2023-09-30T22:11:29.886 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-09-30T22:11:29.887 DEBUG:teuthology.run:Teuthology command: teuthology --name dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi --description fs:upgrade/upgraded_client/{bluestore-bitmap branch/pacific centos_8 clusters/{1-mds-1-client-micro} conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-install 1-mount/mount/fuse 2-upgrade 3-workload/stress_tests/{0-client-upgrade 1-tests/kernel_untar_build}}} --owner scheduled_dparmar@teuthology --archive /home/teuthworker/archive/dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7408044 --verbose -- /home/teuthworker/archive/dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7408044/orig.config.yaml 2023-09-30T22:11:29.940 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-30T22:11:30.036 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7408044 branch: fix-nfs-apply-err-reporting description: fs:upgrade/upgraded_client/{bluestore-bitmap branch/pacific centos_8 clusters/{1-mds-1-client-micro} conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-install 1-mount/mount/fuse 2-upgrade 3-workload/stress_tests/{0-client-upgrade 1-tests/kernel_untar_build}}} email: null first_in_suite: false job_id: '7408044' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 8.stream overrides: admin_socket: branch: fix-nfs-apply-err-reporting ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: bluestore warn on legacy statfs: false bluestore warn on no per pool omap: false mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 mon warn on osd down out interval zero: false osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - scrub mismatch - ScrubResult - wrongly marked - \(POOL_APP_NOT_ENABLED\) - \(SLOW_OPS\) - overall HEALTH_ - \(MON_MSGR2_NOT_ENABLED\) - slow request sha1: 3d6eb91756d99e822eeed7c6b783104a6ad86054 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs check-counter: counters: mds: - mds.dir_split install: ceph: flavor: default sha1: 3d6eb91756d99e822eeed7c6b783104a6ad86054 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-62953 sha1: 6973b24e3d99fd494b753449d2ea12077599cbd6 owner: scheduled_dparmar@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.b - mon.c - mgr.x - mds.a - osd.0 - osd.1 - osd.2 - osd.3 - - client.0 seed: 2683 sha1: 3d6eb91756d99e822eeed7c6b783104a6ad86054 sleep_before_teardown: 0 suite: fs:upgrade suite_branch: wip-62953 suite_path: /home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa suite_relpath: qa suite_repo: https://github.com/dparmar18/ceph suite_sha1: 6973b24e3d99fd494b753449d2ea12077599cbd6 targets: smithi016.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKk3kgw+DuBL1HteXloOxaFL1W90U3HWeJLYcbwgixcY6Xo/kjnjoqxZKByRRTDia3763Cwx7yT37uv+Zj3xRvw= smithi027.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOLuHXzaNM+5O4CIcXkqdy7FJ6Fd0pbrWeXk9n3OuMUKk4LGzMdfrWpI9MHt6/S9S8/UPzCaNJKNF5P3SbAS+Sg= tasks: - install: branch: pacific exclude_packages: - cephadm - ceph-mgr-cephadm - ceph-immutable-object-cache - python3-rados - python3-rgw - python3-rbd - python3-cephfs - ceph-volume extra_packages: - python-rados - python-rgw - python-rbd - python-cephfs extra_system_packages: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - print: '**** done installing pacific' - ceph: conf: global: mon warn on pool no app: false ms bind msgr2: false log-ignorelist: - overall HEALTH_ - \(FS_ - \(MDS_ - \(OSD_ - \(MON_DOWN\) - \(CACHE_POOL_ - \(POOL_ - \(MGR_DOWN\) - \(PG_ - \(SMALLER_PGP_NUM\) - Monitor daemon marked osd - Behind on trimming - Manager daemon - exec: osd.0: - ceph osd set-require-min-compat-client pacific - print: '**** done ceph' - ceph-fuse: null - mds_pre_upgrade: null - print: '**** done mds pre-upgrade sequence' - install.upgrade: mon.a: branch: reef - print: '**** done install.upgrade the host' - ceph.restart: daemons: - mon.* - mgr.* mon-health-to-clog: false wait-for-healthy: false - ceph.healthy: null - ceph.restart: daemons: - osd.* wait-for-healthy: false wait-for-osds-up: true - ceph.stop: - mds.* - ceph.restart: daemons: - mds.* wait-for-healthy: false wait-for-osds-up: true - exec: mon.a: - ceph osd dump -f json-pretty - ceph versions - for f in `ceph osd pool ls` ; do ceph osd pool set $f pg_autoscale_mode off ; done - ceph osd require-osd-release reef - ceph.healthy: null - print: '**** done ceph.restart' - ceph-fuse: client.0: mounted: false - print: '**** done unmount client.0' - install.upgrade: client.0: null - print: '**** done install.upgrade on client.0' - ceph-fuse: client.0: null - print: '**** done remount client.0' - workunit: clients: all: - kernel_untar_build.sh teuthology: branch: pacific fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end - if not is_fuse() then reject() end teuthology_branch: main teuthology_sha1: 54e62bcbac4e53d9685e08328b790d3b20d71cae timestamp: 2023-09-30_21:53:45 tube: smithi user: dparmar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-09-30T22:11:30.036 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa; will attempt to use it 2023-09-30T22:11:30.037 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks 2023-09-30T22:11:30.037 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-09-30T22:11:30.039 INFO:teuthology.task.internal:Checking packages... 2023-09-30T22:11:30.061 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '3d6eb91756d99e822eeed7c6b783104a6ad86054' 2023-09-30T22:11:30.061 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-09-30T22:11:30.061 INFO:teuthology.packaging:ref: None 2023-09-30T22:11:30.061 INFO:teuthology.packaging:tag: None 2023-09-30T22:11:30.062 INFO:teuthology.packaging:branch: fix-nfs-apply-err-reporting 2023-09-30T22:11:30.062 INFO:teuthology.packaging:sha1: 3d6eb91756d99e822eeed7c6b783104a6ad86054 2023-09-30T22:11:30.062 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=fix-nfs-apply-err-reporting 2023-09-30T22:11:30.280 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-5795.g3d6eb917 2023-09-30T22:11:30.500 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-09-30T22:11:30.510 INFO:teuthology.task.internal:no buildpackages task found 2023-09-30T22:11:30.513 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-09-30T22:11:30.525 INFO:teuthology.task.internal:Saving configuration 2023-09-30T22:11:30.568 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-09-30T22:11:30.579 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-30T22:11:30.605 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi016.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7408044', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-30 22:04:55.040715', 'locked_by': 'scheduled_dparmar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKk3kgw+DuBL1HteXloOxaFL1W90U3HWeJLYcbwgixcY6Xo/kjnjoqxZKByRRTDia3763Cwx7yT37uv+Zj3xRvw='} 2023-09-30T22:11:30.676 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi027.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7408044', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-30 22:04:55.038787', 'locked_by': 'scheduled_dparmar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOLuHXzaNM+5O4CIcXkqdy7FJ6Fd0pbrWeXk9n3OuMUKk4LGzMdfrWpI9MHt6/S9S8/UPzCaNJKNF5P3SbAS+Sg='} 2023-09-30T22:11:30.676 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-09-30T22:11:30.685 INFO:teuthology.task.internal:roles: ubuntu@smithi016.front.sepia.ceph.com - ['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3'] 2023-09-30T22:11:30.685 INFO:teuthology.task.internal:roles: ubuntu@smithi027.front.sepia.ceph.com - ['client.0'] 2023-09-30T22:11:30.686 INFO:teuthology.run_tasks:Running task console_log... 2023-09-30T22:11:30.768 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f1b46deef70>, signals=[15]) 2023-09-30T22:11:30.990 INFO:teuthology.run_tasks:Running task internal.connect... 2023-09-30T22:11:30.999 INFO:teuthology.task.internal:Opening connections... 2023-09-30T22:11:30.999 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi016.front.sepia.ceph.com 2023-09-30T22:11:31.003 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-30T22:11:31.072 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi027.front.sepia.ceph.com 2023-09-30T22:11:31.103 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-30T22:11:31.170 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-09-30T22:11:31.181 DEBUG:teuthology.orchestra.run.smithi016:> uname -m 2023-09-30T22:11:31.206 INFO:teuthology.orchestra.run.smithi016.stdout:x86_64 2023-09-30T22:11:31.207 DEBUG:teuthology.orchestra.run.smithi016:> cat /etc/os-release 2023-09-30T22:11:31.269 INFO:teuthology.orchestra.run.smithi016.stdout:NAME="CentOS Stream" 2023-09-30T22:11:31.269 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION="8" 2023-09-30T22:11:31.269 INFO:teuthology.orchestra.run.smithi016.stdout:ID="centos" 2023-09-30T22:11:31.270 INFO:teuthology.orchestra.run.smithi016.stdout:ID_LIKE="rhel fedora" 2023-09-30T22:11:31.270 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION_ID="8" 2023-09-30T22:11:31.270 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM_ID="platform:el8" 2023-09-30T22:11:31.270 INFO:teuthology.orchestra.run.smithi016.stdout:PRETTY_NAME="CentOS Stream 8" 2023-09-30T22:11:31.270 INFO:teuthology.orchestra.run.smithi016.stdout:ANSI_COLOR="0;31" 2023-09-30T22:11:31.270 INFO:teuthology.orchestra.run.smithi016.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-09-30T22:11:31.270 INFO:teuthology.orchestra.run.smithi016.stdout:HOME_URL="https://centos.org/" 2023-09-30T22:11:31.270 INFO:teuthology.orchestra.run.smithi016.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-30T22:11:31.270 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-30T22:11:31.270 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-09-30T22:11:31.271 INFO:teuthology.lock.ops:Updating smithi016.front.sepia.ceph.com on lock server 2023-09-30T22:11:31.296 DEBUG:teuthology.orchestra.run.smithi027:> uname -m 2023-09-30T22:11:31.319 INFO:teuthology.orchestra.run.smithi027.stdout:x86_64 2023-09-30T22:11:31.319 DEBUG:teuthology.orchestra.run.smithi027:> cat /etc/os-release 2023-09-30T22:11:31.382 INFO:teuthology.orchestra.run.smithi027.stdout:NAME="CentOS Stream" 2023-09-30T22:11:31.402 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION="8" 2023-09-30T22:11:31.402 INFO:teuthology.orchestra.run.smithi027.stdout:ID="centos" 2023-09-30T22:11:31.402 INFO:teuthology.orchestra.run.smithi027.stdout:ID_LIKE="rhel fedora" 2023-09-30T22:11:31.402 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION_ID="8" 2023-09-30T22:11:31.402 INFO:teuthology.orchestra.run.smithi027.stdout:PLATFORM_ID="platform:el8" 2023-09-30T22:11:31.402 INFO:teuthology.orchestra.run.smithi027.stdout:PRETTY_NAME="CentOS Stream 8" 2023-09-30T22:11:31.402 INFO:teuthology.orchestra.run.smithi027.stdout:ANSI_COLOR="0;31" 2023-09-30T22:11:31.402 INFO:teuthology.orchestra.run.smithi027.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-09-30T22:11:31.403 INFO:teuthology.orchestra.run.smithi027.stdout:HOME_URL="https://centos.org/" 2023-09-30T22:11:31.403 INFO:teuthology.orchestra.run.smithi027.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-09-30T22:11:31.403 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-09-30T22:11:31.403 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-09-30T22:11:31.404 INFO:teuthology.lock.ops:Updating smithi027.front.sepia.ceph.com on lock server 2023-09-30T22:11:31.436 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-09-30T22:11:31.448 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-09-30T22:11:31.511 INFO:teuthology.task.internal:Checking for old test directory... 2023-09-30T22:11:31.512 DEBUG:teuthology.orchestra.run.smithi016:> test '!' -e /home/ubuntu/cephtest 2023-09-30T22:11:31.514 DEBUG:teuthology.orchestra.run.smithi027:> test '!' -e /home/ubuntu/cephtest 2023-09-30T22:11:31.535 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-09-30T22:11:31.547 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-09-30T22:11:31.548 DEBUG:teuthology.orchestra.run.smithi016:> test -z $(ls -A /var/lib/ceph) 2023-09-30T22:11:31.576 DEBUG:teuthology.orchestra.run.smithi027:> test -z $(ls -A /var/lib/ceph) 2023-09-30T22:11:31.602 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-09-30T22:11:31.673 INFO:teuthology.run_tasks:Running task kernel... 2023-09-30T22:11:31.692 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-09-30T22:11:31.692 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'client.0': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-09-30T22:11:31.692 DEBUG:teuthology.orchestra.run.smithi016:> test -f /run/.containerenv -o -f /.dockerenv 2023-09-30T22:11:31.693 DEBUG:teuthology.orchestra.run.smithi027:> test -f /run/.containerenv -o -f /.dockerenv 2023-09-30T22:11:31.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T22:11:31.715 DEBUG:teuthology.orchestra.run.smithi016:> uname -r 2023-09-30T22:11:31.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T22:11:31.716 DEBUG:teuthology.orchestra.run.smithi027:> uname -r 2023-09-30T22:11:31.777 INFO:teuthology.orchestra.run.smithi016.stdout:4.18.0-489.el8.x86_64 2023-09-30T22:11:31.777 INFO:teuthology.task.kernel:Running kernel on smithi016: 4.18.0-489.el8.x86_64 2023-09-30T22:11:31.777 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum install -y kernel 2023-09-30T22:11:31.778 INFO:teuthology.orchestra.run.smithi027.stdout:4.18.0-489.el8.x86_64 2023-09-30T22:11:31.779 INFO:teuthology.task.kernel:Running kernel on smithi027: 4.18.0-489.el8.x86_64 2023-09-30T22:11:31.779 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum install -y kernel 2023-09-30T22:11:36.755 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 8 - AppStream 13 MB/s | 33 MB 00:02 2023-09-30T22:11:36.862 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 8 - AppStream 13 MB/s | 33 MB 00:02 2023-09-30T22:11:47.538 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 8 - BaseOS 11 MB/s | 48 MB 00:04 2023-09-30T22:11:47.562 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 8 - BaseOS 11 MB/s | 48 MB 00:04 2023-09-30T22:11:55.087 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 8 - Extras 94 kB/s | 18 kB 00:00 2023-09-30T22:11:55.204 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 8 - Extras 51 kB/s | 18 kB 00:00 2023-09-30T22:11:55.929 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 8 - Extras common packages 11 kB/s | 6.9 kB 00:00 2023-09-30T22:11:56.047 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 8 - Extras common packages 11 kB/s | 6.9 kB 00:00 2023-09-30T22:11:56.239 INFO:teuthology.orchestra.run.smithi016.stdout:Copr repo for python3-asyncssh owned by ceph 25 kB/s | 3.5 kB 00:00 2023-09-30T22:11:56.348 INFO:teuthology.orchestra.run.smithi027.stdout:Copr repo for python3-asyncssh owned by ceph 25 kB/s | 3.5 kB 00:00 2023-09-30T22:11:57.948 INFO:teuthology.orchestra.run.smithi016.stdout:Extra Packages for Enterprise Linux 10 MB/s | 16 MB 00:01 2023-09-30T22:11:58.431 INFO:teuthology.orchestra.run.smithi027.stdout:Extra Packages for Enterprise Linux 8.3 MB/s | 16 MB 00:01 2023-09-30T22:12:02.040 INFO:teuthology.orchestra.run.smithi016.stdout:lab-extras 237 kB/s | 24 kB 00:00 2023-09-30T22:12:02.474 INFO:teuthology.orchestra.run.smithi027.stdout:lab-extras 491 kB/s | 24 kB 00:00 2023-09-30T22:12:05.797 INFO:teuthology.orchestra.run.smithi016.stdout:Package kernel-4.18.0-348.el8.x86_64 is already installed. 2023-09-30T22:12:05.798 INFO:teuthology.orchestra.run.smithi016.stdout:Package kernel-4.18.0-489.el8.x86_64 is already installed. 2023-09-30T22:12:06.074 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-09-30T22:12:06.074 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-09-30T22:12:06.075 INFO:teuthology.orchestra.run.smithi016.stdout: Package Architecture Version Repository Size 2023-09-30T22:12:06.075 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-09-30T22:12:06.075 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2023-09-30T22:12:06.075 INFO:teuthology.orchestra.run.smithi016.stdout: kernel x86_64 4.18.0-514.el8 baseos 10 M 2023-09-30T22:12:06.075 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2023-09-30T22:12:06.075 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-core x86_64 4.18.0-514.el8 baseos 43 M 2023-09-30T22:12:06.075 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-modules x86_64 4.18.0-514.el8 baseos 35 M 2023-09-30T22:12:06.075 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:12:06.075 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2023-09-30T22:12:06.075 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-09-30T22:12:06.076 INFO:teuthology.orchestra.run.smithi016.stdout:Install 3 Packages 2023-09-30T22:12:06.076 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:12:06.076 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 88 M 2023-09-30T22:12:06.076 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 96 M 2023-09-30T22:12:06.076 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2023-09-30T22:12:06.198 INFO:teuthology.orchestra.run.smithi027.stdout:Package kernel-4.18.0-348.el8.x86_64 is already installed. 2023-09-30T22:12:06.198 INFO:teuthology.orchestra.run.smithi027.stdout:Package kernel-4.18.0-489.el8.x86_64 is already installed. 2023-09-30T22:12:06.464 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2023-09-30T22:12:06.464 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2023-09-30T22:12:06.464 INFO:teuthology.orchestra.run.smithi027.stdout: Package Architecture Version Repository Size 2023-09-30T22:12:06.465 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2023-09-30T22:12:06.465 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2023-09-30T22:12:06.465 INFO:teuthology.orchestra.run.smithi027.stdout: kernel x86_64 4.18.0-514.el8 baseos 10 M 2023-09-30T22:12:06.465 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2023-09-30T22:12:06.465 INFO:teuthology.orchestra.run.smithi027.stdout: kernel-core x86_64 4.18.0-514.el8 baseos 43 M 2023-09-30T22:12:06.465 INFO:teuthology.orchestra.run.smithi027.stdout: kernel-modules x86_64 4.18.0-514.el8 baseos 35 M 2023-09-30T22:12:06.465 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-09-30T22:12:06.465 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2023-09-30T22:12:06.465 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2023-09-30T22:12:06.465 INFO:teuthology.orchestra.run.smithi027.stdout:Install 3 Packages 2023-09-30T22:12:06.465 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-09-30T22:12:06.466 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 88 M 2023-09-30T22:12:06.466 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 96 M 2023-09-30T22:12:06.466 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2023-09-30T22:12:09.266 INFO:teuthology.orchestra.run.smithi016.stdout:(1/3): kernel-4.18.0-514.el8.x86_64.rpm 3.2 MB/s | 10 MB 00:03 2023-09-30T22:12:10.417 INFO:teuthology.orchestra.run.smithi027.stdout:(1/3): kernel-4.18.0-514.el8.x86_64.rpm 2.7 MB/s | 10 MB 00:03 2023-09-30T22:12:14.383 INFO:teuthology.orchestra.run.smithi016.stdout:(2/3): kernel-modules-4.18.0-514.el8.x86_64.rpm 4.3 MB/s | 35 MB 00:08 2023-09-30T22:12:15.775 INFO:teuthology.orchestra.run.smithi027.stdout:(2/3): kernel-modules-4.18.0-514.el8.x86_64.rpm 3.9 MB/s | 35 MB 00:09 2023-09-30T22:12:15.791 INFO:teuthology.orchestra.run.smithi016.stdout:(3/3): kernel-core-4.18.0-514.el8.x86_64.rpm 4.4 MB/s | 43 MB 00:09 2023-09-30T22:12:15.791 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2023-09-30T22:12:15.791 INFO:teuthology.orchestra.run.smithi016.stdout:Total 9.1 MB/s | 88 MB 00:09 2023-09-30T22:12:16.495 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2023-09-30T22:12:16.936 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2023-09-30T22:12:16.936 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2023-09-30T22:12:18.017 INFO:teuthology.orchestra.run.smithi027.stdout:(3/3): kernel-core-4.18.0-514.el8.x86_64.rpm 3.8 MB/s | 43 MB 00:11 2023-09-30T22:12:18.017 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2023-09-30T22:12:18.017 INFO:teuthology.orchestra.run.smithi027.stdout:Total 7.6 MB/s | 88 MB 00:11 2023-09-30T22:12:18.218 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2023-09-30T22:12:18.254 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2023-09-30T22:12:18.717 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2023-09-30T22:12:19.143 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2023-09-30T22:12:19.144 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2023-09-30T22:12:20.425 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2023-09-30T22:12:20.445 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2023-09-30T22:12:21.187 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2023-09-30T22:12:21.715 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : kernel-core-4.18.0-514.el8.x86_64 1/3 2023-09-30T22:12:23.327 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2023-09-30T22:12:23.933 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : kernel-core-4.18.0-514.el8.x86_64 1/3 2023-09-30T22:12:25.451 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-core-4.18.0-514.el8.x86_64 1/3 2023-09-30T22:12:25.823 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : kernel-modules-4.18.0-514.el8.x86_64 2/3 2023-09-30T22:12:27.706 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: kernel-core-4.18.0-514.el8.x86_64 1/3 2023-09-30T22:12:28.099 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : kernel-modules-4.18.0-514.el8.x86_64 2/3 2023-09-30T22:12:29.442 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-modules-4.18.0-514.el8.x86_64 2/3 2023-09-30T22:12:29.765 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : kernel-4.18.0-514.el8.x86_64 3/3 2023-09-30T22:12:31.718 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: kernel-modules-4.18.0-514.el8.x86_64 2/3 2023-09-30T22:12:32.060 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : kernel-4.18.0-514.el8.x86_64 3/3 2023-09-30T22:12:53.033 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-core-4.18.0-514.el8.x86_64 3/3 2023-09-30T22:12:53.057 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-modules-4.18.0-514.el8.x86_64 3/3 2023-09-30T22:12:53.424 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-4.18.0-514.el8.x86_64 3/3 2023-09-30T22:12:53.424 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : kernel-4.18.0-514.el8.x86_64 1/3 2023-09-30T22:12:53.424 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : kernel-core-4.18.0-514.el8.x86_64 2/3 2023-09-30T22:12:54.225 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : kernel-modules-4.18.0-514.el8.x86_64 3/3 2023-09-30T22:12:54.225 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:12:54.225 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2023-09-30T22:12:54.226 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-4.18.0-514.el8.x86_64 kernel-core-4.18.0-514.el8.x86_64 2023-09-30T22:12:54.226 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-modules-4.18.0-514.el8.x86_64 2023-09-30T22:12:54.226 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:12:54.226 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-09-30T22:12:54.968 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: kernel-core-4.18.0-514.el8.x86_64 3/3 2023-09-30T22:12:54.991 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: kernel-modules-4.18.0-514.el8.x86_64 3/3 2023-09-30T22:12:55.317 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q kernel | sort -rV | head -n 1 2023-09-30T22:12:55.354 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: kernel-4.18.0-514.el8.x86_64 3/3 2023-09-30T22:12:55.354 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : kernel-4.18.0-514.el8.x86_64 1/3 2023-09-30T22:12:55.354 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : kernel-core-4.18.0-514.el8.x86_64 2/3 2023-09-30T22:12:55.545 INFO:teuthology.orchestra.run.smithi016.stdout:kernel-4.18.0-514.el8.x86_64 2023-09-30T22:12:55.545 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-514.el8.x86_64 2023-09-30T22:12:55.545 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-489.el8.x86_64 Expected: 4.18.0-514.el8.x86_64 2023-09-30T22:12:55.545 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-09-30T22:12:55.545 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-09-30T22:12:55.545 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-09-30T22:12:55.545 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi016.front.sepia.ceph.com, path=None, version=distro) 2023-09-30T22:12:55.546 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q kernel | sort -rV | head -n 1 2023-09-30T22:12:55.763 INFO:teuthology.orchestra.run.smithi016.stdout:kernel-4.18.0-514.el8.x86_64 2023-09-30T22:12:55.764 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-514.el8.x86_64 2023-09-30T22:12:55.764 DEBUG:teuthology.orchestra.run.smithi016:> sudo rpm -qi grub2-tools 2023-09-30T22:12:55.798 INFO:teuthology.orchestra.run.smithi016.stdout:Name : grub2-tools 2023-09-30T22:12:55.799 INFO:teuthology.orchestra.run.smithi016.stdout:Epoch : 1 2023-09-30T22:12:55.799 INFO:teuthology.orchestra.run.smithi016.stdout:Version : 2.02 2023-09-30T22:12:55.799 INFO:teuthology.orchestra.run.smithi016.stdout:Release : 129.el8 2023-09-30T22:12:55.799 INFO:teuthology.orchestra.run.smithi016.stdout:Architecture: x86_64 2023-09-30T22:12:55.799 INFO:teuthology.orchestra.run.smithi016.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2023-09-30T22:12:55.799 INFO:teuthology.orchestra.run.smithi016.stdout:Group : System Environment/Base 2023-09-30T22:12:55.799 INFO:teuthology.orchestra.run.smithi016.stdout:Size : 9265586 2023-09-30T22:12:55.799 INFO:teuthology.orchestra.run.smithi016.stdout:License : GPLv3+ 2023-09-30T22:12:55.799 INFO:teuthology.orchestra.run.smithi016.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-09-30T22:12:55.799 INFO:teuthology.orchestra.run.smithi016.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-09-30T22:12:55.799 INFO:teuthology.orchestra.run.smithi016.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-09-30T22:12:55.800 INFO:teuthology.orchestra.run.smithi016.stdout:Build Host : kbuilder.bsys.centos.org 2023-09-30T22:12:55.800 INFO:teuthology.orchestra.run.smithi016.stdout:Relocations : (not relocatable) 2023-09-30T22:12:55.800 INFO:teuthology.orchestra.run.smithi016.stdout:Packager : CentOS BuildSystem 2023-09-30T22:12:55.800 INFO:teuthology.orchestra.run.smithi016.stdout:Vendor : CentOS 2023-09-30T22:12:55.800 INFO:teuthology.orchestra.run.smithi016.stdout:URL : http://www.gnu.org/software/grub/ 2023-09-30T22:12:55.800 INFO:teuthology.orchestra.run.smithi016.stdout:Summary : Support tools for GRUB. 2023-09-30T22:12:55.800 INFO:teuthology.orchestra.run.smithi016.stdout:Description : 2023-09-30T22:12:55.800 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:12:55.800 INFO:teuthology.orchestra.run.smithi016.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-09-30T22:12:55.800 INFO:teuthology.orchestra.run.smithi016.stdout:customizable bootloader with modular architecture. It supports a rich 2023-09-30T22:12:55.800 INFO:teuthology.orchestra.run.smithi016.stdout:variety of kernel formats, file systems, computer architectures and 2023-09-30T22:12:55.800 INFO:teuthology.orchestra.run.smithi016.stdout:hardware devices. 2023-09-30T22:12:55.801 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:12:55.801 INFO:teuthology.orchestra.run.smithi016.stdout:This subpackage provides tools for support of all platforms. 2023-09-30T22:12:55.801 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-09-30T22:12:55.802 INFO:teuthology.task.kernel:Updating grub on smithi016 to boot 4.18.0-514.el8.x86_64 2023-09-30T22:12:55.802 DEBUG:teuthology.orchestra.run.smithi016:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-09-30T22:12:56.325 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : kernel-modules-4.18.0-514.el8.x86_64 3/3 2023-09-30T22:12:56.325 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-09-30T22:12:56.325 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2023-09-30T22:12:56.325 INFO:teuthology.orchestra.run.smithi027.stdout: kernel-4.18.0-514.el8.x86_64 kernel-core-4.18.0-514.el8.x86_64 2023-09-30T22:12:56.325 INFO:teuthology.orchestra.run.smithi027.stdout: kernel-modules-4.18.0-514.el8.x86_64 2023-09-30T22:12:56.325 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-09-30T22:12:56.325 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2023-09-30T22:12:56.464 DEBUG:teuthology.orchestra.run.smithi027:> rpm -q kernel | sort -rV | head -n 1 2023-09-30T22:12:56.689 INFO:teuthology.orchestra.run.smithi027.stdout:kernel-4.18.0-514.el8.x86_64 2023-09-30T22:12:56.689 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-514.el8.x86_64 2023-09-30T22:12:56.689 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-489.el8.x86_64 Expected: 4.18.0-514.el8.x86_64 2023-09-30T22:12:56.689 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-09-30T22:12:56.689 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-09-30T22:12:56.689 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2023-09-30T22:12:56.689 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi027.front.sepia.ceph.com, path=None, version=distro) 2023-09-30T22:12:56.690 DEBUG:teuthology.orchestra.run.smithi027:> rpm -q kernel | sort -rV | head -n 1 2023-09-30T22:12:56.907 INFO:teuthology.orchestra.run.smithi027.stdout:kernel-4.18.0-514.el8.x86_64 2023-09-30T22:12:56.907 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-514.el8.x86_64 2023-09-30T22:12:56.907 DEBUG:teuthology.orchestra.run.smithi027:> sudo rpm -qi grub2-tools 2023-09-30T22:12:56.933 INFO:teuthology.orchestra.run.smithi016.stderr:Generating grub configuration file ... 2023-09-30T22:12:56.941 INFO:teuthology.orchestra.run.smithi027.stdout:Name : grub2-tools 2023-09-30T22:12:56.941 INFO:teuthology.orchestra.run.smithi027.stdout:Epoch : 1 2023-09-30T22:12:56.942 INFO:teuthology.orchestra.run.smithi027.stdout:Version : 2.02 2023-09-30T22:12:56.942 INFO:teuthology.orchestra.run.smithi027.stdout:Release : 129.el8 2023-09-30T22:12:56.942 INFO:teuthology.orchestra.run.smithi027.stdout:Architecture: x86_64 2023-09-30T22:12:56.942 INFO:teuthology.orchestra.run.smithi027.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2023-09-30T22:12:56.942 INFO:teuthology.orchestra.run.smithi027.stdout:Group : System Environment/Base 2023-09-30T22:12:56.942 INFO:teuthology.orchestra.run.smithi027.stdout:Size : 9265586 2023-09-30T22:12:56.942 INFO:teuthology.orchestra.run.smithi027.stdout:License : GPLv3+ 2023-09-30T22:12:56.942 INFO:teuthology.orchestra.run.smithi027.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-09-30T22:12:56.942 INFO:teuthology.orchestra.run.smithi027.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-09-30T22:12:56.942 INFO:teuthology.orchestra.run.smithi027.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-09-30T22:12:56.943 INFO:teuthology.orchestra.run.smithi027.stdout:Build Host : kbuilder.bsys.centos.org 2023-09-30T22:12:56.943 INFO:teuthology.orchestra.run.smithi027.stdout:Relocations : (not relocatable) 2023-09-30T22:12:56.943 INFO:teuthology.orchestra.run.smithi027.stdout:Packager : CentOS BuildSystem 2023-09-30T22:12:56.943 INFO:teuthology.orchestra.run.smithi027.stdout:Vendor : CentOS 2023-09-30T22:12:56.943 INFO:teuthology.orchestra.run.smithi027.stdout:URL : http://www.gnu.org/software/grub/ 2023-09-30T22:12:56.943 INFO:teuthology.orchestra.run.smithi027.stdout:Summary : Support tools for GRUB. 2023-09-30T22:12:56.943 INFO:teuthology.orchestra.run.smithi027.stdout:Description : 2023-09-30T22:12:56.943 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-09-30T22:12:56.943 INFO:teuthology.orchestra.run.smithi027.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-09-30T22:12:56.943 INFO:teuthology.orchestra.run.smithi027.stdout:customizable bootloader with modular architecture. It supports a rich 2023-09-30T22:12:56.943 INFO:teuthology.orchestra.run.smithi027.stdout:variety of kernel formats, file systems, computer architectures and 2023-09-30T22:12:56.944 INFO:teuthology.orchestra.run.smithi027.stdout:hardware devices. 2023-09-30T22:12:56.944 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-09-30T22:12:56.944 INFO:teuthology.orchestra.run.smithi027.stdout:This subpackage provides tools for support of all platforms. 2023-09-30T22:12:56.944 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-09-30T22:12:56.945 INFO:teuthology.task.kernel:Updating grub on smithi027 to boot 4.18.0-514.el8.x86_64 2023-09-30T22:12:56.945 DEBUG:teuthology.orchestra.run.smithi027:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-09-30T22:12:58.211 INFO:teuthology.orchestra.run.smithi027.stderr:Generating grub configuration file ... 2023-09-30T22:12:59.076 INFO:teuthology.orchestra.run.smithi016.stderr:done 2023-09-30T22:12:59.078 DEBUG:teuthology.orchestra.run.smithi016:> mktemp 2023-09-30T22:12:59.102 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/tmp.EJpOLlavfJ 2023-09-30T22:12:59.102 DEBUG:teuthology.orchestra.run.smithi016:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.EJpOLlavfJ 2023-09-30T22:12:59.174 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0666 /tmp/tmp.EJpOLlavfJ 2023-09-30T22:12:59.317 DEBUG:teuthology.orchestra.remote:smithi016:/tmp/tmp.EJpOLlavfJ is 6KB 2023-09-30T22:12:59.365 DEBUG:teuthology.orchestra.run.smithi016:> rm -fr /tmp/tmp.EJpOLlavfJ 2023-09-30T22:12:59.384 DEBUG:teuthology.orchestra.run.smithi016:> sudo /bin/ls /boot/loader/entries || true 2023-09-30T22:12:59.458 INFO:teuthology.orchestra.run.smithi016.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2023-09-30T22:12:59.459 INFO:teuthology.orchestra.run.smithi016.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2023-09-30T22:12:59.459 INFO:teuthology.orchestra.run.smithi016.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2023-09-30T22:12:59.459 INFO:teuthology.orchestra.run.smithi016.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-514.el8.x86_64.conf 2023-09-30T22:12:59.460 DEBUG:teuthology.orchestra.run.smithi016:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-4.18.0-514.el8.x86_64 2023-09-30T22:12:59.603 DEBUG:teuthology.orchestra.run.smithi016:> sudo shutdown -r now 2023-09-30T22:13:00.455 INFO:teuthology.orchestra.run.smithi027.stderr:done 2023-09-30T22:13:00.457 DEBUG:teuthology.orchestra.run.smithi027:> mktemp 2023-09-30T22:13:00.478 INFO:teuthology.orchestra.run.smithi027.stdout:/tmp/tmp.KVaPyEqLe1 2023-09-30T22:13:00.479 DEBUG:teuthology.orchestra.run.smithi027:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.KVaPyEqLe1 2023-09-30T22:13:00.552 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 0666 /tmp/tmp.KVaPyEqLe1 2023-09-30T22:13:00.693 DEBUG:teuthology.orchestra.remote:smithi027:/tmp/tmp.KVaPyEqLe1 is 6KB 2023-09-30T22:13:00.740 DEBUG:teuthology.orchestra.run.smithi027:> rm -fr /tmp/tmp.KVaPyEqLe1 2023-09-30T22:13:00.758 DEBUG:teuthology.orchestra.run.smithi027:> sudo /bin/ls /boot/loader/entries || true 2023-09-30T22:13:00.834 INFO:teuthology.orchestra.run.smithi027.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2023-09-30T22:13:00.835 INFO:teuthology.orchestra.run.smithi027.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2023-09-30T22:13:00.835 INFO:teuthology.orchestra.run.smithi027.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2023-09-30T22:13:00.835 INFO:teuthology.orchestra.run.smithi027.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-514.el8.x86_64.conf 2023-09-30T22:13:00.836 DEBUG:teuthology.orchestra.run.smithi027:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-4.18.0-514.el8.x86_64 2023-09-30T22:13:00.997 DEBUG:teuthology.orchestra.run.smithi027:> sudo shutdown -r now 2023-09-30T22:13:29.634 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-09-30T22:13:29.635 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2023-09-30T22:13:29.635 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-30T22:13:31.001 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-09-30T22:13:31.002 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2023-09-30T22:13:31.002 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-30T22:14:05.691 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2023-09-30T22:14:14.700 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2023-09-30T22:14:14.700 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-30T22:14:17.754 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2023-09-30T22:14:29.646 DEBUG:teuthology.orchestra.remote:timed out 2023-09-30T22:14:29.755 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2023-09-30T22:14:29.755 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-30T22:14:32.826 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2023-09-30T22:14:38.650 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2023-09-30T22:14:38.651 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-30T22:14:38.967 DEBUG:teuthology.orchestra.run.smithi016:> true 2023-09-30T22:14:39.256 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi016.front.sepia.ceph.com' 2023-09-30T22:14:39.257 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "4.18.0-514.el8.x86_64"... 2023-09-30T22:14:39.257 DEBUG:teuthology.orchestra.run.smithi016:> uname -r 2023-09-30T22:14:39.277 INFO:teuthology.orchestra.run.smithi016.stdout:4.18.0-514.el8.x86_64 2023-09-30T22:14:39.277 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-514.el8.x86_64 vs 4.18.0-514.el8.x86_64 2023-09-30T22:14:39.278 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-09-30T22:14:39.278 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-09-30T22:14:40.279 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-09-30T22:14:40.279 DEBUG:teuthology.orchestra.run.smithi016:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-09-30T22:14:40.383 INFO:teuthology.orchestra.run.smithi016.stdout:ttyS1 2023-09-30T22:14:40.396 DEBUG:teuthology.parallel:result is None 2023-09-30T22:14:47.827 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2023-09-30T22:14:47.828 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-30T22:15:06.362 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2023-09-30T22:15:24.364 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2023-09-30T22:15:24.365 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-30T22:15:24.926 DEBUG:teuthology.orchestra.run.smithi027:> true 2023-09-30T22:15:25.391 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi027.front.sepia.ceph.com' 2023-09-30T22:15:25.392 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "4.18.0-514.el8.x86_64"... 2023-09-30T22:15:25.392 DEBUG:teuthology.orchestra.run.smithi027:> uname -r 2023-09-30T22:15:25.449 INFO:teuthology.orchestra.run.smithi027.stdout:4.18.0-514.el8.x86_64 2023-09-30T22:15:25.449 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-514.el8.x86_64 vs 4.18.0-514.el8.x86_64 2023-09-30T22:15:25.449 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-09-30T22:15:25.450 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-09-30T22:15:26.450 INFO:teuthology.task.kernel:Enabling kdb on client.0... 2023-09-30T22:15:26.451 DEBUG:teuthology.orchestra.run.smithi027:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-09-30T22:15:26.534 INFO:teuthology.orchestra.run.smithi027.stdout:ttyS1 2023-09-30T22:15:26.550 DEBUG:teuthology.parallel:result is None 2023-09-30T22:15:26.550 INFO:teuthology.run_tasks:Running task internal.base... 2023-09-30T22:15:26.557 INFO:teuthology.task.internal:Creating test directory... 2023-09-30T22:15:26.557 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-09-30T22:15:26.560 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-09-30T22:15:26.583 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-09-30T22:15:26.589 INFO:teuthology.run_tasks:Running task internal.archive... 2023-09-30T22:15:26.599 INFO:teuthology.task.internal:Creating archive directory... 2023-09-30T22:15:26.599 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-09-30T22:15:26.626 DEBUG:teuthology.orchestra.run.smithi027:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-09-30T22:15:26.672 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-09-30T22:15:26.678 INFO:teuthology.task.internal:Enabling coredump saving... 2023-09-30T22:15:26.679 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-09-30T22:15:26.703 DEBUG:teuthology.orchestra.run.smithi027:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-09-30T22:15:26.741 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-30T22:15:26.754 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-30T22:15:26.768 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-30T22:15:26.780 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-09-30T22:15:26.781 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-09-30T22:15:26.788 INFO:teuthology.task.internal:Configuring sudo... 2023-09-30T22:15:26.789 DEBUG:teuthology.orchestra.run.smithi016:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-09-30T22:15:26.799 DEBUG:teuthology.orchestra.run.smithi027:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-09-30T22:15:26.832 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-09-30T22:15:26.851 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-09-30T22:15:26.852 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-09-30T22:15:26.874 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-09-30T22:15:26.894 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-30T22:15:26.987 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-30T22:15:27.043 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-30T22:15:27.135 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-30T22:15:27.184 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-09-30T22:15:27.184 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-09-30T22:15:27.260 DEBUG:teuthology.orchestra.run.smithi027:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-30T22:15:27.308 DEBUG:teuthology.orchestra.run.smithi027:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-09-30T22:15:27.367 DEBUG:teuthology.orchestra.run.smithi027:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-30T22:15:27.456 DEBUG:teuthology.orchestra.run.smithi027:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-09-30T22:15:27.503 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-09-30T22:15:27.503 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-09-30T22:15:27.574 DEBUG:teuthology.orchestra.run.smithi016:> sudo service rsyslog restart 2023-09-30T22:15:27.576 DEBUG:teuthology.orchestra.run.smithi027:> sudo service rsyslog restart 2023-09-30T22:15:27.671 INFO:teuthology.orchestra.run.smithi027.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-30T22:15:27.678 INFO:teuthology.orchestra.run.smithi016.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-30T22:15:28.157 INFO:teuthology.run_tasks:Running task internal.timer... 2023-09-30T22:15:28.166 INFO:teuthology.task.internal:Starting timer... 2023-09-30T22:15:28.166 INFO:teuthology.run_tasks:Running task pcp... 2023-09-30T22:15:28.178 INFO:teuthology.run_tasks:Running task selinux... 2023-09-30T22:15:28.187 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-09-30T22:15:28.187 DEBUG:teuthology.orchestra.run.smithi016:> sudo service auditd rotate 2023-09-30T22:15:28.295 INFO:teuthology.orchestra.run.smithi016.stdout:Rotating logs: 2023-09-30T22:15:28.297 DEBUG:teuthology.orchestra.run.smithi027:> sudo service auditd rotate 2023-09-30T22:15:28.376 INFO:teuthology.orchestra.run.smithi027.stdout:Rotating logs: 2023-09-30T22:15:28.378 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-09-30T22:15:28.379 DEBUG:teuthology.orchestra.run.smithi016:> /usr/sbin/getenforce 2023-09-30T22:15:28.436 INFO:teuthology.orchestra.run.smithi016.stdout:Permissive 2023-09-30T22:15:28.437 DEBUG:teuthology.orchestra.run.smithi027:> /usr/sbin/getenforce 2023-09-30T22:15:28.472 INFO:teuthology.orchestra.run.smithi027.stdout:Permissive 2023-09-30T22:15:28.472 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi016.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi027.front.sepia.ceph.com': 'permissive'} 2023-09-30T22:15:28.472 DEBUG:teuthology.orchestra.run.smithi016:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-30T22:15:28.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T22:15:28.508 DEBUG:teuthology.orchestra.run.smithi027:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-30T22:15:28.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T22:15:28.542 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-09-30T22:15:28.542 DEBUG:teuthology.orchestra.run.smithi016:> sudo /usr/sbin/setenforce permissive 2023-09-30T22:15:28.582 DEBUG:teuthology.orchestra.run.smithi027:> sudo /usr/sbin/setenforce permissive 2023-09-30T22:15:28.611 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-09-30T22:15:28.620 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-09-30T22:15:28.620 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-09-30T22:15:28.636 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-09-30T22:15:28.637 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi016.front.sepia.ceph.com,smithi027.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-09-30T22:23:39.599 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi016.front.sepia.ceph.com'), Remote(name='ubuntu@smithi027.front.sepia.ceph.com')] 2023-09-30T22:23:39.600 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2023-09-30T22:23:39.600 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-30T22:23:39.671 DEBUG:teuthology.orchestra.run.smithi016:> true 2023-09-30T22:23:39.746 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi016.front.sepia.ceph.com' 2023-09-30T22:23:39.746 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2023-09-30T22:23:39.747 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-09-30T22:23:39.815 DEBUG:teuthology.orchestra.run.smithi027:> true 2023-09-30T22:23:39.891 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi027.front.sepia.ceph.com' 2023-09-30T22:23:39.892 INFO:teuthology.run_tasks:Running task clock... 2023-09-30T22:23:39.903 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-09-30T22:23:39.903 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-09-30T22:23:39.904 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-30T22:23:39.906 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-09-30T22:23:39.906 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-30T22:23:39.944 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-09-30T22:23:39.961 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-09-30T22:23:39.966 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-09-30T22:23:39.981 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-09-30T22:23:40.034 INFO:teuthology.orchestra.run.smithi016.stderr:sudo: ntpd: command not found 2023-09-30T22:23:40.048 INFO:teuthology.orchestra.run.smithi016.stdout:506 Cannot talk to daemon 2023-09-30T22:23:40.048 INFO:teuthology.orchestra.run.smithi027.stderr:sudo: ntpd: command not found 2023-09-30T22:23:40.060 INFO:teuthology.orchestra.run.smithi027.stdout:506 Cannot talk to daemon 2023-09-30T22:23:40.065 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-09-30T22:23:40.075 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-09-30T22:23:40.081 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-09-30T22:23:40.089 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-09-30T22:23:40.161 INFO:teuthology.orchestra.run.smithi027.stderr:bash: ntpq: command not found 2023-09-30T22:23:40.163 INFO:teuthology.orchestra.run.smithi016.stderr:bash: ntpq: command not found 2023-09-30T22:23:40.164 INFO:teuthology.orchestra.run.smithi027.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-30T22:23:40.165 INFO:teuthology.orchestra.run.smithi027.stdout:=============================================================================== 2023-09-30T22:23:40.165 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-30T22:23:40.165 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-30T22:23:40.165 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-30T22:23:40.165 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-30T22:23:40.167 INFO:teuthology.orchestra.run.smithi016.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-30T22:23:40.167 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================== 2023-09-30T22:23:40.167 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-30T22:23:40.167 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-30T22:23:40.167 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-30T22:23:40.167 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-09-30T22:23:40.168 INFO:teuthology.run_tasks:Running task install... 2023-09-30T22:23:40.177 DEBUG:teuthology.task.install:project ceph 2023-09-30T22:23:40.177 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '3d6eb91756d99e822eeed7c6b783104a6ad86054'}} 2023-09-30T22:23:40.177 DEBUG:teuthology.task.install:config {'branch': 'pacific', 'exclude_packages': ['cephadm', 'ceph-mgr-cephadm', 'ceph-immutable-object-cache', 'python3-rados', 'python3-rgw', 'python3-rbd', 'python3-cephfs', 'ceph-volume'], 'extra_packages': ['python-rados', 'python-rgw', 'python-rbd', 'python-cephfs'], 'extra_system_packages': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux'], 'flavor': 'default', 'sha1': '3d6eb91756d99e822eeed7c6b783104a6ad86054'} 2023-09-30T22:23:40.177 INFO:teuthology.task.install:Using flavor: default 2023-09-30T22:23:40.183 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-09-30T22:23:40.183 INFO:teuthology.task.install:extra packages: ['python-rados', 'python-rgw', 'python-rbd', 'python-cephfs'] 2023-09-30T22:23:40.184 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'pacific', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['cephadm', 'ceph-mgr-cephadm', 'ceph-immutable-object-cache', 'python3-rados', 'python3-rgw', 'python3-rbd', 'python3-cephfs', 'ceph-volume'], 'extra_packages': ['python-rados', 'python-rgw', 'python-rbd', 'python-cephfs'], 'extra_system_packages': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux'], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d6eb91756d99e822eeed7c6b783104a6ad86054', 'tag': None, 'wait_for_package': False} 2023-09-30T22:23:40.184 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-09-30T22:23:40.184 INFO:teuthology.packaging:ref: None 2023-09-30T22:23:40.184 INFO:teuthology.packaging:tag: None 2023-09-30T22:23:40.184 INFO:teuthology.packaging:branch: pacific 2023-09-30T22:23:40.184 INFO:teuthology.packaging:sha1: 3d6eb91756d99e822eeed7c6b783104a6ad86054 2023-09-30T22:23:40.184 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=pacific 2023-09-30T22:23:40.186 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'pacific', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['cephadm', 'ceph-mgr-cephadm', 'ceph-immutable-object-cache', 'python3-rados', 'python3-rgw', 'python3-rbd', 'python3-cephfs', 'ceph-volume'], 'extra_packages': ['python-rados', 'python-rgw', 'python-rbd', 'python-cephfs'], 'extra_system_packages': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux'], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d6eb91756d99e822eeed7c6b783104a6ad86054', 'tag': None, 'wait_for_package': False} 2023-09-30T22:23:40.186 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-09-30T22:23:40.186 INFO:teuthology.packaging:ref: None 2023-09-30T22:23:40.187 INFO:teuthology.packaging:tag: None 2023-09-30T22:23:40.187 INFO:teuthology.packaging:branch: pacific 2023-09-30T22:23:40.187 INFO:teuthology.packaging:sha1: 3d6eb91756d99e822eeed7c6b783104a6ad86054 2023-09-30T22:23:40.187 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=pacific 2023-09-30T22:23:40.484 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/pacific/eeb13ac974775bd0cce96a2c687e8bcc2faca43d/centos/8/flavors/default/ 2023-09-30T22:23:40.484 INFO:teuthology.task.install.rpm:Package version is 16.2.14-44.geeb13ac9 2023-09-30T22:23:40.505 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/pacific/eeb13ac974775bd0cce96a2c687e8bcc2faca43d/centos/8/flavors/default/ 2023-09-30T22:23:40.506 INFO:teuthology.task.install.rpm:Package version is 16.2.14-44.geeb13ac9 2023-09-30T22:23:40.594 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/pacific/eeb13ac974775bd0cce96a2c687e8bcc2faca43d/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/pacific/eeb13ac974775bd0cce96a2c687e8bcc2faca43d/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/pacific/eeb13ac974775bd0cce96a2c687e8bcc2faca43d/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-09-30T22:23:40.594 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-09-30T22:23:40.594 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-09-30T22:23:40.629 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, rbd-fuse, rbd-mirror, rbd-nbd, python-rados, python-rgw, python-rbd, python-cephfs, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux on remote rpm x86_64 2023-09-30T22:23:40.629 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-09-30T22:23:40.629 INFO:teuthology.packaging:ref: None 2023-09-30T22:23:40.629 INFO:teuthology.packaging:tag: None 2023-09-30T22:23:40.629 INFO:teuthology.packaging:branch: pacific 2023-09-30T22:23:40.630 INFO:teuthology.packaging:sha1: 3d6eb91756d99e822eeed7c6b783104a6ad86054 2023-09-30T22:23:40.630 DEBUG:teuthology.orchestra.run.smithi016:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/pacific/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-09-30T22:23:40.674 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/pacific/eeb13ac974775bd0cce96a2c687e8bcc2faca43d/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/pacific/eeb13ac974775bd0cce96a2c687e8bcc2faca43d/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/pacific/eeb13ac974775bd0cce96a2c687e8bcc2faca43d/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-09-30T22:23:40.675 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-09-30T22:23:40.675 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-09-30T22:23:40.705 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, rbd-fuse, rbd-mirror, rbd-nbd, python-rados, python-rgw, python-rbd, python-cephfs, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux on remote rpm x86_64 2023-09-30T22:23:40.705 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-09-30T22:23:40.705 INFO:teuthology.packaging:ref: None 2023-09-30T22:23:40.705 INFO:teuthology.packaging:tag: None 2023-09-30T22:23:40.705 INFO:teuthology.packaging:branch: pacific 2023-09-30T22:23:40.705 INFO:teuthology.packaging:sha1: 3d6eb91756d99e822eeed7c6b783104a6ad86054 2023-09-30T22:23:40.705 DEBUG:teuthology.orchestra.run.smithi027:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/pacific/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-09-30T22:23:40.706 DEBUG:teuthology.orchestra.run.smithi016:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-09-30T22:23:40.775 DEBUG:teuthology.orchestra.run.smithi027:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-09-30T22:23:40.795 DEBUG:teuthology.orchestra.run.smithi016:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-09-30T22:23:40.828 INFO:teuthology.orchestra.run.smithi016.stdout:check_obsoletes = 1 2023-09-30T22:23:40.829 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum clean all 2023-09-30T22:23:40.855 DEBUG:teuthology.orchestra.run.smithi027:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-09-30T22:23:40.923 INFO:teuthology.orchestra.run.smithi027.stdout:check_obsoletes = 1 2023-09-30T22:23:40.924 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum clean all 2023-09-30T22:23:41.149 INFO:teuthology.orchestra.run.smithi016.stdout:76 files removed 2023-09-30T22:23:41.182 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-fuse librados-devel libcephfs2 libcephfs-devel librados2 librbd1 rbd-fuse rbd-mirror rbd-nbd python-rados python-rgw python-rbd python-cephfs bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux 2023-09-30T22:23:41.203 INFO:teuthology.orchestra.run.smithi027.stdout:76 files removed 2023-09-30T22:23:41.232 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-fuse librados-devel libcephfs2 libcephfs-devel librados2 librbd1 rbd-fuse rbd-mirror rbd-nbd python-rados python-rgw python-rbd python-cephfs bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux 2023-09-30T22:23:42.037 INFO:teuthology.orchestra.run.smithi016.stdout:ceph packages for x86_64 199 kB/s | 76 kB 00:00 2023-09-30T22:23:42.089 INFO:teuthology.orchestra.run.smithi027.stdout:ceph packages for x86_64 178 kB/s | 76 kB 00:00 2023-09-30T22:23:42.404 INFO:teuthology.orchestra.run.smithi016.stdout:ceph noarch packages 47 kB/s | 15 kB 00:00 2023-09-30T22:23:42.498 INFO:teuthology.orchestra.run.smithi027.stdout:ceph noarch packages 42 kB/s | 15 kB 00:00 2023-09-30T22:23:42.679 INFO:teuthology.orchestra.run.smithi016.stdout:ceph source packages 5.9 kB/s | 1.4 kB 00:00 2023-09-30T22:23:42.857 INFO:teuthology.orchestra.run.smithi027.stdout:ceph source packages 4.4 kB/s | 1.4 kB 00:00 2023-09-30T22:23:44.963 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 8 - AppStream 15 MB/s | 33 MB 00:02 2023-09-30T22:23:45.299 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 8 - AppStream 14 MB/s | 33 MB 00:02 2023-09-30T22:23:55.242 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 8 - BaseOS 14 MB/s | 48 MB 00:03 2023-09-30T22:23:55.246 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 8 - BaseOS 13 MB/s | 48 MB 00:03 2023-09-30T22:24:03.240 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 8 - Extras 84 kB/s | 18 kB 00:00 2023-09-30T22:24:03.284 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 8 - Extras 23 kB/s | 18 kB 00:00 2023-09-30T22:24:04.231 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 8 - Extras common packages 8.8 kB/s | 6.9 kB 00:00 2023-09-30T22:24:04.251 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 8 - Extras common packages 9.0 kB/s | 6.9 kB 00:00 2023-09-30T22:24:04.699 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 8 - PowerTools 22 MB/s | 6.5 MB 00:00 2023-09-30T22:24:04.711 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 8 - PowerTools 22 MB/s | 6.5 MB 00:00 2023-09-30T22:24:06.212 INFO:teuthology.orchestra.run.smithi027.stdout:Copr repo for python3-asyncssh owned by ceph 27 kB/s | 3.5 kB 00:00 2023-09-30T22:24:06.259 INFO:teuthology.orchestra.run.smithi016.stdout:Copr repo for python3-asyncssh owned by ceph 26 kB/s | 3.5 kB 00:00 2023-09-30T22:24:07.335 INFO:teuthology.orchestra.run.smithi016.stdout:Extra Packages for Enterprise Linux 18 MB/s | 16 MB 00:00 2023-09-30T22:24:07.392 INFO:teuthology.orchestra.run.smithi027.stdout:Extra Packages for Enterprise Linux 16 MB/s | 16 MB 00:01 2023-09-30T22:24:11.472 INFO:teuthology.orchestra.run.smithi027.stdout:lab-extras 377 kB/s | 24 kB 00:00 2023-09-30T22:24:11.627 INFO:teuthology.orchestra.run.smithi016.stdout:lab-extras 492 kB/s | 24 kB 00:00 2023-09-30T22:24:15.959 INFO:teuthology.orchestra.run.smithi027.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-30T22:24:15.960 INFO:teuthology.orchestra.run.smithi027.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-30T22:24:15.985 INFO:teuthology.orchestra.run.smithi027.stdout:Package NetworkManager-1:1.40.16-4.el8.x86_64 is already installed. 2023-09-30T22:24:15.985 INFO:teuthology.orchestra.run.smithi027.stdout:Package NetworkManager-initscripts-updown-1:1.40.16-4.el8.noarch is already installed. 2023-09-30T22:24:15.986 INFO:teuthology.orchestra.run.smithi027.stdout:Package iproute-6.2.0-1.el8.x86_64 is already installed. 2023-09-30T22:24:15.987 INFO:teuthology.orchestra.run.smithi027.stdout:Package util-linux-2.32.1-41.el8.x86_64 is already installed. 2023-09-30T22:24:16.209 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2023-09-30T22:24:16.221 INFO:teuthology.orchestra.run.smithi027.stdout:==================================================================================================== 2023-09-30T22:24:16.221 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2023-09-30T22:24:16.221 INFO:teuthology.orchestra.run.smithi027.stdout:==================================================================================================== 2023-09-30T22:24:16.221 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2023-09-30T22:24:16.222 INFO:teuthology.orchestra.run.smithi027.stdout: bison x86_64 3.0.4-10.el8 appstream 688 k 2023-09-30T22:24:16.222 INFO:teuthology.orchestra.run.smithi027.stdout: ceph x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 6.4 k 2023-09-30T22:24:16.222 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-base x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 6.1 M 2023-09-30T22:24:16.222 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-fuse x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 910 k 2023-09-30T22:24:16.222 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 1.6 M 2023-09-30T22:24:16.222 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-dashboard noarch 2:16.2.14-44.geeb13ac9.el8 ceph-noarch 3.1 M 2023-09-30T22:24:16.222 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-diskprediction-local noarch 2:16.2.14-44.geeb13ac9.el8 ceph-noarch 8.6 M 2023-09-30T22:24:16.222 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-rook noarch 2:16.2.14-44.geeb13ac9.el8 ceph-noarch 27 k 2023-09-30T22:24:16.222 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-radosgw x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 12 M 2023-09-30T22:24:16.222 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-test x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 52 M 2023-09-30T22:24:16.222 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 baseos 62 k 2023-09-30T22:24:16.223 INFO:teuthology.orchestra.run.smithi027.stdout: flex x86_64 2.6.1-9.el8 appstream 320 k 2023-09-30T22:24:16.223 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs-devel x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 25 k 2023-09-30T22:24:16.223 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs2 x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 824 k 2023-09-30T22:24:16.223 INFO:teuthology.orchestra.run.smithi027.stdout: librados-devel x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 131 k 2023-09-30T22:24:16.223 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-devel x86_64 1:1.1.1k-9.el8 baseos 3.2 M 2023-09-30T22:24:16.223 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cephfs x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 215 k 2023-09-30T22:24:16.223 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rados x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 390 k 2023-09-30T22:24:16.223 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rbd x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 369 k 2023-09-30T22:24:16.223 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rgw x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 114 k 2023-09-30T22:24:16.223 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-fuse x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 88 k 2023-09-30T22:24:16.223 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-mirror x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 3.7 M 2023-09-30T22:24:16.223 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-nbd x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 173 k 2023-09-30T22:24:16.224 INFO:teuthology.orchestra.run.smithi027.stdout:Upgrading: 2023-09-30T22:24:16.224 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-9.el8 baseos 143 k 2023-09-30T22:24:16.224 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-debuginfod-client x86_64 0.189-3.el8 baseos 76 k 2023-09-30T22:24:16.224 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-libelf x86_64 0.189-3.el8 baseos 232 k 2023-09-30T22:24:16.224 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-libs x86_64 0.189-3.el8 baseos 303 k 2023-09-30T22:24:16.224 INFO:teuthology.orchestra.run.smithi027.stdout: iproute x86_64 6.2.0-2.el8 baseos 880 k 2023-09-30T22:24:16.224 INFO:teuthology.orchestra.run.smithi027.stdout: libblkid x86_64 2.32.1-43.el8 baseos 220 k 2023-09-30T22:24:16.224 INFO:teuthology.orchestra.run.smithi027.stdout: libfdisk x86_64 2.32.1-43.el8 baseos 254 k 2023-09-30T22:24:16.224 INFO:teuthology.orchestra.run.smithi027.stdout: libmount x86_64 2.32.1-43.el8 baseos 236 k 2023-09-30T22:24:16.224 INFO:teuthology.orchestra.run.smithi027.stdout: librados2 x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 3.7 M 2023-09-30T22:24:16.225 INFO:teuthology.orchestra.run.smithi027.stdout: librbd1 x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 4.2 M 2023-09-30T22:24:16.225 INFO:teuthology.orchestra.run.smithi027.stdout: libsmartcols x86_64 2.32.1-43.el8 baseos 179 k 2023-09-30T22:24:16.225 INFO:teuthology.orchestra.run.smithi027.stdout: libuuid x86_64 2.32.1-43.el8 baseos 99 k 2023-09-30T22:24:16.225 INFO:teuthology.orchestra.run.smithi027.stdout: util-linux x86_64 2.32.1-43.el8 baseos 2.5 M 2023-09-30T22:24:16.225 INFO:teuthology.orchestra.run.smithi027.stdout: util-linux-user x86_64 2.32.1-43.el8 baseos 103 k 2023-09-30T22:24:16.225 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2023-09-30T22:24:16.225 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-common x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 24 M 2023-09-30T22:24:16.225 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-grafana-dashboards noarch 2:16.2.14-44.geeb13ac9.el8 ceph-noarch 24 k 2023-09-30T22:24:16.225 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mds x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 2.4 M 2023-09-30T22:24:16.225 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-modules-core noarch 2:16.2.14-44.geeb13ac9.el8 ceph-noarch 244 k 2023-09-30T22:24:16.226 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mon x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 5.0 M 2023-09-30T22:24:16.226 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-osd x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 18 M 2023-09-30T22:24:16.226 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-prometheus-alerts noarch 2:16.2.14-44.geeb13ac9.el8 ceph-noarch 15 k 2023-09-30T22:24:16.226 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-selinux x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 26 k 2023-09-30T22:24:16.226 INFO:teuthology.orchestra.run.smithi027.stdout: fuse x86_64 2.9.7-17.el8 baseos 83 k 2023-09-30T22:24:16.226 INFO:teuthology.orchestra.run.smithi027.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-09-30T22:24:16.226 INFO:teuthology.orchestra.run.smithi027.stdout: jq x86_64 1.6-7.el8 appstream 202 k 2023-09-30T22:24:16.226 INFO:teuthology.orchestra.run.smithi027.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 baseos 48 k 2023-09-30T22:24:16.226 INFO:teuthology.orchestra.run.smithi027.stdout: krb5-devel x86_64 1.18.2-22.el8 baseos 561 k 2023-09-30T22:24:16.226 INFO:teuthology.orchestra.run.smithi027.stdout: leveldb x86_64 1.22-1.el8 epel 181 k 2023-09-30T22:24:16.227 INFO:teuthology.orchestra.run.smithi027.stdout: libcephsqlite x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 174 k 2023-09-30T22:24:16.227 INFO:teuthology.orchestra.run.smithi027.stdout: libcom_err-devel x86_64 1.45.6-5.el8 baseos 39 k 2023-09-30T22:24:16.227 INFO:teuthology.orchestra.run.smithi027.stdout: libgfortran x86_64 8.5.0-20.el8 baseos 645 k 2023-09-30T22:24:16.227 INFO:teuthology.orchestra.run.smithi027.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-09-30T22:24:16.227 INFO:teuthology.orchestra.run.smithi027.stdout: libquadmath x86_64 8.5.0-20.el8 baseos 172 k 2023-09-30T22:24:16.227 INFO:teuthology.orchestra.run.smithi027.stdout: librabbitmq x86_64 0.9.0-4.el8 baseos 47 k 2023-09-30T22:24:16.227 INFO:teuthology.orchestra.run.smithi027.stdout: libradosstriper1 x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 493 k 2023-09-30T22:24:16.227 INFO:teuthology.orchestra.run.smithi027.stdout: librdkafka x86_64 0.11.4-3.el8 appstream 354 k 2023-09-30T22:24:16.227 INFO:teuthology.orchestra.run.smithi027.stdout: librgw2 x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 3.8 M 2023-09-30T22:24:16.227 INFO:teuthology.orchestra.run.smithi027.stdout: libselinux-devel x86_64 2.9-8.el8 baseos 200 k 2023-09-30T22:24:16.228 INFO:teuthology.orchestra.run.smithi027.stdout: libsepol-devel x86_64 2.9-3.el8 baseos 87 k 2023-09-30T22:24:16.228 INFO:teuthology.orchestra.run.smithi027.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-09-30T22:24:16.228 INFO:teuthology.orchestra.run.smithi027.stdout: libverto-devel x86_64 0.3.2-2.el8 baseos 18 k 2023-09-30T22:24:16.228 INFO:teuthology.orchestra.run.smithi027.stdout: libzstd-devel x86_64 1.4.4-1.el8 baseos 44 k 2023-09-30T22:24:16.228 INFO:teuthology.orchestra.run.smithi027.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2023-09-30T22:24:16.228 INFO:teuthology.orchestra.run.smithi027.stdout: oniguruma x86_64 6.8.2-2.el8 appstream 187 k 2023-09-30T22:24:16.228 INFO:teuthology.orchestra.run.smithi027.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2023-09-30T22:24:16.228 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2023-09-30T22:24:16.228 INFO:teuthology.orchestra.run.smithi027.stdout: pcre2-devel x86_64 10.32-3.el8 baseos 605 k 2023-09-30T22:24:16.228 INFO:teuthology.orchestra.run.smithi027.stdout: pcre2-utf16 x86_64 10.32-3.el8 baseos 229 k 2023-09-30T22:24:16.228 INFO:teuthology.orchestra.run.smithi027.stdout: pcre2-utf32 x86_64 10.32-3.el8 baseos 220 k 2023-09-30T22:24:16.229 INFO:teuthology.orchestra.run.smithi027.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-09-30T22:24:16.229 INFO:teuthology.orchestra.run.smithi027.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-09-30T22:24:16.229 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-09-30T22:24:16.229 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-argparse x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 45 k 2023-09-30T22:24:16.229 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-common x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 99 k 2023-09-30T22:24:16.229 INFO:teuthology.orchestra.run.smithi027.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-09-30T22:24:16.229 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2023-09-30T22:24:16.229 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-09-30T22:24:16.229 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-09-30T22:24:16.229 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cryptography x86_64 3.2.1-6.el8 baseos 641 k 2023-09-30T22:24:16.229 INFO:teuthology.orchestra.run.smithi027.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-09-30T22:24:16.230 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-09-30T22:24:16.230 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-09-30T22:24:16.230 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2023-09-30T22:24:16.230 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2023-09-30T22:24:16.230 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2023-09-30T22:24:16.230 INFO:teuthology.orchestra.run.smithi027.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-09-30T22:24:16.230 INFO:teuthology.orchestra.run.smithi027.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-09-30T22:24:16.230 INFO:teuthology.orchestra.run.smithi027.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2023-09-30T22:24:16.230 INFO:teuthology.orchestra.run.smithi027.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2023-09-30T22:24:16.230 INFO:teuthology.orchestra.run.smithi027.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-09-30T22:24:16.230 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2023-09-30T22:24:16.233 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2023-09-30T22:24:16.233 INFO:teuthology.orchestra.run.smithi027.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2023-09-30T22:24:16.233 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-09-30T22:24:16.233 INFO:teuthology.orchestra.run.smithi027.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-09-30T22:24:16.233 INFO:teuthology.orchestra.run.smithi027.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2023-09-30T22:24:16.233 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2023-09-30T22:24:16.233 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2023-09-30T22:24:16.233 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2023-09-30T22:24:16.233 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2023-09-30T22:24:16.234 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2023-09-30T22:24:16.234 INFO:teuthology.orchestra.run.smithi027.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-09-30T22:24:16.234 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2023-09-30T22:24:16.234 INFO:teuthology.orchestra.run.smithi027.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-09-30T22:24:16.234 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-09-30T22:24:16.234 INFO:teuthology.orchestra.run.smithi027.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 appstream 14 M 2023-09-30T22:24:16.234 INFO:teuthology.orchestra.run.smithi027.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-09-30T22:24:16.234 INFO:teuthology.orchestra.run.smithi027.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-09-30T22:24:16.234 INFO:teuthology.orchestra.run.smithi027.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-09-30T22:24:16.234 INFO:teuthology.orchestra.run.smithi027.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-09-30T22:24:16.234 INFO:teuthology.orchestra.run.smithi027.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-09-30T22:24:16.234 INFO:teuthology.orchestra.run.smithi027.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-09-30T22:24:16.235 INFO:teuthology.orchestra.run.smithi027.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-09-30T22:24:16.235 INFO:teuthology.orchestra.run.smithi027.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-09-30T22:24:16.235 INFO:teuthology.orchestra.run.smithi027.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2023-09-30T22:24:16.235 INFO:teuthology.orchestra.run.smithi027.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-09-30T22:24:16.235 INFO:teuthology.orchestra.run.smithi027.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2023-09-30T22:24:16.235 INFO:teuthology.orchestra.run.smithi027.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2023-09-30T22:24:16.235 INFO:teuthology.orchestra.run.smithi027.stdout: zlib-devel x86_64 1.2.11-21.el8 baseos 58 k 2023-09-30T22:24:16.235 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-09-30T22:24:16.235 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2023-09-30T22:24:16.235 INFO:teuthology.orchestra.run.smithi027.stdout:==================================================================================================== 2023-09-30T22:24:16.235 INFO:teuthology.orchestra.run.smithi027.stdout:Install 108 Packages 2023-09-30T22:24:16.235 INFO:teuthology.orchestra.run.smithi027.stdout:Upgrade 14 Packages 2023-09-30T22:24:16.236 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-09-30T22:24:16.236 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 200 M 2023-09-30T22:24:16.236 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2023-09-30T22:24:16.308 INFO:teuthology.orchestra.run.smithi016.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-30T22:24:16.309 INFO:teuthology.orchestra.run.smithi016.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-09-30T22:24:16.336 INFO:teuthology.orchestra.run.smithi016.stdout:Package NetworkManager-1:1.40.16-4.el8.x86_64 is already installed. 2023-09-30T22:24:16.336 INFO:teuthology.orchestra.run.smithi016.stdout:Package NetworkManager-initscripts-updown-1:1.40.16-4.el8.noarch is already installed. 2023-09-30T22:24:16.337 INFO:teuthology.orchestra.run.smithi016.stdout:Package iproute-6.2.0-1.el8.x86_64 is already installed. 2023-09-30T22:24:16.338 INFO:teuthology.orchestra.run.smithi016.stdout:Package util-linux-2.32.1-41.el8.x86_64 is already installed. 2023-09-30T22:24:16.571 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-09-30T22:24:16.584 INFO:teuthology.orchestra.run.smithi016.stdout:==================================================================================================== 2023-09-30T22:24:16.584 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2023-09-30T22:24:16.584 INFO:teuthology.orchestra.run.smithi016.stdout:==================================================================================================== 2023-09-30T22:24:16.584 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2023-09-30T22:24:16.584 INFO:teuthology.orchestra.run.smithi016.stdout: bison x86_64 3.0.4-10.el8 appstream 688 k 2023-09-30T22:24:16.584 INFO:teuthology.orchestra.run.smithi016.stdout: ceph x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 6.4 k 2023-09-30T22:24:16.584 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 6.1 M 2023-09-30T22:24:16.585 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 910 k 2023-09-30T22:24:16.585 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 1.6 M 2023-09-30T22:24:16.585 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard noarch 2:16.2.14-44.geeb13ac9.el8 ceph-noarch 3.1 M 2023-09-30T22:24:16.585 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local noarch 2:16.2.14-44.geeb13ac9.el8 ceph-noarch 8.6 M 2023-09-30T22:24:16.585 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook noarch 2:16.2.14-44.geeb13ac9.el8 ceph-noarch 27 k 2023-09-30T22:24:16.585 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 12 M 2023-09-30T22:24:16.585 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 52 M 2023-09-30T22:24:16.585 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 baseos 62 k 2023-09-30T22:24:16.585 INFO:teuthology.orchestra.run.smithi016.stdout: flex x86_64 2.6.1-9.el8 appstream 320 k 2023-09-30T22:24:16.585 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 25 k 2023-09-30T22:24:16.585 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2 x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 824 k 2023-09-30T22:24:16.586 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 131 k 2023-09-30T22:24:16.586 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-devel x86_64 1:1.1.1k-9.el8 baseos 3.2 M 2023-09-30T22:24:16.586 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 215 k 2023-09-30T22:24:16.586 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 390 k 2023-09-30T22:24:16.586 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 369 k 2023-09-30T22:24:16.586 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 114 k 2023-09-30T22:24:16.586 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 88 k 2023-09-30T22:24:16.586 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 3.7 M 2023-09-30T22:24:16.586 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 173 k 2023-09-30T22:24:16.586 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrading: 2023-09-30T22:24:16.586 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-9.el8 baseos 143 k 2023-09-30T22:24:16.587 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-debuginfod-client x86_64 0.189-3.el8 baseos 76 k 2023-09-30T22:24:16.587 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf x86_64 0.189-3.el8 baseos 232 k 2023-09-30T22:24:16.587 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libs x86_64 0.189-3.el8 baseos 303 k 2023-09-30T22:24:16.587 INFO:teuthology.orchestra.run.smithi016.stdout: iproute x86_64 6.2.0-2.el8 baseos 880 k 2023-09-30T22:24:16.587 INFO:teuthology.orchestra.run.smithi016.stdout: libblkid x86_64 2.32.1-43.el8 baseos 220 k 2023-09-30T22:24:16.587 INFO:teuthology.orchestra.run.smithi016.stdout: libfdisk x86_64 2.32.1-43.el8 baseos 254 k 2023-09-30T22:24:16.587 INFO:teuthology.orchestra.run.smithi016.stdout: libmount x86_64 2.32.1-43.el8 baseos 236 k 2023-09-30T22:24:16.587 INFO:teuthology.orchestra.run.smithi016.stdout: librados2 x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 3.7 M 2023-09-30T22:24:16.587 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1 x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 4.2 M 2023-09-30T22:24:16.587 INFO:teuthology.orchestra.run.smithi016.stdout: libsmartcols x86_64 2.32.1-43.el8 baseos 179 k 2023-09-30T22:24:16.587 INFO:teuthology.orchestra.run.smithi016.stdout: libuuid x86_64 2.32.1-43.el8 baseos 99 k 2023-09-30T22:24:16.588 INFO:teuthology.orchestra.run.smithi016.stdout: util-linux x86_64 2.32.1-43.el8 baseos 2.5 M 2023-09-30T22:24:16.588 INFO:teuthology.orchestra.run.smithi016.stdout: util-linux-user x86_64 2.32.1-43.el8 baseos 103 k 2023-09-30T22:24:16.588 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2023-09-30T22:24:16.588 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 24 M 2023-09-30T22:24:16.588 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards noarch 2:16.2.14-44.geeb13ac9.el8 ceph-noarch 24 k 2023-09-30T22:24:16.588 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 2.4 M 2023-09-30T22:24:16.588 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core noarch 2:16.2.14-44.geeb13ac9.el8 ceph-noarch 244 k 2023-09-30T22:24:16.588 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 5.0 M 2023-09-30T22:24:16.588 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 18 M 2023-09-30T22:24:16.588 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts noarch 2:16.2.14-44.geeb13ac9.el8 ceph-noarch 15 k 2023-09-30T22:24:16.588 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 26 k 2023-09-30T22:24:16.588 INFO:teuthology.orchestra.run.smithi016.stdout: fuse x86_64 2.9.7-17.el8 baseos 83 k 2023-09-30T22:24:16.589 INFO:teuthology.orchestra.run.smithi016.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-09-30T22:24:16.589 INFO:teuthology.orchestra.run.smithi016.stdout: jq x86_64 1.6-7.el8 appstream 202 k 2023-09-30T22:24:16.589 INFO:teuthology.orchestra.run.smithi016.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 baseos 48 k 2023-09-30T22:24:16.589 INFO:teuthology.orchestra.run.smithi016.stdout: krb5-devel x86_64 1.18.2-22.el8 baseos 561 k 2023-09-30T22:24:16.589 INFO:teuthology.orchestra.run.smithi016.stdout: leveldb x86_64 1.22-1.el8 epel 181 k 2023-09-30T22:24:16.589 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 174 k 2023-09-30T22:24:16.589 INFO:teuthology.orchestra.run.smithi016.stdout: libcom_err-devel x86_64 1.45.6-5.el8 baseos 39 k 2023-09-30T22:24:16.589 INFO:teuthology.orchestra.run.smithi016.stdout: libgfortran x86_64 8.5.0-20.el8 baseos 645 k 2023-09-30T22:24:16.589 INFO:teuthology.orchestra.run.smithi016.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-09-30T22:24:16.589 INFO:teuthology.orchestra.run.smithi016.stdout: libquadmath x86_64 8.5.0-20.el8 baseos 172 k 2023-09-30T22:24:16.589 INFO:teuthology.orchestra.run.smithi016.stdout: librabbitmq x86_64 0.9.0-4.el8 baseos 47 k 2023-09-30T22:24:16.589 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1 x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 493 k 2023-09-30T22:24:16.590 INFO:teuthology.orchestra.run.smithi016.stdout: librdkafka x86_64 0.11.4-3.el8 appstream 354 k 2023-09-30T22:24:16.590 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2 x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 3.8 M 2023-09-30T22:24:16.590 INFO:teuthology.orchestra.run.smithi016.stdout: libselinux-devel x86_64 2.9-8.el8 baseos 200 k 2023-09-30T22:24:16.590 INFO:teuthology.orchestra.run.smithi016.stdout: libsepol-devel x86_64 2.9-3.el8 baseos 87 k 2023-09-30T22:24:16.590 INFO:teuthology.orchestra.run.smithi016.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-09-30T22:24:16.590 INFO:teuthology.orchestra.run.smithi016.stdout: libverto-devel x86_64 0.3.2-2.el8 baseos 18 k 2023-09-30T22:24:16.590 INFO:teuthology.orchestra.run.smithi016.stdout: libzstd-devel x86_64 1.4.4-1.el8 baseos 44 k 2023-09-30T22:24:16.590 INFO:teuthology.orchestra.run.smithi016.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2023-09-30T22:24:16.590 INFO:teuthology.orchestra.run.smithi016.stdout: oniguruma x86_64 6.8.2-2.el8 appstream 187 k 2023-09-30T22:24:16.591 INFO:teuthology.orchestra.run.smithi016.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2023-09-30T22:24:16.591 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2023-09-30T22:24:16.591 INFO:teuthology.orchestra.run.smithi016.stdout: pcre2-devel x86_64 10.32-3.el8 baseos 605 k 2023-09-30T22:24:16.591 INFO:teuthology.orchestra.run.smithi016.stdout: pcre2-utf16 x86_64 10.32-3.el8 baseos 229 k 2023-09-30T22:24:16.591 INFO:teuthology.orchestra.run.smithi016.stdout: pcre2-utf32 x86_64 10.32-3.el8 baseos 220 k 2023-09-30T22:24:16.591 INFO:teuthology.orchestra.run.smithi016.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-09-30T22:24:16.591 INFO:teuthology.orchestra.run.smithi016.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-09-30T22:24:16.591 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-09-30T22:24:16.591 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 45 k 2023-09-30T22:24:16.591 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common x86_64 2:16.2.14-44.geeb13ac9.el8 ceph 99 k 2023-09-30T22:24:16.591 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-09-30T22:24:16.592 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2023-09-30T22:24:16.592 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-09-30T22:24:16.592 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-09-30T22:24:16.592 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cryptography x86_64 3.2.1-6.el8 baseos 641 k 2023-09-30T22:24:16.592 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-09-30T22:24:16.592 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-09-30T22:24:16.592 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-09-30T22:24:16.592 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2023-09-30T22:24:16.592 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2023-09-30T22:24:16.592 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2023-09-30T22:24:16.592 INFO:teuthology.orchestra.run.smithi016.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-09-30T22:24:16.592 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-09-30T22:24:16.593 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2023-09-30T22:24:16.593 INFO:teuthology.orchestra.run.smithi016.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2023-09-30T22:24:16.593 INFO:teuthology.orchestra.run.smithi016.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-09-30T22:24:16.593 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2023-09-30T22:24:16.593 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2023-09-30T22:24:16.593 INFO:teuthology.orchestra.run.smithi016.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2023-09-30T22:24:16.593 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-09-30T22:24:16.593 INFO:teuthology.orchestra.run.smithi016.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-09-30T22:24:16.593 INFO:teuthology.orchestra.run.smithi016.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2023-09-30T22:24:16.593 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2023-09-30T22:24:16.593 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2023-09-30T22:24:16.593 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2023-09-30T22:24:16.594 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2023-09-30T22:24:16.594 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2023-09-30T22:24:16.594 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-09-30T22:24:16.594 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2023-09-30T22:24:16.594 INFO:teuthology.orchestra.run.smithi016.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-09-30T22:24:16.594 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-09-30T22:24:16.594 INFO:teuthology.orchestra.run.smithi016.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 appstream 14 M 2023-09-30T22:24:16.594 INFO:teuthology.orchestra.run.smithi016.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-09-30T22:24:16.594 INFO:teuthology.orchestra.run.smithi016.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-09-30T22:24:16.594 INFO:teuthology.orchestra.run.smithi016.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-09-30T22:24:16.594 INFO:teuthology.orchestra.run.smithi016.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-09-30T22:24:16.594 INFO:teuthology.orchestra.run.smithi016.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-09-30T22:24:16.595 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-09-30T22:24:16.595 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-09-30T22:24:16.595 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-09-30T22:24:16.595 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2023-09-30T22:24:16.595 INFO:teuthology.orchestra.run.smithi016.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-09-30T22:24:16.595 INFO:teuthology.orchestra.run.smithi016.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2023-09-30T22:24:16.595 INFO:teuthology.orchestra.run.smithi016.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2023-09-30T22:24:16.595 INFO:teuthology.orchestra.run.smithi016.stdout: zlib-devel x86_64 1.2.11-21.el8 baseos 58 k 2023-09-30T22:24:16.595 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:24:16.595 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2023-09-30T22:24:16.595 INFO:teuthology.orchestra.run.smithi016.stdout:==================================================================================================== 2023-09-30T22:24:16.595 INFO:teuthology.orchestra.run.smithi016.stdout:Install 108 Packages 2023-09-30T22:24:16.596 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrade 14 Packages 2023-09-30T22:24:16.596 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:24:16.596 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 200 M 2023-09-30T22:24:16.596 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2023-09-30T22:24:17.057 INFO:teuthology.orchestra.run.smithi027.stdout:(1/122): ceph-16.2.14-44.geeb13ac9.el8.x86_64.r 17 kB/s | 6.4 kB 00:00 2023-09-30T22:24:17.537 INFO:teuthology.orchestra.run.smithi016.stdout:(1/122): ceph-16.2.14-44.geeb13ac9.el8.x86_64.r 18 kB/s | 6.4 kB 00:00 2023-09-30T22:24:17.774 INFO:teuthology.orchestra.run.smithi027.stdout:(2/122): ceph-fuse-16.2.14-44.geeb13ac9.el8.x86 1.2 MB/s | 910 kB 00:00 2023-09-30T22:24:18.591 INFO:teuthology.orchestra.run.smithi027.stdout:(3/122): ceph-mds-16.2.14-44.geeb13ac9.el8.x86_ 3.0 MB/s | 2.4 MB 00:00 2023-09-30T22:24:18.966 INFO:teuthology.orchestra.run.smithi027.stdout:(4/122): ceph-mgr-16.2.14-44.geeb13ac9.el8.x86_ 4.3 MB/s | 1.6 MB 00:00 2023-09-30T22:24:19.453 INFO:teuthology.orchestra.run.smithi016.stdout:(2/122): ceph-base-16.2.14-44.geeb13ac9.el8.x86 2.7 MB/s | 6.1 MB 00:02 2023-09-30T22:24:19.591 INFO:teuthology.orchestra.run.smithi027.stdout:(5/122): ceph-base-16.2.14-44.geeb13ac9.el8.x86 2.1 MB/s | 6.1 MB 00:02 2023-09-30T22:24:19.895 INFO:teuthology.orchestra.run.smithi016.stdout:(3/122): ceph-mds-16.2.14-44.geeb13ac9.el8.x86_ 5.5 MB/s | 2.4 MB 00:00 2023-09-30T22:24:20.158 INFO:teuthology.orchestra.run.smithi027.stdout:(6/122): ceph-mon-16.2.14-44.geeb13ac9.el8.x86_ 4.2 MB/s | 5.0 MB 00:01 2023-09-30T22:24:20.162 INFO:teuthology.orchestra.run.smithi016.stdout:(4/122): ceph-mgr-16.2.14-44.geeb13ac9.el8.x86_ 6.0 MB/s | 1.6 MB 00:00 2023-09-30T22:24:20.605 INFO:teuthology.orchestra.run.smithi016.stdout:(5/122): ceph-fuse-16.2.14-44.geeb13ac9.el8.x86 297 kB/s | 910 kB 00:03 2023-09-30T22:24:20.771 INFO:teuthology.orchestra.run.smithi016.stdout:(6/122): ceph-mon-16.2.14-44.geeb13ac9.el8.x86_ 8.3 MB/s | 5.0 MB 00:00 2023-09-30T22:24:21.988 INFO:teuthology.orchestra.run.smithi016.stdout:(7/122): ceph-radosgw-16.2.14-44.geeb13ac9.el8. 9.5 MB/s | 12 MB 00:01 2023-09-30T22:24:22.055 INFO:teuthology.orchestra.run.smithi016.stdout:(8/122): ceph-selinux-16.2.14-44.geeb13ac9.el8. 386 kB/s | 26 kB 00:00 2023-09-30T22:24:22.891 INFO:teuthology.orchestra.run.smithi027.stdout:(7/122): ceph-radosgw-16.2.14-44.geeb13ac9.el8. 4.2 MB/s | 12 MB 00:02 2023-09-30T22:24:23.025 INFO:teuthology.orchestra.run.smithi027.stdout:(8/122): ceph-selinux-16.2.14-44.geeb13ac9.el8. 192 kB/s | 26 kB 00:00 2023-09-30T22:24:26.492 INFO:teuthology.orchestra.run.smithi027.stdout:(9/122): ceph-osd-16.2.14-44.geeb13ac9.el8.x86_ 2.7 MB/s | 18 MB 00:06 2023-09-30T22:24:26.559 INFO:teuthology.orchestra.run.smithi027.stdout:(10/122): libcephfs-devel-16.2.14-44.geeb13ac9. 385 kB/s | 25 kB 00:00 2023-09-30T22:24:26.638 INFO:teuthology.orchestra.run.smithi016.stdout:(9/122): ceph-test-16.2.14-44.geeb13ac9.el8.x86 11 MB/s | 52 MB 00:04 2023-09-30T22:24:26.739 INFO:teuthology.orchestra.run.smithi016.stdout:(10/122): libcephfs-devel-16.2.14-44.geeb13ac9. 256 kB/s | 25 kB 00:00 2023-09-30T22:24:26.851 INFO:teuthology.orchestra.run.smithi027.stdout:(11/122): libcephfs2-16.2.14-44.geeb13ac9.el8.x 2.8 MB/s | 824 kB 00:00 2023-09-30T22:24:26.951 INFO:teuthology.orchestra.run.smithi027.stdout:(12/122): libcephsqlite-16.2.14-44.geeb13ac9.el 1.7 MB/s | 174 kB 00:00 2023-09-30T22:24:26.997 INFO:teuthology.orchestra.run.smithi016.stdout:(11/122): libcephfs2-16.2.14-44.geeb13ac9.el8.x 3.1 MB/s | 824 kB 00:00 2023-09-30T22:24:27.034 INFO:teuthology.orchestra.run.smithi027.stdout:(13/122): librados-devel-16.2.14-44.geeb13ac9.e 1.5 MB/s | 131 kB 00:00 2023-09-30T22:24:27.081 INFO:teuthology.orchestra.run.smithi016.stdout:(12/122): libcephsqlite-16.2.14-44.geeb13ac9.el 2.0 MB/s | 174 kB 00:00 2023-09-30T22:24:27.156 INFO:teuthology.orchestra.run.smithi016.stdout:(13/122): librados-devel-16.2.14-44.geeb13ac9.e 1.7 MB/s | 131 kB 00:00 2023-09-30T22:24:27.218 INFO:teuthology.orchestra.run.smithi027.stdout:(14/122): libradosstriper1-16.2.14-44.geeb13ac9 2.6 MB/s | 493 kB 00:00 2023-09-30T22:24:27.256 INFO:teuthology.orchestra.run.smithi016.stdout:(14/122): libradosstriper1-16.2.14-44.geeb13ac9 4.8 MB/s | 493 kB 00:00 2023-09-30T22:24:27.807 INFO:teuthology.orchestra.run.smithi016.stdout:(15/122): librgw2-16.2.14-44.geeb13ac9.el8.x86_ 6.9 MB/s | 3.8 MB 00:00 2023-09-30T22:24:27.891 INFO:teuthology.orchestra.run.smithi016.stdout:(16/122): python3-ceph-argparse-16.2.14-44.geeb 541 kB/s | 45 kB 00:00 2023-09-30T22:24:27.966 INFO:teuthology.orchestra.run.smithi016.stdout:(17/122): python3-ceph-common-16.2.14-44.geeb13 1.3 MB/s | 99 kB 00:00 2023-09-30T22:24:28.183 INFO:teuthology.orchestra.run.smithi016.stdout:(18/122): python3-cephfs-16.2.14-44.geeb13ac9.e 994 kB/s | 215 kB 00:00 2023-09-30T22:24:28.266 INFO:teuthology.orchestra.run.smithi016.stdout:(19/122): python3-rados-16.2.14-44.geeb13ac9.el 4.6 MB/s | 390 kB 00:00 2023-09-30T22:24:28.383 INFO:teuthology.orchestra.run.smithi016.stdout:(20/122): python3-rbd-16.2.14-44.geeb13ac9.el8. 3.1 MB/s | 369 kB 00:00 2023-09-30T22:24:28.450 INFO:teuthology.orchestra.run.smithi016.stdout:(21/122): python3-rgw-16.2.14-44.geeb13ac9.el8. 1.7 MB/s | 114 kB 00:00 2023-09-30T22:24:28.485 INFO:teuthology.orchestra.run.smithi027.stdout:(15/122): librgw2-16.2.14-44.geeb13ac9.el8.x86_ 3.0 MB/s | 3.8 MB 00:01 2023-09-30T22:24:28.509 INFO:teuthology.orchestra.run.smithi016.stdout:(22/122): rbd-fuse-16.2.14-44.geeb13ac9.el8.x86 1.5 MB/s | 88 kB 00:00 2023-09-30T22:24:28.552 INFO:teuthology.orchestra.run.smithi027.stdout:(16/122): python3-ceph-argparse-16.2.14-44.geeb 676 kB/s | 45 kB 00:00 2023-09-30T22:24:28.652 INFO:teuthology.orchestra.run.smithi027.stdout:(17/122): python3-ceph-common-16.2.14-44.geeb13 989 kB/s | 99 kB 00:00 2023-09-30T22:24:28.828 INFO:teuthology.orchestra.run.smithi027.stdout:(18/122): python3-cephfs-16.2.14-44.geeb13ac9.e 1.2 MB/s | 215 kB 00:00 2023-09-30T22:24:28.984 INFO:teuthology.orchestra.run.smithi016.stdout:(23/122): rbd-mirror-16.2.14-44.geeb13ac9.el8.x 7.7 MB/s | 3.7 MB 00:00 2023-09-30T22:24:29.028 INFO:teuthology.orchestra.run.smithi027.stdout:(19/122): python3-rados-16.2.14-44.geeb13ac9.el 1.9 MB/s | 390 kB 00:00 2023-09-30T22:24:29.060 INFO:teuthology.orchestra.run.smithi016.stdout:(24/122): rbd-nbd-16.2.14-44.geeb13ac9.el8.x86_ 2.3 MB/s | 173 kB 00:00 2023-09-30T22:24:29.170 INFO:teuthology.orchestra.run.smithi027.stdout:(20/122): python3-rbd-16.2.14-44.geeb13ac9.el8. 2.5 MB/s | 369 kB 00:00 2023-09-30T22:24:29.237 INFO:teuthology.orchestra.run.smithi027.stdout:(21/122): python3-rgw-16.2.14-44.geeb13ac9.el8. 1.7 MB/s | 114 kB 00:00 2023-09-30T22:24:29.252 INFO:teuthology.orchestra.run.smithi016.stdout:(25/122): ceph-grafana-dashboards-16.2.14-44.ge 127 kB/s | 24 kB 00:00 2023-09-30T22:24:29.312 INFO:teuthology.orchestra.run.smithi027.stdout:(22/122): rbd-fuse-16.2.14-44.geeb13ac9.el8.x86 1.2 MB/s | 88 kB 00:00 2023-09-30T22:24:29.744 INFO:teuthology.orchestra.run.smithi016.stdout:(26/122): ceph-mgr-dashboard-16.2.14-44.geeb13a 6.4 MB/s | 3.1 MB 00:00 2023-09-30T22:24:30.477 INFO:teuthology.orchestra.run.smithi016.stdout:(27/122): ceph-common-16.2.14-44.geeb13ac9.el8. 1.8 MB/s | 24 MB 00:13 2023-09-30T22:24:30.620 INFO:teuthology.orchestra.run.smithi016.stdout:(28/122): ceph-mgr-modules-core-16.2.14-44.geeb 1.7 MB/s | 244 kB 00:00 2023-09-30T22:24:30.741 INFO:teuthology.orchestra.run.smithi016.stdout:(29/122): ceph-mgr-rook-16.2.14-44.geeb13ac9.el 227 kB/s | 27 kB 00:00 2023-09-30T22:24:30.746 INFO:teuthology.orchestra.run.smithi027.stdout:(23/122): ceph-common-16.2.14-44.geeb13ac9.el8. 1.7 MB/s | 24 MB 00:14 2023-09-30T22:24:30.828 INFO:teuthology.orchestra.run.smithi016.stdout:(30/122): ceph-prometheus-alerts-16.2.14-44.gee 170 kB/s | 15 kB 00:00 2023-09-30T22:24:30.913 INFO:teuthology.orchestra.run.smithi027.stdout:(24/122): rbd-nbd-16.2.14-44.geeb13ac9.el8.x86_ 1.0 MB/s | 173 kB 00:00 2023-09-30T22:24:30.997 INFO:teuthology.orchestra.run.smithi027.stdout:(25/122): ceph-grafana-dashboards-16.2.14-44.ge 292 kB/s | 24 kB 00:00 2023-09-30T22:24:31.087 INFO:teuthology.orchestra.run.smithi016.stdout:(31/122): ceph-mgr-diskprediction-local-16.2.14 6.4 MB/s | 8.6 MB 00:01 2023-09-30T22:24:31.147 INFO:teuthology.orchestra.run.smithi027.stdout:(26/122): rbd-mirror-16.2.14-44.geeb13ac9.el8.x 2.0 MB/s | 3.7 MB 00:01 2023-09-30T22:24:31.487 INFO:teuthology.orchestra.run.smithi016.stdout:(32/122): bison-3.0.4-10.el8.x86_64.rpm 1.0 MB/s | 688 kB 00:00 2023-09-30T22:24:31.613 INFO:teuthology.orchestra.run.smithi016.stdout:(33/122): jq-1.6-7.el8.x86_64.rpm 1.6 MB/s | 202 kB 00:00 2023-09-30T22:24:31.655 INFO:teuthology.orchestra.run.smithi016.stdout:(34/122): flex-2.6.1-9.el8.x86_64.rpm 563 kB/s | 320 kB 00:00 2023-09-30T22:24:31.680 INFO:teuthology.orchestra.run.smithi027.stdout:(27/122): ceph-mgr-dashboard-16.2.14-44.geeb13a 4.6 MB/s | 3.1 MB 00:00 2023-09-30T22:24:31.755 INFO:teuthology.orchestra.run.smithi016.stdout:(35/122): librdkafka-0.11.4-3.el8.x86_64.rpm 2.4 MB/s | 354 kB 00:00 2023-09-30T22:24:31.764 INFO:teuthology.orchestra.run.smithi027.stdout:(28/122): ceph-mgr-modules-core-16.2.14-44.geeb 2.9 MB/s | 244 kB 00:00 2023-09-30T22:24:31.830 INFO:teuthology.orchestra.run.smithi016.stdout:(36/122): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.4 MB/s | 259 kB 00:00 2023-09-30T22:24:31.831 INFO:teuthology.orchestra.run.smithi027.stdout:(29/122): ceph-mgr-rook-16.2.14-44.geeb13ac9.el 411 kB/s | 27 kB 00:00 2023-09-30T22:24:31.890 INFO:teuthology.orchestra.run.smithi027.stdout:(30/122): ceph-prometheus-alerts-16.2.14-44.gee 254 kB/s | 15 kB 00:00 2023-09-30T22:24:31.939 INFO:teuthology.orchestra.run.smithi016.stdout:(37/122): oniguruma-6.8.2-2.el8.x86_64.rpm 1.0 MB/s | 187 kB 00:00 2023-09-30T22:24:32.298 INFO:teuthology.orchestra.run.smithi016.stdout:(38/122): openblas-0.3.15-4.el8.x86_64.rpm 10 MB/s | 4.7 MB 00:00 2023-09-30T22:24:32.506 INFO:teuthology.orchestra.run.smithi016.stdout:(39/122): openblas-threads-0.3.15-4.el8.x86_64. 8.6 MB/s | 4.9 MB 00:00 2023-09-30T22:24:32.531 INFO:teuthology.orchestra.run.smithi016.stdout:(40/122): python3-jsonpatch-1.21-2.el8.noarch.r 114 kB/s | 27 kB 00:00 2023-09-30T22:24:32.548 INFO:teuthology.orchestra.run.smithi027.stdout:(31/122): bison-3.0.4-10.el8.x86_64.rpm 1.0 MB/s | 688 kB 00:00 2023-09-30T22:24:32.632 INFO:teuthology.orchestra.run.smithi016.stdout:(41/122): python3-jsonpointer-1.10-11.el8.noarc 159 kB/s | 20 kB 00:00 2023-09-30T22:24:32.657 INFO:teuthology.orchestra.run.smithi016.stdout:(42/122): python3-mako-1.0.6-14.el8.noarch.rpm 1.2 MB/s | 157 kB 00:00 2023-09-30T22:24:32.682 INFO:teuthology.orchestra.run.smithi027.stdout:(32/122): flex-2.6.1-9.el8.x86_64.rpm 2.3 MB/s | 320 kB 00:00 2023-09-30T22:24:32.733 INFO:teuthology.orchestra.run.smithi016.stdout:(43/122): python3-markupsafe-0.23-19.el8.x86_64 387 kB/s | 39 kB 00:00 2023-09-30T22:24:32.857 INFO:teuthology.orchestra.run.smithi027.stdout:(33/122): ceph-mgr-diskprediction-local-16.2.14 5.0 MB/s | 8.6 MB 00:01 2023-09-30T22:24:32.862 INFO:teuthology.orchestra.run.smithi016.stdout:(44/122): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.7 MB/s | 225 kB 00:00 2023-09-30T22:24:32.882 INFO:teuthology.orchestra.run.smithi027.stdout:(34/122): jq-1.6-7.el8.x86_64.rpm 1.0 MB/s | 202 kB 00:00 2023-09-30T22:24:33.000 INFO:teuthology.orchestra.run.smithi016.stdout:(45/122): python3-numpy-1.14.3-10.el8.x86_64.rp 11 MB/s | 3.7 MB 00:00 2023-09-30T22:24:33.025 INFO:teuthology.orchestra.run.smithi016.stdout:(46/122): python3-prettytable-0.7.2-14.el8.noar 270 kB/s | 44 kB 00:00 2023-09-30T22:24:33.100 INFO:teuthology.orchestra.run.smithi016.stdout:(47/122): python3-pyOpenSSL-19.0.0-1.el8.noarch 1.0 MB/s | 103 kB 00:00 2023-09-30T22:24:33.159 INFO:teuthology.orchestra.run.smithi016.stdout:(48/122): python3-pyasn1-0.3.7-6.el8.noarch.rpm 945 kB/s | 126 kB 00:00 2023-09-30T22:24:33.201 INFO:teuthology.orchestra.run.smithi016.stdout:(49/122): python3-pyasn1-modules-0.3.7-6.el8.no 1.1 MB/s | 110 kB 00:00 2023-09-30T22:24:33.251 INFO:teuthology.orchestra.run.smithi016.stdout:(50/122): python3-pytz-2017.2-11.el8.noarch.rpm 622 kB/s | 57 kB 00:00 2023-09-30T22:24:33.341 INFO:teuthology.orchestra.run.smithi027.stdout:(35/122): ceph-test-16.2.14-44.geeb13ac9.el8.x8 5.0 MB/s | 52 MB 00:10 2023-09-30T22:24:33.368 INFO:teuthology.orchestra.run.smithi016.stdout:(51/122): python3-werkzeug-0.12.2-4.el8.noarch. 3.8 MB/s | 457 kB 00:00 2023-09-30T22:24:33.384 INFO:teuthology.orchestra.run.smithi027.stdout:(36/122): lttng-ust-2.8.1-11.el8.x86_64.rpm 517 kB/s | 259 kB 00:00 2023-09-30T22:24:33.485 INFO:teuthology.orchestra.run.smithi016.stdout:(52/122): socat-1.7.4.1-1.el8.x86_64.rpm 2.7 MB/s | 323 kB 00:00 2023-09-30T22:24:33.579 INFO:teuthology.orchestra.run.smithi016.stdout:(53/122): xmlstarlet-1.6.1-20.el8.x86_64.rpm 753 kB/s | 71 kB 00:00 2023-09-30T22:24:33.722 INFO:teuthology.orchestra.run.smithi016.stdout:(54/122): elfutils-libelf-devel-0.189-3.el8.x86 435 kB/s | 62 kB 00:00 2023-09-30T22:24:33.767 INFO:teuthology.orchestra.run.smithi027.stdout:(37/122): openblas-0.3.15-4.el8.x86_64.rpm 12 MB/s | 4.7 MB 00:00 2023-09-30T22:24:33.842 INFO:teuthology.orchestra.run.smithi027.stdout:(38/122): librdkafka-0.11.4-3.el8.x86_64.rpm 359 kB/s | 354 kB 00:00 2023-09-30T22:24:33.859 INFO:teuthology.orchestra.run.smithi027.stdout:(39/122): oniguruma-6.8.2-2.el8.x86_64.rpm 362 kB/s | 187 kB 00:00 2023-09-30T22:24:33.864 INFO:teuthology.orchestra.run.smithi016.stdout:(55/122): python3-scipy-1.0.0-21.module_el8.5.0 21 MB/s | 14 MB 00:00 2023-09-30T22:24:33.889 INFO:teuthology.orchestra.run.smithi016.stdout:(56/122): fuse-2.9.7-17.el8.x86_64.rpm 498 kB/s | 83 kB 00:00 2023-09-30T22:24:33.951 INFO:teuthology.orchestra.run.smithi027.stdout:(40/122): python3-jsonpatch-1.21-2.el8.noarch.r 245 kB/s | 27 kB 00:00 2023-09-30T22:24:33.976 INFO:teuthology.orchestra.run.smithi027.stdout:(41/122): python3-jsonpointer-1.10-11.el8.noarc 170 kB/s | 20 kB 00:00 2023-09-30T22:24:34.006 INFO:teuthology.orchestra.run.smithi016.stdout:(57/122): keyutils-libs-devel-1.5.10-9.el8.x86_ 339 kB/s | 48 kB 00:00 2023-09-30T22:24:34.039 INFO:teuthology.orchestra.run.smithi016.stdout:(58/122): krb5-devel-1.18.2-22.el8.x86_64.rpm 3.6 MB/s | 561 kB 00:00 2023-09-30T22:24:34.043 INFO:teuthology.orchestra.run.smithi027.stdout:(42/122): openblas-threads-0.3.15-4.el8.x86_64. 18 MB/s | 4.9 MB 00:00 2023-09-30T22:24:34.064 INFO:teuthology.orchestra.run.smithi016.stdout:(59/122): libcom_err-devel-1.45.6-5.el8.x86_64. 664 kB/s | 39 kB 00:00 2023-09-30T22:24:34.068 INFO:teuthology.orchestra.run.smithi027.stdout:(43/122): python3-mako-1.0.6-14.el8.noarch.rpm 1.3 MB/s | 157 kB 00:00 2023-09-30T22:24:34.094 INFO:teuthology.orchestra.run.smithi027.stdout:(44/122): python3-markupsafe-0.23-19.el8.x86_64 331 kB/s | 39 kB 00:00 2023-09-30T22:24:34.148 INFO:teuthology.orchestra.run.smithi016.stdout:(60/122): libquadmath-8.5.0-20.el8.x86_64.rpm 2.0 MB/s | 172 kB 00:00 2023-09-30T22:24:34.202 INFO:teuthology.orchestra.run.smithi027.stdout:(45/122): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.6 MB/s | 225 kB 00:00 2023-09-30T22:24:34.210 INFO:teuthology.orchestra.run.smithi016.stdout:(61/122): libgfortran-8.5.0-20.el8.x86_64.rpm 3.7 MB/s | 645 kB 00:00 2023-09-30T22:24:34.249 INFO:teuthology.orchestra.run.smithi027.stdout:(46/122): python3-prettytable-0.7.2-14.el8.noar 284 kB/s | 44 kB 00:00 2023-09-30T22:24:34.344 INFO:teuthology.orchestra.run.smithi027.stdout:(47/122): python3-numpy-1.14.3-10.el8.x86_64.rp 12 MB/s | 3.7 MB 00:00 2023-09-30T22:24:34.361 INFO:teuthology.orchestra.run.smithi027.stdout:(48/122): python3-pyOpenSSL-19.0.0-1.el8.noarch 651 kB/s | 103 kB 00:00 2023-09-30T22:24:34.387 INFO:teuthology.orchestra.run.smithi027.stdout:(49/122): python3-pyasn1-0.3.7-6.el8.noarch.rpm 919 kB/s | 126 kB 00:00 2023-09-30T22:24:34.399 INFO:teuthology.orchestra.run.smithi016.stdout:(62/122): librabbitmq-0.9.0-4.el8.x86_64.rpm 187 kB/s | 47 kB 00:00 2023-09-30T22:24:34.424 INFO:teuthology.orchestra.run.smithi016.stdout:(63/122): libselinux-devel-2.9-8.el8.x86_64.rpm 938 kB/s | 200 kB 00:00 2023-09-30T22:24:34.445 INFO:teuthology.orchestra.run.smithi027.stdout:(50/122): python3-pyasn1-modules-0.3.7-6.el8.no 1.1 MB/s | 110 kB 00:00 2023-09-30T22:24:34.551 INFO:teuthology.orchestra.run.smithi027.stdout:(51/122): python3-pytz-2017.2-11.el8.noarch.rpm 522 kB/s | 57 kB 00:00 2023-09-30T22:24:34.551 INFO:teuthology.orchestra.run.smithi016.stdout:(64/122): libsepol-devel-2.9-3.el8.x86_64.rpm 1.0 MB/s | 87 kB 00:00 2023-09-30T22:24:34.551 INFO:teuthology.orchestra.run.smithi016.stdout:(65/122): libverto-devel-0.3.2-2.el8.x86_64.rpm 196 kB/s | 18 kB 00:00 2023-09-30T22:24:34.551 INFO:teuthology.orchestra.run.smithi016.stdout:(66/122): libzstd-devel-1.4.4-1.el8.x86_64.rpm 724 kB/s | 44 kB 00:00 2023-09-30T22:24:34.562 INFO:teuthology.orchestra.run.smithi027.stdout:(52/122): python3-werkzeug-0.12.2-4.el8.noarch. 3.8 MB/s | 457 kB 00:00 2023-09-30T22:24:34.621 INFO:teuthology.orchestra.run.smithi027.stdout:(53/122): socat-1.7.4.1-1.el8.x86_64.rpm 2.1 MB/s | 323 kB 00:00 2023-09-30T22:24:34.659 INFO:teuthology.orchestra.run.smithi016.stdout:(67/122): pcre2-devel-10.32-3.el8.x86_64.rpm 5.1 MB/s | 605 kB 00:00 2023-09-30T22:24:34.693 INFO:teuthology.orchestra.run.smithi027.stdout:(54/122): xmlstarlet-1.6.1-20.el8.x86_64.rpm 542 kB/s | 71 kB 00:00 2023-09-30T22:24:34.751 INFO:teuthology.orchestra.run.smithi016.stdout:(68/122): pcre2-utf16-10.32-3.el8.x86_64.rpm 2.4 MB/s | 229 kB 00:00 2023-09-30T22:24:34.809 INFO:teuthology.orchestra.run.smithi016.stdout:(69/122): pcre2-utf32-10.32-3.el8.x86_64.rpm 3.7 MB/s | 220 kB 00:00 2023-09-30T22:24:34.868 INFO:teuthology.orchestra.run.smithi016.stdout:(70/122): python3-cffi-1.11.5-6.el8.x86_64.rpm 4.2 MB/s | 249 kB 00:00 2023-09-30T22:24:34.889 INFO:teuthology.orchestra.run.smithi027.stdout:(55/122): elfutils-libelf-devel-0.189-3.el8.x86 231 kB/s | 62 kB 00:00 2023-09-30T22:24:34.914 INFO:teuthology.orchestra.run.smithi027.stdout:(56/122): fuse-2.9.7-17.el8.x86_64.rpm 377 kB/s | 83 kB 00:00 2023-09-30T22:24:34.939 INFO:teuthology.orchestra.run.smithi027.stdout:(57/122): keyutils-libs-devel-1.5.10-9.el8.x86_ 960 kB/s | 48 kB 00:00 2023-09-30T22:24:34.990 INFO:teuthology.orchestra.run.smithi027.stdout:(58/122): libcom_err-devel-1.45.6-5.el8.x86_64. 774 kB/s | 39 kB 00:00 2023-09-30T22:24:35.030 INFO:teuthology.orchestra.run.smithi016.stdout:(71/122): openssl-devel-1.1.1k-9.el8.x86_64.rpm 6.2 MB/s | 3.2 MB 00:00 2023-09-30T22:24:35.048 INFO:teuthology.orchestra.run.smithi027.stdout:(59/122): krb5-devel-1.18.2-22.el8.x86_64.rpm 4.1 MB/s | 561 kB 00:00 2023-09-30T22:24:35.069 INFO:teuthology.orchestra.run.smithi016.stdout:(72/122): python3-cryptography-3.2.1-6.el8.x86_ 3.1 MB/s | 641 kB 00:00 2023-09-30T22:24:35.086 INFO:teuthology.orchestra.run.smithi016.stdout:(73/122): python3-jwt-1.6.1-2.el8.noarch.rpm 768 kB/s | 43 kB 00:00 2023-09-30T22:24:35.128 INFO:teuthology.orchestra.run.smithi016.stdout:(74/122): python3-oauthlib-2.1.0-1.el8.noarch.r 2.6 MB/s | 155 kB 00:00 2023-09-30T22:24:35.132 INFO:teuthology.orchestra.run.smithi027.stdout:(60/122): libgfortran-8.5.0-20.el8.x86_64.rpm 4.4 MB/s | 645 kB 00:00 2023-09-30T22:24:35.153 INFO:teuthology.orchestra.run.smithi016.stdout:(75/122): python3-pycparser-2.14-14.el8.noarch. 1.6 MB/s | 109 kB 00:00 2023-09-30T22:24:35.157 INFO:teuthology.orchestra.run.smithi027.stdout:(61/122): libquadmath-8.5.0-20.el8.x86_64.rpm 1.6 MB/s | 172 kB 00:00 2023-09-30T22:24:35.170 INFO:teuthology.orchestra.run.smithi016.stdout:(76/122): python3-requests-oauthlib-1.0.0-1.el8 1.0 MB/s | 43 kB 00:00 2023-09-30T22:24:35.182 INFO:teuthology.orchestra.run.smithi027.stdout:(62/122): librabbitmq-0.9.0-4.el8.x86_64.rpm 936 kB/s | 47 kB 00:00 2023-09-30T22:24:35.195 INFO:teuthology.orchestra.run.smithi016.stdout:(77/122): zlib-devel-1.2.11-21.el8.x86_64.rpm 1.4 MB/s | 58 kB 00:00 2023-09-30T22:24:35.216 INFO:teuthology.orchestra.run.smithi027.stdout:(63/122): libselinux-devel-2.9-8.el8.x86_64.rpm 3.3 MB/s | 200 kB 00:00 2023-09-30T22:24:35.241 INFO:teuthology.orchestra.run.smithi027.stdout:(64/122): libsepol-devel-2.9-3.el8.x86_64.rpm 1.4 MB/s | 87 kB 00:00 2023-09-30T22:24:35.267 INFO:teuthology.orchestra.run.smithi027.stdout:(65/122): libverto-devel-0.3.2-2.el8.x86_64.rpm 361 kB/s | 18 kB 00:00 2023-09-30T22:24:35.292 INFO:teuthology.orchestra.run.smithi027.stdout:(66/122): libzstd-devel-1.4.4-1.el8.x86_64.rpm 872 kB/s | 44 kB 00:00 2023-09-30T22:24:35.412 INFO:teuthology.orchestra.run.smithi016.stdout:(78/122): ceph-osd-16.2.14-44.geeb13ac9.el8.x86 1.2 MB/s | 18 MB 00:14 2023-09-30T22:24:35.509 INFO:teuthology.orchestra.run.smithi027.stdout:(67/122): python3-scipy-1.0.0-21.module_el8.5.0 12 MB/s | 14 MB 00:01 2023-09-30T22:24:35.559 INFO:teuthology.orchestra.run.smithi027.stdout:(68/122): pcre2-devel-10.32-3.el8.x86_64.rpm 2.2 MB/s | 605 kB 00:00 2023-09-30T22:24:35.618 INFO:teuthology.orchestra.run.smithi027.stdout:(69/122): pcre2-utf32-10.32-3.el8.x86_64.rpm 3.7 MB/s | 220 kB 00:00 2023-09-30T22:24:35.676 INFO:teuthology.orchestra.run.smithi027.stdout:(70/122): python3-cffi-1.11.5-6.el8.x86_64.rpm 4.2 MB/s | 249 kB 00:00 2023-09-30T22:24:35.721 INFO:teuthology.orchestra.run.smithi016.stdout:(79/122): leveldb-1.22-1.el8.x86_64.rpm 344 kB/s | 181 kB 00:00 2023-09-30T22:24:35.726 INFO:teuthology.orchestra.run.smithi027.stdout:(71/122): pcre2-utf16-10.32-3.el8.x86_64.rpm 1.0 MB/s | 229 kB 00:00 2023-09-30T22:24:35.746 INFO:teuthology.orchestra.run.smithi016.stdout:(80/122): liboath-2.6.2-3.el8.x86_64.rpm 176 kB/s | 59 kB 00:00 2023-09-30T22:24:35.777 INFO:teuthology.orchestra.run.smithi027.stdout:(72/122): python3-jwt-1.6.1-2.el8.noarch.rpm 854 kB/s | 43 kB 00:00 2023-09-30T22:24:35.788 INFO:teuthology.orchestra.run.smithi016.stdout:(81/122): gperftools-libs-2.7-9.el8.x86_64.rpm 495 kB/s | 306 kB 00:00 2023-09-30T22:24:35.814 INFO:teuthology.orchestra.run.smithi016.stdout:(82/122): libunwind-1.3.1-3.el8.x86_64.rpm 812 kB/s | 75 kB 00:00 2023-09-30T22:24:35.827 INFO:teuthology.orchestra.run.smithi027.stdout:(73/122): openssl-devel-1.1.1k-9.el8.x86_64.rpm 5.7 MB/s | 3.2 MB 00:00 2023-09-30T22:24:35.847 INFO:teuthology.orchestra.run.smithi016.stdout:(83/122): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 434 kB/s | 44 kB 00:00 2023-09-30T22:24:35.852 INFO:teuthology.orchestra.run.smithi027.stdout:(74/122): python3-oauthlib-2.1.0-1.el8.noarch.r 2.0 MB/s | 155 kB 00:00 2023-09-30T22:24:35.878 INFO:teuthology.orchestra.run.smithi027.stdout:(75/122): python3-cryptography-3.2.1-6.el8.x86_ 3.1 MB/s | 641 kB 00:00 2023-09-30T22:24:35.881 INFO:teuthology.orchestra.run.smithi016.stdout:(84/122): python3-beautifulsoup4-4.6.3-2.el8.1. 2.0 MB/s | 185 kB 00:00 2023-09-30T22:24:35.903 INFO:teuthology.orchestra.run.smithi027.stdout:(76/122): python3-pycparser-2.14-14.el8.noarch. 1.4 MB/s | 109 kB 00:00 2023-09-30T22:24:35.928 INFO:teuthology.orchestra.run.smithi027.stdout:(77/122): python3-requests-oauthlib-1.0.0-1.el8 564 kB/s | 43 kB 00:00 2023-09-30T22:24:35.941 INFO:teuthology.orchestra.run.smithi016.stdout:(85/122): python3-cachetools-3.1.1-4.el8.noarch 259 kB/s | 33 kB 00:00 2023-09-30T22:24:35.953 INFO:teuthology.orchestra.run.smithi027.stdout:(78/122): zlib-devel-1.2.11-21.el8.x86_64.rpm 770 kB/s | 58 kB 00:00 2023-09-30T22:24:35.973 INFO:teuthology.orchestra.run.smithi016.stdout:(86/122): python3-certifi-2018.10.15-7.el8.noar 130 kB/s | 16 kB 00:00 2023-09-30T22:24:35.998 INFO:teuthology.orchestra.run.smithi016.stdout:(87/122): python3-cheroot-8.5.2-1.el8.noarch.rp 1.4 MB/s | 173 kB 00:00 2023-09-30T22:24:36.073 INFO:teuthology.orchestra.run.smithi016.stdout:(88/122): python3-google-auth-1.1.1-10.el8.noar 811 kB/s | 81 kB 00:00 2023-09-30T22:24:36.099 INFO:teuthology.orchestra.run.smithi016.stdout:(89/122): python3-jaraco-6.2-6.el8.noarch.rpm 111 kB/s | 11 kB 00:00 2023-09-30T22:24:36.124 INFO:teuthology.orchestra.run.smithi016.stdout:(90/122): python3-cherrypy-18.4.0-1.el8.noarch. 2.1 MB/s | 384 kB 00:00 2023-09-30T22:24:36.162 INFO:teuthology.orchestra.run.smithi027.stdout:(79/122): liboath-2.6.2-3.el8.x86_64.rpm 282 kB/s | 59 kB 00:00 2023-09-30T22:24:36.166 INFO:teuthology.orchestra.run.smithi016.stdout:(91/122): python3-jaraco-functools-2.0-4.el8.no 190 kB/s | 18 kB 00:00 2023-09-30T22:24:36.196 INFO:teuthology.orchestra.run.smithi027.stdout:(80/122): leveldb-1.22-1.el8.x86_64.rpm 677 kB/s | 181 kB 00:00 2023-09-30T22:24:36.233 INFO:teuthology.orchestra.run.smithi016.stdout:(92/122): python3-logutils-0.3.5-11.el8.noarch. 448 kB/s | 49 kB 00:00 2023-09-30T22:24:36.238 INFO:teuthology.orchestra.run.smithi027.stdout:(81/122): gperftools-libs-2.7-9.el8.x86_64.rpm 915 kB/s | 306 kB 00:00 2023-09-30T22:24:36.258 INFO:teuthology.orchestra.run.smithi016.stdout:(93/122): python3-more-itertools-7.2.0-3.el8.no 639 kB/s | 59 kB 00:00 2023-09-30T22:24:36.280 INFO:teuthology.orchestra.run.smithi027.stdout:(82/122): libunwind-1.3.1-3.el8.x86_64.rpm 640 kB/s | 75 kB 00:00 2023-09-30T22:24:36.305 INFO:teuthology.orchestra.run.smithi027.stdout:(83/122): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 400 kB/s | 44 kB 00:00 2023-09-30T22:24:36.333 INFO:teuthology.orchestra.run.smithi016.stdout:(94/122): python3-pecan-1.3.2-9.el8.noarch.rpm 2.8 MB/s | 283 kB 00:00 2023-09-30T22:24:36.347 INFO:teuthology.orchestra.run.smithi027.stdout:(84/122): python3-beautifulsoup4-4.6.3-2.el8.1. 1.7 MB/s | 185 kB 00:00 2023-09-30T22:24:36.359 INFO:teuthology.orchestra.run.smithi016.stdout:(95/122): python3-portend-2.6-1.el8.noarch.rpm 155 kB/s | 16 kB 00:00 2023-09-30T22:24:36.372 INFO:teuthology.orchestra.run.smithi027.stdout:(85/122): python3-cachetools-3.1.1-4.el8.noarch 358 kB/s | 33 kB 00:00 2023-09-30T22:24:36.416 INFO:teuthology.orchestra.run.smithi027.stdout:(86/122): python3-certifi-2018.10.15-7.el8.noar 148 kB/s | 16 kB 00:00 2023-09-30T22:24:36.448 INFO:teuthology.orchestra.run.smithi027.stdout:(87/122): python3-cheroot-8.5.2-1.el8.noarch.rp 1.7 MB/s | 173 kB 00:00 2023-09-30T22:24:36.465 INFO:teuthology.orchestra.run.smithi016.stdout:(96/122): python3-kubernetes-11.0.0-6.el8.noarc 3.2 MB/s | 1.2 MB 00:00 2023-09-30T22:24:36.484 INFO:teuthology.orchestra.run.smithi016.stdout:(97/122): python3-repoze-lru-0.7-6.el8.noarch.r 223 kB/s | 34 kB 00:00 2023-09-30T22:24:36.507 INFO:teuthology.orchestra.run.smithi027.stdout:(88/122): python3-google-auth-1.1.1-10.el8.noar 898 kB/s | 81 kB 00:00 2023-09-30T22:24:36.510 INFO:teuthology.orchestra.run.smithi016.stdout:(98/122): python3-routes-2.4.1-12.el8.noarch.rp 1.3 MB/s | 196 kB 00:00 2023-09-30T22:24:36.532 INFO:teuthology.orchestra.run.smithi027.stdout:(89/122): python3-cherrypy-18.4.0-1.el8.noarch. 2.4 MB/s | 384 kB 00:00 2023-09-30T22:24:36.549 INFO:teuthology.orchestra.run.smithi027.stdout:(90/122): python3-jaraco-6.2-6.el8.noarch.rpm 110 kB/s | 11 kB 00:00 2023-09-30T22:24:36.560 INFO:teuthology.orchestra.run.smithi016.stdout:(99/122): python3-rsa-4.9-2.el8.noarch.rpm 677 kB/s | 64 kB 00:00 2023-09-30T22:24:36.577 INFO:teuthology.orchestra.run.smithi016.stdout:(100/122): python3-simplegeneric-0.8.1-17.el8.n 203 kB/s | 19 kB 00:00 2023-09-30T22:24:36.591 INFO:teuthology.orchestra.run.smithi027.stdout:(91/122): python3-jaraco-functools-2.0-4.el8.no 210 kB/s | 18 kB 00:00 2023-09-30T22:24:36.602 INFO:teuthology.orchestra.run.smithi016.stdout:(101/122): python3-singledispatch-3.4.0.3-18.el 259 kB/s | 24 kB 00:00 2023-09-30T22:24:36.632 INFO:teuthology.orchestra.run.smithi027.stdout:(92/122): python3-logutils-0.3.5-11.el8.noarch. 583 kB/s | 49 kB 00:00 2023-09-30T22:24:36.661 INFO:teuthology.orchestra.run.smithi016.stdout:(102/122): python3-tempora-1.14.1-5.el8.noarch. 291 kB/s | 29 kB 00:00 2023-09-30T22:24:36.674 INFO:teuthology.orchestra.run.smithi027.stdout:(93/122): python3-more-itertools-7.2.0-3.el8.no 701 kB/s | 59 kB 00:00 2023-09-30T22:24:36.686 INFO:teuthology.orchestra.run.smithi016.stdout:(103/122): python3-trustme-0.6.0-4.el8.noarch.r 253 kB/s | 27 kB 00:00 2023-09-30T22:24:36.733 INFO:teuthology.orchestra.run.smithi027.stdout:(94/122): python3-pecan-1.3.2-9.el8.noarch.rpm 2.8 MB/s | 283 kB 00:00 2023-09-30T22:24:36.735 INFO:teuthology.orchestra.run.smithi016.stdout:(104/122): python3-waitress-1.4.3-1.el8.noarch. 1.9 MB/s | 255 kB 00:00 2023-09-30T22:24:36.767 INFO:teuthology.orchestra.run.smithi027.stdout:(95/122): python3-kubernetes-11.0.0-6.el8.noarc 5.0 MB/s | 1.2 MB 00:00 2023-09-30T22:24:36.770 INFO:teuthology.orchestra.run.smithi016.stdout:(105/122): python3-webob-1.8.5-1.el8.1.noarch.r 2.3 MB/s | 251 kB 00:00 2023-09-30T22:24:36.792 INFO:teuthology.orchestra.run.smithi027.stdout:(96/122): python3-portend-2.6-1.el8.noarch.rpm 133 kB/s | 16 kB 00:00 2023-09-30T22:24:36.795 INFO:teuthology.orchestra.run.smithi016.stdout:(106/122): python3-websocket-client-0.56.0-5.el 563 kB/s | 61 kB 00:00 2023-09-30T22:24:36.817 INFO:teuthology.orchestra.run.smithi027.stdout:(97/122): python3-repoze-lru-0.7-6.el8.noarch.r 402 kB/s | 34 kB 00:00 2023-09-30T22:24:36.829 INFO:teuthology.orchestra.run.smithi016.stdout:(107/122): python3-webtest-2.0.33-1.el8.noarch. 915 kB/s | 85 kB 00:00 2023-09-30T22:24:36.871 INFO:teuthology.orchestra.run.smithi016.stdout:(108/122): python3-zc-lockfile-2.0-2.el8.noarch 228 kB/s | 23 kB 00:00 2023-09-30T22:24:36.876 INFO:teuthology.orchestra.run.smithi027.stdout:(98/122): python3-routes-2.4.1-12.el8.noarch.rp 1.8 MB/s | 196 kB 00:00 2023-09-30T22:24:36.893 INFO:teuthology.orchestra.run.smithi027.stdout:(99/122): python3-simplegeneric-0.8.1-17.el8.no 248 kB/s | 19 kB 00:00 2023-09-30T22:24:36.918 INFO:teuthology.orchestra.run.smithi027.stdout:(100/122): python3-rsa-4.9-2.el8.noarch.rpm 510 kB/s | 64 kB 00:00 2023-09-30T22:24:36.952 INFO:teuthology.orchestra.run.smithi027.stdout:(101/122): python3-singledispatch-3.4.0.3-18.el 317 kB/s | 24 kB 00:00 2023-09-30T22:24:36.977 INFO:teuthology.orchestra.run.smithi027.stdout:(102/122): python3-tempora-1.14.1-5.el8.noarch. 349 kB/s | 29 kB 00:00 2023-09-30T22:24:36.994 INFO:teuthology.orchestra.run.smithi027.stdout:(103/122): python3-trustme-0.6.0-4.el8.noarch.r 365 kB/s | 27 kB 00:00 2023-09-30T22:24:37.030 INFO:teuthology.orchestra.run.smithi016.stdout:(109/122): NetworkManager-initscripts-updown-1. 907 kB/s | 143 kB 00:00 2023-09-30T22:24:37.061 INFO:teuthology.orchestra.run.smithi027.stdout:(104/122): python3-waitress-1.4.3-1.el8.noarch. 2.3 MB/s | 255 kB 00:00 2023-09-30T22:24:37.089 INFO:teuthology.orchestra.run.smithi016.stdout:(110/122): elfutils-debuginfod-client-0.189-3.e 1.3 MB/s | 76 kB 00:00 2023-09-30T22:24:37.103 INFO:teuthology.orchestra.run.smithi027.stdout:(105/122): python3-websocket-client-0.56.0-5.el 564 kB/s | 61 kB 00:00 2023-09-30T22:24:37.137 INFO:teuthology.orchestra.run.smithi027.stdout:(106/122): python3-webob-1.8.5-1.el8.1.noarch.r 1.5 MB/s | 251 kB 00:00 2023-09-30T22:24:37.162 INFO:teuthology.orchestra.run.smithi027.stdout:(107/122): python3-webtest-2.0.33-1.el8.noarch. 850 kB/s | 85 kB 00:00 2023-09-30T22:24:37.164 INFO:teuthology.orchestra.run.smithi016.stdout:(111/122): elfutils-libelf-0.189-3.el8.x86_64.r 3.0 MB/s | 232 kB 00:00 2023-09-30T22:24:37.187 INFO:teuthology.orchestra.run.smithi027.stdout:(108/122): python3-zc-lockfile-2.0-2.el8.noarch 274 kB/s | 23 kB 00:00 2023-09-30T22:24:37.231 INFO:teuthology.orchestra.run.smithi016.stdout:(112/122): librbd1-16.2.14-44.geeb13ac9.el8.x86 10 MB/s | 4.2 MB 00:00 2023-09-30T22:24:37.289 INFO:teuthology.orchestra.run.smithi016.stdout:(113/122): elfutils-libs-0.189-3.el8.x86_64.rpm 2.4 MB/s | 303 kB 00:00 2023-09-30T22:24:37.329 INFO:teuthology.orchestra.run.smithi027.stdout:(109/122): NetworkManager-initscripts-updown-1. 1.0 MB/s | 143 kB 00:00 2023-09-30T22:24:37.348 INFO:teuthology.orchestra.run.smithi016.stdout:(114/122): libblkid-2.32.1-43.el8.x86_64.rpm 3.7 MB/s | 220 kB 00:00 2023-09-30T22:24:37.380 INFO:teuthology.orchestra.run.smithi027.stdout:(110/122): elfutils-debuginfod-client-0.189-3.e 1.5 MB/s | 76 kB 00:00 2023-09-30T22:24:37.432 INFO:teuthology.orchestra.run.smithi016.stdout:(115/122): libfdisk-2.32.1-43.el8.x86_64.rpm 3.0 MB/s | 254 kB 00:00 2023-09-30T22:24:37.455 INFO:teuthology.orchestra.run.smithi027.stdout:(111/122): elfutils-libelf-0.189-3.el8.x86_64.r 3.0 MB/s | 232 kB 00:00 2023-09-30T22:24:37.490 INFO:teuthology.orchestra.run.smithi016.stdout:(116/122): iproute-6.2.0-2.el8.x86_64.rpm 3.3 MB/s | 880 kB 00:00 2023-09-30T22:24:37.538 INFO:teuthology.orchestra.run.smithi016.stdout:(117/122): libmount-2.32.1-43.el8.x86_64.rpm 2.2 MB/s | 236 kB 00:00 2023-09-30T22:24:37.547 INFO:teuthology.orchestra.run.smithi027.stdout:(112/122): elfutils-libs-0.189-3.el8.x86_64.rpm 3.2 MB/s | 303 kB 00:00 2023-09-30T22:24:37.574 INFO:teuthology.orchestra.run.smithi016.stdout:(118/122): libsmartcols-2.32.1-43.el8.x86_64.rp 2.1 MB/s | 179 kB 00:00 2023-09-30T22:24:37.600 INFO:teuthology.orchestra.run.smithi016.stdout:(119/122): libuuid-2.32.1-43.el8.x86_64.rpm 1.6 MB/s | 99 kB 00:00 2023-09-30T22:24:37.642 INFO:teuthology.orchestra.run.smithi016.stdout:(120/122): util-linux-user-2.32.1-43.el8.x86_64 2.4 MB/s | 103 kB 00:00 2023-09-30T22:24:37.681 INFO:teuthology.orchestra.run.smithi027.stdout:(113/122): librados2-16.2.14-44.geeb13ac9.el8.x 6.9 MB/s | 3.7 MB 00:00 2023-09-30T22:24:37.739 INFO:teuthology.orchestra.run.smithi027.stdout:(114/122): iproute-6.2.0-2.el8.x86_64.rpm 4.5 MB/s | 880 kB 00:00 2023-09-30T22:24:37.798 INFO:teuthology.orchestra.run.smithi027.stdout:(115/122): libfdisk-2.32.1-43.el8.x86_64.rpm 4.3 MB/s | 254 kB 00:00 2023-09-30T22:24:37.848 INFO:teuthology.orchestra.run.smithi027.stdout:(116/122): libblkid-2.32.1-43.el8.x86_64.rpm 1.3 MB/s | 220 kB 00:00 2023-09-30T22:24:37.873 INFO:teuthology.orchestra.run.smithi027.stdout:(117/122): libmount-2.32.1-43.el8.x86_64.rpm 3.1 MB/s | 236 kB 00:00 2023-09-30T22:24:37.899 INFO:teuthology.orchestra.run.smithi027.stdout:(118/122): libsmartcols-2.32.1-43.el8.x86_64.rp 3.5 MB/s | 179 kB 00:00 2023-09-30T22:24:37.909 INFO:teuthology.orchestra.run.smithi016.stdout:(121/122): util-linux-2.32.1-43.el8.x86_64.rpm 7.4 MB/s | 2.5 MB 00:00 2023-09-30T22:24:37.957 INFO:teuthology.orchestra.run.smithi027.stdout:(119/122): librbd1-16.2.14-44.geeb13ac9.el8.x86 5.3 MB/s | 4.2 MB 00:00 2023-09-30T22:24:37.982 INFO:teuthology.orchestra.run.smithi027.stdout:(120/122): libuuid-2.32.1-43.el8.x86_64.rpm 909 kB/s | 99 kB 00:00 2023-09-30T22:24:38.034 INFO:teuthology.orchestra.run.smithi016.stdout:(122/122): librados2-16.2.14-44.geeb13ac9.el8.x 3.0 MB/s | 3.7 MB 00:01 2023-09-30T22:24:38.037 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2023-09-30T22:24:38.037 INFO:teuthology.orchestra.run.smithi016.stdout:Total 9.3 MB/s | 200 MB 00:21 2023-09-30T22:24:38.091 INFO:teuthology.orchestra.run.smithi027.stdout:(121/122): util-linux-user-2.32.1-43.el8.x86_64 769 kB/s | 103 kB 00:00 2023-09-30T22:24:38.275 INFO:teuthology.orchestra.run.smithi027.stdout:(122/122): util-linux-2.32.1-43.el8.x86_64.rpm 6.6 MB/s | 2.5 MB 00:00 2023-09-30T22:24:38.277 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2023-09-30T22:24:38.278 INFO:teuthology.orchestra.run.smithi027.stdout:Total 9.1 MB/s | 200 MB 00:22 2023-09-30T22:24:38.792 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2023-09-30T22:24:39.020 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2023-09-30T22:24:39.047 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2023-09-30T22:24:39.047 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2023-09-30T22:24:39.262 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2023-09-30T22:24:39.262 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2023-09-30T22:24:40.538 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2023-09-30T22:24:40.541 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2023-09-30T22:24:40.645 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2023-09-30T22:24:40.648 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2023-09-30T22:24:45.455 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2023-09-30T22:24:45.537 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: liboath-2.6.2-3.el8.x86_64 1/1 2023-09-30T22:24:45.586 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2023-09-30T22:24:45.622 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: liboath-2.6.2-3.el8.x86_64 1/1 2023-09-30T22:24:45.742 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/136 2023-09-30T22:24:45.848 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/136 2023-09-30T22:24:45.912 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : leveldb-1.22-1.el8.x86_64 2/136 2023-09-30T22:24:46.041 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 3/136 2023-09-30T22:24:46.062 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : leveldb-1.22-1.el8.x86_64 2/136 2023-09-30T22:24:46.144 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 3/136 2023-09-30T22:24:46.246 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 3/136 2023-09-30T22:24:46.316 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 4/136 2023-09-30T22:24:46.339 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 3/136 2023-09-30T22:24:46.404 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 4/136 2023-09-30T22:24:46.521 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 4/136 2023-09-30T22:24:46.621 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 5/136 2023-09-30T22:24:46.622 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 4/136 2023-09-30T22:24:46.830 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 6/136 2023-09-30T22:24:46.852 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 5/136 2023-09-30T22:24:47.008 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 7/136 2023-09-30T22:24:47.096 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 6/136 2023-09-30T22:24:47.288 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 7/136 2023-09-30T22:24:47.391 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 7/136 2023-09-30T22:24:47.599 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : librados2-2:16.2.14-44.geeb13ac9.el8.x86_64 8/136 2023-09-30T22:24:47.708 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 7/136 2023-09-30T22:24:47.921 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librados2-2:16.2.14-44.geeb13ac9.el8.x86_64 8/136 2023-09-30T22:24:48.032 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librados2-2:16.2.14-44.geeb13ac9.el8.x86_64 8/136 2023-09-30T22:24:48.191 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : librbd1-2:16.2.14-44.geeb13ac9.el8.x86_64 9/136 2023-09-30T22:24:48.306 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librbd1-2:16.2.14-44.geeb13ac9.el8.x86_64 9/136 2023-09-30T22:24:48.376 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-2:16.2.14-44.geeb13ac9.el8.x86_64 8/136 2023-09-30T22:24:48.474 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcephfs2-2:16.2.14-44.geeb13ac9.el8.x86_64 10/136 2023-09-30T22:24:48.546 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librbd1-2:16.2.14-44.geeb13ac9.el8.x86_64 9/136 2023-09-30T22:24:48.565 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libcephfs2-2:16.2.14-44.geeb13ac9.el8.x86_64 10/136 2023-09-30T22:24:48.714 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-2:16.2.14-44.geeb13ac9.el8.x86_64 9/136 2023-09-30T22:24:48.799 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rados-2:16.2.14-44.geeb13ac9.el8.x86_64 11/136 2023-09-30T22:24:48.921 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephfs2-2:16.2.14-44.geeb13ac9.el8.x86_64 10/136 2023-09-30T22:24:48.949 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 12/136 2023-09-30T22:24:49.036 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephfs2-2:16.2.14-44.geeb13ac9.el8.x86_64 10/136 2023-09-30T22:24:49.274 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rados-2:16.2.14-44.geeb13ac9.el8.x86_64 11/136 2023-09-30T22:24:49.349 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 12/136 2023-09-30T22:24:49.421 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 12/136 2023-09-30T22:24:49.474 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librgw2-2:16.2.14-44.geeb13ac9.el8.x86_64 13/136 2023-09-30T22:24:49.545 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librgw2-2:16.2.14-44.geeb13ac9.el8.x86_64 13/136 2023-09-30T22:24:49.773 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 14/136 2023-09-30T22:24:49.859 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 12/136 2023-09-30T22:24:49.948 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 15/136 2023-09-30T22:24:50.037 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librgw2-2:16.2.14-44.geeb13ac9.el8.x86_64 13/136 2023-09-30T22:24:50.107 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcephsqlite-2:16.2.14-44.geeb13ac9.el8.x86_64 16/136 2023-09-30T22:24:50.125 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librgw2-2:16.2.14-44.geeb13ac9.el8.x86_64 13/136 2023-09-30T22:24:50.212 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libcephsqlite-2:16.2.14-44.geeb13ac9.el8.x86_64 16/136 2023-09-30T22:24:50.340 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libradosstriper1-2:16.2.14-44.geeb13ac9.el8.x86_ 17/136 2023-09-30T22:24:50.380 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 14/136 2023-09-30T22:24:50.462 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libradosstriper1-2:16.2.14-44.geeb13ac9.el8.x86_ 17/136 2023-09-30T22:24:50.638 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 15/136 2023-09-30T22:24:50.655 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 18/136 2023-09-30T22:24:50.754 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephsqlite-2:16.2.14-44.geeb13ac9.el8.x86_64 16/136 2023-09-30T22:24:50.816 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : zlib-devel-1.2.11-21.el8.x86_64 19/136 2023-09-30T22:24:50.901 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephsqlite-2:16.2.14-44.geeb13ac9.el8.x86_64 16/136 2023-09-30T22:24:50.957 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 20/136 2023-09-30T22:24:51.029 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libradosstriper1-2:16.2.14-44.geeb13ac9.el8.x86_ 17/136 2023-09-30T22:24:51.076 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 20/136 2023-09-30T22:24:51.191 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libradosstriper1-2:16.2.14-44.geeb13ac9.el8.x86_ 17/136 2023-09-30T22:24:51.224 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 21/136 2023-09-30T22:24:51.411 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 18/136 2023-09-30T22:24:51.573 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : zlib-devel-1.2.11-21.el8.x86_64 19/136 2023-09-30T22:24:51.712 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 20/136 2023-09-30T22:24:51.849 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 20/136 2023-09-30T22:24:51.855 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 21/136 2023-09-30T22:24:52.012 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 21/136 2023-09-30T22:24:52.057 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : openblas-0.3.15-4.el8.x86_64 22/136 2023-09-30T22:24:52.656 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 21/136 2023-09-30T22:24:52.705 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 22/136 2023-09-30T22:24:52.848 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 23/136 2023-09-30T22:24:52.854 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openblas-0.3.15-4.el8.x86_64 22/136 2023-09-30T22:24:53.537 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 22/136 2023-09-30T22:24:53.580 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 23/136 2023-09-30T22:24:53.687 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 23/136 2023-09-30T22:24:53.897 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 24/136 2023-09-30T22:24:54.146 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 25/136 2023-09-30T22:24:54.363 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-ceph-common-2:16.2.14-44.geeb13ac9.el8.x 26/136 2023-09-30T22:24:54.501 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 23/136 2023-09-30T22:24:54.521 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-ceph-argparse-2:16.2.14-44.geeb13ac9.el8 27/136 2023-09-30T22:24:54.724 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cephfs-2:16.2.14-44.geeb13ac9.el8.x86_64 28/136 2023-09-30T22:24:55.332 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/136 2023-09-30T22:24:55.333 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 24/136 2023-09-30T22:24:55.333 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 25/136 2023-09-30T22:24:55.333 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ceph-common-2:16.2.14-44.geeb13ac9.el8.x 26/136 2023-09-30T22:24:55.512 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ceph-argparse-2:16.2.14-44.geeb13ac9.el8 27/136 2023-09-30T22:24:55.781 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cephfs-2:16.2.14-44.geeb13ac9.el8.x86_64 28/136 2023-09-30T22:24:55.995 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/136 2023-09-30T22:24:56.810 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/136 2023-09-30T22:24:57.112 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 30/136 2023-09-30T22:24:57.284 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 31/136 2023-09-30T22:24:57.544 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rsa-4.9-2.el8.noarch 32/136 2023-09-30T22:24:57.779 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rgw-2:16.2.14-44.geeb13ac9.el8.x86_64 33/136 2023-09-30T22:24:57.995 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rbd-2:16.2.14-44.geeb13ac9.el8.x86_64 34/136 2023-09-30T22:24:58.015 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/136 2023-09-30T22:24:58.243 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librados-devel-2:16.2.14-44.geeb13ac9.el8.x86_64 35/136 2023-09-30T22:24:58.371 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 30/136 2023-09-30T22:24:58.446 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 36/136 2023-09-30T22:24:58.570 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 31/136 2023-09-30T22:24:58.616 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : elfutils-debuginfod-client-0.189-3.el8.x86_64 37/136 2023-09-30T22:24:58.825 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rsa-4.9-2.el8.noarch 32/136 2023-09-30T22:24:58.864 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 38/136 2023-09-30T22:24:58.934 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 38/136 2023-09-30T22:24:59.111 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rgw-2:16.2.14-44.geeb13ac9.el8.x86_64 33/136 2023-09-30T22:24:59.114 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 39/136 2023-09-30T22:24:59.222 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 39/136 2023-09-30T22:24:59.301 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rbd-2:16.2.14-44.geeb13ac9.el8.x86_64 34/136 2023-09-30T22:24:59.381 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 40/136 2023-09-30T22:24:59.562 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librados-devel-2:16.2.14-44.geeb13ac9.el8.x86_64 35/136 2023-09-30T22:24:59.768 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 36/136 2023-09-30T22:24:59.944 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 40/136 2023-09-30T22:24:59.972 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : elfutils-debuginfod-client-0.189-3.el8.x86_64 37/136 2023-09-30T22:25:00.228 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 38/136 2023-09-30T22:25:00.248 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 41/136 2023-09-30T22:25:00.264 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 41/136 2023-09-30T22:25:00.356 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 38/136 2023-09-30T22:25:00.427 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 42/136 2023-09-30T22:25:00.702 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 43/136 2023-09-30T22:25:00.819 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 39/136 2023-09-30T22:25:00.943 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 39/136 2023-09-30T22:25:01.094 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 40/136 2023-09-30T22:25:01.322 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 44/136 2023-09-30T22:25:01.520 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 45/136 2023-09-30T22:25:01.677 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 46/136 2023-09-30T22:25:01.677 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 40/136 2023-09-30T22:25:01.885 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 47/136 2023-09-30T22:25:02.111 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 48/136 2023-09-30T22:25:02.112 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 41/136 2023-09-30T22:25:02.131 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 41/136 2023-09-30T22:25:02.293 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 49/136 2023-09-30T22:25:02.328 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 42/136 2023-09-30T22:25:02.487 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 50/136 2023-09-30T22:25:02.586 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 43/136 2023-09-30T22:25:02.631 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 51/136 2023-09-30T22:25:02.828 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 44/136 2023-09-30T22:25:02.832 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 52/136 2023-09-30T22:25:03.033 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 45/136 2023-09-30T22:25:03.034 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 53/136 2023-09-30T22:25:03.241 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 46/136 2023-09-30T22:25:03.283 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 54/136 2023-09-30T22:25:03.442 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 47/136 2023-09-30T22:25:03.477 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 55/136 2023-09-30T22:25:03.667 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 48/136 2023-09-30T22:25:03.691 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 56/136 2023-09-30T22:25:03.865 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 49/136 2023-09-30T22:25:03.868 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 57/136 2023-09-30T22:25:04.076 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 50/136 2023-09-30T22:25:04.091 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 58/136 2023-09-30T22:25:04.228 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 51/136 2023-09-30T22:25:04.307 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 59/136 2023-09-30T22:25:04.429 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 52/136 2023-09-30T22:25:04.675 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 53/136 2023-09-30T22:25:04.735 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 60/136 2023-09-30T22:25:04.965 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 54/136 2023-09-30T22:25:05.026 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 61/136 2023-09-30T22:25:05.209 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 62/136 2023-09-30T22:25:05.238 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 55/136 2023-09-30T22:25:05.429 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 56/136 2023-09-30T22:25:05.490 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 63/136 2023-09-30T22:25:05.625 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 57/136 2023-09-30T22:25:05.703 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 64/136 2023-09-30T22:25:05.856 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 58/136 2023-09-30T22:25:06.114 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 59/136 2023-09-30T22:25:06.520 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 60/136 2023-09-30T22:25:06.819 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 61/136 2023-09-30T22:25:06.843 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 65/136 2023-09-30T22:25:07.043 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 62/136 2023-09-30T22:25:07.064 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 66/136 2023-09-30T22:25:07.289 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 63/136 2023-09-30T22:25:07.326 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 67/136 2023-09-30T22:25:07.514 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 64/136 2023-09-30T22:25:07.548 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 68/136 2023-09-30T22:25:07.776 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 69/136 2023-09-30T22:25:08.002 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : pcre2-utf32-10.32-3.el8.x86_64 70/136 2023-09-30T22:25:08.304 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : pcre2-utf16-10.32-3.el8.x86_64 71/136 2023-09-30T22:25:08.527 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : pcre2-devel-10.32-3.el8.x86_64 72/136 2023-09-30T22:25:08.665 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 73/136 2023-09-30T22:25:08.729 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 65/136 2023-09-30T22:25:08.863 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libverto-devel-0.3.2-2.el8.x86_64 74/136 2023-09-30T22:25:08.988 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 66/136 2023-09-30T22:25:09.154 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 75/136 2023-09-30T22:25:09.248 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 67/136 2023-09-30T22:25:09.392 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libselinux-devel-2.9-8.el8.x86_64 76/136 2023-09-30T22:25:09.489 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 68/136 2023-09-30T22:25:09.527 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcom_err-devel-1.45.6-5.el8.x86_64 77/136 2023-09-30T22:25:09.743 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 78/136 2023-09-30T22:25:09.765 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 69/136 2023-09-30T22:25:09.940 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : krb5-devel-1.18.2-22.el8.x86_64 79/136 2023-09-30T22:25:09.941 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : pcre2-utf32-10.32-3.el8.x86_64 70/136 2023-09-30T22:25:10.124 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : fuse-2.9.7-17.el8.x86_64 80/136 2023-09-30T22:25:10.259 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : pcre2-utf16-10.32-3.el8.x86_64 71/136 2023-09-30T22:25:10.383 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 81/136 2023-09-30T22:25:10.449 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : pcre2-devel-10.32-3.el8.x86_64 72/136 2023-09-30T22:25:10.663 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 82/136 2023-09-30T22:25:10.687 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 73/136 2023-09-30T22:25:10.843 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 83/136 2023-09-30T22:25:10.895 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libverto-devel-0.3.2-2.el8.x86_64 74/136 2023-09-30T22:25:11.069 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 84/136 2023-09-30T22:25:11.225 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 75/136 2023-09-30T22:25:11.358 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-portend-2.6-1.el8.noarch 85/136 2023-09-30T22:25:11.506 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libselinux-devel-2.9-8.el8.x86_64 76/136 2023-09-30T22:25:11.649 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcom_err-devel-1.45.6-5.el8.x86_64 77/136 2023-09-30T22:25:11.790 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 86/136 2023-09-30T22:25:11.884 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 78/136 2023-09-30T22:25:11.963 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 87/136 2023-09-30T22:25:12.112 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : krb5-devel-1.18.2-22.el8.x86_64 79/136 2023-09-30T22:25:12.330 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : fuse-2.9.7-17.el8.x86_64 80/136 2023-09-30T22:25:12.611 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 81/136 2023-09-30T22:25:12.944 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 82/136 2023-09-30T22:25:13.123 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 83/136 2023-09-30T22:25:13.349 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 84/136 2023-09-30T22:25:13.673 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-portend-2.6-1.el8.noarch 85/136 2023-09-30T22:25:13.953 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 86/136 2023-09-30T22:25:14.127 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 87/136 2023-09-30T22:25:14.591 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 88/136 2023-09-30T22:25:14.805 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 88/136 2023-09-30T22:25:15.551 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 88/136 2023-09-30T22:25:15.854 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-base-2:16.2.14-44.geeb13ac9.el8.x86_64 89/136 2023-09-30T22:25:15.973 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-base-2:16.2.14-44.geeb13ac9.el8.x86_64 89/136 2023-09-30T22:25:16.112 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-selinux-2:16.2.14-44.geeb13ac9.el8.x86_64 90/136 2023-09-30T22:25:16.912 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 88/136 2023-09-30T22:25:17.159 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 88/136 2023-09-30T22:25:17.989 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 88/136 2023-09-30T22:25:18.352 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-base-2:16.2.14-44.geeb13ac9.el8.x86_64 89/136 2023-09-30T22:25:18.463 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-base-2:16.2.14-44.geeb13ac9.el8.x86_64 89/136 2023-09-30T22:25:18.658 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-selinux-2:16.2.14-44.geeb13ac9.el8.x86_64 90/136 2023-09-30T22:25:35.416 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-selinux-2:16.2.14-44.geeb13ac9.el8.x86_64 90/136 2023-09-30T22:25:35.416 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /sys 2023-09-30T22:25:35.416 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /proc 2023-09-30T22:25:35.417 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /dev 2023-09-30T22:25:35.417 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /run 2023-09-30T22:25:35.417 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /mnt 2023-09-30T22:25:35.417 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /var/tmp 2023-09-30T22:25:35.417 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /home 2023-09-30T22:25:35.417 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /tmp 2023-09-30T22:25:35.417 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /dev 2023-09-30T22:25:35.417 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-09-30T22:25:35.841 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mds-2:16.2.14-44.geeb13ac9.el8.x86_64 91/136 2023-09-30T22:25:36.290 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mds-2:16.2.14-44.geeb13ac9.el8.x86_64 91/136 2023-09-30T22:25:36.485 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mon-2:16.2.14-44.geeb13ac9.el8.x86_64 92/136 2023-09-30T22:25:38.060 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mon-2:16.2.14-44.geeb13ac9.el8.x86_64 92/136 2023-09-30T22:25:38.243 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-osd-2:16.2.14-44.geeb13ac9.el8.x86_64 93/136 2023-09-30T22:25:38.279 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-osd-2:16.2.14-44.geeb13ac9.el8.x86_64 93/136 2023-09-30T22:25:38.515 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 94/136 2023-09-30T22:25:38.841 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 95/136 2023-09-30T22:25:39.225 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 96/136 2023-09-30T22:25:39.467 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 97/136 2023-09-30T22:25:39.705 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 98/136 2023-09-30T22:25:39.817 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-selinux-2:16.2.14-44.geeb13ac9.el8.x86_64 90/136 2023-09-30T22:25:39.817 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /sys 2023-09-30T22:25:39.817 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /proc 2023-09-30T22:25:39.817 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /dev 2023-09-30T22:25:39.817 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /run 2023-09-30T22:25:39.817 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /mnt 2023-09-30T22:25:39.817 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /var/tmp 2023-09-30T22:25:39.818 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /home 2023-09-30T22:25:39.818 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /tmp 2023-09-30T22:25:39.818 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /dev 2023-09-30T22:25:39.818 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:25:40.051 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 99/136 2023-09-30T22:25:40.130 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 99/136 2023-09-30T22:25:40.287 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : jq-1.6-7.el8.x86_64 100/136 2023-09-30T22:25:40.408 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mds-2:16.2.14-44.geeb13ac9.el8.x86_64 91/136 2023-09-30T22:25:40.459 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-prometheus-alerts-2:16.2.14-44.geeb13ac9.el 101/136 2023-09-30T22:25:40.878 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mds-2:16.2.14-44.geeb13ac9.el8.x86_64 91/136 2023-09-30T22:25:41.089 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mon-2:16.2.14-44.geeb13ac9.el8.x86_64 92/136 2023-09-30T22:25:41.541 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-grafana-dashboards-2:16.2.14-44.geeb13ac9.e 102/136 2023-09-30T22:25:41.817 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-dashboard-2:16.2.14-44.geeb13ac9.el8.no 103/136 2023-09-30T22:25:42.724 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mon-2:16.2.14-44.geeb13ac9.el8.x86_64 92/136 2023-09-30T22:25:42.919 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-dashboard-2:16.2.14-44.geeb13ac9.el8.no 103/136 2023-09-30T22:25:42.964 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-osd-2:16.2.14-44.geeb13ac9.el8.x86_64 93/136 2023-09-30T22:25:43.007 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-osd-2:16.2.14-44.geeb13ac9.el8.x86_64 93/136 2023-09-30T22:25:43.075 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-diskprediction-local-2:16.2.14-44.geeb1 104/136 2023-09-30T22:25:43.171 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:16.2.14-44.geeb1 104/136 2023-09-30T22:25:43.269 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 94/136 2023-09-30T22:25:43.583 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-modules-core-2:16.2.14-44.geeb13ac9.el8 105/136 2023-09-30T22:25:43.645 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 95/136 2023-09-30T22:25:43.767 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-2:16.2.14-44.geeb13ac9.el8.x86_64 106/136 2023-09-30T22:25:43.793 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-2:16.2.14-44.geeb13ac9.el8.x86_64 106/136 2023-09-30T22:25:43.959 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-rook-2:16.2.14-44.geeb13ac9.el8.noarch 107/136 2023-09-30T22:25:43.970 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-rook-2:16.2.14-44.geeb13ac9.el8.noarch 107/136 2023-09-30T22:25:44.105 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 96/136 2023-09-30T22:25:44.331 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 97/136 2023-09-30T22:25:44.544 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 98/136 2023-09-30T22:25:44.680 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 99/136 2023-09-30T22:25:44.766 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 99/136 2023-09-30T22:25:44.959 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : jq-1.6-7.el8.x86_64 100/136 2023-09-30T22:25:45.113 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-prometheus-alerts-2:16.2.14-44.geeb13ac9.el 101/136 2023-09-30T22:25:46.259 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-grafana-dashboards-2:16.2.14-44.geeb13ac9.e 102/136 2023-09-30T22:25:46.562 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-dashboard-2:16.2.14-44.geeb13ac9.el8.no 103/136 2023-09-30T22:25:47.717 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-dashboard-2:16.2.14-44.geeb13ac9.el8.no 103/136 2023-09-30T22:25:47.888 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-diskprediction-local-2:16.2.14-44.geeb1 104/136 2023-09-30T22:25:47.991 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:16.2.14-44.geeb1 104/136 2023-09-30T22:25:48.411 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-modules-core-2:16.2.14-44.geeb13ac9.el8 105/136 2023-09-30T22:25:48.638 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-2:16.2.14-44.geeb13ac9.el8.x86_64 106/136 2023-09-30T22:25:48.670 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-2:16.2.14-44.geeb13ac9.el8.x86_64 106/136 2023-09-30T22:25:48.865 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-2:16.2.14-44.geeb13ac9.el8.x86_64 108/136 2023-09-30T22:25:48.930 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-rook-2:16.2.14-44.geeb13ac9.el8.noarch 107/136 2023-09-30T22:25:48.944 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-rook-2:16.2.14-44.geeb13ac9.el8.noarch 107/136 2023-09-30T22:25:50.172 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-test-2:16.2.14-44.geeb13ac9.el8.x86_64 109/136 2023-09-30T22:25:50.433 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-radosgw-2:16.2.14-44.geeb13ac9.el8.x86_64 110/136 2023-09-30T22:25:50.857 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-radosgw-2:16.2.14-44.geeb13ac9.el8.x86_64 110/136 2023-09-30T22:25:51.192 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-mirror-2:16.2.14-44.geeb13ac9.el8.x86_64 111/136 2023-09-30T22:25:51.289 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: rbd-mirror-2:16.2.14-44.geeb13ac9.el8.x86_64 111/136 2023-09-30T22:25:52.181 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 112/136 2023-09-30T22:25:52.587 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : openssl-devel-1:1.1.1k-9.el8.x86_64 113/136 2023-09-30T22:25:52.759 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 114/136 2023-09-30T22:25:52.928 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 115/136 2023-09-30T22:25:53.108 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcephfs-devel-2:16.2.14-44.geeb13ac9.el8.x86_6 116/136 2023-09-30T22:25:53.291 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 117/136 2023-09-30T22:25:53.622 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-nbd-2:16.2.14-44.geeb13ac9.el8.x86_64 118/136 2023-09-30T22:25:53.885 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : iproute-6.2.0-2.el8.x86_64 119/136 2023-09-30T22:25:54.020 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-2:16.2.14-44.geeb13ac9.el8.x86_64 108/136 2023-09-30T22:25:54.024 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : NetworkManager-initscripts-updown-1:1.40.16-9.el 120/136 2023-09-30T22:25:54.093 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-9.el 120/136 2023-09-30T22:25:54.324 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flex-2.6.1-9.el8.x86_64 121/136 2023-09-30T22:25:54.417 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 121/136 2023-09-30T22:25:54.599 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : bison-3.0.4-10.el8.x86_64 122/136 2023-09-30T22:25:54.612 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 122/136 2023-09-30T22:25:54.792 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : elfutils-debuginfod-client-0.189-1.el8.x86_64 123/136 2023-09-30T22:25:54.917 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : elfutils-libs-0.189-1.el8.x86_64 124/136 2023-09-30T22:25:54.917 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 125/136 2023-09-30T22:25:55.126 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 125/136 2023-09-30T22:25:55.132 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-4.el 126/136 2023-09-30T22:25:55.270 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : NetworkManager-initscripts-updown-1:1.40.16-4.el 126/136 2023-09-30T22:25:55.270 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 127/136 2023-09-30T22:25:55.382 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-test-2:16.2.14-44.geeb13ac9.el8.x86_64 109/136 2023-09-30T22:25:55.498 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 127/136 2023-09-30T22:25:55.637 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-radosgw-2:16.2.14-44.geeb13ac9.el8.x86_64 110/136 2023-09-30T22:25:55.725 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : iproute-6.2.0-1.el8.x86_64 128/136 2023-09-30T22:25:55.907 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : util-linux-user-2.32.1-41.el8.x86_64 129/136 2023-09-30T22:25:56.074 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-radosgw-2:16.2.14-44.geeb13ac9.el8.x86_64 110/136 2023-09-30T22:25:56.151 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : util-linux-2.32.1-41.el8.x86_64 130/136 2023-09-30T22:25:56.151 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : libfdisk-2.32.1-41.el8.x86_64 131/136 2023-09-30T22:25:56.296 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-mirror-2:16.2.14-44.geeb13ac9.el8.x86_64 111/136 2023-09-30T22:25:56.368 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libfdisk-2.32.1-41.el8.x86_64 131/136 2023-09-30T22:25:56.369 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : libmount-2.32.1-41.el8.x86_64 132/136 2023-09-30T22:25:56.401 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: rbd-mirror-2:16.2.14-44.geeb13ac9.el8.x86_64 111/136 2023-09-30T22:25:56.585 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libmount-2.32.1-41.el8.x86_64 132/136 2023-09-30T22:25:56.585 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : libblkid-2.32.1-41.el8.x86_64 133/136 2023-09-30T22:25:56.769 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libblkid-2.32.1-41.el8.x86_64 133/136 2023-09-30T22:25:56.769 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : libuuid-2.32.1-41.el8.x86_64 134/136 2023-09-30T22:25:56.985 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libuuid-2.32.1-41.el8.x86_64 134/136 2023-09-30T22:25:56.985 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : libsmartcols-2.32.1-41.el8.x86_64 135/136 2023-09-30T22:25:57.161 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libsmartcols-2.32.1-41.el8.x86_64 135/136 2023-09-30T22:25:57.334 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : elfutils-libelf-0.189-1.el8.x86_64 136/136 2023-09-30T22:25:57.376 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 112/136 2023-09-30T22:25:57.668 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openssl-devel-1:1.1.1k-9.el8.x86_64 113/136 2023-09-30T22:25:57.900 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 114/136 2023-09-30T22:25:57.944 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: elfutils-libelf-0.189-1.el8.x86_64 136/136 2023-09-30T22:25:57.944 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-2:16.2.14-44.geeb13ac9.el8.x86_64 1/136 2023-09-30T22:25:57.944 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-base-2:16.2.14-44.geeb13ac9.el8.x86_64 2/136 2023-09-30T22:25:57.944 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 3/136 2023-09-30T22:25:57.944 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 4/136 2023-09-30T22:25:57.944 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mds-2:16.2.14-44.geeb13ac9.el8.x86_64 5/136 2023-09-30T22:25:57.944 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-2:16.2.14-44.geeb13ac9.el8.x86_64 6/136 2023-09-30T22:25:57.944 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mon-2:16.2.14-44.geeb13ac9.el8.x86_64 7/136 2023-09-30T22:25:57.944 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-osd-2:16.2.14-44.geeb13ac9.el8.x86_64 8/136 2023-09-30T22:25:57.945 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-radosgw-2:16.2.14-44.geeb13ac9.el8.x86_64 9/136 2023-09-30T22:25:57.945 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-selinux-2:16.2.14-44.geeb13ac9.el8.x86_64 10/136 2023-09-30T22:25:57.945 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-test-2:16.2.14-44.geeb13ac9.el8.x86_64 11/136 2023-09-30T22:25:57.945 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcephfs-devel-2:16.2.14-44.geeb13ac9.el8.x86_6 12/136 2023-09-30T22:25:57.945 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcephfs2-2:16.2.14-44.geeb13ac9.el8.x86_64 13/136 2023-09-30T22:25:57.945 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcephsqlite-2:16.2.14-44.geeb13ac9.el8.x86_64 14/136 2023-09-30T22:25:57.945 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados-devel-2:16.2.14-44.geeb13ac9.el8.x86_64 15/136 2023-09-30T22:25:57.945 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libradosstriper1-2:16.2.14-44.geeb13ac9.el8.x86_ 16/136 2023-09-30T22:25:57.945 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librgw2-2:16.2.14-44.geeb13ac9.el8.x86_64 17/136 2023-09-30T22:25:57.945 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-ceph-argparse-2:16.2.14-44.geeb13ac9.el8 18/136 2023-09-30T22:25:57.945 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-ceph-common-2:16.2.14-44.geeb13ac9.el8.x 19/136 2023-09-30T22:25:57.946 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cephfs-2:16.2.14-44.geeb13ac9.el8.x86_64 20/136 2023-09-30T22:25:57.946 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rados-2:16.2.14-44.geeb13ac9.el8.x86_64 21/136 2023-09-30T22:25:57.946 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rbd-2:16.2.14-44.geeb13ac9.el8.x86_64 22/136 2023-09-30T22:25:57.946 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rgw-2:16.2.14-44.geeb13ac9.el8.x86_64 23/136 2023-09-30T22:25:57.946 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 24/136 2023-09-30T22:25:57.946 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-mirror-2:16.2.14-44.geeb13ac9.el8.x86_64 25/136 2023-09-30T22:25:57.946 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-nbd-2:16.2.14-44.geeb13ac9.el8.x86_64 26/136 2023-09-30T22:25:57.946 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-grafana-dashboards-2:16.2.14-44.geeb13ac9.e 27/136 2023-09-30T22:25:57.946 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-dashboard-2:16.2.14-44.geeb13ac9.el8.no 28/136 2023-09-30T22:25:57.946 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-diskprediction-local-2:16.2.14-44.geeb1 29/136 2023-09-30T22:25:57.947 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-modules-core-2:16.2.14-44.geeb13ac9.el8 30/136 2023-09-30T22:25:57.947 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-rook-2:16.2.14-44.geeb13ac9.el8.noarch 31/136 2023-09-30T22:25:57.947 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-prometheus-alerts-2:16.2.14-44.geeb13ac9.el 32/136 2023-09-30T22:25:57.947 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : bison-3.0.4-10.el8.x86_64 33/136 2023-09-30T22:25:57.947 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flex-2.6.1-9.el8.x86_64 34/136 2023-09-30T22:25:57.947 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : jq-1.6-7.el8.x86_64 35/136 2023-09-30T22:25:57.947 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 36/136 2023-09-30T22:25:57.947 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 37/136 2023-09-30T22:25:57.947 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 38/136 2023-09-30T22:25:57.947 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 39/136 2023-09-30T22:25:57.947 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 40/136 2023-09-30T22:25:57.948 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 41/136 2023-09-30T22:25:57.948 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 42/136 2023-09-30T22:25:57.948 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 43/136 2023-09-30T22:25:57.948 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 44/136 2023-09-30T22:25:57.948 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 45/136 2023-09-30T22:25:57.948 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 46/136 2023-09-30T22:25:57.948 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 47/136 2023-09-30T22:25:57.948 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 48/136 2023-09-30T22:25:57.948 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 49/136 2023-09-30T22:25:57.948 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 50/136 2023-09-30T22:25:57.948 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 51/136 2023-09-30T22:25:57.948 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 52/136 2023-09-30T22:25:57.949 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 53/136 2023-09-30T22:25:57.949 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 54/136 2023-09-30T22:25:57.949 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 55/136 2023-09-30T22:25:57.949 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 56/136 2023-09-30T22:25:57.949 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 57/136 2023-09-30T22:25:57.949 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 58/136 2023-09-30T22:25:57.949 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : krb5-devel-1.18.2-22.el8.x86_64 59/136 2023-09-30T22:25:57.949 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcom_err-devel-1.45.6-5.el8.x86_64 60/136 2023-09-30T22:25:57.949 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 61/136 2023-09-30T22:25:57.949 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 62/136 2023-09-30T22:25:57.949 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 63/136 2023-09-30T22:25:57.949 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libselinux-devel-2.9-8.el8.x86_64 64/136 2023-09-30T22:25:57.950 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 65/136 2023-09-30T22:25:57.950 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libverto-devel-0.3.2-2.el8.x86_64 66/136 2023-09-30T22:25:57.950 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 67/136 2023-09-30T22:25:57.950 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-devel-1:1.1.1k-9.el8.x86_64 68/136 2023-09-30T22:25:57.950 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : pcre2-devel-10.32-3.el8.x86_64 69/136 2023-09-30T22:25:57.950 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : pcre2-utf16-10.32-3.el8.x86_64 70/136 2023-09-30T22:25:57.950 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : pcre2-utf32-10.32-3.el8.x86_64 71/136 2023-09-30T22:25:57.950 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 72/136 2023-09-30T22:25:57.950 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 73/136 2023-09-30T22:25:57.950 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/136 2023-09-30T22:25:57.950 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 75/136 2023-09-30T22:25:57.951 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 76/136 2023-09-30T22:25:57.951 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 77/136 2023-09-30T22:25:57.951 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : zlib-devel-1.2.11-21.el8.x86_64 78/136 2023-09-30T22:25:57.951 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 79/136 2023-09-30T22:25:57.951 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : leveldb-1.22-1.el8.x86_64 80/136 2023-09-30T22:25:57.951 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 81/136 2023-09-30T22:25:57.951 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 82/136 2023-09-30T22:25:57.951 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 83/136 2023-09-30T22:25:57.951 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 84/136 2023-09-30T22:25:57.951 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 85/136 2023-09-30T22:25:57.951 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 86/136 2023-09-30T22:25:57.952 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 87/136 2023-09-30T22:25:57.952 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 88/136 2023-09-30T22:25:57.952 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 89/136 2023-09-30T22:25:57.952 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 90/136 2023-09-30T22:25:57.952 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 91/136 2023-09-30T22:25:57.952 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 92/136 2023-09-30T22:25:57.952 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 93/136 2023-09-30T22:25:57.952 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 94/136 2023-09-30T22:25:57.952 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 95/136 2023-09-30T22:25:57.952 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-portend-2.6-1.el8.noarch 96/136 2023-09-30T22:25:57.953 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 97/136 2023-09-30T22:25:57.953 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 98/136 2023-09-30T22:25:57.953 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 99/136 2023-09-30T22:25:57.953 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 100/136 2023-09-30T22:25:57.953 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 101/136 2023-09-30T22:25:57.953 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 102/136 2023-09-30T22:25:57.953 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 103/136 2023-09-30T22:25:57.953 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 104/136 2023-09-30T22:25:57.953 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 105/136 2023-09-30T22:25:57.953 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 106/136 2023-09-30T22:25:57.953 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 107/136 2023-09-30T22:25:57.954 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 108/136 2023-09-30T22:25:57.954 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados2-2:16.2.14-44.geeb13ac9.el8.x86_64 109/136 2023-09-30T22:25:57.954 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 110/136 2023-09-30T22:25:57.954 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librbd1-2:16.2.14-44.geeb13ac9.el8.x86_64 111/136 2023-09-30T22:25:57.954 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 112/136 2023-09-30T22:25:57.954 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-9.el 113/136 2023-09-30T22:25:57.954 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-4.el 114/136 2023-09-30T22:25:57.954 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el8.x86_64 115/136 2023-09-30T22:25:57.954 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : elfutils-debuginfod-client-0.189-1.el8.x86_64 116/136 2023-09-30T22:25:57.954 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 117/136 2023-09-30T22:25:57.955 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : elfutils-libelf-0.189-1.el8.x86_64 118/136 2023-09-30T22:25:57.955 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 119/136 2023-09-30T22:25:57.955 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : elfutils-libs-0.189-1.el8.x86_64 120/136 2023-09-30T22:25:57.955 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : iproute-6.2.0-2.el8.x86_64 121/136 2023-09-30T22:25:57.955 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : iproute-6.2.0-1.el8.x86_64 122/136 2023-09-30T22:25:57.955 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 123/136 2023-09-30T22:25:57.955 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libblkid-2.32.1-41.el8.x86_64 124/136 2023-09-30T22:25:57.955 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 125/136 2023-09-30T22:25:57.955 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libfdisk-2.32.1-41.el8.x86_64 126/136 2023-09-30T22:25:57.955 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 127/136 2023-09-30T22:25:57.955 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libmount-2.32.1-41.el8.x86_64 128/136 2023-09-30T22:25:57.955 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 129/136 2023-09-30T22:25:57.956 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libsmartcols-2.32.1-41.el8.x86_64 130/136 2023-09-30T22:25:57.956 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 131/136 2023-09-30T22:25:57.956 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libuuid-2.32.1-41.el8.x86_64 132/136 2023-09-30T22:25:57.956 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 133/136 2023-09-30T22:25:57.956 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : util-linux-2.32.1-41.el8.x86_64 134/136 2023-09-30T22:25:57.956 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 135/136 2023-09-30T22:25:58.116 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 115/136 2023-09-30T22:25:58.329 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephfs-devel-2:16.2.14-44.geeb13ac9.el8.x86_6 116/136 2023-09-30T22:25:58.529 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 117/136 2023-09-30T22:25:58.872 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-nbd-2:16.2.14-44.geeb13ac9.el8.x86_64 118/136 2023-09-30T22:25:59.181 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : iproute-6.2.0-2.el8.x86_64 119/136 2023-09-30T22:25:59.320 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-initscripts-updown-1:1.40.16-9.el 120/136 2023-09-30T22:25:59.392 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-9.el 120/136 2023-09-30T22:25:59.670 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flex-2.6.1-9.el8.x86_64 121/136 2023-09-30T22:25:59.767 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 121/136 2023-09-30T22:25:59.978 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : bison-3.0.4-10.el8.x86_64 122/136 2023-09-30T22:25:59.993 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 122/136 2023-09-30T22:25:59.995 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : util-linux-user-2.32.1-41.el8.x86_64 136/136 2023-09-30T22:25:59.995 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-09-30T22:25:59.996 INFO:teuthology.orchestra.run.smithi027.stdout:Upgraded: 2023-09-30T22:25:59.996 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-initscripts-updown-1:1.40.16-9.el8.noarch 2023-09-30T22:25:59.996 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-debuginfod-client-0.189-3.el8.x86_64 2023-09-30T22:25:59.996 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-libelf-0.189-3.el8.x86_64 2023-09-30T22:25:59.996 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-libs-0.189-3.el8.x86_64 2023-09-30T22:25:59.996 INFO:teuthology.orchestra.run.smithi027.stdout: iproute-6.2.0-2.el8.x86_64 2023-09-30T22:25:59.996 INFO:teuthology.orchestra.run.smithi027.stdout: libblkid-2.32.1-43.el8.x86_64 2023-09-30T22:25:59.996 INFO:teuthology.orchestra.run.smithi027.stdout: libfdisk-2.32.1-43.el8.x86_64 2023-09-30T22:25:59.996 INFO:teuthology.orchestra.run.smithi027.stdout: libmount-2.32.1-43.el8.x86_64 2023-09-30T22:25:59.996 INFO:teuthology.orchestra.run.smithi027.stdout: librados2-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:25:59.996 INFO:teuthology.orchestra.run.smithi027.stdout: librbd1-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:25:59.996 INFO:teuthology.orchestra.run.smithi027.stdout: libsmartcols-2.32.1-43.el8.x86_64 2023-09-30T22:25:59.997 INFO:teuthology.orchestra.run.smithi027.stdout: libuuid-2.32.1-43.el8.x86_64 2023-09-30T22:25:59.997 INFO:teuthology.orchestra.run.smithi027.stdout: util-linux-2.32.1-43.el8.x86_64 2023-09-30T22:25:59.997 INFO:teuthology.orchestra.run.smithi027.stdout: util-linux-user-2.32.1-43.el8.x86_64 2023-09-30T22:25:59.997 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2023-09-30T22:25:59.997 INFO:teuthology.orchestra.run.smithi027.stdout: bison-3.0.4-10.el8.x86_64 2023-09-30T22:25:59.997 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:25:59.997 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-base-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:25:59.997 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:25:59.997 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:25:59.997 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-grafana-dashboards-2:16.2.14-44.geeb13ac9.el8.noarch 2023-09-30T22:25:59.997 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mds-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:25:59.998 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:25:59.998 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-dashboard-2:16.2.14-44.geeb13ac9.el8.noarch 2023-09-30T22:25:59.998 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-diskprediction-local-2:16.2.14-44.geeb13ac9.el8.noarch 2023-09-30T22:25:59.998 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-modules-core-2:16.2.14-44.geeb13ac9.el8.noarch 2023-09-30T22:25:59.998 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-rook-2:16.2.14-44.geeb13ac9.el8.noarch 2023-09-30T22:25:59.998 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mon-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:25:59.998 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-osd-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:25:59.998 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-prometheus-alerts-2:16.2.14-44.geeb13ac9.el8.noarch 2023-09-30T22:25:59.998 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-radosgw-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:25:59.998 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-selinux-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:25:59.998 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-test-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:25:59.998 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2023-09-30T22:25:59.999 INFO:teuthology.orchestra.run.smithi027.stdout: flex-2.6.1-9.el8.x86_64 2023-09-30T22:25:59.999 INFO:teuthology.orchestra.run.smithi027.stdout: fuse-2.9.7-17.el8.x86_64 2023-09-30T22:25:59.999 INFO:teuthology.orchestra.run.smithi027.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-09-30T22:25:59.999 INFO:teuthology.orchestra.run.smithi027.stdout: jq-1.6-7.el8.x86_64 2023-09-30T22:25:59.999 INFO:teuthology.orchestra.run.smithi027.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-09-30T22:25:59.999 INFO:teuthology.orchestra.run.smithi027.stdout: krb5-devel-1.18.2-22.el8.x86_64 2023-09-30T22:25:59.999 INFO:teuthology.orchestra.run.smithi027.stdout: leveldb-1.22-1.el8.x86_64 2023-09-30T22:25:59.999 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs-devel-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:25:59.999 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs2-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:25:59.999 INFO:teuthology.orchestra.run.smithi027.stdout: libcephsqlite-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:25:59.999 INFO:teuthology.orchestra.run.smithi027.stdout: libcom_err-devel-1.45.6-5.el8.x86_64 2023-09-30T22:26:00.000 INFO:teuthology.orchestra.run.smithi027.stdout: libgfortran-8.5.0-20.el8.x86_64 2023-09-30T22:26:00.000 INFO:teuthology.orchestra.run.smithi027.stdout: liboath-2.6.2-3.el8.x86_64 2023-09-30T22:26:00.000 INFO:teuthology.orchestra.run.smithi027.stdout: libquadmath-8.5.0-20.el8.x86_64 2023-09-30T22:26:00.000 INFO:teuthology.orchestra.run.smithi027.stdout: librabbitmq-0.9.0-4.el8.x86_64 2023-09-30T22:26:00.000 INFO:teuthology.orchestra.run.smithi027.stdout: librados-devel-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:26:00.000 INFO:teuthology.orchestra.run.smithi027.stdout: libradosstriper1-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:26:00.000 INFO:teuthology.orchestra.run.smithi027.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-09-30T22:26:00.000 INFO:teuthology.orchestra.run.smithi027.stdout: librgw2-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:26:00.000 INFO:teuthology.orchestra.run.smithi027.stdout: libselinux-devel-2.9-8.el8.x86_64 2023-09-30T22:26:00.000 INFO:teuthology.orchestra.run.smithi027.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-09-30T22:26:00.000 INFO:teuthology.orchestra.run.smithi027.stdout: libunwind-1.3.1-3.el8.x86_64 2023-09-30T22:26:00.000 INFO:teuthology.orchestra.run.smithi027.stdout: libverto-devel-0.3.2-2.el8.x86_64 2023-09-30T22:26:00.001 INFO:teuthology.orchestra.run.smithi027.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2023-09-30T22:26:00.001 INFO:teuthology.orchestra.run.smithi027.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-09-30T22:26:00.001 INFO:teuthology.orchestra.run.smithi027.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-09-30T22:26:00.001 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-0.3.15-4.el8.x86_64 2023-09-30T22:26:00.001 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-threads-0.3.15-4.el8.x86_64 2023-09-30T22:26:00.001 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-devel-1:1.1.1k-9.el8.x86_64 2023-09-30T22:26:00.001 INFO:teuthology.orchestra.run.smithi027.stdout: pcre2-devel-10.32-3.el8.x86_64 2023-09-30T22:26:00.001 INFO:teuthology.orchestra.run.smithi027.stdout: pcre2-utf16-10.32-3.el8.x86_64 2023-09-30T22:26:00.001 INFO:teuthology.orchestra.run.smithi027.stdout: pcre2-utf32-10.32-3.el8.x86_64 2023-09-30T22:26:00.001 INFO:teuthology.orchestra.run.smithi027.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-09-30T22:26:00.001 INFO:teuthology.orchestra.run.smithi027.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-09-30T22:26:00.002 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-09-30T22:26:00.002 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-argparse-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:26:00.002 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:26:00.002 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cephfs-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:26:00.002 INFO:teuthology.orchestra.run.smithi027.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-09-30T22:26:00.002 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cffi-1.11.5-6.el8.x86_64 2023-09-30T22:26:00.002 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-09-30T22:26:00.002 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-09-30T22:26:00.002 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2023-09-30T22:26:00.002 INFO:teuthology.orchestra.run.smithi027.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-09-30T22:26:00.002 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-6.2-6.el8.noarch 2023-09-30T22:26:00.002 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-09-30T22:26:00.003 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-09-30T22:26:00.003 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-09-30T22:26:00.003 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-09-30T22:26:00.003 INFO:teuthology.orchestra.run.smithi027.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-09-30T22:26:00.003 INFO:teuthology.orchestra.run.smithi027.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-09-30T22:26:00.003 INFO:teuthology.orchestra.run.smithi027.stdout: python3-mako-1.0.6-14.el8.noarch 2023-09-30T22:26:00.003 INFO:teuthology.orchestra.run.smithi027.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-09-30T22:26:00.003 INFO:teuthology.orchestra.run.smithi027.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-09-30T22:26:00.003 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-09-30T22:26:00.003 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-09-30T22:26:00.003 INFO:teuthology.orchestra.run.smithi027.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-09-30T22:26:00.004 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-09-30T22:26:00.004 INFO:teuthology.orchestra.run.smithi027.stdout: python3-portend-2.6-1.el8.noarch 2023-09-30T22:26:00.004 INFO:teuthology.orchestra.run.smithi027.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-09-30T22:26:00.004 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-09-30T22:26:00.004 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-09-30T22:26:00.004 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-09-30T22:26:00.004 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pycparser-2.14-14.el8.noarch 2023-09-30T22:26:00.004 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pytz-2017.2-11.el8.noarch 2023-09-30T22:26:00.004 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rados-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:26:00.004 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rbd-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:26:00.004 INFO:teuthology.orchestra.run.smithi027.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-09-30T22:26:00.005 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-09-30T22:26:00.005 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rgw-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:26:00.005 INFO:teuthology.orchestra.run.smithi027.stdout: python3-routes-2.4.1-12.el8.noarch 2023-09-30T22:26:00.005 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rsa-4.9-2.el8.noarch 2023-09-30T22:26:00.005 INFO:teuthology.orchestra.run.smithi027.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2023-09-30T22:26:00.005 INFO:teuthology.orchestra.run.smithi027.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-09-30T22:26:00.005 INFO:teuthology.orchestra.run.smithi027.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-09-30T22:26:00.005 INFO:teuthology.orchestra.run.smithi027.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-09-30T22:26:00.005 INFO:teuthology.orchestra.run.smithi027.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-09-30T22:26:00.005 INFO:teuthology.orchestra.run.smithi027.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-09-30T22:26:00.005 INFO:teuthology.orchestra.run.smithi027.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-09-30T22:26:00.005 INFO:teuthology.orchestra.run.smithi027.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-09-30T22:26:00.006 INFO:teuthology.orchestra.run.smithi027.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-09-30T22:26:00.006 INFO:teuthology.orchestra.run.smithi027.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-09-30T22:26:00.006 INFO:teuthology.orchestra.run.smithi027.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-09-30T22:26:00.006 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:26:00.006 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-mirror-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:26:00.006 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-nbd-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:26:00.006 INFO:teuthology.orchestra.run.smithi027.stdout: socat-1.7.4.1-1.el8.x86_64 2023-09-30T22:26:00.006 INFO:teuthology.orchestra.run.smithi027.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-09-30T22:26:00.007 INFO:teuthology.orchestra.run.smithi027.stdout: zlib-devel-1.2.11-21.el8.x86_64 2023-09-30T22:26:00.007 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-09-30T22:26:00.007 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2023-09-30T22:26:00.147 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : elfutils-debuginfod-client-0.189-1.el8.x86_64 123/136 2023-09-30T22:26:00.179 DEBUG:teuthology.parallel:result is None 2023-09-30T22:26:00.321 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : elfutils-libs-0.189-1.el8.x86_64 124/136 2023-09-30T22:26:00.322 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 125/136 2023-09-30T22:26:00.555 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 125/136 2023-09-30T22:26:00.561 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-4.el 126/136 2023-09-30T22:26:00.673 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-initscripts-updown-1:1.40.16-4.el 126/136 2023-09-30T22:26:00.674 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 127/136 2023-09-30T22:26:01.094 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 127/136 2023-09-30T22:26:01.239 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : iproute-6.2.0-1.el8.x86_64 128/136 2023-09-30T22:26:01.426 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : util-linux-user-2.32.1-41.el8.x86_64 129/136 2023-09-30T22:26:01.663 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : util-linux-2.32.1-41.el8.x86_64 130/136 2023-09-30T22:26:01.664 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : libfdisk-2.32.1-41.el8.x86_64 131/136 2023-09-30T22:26:01.906 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libfdisk-2.32.1-41.el8.x86_64 131/136 2023-09-30T22:26:01.906 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : libmount-2.32.1-41.el8.x86_64 132/136 2023-09-30T22:26:02.081 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libmount-2.32.1-41.el8.x86_64 132/136 2023-09-30T22:26:02.081 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : libblkid-2.32.1-41.el8.x86_64 133/136 2023-09-30T22:26:02.272 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libblkid-2.32.1-41.el8.x86_64 133/136 2023-09-30T22:26:02.273 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : libuuid-2.32.1-41.el8.x86_64 134/136 2023-09-30T22:26:02.481 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libuuid-2.32.1-41.el8.x86_64 134/136 2023-09-30T22:26:02.481 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : libsmartcols-2.32.1-41.el8.x86_64 135/136 2023-09-30T22:26:02.673 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libsmartcols-2.32.1-41.el8.x86_64 135/136 2023-09-30T22:26:02.855 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : elfutils-libelf-0.189-1.el8.x86_64 136/136 2023-09-30T22:26:03.493 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: elfutils-libelf-0.189-1.el8.x86_64 136/136 2023-09-30T22:26:03.493 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-2:16.2.14-44.geeb13ac9.el8.x86_64 1/136 2023-09-30T22:26:03.493 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-base-2:16.2.14-44.geeb13ac9.el8.x86_64 2/136 2023-09-30T22:26:03.493 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 3/136 2023-09-30T22:26:03.494 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 4/136 2023-09-30T22:26:03.494 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mds-2:16.2.14-44.geeb13ac9.el8.x86_64 5/136 2023-09-30T22:26:03.494 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-2:16.2.14-44.geeb13ac9.el8.x86_64 6/136 2023-09-30T22:26:03.494 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mon-2:16.2.14-44.geeb13ac9.el8.x86_64 7/136 2023-09-30T22:26:03.494 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-osd-2:16.2.14-44.geeb13ac9.el8.x86_64 8/136 2023-09-30T22:26:03.494 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-radosgw-2:16.2.14-44.geeb13ac9.el8.x86_64 9/136 2023-09-30T22:26:03.494 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-selinux-2:16.2.14-44.geeb13ac9.el8.x86_64 10/136 2023-09-30T22:26:03.494 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-test-2:16.2.14-44.geeb13ac9.el8.x86_64 11/136 2023-09-30T22:26:03.494 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs-devel-2:16.2.14-44.geeb13ac9.el8.x86_6 12/136 2023-09-30T22:26:03.494 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs2-2:16.2.14-44.geeb13ac9.el8.x86_64 13/136 2023-09-30T22:26:03.494 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephsqlite-2:16.2.14-44.geeb13ac9.el8.x86_64 14/136 2023-09-30T22:26:03.495 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados-devel-2:16.2.14-44.geeb13ac9.el8.x86_64 15/136 2023-09-30T22:26:03.495 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libradosstriper1-2:16.2.14-44.geeb13ac9.el8.x86_ 16/136 2023-09-30T22:26:03.495 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librgw2-2:16.2.14-44.geeb13ac9.el8.x86_64 17/136 2023-09-30T22:26:03.495 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-argparse-2:16.2.14-44.geeb13ac9.el8 18/136 2023-09-30T22:26:03.495 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-common-2:16.2.14-44.geeb13ac9.el8.x 19/136 2023-09-30T22:26:03.495 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cephfs-2:16.2.14-44.geeb13ac9.el8.x86_64 20/136 2023-09-30T22:26:03.495 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rados-2:16.2.14-44.geeb13ac9.el8.x86_64 21/136 2023-09-30T22:26:03.495 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rbd-2:16.2.14-44.geeb13ac9.el8.x86_64 22/136 2023-09-30T22:26:03.495 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rgw-2:16.2.14-44.geeb13ac9.el8.x86_64 23/136 2023-09-30T22:26:03.495 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 24/136 2023-09-30T22:26:03.495 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-mirror-2:16.2.14-44.geeb13ac9.el8.x86_64 25/136 2023-09-30T22:26:03.495 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-nbd-2:16.2.14-44.geeb13ac9.el8.x86_64 26/136 2023-09-30T22:26:03.496 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-grafana-dashboards-2:16.2.14-44.geeb13ac9.e 27/136 2023-09-30T22:26:03.496 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-dashboard-2:16.2.14-44.geeb13ac9.el8.no 28/136 2023-09-30T22:26:03.506 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-diskprediction-local-2:16.2.14-44.geeb1 29/136 2023-09-30T22:26:03.506 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-modules-core-2:16.2.14-44.geeb13ac9.el8 30/136 2023-09-30T22:26:03.506 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-rook-2:16.2.14-44.geeb13ac9.el8.noarch 31/136 2023-09-30T22:26:03.506 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-prometheus-alerts-2:16.2.14-44.geeb13ac9.el 32/136 2023-09-30T22:26:03.506 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : bison-3.0.4-10.el8.x86_64 33/136 2023-09-30T22:26:03.506 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flex-2.6.1-9.el8.x86_64 34/136 2023-09-30T22:26:03.506 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : jq-1.6-7.el8.x86_64 35/136 2023-09-30T22:26:03.507 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 36/136 2023-09-30T22:26:03.507 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 37/136 2023-09-30T22:26:03.507 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 38/136 2023-09-30T22:26:03.507 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 39/136 2023-09-30T22:26:03.507 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 40/136 2023-09-30T22:26:03.507 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 41/136 2023-09-30T22:26:03.507 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 42/136 2023-09-30T22:26:03.508 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 43/136 2023-09-30T22:26:03.508 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 44/136 2023-09-30T22:26:03.508 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 45/136 2023-09-30T22:26:03.508 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 46/136 2023-09-30T22:26:03.508 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 47/136 2023-09-30T22:26:03.508 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 48/136 2023-09-30T22:26:03.508 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 49/136 2023-09-30T22:26:03.508 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 50/136 2023-09-30T22:26:03.508 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 51/136 2023-09-30T22:26:03.508 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 52/136 2023-09-30T22:26:03.508 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 53/136 2023-09-30T22:26:03.508 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 54/136 2023-09-30T22:26:03.509 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 55/136 2023-09-30T22:26:03.509 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 56/136 2023-09-30T22:26:03.509 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 57/136 2023-09-30T22:26:03.509 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 58/136 2023-09-30T22:26:03.509 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : krb5-devel-1.18.2-22.el8.x86_64 59/136 2023-09-30T22:26:03.509 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcom_err-devel-1.45.6-5.el8.x86_64 60/136 2023-09-30T22:26:03.509 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 61/136 2023-09-30T22:26:03.509 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 62/136 2023-09-30T22:26:03.509 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 63/136 2023-09-30T22:26:03.509 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libselinux-devel-2.9-8.el8.x86_64 64/136 2023-09-30T22:26:03.509 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 65/136 2023-09-30T22:26:03.510 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libverto-devel-0.3.2-2.el8.x86_64 66/136 2023-09-30T22:26:03.510 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 67/136 2023-09-30T22:26:03.510 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-devel-1:1.1.1k-9.el8.x86_64 68/136 2023-09-30T22:26:03.510 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : pcre2-devel-10.32-3.el8.x86_64 69/136 2023-09-30T22:26:03.510 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : pcre2-utf16-10.32-3.el8.x86_64 70/136 2023-09-30T22:26:03.510 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : pcre2-utf32-10.32-3.el8.x86_64 71/136 2023-09-30T22:26:03.510 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 72/136 2023-09-30T22:26:03.510 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 73/136 2023-09-30T22:26:03.510 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/136 2023-09-30T22:26:03.510 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 75/136 2023-09-30T22:26:03.510 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 76/136 2023-09-30T22:26:03.511 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 77/136 2023-09-30T22:26:03.511 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : zlib-devel-1.2.11-21.el8.x86_64 78/136 2023-09-30T22:26:03.511 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 79/136 2023-09-30T22:26:03.511 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : leveldb-1.22-1.el8.x86_64 80/136 2023-09-30T22:26:03.511 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 81/136 2023-09-30T22:26:03.511 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 82/136 2023-09-30T22:26:03.511 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 83/136 2023-09-30T22:26:03.511 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 84/136 2023-09-30T22:26:03.511 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 85/136 2023-09-30T22:26:03.511 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 86/136 2023-09-30T22:26:03.511 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 87/136 2023-09-30T22:26:03.511 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 88/136 2023-09-30T22:26:03.512 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 89/136 2023-09-30T22:26:03.512 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 90/136 2023-09-30T22:26:03.512 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 91/136 2023-09-30T22:26:03.512 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 92/136 2023-09-30T22:26:03.512 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 93/136 2023-09-30T22:26:03.512 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 94/136 2023-09-30T22:26:03.512 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 95/136 2023-09-30T22:26:03.512 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-portend-2.6-1.el8.noarch 96/136 2023-09-30T22:26:03.512 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 97/136 2023-09-30T22:26:03.512 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 98/136 2023-09-30T22:26:03.512 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 99/136 2023-09-30T22:26:03.512 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 100/136 2023-09-30T22:26:03.513 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 101/136 2023-09-30T22:26:03.513 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 102/136 2023-09-30T22:26:03.513 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 103/136 2023-09-30T22:26:03.513 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 104/136 2023-09-30T22:26:03.513 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 105/136 2023-09-30T22:26:03.513 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 106/136 2023-09-30T22:26:03.513 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 107/136 2023-09-30T22:26:03.513 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 108/136 2023-09-30T22:26:03.513 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-2:16.2.14-44.geeb13ac9.el8.x86_64 109/136 2023-09-30T22:26:03.513 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 110/136 2023-09-30T22:26:03.513 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-2:16.2.14-44.geeb13ac9.el8.x86_64 111/136 2023-09-30T22:26:03.513 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 112/136 2023-09-30T22:26:03.514 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-9.el 113/136 2023-09-30T22:26:03.514 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-4.el 114/136 2023-09-30T22:26:03.514 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el8.x86_64 115/136 2023-09-30T22:26:03.514 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-debuginfod-client-0.189-1.el8.x86_64 116/136 2023-09-30T22:26:03.514 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 117/136 2023-09-30T22:26:03.514 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libelf-0.189-1.el8.x86_64 118/136 2023-09-30T22:26:03.514 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 119/136 2023-09-30T22:26:03.514 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libs-0.189-1.el8.x86_64 120/136 2023-09-30T22:26:03.514 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : iproute-6.2.0-2.el8.x86_64 121/136 2023-09-30T22:26:03.514 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : iproute-6.2.0-1.el8.x86_64 122/136 2023-09-30T22:26:03.514 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 123/136 2023-09-30T22:26:03.515 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libblkid-2.32.1-41.el8.x86_64 124/136 2023-09-30T22:26:03.515 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 125/136 2023-09-30T22:26:03.515 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libfdisk-2.32.1-41.el8.x86_64 126/136 2023-09-30T22:26:03.515 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 127/136 2023-09-30T22:26:03.515 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libmount-2.32.1-41.el8.x86_64 128/136 2023-09-30T22:26:03.515 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 129/136 2023-09-30T22:26:03.515 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libsmartcols-2.32.1-41.el8.x86_64 130/136 2023-09-30T22:26:03.515 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 131/136 2023-09-30T22:26:03.515 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libuuid-2.32.1-41.el8.x86_64 132/136 2023-09-30T22:26:03.515 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 133/136 2023-09-30T22:26:03.515 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : util-linux-2.32.1-41.el8.x86_64 134/136 2023-09-30T22:26:03.516 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 135/136 2023-09-30T22:26:05.624 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : util-linux-user-2.32.1-41.el8.x86_64 136/136 2023-09-30T22:26:05.624 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:26:05.624 INFO:teuthology.orchestra.run.smithi016.stdout:Upgraded: 2023-09-30T22:26:05.624 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-initscripts-updown-1:1.40.16-9.el8.noarch 2023-09-30T22:26:05.624 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-debuginfod-client-0.189-3.el8.x86_64 2023-09-30T22:26:05.625 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf-0.189-3.el8.x86_64 2023-09-30T22:26:05.625 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libs-0.189-3.el8.x86_64 2023-09-30T22:26:05.625 INFO:teuthology.orchestra.run.smithi016.stdout: iproute-6.2.0-2.el8.x86_64 2023-09-30T22:26:05.625 INFO:teuthology.orchestra.run.smithi016.stdout: libblkid-2.32.1-43.el8.x86_64 2023-09-30T22:26:05.625 INFO:teuthology.orchestra.run.smithi016.stdout: libfdisk-2.32.1-43.el8.x86_64 2023-09-30T22:26:05.625 INFO:teuthology.orchestra.run.smithi016.stdout: libmount-2.32.1-43.el8.x86_64 2023-09-30T22:26:05.625 INFO:teuthology.orchestra.run.smithi016.stdout: librados2-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:26:05.625 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:26:05.625 INFO:teuthology.orchestra.run.smithi016.stdout: libsmartcols-2.32.1-43.el8.x86_64 2023-09-30T22:26:05.625 INFO:teuthology.orchestra.run.smithi016.stdout: libuuid-2.32.1-43.el8.x86_64 2023-09-30T22:26:05.625 INFO:teuthology.orchestra.run.smithi016.stdout: util-linux-2.32.1-43.el8.x86_64 2023-09-30T22:26:05.626 INFO:teuthology.orchestra.run.smithi016.stdout: util-linux-user-2.32.1-43.el8.x86_64 2023-09-30T22:26:05.626 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2023-09-30T22:26:05.626 INFO:teuthology.orchestra.run.smithi016.stdout: bison-3.0.4-10.el8.x86_64 2023-09-30T22:26:05.626 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:26:05.626 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:26:05.626 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:26:05.626 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:26:05.626 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards-2:16.2.14-44.geeb13ac9.el8.noarch 2023-09-30T22:26:05.626 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:26:05.626 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:26:05.626 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard-2:16.2.14-44.geeb13ac9.el8.noarch 2023-09-30T22:26:05.627 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local-2:16.2.14-44.geeb13ac9.el8.noarch 2023-09-30T22:26:05.627 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core-2:16.2.14-44.geeb13ac9.el8.noarch 2023-09-30T22:26:05.627 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook-2:16.2.14-44.geeb13ac9.el8.noarch 2023-09-30T22:26:05.627 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:26:05.627 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:26:05.627 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts-2:16.2.14-44.geeb13ac9.el8.noarch 2023-09-30T22:26:05.627 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:26:05.627 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:26:05.627 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:26:05.627 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2023-09-30T22:26:05.627 INFO:teuthology.orchestra.run.smithi016.stdout: flex-2.6.1-9.el8.x86_64 2023-09-30T22:26:05.628 INFO:teuthology.orchestra.run.smithi016.stdout: fuse-2.9.7-17.el8.x86_64 2023-09-30T22:26:05.628 INFO:teuthology.orchestra.run.smithi016.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-09-30T22:26:05.628 INFO:teuthology.orchestra.run.smithi016.stdout: jq-1.6-7.el8.x86_64 2023-09-30T22:26:05.628 INFO:teuthology.orchestra.run.smithi016.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-09-30T22:26:05.628 INFO:teuthology.orchestra.run.smithi016.stdout: krb5-devel-1.18.2-22.el8.x86_64 2023-09-30T22:26:05.628 INFO:teuthology.orchestra.run.smithi016.stdout: leveldb-1.22-1.el8.x86_64 2023-09-30T22:26:05.628 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:26:05.628 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:26:05.628 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:26:05.628 INFO:teuthology.orchestra.run.smithi016.stdout: libcom_err-devel-1.45.6-5.el8.x86_64 2023-09-30T22:26:05.628 INFO:teuthology.orchestra.run.smithi016.stdout: libgfortran-8.5.0-20.el8.x86_64 2023-09-30T22:26:05.628 INFO:teuthology.orchestra.run.smithi016.stdout: liboath-2.6.2-3.el8.x86_64 2023-09-30T22:26:05.629 INFO:teuthology.orchestra.run.smithi016.stdout: libquadmath-8.5.0-20.el8.x86_64 2023-09-30T22:26:05.629 INFO:teuthology.orchestra.run.smithi016.stdout: librabbitmq-0.9.0-4.el8.x86_64 2023-09-30T22:26:05.629 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:26:05.629 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:26:05.629 INFO:teuthology.orchestra.run.smithi016.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-09-30T22:26:05.629 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:26:05.629 INFO:teuthology.orchestra.run.smithi016.stdout: libselinux-devel-2.9-8.el8.x86_64 2023-09-30T22:26:05.629 INFO:teuthology.orchestra.run.smithi016.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-09-30T22:26:05.629 INFO:teuthology.orchestra.run.smithi016.stdout: libunwind-1.3.1-3.el8.x86_64 2023-09-30T22:26:05.629 INFO:teuthology.orchestra.run.smithi016.stdout: libverto-devel-0.3.2-2.el8.x86_64 2023-09-30T22:26:05.629 INFO:teuthology.orchestra.run.smithi016.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2023-09-30T22:26:05.629 INFO:teuthology.orchestra.run.smithi016.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-09-30T22:26:05.630 INFO:teuthology.orchestra.run.smithi016.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-09-30T22:26:05.630 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-0.3.15-4.el8.x86_64 2023-09-30T22:26:05.630 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-threads-0.3.15-4.el8.x86_64 2023-09-30T22:26:05.630 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-devel-1:1.1.1k-9.el8.x86_64 2023-09-30T22:26:05.630 INFO:teuthology.orchestra.run.smithi016.stdout: pcre2-devel-10.32-3.el8.x86_64 2023-09-30T22:26:05.630 INFO:teuthology.orchestra.run.smithi016.stdout: pcre2-utf16-10.32-3.el8.x86_64 2023-09-30T22:26:05.630 INFO:teuthology.orchestra.run.smithi016.stdout: pcre2-utf32-10.32-3.el8.x86_64 2023-09-30T22:26:05.630 INFO:teuthology.orchestra.run.smithi016.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-09-30T22:26:05.630 INFO:teuthology.orchestra.run.smithi016.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-09-30T22:26:05.630 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-09-30T22:26:05.630 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:26:05.631 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:26:05.631 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:26:05.631 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-09-30T22:26:05.631 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cffi-1.11.5-6.el8.x86_64 2023-09-30T22:26:05.631 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-09-30T22:26:05.631 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-09-30T22:26:05.631 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2023-09-30T22:26:05.631 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-09-30T22:26:05.631 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-6.2-6.el8.noarch 2023-09-30T22:26:05.631 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-09-30T22:26:05.631 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-09-30T22:26:05.631 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-09-30T22:26:05.631 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-09-30T22:26:05.632 INFO:teuthology.orchestra.run.smithi016.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-09-30T22:26:05.632 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-09-30T22:26:05.632 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako-1.0.6-14.el8.noarch 2023-09-30T22:26:05.632 INFO:teuthology.orchestra.run.smithi016.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-09-30T22:26:05.632 INFO:teuthology.orchestra.run.smithi016.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-09-30T22:26:05.632 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-09-30T22:26:05.632 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-09-30T22:26:05.632 INFO:teuthology.orchestra.run.smithi016.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-09-30T22:26:05.632 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-09-30T22:26:05.632 INFO:teuthology.orchestra.run.smithi016.stdout: python3-portend-2.6-1.el8.noarch 2023-09-30T22:26:05.632 INFO:teuthology.orchestra.run.smithi016.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-09-30T22:26:05.632 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-09-30T22:26:05.633 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-09-30T22:26:05.633 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-09-30T22:26:05.633 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pycparser-2.14-14.el8.noarch 2023-09-30T22:26:05.633 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytz-2017.2-11.el8.noarch 2023-09-30T22:26:05.633 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:26:05.633 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:26:05.633 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-09-30T22:26:05.633 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-09-30T22:26:05.633 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:26:05.633 INFO:teuthology.orchestra.run.smithi016.stdout: python3-routes-2.4.1-12.el8.noarch 2023-09-30T22:26:05.633 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rsa-4.9-2.el8.noarch 2023-09-30T22:26:05.633 INFO:teuthology.orchestra.run.smithi016.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2023-09-30T22:26:05.634 INFO:teuthology.orchestra.run.smithi016.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-09-30T22:26:05.634 INFO:teuthology.orchestra.run.smithi016.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-09-30T22:26:05.634 INFO:teuthology.orchestra.run.smithi016.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-09-30T22:26:05.634 INFO:teuthology.orchestra.run.smithi016.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-09-30T22:26:05.634 INFO:teuthology.orchestra.run.smithi016.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-09-30T22:26:05.634 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-09-30T22:26:05.634 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-09-30T22:26:05.634 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-09-30T22:26:05.634 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-09-30T22:26:05.634 INFO:teuthology.orchestra.run.smithi016.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-09-30T22:26:05.634 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:26:05.634 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:26:05.635 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd-2:16.2.14-44.geeb13ac9.el8.x86_64 2023-09-30T22:26:05.635 INFO:teuthology.orchestra.run.smithi016.stdout: socat-1.7.4.1-1.el8.x86_64 2023-09-30T22:26:05.635 INFO:teuthology.orchestra.run.smithi016.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-09-30T22:26:05.635 INFO:teuthology.orchestra.run.smithi016.stdout: zlib-devel-1.2.11-21.el8.x86_64 2023-09-30T22:26:05.635 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:26:05.635 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-09-30T22:26:05.832 DEBUG:teuthology.parallel:result is None 2023-09-30T22:26:05.833 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-09-30T22:26:05.833 INFO:teuthology.packaging:ref: None 2023-09-30T22:26:05.833 INFO:teuthology.packaging:tag: None 2023-09-30T22:26:05.833 INFO:teuthology.packaging:branch: pacific 2023-09-30T22:26:05.833 INFO:teuthology.packaging:sha1: 3d6eb91756d99e822eeed7c6b783104a6ad86054 2023-09-30T22:26:05.833 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=pacific 2023-09-30T22:26:06.133 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-09-30T22:26:06.163 INFO:teuthology.orchestra.run.smithi016.stdout:16.2.14-44.geeb13ac9.el8 2023-09-30T22:26:06.164 INFO:teuthology.packaging:The installed version of ceph is 16.2.14-44.geeb13ac9.el8 2023-09-30T22:26:06.164 INFO:teuthology.task.install:The correct ceph version 16.2.14-44.geeb13ac9 is installed. 2023-09-30T22:26:06.164 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-09-30T22:26:06.165 INFO:teuthology.packaging:ref: None 2023-09-30T22:26:06.165 INFO:teuthology.packaging:tag: None 2023-09-30T22:26:06.165 INFO:teuthology.packaging:branch: pacific 2023-09-30T22:26:06.165 INFO:teuthology.packaging:sha1: 3d6eb91756d99e822eeed7c6b783104a6ad86054 2023-09-30T22:26:06.165 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=pacific 2023-09-30T22:26:06.456 DEBUG:teuthology.orchestra.run.smithi027:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-09-30T22:26:06.485 INFO:teuthology.orchestra.run.smithi027.stdout:16.2.14-44.geeb13ac9.el8 2023-09-30T22:26:06.485 INFO:teuthology.packaging:The installed version of ceph is 16.2.14-44.geeb13ac9.el8 2023-09-30T22:26:06.486 INFO:teuthology.task.install:The correct ceph version 16.2.14-44.geeb13ac9 is installed. 2023-09-30T22:26:06.486 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-09-30T22:26:06.486 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-09-30T22:26:06.486 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-09-30T22:26:06.527 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-09-30T22:26:06.527 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-09-30T22:26:06.563 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-09-30T22:26:06.564 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-09-30T22:26:06.564 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/daemon-helper 2023-09-30T22:26:06.603 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-09-30T22:26:06.679 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-09-30T22:26:06.855 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/daemon-helper 2023-09-30T22:26:06.889 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-09-30T22:26:06.962 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-09-30T22:26:06.962 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-09-30T22:26:06.963 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/adjust-ulimits 2023-09-30T22:26:06.997 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-09-30T22:26:07.073 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-09-30T22:26:07.073 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/adjust-ulimits 2023-09-30T22:26:07.104 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-09-30T22:26:07.175 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-09-30T22:26:07.175 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-09-30T22:26:07.175 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/stdin-killer 2023-09-30T22:26:07.209 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-09-30T22:26:07.285 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-09-30T22:26:07.286 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/stdin-killer 2023-09-30T22:26:07.316 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-09-30T22:26:07.387 INFO:teuthology.run_tasks:Running task print... 2023-09-30T22:26:07.396 INFO:teuthology.task.print:**** done installing pacific 2023-09-30T22:26:07.396 INFO:teuthology.run_tasks:Running task ceph... 2023-09-30T22:26:07.471 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-09-30T22:26:07.471 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 777 /var/log/ceph 2023-09-30T22:26:07.474 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 777 /var/log/ceph 2023-09-30T22:26:07.505 INFO:tasks.ceph:Disabling ceph logrotate... 2023-09-30T22:26:07.505 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /etc/logrotate.d/ceph 2023-09-30T22:26:07.547 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f -- /etc/logrotate.d/ceph 2023-09-30T22:26:07.579 INFO:tasks.ceph:Creating extra log directories... 2023-09-30T22:26:07.579 DEBUG:teuthology.orchestra.run.smithi016:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-09-30T22:26:07.623 DEBUG:teuthology.orchestra.run.smithi027:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-09-30T22:26:07.665 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-09-30T22:26:07.666 INFO:tasks.ceph:config {'conf': {'global': {'mon warn on pool no app': False, 'ms bind msgr2': False, 'bluestore warn on legacy statfs': False, 'bluestore warn on no per pool omap': False, 'mon pg warn min per osd': 0}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon warn on osd down out interval zero': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['overall HEALTH_', '\\(FS_', '\\(MDS_', '\\(OSD_', '\\(MON_DOWN\\)', '\\(CACHE_POOL_', '\\(POOL_', '\\(MGR_DOWN\\)', '\\(PG_', '\\(SMALLER_PGP_NUM\\)', 'Monitor daemon marked osd', 'Behind on trimming', 'Manager daemon', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'scrub mismatch', 'ScrubResult', 'wrongly marked', '\\(POOL_APP_NOT_ENABLED\\)', '\\(SLOW_OPS\\)', 'overall HEALTH_', '\\(MON_MSGR2_NOT_ENABLED\\)', 'slow request'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-09-30T22:26:07.666 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7408044', 'branch': 'fix-nfs-apply-err-reporting', 'description': 'fs:upgrade/upgraded_client/{bluestore-bitmap branch/pacific centos_8 clusters/{1-mds-1-client-micro} conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-install 1-mount/mount/fuse 2-upgrade 3-workload/stress_tests/{0-client-upgrade 1-tests/kernel_untar_build}}}', 'email': None, 'first_in_suite': False, 'job_id': '7408044', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 10}}], 'os_type': 'centos', 'os_version': '8.stream', 'overrides': {'admin_socket': {'branch': 'fix-nfs-apply-err-reporting'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'bluestore warn on legacy statfs': False, 'bluestore warn on no per pool omap': False, 'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon warn on osd down out interval zero': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'scrub mismatch', 'ScrubResult', 'wrongly marked', '\\(POOL_APP_NOT_ENABLED\\)', '\\(SLOW_OPS\\)', 'overall HEALTH_', '\\(MON_MSGR2_NOT_ENABLED\\)', 'slow request'], 'sha1': '3d6eb91756d99e822eeed7c6b783104a6ad86054'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'check-counter': {'counters': {'mds': ['mds.dir_split']}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '3d6eb91756d99e822eeed7c6b783104a6ad86054'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-62953', 'sha1': '6973b24e3d99fd494b753449d2ea12077599cbd6'}}, 'owner': 'scheduled_dparmar@teuthology', 'priority': 70, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'mds.a', 'osd.0', 'osd.1', 'osd.2', 'osd.3'], ['client.0']], 'seed': 2683, 'sha1': '3d6eb91756d99e822eeed7c6b783104a6ad86054', 'sleep_before_teardown': 0, 'suite': 'fs:upgrade', 'suite_branch': 'wip-62953', 'suite_path': '/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/dparmar18/ceph', 'suite_sha1': '6973b24e3d99fd494b753449d2ea12077599cbd6', 'targets': {'smithi016.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKk3kgw+DuBL1HteXloOxaFL1W90U3HWeJLYcbwgixcY6Xo/kjnjoqxZKByRRTDia3763Cwx7yT37uv+Zj3xRvw=', 'smithi027.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOLuHXzaNM+5O4CIcXkqdy7FJ6Fd0pbrWeXk9n3OuMUKk4LGzMdfrWpI9MHt6/S9S8/UPzCaNJKNF5P3SbAS+Sg='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'branch': 'pacific', 'exclude_packages': ['cephadm', 'ceph-mgr-cephadm', 'ceph-immutable-object-cache', 'python3-rados', 'python3-rgw', 'python3-rbd', 'python3-cephfs', 'ceph-volume'], 'extra_packages': ['python-rados', 'python-rgw', 'python-rbd', 'python-cephfs'], 'extra_system_packages': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux'], 'flavor': 'default', 'sha1': '3d6eb91756d99e822eeed7c6b783104a6ad86054'}}, {'print': '**** done installing pacific'}, {'ceph': {'conf': {'global': {'mon warn on pool no app': False, 'ms bind msgr2': False, 'bluestore warn on legacy statfs': False, 'bluestore warn on no per pool omap': False, 'mon pg warn min per osd': 0}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon warn on osd down out interval zero': False}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'log-ignorelist': ['overall HEALTH_', '\\(FS_', '\\(MDS_', '\\(OSD_', '\\(MON_DOWN\\)', '\\(CACHE_POOL_', '\\(POOL_', '\\(MGR_DOWN\\)', '\\(PG_', '\\(SMALLER_PGP_NUM\\)', 'Monitor daemon marked osd', 'Behind on trimming', 'Manager daemon', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'scrub mismatch', 'ScrubResult', 'wrongly marked', '\\(POOL_APP_NOT_ENABLED\\)', '\\(SLOW_OPS\\)', 'overall HEALTH_', '\\(MON_MSGR2_NOT_ENABLED\\)', 'slow request'], 'flavor': 'default', 'fs': 'xfs', 'sha1': '3d6eb91756d99e822eeed7c6b783104a6ad86054', 'cluster': 'ceph'}}, {'exec': {'osd.0': ['ceph osd set-require-min-compat-client pacific']}}, {'print': '**** done ceph'}, {'ceph-fuse': None}, {'mds_pre_upgrade': None}, {'print': '**** done mds pre-upgrade sequence'}, {'install.upgrade': {'mon.a': {'branch': 'reef'}}}, {'print': '**** done install.upgrade the host'}, {'ceph.restart': {'daemons': ['mon.*', 'mgr.*'], 'mon-health-to-clog': False, 'wait-for-healthy': False}}, {'ceph.healthy': None}, {'ceph.restart': {'daemons': ['osd.*'], 'wait-for-healthy': False, 'wait-for-osds-up': True}}, {'ceph.stop': ['mds.*']}, {'ceph.restart': {'daemons': ['mds.*'], 'wait-for-healthy': False, 'wait-for-osds-up': True}}, {'exec': {'mon.a': ['ceph osd dump -f json-pretty', 'ceph versions', 'for f in `ceph osd pool ls` ; do ceph osd pool set $f pg_autoscale_mode off ; done', 'ceph osd require-osd-release reef']}}, {'ceph.healthy': None}, {'print': '**** done ceph.restart'}, {'ceph-fuse': {'client.0': {'mounted': False}}}, {'print': '**** done unmount client.0'}, {'install.upgrade': {'client.0': None}}, {'print': '**** done install.upgrade on client.0'}, {'ceph-fuse': {'client.0': None}}, {'print': '**** done remount client.0'}, {'workunit': {'clients': {'all': ['kernel_untar_build.sh']}}}], 'teuthology': {'branch': 'pacific', 'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end", 'if not is_fuse() then reject() end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '54e62bcbac4e53d9685e08328b790d3b20d71cae', 'timestamp': '2023-09-30_21:53:45', 'tube': 'smithi', 'user': 'dparmar', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2830'} 2023-09-30T22:26:07.667 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-09-30T22:26:07.698 DEBUG:teuthology.orchestra.run.smithi027:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-09-30T22:26:07.718 DEBUG:teuthology.orchestra.run.smithi016:> sudo install -d -m0777 -- /var/run/ceph 2023-09-30T22:26:07.761 DEBUG:teuthology.orchestra.run.smithi027:> sudo install -d -m0777 -- /var/run/ceph 2023-09-30T22:26:07.793 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-09-30T22:26:07.793 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/scratch_devs of=/dev/stdout 2023-09-30T22:26:07.856 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-09-30T22:26:07.857 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_1 2023-09-30T22:26:07.921 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-09-30T22:26:07.921 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-30T22:26:07.921 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 6h/6d Inode: 389493 Links: 1 2023-09-30T22:26:07.921 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-30T22:26:07.921 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2023-09-30T22:26:07.921 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2023-09-30 22:26:03.104844465 +0000 2023-09-30T22:26:07.921 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2023-09-30 22:22:58.387583084 +0000 2023-09-30T22:26:07.921 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2023-09-30 22:22:58.387583084 +0000 2023-09-30T22:26:07.921 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: - 2023-09-30T22:26:07.922 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-09-30T22:26:07.995 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2023-09-30T22:26:07.995 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2023-09-30T22:26:07.995 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000114018 s, 4.5 MB/s 2023-09-30T22:26:07.996 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-09-30T22:26:08.062 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_2 2023-09-30T22:26:08.125 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-09-30T22:26:08.125 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-30T22:26:08.125 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 6h/6d Inode: 391770 Links: 1 2023-09-30T22:26:08.125 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-30T22:26:08.125 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2023-09-30T22:26:08.126 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2023-09-30 22:26:03.104844465 +0000 2023-09-30T22:26:08.126 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2023-09-30 22:22:58.987573632 +0000 2023-09-30T22:26:08.126 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2023-09-30 22:22:58.987573632 +0000 2023-09-30T22:26:08.126 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: - 2023-09-30T22:26:08.126 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-09-30T22:26:08.199 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2023-09-30T22:26:08.199 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2023-09-30T22:26:08.199 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000122549 s, 4.2 MB/s 2023-09-30T22:26:08.200 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-09-30T22:26:08.265 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_3 2023-09-30T22:26:08.328 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-09-30T22:26:08.329 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-30T22:26:08.329 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 6h/6d Inode: 382917 Links: 1 2023-09-30T22:26:08.329 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-30T22:26:08.329 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2023-09-30T22:26:08.329 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2023-09-30 22:26:03.105844449 +0000 2023-09-30T22:26:08.329 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2023-09-30 22:22:59.563564559 +0000 2023-09-30T22:26:08.329 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2023-09-30 22:22:59.563564559 +0000 2023-09-30T22:26:08.329 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: - 2023-09-30T22:26:08.330 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-09-30T22:26:08.402 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2023-09-30T22:26:08.402 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2023-09-30T22:26:08.402 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000119056 s, 4.3 MB/s 2023-09-30T22:26:08.403 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-09-30T22:26:08.468 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_4 2023-09-30T22:26:08.531 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-09-30T22:26:08.532 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-09-30T22:26:08.532 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 6h/6d Inode: 397347 Links: 1 2023-09-30T22:26:08.532 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-09-30T22:26:08.532 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2023-09-30T22:26:08.532 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2023-09-30 22:26:03.105844449 +0000 2023-09-30T22:26:08.532 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2023-09-30 22:23:00.180554839 +0000 2023-09-30T22:26:08.532 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2023-09-30 22:23:00.180554839 +0000 2023-09-30T22:26:08.533 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: - 2023-09-30T22:26:08.533 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-09-30T22:26:08.606 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2023-09-30T22:26:08.606 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2023-09-30T22:26:08.606 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.00012513 s, 4.1 MB/s 2023-09-30T22:26:08.607 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-09-30T22:26:08.672 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-09-30T22:26:08.672 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi016.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}} 2023-09-30T22:26:08.672 INFO:tasks.ceph:Generating config... 2023-09-30T22:26:08.673 INFO:tasks.ceph:[global] mon warn on pool no app = False 2023-09-30T22:26:08.673 INFO:tasks.ceph:[global] ms bind msgr2 = False 2023-09-30T22:26:08.673 INFO:tasks.ceph:[global] bluestore warn on legacy statfs = False 2023-09-30T22:26:08.673 INFO:tasks.ceph:[global] bluestore warn on no per pool omap = False 2023-09-30T22:26:08.673 INFO:tasks.ceph:[global] mon pg warn min per osd = 0 2023-09-30T22:26:08.673 INFO:tasks.ceph:[client] client mount timeout = 600 2023-09-30T22:26:08.674 INFO:tasks.ceph:[client] debug client = 20 2023-09-30T22:26:08.674 INFO:tasks.ceph:[client] debug ms = 1 2023-09-30T22:26:08.674 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-09-30T22:26:08.674 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-09-30T22:26:08.674 INFO:tasks.ceph:[mds] debug mds = 20 2023-09-30T22:26:08.674 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-09-30T22:26:08.674 INFO:tasks.ceph:[mds] debug ms = 1 2023-09-30T22:26:08.674 INFO:tasks.ceph:[mds] mds debug frag = True 2023-09-30T22:26:08.674 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-09-30T22:26:08.674 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-09-30T22:26:08.674 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-09-30T22:26:08.674 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-09-30T22:26:08.675 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-09-30T22:26:08.675 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-09-30T22:26:08.675 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-09-30T22:26:08.675 INFO:tasks.ceph:[mgr] debug ms = 1 2023-09-30T22:26:08.675 INFO:tasks.ceph:[mon] debug mon = 20 2023-09-30T22:26:08.675 INFO:tasks.ceph:[mon] debug ms = 1 2023-09-30T22:26:08.675 INFO:tasks.ceph:[mon] debug paxos = 20 2023-09-30T22:26:08.675 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-09-30T22:26:08.675 INFO:tasks.ceph:[mon] mon warn on osd down out interval zero = False 2023-09-30T22:26:08.675 INFO:tasks.ceph:[osd] bdev async discard = True 2023-09-30T22:26:08.675 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-09-30T22:26:08.675 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-09-30T22:26:08.676 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-09-30T22:26:08.676 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-09-30T22:26:08.676 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-09-30T22:26:08.676 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-09-30T22:26:08.676 INFO:tasks.ceph:[osd] debug ms = 1 2023-09-30T22:26:08.676 INFO:tasks.ceph:[osd] debug osd = 20 2023-09-30T22:26:08.676 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-09-30T22:26:08.676 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-09-30T22:26:08.676 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-09-30T22:26:08.676 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-09-30T22:26:08.676 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-09-30T22:26:08.676 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-09-30T22:26:08.676 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-09-30T22:26:08.677 INFO:tasks.ceph:Setting up mon.a... 2023-09-30T22:26:08.677 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-09-30T22:26:08.980 INFO:teuthology.orchestra.run.smithi016.stdout:creating /etc/ceph/ceph.keyring 2023-09-30T22:26:08.983 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-09-30T22:26:09.034 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-09-30T22:26:09.066 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.16'), ('mon.b', '[v2:172.21.15.16:3301,v1:172.21.15.16:6790]'), ('mon.c', '[v2:172.21.15.16:3302,v1:172.21.15.16:6791]')] 2023-09-30T22:26:09.067 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.16,[v2:172.21.15.16:3301,v1:172.21.15.16:6790],[v2:172.21.15.16:3302,v1:172.21.15.16:6791]', 'mon warn on pool no app': False, 'ms bind msgr2': False, 'bluestore warn on legacy statfs': False, 'bluestore warn on no per pool omap': False, 'mon pg warn min per osd': 0}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120, 'mon warn on osd down out interval zero': False}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-09-30T22:26:09.068 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-09-30T22:26:09.068 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-09-30T22:26:09.130 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.16 --addv b '[v2:172.21.15.16:3301,v1:172.21.15.16:6790]' --addv c '[v2:172.21.15.16:3302,v1:172.21.15.16:6791]' --print /home/ubuntu/cephtest/ceph.monmap 2023-09-30T22:26:09.223 INFO:teuthology.orchestra.run.smithi016.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-09-30T22:26:09.224 INFO:teuthology.orchestra.run.smithi016.stdout:monmaptool: generated fsid 7ede3e14-2bd0-4726-9437-6d13c90cd342 2023-09-30T22:26:09.224 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 0 2023-09-30T22:26:09.224 INFO:teuthology.orchestra.run.smithi016.stdout:fsid 7ede3e14-2bd0-4726-9437-6d13c90cd342 2023-09-30T22:26:09.224 INFO:teuthology.orchestra.run.smithi016.stdout:last_changed 2023-09-30T22:26:09.224413+0000 2023-09-30T22:26:09.224 INFO:teuthology.orchestra.run.smithi016.stdout:created 2023-09-30T22:26:09.224413+0000 2023-09-30T22:26:09.224 INFO:teuthology.orchestra.run.smithi016.stdout:min_mon_release 0 (unknown) 2023-09-30T22:26:09.224 INFO:teuthology.orchestra.run.smithi016.stdout:election_strategy: 1 2023-09-30T22:26:09.224 INFO:teuthology.orchestra.run.smithi016.stdout:0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2023-09-30T22:26:09.224 INFO:teuthology.orchestra.run.smithi016.stdout:1: [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] mon.b 2023-09-30T22:26:09.224 INFO:teuthology.orchestra.run.smithi016.stdout:2: [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] mon.c 2023-09-30T22:26:09.225 INFO:teuthology.orchestra.run.smithi016.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-09-30T22:26:09.226 DEBUG:teuthology.orchestra.run.smithi016:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-09-30T22:26:09.247 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 7ede3e14-2bd0-4726-9437-6d13c90cd342... 2023-09-30T22:26:09.248 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-09-30T22:26:09.289 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-09-30T22:26:09.345 INFO:teuthology.orchestra.run.smithi016.stdout:[global] 2023-09-30T22:26:09.345 INFO:teuthology.orchestra.run.smithi016.stdout: chdir = "" 2023-09-30T22:26:09.345 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-09-30T22:26:09.345 INFO:teuthology.orchestra.run.smithi016.stdout: auth supported = cephx 2023-09-30T22:26:09.345 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:26:09.345 INFO:teuthology.orchestra.run.smithi016.stdout: filestore xattr use omap = true 2023-09-30T22:26:09.345 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:26:09.345 INFO:teuthology.orchestra.run.smithi016.stdout: mon clock drift allowed = 1.000 2023-09-30T22:26:09.345 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:26:09.346 INFO:teuthology.orchestra.run.smithi016.stdout: osd crush chooseleaf type = 0 2023-09-30T22:26:09.346 INFO:teuthology.orchestra.run.smithi016.stdout: auth debug = true 2023-09-30T22:26:09.346 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:26:09.346 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on old message = true 2023-09-30T22:26:09.346 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on bug = true 2023-09-30T22:26:09.346 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:26:09.346 INFO:teuthology.orchestra.run.smithi016.stdout: mon max pg per osd = 10000 # >= luminous 2023-09-30T22:26:09.346 INFO:teuthology.orchestra.run.smithi016.stdout: mon pg warn max object skew = 0 2023-09-30T22:26:09.346 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:26:09.346 INFO:teuthology.orchestra.run.smithi016.stdout: # disable pg_autoscaler by default for new pools 2023-09-30T22:26:09.346 INFO:teuthology.orchestra.run.smithi016.stdout: osd_pool_default_pg_autoscale_mode = off 2023-09-30T22:26:09.346 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:26:09.347 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default size = 2 2023-09-30T22:26:09.347 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:26:09.347 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow primary affinity = true 2023-09-30T22:26:09.347 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow pg remap = true 2023-09-30T22:26:09.347 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on legacy crush tunables = false 2023-09-30T22:26:09.347 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on crush straw calc version zero = false 2023-09-30T22:26:09.347 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on no sortbitwise = false 2023-09-30T22:26:09.347 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on osd down out interval zero = false 2023-09-30T22:26:09.347 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on too few osds = false 2023-09-30T22:26:09.347 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-09-30T22:26:09.347 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_no_redundancy = false 2023-09-30T22:26:09.347 INFO:teuthology.orchestra.run.smithi016.stdout: mon_allow_pool_size_one = true 2023-09-30T22:26:09.348 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:26:09.348 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-09-30T22:26:09.348 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:26:09.348 INFO:teuthology.orchestra.run.smithi016.stdout: osd default data pool replay window = 5 2023-09-30T22:26:09.348 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:26:09.348 INFO:teuthology.orchestra.run.smithi016.stdout: mon allow pool delete = true 2023-09-30T22:26:09.348 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:26:09.348 INFO:teuthology.orchestra.run.smithi016.stdout: mon cluster log file level = debug 2023-09-30T22:26:09.348 INFO:teuthology.orchestra.run.smithi016.stdout: debug asserts on shutdown = true 2023-09-30T22:26:09.348 INFO:teuthology.orchestra.run.smithi016.stdout: mon health detail to clog = false 2023-09-30T22:26:09.348 INFO:teuthology.orchestra.run.smithi016.stdout: mon host = "172.21.15.16,[v2:172.21.15.16:3301,v1:172.21.15.16:6790],[v2:172.21.15.16:3302,v1:172.21.15.16:6791]" 2023-09-30T22:26:09.348 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on pool no app = False 2023-09-30T22:26:09.349 INFO:teuthology.orchestra.run.smithi016.stdout: ms bind msgr2 = False 2023-09-30T22:26:09.349 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore warn on legacy statfs = False 2023-09-30T22:26:09.349 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore warn on no per pool omap = False 2023-09-30T22:26:09.349 INFO:teuthology.orchestra.run.smithi016.stdout: mon pg warn min per osd = 0 2023-09-30T22:26:09.349 INFO:teuthology.orchestra.run.smithi016.stdout: fsid = 7ede3e14-2bd0-4726-9437-6d13c90cd342 2023-09-30T22:26:09.349 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:26:09.349 INFO:teuthology.orchestra.run.smithi016.stdout:[osd] 2023-09-30T22:26:09.349 INFO:teuthology.orchestra.run.smithi016.stdout: osd journal size = 100 2023-09-30T22:26:09.349 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:26:09.349 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub load threshold = 5.0 2023-09-30T22:26:09.349 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub max interval = 600 2023-09-30T22:26:09.349 INFO:teuthology.orchestra.run.smithi016.stdout: osd mclock profile = high_recovery_ops 2023-09-30T22:26:09.350 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:26:09.350 INFO:teuthology.orchestra.run.smithi016.stdout: osd recover clone overlap = true 2023-09-30T22:26:09.350 INFO:teuthology.orchestra.run.smithi016.stdout: osd recovery max chunk = 1048576 2023-09-30T22:26:09.350 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:26:09.350 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug shutdown = true 2023-09-30T22:26:09.350 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug op order = true 2023-09-30T22:26:09.350 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug verify stray on activate = true 2023-09-30T22:26:09.350 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:26:09.350 INFO:teuthology.orchestra.run.smithi016.stdout: osd open classes on start = true 2023-09-30T22:26:09.350 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug pg log writeout = true 2023-09-30T22:26:09.350 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:26:09.350 INFO:teuthology.orchestra.run.smithi016.stdout: osd deep scrub update digest min age = 30 2023-09-30T22:26:09.351 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:26:09.351 INFO:teuthology.orchestra.run.smithi016.stdout: osd map max advance = 10 2023-09-30T22:26:09.351 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:26:09.351 INFO:teuthology.orchestra.run.smithi016.stdout: journal zero on create = true 2023-09-30T22:26:09.351 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:26:09.351 INFO:teuthology.orchestra.run.smithi016.stdout: filestore ondisk finisher threads = 3 2023-09-30T22:26:09.351 INFO:teuthology.orchestra.run.smithi016.stdout: filestore apply finisher threads = 3 2023-09-30T22:26:09.351 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:26:09.351 INFO:teuthology.orchestra.run.smithi016.stdout: bdev debug aio = true 2023-09-30T22:26:09.351 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug misdirected ops = true 2023-09-30T22:26:09.351 INFO:teuthology.orchestra.run.smithi016.stdout: bdev async discard = True 2023-09-30T22:26:09.351 INFO:teuthology.orchestra.run.smithi016.stdout: bdev enable discard = True 2023-09-30T22:26:09.352 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore allocator = bitmap 2023-09-30T22:26:09.352 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore block size = 96636764160 2023-09-30T22:26:09.352 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore fsck on mount = True 2023-09-30T22:26:09.352 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluefs = 1/20 2023-09-30T22:26:09.352 INFO:teuthology.orchestra.run.smithi016.stdout: debug bluestore = 1/20 2023-09-30T22:26:09.352 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-09-30T22:26:09.352 INFO:teuthology.orchestra.run.smithi016.stdout: debug osd = 20 2023-09-30T22:26:09.352 INFO:teuthology.orchestra.run.smithi016.stdout: debug rocksdb = 4/10 2023-09-30T22:26:09.352 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd backfillfull_ratio = 0.85 2023-09-30T22:26:09.352 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd full ratio = 0.9 2023-09-30T22:26:09.352 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd nearfull ratio = 0.8 2023-09-30T22:26:09.352 INFO:teuthology.orchestra.run.smithi016.stdout: osd failsafe full ratio = 0.95 2023-09-30T22:26:09.353 INFO:teuthology.orchestra.run.smithi016.stdout: osd objectstore = bluestore 2023-09-30T22:26:09.353 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2023-09-30T22:26:09.353 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:26:09.353 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr] 2023-09-30T22:26:09.353 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-09-30T22:26:09.353 INFO:teuthology.orchestra.run.smithi016.stdout: debug mgr = 20 2023-09-30T22:26:09.353 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2023-09-30T22:26:09.353 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2023-09-30T22:26:09.353 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2023-09-30T22:26:09.353 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2023-09-30T22:26:09.353 INFO:teuthology.orchestra.run.smithi016.stdout: mgr/telemetry/nag = false 2023-09-30T22:26:09.353 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:26:09.354 INFO:teuthology.orchestra.run.smithi016.stdout:[mon] 2023-09-30T22:26:09.354 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-09-30T22:26:09.354 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2023-09-30T22:26:09.354 INFO:teuthology.orchestra.run.smithi016.stdout: debug paxos = 20 2023-09-30T22:26:09.354 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2023-09-30T22:26:09.354 INFO:teuthology.orchestra.run.smithi016.stdout: mon data avail warn = 5 2023-09-30T22:26:09.354 INFO:teuthology.orchestra.run.smithi016.stdout: mon mgr mkfs grace = 240 2023-09-30T22:26:09.354 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2023-09-30T22:26:09.355 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd reporter subtree level = osd 2023-09-30T22:26:09.355 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd prime pg temp = true 2023-09-30T22:26:09.355 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2023-09-30T22:26:09.355 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:26:09.355 INFO:teuthology.orchestra.run.smithi016.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-09-30T22:26:09.355 INFO:teuthology.orchestra.run.smithi016.stdout: auth mon ticket ttl = 660 # 11m 2023-09-30T22:26:09.355 INFO:teuthology.orchestra.run.smithi016.stdout: auth service ticket ttl = 240 # 4m 2023-09-30T22:26:09.355 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:26:09.355 INFO:teuthology.orchestra.run.smithi016.stdout: # don't complain about insecure global_id in the test suite 2023-09-30T22:26:09.355 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-09-30T22:26:09.355 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-09-30T22:26:09.355 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:26:09.356 INFO:teuthology.orchestra.run.smithi016.stdout: # 1m isn't quite enough 2023-09-30T22:26:09.356 INFO:teuthology.orchestra.run.smithi016.stdout: mon_down_mkfs_grace = 2m 2023-09-30T22:26:09.356 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:26:09.356 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_filestore_osds = false 2023-09-30T22:26:09.356 INFO:teuthology.orchestra.run.smithi016.stdout: mon op complaint time = 120 2023-09-30T22:26:09.356 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on osd down out interval zero = False 2023-09-30T22:26:09.356 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:26:09.356 INFO:teuthology.orchestra.run.smithi016.stdout:[client] 2023-09-30T22:26:09.356 INFO:teuthology.orchestra.run.smithi016.stdout: rgw cache enabled = true 2023-09-30T22:26:09.356 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable ops log = true 2023-09-30T22:26:09.356 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable usage log = true 2023-09-30T22:26:09.356 INFO:teuthology.orchestra.run.smithi016.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-09-30T22:26:09.356 INFO:teuthology.orchestra.run.smithi016.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-09-30T22:26:09.357 INFO:teuthology.orchestra.run.smithi016.stdout: client mount timeout = 600 2023-09-30T22:26:09.357 INFO:teuthology.orchestra.run.smithi016.stdout: debug client = 20 2023-09-30T22:26:09.357 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-09-30T22:26:09.357 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2023-09-30T22:26:09.357 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2023-09-30T22:26:09.357 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.a] 2023-09-30T22:26:09.357 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.b] 2023-09-30T22:26:09.357 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.c] 2023-09-30T22:26:09.357 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.a] 2023-09-30T22:26:09.357 INFO:teuthology.orchestra.run.smithi016.stdout:[mds] 2023-09-30T22:26:09.357 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds = 20 2023-09-30T22:26:09.357 INFO:teuthology.orchestra.run.smithi016.stdout: debug mds balancer = 20 2023-09-30T22:26:09.358 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-09-30T22:26:09.358 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug frag = True 2023-09-30T22:26:09.358 INFO:teuthology.orchestra.run.smithi016.stdout: mds debug scatterstat = True 2023-09-30T22:26:09.358 INFO:teuthology.orchestra.run.smithi016.stdout: mds op complaint time = 180 2023-09-30T22:26:09.358 INFO:teuthology.orchestra.run.smithi016.stdout: mds verify scatter = True 2023-09-30T22:26:09.358 INFO:teuthology.orchestra.run.smithi016.stdout: osd op complaint time = 180 2023-09-30T22:26:09.358 INFO:teuthology.orchestra.run.smithi016.stdout: rados mon op timeout = 900 2023-09-30T22:26:09.358 INFO:teuthology.orchestra.run.smithi016.stdout: rados osd op timeout = 900 2023-09-30T22:26:09.359 INFO:teuthology.orchestra.run.smithi027.stdout:[global] 2023-09-30T22:26:09.360 INFO:teuthology.orchestra.run.smithi027.stdout: chdir = "" 2023-09-30T22:26:09.360 INFO:teuthology.orchestra.run.smithi027.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-09-30T22:26:09.360 INFO:teuthology.orchestra.run.smithi027.stdout: auth supported = cephx 2023-09-30T22:26:09.360 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-09-30T22:26:09.360 INFO:teuthology.orchestra.run.smithi027.stdout: filestore xattr use omap = true 2023-09-30T22:26:09.360 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-09-30T22:26:09.360 INFO:teuthology.orchestra.run.smithi027.stdout: mon clock drift allowed = 1.000 2023-09-30T22:26:09.360 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-09-30T22:26:09.360 INFO:teuthology.orchestra.run.smithi027.stdout: osd crush chooseleaf type = 0 2023-09-30T22:26:09.360 INFO:teuthology.orchestra.run.smithi027.stdout: auth debug = true 2023-09-30T22:26:09.361 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-09-30T22:26:09.361 INFO:teuthology.orchestra.run.smithi027.stdout: ms die on old message = true 2023-09-30T22:26:09.361 INFO:teuthology.orchestra.run.smithi027.stdout: ms die on bug = true 2023-09-30T22:26:09.361 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-09-30T22:26:09.361 INFO:teuthology.orchestra.run.smithi027.stdout: mon max pg per osd = 10000 # >= luminous 2023-09-30T22:26:09.361 INFO:teuthology.orchestra.run.smithi027.stdout: mon pg warn max object skew = 0 2023-09-30T22:26:09.361 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-09-30T22:26:09.361 INFO:teuthology.orchestra.run.smithi027.stdout: # disable pg_autoscaler by default for new pools 2023-09-30T22:26:09.361 INFO:teuthology.orchestra.run.smithi027.stdout: osd_pool_default_pg_autoscale_mode = off 2023-09-30T22:26:09.361 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-09-30T22:26:09.361 INFO:teuthology.orchestra.run.smithi027.stdout: osd pool default size = 2 2023-09-30T22:26:09.361 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-09-30T22:26:09.362 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd allow primary affinity = true 2023-09-30T22:26:09.362 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd allow pg remap = true 2023-09-30T22:26:09.362 INFO:teuthology.orchestra.run.smithi027.stdout: mon warn on legacy crush tunables = false 2023-09-30T22:26:09.362 INFO:teuthology.orchestra.run.smithi027.stdout: mon warn on crush straw calc version zero = false 2023-09-30T22:26:09.362 INFO:teuthology.orchestra.run.smithi027.stdout: mon warn on no sortbitwise = false 2023-09-30T22:26:09.362 INFO:teuthology.orchestra.run.smithi027.stdout: mon warn on osd down out interval zero = false 2023-09-30T22:26:09.362 INFO:teuthology.orchestra.run.smithi027.stdout: mon warn on too few osds = false 2023-09-30T22:26:09.362 INFO:teuthology.orchestra.run.smithi027.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-09-30T22:26:09.362 INFO:teuthology.orchestra.run.smithi027.stdout: mon_warn_on_pool_no_redundancy = false 2023-09-30T22:26:09.362 INFO:teuthology.orchestra.run.smithi027.stdout: mon_allow_pool_size_one = true 2023-09-30T22:26:09.362 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-09-30T22:26:09.362 INFO:teuthology.orchestra.run.smithi027.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-09-30T22:26:09.363 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-09-30T22:26:09.363 INFO:teuthology.orchestra.run.smithi027.stdout: osd default data pool replay window = 5 2023-09-30T22:26:09.363 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-09-30T22:26:09.363 INFO:teuthology.orchestra.run.smithi027.stdout: mon allow pool delete = true 2023-09-30T22:26:09.363 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-09-30T22:26:09.363 INFO:teuthology.orchestra.run.smithi027.stdout: mon cluster log file level = debug 2023-09-30T22:26:09.363 INFO:teuthology.orchestra.run.smithi027.stdout: debug asserts on shutdown = true 2023-09-30T22:26:09.363 INFO:teuthology.orchestra.run.smithi027.stdout: mon health detail to clog = false 2023-09-30T22:26:09.363 INFO:teuthology.orchestra.run.smithi027.stdout: mon host = "172.21.15.16,[v2:172.21.15.16:3301,v1:172.21.15.16:6790],[v2:172.21.15.16:3302,v1:172.21.15.16:6791]" 2023-09-30T22:26:09.363 INFO:teuthology.orchestra.run.smithi027.stdout: mon warn on pool no app = False 2023-09-30T22:26:09.363 INFO:teuthology.orchestra.run.smithi027.stdout: ms bind msgr2 = False 2023-09-30T22:26:09.363 INFO:teuthology.orchestra.run.smithi027.stdout: bluestore warn on legacy statfs = False 2023-09-30T22:26:09.364 INFO:teuthology.orchestra.run.smithi027.stdout: bluestore warn on no per pool omap = False 2023-09-30T22:26:09.364 INFO:teuthology.orchestra.run.smithi027.stdout: mon pg warn min per osd = 0 2023-09-30T22:26:09.364 INFO:teuthology.orchestra.run.smithi027.stdout: fsid = 7ede3e14-2bd0-4726-9437-6d13c90cd342 2023-09-30T22:26:09.364 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-09-30T22:26:09.364 INFO:teuthology.orchestra.run.smithi027.stdout:[osd] 2023-09-30T22:26:09.364 INFO:teuthology.orchestra.run.smithi027.stdout: osd journal size = 100 2023-09-30T22:26:09.364 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-09-30T22:26:09.364 INFO:teuthology.orchestra.run.smithi027.stdout: osd scrub load threshold = 5.0 2023-09-30T22:26:09.364 INFO:teuthology.orchestra.run.smithi027.stdout: osd scrub max interval = 600 2023-09-30T22:26:09.364 INFO:teuthology.orchestra.run.smithi027.stdout: osd mclock profile = high_recovery_ops 2023-09-30T22:26:09.364 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-09-30T22:26:09.364 INFO:teuthology.orchestra.run.smithi027.stdout: osd recover clone overlap = true 2023-09-30T22:26:09.365 INFO:teuthology.orchestra.run.smithi027.stdout: osd recovery max chunk = 1048576 2023-09-30T22:26:09.365 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-09-30T22:26:09.365 INFO:teuthology.orchestra.run.smithi027.stdout: osd debug shutdown = true 2023-09-30T22:26:09.365 INFO:teuthology.orchestra.run.smithi027.stdout: osd debug op order = true 2023-09-30T22:26:09.365 INFO:teuthology.orchestra.run.smithi027.stdout: osd debug verify stray on activate = true 2023-09-30T22:26:09.365 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-09-30T22:26:09.365 INFO:teuthology.orchestra.run.smithi027.stdout: osd open classes on start = true 2023-09-30T22:26:09.365 INFO:teuthology.orchestra.run.smithi027.stdout: osd debug pg log writeout = true 2023-09-30T22:26:09.365 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-09-30T22:26:09.365 INFO:teuthology.orchestra.run.smithi027.stdout: osd deep scrub update digest min age = 30 2023-09-30T22:26:09.365 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-09-30T22:26:09.365 INFO:teuthology.orchestra.run.smithi027.stdout: osd map max advance = 10 2023-09-30T22:26:09.366 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-09-30T22:26:09.366 INFO:teuthology.orchestra.run.smithi027.stdout: journal zero on create = true 2023-09-30T22:26:09.366 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-09-30T22:26:09.366 INFO:teuthology.orchestra.run.smithi027.stdout: filestore ondisk finisher threads = 3 2023-09-30T22:26:09.366 INFO:teuthology.orchestra.run.smithi027.stdout: filestore apply finisher threads = 3 2023-09-30T22:26:09.366 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-09-30T22:26:09.366 INFO:teuthology.orchestra.run.smithi027.stdout: bdev debug aio = true 2023-09-30T22:26:09.366 INFO:teuthology.orchestra.run.smithi027.stdout: osd debug misdirected ops = true 2023-09-30T22:26:09.366 INFO:teuthology.orchestra.run.smithi027.stdout: bdev async discard = True 2023-09-30T22:26:09.366 INFO:teuthology.orchestra.run.smithi027.stdout: bdev enable discard = True 2023-09-30T22:26:09.366 INFO:teuthology.orchestra.run.smithi027.stdout: bluestore allocator = bitmap 2023-09-30T22:26:09.366 INFO:teuthology.orchestra.run.smithi027.stdout: bluestore block size = 96636764160 2023-09-30T22:26:09.367 INFO:teuthology.orchestra.run.smithi027.stdout: bluestore fsck on mount = True 2023-09-30T22:26:09.367 INFO:teuthology.orchestra.run.smithi027.stdout: debug bluefs = 1/20 2023-09-30T22:26:09.367 INFO:teuthology.orchestra.run.smithi027.stdout: debug bluestore = 1/20 2023-09-30T22:26:09.367 INFO:teuthology.orchestra.run.smithi027.stdout: debug ms = 1 2023-09-30T22:26:09.367 INFO:teuthology.orchestra.run.smithi027.stdout: debug osd = 20 2023-09-30T22:26:09.367 INFO:teuthology.orchestra.run.smithi027.stdout: debug rocksdb = 4/10 2023-09-30T22:26:09.367 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd backfillfull_ratio = 0.85 2023-09-30T22:26:09.367 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd full ratio = 0.9 2023-09-30T22:26:09.367 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd nearfull ratio = 0.8 2023-09-30T22:26:09.367 INFO:teuthology.orchestra.run.smithi027.stdout: osd failsafe full ratio = 0.95 2023-09-30T22:26:09.367 INFO:teuthology.orchestra.run.smithi027.stdout: osd objectstore = bluestore 2023-09-30T22:26:09.367 INFO:teuthology.orchestra.run.smithi027.stdout: osd op complaint time = 180 2023-09-30T22:26:09.368 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-09-30T22:26:09.368 INFO:teuthology.orchestra.run.smithi027.stdout:[mgr] 2023-09-30T22:26:09.368 INFO:teuthology.orchestra.run.smithi027.stdout: debug ms = 1 2023-09-30T22:26:09.368 INFO:teuthology.orchestra.run.smithi027.stdout: debug mgr = 20 2023-09-30T22:26:09.368 INFO:teuthology.orchestra.run.smithi027.stdout: debug mon = 20 2023-09-30T22:26:09.368 INFO:teuthology.orchestra.run.smithi027.stdout: debug auth = 20 2023-09-30T22:26:09.368 INFO:teuthology.orchestra.run.smithi027.stdout: mon reweight min pgs per osd = 4 2023-09-30T22:26:09.368 INFO:teuthology.orchestra.run.smithi027.stdout: mon reweight min bytes per osd = 10 2023-09-30T22:26:09.368 INFO:teuthology.orchestra.run.smithi027.stdout: mgr/telemetry/nag = false 2023-09-30T22:26:09.368 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-09-30T22:26:09.368 INFO:teuthology.orchestra.run.smithi027.stdout:[mon] 2023-09-30T22:26:09.368 INFO:teuthology.orchestra.run.smithi027.stdout: debug ms = 1 2023-09-30T22:26:09.369 INFO:teuthology.orchestra.run.smithi027.stdout: debug mon = 20 2023-09-30T22:26:09.369 INFO:teuthology.orchestra.run.smithi027.stdout: debug paxos = 20 2023-09-30T22:26:09.369 INFO:teuthology.orchestra.run.smithi027.stdout: debug auth = 20 2023-09-30T22:26:09.369 INFO:teuthology.orchestra.run.smithi027.stdout: mon data avail warn = 5 2023-09-30T22:26:09.369 INFO:teuthology.orchestra.run.smithi027.stdout: mon mgr mkfs grace = 240 2023-09-30T22:26:09.369 INFO:teuthology.orchestra.run.smithi027.stdout: mon reweight min pgs per osd = 4 2023-09-30T22:26:09.369 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd reporter subtree level = osd 2023-09-30T22:26:09.369 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd prime pg temp = true 2023-09-30T22:26:09.369 INFO:teuthology.orchestra.run.smithi027.stdout: mon reweight min bytes per osd = 10 2023-09-30T22:26:09.369 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-09-30T22:26:09.369 INFO:teuthology.orchestra.run.smithi027.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-09-30T22:26:09.369 INFO:teuthology.orchestra.run.smithi027.stdout: auth mon ticket ttl = 660 # 11m 2023-09-30T22:26:09.370 INFO:teuthology.orchestra.run.smithi027.stdout: auth service ticket ttl = 240 # 4m 2023-09-30T22:26:09.370 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-09-30T22:26:09.370 INFO:teuthology.orchestra.run.smithi027.stdout: # don't complain about insecure global_id in the test suite 2023-09-30T22:26:09.370 INFO:teuthology.orchestra.run.smithi027.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-09-30T22:26:09.370 INFO:teuthology.orchestra.run.smithi027.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-09-30T22:26:09.370 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-09-30T22:26:09.370 INFO:teuthology.orchestra.run.smithi027.stdout: # 1m isn't quite enough 2023-09-30T22:26:09.370 INFO:teuthology.orchestra.run.smithi027.stdout: mon_down_mkfs_grace = 2m 2023-09-30T22:26:09.370 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-09-30T22:26:09.370 INFO:teuthology.orchestra.run.smithi027.stdout: mon_warn_on_filestore_osds = false 2023-09-30T22:26:09.370 INFO:teuthology.orchestra.run.smithi027.stdout: mon op complaint time = 120 2023-09-30T22:26:09.370 INFO:teuthology.orchestra.run.smithi027.stdout: mon warn on osd down out interval zero = False 2023-09-30T22:26:09.371 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-09-30T22:26:09.371 INFO:teuthology.orchestra.run.smithi027.stdout:[client] 2023-09-30T22:26:09.371 INFO:teuthology.orchestra.run.smithi027.stdout: rgw cache enabled = true 2023-09-30T22:26:09.371 INFO:teuthology.orchestra.run.smithi027.stdout: rgw enable ops log = true 2023-09-30T22:26:09.371 INFO:teuthology.orchestra.run.smithi027.stdout: rgw enable usage log = true 2023-09-30T22:26:09.371 INFO:teuthology.orchestra.run.smithi027.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-09-30T22:26:09.371 INFO:teuthology.orchestra.run.smithi027.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-09-30T22:26:09.371 INFO:teuthology.orchestra.run.smithi027.stdout: client mount timeout = 600 2023-09-30T22:26:09.371 INFO:teuthology.orchestra.run.smithi027.stdout: debug client = 20 2023-09-30T22:26:09.371 INFO:teuthology.orchestra.run.smithi027.stdout: debug ms = 1 2023-09-30T22:26:09.371 INFO:teuthology.orchestra.run.smithi027.stdout: rados mon op timeout = 900 2023-09-30T22:26:09.371 INFO:teuthology.orchestra.run.smithi027.stdout: rados osd op timeout = 900 2023-09-30T22:26:09.372 INFO:teuthology.orchestra.run.smithi027.stdout:[mon.a] 2023-09-30T22:26:09.372 INFO:teuthology.orchestra.run.smithi027.stdout:[mon.b] 2023-09-30T22:26:09.372 INFO:teuthology.orchestra.run.smithi027.stdout:[mon.c] 2023-09-30T22:26:09.372 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.a] 2023-09-30T22:26:09.372 INFO:teuthology.orchestra.run.smithi027.stdout:[mds] 2023-09-30T22:26:09.372 INFO:teuthology.orchestra.run.smithi027.stdout: debug mds = 20 2023-09-30T22:26:09.372 INFO:teuthology.orchestra.run.smithi027.stdout: debug mds balancer = 20 2023-09-30T22:26:09.372 INFO:teuthology.orchestra.run.smithi027.stdout: debug ms = 1 2023-09-30T22:26:09.372 INFO:teuthology.orchestra.run.smithi027.stdout: mds debug frag = True 2023-09-30T22:26:09.372 INFO:teuthology.orchestra.run.smithi027.stdout: mds debug scatterstat = True 2023-09-30T22:26:09.372 INFO:teuthology.orchestra.run.smithi027.stdout: mds op complaint time = 180 2023-09-30T22:26:09.372 INFO:teuthology.orchestra.run.smithi027.stdout: mds verify scatter = True 2023-09-30T22:26:09.372 INFO:teuthology.orchestra.run.smithi027.stdout: osd op complaint time = 180 2023-09-30T22:26:09.373 INFO:teuthology.orchestra.run.smithi027.stdout: rados mon op timeout = 900 2023-09-30T22:26:09.373 INFO:teuthology.orchestra.run.smithi027.stdout: rados osd op timeout = 900 2023-09-30T22:26:09.373 INFO:tasks.ceph:Creating admin key on mon.a... 2023-09-30T22:26:09.373 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-09-30T22:26:09.466 INFO:tasks.ceph:Copying monmap to all nodes... 2023-09-30T22:26:09.466 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-09-30T22:26:09.466 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-09-30T22:26:09.487 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-09-30T22:26:09.487 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-09-30T22:26:09.550 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi016.front.sepia.ceph.com 2023-09-30T22:26:09.550 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-09-30T22:26:09.551 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/ceph/ceph.keyring 2023-09-30T22:26:09.551 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-09-30T22:26:09.638 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-09-30T22:26:09.638 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-09-30T22:26:09.701 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi027.front.sepia.ceph.com 2023-09-30T22:26:09.701 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-09-30T22:26:09.701 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/ceph/ceph.keyring 2023-09-30T22:26:09.701 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-09-30T22:26:09.746 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-09-30T22:26:09.746 DEBUG:teuthology.orchestra.run.smithi027:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-09-30T22:26:09.807 INFO:tasks.ceph:Setting up mon nodes... 2023-09-30T22:26:09.808 INFO:tasks.ceph:Setting up mgr nodes... 2023-09-30T22:26:09.808 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-09-30T22:26:09.874 INFO:teuthology.orchestra.run.smithi016.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-09-30T22:26:09.874 INFO:tasks.ceph:Setting up mds nodes... 2023-09-30T22:26:09.875 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-09-30T22:26:09.934 INFO:teuthology.orchestra.run.smithi016.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-09-30T22:26:09.937 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-09-30T22:26:09.969 INFO:tasks.ceph_client:Setting up client nodes... 2023-09-30T22:26:09.969 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-09-30T22:26:10.099 INFO:teuthology.orchestra.run.smithi027.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-09-30T22:26:10.114 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-09-30T22:26:10.114 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi016.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}} 2023-09-30T22:26:10.114 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-09-30T22:26:10.149 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-09-30T22:26:10.149 INFO:tasks.ceph:role: osd.0 2023-09-30T22:26:10.149 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi016.front.sepia.ceph.com 2023-09-30T22:26:10.149 DEBUG:teuthology.orchestra.run.smithi016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-09-30T22:26:10.224 INFO:teuthology.orchestra.run.smithi016.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-09-30T22:26:10.224 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-30T22:26:10.224 INFO:teuthology.orchestra.run.smithi016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-30T22:26:10.224 INFO:teuthology.orchestra.run.smithi016.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-09-30T22:26:10.224 INFO:teuthology.orchestra.run.smithi016.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-30T22:26:10.224 INFO:teuthology.orchestra.run.smithi016.stdout: = sunit=0 swidth=0 blks 2023-09-30T22:26:10.224 INFO:teuthology.orchestra.run.smithi016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-30T22:26:10.225 INFO:teuthology.orchestra.run.smithi016.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-09-30T22:26:10.225 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-30T22:26:10.225 INFO:teuthology.orchestra.run.smithi016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-30T22:26:10.247 INFO:teuthology.orchestra.run.smithi016.stdout:Discarding blocks...Done. 2023-09-30T22:26:10.249 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi016.front.sepia.ceph.com -o noatime 2023-09-30T22:26:10.249 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-09-30T22:26:10.339 DEBUG:teuthology.orchestra.run.smithi016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-09-30T22:26:10.377 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-09-30T22:26:10.453 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-09-30T22:26:10.453 INFO:tasks.ceph:role: osd.1 2023-09-30T22:26:10.453 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi016.front.sepia.ceph.com 2023-09-30T22:26:10.453 DEBUG:teuthology.orchestra.run.smithi016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-09-30T22:26:10.528 INFO:teuthology.orchestra.run.smithi016.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-09-30T22:26:10.528 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-30T22:26:10.528 INFO:teuthology.orchestra.run.smithi016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-30T22:26:10.528 INFO:teuthology.orchestra.run.smithi016.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-09-30T22:26:10.528 INFO:teuthology.orchestra.run.smithi016.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-30T22:26:10.528 INFO:teuthology.orchestra.run.smithi016.stdout: = sunit=0 swidth=0 blks 2023-09-30T22:26:10.529 INFO:teuthology.orchestra.run.smithi016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-30T22:26:10.529 INFO:teuthology.orchestra.run.smithi016.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-09-30T22:26:10.529 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-30T22:26:10.529 INFO:teuthology.orchestra.run.smithi016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-30T22:26:10.551 INFO:teuthology.orchestra.run.smithi016.stdout:Discarding blocks...Done. 2023-09-30T22:26:10.553 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi016.front.sepia.ceph.com -o noatime 2023-09-30T22:26:10.553 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-09-30T22:26:10.641 DEBUG:teuthology.orchestra.run.smithi016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-09-30T22:26:10.678 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-09-30T22:26:10.755 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-09-30T22:26:10.755 INFO:tasks.ceph:role: osd.2 2023-09-30T22:26:10.755 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi016.front.sepia.ceph.com 2023-09-30T22:26:10.755 DEBUG:teuthology.orchestra.run.smithi016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-09-30T22:26:10.830 INFO:teuthology.orchestra.run.smithi016.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-09-30T22:26:10.830 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-30T22:26:10.830 INFO:teuthology.orchestra.run.smithi016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-30T22:26:10.830 INFO:teuthology.orchestra.run.smithi016.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-09-30T22:26:10.830 INFO:teuthology.orchestra.run.smithi016.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-30T22:26:10.830 INFO:teuthology.orchestra.run.smithi016.stdout: = sunit=0 swidth=0 blks 2023-09-30T22:26:10.831 INFO:teuthology.orchestra.run.smithi016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-30T22:26:10.831 INFO:teuthology.orchestra.run.smithi016.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-09-30T22:26:10.831 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-30T22:26:10.831 INFO:teuthology.orchestra.run.smithi016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-30T22:26:10.853 INFO:teuthology.orchestra.run.smithi016.stdout:Discarding blocks...Done. 2023-09-30T22:26:10.854 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi016.front.sepia.ceph.com -o noatime 2023-09-30T22:26:10.855 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-09-30T22:26:10.944 DEBUG:teuthology.orchestra.run.smithi016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-09-30T22:26:10.980 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-09-30T22:26:11.055 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-09-30T22:26:11.055 INFO:tasks.ceph:role: osd.3 2023-09-30T22:26:11.056 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi016.front.sepia.ceph.com 2023-09-30T22:26:11.056 DEBUG:teuthology.orchestra.run.smithi016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-09-30T22:26:11.130 INFO:teuthology.orchestra.run.smithi016.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-09-30T22:26:11.131 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 attr=2, projid32bit=1 2023-09-30T22:26:11.131 INFO:teuthology.orchestra.run.smithi016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-09-30T22:26:11.131 INFO:teuthology.orchestra.run.smithi016.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-09-30T22:26:11.131 INFO:teuthology.orchestra.run.smithi016.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-09-30T22:26:11.131 INFO:teuthology.orchestra.run.smithi016.stdout: = sunit=0 swidth=0 blks 2023-09-30T22:26:11.131 INFO:teuthology.orchestra.run.smithi016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-09-30T22:26:11.131 INFO:teuthology.orchestra.run.smithi016.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-09-30T22:26:11.131 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-09-30T22:26:11.131 INFO:teuthology.orchestra.run.smithi016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-09-30T22:26:11.154 INFO:teuthology.orchestra.run.smithi016.stdout:Discarding blocks...Done. 2023-09-30T22:26:11.156 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi016.front.sepia.ceph.com -o noatime 2023-09-30T22:26:11.156 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-09-30T22:26:11.245 DEBUG:teuthology.orchestra.run.smithi016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-09-30T22:26:11.281 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-30T22:26:11.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:11.386+0000 7ff4f021d200 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-09-30T22:26:11.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:11.386+0000 7ff4f021d200 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-09-30T22:26:11.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:11.386+0000 7ff4f021d200 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-09-30T22:26:11.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:11.386+0000 7ff4f021d200 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-09-30T22:26:11.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:11.386+0000 7ff4f021d200 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-09-30T22:26:11.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:11.387+0000 7ff4f021d200 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-09-30T22:26:14.243 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-09-30T22:26:14.278 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-30T22:26:14.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:14.598+0000 7f3bc487a200 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-09-30T22:26:14.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:14.598+0000 7f3bc487a200 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-09-30T22:26:14.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:14.598+0000 7f3bc487a200 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-09-30T22:26:14.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:14.598+0000 7f3bc487a200 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-09-30T22:26:14.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:14.598+0000 7f3bc487a200 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-09-30T22:26:14.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:14.599+0000 7f3bc487a200 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-09-30T22:26:17.446 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-09-30T22:26:17.482 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-30T22:26:17.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:17.577+0000 7f07623db200 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-09-30T22:26:17.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:17.577+0000 7f07623db200 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-09-30T22:26:17.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:17.577+0000 7f07623db200 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-09-30T22:26:17.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:17.577+0000 7f07623db200 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-09-30T22:26:17.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:17.577+0000 7f07623db200 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-09-30T22:26:17.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:17.577+0000 7f07623db200 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-09-30T22:26:20.420 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-09-30T22:26:20.455 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-09-30T22:26:20.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:20.551+0000 7efe9828f200 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-09-30T22:26:20.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:20.551+0000 7efe9828f200 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-09-30T22:26:20.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:20.551+0000 7efe9828f200 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-09-30T22:26:20.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:20.551+0000 7efe9828f200 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-09-30T22:26:20.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:20.551+0000 7efe9828f200 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-09-30T22:26:20.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:20.552+0000 7efe9828f200 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-09-30T22:26:23.395 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-09-30T22:26:23.431 INFO:tasks.ceph:Reading keys from all nodes... 2023-09-30T22:26:23.431 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-09-30T22:26:23.432 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-09-30T22:26:23.507 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-09-30T22:26:23.507 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-09-30T22:26:23.582 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-09-30T22:26:23.582 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-09-30T22:26:23.657 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-09-30T22:26:23.657 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-09-30T22:26:23.732 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-09-30T22:26:23.732 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-09-30T22:26:23.807 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-09-30T22:26:23.807 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-09-30T22:26:23.882 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-09-30T22:26:23.882 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-09-30T22:26:23.905 INFO:tasks.ceph:Adding keys to all mons... 2023-09-30T22:26:23.906 DEBUG:teuthology.orchestra.run.smithi016:> sudo tee -a /etc/ceph/ceph.keyring 2023-09-30T22:26:23.957 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr.x] 2023-09-30T22:26:23.957 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCBoBhl+K0BNBAAstVd/2I6D0aHSCapyIE/Ug== 2023-09-30T22:26:23.957 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.a] 2023-09-30T22:26:23.957 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCBoBhli1PGNxAAdK2GvLXMl7dwIU9v6H2MLg== 2023-09-30T22:26:23.957 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.0] 2023-09-30T22:26:23.958 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCDoBhlz9IXFxAAXAzx2lJnhfQ7hINtgSKGrA== 2023-09-30T22:26:23.958 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.1] 2023-09-30T22:26:23.958 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCGoBhli8G6IxAAbn4BavB1MS2Rck8XOEA3Qw== 2023-09-30T22:26:23.958 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.2] 2023-09-30T22:26:23.958 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCJoBhlLHh0IhAAnFNXAkixHQp9+5uj3h0Bqw== 2023-09-30T22:26:23.958 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.3] 2023-09-30T22:26:23.958 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCMoBhlz/HsIBAALFix8LpSrOoiOHN1Nue9Pw== 2023-09-30T22:26:23.959 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2023-09-30T22:26:23.959 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQCCoBhlXFv+BRAAjA62biNHLIPA2p2kX5GdIw== 2023-09-30T22:26:23.960 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-09-30T22:26:24.061 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-09-30T22:26:24.122 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-30T22:26:24.178 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-30T22:26:24.234 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-30T22:26:24.289 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-09-30T22:26:24.344 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-09-30T22:26:24.402 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-09-30T22:26:24.403 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-09-30T22:26:24.433 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-09-30T22:26:24.535 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-09-30T22:26:24.567 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-09-30T22:26:24.642 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-09-30T22:26:24.749 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-09-30T22:26:24.781 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-09-30T22:26:24.857 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-09-30T22:26:24.963 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-09-30T22:26:24.996 DEBUG:teuthology.orchestra.run.smithi016:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-09-30T22:26:25.058 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-09-30T22:26:25.058 INFO:tasks.ceph.mon.a:Restarting daemon 2023-09-30T22:26:25.059 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-09-30T22:26:25.100 INFO:tasks.ceph.mon.a:Started 2023-09-30T22:26:25.100 INFO:tasks.ceph.mon.b:Restarting daemon 2023-09-30T22:26:25.101 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-09-30T22:26:25.102 INFO:tasks.ceph.mon.b:Started 2023-09-30T22:26:25.103 INFO:tasks.ceph.mon.c:Restarting daemon 2023-09-30T22:26:25.103 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-09-30T22:26:25.105 INFO:tasks.ceph.mon.c:Started 2023-09-30T22:26:25.105 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-09-30T22:26:25.105 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-09-30T22:26:25.106 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-09-30T22:26:25.108 INFO:tasks.ceph.mgr.x:Started 2023-09-30T22:26:25.108 INFO:tasks.ceph:Setting crush tunables to default 2023-09-30T22:26:25.108 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph osd crush tunables default 2023-09-30T22:26:25.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:25.263+0000 7f7f81f18700 1 Processor -- start 2023-09-30T22:26:25.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:25.263+0000 7f7f81f18700 1 -- start start 2023-09-30T22:26:25.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:25.264+0000 7f7f81f18700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7f7c091d30 0x7f7f7c096620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:25.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:25.264+0000 7f7f81f18700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f7c12e460 0x7f7f7c130850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:25.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:25.264+0000 7f7f81f18700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f7c093f70 con 0x7f7f7c096b10 2023-09-30T22:26:25.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:25.264+0000 7f7f81f18700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f7c092110 con 0x7f7f7c091d30 2023-09-30T22:26:25.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:25.264+0000 7f7f81f18700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c0927f0 con 0x7f7f7c12e460 2023-09-30T22:26:25.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:25.264+0000 7f7f7b7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7f7c091d30 0x7f7f7c096620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:25.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:25.264+0000 7f7f7bfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f7c12e460 0x7f7f7c130850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:25.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:25.264+0000 7f7f7affd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7f7c096b10 0x7f7f7c096ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:42118/0 (socket says 172.21.15.16:42118) 2023-09-30T22:26:25.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:25.264+0000 7f7f7b7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7f7c091d30 0x7f7f7c096620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54936/0 (socket says 172.21.15.16:54936) 2023-09-30T22:26:25.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:25.264+0000 7f7f7affd700 1 -- 172.21.15.16:0/3793125773 learned_addr learned my addr 172.21.15.16:0/3793125773 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:26:25.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:25.264+0000 7f7f7bfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f7c12e460 0x7f7f7c130850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:51342/0 (socket says 172.21.15.16:51342) 2023-09-30T22:26:25.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:25.264+0000 7f7f7bfff700 1 -- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f7c12e460 msgr2=0x7f7f7c130850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-09-30T22:26:25.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:25.264+0000 7f7f7bfff700 1 -- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f7c12e460 msgr2=0x7f7f7c130850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:26:25.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:25.264+0000 7f7f7bfff700 1 --2- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f7c12e460 0x7f7f7c130850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-30T22:26:25.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:25.264+0000 7f7f7b7fe700 1 -- 172.21.15.16:0/3793125773 >> v2:172.21.15.16:3300/0 conn(0x7f7f7c091d30 msgr2=0x7f7f7c096620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-09-30T22:26:25.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:25.264+0000 7f7f7b7fe700 1 -- 172.21.15.16:0/3793125773 >> v2:172.21.15.16:3300/0 conn(0x7f7f7c091d30 msgr2=0x7f7f7c096620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:26:25.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:25.264+0000 7f7f7b7fe700 1 --2- 172.21.15.16:0/3793125773 >> v2:172.21.15.16:3300/0 conn(0x7f7f7c091d30 0x7f7f7c096620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-30T22:26:25.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:25.264+0000 7f7f7bfff700 1 --2- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f7c12e460 0x7f7f7c130850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2023-09-30T22:26:25.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:25.264+0000 7f7f7b7fe700 1 --2- 172.21.15.16:0/3793125773 >> v2:172.21.15.16:3300/0 conn(0x7f7f7c091d30 0x7f7f7c096620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2023-09-30T22:26:25.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:25.465+0000 7f7f7bfff700 1 --2- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f7c12e460 0x7f7f7c130850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:25.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:25.465+0000 7f7f7b7fe700 1 --2- 172.21.15.16:0/3793125773 >> v2:172.21.15.16:3300/0 conn(0x7f7f7c091d30 0x7f7f7c096620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:25.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:25.465+0000 7f7f7bfff700 1 -- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f7c12e460 msgr2=0x7f7f7c130850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-09-30T22:26:25.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:25.465+0000 7f7f7bfff700 1 -- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f7c12e460 msgr2=0x7f7f7c130850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:26:25.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:25.465+0000 7f7f7bfff700 1 --2- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f7c12e460 0x7f7f7c130850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-30T22:26:25.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:25.465+0000 7f7f7b7fe700 1 -- 172.21.15.16:0/3793125773 >> v2:172.21.15.16:3300/0 conn(0x7f7f7c091d30 msgr2=0x7f7f7c096620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-09-30T22:26:25.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:25.465+0000 7f7f7b7fe700 1 -- 172.21.15.16:0/3793125773 >> v2:172.21.15.16:3300/0 conn(0x7f7f7c091d30 msgr2=0x7f7f7c096620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:26:25.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:25.465+0000 7f7f7b7fe700 1 --2- 172.21.15.16:0/3793125773 >> v2:172.21.15.16:3300/0 conn(0x7f7f7c091d30 0x7f7f7c096620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-30T22:26:25.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:25.465+0000 7f7f7bfff700 1 --2- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f7c12e460 0x7f7f7c130850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2023-09-30T22:26:25.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:25.465+0000 7f7f7b7fe700 1 --2- 172.21.15.16:0/3793125773 >> v2:172.21.15.16:3300/0 conn(0x7f7f7c091d30 0x7f7f7c096620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2023-09-30T22:26:25.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:25.866+0000 7f7f7bfff700 1 --2- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f7c12e460 0x7f7f7c130850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:25.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:25.866+0000 7f7f7b7fe700 1 --2- 172.21.15.16:0/3793125773 >> v2:172.21.15.16:3300/0 conn(0x7f7f7c091d30 0x7f7f7c096620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:25.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:25.866+0000 7f7f7bfff700 1 -- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f7c12e460 msgr2=0x7f7f7c130850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-09-30T22:26:25.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:25.866+0000 7f7f7bfff700 1 -- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f7c12e460 msgr2=0x7f7f7c130850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:26:25.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:25.866+0000 7f7f7bfff700 1 --2- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f7c12e460 0x7f7f7c130850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-30T22:26:25.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:25.866+0000 7f7f7bfff700 1 --2- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f7c12e460 0x7f7f7c130850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2023-09-30T22:26:25.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:25.866+0000 7f7f7b7fe700 1 -- 172.21.15.16:0/3793125773 >> v2:172.21.15.16:3300/0 conn(0x7f7f7c091d30 msgr2=0x7f7f7c096620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-09-30T22:26:25.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:25.866+0000 7f7f7b7fe700 1 -- 172.21.15.16:0/3793125773 >> v2:172.21.15.16:3300/0 conn(0x7f7f7c091d30 msgr2=0x7f7f7c096620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:26:25.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:25.866+0000 7f7f7b7fe700 1 --2- 172.21.15.16:0/3793125773 >> v2:172.21.15.16:3300/0 conn(0x7f7f7c091d30 0x7f7f7c096620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-30T22:26:25.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:25.866+0000 7f7f7b7fe700 1 --2- 172.21.15.16:0/3793125773 >> v2:172.21.15.16:3300/0 conn(0x7f7f7c091d30 0x7f7f7c096620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2023-09-30T22:26:26.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:26.667+0000 7f7f7bfff700 1 --2- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f7c12e460 0x7f7f7c130850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:26.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:26.667+0000 7f7f7bfff700 1 -- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f7c12e460 msgr2=0x7f7f7c130850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-09-30T22:26:26.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:26.667+0000 7f7f7bfff700 1 -- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f7c12e460 msgr2=0x7f7f7c130850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:26:26.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:26.667+0000 7f7f7bfff700 1 --2- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f7c12e460 0x7f7f7c130850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-30T22:26:26.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:26.667+0000 7f7f7bfff700 1 --2- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f7c12e460 0x7f7f7c130850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2023-09-30T22:26:26.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:26.667+0000 7f7f7b7fe700 1 --2- 172.21.15.16:0/3793125773 >> v2:172.21.15.16:3300/0 conn(0x7f7f7c091d30 0x7f7f7c096620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:26.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:26.667+0000 7f7f7b7fe700 1 -- 172.21.15.16:0/3793125773 >> v2:172.21.15.16:3300/0 conn(0x7f7f7c091d30 msgr2=0x7f7f7c096620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-09-30T22:26:26.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:26.667+0000 7f7f7b7fe700 1 -- 172.21.15.16:0/3793125773 >> v2:172.21.15.16:3300/0 conn(0x7f7f7c091d30 msgr2=0x7f7f7c096620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:26:26.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:26.667+0000 7f7f7b7fe700 1 --2- 172.21.15.16:0/3793125773 >> v2:172.21.15.16:3300/0 conn(0x7f7f7c091d30 0x7f7f7c096620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-30T22:26:26.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:26.667+0000 7f7f7b7fe700 1 --2- 172.21.15.16:0/3793125773 >> v2:172.21.15.16:3300/0 conn(0x7f7f7c091d30 0x7f7f7c096620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2023-09-30T22:26:28.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:28.264+0000 7f7f797fa700 1 -- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f7c12e460 msgr2=0x7f7f7c130850 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T22:26:28.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:28.264+0000 7f7f797fa700 1 --2- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f7c12e460 0x7f7f7c130850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:28.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:28.264+0000 7f7f797fa700 1 -- 172.21.15.16:0/3793125773 >> v2:172.21.15.16:3300/0 conn(0x7f7f7c091d30 msgr2=0x7f7f7c096620 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T22:26:28.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:28.264+0000 7f7f797fa700 1 --2- 172.21.15.16:0/3793125773 >> v2:172.21.15.16:3300/0 conn(0x7f7f7c091d30 0x7f7f7c096620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:28.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:28.264+0000 7f7f797fa700 1 -- 172.21.15.16:0/3793125773 >> v1:172.21.15.16:6789/0 conn(0x7f7f7c096b10 legacy=0x7f7f7c096ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:28.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:28.264+0000 7f7f797fa700 1 --2- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f5c003500 0x7f7f5c005990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:28.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:28.264+0000 7f7f797fa700 1 -- 172.21.15.16:0/3793125773 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c092110 con 0x7f7f5c003500 2023-09-30T22:26:28.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:28.264+0000 7f7f7b7fe700 1 --2- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f5c003500 0x7f7f5c005990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:28.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:28.264+0000 7f7f7b7fe700 1 -- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f5c003500 msgr2=0x7f7f5c005990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-09-30T22:26:28.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:28.264+0000 7f7f7b7fe700 1 -- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f5c003500 msgr2=0x7f7f5c005990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:26:28.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:28.264+0000 7f7f7b7fe700 1 --2- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f5c003500 0x7f7f5c005990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-30T22:26:28.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:28.264+0000 7f7f7b7fe700 1 --2- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f5c003500 0x7f7f5c005990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2023-09-30T22:26:28.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:28.465+0000 7f7f7b7fe700 1 --2- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f5c003500 0x7f7f5c005990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:28.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:28.465+0000 7f7f7b7fe700 1 -- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f5c003500 msgr2=0x7f7f5c005990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-09-30T22:26:28.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:28.465+0000 7f7f7b7fe700 1 -- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f5c003500 msgr2=0x7f7f5c005990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:26:28.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:28.465+0000 7f7f7b7fe700 1 --2- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f5c003500 0x7f7f5c005990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-30T22:26:28.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:28.465+0000 7f7f7b7fe700 1 --2- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f5c003500 0x7f7f5c005990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2023-09-30T22:26:28.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:28.866+0000 7f7f7b7fe700 1 --2- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f5c003500 0x7f7f5c005990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:28.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:28.866+0000 7f7f7b7fe700 1 -- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f5c003500 msgr2=0x7f7f5c005990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-09-30T22:26:28.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:28.866+0000 7f7f7b7fe700 1 -- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f5c003500 msgr2=0x7f7f5c005990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:26:28.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:28.866+0000 7f7f7b7fe700 1 --2- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f5c003500 0x7f7f5c005990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-30T22:26:28.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:28.866+0000 7f7f7b7fe700 1 --2- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f5c003500 0x7f7f5c005990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2023-09-30T22:26:29.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:29.667+0000 7f7f7b7fe700 1 --2- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f5c003500 0x7f7f5c005990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:29.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:29.667+0000 7f7f7b7fe700 1 -- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f5c003500 msgr2=0x7f7f5c005990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-09-30T22:26:29.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:29.667+0000 7f7f7b7fe700 1 -- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f5c003500 msgr2=0x7f7f5c005990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:26:29.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:29.667+0000 7f7f7b7fe700 1 --2- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f5c003500 0x7f7f5c005990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-30T22:26:29.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:29.667+0000 7f7f7b7fe700 1 --2- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f5c003500 0x7f7f5c005990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2023-09-30T22:26:31.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:31.264+0000 7f7f797fa700 1 -- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f5c003500 msgr2=0x7f7f5c005990 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T22:26:31.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:31.264+0000 7f7f797fa700 1 --2- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f5c003500 0x7f7f5c005990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:31.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:31.264+0000 7f7f797fa700 1 --2- 172.21.15.16:0/3793125773 >> v2:172.21.15.16:3300/0 conn(0x7f7f7c091d30 0x7f7f5c008240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:31.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:31.264+0000 7f7f797fa700 1 --2- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f5c003500 0x7f7f5c005990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:31.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:31.264+0000 7f7f797fa700 1 -- 172.21.15.16:0/3793125773 --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f7c095c40 con 0x7f7f7c096b10 2023-09-30T22:26:31.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:31.264+0000 7f7f797fa700 1 -- 172.21.15.16:0/3793125773 --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f7c092110 con 0x7f7f7c091d30 2023-09-30T22:26:31.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:31.264+0000 7f7f797fa700 1 -- 172.21.15.16:0/3793125773 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c0927f0 con 0x7f7f5c003500 2023-09-30T22:26:31.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:31.264+0000 7f7f7b7fe700 1 --2- 172.21.15.16:0/3793125773 >> v2:172.21.15.16:3300/0 conn(0x7f7f7c091d30 0x7f7f5c008240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:31.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:31.264+0000 7f7f7affd700 1 --2- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f5c003500 0x7f7f5c005990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:31.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:31.264+0000 7f7f7b7fe700 1 -- 172.21.15.16:0/3793125773 >> v2:172.21.15.16:3300/0 conn(0x7f7f7c091d30 msgr2=0x7f7f5c008240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-09-30T22:26:31.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:31.264+0000 7f7f7b7fe700 1 -- 172.21.15.16:0/3793125773 >> v2:172.21.15.16:3300/0 conn(0x7f7f7c091d30 msgr2=0x7f7f5c008240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:26:31.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:31.264+0000 7f7f7b7fe700 1 --2- 172.21.15.16:0/3793125773 >> v2:172.21.15.16:3300/0 conn(0x7f7f7c091d30 0x7f7f5c008240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-30T22:26:31.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:31.265+0000 7f7f7b7fe700 1 --2- 172.21.15.16:0/3793125773 >> v2:172.21.15.16:3300/0 conn(0x7f7f7c091d30 0x7f7f5c008240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2023-09-30T22:26:31.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:31.265+0000 7f7f7affd700 1 -- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f5c003500 msgr2=0x7f7f5c005990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-09-30T22:26:31.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:31.265+0000 7f7f7affd700 1 -- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f5c003500 msgr2=0x7f7f5c005990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:26:31.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:31.265+0000 7f7f7affd700 1 --2- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f5c003500 0x7f7f5c005990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-30T22:26:31.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:31.265+0000 7f7f7affd700 1 --2- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f5c003500 0x7f7f5c005990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2023-09-30T22:26:31.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:31.466+0000 7f7f7b7fe700 1 --2- 172.21.15.16:0/3793125773 >> v2:172.21.15.16:3300/0 conn(0x7f7f7c091d30 0x7f7f5c008240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:31.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:31.466+0000 7f7f7affd700 1 --2- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f5c003500 0x7f7f5c005990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:31.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:31.466+0000 7f7f7b7fe700 1 -- 172.21.15.16:0/3793125773 >> v2:172.21.15.16:3300/0 conn(0x7f7f7c091d30 msgr2=0x7f7f5c008240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-09-30T22:26:31.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:31.466+0000 7f7f7b7fe700 1 -- 172.21.15.16:0/3793125773 >> v2:172.21.15.16:3300/0 conn(0x7f7f7c091d30 msgr2=0x7f7f5c008240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:26:31.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:31.466+0000 7f7f7b7fe700 1 --2- 172.21.15.16:0/3793125773 >> v2:172.21.15.16:3300/0 conn(0x7f7f7c091d30 0x7f7f5c008240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-30T22:26:31.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:31.466+0000 7f7f7affd700 1 -- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f5c003500 msgr2=0x7f7f5c005990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-09-30T22:26:31.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:31.466+0000 7f7f7affd700 1 -- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f5c003500 msgr2=0x7f7f5c005990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:26:31.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:31.466+0000 7f7f7affd700 1 --2- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f5c003500 0x7f7f5c005990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-30T22:26:31.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:31.466+0000 7f7f7b7fe700 1 --2- 172.21.15.16:0/3793125773 >> v2:172.21.15.16:3300/0 conn(0x7f7f7c091d30 0x7f7f5c008240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2023-09-30T22:26:31.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:31.466+0000 7f7f7affd700 1 --2- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f5c003500 0x7f7f5c005990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2023-09-30T22:26:31.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:31.867+0000 7f7f7affd700 1 --2- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f5c003500 0x7f7f5c005990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:31.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:31.867+0000 7f7f7b7fe700 1 --2- 172.21.15.16:0/3793125773 >> v2:172.21.15.16:3300/0 conn(0x7f7f7c091d30 0x7f7f5c008240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:31.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:31.867+0000 7f7f7b7fe700 1 -- 172.21.15.16:0/3793125773 >> v2:172.21.15.16:3300/0 conn(0x7f7f7c091d30 msgr2=0x7f7f5c008240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-09-30T22:26:31.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:31.867+0000 7f7f7b7fe700 1 -- 172.21.15.16:0/3793125773 >> v2:172.21.15.16:3300/0 conn(0x7f7f7c091d30 msgr2=0x7f7f5c008240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:26:31.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:31.867+0000 7f7f7affd700 1 -- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f5c003500 msgr2=0x7f7f5c005990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-09-30T22:26:31.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:31.867+0000 7f7f7affd700 1 -- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f5c003500 msgr2=0x7f7f5c005990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:26:31.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:31.867+0000 7f7f7b7fe700 1 --2- 172.21.15.16:0/3793125773 >> v2:172.21.15.16:3300/0 conn(0x7f7f7c091d30 0x7f7f5c008240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-30T22:26:31.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:31.867+0000 7f7f7affd700 1 --2- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f5c003500 0x7f7f5c005990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-30T22:26:31.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:31.867+0000 7f7f7b7fe700 1 --2- 172.21.15.16:0/3793125773 >> v2:172.21.15.16:3300/0 conn(0x7f7f7c091d30 0x7f7f5c008240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2023-09-30T22:26:31.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:31.867+0000 7f7f7affd700 1 --2- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f5c003500 0x7f7f5c005990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2023-09-30T22:26:32.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:32.668+0000 7f7f7b7fe700 1 --2- 172.21.15.16:0/3793125773 >> v2:172.21.15.16:3300/0 conn(0x7f7f7c091d30 0x7f7f5c008240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:32.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:32.668+0000 7f7f7affd700 1 --2- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f5c003500 0x7f7f5c005990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:32.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:32.668+0000 7f7f7b7fe700 1 -- 172.21.15.16:0/3793125773 >> v2:172.21.15.16:3300/0 conn(0x7f7f7c091d30 msgr2=0x7f7f5c008240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-09-30T22:26:32.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:32.668+0000 7f7f7b7fe700 1 -- 172.21.15.16:0/3793125773 >> v2:172.21.15.16:3300/0 conn(0x7f7f7c091d30 msgr2=0x7f7f5c008240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:26:32.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:32.668+0000 7f7f7b7fe700 1 --2- 172.21.15.16:0/3793125773 >> v2:172.21.15.16:3300/0 conn(0x7f7f7c091d30 0x7f7f5c008240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-30T22:26:32.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:32.668+0000 7f7f7affd700 1 -- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f5c003500 msgr2=0x7f7f5c005990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-09-30T22:26:32.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:32.668+0000 7f7f7affd700 1 -- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f5c003500 msgr2=0x7f7f5c005990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:26:32.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:32.668+0000 7f7f7affd700 1 --2- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f5c003500 0x7f7f5c005990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-30T22:26:32.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:32.668+0000 7f7f7b7fe700 1 --2- 172.21.15.16:0/3793125773 >> v2:172.21.15.16:3300/0 conn(0x7f7f7c091d30 0x7f7f5c008240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2023-09-30T22:26:32.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:32.668+0000 7f7f7affd700 1 --2- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f5c003500 0x7f7f5c005990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2023-09-30T22:26:34.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:34.264+0000 7f7f797fa700 1 -- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f5c003500 msgr2=0x7f7f5c005990 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T22:26:34.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:34.264+0000 7f7f797fa700 1 --2- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f5c003500 0x7f7f5c005990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:34.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:34.264+0000 7f7f797fa700 1 -- 172.21.15.16:0/3793125773 >> v2:172.21.15.16:3300/0 conn(0x7f7f7c091d30 msgr2=0x7f7f5c008240 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T22:26:34.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:34.264+0000 7f7f797fa700 1 --2- 172.21.15.16:0/3793125773 >> v2:172.21.15.16:3300/0 conn(0x7f7f7c091d30 0x7f7f5c008240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:34.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:34.264+0000 7f7f797fa700 1 -- 172.21.15.16:0/3793125773 >> v1:172.21.15.16:6789/0 conn(0x7f7f7c096b10 legacy=0x7f7f5c00a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:34.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:34.265+0000 7f7f797fa700 1 --2- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f7c12e460 0x7f7f5c00c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:34.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:34.265+0000 7f7f797fa700 1 -- 172.21.15.16:0/3793125773 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c092110 con 0x7f7f7c12e460 2023-09-30T22:26:34.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:34.265+0000 7f7f7b7fe700 1 --2- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f7c12e460 0x7f7f5c00c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:34.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:34.265+0000 7f7f7b7fe700 1 -- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f7c12e460 msgr2=0x7f7f5c00c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-09-30T22:26:34.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:34.265+0000 7f7f7b7fe700 1 -- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f7c12e460 msgr2=0x7f7f5c00c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:26:34.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:34.265+0000 7f7f7b7fe700 1 --2- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f7c12e460 0x7f7f5c00c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-30T22:26:34.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:34.265+0000 7f7f7b7fe700 1 --2- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f7c12e460 0x7f7f5c00c0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2023-09-30T22:26:34.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:34.466+0000 7f7f7b7fe700 1 --2- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f7c12e460 0x7f7f5c00c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:34.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:34.466+0000 7f7f7b7fe700 1 -- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f7c12e460 msgr2=0x7f7f5c00c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-09-30T22:26:34.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:34.466+0000 7f7f7b7fe700 1 -- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f7c12e460 msgr2=0x7f7f5c00c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:26:34.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:34.466+0000 7f7f7b7fe700 1 --2- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f7c12e460 0x7f7f5c00c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-30T22:26:34.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:34.466+0000 7f7f7b7fe700 1 --2- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f7c12e460 0x7f7f5c00c0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2023-09-30T22:26:34.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:34.867+0000 7f7f7b7fe700 1 --2- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f7c12e460 0x7f7f5c00c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:34.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:34.867+0000 7f7f7b7fe700 1 -- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f7c12e460 msgr2=0x7f7f5c00c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-09-30T22:26:34.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:34.867+0000 7f7f7b7fe700 1 -- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f7c12e460 msgr2=0x7f7f5c00c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:26:34.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:34.867+0000 7f7f7b7fe700 1 --2- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f7c12e460 0x7f7f5c00c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-09-30T22:26:34.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:34.867+0000 7f7f7b7fe700 1 --2- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f7c12e460 0x7f7f5c00c0d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2023-09-30T22:26:35.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:35.668+0000 7f7f7b7fe700 1 --2- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f7c12e460 0x7f7f5c00c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:35.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:35.668+0000 7f7f7b7fe700 1 -- 172.21.15.16:0/3793125773 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f5c00d060 con 0x7f7f7c12e460 2023-09-30T22:26:35.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:35.668+0000 7f7f7b7fe700 1 --2- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f7c12e460 0x7f7f5c00c0d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f7f6c019b50 tx=0x7f7f6c008040).ready entity=mon.2 client_cookie=b549514eebcfb4fb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:35.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:35.668+0000 7f7f7a7fc700 1 -- 172.21.15.16:0/3793125773 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c01e040 con 0x7f7f7c12e460 2023-09-30T22:26:35.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:35.668+0000 7f7f7a7fc700 1 -- 172.21.15.16:0/3793125773 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f6c017960 con 0x7f7f7c12e460 2023-09-30T22:26:35.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:35.668+0000 7f7f7a7fc700 1 -- 172.21.15.16:0/3793125773 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c017ce0 con 0x7f7f7c12e460 2023-09-30T22:26:35.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:35.669+0000 7f7f81f18700 1 -- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f7c12e460 msgr2=0x7f7f5c00c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:35.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:35.669+0000 7f7f81f18700 1 --2- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f7c12e460 0x7f7f5c00c0d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f7f6c019b50 tx=0x7f7f6c008040).stop 2023-09-30T22:26:35.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:35.669+0000 7f7f81f18700 1 -- 172.21.15.16:0/3793125773 shutdown_connections 2023-09-30T22:26:35.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:35.669+0000 7f7f81f18700 1 --2- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f5c003500 0x7f7f5c005990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:35.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:35.669+0000 7f7f81f18700 1 --2- 172.21.15.16:0/3793125773 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f7c12e460 0x7f7f5c00c0d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:35.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:35.669+0000 7f7f81f18700 1 --2- 172.21.15.16:0/3793125773 >> v2:172.21.15.16:3300/0 conn(0x7f7f7c091d30 0x7f7f5c008240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:35.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:35.669+0000 7f7f81f18700 1 -- 172.21.15.16:0/3793125773 >> 172.21.15.16:0/3793125773 conn(0x7f7f7c098720 msgr2=0x7f7f7c098b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:35.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:35.669+0000 7f7f81f18700 1 -- 172.21.15.16:0/3793125773 shutdown_connections 2023-09-30T22:26:35.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:35.669+0000 7f7f81f18700 1 -- 172.21.15.16:0/3793125773 wait complete. 2023-09-30T22:26:35.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:35.671+0000 7f7f81f18700 1 Processor -- start 2023-09-30T22:26:35.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:35.671+0000 7f7f81f18700 1 -- start start 2023-09-30T22:26:35.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:35.671+0000 7f7f81f18700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f5c003500 0x7f7f7c064540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:35.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:35.671+0000 7f7f81f18700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f7c091d30 0x7f7f7c064a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:35.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:35.671+0000 7f7f81f18700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f7c096b10 0x7f7f7c064f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:35.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:35.671+0000 7f7f81f18700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c065410 con 0x7f7f5c003500 2023-09-30T22:26:35.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:35.671+0000 7f7f81f18700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c0659f0 con 0x7f7f7c096b10 2023-09-30T22:26:35.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:35.671+0000 7f7f81f18700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c11d6b0 con 0x7f7f7c091d30 2023-09-30T22:26:35.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:35.672+0000 7f7f7affd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f7c091d30 0x7f7f7c064a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:35.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:35.672+0000 7f7f7b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f5c003500 0x7f7f7c064540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:35.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:35.672+0000 7f7f7affd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f7c091d30 0x7f7f7c064a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:42228/0 (socket says 172.21.15.16:42228) 2023-09-30T22:26:35.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:35.672+0000 7f7f7affd700 1 -- 172.21.15.16:0/1073461879 learned_addr learned my addr 172.21.15.16:0/1073461879 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:26:35.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:35.672+0000 7f7f7bfff700 1 --2- 172.21.15.16:0/1073461879 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f7c096b10 0x7f7f7c064f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:35.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:35.672+0000 7f7f7affd700 1 -- 172.21.15.16:0/1073461879 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f7c096b10 msgr2=0x7f7f7c064f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:35.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:35.672+0000 7f7f7affd700 1 --2- 172.21.15.16:0/1073461879 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f7c096b10 0x7f7f7c064f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:35.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:35.672+0000 7f7f7affd700 1 -- 172.21.15.16:0/1073461879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f5c003500 msgr2=0x7f7f7c064540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:35.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:35.672+0000 7f7f7affd700 1 --2- 172.21.15.16:0/1073461879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f5c003500 0x7f7f7c064540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:35.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:35.672+0000 7f7f7affd700 1 -- 172.21.15.16:0/1073461879 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f6c00e040 con 0x7f7f7c091d30 2023-09-30T22:26:35.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:35.672+0000 7f7f7b7fe700 1 --2- 172.21.15.16:0/1073461879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f5c003500 0x7f7f7c064540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:35.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:35.672+0000 7f7f7bfff700 1 --2- 172.21.15.16:0/1073461879 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f7c096b10 0x7f7f7c064f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:35.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:35.672+0000 7f7f7affd700 1 --2- 172.21.15.16:0/1073461879 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f7c091d30 0x7f7f7c064a30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f7f64005df0 tx=0x7f7f64009040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:35.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:35.672+0000 7f7f737fe700 1 -- 172.21.15.16:0/1073461879 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f64001be0 con 0x7f7f7c091d30 2023-09-30T22:26:35.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:35.672+0000 7f7f737fe700 1 -- 172.21.15.16:0/1073461879 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f64001d40 con 0x7f7f7c091d30 2023-09-30T22:26:35.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:35.672+0000 7f7f81f18700 1 -- 172.21.15.16:0/1073461879 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f7c11d310 con 0x7f7f7c091d30 2023-09-30T22:26:35.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:35.673+0000 7f7f737fe700 1 -- 172.21.15.16:0/1073461879 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f64017670 con 0x7f7f7c091d30 2023-09-30T22:26:35.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:35.673+0000 7f7f81f18700 1 -- 172.21.15.16:0/1073461879 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f7c11e330 con 0x7f7f7c091d30 2023-09-30T22:26:35.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:35.673+0000 7f7f737fe700 1 -- 172.21.15.16:0/1073461879 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f7f64017cc0 con 0x7f7f7c091d30 2023-09-30T22:26:35.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:35.673+0000 7f7f737fe700 1 -- 172.21.15.16:0/1073461879 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f7f64020c40 con 0x7f7f7c091d30 2023-09-30T22:26:35.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:35.674+0000 7f7f81f18700 1 -- 172.21.15.16:0/1073461879 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f7c099ae0 con 0x7f7f7c091d30 2023-09-30T22:26:35.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:35.676+0000 7f7f737fe700 1 -- 172.21.15.16:0/1073461879 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f7f64020400 con 0x7f7f7c091d30 2023-09-30T22:26:35.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:35.840+0000 7f7f81f18700 1 -- 172.21.15.16:0/1073461879 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f7f7c11db60 con 0x7f7f7c091d30 2023-09-30T22:26:35.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:35.843+0000 7f7f737fe700 1 -- 172.21.15.16:0/1073461879 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f7f6402a430 con 0x7f7f7c091d30 2023-09-30T22:26:35.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:35.845+0000 7f7f81f18700 1 -- 172.21.15.16:0/1073461879 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f7c091d30 msgr2=0x7f7f7c064a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:35.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:35.845+0000 7f7f81f18700 1 --2- 172.21.15.16:0/1073461879 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f7c091d30 0x7f7f7c064a30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f7f64005df0 tx=0x7f7f64009040).stop 2023-09-30T22:26:35.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:35.845+0000 7f7f81f18700 1 -- 172.21.15.16:0/1073461879 shutdown_connections 2023-09-30T22:26:35.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:35.845+0000 7f7f81f18700 1 --2- 172.21.15.16:0/1073461879 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f7c096b10 0x7f7f7c064f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:35.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:35.845+0000 7f7f81f18700 1 --2- 172.21.15.16:0/1073461879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f5c003500 0x7f7f7c064540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:35.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:35.845+0000 7f7f81f18700 1 --2- 172.21.15.16:0/1073461879 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f7c091d30 0x7f7f7c064a30 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:35.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:35.845+0000 7f7f81f18700 1 -- 172.21.15.16:0/1073461879 >> 172.21.15.16:0/1073461879 conn(0x7f7f7c098720 msgr2=0x7f7f7c069390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:35.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:35.845+0000 7f7f81f18700 1 -- 172.21.15.16:0/1073461879 shutdown_connections 2023-09-30T22:26:35.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:35.845+0000 7f7f81f18700 1 -- 172.21.15.16:0/1073461879 wait complete. 2023-09-30T22:26:35.847 INFO:teuthology.orchestra.run.smithi016.stderr:adjusted tunables profile to default 2023-09-30T22:26:35.859 INFO:tasks.ceph:check_enable_crimson: False 2023-09-30T22:26:35.859 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-09-30T22:26:35.859 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-09-30T22:26:35.859 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-09-30T22:26:35.870 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:26:35.870+0000 7f2f069c6680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-30T22:26:35.889 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-09-30T22:26:35.890 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-09-30T22:26:35.954 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:26:35.954+0000 7f2f069c6680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-30T22:26:35.962 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-09-30T22:26:35.963 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-09-30T22:26:36.034 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-09-30T22:26:36.034 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-09-30T22:26:36.050 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:26:36.050+0000 7f2f069c6680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-30T22:26:36.079 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph osd new 19946985-0ee2-487f-9347-929670dbe5ff 0 2023-09-30T22:26:36.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.246+0000 7f29f8051700 1 Processor -- start 2023-09-30T22:26:36.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.247+0000 7f29f8051700 1 -- start start 2023-09-30T22:26:36.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.247+0000 7f29f8051700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f29f0095b00 0x7f29f0095ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:36.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.247+0000 7f29f8051700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f29f012d1d0 0x7f29f012f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:36.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.247+0000 7f29f8051700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29f0096fe0 con 0x7f29f00937b0 2023-09-30T22:26:36.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.247+0000 7f29f8051700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29f0093b90 con 0x7f29f012d1d0 2023-09-30T22:26:36.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.247+0000 7f29f8051700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f29f0094f40 con 0x7f29f0095b00 2023-09-30T22:26:36.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.247+0000 7f29f65ee700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f29f012d1d0 0x7f29f012f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:36.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.247+0000 7f29f55ec700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f29f0095b00 0x7f29f0095ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:36.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.247+0000 7f29f5ded700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f29f00937b0 0x7f29f00953f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34860/0 (socket says 172.21.15.16:34860) 2023-09-30T22:26:36.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.247+0000 7f29f5ded700 1 -- 172.21.15.16:0/4159454018 learned_addr learned my addr 172.21.15.16:0/4159454018 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:26:36.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.248+0000 7f29f55ec700 1 -- 172.21.15.16:0/4159454018 >> v2:172.21.15.16:3300/0 conn(0x7f29f012d1d0 msgr2=0x7f29f012f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:36.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.248+0000 7f29f55ec700 1 --2- 172.21.15.16:0/4159454018 >> v2:172.21.15.16:3300/0 conn(0x7f29f012d1d0 0x7f29f012f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:36.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.248+0000 7f29f55ec700 1 -- 172.21.15.16:0/4159454018 >> v1:172.21.15.16:6789/0 conn(0x7f29f00937b0 legacy=0x7f29f00953f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:36.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.248+0000 7f29f4deb700 1 -- 172.21.15.16:0/4159454018 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1687076737 0 0) 0x7f29f0096fe0 con 0x7f29f00937b0 2023-09-30T22:26:36.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.248+0000 7f29f55ec700 1 -- 172.21.15.16:0/4159454018 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29f0094b10 con 0x7f29f0095b00 2023-09-30T22:26:36.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.248+0000 7f29f65ee700 1 --2- 172.21.15.16:0/4159454018 >> v2:172.21.15.16:3300/0 conn(0x7f29f012d1d0 0x7f29f012f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:36.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.248+0000 7f29f55ec700 1 --2- 172.21.15.16:0/4159454018 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f29f0095b00 0x7f29f0095ee0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f29e0009790 tx=0x7f29e0009060).ready entity=mon.2 client_cookie=28846205746dd2f3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:36.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.248+0000 7f29f4deb700 1 -- 172.21.15.16:0/4159454018 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e000f510 con 0x7f29f0095b00 2023-09-30T22:26:36.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.248+0000 7f29f4deb700 1 -- 172.21.15.16:0/4159454018 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29e000f670 con 0x7f29f0095b00 2023-09-30T22:26:36.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.248+0000 7f29f4deb700 1 -- 172.21.15.16:0/4159454018 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e000f9f0 con 0x7f29f0095b00 2023-09-30T22:26:36.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.249+0000 7f29f8051700 1 -- 172.21.15.16:0/4159454018 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f29f0095b00 msgr2=0x7f29f0095ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:36.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.249+0000 7f29f8051700 1 --2- 172.21.15.16:0/4159454018 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f29f0095b00 0x7f29f0095ee0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f29e0009790 tx=0x7f29e0009060).stop 2023-09-30T22:26:36.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.249+0000 7f29f8051700 1 -- 172.21.15.16:0/4159454018 shutdown_connections 2023-09-30T22:26:36.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.249+0000 7f29f8051700 1 --2- 172.21.15.16:0/4159454018 >> v2:172.21.15.16:3300/0 conn(0x7f29f012d1d0 0x7f29f012f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:36.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.249+0000 7f29f8051700 1 --2- 172.21.15.16:0/4159454018 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f29f0095b00 0x7f29f0095ee0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:36.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.249+0000 7f29f8051700 1 -- 172.21.15.16:0/4159454018 >> 172.21.15.16:0/4159454018 conn(0x7f29f00974f0 msgr2=0x7f29f0097900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:36.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.249+0000 7f29f8051700 1 -- 172.21.15.16:0/4159454018 shutdown_connections 2023-09-30T22:26:36.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.249+0000 7f29f8051700 1 -- 172.21.15.16:0/4159454018 wait complete. 2023-09-30T22:26:36.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.250+0000 7f29f8051700 1 Processor -- start 2023-09-30T22:26:36.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.250+0000 7f29f8051700 1 -- start start 2023-09-30T22:26:36.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.250+0000 7f29f8051700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f29f00937b0 0x7f29f012bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:36.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.250+0000 7f29f8051700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29f0095b00 0x7f29f012c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:36.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.250+0000 7f29f8051700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f29f012d1d0 0x7f29f0116e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:36.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.250+0000 7f29f8051700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29f006a4b0 con 0x7f29f0095b00 2023-09-30T22:26:36.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.250+0000 7f29f8051700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29f011d9b0 con 0x7f29f012d1d0 2023-09-30T22:26:36.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.250+0000 7f29f8051700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f29f011e630 con 0x7f29f00937b0 2023-09-30T22:26:36.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.250+0000 7f29f5ded700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f29f00937b0 0x7f29f012bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:36.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.250+0000 7f29f5ded700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f29f00937b0 0x7f29f012bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:42254/0 (socket says 172.21.15.16:42254) 2023-09-30T22:26:36.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.250+0000 7f29f65ee700 1 --2- 172.21.15.16:0/867356303 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f29f012d1d0 0x7f29f0116e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:36.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.250+0000 7f29f5ded700 1 -- 172.21.15.16:0/867356303 learned_addr learned my addr 172.21.15.16:0/867356303 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:26:36.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.251+0000 7f29f55ec700 1 --2- 172.21.15.16:0/867356303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29f0095b00 0x7f29f012c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:36.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.251+0000 7f29f5ded700 1 -- 172.21.15.16:0/867356303 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f29f012d1d0 msgr2=0x7f29f0116e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:36.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.251+0000 7f29f5ded700 1 --2- 172.21.15.16:0/867356303 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f29f012d1d0 0x7f29f0116e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:36.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.251+0000 7f29f5ded700 1 -- 172.21.15.16:0/867356303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29f0095b00 msgr2=0x7f29f012c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:36.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.251+0000 7f29f5ded700 1 --2- 172.21.15.16:0/867356303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29f0095b00 0x7f29f012c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:36.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.251+0000 7f29f5ded700 1 -- 172.21.15.16:0/867356303 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e80092a0 con 0x7f29f00937b0 2023-09-30T22:26:36.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.251+0000 7f29f55ec700 1 --2- 172.21.15.16:0/867356303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29f0095b00 0x7f29f012c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:36.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.251+0000 7f29f65ee700 1 --2- 172.21.15.16:0/867356303 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f29f012d1d0 0x7f29f0116e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:36.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.251+0000 7f29f5ded700 1 --2- 172.21.15.16:0/867356303 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f29f00937b0 0x7f29f012bf10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f29ec00c810 tx=0x7f29ec007f90).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:36.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.251+0000 7f29e5ffb700 1 -- 172.21.15.16:0/867356303 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ec005d30 con 0x7f29f00937b0 2023-09-30T22:26:36.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.251+0000 7f29e5ffb700 1 -- 172.21.15.16:0/867356303 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29ec005e90 con 0x7f29f00937b0 2023-09-30T22:26:36.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.251+0000 7f29f8051700 1 -- 172.21.15.16:0/867356303 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29e00092a0 con 0x7f29f00937b0 2023-09-30T22:26:36.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.251+0000 7f29e5ffb700 1 -- 172.21.15.16:0/867356303 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ec007320 con 0x7f29f00937b0 2023-09-30T22:26:36.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.252+0000 7f29f8051700 1 -- 172.21.15.16:0/867356303 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29f012c8f0 con 0x7f29f00937b0 2023-09-30T22:26:36.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.252+0000 7f29e5ffb700 1 -- 172.21.15.16:0/867356303 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f29ec01d440 con 0x7f29f00937b0 2023-09-30T22:26:36.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.252+0000 7f29e5ffb700 1 -- 172.21.15.16:0/867356303 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7f29ec013760 con 0x7f29f00937b0 2023-09-30T22:26:36.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.253+0000 7f29f8051700 1 -- 172.21.15.16:0/867356303 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29f011d800 con 0x7f29f00937b0 2023-09-30T22:26:36.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.255+0000 7f29e5ffb700 1 -- 172.21.15.16:0/867356303 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f29ec007970 con 0x7f29f00937b0 2023-09-30T22:26:36.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.339+0000 7f29f8051700 1 -- 172.21.15.16:0/867356303 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd new", "uuid": "19946985-0ee2-487f-9347-929670dbe5ff", "id": 0} v 0) v1 -- 0x7f29f005a260 con 0x7f29f00937b0 2023-09-30T22:26:36.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.342+0000 7f29e5ffb700 1 -- 172.21.15.16:0/867356303 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "19946985-0ee2-487f-9347-929670dbe5ff", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f29ec01d920 con 0x7f29f00937b0 2023-09-30T22:26:36.343 INFO:teuthology.orchestra.run.smithi016.stdout:0 2023-09-30T22:26:36.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.343+0000 7f29f8051700 1 -- 172.21.15.16:0/867356303 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f29f00937b0 msgr2=0x7f29f012bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:36.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.343+0000 7f29f8051700 1 --2- 172.21.15.16:0/867356303 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f29f00937b0 0x7f29f012bf10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f29ec00c810 tx=0x7f29ec007f90).stop 2023-09-30T22:26:36.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.344+0000 7f29f8051700 1 -- 172.21.15.16:0/867356303 shutdown_connections 2023-09-30T22:26:36.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.344+0000 7f29f8051700 1 --2- 172.21.15.16:0/867356303 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f29f012d1d0 0x7f29f0116e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:36.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.344+0000 7f29f8051700 1 --2- 172.21.15.16:0/867356303 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f29f00937b0 0x7f29f012bf10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:36.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.344+0000 7f29f8051700 1 --2- 172.21.15.16:0/867356303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29f0095b00 0x7f29f012c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:36.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.344+0000 7f29f8051700 1 -- 172.21.15.16:0/867356303 >> 172.21.15.16:0/867356303 conn(0x7f29f00974f0 msgr2=0x7f29f0111030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:36.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.344+0000 7f29f8051700 1 -- 172.21.15.16:0/867356303 shutdown_connections 2023-09-30T22:26:36.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.344+0000 7f29f8051700 1 -- 172.21.15.16:0/867356303 wait complete. 2023-09-30T22:26:36.357 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph osd new 8efd4dbc-6ae4-4b82-b530-3b3eac2be445 1 2023-09-30T22:26:36.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.486+0000 7fb95e905700 1 Processor -- start 2023-09-30T22:26:36.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.486+0000 7fb95e905700 1 -- start start 2023-09-30T22:26:36.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.487+0000 7fb95e905700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb958096d30 0x7fb958097110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:36.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.487+0000 7fb95e905700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb95812e460 0x7fb958130850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:36.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.487+0000 7fb95e905700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb958093f70 con 0x7fb958091d30 2023-09-30T22:26:36.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.487+0000 7fb95e905700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb958092110 con 0x7fb95812e460 2023-09-30T22:26:36.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.487+0000 7fb95e905700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb9580927f0 con 0x7fb958096d30 2023-09-30T22:26:36.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.487+0000 7fb95cea2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb95812e460 0x7fb958130850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:36.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.487+0000 7fb9577fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb958096d30 0x7fb958097110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:36.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.487+0000 7fb957fff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb958091d30 0x7fb958096620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34864/0 (socket says 172.21.15.16:34864) 2023-09-30T22:26:36.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.487+0000 7fb95cea2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb95812e460 0x7fb958130850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:45850/0 (socket says 172.21.15.16:45850) 2023-09-30T22:26:36.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.487+0000 7fb957fff700 1 -- 172.21.15.16:0/836680676 learned_addr learned my addr 172.21.15.16:0/836680676 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:26:36.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.487+0000 7fb956ffd700 1 -- 172.21.15.16:0/836680676 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1806132815 0 0) 0x7fb958093f70 con 0x7fb958091d30 2023-09-30T22:26:36.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.488+0000 7fb95cea2700 1 -- 172.21.15.16:0/836680676 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb958096d30 msgr2=0x7fb958097110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:36.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.488+0000 7fb95cea2700 1 --2- 172.21.15.16:0/836680676 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb958096d30 0x7fb958097110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:36.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.488+0000 7fb95cea2700 1 -- 172.21.15.16:0/836680676 >> v1:172.21.15.16:6789/0 conn(0x7fb958091d30 legacy=0x7fb958096620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:36.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.488+0000 7fb95cea2700 1 -- 172.21.15.16:0/836680676 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb958095c40 con 0x7fb95812e460 2023-09-30T22:26:36.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.488+0000 7fb9577fe700 1 --2- 172.21.15.16:0/836680676 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb958096d30 0x7fb958097110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:36.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.488+0000 7fb95cea2700 1 --2- 172.21.15.16:0/836680676 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb95812e460 0x7fb958130850 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7fb950009e90 tx=0x7fb950009780).ready entity=mon.1 client_cookie=26ca34ad2f686c18 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:36.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.488+0000 7fb956ffd700 1 -- 172.21.15.16:0/836680676 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95000e070 con 0x7fb95812e460 2023-09-30T22:26:36.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.488+0000 7fb956ffd700 1 -- 172.21.15.16:0/836680676 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb95000bb60 con 0x7fb95812e460 2023-09-30T22:26:36.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.488+0000 7fb956ffd700 1 -- 172.21.15.16:0/836680676 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb950012410 con 0x7fb95812e460 2023-09-30T22:26:36.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.489+0000 7fb95e905700 1 -- 172.21.15.16:0/836680676 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb95812e460 msgr2=0x7fb958130850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:36.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.489+0000 7fb95e905700 1 --2- 172.21.15.16:0/836680676 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb95812e460 0x7fb958130850 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7fb950009e90 tx=0x7fb950009780).stop 2023-09-30T22:26:36.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.489+0000 7fb95e905700 1 -- 172.21.15.16:0/836680676 shutdown_connections 2023-09-30T22:26:36.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.489+0000 7fb95e905700 1 --2- 172.21.15.16:0/836680676 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb95812e460 0x7fb958130850 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:36.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.489+0000 7fb95e905700 1 --2- 172.21.15.16:0/836680676 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb958096d30 0x7fb958097110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:36.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.489+0000 7fb95e905700 1 -- 172.21.15.16:0/836680676 >> 172.21.15.16:0/836680676 conn(0x7fb958098720 msgr2=0x7fb958098b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:36.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.489+0000 7fb95e905700 1 -- 172.21.15.16:0/836680676 shutdown_connections 2023-09-30T22:26:36.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.489+0000 7fb95e905700 1 -- 172.21.15.16:0/836680676 wait complete. 2023-09-30T22:26:36.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.490+0000 7fb95e905700 1 Processor -- start 2023-09-30T22:26:36.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.490+0000 7fb95e905700 1 -- start start 2023-09-30T22:26:36.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.491+0000 7fb95e905700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb958091d30 0x7fb958118130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:36.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.491+0000 7fb95e905700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb958096d30 0x7fb958118620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:36.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.491+0000 7fb95e905700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb95812e460 0x7fb958118b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:36.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.491+0000 7fb95e905700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb958119000 con 0x7fb95812e460 2023-09-30T22:26:36.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.491+0000 7fb95e905700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9581195a0 con 0x7fb958096d30 2023-09-30T22:26:36.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.491+0000 7fb95e905700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb95811b620 con 0x7fb958091d30 2023-09-30T22:26:36.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.491+0000 7fb9577fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb958096d30 0x7fb958118620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:36.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.491+0000 7fb9577fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb958096d30 0x7fb958118620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:45860/0 (socket says 172.21.15.16:45860) 2023-09-30T22:26:36.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.491+0000 7fb9577fe700 1 -- 172.21.15.16:0/1018124038 learned_addr learned my addr 172.21.15.16:0/1018124038 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:26:36.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.492+0000 7fb957fff700 1 --2- 172.21.15.16:0/1018124038 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb958091d30 0x7fb958118130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:36.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.492+0000 7fb9577fe700 1 -- 172.21.15.16:0/1018124038 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb958091d30 msgr2=0x7fb958118130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:36.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.492+0000 7fb9577fe700 1 --2- 172.21.15.16:0/1018124038 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb958091d30 0x7fb958118130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:36.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.492+0000 7fb9577fe700 1 -- 172.21.15.16:0/1018124038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb95812e460 msgr2=0x7fb958118b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:36.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.492+0000 7fb95cea2700 1 --2- 172.21.15.16:0/1018124038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb95812e460 0x7fb958118b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:36.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.492+0000 7fb9577fe700 1 --2- 172.21.15.16:0/1018124038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb95812e460 0x7fb958118b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:36.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.492+0000 7fb9577fe700 1 -- 172.21.15.16:0/1018124038 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb94c0092a0 con 0x7fb958096d30 2023-09-30T22:26:36.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.492+0000 7fb95cea2700 1 --2- 172.21.15.16:0/1018124038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb95812e460 0x7fb958118b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:26:36.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.492+0000 7fb9577fe700 1 --2- 172.21.15.16:0/1018124038 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb958096d30 0x7fb958118620 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7fb94c00ea20 tx=0x7fb94c009060).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:36.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.494+0000 7fb93ffff700 1 -- 172.21.15.16:0/1018124038 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94c00c850 con 0x7fb958096d30 2023-09-30T22:26:36.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.494+0000 7fb93ffff700 1 -- 172.21.15.16:0/1018124038 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb94c00c9b0 con 0x7fb958096d30 2023-09-30T22:26:36.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.494+0000 7fb95e905700 1 -- 172.21.15.16:0/1018124038 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb950009a20 con 0x7fb958096d30 2023-09-30T22:26:36.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.494+0000 7fb93ffff700 1 -- 172.21.15.16:0/1018124038 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94c00cd30 con 0x7fb958096d30 2023-09-30T22:26:36.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.494+0000 7fb95e905700 1 -- 172.21.15.16:0/1018124038 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb95806a710 con 0x7fb958096d30 2023-09-30T22:26:36.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.494+0000 7fb93ffff700 1 -- 172.21.15.16:0/1018124038 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7fb94c012860 con 0x7fb958096d30 2023-09-30T22:26:36.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.494+0000 7fb93ffff700 1 -- 172.21.15.16:0/1018124038 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7fb94c01b7c0 con 0x7fb958096d30 2023-09-30T22:26:36.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.495+0000 7fb95e905700 1 -- 172.21.15.16:0/1018124038 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb95811bb00 con 0x7fb958096d30 2023-09-30T22:26:36.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.498+0000 7fb93ffff700 1 -- 172.21.15.16:0/1018124038 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7fb94c03ae50 con 0x7fb958096d30 2023-09-30T22:26:36.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.582+0000 7fb95e905700 1 -- 172.21.15.16:0/1018124038 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd new", "uuid": "8efd4dbc-6ae4-4b82-b530-3b3eac2be445", "id": 1} v 0) v1 -- 0x7fb95805a260 con 0x7fb958096d30 2023-09-30T22:26:36.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.586+0000 7fb93ffff700 1 -- 172.21.15.16:0/1018124038 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8efd4dbc-6ae4-4b82-b530-3b3eac2be445", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb94c029070 con 0x7fb958096d30 2023-09-30T22:26:36.586 INFO:teuthology.orchestra.run.smithi016.stdout:1 2023-09-30T22:26:36.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.587+0000 7fb95e905700 1 -- 172.21.15.16:0/1018124038 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb958096d30 msgr2=0x7fb958118620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:36.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.587+0000 7fb95e905700 1 --2- 172.21.15.16:0/1018124038 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb958096d30 0x7fb958118620 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7fb94c00ea20 tx=0x7fb94c009060).stop 2023-09-30T22:26:36.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.587+0000 7fb95e905700 1 -- 172.21.15.16:0/1018124038 shutdown_connections 2023-09-30T22:26:36.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.587+0000 7fb95e905700 1 --2- 172.21.15.16:0/1018124038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb95812e460 0x7fb958118b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:36.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.587+0000 7fb95e905700 1 --2- 172.21.15.16:0/1018124038 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb958091d30 0x7fb958118130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:36.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.587+0000 7fb95e905700 1 --2- 172.21.15.16:0/1018124038 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb958096d30 0x7fb958118620 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:36.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.587+0000 7fb95e905700 1 -- 172.21.15.16:0/1018124038 >> 172.21.15.16:0/1018124038 conn(0x7fb958098720 msgr2=0x7fb9581122c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:36.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.587+0000 7fb95e905700 1 -- 172.21.15.16:0/1018124038 shutdown_connections 2023-09-30T22:26:36.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.587+0000 7fb95e905700 1 -- 172.21.15.16:0/1018124038 wait complete. 2023-09-30T22:26:36.600 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph osd new b47bb1ce-27af-4c0c-868a-b76ebd0d9499 2 2023-09-30T22:26:36.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.729+0000 7f129fd3a700 1 Processor -- start 2023-09-30T22:26:36.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.729+0000 7f129fd3a700 1 -- start start 2023-09-30T22:26:36.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.729+0000 7f129fd3a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1298068590 0x7f1298067870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:36.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.729+0000 7f129fd3a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f129812e080 0x7f1298130470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:36.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.729+0000 7f129fd3a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1298069580 con 0x7f1298066d80 2023-09-30T22:26:36.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.729+0000 7f129fd3a700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1298094430 con 0x7f129812e080 2023-09-30T22:26:36.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.729+0000 7f129fd3a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1298094050 con 0x7f1298068590 2023-09-30T22:26:36.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.730+0000 7f129e2d7700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f129812e080 0x7f1298130470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:36.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.730+0000 7f129d2d5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1298068590 0x7f1298067870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:36.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.730+0000 7f129e2d7700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f129812e080 0x7f1298130470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44514/0 (socket says 172.21.15.16:44514) 2023-09-30T22:26:36.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.730+0000 7f129dad6700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1298066d80 0x7f1298067160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34866/0 (socket says 172.21.15.16:34866) 2023-09-30T22:26:36.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.730+0000 7f129e2d7700 1 -- 172.21.15.16:0/278416380 learned_addr learned my addr 172.21.15.16:0/278416380 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:26:36.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.730+0000 7f129e2d7700 1 -- 172.21.15.16:0/278416380 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1298068590 msgr2=0x7f1298067870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:36.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.730+0000 7f129e2d7700 1 --2- 172.21.15.16:0/278416380 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1298068590 0x7f1298067870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:36.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.730+0000 7f129e2d7700 1 -- 172.21.15.16:0/278416380 >> v1:172.21.15.16:6789/0 conn(0x7f1298066d80 legacy=0x7f1298067160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:36.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.730+0000 7f129cad4700 1 -- 172.21.15.16:0/278416380 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 847254550 0 0) 0x7f1298069580 con 0x7f1298066d80 2023-09-30T22:26:36.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.730+0000 7f129e2d7700 1 -- 172.21.15.16:0/278416380 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1298069840 con 0x7f129812e080 2023-09-30T22:26:36.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.730+0000 7f129e2d7700 1 --2- 172.21.15.16:0/278416380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f129812e080 0x7f1298130470 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f1290009f20 tx=0x7f1290009840).ready entity=mon.0 client_cookie=c1ced58edd14fdfa server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:36.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.730+0000 7f129cad4700 1 -- 172.21.15.16:0/278416380 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129000e070 con 0x7f129812e080 2023-09-30T22:26:36.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.730+0000 7f129cad4700 1 -- 172.21.15.16:0/278416380 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f129000b9d0 con 0x7f129812e080 2023-09-30T22:26:36.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.730+0000 7f129cad4700 1 -- 172.21.15.16:0/278416380 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129000bd50 con 0x7f129812e080 2023-09-30T22:26:36.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.731+0000 7f129fd3a700 1 -- 172.21.15.16:0/278416380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f129812e080 msgr2=0x7f1298130470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:36.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.731+0000 7f129fd3a700 1 --2- 172.21.15.16:0/278416380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f129812e080 0x7f1298130470 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f1290009f20 tx=0x7f1290009840).stop 2023-09-30T22:26:36.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.731+0000 7f129fd3a700 1 -- 172.21.15.16:0/278416380 shutdown_connections 2023-09-30T22:26:36.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.731+0000 7f129fd3a700 1 --2- 172.21.15.16:0/278416380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f129812e080 0x7f1298130470 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:36.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.731+0000 7f129fd3a700 1 --2- 172.21.15.16:0/278416380 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1298068590 0x7f1298067870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:36.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.731+0000 7f129fd3a700 1 -- 172.21.15.16:0/278416380 >> 172.21.15.16:0/278416380 conn(0x7f129809a3d0 msgr2=0x7f129809a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:36.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.731+0000 7f129fd3a700 1 -- 172.21.15.16:0/278416380 shutdown_connections 2023-09-30T22:26:36.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.731+0000 7f129fd3a700 1 -- 172.21.15.16:0/278416380 wait complete. 2023-09-30T22:26:36.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.733+0000 7f129fd3a700 1 Processor -- start 2023-09-30T22:26:36.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.733+0000 7f129fd3a700 1 -- start start 2023-09-30T22:26:36.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.734+0000 7f129fd3a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1298066d80 0x7f12981152e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:36.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.734+0000 7f129fd3a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1298068590 0x7f12981113e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:36.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.734+0000 7f129fd3a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f129812e080 0x7f12981118d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:36.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.734+0000 7f129fd3a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1298129480 con 0x7f1298068590 2023-09-30T22:26:36.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.734+0000 7f129fd3a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1298099bf0 con 0x7f129812e080 2023-09-30T22:26:36.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.734+0000 7f129fd3a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f12980690f0 con 0x7f1298066d80 2023-09-30T22:26:36.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.734+0000 7f129dad6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1298066d80 0x7f12981152e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:36.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.734+0000 7f129e2d7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f129812e080 0x7f12981118d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:36.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.734+0000 7f129e2d7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f129812e080 0x7f12981118d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:45882/0 (socket says 172.21.15.16:45882) 2023-09-30T22:26:36.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.734+0000 7f129e2d7700 1 -- 172.21.15.16:0/3728564614 learned_addr learned my addr 172.21.15.16:0/3728564614 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:26:36.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.734+0000 7f129d2d5700 1 --2- 172.21.15.16:0/3728564614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1298068590 0x7f12981113e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:36.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.734+0000 7f129dad6700 1 -- 172.21.15.16:0/3728564614 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f129812e080 msgr2=0x7f12981118d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:36.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.734+0000 7f129dad6700 1 --2- 172.21.15.16:0/3728564614 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f129812e080 0x7f12981118d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:36.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.734+0000 7f129dad6700 1 -- 172.21.15.16:0/3728564614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1298068590 msgr2=0x7f12981113e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:36.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.734+0000 7f129dad6700 1 --2- 172.21.15.16:0/3728564614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1298068590 0x7f12981113e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:36.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.734+0000 7f129dad6700 1 -- 172.21.15.16:0/3728564614 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1290009ae0 con 0x7f1298066d80 2023-09-30T22:26:36.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.734+0000 7f129d2d5700 1 --2- 172.21.15.16:0/3728564614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1298068590 0x7f12981113e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:36.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.734+0000 7f129e2d7700 1 --2- 172.21.15.16:0/3728564614 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f129812e080 0x7f12981118d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:36.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.734+0000 7f129dad6700 1 --2- 172.21.15.16:0/3728564614 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1298066d80 0x7f12981152e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f129400c3f0 tx=0x7f1294007bb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:36.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.734+0000 7f128dffb700 1 -- 172.21.15.16:0/3728564614 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12940070d0 con 0x7f1298066d80 2023-09-30T22:26:36.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.735+0000 7f128dffb700 1 -- 172.21.15.16:0/3728564614 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1294007250 con 0x7f1298066d80 2023-09-30T22:26:36.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.735+0000 7f129fd3a700 1 -- 172.21.15.16:0/3728564614 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1298112850 con 0x7f1298066d80 2023-09-30T22:26:36.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.735+0000 7f128dffb700 1 -- 172.21.15.16:0/3728564614 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12940056e0 con 0x7f1298066d80 2023-09-30T22:26:36.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.735+0000 7f129fd3a700 1 -- 172.21.15.16:0/3728564614 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1298129a00 con 0x7f1298066d80 2023-09-30T22:26:36.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.735+0000 7f128dffb700 1 -- 172.21.15.16:0/3728564614 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f1294005cf0 con 0x7f1298066d80 2023-09-30T22:26:36.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.735+0000 7f128dffb700 1 -- 172.21.15.16:0/3728564614 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7f1294013bf0 con 0x7f1298066d80 2023-09-30T22:26:36.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.736+0000 7f129fd3a700 1 -- 172.21.15.16:0/3728564614 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f129805a260 con 0x7f1298066d80 2023-09-30T22:26:36.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.740+0000 7f128dffb700 1 -- 172.21.15.16:0/3728564614 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f1294013e30 con 0x7f1298066d80 2023-09-30T22:26:36.866 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:26:36.866+0000 7f2f069c6680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-30T22:26:36.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.872+0000 7f129fd3a700 1 -- 172.21.15.16:0/3728564614 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd new", "uuid": "b47bb1ce-27af-4c0c-868a-b76ebd0d9499", "id": 2} v 0) v1 -- 0x7f1298046c20 con 0x7f1298066d80 2023-09-30T22:26:36.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.875+0000 7f128dffb700 1 -- 172.21.15.16:0/3728564614 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b47bb1ce-27af-4c0c-868a-b76ebd0d9499", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f12940137c0 con 0x7f1298066d80 2023-09-30T22:26:36.876 INFO:teuthology.orchestra.run.smithi016.stdout:2 2023-09-30T22:26:36.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.876+0000 7f129fd3a700 1 -- 172.21.15.16:0/3728564614 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1298066d80 msgr2=0x7f12981152e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:36.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.876+0000 7f129fd3a700 1 --2- 172.21.15.16:0/3728564614 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1298066d80 0x7f12981152e0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f129400c3f0 tx=0x7f1294007bb0).stop 2023-09-30T22:26:36.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.877+0000 7f129fd3a700 1 -- 172.21.15.16:0/3728564614 shutdown_connections 2023-09-30T22:26:36.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.877+0000 7f129fd3a700 1 --2- 172.21.15.16:0/3728564614 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f129812e080 0x7f12981118d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:36.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.877+0000 7f129fd3a700 1 --2- 172.21.15.16:0/3728564614 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1298066d80 0x7f12981152e0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:36.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.877+0000 7f129fd3a700 1 --2- 172.21.15.16:0/3728564614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1298068590 0x7f12981113e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:36.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.877+0000 7f129fd3a700 1 -- 172.21.15.16:0/3728564614 >> 172.21.15.16:0/3728564614 conn(0x7f129809a3d0 msgr2=0x7f12980970b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:36.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.877+0000 7f129fd3a700 1 -- 172.21.15.16:0/3728564614 shutdown_connections 2023-09-30T22:26:36.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:36.877+0000 7f129fd3a700 1 -- 172.21.15.16:0/3728564614 wait complete. 2023-09-30T22:26:36.894 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph osd new 03b53d64-d10d-448a-a081-39ecceab589b 3 2023-09-30T22:26:36.998 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:26:36.998+0000 7f2f069c6680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-30T22:26:37.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.024+0000 7fa957a7f700 1 Processor -- start 2023-09-30T22:26:37.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.025+0000 7fa957a7f700 1 -- start start 2023-09-30T22:26:37.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.025+0000 7fa957a7f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa950067e90 0x7fa950068270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:37.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.025+0000 7fa957a7f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa950068760 0x7fa950127ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:37.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.025+0000 7fa957a7f700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9500690b0 con 0x7fa950128090 2023-09-30T22:26:37.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.025+0000 7fa957a7f700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9500920e0 con 0x7fa950067e90 2023-09-30T22:26:37.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.025+0000 7fa957a7f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa950092330 con 0x7fa950068760 2023-09-30T22:26:37.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.025+0000 7fa95501a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa950068760 0x7fa950127ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:37.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.025+0000 7fa95581b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa950067e90 0x7fa950068270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:37.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.025+0000 7fa95501a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa950068760 0x7fa950127ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:45894/0 (socket says 172.21.15.16:45894) 2023-09-30T22:26:37.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.025+0000 7fa95581b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa950067e90 0x7fa950068270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44534/0 (socket says 172.21.15.16:44534) 2023-09-30T22:26:37.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.025+0000 7fa95501a700 1 -- 172.21.15.16:0/379917870 learned_addr learned my addr 172.21.15.16:0/379917870 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:26:37.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.025+0000 7fa95601c700 1 --1- 172.21.15.16:0/379917870 >> v1:172.21.15.16:6789/0 conn(0x7fa950128090 0x7fa95012edf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34868/0 (socket says 172.21.15.16:34868) 2023-09-30T22:26:37.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.026+0000 7fa95501a700 1 -- 172.21.15.16:0/379917870 >> v2:172.21.15.16:3300/0 conn(0x7fa950067e90 msgr2=0x7fa950068270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:37.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.026+0000 7fa95501a700 1 --2- 172.21.15.16:0/379917870 >> v2:172.21.15.16:3300/0 conn(0x7fa950067e90 0x7fa950068270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:37.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.026+0000 7fa95501a700 1 -- 172.21.15.16:0/379917870 >> v1:172.21.15.16:6789/0 conn(0x7fa950128090 legacy=0x7fa95012edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:37.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.026+0000 7fa954819700 1 -- 172.21.15.16:0/379917870 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3953301569 0 0) 0x7fa9500690b0 con 0x7fa950128090 2023-09-30T22:26:37.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.026+0000 7fa95501a700 1 -- 172.21.15.16:0/379917870 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa950069b20 con 0x7fa950068760 2023-09-30T22:26:37.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.026+0000 7fa95581b700 1 --2- 172.21.15.16:0/379917870 >> v2:172.21.15.16:3300/0 conn(0x7fa950067e90 0x7fa950068270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:37.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.026+0000 7fa95501a700 1 --2- 172.21.15.16:0/379917870 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa950068760 0x7fa950127ba0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7fa940006fd0 tx=0x7fa9400090f0).ready entity=mon.1 client_cookie=9aaf8f468d12e817 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:37.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.026+0000 7fa954819700 1 -- 172.21.15.16:0/379917870 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94000b580 con 0x7fa950068760 2023-09-30T22:26:37.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.026+0000 7fa954819700 1 -- 172.21.15.16:0/379917870 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa94000b6e0 con 0x7fa950068760 2023-09-30T22:26:37.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.026+0000 7fa954819700 1 -- 172.21.15.16:0/379917870 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94000ba60 con 0x7fa950068760 2023-09-30T22:26:37.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.027+0000 7fa957a7f700 1 -- 172.21.15.16:0/379917870 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa950068760 msgr2=0x7fa950127ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:37.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.027+0000 7fa957a7f700 1 --2- 172.21.15.16:0/379917870 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa950068760 0x7fa950127ba0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7fa940006fd0 tx=0x7fa9400090f0).stop 2023-09-30T22:26:37.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.027+0000 7fa957a7f700 1 -- 172.21.15.16:0/379917870 shutdown_connections 2023-09-30T22:26:37.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.027+0000 7fa957a7f700 1 --2- 172.21.15.16:0/379917870 >> v2:172.21.15.16:3300/0 conn(0x7fa950067e90 0x7fa950068270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:37.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.027+0000 7fa957a7f700 1 --2- 172.21.15.16:0/379917870 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa950068760 0x7fa950127ba0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:37.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.027+0000 7fa957a7f700 1 -- 172.21.15.16:0/379917870 >> 172.21.15.16:0/379917870 conn(0x7fa95009a3d0 msgr2=0x7fa95009a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:37.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.027+0000 7fa957a7f700 1 -- 172.21.15.16:0/379917870 shutdown_connections 2023-09-30T22:26:37.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.027+0000 7fa957a7f700 1 -- 172.21.15.16:0/379917870 wait complete. 2023-09-30T22:26:37.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.028+0000 7fa957a7f700 1 Processor -- start 2023-09-30T22:26:37.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.028+0000 7fa957a7f700 1 -- start start 2023-09-30T22:26:37.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.028+0000 7fa957a7f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa950067e90 0x7fa950120b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:37.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.028+0000 7fa957a7f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa950068760 0x7fa950121020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:37.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.028+0000 7fa957a7f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa950128090 0x7fa950121510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:37.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.028+0000 7fa957a7f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa950121a00 con 0x7fa950068760 2023-09-30T22:26:37.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.028+0000 7fa957a7f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa95012e110 con 0x7fa950067e90 2023-09-30T22:26:37.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.028+0000 7fa957a7f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa950127d50 con 0x7fa950128090 2023-09-30T22:26:37.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.028+0000 7fa95501a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa950068760 0x7fa950121020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:37.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.028+0000 7fa95581b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa950067e90 0x7fa950120b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:37.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.028+0000 7fa95501a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa950068760 0x7fa950121020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44536/0 (socket says 172.21.15.16:44536) 2023-09-30T22:26:37.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.028+0000 7fa95501a700 1 -- 172.21.15.16:0/124103312 learned_addr learned my addr 172.21.15.16:0/124103312 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:26:37.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.028+0000 7fa95601c700 1 --2- 172.21.15.16:0/124103312 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa950128090 0x7fa950121510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:37.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.029+0000 7fa95501a700 1 -- 172.21.15.16:0/124103312 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa950128090 msgr2=0x7fa950121510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:37.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.029+0000 7fa95501a700 1 --2- 172.21.15.16:0/124103312 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa950128090 0x7fa950121510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:37.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.029+0000 7fa95501a700 1 -- 172.21.15.16:0/124103312 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa950067e90 msgr2=0x7fa950120b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:37.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.029+0000 7fa95501a700 1 --2- 172.21.15.16:0/124103312 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa950067e90 0x7fa950120b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:37.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.029+0000 7fa95501a700 1 -- 172.21.15.16:0/124103312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa94c0092a0 con 0x7fa950068760 2023-09-30T22:26:37.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.029+0000 7fa95581b700 1 --2- 172.21.15.16:0/124103312 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa950067e90 0x7fa950120b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:37.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.029+0000 7fa95501a700 1 --2- 172.21.15.16:0/124103312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa950068760 0x7fa950121020 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7fa940009130 tx=0x7fa940017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:37.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.029+0000 7fa945ffb700 1 -- 172.21.15.16:0/124103312 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa940000bc0 con 0x7fa950068760 2023-09-30T22:26:37.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.029+0000 7fa945ffb700 1 -- 172.21.15.16:0/124103312 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa94000b850 con 0x7fa950068760 2023-09-30T22:26:37.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.029+0000 7fa957a7f700 1 -- 172.21.15.16:0/124103312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa940009370 con 0x7fa950068760 2023-09-30T22:26:37.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.029+0000 7fa945ffb700 1 -- 172.21.15.16:0/124103312 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa940000d20 con 0x7fa950068760 2023-09-30T22:26:37.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.029+0000 7fa957a7f700 1 -- 172.21.15.16:0/124103312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa95012d2b0 con 0x7fa950068760 2023-09-30T22:26:37.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.030+0000 7fa945ffb700 1 -- 172.21.15.16:0/124103312 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7fa94001cb40 con 0x7fa950068760 2023-09-30T22:26:37.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.030+0000 7fa945ffb700 1 -- 172.21.15.16:0/124103312 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7fa940025940 con 0x7fa950068760 2023-09-30T22:26:37.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.030+0000 7fa957a7f700 1 -- 172.21.15.16:0/124103312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa95005a260 con 0x7fa950068760 2023-09-30T22:26:37.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.033+0000 7fa945ffb700 1 -- 172.21.15.16:0/124103312 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7fa940044e50 con 0x7fa950068760 2023-09-30T22:26:37.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.119+0000 7fa957a7f700 1 -- 172.21.15.16:0/124103312 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd new", "uuid": "03b53d64-d10d-448a-a081-39ecceab589b", "id": 3} v 0) v1 -- 0x7fa950046c20 con 0x7fa950068760 2023-09-30T22:26:37.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.122+0000 7fa945ffb700 1 -- 172.21.15.16:0/124103312 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "03b53d64-d10d-448a-a081-39ecceab589b", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa940033030 con 0x7fa950068760 2023-09-30T22:26:37.122 INFO:teuthology.orchestra.run.smithi016.stdout:3 2023-09-30T22:26:37.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.123+0000 7fa957a7f700 1 -- 172.21.15.16:0/124103312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa950068760 msgr2=0x7fa950121020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:37.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.123+0000 7fa957a7f700 1 --2- 172.21.15.16:0/124103312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa950068760 0x7fa950121020 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7fa940009130 tx=0x7fa940017000).stop 2023-09-30T22:26:37.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.123+0000 7fa957a7f700 1 -- 172.21.15.16:0/124103312 shutdown_connections 2023-09-30T22:26:37.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.123+0000 7fa957a7f700 1 --2- 172.21.15.16:0/124103312 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa950067e90 0x7fa950120b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:37.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.123+0000 7fa957a7f700 1 --2- 172.21.15.16:0/124103312 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa950128090 0x7fa950121510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:37.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.123+0000 7fa957a7f700 1 --2- 172.21.15.16:0/124103312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa950068760 0x7fa950121020 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:37.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.123+0000 7fa957a7f700 1 -- 172.21.15.16:0/124103312 >> 172.21.15.16:0/124103312 conn(0x7fa95009a3d0 msgr2=0x7fa9500954e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:37.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.123+0000 7fa957a7f700 1 -- 172.21.15.16:0/124103312 shutdown_connections 2023-09-30T22:26:37.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.123+0000 7fa957a7f700 1 -- 172.21.15.16:0/124103312 wait complete. 2023-09-30T22:26:37.136 INFO:tasks.ceph.osd.0:Restarting daemon 2023-09-30T22:26:37.136 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-09-30T22:26:37.138 INFO:tasks.ceph.osd.0:Started 2023-09-30T22:26:37.138 INFO:tasks.ceph.osd.1:Restarting daemon 2023-09-30T22:26:37.139 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-09-30T22:26:37.141 INFO:tasks.ceph.osd.1:Started 2023-09-30T22:26:37.141 INFO:tasks.ceph.osd.2:Restarting daemon 2023-09-30T22:26:37.141 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-09-30T22:26:37.143 INFO:tasks.ceph.osd.2:Started 2023-09-30T22:26:37.143 INFO:tasks.ceph.osd.3:Restarting daemon 2023-09-30T22:26:37.143 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-09-30T22:26:37.145 INFO:tasks.ceph.osd.3:Started 2023-09-30T22:26:37.146 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:26:37.161 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:26:37.160+0000 7f2f069c6680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-30T22:26:37.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.302+0000 7fb8bb248700 1 Processor -- start 2023-09-30T22:26:37.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.302+0000 7fb8bb248700 1 -- start start 2023-09-30T22:26:37.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.303+0000 7fb8bb248700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb8b4059070 0x7fb8b41256c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:37.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.303+0000 7fb8bb248700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb8b41251c0 0x7fb8b4125bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:37.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.303+0000 7fb8bb248700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8b4095f00 con 0x7fb8b4093bf0 2023-09-30T22:26:37.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.303+0000 7fb8bb248700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8b4059450 con 0x7fb8b4059070 2023-09-30T22:26:37.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.303+0000 7fb8bb248700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b4096190 con 0x7fb8b41251c0 2023-09-30T22:26:37.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.303+0000 7fb8b3fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb8b41251c0 0x7fb8b4125bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:37.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.303+0000 7fb8b8fe4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb8b4059070 0x7fb8b41256c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:37.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.303+0000 7fb8b8fe4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb8b4059070 0x7fb8b41256c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44582/0 (socket says 172.21.15.16:44582) 2023-09-30T22:26:37.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.303+0000 7fb8b3fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb8b41251c0 0x7fb8b4125bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:45946/0 (socket says 172.21.15.16:45946) 2023-09-30T22:26:37.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.303+0000 7fb8b8fe4700 1 -- 172.21.15.16:0/3999665893 learned_addr learned my addr 172.21.15.16:0/3999665893 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:26:37.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.303+0000 7fb8b97e5700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb8b4093bf0 0x7fb8b4094080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34914/0 (socket says 172.21.15.16:34914) 2023-09-30T22:26:37.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.303+0000 7fb8b8fe4700 1 -- 172.21.15.16:0/3999665893 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb8b41251c0 msgr2=0x7fb8b4125bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:37.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.303+0000 7fb8b8fe4700 1 --2- 172.21.15.16:0/3999665893 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb8b41251c0 0x7fb8b4125bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:37.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.303+0000 7fb8b37fe700 1 -- 172.21.15.16:0/3999665893 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3733078207 0 0) 0x7fb8b4095f00 con 0x7fb8b4093bf0 2023-09-30T22:26:37.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.303+0000 7fb8b8fe4700 1 -- 172.21.15.16:0/3999665893 >> v1:172.21.15.16:6789/0 conn(0x7fb8b4093bf0 legacy=0x7fb8b4094080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:37.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.303+0000 7fb8b8fe4700 1 -- 172.21.15.16:0/3999665893 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b41260a0 con 0x7fb8b4059070 2023-09-30T22:26:37.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.304+0000 7fb8b3fff700 1 --2- 172.21.15.16:0/3999665893 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb8b41251c0 0x7fb8b4125bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:37.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.304+0000 7fb8b8fe4700 1 --2- 172.21.15.16:0/3999665893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8b4059070 0x7fb8b41256c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fb8ac009f20 tx=0x7fb8ac009840).ready entity=mon.0 client_cookie=212845df3165f4c5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:37.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.304+0000 7fb8b37fe700 1 -- 172.21.15.16:0/3999665893 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac00e070 con 0x7fb8b4059070 2023-09-30T22:26:37.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.304+0000 7fb8b37fe700 1 -- 172.21.15.16:0/3999665893 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8ac00b9d0 con 0x7fb8b4059070 2023-09-30T22:26:37.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.304+0000 7fb8b37fe700 1 -- 172.21.15.16:0/3999665893 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ac00bd50 con 0x7fb8b4059070 2023-09-30T22:26:37.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.305+0000 7fb8bb248700 1 -- 172.21.15.16:0/3999665893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8b4059070 msgr2=0x7fb8b41256c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:37.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.305+0000 7fb8bb248700 1 --2- 172.21.15.16:0/3999665893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8b4059070 0x7fb8b41256c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fb8ac009f20 tx=0x7fb8ac009840).stop 2023-09-30T22:26:37.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.305+0000 7fb8bb248700 1 -- 172.21.15.16:0/3999665893 shutdown_connections 2023-09-30T22:26:37.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.305+0000 7fb8bb248700 1 --2- 172.21.15.16:0/3999665893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8b4059070 0x7fb8b41256c0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:37.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.305+0000 7fb8bb248700 1 --2- 172.21.15.16:0/3999665893 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb8b41251c0 0x7fb8b4125bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:37.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.305+0000 7fb8bb248700 1 -- 172.21.15.16:0/3999665893 >> 172.21.15.16:0/3999665893 conn(0x7fb8b4097470 msgr2=0x7fb8b4097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:37.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.305+0000 7fb8bb248700 1 -- 172.21.15.16:0/3999665893 shutdown_connections 2023-09-30T22:26:37.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.305+0000 7fb8bb248700 1 -- 172.21.15.16:0/3999665893 wait complete. 2023-09-30T22:26:37.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.306+0000 7fb8bb248700 1 Processor -- start 2023-09-30T22:26:37.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.306+0000 7fb8bb248700 1 -- start start 2023-09-30T22:26:37.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.306+0000 7fb8bb248700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb8b4059070 0x7fb8b4135f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:37.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.306+0000 7fb8bb248700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8b4093bf0 0x7fb8b4136450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:37.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.306+0000 7fb8bb248700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb8b41251c0 0x7fb8b4067750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:37.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.306+0000 7fb8bb248700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b412ad20 con 0x7fb8b4093bf0 2023-09-30T22:26:37.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.306+0000 7fb8bb248700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b4069d80 con 0x7fb8b4059070 2023-09-30T22:26:37.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.306+0000 7fb8bb248700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b4123720 con 0x7fb8b41251c0 2023-09-30T22:26:37.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.306+0000 7fb8b3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8b4093bf0 0x7fb8b4136450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:37.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.306+0000 7fb8b3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8b4093bf0 0x7fb8b4136450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44598/0 (socket says 172.21.15.16:44598) 2023-09-30T22:26:37.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.306+0000 7fb8b3fff700 1 -- 172.21.15.16:0/1568369603 learned_addr learned my addr 172.21.15.16:0/1568369603 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:26:37.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.307+0000 7fb8b97e5700 1 --2- 172.21.15.16:0/1568369603 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb8b41251c0 0x7fb8b4067750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:37.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.307+0000 7fb8b8fe4700 1 --2- 172.21.15.16:0/1568369603 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb8b4059070 0x7fb8b4135f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:37.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.307+0000 7fb8b3fff700 1 -- 172.21.15.16:0/1568369603 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb8b41251c0 msgr2=0x7fb8b4067750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:37.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.307+0000 7fb8b3fff700 1 --2- 172.21.15.16:0/1568369603 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb8b41251c0 0x7fb8b4067750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:37.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.307+0000 7fb8b3fff700 1 -- 172.21.15.16:0/1568369603 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb8b4059070 msgr2=0x7fb8b4135f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:37.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.307+0000 7fb8b3fff700 1 --2- 172.21.15.16:0/1568369603 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb8b4059070 0x7fb8b4135f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:37.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.307+0000 7fb8b3fff700 1 -- 172.21.15.16:0/1568369603 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb89c00d740 con 0x7fb8b4093bf0 2023-09-30T22:26:37.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.307+0000 7fb8b3fff700 1 --2- 172.21.15.16:0/1568369603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8b4093bf0 0x7fb8b4136450 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fb89c00b460 tx=0x7fb89c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:37.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.307+0000 7fb8b0ff9700 1 -- 172.21.15.16:0/1568369603 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb89c00be70 con 0x7fb8b4093bf0 2023-09-30T22:26:37.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.307+0000 7fb8b0ff9700 1 -- 172.21.15.16:0/1568369603 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb89c01e3f0 con 0x7fb8b4093bf0 2023-09-30T22:26:37.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.307+0000 7fb8bb248700 1 -- 172.21.15.16:0/1568369603 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8ac009ae0 con 0x7fb8b4093bf0 2023-09-30T22:26:37.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.307+0000 7fb8b0ff9700 1 -- 172.21.15.16:0/1568369603 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb89c01e770 con 0x7fb8b4093bf0 2023-09-30T22:26:37.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.307+0000 7fb8bb248700 1 -- 172.21.15.16:0/1568369603 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8b41116f0 con 0x7fb8b4093bf0 2023-09-30T22:26:37.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.308+0000 7fb8b0ff9700 1 -- 172.21.15.16:0/1568369603 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7fb89c028440 con 0x7fb8b4093bf0 2023-09-30T22:26:37.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.308+0000 7fb8b0ff9700 1 -- 172.21.15.16:0/1568369603 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7fb89c01ed80 con 0x7fb8b4093bf0 2023-09-30T22:26:37.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.308+0000 7fb8bb248700 1 -- 172.21.15.16:0/1568369603 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8b405a260 con 0x7fb8b4093bf0 2023-09-30T22:26:37.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.311+0000 7fb8b0ff9700 1 -- 172.21.15.16:0/1568369603 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7fb89c028880 con 0x7fb8b4093bf0 2023-09-30T22:26:37.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.382+0000 7fb8bb248700 1 -- 172.21.15.16:0/1568369603 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb8b4046c20 con 0x7fb8b4093bf0 2023-09-30T22:26:37.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.382+0000 7fb8b0ff9700 1 -- 172.21.15.16:0/1568369603 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v6) v1 ==== 74+0+4109 (secure 0 0 0) 0x7fb89c031460 con 0x7fb8b4093bf0 2023-09-30T22:26:37.412 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:26:37.412 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":6,"fsid":"7ede3e14-2bd0-4726-9437-6d13c90cd342","created":"2023-09-30T22:26:35.483697+0000","modified":"2023-09-30T22:26:37.120906+0000","last_up_change":"0.000000","last_in_change":"2023-09-30T22:26:37.120906+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[],"osds":[{"osd":0,"uuid":"19946985-0ee2-487f-9347-929670dbe5ff","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"8efd4dbc-6ae4-4b82-b530-3b3eac2be445","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b47bb1ce-27af-4c0c-868a-b76ebd0d9499","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"03b53d64-d10d-448a-a081-39ecceab589b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:26:37.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.383+0000 7fb8bb248700 1 -- 172.21.15.16:0/1568369603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8b4093bf0 msgr2=0x7fb8b4136450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:37.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.383+0000 7fb8bb248700 1 --2- 172.21.15.16:0/1568369603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8b4093bf0 0x7fb8b4136450 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fb89c00b460 tx=0x7fb89c008040).stop 2023-09-30T22:26:37.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.383+0000 7fb8bb248700 1 -- 172.21.15.16:0/1568369603 shutdown_connections 2023-09-30T22:26:37.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.383+0000 7fb8bb248700 1 --2- 172.21.15.16:0/1568369603 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb8b4059070 0x7fb8b4135f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:37.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.383+0000 7fb8bb248700 1 --2- 172.21.15.16:0/1568369603 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb8b41251c0 0x7fb8b4067750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:37.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.383+0000 7fb8bb248700 1 --2- 172.21.15.16:0/1568369603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8b4093bf0 0x7fb8b4136450 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:37.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.383+0000 7fb8bb248700 1 -- 172.21.15.16:0/1568369603 >> 172.21.15.16:0/1568369603 conn(0x7fb8b4097470 msgr2=0x7fb8b4129850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:37.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.384+0000 7fb8bb248700 1 -- 172.21.15.16:0/1568369603 shutdown_connections 2023-09-30T22:26:37.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:37.384+0000 7fb8bb248700 1 -- 172.21.15.16:0/1568369603 wait complete. 2023-09-30T22:26:37.416 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-09-30T22:26:37.416 INFO:tasks.ceph:Waiting for OSDs to come up 2023-09-30T22:26:37.686 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:26:37.685+0000 7f2f069c6680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-30T22:26:37.755 INFO:tasks.ceph.osd.3.smithi016.stderr:2023-09-30T22:26:37.754+0000 7ff574d93200 -1 Falling back to public interface 2023-09-30T22:26:37.780 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-09-30T22:26:37.780+0000 7f935d25b200 -1 Falling back to public interface 2023-09-30T22:26:37.785 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-09-30T22:26:37.785+0000 7f54beb82200 -1 Falling back to public interface 2023-09-30T22:26:37.788 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-09-30T22:26:37.788+0000 7fe6aee1b200 -1 Falling back to public interface 2023-09-30T22:26:37.819 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-09-30T22:26:37.886 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:26:37.885+0000 7f2f069c6680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-30T22:26:37.939 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.939+0000 7fd0877aa700 1 Processor -- start 2023-09-30T22:26:37.939 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.939+0000 7fd0877aa700 1 -- start start 2023-09-30T22:26:37.941 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.939+0000 7fd0877aa700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd0800678a0 0x7fd080097230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:37.941 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.939+0000 7fd0877aa700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd080097720 0x7fd0801302c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:37.941 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.939+0000 7fd0877aa700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd080097bb0 con 0x7fd080096740 2023-09-30T22:26:37.942 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.939+0000 7fd0877aa700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd08006a4b0 con 0x7fd080097720 2023-09-30T22:26:37.942 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.939+0000 7fd0877aa700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd080098180 con 0x7fd0800678a0 2023-09-30T22:26:37.942 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.939+0000 7fd084d45700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd0800678a0 0x7fd080097230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:37.942 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.940+0000 7fd085d47700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd080097720 0x7fd0801302c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:37.942 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.940+0000 7fd084d45700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd0800678a0 0x7fd080097230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:50184/0 (socket says 172.21.15.16:50184) 2023-09-30T22:26:37.942 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.940+0000 7fd085d47700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd080097720 0x7fd0801302c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:43218/0 (socket says 172.21.15.16:43218) 2023-09-30T22:26:37.942 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.940+0000 7fd084d45700 1 -- 172.21.15.16:0/2817098346 learned_addr learned my addr 172.21.15.16:0/2817098346 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:26:37.942 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.940+0000 7fd085546700 1 --1- 172.21.15.16:0/2817098346 >> v1:172.21.15.16:6789/0 conn(0x7fd080096740 0x7fd080096b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45696/0 (socket says 172.21.15.16:45696) 2023-09-30T22:26:37.942 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.940+0000 7fd085d47700 1 -- 172.21.15.16:0/2817098346 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd0800678a0 msgr2=0x7fd080097230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:37.942 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.940+0000 7fd085d47700 1 --2- 172.21.15.16:0/2817098346 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd0800678a0 0x7fd080097230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:37.942 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.940+0000 7fd077fff700 1 -- 172.21.15.16:0/2817098346 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3128330504 0 0) 0x7fd080097bb0 con 0x7fd080096740 2023-09-30T22:26:37.943 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.940+0000 7fd085d47700 1 -- 172.21.15.16:0/2817098346 >> v1:172.21.15.16:6789/0 conn(0x7fd080096740 legacy=0x7fd080096b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:37.943 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.940+0000 7fd085d47700 1 -- 172.21.15.16:0/2817098346 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd080095e60 con 0x7fd080097720 2023-09-30T22:26:37.943 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.940+0000 7fd084d45700 1 --2- 172.21.15.16:0/2817098346 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd0800678a0 0x7fd080097230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:37.943 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.940+0000 7fd085d47700 1 --2- 172.21.15.16:0/2817098346 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd080097720 0x7fd0801302c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fd07800f330 tx=0x7fd07800db90).ready entity=mon.1 client_cookie=a1326e53e70b512c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:37.943 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.940+0000 7fd077fff700 1 -- 172.21.15.16:0/2817098346 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd078012070 con 0x7fd080097720 2023-09-30T22:26:37.943 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.940+0000 7fd077fff700 1 -- 172.21.15.16:0/2817098346 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd07800d0d0 con 0x7fd080097720 2023-09-30T22:26:37.943 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.940+0000 7fd077fff700 1 -- 172.21.15.16:0/2817098346 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07800d450 con 0x7fd080097720 2023-09-30T22:26:37.943 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.941+0000 7fd0877aa700 1 -- 172.21.15.16:0/2817098346 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd080097720 msgr2=0x7fd0801302c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:37.943 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.941+0000 7fd0877aa700 1 --2- 172.21.15.16:0/2817098346 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd080097720 0x7fd0801302c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fd07800f330 tx=0x7fd07800db90).stop 2023-09-30T22:26:37.943 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.941+0000 7fd0877aa700 1 -- 172.21.15.16:0/2817098346 shutdown_connections 2023-09-30T22:26:37.943 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.941+0000 7fd0877aa700 1 --2- 172.21.15.16:0/2817098346 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd080097720 0x7fd0801302c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:37.943 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.941+0000 7fd0877aa700 1 --2- 172.21.15.16:0/2817098346 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd0800678a0 0x7fd080097230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:37.944 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.941+0000 7fd0877aa700 1 -- 172.21.15.16:0/2817098346 >> 172.21.15.16:0/2817098346 conn(0x7fd080098570 msgr2=0x7fd080098980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:37.944 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.941+0000 7fd0877aa700 1 -- 172.21.15.16:0/2817098346 shutdown_connections 2023-09-30T22:26:37.944 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.941+0000 7fd0877aa700 1 -- 172.21.15.16:0/2817098346 wait complete. 2023-09-30T22:26:37.946 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.942+0000 7fd0877aa700 1 Processor -- start 2023-09-30T22:26:37.946 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.942+0000 7fd0877aa700 1 -- start start 2023-09-30T22:26:37.946 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.943+0000 7fd0877aa700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0800678a0 0x7fd080135bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:37.946 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.943+0000 7fd0877aa700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd080096740 0x7fd0801360c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:37.946 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.943+0000 7fd0877aa700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd080097720 0x7fd0801365b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:37.947 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.943+0000 7fd0877aa700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd080136aa0 con 0x7fd0800678a0 2023-09-30T22:26:37.947 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.943+0000 7fd0877aa700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd080119940 con 0x7fd080096740 2023-09-30T22:26:37.947 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.943+0000 7fd0877aa700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd080067dc0 con 0x7fd080097720 2023-09-30T22:26:37.947 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.943+0000 7fd084d45700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd080096740 0x7fd0801360c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:37.947 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.943+0000 7fd084d45700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd080096740 0x7fd0801360c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:43226/0 (socket says 172.21.15.16:43226) 2023-09-30T22:26:37.947 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.943+0000 7fd084d45700 1 -- 172.21.15.16:0/252563898 learned_addr learned my addr 172.21.15.16:0/252563898 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:26:37.947 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.943+0000 7fd085d47700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd080097720 0x7fd0801365b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:37.947 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.943+0000 7fd085546700 1 --2- 172.21.15.16:0/252563898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0800678a0 0x7fd080135bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:37.947 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.944+0000 7fd085546700 1 -- 172.21.15.16:0/252563898 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd080097720 msgr2=0x7fd0801365b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:37.947 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.944+0000 7fd085546700 1 --2- 172.21.15.16:0/252563898 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd080097720 0x7fd0801365b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:37.947 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.944+0000 7fd085546700 1 -- 172.21.15.16:0/252563898 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd080096740 msgr2=0x7fd0801360c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:37.948 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.944+0000 7fd085546700 1 --2- 172.21.15.16:0/252563898 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd080096740 0x7fd0801360c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:37.948 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.944+0000 7fd085546700 1 -- 172.21.15.16:0/252563898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd070009370 con 0x7fd0800678a0 2023-09-30T22:26:37.948 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.944+0000 7fd084d45700 1 --2- 172.21.15.16:0/252563898 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd080096740 0x7fd0801360c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:37.948 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.944+0000 7fd085d47700 1 --2- 172.21.15.16:0/252563898 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd080097720 0x7fd0801365b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:37.948 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.944+0000 7fd085546700 1 --2- 172.21.15.16:0/252563898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0800678a0 0x7fd080135bd0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fd07c000c00 tx=0x7fd07c007bb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:37.948 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.944+0000 7fd0757fa700 1 -- 172.21.15.16:0/252563898 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07c0070d0 con 0x7fd0800678a0 2023-09-30T22:26:37.948 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.944+0000 7fd0757fa700 1 -- 172.21.15.16:0/252563898 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd07c007230 con 0x7fd0800678a0 2023-09-30T22:26:37.948 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.944+0000 7fd0877aa700 1 -- 172.21.15.16:0/252563898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd07800de30 con 0x7fd0800678a0 2023-09-30T22:26:37.948 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.944+0000 7fd0757fa700 1 -- 172.21.15.16:0/252563898 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd07c0056e0 con 0x7fd0800678a0 2023-09-30T22:26:37.948 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.944+0000 7fd0877aa700 1 -- 172.21.15.16:0/252563898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd08011a150 con 0x7fd0800678a0 2023-09-30T22:26:37.948 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.944+0000 7fd0757fa700 1 -- 172.21.15.16:0/252563898 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7fd07c005cb0 con 0x7fd0800678a0 2023-09-30T22:26:37.949 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.944+0000 7fd0757fa700 1 -- 172.21.15.16:0/252563898 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7fd07c013df0 con 0x7fd0800678a0 2023-09-30T22:26:37.949 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.945+0000 7fd0877aa700 1 -- 172.21.15.16:0/252563898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd08009b3a0 con 0x7fd0800678a0 2023-09-30T22:26:37.949 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:37.948+0000 7fd0757fa700 1 -- 172.21.15.16:0/252563898 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7fd08011a150 con 0x7fd0800678a0 2023-09-30T22:26:37.962 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:26:37.962+0000 7f2f069c6680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-30T22:26:38.018 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:38.018+0000 7fd0877aa700 1 -- 172.21.15.16:0/252563898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd08005a260 con 0x7fd0800678a0 2023-09-30T22:26:38.019 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:38.019+0000 7fd0757fa700 1 -- 172.21.15.16:0/252563898 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v6) v1 ==== 74+0+4109 (secure 0 0 0) 0x7fd07c0138a0 con 0x7fd0800678a0 2023-09-30T22:26:38.019 INFO:teuthology.misc.health.smithi016.stdout: 2023-09-30T22:26:38.019 INFO:teuthology.misc.health.smithi016.stdout:{"epoch":6,"fsid":"7ede3e14-2bd0-4726-9437-6d13c90cd342","created":"2023-09-30T22:26:35.483697+0000","modified":"2023-09-30T22:26:37.120906+0000","last_up_change":"0.000000","last_in_change":"2023-09-30T22:26:37.120906+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[],"osds":[{"osd":0,"uuid":"19946985-0ee2-487f-9347-929670dbe5ff","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"8efd4dbc-6ae4-4b82-b530-3b3eac2be445","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b47bb1ce-27af-4c0c-868a-b76ebd0d9499","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"03b53d64-d10d-448a-a081-39ecceab589b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:26:38.021 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:38.020+0000 7fd0877aa700 1 -- 172.21.15.16:0/252563898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0800678a0 msgr2=0x7fd080135bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:38.022 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:38.020+0000 7fd0877aa700 1 --2- 172.21.15.16:0/252563898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0800678a0 0x7fd080135bd0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fd07c000c00 tx=0x7fd07c007bb0).stop 2023-09-30T22:26:38.022 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:38.020+0000 7fd0877aa700 1 -- 172.21.15.16:0/252563898 shutdown_connections 2023-09-30T22:26:38.022 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:38.020+0000 7fd0877aa700 1 --2- 172.21.15.16:0/252563898 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd080097720 0x7fd0801365b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:38.022 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:38.020+0000 7fd0877aa700 1 --2- 172.21.15.16:0/252563898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd0800678a0 0x7fd080135bd0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:38.022 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:38.020+0000 7fd0877aa700 1 --2- 172.21.15.16:0/252563898 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd080096740 0x7fd0801360c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:38.022 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:38.020+0000 7fd0877aa700 1 -- 172.21.15.16:0/252563898 >> 172.21.15.16:0/252563898 conn(0x7fd080098570 msgr2=0x7fd080099910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:38.022 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:38.020+0000 7fd0877aa700 1 -- 172.21.15.16:0/252563898 shutdown_connections 2023-09-30T22:26:38.022 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:38.020+0000 7fd0877aa700 1 -- 172.21.15.16:0/252563898 wait complete. 2023-09-30T22:26:38.032 DEBUG:teuthology.misc:0 of 4 OSDs are up 2023-09-30T22:26:38.051 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:26:38.051+0000 7f2f069c6680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-30T22:26:38.130 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:26:38.130+0000 7f2f069c6680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-30T22:26:38.526 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:26:38.526+0000 7f2f069c6680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-30T22:26:38.622 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:26:38.622+0000 7f2f069c6680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-30T22:26:39.349 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:26:39.349+0000 7f2f069c6680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-30T22:26:39.428 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:26:39.428+0000 7f2f069c6680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-30T22:26:39.516 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:26:39.516+0000 7f2f069c6680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-30T22:26:39.849 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:26:39.849+0000 7f2f069c6680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-30T22:26:39.928 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:26:39.928+0000 7f2f069c6680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-30T22:26:40.061 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:26:40.060+0000 7f2f069c6680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-30T22:26:40.183 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:26:40.183+0000 7f2f069c6680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-30T22:26:40.356 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:26:40.356+0000 7f2f069c6680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-30T22:26:40.435 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:26:40.435+0000 7f2f069c6680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-30T22:26:41.227 INFO:tasks.ceph.osd.3.smithi016.stderr:2023-09-30T22:26:41.227+0000 7ff574d93200 -1 osd.3 0 log_to_monitors {default=true} 2023-09-30T22:26:41.283 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-09-30T22:26:41.283+0000 7fe6aee1b200 -1 osd.1 0 log_to_monitors {default=true} 2023-09-30T22:26:41.289 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-09-30T22:26:41.289+0000 7f935d25b200 -1 osd.0 0 log_to_monitors {default=true} 2023-09-30T22:26:41.300 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-09-30T22:26:41.300+0000 7f54beb82200 -1 osd.2 0 log_to_monitors {default=true} 2023-09-30T22:26:41.458 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:26:41.457+0000 7f2ebd403700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-09-30T22:26:43.228 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-09-30T22:26:43.228+0000 7f54b5523700 -1 osd.2 0 waiting for initial osdmap 2023-09-30T22:26:43.233 INFO:tasks.ceph.osd.3.smithi016.stderr:2023-09-30T22:26:43.233+0000 7ff56b734700 -1 osd.3 0 waiting for initial osdmap 2023-09-30T22:26:43.233 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-09-30T22:26:43.233+0000 7fe6a57bc700 -1 osd.1 0 waiting for initial osdmap 2023-09-30T22:26:43.233 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-09-30T22:26:43.233+0000 7f9353bfc700 -1 osd.0 0 waiting for initial osdmap 2023-09-30T22:26:43.234 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-09-30T22:26:43.234+0000 7f54b1c9f700 -1 osd.2 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-30T22:26:43.237 INFO:tasks.ceph.osd.3.smithi016.stderr:2023-09-30T22:26:43.237+0000 7ff567eb0700 -1 osd.3 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-30T22:26:43.237 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-09-30T22:26:43.237+0000 7fe6a1f38700 -1 osd.1 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-30T22:26:43.237 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-09-30T22:26:43.238+0000 7f9350378700 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-30T22:26:44.437 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-09-30T22:26:44.565 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.565+0000 7f0866527700 1 Processor -- start 2023-09-30T22:26:44.565 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.565+0000 7f0866527700 1 -- start start 2023-09-30T22:26:44.567 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.565+0000 7f0866527700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0860094a60 0x7f08600958e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:44.567 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.565+0000 7f0866527700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f086012d1d0 0x7f086012f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:44.567 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.565+0000 7f0866527700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0860096d40 con 0x7f0860095dd0 2023-09-30T22:26:44.568 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.565+0000 7f0866527700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0860094e40 con 0x7f086012d1d0 2023-09-30T22:26:44.568 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.565+0000 7f0866527700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0860091b90 con 0x7f0860094a60 2023-09-30T22:26:44.568 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.566+0000 7f085ffff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0860094a60 0x7f08600958e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:44.568 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.566+0000 7f085ffff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0860094a60 0x7f08600958e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:50250/0 (socket says 172.21.15.16:50250) 2023-09-30T22:26:44.568 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.566+0000 7f0864ac4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f086012d1d0 0x7f086012f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:44.568 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.566+0000 7f085ffff700 1 -- 172.21.15.16:0/1958766794 learned_addr learned my addr 172.21.15.16:0/1958766794 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:26:44.568 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.566+0000 7f085ffff700 1 -- 172.21.15.16:0/1958766794 >> v2:172.21.15.16:3300/0 conn(0x7f086012d1d0 msgr2=0x7f086012f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:44.568 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.566+0000 7f085ffff700 1 --2- 172.21.15.16:0/1958766794 >> v2:172.21.15.16:3300/0 conn(0x7f086012d1d0 0x7f086012f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:44.568 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.566+0000 7f085ffff700 1 -- 172.21.15.16:0/1958766794 >> v1:172.21.15.16:6789/0 conn(0x7f0860095dd0 legacy=0x7f08600961d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:44.568 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.566+0000 7f085ffff700 1 -- 172.21.15.16:0/1958766794 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08600934f0 con 0x7f0860094a60 2023-09-30T22:26:44.568 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.566+0000 7f0864ac4700 1 --2- 172.21.15.16:0/1958766794 >> v2:172.21.15.16:3300/0 conn(0x7f086012d1d0 0x7f086012f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:44.568 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.566+0000 7f085ffff700 1 --2- 172.21.15.16:0/1958766794 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0860094a60 0x7f08600958e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f0850006fd0 tx=0x7f08500090f0).ready entity=mon.2 client_cookie=1e53cef4a7219295 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:44.569 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.566+0000 7f085effd700 1 -- 172.21.15.16:0/1958766794 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0850009790 con 0x7f0860094a60 2023-09-30T22:26:44.569 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.566+0000 7f085effd700 1 -- 172.21.15.16:0/1958766794 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08500098f0 con 0x7f0860094a60 2023-09-30T22:26:44.569 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.566+0000 7f085effd700 1 -- 172.21.15.16:0/1958766794 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0850009c70 con 0x7f0860094a60 2023-09-30T22:26:44.569 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.567+0000 7f0866527700 1 -- 172.21.15.16:0/1958766794 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0860094a60 msgr2=0x7f08600958e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:44.569 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.567+0000 7f0866527700 1 --2- 172.21.15.16:0/1958766794 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0860094a60 0x7f08600958e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f0850006fd0 tx=0x7f08500090f0).stop 2023-09-30T22:26:44.569 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.567+0000 7f0866527700 1 -- 172.21.15.16:0/1958766794 shutdown_connections 2023-09-30T22:26:44.569 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.567+0000 7f0866527700 1 --2- 172.21.15.16:0/1958766794 >> v2:172.21.15.16:3300/0 conn(0x7f086012d1d0 0x7f086012f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:44.569 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.567+0000 7f0866527700 1 --2- 172.21.15.16:0/1958766794 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0860094a60 0x7f08600958e0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:44.569 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.567+0000 7f0866527700 1 -- 172.21.15.16:0/1958766794 >> 172.21.15.16:0/1958766794 conn(0x7f086009ab90 msgr2=0x7f086009afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:44.569 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.567+0000 7f0866527700 1 -- 172.21.15.16:0/1958766794 shutdown_connections 2023-09-30T22:26:44.569 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.567+0000 7f0866527700 1 -- 172.21.15.16:0/1958766794 wait complete. 2023-09-30T22:26:44.572 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.568+0000 7f0866527700 1 Processor -- start 2023-09-30T22:26:44.572 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.568+0000 7f0866527700 1 -- start start 2023-09-30T22:26:44.572 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.569+0000 7f0866527700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0860094a60 0x7f08601367d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:44.572 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.569+0000 7f0866527700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0860095dd0 0x7f0860136cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:44.572 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.569+0000 7f0866527700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f086012d1d0 0x7f0860116210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:44.572 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.569+0000 7f0866527700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0860119700 con 0x7f0860095dd0 2023-09-30T22:26:44.572 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.569+0000 7f0866527700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f086012d5b0 con 0x7f086012d1d0 2023-09-30T22:26:44.572 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.569+0000 7f0866527700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f086006a720 con 0x7f0860094a60 2023-09-30T22:26:44.572 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.569+0000 7f085ffff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0860094a60 0x7f08601367d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:44.572 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.569+0000 7f085ffff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0860094a60 0x7f08601367d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:50252/0 (socket says 172.21.15.16:50252) 2023-09-30T22:26:44.573 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.569+0000 7f0864ac4700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f086012d1d0 0x7f0860116210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:44.573 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.569+0000 7f085ffff700 1 -- 172.21.15.16:0/421046431 learned_addr learned my addr 172.21.15.16:0/421046431 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:26:44.573 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.569+0000 7f085f7fe700 1 --2- 172.21.15.16:0/421046431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0860095dd0 0x7f0860136cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:44.573 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.569+0000 7f0864ac4700 1 -- 172.21.15.16:0/421046431 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0860094a60 msgr2=0x7f08601367d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:44.573 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.569+0000 7f0864ac4700 1 --2- 172.21.15.16:0/421046431 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0860094a60 0x7f08601367d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:44.573 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.569+0000 7f0864ac4700 1 -- 172.21.15.16:0/421046431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0860095dd0 msgr2=0x7f0860136cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:44.573 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.569+0000 7f0864ac4700 1 --2- 172.21.15.16:0/421046431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0860095dd0 0x7f0860136cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:44.573 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.569+0000 7f0864ac4700 1 -- 172.21.15.16:0/421046431 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0858009a20 con 0x7f086012d1d0 2023-09-30T22:26:44.573 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.569+0000 7f085ffff700 1 --2- 172.21.15.16:0/421046431 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0860094a60 0x7f08601367d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:44.573 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.569+0000 7f0864ac4700 1 --2- 172.21.15.16:0/421046431 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f086012d1d0 0x7f0860116210 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f0858009ee0 tx=0x7f085800b040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:44.573 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.569+0000 7f0847fff700 1 -- 172.21.15.16:0/421046431 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085800cb70 con 0x7f086012d1d0 2023-09-30T22:26:44.573 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.569+0000 7f0847fff700 1 -- 172.21.15.16:0/421046431 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f085800f040 con 0x7f086012d1d0 2023-09-30T22:26:44.574 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.569+0000 7f0866527700 1 -- 172.21.15.16:0/421046431 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f085000d3f0 con 0x7f086012d1d0 2023-09-30T22:26:44.574 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.569+0000 7f0847fff700 1 -- 172.21.15.16:0/421046431 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0858013400 con 0x7f086012d1d0 2023-09-30T22:26:44.574 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.570+0000 7f0866527700 1 -- 172.21.15.16:0/421046431 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0860119400 con 0x7f086012d1d0 2023-09-30T22:26:44.574 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.570+0000 7f0847fff700 1 -- 172.21.15.16:0/421046431 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f08580137b0 con 0x7f086012d1d0 2023-09-30T22:26:44.574 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.570+0000 7f0847fff700 1 -- 172.21.15.16:0/421046431 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 2069+0+0 (secure 0 0 0) 0x7f085803b330 con 0x7f086012d1d0 2023-09-30T22:26:44.574 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.571+0000 7f0866527700 1 -- 172.21.15.16:0/421046431 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0860119c00 con 0x7f086012d1d0 2023-09-30T22:26:44.575 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.576+0000 7f0847fff700 1 -- 172.21.15.16:0/421046431 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f0860119c00 con 0x7f086012d1d0 2023-09-30T22:26:44.756 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.756+0000 7f0866527700 1 -- 172.21.15.16:0/421046431 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0860119c00 con 0x7f086012d1d0 2023-09-30T22:26:44.757 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.756+0000 7f0847fff700 1 -- 172.21.15.16:0/421046431 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v9) v1 ==== 74+0+4936 (secure 0 0 0) 0x7f0860119c00 con 0x7f086012d1d0 2023-09-30T22:26:44.757 INFO:teuthology.misc.health.smithi016.stdout: 2023-09-30T22:26:44.758 INFO:teuthology.misc.health.smithi016.stdout:{"epoch":9,"fsid":"7ede3e14-2bd0-4726-9437-6d13c90cd342","created":"2023-09-30T22:26:35.483697+0000","modified":"2023-09-30T22:26:44.227199+0000","last_up_change":"2023-09-30T22:26:44.227199+0000","last_in_change":"2023-09-30T22:26:37.120906+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[],"osds":[{"osd":0,"uuid":"19946985-0ee2-487f-9347-929670dbe5ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6804","nonce":118824}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6805","nonce":118824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6807","nonce":118824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6806","nonce":118824}]},"public_addr":"172.21.15.16:6804/118824","cluster_addr":"172.21.15.16:6805/118824","heartbeat_back_addr":"172.21.15.16:6807/118824","heartbeat_front_addr":"172.21.15.16:6806/118824","state":["exists","up"]},{"osd":1,"uuid":"8efd4dbc-6ae4-4b82-b530-3b3eac2be445","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6812","nonce":118832}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6813","nonce":118832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6815","nonce":118832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6814","nonce":118832}]},"public_addr":"172.21.15.16:6812/118832","cluster_addr":"172.21.15.16:6813/118832","heartbeat_back_addr":"172.21.15.16:6815/118832","heartbeat_front_addr":"172.21.15.16:6814/118832","state":["exists","up"]},{"osd":2,"uuid":"b47bb1ce-27af-4c0c-868a-b76ebd0d9499","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6808","nonce":118840}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6809","nonce":118840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6811","nonce":118840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6810","nonce":118840}]},"public_addr":"172.21.15.16:6808/118840","cluster_addr":"172.21.15.16:6809/118840","heartbeat_back_addr":"172.21.15.16:6811/118840","heartbeat_front_addr":"172.21.15.16:6810/118840","state":["exists","up"]},{"osd":3,"uuid":"03b53d64-d10d-448a-a081-39ecceab589b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6800","nonce":118823}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6801","nonce":118823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6803","nonce":118823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6802","nonce":118823}]},"public_addr":"172.21.15.16:6800/118823","cluster_addr":"172.21.15.16:6801/118823","heartbeat_back_addr":"172.21.15.16:6803/118823","heartbeat_front_addr":"172.21.15.16:6802/118823","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:26:44.759 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.758+0000 7f0866527700 1 -- 172.21.15.16:0/421046431 >> v1:172.21.15.16:6816/118285 conn(0x7f084c02bb90 legacy=0x7f084c02e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:44.760 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.758+0000 7f0866527700 1 -- 172.21.15.16:0/421046431 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f086012d1d0 msgr2=0x7f0860116210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:44.760 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.758+0000 7f0866527700 1 --2- 172.21.15.16:0/421046431 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f086012d1d0 0x7f0860116210 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f0858009ee0 tx=0x7f085800b040).stop 2023-09-30T22:26:44.760 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.758+0000 7f0866527700 1 -- 172.21.15.16:0/421046431 shutdown_connections 2023-09-30T22:26:44.760 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.758+0000 7f0866527700 1 --2- 172.21.15.16:0/421046431 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f086012d1d0 0x7f0860116210 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:44.760 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.758+0000 7f0866527700 1 --2- 172.21.15.16:0/421046431 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0860094a60 0x7f08601367d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:44.760 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.758+0000 7f0866527700 1 --2- 172.21.15.16:0/421046431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0860095dd0 0x7f0860136cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:44.760 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.758+0000 7f0866527700 1 -- 172.21.15.16:0/421046431 >> 172.21.15.16:0/421046431 conn(0x7f086009ab90 msgr2=0x7f0860098800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:44.760 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.759+0000 7f0866527700 1 -- 172.21.15.16:0/421046431 shutdown_connections 2023-09-30T22:26:44.760 INFO:teuthology.misc.health.smithi016.stderr:2023-09-30T22:26:44.759+0000 7f0866527700 1 -- 172.21.15.16:0/421046431 wait complete. 2023-09-30T22:26:44.771 DEBUG:teuthology.misc:4 of 4 OSDs are up 2023-09-30T22:26:44.771 INFO:tasks.ceph:Creating RBD pool 2023-09-30T22:26:44.772 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph osd pool create rbd 8 2023-09-30T22:26:44.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:44.900+0000 7f15d3881700 1 Processor -- start 2023-09-30T22:26:44.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:44.901+0000 7f15d3881700 1 -- start start 2023-09-30T22:26:44.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:44.901+0000 7f15d3881700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f15cc0926d0 0x7f15cc1275c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:44.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:44.901+0000 7f15d3881700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f15cc127be0 0x7f15cc128000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:44.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:44.901+0000 7f15d3881700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15cc091c80 con 0x7f15cc12a9a0 2023-09-30T22:26:44.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:44.901+0000 7f15d3881700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15cc099dd0 con 0x7f15cc0926d0 2023-09-30T22:26:44.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:44.901+0000 7f15d3881700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15cc0999d0 con 0x7f15cc127be0 2023-09-30T22:26:44.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:44.901+0000 7f15d0e1c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f15cc127be0 0x7f15cc128000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:44.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:44.901+0000 7f15d161d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f15cc0926d0 0x7f15cc1275c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:44.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:44.901+0000 7f15d1e1e700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f15cc12a9a0 0x7f15cc12ae50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45738/0 (socket says 172.21.15.16:45738) 2023-09-30T22:26:44.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:44.901+0000 7f15d1e1e700 1 -- 172.21.15.16:0/2359632569 learned_addr learned my addr 172.21.15.16:0/2359632569 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:26:44.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:44.902+0000 7f15d0e1c700 1 -- 172.21.15.16:0/2359632569 >> v2:172.21.15.16:3300/0 conn(0x7f15cc0926d0 msgr2=0x7f15cc1275c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:44.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:44.902+0000 7f15d0e1c700 1 --2- 172.21.15.16:0/2359632569 >> v2:172.21.15.16:3300/0 conn(0x7f15cc0926d0 0x7f15cc1275c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:44.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:44.902+0000 7f15d0e1c700 1 -- 172.21.15.16:0/2359632569 >> v1:172.21.15.16:6789/0 conn(0x7f15cc12a9a0 legacy=0x7f15cc12ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:44.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:44.902+0000 7f15c3fff700 1 -- 172.21.15.16:0/2359632569 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1817808347 0 0) 0x7f15cc091c80 con 0x7f15cc12a9a0 2023-09-30T22:26:44.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:44.902+0000 7f15d0e1c700 1 -- 172.21.15.16:0/2359632569 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15cc09a280 con 0x7f15cc127be0 2023-09-30T22:26:44.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:44.902+0000 7f15d161d700 1 --2- 172.21.15.16:0/2359632569 >> v2:172.21.15.16:3300/0 conn(0x7f15cc0926d0 0x7f15cc1275c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:44.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:44.902+0000 7f15d0e1c700 1 --2- 172.21.15.16:0/2359632569 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f15cc127be0 0x7f15cc128000 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f15bc009790 tx=0x7f15bc009060).ready entity=mon.1 client_cookie=ede0e40e57b91dd5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:44.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:44.902+0000 7f15c3fff700 1 -- 172.21.15.16:0/2359632569 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15bc00f510 con 0x7f15cc127be0 2023-09-30T22:26:44.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:44.902+0000 7f15c3fff700 1 -- 172.21.15.16:0/2359632569 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15bc00f670 con 0x7f15cc127be0 2023-09-30T22:26:44.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:44.902+0000 7f15c3fff700 1 -- 172.21.15.16:0/2359632569 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15bc00f9f0 con 0x7f15cc127be0 2023-09-30T22:26:44.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:44.903+0000 7f15d3881700 1 -- 172.21.15.16:0/2359632569 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f15cc127be0 msgr2=0x7f15cc128000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:44.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:44.903+0000 7f15d3881700 1 --2- 172.21.15.16:0/2359632569 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f15cc127be0 0x7f15cc128000 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f15bc009790 tx=0x7f15bc009060).stop 2023-09-30T22:26:44.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:44.903+0000 7f15d3881700 1 -- 172.21.15.16:0/2359632569 shutdown_connections 2023-09-30T22:26:44.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:44.903+0000 7f15d3881700 1 --2- 172.21.15.16:0/2359632569 >> v2:172.21.15.16:3300/0 conn(0x7f15cc0926d0 0x7f15cc1275c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:44.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:44.903+0000 7f15d3881700 1 --2- 172.21.15.16:0/2359632569 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f15cc127be0 0x7f15cc128000 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:44.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:44.903+0000 7f15d3881700 1 -- 172.21.15.16:0/2359632569 >> 172.21.15.16:0/2359632569 conn(0x7f15cc097480 msgr2=0x7f15cc097890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:44.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:44.903+0000 7f15d3881700 1 -- 172.21.15.16:0/2359632569 shutdown_connections 2023-09-30T22:26:44.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:44.903+0000 7f15d3881700 1 -- 172.21.15.16:0/2359632569 wait complete. 2023-09-30T22:26:44.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:44.904+0000 7f15d3881700 1 Processor -- start 2023-09-30T22:26:44.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:44.904+0000 7f15d3881700 1 -- start start 2023-09-30T22:26:44.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:44.904+0000 7f15d3881700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f15cc0926d0 0x7f15cc1179d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:44.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:44.904+0000 7f15d3881700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15cc127be0 0x7f15cc113a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:44.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:44.904+0000 7f15d3881700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f15cc12a9a0 0x7f15cc113f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:44.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:44.904+0000 7f15d3881700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15cc124b00 con 0x7f15cc127be0 2023-09-30T22:26:44.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:44.904+0000 7f15d3881700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15cc09a4b0 con 0x7f15cc0926d0 2023-09-30T22:26:44.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:44.904+0000 7f15d3881700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f15cc1174d0 con 0x7f15cc12a9a0 2023-09-30T22:26:44.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:44.904+0000 7f15d0e1c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15cc127be0 0x7f15cc113a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:44.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:44.904+0000 7f15d0e1c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15cc127be0 0x7f15cc113a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53018/0 (socket says 172.21.15.16:53018) 2023-09-30T22:26:44.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:44.904+0000 7f15d0e1c700 1 -- 172.21.15.16:0/2683238794 learned_addr learned my addr 172.21.15.16:0/2683238794 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:26:44.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:44.904+0000 7f15d1e1e700 1 --2- 172.21.15.16:0/2683238794 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f15cc12a9a0 0x7f15cc113f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:44.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:44.905+0000 7f15d161d700 1 --2- 172.21.15.16:0/2683238794 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f15cc0926d0 0x7f15cc1179d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:44.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:44.905+0000 7f15d0e1c700 1 -- 172.21.15.16:0/2683238794 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f15cc12a9a0 msgr2=0x7f15cc113f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:44.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:44.905+0000 7f15d0e1c700 1 --2- 172.21.15.16:0/2683238794 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f15cc12a9a0 0x7f15cc113f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:44.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:44.905+0000 7f15d0e1c700 1 -- 172.21.15.16:0/2683238794 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f15cc0926d0 msgr2=0x7f15cc1179d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:44.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:44.905+0000 7f15d0e1c700 1 --2- 172.21.15.16:0/2683238794 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f15cc0926d0 0x7f15cc1179d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:44.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:44.905+0000 7f15d0e1c700 1 -- 172.21.15.16:0/2683238794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15c80092a0 con 0x7f15cc127be0 2023-09-30T22:26:44.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:44.905+0000 7f15d0e1c700 1 --2- 172.21.15.16:0/2683238794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15cc127be0 0x7f15cc113a80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f15cc099b10 tx=0x7f15bc017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:44.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:44.905+0000 7f15c17fa700 1 -- 172.21.15.16:0/2683238794 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15bc00f7e0 con 0x7f15cc127be0 2023-09-30T22:26:44.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:44.905+0000 7f15c17fa700 1 -- 172.21.15.16:0/2683238794 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15bc00f960 con 0x7f15cc127be0 2023-09-30T22:26:44.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:44.905+0000 7f15d3881700 1 -- 172.21.15.16:0/2683238794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15bc0092a0 con 0x7f15cc127be0 2023-09-30T22:26:44.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:44.905+0000 7f15c17fa700 1 -- 172.21.15.16:0/2683238794 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15bc01b660 con 0x7f15cc127be0 2023-09-30T22:26:44.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:44.905+0000 7f15d3881700 1 -- 172.21.15.16:0/2683238794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15cc1155c0 con 0x7f15cc127be0 2023-09-30T22:26:44.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:44.906+0000 7f15c17fa700 1 -- 172.21.15.16:0/2683238794 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f15bc01ba10 con 0x7f15cc127be0 2023-09-30T22:26:44.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:44.906+0000 7f15c17fa700 1 -- 172.21.15.16:0/2683238794 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 2069+0+0 (secure 0 0 0) 0x7f15bc03c1b0 con 0x7f15cc127be0 2023-09-30T22:26:44.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:44.906+0000 7f15d3881700 1 -- 172.21.15.16:0/2683238794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15cc115780 con 0x7f15cc127be0 2023-09-30T22:26:44.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:44.911+0000 7f15c17fa700 1 -- 172.21.15.16:0/2683238794 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f15bc029080 con 0x7f15cc127be0 2023-09-30T22:26:45.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.085+0000 7f15d3881700 1 -- 172.21.15.16:0/2683238794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f15cc046c20 con 0x7f15cc127be0 2023-09-30T22:26:45.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.236+0000 7f15c17fa700 1 -- 172.21.15.16:0/2683238794 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v10) v1 ==== 109+0+0 (secure 0 0 0) 0x7f15bc01f020 con 0x7f15cc127be0 2023-09-30T22:26:45.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.241+0000 7f15d3881700 1 -- 172.21.15.16:0/2683238794 >> v1:172.21.15.16:6816/118285 conn(0x7f15b002bb60 legacy=0x7f15b002e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:45.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.241+0000 7f15d3881700 1 -- 172.21.15.16:0/2683238794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15cc127be0 msgr2=0x7f15cc113a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:45.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.241+0000 7f15d3881700 1 --2- 172.21.15.16:0/2683238794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15cc127be0 0x7f15cc113a80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f15cc099b10 tx=0x7f15bc017000).stop 2023-09-30T22:26:45.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.242+0000 7f15d3881700 1 -- 172.21.15.16:0/2683238794 shutdown_connections 2023-09-30T22:26:45.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.242+0000 7f15d3881700 1 --2- 172.21.15.16:0/2683238794 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f15cc0926d0 0x7f15cc1179d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:45.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.242+0000 7f15d3881700 1 --2- 172.21.15.16:0/2683238794 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f15cc12a9a0 0x7f15cc113f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:45.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.242+0000 7f15d3881700 1 --2- 172.21.15.16:0/2683238794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15cc127be0 0x7f15cc113a80 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:45.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.242+0000 7f15d3881700 1 -- 172.21.15.16:0/2683238794 >> 172.21.15.16:0/2683238794 conn(0x7f15cc097480 msgr2=0x7f15cc093da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:45.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.242+0000 7f15d3881700 1 -- 172.21.15.16:0/2683238794 shutdown_connections 2023-09-30T22:26:45.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.242+0000 7f15d3881700 1 -- 172.21.15.16:0/2683238794 wait complete. 2023-09-30T22:26:45.244 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'rbd' created 2023-09-30T22:26:45.256 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-09-30T22:26:45.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.393+0000 7f7516c89700 1 Processor -- start 2023-09-30T22:26:45.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.393+0000 7f7516c89700 1 -- start start 2023-09-30T22:26:45.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.394+0000 7f7516c89700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7510097470 0x7f7510095b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:45.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.394+0000 7f7516c89700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7510096450 0x7f7510127fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:45.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.394+0000 7f7516c89700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75100615b0 con 0x7f7510096050 2023-09-30T22:26:45.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.394+0000 7f7516c89700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7510097850 con 0x7f7510096450 2023-09-30T22:26:45.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.394+0000 7f7516c89700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75100931e0 con 0x7f7510097470 2023-09-30T22:26:45.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.394+0000 7f7514a25700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7510097470 0x7f7510095b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:45.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.394+0000 7f750ffff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7510096050 0x7f7510127890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45754/0 (socket says 172.21.15.16:45754) 2023-09-30T22:26:45.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.394+0000 7f750ffff700 1 -- 172.21.15.16:0/3291028076 learned_addr learned my addr 172.21.15.16:0/3291028076 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:26:45.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.394+0000 7f7515226700 1 --2- 172.21.15.16:0/3291028076 >> v2:172.21.15.16:3300/0 conn(0x7f7510096450 0x7f7510127fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:45.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.395+0000 7f7514a25700 1 -- 172.21.15.16:0/3291028076 >> v2:172.21.15.16:3300/0 conn(0x7f7510096450 msgr2=0x7f7510127fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:45.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.395+0000 7f7514a25700 1 --2- 172.21.15.16:0/3291028076 >> v2:172.21.15.16:3300/0 conn(0x7f7510096450 0x7f7510127fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:45.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.395+0000 7f750f7fe700 1 -- 172.21.15.16:0/3291028076 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 976890699 0 0) 0x7f75100615b0 con 0x7f7510096050 2023-09-30T22:26:45.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.395+0000 7f7514a25700 1 -- 172.21.15.16:0/3291028076 >> v1:172.21.15.16:6789/0 conn(0x7f7510096050 legacy=0x7f7510127890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:45.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.395+0000 7f7514a25700 1 -- 172.21.15.16:0/3291028076 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7510096900 con 0x7f7510097470 2023-09-30T22:26:45.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.395+0000 7f7514a25700 1 --2- 172.21.15.16:0/3291028076 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7510097470 0x7f7510095b60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f7500006fd0 tx=0x7f7500006f40).ready entity=mon.1 client_cookie=15f90a35ca36edcd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:45.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.395+0000 7f750f7fe700 1 -- 172.21.15.16:0/3291028076 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7500005880 con 0x7f7510097470 2023-09-30T22:26:45.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.395+0000 7f750f7fe700 1 -- 172.21.15.16:0/3291028076 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75000059e0 con 0x7f7510097470 2023-09-30T22:26:45.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.395+0000 7f750f7fe700 1 -- 172.21.15.16:0/3291028076 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7500005d60 con 0x7f7510097470 2023-09-30T22:26:45.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.396+0000 7f7516c89700 1 -- 172.21.15.16:0/3291028076 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7510097470 msgr2=0x7f7510095b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:45.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.396+0000 7f7516c89700 1 --2- 172.21.15.16:0/3291028076 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7510097470 0x7f7510095b60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f7500006fd0 tx=0x7f7500006f40).stop 2023-09-30T22:26:45.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.396+0000 7f7516c89700 1 -- 172.21.15.16:0/3291028076 shutdown_connections 2023-09-30T22:26:45.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.396+0000 7f7516c89700 1 --2- 172.21.15.16:0/3291028076 >> v2:172.21.15.16:3300/0 conn(0x7f7510096450 0x7f7510127fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:45.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.396+0000 7f7516c89700 1 --2- 172.21.15.16:0/3291028076 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7510097470 0x7f7510095b60 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:45.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.396+0000 7f7516c89700 1 -- 172.21.15.16:0/3291028076 >> 172.21.15.16:0/3291028076 conn(0x7f751009ab00 msgr2=0x7f7510096b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:45.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.396+0000 7f7516c89700 1 -- 172.21.15.16:0/3291028076 shutdown_connections 2023-09-30T22:26:45.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.396+0000 7f7516c89700 1 -- 172.21.15.16:0/3291028076 wait complete. 2023-09-30T22:26:45.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.397+0000 7f7516c89700 1 Processor -- start 2023-09-30T22:26:45.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.397+0000 7f7516c89700 1 -- start start 2023-09-30T22:26:45.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.398+0000 7f7516c89700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7510096050 0x7f7510120f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:45.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.398+0000 7f7516c89700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7510096450 0x7f7510121410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:45.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.398+0000 7f7516c89700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7510097470 0x7f7510121900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:45.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.398+0000 7f7516c89700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7510092ce0 con 0x7f7510097470 2023-09-30T22:26:45.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.398+0000 7f7516c89700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75101243e0 con 0x7f7510096050 2023-09-30T22:26:45.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.398+0000 7f7516c89700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7510123e40 con 0x7f7510096450 2023-09-30T22:26:45.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.398+0000 7f7514a25700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7510096050 0x7f7510120f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:45.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.398+0000 7f7514a25700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7510096050 0x7f7510120f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:43342/0 (socket says 172.21.15.16:43342) 2023-09-30T22:26:45.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.398+0000 7f7514a25700 1 -- 172.21.15.16:0/1399335647 learned_addr learned my addr 172.21.15.16:0/1399335647 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:26:45.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.398+0000 7f7515226700 1 --2- 172.21.15.16:0/1399335647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7510097470 0x7f7510121900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:45.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.398+0000 7f750ffff700 1 --2- 172.21.15.16:0/1399335647 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7510096450 0x7f7510121410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:45.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.398+0000 7f7514a25700 1 -- 172.21.15.16:0/1399335647 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7510096450 msgr2=0x7f7510121410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:45.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.398+0000 7f7514a25700 1 --2- 172.21.15.16:0/1399335647 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7510096450 0x7f7510121410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:45.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.398+0000 7f7514a25700 1 -- 172.21.15.16:0/1399335647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7510097470 msgr2=0x7f7510121900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:45.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.398+0000 7f7514a25700 1 --2- 172.21.15.16:0/1399335647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7510097470 0x7f7510121900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:45.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.398+0000 7f7514a25700 1 -- 172.21.15.16:0/1399335647 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7500009250 con 0x7f7510096050 2023-09-30T22:26:45.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.398+0000 7f7515226700 1 --2- 172.21.15.16:0/1399335647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7510097470 0x7f7510121900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:45.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.399+0000 7f7514a25700 1 --2- 172.21.15.16:0/1399335647 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7510096050 0x7f7510120f20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7f7500009dd0 tx=0x7f7500016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:45.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.399+0000 7f750cff9700 1 -- 172.21.15.16:0/1399335647 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750000f720 con 0x7f7510096050 2023-09-30T22:26:45.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.399+0000 7f750cff9700 1 -- 172.21.15.16:0/1399335647 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f750000f880 con 0x7f7510096050 2023-09-30T22:26:45.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.399+0000 7f7516c89700 1 -- 172.21.15.16:0/1399335647 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7510124fc0 con 0x7f7510096050 2023-09-30T22:26:45.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.399+0000 7f750cff9700 1 -- 172.21.15.16:0/1399335647 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750000fc20 con 0x7f7510096050 2023-09-30T22:26:45.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.399+0000 7f7516c89700 1 -- 172.21.15.16:0/1399335647 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7510124c80 con 0x7f7510096050 2023-09-30T22:26:45.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.400+0000 7f750cff9700 1 -- 172.21.15.16:0/1399335647 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f7500027460 con 0x7f7510096050 2023-09-30T22:26:45.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.400+0000 7f750cff9700 1 -- 172.21.15.16:0/1399335647 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 2803+0+0 (secure 0 0 0) 0x7f750003c1f0 con 0x7f7510096050 2023-09-30T22:26:45.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.400+0000 7f7516c89700 1 -- 172.21.15.16:0/1399335647 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7510124c80 con 0x7f7510096050 2023-09-30T22:26:45.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.405+0000 7f750cff9700 1 -- 172.21.15.16:0/1399335647 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f7510124c80 con 0x7f7510096050 2023-09-30T22:26:45.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:45.582+0000 7f7516c89700 1 -- 172.21.15.16:0/1399335647 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f7510124c80 con 0x7f7510096050 2023-09-30T22:26:46.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.240+0000 7f750cff9700 1 -- 172.21.15.16:0/1399335647 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v11) v1 ==== 173+0+0 (secure 0 0 0) 0x7f7510124c80 con 0x7f7510096050 2023-09-30T22:26:46.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.245+0000 7f7516c89700 1 -- 172.21.15.16:0/1399335647 >> v1:172.21.15.16:6816/118285 conn(0x7f74fc02bb60 legacy=0x7f74fc02e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:46.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.245+0000 7f7516c89700 1 -- 172.21.15.16:0/1399335647 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7510096050 msgr2=0x7f7510120f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:46.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.245+0000 7f7516c89700 1 --2- 172.21.15.16:0/1399335647 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7510096050 0x7f7510120f20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7f7500009dd0 tx=0x7f7500016000).stop 2023-09-30T22:26:46.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.246+0000 7f7516c89700 1 -- 172.21.15.16:0/1399335647 shutdown_connections 2023-09-30T22:26:46.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.246+0000 7f7516c89700 1 --2- 172.21.15.16:0/1399335647 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7510096050 0x7f7510120f20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:46.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.246+0000 7f7516c89700 1 --2- 172.21.15.16:0/1399335647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7510097470 0x7f7510121900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:46.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.246+0000 7f7516c89700 1 --2- 172.21.15.16:0/1399335647 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7510096450 0x7f7510121410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:46.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.246+0000 7f7516c89700 1 -- 172.21.15.16:0/1399335647 >> 172.21.15.16:0/1399335647 conn(0x7f751009ab00 msgr2=0x7f7510127890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:46.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.246+0000 7f7516c89700 1 -- 172.21.15.16:0/1399335647 shutdown_connections 2023-09-30T22:26:46.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.246+0000 7f7516c89700 1 -- 172.21.15.16:0/1399335647 wait complete. 2023-09-30T22:26:46.247 INFO:teuthology.orchestra.run.smithi016.stderr:enabled application 'rbd' on pool 'rbd' 2023-09-30T22:26:46.261 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-09-30T22:26:46.261 INFO:tasks.ceph.mds.a:Restarting daemon 2023-09-30T22:26:46.261 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-09-30T22:26:46.263 INFO:tasks.ceph.mds.a:Started 2023-09-30T22:26:46.263 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-09-30T22:26:46.264 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:26:46.350 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2023-09-30T22:26:46.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.400+0000 7f2135235700 1 Processor -- start 2023-09-30T22:26:46.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.400+0000 7f2135235700 1 -- start start 2023-09-30T22:26:46.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.400+0000 7f2135235700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2130091b00 0x7f213012c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:46.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.400+0000 7f2135235700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f213012ce60 0x7f213012f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:46.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.400+0000 7f2135235700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2130096760 con 0x7f2130096380 2023-09-30T22:26:46.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.400+0000 7f2135235700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2130091ff0 con 0x7f213012ce60 2023-09-30T22:26:46.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.400+0000 7f2135235700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2130092c80 con 0x7f2130091b00 2023-09-30T22:26:46.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.401+0000 7f212e59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2130091b00 0x7f213012c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:46.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.401+0000 7f212f59e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f213012ce60 0x7f213012f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:46.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.401+0000 7f212e59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2130091b00 0x7f213012c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:50292/0 (socket says 172.21.15.16:50292) 2023-09-30T22:26:46.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.401+0000 7f212ed9d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2130096380 0x7f2130094730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45776/0 (socket says 172.21.15.16:45776) 2023-09-30T22:26:46.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.401+0000 7f212e59c700 1 -- 172.21.15.16:0/632218525 learned_addr learned my addr 172.21.15.16:0/632218525 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:26:46.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.401+0000 7f212e59c700 1 -- 172.21.15.16:0/632218525 >> v2:172.21.15.16:3300/0 conn(0x7f213012ce60 msgr2=0x7f213012f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:46.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.401+0000 7f212e59c700 1 --2- 172.21.15.16:0/632218525 >> v2:172.21.15.16:3300/0 conn(0x7f213012ce60 0x7f213012f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:46.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.401+0000 7f212e59c700 1 -- 172.21.15.16:0/632218525 >> v1:172.21.15.16:6789/0 conn(0x7f2130096380 legacy=0x7f2130094730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:46.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.401+0000 7f212e59c700 1 -- 172.21.15.16:0/632218525 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2130094e40 con 0x7f2130091b00 2023-09-30T22:26:46.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.401+0000 7f212e59c700 1 --2- 172.21.15.16:0/632218525 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2130091b00 0x7f213012c970 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f2124006fd0 tx=0x7f21240090f0).ready entity=mon.2 client_cookie=3f99a53a81f7b898 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:46.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.401+0000 7f212dd9b700 1 -- 172.21.15.16:0/632218525 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212400b580 con 0x7f2130091b00 2023-09-30T22:26:46.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.401+0000 7f212dd9b700 1 -- 172.21.15.16:0/632218525 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f212400b6e0 con 0x7f2130091b00 2023-09-30T22:26:46.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.401+0000 7f212dd9b700 1 -- 172.21.15.16:0/632218525 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f212400ba60 con 0x7f2130091b00 2023-09-30T22:26:46.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.402+0000 7f2135235700 1 -- 172.21.15.16:0/632218525 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2130091b00 msgr2=0x7f213012c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:46.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.402+0000 7f2135235700 1 --2- 172.21.15.16:0/632218525 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2130091b00 0x7f213012c970 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f2124006fd0 tx=0x7f21240090f0).stop 2023-09-30T22:26:46.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.402+0000 7f2135235700 1 -- 172.21.15.16:0/632218525 shutdown_connections 2023-09-30T22:26:46.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.402+0000 7f2135235700 1 --2- 172.21.15.16:0/632218525 >> v2:172.21.15.16:3300/0 conn(0x7f213012ce60 0x7f213012f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:46.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.402+0000 7f2135235700 1 --2- 172.21.15.16:0/632218525 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2130091b00 0x7f213012c970 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:46.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.402+0000 7f2135235700 1 -- 172.21.15.16:0/632218525 >> 172.21.15.16:0/632218525 conn(0x7f2130097470 msgr2=0x7f2130097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:46.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.402+0000 7f2135235700 1 -- 172.21.15.16:0/632218525 shutdown_connections 2023-09-30T22:26:46.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.402+0000 7f2135235700 1 -- 172.21.15.16:0/632218525 wait complete. 2023-09-30T22:26:46.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.403+0000 7f2135235700 1 Processor -- start 2023-09-30T22:26:46.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.403+0000 7f2135235700 1 -- start start 2023-09-30T22:26:46.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.403+0000 7f2135235700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2130091b00 0x7f213013a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:46.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.403+0000 7f2135235700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2130096380 0x7f213013ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:46.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.403+0000 7f2135235700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f213012ce60 0x7f2130110fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:46.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.403+0000 7f2135235700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21301146e0 con 0x7f213012ce60 2023-09-30T22:26:46.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.403+0000 7f2135235700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2130099090 con 0x7f2130091b00 2023-09-30T22:26:46.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.403+0000 7f2135235700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f213012f470 con 0x7f2130096380 2023-09-30T22:26:46.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.404+0000 7f212e59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2130096380 0x7f213013ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:46.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.404+0000 7f212ed9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2130091b00 0x7f213013a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:46.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.404+0000 7f212e59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2130096380 0x7f213013ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:50302/0 (socket says 172.21.15.16:50302) 2023-09-30T22:26:46.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.404+0000 7f212f59e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f213012ce60 0x7f2130110fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:46.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.404+0000 7f212e59c700 1 -- 172.21.15.16:0/3750927529 learned_addr learned my addr 172.21.15.16:0/3750927529 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:26:46.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.404+0000 7f212e59c700 1 -- 172.21.15.16:0/3750927529 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2130091b00 msgr2=0x7f213013a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:46.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.404+0000 7f212e59c700 1 --2- 172.21.15.16:0/3750927529 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2130091b00 0x7f213013a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:46.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.404+0000 7f212e59c700 1 -- 172.21.15.16:0/3750927529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f213012ce60 msgr2=0x7f2130110fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:46.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.404+0000 7f212e59c700 1 --2- 172.21.15.16:0/3750927529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f213012ce60 0x7f2130110fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:46.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.404+0000 7f212e59c700 1 -- 172.21.15.16:0/3750927529 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2124009370 con 0x7f2130096380 2023-09-30T22:26:46.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.404+0000 7f212e59c700 1 --2- 172.21.15.16:0/3750927529 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2130096380 0x7f213013ac10 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f2124009fd0 tx=0x7f2124017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:46.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.404+0000 7f2116ffd700 1 -- 172.21.15.16:0/3750927529 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2124000bc0 con 0x7f2130096380 2023-09-30T22:26:46.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.404+0000 7f2116ffd700 1 -- 172.21.15.16:0/3750927529 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f212400b850 con 0x7f2130096380 2023-09-30T22:26:46.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.404+0000 7f2135235700 1 -- 172.21.15.16:0/3750927529 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f213006a450 con 0x7f2130096380 2023-09-30T22:26:46.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.404+0000 7f2116ffd700 1 -- 172.21.15.16:0/3750927529 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2124000d20 con 0x7f2130096380 2023-09-30T22:26:46.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.404+0000 7f2135235700 1 -- 172.21.15.16:0/3750927529 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2130113ac0 con 0x7f2130096380 2023-09-30T22:26:46.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.405+0000 7f2116ffd700 1 -- 172.21.15.16:0/3750927529 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f212401cad0 con 0x7f2130096380 2023-09-30T22:26:46.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.405+0000 7f2116ffd700 1 -- 172.21.15.16:0/3750927529 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 2838+0+0 (secure 0 0 0) 0x7f212403ca60 con 0x7f2130096380 2023-09-30T22:26:46.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.405+0000 7f2135235700 1 -- 172.21.15.16:0/3750927529 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f213005a260 con 0x7f2130096380 2023-09-30T22:26:46.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.410+0000 7f2116ffd700 1 -- 172.21.15.16:0/3750927529 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f212402a020 con 0x7f2130096380 2023-09-30T22:26:46.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.593+0000 7f2135235700 1 -- 172.21.15.16:0/3750927529 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2130093970 con 0x7f2130096380 2023-09-30T22:26:46.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.594+0000 7f2116ffd700 1 -- 172.21.15.16:0/3750927529 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v11) v1 ==== 74+0+8150 (secure 0 0 0) 0x7f212401cd80 con 0x7f2130096380 2023-09-30T22:26:46.594 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:26:46.594 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":11,"fsid":"7ede3e14-2bd0-4726-9437-6d13c90cd342","created":"2023-09-30T22:26:35.483697+0000","modified":"2023-09-30T22:26:46.235485+0000","last_up_change":"2023-09-30T22:26:44.227199+0000","last_in_change":"2023-09-30T22:26:37.120906+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-09-30T22:26:44.285133+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:26:45.132921+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"19946985-0ee2-487f-9347-929670dbe5ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6804","nonce":118824}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6805","nonce":118824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6807","nonce":118824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6806","nonce":118824}]},"public_addr":"172.21.15.16:6804/118824","cluster_addr":"172.21.15.16:6805/118824","heartbeat_back_addr":"172.21.15.16:6807/118824","heartbeat_front_addr":"172.21.15.16:6806/118824","state":["exists","up"]},{"osd":1,"uuid":"8efd4dbc-6ae4-4b82-b530-3b3eac2be445","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6812","nonce":118832}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6813","nonce":118832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6815","nonce":118832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6814","nonce":118832}]},"public_addr":"172.21.15.16:6812/118832","cluster_addr":"172.21.15.16:6813/118832","heartbeat_back_addr":"172.21.15.16:6815/118832","heartbeat_front_addr":"172.21.15.16:6814/118832","state":["exists","up"]},{"osd":2,"uuid":"b47bb1ce-27af-4c0c-868a-b76ebd0d9499","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6808","nonce":118840}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6809","nonce":118840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6811","nonce":118840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6810","nonce":118840}]},"public_addr":"172.21.15.16:6808/118840","cluster_addr":"172.21.15.16:6809/118840","heartbeat_back_addr":"172.21.15.16:6811/118840","heartbeat_front_addr":"172.21.15.16:6810/118840","state":["exists","up"]},{"osd":3,"uuid":"03b53d64-d10d-448a-a081-39ecceab589b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6800","nonce":118823}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6801","nonce":118823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6803","nonce":118823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6802","nonce":118823}]},"public_addr":"172.21.15.16:6800/118823","cluster_addr":"172.21.15.16:6801/118823","heartbeat_back_addr":"172.21.15.16:6803/118823","heartbeat_front_addr":"172.21.15.16:6802/118823","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.305953+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.307488+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.341023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.192071+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:26:46.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.596+0000 7f2135235700 1 -- 172.21.15.16:0/3750927529 >> v1:172.21.15.16:6816/118285 conn(0x7f211c02bb60 legacy=0x7f211c02e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:46.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.596+0000 7f2135235700 1 -- 172.21.15.16:0/3750927529 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2130096380 msgr2=0x7f213013ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:46.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.596+0000 7f2135235700 1 --2- 172.21.15.16:0/3750927529 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2130096380 0x7f213013ac10 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f2124009fd0 tx=0x7f2124017000).stop 2023-09-30T22:26:46.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.596+0000 7f2135235700 1 -- 172.21.15.16:0/3750927529 shutdown_connections 2023-09-30T22:26:46.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.596+0000 7f2135235700 1 --2- 172.21.15.16:0/3750927529 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2130091b00 0x7f213013a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:46.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.596+0000 7f2135235700 1 --2- 172.21.15.16:0/3750927529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f213012ce60 0x7f2130110fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:46.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.596+0000 7f2135235700 1 --2- 172.21.15.16:0/3750927529 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2130096380 0x7f213013ac10 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:46.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.596+0000 7f2135235700 1 -- 172.21.15.16:0/3750927529 >> 172.21.15.16:0/3750927529 conn(0x7f2130097470 msgr2=0x7f2130125410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:46.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.596+0000 7f2135235700 1 -- 172.21.15.16:0/3750927529 shutdown_connections 2023-09-30T22:26:46.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.596+0000 7f2135235700 1 -- 172.21.15.16:0/3750927529 wait complete. 2023-09-30T22:26:46.610 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-09-30T22:26:44.285133+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '11', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-09-30T22:26:45.132921+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '11', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-09-30T22:26:46.611 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-09-30T22:26:46.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.744+0000 7fc4daa45700 1 Processor -- start 2023-09-30T22:26:46.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.744+0000 7fc4daa45700 1 -- start start 2023-09-30T22:26:46.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.745+0000 7fc4daa45700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc4d41275c0 0x7fc4d412fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:46.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.745+0000 7fc4daa45700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc4d4130220 0x7fc4d4132610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:46.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.745+0000 7fc4daa45700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4d4091c80 con 0x7fc4d40926d0 2023-09-30T22:26:46.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.745+0000 7fc4daa45700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4d41279e0 con 0x7fc4d4130220 2023-09-30T22:26:46.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.745+0000 7fc4daa45700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d40999d0 con 0x7fc4d41275c0 2023-09-30T22:26:46.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.745+0000 7fc4d37fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc4d41275c0 0x7fc4d412fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:46.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.745+0000 7fc4d37fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc4d41275c0 0x7fc4d412fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:50308/0 (socket says 172.21.15.16:50308) 2023-09-30T22:26:46.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.745+0000 7fc4d37fe700 1 -- 172.21.15.16:0/3347799879 learned_addr learned my addr 172.21.15.16:0/3347799879 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:26:46.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.745+0000 7fc4d8fe2700 1 --2- 172.21.15.16:0/3347799879 >> v2:172.21.15.16:3300/0 conn(0x7fc4d4130220 0x7fc4d4132610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:46.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.745+0000 7fc4d37fe700 1 -- 172.21.15.16:0/3347799879 >> v2:172.21.15.16:3300/0 conn(0x7fc4d4130220 msgr2=0x7fc4d4132610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:46.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.745+0000 7fc4d37fe700 1 --2- 172.21.15.16:0/3347799879 >> v2:172.21.15.16:3300/0 conn(0x7fc4d4130220 0x7fc4d4132610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:46.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.745+0000 7fc4d37fe700 1 -- 172.21.15.16:0/3347799879 >> v1:172.21.15.16:6789/0 conn(0x7fc4d40926d0 legacy=0x7fc4d4127be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:46.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.745+0000 7fc4d2ffd700 1 -- 172.21.15.16:0/3347799879 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2486374512 0 0) 0x7fc4d4091c80 con 0x7fc4d40926d0 2023-09-30T22:26:46.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.745+0000 7fc4d37fe700 1 -- 172.21.15.16:0/3347799879 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4d409a280 con 0x7fc4d41275c0 2023-09-30T22:26:46.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.745+0000 7fc4d8fe2700 1 --2- 172.21.15.16:0/3347799879 >> v2:172.21.15.16:3300/0 conn(0x7fc4d4130220 0x7fc4d4132610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:46.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.745+0000 7fc4d37fe700 1 --2- 172.21.15.16:0/3347799879 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc4d41275c0 0x7fc4d412fd30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fc4c4006fd0 tx=0x7fc4c40090f0).ready entity=mon.2 client_cookie=45129ff1ff3c5f67 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:46.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.746+0000 7fc4d2ffd700 1 -- 172.21.15.16:0/3347799879 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c400b580 con 0x7fc4d41275c0 2023-09-30T22:26:46.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.746+0000 7fc4d2ffd700 1 -- 172.21.15.16:0/3347799879 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4c400b6e0 con 0x7fc4d41275c0 2023-09-30T22:26:46.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.746+0000 7fc4d2ffd700 1 -- 172.21.15.16:0/3347799879 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c400ba60 con 0x7fc4d41275c0 2023-09-30T22:26:46.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.746+0000 7fc4daa45700 1 -- 172.21.15.16:0/3347799879 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc4d41275c0 msgr2=0x7fc4d412fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:46.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.746+0000 7fc4daa45700 1 --2- 172.21.15.16:0/3347799879 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc4d41275c0 0x7fc4d412fd30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fc4c4006fd0 tx=0x7fc4c40090f0).stop 2023-09-30T22:26:46.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.746+0000 7fc4daa45700 1 -- 172.21.15.16:0/3347799879 shutdown_connections 2023-09-30T22:26:46.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.746+0000 7fc4daa45700 1 --2- 172.21.15.16:0/3347799879 >> v2:172.21.15.16:3300/0 conn(0x7fc4d4130220 0x7fc4d4132610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:46.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.746+0000 7fc4daa45700 1 --2- 172.21.15.16:0/3347799879 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc4d41275c0 0x7fc4d412fd30 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:46.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.746+0000 7fc4daa45700 1 -- 172.21.15.16:0/3347799879 >> 172.21.15.16:0/3347799879 conn(0x7fc4d4097480 msgr2=0x7fc4d4097890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:46.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.746+0000 7fc4daa45700 1 -- 172.21.15.16:0/3347799879 shutdown_connections 2023-09-30T22:26:46.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.746+0000 7fc4daa45700 1 -- 172.21.15.16:0/3347799879 wait complete. 2023-09-30T22:26:46.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.747+0000 7fc4daa45700 1 Processor -- start 2023-09-30T22:26:46.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.747+0000 7fc4daa45700 1 -- start start 2023-09-30T22:26:46.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.748+0000 7fc4daa45700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4d40926d0 0x7fc4d412e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:46.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.748+0000 7fc4daa45700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc4d41275c0 0x7fc4d412e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:46.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.748+0000 7fc4daa45700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc4d4130220 0x7fc4d412ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:46.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.748+0000 7fc4daa45700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d412ef30 con 0x7fc4d40926d0 2023-09-30T22:26:46.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.748+0000 7fc4daa45700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d4113880 con 0x7fc4d41275c0 2023-09-30T22:26:46.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.748+0000 7fc4daa45700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d4112700 con 0x7fc4d4130220 2023-09-30T22:26:46.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.748+0000 7fc4d3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4d40926d0 0x7fc4d412e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:46.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.748+0000 7fc4d37fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc4d41275c0 0x7fc4d412e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:46.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.748+0000 7fc4d3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4d40926d0 0x7fc4d412e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53058/0 (socket says 172.21.15.16:53058) 2023-09-30T22:26:46.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.748+0000 7fc4d3fff700 1 -- 172.21.15.16:0/368176797 learned_addr learned my addr 172.21.15.16:0/368176797 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:26:46.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.748+0000 7fc4d37fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc4d41275c0 0x7fc4d412e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:43372/0 (socket says 172.21.15.16:43372) 2023-09-30T22:26:46.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.748+0000 7fc4d8fe2700 1 --2- 172.21.15.16:0/368176797 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc4d4130220 0x7fc4d412ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:46.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.748+0000 7fc4d3fff700 1 -- 172.21.15.16:0/368176797 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc4d4130220 msgr2=0x7fc4d412ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:46.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.748+0000 7fc4d3fff700 1 --2- 172.21.15.16:0/368176797 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc4d4130220 0x7fc4d412ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:46.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.748+0000 7fc4d3fff700 1 -- 172.21.15.16:0/368176797 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc4d41275c0 msgr2=0x7fc4d412e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:46.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.748+0000 7fc4d3fff700 1 --2- 172.21.15.16:0/368176797 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc4d41275c0 0x7fc4d412e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:46.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.748+0000 7fc4d3fff700 1 -- 172.21.15.16:0/368176797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4cc009a20 con 0x7fc4d40926d0 2023-09-30T22:26:46.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.749+0000 7fc4d3fff700 1 --2- 172.21.15.16:0/368176797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4d40926d0 0x7fc4d412e060 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fc4bc007fd0 tx=0x7fc4bc007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:46.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.749+0000 7fc4cbfff700 1 -- 172.21.15.16:0/368176797 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc00f470 con 0x7fc4d40926d0 2023-09-30T22:26:46.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.749+0000 7fc4cbfff700 1 -- 172.21.15.16:0/368176797 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4bc00f5f0 con 0x7fc4d40926d0 2023-09-30T22:26:46.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.749+0000 7fc4daa45700 1 -- 172.21.15.16:0/368176797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4c4009370 con 0x7fc4d40926d0 2023-09-30T22:26:46.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.749+0000 7fc4cbfff700 1 -- 172.21.15.16:0/368176797 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc00f9b0 con 0x7fc4d40926d0 2023-09-30T22:26:46.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.749+0000 7fc4daa45700 1 -- 172.21.15.16:0/368176797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4d4113380 con 0x7fc4d40926d0 2023-09-30T22:26:46.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.749+0000 7fc4cbfff700 1 -- 172.21.15.16:0/368176797 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fc4bc00fd20 con 0x7fc4d40926d0 2023-09-30T22:26:46.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.750+0000 7fc4cbfff700 1 -- 172.21.15.16:0/368176797 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 2838+0+0 (secure 0 0 0) 0x7fc4bc03a210 con 0x7fc4d40926d0 2023-09-30T22:26:46.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.750+0000 7fc4daa45700 1 -- 172.21.15.16:0/368176797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4d4112570 con 0x7fc4d40926d0 2023-09-30T22:26:46.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.755+0000 7fc4cbfff700 1 -- 172.21.15.16:0/368176797 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fc4bc018260 con 0x7fc4d40926d0 2023-09-30T22:26:46.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.928+0000 7fc4daa45700 1 -- 172.21.15.16:0/368176797 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fc4d409a4b0 con 0x7fc4d40926d0 2023-09-30T22:26:46.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.928+0000 7fc4cbfff700 1 -- 172.21.15.16:0/368176797 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v11) v1 ==== 110+0+10 (secure 0 0 0) 0x7fc4bc010af0 con 0x7fc4d40926d0 2023-09-30T22:26:46.929 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-09-30T22:26:46.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.930+0000 7fc4daa45700 1 -- 172.21.15.16:0/368176797 >> v1:172.21.15.16:6816/118285 conn(0x7fc4b402bb60 legacy=0x7fc4b402e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:46.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.930+0000 7fc4daa45700 1 -- 172.21.15.16:0/368176797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4d40926d0 msgr2=0x7fc4d412e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:46.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.930+0000 7fc4daa45700 1 --2- 172.21.15.16:0/368176797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4d40926d0 0x7fc4d412e060 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fc4bc007fd0 tx=0x7fc4bc007c20).stop 2023-09-30T22:26:46.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.930+0000 7fc4daa45700 1 -- 172.21.15.16:0/368176797 shutdown_connections 2023-09-30T22:26:46.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.930+0000 7fc4daa45700 1 --2- 172.21.15.16:0/368176797 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc4d4130220 0x7fc4d412ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:46.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.930+0000 7fc4daa45700 1 --2- 172.21.15.16:0/368176797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4d40926d0 0x7fc4d412e060 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:46.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.930+0000 7fc4daa45700 1 --2- 172.21.15.16:0/368176797 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc4d41275c0 0x7fc4d412e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:46.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.930+0000 7fc4daa45700 1 -- 172.21.15.16:0/368176797 >> 172.21.15.16:0/368176797 conn(0x7fc4d4097480 msgr2=0x7fc4d4092d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:46.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.930+0000 7fc4daa45700 1 -- 172.21.15.16:0/368176797 shutdown_connections 2023-09-30T22:26:46.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:46.930+0000 7fc4daa45700 1 -- 172.21.15.16:0/368176797 wait complete. 2023-09-30T22:26:46.943 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-09-30T22:26:47.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.076+0000 7feefa216700 1 Processor -- start 2023-09-30T22:26:47.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.076+0000 7feefa216700 1 -- start start 2023-09-30T22:26:47.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.076+0000 7feefa216700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7feef4059070 0x7feef4125250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:47.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.076+0000 7feefa216700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7feef412db40 0x7feef412ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:47.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.076+0000 7feefa216700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feef4098790 con 0x7feef41257f0 2023-09-30T22:26:47.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.076+0000 7feefa216700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feef4059450 con 0x7feef4059070 2023-09-30T22:26:47.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.076+0000 7feefa216700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feef4059da0 con 0x7feef412db40 2023-09-30T22:26:47.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.076+0000 7feef37fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7feef4059070 0x7feef4125250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:47.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.076+0000 7feef3fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7feef412db40 0x7feef412ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:47.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.076+0000 7feef2ffd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7feef41257f0 0x7feef4125c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:45806/0 (socket says 172.21.15.16:45806) 2023-09-30T22:26:47.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.076+0000 7feef2ffd700 1 -- 172.21.15.16:0/1740089674 learned_addr learned my addr 172.21.15.16:0/1740089674 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:26:47.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.077+0000 7feef27fc700 1 -- 172.21.15.16:0/1740089674 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1424772163 0 0) 0x7feef4098790 con 0x7feef41257f0 2023-09-30T22:26:47.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.077+0000 7feef3fff700 1 -- 172.21.15.16:0/1740089674 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7feef4059070 msgr2=0x7feef4125250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:47.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.077+0000 7feef3fff700 1 --2- 172.21.15.16:0/1740089674 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7feef4059070 0x7feef4125250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:47.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.077+0000 7feef3fff700 1 -- 172.21.15.16:0/1740089674 >> v1:172.21.15.16:6789/0 conn(0x7feef41257f0 legacy=0x7feef4125c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:47.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.077+0000 7feef3fff700 1 -- 172.21.15.16:0/1740089674 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feef40996f0 con 0x7feef412db40 2023-09-30T22:26:47.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.077+0000 7feef37fe700 1 --2- 172.21.15.16:0/1740089674 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7feef4059070 0x7feef4125250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:47.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.077+0000 7feef3fff700 1 --2- 172.21.15.16:0/1740089674 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7feef412db40 0x7feef412ff30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7feeec009e90 tx=0x7feeec009780).ready entity=mon.2 client_cookie=d8fd364e4bff4d42 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:47.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.077+0000 7feef27fc700 1 -- 172.21.15.16:0/1740089674 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeec00e070 con 0x7feef412db40 2023-09-30T22:26:47.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.077+0000 7feef27fc700 1 -- 172.21.15.16:0/1740089674 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeec00bb60 con 0x7feef412db40 2023-09-30T22:26:47.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.077+0000 7feef27fc700 1 -- 172.21.15.16:0/1740089674 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeec012410 con 0x7feef412db40 2023-09-30T22:26:47.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.078+0000 7feefa216700 1 -- 172.21.15.16:0/1740089674 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7feef412db40 msgr2=0x7feef412ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:47.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.078+0000 7feefa216700 1 --2- 172.21.15.16:0/1740089674 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7feef412db40 0x7feef412ff30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7feeec009e90 tx=0x7feeec009780).stop 2023-09-30T22:26:47.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.078+0000 7feefa216700 1 -- 172.21.15.16:0/1740089674 shutdown_connections 2023-09-30T22:26:47.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.078+0000 7feefa216700 1 --2- 172.21.15.16:0/1740089674 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7feef412db40 0x7feef412ff30 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:47.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.078+0000 7feefa216700 1 --2- 172.21.15.16:0/1740089674 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7feef4059070 0x7feef4125250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:47.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.078+0000 7feefa216700 1 -- 172.21.15.16:0/1740089674 >> 172.21.15.16:0/1740089674 conn(0x7feef4097480 msgr2=0x7feef4097890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:47.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.078+0000 7feefa216700 1 -- 172.21.15.16:0/1740089674 shutdown_connections 2023-09-30T22:26:47.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.078+0000 7feefa216700 1 -- 172.21.15.16:0/1740089674 wait complete. 2023-09-30T22:26:47.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.079+0000 7feefa216700 1 Processor -- start 2023-09-30T22:26:47.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.079+0000 7feefa216700 1 -- start start 2023-09-30T22:26:47.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.079+0000 7feefa216700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feef4059070 0x7feef4069370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:47.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.079+0000 7feefa216700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7feef41257f0 0x7feef412bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:47.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.079+0000 7feefa216700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7feef412db40 0x7feef412c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:47.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.079+0000 7feefa216700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef406a000 con 0x7feef4059070 2023-09-30T22:26:47.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.079+0000 7feefa216700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feef4069e10 con 0x7feef41257f0 2023-09-30T22:26:47.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.079+0000 7feefa216700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feef4098ea0 con 0x7feef412db40 2023-09-30T22:26:47.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.079+0000 7feef37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feef4059070 0x7feef4069370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:47.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.080+0000 7feef37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feef4059070 0x7feef4069370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53068/0 (socket says 172.21.15.16:53068) 2023-09-30T22:26:47.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.080+0000 7feef37fe700 1 -- 172.21.15.16:0/3133955590 learned_addr learned my addr 172.21.15.16:0/3133955590 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:26:47.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.080+0000 7feef2ffd700 1 --2- 172.21.15.16:0/3133955590 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7feef41257f0 0x7feef412bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:47.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.080+0000 7feef3fff700 1 --2- 172.21.15.16:0/3133955590 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7feef412db40 0x7feef412c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:47.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.080+0000 7feef37fe700 1 -- 172.21.15.16:0/3133955590 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7feef412db40 msgr2=0x7feef412c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:47.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.080+0000 7feef37fe700 1 --2- 172.21.15.16:0/3133955590 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7feef412db40 0x7feef412c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:47.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.080+0000 7feef37fe700 1 -- 172.21.15.16:0/3133955590 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7feef41257f0 msgr2=0x7feef412bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:47.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.080+0000 7feef37fe700 1 --2- 172.21.15.16:0/3133955590 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7feef41257f0 0x7feef412bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:47.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.080+0000 7feef37fe700 1 -- 172.21.15.16:0/3133955590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee40092a0 con 0x7feef4059070 2023-09-30T22:26:47.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.080+0000 7feef2ffd700 1 --2- 172.21.15.16:0/3133955590 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7feef41257f0 0x7feef412bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:47.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.080+0000 7feef37fe700 1 --2- 172.21.15.16:0/3133955590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feef4059070 0x7feef4069370 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7feee400ea20 tx=0x7feee4009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:47.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.080+0000 7feedb7fe700 1 -- 172.21.15.16:0/3133955590 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee400ee60 con 0x7feef4059070 2023-09-30T22:26:47.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.080+0000 7feedb7fe700 1 -- 172.21.15.16:0/3133955590 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feee4009600 con 0x7feef4059070 2023-09-30T22:26:47.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.080+0000 7feefa216700 1 -- 172.21.15.16:0/3133955590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feeec009a20 con 0x7feef4059070 2023-09-30T22:26:47.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.080+0000 7feedb7fe700 1 -- 172.21.15.16:0/3133955590 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee400ca60 con 0x7feef4059070 2023-09-30T22:26:47.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.080+0000 7feefa216700 1 -- 172.21.15.16:0/3133955590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feef412ab40 con 0x7feef4059070 2023-09-30T22:26:47.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.081+0000 7feedb7fe700 1 -- 172.21.15.16:0/3133955590 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7feee401b400 con 0x7feef4059070 2023-09-30T22:26:47.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.081+0000 7feedb7fe700 1 -- 172.21.15.16:0/3133955590 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 2838+0+0 (secure 0 0 0) 0x7feee403b110 con 0x7feef4059070 2023-09-30T22:26:47.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.082+0000 7feefa216700 1 -- 172.21.15.16:0/3133955590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feef405a260 con 0x7feef4059070 2023-09-30T22:26:47.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.086+0000 7feedb7fe700 1 -- 172.21.15.16:0/3133955590 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7feee4011d20 con 0x7feef4059070 2023-09-30T22:26:47.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.262+0000 7feefa216700 1 -- 172.21.15.16:0/3133955590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7feef409a3e0 con 0x7feef4059070 2023-09-30T22:26:47.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.262+0000 7feedb7fe700 1 -- 172.21.15.16:0/3133955590 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v12) v1 ==== 92+0+10 (secure 0 0 0) 0x7feee4016020 con 0x7feef4059070 2023-09-30T22:26:47.263 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 8 2023-09-30T22:26:47.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.264+0000 7feefa216700 1 -- 172.21.15.16:0/3133955590 >> v1:172.21.15.16:6816/118285 conn(0x7feee002bb90 legacy=0x7feee002e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:47.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.264+0000 7feefa216700 1 -- 172.21.15.16:0/3133955590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feef4059070 msgr2=0x7feef4069370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:47.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.264+0000 7feefa216700 1 --2- 172.21.15.16:0/3133955590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feef4059070 0x7feef4069370 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7feee400ea20 tx=0x7feee4009060).stop 2023-09-30T22:26:47.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.264+0000 7feefa216700 1 -- 172.21.15.16:0/3133955590 shutdown_connections 2023-09-30T22:26:47.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.264+0000 7feefa216700 1 --2- 172.21.15.16:0/3133955590 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7feef412db40 0x7feef412c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:47.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.264+0000 7feefa216700 1 --2- 172.21.15.16:0/3133955590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feef4059070 0x7feef4069370 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:47.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.264+0000 7feefa216700 1 --2- 172.21.15.16:0/3133955590 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7feef41257f0 0x7feef412bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:47.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.264+0000 7feefa216700 1 -- 172.21.15.16:0/3133955590 >> 172.21.15.16:0/3133955590 conn(0x7feef4097480 msgr2=0x7feef412f780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:47.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.264+0000 7feefa216700 1 -- 172.21.15.16:0/3133955590 shutdown_connections 2023-09-30T22:26:47.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.264+0000 7feefa216700 1 -- 172.21.15.16:0/3133955590 wait complete. 2023-09-30T22:26:47.277 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-30T22:26:47.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.407+0000 7feaa08d7700 1 Processor -- start 2023-09-30T22:26:47.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.407+0000 7feaa08d7700 1 -- start start 2023-09-30T22:26:47.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.408+0000 7feaa08d7700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fea9c0613a0 0x7fea9c12ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:47.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.408+0000 7feaa08d7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea9c12b6d0 0x7fea9c096500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:47.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.408+0000 7feaa08d7700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea9c12abd0 con 0x7fea9c12b2f0 2023-09-30T22:26:47.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.408+0000 7feaa08d7700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea9c061780 con 0x7fea9c0613a0 2023-09-30T22:26:47.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.408+0000 7feaa08d7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea9c067530 con 0x7fea9c12b6d0 2023-09-30T22:26:47.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.408+0000 7fea9a59c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fea9c0613a0 0x7fea9c12ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:47.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.408+0000 7fea9ad9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea9c12b6d0 0x7fea9c096500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:47.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.408+0000 7fea99d9b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fea9c12b2f0 0x7fea9c095df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33626/0 (socket says 172.21.15.16:33626) 2023-09-30T22:26:47.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.408+0000 7fea99d9b700 1 -- 172.21.15.16:0/4240079714 learned_addr learned my addr 172.21.15.16:0/4240079714 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:26:47.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.409+0000 7fea9ad9d700 1 -- 172.21.15.16:0/4240079714 >> v2:172.21.15.16:3300/0 conn(0x7fea9c0613a0 msgr2=0x7fea9c12ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:47.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.409+0000 7fea9ad9d700 1 --2- 172.21.15.16:0/4240079714 >> v2:172.21.15.16:3300/0 conn(0x7fea9c0613a0 0x7fea9c12ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:47.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.409+0000 7fea9ad9d700 1 -- 172.21.15.16:0/4240079714 >> v1:172.21.15.16:6789/0 conn(0x7fea9c12b2f0 legacy=0x7fea9c095df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:47.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.409+0000 7fea9ad9d700 1 -- 172.21.15.16:0/4240079714 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea9c0683c0 con 0x7fea9c12b6d0 2023-09-30T22:26:47.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.409+0000 7fea9a59c700 1 --2- 172.21.15.16:0/4240079714 >> v2:172.21.15.16:3300/0 conn(0x7fea9c0613a0 0x7fea9c12ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:47.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.409+0000 7fea9ad9d700 1 --2- 172.21.15.16:0/4240079714 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea9c12b6d0 0x7fea9c096500 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7fea94009e90 tx=0x7fea94009780).ready entity=mon.1 client_cookie=3c901ad73f68a30 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:47.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.409+0000 7fea9959a700 1 -- 172.21.15.16:0/4240079714 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea9400e070 con 0x7fea9c12b6d0 2023-09-30T22:26:47.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.409+0000 7fea9959a700 1 -- 172.21.15.16:0/4240079714 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea9400bb20 con 0x7fea9c12b6d0 2023-09-30T22:26:47.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.409+0000 7fea9959a700 1 -- 172.21.15.16:0/4240079714 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea9400bea0 con 0x7fea9c12b6d0 2023-09-30T22:26:47.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.410+0000 7feaa08d7700 1 -- 172.21.15.16:0/4240079714 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea9c12b6d0 msgr2=0x7fea9c096500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:47.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.410+0000 7feaa08d7700 1 --2- 172.21.15.16:0/4240079714 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea9c12b6d0 0x7fea9c096500 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7fea94009e90 tx=0x7fea94009780).stop 2023-09-30T22:26:47.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.410+0000 7feaa08d7700 1 -- 172.21.15.16:0/4240079714 shutdown_connections 2023-09-30T22:26:47.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.410+0000 7feaa08d7700 1 --2- 172.21.15.16:0/4240079714 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea9c12b6d0 0x7fea9c096500 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:47.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.410+0000 7feaa08d7700 1 --2- 172.21.15.16:0/4240079714 >> v2:172.21.15.16:3300/0 conn(0x7fea9c0613a0 0x7fea9c12ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:47.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.410+0000 7feaa08d7700 1 -- 172.21.15.16:0/4240079714 >> 172.21.15.16:0/4240079714 conn(0x7fea9c09a220 msgr2=0x7fea9c09a630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:47.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.410+0000 7feaa08d7700 1 -- 172.21.15.16:0/4240079714 shutdown_connections 2023-09-30T22:26:47.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.410+0000 7feaa08d7700 1 -- 172.21.15.16:0/4240079714 wait complete. 2023-09-30T22:26:47.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.411+0000 7feaa08d7700 1 Processor -- start 2023-09-30T22:26:47.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.411+0000 7feaa08d7700 1 -- start start 2023-09-30T22:26:47.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.412+0000 7feaa08d7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea9c0613a0 0x7fea9c119110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:47.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.412+0000 7feaa08d7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea9c12b2f0 0x7fea9c1159a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:47.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.412+0000 7feaa08d7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea9c12b6d0 0x7fea9c115e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:47.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.412+0000 7feaa08d7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea9c129b20 con 0x7fea9c12b2f0 2023-09-30T22:26:47.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.412+0000 7feaa08d7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea9c067130 con 0x7fea9c0613a0 2023-09-30T22:26:47.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.412+0000 7feaa08d7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fea9c118670 con 0x7fea9c12b6d0 2023-09-30T22:26:47.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.412+0000 7fea9a59c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea9c0613a0 0x7fea9c119110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:47.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.412+0000 7fea9a59c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea9c0613a0 0x7fea9c119110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:44424/0 (socket says 172.21.15.16:44424) 2023-09-30T22:26:47.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.412+0000 7fea99d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea9c12b2f0 0x7fea9c1159a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:47.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.412+0000 7fea9a59c700 1 -- 172.21.15.16:0/253265194 learned_addr learned my addr 172.21.15.16:0/253265194 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:26:47.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.412+0000 7fea9ad9d700 1 --2- 172.21.15.16:0/253265194 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea9c12b6d0 0x7fea9c115e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:47.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.412+0000 7fea9a59c700 1 -- 172.21.15.16:0/253265194 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea9c12b6d0 msgr2=0x7fea9c115e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:47.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.412+0000 7fea9a59c700 1 --2- 172.21.15.16:0/253265194 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea9c12b6d0 0x7fea9c115e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:47.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.412+0000 7fea9a59c700 1 -- 172.21.15.16:0/253265194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea9c12b2f0 msgr2=0x7fea9c1159a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:47.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.412+0000 7fea9a59c700 1 --2- 172.21.15.16:0/253265194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea9c12b2f0 0x7fea9c1159a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:47.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.412+0000 7fea9a59c700 1 -- 172.21.15.16:0/253265194 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea8c0092a0 con 0x7fea9c0613a0 2023-09-30T22:26:47.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.412+0000 7fea99d9b700 1 --2- 172.21.15.16:0/253265194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea9c12b2f0 0x7fea9c1159a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:47.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.412+0000 7fea9a59c700 1 --2- 172.21.15.16:0/253265194 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea9c0613a0 0x7fea9c119110 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fea8c00ea20 tx=0x7fea8c009060).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:47.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.413+0000 7fea8a7fc700 1 -- 172.21.15.16:0/253265194 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8c00ee60 con 0x7fea9c0613a0 2023-09-30T22:26:47.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.413+0000 7fea8a7fc700 1 -- 172.21.15.16:0/253265194 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea8c00c850 con 0x7fea9c0613a0 2023-09-30T22:26:47.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.413+0000 7feaa08d7700 1 -- 172.21.15.16:0/253265194 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea94009a20 con 0x7fea9c0613a0 2023-09-30T22:26:47.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.413+0000 7fea8a7fc700 1 -- 172.21.15.16:0/253265194 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8c00cbd0 con 0x7fea9c0613a0 2023-09-30T22:26:47.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.413+0000 7feaa08d7700 1 -- 172.21.15.16:0/253265194 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea9c115600 con 0x7fea9c0613a0 2023-09-30T22:26:47.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.414+0000 7feaa08d7700 1 -- 172.21.15.16:0/253265194 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea9c067730 con 0x7fea9c0613a0 2023-09-30T22:26:47.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.414+0000 7fea8a7fc700 1 -- 172.21.15.16:0/253265194 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fea8c01b400 con 0x7fea9c0613a0 2023-09-30T22:26:47.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.414+0000 7fea8a7fc700 1 -- 172.21.15.16:0/253265194 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2838+0+0 (secure 0 0 0) 0x7fea8c03b770 con 0x7fea9c0613a0 2023-09-30T22:26:47.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.419+0000 7fea8a7fc700 1 -- 172.21.15.16:0/253265194 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fea8c011760 con 0x7fea9c0613a0 2023-09-30T22:26:47.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.636+0000 7feaa08d7700 1 -- 172.21.15.16:0/253265194 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fea9c069430 con 0x7fea9c0613a0 2023-09-30T22:26:47.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.637+0000 7fea8a7fc700 1 -- 172.21.15.16:0/253265194 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1196 (secure 0 0 0) 0x7fea8c029070 con 0x7fea9c0613a0 2023-09-30T22:26:47.637 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:26:47.637 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4218,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6817/1424366610","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6817","nonce":1424366610}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-09-30T22:26:47.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.639+0000 7feaa08d7700 1 -- 172.21.15.16:0/253265194 >> v1:172.21.15.16:6816/118285 conn(0x7fea7c02bb90 legacy=0x7fea7c02e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:47.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.639+0000 7feaa08d7700 1 -- 172.21.15.16:0/253265194 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea9c0613a0 msgr2=0x7fea9c119110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:47.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.639+0000 7feaa08d7700 1 --2- 172.21.15.16:0/253265194 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea9c0613a0 0x7fea9c119110 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fea8c00ea20 tx=0x7fea8c009060).stop 2023-09-30T22:26:47.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.639+0000 7feaa08d7700 1 -- 172.21.15.16:0/253265194 shutdown_connections 2023-09-30T22:26:47.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.639+0000 7feaa08d7700 1 --2- 172.21.15.16:0/253265194 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea9c0613a0 0x7fea9c119110 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:47.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.639+0000 7feaa08d7700 1 --2- 172.21.15.16:0/253265194 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea9c12b6d0 0x7fea9c115e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:47.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.639+0000 7feaa08d7700 1 --2- 172.21.15.16:0/253265194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea9c12b2f0 0x7fea9c1159a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:47.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.639+0000 7feaa08d7700 1 -- 172.21.15.16:0/253265194 >> 172.21.15.16:0/253265194 conn(0x7fea9c09a220 msgr2=0x7fea9c12e040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:47.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.639+0000 7feaa08d7700 1 -- 172.21.15.16:0/253265194 shutdown_connections 2023-09-30T22:26:47.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.639+0000 7feaa08d7700 1 -- 172.21.15.16:0/253265194 wait complete. 2023-09-30T22:26:47.641 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 2 2023-09-30T22:26:47.652 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:26:47.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.781+0000 7fcf3dcda700 1 Processor -- start 2023-09-30T22:26:47.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.781+0000 7fcf3dcda700 1 -- start start 2023-09-30T22:26:47.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.782+0000 7fcf3dcda700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcf38127390 0x7fcf3812fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:47.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.782+0000 7fcf3dcda700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcf38127fa0 0x7fcf38132170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:47.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.782+0000 7fcf3dcda700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf38091b60 con 0x7fcf38059070 2023-09-30T22:26:47.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.782+0000 7fcf3dcda700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf38096940 con 0x7fcf38127390 2023-09-30T22:26:47.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.782+0000 7fcf3dcda700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcf38059c00 con 0x7fcf38127fa0 2023-09-30T22:26:47.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.782+0000 7fcf36ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcf38127390 0x7fcf3812fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:47.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.782+0000 7fcf37fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcf38127fa0 0x7fcf38132170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:47.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.782+0000 7fcf377fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fcf38059070 0x7fcf38127890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33632/0 (socket says 172.21.15.16:33632) 2023-09-30T22:26:47.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.782+0000 7fcf377fe700 1 -- 172.21.15.16:0/2793494558 learned_addr learned my addr 172.21.15.16:0/2793494558 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:26:47.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.782+0000 7fcf36ffd700 1 -- 172.21.15.16:0/2793494558 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcf38127fa0 msgr2=0x7fcf38132170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:47.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.782+0000 7fcf36ffd700 1 --2- 172.21.15.16:0/2793494558 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcf38127fa0 0x7fcf38132170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:47.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.782+0000 7fcf36ffd700 1 -- 172.21.15.16:0/2793494558 >> v1:172.21.15.16:6789/0 conn(0x7fcf38059070 legacy=0x7fcf38127890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:47.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.782+0000 7fcf36ffd700 1 -- 172.21.15.16:0/2793494558 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf380929d0 con 0x7fcf38127390 2023-09-30T22:26:47.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.782+0000 7fcf367fc700 1 -- 172.21.15.16:0/2793494558 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3931734468 0 0) 0x7fcf38091b60 con 0x7fcf38059070 2023-09-30T22:26:47.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.783+0000 7fcf36ffd700 1 --2- 172.21.15.16:0/2793494558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf38127390 0x7fcf3812fc70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fcf20009760 tx=0x7fcf20009060).ready entity=mon.0 client_cookie=414d44556a4e8926 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:47.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.783+0000 7fcf367fc700 1 -- 172.21.15.16:0/2793494558 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf200090d0 con 0x7fcf38127390 2023-09-30T22:26:47.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.783+0000 7fcf367fc700 1 -- 172.21.15.16:0/2793494558 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf2000f510 con 0x7fcf38127390 2023-09-30T22:26:47.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.783+0000 7fcf367fc700 1 -- 172.21.15.16:0/2793494558 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2000f890 con 0x7fcf38127390 2023-09-30T22:26:47.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.783+0000 7fcf3dcda700 1 -- 172.21.15.16:0/2793494558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf38127390 msgr2=0x7fcf3812fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:47.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.783+0000 7fcf3dcda700 1 --2- 172.21.15.16:0/2793494558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf38127390 0x7fcf3812fc70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fcf20009760 tx=0x7fcf20009060).stop 2023-09-30T22:26:47.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.783+0000 7fcf3dcda700 1 -- 172.21.15.16:0/2793494558 shutdown_connections 2023-09-30T22:26:47.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.783+0000 7fcf3dcda700 1 --2- 172.21.15.16:0/2793494558 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcf38127fa0 0x7fcf38132170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:47.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.783+0000 7fcf3dcda700 1 --2- 172.21.15.16:0/2793494558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf38127390 0x7fcf3812fc70 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:47.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.783+0000 7fcf3dcda700 1 -- 172.21.15.16:0/2793494558 >> 172.21.15.16:0/2793494558 conn(0x7fcf38097470 msgr2=0x7fcf38097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:47.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.783+0000 7fcf3dcda700 1 -- 172.21.15.16:0/2793494558 shutdown_connections 2023-09-30T22:26:47.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.783+0000 7fcf3dcda700 1 -- 172.21.15.16:0/2793494558 wait complete. 2023-09-30T22:26:47.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.784+0000 7fcf3dcda700 1 Processor -- start 2023-09-30T22:26:47.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.785+0000 7fcf3dcda700 1 -- start start 2023-09-30T22:26:47.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.785+0000 7fcf3dcda700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcf38059070 0x7fcf381248d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:47.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.785+0000 7fcf3dcda700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcf38127390 0x7fcf3812ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:47.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.785+0000 7fcf3dcda700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf38127fa0 0x7fcf3812bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:47.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.785+0000 7fcf3dcda700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf38131b70 con 0x7fcf38127fa0 2023-09-30T22:26:47.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.785+0000 7fcf3dcda700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf381317c0 con 0x7fcf38059070 2023-09-30T22:26:47.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.785+0000 7fcf3dcda700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcf38096f20 con 0x7fcf38127390 2023-09-30T22:26:47.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.785+0000 7fcf36ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcf38127390 0x7fcf3812ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:47.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.785+0000 7fcf37fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf38127fa0 0x7fcf3812bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:47.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.785+0000 7fcf377fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcf38059070 0x7fcf381248d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:47.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.785+0000 7fcf36ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcf38127390 0x7fcf3812ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:46098/0 (socket says 172.21.15.16:46098) 2023-09-30T22:26:47.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.785+0000 7fcf36ffd700 1 -- 172.21.15.16:0/3130981299 learned_addr learned my addr 172.21.15.16:0/3130981299 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:26:47.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.785+0000 7fcf37fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf38127fa0 0x7fcf3812bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38528/0 (socket says 172.21.15.16:38528) 2023-09-30T22:26:47.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.785+0000 7fcf36ffd700 1 -- 172.21.15.16:0/3130981299 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcf38059070 msgr2=0x7fcf381248d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:47.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.785+0000 7fcf36ffd700 1 --2- 172.21.15.16:0/3130981299 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcf38059070 0x7fcf381248d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:47.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.785+0000 7fcf36ffd700 1 -- 172.21.15.16:0/3130981299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf38127fa0 msgr2=0x7fcf3812bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:47.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.785+0000 7fcf36ffd700 1 --2- 172.21.15.16:0/3130981299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf38127fa0 0x7fcf3812bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:47.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.785+0000 7fcf36ffd700 1 -- 172.21.15.16:0/3130981299 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf200092a0 con 0x7fcf38127390 2023-09-30T22:26:47.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.785+0000 7fcf377fe700 1 --2- 172.21.15.16:0/3130981299 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcf38059070 0x7fcf381248d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:47.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.786+0000 7fcf37fff700 1 --2- 172.21.15.16:0/3130981299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf38127fa0 0x7fcf3812bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:47.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.786+0000 7fcf36ffd700 1 --2- 172.21.15.16:0/3130981299 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcf38127390 0x7fcf3812ba40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fcf20009190 tx=0x7fcf20017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:47.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.786+0000 7fcf2f7fe700 1 -- 172.21.15.16:0/3130981299 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2000f890 con 0x7fcf38127390 2023-09-30T22:26:47.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.786+0000 7fcf2f7fe700 1 -- 172.21.15.16:0/3130981299 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf20006930 con 0x7fcf38127390 2023-09-30T22:26:47.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.786+0000 7fcf3dcda700 1 -- 172.21.15.16:0/3130981299 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf3812ec40 con 0x7fcf38127390 2023-09-30T22:26:47.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.786+0000 7fcf2f7fe700 1 -- 172.21.15.16:0/3130981299 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2000fea0 con 0x7fcf38127390 2023-09-30T22:26:47.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.786+0000 7fcf3dcda700 1 -- 172.21.15.16:0/3130981299 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf3812c420 con 0x7fcf38127390 2023-09-30T22:26:47.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.787+0000 7fcf2f7fe700 1 -- 172.21.15.16:0/3130981299 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fcf2001b620 con 0x7fcf38127390 2023-09-30T22:26:47.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.787+0000 7fcf2f7fe700 1 -- 172.21.15.16:0/3130981299 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2838+0+0 (secure 0 0 0) 0x7fcf2003c840 con 0x7fcf38127390 2023-09-30T22:26:47.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.787+0000 7fcf3dcda700 1 -- 172.21.15.16:0/3130981299 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf38064f80 con 0x7fcf38127390 2023-09-30T22:26:47.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.792+0000 7fcf2f7fe700 1 -- 172.21.15.16:0/3130981299 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fcf2001f020 con 0x7fcf38127390 2023-09-30T22:26:47.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.967+0000 7fcf3dcda700 1 -- 172.21.15.16:0/3130981299 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcf3812e380 con 0x7fcf38127390 2023-09-30T22:26:47.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.968+0000 7fcf2f7fe700 1 -- 172.21.15.16:0/3130981299 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+8124 (secure 0 0 0) 0x7fcf200193c0 con 0x7fcf38127390 2023-09-30T22:26:47.968 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:26:47.968 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":12,"fsid":"7ede3e14-2bd0-4726-9437-6d13c90cd342","created":"2023-09-30T22:26:35.483697+0000","modified":"2023-09-30T22:26:47.242248+0000","last_up_change":"2023-09-30T22:26:44.227199+0000","last_in_change":"2023-09-30T22:26:37.120906+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-09-30T22:26:44.285133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:26:45.132921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"19946985-0ee2-487f-9347-929670dbe5ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6804","nonce":118824}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6805","nonce":118824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6807","nonce":118824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6806","nonce":118824}]},"public_addr":"172.21.15.16:6804/118824","cluster_addr":"172.21.15.16:6805/118824","heartbeat_back_addr":"172.21.15.16:6807/118824","heartbeat_front_addr":"172.21.15.16:6806/118824","state":["exists","up"]},{"osd":1,"uuid":"8efd4dbc-6ae4-4b82-b530-3b3eac2be445","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6812","nonce":118832}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6813","nonce":118832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6815","nonce":118832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6814","nonce":118832}]},"public_addr":"172.21.15.16:6812/118832","cluster_addr":"172.21.15.16:6813/118832","heartbeat_back_addr":"172.21.15.16:6815/118832","heartbeat_front_addr":"172.21.15.16:6814/118832","state":["exists","up"]},{"osd":2,"uuid":"b47bb1ce-27af-4c0c-868a-b76ebd0d9499","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6808","nonce":118840}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6809","nonce":118840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6811","nonce":118840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6810","nonce":118840}]},"public_addr":"172.21.15.16:6808/118840","cluster_addr":"172.21.15.16:6809/118840","heartbeat_back_addr":"172.21.15.16:6811/118840","heartbeat_front_addr":"172.21.15.16:6810/118840","state":["exists","up"]},{"osd":3,"uuid":"03b53d64-d10d-448a-a081-39ecceab589b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6800","nonce":118823}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6801","nonce":118823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6803","nonce":118823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6802","nonce":118823}]},"public_addr":"172.21.15.16:6800/118823","cluster_addr":"172.21.15.16:6801/118823","heartbeat_back_addr":"172.21.15.16:6803/118823","heartbeat_front_addr":"172.21.15.16:6802/118823","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.305953+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.307488+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.341023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.192071+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:26:47.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.970+0000 7fcf3dcda700 1 -- 172.21.15.16:0/3130981299 >> v1:172.21.15.16:6816/118285 conn(0x7fcf1802bb90 legacy=0x7fcf1802e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:47.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.970+0000 7fcf3dcda700 1 -- 172.21.15.16:0/3130981299 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcf38127390 msgr2=0x7fcf3812ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:47.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.970+0000 7fcf3dcda700 1 --2- 172.21.15.16:0/3130981299 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcf38127390 0x7fcf3812ba40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fcf20009190 tx=0x7fcf20017000).stop 2023-09-30T22:26:47.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.970+0000 7fcf3dcda700 1 -- 172.21.15.16:0/3130981299 shutdown_connections 2023-09-30T22:26:47.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.970+0000 7fcf3dcda700 1 --2- 172.21.15.16:0/3130981299 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcf38059070 0x7fcf381248d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:47.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.970+0000 7fcf3dcda700 1 --2- 172.21.15.16:0/3130981299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf38127fa0 0x7fcf3812bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:47.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.970+0000 7fcf3dcda700 1 --2- 172.21.15.16:0/3130981299 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcf38127390 0x7fcf3812ba40 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:47.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.970+0000 7fcf3dcda700 1 -- 172.21.15.16:0/3130981299 >> 172.21.15.16:0/3130981299 conn(0x7fcf38097470 msgr2=0x7fcf38098780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:47.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.970+0000 7fcf3dcda700 1 -- 172.21.15.16:0/3130981299 shutdown_connections 2023-09-30T22:26:47.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:47.970+0000 7fcf3dcda700 1 -- 172.21.15.16:0/3130981299 wait complete. 2023-09-30T22:26:47.983 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-09-30T22:26:44.285133+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-09-30T22:26:45.132921+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-09-30T22:26:47.983 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-09-30T22:26:48.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.116+0000 7fc0868da700 1 Processor -- start 2023-09-30T22:26:48.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.116+0000 7fc0868da700 1 -- start start 2023-09-30T22:26:48.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.117+0000 7fc0868da700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc080093840 0x7fc080098790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:48.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.117+0000 7fc0868da700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc080098c80 0x7fc080099060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:48.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.117+0000 7fc0868da700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc08012c150 0x7fc08012e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:48.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.117+0000 7fc0868da700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc080093c20 con 0x7fc08012c150 2023-09-30T22:26:48.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.117+0000 7fc0868da700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc080092b90 con 0x7fc080098c80 2023-09-30T22:26:48.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.117+0000 7fc0868da700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc080092d80 con 0x7fc080093840 2023-09-30T22:26:48.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.117+0000 7fc07ffff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc080093840 0x7fc080098790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:48.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.117+0000 7fc07f7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc080098c80 0x7fc080099060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:48.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.117+0000 7fc084e77700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc08012c150 0x7fc08012e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:48.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.117+0000 7fc07f7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc080098c80 0x7fc080099060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:44442/0 (socket says 172.21.15.16:44442) 2023-09-30T22:26:48.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.117+0000 7fc07ffff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc080093840 0x7fc080098790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:46106/0 (socket says 172.21.15.16:46106) 2023-09-30T22:26:48.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.117+0000 7fc084e77700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc08012c150 0x7fc08012e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38542/0 (socket says 172.21.15.16:38542) 2023-09-30T22:26:48.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.117+0000 7fc07f7fe700 1 -- 172.21.15.16:0/4106383554 learned_addr learned my addr 172.21.15.16:0/4106383554 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:26:48.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.117+0000 7fc084e77700 1 -- 172.21.15.16:0/4106383554 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc080093840 msgr2=0x7fc080098790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:48.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.117+0000 7fc084e77700 1 --2- 172.21.15.16:0/4106383554 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc080093840 0x7fc080098790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:48.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.117+0000 7fc084e77700 1 -- 172.21.15.16:0/4106383554 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc080098c80 msgr2=0x7fc080099060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:48.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.117+0000 7fc084e77700 1 --2- 172.21.15.16:0/4106383554 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc080098c80 0x7fc080099060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:48.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.117+0000 7fc084e77700 1 -- 172.21.15.16:0/4106383554 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc080099b70 con 0x7fc08012c150 2023-09-30T22:26:48.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.117+0000 7fc07f7fe700 1 --2- 172.21.15.16:0/4106383554 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc080098c80 0x7fc080099060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:48.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.117+0000 7fc084e77700 1 --2- 172.21.15.16:0/4106383554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc08012c150 0x7fc08012e540 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fc07800d0d0 tx=0x7fc07800b990).ready entity=mon.0 client_cookie=a78dd9ae0567af5b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:48.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.118+0000 7fc07effd700 1 -- 172.21.15.16:0/4106383554 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc078010070 con 0x7fc08012c150 2023-09-30T22:26:48.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.118+0000 7fc07effd700 1 -- 172.21.15.16:0/4106383554 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc07800b2e0 con 0x7fc08012c150 2023-09-30T22:26:48.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.118+0000 7fc07effd700 1 -- 172.21.15.16:0/4106383554 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc07800dc90 con 0x7fc08012c150 2023-09-30T22:26:48.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.118+0000 7fc0868da700 1 -- 172.21.15.16:0/4106383554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc08012c150 msgr2=0x7fc08012e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:48.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.118+0000 7fc0868da700 1 --2- 172.21.15.16:0/4106383554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc08012c150 0x7fc08012e540 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fc07800d0d0 tx=0x7fc07800b990).stop 2023-09-30T22:26:48.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.118+0000 7fc0868da700 1 -- 172.21.15.16:0/4106383554 shutdown_connections 2023-09-30T22:26:48.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.118+0000 7fc0868da700 1 --2- 172.21.15.16:0/4106383554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc08012c150 0x7fc08012e540 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:48.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.118+0000 7fc0868da700 1 --2- 172.21.15.16:0/4106383554 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc080093840 0x7fc080098790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:48.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.118+0000 7fc0868da700 1 --2- 172.21.15.16:0/4106383554 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc080098c80 0x7fc080099060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:48.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.118+0000 7fc0868da700 1 -- 172.21.15.16:0/4106383554 >> 172.21.15.16:0/4106383554 conn(0x7fc080097480 msgr2=0x7fc080097890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:48.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.118+0000 7fc0868da700 1 -- 172.21.15.16:0/4106383554 shutdown_connections 2023-09-30T22:26:48.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.118+0000 7fc0868da700 1 -- 172.21.15.16:0/4106383554 wait complete. 2023-09-30T22:26:48.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.119+0000 7fc0868da700 1 Processor -- start 2023-09-30T22:26:48.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.119+0000 7fc0868da700 1 -- start start 2023-09-30T22:26:48.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.120+0000 7fc0868da700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc080093840 0x7fc080115a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:48.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.120+0000 7fc0868da700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc080098c80 0x7fc080115f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:48.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.120+0000 7fc0868da700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc08012c150 0x7fc080116400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:48.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.120+0000 7fc0868da700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0801191f0 con 0x7fc080098c80 2023-09-30T22:26:48.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.120+0000 7fc0868da700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc08012bab0 con 0x7fc080093840 2023-09-30T22:26:48.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.120+0000 7fc0868da700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc080118ca0 con 0x7fc08012c150 2023-09-30T22:26:48.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.120+0000 7fc07ffff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc080093840 0x7fc080115a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:48.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.120+0000 7fc07ffff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc080093840 0x7fc080115a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:44446/0 (socket says 172.21.15.16:44446) 2023-09-30T22:26:48.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.120+0000 7fc084e77700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc08012c150 0x7fc080116400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:48.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.120+0000 7fc07ffff700 1 -- 172.21.15.16:0/2859831998 learned_addr learned my addr 172.21.15.16:0/2859831998 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:26:48.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.120+0000 7fc07f7fe700 1 --2- 172.21.15.16:0/2859831998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc080098c80 0x7fc080115f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:48.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.120+0000 7fc07ffff700 1 -- 172.21.15.16:0/2859831998 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc08012c150 msgr2=0x7fc080116400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:48.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.120+0000 7fc07ffff700 1 --2- 172.21.15.16:0/2859831998 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc08012c150 0x7fc080116400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:48.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.120+0000 7fc07ffff700 1 -- 172.21.15.16:0/2859831998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc080098c80 msgr2=0x7fc080115f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:48.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.120+0000 7fc07ffff700 1 --2- 172.21.15.16:0/2859831998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc080098c80 0x7fc080115f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:48.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.120+0000 7fc07ffff700 1 -- 172.21.15.16:0/2859831998 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc068009370 con 0x7fc080093840 2023-09-30T22:26:48.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.120+0000 7fc084e77700 1 --2- 172.21.15.16:0/2859831998 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc08012c150 0x7fc080116400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:48.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.120+0000 7fc07f7fe700 1 --2- 172.21.15.16:0/2859831998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc080098c80 0x7fc080115f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:48.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.120+0000 7fc07ffff700 1 --2- 172.21.15.16:0/2859831998 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc080093840 0x7fc080115a20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fc07000e9b0 tx=0x7fc070009060).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:48.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.121+0000 7fc077fff700 1 -- 172.21.15.16:0/2859831998 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc070009510 con 0x7fc080093840 2023-09-30T22:26:48.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.121+0000 7fc077fff700 1 -- 172.21.15.16:0/2859831998 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc070012600 con 0x7fc080093840 2023-09-30T22:26:48.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.121+0000 7fc0868da700 1 -- 172.21.15.16:0/2859831998 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc07800bc30 con 0x7fc080093840 2023-09-30T22:26:48.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.121+0000 7fc077fff700 1 -- 172.21.15.16:0/2859831998 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc070012980 con 0x7fc080093840 2023-09-30T22:26:48.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.121+0000 7fc0868da700 1 -- 172.21.15.16:0/2859831998 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc080117550 con 0x7fc080093840 2023-09-30T22:26:48.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.121+0000 7fc077fff700 1 -- 172.21.15.16:0/2859831998 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fc07001b400 con 0x7fc080093840 2023-09-30T22:26:48.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.122+0000 7fc077fff700 1 -- 172.21.15.16:0/2859831998 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2838+0+0 (secure 0 0 0) 0x7fc07003b130 con 0x7fc080093840 2023-09-30T22:26:48.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.122+0000 7fc0868da700 1 -- 172.21.15.16:0/2859831998 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc08005a260 con 0x7fc080093840 2023-09-30T22:26:48.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.126+0000 7fc077fff700 1 -- 172.21.15.16:0/2859831998 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fc070019310 con 0x7fc080093840 2023-09-30T22:26:48.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.302+0000 7fc0868da700 1 -- 172.21.15.16:0/2859831998 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fc080046c20 con 0x7fc080093840 2023-09-30T22:26:48.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.302+0000 7fc077fff700 1 -- 172.21.15.16:0/2859831998 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v12) v1 ==== 110+0+10 (secure 0 0 0) 0x7fc070023350 con 0x7fc080093840 2023-09-30T22:26:48.303 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-09-30T22:26:48.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.304+0000 7fc0868da700 1 -- 172.21.15.16:0/2859831998 >> v1:172.21.15.16:6816/118285 conn(0x7fc06002bb60 legacy=0x7fc06002e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:48.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.304+0000 7fc0868da700 1 -- 172.21.15.16:0/2859831998 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc080093840 msgr2=0x7fc080115a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:48.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.304+0000 7fc0868da700 1 --2- 172.21.15.16:0/2859831998 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc080093840 0x7fc080115a20 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fc07000e9b0 tx=0x7fc070009060).stop 2023-09-30T22:26:48.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.304+0000 7fc0868da700 1 -- 172.21.15.16:0/2859831998 shutdown_connections 2023-09-30T22:26:48.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.304+0000 7fc0868da700 1 --2- 172.21.15.16:0/2859831998 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc080093840 0x7fc080115a20 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:48.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.304+0000 7fc0868da700 1 --2- 172.21.15.16:0/2859831998 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc08012c150 0x7fc080116400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:48.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.304+0000 7fc0868da700 1 --2- 172.21.15.16:0/2859831998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc080098c80 0x7fc080115f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:48.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.304+0000 7fc0868da700 1 -- 172.21.15.16:0/2859831998 >> 172.21.15.16:0/2859831998 conn(0x7fc080097480 msgr2=0x7fc080069110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:48.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.304+0000 7fc0868da700 1 -- 172.21.15.16:0/2859831998 shutdown_connections 2023-09-30T22:26:48.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.304+0000 7fc0868da700 1 -- 172.21.15.16:0/2859831998 wait complete. 2023-09-30T22:26:48.317 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-09-30T22:26:48.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.451+0000 7fc562121700 1 Processor -- start 2023-09-30T22:26:48.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.451+0000 7fc562121700 1 -- start start 2023-09-30T22:26:48.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.451+0000 7fc562121700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc55c0694d0 0x7fc55c096410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:48.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.451+0000 7fc562121700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc55c096900 0x7fc55c096d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:48.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.451+0000 7fc562121700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc55c095e60 con 0x7fc55c12d230 2023-09-30T22:26:48.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.451+0000 7fc562121700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc55c068d40 con 0x7fc55c096900 2023-09-30T22:26:48.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.451+0000 7fc562121700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc55c068fb0 con 0x7fc55c0694d0 2023-09-30T22:26:48.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.452+0000 7fc55affd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc55c096900 0x7fc55c096d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:48.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.452+0000 7fc55b7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc55c0694d0 0x7fc55c096410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:48.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.452+0000 7fc55bfff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc55c12d230 0x7fc55c12f620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33636/0 (socket says 172.21.15.16:33636) 2023-09-30T22:26:48.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.452+0000 7fc55affd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc55c096900 0x7fc55c096d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38564/0 (socket says 172.21.15.16:38564) 2023-09-30T22:26:48.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.452+0000 7fc55bfff700 1 -- 172.21.15.16:0/1834002780 learned_addr learned my addr 172.21.15.16:0/1834002780 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:26:48.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.452+0000 7fc55b7fe700 1 --2- 172.21.15.16:0/1834002780 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc55c0694d0 0x7fc55c096410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:46114/0 (socket says 172.21.15.16:46114) 2023-09-30T22:26:48.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.452+0000 7fc55b7fe700 1 -- 172.21.15.16:0/1834002780 >> v2:172.21.15.16:3300/0 conn(0x7fc55c096900 msgr2=0x7fc55c096d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:48.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.452+0000 7fc55b7fe700 1 --2- 172.21.15.16:0/1834002780 >> v2:172.21.15.16:3300/0 conn(0x7fc55c096900 0x7fc55c096d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:48.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.452+0000 7fc55b7fe700 1 -- 172.21.15.16:0/1834002780 >> v1:172.21.15.16:6789/0 conn(0x7fc55c12d230 legacy=0x7fc55c12f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:48.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.452+0000 7fc55b7fe700 1 -- 172.21.15.16:0/1834002780 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc55c068a10 con 0x7fc55c0694d0 2023-09-30T22:26:48.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.452+0000 7fc55a7fc700 1 -- 172.21.15.16:0/1834002780 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 383485938 0 0) 0x7fc55c095e60 con 0x7fc55c12d230 2023-09-30T22:26:48.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.452+0000 7fc55affd700 1 --2- 172.21.15.16:0/1834002780 >> v2:172.21.15.16:3300/0 conn(0x7fc55c096900 0x7fc55c096d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:48.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.452+0000 7fc55b7fe700 1 --2- 172.21.15.16:0/1834002780 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc55c0694d0 0x7fc55c096410 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fc554009760 tx=0x7fc554009060).ready entity=mon.2 client_cookie=580a899a57773711 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:48.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.452+0000 7fc55a7fc700 1 -- 172.21.15.16:0/1834002780 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc55400f510 con 0x7fc55c0694d0 2023-09-30T22:26:48.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.452+0000 7fc55a7fc700 1 -- 172.21.15.16:0/1834002780 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc55400f670 con 0x7fc55c0694d0 2023-09-30T22:26:48.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.452+0000 7fc55a7fc700 1 -- 172.21.15.16:0/1834002780 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc55400f9f0 con 0x7fc55c0694d0 2023-09-30T22:26:48.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.453+0000 7fc562121700 1 -- 172.21.15.16:0/1834002780 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc55c0694d0 msgr2=0x7fc55c096410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:48.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.453+0000 7fc562121700 1 --2- 172.21.15.16:0/1834002780 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc55c0694d0 0x7fc55c096410 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fc554009760 tx=0x7fc554009060).stop 2023-09-30T22:26:48.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.453+0000 7fc562121700 1 -- 172.21.15.16:0/1834002780 shutdown_connections 2023-09-30T22:26:48.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.453+0000 7fc562121700 1 --2- 172.21.15.16:0/1834002780 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc55c0694d0 0x7fc55c096410 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:48.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.453+0000 7fc562121700 1 --2- 172.21.15.16:0/1834002780 >> v2:172.21.15.16:3300/0 conn(0x7fc55c096900 0x7fc55c096d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:48.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.453+0000 7fc562121700 1 -- 172.21.15.16:0/1834002780 >> 172.21.15.16:0/1834002780 conn(0x7fc55c09ba60 msgr2=0x7fc55c09be70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:48.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.453+0000 7fc562121700 1 -- 172.21.15.16:0/1834002780 shutdown_connections 2023-09-30T22:26:48.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.453+0000 7fc562121700 1 -- 172.21.15.16:0/1834002780 wait complete. 2023-09-30T22:26:48.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.454+0000 7fc562121700 1 Processor -- start 2023-09-30T22:26:48.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.454+0000 7fc562121700 1 -- start start 2023-09-30T22:26:48.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.455+0000 7fc562121700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc55c0694d0 0x7fc55c117e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:48.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.455+0000 7fc562121700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc55c096900 0x7fc55c118320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:48.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.455+0000 7fc562121700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc55c12d230 0x7fc55c118810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:48.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.455+0000 7fc562121700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc55c11afb0 con 0x7fc55c0694d0 2023-09-30T22:26:48.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.455+0000 7fc562121700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc55c091bf0 con 0x7fc55c096900 2023-09-30T22:26:48.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.455+0000 7fc562121700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc55c067d80 con 0x7fc55c12d230 2023-09-30T22:26:48.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.455+0000 7fc55b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc55c0694d0 0x7fc55c117e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:48.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.455+0000 7fc55b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc55c0694d0 0x7fc55c117e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38568/0 (socket says 172.21.15.16:38568) 2023-09-30T22:26:48.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.455+0000 7fc55b7fe700 1 -- 172.21.15.16:0/476034161 learned_addr learned my addr 172.21.15.16:0/476034161 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:26:48.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.455+0000 7fc55bfff700 1 --2- 172.21.15.16:0/476034161 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc55c12d230 0x7fc55c118810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:48.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.455+0000 7fc55affd700 1 --2- 172.21.15.16:0/476034161 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc55c096900 0x7fc55c118320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:48.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.455+0000 7fc55b7fe700 1 -- 172.21.15.16:0/476034161 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc55c12d230 msgr2=0x7fc55c118810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:48.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.455+0000 7fc55b7fe700 1 --2- 172.21.15.16:0/476034161 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc55c12d230 0x7fc55c118810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:48.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.455+0000 7fc55b7fe700 1 -- 172.21.15.16:0/476034161 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc55c096900 msgr2=0x7fc55c118320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:48.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.455+0000 7fc55b7fe700 1 --2- 172.21.15.16:0/476034161 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc55c096900 0x7fc55c118320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:48.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.455+0000 7fc55b7fe700 1 -- 172.21.15.16:0/476034161 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5440092a0 con 0x7fc55c0694d0 2023-09-30T22:26:48.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.455+0000 7fc55b7fe700 1 --2- 172.21.15.16:0/476034161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc55c0694d0 0x7fc55c117e30 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fc554009e00 tx=0x7fc554017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:48.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.456+0000 7fc5537fe700 1 -- 172.21.15.16:0/476034161 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc55400f7e0 con 0x7fc55c0694d0 2023-09-30T22:26:48.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.456+0000 7fc5537fe700 1 -- 172.21.15.16:0/476034161 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc55400f940 con 0x7fc55c0694d0 2023-09-30T22:26:48.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.456+0000 7fc562121700 1 -- 172.21.15.16:0/476034161 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5540092a0 con 0x7fc55c0694d0 2023-09-30T22:26:48.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.456+0000 7fc5537fe700 1 -- 172.21.15.16:0/476034161 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc55401b660 con 0x7fc55c0694d0 2023-09-30T22:26:48.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.456+0000 7fc562121700 1 -- 172.21.15.16:0/476034161 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc55c1191a0 con 0x7fc55c0694d0 2023-09-30T22:26:48.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.456+0000 7fc5537fe700 1 -- 172.21.15.16:0/476034161 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fc55401ba10 con 0x7fc55c0694d0 2023-09-30T22:26:48.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.457+0000 7fc5537fe700 1 -- 172.21.15.16:0/476034161 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2838+0+0 (secure 0 0 0) 0x7fc55403c500 con 0x7fc55c0694d0 2023-09-30T22:26:48.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.457+0000 7fc562121700 1 -- 172.21.15.16:0/476034161 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc55c05a260 con 0x7fc55c0694d0 2023-09-30T22:26:48.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.461+0000 7fc5537fe700 1 -- 172.21.15.16:0/476034161 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fc55401f080 con 0x7fc55c0694d0 2023-09-30T22:26:48.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.634+0000 7fc562121700 1 -- 172.21.15.16:0/476034161 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc55c092f90 con 0x7fc55c0694d0 2023-09-30T22:26:48.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.635+0000 7fc5537fe700 1 -- 172.21.15.16:0/476034161 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v12) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc554029020 con 0x7fc55c0694d0 2023-09-30T22:26:48.635 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 8 2023-09-30T22:26:48.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.637+0000 7fc562121700 1 -- 172.21.15.16:0/476034161 >> v1:172.21.15.16:6816/118285 conn(0x7fc53c02bb60 legacy=0x7fc53c02e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:48.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.637+0000 7fc562121700 1 -- 172.21.15.16:0/476034161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc55c0694d0 msgr2=0x7fc55c117e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:48.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.637+0000 7fc562121700 1 --2- 172.21.15.16:0/476034161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc55c0694d0 0x7fc55c117e30 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fc554009e00 tx=0x7fc554017000).stop 2023-09-30T22:26:48.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.637+0000 7fc562121700 1 -- 172.21.15.16:0/476034161 shutdown_connections 2023-09-30T22:26:48.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.637+0000 7fc562121700 1 --2- 172.21.15.16:0/476034161 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc55c12d230 0x7fc55c118810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:48.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.637+0000 7fc562121700 1 --2- 172.21.15.16:0/476034161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc55c0694d0 0x7fc55c117e30 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:48.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.637+0000 7fc562121700 1 --2- 172.21.15.16:0/476034161 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc55c096900 0x7fc55c118320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:48.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.637+0000 7fc562121700 1 -- 172.21.15.16:0/476034161 >> 172.21.15.16:0/476034161 conn(0x7fc55c09ba60 msgr2=0x7fc55c094c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:48.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.637+0000 7fc562121700 1 -- 172.21.15.16:0/476034161 shutdown_connections 2023-09-30T22:26:48.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.637+0000 7fc562121700 1 -- 172.21.15.16:0/476034161 wait complete. 2023-09-30T22:26:48.651 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-09-30T22:26:48.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.780+0000 7f2dff31c700 1 Processor -- start 2023-09-30T22:26:48.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.780+0000 7f2dff31c700 1 -- start start 2023-09-30T22:26:48.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.781+0000 7f2dff31c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2df8091b00 0x7f2df812c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:48.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.781+0000 7f2dff31c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2df812ce60 0x7f2df812f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:48.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.781+0000 7f2dff31c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2df8096760 con 0x7f2df8096380 2023-09-30T22:26:48.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.781+0000 7f2dff31c700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2df8091ff0 con 0x7f2df812ce60 2023-09-30T22:26:48.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.781+0000 7f2dff31c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2df8092c80 con 0x7f2df8091b00 2023-09-30T22:26:48.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.781+0000 7f2dfc8b7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2df8091b00 0x7f2df812c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:48.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.781+0000 7f2dfc8b7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2df8091b00 0x7f2df812c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:44474/0 (socket says 172.21.15.16:44474) 2023-09-30T22:26:48.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.781+0000 7f2dfd8b9700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2df812ce60 0x7f2df812f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:48.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.781+0000 7f2dfc8b7700 1 -- 172.21.15.16:0/3551491535 learned_addr learned my addr 172.21.15.16:0/3551491535 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:26:48.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.782+0000 7f2deffff700 1 -- 172.21.15.16:0/3551491535 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1937195353 0 0) 0x7f2df8096760 con 0x7f2df8096380 2023-09-30T22:26:48.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.782+0000 7f2deffff700 1 -- 172.21.15.16:0/3551491535 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2de40033f0 con 0x7f2df8096380 2023-09-30T22:26:48.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.782+0000 7f2dfd8b9700 1 -- 172.21.15.16:0/3551491535 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2df8091b00 msgr2=0x7f2df812c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:48.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.782+0000 7f2dfd8b9700 1 --2- 172.21.15.16:0/3551491535 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2df8091b00 0x7f2df812c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:48.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.782+0000 7f2dfd8b9700 1 -- 172.21.15.16:0/3551491535 >> v1:172.21.15.16:6789/0 conn(0x7f2df8096380 legacy=0x7f2df8094730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:48.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.782+0000 7f2dfd8b9700 1 -- 172.21.15.16:0/3551491535 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df8094e40 con 0x7f2df812ce60 2023-09-30T22:26:48.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.782+0000 7f2dfc8b7700 1 --2- 172.21.15.16:0/3551491535 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2df8091b00 0x7f2df812c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:48.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.782+0000 7f2dfd8b9700 1 --2- 172.21.15.16:0/3551491535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df812ce60 0x7f2df812f250 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f2df0009e60 tx=0x7f2df0009780).ready entity=mon.0 client_cookie=f6addd5546888a62 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:48.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.782+0000 7f2deffff700 1 -- 172.21.15.16:0/3551491535 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df000e070 con 0x7f2df812ce60 2023-09-30T22:26:48.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.782+0000 7f2deffff700 1 -- 172.21.15.16:0/3551491535 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2df000bab0 con 0x7f2df812ce60 2023-09-30T22:26:48.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.782+0000 7f2deffff700 1 -- 172.21.15.16:0/3551491535 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df000be30 con 0x7f2df812ce60 2023-09-30T22:26:48.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.783+0000 7f2dff31c700 1 -- 172.21.15.16:0/3551491535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df812ce60 msgr2=0x7f2df812f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:48.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.783+0000 7f2dff31c700 1 --2- 172.21.15.16:0/3551491535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df812ce60 0x7f2df812f250 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f2df0009e60 tx=0x7f2df0009780).stop 2023-09-30T22:26:48.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.783+0000 7f2dff31c700 1 -- 172.21.15.16:0/3551491535 shutdown_connections 2023-09-30T22:26:48.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.783+0000 7f2dff31c700 1 --2- 172.21.15.16:0/3551491535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df812ce60 0x7f2df812f250 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:48.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.783+0000 7f2dff31c700 1 --2- 172.21.15.16:0/3551491535 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2df8091b00 0x7f2df812c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:48.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.783+0000 7f2dff31c700 1 -- 172.21.15.16:0/3551491535 >> 172.21.15.16:0/3551491535 conn(0x7f2df8097470 msgr2=0x7f2df8097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:48.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.783+0000 7f2dff31c700 1 -- 172.21.15.16:0/3551491535 shutdown_connections 2023-09-30T22:26:48.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.783+0000 7f2dff31c700 1 -- 172.21.15.16:0/3551491535 wait complete. 2023-09-30T22:26:48.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.784+0000 7f2dff31c700 1 Processor -- start 2023-09-30T22:26:48.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.784+0000 7f2dff31c700 1 -- start start 2023-09-30T22:26:48.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.784+0000 7f2dff31c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2df8091b00 0x7f2df8124e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:48.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.784+0000 7f2dff31c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df8096380 0x7f2df8129580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:48.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.784+0000 7f2dff31c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2df812ce60 0x7f2df8129a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:48.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.784+0000 7f2dff31c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df806a520 con 0x7f2df8096380 2023-09-30T22:26:48.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.784+0000 7f2dff31c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2df812ca60 con 0x7f2df8091b00 2023-09-30T22:26:48.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.784+0000 7f2dff31c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2df8111ec0 con 0x7f2df812ce60 2023-09-30T22:26:48.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.785+0000 7f2dfc8b7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df8096380 0x7f2df8129580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:48.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.785+0000 7f2dfd0b8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2df8091b00 0x7f2df8124e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:48.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.785+0000 7f2dfd8b9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2df812ce60 0x7f2df8129a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:48.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.785+0000 7f2dfc8b7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df8096380 0x7f2df8129580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38580/0 (socket says 172.21.15.16:38580) 2023-09-30T22:26:48.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.785+0000 7f2dfc8b7700 1 -- 172.21.15.16:0/3289441475 learned_addr learned my addr 172.21.15.16:0/3289441475 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:26:48.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.785+0000 7f2dfc8b7700 1 -- 172.21.15.16:0/3289441475 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2df812ce60 msgr2=0x7f2df8129a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:48.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.785+0000 7f2dfc8b7700 1 --2- 172.21.15.16:0/3289441475 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2df812ce60 0x7f2df8129a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:48.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.785+0000 7f2dfc8b7700 1 -- 172.21.15.16:0/3289441475 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2df8091b00 msgr2=0x7f2df8124e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:48.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.785+0000 7f2dfc8b7700 1 --2- 172.21.15.16:0/3289441475 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2df8091b00 0x7f2df8124e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:48.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.785+0000 7f2dfc8b7700 1 -- 172.21.15.16:0/3289441475 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de800b610 con 0x7f2df8096380 2023-09-30T22:26:48.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.785+0000 7f2dfc8b7700 1 --2- 172.21.15.16:0/3289441475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df8096380 0x7f2df8129580 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f2de8010e90 tx=0x7f2de800b3b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:48.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.785+0000 7f2ded7fa700 1 -- 172.21.15.16:0/3289441475 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de800ed40 con 0x7f2df8096380 2023-09-30T22:26:48.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.785+0000 7f2ded7fa700 1 -- 172.21.15.16:0/3289441475 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2de801a470 con 0x7f2df8096380 2023-09-30T22:26:48.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.785+0000 7f2dff31c700 1 -- 172.21.15.16:0/3289441475 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2df0009a20 con 0x7f2df8096380 2023-09-30T22:26:48.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.785+0000 7f2ded7fa700 1 -- 172.21.15.16:0/3289441475 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de800eea0 con 0x7f2df8096380 2023-09-30T22:26:48.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.785+0000 7f2dff31c700 1 -- 172.21.15.16:0/3289441475 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2df812ba80 con 0x7f2df8096380 2023-09-30T22:26:48.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.786+0000 7f2ded7fa700 1 -- 172.21.15.16:0/3289441475 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f2de8017070 con 0x7f2df8096380 2023-09-30T22:26:48.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.786+0000 7f2ded7fa700 1 -- 172.21.15.16:0/3289441475 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2838+0+0 (secure 0 0 0) 0x7f2de803d9b0 con 0x7f2df8096380 2023-09-30T22:26:48.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.786+0000 7f2dff31c700 1 -- 172.21.15.16:0/3289441475 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2df81117a0 con 0x7f2df8096380 2023-09-30T22:26:48.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.791+0000 7f2ded7fa700 1 -- 172.21.15.16:0/3289441475 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f2df81117a0 con 0x7f2df8096380 2023-09-30T22:26:48.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.970+0000 7f2dff31c700 1 -- 172.21.15.16:0/3289441475 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f2df81117a0 con 0x7f2df8096380 2023-09-30T22:26:48.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.970+0000 7f2ded7fa700 1 -- 172.21.15.16:0/3289441475 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v12) v1 ==== 84+0+143 (secure 0 0 0) 0x7f2df81117a0 con 0x7f2df8096380 2023-09-30T22:26:48.970 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:26:48.971 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2023-09-30T22:26:48.971 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-09-30T22:26:48.971 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2023-09-30T22:26:48.971 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": "device_health_metrics" 2023-09-30T22:26:48.971 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-09-30T22:26:48.971 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-09-30T22:26:48.971 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 2, 2023-09-30T22:26:48.971 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": "rbd" 2023-09-30T22:26:48.971 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-09-30T22:26:48.971 INFO:teuthology.orchestra.run.smithi016.stdout:] 2023-09-30T22:26:48.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.972+0000 7f2dff31c700 1 -- 172.21.15.16:0/3289441475 >> v1:172.21.15.16:6816/118285 conn(0x7f2ddc02bb90 legacy=0x7f2ddc02e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:48.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.972+0000 7f2dff31c700 1 -- 172.21.15.16:0/3289441475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df8096380 msgr2=0x7f2df8129580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:48.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.972+0000 7f2dff31c700 1 --2- 172.21.15.16:0/3289441475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df8096380 0x7f2df8129580 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f2de8010e90 tx=0x7f2de800b3b0).stop 2023-09-30T22:26:48.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.972+0000 7f2dff31c700 1 -- 172.21.15.16:0/3289441475 shutdown_connections 2023-09-30T22:26:48.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.972+0000 7f2dff31c700 1 --2- 172.21.15.16:0/3289441475 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2df8091b00 0x7f2df8124e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:48.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.972+0000 7f2dff31c700 1 --2- 172.21.15.16:0/3289441475 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2df812ce60 0x7f2df8129a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:48.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.972+0000 7f2dff31c700 1 --2- 172.21.15.16:0/3289441475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df8096380 0x7f2df8129580 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:48.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.972+0000 7f2dff31c700 1 -- 172.21.15.16:0/3289441475 >> 172.21.15.16:0/3289441475 conn(0x7f2df8097470 msgr2=0x7f2df8127d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:48.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.972+0000 7f2dff31c700 1 -- 172.21.15.16:0/3289441475 shutdown_connections 2023-09-30T22:26:48.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:48.972+0000 7f2dff31c700 1 -- 172.21.15.16:0/3289441475 wait complete. 2023-09-30T22:26:48.985 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-09-30T22:26:48.985 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-09-30T22:26:49.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.119+0000 7f1076a43700 1 Processor -- start 2023-09-30T22:26:49.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.119+0000 7f1076a43700 1 -- start start 2023-09-30T22:26:49.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.119+0000 7f1076a43700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1070069370 0x7f1070096410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:49.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.119+0000 7f1076a43700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1070096900 0x7f1070096ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:49.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.119+0000 7f1076a43700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f107006a770 con 0x7f107012d2b0 2023-09-30T22:26:49.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.119+0000 7f1076a43700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1070069750 con 0x7f1070069370 2023-09-30T22:26:49.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.119+0000 7f1076a43700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1070069a30 con 0x7f1070096900 2023-09-30T22:26:49.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.119+0000 7f106f7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1070096900 0x7f1070096ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:49.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.119+0000 7f106ffff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1070069370 0x7f1070096410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:49.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.120+0000 7f106ffff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1070069370 0x7f1070096410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38596/0 (socket says 172.21.15.16:38596) 2023-09-30T22:26:49.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.120+0000 7f106f7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1070096900 0x7f1070096ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:46134/0 (socket says 172.21.15.16:46134) 2023-09-30T22:26:49.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.120+0000 7f106ffff700 1 -- 172.21.15.16:0/3806215230 learned_addr learned my addr 172.21.15.16:0/3806215230 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:26:49.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.120+0000 7f1074fe0700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f107012d2b0 0x7f107012f6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33652/0 (socket says 172.21.15.16:33652) 2023-09-30T22:26:49.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.120+0000 7f106ffff700 1 -- 172.21.15.16:0/3806215230 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1070096900 msgr2=0x7f1070096ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:49.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.120+0000 7f106ffff700 1 --2- 172.21.15.16:0/3806215230 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1070096900 0x7f1070096ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:49.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.120+0000 7f106ffff700 1 -- 172.21.15.16:0/3806215230 >> v1:172.21.15.16:6789/0 conn(0x7f107012d2b0 legacy=0x7f107012f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:49.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.120+0000 7f106ffff700 1 -- 172.21.15.16:0/3806215230 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1070097810 con 0x7f1070069370 2023-09-30T22:26:49.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.120+0000 7f106effd700 1 -- 172.21.15.16:0/3806215230 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2780446251 0 0) 0x7f107006a770 con 0x7f107012d2b0 2023-09-30T22:26:49.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.120+0000 7f106f7fe700 1 --2- 172.21.15.16:0/3806215230 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1070096900 0x7f1070096ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:49.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.120+0000 7f106ffff700 1 --2- 172.21.15.16:0/3806215230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1070069370 0x7f1070096410 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f1060006fd0 tx=0x7f10600090f0).ready entity=mon.0 client_cookie=94249cfa8eebf369 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:49.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.120+0000 7f106effd700 1 -- 172.21.15.16:0/3806215230 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106000b580 con 0x7f1070069370 2023-09-30T22:26:49.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.120+0000 7f106effd700 1 -- 172.21.15.16:0/3806215230 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f106000b6e0 con 0x7f1070069370 2023-09-30T22:26:49.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.120+0000 7f106effd700 1 -- 172.21.15.16:0/3806215230 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106000ba60 con 0x7f1070069370 2023-09-30T22:26:49.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.121+0000 7f1076a43700 1 -- 172.21.15.16:0/3806215230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1070069370 msgr2=0x7f1070096410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:49.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.121+0000 7f1076a43700 1 --2- 172.21.15.16:0/3806215230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1070069370 0x7f1070096410 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f1060006fd0 tx=0x7f10600090f0).stop 2023-09-30T22:26:49.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.121+0000 7f1076a43700 1 -- 172.21.15.16:0/3806215230 shutdown_connections 2023-09-30T22:26:49.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.121+0000 7f1076a43700 1 --2- 172.21.15.16:0/3806215230 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1070069370 0x7f1070096410 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:49.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.121+0000 7f1076a43700 1 --2- 172.21.15.16:0/3806215230 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1070096900 0x7f1070096ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:49.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.121+0000 7f1076a43700 1 -- 172.21.15.16:0/3806215230 >> 172.21.15.16:0/3806215230 conn(0x7f1070059cd0 msgr2=0x7f107009bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:49.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.121+0000 7f1076a43700 1 -- 172.21.15.16:0/3806215230 shutdown_connections 2023-09-30T22:26:49.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.121+0000 7f1076a43700 1 -- 172.21.15.16:0/3806215230 wait complete. 2023-09-30T22:26:49.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.122+0000 7f1076a43700 1 Processor -- start 2023-09-30T22:26:49.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.122+0000 7f1076a43700 1 -- start start 2023-09-30T22:26:49.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.123+0000 7f1076a43700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1070069370 0x7f107011df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:49.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.123+0000 7f1076a43700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1070096900 0x7f1070064540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:49.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.123+0000 7f1076a43700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f107012d2b0 0x7f1070064a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:49.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.123+0000 7f1076a43700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1070067bb0 con 0x7f1070096900 2023-09-30T22:26:49.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.123+0000 7f1076a43700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f107011d7c0 con 0x7f107012d2b0 2023-09-30T22:26:49.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.123+0000 7f1076a43700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f107011d0f0 con 0x7f1070069370 2023-09-30T22:26:49.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.123+0000 7f106ffff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1070069370 0x7f107011df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:49.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.123+0000 7f106ffff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1070069370 0x7f107011df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:46144/0 (socket says 172.21.15.16:46144) 2023-09-30T22:26:49.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.123+0000 7f106ffff700 1 -- 172.21.15.16:0/1876283677 learned_addr learned my addr 172.21.15.16:0/1876283677 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:26:49.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.123+0000 7f106f7fe700 1 --2- 172.21.15.16:0/1876283677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1070096900 0x7f1070064540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:49.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.123+0000 7f1074fe0700 1 --2- 172.21.15.16:0/1876283677 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f107012d2b0 0x7f1070064a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:49.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.123+0000 7f106ffff700 1 -- 172.21.15.16:0/1876283677 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f107012d2b0 msgr2=0x7f1070064a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:49.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.123+0000 7f106ffff700 1 --2- 172.21.15.16:0/1876283677 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f107012d2b0 0x7f1070064a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:49.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.123+0000 7f106ffff700 1 -- 172.21.15.16:0/1876283677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1070096900 msgr2=0x7f1070064540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:49.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.123+0000 7f106ffff700 1 --2- 172.21.15.16:0/1876283677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1070096900 0x7f1070064540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:49.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.123+0000 7f106ffff700 1 -- 172.21.15.16:0/1876283677 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10640092a0 con 0x7f1070069370 2023-09-30T22:26:49.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.123+0000 7f106f7fe700 1 --2- 172.21.15.16:0/1876283677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1070096900 0x7f1070064540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:49.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.123+0000 7f106ffff700 1 --2- 172.21.15.16:0/1876283677 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1070069370 0x7f107011df30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f1060009ea0 tx=0x7f1060017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:49.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.123+0000 7f1057fff700 1 -- 172.21.15.16:0/1876283677 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1060000bc0 con 0x7f1070069370 2023-09-30T22:26:49.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.123+0000 7f1057fff700 1 -- 172.21.15.16:0/1876283677 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f106000b850 con 0x7f1070069370 2023-09-30T22:26:49.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.123+0000 7f1057fff700 1 -- 172.21.15.16:0/1876283677 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1060000d20 con 0x7f1070069370 2023-09-30T22:26:49.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.123+0000 7f1076a43700 1 -- 172.21.15.16:0/1876283677 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1060009370 con 0x7f1070069370 2023-09-30T22:26:49.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.123+0000 7f1076a43700 1 -- 172.21.15.16:0/1876283677 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10700660d0 con 0x7f1070069370 2023-09-30T22:26:49.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.124+0000 7f1076a43700 1 -- 172.21.15.16:0/1876283677 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f107005a260 con 0x7f1070069370 2023-09-30T22:26:49.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.125+0000 7f1057fff700 1 -- 172.21.15.16:0/1876283677 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f106001c900 con 0x7f1070069370 2023-09-30T22:26:49.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.125+0000 7f1057fff700 1 -- 172.21.15.16:0/1876283677 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2838+0+0 (secure 0 0 0) 0x7f106003cc90 con 0x7f1070069370 2023-09-30T22:26:49.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.129+0000 7f1057fff700 1 -- 172.21.15.16:0/1876283677 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f106002e9d0 con 0x7f1070069370 2023-09-30T22:26:49.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:49.306+0000 7f1076a43700 1 -- 172.21.15.16:0/1876283677 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f107011d450 con 0x7f1070069370 2023-09-30T22:26:50.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.257+0000 7f1057fff700 1 -- 172.21.15.16:0/1876283677 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v13) v1 ==== 138+0+0 (secure 0 0 0) 0x7f10600340b0 con 0x7f1070069370 2023-09-30T22:26:50.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.262+0000 7f1076a43700 1 -- 172.21.15.16:0/1876283677 >> v1:172.21.15.16:6816/118285 conn(0x7f105c02bb60 legacy=0x7f105c02e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:50.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.262+0000 7f1076a43700 1 -- 172.21.15.16:0/1876283677 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1070069370 msgr2=0x7f107011df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:50.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.262+0000 7f1076a43700 1 --2- 172.21.15.16:0/1876283677 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1070069370 0x7f107011df30 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f1060009ea0 tx=0x7f1060017000).stop 2023-09-30T22:26:50.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.267+0000 7f1076a43700 1 -- 172.21.15.16:0/1876283677 shutdown_connections 2023-09-30T22:26:50.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.267+0000 7f1076a43700 1 --2- 172.21.15.16:0/1876283677 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f107012d2b0 0x7f1070064a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:50.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.267+0000 7f1076a43700 1 --2- 172.21.15.16:0/1876283677 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1070069370 0x7f107011df30 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:50.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.267+0000 7f1076a43700 1 --2- 172.21.15.16:0/1876283677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1070096900 0x7f1070064540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:50.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.267+0000 7f1076a43700 1 -- 172.21.15.16:0/1876283677 >> 172.21.15.16:0/1876283677 conn(0x7f1070059cd0 msgr2=0x7f1070094c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:50.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.267+0000 7f1076a43700 1 -- 172.21.15.16:0/1876283677 shutdown_connections 2023-09-30T22:26:50.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.267+0000 7f1076a43700 1 -- 172.21.15.16:0/1876283677 wait complete. 2023-09-30T22:26:50.268 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2023-09-30T22:26:50.283 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-09-30T22:26:50.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.422+0000 7f4c52171700 1 Processor -- start 2023-09-30T22:26:50.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.422+0000 7f4c52171700 1 -- start start 2023-09-30T22:26:50.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.423+0000 7f4c52171700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4c4c066d80 0x7f4c4c067160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:50.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.423+0000 7f4c52171700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4c4c068590 0x7f4c4c067650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:50.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.423+0000 7f4c52171700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c4c069580 con 0x7f4c4c128bb0 2023-09-30T22:26:50.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.423+0000 7f4c52171700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4c094430 con 0x7f4c4c066d80 2023-09-30T22:26:50.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.423+0000 7f4c52171700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4c094050 con 0x7f4c4c068590 2023-09-30T22:26:50.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.423+0000 7f4c4b7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4c4c066d80 0x7f4c4c067160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:50.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.423+0000 7f4c4bfff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4c4c128bb0 0x7f4c4c129040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33666/0 (socket says 172.21.15.16:33666) 2023-09-30T22:26:50.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.423+0000 7f4c4bfff700 1 -- 172.21.15.16:0/467831189 learned_addr learned my addr 172.21.15.16:0/467831189 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:26:50.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.423+0000 7f4c4affd700 1 --2- 172.21.15.16:0/467831189 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4c4c068590 0x7f4c4c067650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:50.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.423+0000 7f4c4a7fc700 1 -- 172.21.15.16:0/467831189 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3172576839 0 0) 0x7f4c4c069580 con 0x7f4c4c128bb0 2023-09-30T22:26:50.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.423+0000 7f4c4b7fe700 1 -- 172.21.15.16:0/467831189 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4c4c068590 msgr2=0x7f4c4c067650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:50.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.423+0000 7f4c4b7fe700 1 --2- 172.21.15.16:0/467831189 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4c4c068590 0x7f4c4c067650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:50.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.423+0000 7f4c4b7fe700 1 -- 172.21.15.16:0/467831189 >> v1:172.21.15.16:6789/0 conn(0x7f4c4c128bb0 legacy=0x7f4c4c129040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:50.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.423+0000 7f4c4b7fe700 1 -- 172.21.15.16:0/467831189 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c4c069840 con 0x7f4c4c066d80 2023-09-30T22:26:50.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.423+0000 7f4c4b7fe700 1 --2- 172.21.15.16:0/467831189 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4c4c066d80 0x7f4c4c067160 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f4c3c009790 tx=0x7f4c3c009060).ready entity=mon.1 client_cookie=120a76d911d7b9e2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:50.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.423+0000 7f4c4a7fc700 1 -- 172.21.15.16:0/467831189 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c0090d0 con 0x7f4c4c066d80 2023-09-30T22:26:50.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.424+0000 7f4c4a7fc700 1 -- 172.21.15.16:0/467831189 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c3c00f510 con 0x7f4c4c066d80 2023-09-30T22:26:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.424+0000 7f4c4a7fc700 1 -- 172.21.15.16:0/467831189 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c00f890 con 0x7f4c4c066d80 2023-09-30T22:26:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.424+0000 7f4c52171700 1 -- 172.21.15.16:0/467831189 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4c4c066d80 msgr2=0x7f4c4c067160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.424+0000 7f4c52171700 1 --2- 172.21.15.16:0/467831189 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4c4c066d80 0x7f4c4c067160 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f4c3c009790 tx=0x7f4c3c009060).stop 2023-09-30T22:26:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.424+0000 7f4c52171700 1 -- 172.21.15.16:0/467831189 shutdown_connections 2023-09-30T22:26:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.424+0000 7f4c52171700 1 --2- 172.21.15.16:0/467831189 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4c4c066d80 0x7f4c4c067160 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.424+0000 7f4c52171700 1 --2- 172.21.15.16:0/467831189 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4c4c068590 0x7f4c4c067650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.424+0000 7f4c52171700 1 -- 172.21.15.16:0/467831189 >> 172.21.15.16:0/467831189 conn(0x7f4c4c09a3d0 msgr2=0x7f4c4c09a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.424+0000 7f4c52171700 1 -- 172.21.15.16:0/467831189 shutdown_connections 2023-09-30T22:26:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.424+0000 7f4c52171700 1 -- 172.21.15.16:0/467831189 wait complete. 2023-09-30T22:26:50.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.425+0000 7f4c52171700 1 Processor -- start 2023-09-30T22:26:50.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.425+0000 7f4c52171700 1 -- start start 2023-09-30T22:26:50.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.426+0000 7f4c52171700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4c4c066d80 0x7f4c4c13a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:50.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.426+0000 7f4c52171700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4c4c068590 0x7f4c4c13aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:50.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.426+0000 7f4c52171700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c4c128bb0 0x7f4c4c091a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:50.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.426+0000 7f4c52171700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4c12d340 con 0x7f4c4c128bb0 2023-09-30T22:26:50.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.426+0000 7f4c52171700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4c099480 con 0x7f4c4c066d80 2023-09-30T22:26:50.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.426+0000 7f4c52171700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4c0690f0 con 0x7f4c4c068590 2023-09-30T22:26:50.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.426+0000 7f4c4b7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4c4c066d80 0x7f4c4c13a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:50.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.426+0000 7f4c4b7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4c4c066d80 0x7f4c4c13a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:44502/0 (socket says 172.21.15.16:44502) 2023-09-30T22:26:50.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.426+0000 7f4c4b7fe700 1 -- 172.21.15.16:0/1543516714 learned_addr learned my addr 172.21.15.16:0/1543516714 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:26:50.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.426+0000 7f4c4bfff700 1 --2- 172.21.15.16:0/1543516714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c4c128bb0 0x7f4c4c091a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:50.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.426+0000 7f4c4affd700 1 --2- 172.21.15.16:0/1543516714 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4c4c068590 0x7f4c4c13aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:50.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.426+0000 7f4c4b7fe700 1 -- 172.21.15.16:0/1543516714 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4c4c068590 msgr2=0x7f4c4c13aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.426+0000 7f4c4b7fe700 1 --2- 172.21.15.16:0/1543516714 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4c4c068590 0x7f4c4c13aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.426+0000 7f4c4b7fe700 1 -- 172.21.15.16:0/1543516714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c4c128bb0 msgr2=0x7f4c4c091a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.426+0000 7f4c4b7fe700 1 --2- 172.21.15.16:0/1543516714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c4c128bb0 0x7f4c4c091a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.426+0000 7f4c4b7fe700 1 -- 172.21.15.16:0/1543516714 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c3c0092a0 con 0x7f4c4c066d80 2023-09-30T22:26:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.426+0000 7f4c4b7fe700 1 --2- 172.21.15.16:0/1543516714 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4c4c066d80 0x7f4c4c13a5d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f4c3c009e30 tx=0x7f4c3c017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.427+0000 7f4c437fe700 1 -- 172.21.15.16:0/1543516714 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c00f890 con 0x7f4c4c066d80 2023-09-30T22:26:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.427+0000 7f4c437fe700 1 -- 172.21.15.16:0/1543516714 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c3c006910 con 0x7f4c4c066d80 2023-09-30T22:26:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.427+0000 7f4c52171700 1 -- 172.21.15.16:0/1543516714 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c4c1195d0 con 0x7f4c4c066d80 2023-09-30T22:26:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.427+0000 7f4c437fe700 1 -- 172.21.15.16:0/1543516714 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c00fea0 con 0x7f4c4c066d80 2023-09-30T22:26:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.427+0000 7f4c52171700 1 -- 172.21.15.16:0/1543516714 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c4c118270 con 0x7f4c4c066d80 2023-09-30T22:26:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.427+0000 7f4c437fe700 1 -- 172.21.15.16:0/1543516714 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f4c3c01b620 con 0x7f4c4c066d80 2023-09-30T22:26:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.428+0000 7f4c437fe700 1 -- 172.21.15.16:0/1543516714 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 3208+0+0 (secure 0 0 0) 0x7f4c3c03c5f0 con 0x7f4c4c066d80 2023-09-30T22:26:50.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.428+0000 7f4c52171700 1 -- 172.21.15.16:0/1543516714 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c4c119270 con 0x7f4c4c066d80 2023-09-30T22:26:50.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.433+0000 7f4c437fe700 1 -- 172.21.15.16:0/1543516714 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f4c4c119270 con 0x7f4c4c066d80 2023-09-30T22:26:50.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:50.607+0000 7f4c52171700 1 -- 172.21.15.16:0/1543516714 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f4c4c119430 con 0x7f4c4c066d80 2023-09-30T22:26:51.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.263+0000 7f4c437fe700 1 -- 172.21.15.16:0/1543516714 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v14) v1 ==== 170+0+0 (secure 0 0 0) 0x7f4c4c119430 con 0x7f4c4c066d80 2023-09-30T22:26:51.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.268+0000 7f4c52171700 1 -- 172.21.15.16:0/1543516714 >> v1:172.21.15.16:6816/118285 conn(0x7f4c2c02bb90 legacy=0x7f4c2c02e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:51.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.268+0000 7f4c52171700 1 -- 172.21.15.16:0/1543516714 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4c4c066d80 msgr2=0x7f4c4c13a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:51.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.268+0000 7f4c52171700 1 --2- 172.21.15.16:0/1543516714 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4c4c066d80 0x7f4c4c13a5d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f4c3c009e30 tx=0x7f4c3c017000).stop 2023-09-30T22:26:51.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.268+0000 7f4c52171700 1 -- 172.21.15.16:0/1543516714 shutdown_connections 2023-09-30T22:26:51.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.268+0000 7f4c52171700 1 --2- 172.21.15.16:0/1543516714 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4c4c066d80 0x7f4c4c13a5d0 secure :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0x7f4c3c009e30 tx=0x7f4c3c017000).stop 2023-09-30T22:26:51.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.268+0000 7f4c52171700 1 --2- 172.21.15.16:0/1543516714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c4c128bb0 0x7f4c4c091a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:51.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.268+0000 7f4c52171700 1 --2- 172.21.15.16:0/1543516714 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4c4c068590 0x7f4c4c13aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:51.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.268+0000 7f4c52171700 1 -- 172.21.15.16:0/1543516714 >> 172.21.15.16:0/1543516714 conn(0x7f4c4c09a3d0 msgr2=0x7f4c4c129c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:51.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.281+0000 7f4c52171700 1 -- 172.21.15.16:0/1543516714 shutdown_connections 2023-09-30T22:26:51.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.282+0000 7f4c52171700 1 -- 172.21.15.16:0/1543516714 wait complete. 2023-09-30T22:26:51.284 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2023-09-30T22:26:51.308 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-09-30T22:26:51.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.442+0000 7f5a45f53700 1 Processor -- start 2023-09-30T22:26:51.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.442+0000 7f5a45f53700 1 -- start start 2023-09-30T22:26:51.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.443+0000 7f5a45f53700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5a40059070 0x7f5a40125260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:51.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.443+0000 7f5a45f53700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a401257f0 0x7f5a40125c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:51.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.443+0000 7f5a45f53700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5a40093bf0 0x7f5a40094070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:51.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.443+0000 7f5a45f53700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a40059450 con 0x7f5a40093bf0 2023-09-30T22:26:51.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.443+0000 7f5a45f53700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a400996f0 con 0x7f5a40059070 2023-09-30T22:26:51.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.443+0000 7f5a45f53700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5a40098790 con 0x7f5a401257f0 2023-09-30T22:26:51.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.443+0000 7f5a3effd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a401257f0 0x7f5a40125c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:51.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.443+0000 7f5a3f7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5a40059070 0x7f5a40125260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:51.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.443+0000 7f5a3effd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a401257f0 0x7f5a40125c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:46174/0 (socket says 172.21.15.16:46174) 2023-09-30T22:26:51.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.443+0000 7f5a3effd700 1 -- 172.21.15.16:0/71110702 learned_addr learned my addr 172.21.15.16:0/71110702 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:26:51.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.443+0000 7f5a3ffff700 1 --2- 172.21.15.16:0/71110702 >> v2:172.21.15.16:3300/0 conn(0x7f5a40093bf0 0x7f5a40094070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:51.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.443+0000 7f5a3effd700 1 -- 172.21.15.16:0/71110702 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5a40059070 msgr2=0x7f5a40125260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:51.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.443+0000 7f5a3effd700 1 --2- 172.21.15.16:0/71110702 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5a40059070 0x7f5a40125260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:51.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.443+0000 7f5a3effd700 1 -- 172.21.15.16:0/71110702 >> v2:172.21.15.16:3300/0 conn(0x7f5a40093bf0 msgr2=0x7f5a40094070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:51.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.443+0000 7f5a3effd700 1 --2- 172.21.15.16:0/71110702 >> v2:172.21.15.16:3300/0 conn(0x7f5a40093bf0 0x7f5a40094070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:51.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.443+0000 7f5a3effd700 1 -- 172.21.15.16:0/71110702 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a4009a5a0 con 0x7f5a401257f0 2023-09-30T22:26:51.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.443+0000 7f5a3f7fe700 1 --2- 172.21.15.16:0/71110702 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5a40059070 0x7f5a40125260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:51.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.444+0000 7f5a3effd700 1 --2- 172.21.15.16:0/71110702 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a401257f0 0x7f5a40125c70 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f5a30009830 tx=0x7f5a300090f0).ready entity=mon.2 client_cookie=39eaaaf8d35395af server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:51.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.444+0000 7f5a3e7fc700 1 -- 172.21.15.16:0/71110702 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3000f510 con 0x7f5a401257f0 2023-09-30T22:26:51.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.444+0000 7f5a3e7fc700 1 -- 172.21.15.16:0/71110702 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a3000f670 con 0x7f5a401257f0 2023-09-30T22:26:51.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.444+0000 7f5a3e7fc700 1 -- 172.21.15.16:0/71110702 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3000f9f0 con 0x7f5a401257f0 2023-09-30T22:26:51.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.444+0000 7f5a45f53700 1 -- 172.21.15.16:0/71110702 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a401257f0 msgr2=0x7f5a40125c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:51.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.444+0000 7f5a45f53700 1 --2- 172.21.15.16:0/71110702 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a401257f0 0x7f5a40125c70 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f5a30009830 tx=0x7f5a300090f0).stop 2023-09-30T22:26:51.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.444+0000 7f5a45f53700 1 -- 172.21.15.16:0/71110702 shutdown_connections 2023-09-30T22:26:51.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.444+0000 7f5a45f53700 1 --2- 172.21.15.16:0/71110702 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5a40059070 0x7f5a40125260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:51.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.444+0000 7f5a45f53700 1 --2- 172.21.15.16:0/71110702 >> v2:172.21.15.16:3300/0 conn(0x7f5a40093bf0 0x7f5a40094070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:51.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.444+0000 7f5a45f53700 1 --2- 172.21.15.16:0/71110702 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a401257f0 0x7f5a40125c70 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:51.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.444+0000 7f5a45f53700 1 -- 172.21.15.16:0/71110702 >> 172.21.15.16:0/71110702 conn(0x7f5a40097480 msgr2=0x7f5a40097890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:51.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.444+0000 7f5a45f53700 1 -- 172.21.15.16:0/71110702 shutdown_connections 2023-09-30T22:26:51.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.445+0000 7f5a45f53700 1 -- 172.21.15.16:0/71110702 wait complete. 2023-09-30T22:26:51.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.446+0000 7f5a45f53700 1 Processor -- start 2023-09-30T22:26:51.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.446+0000 7f5a45f53700 1 -- start start 2023-09-30T22:26:51.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.446+0000 7f5a45f53700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a40059070 0x7f5a40138100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:51.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.446+0000 7f5a45f53700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5a40093bf0 0x7f5a401385f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:51.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.446+0000 7f5a45f53700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a401257f0 0x7f5a40128d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:51.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.446+0000 7f5a45f53700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a40064f70 con 0x7f5a40059070 2023-09-30T22:26:51.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.446+0000 7f5a45f53700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4012cec0 con 0x7f5a40093bf0 2023-09-30T22:26:51.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.446+0000 7f5a45f53700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5a40098ea0 con 0x7f5a401257f0 2023-09-30T22:26:51.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.446+0000 7f5a3f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a40059070 0x7f5a40138100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:51.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.446+0000 7f5a3ffff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a401257f0 0x7f5a40128d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:51.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.446+0000 7f5a3f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a40059070 0x7f5a40138100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38640/0 (socket says 172.21.15.16:38640) 2023-09-30T22:26:51.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.446+0000 7f5a3ffff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a401257f0 0x7f5a40128d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:46182/0 (socket says 172.21.15.16:46182) 2023-09-30T22:26:51.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.446+0000 7f5a3f7fe700 1 -- 172.21.15.16:0/826812021 learned_addr learned my addr 172.21.15.16:0/826812021 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:26:51.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.446+0000 7f5a3effd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5a40093bf0 0x7f5a401385f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:51.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.447+0000 7f5a3ffff700 1 -- 172.21.15.16:0/826812021 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5a40093bf0 msgr2=0x7f5a401385f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:51.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.447+0000 7f5a3ffff700 1 --2- 172.21.15.16:0/826812021 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5a40093bf0 0x7f5a401385f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:51.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.447+0000 7f5a3ffff700 1 -- 172.21.15.16:0/826812021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a40059070 msgr2=0x7f5a40138100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:51.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.447+0000 7f5a3ffff700 1 --2- 172.21.15.16:0/826812021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a40059070 0x7f5a40138100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:51.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.447+0000 7f5a3ffff700 1 -- 172.21.15.16:0/826812021 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a280092a0 con 0x7f5a401257f0 2023-09-30T22:26:51.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.447+0000 7f5a3f7fe700 1 --2- 172.21.15.16:0/826812021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a40059070 0x7f5a40138100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:51.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.447+0000 7f5a3ffff700 1 --2- 172.21.15.16:0/826812021 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a401257f0 0x7f5a40128d20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f5a38009e80 tx=0x7f5a3800b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:51.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.447+0000 7f5a377fe700 1 -- 172.21.15.16:0/826812021 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3800caa0 con 0x7f5a401257f0 2023-09-30T22:26:51.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.447+0000 7f5a377fe700 1 -- 172.21.15.16:0/826812021 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a3800f040 con 0x7f5a401257f0 2023-09-30T22:26:51.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.447+0000 7f5a45f53700 1 -- 172.21.15.16:0/826812021 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a30009370 con 0x7f5a401257f0 2023-09-30T22:26:51.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.447+0000 7f5a377fe700 1 -- 172.21.15.16:0/826812021 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3800ce10 con 0x7f5a401257f0 2023-09-30T22:26:51.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.447+0000 7f5a45f53700 1 -- 172.21.15.16:0/826812021 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a4012a940 con 0x7f5a401257f0 2023-09-30T22:26:51.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.448+0000 7f5a377fe700 1 -- 172.21.15.16:0/826812021 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f5a38013610 con 0x7f5a401257f0 2023-09-30T22:26:51.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.448+0000 7f5a377fe700 1 -- 172.21.15.16:0/826812021 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 3590+0+0 (secure 0 0 0) 0x7f5a3803b6a0 con 0x7f5a401257f0 2023-09-30T22:26:51.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.448+0000 7f5a45f53700 1 -- 172.21.15.16:0/826812021 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a4005a260 con 0x7f5a401257f0 2023-09-30T22:26:51.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.453+0000 7f5a377fe700 1 -- 172.21.15.16:0/826812021 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f5a3801d460 con 0x7f5a401257f0 2023-09-30T22:26:51.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.672+0000 7f5a45f53700 1 -- 172.21.15.16:0/826812021 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f5a40046c20 con 0x7f5a401257f0 2023-09-30T22:26:51.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.681+0000 7f5a377fe700 1 -- 172.21.15.16:0/826812021 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v3) v1 ==== 172+0+0 (secure 0 0 0) 0x7f5a3802c0c0 con 0x7f5a401257f0 2023-09-30T22:26:51.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.695+0000 7f5a45f53700 1 -- 172.21.15.16:0/826812021 >> v1:172.21.15.16:6816/118285 conn(0x7f5a2002bb90 legacy=0x7f5a2002e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:51.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.695+0000 7f5a45f53700 1 -- 172.21.15.16:0/826812021 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a401257f0 msgr2=0x7f5a40128d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:51.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.695+0000 7f5a45f53700 1 --2- 172.21.15.16:0/826812021 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a401257f0 0x7f5a40128d20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f5a38009e80 tx=0x7f5a3800b040).stop 2023-09-30T22:26:51.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.696+0000 7f5a45f53700 1 -- 172.21.15.16:0/826812021 shutdown_connections 2023-09-30T22:26:51.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.696+0000 7f5a45f53700 1 --2- 172.21.15.16:0/826812021 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a401257f0 0x7f5a40128d20 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:51.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.696+0000 7f5a45f53700 1 --2- 172.21.15.16:0/826812021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a40059070 0x7f5a40138100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:51.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.696+0000 7f5a45f53700 1 --2- 172.21.15.16:0/826812021 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5a40093bf0 0x7f5a401385f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:51.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.696+0000 7f5a45f53700 1 -- 172.21.15.16:0/826812021 >> 172.21.15.16:0/826812021 conn(0x7f5a40097480 msgr2=0x7f5a40096280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:51.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.699+0000 7f5a45f53700 1 -- 172.21.15.16:0/826812021 shutdown_connections 2023-09-30T22:26:51.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.701+0000 7f5a45f53700 1 -- 172.21.15.16:0/826812021 wait complete. 2023-09-30T22:26:51.705 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 3 and data pool 4 2023-09-30T22:26:51.725 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:26:51.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.855+0000 7f4f874b5700 1 Processor -- start 2023-09-30T22:26:51.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.855+0000 7f4f874b5700 1 -- start start 2023-09-30T22:26:51.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.856+0000 7f4f874b5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4f80066200 0x7f4f80097780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:51.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.856+0000 7f4f874b5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4f8012b300 0x7f4f80127500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:51.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.856+0000 7f4f874b5700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f80065740 con 0x7f4f8012b720 2023-09-30T22:26:51.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.856+0000 7f4f874b5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f80064ef0 con 0x7f4f80066200 2023-09-30T22:26:51.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.856+0000 7f4f874b5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4f80065a00 con 0x7f4f8012b300 2023-09-30T22:26:51.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.856+0000 7f4f84a50700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4f8012b300 0x7f4f80127500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:51.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.856+0000 7f4f85251700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4f80066200 0x7f4f80097780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:51.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.856+0000 7f4f85a52700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4f8012b720 0x7f4f801279f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33674/0 (socket says 172.21.15.16:33674) 2023-09-30T22:26:51.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.856+0000 7f4f85a52700 1 -- 172.21.15.16:0/628105330 learned_addr learned my addr 172.21.15.16:0/628105330 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:26:51.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.856+0000 7f4f84a50700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4f8012b300 0x7f4f80127500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:46194/0 (socket says 172.21.15.16:46194) 2023-09-30T22:26:51.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.856+0000 7f4f85251700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4f80066200 0x7f4f80097780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:44534/0 (socket says 172.21.15.16:44534) 2023-09-30T22:26:51.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.856+0000 7f4f7ffff700 1 -- 172.21.15.16:0/628105330 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2777116633 0 0) 0x7f4f80065740 con 0x7f4f8012b720 2023-09-30T22:26:51.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.856+0000 7f4f7ffff700 1 -- 172.21.15.16:0/628105330 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f680033f0 con 0x7f4f8012b720 2023-09-30T22:26:51.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.856+0000 7f4f84a50700 1 -- 172.21.15.16:0/628105330 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4f80066200 msgr2=0x7f4f80097780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:51.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.856+0000 7f4f84a50700 1 --2- 172.21.15.16:0/628105330 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4f80066200 0x7f4f80097780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:51.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.856+0000 7f4f84a50700 1 -- 172.21.15.16:0/628105330 >> v1:172.21.15.16:6789/0 conn(0x7f4f8012b720 legacy=0x7f4f801279f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:51.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.856+0000 7f4f84a50700 1 -- 172.21.15.16:0/628105330 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f800653c0 con 0x7f4f8012b300 2023-09-30T22:26:51.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.856+0000 7f4f85251700 1 --2- 172.21.15.16:0/628105330 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4f80066200 0x7f4f80097780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:51.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.856+0000 7f4f84a50700 1 --2- 172.21.15.16:0/628105330 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4f8012b300 0x7f4f80127500 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f4f74009790 tx=0x7f4f74009060).ready entity=mon.2 client_cookie=36fade82446423d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:51.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.857+0000 7f4f7ffff700 1 -- 172.21.15.16:0/628105330 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f740090d0 con 0x7f4f8012b300 2023-09-30T22:26:51.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.857+0000 7f4f7ffff700 1 -- 172.21.15.16:0/628105330 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f7400f510 con 0x7f4f8012b300 2023-09-30T22:26:51.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.857+0000 7f4f7ffff700 1 -- 172.21.15.16:0/628105330 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f7400f890 con 0x7f4f8012b300 2023-09-30T22:26:51.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.857+0000 7f4f874b5700 1 -- 172.21.15.16:0/628105330 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4f8012b300 msgr2=0x7f4f80127500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:51.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.857+0000 7f4f874b5700 1 --2- 172.21.15.16:0/628105330 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4f8012b300 0x7f4f80127500 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f4f74009790 tx=0x7f4f74009060).stop 2023-09-30T22:26:51.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.857+0000 7f4f874b5700 1 -- 172.21.15.16:0/628105330 shutdown_connections 2023-09-30T22:26:51.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.857+0000 7f4f874b5700 1 --2- 172.21.15.16:0/628105330 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4f80066200 0x7f4f80097780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:51.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.857+0000 7f4f874b5700 1 --2- 172.21.15.16:0/628105330 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4f8012b300 0x7f4f80127500 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:51.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.857+0000 7f4f874b5700 1 -- 172.21.15.16:0/628105330 >> 172.21.15.16:0/628105330 conn(0x7f4f8009ba50 msgr2=0x7f4f8009be60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:51.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.857+0000 7f4f874b5700 1 -- 172.21.15.16:0/628105330 shutdown_connections 2023-09-30T22:26:51.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.857+0000 7f4f874b5700 1 -- 172.21.15.16:0/628105330 wait complete. 2023-09-30T22:26:51.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.858+0000 7f4f874b5700 1 Processor -- start 2023-09-30T22:26:51.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.858+0000 7f4f874b5700 1 -- start start 2023-09-30T22:26:51.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.859+0000 7f4f874b5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4f80066200 0x7f4f801140b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:51.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.859+0000 7f4f874b5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4f8012b300 0x7f4f801145a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:51.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.859+0000 7f4f874b5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f8012b720 0x7f4f80114a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:51.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.859+0000 7f4f874b5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f80114f80 con 0x7f4f8012b720 2023-09-30T22:26:51.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.859+0000 7f4f874b5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f8012a300 con 0x7f4f80066200 2023-09-30T22:26:51.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.859+0000 7f4f874b5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4f801186e0 con 0x7f4f8012b300 2023-09-30T22:26:51.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.859+0000 7f4f84a50700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4f8012b300 0x7f4f801145a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:51.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.859+0000 7f4f85a52700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f8012b720 0x7f4f80114a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:51.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.859+0000 7f4f84a50700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4f8012b300 0x7f4f801145a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:46200/0 (socket says 172.21.15.16:46200) 2023-09-30T22:26:51.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.859+0000 7f4f84a50700 1 -- 172.21.15.16:0/3202891012 learned_addr learned my addr 172.21.15.16:0/3202891012 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:26:51.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.859+0000 7f4f85251700 1 --2- 172.21.15.16:0/3202891012 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4f80066200 0x7f4f801140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:51.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.859+0000 7f4f85a52700 1 -- 172.21.15.16:0/3202891012 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4f8012b300 msgr2=0x7f4f801145a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:51.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.859+0000 7f4f85a52700 1 --2- 172.21.15.16:0/3202891012 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4f8012b300 0x7f4f801145a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:51.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.859+0000 7f4f85a52700 1 -- 172.21.15.16:0/3202891012 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4f80066200 msgr2=0x7f4f801140b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:51.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.859+0000 7f4f85a52700 1 --2- 172.21.15.16:0/3202891012 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4f80066200 0x7f4f801140b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:51.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.859+0000 7f4f85a52700 1 -- 172.21.15.16:0/3202891012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f70009a20 con 0x7f4f8012b720 2023-09-30T22:26:51.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.859+0000 7f4f85251700 1 --2- 172.21.15.16:0/3202891012 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4f80066200 0x7f4f801140b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:51.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.859+0000 7f4f84a50700 1 --2- 172.21.15.16:0/3202891012 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4f8012b300 0x7f4f801145a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:51.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.859+0000 7f4f85a52700 1 --2- 172.21.15.16:0/3202891012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f8012b720 0x7f4f80114a90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f4f7800c410 tx=0x7f4f78007b90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:51.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.860+0000 7f4f7d7fa700 1 -- 172.21.15.16:0/3202891012 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f780070d0 con 0x7f4f8012b720 2023-09-30T22:26:51.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.860+0000 7f4f7d7fa700 1 -- 172.21.15.16:0/3202891012 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f78007230 con 0x7f4f8012b720 2023-09-30T22:26:51.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.860+0000 7f4f874b5700 1 -- 172.21.15.16:0/3202891012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f740092a0 con 0x7f4f8012b720 2023-09-30T22:26:51.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.860+0000 7f4f7d7fa700 1 -- 172.21.15.16:0/3202891012 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f780075b0 con 0x7f4f8012b720 2023-09-30T22:26:51.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.860+0000 7f4f874b5700 1 -- 172.21.15.16:0/3202891012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f80117820 con 0x7f4f8012b720 2023-09-30T22:26:51.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.861+0000 7f4f7d7fa700 1 -- 172.21.15.16:0/3202891012 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f4f780059b0 con 0x7f4f8012b720 2023-09-30T22:26:51.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.861+0000 7f4f7d7fa700 1 -- 172.21.15.16:0/3202891012 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f4f7803a4c0 con 0x7f4f8012b720 2023-09-30T22:26:51.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.861+0000 7f4f874b5700 1 -- 172.21.15.16:0/3202891012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f8005a260 con 0x7f4f8012b720 2023-09-30T22:26:51.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:51.866+0000 7f4f7d7fa700 1 -- 172.21.15.16:0/3202891012 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f4f78010d40 con 0x7f4f8012b720 2023-09-30T22:26:52.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.041+0000 7f4f874b5700 1 -- 172.21.15.16:0/3202891012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4f80117a40 con 0x7f4f8012b720 2023-09-30T22:26:52.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.042+0000 7f4f7d7fa700 1 -- 172.21.15.16:0/3202891012 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+11354 (secure 0 0 0) 0x7f4f78010300 con 0x7f4f8012b720 2023-09-30T22:26:52.042 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:26:52.042 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":15,"fsid":"7ede3e14-2bd0-4726-9437-6d13c90cd342","created":"2023-09-30T22:26:35.483697+0000","modified":"2023-09-30T22:26:51.674239+0000","last_up_change":"2023-09-30T22:26:44.227199+0000","last_in_change":"2023-09-30T22:26:37.120906+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-09-30T22:26:44.285133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:26:45.132921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:26:49.353683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:26:50.655685+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"19946985-0ee2-487f-9347-929670dbe5ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6804","nonce":118824}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6805","nonce":118824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6807","nonce":118824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6806","nonce":118824}]},"public_addr":"172.21.15.16:6804/118824","cluster_addr":"172.21.15.16:6805/118824","heartbeat_back_addr":"172.21.15.16:6807/118824","heartbeat_front_addr":"172.21.15.16:6806/118824","state":["exists","up"]},{"osd":1,"uuid":"8efd4dbc-6ae4-4b82-b530-3b3eac2be445","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6812","nonce":118832}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6813","nonce":118832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6815","nonce":118832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6814","nonce":118832}]},"public_addr":"172.21.15.16:6812/118832","cluster_addr":"172.21.15.16:6813/118832","heartbeat_back_addr":"172.21.15.16:6815/118832","heartbeat_front_addr":"172.21.15.16:6814/118832","state":["exists","up"]},{"osd":2,"uuid":"b47bb1ce-27af-4c0c-868a-b76ebd0d9499","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6808","nonce":118840}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6809","nonce":118840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6811","nonce":118840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6810","nonce":118840}]},"public_addr":"172.21.15.16:6808/118840","cluster_addr":"172.21.15.16:6809/118840","heartbeat_back_addr":"172.21.15.16:6811/118840","heartbeat_front_addr":"172.21.15.16:6810/118840","state":["exists","up"]},{"osd":3,"uuid":"03b53d64-d10d-448a-a081-39ecceab589b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6800","nonce":118823}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6801","nonce":118823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6803","nonce":118823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6802","nonce":118823}]},"public_addr":"172.21.15.16:6800/118823","cluster_addr":"172.21.15.16:6801/118823","heartbeat_back_addr":"172.21.15.16:6803/118823","heartbeat_front_addr":"172.21.15.16:6802/118823","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.305953+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.307488+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.341023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.192071+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:26:52.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.044+0000 7f4f874b5700 1 -- 172.21.15.16:0/3202891012 >> v1:172.21.15.16:6816/118285 conn(0x7f4f6c02bb90 legacy=0x7f4f6c02e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:52.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.044+0000 7f4f874b5700 1 -- 172.21.15.16:0/3202891012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f8012b720 msgr2=0x7f4f80114a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:52.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.044+0000 7f4f874b5700 1 --2- 172.21.15.16:0/3202891012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f8012b720 0x7f4f80114a90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f4f7800c410 tx=0x7f4f78007b90).stop 2023-09-30T22:26:52.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.044+0000 7f4f874b5700 1 -- 172.21.15.16:0/3202891012 shutdown_connections 2023-09-30T22:26:52.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.044+0000 7f4f874b5700 1 --2- 172.21.15.16:0/3202891012 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4f80066200 0x7f4f801140b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:52.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.044+0000 7f4f874b5700 1 --2- 172.21.15.16:0/3202891012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f8012b720 0x7f4f80114a90 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:52.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.044+0000 7f4f874b5700 1 --2- 172.21.15.16:0/3202891012 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4f8012b300 0x7f4f801145a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:52.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.044+0000 7f4f874b5700 1 -- 172.21.15.16:0/3202891012 >> 172.21.15.16:0/3202891012 conn(0x7f4f8009ba50 msgr2=0x7f4f8009b260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:52.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.044+0000 7f4f874b5700 1 -- 172.21.15.16:0/3202891012 shutdown_connections 2023-09-30T22:26:52.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.044+0000 7f4f874b5700 1 -- 172.21.15.16:0/3202891012 wait complete. 2023-09-30T22:26:52.057 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:26:52.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.186+0000 7f8d02942700 1 Processor -- start 2023-09-30T22:26:52.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.187+0000 7f8d02942700 1 -- start start 2023-09-30T22:26:52.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.187+0000 7f8d02942700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8cfc091b00 0x7f8cfc12c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:52.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.187+0000 7f8d02942700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8cfc12ce60 0x7f8cfc12f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:52.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.187+0000 7f8d02942700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cfc096760 con 0x7f8cfc096380 2023-09-30T22:26:52.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.187+0000 7f8d02942700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cfc091ff0 con 0x7f8cfc12ce60 2023-09-30T22:26:52.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.187+0000 7f8d02942700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc092c80 con 0x7f8cfc091b00 2023-09-30T22:26:52.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.187+0000 7f8cfb7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8cfc091b00 0x7f8cfc12c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:52.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.187+0000 7f8cfbfff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8cfc096380 0x7f8cfc094730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33686/0 (socket says 172.21.15.16:33686) 2023-09-30T22:26:52.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.187+0000 7f8cfbfff700 1 -- 172.21.15.16:0/2612772628 learned_addr learned my addr 172.21.15.16:0/2612772628 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:26:52.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.187+0000 7f8d00edf700 1 --2- 172.21.15.16:0/2612772628 >> v2:172.21.15.16:3300/0 conn(0x7f8cfc12ce60 0x7f8cfc12f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:52.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.188+0000 7f8d00edf700 1 -- 172.21.15.16:0/2612772628 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8cfc091b00 msgr2=0x7f8cfc12c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:52.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.188+0000 7f8d00edf700 1 --2- 172.21.15.16:0/2612772628 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8cfc091b00 0x7f8cfc12c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:52.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.188+0000 7f8d00edf700 1 -- 172.21.15.16:0/2612772628 >> v1:172.21.15.16:6789/0 conn(0x7f8cfc096380 legacy=0x7f8cfc094730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:52.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.188+0000 7f8cfaffd700 1 -- 172.21.15.16:0/2612772628 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 483353575 0 0) 0x7f8cfc096760 con 0x7f8cfc096380 2023-09-30T22:26:52.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.188+0000 7f8d00edf700 1 -- 172.21.15.16:0/2612772628 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cfc094e40 con 0x7f8cfc12ce60 2023-09-30T22:26:52.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.188+0000 7f8cfb7fe700 1 --2- 172.21.15.16:0/2612772628 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8cfc091b00 0x7f8cfc12c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:52.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.188+0000 7f8d00edf700 1 --2- 172.21.15.16:0/2612772628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cfc12ce60 0x7f8cfc12f250 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f8cf4009e60 tx=0x7f8cf4009780).ready entity=mon.0 client_cookie=1185de9c919e4c5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:52.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.188+0000 7f8cfaffd700 1 -- 172.21.15.16:0/2612772628 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf400e070 con 0x7f8cfc12ce60 2023-09-30T22:26:52.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.188+0000 7f8cfaffd700 1 -- 172.21.15.16:0/2612772628 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cf400bb60 con 0x7f8cfc12ce60 2023-09-30T22:26:52.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.188+0000 7f8cfaffd700 1 -- 172.21.15.16:0/2612772628 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf4012410 con 0x7f8cfc12ce60 2023-09-30T22:26:52.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.188+0000 7f8d02942700 1 -- 172.21.15.16:0/2612772628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cfc12ce60 msgr2=0x7f8cfc12f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:52.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.188+0000 7f8d02942700 1 --2- 172.21.15.16:0/2612772628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cfc12ce60 0x7f8cfc12f250 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f8cf4009e60 tx=0x7f8cf4009780).stop 2023-09-30T22:26:52.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.189+0000 7f8d02942700 1 -- 172.21.15.16:0/2612772628 shutdown_connections 2023-09-30T22:26:52.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.189+0000 7f8d02942700 1 --2- 172.21.15.16:0/2612772628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cfc12ce60 0x7f8cfc12f250 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:52.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.189+0000 7f8d02942700 1 --2- 172.21.15.16:0/2612772628 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8cfc091b00 0x7f8cfc12c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:52.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.189+0000 7f8d02942700 1 -- 172.21.15.16:0/2612772628 >> 172.21.15.16:0/2612772628 conn(0x7f8cfc097470 msgr2=0x7f8cfc097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:52.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.189+0000 7f8d02942700 1 -- 172.21.15.16:0/2612772628 shutdown_connections 2023-09-30T22:26:52.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.189+0000 7f8d02942700 1 -- 172.21.15.16:0/2612772628 wait complete. 2023-09-30T22:26:52.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.189+0000 7f8d02942700 1 Processor -- start 2023-09-30T22:26:52.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.190+0000 7f8d02942700 1 -- start start 2023-09-30T22:26:52.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.190+0000 7f8d02942700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8cfc091b00 0x7f8cfc0590a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:52.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.190+0000 7f8d02942700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cfc096380 0x7f8cfc1246a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:52.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.190+0000 7f8d02942700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8cfc12ce60 0x7f8cfc124b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:52.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.190+0000 7f8d02942700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc1116f0 con 0x7f8cfc096380 2023-09-30T22:26:52.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.190+0000 7f8d02942700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc099bf0 con 0x7f8cfc12ce60 2023-09-30T22:26:52.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.190+0000 7f8d02942700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc06a550 con 0x7f8cfc091b00 2023-09-30T22:26:52.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.190+0000 7f8cfb7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cfc096380 0x7f8cfc1246a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:52.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.190+0000 7f8d00edf700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8cfc12ce60 0x7f8cfc124b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:52.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.190+0000 7f8cfbfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8cfc091b00 0x7f8cfc0590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:52.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.190+0000 7f8cfb7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cfc096380 0x7f8cfc1246a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38666/0 (socket says 172.21.15.16:38666) 2023-09-30T22:26:52.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.190+0000 7f8d00edf700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8cfc12ce60 0x7f8cfc124b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:44558/0 (socket says 172.21.15.16:44558) 2023-09-30T22:26:52.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.190+0000 7f8cfb7fe700 1 -- 172.21.15.16:0/290957692 learned_addr learned my addr 172.21.15.16:0/290957692 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:26:52.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.191+0000 7f8d00edf700 1 -- 172.21.15.16:0/290957692 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8cfc091b00 msgr2=0x7f8cfc0590a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:52.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.191+0000 7f8d00edf700 1 --2- 172.21.15.16:0/290957692 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8cfc091b00 0x7f8cfc0590a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:52.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.191+0000 7f8d00edf700 1 -- 172.21.15.16:0/290957692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cfc096380 msgr2=0x7f8cfc1246a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:52.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.191+0000 7f8d00edf700 1 --2- 172.21.15.16:0/290957692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cfc096380 0x7f8cfc1246a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:52.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.191+0000 7f8d00edf700 1 -- 172.21.15.16:0/290957692 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cf00092a0 con 0x7f8cfc12ce60 2023-09-30T22:26:52.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.191+0000 7f8cfb7fe700 1 --2- 172.21.15.16:0/290957692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cfc096380 0x7f8cfc1246a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:52.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.191+0000 7f8d00edf700 1 --2- 172.21.15.16:0/290957692 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8cfc12ce60 0x7f8cfc124b90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f8cf4014000 tx=0x7f8cf4009780).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:52.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.191+0000 7f8ce3fff700 1 -- 172.21.15.16:0/290957692 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf4012410 con 0x7f8cfc12ce60 2023-09-30T22:26:52.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.191+0000 7f8ce3fff700 1 -- 172.21.15.16:0/290957692 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cf401d630 con 0x7f8cfc12ce60 2023-09-30T22:26:52.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.191+0000 7f8d02942700 1 -- 172.21.15.16:0/290957692 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cf4009a20 con 0x7f8cfc12ce60 2023-09-30T22:26:52.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.191+0000 7f8ce3fff700 1 -- 172.21.15.16:0/290957692 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf401d9d0 con 0x7f8cfc12ce60 2023-09-30T22:26:52.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.191+0000 7f8d02942700 1 -- 172.21.15.16:0/290957692 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cfc059db0 con 0x7f8cfc12ce60 2023-09-30T22:26:52.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.192+0000 7f8ce3fff700 1 -- 172.21.15.16:0/290957692 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f8cf4003040 con 0x7f8cfc12ce60 2023-09-30T22:26:52.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.192+0000 7f8ce3fff700 1 -- 172.21.15.16:0/290957692 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f8cf403cc90 con 0x7f8cfc12ce60 2023-09-30T22:26:52.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.193+0000 7f8d02942700 1 -- 172.21.15.16:0/290957692 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cfc05a260 con 0x7f8cfc12ce60 2023-09-30T22:26:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.197+0000 7f8ce3fff700 1 -- 172.21.15.16:0/290957692 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f8cf401de00 con 0x7f8cfc12ce60 2023-09-30T22:26:52.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.375+0000 7f8d02942700 1 -- 172.21.15.16:0/290957692 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8cfc093930 con 0x7f8cfc12ce60 2023-09-30T22:26:52.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.376+0000 7f8ce3fff700 1 -- 172.21.15.16:0/290957692 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v15) v1 ==== 74+0+11354 (secure 0 0 0) 0x7f8cf4010b00 con 0x7f8cfc12ce60 2023-09-30T22:26:52.377 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:26:52.377 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":15,"fsid":"7ede3e14-2bd0-4726-9437-6d13c90cd342","created":"2023-09-30T22:26:35.483697+0000","modified":"2023-09-30T22:26:51.674239+0000","last_up_change":"2023-09-30T22:26:44.227199+0000","last_in_change":"2023-09-30T22:26:37.120906+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-09-30T22:26:44.285133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:26:45.132921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:26:49.353683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:26:50.655685+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"19946985-0ee2-487f-9347-929670dbe5ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6804","nonce":118824}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6805","nonce":118824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6807","nonce":118824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6806","nonce":118824}]},"public_addr":"172.21.15.16:6804/118824","cluster_addr":"172.21.15.16:6805/118824","heartbeat_back_addr":"172.21.15.16:6807/118824","heartbeat_front_addr":"172.21.15.16:6806/118824","state":["exists","up"]},{"osd":1,"uuid":"8efd4dbc-6ae4-4b82-b530-3b3eac2be445","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6812","nonce":118832}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6813","nonce":118832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6815","nonce":118832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6814","nonce":118832}]},"public_addr":"172.21.15.16:6812/118832","cluster_addr":"172.21.15.16:6813/118832","heartbeat_back_addr":"172.21.15.16:6815/118832","heartbeat_front_addr":"172.21.15.16:6814/118832","state":["exists","up"]},{"osd":2,"uuid":"b47bb1ce-27af-4c0c-868a-b76ebd0d9499","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6808","nonce":118840}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6809","nonce":118840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6811","nonce":118840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6810","nonce":118840}]},"public_addr":"172.21.15.16:6808/118840","cluster_addr":"172.21.15.16:6809/118840","heartbeat_back_addr":"172.21.15.16:6811/118840","heartbeat_front_addr":"172.21.15.16:6810/118840","state":["exists","up"]},{"osd":3,"uuid":"03b53d64-d10d-448a-a081-39ecceab589b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6800","nonce":118823}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6801","nonce":118823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6803","nonce":118823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6802","nonce":118823}]},"public_addr":"172.21.15.16:6800/118823","cluster_addr":"172.21.15.16:6801/118823","heartbeat_back_addr":"172.21.15.16:6803/118823","heartbeat_front_addr":"172.21.15.16:6802/118823","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.305953+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.307488+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.341023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.192071+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:26:52.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.378+0000 7f8d02942700 1 -- 172.21.15.16:0/290957692 >> v1:172.21.15.16:6816/118285 conn(0x7f8ce802bb90 legacy=0x7f8ce802e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:52.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.378+0000 7f8d02942700 1 -- 172.21.15.16:0/290957692 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8cfc12ce60 msgr2=0x7f8cfc124b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:52.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.378+0000 7f8d02942700 1 --2- 172.21.15.16:0/290957692 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8cfc12ce60 0x7f8cfc124b90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f8cf4014000 tx=0x7f8cf4009780).stop 2023-09-30T22:26:52.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.378+0000 7f8d02942700 1 -- 172.21.15.16:0/290957692 shutdown_connections 2023-09-30T22:26:52.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.378+0000 7f8d02942700 1 --2- 172.21.15.16:0/290957692 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8cfc12ce60 0x7f8cfc124b90 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:52.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.378+0000 7f8d02942700 1 --2- 172.21.15.16:0/290957692 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8cfc091b00 0x7f8cfc0590a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:52.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.378+0000 7f8d02942700 1 --2- 172.21.15.16:0/290957692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cfc096380 0x7f8cfc1246a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:52.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.378+0000 7f8d02942700 1 -- 172.21.15.16:0/290957692 >> 172.21.15.16:0/290957692 conn(0x7f8cfc097470 msgr2=0x7f8cfc0664e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:52.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.379+0000 7f8d02942700 1 -- 172.21.15.16:0/290957692 shutdown_connections 2023-09-30T22:26:52.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.379+0000 7f8d02942700 1 -- 172.21.15.16:0/290957692 wait complete. 2023-09-30T22:26:52.392 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-09-30T22:26:52.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.525+0000 7f06d87e6700 1 Processor -- start 2023-09-30T22:26:52.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.525+0000 7f06d87e6700 1 -- start start 2023-09-30T22:26:52.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.526+0000 7f06d87e6700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f06d00613a0 0x7f06d00919e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:52.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.526+0000 7f06d87e6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f06d012cde0 0x7f06d012f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:52.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.526+0000 7f06d87e6700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06d0092c00 con 0x7f06d0091ed0 2023-09-30T22:26:52.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.526+0000 7f06d87e6700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06d0061780 con 0x7f06d00613a0 2023-09-30T22:26:52.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.526+0000 7f06d87e6700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f06d0092ff0 con 0x7f06d012cde0 2023-09-30T22:26:52.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.526+0000 7f06d6582700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f06d00613a0 0x7f06d00919e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:52.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.526+0000 7f06d6d83700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f06d012cde0 0x7f06d012f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:52.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.526+0000 7f06d5d81700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f06d0091ed0 0x7f06d00922b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33692/0 (socket says 172.21.15.16:33692) 2023-09-30T22:26:52.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.526+0000 7f06d5d81700 1 -- 172.21.15.16:0/818745926 learned_addr learned my addr 172.21.15.16:0/818745926 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:26:52.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.526+0000 7f06d6582700 1 -- 172.21.15.16:0/818745926 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f06d012cde0 msgr2=0x7f06d012f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:52.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.526+0000 7f06d6582700 1 --2- 172.21.15.16:0/818745926 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f06d012cde0 0x7f06d012f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:52.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.526+0000 7f06d6582700 1 -- 172.21.15.16:0/818745926 >> v1:172.21.15.16:6789/0 conn(0x7f06d0091ed0 legacy=0x7f06d00922b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:52.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.526+0000 7f06d5580700 1 -- 172.21.15.16:0/818745926 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3704527981 0 0) 0x7f06d0092c00 con 0x7f06d0091ed0 2023-09-30T22:26:52.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.526+0000 7f06d6582700 1 -- 172.21.15.16:0/818745926 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06d0093b20 con 0x7f06d00613a0 2023-09-30T22:26:52.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.526+0000 7f06d6d83700 1 --2- 172.21.15.16:0/818745926 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f06d012cde0 0x7f06d012f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:52.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.526+0000 7f06d6582700 1 --2- 172.21.15.16:0/818745926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06d00613a0 0x7f06d00919e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f06cc009760 tx=0x7f06cc009060).ready entity=mon.0 client_cookie=f01f4ccb6d5c1dc4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:52.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.527+0000 7f06d5580700 1 -- 172.21.15.16:0/818745926 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06cc00f510 con 0x7f06d00613a0 2023-09-30T22:26:52.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.527+0000 7f06d5580700 1 -- 172.21.15.16:0/818745926 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06cc00f670 con 0x7f06d00613a0 2023-09-30T22:26:52.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.527+0000 7f06d5580700 1 -- 172.21.15.16:0/818745926 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06cc00f9f0 con 0x7f06d00613a0 2023-09-30T22:26:52.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.527+0000 7f06d87e6700 1 -- 172.21.15.16:0/818745926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06d00613a0 msgr2=0x7f06d00919e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:52.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.527+0000 7f06d87e6700 1 --2- 172.21.15.16:0/818745926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06d00613a0 0x7f06d00919e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f06cc009760 tx=0x7f06cc009060).stop 2023-09-30T22:26:52.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.527+0000 7f06d87e6700 1 -- 172.21.15.16:0/818745926 shutdown_connections 2023-09-30T22:26:52.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.527+0000 7f06d87e6700 1 --2- 172.21.15.16:0/818745926 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f06d012cde0 0x7f06d012f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:52.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.527+0000 7f06d87e6700 1 --2- 172.21.15.16:0/818745926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06d00613a0 0x7f06d00919e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:52.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.527+0000 7f06d87e6700 1 -- 172.21.15.16:0/818745926 >> 172.21.15.16:0/818745926 conn(0x7f06d009a1c0 msgr2=0x7f06d009a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:52.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.527+0000 7f06d87e6700 1 -- 172.21.15.16:0/818745926 shutdown_connections 2023-09-30T22:26:52.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.528+0000 7f06d87e6700 1 -- 172.21.15.16:0/818745926 wait complete. 2023-09-30T22:26:52.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.528+0000 7f06d87e6700 1 Processor -- start 2023-09-30T22:26:52.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.528+0000 7f06d87e6700 1 -- start start 2023-09-30T22:26:52.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.529+0000 7f06d87e6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06d00613a0 0x7f06d0117cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:52.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.529+0000 7f06d87e6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f06d0091ed0 0x7f06d0127940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:52.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.529+0000 7f06d87e6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f06d012cde0 0x7f06d0127e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:52.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.529+0000 7f06d87e6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d0128e80 con 0x7f06d00613a0 2023-09-30T22:26:52.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.529+0000 7f06d87e6700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06d0093d70 con 0x7f06d012cde0 2023-09-30T22:26:52.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.529+0000 7f06d87e6700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f06d0117250 con 0x7f06d0091ed0 2023-09-30T22:26:52.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.529+0000 7f06d5d81700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f06d0091ed0 0x7f06d0127940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:52.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.529+0000 7f06d6582700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06d00613a0 0x7f06d0117cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:52.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.529+0000 7f06d5d81700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f06d0091ed0 0x7f06d0127940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:46228/0 (socket says 172.21.15.16:46228) 2023-09-30T22:26:52.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.529+0000 7f06d5d81700 1 -- 172.21.15.16:0/1358554695 learned_addr learned my addr 172.21.15.16:0/1358554695 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:26:52.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.529+0000 7f06d6582700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06d00613a0 0x7f06d0117cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38694/0 (socket says 172.21.15.16:38694) 2023-09-30T22:26:52.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.529+0000 7f06d6d83700 1 --2- 172.21.15.16:0/1358554695 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f06d012cde0 0x7f06d0127e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:52.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.529+0000 7f06d5d81700 1 -- 172.21.15.16:0/1358554695 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f06d012cde0 msgr2=0x7f06d0127e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:52.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.529+0000 7f06d5d81700 1 --2- 172.21.15.16:0/1358554695 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f06d012cde0 0x7f06d0127e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:52.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.529+0000 7f06d5d81700 1 -- 172.21.15.16:0/1358554695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06d00613a0 msgr2=0x7f06d0117cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:52.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.529+0000 7f06d5d81700 1 --2- 172.21.15.16:0/1358554695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06d00613a0 0x7f06d0117cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:52.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.529+0000 7f06d5d81700 1 -- 172.21.15.16:0/1358554695 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c8009a20 con 0x7f06d0091ed0 2023-09-30T22:26:52.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.529+0000 7f06d6582700 1 --2- 172.21.15.16:0/1358554695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06d00613a0 0x7f06d0117cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:52.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.530+0000 7f06d5d81700 1 --2- 172.21.15.16:0/1358554695 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f06d0091ed0 0x7f06d0127940 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f06c400c4e0 tx=0x7f06c4007bf0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:52.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.530+0000 7f06c27fc700 1 -- 172.21.15.16:0/1358554695 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c400cea0 con 0x7f06d0091ed0 2023-09-30T22:26:52.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.530+0000 7f06c27fc700 1 -- 172.21.15.16:0/1358554695 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06c40072e0 con 0x7f06d0091ed0 2023-09-30T22:26:52.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.530+0000 7f06d87e6700 1 -- 172.21.15.16:0/1358554695 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06cc0092a0 con 0x7f06d0091ed0 2023-09-30T22:26:52.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.530+0000 7f06c27fc700 1 -- 172.21.15.16:0/1358554695 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c40057a0 con 0x7f06d0091ed0 2023-09-30T22:26:52.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.530+0000 7f06d87e6700 1 -- 172.21.15.16:0/1358554695 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06d0115aa0 con 0x7f06d0091ed0 2023-09-30T22:26:52.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.530+0000 7f06c27fc700 1 -- 172.21.15.16:0/1358554695 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f06c4005bd0 con 0x7f06d0091ed0 2023-09-30T22:26:52.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.531+0000 7f06c27fc700 1 -- 172.21.15.16:0/1358554695 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f06c403a6e0 con 0x7f06d0091ed0 2023-09-30T22:26:52.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.531+0000 7f06d87e6700 1 -- 172.21.15.16:0/1358554695 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06d005a260 con 0x7f06d0091ed0 2023-09-30T22:26:52.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.536+0000 7f06c27fc700 1 -- 172.21.15.16:0/1358554695 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f06c401d3f0 con 0x7f06d0091ed0 2023-09-30T22:26:52.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:52.756+0000 7f06d87e6700 1 -- 172.21.15.16:0/1358554695 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f06d0046c20 con 0x7f06d0091ed0 2023-09-30T22:26:53.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.681+0000 7f06c27fc700 1 -- 172.21.15.16:0/1358554695 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) v1 ==== 118+0+0 (secure 0 0 0) 0x7f06c401c350 con 0x7f06d0091ed0 2023-09-30T22:26:53.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.684+0000 7f06d87e6700 1 -- 172.21.15.16:0/1358554695 >> v1:172.21.15.16:6816/118285 conn(0x7f06b402bb60 legacy=0x7f06b402e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:53.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.684+0000 7f06d87e6700 1 -- 172.21.15.16:0/1358554695 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f06d0091ed0 msgr2=0x7f06d0127940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:53.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.684+0000 7f06d87e6700 1 --2- 172.21.15.16:0/1358554695 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f06d0091ed0 0x7f06d0127940 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f06c400c4e0 tx=0x7f06c4007bf0).stop 2023-09-30T22:26:53.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.684+0000 7f06d87e6700 1 -- 172.21.15.16:0/1358554695 shutdown_connections 2023-09-30T22:26:53.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.684+0000 7f06d87e6700 1 --2- 172.21.15.16:0/1358554695 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f06d012cde0 0x7f06d0127e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:53.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.684+0000 7f06d87e6700 1 --2- 172.21.15.16:0/1358554695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06d00613a0 0x7f06d0117cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:53.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.684+0000 7f06d87e6700 1 --2- 172.21.15.16:0/1358554695 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f06d0091ed0 0x7f06d0127940 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:53.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.684+0000 7f06d87e6700 1 -- 172.21.15.16:0/1358554695 >> 172.21.15.16:0/1358554695 conn(0x7f06d009a1c0 msgr2=0x7f06d0068d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:53.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.684+0000 7f06d87e6700 1 -- 172.21.15.16:0/1358554695 shutdown_connections 2023-09-30T22:26:53.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.684+0000 7f06d87e6700 1 -- 172.21.15.16:0/1358554695 wait complete. 2023-09-30T22:26:53.699 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-09-30T22:26:53.699 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-09-30T22:26:53.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.836+0000 7fa468805700 1 Processor -- start 2023-09-30T22:26:53.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.836+0000 7fa468805700 1 -- start start 2023-09-30T22:26:53.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.836+0000 7fa468805700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa464093840 0x7fa464098790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:53.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.836+0000 7fa468805700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa464098c80 0x7fa464099060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:53.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.836+0000 7fa468805700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa46412c150 0x7fa46412e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:53.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.836+0000 7fa468805700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa464093c20 con 0x7fa464098c80 2023-09-30T22:26:53.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.836+0000 7fa468805700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa464092b90 con 0x7fa46412c150 2023-09-30T22:26:53.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.836+0000 7fa468805700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa464092d80 con 0x7fa464093840 2023-09-30T22:26:53.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.837+0000 7fa461d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa464098c80 0x7fa464099060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:53.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.837+0000 7fa46259c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa464093840 0x7fa464098790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:53.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.837+0000 7fa462d9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa46412c150 0x7fa46412e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:53.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.837+0000 7fa461d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa464098c80 0x7fa464099060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38704/0 (socket says 172.21.15.16:38704) 2023-09-30T22:26:53.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.837+0000 7fa462d9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa46412c150 0x7fa46412e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:44576/0 (socket says 172.21.15.16:44576) 2023-09-30T22:26:53.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.837+0000 7fa46259c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa464093840 0x7fa464098790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:46236/0 (socket says 172.21.15.16:46236) 2023-09-30T22:26:53.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.837+0000 7fa461d9b700 1 -- 172.21.15.16:0/4108422170 learned_addr learned my addr 172.21.15.16:0/4108422170 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:26:53.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.837+0000 7fa46259c700 1 -- 172.21.15.16:0/4108422170 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa46412c150 msgr2=0x7fa46412e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:53.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.837+0000 7fa46259c700 1 --2- 172.21.15.16:0/4108422170 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa46412c150 0x7fa46412e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:53.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.837+0000 7fa46259c700 1 -- 172.21.15.16:0/4108422170 >> v2:172.21.15.16:3300/0 conn(0x7fa464098c80 msgr2=0x7fa464099060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:53.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.837+0000 7fa46259c700 1 --2- 172.21.15.16:0/4108422170 >> v2:172.21.15.16:3300/0 conn(0x7fa464098c80 0x7fa464099060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:53.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.837+0000 7fa46259c700 1 -- 172.21.15.16:0/4108422170 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa464099b70 con 0x7fa464093840 2023-09-30T22:26:53.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.837+0000 7fa462d9d700 1 --2- 172.21.15.16:0/4108422170 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa46412c150 0x7fa46412e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:53.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.837+0000 7fa461d9b700 1 --2- 172.21.15.16:0/4108422170 >> v2:172.21.15.16:3300/0 conn(0x7fa464098c80 0x7fa464099060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:53.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.837+0000 7fa46259c700 1 --2- 172.21.15.16:0/4108422170 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa464093840 0x7fa464098790 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fa454009760 tx=0x7fa454009060).ready entity=mon.2 client_cookie=f6b8469a9d69475e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:53.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.837+0000 7fa46159a700 1 -- 172.21.15.16:0/4108422170 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45400d4a0 con 0x7fa464093840 2023-09-30T22:26:53.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.837+0000 7fa46159a700 1 -- 172.21.15.16:0/4108422170 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa45400d600 con 0x7fa464093840 2023-09-30T22:26:53.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.837+0000 7fa46159a700 1 -- 172.21.15.16:0/4108422170 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45400d980 con 0x7fa464093840 2023-09-30T22:26:53.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.838+0000 7fa468805700 1 -- 172.21.15.16:0/4108422170 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa464093840 msgr2=0x7fa464098790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:53.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.838+0000 7fa468805700 1 --2- 172.21.15.16:0/4108422170 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa464093840 0x7fa464098790 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fa454009760 tx=0x7fa454009060).stop 2023-09-30T22:26:53.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.838+0000 7fa468805700 1 -- 172.21.15.16:0/4108422170 shutdown_connections 2023-09-30T22:26:53.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.838+0000 7fa468805700 1 --2- 172.21.15.16:0/4108422170 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa46412c150 0x7fa46412e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:53.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.838+0000 7fa468805700 1 --2- 172.21.15.16:0/4108422170 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa464093840 0x7fa464098790 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:53.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.838+0000 7fa468805700 1 --2- 172.21.15.16:0/4108422170 >> v2:172.21.15.16:3300/0 conn(0x7fa464098c80 0x7fa464099060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:53.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.838+0000 7fa468805700 1 -- 172.21.15.16:0/4108422170 >> 172.21.15.16:0/4108422170 conn(0x7fa464097480 msgr2=0x7fa464097890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:53.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.838+0000 7fa468805700 1 -- 172.21.15.16:0/4108422170 shutdown_connections 2023-09-30T22:26:53.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.838+0000 7fa468805700 1 -- 172.21.15.16:0/4108422170 wait complete. 2023-09-30T22:26:53.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.839+0000 7fa468805700 1 Processor -- start 2023-09-30T22:26:53.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.839+0000 7fa468805700 1 -- start start 2023-09-30T22:26:53.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.839+0000 7fa468805700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa464093840 0x7fa464116010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:53.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.839+0000 7fa468805700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa464098c80 0x7fa464116500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:53.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.839+0000 7fa468805700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa46412c150 0x7fa4641169f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:53.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.839+0000 7fa468805700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa464116ee0 con 0x7fa46412c150 2023-09-30T22:26:53.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.839+0000 7fa468805700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa464117aa0 con 0x7fa464098c80 2023-09-30T22:26:53.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.839+0000 7fa468805700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa464119170 con 0x7fa464093840 2023-09-30T22:26:53.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.839+0000 7fa46259c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa464093840 0x7fa464116010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:53.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.839+0000 7fa461d9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa464098c80 0x7fa464116500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:53.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.839+0000 7fa46259c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa464093840 0x7fa464116010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:46248/0 (socket says 172.21.15.16:46248) 2023-09-30T22:26:53.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.840+0000 7fa46259c700 1 -- 172.21.15.16:0/3025382928 learned_addr learned my addr 172.21.15.16:0/3025382928 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:26:53.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.840+0000 7fa462d9d700 1 --2- 172.21.15.16:0/3025382928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa46412c150 0x7fa4641169f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:53.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.840+0000 7fa461d9b700 1 -- 172.21.15.16:0/3025382928 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa464093840 msgr2=0x7fa464116010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:53.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.840+0000 7fa461d9b700 1 --2- 172.21.15.16:0/3025382928 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa464093840 0x7fa464116010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:53.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.840+0000 7fa461d9b700 1 -- 172.21.15.16:0/3025382928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa46412c150 msgr2=0x7fa4641169f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:53.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.840+0000 7fa461d9b700 1 --2- 172.21.15.16:0/3025382928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa46412c150 0x7fa4641169f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:53.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.840+0000 7fa461d9b700 1 -- 172.21.15.16:0/3025382928 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa45800b610 con 0x7fa464098c80 2023-09-30T22:26:53.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.840+0000 7fa46259c700 1 --2- 172.21.15.16:0/3025382928 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa464093840 0x7fa464116010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:53.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.840+0000 7fa462d9d700 1 --2- 172.21.15.16:0/3025382928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa46412c150 0x7fa4641169f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:53.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.840+0000 7fa461d9b700 1 --2- 172.21.15.16:0/3025382928 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa464098c80 0x7fa464116500 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fa45800bfd0 tx=0x7fa45800b3b0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:53.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.840+0000 7fa4527fc700 1 -- 172.21.15.16:0/3025382928 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45800b8f0 con 0x7fa464098c80 2023-09-30T22:26:53.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.840+0000 7fa4527fc700 1 -- 172.21.15.16:0/3025382928 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa45800ee10 con 0x7fa464098c80 2023-09-30T22:26:53.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.840+0000 7fa468805700 1 -- 172.21.15.16:0/3025382928 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa45c009a20 con 0x7fa464098c80 2023-09-30T22:26:53.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.840+0000 7fa4527fc700 1 -- 172.21.15.16:0/3025382928 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa45801a680 con 0x7fa464098c80 2023-09-30T22:26:53.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.840+0000 7fa468805700 1 -- 172.21.15.16:0/3025382928 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4540092a0 con 0x7fa464098c80 2023-09-30T22:26:53.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.841+0000 7fa4527fc700 1 -- 172.21.15.16:0/3025382928 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fa458017070 con 0x7fa464098c80 2023-09-30T22:26:53.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.841+0000 7fa4527fc700 1 -- 172.21.15.16:0/3025382928 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fa45803dc00 con 0x7fa464098c80 2023-09-30T22:26:53.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.841+0000 7fa468805700 1 -- 172.21.15.16:0/3025382928 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa46405a260 con 0x7fa464098c80 2023-09-30T22:26:53.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:53.846+0000 7fa4527fc700 1 -- 172.21.15.16:0/3025382928 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fa4580243f0 con 0x7fa464098c80 2023-09-30T22:26:54.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.060+0000 7fa468805700 1 -- 172.21.15.16:0/3025382928 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fa464096ca0 con 0x7fa464098c80 2023-09-30T22:26:54.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.685+0000 7fa4527fc700 1 -- 172.21.15.16:0/3025382928 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa458018370 con 0x7fa464098c80 2023-09-30T22:26:54.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.687+0000 7fa468805700 1 -- 172.21.15.16:0/3025382928 >> v1:172.21.15.16:6816/118285 conn(0x7fa44402bb60 legacy=0x7fa44402e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:54.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.687+0000 7fa468805700 1 -- 172.21.15.16:0/3025382928 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa464098c80 msgr2=0x7fa464116500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:54.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.687+0000 7fa468805700 1 --2- 172.21.15.16:0/3025382928 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa464098c80 0x7fa464116500 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fa45800bfd0 tx=0x7fa45800b3b0).stop 2023-09-30T22:26:54.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.687+0000 7fa468805700 1 -- 172.21.15.16:0/3025382928 shutdown_connections 2023-09-30T22:26:54.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.687+0000 7fa468805700 1 --2- 172.21.15.16:0/3025382928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa46412c150 0x7fa4641169f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:54.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.687+0000 7fa468805700 1 --2- 172.21.15.16:0/3025382928 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa464093840 0x7fa464116010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:54.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.687+0000 7fa468805700 1 --2- 172.21.15.16:0/3025382928 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa464098c80 0x7fa464116500 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:54.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.687+0000 7fa468805700 1 -- 172.21.15.16:0/3025382928 >> 172.21.15.16:0/3025382928 conn(0x7fa464097480 msgr2=0x7fa464068aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:54.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.688+0000 7fa468805700 1 -- 172.21.15.16:0/3025382928 shutdown_connections 2023-09-30T22:26:54.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.688+0000 7fa468805700 1 -- 172.21.15.16:0/3025382928 wait complete. 2023-09-30T22:26:54.702 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-09-30T22:26:54.703 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-30T22:26:54.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.837+0000 7f75b9908700 1 Processor -- start 2023-09-30T22:26:54.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.837+0000 7f75b9908700 1 -- start start 2023-09-30T22:26:54.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.837+0000 7f75b9908700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f75b4097c30 0x7f75b4098db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:54.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.837+0000 7f75b9908700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f75b40992a0 0x7f75b4132480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:54.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.837+0000 7f75b9908700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75b4096160 con 0x7f75b40650f0 2023-09-30T22:26:54.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.837+0000 7f75b9908700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75b40654d0 con 0x7f75b40992a0 2023-09-30T22:26:54.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.837+0000 7f75b9908700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f75b4065dd0 con 0x7f75b4097c30 2023-09-30T22:26:54.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.837+0000 7f75b37fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f75b40992a0 0x7f75b4132480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:54.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.837+0000 7f75b27fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f75b4097c30 0x7f75b4098db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:54.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.837+0000 7f75b2ffd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f75b40650f0 0x7f75b40986a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33698/0 (socket says 172.21.15.16:33698) 2023-09-30T22:26:54.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.837+0000 7f75b2ffd700 1 -- 172.21.15.16:0/2502198898 learned_addr learned my addr 172.21.15.16:0/2502198898 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:26:54.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.838+0000 7f75b27fc700 1 -- 172.21.15.16:0/2502198898 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f75b40992a0 msgr2=0x7f75b4132480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:54.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.838+0000 7f75b27fc700 1 --2- 172.21.15.16:0/2502198898 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f75b40992a0 0x7f75b4132480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:54.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.838+0000 7f75b27fc700 1 -- 172.21.15.16:0/2502198898 >> v1:172.21.15.16:6789/0 conn(0x7f75b40650f0 legacy=0x7f75b40986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:54.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.838+0000 7f75b27fc700 1 -- 172.21.15.16:0/2502198898 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b4099790 con 0x7f75b4097c30 2023-09-30T22:26:54.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.838+0000 7f75b1ffb700 1 -- 172.21.15.16:0/2502198898 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2732961641 0 0) 0x7f75b4096160 con 0x7f75b40650f0 2023-09-30T22:26:54.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.838+0000 7f75b37fe700 1 --2- 172.21.15.16:0/2502198898 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f75b40992a0 0x7f75b4132480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:54.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.838+0000 7f75b27fc700 1 --2- 172.21.15.16:0/2502198898 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f75b4097c30 0x7f75b4098db0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f759c009790 tx=0x7f759c009060).ready entity=mon.2 client_cookie=b67337a332c9881a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:54.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.838+0000 7f75b1ffb700 1 -- 172.21.15.16:0/2502198898 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759c00f510 con 0x7f75b4097c30 2023-09-30T22:26:54.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.838+0000 7f75b1ffb700 1 -- 172.21.15.16:0/2502198898 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f759c00f670 con 0x7f75b4097c30 2023-09-30T22:26:54.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.838+0000 7f75b1ffb700 1 -- 172.21.15.16:0/2502198898 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759c00f9f0 con 0x7f75b4097c30 2023-09-30T22:26:54.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.838+0000 7f75b9908700 1 -- 172.21.15.16:0/2502198898 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f75b4097c30 msgr2=0x7f75b4098db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:54.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.838+0000 7f75b9908700 1 --2- 172.21.15.16:0/2502198898 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f75b4097c30 0x7f75b4098db0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f759c009790 tx=0x7f759c009060).stop 2023-09-30T22:26:54.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.839+0000 7f75b9908700 1 -- 172.21.15.16:0/2502198898 shutdown_connections 2023-09-30T22:26:54.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.839+0000 7f75b9908700 1 --2- 172.21.15.16:0/2502198898 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f75b40992a0 0x7f75b4132480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:54.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.839+0000 7f75b9908700 1 --2- 172.21.15.16:0/2502198898 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f75b4097c30 0x7f75b4098db0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:54.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.839+0000 7f75b9908700 1 -- 172.21.15.16:0/2502198898 >> 172.21.15.16:0/2502198898 conn(0x7f75b409dc70 msgr2=0x7f75b409e050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:54.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.839+0000 7f75b9908700 1 -- 172.21.15.16:0/2502198898 shutdown_connections 2023-09-30T22:26:54.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.839+0000 7f75b9908700 1 -- 172.21.15.16:0/2502198898 wait complete. 2023-09-30T22:26:54.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.840+0000 7f75b9908700 1 Processor -- start 2023-09-30T22:26:54.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.840+0000 7f75b9908700 1 -- start start 2023-09-30T22:26:54.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.840+0000 7f75b9908700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f75b40650f0 0x7f75b411a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:54.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.840+0000 7f75b9908700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f75b40992a0 0x7f75b411acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:54.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.840+0000 7f75b9908700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75b411b1c0 0x7f75b413ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:54.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.840+0000 7f75b9908700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b411bc50 con 0x7f75b411b1c0 2023-09-30T22:26:54.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.840+0000 7f75b9908700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75b411b640 con 0x7f75b40650f0 2023-09-30T22:26:54.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.840+0000 7f75b9908700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f75b411da20 con 0x7f75b40992a0 2023-09-30T22:26:54.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.840+0000 7f75b2ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f75b40650f0 0x7f75b411a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:54.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.840+0000 7f75b37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75b411b1c0 0x7f75b413ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:54.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.840+0000 7f75b2ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f75b40650f0 0x7f75b411a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:44592/0 (socket says 172.21.15.16:44592) 2023-09-30T22:26:54.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.840+0000 7f75b2ffd700 1 -- 172.21.15.16:0/446007745 learned_addr learned my addr 172.21.15.16:0/446007745 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:26:54.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.840+0000 7f75b27fc700 1 --2- 172.21.15.16:0/446007745 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f75b40992a0 0x7f75b411acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:54.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.841+0000 7f75b37fe700 1 -- 172.21.15.16:0/446007745 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f75b40992a0 msgr2=0x7f75b411acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:54.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.841+0000 7f75b37fe700 1 --2- 172.21.15.16:0/446007745 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f75b40992a0 0x7f75b411acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:54.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.841+0000 7f75b37fe700 1 -- 172.21.15.16:0/446007745 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f75b40650f0 msgr2=0x7f75b411a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:54.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.841+0000 7f75b37fe700 1 --2- 172.21.15.16:0/446007745 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f75b40650f0 0x7f75b411a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:54.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.841+0000 7f75b37fe700 1 -- 172.21.15.16:0/446007745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75ac0092a0 con 0x7f75b411b1c0 2023-09-30T22:26:54.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.841+0000 7f75b37fe700 1 --2- 172.21.15.16:0/446007745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75b411b1c0 0x7f75b413ab10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f75ac00e930 tx=0x7f75ac009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:54.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.841+0000 7f75aaffd700 1 -- 172.21.15.16:0/446007745 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ac0094f0 con 0x7f75b411b1c0 2023-09-30T22:26:54.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.841+0000 7f75aaffd700 1 -- 172.21.15.16:0/446007745 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75ac012600 con 0x7f75b411b1c0 2023-09-30T22:26:54.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.841+0000 7f75b9908700 1 -- 172.21.15.16:0/446007745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f759c0092a0 con 0x7f75b411b1c0 2023-09-30T22:26:54.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.841+0000 7f75aaffd700 1 -- 172.21.15.16:0/446007745 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ac012980 con 0x7f75b411b1c0 2023-09-30T22:26:54.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.841+0000 7f75b9908700 1 -- 172.21.15.16:0/446007745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75b409cb60 con 0x7f75b411b1c0 2023-09-30T22:26:54.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.842+0000 7f75aaffd700 1 -- 172.21.15.16:0/446007745 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f75ac01a3b0 con 0x7f75b411b1c0 2023-09-30T22:26:54.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.842+0000 7f75aaffd700 1 -- 172.21.15.16:0/446007745 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f75ac03ad30 con 0x7f75b411b1c0 2023-09-30T22:26:54.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.842+0000 7f75b9908700 1 -- 172.21.15.16:0/446007745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75b4099a80 con 0x7f75b411b1c0 2023-09-30T22:26:54.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:54.847+0000 7f75aaffd700 1 -- 172.21.15.16:0/446007745 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f75ac019310 con 0x7f75b411b1c0 2023-09-30T22:26:55.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.066+0000 7f75b9908700 1 -- 172.21.15.16:0/446007745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f75b405a260 con 0x7f75b411b1c0 2023-09-30T22:26:55.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.067+0000 7f75aaffd700 1 -- 172.21.15.16:0/446007745 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+2156 (secure 0 0 0) 0x7f75ac018ab0 con 0x7f75b411b1c0 2023-09-30T22:26:55.067 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:26:55.067 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-30T22:26:51.674561+0000","modified":"2023-09-30T22:26:54.680606+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4218},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4218":{"gid":4218,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6817/1424366610","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6817","nonce":1424366610}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-09-30T22:26:55.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.068+0000 7f75b9908700 1 -- 172.21.15.16:0/446007745 >> v1:172.21.15.16:6816/118285 conn(0x7f759402bb60 legacy=0x7f759402e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:55.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.068+0000 7f75b9908700 1 -- 172.21.15.16:0/446007745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75b411b1c0 msgr2=0x7f75b413ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:55.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.068+0000 7f75b9908700 1 --2- 172.21.15.16:0/446007745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75b411b1c0 0x7f75b413ab10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f75ac00e930 tx=0x7f75ac009060).stop 2023-09-30T22:26:55.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.068+0000 7f75b9908700 1 -- 172.21.15.16:0/446007745 shutdown_connections 2023-09-30T22:26:55.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.068+0000 7f75b9908700 1 --2- 172.21.15.16:0/446007745 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f75b40650f0 0x7f75b411a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:55.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.068+0000 7f75b9908700 1 --2- 172.21.15.16:0/446007745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75b411b1c0 0x7f75b413ab10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:55.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.068+0000 7f75b9908700 1 --2- 172.21.15.16:0/446007745 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f75b40992a0 0x7f75b411acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:55.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.068+0000 7f75b9908700 1 -- 172.21.15.16:0/446007745 >> 172.21.15.16:0/446007745 conn(0x7f75b409dc70 msgr2=0x7f75b41142d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:55.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.069+0000 7f75b9908700 1 -- 172.21.15.16:0/446007745 shutdown_connections 2023-09-30T22:26:55.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.069+0000 7f75b9908700 1 -- 172.21.15.16:0/446007745 wait complete. 2023-09-30T22:26:55.070 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 6 2023-09-30T22:26:55.082 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:26:55.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.211+0000 7f6c1361b700 1 Processor -- start 2023-09-30T22:26:55.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.211+0000 7f6c1361b700 1 -- start start 2023-09-30T22:26:55.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.212+0000 7f6c1361b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6c0c0613a0 0x7f6c0c12ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:55.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.212+0000 7f6c1361b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6c0c097cb0 0x7f6c0c12b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:55.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.212+0000 7f6c1361b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6c0c12b660 0x7f6c0c095df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:55.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.212+0000 7f6c1361b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c0c061780 con 0x7f6c0c12b660 2023-09-30T22:26:55.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.212+0000 7f6c1361b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0c067530 con 0x7f6c0c097cb0 2023-09-30T22:26:55.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.212+0000 7f6c1361b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0c066ee0 con 0x7f6c0c0613a0 2023-09-30T22:26:55.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.212+0000 7f6c113b7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6c0c0613a0 0x7f6c0c12ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:55.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.212+0000 7f6c10bb6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6c0c097cb0 0x7f6c0c12b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:55.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.212+0000 7f6c11bb8700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6c0c12b660 0x7f6c0c095df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:55.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.212+0000 7f6c113b7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6c0c0613a0 0x7f6c0c12ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:46272/0 (socket says 172.21.15.16:46272) 2023-09-30T22:26:55.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.212+0000 7f6c10bb6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6c0c097cb0 0x7f6c0c12b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:44594/0 (socket says 172.21.15.16:44594) 2023-09-30T22:26:55.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.212+0000 7f6c113b7700 1 -- 172.21.15.16:0/3450321774 learned_addr learned my addr 172.21.15.16:0/3450321774 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:26:55.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.212+0000 7f6c11bb8700 1 --2- 172.21.15.16:0/3450321774 >> v2:172.21.15.16:3300/0 conn(0x7f6c0c12b660 0x7f6c0c095df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38718/0 (socket says 172.21.15.16:38718) 2023-09-30T22:26:55.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.212+0000 7f6c10bb6700 1 -- 172.21.15.16:0/3450321774 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6c0c0613a0 msgr2=0x7f6c0c12ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:55.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.212+0000 7f6c10bb6700 1 --2- 172.21.15.16:0/3450321774 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6c0c0613a0 0x7f6c0c12ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:55.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.212+0000 7f6c10bb6700 1 -- 172.21.15.16:0/3450321774 >> v2:172.21.15.16:3300/0 conn(0x7f6c0c12b660 msgr2=0x7f6c0c095df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:55.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.212+0000 7f6c10bb6700 1 --2- 172.21.15.16:0/3450321774 >> v2:172.21.15.16:3300/0 conn(0x7f6c0c12b660 0x7f6c0c095df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:55.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.212+0000 7f6c10bb6700 1 -- 172.21.15.16:0/3450321774 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c0c0962e0 con 0x7f6c0c097cb0 2023-09-30T22:26:55.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.212+0000 7f6c113b7700 1 --2- 172.21.15.16:0/3450321774 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6c0c0613a0 0x7f6c0c12ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:55.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.212+0000 7f6c11bb8700 1 --2- 172.21.15.16:0/3450321774 >> v2:172.21.15.16:3300/0 conn(0x7f6c0c12b660 0x7f6c0c095df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:55.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.212+0000 7f6c10bb6700 1 --2- 172.21.15.16:0/3450321774 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6c0c097cb0 0x7f6c0c12b170 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f6bfc009760 tx=0x7f6bfc009060).ready entity=mon.1 client_cookie=c6bf2ad0c62e6e1f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:55.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.213+0000 7f6c03fff700 1 -- 172.21.15.16:0/3450321774 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc00d4a0 con 0x7f6c0c097cb0 2023-09-30T22:26:55.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.213+0000 7f6c03fff700 1 -- 172.21.15.16:0/3450321774 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bfc00d600 con 0x7f6c0c097cb0 2023-09-30T22:26:55.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.213+0000 7f6c03fff700 1 -- 172.21.15.16:0/3450321774 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bfc00d980 con 0x7f6c0c097cb0 2023-09-30T22:26:55.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.213+0000 7f6c1361b700 1 -- 172.21.15.16:0/3450321774 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6c0c097cb0 msgr2=0x7f6c0c12b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:55.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.213+0000 7f6c1361b700 1 --2- 172.21.15.16:0/3450321774 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6c0c097cb0 0x7f6c0c12b170 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f6bfc009760 tx=0x7f6bfc009060).stop 2023-09-30T22:26:55.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.213+0000 7f6c1361b700 1 -- 172.21.15.16:0/3450321774 shutdown_connections 2023-09-30T22:26:55.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.213+0000 7f6c1361b700 1 --2- 172.21.15.16:0/3450321774 >> v2:172.21.15.16:3300/0 conn(0x7f6c0c12b660 0x7f6c0c095df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:55.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.213+0000 7f6c1361b700 1 --2- 172.21.15.16:0/3450321774 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6c0c0613a0 0x7f6c0c12ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:55.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.213+0000 7f6c1361b700 1 --2- 172.21.15.16:0/3450321774 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6c0c097cb0 0x7f6c0c12b170 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:55.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.213+0000 7f6c1361b700 1 -- 172.21.15.16:0/3450321774 >> 172.21.15.16:0/3450321774 conn(0x7f6c0c09a220 msgr2=0x7f6c0c09a630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:55.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.213+0000 7f6c1361b700 1 -- 172.21.15.16:0/3450321774 shutdown_connections 2023-09-30T22:26:55.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.213+0000 7f6c1361b700 1 -- 172.21.15.16:0/3450321774 wait complete. 2023-09-30T22:26:55.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.214+0000 7f6c1361b700 1 Processor -- start 2023-09-30T22:26:55.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.214+0000 7f6c1361b700 1 -- start start 2023-09-30T22:26:55.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.215+0000 7f6c1361b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6c0c0613a0 0x7f6c0c13a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:55.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.215+0000 7f6c1361b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c0c097cb0 0x7f6c0c13ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:55.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.215+0000 7f6c1361b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6c0c12b660 0x7f6c0c114ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:55.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.215+0000 7f6c1361b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0c118c50 con 0x7f6c0c097cb0 2023-09-30T22:26:55.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.215+0000 7f6c1361b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0c116350 con 0x7f6c0c12b660 2023-09-30T22:26:55.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.215+0000 7f6c1361b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0c098cb0 con 0x7f6c0c0613a0 2023-09-30T22:26:55.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.215+0000 7f6c113b7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6c0c0613a0 0x7f6c0c13a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:55.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.215+0000 7f6c11bb8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6c0c12b660 0x7f6c0c114ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:55.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.215+0000 7f6c113b7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6c0c0613a0 0x7f6c0c13a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:46286/0 (socket says 172.21.15.16:46286) 2023-09-30T22:26:55.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.215+0000 7f6c113b7700 1 -- 172.21.15.16:0/2842899039 learned_addr learned my addr 172.21.15.16:0/2842899039 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:26:55.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.215+0000 7f6c10bb6700 1 --2- 172.21.15.16:0/2842899039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c0c097cb0 0x7f6c0c13ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:55.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.215+0000 7f6c113b7700 1 -- 172.21.15.16:0/2842899039 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6c0c12b660 msgr2=0x7f6c0c114ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:55.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.215+0000 7f6c113b7700 1 --2- 172.21.15.16:0/2842899039 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6c0c12b660 0x7f6c0c114ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:55.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.215+0000 7f6c113b7700 1 -- 172.21.15.16:0/2842899039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c0c097cb0 msgr2=0x7f6c0c13ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:55.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.215+0000 7f6c113b7700 1 --2- 172.21.15.16:0/2842899039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c0c097cb0 0x7f6c0c13ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:55.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.215+0000 7f6c113b7700 1 -- 172.21.15.16:0/2842899039 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c08009370 con 0x7f6c0c0613a0 2023-09-30T22:26:55.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.216+0000 7f6c113b7700 1 --2- 172.21.15.16:0/2842899039 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6c0c0613a0 0x7f6c0c13a620 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f6c0800ea20 tx=0x7f6c080090f0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:55.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.216+0000 7f6c017fa700 1 -- 172.21.15.16:0/2842899039 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c08009650 con 0x7f6c0c0613a0 2023-09-30T22:26:55.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.216+0000 7f6c017fa700 1 -- 172.21.15.16:0/2842899039 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c0800c850 con 0x7f6c0c0613a0 2023-09-30T22:26:55.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.216+0000 7f6c017fa700 1 -- 172.21.15.16:0/2842899039 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0800cbd0 con 0x7f6c0c0613a0 2023-09-30T22:26:55.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.216+0000 7f6c1361b700 1 -- 172.21.15.16:0/2842899039 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c04009a20 con 0x7f6c0c0613a0 2023-09-30T22:26:55.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.216+0000 7f6c1361b700 1 -- 172.21.15.16:0/2842899039 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bfc0092a0 con 0x7f6c0c0613a0 2023-09-30T22:26:55.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.217+0000 7f6c017fa700 1 -- 172.21.15.16:0/2842899039 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f6c0801b400 con 0x7f6c0c0613a0 2023-09-30T22:26:55.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.217+0000 7f6c017fa700 1 -- 172.21.15.16:0/2842899039 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f6c0803b520 con 0x7f6c0c0613a0 2023-09-30T22:26:55.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.217+0000 7f6c1361b700 1 -- 172.21.15.16:0/2842899039 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c0c05a260 con 0x7f6c0c0613a0 2023-09-30T22:26:55.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.223+0000 7f6c017fa700 1 -- 172.21.15.16:0/2842899039 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f6c08016070 con 0x7f6c0c0613a0 2023-09-30T22:26:55.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.401+0000 7f6c1361b700 1 -- 172.21.15.16:0/2842899039 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6c0c0696c0 con 0x7f6c0c0613a0 2023-09-30T22:26:55.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.402+0000 7f6c017fa700 1 -- 172.21.15.16:0/2842899039 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+11341 (secure 0 0 0) 0x7f6c08016020 con 0x7f6c0c0613a0 2023-09-30T22:26:55.402 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:26:55.403 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":16,"fsid":"7ede3e14-2bd0-4726-9437-6d13c90cd342","created":"2023-09-30T22:26:35.483697+0000","modified":"2023-09-30T22:26:52.675888+0000","last_up_change":"2023-09-30T22:26:44.227199+0000","last_in_change":"2023-09-30T22:26:37.120906+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-09-30T22:26:44.285133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:26:45.132921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:26:49.353683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:26:50.655685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"19946985-0ee2-487f-9347-929670dbe5ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6804","nonce":118824}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6805","nonce":118824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6807","nonce":118824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6806","nonce":118824}]},"public_addr":"172.21.15.16:6804/118824","cluster_addr":"172.21.15.16:6805/118824","heartbeat_back_addr":"172.21.15.16:6807/118824","heartbeat_front_addr":"172.21.15.16:6806/118824","state":["exists","up"]},{"osd":1,"uuid":"8efd4dbc-6ae4-4b82-b530-3b3eac2be445","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6812","nonce":118832}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6813","nonce":118832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6815","nonce":118832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6814","nonce":118832}]},"public_addr":"172.21.15.16:6812/118832","cluster_addr":"172.21.15.16:6813/118832","heartbeat_back_addr":"172.21.15.16:6815/118832","heartbeat_front_addr":"172.21.15.16:6814/118832","state":["exists","up"]},{"osd":2,"uuid":"b47bb1ce-27af-4c0c-868a-b76ebd0d9499","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6808","nonce":118840}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6809","nonce":118840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6811","nonce":118840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6810","nonce":118840}]},"public_addr":"172.21.15.16:6808/118840","cluster_addr":"172.21.15.16:6809/118840","heartbeat_back_addr":"172.21.15.16:6811/118840","heartbeat_front_addr":"172.21.15.16:6810/118840","state":["exists","up"]},{"osd":3,"uuid":"03b53d64-d10d-448a-a081-39ecceab589b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6800","nonce":118823}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6801","nonce":118823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6803","nonce":118823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6802","nonce":118823}]},"public_addr":"172.21.15.16:6800/118823","cluster_addr":"172.21.15.16:6801/118823","heartbeat_back_addr":"172.21.15.16:6803/118823","heartbeat_front_addr":"172.21.15.16:6802/118823","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.305953+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.307488+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.341023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.192071+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:26:55.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.404+0000 7f6c1361b700 1 -- 172.21.15.16:0/2842899039 >> v1:172.21.15.16:6816/118285 conn(0x7f6bf002bb90 legacy=0x7f6bf002e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:55.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.404+0000 7f6c1361b700 1 -- 172.21.15.16:0/2842899039 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6c0c0613a0 msgr2=0x7f6c0c13a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:55.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.404+0000 7f6c1361b700 1 --2- 172.21.15.16:0/2842899039 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6c0c0613a0 0x7f6c0c13a620 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f6c0800ea20 tx=0x7f6c080090f0).stop 2023-09-30T22:26:55.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.404+0000 7f6c1361b700 1 -- 172.21.15.16:0/2842899039 shutdown_connections 2023-09-30T22:26:55.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.404+0000 7f6c1361b700 1 --2- 172.21.15.16:0/2842899039 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6c0c12b660 0x7f6c0c114ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:55.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.404+0000 7f6c1361b700 1 --2- 172.21.15.16:0/2842899039 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6c0c0613a0 0x7f6c0c13a620 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:55.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.404+0000 7f6c1361b700 1 --2- 172.21.15.16:0/2842899039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c0c097cb0 0x7f6c0c13ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:55.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.404+0000 7f6c1361b700 1 -- 172.21.15.16:0/2842899039 >> 172.21.15.16:0/2842899039 conn(0x7f6c0c09a220 msgr2=0x7f6c0c065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:55.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.404+0000 7f6c1361b700 1 -- 172.21.15.16:0/2842899039 shutdown_connections 2023-09-30T22:26:55.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.404+0000 7f6c1361b700 1 -- 172.21.15.16:0/2842899039 wait complete. 2023-09-30T22:26:55.418 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-09-30T22:26:55.418 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-30T22:26:55.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.548+0000 7fc337380700 1 Processor -- start 2023-09-30T22:26:55.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.548+0000 7fc337380700 1 -- start start 2023-09-30T22:26:55.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.549+0000 7fc337380700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc3300661b0 0x7fc33012aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:55.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.549+0000 7fc337380700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc330064570 0x7fc33012b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:55.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.549+0000 7fc337380700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3300656f0 con 0x7fc330127570 2023-09-30T22:26:55.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.549+0000 7fc337380700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc330066590 con 0x7fc3300661b0 2023-09-30T22:26:55.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.549+0000 7fc337380700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc3300649f0 con 0x7fc330064570 2023-09-30T22:26:55.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.549+0000 7fc33491b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc330064570 0x7fc33012b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:55.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.549+0000 7fc33511c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc3300661b0 0x7fc33012aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:55.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.549+0000 7fc33591d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc330127570 0x7fc3301279f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33700/0 (socket says 172.21.15.16:33700) 2023-09-30T22:26:55.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.549+0000 7fc33491b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc330064570 0x7fc33012b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:46298/0 (socket says 172.21.15.16:46298) 2023-09-30T22:26:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.549+0000 7fc33591d700 1 -- 172.21.15.16:0/2762673991 learned_addr learned my addr 172.21.15.16:0/2762673991 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:26:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.549+0000 7fc327fff700 1 -- 172.21.15.16:0/2762673991 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3882770973 0 0) 0x7fc3300656f0 con 0x7fc330127570 2023-09-30T22:26:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.549+0000 7fc327fff700 1 -- 172.21.15.16:0/2762673991 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc31c0033f0 con 0x7fc330127570 2023-09-30T22:26:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.550+0000 7fc33511c700 1 -- 172.21.15.16:0/2762673991 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc330064570 msgr2=0x7fc33012b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.550+0000 7fc33511c700 1 --2- 172.21.15.16:0/2762673991 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc330064570 0x7fc33012b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.550+0000 7fc33511c700 1 -- 172.21.15.16:0/2762673991 >> v1:172.21.15.16:6789/0 conn(0x7fc330127570 legacy=0x7fc3301279f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.550+0000 7fc33511c700 1 -- 172.21.15.16:0/2762673991 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc33012af80 con 0x7fc3300661b0 2023-09-30T22:26:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.550+0000 7fc33491b700 1 --2- 172.21.15.16:0/2762673991 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc330064570 0x7fc33012b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.550+0000 7fc33511c700 1 --2- 172.21.15.16:0/2762673991 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc3300661b0 0x7fc33012aa90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fc32c009790 tx=0x7fc32c009060).ready entity=mon.1 client_cookie=5006f653b2114752 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.550+0000 7fc327fff700 1 -- 172.21.15.16:0/2762673991 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32c00f510 con 0x7fc3300661b0 2023-09-30T22:26:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.550+0000 7fc327fff700 1 -- 172.21.15.16:0/2762673991 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc32c00f670 con 0x7fc3300661b0 2023-09-30T22:26:55.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.550+0000 7fc327fff700 1 -- 172.21.15.16:0/2762673991 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32c00f9f0 con 0x7fc3300661b0 2023-09-30T22:26:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.550+0000 7fc337380700 1 -- 172.21.15.16:0/2762673991 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc3300661b0 msgr2=0x7fc33012aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.550+0000 7fc337380700 1 --2- 172.21.15.16:0/2762673991 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc3300661b0 0x7fc33012aa90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fc32c009790 tx=0x7fc32c009060).stop 2023-09-30T22:26:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.550+0000 7fc337380700 1 -- 172.21.15.16:0/2762673991 shutdown_connections 2023-09-30T22:26:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.550+0000 7fc337380700 1 --2- 172.21.15.16:0/2762673991 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc3300661b0 0x7fc33012aa90 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.550+0000 7fc337380700 1 --2- 172.21.15.16:0/2762673991 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc330064570 0x7fc33012b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.551+0000 7fc337380700 1 -- 172.21.15.16:0/2762673991 >> 172.21.15.16:0/2762673991 conn(0x7fc33009ba50 msgr2=0x7fc33009be60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.551+0000 7fc337380700 1 -- 172.21.15.16:0/2762673991 shutdown_connections 2023-09-30T22:26:55.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.551+0000 7fc337380700 1 -- 172.21.15.16:0/2762673991 wait complete. 2023-09-30T22:26:55.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.551+0000 7fc337380700 1 Processor -- start 2023-09-30T22:26:55.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.552+0000 7fc337380700 1 -- start start 2023-09-30T22:26:55.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.552+0000 7fc337380700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc330064570 0x7fc330124ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:55.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.552+0000 7fc337380700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc3300661b0 0x7fc330113d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:55.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.552+0000 7fc337380700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc330127570 0x7fc330114200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:55.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.552+0000 7fc337380700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc330117780 con 0x7fc330064570 2023-09-30T22:26:55.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.552+0000 7fc337380700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3300650b0 con 0x7fc3300661b0 2023-09-30T22:26:55.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.552+0000 7fc337380700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc3301180b0 con 0x7fc330127570 2023-09-30T22:26:55.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.552+0000 7fc33511c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc330064570 0x7fc330124ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:55.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.552+0000 7fc33491b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc3300661b0 0x7fc330113d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:55.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.552+0000 7fc33591d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc330127570 0x7fc330114200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:55.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.552+0000 7fc33511c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc330064570 0x7fc330124ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38734/0 (socket says 172.21.15.16:38734) 2023-09-30T22:26:55.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.552+0000 7fc33491b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc3300661b0 0x7fc330113d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:44632/0 (socket says 172.21.15.16:44632) 2023-09-30T22:26:55.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.552+0000 7fc33511c700 1 -- 172.21.15.16:0/1193488512 learned_addr learned my addr 172.21.15.16:0/1193488512 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:26:55.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.553+0000 7fc33491b700 1 -- 172.21.15.16:0/1193488512 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc330127570 msgr2=0x7fc330114200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:55.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.553+0000 7fc33491b700 1 --2- 172.21.15.16:0/1193488512 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc330127570 0x7fc330114200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:55.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.553+0000 7fc33491b700 1 -- 172.21.15.16:0/1193488512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc330064570 msgr2=0x7fc330124ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:55.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.553+0000 7fc33491b700 1 --2- 172.21.15.16:0/1193488512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc330064570 0x7fc330124ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:55.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.553+0000 7fc33491b700 1 -- 172.21.15.16:0/1193488512 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3200092a0 con 0x7fc3300661b0 2023-09-30T22:26:55.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.553+0000 7fc33511c700 1 --2- 172.21.15.16:0/1193488512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc330064570 0x7fc330124ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:55.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.553+0000 7fc33491b700 1 --2- 172.21.15.16:0/1193488512 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc3300661b0 0x7fc330113d10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fc32000e950 tx=0x7fc320009060).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:55.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.553+0000 7fc3257fa700 1 -- 172.21.15.16:0/1193488512 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc320009580 con 0x7fc3300661b0 2023-09-30T22:26:55.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.553+0000 7fc3257fa700 1 -- 172.21.15.16:0/1193488512 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc320012600 con 0x7fc3300661b0 2023-09-30T22:26:55.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.553+0000 7fc337380700 1 -- 172.21.15.16:0/1193488512 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc32c0092a0 con 0x7fc3300661b0 2023-09-30T22:26:55.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.553+0000 7fc3257fa700 1 -- 172.21.15.16:0/1193488512 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3200129c0 con 0x7fc3300661b0 2023-09-30T22:26:55.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.553+0000 7fc337380700 1 -- 172.21.15.16:0/1193488512 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3301171c0 con 0x7fc3300661b0 2023-09-30T22:26:55.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.554+0000 7fc3257fa700 1 -- 172.21.15.16:0/1193488512 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fc320012d30 con 0x7fc3300661b0 2023-09-30T22:26:55.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.554+0000 7fc3257fa700 1 -- 172.21.15.16:0/1193488512 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fc32003b3b0 con 0x7fc3300661b0 2023-09-30T22:26:55.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.554+0000 7fc337380700 1 -- 172.21.15.16:0/1193488512 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc330097df0 con 0x7fc3300661b0 2023-09-30T22:26:55.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.559+0000 7fc3257fa700 1 -- 172.21.15.16:0/1193488512 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fc320011b40 con 0x7fc3300661b0 2023-09-30T22:26:55.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.730+0000 7fc337380700 1 -- 172.21.15.16:0/1193488512 --> v1:172.21.15.16:6816/118285 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc3301175c0 con 0x7fc31402bb90 2023-09-30T22:26:55.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.737+0000 7fc3257fa700 1 -- 172.21.15.16:0/1193488512 <== mgr.4098 v1:172.21.15.16:6816/118285 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+242444 (unknown 2965378022 0 222514928) 0x7fc3301175c0 con 0x7fc31402bb90 2023-09-30T22:26:55.740 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:26:55.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.739+0000 7fc337380700 1 -- 172.21.15.16:0/1193488512 >> v1:172.21.15.16:6816/118285 conn(0x7fc31402bb90 legacy=0x7fc31402e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:55.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.740+0000 7fc337380700 1 -- 172.21.15.16:0/1193488512 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc3300661b0 msgr2=0x7fc330113d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:55.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.740+0000 7fc337380700 1 --2- 172.21.15.16:0/1193488512 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc3300661b0 0x7fc330113d10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fc32000e950 tx=0x7fc320009060).stop 2023-09-30T22:26:55.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.740+0000 7fc337380700 1 -- 172.21.15.16:0/1193488512 shutdown_connections 2023-09-30T22:26:55.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.740+0000 7fc337380700 1 --2- 172.21.15.16:0/1193488512 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc330127570 0x7fc330114200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:55.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.740+0000 7fc337380700 1 --2- 172.21.15.16:0/1193488512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc330064570 0x7fc330124ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:55.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.740+0000 7fc337380700 1 --2- 172.21.15.16:0/1193488512 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc3300661b0 0x7fc330113d10 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:55.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.740+0000 7fc337380700 1 -- 172.21.15.16:0/1193488512 >> 172.21.15.16:0/1193488512 conn(0x7fc33009ba50 msgr2=0x7fc33009aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:55.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.740+0000 7fc337380700 1 -- 172.21.15.16:0/1193488512 shutdown_connections 2023-09-30T22:26:55.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.740+0000 7fc337380700 1 -- 172.21.15.16:0/1193488512 wait complete. 2023-09-30T22:26:55.743 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-09-30T22:26:55.753 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2023-09-30T22:26:54.455185+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":12,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1188796,"kb_used_data":412,"kb_used_omap":0,"kb_used_meta":1188352,"kb_avail":376298564,"statfs":{"total":386547056640,"available":385329729536,"internally_reserved":0,"allocated":421888,"data_stored":108906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1216872448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":11,"state":"active+clean","last_fresh":"2023-09-30T22:26:46.276231+0000","last_change":"2023-09-30T22:26:46.273891+0000","last_active":"2023-09-30T22:26:46.276231+0000","last_peered":"2023-09-30T22:26:46.276231+0000","last_clean":"2023-09-30T22:26:46.276231+0000","last_became_active":"2023-09-30T22:26:46.273588+0000","last_became_peered":"2023-09-30T22:26:46.273588+0000","last_unstale":"2023-09-30T22:26:46.276231+0000","last_undegraded":"2023-09-30T22:26:46.276231+0000","last_fullsized":"2023-09-30T22:26:46.276231+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":11,"state":"active+clean","last_fresh":"2023-09-30T22:26:46.248332+0000","last_change":"2023-09-30T22:26:46.245946+0000","last_active":"2023-09-30T22:26:46.248332+0000","last_peered":"2023-09-30T22:26:46.248332+0000","last_clean":"2023-09-30T22:26:46.248332+0000","last_became_active":"2023-09-30T22:26:46.245602+0000","last_became_peered":"2023-09-30T22:26:46.245602+0000","last_unstale":"2023-09-30T22:26:46.248332+0000","last_undegraded":"2023-09-30T22:26:46.248332+0000","last_fullsized":"2023-09-30T22:26:46.248332+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":11,"state":"active+clean","last_fresh":"2023-09-30T22:26:46.243326+0000","last_change":"2023-09-30T22:26:46.242607+0000","last_active":"2023-09-30T22:26:46.243326+0000","last_peered":"2023-09-30T22:26:46.243326+0000","last_clean":"2023-09-30T22:26:46.243326+0000","last_became_active":"2023-09-30T22:26:46.242283+0000","last_became_peered":"2023-09-30T22:26:46.242283+0000","last_unstale":"2023-09-30T22:26:46.243326+0000","last_undegraded":"2023-09-30T22:26:46.243326+0000","last_fullsized":"2023-09-30T22:26:46.243326+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":11,"state":"active+clean","last_fresh":"2023-09-30T22:26:46.243288+0000","last_change":"2023-09-30T22:26:46.242752+0000","last_active":"2023-09-30T22:26:46.243288+0000","last_peered":"2023-09-30T22:26:46.243288+0000","last_clean":"2023-09-30T22:26:46.243288+0000","last_became_active":"2023-09-30T22:26:46.242385+0000","last_became_peered":"2023-09-30T22:26:46.242385+0000","last_unstale":"2023-09-30T22:26:46.243288+0000","last_undegraded":"2023-09-30T22:26:46.243288+0000","last_fullsized":"2023-09-30T22:26:46.243288+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":9,"reported_epoch":11,"state":"active+clean","last_fresh":"2023-09-30T22:26:46.276288+0000","last_change":"2023-09-30T22:26:46.273930+0000","last_active":"2023-09-30T22:26:46.276288+0000","last_peered":"2023-09-30T22:26:46.276288+0000","last_clean":"2023-09-30T22:26:46.276288+0000","last_became_active":"2023-09-30T22:26:46.273621+0000","last_became_peered":"2023-09-30T22:26:46.273621+0000","last_unstale":"2023-09-30T22:26:46.276288+0000","last_undegraded":"2023-09-30T22:26:46.276288+0000","last_fullsized":"2023-09-30T22:26:46.276288+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":11,"state":"active+clean","last_fresh":"2023-09-30T22:26:46.248271+0000","last_change":"2023-09-30T22:26:46.245668+0000","last_active":"2023-09-30T22:26:46.248271+0000","last_peered":"2023-09-30T22:26:46.248271+0000","last_clean":"2023-09-30T22:26:46.248271+0000","last_became_active":"2023-09-30T22:26:46.245319+0000","last_became_peered":"2023-09-30T22:26:46.245319+0000","last_unstale":"2023-09-30T22:26:46.248271+0000","last_undegraded":"2023-09-30T22:26:46.248271+0000","last_fullsized":"2023-09-30T22:26:46.248271+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":11,"state":"active+clean","last_fresh":"2023-09-30T22:26:46.276351+0000","last_change":"2023-09-30T22:26:46.274062+0000","last_active":"2023-09-30T22:26:46.276351+0000","last_peered":"2023-09-30T22:26:46.276351+0000","last_clean":"2023-09-30T22:26:46.276351+0000","last_became_active":"2023-09-30T22:26:46.273752+0000","last_became_peered":"2023-09-30T22:26:46.273752+0000","last_unstale":"2023-09-30T22:26:46.276351+0000","last_undegraded":"2023-09-30T22:26:46.276351+0000","last_fullsized":"2023-09-30T22:26:46.276351+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":11,"state":"active+clean","last_fresh":"2023-09-30T22:26:46.276749+0000","last_change":"2023-09-30T22:26:46.273888+0000","last_active":"2023-09-30T22:26:46.276749+0000","last_peered":"2023-09-30T22:26:46.276749+0000","last_clean":"2023-09-30T22:26:46.276749+0000","last_became_active":"2023-09-30T22:26:46.273549+0000","last_became_peered":"2023-09-30T22:26:46.273549+0000","last_unstale":"2023-09-30T22:26:46.276749+0000","last_undegraded":"2023-09-30T22:26:46.276749+0000","last_fullsized":"2023-09-30T22:26:46.276749+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":11,"state":"active+clean","last_fresh":"2023-09-30T22:26:46.276604+0000","last_change":"2023-09-30T22:26:46.274508+0000","last_active":"2023-09-30T22:26:46.276604+0000","last_peered":"2023-09-30T22:26:46.276604+0000","last_clean":"2023-09-30T22:26:46.276604+0000","last_became_active":"2023-09-30T22:26:46.273567+0000","last_became_peered":"2023-09-30T22:26:46.273567+0000","last_unstale":"2023-09-30T22:26:46.276604+0000","last_undegraded":"2023-09-30T22:26:46.276604+0000","last_fullsized":"2023-09-30T22:26:46.276604+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705666,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297200,"kb_used_data":104,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074640,"statfs":{"total":96636764160,"available":96332431360,"internally_reserved":0,"allocated":106496,"data_stored":27519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]}]},{"osd":2,"up_from":9,"seq":38654705667,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297200,"kb_used_data":104,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074640,"statfs":{"total":96636764160,"available":96332431360,"internally_reserved":0,"allocated":106496,"data_stored":27519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.223}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32900000000000001}]}]},{"osd":0,"up_from":9,"seq":38654705666,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297196,"kb_used_data":100,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074644,"statfs":{"total":96636764160,"available":96332435456,"internally_reserved":0,"allocated":102400,"data_stored":26349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]}]},{"osd":1,"up_from":9,"seq":38654705666,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297200,"kb_used_data":104,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074640,"statfs":{"total":96636764160,"available":96332431360,"internally_reserved":0,"allocated":106496,"data_stored":27519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.313}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-30T22:26:55.760 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-30T22:26:55.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.924+0000 7f76e3502700 1 Processor -- start 2023-09-30T22:26:55.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.924+0000 7f76e3502700 1 -- start start 2023-09-30T22:26:55.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.925+0000 7f76e3502700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f76dc095a10 0x7f76dc095df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:55.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.925+0000 7f76e3502700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f76dc0962e0 0x7f76dc12c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:55.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.925+0000 7f76e3502700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f76dc12c730 0x7f76dc12eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:55.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.925+0000 7f76e3502700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76dc06acb0 con 0x7f76dc12c730 2023-09-30T22:26:55.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.925+0000 7f76e3502700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76dc06aac0 con 0x7f76dc095a10 2023-09-30T22:26:55.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.925+0000 7f76e3502700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f76dc093ba0 con 0x7f76dc0962e0 2023-09-30T22:26:55.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.925+0000 7f76e1a9f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f76dc12c730 0x7f76dc12eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:55.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.925+0000 7f76e129e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f76dc095a10 0x7f76dc095df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:55.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.925+0000 7f76e1a9f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f76dc12c730 0x7f76dc12eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38738/0 (socket says 172.21.15.16:38738) 2023-09-30T22:26:55.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.925+0000 7f76e1a9f700 1 -- 172.21.15.16:0/4219872256 learned_addr learned my addr 172.21.15.16:0/4219872256 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:26:55.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.925+0000 7f76e129e700 1 --2- 172.21.15.16:0/4219872256 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f76dc095a10 0x7f76dc095df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:44648/0 (socket says 172.21.15.16:44648) 2023-09-30T22:26:55.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.925+0000 7f76e0a9d700 1 --2- 172.21.15.16:0/4219872256 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f76dc0962e0 0x7f76dc12c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:55.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.925+0000 7f76e0a9d700 1 -- 172.21.15.16:0/4219872256 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f76dc095a10 msgr2=0x7f76dc095df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:55.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.925+0000 7f76e0a9d700 1 --2- 172.21.15.16:0/4219872256 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f76dc095a10 0x7f76dc095df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:55.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.925+0000 7f76e0a9d700 1 -- 172.21.15.16:0/4219872256 >> v2:172.21.15.16:3300/0 conn(0x7f76dc12c730 msgr2=0x7f76dc12eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:55.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.925+0000 7f76e0a9d700 1 --2- 172.21.15.16:0/4219872256 >> v2:172.21.15.16:3300/0 conn(0x7f76dc12c730 0x7f76dc12eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:55.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.925+0000 7f76e0a9d700 1 -- 172.21.15.16:0/4219872256 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76dc0615b0 con 0x7f76dc0962e0 2023-09-30T22:26:55.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.925+0000 7f76e1a9f700 1 --2- 172.21.15.16:0/4219872256 >> v2:172.21.15.16:3300/0 conn(0x7f76dc12c730 0x7f76dc12eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:55.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.925+0000 7f76e129e700 1 --2- 172.21.15.16:0/4219872256 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f76dc095a10 0x7f76dc095df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:55.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.926+0000 7f76e0a9d700 1 --2- 172.21.15.16:0/4219872256 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f76dc0962e0 0x7f76dc12c240 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f76cc0097c0 tx=0x7f76cc009060).ready entity=mon.2 client_cookie=3f8a7a303a8ee7ff server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:55.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.926+0000 7f76d3fff700 1 -- 172.21.15.16:0/4219872256 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76cc00d4a0 con 0x7f76dc0962e0 2023-09-30T22:26:55.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.926+0000 7f76d3fff700 1 -- 172.21.15.16:0/4219872256 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76cc00d600 con 0x7f76dc0962e0 2023-09-30T22:26:55.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.926+0000 7f76d3fff700 1 -- 172.21.15.16:0/4219872256 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76cc00d980 con 0x7f76dc0962e0 2023-09-30T22:26:55.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.926+0000 7f76e3502700 1 -- 172.21.15.16:0/4219872256 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f76dc0962e0 msgr2=0x7f76dc12c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:55.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.926+0000 7f76e3502700 1 --2- 172.21.15.16:0/4219872256 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f76dc0962e0 0x7f76dc12c240 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f76cc0097c0 tx=0x7f76cc009060).stop 2023-09-30T22:26:55.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.926+0000 7f76e3502700 1 -- 172.21.15.16:0/4219872256 shutdown_connections 2023-09-30T22:26:55.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.926+0000 7f76e3502700 1 --2- 172.21.15.16:0/4219872256 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f76dc095a10 0x7f76dc095df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:55.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.926+0000 7f76e3502700 1 --2- 172.21.15.16:0/4219872256 >> v2:172.21.15.16:3300/0 conn(0x7f76dc12c730 0x7f76dc12eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:55.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.926+0000 7f76e3502700 1 --2- 172.21.15.16:0/4219872256 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f76dc0962e0 0x7f76dc12c240 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:55.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.926+0000 7f76e3502700 1 -- 172.21.15.16:0/4219872256 >> 172.21.15.16:0/4219872256 conn(0x7f76dc09ab00 msgr2=0x7f76dc096b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:55.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.927+0000 7f76e3502700 1 -- 172.21.15.16:0/4219872256 shutdown_connections 2023-09-30T22:26:55.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.927+0000 7f76e3502700 1 -- 172.21.15.16:0/4219872256 wait complete. 2023-09-30T22:26:55.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.928+0000 7f76e3502700 1 Processor -- start 2023-09-30T22:26:55.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.928+0000 7f76e3502700 1 -- start start 2023-09-30T22:26:55.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.929+0000 7f76e3502700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76dc095a10 0x7f76dc127e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:55.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.929+0000 7f76e3502700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f76dc0962e0 0x7f76dc123e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:55.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.929+0000 7f76e3502700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f76dc12c730 0x7f76dc1242f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:55.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.929+0000 7f76e3502700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76dc12be70 con 0x7f76dc095a10 2023-09-30T22:26:55.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.929+0000 7f76e3502700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76dc069880 con 0x7f76dc12c730 2023-09-30T22:26:55.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.929+0000 7f76e3502700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f76dc0933c0 con 0x7f76dc0962e0 2023-09-30T22:26:55.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.929+0000 7f76e129e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76dc095a10 0x7f76dc127e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:55.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.929+0000 7f76e129e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76dc095a10 0x7f76dc127e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38754/0 (socket says 172.21.15.16:38754) 2023-09-30T22:26:55.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.929+0000 7f76e129e700 1 -- 172.21.15.16:0/2802687771 learned_addr learned my addr 172.21.15.16:0/2802687771 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:26:55.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.929+0000 7f76e0a9d700 1 --2- 172.21.15.16:0/2802687771 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f76dc0962e0 0x7f76dc123e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:55.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.929+0000 7f76e1a9f700 1 --2- 172.21.15.16:0/2802687771 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f76dc12c730 0x7f76dc1242f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:55.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.929+0000 7f76e0a9d700 1 -- 172.21.15.16:0/2802687771 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f76dc12c730 msgr2=0x7f76dc1242f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:55.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.929+0000 7f76e0a9d700 1 --2- 172.21.15.16:0/2802687771 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f76dc12c730 0x7f76dc1242f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:55.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.929+0000 7f76e0a9d700 1 -- 172.21.15.16:0/2802687771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76dc095a10 msgr2=0x7f76dc127e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:55.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.929+0000 7f76e0a9d700 1 --2- 172.21.15.16:0/2802687771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76dc095a10 0x7f76dc127e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:55.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.929+0000 7f76e0a9d700 1 -- 172.21.15.16:0/2802687771 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d80092a0 con 0x7f76dc0962e0 2023-09-30T22:26:55.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.929+0000 7f76e0a9d700 1 --2- 172.21.15.16:0/2802687771 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f76dc0962e0 0x7f76dc123e00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f76cc013000 tx=0x7f76cc016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:55.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.930+0000 7f76d17fa700 1 -- 172.21.15.16:0/2802687771 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76cc00dcf0 con 0x7f76dc0962e0 2023-09-30T22:26:55.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.930+0000 7f76d17fa700 1 -- 172.21.15.16:0/2802687771 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76cc00de70 con 0x7f76dc0962e0 2023-09-30T22:26:55.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.930+0000 7f76e3502700 1 -- 172.21.15.16:0/2802687771 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76d4009ae0 con 0x7f76dc0962e0 2023-09-30T22:26:55.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.930+0000 7f76d17fa700 1 -- 172.21.15.16:0/2802687771 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76cc01d430 con 0x7f76dc0962e0 2023-09-30T22:26:55.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.930+0000 7f76e3502700 1 -- 172.21.15.16:0/2802687771 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76cc0092a0 con 0x7f76dc0962e0 2023-09-30T22:26:55.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.930+0000 7f76d17fa700 1 -- 172.21.15.16:0/2802687771 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f76cc01d820 con 0x7f76dc0962e0 2023-09-30T22:26:55.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.931+0000 7f76d17fa700 1 -- 172.21.15.16:0/2802687771 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f76cc03bd80 con 0x7f76dc0962e0 2023-09-30T22:26:55.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.931+0000 7f76e3502700 1 -- 172.21.15.16:0/2802687771 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76dc095ea0 con 0x7f76dc0962e0 2023-09-30T22:26:55.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:55.936+0000 7f76d17fa700 1 -- 172.21.15.16:0/2802687771 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f76cc022030 con 0x7f76dc0962e0 2023-09-30T22:26:56.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.104+0000 7f76e3502700 1 -- 172.21.15.16:0/2802687771 --> v1:172.21.15.16:6816/118285 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f76dc1275b0 con 0x7f76c002bb60 2023-09-30T22:26:56.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.110+0000 7f76d17fa700 1 -- 172.21.15.16:0/2802687771 <== mgr.4098 v1:172.21.15.16:6816/118285 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+242444 (unknown 2965378022 0 222514928) 0x7f76d8009930 con 0x7f76c002bb60 2023-09-30T22:26:56.113 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:26:56.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.113+0000 7f76e3502700 1 -- 172.21.15.16:0/2802687771 >> v1:172.21.15.16:6816/118285 conn(0x7f76c002bb60 legacy=0x7f76c002e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:56.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.113+0000 7f76e3502700 1 -- 172.21.15.16:0/2802687771 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f76dc0962e0 msgr2=0x7f76dc123e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:56.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.113+0000 7f76e3502700 1 --2- 172.21.15.16:0/2802687771 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f76dc0962e0 0x7f76dc123e00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f76cc013000 tx=0x7f76cc016000).stop 2023-09-30T22:26:56.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.114+0000 7f76e3502700 1 -- 172.21.15.16:0/2802687771 shutdown_connections 2023-09-30T22:26:56.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.114+0000 7f76e3502700 1 --2- 172.21.15.16:0/2802687771 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f76dc12c730 0x7f76dc1242f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:56.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.114+0000 7f76e3502700 1 --2- 172.21.15.16:0/2802687771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76dc095a10 0x7f76dc127e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:56.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.114+0000 7f76e3502700 1 --2- 172.21.15.16:0/2802687771 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f76dc0962e0 0x7f76dc123e00 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:56.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.114+0000 7f76e3502700 1 -- 172.21.15.16:0/2802687771 >> 172.21.15.16:0/2802687771 conn(0x7f76dc09ab00 msgr2=0x7f76dc068fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:56.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.114+0000 7f76e3502700 1 -- 172.21.15.16:0/2802687771 shutdown_connections 2023-09-30T22:26:56.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.114+0000 7f76e3502700 1 -- 172.21.15.16:0/2802687771 wait complete. 2023-09-30T22:26:56.116 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-09-30T22:26:56.127 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2023-09-30T22:26:54.455185+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":12,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1188796,"kb_used_data":412,"kb_used_omap":0,"kb_used_meta":1188352,"kb_avail":376298564,"statfs":{"total":386547056640,"available":385329729536,"internally_reserved":0,"allocated":421888,"data_stored":108906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1216872448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":11,"state":"active+clean","last_fresh":"2023-09-30T22:26:46.276231+0000","last_change":"2023-09-30T22:26:46.273891+0000","last_active":"2023-09-30T22:26:46.276231+0000","last_peered":"2023-09-30T22:26:46.276231+0000","last_clean":"2023-09-30T22:26:46.276231+0000","last_became_active":"2023-09-30T22:26:46.273588+0000","last_became_peered":"2023-09-30T22:26:46.273588+0000","last_unstale":"2023-09-30T22:26:46.276231+0000","last_undegraded":"2023-09-30T22:26:46.276231+0000","last_fullsized":"2023-09-30T22:26:46.276231+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":11,"state":"active+clean","last_fresh":"2023-09-30T22:26:46.248332+0000","last_change":"2023-09-30T22:26:46.245946+0000","last_active":"2023-09-30T22:26:46.248332+0000","last_peered":"2023-09-30T22:26:46.248332+0000","last_clean":"2023-09-30T22:26:46.248332+0000","last_became_active":"2023-09-30T22:26:46.245602+0000","last_became_peered":"2023-09-30T22:26:46.245602+0000","last_unstale":"2023-09-30T22:26:46.248332+0000","last_undegraded":"2023-09-30T22:26:46.248332+0000","last_fullsized":"2023-09-30T22:26:46.248332+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":11,"state":"active+clean","last_fresh":"2023-09-30T22:26:46.243326+0000","last_change":"2023-09-30T22:26:46.242607+0000","last_active":"2023-09-30T22:26:46.243326+0000","last_peered":"2023-09-30T22:26:46.243326+0000","last_clean":"2023-09-30T22:26:46.243326+0000","last_became_active":"2023-09-30T22:26:46.242283+0000","last_became_peered":"2023-09-30T22:26:46.242283+0000","last_unstale":"2023-09-30T22:26:46.243326+0000","last_undegraded":"2023-09-30T22:26:46.243326+0000","last_fullsized":"2023-09-30T22:26:46.243326+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":11,"state":"active+clean","last_fresh":"2023-09-30T22:26:46.243288+0000","last_change":"2023-09-30T22:26:46.242752+0000","last_active":"2023-09-30T22:26:46.243288+0000","last_peered":"2023-09-30T22:26:46.243288+0000","last_clean":"2023-09-30T22:26:46.243288+0000","last_became_active":"2023-09-30T22:26:46.242385+0000","last_became_peered":"2023-09-30T22:26:46.242385+0000","last_unstale":"2023-09-30T22:26:46.243288+0000","last_undegraded":"2023-09-30T22:26:46.243288+0000","last_fullsized":"2023-09-30T22:26:46.243288+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":9,"reported_epoch":11,"state":"active+clean","last_fresh":"2023-09-30T22:26:46.276288+0000","last_change":"2023-09-30T22:26:46.273930+0000","last_active":"2023-09-30T22:26:46.276288+0000","last_peered":"2023-09-30T22:26:46.276288+0000","last_clean":"2023-09-30T22:26:46.276288+0000","last_became_active":"2023-09-30T22:26:46.273621+0000","last_became_peered":"2023-09-30T22:26:46.273621+0000","last_unstale":"2023-09-30T22:26:46.276288+0000","last_undegraded":"2023-09-30T22:26:46.276288+0000","last_fullsized":"2023-09-30T22:26:46.276288+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":11,"state":"active+clean","last_fresh":"2023-09-30T22:26:46.248271+0000","last_change":"2023-09-30T22:26:46.245668+0000","last_active":"2023-09-30T22:26:46.248271+0000","last_peered":"2023-09-30T22:26:46.248271+0000","last_clean":"2023-09-30T22:26:46.248271+0000","last_became_active":"2023-09-30T22:26:46.245319+0000","last_became_peered":"2023-09-30T22:26:46.245319+0000","last_unstale":"2023-09-30T22:26:46.248271+0000","last_undegraded":"2023-09-30T22:26:46.248271+0000","last_fullsized":"2023-09-30T22:26:46.248271+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":11,"state":"active+clean","last_fresh":"2023-09-30T22:26:46.276351+0000","last_change":"2023-09-30T22:26:46.274062+0000","last_active":"2023-09-30T22:26:46.276351+0000","last_peered":"2023-09-30T22:26:46.276351+0000","last_clean":"2023-09-30T22:26:46.276351+0000","last_became_active":"2023-09-30T22:26:46.273752+0000","last_became_peered":"2023-09-30T22:26:46.273752+0000","last_unstale":"2023-09-30T22:26:46.276351+0000","last_undegraded":"2023-09-30T22:26:46.276351+0000","last_fullsized":"2023-09-30T22:26:46.276351+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":11,"state":"active+clean","last_fresh":"2023-09-30T22:26:46.276749+0000","last_change":"2023-09-30T22:26:46.273888+0000","last_active":"2023-09-30T22:26:46.276749+0000","last_peered":"2023-09-30T22:26:46.276749+0000","last_clean":"2023-09-30T22:26:46.276749+0000","last_became_active":"2023-09-30T22:26:46.273549+0000","last_became_peered":"2023-09-30T22:26:46.273549+0000","last_unstale":"2023-09-30T22:26:46.276749+0000","last_undegraded":"2023-09-30T22:26:46.276749+0000","last_fullsized":"2023-09-30T22:26:46.276749+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":11,"state":"active+clean","last_fresh":"2023-09-30T22:26:46.276604+0000","last_change":"2023-09-30T22:26:46.274508+0000","last_active":"2023-09-30T22:26:46.276604+0000","last_peered":"2023-09-30T22:26:46.276604+0000","last_clean":"2023-09-30T22:26:46.276604+0000","last_became_active":"2023-09-30T22:26:46.273567+0000","last_became_peered":"2023-09-30T22:26:46.273567+0000","last_unstale":"2023-09-30T22:26:46.276604+0000","last_undegraded":"2023-09-30T22:26:46.276604+0000","last_fullsized":"2023-09-30T22:26:46.276604+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:50.251859+0000","last_change":"2023-09-30T22:26:50.251859+0000","last_active":"2023-09-30T22:26:50.251859+0000","last_peered":"2023-09-30T22:26:50.251859+0000","last_clean":"2023-09-30T22:26:50.251859+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:50.251859+0000","last_undegraded":"2023-09-30T22:26:50.251859+0000","last_fullsized":"2023-09-30T22:26:50.251859+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-09-30T22:26:51.255999+0000","last_change":"2023-09-30T22:26:51.255999+0000","last_active":"2023-09-30T22:26:51.255999+0000","last_peered":"2023-09-30T22:26:51.255999+0000","last_clean":"2023-09-30T22:26:51.255999+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-09-30T22:26:51.255999+0000","last_undegraded":"2023-09-30T22:26:51.255999+0000","last_fullsized":"2023-09-30T22:26:51.255999+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705666,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297200,"kb_used_data":104,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074640,"statfs":{"total":96636764160,"available":96332431360,"internally_reserved":0,"allocated":106496,"data_stored":27519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]}]},{"osd":2,"up_from":9,"seq":38654705667,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297200,"kb_used_data":104,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074640,"statfs":{"total":96636764160,"available":96332431360,"internally_reserved":0,"allocated":106496,"data_stored":27519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.223}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32900000000000001}]}]},{"osd":0,"up_from":9,"seq":38654705666,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297196,"kb_used_data":100,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074644,"statfs":{"total":96636764160,"available":96332435456,"internally_reserved":0,"allocated":102400,"data_stored":26349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]}]},{"osd":1,"up_from":9,"seq":38654705666,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297200,"kb_used_data":104,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074640,"statfs":{"total":96636764160,"available":96332431360,"internally_reserved":0,"allocated":106496,"data_stored":27519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.313}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-30T22:26:56.131 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-30T22:26:56.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.299+0000 7fb45d94a700 1 Processor -- start 2023-09-30T22:26:56.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.299+0000 7fb45d94a700 1 -- start start 2023-09-30T22:26:56.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.300+0000 7fb45d94a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb4580650f0 0x7fb4580986a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:56.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.300+0000 7fb45d94a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb4580992a0 0x7fb458132480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:56.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.300+0000 7fb45d94a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb458096160 con 0x7fb458097c30 2023-09-30T22:26:56.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.300+0000 7fb45d94a700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4580654d0 con 0x7fb4580992a0 2023-09-30T22:26:56.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.300+0000 7fb45d94a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb458065dd0 con 0x7fb4580650f0 2023-09-30T22:26:56.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.300+0000 7fb456ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb4580650f0 0x7fb4580986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:56.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.300+0000 7fb456ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb4580650f0 0x7fb4580986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:46330/0 (socket says 172.21.15.16:46330) 2023-09-30T22:26:56.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.300+0000 7fb4577fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb4580992a0 0x7fb458132480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:56.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.300+0000 7fb456ffd700 1 -- 172.21.15.16:0/2165073282 learned_addr learned my addr 172.21.15.16:0/2165073282 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:26:56.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.300+0000 7fb456ffd700 1 -- 172.21.15.16:0/2165073282 >> v2:172.21.15.16:3300/0 conn(0x7fb4580992a0 msgr2=0x7fb458132480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:56.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.300+0000 7fb456ffd700 1 --2- 172.21.15.16:0/2165073282 >> v2:172.21.15.16:3300/0 conn(0x7fb4580992a0 0x7fb458132480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:56.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.300+0000 7fb456ffd700 1 -- 172.21.15.16:0/2165073282 >> v1:172.21.15.16:6789/0 conn(0x7fb458097c30 legacy=0x7fb458098b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:56.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.300+0000 7fb456ffd700 1 -- 172.21.15.16:0/2165073282 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb458099790 con 0x7fb4580650f0 2023-09-30T22:26:56.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.300+0000 7fb4577fe700 1 --2- 172.21.15.16:0/2165073282 >> v2:172.21.15.16:3300/0 conn(0x7fb4580992a0 0x7fb458132480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:56.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.300+0000 7fb456ffd700 1 --2- 172.21.15.16:0/2165073282 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb4580650f0 0x7fb4580986a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fb448006fd0 tx=0x7fb4480090f0).ready entity=mon.2 client_cookie=aca8065de3d6af77 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:56.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.301+0000 7fb455ffb700 1 -- 172.21.15.16:0/2165073282 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb44800b580 con 0x7fb4580650f0 2023-09-30T22:26:56.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.301+0000 7fb455ffb700 1 -- 172.21.15.16:0/2165073282 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb44800b6e0 con 0x7fb4580650f0 2023-09-30T22:26:56.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.301+0000 7fb455ffb700 1 -- 172.21.15.16:0/2165073282 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb44800ba60 con 0x7fb4580650f0 2023-09-30T22:26:56.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.301+0000 7fb45d94a700 1 -- 172.21.15.16:0/2165073282 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb4580650f0 msgr2=0x7fb4580986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:56.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.301+0000 7fb45d94a700 1 --2- 172.21.15.16:0/2165073282 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb4580650f0 0x7fb4580986a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fb448006fd0 tx=0x7fb4480090f0).stop 2023-09-30T22:26:56.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.301+0000 7fb45d94a700 1 -- 172.21.15.16:0/2165073282 shutdown_connections 2023-09-30T22:26:56.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.301+0000 7fb45d94a700 1 --2- 172.21.15.16:0/2165073282 >> v2:172.21.15.16:3300/0 conn(0x7fb4580992a0 0x7fb458132480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:56.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.301+0000 7fb45d94a700 1 --2- 172.21.15.16:0/2165073282 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb4580650f0 0x7fb4580986a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:56.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.301+0000 7fb45d94a700 1 -- 172.21.15.16:0/2165073282 >> 172.21.15.16:0/2165073282 conn(0x7fb45809dc70 msgr2=0x7fb45809e050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:56.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.301+0000 7fb45d94a700 1 -- 172.21.15.16:0/2165073282 shutdown_connections 2023-09-30T22:26:56.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.301+0000 7fb45d94a700 1 -- 172.21.15.16:0/2165073282 wait complete. 2023-09-30T22:26:56.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.302+0000 7fb45d94a700 1 Processor -- start 2023-09-30T22:26:56.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.302+0000 7fb45d94a700 1 -- start start 2023-09-30T22:26:56.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.303+0000 7fb45d94a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb4580650f0 0x7fb458117440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:56.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.303+0000 7fb45d94a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4580992a0 0x7fb458117930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:56.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.303+0000 7fb45d94a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb458117e20 0x7fb45813aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:56.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.303+0000 7fb45d94a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb458119490 con 0x7fb4580992a0 2023-09-30T22:26:56.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.303+0000 7fb45d94a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb45811ab80 con 0x7fb458117e20 2023-09-30T22:26:56.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.303+0000 7fb45d94a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb45811ba40 con 0x7fb4580650f0 2023-09-30T22:26:56.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.303+0000 7fb4567fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4580992a0 0x7fb458117930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:56.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.303+0000 7fb4577fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb458117e20 0x7fb45813aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:56.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.303+0000 7fb4567fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4580992a0 0x7fb458117930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38768/0 (socket says 172.21.15.16:38768) 2023-09-30T22:26:56.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.303+0000 7fb4567fc700 1 -- 172.21.15.16:0/2717457617 learned_addr learned my addr 172.21.15.16:0/2717457617 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:26:56.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.303+0000 7fb456ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb4580650f0 0x7fb458117440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:56.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.303+0000 7fb456ffd700 1 -- 172.21.15.16:0/2717457617 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb458117e20 msgr2=0x7fb45813aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:56.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.303+0000 7fb456ffd700 1 --2- 172.21.15.16:0/2717457617 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb458117e20 0x7fb45813aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:56.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.303+0000 7fb456ffd700 1 -- 172.21.15.16:0/2717457617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4580992a0 msgr2=0x7fb458117930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:56.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.303+0000 7fb456ffd700 1 --2- 172.21.15.16:0/2717457617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4580992a0 0x7fb458117930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:56.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.303+0000 7fb456ffd700 1 -- 172.21.15.16:0/2717457617 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb450009a20 con 0x7fb4580650f0 2023-09-30T22:26:56.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.303+0000 7fb4567fc700 1 --2- 172.21.15.16:0/2717457617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4580992a0 0x7fb458117930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:56.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.304+0000 7fb4577fe700 1 --2- 172.21.15.16:0/2717457617 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb458117e20 0x7fb45813aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:56.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.304+0000 7fb456ffd700 1 --2- 172.21.15.16:0/2717457617 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb4580650f0 0x7fb458117440 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fb448014000 tx=0x7fb448017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:56.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.304+0000 7fb44effd700 1 -- 172.21.15.16:0/2717457617 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb448000bc0 con 0x7fb4580650f0 2023-09-30T22:26:56.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.304+0000 7fb44effd700 1 -- 172.21.15.16:0/2717457617 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb44800b850 con 0x7fb4580650f0 2023-09-30T22:26:56.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.304+0000 7fb45d94a700 1 -- 172.21.15.16:0/2717457617 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb448009370 con 0x7fb4580650f0 2023-09-30T22:26:56.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.304+0000 7fb44effd700 1 -- 172.21.15.16:0/2717457617 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb448000d20 con 0x7fb4580650f0 2023-09-30T22:26:56.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.304+0000 7fb45d94a700 1 -- 172.21.15.16:0/2717457617 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb458118820 con 0x7fb4580650f0 2023-09-30T22:26:56.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.305+0000 7fb44effd700 1 -- 172.21.15.16:0/2717457617 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fb44801cb50 con 0x7fb4580650f0 2023-09-30T22:26:56.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.305+0000 7fb44effd700 1 -- 172.21.15.16:0/2717457617 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fb44803d2a0 con 0x7fb4580650f0 2023-09-30T22:26:56.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.305+0000 7fb45d94a700 1 -- 172.21.15.16:0/2717457617 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb45811acc0 con 0x7fb4580650f0 2023-09-30T22:26:56.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.310+0000 7fb44effd700 1 -- 172.21.15.16:0/2717457617 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fb45811acc0 con 0x7fb4580650f0 2023-09-30T22:26:56.370 INFO:tasks.ceph.mds.a.smithi016.stderr:2023-09-30T22:26:56.369+0000 7fe3e7dad700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-09-30T22:26:56.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.480+0000 7fb45d94a700 1 -- 172.21.15.16:0/2717457617 --> v1:172.21.15.16:6816/118285 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb45811a800 con 0x7fb43802bb60 2023-09-30T22:26:56.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.487+0000 7fb44effd700 1 -- 172.21.15.16:0/2717457617 <== mgr.4098 v1:172.21.15.16:6816/118285 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+249995 (unknown 2965378022 0 1350139876) 0x7fb45811a800 con 0x7fb43802bb60 2023-09-30T22:26:56.490 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:26:56.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.490+0000 7fb45d94a700 1 -- 172.21.15.16:0/2717457617 >> v1:172.21.15.16:6816/118285 conn(0x7fb43802bb60 legacy=0x7fb43802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:56.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.490+0000 7fb45d94a700 1 -- 172.21.15.16:0/2717457617 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb4580650f0 msgr2=0x7fb458117440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:56.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.490+0000 7fb45d94a700 1 --2- 172.21.15.16:0/2717457617 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb4580650f0 0x7fb458117440 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fb448014000 tx=0x7fb448017000).stop 2023-09-30T22:26:56.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.490+0000 7fb45d94a700 1 -- 172.21.15.16:0/2717457617 shutdown_connections 2023-09-30T22:26:56.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.490+0000 7fb45d94a700 1 --2- 172.21.15.16:0/2717457617 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb458117e20 0x7fb45813aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:56.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.490+0000 7fb45d94a700 1 --2- 172.21.15.16:0/2717457617 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb4580650f0 0x7fb458117440 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:56.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.490+0000 7fb45d94a700 1 --2- 172.21.15.16:0/2717457617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4580992a0 0x7fb458117930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:56.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.490+0000 7fb45d94a700 1 -- 172.21.15.16:0/2717457617 >> 172.21.15.16:0/2717457617 conn(0x7fb45809dc70 msgr2=0x7fb45812f690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:56.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.490+0000 7fb45d94a700 1 -- 172.21.15.16:0/2717457617 shutdown_connections 2023-09-30T22:26:56.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.490+0000 7fb45d94a700 1 -- 172.21.15.16:0/2717457617 wait complete. 2023-09-30T22:26:56.493 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-09-30T22:26:56.503 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2023-09-30T22:26:56.455546+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":159,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1188988,"kb_used_data":604,"kb_used_omap":0,"kb_used_meta":1188352,"kb_avail":376298372,"statfs":{"total":386547056640,"available":385329532928,"internally_reserved":0,"allocated":618496,"data_stored":181498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1216872448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.718509+0000","last_change":"2023-09-30T22:26:51.708966+0000","last_active":"2023-09-30T22:26:51.718509+0000","last_peered":"2023-09-30T22:26:51.718509+0000","last_clean":"2023-09-30T22:26:51.718509+0000","last_became_active":"2023-09-30T22:26:51.708653+0000","last_became_peered":"2023-09-30T22:26:51.708653+0000","last_unstale":"2023-09-30T22:26:51.718509+0000","last_undegraded":"2023-09-30T22:26:51.718509+0000","last_fullsized":"2023-09-30T22:26:51.718509+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.729225+0000","last_change":"2023-09-30T22:26:51.727077+0000","last_active":"2023-09-30T22:26:51.729225+0000","last_peered":"2023-09-30T22:26:51.729225+0000","last_clean":"2023-09-30T22:26:51.729225+0000","last_became_active":"2023-09-30T22:26:51.726778+0000","last_became_peered":"2023-09-30T22:26:51.726778+0000","last_unstale":"2023-09-30T22:26:51.729225+0000","last_undegraded":"2023-09-30T22:26:51.729225+0000","last_fullsized":"2023-09-30T22:26:51.729225+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.714151+0000","last_change":"2023-09-30T22:26:51.706919+0000","last_active":"2023-09-30T22:26:51.714151+0000","last_peered":"2023-09-30T22:26:51.714151+0000","last_clean":"2023-09-30T22:26:51.714151+0000","last_became_active":"2023-09-30T22:26:51.706676+0000","last_became_peered":"2023-09-30T22:26:51.706676+0000","last_unstale":"2023-09-30T22:26:51.714151+0000","last_undegraded":"2023-09-30T22:26:51.714151+0000","last_fullsized":"2023-09-30T22:26:51.714151+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.728676+0000","last_change":"2023-09-30T22:26:51.727386+0000","last_active":"2023-09-30T22:26:51.728676+0000","last_peered":"2023-09-30T22:26:51.728676+0000","last_clean":"2023-09-30T22:26:51.728676+0000","last_became_active":"2023-09-30T22:26:51.727091+0000","last_became_peered":"2023-09-30T22:26:51.727091+0000","last_unstale":"2023-09-30T22:26:51.728676+0000","last_undegraded":"2023-09-30T22:26:51.728676+0000","last_fullsized":"2023-09-30T22:26:51.728676+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.723905+0000","last_change":"2023-09-30T22:26:51.719993+0000","last_active":"2023-09-30T22:26:51.723905+0000","last_peered":"2023-09-30T22:26:51.723905+0000","last_clean":"2023-09-30T22:26:51.723905+0000","last_became_active":"2023-09-30T22:26:51.719495+0000","last_became_peered":"2023-09-30T22:26:51.719495+0000","last_unstale":"2023-09-30T22:26:51.723905+0000","last_undegraded":"2023-09-30T22:26:51.723905+0000","last_fullsized":"2023-09-30T22:26:51.723905+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.715438+0000","last_change":"2023-09-30T22:26:51.709581+0000","last_active":"2023-09-30T22:26:51.715438+0000","last_peered":"2023-09-30T22:26:51.715438+0000","last_clean":"2023-09-30T22:26:51.715438+0000","last_became_active":"2023-09-30T22:26:51.709309+0000","last_became_peered":"2023-09-30T22:26:51.709309+0000","last_unstale":"2023-09-30T22:26:51.715438+0000","last_undegraded":"2023-09-30T22:26:51.715438+0000","last_fullsized":"2023-09-30T22:26:51.715438+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.713991+0000","last_change":"2023-09-30T22:26:51.705826+0000","last_active":"2023-09-30T22:26:51.713991+0000","last_peered":"2023-09-30T22:26:51.713991+0000","last_clean":"2023-09-30T22:26:51.713991+0000","last_became_active":"2023-09-30T22:26:51.705037+0000","last_became_peered":"2023-09-30T22:26:51.705037+0000","last_unstale":"2023-09-30T22:26:51.713991+0000","last_undegraded":"2023-09-30T22:26:51.713991+0000","last_fullsized":"2023-09-30T22:26:51.713991+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730836+0000","last_change":"2023-09-30T22:26:51.726640+0000","last_active":"2023-09-30T22:26:51.730836+0000","last_peered":"2023-09-30T22:26:51.730836+0000","last_clean":"2023-09-30T22:26:51.730836+0000","last_became_active":"2023-09-30T22:26:51.726278+0000","last_became_peered":"2023-09-30T22:26:51.726278+0000","last_unstale":"2023-09-30T22:26:51.730836+0000","last_undegraded":"2023-09-30T22:26:51.730836+0000","last_fullsized":"2023-09-30T22:26:51.730836+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.724179+0000","last_change":"2023-09-30T22:26:51.721817+0000","last_active":"2023-09-30T22:26:51.724179+0000","last_peered":"2023-09-30T22:26:51.724179+0000","last_clean":"2023-09-30T22:26:51.724179+0000","last_became_active":"2023-09-30T22:26:51.721373+0000","last_became_peered":"2023-09-30T22:26:51.721373+0000","last_unstale":"2023-09-30T22:26:51.724179+0000","last_undegraded":"2023-09-30T22:26:51.724179+0000","last_fullsized":"2023-09-30T22:26:51.724179+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730810+0000","last_change":"2023-09-30T22:26:51.726348+0000","last_active":"2023-09-30T22:26:51.730810+0000","last_peered":"2023-09-30T22:26:51.730810+0000","last_clean":"2023-09-30T22:26:51.730810+0000","last_became_active":"2023-09-30T22:26:51.726030+0000","last_became_peered":"2023-09-30T22:26:51.726030+0000","last_unstale":"2023-09-30T22:26:51.730810+0000","last_undegraded":"2023-09-30T22:26:51.730810+0000","last_fullsized":"2023-09-30T22:26:51.730810+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730607+0000","last_change":"2023-09-30T22:26:51.722619+0000","last_active":"2023-09-30T22:26:51.730607+0000","last_peered":"2023-09-30T22:26:51.730607+0000","last_clean":"2023-09-30T22:26:51.730607+0000","last_became_active":"2023-09-30T22:26:51.722317+0000","last_became_peered":"2023-09-30T22:26:51.722317+0000","last_unstale":"2023-09-30T22:26:51.730607+0000","last_undegraded":"2023-09-30T22:26:51.730607+0000","last_fullsized":"2023-09-30T22:26:51.730607+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.718852+0000","last_change":"2023-09-30T22:26:51.709697+0000","last_active":"2023-09-30T22:26:51.718852+0000","last_peered":"2023-09-30T22:26:51.718852+0000","last_clean":"2023-09-30T22:26:51.718852+0000","last_became_active":"2023-09-30T22:26:51.709342+0000","last_became_peered":"2023-09-30T22:26:51.709342+0000","last_unstale":"2023-09-30T22:26:51.718852+0000","last_undegraded":"2023-09-30T22:26:51.718852+0000","last_fullsized":"2023-09-30T22:26:51.718852+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.719514+0000","last_change":"2023-09-30T22:26:51.716276+0000","last_active":"2023-09-30T22:26:51.719514+0000","last_peered":"2023-09-30T22:26:51.719514+0000","last_clean":"2023-09-30T22:26:51.719514+0000","last_became_active":"2023-09-30T22:26:51.715634+0000","last_became_peered":"2023-09-30T22:26:51.715634+0000","last_unstale":"2023-09-30T22:26:51.719514+0000","last_undegraded":"2023-09-30T22:26:51.719514+0000","last_fullsized":"2023-09-30T22:26:51.719514+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.707967+0000","last_change":"2023-09-30T22:26:51.695948+0000","last_active":"2023-09-30T22:26:51.707967+0000","last_peered":"2023-09-30T22:26:51.707967+0000","last_clean":"2023-09-30T22:26:51.707967+0000","last_became_active":"2023-09-30T22:26:51.695661+0000","last_became_peered":"2023-09-30T22:26:51.695661+0000","last_unstale":"2023-09-30T22:26:51.707967+0000","last_undegraded":"2023-09-30T22:26:51.707967+0000","last_fullsized":"2023-09-30T22:26:51.707967+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.716656+0000","last_change":"2023-09-30T22:26:51.708532+0000","last_active":"2023-09-30T22:26:51.716656+0000","last_peered":"2023-09-30T22:26:51.716656+0000","last_clean":"2023-09-30T22:26:51.716656+0000","last_became_active":"2023-09-30T22:26:51.708204+0000","last_became_peered":"2023-09-30T22:26:51.708204+0000","last_unstale":"2023-09-30T22:26:51.716656+0000","last_undegraded":"2023-09-30T22:26:51.716656+0000","last_fullsized":"2023-09-30T22:26:51.716656+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.714211+0000","last_change":"2023-09-30T22:26:51.704597+0000","last_active":"2023-09-30T22:26:51.714211+0000","last_peered":"2023-09-30T22:26:51.714211+0000","last_clean":"2023-09-30T22:26:51.714211+0000","last_became_active":"2023-09-30T22:26:51.704307+0000","last_became_peered":"2023-09-30T22:26:51.704307+0000","last_unstale":"2023-09-30T22:26:51.714211+0000","last_undegraded":"2023-09-30T22:26:51.714211+0000","last_fullsized":"2023-09-30T22:26:51.714211+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.724159+0000","last_change":"2023-09-30T22:26:51.721640+0000","last_active":"2023-09-30T22:26:51.724159+0000","last_peered":"2023-09-30T22:26:51.724159+0000","last_clean":"2023-09-30T22:26:51.724159+0000","last_became_active":"2023-09-30T22:26:51.720950+0000","last_became_peered":"2023-09-30T22:26:51.720950+0000","last_unstale":"2023-09-30T22:26:51.724159+0000","last_undegraded":"2023-09-30T22:26:51.724159+0000","last_fullsized":"2023-09-30T22:26:51.724159+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.707139+0000","last_change":"2023-09-30T22:26:51.697513+0000","last_active":"2023-09-30T22:26:51.707139+0000","last_peered":"2023-09-30T22:26:51.707139+0000","last_clean":"2023-09-30T22:26:51.707139+0000","last_became_active":"2023-09-30T22:26:51.697137+0000","last_became_peered":"2023-09-30T22:26:51.697137+0000","last_unstale":"2023-09-30T22:26:51.707139+0000","last_undegraded":"2023-09-30T22:26:51.707139+0000","last_fullsized":"2023-09-30T22:26:51.707139+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.718540+0000","last_change":"2023-09-30T22:26:51.715476+0000","last_active":"2023-09-30T22:26:51.718540+0000","last_peered":"2023-09-30T22:26:51.718540+0000","last_clean":"2023-09-30T22:26:51.718540+0000","last_became_active":"2023-09-30T22:26:51.715176+0000","last_became_peered":"2023-09-30T22:26:51.715176+0000","last_unstale":"2023-09-30T22:26:51.718540+0000","last_undegraded":"2023-09-30T22:26:51.718540+0000","last_fullsized":"2023-09-30T22:26:51.718540+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730693+0000","last_change":"2023-09-30T22:26:51.724991+0000","last_active":"2023-09-30T22:26:51.730693+0000","last_peered":"2023-09-30T22:26:51.730693+0000","last_clean":"2023-09-30T22:26:51.730693+0000","last_became_active":"2023-09-30T22:26:51.724240+0000","last_became_peered":"2023-09-30T22:26:51.724240+0000","last_unstale":"2023-09-30T22:26:51.730693+0000","last_undegraded":"2023-09-30T22:26:51.730693+0000","last_fullsized":"2023-09-30T22:26:51.730693+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730391+0000","last_change":"2023-09-30T22:26:51.717833+0000","last_active":"2023-09-30T22:26:51.730391+0000","last_peered":"2023-09-30T22:26:51.730391+0000","last_clean":"2023-09-30T22:26:51.730391+0000","last_became_active":"2023-09-30T22:26:51.717517+0000","last_became_peered":"2023-09-30T22:26:51.717517+0000","last_unstale":"2023-09-30T22:26:51.730391+0000","last_undegraded":"2023-09-30T22:26:51.730391+0000","last_fullsized":"2023-09-30T22:26:51.730391+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.723937+0000","last_change":"2023-09-30T22:26:51.719030+0000","last_active":"2023-09-30T22:26:51.723937+0000","last_peered":"2023-09-30T22:26:51.723937+0000","last_clean":"2023-09-30T22:26:51.723937+0000","last_became_active":"2023-09-30T22:26:51.715370+0000","last_became_peered":"2023-09-30T22:26:51.715370+0000","last_unstale":"2023-09-30T22:26:51.723937+0000","last_undegraded":"2023-09-30T22:26:51.723937+0000","last_fullsized":"2023-09-30T22:26:51.723937+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.713970+0000","last_change":"2023-09-30T22:26:51.698664+0000","last_active":"2023-09-30T22:26:51.713970+0000","last_peered":"2023-09-30T22:26:51.713970+0000","last_clean":"2023-09-30T22:26:51.713970+0000","last_became_active":"2023-09-30T22:26:51.698321+0000","last_became_peered":"2023-09-30T22:26:51.698321+0000","last_unstale":"2023-09-30T22:26:51.713970+0000","last_undegraded":"2023-09-30T22:26:51.713970+0000","last_fullsized":"2023-09-30T22:26:51.713970+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730972+0000","last_change":"2023-09-30T22:26:51.727955+0000","last_active":"2023-09-30T22:26:51.730972+0000","last_peered":"2023-09-30T22:26:51.730972+0000","last_clean":"2023-09-30T22:26:51.730972+0000","last_became_active":"2023-09-30T22:26:51.727209+0000","last_became_peered":"2023-09-30T22:26:51.727209+0000","last_unstale":"2023-09-30T22:26:51.730972+0000","last_undegraded":"2023-09-30T22:26:51.730972+0000","last_fullsized":"2023-09-30T22:26:51.730972+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730499+0000","last_change":"2023-09-30T22:26:51.721154+0000","last_active":"2023-09-30T22:26:51.730499+0000","last_peered":"2023-09-30T22:26:51.730499+0000","last_clean":"2023-09-30T22:26:51.730499+0000","last_became_active":"2023-09-30T22:26:51.720703+0000","last_became_peered":"2023-09-30T22:26:51.720703+0000","last_unstale":"2023-09-30T22:26:51.730499+0000","last_undegraded":"2023-09-30T22:26:51.730499+0000","last_fullsized":"2023-09-30T22:26:51.730499+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.725830+0000","last_change":"2023-09-30T22:26:51.701309+0000","last_active":"2023-09-30T22:26:51.725830+0000","last_peered":"2023-09-30T22:26:51.725830+0000","last_clean":"2023-09-30T22:26:51.725830+0000","last_became_active":"2023-09-30T22:26:51.701012+0000","last_became_peered":"2023-09-30T22:26:51.701012+0000","last_unstale":"2023-09-30T22:26:51.725830+0000","last_undegraded":"2023-09-30T22:26:51.725830+0000","last_fullsized":"2023-09-30T22:26:51.725830+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.718423+0000","last_change":"2023-09-30T22:26:51.715042+0000","last_active":"2023-09-30T22:26:51.718423+0000","last_peered":"2023-09-30T22:26:51.718423+0000","last_clean":"2023-09-30T22:26:51.718423+0000","last_became_active":"2023-09-30T22:26:51.712158+0000","last_became_peered":"2023-09-30T22:26:51.712158+0000","last_unstale":"2023-09-30T22:26:51.718423+0000","last_undegraded":"2023-09-30T22:26:51.718423+0000","last_fullsized":"2023-09-30T22:26:51.718423+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"15'1","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.713320+0000","last_change":"2023-09-30T22:26:51.296397+0000","last_active":"2023-09-30T22:26:51.713320+0000","last_peered":"2023-09-30T22:26:51.713320+0000","last_clean":"2023-09-30T22:26:51.713320+0000","last_became_active":"2023-09-30T22:26:51.295860+0000","last_became_peered":"2023-09-30T22:26:51.295860+0000","last_unstale":"2023-09-30T22:26:51.713320+0000","last_undegraded":"2023-09-30T22:26:51.713320+0000","last_fullsized":"2023-09-30T22:26:51.713320+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.716849+0000","last_change":"2023-09-30T22:26:51.712472+0000","last_active":"2023-09-30T22:26:51.716849+0000","last_peered":"2023-09-30T22:26:51.716849+0000","last_clean":"2023-09-30T22:26:51.716849+0000","last_became_active":"2023-09-30T22:26:51.711527+0000","last_became_peered":"2023-09-30T22:26:51.711527+0000","last_unstale":"2023-09-30T22:26:51.716849+0000","last_undegraded":"2023-09-30T22:26:51.716849+0000","last_fullsized":"2023-09-30T22:26:51.716849+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.701600+0000","last_change":"2023-09-30T22:26:51.298004+0000","last_active":"2023-09-30T22:26:51.701600+0000","last_peered":"2023-09-30T22:26:51.701600+0000","last_clean":"2023-09-30T22:26:51.701600+0000","last_became_active":"2023-09-30T22:26:51.297714+0000","last_became_peered":"2023-09-30T22:26:51.297714+0000","last_unstale":"2023-09-30T22:26:51.701600+0000","last_undegraded":"2023-09-30T22:26:51.701600+0000","last_fullsized":"2023-09-30T22:26:51.701600+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.729456+0000","last_change":"2023-09-30T22:26:51.727608+0000","last_active":"2023-09-30T22:26:51.729456+0000","last_peered":"2023-09-30T22:26:51.729456+0000","last_clean":"2023-09-30T22:26:51.729456+0000","last_became_active":"2023-09-30T22:26:51.727154+0000","last_became_peered":"2023-09-30T22:26:51.727154+0000","last_unstale":"2023-09-30T22:26:51.729456+0000","last_undegraded":"2023-09-30T22:26:51.729456+0000","last_fullsized":"2023-09-30T22:26:51.729456+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.684923+0000","last_change":"2023-09-30T22:26:51.285214+0000","last_active":"2023-09-30T22:26:51.684923+0000","last_peered":"2023-09-30T22:26:51.684923+0000","last_clean":"2023-09-30T22:26:51.684923+0000","last_became_active":"2023-09-30T22:26:51.284749+0000","last_became_peered":"2023-09-30T22:26:51.284749+0000","last_unstale":"2023-09-30T22:26:51.684923+0000","last_undegraded":"2023-09-30T22:26:51.684923+0000","last_fullsized":"2023-09-30T22:26:51.684923+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.716115+0000","last_change":"2023-09-30T22:26:51.706697+0000","last_active":"2023-09-30T22:26:51.716115+0000","last_peered":"2023-09-30T22:26:51.716115+0000","last_clean":"2023-09-30T22:26:51.716115+0000","last_became_active":"2023-09-30T22:26:51.706373+0000","last_became_peered":"2023-09-30T22:26:51.706373+0000","last_unstale":"2023-09-30T22:26:51.716115+0000","last_undegraded":"2023-09-30T22:26:51.716115+0000","last_fullsized":"2023-09-30T22:26:51.716115+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.689119+0000","last_change":"2023-09-30T22:26:51.281642+0000","last_active":"2023-09-30T22:26:51.689119+0000","last_peered":"2023-09-30T22:26:51.689119+0000","last_clean":"2023-09-30T22:26:51.689119+0000","last_became_active":"2023-09-30T22:26:51.281319+0000","last_became_peered":"2023-09-30T22:26:51.281319+0000","last_unstale":"2023-09-30T22:26:51.689119+0000","last_undegraded":"2023-09-30T22:26:51.689119+0000","last_fullsized":"2023-09-30T22:26:51.689119+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.714232+0000","last_change":"2023-09-30T22:26:51.705192+0000","last_active":"2023-09-30T22:26:51.714232+0000","last_peered":"2023-09-30T22:26:51.714232+0000","last_clean":"2023-09-30T22:26:51.714232+0000","last_became_active":"2023-09-30T22:26:51.704870+0000","last_became_peered":"2023-09-30T22:26:51.704870+0000","last_unstale":"2023-09-30T22:26:51.714232+0000","last_undegraded":"2023-09-30T22:26:51.714232+0000","last_fullsized":"2023-09-30T22:26:51.714232+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.680797+0000","last_change":"2023-09-30T22:26:51.293711+0000","last_active":"2023-09-30T22:26:51.680797+0000","last_peered":"2023-09-30T22:26:51.680797+0000","last_clean":"2023-09-30T22:26:51.680797+0000","last_became_active":"2023-09-30T22:26:51.293410+0000","last_became_peered":"2023-09-30T22:26:51.293410+0000","last_unstale":"2023-09-30T22:26:51.680797+0000","last_undegraded":"2023-09-30T22:26:51.680797+0000","last_fullsized":"2023-09-30T22:26:51.680797+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.714029+0000","last_change":"2023-09-30T22:26:51.706005+0000","last_active":"2023-09-30T22:26:51.714029+0000","last_peered":"2023-09-30T22:26:51.714029+0000","last_clean":"2023-09-30T22:26:51.714029+0000","last_became_active":"2023-09-30T22:26:51.705491+0000","last_became_peered":"2023-09-30T22:26:51.705491+0000","last_unstale":"2023-09-30T22:26:51.714029+0000","last_undegraded":"2023-09-30T22:26:51.714029+0000","last_fullsized":"2023-09-30T22:26:51.714029+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"15'3","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.719469+0000","last_change":"2023-09-30T22:26:51.279976+0000","last_active":"2023-09-30T22:26:51.719469+0000","last_peered":"2023-09-30T22:26:51.719469+0000","last_clean":"2023-09-30T22:26:51.719469+0000","last_became_active":"2023-09-30T22:26:51.279669+0000","last_became_peered":"2023-09-30T22:26:51.279669+0000","last_unstale":"2023-09-30T22:26:51.719469+0000","last_undegraded":"2023-09-30T22:26:51.719469+0000","last_fullsized":"2023-09-30T22:26:51.719469+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730714+0000","last_change":"2023-09-30T22:26:51.725947+0000","last_active":"2023-09-30T22:26:51.730714+0000","last_peered":"2023-09-30T22:26:51.730714+0000","last_clean":"2023-09-30T22:26:51.730714+0000","last_became_active":"2023-09-30T22:26:51.723294+0000","last_became_peered":"2023-09-30T22:26:51.723294+0000","last_unstale":"2023-09-30T22:26:51.730714+0000","last_undegraded":"2023-09-30T22:26:51.730714+0000","last_fullsized":"2023-09-30T22:26:51.730714+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"15'6","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.723849+0000","last_change":"2023-09-30T22:26:51.297527+0000","last_active":"2023-09-30T22:26:51.723849+0000","last_peered":"2023-09-30T22:26:51.723849+0000","last_clean":"2023-09-30T22:26:51.723849+0000","last_became_active":"2023-09-30T22:26:51.297246+0000","last_became_peered":"2023-09-30T22:26:51.297246+0000","last_unstale":"2023-09-30T22:26:51.723849+0000","last_undegraded":"2023-09-30T22:26:51.723849+0000","last_fullsized":"2023-09-30T22:26:51.723849+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.718810+0000","last_change":"2023-09-30T22:26:51.710781+0000","last_active":"2023-09-30T22:26:51.718810+0000","last_peered":"2023-09-30T22:26:51.718810+0000","last_clean":"2023-09-30T22:26:51.718810+0000","last_became_active":"2023-09-30T22:26:51.710107+0000","last_became_peered":"2023-09-30T22:26:51.710107+0000","last_unstale":"2023-09-30T22:26:51.718810+0000","last_undegraded":"2023-09-30T22:26:51.718810+0000","last_fullsized":"2023-09-30T22:26:51.718810+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"15'2","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.714029+0000","last_change":"2023-09-30T22:26:51.290194+0000","last_active":"2023-09-30T22:26:51.714029+0000","last_peered":"2023-09-30T22:26:51.714029+0000","last_clean":"2023-09-30T22:26:51.714029+0000","last_became_active":"2023-09-30T22:26:51.288688+0000","last_became_peered":"2023-09-30T22:26:51.288688+0000","last_unstale":"2023-09-30T22:26:51.714029+0000","last_undegraded":"2023-09-30T22:26:51.714029+0000","last_fullsized":"2023-09-30T22:26:51.714029+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.728819+0000","last_change":"2023-09-30T22:26:51.726780+0000","last_active":"2023-09-30T22:26:51.728819+0000","last_peered":"2023-09-30T22:26:51.728819+0000","last_clean":"2023-09-30T22:26:51.728819+0000","last_became_active":"2023-09-30T22:26:51.726408+0000","last_became_peered":"2023-09-30T22:26:51.726408+0000","last_unstale":"2023-09-30T22:26:51.728819+0000","last_undegraded":"2023-09-30T22:26:51.728819+0000","last_fullsized":"2023-09-30T22:26:51.728819+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.685836+0000","last_change":"2023-09-30T22:26:51.281127+0000","last_active":"2023-09-30T22:26:51.685836+0000","last_peered":"2023-09-30T22:26:51.685836+0000","last_clean":"2023-09-30T22:26:51.685836+0000","last_became_active":"2023-09-30T22:26:51.280507+0000","last_became_peered":"2023-09-30T22:26:51.280507+0000","last_unstale":"2023-09-30T22:26:51.685836+0000","last_undegraded":"2023-09-30T22:26:51.685836+0000","last_fullsized":"2023-09-30T22:26:51.685836+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.716876+0000","last_change":"2023-09-30T22:26:51.712671+0000","last_active":"2023-09-30T22:26:51.716876+0000","last_peered":"2023-09-30T22:26:51.716876+0000","last_clean":"2023-09-30T22:26:51.716876+0000","last_became_active":"2023-09-30T22:26:51.711836+0000","last_became_peered":"2023-09-30T22:26:51.711836+0000","last_unstale":"2023-09-30T22:26:51.716876+0000","last_undegraded":"2023-09-30T22:26:51.716876+0000","last_fullsized":"2023-09-30T22:26:51.716876+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.683587+0000","last_change":"2023-09-30T22:26:51.297467+0000","last_active":"2023-09-30T22:26:51.683587+0000","last_peered":"2023-09-30T22:26:51.683587+0000","last_clean":"2023-09-30T22:26:51.683587+0000","last_became_active":"2023-09-30T22:26:51.296950+0000","last_became_peered":"2023-09-30T22:26:51.296950+0000","last_unstale":"2023-09-30T22:26:51.683587+0000","last_undegraded":"2023-09-30T22:26:51.683587+0000","last_fullsized":"2023-09-30T22:26:51.683587+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.724064+0000","last_change":"2023-09-30T22:26:51.719411+0000","last_active":"2023-09-30T22:26:51.724064+0000","last_peered":"2023-09-30T22:26:51.724064+0000","last_clean":"2023-09-30T22:26:51.724064+0000","last_became_active":"2023-09-30T22:26:51.719111+0000","last_became_peered":"2023-09-30T22:26:51.719111+0000","last_unstale":"2023-09-30T22:26:51.724064+0000","last_undegraded":"2023-09-30T22:26:51.724064+0000","last_fullsized":"2023-09-30T22:26:51.724064+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"15'8","reported_seq":18,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.727696+0000","last_change":"2023-09-30T22:26:51.287702+0000","last_active":"2023-09-30T22:26:51.727696+0000","last_peered":"2023-09-30T22:26:51.727696+0000","last_clean":"2023-09-30T22:26:51.727696+0000","last_became_active":"2023-09-30T22:26:51.287434+0000","last_became_peered":"2023-09-30T22:26:51.287434+0000","last_unstale":"2023-09-30T22:26:51.727696+0000","last_undegraded":"2023-09-30T22:26:51.727696+0000","last_fullsized":"2023-09-30T22:26:51.727696+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.716138+0000","last_change":"2023-09-30T22:26:51.706663+0000","last_active":"2023-09-30T22:26:51.716138+0000","last_peered":"2023-09-30T22:26:51.716138+0000","last_clean":"2023-09-30T22:26:51.716138+0000","last_became_active":"2023-09-30T22:26:51.706363+0000","last_became_peered":"2023-09-30T22:26:51.706363+0000","last_unstale":"2023-09-30T22:26:51.716138+0000","last_undegraded":"2023-09-30T22:26:51.716138+0000","last_fullsized":"2023-09-30T22:26:51.716138+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"15'1","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.721973+0000","last_change":"2023-09-30T22:26:51.291996+0000","last_active":"2023-09-30T22:26:51.721973+0000","last_peered":"2023-09-30T22:26:51.721973+0000","last_clean":"2023-09-30T22:26:51.721973+0000","last_became_active":"2023-09-30T22:26:51.291656+0000","last_became_peered":"2023-09-30T22:26:51.291656+0000","last_unstale":"2023-09-30T22:26:51.721973+0000","last_undegraded":"2023-09-30T22:26:51.721973+0000","last_fullsized":"2023-09-30T22:26:51.721973+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.715557+0000","last_change":"2023-09-30T22:26:51.710326+0000","last_active":"2023-09-30T22:26:51.715557+0000","last_peered":"2023-09-30T22:26:51.715557+0000","last_clean":"2023-09-30T22:26:51.715557+0000","last_became_active":"2023-09-30T22:26:51.708793+0000","last_became_peered":"2023-09-30T22:26:51.708793+0000","last_unstale":"2023-09-30T22:26:51.715557+0000","last_undegraded":"2023-09-30T22:26:51.715557+0000","last_fullsized":"2023-09-30T22:26:51.715557+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"15'2","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.699110+0000","last_change":"2023-09-30T22:26:51.300700+0000","last_active":"2023-09-30T22:26:51.699110+0000","last_peered":"2023-09-30T22:26:51.699110+0000","last_clean":"2023-09-30T22:26:51.699110+0000","last_became_active":"2023-09-30T22:26:51.300416+0000","last_became_peered":"2023-09-30T22:26:51.300416+0000","last_unstale":"2023-09-30T22:26:51.699110+0000","last_undegraded":"2023-09-30T22:26:51.699110+0000","last_fullsized":"2023-09-30T22:26:51.699110+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730623+0000","last_change":"2023-09-30T22:26:51.722902+0000","last_active":"2023-09-30T22:26:51.730623+0000","last_peered":"2023-09-30T22:26:51.730623+0000","last_clean":"2023-09-30T22:26:51.730623+0000","last_became_active":"2023-09-30T22:26:51.717919+0000","last_became_peered":"2023-09-30T22:26:51.717919+0000","last_unstale":"2023-09-30T22:26:51.730623+0000","last_undegraded":"2023-09-30T22:26:51.730623+0000","last_fullsized":"2023-09-30T22:26:51.730623+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"15'1","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.716626+0000","last_change":"2023-09-30T22:26:51.292120+0000","last_active":"2023-09-30T22:26:51.716626+0000","last_peered":"2023-09-30T22:26:51.716626+0000","last_clean":"2023-09-30T22:26:51.716626+0000","last_became_active":"2023-09-30T22:26:51.289469+0000","last_became_peered":"2023-09-30T22:26:51.289469+0000","last_unstale":"2023-09-30T22:26:51.716626+0000","last_undegraded":"2023-09-30T22:26:51.716626+0000","last_fullsized":"2023-09-30T22:26:51.716626+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.718700+0000","last_change":"2023-09-30T22:26:51.711649+0000","last_active":"2023-09-30T22:26:51.718700+0000","last_peered":"2023-09-30T22:26:51.718700+0000","last_clean":"2023-09-30T22:26:51.718700+0000","last_became_active":"2023-09-30T22:26:51.711384+0000","last_became_peered":"2023-09-30T22:26:51.711384+0000","last_unstale":"2023-09-30T22:26:51.718700+0000","last_undegraded":"2023-09-30T22:26:51.718700+0000","last_fullsized":"2023-09-30T22:26:51.718700+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.686889+0000","last_change":"2023-09-30T22:26:51.281627+0000","last_active":"2023-09-30T22:26:51.686889+0000","last_peered":"2023-09-30T22:26:51.686889+0000","last_clean":"2023-09-30T22:26:51.686889+0000","last_became_active":"2023-09-30T22:26:51.281311+0000","last_became_peered":"2023-09-30T22:26:51.281311+0000","last_unstale":"2023-09-30T22:26:51.686889+0000","last_undegraded":"2023-09-30T22:26:51.686889+0000","last_fullsized":"2023-09-30T22:26:51.686889+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.707659+0000","last_change":"2023-09-30T22:26:51.700837+0000","last_active":"2023-09-30T22:26:51.707659+0000","last_peered":"2023-09-30T22:26:51.707659+0000","last_clean":"2023-09-30T22:26:51.707659+0000","last_became_active":"2023-09-30T22:26:51.700523+0000","last_became_peered":"2023-09-30T22:26:51.700523+0000","last_unstale":"2023-09-30T22:26:51.707659+0000","last_undegraded":"2023-09-30T22:26:51.707659+0000","last_fullsized":"2023-09-30T22:26:51.707659+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"15'2","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.721892+0000","last_change":"2023-09-30T22:26:51.290248+0000","last_active":"2023-09-30T22:26:51.721892+0000","last_peered":"2023-09-30T22:26:51.721892+0000","last_clean":"2023-09-30T22:26:51.721892+0000","last_became_active":"2023-09-30T22:26:51.289168+0000","last_became_peered":"2023-09-30T22:26:51.289168+0000","last_unstale":"2023-09-30T22:26:51.721892+0000","last_undegraded":"2023-09-30T22:26:51.721892+0000","last_fullsized":"2023-09-30T22:26:51.721892+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.723965+0000","last_change":"2023-09-30T22:26:51.719224+0000","last_active":"2023-09-30T22:26:51.723965+0000","last_peered":"2023-09-30T22:26:51.723965+0000","last_clean":"2023-09-30T22:26:51.723965+0000","last_became_active":"2023-09-30T22:26:51.718890+0000","last_became_peered":"2023-09-30T22:26:51.718890+0000","last_unstale":"2023-09-30T22:26:51.723965+0000","last_undegraded":"2023-09-30T22:26:51.723965+0000","last_fullsized":"2023-09-30T22:26:51.723965+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.687008+0000","last_change":"2023-09-30T22:26:51.288204+0000","last_active":"2023-09-30T22:26:51.687008+0000","last_peered":"2023-09-30T22:26:51.687008+0000","last_clean":"2023-09-30T22:26:51.687008+0000","last_became_active":"2023-09-30T22:26:51.287903+0000","last_became_peered":"2023-09-30T22:26:51.287903+0000","last_unstale":"2023-09-30T22:26:51.687008+0000","last_undegraded":"2023-09-30T22:26:51.687008+0000","last_fullsized":"2023-09-30T22:26:51.687008+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.718399+0000","last_change":"2023-09-30T22:26:51.713792+0000","last_active":"2023-09-30T22:26:51.718399+0000","last_peered":"2023-09-30T22:26:51.718399+0000","last_clean":"2023-09-30T22:26:51.718399+0000","last_became_active":"2023-09-30T22:26:51.713334+0000","last_became_peered":"2023-09-30T22:26:51.713334+0000","last_unstale":"2023-09-30T22:26:51.718399+0000","last_undegraded":"2023-09-30T22:26:51.718399+0000","last_fullsized":"2023-09-30T22:26:51.718399+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.683106+0000","last_change":"2023-09-30T22:26:51.297240+0000","last_active":"2023-09-30T22:26:51.683106+0000","last_peered":"2023-09-30T22:26:51.683106+0000","last_clean":"2023-09-30T22:26:51.683106+0000","last_became_active":"2023-09-30T22:26:51.296547+0000","last_became_peered":"2023-09-30T22:26:51.296547+0000","last_unstale":"2023-09-30T22:26:51.683106+0000","last_undegraded":"2023-09-30T22:26:51.683106+0000","last_fullsized":"2023-09-30T22:26:51.683106+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.718663+0000","last_change":"2023-09-30T22:26:51.711889+0000","last_active":"2023-09-30T22:26:51.718663+0000","last_peered":"2023-09-30T22:26:51.718663+0000","last_clean":"2023-09-30T22:26:51.718663+0000","last_became_active":"2023-09-30T22:26:51.711570+0000","last_became_peered":"2023-09-30T22:26:51.711570+0000","last_unstale":"2023-09-30T22:26:51.718663+0000","last_undegraded":"2023-09-30T22:26:51.718663+0000","last_fullsized":"2023-09-30T22:26:51.718663+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.684383+0000","last_change":"2023-09-30T22:26:51.290699+0000","last_active":"2023-09-30T22:26:51.684383+0000","last_peered":"2023-09-30T22:26:51.684383+0000","last_clean":"2023-09-30T22:26:51.684383+0000","last_became_active":"2023-09-30T22:26:51.290396+0000","last_became_peered":"2023-09-30T22:26:51.290396+0000","last_unstale":"2023-09-30T22:26:51.684383+0000","last_undegraded":"2023-09-30T22:26:51.684383+0000","last_fullsized":"2023-09-30T22:26:51.684383+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730474+0000","last_change":"2023-09-30T22:26:51.720976+0000","last_active":"2023-09-30T22:26:51.730474+0000","last_peered":"2023-09-30T22:26:51.730474+0000","last_clean":"2023-09-30T22:26:51.730474+0000","last_became_active":"2023-09-30T22:26:51.720378+0000","last_became_peered":"2023-09-30T22:26:51.720378+0000","last_unstale":"2023-09-30T22:26:51.730474+0000","last_undegraded":"2023-09-30T22:26:51.730474+0000","last_fullsized":"2023-09-30T22:26:51.730474+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.702412+0000","last_change":"2023-09-30T22:26:46.273891+0000","last_active":"2023-09-30T22:26:51.702412+0000","last_peered":"2023-09-30T22:26:51.702412+0000","last_clean":"2023-09-30T22:26:51.702412+0000","last_became_active":"2023-09-30T22:26:46.273588+0000","last_became_peered":"2023-09-30T22:26:46.273588+0000","last_unstale":"2023-09-30T22:26:51.702412+0000","last_undegraded":"2023-09-30T22:26:51.702412+0000","last_fullsized":"2023-09-30T22:26:51.702412+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.680672+0000","last_change":"2023-09-30T22:26:51.294682+0000","last_active":"2023-09-30T22:26:51.680672+0000","last_peered":"2023-09-30T22:26:51.680672+0000","last_clean":"2023-09-30T22:26:51.680672+0000","last_became_active":"2023-09-30T22:26:51.294422+0000","last_became_peered":"2023-09-30T22:26:51.294422+0000","last_unstale":"2023-09-30T22:26:51.680672+0000","last_undegraded":"2023-09-30T22:26:51.680672+0000","last_fullsized":"2023-09-30T22:26:51.680672+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730874+0000","last_change":"2023-09-30T22:26:51.726954+0000","last_active":"2023-09-30T22:26:51.730874+0000","last_peered":"2023-09-30T22:26:51.730874+0000","last_clean":"2023-09-30T22:26:51.730874+0000","last_became_active":"2023-09-30T22:26:51.726641+0000","last_became_peered":"2023-09-30T22:26:51.726641+0000","last_unstale":"2023-09-30T22:26:51.730874+0000","last_undegraded":"2023-09-30T22:26:51.730874+0000","last_fullsized":"2023-09-30T22:26:51.730874+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.687452+0000","last_change":"2023-09-30T22:26:46.245946+0000","last_active":"2023-09-30T22:26:51.687452+0000","last_peered":"2023-09-30T22:26:51.687452+0000","last_clean":"2023-09-30T22:26:51.687452+0000","last_became_active":"2023-09-30T22:26:46.245602+0000","last_became_peered":"2023-09-30T22:26:46.245602+0000","last_unstale":"2023-09-30T22:26:51.687452+0000","last_undegraded":"2023-09-30T22:26:51.687452+0000","last_fullsized":"2023-09-30T22:26:51.687452+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.685668+0000","last_change":"2023-09-30T22:26:51.294691+0000","last_active":"2023-09-30T22:26:51.685668+0000","last_peered":"2023-09-30T22:26:51.685668+0000","last_clean":"2023-09-30T22:26:51.685668+0000","last_became_active":"2023-09-30T22:26:51.294411+0000","last_became_peered":"2023-09-30T22:26:51.294411+0000","last_unstale":"2023-09-30T22:26:51.685668+0000","last_undegraded":"2023-09-30T22:26:51.685668+0000","last_fullsized":"2023-09-30T22:26:51.685668+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730680+0000","last_change":"2023-09-30T22:26:51.724759+0000","last_active":"2023-09-30T22:26:51.730680+0000","last_peered":"2023-09-30T22:26:51.730680+0000","last_clean":"2023-09-30T22:26:51.730680+0000","last_became_active":"2023-09-30T22:26:51.723221+0000","last_became_peered":"2023-09-30T22:26:51.723221+0000","last_unstale":"2023-09-30T22:26:51.730680+0000","last_undegraded":"2023-09-30T22:26:51.730680+0000","last_fullsized":"2023-09-30T22:26:51.730680+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.681986+0000","last_change":"2023-09-30T22:26:46.242607+0000","last_active":"2023-09-30T22:26:51.681986+0000","last_peered":"2023-09-30T22:26:51.681986+0000","last_clean":"2023-09-30T22:26:51.681986+0000","last_became_active":"2023-09-30T22:26:46.242283+0000","last_became_peered":"2023-09-30T22:26:46.242283+0000","last_unstale":"2023-09-30T22:26:51.681986+0000","last_undegraded":"2023-09-30T22:26:51.681986+0000","last_fullsized":"2023-09-30T22:26:51.681986+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.703666+0000","last_change":"2023-09-30T22:26:51.295409+0000","last_active":"2023-09-30T22:26:51.703666+0000","last_peered":"2023-09-30T22:26:51.703666+0000","last_clean":"2023-09-30T22:26:51.703666+0000","last_became_active":"2023-09-30T22:26:51.295140+0000","last_became_peered":"2023-09-30T22:26:51.295140+0000","last_unstale":"2023-09-30T22:26:51.703666+0000","last_undegraded":"2023-09-30T22:26:51.703666+0000","last_fullsized":"2023-09-30T22:26:51.703666+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730344+0000","last_change":"2023-09-30T22:26:51.715504+0000","last_active":"2023-09-30T22:26:51.730344+0000","last_peered":"2023-09-30T22:26:51.730344+0000","last_clean":"2023-09-30T22:26:51.730344+0000","last_became_active":"2023-09-30T22:26:51.715258+0000","last_became_peered":"2023-09-30T22:26:51.715258+0000","last_unstale":"2023-09-30T22:26:51.730344+0000","last_undegraded":"2023-09-30T22:26:51.730344+0000","last_fullsized":"2023-09-30T22:26:51.730344+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.682511+0000","last_change":"2023-09-30T22:26:46.242752+0000","last_active":"2023-09-30T22:26:51.682511+0000","last_peered":"2023-09-30T22:26:51.682511+0000","last_clean":"2023-09-30T22:26:51.682511+0000","last_became_active":"2023-09-30T22:26:46.242385+0000","last_became_peered":"2023-09-30T22:26:46.242385+0000","last_unstale":"2023-09-30T22:26:51.682511+0000","last_undegraded":"2023-09-30T22:26:51.682511+0000","last_fullsized":"2023-09-30T22:26:51.682511+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.703565+0000","last_change":"2023-09-30T22:26:46.273930+0000","last_active":"2023-09-30T22:26:51.703565+0000","last_peered":"2023-09-30T22:26:51.703565+0000","last_clean":"2023-09-30T22:26:51.703565+0000","last_became_active":"2023-09-30T22:26:46.273621+0000","last_became_peered":"2023-09-30T22:26:46.273621+0000","last_unstale":"2023-09-30T22:26:51.703565+0000","last_undegraded":"2023-09-30T22:26:51.703565+0000","last_fullsized":"2023-09-30T22:26:51.703565+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.686892+0000","last_change":"2023-09-30T22:26:51.298163+0000","last_active":"2023-09-30T22:26:51.686892+0000","last_peered":"2023-09-30T22:26:51.686892+0000","last_clean":"2023-09-30T22:26:51.686892+0000","last_became_active":"2023-09-30T22:26:51.297897+0000","last_became_peered":"2023-09-30T22:26:51.297897+0000","last_unstale":"2023-09-30T22:26:51.686892+0000","last_undegraded":"2023-09-30T22:26:51.686892+0000","last_fullsized":"2023-09-30T22:26:51.686892+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.718377+0000","last_change":"2023-09-30T22:26:51.713608+0000","last_active":"2023-09-30T22:26:51.718377+0000","last_peered":"2023-09-30T22:26:51.718377+0000","last_clean":"2023-09-30T22:26:51.718377+0000","last_became_active":"2023-09-30T22:26:51.712992+0000","last_became_peered":"2023-09-30T22:26:51.712992+0000","last_unstale":"2023-09-30T22:26:51.718377+0000","last_undegraded":"2023-09-30T22:26:51.718377+0000","last_fullsized":"2023-09-30T22:26:51.718377+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.684824+0000","last_change":"2023-09-30T22:26:46.245668+0000","last_active":"2023-09-30T22:26:51.684824+0000","last_peered":"2023-09-30T22:26:51.684824+0000","last_clean":"2023-09-30T22:26:51.684824+0000","last_became_active":"2023-09-30T22:26:46.245319+0000","last_became_peered":"2023-09-30T22:26:46.245319+0000","last_unstale":"2023-09-30T22:26:51.684824+0000","last_undegraded":"2023-09-30T22:26:51.684824+0000","last_fullsized":"2023-09-30T22:26:51.684824+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.681181+0000","last_change":"2023-09-30T22:26:51.294664+0000","last_active":"2023-09-30T22:26:51.681181+0000","last_peered":"2023-09-30T22:26:51.681181+0000","last_clean":"2023-09-30T22:26:51.681181+0000","last_became_active":"2023-09-30T22:26:51.294357+0000","last_became_peered":"2023-09-30T22:26:51.294357+0000","last_unstale":"2023-09-30T22:26:51.681181+0000","last_undegraded":"2023-09-30T22:26:51.681181+0000","last_fullsized":"2023-09-30T22:26:51.681181+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.707651+0000","last_change":"2023-09-30T22:26:51.705784+0000","last_active":"2023-09-30T22:26:51.707651+0000","last_peered":"2023-09-30T22:26:51.707651+0000","last_clean":"2023-09-30T22:26:51.707651+0000","last_became_active":"2023-09-30T22:26:51.705480+0000","last_became_peered":"2023-09-30T22:26:51.705480+0000","last_unstale":"2023-09-30T22:26:51.707651+0000","last_undegraded":"2023-09-30T22:26:51.707651+0000","last_fullsized":"2023-09-30T22:26:51.707651+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.689511+0000","last_change":"2023-09-30T22:26:46.274062+0000","last_active":"2023-09-30T22:26:51.689511+0000","last_peered":"2023-09-30T22:26:51.689511+0000","last_clean":"2023-09-30T22:26:51.689511+0000","last_became_active":"2023-09-30T22:26:46.273752+0000","last_became_peered":"2023-09-30T22:26:46.273752+0000","last_unstale":"2023-09-30T22:26:51.689511+0000","last_undegraded":"2023-09-30T22:26:51.689511+0000","last_fullsized":"2023-09-30T22:26:51.689511+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.702185+0000","last_change":"2023-09-30T22:26:51.295181+0000","last_active":"2023-09-30T22:26:51.702185+0000","last_peered":"2023-09-30T22:26:51.702185+0000","last_clean":"2023-09-30T22:26:51.702185+0000","last_became_active":"2023-09-30T22:26:51.294889+0000","last_became_peered":"2023-09-30T22:26:51.294889+0000","last_unstale":"2023-09-30T22:26:51.702185+0000","last_undegraded":"2023-09-30T22:26:51.702185+0000","last_fullsized":"2023-09-30T22:26:51.702185+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.698806+0000","last_change":"2023-09-30T22:26:51.690467+0000","last_active":"2023-09-30T22:26:51.698806+0000","last_peered":"2023-09-30T22:26:51.698806+0000","last_clean":"2023-09-30T22:26:51.698806+0000","last_became_active":"2023-09-30T22:26:51.690163+0000","last_became_peered":"2023-09-30T22:26:51.690163+0000","last_unstale":"2023-09-30T22:26:51.698806+0000","last_undegraded":"2023-09-30T22:26:51.698806+0000","last_fullsized":"2023-09-30T22:26:51.698806+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"15'2","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.716522+0000","last_change":"2023-09-30T22:26:51.290268+0000","last_active":"2023-09-30T22:26:51.716522+0000","last_peered":"2023-09-30T22:26:51.716522+0000","last_clean":"2023-09-30T22:26:51.716522+0000","last_became_active":"2023-09-30T22:26:51.289409+0000","last_became_peered":"2023-09-30T22:26:51.289409+0000","last_unstale":"2023-09-30T22:26:51.716522+0000","last_undegraded":"2023-09-30T22:26:51.716522+0000","last_fullsized":"2023-09-30T22:26:51.716522+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.712603+0000","last_change":"2023-09-30T22:26:46.273888+0000","last_active":"2023-09-30T22:26:51.712603+0000","last_peered":"2023-09-30T22:26:51.712603+0000","last_clean":"2023-09-30T22:26:51.712603+0000","last_became_active":"2023-09-30T22:26:46.273549+0000","last_became_peered":"2023-09-30T22:26:46.273549+0000","last_unstale":"2023-09-30T22:26:51.712603+0000","last_undegraded":"2023-09-30T22:26:51.712603+0000","last_fullsized":"2023-09-30T22:26:51.712603+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.714861+0000","last_change":"2023-09-30T22:26:51.709356+0000","last_active":"2023-09-30T22:26:51.714861+0000","last_peered":"2023-09-30T22:26:51.714861+0000","last_clean":"2023-09-30T22:26:51.714861+0000","last_became_active":"2023-09-30T22:26:51.709059+0000","last_became_peered":"2023-09-30T22:26:51.709059+0000","last_unstale":"2023-09-30T22:26:51.714861+0000","last_undegraded":"2023-09-30T22:26:51.714861+0000","last_fullsized":"2023-09-30T22:26:51.714861+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.682520+0000","last_change":"2023-09-30T22:26:46.274508+0000","last_active":"2023-09-30T22:26:51.682520+0000","last_peered":"2023-09-30T22:26:51.682520+0000","last_clean":"2023-09-30T22:26:51.682520+0000","last_became_active":"2023-09-30T22:26:46.273567+0000","last_became_peered":"2023-09-30T22:26:46.273567+0000","last_unstale":"2023-09-30T22:26:51.682520+0000","last_undegraded":"2023-09-30T22:26:51.682520+0000","last_fullsized":"2023-09-30T22:26:51.682520+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.690008+0000","last_change":"2023-09-30T22:26:51.288625+0000","last_active":"2023-09-30T22:26:51.690008+0000","last_peered":"2023-09-30T22:26:51.690008+0000","last_clean":"2023-09-30T22:26:51.690008+0000","last_became_active":"2023-09-30T22:26:51.288286+0000","last_became_peered":"2023-09-30T22:26:51.288286+0000","last_unstale":"2023-09-30T22:26:51.690008+0000","last_undegraded":"2023-09-30T22:26:51.690008+0000","last_fullsized":"2023-09-30T22:26:51.690008+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.718567+0000","last_change":"2023-09-30T22:26:51.708965+0000","last_active":"2023-09-30T22:26:51.718567+0000","last_peered":"2023-09-30T22:26:51.718567+0000","last_clean":"2023-09-30T22:26:51.718567+0000","last_became_active":"2023-09-30T22:26:51.708607+0000","last_became_peered":"2023-09-30T22:26:51.708607+0000","last_unstale":"2023-09-30T22:26:51.718567+0000","last_undegraded":"2023-09-30T22:26:51.718567+0000","last_fullsized":"2023-09-30T22:26:51.718567+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"15'1","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.719553+0000","last_change":"2023-09-30T22:26:51.282125+0000","last_active":"2023-09-30T22:26:51.719553+0000","last_peered":"2023-09-30T22:26:51.719553+0000","last_clean":"2023-09-30T22:26:51.719553+0000","last_became_active":"2023-09-30T22:26:51.281780+0000","last_became_peered":"2023-09-30T22:26:51.281780+0000","last_unstale":"2023-09-30T22:26:51.719553+0000","last_undegraded":"2023-09-30T22:26:51.719553+0000","last_fullsized":"2023-09-30T22:26:51.719553+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.709532+0000","last_change":"2023-09-30T22:26:51.703281+0000","last_active":"2023-09-30T22:26:51.709532+0000","last_peered":"2023-09-30T22:26:51.709532+0000","last_clean":"2023-09-30T22:26:51.709532+0000","last_became_active":"2023-09-30T22:26:51.703007+0000","last_became_peered":"2023-09-30T22:26:51.703007+0000","last_unstale":"2023-09-30T22:26:51.709532+0000","last_undegraded":"2023-09-30T22:26:51.709532+0000","last_fullsized":"2023-09-30T22:26:51.709532+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"15'2","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.726495+0000","last_change":"2023-09-30T22:26:51.287630+0000","last_active":"2023-09-30T22:26:51.726495+0000","last_peered":"2023-09-30T22:26:51.726495+0000","last_clean":"2023-09-30T22:26:51.726495+0000","last_became_active":"2023-09-30T22:26:51.287337+0000","last_became_peered":"2023-09-30T22:26:51.287337+0000","last_unstale":"2023-09-30T22:26:51.726495+0000","last_undegraded":"2023-09-30T22:26:51.726495+0000","last_fullsized":"2023-09-30T22:26:51.726495+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.719532+0000","last_change":"2023-09-30T22:26:51.716458+0000","last_active":"2023-09-30T22:26:51.719532+0000","last_peered":"2023-09-30T22:26:51.719532+0000","last_clean":"2023-09-30T22:26:51.719532+0000","last_became_active":"2023-09-30T22:26:51.715957+0000","last_became_peered":"2023-09-30T22:26:51.715957+0000","last_unstale":"2023-09-30T22:26:51.719532+0000","last_undegraded":"2023-09-30T22:26:51.719532+0000","last_fullsized":"2023-09-30T22:26:51.719532+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"15'2","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.716732+0000","last_change":"2023-09-30T22:26:51.285956+0000","last_active":"2023-09-30T22:26:51.716732+0000","last_peered":"2023-09-30T22:26:51.716732+0000","last_clean":"2023-09-30T22:26:51.716732+0000","last_became_active":"2023-09-30T22:26:51.285690+0000","last_became_peered":"2023-09-30T22:26:51.285690+0000","last_unstale":"2023-09-30T22:26:51.716732+0000","last_undegraded":"2023-09-30T22:26:51.716732+0000","last_fullsized":"2023-09-30T22:26:51.716732+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.719485+0000","last_change":"2023-09-30T22:26:51.716290+0000","last_active":"2023-09-30T22:26:51.719485+0000","last_peered":"2023-09-30T22:26:51.719485+0000","last_clean":"2023-09-30T22:26:51.719485+0000","last_became_active":"2023-09-30T22:26:51.716007+0000","last_became_peered":"2023-09-30T22:26:51.716007+0000","last_unstale":"2023-09-30T22:26:51.719485+0000","last_undegraded":"2023-09-30T22:26:51.719485+0000","last_fullsized":"2023-09-30T22:26:51.719485+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1e","version":"15'1","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.719629+0000","last_change":"2023-09-30T22:26:51.296949+0000","last_active":"2023-09-30T22:26:51.719629+0000","last_peered":"2023-09-30T22:26:51.719629+0000","last_clean":"2023-09-30T22:26:51.719629+0000","last_became_active":"2023-09-30T22:26:51.296602+0000","last_became_peered":"2023-09-30T22:26:51.296602+0000","last_unstale":"2023-09-30T22:26:51.719629+0000","last_undegraded":"2023-09-30T22:26:51.719629+0000","last_fullsized":"2023-09-30T22:26:51.719629+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.714046+0000","last_change":"2023-09-30T22:26:51.706034+0000","last_active":"2023-09-30T22:26:51.714046+0000","last_peered":"2023-09-30T22:26:51.714046+0000","last_clean":"2023-09-30T22:26:51.714046+0000","last_became_active":"2023-09-30T22:26:51.705633+0000","last_became_peered":"2023-09-30T22:26:51.705633+0000","last_unstale":"2023-09-30T22:26:51.714046+0000","last_undegraded":"2023-09-30T22:26:51.714046+0000","last_fullsized":"2023-09-30T22:26:51.714046+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"15'4","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.722169+0000","last_change":"2023-09-30T22:26:51.291976+0000","last_active":"2023-09-30T22:26:51.722169+0000","last_peered":"2023-09-30T22:26:51.722169+0000","last_clean":"2023-09-30T22:26:51.722169+0000","last_became_active":"2023-09-30T22:26:51.291255+0000","last_became_peered":"2023-09-30T22:26:51.291255+0000","last_unstale":"2023-09-30T22:26:51.722169+0000","last_undegraded":"2023-09-30T22:26:51.722169+0000","last_fullsized":"2023-09-30T22:26:51.722169+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.718466+0000","last_change":"2023-09-30T22:26:51.713485+0000","last_active":"2023-09-30T22:26:51.718466+0000","last_peered":"2023-09-30T22:26:51.718466+0000","last_clean":"2023-09-30T22:26:51.718466+0000","last_became_active":"2023-09-30T22:26:51.713188+0000","last_became_peered":"2023-09-30T22:26:51.713188+0000","last_unstale":"2023-09-30T22:26:51.718466+0000","last_undegraded":"2023-09-30T22:26:51.718466+0000","last_fullsized":"2023-09-30T22:26:51.718466+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.709789+0000","last_change":"2023-09-30T22:26:51.704082+0000","last_active":"2023-09-30T22:26:51.709789+0000","last_peered":"2023-09-30T22:26:51.709789+0000","last_clean":"2023-09-30T22:26:51.709789+0000","last_became_active":"2023-09-30T22:26:51.703280+0000","last_became_peered":"2023-09-30T22:26:51.703280+0000","last_unstale":"2023-09-30T22:26:51.709789+0000","last_undegraded":"2023-09-30T22:26:51.709789+0000","last_fullsized":"2023-09-30T22:26:51.709789+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.714010+0000","last_change":"2023-09-30T22:26:51.705877+0000","last_active":"2023-09-30T22:26:51.714010+0000","last_peered":"2023-09-30T22:26:51.714010+0000","last_clean":"2023-09-30T22:26:51.714010+0000","last_became_active":"2023-09-30T22:26:51.705352+0000","last_became_peered":"2023-09-30T22:26:51.705352+0000","last_unstale":"2023-09-30T22:26:51.714010+0000","last_undegraded":"2023-09-30T22:26:51.714010+0000","last_fullsized":"2023-09-30T22:26:51.714010+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.719132+0000","last_change":"2023-09-30T22:26:51.712201+0000","last_active":"2023-09-30T22:26:51.719132+0000","last_peered":"2023-09-30T22:26:51.719132+0000","last_clean":"2023-09-30T22:26:51.719132+0000","last_became_active":"2023-09-30T22:26:51.709083+0000","last_became_peered":"2023-09-30T22:26:51.709083+0000","last_unstale":"2023-09-30T22:26:51.719132+0000","last_undegraded":"2023-09-30T22:26:51.719132+0000","last_fullsized":"2023-09-30T22:26:51.719132+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.715082+0000","last_change":"2023-09-30T22:26:51.706543+0000","last_active":"2023-09-30T22:26:51.715082+0000","last_peered":"2023-09-30T22:26:51.715082+0000","last_clean":"2023-09-30T22:26:51.715082+0000","last_became_active":"2023-09-30T22:26:51.706251+0000","last_became_peered":"2023-09-30T22:26:51.706251+0000","last_unstale":"2023-09-30T22:26:51.715082+0000","last_undegraded":"2023-09-30T22:26:51.715082+0000","last_fullsized":"2023-09-30T22:26:51.715082+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730665+0000","last_change":"2023-09-30T22:26:51.724591+0000","last_active":"2023-09-30T22:26:51.730665+0000","last_peered":"2023-09-30T22:26:51.730665+0000","last_clean":"2023-09-30T22:26:51.730665+0000","last_became_active":"2023-09-30T22:26:51.722847+0000","last_became_peered":"2023-09-30T22:26:51.722847+0000","last_unstale":"2023-09-30T22:26:51.730665+0000","last_undegraded":"2023-09-30T22:26:51.730665+0000","last_fullsized":"2023-09-30T22:26:51.730665+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705667,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297256,"kb_used_data":160,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074584,"statfs":{"total":96636764160,"available":96332374016,"internally_reserved":0,"allocated":163840,"data_stored":46244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38800000000000001}]}]},{"osd":2,"up_from":9,"seq":38654705668,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297244,"kb_used_data":148,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074596,"statfs":{"total":96636764160,"available":96332386304,"internally_reserved":0,"allocated":151552,"data_stored":46050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.315}]}]},{"osd":0,"up_from":9,"seq":38654705667,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297244,"kb_used_data":148,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074596,"statfs":{"total":96636764160,"available":96332386304,"internally_reserved":0,"allocated":151552,"data_stored":44044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]}]},{"osd":1,"up_from":9,"seq":38654705667,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297244,"kb_used_data":148,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074596,"statfs":{"total":96636764160,"available":96332386304,"internally_reserved":0,"allocated":151552,"data_stored":45160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38300000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-30T22:26:56.509 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-09-30T22:26:56.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.676+0000 7ff72683d700 1 Processor -- start 2023-09-30T22:26:56.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.676+0000 7ff72683d700 1 -- start start 2023-09-30T22:26:56.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.676+0000 7ff72683d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff7200969b0 0x7ff720096d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:56.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.677+0000 7ff72683d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff720097280 0x7ff720066cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:56.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.677+0000 7ff72683d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff720092fe0 con 0x7ff7200671c0 2023-09-30T22:26:56.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.677+0000 7ff72683d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff720094560 con 0x7ff720097280 2023-09-30T22:26:56.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.677+0000 7ff72683d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff720093260 con 0x7ff7200969b0 2023-09-30T22:26:56.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.677+0000 7ff71f7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff720097280 0x7ff720066cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:56.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.677+0000 7ff71ffff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff7200969b0 0x7ff720096d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:56.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.677+0000 7ff724dda700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff7200671c0 0x7ff72012fcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33726/0 (socket says 172.21.15.16:33726) 2023-09-30T22:26:56.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.677+0000 7ff724dda700 1 -- 172.21.15.16:0/1048549835 learned_addr learned my addr 172.21.15.16:0/1048549835 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:26:56.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.677+0000 7ff71f7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff720097280 0x7ff720066cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38776/0 (socket says 172.21.15.16:38776) 2023-09-30T22:26:56.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.677+0000 7ff71f7fe700 1 -- 172.21.15.16:0/1048549835 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff7200969b0 msgr2=0x7ff720096d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:56.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.677+0000 7ff71f7fe700 1 --2- 172.21.15.16:0/1048549835 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff7200969b0 0x7ff720096d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:56.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.677+0000 7ff71f7fe700 1 -- 172.21.15.16:0/1048549835 >> v1:172.21.15.16:6789/0 conn(0x7ff7200671c0 legacy=0x7ff72012fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:56.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.677+0000 7ff71f7fe700 1 -- 172.21.15.16:0/1048549835 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff720092b30 con 0x7ff720097280 2023-09-30T22:26:56.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.677+0000 7ff71effd700 1 -- 172.21.15.16:0/1048549835 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 388416853 0 0) 0x7ff720092fe0 con 0x7ff7200671c0 2023-09-30T22:26:56.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.677+0000 7ff71ffff700 1 --2- 172.21.15.16:0/1048549835 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff7200969b0 0x7ff720096d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:56.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.677+0000 7ff71f7fe700 1 --2- 172.21.15.16:0/1048549835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff720097280 0x7ff720066cd0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7ff714009760 tx=0x7ff714009060).ready entity=mon.0 client_cookie=56beeef9477c2033 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:56.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.677+0000 7ff71effd700 1 -- 172.21.15.16:0/1048549835 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71400f510 con 0x7ff720097280 2023-09-30T22:26:56.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.677+0000 7ff71effd700 1 -- 172.21.15.16:0/1048549835 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff71400f670 con 0x7ff720097280 2023-09-30T22:26:56.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.677+0000 7ff71effd700 1 -- 172.21.15.16:0/1048549835 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71400f9f0 con 0x7ff720097280 2023-09-30T22:26:56.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.678+0000 7ff72683d700 1 -- 172.21.15.16:0/1048549835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff720097280 msgr2=0x7ff720066cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:56.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.678+0000 7ff72683d700 1 --2- 172.21.15.16:0/1048549835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff720097280 0x7ff720066cd0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7ff714009760 tx=0x7ff714009060).stop 2023-09-30T22:26:56.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.678+0000 7ff72683d700 1 -- 172.21.15.16:0/1048549835 shutdown_connections 2023-09-30T22:26:56.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.678+0000 7ff72683d700 1 --2- 172.21.15.16:0/1048549835 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff7200969b0 0x7ff720096d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:56.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.678+0000 7ff72683d700 1 --2- 172.21.15.16:0/1048549835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff720097280 0x7ff720066cd0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:56.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.678+0000 7ff72683d700 1 -- 172.21.15.16:0/1048549835 >> 172.21.15.16:0/1048549835 conn(0x7ff72009a220 msgr2=0x7ff72009a630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:56.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.678+0000 7ff72683d700 1 -- 172.21.15.16:0/1048549835 shutdown_connections 2023-09-30T22:26:56.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.678+0000 7ff72683d700 1 -- 172.21.15.16:0/1048549835 wait complete. 2023-09-30T22:26:56.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.679+0000 7ff72683d700 1 Processor -- start 2023-09-30T22:26:56.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.679+0000 7ff72683d700 1 -- start start 2023-09-30T22:26:56.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.680+0000 7ff72683d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff7200671c0 0x7ff720138470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:56.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.680+0000 7ff72683d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff7200969b0 0x7ff720138960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:56.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.680+0000 7ff72683d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff720097280 0x7ff720116f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:56.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.680+0000 7ff72683d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7201183d0 con 0x7ff720097280 2023-09-30T22:26:56.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.680+0000 7ff72683d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff72012d1c0 con 0x7ff7200671c0 2023-09-30T22:26:56.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.680+0000 7ff72683d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff7200669c0 con 0x7ff7200969b0 2023-09-30T22:26:56.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.680+0000 7ff71ffff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff7200671c0 0x7ff720138470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:56.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.680+0000 7ff71f7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff7200969b0 0x7ff720138960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:56.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.680+0000 7ff71ffff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff7200671c0 0x7ff720138470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:44670/0 (socket says 172.21.15.16:44670) 2023-09-30T22:26:56.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.680+0000 7ff71ffff700 1 -- 172.21.15.16:0/1242628738 learned_addr learned my addr 172.21.15.16:0/1242628738 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:26:56.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.680+0000 7ff71f7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff7200969b0 0x7ff720138960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:46364/0 (socket says 172.21.15.16:46364) 2023-09-30T22:26:56.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.680+0000 7ff724dda700 1 --2- 172.21.15.16:0/1242628738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff720097280 0x7ff720116f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:56.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.680+0000 7ff71f7fe700 1 -- 172.21.15.16:0/1242628738 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff7200671c0 msgr2=0x7ff720138470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:56.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.680+0000 7ff71f7fe700 1 --2- 172.21.15.16:0/1242628738 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff7200671c0 0x7ff720138470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:56.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.680+0000 7ff71f7fe700 1 -- 172.21.15.16:0/1242628738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff720097280 msgr2=0x7ff720116f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:56.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.680+0000 7ff71f7fe700 1 --2- 172.21.15.16:0/1242628738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff720097280 0x7ff720116f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:56.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.680+0000 7ff71f7fe700 1 -- 172.21.15.16:0/1242628738 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7100092a0 con 0x7ff7200969b0 2023-09-30T22:26:56.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.680+0000 7ff724dda700 1 --2- 172.21.15.16:0/1242628738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff720097280 0x7ff720116f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:56.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.680+0000 7ff71ffff700 1 --2- 172.21.15.16:0/1242628738 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff7200671c0 0x7ff720138470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:56.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.680+0000 7ff71f7fe700 1 --2- 172.21.15.16:0/1242628738 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff7200969b0 0x7ff720138960 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7ff714014000 tx=0x7ff714017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:56.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.681+0000 7ff707fff700 1 -- 172.21.15.16:0/1242628738 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71400f7e0 con 0x7ff7200969b0 2023-09-30T22:26:56.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.681+0000 7ff707fff700 1 -- 172.21.15.16:0/1242628738 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff71400f940 con 0x7ff7200969b0 2023-09-30T22:26:56.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.681+0000 7ff707fff700 1 -- 172.21.15.16:0/1242628738 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71401b660 con 0x7ff7200969b0 2023-09-30T22:26:56.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.681+0000 7ff72683d700 1 -- 172.21.15.16:0/1242628738 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7140092a0 con 0x7ff7200969b0 2023-09-30T22:26:56.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.681+0000 7ff72683d700 1 -- 172.21.15.16:0/1242628738 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff720119430 con 0x7ff7200969b0 2023-09-30T22:26:56.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.681+0000 7ff707fff700 1 -- 172.21.15.16:0/1242628738 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7ff71401b9d0 con 0x7ff7200969b0 2023-09-30T22:26:56.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.682+0000 7ff707fff700 1 -- 172.21.15.16:0/1242628738 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3690+0+0 (secure 0 0 0) 0x7ff71403c940 con 0x7ff7200969b0 2023-09-30T22:26:56.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.682+0000 7ff72683d700 1 -- 172.21.15.16:0/1242628738 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff72005a260 con 0x7ff7200969b0 2023-09-30T22:26:56.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.687+0000 7ff707fff700 1 -- 172.21.15.16:0/1242628738 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7ff714028030 con 0x7ff7200969b0 2023-09-30T22:26:56.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.940+0000 7ff72683d700 1 -- 172.21.15.16:0/1242628738 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7ff720046c20 con 0x7ff7200969b0 2023-09-30T22:26:56.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.940+0000 7ff707fff700 1 -- 172.21.15.16:0/1242628738 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+951 (secure 0 0 0) 0x7ff714019380 con 0x7ff7200969b0 2023-09-30T22:26:56.942 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:26:56.942 INFO:teuthology.orchestra.run.smithi016.stdout:{"fsid":"7ede3e14-2bd0-4726-9437-6d13c90cd342","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":21,"monmap":{"epoch":1,"min_mon_release_name":"pacific","num_mons":3},"osdmap":{"epoch":16,"num_osds":4,"num_up_osds":4,"osd_up_since":1696112804,"num_in_osds":4,"osd_in_since":1696112797,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105}],"num_pgs":105,"num_pools":4,"num_objects":21,"data_bytes":2306,"bytes_used":1217523712,"bytes_avail":385329532928,"bytes_total":386547056640},"fsmap":{"epoch":6,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"a","status":"up:active","gid":4218}],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2023-09-30T22:26:35.485048+0000","services":{}},"progress_events":{}} 2023-09-30T22:26:56.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.942+0000 7ff72683d700 1 -- 172.21.15.16:0/1242628738 >> v1:172.21.15.16:6816/118285 conn(0x7ff70c02bb90 legacy=0x7ff70c02e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:56.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.942+0000 7ff72683d700 1 -- 172.21.15.16:0/1242628738 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff7200969b0 msgr2=0x7ff720138960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:56.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.942+0000 7ff72683d700 1 --2- 172.21.15.16:0/1242628738 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff7200969b0 0x7ff720138960 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7ff714014000 tx=0x7ff714017000).stop 2023-09-30T22:26:56.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.942+0000 7ff72683d700 1 -- 172.21.15.16:0/1242628738 shutdown_connections 2023-09-30T22:26:56.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.942+0000 7ff72683d700 1 --2- 172.21.15.16:0/1242628738 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff7200671c0 0x7ff720138470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:56.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.942+0000 7ff72683d700 1 --2- 172.21.15.16:0/1242628738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff720097280 0x7ff720116f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:56.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.942+0000 7ff72683d700 1 --2- 172.21.15.16:0/1242628738 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff7200969b0 0x7ff720138960 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:56.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.942+0000 7ff72683d700 1 -- 172.21.15.16:0/1242628738 >> 172.21.15.16:0/1242628738 conn(0x7ff72009a220 msgr2=0x7ff720095590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:56.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.943+0000 7ff72683d700 1 -- 172.21.15.16:0/1242628738 shutdown_connections 2023-09-30T22:26:56.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:56.943+0000 7ff72683d700 1 -- 172.21.15.16:0/1242628738 wait complete. 2023-09-30T22:26:56.957 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-09-30T22:26:56.957 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-30T22:26:57.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.085+0000 7f0963ea8700 1 Processor -- start 2023-09-30T22:26:57.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.086+0000 7f0963ea8700 1 -- start start 2023-09-30T22:26:57.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.086+0000 7f0963ea8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f095c1251c0 0x7f095c12da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:57.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.086+0000 7f0963ea8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f095c125dd0 0x7f095c12ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:57.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.086+0000 7f0963ea8700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f095c095f00 con 0x7f095c059070 2023-09-30T22:26:57.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.086+0000 7f0963ea8700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f095c059450 con 0x7f095c125dd0 2023-09-30T22:26:57.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.086+0000 7f0963ea8700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f095c096190 con 0x7f095c1251c0 2023-09-30T22:26:57.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.086+0000 7f0962445700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f095c125dd0 0x7f095c12ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:57.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.086+0000 7f0961443700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f095c1251c0 0x7f095c12da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:57.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.086+0000 7f0961c44700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f095c059070 0x7f095c1256c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33730/0 (socket says 172.21.15.16:33730) 2023-09-30T22:26:57.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.086+0000 7f0961c44700 1 -- 172.21.15.16:0/2629190445 learned_addr learned my addr 172.21.15.16:0/2629190445 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:26:57.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.087+0000 7f0960c42700 1 -- 172.21.15.16:0/2629190445 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1834854630 0 0) 0x7f095c095f00 con 0x7f095c059070 2023-09-30T22:26:57.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.087+0000 7f0962445700 1 -- 172.21.15.16:0/2629190445 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f095c1251c0 msgr2=0x7f095c12da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:26:57.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.087+0000 7f0962445700 1 --2- 172.21.15.16:0/2629190445 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f095c1251c0 0x7f095c12da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:57.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.087+0000 7f0962445700 1 -- 172.21.15.16:0/2629190445 >> v1:172.21.15.16:6789/0 conn(0x7f095c059070 legacy=0x7f095c1256c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:57.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.087+0000 7f0962445700 1 -- 172.21.15.16:0/2629190445 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f095c059dd0 con 0x7f095c125dd0 2023-09-30T22:26:57.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.087+0000 7f0961443700 1 --2- 172.21.15.16:0/2629190445 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f095c1251c0 0x7f095c12da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:57.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.087+0000 7f0962445700 1 --2- 172.21.15.16:0/2629190445 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f095c125dd0 0x7f095c12ff80 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f0954009e90 tx=0x7f0954009780).ready entity=mon.1 client_cookie=cb85d52d50c90742 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:57.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.087+0000 7f0960c42700 1 -- 172.21.15.16:0/2629190445 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095400e070 con 0x7f095c125dd0 2023-09-30T22:26:57.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.087+0000 7f0960c42700 1 -- 172.21.15.16:0/2629190445 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f095400bb60 con 0x7f095c125dd0 2023-09-30T22:26:57.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.087+0000 7f0960c42700 1 -- 172.21.15.16:0/2629190445 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0954012410 con 0x7f095c125dd0 2023-09-30T22:26:57.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.087+0000 7f0963ea8700 1 -- 172.21.15.16:0/2629190445 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f095c125dd0 msgr2=0x7f095c12ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:57.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.087+0000 7f0963ea8700 1 --2- 172.21.15.16:0/2629190445 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f095c125dd0 0x7f095c12ff80 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f0954009e90 tx=0x7f0954009780).stop 2023-09-30T22:26:57.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.088+0000 7f0963ea8700 1 -- 172.21.15.16:0/2629190445 shutdown_connections 2023-09-30T22:26:57.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.088+0000 7f0963ea8700 1 --2- 172.21.15.16:0/2629190445 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f095c125dd0 0x7f095c12ff80 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:57.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.088+0000 7f0963ea8700 1 --2- 172.21.15.16:0/2629190445 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f095c1251c0 0x7f095c12da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:57.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.088+0000 7f0963ea8700 1 -- 172.21.15.16:0/2629190445 >> 172.21.15.16:0/2629190445 conn(0x7f095c097470 msgr2=0x7f095c097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:57.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.088+0000 7f0963ea8700 1 -- 172.21.15.16:0/2629190445 shutdown_connections 2023-09-30T22:26:57.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.088+0000 7f0963ea8700 1 -- 172.21.15.16:0/2629190445 wait complete. 2023-09-30T22:26:57.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.088+0000 7f0963ea8700 1 Processor -- start 2023-09-30T22:26:57.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.089+0000 7f0963ea8700 1 -- start start 2023-09-30T22:26:57.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.089+0000 7f0963ea8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f095c059070 0x7f095c122b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:57.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.089+0000 7f0963ea8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f095c1251c0 0x7f095c0664e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:57.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.089+0000 7f0963ea8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f095c125dd0 0x7f095c0669d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:26:57.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.089+0000 7f0963ea8700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f095c069420 con 0x7f095c125dd0 2023-09-30T22:26:57.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.089+0000 7f0963ea8700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f095c06a290 con 0x7f095c1251c0 2023-09-30T22:26:57.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.089+0000 7f0963ea8700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f095c12fe90 con 0x7f095c059070 2023-09-30T22:26:57.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.089+0000 7f0961c44700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f095c059070 0x7f095c122b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:57.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.089+0000 7f0962445700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f095c125dd0 0x7f095c0669d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:57.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.089+0000 7f0961443700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f095c1251c0 0x7f095c0664e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:26:57.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.089+0000 7f0961c44700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f095c059070 0x7f095c122b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:46376/0 (socket says 172.21.15.16:46376) 2023-09-30T22:26:57.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.089+0000 7f0962445700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f095c125dd0 0x7f095c0669d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38788/0 (socket says 172.21.15.16:38788) 2023-09-30T22:26:57.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.089+0000 7f0961c44700 1 -- 172.21.15.16:0/3758686023 learned_addr learned my addr 172.21.15.16:0/3758686023 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:26:57.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.089+0000 7f0961443700 1 -- 172.21.15.16:0/3758686023 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f095c059070 msgr2=0x7f095c122b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:57.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.089+0000 7f0961443700 1 --2- 172.21.15.16:0/3758686023 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f095c059070 0x7f095c122b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:57.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.089+0000 7f0961443700 1 -- 172.21.15.16:0/3758686023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f095c125dd0 msgr2=0x7f095c0669d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:57.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.089+0000 7f0961443700 1 --2- 172.21.15.16:0/3758686023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f095c125dd0 0x7f095c0669d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:57.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.089+0000 7f0961443700 1 -- 172.21.15.16:0/3758686023 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f094c0092a0 con 0x7f095c1251c0 2023-09-30T22:26:57.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.089+0000 7f0962445700 1 --2- 172.21.15.16:0/3758686023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f095c125dd0 0x7f095c0669d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:26:57.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.090+0000 7f0961443700 1 --2- 172.21.15.16:0/3758686023 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f095c1251c0 0x7f095c0664e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f094c00e980 tx=0x7f094c009060).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:26:57.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.090+0000 7f0951ffb700 1 -- 172.21.15.16:0/3758686023 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c012600 con 0x7f095c1251c0 2023-09-30T22:26:57.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.090+0000 7f0951ffb700 1 -- 172.21.15.16:0/3758686023 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f094c012760 con 0x7f095c1251c0 2023-09-30T22:26:57.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.090+0000 7f0963ea8700 1 -- 172.21.15.16:0/3758686023 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0954009a20 con 0x7f095c1251c0 2023-09-30T22:26:57.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.090+0000 7f0951ffb700 1 -- 172.21.15.16:0/3758686023 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c012ae0 con 0x7f095c1251c0 2023-09-30T22:26:57.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.090+0000 7f0963ea8700 1 -- 172.21.15.16:0/3758686023 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f095c095930 con 0x7f095c1251c0 2023-09-30T22:26:57.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.091+0000 7f0951ffb700 1 -- 172.21.15.16:0/3758686023 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f094c01b400 con 0x7f095c1251c0 2023-09-30T22:26:57.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.091+0000 7f0951ffb700 1 -- 172.21.15.16:0/3758686023 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f094c03b510 con 0x7f095c1251c0 2023-09-30T22:26:57.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.091+0000 7f0963ea8700 1 -- 172.21.15.16:0/3758686023 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f095c05a260 con 0x7f095c1251c0 2023-09-30T22:26:57.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.096+0000 7f0951ffb700 1 -- 172.21.15.16:0/3758686023 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f094c016070 con 0x7f095c1251c0 2023-09-30T22:26:57.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.265+0000 7f0963ea8700 1 -- 172.21.15.16:0/3758686023 --> v1:172.21.15.16:6816/118285 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f095c046c20 con 0x7f094002bb90 2023-09-30T22:26:57.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.272+0000 7f0951ffb700 1 -- 172.21.15.16:0/3758686023 <== mgr.4098 v1:172.21.15.16:6816/118285 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+249995 (unknown 2965378022 0 1350139876) 0x7f095c046c20 con 0x7f094002bb90 2023-09-30T22:26:57.276 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:26:57.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.275+0000 7f0963ea8700 1 -- 172.21.15.16:0/3758686023 >> v1:172.21.15.16:6816/118285 conn(0x7f094002bb90 legacy=0x7f094002e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:57.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.275+0000 7f0963ea8700 1 -- 172.21.15.16:0/3758686023 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f095c1251c0 msgr2=0x7f095c0664e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:26:57.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.275+0000 7f0963ea8700 1 --2- 172.21.15.16:0/3758686023 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f095c1251c0 0x7f095c0664e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f094c00e980 tx=0x7f094c009060).stop 2023-09-30T22:26:57.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.275+0000 7f0963ea8700 1 -- 172.21.15.16:0/3758686023 shutdown_connections 2023-09-30T22:26:57.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.275+0000 7f0963ea8700 1 --2- 172.21.15.16:0/3758686023 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f095c125dd0 0x7f095c0669d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:57.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.275+0000 7f0963ea8700 1 --2- 172.21.15.16:0/3758686023 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f095c059070 0x7f095c122b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:57.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.275+0000 7f0963ea8700 1 --2- 172.21.15.16:0/3758686023 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f095c1251c0 0x7f095c0664e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:26:57.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.275+0000 7f0963ea8700 1 -- 172.21.15.16:0/3758686023 >> 172.21.15.16:0/3758686023 conn(0x7f095c097470 msgr2=0x7f095c12f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:26:57.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.275+0000 7f0963ea8700 1 -- 172.21.15.16:0/3758686023 shutdown_connections 2023-09-30T22:26:57.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:26:57.275+0000 7f0963ea8700 1 -- 172.21.15.16:0/3758686023 wait complete. 2023-09-30T22:26:57.279 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-09-30T22:26:57.289 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2023-09-30T22:26:56.455546+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":159,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1188988,"kb_used_data":604,"kb_used_omap":0,"kb_used_meta":1188352,"kb_avail":376298372,"statfs":{"total":386547056640,"available":385329532928,"internally_reserved":0,"allocated":618496,"data_stored":181498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1216872448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.718509+0000","last_change":"2023-09-30T22:26:51.708966+0000","last_active":"2023-09-30T22:26:51.718509+0000","last_peered":"2023-09-30T22:26:51.718509+0000","last_clean":"2023-09-30T22:26:51.718509+0000","last_became_active":"2023-09-30T22:26:51.708653+0000","last_became_peered":"2023-09-30T22:26:51.708653+0000","last_unstale":"2023-09-30T22:26:51.718509+0000","last_undegraded":"2023-09-30T22:26:51.718509+0000","last_fullsized":"2023-09-30T22:26:51.718509+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.729225+0000","last_change":"2023-09-30T22:26:51.727077+0000","last_active":"2023-09-30T22:26:51.729225+0000","last_peered":"2023-09-30T22:26:51.729225+0000","last_clean":"2023-09-30T22:26:51.729225+0000","last_became_active":"2023-09-30T22:26:51.726778+0000","last_became_peered":"2023-09-30T22:26:51.726778+0000","last_unstale":"2023-09-30T22:26:51.729225+0000","last_undegraded":"2023-09-30T22:26:51.729225+0000","last_fullsized":"2023-09-30T22:26:51.729225+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.714151+0000","last_change":"2023-09-30T22:26:51.706919+0000","last_active":"2023-09-30T22:26:51.714151+0000","last_peered":"2023-09-30T22:26:51.714151+0000","last_clean":"2023-09-30T22:26:51.714151+0000","last_became_active":"2023-09-30T22:26:51.706676+0000","last_became_peered":"2023-09-30T22:26:51.706676+0000","last_unstale":"2023-09-30T22:26:51.714151+0000","last_undegraded":"2023-09-30T22:26:51.714151+0000","last_fullsized":"2023-09-30T22:26:51.714151+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.728676+0000","last_change":"2023-09-30T22:26:51.727386+0000","last_active":"2023-09-30T22:26:51.728676+0000","last_peered":"2023-09-30T22:26:51.728676+0000","last_clean":"2023-09-30T22:26:51.728676+0000","last_became_active":"2023-09-30T22:26:51.727091+0000","last_became_peered":"2023-09-30T22:26:51.727091+0000","last_unstale":"2023-09-30T22:26:51.728676+0000","last_undegraded":"2023-09-30T22:26:51.728676+0000","last_fullsized":"2023-09-30T22:26:51.728676+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.723905+0000","last_change":"2023-09-30T22:26:51.719993+0000","last_active":"2023-09-30T22:26:51.723905+0000","last_peered":"2023-09-30T22:26:51.723905+0000","last_clean":"2023-09-30T22:26:51.723905+0000","last_became_active":"2023-09-30T22:26:51.719495+0000","last_became_peered":"2023-09-30T22:26:51.719495+0000","last_unstale":"2023-09-30T22:26:51.723905+0000","last_undegraded":"2023-09-30T22:26:51.723905+0000","last_fullsized":"2023-09-30T22:26:51.723905+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.715438+0000","last_change":"2023-09-30T22:26:51.709581+0000","last_active":"2023-09-30T22:26:51.715438+0000","last_peered":"2023-09-30T22:26:51.715438+0000","last_clean":"2023-09-30T22:26:51.715438+0000","last_became_active":"2023-09-30T22:26:51.709309+0000","last_became_peered":"2023-09-30T22:26:51.709309+0000","last_unstale":"2023-09-30T22:26:51.715438+0000","last_undegraded":"2023-09-30T22:26:51.715438+0000","last_fullsized":"2023-09-30T22:26:51.715438+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.713991+0000","last_change":"2023-09-30T22:26:51.705826+0000","last_active":"2023-09-30T22:26:51.713991+0000","last_peered":"2023-09-30T22:26:51.713991+0000","last_clean":"2023-09-30T22:26:51.713991+0000","last_became_active":"2023-09-30T22:26:51.705037+0000","last_became_peered":"2023-09-30T22:26:51.705037+0000","last_unstale":"2023-09-30T22:26:51.713991+0000","last_undegraded":"2023-09-30T22:26:51.713991+0000","last_fullsized":"2023-09-30T22:26:51.713991+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730836+0000","last_change":"2023-09-30T22:26:51.726640+0000","last_active":"2023-09-30T22:26:51.730836+0000","last_peered":"2023-09-30T22:26:51.730836+0000","last_clean":"2023-09-30T22:26:51.730836+0000","last_became_active":"2023-09-30T22:26:51.726278+0000","last_became_peered":"2023-09-30T22:26:51.726278+0000","last_unstale":"2023-09-30T22:26:51.730836+0000","last_undegraded":"2023-09-30T22:26:51.730836+0000","last_fullsized":"2023-09-30T22:26:51.730836+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.724179+0000","last_change":"2023-09-30T22:26:51.721817+0000","last_active":"2023-09-30T22:26:51.724179+0000","last_peered":"2023-09-30T22:26:51.724179+0000","last_clean":"2023-09-30T22:26:51.724179+0000","last_became_active":"2023-09-30T22:26:51.721373+0000","last_became_peered":"2023-09-30T22:26:51.721373+0000","last_unstale":"2023-09-30T22:26:51.724179+0000","last_undegraded":"2023-09-30T22:26:51.724179+0000","last_fullsized":"2023-09-30T22:26:51.724179+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730810+0000","last_change":"2023-09-30T22:26:51.726348+0000","last_active":"2023-09-30T22:26:51.730810+0000","last_peered":"2023-09-30T22:26:51.730810+0000","last_clean":"2023-09-30T22:26:51.730810+0000","last_became_active":"2023-09-30T22:26:51.726030+0000","last_became_peered":"2023-09-30T22:26:51.726030+0000","last_unstale":"2023-09-30T22:26:51.730810+0000","last_undegraded":"2023-09-30T22:26:51.730810+0000","last_fullsized":"2023-09-30T22:26:51.730810+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730607+0000","last_change":"2023-09-30T22:26:51.722619+0000","last_active":"2023-09-30T22:26:51.730607+0000","last_peered":"2023-09-30T22:26:51.730607+0000","last_clean":"2023-09-30T22:26:51.730607+0000","last_became_active":"2023-09-30T22:26:51.722317+0000","last_became_peered":"2023-09-30T22:26:51.722317+0000","last_unstale":"2023-09-30T22:26:51.730607+0000","last_undegraded":"2023-09-30T22:26:51.730607+0000","last_fullsized":"2023-09-30T22:26:51.730607+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.718852+0000","last_change":"2023-09-30T22:26:51.709697+0000","last_active":"2023-09-30T22:26:51.718852+0000","last_peered":"2023-09-30T22:26:51.718852+0000","last_clean":"2023-09-30T22:26:51.718852+0000","last_became_active":"2023-09-30T22:26:51.709342+0000","last_became_peered":"2023-09-30T22:26:51.709342+0000","last_unstale":"2023-09-30T22:26:51.718852+0000","last_undegraded":"2023-09-30T22:26:51.718852+0000","last_fullsized":"2023-09-30T22:26:51.718852+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.719514+0000","last_change":"2023-09-30T22:26:51.716276+0000","last_active":"2023-09-30T22:26:51.719514+0000","last_peered":"2023-09-30T22:26:51.719514+0000","last_clean":"2023-09-30T22:26:51.719514+0000","last_became_active":"2023-09-30T22:26:51.715634+0000","last_became_peered":"2023-09-30T22:26:51.715634+0000","last_unstale":"2023-09-30T22:26:51.719514+0000","last_undegraded":"2023-09-30T22:26:51.719514+0000","last_fullsized":"2023-09-30T22:26:51.719514+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.707967+0000","last_change":"2023-09-30T22:26:51.695948+0000","last_active":"2023-09-30T22:26:51.707967+0000","last_peered":"2023-09-30T22:26:51.707967+0000","last_clean":"2023-09-30T22:26:51.707967+0000","last_became_active":"2023-09-30T22:26:51.695661+0000","last_became_peered":"2023-09-30T22:26:51.695661+0000","last_unstale":"2023-09-30T22:26:51.707967+0000","last_undegraded":"2023-09-30T22:26:51.707967+0000","last_fullsized":"2023-09-30T22:26:51.707967+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.716656+0000","last_change":"2023-09-30T22:26:51.708532+0000","last_active":"2023-09-30T22:26:51.716656+0000","last_peered":"2023-09-30T22:26:51.716656+0000","last_clean":"2023-09-30T22:26:51.716656+0000","last_became_active":"2023-09-30T22:26:51.708204+0000","last_became_peered":"2023-09-30T22:26:51.708204+0000","last_unstale":"2023-09-30T22:26:51.716656+0000","last_undegraded":"2023-09-30T22:26:51.716656+0000","last_fullsized":"2023-09-30T22:26:51.716656+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.714211+0000","last_change":"2023-09-30T22:26:51.704597+0000","last_active":"2023-09-30T22:26:51.714211+0000","last_peered":"2023-09-30T22:26:51.714211+0000","last_clean":"2023-09-30T22:26:51.714211+0000","last_became_active":"2023-09-30T22:26:51.704307+0000","last_became_peered":"2023-09-30T22:26:51.704307+0000","last_unstale":"2023-09-30T22:26:51.714211+0000","last_undegraded":"2023-09-30T22:26:51.714211+0000","last_fullsized":"2023-09-30T22:26:51.714211+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.724159+0000","last_change":"2023-09-30T22:26:51.721640+0000","last_active":"2023-09-30T22:26:51.724159+0000","last_peered":"2023-09-30T22:26:51.724159+0000","last_clean":"2023-09-30T22:26:51.724159+0000","last_became_active":"2023-09-30T22:26:51.720950+0000","last_became_peered":"2023-09-30T22:26:51.720950+0000","last_unstale":"2023-09-30T22:26:51.724159+0000","last_undegraded":"2023-09-30T22:26:51.724159+0000","last_fullsized":"2023-09-30T22:26:51.724159+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.707139+0000","last_change":"2023-09-30T22:26:51.697513+0000","last_active":"2023-09-30T22:26:51.707139+0000","last_peered":"2023-09-30T22:26:51.707139+0000","last_clean":"2023-09-30T22:26:51.707139+0000","last_became_active":"2023-09-30T22:26:51.697137+0000","last_became_peered":"2023-09-30T22:26:51.697137+0000","last_unstale":"2023-09-30T22:26:51.707139+0000","last_undegraded":"2023-09-30T22:26:51.707139+0000","last_fullsized":"2023-09-30T22:26:51.707139+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.718540+0000","last_change":"2023-09-30T22:26:51.715476+0000","last_active":"2023-09-30T22:26:51.718540+0000","last_peered":"2023-09-30T22:26:51.718540+0000","last_clean":"2023-09-30T22:26:51.718540+0000","last_became_active":"2023-09-30T22:26:51.715176+0000","last_became_peered":"2023-09-30T22:26:51.715176+0000","last_unstale":"2023-09-30T22:26:51.718540+0000","last_undegraded":"2023-09-30T22:26:51.718540+0000","last_fullsized":"2023-09-30T22:26:51.718540+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730693+0000","last_change":"2023-09-30T22:26:51.724991+0000","last_active":"2023-09-30T22:26:51.730693+0000","last_peered":"2023-09-30T22:26:51.730693+0000","last_clean":"2023-09-30T22:26:51.730693+0000","last_became_active":"2023-09-30T22:26:51.724240+0000","last_became_peered":"2023-09-30T22:26:51.724240+0000","last_unstale":"2023-09-30T22:26:51.730693+0000","last_undegraded":"2023-09-30T22:26:51.730693+0000","last_fullsized":"2023-09-30T22:26:51.730693+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730391+0000","last_change":"2023-09-30T22:26:51.717833+0000","last_active":"2023-09-30T22:26:51.730391+0000","last_peered":"2023-09-30T22:26:51.730391+0000","last_clean":"2023-09-30T22:26:51.730391+0000","last_became_active":"2023-09-30T22:26:51.717517+0000","last_became_peered":"2023-09-30T22:26:51.717517+0000","last_unstale":"2023-09-30T22:26:51.730391+0000","last_undegraded":"2023-09-30T22:26:51.730391+0000","last_fullsized":"2023-09-30T22:26:51.730391+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.723937+0000","last_change":"2023-09-30T22:26:51.719030+0000","last_active":"2023-09-30T22:26:51.723937+0000","last_peered":"2023-09-30T22:26:51.723937+0000","last_clean":"2023-09-30T22:26:51.723937+0000","last_became_active":"2023-09-30T22:26:51.715370+0000","last_became_peered":"2023-09-30T22:26:51.715370+0000","last_unstale":"2023-09-30T22:26:51.723937+0000","last_undegraded":"2023-09-30T22:26:51.723937+0000","last_fullsized":"2023-09-30T22:26:51.723937+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.713970+0000","last_change":"2023-09-30T22:26:51.698664+0000","last_active":"2023-09-30T22:26:51.713970+0000","last_peered":"2023-09-30T22:26:51.713970+0000","last_clean":"2023-09-30T22:26:51.713970+0000","last_became_active":"2023-09-30T22:26:51.698321+0000","last_became_peered":"2023-09-30T22:26:51.698321+0000","last_unstale":"2023-09-30T22:26:51.713970+0000","last_undegraded":"2023-09-30T22:26:51.713970+0000","last_fullsized":"2023-09-30T22:26:51.713970+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730972+0000","last_change":"2023-09-30T22:26:51.727955+0000","last_active":"2023-09-30T22:26:51.730972+0000","last_peered":"2023-09-30T22:26:51.730972+0000","last_clean":"2023-09-30T22:26:51.730972+0000","last_became_active":"2023-09-30T22:26:51.727209+0000","last_became_peered":"2023-09-30T22:26:51.727209+0000","last_unstale":"2023-09-30T22:26:51.730972+0000","last_undegraded":"2023-09-30T22:26:51.730972+0000","last_fullsized":"2023-09-30T22:26:51.730972+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730499+0000","last_change":"2023-09-30T22:26:51.721154+0000","last_active":"2023-09-30T22:26:51.730499+0000","last_peered":"2023-09-30T22:26:51.730499+0000","last_clean":"2023-09-30T22:26:51.730499+0000","last_became_active":"2023-09-30T22:26:51.720703+0000","last_became_peered":"2023-09-30T22:26:51.720703+0000","last_unstale":"2023-09-30T22:26:51.730499+0000","last_undegraded":"2023-09-30T22:26:51.730499+0000","last_fullsized":"2023-09-30T22:26:51.730499+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.725830+0000","last_change":"2023-09-30T22:26:51.701309+0000","last_active":"2023-09-30T22:26:51.725830+0000","last_peered":"2023-09-30T22:26:51.725830+0000","last_clean":"2023-09-30T22:26:51.725830+0000","last_became_active":"2023-09-30T22:26:51.701012+0000","last_became_peered":"2023-09-30T22:26:51.701012+0000","last_unstale":"2023-09-30T22:26:51.725830+0000","last_undegraded":"2023-09-30T22:26:51.725830+0000","last_fullsized":"2023-09-30T22:26:51.725830+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.718423+0000","last_change":"2023-09-30T22:26:51.715042+0000","last_active":"2023-09-30T22:26:51.718423+0000","last_peered":"2023-09-30T22:26:51.718423+0000","last_clean":"2023-09-30T22:26:51.718423+0000","last_became_active":"2023-09-30T22:26:51.712158+0000","last_became_peered":"2023-09-30T22:26:51.712158+0000","last_unstale":"2023-09-30T22:26:51.718423+0000","last_undegraded":"2023-09-30T22:26:51.718423+0000","last_fullsized":"2023-09-30T22:26:51.718423+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"15'1","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.713320+0000","last_change":"2023-09-30T22:26:51.296397+0000","last_active":"2023-09-30T22:26:51.713320+0000","last_peered":"2023-09-30T22:26:51.713320+0000","last_clean":"2023-09-30T22:26:51.713320+0000","last_became_active":"2023-09-30T22:26:51.295860+0000","last_became_peered":"2023-09-30T22:26:51.295860+0000","last_unstale":"2023-09-30T22:26:51.713320+0000","last_undegraded":"2023-09-30T22:26:51.713320+0000","last_fullsized":"2023-09-30T22:26:51.713320+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.716849+0000","last_change":"2023-09-30T22:26:51.712472+0000","last_active":"2023-09-30T22:26:51.716849+0000","last_peered":"2023-09-30T22:26:51.716849+0000","last_clean":"2023-09-30T22:26:51.716849+0000","last_became_active":"2023-09-30T22:26:51.711527+0000","last_became_peered":"2023-09-30T22:26:51.711527+0000","last_unstale":"2023-09-30T22:26:51.716849+0000","last_undegraded":"2023-09-30T22:26:51.716849+0000","last_fullsized":"2023-09-30T22:26:51.716849+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.701600+0000","last_change":"2023-09-30T22:26:51.298004+0000","last_active":"2023-09-30T22:26:51.701600+0000","last_peered":"2023-09-30T22:26:51.701600+0000","last_clean":"2023-09-30T22:26:51.701600+0000","last_became_active":"2023-09-30T22:26:51.297714+0000","last_became_peered":"2023-09-30T22:26:51.297714+0000","last_unstale":"2023-09-30T22:26:51.701600+0000","last_undegraded":"2023-09-30T22:26:51.701600+0000","last_fullsized":"2023-09-30T22:26:51.701600+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.729456+0000","last_change":"2023-09-30T22:26:51.727608+0000","last_active":"2023-09-30T22:26:51.729456+0000","last_peered":"2023-09-30T22:26:51.729456+0000","last_clean":"2023-09-30T22:26:51.729456+0000","last_became_active":"2023-09-30T22:26:51.727154+0000","last_became_peered":"2023-09-30T22:26:51.727154+0000","last_unstale":"2023-09-30T22:26:51.729456+0000","last_undegraded":"2023-09-30T22:26:51.729456+0000","last_fullsized":"2023-09-30T22:26:51.729456+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.684923+0000","last_change":"2023-09-30T22:26:51.285214+0000","last_active":"2023-09-30T22:26:51.684923+0000","last_peered":"2023-09-30T22:26:51.684923+0000","last_clean":"2023-09-30T22:26:51.684923+0000","last_became_active":"2023-09-30T22:26:51.284749+0000","last_became_peered":"2023-09-30T22:26:51.284749+0000","last_unstale":"2023-09-30T22:26:51.684923+0000","last_undegraded":"2023-09-30T22:26:51.684923+0000","last_fullsized":"2023-09-30T22:26:51.684923+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.716115+0000","last_change":"2023-09-30T22:26:51.706697+0000","last_active":"2023-09-30T22:26:51.716115+0000","last_peered":"2023-09-30T22:26:51.716115+0000","last_clean":"2023-09-30T22:26:51.716115+0000","last_became_active":"2023-09-30T22:26:51.706373+0000","last_became_peered":"2023-09-30T22:26:51.706373+0000","last_unstale":"2023-09-30T22:26:51.716115+0000","last_undegraded":"2023-09-30T22:26:51.716115+0000","last_fullsized":"2023-09-30T22:26:51.716115+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.689119+0000","last_change":"2023-09-30T22:26:51.281642+0000","last_active":"2023-09-30T22:26:51.689119+0000","last_peered":"2023-09-30T22:26:51.689119+0000","last_clean":"2023-09-30T22:26:51.689119+0000","last_became_active":"2023-09-30T22:26:51.281319+0000","last_became_peered":"2023-09-30T22:26:51.281319+0000","last_unstale":"2023-09-30T22:26:51.689119+0000","last_undegraded":"2023-09-30T22:26:51.689119+0000","last_fullsized":"2023-09-30T22:26:51.689119+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.714232+0000","last_change":"2023-09-30T22:26:51.705192+0000","last_active":"2023-09-30T22:26:51.714232+0000","last_peered":"2023-09-30T22:26:51.714232+0000","last_clean":"2023-09-30T22:26:51.714232+0000","last_became_active":"2023-09-30T22:26:51.704870+0000","last_became_peered":"2023-09-30T22:26:51.704870+0000","last_unstale":"2023-09-30T22:26:51.714232+0000","last_undegraded":"2023-09-30T22:26:51.714232+0000","last_fullsized":"2023-09-30T22:26:51.714232+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.680797+0000","last_change":"2023-09-30T22:26:51.293711+0000","last_active":"2023-09-30T22:26:51.680797+0000","last_peered":"2023-09-30T22:26:51.680797+0000","last_clean":"2023-09-30T22:26:51.680797+0000","last_became_active":"2023-09-30T22:26:51.293410+0000","last_became_peered":"2023-09-30T22:26:51.293410+0000","last_unstale":"2023-09-30T22:26:51.680797+0000","last_undegraded":"2023-09-30T22:26:51.680797+0000","last_fullsized":"2023-09-30T22:26:51.680797+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.714029+0000","last_change":"2023-09-30T22:26:51.706005+0000","last_active":"2023-09-30T22:26:51.714029+0000","last_peered":"2023-09-30T22:26:51.714029+0000","last_clean":"2023-09-30T22:26:51.714029+0000","last_became_active":"2023-09-30T22:26:51.705491+0000","last_became_peered":"2023-09-30T22:26:51.705491+0000","last_unstale":"2023-09-30T22:26:51.714029+0000","last_undegraded":"2023-09-30T22:26:51.714029+0000","last_fullsized":"2023-09-30T22:26:51.714029+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"15'3","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.719469+0000","last_change":"2023-09-30T22:26:51.279976+0000","last_active":"2023-09-30T22:26:51.719469+0000","last_peered":"2023-09-30T22:26:51.719469+0000","last_clean":"2023-09-30T22:26:51.719469+0000","last_became_active":"2023-09-30T22:26:51.279669+0000","last_became_peered":"2023-09-30T22:26:51.279669+0000","last_unstale":"2023-09-30T22:26:51.719469+0000","last_undegraded":"2023-09-30T22:26:51.719469+0000","last_fullsized":"2023-09-30T22:26:51.719469+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730714+0000","last_change":"2023-09-30T22:26:51.725947+0000","last_active":"2023-09-30T22:26:51.730714+0000","last_peered":"2023-09-30T22:26:51.730714+0000","last_clean":"2023-09-30T22:26:51.730714+0000","last_became_active":"2023-09-30T22:26:51.723294+0000","last_became_peered":"2023-09-30T22:26:51.723294+0000","last_unstale":"2023-09-30T22:26:51.730714+0000","last_undegraded":"2023-09-30T22:26:51.730714+0000","last_fullsized":"2023-09-30T22:26:51.730714+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"15'6","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.723849+0000","last_change":"2023-09-30T22:26:51.297527+0000","last_active":"2023-09-30T22:26:51.723849+0000","last_peered":"2023-09-30T22:26:51.723849+0000","last_clean":"2023-09-30T22:26:51.723849+0000","last_became_active":"2023-09-30T22:26:51.297246+0000","last_became_peered":"2023-09-30T22:26:51.297246+0000","last_unstale":"2023-09-30T22:26:51.723849+0000","last_undegraded":"2023-09-30T22:26:51.723849+0000","last_fullsized":"2023-09-30T22:26:51.723849+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.718810+0000","last_change":"2023-09-30T22:26:51.710781+0000","last_active":"2023-09-30T22:26:51.718810+0000","last_peered":"2023-09-30T22:26:51.718810+0000","last_clean":"2023-09-30T22:26:51.718810+0000","last_became_active":"2023-09-30T22:26:51.710107+0000","last_became_peered":"2023-09-30T22:26:51.710107+0000","last_unstale":"2023-09-30T22:26:51.718810+0000","last_undegraded":"2023-09-30T22:26:51.718810+0000","last_fullsized":"2023-09-30T22:26:51.718810+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"15'2","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.714029+0000","last_change":"2023-09-30T22:26:51.290194+0000","last_active":"2023-09-30T22:26:51.714029+0000","last_peered":"2023-09-30T22:26:51.714029+0000","last_clean":"2023-09-30T22:26:51.714029+0000","last_became_active":"2023-09-30T22:26:51.288688+0000","last_became_peered":"2023-09-30T22:26:51.288688+0000","last_unstale":"2023-09-30T22:26:51.714029+0000","last_undegraded":"2023-09-30T22:26:51.714029+0000","last_fullsized":"2023-09-30T22:26:51.714029+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.728819+0000","last_change":"2023-09-30T22:26:51.726780+0000","last_active":"2023-09-30T22:26:51.728819+0000","last_peered":"2023-09-30T22:26:51.728819+0000","last_clean":"2023-09-30T22:26:51.728819+0000","last_became_active":"2023-09-30T22:26:51.726408+0000","last_became_peered":"2023-09-30T22:26:51.726408+0000","last_unstale":"2023-09-30T22:26:51.728819+0000","last_undegraded":"2023-09-30T22:26:51.728819+0000","last_fullsized":"2023-09-30T22:26:51.728819+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.685836+0000","last_change":"2023-09-30T22:26:51.281127+0000","last_active":"2023-09-30T22:26:51.685836+0000","last_peered":"2023-09-30T22:26:51.685836+0000","last_clean":"2023-09-30T22:26:51.685836+0000","last_became_active":"2023-09-30T22:26:51.280507+0000","last_became_peered":"2023-09-30T22:26:51.280507+0000","last_unstale":"2023-09-30T22:26:51.685836+0000","last_undegraded":"2023-09-30T22:26:51.685836+0000","last_fullsized":"2023-09-30T22:26:51.685836+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.716876+0000","last_change":"2023-09-30T22:26:51.712671+0000","last_active":"2023-09-30T22:26:51.716876+0000","last_peered":"2023-09-30T22:26:51.716876+0000","last_clean":"2023-09-30T22:26:51.716876+0000","last_became_active":"2023-09-30T22:26:51.711836+0000","last_became_peered":"2023-09-30T22:26:51.711836+0000","last_unstale":"2023-09-30T22:26:51.716876+0000","last_undegraded":"2023-09-30T22:26:51.716876+0000","last_fullsized":"2023-09-30T22:26:51.716876+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.683587+0000","last_change":"2023-09-30T22:26:51.297467+0000","last_active":"2023-09-30T22:26:51.683587+0000","last_peered":"2023-09-30T22:26:51.683587+0000","last_clean":"2023-09-30T22:26:51.683587+0000","last_became_active":"2023-09-30T22:26:51.296950+0000","last_became_peered":"2023-09-30T22:26:51.296950+0000","last_unstale":"2023-09-30T22:26:51.683587+0000","last_undegraded":"2023-09-30T22:26:51.683587+0000","last_fullsized":"2023-09-30T22:26:51.683587+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.724064+0000","last_change":"2023-09-30T22:26:51.719411+0000","last_active":"2023-09-30T22:26:51.724064+0000","last_peered":"2023-09-30T22:26:51.724064+0000","last_clean":"2023-09-30T22:26:51.724064+0000","last_became_active":"2023-09-30T22:26:51.719111+0000","last_became_peered":"2023-09-30T22:26:51.719111+0000","last_unstale":"2023-09-30T22:26:51.724064+0000","last_undegraded":"2023-09-30T22:26:51.724064+0000","last_fullsized":"2023-09-30T22:26:51.724064+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"15'8","reported_seq":18,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.727696+0000","last_change":"2023-09-30T22:26:51.287702+0000","last_active":"2023-09-30T22:26:51.727696+0000","last_peered":"2023-09-30T22:26:51.727696+0000","last_clean":"2023-09-30T22:26:51.727696+0000","last_became_active":"2023-09-30T22:26:51.287434+0000","last_became_peered":"2023-09-30T22:26:51.287434+0000","last_unstale":"2023-09-30T22:26:51.727696+0000","last_undegraded":"2023-09-30T22:26:51.727696+0000","last_fullsized":"2023-09-30T22:26:51.727696+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.716138+0000","last_change":"2023-09-30T22:26:51.706663+0000","last_active":"2023-09-30T22:26:51.716138+0000","last_peered":"2023-09-30T22:26:51.716138+0000","last_clean":"2023-09-30T22:26:51.716138+0000","last_became_active":"2023-09-30T22:26:51.706363+0000","last_became_peered":"2023-09-30T22:26:51.706363+0000","last_unstale":"2023-09-30T22:26:51.716138+0000","last_undegraded":"2023-09-30T22:26:51.716138+0000","last_fullsized":"2023-09-30T22:26:51.716138+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"15'1","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.721973+0000","last_change":"2023-09-30T22:26:51.291996+0000","last_active":"2023-09-30T22:26:51.721973+0000","last_peered":"2023-09-30T22:26:51.721973+0000","last_clean":"2023-09-30T22:26:51.721973+0000","last_became_active":"2023-09-30T22:26:51.291656+0000","last_became_peered":"2023-09-30T22:26:51.291656+0000","last_unstale":"2023-09-30T22:26:51.721973+0000","last_undegraded":"2023-09-30T22:26:51.721973+0000","last_fullsized":"2023-09-30T22:26:51.721973+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.715557+0000","last_change":"2023-09-30T22:26:51.710326+0000","last_active":"2023-09-30T22:26:51.715557+0000","last_peered":"2023-09-30T22:26:51.715557+0000","last_clean":"2023-09-30T22:26:51.715557+0000","last_became_active":"2023-09-30T22:26:51.708793+0000","last_became_peered":"2023-09-30T22:26:51.708793+0000","last_unstale":"2023-09-30T22:26:51.715557+0000","last_undegraded":"2023-09-30T22:26:51.715557+0000","last_fullsized":"2023-09-30T22:26:51.715557+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"15'2","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.699110+0000","last_change":"2023-09-30T22:26:51.300700+0000","last_active":"2023-09-30T22:26:51.699110+0000","last_peered":"2023-09-30T22:26:51.699110+0000","last_clean":"2023-09-30T22:26:51.699110+0000","last_became_active":"2023-09-30T22:26:51.300416+0000","last_became_peered":"2023-09-30T22:26:51.300416+0000","last_unstale":"2023-09-30T22:26:51.699110+0000","last_undegraded":"2023-09-30T22:26:51.699110+0000","last_fullsized":"2023-09-30T22:26:51.699110+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730623+0000","last_change":"2023-09-30T22:26:51.722902+0000","last_active":"2023-09-30T22:26:51.730623+0000","last_peered":"2023-09-30T22:26:51.730623+0000","last_clean":"2023-09-30T22:26:51.730623+0000","last_became_active":"2023-09-30T22:26:51.717919+0000","last_became_peered":"2023-09-30T22:26:51.717919+0000","last_unstale":"2023-09-30T22:26:51.730623+0000","last_undegraded":"2023-09-30T22:26:51.730623+0000","last_fullsized":"2023-09-30T22:26:51.730623+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"15'1","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.716626+0000","last_change":"2023-09-30T22:26:51.292120+0000","last_active":"2023-09-30T22:26:51.716626+0000","last_peered":"2023-09-30T22:26:51.716626+0000","last_clean":"2023-09-30T22:26:51.716626+0000","last_became_active":"2023-09-30T22:26:51.289469+0000","last_became_peered":"2023-09-30T22:26:51.289469+0000","last_unstale":"2023-09-30T22:26:51.716626+0000","last_undegraded":"2023-09-30T22:26:51.716626+0000","last_fullsized":"2023-09-30T22:26:51.716626+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.718700+0000","last_change":"2023-09-30T22:26:51.711649+0000","last_active":"2023-09-30T22:26:51.718700+0000","last_peered":"2023-09-30T22:26:51.718700+0000","last_clean":"2023-09-30T22:26:51.718700+0000","last_became_active":"2023-09-30T22:26:51.711384+0000","last_became_peered":"2023-09-30T22:26:51.711384+0000","last_unstale":"2023-09-30T22:26:51.718700+0000","last_undegraded":"2023-09-30T22:26:51.718700+0000","last_fullsized":"2023-09-30T22:26:51.718700+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.686889+0000","last_change":"2023-09-30T22:26:51.281627+0000","last_active":"2023-09-30T22:26:51.686889+0000","last_peered":"2023-09-30T22:26:51.686889+0000","last_clean":"2023-09-30T22:26:51.686889+0000","last_became_active":"2023-09-30T22:26:51.281311+0000","last_became_peered":"2023-09-30T22:26:51.281311+0000","last_unstale":"2023-09-30T22:26:51.686889+0000","last_undegraded":"2023-09-30T22:26:51.686889+0000","last_fullsized":"2023-09-30T22:26:51.686889+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.707659+0000","last_change":"2023-09-30T22:26:51.700837+0000","last_active":"2023-09-30T22:26:51.707659+0000","last_peered":"2023-09-30T22:26:51.707659+0000","last_clean":"2023-09-30T22:26:51.707659+0000","last_became_active":"2023-09-30T22:26:51.700523+0000","last_became_peered":"2023-09-30T22:26:51.700523+0000","last_unstale":"2023-09-30T22:26:51.707659+0000","last_undegraded":"2023-09-30T22:26:51.707659+0000","last_fullsized":"2023-09-30T22:26:51.707659+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"15'2","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.721892+0000","last_change":"2023-09-30T22:26:51.290248+0000","last_active":"2023-09-30T22:26:51.721892+0000","last_peered":"2023-09-30T22:26:51.721892+0000","last_clean":"2023-09-30T22:26:51.721892+0000","last_became_active":"2023-09-30T22:26:51.289168+0000","last_became_peered":"2023-09-30T22:26:51.289168+0000","last_unstale":"2023-09-30T22:26:51.721892+0000","last_undegraded":"2023-09-30T22:26:51.721892+0000","last_fullsized":"2023-09-30T22:26:51.721892+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.723965+0000","last_change":"2023-09-30T22:26:51.719224+0000","last_active":"2023-09-30T22:26:51.723965+0000","last_peered":"2023-09-30T22:26:51.723965+0000","last_clean":"2023-09-30T22:26:51.723965+0000","last_became_active":"2023-09-30T22:26:51.718890+0000","last_became_peered":"2023-09-30T22:26:51.718890+0000","last_unstale":"2023-09-30T22:26:51.723965+0000","last_undegraded":"2023-09-30T22:26:51.723965+0000","last_fullsized":"2023-09-30T22:26:51.723965+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.687008+0000","last_change":"2023-09-30T22:26:51.288204+0000","last_active":"2023-09-30T22:26:51.687008+0000","last_peered":"2023-09-30T22:26:51.687008+0000","last_clean":"2023-09-30T22:26:51.687008+0000","last_became_active":"2023-09-30T22:26:51.287903+0000","last_became_peered":"2023-09-30T22:26:51.287903+0000","last_unstale":"2023-09-30T22:26:51.687008+0000","last_undegraded":"2023-09-30T22:26:51.687008+0000","last_fullsized":"2023-09-30T22:26:51.687008+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.718399+0000","last_change":"2023-09-30T22:26:51.713792+0000","last_active":"2023-09-30T22:26:51.718399+0000","last_peered":"2023-09-30T22:26:51.718399+0000","last_clean":"2023-09-30T22:26:51.718399+0000","last_became_active":"2023-09-30T22:26:51.713334+0000","last_became_peered":"2023-09-30T22:26:51.713334+0000","last_unstale":"2023-09-30T22:26:51.718399+0000","last_undegraded":"2023-09-30T22:26:51.718399+0000","last_fullsized":"2023-09-30T22:26:51.718399+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.683106+0000","last_change":"2023-09-30T22:26:51.297240+0000","last_active":"2023-09-30T22:26:51.683106+0000","last_peered":"2023-09-30T22:26:51.683106+0000","last_clean":"2023-09-30T22:26:51.683106+0000","last_became_active":"2023-09-30T22:26:51.296547+0000","last_became_peered":"2023-09-30T22:26:51.296547+0000","last_unstale":"2023-09-30T22:26:51.683106+0000","last_undegraded":"2023-09-30T22:26:51.683106+0000","last_fullsized":"2023-09-30T22:26:51.683106+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.718663+0000","last_change":"2023-09-30T22:26:51.711889+0000","last_active":"2023-09-30T22:26:51.718663+0000","last_peered":"2023-09-30T22:26:51.718663+0000","last_clean":"2023-09-30T22:26:51.718663+0000","last_became_active":"2023-09-30T22:26:51.711570+0000","last_became_peered":"2023-09-30T22:26:51.711570+0000","last_unstale":"2023-09-30T22:26:51.718663+0000","last_undegraded":"2023-09-30T22:26:51.718663+0000","last_fullsized":"2023-09-30T22:26:51.718663+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.684383+0000","last_change":"2023-09-30T22:26:51.290699+0000","last_active":"2023-09-30T22:26:51.684383+0000","last_peered":"2023-09-30T22:26:51.684383+0000","last_clean":"2023-09-30T22:26:51.684383+0000","last_became_active":"2023-09-30T22:26:51.290396+0000","last_became_peered":"2023-09-30T22:26:51.290396+0000","last_unstale":"2023-09-30T22:26:51.684383+0000","last_undegraded":"2023-09-30T22:26:51.684383+0000","last_fullsized":"2023-09-30T22:26:51.684383+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730474+0000","last_change":"2023-09-30T22:26:51.720976+0000","last_active":"2023-09-30T22:26:51.730474+0000","last_peered":"2023-09-30T22:26:51.730474+0000","last_clean":"2023-09-30T22:26:51.730474+0000","last_became_active":"2023-09-30T22:26:51.720378+0000","last_became_peered":"2023-09-30T22:26:51.720378+0000","last_unstale":"2023-09-30T22:26:51.730474+0000","last_undegraded":"2023-09-30T22:26:51.730474+0000","last_fullsized":"2023-09-30T22:26:51.730474+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.702412+0000","last_change":"2023-09-30T22:26:46.273891+0000","last_active":"2023-09-30T22:26:51.702412+0000","last_peered":"2023-09-30T22:26:51.702412+0000","last_clean":"2023-09-30T22:26:51.702412+0000","last_became_active":"2023-09-30T22:26:46.273588+0000","last_became_peered":"2023-09-30T22:26:46.273588+0000","last_unstale":"2023-09-30T22:26:51.702412+0000","last_undegraded":"2023-09-30T22:26:51.702412+0000","last_fullsized":"2023-09-30T22:26:51.702412+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.680672+0000","last_change":"2023-09-30T22:26:51.294682+0000","last_active":"2023-09-30T22:26:51.680672+0000","last_peered":"2023-09-30T22:26:51.680672+0000","last_clean":"2023-09-30T22:26:51.680672+0000","last_became_active":"2023-09-30T22:26:51.294422+0000","last_became_peered":"2023-09-30T22:26:51.294422+0000","last_unstale":"2023-09-30T22:26:51.680672+0000","last_undegraded":"2023-09-30T22:26:51.680672+0000","last_fullsized":"2023-09-30T22:26:51.680672+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730874+0000","last_change":"2023-09-30T22:26:51.726954+0000","last_active":"2023-09-30T22:26:51.730874+0000","last_peered":"2023-09-30T22:26:51.730874+0000","last_clean":"2023-09-30T22:26:51.730874+0000","last_became_active":"2023-09-30T22:26:51.726641+0000","last_became_peered":"2023-09-30T22:26:51.726641+0000","last_unstale":"2023-09-30T22:26:51.730874+0000","last_undegraded":"2023-09-30T22:26:51.730874+0000","last_fullsized":"2023-09-30T22:26:51.730874+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.687452+0000","last_change":"2023-09-30T22:26:46.245946+0000","last_active":"2023-09-30T22:26:51.687452+0000","last_peered":"2023-09-30T22:26:51.687452+0000","last_clean":"2023-09-30T22:26:51.687452+0000","last_became_active":"2023-09-30T22:26:46.245602+0000","last_became_peered":"2023-09-30T22:26:46.245602+0000","last_unstale":"2023-09-30T22:26:51.687452+0000","last_undegraded":"2023-09-30T22:26:51.687452+0000","last_fullsized":"2023-09-30T22:26:51.687452+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.685668+0000","last_change":"2023-09-30T22:26:51.294691+0000","last_active":"2023-09-30T22:26:51.685668+0000","last_peered":"2023-09-30T22:26:51.685668+0000","last_clean":"2023-09-30T22:26:51.685668+0000","last_became_active":"2023-09-30T22:26:51.294411+0000","last_became_peered":"2023-09-30T22:26:51.294411+0000","last_unstale":"2023-09-30T22:26:51.685668+0000","last_undegraded":"2023-09-30T22:26:51.685668+0000","last_fullsized":"2023-09-30T22:26:51.685668+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730680+0000","last_change":"2023-09-30T22:26:51.724759+0000","last_active":"2023-09-30T22:26:51.730680+0000","last_peered":"2023-09-30T22:26:51.730680+0000","last_clean":"2023-09-30T22:26:51.730680+0000","last_became_active":"2023-09-30T22:26:51.723221+0000","last_became_peered":"2023-09-30T22:26:51.723221+0000","last_unstale":"2023-09-30T22:26:51.730680+0000","last_undegraded":"2023-09-30T22:26:51.730680+0000","last_fullsized":"2023-09-30T22:26:51.730680+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.681986+0000","last_change":"2023-09-30T22:26:46.242607+0000","last_active":"2023-09-30T22:26:51.681986+0000","last_peered":"2023-09-30T22:26:51.681986+0000","last_clean":"2023-09-30T22:26:51.681986+0000","last_became_active":"2023-09-30T22:26:46.242283+0000","last_became_peered":"2023-09-30T22:26:46.242283+0000","last_unstale":"2023-09-30T22:26:51.681986+0000","last_undegraded":"2023-09-30T22:26:51.681986+0000","last_fullsized":"2023-09-30T22:26:51.681986+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.703666+0000","last_change":"2023-09-30T22:26:51.295409+0000","last_active":"2023-09-30T22:26:51.703666+0000","last_peered":"2023-09-30T22:26:51.703666+0000","last_clean":"2023-09-30T22:26:51.703666+0000","last_became_active":"2023-09-30T22:26:51.295140+0000","last_became_peered":"2023-09-30T22:26:51.295140+0000","last_unstale":"2023-09-30T22:26:51.703666+0000","last_undegraded":"2023-09-30T22:26:51.703666+0000","last_fullsized":"2023-09-30T22:26:51.703666+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730344+0000","last_change":"2023-09-30T22:26:51.715504+0000","last_active":"2023-09-30T22:26:51.730344+0000","last_peered":"2023-09-30T22:26:51.730344+0000","last_clean":"2023-09-30T22:26:51.730344+0000","last_became_active":"2023-09-30T22:26:51.715258+0000","last_became_peered":"2023-09-30T22:26:51.715258+0000","last_unstale":"2023-09-30T22:26:51.730344+0000","last_undegraded":"2023-09-30T22:26:51.730344+0000","last_fullsized":"2023-09-30T22:26:51.730344+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.682511+0000","last_change":"2023-09-30T22:26:46.242752+0000","last_active":"2023-09-30T22:26:51.682511+0000","last_peered":"2023-09-30T22:26:51.682511+0000","last_clean":"2023-09-30T22:26:51.682511+0000","last_became_active":"2023-09-30T22:26:46.242385+0000","last_became_peered":"2023-09-30T22:26:46.242385+0000","last_unstale":"2023-09-30T22:26:51.682511+0000","last_undegraded":"2023-09-30T22:26:51.682511+0000","last_fullsized":"2023-09-30T22:26:51.682511+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.703565+0000","last_change":"2023-09-30T22:26:46.273930+0000","last_active":"2023-09-30T22:26:51.703565+0000","last_peered":"2023-09-30T22:26:51.703565+0000","last_clean":"2023-09-30T22:26:51.703565+0000","last_became_active":"2023-09-30T22:26:46.273621+0000","last_became_peered":"2023-09-30T22:26:46.273621+0000","last_unstale":"2023-09-30T22:26:51.703565+0000","last_undegraded":"2023-09-30T22:26:51.703565+0000","last_fullsized":"2023-09-30T22:26:51.703565+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.686892+0000","last_change":"2023-09-30T22:26:51.298163+0000","last_active":"2023-09-30T22:26:51.686892+0000","last_peered":"2023-09-30T22:26:51.686892+0000","last_clean":"2023-09-30T22:26:51.686892+0000","last_became_active":"2023-09-30T22:26:51.297897+0000","last_became_peered":"2023-09-30T22:26:51.297897+0000","last_unstale":"2023-09-30T22:26:51.686892+0000","last_undegraded":"2023-09-30T22:26:51.686892+0000","last_fullsized":"2023-09-30T22:26:51.686892+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.718377+0000","last_change":"2023-09-30T22:26:51.713608+0000","last_active":"2023-09-30T22:26:51.718377+0000","last_peered":"2023-09-30T22:26:51.718377+0000","last_clean":"2023-09-30T22:26:51.718377+0000","last_became_active":"2023-09-30T22:26:51.712992+0000","last_became_peered":"2023-09-30T22:26:51.712992+0000","last_unstale":"2023-09-30T22:26:51.718377+0000","last_undegraded":"2023-09-30T22:26:51.718377+0000","last_fullsized":"2023-09-30T22:26:51.718377+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.684824+0000","last_change":"2023-09-30T22:26:46.245668+0000","last_active":"2023-09-30T22:26:51.684824+0000","last_peered":"2023-09-30T22:26:51.684824+0000","last_clean":"2023-09-30T22:26:51.684824+0000","last_became_active":"2023-09-30T22:26:46.245319+0000","last_became_peered":"2023-09-30T22:26:46.245319+0000","last_unstale":"2023-09-30T22:26:51.684824+0000","last_undegraded":"2023-09-30T22:26:51.684824+0000","last_fullsized":"2023-09-30T22:26:51.684824+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.681181+0000","last_change":"2023-09-30T22:26:51.294664+0000","last_active":"2023-09-30T22:26:51.681181+0000","last_peered":"2023-09-30T22:26:51.681181+0000","last_clean":"2023-09-30T22:26:51.681181+0000","last_became_active":"2023-09-30T22:26:51.294357+0000","last_became_peered":"2023-09-30T22:26:51.294357+0000","last_unstale":"2023-09-30T22:26:51.681181+0000","last_undegraded":"2023-09-30T22:26:51.681181+0000","last_fullsized":"2023-09-30T22:26:51.681181+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.707651+0000","last_change":"2023-09-30T22:26:51.705784+0000","last_active":"2023-09-30T22:26:51.707651+0000","last_peered":"2023-09-30T22:26:51.707651+0000","last_clean":"2023-09-30T22:26:51.707651+0000","last_became_active":"2023-09-30T22:26:51.705480+0000","last_became_peered":"2023-09-30T22:26:51.705480+0000","last_unstale":"2023-09-30T22:26:51.707651+0000","last_undegraded":"2023-09-30T22:26:51.707651+0000","last_fullsized":"2023-09-30T22:26:51.707651+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.689511+0000","last_change":"2023-09-30T22:26:46.274062+0000","last_active":"2023-09-30T22:26:51.689511+0000","last_peered":"2023-09-30T22:26:51.689511+0000","last_clean":"2023-09-30T22:26:51.689511+0000","last_became_active":"2023-09-30T22:26:46.273752+0000","last_became_peered":"2023-09-30T22:26:46.273752+0000","last_unstale":"2023-09-30T22:26:51.689511+0000","last_undegraded":"2023-09-30T22:26:51.689511+0000","last_fullsized":"2023-09-30T22:26:51.689511+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.702185+0000","last_change":"2023-09-30T22:26:51.295181+0000","last_active":"2023-09-30T22:26:51.702185+0000","last_peered":"2023-09-30T22:26:51.702185+0000","last_clean":"2023-09-30T22:26:51.702185+0000","last_became_active":"2023-09-30T22:26:51.294889+0000","last_became_peered":"2023-09-30T22:26:51.294889+0000","last_unstale":"2023-09-30T22:26:51.702185+0000","last_undegraded":"2023-09-30T22:26:51.702185+0000","last_fullsized":"2023-09-30T22:26:51.702185+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.698806+0000","last_change":"2023-09-30T22:26:51.690467+0000","last_active":"2023-09-30T22:26:51.698806+0000","last_peered":"2023-09-30T22:26:51.698806+0000","last_clean":"2023-09-30T22:26:51.698806+0000","last_became_active":"2023-09-30T22:26:51.690163+0000","last_became_peered":"2023-09-30T22:26:51.690163+0000","last_unstale":"2023-09-30T22:26:51.698806+0000","last_undegraded":"2023-09-30T22:26:51.698806+0000","last_fullsized":"2023-09-30T22:26:51.698806+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"15'2","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.716522+0000","last_change":"2023-09-30T22:26:51.290268+0000","last_active":"2023-09-30T22:26:51.716522+0000","last_peered":"2023-09-30T22:26:51.716522+0000","last_clean":"2023-09-30T22:26:51.716522+0000","last_became_active":"2023-09-30T22:26:51.289409+0000","last_became_peered":"2023-09-30T22:26:51.289409+0000","last_unstale":"2023-09-30T22:26:51.716522+0000","last_undegraded":"2023-09-30T22:26:51.716522+0000","last_fullsized":"2023-09-30T22:26:51.716522+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.712603+0000","last_change":"2023-09-30T22:26:46.273888+0000","last_active":"2023-09-30T22:26:51.712603+0000","last_peered":"2023-09-30T22:26:51.712603+0000","last_clean":"2023-09-30T22:26:51.712603+0000","last_became_active":"2023-09-30T22:26:46.273549+0000","last_became_peered":"2023-09-30T22:26:46.273549+0000","last_unstale":"2023-09-30T22:26:51.712603+0000","last_undegraded":"2023-09-30T22:26:51.712603+0000","last_fullsized":"2023-09-30T22:26:51.712603+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.714861+0000","last_change":"2023-09-30T22:26:51.709356+0000","last_active":"2023-09-30T22:26:51.714861+0000","last_peered":"2023-09-30T22:26:51.714861+0000","last_clean":"2023-09-30T22:26:51.714861+0000","last_became_active":"2023-09-30T22:26:51.709059+0000","last_became_peered":"2023-09-30T22:26:51.709059+0000","last_unstale":"2023-09-30T22:26:51.714861+0000","last_undegraded":"2023-09-30T22:26:51.714861+0000","last_fullsized":"2023-09-30T22:26:51.714861+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.682520+0000","last_change":"2023-09-30T22:26:46.274508+0000","last_active":"2023-09-30T22:26:51.682520+0000","last_peered":"2023-09-30T22:26:51.682520+0000","last_clean":"2023-09-30T22:26:51.682520+0000","last_became_active":"2023-09-30T22:26:46.273567+0000","last_became_peered":"2023-09-30T22:26:46.273567+0000","last_unstale":"2023-09-30T22:26:51.682520+0000","last_undegraded":"2023-09-30T22:26:51.682520+0000","last_fullsized":"2023-09-30T22:26:51.682520+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.690008+0000","last_change":"2023-09-30T22:26:51.288625+0000","last_active":"2023-09-30T22:26:51.690008+0000","last_peered":"2023-09-30T22:26:51.690008+0000","last_clean":"2023-09-30T22:26:51.690008+0000","last_became_active":"2023-09-30T22:26:51.288286+0000","last_became_peered":"2023-09-30T22:26:51.288286+0000","last_unstale":"2023-09-30T22:26:51.690008+0000","last_undegraded":"2023-09-30T22:26:51.690008+0000","last_fullsized":"2023-09-30T22:26:51.690008+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.718567+0000","last_change":"2023-09-30T22:26:51.708965+0000","last_active":"2023-09-30T22:26:51.718567+0000","last_peered":"2023-09-30T22:26:51.718567+0000","last_clean":"2023-09-30T22:26:51.718567+0000","last_became_active":"2023-09-30T22:26:51.708607+0000","last_became_peered":"2023-09-30T22:26:51.708607+0000","last_unstale":"2023-09-30T22:26:51.718567+0000","last_undegraded":"2023-09-30T22:26:51.718567+0000","last_fullsized":"2023-09-30T22:26:51.718567+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"15'1","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.719553+0000","last_change":"2023-09-30T22:26:51.282125+0000","last_active":"2023-09-30T22:26:51.719553+0000","last_peered":"2023-09-30T22:26:51.719553+0000","last_clean":"2023-09-30T22:26:51.719553+0000","last_became_active":"2023-09-30T22:26:51.281780+0000","last_became_peered":"2023-09-30T22:26:51.281780+0000","last_unstale":"2023-09-30T22:26:51.719553+0000","last_undegraded":"2023-09-30T22:26:51.719553+0000","last_fullsized":"2023-09-30T22:26:51.719553+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.709532+0000","last_change":"2023-09-30T22:26:51.703281+0000","last_active":"2023-09-30T22:26:51.709532+0000","last_peered":"2023-09-30T22:26:51.709532+0000","last_clean":"2023-09-30T22:26:51.709532+0000","last_became_active":"2023-09-30T22:26:51.703007+0000","last_became_peered":"2023-09-30T22:26:51.703007+0000","last_unstale":"2023-09-30T22:26:51.709532+0000","last_undegraded":"2023-09-30T22:26:51.709532+0000","last_fullsized":"2023-09-30T22:26:51.709532+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"15'2","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.726495+0000","last_change":"2023-09-30T22:26:51.287630+0000","last_active":"2023-09-30T22:26:51.726495+0000","last_peered":"2023-09-30T22:26:51.726495+0000","last_clean":"2023-09-30T22:26:51.726495+0000","last_became_active":"2023-09-30T22:26:51.287337+0000","last_became_peered":"2023-09-30T22:26:51.287337+0000","last_unstale":"2023-09-30T22:26:51.726495+0000","last_undegraded":"2023-09-30T22:26:51.726495+0000","last_fullsized":"2023-09-30T22:26:51.726495+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.719532+0000","last_change":"2023-09-30T22:26:51.716458+0000","last_active":"2023-09-30T22:26:51.719532+0000","last_peered":"2023-09-30T22:26:51.719532+0000","last_clean":"2023-09-30T22:26:51.719532+0000","last_became_active":"2023-09-30T22:26:51.715957+0000","last_became_peered":"2023-09-30T22:26:51.715957+0000","last_unstale":"2023-09-30T22:26:51.719532+0000","last_undegraded":"2023-09-30T22:26:51.719532+0000","last_fullsized":"2023-09-30T22:26:51.719532+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"15'2","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.716732+0000","last_change":"2023-09-30T22:26:51.285956+0000","last_active":"2023-09-30T22:26:51.716732+0000","last_peered":"2023-09-30T22:26:51.716732+0000","last_clean":"2023-09-30T22:26:51.716732+0000","last_became_active":"2023-09-30T22:26:51.285690+0000","last_became_peered":"2023-09-30T22:26:51.285690+0000","last_unstale":"2023-09-30T22:26:51.716732+0000","last_undegraded":"2023-09-30T22:26:51.716732+0000","last_fullsized":"2023-09-30T22:26:51.716732+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.719485+0000","last_change":"2023-09-30T22:26:51.716290+0000","last_active":"2023-09-30T22:26:51.719485+0000","last_peered":"2023-09-30T22:26:51.719485+0000","last_clean":"2023-09-30T22:26:51.719485+0000","last_became_active":"2023-09-30T22:26:51.716007+0000","last_became_peered":"2023-09-30T22:26:51.716007+0000","last_unstale":"2023-09-30T22:26:51.719485+0000","last_undegraded":"2023-09-30T22:26:51.719485+0000","last_fullsized":"2023-09-30T22:26:51.719485+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1e","version":"15'1","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.719629+0000","last_change":"2023-09-30T22:26:51.296949+0000","last_active":"2023-09-30T22:26:51.719629+0000","last_peered":"2023-09-30T22:26:51.719629+0000","last_clean":"2023-09-30T22:26:51.719629+0000","last_became_active":"2023-09-30T22:26:51.296602+0000","last_became_peered":"2023-09-30T22:26:51.296602+0000","last_unstale":"2023-09-30T22:26:51.719629+0000","last_undegraded":"2023-09-30T22:26:51.719629+0000","last_fullsized":"2023-09-30T22:26:51.719629+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.714046+0000","last_change":"2023-09-30T22:26:51.706034+0000","last_active":"2023-09-30T22:26:51.714046+0000","last_peered":"2023-09-30T22:26:51.714046+0000","last_clean":"2023-09-30T22:26:51.714046+0000","last_became_active":"2023-09-30T22:26:51.705633+0000","last_became_peered":"2023-09-30T22:26:51.705633+0000","last_unstale":"2023-09-30T22:26:51.714046+0000","last_undegraded":"2023-09-30T22:26:51.714046+0000","last_fullsized":"2023-09-30T22:26:51.714046+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"15'4","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.722169+0000","last_change":"2023-09-30T22:26:51.291976+0000","last_active":"2023-09-30T22:26:51.722169+0000","last_peered":"2023-09-30T22:26:51.722169+0000","last_clean":"2023-09-30T22:26:51.722169+0000","last_became_active":"2023-09-30T22:26:51.291255+0000","last_became_peered":"2023-09-30T22:26:51.291255+0000","last_unstale":"2023-09-30T22:26:51.722169+0000","last_undegraded":"2023-09-30T22:26:51.722169+0000","last_fullsized":"2023-09-30T22:26:51.722169+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.718466+0000","last_change":"2023-09-30T22:26:51.713485+0000","last_active":"2023-09-30T22:26:51.718466+0000","last_peered":"2023-09-30T22:26:51.718466+0000","last_clean":"2023-09-30T22:26:51.718466+0000","last_became_active":"2023-09-30T22:26:51.713188+0000","last_became_peered":"2023-09-30T22:26:51.713188+0000","last_unstale":"2023-09-30T22:26:51.718466+0000","last_undegraded":"2023-09-30T22:26:51.718466+0000","last_fullsized":"2023-09-30T22:26:51.718466+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.709789+0000","last_change":"2023-09-30T22:26:51.704082+0000","last_active":"2023-09-30T22:26:51.709789+0000","last_peered":"2023-09-30T22:26:51.709789+0000","last_clean":"2023-09-30T22:26:51.709789+0000","last_became_active":"2023-09-30T22:26:51.703280+0000","last_became_peered":"2023-09-30T22:26:51.703280+0000","last_unstale":"2023-09-30T22:26:51.709789+0000","last_undegraded":"2023-09-30T22:26:51.709789+0000","last_fullsized":"2023-09-30T22:26:51.709789+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.714010+0000","last_change":"2023-09-30T22:26:51.705877+0000","last_active":"2023-09-30T22:26:51.714010+0000","last_peered":"2023-09-30T22:26:51.714010+0000","last_clean":"2023-09-30T22:26:51.714010+0000","last_became_active":"2023-09-30T22:26:51.705352+0000","last_became_peered":"2023-09-30T22:26:51.705352+0000","last_unstale":"2023-09-30T22:26:51.714010+0000","last_undegraded":"2023-09-30T22:26:51.714010+0000","last_fullsized":"2023-09-30T22:26:51.714010+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.719132+0000","last_change":"2023-09-30T22:26:51.712201+0000","last_active":"2023-09-30T22:26:51.719132+0000","last_peered":"2023-09-30T22:26:51.719132+0000","last_clean":"2023-09-30T22:26:51.719132+0000","last_became_active":"2023-09-30T22:26:51.709083+0000","last_became_peered":"2023-09-30T22:26:51.709083+0000","last_unstale":"2023-09-30T22:26:51.719132+0000","last_undegraded":"2023-09-30T22:26:51.719132+0000","last_fullsized":"2023-09-30T22:26:51.719132+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.715082+0000","last_change":"2023-09-30T22:26:51.706543+0000","last_active":"2023-09-30T22:26:51.715082+0000","last_peered":"2023-09-30T22:26:51.715082+0000","last_clean":"2023-09-30T22:26:51.715082+0000","last_became_active":"2023-09-30T22:26:51.706251+0000","last_became_peered":"2023-09-30T22:26:51.706251+0000","last_unstale":"2023-09-30T22:26:51.715082+0000","last_undegraded":"2023-09-30T22:26:51.715082+0000","last_fullsized":"2023-09-30T22:26:51.715082+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730665+0000","last_change":"2023-09-30T22:26:51.724591+0000","last_active":"2023-09-30T22:26:51.730665+0000","last_peered":"2023-09-30T22:26:51.730665+0000","last_clean":"2023-09-30T22:26:51.730665+0000","last_became_active":"2023-09-30T22:26:51.722847+0000","last_became_peered":"2023-09-30T22:26:51.722847+0000","last_unstale":"2023-09-30T22:26:51.730665+0000","last_undegraded":"2023-09-30T22:26:51.730665+0000","last_fullsized":"2023-09-30T22:26:51.730665+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705667,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297256,"kb_used_data":160,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074584,"statfs":{"total":96636764160,"available":96332374016,"internally_reserved":0,"allocated":163840,"data_stored":46244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38800000000000001}]}]},{"osd":2,"up_from":9,"seq":38654705668,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297244,"kb_used_data":148,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074596,"statfs":{"total":96636764160,"available":96332386304,"internally_reserved":0,"allocated":151552,"data_stored":46050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.315}]}]},{"osd":0,"up_from":9,"seq":38654705667,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297244,"kb_used_data":148,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074596,"statfs":{"total":96636764160,"available":96332386304,"internally_reserved":0,"allocated":151552,"data_stored":44044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]}]},{"osd":1,"up_from":9,"seq":38654705667,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297244,"kb_used_data":148,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074596,"statfs":{"total":96636764160,"available":96332386304,"internally_reserved":0,"allocated":151552,"data_stored":45160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38300000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-30T22:27:00.296 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-30T22:27:00.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.434+0000 7f8f5aa6c700 1 Processor -- start 2023-09-30T22:27:00.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.434+0000 7f8f5aa6c700 1 -- start start 2023-09-30T22:27:00.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.434+0000 7f8f5aa6c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8f54091b00 0x7f8f5412c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:00.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.434+0000 7f8f5aa6c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8f5412ce60 0x7f8f5412f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:00.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.434+0000 7f8f5aa6c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f54096760 con 0x7f8f54096380 2023-09-30T22:27:00.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.434+0000 7f8f5aa6c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f54091ff0 con 0x7f8f54091b00 2023-09-30T22:27:00.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.434+0000 7f8f5aa6c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8f54092c80 con 0x7f8f5412ce60 2023-09-30T22:27:00.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.434+0000 7f8f59009700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8f5412ce60 0x7f8f5412f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:00.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.434+0000 7f8f53fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8f54091b00 0x7f8f5412c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:00.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.434+0000 7f8f58808700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8f54096380 0x7f8f54094730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51402/0 (socket says 172.21.15.16:51402) 2023-09-30T22:27:00.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.434+0000 7f8f58808700 1 -- 172.21.15.16:0/3959885778 learned_addr learned my addr 172.21.15.16:0/3959885778 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:27:00.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.435+0000 7f8f53fff700 1 -- 172.21.15.16:0/3959885778 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8f5412ce60 msgr2=0x7f8f5412f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:00.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.435+0000 7f8f53fff700 1 --2- 172.21.15.16:0/3959885778 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8f5412ce60 0x7f8f5412f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:00.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.435+0000 7f8f53fff700 1 -- 172.21.15.16:0/3959885778 >> v1:172.21.15.16:6789/0 conn(0x7f8f54096380 legacy=0x7f8f54094730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:00.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.435+0000 7f8f537fe700 1 -- 172.21.15.16:0/3959885778 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2216394519 0 0) 0x7f8f54096760 con 0x7f8f54096380 2023-09-30T22:27:00.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.435+0000 7f8f53fff700 1 -- 172.21.15.16:0/3959885778 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f54094e40 con 0x7f8f54091b00 2023-09-30T22:27:00.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.435+0000 7f8f59009700 1 --2- 172.21.15.16:0/3959885778 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8f5412ce60 0x7f8f5412f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:00.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.435+0000 7f8f53fff700 1 --2- 172.21.15.16:0/3959885778 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8f54091b00 0x7f8f5412c970 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f8f48009790 tx=0x7f8f48009060).ready entity=mon.1 client_cookie=9d572aa01604e0d5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:00.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.435+0000 7f8f537fe700 1 -- 172.21.15.16:0/3959885778 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4800f6b0 con 0x7f8f54091b00 2023-09-30T22:27:00.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.435+0000 7f8f537fe700 1 -- 172.21.15.16:0/3959885778 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f4800f810 con 0x7f8f54091b00 2023-09-30T22:27:00.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.435+0000 7f8f537fe700 1 -- 172.21.15.16:0/3959885778 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4800fb90 con 0x7f8f54091b00 2023-09-30T22:27:00.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.436+0000 7f8f5aa6c700 1 -- 172.21.15.16:0/3959885778 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8f54091b00 msgr2=0x7f8f5412c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:00.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.436+0000 7f8f5aa6c700 1 --2- 172.21.15.16:0/3959885778 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8f54091b00 0x7f8f5412c970 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f8f48009790 tx=0x7f8f48009060).stop 2023-09-30T22:27:00.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.436+0000 7f8f5aa6c700 1 -- 172.21.15.16:0/3959885778 shutdown_connections 2023-09-30T22:27:00.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.436+0000 7f8f5aa6c700 1 --2- 172.21.15.16:0/3959885778 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8f5412ce60 0x7f8f5412f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:00.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.436+0000 7f8f5aa6c700 1 --2- 172.21.15.16:0/3959885778 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8f54091b00 0x7f8f5412c970 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:00.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.436+0000 7f8f5aa6c700 1 -- 172.21.15.16:0/3959885778 >> 172.21.15.16:0/3959885778 conn(0x7f8f54097470 msgr2=0x7f8f54097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:00.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.436+0000 7f8f5aa6c700 1 -- 172.21.15.16:0/3959885778 shutdown_connections 2023-09-30T22:27:00.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.436+0000 7f8f5aa6c700 1 -- 172.21.15.16:0/3959885778 wait complete. 2023-09-30T22:27:00.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.437+0000 7f8f5aa6c700 1 Processor -- start 2023-09-30T22:27:00.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.437+0000 7f8f5aa6c700 1 -- start start 2023-09-30T22:27:00.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.437+0000 7f8f5aa6c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8f54091b00 0x7f8f5413a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:00.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.437+0000 7f8f5aa6c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8f54096380 0x7f8f5413a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:00.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.437+0000 7f8f5aa6c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f5412ce60 0x7f8f5413ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:00.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.437+0000 7f8f5aa6c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5412c310 con 0x7f8f5412ce60 2023-09-30T22:27:00.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.437+0000 7f8f5aa6c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f54098ca0 con 0x7f8f54091b00 2023-09-30T22:27:00.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.437+0000 7f8f5aa6c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5406a520 con 0x7f8f54096380 2023-09-30T22:27:00.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.437+0000 7f8f59009700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f5412ce60 0x7f8f5413ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:00.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.437+0000 7f8f59009700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f5412ce60 0x7f8f5413ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54368/0 (socket says 172.21.15.16:54368) 2023-09-30T22:27:00.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.437+0000 7f8f58808700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8f54091b00 0x7f8f5413a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:00.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.437+0000 7f8f59009700 1 -- 172.21.15.16:0/255117913 learned_addr learned my addr 172.21.15.16:0/255117913 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:00.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.437+0000 7f8f53fff700 1 --2- 172.21.15.16:0/255117913 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8f54096380 0x7f8f5413a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:00.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.438+0000 7f8f53fff700 1 -- 172.21.15.16:0/255117913 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8f54091b00 msgr2=0x7f8f5413a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:00.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.438+0000 7f8f53fff700 1 --2- 172.21.15.16:0/255117913 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8f54091b00 0x7f8f5413a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:00.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.438+0000 7f8f53fff700 1 -- 172.21.15.16:0/255117913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f5412ce60 msgr2=0x7f8f5413ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:00.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.438+0000 7f8f53fff700 1 --2- 172.21.15.16:0/255117913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f5412ce60 0x7f8f5413ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:00.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.438+0000 7f8f53fff700 1 -- 172.21.15.16:0/255117913 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f4c009a20 con 0x7f8f54096380 2023-09-30T22:27:00.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.438+0000 7f8f59009700 1 --2- 172.21.15.16:0/255117913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f5412ce60 0x7f8f5413ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:00.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.438+0000 7f8f58808700 1 --2- 172.21.15.16:0/255117913 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8f54091b00 0x7f8f5413a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:00.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.438+0000 7f8f53fff700 1 --2- 172.21.15.16:0/255117913 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8f54096380 0x7f8f5413a870 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f8f48016000 tx=0x7f8f48019000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:00.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.438+0000 7f8f50ff9700 1 -- 172.21.15.16:0/255117913 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4800f980 con 0x7f8f54096380 2023-09-30T22:27:00.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.438+0000 7f8f50ff9700 1 -- 172.21.15.16:0/255117913 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f4800fae0 con 0x7f8f54096380 2023-09-30T22:27:00.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.438+0000 7f8f5aa6c700 1 -- 172.21.15.16:0/255117913 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f480092a0 con 0x7f8f54096380 2023-09-30T22:27:00.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.438+0000 7f8f50ff9700 1 -- 172.21.15.16:0/255117913 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f480196a0 con 0x7f8f54096380 2023-09-30T22:27:00.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.438+0000 7f8f5aa6c700 1 -- 172.21.15.16:0/255117913 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f54116180 con 0x7f8f54096380 2023-09-30T22:27:00.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.439+0000 7f8f50ff9700 1 -- 172.21.15.16:0/255117913 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f8f48019a50 con 0x7f8f54096380 2023-09-30T22:27:00.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.439+0000 7f8f50ff9700 1 -- 172.21.15.16:0/255117913 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f8f4803c510 con 0x7f8f54096380 2023-09-30T22:27:00.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.439+0000 7f8f5aa6c700 1 -- 172.21.15.16:0/255117913 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f54115d50 con 0x7f8f54096380 2023-09-30T22:27:00.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.444+0000 7f8f50ff9700 1 -- 172.21.15.16:0/255117913 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f8f48027070 con 0x7f8f54096380 2023-09-30T22:27:00.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.616+0000 7f8f5aa6c700 1 -- 172.21.15.16:0/255117913 --> v1:172.21.15.16:6816/118285 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8f5405a260 con 0x7f8f4002bb60 2023-09-30T22:27:00.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.622+0000 7f8f50ff9700 1 -- 172.21.15.16:0/255117913 <== mgr.4098 v1:172.21.15.16:6816/118285 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+249933 (unknown 2965378022 0 1639013670) 0x7f8f5405a260 con 0x7f8f4002bb60 2023-09-30T22:27:00.627 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:27:00.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.625+0000 7f8f5aa6c700 1 -- 172.21.15.16:0/255117913 >> v1:172.21.15.16:6816/118285 conn(0x7f8f4002bb60 legacy=0x7f8f4002e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:00.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.625+0000 7f8f5aa6c700 1 -- 172.21.15.16:0/255117913 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8f54096380 msgr2=0x7f8f5413a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:00.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.625+0000 7f8f5aa6c700 1 --2- 172.21.15.16:0/255117913 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8f54096380 0x7f8f5413a870 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f8f48016000 tx=0x7f8f48019000).stop 2023-09-30T22:27:00.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.626+0000 7f8f5aa6c700 1 -- 172.21.15.16:0/255117913 shutdown_connections 2023-09-30T22:27:00.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.626+0000 7f8f5aa6c700 1 --2- 172.21.15.16:0/255117913 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8f54091b00 0x7f8f5413a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:00.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.626+0000 7f8f5aa6c700 1 --2- 172.21.15.16:0/255117913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f5412ce60 0x7f8f5413ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:00.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.626+0000 7f8f5aa6c700 1 --2- 172.21.15.16:0/255117913 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8f54096380 0x7f8f5413a870 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:00.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.626+0000 7f8f5aa6c700 1 -- 172.21.15.16:0/255117913 >> 172.21.15.16:0/255117913 conn(0x7f8f54097470 msgr2=0x7f8f540952a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:00.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.626+0000 7f8f5aa6c700 1 -- 172.21.15.16:0/255117913 shutdown_connections 2023-09-30T22:27:00.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.626+0000 7f8f5aa6c700 1 -- 172.21.15.16:0/255117913 wait complete. 2023-09-30T22:27:00.630 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-09-30T22:27:00.640 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-09-30T22:27:00.456472+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1188988,"kb_used_data":604,"kb_used_omap":0,"kb_used_meta":1188352,"kb_avail":376298372,"statfs":{"total":386547056640,"available":385329532928,"internally_reserved":0,"allocated":618496,"data_stored":181498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1216872448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000926"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.718509+0000","last_change":"2023-09-30T22:26:51.708966+0000","last_active":"2023-09-30T22:26:51.718509+0000","last_peered":"2023-09-30T22:26:51.718509+0000","last_clean":"2023-09-30T22:26:51.718509+0000","last_became_active":"2023-09-30T22:26:51.708653+0000","last_became_peered":"2023-09-30T22:26:51.708653+0000","last_unstale":"2023-09-30T22:26:51.718509+0000","last_undegraded":"2023-09-30T22:26:51.718509+0000","last_fullsized":"2023-09-30T22:26:51.718509+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.729225+0000","last_change":"2023-09-30T22:26:51.727077+0000","last_active":"2023-09-30T22:26:51.729225+0000","last_peered":"2023-09-30T22:26:51.729225+0000","last_clean":"2023-09-30T22:26:51.729225+0000","last_became_active":"2023-09-30T22:26:51.726778+0000","last_became_peered":"2023-09-30T22:26:51.726778+0000","last_unstale":"2023-09-30T22:26:51.729225+0000","last_undegraded":"2023-09-30T22:26:51.729225+0000","last_fullsized":"2023-09-30T22:26:51.729225+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.714151+0000","last_change":"2023-09-30T22:26:51.706919+0000","last_active":"2023-09-30T22:26:51.714151+0000","last_peered":"2023-09-30T22:26:51.714151+0000","last_clean":"2023-09-30T22:26:51.714151+0000","last_became_active":"2023-09-30T22:26:51.706676+0000","last_became_peered":"2023-09-30T22:26:51.706676+0000","last_unstale":"2023-09-30T22:26:51.714151+0000","last_undegraded":"2023-09-30T22:26:51.714151+0000","last_fullsized":"2023-09-30T22:26:51.714151+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.728676+0000","last_change":"2023-09-30T22:26:51.727386+0000","last_active":"2023-09-30T22:26:51.728676+0000","last_peered":"2023-09-30T22:26:51.728676+0000","last_clean":"2023-09-30T22:26:51.728676+0000","last_became_active":"2023-09-30T22:26:51.727091+0000","last_became_peered":"2023-09-30T22:26:51.727091+0000","last_unstale":"2023-09-30T22:26:51.728676+0000","last_undegraded":"2023-09-30T22:26:51.728676+0000","last_fullsized":"2023-09-30T22:26:51.728676+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.723905+0000","last_change":"2023-09-30T22:26:51.719993+0000","last_active":"2023-09-30T22:26:51.723905+0000","last_peered":"2023-09-30T22:26:51.723905+0000","last_clean":"2023-09-30T22:26:51.723905+0000","last_became_active":"2023-09-30T22:26:51.719495+0000","last_became_peered":"2023-09-30T22:26:51.719495+0000","last_unstale":"2023-09-30T22:26:51.723905+0000","last_undegraded":"2023-09-30T22:26:51.723905+0000","last_fullsized":"2023-09-30T22:26:51.723905+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.715438+0000","last_change":"2023-09-30T22:26:51.709581+0000","last_active":"2023-09-30T22:26:51.715438+0000","last_peered":"2023-09-30T22:26:51.715438+0000","last_clean":"2023-09-30T22:26:51.715438+0000","last_became_active":"2023-09-30T22:26:51.709309+0000","last_became_peered":"2023-09-30T22:26:51.709309+0000","last_unstale":"2023-09-30T22:26:51.715438+0000","last_undegraded":"2023-09-30T22:26:51.715438+0000","last_fullsized":"2023-09-30T22:26:51.715438+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.713991+0000","last_change":"2023-09-30T22:26:51.705826+0000","last_active":"2023-09-30T22:26:51.713991+0000","last_peered":"2023-09-30T22:26:51.713991+0000","last_clean":"2023-09-30T22:26:51.713991+0000","last_became_active":"2023-09-30T22:26:51.705037+0000","last_became_peered":"2023-09-30T22:26:51.705037+0000","last_unstale":"2023-09-30T22:26:51.713991+0000","last_undegraded":"2023-09-30T22:26:51.713991+0000","last_fullsized":"2023-09-30T22:26:51.713991+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730836+0000","last_change":"2023-09-30T22:26:51.726640+0000","last_active":"2023-09-30T22:26:51.730836+0000","last_peered":"2023-09-30T22:26:51.730836+0000","last_clean":"2023-09-30T22:26:51.730836+0000","last_became_active":"2023-09-30T22:26:51.726278+0000","last_became_peered":"2023-09-30T22:26:51.726278+0000","last_unstale":"2023-09-30T22:26:51.730836+0000","last_undegraded":"2023-09-30T22:26:51.730836+0000","last_fullsized":"2023-09-30T22:26:51.730836+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.724179+0000","last_change":"2023-09-30T22:26:51.721817+0000","last_active":"2023-09-30T22:26:51.724179+0000","last_peered":"2023-09-30T22:26:51.724179+0000","last_clean":"2023-09-30T22:26:51.724179+0000","last_became_active":"2023-09-30T22:26:51.721373+0000","last_became_peered":"2023-09-30T22:26:51.721373+0000","last_unstale":"2023-09-30T22:26:51.724179+0000","last_undegraded":"2023-09-30T22:26:51.724179+0000","last_fullsized":"2023-09-30T22:26:51.724179+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730810+0000","last_change":"2023-09-30T22:26:51.726348+0000","last_active":"2023-09-30T22:26:51.730810+0000","last_peered":"2023-09-30T22:26:51.730810+0000","last_clean":"2023-09-30T22:26:51.730810+0000","last_became_active":"2023-09-30T22:26:51.726030+0000","last_became_peered":"2023-09-30T22:26:51.726030+0000","last_unstale":"2023-09-30T22:26:51.730810+0000","last_undegraded":"2023-09-30T22:26:51.730810+0000","last_fullsized":"2023-09-30T22:26:51.730810+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730607+0000","last_change":"2023-09-30T22:26:51.722619+0000","last_active":"2023-09-30T22:26:51.730607+0000","last_peered":"2023-09-30T22:26:51.730607+0000","last_clean":"2023-09-30T22:26:51.730607+0000","last_became_active":"2023-09-30T22:26:51.722317+0000","last_became_peered":"2023-09-30T22:26:51.722317+0000","last_unstale":"2023-09-30T22:26:51.730607+0000","last_undegraded":"2023-09-30T22:26:51.730607+0000","last_fullsized":"2023-09-30T22:26:51.730607+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.718852+0000","last_change":"2023-09-30T22:26:51.709697+0000","last_active":"2023-09-30T22:26:51.718852+0000","last_peered":"2023-09-30T22:26:51.718852+0000","last_clean":"2023-09-30T22:26:51.718852+0000","last_became_active":"2023-09-30T22:26:51.709342+0000","last_became_peered":"2023-09-30T22:26:51.709342+0000","last_unstale":"2023-09-30T22:26:51.718852+0000","last_undegraded":"2023-09-30T22:26:51.718852+0000","last_fullsized":"2023-09-30T22:26:51.718852+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.719514+0000","last_change":"2023-09-30T22:26:51.716276+0000","last_active":"2023-09-30T22:26:51.719514+0000","last_peered":"2023-09-30T22:26:51.719514+0000","last_clean":"2023-09-30T22:26:51.719514+0000","last_became_active":"2023-09-30T22:26:51.715634+0000","last_became_peered":"2023-09-30T22:26:51.715634+0000","last_unstale":"2023-09-30T22:26:51.719514+0000","last_undegraded":"2023-09-30T22:26:51.719514+0000","last_fullsized":"2023-09-30T22:26:51.719514+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.707967+0000","last_change":"2023-09-30T22:26:51.695948+0000","last_active":"2023-09-30T22:26:51.707967+0000","last_peered":"2023-09-30T22:26:51.707967+0000","last_clean":"2023-09-30T22:26:51.707967+0000","last_became_active":"2023-09-30T22:26:51.695661+0000","last_became_peered":"2023-09-30T22:26:51.695661+0000","last_unstale":"2023-09-30T22:26:51.707967+0000","last_undegraded":"2023-09-30T22:26:51.707967+0000","last_fullsized":"2023-09-30T22:26:51.707967+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.716656+0000","last_change":"2023-09-30T22:26:51.708532+0000","last_active":"2023-09-30T22:26:51.716656+0000","last_peered":"2023-09-30T22:26:51.716656+0000","last_clean":"2023-09-30T22:26:51.716656+0000","last_became_active":"2023-09-30T22:26:51.708204+0000","last_became_peered":"2023-09-30T22:26:51.708204+0000","last_unstale":"2023-09-30T22:26:51.716656+0000","last_undegraded":"2023-09-30T22:26:51.716656+0000","last_fullsized":"2023-09-30T22:26:51.716656+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.714211+0000","last_change":"2023-09-30T22:26:51.704597+0000","last_active":"2023-09-30T22:26:51.714211+0000","last_peered":"2023-09-30T22:26:51.714211+0000","last_clean":"2023-09-30T22:26:51.714211+0000","last_became_active":"2023-09-30T22:26:51.704307+0000","last_became_peered":"2023-09-30T22:26:51.704307+0000","last_unstale":"2023-09-30T22:26:51.714211+0000","last_undegraded":"2023-09-30T22:26:51.714211+0000","last_fullsized":"2023-09-30T22:26:51.714211+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.724159+0000","last_change":"2023-09-30T22:26:51.721640+0000","last_active":"2023-09-30T22:26:51.724159+0000","last_peered":"2023-09-30T22:26:51.724159+0000","last_clean":"2023-09-30T22:26:51.724159+0000","last_became_active":"2023-09-30T22:26:51.720950+0000","last_became_peered":"2023-09-30T22:26:51.720950+0000","last_unstale":"2023-09-30T22:26:51.724159+0000","last_undegraded":"2023-09-30T22:26:51.724159+0000","last_fullsized":"2023-09-30T22:26:51.724159+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.707139+0000","last_change":"2023-09-30T22:26:51.697513+0000","last_active":"2023-09-30T22:26:51.707139+0000","last_peered":"2023-09-30T22:26:51.707139+0000","last_clean":"2023-09-30T22:26:51.707139+0000","last_became_active":"2023-09-30T22:26:51.697137+0000","last_became_peered":"2023-09-30T22:26:51.697137+0000","last_unstale":"2023-09-30T22:26:51.707139+0000","last_undegraded":"2023-09-30T22:26:51.707139+0000","last_fullsized":"2023-09-30T22:26:51.707139+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.718540+0000","last_change":"2023-09-30T22:26:51.715476+0000","last_active":"2023-09-30T22:26:51.718540+0000","last_peered":"2023-09-30T22:26:51.718540+0000","last_clean":"2023-09-30T22:26:51.718540+0000","last_became_active":"2023-09-30T22:26:51.715176+0000","last_became_peered":"2023-09-30T22:26:51.715176+0000","last_unstale":"2023-09-30T22:26:51.718540+0000","last_undegraded":"2023-09-30T22:26:51.718540+0000","last_fullsized":"2023-09-30T22:26:51.718540+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730693+0000","last_change":"2023-09-30T22:26:51.724991+0000","last_active":"2023-09-30T22:26:51.730693+0000","last_peered":"2023-09-30T22:26:51.730693+0000","last_clean":"2023-09-30T22:26:51.730693+0000","last_became_active":"2023-09-30T22:26:51.724240+0000","last_became_peered":"2023-09-30T22:26:51.724240+0000","last_unstale":"2023-09-30T22:26:51.730693+0000","last_undegraded":"2023-09-30T22:26:51.730693+0000","last_fullsized":"2023-09-30T22:26:51.730693+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730391+0000","last_change":"2023-09-30T22:26:51.717833+0000","last_active":"2023-09-30T22:26:51.730391+0000","last_peered":"2023-09-30T22:26:51.730391+0000","last_clean":"2023-09-30T22:26:51.730391+0000","last_became_active":"2023-09-30T22:26:51.717517+0000","last_became_peered":"2023-09-30T22:26:51.717517+0000","last_unstale":"2023-09-30T22:26:51.730391+0000","last_undegraded":"2023-09-30T22:26:51.730391+0000","last_fullsized":"2023-09-30T22:26:51.730391+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.723937+0000","last_change":"2023-09-30T22:26:51.719030+0000","last_active":"2023-09-30T22:26:51.723937+0000","last_peered":"2023-09-30T22:26:51.723937+0000","last_clean":"2023-09-30T22:26:51.723937+0000","last_became_active":"2023-09-30T22:26:51.715370+0000","last_became_peered":"2023-09-30T22:26:51.715370+0000","last_unstale":"2023-09-30T22:26:51.723937+0000","last_undegraded":"2023-09-30T22:26:51.723937+0000","last_fullsized":"2023-09-30T22:26:51.723937+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.713970+0000","last_change":"2023-09-30T22:26:51.698664+0000","last_active":"2023-09-30T22:26:51.713970+0000","last_peered":"2023-09-30T22:26:51.713970+0000","last_clean":"2023-09-30T22:26:51.713970+0000","last_became_active":"2023-09-30T22:26:51.698321+0000","last_became_peered":"2023-09-30T22:26:51.698321+0000","last_unstale":"2023-09-30T22:26:51.713970+0000","last_undegraded":"2023-09-30T22:26:51.713970+0000","last_fullsized":"2023-09-30T22:26:51.713970+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730972+0000","last_change":"2023-09-30T22:26:51.727955+0000","last_active":"2023-09-30T22:26:51.730972+0000","last_peered":"2023-09-30T22:26:51.730972+0000","last_clean":"2023-09-30T22:26:51.730972+0000","last_became_active":"2023-09-30T22:26:51.727209+0000","last_became_peered":"2023-09-30T22:26:51.727209+0000","last_unstale":"2023-09-30T22:26:51.730972+0000","last_undegraded":"2023-09-30T22:26:51.730972+0000","last_fullsized":"2023-09-30T22:26:51.730972+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730499+0000","last_change":"2023-09-30T22:26:51.721154+0000","last_active":"2023-09-30T22:26:51.730499+0000","last_peered":"2023-09-30T22:26:51.730499+0000","last_clean":"2023-09-30T22:26:51.730499+0000","last_became_active":"2023-09-30T22:26:51.720703+0000","last_became_peered":"2023-09-30T22:26:51.720703+0000","last_unstale":"2023-09-30T22:26:51.730499+0000","last_undegraded":"2023-09-30T22:26:51.730499+0000","last_fullsized":"2023-09-30T22:26:51.730499+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.725830+0000","last_change":"2023-09-30T22:26:51.701309+0000","last_active":"2023-09-30T22:26:51.725830+0000","last_peered":"2023-09-30T22:26:51.725830+0000","last_clean":"2023-09-30T22:26:51.725830+0000","last_became_active":"2023-09-30T22:26:51.701012+0000","last_became_peered":"2023-09-30T22:26:51.701012+0000","last_unstale":"2023-09-30T22:26:51.725830+0000","last_undegraded":"2023-09-30T22:26:51.725830+0000","last_fullsized":"2023-09-30T22:26:51.725830+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.718423+0000","last_change":"2023-09-30T22:26:51.715042+0000","last_active":"2023-09-30T22:26:51.718423+0000","last_peered":"2023-09-30T22:26:51.718423+0000","last_clean":"2023-09-30T22:26:51.718423+0000","last_became_active":"2023-09-30T22:26:51.712158+0000","last_became_peered":"2023-09-30T22:26:51.712158+0000","last_unstale":"2023-09-30T22:26:51.718423+0000","last_undegraded":"2023-09-30T22:26:51.718423+0000","last_fullsized":"2023-09-30T22:26:51.718423+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"15'1","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.713320+0000","last_change":"2023-09-30T22:26:51.296397+0000","last_active":"2023-09-30T22:26:51.713320+0000","last_peered":"2023-09-30T22:26:51.713320+0000","last_clean":"2023-09-30T22:26:51.713320+0000","last_became_active":"2023-09-30T22:26:51.295860+0000","last_became_peered":"2023-09-30T22:26:51.295860+0000","last_unstale":"2023-09-30T22:26:51.713320+0000","last_undegraded":"2023-09-30T22:26:51.713320+0000","last_fullsized":"2023-09-30T22:26:51.713320+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.716849+0000","last_change":"2023-09-30T22:26:51.712472+0000","last_active":"2023-09-30T22:26:51.716849+0000","last_peered":"2023-09-30T22:26:51.716849+0000","last_clean":"2023-09-30T22:26:51.716849+0000","last_became_active":"2023-09-30T22:26:51.711527+0000","last_became_peered":"2023-09-30T22:26:51.711527+0000","last_unstale":"2023-09-30T22:26:51.716849+0000","last_undegraded":"2023-09-30T22:26:51.716849+0000","last_fullsized":"2023-09-30T22:26:51.716849+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.701600+0000","last_change":"2023-09-30T22:26:51.298004+0000","last_active":"2023-09-30T22:26:51.701600+0000","last_peered":"2023-09-30T22:26:51.701600+0000","last_clean":"2023-09-30T22:26:51.701600+0000","last_became_active":"2023-09-30T22:26:51.297714+0000","last_became_peered":"2023-09-30T22:26:51.297714+0000","last_unstale":"2023-09-30T22:26:51.701600+0000","last_undegraded":"2023-09-30T22:26:51.701600+0000","last_fullsized":"2023-09-30T22:26:51.701600+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.729456+0000","last_change":"2023-09-30T22:26:51.727608+0000","last_active":"2023-09-30T22:26:51.729456+0000","last_peered":"2023-09-30T22:26:51.729456+0000","last_clean":"2023-09-30T22:26:51.729456+0000","last_became_active":"2023-09-30T22:26:51.727154+0000","last_became_peered":"2023-09-30T22:26:51.727154+0000","last_unstale":"2023-09-30T22:26:51.729456+0000","last_undegraded":"2023-09-30T22:26:51.729456+0000","last_fullsized":"2023-09-30T22:26:51.729456+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.684923+0000","last_change":"2023-09-30T22:26:51.285214+0000","last_active":"2023-09-30T22:26:51.684923+0000","last_peered":"2023-09-30T22:26:51.684923+0000","last_clean":"2023-09-30T22:26:51.684923+0000","last_became_active":"2023-09-30T22:26:51.284749+0000","last_became_peered":"2023-09-30T22:26:51.284749+0000","last_unstale":"2023-09-30T22:26:51.684923+0000","last_undegraded":"2023-09-30T22:26:51.684923+0000","last_fullsized":"2023-09-30T22:26:51.684923+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.716115+0000","last_change":"2023-09-30T22:26:51.706697+0000","last_active":"2023-09-30T22:26:51.716115+0000","last_peered":"2023-09-30T22:26:51.716115+0000","last_clean":"2023-09-30T22:26:51.716115+0000","last_became_active":"2023-09-30T22:26:51.706373+0000","last_became_peered":"2023-09-30T22:26:51.706373+0000","last_unstale":"2023-09-30T22:26:51.716115+0000","last_undegraded":"2023-09-30T22:26:51.716115+0000","last_fullsized":"2023-09-30T22:26:51.716115+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.689119+0000","last_change":"2023-09-30T22:26:51.281642+0000","last_active":"2023-09-30T22:26:51.689119+0000","last_peered":"2023-09-30T22:26:51.689119+0000","last_clean":"2023-09-30T22:26:51.689119+0000","last_became_active":"2023-09-30T22:26:51.281319+0000","last_became_peered":"2023-09-30T22:26:51.281319+0000","last_unstale":"2023-09-30T22:26:51.689119+0000","last_undegraded":"2023-09-30T22:26:51.689119+0000","last_fullsized":"2023-09-30T22:26:51.689119+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.714232+0000","last_change":"2023-09-30T22:26:51.705192+0000","last_active":"2023-09-30T22:26:51.714232+0000","last_peered":"2023-09-30T22:26:51.714232+0000","last_clean":"2023-09-30T22:26:51.714232+0000","last_became_active":"2023-09-30T22:26:51.704870+0000","last_became_peered":"2023-09-30T22:26:51.704870+0000","last_unstale":"2023-09-30T22:26:51.714232+0000","last_undegraded":"2023-09-30T22:26:51.714232+0000","last_fullsized":"2023-09-30T22:26:51.714232+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.680797+0000","last_change":"2023-09-30T22:26:51.293711+0000","last_active":"2023-09-30T22:26:51.680797+0000","last_peered":"2023-09-30T22:26:51.680797+0000","last_clean":"2023-09-30T22:26:51.680797+0000","last_became_active":"2023-09-30T22:26:51.293410+0000","last_became_peered":"2023-09-30T22:26:51.293410+0000","last_unstale":"2023-09-30T22:26:51.680797+0000","last_undegraded":"2023-09-30T22:26:51.680797+0000","last_fullsized":"2023-09-30T22:26:51.680797+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.714029+0000","last_change":"2023-09-30T22:26:51.706005+0000","last_active":"2023-09-30T22:26:51.714029+0000","last_peered":"2023-09-30T22:26:51.714029+0000","last_clean":"2023-09-30T22:26:51.714029+0000","last_became_active":"2023-09-30T22:26:51.705491+0000","last_became_peered":"2023-09-30T22:26:51.705491+0000","last_unstale":"2023-09-30T22:26:51.714029+0000","last_undegraded":"2023-09-30T22:26:51.714029+0000","last_fullsized":"2023-09-30T22:26:51.714029+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"15'3","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.719469+0000","last_change":"2023-09-30T22:26:51.279976+0000","last_active":"2023-09-30T22:26:51.719469+0000","last_peered":"2023-09-30T22:26:51.719469+0000","last_clean":"2023-09-30T22:26:51.719469+0000","last_became_active":"2023-09-30T22:26:51.279669+0000","last_became_peered":"2023-09-30T22:26:51.279669+0000","last_unstale":"2023-09-30T22:26:51.719469+0000","last_undegraded":"2023-09-30T22:26:51.719469+0000","last_fullsized":"2023-09-30T22:26:51.719469+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730714+0000","last_change":"2023-09-30T22:26:51.725947+0000","last_active":"2023-09-30T22:26:51.730714+0000","last_peered":"2023-09-30T22:26:51.730714+0000","last_clean":"2023-09-30T22:26:51.730714+0000","last_became_active":"2023-09-30T22:26:51.723294+0000","last_became_peered":"2023-09-30T22:26:51.723294+0000","last_unstale":"2023-09-30T22:26:51.730714+0000","last_undegraded":"2023-09-30T22:26:51.730714+0000","last_fullsized":"2023-09-30T22:26:51.730714+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"15'6","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.723849+0000","last_change":"2023-09-30T22:26:51.297527+0000","last_active":"2023-09-30T22:26:51.723849+0000","last_peered":"2023-09-30T22:26:51.723849+0000","last_clean":"2023-09-30T22:26:51.723849+0000","last_became_active":"2023-09-30T22:26:51.297246+0000","last_became_peered":"2023-09-30T22:26:51.297246+0000","last_unstale":"2023-09-30T22:26:51.723849+0000","last_undegraded":"2023-09-30T22:26:51.723849+0000","last_fullsized":"2023-09-30T22:26:51.723849+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.718810+0000","last_change":"2023-09-30T22:26:51.710781+0000","last_active":"2023-09-30T22:26:51.718810+0000","last_peered":"2023-09-30T22:26:51.718810+0000","last_clean":"2023-09-30T22:26:51.718810+0000","last_became_active":"2023-09-30T22:26:51.710107+0000","last_became_peered":"2023-09-30T22:26:51.710107+0000","last_unstale":"2023-09-30T22:26:51.718810+0000","last_undegraded":"2023-09-30T22:26:51.718810+0000","last_fullsized":"2023-09-30T22:26:51.718810+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"15'2","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.714029+0000","last_change":"2023-09-30T22:26:51.290194+0000","last_active":"2023-09-30T22:26:51.714029+0000","last_peered":"2023-09-30T22:26:51.714029+0000","last_clean":"2023-09-30T22:26:51.714029+0000","last_became_active":"2023-09-30T22:26:51.288688+0000","last_became_peered":"2023-09-30T22:26:51.288688+0000","last_unstale":"2023-09-30T22:26:51.714029+0000","last_undegraded":"2023-09-30T22:26:51.714029+0000","last_fullsized":"2023-09-30T22:26:51.714029+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.728819+0000","last_change":"2023-09-30T22:26:51.726780+0000","last_active":"2023-09-30T22:26:51.728819+0000","last_peered":"2023-09-30T22:26:51.728819+0000","last_clean":"2023-09-30T22:26:51.728819+0000","last_became_active":"2023-09-30T22:26:51.726408+0000","last_became_peered":"2023-09-30T22:26:51.726408+0000","last_unstale":"2023-09-30T22:26:51.728819+0000","last_undegraded":"2023-09-30T22:26:51.728819+0000","last_fullsized":"2023-09-30T22:26:51.728819+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.685836+0000","last_change":"2023-09-30T22:26:51.281127+0000","last_active":"2023-09-30T22:26:51.685836+0000","last_peered":"2023-09-30T22:26:51.685836+0000","last_clean":"2023-09-30T22:26:51.685836+0000","last_became_active":"2023-09-30T22:26:51.280507+0000","last_became_peered":"2023-09-30T22:26:51.280507+0000","last_unstale":"2023-09-30T22:26:51.685836+0000","last_undegraded":"2023-09-30T22:26:51.685836+0000","last_fullsized":"2023-09-30T22:26:51.685836+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.716876+0000","last_change":"2023-09-30T22:26:51.712671+0000","last_active":"2023-09-30T22:26:51.716876+0000","last_peered":"2023-09-30T22:26:51.716876+0000","last_clean":"2023-09-30T22:26:51.716876+0000","last_became_active":"2023-09-30T22:26:51.711836+0000","last_became_peered":"2023-09-30T22:26:51.711836+0000","last_unstale":"2023-09-30T22:26:51.716876+0000","last_undegraded":"2023-09-30T22:26:51.716876+0000","last_fullsized":"2023-09-30T22:26:51.716876+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.683587+0000","last_change":"2023-09-30T22:26:51.297467+0000","last_active":"2023-09-30T22:26:51.683587+0000","last_peered":"2023-09-30T22:26:51.683587+0000","last_clean":"2023-09-30T22:26:51.683587+0000","last_became_active":"2023-09-30T22:26:51.296950+0000","last_became_peered":"2023-09-30T22:26:51.296950+0000","last_unstale":"2023-09-30T22:26:51.683587+0000","last_undegraded":"2023-09-30T22:26:51.683587+0000","last_fullsized":"2023-09-30T22:26:51.683587+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.724064+0000","last_change":"2023-09-30T22:26:51.719411+0000","last_active":"2023-09-30T22:26:51.724064+0000","last_peered":"2023-09-30T22:26:51.724064+0000","last_clean":"2023-09-30T22:26:51.724064+0000","last_became_active":"2023-09-30T22:26:51.719111+0000","last_became_peered":"2023-09-30T22:26:51.719111+0000","last_unstale":"2023-09-30T22:26:51.724064+0000","last_undegraded":"2023-09-30T22:26:51.724064+0000","last_fullsized":"2023-09-30T22:26:51.724064+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"15'8","reported_seq":18,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.727696+0000","last_change":"2023-09-30T22:26:51.287702+0000","last_active":"2023-09-30T22:26:51.727696+0000","last_peered":"2023-09-30T22:26:51.727696+0000","last_clean":"2023-09-30T22:26:51.727696+0000","last_became_active":"2023-09-30T22:26:51.287434+0000","last_became_peered":"2023-09-30T22:26:51.287434+0000","last_unstale":"2023-09-30T22:26:51.727696+0000","last_undegraded":"2023-09-30T22:26:51.727696+0000","last_fullsized":"2023-09-30T22:26:51.727696+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.716138+0000","last_change":"2023-09-30T22:26:51.706663+0000","last_active":"2023-09-30T22:26:51.716138+0000","last_peered":"2023-09-30T22:26:51.716138+0000","last_clean":"2023-09-30T22:26:51.716138+0000","last_became_active":"2023-09-30T22:26:51.706363+0000","last_became_peered":"2023-09-30T22:26:51.706363+0000","last_unstale":"2023-09-30T22:26:51.716138+0000","last_undegraded":"2023-09-30T22:26:51.716138+0000","last_fullsized":"2023-09-30T22:26:51.716138+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"15'1","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.721973+0000","last_change":"2023-09-30T22:26:51.291996+0000","last_active":"2023-09-30T22:26:51.721973+0000","last_peered":"2023-09-30T22:26:51.721973+0000","last_clean":"2023-09-30T22:26:51.721973+0000","last_became_active":"2023-09-30T22:26:51.291656+0000","last_became_peered":"2023-09-30T22:26:51.291656+0000","last_unstale":"2023-09-30T22:26:51.721973+0000","last_undegraded":"2023-09-30T22:26:51.721973+0000","last_fullsized":"2023-09-30T22:26:51.721973+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.715557+0000","last_change":"2023-09-30T22:26:51.710326+0000","last_active":"2023-09-30T22:26:51.715557+0000","last_peered":"2023-09-30T22:26:51.715557+0000","last_clean":"2023-09-30T22:26:51.715557+0000","last_became_active":"2023-09-30T22:26:51.708793+0000","last_became_peered":"2023-09-30T22:26:51.708793+0000","last_unstale":"2023-09-30T22:26:51.715557+0000","last_undegraded":"2023-09-30T22:26:51.715557+0000","last_fullsized":"2023-09-30T22:26:51.715557+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"15'2","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.699110+0000","last_change":"2023-09-30T22:26:51.300700+0000","last_active":"2023-09-30T22:26:51.699110+0000","last_peered":"2023-09-30T22:26:51.699110+0000","last_clean":"2023-09-30T22:26:51.699110+0000","last_became_active":"2023-09-30T22:26:51.300416+0000","last_became_peered":"2023-09-30T22:26:51.300416+0000","last_unstale":"2023-09-30T22:26:51.699110+0000","last_undegraded":"2023-09-30T22:26:51.699110+0000","last_fullsized":"2023-09-30T22:26:51.699110+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730623+0000","last_change":"2023-09-30T22:26:51.722902+0000","last_active":"2023-09-30T22:26:51.730623+0000","last_peered":"2023-09-30T22:26:51.730623+0000","last_clean":"2023-09-30T22:26:51.730623+0000","last_became_active":"2023-09-30T22:26:51.717919+0000","last_became_peered":"2023-09-30T22:26:51.717919+0000","last_unstale":"2023-09-30T22:26:51.730623+0000","last_undegraded":"2023-09-30T22:26:51.730623+0000","last_fullsized":"2023-09-30T22:26:51.730623+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"15'1","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.716626+0000","last_change":"2023-09-30T22:26:51.292120+0000","last_active":"2023-09-30T22:26:51.716626+0000","last_peered":"2023-09-30T22:26:51.716626+0000","last_clean":"2023-09-30T22:26:51.716626+0000","last_became_active":"2023-09-30T22:26:51.289469+0000","last_became_peered":"2023-09-30T22:26:51.289469+0000","last_unstale":"2023-09-30T22:26:51.716626+0000","last_undegraded":"2023-09-30T22:26:51.716626+0000","last_fullsized":"2023-09-30T22:26:51.716626+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.718700+0000","last_change":"2023-09-30T22:26:51.711649+0000","last_active":"2023-09-30T22:26:51.718700+0000","last_peered":"2023-09-30T22:26:51.718700+0000","last_clean":"2023-09-30T22:26:51.718700+0000","last_became_active":"2023-09-30T22:26:51.711384+0000","last_became_peered":"2023-09-30T22:26:51.711384+0000","last_unstale":"2023-09-30T22:26:51.718700+0000","last_undegraded":"2023-09-30T22:26:51.718700+0000","last_fullsized":"2023-09-30T22:26:51.718700+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.686889+0000","last_change":"2023-09-30T22:26:51.281627+0000","last_active":"2023-09-30T22:26:51.686889+0000","last_peered":"2023-09-30T22:26:51.686889+0000","last_clean":"2023-09-30T22:26:51.686889+0000","last_became_active":"2023-09-30T22:26:51.281311+0000","last_became_peered":"2023-09-30T22:26:51.281311+0000","last_unstale":"2023-09-30T22:26:51.686889+0000","last_undegraded":"2023-09-30T22:26:51.686889+0000","last_fullsized":"2023-09-30T22:26:51.686889+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.707659+0000","last_change":"2023-09-30T22:26:51.700837+0000","last_active":"2023-09-30T22:26:51.707659+0000","last_peered":"2023-09-30T22:26:51.707659+0000","last_clean":"2023-09-30T22:26:51.707659+0000","last_became_active":"2023-09-30T22:26:51.700523+0000","last_became_peered":"2023-09-30T22:26:51.700523+0000","last_unstale":"2023-09-30T22:26:51.707659+0000","last_undegraded":"2023-09-30T22:26:51.707659+0000","last_fullsized":"2023-09-30T22:26:51.707659+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"15'2","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.721892+0000","last_change":"2023-09-30T22:26:51.290248+0000","last_active":"2023-09-30T22:26:51.721892+0000","last_peered":"2023-09-30T22:26:51.721892+0000","last_clean":"2023-09-30T22:26:51.721892+0000","last_became_active":"2023-09-30T22:26:51.289168+0000","last_became_peered":"2023-09-30T22:26:51.289168+0000","last_unstale":"2023-09-30T22:26:51.721892+0000","last_undegraded":"2023-09-30T22:26:51.721892+0000","last_fullsized":"2023-09-30T22:26:51.721892+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.723965+0000","last_change":"2023-09-30T22:26:51.719224+0000","last_active":"2023-09-30T22:26:51.723965+0000","last_peered":"2023-09-30T22:26:51.723965+0000","last_clean":"2023-09-30T22:26:51.723965+0000","last_became_active":"2023-09-30T22:26:51.718890+0000","last_became_peered":"2023-09-30T22:26:51.718890+0000","last_unstale":"2023-09-30T22:26:51.723965+0000","last_undegraded":"2023-09-30T22:26:51.723965+0000","last_fullsized":"2023-09-30T22:26:51.723965+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.687008+0000","last_change":"2023-09-30T22:26:51.288204+0000","last_active":"2023-09-30T22:26:51.687008+0000","last_peered":"2023-09-30T22:26:51.687008+0000","last_clean":"2023-09-30T22:26:51.687008+0000","last_became_active":"2023-09-30T22:26:51.287903+0000","last_became_peered":"2023-09-30T22:26:51.287903+0000","last_unstale":"2023-09-30T22:26:51.687008+0000","last_undegraded":"2023-09-30T22:26:51.687008+0000","last_fullsized":"2023-09-30T22:26:51.687008+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.718399+0000","last_change":"2023-09-30T22:26:51.713792+0000","last_active":"2023-09-30T22:26:51.718399+0000","last_peered":"2023-09-30T22:26:51.718399+0000","last_clean":"2023-09-30T22:26:51.718399+0000","last_became_active":"2023-09-30T22:26:51.713334+0000","last_became_peered":"2023-09-30T22:26:51.713334+0000","last_unstale":"2023-09-30T22:26:51.718399+0000","last_undegraded":"2023-09-30T22:26:51.718399+0000","last_fullsized":"2023-09-30T22:26:51.718399+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.683106+0000","last_change":"2023-09-30T22:26:51.297240+0000","last_active":"2023-09-30T22:26:51.683106+0000","last_peered":"2023-09-30T22:26:51.683106+0000","last_clean":"2023-09-30T22:26:51.683106+0000","last_became_active":"2023-09-30T22:26:51.296547+0000","last_became_peered":"2023-09-30T22:26:51.296547+0000","last_unstale":"2023-09-30T22:26:51.683106+0000","last_undegraded":"2023-09-30T22:26:51.683106+0000","last_fullsized":"2023-09-30T22:26:51.683106+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.718663+0000","last_change":"2023-09-30T22:26:51.711889+0000","last_active":"2023-09-30T22:26:51.718663+0000","last_peered":"2023-09-30T22:26:51.718663+0000","last_clean":"2023-09-30T22:26:51.718663+0000","last_became_active":"2023-09-30T22:26:51.711570+0000","last_became_peered":"2023-09-30T22:26:51.711570+0000","last_unstale":"2023-09-30T22:26:51.718663+0000","last_undegraded":"2023-09-30T22:26:51.718663+0000","last_fullsized":"2023-09-30T22:26:51.718663+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.684383+0000","last_change":"2023-09-30T22:26:51.290699+0000","last_active":"2023-09-30T22:26:51.684383+0000","last_peered":"2023-09-30T22:26:51.684383+0000","last_clean":"2023-09-30T22:26:51.684383+0000","last_became_active":"2023-09-30T22:26:51.290396+0000","last_became_peered":"2023-09-30T22:26:51.290396+0000","last_unstale":"2023-09-30T22:26:51.684383+0000","last_undegraded":"2023-09-30T22:26:51.684383+0000","last_fullsized":"2023-09-30T22:26:51.684383+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730474+0000","last_change":"2023-09-30T22:26:51.720976+0000","last_active":"2023-09-30T22:26:51.730474+0000","last_peered":"2023-09-30T22:26:51.730474+0000","last_clean":"2023-09-30T22:26:51.730474+0000","last_became_active":"2023-09-30T22:26:51.720378+0000","last_became_peered":"2023-09-30T22:26:51.720378+0000","last_unstale":"2023-09-30T22:26:51.730474+0000","last_undegraded":"2023-09-30T22:26:51.730474+0000","last_fullsized":"2023-09-30T22:26:51.730474+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.702412+0000","last_change":"2023-09-30T22:26:46.273891+0000","last_active":"2023-09-30T22:26:51.702412+0000","last_peered":"2023-09-30T22:26:51.702412+0000","last_clean":"2023-09-30T22:26:51.702412+0000","last_became_active":"2023-09-30T22:26:46.273588+0000","last_became_peered":"2023-09-30T22:26:46.273588+0000","last_unstale":"2023-09-30T22:26:51.702412+0000","last_undegraded":"2023-09-30T22:26:51.702412+0000","last_fullsized":"2023-09-30T22:26:51.702412+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.680672+0000","last_change":"2023-09-30T22:26:51.294682+0000","last_active":"2023-09-30T22:26:51.680672+0000","last_peered":"2023-09-30T22:26:51.680672+0000","last_clean":"2023-09-30T22:26:51.680672+0000","last_became_active":"2023-09-30T22:26:51.294422+0000","last_became_peered":"2023-09-30T22:26:51.294422+0000","last_unstale":"2023-09-30T22:26:51.680672+0000","last_undegraded":"2023-09-30T22:26:51.680672+0000","last_fullsized":"2023-09-30T22:26:51.680672+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730874+0000","last_change":"2023-09-30T22:26:51.726954+0000","last_active":"2023-09-30T22:26:51.730874+0000","last_peered":"2023-09-30T22:26:51.730874+0000","last_clean":"2023-09-30T22:26:51.730874+0000","last_became_active":"2023-09-30T22:26:51.726641+0000","last_became_peered":"2023-09-30T22:26:51.726641+0000","last_unstale":"2023-09-30T22:26:51.730874+0000","last_undegraded":"2023-09-30T22:26:51.730874+0000","last_fullsized":"2023-09-30T22:26:51.730874+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.687452+0000","last_change":"2023-09-30T22:26:46.245946+0000","last_active":"2023-09-30T22:26:51.687452+0000","last_peered":"2023-09-30T22:26:51.687452+0000","last_clean":"2023-09-30T22:26:51.687452+0000","last_became_active":"2023-09-30T22:26:46.245602+0000","last_became_peered":"2023-09-30T22:26:46.245602+0000","last_unstale":"2023-09-30T22:26:51.687452+0000","last_undegraded":"2023-09-30T22:26:51.687452+0000","last_fullsized":"2023-09-30T22:26:51.687452+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.685668+0000","last_change":"2023-09-30T22:26:51.294691+0000","last_active":"2023-09-30T22:26:51.685668+0000","last_peered":"2023-09-30T22:26:51.685668+0000","last_clean":"2023-09-30T22:26:51.685668+0000","last_became_active":"2023-09-30T22:26:51.294411+0000","last_became_peered":"2023-09-30T22:26:51.294411+0000","last_unstale":"2023-09-30T22:26:51.685668+0000","last_undegraded":"2023-09-30T22:26:51.685668+0000","last_fullsized":"2023-09-30T22:26:51.685668+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730680+0000","last_change":"2023-09-30T22:26:51.724759+0000","last_active":"2023-09-30T22:26:51.730680+0000","last_peered":"2023-09-30T22:26:51.730680+0000","last_clean":"2023-09-30T22:26:51.730680+0000","last_became_active":"2023-09-30T22:26:51.723221+0000","last_became_peered":"2023-09-30T22:26:51.723221+0000","last_unstale":"2023-09-30T22:26:51.730680+0000","last_undegraded":"2023-09-30T22:26:51.730680+0000","last_fullsized":"2023-09-30T22:26:51.730680+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.681986+0000","last_change":"2023-09-30T22:26:46.242607+0000","last_active":"2023-09-30T22:26:51.681986+0000","last_peered":"2023-09-30T22:26:51.681986+0000","last_clean":"2023-09-30T22:26:51.681986+0000","last_became_active":"2023-09-30T22:26:46.242283+0000","last_became_peered":"2023-09-30T22:26:46.242283+0000","last_unstale":"2023-09-30T22:26:51.681986+0000","last_undegraded":"2023-09-30T22:26:51.681986+0000","last_fullsized":"2023-09-30T22:26:51.681986+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.703666+0000","last_change":"2023-09-30T22:26:51.295409+0000","last_active":"2023-09-30T22:26:51.703666+0000","last_peered":"2023-09-30T22:26:51.703666+0000","last_clean":"2023-09-30T22:26:51.703666+0000","last_became_active":"2023-09-30T22:26:51.295140+0000","last_became_peered":"2023-09-30T22:26:51.295140+0000","last_unstale":"2023-09-30T22:26:51.703666+0000","last_undegraded":"2023-09-30T22:26:51.703666+0000","last_fullsized":"2023-09-30T22:26:51.703666+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730344+0000","last_change":"2023-09-30T22:26:51.715504+0000","last_active":"2023-09-30T22:26:51.730344+0000","last_peered":"2023-09-30T22:26:51.730344+0000","last_clean":"2023-09-30T22:26:51.730344+0000","last_became_active":"2023-09-30T22:26:51.715258+0000","last_became_peered":"2023-09-30T22:26:51.715258+0000","last_unstale":"2023-09-30T22:26:51.730344+0000","last_undegraded":"2023-09-30T22:26:51.730344+0000","last_fullsized":"2023-09-30T22:26:51.730344+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.682511+0000","last_change":"2023-09-30T22:26:46.242752+0000","last_active":"2023-09-30T22:26:51.682511+0000","last_peered":"2023-09-30T22:26:51.682511+0000","last_clean":"2023-09-30T22:26:51.682511+0000","last_became_active":"2023-09-30T22:26:46.242385+0000","last_became_peered":"2023-09-30T22:26:46.242385+0000","last_unstale":"2023-09-30T22:26:51.682511+0000","last_undegraded":"2023-09-30T22:26:51.682511+0000","last_fullsized":"2023-09-30T22:26:51.682511+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.703565+0000","last_change":"2023-09-30T22:26:46.273930+0000","last_active":"2023-09-30T22:26:51.703565+0000","last_peered":"2023-09-30T22:26:51.703565+0000","last_clean":"2023-09-30T22:26:51.703565+0000","last_became_active":"2023-09-30T22:26:46.273621+0000","last_became_peered":"2023-09-30T22:26:46.273621+0000","last_unstale":"2023-09-30T22:26:51.703565+0000","last_undegraded":"2023-09-30T22:26:51.703565+0000","last_fullsized":"2023-09-30T22:26:51.703565+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.686892+0000","last_change":"2023-09-30T22:26:51.298163+0000","last_active":"2023-09-30T22:26:51.686892+0000","last_peered":"2023-09-30T22:26:51.686892+0000","last_clean":"2023-09-30T22:26:51.686892+0000","last_became_active":"2023-09-30T22:26:51.297897+0000","last_became_peered":"2023-09-30T22:26:51.297897+0000","last_unstale":"2023-09-30T22:26:51.686892+0000","last_undegraded":"2023-09-30T22:26:51.686892+0000","last_fullsized":"2023-09-30T22:26:51.686892+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.718377+0000","last_change":"2023-09-30T22:26:51.713608+0000","last_active":"2023-09-30T22:26:51.718377+0000","last_peered":"2023-09-30T22:26:51.718377+0000","last_clean":"2023-09-30T22:26:51.718377+0000","last_became_active":"2023-09-30T22:26:51.712992+0000","last_became_peered":"2023-09-30T22:26:51.712992+0000","last_unstale":"2023-09-30T22:26:51.718377+0000","last_undegraded":"2023-09-30T22:26:51.718377+0000","last_fullsized":"2023-09-30T22:26:51.718377+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.684824+0000","last_change":"2023-09-30T22:26:46.245668+0000","last_active":"2023-09-30T22:26:51.684824+0000","last_peered":"2023-09-30T22:26:51.684824+0000","last_clean":"2023-09-30T22:26:51.684824+0000","last_became_active":"2023-09-30T22:26:46.245319+0000","last_became_peered":"2023-09-30T22:26:46.245319+0000","last_unstale":"2023-09-30T22:26:51.684824+0000","last_undegraded":"2023-09-30T22:26:51.684824+0000","last_fullsized":"2023-09-30T22:26:51.684824+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.681181+0000","last_change":"2023-09-30T22:26:51.294664+0000","last_active":"2023-09-30T22:26:51.681181+0000","last_peered":"2023-09-30T22:26:51.681181+0000","last_clean":"2023-09-30T22:26:51.681181+0000","last_became_active":"2023-09-30T22:26:51.294357+0000","last_became_peered":"2023-09-30T22:26:51.294357+0000","last_unstale":"2023-09-30T22:26:51.681181+0000","last_undegraded":"2023-09-30T22:26:51.681181+0000","last_fullsized":"2023-09-30T22:26:51.681181+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.707651+0000","last_change":"2023-09-30T22:26:51.705784+0000","last_active":"2023-09-30T22:26:51.707651+0000","last_peered":"2023-09-30T22:26:51.707651+0000","last_clean":"2023-09-30T22:26:51.707651+0000","last_became_active":"2023-09-30T22:26:51.705480+0000","last_became_peered":"2023-09-30T22:26:51.705480+0000","last_unstale":"2023-09-30T22:26:51.707651+0000","last_undegraded":"2023-09-30T22:26:51.707651+0000","last_fullsized":"2023-09-30T22:26:51.707651+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.689511+0000","last_change":"2023-09-30T22:26:46.274062+0000","last_active":"2023-09-30T22:26:51.689511+0000","last_peered":"2023-09-30T22:26:51.689511+0000","last_clean":"2023-09-30T22:26:51.689511+0000","last_became_active":"2023-09-30T22:26:46.273752+0000","last_became_peered":"2023-09-30T22:26:46.273752+0000","last_unstale":"2023-09-30T22:26:51.689511+0000","last_undegraded":"2023-09-30T22:26:51.689511+0000","last_fullsized":"2023-09-30T22:26:51.689511+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.702185+0000","last_change":"2023-09-30T22:26:51.295181+0000","last_active":"2023-09-30T22:26:51.702185+0000","last_peered":"2023-09-30T22:26:51.702185+0000","last_clean":"2023-09-30T22:26:51.702185+0000","last_became_active":"2023-09-30T22:26:51.294889+0000","last_became_peered":"2023-09-30T22:26:51.294889+0000","last_unstale":"2023-09-30T22:26:51.702185+0000","last_undegraded":"2023-09-30T22:26:51.702185+0000","last_fullsized":"2023-09-30T22:26:51.702185+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.698806+0000","last_change":"2023-09-30T22:26:51.690467+0000","last_active":"2023-09-30T22:26:51.698806+0000","last_peered":"2023-09-30T22:26:51.698806+0000","last_clean":"2023-09-30T22:26:51.698806+0000","last_became_active":"2023-09-30T22:26:51.690163+0000","last_became_peered":"2023-09-30T22:26:51.690163+0000","last_unstale":"2023-09-30T22:26:51.698806+0000","last_undegraded":"2023-09-30T22:26:51.698806+0000","last_fullsized":"2023-09-30T22:26:51.698806+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"15'2","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.716522+0000","last_change":"2023-09-30T22:26:51.290268+0000","last_active":"2023-09-30T22:26:51.716522+0000","last_peered":"2023-09-30T22:26:51.716522+0000","last_clean":"2023-09-30T22:26:51.716522+0000","last_became_active":"2023-09-30T22:26:51.289409+0000","last_became_peered":"2023-09-30T22:26:51.289409+0000","last_unstale":"2023-09-30T22:26:51.716522+0000","last_undegraded":"2023-09-30T22:26:51.716522+0000","last_fullsized":"2023-09-30T22:26:51.716522+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.712603+0000","last_change":"2023-09-30T22:26:46.273888+0000","last_active":"2023-09-30T22:26:51.712603+0000","last_peered":"2023-09-30T22:26:51.712603+0000","last_clean":"2023-09-30T22:26:51.712603+0000","last_became_active":"2023-09-30T22:26:46.273549+0000","last_became_peered":"2023-09-30T22:26:46.273549+0000","last_unstale":"2023-09-30T22:26:51.712603+0000","last_undegraded":"2023-09-30T22:26:51.712603+0000","last_fullsized":"2023-09-30T22:26:51.712603+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.714861+0000","last_change":"2023-09-30T22:26:51.709356+0000","last_active":"2023-09-30T22:26:51.714861+0000","last_peered":"2023-09-30T22:26:51.714861+0000","last_clean":"2023-09-30T22:26:51.714861+0000","last_became_active":"2023-09-30T22:26:51.709059+0000","last_became_peered":"2023-09-30T22:26:51.709059+0000","last_unstale":"2023-09-30T22:26:51.714861+0000","last_undegraded":"2023-09-30T22:26:51.714861+0000","last_fullsized":"2023-09-30T22:26:51.714861+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.682520+0000","last_change":"2023-09-30T22:26:46.274508+0000","last_active":"2023-09-30T22:26:51.682520+0000","last_peered":"2023-09-30T22:26:51.682520+0000","last_clean":"2023-09-30T22:26:51.682520+0000","last_became_active":"2023-09-30T22:26:46.273567+0000","last_became_peered":"2023-09-30T22:26:46.273567+0000","last_unstale":"2023-09-30T22:26:51.682520+0000","last_undegraded":"2023-09-30T22:26:51.682520+0000","last_fullsized":"2023-09-30T22:26:51.682520+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:26:52.891144+0000","last_change":"2023-09-30T22:26:51.288625+0000","last_active":"2023-09-30T22:26:52.891144+0000","last_peered":"2023-09-30T22:26:52.891144+0000","last_clean":"2023-09-30T22:26:52.891144+0000","last_became_active":"2023-09-30T22:26:51.288286+0000","last_became_peered":"2023-09-30T22:26:51.288286+0000","last_unstale":"2023-09-30T22:26:52.891144+0000","last_undegraded":"2023-09-30T22:26:52.891144+0000","last_fullsized":"2023-09-30T22:26:52.891144+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.718567+0000","last_change":"2023-09-30T22:26:51.708965+0000","last_active":"2023-09-30T22:26:51.718567+0000","last_peered":"2023-09-30T22:26:51.718567+0000","last_clean":"2023-09-30T22:26:51.718567+0000","last_became_active":"2023-09-30T22:26:51.708607+0000","last_became_peered":"2023-09-30T22:26:51.708607+0000","last_unstale":"2023-09-30T22:26:51.718567+0000","last_undegraded":"2023-09-30T22:26:51.718567+0000","last_fullsized":"2023-09-30T22:26:51.718567+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"15'1","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.719553+0000","last_change":"2023-09-30T22:26:51.282125+0000","last_active":"2023-09-30T22:26:51.719553+0000","last_peered":"2023-09-30T22:26:51.719553+0000","last_clean":"2023-09-30T22:26:51.719553+0000","last_became_active":"2023-09-30T22:26:51.281780+0000","last_became_peered":"2023-09-30T22:26:51.281780+0000","last_unstale":"2023-09-30T22:26:51.719553+0000","last_undegraded":"2023-09-30T22:26:51.719553+0000","last_fullsized":"2023-09-30T22:26:51.719553+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.709532+0000","last_change":"2023-09-30T22:26:51.703281+0000","last_active":"2023-09-30T22:26:51.709532+0000","last_peered":"2023-09-30T22:26:51.709532+0000","last_clean":"2023-09-30T22:26:51.709532+0000","last_became_active":"2023-09-30T22:26:51.703007+0000","last_became_peered":"2023-09-30T22:26:51.703007+0000","last_unstale":"2023-09-30T22:26:51.709532+0000","last_undegraded":"2023-09-30T22:26:51.709532+0000","last_fullsized":"2023-09-30T22:26:51.709532+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"15'2","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.726495+0000","last_change":"2023-09-30T22:26:51.287630+0000","last_active":"2023-09-30T22:26:51.726495+0000","last_peered":"2023-09-30T22:26:51.726495+0000","last_clean":"2023-09-30T22:26:51.726495+0000","last_became_active":"2023-09-30T22:26:51.287337+0000","last_became_peered":"2023-09-30T22:26:51.287337+0000","last_unstale":"2023-09-30T22:26:51.726495+0000","last_undegraded":"2023-09-30T22:26:51.726495+0000","last_fullsized":"2023-09-30T22:26:51.726495+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.719532+0000","last_change":"2023-09-30T22:26:51.716458+0000","last_active":"2023-09-30T22:26:51.719532+0000","last_peered":"2023-09-30T22:26:51.719532+0000","last_clean":"2023-09-30T22:26:51.719532+0000","last_became_active":"2023-09-30T22:26:51.715957+0000","last_became_peered":"2023-09-30T22:26:51.715957+0000","last_unstale":"2023-09-30T22:26:51.719532+0000","last_undegraded":"2023-09-30T22:26:51.719532+0000","last_fullsized":"2023-09-30T22:26:51.719532+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"15'2","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.716732+0000","last_change":"2023-09-30T22:26:51.285956+0000","last_active":"2023-09-30T22:26:51.716732+0000","last_peered":"2023-09-30T22:26:51.716732+0000","last_clean":"2023-09-30T22:26:51.716732+0000","last_became_active":"2023-09-30T22:26:51.285690+0000","last_became_peered":"2023-09-30T22:26:51.285690+0000","last_unstale":"2023-09-30T22:26:51.716732+0000","last_undegraded":"2023-09-30T22:26:51.716732+0000","last_fullsized":"2023-09-30T22:26:51.716732+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.719485+0000","last_change":"2023-09-30T22:26:51.716290+0000","last_active":"2023-09-30T22:26:51.719485+0000","last_peered":"2023-09-30T22:26:51.719485+0000","last_clean":"2023-09-30T22:26:51.719485+0000","last_became_active":"2023-09-30T22:26:51.716007+0000","last_became_peered":"2023-09-30T22:26:51.716007+0000","last_unstale":"2023-09-30T22:26:51.719485+0000","last_undegraded":"2023-09-30T22:26:51.719485+0000","last_fullsized":"2023-09-30T22:26:51.719485+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1e","version":"15'1","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.719629+0000","last_change":"2023-09-30T22:26:51.296949+0000","last_active":"2023-09-30T22:26:51.719629+0000","last_peered":"2023-09-30T22:26:51.719629+0000","last_clean":"2023-09-30T22:26:51.719629+0000","last_became_active":"2023-09-30T22:26:51.296602+0000","last_became_peered":"2023-09-30T22:26:51.296602+0000","last_unstale":"2023-09-30T22:26:51.719629+0000","last_undegraded":"2023-09-30T22:26:51.719629+0000","last_fullsized":"2023-09-30T22:26:51.719629+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.714046+0000","last_change":"2023-09-30T22:26:51.706034+0000","last_active":"2023-09-30T22:26:51.714046+0000","last_peered":"2023-09-30T22:26:51.714046+0000","last_clean":"2023-09-30T22:26:51.714046+0000","last_became_active":"2023-09-30T22:26:51.705633+0000","last_became_peered":"2023-09-30T22:26:51.705633+0000","last_unstale":"2023-09-30T22:26:51.714046+0000","last_undegraded":"2023-09-30T22:26:51.714046+0000","last_fullsized":"2023-09-30T22:26:51.714046+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"15'4","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.722169+0000","last_change":"2023-09-30T22:26:51.291976+0000","last_active":"2023-09-30T22:26:51.722169+0000","last_peered":"2023-09-30T22:26:51.722169+0000","last_clean":"2023-09-30T22:26:51.722169+0000","last_became_active":"2023-09-30T22:26:51.291255+0000","last_became_peered":"2023-09-30T22:26:51.291255+0000","last_unstale":"2023-09-30T22:26:51.722169+0000","last_undegraded":"2023-09-30T22:26:51.722169+0000","last_fullsized":"2023-09-30T22:26:51.722169+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.718466+0000","last_change":"2023-09-30T22:26:51.713485+0000","last_active":"2023-09-30T22:26:51.718466+0000","last_peered":"2023-09-30T22:26:51.718466+0000","last_clean":"2023-09-30T22:26:51.718466+0000","last_became_active":"2023-09-30T22:26:51.713188+0000","last_became_peered":"2023-09-30T22:26:51.713188+0000","last_unstale":"2023-09-30T22:26:51.718466+0000","last_undegraded":"2023-09-30T22:26:51.718466+0000","last_fullsized":"2023-09-30T22:26:51.718466+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.709789+0000","last_change":"2023-09-30T22:26:51.704082+0000","last_active":"2023-09-30T22:26:51.709789+0000","last_peered":"2023-09-30T22:26:51.709789+0000","last_clean":"2023-09-30T22:26:51.709789+0000","last_became_active":"2023-09-30T22:26:51.703280+0000","last_became_peered":"2023-09-30T22:26:51.703280+0000","last_unstale":"2023-09-30T22:26:51.709789+0000","last_undegraded":"2023-09-30T22:26:51.709789+0000","last_fullsized":"2023-09-30T22:26:51.709789+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.714010+0000","last_change":"2023-09-30T22:26:51.705877+0000","last_active":"2023-09-30T22:26:51.714010+0000","last_peered":"2023-09-30T22:26:51.714010+0000","last_clean":"2023-09-30T22:26:51.714010+0000","last_became_active":"2023-09-30T22:26:51.705352+0000","last_became_peered":"2023-09-30T22:26:51.705352+0000","last_unstale":"2023-09-30T22:26:51.714010+0000","last_undegraded":"2023-09-30T22:26:51.714010+0000","last_fullsized":"2023-09-30T22:26:51.714010+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.719132+0000","last_change":"2023-09-30T22:26:51.712201+0000","last_active":"2023-09-30T22:26:51.719132+0000","last_peered":"2023-09-30T22:26:51.719132+0000","last_clean":"2023-09-30T22:26:51.719132+0000","last_became_active":"2023-09-30T22:26:51.709083+0000","last_became_peered":"2023-09-30T22:26:51.709083+0000","last_unstale":"2023-09-30T22:26:51.719132+0000","last_undegraded":"2023-09-30T22:26:51.719132+0000","last_fullsized":"2023-09-30T22:26:51.719132+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.715082+0000","last_change":"2023-09-30T22:26:51.706543+0000","last_active":"2023-09-30T22:26:51.715082+0000","last_peered":"2023-09-30T22:26:51.715082+0000","last_clean":"2023-09-30T22:26:51.715082+0000","last_became_active":"2023-09-30T22:26:51.706251+0000","last_became_peered":"2023-09-30T22:26:51.706251+0000","last_unstale":"2023-09-30T22:26:51.715082+0000","last_undegraded":"2023-09-30T22:26:51.715082+0000","last_fullsized":"2023-09-30T22:26:51.715082+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730665+0000","last_change":"2023-09-30T22:26:51.724591+0000","last_active":"2023-09-30T22:26:51.730665+0000","last_peered":"2023-09-30T22:26:51.730665+0000","last_clean":"2023-09-30T22:26:51.730665+0000","last_became_active":"2023-09-30T22:26:51.722847+0000","last_became_peered":"2023-09-30T22:26:51.722847+0000","last_unstale":"2023-09-30T22:26:51.730665+0000","last_undegraded":"2023-09-30T22:26:51.730665+0000","last_fullsized":"2023-09-30T22:26:51.730665+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705668,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297256,"kb_used_data":160,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074584,"statfs":{"total":96636764160,"available":96332374016,"internally_reserved":0,"allocated":163840,"data_stored":46244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32400000000000001}]}]},{"osd":2,"up_from":9,"seq":38654705669,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297244,"kb_used_data":148,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074596,"statfs":{"total":96636764160,"available":96332386304,"internally_reserved":0,"allocated":151552,"data_stored":46050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]}]},{"osd":0,"up_from":9,"seq":38654705668,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297244,"kb_used_data":148,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074596,"statfs":{"total":96636764160,"available":96332386304,"internally_reserved":0,"allocated":151552,"data_stored":44044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]}]},{"osd":1,"up_from":9,"seq":38654705668,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297244,"kb_used_data":148,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074596,"statfs":{"total":96636764160,"available":96332386304,"internally_reserved":0,"allocated":151552,"data_stored":45160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40699999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-30T22:27:00.645 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-09-30T22:27:00.646 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-09-30T22:27:00.646 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-09-30T22:27:00.646 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-09-30T22:27:00.647 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-09-30T22:27:00.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.812+0000 7fc9fdb02700 1 Processor -- start 2023-09-30T22:27:00.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.812+0000 7fc9fdb02700 1 -- start start 2023-09-30T22:27:00.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.812+0000 7fc9fdb02700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc9f8059070 0x7fc9f81256c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:00.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.813+0000 7fc9fdb02700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc9f8125bb0 0x7fc9f8094300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:00.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.813+0000 7fc9fdb02700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9f8095f00 con 0x7fc9f81251c0 2023-09-30T22:27:00.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.813+0000 7fc9fdb02700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9f8059450 con 0x7fc9f8059070 2023-09-30T22:27:00.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.813+0000 7fc9fdb02700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f8096190 con 0x7fc9f8125bb0 2023-09-30T22:27:00.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.813+0000 7fc9f77fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc9f8059070 0x7fc9f81256c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:00.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.813+0000 7fc9f7fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc9f8125bb0 0x7fc9f8094300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:00.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.813+0000 7fc9f77fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc9f8059070 0x7fc9f81256c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54370/0 (socket says 172.21.15.16:54370) 2023-09-30T22:27:00.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.813+0000 7fc9f7fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc9f8125bb0 0x7fc9f8094300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:40062/0 (socket says 172.21.15.16:40062) 2023-09-30T22:27:00.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.813+0000 7fc9f77fe700 1 -- 172.21.15.16:0/868805051 learned_addr learned my addr 172.21.15.16:0/868805051 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:00.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.813+0000 7fc9f6ffd700 1 --1- 172.21.15.16:0/868805051 >> v1:172.21.15.16:6789/0 conn(0x7fc9f81251c0 0x7fc9f8093bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51410/0 (socket says 172.21.15.16:51410) 2023-09-30T22:27:00.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.813+0000 7fc9f77fe700 1 -- 172.21.15.16:0/868805051 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc9f8125bb0 msgr2=0x7fc9f8094300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:00.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.813+0000 7fc9f77fe700 1 --2- 172.21.15.16:0/868805051 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc9f8125bb0 0x7fc9f8094300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:00.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.813+0000 7fc9f77fe700 1 -- 172.21.15.16:0/868805051 >> v1:172.21.15.16:6789/0 conn(0x7fc9f81251c0 legacy=0x7fc9f8093bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:00.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.813+0000 7fc9f77fe700 1 -- 172.21.15.16:0/868805051 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f8059dd0 con 0x7fc9f8059070 2023-09-30T22:27:00.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.813+0000 7fc9f67fc700 1 -- 172.21.15.16:0/868805051 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3906869932 0 0) 0x7fc9f8095f00 con 0x7fc9f81251c0 2023-09-30T22:27:00.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.813+0000 7fc9f7fff700 1 --2- 172.21.15.16:0/868805051 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc9f8125bb0 0x7fc9f8094300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:00.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.813+0000 7fc9f77fe700 1 --2- 172.21.15.16:0/868805051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9f8059070 0x7fc9f81256c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7fc9e8006fd0 tx=0x7fc9e80090f0).ready entity=mon.0 client_cookie=72f6fad39dc079ce server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:00.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.813+0000 7fc9f67fc700 1 -- 172.21.15.16:0/868805051 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e800b580 con 0x7fc9f8059070 2023-09-30T22:27:00.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.813+0000 7fc9f67fc700 1 -- 172.21.15.16:0/868805051 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9e800b6e0 con 0x7fc9f8059070 2023-09-30T22:27:00.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.813+0000 7fc9f67fc700 1 -- 172.21.15.16:0/868805051 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e800ba60 con 0x7fc9f8059070 2023-09-30T22:27:00.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.814+0000 7fc9fdb02700 1 -- 172.21.15.16:0/868805051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9f8059070 msgr2=0x7fc9f81256c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:00.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.814+0000 7fc9fdb02700 1 --2- 172.21.15.16:0/868805051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9f8059070 0x7fc9f81256c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7fc9e8006fd0 tx=0x7fc9e80090f0).stop 2023-09-30T22:27:00.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.814+0000 7fc9fdb02700 1 -- 172.21.15.16:0/868805051 shutdown_connections 2023-09-30T22:27:00.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.814+0000 7fc9fdb02700 1 --2- 172.21.15.16:0/868805051 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc9f8125bb0 0x7fc9f8094300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:00.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.814+0000 7fc9fdb02700 1 --2- 172.21.15.16:0/868805051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9f8059070 0x7fc9f81256c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:00.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.814+0000 7fc9fdb02700 1 -- 172.21.15.16:0/868805051 >> 172.21.15.16:0/868805051 conn(0x7fc9f8097470 msgr2=0x7fc9f8097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:00.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.814+0000 7fc9fdb02700 1 -- 172.21.15.16:0/868805051 shutdown_connections 2023-09-30T22:27:00.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.814+0000 7fc9fdb02700 1 -- 172.21.15.16:0/868805051 wait complete. 2023-09-30T22:27:00.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.815+0000 7fc9fdb02700 1 Processor -- start 2023-09-30T22:27:00.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.815+0000 7fc9fdb02700 1 -- start start 2023-09-30T22:27:00.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.816+0000 7fc9fdb02700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc9f8059070 0x7fc9f8115a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:00.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.816+0000 7fc9fdb02700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc9f81251c0 0x7fc9f8111b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:00.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.816+0000 7fc9fdb02700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9f8125bb0 0x7fc9f8112010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:00.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.816+0000 7fc9fdb02700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f8113f30 con 0x7fc9f8125bb0 2023-09-30T22:27:00.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.816+0000 7fc9fdb02700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f8122ba0 con 0x7fc9f8059070 2023-09-30T22:27:00.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.816+0000 7fc9fdb02700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f80958d0 con 0x7fc9f81251c0 2023-09-30T22:27:00.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.816+0000 7fc9f6ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc9f81251c0 0x7fc9f8111b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:00.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.816+0000 7fc9f7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9f8125bb0 0x7fc9f8112010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:00.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.816+0000 7fc9f77fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc9f8059070 0x7fc9f8115a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:00.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.816+0000 7fc9f7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9f8125bb0 0x7fc9f8112010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54382/0 (socket says 172.21.15.16:54382) 2023-09-30T22:27:00.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.816+0000 7fc9f6ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc9f81251c0 0x7fc9f8111b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:40066/0 (socket says 172.21.15.16:40066) 2023-09-30T22:27:00.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.816+0000 7fc9f7fff700 1 -- 172.21.15.16:0/1567495468 learned_addr learned my addr 172.21.15.16:0/1567495468 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:00.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.816+0000 7fc9f7fff700 1 -- 172.21.15.16:0/1567495468 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc9f81251c0 msgr2=0x7fc9f8111b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:00.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.816+0000 7fc9f7fff700 1 --2- 172.21.15.16:0/1567495468 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc9f81251c0 0x7fc9f8111b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:00.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.816+0000 7fc9f7fff700 1 -- 172.21.15.16:0/1567495468 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc9f8059070 msgr2=0x7fc9f8115a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:00.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.816+0000 7fc9f7fff700 1 --2- 172.21.15.16:0/1567495468 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc9f8059070 0x7fc9f8115a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:00.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.816+0000 7fc9f7fff700 1 -- 172.21.15.16:0/1567495468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f0009a20 con 0x7fc9f8125bb0 2023-09-30T22:27:00.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.816+0000 7fc9f6ffd700 1 --2- 172.21.15.16:0/1567495468 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc9f81251c0 0x7fc9f8111b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:00.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.816+0000 7fc9f77fe700 1 --2- 172.21.15.16:0/1567495468 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc9f8059070 0x7fc9f8115a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:00.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.816+0000 7fc9f7fff700 1 --2- 172.21.15.16:0/1567495468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9f8125bb0 0x7fc9f8112010 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fc9f0009fc0 tx=0x7fc9f000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:00.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.817+0000 7fc9ef7fe700 1 -- 172.21.15.16:0/1567495468 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f0009d20 con 0x7fc9f8125bb0 2023-09-30T22:27:00.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.817+0000 7fc9ef7fe700 1 -- 172.21.15.16:0/1567495468 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9f000f040 con 0x7fc9f8125bb0 2023-09-30T22:27:00.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.817+0000 7fc9fdb02700 1 -- 172.21.15.16:0/1567495468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9e8009370 con 0x7fc9f8125bb0 2023-09-30T22:27:00.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.817+0000 7fc9ef7fe700 1 -- 172.21.15.16:0/1567495468 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f000cde0 con 0x7fc9f8125bb0 2023-09-30T22:27:00.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.817+0000 7fc9fdb02700 1 -- 172.21.15.16:0/1567495468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9f8114fd0 con 0x7fc9f8125bb0 2023-09-30T22:27:00.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.818+0000 7fc9ef7fe700 1 -- 172.21.15.16:0/1567495468 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fc9f0013610 con 0x7fc9f8125bb0 2023-09-30T22:27:00.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.818+0000 7fc9ef7fe700 1 -- 172.21.15.16:0/1567495468 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fc9f003b8a0 con 0x7fc9f8125bb0 2023-09-30T22:27:00.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.818+0000 7fc9fdb02700 1 -- 172.21.15.16:0/1567495468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9f81233e0 con 0x7fc9f8125bb0 2023-09-30T22:27:00.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:00.823+0000 7fc9ef7fe700 1 -- 172.21.15.16:0/1567495468 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fc9f001d460 con 0x7fc9f8125bb0 2023-09-30T22:27:01.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.044+0000 7fc9fdb02700 1 -- 172.21.15.16:0/1567495468 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fc9f805a260 con 0x7fc9f8125bb0 2023-09-30T22:27:01.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.046+0000 7fc9ef7fe700 1 -- 172.21.15.16:0/1567495468 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v4) v1 ==== 74+0+64130 (secure 0 0 0) 0x7fc9f00077b0 con 0x7fc9f8125bb0 2023-09-30T22:27:01.048 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:27:01.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.048+0000 7fc9fdb02700 1 -- 172.21.15.16:0/1567495468 >> v1:172.21.15.16:6816/118285 conn(0x7fc9d802bb60 legacy=0x7fc9d802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.048+0000 7fc9fdb02700 1 -- 172.21.15.16:0/1567495468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9f8125bb0 msgr2=0x7fc9f8112010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.048+0000 7fc9fdb02700 1 --2- 172.21.15.16:0/1567495468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9f8125bb0 0x7fc9f8112010 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fc9f0009fc0 tx=0x7fc9f000b040).stop 2023-09-30T22:27:01.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.048+0000 7fc9fdb02700 1 -- 172.21.15.16:0/1567495468 shutdown_connections 2023-09-30T22:27:01.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.048+0000 7fc9fdb02700 1 --2- 172.21.15.16:0/1567495468 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc9f8059070 0x7fc9f8115a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:01.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.048+0000 7fc9fdb02700 1 --2- 172.21.15.16:0/1567495468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9f8125bb0 0x7fc9f8112010 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:01.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.048+0000 7fc9fdb02700 1 --2- 172.21.15.16:0/1567495468 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc9f81251c0 0x7fc9f8111b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:01.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.048+0000 7fc9fdb02700 1 -- 172.21.15.16:0/1567495468 >> 172.21.15.16:0/1567495468 conn(0x7fc9f8097470 msgr2=0x7fc9f8128be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:01.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.048+0000 7fc9fdb02700 1 -- 172.21.15.16:0/1567495468 shutdown_connections 2023-09-30T22:27:01.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.048+0000 7fc9fdb02700 1 -- 172.21.15.16:0/1567495468 wait complete. 2023-09-30T22:27:01.062 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":4,"active_gid":4098,"active_name":"x","active_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6816","nonce":118285}]},"active_addr":"172.21.15.16:6816/118285","active_change":"2023-09-30T22:26:40.448871+0000","active_mgr_features":4540138314316775423,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":1101648925}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":2249754484}]}]} 2023-09-30T22:27:01.064 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-09-30T22:27:01.064 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-09-30T22:27:01.064 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:27:01.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.193+0000 7ffa076f8700 1 Processor -- start 2023-09-30T22:27:01.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.193+0000 7ffa076f8700 1 -- start start 2023-09-30T22:27:01.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.194+0000 7ffa076f8700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ffa00091b00 0x7ffa0012c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:01.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.194+0000 7ffa076f8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa0012ce60 0x7ffa0012f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:01.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.194+0000 7ffa076f8700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa00096760 con 0x7ffa00096380 2023-09-30T22:27:01.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.194+0000 7ffa076f8700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa00091ff0 con 0x7ffa00091b00 2023-09-30T22:27:01.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.194+0000 7ffa076f8700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa00092c80 con 0x7ffa0012ce60 2023-09-30T22:27:01.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.194+0000 7ffa04c93700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ffa00091b00 0x7ffa0012c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:01.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.194+0000 7ffa05c95700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa0012ce60 0x7ffa0012f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:01.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.194+0000 7ffa05c95700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa0012ce60 0x7ffa0012f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:42106/0 (socket says 172.21.15.16:42106) 2023-09-30T22:27:01.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.194+0000 7ffa04c93700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ffa00091b00 0x7ffa0012c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54398/0 (socket says 172.21.15.16:54398) 2023-09-30T22:27:01.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.194+0000 7ffa05494700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ffa00096380 0x7ffa00094730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51418/0 (socket says 172.21.15.16:51418) 2023-09-30T22:27:01.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.194+0000 7ffa05c95700 1 -- 172.21.15.16:0/3177666687 learned_addr learned my addr 172.21.15.16:0/3177666687 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:01.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.194+0000 7ffa04c93700 1 -- 172.21.15.16:0/3177666687 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa0012ce60 msgr2=0x7ffa0012f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:01.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.194+0000 7ffa04c93700 1 --2- 172.21.15.16:0/3177666687 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa0012ce60 0x7ffa0012f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:01.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.194+0000 7ffa04c93700 1 -- 172.21.15.16:0/3177666687 >> v1:172.21.15.16:6789/0 conn(0x7ffa00096380 legacy=0x7ffa00094730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.194+0000 7ff9f7fff700 1 -- 172.21.15.16:0/3177666687 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4245926222 0 0) 0x7ffa00096760 con 0x7ffa00096380 2023-09-30T22:27:01.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.194+0000 7ffa04c93700 1 -- 172.21.15.16:0/3177666687 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa00094e40 con 0x7ffa00091b00 2023-09-30T22:27:01.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.194+0000 7ffa05c95700 1 --2- 172.21.15.16:0/3177666687 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa0012ce60 0x7ffa0012f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:01.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.194+0000 7ffa04c93700 1 --2- 172.21.15.16:0/3177666687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa00091b00 0x7ffa0012c970 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7ff9fc009760 tx=0x7ff9fc009060).ready entity=mon.0 client_cookie=b6c8f8402d238cd4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:01.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.195+0000 7ff9f7fff700 1 -- 172.21.15.16:0/3177666687 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc00f510 con 0x7ffa00091b00 2023-09-30T22:27:01.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.195+0000 7ff9f7fff700 1 -- 172.21.15.16:0/3177666687 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9fc00f670 con 0x7ffa00091b00 2023-09-30T22:27:01.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.195+0000 7ff9f7fff700 1 -- 172.21.15.16:0/3177666687 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc00f9f0 con 0x7ffa00091b00 2023-09-30T22:27:01.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.195+0000 7ffa076f8700 1 -- 172.21.15.16:0/3177666687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa00091b00 msgr2=0x7ffa0012c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.195+0000 7ffa076f8700 1 --2- 172.21.15.16:0/3177666687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa00091b00 0x7ffa0012c970 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7ff9fc009760 tx=0x7ff9fc009060).stop 2023-09-30T22:27:01.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.195+0000 7ffa076f8700 1 -- 172.21.15.16:0/3177666687 shutdown_connections 2023-09-30T22:27:01.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.195+0000 7ffa076f8700 1 --2- 172.21.15.16:0/3177666687 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa0012ce60 0x7ffa0012f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:01.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.195+0000 7ffa076f8700 1 --2- 172.21.15.16:0/3177666687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa00091b00 0x7ffa0012c970 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:01.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.195+0000 7ffa076f8700 1 -- 172.21.15.16:0/3177666687 >> 172.21.15.16:0/3177666687 conn(0x7ffa00097470 msgr2=0x7ffa00097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:01.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.195+0000 7ffa076f8700 1 -- 172.21.15.16:0/3177666687 shutdown_connections 2023-09-30T22:27:01.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.195+0000 7ffa076f8700 1 -- 172.21.15.16:0/3177666687 wait complete. 2023-09-30T22:27:01.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.196+0000 7ffa076f8700 1 Processor -- start 2023-09-30T22:27:01.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.196+0000 7ffa076f8700 1 -- start start 2023-09-30T22:27:01.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.197+0000 7ffa076f8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa00091b00 0x7ffa00123cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:01.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.197+0000 7ffa076f8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa00096380 0x7ffa001241e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:01.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.197+0000 7ffa076f8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa0012ce60 0x7ffa001246d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:01.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.197+0000 7ffa076f8700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa00112a30 con 0x7ffa00096380 2023-09-30T22:27:01.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.197+0000 7ffa076f8700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0012c190 con 0x7ffa0012ce60 2023-09-30T22:27:01.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.197+0000 7ffa076f8700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffa00098da0 con 0x7ffa00091b00 2023-09-30T22:27:01.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.197+0000 7ffa05c95700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa0012ce60 0x7ffa001246d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:01.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.197+0000 7ffa05494700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa00091b00 0x7ffa00123cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:01.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.197+0000 7ffa05c95700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa0012ce60 0x7ffa001246d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:42120/0 (socket says 172.21.15.16:42120) 2023-09-30T22:27:01.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.197+0000 7ffa05494700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa00091b00 0x7ffa00123cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:40070/0 (socket says 172.21.15.16:40070) 2023-09-30T22:27:01.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.197+0000 7ffa05c95700 1 -- 172.21.15.16:0/650699544 learned_addr learned my addr 172.21.15.16:0/650699544 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:01.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.197+0000 7ffa04c93700 1 --2- 172.21.15.16:0/650699544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa00096380 0x7ffa001241e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:01.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.198+0000 7ffa05c95700 1 -- 172.21.15.16:0/650699544 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa00091b00 msgr2=0x7ffa00123cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.198+0000 7ffa05c95700 1 --2- 172.21.15.16:0/650699544 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa00091b00 0x7ffa00123cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:01.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.198+0000 7ffa05c95700 1 -- 172.21.15.16:0/650699544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa00096380 msgr2=0x7ffa001241e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.198+0000 7ffa05c95700 1 --2- 172.21.15.16:0/650699544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa00096380 0x7ffa001241e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:01.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.198+0000 7ffa05c95700 1 -- 172.21.15.16:0/650699544 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f800bcf0 con 0x7ffa0012ce60 2023-09-30T22:27:01.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.198+0000 7ffa04c93700 1 --2- 172.21.15.16:0/650699544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa00096380 0x7ffa001241e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.198+0000 7ffa05c95700 1 --2- 172.21.15.16:0/650699544 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa0012ce60 0x7ffa001246d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7ff9f80094c0 tx=0x7ff9f800d040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.198+0000 7ff9f57fa700 1 -- 172.21.15.16:0/650699544 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f8010040 con 0x7ffa0012ce60 2023-09-30T22:27:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.198+0000 7ff9f57fa700 1 -- 172.21.15.16:0/650699544 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9f800b2e0 con 0x7ffa0012ce60 2023-09-30T22:27:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.198+0000 7ffa076f8700 1 -- 172.21.15.16:0/650699544 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9fc0092a0 con 0x7ffa0012ce60 2023-09-30T22:27:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.198+0000 7ff9f57fa700 1 -- 172.21.15.16:0/650699544 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f800fc00 con 0x7ffa0012ce60 2023-09-30T22:27:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.198+0000 7ffa076f8700 1 -- 172.21.15.16:0/650699544 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa00112490 con 0x7ffa0012ce60 2023-09-30T22:27:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.199+0000 7ff9f57fa700 1 -- 172.21.15.16:0/650699544 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7ff9f8009610 con 0x7ffa0012ce60 2023-09-30T22:27:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.199+0000 7ff9f57fa700 1 -- 172.21.15.16:0/650699544 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3690+0+0 (secure 0 0 0) 0x7ff9f803b820 con 0x7ffa0012ce60 2023-09-30T22:27:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.199+0000 7ffa076f8700 1 -- 172.21.15.16:0/650699544 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa0005a260 con 0x7ffa0012ce60 2023-09-30T22:27:01.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.204+0000 7ff9f57fa700 1 -- 172.21.15.16:0/650699544 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7ff9f8013940 con 0x7ffa0012ce60 2023-09-30T22:27:01.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.379+0000 7ffa076f8700 1 -- 172.21.15.16:0/650699544 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffa00093980 con 0x7ffa0012ce60 2023-09-30T22:27:01.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.380+0000 7ff9f57fa700 1 -- 172.21.15.16:0/650699544 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+11341 (secure 0 0 0) 0x7ff9f8013350 con 0x7ffa0012ce60 2023-09-30T22:27:01.381 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:27:01.382 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":16,"fsid":"7ede3e14-2bd0-4726-9437-6d13c90cd342","created":"2023-09-30T22:26:35.483697+0000","modified":"2023-09-30T22:26:52.675888+0000","last_up_change":"2023-09-30T22:26:44.227199+0000","last_in_change":"2023-09-30T22:26:37.120906+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-09-30T22:26:44.285133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:26:45.132921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:26:49.353683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:26:50.655685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"19946985-0ee2-487f-9347-929670dbe5ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6804","nonce":118824}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6805","nonce":118824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6807","nonce":118824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6806","nonce":118824}]},"public_addr":"172.21.15.16:6804/118824","cluster_addr":"172.21.15.16:6805/118824","heartbeat_back_addr":"172.21.15.16:6807/118824","heartbeat_front_addr":"172.21.15.16:6806/118824","state":["exists","up"]},{"osd":1,"uuid":"8efd4dbc-6ae4-4b82-b530-3b3eac2be445","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6812","nonce":118832}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6813","nonce":118832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6815","nonce":118832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6814","nonce":118832}]},"public_addr":"172.21.15.16:6812/118832","cluster_addr":"172.21.15.16:6813/118832","heartbeat_back_addr":"172.21.15.16:6815/118832","heartbeat_front_addr":"172.21.15.16:6814/118832","state":["exists","up"]},{"osd":2,"uuid":"b47bb1ce-27af-4c0c-868a-b76ebd0d9499","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6808","nonce":118840}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6809","nonce":118840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6811","nonce":118840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6810","nonce":118840}]},"public_addr":"172.21.15.16:6808/118840","cluster_addr":"172.21.15.16:6809/118840","heartbeat_back_addr":"172.21.15.16:6811/118840","heartbeat_front_addr":"172.21.15.16:6810/118840","state":["exists","up"]},{"osd":3,"uuid":"03b53d64-d10d-448a-a081-39ecceab589b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6800","nonce":118823}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6801","nonce":118823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6803","nonce":118823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6802","nonce":118823}]},"public_addr":"172.21.15.16:6800/118823","cluster_addr":"172.21.15.16:6801/118823","heartbeat_back_addr":"172.21.15.16:6803/118823","heartbeat_front_addr":"172.21.15.16:6802/118823","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.305953+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.307488+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.341023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.192071+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:27:01.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.382+0000 7ffa076f8700 1 -- 172.21.15.16:0/650699544 >> v1:172.21.15.16:6816/118285 conn(0x7ff9e402bb60 legacy=0x7ff9e402e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.382+0000 7ffa076f8700 1 -- 172.21.15.16:0/650699544 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa0012ce60 msgr2=0x7ffa001246d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.382+0000 7ffa076f8700 1 --2- 172.21.15.16:0/650699544 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa0012ce60 0x7ffa001246d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7ff9f80094c0 tx=0x7ff9f800d040).stop 2023-09-30T22:27:01.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.382+0000 7ffa076f8700 1 -- 172.21.15.16:0/650699544 shutdown_connections 2023-09-30T22:27:01.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.382+0000 7ffa076f8700 1 --2- 172.21.15.16:0/650699544 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa0012ce60 0x7ffa001246d0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:01.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.382+0000 7ffa076f8700 1 --2- 172.21.15.16:0/650699544 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa00091b00 0x7ffa00123cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:01.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.382+0000 7ffa076f8700 1 --2- 172.21.15.16:0/650699544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa00096380 0x7ffa001241e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:01.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.382+0000 7ffa076f8700 1 -- 172.21.15.16:0/650699544 >> 172.21.15.16:0/650699544 conn(0x7ffa00097470 msgr2=0x7ffa00059070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:01.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.382+0000 7ffa076f8700 1 -- 172.21.15.16:0/650699544 shutdown_connections 2023-09-30T22:27:01.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.382+0000 7ffa076f8700 1 -- 172.21.15.16:0/650699544 wait complete. 2023-09-30T22:27:01.397 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-09-30T22:27:01.397 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:27:01.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.526+0000 7f0f8835b700 1 Processor -- start 2023-09-30T22:27:01.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.526+0000 7f0f8835b700 1 -- start start 2023-09-30T22:27:01.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.526+0000 7f0f8835b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0f80127390 0x7f0f8012fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:01.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.527+0000 7f0f8835b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0f80127fa0 0x7f0f80132170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:01.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.527+0000 7f0f8835b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f80091b60 con 0x7f0f80059070 2023-09-30T22:27:01.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.527+0000 7f0f8835b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f80096940 con 0x7f0f80127390 2023-09-30T22:27:01.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.527+0000 7f0f8835b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0f80059c00 con 0x7f0f80127fa0 2023-09-30T22:27:01.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.527+0000 7f0f858f6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0f80127390 0x7f0f8012fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:01.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.527+0000 7f0f868f8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0f80127fa0 0x7f0f80132170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:01.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.527+0000 7f0f860f7700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0f80059070 0x7f0f80127890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51434/0 (socket says 172.21.15.16:51434) 2023-09-30T22:27:01.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.527+0000 7f0f858f6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0f80127390 0x7f0f8012fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:42126/0 (socket says 172.21.15.16:42126) 2023-09-30T22:27:01.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.527+0000 7f0f860f7700 1 -- 172.21.15.16:0/1269655938 learned_addr learned my addr 172.21.15.16:0/1269655938 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:27:01.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.527+0000 7f0f868f8700 1 -- 172.21.15.16:0/1269655938 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0f80127390 msgr2=0x7f0f8012fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:01.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.527+0000 7f0f850f5700 1 -- 172.21.15.16:0/1269655938 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3362732212 0 0) 0x7f0f80091b60 con 0x7f0f80059070 2023-09-30T22:27:01.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.527+0000 7f0f868f8700 1 --2- 172.21.15.16:0/1269655938 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0f80127390 0x7f0f8012fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:01.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.527+0000 7f0f868f8700 1 -- 172.21.15.16:0/1269655938 >> v1:172.21.15.16:6789/0 conn(0x7f0f80059070 legacy=0x7f0f80127890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.527+0000 7f0f868f8700 1 -- 172.21.15.16:0/1269655938 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f800929d0 con 0x7f0f80127fa0 2023-09-30T22:27:01.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.527+0000 7f0f858f6700 1 --2- 172.21.15.16:0/1269655938 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0f80127390 0x7f0f8012fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:01.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.527+0000 7f0f868f8700 1 --2- 172.21.15.16:0/1269655938 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0f80127fa0 0x7f0f80132170 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f0f78009e90 tx=0x7f0f78009780).ready entity=mon.2 client_cookie=3dad40017bd8d9b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:01.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.527+0000 7f0f850f5700 1 -- 172.21.15.16:0/1269655938 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f7800e070 con 0x7f0f80127fa0 2023-09-30T22:27:01.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.527+0000 7f0f850f5700 1 -- 172.21.15.16:0/1269655938 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f7800bb60 con 0x7f0f80127fa0 2023-09-30T22:27:01.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.527+0000 7f0f850f5700 1 -- 172.21.15.16:0/1269655938 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f78012410 con 0x7f0f80127fa0 2023-09-30T22:27:01.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.528+0000 7f0f8835b700 1 -- 172.21.15.16:0/1269655938 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0f80127fa0 msgr2=0x7f0f80132170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.528+0000 7f0f8835b700 1 --2- 172.21.15.16:0/1269655938 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0f80127fa0 0x7f0f80132170 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f0f78009e90 tx=0x7f0f78009780).stop 2023-09-30T22:27:01.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.528+0000 7f0f8835b700 1 -- 172.21.15.16:0/1269655938 shutdown_connections 2023-09-30T22:27:01.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.528+0000 7f0f8835b700 1 --2- 172.21.15.16:0/1269655938 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0f80127fa0 0x7f0f80132170 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:01.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.528+0000 7f0f8835b700 1 --2- 172.21.15.16:0/1269655938 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0f80127390 0x7f0f8012fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:01.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.528+0000 7f0f8835b700 1 -- 172.21.15.16:0/1269655938 >> 172.21.15.16:0/1269655938 conn(0x7f0f80097470 msgr2=0x7f0f80097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:01.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.528+0000 7f0f8835b700 1 -- 172.21.15.16:0/1269655938 shutdown_connections 2023-09-30T22:27:01.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.528+0000 7f0f8835b700 1 -- 172.21.15.16:0/1269655938 wait complete. 2023-09-30T22:27:01.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.529+0000 7f0f8835b700 1 Processor -- start 2023-09-30T22:27:01.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.529+0000 7f0f8835b700 1 -- start start 2023-09-30T22:27:01.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.529+0000 7f0f8835b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f80059070 0x7f0f80125090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:01.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.529+0000 7f0f8835b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0f80127390 0x7f0f8012dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:01.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.530+0000 7f0f8835b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0f80127fa0 0x7f0f8012e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:01.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.530+0000 7f0f8835b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8012fe50 con 0x7f0f80059070 2023-09-30T22:27:01.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.530+0000 7f0f8835b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8012f120 con 0x7f0f80127390 2023-09-30T22:27:01.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.530+0000 7f0f8835b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0f80131d50 con 0x7f0f80127fa0 2023-09-30T22:27:01.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.530+0000 7f0f858f6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0f80127390 0x7f0f8012dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:01.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.530+0000 7f0f858f6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0f80127390 0x7f0f8012dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:42134/0 (socket says 172.21.15.16:42134) 2023-09-30T22:27:01.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.530+0000 7f0f858f6700 1 -- 172.21.15.16:0/3860463909 learned_addr learned my addr 172.21.15.16:0/3860463909 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:01.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.530+0000 7f0f860f7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f80059070 0x7f0f80125090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:01.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.530+0000 7f0f868f8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0f80127fa0 0x7f0f8012e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:01.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.530+0000 7f0f858f6700 1 -- 172.21.15.16:0/3860463909 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0f80127fa0 msgr2=0x7f0f8012e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.530+0000 7f0f858f6700 1 --2- 172.21.15.16:0/3860463909 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0f80127fa0 0x7f0f8012e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:01.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.530+0000 7f0f858f6700 1 -- 172.21.15.16:0/3860463909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f80059070 msgr2=0x7f0f80125090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.530+0000 7f0f858f6700 1 --2- 172.21.15.16:0/3860463909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f80059070 0x7f0f80125090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:01.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.530+0000 7f0f858f6700 1 -- 172.21.15.16:0/3860463909 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f7c0092a0 con 0x7f0f80127390 2023-09-30T22:27:01.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.530+0000 7f0f868f8700 1 --2- 172.21.15.16:0/3860463909 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0f80127fa0 0x7f0f8012e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:01.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.530+0000 7f0f860f7700 1 --2- 172.21.15.16:0/3860463909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f80059070 0x7f0f80125090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:01.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.530+0000 7f0f858f6700 1 --2- 172.21.15.16:0/3860463909 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0f80127390 0x7f0f8012dd30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f0f7c00ea00 tx=0x7f0f7c009060).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:01.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.530+0000 7f0f767fc700 1 -- 172.21.15.16:0/3860463909 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c00c6a0 con 0x7f0f80127390 2023-09-30T22:27:01.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.530+0000 7f0f767fc700 1 -- 172.21.15.16:0/3860463909 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f7c00c800 con 0x7f0f80127390 2023-09-30T22:27:01.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.530+0000 7f0f8835b700 1 -- 172.21.15.16:0/3860463909 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f78009a20 con 0x7f0f80127390 2023-09-30T22:27:01.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.530+0000 7f0f767fc700 1 -- 172.21.15.16:0/3860463909 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c00cb80 con 0x7f0f80127390 2023-09-30T22:27:01.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.530+0000 7f0f8835b700 1 -- 172.21.15.16:0/3860463909 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f801300a0 con 0x7f0f80127390 2023-09-30T22:27:01.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.531+0000 7f0f767fc700 1 -- 172.21.15.16:0/3860463909 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f0f7c01b400 con 0x7f0f80127390 2023-09-30T22:27:01.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.531+0000 7f0f767fc700 1 -- 172.21.15.16:0/3860463909 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f0f7c03b500 con 0x7f0f80127390 2023-09-30T22:27:01.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.531+0000 7f0f8835b700 1 -- 172.21.15.16:0/3860463909 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f80064da0 con 0x7f0f80127390 2023-09-30T22:27:01.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.536+0000 7f0f767fc700 1 -- 172.21.15.16:0/3860463909 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f0f7c016070 con 0x7f0f80127390 2023-09-30T22:27:01.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.714+0000 7f0f8835b700 1 -- 172.21.15.16:0/3860463909 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0f8012f9f0 con 0x7f0f80127390 2023-09-30T22:27:01.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.715+0000 7f0f767fc700 1 -- 172.21.15.16:0/3860463909 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+11341 (secure 0 0 0) 0x7f0f7c016020 con 0x7f0f80127390 2023-09-30T22:27:01.717 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:27:01.717 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":16,"fsid":"7ede3e14-2bd0-4726-9437-6d13c90cd342","created":"2023-09-30T22:26:35.483697+0000","modified":"2023-09-30T22:26:52.675888+0000","last_up_change":"2023-09-30T22:26:44.227199+0000","last_in_change":"2023-09-30T22:26:37.120906+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-09-30T22:26:44.285133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:26:45.132921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:26:49.353683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:26:50.655685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"19946985-0ee2-487f-9347-929670dbe5ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6804","nonce":118824}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6805","nonce":118824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6807","nonce":118824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6806","nonce":118824}]},"public_addr":"172.21.15.16:6804/118824","cluster_addr":"172.21.15.16:6805/118824","heartbeat_back_addr":"172.21.15.16:6807/118824","heartbeat_front_addr":"172.21.15.16:6806/118824","state":["exists","up"]},{"osd":1,"uuid":"8efd4dbc-6ae4-4b82-b530-3b3eac2be445","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6812","nonce":118832}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6813","nonce":118832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6815","nonce":118832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6814","nonce":118832}]},"public_addr":"172.21.15.16:6812/118832","cluster_addr":"172.21.15.16:6813/118832","heartbeat_back_addr":"172.21.15.16:6815/118832","heartbeat_front_addr":"172.21.15.16:6814/118832","state":["exists","up"]},{"osd":2,"uuid":"b47bb1ce-27af-4c0c-868a-b76ebd0d9499","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6808","nonce":118840}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6809","nonce":118840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6811","nonce":118840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6810","nonce":118840}]},"public_addr":"172.21.15.16:6808/118840","cluster_addr":"172.21.15.16:6809/118840","heartbeat_back_addr":"172.21.15.16:6811/118840","heartbeat_front_addr":"172.21.15.16:6810/118840","state":["exists","up"]},{"osd":3,"uuid":"03b53d64-d10d-448a-a081-39ecceab589b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6800","nonce":118823}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6801","nonce":118823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6803","nonce":118823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6802","nonce":118823}]},"public_addr":"172.21.15.16:6800/118823","cluster_addr":"172.21.15.16:6801/118823","heartbeat_back_addr":"172.21.15.16:6803/118823","heartbeat_front_addr":"172.21.15.16:6802/118823","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.305953+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.307488+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.341023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.192071+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:27:01.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.717+0000 7f0f8835b700 1 -- 172.21.15.16:0/3860463909 >> v1:172.21.15.16:6816/118285 conn(0x7f0f6402bb60 legacy=0x7f0f6402e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.717+0000 7f0f8835b700 1 -- 172.21.15.16:0/3860463909 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0f80127390 msgr2=0x7f0f8012dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.717+0000 7f0f8835b700 1 --2- 172.21.15.16:0/3860463909 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0f80127390 0x7f0f8012dd30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f0f7c00ea00 tx=0x7f0f7c009060).stop 2023-09-30T22:27:01.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.717+0000 7f0f8835b700 1 -- 172.21.15.16:0/3860463909 shutdown_connections 2023-09-30T22:27:01.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.717+0000 7f0f8835b700 1 --2- 172.21.15.16:0/3860463909 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0f80127fa0 0x7f0f8012e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:01.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.717+0000 7f0f8835b700 1 --2- 172.21.15.16:0/3860463909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f80059070 0x7f0f80125090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:01.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.717+0000 7f0f8835b700 1 --2- 172.21.15.16:0/3860463909 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0f80127390 0x7f0f8012dd30 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:01.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.717+0000 7f0f8835b700 1 -- 172.21.15.16:0/3860463909 >> 172.21.15.16:0/3860463909 conn(0x7f0f80097470 msgr2=0x7f0f80098780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:01.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.718+0000 7f0f8835b700 1 -- 172.21.15.16:0/3860463909 shutdown_connections 2023-09-30T22:27:01.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.718+0000 7f0f8835b700 1 -- 172.21.15.16:0/3860463909 wait complete. 2023-09-30T22:27:01.732 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-09-30T22:27:01.733 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-09-30T22:27:01.733 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-09-30T22:27:01.734 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-09-30T22:27:01.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.872+0000 7f775bc83700 1 Processor -- start 2023-09-30T22:27:01.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.873+0000 7f775bc83700 1 -- start start 2023-09-30T22:27:01.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.873+0000 7f775bc83700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f77540965e0 0x7f77540969c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:01.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.873+0000 7f775bc83700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7754096eb0 0x7f7754067de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:01.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.873+0000 7f775bc83700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f77540682d0 0x7f7754068740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:01.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.873+0000 7f775bc83700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7754068c30 con 0x7f77540965e0 2023-09-30T22:27:01.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.873+0000 7f775bc83700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77540933b0 con 0x7f77540682d0 2023-09-30T22:27:01.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.873+0000 7f775bc83700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7754092190 con 0x7f7754096eb0 2023-09-30T22:27:01.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.873+0000 7f7759a1f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f77540965e0 0x7f77540969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:01.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.874+0000 7f7759a1f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f77540965e0 0x7f77540969c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54432/0 (socket says 172.21.15.16:54432) 2023-09-30T22:27:01.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.874+0000 7f7759a1f700 1 -- 172.21.15.16:0/772204649 learned_addr learned my addr 172.21.15.16:0/772204649 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:01.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.874+0000 7f775a220700 1 --2- 172.21.15.16:0/772204649 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f77540682d0 0x7f7754068740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:01.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.874+0000 7f7759a1f700 1 -- 172.21.15.16:0/772204649 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7754096eb0 msgr2=0x7f7754067de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:01.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.874+0000 7f7759a1f700 1 --2- 172.21.15.16:0/772204649 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7754096eb0 0x7f7754067de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-09-30T22:27:01.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.874+0000 7f7759a1f700 1 -- 172.21.15.16:0/772204649 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f77540682d0 msgr2=0x7f7754068740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:01.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.874+0000 7f7759a1f700 1 --2- 172.21.15.16:0/772204649 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f77540682d0 0x7f7754068740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:01.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.874+0000 7f7759a1f700 1 -- 172.21.15.16:0/772204649 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7754069a20 con 0x7f77540965e0 2023-09-30T22:27:01.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.874+0000 7f775a220700 1 --2- 172.21.15.16:0/772204649 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f77540682d0 0x7f7754068740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:01.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.874+0000 7f7759a1f700 1 --2- 172.21.15.16:0/772204649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77540965e0 0x7f77540969c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f7750006fd0 tx=0x7f77500090f0).ready entity=mon.0 client_cookie=da7a963ae2530ea6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:01.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.874+0000 7f7758a1d700 1 -- 172.21.15.16:0/772204649 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775000b580 con 0x7f77540965e0 2023-09-30T22:27:01.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.874+0000 7f7758a1d700 1 -- 172.21.15.16:0/772204649 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f775000b6e0 con 0x7f77540965e0 2023-09-30T22:27:01.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.874+0000 7f7758a1d700 1 -- 172.21.15.16:0/772204649 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775000ba60 con 0x7f77540965e0 2023-09-30T22:27:01.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.875+0000 7f775bc83700 1 -- 172.21.15.16:0/772204649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77540965e0 msgr2=0x7f77540969c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.875+0000 7f775bc83700 1 --2- 172.21.15.16:0/772204649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77540965e0 0x7f77540969c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f7750006fd0 tx=0x7f77500090f0).stop 2023-09-30T22:27:01.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.875+0000 7f775bc83700 1 -- 172.21.15.16:0/772204649 shutdown_connections 2023-09-30T22:27:01.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.875+0000 7f775bc83700 1 --2- 172.21.15.16:0/772204649 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f77540682d0 0x7f7754068740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:01.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.875+0000 7f775bc83700 1 --2- 172.21.15.16:0/772204649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77540965e0 0x7f77540969c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:01.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.875+0000 7f775bc83700 1 --2- 172.21.15.16:0/772204649 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7754096eb0 0x7f7754067de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-09-30T22:27:01.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.876+0000 7f775bc83700 1 -- 172.21.15.16:0/772204649 >> 172.21.15.16:0/772204649 conn(0x7f775409a220 msgr2=0x7f775409a630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:01.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.876+0000 7f775bc83700 1 -- 172.21.15.16:0/772204649 shutdown_connections 2023-09-30T22:27:01.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.876+0000 7f775bc83700 1 -- 172.21.15.16:0/772204649 wait complete. 2023-09-30T22:27:01.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.877+0000 7f775bc83700 1 Processor -- start 2023-09-30T22:27:01.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.877+0000 7f775bc83700 1 -- start start 2023-09-30T22:27:01.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.877+0000 7f775bc83700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f77540682d0 0x7f77541384e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:01.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.877+0000 7f775bc83700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77540965e0 0x7f77541389d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:01.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.877+0000 7f775bc83700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7754096eb0 0x7f7754138ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:01.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.877+0000 7f775bc83700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f775411c250 con 0x7f77540965e0 2023-09-30T22:27:01.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.877+0000 7f775bc83700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7754091bf0 con 0x7f77540682d0 2023-09-30T22:27:01.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.877+0000 7f775bc83700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7754066770 con 0x7f7754096eb0 2023-09-30T22:27:01.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.877+0000 7f7759a1f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f77540682d0 0x7f77541384e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:01.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.877+0000 7f7759a1f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f77540682d0 0x7f77541384e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:42152/0 (socket says 172.21.15.16:42152) 2023-09-30T22:27:01.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.877+0000 7f7759a1f700 1 -- 172.21.15.16:0/2307504703 learned_addr learned my addr 172.21.15.16:0/2307504703 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:01.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.877+0000 7f775a220700 1 --2- 172.21.15.16:0/2307504703 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7754096eb0 0x7f7754138ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:01.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.878+0000 7f775921e700 1 --2- 172.21.15.16:0/2307504703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77540965e0 0x7f77541389d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:01.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.878+0000 7f7759a1f700 1 -- 172.21.15.16:0/2307504703 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7754096eb0 msgr2=0x7f7754138ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.878+0000 7f7759a1f700 1 --2- 172.21.15.16:0/2307504703 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7754096eb0 0x7f7754138ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:01.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.878+0000 7f7759a1f700 1 -- 172.21.15.16:0/2307504703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77540965e0 msgr2=0x7f77541389d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.878+0000 7f7759a1f700 1 --2- 172.21.15.16:0/2307504703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77540965e0 0x7f77541389d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:01.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.878+0000 7f7759a1f700 1 -- 172.21.15.16:0/2307504703 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f774c009a20 con 0x7f77540682d0 2023-09-30T22:27:01.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.878+0000 7f7759a1f700 1 --2- 172.21.15.16:0/2307504703 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f77540682d0 0x7f77541384e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f7750009130 tx=0x7f7750017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:01.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.878+0000 7f7749ffb700 1 -- 172.21.15.16:0/2307504703 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775000bea0 con 0x7f77540682d0 2023-09-30T22:27:01.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.878+0000 7f7749ffb700 1 -- 172.21.15.16:0/2307504703 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7750006a40 con 0x7f77540682d0 2023-09-30T22:27:01.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.878+0000 7f775bc83700 1 -- 172.21.15.16:0/2307504703 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7750009370 con 0x7f77540682d0 2023-09-30T22:27:01.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.878+0000 7f7749ffb700 1 -- 172.21.15.16:0/2307504703 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775001c420 con 0x7f77540682d0 2023-09-30T22:27:01.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.879+0000 7f775bc83700 1 -- 172.21.15.16:0/2307504703 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f775411ca60 con 0x7f77540682d0 2023-09-30T22:27:01.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.880+0000 7f775bc83700 1 -- 172.21.15.16:0/2307504703 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f775405a260 con 0x7f77540682d0 2023-09-30T22:27:01.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.881+0000 7f7749ffb700 1 -- 172.21.15.16:0/2307504703 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f775001c790 con 0x7f77540682d0 2023-09-30T22:27:01.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.881+0000 7f7749ffb700 1 -- 172.21.15.16:0/2307504703 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f775003d020 con 0x7f77540682d0 2023-09-30T22:27:01.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.881+0000 7f7749ffb700 1 -- 172.21.15.16:0/2307504703 --> v1:172.21.15.16:6812/118832 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f77380384e0 con 0x7f77380347f0 2023-09-30T22:27:01.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.881+0000 7f7749ffb700 1 -- 172.21.15.16:0/2307504703 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f775003d360 con 0x7f77540682d0 2023-09-30T22:27:01.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.883+0000 7f7749ffb700 1 -- 172.21.15.16:0/2307504703 <== osd.1 v1:172.21.15.16:6812/118832 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (unknown 0 0 1859947542) 0x7f77380384e0 con 0x7f77380347f0 2023-09-30T22:27:01.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.883+0000 7f9fdc058700 1 Processor -- start 2023-09-30T22:27:01.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.883+0000 7f9fdc058700 1 -- start start 2023-09-30T22:27:01.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.884+0000 7f9fdc058700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9fd40654a0 0x7f9fd4098640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:01.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.884+0000 7f9fdc058700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9fd4098fa0 0x7f9fd4132ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:01.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.884+0000 7f9fdc058700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fd4065ed0 con 0x7f9fd4098b30 2023-09-30T22:27:01.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.884+0000 7f9fdc058700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fd4095880 con 0x7f9fd40654a0 2023-09-30T22:27:01.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.884+0000 7f9fdc058700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd4096120 con 0x7f9fd4098fa0 2023-09-30T22:27:01.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.884+0000 7f9fd9df4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9fd40654a0 0x7f9fd4098640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:01.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.884+0000 7f9fd9df4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9fd40654a0 0x7f9fd4098640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54454/0 (socket says 172.21.15.16:54454) 2023-09-30T22:27:01.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.884+0000 7f9fd9df4700 1 -- 172.21.15.16:0/3152887482 learned_addr learned my addr 172.21.15.16:0/3152887482 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:01.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.884+0000 7f9fd95f3700 1 --1- 172.21.15.16:0/3152887482 >> v1:172.21.15.16:6789/0 conn(0x7f9fd4098b30 0x7f9fd412f470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51442/0 (socket says 172.21.15.16:51442) 2023-09-30T22:27:01.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.884+0000 7f9fda5f5700 1 --2- 172.21.15.16:0/3152887482 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9fd4098fa0 0x7f9fd4132ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:01.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.885+0000 7f9fd9df4700 1 -- 172.21.15.16:0/3152887482 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9fd4098fa0 msgr2=0x7f9fd4132ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:01.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.885+0000 7f9fd9df4700 1 --2- 172.21.15.16:0/3152887482 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9fd4098fa0 0x7f9fd4132ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:01.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.885+0000 7f9fd9df4700 1 -- 172.21.15.16:0/3152887482 >> v1:172.21.15.16:6789/0 conn(0x7f9fd4098b30 legacy=0x7f9fd412f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.885+0000 7f9fd9df4700 1 -- 172.21.15.16:0/3152887482 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd4066280 con 0x7f9fd40654a0 2023-09-30T22:27:01.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.885+0000 7f9fd8df2700 1 -- 172.21.15.16:0/3152887482 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 659445012 0 0) 0x7f9fd4065ed0 con 0x7f9fd4098b30 2023-09-30T22:27:01.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.885+0000 7f9fda5f5700 1 --2- 172.21.15.16:0/3152887482 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9fd4098fa0 0x7f9fd4132ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:01.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.885+0000 7f9fd9df4700 1 --2- 172.21.15.16:0/3152887482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fd40654a0 0x7f9fd4098640 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f9fd0006fd0 tx=0x7f9fd00090f0).ready entity=mon.0 client_cookie=5b76a360a5e74175 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:01.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.885+0000 7f9fd8df2700 1 -- 172.21.15.16:0/3152887482 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd000b580 con 0x7f9fd40654a0 2023-09-30T22:27:01.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.885+0000 7f9fd8df2700 1 -- 172.21.15.16:0/3152887482 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fd000b6e0 con 0x7f9fd40654a0 2023-09-30T22:27:01.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.885+0000 7f9fd8df2700 1 -- 172.21.15.16:0/3152887482 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd000ba60 con 0x7f9fd40654a0 2023-09-30T22:27:01.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.886+0000 7f9fdc058700 1 -- 172.21.15.16:0/3152887482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fd40654a0 msgr2=0x7f9fd4098640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.886+0000 7f9fdc058700 1 --2- 172.21.15.16:0/3152887482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fd40654a0 0x7f9fd4098640 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f9fd0006fd0 tx=0x7f9fd00090f0).stop 2023-09-30T22:27:01.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.886+0000 7f9fdc058700 1 -- 172.21.15.16:0/3152887482 shutdown_connections 2023-09-30T22:27:01.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.886+0000 7f9fdc058700 1 --2- 172.21.15.16:0/3152887482 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9fd4098fa0 0x7f9fd4132ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:01.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.886+0000 7f9fdc058700 1 --2- 172.21.15.16:0/3152887482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fd40654a0 0x7f9fd4098640 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:01.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.886+0000 7f9fdc058700 1 -- 172.21.15.16:0/3152887482 >> 172.21.15.16:0/3152887482 conn(0x7f9fd409a740 msgr2=0x7f9fd409ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:01.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.886+0000 7f9fdc058700 1 -- 172.21.15.16:0/3152887482 shutdown_connections 2023-09-30T22:27:01.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.886+0000 7f9fdc058700 1 -- 172.21.15.16:0/3152887482 wait complete. 2023-09-30T22:27:01.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.888+0000 7f9fdc058700 1 Processor -- start 2023-09-30T22:27:01.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.888+0000 7f9fdc058700 1 -- start start 2023-09-30T22:27:01.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.888+0000 7f9fdc058700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fd40654a0 0x7f9fd413a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:01.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.888+0000 7f9fdc058700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9fd4098b30 0x7f9fd413a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:01.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.888+0000 7f9fdc058700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9fd4098fa0 0x7f9fd413ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:01.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.888+0000 7f9fdc058700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd411e0c0 con 0x7f9fd40654a0 2023-09-30T22:27:01.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.888+0000 7f9fdc058700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd409ccb0 con 0x7f9fd4098b30 2023-09-30T22:27:01.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.888+0000 7f9fdc058700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd4099b00 con 0x7f9fd4098fa0 2023-09-30T22:27:01.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.889+0000 7f9fd95f3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9fd4098b30 0x7f9fd413a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:01.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.889+0000 7f9fd95f3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9fd4098b30 0x7f9fd413a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:42176/0 (socket says 172.21.15.16:42176) 2023-09-30T22:27:01.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.889+0000 7f9fd95f3700 1 -- 172.21.15.16:0/2853743865 learned_addr learned my addr 172.21.15.16:0/2853743865 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:01.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.889+0000 7f9fd9df4700 1 --2- 172.21.15.16:0/2853743865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fd40654a0 0x7f9fd413a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:01.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.889+0000 7f9fd95f3700 1 -- 172.21.15.16:0/2853743865 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9fd4098fa0 msgr2=0x7f9fd413ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.889+0000 7f9fd95f3700 1 --2- 172.21.15.16:0/2853743865 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9fd4098fa0 0x7f9fd413ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-09-30T22:27:01.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.889+0000 7f9fd95f3700 1 -- 172.21.15.16:0/2853743865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fd40654a0 msgr2=0x7f9fd413a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.889+0000 7f9fd95f3700 1 --2- 172.21.15.16:0/2853743865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fd40654a0 0x7f9fd413a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:01.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.889+0000 7f9fd95f3700 1 -- 172.21.15.16:0/2853743865 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fcc009a20 con 0x7f9fd4098b30 2023-09-30T22:27:01.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.889+0000 7f9fd95f3700 1 --2- 172.21.15.16:0/2853743865 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9fd4098b30 0x7f9fd413a840 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f9fc400c520 tx=0x7f9fc4007c10).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:01.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.890+0000 7f9fc9ffb700 1 -- 172.21.15.16:0/2853743865 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc40072e0 con 0x7f9fd4098b30 2023-09-30T22:27:01.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.890+0000 7f9fc9ffb700 1 -- 172.21.15.16:0/2853743865 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fc4007440 con 0x7f9fd4098b30 2023-09-30T22:27:01.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.890+0000 7f9fc9ffb700 1 -- 172.21.15.16:0/2853743865 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc4005930 con 0x7f9fd4098b30 2023-09-30T22:27:01.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.890+0000 7f9fdc058700 1 -- 172.21.15.16:0/2853743865 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fd0009370 con 0x7f9fd4098b30 2023-09-30T22:27:01.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.890+0000 7f9fdc058700 1 -- 172.21.15.16:0/2853743865 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fd411e8d0 con 0x7f9fd4098b30 2023-09-30T22:27:01.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.890+0000 7ff424f51700 1 Processor -- start 2023-09-30T22:27:01.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.891+0000 7f9fc9ffb700 1 -- 172.21.15.16:0/2853743865 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f9fc4005ca0 con 0x7f9fd4098b30 2023-09-30T22:27:01.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.891+0000 7f9fc9ffb700 1 -- 172.21.15.16:0/2853743865 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f9fc403a580 con 0x7f9fd4098b30 2023-09-30T22:27:01.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.890+0000 7ff424f51700 1 -- start start 2023-09-30T22:27:01.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.891+0000 7ff424f51700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff4200677c0 0x7ff420134430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:01.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.891+0000 7ff424f51700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff420134920 0x7ff420136d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:01.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.891+0000 7ff424f51700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff420098000 con 0x7ff420066cd0 2023-09-30T22:27:01.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.891+0000 7ff424f51700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff420067ba0 con 0x7ff420134920 2023-09-30T22:27:01.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.891+0000 7ff424f51700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff420068b90 con 0x7ff4200677c0 2023-09-30T22:27:01.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.891+0000 7ff41dd9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff4200677c0 0x7ff420134430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:01.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.891+0000 7ff41dd9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff4200677c0 0x7ff420134430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:42192/0 (socket says 172.21.15.16:42192) 2023-09-30T22:27:01.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.891+0000 7ff41dd9b700 1 -- 172.21.15.16:0/636899525 learned_addr learned my addr 172.21.15.16:0/636899525 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:01.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.891+0000 7ff41ed9d700 1 --2- 172.21.15.16:0/636899525 >> v2:172.21.15.16:3300/0 conn(0x7ff420134920 0x7ff420136d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:01.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.892+0000 7ff41dd9b700 1 -- 172.21.15.16:0/636899525 >> v2:172.21.15.16:3300/0 conn(0x7ff420134920 msgr2=0x7ff420136d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:01.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.892+0000 7ff41dd9b700 1 --2- 172.21.15.16:0/636899525 >> v2:172.21.15.16:3300/0 conn(0x7ff420134920 0x7ff420136d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.892+0000 7ff41dd9b700 1 -- 172.21.15.16:0/636899525 >> v1:172.21.15.16:6789/0 conn(0x7ff420066cd0 legacy=0x7ff4200670b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.892+0000 7ff41dd9b700 1 -- 172.21.15.16:0/636899525 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4200697b0 con 0x7ff4200677c0 2023-09-30T22:27:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.892+0000 7ff41dd9b700 1 --2- 172.21.15.16:0/636899525 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff4200677c0 0x7ff420134430 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7ff410009f20 tx=0x7ff410009840).ready entity=mon.1 client_cookie=c6f60693687ebdb6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.892+0000 7ff41d59a700 1 -- 172.21.15.16:0/636899525 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff41000e070 con 0x7ff4200677c0 2023-09-30T22:27:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.892+0000 7ff41d59a700 1 -- 172.21.15.16:0/636899525 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff41000b9d0 con 0x7ff4200677c0 2023-09-30T22:27:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.893+0000 7ff424f51700 1 -- 172.21.15.16:0/636899525 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff4200677c0 msgr2=0x7ff420134430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.893+0000 7ff424f51700 1 --2- 172.21.15.16:0/636899525 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff4200677c0 0x7ff420134430 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7ff410009f20 tx=0x7ff410009840).stop 2023-09-30T22:27:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.893+0000 7ff424f51700 1 -- 172.21.15.16:0/636899525 shutdown_connections 2023-09-30T22:27:01.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.893+0000 7ff424f51700 1 --2- 172.21.15.16:0/636899525 >> v2:172.21.15.16:3300/0 conn(0x7ff420134920 0x7ff420136d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:01.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.893+0000 7ff424f51700 1 --2- 172.21.15.16:0/636899525 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff4200677c0 0x7ff420134430 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:01.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.893+0000 7ff424f51700 1 -- 172.21.15.16:0/636899525 >> 172.21.15.16:0/636899525 conn(0x7ff42009a220 msgr2=0x7ff42009a630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:01.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.893+0000 7ff424f51700 1 -- 172.21.15.16:0/636899525 shutdown_connections 2023-09-30T22:27:01.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.893+0000 7ff424f51700 1 -- 172.21.15.16:0/636899525 wait complete. 2023-09-30T22:27:01.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.894+0000 7f9fdc058700 1 -- 172.21.15.16:0/2853743865 --> v1:172.21.15.16:6804/118824 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9fd405f0a0 con 0x7f9fd411b090 2023-09-30T22:27:01.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.894+0000 7ff424f51700 1 Processor -- start 2023-09-30T22:27:01.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.894+0000 7ff424f51700 1 -- start start 2023-09-30T22:27:01.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.895+0000 7ff424f51700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff420066cd0 0x7ff420113220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:01.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.895+0000 7ff424f51700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff4200677c0 0x7ff4201339d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:01.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.895+0000 7ff424f51700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff420134920 0x7ff420133ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:01.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.895+0000 7ff424f51700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff420115710 con 0x7ff420134920 2023-09-30T22:27:01.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.895+0000 7ff424f51700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff42012fac0 con 0x7ff4200677c0 2023-09-30T22:27:01.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.895+0000 7ff424f51700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff4200665b0 con 0x7ff420066cd0 2023-09-30T22:27:01.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.895+0000 7ff41e59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff420066cd0 0x7ff420113220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:01.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.895+0000 7ff41e59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff420066cd0 0x7ff420113220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:40130/0 (socket says 172.21.15.16:40130) 2023-09-30T22:27:01.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.895+0000 7ff41e59c700 1 -- 172.21.15.16:0/4280022366 learned_addr learned my addr 172.21.15.16:0/4280022366 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:01.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.896+0000 7ff41ed9d700 1 --2- 172.21.15.16:0/4280022366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff420134920 0x7ff420133ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:01.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.896+0000 7ff41e59c700 1 -- 172.21.15.16:0/4280022366 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff4200677c0 msgr2=0x7ff4201339d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.896+0000 7ff41e59c700 1 --2- 172.21.15.16:0/4280022366 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff4200677c0 0x7ff4201339d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-09-30T22:27:01.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.896+0000 7ff41e59c700 1 -- 172.21.15.16:0/4280022366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff420134920 msgr2=0x7ff420133ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.896+0000 7ff41e59c700 1 --2- 172.21.15.16:0/4280022366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff420134920 0x7ff420133ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:01.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.896+0000 7ff41e59c700 1 -- 172.21.15.16:0/4280022366 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff410009ae0 con 0x7ff420066cd0 2023-09-30T22:27:01.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.896+0000 7ff41ed9d700 1 --2- 172.21.15.16:0/4280022366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff420134920 0x7ff420133ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:27:01.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.896+0000 7f9fc9ffb700 1 -- 172.21.15.16:0/2853743865 <== osd.0 v1:172.21.15.16:6804/118824 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (unknown 0 0 1859947542) 0x7f9fd405f0a0 con 0x7f9fd411b090 2023-09-30T22:27:01.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.896+0000 7ff41e59c700 1 --2- 172.21.15.16:0/4280022366 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff420066cd0 0x7ff420113220 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7ff41400c4a0 tx=0x7ff414007e00).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:01.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.899+0000 7ff40e7fc700 1 -- 172.21.15.16:0/4280022366 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff414007020 con 0x7ff420066cd0 2023-09-30T22:27:01.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.899+0000 7ff40e7fc700 1 -- 172.21.15.16:0/4280022366 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff414007180 con 0x7ff420066cd0 2023-09-30T22:27:01.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.899+0000 7ff40e7fc700 1 -- 172.21.15.16:0/4280022366 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff414007520 con 0x7ff420066cd0 2023-09-30T22:27:01.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.899+0000 7ff424f51700 1 -- 172.21.15.16:0/4280022366 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff420069270 con 0x7ff420066cd0 2023-09-30T22:27:01.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.899+0000 7ff424f51700 1 -- 172.21.15.16:0/4280022366 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff420116550 con 0x7ff420066cd0 2023-09-30T22:27:01.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.900+0000 7ff424f51700 1 -- 172.21.15.16:0/4280022366 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff420068830 con 0x7ff420066cd0 2023-09-30T22:27:01.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.900+0000 7ff40e7fc700 1 -- 172.21.15.16:0/4280022366 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7ff4140068b0 con 0x7ff420066cd0 2023-09-30T22:27:01.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.900+0000 7ff40e7fc700 1 -- 172.21.15.16:0/4280022366 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3690+0+0 (secure 0 0 0) 0x7ff41403a2a0 con 0x7ff420066cd0 2023-09-30T22:27:01.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.900+0000 7ff40e7fc700 1 -- 172.21.15.16:0/4280022366 --> v1:172.21.15.16:6800/118823 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff400038530 con 0x7ff400034840 2023-09-30T22:27:01.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.900+0000 7f31c19e2700 1 Processor -- start 2023-09-30T22:27:01.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.900+0000 7f31c19e2700 1 -- start start 2023-09-30T22:27:01.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.901+0000 7f31c19e2700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f31bc067ce0 0x7f31bc092340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:01.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.901+0000 7f31c19e2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f31bc0613a0 0x7f31bc123a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:01.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.901+0000 7ff40e7fc700 1 -- 172.21.15.16:0/4280022366 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4140198e0 con 0x7ff420066cd0 2023-09-30T22:27:01.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.901+0000 7f31c19e2700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31bc068880 con 0x7f31bc123f10 2023-09-30T22:27:01.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.901+0000 7f31c19e2700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31bc0680c0 con 0x7f31bc067ce0 2023-09-30T22:27:01.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.901+0000 7f31c19e2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31bc068c90 con 0x7f31bc0613a0 2023-09-30T22:27:01.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.901+0000 7f31bb7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f31bc123f10 0x7f31bc126f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51452/0 (socket says 172.21.15.16:51452) 2023-09-30T22:27:01.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.901+0000 7f31bb7fe700 1 -- 172.21.15.16:0/3964857688 learned_addr learned my addr 172.21.15.16:0/3964857688 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:27:01.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.901+0000 7f31baffd700 1 --2- 172.21.15.16:0/3964857688 >> v2:172.21.15.16:3300/0 conn(0x7f31bc067ce0 0x7f31bc092340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:01.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.902+0000 7f31b9ffb700 1 -- 172.21.15.16:0/3964857688 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1359809988 0 0) 0x7f31bc068880 con 0x7f31bc123f10 2023-09-30T22:27:01.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.902+0000 7f31b9ffb700 1 -- 172.21.15.16:0/3964857688 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31a40033f0 con 0x7f31bc123f10 2023-09-30T22:27:01.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.902+0000 7f31ba7fc700 1 --2- 172.21.15.16:0/3964857688 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f31bc0613a0 0x7f31bc123a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:01.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.902+0000 7f31b9ffb700 1 -- 172.21.15.16:0/3964857688 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 525165496 0 0) 0x7f31a40033f0 con 0x7f31bc123f10 2023-09-30T22:27:01.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.902+0000 7f31b9ffb700 1 -- 172.21.15.16:0/3964857688 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f31bc0613a0 msgr2=0x7f31bc123a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:01.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.902+0000 7f31b9ffb700 1 --2- 172.21.15.16:0/3964857688 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f31bc0613a0 0x7f31bc123a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:01.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.902+0000 7f31b9ffb700 1 -- 172.21.15.16:0/3964857688 >> v2:172.21.15.16:3300/0 conn(0x7f31bc067ce0 msgr2=0x7f31bc092340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:01.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.902+0000 7f31b9ffb700 1 --2- 172.21.15.16:0/3964857688 >> v2:172.21.15.16:3300/0 conn(0x7f31bc067ce0 0x7f31bc092340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:01.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.902+0000 7f31b9ffb700 1 -- 172.21.15.16:0/3964857688 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31bc124430 con 0x7f31bc123f10 2023-09-30T22:27:01.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.902+0000 7f31b9ffb700 1 -- 172.21.15.16:0/3964857688 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1831954289 0 0) 0x7f31b40017f0 con 0x7f31bc123f10 2023-09-30T22:27:01.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.902+0000 7f31b9ffb700 1 -- 172.21.15.16:0/3964857688 >> v1:172.21.15.16:6789/0 conn(0x7f31bc123f10 legacy=0x7f31bc126f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.902+0000 7f31b9ffb700 1 --2- 172.21.15.16:0/3964857688 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f31a40038e0 0x7f31a4003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:01.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.902+0000 7f31b9ffb700 1 --2- 172.21.15.16:0/3964857688 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f31a4004230 0x7f31a4004710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:01.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.902+0000 7f31b9ffb700 1 --2- 172.21.15.16:0/3964857688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31a4004e10 0x7f31a400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:01.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.902+0000 7ff40e7fc700 1 -- 172.21.15.16:0/4280022366 <== osd.3 v1:172.21.15.16:6800/118823 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (unknown 0 0 1859947542) 0x7ff400038530 con 0x7ff400034840 2023-09-30T22:27:01.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.902+0000 7f31b9ffb700 1 -- 172.21.15.16:0/3964857688 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31bc0680c0 con 0x7f31a4004e10 2023-09-30T22:27:01.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.902+0000 7f31b9ffb700 1 -- 172.21.15.16:0/3964857688 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31bc068c90 con 0x7f31a40038e0 2023-09-30T22:27:01.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.902+0000 7f31b9ffb700 1 -- 172.21.15.16:0/3964857688 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f31a400be20 con 0x7f31a4004230 2023-09-30T22:27:01.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.902+0000 7f31baffd700 1 --2- 172.21.15.16:0/3964857688 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f31a40038e0 0x7f31a4003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:01.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.902+0000 7f31bb7fe700 1 --2- 172.21.15.16:0/3964857688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31a4004e10 0x7f31a400b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:01.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.903+0000 7f31baffd700 1 -- 172.21.15.16:0/3964857688 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f31a4004230 msgr2=0x7f31a4004710 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T22:27:01.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.903+0000 7f31baffd700 1 --2- 172.21.15.16:0/3964857688 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f31a4004230 0x7f31a4004710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-09-30T22:27:01.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.903+0000 7f31baffd700 1 -- 172.21.15.16:0/3964857688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31a4004e10 msgr2=0x7f31a400b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:01.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.903+0000 7f31baffd700 1 --2- 172.21.15.16:0/3964857688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31a4004e10 0x7f31a400b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:01.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.903+0000 7f31baffd700 1 -- 172.21.15.16:0/3964857688 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a40033f0 con 0x7f31a40038e0 2023-09-30T22:27:01.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.903+0000 7f31bb7fe700 1 --2- 172.21.15.16:0/3964857688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31a4004e10 0x7f31a400b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:27:01.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.903+0000 7f31baffd700 1 --2- 172.21.15.16:0/3964857688 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f31a40038e0 0x7f31a4003d00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f31ac009a80 tx=0x7f31ac009060).ready entity=mon.1 client_cookie=e7ef1328e1ed44e5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:01.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.903+0000 7f31b9ffb700 1 -- 172.21.15.16:0/3964857688 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ac009430 con 0x7f31a40038e0 2023-09-30T22:27:01.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.903+0000 7f31b9ffb700 1 -- 172.21.15.16:0/3964857688 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31ac00e640 con 0x7f31a40038e0 2023-09-30T22:27:01.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.903+0000 7f31b9ffb700 1 -- 172.21.15.16:0/3964857688 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ac00e9c0 con 0x7f31a40038e0 2023-09-30T22:27:01.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.903+0000 7f775bc83700 1 -- 172.21.15.16:0/2307504703 --> v1:172.21.15.16:6812/118832 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f775411ca60 con 0x7f77380347f0 2023-09-30T22:27:01.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.903+0000 7f7749ffb700 1 -- 172.21.15.16:0/2307504703 <== osd.1 v1:172.21.15.16:6812/118832 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 675830894) 0x7f775411ca60 con 0x7f77380347f0 2023-09-30T22:27:01.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.904+0000 7f775bc83700 1 -- 172.21.15.16:0/2307504703 >> v1:172.21.15.16:6812/118832 conn(0x7f77380347f0 legacy=0x7f7738036c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.904+0000 7f31c19e2700 1 -- 172.21.15.16:0/3964857688 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f31a40038e0 msgr2=0x7f31a4003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.904+0000 7f31c19e2700 1 --2- 172.21.15.16:0/3964857688 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f31a40038e0 0x7f31a4003d00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f31ac009a80 tx=0x7f31ac009060).stop 2023-09-30T22:27:01.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.904+0000 7f31bb7fe700 1 -- 172.21.15.16:0/3964857688 reap_dead start 2023-09-30T22:27:01.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.904+0000 7f775bc83700 1 -- 172.21.15.16:0/2307504703 >> v1:172.21.15.16:6816/118285 conn(0x7f773802bb60 legacy=0x7f773802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.904+0000 7f775bc83700 1 -- 172.21.15.16:0/2307504703 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f77540682d0 msgr2=0x7f77541384e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.904+0000 7f775bc83700 1 --2- 172.21.15.16:0/2307504703 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f77540682d0 0x7f77541384e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f7750009130 tx=0x7f7750017000).stop 2023-09-30T22:27:01.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.904+0000 7f775a220700 1 -- 172.21.15.16:0/2307504703 reap_dead start 2023-09-30T22:27:01.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.904+0000 7f775bc83700 1 -- 172.21.15.16:0/2307504703 shutdown_connections 2023-09-30T22:27:01.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.904+0000 7f775bc83700 1 -- 172.21.15.16:0/2307504703 >> 172.21.15.16:0/2307504703 conn(0x7f775409a220 msgr2=0x7f77540674b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:01.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.904+0000 7f775bc83700 1 -- 172.21.15.16:0/2307504703 shutdown_connections 2023-09-30T22:27:01.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.904+0000 7f31c19e2700 1 -- 172.21.15.16:0/3964857688 shutdown_connections 2023-09-30T22:27:01.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.904+0000 7f31c19e2700 1 -- 172.21.15.16:0/3964857688 >> 172.21.15.16:0/3964857688 conn(0x7f31bc09a220 msgr2=0x7f31bc09a630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:01.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.904+0000 7f775bc83700 1 -- 172.21.15.16:0/2307504703 wait complete. 2023-09-30T22:27:01.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.904+0000 7f31c19e2700 1 -- 172.21.15.16:0/3964857688 shutdown_connections 2023-09-30T22:27:01.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.904+0000 7f31c19e2700 1 -- 172.21.15.16:0/3964857688 wait complete. 2023-09-30T22:27:01.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.905+0000 7f31c19e2700 1 Processor -- start 2023-09-30T22:27:01.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.906+0000 7f31c19e2700 1 -- start start 2023-09-30T22:27:01.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.906+0000 7f31c19e2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f31bc122880 0x7f31bc122c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:01.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.906+0000 7f31c19e2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f31bc123150 0x7f31bc114c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:01.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.906+0000 7f31c19e2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31bc115170 0x7f31bc1155f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:01.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.906+0000 7f31c19e2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31bc0670a0 con 0x7f31bc115170 2023-09-30T22:27:01.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.906+0000 7f31c19e2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31bc0697c0 con 0x7f31bc122880 2023-09-30T22:27:01.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.906+0000 7f31c19e2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f31bc115ae0 con 0x7f31bc123150 2023-09-30T22:27:01.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.906+0000 7f31baffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f31bc122880 0x7f31bc122c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:01.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.906+0000 7f31baffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f31bc122880 0x7f31bc122c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:42238/0 (socket says 172.21.15.16:42238) 2023-09-30T22:27:01.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.906+0000 7f31baffd700 1 -- 172.21.15.16:0/3089777357 learned_addr learned my addr 172.21.15.16:0/3089777357 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:01.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.906+0000 7f31bb7fe700 1 --2- 172.21.15.16:0/3089777357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31bc115170 0x7f31bc1155f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:01.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.907+0000 7f31baffd700 1 -- 172.21.15.16:0/3089777357 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f31bc123150 msgr2=0x7f31bc114c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.907+0000 7f31baffd700 1 --2- 172.21.15.16:0/3089777357 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f31bc123150 0x7f31bc114c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-09-30T22:27:01.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.907+0000 7f31baffd700 1 -- 172.21.15.16:0/3089777357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31bc115170 msgr2=0x7f31bc1155f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.907+0000 7f31baffd700 1 --2- 172.21.15.16:0/3089777357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31bc115170 0x7f31bc1155f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:01.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.907+0000 7f31baffd700 1 -- 172.21.15.16:0/3089777357 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31bc119400 con 0x7f31bc122880 2023-09-30T22:27:01.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.907+0000 7f31baffd700 1 --2- 172.21.15.16:0/3089777357 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f31bc122880 0x7f31bc122c60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f31ac00e4b0 tx=0x7f31ac01a000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:01.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.907+0000 7f31a2ffd700 1 -- 172.21.15.16:0/3089777357 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ac00ed30 con 0x7f31bc122880 2023-09-30T22:27:01.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.907+0000 7f31a2ffd700 1 -- 172.21.15.16:0/3089777357 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31ac00ee90 con 0x7f31bc122880 2023-09-30T22:27:01.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.907+0000 7f31a2ffd700 1 -- 172.21.15.16:0/3089777357 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ac025440 con 0x7f31bc122880 2023-09-30T22:27:01.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.908+0000 7f31c19e2700 1 -- 172.21.15.16:0/3089777357 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31bc1184a0 con 0x7f31bc122880 2023-09-30T22:27:01.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.908+0000 7f31c19e2700 1 -- 172.21.15.16:0/3089777357 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31bc118df0 con 0x7f31bc122880 2023-09-30T22:27:01.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.909+0000 7f31c19e2700 1 -- 172.21.15.16:0/3089777357 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f31bc05a260 con 0x7f31bc122880 2023-09-30T22:27:01.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.909+0000 7f31a2ffd700 1 -- 172.21.15.16:0/3089777357 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f31ac02e440 con 0x7f31bc122880 2023-09-30T22:27:01.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.909+0000 7f31a2ffd700 1 -- 172.21.15.16:0/3089777357 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f31ac03c6e0 con 0x7f31bc122880 2023-09-30T22:27:01.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.909+0000 7f31a2ffd700 1 -- 172.21.15.16:0/3089777357 --> v1:172.21.15.16:6808/118840 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f31a8038530 con 0x7f31a8034840 2023-09-30T22:27:01.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.909+0000 7f31a2ffd700 1 -- 172.21.15.16:0/3089777357 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7f31ac006ad0 con 0x7f31bc122880 2023-09-30T22:27:01.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.911+0000 7f31a2ffd700 1 -- 172.21.15.16:0/3089777357 <== osd.2 v1:172.21.15.16:6808/118840 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (unknown 0 0 1859947542) 0x7f31a8038530 con 0x7f31a8034840 2023-09-30T22:27:01.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.915+0000 7f9fdc058700 1 -- 172.21.15.16:0/2853743865 --> v1:172.21.15.16:6804/118824 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9fd411df00 con 0x7f9fd411b090 2023-09-30T22:27:01.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.915+0000 7f9fc9ffb700 1 -- 172.21.15.16:0/2853743865 <== osd.0 v1:172.21.15.16:6804/118824 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 675830894) 0x7f9fd411df00 con 0x7f9fd411b090 2023-09-30T22:27:01.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.916+0000 7f9fdc058700 1 -- 172.21.15.16:0/2853743865 >> v1:172.21.15.16:6804/118824 conn(0x7f9fd411b090 legacy=0x7f9fd411b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.916+0000 7f9fdc058700 1 -- 172.21.15.16:0/2853743865 >> v1:172.21.15.16:6816/118285 conn(0x7f9fb802bb60 legacy=0x7f9fb802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.916+0000 7f9fdc058700 1 -- 172.21.15.16:0/2853743865 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9fd4098b30 msgr2=0x7f9fd413a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.916+0000 7f9fdc058700 1 --2- 172.21.15.16:0/2853743865 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9fd4098b30 0x7f9fd413a840 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f9fc400c520 tx=0x7f9fc4007c10).stop 2023-09-30T22:27:01.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.916+0000 7f9fda5f5700 1 -- 172.21.15.16:0/2853743865 reap_dead start 2023-09-30T22:27:01.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.916+0000 7f9fdc058700 1 -- 172.21.15.16:0/2853743865 shutdown_connections 2023-09-30T22:27:01.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.916+0000 7f9fdc058700 1 -- 172.21.15.16:0/2853743865 >> 172.21.15.16:0/2853743865 conn(0x7f9fd409a740 msgr2=0x7f9fd41142e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:01.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.916+0000 7f9fdc058700 1 -- 172.21.15.16:0/2853743865 shutdown_connections 2023-09-30T22:27:01.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.916+0000 7f9fdc058700 1 -- 172.21.15.16:0/2853743865 wait complete. 2023-09-30T22:27:01.927 INFO:teuthology.orchestra.run.smithi016.stdout:38654705669 2023-09-30T22:27:01.928 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:27:01.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.921+0000 7ff424f51700 1 -- 172.21.15.16:0/4280022366 --> v1:172.21.15.16:6800/118823 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff420115c10 con 0x7ff400034840 2023-09-30T22:27:01.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.921+0000 7ff40e7fc700 1 -- 172.21.15.16:0/4280022366 <== osd.3 v1:172.21.15.16:6800/118823 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 675830894) 0x7ff418009480 con 0x7ff400034840 2023-09-30T22:27:01.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.922+0000 7ff424f51700 1 -- 172.21.15.16:0/4280022366 >> v1:172.21.15.16:6800/118823 conn(0x7ff400034840 legacy=0x7ff400036cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.922+0000 7ff424f51700 1 -- 172.21.15.16:0/4280022366 >> v1:172.21.15.16:6816/118285 conn(0x7ff40002bbb0 legacy=0x7ff40002e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.922+0000 7ff424f51700 1 -- 172.21.15.16:0/4280022366 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff420066cd0 msgr2=0x7ff420113220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.922+0000 7ff424f51700 1 --2- 172.21.15.16:0/4280022366 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff420066cd0 0x7ff420113220 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7ff41400c4a0 tx=0x7ff414007e00).stop 2023-09-30T22:27:01.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.922+0000 7ff41ed9d700 1 -- 172.21.15.16:0/4280022366 reap_dead start 2023-09-30T22:27:01.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.922+0000 7ff424f51700 1 -- 172.21.15.16:0/4280022366 shutdown_connections 2023-09-30T22:27:01.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.922+0000 7ff424f51700 1 -- 172.21.15.16:0/4280022366 >> 172.21.15.16:0/4280022366 conn(0x7ff42009a220 msgr2=0x7ff420137650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:01.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.922+0000 7ff424f51700 1 -- 172.21.15.16:0/4280022366 shutdown_connections 2023-09-30T22:27:01.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.922+0000 7ff424f51700 1 -- 172.21.15.16:0/4280022366 wait complete. 2023-09-30T22:27:01.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.927+0000 7f31c19e2700 1 -- 172.21.15.16:0/3089777357 --> v1:172.21.15.16:6808/118840 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f31bc1180d0 con 0x7f31a8034840 2023-09-30T22:27:01.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.928+0000 7f31a2ffd700 1 -- 172.21.15.16:0/3089777357 <== osd.2 v1:172.21.15.16:6808/118840 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 1129848789) 0x7f31bc1180d0 con 0x7f31a8034840 2023-09-30T22:27:01.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.928+0000 7f31c19e2700 1 -- 172.21.15.16:0/3089777357 >> v1:172.21.15.16:6808/118840 conn(0x7f31a8034840 legacy=0x7f31a8036cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.928+0000 7f31c19e2700 1 -- 172.21.15.16:0/3089777357 >> v1:172.21.15.16:6816/118285 conn(0x7f31a802bbb0 legacy=0x7f31a802e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.928+0000 7f31c19e2700 1 -- 172.21.15.16:0/3089777357 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f31bc122880 msgr2=0x7f31bc122c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:01.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.928+0000 7f31c19e2700 1 --2- 172.21.15.16:0/3089777357 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f31bc122880 0x7f31bc122c60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f31ac00e4b0 tx=0x7f31ac01a000).stop 2023-09-30T22:27:01.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.928+0000 7f31bb7fe700 1 -- 172.21.15.16:0/3089777357 reap_dead start 2023-09-30T22:27:01.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.928+0000 7f31c19e2700 1 -- 172.21.15.16:0/3089777357 shutdown_connections 2023-09-30T22:27:01.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.928+0000 7f31c19e2700 1 -- 172.21.15.16:0/3089777357 >> 172.21.15.16:0/3089777357 conn(0x7f31bc09a220 msgr2=0x7f31bc098e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:01.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.929+0000 7f31c19e2700 1 -- 172.21.15.16:0/3089777357 shutdown_connections 2023-09-30T22:27:01.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:01.929+0000 7f31c19e2700 1 -- 172.21.15.16:0/3089777357 wait complete. 2023-09-30T22:27:01.933 INFO:teuthology.orchestra.run.smithi016.stdout:38654705669 2023-09-30T22:27:01.934 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:27:01.939 INFO:teuthology.orchestra.run.smithi016.stdout:38654705669 2023-09-30T22:27:01.939 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:27:01.944 INFO:teuthology.orchestra.run.smithi016.stdout:38654705670 2023-09-30T22:27:01.944 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:27:02.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.081+0000 7f589c86c700 1 Processor -- start 2023-09-30T22:27:02.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.081+0000 7f589c86c700 1 -- start start 2023-09-30T22:27:02.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.082+0000 7f589c86c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5898059070 0x7f5898127890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:02.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.082+0000 7f589c86c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5898127d80 0x7f5898095a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:02.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.082+0000 7f589c86c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5898091b60 con 0x7f5898127390 2023-09-30T22:27:02.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.082+0000 7f589c86c700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5898059450 con 0x7f5898059070 2023-09-30T22:27:02.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.082+0000 7f589c86c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58980968c0 con 0x7f5898127d80 2023-09-30T22:27:02.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.082+0000 7f1b7dd1c700 1 Processor -- start 2023-09-30T22:27:02.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.083+0000 7f589659c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5898059070 0x7f5898127890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:02.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.082+0000 7f1b7dd1c700 1 -- start start 2023-09-30T22:27:02.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.083+0000 7f589659c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5898059070 0x7f5898127890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54516/0 (socket says 172.21.15.16:54516) 2023-09-30T22:27:02.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.083+0000 7f589659c700 1 -- 172.21.15.16:0/3601128351 learned_addr learned my addr 172.21.15.16:0/3601128351 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:02.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.083+0000 7f589659c700 1 -- 172.21.15.16:0/3601128351 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5898127d80 msgr2=0x7f5898095a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:02.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.083+0000 7f589659c700 1 --2- 172.21.15.16:0/3601128351 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5898127d80 0x7f5898095a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-09-30T22:27:02.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.083+0000 7f589659c700 1 -- 172.21.15.16:0/3601128351 >> v1:172.21.15.16:6789/0 conn(0x7f5898127390 legacy=0x7f5898095370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:02.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.083+0000 7f589559a700 1 -- 172.21.15.16:0/3601128351 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 51313054 0 0) 0x7f5898091b60 con 0x7f5898127390 2023-09-30T22:27:02.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.083+0000 7f589659c700 1 -- 172.21.15.16:0/3601128351 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5898092a10 con 0x7f5898059070 2023-09-30T22:27:02.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.083+0000 7f589659c700 1 --2- 172.21.15.16:0/3601128351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5898059070 0x7f5898127890 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f5888006fd0 tx=0x7f58880090f0).ready entity=mon.0 client_cookie=1483034e789076e2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:02.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.083+0000 7f1b7dd1c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1b78096820 0x7f1b78096c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:02.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.083+0000 7f1b7dd1c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1b78067940 0x7f1b780970f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:02.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.083+0000 7f1b7dd1c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b78068df0 con 0x7f1b780975e0 2023-09-30T22:27:02.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.083+0000 7f1b7dd1c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7806a4c0 con 0x7f1b78096820 2023-09-30T22:27:02.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.083+0000 7f1b7dd1c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1b78098260 con 0x7f1b78067940 2023-09-30T22:27:02.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.083+0000 7f589559a700 1 -- 172.21.15.16:0/3601128351 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f588800b600 con 0x7f5898059070 2023-09-30T22:27:02.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.083+0000 7f589559a700 1 -- 172.21.15.16:0/3601128351 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f588800b760 con 0x7f5898059070 2023-09-30T22:27:02.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.084+0000 7f589559a700 1 -- 172.21.15.16:0/3601128351 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f588800bb00 con 0x7f5898059070 2023-09-30T22:27:02.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.084+0000 7f1b77fff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1b780975e0 0x7f1b7812f270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51456/0 (socket says 172.21.15.16:51456) 2023-09-30T22:27:02.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.084+0000 7f1b77fff700 1 -- 172.21.15.16:0/1479162459 learned_addr learned my addr 172.21.15.16:0/1479162459 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:27:02.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.084+0000 7f1b777fe700 1 --2- 172.21.15.16:0/1479162459 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1b78096820 0x7f1b78096c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:02.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.084+0000 7f1b76ffd700 1 --2- 172.21.15.16:0/1479162459 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1b78067940 0x7f1b780970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:02.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.084+0000 7f1b767fc700 1 -- 172.21.15.16:0/1479162459 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1221674959 0 0) 0x7f1b78068df0 con 0x7f1b780975e0 2023-09-30T22:27:02.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.084+0000 7f1b767fc700 1 -- 172.21.15.16:0/1479162459 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b600033f0 con 0x7f1b780975e0 2023-09-30T22:27:02.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.084+0000 7f1b76ffd700 1 -- 172.21.15.16:0/1479162459 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1b78096820 msgr2=0x7f1b78096c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:02.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.084+0000 7f1b76ffd700 1 --2- 172.21.15.16:0/1479162459 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1b78096820 0x7f1b78096c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:02.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.084+0000 7f1b76ffd700 1 -- 172.21.15.16:0/1479162459 >> v1:172.21.15.16:6789/0 conn(0x7f1b780975e0 legacy=0x7f1b7812f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:02.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.084+0000 7f1b767fc700 1 -- 172.21.15.16:0/1479162459 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2480675435 0 0) 0x7f1b600033f0 con 0x7f1b780975e0 2023-09-30T22:27:02.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.084+0000 7f1b76ffd700 1 -- 172.21.15.16:0/1479162459 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b78095e60 con 0x7f1b78067940 2023-09-30T22:27:02.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.084+0000 7f589c86c700 1 -- 172.21.15.16:0/3601128351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5898059070 msgr2=0x7f5898127890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:02.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.084+0000 7f589c86c700 1 --2- 172.21.15.16:0/3601128351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5898059070 0x7f5898127890 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f5888006fd0 tx=0x7f58880090f0).stop 2023-09-30T22:27:02.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.085+0000 7f1b76ffd700 1 --2- 172.21.15.16:0/1479162459 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1b78067940 0x7f1b780970f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f1b68009790 tx=0x7f1b68009060).ready entity=mon.2 client_cookie=bd70f7279123b913 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:02.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.085+0000 7f589c86c700 1 -- 172.21.15.16:0/3601128351 shutdown_connections 2023-09-30T22:27:02.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.085+0000 7f589c86c700 1 --2- 172.21.15.16:0/3601128351 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5898127d80 0x7f5898095a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-09-30T22:27:02.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.085+0000 7f589c86c700 1 --2- 172.21.15.16:0/3601128351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5898059070 0x7f5898127890 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:02.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.085+0000 7f589c86c700 1 -- 172.21.15.16:0/3601128351 >> 172.21.15.16:0/3601128351 conn(0x7f5898097470 msgr2=0x7f5898097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:02.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.085+0000 7f589c86c700 1 -- 172.21.15.16:0/3601128351 shutdown_connections 2023-09-30T22:27:02.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.085+0000 7f589c86c700 1 -- 172.21.15.16:0/3601128351 wait complete. 2023-09-30T22:27:02.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.085+0000 7f1b767fc700 1 -- 172.21.15.16:0/1479162459 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b700017f0 con 0x7f1b78067940 2023-09-30T22:27:02.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.085+0000 7f1b767fc700 1 -- 172.21.15.16:0/1479162459 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b68011400 con 0x7f1b78067940 2023-09-30T22:27:02.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.085+0000 7f1b767fc700 1 -- 172.21.15.16:0/1479162459 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b68011780 con 0x7f1b78067940 2023-09-30T22:27:02.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.086+0000 7f1b7dd1c700 1 -- 172.21.15.16:0/1479162459 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1b78067940 msgr2=0x7f1b780970f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:02.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.086+0000 7f1b7dd1c700 1 --2- 172.21.15.16:0/1479162459 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1b78067940 0x7f1b780970f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f1b68009790 tx=0x7f1b68009060).stop 2023-09-30T22:27:02.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.086+0000 7f589c86c700 1 Processor -- start 2023-09-30T22:27:02.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.086+0000 7f1b7dd1c700 1 -- 172.21.15.16:0/1479162459 shutdown_connections 2023-09-30T22:27:02.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.086+0000 7f1b7dd1c700 1 --2- 172.21.15.16:0/1479162459 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1b78096820 0x7f1b78096c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:02.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.086+0000 7f1b7dd1c700 1 --2- 172.21.15.16:0/1479162459 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1b78067940 0x7f1b780970f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:02.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.086+0000 7f1b7dd1c700 1 -- 172.21.15.16:0/1479162459 >> 172.21.15.16:0/1479162459 conn(0x7f1b7809ba50 msgr2=0x7f1b7809be60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:02.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.086+0000 7f1b7dd1c700 1 -- 172.21.15.16:0/1479162459 shutdown_connections 2023-09-30T22:27:02.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.086+0000 7f589c86c700 1 -- start start 2023-09-30T22:27:02.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.086+0000 7f1b7dd1c700 1 -- 172.21.15.16:0/1479162459 wait complete. 2023-09-30T22:27:02.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.086+0000 7f589c86c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5898059070 0x7f58981259d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:02.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.086+0000 7f589c86c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5898127390 0x7f5898114900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:02.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.086+0000 7f589c86c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5898127d80 0x7f5898114df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:02.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.087+0000 7f589c86c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5898118410 con 0x7f5898127390 2023-09-30T22:27:02.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.087+0000 7f589c86c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5898129040 con 0x7f5898127d80 2023-09-30T22:27:02.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.087+0000 7f589c86c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5898096ea0 con 0x7f5898059070 2023-09-30T22:27:02.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.087+0000 7f5895d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5898127390 0x7f5898114900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:02.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.087+0000 7f589659c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5898059070 0x7f58981259d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:02.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.087+0000 7f589659c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5898059070 0x7f58981259d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:40150/0 (socket says 172.21.15.16:40150) 2023-09-30T22:27:02.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.087+0000 7f5895d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5898127390 0x7f5898114900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54520/0 (socket says 172.21.15.16:54520) 2023-09-30T22:27:02.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.087+0000 7f589659c700 1 -- 172.21.15.16:0/739913776 learned_addr learned my addr 172.21.15.16:0/739913776 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:02.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.087+0000 7f5896d9d700 1 --2- 172.21.15.16:0/739913776 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5898127d80 0x7f5898114df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:02.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.087+0000 7f589659c700 1 -- 172.21.15.16:0/739913776 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5898127d80 msgr2=0x7f5898114df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:02.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.087+0000 7f589659c700 1 --2- 172.21.15.16:0/739913776 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5898127d80 0x7f5898114df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:02.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.087+0000 7f589659c700 1 -- 172.21.15.16:0/739913776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5898127390 msgr2=0x7f5898114900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:02.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.087+0000 7f589659c700 1 --2- 172.21.15.16:0/739913776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5898127390 0x7f5898114900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:02.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.087+0000 7f589659c700 1 -- 172.21.15.16:0/739913776 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5888009370 con 0x7f5898059070 2023-09-30T22:27:02.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.087+0000 7f5896d9d700 1 --2- 172.21.15.16:0/739913776 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5898127d80 0x7f5898114df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:27:02.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.087+0000 7f589659c700 1 --2- 172.21.15.16:0/739913776 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5898059070 0x7f58981259d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f5888011fc0 tx=0x7f5888017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:02.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.087+0000 7f1b7dd1c700 1 Processor -- start 2023-09-30T22:27:02.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.088+0000 7f58867fc700 1 -- 172.21.15.16:0/739913776 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5888002f70 con 0x7f5898059070 2023-09-30T22:27:02.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.088+0000 7f1b7dd1c700 1 -- start start 2023-09-30T22:27:02.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.088+0000 7f58867fc700 1 -- 172.21.15.16:0/739913776 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f588800b8d0 con 0x7f5898059070 2023-09-30T22:27:02.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.088+0000 7f58867fc700 1 -- 172.21.15.16:0/739913776 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f588801c420 con 0x7f5898059070 2023-09-30T22:27:02.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.088+0000 7f589c86c700 1 -- 172.21.15.16:0/739913776 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5898117e00 con 0x7f5898059070 2023-09-30T22:27:02.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.088+0000 7f1b7dd1c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1b78067940 0x7f1b7813edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:02.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.088+0000 7f1b7dd1c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b78096820 0x7f1b7813f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:02.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.088+0000 7f1b7dd1c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1b780975e0 0x7f1b7813f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:02.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.088+0000 7f1b7dd1c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7813fca0 con 0x7f1b78096820 2023-09-30T22:27:02.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.088+0000 7f1b7dd1c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b78122b40 con 0x7f1b78067940 2023-09-30T22:27:02.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.088+0000 7f1b7dd1c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1b78067e60 con 0x7f1b780975e0 2023-09-30T22:27:02.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.088+0000 7f1b777fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1b78067940 0x7f1b7813edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:02.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.088+0000 7f1b76ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b78096820 0x7f1b7813f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:02.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.088+0000 7f1b777fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1b78067940 0x7f1b7813edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:42270/0 (socket says 172.21.15.16:42270) 2023-09-30T22:27:02.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.088+0000 7f1b76ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b78096820 0x7f1b7813f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54530/0 (socket says 172.21.15.16:54530) 2023-09-30T22:27:02.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.088+0000 7f1b777fe700 1 -- 172.21.15.16:0/3516608951 learned_addr learned my addr 172.21.15.16:0/3516608951 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:02.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.089+0000 7f589c86c700 1 -- 172.21.15.16:0/739913776 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5898118a80 con 0x7f5898059070 2023-09-30T22:27:02.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.089+0000 7f1b77fff700 1 --2- 172.21.15.16:0/3516608951 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1b780975e0 0x7f1b7813f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:02.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.089+0000 7f1b76ffd700 1 -- 172.21.15.16:0/3516608951 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1b780975e0 msgr2=0x7f1b7813f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:02.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.089+0000 7f1b76ffd700 1 --2- 172.21.15.16:0/3516608951 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1b780975e0 0x7f1b7813f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:02.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.089+0000 7f1b76ffd700 1 -- 172.21.15.16:0/3516608951 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1b78067940 msgr2=0x7f1b7813edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:02.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.089+0000 7f1b76ffd700 1 --2- 172.21.15.16:0/3516608951 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1b78067940 0x7f1b7813edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:02.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.089+0000 7f1b76ffd700 1 -- 172.21.15.16:0/3516608951 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b680092a0 con 0x7f1b78096820 2023-09-30T22:27:02.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.089+0000 7f1b777fe700 1 --2- 172.21.15.16:0/3516608951 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1b78067940 0x7f1b7813edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:02.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.089+0000 7f1b76ffd700 1 --2- 172.21.15.16:0/3516608951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b78096820 0x7f1b7813f2c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f1b68013000 tx=0x7f1b68016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:02.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.089+0000 7f1b5f7fe700 1 -- 172.21.15.16:0/3516608951 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b680107c0 con 0x7f1b78096820 2023-09-30T22:27:02.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.089+0000 7f1b5f7fe700 1 -- 172.21.15.16:0/3516608951 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b68010920 con 0x7f1b78096820 2023-09-30T22:27:02.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.089+0000 7f1b5f7fe700 1 -- 172.21.15.16:0/3516608951 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b68010ce0 con 0x7f1b78096820 2023-09-30T22:27:02.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.089+0000 7f1b7dd1c700 1 -- 172.21.15.16:0/3516608951 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b781215d0 con 0x7f1b78096820 2023-09-30T22:27:02.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.090+0000 7f1b7dd1c700 1 -- 172.21.15.16:0/3516608951 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b78123350 con 0x7f1b78096820 2023-09-30T22:27:02.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.090+0000 7f58867fc700 1 -- 172.21.15.16:0/739913776 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f588801c850 con 0x7f5898059070 2023-09-30T22:27:02.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.090+0000 7f589c86c700 1 -- 172.21.15.16:0/739913776 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5898091d80 con 0x7f5898059070 2023-09-30T22:27:02.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.090+0000 7f58867fc700 1 -- 172.21.15.16:0/739913776 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f588803cdd0 con 0x7f5898059070 2023-09-30T22:27:02.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.091+0000 7f1b5f7fe700 1 -- 172.21.15.16:0/3516608951 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f1b6801d640 con 0x7f1b78096820 2023-09-30T22:27:02.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.091+0000 7f1b7dd1c700 1 -- 172.21.15.16:0/3516608951 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b58005120 con 0x7f1b78096820 2023-09-30T22:27:02.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.091+0000 7f1b5f7fe700 1 -- 172.21.15.16:0/3516608951 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f1b6803c250 con 0x7f1b78096820 2023-09-30T22:27:02.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.096+0000 7f1b5f7fe700 1 -- 172.21.15.16:0/3516608951 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f1b6801dc70 con 0x7f1b78096820 2023-09-30T22:27:02.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.099+0000 7f58867fc700 1 -- 172.21.15.16:0/739913776 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f5888020070 con 0x7f5898059070 2023-09-30T22:27:02.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.123+0000 7f5f70c47700 1 Processor -- start 2023-09-30T22:27:02.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.123+0000 7f5f70c47700 1 -- start start 2023-09-30T22:27:02.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.124+0000 7f5f70c47700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5f6c123a90 0x7f5f6c123e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:02.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.124+0000 7f5f70c47700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5f6c067ce0 0x7f5f6c092340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:02.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.124+0000 7f5f70c47700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5f6c126f70 0x7f5f6c1273f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:02.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.124+0000 7f5f70c47700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f6c0680c0 con 0x7f5f6c126f70 2023-09-30T22:27:02.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.124+0000 7f5f70c47700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6c068880 con 0x7f5f6c067ce0 2023-09-30T22:27:02.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.124+0000 7f5f70c47700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6c068c90 con 0x7f5f6c123a90 2023-09-30T22:27:02.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.125+0000 7f5f69d9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5f6c067ce0 0x7f5f6c092340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:02.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.125+0000 7f5f69d9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5f6c067ce0 0x7f5f6c092340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:42280/0 (socket says 172.21.15.16:42280) 2023-09-30T22:27:02.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.125+0000 7f5f69d9b700 1 -- 172.21.15.16:0/4211625342 learned_addr learned my addr 172.21.15.16:0/4211625342 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:02.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.125+0000 7f5f6ad9d700 1 --2- 172.21.15.16:0/4211625342 >> v2:172.21.15.16:3300/0 conn(0x7f5f6c126f70 0x7f5f6c1273f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:02.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.125+0000 7f5f6a59c700 1 --2- 172.21.15.16:0/4211625342 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5f6c123a90 0x7f5f6c123e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:02.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.125+0000 7f5f69d9b700 1 -- 172.21.15.16:0/4211625342 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5f6c123a90 msgr2=0x7f5f6c123e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:02.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.125+0000 7f5f69d9b700 1 --2- 172.21.15.16:0/4211625342 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5f6c123a90 0x7f5f6c123e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:02.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.125+0000 7f5f69d9b700 1 -- 172.21.15.16:0/4211625342 >> v2:172.21.15.16:3300/0 conn(0x7f5f6c126f70 msgr2=0x7f5f6c1273f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:02.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.125+0000 7f5f69d9b700 1 --2- 172.21.15.16:0/4211625342 >> v2:172.21.15.16:3300/0 conn(0x7f5f6c126f70 0x7f5f6c1273f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:02.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.125+0000 7f5f69d9b700 1 -- 172.21.15.16:0/4211625342 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f6c124430 con 0x7f5f6c067ce0 2023-09-30T22:27:02.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.125+0000 7f5f6ad9d700 1 --2- 172.21.15.16:0/4211625342 >> v2:172.21.15.16:3300/0 conn(0x7f5f6c126f70 0x7f5f6c1273f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:02.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.125+0000 7f5f6a59c700 1 --2- 172.21.15.16:0/4211625342 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5f6c123a90 0x7f5f6c123e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:27:02.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.125+0000 7f5f69d9b700 1 --2- 172.21.15.16:0/4211625342 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5f6c067ce0 0x7f5f6c092340 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f5f6000bac0 tx=0x7f5f6000b370).ready entity=mon.1 client_cookie=51d5bb5286a4b7e3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:02.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.126+0000 7f5f6959a700 1 -- 172.21.15.16:0/4211625342 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6000db70 con 0x7f5f6c067ce0 2023-09-30T22:27:02.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.126+0000 7f5f6959a700 1 -- 172.21.15.16:0/4211625342 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f6000dcd0 con 0x7f5f6c067ce0 2023-09-30T22:27:02.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.126+0000 7f5f6959a700 1 -- 172.21.15.16:0/4211625342 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f60015600 con 0x7f5f6c067ce0 2023-09-30T22:27:02.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.126+0000 7f5f70c47700 1 -- 172.21.15.16:0/4211625342 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5f6c067ce0 msgr2=0x7f5f6c092340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:02.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.126+0000 7f5f70c47700 1 --2- 172.21.15.16:0/4211625342 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5f6c067ce0 0x7f5f6c092340 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f5f6000bac0 tx=0x7f5f6000b370).stop 2023-09-30T22:27:02.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.126+0000 7f5f70c47700 1 -- 172.21.15.16:0/4211625342 shutdown_connections 2023-09-30T22:27:02.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.126+0000 7f5f70c47700 1 --2- 172.21.15.16:0/4211625342 >> v2:172.21.15.16:3300/0 conn(0x7f5f6c126f70 0x7f5f6c1273f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:02.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.126+0000 7f5f70c47700 1 --2- 172.21.15.16:0/4211625342 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5f6c123a90 0x7f5f6c123e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:02.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.126+0000 7f5f70c47700 1 --2- 172.21.15.16:0/4211625342 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5f6c067ce0 0x7f5f6c092340 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:02.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.126+0000 7f5f70c47700 1 -- 172.21.15.16:0/4211625342 >> 172.21.15.16:0/4211625342 conn(0x7f5f6c09a220 msgr2=0x7f5f6c09a630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:02.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.127+0000 7f5f70c47700 1 -- 172.21.15.16:0/4211625342 shutdown_connections 2023-09-30T22:27:02.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.127+0000 7f5f70c47700 1 -- 172.21.15.16:0/4211625342 wait complete. 2023-09-30T22:27:02.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.128+0000 7f5f70c47700 1 Processor -- start 2023-09-30T22:27:02.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.128+0000 7f5f70c47700 1 -- start start 2023-09-30T22:27:02.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.129+0000 7f5f70c47700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f6c067ce0 0x7f5f6c111650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:02.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.129+0000 7f5f70c47700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5f6c123a90 0x7f5f6c111b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:02.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.129+0000 7f5f70c47700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5f6c126f70 0x7f5f6c112030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:02.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.129+0000 7f5f70c47700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6c112520 con 0x7f5f6c067ce0 2023-09-30T22:27:02.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.129+0000 7f5f70c47700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6c114730 con 0x7f5f6c126f70 2023-09-30T22:27:02.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.129+0000 7f5f70c47700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6c0697c0 con 0x7f5f6c123a90 2023-09-30T22:27:02.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.129+0000 7f5f6ad9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5f6c126f70 0x7f5f6c112030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:02.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.129+0000 7f5f6ad9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5f6c126f70 0x7f5f6c112030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:42286/0 (socket says 172.21.15.16:42286) 2023-09-30T22:27:02.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.129+0000 7f5f6ad9d700 1 -- 172.21.15.16:0/466948315 learned_addr learned my addr 172.21.15.16:0/466948315 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:02.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.129+0000 7f5f69d9b700 1 --2- 172.21.15.16:0/466948315 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5f6c123a90 0x7f5f6c111b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:02.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.129+0000 7f5f6a59c700 1 --2- 172.21.15.16:0/466948315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f6c067ce0 0x7f5f6c111650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:02.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.129+0000 7f5f6ad9d700 1 -- 172.21.15.16:0/466948315 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5f6c123a90 msgr2=0x7f5f6c111b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:02.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.129+0000 7f5f6ad9d700 1 --2- 172.21.15.16:0/466948315 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5f6c123a90 0x7f5f6c111b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:02.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.129+0000 7f5f6ad9d700 1 -- 172.21.15.16:0/466948315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f6c067ce0 msgr2=0x7f5f6c111650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:02.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.129+0000 7f5f6ad9d700 1 --2- 172.21.15.16:0/466948315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f6c067ce0 0x7f5f6c111650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:02.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.129+0000 7f5f6ad9d700 1 -- 172.21.15.16:0/466948315 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f64009a20 con 0x7f5f6c126f70 2023-09-30T22:27:02.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.130+0000 7f5f6ad9d700 1 --2- 172.21.15.16:0/466948315 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5f6c126f70 0x7f5f6c112030 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f5f64009fc0 tx=0x7f5f6400b040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:02.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.130+0000 7f5f5a7fc700 1 -- 172.21.15.16:0/466948315 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6400cbd0 con 0x7f5f6c126f70 2023-09-30T22:27:02.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.130+0000 7f5f5a7fc700 1 -- 172.21.15.16:0/466948315 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f6400f040 con 0x7f5f6c126f70 2023-09-30T22:27:02.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.130+0000 7f5f70c47700 1 -- 172.21.15.16:0/466948315 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f6000b6a0 con 0x7f5f6c126f70 2023-09-30T22:27:02.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.130+0000 7f5f5a7fc700 1 -- 172.21.15.16:0/466948315 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f64013400 con 0x7f5f6c126f70 2023-09-30T22:27:02.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.130+0000 7f5f70c47700 1 -- 172.21.15.16:0/466948315 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f6c114960 con 0x7f5f6c126f70 2023-09-30T22:27:02.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.131+0000 7f5f70c47700 1 -- 172.21.15.16:0/466948315 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f6c114da0 con 0x7f5f6c126f70 2023-09-30T22:27:02.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.132+0000 7f5f5a7fc700 1 -- 172.21.15.16:0/466948315 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f5f640137b0 con 0x7f5f6c126f70 2023-09-30T22:27:02.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.132+0000 7f5f5a7fc700 1 -- 172.21.15.16:0/466948315 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f5f6403b950 con 0x7f5f6c126f70 2023-09-30T22:27:02.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.139+0000 7f5f5a7fc700 1 -- 172.21.15.16:0/466948315 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f5f64007990 con 0x7f5f6c126f70 2023-09-30T22:27:02.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.145+0000 7f6bd3c30700 1 Processor -- start 2023-09-30T22:27:02.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.145+0000 7f6bd3c30700 1 -- start start 2023-09-30T22:27:02.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.146+0000 7f6bd3c30700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6bcc096610 0x7f6bcc0969f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:02.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.146+0000 7f6bd3c30700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6bcc0684f0 0x7f6bcc12fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:02.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.146+0000 7f6bd3c30700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bcc069ad0 con 0x7f6bcc096ee0 2023-09-30T22:27:02.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.146+0000 7f6bd3c30700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bcc093420 con 0x7f6bcc096610 2023-09-30T22:27:02.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.146+0000 7f6bd3c30700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bcc091ef0 con 0x7f6bcc0684f0 2023-09-30T22:27:02.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.146+0000 7f6bd19cc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6bcc096610 0x7f6bcc0969f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:02.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.146+0000 7f6bd19cc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6bcc096610 0x7f6bcc0969f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54558/0 (socket says 172.21.15.16:54558) 2023-09-30T22:27:02.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.146+0000 7f6bd19cc700 1 -- 172.21.15.16:0/1511312598 learned_addr learned my addr 172.21.15.16:0/1511312598 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:02.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.146+0000 7f6bd19cc700 1 -- 172.21.15.16:0/1511312598 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6bcc0684f0 msgr2=0x7f6bcc12fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:02.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.146+0000 7f6bd19cc700 1 --2- 172.21.15.16:0/1511312598 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6bcc0684f0 0x7f6bcc12fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-09-30T22:27:02.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.146+0000 7f6bd19cc700 1 -- 172.21.15.16:0/1511312598 >> v1:172.21.15.16:6789/0 conn(0x7f6bcc096ee0 legacy=0x7f6bcc067de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:02.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.146+0000 7f6bd19cc700 1 -- 172.21.15.16:0/1511312598 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bcc092450 con 0x7f6bcc096610 2023-09-30T22:27:02.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.147+0000 7f6bd19cc700 1 --2- 172.21.15.16:0/1511312598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bcc096610 0x7f6bcc0969f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f6bc8006fd0 tx=0x7f6bc800b3b0).ready entity=mon.0 client_cookie=2422c9a3bf3d9c20 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:02.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.147+0000 7f6bd09ca700 1 -- 172.21.15.16:0/1511312598 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc800db40 con 0x7f6bcc096610 2023-09-30T22:27:02.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.147+0000 7f6bd09ca700 1 -- 172.21.15.16:0/1511312598 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bc800dca0 con 0x7f6bcc096610 2023-09-30T22:27:02.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.147+0000 7f6bd09ca700 1 -- 172.21.15.16:0/1511312598 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc8010680 con 0x7f6bcc096610 2023-09-30T22:27:02.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.147+0000 7f6bd3c30700 1 -- 172.21.15.16:0/1511312598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bcc096610 msgr2=0x7f6bcc0969f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:02.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.147+0000 7f6bd3c30700 1 --2- 172.21.15.16:0/1511312598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bcc096610 0x7f6bcc0969f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f6bc8006fd0 tx=0x7f6bc800b3b0).stop 2023-09-30T22:27:02.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.147+0000 7f6bd3c30700 1 -- 172.21.15.16:0/1511312598 shutdown_connections 2023-09-30T22:27:02.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.147+0000 7f6bd3c30700 1 --2- 172.21.15.16:0/1511312598 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6bcc0684f0 0x7f6bcc12fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-09-30T22:27:02.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.147+0000 7f6bd3c30700 1 --2- 172.21.15.16:0/1511312598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bcc096610 0x7f6bcc0969f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:02.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.147+0000 7f6bd3c30700 1 -- 172.21.15.16:0/1511312598 >> 172.21.15.16:0/1511312598 conn(0x7f6bcc09a220 msgr2=0x7f6bcc09a630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:02.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.148+0000 7f6bd3c30700 1 -- 172.21.15.16:0/1511312598 shutdown_connections 2023-09-30T22:27:02.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.148+0000 7f6bd3c30700 1 -- 172.21.15.16:0/1511312598 wait complete. 2023-09-30T22:27:02.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.149+0000 7f6bd3c30700 1 Processor -- start 2023-09-30T22:27:02.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.149+0000 7f6bd3c30700 1 -- start start 2023-09-30T22:27:02.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.150+0000 7f6bd3c30700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6bcc0684f0 0x7f6bcc1199a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:02.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.150+0000 7f6bd3c30700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bcc096610 0x7f6bcc119e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:02.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.150+0000 7f6bd3c30700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6bcc096ee0 0x7f6bcc11a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:02.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.150+0000 7f6bd3c30700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bcc066b20 con 0x7f6bcc096610 2023-09-30T22:27:02.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.150+0000 7f6bd3c30700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bcc11ca20 con 0x7f6bcc0684f0 2023-09-30T22:27:02.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.150+0000 7f6bd3c30700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6bcc091bf0 con 0x7f6bcc096ee0 2023-09-30T22:27:02.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.150+0000 7f6bd21cd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6bcc096ee0 0x7f6bcc11a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:02.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.150+0000 7f6bd21cd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6bcc096ee0 0x7f6bcc11a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:40198/0 (socket says 172.21.15.16:40198) 2023-09-30T22:27:02.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.150+0000 7f6bd21cd700 1 -- 172.21.15.16:0/1559789404 learned_addr learned my addr 172.21.15.16:0/1559789404 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:02.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.151+0000 7f6bd11cb700 1 --2- 172.21.15.16:0/1559789404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bcc096610 0x7f6bcc119e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:02.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.151+0000 7f6bd21cd700 1 -- 172.21.15.16:0/1559789404 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6bcc0684f0 msgr2=0x7f6bcc1199a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:02.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.151+0000 7f6bd21cd700 1 --2- 172.21.15.16:0/1559789404 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6bcc0684f0 0x7f6bcc1199a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-09-30T22:27:02.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.151+0000 7f6bd21cd700 1 -- 172.21.15.16:0/1559789404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bcc096610 msgr2=0x7f6bcc119e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:02.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.151+0000 7f6bd21cd700 1 --2- 172.21.15.16:0/1559789404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bcc096610 0x7f6bcc119e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:02.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.151+0000 7f6bd21cd700 1 -- 172.21.15.16:0/1559789404 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bc800b610 con 0x7f6bcc096ee0 2023-09-30T22:27:02.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.151+0000 7f6bd21cd700 1 --2- 172.21.15.16:0/1559789404 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6bcc096ee0 0x7f6bcc11a380 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f6bc400bc60 tx=0x7f6bc4009840).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:02.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.151+0000 7f6bc1ffb700 1 -- 172.21.15.16:0/1559789404 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc400c7b0 con 0x7f6bcc096ee0 2023-09-30T22:27:02.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.151+0000 7f6bc1ffb700 1 -- 172.21.15.16:0/1559789404 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bc4010070 con 0x7f6bcc096ee0 2023-09-30T22:27:02.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.151+0000 7f6bc1ffb700 1 -- 172.21.15.16:0/1559789404 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc400c970 con 0x7f6bcc096ee0 2023-09-30T22:27:02.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.151+0000 7f6bd3c30700 1 -- 172.21.15.16:0/1559789404 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bcc1195c0 con 0x7f6bcc096ee0 2023-09-30T22:27:02.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.152+0000 7f6bd3c30700 1 -- 172.21.15.16:0/1559789404 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bcc11adf0 con 0x7f6bcc096ee0 2023-09-30T22:27:02.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.153+0000 7f6bd3c30700 1 -- 172.21.15.16:0/1559789404 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bcc11c7c0 con 0x7f6bcc096ee0 2023-09-30T22:27:02.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.154+0000 7f6bc1ffb700 1 -- 172.21.15.16:0/1559789404 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f6bc400cd20 con 0x7f6bcc096ee0 2023-09-30T22:27:02.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.154+0000 7f6bc1ffb700 1 -- 172.21.15.16:0/1559789404 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f6bc403ad20 con 0x7f6bcc096ee0 2023-09-30T22:27:02.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.163+0000 7f6bc1ffb700 1 -- 172.21.15.16:0/1559789404 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f6bcc11c7c0 con 0x7f6bcc096ee0 2023-09-30T22:27:02.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.281+0000 7f589c86c700 1 -- 172.21.15.16:0/739913776 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5898118250 con 0x7f5898059070 2023-09-30T22:27:02.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.282+0000 7f58867fc700 1 -- 172.21.15.16:0/739913776 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5888020020 con 0x7f5898059070 2023-09-30T22:27:02.283 INFO:teuthology.orchestra.run.smithi016.stdout:38654705668 2023-09-30T22:27:02.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.285+0000 7f589c86c700 1 -- 172.21.15.16:0/739913776 >> v1:172.21.15.16:6816/118285 conn(0x7f587802bb90 legacy=0x7f587802e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:02.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.285+0000 7f589c86c700 1 -- 172.21.15.16:0/739913776 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5898059070 msgr2=0x7f58981259d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:02.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.285+0000 7f589c86c700 1 --2- 172.21.15.16:0/739913776 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5898059070 0x7f58981259d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f5888011fc0 tx=0x7f5888017000).stop 2023-09-30T22:27:02.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.285+0000 7f589c86c700 1 -- 172.21.15.16:0/739913776 shutdown_connections 2023-09-30T22:27:02.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.285+0000 7f589c86c700 1 --2- 172.21.15.16:0/739913776 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5898127d80 0x7f5898114df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:02.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.285+0000 7f589c86c700 1 --2- 172.21.15.16:0/739913776 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5898059070 0x7f58981259d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:02.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.285+0000 7f589c86c700 1 --2- 172.21.15.16:0/739913776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5898127390 0x7f5898114900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:02.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.285+0000 7f589c86c700 1 -- 172.21.15.16:0/739913776 >> 172.21.15.16:0/739913776 conn(0x7f5898097470 msgr2=0x7f5898099e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:02.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.285+0000 7f589c86c700 1 -- 172.21.15.16:0/739913776 shutdown_connections 2023-09-30T22:27:02.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.285+0000 7f589c86c700 1 -- 172.21.15.16:0/739913776 wait complete. 2023-09-30T22:27:02.300 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705668 for osd.1 2023-09-30T22:27:02.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.309+0000 7f1b7dd1c700 1 -- 172.21.15.16:0/3516608951 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1b580057f0 con 0x7f1b78096820 2023-09-30T22:27:02.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.310+0000 7f1b5f7fe700 1 -- 172.21.15.16:0/3516608951 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1b680263c0 con 0x7f1b78096820 2023-09-30T22:27:02.311 INFO:teuthology.orchestra.run.smithi016.stdout:38654705668 2023-09-30T22:27:02.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.311+0000 7f1b7dd1c700 1 -- 172.21.15.16:0/3516608951 >> v1:172.21.15.16:6816/118285 conn(0x7f1b6402bbe0 legacy=0x7f1b6402e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:02.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.311+0000 7f1b7dd1c700 1 -- 172.21.15.16:0/3516608951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b78096820 msgr2=0x7f1b7813f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:02.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.311+0000 7f1b7dd1c700 1 --2- 172.21.15.16:0/3516608951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b78096820 0x7f1b7813f2c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f1b68013000 tx=0x7f1b68016000).stop 2023-09-30T22:27:02.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.312+0000 7f1b7dd1c700 1 -- 172.21.15.16:0/3516608951 shutdown_connections 2023-09-30T22:27:02.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.312+0000 7f1b7dd1c700 1 --2- 172.21.15.16:0/3516608951 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1b78067940 0x7f1b7813edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:02.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.312+0000 7f1b7dd1c700 1 --2- 172.21.15.16:0/3516608951 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1b780975e0 0x7f1b7813f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:02.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.312+0000 7f1b7dd1c700 1 --2- 172.21.15.16:0/3516608951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b78096820 0x7f1b7813f2c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:02.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.312+0000 7f1b7dd1c700 1 -- 172.21.15.16:0/3516608951 >> 172.21.15.16:0/3516608951 conn(0x7f1b7809ba50 msgr2=0x7f1b780998d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:02.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.312+0000 7f1b7dd1c700 1 -- 172.21.15.16:0/3516608951 shutdown_connections 2023-09-30T22:27:02.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.312+0000 7f1b7dd1c700 1 -- 172.21.15.16:0/3516608951 wait complete. 2023-09-30T22:27:02.326 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705668 for osd.0 2023-09-30T22:27:02.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.332+0000 7f5f70c47700 1 -- 172.21.15.16:0/466948315 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f5f6c05a260 con 0x7f5f6c126f70 2023-09-30T22:27:02.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.332+0000 7f5f5a7fc700 1 -- 172.21.15.16:0/466948315 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5f64007340 con 0x7f5f6c126f70 2023-09-30T22:27:02.334 INFO:teuthology.orchestra.run.smithi016.stdout:38654705669 2023-09-30T22:27:02.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.334+0000 7f5f70c47700 1 -- 172.21.15.16:0/466948315 >> v1:172.21.15.16:6816/118285 conn(0x7f5f4c02bb60 legacy=0x7f5f4c02e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:02.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.334+0000 7f5f70c47700 1 -- 172.21.15.16:0/466948315 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5f6c126f70 msgr2=0x7f5f6c112030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:02.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.334+0000 7f5f70c47700 1 --2- 172.21.15.16:0/466948315 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5f6c126f70 0x7f5f6c112030 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f5f64009fc0 tx=0x7f5f6400b040).stop 2023-09-30T22:27:02.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.334+0000 7f5f70c47700 1 -- 172.21.15.16:0/466948315 shutdown_connections 2023-09-30T22:27:02.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.334+0000 7f5f70c47700 1 --2- 172.21.15.16:0/466948315 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5f6c126f70 0x7f5f6c112030 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:02.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.334+0000 7f5f70c47700 1 --2- 172.21.15.16:0/466948315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f6c067ce0 0x7f5f6c111650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:02.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.334+0000 7f5f70c47700 1 --2- 172.21.15.16:0/466948315 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5f6c123a90 0x7f5f6c111b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:02.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.334+0000 7f5f70c47700 1 -- 172.21.15.16:0/466948315 >> 172.21.15.16:0/466948315 conn(0x7f5f6c09a220 msgr2=0x7f5f6c098c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:02.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.334+0000 7f5f70c47700 1 -- 172.21.15.16:0/466948315 shutdown_connections 2023-09-30T22:27:02.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.334+0000 7f5f70c47700 1 -- 172.21.15.16:0/466948315 wait complete. 2023-09-30T22:27:02.348 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705669 for osd.2 2023-09-30T22:27:02.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.362+0000 7f6bd3c30700 1 -- 172.21.15.16:0/1559789404 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6bcc11c7c0 con 0x7f6bcc096ee0 2023-09-30T22:27:02.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.362+0000 7f6bc1ffb700 1 -- 172.21.15.16:0/1559789404 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6bcc11c7c0 con 0x7f6bcc096ee0 2023-09-30T22:27:02.364 INFO:teuthology.orchestra.run.smithi016.stdout:38654705668 2023-09-30T22:27:02.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.364+0000 7f6bd3c30700 1 -- 172.21.15.16:0/1559789404 >> v1:172.21.15.16:6816/118285 conn(0x7f6bb002bb60 legacy=0x7f6bb002e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:02.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.364+0000 7f6bd3c30700 1 -- 172.21.15.16:0/1559789404 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6bcc096ee0 msgr2=0x7f6bcc11a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:02.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.364+0000 7f6bd3c30700 1 --2- 172.21.15.16:0/1559789404 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6bcc096ee0 0x7f6bcc11a380 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f6bc400bc60 tx=0x7f6bc4009840).stop 2023-09-30T22:27:02.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.364+0000 7f6bd3c30700 1 -- 172.21.15.16:0/1559789404 shutdown_connections 2023-09-30T22:27:02.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.364+0000 7f6bd3c30700 1 --2- 172.21.15.16:0/1559789404 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6bcc0684f0 0x7f6bcc1199a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-09-30T22:27:02.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.364+0000 7f6bd3c30700 1 --2- 172.21.15.16:0/1559789404 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6bcc096ee0 0x7f6bcc11a380 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:02.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.364+0000 7f6bd3c30700 1 --2- 172.21.15.16:0/1559789404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bcc096610 0x7f6bcc119e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:02.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.364+0000 7f6bd3c30700 1 -- 172.21.15.16:0/1559789404 >> 172.21.15.16:0/1559789404 conn(0x7f6bcc09a220 msgr2=0x7f6bcc129e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:02.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.364+0000 7f6bd3c30700 1 -- 172.21.15.16:0/1559789404 shutdown_connections 2023-09-30T22:27:02.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:02.365+0000 7f6bd3c30700 1 -- 172.21.15.16:0/1559789404 wait complete. 2023-09-30T22:27:02.379 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705668 for osd.3 2023-09-30T22:27:03.301 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:27:03.327 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:27:03.349 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:27:03.380 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:27:03.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.445+0000 7fabc939f700 1 Processor -- start 2023-09-30T22:27:03.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.445+0000 7fabc939f700 1 -- start start 2023-09-30T22:27:03.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.445+0000 7fabc939f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fabc4096380 0x7fabc4094730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:03.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.445+0000 7fabc939f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fabc4094c20 0x7fabc4127d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:03.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.446+0000 7fabc939f700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabc4096760 con 0x7fabc4091b00 2023-09-30T22:27:03.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.446+0000 7fabc939f700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabc4091ff0 con 0x7fabc4096380 2023-09-30T22:27:03.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.446+0000 7fabc939f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fabc4092c80 con 0x7fabc4094c20 2023-09-30T22:27:03.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.446+0000 7fabc37fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fabc4094c20 0x7fabc4127d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:03.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.446+0000 7fabc27fc700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fabc4091b00 0x7fabc4127600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51480/0 (socket says 172.21.15.16:51480) 2023-09-30T22:27:03.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.446+0000 7fabc27fc700 1 -- 172.21.15.16:0/2455947136 learned_addr learned my addr 172.21.15.16:0/2455947136 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:27:03.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.446+0000 7fabc2ffd700 1 --2- 172.21.15.16:0/2455947136 >> v2:172.21.15.16:3300/0 conn(0x7fabc4096380 0x7fabc4094730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:03.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.446+0000 7fabc37fe700 1 -- 172.21.15.16:0/2455947136 >> v2:172.21.15.16:3300/0 conn(0x7fabc4096380 msgr2=0x7fabc4094730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:03.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.446+0000 7fabc37fe700 1 --2- 172.21.15.16:0/2455947136 >> v2:172.21.15.16:3300/0 conn(0x7fabc4096380 0x7fabc4094730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:03.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.446+0000 7fabc37fe700 1 -- 172.21.15.16:0/2455947136 >> v1:172.21.15.16:6789/0 conn(0x7fabc4091b00 legacy=0x7fabc4127600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:03.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.446+0000 7fabc1ffb700 1 -- 172.21.15.16:0/2455947136 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2786825650 0 0) 0x7fabc4096760 con 0x7fabc4091b00 2023-09-30T22:27:03.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.446+0000 7fabc37fe700 1 -- 172.21.15.16:0/2455947136 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc4093900 con 0x7fabc4094c20 2023-09-30T22:27:03.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.446+0000 7fabc37fe700 1 --2- 172.21.15.16:0/2455947136 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fabc4094c20 0x7fabc4127d10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fabbc009e60 tx=0x7fabbc009780).ready entity=mon.2 client_cookie=be15f7e6b037adf2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:03.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.447+0000 7fabc1ffb700 1 -- 172.21.15.16:0/2455947136 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabbc00e070 con 0x7fabc4094c20 2023-09-30T22:27:03.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.447+0000 7fabc1ffb700 1 -- 172.21.15.16:0/2455947136 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabbc00bb60 con 0x7fabc4094c20 2023-09-30T22:27:03.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.447+0000 7fabc1ffb700 1 -- 172.21.15.16:0/2455947136 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabbc012410 con 0x7fabc4094c20 2023-09-30T22:27:03.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.447+0000 7fabc939f700 1 -- 172.21.15.16:0/2455947136 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fabc4094c20 msgr2=0x7fabc4127d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:03.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.447+0000 7fabc939f700 1 --2- 172.21.15.16:0/2455947136 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fabc4094c20 0x7fabc4127d10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fabbc009e60 tx=0x7fabbc009780).stop 2023-09-30T22:27:03.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.448+0000 7fabc939f700 1 -- 172.21.15.16:0/2455947136 shutdown_connections 2023-09-30T22:27:03.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.448+0000 7fabc939f700 1 --2- 172.21.15.16:0/2455947136 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fabc4094c20 0x7fabc4127d10 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:03.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.448+0000 7fabc939f700 1 --2- 172.21.15.16:0/2455947136 >> v2:172.21.15.16:3300/0 conn(0x7fabc4096380 0x7fabc4094730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:03.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.448+0000 7fabc939f700 1 -- 172.21.15.16:0/2455947136 >> 172.21.15.16:0/2455947136 conn(0x7fabc4097470 msgr2=0x7fabc4097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:03.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.448+0000 7fabc939f700 1 -- 172.21.15.16:0/2455947136 shutdown_connections 2023-09-30T22:27:03.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.448+0000 7fabc939f700 1 -- 172.21.15.16:0/2455947136 wait complete. 2023-09-30T22:27:03.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.449+0000 7fabc939f700 1 Processor -- start 2023-09-30T22:27:03.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.449+0000 7fabc939f700 1 -- start start 2023-09-30T22:27:03.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.449+0000 7fabc939f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fabc4091b00 0x7fabc4069000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:03.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.449+0000 7fabc939f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabc4094c20 0x7fabc40694f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:03.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.449+0000 7fabc939f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fabc4096380 0x7fabc40699e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:03.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.449+0000 7fabc939f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc4098da0 con 0x7fabc4094c20 2023-09-30T22:27:03.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.449+0000 7fabc939f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabc4112c50 con 0x7fabc4096380 2023-09-30T22:27:03.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.449+0000 7fabc939f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fabc4112990 con 0x7fabc4091b00 2023-09-30T22:27:03.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.449+0000 7fabc37fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fabc4096380 0x7fabc40699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:03.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.449+0000 7fabc37fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fabc4096380 0x7fabc40699e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:42310/0 (socket says 172.21.15.16:42310) 2023-09-30T22:27:03.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.450+0000 7fabc37fe700 1 -- 172.21.15.16:0/1901683723 learned_addr learned my addr 172.21.15.16:0/1901683723 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:03.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.450+0000 7fabc2ffd700 1 --2- 172.21.15.16:0/1901683723 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fabc4091b00 0x7fabc4069000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:03.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.450+0000 7fabc37fe700 1 -- 172.21.15.16:0/1901683723 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fabc4091b00 msgr2=0x7fabc4069000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:03.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.450+0000 7fabc37fe700 1 --2- 172.21.15.16:0/1901683723 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fabc4091b00 0x7fabc4069000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:03.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.450+0000 7fabc37fe700 1 -- 172.21.15.16:0/1901683723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabc4094c20 msgr2=0x7fabc40694f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:03.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.450+0000 7fabc37fe700 1 --2- 172.21.15.16:0/1901683723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabc4094c20 0x7fabc40694f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-09-30T22:27:03.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.450+0000 7fabc37fe700 1 -- 172.21.15.16:0/1901683723 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabbc009a20 con 0x7fabc4096380 2023-09-30T22:27:03.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.450+0000 7fabc37fe700 1 --2- 172.21.15.16:0/1901683723 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fabc4096380 0x7fabc40699e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fabc4092dc0 tx=0x7fabbc009780).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:03.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.450+0000 7fabaaffd700 1 -- 172.21.15.16:0/1901683723 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabbc00b6f0 con 0x7fabc4096380 2023-09-30T22:27:03.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.450+0000 7fabaaffd700 1 -- 172.21.15.16:0/1901683723 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabbc01d630 con 0x7fabc4096380 2023-09-30T22:27:03.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.450+0000 7fabaaffd700 1 -- 172.21.15.16:0/1901683723 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabbc01d9b0 con 0x7fabc4096380 2023-09-30T22:27:03.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.450+0000 7fabc939f700 1 -- 172.21.15.16:0/1901683723 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabc406a500 con 0x7fabc4096380 2023-09-30T22:27:03.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.450+0000 7fabc939f700 1 -- 172.21.15.16:0/1901683723 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabc41137d0 con 0x7fabc4096380 2023-09-30T22:27:03.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.451+0000 7fabaaffd700 1 -- 172.21.15.16:0/1901683723 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fabbc003020 con 0x7fabc4096380 2023-09-30T22:27:03.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.452+0000 7fabaaffd700 1 -- 172.21.15.16:0/1901683723 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fabbc03cbe0 con 0x7fabc4096380 2023-09-30T22:27:03.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.452+0000 7fabc939f700 1 -- 172.21.15.16:0/1901683723 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabc41137d0 con 0x7fabc4096380 2023-09-30T22:27:03.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.457+0000 7fabaaffd700 1 -- 172.21.15.16:0/1901683723 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fabc41137d0 con 0x7fabc4096380 2023-09-30T22:27:03.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.492+0000 7f1f68762700 1 Processor -- start 2023-09-30T22:27:03.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.492+0000 7f1f68762700 1 -- start start 2023-09-30T22:27:03.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.493+0000 7f1f68762700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f60096380 0x7f1f60094730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:03.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.493+0000 7f1f68762700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f60091b00 0x7f1f60127600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:03.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.493+0000 7f1f68762700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1f60094c20 0x7f1f60127af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:03.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.493+0000 7f1f68762700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f60096760 con 0x7f1f60094c20 2023-09-30T22:27:03.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.493+0000 7f1f68762700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f60091ff0 con 0x7f1f60096380 2023-09-30T22:27:03.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.493+0000 7f1f68762700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1f60092c80 con 0x7f1f60091b00 2023-09-30T22:27:03.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.493+0000 7f1f664fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f60096380 0x7f1f60094730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:03.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.493+0000 7f1f664fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f60096380 0x7f1f60094730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:42322/0 (socket says 172.21.15.16:42322) 2023-09-30T22:27:03.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.493+0000 7f1f664fe700 1 -- 172.21.15.16:0/3180219139 learned_addr learned my addr 172.21.15.16:0/3180219139 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:03.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.493+0000 7f1f65cfd700 1 --2- 172.21.15.16:0/3180219139 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f60091b00 0x7f1f60127600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:03.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.493+0000 7f1f66cff700 1 --2- 172.21.15.16:0/3180219139 >> v2:172.21.15.16:3300/0 conn(0x7f1f60094c20 0x7f1f60127af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:03.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.493+0000 7f1f664fe700 1 -- 172.21.15.16:0/3180219139 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f60091b00 msgr2=0x7f1f60127600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:03.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.493+0000 7f1f664fe700 1 --2- 172.21.15.16:0/3180219139 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f60091b00 0x7f1f60127600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:03.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.494+0000 7f1f664fe700 1 -- 172.21.15.16:0/3180219139 >> v2:172.21.15.16:3300/0 conn(0x7f1f60094c20 msgr2=0x7f1f60127af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:03.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.494+0000 7f1f664fe700 1 --2- 172.21.15.16:0/3180219139 >> v2:172.21.15.16:3300/0 conn(0x7f1f60094c20 0x7f1f60127af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:03.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.494+0000 7f1f664fe700 1 -- 172.21.15.16:0/3180219139 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f60093900 con 0x7f1f60096380 2023-09-30T22:27:03.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.494+0000 7f1f66cff700 1 --2- 172.21.15.16:0/3180219139 >> v2:172.21.15.16:3300/0 conn(0x7f1f60094c20 0x7f1f60127af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:27:03.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.494+0000 7f1f65cfd700 1 --2- 172.21.15.16:0/3180219139 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f60091b00 0x7f1f60127600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:03.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.494+0000 7f1f664fe700 1 --2- 172.21.15.16:0/3180219139 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f60096380 0x7f1f60094730 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f1f5c009830 tx=0x7f1f5c0090f0).ready entity=mon.1 client_cookie=c52734230d0655d6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:03.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.494+0000 7f1f654fc700 1 -- 172.21.15.16:0/3180219139 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c012980 con 0x7f1f60096380 2023-09-30T22:27:03.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.494+0000 7f1f654fc700 1 -- 172.21.15.16:0/3180219139 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f5c012ae0 con 0x7f1f60096380 2023-09-30T22:27:03.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.494+0000 7f1f654fc700 1 -- 172.21.15.16:0/3180219139 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c012e60 con 0x7f1f60096380 2023-09-30T22:27:03.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.495+0000 7f1f68762700 1 -- 172.21.15.16:0/3180219139 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f60096380 msgr2=0x7f1f60094730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:03.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.495+0000 7f1f68762700 1 --2- 172.21.15.16:0/3180219139 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f60096380 0x7f1f60094730 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f1f5c009830 tx=0x7f1f5c0090f0).stop 2023-09-30T22:27:03.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.495+0000 7f1f68762700 1 -- 172.21.15.16:0/3180219139 shutdown_connections 2023-09-30T22:27:03.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.495+0000 7f1f68762700 1 --2- 172.21.15.16:0/3180219139 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f60096380 0x7f1f60094730 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:03.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.495+0000 7f1f68762700 1 --2- 172.21.15.16:0/3180219139 >> v2:172.21.15.16:3300/0 conn(0x7f1f60094c20 0x7f1f60127af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:03.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.495+0000 7f1f68762700 1 --2- 172.21.15.16:0/3180219139 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f60091b00 0x7f1f60127600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:03.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.495+0000 7f1f68762700 1 -- 172.21.15.16:0/3180219139 >> 172.21.15.16:0/3180219139 conn(0x7f1f60097470 msgr2=0x7f1f60097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:03.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.495+0000 7f1f68762700 1 -- 172.21.15.16:0/3180219139 shutdown_connections 2023-09-30T22:27:03.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.495+0000 7f1f68762700 1 -- 172.21.15.16:0/3180219139 wait complete. 2023-09-30T22:27:03.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.497+0000 7f1f68762700 1 Processor -- start 2023-09-30T22:27:03.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.497+0000 7f1f68762700 1 -- start start 2023-09-30T22:27:03.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.497+0000 7f1f68762700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f60091b00 0x7f1f601248f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:03.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.497+0000 7f1f68762700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f60094c20 0x7f1f601127d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:03.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.497+0000 7f1f68762700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f60096380 0x7f1f60112cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:03.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.497+0000 7f1f68762700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f60098f80 con 0x7f1f60094c20 2023-09-30T22:27:03.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.497+0000 7f1f68762700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6006a550 con 0x7f1f60096380 2023-09-30T22:27:03.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.497+0000 7f1f68762700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1f60116330 con 0x7f1f60091b00 2023-09-30T22:27:03.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.497+0000 7f1f66cff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f60096380 0x7f1f60112cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:03.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.497+0000 7f1f66cff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f60096380 0x7f1f60112cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:42334/0 (socket says 172.21.15.16:42334) 2023-09-30T22:27:03.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.497+0000 7f1f66cff700 1 -- 172.21.15.16:0/1064854638 learned_addr learned my addr 172.21.15.16:0/1064854638 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:03.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.497+0000 7f1f664fe700 1 --2- 172.21.15.16:0/1064854638 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f60091b00 0x7f1f601248f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:03.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.497+0000 7f1f65cfd700 1 --2- 172.21.15.16:0/1064854638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f60094c20 0x7f1f601127d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:03.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.498+0000 7f1f66cff700 1 -- 172.21.15.16:0/1064854638 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f60091b00 msgr2=0x7f1f601248f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:03.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.498+0000 7f1f66cff700 1 --2- 172.21.15.16:0/1064854638 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f60091b00 0x7f1f601248f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:03.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.498+0000 7f1f66cff700 1 -- 172.21.15.16:0/1064854638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f60094c20 msgr2=0x7f1f601127d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:03.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.498+0000 7f1f66cff700 1 --2- 172.21.15.16:0/1064854638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f60094c20 0x7f1f601127d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:03.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.498+0000 7f1f66cff700 1 -- 172.21.15.16:0/1064854638 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f500092a0 con 0x7f1f60096380 2023-09-30T22:27:03.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.498+0000 7f1f66cff700 1 --2- 172.21.15.16:0/1064854638 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f60096380 0x7f1f60112cc0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f1f5800d180 tx=0x7f1f58009a50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:03.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.498+0000 7f1f567fc700 1 -- 172.21.15.16:0/1064854638 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5800dcc0 con 0x7f1f60096380 2023-09-30T22:27:03.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.498+0000 7f1f567fc700 1 -- 172.21.15.16:0/1064854638 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f5800f040 con 0x7f1f60096380 2023-09-30T22:27:03.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.498+0000 7f1f567fc700 1 -- 172.21.15.16:0/1064854638 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f58013600 con 0x7f1f60096380 2023-09-30T22:27:03.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.498+0000 7f1f68762700 1 -- 172.21.15.16:0/1064854638 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f5c009370 con 0x7f1f60096380 2023-09-30T22:27:03.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.499+0000 7f1f68762700 1 -- 172.21.15.16:0/1064854638 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f60116b50 con 0x7f1f60096380 2023-09-30T22:27:03.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.500+0000 7f1f567fc700 1 -- 172.21.15.16:0/1064854638 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f1f580139f0 con 0x7f1f60096380 2023-09-30T22:27:03.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.500+0000 7f1f567fc700 1 -- 172.21.15.16:0/1064854638 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f1f5803b470 con 0x7f1f60096380 2023-09-30T22:27:03.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.500+0000 7f1f68762700 1 -- 172.21.15.16:0/1064854638 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f60116b50 con 0x7f1f60096380 2023-09-30T22:27:03.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.505+0000 7f1f567fc700 1 -- 172.21.15.16:0/1064854638 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f1f60116b50 con 0x7f1f60096380 2023-09-30T22:27:03.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.530+0000 7fd235a1b700 1 Processor -- start 2023-09-30T22:27:03.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.530+0000 7fd235a1b700 1 -- start start 2023-09-30T22:27:03.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.531+0000 7fd235a1b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd2301251c0 0x7fd23012da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:03.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.531+0000 7fd235a1b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd230125dd0 0x7fd23012ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:03.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.531+0000 7fd235a1b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd230095f00 con 0x7fd230059070 2023-09-30T22:27:03.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.531+0000 7fd235a1b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd230059450 con 0x7fd2301251c0 2023-09-30T22:27:03.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.531+0000 7fd235a1b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd230096190 con 0x7fd230125dd0 2023-09-30T22:27:03.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.531+0000 7fd22f7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd230125dd0 0x7fd23012ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:03.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.531+0000 7fd22f7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd230125dd0 0x7fd23012ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:40244/0 (socket says 172.21.15.16:40244) 2023-09-30T22:27:03.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.531+0000 7fd22f7fe700 1 -- 172.21.15.16:0/3889160111 learned_addr learned my addr 172.21.15.16:0/3889160111 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:03.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.531+0000 7fd22e7fc700 1 --2- 172.21.15.16:0/3889160111 >> v2:172.21.15.16:3300/0 conn(0x7fd2301251c0 0x7fd23012da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:03.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.531+0000 7fd22f7fe700 1 -- 172.21.15.16:0/3889160111 >> v2:172.21.15.16:3300/0 conn(0x7fd2301251c0 msgr2=0x7fd23012da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:03.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.531+0000 7fd22f7fe700 1 --2- 172.21.15.16:0/3889160111 >> v2:172.21.15.16:3300/0 conn(0x7fd2301251c0 0x7fd23012da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:03.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.531+0000 7fd22f7fe700 1 -- 172.21.15.16:0/3889160111 >> v1:172.21.15.16:6789/0 conn(0x7fd230059070 legacy=0x7fd2301256c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:03.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.531+0000 7fd22f7fe700 1 -- 172.21.15.16:0/3889160111 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd230059dd0 con 0x7fd230125dd0 2023-09-30T22:27:03.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.532+0000 7fd22f7fe700 1 --2- 172.21.15.16:0/3889160111 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd230125dd0 0x7fd23012ff80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fd228009f20 tx=0x7fd228009840).ready entity=mon.2 client_cookie=c78ff767db3ad79b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:03.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.532+0000 7fd22dffb700 1 -- 172.21.15.16:0/3889160111 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd22800e070 con 0x7fd230125dd0 2023-09-30T22:27:03.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.532+0000 7fd22dffb700 1 -- 172.21.15.16:0/3889160111 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd22800b9d0 con 0x7fd230125dd0 2023-09-30T22:27:03.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.532+0000 7fd22dffb700 1 -- 172.21.15.16:0/3889160111 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd22800bd50 con 0x7fd230125dd0 2023-09-30T22:27:03.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.533+0000 7fd235a1b700 1 -- 172.21.15.16:0/3889160111 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd230125dd0 msgr2=0x7fd23012ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:03.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.533+0000 7fd235a1b700 1 --2- 172.21.15.16:0/3889160111 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd230125dd0 0x7fd23012ff80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fd228009f20 tx=0x7fd228009840).stop 2023-09-30T22:27:03.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.533+0000 7fd235a1b700 1 -- 172.21.15.16:0/3889160111 shutdown_connections 2023-09-30T22:27:03.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.533+0000 7fd235a1b700 1 --2- 172.21.15.16:0/3889160111 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd230125dd0 0x7fd23012ff80 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:03.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.533+0000 7fd235a1b700 1 --2- 172.21.15.16:0/3889160111 >> v2:172.21.15.16:3300/0 conn(0x7fd2301251c0 0x7fd23012da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:03.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.533+0000 7fd235a1b700 1 -- 172.21.15.16:0/3889160111 >> 172.21.15.16:0/3889160111 conn(0x7fd230097470 msgr2=0x7fd230097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:03.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.533+0000 7fd235a1b700 1 -- 172.21.15.16:0/3889160111 shutdown_connections 2023-09-30T22:27:03.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.533+0000 7fd235a1b700 1 -- 172.21.15.16:0/3889160111 wait complete. 2023-09-30T22:27:03.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.534+0000 7fd235a1b700 1 Processor -- start 2023-09-30T22:27:03.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.534+0000 7fd235a1b700 1 -- start start 2023-09-30T22:27:03.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.535+0000 7fd235a1b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd230059070 0x7fd23012a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:03.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.535+0000 7fd235a1b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd2301251c0 0x7fd23012dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:03.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.535+0000 7fd235a1b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd230125dd0 0x7fd23012dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:03.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.535+0000 7fd235a1b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2300958d0 con 0x7fd230125dd0 2023-09-30T22:27:03.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.535+0000 7fd235a1b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd23012fcf0 con 0x7fd2301251c0 2023-09-30T22:27:03.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.535+0000 7fd235a1b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd23012cc70 con 0x7fd230059070 2023-09-30T22:27:03.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.535+0000 7fd22effd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd230059070 0x7fd23012a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:03.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.535+0000 7fd22effd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd230059070 0x7fd23012a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:40252/0 (socket says 172.21.15.16:40252) 2023-09-30T22:27:03.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.535+0000 7fd22effd700 1 -- 172.21.15.16:0/3544509886 learned_addr learned my addr 172.21.15.16:0/3544509886 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:03.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.535+0000 7fd22f7fe700 1 --2- 172.21.15.16:0/3544509886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd230125dd0 0x7fd23012dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:03.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.535+0000 7fd22effd700 1 -- 172.21.15.16:0/3544509886 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd2301251c0 msgr2=0x7fd23012dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:03.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.535+0000 7fd22effd700 1 --2- 172.21.15.16:0/3544509886 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd2301251c0 0x7fd23012dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-09-30T22:27:03.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.536+0000 7fd22effd700 1 -- 172.21.15.16:0/3544509886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd230125dd0 msgr2=0x7fd23012dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:03.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.536+0000 7fd22effd700 1 --2- 172.21.15.16:0/3544509886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd230125dd0 0x7fd23012dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:03.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.536+0000 7fd22effd700 1 -- 172.21.15.16:0/3544509886 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd228009ae0 con 0x7fd230059070 2023-09-30T22:27:03.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.536+0000 7fd22f7fe700 1 --2- 172.21.15.16:0/3544509886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd230125dd0 0x7fd23012dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:27:03.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.536+0000 7fd22effd700 1 --2- 172.21.15.16:0/3544509886 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd230059070 0x7fd23012a860 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fd22000c4c0 tx=0x7fd220007e00).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:03.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.536+0000 7fd216ffd700 1 -- 172.21.15.16:0/3544509886 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd220007020 con 0x7fd230059070 2023-09-30T22:27:03.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.536+0000 7fd216ffd700 1 -- 172.21.15.16:0/3544509886 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd220001e90 con 0x7fd230059070 2023-09-30T22:27:03.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.536+0000 7fd235a1b700 1 -- 172.21.15.16:0/3544509886 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd23012cee0 con 0x7fd230059070 2023-09-30T22:27:03.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.536+0000 7fd216ffd700 1 -- 172.21.15.16:0/3544509886 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2200073b0 con 0x7fd230059070 2023-09-30T22:27:03.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.536+0000 7fd235a1b700 1 -- 172.21.15.16:0/3544509886 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd23012c940 con 0x7fd230059070 2023-09-30T22:27:03.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.537+0000 7fd216ffd700 1 -- 172.21.15.16:0/3544509886 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fd2200068b0 con 0x7fd230059070 2023-09-30T22:27:03.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.537+0000 7fd216ffd700 1 -- 172.21.15.16:0/3544509886 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fd22003aa40 con 0x7fd230059070 2023-09-30T22:27:03.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.539+0000 7fd235a1b700 1 -- 172.21.15.16:0/3544509886 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd23005a260 con 0x7fd230059070 2023-09-30T22:27:03.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.541+0000 7f3ba462e700 1 Processor -- start 2023-09-30T22:27:03.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.541+0000 7f3ba462e700 1 -- start start 2023-09-30T22:27:03.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.541+0000 7f3ba462e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3b9c059070 0x7f3b9c127890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:03.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.541+0000 7f3ba462e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3b9c127390 0x7f3b9c127d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:03.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.541+0000 7f3ba462e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3b9c095370 0x7f3b9c0957f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:03.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.541+0000 7f3ba462e700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b9c096940 con 0x7f3b9c095370 2023-09-30T22:27:03.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.541+0000 7f3ba462e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9c059c00 con 0x7f3b9c127390 2023-09-30T22:27:03.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.541+0000 7f3ba462e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9c091b60 con 0x7f3b9c059070 2023-09-30T22:27:03.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.542+0000 7f3ba23ca700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3b9c059070 0x7f3b9c127890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:03.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.542+0000 7f3ba23ca700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3b9c059070 0x7f3b9c127890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:40258/0 (socket says 172.21.15.16:40258) 2023-09-30T22:27:03.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.542+0000 7f3ba23ca700 1 -- 172.21.15.16:0/132111148 learned_addr learned my addr 172.21.15.16:0/132111148 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:03.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.542+0000 7f3ba1bc9700 1 --2- 172.21.15.16:0/132111148 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3b9c127390 0x7f3b9c127d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:03.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.542+0000 7f3ba23ca700 1 -- 172.21.15.16:0/132111148 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3b9c127390 msgr2=0x7f3b9c127d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:03.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.542+0000 7f3ba23ca700 1 --2- 172.21.15.16:0/132111148 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3b9c127390 0x7f3b9c127d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:03.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.542+0000 7f3ba23ca700 1 -- 172.21.15.16:0/132111148 >> v2:172.21.15.16:3300/0 conn(0x7f3b9c095370 msgr2=0x7f3b9c0957f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:03.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.542+0000 7f3ba23ca700 1 --2- 172.21.15.16:0/132111148 >> v2:172.21.15.16:3300/0 conn(0x7f3b9c095370 0x7f3b9c0957f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-09-30T22:27:03.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.542+0000 7f3ba23ca700 1 -- 172.21.15.16:0/132111148 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b9c0929d0 con 0x7f3b9c059070 2023-09-30T22:27:03.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.542+0000 7f3ba23ca700 1 --2- 172.21.15.16:0/132111148 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3b9c059070 0x7f3b9c127890 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f3b98009830 tx=0x7f3b980090f0).ready entity=mon.2 client_cookie=abb8bfc8ad8dc6ca server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:03.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.548+0000 7f3ba13c8700 1 -- 172.21.15.16:0/132111148 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9800b580 con 0x7f3b9c059070 2023-09-30T22:27:03.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.548+0000 7f3ba13c8700 1 -- 172.21.15.16:0/132111148 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b9800b6e0 con 0x7f3b9c059070 2023-09-30T22:27:03.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.548+0000 7f3ba13c8700 1 -- 172.21.15.16:0/132111148 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9800ba80 con 0x7f3b9c059070 2023-09-30T22:27:03.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.548+0000 7fd216ffd700 1 -- 172.21.15.16:0/3544509886 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fd22000f470 con 0x7fd230059070 2023-09-30T22:27:03.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.548+0000 7f3ba462e700 1 -- 172.21.15.16:0/132111148 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3b9c059070 msgr2=0x7f3b9c127890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:03.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.548+0000 7f3ba462e700 1 --2- 172.21.15.16:0/132111148 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3b9c059070 0x7f3b9c127890 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f3b98009830 tx=0x7f3b980090f0).stop 2023-09-30T22:27:03.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.549+0000 7f3ba462e700 1 -- 172.21.15.16:0/132111148 shutdown_connections 2023-09-30T22:27:03.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.549+0000 7f3ba462e700 1 --2- 172.21.15.16:0/132111148 >> v2:172.21.15.16:3300/0 conn(0x7f3b9c095370 0x7f3b9c0957f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-09-30T22:27:03.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.549+0000 7f3ba462e700 1 --2- 172.21.15.16:0/132111148 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3b9c059070 0x7f3b9c127890 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:03.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.549+0000 7f3ba462e700 1 --2- 172.21.15.16:0/132111148 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3b9c127390 0x7f3b9c127d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:03.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.549+0000 7f3ba462e700 1 -- 172.21.15.16:0/132111148 >> 172.21.15.16:0/132111148 conn(0x7f3b9c097470 msgr2=0x7f3b9c097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:03.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.549+0000 7f3ba462e700 1 -- 172.21.15.16:0/132111148 shutdown_connections 2023-09-30T22:27:03.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.549+0000 7f3ba462e700 1 -- 172.21.15.16:0/132111148 wait complete. 2023-09-30T22:27:03.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.550+0000 7f3ba462e700 1 Processor -- start 2023-09-30T22:27:03.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.550+0000 7f3ba462e700 1 -- start start 2023-09-30T22:27:03.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.550+0000 7f3ba462e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b9c059070 0x7f3b9c117b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:03.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.550+0000 7f3ba462e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3b9c095370 0x7f3b9c113ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:03.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.550+0000 7f3ba462e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3b9c127390 0x7f3b9c114090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:03.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.550+0000 7f3ba462e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9c1254a0 con 0x7f3b9c059070 2023-09-30T22:27:03.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.550+0000 7f3ba462e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9c096f20 con 0x7f3b9c095370 2023-09-30T22:27:03.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.550+0000 7f3ba462e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9c114c50 con 0x7f3b9c127390 2023-09-30T22:27:03.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.551+0000 7f3ba23ca700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b9c059070 0x7f3b9c117b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:03.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.551+0000 7f3ba23ca700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b9c059070 0x7f3b9c117b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54622/0 (socket says 172.21.15.16:54622) 2023-09-30T22:27:03.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.551+0000 7f3ba23ca700 1 -- 172.21.15.16:0/3805489439 learned_addr learned my addr 172.21.15.16:0/3805489439 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:03.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.551+0000 7f3ba2bcb700 1 --2- 172.21.15.16:0/3805489439 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3b9c127390 0x7f3b9c114090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:03.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.551+0000 7f3ba23ca700 1 -- 172.21.15.16:0/3805489439 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3b9c127390 msgr2=0x7f3b9c114090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:03.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.551+0000 7f3ba23ca700 1 --2- 172.21.15.16:0/3805489439 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3b9c127390 0x7f3b9c114090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:03.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.551+0000 7f3ba23ca700 1 -- 172.21.15.16:0/3805489439 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3b9c095370 msgr2=0x7f3b9c113ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:03.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.551+0000 7f3ba23ca700 1 --2- 172.21.15.16:0/3805489439 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3b9c095370 0x7f3b9c113ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-09-30T22:27:03.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.551+0000 7f3ba23ca700 1 -- 172.21.15.16:0/3805489439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b98009370 con 0x7f3b9c059070 2023-09-30T22:27:03.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.551+0000 7f3ba23ca700 1 --2- 172.21.15.16:0/3805489439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b9c059070 0x7f3b9c117b50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f3b98009ed0 tx=0x7f3b98017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:03.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.551+0000 7f3b927fc700 1 -- 172.21.15.16:0/3805489439 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b980069b0 con 0x7f3b9c059070 2023-09-30T22:27:03.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.551+0000 7f3b927fc700 1 -- 172.21.15.16:0/3805489439 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b98000bc0 con 0x7f3b9c059070 2023-09-30T22:27:03.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.551+0000 7f3ba462e700 1 -- 172.21.15.16:0/3805489439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b9c117370 con 0x7f3b9c059070 2023-09-30T22:27:03.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.551+0000 7f3ba462e700 1 -- 172.21.15.16:0/3805489439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b9c1156d0 con 0x7f3b9c059070 2023-09-30T22:27:03.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.552+0000 7f3b927fc700 1 -- 172.21.15.16:0/3805489439 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9800ba80 con 0x7f3b9c059070 2023-09-30T22:27:03.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.552+0000 7f3b927fc700 1 -- 172.21.15.16:0/3805489439 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f3b9801c630 con 0x7f3b9c059070 2023-09-30T22:27:03.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.552+0000 7f3b927fc700 1 -- 172.21.15.16:0/3805489439 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f3b9803cce0 con 0x7f3b9c059070 2023-09-30T22:27:03.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.553+0000 7f3ba462e700 1 -- 172.21.15.16:0/3805489439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b9c099980 con 0x7f3b9c059070 2023-09-30T22:27:03.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.558+0000 7f3b927fc700 1 -- 172.21.15.16:0/3805489439 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f3b9802a020 con 0x7f3b9c059070 2023-09-30T22:27:03.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.660+0000 7fabc939f700 1 -- 172.21.15.16:0/1901683723 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fabc41137d0 con 0x7fabc4096380 2023-09-30T22:27:03.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.661+0000 7fabaaffd700 1 -- 172.21.15.16:0/1901683723 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fabc41137d0 con 0x7fabc4096380 2023-09-30T22:27:03.662 INFO:teuthology.orchestra.run.smithi016.stdout:38654705669 2023-09-30T22:27:03.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.663+0000 7fabc939f700 1 -- 172.21.15.16:0/1901683723 >> v1:172.21.15.16:6816/118285 conn(0x7fabb002bb90 legacy=0x7fabb002e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:03.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.663+0000 7fabc939f700 1 -- 172.21.15.16:0/1901683723 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fabc4096380 msgr2=0x7fabc40699e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:03.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.663+0000 7fabc939f700 1 --2- 172.21.15.16:0/1901683723 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fabc4096380 0x7fabc40699e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fabc4092dc0 tx=0x7fabbc009780).stop 2023-09-30T22:27:03.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.663+0000 7fabc939f700 1 -- 172.21.15.16:0/1901683723 shutdown_connections 2023-09-30T22:27:03.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.663+0000 7fabc939f700 1 --2- 172.21.15.16:0/1901683723 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fabc4096380 0x7fabc40699e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:03.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.663+0000 7fabc939f700 1 --2- 172.21.15.16:0/1901683723 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fabc4091b00 0x7fabc4069000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:03.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.663+0000 7fabc939f700 1 --2- 172.21.15.16:0/1901683723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabc4094c20 0x7fabc40694f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-09-30T22:27:03.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.663+0000 7fabc939f700 1 -- 172.21.15.16:0/1901683723 >> 172.21.15.16:0/1901683723 conn(0x7fabc4097470 msgr2=0x7fabc4128740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:03.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.663+0000 7fabc939f700 1 -- 172.21.15.16:0/1901683723 shutdown_connections 2023-09-30T22:27:03.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.663+0000 7fabc939f700 1 -- 172.21.15.16:0/1901683723 wait complete. 2023-09-30T22:27:03.683 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705669 for osd.1 2023-09-30T22:27:03.683 DEBUG:teuthology.parallel:result is None 2023-09-30T22:27:03.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.713+0000 7f1f68762700 1 -- 172.21.15.16:0/1064854638 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1f60116b50 con 0x7f1f60096380 2023-09-30T22:27:03.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.713+0000 7f1f567fc700 1 -- 172.21.15.16:0/1064854638 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1f60116b50 con 0x7f1f60096380 2023-09-30T22:27:03.715 INFO:teuthology.orchestra.run.smithi016.stdout:38654705669 2023-09-30T22:27:03.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.715+0000 7f1f68762700 1 -- 172.21.15.16:0/1064854638 >> v1:172.21.15.16:6816/118285 conn(0x7f1f4402bb60 legacy=0x7f1f4402e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:03.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.715+0000 7f1f68762700 1 -- 172.21.15.16:0/1064854638 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f60096380 msgr2=0x7f1f60112cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:03.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.715+0000 7f1f68762700 1 --2- 172.21.15.16:0/1064854638 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f60096380 0x7f1f60112cc0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f1f5800d180 tx=0x7f1f58009a50).stop 2023-09-30T22:27:03.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.715+0000 7f1f68762700 1 -- 172.21.15.16:0/1064854638 shutdown_connections 2023-09-30T22:27:03.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.715+0000 7f1f68762700 1 --2- 172.21.15.16:0/1064854638 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f60096380 0x7f1f60112cc0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:03.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.715+0000 7f1f68762700 1 --2- 172.21.15.16:0/1064854638 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f60091b00 0x7f1f601248f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:03.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.715+0000 7f1f68762700 1 --2- 172.21.15.16:0/1064854638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f60094c20 0x7f1f601127d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:03.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.715+0000 7f1f68762700 1 -- 172.21.15.16:0/1064854638 >> 172.21.15.16:0/1064854638 conn(0x7f1f60097470 msgr2=0x7f1f60126af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:03.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.716+0000 7f1f68762700 1 -- 172.21.15.16:0/1064854638 shutdown_connections 2023-09-30T22:27:03.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.716+0000 7f1f68762700 1 -- 172.21.15.16:0/1064854638 wait complete. 2023-09-30T22:27:03.730 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705669 for osd.0 2023-09-30T22:27:03.730 DEBUG:teuthology.parallel:result is None 2023-09-30T22:27:03.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.737+0000 7f3ba462e700 1 -- 172.21.15.16:0/3805489439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f3b9c099d80 con 0x7f3b9c059070 2023-09-30T22:27:03.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.737+0000 7f3b927fc700 1 -- 172.21.15.16:0/3805489439 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3b98034090 con 0x7f3b9c059070 2023-09-30T22:27:03.739 INFO:teuthology.orchestra.run.smithi016.stdout:38654705669 2023-09-30T22:27:03.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.739+0000 7f3ba462e700 1 -- 172.21.15.16:0/3805489439 >> v1:172.21.15.16:6816/118285 conn(0x7f3b8002bb60 legacy=0x7f3b8002e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:03.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.739+0000 7f3ba462e700 1 -- 172.21.15.16:0/3805489439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b9c059070 msgr2=0x7f3b9c117b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:03.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.739+0000 7f3ba462e700 1 --2- 172.21.15.16:0/3805489439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b9c059070 0x7f3b9c117b50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f3b98009ed0 tx=0x7f3b98017000).stop 2023-09-30T22:27:03.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.740+0000 7f3ba462e700 1 -- 172.21.15.16:0/3805489439 shutdown_connections 2023-09-30T22:27:03.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.740+0000 7f3ba462e700 1 --2- 172.21.15.16:0/3805489439 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3b9c127390 0x7f3b9c114090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:03.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.740+0000 7f3ba462e700 1 --2- 172.21.15.16:0/3805489439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b9c059070 0x7f3b9c117b50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:03.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.740+0000 7f3ba462e700 1 --2- 172.21.15.16:0/3805489439 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3b9c095370 0x7f3b9c113ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-09-30T22:27:03.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.740+0000 7f3ba462e700 1 -- 172.21.15.16:0/3805489439 >> 172.21.15.16:0/3805489439 conn(0x7f3b9c097470 msgr2=0x7f3b9c094390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:03.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.740+0000 7f3ba462e700 1 -- 172.21.15.16:0/3805489439 shutdown_connections 2023-09-30T22:27:03.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.740+0000 7f3ba462e700 1 -- 172.21.15.16:0/3805489439 wait complete. 2023-09-30T22:27:03.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.742+0000 7fd235a1b700 1 -- 172.21.15.16:0/3544509886 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd230096fa0 con 0x7fd230059070 2023-09-30T22:27:03.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.742+0000 7fd216ffd700 1 -- 172.21.15.16:0/3544509886 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd22001ab80 con 0x7fd230059070 2023-09-30T22:27:03.744 INFO:teuthology.orchestra.run.smithi016.stdout:38654705670 2023-09-30T22:27:03.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.744+0000 7fd235a1b700 1 -- 172.21.15.16:0/3544509886 >> v1:172.21.15.16:6816/118285 conn(0x7fd21c02bb60 legacy=0x7fd21c02e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:03.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.744+0000 7fd235a1b700 1 -- 172.21.15.16:0/3544509886 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd230059070 msgr2=0x7fd23012a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:03.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.744+0000 7fd235a1b700 1 --2- 172.21.15.16:0/3544509886 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd230059070 0x7fd23012a860 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fd22000c4c0 tx=0x7fd220007e00).stop 2023-09-30T22:27:03.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.745+0000 7fd235a1b700 1 -- 172.21.15.16:0/3544509886 shutdown_connections 2023-09-30T22:27:03.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.745+0000 7fd235a1b700 1 --2- 172.21.15.16:0/3544509886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd230125dd0 0x7fd23012dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:03.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.745+0000 7fd235a1b700 1 --2- 172.21.15.16:0/3544509886 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd230059070 0x7fd23012a860 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:03.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.745+0000 7fd235a1b700 1 --2- 172.21.15.16:0/3544509886 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd2301251c0 0x7fd23012dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-09-30T22:27:03.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.745+0000 7fd235a1b700 1 -- 172.21.15.16:0/3544509886 >> 172.21.15.16:0/3544509886 conn(0x7fd230097470 msgr2=0x7fd230099ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:03.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.745+0000 7fd235a1b700 1 -- 172.21.15.16:0/3544509886 shutdown_connections 2023-09-30T22:27:03.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.745+0000 7fd235a1b700 1 -- 172.21.15.16:0/3544509886 wait complete. 2023-09-30T22:27:03.759 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705669 for osd.3 2023-09-30T22:27:03.759 DEBUG:teuthology.parallel:result is None 2023-09-30T22:27:03.762 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705670 got 38654705670 for osd.2 2023-09-30T22:27:03.763 DEBUG:teuthology.parallel:result is None 2023-09-30T22:27:03.763 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-09-30T22:27:03.763 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-30T22:27:03.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.935+0000 7f53b17f7700 1 Processor -- start 2023-09-30T22:27:03.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.935+0000 7f53b17f7700 1 -- start start 2023-09-30T22:27:03.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.936+0000 7f53b17f7700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f53ac096820 0x7f53ac096c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:03.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.936+0000 7f53b17f7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f53ac097800 0x7f53ac130280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:03.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.936+0000 7f53b17f7700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53ac068df0 con 0x7f53ac067940 2023-09-30T22:27:03.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.936+0000 7f53b17f7700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53ac097c90 con 0x7f53ac096820 2023-09-30T22:27:03.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.936+0000 7f53b17f7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53ac06a4c0 con 0x7f53ac097800 2023-09-30T22:27:03.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.936+0000 7f53ab7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f53ac097800 0x7f53ac130280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:03.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.936+0000 7f53aaffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f53ac096820 0x7f53ac096c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:03.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.936+0000 7f53ab7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f53ac097800 0x7f53ac130280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:42360/0 (socket says 172.21.15.16:42360) 2023-09-30T22:27:03.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.936+0000 7f53aaffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f53ac096820 0x7f53ac096c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54638/0 (socket says 172.21.15.16:54638) 2023-09-30T22:27:03.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.936+0000 7f53aa7fc700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f53ac067940 0x7f53ac0970f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51508/0 (socket says 172.21.15.16:51508) 2023-09-30T22:27:03.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.936+0000 7f53ab7fe700 1 -- 172.21.15.16:0/70527645 learned_addr learned my addr 172.21.15.16:0/70527645 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:03.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.936+0000 7f53ab7fe700 1 -- 172.21.15.16:0/70527645 >> v2:172.21.15.16:3300/0 conn(0x7f53ac096820 msgr2=0x7f53ac096c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:03.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.936+0000 7f53ab7fe700 1 --2- 172.21.15.16:0/70527645 >> v2:172.21.15.16:3300/0 conn(0x7f53ac096820 0x7f53ac096c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:03.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.936+0000 7f53ab7fe700 1 -- 172.21.15.16:0/70527645 >> v1:172.21.15.16:6789/0 conn(0x7f53ac067940 legacy=0x7f53ac0970f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:03.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.936+0000 7f53ab7fe700 1 -- 172.21.15.16:0/70527645 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53ac098260 con 0x7f53ac097800 2023-09-30T22:27:03.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.936+0000 7f53a9ffb700 1 -- 172.21.15.16:0/70527645 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 266357230 0 0) 0x7f53ac068df0 con 0x7f53ac067940 2023-09-30T22:27:03.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.937+0000 7f53aaffd700 1 --2- 172.21.15.16:0/70527645 >> v2:172.21.15.16:3300/0 conn(0x7f53ac096820 0x7f53ac096c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:03.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.937+0000 7f53ab7fe700 1 --2- 172.21.15.16:0/70527645 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f53ac097800 0x7f53ac130280 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f53a4009f20 tx=0x7f53a4009840).ready entity=mon.1 client_cookie=c6d7499aa4e8b29c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:03.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.937+0000 7f53a9ffb700 1 -- 172.21.15.16:0/70527645 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a400e070 con 0x7f53ac097800 2023-09-30T22:27:03.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.937+0000 7f53a9ffb700 1 -- 172.21.15.16:0/70527645 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53a400b9d0 con 0x7f53ac097800 2023-09-30T22:27:03.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.937+0000 7f53a9ffb700 1 -- 172.21.15.16:0/70527645 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a400bd50 con 0x7f53ac097800 2023-09-30T22:27:03.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.937+0000 7f53b17f7700 1 -- 172.21.15.16:0/70527645 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f53ac097800 msgr2=0x7f53ac130280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:03.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.937+0000 7f53b17f7700 1 --2- 172.21.15.16:0/70527645 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f53ac097800 0x7f53ac130280 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f53a4009f20 tx=0x7f53a4009840).stop 2023-09-30T22:27:03.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.937+0000 7f53b17f7700 1 -- 172.21.15.16:0/70527645 shutdown_connections 2023-09-30T22:27:03.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.937+0000 7f53b17f7700 1 --2- 172.21.15.16:0/70527645 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f53ac097800 0x7f53ac130280 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:03.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.937+0000 7f53b17f7700 1 --2- 172.21.15.16:0/70527645 >> v2:172.21.15.16:3300/0 conn(0x7f53ac096820 0x7f53ac096c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:03.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.937+0000 7f53b17f7700 1 -- 172.21.15.16:0/70527645 >> 172.21.15.16:0/70527645 conn(0x7f53ac09ba50 msgr2=0x7f53ac09be60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:03.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.937+0000 7f53b17f7700 1 -- 172.21.15.16:0/70527645 shutdown_connections 2023-09-30T22:27:03.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.937+0000 7f53b17f7700 1 -- 172.21.15.16:0/70527645 wait complete. 2023-09-30T22:27:03.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.939+0000 7f53b17f7700 1 Processor -- start 2023-09-30T22:27:03.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.939+0000 7f53b17f7700 1 -- start start 2023-09-30T22:27:03.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.940+0000 7f53b17f7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53ac067940 0x7f53ac12ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:03.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.940+0000 7f53b17f7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f53ac096820 0x7f53ac117e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:03.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.940+0000 7f53b17f7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f53ac097800 0x7f53ac118380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:03.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.940+0000 7f53b17f7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ac119350 con 0x7f53ac067940 2023-09-30T22:27:03.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.940+0000 7f53b17f7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53ac11a4c0 con 0x7f53ac096820 2023-09-30T22:27:03.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.940+0000 7f53b17f7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f53ac067e60 con 0x7f53ac097800 2023-09-30T22:27:03.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.940+0000 7f53aaffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53ac067940 0x7f53ac12ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:03.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.940+0000 7f53aa7fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f53ac096820 0x7f53ac117e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:03.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.940+0000 7f53aaffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53ac067940 0x7f53ac12ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54652/0 (socket says 172.21.15.16:54652) 2023-09-30T22:27:03.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.940+0000 7f53aaffd700 1 -- 172.21.15.16:0/2265743328 learned_addr learned my addr 172.21.15.16:0/2265743328 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:03.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.940+0000 7f53ab7fe700 1 --2- 172.21.15.16:0/2265743328 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f53ac097800 0x7f53ac118380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:03.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.940+0000 7f53aaffd700 1 -- 172.21.15.16:0/2265743328 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f53ac097800 msgr2=0x7f53ac118380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:03.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.940+0000 7f53aaffd700 1 --2- 172.21.15.16:0/2265743328 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f53ac097800 0x7f53ac118380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:03.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.940+0000 7f53aaffd700 1 -- 172.21.15.16:0/2265743328 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f53ac096820 msgr2=0x7f53ac117e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:03.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.940+0000 7f53aaffd700 1 --2- 172.21.15.16:0/2265743328 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f53ac096820 0x7f53ac117e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:03.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.940+0000 7f53aaffd700 1 -- 172.21.15.16:0/2265743328 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f539c0092a0 con 0x7f53ac067940 2023-09-30T22:27:03.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.940+0000 7f53aa7fc700 1 --2- 172.21.15.16:0/2265743328 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f53ac096820 0x7f53ac117e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:03.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.940+0000 7f53aaffd700 1 --2- 172.21.15.16:0/2265743328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53ac067940 0x7f53ac12ffd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f539c00ea20 tx=0x7f539c009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:03.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.940+0000 7f53a2ffd700 1 -- 172.21.15.16:0/2265743328 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539c00ee60 con 0x7f53ac067940 2023-09-30T22:27:03.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.940+0000 7f53a2ffd700 1 -- 172.21.15.16:0/2265743328 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f539c00c850 con 0x7f53ac067940 2023-09-30T22:27:03.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.941+0000 7f53a2ffd700 1 -- 172.21.15.16:0/2265743328 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539c00cbd0 con 0x7f53ac067940 2023-09-30T22:27:03.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.941+0000 7f53b17f7700 1 -- 172.21.15.16:0/2265743328 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53a4009ae0 con 0x7f53ac067940 2023-09-30T22:27:03.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.941+0000 7f53b17f7700 1 -- 172.21.15.16:0/2265743328 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53ac1304c0 con 0x7f53ac067940 2023-09-30T22:27:03.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.941+0000 7f53a2ffd700 1 -- 172.21.15.16:0/2265743328 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f539c01b400 con 0x7f53ac067940 2023-09-30T22:27:03.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.942+0000 7f53a2ffd700 1 -- 172.21.15.16:0/2265743328 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f539c03b5f0 con 0x7f53ac067940 2023-09-30T22:27:03.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.942+0000 7f53b17f7700 1 -- 172.21.15.16:0/2265743328 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53ac05a260 con 0x7f53ac067940 2023-09-30T22:27:03.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:03.946+0000 7f53a2ffd700 1 -- 172.21.15.16:0/2265743328 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f539c019900 con 0x7f53ac067940 2023-09-30T22:27:04.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.120+0000 7f53b17f7700 1 -- 172.21.15.16:0/2265743328 --> v1:172.21.15.16:6816/118285 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f53ac118c90 con 0x7f538c02bb60 2023-09-30T22:27:04.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.126+0000 7f53a2ffd700 1 -- 172.21.15.16:0/2265743328 <== mgr.4098 v1:172.21.15.16:6816/118285 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+249919 (unknown 2965378022 0 2284131050) 0x7f53ac118c90 con 0x7f538c02bb60 2023-09-30T22:27:04.131 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:27:04.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.130+0000 7f53b17f7700 1 -- 172.21.15.16:0/2265743328 >> v1:172.21.15.16:6816/118285 conn(0x7f538c02bb60 legacy=0x7f538c02e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:04.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.130+0000 7f53b17f7700 1 -- 172.21.15.16:0/2265743328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53ac067940 msgr2=0x7f53ac12ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:04.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.130+0000 7f53b17f7700 1 --2- 172.21.15.16:0/2265743328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53ac067940 0x7f53ac12ffd0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f539c00ea20 tx=0x7f539c009060).stop 2023-09-30T22:27:04.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.130+0000 7f53b17f7700 1 -- 172.21.15.16:0/2265743328 shutdown_connections 2023-09-30T22:27:04.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.130+0000 7f53b17f7700 1 --2- 172.21.15.16:0/2265743328 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f53ac097800 0x7f53ac118380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:04.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.130+0000 7f53b17f7700 1 --2- 172.21.15.16:0/2265743328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53ac067940 0x7f53ac12ffd0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:04.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.130+0000 7f53b17f7700 1 --2- 172.21.15.16:0/2265743328 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f53ac096820 0x7f53ac117e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:04.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.130+0000 7f53b17f7700 1 -- 172.21.15.16:0/2265743328 >> 172.21.15.16:0/2265743328 conn(0x7f53ac09ba50 msgr2=0x7f53ac099880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:04.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.130+0000 7f53b17f7700 1 -- 172.21.15.16:0/2265743328 shutdown_connections 2023-09-30T22:27:04.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.130+0000 7f53b17f7700 1 -- 172.21.15.16:0/2265743328 wait complete. 2023-09-30T22:27:04.134 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-09-30T22:27:04.144 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-09-30T22:27:02.456931+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1188988,"kb_used_data":604,"kb_used_omap":0,"kb_used_meta":1188352,"kb_avail":376298372,"statfs":{"total":386547056640,"available":385329532928,"internally_reserved":0,"allocated":618496,"data_stored":181498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1216872448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001385"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.718509+0000","last_change":"2023-09-30T22:26:51.708966+0000","last_active":"2023-09-30T22:26:51.718509+0000","last_peered":"2023-09-30T22:26:51.718509+0000","last_clean":"2023-09-30T22:26:51.718509+0000","last_became_active":"2023-09-30T22:26:51.708653+0000","last_became_peered":"2023-09-30T22:26:51.708653+0000","last_unstale":"2023-09-30T22:26:51.718509+0000","last_undegraded":"2023-09-30T22:26:51.718509+0000","last_fullsized":"2023-09-30T22:26:51.718509+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.729225+0000","last_change":"2023-09-30T22:26:51.727077+0000","last_active":"2023-09-30T22:26:51.729225+0000","last_peered":"2023-09-30T22:26:51.729225+0000","last_clean":"2023-09-30T22:26:51.729225+0000","last_became_active":"2023-09-30T22:26:51.726778+0000","last_became_peered":"2023-09-30T22:26:51.726778+0000","last_unstale":"2023-09-30T22:26:51.729225+0000","last_undegraded":"2023-09-30T22:26:51.729225+0000","last_fullsized":"2023-09-30T22:26:51.729225+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.714151+0000","last_change":"2023-09-30T22:26:51.706919+0000","last_active":"2023-09-30T22:26:51.714151+0000","last_peered":"2023-09-30T22:26:51.714151+0000","last_clean":"2023-09-30T22:26:51.714151+0000","last_became_active":"2023-09-30T22:26:51.706676+0000","last_became_peered":"2023-09-30T22:26:51.706676+0000","last_unstale":"2023-09-30T22:26:51.714151+0000","last_undegraded":"2023-09-30T22:26:51.714151+0000","last_fullsized":"2023-09-30T22:26:51.714151+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.728676+0000","last_change":"2023-09-30T22:26:51.727386+0000","last_active":"2023-09-30T22:26:51.728676+0000","last_peered":"2023-09-30T22:26:51.728676+0000","last_clean":"2023-09-30T22:26:51.728676+0000","last_became_active":"2023-09-30T22:26:51.727091+0000","last_became_peered":"2023-09-30T22:26:51.727091+0000","last_unstale":"2023-09-30T22:26:51.728676+0000","last_undegraded":"2023-09-30T22:26:51.728676+0000","last_fullsized":"2023-09-30T22:26:51.728676+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.723905+0000","last_change":"2023-09-30T22:26:51.719993+0000","last_active":"2023-09-30T22:26:51.723905+0000","last_peered":"2023-09-30T22:26:51.723905+0000","last_clean":"2023-09-30T22:26:51.723905+0000","last_became_active":"2023-09-30T22:26:51.719495+0000","last_became_peered":"2023-09-30T22:26:51.719495+0000","last_unstale":"2023-09-30T22:26:51.723905+0000","last_undegraded":"2023-09-30T22:26:51.723905+0000","last_fullsized":"2023-09-30T22:26:51.723905+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.715438+0000","last_change":"2023-09-30T22:26:51.709581+0000","last_active":"2023-09-30T22:26:51.715438+0000","last_peered":"2023-09-30T22:26:51.715438+0000","last_clean":"2023-09-30T22:26:51.715438+0000","last_became_active":"2023-09-30T22:26:51.709309+0000","last_became_peered":"2023-09-30T22:26:51.709309+0000","last_unstale":"2023-09-30T22:26:51.715438+0000","last_undegraded":"2023-09-30T22:26:51.715438+0000","last_fullsized":"2023-09-30T22:26:51.715438+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.713991+0000","last_change":"2023-09-30T22:26:51.705826+0000","last_active":"2023-09-30T22:26:51.713991+0000","last_peered":"2023-09-30T22:26:51.713991+0000","last_clean":"2023-09-30T22:26:51.713991+0000","last_became_active":"2023-09-30T22:26:51.705037+0000","last_became_peered":"2023-09-30T22:26:51.705037+0000","last_unstale":"2023-09-30T22:26:51.713991+0000","last_undegraded":"2023-09-30T22:26:51.713991+0000","last_fullsized":"2023-09-30T22:26:51.713991+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730836+0000","last_change":"2023-09-30T22:26:51.726640+0000","last_active":"2023-09-30T22:26:51.730836+0000","last_peered":"2023-09-30T22:26:51.730836+0000","last_clean":"2023-09-30T22:26:51.730836+0000","last_became_active":"2023-09-30T22:26:51.726278+0000","last_became_peered":"2023-09-30T22:26:51.726278+0000","last_unstale":"2023-09-30T22:26:51.730836+0000","last_undegraded":"2023-09-30T22:26:51.730836+0000","last_fullsized":"2023-09-30T22:26:51.730836+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.724179+0000","last_change":"2023-09-30T22:26:51.721817+0000","last_active":"2023-09-30T22:26:51.724179+0000","last_peered":"2023-09-30T22:26:51.724179+0000","last_clean":"2023-09-30T22:26:51.724179+0000","last_became_active":"2023-09-30T22:26:51.721373+0000","last_became_peered":"2023-09-30T22:26:51.721373+0000","last_unstale":"2023-09-30T22:26:51.724179+0000","last_undegraded":"2023-09-30T22:26:51.724179+0000","last_fullsized":"2023-09-30T22:26:51.724179+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730810+0000","last_change":"2023-09-30T22:26:51.726348+0000","last_active":"2023-09-30T22:26:51.730810+0000","last_peered":"2023-09-30T22:26:51.730810+0000","last_clean":"2023-09-30T22:26:51.730810+0000","last_became_active":"2023-09-30T22:26:51.726030+0000","last_became_peered":"2023-09-30T22:26:51.726030+0000","last_unstale":"2023-09-30T22:26:51.730810+0000","last_undegraded":"2023-09-30T22:26:51.730810+0000","last_fullsized":"2023-09-30T22:26:51.730810+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730607+0000","last_change":"2023-09-30T22:26:51.722619+0000","last_active":"2023-09-30T22:26:51.730607+0000","last_peered":"2023-09-30T22:26:51.730607+0000","last_clean":"2023-09-30T22:26:51.730607+0000","last_became_active":"2023-09-30T22:26:51.722317+0000","last_became_peered":"2023-09-30T22:26:51.722317+0000","last_unstale":"2023-09-30T22:26:51.730607+0000","last_undegraded":"2023-09-30T22:26:51.730607+0000","last_fullsized":"2023-09-30T22:26:51.730607+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.718852+0000","last_change":"2023-09-30T22:26:51.709697+0000","last_active":"2023-09-30T22:26:51.718852+0000","last_peered":"2023-09-30T22:26:51.718852+0000","last_clean":"2023-09-30T22:26:51.718852+0000","last_became_active":"2023-09-30T22:26:51.709342+0000","last_became_peered":"2023-09-30T22:26:51.709342+0000","last_unstale":"2023-09-30T22:26:51.718852+0000","last_undegraded":"2023-09-30T22:26:51.718852+0000","last_fullsized":"2023-09-30T22:26:51.718852+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.719514+0000","last_change":"2023-09-30T22:26:51.716276+0000","last_active":"2023-09-30T22:26:51.719514+0000","last_peered":"2023-09-30T22:26:51.719514+0000","last_clean":"2023-09-30T22:26:51.719514+0000","last_became_active":"2023-09-30T22:26:51.715634+0000","last_became_peered":"2023-09-30T22:26:51.715634+0000","last_unstale":"2023-09-30T22:26:51.719514+0000","last_undegraded":"2023-09-30T22:26:51.719514+0000","last_fullsized":"2023-09-30T22:26:51.719514+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.707967+0000","last_change":"2023-09-30T22:26:51.695948+0000","last_active":"2023-09-30T22:26:51.707967+0000","last_peered":"2023-09-30T22:26:51.707967+0000","last_clean":"2023-09-30T22:26:51.707967+0000","last_became_active":"2023-09-30T22:26:51.695661+0000","last_became_peered":"2023-09-30T22:26:51.695661+0000","last_unstale":"2023-09-30T22:26:51.707967+0000","last_undegraded":"2023-09-30T22:26:51.707967+0000","last_fullsized":"2023-09-30T22:26:51.707967+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.716656+0000","last_change":"2023-09-30T22:26:51.708532+0000","last_active":"2023-09-30T22:26:51.716656+0000","last_peered":"2023-09-30T22:26:51.716656+0000","last_clean":"2023-09-30T22:26:51.716656+0000","last_became_active":"2023-09-30T22:26:51.708204+0000","last_became_peered":"2023-09-30T22:26:51.708204+0000","last_unstale":"2023-09-30T22:26:51.716656+0000","last_undegraded":"2023-09-30T22:26:51.716656+0000","last_fullsized":"2023-09-30T22:26:51.716656+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.714211+0000","last_change":"2023-09-30T22:26:51.704597+0000","last_active":"2023-09-30T22:26:51.714211+0000","last_peered":"2023-09-30T22:26:51.714211+0000","last_clean":"2023-09-30T22:26:51.714211+0000","last_became_active":"2023-09-30T22:26:51.704307+0000","last_became_peered":"2023-09-30T22:26:51.704307+0000","last_unstale":"2023-09-30T22:26:51.714211+0000","last_undegraded":"2023-09-30T22:26:51.714211+0000","last_fullsized":"2023-09-30T22:26:51.714211+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.724159+0000","last_change":"2023-09-30T22:26:51.721640+0000","last_active":"2023-09-30T22:26:51.724159+0000","last_peered":"2023-09-30T22:26:51.724159+0000","last_clean":"2023-09-30T22:26:51.724159+0000","last_became_active":"2023-09-30T22:26:51.720950+0000","last_became_peered":"2023-09-30T22:26:51.720950+0000","last_unstale":"2023-09-30T22:26:51.724159+0000","last_undegraded":"2023-09-30T22:26:51.724159+0000","last_fullsized":"2023-09-30T22:26:51.724159+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.707139+0000","last_change":"2023-09-30T22:26:51.697513+0000","last_active":"2023-09-30T22:26:51.707139+0000","last_peered":"2023-09-30T22:26:51.707139+0000","last_clean":"2023-09-30T22:26:51.707139+0000","last_became_active":"2023-09-30T22:26:51.697137+0000","last_became_peered":"2023-09-30T22:26:51.697137+0000","last_unstale":"2023-09-30T22:26:51.707139+0000","last_undegraded":"2023-09-30T22:26:51.707139+0000","last_fullsized":"2023-09-30T22:26:51.707139+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.718540+0000","last_change":"2023-09-30T22:26:51.715476+0000","last_active":"2023-09-30T22:26:51.718540+0000","last_peered":"2023-09-30T22:26:51.718540+0000","last_clean":"2023-09-30T22:26:51.718540+0000","last_became_active":"2023-09-30T22:26:51.715176+0000","last_became_peered":"2023-09-30T22:26:51.715176+0000","last_unstale":"2023-09-30T22:26:51.718540+0000","last_undegraded":"2023-09-30T22:26:51.718540+0000","last_fullsized":"2023-09-30T22:26:51.718540+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730693+0000","last_change":"2023-09-30T22:26:51.724991+0000","last_active":"2023-09-30T22:26:51.730693+0000","last_peered":"2023-09-30T22:26:51.730693+0000","last_clean":"2023-09-30T22:26:51.730693+0000","last_became_active":"2023-09-30T22:26:51.724240+0000","last_became_peered":"2023-09-30T22:26:51.724240+0000","last_unstale":"2023-09-30T22:26:51.730693+0000","last_undegraded":"2023-09-30T22:26:51.730693+0000","last_fullsized":"2023-09-30T22:26:51.730693+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730391+0000","last_change":"2023-09-30T22:26:51.717833+0000","last_active":"2023-09-30T22:26:51.730391+0000","last_peered":"2023-09-30T22:26:51.730391+0000","last_clean":"2023-09-30T22:26:51.730391+0000","last_became_active":"2023-09-30T22:26:51.717517+0000","last_became_peered":"2023-09-30T22:26:51.717517+0000","last_unstale":"2023-09-30T22:26:51.730391+0000","last_undegraded":"2023-09-30T22:26:51.730391+0000","last_fullsized":"2023-09-30T22:26:51.730391+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.723937+0000","last_change":"2023-09-30T22:26:51.719030+0000","last_active":"2023-09-30T22:26:51.723937+0000","last_peered":"2023-09-30T22:26:51.723937+0000","last_clean":"2023-09-30T22:26:51.723937+0000","last_became_active":"2023-09-30T22:26:51.715370+0000","last_became_peered":"2023-09-30T22:26:51.715370+0000","last_unstale":"2023-09-30T22:26:51.723937+0000","last_undegraded":"2023-09-30T22:26:51.723937+0000","last_fullsized":"2023-09-30T22:26:51.723937+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.713970+0000","last_change":"2023-09-30T22:26:51.698664+0000","last_active":"2023-09-30T22:26:51.713970+0000","last_peered":"2023-09-30T22:26:51.713970+0000","last_clean":"2023-09-30T22:26:51.713970+0000","last_became_active":"2023-09-30T22:26:51.698321+0000","last_became_peered":"2023-09-30T22:26:51.698321+0000","last_unstale":"2023-09-30T22:26:51.713970+0000","last_undegraded":"2023-09-30T22:26:51.713970+0000","last_fullsized":"2023-09-30T22:26:51.713970+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730972+0000","last_change":"2023-09-30T22:26:51.727955+0000","last_active":"2023-09-30T22:26:51.730972+0000","last_peered":"2023-09-30T22:26:51.730972+0000","last_clean":"2023-09-30T22:26:51.730972+0000","last_became_active":"2023-09-30T22:26:51.727209+0000","last_became_peered":"2023-09-30T22:26:51.727209+0000","last_unstale":"2023-09-30T22:26:51.730972+0000","last_undegraded":"2023-09-30T22:26:51.730972+0000","last_fullsized":"2023-09-30T22:26:51.730972+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730499+0000","last_change":"2023-09-30T22:26:51.721154+0000","last_active":"2023-09-30T22:26:51.730499+0000","last_peered":"2023-09-30T22:26:51.730499+0000","last_clean":"2023-09-30T22:26:51.730499+0000","last_became_active":"2023-09-30T22:26:51.720703+0000","last_became_peered":"2023-09-30T22:26:51.720703+0000","last_unstale":"2023-09-30T22:26:51.730499+0000","last_undegraded":"2023-09-30T22:26:51.730499+0000","last_fullsized":"2023-09-30T22:26:51.730499+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.725830+0000","last_change":"2023-09-30T22:26:51.701309+0000","last_active":"2023-09-30T22:26:51.725830+0000","last_peered":"2023-09-30T22:26:51.725830+0000","last_clean":"2023-09-30T22:26:51.725830+0000","last_became_active":"2023-09-30T22:26:51.701012+0000","last_became_peered":"2023-09-30T22:26:51.701012+0000","last_unstale":"2023-09-30T22:26:51.725830+0000","last_undegraded":"2023-09-30T22:26:51.725830+0000","last_fullsized":"2023-09-30T22:26:51.725830+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.718423+0000","last_change":"2023-09-30T22:26:51.715042+0000","last_active":"2023-09-30T22:26:51.718423+0000","last_peered":"2023-09-30T22:26:51.718423+0000","last_clean":"2023-09-30T22:26:51.718423+0000","last_became_active":"2023-09-30T22:26:51.712158+0000","last_became_peered":"2023-09-30T22:26:51.712158+0000","last_unstale":"2023-09-30T22:26:51.718423+0000","last_undegraded":"2023-09-30T22:26:51.718423+0000","last_fullsized":"2023-09-30T22:26:51.718423+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"15'1","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.713320+0000","last_change":"2023-09-30T22:26:51.296397+0000","last_active":"2023-09-30T22:26:51.713320+0000","last_peered":"2023-09-30T22:26:51.713320+0000","last_clean":"2023-09-30T22:26:51.713320+0000","last_became_active":"2023-09-30T22:26:51.295860+0000","last_became_peered":"2023-09-30T22:26:51.295860+0000","last_unstale":"2023-09-30T22:26:51.713320+0000","last_undegraded":"2023-09-30T22:26:51.713320+0000","last_fullsized":"2023-09-30T22:26:51.713320+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.716849+0000","last_change":"2023-09-30T22:26:51.712472+0000","last_active":"2023-09-30T22:26:51.716849+0000","last_peered":"2023-09-30T22:26:51.716849+0000","last_clean":"2023-09-30T22:26:51.716849+0000","last_became_active":"2023-09-30T22:26:51.711527+0000","last_became_peered":"2023-09-30T22:26:51.711527+0000","last_unstale":"2023-09-30T22:26:51.716849+0000","last_undegraded":"2023-09-30T22:26:51.716849+0000","last_fullsized":"2023-09-30T22:26:51.716849+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.701600+0000","last_change":"2023-09-30T22:26:51.298004+0000","last_active":"2023-09-30T22:26:51.701600+0000","last_peered":"2023-09-30T22:26:51.701600+0000","last_clean":"2023-09-30T22:26:51.701600+0000","last_became_active":"2023-09-30T22:26:51.297714+0000","last_became_peered":"2023-09-30T22:26:51.297714+0000","last_unstale":"2023-09-30T22:26:51.701600+0000","last_undegraded":"2023-09-30T22:26:51.701600+0000","last_fullsized":"2023-09-30T22:26:51.701600+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.729456+0000","last_change":"2023-09-30T22:26:51.727608+0000","last_active":"2023-09-30T22:26:51.729456+0000","last_peered":"2023-09-30T22:26:51.729456+0000","last_clean":"2023-09-30T22:26:51.729456+0000","last_became_active":"2023-09-30T22:26:51.727154+0000","last_became_peered":"2023-09-30T22:26:51.727154+0000","last_unstale":"2023-09-30T22:26:51.729456+0000","last_undegraded":"2023-09-30T22:26:51.729456+0000","last_fullsized":"2023-09-30T22:26:51.729456+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.684923+0000","last_change":"2023-09-30T22:26:51.285214+0000","last_active":"2023-09-30T22:26:51.684923+0000","last_peered":"2023-09-30T22:26:51.684923+0000","last_clean":"2023-09-30T22:26:51.684923+0000","last_became_active":"2023-09-30T22:26:51.284749+0000","last_became_peered":"2023-09-30T22:26:51.284749+0000","last_unstale":"2023-09-30T22:26:51.684923+0000","last_undegraded":"2023-09-30T22:26:51.684923+0000","last_fullsized":"2023-09-30T22:26:51.684923+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.716115+0000","last_change":"2023-09-30T22:26:51.706697+0000","last_active":"2023-09-30T22:26:51.716115+0000","last_peered":"2023-09-30T22:26:51.716115+0000","last_clean":"2023-09-30T22:26:51.716115+0000","last_became_active":"2023-09-30T22:26:51.706373+0000","last_became_peered":"2023-09-30T22:26:51.706373+0000","last_unstale":"2023-09-30T22:26:51.716115+0000","last_undegraded":"2023-09-30T22:26:51.716115+0000","last_fullsized":"2023-09-30T22:26:51.716115+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.689119+0000","last_change":"2023-09-30T22:26:51.281642+0000","last_active":"2023-09-30T22:26:51.689119+0000","last_peered":"2023-09-30T22:26:51.689119+0000","last_clean":"2023-09-30T22:26:51.689119+0000","last_became_active":"2023-09-30T22:26:51.281319+0000","last_became_peered":"2023-09-30T22:26:51.281319+0000","last_unstale":"2023-09-30T22:26:51.689119+0000","last_undegraded":"2023-09-30T22:26:51.689119+0000","last_fullsized":"2023-09-30T22:26:51.689119+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.714232+0000","last_change":"2023-09-30T22:26:51.705192+0000","last_active":"2023-09-30T22:26:51.714232+0000","last_peered":"2023-09-30T22:26:51.714232+0000","last_clean":"2023-09-30T22:26:51.714232+0000","last_became_active":"2023-09-30T22:26:51.704870+0000","last_became_peered":"2023-09-30T22:26:51.704870+0000","last_unstale":"2023-09-30T22:26:51.714232+0000","last_undegraded":"2023-09-30T22:26:51.714232+0000","last_fullsized":"2023-09-30T22:26:51.714232+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.680797+0000","last_change":"2023-09-30T22:26:51.293711+0000","last_active":"2023-09-30T22:26:51.680797+0000","last_peered":"2023-09-30T22:26:51.680797+0000","last_clean":"2023-09-30T22:26:51.680797+0000","last_became_active":"2023-09-30T22:26:51.293410+0000","last_became_peered":"2023-09-30T22:26:51.293410+0000","last_unstale":"2023-09-30T22:26:51.680797+0000","last_undegraded":"2023-09-30T22:26:51.680797+0000","last_fullsized":"2023-09-30T22:26:51.680797+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.714029+0000","last_change":"2023-09-30T22:26:51.706005+0000","last_active":"2023-09-30T22:26:51.714029+0000","last_peered":"2023-09-30T22:26:51.714029+0000","last_clean":"2023-09-30T22:26:51.714029+0000","last_became_active":"2023-09-30T22:26:51.705491+0000","last_became_peered":"2023-09-30T22:26:51.705491+0000","last_unstale":"2023-09-30T22:26:51.714029+0000","last_undegraded":"2023-09-30T22:26:51.714029+0000","last_fullsized":"2023-09-30T22:26:51.714029+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"15'3","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.719469+0000","last_change":"2023-09-30T22:26:51.279976+0000","last_active":"2023-09-30T22:26:51.719469+0000","last_peered":"2023-09-30T22:26:51.719469+0000","last_clean":"2023-09-30T22:26:51.719469+0000","last_became_active":"2023-09-30T22:26:51.279669+0000","last_became_peered":"2023-09-30T22:26:51.279669+0000","last_unstale":"2023-09-30T22:26:51.719469+0000","last_undegraded":"2023-09-30T22:26:51.719469+0000","last_fullsized":"2023-09-30T22:26:51.719469+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730714+0000","last_change":"2023-09-30T22:26:51.725947+0000","last_active":"2023-09-30T22:26:51.730714+0000","last_peered":"2023-09-30T22:26:51.730714+0000","last_clean":"2023-09-30T22:26:51.730714+0000","last_became_active":"2023-09-30T22:26:51.723294+0000","last_became_peered":"2023-09-30T22:26:51.723294+0000","last_unstale":"2023-09-30T22:26:51.730714+0000","last_undegraded":"2023-09-30T22:26:51.730714+0000","last_fullsized":"2023-09-30T22:26:51.730714+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"15'6","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.723849+0000","last_change":"2023-09-30T22:26:51.297527+0000","last_active":"2023-09-30T22:26:51.723849+0000","last_peered":"2023-09-30T22:26:51.723849+0000","last_clean":"2023-09-30T22:26:51.723849+0000","last_became_active":"2023-09-30T22:26:51.297246+0000","last_became_peered":"2023-09-30T22:26:51.297246+0000","last_unstale":"2023-09-30T22:26:51.723849+0000","last_undegraded":"2023-09-30T22:26:51.723849+0000","last_fullsized":"2023-09-30T22:26:51.723849+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.718810+0000","last_change":"2023-09-30T22:26:51.710781+0000","last_active":"2023-09-30T22:26:51.718810+0000","last_peered":"2023-09-30T22:26:51.718810+0000","last_clean":"2023-09-30T22:26:51.718810+0000","last_became_active":"2023-09-30T22:26:51.710107+0000","last_became_peered":"2023-09-30T22:26:51.710107+0000","last_unstale":"2023-09-30T22:26:51.718810+0000","last_undegraded":"2023-09-30T22:26:51.718810+0000","last_fullsized":"2023-09-30T22:26:51.718810+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"15'2","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.714029+0000","last_change":"2023-09-30T22:26:51.290194+0000","last_active":"2023-09-30T22:26:51.714029+0000","last_peered":"2023-09-30T22:26:51.714029+0000","last_clean":"2023-09-30T22:26:51.714029+0000","last_became_active":"2023-09-30T22:26:51.288688+0000","last_became_peered":"2023-09-30T22:26:51.288688+0000","last_unstale":"2023-09-30T22:26:51.714029+0000","last_undegraded":"2023-09-30T22:26:51.714029+0000","last_fullsized":"2023-09-30T22:26:51.714029+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.728819+0000","last_change":"2023-09-30T22:26:51.726780+0000","last_active":"2023-09-30T22:26:51.728819+0000","last_peered":"2023-09-30T22:26:51.728819+0000","last_clean":"2023-09-30T22:26:51.728819+0000","last_became_active":"2023-09-30T22:26:51.726408+0000","last_became_peered":"2023-09-30T22:26:51.726408+0000","last_unstale":"2023-09-30T22:26:51.728819+0000","last_undegraded":"2023-09-30T22:26:51.728819+0000","last_fullsized":"2023-09-30T22:26:51.728819+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.685836+0000","last_change":"2023-09-30T22:26:51.281127+0000","last_active":"2023-09-30T22:26:51.685836+0000","last_peered":"2023-09-30T22:26:51.685836+0000","last_clean":"2023-09-30T22:26:51.685836+0000","last_became_active":"2023-09-30T22:26:51.280507+0000","last_became_peered":"2023-09-30T22:26:51.280507+0000","last_unstale":"2023-09-30T22:26:51.685836+0000","last_undegraded":"2023-09-30T22:26:51.685836+0000","last_fullsized":"2023-09-30T22:26:51.685836+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.716876+0000","last_change":"2023-09-30T22:26:51.712671+0000","last_active":"2023-09-30T22:26:51.716876+0000","last_peered":"2023-09-30T22:26:51.716876+0000","last_clean":"2023-09-30T22:26:51.716876+0000","last_became_active":"2023-09-30T22:26:51.711836+0000","last_became_peered":"2023-09-30T22:26:51.711836+0000","last_unstale":"2023-09-30T22:26:51.716876+0000","last_undegraded":"2023-09-30T22:26:51.716876+0000","last_fullsized":"2023-09-30T22:26:51.716876+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.683587+0000","last_change":"2023-09-30T22:26:51.297467+0000","last_active":"2023-09-30T22:26:51.683587+0000","last_peered":"2023-09-30T22:26:51.683587+0000","last_clean":"2023-09-30T22:26:51.683587+0000","last_became_active":"2023-09-30T22:26:51.296950+0000","last_became_peered":"2023-09-30T22:26:51.296950+0000","last_unstale":"2023-09-30T22:26:51.683587+0000","last_undegraded":"2023-09-30T22:26:51.683587+0000","last_fullsized":"2023-09-30T22:26:51.683587+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.724064+0000","last_change":"2023-09-30T22:26:51.719411+0000","last_active":"2023-09-30T22:26:51.724064+0000","last_peered":"2023-09-30T22:26:51.724064+0000","last_clean":"2023-09-30T22:26:51.724064+0000","last_became_active":"2023-09-30T22:26:51.719111+0000","last_became_peered":"2023-09-30T22:26:51.719111+0000","last_unstale":"2023-09-30T22:26:51.724064+0000","last_undegraded":"2023-09-30T22:26:51.724064+0000","last_fullsized":"2023-09-30T22:26:51.724064+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"15'8","reported_seq":18,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.727696+0000","last_change":"2023-09-30T22:26:51.287702+0000","last_active":"2023-09-30T22:26:51.727696+0000","last_peered":"2023-09-30T22:26:51.727696+0000","last_clean":"2023-09-30T22:26:51.727696+0000","last_became_active":"2023-09-30T22:26:51.287434+0000","last_became_peered":"2023-09-30T22:26:51.287434+0000","last_unstale":"2023-09-30T22:26:51.727696+0000","last_undegraded":"2023-09-30T22:26:51.727696+0000","last_fullsized":"2023-09-30T22:26:51.727696+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.716138+0000","last_change":"2023-09-30T22:26:51.706663+0000","last_active":"2023-09-30T22:26:51.716138+0000","last_peered":"2023-09-30T22:26:51.716138+0000","last_clean":"2023-09-30T22:26:51.716138+0000","last_became_active":"2023-09-30T22:26:51.706363+0000","last_became_peered":"2023-09-30T22:26:51.706363+0000","last_unstale":"2023-09-30T22:26:51.716138+0000","last_undegraded":"2023-09-30T22:26:51.716138+0000","last_fullsized":"2023-09-30T22:26:51.716138+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"15'1","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.721973+0000","last_change":"2023-09-30T22:26:51.291996+0000","last_active":"2023-09-30T22:26:51.721973+0000","last_peered":"2023-09-30T22:26:51.721973+0000","last_clean":"2023-09-30T22:26:51.721973+0000","last_became_active":"2023-09-30T22:26:51.291656+0000","last_became_peered":"2023-09-30T22:26:51.291656+0000","last_unstale":"2023-09-30T22:26:51.721973+0000","last_undegraded":"2023-09-30T22:26:51.721973+0000","last_fullsized":"2023-09-30T22:26:51.721973+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.715557+0000","last_change":"2023-09-30T22:26:51.710326+0000","last_active":"2023-09-30T22:26:51.715557+0000","last_peered":"2023-09-30T22:26:51.715557+0000","last_clean":"2023-09-30T22:26:51.715557+0000","last_became_active":"2023-09-30T22:26:51.708793+0000","last_became_peered":"2023-09-30T22:26:51.708793+0000","last_unstale":"2023-09-30T22:26:51.715557+0000","last_undegraded":"2023-09-30T22:26:51.715557+0000","last_fullsized":"2023-09-30T22:26:51.715557+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"15'2","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.699110+0000","last_change":"2023-09-30T22:26:51.300700+0000","last_active":"2023-09-30T22:26:51.699110+0000","last_peered":"2023-09-30T22:26:51.699110+0000","last_clean":"2023-09-30T22:26:51.699110+0000","last_became_active":"2023-09-30T22:26:51.300416+0000","last_became_peered":"2023-09-30T22:26:51.300416+0000","last_unstale":"2023-09-30T22:26:51.699110+0000","last_undegraded":"2023-09-30T22:26:51.699110+0000","last_fullsized":"2023-09-30T22:26:51.699110+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730623+0000","last_change":"2023-09-30T22:26:51.722902+0000","last_active":"2023-09-30T22:26:51.730623+0000","last_peered":"2023-09-30T22:26:51.730623+0000","last_clean":"2023-09-30T22:26:51.730623+0000","last_became_active":"2023-09-30T22:26:51.717919+0000","last_became_peered":"2023-09-30T22:26:51.717919+0000","last_unstale":"2023-09-30T22:26:51.730623+0000","last_undegraded":"2023-09-30T22:26:51.730623+0000","last_fullsized":"2023-09-30T22:26:51.730623+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"15'1","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.716626+0000","last_change":"2023-09-30T22:26:51.292120+0000","last_active":"2023-09-30T22:26:51.716626+0000","last_peered":"2023-09-30T22:26:51.716626+0000","last_clean":"2023-09-30T22:26:51.716626+0000","last_became_active":"2023-09-30T22:26:51.289469+0000","last_became_peered":"2023-09-30T22:26:51.289469+0000","last_unstale":"2023-09-30T22:26:51.716626+0000","last_undegraded":"2023-09-30T22:26:51.716626+0000","last_fullsized":"2023-09-30T22:26:51.716626+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.718700+0000","last_change":"2023-09-30T22:26:51.711649+0000","last_active":"2023-09-30T22:26:51.718700+0000","last_peered":"2023-09-30T22:26:51.718700+0000","last_clean":"2023-09-30T22:26:51.718700+0000","last_became_active":"2023-09-30T22:26:51.711384+0000","last_became_peered":"2023-09-30T22:26:51.711384+0000","last_unstale":"2023-09-30T22:26:51.718700+0000","last_undegraded":"2023-09-30T22:26:51.718700+0000","last_fullsized":"2023-09-30T22:26:51.718700+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.686889+0000","last_change":"2023-09-30T22:26:51.281627+0000","last_active":"2023-09-30T22:26:51.686889+0000","last_peered":"2023-09-30T22:26:51.686889+0000","last_clean":"2023-09-30T22:26:51.686889+0000","last_became_active":"2023-09-30T22:26:51.281311+0000","last_became_peered":"2023-09-30T22:26:51.281311+0000","last_unstale":"2023-09-30T22:26:51.686889+0000","last_undegraded":"2023-09-30T22:26:51.686889+0000","last_fullsized":"2023-09-30T22:26:51.686889+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.707659+0000","last_change":"2023-09-30T22:26:51.700837+0000","last_active":"2023-09-30T22:26:51.707659+0000","last_peered":"2023-09-30T22:26:51.707659+0000","last_clean":"2023-09-30T22:26:51.707659+0000","last_became_active":"2023-09-30T22:26:51.700523+0000","last_became_peered":"2023-09-30T22:26:51.700523+0000","last_unstale":"2023-09-30T22:26:51.707659+0000","last_undegraded":"2023-09-30T22:26:51.707659+0000","last_fullsized":"2023-09-30T22:26:51.707659+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"15'2","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.721892+0000","last_change":"2023-09-30T22:26:51.290248+0000","last_active":"2023-09-30T22:26:51.721892+0000","last_peered":"2023-09-30T22:26:51.721892+0000","last_clean":"2023-09-30T22:26:51.721892+0000","last_became_active":"2023-09-30T22:26:51.289168+0000","last_became_peered":"2023-09-30T22:26:51.289168+0000","last_unstale":"2023-09-30T22:26:51.721892+0000","last_undegraded":"2023-09-30T22:26:51.721892+0000","last_fullsized":"2023-09-30T22:26:51.721892+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.723965+0000","last_change":"2023-09-30T22:26:51.719224+0000","last_active":"2023-09-30T22:26:51.723965+0000","last_peered":"2023-09-30T22:26:51.723965+0000","last_clean":"2023-09-30T22:26:51.723965+0000","last_became_active":"2023-09-30T22:26:51.718890+0000","last_became_peered":"2023-09-30T22:26:51.718890+0000","last_unstale":"2023-09-30T22:26:51.723965+0000","last_undegraded":"2023-09-30T22:26:51.723965+0000","last_fullsized":"2023-09-30T22:26:51.723965+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.687008+0000","last_change":"2023-09-30T22:26:51.288204+0000","last_active":"2023-09-30T22:26:51.687008+0000","last_peered":"2023-09-30T22:26:51.687008+0000","last_clean":"2023-09-30T22:26:51.687008+0000","last_became_active":"2023-09-30T22:26:51.287903+0000","last_became_peered":"2023-09-30T22:26:51.287903+0000","last_unstale":"2023-09-30T22:26:51.687008+0000","last_undegraded":"2023-09-30T22:26:51.687008+0000","last_fullsized":"2023-09-30T22:26:51.687008+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.718399+0000","last_change":"2023-09-30T22:26:51.713792+0000","last_active":"2023-09-30T22:26:51.718399+0000","last_peered":"2023-09-30T22:26:51.718399+0000","last_clean":"2023-09-30T22:26:51.718399+0000","last_became_active":"2023-09-30T22:26:51.713334+0000","last_became_peered":"2023-09-30T22:26:51.713334+0000","last_unstale":"2023-09-30T22:26:51.718399+0000","last_undegraded":"2023-09-30T22:26:51.718399+0000","last_fullsized":"2023-09-30T22:26:51.718399+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.683106+0000","last_change":"2023-09-30T22:26:51.297240+0000","last_active":"2023-09-30T22:26:51.683106+0000","last_peered":"2023-09-30T22:26:51.683106+0000","last_clean":"2023-09-30T22:26:51.683106+0000","last_became_active":"2023-09-30T22:26:51.296547+0000","last_became_peered":"2023-09-30T22:26:51.296547+0000","last_unstale":"2023-09-30T22:26:51.683106+0000","last_undegraded":"2023-09-30T22:26:51.683106+0000","last_fullsized":"2023-09-30T22:26:51.683106+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.718663+0000","last_change":"2023-09-30T22:26:51.711889+0000","last_active":"2023-09-30T22:26:51.718663+0000","last_peered":"2023-09-30T22:26:51.718663+0000","last_clean":"2023-09-30T22:26:51.718663+0000","last_became_active":"2023-09-30T22:26:51.711570+0000","last_became_peered":"2023-09-30T22:26:51.711570+0000","last_unstale":"2023-09-30T22:26:51.718663+0000","last_undegraded":"2023-09-30T22:26:51.718663+0000","last_fullsized":"2023-09-30T22:26:51.718663+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.684383+0000","last_change":"2023-09-30T22:26:51.290699+0000","last_active":"2023-09-30T22:26:51.684383+0000","last_peered":"2023-09-30T22:26:51.684383+0000","last_clean":"2023-09-30T22:26:51.684383+0000","last_became_active":"2023-09-30T22:26:51.290396+0000","last_became_peered":"2023-09-30T22:26:51.290396+0000","last_unstale":"2023-09-30T22:26:51.684383+0000","last_undegraded":"2023-09-30T22:26:51.684383+0000","last_fullsized":"2023-09-30T22:26:51.684383+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730474+0000","last_change":"2023-09-30T22:26:51.720976+0000","last_active":"2023-09-30T22:26:51.730474+0000","last_peered":"2023-09-30T22:26:51.730474+0000","last_clean":"2023-09-30T22:26:51.730474+0000","last_became_active":"2023-09-30T22:26:51.720378+0000","last_became_peered":"2023-09-30T22:26:51.720378+0000","last_unstale":"2023-09-30T22:26:51.730474+0000","last_undegraded":"2023-09-30T22:26:51.730474+0000","last_fullsized":"2023-09-30T22:26:51.730474+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.702412+0000","last_change":"2023-09-30T22:26:46.273891+0000","last_active":"2023-09-30T22:26:51.702412+0000","last_peered":"2023-09-30T22:26:51.702412+0000","last_clean":"2023-09-30T22:26:51.702412+0000","last_became_active":"2023-09-30T22:26:46.273588+0000","last_became_peered":"2023-09-30T22:26:46.273588+0000","last_unstale":"2023-09-30T22:26:51.702412+0000","last_undegraded":"2023-09-30T22:26:51.702412+0000","last_fullsized":"2023-09-30T22:26:51.702412+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.680672+0000","last_change":"2023-09-30T22:26:51.294682+0000","last_active":"2023-09-30T22:26:51.680672+0000","last_peered":"2023-09-30T22:26:51.680672+0000","last_clean":"2023-09-30T22:26:51.680672+0000","last_became_active":"2023-09-30T22:26:51.294422+0000","last_became_peered":"2023-09-30T22:26:51.294422+0000","last_unstale":"2023-09-30T22:26:51.680672+0000","last_undegraded":"2023-09-30T22:26:51.680672+0000","last_fullsized":"2023-09-30T22:26:51.680672+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730874+0000","last_change":"2023-09-30T22:26:51.726954+0000","last_active":"2023-09-30T22:26:51.730874+0000","last_peered":"2023-09-30T22:26:51.730874+0000","last_clean":"2023-09-30T22:26:51.730874+0000","last_became_active":"2023-09-30T22:26:51.726641+0000","last_became_peered":"2023-09-30T22:26:51.726641+0000","last_unstale":"2023-09-30T22:26:51.730874+0000","last_undegraded":"2023-09-30T22:26:51.730874+0000","last_fullsized":"2023-09-30T22:26:51.730874+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.687452+0000","last_change":"2023-09-30T22:26:46.245946+0000","last_active":"2023-09-30T22:26:51.687452+0000","last_peered":"2023-09-30T22:26:51.687452+0000","last_clean":"2023-09-30T22:26:51.687452+0000","last_became_active":"2023-09-30T22:26:46.245602+0000","last_became_peered":"2023-09-30T22:26:46.245602+0000","last_unstale":"2023-09-30T22:26:51.687452+0000","last_undegraded":"2023-09-30T22:26:51.687452+0000","last_fullsized":"2023-09-30T22:26:51.687452+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.685668+0000","last_change":"2023-09-30T22:26:51.294691+0000","last_active":"2023-09-30T22:26:51.685668+0000","last_peered":"2023-09-30T22:26:51.685668+0000","last_clean":"2023-09-30T22:26:51.685668+0000","last_became_active":"2023-09-30T22:26:51.294411+0000","last_became_peered":"2023-09-30T22:26:51.294411+0000","last_unstale":"2023-09-30T22:26:51.685668+0000","last_undegraded":"2023-09-30T22:26:51.685668+0000","last_fullsized":"2023-09-30T22:26:51.685668+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730680+0000","last_change":"2023-09-30T22:26:51.724759+0000","last_active":"2023-09-30T22:26:51.730680+0000","last_peered":"2023-09-30T22:26:51.730680+0000","last_clean":"2023-09-30T22:26:51.730680+0000","last_became_active":"2023-09-30T22:26:51.723221+0000","last_became_peered":"2023-09-30T22:26:51.723221+0000","last_unstale":"2023-09-30T22:26:51.730680+0000","last_undegraded":"2023-09-30T22:26:51.730680+0000","last_fullsized":"2023-09-30T22:26:51.730680+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.681986+0000","last_change":"2023-09-30T22:26:46.242607+0000","last_active":"2023-09-30T22:26:51.681986+0000","last_peered":"2023-09-30T22:26:51.681986+0000","last_clean":"2023-09-30T22:26:51.681986+0000","last_became_active":"2023-09-30T22:26:46.242283+0000","last_became_peered":"2023-09-30T22:26:46.242283+0000","last_unstale":"2023-09-30T22:26:51.681986+0000","last_undegraded":"2023-09-30T22:26:51.681986+0000","last_fullsized":"2023-09-30T22:26:51.681986+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.703666+0000","last_change":"2023-09-30T22:26:51.295409+0000","last_active":"2023-09-30T22:26:51.703666+0000","last_peered":"2023-09-30T22:26:51.703666+0000","last_clean":"2023-09-30T22:26:51.703666+0000","last_became_active":"2023-09-30T22:26:51.295140+0000","last_became_peered":"2023-09-30T22:26:51.295140+0000","last_unstale":"2023-09-30T22:26:51.703666+0000","last_undegraded":"2023-09-30T22:26:51.703666+0000","last_fullsized":"2023-09-30T22:26:51.703666+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730344+0000","last_change":"2023-09-30T22:26:51.715504+0000","last_active":"2023-09-30T22:26:51.730344+0000","last_peered":"2023-09-30T22:26:51.730344+0000","last_clean":"2023-09-30T22:26:51.730344+0000","last_became_active":"2023-09-30T22:26:51.715258+0000","last_became_peered":"2023-09-30T22:26:51.715258+0000","last_unstale":"2023-09-30T22:26:51.730344+0000","last_undegraded":"2023-09-30T22:26:51.730344+0000","last_fullsized":"2023-09-30T22:26:51.730344+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.682511+0000","last_change":"2023-09-30T22:26:46.242752+0000","last_active":"2023-09-30T22:26:51.682511+0000","last_peered":"2023-09-30T22:26:51.682511+0000","last_clean":"2023-09-30T22:26:51.682511+0000","last_became_active":"2023-09-30T22:26:46.242385+0000","last_became_peered":"2023-09-30T22:26:46.242385+0000","last_unstale":"2023-09-30T22:26:51.682511+0000","last_undegraded":"2023-09-30T22:26:51.682511+0000","last_fullsized":"2023-09-30T22:26:51.682511+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.703565+0000","last_change":"2023-09-30T22:26:46.273930+0000","last_active":"2023-09-30T22:26:51.703565+0000","last_peered":"2023-09-30T22:26:51.703565+0000","last_clean":"2023-09-30T22:26:51.703565+0000","last_became_active":"2023-09-30T22:26:46.273621+0000","last_became_peered":"2023-09-30T22:26:46.273621+0000","last_unstale":"2023-09-30T22:26:51.703565+0000","last_undegraded":"2023-09-30T22:26:51.703565+0000","last_fullsized":"2023-09-30T22:26:51.703565+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.686892+0000","last_change":"2023-09-30T22:26:51.298163+0000","last_active":"2023-09-30T22:26:51.686892+0000","last_peered":"2023-09-30T22:26:51.686892+0000","last_clean":"2023-09-30T22:26:51.686892+0000","last_became_active":"2023-09-30T22:26:51.297897+0000","last_became_peered":"2023-09-30T22:26:51.297897+0000","last_unstale":"2023-09-30T22:26:51.686892+0000","last_undegraded":"2023-09-30T22:26:51.686892+0000","last_fullsized":"2023-09-30T22:26:51.686892+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.718377+0000","last_change":"2023-09-30T22:26:51.713608+0000","last_active":"2023-09-30T22:26:51.718377+0000","last_peered":"2023-09-30T22:26:51.718377+0000","last_clean":"2023-09-30T22:26:51.718377+0000","last_became_active":"2023-09-30T22:26:51.712992+0000","last_became_peered":"2023-09-30T22:26:51.712992+0000","last_unstale":"2023-09-30T22:26:51.718377+0000","last_undegraded":"2023-09-30T22:26:51.718377+0000","last_fullsized":"2023-09-30T22:26:51.718377+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.684824+0000","last_change":"2023-09-30T22:26:46.245668+0000","last_active":"2023-09-30T22:26:51.684824+0000","last_peered":"2023-09-30T22:26:51.684824+0000","last_clean":"2023-09-30T22:26:51.684824+0000","last_became_active":"2023-09-30T22:26:46.245319+0000","last_became_peered":"2023-09-30T22:26:46.245319+0000","last_unstale":"2023-09-30T22:26:51.684824+0000","last_undegraded":"2023-09-30T22:26:51.684824+0000","last_fullsized":"2023-09-30T22:26:51.684824+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.681181+0000","last_change":"2023-09-30T22:26:51.294664+0000","last_active":"2023-09-30T22:26:51.681181+0000","last_peered":"2023-09-30T22:26:51.681181+0000","last_clean":"2023-09-30T22:26:51.681181+0000","last_became_active":"2023-09-30T22:26:51.294357+0000","last_became_peered":"2023-09-30T22:26:51.294357+0000","last_unstale":"2023-09-30T22:26:51.681181+0000","last_undegraded":"2023-09-30T22:26:51.681181+0000","last_fullsized":"2023-09-30T22:26:51.681181+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.707651+0000","last_change":"2023-09-30T22:26:51.705784+0000","last_active":"2023-09-30T22:26:51.707651+0000","last_peered":"2023-09-30T22:26:51.707651+0000","last_clean":"2023-09-30T22:26:51.707651+0000","last_became_active":"2023-09-30T22:26:51.705480+0000","last_became_peered":"2023-09-30T22:26:51.705480+0000","last_unstale":"2023-09-30T22:26:51.707651+0000","last_undegraded":"2023-09-30T22:26:51.707651+0000","last_fullsized":"2023-09-30T22:26:51.707651+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.689511+0000","last_change":"2023-09-30T22:26:46.274062+0000","last_active":"2023-09-30T22:26:51.689511+0000","last_peered":"2023-09-30T22:26:51.689511+0000","last_clean":"2023-09-30T22:26:51.689511+0000","last_became_active":"2023-09-30T22:26:46.273752+0000","last_became_peered":"2023-09-30T22:26:46.273752+0000","last_unstale":"2023-09-30T22:26:51.689511+0000","last_undegraded":"2023-09-30T22:26:51.689511+0000","last_fullsized":"2023-09-30T22:26:51.689511+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.702185+0000","last_change":"2023-09-30T22:26:51.295181+0000","last_active":"2023-09-30T22:26:51.702185+0000","last_peered":"2023-09-30T22:26:51.702185+0000","last_clean":"2023-09-30T22:26:51.702185+0000","last_became_active":"2023-09-30T22:26:51.294889+0000","last_became_peered":"2023-09-30T22:26:51.294889+0000","last_unstale":"2023-09-30T22:26:51.702185+0000","last_undegraded":"2023-09-30T22:26:51.702185+0000","last_fullsized":"2023-09-30T22:26:51.702185+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.698806+0000","last_change":"2023-09-30T22:26:51.690467+0000","last_active":"2023-09-30T22:26:51.698806+0000","last_peered":"2023-09-30T22:26:51.698806+0000","last_clean":"2023-09-30T22:26:51.698806+0000","last_became_active":"2023-09-30T22:26:51.690163+0000","last_became_peered":"2023-09-30T22:26:51.690163+0000","last_unstale":"2023-09-30T22:26:51.698806+0000","last_undegraded":"2023-09-30T22:26:51.698806+0000","last_fullsized":"2023-09-30T22:26:51.698806+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"15'2","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.716522+0000","last_change":"2023-09-30T22:26:51.290268+0000","last_active":"2023-09-30T22:26:51.716522+0000","last_peered":"2023-09-30T22:26:51.716522+0000","last_clean":"2023-09-30T22:26:51.716522+0000","last_became_active":"2023-09-30T22:26:51.289409+0000","last_became_peered":"2023-09-30T22:26:51.289409+0000","last_unstale":"2023-09-30T22:26:51.716522+0000","last_undegraded":"2023-09-30T22:26:51.716522+0000","last_fullsized":"2023-09-30T22:26:51.716522+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.712603+0000","last_change":"2023-09-30T22:26:46.273888+0000","last_active":"2023-09-30T22:26:51.712603+0000","last_peered":"2023-09-30T22:26:51.712603+0000","last_clean":"2023-09-30T22:26:51.712603+0000","last_became_active":"2023-09-30T22:26:46.273549+0000","last_became_peered":"2023-09-30T22:26:46.273549+0000","last_unstale":"2023-09-30T22:26:51.712603+0000","last_undegraded":"2023-09-30T22:26:51.712603+0000","last_fullsized":"2023-09-30T22:26:51.712603+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.714861+0000","last_change":"2023-09-30T22:26:51.709356+0000","last_active":"2023-09-30T22:26:51.714861+0000","last_peered":"2023-09-30T22:26:51.714861+0000","last_clean":"2023-09-30T22:26:51.714861+0000","last_became_active":"2023-09-30T22:26:51.709059+0000","last_became_peered":"2023-09-30T22:26:51.709059+0000","last_unstale":"2023-09-30T22:26:51.714861+0000","last_undegraded":"2023-09-30T22:26:51.714861+0000","last_fullsized":"2023-09-30T22:26:51.714861+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.682520+0000","last_change":"2023-09-30T22:26:46.274508+0000","last_active":"2023-09-30T22:26:51.682520+0000","last_peered":"2023-09-30T22:26:51.682520+0000","last_clean":"2023-09-30T22:26:51.682520+0000","last_became_active":"2023-09-30T22:26:46.273567+0000","last_became_peered":"2023-09-30T22:26:46.273567+0000","last_unstale":"2023-09-30T22:26:51.682520+0000","last_undegraded":"2023-09-30T22:26:51.682520+0000","last_fullsized":"2023-09-30T22:26:51.682520+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:26:52.891144+0000","last_change":"2023-09-30T22:26:51.288625+0000","last_active":"2023-09-30T22:26:52.891144+0000","last_peered":"2023-09-30T22:26:52.891144+0000","last_clean":"2023-09-30T22:26:52.891144+0000","last_became_active":"2023-09-30T22:26:51.288286+0000","last_became_peered":"2023-09-30T22:26:51.288286+0000","last_unstale":"2023-09-30T22:26:52.891144+0000","last_undegraded":"2023-09-30T22:26:52.891144+0000","last_fullsized":"2023-09-30T22:26:52.891144+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.718567+0000","last_change":"2023-09-30T22:26:51.708965+0000","last_active":"2023-09-30T22:26:51.718567+0000","last_peered":"2023-09-30T22:26:51.718567+0000","last_clean":"2023-09-30T22:26:51.718567+0000","last_became_active":"2023-09-30T22:26:51.708607+0000","last_became_peered":"2023-09-30T22:26:51.708607+0000","last_unstale":"2023-09-30T22:26:51.718567+0000","last_undegraded":"2023-09-30T22:26:51.718567+0000","last_fullsized":"2023-09-30T22:26:51.718567+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"15'1","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.719553+0000","last_change":"2023-09-30T22:26:51.282125+0000","last_active":"2023-09-30T22:26:51.719553+0000","last_peered":"2023-09-30T22:26:51.719553+0000","last_clean":"2023-09-30T22:26:51.719553+0000","last_became_active":"2023-09-30T22:26:51.281780+0000","last_became_peered":"2023-09-30T22:26:51.281780+0000","last_unstale":"2023-09-30T22:26:51.719553+0000","last_undegraded":"2023-09-30T22:26:51.719553+0000","last_fullsized":"2023-09-30T22:26:51.719553+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.709532+0000","last_change":"2023-09-30T22:26:51.703281+0000","last_active":"2023-09-30T22:26:51.709532+0000","last_peered":"2023-09-30T22:26:51.709532+0000","last_clean":"2023-09-30T22:26:51.709532+0000","last_became_active":"2023-09-30T22:26:51.703007+0000","last_became_peered":"2023-09-30T22:26:51.703007+0000","last_unstale":"2023-09-30T22:26:51.709532+0000","last_undegraded":"2023-09-30T22:26:51.709532+0000","last_fullsized":"2023-09-30T22:26:51.709532+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"15'2","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.726495+0000","last_change":"2023-09-30T22:26:51.287630+0000","last_active":"2023-09-30T22:26:51.726495+0000","last_peered":"2023-09-30T22:26:51.726495+0000","last_clean":"2023-09-30T22:26:51.726495+0000","last_became_active":"2023-09-30T22:26:51.287337+0000","last_became_peered":"2023-09-30T22:26:51.287337+0000","last_unstale":"2023-09-30T22:26:51.726495+0000","last_undegraded":"2023-09-30T22:26:51.726495+0000","last_fullsized":"2023-09-30T22:26:51.726495+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.719532+0000","last_change":"2023-09-30T22:26:51.716458+0000","last_active":"2023-09-30T22:26:51.719532+0000","last_peered":"2023-09-30T22:26:51.719532+0000","last_clean":"2023-09-30T22:26:51.719532+0000","last_became_active":"2023-09-30T22:26:51.715957+0000","last_became_peered":"2023-09-30T22:26:51.715957+0000","last_unstale":"2023-09-30T22:26:51.719532+0000","last_undegraded":"2023-09-30T22:26:51.719532+0000","last_fullsized":"2023-09-30T22:26:51.719532+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"15'2","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.716732+0000","last_change":"2023-09-30T22:26:51.285956+0000","last_active":"2023-09-30T22:26:51.716732+0000","last_peered":"2023-09-30T22:26:51.716732+0000","last_clean":"2023-09-30T22:26:51.716732+0000","last_became_active":"2023-09-30T22:26:51.285690+0000","last_became_peered":"2023-09-30T22:26:51.285690+0000","last_unstale":"2023-09-30T22:26:51.716732+0000","last_undegraded":"2023-09-30T22:26:51.716732+0000","last_fullsized":"2023-09-30T22:26:51.716732+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.719485+0000","last_change":"2023-09-30T22:26:51.716290+0000","last_active":"2023-09-30T22:26:51.719485+0000","last_peered":"2023-09-30T22:26:51.719485+0000","last_clean":"2023-09-30T22:26:51.719485+0000","last_became_active":"2023-09-30T22:26:51.716007+0000","last_became_peered":"2023-09-30T22:26:51.716007+0000","last_unstale":"2023-09-30T22:26:51.719485+0000","last_undegraded":"2023-09-30T22:26:51.719485+0000","last_fullsized":"2023-09-30T22:26:51.719485+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1e","version":"15'1","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.719629+0000","last_change":"2023-09-30T22:26:51.296949+0000","last_active":"2023-09-30T22:26:51.719629+0000","last_peered":"2023-09-30T22:26:51.719629+0000","last_clean":"2023-09-30T22:26:51.719629+0000","last_became_active":"2023-09-30T22:26:51.296602+0000","last_became_peered":"2023-09-30T22:26:51.296602+0000","last_unstale":"2023-09-30T22:26:51.719629+0000","last_undegraded":"2023-09-30T22:26:51.719629+0000","last_fullsized":"2023-09-30T22:26:51.719629+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.714046+0000","last_change":"2023-09-30T22:26:51.706034+0000","last_active":"2023-09-30T22:26:51.714046+0000","last_peered":"2023-09-30T22:26:51.714046+0000","last_clean":"2023-09-30T22:26:51.714046+0000","last_became_active":"2023-09-30T22:26:51.705633+0000","last_became_peered":"2023-09-30T22:26:51.705633+0000","last_unstale":"2023-09-30T22:26:51.714046+0000","last_undegraded":"2023-09-30T22:26:51.714046+0000","last_fullsized":"2023-09-30T22:26:51.714046+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"15'4","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.722169+0000","last_change":"2023-09-30T22:26:51.291976+0000","last_active":"2023-09-30T22:26:51.722169+0000","last_peered":"2023-09-30T22:26:51.722169+0000","last_clean":"2023-09-30T22:26:51.722169+0000","last_became_active":"2023-09-30T22:26:51.291255+0000","last_became_peered":"2023-09-30T22:26:51.291255+0000","last_unstale":"2023-09-30T22:26:51.722169+0000","last_undegraded":"2023-09-30T22:26:51.722169+0000","last_fullsized":"2023-09-30T22:26:51.722169+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.718466+0000","last_change":"2023-09-30T22:26:51.713485+0000","last_active":"2023-09-30T22:26:51.718466+0000","last_peered":"2023-09-30T22:26:51.718466+0000","last_clean":"2023-09-30T22:26:51.718466+0000","last_became_active":"2023-09-30T22:26:51.713188+0000","last_became_peered":"2023-09-30T22:26:51.713188+0000","last_unstale":"2023-09-30T22:26:51.718466+0000","last_undegraded":"2023-09-30T22:26:51.718466+0000","last_fullsized":"2023-09-30T22:26:51.718466+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.709789+0000","last_change":"2023-09-30T22:26:51.704082+0000","last_active":"2023-09-30T22:26:51.709789+0000","last_peered":"2023-09-30T22:26:51.709789+0000","last_clean":"2023-09-30T22:26:51.709789+0000","last_became_active":"2023-09-30T22:26:51.703280+0000","last_became_peered":"2023-09-30T22:26:51.703280+0000","last_unstale":"2023-09-30T22:26:51.709789+0000","last_undegraded":"2023-09-30T22:26:51.709789+0000","last_fullsized":"2023-09-30T22:26:51.709789+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.714010+0000","last_change":"2023-09-30T22:26:51.705877+0000","last_active":"2023-09-30T22:26:51.714010+0000","last_peered":"2023-09-30T22:26:51.714010+0000","last_clean":"2023-09-30T22:26:51.714010+0000","last_became_active":"2023-09-30T22:26:51.705352+0000","last_became_peered":"2023-09-30T22:26:51.705352+0000","last_unstale":"2023-09-30T22:26:51.714010+0000","last_undegraded":"2023-09-30T22:26:51.714010+0000","last_fullsized":"2023-09-30T22:26:51.714010+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.719132+0000","last_change":"2023-09-30T22:26:51.712201+0000","last_active":"2023-09-30T22:26:51.719132+0000","last_peered":"2023-09-30T22:26:51.719132+0000","last_clean":"2023-09-30T22:26:51.719132+0000","last_became_active":"2023-09-30T22:26:51.709083+0000","last_became_peered":"2023-09-30T22:26:51.709083+0000","last_unstale":"2023-09-30T22:26:51.719132+0000","last_undegraded":"2023-09-30T22:26:51.719132+0000","last_fullsized":"2023-09-30T22:26:51.719132+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.715082+0000","last_change":"2023-09-30T22:26:51.706543+0000","last_active":"2023-09-30T22:26:51.715082+0000","last_peered":"2023-09-30T22:26:51.715082+0000","last_clean":"2023-09-30T22:26:51.715082+0000","last_became_active":"2023-09-30T22:26:51.706251+0000","last_became_peered":"2023-09-30T22:26:51.706251+0000","last_unstale":"2023-09-30T22:26:51.715082+0000","last_undegraded":"2023-09-30T22:26:51.715082+0000","last_fullsized":"2023-09-30T22:26:51.715082+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730665+0000","last_change":"2023-09-30T22:26:51.724591+0000","last_active":"2023-09-30T22:26:51.730665+0000","last_peered":"2023-09-30T22:26:51.730665+0000","last_clean":"2023-09-30T22:26:51.730665+0000","last_became_active":"2023-09-30T22:26:51.722847+0000","last_became_peered":"2023-09-30T22:26:51.722847+0000","last_unstale":"2023-09-30T22:26:51.730665+0000","last_undegraded":"2023-09-30T22:26:51.730665+0000","last_fullsized":"2023-09-30T22:26:51.730665+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705669,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297256,"kb_used_data":160,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074584,"statfs":{"total":96636764160,"available":96332374016,"internally_reserved":0,"allocated":163840,"data_stored":46244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.314}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]}]},{"osd":2,"up_from":9,"seq":38654705670,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297244,"kb_used_data":148,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074596,"statfs":{"total":96636764160,"available":96332386304,"internally_reserved":0,"allocated":151552,"data_stored":46050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]}]},{"osd":0,"up_from":9,"seq":38654705669,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297244,"kb_used_data":148,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074596,"statfs":{"total":96636764160,"available":96332386304,"internally_reserved":0,"allocated":151552,"data_stored":44044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]}]},{"osd":1,"up_from":9,"seq":38654705669,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297244,"kb_used_data":148,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074596,"statfs":{"total":96636764160,"available":96332386304,"internally_reserved":0,"allocated":151552,"data_stored":45160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33400000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-30T22:27:04.150 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-30T22:27:04.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.316+0000 7f22460e0700 1 Processor -- start 2023-09-30T22:27:04.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.316+0000 7f22460e0700 1 -- start start 2023-09-30T22:27:04.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.317+0000 7f22460e0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2240059070 0x7f22401256c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:04.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.317+0000 7f22460e0700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f22401251c0 0x7f2240125bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:04.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.317+0000 7f22460e0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2240093bf0 0x7f2240094050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:04.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.317+0000 7f22460e0700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2240059450 con 0x7f22401251c0 2023-09-30T22:27:04.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.317+0000 7f22460e0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2240095f00 con 0x7f2240093bf0 2023-09-30T22:27:04.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.317+0000 7f22460e0700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2240096190 con 0x7f2240059070 2023-09-30T22:27:04.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.317+0000 7f223ffff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2240093bf0 0x7f2240094050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:04.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.317+0000 7f223f7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2240059070 0x7f22401256c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:04.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.317+0000 7f223effd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f22401251c0 0x7f2240125bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:04.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.317+0000 7f223f7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2240059070 0x7f22401256c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:40288/0 (socket says 172.21.15.16:40288) 2023-09-30T22:27:04.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.317+0000 7f223ffff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2240093bf0 0x7f2240094050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:42388/0 (socket says 172.21.15.16:42388) 2023-09-30T22:27:04.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.317+0000 7f223f7fe700 1 -- 172.21.15.16:0/2585443323 learned_addr learned my addr 172.21.15.16:0/2585443323 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:04.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.317+0000 7f223f7fe700 1 -- 172.21.15.16:0/2585443323 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2240093bf0 msgr2=0x7f2240094050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:04.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.317+0000 7f223f7fe700 1 --2- 172.21.15.16:0/2585443323 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2240093bf0 0x7f2240094050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:04.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.317+0000 7f223f7fe700 1 -- 172.21.15.16:0/2585443323 >> v2:172.21.15.16:3300/0 conn(0x7f22401251c0 msgr2=0x7f2240125bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:04.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.317+0000 7f223f7fe700 1 --2- 172.21.15.16:0/2585443323 >> v2:172.21.15.16:3300/0 conn(0x7f22401251c0 0x7f2240125bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:04.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.317+0000 7f223f7fe700 1 -- 172.21.15.16:0/2585443323 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22401260a0 con 0x7f2240059070 2023-09-30T22:27:04.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.317+0000 7f223effd700 1 --2- 172.21.15.16:0/2585443323 >> v2:172.21.15.16:3300/0 conn(0x7f22401251c0 0x7f2240125bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:04.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.317+0000 7f223ffff700 1 --2- 172.21.15.16:0/2585443323 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2240093bf0 0x7f2240094050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:04.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.318+0000 7f223f7fe700 1 --2- 172.21.15.16:0/2585443323 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2240059070 0x7f22401256c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f2230006fd0 tx=0x7f22300090f0).ready entity=mon.2 client_cookie=837bd215de4e0d20 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:04.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.318+0000 7f223e7fc700 1 -- 172.21.15.16:0/2585443323 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223000f510 con 0x7f2240059070 2023-09-30T22:27:04.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.318+0000 7f223e7fc700 1 -- 172.21.15.16:0/2585443323 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f223000f670 con 0x7f2240059070 2023-09-30T22:27:04.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.318+0000 7f223e7fc700 1 -- 172.21.15.16:0/2585443323 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223000f9f0 con 0x7f2240059070 2023-09-30T22:27:04.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.318+0000 7f22460e0700 1 -- 172.21.15.16:0/2585443323 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2240059070 msgr2=0x7f22401256c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:04.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.318+0000 7f22460e0700 1 --2- 172.21.15.16:0/2585443323 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2240059070 0x7f22401256c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f2230006fd0 tx=0x7f22300090f0).stop 2023-09-30T22:27:04.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.318+0000 7f22460e0700 1 -- 172.21.15.16:0/2585443323 shutdown_connections 2023-09-30T22:27:04.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.318+0000 7f22460e0700 1 --2- 172.21.15.16:0/2585443323 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2240093bf0 0x7f2240094050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:04.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.318+0000 7f22460e0700 1 --2- 172.21.15.16:0/2585443323 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2240059070 0x7f22401256c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:04.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.318+0000 7f22460e0700 1 --2- 172.21.15.16:0/2585443323 >> v2:172.21.15.16:3300/0 conn(0x7f22401251c0 0x7f2240125bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:04.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.318+0000 7f22460e0700 1 -- 172.21.15.16:0/2585443323 >> 172.21.15.16:0/2585443323 conn(0x7f2240097470 msgr2=0x7f2240097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:04.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.318+0000 7f22460e0700 1 -- 172.21.15.16:0/2585443323 shutdown_connections 2023-09-30T22:27:04.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.318+0000 7f22460e0700 1 -- 172.21.15.16:0/2585443323 wait complete. 2023-09-30T22:27:04.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.319+0000 7f22460e0700 1 Processor -- start 2023-09-30T22:27:04.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.319+0000 7f22460e0700 1 -- start start 2023-09-30T22:27:04.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.320+0000 7f22460e0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2240059070 0x7f22401228a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:04.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.320+0000 7f22460e0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2240093bf0 0x7f2240111790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:04.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.320+0000 7f22460e0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f22401251c0 0x7f2240111c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:04.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.320+0000 7f22460e0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2240113c30 con 0x7f2240093bf0 2023-09-30T22:27:04.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.320+0000 7f22460e0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2240095900 con 0x7f2240059070 2023-09-30T22:27:04.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.320+0000 7f22460e0700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f22401152f0 con 0x7f22401251c0 2023-09-30T22:27:04.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.320+0000 7f223f7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2240059070 0x7f22401228a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:04.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.320+0000 7f223f7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2240059070 0x7f22401228a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:42404/0 (socket says 172.21.15.16:42404) 2023-09-30T22:27:04.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.320+0000 7f223ffff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f22401251c0 0x7f2240111c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:04.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.320+0000 7f223f7fe700 1 -- 172.21.15.16:0/3395604836 learned_addr learned my addr 172.21.15.16:0/3395604836 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:04.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.320+0000 7f223effd700 1 --2- 172.21.15.16:0/3395604836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2240093bf0 0x7f2240111790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:04.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.320+0000 7f223ffff700 1 -- 172.21.15.16:0/3395604836 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2240059070 msgr2=0x7f22401228a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:04.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.320+0000 7f223ffff700 1 --2- 172.21.15.16:0/3395604836 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2240059070 0x7f22401228a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:04.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.320+0000 7f223ffff700 1 -- 172.21.15.16:0/3395604836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2240093bf0 msgr2=0x7f2240111790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:04.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.320+0000 7f223ffff700 1 --2- 172.21.15.16:0/3395604836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2240093bf0 0x7f2240111790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:04.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.320+0000 7f223ffff700 1 -- 172.21.15.16:0/3395604836 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2238009a20 con 0x7f22401251c0 2023-09-30T22:27:04.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.320+0000 7f223f7fe700 1 --2- 172.21.15.16:0/3395604836 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2240059070 0x7f22401228a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:04.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.321+0000 7f223ffff700 1 --2- 172.21.15.16:0/3395604836 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f22401251c0 0x7f2240111c80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f2238009f10 tx=0x7f223800b040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:04.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.321+0000 7f22377fe700 1 -- 172.21.15.16:0/3395604836 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2238009c70 con 0x7f22401251c0 2023-09-30T22:27:04.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.321+0000 7f22377fe700 1 -- 172.21.15.16:0/3395604836 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f223800f040 con 0x7f22401251c0 2023-09-30T22:27:04.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.321+0000 7f22460e0700 1 -- 172.21.15.16:0/3395604836 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22280092a0 con 0x7f22401251c0 2023-09-30T22:27:04.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.321+0000 7f22377fe700 1 -- 172.21.15.16:0/3395604836 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223800cd80 con 0x7f22401251c0 2023-09-30T22:27:04.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.321+0000 7f22460e0700 1 -- 172.21.15.16:0/3395604836 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2230009370 con 0x7f22401251c0 2023-09-30T22:27:04.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.322+0000 7f22377fe700 1 -- 172.21.15.16:0/3395604836 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f2238013610 con 0x7f22401251c0 2023-09-30T22:27:04.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.322+0000 7f22377fe700 1 -- 172.21.15.16:0/3395604836 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f223803b730 con 0x7f22401251c0 2023-09-30T22:27:04.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.322+0000 7f22460e0700 1 -- 172.21.15.16:0/3395604836 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f224005a260 con 0x7f22401251c0 2023-09-30T22:27:04.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.327+0000 7f22377fe700 1 -- 172.21.15.16:0/3395604836 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f223801c410 con 0x7f22401251c0 2023-09-30T22:27:04.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.498+0000 7f22460e0700 1 -- 172.21.15.16:0/3395604836 --> v1:172.21.15.16:6816/118285 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2240114ef0 con 0x7f222002bb90 2023-09-30T22:27:04.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.505+0000 7f22377fe700 1 -- 172.21.15.16:0/3395604836 <== mgr.4098 v1:172.21.15.16:6816/118285 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+249919 (unknown 2965378022 0 3540306068) 0x7f223000fd60 con 0x7f222002bb90 2023-09-30T22:27:04.509 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:27:04.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.508+0000 7f22460e0700 1 -- 172.21.15.16:0/3395604836 >> v1:172.21.15.16:6816/118285 conn(0x7f222002bb90 legacy=0x7f222002e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:04.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.508+0000 7f22460e0700 1 -- 172.21.15.16:0/3395604836 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f22401251c0 msgr2=0x7f2240111c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:04.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.508+0000 7f22460e0700 1 --2- 172.21.15.16:0/3395604836 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f22401251c0 0x7f2240111c80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f2238009f10 tx=0x7f223800b040).stop 2023-09-30T22:27:04.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.508+0000 7f22460e0700 1 -- 172.21.15.16:0/3395604836 shutdown_connections 2023-09-30T22:27:04.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.508+0000 7f22460e0700 1 --2- 172.21.15.16:0/3395604836 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2240059070 0x7f22401228a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:04.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.508+0000 7f22460e0700 1 --2- 172.21.15.16:0/3395604836 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f22401251c0 0x7f2240111c80 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:04.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.508+0000 7f22460e0700 1 --2- 172.21.15.16:0/3395604836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2240093bf0 0x7f2240111790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:04.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.508+0000 7f22460e0700 1 -- 172.21.15.16:0/3395604836 >> 172.21.15.16:0/3395604836 conn(0x7f2240097470 msgr2=0x7f2240128930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:04.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.508+0000 7f22460e0700 1 -- 172.21.15.16:0/3395604836 shutdown_connections 2023-09-30T22:27:04.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.508+0000 7f22460e0700 1 -- 172.21.15.16:0/3395604836 wait complete. 2023-09-30T22:27:04.512 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-09-30T22:27:04.522 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-09-30T22:27:04.457435+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1188988,"kb_used_data":604,"kb_used_omap":0,"kb_used_meta":1188352,"kb_avail":376298372,"statfs":{"total":386547056640,"available":385329532928,"internally_reserved":0,"allocated":618496,"data_stored":181498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1216872448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001889"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.718509+0000","last_change":"2023-09-30T22:26:51.708966+0000","last_active":"2023-09-30T22:26:51.718509+0000","last_peered":"2023-09-30T22:26:51.718509+0000","last_clean":"2023-09-30T22:26:51.718509+0000","last_became_active":"2023-09-30T22:26:51.708653+0000","last_became_peered":"2023-09-30T22:26:51.708653+0000","last_unstale":"2023-09-30T22:26:51.718509+0000","last_undegraded":"2023-09-30T22:26:51.718509+0000","last_fullsized":"2023-09-30T22:26:51.718509+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.729225+0000","last_change":"2023-09-30T22:26:51.727077+0000","last_active":"2023-09-30T22:26:51.729225+0000","last_peered":"2023-09-30T22:26:51.729225+0000","last_clean":"2023-09-30T22:26:51.729225+0000","last_became_active":"2023-09-30T22:26:51.726778+0000","last_became_peered":"2023-09-30T22:26:51.726778+0000","last_unstale":"2023-09-30T22:26:51.729225+0000","last_undegraded":"2023-09-30T22:26:51.729225+0000","last_fullsized":"2023-09-30T22:26:51.729225+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.714151+0000","last_change":"2023-09-30T22:26:51.706919+0000","last_active":"2023-09-30T22:26:51.714151+0000","last_peered":"2023-09-30T22:26:51.714151+0000","last_clean":"2023-09-30T22:26:51.714151+0000","last_became_active":"2023-09-30T22:26:51.706676+0000","last_became_peered":"2023-09-30T22:26:51.706676+0000","last_unstale":"2023-09-30T22:26:51.714151+0000","last_undegraded":"2023-09-30T22:26:51.714151+0000","last_fullsized":"2023-09-30T22:26:51.714151+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.728676+0000","last_change":"2023-09-30T22:26:51.727386+0000","last_active":"2023-09-30T22:26:51.728676+0000","last_peered":"2023-09-30T22:26:51.728676+0000","last_clean":"2023-09-30T22:26:51.728676+0000","last_became_active":"2023-09-30T22:26:51.727091+0000","last_became_peered":"2023-09-30T22:26:51.727091+0000","last_unstale":"2023-09-30T22:26:51.728676+0000","last_undegraded":"2023-09-30T22:26:51.728676+0000","last_fullsized":"2023-09-30T22:26:51.728676+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.723905+0000","last_change":"2023-09-30T22:26:51.719993+0000","last_active":"2023-09-30T22:26:51.723905+0000","last_peered":"2023-09-30T22:26:51.723905+0000","last_clean":"2023-09-30T22:26:51.723905+0000","last_became_active":"2023-09-30T22:26:51.719495+0000","last_became_peered":"2023-09-30T22:26:51.719495+0000","last_unstale":"2023-09-30T22:26:51.723905+0000","last_undegraded":"2023-09-30T22:26:51.723905+0000","last_fullsized":"2023-09-30T22:26:51.723905+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.715438+0000","last_change":"2023-09-30T22:26:51.709581+0000","last_active":"2023-09-30T22:26:51.715438+0000","last_peered":"2023-09-30T22:26:51.715438+0000","last_clean":"2023-09-30T22:26:51.715438+0000","last_became_active":"2023-09-30T22:26:51.709309+0000","last_became_peered":"2023-09-30T22:26:51.709309+0000","last_unstale":"2023-09-30T22:26:51.715438+0000","last_undegraded":"2023-09-30T22:26:51.715438+0000","last_fullsized":"2023-09-30T22:26:51.715438+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.713991+0000","last_change":"2023-09-30T22:26:51.705826+0000","last_active":"2023-09-30T22:26:51.713991+0000","last_peered":"2023-09-30T22:26:51.713991+0000","last_clean":"2023-09-30T22:26:51.713991+0000","last_became_active":"2023-09-30T22:26:51.705037+0000","last_became_peered":"2023-09-30T22:26:51.705037+0000","last_unstale":"2023-09-30T22:26:51.713991+0000","last_undegraded":"2023-09-30T22:26:51.713991+0000","last_fullsized":"2023-09-30T22:26:51.713991+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730836+0000","last_change":"2023-09-30T22:26:51.726640+0000","last_active":"2023-09-30T22:26:51.730836+0000","last_peered":"2023-09-30T22:26:51.730836+0000","last_clean":"2023-09-30T22:26:51.730836+0000","last_became_active":"2023-09-30T22:26:51.726278+0000","last_became_peered":"2023-09-30T22:26:51.726278+0000","last_unstale":"2023-09-30T22:26:51.730836+0000","last_undegraded":"2023-09-30T22:26:51.730836+0000","last_fullsized":"2023-09-30T22:26:51.730836+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.724179+0000","last_change":"2023-09-30T22:26:51.721817+0000","last_active":"2023-09-30T22:26:51.724179+0000","last_peered":"2023-09-30T22:26:51.724179+0000","last_clean":"2023-09-30T22:26:51.724179+0000","last_became_active":"2023-09-30T22:26:51.721373+0000","last_became_peered":"2023-09-30T22:26:51.721373+0000","last_unstale":"2023-09-30T22:26:51.724179+0000","last_undegraded":"2023-09-30T22:26:51.724179+0000","last_fullsized":"2023-09-30T22:26:51.724179+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730810+0000","last_change":"2023-09-30T22:26:51.726348+0000","last_active":"2023-09-30T22:26:51.730810+0000","last_peered":"2023-09-30T22:26:51.730810+0000","last_clean":"2023-09-30T22:26:51.730810+0000","last_became_active":"2023-09-30T22:26:51.726030+0000","last_became_peered":"2023-09-30T22:26:51.726030+0000","last_unstale":"2023-09-30T22:26:51.730810+0000","last_undegraded":"2023-09-30T22:26:51.730810+0000","last_fullsized":"2023-09-30T22:26:51.730810+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730607+0000","last_change":"2023-09-30T22:26:51.722619+0000","last_active":"2023-09-30T22:26:51.730607+0000","last_peered":"2023-09-30T22:26:51.730607+0000","last_clean":"2023-09-30T22:26:51.730607+0000","last_became_active":"2023-09-30T22:26:51.722317+0000","last_became_peered":"2023-09-30T22:26:51.722317+0000","last_unstale":"2023-09-30T22:26:51.730607+0000","last_undegraded":"2023-09-30T22:26:51.730607+0000","last_fullsized":"2023-09-30T22:26:51.730607+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.718852+0000","last_change":"2023-09-30T22:26:51.709697+0000","last_active":"2023-09-30T22:26:51.718852+0000","last_peered":"2023-09-30T22:26:51.718852+0000","last_clean":"2023-09-30T22:26:51.718852+0000","last_became_active":"2023-09-30T22:26:51.709342+0000","last_became_peered":"2023-09-30T22:26:51.709342+0000","last_unstale":"2023-09-30T22:26:51.718852+0000","last_undegraded":"2023-09-30T22:26:51.718852+0000","last_fullsized":"2023-09-30T22:26:51.718852+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.719514+0000","last_change":"2023-09-30T22:26:51.716276+0000","last_active":"2023-09-30T22:26:51.719514+0000","last_peered":"2023-09-30T22:26:51.719514+0000","last_clean":"2023-09-30T22:26:51.719514+0000","last_became_active":"2023-09-30T22:26:51.715634+0000","last_became_peered":"2023-09-30T22:26:51.715634+0000","last_unstale":"2023-09-30T22:26:51.719514+0000","last_undegraded":"2023-09-30T22:26:51.719514+0000","last_fullsized":"2023-09-30T22:26:51.719514+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.707967+0000","last_change":"2023-09-30T22:26:51.695948+0000","last_active":"2023-09-30T22:26:51.707967+0000","last_peered":"2023-09-30T22:26:51.707967+0000","last_clean":"2023-09-30T22:26:51.707967+0000","last_became_active":"2023-09-30T22:26:51.695661+0000","last_became_peered":"2023-09-30T22:26:51.695661+0000","last_unstale":"2023-09-30T22:26:51.707967+0000","last_undegraded":"2023-09-30T22:26:51.707967+0000","last_fullsized":"2023-09-30T22:26:51.707967+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.716656+0000","last_change":"2023-09-30T22:26:51.708532+0000","last_active":"2023-09-30T22:26:51.716656+0000","last_peered":"2023-09-30T22:26:51.716656+0000","last_clean":"2023-09-30T22:26:51.716656+0000","last_became_active":"2023-09-30T22:26:51.708204+0000","last_became_peered":"2023-09-30T22:26:51.708204+0000","last_unstale":"2023-09-30T22:26:51.716656+0000","last_undegraded":"2023-09-30T22:26:51.716656+0000","last_fullsized":"2023-09-30T22:26:51.716656+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.714211+0000","last_change":"2023-09-30T22:26:51.704597+0000","last_active":"2023-09-30T22:26:51.714211+0000","last_peered":"2023-09-30T22:26:51.714211+0000","last_clean":"2023-09-30T22:26:51.714211+0000","last_became_active":"2023-09-30T22:26:51.704307+0000","last_became_peered":"2023-09-30T22:26:51.704307+0000","last_unstale":"2023-09-30T22:26:51.714211+0000","last_undegraded":"2023-09-30T22:26:51.714211+0000","last_fullsized":"2023-09-30T22:26:51.714211+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.724159+0000","last_change":"2023-09-30T22:26:51.721640+0000","last_active":"2023-09-30T22:26:51.724159+0000","last_peered":"2023-09-30T22:26:51.724159+0000","last_clean":"2023-09-30T22:26:51.724159+0000","last_became_active":"2023-09-30T22:26:51.720950+0000","last_became_peered":"2023-09-30T22:26:51.720950+0000","last_unstale":"2023-09-30T22:26:51.724159+0000","last_undegraded":"2023-09-30T22:26:51.724159+0000","last_fullsized":"2023-09-30T22:26:51.724159+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.707139+0000","last_change":"2023-09-30T22:26:51.697513+0000","last_active":"2023-09-30T22:26:51.707139+0000","last_peered":"2023-09-30T22:26:51.707139+0000","last_clean":"2023-09-30T22:26:51.707139+0000","last_became_active":"2023-09-30T22:26:51.697137+0000","last_became_peered":"2023-09-30T22:26:51.697137+0000","last_unstale":"2023-09-30T22:26:51.707139+0000","last_undegraded":"2023-09-30T22:26:51.707139+0000","last_fullsized":"2023-09-30T22:26:51.707139+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.718540+0000","last_change":"2023-09-30T22:26:51.715476+0000","last_active":"2023-09-30T22:26:51.718540+0000","last_peered":"2023-09-30T22:26:51.718540+0000","last_clean":"2023-09-30T22:26:51.718540+0000","last_became_active":"2023-09-30T22:26:51.715176+0000","last_became_peered":"2023-09-30T22:26:51.715176+0000","last_unstale":"2023-09-30T22:26:51.718540+0000","last_undegraded":"2023-09-30T22:26:51.718540+0000","last_fullsized":"2023-09-30T22:26:51.718540+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730693+0000","last_change":"2023-09-30T22:26:51.724991+0000","last_active":"2023-09-30T22:26:51.730693+0000","last_peered":"2023-09-30T22:26:51.730693+0000","last_clean":"2023-09-30T22:26:51.730693+0000","last_became_active":"2023-09-30T22:26:51.724240+0000","last_became_peered":"2023-09-30T22:26:51.724240+0000","last_unstale":"2023-09-30T22:26:51.730693+0000","last_undegraded":"2023-09-30T22:26:51.730693+0000","last_fullsized":"2023-09-30T22:26:51.730693+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730391+0000","last_change":"2023-09-30T22:26:51.717833+0000","last_active":"2023-09-30T22:26:51.730391+0000","last_peered":"2023-09-30T22:26:51.730391+0000","last_clean":"2023-09-30T22:26:51.730391+0000","last_became_active":"2023-09-30T22:26:51.717517+0000","last_became_peered":"2023-09-30T22:26:51.717517+0000","last_unstale":"2023-09-30T22:26:51.730391+0000","last_undegraded":"2023-09-30T22:26:51.730391+0000","last_fullsized":"2023-09-30T22:26:51.730391+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.723937+0000","last_change":"2023-09-30T22:26:51.719030+0000","last_active":"2023-09-30T22:26:51.723937+0000","last_peered":"2023-09-30T22:26:51.723937+0000","last_clean":"2023-09-30T22:26:51.723937+0000","last_became_active":"2023-09-30T22:26:51.715370+0000","last_became_peered":"2023-09-30T22:26:51.715370+0000","last_unstale":"2023-09-30T22:26:51.723937+0000","last_undegraded":"2023-09-30T22:26:51.723937+0000","last_fullsized":"2023-09-30T22:26:51.723937+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.713970+0000","last_change":"2023-09-30T22:26:51.698664+0000","last_active":"2023-09-30T22:26:51.713970+0000","last_peered":"2023-09-30T22:26:51.713970+0000","last_clean":"2023-09-30T22:26:51.713970+0000","last_became_active":"2023-09-30T22:26:51.698321+0000","last_became_peered":"2023-09-30T22:26:51.698321+0000","last_unstale":"2023-09-30T22:26:51.713970+0000","last_undegraded":"2023-09-30T22:26:51.713970+0000","last_fullsized":"2023-09-30T22:26:51.713970+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730972+0000","last_change":"2023-09-30T22:26:51.727955+0000","last_active":"2023-09-30T22:26:51.730972+0000","last_peered":"2023-09-30T22:26:51.730972+0000","last_clean":"2023-09-30T22:26:51.730972+0000","last_became_active":"2023-09-30T22:26:51.727209+0000","last_became_peered":"2023-09-30T22:26:51.727209+0000","last_unstale":"2023-09-30T22:26:51.730972+0000","last_undegraded":"2023-09-30T22:26:51.730972+0000","last_fullsized":"2023-09-30T22:26:51.730972+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730499+0000","last_change":"2023-09-30T22:26:51.721154+0000","last_active":"2023-09-30T22:26:51.730499+0000","last_peered":"2023-09-30T22:26:51.730499+0000","last_clean":"2023-09-30T22:26:51.730499+0000","last_became_active":"2023-09-30T22:26:51.720703+0000","last_became_peered":"2023-09-30T22:26:51.720703+0000","last_unstale":"2023-09-30T22:26:51.730499+0000","last_undegraded":"2023-09-30T22:26:51.730499+0000","last_fullsized":"2023-09-30T22:26:51.730499+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.725830+0000","last_change":"2023-09-30T22:26:51.701309+0000","last_active":"2023-09-30T22:26:51.725830+0000","last_peered":"2023-09-30T22:26:51.725830+0000","last_clean":"2023-09-30T22:26:51.725830+0000","last_became_active":"2023-09-30T22:26:51.701012+0000","last_became_peered":"2023-09-30T22:26:51.701012+0000","last_unstale":"2023-09-30T22:26:51.725830+0000","last_undegraded":"2023-09-30T22:26:51.725830+0000","last_fullsized":"2023-09-30T22:26:51.725830+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.718423+0000","last_change":"2023-09-30T22:26:51.715042+0000","last_active":"2023-09-30T22:26:51.718423+0000","last_peered":"2023-09-30T22:26:51.718423+0000","last_clean":"2023-09-30T22:26:51.718423+0000","last_became_active":"2023-09-30T22:26:51.712158+0000","last_became_peered":"2023-09-30T22:26:51.712158+0000","last_unstale":"2023-09-30T22:26:51.718423+0000","last_undegraded":"2023-09-30T22:26:51.718423+0000","last_fullsized":"2023-09-30T22:26:51.718423+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"15'1","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.713320+0000","last_change":"2023-09-30T22:26:51.296397+0000","last_active":"2023-09-30T22:26:51.713320+0000","last_peered":"2023-09-30T22:26:51.713320+0000","last_clean":"2023-09-30T22:26:51.713320+0000","last_became_active":"2023-09-30T22:26:51.295860+0000","last_became_peered":"2023-09-30T22:26:51.295860+0000","last_unstale":"2023-09-30T22:26:51.713320+0000","last_undegraded":"2023-09-30T22:26:51.713320+0000","last_fullsized":"2023-09-30T22:26:51.713320+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.716849+0000","last_change":"2023-09-30T22:26:51.712472+0000","last_active":"2023-09-30T22:26:51.716849+0000","last_peered":"2023-09-30T22:26:51.716849+0000","last_clean":"2023-09-30T22:26:51.716849+0000","last_became_active":"2023-09-30T22:26:51.711527+0000","last_became_peered":"2023-09-30T22:26:51.711527+0000","last_unstale":"2023-09-30T22:26:51.716849+0000","last_undegraded":"2023-09-30T22:26:51.716849+0000","last_fullsized":"2023-09-30T22:26:51.716849+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.701600+0000","last_change":"2023-09-30T22:26:51.298004+0000","last_active":"2023-09-30T22:26:51.701600+0000","last_peered":"2023-09-30T22:26:51.701600+0000","last_clean":"2023-09-30T22:26:51.701600+0000","last_became_active":"2023-09-30T22:26:51.297714+0000","last_became_peered":"2023-09-30T22:26:51.297714+0000","last_unstale":"2023-09-30T22:26:51.701600+0000","last_undegraded":"2023-09-30T22:26:51.701600+0000","last_fullsized":"2023-09-30T22:26:51.701600+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.729456+0000","last_change":"2023-09-30T22:26:51.727608+0000","last_active":"2023-09-30T22:26:51.729456+0000","last_peered":"2023-09-30T22:26:51.729456+0000","last_clean":"2023-09-30T22:26:51.729456+0000","last_became_active":"2023-09-30T22:26:51.727154+0000","last_became_peered":"2023-09-30T22:26:51.727154+0000","last_unstale":"2023-09-30T22:26:51.729456+0000","last_undegraded":"2023-09-30T22:26:51.729456+0000","last_fullsized":"2023-09-30T22:26:51.729456+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.684923+0000","last_change":"2023-09-30T22:26:51.285214+0000","last_active":"2023-09-30T22:26:51.684923+0000","last_peered":"2023-09-30T22:26:51.684923+0000","last_clean":"2023-09-30T22:26:51.684923+0000","last_became_active":"2023-09-30T22:26:51.284749+0000","last_became_peered":"2023-09-30T22:26:51.284749+0000","last_unstale":"2023-09-30T22:26:51.684923+0000","last_undegraded":"2023-09-30T22:26:51.684923+0000","last_fullsized":"2023-09-30T22:26:51.684923+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.716115+0000","last_change":"2023-09-30T22:26:51.706697+0000","last_active":"2023-09-30T22:26:51.716115+0000","last_peered":"2023-09-30T22:26:51.716115+0000","last_clean":"2023-09-30T22:26:51.716115+0000","last_became_active":"2023-09-30T22:26:51.706373+0000","last_became_peered":"2023-09-30T22:26:51.706373+0000","last_unstale":"2023-09-30T22:26:51.716115+0000","last_undegraded":"2023-09-30T22:26:51.716115+0000","last_fullsized":"2023-09-30T22:26:51.716115+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.689119+0000","last_change":"2023-09-30T22:26:51.281642+0000","last_active":"2023-09-30T22:26:51.689119+0000","last_peered":"2023-09-30T22:26:51.689119+0000","last_clean":"2023-09-30T22:26:51.689119+0000","last_became_active":"2023-09-30T22:26:51.281319+0000","last_became_peered":"2023-09-30T22:26:51.281319+0000","last_unstale":"2023-09-30T22:26:51.689119+0000","last_undegraded":"2023-09-30T22:26:51.689119+0000","last_fullsized":"2023-09-30T22:26:51.689119+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.714232+0000","last_change":"2023-09-30T22:26:51.705192+0000","last_active":"2023-09-30T22:26:51.714232+0000","last_peered":"2023-09-30T22:26:51.714232+0000","last_clean":"2023-09-30T22:26:51.714232+0000","last_became_active":"2023-09-30T22:26:51.704870+0000","last_became_peered":"2023-09-30T22:26:51.704870+0000","last_unstale":"2023-09-30T22:26:51.714232+0000","last_undegraded":"2023-09-30T22:26:51.714232+0000","last_fullsized":"2023-09-30T22:26:51.714232+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.680797+0000","last_change":"2023-09-30T22:26:51.293711+0000","last_active":"2023-09-30T22:26:51.680797+0000","last_peered":"2023-09-30T22:26:51.680797+0000","last_clean":"2023-09-30T22:26:51.680797+0000","last_became_active":"2023-09-30T22:26:51.293410+0000","last_became_peered":"2023-09-30T22:26:51.293410+0000","last_unstale":"2023-09-30T22:26:51.680797+0000","last_undegraded":"2023-09-30T22:26:51.680797+0000","last_fullsized":"2023-09-30T22:26:51.680797+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.714029+0000","last_change":"2023-09-30T22:26:51.706005+0000","last_active":"2023-09-30T22:26:51.714029+0000","last_peered":"2023-09-30T22:26:51.714029+0000","last_clean":"2023-09-30T22:26:51.714029+0000","last_became_active":"2023-09-30T22:26:51.705491+0000","last_became_peered":"2023-09-30T22:26:51.705491+0000","last_unstale":"2023-09-30T22:26:51.714029+0000","last_undegraded":"2023-09-30T22:26:51.714029+0000","last_fullsized":"2023-09-30T22:26:51.714029+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"15'3","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.719469+0000","last_change":"2023-09-30T22:26:51.279976+0000","last_active":"2023-09-30T22:26:51.719469+0000","last_peered":"2023-09-30T22:26:51.719469+0000","last_clean":"2023-09-30T22:26:51.719469+0000","last_became_active":"2023-09-30T22:26:51.279669+0000","last_became_peered":"2023-09-30T22:26:51.279669+0000","last_unstale":"2023-09-30T22:26:51.719469+0000","last_undegraded":"2023-09-30T22:26:51.719469+0000","last_fullsized":"2023-09-30T22:26:51.719469+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730714+0000","last_change":"2023-09-30T22:26:51.725947+0000","last_active":"2023-09-30T22:26:51.730714+0000","last_peered":"2023-09-30T22:26:51.730714+0000","last_clean":"2023-09-30T22:26:51.730714+0000","last_became_active":"2023-09-30T22:26:51.723294+0000","last_became_peered":"2023-09-30T22:26:51.723294+0000","last_unstale":"2023-09-30T22:26:51.730714+0000","last_undegraded":"2023-09-30T22:26:51.730714+0000","last_fullsized":"2023-09-30T22:26:51.730714+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"15'6","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.723849+0000","last_change":"2023-09-30T22:26:51.297527+0000","last_active":"2023-09-30T22:26:51.723849+0000","last_peered":"2023-09-30T22:26:51.723849+0000","last_clean":"2023-09-30T22:26:51.723849+0000","last_became_active":"2023-09-30T22:26:51.297246+0000","last_became_peered":"2023-09-30T22:26:51.297246+0000","last_unstale":"2023-09-30T22:26:51.723849+0000","last_undegraded":"2023-09-30T22:26:51.723849+0000","last_fullsized":"2023-09-30T22:26:51.723849+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.718810+0000","last_change":"2023-09-30T22:26:51.710781+0000","last_active":"2023-09-30T22:26:51.718810+0000","last_peered":"2023-09-30T22:26:51.718810+0000","last_clean":"2023-09-30T22:26:51.718810+0000","last_became_active":"2023-09-30T22:26:51.710107+0000","last_became_peered":"2023-09-30T22:26:51.710107+0000","last_unstale":"2023-09-30T22:26:51.718810+0000","last_undegraded":"2023-09-30T22:26:51.718810+0000","last_fullsized":"2023-09-30T22:26:51.718810+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"15'2","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.714029+0000","last_change":"2023-09-30T22:26:51.290194+0000","last_active":"2023-09-30T22:26:51.714029+0000","last_peered":"2023-09-30T22:26:51.714029+0000","last_clean":"2023-09-30T22:26:51.714029+0000","last_became_active":"2023-09-30T22:26:51.288688+0000","last_became_peered":"2023-09-30T22:26:51.288688+0000","last_unstale":"2023-09-30T22:26:51.714029+0000","last_undegraded":"2023-09-30T22:26:51.714029+0000","last_fullsized":"2023-09-30T22:26:51.714029+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.728819+0000","last_change":"2023-09-30T22:26:51.726780+0000","last_active":"2023-09-30T22:26:51.728819+0000","last_peered":"2023-09-30T22:26:51.728819+0000","last_clean":"2023-09-30T22:26:51.728819+0000","last_became_active":"2023-09-30T22:26:51.726408+0000","last_became_peered":"2023-09-30T22:26:51.726408+0000","last_unstale":"2023-09-30T22:26:51.728819+0000","last_undegraded":"2023-09-30T22:26:51.728819+0000","last_fullsized":"2023-09-30T22:26:51.728819+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.685836+0000","last_change":"2023-09-30T22:26:51.281127+0000","last_active":"2023-09-30T22:26:51.685836+0000","last_peered":"2023-09-30T22:26:51.685836+0000","last_clean":"2023-09-30T22:26:51.685836+0000","last_became_active":"2023-09-30T22:26:51.280507+0000","last_became_peered":"2023-09-30T22:26:51.280507+0000","last_unstale":"2023-09-30T22:26:51.685836+0000","last_undegraded":"2023-09-30T22:26:51.685836+0000","last_fullsized":"2023-09-30T22:26:51.685836+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.716876+0000","last_change":"2023-09-30T22:26:51.712671+0000","last_active":"2023-09-30T22:26:51.716876+0000","last_peered":"2023-09-30T22:26:51.716876+0000","last_clean":"2023-09-30T22:26:51.716876+0000","last_became_active":"2023-09-30T22:26:51.711836+0000","last_became_peered":"2023-09-30T22:26:51.711836+0000","last_unstale":"2023-09-30T22:26:51.716876+0000","last_undegraded":"2023-09-30T22:26:51.716876+0000","last_fullsized":"2023-09-30T22:26:51.716876+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.683587+0000","last_change":"2023-09-30T22:26:51.297467+0000","last_active":"2023-09-30T22:26:51.683587+0000","last_peered":"2023-09-30T22:26:51.683587+0000","last_clean":"2023-09-30T22:26:51.683587+0000","last_became_active":"2023-09-30T22:26:51.296950+0000","last_became_peered":"2023-09-30T22:26:51.296950+0000","last_unstale":"2023-09-30T22:26:51.683587+0000","last_undegraded":"2023-09-30T22:26:51.683587+0000","last_fullsized":"2023-09-30T22:26:51.683587+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.724064+0000","last_change":"2023-09-30T22:26:51.719411+0000","last_active":"2023-09-30T22:26:51.724064+0000","last_peered":"2023-09-30T22:26:51.724064+0000","last_clean":"2023-09-30T22:26:51.724064+0000","last_became_active":"2023-09-30T22:26:51.719111+0000","last_became_peered":"2023-09-30T22:26:51.719111+0000","last_unstale":"2023-09-30T22:26:51.724064+0000","last_undegraded":"2023-09-30T22:26:51.724064+0000","last_fullsized":"2023-09-30T22:26:51.724064+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"15'8","reported_seq":18,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.727696+0000","last_change":"2023-09-30T22:26:51.287702+0000","last_active":"2023-09-30T22:26:51.727696+0000","last_peered":"2023-09-30T22:26:51.727696+0000","last_clean":"2023-09-30T22:26:51.727696+0000","last_became_active":"2023-09-30T22:26:51.287434+0000","last_became_peered":"2023-09-30T22:26:51.287434+0000","last_unstale":"2023-09-30T22:26:51.727696+0000","last_undegraded":"2023-09-30T22:26:51.727696+0000","last_fullsized":"2023-09-30T22:26:51.727696+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.716138+0000","last_change":"2023-09-30T22:26:51.706663+0000","last_active":"2023-09-30T22:26:51.716138+0000","last_peered":"2023-09-30T22:26:51.716138+0000","last_clean":"2023-09-30T22:26:51.716138+0000","last_became_active":"2023-09-30T22:26:51.706363+0000","last_became_peered":"2023-09-30T22:26:51.706363+0000","last_unstale":"2023-09-30T22:26:51.716138+0000","last_undegraded":"2023-09-30T22:26:51.716138+0000","last_fullsized":"2023-09-30T22:26:51.716138+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"15'1","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.721973+0000","last_change":"2023-09-30T22:26:51.291996+0000","last_active":"2023-09-30T22:26:51.721973+0000","last_peered":"2023-09-30T22:26:51.721973+0000","last_clean":"2023-09-30T22:26:51.721973+0000","last_became_active":"2023-09-30T22:26:51.291656+0000","last_became_peered":"2023-09-30T22:26:51.291656+0000","last_unstale":"2023-09-30T22:26:51.721973+0000","last_undegraded":"2023-09-30T22:26:51.721973+0000","last_fullsized":"2023-09-30T22:26:51.721973+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.715557+0000","last_change":"2023-09-30T22:26:51.710326+0000","last_active":"2023-09-30T22:26:51.715557+0000","last_peered":"2023-09-30T22:26:51.715557+0000","last_clean":"2023-09-30T22:26:51.715557+0000","last_became_active":"2023-09-30T22:26:51.708793+0000","last_became_peered":"2023-09-30T22:26:51.708793+0000","last_unstale":"2023-09-30T22:26:51.715557+0000","last_undegraded":"2023-09-30T22:26:51.715557+0000","last_fullsized":"2023-09-30T22:26:51.715557+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"15'2","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.699110+0000","last_change":"2023-09-30T22:26:51.300700+0000","last_active":"2023-09-30T22:26:51.699110+0000","last_peered":"2023-09-30T22:26:51.699110+0000","last_clean":"2023-09-30T22:26:51.699110+0000","last_became_active":"2023-09-30T22:26:51.300416+0000","last_became_peered":"2023-09-30T22:26:51.300416+0000","last_unstale":"2023-09-30T22:26:51.699110+0000","last_undegraded":"2023-09-30T22:26:51.699110+0000","last_fullsized":"2023-09-30T22:26:51.699110+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730623+0000","last_change":"2023-09-30T22:26:51.722902+0000","last_active":"2023-09-30T22:26:51.730623+0000","last_peered":"2023-09-30T22:26:51.730623+0000","last_clean":"2023-09-30T22:26:51.730623+0000","last_became_active":"2023-09-30T22:26:51.717919+0000","last_became_peered":"2023-09-30T22:26:51.717919+0000","last_unstale":"2023-09-30T22:26:51.730623+0000","last_undegraded":"2023-09-30T22:26:51.730623+0000","last_fullsized":"2023-09-30T22:26:51.730623+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"15'1","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.716626+0000","last_change":"2023-09-30T22:26:51.292120+0000","last_active":"2023-09-30T22:26:51.716626+0000","last_peered":"2023-09-30T22:26:51.716626+0000","last_clean":"2023-09-30T22:26:51.716626+0000","last_became_active":"2023-09-30T22:26:51.289469+0000","last_became_peered":"2023-09-30T22:26:51.289469+0000","last_unstale":"2023-09-30T22:26:51.716626+0000","last_undegraded":"2023-09-30T22:26:51.716626+0000","last_fullsized":"2023-09-30T22:26:51.716626+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.718700+0000","last_change":"2023-09-30T22:26:51.711649+0000","last_active":"2023-09-30T22:26:51.718700+0000","last_peered":"2023-09-30T22:26:51.718700+0000","last_clean":"2023-09-30T22:26:51.718700+0000","last_became_active":"2023-09-30T22:26:51.711384+0000","last_became_peered":"2023-09-30T22:26:51.711384+0000","last_unstale":"2023-09-30T22:26:51.718700+0000","last_undegraded":"2023-09-30T22:26:51.718700+0000","last_fullsized":"2023-09-30T22:26:51.718700+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.686889+0000","last_change":"2023-09-30T22:26:51.281627+0000","last_active":"2023-09-30T22:26:51.686889+0000","last_peered":"2023-09-30T22:26:51.686889+0000","last_clean":"2023-09-30T22:26:51.686889+0000","last_became_active":"2023-09-30T22:26:51.281311+0000","last_became_peered":"2023-09-30T22:26:51.281311+0000","last_unstale":"2023-09-30T22:26:51.686889+0000","last_undegraded":"2023-09-30T22:26:51.686889+0000","last_fullsized":"2023-09-30T22:26:51.686889+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.707659+0000","last_change":"2023-09-30T22:26:51.700837+0000","last_active":"2023-09-30T22:26:51.707659+0000","last_peered":"2023-09-30T22:26:51.707659+0000","last_clean":"2023-09-30T22:26:51.707659+0000","last_became_active":"2023-09-30T22:26:51.700523+0000","last_became_peered":"2023-09-30T22:26:51.700523+0000","last_unstale":"2023-09-30T22:26:51.707659+0000","last_undegraded":"2023-09-30T22:26:51.707659+0000","last_fullsized":"2023-09-30T22:26:51.707659+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"15'2","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.721892+0000","last_change":"2023-09-30T22:26:51.290248+0000","last_active":"2023-09-30T22:26:51.721892+0000","last_peered":"2023-09-30T22:26:51.721892+0000","last_clean":"2023-09-30T22:26:51.721892+0000","last_became_active":"2023-09-30T22:26:51.289168+0000","last_became_peered":"2023-09-30T22:26:51.289168+0000","last_unstale":"2023-09-30T22:26:51.721892+0000","last_undegraded":"2023-09-30T22:26:51.721892+0000","last_fullsized":"2023-09-30T22:26:51.721892+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.723965+0000","last_change":"2023-09-30T22:26:51.719224+0000","last_active":"2023-09-30T22:26:51.723965+0000","last_peered":"2023-09-30T22:26:51.723965+0000","last_clean":"2023-09-30T22:26:51.723965+0000","last_became_active":"2023-09-30T22:26:51.718890+0000","last_became_peered":"2023-09-30T22:26:51.718890+0000","last_unstale":"2023-09-30T22:26:51.723965+0000","last_undegraded":"2023-09-30T22:26:51.723965+0000","last_fullsized":"2023-09-30T22:26:51.723965+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.687008+0000","last_change":"2023-09-30T22:26:51.288204+0000","last_active":"2023-09-30T22:26:51.687008+0000","last_peered":"2023-09-30T22:26:51.687008+0000","last_clean":"2023-09-30T22:26:51.687008+0000","last_became_active":"2023-09-30T22:26:51.287903+0000","last_became_peered":"2023-09-30T22:26:51.287903+0000","last_unstale":"2023-09-30T22:26:51.687008+0000","last_undegraded":"2023-09-30T22:26:51.687008+0000","last_fullsized":"2023-09-30T22:26:51.687008+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.718399+0000","last_change":"2023-09-30T22:26:51.713792+0000","last_active":"2023-09-30T22:26:51.718399+0000","last_peered":"2023-09-30T22:26:51.718399+0000","last_clean":"2023-09-30T22:26:51.718399+0000","last_became_active":"2023-09-30T22:26:51.713334+0000","last_became_peered":"2023-09-30T22:26:51.713334+0000","last_unstale":"2023-09-30T22:26:51.718399+0000","last_undegraded":"2023-09-30T22:26:51.718399+0000","last_fullsized":"2023-09-30T22:26:51.718399+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.683106+0000","last_change":"2023-09-30T22:26:51.297240+0000","last_active":"2023-09-30T22:26:51.683106+0000","last_peered":"2023-09-30T22:26:51.683106+0000","last_clean":"2023-09-30T22:26:51.683106+0000","last_became_active":"2023-09-30T22:26:51.296547+0000","last_became_peered":"2023-09-30T22:26:51.296547+0000","last_unstale":"2023-09-30T22:26:51.683106+0000","last_undegraded":"2023-09-30T22:26:51.683106+0000","last_fullsized":"2023-09-30T22:26:51.683106+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.718663+0000","last_change":"2023-09-30T22:26:51.711889+0000","last_active":"2023-09-30T22:26:51.718663+0000","last_peered":"2023-09-30T22:26:51.718663+0000","last_clean":"2023-09-30T22:26:51.718663+0000","last_became_active":"2023-09-30T22:26:51.711570+0000","last_became_peered":"2023-09-30T22:26:51.711570+0000","last_unstale":"2023-09-30T22:26:51.718663+0000","last_undegraded":"2023-09-30T22:26:51.718663+0000","last_fullsized":"2023-09-30T22:26:51.718663+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.684383+0000","last_change":"2023-09-30T22:26:51.290699+0000","last_active":"2023-09-30T22:26:51.684383+0000","last_peered":"2023-09-30T22:26:51.684383+0000","last_clean":"2023-09-30T22:26:51.684383+0000","last_became_active":"2023-09-30T22:26:51.290396+0000","last_became_peered":"2023-09-30T22:26:51.290396+0000","last_unstale":"2023-09-30T22:26:51.684383+0000","last_undegraded":"2023-09-30T22:26:51.684383+0000","last_fullsized":"2023-09-30T22:26:51.684383+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730474+0000","last_change":"2023-09-30T22:26:51.720976+0000","last_active":"2023-09-30T22:26:51.730474+0000","last_peered":"2023-09-30T22:26:51.730474+0000","last_clean":"2023-09-30T22:26:51.730474+0000","last_became_active":"2023-09-30T22:26:51.720378+0000","last_became_peered":"2023-09-30T22:26:51.720378+0000","last_unstale":"2023-09-30T22:26:51.730474+0000","last_undegraded":"2023-09-30T22:26:51.730474+0000","last_fullsized":"2023-09-30T22:26:51.730474+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.702412+0000","last_change":"2023-09-30T22:26:46.273891+0000","last_active":"2023-09-30T22:26:51.702412+0000","last_peered":"2023-09-30T22:26:51.702412+0000","last_clean":"2023-09-30T22:26:51.702412+0000","last_became_active":"2023-09-30T22:26:46.273588+0000","last_became_peered":"2023-09-30T22:26:46.273588+0000","last_unstale":"2023-09-30T22:26:51.702412+0000","last_undegraded":"2023-09-30T22:26:51.702412+0000","last_fullsized":"2023-09-30T22:26:51.702412+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.680672+0000","last_change":"2023-09-30T22:26:51.294682+0000","last_active":"2023-09-30T22:26:51.680672+0000","last_peered":"2023-09-30T22:26:51.680672+0000","last_clean":"2023-09-30T22:26:51.680672+0000","last_became_active":"2023-09-30T22:26:51.294422+0000","last_became_peered":"2023-09-30T22:26:51.294422+0000","last_unstale":"2023-09-30T22:26:51.680672+0000","last_undegraded":"2023-09-30T22:26:51.680672+0000","last_fullsized":"2023-09-30T22:26:51.680672+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730874+0000","last_change":"2023-09-30T22:26:51.726954+0000","last_active":"2023-09-30T22:26:51.730874+0000","last_peered":"2023-09-30T22:26:51.730874+0000","last_clean":"2023-09-30T22:26:51.730874+0000","last_became_active":"2023-09-30T22:26:51.726641+0000","last_became_peered":"2023-09-30T22:26:51.726641+0000","last_unstale":"2023-09-30T22:26:51.730874+0000","last_undegraded":"2023-09-30T22:26:51.730874+0000","last_fullsized":"2023-09-30T22:26:51.730874+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.687452+0000","last_change":"2023-09-30T22:26:46.245946+0000","last_active":"2023-09-30T22:26:51.687452+0000","last_peered":"2023-09-30T22:26:51.687452+0000","last_clean":"2023-09-30T22:26:51.687452+0000","last_became_active":"2023-09-30T22:26:46.245602+0000","last_became_peered":"2023-09-30T22:26:46.245602+0000","last_unstale":"2023-09-30T22:26:51.687452+0000","last_undegraded":"2023-09-30T22:26:51.687452+0000","last_fullsized":"2023-09-30T22:26:51.687452+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.685668+0000","last_change":"2023-09-30T22:26:51.294691+0000","last_active":"2023-09-30T22:26:51.685668+0000","last_peered":"2023-09-30T22:26:51.685668+0000","last_clean":"2023-09-30T22:26:51.685668+0000","last_became_active":"2023-09-30T22:26:51.294411+0000","last_became_peered":"2023-09-30T22:26:51.294411+0000","last_unstale":"2023-09-30T22:26:51.685668+0000","last_undegraded":"2023-09-30T22:26:51.685668+0000","last_fullsized":"2023-09-30T22:26:51.685668+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730680+0000","last_change":"2023-09-30T22:26:51.724759+0000","last_active":"2023-09-30T22:26:51.730680+0000","last_peered":"2023-09-30T22:26:51.730680+0000","last_clean":"2023-09-30T22:26:51.730680+0000","last_became_active":"2023-09-30T22:26:51.723221+0000","last_became_peered":"2023-09-30T22:26:51.723221+0000","last_unstale":"2023-09-30T22:26:51.730680+0000","last_undegraded":"2023-09-30T22:26:51.730680+0000","last_fullsized":"2023-09-30T22:26:51.730680+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.681986+0000","last_change":"2023-09-30T22:26:46.242607+0000","last_active":"2023-09-30T22:26:51.681986+0000","last_peered":"2023-09-30T22:26:51.681986+0000","last_clean":"2023-09-30T22:26:51.681986+0000","last_became_active":"2023-09-30T22:26:46.242283+0000","last_became_peered":"2023-09-30T22:26:46.242283+0000","last_unstale":"2023-09-30T22:26:51.681986+0000","last_undegraded":"2023-09-30T22:26:51.681986+0000","last_fullsized":"2023-09-30T22:26:51.681986+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.703666+0000","last_change":"2023-09-30T22:26:51.295409+0000","last_active":"2023-09-30T22:26:51.703666+0000","last_peered":"2023-09-30T22:26:51.703666+0000","last_clean":"2023-09-30T22:26:51.703666+0000","last_became_active":"2023-09-30T22:26:51.295140+0000","last_became_peered":"2023-09-30T22:26:51.295140+0000","last_unstale":"2023-09-30T22:26:51.703666+0000","last_undegraded":"2023-09-30T22:26:51.703666+0000","last_fullsized":"2023-09-30T22:26:51.703666+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730344+0000","last_change":"2023-09-30T22:26:51.715504+0000","last_active":"2023-09-30T22:26:51.730344+0000","last_peered":"2023-09-30T22:26:51.730344+0000","last_clean":"2023-09-30T22:26:51.730344+0000","last_became_active":"2023-09-30T22:26:51.715258+0000","last_became_peered":"2023-09-30T22:26:51.715258+0000","last_unstale":"2023-09-30T22:26:51.730344+0000","last_undegraded":"2023-09-30T22:26:51.730344+0000","last_fullsized":"2023-09-30T22:26:51.730344+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.682511+0000","last_change":"2023-09-30T22:26:46.242752+0000","last_active":"2023-09-30T22:26:51.682511+0000","last_peered":"2023-09-30T22:26:51.682511+0000","last_clean":"2023-09-30T22:26:51.682511+0000","last_became_active":"2023-09-30T22:26:46.242385+0000","last_became_peered":"2023-09-30T22:26:46.242385+0000","last_unstale":"2023-09-30T22:26:51.682511+0000","last_undegraded":"2023-09-30T22:26:51.682511+0000","last_fullsized":"2023-09-30T22:26:51.682511+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.703565+0000","last_change":"2023-09-30T22:26:46.273930+0000","last_active":"2023-09-30T22:26:51.703565+0000","last_peered":"2023-09-30T22:26:51.703565+0000","last_clean":"2023-09-30T22:26:51.703565+0000","last_became_active":"2023-09-30T22:26:46.273621+0000","last_became_peered":"2023-09-30T22:26:46.273621+0000","last_unstale":"2023-09-30T22:26:51.703565+0000","last_undegraded":"2023-09-30T22:26:51.703565+0000","last_fullsized":"2023-09-30T22:26:51.703565+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.686892+0000","last_change":"2023-09-30T22:26:51.298163+0000","last_active":"2023-09-30T22:26:51.686892+0000","last_peered":"2023-09-30T22:26:51.686892+0000","last_clean":"2023-09-30T22:26:51.686892+0000","last_became_active":"2023-09-30T22:26:51.297897+0000","last_became_peered":"2023-09-30T22:26:51.297897+0000","last_unstale":"2023-09-30T22:26:51.686892+0000","last_undegraded":"2023-09-30T22:26:51.686892+0000","last_fullsized":"2023-09-30T22:26:51.686892+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.718377+0000","last_change":"2023-09-30T22:26:51.713608+0000","last_active":"2023-09-30T22:26:51.718377+0000","last_peered":"2023-09-30T22:26:51.718377+0000","last_clean":"2023-09-30T22:26:51.718377+0000","last_became_active":"2023-09-30T22:26:51.712992+0000","last_became_peered":"2023-09-30T22:26:51.712992+0000","last_unstale":"2023-09-30T22:26:51.718377+0000","last_undegraded":"2023-09-30T22:26:51.718377+0000","last_fullsized":"2023-09-30T22:26:51.718377+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.684824+0000","last_change":"2023-09-30T22:26:46.245668+0000","last_active":"2023-09-30T22:26:51.684824+0000","last_peered":"2023-09-30T22:26:51.684824+0000","last_clean":"2023-09-30T22:26:51.684824+0000","last_became_active":"2023-09-30T22:26:46.245319+0000","last_became_peered":"2023-09-30T22:26:46.245319+0000","last_unstale":"2023-09-30T22:26:51.684824+0000","last_undegraded":"2023-09-30T22:26:51.684824+0000","last_fullsized":"2023-09-30T22:26:51.684824+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.681181+0000","last_change":"2023-09-30T22:26:51.294664+0000","last_active":"2023-09-30T22:26:51.681181+0000","last_peered":"2023-09-30T22:26:51.681181+0000","last_clean":"2023-09-30T22:26:51.681181+0000","last_became_active":"2023-09-30T22:26:51.294357+0000","last_became_peered":"2023-09-30T22:26:51.294357+0000","last_unstale":"2023-09-30T22:26:51.681181+0000","last_undegraded":"2023-09-30T22:26:51.681181+0000","last_fullsized":"2023-09-30T22:26:51.681181+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.707651+0000","last_change":"2023-09-30T22:26:51.705784+0000","last_active":"2023-09-30T22:26:51.707651+0000","last_peered":"2023-09-30T22:26:51.707651+0000","last_clean":"2023-09-30T22:26:51.707651+0000","last_became_active":"2023-09-30T22:26:51.705480+0000","last_became_peered":"2023-09-30T22:26:51.705480+0000","last_unstale":"2023-09-30T22:26:51.707651+0000","last_undegraded":"2023-09-30T22:26:51.707651+0000","last_fullsized":"2023-09-30T22:26:51.707651+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.689511+0000","last_change":"2023-09-30T22:26:46.274062+0000","last_active":"2023-09-30T22:26:51.689511+0000","last_peered":"2023-09-30T22:26:51.689511+0000","last_clean":"2023-09-30T22:26:51.689511+0000","last_became_active":"2023-09-30T22:26:46.273752+0000","last_became_peered":"2023-09-30T22:26:46.273752+0000","last_unstale":"2023-09-30T22:26:51.689511+0000","last_undegraded":"2023-09-30T22:26:51.689511+0000","last_fullsized":"2023-09-30T22:26:51.689511+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.702185+0000","last_change":"2023-09-30T22:26:51.295181+0000","last_active":"2023-09-30T22:26:51.702185+0000","last_peered":"2023-09-30T22:26:51.702185+0000","last_clean":"2023-09-30T22:26:51.702185+0000","last_became_active":"2023-09-30T22:26:51.294889+0000","last_became_peered":"2023-09-30T22:26:51.294889+0000","last_unstale":"2023-09-30T22:26:51.702185+0000","last_undegraded":"2023-09-30T22:26:51.702185+0000","last_fullsized":"2023-09-30T22:26:51.702185+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.698806+0000","last_change":"2023-09-30T22:26:51.690467+0000","last_active":"2023-09-30T22:26:51.698806+0000","last_peered":"2023-09-30T22:26:51.698806+0000","last_clean":"2023-09-30T22:26:51.698806+0000","last_became_active":"2023-09-30T22:26:51.690163+0000","last_became_peered":"2023-09-30T22:26:51.690163+0000","last_unstale":"2023-09-30T22:26:51.698806+0000","last_undegraded":"2023-09-30T22:26:51.698806+0000","last_fullsized":"2023-09-30T22:26:51.698806+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"15'2","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.716522+0000","last_change":"2023-09-30T22:26:51.290268+0000","last_active":"2023-09-30T22:26:51.716522+0000","last_peered":"2023-09-30T22:26:51.716522+0000","last_clean":"2023-09-30T22:26:51.716522+0000","last_became_active":"2023-09-30T22:26:51.289409+0000","last_became_peered":"2023-09-30T22:26:51.289409+0000","last_unstale":"2023-09-30T22:26:51.716522+0000","last_undegraded":"2023-09-30T22:26:51.716522+0000","last_fullsized":"2023-09-30T22:26:51.716522+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.712603+0000","last_change":"2023-09-30T22:26:46.273888+0000","last_active":"2023-09-30T22:26:51.712603+0000","last_peered":"2023-09-30T22:26:51.712603+0000","last_clean":"2023-09-30T22:26:51.712603+0000","last_became_active":"2023-09-30T22:26:46.273549+0000","last_became_peered":"2023-09-30T22:26:46.273549+0000","last_unstale":"2023-09-30T22:26:51.712603+0000","last_undegraded":"2023-09-30T22:26:51.712603+0000","last_fullsized":"2023-09-30T22:26:51.712603+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.714861+0000","last_change":"2023-09-30T22:26:51.709356+0000","last_active":"2023-09-30T22:26:51.714861+0000","last_peered":"2023-09-30T22:26:51.714861+0000","last_clean":"2023-09-30T22:26:51.714861+0000","last_became_active":"2023-09-30T22:26:51.709059+0000","last_became_peered":"2023-09-30T22:26:51.709059+0000","last_unstale":"2023-09-30T22:26:51.714861+0000","last_undegraded":"2023-09-30T22:26:51.714861+0000","last_fullsized":"2023-09-30T22:26:51.714861+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.682520+0000","last_change":"2023-09-30T22:26:46.274508+0000","last_active":"2023-09-30T22:26:51.682520+0000","last_peered":"2023-09-30T22:26:51.682520+0000","last_clean":"2023-09-30T22:26:51.682520+0000","last_became_active":"2023-09-30T22:26:46.273567+0000","last_became_peered":"2023-09-30T22:26:46.273567+0000","last_unstale":"2023-09-30T22:26:51.682520+0000","last_undegraded":"2023-09-30T22:26:51.682520+0000","last_fullsized":"2023-09-30T22:26:51.682520+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"16'1","reported_seq":11,"reported_epoch":16,"state":"active+clean","last_fresh":"2023-09-30T22:26:52.891144+0000","last_change":"2023-09-30T22:26:51.288625+0000","last_active":"2023-09-30T22:26:52.891144+0000","last_peered":"2023-09-30T22:26:52.891144+0000","last_clean":"2023-09-30T22:26:52.891144+0000","last_became_active":"2023-09-30T22:26:51.288286+0000","last_became_peered":"2023-09-30T22:26:51.288286+0000","last_unstale":"2023-09-30T22:26:52.891144+0000","last_undegraded":"2023-09-30T22:26:52.891144+0000","last_fullsized":"2023-09-30T22:26:52.891144+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.718567+0000","last_change":"2023-09-30T22:26:51.708965+0000","last_active":"2023-09-30T22:26:51.718567+0000","last_peered":"2023-09-30T22:26:51.718567+0000","last_clean":"2023-09-30T22:26:51.718567+0000","last_became_active":"2023-09-30T22:26:51.708607+0000","last_became_peered":"2023-09-30T22:26:51.708607+0000","last_unstale":"2023-09-30T22:26:51.718567+0000","last_undegraded":"2023-09-30T22:26:51.718567+0000","last_fullsized":"2023-09-30T22:26:51.718567+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"15'1","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.719553+0000","last_change":"2023-09-30T22:26:51.282125+0000","last_active":"2023-09-30T22:26:51.719553+0000","last_peered":"2023-09-30T22:26:51.719553+0000","last_clean":"2023-09-30T22:26:51.719553+0000","last_became_active":"2023-09-30T22:26:51.281780+0000","last_became_peered":"2023-09-30T22:26:51.281780+0000","last_unstale":"2023-09-30T22:26:51.719553+0000","last_undegraded":"2023-09-30T22:26:51.719553+0000","last_fullsized":"2023-09-30T22:26:51.719553+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.709532+0000","last_change":"2023-09-30T22:26:51.703281+0000","last_active":"2023-09-30T22:26:51.709532+0000","last_peered":"2023-09-30T22:26:51.709532+0000","last_clean":"2023-09-30T22:26:51.709532+0000","last_became_active":"2023-09-30T22:26:51.703007+0000","last_became_peered":"2023-09-30T22:26:51.703007+0000","last_unstale":"2023-09-30T22:26:51.709532+0000","last_undegraded":"2023-09-30T22:26:51.709532+0000","last_fullsized":"2023-09-30T22:26:51.709532+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"15'2","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.726495+0000","last_change":"2023-09-30T22:26:51.287630+0000","last_active":"2023-09-30T22:26:51.726495+0000","last_peered":"2023-09-30T22:26:51.726495+0000","last_clean":"2023-09-30T22:26:51.726495+0000","last_became_active":"2023-09-30T22:26:51.287337+0000","last_became_peered":"2023-09-30T22:26:51.287337+0000","last_unstale":"2023-09-30T22:26:51.726495+0000","last_undegraded":"2023-09-30T22:26:51.726495+0000","last_fullsized":"2023-09-30T22:26:51.726495+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.719532+0000","last_change":"2023-09-30T22:26:51.716458+0000","last_active":"2023-09-30T22:26:51.719532+0000","last_peered":"2023-09-30T22:26:51.719532+0000","last_clean":"2023-09-30T22:26:51.719532+0000","last_became_active":"2023-09-30T22:26:51.715957+0000","last_became_peered":"2023-09-30T22:26:51.715957+0000","last_unstale":"2023-09-30T22:26:51.719532+0000","last_undegraded":"2023-09-30T22:26:51.719532+0000","last_fullsized":"2023-09-30T22:26:51.719532+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"15'2","reported_seq":12,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.716732+0000","last_change":"2023-09-30T22:26:51.285956+0000","last_active":"2023-09-30T22:26:51.716732+0000","last_peered":"2023-09-30T22:26:51.716732+0000","last_clean":"2023-09-30T22:26:51.716732+0000","last_became_active":"2023-09-30T22:26:51.285690+0000","last_became_peered":"2023-09-30T22:26:51.285690+0000","last_unstale":"2023-09-30T22:26:51.716732+0000","last_undegraded":"2023-09-30T22:26:51.716732+0000","last_fullsized":"2023-09-30T22:26:51.716732+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.719485+0000","last_change":"2023-09-30T22:26:51.716290+0000","last_active":"2023-09-30T22:26:51.719485+0000","last_peered":"2023-09-30T22:26:51.719485+0000","last_clean":"2023-09-30T22:26:51.719485+0000","last_became_active":"2023-09-30T22:26:51.716007+0000","last_became_peered":"2023-09-30T22:26:51.716007+0000","last_unstale":"2023-09-30T22:26:51.719485+0000","last_undegraded":"2023-09-30T22:26:51.719485+0000","last_fullsized":"2023-09-30T22:26:51.719485+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1e","version":"15'1","reported_seq":11,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.719629+0000","last_change":"2023-09-30T22:26:51.296949+0000","last_active":"2023-09-30T22:26:51.719629+0000","last_peered":"2023-09-30T22:26:51.719629+0000","last_clean":"2023-09-30T22:26:51.719629+0000","last_became_active":"2023-09-30T22:26:51.296602+0000","last_became_peered":"2023-09-30T22:26:51.296602+0000","last_unstale":"2023-09-30T22:26:51.719629+0000","last_undegraded":"2023-09-30T22:26:51.719629+0000","last_fullsized":"2023-09-30T22:26:51.719629+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.714046+0000","last_change":"2023-09-30T22:26:51.706034+0000","last_active":"2023-09-30T22:26:51.714046+0000","last_peered":"2023-09-30T22:26:51.714046+0000","last_clean":"2023-09-30T22:26:51.714046+0000","last_became_active":"2023-09-30T22:26:51.705633+0000","last_became_peered":"2023-09-30T22:26:51.705633+0000","last_unstale":"2023-09-30T22:26:51.714046+0000","last_undegraded":"2023-09-30T22:26:51.714046+0000","last_fullsized":"2023-09-30T22:26:51.714046+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"15'4","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.722169+0000","last_change":"2023-09-30T22:26:51.291976+0000","last_active":"2023-09-30T22:26:51.722169+0000","last_peered":"2023-09-30T22:26:51.722169+0000","last_clean":"2023-09-30T22:26:51.722169+0000","last_became_active":"2023-09-30T22:26:51.291255+0000","last_became_peered":"2023-09-30T22:26:51.291255+0000","last_unstale":"2023-09-30T22:26:51.722169+0000","last_undegraded":"2023-09-30T22:26:51.722169+0000","last_fullsized":"2023-09-30T22:26:51.722169+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.718466+0000","last_change":"2023-09-30T22:26:51.713485+0000","last_active":"2023-09-30T22:26:51.718466+0000","last_peered":"2023-09-30T22:26:51.718466+0000","last_clean":"2023-09-30T22:26:51.718466+0000","last_became_active":"2023-09-30T22:26:51.713188+0000","last_became_peered":"2023-09-30T22:26:51.713188+0000","last_unstale":"2023-09-30T22:26:51.718466+0000","last_undegraded":"2023-09-30T22:26:51.718466+0000","last_fullsized":"2023-09-30T22:26:51.718466+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.709789+0000","last_change":"2023-09-30T22:26:51.704082+0000","last_active":"2023-09-30T22:26:51.709789+0000","last_peered":"2023-09-30T22:26:51.709789+0000","last_clean":"2023-09-30T22:26:51.709789+0000","last_became_active":"2023-09-30T22:26:51.703280+0000","last_became_peered":"2023-09-30T22:26:51.703280+0000","last_unstale":"2023-09-30T22:26:51.709789+0000","last_undegraded":"2023-09-30T22:26:51.709789+0000","last_fullsized":"2023-09-30T22:26:51.709789+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.714010+0000","last_change":"2023-09-30T22:26:51.705877+0000","last_active":"2023-09-30T22:26:51.714010+0000","last_peered":"2023-09-30T22:26:51.714010+0000","last_clean":"2023-09-30T22:26:51.714010+0000","last_became_active":"2023-09-30T22:26:51.705352+0000","last_became_peered":"2023-09-30T22:26:51.705352+0000","last_unstale":"2023-09-30T22:26:51.714010+0000","last_undegraded":"2023-09-30T22:26:51.714010+0000","last_fullsized":"2023-09-30T22:26:51.714010+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.719132+0000","last_change":"2023-09-30T22:26:51.712201+0000","last_active":"2023-09-30T22:26:51.719132+0000","last_peered":"2023-09-30T22:26:51.719132+0000","last_clean":"2023-09-30T22:26:51.719132+0000","last_became_active":"2023-09-30T22:26:51.709083+0000","last_became_peered":"2023-09-30T22:26:51.709083+0000","last_unstale":"2023-09-30T22:26:51.719132+0000","last_undegraded":"2023-09-30T22:26:51.719132+0000","last_fullsized":"2023-09-30T22:26:51.719132+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.715082+0000","last_change":"2023-09-30T22:26:51.706543+0000","last_active":"2023-09-30T22:26:51.715082+0000","last_peered":"2023-09-30T22:26:51.715082+0000","last_clean":"2023-09-30T22:26:51.715082+0000","last_became_active":"2023-09-30T22:26:51.706251+0000","last_became_peered":"2023-09-30T22:26:51.706251+0000","last_unstale":"2023-09-30T22:26:51.715082+0000","last_undegraded":"2023-09-30T22:26:51.715082+0000","last_fullsized":"2023-09-30T22:26:51.715082+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2023-09-30T22:26:51.730665+0000","last_change":"2023-09-30T22:26:51.724591+0000","last_active":"2023-09-30T22:26:51.730665+0000","last_peered":"2023-09-30T22:26:51.730665+0000","last_clean":"2023-09-30T22:26:51.730665+0000","last_became_active":"2023-09-30T22:26:51.722847+0000","last_became_peered":"2023-09-30T22:26:51.722847+0000","last_unstale":"2023-09-30T22:26:51.730665+0000","last_undegraded":"2023-09-30T22:26:51.730665+0000","last_fullsized":"2023-09-30T22:26:51.730665+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705669,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297256,"kb_used_data":160,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074584,"statfs":{"total":96636764160,"available":96332374016,"internally_reserved":0,"allocated":163840,"data_stored":46244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.314}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]}]},{"osd":2,"up_from":9,"seq":38654705670,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297244,"kb_used_data":148,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074596,"statfs":{"total":96636764160,"available":96332386304,"internally_reserved":0,"allocated":151552,"data_stored":46050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]}]},{"osd":0,"up_from":9,"seq":38654705669,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297244,"kb_used_data":148,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074596,"statfs":{"total":96636764160,"available":96332386304,"internally_reserved":0,"allocated":151552,"data_stored":44044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]}]},{"osd":1,"up_from":9,"seq":38654705669,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297244,"kb_used_data":148,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074596,"statfs":{"total":96636764160,"available":96332386304,"internally_reserved":0,"allocated":151552,"data_stored":45160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33400000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-30T22:27:04.528 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-09-30T22:27:04.528 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-09-30T22:27:04.528 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-09-30T22:27:04.528 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T22:27:04.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.693+0000 7f0e27132700 1 Processor -- start 2023-09-30T22:27:04.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.693+0000 7f0e27132700 1 -- start start 2023-09-30T22:27:04.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.694+0000 7f0e27132700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0e2012bc30 0x7f0e2012e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:04.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.694+0000 7f0e27132700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0e2012e510 0x7f0e20130900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:04.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.694+0000 7f0e27132700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e20092460 con 0x7f0e20096830 2023-09-30T22:27:04.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.694+0000 7f0e27132700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e20094430 con 0x7f0e2012e510 2023-09-30T22:27:04.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.694+0000 7f0e27132700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0e20092ae0 con 0x7f0e2012bc30 2023-09-30T22:27:04.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.694+0000 7f0e1ffff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0e2012bc30 0x7f0e2012e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:04.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.694+0000 7f0e256cf700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0e2012e510 0x7f0e20130900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:04.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.694+0000 7f0e24ece700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0e20096830 0x7f0e20096c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51524/0 (socket says 172.21.15.16:51524) 2023-09-30T22:27:04.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.694+0000 7f0e24ece700 1 -- 172.21.15.16:0/1861889226 learned_addr learned my addr 172.21.15.16:0/1861889226 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:27:04.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.695+0000 7f0e256cf700 1 -- 172.21.15.16:0/1861889226 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0e2012bc30 msgr2=0x7f0e2012e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:04.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.695+0000 7f0e256cf700 1 --2- 172.21.15.16:0/1861889226 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0e2012bc30 0x7f0e2012e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:04.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.695+0000 7f0e1f7fe700 1 -- 172.21.15.16:0/1861889226 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1621699792 0 0) 0x7f0e20092460 con 0x7f0e20096830 2023-09-30T22:27:04.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.695+0000 7f0e256cf700 1 -- 172.21.15.16:0/1861889226 >> v1:172.21.15.16:6789/0 conn(0x7f0e20096830 legacy=0x7f0e20096c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:04.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.695+0000 7f0e256cf700 1 -- 172.21.15.16:0/1861889226 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e200934d0 con 0x7f0e2012e510 2023-09-30T22:27:04.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.695+0000 7f0e256cf700 1 --2- 172.21.15.16:0/1861889226 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0e2012e510 0x7f0e20130900 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f0e18009e90 tx=0x7f0e18009780).ready entity=mon.1 client_cookie=aafc58288e10f1c9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:04.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.695+0000 7f0e1f7fe700 1 -- 172.21.15.16:0/1861889226 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1800e070 con 0x7f0e2012e510 2023-09-30T22:27:04.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.695+0000 7f0e1f7fe700 1 -- 172.21.15.16:0/1861889226 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e1800bb60 con 0x7f0e2012e510 2023-09-30T22:27:04.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.695+0000 7f0e1f7fe700 1 -- 172.21.15.16:0/1861889226 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e18012410 con 0x7f0e2012e510 2023-09-30T22:27:04.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.696+0000 7f0e27132700 1 -- 172.21.15.16:0/1861889226 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0e2012e510 msgr2=0x7f0e20130900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:04.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.696+0000 7f0e27132700 1 --2- 172.21.15.16:0/1861889226 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0e2012e510 0x7f0e20130900 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f0e18009e90 tx=0x7f0e18009780).stop 2023-09-30T22:27:04.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.696+0000 7f0e27132700 1 -- 172.21.15.16:0/1861889226 shutdown_connections 2023-09-30T22:27:04.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.696+0000 7f0e27132700 1 --2- 172.21.15.16:0/1861889226 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0e2012e510 0x7f0e20130900 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:04.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.696+0000 7f0e27132700 1 --2- 172.21.15.16:0/1861889226 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0e2012bc30 0x7f0e2012e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:04.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.696+0000 7f0e27132700 1 -- 172.21.15.16:0/1861889226 >> 172.21.15.16:0/1861889226 conn(0x7f0e2009a220 msgr2=0x7f0e2009a630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:04.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.696+0000 7f0e27132700 1 -- 172.21.15.16:0/1861889226 shutdown_connections 2023-09-30T22:27:04.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.696+0000 7f0e27132700 1 -- 172.21.15.16:0/1861889226 wait complete. 2023-09-30T22:27:04.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.697+0000 7f0e27132700 1 Processor -- start 2023-09-30T22:27:04.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.697+0000 7f0e27132700 1 -- start start 2023-09-30T22:27:04.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.698+0000 7f0e27132700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0e20096830 0x7f0e20124170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:04.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.698+0000 7f0e27132700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e2012bc30 0x7f0e2012c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:04.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.698+0000 7f0e27132700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0e2012e510 0x7f0e2012c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:04.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.698+0000 7f0e27132700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e20112550 con 0x7f0e2012bc30 2023-09-30T22:27:04.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.698+0000 7f0e27132700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e20111360 con 0x7f0e20096830 2023-09-30T22:27:04.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.698+0000 7f0e27132700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0e201118b0 con 0x7f0e2012e510 2023-09-30T22:27:04.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.698+0000 7f0e24ece700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0e20096830 0x7f0e20124170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:04.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.698+0000 7f0e1ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e2012bc30 0x7f0e2012c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:04.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.698+0000 7f0e24ece700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0e20096830 0x7f0e20124170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:42416/0 (socket says 172.21.15.16:42416) 2023-09-30T22:27:04.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.698+0000 7f0e24ece700 1 -- 172.21.15.16:0/2082112325 learned_addr learned my addr 172.21.15.16:0/2082112325 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:04.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.698+0000 7f0e256cf700 1 --2- 172.21.15.16:0/2082112325 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0e2012e510 0x7f0e2012c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:04.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.698+0000 7f0e24ece700 1 -- 172.21.15.16:0/2082112325 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0e2012e510 msgr2=0x7f0e2012c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:04.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.698+0000 7f0e24ece700 1 --2- 172.21.15.16:0/2082112325 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0e2012e510 0x7f0e2012c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:04.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.698+0000 7f0e24ece700 1 -- 172.21.15.16:0/2082112325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e2012bc30 msgr2=0x7f0e2012c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:04.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.698+0000 7f0e24ece700 1 --2- 172.21.15.16:0/2082112325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e2012bc30 0x7f0e2012c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:04.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.698+0000 7f0e24ece700 1 -- 172.21.15.16:0/2082112325 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e18009a20 con 0x7f0e20096830 2023-09-30T22:27:04.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.699+0000 7f0e24ece700 1 --2- 172.21.15.16:0/2082112325 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0e20096830 0x7f0e20124170 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f0e1000c4f0 tx=0x7f0e10007c10).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:04.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.699+0000 7f0e1cff9700 1 -- 172.21.15.16:0/2082112325 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1000ce20 con 0x7f0e20096830 2023-09-30T22:27:04.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.699+0000 7f0e1cff9700 1 -- 172.21.15.16:0/2082112325 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e100072e0 con 0x7f0e20096830 2023-09-30T22:27:04.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.699+0000 7f0e27132700 1 -- 172.21.15.16:0/2082112325 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e200674b0 con 0x7f0e20096830 2023-09-30T22:27:04.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.699+0000 7f0e1cff9700 1 -- 172.21.15.16:0/2082112325 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e100057a0 con 0x7f0e20096830 2023-09-30T22:27:04.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.699+0000 7f0e27132700 1 -- 172.21.15.16:0/2082112325 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e2012dc50 con 0x7f0e20096830 2023-09-30T22:27:04.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.700+0000 7f0e1cff9700 1 -- 172.21.15.16:0/2082112325 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f0e10005bd0 con 0x7f0e20096830 2023-09-30T22:27:04.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.700+0000 7f0e1cff9700 1 -- 172.21.15.16:0/2082112325 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f0e1003a730 con 0x7f0e20096830 2023-09-30T22:27:04.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.700+0000 7f0e27132700 1 -- 172.21.15.16:0/2082112325 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e2005a260 con 0x7f0e20096830 2023-09-30T22:27:04.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.705+0000 7f0e1cff9700 1 -- 172.21.15.16:0/2082112325 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f0e10018030 con 0x7f0e20096830 2023-09-30T22:27:04.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.919+0000 7f0e27132700 1 -- 172.21.15.16:0/2082112325 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f0e2012ceb0 con 0x7f0e20096830 2023-09-30T22:27:04.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.919+0000 7f0e1cff9700 1 -- 172.21.15.16:0/2082112325 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+45 (secure 0 0 0) 0x7f0e1001cd70 con 0x7f0e20096830 2023-09-30T22:27:04.921 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:27:04.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.921+0000 7f0e27132700 1 -- 172.21.15.16:0/2082112325 >> v1:172.21.15.16:6816/118285 conn(0x7f0e0002bb60 legacy=0x7f0e0002e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:04.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.921+0000 7f0e27132700 1 -- 172.21.15.16:0/2082112325 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0e20096830 msgr2=0x7f0e20124170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:04.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.921+0000 7f0e27132700 1 --2- 172.21.15.16:0/2082112325 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0e20096830 0x7f0e20124170 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f0e1000c4f0 tx=0x7f0e10007c10).stop 2023-09-30T22:27:04.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.921+0000 7f0e27132700 1 -- 172.21.15.16:0/2082112325 shutdown_connections 2023-09-30T22:27:04.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.921+0000 7f0e27132700 1 --2- 172.21.15.16:0/2082112325 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0e20096830 0x7f0e20124170 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:04.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.921+0000 7f0e27132700 1 --2- 172.21.15.16:0/2082112325 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0e2012e510 0x7f0e2012c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:04.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.921+0000 7f0e27132700 1 --2- 172.21.15.16:0/2082112325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e2012bc30 0x7f0e2012c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:04.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.921+0000 7f0e27132700 1 -- 172.21.15.16:0/2082112325 >> 172.21.15.16:0/2082112325 conn(0x7f0e2009a220 msgr2=0x7f0e20126c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:04.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.921+0000 7f0e27132700 1 -- 172.21.15.16:0/2082112325 shutdown_connections 2023-09-30T22:27:04.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:04.921+0000 7f0e27132700 1 -- 172.21.15.16:0/2082112325 wait complete. 2023-09-30T22:27:04.935 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-09-30T22:27:04.935 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-09-30T22:27:04.936 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:27:05.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.064+0000 7f77d4be7700 1 Processor -- start 2023-09-30T22:27:05.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.064+0000 7f77d4be7700 1 -- start start 2023-09-30T22:27:05.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.065+0000 7f77d4be7700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f77d0066200 0x7f77d0097780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:05.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.065+0000 7f77d4be7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f77d012b720 0x7f77d0127c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:05.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.065+0000 7f77d4be7700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77d0065740 con 0x7f77d012b300 2023-09-30T22:27:05.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.065+0000 7f77d4be7700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77d0064ef0 con 0x7f77d0066200 2023-09-30T22:27:05.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.065+0000 7f77d4be7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f77d0065a00 con 0x7f77d012b720 2023-09-30T22:27:05.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.065+0000 7f77ced9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f77d012b720 0x7f77d0127c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:05.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.065+0000 7f77ce59c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f77d0066200 0x7f77d0097780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:05.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.065+0000 7f77cdd9b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f77d012b300 0x7f77d0127500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51532/0 (socket says 172.21.15.16:51532) 2023-09-30T22:27:05.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.065+0000 7f77ce59c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f77d0066200 0x7f77d0097780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54684/0 (socket says 172.21.15.16:54684) 2023-09-30T22:27:05.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.065+0000 7f77cdd9b700 1 -- 172.21.15.16:0/2672540261 learned_addr learned my addr 172.21.15.16:0/2672540261 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:27:05.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.065+0000 7f77ced9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f77d012b720 0x7f77d0127c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:40320/0 (socket says 172.21.15.16:40320) 2023-09-30T22:27:05.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.065+0000 7f77ced9d700 1 -- 172.21.15.16:0/2672540261 >> v2:172.21.15.16:3300/0 conn(0x7f77d0066200 msgr2=0x7f77d0097780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:05.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.065+0000 7f77ced9d700 1 --2- 172.21.15.16:0/2672540261 >> v2:172.21.15.16:3300/0 conn(0x7f77d0066200 0x7f77d0097780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:05.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.065+0000 7f77cd59a700 1 -- 172.21.15.16:0/2672540261 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3538316843 0 0) 0x7f77d0065740 con 0x7f77d012b300 2023-09-30T22:27:05.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.065+0000 7f77ced9d700 1 -- 172.21.15.16:0/2672540261 >> v1:172.21.15.16:6789/0 conn(0x7f77d012b300 legacy=0x7f77d0127500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:05.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.065+0000 7f77ced9d700 1 -- 172.21.15.16:0/2672540261 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d00653c0 con 0x7f77d012b720 2023-09-30T22:27:05.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.065+0000 7f77ced9d700 1 --2- 172.21.15.16:0/2672540261 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f77d012b720 0x7f77d0127c10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f77c8009e60 tx=0x7f77c8009780).ready entity=mon.2 client_cookie=3f8e910dd370dc8a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:05.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.066+0000 7f77cd59a700 1 -- 172.21.15.16:0/2672540261 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c800e070 con 0x7f77d012b720 2023-09-30T22:27:05.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.066+0000 7f77cd59a700 1 -- 172.21.15.16:0/2672540261 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77c800bb60 con 0x7f77d012b720 2023-09-30T22:27:05.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.066+0000 7f77cd59a700 1 -- 172.21.15.16:0/2672540261 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c8012410 con 0x7f77d012b720 2023-09-30T22:27:05.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.066+0000 7f77d4be7700 1 -- 172.21.15.16:0/2672540261 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f77d012b720 msgr2=0x7f77d0127c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:05.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.066+0000 7f77d4be7700 1 --2- 172.21.15.16:0/2672540261 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f77d012b720 0x7f77d0127c10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f77c8009e60 tx=0x7f77c8009780).stop 2023-09-30T22:27:05.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.066+0000 7f77d4be7700 1 -- 172.21.15.16:0/2672540261 shutdown_connections 2023-09-30T22:27:05.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.066+0000 7f77d4be7700 1 --2- 172.21.15.16:0/2672540261 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f77d012b720 0x7f77d0127c10 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:05.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.066+0000 7f77d4be7700 1 --2- 172.21.15.16:0/2672540261 >> v2:172.21.15.16:3300/0 conn(0x7f77d0066200 0x7f77d0097780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:05.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.066+0000 7f77d4be7700 1 -- 172.21.15.16:0/2672540261 >> 172.21.15.16:0/2672540261 conn(0x7f77d009ba50 msgr2=0x7f77d009be60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:05.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.066+0000 7f77d4be7700 1 -- 172.21.15.16:0/2672540261 shutdown_connections 2023-09-30T22:27:05.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.066+0000 7f77d4be7700 1 -- 172.21.15.16:0/2672540261 wait complete. 2023-09-30T22:27:05.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.067+0000 7f77d4be7700 1 Processor -- start 2023-09-30T22:27:05.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.067+0000 7f77d4be7700 1 -- start start 2023-09-30T22:27:05.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.068+0000 7f77d4be7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77d0066200 0x7f77d0117cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:05.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.068+0000 7f77d4be7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f77d012b300 0x7f77d0113d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:05.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.068+0000 7f77d4be7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f77d012b720 0x7f77d0114220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:05.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.068+0000 7f77d4be7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d0117230 con 0x7f77d0066200 2023-09-30T22:27:05.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.068+0000 7f77d4be7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77d012a010 con 0x7f77d012b300 2023-09-30T22:27:05.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.068+0000 7f77d4be7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f77d0065120 con 0x7f77d012b720 2023-09-30T22:27:05.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.068+0000 7f77ce59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77d0066200 0x7f77d0117cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:05.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.068+0000 7f77ced9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f77d012b720 0x7f77d0114220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:05.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.068+0000 7f77ce59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77d0066200 0x7f77d0117cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54686/0 (socket says 172.21.15.16:54686) 2023-09-30T22:27:05.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.068+0000 7f77ce59c700 1 -- 172.21.15.16:0/1461243765 learned_addr learned my addr 172.21.15.16:0/1461243765 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:05.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.068+0000 7f77cdd9b700 1 --2- 172.21.15.16:0/1461243765 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f77d012b300 0x7f77d0113d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:05.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.068+0000 7f77ced9d700 1 -- 172.21.15.16:0/1461243765 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f77d012b300 msgr2=0x7f77d0113d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:05.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.068+0000 7f77ced9d700 1 --2- 172.21.15.16:0/1461243765 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f77d012b300 0x7f77d0113d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:05.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.068+0000 7f77ced9d700 1 -- 172.21.15.16:0/1461243765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77d0066200 msgr2=0x7f77d0117cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:05.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.068+0000 7f77ced9d700 1 --2- 172.21.15.16:0/1461243765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77d0066200 0x7f77d0117cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:05.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.068+0000 7f77ced9d700 1 -- 172.21.15.16:0/1461243765 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77c8009a20 con 0x7f77d012b720 2023-09-30T22:27:05.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.068+0000 7f77ce59c700 1 --2- 172.21.15.16:0/1461243765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77d0066200 0x7f77d0117cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:05.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.068+0000 7f77ced9d700 1 --2- 172.21.15.16:0/1461243765 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f77d012b720 0x7f77d0114220 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f77c8014000 tx=0x7f77c8009780).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:05.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.069+0000 7f77c67fc700 1 -- 172.21.15.16:0/1461243765 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c8012410 con 0x7f77d012b720 2023-09-30T22:27:05.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.069+0000 7f77c67fc700 1 -- 172.21.15.16:0/1461243765 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77c801d630 con 0x7f77d012b720 2023-09-30T22:27:05.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.069+0000 7f77d4be7700 1 -- 172.21.15.16:0/1461243765 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77d0124ee0 con 0x7f77d012b720 2023-09-30T22:27:05.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.069+0000 7f77c67fc700 1 -- 172.21.15.16:0/1461243765 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c80070e0 con 0x7f77d012b720 2023-09-30T22:27:05.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.069+0000 7f77d4be7700 1 -- 172.21.15.16:0/1461243765 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77d0114f70 con 0x7f77d012b720 2023-09-30T22:27:05.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.069+0000 7f77c67fc700 1 -- 172.21.15.16:0/1461243765 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f77c8003040 con 0x7f77d012b720 2023-09-30T22:27:05.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.070+0000 7f77c67fc700 1 -- 172.21.15.16:0/1461243765 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f77c803cc70 con 0x7f77d012b720 2023-09-30T22:27:05.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.070+0000 7f77d4be7700 1 -- 172.21.15.16:0/1461243765 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77d005a260 con 0x7f77d012b720 2023-09-30T22:27:05.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.074+0000 7f77c67fc700 1 -- 172.21.15.16:0/1461243765 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f77c8025dc0 con 0x7f77d012b720 2023-09-30T22:27:05.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.251+0000 7f77d4be7700 1 -- 172.21.15.16:0/1461243765 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f77d012af40 con 0x7f77d012b720 2023-09-30T22:27:05.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.252+0000 7f77c67fc700 1 -- 172.21.15.16:0/1461243765 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+11341 (secure 0 0 0) 0x7f77c8025360 con 0x7f77d012b720 2023-09-30T22:27:05.253 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:27:05.253 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":16,"fsid":"7ede3e14-2bd0-4726-9437-6d13c90cd342","created":"2023-09-30T22:26:35.483697+0000","modified":"2023-09-30T22:26:52.675888+0000","last_up_change":"2023-09-30T22:26:44.227199+0000","last_in_change":"2023-09-30T22:26:37.120906+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-09-30T22:26:44.285133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:26:45.132921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:26:49.353683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:26:50.655685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"19946985-0ee2-487f-9347-929670dbe5ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6804","nonce":118824}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6805","nonce":118824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6807","nonce":118824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6806","nonce":118824}]},"public_addr":"172.21.15.16:6804/118824","cluster_addr":"172.21.15.16:6805/118824","heartbeat_back_addr":"172.21.15.16:6807/118824","heartbeat_front_addr":"172.21.15.16:6806/118824","state":["exists","up"]},{"osd":1,"uuid":"8efd4dbc-6ae4-4b82-b530-3b3eac2be445","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6812","nonce":118832}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6813","nonce":118832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6815","nonce":118832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6814","nonce":118832}]},"public_addr":"172.21.15.16:6812/118832","cluster_addr":"172.21.15.16:6813/118832","heartbeat_back_addr":"172.21.15.16:6815/118832","heartbeat_front_addr":"172.21.15.16:6814/118832","state":["exists","up"]},{"osd":2,"uuid":"b47bb1ce-27af-4c0c-868a-b76ebd0d9499","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6808","nonce":118840}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6809","nonce":118840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6811","nonce":118840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6810","nonce":118840}]},"public_addr":"172.21.15.16:6808/118840","cluster_addr":"172.21.15.16:6809/118840","heartbeat_back_addr":"172.21.15.16:6811/118840","heartbeat_front_addr":"172.21.15.16:6810/118840","state":["exists","up"]},{"osd":3,"uuid":"03b53d64-d10d-448a-a081-39ecceab589b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6800","nonce":118823}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6801","nonce":118823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6803","nonce":118823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6802","nonce":118823}]},"public_addr":"172.21.15.16:6800/118823","cluster_addr":"172.21.15.16:6801/118823","heartbeat_back_addr":"172.21.15.16:6803/118823","heartbeat_front_addr":"172.21.15.16:6802/118823","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.305953+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.307488+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.341023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.192071+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:27:05.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.254+0000 7f77d4be7700 1 -- 172.21.15.16:0/1461243765 >> v1:172.21.15.16:6816/118285 conn(0x7f77b002bb60 legacy=0x7f77b002e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:05.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.254+0000 7f77d4be7700 1 -- 172.21.15.16:0/1461243765 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f77d012b720 msgr2=0x7f77d0114220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:05.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.254+0000 7f77d4be7700 1 --2- 172.21.15.16:0/1461243765 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f77d012b720 0x7f77d0114220 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f77c8014000 tx=0x7f77c8009780).stop 2023-09-30T22:27:05.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.254+0000 7f77d4be7700 1 -- 172.21.15.16:0/1461243765 shutdown_connections 2023-09-30T22:27:05.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.254+0000 7f77d4be7700 1 --2- 172.21.15.16:0/1461243765 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f77d012b720 0x7f77d0114220 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:05.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.254+0000 7f77d4be7700 1 --2- 172.21.15.16:0/1461243765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77d0066200 0x7f77d0117cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:05.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.254+0000 7f77d4be7700 1 --2- 172.21.15.16:0/1461243765 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f77d012b300 0x7f77d0113d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:05.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.254+0000 7f77d4be7700 1 -- 172.21.15.16:0/1461243765 >> 172.21.15.16:0/1461243765 conn(0x7f77d009ba50 msgr2=0x7f77d01277b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:05.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.254+0000 7f77d4be7700 1 -- 172.21.15.16:0/1461243765 shutdown_connections 2023-09-30T22:27:05.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.254+0000 7f77d4be7700 1 -- 172.21.15.16:0/1461243765 wait complete. 2023-09-30T22:27:05.270 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-09-30T22:26:44.285133+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-09-30T22:26:45.132921+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-30T22:26:49.353683+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-09-30T22:26:50.655685+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-30T22:27:05.270 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-09-30T22:27:05.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.403+0000 7f5bdc2b9700 1 Processor -- start 2023-09-30T22:27:05.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.403+0000 7f5bdc2b9700 1 -- start start 2023-09-30T22:27:05.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.404+0000 7f5bdc2b9700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5bd4098740 0x7f5bd4098b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:05.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.404+0000 7f5bdc2b9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5bd4069430 0x7f5bd4130c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:05.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.404+0000 7f5bdc2b9700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bd4095270 con 0x7f5bd4099010 2023-09-30T22:27:05.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.404+0000 7f5bdc2b9700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bd4094d20 con 0x7f5bd4098740 2023-09-30T22:27:05.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.404+0000 7f5bdc2b9700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd4096520 con 0x7f5bd4069430 2023-09-30T22:27:05.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.404+0000 7f5bda055700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5bd4098740 0x7f5bd4098b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:05.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.404+0000 7f5bda856700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5bd4069430 0x7f5bd4130c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:05.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.404+0000 7f5bda055700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5bd4098740 0x7f5bd4098b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54696/0 (socket says 172.21.15.16:54696) 2023-09-30T22:27:05.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.404+0000 7f5bd9854700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5bd4099010 0x7f5bd4068d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51540/0 (socket says 172.21.15.16:51540) 2023-09-30T22:27:05.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.404+0000 7f5bda856700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5bd4069430 0x7f5bd4130c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:42446/0 (socket says 172.21.15.16:42446) 2023-09-30T22:27:05.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.404+0000 7f5bda055700 1 -- 172.21.15.16:0/4285200419 learned_addr learned my addr 172.21.15.16:0/4285200419 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:05.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.404+0000 7f5bda055700 1 -- 172.21.15.16:0/4285200419 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5bd4069430 msgr2=0x7f5bd4130c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:05.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.404+0000 7f5bda055700 1 --2- 172.21.15.16:0/4285200419 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5bd4069430 0x7f5bd4130c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:05.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.404+0000 7f5bda055700 1 -- 172.21.15.16:0/4285200419 >> v1:172.21.15.16:6789/0 conn(0x7f5bd4099010 legacy=0x7f5bd4068d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:05.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.404+0000 7f5bd9053700 1 -- 172.21.15.16:0/4285200419 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2736644336 0 0) 0x7f5bd4095270 con 0x7f5bd4099010 2023-09-30T22:27:05.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.404+0000 7f5bda055700 1 -- 172.21.15.16:0/4285200419 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bd4094870 con 0x7f5bd4098740 2023-09-30T22:27:05.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.404+0000 7f5bda856700 1 --2- 172.21.15.16:0/4285200419 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5bd4069430 0x7f5bd4130c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:05.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.404+0000 7f5bda055700 1 --2- 172.21.15.16:0/4285200419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bd4098740 0x7f5bd4098b20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f5bd0006fd0 tx=0x7f5bd00090f0).ready entity=mon.0 client_cookie=b770a8e22cff8254 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:05.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.405+0000 7f5bd9053700 1 -- 172.21.15.16:0/4285200419 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd000b580 con 0x7f5bd4098740 2023-09-30T22:27:05.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.405+0000 7f5bd9053700 1 -- 172.21.15.16:0/4285200419 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bd000b6e0 con 0x7f5bd4098740 2023-09-30T22:27:05.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.405+0000 7f5bd9053700 1 -- 172.21.15.16:0/4285200419 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd000ba80 con 0x7f5bd4098740 2023-09-30T22:27:05.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.405+0000 7f5bdc2b9700 1 -- 172.21.15.16:0/4285200419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bd4098740 msgr2=0x7f5bd4098b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:05.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.405+0000 7f5bdc2b9700 1 --2- 172.21.15.16:0/4285200419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bd4098740 0x7f5bd4098b20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f5bd0006fd0 tx=0x7f5bd00090f0).stop 2023-09-30T22:27:05.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.405+0000 7f5bdc2b9700 1 -- 172.21.15.16:0/4285200419 shutdown_connections 2023-09-30T22:27:05.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.405+0000 7f5bdc2b9700 1 --2- 172.21.15.16:0/4285200419 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5bd4069430 0x7f5bd4130c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:05.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.405+0000 7f5bdc2b9700 1 --2- 172.21.15.16:0/4285200419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bd4098740 0x7f5bd4098b20 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:05.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.405+0000 7f5bdc2b9700 1 -- 172.21.15.16:0/4285200419 >> 172.21.15.16:0/4285200419 conn(0x7f5bd409a1c0 msgr2=0x7f5bd409a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:05.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.405+0000 7f5bdc2b9700 1 -- 172.21.15.16:0/4285200419 shutdown_connections 2023-09-30T22:27:05.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.405+0000 7f5bdc2b9700 1 -- 172.21.15.16:0/4285200419 wait complete. 2023-09-30T22:27:05.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.406+0000 7f5bdc2b9700 1 Processor -- start 2023-09-30T22:27:05.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.406+0000 7f5bdc2b9700 1 -- start start 2023-09-30T22:27:05.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.407+0000 7f5bdc2b9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5bd4069430 0x7f5bd411d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:05.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.407+0000 7f5bdc2b9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5bd4098740 0x7f5bd4091a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:05.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.407+0000 7f5bdc2b9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bd4099010 0x7f5bd4091f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:05.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.407+0000 7f5bdc2b9700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd4094f70 con 0x7f5bd4099010 2023-09-30T22:27:05.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.407+0000 7f5bdc2b9700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd411c960 con 0x7f5bd4069430 2023-09-30T22:27:05.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.407+0000 7f5bdc2b9700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd411d000 con 0x7f5bd4098740 2023-09-30T22:27:05.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.407+0000 7f5bda055700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5bd4069430 0x7f5bd411d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:05.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.407+0000 7f5bda856700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bd4099010 0x7f5bd4091f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:05.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.407+0000 7f5bda055700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5bd4069430 0x7f5bd411d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:42462/0 (socket says 172.21.15.16:42462) 2023-09-30T22:27:05.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.407+0000 7f5bda055700 1 -- 172.21.15.16:0/2151715102 learned_addr learned my addr 172.21.15.16:0/2151715102 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:05.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.407+0000 7f5bda856700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bd4099010 0x7f5bd4091f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54710/0 (socket says 172.21.15.16:54710) 2023-09-30T22:27:05.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.407+0000 7f5bd9854700 1 --2- 172.21.15.16:0/2151715102 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5bd4098740 0x7f5bd4091a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:05.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.407+0000 7f5bda055700 1 -- 172.21.15.16:0/2151715102 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5bd4098740 msgr2=0x7f5bd4091a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:05.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.407+0000 7f5bda055700 1 --2- 172.21.15.16:0/2151715102 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5bd4098740 0x7f5bd4091a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:05.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.407+0000 7f5bda055700 1 -- 172.21.15.16:0/2151715102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bd4099010 msgr2=0x7f5bd4091f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:05.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.407+0000 7f5bda055700 1 --2- 172.21.15.16:0/2151715102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bd4099010 0x7f5bd4091f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:05.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.407+0000 7f5bda055700 1 -- 172.21.15.16:0/2151715102 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bcc009a20 con 0x7f5bd4069430 2023-09-30T22:27:05.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.407+0000 7f5bda055700 1 --2- 172.21.15.16:0/2151715102 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5bd4069430 0x7f5bd411d400 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f5bd4094e60 tx=0x7f5bd0017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:05.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.408+0000 7f5bc67fc700 1 -- 172.21.15.16:0/2151715102 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd0006950 con 0x7f5bd4069430 2023-09-30T22:27:05.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.408+0000 7f5bc67fc700 1 -- 172.21.15.16:0/2151715102 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bd000be40 con 0x7f5bd4069430 2023-09-30T22:27:05.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.408+0000 7f5bc67fc700 1 -- 172.21.15.16:0/2151715102 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd000ba80 con 0x7f5bd4069430 2023-09-30T22:27:05.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.408+0000 7f5bdc2b9700 1 -- 172.21.15.16:0/2151715102 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bd0009370 con 0x7f5bd4069430 2023-09-30T22:27:05.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.408+0000 7f5bdc2b9700 1 -- 172.21.15.16:0/2151715102 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bd412b5c0 con 0x7f5bd4069430 2023-09-30T22:27:05.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.409+0000 7f5bdc2b9700 1 -- 172.21.15.16:0/2151715102 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bd405a260 con 0x7f5bd4069430 2023-09-30T22:27:05.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.409+0000 7f5bc67fc700 1 -- 172.21.15.16:0/2151715102 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f5bd001c630 con 0x7f5bd4069430 2023-09-30T22:27:05.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.409+0000 7f5bc67fc700 1 -- 172.21.15.16:0/2151715102 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f5bd003cc30 con 0x7f5bd4069430 2023-09-30T22:27:05.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.414+0000 7f5bc67fc700 1 -- 172.21.15.16:0/2151715102 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f5bd001b3b0 con 0x7f5bd4069430 2023-09-30T22:27:05.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.587+0000 7f5bdc2b9700 1 -- 172.21.15.16:0/2151715102 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f5bd4094540 con 0x7f5bd4069430 2023-09-30T22:27:05.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.588+0000 7f5bc67fc700 1 -- 172.21.15.16:0/2151715102 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v16) v1 ==== 110+0+10 (secure 0 0 0) 0x7f5bd002a020 con 0x7f5bd4069430 2023-09-30T22:27:05.589 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-09-30T22:27:05.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.590+0000 7f5bdc2b9700 1 -- 172.21.15.16:0/2151715102 >> v1:172.21.15.16:6816/118285 conn(0x7f5bb802bb60 legacy=0x7f5bb802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:05.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.590+0000 7f5bdc2b9700 1 -- 172.21.15.16:0/2151715102 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5bd4069430 msgr2=0x7f5bd411d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:05.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.590+0000 7f5bdc2b9700 1 --2- 172.21.15.16:0/2151715102 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5bd4069430 0x7f5bd411d400 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f5bd4094e60 tx=0x7f5bd0017000).stop 2023-09-30T22:27:05.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.590+0000 7f5bdc2b9700 1 -- 172.21.15.16:0/2151715102 shutdown_connections 2023-09-30T22:27:05.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.590+0000 7f5bdc2b9700 1 --2- 172.21.15.16:0/2151715102 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5bd4069430 0x7f5bd411d400 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:05.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.590+0000 7f5bdc2b9700 1 --2- 172.21.15.16:0/2151715102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bd4099010 0x7f5bd4091f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:05.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.590+0000 7f5bdc2b9700 1 --2- 172.21.15.16:0/2151715102 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5bd4098740 0x7f5bd4091a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:05.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.590+0000 7f5bdc2b9700 1 -- 172.21.15.16:0/2151715102 >> 172.21.15.16:0/2151715102 conn(0x7f5bd409a1c0 msgr2=0x7f5bd4065d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:05.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.590+0000 7f5bdc2b9700 1 -- 172.21.15.16:0/2151715102 shutdown_connections 2023-09-30T22:27:05.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.590+0000 7f5bdc2b9700 1 -- 172.21.15.16:0/2151715102 wait complete. 2023-09-30T22:27:05.604 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-09-30T22:27:05.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.735+0000 7f9359089700 1 Processor -- start 2023-09-30T22:27:05.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.735+0000 7f9359089700 1 -- start start 2023-09-30T22:27:05.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.736+0000 7f9359089700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9354093840 0x7f9354098790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:05.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.736+0000 7f9359089700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9354098c80 0x7f9354099060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:05.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.736+0000 7f9359089700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9354092d80 con 0x7f935412c150 2023-09-30T22:27:05.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.736+0000 7f9359089700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9354093c20 con 0x7f9354093840 2023-09-30T22:27:05.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.736+0000 7f9359089700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9354092b90 con 0x7f9354098c80 2023-09-30T22:27:05.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.736+0000 7f935259c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9354098c80 0x7f9354099060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:05.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.736+0000 7f9352d9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9354093840 0x7f9354098790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:05.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.736+0000 7f935259c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9354098c80 0x7f9354099060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:40338/0 (socket says 172.21.15.16:40338) 2023-09-30T22:27:05.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.736+0000 7f935359e700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f935412c150 0x7f935412e540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51542/0 (socket says 172.21.15.16:51542) 2023-09-30T22:27:05.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.736+0000 7f9352d9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9354093840 0x7f9354098790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54726/0 (socket says 172.21.15.16:54726) 2023-09-30T22:27:05.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.736+0000 7f935259c700 1 -- 172.21.15.16:0/2332117706 learned_addr learned my addr 172.21.15.16:0/2332117706 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:05.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.736+0000 7f935259c700 1 -- 172.21.15.16:0/2332117706 >> v2:172.21.15.16:3300/0 conn(0x7f9354093840 msgr2=0x7f9354098790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:05.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.736+0000 7f935259c700 1 --2- 172.21.15.16:0/2332117706 >> v2:172.21.15.16:3300/0 conn(0x7f9354093840 0x7f9354098790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:05.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.736+0000 7f935259c700 1 -- 172.21.15.16:0/2332117706 >> v1:172.21.15.16:6789/0 conn(0x7f935412c150 legacy=0x7f935412e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:05.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.736+0000 7f935259c700 1 -- 172.21.15.16:0/2332117706 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9354099b70 con 0x7f9354098c80 2023-09-30T22:27:05.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.736+0000 7f9351d9b700 1 -- 172.21.15.16:0/2332117706 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 538071233 0 0) 0x7f9354092d80 con 0x7f935412c150 2023-09-30T22:27:05.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.736+0000 7f9352d9d700 1 --2- 172.21.15.16:0/2332117706 >> v2:172.21.15.16:3300/0 conn(0x7f9354093840 0x7f9354098790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:05.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.736+0000 7f935259c700 1 --2- 172.21.15.16:0/2332117706 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9354098c80 0x7f9354099060 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f933c006fd0 tx=0x7f933c0090f0).ready entity=mon.2 client_cookie=5751d9421fd6f33e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:05.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.736+0000 7f9351d9b700 1 -- 172.21.15.16:0/2332117706 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933c00b580 con 0x7f9354098c80 2023-09-30T22:27:05.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.737+0000 7f9351d9b700 1 -- 172.21.15.16:0/2332117706 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f933c00b6e0 con 0x7f9354098c80 2023-09-30T22:27:05.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.737+0000 7f9351d9b700 1 -- 172.21.15.16:0/2332117706 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933c00ba60 con 0x7f9354098c80 2023-09-30T22:27:05.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.737+0000 7f9359089700 1 -- 172.21.15.16:0/2332117706 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9354098c80 msgr2=0x7f9354099060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:05.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.737+0000 7f9359089700 1 --2- 172.21.15.16:0/2332117706 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9354098c80 0x7f9354099060 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f933c006fd0 tx=0x7f933c0090f0).stop 2023-09-30T22:27:05.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.737+0000 7f9359089700 1 -- 172.21.15.16:0/2332117706 shutdown_connections 2023-09-30T22:27:05.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.737+0000 7f9359089700 1 --2- 172.21.15.16:0/2332117706 >> v2:172.21.15.16:3300/0 conn(0x7f9354093840 0x7f9354098790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:05.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.737+0000 7f9359089700 1 --2- 172.21.15.16:0/2332117706 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9354098c80 0x7f9354099060 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:05.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.737+0000 7f9359089700 1 -- 172.21.15.16:0/2332117706 >> 172.21.15.16:0/2332117706 conn(0x7f9354097480 msgr2=0x7f9354097890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:05.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.737+0000 7f9359089700 1 -- 172.21.15.16:0/2332117706 shutdown_connections 2023-09-30T22:27:05.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.737+0000 7f9359089700 1 -- 172.21.15.16:0/2332117706 wait complete. 2023-09-30T22:27:05.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.738+0000 7f9359089700 1 Processor -- start 2023-09-30T22:27:05.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.738+0000 7f9359089700 1 -- start start 2023-09-30T22:27:05.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.739+0000 7f9359089700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9354093840 0x7f9354115d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:05.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.739+0000 7f9359089700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9354098c80 0x7f9354116270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:05.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.739+0000 7f9359089700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f935412c150 0x7f9354116760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:05.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.739+0000 7f9359089700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9354116c50 con 0x7f9354098c80 2023-09-30T22:27:05.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.739+0000 7f9359089700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93541171f0 con 0x7f9354093840 2023-09-30T22:27:05.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.739+0000 7f9359089700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9354119310 con 0x7f935412c150 2023-09-30T22:27:05.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.739+0000 7f9352d9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9354093840 0x7f9354115d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:05.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.739+0000 7f935259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9354098c80 0x7f9354116270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:05.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.739+0000 7f9352d9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9354093840 0x7f9354115d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:42476/0 (socket says 172.21.15.16:42476) 2023-09-30T22:27:05.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.739+0000 7f9352d9d700 1 -- 172.21.15.16:0/468050700 learned_addr learned my addr 172.21.15.16:0/468050700 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:05.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.739+0000 7f935259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9354098c80 0x7f9354116270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54736/0 (socket says 172.21.15.16:54736) 2023-09-30T22:27:05.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.739+0000 7f935359e700 1 --2- 172.21.15.16:0/468050700 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f935412c150 0x7f9354116760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:05.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.739+0000 7f935259c700 1 -- 172.21.15.16:0/468050700 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f935412c150 msgr2=0x7f9354116760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:05.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.739+0000 7f935259c700 1 --2- 172.21.15.16:0/468050700 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f935412c150 0x7f9354116760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:05.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.739+0000 7f935259c700 1 -- 172.21.15.16:0/468050700 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9354093840 msgr2=0x7f9354115d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:05.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.739+0000 7f935259c700 1 --2- 172.21.15.16:0/468050700 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9354093840 0x7f9354115d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:05.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.739+0000 7f935259c700 1 -- 172.21.15.16:0/468050700 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93440092a0 con 0x7f9354098c80 2023-09-30T22:27:05.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.739+0000 7f935359e700 1 --2- 172.21.15.16:0/468050700 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f935412c150 0x7f9354116760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:05.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.739+0000 7f9352d9d700 1 --2- 172.21.15.16:0/468050700 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9354093840 0x7f9354115d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:05.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.739+0000 7f935259c700 1 --2- 172.21.15.16:0/468050700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9354098c80 0x7f9354116270 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f9354094600 tx=0x7f933c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:05.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.740+0000 7f934affd700 1 -- 172.21.15.16:0/468050700 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933c006d90 con 0x7f9354098c80 2023-09-30T22:27:05.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.740+0000 7f934affd700 1 -- 172.21.15.16:0/468050700 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f933c006a40 con 0x7f9354098c80 2023-09-30T22:27:05.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.740+0000 7f9359089700 1 -- 172.21.15.16:0/468050700 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f933c009370 con 0x7f9354098c80 2023-09-30T22:27:05.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.740+0000 7f934affd700 1 -- 172.21.15.16:0/468050700 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933c00b850 con 0x7f9354098c80 2023-09-30T22:27:05.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.740+0000 7f9359089700 1 -- 172.21.15.16:0/468050700 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9354118fc0 con 0x7f9354098c80 2023-09-30T22:27:05.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.740+0000 7f934affd700 1 -- 172.21.15.16:0/468050700 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f933c01c630 con 0x7f9354098c80 2023-09-30T22:27:05.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.741+0000 7f934affd700 1 -- 172.21.15.16:0/468050700 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f933c03ce60 con 0x7f9354098c80 2023-09-30T22:27:05.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.741+0000 7f9359089700 1 -- 172.21.15.16:0/468050700 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93541197f0 con 0x7f9354098c80 2023-09-30T22:27:05.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.746+0000 7f934affd700 1 -- 172.21.15.16:0/468050700 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f93541197f0 con 0x7f9354098c80 2023-09-30T22:27:05.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.919+0000 7f9359089700 1 -- 172.21.15.16:0/468050700 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f93541197f0 con 0x7f9354098c80 2023-09-30T22:27:05.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.919+0000 7f934affd700 1 -- 172.21.15.16:0/468050700 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v16) v1 ==== 92+0+10 (secure 0 0 0) 0x7f93541197f0 con 0x7f9354098c80 2023-09-30T22:27:05.921 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 8 2023-09-30T22:27:05.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.921+0000 7f9359089700 1 -- 172.21.15.16:0/468050700 >> v1:172.21.15.16:6816/118285 conn(0x7f933402bb60 legacy=0x7f933402e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:05.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.921+0000 7f9359089700 1 -- 172.21.15.16:0/468050700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9354098c80 msgr2=0x7f9354116270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:05.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.921+0000 7f9359089700 1 --2- 172.21.15.16:0/468050700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9354098c80 0x7f9354116270 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f9354094600 tx=0x7f933c017000).stop 2023-09-30T22:27:05.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.921+0000 7f9359089700 1 -- 172.21.15.16:0/468050700 shutdown_connections 2023-09-30T22:27:05.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.921+0000 7f9359089700 1 --2- 172.21.15.16:0/468050700 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9354093840 0x7f9354115d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:05.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.921+0000 7f9359089700 1 --2- 172.21.15.16:0/468050700 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f935412c150 0x7f9354116760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:05.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.921+0000 7f9359089700 1 --2- 172.21.15.16:0/468050700 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9354098c80 0x7f9354116270 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:05.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.921+0000 7f9359089700 1 -- 172.21.15.16:0/468050700 >> 172.21.15.16:0/468050700 conn(0x7f9354097480 msgr2=0x7f9354068980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:05.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.921+0000 7f9359089700 1 -- 172.21.15.16:0/468050700 shutdown_connections 2023-09-30T22:27:05.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:05.922+0000 7f9359089700 1 -- 172.21.15.16:0/468050700 wait complete. 2023-09-30T22:27:05.936 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-30T22:27:06.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.067+0000 7fbb1f8c3700 1 Processor -- start 2023-09-30T22:27:06.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.067+0000 7fbb1f8c3700 1 -- start start 2023-09-30T22:27:06.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.067+0000 7fbb1f8c3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbb18098ea0 0x7fbb18099280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:06.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.067+0000 7fbb1f8c3700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbb1812d160 0x7fbb1812f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:06.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.067+0000 7fbb1f8c3700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb18092d80 con 0x7fbb18093840 2023-09-30T22:27:06.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.067+0000 7fbb1f8c3700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb18093c20 con 0x7fbb1812d160 2023-09-30T22:27:06.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.067+0000 7fbb1f8c3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb18092b90 con 0x7fbb18098ea0 2023-09-30T22:27:06.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.068+0000 7fbb1de60700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbb1812d160 0x7fbb1812f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:06.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.068+0000 7fbb1de60700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbb1812d160 0x7fbb1812f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54752/0 (socket says 172.21.15.16:54752) 2023-09-30T22:27:06.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.068+0000 7fbb1d65f700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fbb18093840 0x7fbb18098790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51556/0 (socket says 172.21.15.16:51556) 2023-09-30T22:27:06.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.068+0000 7fbb1de60700 1 -- 172.21.15.16:0/3899984780 learned_addr learned my addr 172.21.15.16:0/3899984780 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:06.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.068+0000 7fbb1ce5e700 1 --2- 172.21.15.16:0/3899984780 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbb18098ea0 0x7fbb18099280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:06.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.068+0000 7fbb1de60700 1 -- 172.21.15.16:0/3899984780 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbb18098ea0 msgr2=0x7fbb18099280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:06.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.068+0000 7fbb1de60700 1 --2- 172.21.15.16:0/3899984780 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbb18098ea0 0x7fbb18099280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:06.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.068+0000 7fbb1de60700 1 -- 172.21.15.16:0/3899984780 >> v1:172.21.15.16:6789/0 conn(0x7fbb18093840 legacy=0x7fbb18098790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:06.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.068+0000 7fbb0ffff700 1 -- 172.21.15.16:0/3899984780 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 849046655 0 0) 0x7fbb18092d80 con 0x7fbb18093840 2023-09-30T22:27:06.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.068+0000 7fbb1de60700 1 -- 172.21.15.16:0/3899984780 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb18099b70 con 0x7fbb1812d160 2023-09-30T22:27:06.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.068+0000 7fbb1ce5e700 1 --2- 172.21.15.16:0/3899984780 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbb18098ea0 0x7fbb18099280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:06.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.068+0000 7fbb1de60700 1 --2- 172.21.15.16:0/3899984780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb1812d160 0x7fbb1812f550 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7fbb10009f50 tx=0x7fbb10009840).ready entity=mon.0 client_cookie=4100ef3c44b2249e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:06.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.068+0000 7fbb0ffff700 1 -- 172.21.15.16:0/3899984780 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1000e070 con 0x7fbb1812d160 2023-09-30T22:27:06.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.068+0000 7fbb0ffff700 1 -- 172.21.15.16:0/3899984780 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb1000b9c0 con 0x7fbb1812d160 2023-09-30T22:27:06.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.068+0000 7fbb0ffff700 1 -- 172.21.15.16:0/3899984780 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1000bd40 con 0x7fbb1812d160 2023-09-30T22:27:06.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.069+0000 7fbb1f8c3700 1 -- 172.21.15.16:0/3899984780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb1812d160 msgr2=0x7fbb1812f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:06.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.069+0000 7fbb1f8c3700 1 --2- 172.21.15.16:0/3899984780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb1812d160 0x7fbb1812f550 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7fbb10009f50 tx=0x7fbb10009840).stop 2023-09-30T22:27:06.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.069+0000 7fbb1f8c3700 1 -- 172.21.15.16:0/3899984780 shutdown_connections 2023-09-30T22:27:06.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.069+0000 7fbb1f8c3700 1 --2- 172.21.15.16:0/3899984780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb1812d160 0x7fbb1812f550 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:06.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.069+0000 7fbb1f8c3700 1 --2- 172.21.15.16:0/3899984780 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbb18098ea0 0x7fbb18099280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:06.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.069+0000 7fbb1f8c3700 1 -- 172.21.15.16:0/3899984780 >> 172.21.15.16:0/3899984780 conn(0x7fbb18097480 msgr2=0x7fbb18097890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:06.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.069+0000 7fbb1f8c3700 1 -- 172.21.15.16:0/3899984780 shutdown_connections 2023-09-30T22:27:06.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.069+0000 7fbb1f8c3700 1 -- 172.21.15.16:0/3899984780 wait complete. 2023-09-30T22:27:06.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.070+0000 7fbb1f8c3700 1 Processor -- start 2023-09-30T22:27:06.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.070+0000 7fbb1f8c3700 1 -- start start 2023-09-30T22:27:06.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.071+0000 7fbb1f8c3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb18093840 0x7fbb1812bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:06.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.071+0000 7fbb1f8c3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbb18098ea0 0x7fbb1812c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:06.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.071+0000 7fbb1f8c3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbb1812d160 0x7fbb18116c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:06.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.071+0000 7fbb1f8c3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb18096c60 con 0x7fbb18093840 2023-09-30T22:27:06.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.071+0000 7fbb1f8c3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1806a520 con 0x7fbb1812d160 2023-09-30T22:27:06.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.071+0000 7fbb1f8c3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1811e3f0 con 0x7fbb18098ea0 2023-09-30T22:27:06.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.071+0000 7fbb1d65f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb18093840 0x7fbb1812bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:06.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.071+0000 7fbb1ce5e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbb18098ea0 0x7fbb1812c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:06.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.071+0000 7fbb1d65f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb18093840 0x7fbb1812bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54758/0 (socket says 172.21.15.16:54758) 2023-09-30T22:27:06.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.071+0000 7fbb1d65f700 1 -- 172.21.15.16:0/2534807910 learned_addr learned my addr 172.21.15.16:0/2534807910 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:06.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.071+0000 7fbb1de60700 1 --2- 172.21.15.16:0/2534807910 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbb1812d160 0x7fbb18116c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:06.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.072+0000 7fbb1ce5e700 1 -- 172.21.15.16:0/2534807910 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbb1812d160 msgr2=0x7fbb18116c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:06.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.072+0000 7fbb1ce5e700 1 --2- 172.21.15.16:0/2534807910 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbb1812d160 0x7fbb18116c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:06.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.072+0000 7fbb1ce5e700 1 -- 172.21.15.16:0/2534807910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb18093840 msgr2=0x7fbb1812bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:06.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.072+0000 7fbb1ce5e700 1 --2- 172.21.15.16:0/2534807910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb18093840 0x7fbb1812bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:06.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.072+0000 7fbb1ce5e700 1 -- 172.21.15.16:0/2534807910 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb080092a0 con 0x7fbb18098ea0 2023-09-30T22:27:06.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.072+0000 7fbb1d65f700 1 --2- 172.21.15.16:0/2534807910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb18093840 0x7fbb1812bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:06.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.072+0000 7fbb1de60700 1 --2- 172.21.15.16:0/2534807910 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbb1812d160 0x7fbb18116c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:06.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.072+0000 7fbb1ce5e700 1 --2- 172.21.15.16:0/2534807910 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbb18098ea0 0x7fbb1812c0b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fbb0800e980 tx=0x7fbb08009060).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:06.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.072+0000 7fbb0d7fa700 1 -- 172.21.15.16:0/2534807910 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb08009580 con 0x7fbb18098ea0 2023-09-30T22:27:06.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.072+0000 7fbb0d7fa700 1 -- 172.21.15.16:0/2534807910 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb08012600 con 0x7fbb18098ea0 2023-09-30T22:27:06.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.072+0000 7fbb1f8c3700 1 -- 172.21.15.16:0/2534807910 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb10009ae0 con 0x7fbb18098ea0 2023-09-30T22:27:06.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.072+0000 7fbb0d7fa700 1 -- 172.21.15.16:0/2534807910 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb08012980 con 0x7fbb18098ea0 2023-09-30T22:27:06.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.072+0000 7fbb1f8c3700 1 -- 172.21.15.16:0/2534807910 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb18059230 con 0x7fbb18098ea0 2023-09-30T22:27:06.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.073+0000 7fbb0d7fa700 1 -- 172.21.15.16:0/2534807910 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fbb08012d30 con 0x7fbb18098ea0 2023-09-30T22:27:06.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.073+0000 7fbb0d7fa700 1 -- 172.21.15.16:0/2534807910 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fbb0803b370 con 0x7fbb18098ea0 2023-09-30T22:27:06.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.073+0000 7fbb1f8c3700 1 -- 172.21.15.16:0/2534807910 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb18096a00 con 0x7fbb18098ea0 2023-09-30T22:27:06.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.078+0000 7fbb0d7fa700 1 -- 172.21.15.16:0/2534807910 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fbb08016070 con 0x7fbb18098ea0 2023-09-30T22:27:06.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.251+0000 7fbb1f8c3700 1 -- 172.21.15.16:0/2534807910 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbb1805a260 con 0x7fbb18098ea0 2023-09-30T22:27:06.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.251+0000 7fbb0d7fa700 1 -- 172.21.15.16:0/2534807910 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v16) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbb080290a0 con 0x7fbb18098ea0 2023-09-30T22:27:06.253 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-09-30T22:27:06.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.253+0000 7fbb1f8c3700 1 -- 172.21.15.16:0/2534807910 >> v1:172.21.15.16:6816/118285 conn(0x7fbafc02bb60 legacy=0x7fbafc02e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:06.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.253+0000 7fbb1f8c3700 1 -- 172.21.15.16:0/2534807910 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbb18098ea0 msgr2=0x7fbb1812c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:06.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.253+0000 7fbb1f8c3700 1 --2- 172.21.15.16:0/2534807910 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbb18098ea0 0x7fbb1812c0b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fbb0800e980 tx=0x7fbb08009060).stop 2023-09-30T22:27:06.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.253+0000 7fbb1f8c3700 1 -- 172.21.15.16:0/2534807910 shutdown_connections 2023-09-30T22:27:06.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.253+0000 7fbb1f8c3700 1 --2- 172.21.15.16:0/2534807910 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbb1812d160 0x7fbb18116c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:06.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.253+0000 7fbb1f8c3700 1 --2- 172.21.15.16:0/2534807910 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb18093840 0x7fbb1812bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:06.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.253+0000 7fbb1f8c3700 1 --2- 172.21.15.16:0/2534807910 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbb18098ea0 0x7fbb1812c0b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:06.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.253+0000 7fbb1f8c3700 1 -- 172.21.15.16:0/2534807910 >> 172.21.15.16:0/2534807910 conn(0x7fbb18097480 msgr2=0x7fbb180693c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:06.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.254+0000 7fbb1f8c3700 1 -- 172.21.15.16:0/2534807910 shutdown_connections 2023-09-30T22:27:06.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.254+0000 7fbb1f8c3700 1 -- 172.21.15.16:0/2534807910 wait complete. 2023-09-30T22:27:06.268 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-30T22:27:06.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.400+0000 7f8ece5c4700 1 Processor -- start 2023-09-30T22:27:06.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.400+0000 7f8ece5c4700 1 -- start start 2023-09-30T22:27:06.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.400+0000 7f8ece5c4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8ec80613a0 0x7f8ec812ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:06.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.400+0000 7f8ece5c4700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8ec812b610 0x7f8ec8069430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:06.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.400+0000 7f8ece5c4700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ec80985c0 con 0x7f8ec812b230 2023-09-30T22:27:06.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.400+0000 7f8ece5c4700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ec8061780 con 0x7f8ec80613a0 2023-09-30T22:27:06.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.400+0000 7f8ece5c4700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec8091dd0 con 0x7f8ec812b610 2023-09-30T22:27:06.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.400+0000 7f8ec7fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8ec80613a0 0x7f8ec812ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:06.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.400+0000 7f8eccb61700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8ec812b610 0x7f8ec8069430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:06.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.400+0000 7f8ec77fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8ec812b230 0x7f8ec8068d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51564/0 (socket says 172.21.15.16:51564) 2023-09-30T22:27:06.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.400+0000 7f8ec7fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8ec80613a0 0x7f8ec812ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54764/0 (socket says 172.21.15.16:54764) 2023-09-30T22:27:06.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.400+0000 7f8ec77fe700 1 -- 172.21.15.16:0/3524578807 learned_addr learned my addr 172.21.15.16:0/3524578807 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:27:06.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.401+0000 7f8ec7fff700 1 -- 172.21.15.16:0/3524578807 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8ec812b610 msgr2=0x7f8ec8069430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:06.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.401+0000 7f8ec7fff700 1 --2- 172.21.15.16:0/3524578807 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8ec812b610 0x7f8ec8069430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:06.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.401+0000 7f8ec7fff700 1 -- 172.21.15.16:0/3524578807 >> v1:172.21.15.16:6789/0 conn(0x7f8ec812b230 legacy=0x7f8ec8068d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:06.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.401+0000 7f8ec7fff700 1 -- 172.21.15.16:0/3524578807 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ec80996a0 con 0x7f8ec80613a0 2023-09-30T22:27:06.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.401+0000 7f8ec6ffd700 1 -- 172.21.15.16:0/3524578807 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1252823765 0 0) 0x7f8ec80985c0 con 0x7f8ec812b230 2023-09-30T22:27:06.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.401+0000 7f8eccb61700 1 --2- 172.21.15.16:0/3524578807 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8ec812b610 0x7f8ec8069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:06.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.401+0000 7f8ec7fff700 1 --2- 172.21.15.16:0/3524578807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ec80613a0 0x7f8ec812ad40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f8eb8009760 tx=0x7f8eb8009060).ready entity=mon.0 client_cookie=688ef564e0e751b1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:06.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.401+0000 7f8ec6ffd700 1 -- 172.21.15.16:0/3524578807 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb800f510 con 0x7f8ec80613a0 2023-09-30T22:27:06.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.401+0000 7f8ec6ffd700 1 -- 172.21.15.16:0/3524578807 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eb800f670 con 0x7f8ec80613a0 2023-09-30T22:27:06.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.401+0000 7f8ec6ffd700 1 -- 172.21.15.16:0/3524578807 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb800f9f0 con 0x7f8ec80613a0 2023-09-30T22:27:06.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.402+0000 7f8ece5c4700 1 -- 172.21.15.16:0/3524578807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ec80613a0 msgr2=0x7f8ec812ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:06.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.402+0000 7f8ece5c4700 1 --2- 172.21.15.16:0/3524578807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ec80613a0 0x7f8ec812ad40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f8eb8009760 tx=0x7f8eb8009060).stop 2023-09-30T22:27:06.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.402+0000 7f8ece5c4700 1 -- 172.21.15.16:0/3524578807 shutdown_connections 2023-09-30T22:27:06.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.402+0000 7f8ece5c4700 1 --2- 172.21.15.16:0/3524578807 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8ec812b610 0x7f8ec8069430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:06.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.402+0000 7f8ece5c4700 1 --2- 172.21.15.16:0/3524578807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ec80613a0 0x7f8ec812ad40 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:06.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.402+0000 7f8ece5c4700 1 -- 172.21.15.16:0/3524578807 >> 172.21.15.16:0/3524578807 conn(0x7f8ec809a1c0 msgr2=0x7f8ec809a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:06.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.402+0000 7f8ece5c4700 1 -- 172.21.15.16:0/3524578807 shutdown_connections 2023-09-30T22:27:06.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.402+0000 7f8ece5c4700 1 -- 172.21.15.16:0/3524578807 wait complete. 2023-09-30T22:27:06.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.403+0000 7f8ece5c4700 1 Processor -- start 2023-09-30T22:27:06.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.403+0000 7f8ece5c4700 1 -- start start 2023-09-30T22:27:06.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.403+0000 7f8ece5c4700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8ec80613a0 0x7f8ec813a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:06.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.403+0000 7f8ece5c4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8ec812b230 0x7f8ec813a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:06.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.403+0000 7f8ece5c4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ec812b610 0x7f8ec8115b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:06.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.403+0000 7f8ece5c4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec8068630 con 0x7f8ec812b610 2023-09-30T22:27:06.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.403+0000 7f8ece5c4700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec812f190 con 0x7f8ec80613a0 2023-09-30T22:27:06.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.403+0000 7f8ece5c4700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec8117840 con 0x7f8ec812b230 2023-09-30T22:27:06.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.403+0000 7f8ec7fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8ec80613a0 0x7f8ec813a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:06.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.404+0000 7f8ec7fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8ec80613a0 0x7f8ec813a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:42520/0 (socket says 172.21.15.16:42520) 2023-09-30T22:27:06.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.404+0000 7f8ec7fff700 1 -- 172.21.15.16:0/1748592616 learned_addr learned my addr 172.21.15.16:0/1748592616 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:06.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.404+0000 7f8eccb61700 1 --2- 172.21.15.16:0/1748592616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ec812b610 0x7f8ec8115b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:06.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.404+0000 7f8ec77fe700 1 --2- 172.21.15.16:0/1748592616 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8ec812b230 0x7f8ec813a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:06.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.404+0000 7f8ec7fff700 1 -- 172.21.15.16:0/1748592616 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8ec812b230 msgr2=0x7f8ec813a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:06.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.404+0000 7f8ec7fff700 1 --2- 172.21.15.16:0/1748592616 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8ec812b230 0x7f8ec813a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:06.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.404+0000 7f8ec7fff700 1 -- 172.21.15.16:0/1748592616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ec812b610 msgr2=0x7f8ec8115b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:06.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.404+0000 7f8ec7fff700 1 --2- 172.21.15.16:0/1748592616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ec812b610 0x7f8ec8115b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:06.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.404+0000 7f8ec7fff700 1 -- 172.21.15.16:0/1748592616 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ec0009a20 con 0x7f8ec80613a0 2023-09-30T22:27:06.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.404+0000 7f8eccb61700 1 --2- 172.21.15.16:0/1748592616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ec812b610 0x7f8ec8115b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:06.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.404+0000 7f8ec7fff700 1 --2- 172.21.15.16:0/1748592616 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8ec80613a0 0x7f8ec813a440 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f8eb8006d90 tx=0x7f8eb8017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:06.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.404+0000 7f8eaffff700 1 -- 172.21.15.16:0/1748592616 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb800f7e0 con 0x7f8ec80613a0 2023-09-30T22:27:06.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.404+0000 7f8eaffff700 1 -- 172.21.15.16:0/1748592616 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eb800f940 con 0x7f8ec80613a0 2023-09-30T22:27:06.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.404+0000 7f8ece5c4700 1 -- 172.21.15.16:0/1748592616 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8eb80092a0 con 0x7f8ec80613a0 2023-09-30T22:27:06.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.404+0000 7f8eaffff700 1 -- 172.21.15.16:0/1748592616 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb801b620 con 0x7f8ec80613a0 2023-09-30T22:27:06.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.404+0000 7f8ece5c4700 1 -- 172.21.15.16:0/1748592616 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ec8119d90 con 0x7f8ec80613a0 2023-09-30T22:27:06.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.405+0000 7f8eaffff700 1 -- 172.21.15.16:0/1748592616 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f8eb801ba10 con 0x7f8ec80613a0 2023-09-30T22:27:06.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.405+0000 7f8eaffff700 1 -- 172.21.15.16:0/1748592616 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f8eb803ca70 con 0x7f8ec80613a0 2023-09-30T22:27:06.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.405+0000 7f8ece5c4700 1 -- 172.21.15.16:0/1748592616 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ec80930d0 con 0x7f8ec80613a0 2023-09-30T22:27:06.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.410+0000 7f8eaffff700 1 -- 172.21.15.16:0/1748592616 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f8eb8006a40 con 0x7f8ec80613a0 2023-09-30T22:27:06.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.584+0000 7f8ece5c4700 1 -- 172.21.15.16:0/1748592616 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8ec805a260 con 0x7f8ec80613a0 2023-09-30T22:27:06.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.585+0000 7f8eaffff700 1 -- 172.21.15.16:0/1748592616 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v16) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8eb8019b70 con 0x7f8ec80613a0 2023-09-30T22:27:06.586 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-09-30T22:27:06.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.587+0000 7f8ece5c4700 1 -- 172.21.15.16:0/1748592616 >> v1:172.21.15.16:6816/118285 conn(0x7f8eb402bb90 legacy=0x7f8eb402e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:06.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.587+0000 7f8ece5c4700 1 -- 172.21.15.16:0/1748592616 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8ec80613a0 msgr2=0x7f8ec813a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:06.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.587+0000 7f8ece5c4700 1 --2- 172.21.15.16:0/1748592616 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8ec80613a0 0x7f8ec813a440 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f8eb8006d90 tx=0x7f8eb8017000).stop 2023-09-30T22:27:06.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.587+0000 7f8ece5c4700 1 -- 172.21.15.16:0/1748592616 shutdown_connections 2023-09-30T22:27:06.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.587+0000 7f8ece5c4700 1 --2- 172.21.15.16:0/1748592616 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8ec80613a0 0x7f8ec813a440 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:06.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.587+0000 7f8ece5c4700 1 --2- 172.21.15.16:0/1748592616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ec812b610 0x7f8ec8115b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:06.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.587+0000 7f8ece5c4700 1 --2- 172.21.15.16:0/1748592616 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8ec812b230 0x7f8ec813a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:06.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.587+0000 7f8ece5c4700 1 -- 172.21.15.16:0/1748592616 >> 172.21.15.16:0/1748592616 conn(0x7f8ec809a1c0 msgr2=0x7f8ec812df70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:06.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.587+0000 7f8ece5c4700 1 -- 172.21.15.16:0/1748592616 shutdown_connections 2023-09-30T22:27:06.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.587+0000 7f8ece5c4700 1 -- 172.21.15.16:0/1748592616 wait complete. 2023-09-30T22:27:06.601 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-30T22:27:06.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.729+0000 7f7eba2e4700 1 Processor -- start 2023-09-30T22:27:06.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.730+0000 7f7eba2e4700 1 -- start start 2023-09-30T22:27:06.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.730+0000 7f7eba2e4700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7eb4059070 0x7f7eb4127890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:06.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.730+0000 7f7eba2e4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7eb4127390 0x7f7eb4127d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:06.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.730+0000 7f7eba2e4700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7eb4091b60 con 0x7f7eb4095370 2023-09-30T22:27:06.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.730+0000 7f7eba2e4700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7eb4059450 con 0x7f7eb4127390 2023-09-30T22:27:06.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.730+0000 7f7eba2e4700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb40968c0 con 0x7f7eb4059070 2023-09-30T22:27:06.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.730+0000 7f7eb3fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7eb4059070 0x7f7eb4127890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:06.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.730+0000 7f7eb37fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7eb4127390 0x7f7eb4127d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:06.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.730+0000 7f7eb3fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7eb4059070 0x7f7eb4127890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:42524/0 (socket says 172.21.15.16:42524) 2023-09-30T22:27:06.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.730+0000 7f7eb3fff700 1 -- 172.21.15.16:0/1063756122 learned_addr learned my addr 172.21.15.16:0/1063756122 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:06.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.731+0000 7f7eb3fff700 1 -- 172.21.15.16:0/1063756122 >> v2:172.21.15.16:3300/0 conn(0x7f7eb4127390 msgr2=0x7f7eb4127d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:06.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.731+0000 7f7eb3fff700 1 --2- 172.21.15.16:0/1063756122 >> v2:172.21.15.16:3300/0 conn(0x7f7eb4127390 0x7f7eb4127d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:06.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.731+0000 7f7eb3fff700 1 -- 172.21.15.16:0/1063756122 >> v1:172.21.15.16:6789/0 conn(0x7f7eb4095370 legacy=0x7f7eb4095820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:06.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.731+0000 7f7eb3fff700 1 -- 172.21.15.16:0/1063756122 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eb4092a10 con 0x7f7eb4059070 2023-09-30T22:27:06.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.731+0000 7f7eb37fe700 1 --2- 172.21.15.16:0/1063756122 >> v2:172.21.15.16:3300/0 conn(0x7f7eb4127390 0x7f7eb4127d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:06.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.731+0000 7f7eb3fff700 1 --2- 172.21.15.16:0/1063756122 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7eb4059070 0x7f7eb4127890 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f7ea4009830 tx=0x7f7ea40090f0).ready entity=mon.1 client_cookie=37d64c97cd24a883 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:06.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.731+0000 7f7eb2ffd700 1 -- 172.21.15.16:0/1063756122 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea400b580 con 0x7f7eb4059070 2023-09-30T22:27:06.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.731+0000 7f7eb2ffd700 1 -- 172.21.15.16:0/1063756122 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ea400b6e0 con 0x7f7eb4059070 2023-09-30T22:27:06.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.731+0000 7f7eb2ffd700 1 -- 172.21.15.16:0/1063756122 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea400ba80 con 0x7f7eb4059070 2023-09-30T22:27:06.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.732+0000 7f7eba2e4700 1 -- 172.21.15.16:0/1063756122 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7eb4059070 msgr2=0x7f7eb4127890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:06.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.732+0000 7f7eba2e4700 1 --2- 172.21.15.16:0/1063756122 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7eb4059070 0x7f7eb4127890 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f7ea4009830 tx=0x7f7ea40090f0).stop 2023-09-30T22:27:06.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.732+0000 7f7eba2e4700 1 -- 172.21.15.16:0/1063756122 shutdown_connections 2023-09-30T22:27:06.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.732+0000 7f7eba2e4700 1 --2- 172.21.15.16:0/1063756122 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7eb4059070 0x7f7eb4127890 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:06.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.732+0000 7f7eba2e4700 1 --2- 172.21.15.16:0/1063756122 >> v2:172.21.15.16:3300/0 conn(0x7f7eb4127390 0x7f7eb4127d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:06.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.732+0000 7f7eba2e4700 1 -- 172.21.15.16:0/1063756122 >> 172.21.15.16:0/1063756122 conn(0x7f7eb4097470 msgr2=0x7f7eb4097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:06.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.732+0000 7f7eba2e4700 1 -- 172.21.15.16:0/1063756122 shutdown_connections 2023-09-30T22:27:06.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.732+0000 7f7eba2e4700 1 -- 172.21.15.16:0/1063756122 wait complete. 2023-09-30T22:27:06.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.732+0000 7f7eba2e4700 1 Processor -- start 2023-09-30T22:27:06.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.733+0000 7f7eba2e4700 1 -- start start 2023-09-30T22:27:06.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.733+0000 7f7eba2e4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7eb4059070 0x7f7eb4113a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:06.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.733+0000 7f7eba2e4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7eb4095370 0x7f7eb4113f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:06.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.733+0000 7f7eba2e4700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7eb4127390 0x7f7eb4114440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:06.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.733+0000 7f7eba2e4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb4124ae0 con 0x7f7eb4059070 2023-09-30T22:27:06.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.733+0000 7f7eba2e4700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb4096ea0 con 0x7f7eb4127390 2023-09-30T22:27:06.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.733+0000 7f7eba2e4700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb4116f10 con 0x7f7eb4095370 2023-09-30T22:27:06.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.733+0000 7f7eb3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7eb4059070 0x7f7eb4113a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:06.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.733+0000 7f7eb3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7eb4059070 0x7f7eb4113a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54800/0 (socket says 172.21.15.16:54800) 2023-09-30T22:27:06.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.733+0000 7f7eb3fff700 1 -- 172.21.15.16:0/2124055748 learned_addr learned my addr 172.21.15.16:0/2124055748 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:06.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.733+0000 7f7eb37fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7eb4095370 0x7f7eb4113f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:06.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.733+0000 7f7eb8881700 1 --2- 172.21.15.16:0/2124055748 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7eb4127390 0x7f7eb4114440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:06.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.734+0000 7f7eb3fff700 1 -- 172.21.15.16:0/2124055748 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7eb4095370 msgr2=0x7f7eb4113f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:06.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.734+0000 7f7eb3fff700 1 --2- 172.21.15.16:0/2124055748 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7eb4095370 0x7f7eb4113f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:06.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.734+0000 7f7eb3fff700 1 -- 172.21.15.16:0/2124055748 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7eb4127390 msgr2=0x7f7eb4114440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:06.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.734+0000 7f7eb3fff700 1 --2- 172.21.15.16:0/2124055748 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7eb4127390 0x7f7eb4114440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:06.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.734+0000 7f7eb3fff700 1 -- 172.21.15.16:0/2124055748 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea80092a0 con 0x7f7eb4059070 2023-09-30T22:27:06.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.734+0000 7f7eb3fff700 1 --2- 172.21.15.16:0/2124055748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7eb4059070 0x7f7eb4113a60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f7eb4092390 tx=0x7f7ea4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:06.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.734+0000 7f7e9bfff700 1 -- 172.21.15.16:0/2124055748 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea4006d90 con 0x7f7eb4059070 2023-09-30T22:27:06.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.734+0000 7f7e9bfff700 1 -- 172.21.15.16:0/2124055748 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ea400be40 con 0x7f7eb4059070 2023-09-30T22:27:06.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.734+0000 7f7e9bfff700 1 -- 172.21.15.16:0/2124055748 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea401c420 con 0x7f7eb4059070 2023-09-30T22:27:06.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.734+0000 7f7eba2e4700 1 -- 172.21.15.16:0/2124055748 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ea4009370 con 0x7f7eb4059070 2023-09-30T22:27:06.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.734+0000 7f7eba2e4700 1 -- 172.21.15.16:0/2124055748 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7eb4117b90 con 0x7f7eb4059070 2023-09-30T22:27:06.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.735+0000 7f7e9bfff700 1 -- 172.21.15.16:0/2124055748 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f7ea401c790 con 0x7f7eb4059070 2023-09-30T22:27:06.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.735+0000 7f7e9bfff700 1 -- 172.21.15.16:0/2124055748 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f7ea403cc30 con 0x7f7eb4059070 2023-09-30T22:27:06.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.735+0000 7f7eba2e4700 1 -- 172.21.15.16:0/2124055748 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7eb4091d80 con 0x7f7eb4059070 2023-09-30T22:27:06.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.740+0000 7f7e9bfff700 1 -- 172.21.15.16:0/2124055748 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f7ea402a020 con 0x7f7eb4059070 2023-09-30T22:27:06.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.955+0000 7f7eba2e4700 1 -- 172.21.15.16:0/2124055748 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7eb4129220 con 0x7f7eb4059070 2023-09-30T22:27:06.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.955+0000 7f7e9bfff700 1 -- 172.21.15.16:0/2124055748 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+2156 (secure 0 0 0) 0x7f7ea4034090 con 0x7f7eb4059070 2023-09-30T22:27:06.957 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:27:06.957 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-30T22:26:51.674561+0000","modified":"2023-09-30T22:26:54.680606+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4218},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4218":{"gid":4218,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6817/1424366610","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6817","nonce":1424366610}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-09-30T22:27:06.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.957+0000 7f7eba2e4700 1 -- 172.21.15.16:0/2124055748 >> v1:172.21.15.16:6816/118285 conn(0x7f7ea002bb60 legacy=0x7f7ea002e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:06.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.957+0000 7f7eba2e4700 1 -- 172.21.15.16:0/2124055748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7eb4059070 msgr2=0x7f7eb4113a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:06.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.957+0000 7f7eba2e4700 1 --2- 172.21.15.16:0/2124055748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7eb4059070 0x7f7eb4113a60 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f7eb4092390 tx=0x7f7ea4017000).stop 2023-09-30T22:27:06.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.957+0000 7f7eba2e4700 1 -- 172.21.15.16:0/2124055748 shutdown_connections 2023-09-30T22:27:06.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.957+0000 7f7eba2e4700 1 --2- 172.21.15.16:0/2124055748 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7eb4127390 0x7f7eb4114440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:06.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.957+0000 7f7eba2e4700 1 --2- 172.21.15.16:0/2124055748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7eb4059070 0x7f7eb4113a60 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:06.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.957+0000 7f7eba2e4700 1 --2- 172.21.15.16:0/2124055748 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7eb4095370 0x7f7eb4113f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:06.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.957+0000 7f7eba2e4700 1 -- 172.21.15.16:0/2124055748 >> 172.21.15.16:0/2124055748 conn(0x7f7eb4097470 msgr2=0x7f7eb4129750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:06.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.957+0000 7f7eba2e4700 1 -- 172.21.15.16:0/2124055748 shutdown_connections 2023-09-30T22:27:06.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:06.957+0000 7f7eba2e4700 1 -- 172.21.15.16:0/2124055748 wait complete. 2023-09-30T22:27:06.960 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 6 2023-09-30T22:27:06.972 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:27:07.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.099+0000 7f2ad78ed700 1 Processor -- start 2023-09-30T22:27:07.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.100+0000 7f2ad78ed700 1 -- start start 2023-09-30T22:27:07.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.100+0000 7f2ad78ed700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2ad012b510 0x7f2ad012ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:07.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.100+0000 7f2ad78ed700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2ad01303f0 0x7f2ad01327e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:07.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.100+0000 7f2ad78ed700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ad012abd0 con 0x7f2ad00613a0 2023-09-30T22:27:07.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.100+0000 7f2ad78ed700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ad0061780 con 0x7f2ad012b510 2023-09-30T22:27:07.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.100+0000 7f2ad78ed700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad0067530 con 0x7f2ad01303f0 2023-09-30T22:27:07.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.100+0000 7f2ad4e88700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2ad012b510 0x7f2ad012ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:07.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.100+0000 7f2ad5e8a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2ad01303f0 0x7f2ad01327e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:07.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.100+0000 7f2ad4e88700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2ad012b510 0x7f2ad012ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54816/0 (socket says 172.21.15.16:54816) 2023-09-30T22:27:07.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.100+0000 7f2ad4e88700 1 -- 172.21.15.16:0/3248175718 learned_addr learned my addr 172.21.15.16:0/3248175718 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:07.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.100+0000 7f2ad5689700 1 --1- 172.21.15.16:0/3248175718 >> v1:172.21.15.16:6789/0 conn(0x7f2ad00613a0 0x7f2ad012ae00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51582/0 (socket says 172.21.15.16:51582) 2023-09-30T22:27:07.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.101+0000 7f2ad5e8a700 1 -- 172.21.15.16:0/3248175718 >> v2:172.21.15.16:3300/0 conn(0x7f2ad012b510 msgr2=0x7f2ad012ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:07.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.101+0000 7f2ad5e8a700 1 --2- 172.21.15.16:0/3248175718 >> v2:172.21.15.16:3300/0 conn(0x7f2ad012b510 0x7f2ad012ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:07.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.101+0000 7f2ad5e8a700 1 -- 172.21.15.16:0/3248175718 >> v1:172.21.15.16:6789/0 conn(0x7f2ad00613a0 legacy=0x7f2ad012ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:07.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.101+0000 7f2ac7fff700 1 -- 172.21.15.16:0/3248175718 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3881357409 0 0) 0x7f2ad012abd0 con 0x7f2ad00613a0 2023-09-30T22:27:07.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.101+0000 7f2ad5e8a700 1 -- 172.21.15.16:0/3248175718 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ad00683c0 con 0x7f2ad01303f0 2023-09-30T22:27:07.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.101+0000 7f2ad4e88700 1 --2- 172.21.15.16:0/3248175718 >> v2:172.21.15.16:3300/0 conn(0x7f2ad012b510 0x7f2ad012ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:07.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.101+0000 7f2ad5e8a700 1 --2- 172.21.15.16:0/3248175718 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2ad01303f0 0x7f2ad01327e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f2ac8009e90 tx=0x7f2ac8009780).ready entity=mon.1 client_cookie=2d1ca668337003a2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:07.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.101+0000 7f2ac7fff700 1 -- 172.21.15.16:0/3248175718 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac800e070 con 0x7f2ad01303f0 2023-09-30T22:27:07.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.101+0000 7f2ac7fff700 1 -- 172.21.15.16:0/3248175718 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ac800bb20 con 0x7f2ad01303f0 2023-09-30T22:27:07.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.101+0000 7f2ac7fff700 1 -- 172.21.15.16:0/3248175718 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac800bea0 con 0x7f2ad01303f0 2023-09-30T22:27:07.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.102+0000 7f2ad78ed700 1 -- 172.21.15.16:0/3248175718 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2ad01303f0 msgr2=0x7f2ad01327e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:07.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.102+0000 7f2ad78ed700 1 --2- 172.21.15.16:0/3248175718 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2ad01303f0 0x7f2ad01327e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f2ac8009e90 tx=0x7f2ac8009780).stop 2023-09-30T22:27:07.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.102+0000 7f2ad78ed700 1 -- 172.21.15.16:0/3248175718 shutdown_connections 2023-09-30T22:27:07.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.102+0000 7f2ad78ed700 1 --2- 172.21.15.16:0/3248175718 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2ad01303f0 0x7f2ad01327e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:07.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.102+0000 7f2ad78ed700 1 --2- 172.21.15.16:0/3248175718 >> v2:172.21.15.16:3300/0 conn(0x7f2ad012b510 0x7f2ad012ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:07.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.102+0000 7f2ad78ed700 1 -- 172.21.15.16:0/3248175718 >> 172.21.15.16:0/3248175718 conn(0x7f2ad009a220 msgr2=0x7f2ad009a630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:07.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.102+0000 7f2ad78ed700 1 -- 172.21.15.16:0/3248175718 shutdown_connections 2023-09-30T22:27:07.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.102+0000 7f2ad78ed700 1 -- 172.21.15.16:0/3248175718 wait complete. 2023-09-30T22:27:07.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.103+0000 7f2ad78ed700 1 Processor -- start 2023-09-30T22:27:07.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.103+0000 7f2ad78ed700 1 -- start start 2023-09-30T22:27:07.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.103+0000 7f2ad78ed700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2ad00613a0 0x7f2ad012a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:07.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.103+0000 7f2ad78ed700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2ad012b510 0x7f2ad01145b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:07.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.103+0000 7f2ad78ed700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ad01303f0 0x7f2ad0114aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:07.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.103+0000 7f2ad78ed700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad0067130 con 0x7f2ad01303f0 2023-09-30T22:27:07.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.103+0000 7f2ad78ed700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad0118620 con 0x7f2ad00613a0 2023-09-30T22:27:07.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.103+0000 7f2ad78ed700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad01159b0 con 0x7f2ad012b510 2023-09-30T22:27:07.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.103+0000 7f2ad5689700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2ad00613a0 0x7f2ad012a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:07.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.103+0000 7f2ad5e8a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ad01303f0 0x7f2ad0114aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:07.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.103+0000 7f2ad4e88700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2ad012b510 0x7f2ad01145b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:07.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.103+0000 7f2ad5e8a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ad01303f0 0x7f2ad0114aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54826/0 (socket says 172.21.15.16:54826) 2023-09-30T22:27:07.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.103+0000 7f2ad5689700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2ad00613a0 0x7f2ad012a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:42538/0 (socket says 172.21.15.16:42538) 2023-09-30T22:27:07.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.104+0000 7f2ad5e8a700 1 -- 172.21.15.16:0/3508590664 learned_addr learned my addr 172.21.15.16:0/3508590664 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:07.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.104+0000 7f2ad5e8a700 1 -- 172.21.15.16:0/3508590664 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2ad012b510 msgr2=0x7f2ad01145b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:07.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.104+0000 7f2ad5e8a700 1 --2- 172.21.15.16:0/3508590664 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2ad012b510 0x7f2ad01145b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:07.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.104+0000 7f2ad5e8a700 1 -- 172.21.15.16:0/3508590664 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2ad00613a0 msgr2=0x7f2ad012a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:07.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.104+0000 7f2ad5e8a700 1 --2- 172.21.15.16:0/3508590664 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2ad00613a0 0x7f2ad012a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:07.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.104+0000 7f2ad5e8a700 1 -- 172.21.15.16:0/3508590664 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ac0009370 con 0x7f2ad01303f0 2023-09-30T22:27:07.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.104+0000 7f2ad5e8a700 1 --2- 172.21.15.16:0/3508590664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ad01303f0 0x7f2ad0114aa0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f2ac8011000 tx=0x7f2ac8009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:07.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.104+0000 7f2ac57fa700 1 -- 172.21.15.16:0/3508590664 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac8009440 con 0x7f2ad01303f0 2023-09-30T22:27:07.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.104+0000 7f2ac57fa700 1 -- 172.21.15.16:0/3508590664 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ac801d630 con 0x7f2ad01303f0 2023-09-30T22:27:07.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.104+0000 7f2ad78ed700 1 -- 172.21.15.16:0/3508590664 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ac8009a20 con 0x7f2ad01303f0 2023-09-30T22:27:07.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.104+0000 7f2ac57fa700 1 -- 172.21.15.16:0/3508590664 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac801d9f0 con 0x7f2ad01303f0 2023-09-30T22:27:07.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.104+0000 7f2ad78ed700 1 -- 172.21.15.16:0/3508590664 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ad01168d0 con 0x7f2ad01303f0 2023-09-30T22:27:07.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.105+0000 7f2ac57fa700 1 -- 172.21.15.16:0/3508590664 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f2ac8003040 con 0x7f2ad01303f0 2023-09-30T22:27:07.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.105+0000 7f2ac57fa700 1 -- 172.21.15.16:0/3508590664 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f2ac803cdc0 con 0x7f2ad01303f0 2023-09-30T22:27:07.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.105+0000 7f2ad78ed700 1 -- 172.21.15.16:0/3508590664 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ad0117be0 con 0x7f2ad01303f0 2023-09-30T22:27:07.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.110+0000 7f2ac57fa700 1 -- 172.21.15.16:0/3508590664 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f2ad0117be0 con 0x7f2ad01303f0 2023-09-30T22:27:07.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.287+0000 7f2ad78ed700 1 -- 172.21.15.16:0/3508590664 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2ad0117be0 con 0x7f2ad01303f0 2023-09-30T22:27:07.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.287+0000 7f2ac57fa700 1 -- 172.21.15.16:0/3508590664 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+11341 (secure 0 0 0) 0x7f2ad0117be0 con 0x7f2ad01303f0 2023-09-30T22:27:07.289 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:27:07.289 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":16,"fsid":"7ede3e14-2bd0-4726-9437-6d13c90cd342","created":"2023-09-30T22:26:35.483697+0000","modified":"2023-09-30T22:26:52.675888+0000","last_up_change":"2023-09-30T22:26:44.227199+0000","last_in_change":"2023-09-30T22:26:37.120906+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-09-30T22:26:44.285133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:26:45.132921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:26:49.353683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:26:50.655685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"19946985-0ee2-487f-9347-929670dbe5ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6804","nonce":118824}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6805","nonce":118824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6807","nonce":118824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6806","nonce":118824}]},"public_addr":"172.21.15.16:6804/118824","cluster_addr":"172.21.15.16:6805/118824","heartbeat_back_addr":"172.21.15.16:6807/118824","heartbeat_front_addr":"172.21.15.16:6806/118824","state":["exists","up"]},{"osd":1,"uuid":"8efd4dbc-6ae4-4b82-b530-3b3eac2be445","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6812","nonce":118832}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6813","nonce":118832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6815","nonce":118832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6814","nonce":118832}]},"public_addr":"172.21.15.16:6812/118832","cluster_addr":"172.21.15.16:6813/118832","heartbeat_back_addr":"172.21.15.16:6815/118832","heartbeat_front_addr":"172.21.15.16:6814/118832","state":["exists","up"]},{"osd":2,"uuid":"b47bb1ce-27af-4c0c-868a-b76ebd0d9499","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6808","nonce":118840}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6809","nonce":118840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6811","nonce":118840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6810","nonce":118840}]},"public_addr":"172.21.15.16:6808/118840","cluster_addr":"172.21.15.16:6809/118840","heartbeat_back_addr":"172.21.15.16:6811/118840","heartbeat_front_addr":"172.21.15.16:6810/118840","state":["exists","up"]},{"osd":3,"uuid":"03b53d64-d10d-448a-a081-39ecceab589b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6800","nonce":118823}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6801","nonce":118823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6803","nonce":118823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6802","nonce":118823}]},"public_addr":"172.21.15.16:6800/118823","cluster_addr":"172.21.15.16:6801/118823","heartbeat_back_addr":"172.21.15.16:6803/118823","heartbeat_front_addr":"172.21.15.16:6802/118823","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.305953+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.307488+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.341023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.192071+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:27:07.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.290+0000 7f2ad78ed700 1 -- 172.21.15.16:0/3508590664 >> v1:172.21.15.16:6816/118285 conn(0x7f2ab402bb60 legacy=0x7f2ab402e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:07.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.290+0000 7f2ad78ed700 1 -- 172.21.15.16:0/3508590664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ad01303f0 msgr2=0x7f2ad0114aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:07.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.290+0000 7f2ad78ed700 1 --2- 172.21.15.16:0/3508590664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ad01303f0 0x7f2ad0114aa0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f2ac8011000 tx=0x7f2ac8009780).stop 2023-09-30T22:27:07.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.290+0000 7f2ad78ed700 1 -- 172.21.15.16:0/3508590664 shutdown_connections 2023-09-30T22:27:07.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.290+0000 7f2ad78ed700 1 --2- 172.21.15.16:0/3508590664 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2ad00613a0 0x7f2ad012a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:07.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.290+0000 7f2ad78ed700 1 --2- 172.21.15.16:0/3508590664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ad01303f0 0x7f2ad0114aa0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:07.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.290+0000 7f2ad78ed700 1 --2- 172.21.15.16:0/3508590664 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2ad012b510 0x7f2ad01145b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:07.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.290+0000 7f2ad78ed700 1 -- 172.21.15.16:0/3508590664 >> 172.21.15.16:0/3508590664 conn(0x7f2ad009a220 msgr2=0x7f2ad0095df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:07.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.290+0000 7f2ad78ed700 1 -- 172.21.15.16:0/3508590664 shutdown_connections 2023-09-30T22:27:07.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.290+0000 7f2ad78ed700 1 -- 172.21.15.16:0/3508590664 wait complete. 2023-09-30T22:27:07.305 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-09-30T22:26:44.285133+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-09-30T22:26:45.132921+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-30T22:26:49.353683+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-09-30T22:26:50.655685+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-30T22:27:07.306 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-09-30T22:27:07.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.438+0000 7f02b1e3a700 1 Processor -- start 2023-09-30T22:27:07.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.438+0000 7f02b1e3a700 1 -- start start 2023-09-30T22:27:07.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.438+0000 7f02b1e3a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f02ac093840 0x7f02ac098790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:07.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.438+0000 7f02b1e3a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f02ac098c80 0x7f02ac099060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:07.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.438+0000 7f02b1e3a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f02ac12c150 0x7f02ac12e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:07.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.438+0000 7f02b1e3a700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02ac093c20 con 0x7f02ac12c150 2023-09-30T22:27:07.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.438+0000 7f02b1e3a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02ac092b90 con 0x7f02ac093840 2023-09-30T22:27:07.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.438+0000 7f02b1e3a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f02ac092d80 con 0x7f02ac098c80 2023-09-30T22:27:07.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.438+0000 7f02ab7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f02ac093840 0x7f02ac098790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:07.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.438+0000 7f02aaffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f02ac098c80 0x7f02ac099060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:07.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.438+0000 7f02ab7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f02ac093840 0x7f02ac098790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:49028/0 (socket says 172.21.15.16:49028) 2023-09-30T22:27:07.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.438+0000 7f02abfff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f02ac12c150 0x7f02ac12e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:07.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.438+0000 7f02ab7fe700 1 -- 172.21.15.16:0/360436225 learned_addr learned my addr 172.21.15.16:0/360436225 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:07.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.439+0000 7f02ab7fe700 1 -- 172.21.15.16:0/360436225 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f02ac098c80 msgr2=0x7f02ac099060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:07.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.439+0000 7f02ab7fe700 1 --2- 172.21.15.16:0/360436225 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f02ac098c80 0x7f02ac099060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:07.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.439+0000 7f02ab7fe700 1 -- 172.21.15.16:0/360436225 >> v2:172.21.15.16:3300/0 conn(0x7f02ac12c150 msgr2=0x7f02ac12e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:07.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.439+0000 7f02ab7fe700 1 --2- 172.21.15.16:0/360436225 >> v2:172.21.15.16:3300/0 conn(0x7f02ac12c150 0x7f02ac12e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:07.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.439+0000 7f02ab7fe700 1 -- 172.21.15.16:0/360436225 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02ac099b70 con 0x7f02ac093840 2023-09-30T22:27:07.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.439+0000 7f02aaffd700 1 --2- 172.21.15.16:0/360436225 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f02ac098c80 0x7f02ac099060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:07.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.439+0000 7f02abfff700 1 --2- 172.21.15.16:0/360436225 >> v2:172.21.15.16:3300/0 conn(0x7f02ac12c150 0x7f02ac12e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:07.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.439+0000 7f02ab7fe700 1 --2- 172.21.15.16:0/360436225 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f02ac093840 0x7f02ac098790 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f029c006fd0 tx=0x7f029c0090f0).ready entity=mon.1 client_cookie=1af70eaab7ec1bbe server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:07.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.439+0000 7f02aa7fc700 1 -- 172.21.15.16:0/360436225 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029c00f510 con 0x7f02ac093840 2023-09-30T22:27:07.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.439+0000 7f02aa7fc700 1 -- 172.21.15.16:0/360436225 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f029c00f670 con 0x7f02ac093840 2023-09-30T22:27:07.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.439+0000 7f02aa7fc700 1 -- 172.21.15.16:0/360436225 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029c00f9f0 con 0x7f02ac093840 2023-09-30T22:27:07.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.440+0000 7f02b1e3a700 1 -- 172.21.15.16:0/360436225 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f02ac093840 msgr2=0x7f02ac098790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:07.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.440+0000 7f02b1e3a700 1 --2- 172.21.15.16:0/360436225 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f02ac093840 0x7f02ac098790 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f029c006fd0 tx=0x7f029c0090f0).stop 2023-09-30T22:27:07.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.440+0000 7f02b1e3a700 1 -- 172.21.15.16:0/360436225 shutdown_connections 2023-09-30T22:27:07.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.440+0000 7f02b1e3a700 1 --2- 172.21.15.16:0/360436225 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f02ac093840 0x7f02ac098790 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:07.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.440+0000 7f02b1e3a700 1 --2- 172.21.15.16:0/360436225 >> v2:172.21.15.16:3300/0 conn(0x7f02ac12c150 0x7f02ac12e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:07.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.440+0000 7f02b1e3a700 1 --2- 172.21.15.16:0/360436225 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f02ac098c80 0x7f02ac099060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:07.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.440+0000 7f02b1e3a700 1 -- 172.21.15.16:0/360436225 >> 172.21.15.16:0/360436225 conn(0x7f02ac097480 msgr2=0x7f02ac097890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:07.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.440+0000 7f02b1e3a700 1 -- 172.21.15.16:0/360436225 shutdown_connections 2023-09-30T22:27:07.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.440+0000 7f02b1e3a700 1 -- 172.21.15.16:0/360436225 wait complete. 2023-09-30T22:27:07.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.441+0000 7f02b1e3a700 1 Processor -- start 2023-09-30T22:27:07.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.441+0000 7f02b1e3a700 1 -- start start 2023-09-30T22:27:07.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.441+0000 7f02b1e3a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f02ac093840 0x7f02ac138780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:07.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.441+0000 7f02b1e3a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f02ac098c80 0x7f02ac138c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:07.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.441+0000 7f02b1e3a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02ac12c150 0x7f02ac119210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:07.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.442+0000 7f02b1e3a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ac11c630 con 0x7f02ac12c150 2023-09-30T22:27:07.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.442+0000 7f02b1e3a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02ac12c530 con 0x7f02ac093840 2023-09-30T22:27:07.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.442+0000 7f02b1e3a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f02ac11b6f0 con 0x7f02ac098c80 2023-09-30T22:27:07.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.442+0000 7f02ab7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f02ac093840 0x7f02ac138780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:07.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.442+0000 7f02ab7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f02ac093840 0x7f02ac138780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:49040/0 (socket says 172.21.15.16:49040) 2023-09-30T22:27:07.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.442+0000 7f02ab7fe700 1 -- 172.21.15.16:0/833770314 learned_addr learned my addr 172.21.15.16:0/833770314 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:07.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.442+0000 7f02aaffd700 1 --2- 172.21.15.16:0/833770314 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f02ac098c80 0x7f02ac138c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:07.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.442+0000 7f02abfff700 1 --2- 172.21.15.16:0/833770314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02ac12c150 0x7f02ac119210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:07.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.442+0000 7f02ab7fe700 1 -- 172.21.15.16:0/833770314 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f02ac098c80 msgr2=0x7f02ac138c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:07.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.442+0000 7f02ab7fe700 1 --2- 172.21.15.16:0/833770314 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f02ac098c80 0x7f02ac138c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:07.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.442+0000 7f02ab7fe700 1 -- 172.21.15.16:0/833770314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02ac12c150 msgr2=0x7f02ac119210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:07.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.442+0000 7f02ab7fe700 1 --2- 172.21.15.16:0/833770314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02ac12c150 0x7f02ac119210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:07.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.442+0000 7f02ab7fe700 1 -- 172.21.15.16:0/833770314 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02a00092a0 con 0x7f02ac093840 2023-09-30T22:27:07.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.442+0000 7f02abfff700 1 --2- 172.21.15.16:0/833770314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02ac12c150 0x7f02ac119210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:07.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.442+0000 7f02ab7fe700 1 --2- 172.21.15.16:0/833770314 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f02ac093840 0x7f02ac138780 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f02ac09a710 tx=0x7f029c017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:07.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.442+0000 7f02937fe700 1 -- 172.21.15.16:0/833770314 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029c00f7e0 con 0x7f02ac093840 2023-09-30T22:27:07.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.442+0000 7f02937fe700 1 -- 172.21.15.16:0/833770314 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f029c00f940 con 0x7f02ac093840 2023-09-30T22:27:07.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.442+0000 7f02b1e3a700 1 -- 172.21.15.16:0/833770314 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02a4009a20 con 0x7f02ac093840 2023-09-30T22:27:07.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.442+0000 7f02937fe700 1 -- 172.21.15.16:0/833770314 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029c01c670 con 0x7f02ac093840 2023-09-30T22:27:07.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.442+0000 7f02b1e3a700 1 -- 172.21.15.16:0/833770314 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f029c009370 con 0x7f02ac093840 2023-09-30T22:27:07.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.443+0000 7f02937fe700 1 -- 172.21.15.16:0/833770314 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f029c01ca60 con 0x7f02ac093840 2023-09-30T22:27:07.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.443+0000 7f02937fe700 1 -- 172.21.15.16:0/833770314 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f029c03cc40 con 0x7f02ac093840 2023-09-30T22:27:07.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.443+0000 7f02b1e3a700 1 -- 172.21.15.16:0/833770314 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02ac05a260 con 0x7f02ac093840 2023-09-30T22:27:07.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.448+0000 7f02937fe700 1 -- 172.21.15.16:0/833770314 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f029c020030 con 0x7f02ac093840 2023-09-30T22:27:07.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.621+0000 7f02b1e3a700 1 -- 172.21.15.16:0/833770314 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f02ac0993b0 con 0x7f02ac093840 2023-09-30T22:27:07.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.622+0000 7f02937fe700 1 -- 172.21.15.16:0/833770314 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v16) v1 ==== 110+0+10 (secure 0 0 0) 0x7f029c0340b0 con 0x7f02ac093840 2023-09-30T22:27:07.623 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-09-30T22:27:07.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.623+0000 7f02b1e3a700 1 -- 172.21.15.16:0/833770314 >> v1:172.21.15.16:6816/118285 conn(0x7f029802bb90 legacy=0x7f029802e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:07.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.623+0000 7f02b1e3a700 1 -- 172.21.15.16:0/833770314 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f02ac093840 msgr2=0x7f02ac138780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:07.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.623+0000 7f02b1e3a700 1 --2- 172.21.15.16:0/833770314 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f02ac093840 0x7f02ac138780 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f02ac09a710 tx=0x7f029c017000).stop 2023-09-30T22:27:07.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.623+0000 7f02b1e3a700 1 -- 172.21.15.16:0/833770314 shutdown_connections 2023-09-30T22:27:07.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.623+0000 7f02b1e3a700 1 --2- 172.21.15.16:0/833770314 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f02ac093840 0x7f02ac138780 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:07.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.623+0000 7f02b1e3a700 1 --2- 172.21.15.16:0/833770314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02ac12c150 0x7f02ac119210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:07.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.623+0000 7f02b1e3a700 1 --2- 172.21.15.16:0/833770314 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f02ac098c80 0x7f02ac138c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:07.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.623+0000 7f02b1e3a700 1 -- 172.21.15.16:0/833770314 >> 172.21.15.16:0/833770314 conn(0x7f02ac097480 msgr2=0x7f02ac110fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:07.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.623+0000 7f02b1e3a700 1 -- 172.21.15.16:0/833770314 shutdown_connections 2023-09-30T22:27:07.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.624+0000 7f02b1e3a700 1 -- 172.21.15.16:0/833770314 wait complete. 2023-09-30T22:27:07.638 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-09-30T22:27:07.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.774+0000 7fbe5bc22700 1 Processor -- start 2023-09-30T22:27:07.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.774+0000 7fbe5bc22700 1 -- start start 2023-09-30T22:27:07.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.775+0000 7fbe5bc22700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbe54093840 0x7fbe54098790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:07.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.775+0000 7fbe5bc22700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbe5412d160 0x7fbe5412f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:07.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.775+0000 7fbe5bc22700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe54092d80 con 0x7fbe54098c80 2023-09-30T22:27:07.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.775+0000 7fbe5bc22700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe54093c20 con 0x7fbe54093840 2023-09-30T22:27:07.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.775+0000 7fbe5bc22700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe54092b90 con 0x7fbe5412d160 2023-09-30T22:27:07.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.775+0000 7fbe599be700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbe54093840 0x7fbe54098790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:07.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.775+0000 7fbe5a1bf700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbe5412d160 0x7fbe5412f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:07.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.775+0000 7fbe591bd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fbe54098c80 0x7fbe54099090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53210/0 (socket says 172.21.15.16:53210) 2023-09-30T22:27:07.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.775+0000 7fbe591bd700 1 -- 172.21.15.16:0/616197319 learned_addr learned my addr 172.21.15.16:0/616197319 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:27:07.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.775+0000 7fbe599be700 1 -- 172.21.15.16:0/616197319 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbe5412d160 msgr2=0x7fbe5412f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:07.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.775+0000 7fbe599be700 1 --2- 172.21.15.16:0/616197319 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbe5412d160 0x7fbe5412f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:07.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.775+0000 7fbe599be700 1 -- 172.21.15.16:0/616197319 >> v1:172.21.15.16:6789/0 conn(0x7fbe54098c80 legacy=0x7fbe54099090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:07.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.775+0000 7fbe599be700 1 -- 172.21.15.16:0/616197319 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe54099b70 con 0x7fbe54093840 2023-09-30T22:27:07.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.775+0000 7fbe589bc700 1 -- 172.21.15.16:0/616197319 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2261815678 0 0) 0x7fbe54092d80 con 0x7fbe54098c80 2023-09-30T22:27:07.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.775+0000 7fbe5a1bf700 1 --2- 172.21.15.16:0/616197319 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbe5412d160 0x7fbe5412f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:07.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.775+0000 7fbe599be700 1 --2- 172.21.15.16:0/616197319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe54093840 0x7fbe54098790 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7fbe50009760 tx=0x7fbe50009060).ready entity=mon.0 client_cookie=4d024d5c704207d4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:07.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.776+0000 7fbe589bc700 1 -- 172.21.15.16:0/616197319 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe5000f510 con 0x7fbe54093840 2023-09-30T22:27:07.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.776+0000 7fbe589bc700 1 -- 172.21.15.16:0/616197319 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe5000f670 con 0x7fbe54093840 2023-09-30T22:27:07.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.776+0000 7fbe589bc700 1 -- 172.21.15.16:0/616197319 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe5000f9f0 con 0x7fbe54093840 2023-09-30T22:27:07.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.776+0000 7fbe5bc22700 1 -- 172.21.15.16:0/616197319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe54093840 msgr2=0x7fbe54098790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:07.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.776+0000 7fbe5bc22700 1 --2- 172.21.15.16:0/616197319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe54093840 0x7fbe54098790 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7fbe50009760 tx=0x7fbe50009060).stop 2023-09-30T22:27:07.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.776+0000 7fbe5bc22700 1 -- 172.21.15.16:0/616197319 shutdown_connections 2023-09-30T22:27:07.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.776+0000 7fbe5bc22700 1 --2- 172.21.15.16:0/616197319 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbe5412d160 0x7fbe5412f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:07.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.776+0000 7fbe5bc22700 1 --2- 172.21.15.16:0/616197319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe54093840 0x7fbe54098790 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:07.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.776+0000 7fbe5bc22700 1 -- 172.21.15.16:0/616197319 >> 172.21.15.16:0/616197319 conn(0x7fbe54097480 msgr2=0x7fbe54097890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:07.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.776+0000 7fbe5bc22700 1 -- 172.21.15.16:0/616197319 shutdown_connections 2023-09-30T22:27:07.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.776+0000 7fbe5bc22700 1 -- 172.21.15.16:0/616197319 wait complete. 2023-09-30T22:27:07.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.777+0000 7fbe5bc22700 1 Processor -- start 2023-09-30T22:27:07.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.777+0000 7fbe5bc22700 1 -- start start 2023-09-30T22:27:07.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.778+0000 7fbe5bc22700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbe54093840 0x7fbe5411bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:07.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.778+0000 7fbe5bc22700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbe54098c80 0x7fbe54117d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:07.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.778+0000 7fbe5bc22700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe5412d160 0x7fbe54118200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:07.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.778+0000 7fbe5bc22700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5411b400 con 0x7fbe5412d160 2023-09-30T22:27:07.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.778+0000 7fbe5bc22700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5412ccb0 con 0x7fbe54093840 2023-09-30T22:27:07.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.778+0000 7fbe5bc22700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5411aeb0 con 0x7fbe54098c80 2023-09-30T22:27:07.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.778+0000 7fbe599be700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbe54093840 0x7fbe5411bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:07.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.778+0000 7fbe5a1bf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe5412d160 0x7fbe54118200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:07.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.778+0000 7fbe599be700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbe54093840 0x7fbe5411bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:49064/0 (socket says 172.21.15.16:49064) 2023-09-30T22:27:07.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.778+0000 7fbe599be700 1 -- 172.21.15.16:0/2630836557 learned_addr learned my addr 172.21.15.16:0/2630836557 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:07.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.778+0000 7fbe591bd700 1 --2- 172.21.15.16:0/2630836557 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbe54098c80 0x7fbe54117d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:07.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.778+0000 7fbe599be700 1 -- 172.21.15.16:0/2630836557 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbe54098c80 msgr2=0x7fbe54117d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:07.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.778+0000 7fbe599be700 1 --2- 172.21.15.16:0/2630836557 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbe54098c80 0x7fbe54117d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:07.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.778+0000 7fbe599be700 1 -- 172.21.15.16:0/2630836557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe5412d160 msgr2=0x7fbe54118200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:07.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.778+0000 7fbe599be700 1 --2- 172.21.15.16:0/2630836557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe5412d160 0x7fbe54118200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:07.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.778+0000 7fbe599be700 1 -- 172.21.15.16:0/2630836557 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe4c009a20 con 0x7fbe54093840 2023-09-30T22:27:07.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.778+0000 7fbe5a1bf700 1 --2- 172.21.15.16:0/2630836557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe5412d160 0x7fbe54118200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:07.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.778+0000 7fbe599be700 1 --2- 172.21.15.16:0/2630836557 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbe54093840 0x7fbe5411bbe0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fbe50009e00 tx=0x7fbe50017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:07.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.779+0000 7fbe45ffb700 1 -- 172.21.15.16:0/2630836557 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe5000f7e0 con 0x7fbe54093840 2023-09-30T22:27:07.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.779+0000 7fbe45ffb700 1 -- 172.21.15.16:0/2630836557 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe5000f940 con 0x7fbe54093840 2023-09-30T22:27:07.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.779+0000 7fbe45ffb700 1 -- 172.21.15.16:0/2630836557 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe5001b660 con 0x7fbe54093840 2023-09-30T22:27:07.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.779+0000 7fbe5bc22700 1 -- 172.21.15.16:0/2630836557 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe500092a0 con 0x7fbe54093840 2023-09-30T22:27:07.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.779+0000 7fbe5bc22700 1 -- 172.21.15.16:0/2630836557 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe54119f50 con 0x7fbe54093840 2023-09-30T22:27:07.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.780+0000 7fbe45ffb700 1 -- 172.21.15.16:0/2630836557 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fbe5001b9d0 con 0x7fbe54093840 2023-09-30T22:27:07.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.780+0000 7fbe45ffb700 1 -- 172.21.15.16:0/2630836557 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fbe5003c910 con 0x7fbe54093840 2023-09-30T22:27:07.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.780+0000 7fbe5bc22700 1 -- 172.21.15.16:0/2630836557 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe5405a260 con 0x7fbe54093840 2023-09-30T22:27:07.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.785+0000 7fbe45ffb700 1 -- 172.21.15.16:0/2630836557 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fbe50029080 con 0x7fbe54093840 2023-09-30T22:27:07.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.957+0000 7fbe5bc22700 1 -- 172.21.15.16:0/2630836557 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fbe54046c20 con 0x7fbe54093840 2023-09-30T22:27:07.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.958+0000 7fbe45ffb700 1 -- 172.21.15.16:0/2630836557 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v16) v1 ==== 92+0+10 (secure 0 0 0) 0x7fbe5001f020 con 0x7fbe54093840 2023-09-30T22:27:07.959 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 8 2023-09-30T22:27:07.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.960+0000 7fbe5bc22700 1 -- 172.21.15.16:0/2630836557 >> v1:172.21.15.16:6816/118285 conn(0x7fbe3802bb60 legacy=0x7fbe3802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:07.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.960+0000 7fbe5bc22700 1 -- 172.21.15.16:0/2630836557 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbe54093840 msgr2=0x7fbe5411bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:07.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.960+0000 7fbe5bc22700 1 --2- 172.21.15.16:0/2630836557 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbe54093840 0x7fbe5411bbe0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fbe50009e00 tx=0x7fbe50017000).stop 2023-09-30T22:27:07.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.960+0000 7fbe5bc22700 1 -- 172.21.15.16:0/2630836557 shutdown_connections 2023-09-30T22:27:07.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.960+0000 7fbe5bc22700 1 --2- 172.21.15.16:0/2630836557 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbe54093840 0x7fbe5411bbe0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:07.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.960+0000 7fbe5bc22700 1 --2- 172.21.15.16:0/2630836557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe5412d160 0x7fbe54118200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:07.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.960+0000 7fbe5bc22700 1 --2- 172.21.15.16:0/2630836557 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbe54098c80 0x7fbe54117d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:07.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.960+0000 7fbe5bc22700 1 -- 172.21.15.16:0/2630836557 >> 172.21.15.16:0/2630836557 conn(0x7fbe54097480 msgr2=0x7fbe5412e000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:07.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.960+0000 7fbe5bc22700 1 -- 172.21.15.16:0/2630836557 shutdown_connections 2023-09-30T22:27:07.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:07.960+0000 7fbe5bc22700 1 -- 172.21.15.16:0/2630836557 wait complete. 2023-09-30T22:27:07.975 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-30T22:27:08.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.106+0000 7ffb7b015700 1 Processor -- start 2023-09-30T22:27:08.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.106+0000 7ffb7b015700 1 -- start start 2023-09-30T22:27:08.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.107+0000 7ffb7b015700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ffb74066110 0x7ffb74097420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:08.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.107+0000 7ffb7b015700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffb7412add0 0x7ffb7412b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:08.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.107+0000 7ffb7b015700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb74064e00 con 0x7ffb741274d0 2023-09-30T22:27:08.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.107+0000 7ffb7b015700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb740664f0 con 0x7ffb74066110 2023-09-30T22:27:08.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.107+0000 7ffb7b015700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb74065d80 con 0x7ffb7412add0 2023-09-30T22:27:08.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.107+0000 7ffb73fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffb7412add0 0x7ffb7412b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:08.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.107+0000 7ffb78db1700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ffb74066110 0x7ffb74097420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:08.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.107+0000 7ffb78db1700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ffb74066110 0x7ffb74097420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40792/0 (socket says 172.21.15.16:40792) 2023-09-30T22:27:08.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.107+0000 7ffb73fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffb7412add0 0x7ffb7412b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:49068/0 (socket says 172.21.15.16:49068) 2023-09-30T22:27:08.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.107+0000 7ffb78db1700 1 -- 172.21.15.16:0/1473675847 learned_addr learned my addr 172.21.15.16:0/1473675847 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:08.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.107+0000 7ffb795b2700 1 --1- 172.21.15.16:0/1473675847 >> v1:172.21.15.16:6789/0 conn(0x7ffb741274d0 0x7ffb74127980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53218/0 (socket says 172.21.15.16:53218) 2023-09-30T22:27:08.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.107+0000 7ffb78db1700 1 -- 172.21.15.16:0/1473675847 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffb7412add0 msgr2=0x7ffb7412b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:08.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.107+0000 7ffb78db1700 1 --2- 172.21.15.16:0/1473675847 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffb7412add0 0x7ffb7412b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:08.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.107+0000 7ffb78db1700 1 -- 172.21.15.16:0/1473675847 >> v1:172.21.15.16:6789/0 conn(0x7ffb741274d0 legacy=0x7ffb74127980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:08.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.107+0000 7ffb78db1700 1 -- 172.21.15.16:0/1473675847 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb74065910 con 0x7ffb74066110 2023-09-30T22:27:08.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.107+0000 7ffb737fe700 1 -- 172.21.15.16:0/1473675847 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2778579800 0 0) 0x7ffb74064e00 con 0x7ffb741274d0 2023-09-30T22:27:08.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.107+0000 7ffb73fff700 1 --2- 172.21.15.16:0/1473675847 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffb7412add0 0x7ffb7412b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:08.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.108+0000 7ffb78db1700 1 --2- 172.21.15.16:0/1473675847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb74066110 0x7ffb74097420 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7ffb64006fd0 tx=0x7ffb640090f0).ready entity=mon.0 client_cookie=54c5cb67b844619e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:08.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.108+0000 7ffb737fe700 1 -- 172.21.15.16:0/1473675847 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6400b580 con 0x7ffb74066110 2023-09-30T22:27:08.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.108+0000 7ffb737fe700 1 -- 172.21.15.16:0/1473675847 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb6400b6e0 con 0x7ffb74066110 2023-09-30T22:27:08.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.108+0000 7ffb737fe700 1 -- 172.21.15.16:0/1473675847 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6400ba60 con 0x7ffb74066110 2023-09-30T22:27:08.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.108+0000 7ffb7b015700 1 -- 172.21.15.16:0/1473675847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb74066110 msgr2=0x7ffb74097420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:08.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.108+0000 7ffb7b015700 1 --2- 172.21.15.16:0/1473675847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb74066110 0x7ffb74097420 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7ffb64006fd0 tx=0x7ffb640090f0).stop 2023-09-30T22:27:08.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.108+0000 7ffb7b015700 1 -- 172.21.15.16:0/1473675847 shutdown_connections 2023-09-30T22:27:08.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.108+0000 7ffb7b015700 1 --2- 172.21.15.16:0/1473675847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb74066110 0x7ffb74097420 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:08.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.108+0000 7ffb7b015700 1 --2- 172.21.15.16:0/1473675847 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffb7412add0 0x7ffb7412b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:08.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.108+0000 7ffb7b015700 1 -- 172.21.15.16:0/1473675847 >> 172.21.15.16:0/1473675847 conn(0x7ffb7409ba60 msgr2=0x7ffb7409be70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:08.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.108+0000 7ffb7b015700 1 -- 172.21.15.16:0/1473675847 shutdown_connections 2023-09-30T22:27:08.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.108+0000 7ffb7b015700 1 -- 172.21.15.16:0/1473675847 wait complete. 2023-09-30T22:27:08.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.110+0000 7ffb7b015700 1 Processor -- start 2023-09-30T22:27:08.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.110+0000 7ffb7b015700 1 -- start start 2023-09-30T22:27:08.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.112+0000 7ffb7b015700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffb74066110 0x7ffb74116d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:08.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.112+0000 7ffb7b015700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffb741274d0 0x7ffb74113de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:08.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.112+0000 7ffb7b015700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb7412add0 0x7ffb741142d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:08.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.112+0000 7ffb7b015700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb74117680 con 0x7ffb7412add0 2023-09-30T22:27:08.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.112+0000 7ffb7b015700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb74065030 con 0x7ffb74066110 2023-09-30T22:27:08.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.112+0000 7ffb7b015700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffb741172e0 con 0x7ffb741274d0 2023-09-30T22:27:08.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.112+0000 7ffb78db1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffb74066110 0x7ffb74116d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:08.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.112+0000 7ffb78db1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffb74066110 0x7ffb74116d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:49072/0 (socket says 172.21.15.16:49072) 2023-09-30T22:27:08.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.112+0000 7ffb795b2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb7412add0 0x7ffb741142d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:08.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.112+0000 7ffb78db1700 1 -- 172.21.15.16:0/1824058999 learned_addr learned my addr 172.21.15.16:0/1824058999 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:08.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.112+0000 7ffb73fff700 1 --2- 172.21.15.16:0/1824058999 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffb741274d0 0x7ffb74113de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:08.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.112+0000 7ffb795b2700 1 -- 172.21.15.16:0/1824058999 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffb741274d0 msgr2=0x7ffb74113de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:08.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.112+0000 7ffb795b2700 1 --2- 172.21.15.16:0/1824058999 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffb741274d0 0x7ffb74113de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:08.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.112+0000 7ffb795b2700 1 -- 172.21.15.16:0/1824058999 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffb74066110 msgr2=0x7ffb74116d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:08.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.112+0000 7ffb795b2700 1 --2- 172.21.15.16:0/1824058999 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffb74066110 0x7ffb74116d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:08.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.112+0000 7ffb795b2700 1 -- 172.21.15.16:0/1824058999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb5c0092a0 con 0x7ffb7412add0 2023-09-30T22:27:08.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.112+0000 7ffb78db1700 1 --2- 172.21.15.16:0/1824058999 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffb74066110 0x7ffb74116d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:08.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.112+0000 7ffb795b2700 1 --2- 172.21.15.16:0/1824058999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb7412add0 0x7ffb741142d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7ffb6c00c790 tx=0x7ffb6c007ea0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:08.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.112+0000 7ffb70ff9700 1 -- 172.21.15.16:0/1824058999 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c005cc0 con 0x7ffb7412add0 2023-09-30T22:27:08.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.112+0000 7ffb70ff9700 1 -- 172.21.15.16:0/1824058999 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb6c005e40 con 0x7ffb7412add0 2023-09-30T22:27:08.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.112+0000 7ffb7b015700 1 -- 172.21.15.16:0/1824058999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb64009370 con 0x7ffb7412add0 2023-09-30T22:27:08.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.112+0000 7ffb70ff9700 1 -- 172.21.15.16:0/1824058999 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c007320 con 0x7ffb7412add0 2023-09-30T22:27:08.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.112+0000 7ffb7b015700 1 -- 172.21.15.16:0/1824058999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb74115970 con 0x7ffb7412add0 2023-09-30T22:27:08.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.112+0000 7ffb70ff9700 1 -- 172.21.15.16:0/1824058999 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7ffb6c007690 con 0x7ffb7412add0 2023-09-30T22:27:08.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.112+0000 7ffb70ff9700 1 -- 172.21.15.16:0/1824058999 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3690+0+0 (secure 0 0 0) 0x7ffb6c03a850 con 0x7ffb7412add0 2023-09-30T22:27:08.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.113+0000 7ffb7b015700 1 -- 172.21.15.16:0/1824058999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb74115b30 con 0x7ffb7412add0 2023-09-30T22:27:08.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.117+0000 7ffb70ff9700 1 -- 172.21.15.16:0/1824058999 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7ffb6c013cb0 con 0x7ffb7412add0 2023-09-30T22:27:08.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.292+0000 7ffb7b015700 1 -- 172.21.15.16:0/1824058999 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ffb7405a260 con 0x7ffb7412add0 2023-09-30T22:27:08.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.292+0000 7ffb70ff9700 1 -- 172.21.15.16:0/1824058999 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v16) v1 ==== 104+0+11 (secure 0 0 0) 0x7ffb6c010af0 con 0x7ffb7412add0 2023-09-30T22:27:08.294 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-09-30T22:27:08.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.294+0000 7ffb7b015700 1 -- 172.21.15.16:0/1824058999 >> v1:172.21.15.16:6816/118285 conn(0x7ffb5402bb60 legacy=0x7ffb5402e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:08.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.294+0000 7ffb7b015700 1 -- 172.21.15.16:0/1824058999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb7412add0 msgr2=0x7ffb741142d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:08.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.294+0000 7ffb7b015700 1 --2- 172.21.15.16:0/1824058999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb7412add0 0x7ffb741142d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7ffb6c00c790 tx=0x7ffb6c007ea0).stop 2023-09-30T22:27:08.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.294+0000 7ffb7b015700 1 -- 172.21.15.16:0/1824058999 shutdown_connections 2023-09-30T22:27:08.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.294+0000 7ffb7b015700 1 --2- 172.21.15.16:0/1824058999 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffb74066110 0x7ffb74116d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:08.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.294+0000 7ffb7b015700 1 --2- 172.21.15.16:0/1824058999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb7412add0 0x7ffb741142d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:08.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.295+0000 7ffb7b015700 1 --2- 172.21.15.16:0/1824058999 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffb741274d0 0x7ffb74113de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:08.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.295+0000 7ffb7b015700 1 -- 172.21.15.16:0/1824058999 >> 172.21.15.16:0/1824058999 conn(0x7ffb7409ba60 msgr2=0x7ffb74129740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:08.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.295+0000 7ffb7b015700 1 -- 172.21.15.16:0/1824058999 shutdown_connections 2023-09-30T22:27:08.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.295+0000 7ffb7b015700 1 -- 172.21.15.16:0/1824058999 wait complete. 2023-09-30T22:27:08.309 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-30T22:27:08.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.441+0000 7f6e795f5700 1 Processor -- start 2023-09-30T22:27:08.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.441+0000 7f6e795f5700 1 -- start start 2023-09-30T22:27:08.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.441+0000 7f6e795f5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6e740653a0 0x7f6e74098640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:08.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.441+0000 7f6e795f5700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6e74098b30 0x7f6e74098f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:08.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.441+0000 7f6e795f5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e7412f430 0x7f6e74131820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:08.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.441+0000 7f6e795f5700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e74065e80 con 0x7f6e74098b30 2023-09-30T22:27:08.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.441+0000 7f6e795f5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e740663b0 con 0x7f6e740653a0 2023-09-30T22:27:08.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.441+0000 7f6e795f5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6e74065780 con 0x7f6e7412f430 2023-09-30T22:27:08.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.441+0000 7f6e727fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6e74098b30 0x7f6e74098f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:08.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.441+0000 7f6e72ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6e740653a0 0x7f6e74098640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:08.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.441+0000 7f6e737fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e7412f430 0x7f6e74131820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:08.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.441+0000 7f6e727fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6e74098b30 0x7f6e74098f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40810/0 (socket says 172.21.15.16:40810) 2023-09-30T22:27:08.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.441+0000 7f6e727fc700 1 -- 172.21.15.16:0/3852451889 learned_addr learned my addr 172.21.15.16:0/3852451889 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:08.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.441+0000 7f6e727fc700 1 -- 172.21.15.16:0/3852451889 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e7412f430 msgr2=0x7f6e74131820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:08.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.441+0000 7f6e727fc700 1 --2- 172.21.15.16:0/3852451889 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e7412f430 0x7f6e74131820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:08.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.441+0000 7f6e727fc700 1 -- 172.21.15.16:0/3852451889 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6e740653a0 msgr2=0x7f6e74098640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:08.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.441+0000 7f6e727fc700 1 --2- 172.21.15.16:0/3852451889 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6e740653a0 0x7f6e74098640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:08.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.441+0000 7f6e727fc700 1 -- 172.21.15.16:0/3852451889 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e74096180 con 0x7f6e74098b30 2023-09-30T22:27:08.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.441+0000 7f6e737fe700 1 --2- 172.21.15.16:0/3852451889 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e7412f430 0x7f6e74131820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:08.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.441+0000 7f6e727fc700 1 --2- 172.21.15.16:0/3852451889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e74098b30 0x7f6e74098f10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f6e68006fd0 tx=0x7f6e680090f0).ready entity=mon.0 client_cookie=a184c901a34091cd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:08.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.442+0000 7f6e71ffb700 1 -- 172.21.15.16:0/3852451889 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6800f510 con 0x7f6e74098b30 2023-09-30T22:27:08.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.442+0000 7f6e71ffb700 1 -- 172.21.15.16:0/3852451889 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e6800f670 con 0x7f6e74098b30 2023-09-30T22:27:08.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.442+0000 7f6e71ffb700 1 -- 172.21.15.16:0/3852451889 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6800f9f0 con 0x7f6e74098b30 2023-09-30T22:27:08.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.442+0000 7f6e795f5700 1 -- 172.21.15.16:0/3852451889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e74098b30 msgr2=0x7f6e74098f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:08.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.442+0000 7f6e795f5700 1 --2- 172.21.15.16:0/3852451889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e74098b30 0x7f6e74098f10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f6e68006fd0 tx=0x7f6e680090f0).stop 2023-09-30T22:27:08.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.442+0000 7f6e795f5700 1 -- 172.21.15.16:0/3852451889 shutdown_connections 2023-09-30T22:27:08.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.442+0000 7f6e795f5700 1 --2- 172.21.15.16:0/3852451889 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6e740653a0 0x7f6e74098640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:08.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.442+0000 7f6e795f5700 1 --2- 172.21.15.16:0/3852451889 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e7412f430 0x7f6e74131820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:08.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.442+0000 7f6e795f5700 1 --2- 172.21.15.16:0/3852451889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e74098b30 0x7f6e74098f10 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:08.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.442+0000 7f6e795f5700 1 -- 172.21.15.16:0/3852451889 >> 172.21.15.16:0/3852451889 conn(0x7f6e7409dc20 msgr2=0x7f6e7409e030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:08.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.442+0000 7f6e795f5700 1 -- 172.21.15.16:0/3852451889 shutdown_connections 2023-09-30T22:27:08.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.442+0000 7f6e795f5700 1 -- 172.21.15.16:0/3852451889 wait complete. 2023-09-30T22:27:08.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.443+0000 7f6e795f5700 1 Processor -- start 2023-09-30T22:27:08.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.443+0000 7f6e795f5700 1 -- start start 2023-09-30T22:27:08.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.444+0000 7f6e795f5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e740653a0 0x7f6e7411a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:08.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.444+0000 7f6e795f5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e74098b30 0x7f6e7411a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:08.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.444+0000 7f6e795f5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6e7412f430 0x7f6e7411ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:08.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.444+0000 7f6e795f5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7411b280 con 0x7f6e74098b30 2023-09-30T22:27:08.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.444+0000 7f6e795f5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7411d940 con 0x7f6e7412f430 2023-09-30T22:27:08.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.444+0000 7f6e795f5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7411c150 con 0x7f6e740653a0 2023-09-30T22:27:08.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.444+0000 7f6e72ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e740653a0 0x7f6e7411a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:08.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.444+0000 7f6e737fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6e7412f430 0x7f6e7411ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:08.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.444+0000 7f6e72ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e740653a0 0x7f6e7411a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37614/0 (socket says 172.21.15.16:37614) 2023-09-30T22:27:08.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.444+0000 7f6e72ffd700 1 -- 172.21.15.16:0/890343802 learned_addr learned my addr 172.21.15.16:0/890343802 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:08.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.444+0000 7f6e727fc700 1 --2- 172.21.15.16:0/890343802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e74098b30 0x7f6e7411a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:08.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.444+0000 7f6e72ffd700 1 -- 172.21.15.16:0/890343802 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6e7412f430 msgr2=0x7f6e7411ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:08.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.444+0000 7f6e72ffd700 1 --2- 172.21.15.16:0/890343802 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6e7412f430 0x7f6e7411ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:08.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.444+0000 7f6e72ffd700 1 -- 172.21.15.16:0/890343802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e74098b30 msgr2=0x7f6e7411a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:08.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.444+0000 7f6e72ffd700 1 --2- 172.21.15.16:0/890343802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e74098b30 0x7f6e7411a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:08.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.444+0000 7f6e72ffd700 1 -- 172.21.15.16:0/890343802 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e6c009a20 con 0x7f6e740653a0 2023-09-30T22:27:08.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.444+0000 7f6e737fe700 1 --2- 172.21.15.16:0/890343802 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6e7412f430 0x7f6e7411ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:08.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.444+0000 7f6e727fc700 1 --2- 172.21.15.16:0/890343802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e74098b30 0x7f6e7411a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:08.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.444+0000 7f6e72ffd700 1 --2- 172.21.15.16:0/890343802 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e740653a0 0x7f6e7411a3b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f6e6400e930 tx=0x7f6e64009060).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:08.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.445+0000 7f6e5affd700 1 -- 172.21.15.16:0/890343802 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e64009510 con 0x7f6e740653a0 2023-09-30T22:27:08.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.445+0000 7f6e5affd700 1 -- 172.21.15.16:0/890343802 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e64012600 con 0x7f6e740653a0 2023-09-30T22:27:08.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.445+0000 7f6e795f5700 1 -- 172.21.15.16:0/890343802 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e68009370 con 0x7f6e740653a0 2023-09-30T22:27:08.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.445+0000 7f6e5affd700 1 -- 172.21.15.16:0/890343802 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e64012980 con 0x7f6e740653a0 2023-09-30T22:27:08.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.445+0000 7f6e795f5700 1 -- 172.21.15.16:0/890343802 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e74099a90 con 0x7f6e740653a0 2023-09-30T22:27:08.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.445+0000 7f6e5affd700 1 -- 172.21.15.16:0/890343802 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f6e6401b400 con 0x7f6e740653a0 2023-09-30T22:27:08.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.446+0000 7f6e5affd700 1 -- 172.21.15.16:0/890343802 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f6e6403b3e0 con 0x7f6e740653a0 2023-09-30T22:27:08.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.446+0000 7f6e795f5700 1 -- 172.21.15.16:0/890343802 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e7411de40 con 0x7f6e740653a0 2023-09-30T22:27:08.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.451+0000 7f6e5affd700 1 -- 172.21.15.16:0/890343802 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f6e64019310 con 0x7f6e740653a0 2023-09-30T22:27:08.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.624+0000 7f6e795f5700 1 -- 172.21.15.16:0/890343802 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6e7405a260 con 0x7f6e740653a0 2023-09-30T22:27:08.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.624+0000 7f6e5affd700 1 -- 172.21.15.16:0/890343802 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v16) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6e64023350 con 0x7f6e740653a0 2023-09-30T22:27:08.626 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-09-30T22:27:08.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.627+0000 7f6e795f5700 1 -- 172.21.15.16:0/890343802 >> v1:172.21.15.16:6816/118285 conn(0x7f6e6002bb60 legacy=0x7f6e6002e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:08.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.627+0000 7f6e795f5700 1 -- 172.21.15.16:0/890343802 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e740653a0 msgr2=0x7f6e7411a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:08.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.627+0000 7f6e795f5700 1 --2- 172.21.15.16:0/890343802 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e740653a0 0x7f6e7411a3b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f6e6400e930 tx=0x7f6e64009060).stop 2023-09-30T22:27:08.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.627+0000 7f6e795f5700 1 -- 172.21.15.16:0/890343802 shutdown_connections 2023-09-30T22:27:08.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.627+0000 7f6e795f5700 1 --2- 172.21.15.16:0/890343802 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6e7412f430 0x7f6e7411ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:08.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.627+0000 7f6e795f5700 1 --2- 172.21.15.16:0/890343802 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e740653a0 0x7f6e7411a3b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:08.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.627+0000 7f6e795f5700 1 --2- 172.21.15.16:0/890343802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e74098b30 0x7f6e7411a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:08.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.627+0000 7f6e795f5700 1 -- 172.21.15.16:0/890343802 >> 172.21.15.16:0/890343802 conn(0x7f6e7409dc20 msgr2=0x7f6e7409bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:08.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.627+0000 7f6e795f5700 1 -- 172.21.15.16:0/890343802 shutdown_connections 2023-09-30T22:27:08.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.627+0000 7f6e795f5700 1 -- 172.21.15.16:0/890343802 wait complete. 2023-09-30T22:27:08.641 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-30T22:27:08.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.769+0000 7fbf88781700 1 Processor -- start 2023-09-30T22:27:08.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.769+0000 7fbf88781700 1 -- start start 2023-09-30T22:27:08.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.770+0000 7fbf88781700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbf80059070 0x7fbf80127890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:08.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.770+0000 7fbf88781700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbf80127390 0x7fbf80127d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:08.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.770+0000 7fbf88781700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbf80095370 0x7fbf80095820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:08.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.770+0000 7fbf88781700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf80059450 con 0x7fbf80059070 2023-09-30T22:27:08.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.770+0000 7fbf88781700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf800968c0 con 0x7fbf80127390 2023-09-30T22:27:08.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.770+0000 7fbf88781700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbf80094f30 con 0x7fbf80095370 2023-09-30T22:27:08.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.770+0000 7fbf8651d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbf80059070 0x7fbf80127890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:08.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.770+0000 7fbf85d1c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbf80127390 0x7fbf80127d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:08.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.770+0000 7fbf86d1e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbf80095370 0x7fbf80095820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:08.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.770+0000 7fbf8651d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbf80059070 0x7fbf80127890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40828/0 (socket says 172.21.15.16:40828) 2023-09-30T22:27:08.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.770+0000 7fbf85d1c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbf80127390 0x7fbf80127d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:49090/0 (socket says 172.21.15.16:49090) 2023-09-30T22:27:08.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.770+0000 7fbf85d1c700 1 -- 172.21.15.16:0/2965516367 learned_addr learned my addr 172.21.15.16:0/2965516367 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:08.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.772+0000 7fbf85d1c700 1 -- 172.21.15.16:0/2965516367 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbf80095370 msgr2=0x7fbf80095820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:08.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.772+0000 7fbf85d1c700 1 --2- 172.21.15.16:0/2965516367 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbf80095370 0x7fbf80095820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:08.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.772+0000 7fbf85d1c700 1 -- 172.21.15.16:0/2965516367 >> v2:172.21.15.16:3300/0 conn(0x7fbf80059070 msgr2=0x7fbf80127890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:08.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.772+0000 7fbf85d1c700 1 --2- 172.21.15.16:0/2965516367 >> v2:172.21.15.16:3300/0 conn(0x7fbf80059070 0x7fbf80127890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:08.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.772+0000 7fbf85d1c700 1 -- 172.21.15.16:0/2965516367 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf80091b60 con 0x7fbf80127390 2023-09-30T22:27:08.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.772+0000 7fbf8651d700 1 --2- 172.21.15.16:0/2965516367 >> v2:172.21.15.16:3300/0 conn(0x7fbf80059070 0x7fbf80127890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:08.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.772+0000 7fbf85d1c700 1 --2- 172.21.15.16:0/2965516367 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbf80127390 0x7fbf80127d80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fbf70006fd0 tx=0x7fbf700090f0).ready entity=mon.1 client_cookie=14c73817734a9442 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:08.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.772+0000 7fbf8551b700 1 -- 172.21.15.16:0/2965516367 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7000f510 con 0x7fbf80127390 2023-09-30T22:27:08.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.772+0000 7fbf8551b700 1 -- 172.21.15.16:0/2965516367 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf7000f670 con 0x7fbf80127390 2023-09-30T22:27:08.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.772+0000 7fbf8551b700 1 -- 172.21.15.16:0/2965516367 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7000fa10 con 0x7fbf80127390 2023-09-30T22:27:08.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.773+0000 7fbf88781700 1 -- 172.21.15.16:0/2965516367 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbf80127390 msgr2=0x7fbf80127d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:08.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.773+0000 7fbf88781700 1 --2- 172.21.15.16:0/2965516367 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbf80127390 0x7fbf80127d80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fbf70006fd0 tx=0x7fbf700090f0).stop 2023-09-30T22:27:08.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.773+0000 7fbf88781700 1 -- 172.21.15.16:0/2965516367 shutdown_connections 2023-09-30T22:27:08.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.773+0000 7fbf88781700 1 --2- 172.21.15.16:0/2965516367 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbf80095370 0x7fbf80095820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:08.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.773+0000 7fbf88781700 1 --2- 172.21.15.16:0/2965516367 >> v2:172.21.15.16:3300/0 conn(0x7fbf80059070 0x7fbf80127890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:08.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.773+0000 7fbf88781700 1 --2- 172.21.15.16:0/2965516367 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbf80127390 0x7fbf80127d80 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:08.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.773+0000 7fbf88781700 1 -- 172.21.15.16:0/2965516367 >> 172.21.15.16:0/2965516367 conn(0x7fbf80097470 msgr2=0x7fbf80097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:08.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.773+0000 7fbf88781700 1 -- 172.21.15.16:0/2965516367 shutdown_connections 2023-09-30T22:27:08.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.773+0000 7fbf88781700 1 -- 172.21.15.16:0/2965516367 wait complete. 2023-09-30T22:27:08.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.774+0000 7fbf88781700 1 Processor -- start 2023-09-30T22:27:08.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.774+0000 7fbf88781700 1 -- start start 2023-09-30T22:27:08.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.774+0000 7fbf88781700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf80059070 0x7fbf80113a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:08.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.774+0000 7fbf88781700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbf80095370 0x7fbf80113f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:08.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.774+0000 7fbf88781700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbf80127390 0x7fbf80114450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:08.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.774+0000 7fbf88781700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf80129190 con 0x7fbf80059070 2023-09-30T22:27:08.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.774+0000 7fbf88781700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf80096ea0 con 0x7fbf80095370 2023-09-30T22:27:08.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.774+0000 7fbf88781700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbf80124fc0 con 0x7fbf80127390 2023-09-30T22:27:08.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.774+0000 7fbf8651d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf80059070 0x7fbf80113a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:08.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.774+0000 7fbf85d1c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbf80095370 0x7fbf80113f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:08.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.774+0000 7fbf8651d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf80059070 0x7fbf80113a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40844/0 (socket says 172.21.15.16:40844) 2023-09-30T22:27:08.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.774+0000 7fbf8651d700 1 -- 172.21.15.16:0/1347759820 learned_addr learned my addr 172.21.15.16:0/1347759820 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:08.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.774+0000 7fbf86d1e700 1 --2- 172.21.15.16:0/1347759820 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbf80127390 0x7fbf80114450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:08.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.775+0000 7fbf85d1c700 1 -- 172.21.15.16:0/1347759820 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbf80127390 msgr2=0x7fbf80114450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:08.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.775+0000 7fbf85d1c700 1 --2- 172.21.15.16:0/1347759820 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbf80127390 0x7fbf80114450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:08.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.775+0000 7fbf85d1c700 1 -- 172.21.15.16:0/1347759820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf80059070 msgr2=0x7fbf80113a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:08.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.775+0000 7fbf85d1c700 1 --2- 172.21.15.16:0/1347759820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf80059070 0x7fbf80113a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:08.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.775+0000 7fbf85d1c700 1 -- 172.21.15.16:0/1347759820 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf7c0092a0 con 0x7fbf80095370 2023-09-30T22:27:08.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.775+0000 7fbf8651d700 1 --2- 172.21.15.16:0/1347759820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf80059070 0x7fbf80113a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:08.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.775+0000 7fbf85d1c700 1 --2- 172.21.15.16:0/1347759820 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbf80095370 0x7fbf80113f60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fbf80098320 tx=0x7fbf70017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:08.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.775+0000 7fbf767fc700 1 -- 172.21.15.16:0/1347759820 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf70006d90 con 0x7fbf80095370 2023-09-30T22:27:08.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.775+0000 7fbf767fc700 1 -- 172.21.15.16:0/1347759820 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf70006910 con 0x7fbf80095370 2023-09-30T22:27:08.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.775+0000 7fbf88781700 1 -- 172.21.15.16:0/1347759820 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf70009370 con 0x7fbf80095370 2023-09-30T22:27:08.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.775+0000 7fbf767fc700 1 -- 172.21.15.16:0/1347759820 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf70000bc0 con 0x7fbf80095370 2023-09-30T22:27:08.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.775+0000 7fbf88781700 1 -- 172.21.15.16:0/1347759820 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf80117ba0 con 0x7fbf80095370 2023-09-30T22:27:08.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.776+0000 7fbf767fc700 1 -- 172.21.15.16:0/1347759820 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fbf70022020 con 0x7fbf80095370 2023-09-30T22:27:08.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.776+0000 7fbf767fc700 1 -- 172.21.15.16:0/1347759820 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fbf7003cab0 con 0x7fbf80095370 2023-09-30T22:27:08.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.776+0000 7fbf88781700 1 -- 172.21.15.16:0/1347759820 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf80117d30 con 0x7fbf80095370 2023-09-30T22:27:08.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.780+0000 7fbf767fc700 1 -- 172.21.15.16:0/1347759820 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fbf7002a070 con 0x7fbf80095370 2023-09-30T22:27:08.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.996+0000 7fbf88781700 1 -- 172.21.15.16:0/1347759820 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbf8005a260 con 0x7fbf80095370 2023-09-30T22:27:08.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.996+0000 7fbf767fc700 1 -- 172.21.15.16:0/1347759820 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+2156 (secure 0 0 0) 0x7fbf70034090 con 0x7fbf80095370 2023-09-30T22:27:08.997 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:27:08.998 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-30T22:26:51.674561+0000","modified":"2023-09-30T22:26:54.680606+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4218},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4218":{"gid":4218,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6817/1424366610","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6817","nonce":1424366610}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-09-30T22:27:09.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.998+0000 7fbf88781700 1 -- 172.21.15.16:0/1347759820 >> v1:172.21.15.16:6816/118285 conn(0x7fbf6402bb60 legacy=0x7fbf6402e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:09.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.998+0000 7fbf88781700 1 -- 172.21.15.16:0/1347759820 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbf80095370 msgr2=0x7fbf80113f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:09.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.998+0000 7fbf88781700 1 --2- 172.21.15.16:0/1347759820 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbf80095370 0x7fbf80113f60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fbf80098320 tx=0x7fbf70017000).stop 2023-09-30T22:27:09.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.998+0000 7fbf88781700 1 -- 172.21.15.16:0/1347759820 shutdown_connections 2023-09-30T22:27:09.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.998+0000 7fbf88781700 1 --2- 172.21.15.16:0/1347759820 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbf80127390 0x7fbf80114450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:09.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.998+0000 7fbf88781700 1 --2- 172.21.15.16:0/1347759820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf80059070 0x7fbf80113a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:09.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.998+0000 7fbf88781700 1 --2- 172.21.15.16:0/1347759820 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbf80095370 0x7fbf80113f60 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:09.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.998+0000 7fbf88781700 1 -- 172.21.15.16:0/1347759820 >> 172.21.15.16:0/1347759820 conn(0x7fbf80097470 msgr2=0x7fbf8012add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:09.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.999+0000 7fbf88781700 1 -- 172.21.15.16:0/1347759820 shutdown_connections 2023-09-30T22:27:09.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:08.999+0000 7fbf88781700 1 -- 172.21.15.16:0/1347759820 wait complete. 2023-09-30T22:27:09.002 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 6 2023-09-30T22:27:09.012 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:27:09.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.141+0000 7fd29bd7d700 1 Processor -- start 2023-09-30T22:27:09.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.141+0000 7fd29bd7d700 1 -- start start 2023-09-30T22:27:09.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.141+0000 7fd29bd7d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd29412ecd0 0x7fd29412f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:09.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.142+0000 7fd29bd7d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd294134060 0x7fd294136450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:09.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.142+0000 7fd29bd7d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd294059ba0 con 0x7fd294059070 2023-09-30T22:27:09.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.142+0000 7fd29bd7d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2940969b0 con 0x7fd29412ecd0 2023-09-30T22:27:09.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.142+0000 7fd29bd7d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd294092320 con 0x7fd294134060 2023-09-30T22:27:09.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.142+0000 7fd299318700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd29412ecd0 0x7fd29412f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:09.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.142+0000 7fd29a31a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd294134060 0x7fd294136450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:09.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.142+0000 7fd29a31a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd294134060 0x7fd294136450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37632/0 (socket says 172.21.15.16:37632) 2023-09-30T22:27:09.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.142+0000 7fd29a31a700 1 -- 172.21.15.16:0/1618707884 learned_addr learned my addr 172.21.15.16:0/1618707884 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:09.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.142+0000 7fd299318700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd29412ecd0 0x7fd29412f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:49100/0 (socket says 172.21.15.16:49100) 2023-09-30T22:27:09.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.142+0000 7fd298b17700 1 -- 172.21.15.16:0/1618707884 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2975703799 0 0) 0x7fd294059ba0 con 0x7fd294059070 2023-09-30T22:27:09.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.142+0000 7fd29a31a700 1 -- 172.21.15.16:0/1618707884 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd29412ecd0 msgr2=0x7fd29412f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:09.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.142+0000 7fd29a31a700 1 --2- 172.21.15.16:0/1618707884 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd29412ecd0 0x7fd29412f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:09.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.142+0000 7fd29a31a700 1 -- 172.21.15.16:0/1618707884 >> v1:172.21.15.16:6789/0 conn(0x7fd294059070 legacy=0x7fd29412f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:09.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.142+0000 7fd29a31a700 1 -- 172.21.15.16:0/1618707884 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd294091c10 con 0x7fd294134060 2023-09-30T22:27:09.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.142+0000 7fd299318700 1 --2- 172.21.15.16:0/1618707884 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd29412ecd0 0x7fd29412f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:09.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.142+0000 7fd29a31a700 1 --2- 172.21.15.16:0/1618707884 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd294134060 0x7fd294136450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7fd28c009f50 tx=0x7fd28c009840).ready entity=mon.2 client_cookie=48e6782e021f98ac server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:09.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.142+0000 7fd298b17700 1 -- 172.21.15.16:0/1618707884 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28c00e070 con 0x7fd294134060 2023-09-30T22:27:09.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.142+0000 7fd298b17700 1 -- 172.21.15.16:0/1618707884 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd28c00ba00 con 0x7fd294134060 2023-09-30T22:27:09.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.142+0000 7fd298b17700 1 -- 172.21.15.16:0/1618707884 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28c00bd80 con 0x7fd294134060 2023-09-30T22:27:09.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.143+0000 7fd29bd7d700 1 -- 172.21.15.16:0/1618707884 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd294134060 msgr2=0x7fd294136450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:09.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.143+0000 7fd29bd7d700 1 --2- 172.21.15.16:0/1618707884 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd294134060 0x7fd294136450 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7fd28c009f50 tx=0x7fd28c009840).stop 2023-09-30T22:27:09.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.143+0000 7fd29bd7d700 1 -- 172.21.15.16:0/1618707884 shutdown_connections 2023-09-30T22:27:09.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.143+0000 7fd29bd7d700 1 --2- 172.21.15.16:0/1618707884 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd294134060 0x7fd294136450 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:09.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.143+0000 7fd29bd7d700 1 --2- 172.21.15.16:0/1618707884 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd29412ecd0 0x7fd29412f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:09.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.143+0000 7fd29bd7d700 1 -- 172.21.15.16:0/1618707884 >> 172.21.15.16:0/1618707884 conn(0x7fd294097470 msgr2=0x7fd294097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:09.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.143+0000 7fd29bd7d700 1 -- 172.21.15.16:0/1618707884 shutdown_connections 2023-09-30T22:27:09.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.143+0000 7fd29bd7d700 1 -- 172.21.15.16:0/1618707884 wait complete. 2023-09-30T22:27:09.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.144+0000 7fd29bd7d700 1 Processor -- start 2023-09-30T22:27:09.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.144+0000 7fd29bd7d700 1 -- start start 2023-09-30T22:27:09.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.145+0000 7fd29bd7d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd294059070 0x7fd2940693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:09.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.145+0000 7fd29bd7d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd294134060 0x7fd29412fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:09.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.145+0000 7fd29bd7d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd294130280 0x7fd2941306c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:09.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.145+0000 7fd29bd7d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2941330c0 con 0x7fd294059070 2023-09-30T22:27:09.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.145+0000 7fd29bd7d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2941332a0 con 0x7fd294130280 2023-09-30T22:27:09.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.145+0000 7fd29bd7d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd294130bb0 con 0x7fd294134060 2023-09-30T22:27:09.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.145+0000 7fd29a31a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd294130280 0x7fd2941306c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:09.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.145+0000 7fd299318700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd294134060 0x7fd29412fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:09.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.145+0000 7fd29a31a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd294130280 0x7fd2941306c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:49106/0 (socket says 172.21.15.16:49106) 2023-09-30T22:27:09.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.145+0000 7fd29a31a700 1 -- 172.21.15.16:0/1542429771 learned_addr learned my addr 172.21.15.16:0/1542429771 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:09.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.145+0000 7fd299b19700 1 --2- 172.21.15.16:0/1542429771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd294059070 0x7fd2940693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:09.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.145+0000 7fd299318700 1 -- 172.21.15.16:0/1542429771 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd294130280 msgr2=0x7fd2941306c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:09.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.145+0000 7fd299318700 1 --2- 172.21.15.16:0/1542429771 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd294130280 0x7fd2941306c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:09.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.145+0000 7fd299318700 1 -- 172.21.15.16:0/1542429771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd294059070 msgr2=0x7fd2940693b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:09.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.145+0000 7fd299318700 1 --2- 172.21.15.16:0/1542429771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd294059070 0x7fd2940693b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:09.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.145+0000 7fd299318700 1 -- 172.21.15.16:0/1542429771 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2840092a0 con 0x7fd294134060 2023-09-30T22:27:09.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.145+0000 7fd29a31a700 1 --2- 172.21.15.16:0/1542429771 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd294130280 0x7fd2941306c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:09.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.145+0000 7fd299318700 1 --2- 172.21.15.16:0/1542429771 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd294134060 0x7fd29412fd90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7fd28400e980 tx=0x7fd284009060).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:09.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.145+0000 7fd289ffb700 1 -- 172.21.15.16:0/1542429771 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd284012600 con 0x7fd294134060 2023-09-30T22:27:09.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.145+0000 7fd289ffb700 1 -- 172.21.15.16:0/1542429771 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd284012760 con 0x7fd294134060 2023-09-30T22:27:09.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.145+0000 7fd29bd7d700 1 -- 172.21.15.16:0/1542429771 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd28c009ae0 con 0x7fd294134060 2023-09-30T22:27:09.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.145+0000 7fd289ffb700 1 -- 172.21.15.16:0/1542429771 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd284012b00 con 0x7fd294134060 2023-09-30T22:27:09.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.146+0000 7fd29bd7d700 1 -- 172.21.15.16:0/1542429771 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2941323e0 con 0x7fd294134060 2023-09-30T22:27:09.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.146+0000 7fd289ffb700 1 -- 172.21.15.16:0/1542429771 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fd28401b400 con 0x7fd294134060 2023-09-30T22:27:09.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.147+0000 7fd289ffb700 1 -- 172.21.15.16:0/1542429771 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fd28403b490 con 0x7fd294134060 2023-09-30T22:27:09.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.147+0000 7fd29bd7d700 1 -- 172.21.15.16:0/1542429771 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd29406a4e0 con 0x7fd294134060 2023-09-30T22:27:09.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.151+0000 7fd289ffb700 1 -- 172.21.15.16:0/1542429771 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7fd284019900 con 0x7fd294134060 2023-09-30T22:27:09.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.327+0000 7fd29bd7d700 1 -- 172.21.15.16:0/1542429771 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd294136940 con 0x7fd294134060 2023-09-30T22:27:09.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.328+0000 7fd289ffb700 1 -- 172.21.15.16:0/1542429771 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+11341 (secure 0 0 0) 0x7fd284019310 con 0x7fd294134060 2023-09-30T22:27:09.328 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:27:09.329 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":16,"fsid":"7ede3e14-2bd0-4726-9437-6d13c90cd342","created":"2023-09-30T22:26:35.483697+0000","modified":"2023-09-30T22:26:52.675888+0000","last_up_change":"2023-09-30T22:26:44.227199+0000","last_in_change":"2023-09-30T22:26:37.120906+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-09-30T22:26:44.285133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:26:45.132921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:26:49.353683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:26:50.655685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"19946985-0ee2-487f-9347-929670dbe5ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6804","nonce":118824}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6805","nonce":118824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6807","nonce":118824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6806","nonce":118824}]},"public_addr":"172.21.15.16:6804/118824","cluster_addr":"172.21.15.16:6805/118824","heartbeat_back_addr":"172.21.15.16:6807/118824","heartbeat_front_addr":"172.21.15.16:6806/118824","state":["exists","up"]},{"osd":1,"uuid":"8efd4dbc-6ae4-4b82-b530-3b3eac2be445","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6812","nonce":118832}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6813","nonce":118832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6815","nonce":118832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6814","nonce":118832}]},"public_addr":"172.21.15.16:6812/118832","cluster_addr":"172.21.15.16:6813/118832","heartbeat_back_addr":"172.21.15.16:6815/118832","heartbeat_front_addr":"172.21.15.16:6814/118832","state":["exists","up"]},{"osd":2,"uuid":"b47bb1ce-27af-4c0c-868a-b76ebd0d9499","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6808","nonce":118840}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6809","nonce":118840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6811","nonce":118840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6810","nonce":118840}]},"public_addr":"172.21.15.16:6808/118840","cluster_addr":"172.21.15.16:6809/118840","heartbeat_back_addr":"172.21.15.16:6811/118840","heartbeat_front_addr":"172.21.15.16:6810/118840","state":["exists","up"]},{"osd":3,"uuid":"03b53d64-d10d-448a-a081-39ecceab589b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6800","nonce":118823}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6801","nonce":118823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6803","nonce":118823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6802","nonce":118823}]},"public_addr":"172.21.15.16:6800/118823","cluster_addr":"172.21.15.16:6801/118823","heartbeat_back_addr":"172.21.15.16:6803/118823","heartbeat_front_addr":"172.21.15.16:6802/118823","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.305953+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.307488+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.341023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.192071+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:27:09.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.330+0000 7fd29bd7d700 1 -- 172.21.15.16:0/1542429771 >> v1:172.21.15.16:6816/118285 conn(0x7fd27802bb60 legacy=0x7fd27802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:09.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.330+0000 7fd29bd7d700 1 -- 172.21.15.16:0/1542429771 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd294134060 msgr2=0x7fd29412fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:09.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.330+0000 7fd29bd7d700 1 --2- 172.21.15.16:0/1542429771 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd294134060 0x7fd29412fd90 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7fd28400e980 tx=0x7fd284009060).stop 2023-09-30T22:27:09.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.330+0000 7fd29bd7d700 1 -- 172.21.15.16:0/1542429771 shutdown_connections 2023-09-30T22:27:09.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.330+0000 7fd29bd7d700 1 --2- 172.21.15.16:0/1542429771 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd294130280 0x7fd2941306c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:09.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.330+0000 7fd29bd7d700 1 --2- 172.21.15.16:0/1542429771 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd294059070 0x7fd2940693b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:09.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.330+0000 7fd29bd7d700 1 --2- 172.21.15.16:0/1542429771 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd294134060 0x7fd29412fd90 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:09.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.330+0000 7fd29bd7d700 1 -- 172.21.15.16:0/1542429771 >> 172.21.15.16:0/1542429771 conn(0x7fd294097470 msgr2=0x7fd29412deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:09.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.330+0000 7fd29bd7d700 1 -- 172.21.15.16:0/1542429771 shutdown_connections 2023-09-30T22:27:09.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.330+0000 7fd29bd7d700 1 -- 172.21.15.16:0/1542429771 wait complete. 2023-09-30T22:27:09.344 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-30T22:27:09.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.474+0000 7f33fce0a700 1 Processor -- start 2023-09-30T22:27:09.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.474+0000 7f33fce0a700 1 -- start start 2023-09-30T22:27:09.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.475+0000 7f33fce0a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f33f8096820 0x7f33f8096c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:09.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.475+0000 7f33fce0a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f33f8067940 0x7f33f80970f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:09.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.475+0000 7f33fce0a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f33f80975e0 0x7f33f812f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:09.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.475+0000 7f33fce0a700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33f8068df0 con 0x7f33f8067940 2023-09-30T22:27:09.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.475+0000 7f33fce0a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33f806a4c0 con 0x7f33f8096820 2023-09-30T22:27:09.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.475+0000 7f33fce0a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f33f8098260 con 0x7f33f80975e0 2023-09-30T22:27:09.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.475+0000 7f33f659c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f33f8096820 0x7f33f8096c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:09.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.475+0000 7f33f5d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f33f8067940 0x7f33f80970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:09.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.475+0000 7f33f6d9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f33f80975e0 0x7f33f812f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:09.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.475+0000 7f33f659c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f33f8096820 0x7f33f8096c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:49122/0 (socket says 172.21.15.16:49122) 2023-09-30T22:27:09.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.475+0000 7f33f5d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f33f8067940 0x7f33f80970f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40854/0 (socket says 172.21.15.16:40854) 2023-09-30T22:27:09.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.475+0000 7f33f6d9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f33f80975e0 0x7f33f812f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37644/0 (socket says 172.21.15.16:37644) 2023-09-30T22:27:09.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.475+0000 7f33f659c700 1 -- 172.21.15.16:0/4234191927 learned_addr learned my addr 172.21.15.16:0/4234191927 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:09.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.476+0000 7f33f659c700 1 -- 172.21.15.16:0/4234191927 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f33f80975e0 msgr2=0x7f33f812f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:09.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.476+0000 7f33f659c700 1 --2- 172.21.15.16:0/4234191927 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f33f80975e0 0x7f33f812f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:09.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.476+0000 7f33f659c700 1 -- 172.21.15.16:0/4234191927 >> v2:172.21.15.16:3300/0 conn(0x7f33f8067940 msgr2=0x7f33f80970f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:09.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.476+0000 7f33f659c700 1 --2- 172.21.15.16:0/4234191927 >> v2:172.21.15.16:3300/0 conn(0x7f33f8067940 0x7f33f80970f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:09.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.476+0000 7f33f659c700 1 -- 172.21.15.16:0/4234191927 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f8095e60 con 0x7f33f8096820 2023-09-30T22:27:09.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.476+0000 7f33f6d9d700 1 --2- 172.21.15.16:0/4234191927 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f33f80975e0 0x7f33f812f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:09.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.476+0000 7f33f5d9b700 1 --2- 172.21.15.16:0/4234191927 >> v2:172.21.15.16:3300/0 conn(0x7f33f8067940 0x7f33f80970f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:09.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.476+0000 7f33f659c700 1 --2- 172.21.15.16:0/4234191927 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f33f8096820 0x7f33f8096c00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f33e8006fd0 tx=0x7f33e80090f0).ready entity=mon.1 client_cookie=7c386cb2c7f0cc68 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:09.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.476+0000 7f33f559a700 1 -- 172.21.15.16:0/4234191927 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e800f510 con 0x7f33f8096820 2023-09-30T22:27:09.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.476+0000 7f33f559a700 1 -- 172.21.15.16:0/4234191927 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33e800f670 con 0x7f33f8096820 2023-09-30T22:27:09.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.476+0000 7f33f559a700 1 -- 172.21.15.16:0/4234191927 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e800f9f0 con 0x7f33f8096820 2023-09-30T22:27:09.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.476+0000 7f33fce0a700 1 -- 172.21.15.16:0/4234191927 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f33f8096820 msgr2=0x7f33f8096c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:09.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.476+0000 7f33fce0a700 1 --2- 172.21.15.16:0/4234191927 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f33f8096820 0x7f33f8096c00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f33e8006fd0 tx=0x7f33e80090f0).stop 2023-09-30T22:27:09.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.477+0000 7f33fce0a700 1 -- 172.21.15.16:0/4234191927 shutdown_connections 2023-09-30T22:27:09.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.477+0000 7f33fce0a700 1 --2- 172.21.15.16:0/4234191927 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f33f8096820 0x7f33f8096c00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:09.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.477+0000 7f33fce0a700 1 --2- 172.21.15.16:0/4234191927 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f33f80975e0 0x7f33f812f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:09.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.477+0000 7f33fce0a700 1 --2- 172.21.15.16:0/4234191927 >> v2:172.21.15.16:3300/0 conn(0x7f33f8067940 0x7f33f80970f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:09.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.477+0000 7f33fce0a700 1 -- 172.21.15.16:0/4234191927 >> 172.21.15.16:0/4234191927 conn(0x7f33f809ba50 msgr2=0x7f33f809be60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:09.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.477+0000 7f33fce0a700 1 -- 172.21.15.16:0/4234191927 shutdown_connections 2023-09-30T22:27:09.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.477+0000 7f33fce0a700 1 -- 172.21.15.16:0/4234191927 wait complete. 2023-09-30T22:27:09.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.478+0000 7f33fce0a700 1 Processor -- start 2023-09-30T22:27:09.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.478+0000 7f33fce0a700 1 -- start start 2023-09-30T22:27:09.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.479+0000 7f33fce0a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f33f8067940 0x7f33f8138340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:09.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.479+0000 7f33fce0a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33f8096820 0x7f33f8138830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:09.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.479+0000 7f33fce0a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f33f80975e0 0x7f33f8138d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:09.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.479+0000 7f33fce0a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f811c0b0 con 0x7f33f8096820 2023-09-30T22:27:09.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.479+0000 7f33fce0a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33f8067e30 con 0x7f33f80975e0 2023-09-30T22:27:09.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.479+0000 7f33fce0a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f33f8091f60 con 0x7f33f8067940 2023-09-30T22:27:09.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.479+0000 7f33f6d9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f33f80975e0 0x7f33f8138d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:09.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.479+0000 7f33f5d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33f8096820 0x7f33f8138830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:09.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.479+0000 7f33f6d9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f33f80975e0 0x7f33f8138d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:49134/0 (socket says 172.21.15.16:49134) 2023-09-30T22:27:09.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.479+0000 7f33f6d9d700 1 -- 172.21.15.16:0/4061597297 learned_addr learned my addr 172.21.15.16:0/4061597297 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:09.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.479+0000 7f33f659c700 1 --2- 172.21.15.16:0/4061597297 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f33f8067940 0x7f33f8138340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:09.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.479+0000 7f33f5d9b700 1 -- 172.21.15.16:0/4061597297 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f33f8067940 msgr2=0x7f33f8138340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:09.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.479+0000 7f33f5d9b700 1 --2- 172.21.15.16:0/4061597297 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f33f8067940 0x7f33f8138340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:09.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.479+0000 7f33f5d9b700 1 -- 172.21.15.16:0/4061597297 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f33f80975e0 msgr2=0x7f33f8138d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:09.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.479+0000 7f33f5d9b700 1 --2- 172.21.15.16:0/4061597297 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f33f80975e0 0x7f33f8138d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:09.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.479+0000 7f33f5d9b700 1 -- 172.21.15.16:0/4061597297 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33ec0092a0 con 0x7f33f8096820 2023-09-30T22:27:09.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.479+0000 7f33f5d9b700 1 --2- 172.21.15.16:0/4061597297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33f8096820 0x7f33f8138830 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f33ec00e9b0 tx=0x7f33ec009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:09.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.480+0000 7f33e67fc700 1 -- 172.21.15.16:0/4061597297 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ec012600 con 0x7f33f8096820 2023-09-30T22:27:09.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.480+0000 7f33e67fc700 1 -- 172.21.15.16:0/4061597297 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33ec012760 con 0x7f33f8096820 2023-09-30T22:27:09.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.480+0000 7f33fce0a700 1 -- 172.21.15.16:0/4061597297 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33f0009a20 con 0x7f33f8096820 2023-09-30T22:27:09.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.480+0000 7f33e67fc700 1 -- 172.21.15.16:0/4061597297 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ec012ae0 con 0x7f33f8096820 2023-09-30T22:27:09.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.480+0000 7f33fce0a700 1 -- 172.21.15.16:0/4061597297 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33e8009370 con 0x7f33f8096820 2023-09-30T22:27:09.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.480+0000 7f33e67fc700 1 -- 172.21.15.16:0/4061597297 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f33ec01b400 con 0x7f33f8096820 2023-09-30T22:27:09.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.481+0000 7f33e67fc700 1 -- 172.21.15.16:0/4061597297 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f33ec03b430 con 0x7f33f8096820 2023-09-30T22:27:09.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.481+0000 7f33fce0a700 1 -- 172.21.15.16:0/4061597297 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33f8091a90 con 0x7f33f8096820 2023-09-30T22:27:09.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.485+0000 7f33e67fc700 1 -- 172.21.15.16:0/4061597297 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f33ec019900 con 0x7f33f8096820 2023-09-30T22:27:09.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.701+0000 7f33fce0a700 1 -- 172.21.15.16:0/4061597297 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f33f805a260 con 0x7f33f8096820 2023-09-30T22:27:09.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.701+0000 7f33e67fc700 1 -- 172.21.15.16:0/4061597297 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+2156 (secure 0 0 0) 0x7f33ec023350 con 0x7f33f8096820 2023-09-30T22:27:09.702 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:27:09.703 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-30T22:26:51.674561+0000","modified":"2023-09-30T22:26:54.680606+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4218},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4218":{"gid":4218,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6817/1424366610","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6817","nonce":1424366610}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-09-30T22:27:09.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.703+0000 7f33fce0a700 1 -- 172.21.15.16:0/4061597297 >> v1:172.21.15.16:6816/118285 conn(0x7f33d802bb60 legacy=0x7f33d802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:09.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.703+0000 7f33fce0a700 1 -- 172.21.15.16:0/4061597297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33f8096820 msgr2=0x7f33f8138830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:09.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.703+0000 7f33fce0a700 1 --2- 172.21.15.16:0/4061597297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33f8096820 0x7f33f8138830 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f33ec00e9b0 tx=0x7f33ec009060).stop 2023-09-30T22:27:09.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.703+0000 7f33fce0a700 1 -- 172.21.15.16:0/4061597297 shutdown_connections 2023-09-30T22:27:09.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.703+0000 7f33fce0a700 1 --2- 172.21.15.16:0/4061597297 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f33f80975e0 0x7f33f8138d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:09.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.703+0000 7f33fce0a700 1 --2- 172.21.15.16:0/4061597297 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f33f8067940 0x7f33f8138340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:09.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.703+0000 7f33fce0a700 1 --2- 172.21.15.16:0/4061597297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33f8096820 0x7f33f8138830 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:09.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.704+0000 7f33fce0a700 1 -- 172.21.15.16:0/4061597297 >> 172.21.15.16:0/4061597297 conn(0x7f33f809ba50 msgr2=0x7f33f8099970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:09.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.704+0000 7f33fce0a700 1 -- 172.21.15.16:0/4061597297 shutdown_connections 2023-09-30T22:27:09.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.704+0000 7f33fce0a700 1 -- 172.21.15.16:0/4061597297 wait complete. 2023-09-30T22:27:09.706 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 6 2023-09-30T22:27:09.818 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-09-30T22:26:51.674561+0000', 'modified': '2023-09-30T22:26:54.680606+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4218}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4218': {'gid': 4218, 'name': 'a', 'rank': 0, 'incarnation': 3, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.16:6817/1424366610', 'addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.16:6817', 'nonce': 1424366610}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-09-30T22:27:09.818 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-09-30T22:27:09.818 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-09-30T22:27:09.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.955+0000 7fb84b1c2700 1 Processor -- start 2023-09-30T22:27:09.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.955+0000 7fb84b1c2700 1 -- start start 2023-09-30T22:27:09.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.956+0000 7fb84b1c2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb844059070 0x7fb844127890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:09.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.956+0000 7fb84b1c2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb844127390 0x7fb844127d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:09.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.956+0000 7fb84b1c2700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb844091b60 con 0x7fb844095370 2023-09-30T22:27:09.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.956+0000 7fb84b1c2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb844096940 con 0x7fb844127390 2023-09-30T22:27:09.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.956+0000 7fb84b1c2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb844059c00 con 0x7fb844059070 2023-09-30T22:27:09.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.956+0000 7fb843fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb844127390 0x7fb844127d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:09.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.956+0000 7fb84975f700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb844095370 0x7fb8440957f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53240/0 (socket says 172.21.15.16:53240) 2023-09-30T22:27:09.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.956+0000 7fb84975f700 1 -- 172.21.15.16:0/3727534772 learned_addr learned my addr 172.21.15.16:0/3727534772 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:27:09.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.956+0000 7fb848f5e700 1 --2- 172.21.15.16:0/3727534772 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb844059070 0x7fb844127890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:09.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.956+0000 7fb843fff700 1 -- 172.21.15.16:0/3727534772 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb844059070 msgr2=0x7fb844127890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:09.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.956+0000 7fb843fff700 1 --2- 172.21.15.16:0/3727534772 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb844059070 0x7fb844127890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:09.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.956+0000 7fb8437fe700 1 -- 172.21.15.16:0/3727534772 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1620821073 0 0) 0x7fb844091b60 con 0x7fb844095370 2023-09-30T22:27:09.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.956+0000 7fb843fff700 1 -- 172.21.15.16:0/3727534772 >> v1:172.21.15.16:6789/0 conn(0x7fb844095370 legacy=0x7fb8440957f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:09.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.956+0000 7fb843fff700 1 -- 172.21.15.16:0/3727534772 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8440929d0 con 0x7fb844127390 2023-09-30T22:27:09.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.956+0000 7fb848f5e700 1 --2- 172.21.15.16:0/3727534772 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb844059070 0x7fb844127890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:09.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.957+0000 7fb843fff700 1 --2- 172.21.15.16:0/3727534772 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb844127390 0x7fb844127d80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7fb82c0097c0 tx=0x7fb82c009060).ready entity=mon.1 client_cookie=1cfb1919de6deee5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:09.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.957+0000 7fb8437fe700 1 -- 172.21.15.16:0/3727534772 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82c0090d0 con 0x7fb844127390 2023-09-30T22:27:09.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.957+0000 7fb8437fe700 1 -- 172.21.15.16:0/3727534772 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb82c00f510 con 0x7fb844127390 2023-09-30T22:27:09.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.957+0000 7fb8437fe700 1 -- 172.21.15.16:0/3727534772 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82c00f890 con 0x7fb844127390 2023-09-30T22:27:09.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.957+0000 7fb84b1c2700 1 -- 172.21.15.16:0/3727534772 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb844127390 msgr2=0x7fb844127d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:09.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.957+0000 7fb84b1c2700 1 --2- 172.21.15.16:0/3727534772 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb844127390 0x7fb844127d80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7fb82c0097c0 tx=0x7fb82c009060).stop 2023-09-30T22:27:09.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.957+0000 7fb84b1c2700 1 -- 172.21.15.16:0/3727534772 shutdown_connections 2023-09-30T22:27:09.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.957+0000 7fb84b1c2700 1 --2- 172.21.15.16:0/3727534772 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb844059070 0x7fb844127890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:09.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.957+0000 7fb84b1c2700 1 --2- 172.21.15.16:0/3727534772 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb844127390 0x7fb844127d80 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:09.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.957+0000 7fb84b1c2700 1 -- 172.21.15.16:0/3727534772 >> 172.21.15.16:0/3727534772 conn(0x7fb844097470 msgr2=0x7fb844097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:09.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.957+0000 7fb84b1c2700 1 -- 172.21.15.16:0/3727534772 shutdown_connections 2023-09-30T22:27:09.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.957+0000 7fb84b1c2700 1 -- 172.21.15.16:0/3727534772 wait complete. 2023-09-30T22:27:09.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.958+0000 7fb84b1c2700 1 Processor -- start 2023-09-30T22:27:09.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.959+0000 7fb84b1c2700 1 -- start start 2023-09-30T22:27:09.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.959+0000 7fb84b1c2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb844059070 0x7fb844117940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:09.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.959+0000 7fb84b1c2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb844095370 0x7fb844124a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:09.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.959+0000 7fb84b1c2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb844127390 0x7fb844113960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:09.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.959+0000 7fb84b1c2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb844125430 con 0x7fb844059070 2023-09-30T22:27:09.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.959+0000 7fb84b1c2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb844096f20 con 0x7fb844095370 2023-09-30T22:27:09.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.959+0000 7fb84b1c2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb844117440 con 0x7fb844127390 2023-09-30T22:27:09.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.959+0000 7fb848f5e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb844059070 0x7fb844117940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:09.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.959+0000 7fb848f5e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb844059070 0x7fb844117940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40872/0 (socket says 172.21.15.16:40872) 2023-09-30T22:27:09.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.959+0000 7fb848f5e700 1 -- 172.21.15.16:0/2364406273 learned_addr learned my addr 172.21.15.16:0/2364406273 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:09.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.959+0000 7fb84975f700 1 --2- 172.21.15.16:0/2364406273 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb844127390 0x7fb844113960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:09.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.959+0000 7fb843fff700 1 --2- 172.21.15.16:0/2364406273 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb844095370 0x7fb844124a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:09.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.959+0000 7fb848f5e700 1 -- 172.21.15.16:0/2364406273 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb844127390 msgr2=0x7fb844113960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:09.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.959+0000 7fb848f5e700 1 --2- 172.21.15.16:0/2364406273 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb844127390 0x7fb844113960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:09.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.960+0000 7fb848f5e700 1 -- 172.21.15.16:0/2364406273 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb844095370 msgr2=0x7fb844124a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:09.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.960+0000 7fb848f5e700 1 --2- 172.21.15.16:0/2364406273 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb844095370 0x7fb844124a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:09.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.960+0000 7fb848f5e700 1 -- 172.21.15.16:0/2364406273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8340092a0 con 0x7fb844059070 2023-09-30T22:27:09.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.960+0000 7fb84975f700 1 --2- 172.21.15.16:0/2364406273 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb844127390 0x7fb844113960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:09.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.960+0000 7fb848f5e700 1 --2- 172.21.15.16:0/2364406273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb844059070 0x7fb844117940 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7fb83400ea20 tx=0x7fb834009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:09.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.960+0000 7fb840ff9700 1 -- 172.21.15.16:0/2364406273 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83400ee60 con 0x7fb844059070 2023-09-30T22:27:09.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.960+0000 7fb840ff9700 1 -- 172.21.15.16:0/2364406273 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb834009600 con 0x7fb844059070 2023-09-30T22:27:09.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.960+0000 7fb840ff9700 1 -- 172.21.15.16:0/2364406273 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83400ca60 con 0x7fb844059070 2023-09-30T22:27:09.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.960+0000 7fb84b1c2700 1 -- 172.21.15.16:0/2364406273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb82c0092a0 con 0x7fb844059070 2023-09-30T22:27:09.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.960+0000 7fb84b1c2700 1 -- 172.21.15.16:0/2364406273 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8441156f0 con 0x7fb844059070 2023-09-30T22:27:09.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.961+0000 7fb840ff9700 1 -- 172.21.15.16:0/2364406273 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7fb83401b400 con 0x7fb844059070 2023-09-30T22:27:09.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.961+0000 7fb840ff9700 1 -- 172.21.15.16:0/2364406273 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fb83403b460 con 0x7fb844059070 2023-09-30T22:27:09.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.964+0000 7fb85cb75b80 1 Processor -- start 2023-09-30T22:27:09.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.964+0000 7fb85cb75b80 1 -- start start 2023-09-30T22:27:09.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.964+0000 7fb85cb75b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592f19ab170 0x5592f19ab550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:09.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.964+0000 7fb85cb75b80 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x5592f19aba40 0x5592f19d50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:09.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.964+0000 7fb85cb75b80 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x5592f19d55b0 0x5592f19d79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:09.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.964+0000 7fb85cb75b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592f1975610 con 0x5592f19ab170 2023-09-30T22:27:09.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.964+0000 7fb85cb75b80 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x5592f18ebdb0 con 0x5592f19d55b0 2023-09-30T22:27:09.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.964+0000 7fb85cb75b80 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x5592f1901030 con 0x5592f19aba40 2023-09-30T22:27:09.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.965+0000 7fb848f5e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592f19ab170 0x5592f19ab550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:09.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.965+0000 7fb84975f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x5592f19d55b0 0x5592f19d79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:09.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.965+0000 7fb848f5e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592f19ab170 0x5592f19ab550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40884/0 (socket says 172.21.15.16:40884) 2023-09-30T22:27:09.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.965+0000 7fb848f5e700 1 -- 172.21.15.16:0/3672542448 learned_addr learned my addr 172.21.15.16:0/3672542448 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:09.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.965+0000 7fb843fff700 1 --2- 172.21.15.16:0/3672542448 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x5592f19aba40 0x5592f19d50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:09.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.965+0000 7fb84975f700 1 -- 172.21.15.16:0/3672542448 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x5592f19aba40 msgr2=0x5592f19d50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:09.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.965+0000 7fb84975f700 1 --2- 172.21.15.16:0/3672542448 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x5592f19aba40 0x5592f19d50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:09.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.965+0000 7fb84975f700 1 -- 172.21.15.16:0/3672542448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592f19ab170 msgr2=0x5592f19ab550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:09.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.965+0000 7fb84975f700 1 --2- 172.21.15.16:0/3672542448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592f19ab170 0x5592f19ab550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:09.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.965+0000 7fb84975f700 1 -- 172.21.15.16:0/3672542448 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592f179c710 con 0x5592f19d55b0 2023-09-30T22:27:09.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.965+0000 7fb848f5e700 1 --2- 172.21.15.16:0/3672542448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592f19ab170 0x5592f19ab550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:09.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.965+0000 7fb84975f700 1 --2- 172.21.15.16:0/3672542448 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x5592f19d55b0 0x5592f19d79a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7fb83c007fd0 tx=0x7fb83c009040).ready entity=mon.1 client_cookie=a3d4db6ac7d1a186 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:09.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.965+0000 7fb839ffb700 1 -- 172.21.15.16:0/3672542448 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83c001d10 con 0x5592f19d55b0 2023-09-30T22:27:09.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.965+0000 7fb839ffb700 1 -- 172.21.15.16:0/3672542448 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb83c005d10 con 0x5592f19d55b0 2023-09-30T22:27:09.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.965+0000 7fb839ffb700 1 -- 172.21.15.16:0/3672542448 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83c00fd30 con 0x5592f19d55b0 2023-09-30T22:27:09.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.966+0000 7fb85cb75b80 1 -- 172.21.15.16:0/3672542448 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x5592f19d55b0 msgr2=0x5592f19d79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:09.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.966+0000 7fb85cb75b80 1 --2- 172.21.15.16:0/3672542448 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x5592f19d55b0 0x5592f19d79a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7fb83c007fd0 tx=0x7fb83c009040).stop 2023-09-30T22:27:09.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.966+0000 7fb85cb75b80 1 -- 172.21.15.16:0/3672542448 shutdown_connections 2023-09-30T22:27:09.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.966+0000 7fb85cb75b80 1 --2- 172.21.15.16:0/3672542448 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x5592f19d55b0 0x5592f19d79a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:09.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.966+0000 7fb85cb75b80 1 --2- 172.21.15.16:0/3672542448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592f19ab170 0x5592f19ab550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:09.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.966+0000 7fb85cb75b80 1 --2- 172.21.15.16:0/3672542448 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x5592f19aba40 0x5592f19d50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:09.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.966+0000 7fb85cb75b80 1 -- 172.21.15.16:0/3672542448 >> 172.21.15.16:0/3672542448 conn(0x5592f1819940 msgr2=0x5592f19cbcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:09.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.966+0000 7fb85cb75b80 1 -- 172.21.15.16:0/3672542448 shutdown_connections 2023-09-30T22:27:09.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.966+0000 7fb85cb75b80 1 -- 172.21.15.16:0/3672542448 wait complete. 2023-09-30T22:27:09.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.966+0000 7fb85cb75b80 1 -- start start 2023-09-30T22:27:09.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.967+0000 7fb85cb75b80 1 Processor -- start 2023-09-30T22:27:09.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.967+0000 7fb85cb75b80 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x5592f19ab170 0x5592f19dacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:09.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.967+0000 7fb85cb75b80 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x5592f19aba40 0x5592f19db1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:09.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.967+0000 7fb85cb75b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592f19d55b0 0x5592f19d0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:09.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.967+0000 7fb85cb75b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592f1901170 con 0x5592f19d55b0 2023-09-30T22:27:09.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.967+0000 7fb85cb75b80 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x5592f19013f0 con 0x5592f19aba40 2023-09-30T22:27:09.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.967+0000 7fb85cb75b80 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x5592f18cc920 con 0x5592f19ab170 2023-09-30T22:27:09.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.967+0000 7fb85cb75b80 10 client.0 ll_register_callbacks cb 0x5592f18a1ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T22:27:09.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.967+0000 7fb84975f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592f19d55b0 0x5592f19d0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:09.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.967+0000 7fb848f5e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x5592f19ab170 0x5592f19dacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:09.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.968+0000 7fb843fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x5592f19aba40 0x5592f19db1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:09.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.968+0000 7fb84975f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592f19d55b0 0x5592f19d0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40900/0 (socket says 172.21.15.16:40900) 2023-09-30T22:27:09.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.968+0000 7fb848f5e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x5592f19ab170 0x5592f19dacf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37680/0 (socket says 172.21.15.16:37680) 2023-09-30T22:27:09.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.968+0000 7fb84975f700 1 -- 172.21.15.16:0/2972799233 learned_addr learned my addr 172.21.15.16:0/2972799233 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:09.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.968+0000 7fb84975f700 1 -- 172.21.15.16:0/2972799233 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x5592f19ab170 msgr2=0x5592f19dacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:09.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.968+0000 7fb84975f700 1 --2- 172.21.15.16:0/2972799233 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x5592f19ab170 0x5592f19dacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:09.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.968+0000 7fb84975f700 1 -- 172.21.15.16:0/2972799233 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x5592f19aba40 msgr2=0x5592f19db1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:09.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.968+0000 7fb84975f700 1 --2- 172.21.15.16:0/2972799233 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x5592f19aba40 0x5592f19db1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:09.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.968+0000 7fb84975f700 1 -- 172.21.15.16:0/2972799233 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb83c001f00 con 0x5592f19d55b0 2023-09-30T22:27:09.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.968+0000 7fb848f5e700 1 --2- 172.21.15.16:0/2972799233 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x5592f19ab170 0x5592f19dacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:09.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.968+0000 7fb843fff700 1 --2- 172.21.15.16:0/2972799233 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x5592f19aba40 0x5592f19db1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:09.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.968+0000 7fb84975f700 1 --2- 172.21.15.16:0/2972799233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592f19d55b0 0x5592f19d0960 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7fb83c007000 tx=0x7fb83c018000).ready entity=mon.0 client_cookie=1f9ddd546be49a02 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:09.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.968+0000 7fb822ffd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-09-30T22:27:09.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.968+0000 7fb822ffd700 1 -- 172.21.15.16:0/2972799233 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83c0075c0 con 0x5592f19d55b0 2023-09-30T22:27:09.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.968+0000 7fb822ffd700 1 -- 172.21.15.16:0/2972799233 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(16..16 src has 1..16) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fb83c007720 con 0x5592f19d55b0 2023-09-30T22:27:09.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.968+0000 7fb85cb75b80 1 -- 172.21.15.16:0/2972799233 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5592f19a3f20 con 0x5592f19d55b0 2023-09-30T22:27:09.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.968+0000 7fb822ffd700 1 -- 172.21.15.16:0/2972799233 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb83c01f400 con 0x5592f19d55b0 2023-09-30T22:27:09.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.968+0000 7fb85cb75b80 10 client.4638 fetch_fsmap learned FSMap version 6 2023-09-30T22:27:09.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.968+0000 7fb85cb75b80 1 -- 172.21.15.16:0/2972799233 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x5592f18d3b00 con 0x5592f19d55b0 2023-09-30T22:27:09.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.969+0000 7fb822ffd700 1 -- 172.21.15.16:0/2972799233 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5592f19a3f20 con 0x5592f19d55b0 2023-09-30T22:27:09.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.969+0000 7fb822ffd700 1 -- 172.21.15.16:0/2972799233 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 6) v1 ==== 1430+0+0 (secure 0 0 0) 0x7fb83c012070 con 0x5592f19d55b0 2023-09-30T22:27:09.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.969+0000 7fb85cb75b80 10 client.4638 fetch_fsmap finished waiting for FSMap version 6 2023-09-30T22:27:09.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.969+0000 7fb85cb75b80 10 client.4638 resolve_mds: resolved name 'a' to daemon mds.a 2023-09-30T22:27:09.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.969+0000 7fb85cb75b80 20 client.4638 populate_metadata read hostname 'smithi016' 2023-09-30T22:27:09.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.969+0000 7fb85cb75b80 4 client.4638 mds_command: new command op to 4218 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-30T22:27:09.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.969+0000 7fb85cb75b80 1 -- 172.21.15.16:0/2972799233 --> v1:172.21.15.16:6817/1424366610 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5592f1979250 con 0x5592f19d1470 2023-09-30T22:27:09.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.969+0000 7fb822ffd700 10 client.4638 ms_handle_connect on v1:172.21.15.16:6817/1424366610 2023-09-30T22:27:09.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.970+0000 7fb822ffd700 1 -- 172.21.15.16:0/2972799233 <== mds.0 v1:172.21.15.16:6817/1424366610 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (unknown 0 0 3608088055) 0x5592f1979250 con 0x5592f19d1470 2023-09-30T22:27:09.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.970+0000 7fb822ffd700 10 client.4638 handle_command_reply: tid=0 2023-09-30T22:27:09.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.970+0000 7fb85cb75b80 1 client.4638 shutdown 2023-09-30T22:27:09.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.970+0000 7fb85cb75b80 1 -- 172.21.15.16:0/2972799233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592f19d55b0 msgr2=0x5592f19d0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:09.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.970+0000 7fb85cb75b80 1 --2- 172.21.15.16:0/2972799233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592f19d55b0 0x5592f19d0960 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7fb83c007000 tx=0x7fb83c018000).stop 2023-09-30T22:27:09.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.970+0000 7fb85cb75b80 1 -- 172.21.15.16:0/2972799233 shutdown_connections 2023-09-30T22:27:09.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.970+0000 7fb85cb75b80 1 --2- 172.21.15.16:0/2972799233 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592f19d55b0 0x5592f19d0960 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:09.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.970+0000 7fb85cb75b80 1 --2- 172.21.15.16:0/2972799233 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x5592f19ab170 0x5592f19dacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:09.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.970+0000 7fb822ffd700 0 client.4638 ms_handle_reset on v1:172.21.15.16:6817/1424366610 2023-09-30T22:27:09.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.970+0000 7fb85cb75b80 1 --2- 172.21.15.16:0/2972799233 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x5592f19aba40 0x5592f19db1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:09.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.970+0000 7fb85cb75b80 1 -- 172.21.15.16:0/2972799233 >> 172.21.15.16:0/2972799233 conn(0x5592f1819940 msgr2=0x5592f19cbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:09.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.970+0000 7fb85cb75b80 1 -- 172.21.15.16:0/2972799233 shutdown_connections 2023-09-30T22:27:09.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.970+0000 7fb85cb75b80 1 -- 172.21.15.16:0/2972799233 wait complete. 2023-09-30T22:27:09.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.970+0000 7fb85cb75b80 20 client.4638 trim_cache size 0 max 16384 2023-09-30T22:27:09.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.987+0000 7fb85cb75b80 1 Processor -- start 2023-09-30T22:27:09.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.987+0000 7fb85cb75b80 1 -- start start 2023-09-30T22:27:09.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.988+0000 7fb85cb75b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592f19ab170 0x5592f19abe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:09.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.988+0000 7fb85cb75b80 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x5592f19aba40 0x5592f19ad3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:09.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.988+0000 7fb85cb75b80 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x5592f19d55b0 0x5592f19cbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:09.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.988+0000 7fb85cb75b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592f19704c0 con 0x5592f19ab170 2023-09-30T22:27:09.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.988+0000 7fb85cb75b80 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x5592f19aaf70 con 0x5592f19aba40 2023-09-30T22:27:09.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.988+0000 7fb85cb75b80 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x5592f19d43f0 con 0x5592f19d55b0 2023-09-30T22:27:09.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.988+0000 7fb843fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x5592f19aba40 0x5592f19ad3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:09.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.988+0000 7fb848f5e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592f19ab170 0x5592f19abe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:09.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.988+0000 7fb843fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x5592f19aba40 0x5592f19ad3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:49184/0 (socket says 172.21.15.16:49184) 2023-09-30T22:27:09.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.988+0000 7fb843fff700 1 -- 172.21.15.16:0/3781508274 learned_addr learned my addr 172.21.15.16:0/3781508274 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:09.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.988+0000 7fb84975f700 1 --2- 172.21.15.16:0/3781508274 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x5592f19d55b0 0x5592f19cbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:09.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.988+0000 7fb848f5e700 1 -- 172.21.15.16:0/3781508274 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x5592f19d55b0 msgr2=0x5592f19cbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:09.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.988+0000 7fb848f5e700 1 --2- 172.21.15.16:0/3781508274 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x5592f19d55b0 0x5592f19cbbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:09.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.988+0000 7fb848f5e700 1 -- 172.21.15.16:0/3781508274 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x5592f19aba40 msgr2=0x5592f19ad3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:09.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.988+0000 7fb848f5e700 1 --2- 172.21.15.16:0/3781508274 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x5592f19aba40 0x5592f19ad3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:09.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.988+0000 7fb848f5e700 1 -- 172.21.15.16:0/3781508274 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592f18f9f70 con 0x5592f19ab170 2023-09-30T22:27:09.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.988+0000 7fb843fff700 1 --2- 172.21.15.16:0/3781508274 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x5592f19aba40 0x5592f19ad3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:09.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.988+0000 7fb848f5e700 1 --2- 172.21.15.16:0/3781508274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592f19ab170 0x5592f19abe20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7fb834000c00 tx=0x7fb834018000).ready entity=mon.0 client_cookie=4ee74dcb2bacb536 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:09.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.989+0000 7fb83a7fc700 1 -- 172.21.15.16:0/3781508274 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb834031070 con 0x5592f19ab170 2023-09-30T22:27:09.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.989+0000 7fb83a7fc700 1 -- 172.21.15.16:0/3781508274 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb83401bca0 con 0x5592f19ab170 2023-09-30T22:27:09.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.989+0000 7fb83a7fc700 1 -- 172.21.15.16:0/3781508274 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb834011900 con 0x5592f19ab170 2023-09-30T22:27:09.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.989+0000 7fb85cb75b80 1 -- 172.21.15.16:0/3781508274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592f19ab170 msgr2=0x5592f19abe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:09.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.989+0000 7fb85cb75b80 1 --2- 172.21.15.16:0/3781508274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592f19ab170 0x5592f19abe20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7fb834000c00 tx=0x7fb834018000).stop 2023-09-30T22:27:09.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.989+0000 7fb85cb75b80 1 -- 172.21.15.16:0/3781508274 shutdown_connections 2023-09-30T22:27:09.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.989+0000 7fb85cb75b80 1 --2- 172.21.15.16:0/3781508274 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x5592f19d55b0 0x5592f19cbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:09.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.989+0000 7fb85cb75b80 1 --2- 172.21.15.16:0/3781508274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592f19ab170 0x5592f19abe20 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:09.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.989+0000 7fb85cb75b80 1 --2- 172.21.15.16:0/3781508274 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x5592f19aba40 0x5592f19ad3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:09.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.989+0000 7fb85cb75b80 1 -- 172.21.15.16:0/3781508274 >> 172.21.15.16:0/3781508274 conn(0x5592f1819940 msgr2=0x5592f19af570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:09.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.989+0000 7fb85cb75b80 1 -- 172.21.15.16:0/3781508274 shutdown_connections 2023-09-30T22:27:09.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.989+0000 7fb85cb75b80 1 -- 172.21.15.16:0/3781508274 wait complete. 2023-09-30T22:27:09.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.990+0000 7fb85cb75b80 1 -- start start 2023-09-30T22:27:09.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.990+0000 7fb85cb75b80 1 Processor -- start 2023-09-30T22:27:09.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.991+0000 7fb85cb75b80 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x5592f19ab170 0x5592f1a10650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:09.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.991+0000 7fb85cb75b80 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x5592f19aba40 0x5592f1a10b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:09.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.991+0000 7fb85cb75b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592f19d55b0 0x5592f1a11030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:09.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.991+0000 7fb85cb75b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592f1a14250 con 0x5592f19d55b0 2023-09-30T22:27:09.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.991+0000 7fb85cb75b80 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x5592f1a133d0 con 0x5592f19aba40 2023-09-30T22:27:09.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.991+0000 7fb85cb75b80 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x5592f19acb80 con 0x5592f19ab170 2023-09-30T22:27:09.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.991+0000 7fb85cb75b80 10 client.0 ll_register_callbacks cb 0x5592f18a1ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T22:27:09.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.991+0000 7fb843fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x5592f19aba40 0x5592f1a10b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:09.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.991+0000 7fb843fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x5592f19aba40 0x5592f1a10b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:49186/0 (socket says 172.21.15.16:49186) 2023-09-30T22:27:09.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.991+0000 7fb843fff700 1 -- 172.21.15.16:0/1581691945 learned_addr learned my addr 172.21.15.16:0/1581691945 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:09.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.991+0000 7fb84975f700 1 --2- 172.21.15.16:0/1581691945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592f19d55b0 0x5592f1a11030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:09.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.991+0000 7fb848f5e700 1 --2- 172.21.15.16:0/1581691945 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x5592f19ab170 0x5592f1a10650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:09.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.991+0000 7fb843fff700 1 -- 172.21.15.16:0/1581691945 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x5592f19ab170 msgr2=0x5592f1a10650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:09.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.991+0000 7fb843fff700 1 --2- 172.21.15.16:0/1581691945 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x5592f19ab170 0x5592f1a10650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:09.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.991+0000 7fb843fff700 1 -- 172.21.15.16:0/1581691945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592f19d55b0 msgr2=0x5592f1a11030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:09.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.991+0000 7fb843fff700 1 --2- 172.21.15.16:0/1581691945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592f19d55b0 0x5592f1a11030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:09.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.991+0000 7fb843fff700 1 -- 172.21.15.16:0/1581691945 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8440929d0 con 0x5592f19aba40 2023-09-30T22:27:09.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.991+0000 7fb84975f700 1 --2- 172.21.15.16:0/1581691945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592f19d55b0 0x5592f1a11030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:09.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.991+0000 7fb848f5e700 1 --2- 172.21.15.16:0/1581691945 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x5592f19ab170 0x5592f1a10650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:27:09.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.991+0000 7fb843fff700 1 --2- 172.21.15.16:0/1581691945 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x5592f19aba40 0x5592f1a10b40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7fb844097340 tx=0x7fb82c017000).ready entity=mon.1 client_cookie=a45b82525626577b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:09.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.991+0000 7fb822ffd700 10 client.0 ms_handle_connect on v2:172.21.15.16:3301/0 2023-09-30T22:27:09.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.991+0000 7fb822ffd700 1 -- 172.21.15.16:0/1581691945 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82c01e440 con 0x5592f19aba40 2023-09-30T22:27:09.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.991+0000 7fb822ffd700 1 -- 172.21.15.16:0/1581691945 <== mon.1 v2:172.21.15.16:3301/0 2 ==== osd_map(16..16 src has 1..16) v4 ==== 3690+0+0 (secure 0 0 0) 0x7fb82c01d710 con 0x5592f19aba40 2023-09-30T22:27:09.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.991+0000 7fb85cb75b80 1 -- 172.21.15.16:0/1581691945 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5592f19c6660 con 0x5592f19aba40 2023-09-30T22:27:09.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.992+0000 7fb822ffd700 1 -- 172.21.15.16:0/1581691945 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb82c01dae0 con 0x5592f19aba40 2023-09-30T22:27:09.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.992+0000 7fb85cb75b80 10 client.4468 fetch_fsmap learned FSMap version 6 2023-09-30T22:27:09.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.992+0000 7fb85cb75b80 1 -- 172.21.15.16:0/1581691945 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x7fb82c0092a0 con 0x5592f19aba40 2023-09-30T22:27:09.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.992+0000 7fb822ffd700 1 -- 172.21.15.16:0/1581691945 <== mon.1 v2:172.21.15.16:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5592f19c6660 con 0x5592f19aba40 2023-09-30T22:27:09.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.992+0000 7fb822ffd700 1 -- 172.21.15.16:0/1581691945 <== mon.1 v2:172.21.15.16:3301/0 5 ==== fsmap(e 6) v1 ==== 1430+0+0 (secure 0 0 0) 0x7fb82c0097f0 con 0x5592f19aba40 2023-09-30T22:27:09.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.992+0000 7fb85cb75b80 10 client.4468 fetch_fsmap finished waiting for FSMap version 6 2023-09-30T22:27:09.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.992+0000 7fb85cb75b80 10 client.4468 resolve_mds: resolved name 'a' to daemon mds.a 2023-09-30T22:27:09.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.992+0000 7fb85cb75b80 20 client.4468 populate_metadata read hostname 'smithi016' 2023-09-30T22:27:09.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.992+0000 7fb85cb75b80 4 client.4468 mds_command: new command op to 4218 tid=0[{"prefix": "status"}] 2023-09-30T22:27:09.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.992+0000 7fb85cb75b80 1 -- 172.21.15.16:0/1581691945 --> v1:172.21.15.16:6817/1424366610 -- command(tid 0: {"prefix": "status"}) v1 -- 0x7fb83c005d10 con 0x5592f1a30570 2023-09-30T22:27:09.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.992+0000 7fb822ffd700 10 client.4468 ms_handle_connect on v1:172.21.15.16:6817/1424366610 2023-09-30T22:27:09.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.993+0000 7fb822ffd700 1 -- 172.21.15.16:0/1581691945 <== mds.0 v1:172.21.15.16:6817/1424366610 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (unknown 0 0 1776852651) 0x7fb83c005d10 con 0x5592f1a30570 2023-09-30T22:27:09.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.993+0000 7fb822ffd700 10 client.4468 handle_command_reply: tid=0 2023-09-30T22:27:09.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.993+0000 7fb85cb75b80 1 client.4468 shutdown 2023-09-30T22:27:09.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.993+0000 7fb85cb75b80 1 -- 172.21.15.16:0/1581691945 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x5592f19aba40 msgr2=0x5592f1a10b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:09.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.993+0000 7fb85cb75b80 1 --2- 172.21.15.16:0/1581691945 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x5592f19aba40 0x5592f1a10b40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7fb844097340 tx=0x7fb82c017000).stop 2023-09-30T22:27:09.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.993+0000 7fb85cb75b80 1 -- 172.21.15.16:0/1581691945 shutdown_connections 2023-09-30T22:27:09.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.993+0000 7fb85cb75b80 1 --2- 172.21.15.16:0/1581691945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592f19d55b0 0x5592f1a11030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:09.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.993+0000 7fb85cb75b80 1 --2- 172.21.15.16:0/1581691945 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x5592f19ab170 0x5592f1a10650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:09.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.993+0000 7fb85cb75b80 1 --2- 172.21.15.16:0/1581691945 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x5592f19aba40 0x5592f1a10b40 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:10.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.993+0000 7fb822ffd700 0 client.4468 ms_handle_reset on v1:172.21.15.16:6817/1424366610 2023-09-30T22:27:10.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.993+0000 7fb85cb75b80 1 -- 172.21.15.16:0/1581691945 >> 172.21.15.16:0/1581691945 conn(0x5592f1819940 msgr2=0x5592f19af420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:10.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.993+0000 7fb85cb75b80 1 -- 172.21.15.16:0/1581691945 shutdown_connections 2023-09-30T22:27:10.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.993+0000 7fb85cb75b80 1 -- 172.21.15.16:0/1581691945 wait complete. 2023-09-30T22:27:10.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.993+0000 7fb85cb75b80 20 client.4468 trim_cache size 0 max 16384 2023-09-30T22:27:10.000 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-09-30T22:27:10.000 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "7ede3e14-2bd0-4726-9437-6d13c90cd342", 2023-09-30T22:27:10.000 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2023-09-30T22:27:10.000 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 4218, 2023-09-30T22:27:10.001 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2023-09-30T22:27:10.001 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2023-09-30T22:27:10.001 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2023-09-30T22:27:10.001 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 18.317820077, 2023-09-30T22:27:10.001 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 6, 2023-09-30T22:27:10.001 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 16, 2023-09-30T22:27:10.001 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 16, 2023-09-30T22:27:10.001 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 23.642413734000002 2023-09-30T22:27:10.001 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-09-30T22:27:10.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.994+0000 7fb84b1c2700 1 -- 172.21.15.16:0/2364406273 >> v1:172.21.15.16:6816/118285 conn(0x7fb82402bb60 legacy=0x7fb82402e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:10.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.994+0000 7fb84b1c2700 1 -- 172.21.15.16:0/2364406273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb844059070 msgr2=0x7fb844117940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:10.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.994+0000 7fb84b1c2700 1 --2- 172.21.15.16:0/2364406273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb844059070 0x7fb844117940 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7fb83400ea20 tx=0x7fb834009060).stop 2023-09-30T22:27:10.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.994+0000 7fb84b1c2700 1 -- 172.21.15.16:0/2364406273 shutdown_connections 2023-09-30T22:27:10.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.994+0000 7fb84b1c2700 1 --2- 172.21.15.16:0/2364406273 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb844127390 0x7fb844113960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:10.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.994+0000 7fb84b1c2700 1 --2- 172.21.15.16:0/2364406273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb844059070 0x7fb844117940 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:10.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.994+0000 7fb84b1c2700 1 --2- 172.21.15.16:0/2364406273 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb844095370 0x7fb844124a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:10.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.994+0000 7fb84b1c2700 1 -- 172.21.15.16:0/2364406273 >> 172.21.15.16:0/2364406273 conn(0x7fb844097470 msgr2=0x7fb8440941d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:10.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.994+0000 7fb84b1c2700 1 -- 172.21.15.16:0/2364406273 shutdown_connections 2023-09-30T22:27:10.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:09.994+0000 7fb84b1c2700 1 -- 172.21.15.16:0/2364406273 wait complete. 2023-09-30T22:27:10.007 INFO:teuthology.run_tasks:Running task exec... 2023-09-30T22:27:10.016 INFO:teuthology.task.exec:Executing custom commands... 2023-09-30T22:27:10.016 INFO:teuthology.task.exec:Running commands on role osd.0 host ubuntu@smithi016.front.sepia.ceph.com 2023-09-30T22:27:10.016 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph osd set-require-min-compat-client pacific' 2023-09-30T22:27:10.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.144+0000 7f8cf1c18700 1 Processor -- start 2023-09-30T22:27:10.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.144+0000 7f8cf1c18700 1 -- start start 2023-09-30T22:27:10.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.144+0000 7f8cf1c18700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8cec127e70 0x7f8cec12fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:10.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.144+0000 7f8cf1c18700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8cec130170 0x7f8cec132560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:10.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.144+0000 7f8cf1c18700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cec092ab0 con 0x7f8cec062650 2023-09-30T22:27:10.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.144+0000 7f8cf1c18700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cec062a30 con 0x7f8cec130170 2023-09-30T22:27:10.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.144+0000 7f8cf1c18700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8cec1282b0 con 0x7f8cec127e70 2023-09-30T22:27:10.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.145+0000 7f8cebfff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8cec130170 0x7f8cec132560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:10.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.145+0000 7f8ceaffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8cec127e70 0x7f8cec12fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:10.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.145+0000 7f8ceb7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8cec062650 0x7f8cec127760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53246/0 (socket says 172.21.15.16:53246) 2023-09-30T22:27:10.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.145+0000 7f8ceb7fe700 1 -- 172.21.15.16:0/999373095 learned_addr learned my addr 172.21.15.16:0/999373095 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:27:10.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.145+0000 7f8cebfff700 1 -- 172.21.15.16:0/999373095 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8cec127e70 msgr2=0x7f8cec12fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:10.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.145+0000 7f8cebfff700 1 --2- 172.21.15.16:0/999373095 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8cec127e70 0x7f8cec12fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:10.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.145+0000 7f8cea7fc700 1 -- 172.21.15.16:0/999373095 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4203442829 0 0) 0x7f8cdc001110 con 0x7f8cec062650 2023-09-30T22:27:10.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.145+0000 7f8cebfff700 1 -- 172.21.15.16:0/999373095 >> v1:172.21.15.16:6789/0 conn(0x7f8cec062650 legacy=0x7f8cec127760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:10.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.145+0000 7f8cebfff700 1 -- 172.21.15.16:0/999373095 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cec091b70 con 0x7f8cec130170 2023-09-30T22:27:10.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.145+0000 7f8ceaffd700 1 --2- 172.21.15.16:0/999373095 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8cec127e70 0x7f8cec12fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:10.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.145+0000 7f8cebfff700 1 --2- 172.21.15.16:0/999373095 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8cec130170 0x7f8cec132560 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f8ce4009e90 tx=0x7f8ce4009780).ready entity=mon.1 client_cookie=dfd2df5e9b45a48a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:10.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.145+0000 7f8cea7fc700 1 -- 172.21.15.16:0/999373095 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce400e070 con 0x7f8cec130170 2023-09-30T22:27:10.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.145+0000 7f8cea7fc700 1 -- 172.21.15.16:0/999373095 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ce400bb00 con 0x7f8cec130170 2023-09-30T22:27:10.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.145+0000 7f8cea7fc700 1 -- 172.21.15.16:0/999373095 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce400be80 con 0x7f8cec130170 2023-09-30T22:27:10.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.146+0000 7f8cf1c18700 1 -- 172.21.15.16:0/999373095 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8cec130170 msgr2=0x7f8cec132560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:10.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.146+0000 7f8cf1c18700 1 --2- 172.21.15.16:0/999373095 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8cec130170 0x7f8cec132560 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f8ce4009e90 tx=0x7f8ce4009780).stop 2023-09-30T22:27:10.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.146+0000 7f8cf1c18700 1 -- 172.21.15.16:0/999373095 shutdown_connections 2023-09-30T22:27:10.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.146+0000 7f8cf1c18700 1 --2- 172.21.15.16:0/999373095 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8cec130170 0x7f8cec132560 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:10.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.146+0000 7f8cf1c18700 1 --2- 172.21.15.16:0/999373095 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8cec127e70 0x7f8cec12fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:10.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.146+0000 7f8cf1c18700 1 -- 172.21.15.16:0/999373095 >> 172.21.15.16:0/999373095 conn(0x7f8cec059b00 msgr2=0x7f8cec097410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:10.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.146+0000 7f8cf1c18700 1 -- 172.21.15.16:0/999373095 shutdown_connections 2023-09-30T22:27:10.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.146+0000 7f8cf1c18700 1 -- 172.21.15.16:0/999373095 wait complete. 2023-09-30T22:27:10.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.147+0000 7f8cf1c18700 1 Processor -- start 2023-09-30T22:27:10.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.147+0000 7f8cf1c18700 1 -- start start 2023-09-30T22:27:10.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.148+0000 7f8cf1c18700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cec062650 0x7f8cec12eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:10.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.148+0000 7f8cf1c18700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8cec127e70 0x7f8cec110fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:10.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.148+0000 7f8cf1c18700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8cec130170 0x7f8cec1114c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:10.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.148+0000 7f8cf1c18700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cec113350 con 0x7f8cec062650 2023-09-30T22:27:10.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.148+0000 7f8cf1c18700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cec09a3a0 con 0x7f8cec130170 2023-09-30T22:27:10.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.148+0000 7f8cf1c18700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8cec12fca0 con 0x7f8cec127e70 2023-09-30T22:27:10.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.148+0000 7f8ceb7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cec062650 0x7f8cec12eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:10.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.148+0000 7f8ceb7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cec062650 0x7f8cec12eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40934/0 (socket says 172.21.15.16:40934) 2023-09-30T22:27:10.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.148+0000 7f8ceb7fe700 1 -- 172.21.15.16:0/373032053 learned_addr learned my addr 172.21.15.16:0/373032053 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:10.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.148+0000 7f8cebfff700 1 --2- 172.21.15.16:0/373032053 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8cec130170 0x7f8cec1114c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:10.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.148+0000 7f8ceaffd700 1 --2- 172.21.15.16:0/373032053 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8cec127e70 0x7f8cec110fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:10.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.148+0000 7f8cebfff700 1 -- 172.21.15.16:0/373032053 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8cec127e70 msgr2=0x7f8cec110fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:10.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.148+0000 7f8cebfff700 1 --2- 172.21.15.16:0/373032053 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8cec127e70 0x7f8cec110fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:10.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.148+0000 7f8cebfff700 1 -- 172.21.15.16:0/373032053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cec062650 msgr2=0x7f8cec12eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:10.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.148+0000 7f8cebfff700 1 --2- 172.21.15.16:0/373032053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cec062650 0x7f8cec12eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:10.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.148+0000 7f8cebfff700 1 -- 172.21.15.16:0/373032053 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ce00092a0 con 0x7f8cec130170 2023-09-30T22:27:10.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.148+0000 7f8ceaffd700 1 --2- 172.21.15.16:0/373032053 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8cec127e70 0x7f8cec110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:10.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.148+0000 7f8ceb7fe700 1 --2- 172.21.15.16:0/373032053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cec062650 0x7f8cec12eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:10.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.148+0000 7f8cebfff700 1 --2- 172.21.15.16:0/373032053 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8cec130170 0x7f8cec1114c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f8ce4014000 tx=0x7f8cec092530).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:10.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.148+0000 7f8cd37fe700 1 -- 172.21.15.16:0/373032053 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce40070a0 con 0x7f8cec130170 2023-09-30T22:27:10.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.148+0000 7f8cd37fe700 1 -- 172.21.15.16:0/373032053 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ce4013e60 con 0x7f8cec130170 2023-09-30T22:27:10.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.148+0000 7f8cf1c18700 1 -- 172.21.15.16:0/373032053 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ce4009a20 con 0x7f8cec130170 2023-09-30T22:27:10.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.148+0000 7f8cd37fe700 1 -- 172.21.15.16:0/373032053 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ce401d8a0 con 0x7f8cec130170 2023-09-30T22:27:10.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.149+0000 7f8cf1c18700 1 -- 172.21.15.16:0/373032053 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cec1139a0 con 0x7f8cec130170 2023-09-30T22:27:10.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.149+0000 7f8cd37fe700 1 -- 172.21.15.16:0/373032053 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f8ce4003040 con 0x7f8cec130170 2023-09-30T22:27:10.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.150+0000 7f8cd37fe700 1 -- 172.21.15.16:0/373032053 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f8ce403cc00 con 0x7f8cec130170 2023-09-30T22:27:10.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.150+0000 7f8cf1c18700 1 -- 172.21.15.16:0/373032053 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cec113540 con 0x7f8cec130170 2023-09-30T22:27:10.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.154+0000 7f8cd37fe700 1 -- 172.21.15.16:0/373032053 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f8cec113540 con 0x7f8cec130170 2023-09-30T22:27:10.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:10.345+0000 7f8cf1c18700 1 -- 172.21.15.16:0/373032053 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd set-require-min-compat-client", "version": "pacific"} v 0) v1 -- 0x7f8cec113540 con 0x7f8cec130170 2023-09-30T22:27:11.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.250+0000 7f8cd37fe700 1 -- 172.21.15.16:0/373032053 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd set-require-min-compat-client", "version": "pacific"}]=0 set require_min_compat_client to pacific v17) v1 ==== 143+0+0 (secure 0 0 0) 0x7f8cec113540 con 0x7f8cec130170 2023-09-30T22:27:11.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.254+0000 7f8cf1c18700 1 -- 172.21.15.16:0/373032053 >> v1:172.21.15.16:6816/118285 conn(0x7f8cd802bb90 legacy=0x7f8cd802e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:11.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.254+0000 7f8cf1c18700 1 -- 172.21.15.16:0/373032053 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8cec130170 msgr2=0x7f8cec1114c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:11.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.254+0000 7f8cf1c18700 1 --2- 172.21.15.16:0/373032053 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8cec130170 0x7f8cec1114c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f8ce4014000 tx=0x7f8cec092530).stop 2023-09-30T22:27:11.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.254+0000 7f8cf1c18700 1 -- 172.21.15.16:0/373032053 shutdown_connections 2023-09-30T22:27:11.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.254+0000 7f8cf1c18700 1 --2- 172.21.15.16:0/373032053 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8cec130170 0x7f8cec1114c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:11.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.254+0000 7f8cf1c18700 1 --2- 172.21.15.16:0/373032053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cec062650 0x7f8cec12eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:11.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.254+0000 7f8cf1c18700 1 --2- 172.21.15.16:0/373032053 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8cec127e70 0x7f8cec110fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:11.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.254+0000 7f8cf1c18700 1 -- 172.21.15.16:0/373032053 >> 172.21.15.16:0/373032053 conn(0x7f8cec059b00 msgr2=0x7f8cec12a730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:11.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.254+0000 7f8cf1c18700 1 -- 172.21.15.16:0/373032053 shutdown_connections 2023-09-30T22:27:11.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.254+0000 7f8cf1c18700 1 -- 172.21.15.16:0/373032053 wait complete. 2023-09-30T22:27:11.255 INFO:teuthology.orchestra.run.smithi016.stderr:set require_min_compat_client to pacific 2023-09-30T22:27:11.269 INFO:teuthology.run_tasks:Running task print... 2023-09-30T22:27:11.278 INFO:teuthology.task.print:**** done ceph 2023-09-30T22:27:11.279 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-09-30T22:27:11.309 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-09-30T22:27:11.309 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2023-09-30T22:27:11.309 INFO:tasks.ceph_fuse:client.0 config is {} 2023-09-30T22:27:11.309 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-09-30T22:27:11.309 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-09-30T22:27:11.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:27:11.310 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2023-09-30T22:27:11.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:27:11.335 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link delete ceph-brx 2023-09-30T22:27:11.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T22:27:11.414 INFO:teuthology.orchestra.run.smithi027.stderr:Cannot find device "ceph-brx" 2023-09-30T22:27:11.415 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-09-30T22:27:11.415 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-09-30T22:27:11.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.553+0000 7feb3d334700 1 Processor -- start 2023-09-30T22:27:11.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.553+0000 7feb3d334700 1 -- start start 2023-09-30T22:27:11.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.554+0000 7feb3d334700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7feb38096380 0x7feb38094730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:11.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.554+0000 7feb3d334700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7feb38094c20 0x7feb38127d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:11.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.554+0000 7feb3d334700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb38096760 con 0x7feb38091b00 2023-09-30T22:27:11.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.554+0000 7feb3d334700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb38091ff0 con 0x7feb38096380 2023-09-30T22:27:11.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.554+0000 7feb3d334700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb38092c80 con 0x7feb38094c20 2023-09-30T22:27:11.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.554+0000 7feb36ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7feb38096380 0x7feb38094730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:11.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.554+0000 7feb377fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7feb38094c20 0x7feb38127d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:11.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.554+0000 7feb367fc700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7feb38091b00 0x7feb38127600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53254/0 (socket says 172.21.15.16:53254) 2023-09-30T22:27:11.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.554+0000 7feb36ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7feb38096380 0x7feb38094730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40946/0 (socket says 172.21.15.16:40946) 2023-09-30T22:27:11.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.554+0000 7feb367fc700 1 -- 172.21.15.16:0/1676662570 learned_addr learned my addr 172.21.15.16:0/1676662570 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:27:11.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.554+0000 7feb36ffd700 1 -- 172.21.15.16:0/1676662570 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7feb38094c20 msgr2=0x7feb38127d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:11.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.554+0000 7feb36ffd700 1 --2- 172.21.15.16:0/1676662570 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7feb38094c20 0x7feb38127d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:11.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.554+0000 7feb36ffd700 1 -- 172.21.15.16:0/1676662570 >> v1:172.21.15.16:6789/0 conn(0x7feb38091b00 legacy=0x7feb38127600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:11.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.554+0000 7feb36ffd700 1 -- 172.21.15.16:0/1676662570 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb38093900 con 0x7feb38096380 2023-09-30T22:27:11.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.554+0000 7feb35ffb700 1 -- 172.21.15.16:0/1676662570 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4247017247 0 0) 0x7feb38096760 con 0x7feb38091b00 2023-09-30T22:27:11.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.554+0000 7feb377fe700 1 --2- 172.21.15.16:0/1676662570 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7feb38094c20 0x7feb38127d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:11.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.555+0000 7feb36ffd700 1 --2- 172.21.15.16:0/1676662570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb38096380 0x7feb38094730 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7feb28009760 tx=0x7feb28009060).ready entity=mon.0 client_cookie=7aee70f78cbd1462 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:11.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.555+0000 7feb35ffb700 1 -- 172.21.15.16:0/1676662570 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2800f510 con 0x7feb38096380 2023-09-30T22:27:11.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.555+0000 7feb35ffb700 1 -- 172.21.15.16:0/1676662570 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb2800f670 con 0x7feb38096380 2023-09-30T22:27:11.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.555+0000 7feb35ffb700 1 -- 172.21.15.16:0/1676662570 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2800f9f0 con 0x7feb38096380 2023-09-30T22:27:11.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.555+0000 7feb3d334700 1 -- 172.21.15.16:0/1676662570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb38096380 msgr2=0x7feb38094730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:11.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.555+0000 7feb3d334700 1 --2- 172.21.15.16:0/1676662570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb38096380 0x7feb38094730 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7feb28009760 tx=0x7feb28009060).stop 2023-09-30T22:27:11.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.555+0000 7feb3d334700 1 -- 172.21.15.16:0/1676662570 shutdown_connections 2023-09-30T22:27:11.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.555+0000 7feb3d334700 1 --2- 172.21.15.16:0/1676662570 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7feb38094c20 0x7feb38127d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:11.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.555+0000 7feb3d334700 1 --2- 172.21.15.16:0/1676662570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb38096380 0x7feb38094730 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:11.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.555+0000 7feb3d334700 1 -- 172.21.15.16:0/1676662570 >> 172.21.15.16:0/1676662570 conn(0x7feb38097470 msgr2=0x7feb38097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:11.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.555+0000 7feb3d334700 1 -- 172.21.15.16:0/1676662570 shutdown_connections 2023-09-30T22:27:11.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.555+0000 7feb3d334700 1 -- 172.21.15.16:0/1676662570 wait complete. 2023-09-30T22:27:11.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.556+0000 7feb3d334700 1 Processor -- start 2023-09-30T22:27:11.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.557+0000 7feb3d334700 1 -- start start 2023-09-30T22:27:11.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.557+0000 7feb3d334700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb38091b00 0x7feb380590a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:11.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.557+0000 7feb3d334700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7feb38094c20 0x7feb38115fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:11.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.557+0000 7feb3d334700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7feb38096380 0x7feb381164c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:11.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.557+0000 7feb3d334700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb38118190 con 0x7feb38091b00 2023-09-30T22:27:11.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.557+0000 7feb3d334700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb3809a240 con 0x7feb38096380 2023-09-30T22:27:11.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.557+0000 7feb3d334700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feb3806a520 con 0x7feb38094c20 2023-09-30T22:27:11.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.557+0000 7feb36ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb38091b00 0x7feb380590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:11.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.557+0000 7feb36ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb38091b00 0x7feb380590a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40960/0 (socket says 172.21.15.16:40960) 2023-09-30T22:27:11.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.557+0000 7feb367fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7feb38094c20 0x7feb38115fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:11.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.557+0000 7feb36ffd700 1 -- 172.21.15.16:0/2374137479 learned_addr learned my addr 172.21.15.16:0/2374137479 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:11.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.557+0000 7feb377fe700 1 --2- 172.21.15.16:0/2374137479 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7feb38096380 0x7feb381164c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:11.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.557+0000 7feb36ffd700 1 -- 172.21.15.16:0/2374137479 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7feb38094c20 msgr2=0x7feb38115fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:11.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.557+0000 7feb36ffd700 1 --2- 172.21.15.16:0/2374137479 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7feb38094c20 0x7feb38115fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:11.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.557+0000 7feb36ffd700 1 -- 172.21.15.16:0/2374137479 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7feb38096380 msgr2=0x7feb381164c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:11.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.557+0000 7feb36ffd700 1 --2- 172.21.15.16:0/2374137479 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7feb38096380 0x7feb381164c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:11.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.557+0000 7feb36ffd700 1 -- 172.21.15.16:0/2374137479 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb30009a20 con 0x7feb38091b00 2023-09-30T22:27:11.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.558+0000 7feb36ffd700 1 --2- 172.21.15.16:0/2374137479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb38091b00 0x7feb380590a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7feb28009e00 tx=0x7feb28017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:11.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.558+0000 7feb2effd700 1 -- 172.21.15.16:0/2374137479 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2800f7e0 con 0x7feb38091b00 2023-09-30T22:27:11.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.558+0000 7feb2effd700 1 -- 172.21.15.16:0/2374137479 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb2800f940 con 0x7feb38091b00 2023-09-30T22:27:11.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.558+0000 7feb3d334700 1 -- 172.21.15.16:0/2374137479 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb280092a0 con 0x7feb38091b00 2023-09-30T22:27:11.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.558+0000 7feb2effd700 1 -- 172.21.15.16:0/2374137479 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2801b660 con 0x7feb38091b00 2023-09-30T22:27:11.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.558+0000 7feb3d334700 1 -- 172.21.15.16:0/2374137479 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb38059c20 con 0x7feb38091b00 2023-09-30T22:27:11.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.559+0000 7feb2effd700 1 -- 172.21.15.16:0/2374137479 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7feb2801b9d0 con 0x7feb38091b00 2023-09-30T22:27:11.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.559+0000 7feb2effd700 1 -- 172.21.15.16:0/2374137479 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7feb2803c8c0 con 0x7feb38091b00 2023-09-30T22:27:11.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.559+0000 7feb3d334700 1 -- 172.21.15.16:0/2374137479 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb38059e10 con 0x7feb38091b00 2023-09-30T22:27:11.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.564+0000 7feb2effd700 1 -- 172.21.15.16:0/2374137479 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7feb28029080 con 0x7feb38091b00 2023-09-30T22:27:11.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.782+0000 7feb3d334700 1 -- 172.21.15.16:0/2374137479 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7feb38118650 con 0x7feb38091b00 2023-09-30T22:27:11.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.782+0000 7feb2effd700 1 -- 172.21.15.16:0/2374137479 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7feb38118650 con 0x7feb38091b00 2023-09-30T22:27:11.783 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-09-30T22:27:11.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.784+0000 7feb3d334700 1 -- 172.21.15.16:0/2374137479 >> v1:172.21.15.16:6816/118285 conn(0x7feb1802bb90 legacy=0x7feb1802e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:11.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.784+0000 7feb3d334700 1 -- 172.21.15.16:0/2374137479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb38091b00 msgr2=0x7feb380590a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:11.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.784+0000 7feb3d334700 1 --2- 172.21.15.16:0/2374137479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb38091b00 0x7feb380590a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7feb28009e00 tx=0x7feb28017000).stop 2023-09-30T22:27:11.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.784+0000 7feb3d334700 1 -- 172.21.15.16:0/2374137479 shutdown_connections 2023-09-30T22:27:11.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.784+0000 7feb3d334700 1 --2- 172.21.15.16:0/2374137479 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7feb38096380 0x7feb381164c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:11.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.784+0000 7feb3d334700 1 --2- 172.21.15.16:0/2374137479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feb38091b00 0x7feb380590a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:11.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.784+0000 7feb3d334700 1 --2- 172.21.15.16:0/2374137479 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7feb38094c20 0x7feb38115fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:11.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.784+0000 7feb3d334700 1 -- 172.21.15.16:0/2374137479 >> 172.21.15.16:0/2374137479 conn(0x7feb38097470 msgr2=0x7feb38098780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:11.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.784+0000 7feb3d334700 1 -- 172.21.15.16:0/2374137479 shutdown_connections 2023-09-30T22:27:11.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:11.784+0000 7feb3d334700 1 -- 172.21.15.16:0/2374137479 wait complete. 2023-09-30T22:27:11.798 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-09-30T22:27:11.798 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-09-30T22:27:11.798 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi027.front.sepia.ceph.com 2023-09-30T22:27:11.798 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-09-30T22:27:11.798 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-09-30T22:27:11.798 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-09-30T22:27:11.798 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-09-30T22:27:11.798 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-09-30T22:27:11.798 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-09-30T22:27:11.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:27:11.799 DEBUG:teuthology.orchestra.run.smithi027:> ip addr 2023-09-30T22:27:11.822 INFO:teuthology.orchestra.run.smithi027.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-09-30T22:27:11.822 INFO:teuthology.orchestra.run.smithi027.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-09-30T22:27:11.822 INFO:teuthology.orchestra.run.smithi027.stdout: inet 127.0.0.1/8 scope host lo 2023-09-30T22:27:11.822 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-09-30T22:27:11.822 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 ::1/128 scope host 2023-09-30T22:27:11.822 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-09-30T22:27:11.822 INFO:teuthology.orchestra.run.smithi027.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-30T22:27:11.822 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c8 brd ff:ff:ff:ff:ff:ff 2023-09-30T22:27:11.823 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f0 2023-09-30T22:27:11.823 INFO:teuthology.orchestra.run.smithi027.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-30T22:27:11.823 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c9 brd ff:ff:ff:ff:ff:ff 2023-09-30T22:27:11.823 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f1 2023-09-30T22:27:11.823 INFO:teuthology.orchestra.run.smithi027.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-09-30T22:27:11.823 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5a brd ff:ff:ff:ff:ff:ff 2023-09-30T22:27:11.823 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f0 2023-09-30T22:27:11.823 INFO:teuthology.orchestra.run.smithi027.stdout: inet 172.21.15.27/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-09-30T22:27:11.823 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft 42484sec preferred_lft 42484sec 2023-09-30T22:27:11.823 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::ae1f:6bff:fef8:395a/64 scope link 2023-09-30T22:27:11.823 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-09-30T22:27:11.824 INFO:teuthology.orchestra.run.smithi027.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-09-30T22:27:11.824 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5b brd ff:ff:ff:ff:ff:ff 2023-09-30T22:27:11.824 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f1 2023-09-30T22:27:11.824 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-09-30T22:27:11.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:27:11.824 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec bash -c ' 2023-09-30T22:27:11.824 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2023-09-30T22:27:11.825 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link add name ceph-brx type bridge 2023-09-30T22:27:11.825 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip addr flush dev ceph-brx 2023-09-30T22:27:11.825 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set ceph-brx up 2023-09-30T22:27:11.825 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-09-30T22:27:11.825 DEBUG:teuthology.orchestra.run.smithi027:> ') 2023-09-30T22:27:11.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:27:11.996 DEBUG:teuthology.orchestra.run.smithi027:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-09-30T22:27:12.030 INFO:teuthology.orchestra.run.smithi027.stdout:1 2023-09-30T22:27:12.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:27:12.031 DEBUG:teuthology.orchestra.run.smithi027:> route 2023-09-30T22:27:12.106 INFO:teuthology.orchestra.run.smithi027.stdout:Kernel IP routing table 2023-09-30T22:27:12.106 INFO:teuthology.orchestra.run.smithi027.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-09-30T22:27:12.106 INFO:teuthology.orchestra.run.smithi027.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-09-30T22:27:12.106 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-09-30T22:27:12.107 INFO:teuthology.orchestra.run.smithi027.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-09-30T22:27:12.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:27:12.107 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec bash -c ' 2023-09-30T22:27:12.108 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2023-09-30T22:27:12.108 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-09-30T22:27:12.109 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-09-30T22:27:12.109 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-09-30T22:27:12.109 DEBUG:teuthology.orchestra.run.smithi027:> ') 2023-09-30T22:27:12.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:27:12.375 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2023-09-30T22:27:12.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:27:12.394 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list-id 2023-09-30T22:27:12.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:27:12.452 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec bash -c ' 2023-09-30T22:27:12.452 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2023-09-30T22:27:12.453 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-30T22:27:12.453 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-09-30T22:27:12.453 DEBUG:teuthology.orchestra.run.smithi027:> ') 2023-09-30T22:27:12.548 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-09-30T22:27:12.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:27:12.549 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec bash -c ' 2023-09-30T22:27:12.549 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2023-09-30T22:27:12.549 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-09-30T22:27:12.549 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-09-30T22:27:12.549 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-09-30T22:27:12.549 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-09-30T22:27:12.550 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-09-30T22:27:12.550 DEBUG:teuthology.orchestra.run.smithi027:> ') 2023-09-30T22:27:12.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:27:12.767 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec bash -c ' 2023-09-30T22:27:12.767 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2023-09-30T22:27:12.767 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set brx.0 up 2023-09-30T22:27:12.767 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set dev brx.0 master ceph-brx 2023-09-30T22:27:12.767 DEBUG:teuthology.orchestra.run.smithi027:> ') 2023-09-30T22:27:12.866 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-09-30T22:27:12.867 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-30T22:27:12.867 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-09-30T22:27:12.885 INFO:teuthology.orchestra.run.smithi027.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-09-30T22:27:12.885 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-09-30T22:27:12.885 DEBUG:teuthology.orchestra.run.smithi027:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-09-30T22:27:12.943 DEBUG:teuthology.orchestra.run.smithi027:> sudo modprobe fuse 2023-09-30T22:27:13.012 DEBUG:teuthology.orchestra.run.smithi027:> cat /proc/self/mounts | awk '{print $2}' 2023-09-30T22:27:13.071 INFO:teuthology.orchestra.run.smithi027.stdout:/sys 2023-09-30T22:27:13.071 INFO:teuthology.orchestra.run.smithi027.stdout:/proc 2023-09-30T22:27:13.071 INFO:teuthology.orchestra.run.smithi027.stdout:/dev 2023-09-30T22:27:13.071 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/security 2023-09-30T22:27:13.071 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/shm 2023-09-30T22:27:13.071 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/pts 2023-09-30T22:27:13.071 INFO:teuthology.orchestra.run.smithi027.stdout:/run 2023-09-30T22:27:13.071 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/cgroup 2023-09-30T22:27:13.071 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/cgroup/systemd 2023-09-30T22:27:13.072 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/pstore 2023-09-30T22:27:13.072 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/bpf 2023-09-30T22:27:13.072 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/cgroup/hugetlb 2023-09-30T22:27:13.072 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/cgroup/blkio 2023-09-30T22:27:13.072 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/cgroup/perf_event 2023-09-30T22:27:13.072 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/cgroup/net_cls,net_prio 2023-09-30T22:27:13.072 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/cgroup/cpu,cpuacct 2023-09-30T22:27:13.072 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/cgroup/cpuset 2023-09-30T22:27:13.072 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/cgroup/pids 2023-09-30T22:27:13.072 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/cgroup/devices 2023-09-30T22:27:13.072 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/cgroup/freezer 2023-09-30T22:27:13.072 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/cgroup/rdma 2023-09-30T22:27:13.073 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/cgroup/memory 2023-09-30T22:27:13.073 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/tracing 2023-09-30T22:27:13.073 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/config 2023-09-30T22:27:13.073 INFO:teuthology.orchestra.run.smithi027.stdout:/ 2023-09-30T22:27:13.073 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/selinux 2023-09-30T22:27:13.073 INFO:teuthology.orchestra.run.smithi027.stdout:/proc/sys/fs/binfmt_misc 2023-09-30T22:27:13.073 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/debug 2023-09-30T22:27:13.073 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/mqueue 2023-09-30T22:27:13.073 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/hugepages 2023-09-30T22:27:13.073 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/fuse/connections 2023-09-30T22:27:13.073 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/nfs/rpc_pipefs 2023-09-30T22:27:13.073 INFO:teuthology.orchestra.run.smithi027.stdout:/run/user/1000 2023-09-30T22:27:13.074 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/ceph 2023-09-30T22:27:13.074 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns 2023-09-30T22:27:13.074 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-30T22:27:13.074 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-30T22:27:13.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:27:13.075 DEBUG:teuthology.orchestra.run.smithi027:> ls /sys/fs/fuse/connections 2023-09-30T22:27:13.134 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-09-30T22:27:13.134 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-09-30T22:27:13.177 DEBUG:teuthology.orchestra.run.smithi027:> sudo modprobe fuse 2023-09-30T22:27:13.205 DEBUG:teuthology.orchestra.run.smithi027:> cat /proc/self/mounts | awk '{print $2}' 2023-09-30T22:27:13.267 INFO:teuthology.orchestra.run.smithi027.stdout:/sys 2023-09-30T22:27:13.267 INFO:teuthology.orchestra.run.smithi027.stdout:/proc 2023-09-30T22:27:13.267 INFO:teuthology.orchestra.run.smithi027.stdout:/dev 2023-09-30T22:27:13.267 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/security 2023-09-30T22:27:13.267 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/shm 2023-09-30T22:27:13.267 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/pts 2023-09-30T22:27:13.267 INFO:teuthology.orchestra.run.smithi027.stdout:/run 2023-09-30T22:27:13.267 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/cgroup 2023-09-30T22:27:13.267 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/cgroup/systemd 2023-09-30T22:27:13.268 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/pstore 2023-09-30T22:27:13.268 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/bpf 2023-09-30T22:27:13.268 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/cgroup/hugetlb 2023-09-30T22:27:13.268 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/cgroup/blkio 2023-09-30T22:27:13.268 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/cgroup/perf_event 2023-09-30T22:27:13.268 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/cgroup/net_cls,net_prio 2023-09-30T22:27:13.268 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/cgroup/cpu,cpuacct 2023-09-30T22:27:13.268 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/cgroup/cpuset 2023-09-30T22:27:13.268 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/cgroup/pids 2023-09-30T22:27:13.268 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/cgroup/devices 2023-09-30T22:27:13.269 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/cgroup/freezer 2023-09-30T22:27:13.269 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/cgroup/rdma 2023-09-30T22:27:13.269 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/cgroup/memory 2023-09-30T22:27:13.269 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/tracing 2023-09-30T22:27:13.269 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/config 2023-09-30T22:27:13.269 INFO:teuthology.orchestra.run.smithi027.stdout:/ 2023-09-30T22:27:13.269 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/selinux 2023-09-30T22:27:13.269 INFO:teuthology.orchestra.run.smithi027.stdout:/proc/sys/fs/binfmt_misc 2023-09-30T22:27:13.269 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/kernel/debug 2023-09-30T22:27:13.269 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/mqueue 2023-09-30T22:27:13.270 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/hugepages 2023-09-30T22:27:13.270 INFO:teuthology.orchestra.run.smithi027.stdout:/sys/fs/fuse/connections 2023-09-30T22:27:13.270 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/nfs/rpc_pipefs 2023-09-30T22:27:13.270 INFO:teuthology.orchestra.run.smithi027.stdout:/run/user/1000 2023-09-30T22:27:13.270 INFO:teuthology.orchestra.run.smithi027.stdout:/var/lib/ceph 2023-09-30T22:27:13.270 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns 2023-09-30T22:27:13.270 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-30T22:27:13.270 INFO:teuthology.orchestra.run.smithi027.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-30T22:27:13.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:27:13.271 DEBUG:teuthology.orchestra.run.smithi027:> ls /sys/fs/fuse/connections 2023-09-30T22:27:13.273 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:2023-09-30T22:27:13.274+0000 7f9375d363c0 -1 init, newargv = 0x5638cd7c5230 newargc=15 2023-09-30T22:27:13.274 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:ceph-fuse[112319]: starting ceph client 2023-09-30T22:27:13.279 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:ceph-fuse[112319]: starting fuse 2023-09-30T22:27:13.292 INFO:teuthology.orchestra.run.smithi027.stdout:46 2023-09-30T22:27:13.293 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [46] 2023-09-30T22:27:13.293 DEBUG:teuthology.orchestra.run.smithi027:> sudo stdin-killer -- python3 -c ' 2023-09-30T22:27:13.293 DEBUG:teuthology.orchestra.run.smithi027:> import glob 2023-09-30T22:27:13.293 DEBUG:teuthology.orchestra.run.smithi027:> import re 2023-09-30T22:27:13.293 DEBUG:teuthology.orchestra.run.smithi027:> import os 2023-09-30T22:27:13.293 DEBUG:teuthology.orchestra.run.smithi027:> import subprocess 2023-09-30T22:27:13.293 DEBUG:teuthology.orchestra.run.smithi027:> 2023-09-30T22:27:13.294 DEBUG:teuthology.orchestra.run.smithi027:> def _find_admin_socket(client_name): 2023-09-30T22:27:13.294 DEBUG:teuthology.orchestra.run.smithi027:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-09-30T22:27:13.294 DEBUG:teuthology.orchestra.run.smithi027:> files = glob.glob(asok_path) 2023-09-30T22:27:13.294 DEBUG:teuthology.orchestra.run.smithi027:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-09-30T22:27:13.294 DEBUG:teuthology.orchestra.run.smithi027:> 2023-09-30T22:27:13.294 DEBUG:teuthology.orchestra.run.smithi027:> # Given a non-glob path, it better be there 2023-09-30T22:27:13.294 DEBUG:teuthology.orchestra.run.smithi027:> if "*" not in asok_path: 2023-09-30T22:27:13.294 DEBUG:teuthology.orchestra.run.smithi027:> assert(len(files) == 1) 2023-09-30T22:27:13.294 DEBUG:teuthology.orchestra.run.smithi027:> return files[0] 2023-09-30T22:27:13.294 DEBUG:teuthology.orchestra.run.smithi027:> 2023-09-30T22:27:13.294 DEBUG:teuthology.orchestra.run.smithi027:> for f in files: 2023-09-30T22:27:13.295 DEBUG:teuthology.orchestra.run.smithi027:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-09-30T22:27:13.295 DEBUG:teuthology.orchestra.run.smithi027:> if os.path.exists("/proc/{0}".format(pid)): 2023-09-30T22:27:13.295 DEBUG:teuthology.orchestra.run.smithi027:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-09-30T22:27:13.295 DEBUG:teuthology.orchestra.run.smithi027:> contents = proc_f.read() 2023-09-30T22:27:13.295 DEBUG:teuthology.orchestra.run.smithi027:> if mountpoint in contents: 2023-09-30T22:27:13.295 DEBUG:teuthology.orchestra.run.smithi027:> return f 2023-09-30T22:27:13.295 DEBUG:teuthology.orchestra.run.smithi027:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-09-30T22:27:13.295 DEBUG:teuthology.orchestra.run.smithi027:> 2023-09-30T22:27:13.295 DEBUG:teuthology.orchestra.run.smithi027:> print(_find_admin_socket("client.0")) 2023-09-30T22:27:13.295 DEBUG:teuthology.orchestra.run.smithi027:> ' 2023-09-30T22:27:13.418 INFO:teuthology.orchestra.run.smithi027.stdout:/var/run/ceph/ceph-client.0.112319.asok 2023-09-30T22:27:13.421 INFO:teuthology.orchestra.run.smithi027.stderr:2023-09-30T22:27:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-09-30T22:27:13.428 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.112319.asok 2023-09-30T22:27:13.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:27:13.428 DEBUG:teuthology.orchestra.run.smithi027:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.112319.asok status 2023-09-30T22:27:13.584 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-09-30T22:27:13.584 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": { 2023-09-30T22:27:13.584 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_sha1": "eeb13ac974775bd0cce96a2c687e8bcc2faca43d", 2023-09-30T22:27:13.584 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_version": "ceph version 16.2.14-44-geeb13ac9 (eeb13ac974775bd0cce96a2c687e8bcc2faca43d) pacific (stable)", 2023-09-30T22:27:13.584 INFO:teuthology.orchestra.run.smithi027.stdout: "entity_id": "0", 2023-09-30T22:27:13.585 INFO:teuthology.orchestra.run.smithi027.stdout: "hostname": "smithi027", 2023-09-30T22:27:13.585 INFO:teuthology.orchestra.run.smithi027.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-09-30T22:27:13.585 INFO:teuthology.orchestra.run.smithi027.stdout: "pid": "112319", 2023-09-30T22:27:13.585 INFO:teuthology.orchestra.run.smithi027.stdout: "root": "/" 2023-09-30T22:27:13.585 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-09-30T22:27:13.585 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_count": 0, 2023-09-30T22:27:13.585 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_pinned_count": 0, 2023-09-30T22:27:13.585 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 4665, 2023-09-30T22:27:13.585 INFO:teuthology.orchestra.run.smithi027.stdout: "inst": { 2023-09-30T22:27:13.585 INFO:teuthology.orchestra.run.smithi027.stdout: "name": { 2023-09-30T22:27:13.585 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "client", 2023-09-30T22:27:13.586 INFO:teuthology.orchestra.run.smithi027.stdout: "num": 4665 2023-09-30T22:27:13.586 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-09-30T22:27:13.586 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2023-09-30T22:27:13.586 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-09-30T22:27:13.586 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:0", 2023-09-30T22:27:13.586 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 4177555180 2023-09-30T22:27:13.586 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-09-30T22:27:13.586 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-09-30T22:27:13.586 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2023-09-30T22:27:13.586 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-09-30T22:27:13.586 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:0", 2023-09-30T22:27:13.587 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 4177555180 2023-09-30T22:27:13.587 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-09-30T22:27:13.587 INFO:teuthology.orchestra.run.smithi027.stdout: "inst_str": "client.4665 172.21.15.27:0/4177555180", 2023-09-30T22:27:13.587 INFO:teuthology.orchestra.run.smithi027.stdout: "addr_str": "172.21.15.27:0/4177555180", 2023-09-30T22:27:13.587 INFO:teuthology.orchestra.run.smithi027.stdout: "inode_count": 1, 2023-09-30T22:27:13.587 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_epoch": 6, 2023-09-30T22:27:13.587 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch": 17, 2023-09-30T22:27:13.587 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch_barrier": 0, 2023-09-30T22:27:13.587 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklisted": false, 2023-09-30T22:27:13.587 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs" 2023-09-30T22:27:13.587 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-09-30T22:27:13.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:27:13.594 DEBUG:teuthology.orchestra.run.smithi027:> stat --file-system '--printf=%T 2023-09-30T22:27:13.594 DEBUG:teuthology.orchestra.run.smithi027:> ' -- /home/ubuntu/cephtest/mnt.0 2023-09-30T22:27:13.613 INFO:teuthology.orchestra.run.smithi027.stdout:fuseblk 2023-09-30T22:27:13.613 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-09-30T22:27:13.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:27:13.613 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-09-30T22:27:13.682 INFO:teuthology.run_tasks:Running task mds_pre_upgrade... 2023-09-30T22:27:13.693 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:27:13.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.831+0000 7ff6e7f42700 1 Processor -- start 2023-09-30T22:27:13.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.831+0000 7ff6e7f42700 1 -- start start 2023-09-30T22:27:13.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.831+0000 7ff6e7f42700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff6e0127390 0x7ff6e012fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:13.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.831+0000 7ff6e7f42700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff6e0127fa0 0x7ff6e0132170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:13.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.831+0000 7ff6e7f42700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6e0091b60 con 0x7ff6e0059070 2023-09-30T22:27:13.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.831+0000 7ff6e7f42700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e0096940 con 0x7ff6e0127390 2023-09-30T22:27:13.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.831+0000 7ff6e7f42700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e0059c00 con 0x7ff6e0127fa0 2023-09-30T22:27:13.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.831+0000 7ff6e54dd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff6e0127390 0x7ff6e012fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:13.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.831+0000 7ff6e64df700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff6e0127fa0 0x7ff6e0132170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:13.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.832+0000 7ff6e54dd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff6e0127390 0x7ff6e012fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:49238/0 (socket says 172.21.15.16:49238) 2023-09-30T22:27:13.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.832+0000 7ff6e64df700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff6e0127fa0 0x7ff6e0132170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37738/0 (socket says 172.21.15.16:37738) 2023-09-30T22:27:13.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.832+0000 7ff6e54dd700 1 -- 172.21.15.16:0/5617565 learned_addr learned my addr 172.21.15.16:0/5617565 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:13.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.832+0000 7ff6e5cde700 1 --1- 172.21.15.16:0/5617565 >> v1:172.21.15.16:6789/0 conn(0x7ff6e0059070 0x7ff6e0127890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53258/0 (socket says 172.21.15.16:53258) 2023-09-30T22:27:13.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.832+0000 7ff6e54dd700 1 -- 172.21.15.16:0/5617565 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff6e0127fa0 msgr2=0x7ff6e0132170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:13.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.832+0000 7ff6e54dd700 1 --2- 172.21.15.16:0/5617565 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff6e0127fa0 0x7ff6e0132170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:13.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.832+0000 7ff6e54dd700 1 -- 172.21.15.16:0/5617565 >> v1:172.21.15.16:6789/0 conn(0x7ff6e0059070 legacy=0x7ff6e0127890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:13.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.832+0000 7ff6e4cdc700 1 -- 172.21.15.16:0/5617565 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1828464754 0 0) 0x7ff6e0091b60 con 0x7ff6e0059070 2023-09-30T22:27:13.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.832+0000 7ff6e54dd700 1 -- 172.21.15.16:0/5617565 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e00929d0 con 0x7ff6e0127390 2023-09-30T22:27:13.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.832+0000 7ff6e64df700 1 --2- 172.21.15.16:0/5617565 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff6e0127fa0 0x7ff6e0132170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:13.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.832+0000 7ff6e54dd700 1 --2- 172.21.15.16:0/5617565 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff6e0127390 0x7ff6e012fc70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7ff6d4009830 tx=0x7ff6d40090f0).ready entity=mon.1 client_cookie=1a461019f27e521e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:13.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.832+0000 7ff6e4cdc700 1 -- 172.21.15.16:0/5617565 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d400b580 con 0x7ff6e0127390 2023-09-30T22:27:13.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.832+0000 7ff6e4cdc700 1 -- 172.21.15.16:0/5617565 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6d400b6e0 con 0x7ff6e0127390 2023-09-30T22:27:13.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.832+0000 7ff6e4cdc700 1 -- 172.21.15.16:0/5617565 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d400ba80 con 0x7ff6e0127390 2023-09-30T22:27:13.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.833+0000 7ff6e7f42700 1 -- 172.21.15.16:0/5617565 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff6e0127390 msgr2=0x7ff6e012fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:13.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.833+0000 7ff6e7f42700 1 --2- 172.21.15.16:0/5617565 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff6e0127390 0x7ff6e012fc70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7ff6d4009830 tx=0x7ff6d40090f0).stop 2023-09-30T22:27:13.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.833+0000 7ff6e7f42700 1 -- 172.21.15.16:0/5617565 shutdown_connections 2023-09-30T22:27:13.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.833+0000 7ff6e7f42700 1 --2- 172.21.15.16:0/5617565 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff6e0127fa0 0x7ff6e0132170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:13.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.833+0000 7ff6e7f42700 1 --2- 172.21.15.16:0/5617565 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff6e0127390 0x7ff6e012fc70 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:13.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.833+0000 7ff6e7f42700 1 -- 172.21.15.16:0/5617565 >> 172.21.15.16:0/5617565 conn(0x7ff6e0097470 msgr2=0x7ff6e0097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:13.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.833+0000 7ff6e7f42700 1 -- 172.21.15.16:0/5617565 shutdown_connections 2023-09-30T22:27:13.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.833+0000 7ff6e7f42700 1 -- 172.21.15.16:0/5617565 wait complete. 2023-09-30T22:27:13.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.834+0000 7ff6e7f42700 1 Processor -- start 2023-09-30T22:27:13.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.834+0000 7ff6e7f42700 1 -- start start 2023-09-30T22:27:13.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.834+0000 7ff6e7f42700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff6e0059070 0x7ff6e01247d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:13.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.834+0000 7ff6e7f42700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6e0127390 0x7ff6e012ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:13.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.834+0000 7ff6e7f42700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff6e0127fa0 0x7ff6e012bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:13.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.834+0000 7ff6e7f42700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e012e290 con 0x7ff6e0127390 2023-09-30T22:27:13.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.834+0000 7ff6e7f42700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e0131de0 con 0x7ff6e0127fa0 2023-09-30T22:27:13.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.834+0000 7ff6e7f42700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e0096f20 con 0x7ff6e0059070 2023-09-30T22:27:13.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.834+0000 7ff6e5cde700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff6e0059070 0x7ff6e01247d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:13.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.834+0000 7ff6e64df700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff6e0127fa0 0x7ff6e012bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:13.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.834+0000 7ff6e54dd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6e0127390 0x7ff6e012ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:13.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.834+0000 7ff6e64df700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff6e0127fa0 0x7ff6e012bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:49242/0 (socket says 172.21.15.16:49242) 2023-09-30T22:27:13.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.834+0000 7ff6e5cde700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff6e0059070 0x7ff6e01247d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37752/0 (socket says 172.21.15.16:37752) 2023-09-30T22:27:13.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.834+0000 7ff6e64df700 1 -- 172.21.15.16:0/550123518 learned_addr learned my addr 172.21.15.16:0/550123518 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:13.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.835+0000 7ff6e5cde700 1 -- 172.21.15.16:0/550123518 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff6e0127fa0 msgr2=0x7ff6e012bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:13.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.835+0000 7ff6e5cde700 1 --2- 172.21.15.16:0/550123518 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff6e0127fa0 0x7ff6e012bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:13.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.835+0000 7ff6e5cde700 1 -- 172.21.15.16:0/550123518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6e0127390 msgr2=0x7ff6e012ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:13.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.835+0000 7ff6e5cde700 1 --2- 172.21.15.16:0/550123518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6e0127390 0x7ff6e012ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:13.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.835+0000 7ff6e5cde700 1 -- 172.21.15.16:0/550123518 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6d8009a20 con 0x7ff6e0059070 2023-09-30T22:27:13.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.835+0000 7ff6e64df700 1 --2- 172.21.15.16:0/550123518 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff6e0127fa0 0x7ff6e012bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:13.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.835+0000 7ff6e5cde700 1 --2- 172.21.15.16:0/550123518 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff6e0059070 0x7ff6e01247d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7ff6dc00c3f0 tx=0x7ff6dc007bb0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:13.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.835+0000 7ff6d1ffb700 1 -- 172.21.15.16:0/550123518 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc0070d0 con 0x7ff6e0059070 2023-09-30T22:27:13.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.835+0000 7ff6d1ffb700 1 -- 172.21.15.16:0/550123518 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6dc007230 con 0x7ff6e0059070 2023-09-30T22:27:13.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.835+0000 7ff6e7f42700 1 -- 172.21.15.16:0/550123518 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6d4009370 con 0x7ff6e0059070 2023-09-30T22:27:13.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.835+0000 7ff6d1ffb700 1 -- 172.21.15.16:0/550123518 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc0056e0 con 0x7ff6e0059070 2023-09-30T22:27:13.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.835+0000 7ff6e7f42700 1 -- 172.21.15.16:0/550123518 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6e012eb30 con 0x7ff6e0059070 2023-09-30T22:27:13.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.836+0000 7ff6d1ffb700 1 -- 172.21.15.16:0/550123518 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7ff6dc005a90 con 0x7ff6e0059070 2023-09-30T22:27:13.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.836+0000 7ff6d1ffb700 1 -- 172.21.15.16:0/550123518 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7ff6dc03a680 con 0x7ff6e0059070 2023-09-30T22:27:13.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.837+0000 7ff6e7f42700 1 -- 172.21.15.16:0/550123518 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6e005a260 con 0x7ff6e0059070 2023-09-30T22:27:13.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:13.841+0000 7ff6d1ffb700 1 -- 172.21.15.16:0/550123518 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7ff6dc018030 con 0x7ff6e0059070 2023-09-30T22:27:14.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.016+0000 7ff6e7f42700 1 -- 172.21.15.16:0/550123518 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff6e012c840 con 0x7ff6e0059070 2023-09-30T22:27:14.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.016+0000 7ff6d1ffb700 1 -- 172.21.15.16:0/550123518 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+11340 (secure 0 0 0) 0x7ff6dc013970 con 0x7ff6e0059070 2023-09-30T22:27:14.017 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:27:14.017 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":17,"fsid":"7ede3e14-2bd0-4726-9437-6d13c90cd342","created":"2023-09-30T22:26:35.483697+0000","modified":"2023-09-30T22:27:11.244326+0000","last_up_change":"2023-09-30T22:26:44.227199+0000","last_in_change":"2023-09-30T22:26:37.120906+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-09-30T22:26:44.285133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:26:45.132921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:26:49.353683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:26:50.655685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"19946985-0ee2-487f-9347-929670dbe5ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6804","nonce":118824}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6805","nonce":118824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6807","nonce":118824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6806","nonce":118824}]},"public_addr":"172.21.15.16:6804/118824","cluster_addr":"172.21.15.16:6805/118824","heartbeat_back_addr":"172.21.15.16:6807/118824","heartbeat_front_addr":"172.21.15.16:6806/118824","state":["exists","up"]},{"osd":1,"uuid":"8efd4dbc-6ae4-4b82-b530-3b3eac2be445","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6812","nonce":118832}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6813","nonce":118832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6815","nonce":118832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6814","nonce":118832}]},"public_addr":"172.21.15.16:6812/118832","cluster_addr":"172.21.15.16:6813/118832","heartbeat_back_addr":"172.21.15.16:6815/118832","heartbeat_front_addr":"172.21.15.16:6814/118832","state":["exists","up"]},{"osd":2,"uuid":"b47bb1ce-27af-4c0c-868a-b76ebd0d9499","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6808","nonce":118840}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6809","nonce":118840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6811","nonce":118840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6810","nonce":118840}]},"public_addr":"172.21.15.16:6808/118840","cluster_addr":"172.21.15.16:6809/118840","heartbeat_back_addr":"172.21.15.16:6811/118840","heartbeat_front_addr":"172.21.15.16:6810/118840","state":["exists","up"]},{"osd":3,"uuid":"03b53d64-d10d-448a-a081-39ecceab589b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6800","nonce":118823}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6801","nonce":118823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6803","nonce":118823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6802","nonce":118823}]},"public_addr":"172.21.15.16:6800/118823","cluster_addr":"172.21.15.16:6801/118823","heartbeat_back_addr":"172.21.15.16:6803/118823","heartbeat_front_addr":"172.21.15.16:6802/118823","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.305953+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.307488+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.341023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.192071+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:27:14.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.019+0000 7ff6e7f42700 1 -- 172.21.15.16:0/550123518 >> v1:172.21.15.16:6816/118285 conn(0x7ff6c402bb90 legacy=0x7ff6c402e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:14.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.019+0000 7ff6e7f42700 1 -- 172.21.15.16:0/550123518 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff6e0059070 msgr2=0x7ff6e01247d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:14.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.019+0000 7ff6e7f42700 1 --2- 172.21.15.16:0/550123518 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff6e0059070 0x7ff6e01247d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7ff6dc00c3f0 tx=0x7ff6dc007bb0).stop 2023-09-30T22:27:14.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.019+0000 7ff6e7f42700 1 -- 172.21.15.16:0/550123518 shutdown_connections 2023-09-30T22:27:14.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.019+0000 7ff6e7f42700 1 --2- 172.21.15.16:0/550123518 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff6e0127fa0 0x7ff6e012bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:14.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.019+0000 7ff6e7f42700 1 --2- 172.21.15.16:0/550123518 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff6e0059070 0x7ff6e01247d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:14.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.019+0000 7ff6e7f42700 1 --2- 172.21.15.16:0/550123518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6e0127390 0x7ff6e012ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:14.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.019+0000 7ff6e7f42700 1 -- 172.21.15.16:0/550123518 >> 172.21.15.16:0/550123518 conn(0x7ff6e0097470 msgr2=0x7ff6e0098780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:14.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.019+0000 7ff6e7f42700 1 -- 172.21.15.16:0/550123518 shutdown_connections 2023-09-30T22:27:14.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.019+0000 7ff6e7f42700 1 -- 172.21.15.16:0/550123518 wait complete. 2023-09-30T22:27:14.032 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-09-30T22:26:44.285133+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-09-30T22:26:45.132921+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-30T22:26:49.353683+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-09-30T22:26:50.655685+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-09-30T22:27:14.033 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-09-30T22:27:14.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.166+0000 7f43ade18700 1 Processor -- start 2023-09-30T22:27:14.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.166+0000 7f43ade18700 1 -- start start 2023-09-30T22:27:14.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.167+0000 7f43ade18700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f43a80926d0 0x7f43a81275c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:14.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.167+0000 7f43ade18700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f43a8127be0 0x7f43a8128000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:14.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.167+0000 7f43ade18700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f43a812a9a0 0x7f43a812ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:14.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.167+0000 7f43ade18700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43a8099dd0 con 0x7f43a812a9a0 2023-09-30T22:27:14.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.167+0000 7f43ade18700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43a8091c80 con 0x7f43a8127be0 2023-09-30T22:27:14.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.167+0000 7f43ade18700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f43a80999d0 con 0x7f43a80926d0 2023-09-30T22:27:14.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.167+0000 7f43a77fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f43a80926d0 0x7f43a81275c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:14.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.167+0000 7f43a6ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f43a8127be0 0x7f43a8128000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:14.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.167+0000 7f43a77fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f43a80926d0 0x7f43a81275c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37768/0 (socket says 172.21.15.16:37768) 2023-09-30T22:27:14.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.167+0000 7f43a6ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f43a8127be0 0x7f43a8128000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:49258/0 (socket says 172.21.15.16:49258) 2023-09-30T22:27:14.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.167+0000 7f43a77fe700 1 -- 172.21.15.16:0/716806792 learned_addr learned my addr 172.21.15.16:0/716806792 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:14.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.167+0000 7f43a7fff700 1 --2- 172.21.15.16:0/716806792 >> v2:172.21.15.16:3300/0 conn(0x7f43a812a9a0 0x7f43a812ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:14.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.167+0000 7f43a6ffd700 1 -- 172.21.15.16:0/716806792 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f43a80926d0 msgr2=0x7f43a81275c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:14.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.167+0000 7f43a6ffd700 1 --2- 172.21.15.16:0/716806792 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f43a80926d0 0x7f43a81275c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:14.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.167+0000 7f43a6ffd700 1 -- 172.21.15.16:0/716806792 >> v2:172.21.15.16:3300/0 conn(0x7f43a812a9a0 msgr2=0x7f43a812ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:14.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.167+0000 7f43a6ffd700 1 --2- 172.21.15.16:0/716806792 >> v2:172.21.15.16:3300/0 conn(0x7f43a812a9a0 0x7f43a812ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:14.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.167+0000 7f43a6ffd700 1 -- 172.21.15.16:0/716806792 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43a809a280 con 0x7f43a8127be0 2023-09-30T22:27:14.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.167+0000 7f43a77fe700 1 --2- 172.21.15.16:0/716806792 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f43a80926d0 0x7f43a81275c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:14.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.167+0000 7f43a7fff700 1 --2- 172.21.15.16:0/716806792 >> v2:172.21.15.16:3300/0 conn(0x7f43a812a9a0 0x7f43a812ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:14.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.167+0000 7f43a6ffd700 1 --2- 172.21.15.16:0/716806792 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f43a8127be0 0x7f43a8128000 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f4390009760 tx=0x7f4390009060).ready entity=mon.1 client_cookie=1a586d31dd708321 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:14.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.167+0000 7f43a67fc700 1 -- 172.21.15.16:0/716806792 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f439000d4a0 con 0x7f43a8127be0 2023-09-30T22:27:14.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.168+0000 7f43a67fc700 1 -- 172.21.15.16:0/716806792 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f439000d600 con 0x7f43a8127be0 2023-09-30T22:27:14.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.168+0000 7f43a67fc700 1 -- 172.21.15.16:0/716806792 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f439000d980 con 0x7f43a8127be0 2023-09-30T22:27:14.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.168+0000 7f43ade18700 1 -- 172.21.15.16:0/716806792 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f43a8127be0 msgr2=0x7f43a8128000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:14.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.168+0000 7f43ade18700 1 --2- 172.21.15.16:0/716806792 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f43a8127be0 0x7f43a8128000 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f4390009760 tx=0x7f4390009060).stop 2023-09-30T22:27:14.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.168+0000 7f43ade18700 1 -- 172.21.15.16:0/716806792 shutdown_connections 2023-09-30T22:27:14.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.168+0000 7f43ade18700 1 --2- 172.21.15.16:0/716806792 >> v2:172.21.15.16:3300/0 conn(0x7f43a812a9a0 0x7f43a812ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:14.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.168+0000 7f43ade18700 1 --2- 172.21.15.16:0/716806792 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f43a80926d0 0x7f43a81275c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:14.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.168+0000 7f43ade18700 1 --2- 172.21.15.16:0/716806792 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f43a8127be0 0x7f43a8128000 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:14.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.168+0000 7f43ade18700 1 -- 172.21.15.16:0/716806792 >> 172.21.15.16:0/716806792 conn(0x7f43a8097480 msgr2=0x7f43a8097890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:14.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.168+0000 7f43ade18700 1 -- 172.21.15.16:0/716806792 shutdown_connections 2023-09-30T22:27:14.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.168+0000 7f43ade18700 1 -- 172.21.15.16:0/716806792 wait complete. 2023-09-30T22:27:14.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.169+0000 7f43ade18700 1 Processor -- start 2023-09-30T22:27:14.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.169+0000 7f43ade18700 1 -- start start 2023-09-30T22:27:14.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.170+0000 7f43ade18700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f43a80926d0 0x7f43a81139e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:14.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.170+0000 7f43ade18700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f43a8127be0 0x7f43a8113ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:14.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.170+0000 7f43ade18700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43a812a9a0 0x7f43a81143c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:14.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.170+0000 7f43ade18700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43a8124b40 con 0x7f43a812a9a0 2023-09-30T22:27:14.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.170+0000 7f43ade18700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43a80938f0 con 0x7f43a8127be0 2023-09-30T22:27:14.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.170+0000 7f43ade18700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f43a809a4b0 con 0x7f43a80926d0 2023-09-30T22:27:14.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.170+0000 7f43a77fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f43a80926d0 0x7f43a81139e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:14.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.170+0000 7f43a77fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f43a80926d0 0x7f43a81139e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37770/0 (socket says 172.21.15.16:37770) 2023-09-30T22:27:14.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.170+0000 7f43a77fe700 1 -- 172.21.15.16:0/1474609030 learned_addr learned my addr 172.21.15.16:0/1474609030 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:14.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.170+0000 7f43a7fff700 1 --2- 172.21.15.16:0/1474609030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43a812a9a0 0x7f43a81143c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:14.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.170+0000 7f43a6ffd700 1 --2- 172.21.15.16:0/1474609030 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f43a8127be0 0x7f43a8113ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:14.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.170+0000 7f43a77fe700 1 -- 172.21.15.16:0/1474609030 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f43a8127be0 msgr2=0x7f43a8113ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:14.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.170+0000 7f43a77fe700 1 --2- 172.21.15.16:0/1474609030 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f43a8127be0 0x7f43a8113ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:14.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.170+0000 7f43a77fe700 1 -- 172.21.15.16:0/1474609030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43a812a9a0 msgr2=0x7f43a81143c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:14.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.170+0000 7f43a77fe700 1 --2- 172.21.15.16:0/1474609030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43a812a9a0 0x7f43a81143c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:14.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.170+0000 7f43a77fe700 1 -- 172.21.15.16:0/1474609030 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4398009370 con 0x7f43a80926d0 2023-09-30T22:27:14.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.170+0000 7f43a6ffd700 1 --2- 172.21.15.16:0/1474609030 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f43a8127be0 0x7f43a8113ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:14.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.170+0000 7f43a7fff700 1 --2- 172.21.15.16:0/1474609030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43a812a9a0 0x7f43a81143c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:14.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.170+0000 7f43a77fe700 1 --2- 172.21.15.16:0/1474609030 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f43a80926d0 0x7f43a81139e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f439800ea20 tx=0x7f43980090f0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:14.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.171+0000 7f439f7fe700 1 -- 172.21.15.16:0/1474609030 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4398009650 con 0x7f43a80926d0 2023-09-30T22:27:14.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.171+0000 7f439f7fe700 1 -- 172.21.15.16:0/1474609030 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f439800c850 con 0x7f43a80926d0 2023-09-30T22:27:14.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.171+0000 7f43ade18700 1 -- 172.21.15.16:0/1474609030 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43a0009a20 con 0x7f43a80926d0 2023-09-30T22:27:14.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.171+0000 7f439f7fe700 1 -- 172.21.15.16:0/1474609030 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f439800cbd0 con 0x7f43a80926d0 2023-09-30T22:27:14.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.171+0000 7f43ade18700 1 -- 172.21.15.16:0/1474609030 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43900092a0 con 0x7f43a80926d0 2023-09-30T22:27:14.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.171+0000 7f439f7fe700 1 -- 172.21.15.16:0/1474609030 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f439801b400 con 0x7f43a80926d0 2023-09-30T22:27:14.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.172+0000 7f439f7fe700 1 -- 172.21.15.16:0/1474609030 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f439803b520 con 0x7f43a80926d0 2023-09-30T22:27:14.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.172+0000 7f43ade18700 1 -- 172.21.15.16:0/1474609030 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43a8117470 con 0x7f43a80926d0 2023-09-30T22:27:14.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.176+0000 7f439f7fe700 1 -- 172.21.15.16:0/1474609030 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f4398016070 con 0x7f43a80926d0 2023-09-30T22:27:14.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.352+0000 7f43ade18700 1 -- 172.21.15.16:0/1474609030 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f43a805a2c0 con 0x7f43a80926d0 2023-09-30T22:27:14.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.352+0000 7f439f7fe700 1 -- 172.21.15.16:0/1474609030 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v17) v1 ==== 110+0+10 (secure 0 0 0) 0x7f4398016020 con 0x7f43a80926d0 2023-09-30T22:27:14.352 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-09-30T22:27:14.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.354+0000 7f43ade18700 1 -- 172.21.15.16:0/1474609030 >> v1:172.21.15.16:6816/118285 conn(0x7f438802bb60 legacy=0x7f438802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:14.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.354+0000 7f43ade18700 1 -- 172.21.15.16:0/1474609030 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f43a80926d0 msgr2=0x7f43a81139e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:14.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.354+0000 7f43ade18700 1 --2- 172.21.15.16:0/1474609030 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f43a80926d0 0x7f43a81139e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f439800ea20 tx=0x7f43980090f0).stop 2023-09-30T22:27:14.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.354+0000 7f43ade18700 1 -- 172.21.15.16:0/1474609030 shutdown_connections 2023-09-30T22:27:14.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.354+0000 7f43ade18700 1 --2- 172.21.15.16:0/1474609030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43a812a9a0 0x7f43a81143c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:14.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.354+0000 7f43ade18700 1 --2- 172.21.15.16:0/1474609030 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f43a80926d0 0x7f43a81139e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:14.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.354+0000 7f43ade18700 1 --2- 172.21.15.16:0/1474609030 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f43a8127be0 0x7f43a8113ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:14.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.354+0000 7f43ade18700 1 -- 172.21.15.16:0/1474609030 >> 172.21.15.16:0/1474609030 conn(0x7f43a8097480 msgr2=0x7f43a8095f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:14.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.354+0000 7f43ade18700 1 -- 172.21.15.16:0/1474609030 shutdown_connections 2023-09-30T22:27:14.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.354+0000 7f43ade18700 1 -- 172.21.15.16:0/1474609030 wait complete. 2023-09-30T22:27:14.367 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-09-30T22:27:14.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.501+0000 7f239bea1700 1 Processor -- start 2023-09-30T22:27:14.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.501+0000 7f239bea1700 1 -- start start 2023-09-30T22:27:14.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.501+0000 7f239bea1700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2394093840 0x7f2394098790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:14.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.501+0000 7f239bea1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2394098c80 0x7f2394099060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:14.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.501+0000 7f239bea1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f239412c150 0x7f239412e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:14.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.501+0000 7f239bea1700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2394093c20 con 0x7f2394093840 2023-09-30T22:27:14.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.501+0000 7f239bea1700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2394092b90 con 0x7f239412c150 2023-09-30T22:27:14.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.501+0000 7f239bea1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2394092d80 con 0x7f2394098c80 2023-09-30T22:27:14.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.502+0000 7f2399c3d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2394093840 0x7f2394098790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:14.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.502+0000 7f239943c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2394098c80 0x7f2394099060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:14.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.502+0000 7f239a43e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f239412c150 0x7f239412e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:14.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.502+0000 7f2399c3d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2394093840 0x7f2394098790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40998/0 (socket says 172.21.15.16:40998) 2023-09-30T22:27:14.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.502+0000 7f239943c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2394098c80 0x7f2394099060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37780/0 (socket says 172.21.15.16:37780) 2023-09-30T22:27:14.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.502+0000 7f2399c3d700 1 -- 172.21.15.16:0/177457523 learned_addr learned my addr 172.21.15.16:0/177457523 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:14.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.502+0000 7f2399c3d700 1 -- 172.21.15.16:0/177457523 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2394098c80 msgr2=0x7f2394099060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:14.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.502+0000 7f2399c3d700 1 --2- 172.21.15.16:0/177457523 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2394098c80 0x7f2394099060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:14.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.502+0000 7f2399c3d700 1 -- 172.21.15.16:0/177457523 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f239412c150 msgr2=0x7f239412e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:14.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.502+0000 7f2399c3d700 1 --2- 172.21.15.16:0/177457523 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f239412c150 0x7f239412e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:14.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.502+0000 7f2399c3d700 1 -- 172.21.15.16:0/177457523 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2394099b70 con 0x7f2394093840 2023-09-30T22:27:14.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.502+0000 7f239943c700 1 --2- 172.21.15.16:0/177457523 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2394098c80 0x7f2394099060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:14.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.502+0000 7f239a43e700 1 --2- 172.21.15.16:0/177457523 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f239412c150 0x7f239412e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:14.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.502+0000 7f2399c3d700 1 --2- 172.21.15.16:0/177457523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2394093840 0x7f2394098790 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f2390006fd0 tx=0x7f23900090f0).ready entity=mon.0 client_cookie=9caea64fa9996f63 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:14.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.502+0000 7f2398c3b700 1 -- 172.21.15.16:0/177457523 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f239000f510 con 0x7f2394093840 2023-09-30T22:27:14.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.502+0000 7f2398c3b700 1 -- 172.21.15.16:0/177457523 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f239000f670 con 0x7f2394093840 2023-09-30T22:27:14.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.502+0000 7f2398c3b700 1 -- 172.21.15.16:0/177457523 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f239000f9f0 con 0x7f2394093840 2023-09-30T22:27:14.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.503+0000 7f239bea1700 1 -- 172.21.15.16:0/177457523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2394093840 msgr2=0x7f2394098790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:14.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.503+0000 7f239bea1700 1 --2- 172.21.15.16:0/177457523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2394093840 0x7f2394098790 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f2390006fd0 tx=0x7f23900090f0).stop 2023-09-30T22:27:14.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.503+0000 7f239bea1700 1 -- 172.21.15.16:0/177457523 shutdown_connections 2023-09-30T22:27:14.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.503+0000 7f239bea1700 1 --2- 172.21.15.16:0/177457523 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f239412c150 0x7f239412e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:14.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.503+0000 7f239bea1700 1 --2- 172.21.15.16:0/177457523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2394093840 0x7f2394098790 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:14.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.503+0000 7f239bea1700 1 --2- 172.21.15.16:0/177457523 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2394098c80 0x7f2394099060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:14.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.503+0000 7f239bea1700 1 -- 172.21.15.16:0/177457523 >> 172.21.15.16:0/177457523 conn(0x7f2394097480 msgr2=0x7f2394097890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:14.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.503+0000 7f239bea1700 1 -- 172.21.15.16:0/177457523 shutdown_connections 2023-09-30T22:27:14.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.503+0000 7f239bea1700 1 -- 172.21.15.16:0/177457523 wait complete. 2023-09-30T22:27:14.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.504+0000 7f239bea1700 1 Processor -- start 2023-09-30T22:27:14.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.504+0000 7f239bea1700 1 -- start start 2023-09-30T22:27:14.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.504+0000 7f239bea1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2394093840 0x7f239411bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:14.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.504+0000 7f239bea1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2394098c80 0x7f2394117ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:14.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.504+0000 7f239bea1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f239412c150 0x7f23941181d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:14.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.504+0000 7f239bea1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f239412bcc0 con 0x7f239412c150 2023-09-30T22:27:14.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.504+0000 7f239bea1700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f239411a290 con 0x7f2394098c80 2023-09-30T22:27:14.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.504+0000 7f239bea1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f239411b420 con 0x7f2394093840 2023-09-30T22:27:14.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.505+0000 7f239943c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2394098c80 0x7f2394117ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:14.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.505+0000 7f239a43e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f239412c150 0x7f23941181d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:14.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.505+0000 7f2399c3d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2394093840 0x7f239411bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:14.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.505+0000 7f239943c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2394098c80 0x7f2394117ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:49290/0 (socket says 172.21.15.16:49290) 2023-09-30T22:27:14.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.505+0000 7f239a43e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f239412c150 0x7f23941181d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41014/0 (socket says 172.21.15.16:41014) 2023-09-30T22:27:14.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.505+0000 7f239943c700 1 -- 172.21.15.16:0/186277463 learned_addr learned my addr 172.21.15.16:0/186277463 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:14.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.505+0000 7f2399c3d700 1 -- 172.21.15.16:0/186277463 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2394098c80 msgr2=0x7f2394117ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:14.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.505+0000 7f2399c3d700 1 --2- 172.21.15.16:0/186277463 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2394098c80 0x7f2394117ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:14.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.505+0000 7f2399c3d700 1 -- 172.21.15.16:0/186277463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f239412c150 msgr2=0x7f23941181d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:14.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.505+0000 7f2399c3d700 1 --2- 172.21.15.16:0/186277463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f239412c150 0x7f23941181d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:14.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.505+0000 7f2399c3d700 1 -- 172.21.15.16:0/186277463 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23840092a0 con 0x7f2394093840 2023-09-30T22:27:14.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.505+0000 7f2399c3d700 1 --2- 172.21.15.16:0/186277463 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2394093840 0x7f239411bc00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f2390014000 tx=0x7f2390017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:14.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.505+0000 7f2389ffb700 1 -- 172.21.15.16:0/186277463 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f239000fe30 con 0x7f2394093840 2023-09-30T22:27:14.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.505+0000 7f2389ffb700 1 -- 172.21.15.16:0/186277463 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f239000f7e0 con 0x7f2394093840 2023-09-30T22:27:14.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.505+0000 7f239bea1700 1 -- 172.21.15.16:0/186277463 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f238c009a20 con 0x7f2394093840 2023-09-30T22:27:14.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.505+0000 7f2389ffb700 1 -- 172.21.15.16:0/186277463 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f239001c650 con 0x7f2394093840 2023-09-30T22:27:14.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.505+0000 7f239bea1700 1 -- 172.21.15.16:0/186277463 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2390009370 con 0x7f2394093840 2023-09-30T22:27:14.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.506+0000 7f2389ffb700 1 -- 172.21.15.16:0/186277463 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f239001ca40 con 0x7f2394093840 2023-09-30T22:27:14.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.506+0000 7f2389ffb700 1 -- 172.21.15.16:0/186277463 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f239003c540 con 0x7f2394093840 2023-09-30T22:27:14.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.507+0000 7f239bea1700 1 -- 172.21.15.16:0/186277463 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f239411b280 con 0x7f2394093840 2023-09-30T22:27:14.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.511+0000 7f2389ffb700 1 -- 172.21.15.16:0/186277463 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f239411b280 con 0x7f2394093840 2023-09-30T22:27:14.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.685+0000 7f239bea1700 1 -- 172.21.15.16:0/186277463 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f239411b280 con 0x7f2394093840 2023-09-30T22:27:14.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.685+0000 7f2389ffb700 1 -- 172.21.15.16:0/186277463 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v17) v1 ==== 92+0+10 (secure 0 0 0) 0x7f239411b280 con 0x7f2394093840 2023-09-30T22:27:14.686 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 8 2023-09-30T22:27:14.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.688+0000 7f239bea1700 1 -- 172.21.15.16:0/186277463 >> v1:172.21.15.16:6816/118285 conn(0x7f237802bb30 legacy=0x7f237802dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:14.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.688+0000 7f239bea1700 1 -- 172.21.15.16:0/186277463 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2394093840 msgr2=0x7f239411bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:14.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.688+0000 7f239bea1700 1 --2- 172.21.15.16:0/186277463 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2394093840 0x7f239411bc00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f2390014000 tx=0x7f2390017000).stop 2023-09-30T22:27:14.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.688+0000 7f239bea1700 1 -- 172.21.15.16:0/186277463 shutdown_connections 2023-09-30T22:27:14.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.688+0000 7f239bea1700 1 --2- 172.21.15.16:0/186277463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f239412c150 0x7f23941181d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:14.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.688+0000 7f239bea1700 1 --2- 172.21.15.16:0/186277463 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2394093840 0x7f239411bc00 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:14.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.688+0000 7f239bea1700 1 --2- 172.21.15.16:0/186277463 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2394098c80 0x7f2394117ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:14.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.688+0000 7f239bea1700 1 -- 172.21.15.16:0/186277463 >> 172.21.15.16:0/186277463 conn(0x7f2394097480 msgr2=0x7f239412d430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:14.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.688+0000 7f239bea1700 1 -- 172.21.15.16:0/186277463 shutdown_connections 2023-09-30T22:27:14.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.688+0000 7f239bea1700 1 -- 172.21.15.16:0/186277463 wait complete. 2023-09-30T22:27:14.701 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-30T22:27:14.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.834+0000 7f4e0df70700 1 Processor -- start 2023-09-30T22:27:14.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.834+0000 7f4e0df70700 1 -- start start 2023-09-30T22:27:14.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.835+0000 7f4e0df70700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4e080613a0 0x7f4e080919e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:14.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.835+0000 7f4e0df70700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4e0812cde0 0x7f4e0812f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:14.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.835+0000 7f4e0df70700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e08092c00 con 0x7f4e08091ed0 2023-09-30T22:27:14.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.835+0000 7f4e0df70700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e08061780 con 0x7f4e0812cde0 2023-09-30T22:27:14.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.835+0000 7f4e0df70700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e08092ff0 con 0x7f4e080613a0 2023-09-30T22:27:14.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.835+0000 7f4e07fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4e0812cde0 0x7f4e0812f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:14.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.835+0000 7f4e077fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4e080613a0 0x7f4e080919e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:14.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.835+0000 7f4e07fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4e0812cde0 0x7f4e0812f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41026/0 (socket says 172.21.15.16:41026) 2023-09-30T22:27:14.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.835+0000 7f4e06ffd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4e08091ed0 0x7f4e080922b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53262/0 (socket says 172.21.15.16:53262) 2023-09-30T22:27:14.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.835+0000 7f4e07fff700 1 -- 172.21.15.16:0/3110141135 learned_addr learned my addr 172.21.15.16:0/3110141135 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:14.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.835+0000 7f4e07fff700 1 -- 172.21.15.16:0/3110141135 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4e080613a0 msgr2=0x7f4e080919e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:14.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.835+0000 7f4e07fff700 1 --2- 172.21.15.16:0/3110141135 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4e080613a0 0x7f4e080919e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:14.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.835+0000 7f4e07fff700 1 -- 172.21.15.16:0/3110141135 >> v1:172.21.15.16:6789/0 conn(0x7f4e08091ed0 legacy=0x7f4e080922b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:14.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.835+0000 7f4e07fff700 1 -- 172.21.15.16:0/3110141135 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e08093b20 con 0x7f4e0812cde0 2023-09-30T22:27:14.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.835+0000 7f4e067fc700 1 -- 172.21.15.16:0/3110141135 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3723679518 0 0) 0x7f4e08092c00 con 0x7f4e08091ed0 2023-09-30T22:27:14.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.835+0000 7f4e077fe700 1 --2- 172.21.15.16:0/3110141135 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4e080613a0 0x7f4e080919e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:14.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.835+0000 7f4e07fff700 1 --2- 172.21.15.16:0/3110141135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e0812cde0 0x7f4e0812f1d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f4e00009f20 tx=0x7f4e00009840).ready entity=mon.0 client_cookie=5f1373ddefeb804b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:14.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.835+0000 7f4e067fc700 1 -- 172.21.15.16:0/3110141135 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0000e070 con 0x7f4e0812cde0 2023-09-30T22:27:14.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.836+0000 7f4e067fc700 1 -- 172.21.15.16:0/3110141135 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e0000b9d0 con 0x7f4e0812cde0 2023-09-30T22:27:14.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.836+0000 7f4e067fc700 1 -- 172.21.15.16:0/3110141135 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0000bd50 con 0x7f4e0812cde0 2023-09-30T22:27:14.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.836+0000 7f4e0df70700 1 -- 172.21.15.16:0/3110141135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e0812cde0 msgr2=0x7f4e0812f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:14.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.836+0000 7f4e0df70700 1 --2- 172.21.15.16:0/3110141135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e0812cde0 0x7f4e0812f1d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f4e00009f20 tx=0x7f4e00009840).stop 2023-09-30T22:27:14.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.836+0000 7f4e0df70700 1 -- 172.21.15.16:0/3110141135 shutdown_connections 2023-09-30T22:27:14.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.836+0000 7f4e0df70700 1 --2- 172.21.15.16:0/3110141135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e0812cde0 0x7f4e0812f1d0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:14.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.836+0000 7f4e0df70700 1 --2- 172.21.15.16:0/3110141135 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4e080613a0 0x7f4e080919e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:14.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.836+0000 7f4e0df70700 1 -- 172.21.15.16:0/3110141135 >> 172.21.15.16:0/3110141135 conn(0x7f4e0809a1c0 msgr2=0x7f4e0809a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:14.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.836+0000 7f4e0df70700 1 -- 172.21.15.16:0/3110141135 shutdown_connections 2023-09-30T22:27:14.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.836+0000 7f4e0df70700 1 -- 172.21.15.16:0/3110141135 wait complete. 2023-09-30T22:27:14.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.838+0000 7f4e0df70700 1 Processor -- start 2023-09-30T22:27:14.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.838+0000 7f4e0df70700 1 -- start start 2023-09-30T22:27:14.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.838+0000 7f4e0df70700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4e080613a0 0x7f4e08111c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:14.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.838+0000 7f4e0df70700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e08091ed0 0x7f4e08112160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:14.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.838+0000 7f4e0df70700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4e0812cde0 0x7f4e08112650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:14.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.838+0000 7f4e0df70700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e081141a0 con 0x7f4e08091ed0 2023-09-30T22:27:14.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.838+0000 7f4e0df70700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e08099230 con 0x7f4e0812cde0 2023-09-30T22:27:14.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.838+0000 7f4e0df70700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4e08093d70 con 0x7f4e080613a0 2023-09-30T22:27:14.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.839+0000 7f4e07fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4e0812cde0 0x7f4e08112650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:14.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.839+0000 7f4e07fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4e0812cde0 0x7f4e08112650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:49312/0 (socket says 172.21.15.16:49312) 2023-09-30T22:27:14.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.839+0000 7f4e07fff700 1 -- 172.21.15.16:0/1729211443 learned_addr learned my addr 172.21.15.16:0/1729211443 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:14.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.839+0000 7f4e06ffd700 1 --2- 172.21.15.16:0/1729211443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e08091ed0 0x7f4e08112160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:14.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.839+0000 7f4e077fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4e080613a0 0x7f4e08111c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:14.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.839+0000 7f4e07fff700 1 -- 172.21.15.16:0/1729211443 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4e080613a0 msgr2=0x7f4e08111c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:14.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.839+0000 7f4e07fff700 1 --2- 172.21.15.16:0/1729211443 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4e080613a0 0x7f4e08111c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:14.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.839+0000 7f4e07fff700 1 -- 172.21.15.16:0/1729211443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e08091ed0 msgr2=0x7f4e08112160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:14.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.839+0000 7f4e07fff700 1 --2- 172.21.15.16:0/1729211443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e08091ed0 0x7f4e08112160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:14.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.839+0000 7f4e07fff700 1 -- 172.21.15.16:0/1729211443 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df80092a0 con 0x7f4e0812cde0 2023-09-30T22:27:14.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.839+0000 7f4e07fff700 1 --2- 172.21.15.16:0/1729211443 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4e0812cde0 0x7f4e08112650 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f4e0000be70 tx=0x7f4e00009840).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:14.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.839+0000 7f4dff7fe700 1 -- 172.21.15.16:0/1729211443 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0000b640 con 0x7f4e0812cde0 2023-09-30T22:27:14.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.839+0000 7f4dff7fe700 1 -- 172.21.15.16:0/1729211443 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e000092e0 con 0x7f4e0812cde0 2023-09-30T22:27:14.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.839+0000 7f4dff7fe700 1 -- 172.21.15.16:0/1729211443 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e00007840 con 0x7f4e0812cde0 2023-09-30T22:27:14.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.839+0000 7f4e0df70700 1 -- 172.21.15.16:0/1729211443 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e00009ae0 con 0x7f4e0812cde0 2023-09-30T22:27:14.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.839+0000 7f4e0df70700 1 -- 172.21.15.16:0/1729211443 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e081161b0 con 0x7f4e0812cde0 2023-09-30T22:27:14.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.840+0000 7f4dff7fe700 1 -- 172.21.15.16:0/1729211443 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f4e00003040 con 0x7f4e0812cde0 2023-09-30T22:27:14.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.840+0000 7f4dff7fe700 1 -- 172.21.15.16:0/1729211443 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f4e0003cb60 con 0x7f4e0812cde0 2023-09-30T22:27:14.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.840+0000 7f4e0df70700 1 -- 172.21.15.16:0/1729211443 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e081151c0 con 0x7f4e0812cde0 2023-09-30T22:27:14.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:14.845+0000 7f4dff7fe700 1 -- 172.21.15.16:0/1729211443 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f4e081151c0 con 0x7f4e0812cde0 2023-09-30T22:27:15.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.017+0000 7f4e0df70700 1 -- 172.21.15.16:0/1729211443 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4e081151c0 con 0x7f4e0812cde0 2023-09-30T22:27:15.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.018+0000 7f4dff7fe700 1 -- 172.21.15.16:0/1729211443 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v17) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4e081151c0 con 0x7f4e0812cde0 2023-09-30T22:27:15.018 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-09-30T22:27:15.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.020+0000 7f4e0df70700 1 -- 172.21.15.16:0/1729211443 >> v1:172.21.15.16:6816/118285 conn(0x7f4de802bb60 legacy=0x7f4de802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:15.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.020+0000 7f4e0df70700 1 -- 172.21.15.16:0/1729211443 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4e0812cde0 msgr2=0x7f4e08112650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:15.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.020+0000 7f4e0df70700 1 --2- 172.21.15.16:0/1729211443 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4e0812cde0 0x7f4e08112650 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f4e0000be70 tx=0x7f4e00009840).stop 2023-09-30T22:27:15.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.020+0000 7f4e0df70700 1 -- 172.21.15.16:0/1729211443 shutdown_connections 2023-09-30T22:27:15.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.020+0000 7f4e0df70700 1 --2- 172.21.15.16:0/1729211443 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4e0812cde0 0x7f4e08112650 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:15.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.020+0000 7f4e0df70700 1 --2- 172.21.15.16:0/1729211443 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4e080613a0 0x7f4e08111c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:15.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.020+0000 7f4e0df70700 1 --2- 172.21.15.16:0/1729211443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e08091ed0 0x7f4e08112160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:15.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.020+0000 7f4e0df70700 1 -- 172.21.15.16:0/1729211443 >> 172.21.15.16:0/1729211443 conn(0x7f4e0809a1c0 msgr2=0x7f4e0812a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:15.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.020+0000 7f4e0df70700 1 -- 172.21.15.16:0/1729211443 shutdown_connections 2023-09-30T22:27:15.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.020+0000 7f4e0df70700 1 -- 172.21.15.16:0/1729211443 wait complete. 2023-09-30T22:27:15.033 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-30T22:27:15.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.166+0000 7f7f18c79700 1 Processor -- start 2023-09-30T22:27:15.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.166+0000 7f7f18c79700 1 -- start start 2023-09-30T22:27:15.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.167+0000 7f7f18c79700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f141275c0 0x7f7f1412fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:15.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.167+0000 7f7f18c79700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7f14130220 0x7f7f14132610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:15.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.167+0000 7f7f18c79700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f14091c80 con 0x7f7f140926d0 2023-09-30T22:27:15.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.167+0000 7f7f18c79700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f141279e0 con 0x7f7f14130220 2023-09-30T22:27:15.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.167+0000 7f7f18c79700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7f140999d0 con 0x7f7f141275c0 2023-09-30T22:27:15.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.167+0000 7f7f12d9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7f14130220 0x7f7f14132610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:15.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.167+0000 7f7f1259c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7f140926d0 0x7f7f14127be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53274/0 (socket says 172.21.15.16:53274) 2023-09-30T22:27:15.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.167+0000 7f7f11d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f141275c0 0x7f7f1412fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:15.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.167+0000 7f7f1259c700 1 -- 172.21.15.16:0/1353063582 learned_addr learned my addr 172.21.15.16:0/1353063582 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:27:15.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.167+0000 7f7f11d9b700 1 -- 172.21.15.16:0/1353063582 >> v2:172.21.15.16:3300/0 conn(0x7f7f14130220 msgr2=0x7f7f14132610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:15.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.167+0000 7f7f11d9b700 1 --2- 172.21.15.16:0/1353063582 >> v2:172.21.15.16:3300/0 conn(0x7f7f14130220 0x7f7f14132610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:15.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.167+0000 7f7f11d9b700 1 -- 172.21.15.16:0/1353063582 >> v1:172.21.15.16:6789/0 conn(0x7f7f140926d0 legacy=0x7f7f14127be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:15.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.167+0000 7f7f11d9b700 1 -- 172.21.15.16:0/1353063582 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f1409a280 con 0x7f7f141275c0 2023-09-30T22:27:15.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.167+0000 7f7f1159a700 1 -- 172.21.15.16:0/1353063582 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1275313790 0 0) 0x7f7f14091c80 con 0x7f7f140926d0 2023-09-30T22:27:15.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.167+0000 7f7f12d9d700 1 --2- 172.21.15.16:0/1353063582 >> v2:172.21.15.16:3300/0 conn(0x7f7f14130220 0x7f7f14132610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:15.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.167+0000 7f7f11d9b700 1 --2- 172.21.15.16:0/1353063582 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f141275c0 0x7f7f1412fd30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f7efc009760 tx=0x7f7efc009060).ready entity=mon.2 client_cookie=9a15a1209bb6e023 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:15.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.167+0000 7f7f1159a700 1 -- 172.21.15.16:0/1353063582 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7efc00f510 con 0x7f7f141275c0 2023-09-30T22:27:15.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.168+0000 7f7f1159a700 1 -- 172.21.15.16:0/1353063582 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7efc00f670 con 0x7f7f141275c0 2023-09-30T22:27:15.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.168+0000 7f7f1159a700 1 -- 172.21.15.16:0/1353063582 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7efc00f9f0 con 0x7f7f141275c0 2023-09-30T22:27:15.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.168+0000 7f7f18c79700 1 -- 172.21.15.16:0/1353063582 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f141275c0 msgr2=0x7f7f1412fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:15.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.168+0000 7f7f18c79700 1 --2- 172.21.15.16:0/1353063582 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f141275c0 0x7f7f1412fd30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f7efc009760 tx=0x7f7efc009060).stop 2023-09-30T22:27:15.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.168+0000 7f7f18c79700 1 -- 172.21.15.16:0/1353063582 shutdown_connections 2023-09-30T22:27:15.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.168+0000 7f7f18c79700 1 --2- 172.21.15.16:0/1353063582 >> v2:172.21.15.16:3300/0 conn(0x7f7f14130220 0x7f7f14132610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:15.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.168+0000 7f7f18c79700 1 --2- 172.21.15.16:0/1353063582 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f141275c0 0x7f7f1412fd30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:15.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.168+0000 7f7f18c79700 1 -- 172.21.15.16:0/1353063582 >> 172.21.15.16:0/1353063582 conn(0x7f7f14097480 msgr2=0x7f7f14097890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:15.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.168+0000 7f7f18c79700 1 -- 172.21.15.16:0/1353063582 shutdown_connections 2023-09-30T22:27:15.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.168+0000 7f7f18c79700 1 -- 172.21.15.16:0/1353063582 wait complete. 2023-09-30T22:27:15.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.169+0000 7f7f18c79700 1 Processor -- start 2023-09-30T22:27:15.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.169+0000 7f7f18c79700 1 -- start start 2023-09-30T22:27:15.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.170+0000 7f7f18c79700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f140926d0 0x7f7f1412e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:15.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.170+0000 7f7f18c79700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f141275c0 0x7f7f1412e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:15.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.170+0000 7f7f18c79700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f14130220 0x7f7f1412ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:15.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.170+0000 7f7f18c79700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1412ef30 con 0x7f7f140926d0 2023-09-30T22:27:15.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.170+0000 7f7f18c79700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f14113880 con 0x7f7f14130220 2023-09-30T22:27:15.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.170+0000 7f7f18c79700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7f14112700 con 0x7f7f141275c0 2023-09-30T22:27:15.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.170+0000 7f7f12d9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f14130220 0x7f7f1412ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:15.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.170+0000 7f7f12d9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f14130220 0x7f7f1412ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:49316/0 (socket says 172.21.15.16:49316) 2023-09-30T22:27:15.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.170+0000 7f7f12d9d700 1 -- 172.21.15.16:0/1204007750 learned_addr learned my addr 172.21.15.16:0/1204007750 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:15.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.170+0000 7f7f11d9b700 1 --2- 172.21.15.16:0/1204007750 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f141275c0 0x7f7f1412e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:15.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.170+0000 7f7f1259c700 1 --2- 172.21.15.16:0/1204007750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f140926d0 0x7f7f1412e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:15.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.170+0000 7f7f11d9b700 1 -- 172.21.15.16:0/1204007750 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f14130220 msgr2=0x7f7f1412ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:15.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.170+0000 7f7f11d9b700 1 --2- 172.21.15.16:0/1204007750 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f14130220 0x7f7f1412ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:15.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.170+0000 7f7f11d9b700 1 -- 172.21.15.16:0/1204007750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f140926d0 msgr2=0x7f7f1412e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:15.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.170+0000 7f7f11d9b700 1 --2- 172.21.15.16:0/1204007750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f140926d0 0x7f7f1412e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:15.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.170+0000 7f7f11d9b700 1 -- 172.21.15.16:0/1204007750 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f0c009a20 con 0x7f7f141275c0 2023-09-30T22:27:15.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.170+0000 7f7f11d9b700 1 --2- 172.21.15.16:0/1204007750 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f141275c0 0x7f7f1412e550 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f7f14065730 tx=0x7f7efc017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:15.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.170+0000 7f7f0a7fc700 1 -- 172.21.15.16:0/1204007750 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7efc00f7e0 con 0x7f7f141275c0 2023-09-30T22:27:15.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.170+0000 7f7f0a7fc700 1 -- 172.21.15.16:0/1204007750 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7efc00f940 con 0x7f7f141275c0 2023-09-30T22:27:15.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.170+0000 7f7f18c79700 1 -- 172.21.15.16:0/1204007750 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7efc0092a0 con 0x7f7f141275c0 2023-09-30T22:27:15.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.171+0000 7f7f0a7fc700 1 -- 172.21.15.16:0/1204007750 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7efc01b660 con 0x7f7f141275c0 2023-09-30T22:27:15.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.171+0000 7f7f18c79700 1 -- 172.21.15.16:0/1204007750 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f141124c0 con 0x7f7f141275c0 2023-09-30T22:27:15.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.171+0000 7f7f0a7fc700 1 -- 172.21.15.16:0/1204007750 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f7efc01ba50 con 0x7f7f141275c0 2023-09-30T22:27:15.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.171+0000 7f7f0a7fc700 1 -- 172.21.15.16:0/1204007750 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f7efc03c6c0 con 0x7f7f141275c0 2023-09-30T22:27:15.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.172+0000 7f7f18c79700 1 -- 172.21.15.16:0/1204007750 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f1409a4b0 con 0x7f7f141275c0 2023-09-30T22:27:15.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.176+0000 7f7f0a7fc700 1 -- 172.21.15.16:0/1204007750 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f7efc01f080 con 0x7f7f141275c0 2023-09-30T22:27:15.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.354+0000 7f7f18c79700 1 -- 172.21.15.16:0/1204007750 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7f1405a260 con 0x7f7f141275c0 2023-09-30T22:27:15.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.355+0000 7f7f0a7fc700 1 -- 172.21.15.16:0/1204007750 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v17) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7efc029020 con 0x7f7f141275c0 2023-09-30T22:27:15.355 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-09-30T22:27:15.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.357+0000 7f7f18c79700 1 -- 172.21.15.16:0/1204007750 >> v1:172.21.15.16:6816/118285 conn(0x7f7ef402bb60 legacy=0x7f7ef402e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:15.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.357+0000 7f7f18c79700 1 -- 172.21.15.16:0/1204007750 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f141275c0 msgr2=0x7f7f1412e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:15.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.357+0000 7f7f18c79700 1 --2- 172.21.15.16:0/1204007750 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f141275c0 0x7f7f1412e550 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f7f14065730 tx=0x7f7efc017000).stop 2023-09-30T22:27:15.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.357+0000 7f7f18c79700 1 -- 172.21.15.16:0/1204007750 shutdown_connections 2023-09-30T22:27:15.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.357+0000 7f7f18c79700 1 --2- 172.21.15.16:0/1204007750 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f14130220 0x7f7f1412ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:15.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.357+0000 7f7f18c79700 1 --2- 172.21.15.16:0/1204007750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f140926d0 0x7f7f1412e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:15.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.357+0000 7f7f18c79700 1 --2- 172.21.15.16:0/1204007750 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f141275c0 0x7f7f1412e550 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:15.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.357+0000 7f7f18c79700 1 -- 172.21.15.16:0/1204007750 >> 172.21.15.16:0/1204007750 conn(0x7f7f14097480 msgr2=0x7f7f14092d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:15.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.357+0000 7f7f18c79700 1 -- 172.21.15.16:0/1204007750 shutdown_connections 2023-09-30T22:27:15.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.357+0000 7f7f18c79700 1 -- 172.21.15.16:0/1204007750 wait complete. 2023-09-30T22:27:15.370 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-30T22:27:15.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.500+0000 7f747df98700 1 Processor -- start 2023-09-30T22:27:15.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.500+0000 7f747df98700 1 -- start start 2023-09-30T22:27:15.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.501+0000 7f747df98700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7478091b00 0x7f747812c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:15.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.501+0000 7f747df98700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f747812ce60 0x7f747812f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:15.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.501+0000 7f747df98700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7478096760 con 0x7f7478096380 2023-09-30T22:27:15.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.501+0000 7f747df98700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7478091ff0 con 0x7f7478091b00 2023-09-30T22:27:15.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.501+0000 7f747df98700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7478092c80 con 0x7f747812ce60 2023-09-30T22:27:15.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.501+0000 7f7477fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f747812ce60 0x7f747812f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:15.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.501+0000 7f7476ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7478091b00 0x7f747812c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:15.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.501+0000 7f74777fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7478096380 0x7f7478094730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53282/0 (socket says 172.21.15.16:53282) 2023-09-30T22:27:15.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.501+0000 7f74777fe700 1 -- 172.21.15.16:0/2739300146 learned_addr learned my addr 172.21.15.16:0/2739300146 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:27:15.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.501+0000 7f7476ffd700 1 -- 172.21.15.16:0/2739300146 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f747812ce60 msgr2=0x7f747812f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:15.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.502+0000 7f7476ffd700 1 --2- 172.21.15.16:0/2739300146 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f747812ce60 0x7f747812f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:15.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.502+0000 7f7476ffd700 1 -- 172.21.15.16:0/2739300146 >> v1:172.21.15.16:6789/0 conn(0x7f7478096380 legacy=0x7f7478094730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:15.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.502+0000 7f7476ffd700 1 -- 172.21.15.16:0/2739300146 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7478094e40 con 0x7f7478091b00 2023-09-30T22:27:15.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.502+0000 7f7477fff700 1 --2- 172.21.15.16:0/2739300146 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f747812ce60 0x7f747812f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:15.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.502+0000 7f7476ffd700 1 --2- 172.21.15.16:0/2739300146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7478091b00 0x7f747812c970 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f7460009760 tx=0x7f7460009060).ready entity=mon.0 client_cookie=1d6e7d8f0c3119d3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:15.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.502+0000 7f74767fc700 1 -- 172.21.15.16:0/2739300146 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746000f510 con 0x7f7478091b00 2023-09-30T22:27:15.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.502+0000 7f74767fc700 1 -- 172.21.15.16:0/2739300146 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f746000f670 con 0x7f7478091b00 2023-09-30T22:27:15.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.502+0000 7f74767fc700 1 -- 172.21.15.16:0/2739300146 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746000f9f0 con 0x7f7478091b00 2023-09-30T22:27:15.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.502+0000 7f747df98700 1 -- 172.21.15.16:0/2739300146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7478091b00 msgr2=0x7f747812c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:15.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.502+0000 7f747df98700 1 --2- 172.21.15.16:0/2739300146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7478091b00 0x7f747812c970 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f7460009760 tx=0x7f7460009060).stop 2023-09-30T22:27:15.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.502+0000 7f747df98700 1 -- 172.21.15.16:0/2739300146 shutdown_connections 2023-09-30T22:27:15.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.502+0000 7f747df98700 1 --2- 172.21.15.16:0/2739300146 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f747812ce60 0x7f747812f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:15.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.502+0000 7f747df98700 1 --2- 172.21.15.16:0/2739300146 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7478091b00 0x7f747812c970 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:15.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.502+0000 7f747df98700 1 -- 172.21.15.16:0/2739300146 >> 172.21.15.16:0/2739300146 conn(0x7f7478097470 msgr2=0x7f7478097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:15.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.503+0000 7f747df98700 1 -- 172.21.15.16:0/2739300146 shutdown_connections 2023-09-30T22:27:15.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.503+0000 7f747df98700 1 -- 172.21.15.16:0/2739300146 wait complete. 2023-09-30T22:27:15.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.504+0000 7f747df98700 1 Processor -- start 2023-09-30T22:27:15.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.504+0000 7f747df98700 1 -- start start 2023-09-30T22:27:15.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.504+0000 7f747df98700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7478091b00 0x7f7478137470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:15.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.504+0000 7f747df98700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7478096380 0x7f7478137960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:15.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.504+0000 7f747df98700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f747812ce60 0x7f7478068740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:15.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.504+0000 7f747df98700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f747812d240 con 0x7f7478096380 2023-09-30T22:27:15.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.504+0000 7f747df98700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f747809a060 con 0x7f7478091b00 2023-09-30T22:27:15.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.504+0000 7f747df98700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f74781129e0 con 0x7f747812ce60 2023-09-30T22:27:15.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.504+0000 7f74777fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7478091b00 0x7f7478137470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:15.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.504+0000 7f7476ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7478096380 0x7f7478137960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:15.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.504+0000 7f7477fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f747812ce60 0x7f7478068740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:15.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.504+0000 7f7476ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7478096380 0x7f7478137960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41076/0 (socket says 172.21.15.16:41076) 2023-09-30T22:27:15.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.504+0000 7f74777fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7478091b00 0x7f7478137470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:49336/0 (socket says 172.21.15.16:49336) 2023-09-30T22:27:15.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.504+0000 7f7476ffd700 1 -- 172.21.15.16:0/157092138 learned_addr learned my addr 172.21.15.16:0/157092138 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:15.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.505+0000 7f7476ffd700 1 -- 172.21.15.16:0/157092138 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f747812ce60 msgr2=0x7f7478068740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:15.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.505+0000 7f7476ffd700 1 --2- 172.21.15.16:0/157092138 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f747812ce60 0x7f7478068740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:15.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.505+0000 7f7476ffd700 1 -- 172.21.15.16:0/157092138 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7478091b00 msgr2=0x7f7478137470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:15.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.505+0000 7f7476ffd700 1 --2- 172.21.15.16:0/157092138 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7478091b00 0x7f7478137470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:15.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.505+0000 7f7476ffd700 1 -- 172.21.15.16:0/157092138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7470009a20 con 0x7f7478096380 2023-09-30T22:27:15.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.505+0000 7f7476ffd700 1 --2- 172.21.15.16:0/157092138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7478096380 0x7f7478137960 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f7460009e00 tx=0x7f7460017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:15.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.505+0000 7f746f7fe700 1 -- 172.21.15.16:0/157092138 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746000f7e0 con 0x7f7478096380 2023-09-30T22:27:15.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.505+0000 7f746f7fe700 1 -- 172.21.15.16:0/157092138 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f746000f960 con 0x7f7478096380 2023-09-30T22:27:15.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.505+0000 7f747df98700 1 -- 172.21.15.16:0/157092138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74600092a0 con 0x7f7478096380 2023-09-30T22:27:15.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.505+0000 7f746f7fe700 1 -- 172.21.15.16:0/157092138 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746001b660 con 0x7f7478096380 2023-09-30T22:27:15.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.505+0000 7f747df98700 1 -- 172.21.15.16:0/157092138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74781133a0 con 0x7f7478096380 2023-09-30T22:27:15.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.506+0000 7f746f7fe700 1 -- 172.21.15.16:0/157092138 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f746001b9d0 con 0x7f7478096380 2023-09-30T22:27:15.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.506+0000 7f747df98700 1 -- 172.21.15.16:0/157092138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f747805a260 con 0x7f7478096380 2023-09-30T22:27:15.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.506+0000 7f746f7fe700 1 -- 172.21.15.16:0/157092138 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f746003c810 con 0x7f7478096380 2023-09-30T22:27:15.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.511+0000 7f746f7fe700 1 -- 172.21.15.16:0/157092138 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f746002c940 con 0x7f7478096380 2023-09-30T22:27:15.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.733+0000 7f747df98700 1 -- 172.21.15.16:0/157092138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f74780939f0 con 0x7f7478096380 2023-09-30T22:27:15.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.733+0000 7f746f7fe700 1 -- 172.21.15.16:0/157092138 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+2156 (secure 0 0 0) 0x7f746001f070 con 0x7f7478096380 2023-09-30T22:27:15.734 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:27:15.734 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-30T22:26:51.674561+0000","modified":"2023-09-30T22:26:54.680606+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4218},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4218":{"gid":4218,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6817/1424366610","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6817","nonce":1424366610}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-09-30T22:27:15.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.735+0000 7f747df98700 1 -- 172.21.15.16:0/157092138 >> v1:172.21.15.16:6816/118285 conn(0x7f745802bb60 legacy=0x7f745802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:15.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.735+0000 7f747df98700 1 -- 172.21.15.16:0/157092138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7478096380 msgr2=0x7f7478137960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:15.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.735+0000 7f747df98700 1 --2- 172.21.15.16:0/157092138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7478096380 0x7f7478137960 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f7460009e00 tx=0x7f7460017000).stop 2023-09-30T22:27:15.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.735+0000 7f747df98700 1 -- 172.21.15.16:0/157092138 shutdown_connections 2023-09-30T22:27:15.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.735+0000 7f747df98700 1 --2- 172.21.15.16:0/157092138 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7478091b00 0x7f7478137470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:15.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.735+0000 7f747df98700 1 --2- 172.21.15.16:0/157092138 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f747812ce60 0x7f7478068740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:15.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.735+0000 7f747df98700 1 --2- 172.21.15.16:0/157092138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7478096380 0x7f7478137960 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:15.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.735+0000 7f747df98700 1 -- 172.21.15.16:0/157092138 >> 172.21.15.16:0/157092138 conn(0x7f7478097470 msgr2=0x7f7478126780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:15.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.735+0000 7f747df98700 1 -- 172.21.15.16:0/157092138 shutdown_connections 2023-09-30T22:27:15.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.736+0000 7f747df98700 1 -- 172.21.15.16:0/157092138 wait complete. 2023-09-30T22:27:15.737 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 6 2023-09-30T22:27:15.749 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:27:15.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.878+0000 7f25e68b1700 1 Processor -- start 2023-09-30T22:27:15.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.878+0000 7f25e68b1700 1 -- start start 2023-09-30T22:27:15.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.879+0000 7f25e68b1700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f25e0096820 0x7f25e0096c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:15.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.879+0000 7f25e68b1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f25e0097800 0x7f25e0130280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:15.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.879+0000 7f25e68b1700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25e0068df0 con 0x7f25e0067940 2023-09-30T22:27:15.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.879+0000 7f25e68b1700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25e0097c90 con 0x7f25e0096820 2023-09-30T22:27:15.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.879+0000 7f25e68b1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f25e006a4c0 con 0x7f25e0097800 2023-09-30T22:27:15.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.879+0000 7f25e4e4e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f25e0097800 0x7f25e0130280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:15.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.879+0000 7f25dffff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f25e0096820 0x7f25e0096c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:15.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.879+0000 7f25e4e4e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f25e0097800 0x7f25e0130280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37866/0 (socket says 172.21.15.16:37866) 2023-09-30T22:27:15.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.879+0000 7f25e4e4e700 1 -- 172.21.15.16:0/2608134235 learned_addr learned my addr 172.21.15.16:0/2608134235 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:15.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.879+0000 7f25dffff700 1 -- 172.21.15.16:0/2608134235 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f25e0097800 msgr2=0x7f25e0130280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:15.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.879+0000 7f25dffff700 1 --2- 172.21.15.16:0/2608134235 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f25e0097800 0x7f25e0130280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:15.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.879+0000 7f25dffff700 1 -- 172.21.15.16:0/2608134235 >> v1:172.21.15.16:6789/0 conn(0x7f25e0067940 legacy=0x7f25e00970f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:15.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.880+0000 7f25deffd700 1 -- 172.21.15.16:0/2608134235 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2761602686 0 0) 0x7f25e0068df0 con 0x7f25e0067940 2023-09-30T22:27:15.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.880+0000 7f25dffff700 1 -- 172.21.15.16:0/2608134235 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e0098260 con 0x7f25e0096820 2023-09-30T22:27:15.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.880+0000 7f25e4e4e700 1 --2- 172.21.15.16:0/2608134235 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f25e0097800 0x7f25e0130280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:15.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.880+0000 7f25dffff700 1 --2- 172.21.15.16:0/2608134235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25e0096820 0x7f25e0096c00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f25c8009760 tx=0x7f25c8009060).ready entity=mon.0 client_cookie=2fe66391bd8cf0fd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:15.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.880+0000 7f25deffd700 1 -- 172.21.15.16:0/2608134235 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c800f510 con 0x7f25e0096820 2023-09-30T22:27:15.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.880+0000 7f25deffd700 1 -- 172.21.15.16:0/2608134235 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25c800f670 con 0x7f25e0096820 2023-09-30T22:27:15.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.880+0000 7f25deffd700 1 -- 172.21.15.16:0/2608134235 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c800f9f0 con 0x7f25e0096820 2023-09-30T22:27:15.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.880+0000 7f25e68b1700 1 -- 172.21.15.16:0/2608134235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25e0096820 msgr2=0x7f25e0096c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:15.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.880+0000 7f25e68b1700 1 --2- 172.21.15.16:0/2608134235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25e0096820 0x7f25e0096c00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f25c8009760 tx=0x7f25c8009060).stop 2023-09-30T22:27:15.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.880+0000 7f25e68b1700 1 -- 172.21.15.16:0/2608134235 shutdown_connections 2023-09-30T22:27:15.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.880+0000 7f25e68b1700 1 --2- 172.21.15.16:0/2608134235 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f25e0097800 0x7f25e0130280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:15.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.880+0000 7f25e68b1700 1 --2- 172.21.15.16:0/2608134235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25e0096820 0x7f25e0096c00 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:15.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.880+0000 7f25e68b1700 1 -- 172.21.15.16:0/2608134235 >> 172.21.15.16:0/2608134235 conn(0x7f25e009ba50 msgr2=0x7f25e009be60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:15.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.880+0000 7f25e68b1700 1 -- 172.21.15.16:0/2608134235 shutdown_connections 2023-09-30T22:27:15.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.881+0000 7f25e68b1700 1 -- 172.21.15.16:0/2608134235 wait complete. 2023-09-30T22:27:15.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.881+0000 7f25e68b1700 1 Processor -- start 2023-09-30T22:27:15.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.882+0000 7f25e68b1700 1 -- start start 2023-09-30T22:27:15.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.882+0000 7f25e68b1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f25e0067940 0x7f25e011c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:15.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.882+0000 7f25e68b1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f25e0096820 0x7f25e0118360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:15.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.882+0000 7f25e68b1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25e0097800 0x7f25e0118850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:15.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.882+0000 7f25e68b1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e0119800 con 0x7f25e0097800 2023-09-30T22:27:15.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.882+0000 7f25e68b1700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25e0130380 con 0x7f25e0067940 2023-09-30T22:27:15.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.882+0000 7f25e68b1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f25e011b550 con 0x7f25e0096820 2023-09-30T22:27:15.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.882+0000 7f25dffff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f25e0067940 0x7f25e011c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:15.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.882+0000 7f25e4e4e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25e0097800 0x7f25e0118850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:15.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.882+0000 7f25dffff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f25e0067940 0x7f25e011c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:49346/0 (socket says 172.21.15.16:49346) 2023-09-30T22:27:15.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.882+0000 7f25dffff700 1 -- 172.21.15.16:0/993882804 learned_addr learned my addr 172.21.15.16:0/993882804 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:15.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.882+0000 7f25e4e4e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25e0097800 0x7f25e0118850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41088/0 (socket says 172.21.15.16:41088) 2023-09-30T22:27:15.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.882+0000 7f25df7fe700 1 --2- 172.21.15.16:0/993882804 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f25e0096820 0x7f25e0118360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:15.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.883+0000 7f25e4e4e700 1 -- 172.21.15.16:0/993882804 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f25e0096820 msgr2=0x7f25e0118360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:15.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.883+0000 7f25e4e4e700 1 --2- 172.21.15.16:0/993882804 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f25e0096820 0x7f25e0118360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:15.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.883+0000 7f25e4e4e700 1 -- 172.21.15.16:0/993882804 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f25e0067940 msgr2=0x7f25e011c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:15.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.883+0000 7f25e4e4e700 1 --2- 172.21.15.16:0/993882804 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f25e0067940 0x7f25e011c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:15.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.883+0000 7f25e4e4e700 1 -- 172.21.15.16:0/993882804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d8009ae0 con 0x7f25e0097800 2023-09-30T22:27:15.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.883+0000 7f25dffff700 1 --2- 172.21.15.16:0/993882804 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f25e0067940 0x7f25e011c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:15.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.883+0000 7f25df7fe700 1 --2- 172.21.15.16:0/993882804 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f25e0096820 0x7f25e0118360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:15.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.883+0000 7f25e4e4e700 1 --2- 172.21.15.16:0/993882804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25e0097800 0x7f25e0118850 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f25d8009f10 tx=0x7f25d800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:15.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.883+0000 7f25d7fff700 1 -- 172.21.15.16:0/993882804 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d80092e0 con 0x7f25e0097800 2023-09-30T22:27:15.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.883+0000 7f25d7fff700 1 -- 172.21.15.16:0/993882804 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25d800f040 con 0x7f25e0097800 2023-09-30T22:27:15.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.883+0000 7f25e68b1700 1 -- 172.21.15.16:0/993882804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25c80092a0 con 0x7f25e0097800 2023-09-30T22:27:15.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.883+0000 7f25d7fff700 1 -- 172.21.15.16:0/993882804 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d800cb70 con 0x7f25e0097800 2023-09-30T22:27:15.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.883+0000 7f25e68b1700 1 -- 172.21.15.16:0/993882804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25e011a5f0 con 0x7f25e0097800 2023-09-30T22:27:15.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.884+0000 7f25d7fff700 1 -- 172.21.15.16:0/993882804 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f25d8014460 con 0x7f25e0097800 2023-09-30T22:27:15.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.884+0000 7f25d7fff700 1 -- 172.21.15.16:0/993882804 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f25d803afa0 con 0x7f25e0097800 2023-09-30T22:27:15.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.884+0000 7f25e68b1700 1 -- 172.21.15.16:0/993882804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25e009b790 con 0x7f25e0097800 2023-09-30T22:27:15.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:15.889+0000 7f25d7fff700 1 -- 172.21.15.16:0/993882804 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f25d8012360 con 0x7f25e0097800 2023-09-30T22:27:16.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.065+0000 7f25e68b1700 1 -- 172.21.15.16:0/993882804 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f25e011b9d0 con 0x7f25e0097800 2023-09-30T22:27:16.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.066+0000 7f25d7fff700 1 -- 172.21.15.16:0/993882804 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+11340 (secure 0 0 0) 0x7f25d801cb10 con 0x7f25e0097800 2023-09-30T22:27:16.067 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:27:16.067 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":17,"fsid":"7ede3e14-2bd0-4726-9437-6d13c90cd342","created":"2023-09-30T22:26:35.483697+0000","modified":"2023-09-30T22:27:11.244326+0000","last_up_change":"2023-09-30T22:26:44.227199+0000","last_in_change":"2023-09-30T22:26:37.120906+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-09-30T22:26:44.285133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:26:45.132921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:26:49.353683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:26:50.655685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"19946985-0ee2-487f-9347-929670dbe5ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6804","nonce":118824}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6805","nonce":118824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6807","nonce":118824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6806","nonce":118824}]},"public_addr":"172.21.15.16:6804/118824","cluster_addr":"172.21.15.16:6805/118824","heartbeat_back_addr":"172.21.15.16:6807/118824","heartbeat_front_addr":"172.21.15.16:6806/118824","state":["exists","up"]},{"osd":1,"uuid":"8efd4dbc-6ae4-4b82-b530-3b3eac2be445","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6812","nonce":118832}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6813","nonce":118832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6815","nonce":118832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6814","nonce":118832}]},"public_addr":"172.21.15.16:6812/118832","cluster_addr":"172.21.15.16:6813/118832","heartbeat_back_addr":"172.21.15.16:6815/118832","heartbeat_front_addr":"172.21.15.16:6814/118832","state":["exists","up"]},{"osd":2,"uuid":"b47bb1ce-27af-4c0c-868a-b76ebd0d9499","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6808","nonce":118840}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6809","nonce":118840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6811","nonce":118840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6810","nonce":118840}]},"public_addr":"172.21.15.16:6808/118840","cluster_addr":"172.21.15.16:6809/118840","heartbeat_back_addr":"172.21.15.16:6811/118840","heartbeat_front_addr":"172.21.15.16:6810/118840","state":["exists","up"]},{"osd":3,"uuid":"03b53d64-d10d-448a-a081-39ecceab589b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6800","nonce":118823}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6801","nonce":118823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6803","nonce":118823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6802","nonce":118823}]},"public_addr":"172.21.15.16:6800/118823","cluster_addr":"172.21.15.16:6801/118823","heartbeat_back_addr":"172.21.15.16:6803/118823","heartbeat_front_addr":"172.21.15.16:6802/118823","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.305953+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.307488+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.341023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.192071+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:27:16.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.068+0000 7f25e68b1700 1 -- 172.21.15.16:0/993882804 >> v1:172.21.15.16:6816/118285 conn(0x7f25c002bb60 legacy=0x7f25c002e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:16.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.068+0000 7f25e68b1700 1 -- 172.21.15.16:0/993882804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25e0097800 msgr2=0x7f25e0118850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:16.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.068+0000 7f25e68b1700 1 --2- 172.21.15.16:0/993882804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25e0097800 0x7f25e0118850 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f25d8009f10 tx=0x7f25d800b040).stop 2023-09-30T22:27:16.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.068+0000 7f25e68b1700 1 -- 172.21.15.16:0/993882804 shutdown_connections 2023-09-30T22:27:16.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.068+0000 7f25e68b1700 1 --2- 172.21.15.16:0/993882804 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f25e0067940 0x7f25e011c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:16.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.068+0000 7f25e68b1700 1 --2- 172.21.15.16:0/993882804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25e0097800 0x7f25e0118850 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:16.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.068+0000 7f25e68b1700 1 --2- 172.21.15.16:0/993882804 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f25e0096820 0x7f25e0118360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:16.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.068+0000 7f25e68b1700 1 -- 172.21.15.16:0/993882804 >> 172.21.15.16:0/993882804 conn(0x7f25e009ba50 msgr2=0x7f25e0099880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:16.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.069+0000 7f25e68b1700 1 -- 172.21.15.16:0/993882804 shutdown_connections 2023-09-30T22:27:16.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.069+0000 7f25e68b1700 1 -- 172.21.15.16:0/993882804 wait complete. 2023-09-30T22:27:16.082 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-09-30T22:27:16.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.215+0000 7f320abc0700 1 Processor -- start 2023-09-30T22:27:16.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.215+0000 7f320abc0700 1 -- start start 2023-09-30T22:27:16.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.216+0000 7f320abc0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3204099230 0x7f320412dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:16.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.216+0000 7f320abc0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f320412e0d0 0x7f32041304c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:16.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.216+0000 7f320abc0700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3204095270 con 0x7f3204098740 2023-09-30T22:27:16.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.216+0000 7f320abc0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3204094d20 con 0x7f320412e0d0 2023-09-30T22:27:16.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.216+0000 7f320abc0700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3204096520 con 0x7f3204099230 2023-09-30T22:27:16.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.216+0000 7f320915d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f320412e0d0 0x7f32041304c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:16.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.216+0000 7f320895c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3204098740 0x7f3204098b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53304/0 (socket says 172.21.15.16:53304) 2023-09-30T22:27:16.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.216+0000 7f320895c700 1 -- 172.21.15.16:0/1344836362 learned_addr learned my addr 172.21.15.16:0/1344836362 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:27:16.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.216+0000 7f3203fff700 1 --2- 172.21.15.16:0/1344836362 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3204099230 0x7f320412dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:16.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.216+0000 7f3203fff700 1 -- 172.21.15.16:0/1344836362 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f320412e0d0 msgr2=0x7f32041304c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:16.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.216+0000 7f3203fff700 1 --2- 172.21.15.16:0/1344836362 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f320412e0d0 0x7f32041304c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:16.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.216+0000 7f3203fff700 1 -- 172.21.15.16:0/1344836362 >> v1:172.21.15.16:6789/0 conn(0x7f3204098740 legacy=0x7f3204098b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:16.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.216+0000 7f32037fe700 1 -- 172.21.15.16:0/1344836362 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3523012037 0 0) 0x7f3204095270 con 0x7f3204098740 2023-09-30T22:27:16.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.216+0000 7f3203fff700 1 -- 172.21.15.16:0/1344836362 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3204094870 con 0x7f3204099230 2023-09-30T22:27:16.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.216+0000 7f3203fff700 1 --2- 172.21.15.16:0/1344836362 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3204099230 0x7f320412dbe0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f31ec009790 tx=0x7f31ec009060).ready entity=mon.2 client_cookie=ad17a23ea0dda04e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:16.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.217+0000 7f32037fe700 1 -- 172.21.15.16:0/1344836362 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ec0090d0 con 0x7f3204099230 2023-09-30T22:27:16.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.217+0000 7f32037fe700 1 -- 172.21.15.16:0/1344836362 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31ec00f510 con 0x7f3204099230 2023-09-30T22:27:16.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.217+0000 7f32037fe700 1 -- 172.21.15.16:0/1344836362 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ec00f890 con 0x7f3204099230 2023-09-30T22:27:16.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.217+0000 7f320abc0700 1 -- 172.21.15.16:0/1344836362 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3204099230 msgr2=0x7f320412dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:16.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.217+0000 7f320abc0700 1 --2- 172.21.15.16:0/1344836362 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3204099230 0x7f320412dbe0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f31ec009790 tx=0x7f31ec009060).stop 2023-09-30T22:27:16.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.217+0000 7f320abc0700 1 -- 172.21.15.16:0/1344836362 shutdown_connections 2023-09-30T22:27:16.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.217+0000 7f320abc0700 1 --2- 172.21.15.16:0/1344836362 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f320412e0d0 0x7f32041304c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:16.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.217+0000 7f320abc0700 1 --2- 172.21.15.16:0/1344836362 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3204099230 0x7f320412dbe0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:16.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.217+0000 7f320abc0700 1 -- 172.21.15.16:0/1344836362 >> 172.21.15.16:0/1344836362 conn(0x7f320409a1c0 msgr2=0x7f320409a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:16.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.217+0000 7f320abc0700 1 -- 172.21.15.16:0/1344836362 shutdown_connections 2023-09-30T22:27:16.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.217+0000 7f320abc0700 1 -- 172.21.15.16:0/1344836362 wait complete. 2023-09-30T22:27:16.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.218+0000 7f320abc0700 1 Processor -- start 2023-09-30T22:27:16.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.218+0000 7f320abc0700 1 -- start start 2023-09-30T22:27:16.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.219+0000 7f320abc0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3204098740 0x7f320413a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:16.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.219+0000 7f320abc0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3204099230 0x7f320413ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:16.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.219+0000 7f320abc0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f320412e0d0 0x7f320413b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:16.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.219+0000 7f320abc0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f320411d520 con 0x7f3204099230 2023-09-30T22:27:16.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.219+0000 7f320abc0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3204066d80 con 0x7f320412e0d0 2023-09-30T22:27:16.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.219+0000 7f320abc0700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3204094f70 con 0x7f3204098740 2023-09-30T22:27:16.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.219+0000 7f320895c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3204098740 0x7f320413a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:16.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.219+0000 7f320915d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f320412e0d0 0x7f320413b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:16.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.219+0000 7f3203fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3204099230 0x7f320413ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:16.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.219+0000 7f3203fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3204099230 0x7f320413ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41090/0 (socket says 172.21.15.16:41090) 2023-09-30T22:27:16.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.219+0000 7f320895c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3204098740 0x7f320413a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37892/0 (socket says 172.21.15.16:37892) 2023-09-30T22:27:16.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.219+0000 7f320915d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f320412e0d0 0x7f320413b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:49352/0 (socket says 172.21.15.16:49352) 2023-09-30T22:27:16.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.219+0000 7f3203fff700 1 -- 172.21.15.16:0/2881472887 learned_addr learned my addr 172.21.15.16:0/2881472887 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:16.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.219+0000 7f320915d700 1 -- 172.21.15.16:0/2881472887 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3204098740 msgr2=0x7f320413a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:16.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.219+0000 7f320915d700 1 --2- 172.21.15.16:0/2881472887 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3204098740 0x7f320413a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:16.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.219+0000 7f320915d700 1 -- 172.21.15.16:0/2881472887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3204099230 msgr2=0x7f320413ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:16.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.219+0000 7f320915d700 1 --2- 172.21.15.16:0/2881472887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3204099230 0x7f320413ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:16.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.219+0000 7f320915d700 1 -- 172.21.15.16:0/2881472887 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31ec0092a0 con 0x7f320412e0d0 2023-09-30T22:27:16.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.219+0000 7f320895c700 1 --2- 172.21.15.16:0/2881472887 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3204098740 0x7f320413a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:16.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.219+0000 7f3203fff700 1 --2- 172.21.15.16:0/2881472887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3204099230 0x7f320413ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:16.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.220+0000 7f320915d700 1 --2- 172.21.15.16:0/2881472887 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f320412e0d0 0x7f320413b030 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f31fc009e20 tx=0x7f31fc00b040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:16.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.220+0000 7f3200ff9700 1 -- 172.21.15.16:0/2881472887 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31fc009bb0 con 0x7f320412e0d0 2023-09-30T22:27:16.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.220+0000 7f3200ff9700 1 -- 172.21.15.16:0/2881472887 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31fc00f040 con 0x7f320412e0d0 2023-09-30T22:27:16.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.220+0000 7f3200ff9700 1 -- 172.21.15.16:0/2881472887 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31fc00ccd0 con 0x7f320412e0d0 2023-09-30T22:27:16.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.220+0000 7f320abc0700 1 -- 172.21.15.16:0/2881472887 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f320411d200 con 0x7f320412e0d0 2023-09-30T22:27:16.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.220+0000 7f320abc0700 1 -- 172.21.15.16:0/2881472887 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f320411d770 con 0x7f320412e0d0 2023-09-30T22:27:16.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.221+0000 7f320abc0700 1 -- 172.21.15.16:0/2881472887 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f320411d390 con 0x7f320412e0d0 2023-09-30T22:27:16.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.221+0000 7f3200ff9700 1 -- 172.21.15.16:0/2881472887 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f31fc013610 con 0x7f320412e0d0 2023-09-30T22:27:16.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.221+0000 7f3200ff9700 1 -- 172.21.15.16:0/2881472887 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f31fc03b6c0 con 0x7f320412e0d0 2023-09-30T22:27:16.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.226+0000 7f3200ff9700 1 -- 172.21.15.16:0/2881472887 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f31fc007750 con 0x7f320412e0d0 2023-09-30T22:27:16.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:16.443+0000 7f320abc0700 1 -- 172.21.15.16:0/2881472887 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f3204094540 con 0x7f320412e0d0 2023-09-30T22:27:17.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.265+0000 7f3200ff9700 1 -- 172.21.15.16:0/2881472887 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) v1 ==== 122+0+0 (secure 0 0 0) 0x7f31fc007340 con 0x7f320412e0d0 2023-09-30T22:27:17.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.267+0000 7f320abc0700 1 -- 172.21.15.16:0/2881472887 >> v1:172.21.15.16:6816/118285 conn(0x7f31e402bb60 legacy=0x7f31e402e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:17.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.267+0000 7f320abc0700 1 -- 172.21.15.16:0/2881472887 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f320412e0d0 msgr2=0x7f320413b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:17.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.267+0000 7f320abc0700 1 --2- 172.21.15.16:0/2881472887 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f320412e0d0 0x7f320413b030 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f31fc009e20 tx=0x7f31fc00b040).stop 2023-09-30T22:27:17.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.267+0000 7f320abc0700 1 -- 172.21.15.16:0/2881472887 shutdown_connections 2023-09-30T22:27:17.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.267+0000 7f320abc0700 1 --2- 172.21.15.16:0/2881472887 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f320412e0d0 0x7f320413b030 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:17.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.267+0000 7f320abc0700 1 --2- 172.21.15.16:0/2881472887 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3204098740 0x7f320413a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:17.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.267+0000 7f320abc0700 1 --2- 172.21.15.16:0/2881472887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3204099230 0x7f320413ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:17.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.267+0000 7f320abc0700 1 -- 172.21.15.16:0/2881472887 >> 172.21.15.16:0/2881472887 conn(0x7f320409a1c0 msgr2=0x7f3204065d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:17.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.268+0000 7f320abc0700 1 -- 172.21.15.16:0/2881472887 shutdown_connections 2023-09-30T22:27:17.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.268+0000 7f320abc0700 1 -- 172.21.15.16:0/2881472887 wait complete. 2023-09-30T22:27:17.283 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 1 2023-09-30T22:27:17.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.419+0000 7f46efcec700 1 Processor -- start 2023-09-30T22:27:17.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.419+0000 7f46efcec700 1 -- start start 2023-09-30T22:27:17.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.420+0000 7f46efcec700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f46e8098740 0x7f46e8098b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:17.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.420+0000 7f46efcec700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f46e8069430 0x7f46e8130c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:17.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.420+0000 7f46efcec700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46e8095270 con 0x7f46e8099010 2023-09-30T22:27:17.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.420+0000 7f46efcec700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46e8094d20 con 0x7f46e8098740 2023-09-30T22:27:17.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.420+0000 7f46efcec700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46e8096520 con 0x7f46e8069430 2023-09-30T22:27:17.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.420+0000 7f46eda88700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f46e8098740 0x7f46e8098b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:17.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.420+0000 7f46ee289700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f46e8069430 0x7f46e8130c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:17.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.420+0000 7f46ed287700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f46e8099010 0x7f46e8068d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48178/0 (socket says 172.21.15.16:48178) 2023-09-30T22:27:17.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.420+0000 7f46eda88700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f46e8098740 0x7f46e8098b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45886/0 (socket says 172.21.15.16:45886) 2023-09-30T22:27:17.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.420+0000 7f46ed287700 1 -- 172.21.15.16:0/548388041 learned_addr learned my addr 172.21.15.16:0/548388041 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:27:17.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.420+0000 7f46eda88700 1 -- 172.21.15.16:0/548388041 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f46e8069430 msgr2=0x7f46e8130c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:17.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.420+0000 7f46eda88700 1 --2- 172.21.15.16:0/548388041 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f46e8069430 0x7f46e8130c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:17.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.420+0000 7f46eda88700 1 -- 172.21.15.16:0/548388041 >> v1:172.21.15.16:6789/0 conn(0x7f46e8099010 legacy=0x7f46e8068d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:17.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.420+0000 7f46eda88700 1 -- 172.21.15.16:0/548388041 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e8094870 con 0x7f46e8098740 2023-09-30T22:27:17.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.420+0000 7f46ee289700 1 --2- 172.21.15.16:0/548388041 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f46e8069430 0x7f46e8130c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:17.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.420+0000 7f46eca86700 1 -- 172.21.15.16:0/548388041 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3909198218 0 0) 0x7f46e8095270 con 0x7f46e8099010 2023-09-30T22:27:17.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.420+0000 7f46eda88700 1 --2- 172.21.15.16:0/548388041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46e8098740 0x7f46e8098b20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f46e4009760 tx=0x7f46e4009060).ready entity=mon.0 client_cookie=175ed428c7faa87a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:17.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.420+0000 7f46eca86700 1 -- 172.21.15.16:0/548388041 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e40090d0 con 0x7f46e8098740 2023-09-30T22:27:17.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.420+0000 7f46eca86700 1 -- 172.21.15.16:0/548388041 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46e400f510 con 0x7f46e8098740 2023-09-30T22:27:17.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.420+0000 7f46eca86700 1 -- 172.21.15.16:0/548388041 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e400f890 con 0x7f46e8098740 2023-09-30T22:27:17.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.421+0000 7f46efcec700 1 -- 172.21.15.16:0/548388041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46e8098740 msgr2=0x7f46e8098b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:17.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.421+0000 7f46efcec700 1 --2- 172.21.15.16:0/548388041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46e8098740 0x7f46e8098b20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f46e4009760 tx=0x7f46e4009060).stop 2023-09-30T22:27:17.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.421+0000 7f46efcec700 1 -- 172.21.15.16:0/548388041 shutdown_connections 2023-09-30T22:27:17.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.421+0000 7f46efcec700 1 --2- 172.21.15.16:0/548388041 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f46e8069430 0x7f46e8130c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:17.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.421+0000 7f46efcec700 1 --2- 172.21.15.16:0/548388041 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46e8098740 0x7f46e8098b20 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:17.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.421+0000 7f46efcec700 1 -- 172.21.15.16:0/548388041 >> 172.21.15.16:0/548388041 conn(0x7f46e809a1c0 msgr2=0x7f46e809a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:17.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.421+0000 7f46efcec700 1 -- 172.21.15.16:0/548388041 shutdown_connections 2023-09-30T22:27:17.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.421+0000 7f46efcec700 1 -- 172.21.15.16:0/548388041 wait complete. 2023-09-30T22:27:17.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.422+0000 7f46efcec700 1 Processor -- start 2023-09-30T22:27:17.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.422+0000 7f46efcec700 1 -- start start 2023-09-30T22:27:17.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.423+0000 7f46efcec700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f46e8069430 0x7f46e8117810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:17.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.423+0000 7f46efcec700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46e8098740 0x7f46e8117d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:17.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.423+0000 7f46efcec700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f46e8099010 0x7f46e81181f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:17.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.423+0000 7f46efcec700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e81175e0 con 0x7f46e8098740 2023-09-30T22:27:17.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.423+0000 7f46efcec700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46e8094f70 con 0x7f46e8069430 2023-09-30T22:27:17.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.423+0000 7f46efcec700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f46e811a9d0 con 0x7f46e8099010 2023-09-30T22:27:17.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.423+0000 7f46ed287700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46e8098740 0x7f46e8117d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:17.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.423+0000 7f46ee289700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f46e8099010 0x7f46e81181f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:17.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.423+0000 7f46ed287700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46e8098740 0x7f46e8117d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45890/0 (socket says 172.21.15.16:45890) 2023-09-30T22:27:17.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.423+0000 7f46eda88700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f46e8069430 0x7f46e8117810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:17.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.423+0000 7f46ed287700 1 -- 172.21.15.16:0/1304081345 learned_addr learned my addr 172.21.15.16:0/1304081345 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:17.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.423+0000 7f46eda88700 1 -- 172.21.15.16:0/1304081345 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f46e8099010 msgr2=0x7f46e81181f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:17.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.423+0000 7f46eda88700 1 --2- 172.21.15.16:0/1304081345 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f46e8099010 0x7f46e81181f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:17.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.423+0000 7f46eda88700 1 -- 172.21.15.16:0/1304081345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46e8098740 msgr2=0x7f46e8117d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:17.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.423+0000 7f46eda88700 1 --2- 172.21.15.16:0/1304081345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46e8098740 0x7f46e8117d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:17.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.423+0000 7f46eda88700 1 -- 172.21.15.16:0/1304081345 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e0009a20 con 0x7f46e8069430 2023-09-30T22:27:17.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.423+0000 7f46ee289700 1 --2- 172.21.15.16:0/1304081345 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f46e8099010 0x7f46e81181f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:17.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.423+0000 7f46ed287700 1 --2- 172.21.15.16:0/1304081345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46e8098740 0x7f46e8117d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:17.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.423+0000 7f46eda88700 1 --2- 172.21.15.16:0/1304081345 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f46e8069430 0x7f46e8117810 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f46e4014000 tx=0x7f46e4017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:17.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.423+0000 7f46ddffb700 1 -- 172.21.15.16:0/1304081345 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e400f890 con 0x7f46e8069430 2023-09-30T22:27:17.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.423+0000 7f46ddffb700 1 -- 172.21.15.16:0/1304081345 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46e4006930 con 0x7f46e8069430 2023-09-30T22:27:17.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.423+0000 7f46efcec700 1 -- 172.21.15.16:0/1304081345 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46e40092a0 con 0x7f46e8069430 2023-09-30T22:27:17.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.423+0000 7f46ddffb700 1 -- 172.21.15.16:0/1304081345 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46e400fea0 con 0x7f46e8069430 2023-09-30T22:27:17.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.424+0000 7f46efcec700 1 -- 172.21.15.16:0/1304081345 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46e811b390 con 0x7f46e8069430 2023-09-30T22:27:17.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.425+0000 7f46efcec700 1 -- 172.21.15.16:0/1304081345 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46e805a260 con 0x7f46e8069430 2023-09-30T22:27:17.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.425+0000 7f46ddffb700 1 -- 172.21.15.16:0/1304081345 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f46e401b620 con 0x7f46e8069430 2023-09-30T22:27:17.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.425+0000 7f46ddffb700 1 -- 172.21.15.16:0/1304081345 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f46e403cd90 con 0x7f46e8069430 2023-09-30T22:27:17.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.430+0000 7f46ddffb700 1 -- 172.21.15.16:0/1304081345 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f46e402e430 con 0x7f46e8069430 2023-09-30T22:27:17.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:17.650+0000 7f46efcec700 1 -- 172.21.15.16:0/1304081345 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"} v 0) v1 -- 0x7f46e811abf0 con 0x7f46e8069430 2023-09-30T22:27:18.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.268+0000 7f46ddffb700 1 -- 172.21.15.16:0/1304081345 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]=0 v8) v1 ==== 105+0+0 (secure 0 0 0) 0x7f46e40320b0 con 0x7f46e8069430 2023-09-30T22:27:18.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.271+0000 7f46efcec700 1 -- 172.21.15.16:0/1304081345 >> v1:172.21.15.16:6816/118285 conn(0x7f46cc02bb60 legacy=0x7f46cc02e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:18.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.271+0000 7f46efcec700 1 -- 172.21.15.16:0/1304081345 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f46e8069430 msgr2=0x7f46e8117810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:18.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.271+0000 7f46efcec700 1 --2- 172.21.15.16:0/1304081345 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f46e8069430 0x7f46e8117810 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f46e4014000 tx=0x7f46e4017000).stop 2023-09-30T22:27:18.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.271+0000 7f46efcec700 1 -- 172.21.15.16:0/1304081345 shutdown_connections 2023-09-30T22:27:18.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.271+0000 7f46efcec700 1 --2- 172.21.15.16:0/1304081345 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f46e8069430 0x7f46e8117810 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:18.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.271+0000 7f46efcec700 1 --2- 172.21.15.16:0/1304081345 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f46e8099010 0x7f46e81181f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:18.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.271+0000 7f46efcec700 1 --2- 172.21.15.16:0/1304081345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46e8098740 0x7f46e8117d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:18.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.271+0000 7f46efcec700 1 -- 172.21.15.16:0/1304081345 >> 172.21.15.16:0/1304081345 conn(0x7f46e809a1c0 msgr2=0x7f46e8065d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:18.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.271+0000 7f46efcec700 1 -- 172.21.15.16:0/1304081345 shutdown_connections 2023-09-30T22:27:18.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.271+0000 7f46efcec700 1 -- 172.21.15.16:0/1304081345 wait complete. 2023-09-30T22:27:18.287 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-30T22:27:18.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.420+0000 7f947cdce700 1 Processor -- start 2023-09-30T22:27:18.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.420+0000 7f947cdce700 1 -- start start 2023-09-30T22:27:18.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.421+0000 7f947cdce700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9478127390 0x7f947812fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:18.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.421+0000 7f947cdce700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9478127fa0 0x7f9478132170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:18.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.421+0000 7f947cdce700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9478091b60 con 0x7f9478059070 2023-09-30T22:27:18.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.421+0000 7f947cdce700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9478096940 con 0x7f9478127390 2023-09-30T22:27:18.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.421+0000 7f947cdce700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9478059c00 con 0x7f9478127fa0 2023-09-30T22:27:18.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.421+0000 7f9476d9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9478127fa0 0x7f9478132170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:18.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.421+0000 7f9475d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9478127390 0x7f947812fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:18.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.421+0000 7f947659c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9478059070 0x7f9478127890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48184/0 (socket says 172.21.15.16:48184) 2023-09-30T22:27:18.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.421+0000 7f947659c700 1 -- 172.21.15.16:0/481999592 learned_addr learned my addr 172.21.15.16:0/481999592 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:27:18.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.421+0000 7f9475d9b700 1 -- 172.21.15.16:0/481999592 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9478127fa0 msgr2=0x7f9478132170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:18.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.422+0000 7f9475d9b700 1 --2- 172.21.15.16:0/481999592 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9478127fa0 0x7f9478132170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:18.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.422+0000 7f9475d9b700 1 -- 172.21.15.16:0/481999592 >> v1:172.21.15.16:6789/0 conn(0x7f9478059070 legacy=0x7f9478127890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:18.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.422+0000 7f9475d9b700 1 -- 172.21.15.16:0/481999592 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94780929d0 con 0x7f9478127390 2023-09-30T22:27:18.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.422+0000 7f947559a700 1 -- 172.21.15.16:0/481999592 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3638888238 0 0) 0x7f9478091b60 con 0x7f9478059070 2023-09-30T22:27:18.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.422+0000 7f9476d9d700 1 --2- 172.21.15.16:0/481999592 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9478127fa0 0x7f9478132170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:18.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.422+0000 7f9475d9b700 1 --2- 172.21.15.16:0/481999592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9478127390 0x7f947812fc70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f946c009760 tx=0x7f946c009060).ready entity=mon.0 client_cookie=a60057d8290fa3e7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:18.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.422+0000 7f947559a700 1 -- 172.21.15.16:0/481999592 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946c0090d0 con 0x7f9478127390 2023-09-30T22:27:18.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.422+0000 7f947559a700 1 -- 172.21.15.16:0/481999592 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f946c00f510 con 0x7f9478127390 2023-09-30T22:27:18.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.422+0000 7f947559a700 1 -- 172.21.15.16:0/481999592 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946c00f890 con 0x7f9478127390 2023-09-30T22:27:18.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.422+0000 7f947cdce700 1 -- 172.21.15.16:0/481999592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9478127390 msgr2=0x7f947812fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:18.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.422+0000 7f947cdce700 1 --2- 172.21.15.16:0/481999592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9478127390 0x7f947812fc70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f946c009760 tx=0x7f946c009060).stop 2023-09-30T22:27:18.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.422+0000 7f947cdce700 1 -- 172.21.15.16:0/481999592 shutdown_connections 2023-09-30T22:27:18.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.422+0000 7f947cdce700 1 --2- 172.21.15.16:0/481999592 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9478127fa0 0x7f9478132170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:18.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.422+0000 7f947cdce700 1 --2- 172.21.15.16:0/481999592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9478127390 0x7f947812fc70 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:18.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.422+0000 7f947cdce700 1 -- 172.21.15.16:0/481999592 >> 172.21.15.16:0/481999592 conn(0x7f9478097470 msgr2=0x7f9478097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:18.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.423+0000 7f947cdce700 1 -- 172.21.15.16:0/481999592 shutdown_connections 2023-09-30T22:27:18.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.423+0000 7f947cdce700 1 -- 172.21.15.16:0/481999592 wait complete. 2023-09-30T22:27:18.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.423+0000 7f947cdce700 1 Processor -- start 2023-09-30T22:27:18.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.423+0000 7f947cdce700 1 -- start start 2023-09-30T22:27:18.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.424+0000 7f947cdce700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9478059070 0x7f947812e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:18.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.424+0000 7f947cdce700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9478127390 0x7f947812e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:18.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.424+0000 7f947cdce700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9478127fa0 0x7f947812eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:18.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.424+0000 7f947cdce700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9478131860 con 0x7f9478127fa0 2023-09-30T22:27:18.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.424+0000 7f947cdce700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9478125330 con 0x7f9478127390 2023-09-30T22:27:18.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.424+0000 7f947cdce700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9478131bb0 con 0x7f9478059070 2023-09-30T22:27:18.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.424+0000 7f947659c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9478059070 0x7f947812e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:18.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.424+0000 7f947659c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9478059070 0x7f947812e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37518/0 (socket says 172.21.15.16:37518) 2023-09-30T22:27:18.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.424+0000 7f947659c700 1 -- 172.21.15.16:0/3076399290 learned_addr learned my addr 172.21.15.16:0/3076399290 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:18.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.424+0000 7f9475d9b700 1 --2- 172.21.15.16:0/3076399290 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9478127390 0x7f947812e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:18.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.424+0000 7f9476d9d700 1 --2- 172.21.15.16:0/3076399290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9478127fa0 0x7f947812eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:18.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.424+0000 7f9476d9d700 1 -- 172.21.15.16:0/3076399290 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9478059070 msgr2=0x7f947812e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:18.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.425+0000 7f9476d9d700 1 --2- 172.21.15.16:0/3076399290 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9478059070 0x7f947812e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:18.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.425+0000 7f9476d9d700 1 -- 172.21.15.16:0/3076399290 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9478127390 msgr2=0x7f947812e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:18.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.425+0000 7f9476d9d700 1 --2- 172.21.15.16:0/3076399290 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9478127390 0x7f947812e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:18.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.425+0000 7f9476d9d700 1 -- 172.21.15.16:0/3076399290 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94700092a0 con 0x7f9478127fa0 2023-09-30T22:27:18.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.425+0000 7f9475d9b700 1 --2- 172.21.15.16:0/3076399290 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9478127390 0x7f947812e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:18.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.425+0000 7f947659c700 1 --2- 172.21.15.16:0/3076399290 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9478059070 0x7f947812e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:18.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.425+0000 7f9476d9d700 1 --2- 172.21.15.16:0/3076399290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9478127fa0 0x7f947812eb00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f947000e930 tx=0x7f9470009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:18.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.425+0000 7f94667fc700 1 -- 172.21.15.16:0/3076399290 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9470012600 con 0x7f9478127fa0 2023-09-30T22:27:18.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.425+0000 7f94667fc700 1 -- 172.21.15.16:0/3076399290 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9470012760 con 0x7f9478127fa0 2023-09-30T22:27:18.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.425+0000 7f947cdce700 1 -- 172.21.15.16:0/3076399290 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f946c0092a0 con 0x7f9478127fa0 2023-09-30T22:27:18.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.425+0000 7f94667fc700 1 -- 172.21.15.16:0/3076399290 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9470012ae0 con 0x7f9478127fa0 2023-09-30T22:27:18.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.425+0000 7f947cdce700 1 -- 172.21.15.16:0/3076399290 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f947812fbc0 con 0x7f9478127fa0 2023-09-30T22:27:18.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.426+0000 7f94667fc700 1 -- 172.21.15.16:0/3076399290 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f947001b400 con 0x7f9478127fa0 2023-09-30T22:27:18.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.426+0000 7f94667fc700 1 -- 172.21.15.16:0/3076399290 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f947003b430 con 0x7f9478127fa0 2023-09-30T22:27:18.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.426+0000 7f947cdce700 1 -- 172.21.15.16:0/3076399290 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f947805a260 con 0x7f9478127fa0 2023-09-30T22:27:18.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.431+0000 7f94667fc700 1 -- 172.21.15.16:0/3076399290 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156992 (secure 0 0 0) 0x7f9470016030 con 0x7f9478127fa0 2023-09-30T22:27:18.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.647+0000 7f947cdce700 1 -- 172.21.15.16:0/3076399290 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9478131de0 con 0x7f9478127fa0 2023-09-30T22:27:18.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.647+0000 7f94667fc700 1 -- 172.21.15.16:0/3076399290 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+2156 (secure 0 0 0) 0x7f94700228f0 con 0x7f9478127fa0 2023-09-30T22:27:18.648 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:27:18.648 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-30T22:26:51.674561+0000","modified":"2023-09-30T22:27:18.263912+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4218},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4218":{"gid":4218,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6817/1424366610","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6817","nonce":1424366610}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-09-30T22:27:18.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.649+0000 7f947cdce700 1 -- 172.21.15.16:0/3076399290 >> v1:172.21.15.16:6816/118285 conn(0x7f945802bb60 legacy=0x7f945802e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:18.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.649+0000 7f947cdce700 1 -- 172.21.15.16:0/3076399290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9478127fa0 msgr2=0x7f947812eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:18.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.649+0000 7f947cdce700 1 --2- 172.21.15.16:0/3076399290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9478127fa0 0x7f947812eb00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f947000e930 tx=0x7f9470009060).stop 2023-09-30T22:27:18.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.649+0000 7f947cdce700 1 -- 172.21.15.16:0/3076399290 shutdown_connections 2023-09-30T22:27:18.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.649+0000 7f947cdce700 1 --2- 172.21.15.16:0/3076399290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9478127fa0 0x7f947812eb00 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:18.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.649+0000 7f947cdce700 1 --2- 172.21.15.16:0/3076399290 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9478059070 0x7f947812e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:18.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.649+0000 7f947cdce700 1 --2- 172.21.15.16:0/3076399290 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9478127390 0x7f947812e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:18.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.649+0000 7f947cdce700 1 -- 172.21.15.16:0/3076399290 >> 172.21.15.16:0/3076399290 conn(0x7f9478097470 msgr2=0x7f9478126320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:18.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.649+0000 7f947cdce700 1 -- 172.21.15.16:0/3076399290 shutdown_connections 2023-09-30T22:27:18.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.649+0000 7f947cdce700 1 -- 172.21.15.16:0/3076399290 wait complete. 2023-09-30T22:27:18.651 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2023-09-30T22:27:18.763 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-09-30T22:26:51.674561+0000', 'modified': '2023-09-30T22:27:18.263912+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4218}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4218': {'gid': 4218, 'name': 'a', 'rank': 0, 'incarnation': 3, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.16:6817/1424366610', 'addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.16:6817', 'nonce': 1424366610}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-09-30T22:27:18.764 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-09-30T22:27:18.764 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-09-30T22:27:18.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.900+0000 7f732b7b2700 1 Processor -- start 2023-09-30T22:27:18.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.901+0000 7f732b7b2700 1 -- start start 2023-09-30T22:27:18.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.901+0000 7f732b7b2700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7324059070 0x7f7324127890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:18.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.901+0000 7f732b7b2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7324127d80 0x7f7324095a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:18.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.901+0000 7f732b7b2700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7324091b60 con 0x7f7324127390 2023-09-30T22:27:18.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.901+0000 7f732b7b2700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7324096940 con 0x7f7324059070 2023-09-30T22:27:18.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.901+0000 7f732b7b2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7324059c00 con 0x7f7324127d80 2023-09-30T22:27:18.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.901+0000 7f732954e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7324059070 0x7f7324127890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:18.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.901+0000 7f7328d4d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7324127390 0x7f7324095370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48194/0 (socket says 172.21.15.16:48194) 2023-09-30T22:27:18.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.901+0000 7f7328d4d700 1 -- 172.21.15.16:0/3436180492 learned_addr learned my addr 172.21.15.16:0/3436180492 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:27:18.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.901+0000 7f7329d4f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7324127d80 0x7f7324095a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:18.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.902+0000 7f7329d4f700 1 -- 172.21.15.16:0/3436180492 >> v2:172.21.15.16:3300/0 conn(0x7f7324059070 msgr2=0x7f7324127890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:18.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.902+0000 7f7329d4f700 1 --2- 172.21.15.16:0/3436180492 >> v2:172.21.15.16:3300/0 conn(0x7f7324059070 0x7f7324127890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:18.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.902+0000 7f7329d4f700 1 -- 172.21.15.16:0/3436180492 >> v1:172.21.15.16:6789/0 conn(0x7f7324127390 legacy=0x7f7324095370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:18.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.902+0000 7f731bfff700 1 -- 172.21.15.16:0/3436180492 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1308025936 0 0) 0x7f7324091b60 con 0x7f7324127390 2023-09-30T22:27:18.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.902+0000 7f7329d4f700 1 -- 172.21.15.16:0/3436180492 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73240929d0 con 0x7f7324127d80 2023-09-30T22:27:18.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.902+0000 7f732954e700 1 --2- 172.21.15.16:0/3436180492 >> v2:172.21.15.16:3300/0 conn(0x7f7324059070 0x7f7324127890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:18.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.902+0000 7f7329d4f700 1 --2- 172.21.15.16:0/3436180492 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7324127d80 0x7f7324095a80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f731c009e60 tx=0x7f731c009780).ready entity=mon.2 client_cookie=e0991627429be06 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:18.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.902+0000 7f731bfff700 1 -- 172.21.15.16:0/3436180492 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731c00e070 con 0x7f7324127d80 2023-09-30T22:27:18.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.902+0000 7f731bfff700 1 -- 172.21.15.16:0/3436180492 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f731c00bb60 con 0x7f7324127d80 2023-09-30T22:27:18.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.902+0000 7f731bfff700 1 -- 172.21.15.16:0/3436180492 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731c012410 con 0x7f7324127d80 2023-09-30T22:27:18.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.903+0000 7f732b7b2700 1 -- 172.21.15.16:0/3436180492 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7324127d80 msgr2=0x7f7324095a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:18.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.903+0000 7f732b7b2700 1 --2- 172.21.15.16:0/3436180492 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7324127d80 0x7f7324095a80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f731c009e60 tx=0x7f731c009780).stop 2023-09-30T22:27:18.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.903+0000 7f732b7b2700 1 -- 172.21.15.16:0/3436180492 shutdown_connections 2023-09-30T22:27:18.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.903+0000 7f732b7b2700 1 --2- 172.21.15.16:0/3436180492 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7324127d80 0x7f7324095a80 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:18.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.903+0000 7f732b7b2700 1 --2- 172.21.15.16:0/3436180492 >> v2:172.21.15.16:3300/0 conn(0x7f7324059070 0x7f7324127890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:18.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.903+0000 7f732b7b2700 1 -- 172.21.15.16:0/3436180492 >> 172.21.15.16:0/3436180492 conn(0x7f7324097470 msgr2=0x7f7324097880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:18.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.903+0000 7f732b7b2700 1 -- 172.21.15.16:0/3436180492 shutdown_connections 2023-09-30T22:27:18.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.903+0000 7f732b7b2700 1 -- 172.21.15.16:0/3436180492 wait complete. 2023-09-30T22:27:18.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.904+0000 7f732b7b2700 1 Processor -- start 2023-09-30T22:27:18.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.904+0000 7f732b7b2700 1 -- start start 2023-09-30T22:27:18.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.904+0000 7f732b7b2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7324059070 0x7f7324117450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:18.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.904+0000 7f732b7b2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7324127390 0x7f7324113bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:18.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.904+0000 7f732b7b2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7324127d80 0x7f73241140a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:18.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.904+0000 7f732b7b2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7324096f20 con 0x7f7324059070 2023-09-30T22:27:18.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.904+0000 7f732b7b2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7324116c70 con 0x7f7324127390 2023-09-30T22:27:18.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.904+0000 7f732b7b2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7324113930 con 0x7f7324127d80 2023-09-30T22:27:18.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.904+0000 7f7329d4f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7324127d80 0x7f73241140a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:18.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.904+0000 7f732954e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7324059070 0x7f7324117450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:18.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.904+0000 7f7329d4f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7324127d80 0x7f73241140a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37532/0 (socket says 172.21.15.16:37532) 2023-09-30T22:27:18.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.904+0000 7f7329d4f700 1 -- 172.21.15.16:0/3721815126 learned_addr learned my addr 172.21.15.16:0/3721815126 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:18.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.904+0000 7f7328d4d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7324127390 0x7f7324113bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:18.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.904+0000 7f732954e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7324059070 0x7f7324117450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45930/0 (socket says 172.21.15.16:45930) 2023-09-30T22:27:18.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.905+0000 7f7329d4f700 1 -- 172.21.15.16:0/3721815126 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7324127390 msgr2=0x7f7324113bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:18.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.905+0000 7f7329d4f700 1 --2- 172.21.15.16:0/3721815126 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7324127390 0x7f7324113bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:18.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.905+0000 7f7329d4f700 1 -- 172.21.15.16:0/3721815126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7324059070 msgr2=0x7f7324117450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:18.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.905+0000 7f7329d4f700 1 --2- 172.21.15.16:0/3721815126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7324059070 0x7f7324117450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:18.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.905+0000 7f7329d4f700 1 -- 172.21.15.16:0/3721815126 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73200092a0 con 0x7f7324127d80 2023-09-30T22:27:18.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.905+0000 7f732954e700 1 --2- 172.21.15.16:0/3721815126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7324059070 0x7f7324117450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:18.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.905+0000 7f7329d4f700 1 --2- 172.21.15.16:0/3721815126 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7324127d80 0x7f73241140a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f731c00b6c0 tx=0x7f7324097340).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:18.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.905+0000 7f73197fa700 1 -- 172.21.15.16:0/3721815126 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731c012410 con 0x7f7324127d80 2023-09-30T22:27:18.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.905+0000 7f73197fa700 1 -- 172.21.15.16:0/3721815126 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f731c01d630 con 0x7f7324127d80 2023-09-30T22:27:18.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.905+0000 7f732b7b2700 1 -- 172.21.15.16:0/3721815126 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f731c009a20 con 0x7f7324127d80 2023-09-30T22:27:18.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.905+0000 7f73197fa700 1 -- 172.21.15.16:0/3721815126 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731c0070e0 con 0x7f7324127d80 2023-09-30T22:27:18.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.905+0000 7f732b7b2700 1 -- 172.21.15.16:0/3721815126 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7324115200 con 0x7f7324127d80 2023-09-30T22:27:18.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.906+0000 7f73197fa700 1 -- 172.21.15.16:0/3721815126 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 4) v1 ==== 31548+0+0 (secure 0 0 0) 0x7f731c003040 con 0x7f7324127d80 2023-09-30T22:27:18.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.906+0000 7f73197fa700 1 -- 172.21.15.16:0/3721815126 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f731c03cd30 con 0x7f7324127d80 2023-09-30T22:27:18.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.909+0000 7f733d165b80 1 Processor -- start 2023-09-30T22:27:18.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.909+0000 7f733d165b80 1 -- start start 2023-09-30T22:27:18.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.909+0000 7f733d165b80 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x55b1e1a34170 0x55b1e1a34550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:18.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.910+0000 7f733d165b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1e1a34a40 0x55b1e1a5e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:18.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.910+0000 7f733d165b80 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x55b1e1a5e5b0 0x55b1e1a609a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:18.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.910+0000 7f733d165b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1e19fe610 con 0x55b1e1a34a40 2023-09-30T22:27:18.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.910+0000 7f733d165b80 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b1e1974db0 con 0x55b1e1a5e5b0 2023-09-30T22:27:18.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.910+0000 7f733d165b80 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x55b1e198a030 con 0x55b1e1a34170 2023-09-30T22:27:18.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.910+0000 7f732954e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x55b1e1a34170 0x55b1e1a34550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:18.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.910+0000 7f7329d4f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x55b1e1a5e5b0 0x55b1e1a609a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:18.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.910+0000 7f732954e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x55b1e1a34170 0x55b1e1a34550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37534/0 (socket says 172.21.15.16:37534) 2023-09-30T22:27:18.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.910+0000 7f7329d4f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x55b1e1a5e5b0 0x55b1e1a609a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:42248/0 (socket says 172.21.15.16:42248) 2023-09-30T22:27:18.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.910+0000 7f732954e700 1 -- 172.21.15.16:0/756427156 learned_addr learned my addr 172.21.15.16:0/756427156 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:18.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.910+0000 7f7328d4d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1e1a34a40 0x55b1e1a5e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:18.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.910+0000 7f7328d4d700 1 -- 172.21.15.16:0/756427156 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x55b1e1a34170 msgr2=0x55b1e1a34550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:18.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.910+0000 7f7328d4d700 1 --2- 172.21.15.16:0/756427156 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x55b1e1a34170 0x55b1e1a34550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:18.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.910+0000 7f7328d4d700 1 -- 172.21.15.16:0/756427156 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x55b1e1a5e5b0 msgr2=0x55b1e1a609a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:18.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.910+0000 7f7328d4d700 1 --2- 172.21.15.16:0/756427156 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x55b1e1a5e5b0 0x55b1e1a609a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:18.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.910+0000 7f7328d4d700 1 -- 172.21.15.16:0/756427156 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b1e1825710 con 0x55b1e1a34a40 2023-09-30T22:27:18.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.910+0000 7f7329d4f700 1 --2- 172.21.15.16:0/756427156 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x55b1e1a5e5b0 0x55b1e1a609a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:18.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.910+0000 7f7328d4d700 1 --2- 172.21.15.16:0/756427156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1e1a34a40 0x55b1e1a5e0c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f732405a0c0 tx=0x7f7314007b90).ready entity=mon.0 client_cookie=a739dee2f3cbd2b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:18.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.910+0000 7f730e7fc700 1 -- 172.21.15.16:0/756427156 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73140075d0 con 0x55b1e1a34a40 2023-09-30T22:27:18.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.910+0000 7f730e7fc700 1 -- 172.21.15.16:0/756427156 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7314007040 con 0x55b1e1a34a40 2023-09-30T22:27:18.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.910+0000 7f730e7fc700 1 -- 172.21.15.16:0/756427156 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73140016f0 con 0x55b1e1a34a40 2023-09-30T22:27:18.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.911+0000 7f733d165b80 1 -- 172.21.15.16:0/756427156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1e1a34a40 msgr2=0x55b1e1a5e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:18.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.911+0000 7f733d165b80 1 --2- 172.21.15.16:0/756427156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1e1a34a40 0x55b1e1a5e0c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f732405a0c0 tx=0x7f7314007b90).stop 2023-09-30T22:27:18.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.911+0000 7f733d165b80 1 -- 172.21.15.16:0/756427156 shutdown_connections 2023-09-30T22:27:18.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.911+0000 7f733d165b80 1 --2- 172.21.15.16:0/756427156 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x55b1e1a5e5b0 0x55b1e1a609a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:18.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.911+0000 7f733d165b80 1 --2- 172.21.15.16:0/756427156 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x55b1e1a34170 0x55b1e1a34550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:18.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.911+0000 7f733d165b80 1 --2- 172.21.15.16:0/756427156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1e1a34a40 0x55b1e1a5e0c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:18.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.911+0000 7f733d165b80 1 -- 172.21.15.16:0/756427156 >> 172.21.15.16:0/756427156 conn(0x55b1e18a2940 msgr2=0x55b1e1a54cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:18.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.911+0000 7f733d165b80 1 -- 172.21.15.16:0/756427156 shutdown_connections 2023-09-30T22:27:18.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.911+0000 7f733d165b80 1 -- 172.21.15.16:0/756427156 wait complete. 2023-09-30T22:27:18.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.911+0000 7f733d165b80 1 -- start start 2023-09-30T22:27:18.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.912+0000 7f733d165b80 1 Processor -- start 2023-09-30T22:27:18.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.912+0000 7f733d165b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1e1a34170 0x55b1e1a63cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:18.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.912+0000 7f733d165b80 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x55b1e1a34a40 0x55b1e1a641e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:18.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.912+0000 7f733d165b80 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x55b1e1a5e5b0 0x55b1e1a59ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:18.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.912+0000 7f733d165b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1e198a170 con 0x55b1e1a34170 2023-09-30T22:27:18.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.912+0000 7f733d165b80 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b1e198a3f0 con 0x55b1e1a34a40 2023-09-30T22:27:18.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.912+0000 7f733d165b80 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x55b1e1955920 con 0x55b1e1a5e5b0 2023-09-30T22:27:18.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.912+0000 7f733d165b80 10 client.0 ll_register_callbacks cb 0x55b1e192ace0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T22:27:18.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.912+0000 7f7328d4d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x55b1e1a34a40 0x55b1e1a641e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:18.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.912+0000 7f7328d4d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x55b1e1a34a40 0x55b1e1a641e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:42262/0 (socket says 172.21.15.16:42262) 2023-09-30T22:27:18.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.912+0000 7f7328d4d700 1 -- 172.21.15.16:0/2942910392 learned_addr learned my addr 172.21.15.16:0/2942910392 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:18.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.913+0000 7f7329d4f700 1 --2- 172.21.15.16:0/2942910392 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x55b1e1a5e5b0 0x55b1e1a59ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:18.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.913+0000 7f732954e700 1 --2- 172.21.15.16:0/2942910392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1e1a34170 0x55b1e1a63cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:18.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.913+0000 7f7328d4d700 1 -- 172.21.15.16:0/2942910392 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x55b1e1a5e5b0 msgr2=0x55b1e1a59ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:18.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.913+0000 7f7328d4d700 1 --2- 172.21.15.16:0/2942910392 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x55b1e1a5e5b0 0x55b1e1a59ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:18.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.913+0000 7f7328d4d700 1 -- 172.21.15.16:0/2942910392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1e1a34170 msgr2=0x55b1e1a63cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:18.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.913+0000 7f7328d4d700 1 --2- 172.21.15.16:0/2942910392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1e1a34170 0x55b1e1a63cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:18.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.913+0000 7f7328d4d700 1 -- 172.21.15.16:0/2942910392 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7314007230 con 0x55b1e1a34a40 2023-09-30T22:27:18.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.913+0000 7f7328d4d700 1 --2- 172.21.15.16:0/2942910392 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x55b1e1a34a40 0x55b1e1a641e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f7314000c00 tx=0x7f7314018000).ready entity=mon.1 client_cookie=9bb4f6e39ca8752b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:18.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.913+0000 7f73037fe700 10 client.0 ms_handle_connect on v2:172.21.15.16:3301/0 2023-09-30T22:27:18.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.913+0000 7f73037fe700 1 -- 172.21.15.16:0/2942910392 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f731401f400 con 0x55b1e1a34a40 2023-09-30T22:27:18.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.913+0000 7f73037fe700 1 -- 172.21.15.16:0/2942910392 <== mon.1 v2:172.21.15.16:3301/0 2 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f731401f560 con 0x55b1e1a34a40 2023-09-30T22:27:18.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.913+0000 7f733d165b80 1 -- 172.21.15.16:0/2942910392 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b1e1a2cf20 con 0x55b1e1a34a40 2023-09-30T22:27:18.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.913+0000 7f73037fe700 1 -- 172.21.15.16:0/2942910392 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f731401f930 con 0x55b1e1a34a40 2023-09-30T22:27:18.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.913+0000 7f733d165b80 10 client.4546 fetch_fsmap learned FSMap version 8 2023-09-30T22:27:18.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.913+0000 7f733d165b80 1 -- 172.21.15.16:0/2942910392 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55b1e195cb00 con 0x55b1e1a34a40 2023-09-30T22:27:18.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.913+0000 7f73037fe700 1 -- 172.21.15.16:0/2942910392 <== mon.1 v2:172.21.15.16:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b1e1a2cf20 con 0x55b1e1a34a40 2023-09-30T22:27:18.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.914+0000 7f73037fe700 1 -- 172.21.15.16:0/2942910392 <== mon.1 v2:172.21.15.16:3301/0 5 ==== fsmap(e 8) v1 ==== 1430+0+0 (secure 0 0 0) 0x7f7314006730 con 0x55b1e1a34a40 2023-09-30T22:27:18.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.914+0000 7f733d165b80 10 client.4546 fetch_fsmap finished waiting for FSMap version 8 2023-09-30T22:27:18.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.914+0000 7f733d165b80 10 client.4546 resolve_mds: resolved name 'a' to daemon mds.a 2023-09-30T22:27:18.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.914+0000 7f733d165b80 20 client.4546 populate_metadata read hostname 'smithi016' 2023-09-30T22:27:18.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.914+0000 7f733d165b80 4 client.4546 mds_command: new command op to 4218 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-30T22:27:18.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.914+0000 7f733d165b80 1 -- 172.21.15.16:0/2942910392 --> v1:172.21.15.16:6817/1424366610 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b1e1a02250 con 0x55b1e1a5a6f0 2023-09-30T22:27:18.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.914+0000 7f73037fe700 10 client.4546 ms_handle_connect on v1:172.21.15.16:6817/1424366610 2023-09-30T22:27:18.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.915+0000 7f73037fe700 1 -- 172.21.15.16:0/2942910392 <== mds.0 v1:172.21.15.16:6817/1424366610 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (unknown 0 0 3608088055) 0x55b1e1a02250 con 0x55b1e1a5a6f0 2023-09-30T22:27:18.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.915+0000 7f73037fe700 10 client.4546 handle_command_reply: tid=0 2023-09-30T22:27:18.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.915+0000 7f733d165b80 1 client.4546 shutdown 2023-09-30T22:27:18.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.915+0000 7f733d165b80 1 -- 172.21.15.16:0/2942910392 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x55b1e1a34a40 msgr2=0x55b1e1a641e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:18.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.915+0000 7f733d165b80 1 --2- 172.21.15.16:0/2942910392 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x55b1e1a34a40 0x55b1e1a641e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f7314000c00 tx=0x7f7314018000).stop 2023-09-30T22:27:18.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.915+0000 7f733d165b80 1 -- 172.21.15.16:0/2942910392 shutdown_connections 2023-09-30T22:27:18.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.915+0000 7f733d165b80 1 --2- 172.21.15.16:0/2942910392 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x55b1e1a5e5b0 0x55b1e1a59ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:18.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.915+0000 7f733d165b80 1 --2- 172.21.15.16:0/2942910392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1e1a34170 0x55b1e1a63cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:18.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.915+0000 7f73037fe700 0 client.4546 ms_handle_reset on v1:172.21.15.16:6817/1424366610 2023-09-30T22:27:18.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.915+0000 7f733d165b80 1 --2- 172.21.15.16:0/2942910392 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x55b1e1a34a40 0x55b1e1a641e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:18.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.915+0000 7f733d165b80 1 -- 172.21.15.16:0/2942910392 >> 172.21.15.16:0/2942910392 conn(0x55b1e18a2940 msgr2=0x55b1e1a54c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:18.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.915+0000 7f733d165b80 1 -- 172.21.15.16:0/2942910392 shutdown_connections 2023-09-30T22:27:18.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.915+0000 7f733d165b80 1 -- 172.21.15.16:0/2942910392 wait complete. 2023-09-30T22:27:18.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.915+0000 7f733d165b80 20 client.4546 trim_cache size 0 max 16384 2023-09-30T22:27:18.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.932+0000 7f733d165b80 1 Processor -- start 2023-09-30T22:27:18.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.932+0000 7f733d165b80 1 -- start start 2023-09-30T22:27:18.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.933+0000 7f733d165b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1e1a34170 0x55b1e1a554d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:18.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.933+0000 7f733d165b80 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x55b1e1a34a40 0x55b1e1a6fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:18.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.933+0000 7f733d165b80 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x55b1e1a5e5b0 0x55b1e1a50030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:18.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.933+0000 7f733d165b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1e1a5dfd0 con 0x55b1e1a34170 2023-09-30T22:27:18.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.933+0000 7f733d165b80 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b1e1a37380 con 0x55b1e1a5e5b0 2023-09-30T22:27:18.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.933+0000 7f733d165b80 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x55b1e1975c40 con 0x55b1e1a34a40 2023-09-30T22:27:18.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.933+0000 7f7328d4d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x55b1e1a34a40 0x55b1e1a6fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:18.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.933+0000 7f7329d4f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x55b1e1a5e5b0 0x55b1e1a50030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:18.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.933+0000 7f732954e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1e1a34170 0x55b1e1a554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:18.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.933+0000 7f7329d4f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x55b1e1a5e5b0 0x55b1e1a50030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:42264/0 (socket says 172.21.15.16:42264) 2023-09-30T22:27:18.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.933+0000 7f7328d4d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x55b1e1a34a40 0x55b1e1a6fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37552/0 (socket says 172.21.15.16:37552) 2023-09-30T22:27:18.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.933+0000 7f7329d4f700 1 -- 172.21.15.16:0/868956929 learned_addr learned my addr 172.21.15.16:0/868956929 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:18.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.934+0000 7f732954e700 1 -- 172.21.15.16:0/868956929 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x55b1e1a34a40 msgr2=0x55b1e1a6fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:18.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.934+0000 7f732954e700 1 --2- 172.21.15.16:0/868956929 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x55b1e1a34a40 0x55b1e1a6fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:18.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.934+0000 7f732954e700 1 -- 172.21.15.16:0/868956929 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x55b1e1a5e5b0 msgr2=0x55b1e1a50030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:18.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.934+0000 7f732954e700 1 --2- 172.21.15.16:0/868956929 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x55b1e1a5e5b0 0x55b1e1a50030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:18.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.934+0000 7f732954e700 1 -- 172.21.15.16:0/868956929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b1e1982f70 con 0x55b1e1a34170 2023-09-30T22:27:18.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.934+0000 7f7329d4f700 1 --2- 172.21.15.16:0/868956929 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x55b1e1a5e5b0 0x55b1e1a50030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:18.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.934+0000 7f732954e700 1 --2- 172.21.15.16:0/868956929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1e1a34170 0x55b1e1a554d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x55b1e1a708c0 tx=0x7f7320008040).ready entity=mon.0 client_cookie=46c9a95fa7f82350 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:18.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.934+0000 7f730effd700 1 -- 172.21.15.16:0/868956929 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f732001b070 con 0x55b1e1a34170 2023-09-30T22:27:18.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.934+0000 7f730effd700 1 -- 172.21.15.16:0/868956929 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73200116a0 con 0x55b1e1a34170 2023-09-30T22:27:18.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.934+0000 7f730effd700 1 -- 172.21.15.16:0/868956929 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7320011ee0 con 0x55b1e1a34170 2023-09-30T22:27:18.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.934+0000 7f733d165b80 1 -- 172.21.15.16:0/868956929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1e1a34170 msgr2=0x55b1e1a554d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:18.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.934+0000 7f733d165b80 1 --2- 172.21.15.16:0/868956929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1e1a34170 0x55b1e1a554d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x55b1e1a708c0 tx=0x7f7320008040).stop 2023-09-30T22:27:18.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.934+0000 7f733d165b80 1 -- 172.21.15.16:0/868956929 shutdown_connections 2023-09-30T22:27:18.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.934+0000 7f733d165b80 1 --2- 172.21.15.16:0/868956929 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x55b1e1a5e5b0 0x55b1e1a50030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:18.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.934+0000 7f733d165b80 1 --2- 172.21.15.16:0/868956929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1e1a34170 0x55b1e1a554d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:18.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.934+0000 7f733d165b80 1 --2- 172.21.15.16:0/868956929 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x55b1e1a34a40 0x55b1e1a6fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:18.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.934+0000 7f733d165b80 1 -- 172.21.15.16:0/868956929 >> 172.21.15.16:0/868956929 conn(0x55b1e18a2940 msgr2=0x55b1e1a54c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:18.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.935+0000 7f733d165b80 1 -- 172.21.15.16:0/868956929 shutdown_connections 2023-09-30T22:27:18.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.935+0000 7f733d165b80 1 -- 172.21.15.16:0/868956929 wait complete. 2023-09-30T22:27:18.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.935+0000 7f733d165b80 1 -- start start 2023-09-30T22:27:18.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.935+0000 7f733d165b80 1 Processor -- start 2023-09-30T22:27:18.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.936+0000 7f733d165b80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1e1a34170 0x55b1e1a608b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:18.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.936+0000 7f733d165b80 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x55b1e1a34a40 0x55b1e1a60da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:18.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.936+0000 7f733d165b80 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x55b1e1a5e5b0 0x55b1e1ab8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-09-30T22:27:18.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.936+0000 7f733d165b80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b1e1a9cff0 con 0x55b1e1a34170 2023-09-30T22:27:18.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.936+0000 7f733d165b80 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b1e1a5dd00 con 0x55b1e1a5e5b0 2023-09-30T22:27:18.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.936+0000 7f733d165b80 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x55b1e1a63a30 con 0x55b1e1a34a40 2023-09-30T22:27:18.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.936+0000 7f733d165b80 10 client.0 ll_register_callbacks cb 0x55b1e192ace0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T22:27:18.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.936+0000 7f7329d4f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x55b1e1a5e5b0 0x55b1e1ab8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:18.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.936+0000 7f732954e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1e1a34170 0x55b1e1a608b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:18.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.936+0000 7f7328d4d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x55b1e1a34a40 0x55b1e1a60da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-09-30T22:27:18.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.936+0000 7f7329d4f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x55b1e1a5e5b0 0x55b1e1ab8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:42282/0 (socket says 172.21.15.16:42282) 2023-09-30T22:27:18.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.936+0000 7f7329d4f700 1 -- 172.21.15.16:0/1191952833 learned_addr learned my addr 172.21.15.16:0/1191952833 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:27:18.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.936+0000 7f7328d4d700 1 -- 172.21.15.16:0/1191952833 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x55b1e1a5e5b0 msgr2=0x55b1e1ab8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:18.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.936+0000 7f7328d4d700 1 --2- 172.21.15.16:0/1191952833 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x55b1e1a5e5b0 0x55b1e1ab8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:18.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.936+0000 7f7328d4d700 1 -- 172.21.15.16:0/1191952833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1e1a34170 msgr2=0x55b1e1a608b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:27:18.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.936+0000 7f7328d4d700 1 --2- 172.21.15.16:0/1191952833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1e1a34170 0x55b1e1a608b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:18.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.936+0000 7f7328d4d700 1 -- 172.21.15.16:0/1191952833 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f731c02c070 con 0x55b1e1a34a40 2023-09-30T22:27:18.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.936+0000 7f7329d4f700 1 --2- 172.21.15.16:0/1191952833 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x55b1e1a5e5b0 0x55b1e1ab8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:27:18.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.937+0000 7f7328d4d700 1 --2- 172.21.15.16:0/1191952833 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x55b1e1a34a40 0x55b1e1a60da0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f73140011e0 tx=0x7f7314020000).ready entity=mon.2 client_cookie=86350800ee6f287f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:27:18.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.937+0000 7f73037fe700 10 client.0 ms_handle_connect on v2:172.21.15.16:3302/0 2023-09-30T22:27:18.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.937+0000 7f73037fe700 1 -- 172.21.15.16:0/1191952833 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7314000c30 con 0x55b1e1a34a40 2023-09-30T22:27:18.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.937+0000 7f73037fe700 1 -- 172.21.15.16:0/1191952833 <== mon.2 v2:172.21.15.16:3302/0 2 ==== osd_map(17..17 src has 1..17) v4 ==== 3690+0+0 (secure 0 0 0) 0x7f7314000d90 con 0x55b1e1a34a40 2023-09-30T22:27:18.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.937+0000 7f733d165b80 1 -- 172.21.15.16:0/1191952833 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7314007040 con 0x55b1e1a34a40 2023-09-30T22:27:18.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.937+0000 7f73037fe700 1 -- 172.21.15.16:0/1191952833 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7314028c60 con 0x55b1e1a34a40 2023-09-30T22:27:18.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.937+0000 7f733d165b80 10 client.4559 fetch_fsmap learned FSMap version 8 2023-09-30T22:27:18.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.937+0000 7f733d165b80 1 -- 172.21.15.16:0/1191952833 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x7f7320011830 con 0x55b1e1a34a40 2023-09-30T22:27:18.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.937+0000 7f73037fe700 1 -- 172.21.15.16:0/1191952833 <== mon.2 v2:172.21.15.16:3302/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7314007040 con 0x55b1e1a34a40 2023-09-30T22:27:18.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.937+0000 7f73037fe700 1 -- 172.21.15.16:0/1191952833 <== mon.2 v2:172.21.15.16:3302/0 5 ==== fsmap(e 8) v1 ==== 1430+0+0 (secure 0 0 0) 0x7f7314012070 con 0x55b1e1a34a40 2023-09-30T22:27:18.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.937+0000 7f733d165b80 10 client.4559 fetch_fsmap finished waiting for FSMap version 8 2023-09-30T22:27:18.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.937+0000 7f733d165b80 10 client.4559 resolve_mds: resolved name 'a' to daemon mds.a 2023-09-30T22:27:18.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.937+0000 7f733d165b80 20 client.4559 populate_metadata read hostname 'smithi016' 2023-09-30T22:27:18.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.937+0000 7f733d165b80 4 client.4559 mds_command: new command op to 4218 tid=0[{"prefix": "status"}] 2023-09-30T22:27:18.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.937+0000 7f733d165b80 1 -- 172.21.15.16:0/1191952833 --> v1:172.21.15.16:6817/1424366610 -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b1e17f2ef0 con 0x55b1e1ab8eb0 2023-09-30T22:27:18.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.938+0000 7f73037fe700 10 client.4559 ms_handle_connect on v1:172.21.15.16:6817/1424366610 2023-09-30T22:27:18.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.938+0000 7f73037fe700 1 -- 172.21.15.16:0/1191952833 <== mds.0 v1:172.21.15.16:6817/1424366610 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (unknown 0 0 2195945002) 0x55b1e17f2ef0 con 0x55b1e1ab8eb0 2023-09-30T22:27:18.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.938+0000 7f73037fe700 10 client.4559 handle_command_reply: tid=0 2023-09-30T22:27:18.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.938+0000 7f733d165b80 1 client.4559 shutdown 2023-09-30T22:27:18.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.938+0000 7f733d165b80 1 -- 172.21.15.16:0/1191952833 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x55b1e1a34a40 msgr2=0x55b1e1a60da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:18.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.938+0000 7f733d165b80 1 --2- 172.21.15.16:0/1191952833 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x55b1e1a34a40 0x55b1e1a60da0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f73140011e0 tx=0x7f7314020000).stop 2023-09-30T22:27:18.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.938+0000 7f733d165b80 1 -- 172.21.15.16:0/1191952833 shutdown_connections 2023-09-30T22:27:18.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.938+0000 7f733d165b80 1 --2- 172.21.15.16:0/1191952833 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x55b1e1a5e5b0 0x55b1e1ab8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:18.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.938+0000 7f733d165b80 1 --2- 172.21.15.16:0/1191952833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b1e1a34170 0x55b1e1a608b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:18.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.938+0000 7f733d165b80 1 --2- 172.21.15.16:0/1191952833 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x55b1e1a34a40 0x55b1e1a60da0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:18.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.938+0000 7f73037fe700 0 client.4559 ms_handle_reset on v1:172.21.15.16:6817/1424366610 2023-09-30T22:27:18.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.938+0000 7f733d165b80 1 -- 172.21.15.16:0/1191952833 >> 172.21.15.16:0/1191952833 conn(0x55b1e18a2940 msgr2=0x55b1e1a4ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:18.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.938+0000 7f733d165b80 1 -- 172.21.15.16:0/1191952833 shutdown_connections 2023-09-30T22:27:18.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.938+0000 7f733d165b80 1 -- 172.21.15.16:0/1191952833 wait complete. 2023-09-30T22:27:18.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.938+0000 7f733d165b80 20 client.4559 trim_cache size 0 max 16384 2023-09-30T22:27:18.945 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-09-30T22:27:18.945 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "7ede3e14-2bd0-4726-9437-6d13c90cd342", 2023-09-30T22:27:18.945 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2023-09-30T22:27:18.945 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 4218, 2023-09-30T22:27:18.945 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2023-09-30T22:27:18.946 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2023-09-30T22:27:18.946 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2023-09-30T22:27:18.946 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 27.263064517, 2023-09-30T22:27:18.946 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 8, 2023-09-30T22:27:18.946 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 17, 2023-09-30T22:27:18.946 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 17, 2023-09-30T22:27:18.946 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 32.587658038000001 2023-09-30T22:27:18.946 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-09-30T22:27:18.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.939+0000 7f732b7b2700 1 -- 172.21.15.16:0/3721815126 >> v1:172.21.15.16:6816/118285 conn(0x7f730802bb90 legacy=0x7f730802e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:18.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.939+0000 7f732b7b2700 1 -- 172.21.15.16:0/3721815126 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7324127d80 msgr2=0x7f73241140a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:27:18.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.939+0000 7f732b7b2700 1 --2- 172.21.15.16:0/3721815126 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7324127d80 0x7f73241140a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f731c00b6c0 tx=0x7f7324097340).stop 2023-09-30T22:27:18.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.939+0000 7f732b7b2700 1 -- 172.21.15.16:0/3721815126 shutdown_connections 2023-09-30T22:27:18.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.939+0000 7f732b7b2700 1 --2- 172.21.15.16:0/3721815126 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7324127d80 0x7f73241140a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:18.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.939+0000 7f732b7b2700 1 --2- 172.21.15.16:0/3721815126 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7324059070 0x7f7324117450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:18.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.939+0000 7f732b7b2700 1 --2- 172.21.15.16:0/3721815126 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7324127390 0x7f7324113bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-09-30T22:27:18.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.939+0000 7f732b7b2700 1 -- 172.21.15.16:0/3721815126 >> 172.21.15.16:0/3721815126 conn(0x7f7324097470 msgr2=0x7f732412a720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:27:18.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.939+0000 7f732b7b2700 1 -- 172.21.15.16:0/3721815126 shutdown_connections 2023-09-30T22:27:18.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:27:18.939+0000 7f732b7b2700 1 -- 172.21.15.16:0/3721815126 wait complete. 2023-09-30T22:27:18.952 INFO:teuthology.run_tasks:Running task print... 2023-09-30T22:27:18.961 INFO:teuthology.task.print:**** done mds pre-upgrade sequence 2023-09-30T22:27:18.961 INFO:teuthology.run_tasks:Running task install.upgrade... 2023-09-30T22:27:18.970 INFO:teuthology.task.install:project ceph config {'mon.a': {'branch': 'reef'}} overrides {'flavor': 'default', 'sha1': '3d6eb91756d99e822eeed7c6b783104a6ad86054'} 2023-09-30T22:27:18.970 INFO:teuthology.task.install:config contains sha1|tag|branch, removing those keys from override 2023-09-30T22:27:18.970 INFO:teuthology.task.install:remote ubuntu@smithi016.front.sepia.ceph.com config {'branch': 'reef', 'flavor': 'default'} 2023-09-30T22:27:18.970 INFO:teuthology.task.install:extra packages: [] 2023-09-30T22:27:18.978 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-09-30T22:27:18.978 INFO:teuthology.task.install:Upgrading ceph rpm packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd 2023-09-30T22:27:18.979 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q ceph-common --qf '%{VERSION}-%{RELEASE}' 2023-09-30T22:27:19.005 INFO:teuthology.orchestra.run.smithi016.stdout:16.2.14-44.geeb13ac9.el8 2023-09-30T22:27:19.006 INFO:teuthology.packaging:The installed version of ceph-common is 16.2.14-44.geeb13ac9.el8 2023-09-30T22:27:19.006 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=reef 2023-09-30T22:27:19.314 INFO:teuthology.task.install:Ceph rpm upgrade from 16.2.14-44.geeb13ac9.el8 to 18.2.0-501.gfc1e90a3 2023-09-30T22:27:19.315 INFO:teuthology.task.install.rpm:Host smithi016 is: centos 8 x86_64 2023-09-30T22:27:19.315 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=reef 2023-09-30T22:27:19.639 INFO:teuthology.task.install.rpm:Repo base URL: https://3.chacra.ceph.com/r/ceph/reef/fc1e90a395a6a5631142eb70fd9bf3d22740a9da/centos/8/flavors/default/ 2023-09-30T22:27:19.639 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f /etc/yum.repos.d/ceph.repo 2023-09-30T22:27:20.016 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/reef/fc1e90a395a6a5631142eb70fd9bf3d22740a9da/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/reef/fc1e90a395a6a5631142eb70fd9bf3d22740a9da/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/reef/fc1e90a395a6a5631142eb70fd9bf3d22740a9da/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-09-30T22:27:20.016 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-09-30T22:27:20.016 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-09-30T22:27:20.053 DEBUG:teuthology.orchestra.run.smithi016:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/reef/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-09-30T22:27:20.131 DEBUG:teuthology.orchestra.run.smithi016:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-09-30T22:27:20.207 DEBUG:teuthology.orchestra.run.smithi016:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-09-30T22:27:20.270 INFO:teuthology.orchestra.run.smithi016.stdout:check_obsoletes = 1 2023-09-30T22:27:20.285 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum clean all 2023-09-30T22:27:20.596 INFO:teuthology.orchestra.run.smithi016.stdout:71 files removed 2023-09-30T22:27:20.630 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y upgrade ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd 2023-09-30T22:27:22.090 INFO:teuthology.orchestra.run.smithi016.stdout:ceph packages for x86_64 82 kB/s | 80 kB 00:00 2023-09-30T22:27:22.924 INFO:teuthology.orchestra.run.smithi016.stdout:ceph noarch packages 12 kB/s | 9.4 kB 00:00 2023-09-30T22:27:23.750 INFO:teuthology.orchestra.run.smithi016.stdout:ceph source packages 1.9 kB/s | 1.5 kB 00:00 2023-09-30T22:27:26.133 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 8 - AppStream 14 MB/s | 33 MB 00:02 2023-09-30T22:27:35.551 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 8 - BaseOS 18 MB/s | 48 MB 00:02 2023-09-30T22:27:43.368 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 8 - Extras 107 kB/s | 18 kB 00:00 2023-09-30T22:27:43.968 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 8 - Extras common packages 17 kB/s | 6.9 kB 00:00 2023-09-30T22:27:44.620 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 8 - PowerTools 13 MB/s | 6.5 MB 00:00 2023-09-30T22:27:46.339 INFO:teuthology.orchestra.run.smithi016.stdout:Copr repo for python3-asyncssh owned by ceph 11 kB/s | 3.5 kB 00:00 2023-09-30T22:27:47.567 INFO:teuthology.orchestra.run.smithi016.stdout:Extra Packages for Enterprise Linux 15 MB/s | 16 MB 00:01 2023-09-30T22:27:51.840 INFO:teuthology.orchestra.run.smithi016.stdout:lab-extras 535 kB/s | 24 kB 00:00 2023-09-30T22:27:56.522 INFO:teuthology.orchestra.run.smithi016.stderr:Package cephadm available, but not installed. 2023-09-30T22:27:56.523 INFO:teuthology.orchestra.run.smithi016.stdout:No match for argument: cephadm 2023-09-30T22:27:56.523 INFO:teuthology.orchestra.run.smithi016.stderr:Package ceph-immutable-object-cache available, but not installed. 2023-09-30T22:27:56.523 INFO:teuthology.orchestra.run.smithi016.stdout:No match for argument: ceph-immutable-object-cache 2023-09-30T22:27:56.538 INFO:teuthology.orchestra.run.smithi016.stderr:Package ceph-mgr-cephadm available, but not installed. 2023-09-30T22:27:56.539 INFO:teuthology.orchestra.run.smithi016.stdout:No match for argument: ceph-mgr-cephadm 2023-09-30T22:27:56.543 INFO:teuthology.orchestra.run.smithi016.stderr:Package ceph-volume available, but not installed. 2023-09-30T22:27:56.543 INFO:teuthology.orchestra.run.smithi016.stdout:No match for argument: ceph-volume 2023-09-30T22:27:56.854 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-09-30T22:27:56.860 INFO:teuthology.orchestra.run.smithi016.stdout:====================================================================================== 2023-09-30T22:27:56.860 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2023-09-30T22:27:56.860 INFO:teuthology.orchestra.run.smithi016.stdout:====================================================================================== 2023-09-30T22:27:56.860 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrading: 2023-09-30T22:27:56.860 INFO:teuthology.orchestra.run.smithi016.stdout: ceph x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 6.4 k 2023-09-30T22:27:56.860 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 5.4 M 2023-09-30T22:27:56.861 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 22 M 2023-09-30T22:27:56.861 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 847 k 2023-09-30T22:27:56.861 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards noarch 2:18.2.0-501.gfc1e90a3.el8 ceph-noarch 23 k 2023-09-30T22:27:56.861 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 2.3 M 2023-09-30T22:27:56.861 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 1.5 M 2023-09-30T22:27:56.861 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard noarch 2:18.2.0-501.gfc1e90a3.el8 ceph-noarch 1.7 M 2023-09-30T22:27:56.861 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.0-501.gfc1e90a3.el8 ceph-noarch 7.4 M 2023-09-30T22:27:56.861 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core noarch 2:18.2.0-501.gfc1e90a3.el8 ceph-noarch 241 k 2023-09-30T22:27:56.861 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook noarch 2:18.2.0-501.gfc1e90a3.el8 ceph-noarch 47 k 2023-09-30T22:27:56.861 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 4.6 M 2023-09-30T22:27:56.861 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 18 M 2023-09-30T22:27:56.861 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts noarch 2:18.2.0-501.gfc1e90a3.el8 ceph-noarch 15 k 2023-09-30T22:27:56.861 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 13 M 2023-09-30T22:27:56.862 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 24 k 2023-09-30T22:27:56.862 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 47 M 2023-09-30T22:27:56.862 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 30 k 2023-09-30T22:27:56.862 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2 x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 769 k 2023-09-30T22:27:56.862 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 137 k 2023-09-30T22:27:56.862 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 139 k 2023-09-30T22:27:56.862 INFO:teuthology.orchestra.run.smithi016.stdout: librados2 x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 3.7 M 2023-09-30T22:27:56.862 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1 x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 474 k 2023-09-30T22:27:56.862 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1 x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 3.6 M 2023-09-30T22:27:56.862 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2 x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 6.5 M 2023-09-30T22:27:56.862 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 45 k 2023-09-30T22:27:56.862 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 123 k 2023-09-30T22:27:56.863 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 190 k 2023-09-30T22:27:56.863 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 371 k 2023-09-30T22:27:56.863 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 364 k 2023-09-30T22:27:56.863 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 107 k 2023-09-30T22:27:56.863 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 94 k 2023-09-30T22:27:56.863 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 3.4 M 2023-09-30T22:27:56.863 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd x86_64 2:18.2.0-501.gfc1e90a3.el8 ceph 179 k 2023-09-30T22:27:56.863 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2023-09-30T22:27:56.863 INFO:teuthology.orchestra.run.smithi016.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-09-30T22:27:56.863 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:27:56.863 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2023-09-30T22:27:56.863 INFO:teuthology.orchestra.run.smithi016.stdout:====================================================================================== 2023-09-30T22:27:56.863 INFO:teuthology.orchestra.run.smithi016.stdout:Install 1 Package 2023-09-30T22:27:56.864 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrade 34 Packages 2023-09-30T22:27:56.864 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:27:56.864 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 145 M 2023-09-30T22:27:56.864 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2023-09-30T22:27:57.125 INFO:teuthology.orchestra.run.smithi016.stdout:(1/35): thrift-0.13.0-2.el8.x86_64.rpm 9.0 MB/s | 1.7 MB 00:00 2023-09-30T22:27:57.150 INFO:teuthology.orchestra.run.smithi016.stdout:(2/35): ceph-18.2.0-501.gfc1e90a3.el8.x86_64.rp 29 kB/s | 6.4 kB 00:00 2023-09-30T22:27:57.734 INFO:teuthology.orchestra.run.smithi016.stdout:(3/35): ceph-fuse-18.2.0-501.gfc1e90a3.el8.x86_ 1.4 MB/s | 847 kB 00:00 2023-09-30T22:27:58.384 INFO:teuthology.orchestra.run.smithi016.stdout:(4/35): ceph-base-18.2.0-501.gfc1e90a3.el8.x86_ 3.7 MB/s | 5.4 MB 00:01 2023-09-30T22:27:58.468 INFO:teuthology.orchestra.run.smithi016.stdout:(5/35): ceph-mds-18.2.0-501.gfc1e90a3.el8.x86_6 3.1 MB/s | 2.3 MB 00:00 2023-09-30T22:27:58.776 INFO:teuthology.orchestra.run.smithi016.stdout:(6/35): ceph-mgr-18.2.0-501.gfc1e90a3.el8.x86_6 3.9 MB/s | 1.5 MB 00:00 2023-09-30T22:27:59.368 INFO:teuthology.orchestra.run.smithi016.stdout:(7/35): ceph-mon-18.2.0-501.gfc1e90a3.el8.x86_6 5.1 MB/s | 4.6 MB 00:00 2023-09-30T22:28:00.785 INFO:teuthology.orchestra.run.smithi016.stdout:(8/35): ceph-radosgw-18.2.0-501.gfc1e90a3.el8.x 9.2 MB/s | 13 MB 00:01 2023-09-30T22:28:00.876 INFO:teuthology.orchestra.run.smithi016.stdout:(9/35): ceph-selinux-18.2.0-501.gfc1e90a3.el8.x 269 kB/s | 24 kB 00:00 2023-09-30T22:28:03.281 INFO:teuthology.orchestra.run.smithi016.stdout:(10/35): ceph-osd-18.2.0-501.gfc1e90a3.el8.x86_ 3.9 MB/s | 18 MB 00:04 2023-09-30T22:28:03.377 INFO:teuthology.orchestra.run.smithi016.stdout:(11/35): libcephfs-devel-18.2.0-501.gfc1e90a3.e 319 kB/s | 30 kB 00:00 2023-09-30T22:28:03.686 INFO:teuthology.orchestra.run.smithi016.stdout:(12/35): libcephfs2-18.2.0-501.gfc1e90a3.el8.x8 2.4 MB/s | 769 kB 00:00 2023-09-30T22:28:03.770 INFO:teuthology.orchestra.run.smithi016.stdout:(13/35): libcephsqlite-18.2.0-501.gfc1e90a3.el8 1.6 MB/s | 137 kB 00:00 2023-09-30T22:28:03.862 INFO:teuthology.orchestra.run.smithi016.stdout:(14/35): librados-devel-18.2.0-501.gfc1e90a3.el 1.5 MB/s | 139 kB 00:00 2023-09-30T22:28:04.229 INFO:teuthology.orchestra.run.smithi016.stdout:(15/35): ceph-common-18.2.0-501.gfc1e90a3.el8.x 3.0 MB/s | 22 MB 00:07 2023-09-30T22:28:04.454 INFO:teuthology.orchestra.run.smithi016.stdout:(16/35): libradosstriper1-18.2.0-501.gfc1e90a3. 2.1 MB/s | 474 kB 00:00 2023-09-30T22:28:04.996 INFO:teuthology.orchestra.run.smithi016.stdout:(17/35): librados2-18.2.0-501.gfc1e90a3.el8.x86 3.2 MB/s | 3.7 MB 00:01 2023-09-30T22:28:05.480 INFO:teuthology.orchestra.run.smithi016.stdout:(18/35): librbd1-18.2.0-501.gfc1e90a3.el8.x86_6 3.5 MB/s | 3.6 MB 00:01 2023-09-30T22:28:05.555 INFO:teuthology.orchestra.run.smithi016.stdout:(19/35): python3-ceph-argparse-18.2.0-501.gfc1e 595 kB/s | 45 kB 00:00 2023-09-30T22:28:06.206 INFO:teuthology.orchestra.run.smithi016.stdout:(20/35): ceph-test-18.2.0-501.gfc1e90a3.el8.x86 8.8 MB/s | 47 MB 00:05 2023-09-30T22:28:06.231 INFO:teuthology.orchestra.run.smithi016.stdout:(21/35): python3-ceph-common-18.2.0-501.gfc1e90 183 kB/s | 123 kB 00:00 2023-09-30T22:28:06.323 INFO:teuthology.orchestra.run.smithi016.stdout:(22/35): python3-cephfs-18.2.0-501.gfc1e90a3.el 1.6 MB/s | 190 kB 00:00 2023-09-30T22:28:06.390 INFO:teuthology.orchestra.run.smithi016.stdout:(23/35): python3-rados-18.2.0-501.gfc1e90a3.el8 2.3 MB/s | 371 kB 00:00 2023-09-30T22:28:06.449 INFO:teuthology.orchestra.run.smithi016.stdout:(24/35): python3-rbd-18.2.0-501.gfc1e90a3.el8.x 2.8 MB/s | 364 kB 00:00 2023-09-30T22:28:06.474 INFO:teuthology.orchestra.run.smithi016.stdout:(25/35): python3-rgw-18.2.0-501.gfc1e90a3.el8.x 1.2 MB/s | 107 kB 00:00 2023-09-30T22:28:06.508 INFO:teuthology.orchestra.run.smithi016.stdout:(26/35): rbd-fuse-18.2.0-501.gfc1e90a3.el8.x86_ 1.6 MB/s | 94 kB 00:00 2023-09-30T22:28:06.625 INFO:teuthology.orchestra.run.smithi016.stdout:(27/35): librgw2-18.2.0-501.gfc1e90a3.el8.x86_6 4.0 MB/s | 6.5 MB 00:01 2023-09-30T22:28:06.650 INFO:teuthology.orchestra.run.smithi016.stdout:(28/35): rbd-nbd-18.2.0-501.gfc1e90a3.el8.x86_6 1.2 MB/s | 179 kB 00:00 2023-09-30T22:28:06.692 INFO:teuthology.orchestra.run.smithi016.stdout:(29/35): ceph-grafana-dashboards-18.2.0-501.gfc 346 kB/s | 23 kB 00:00 2023-09-30T22:28:06.892 INFO:teuthology.orchestra.run.smithi016.stdout:(30/35): ceph-mgr-dashboard-18.2.0-501.gfc1e90a 7.2 MB/s | 1.7 MB 00:00 2023-09-30T22:28:06.976 INFO:teuthology.orchestra.run.smithi016.stdout:(31/35): ceph-mgr-modules-core-18.2.0-501.gfc1e 2.8 MB/s | 241 kB 00:00 2023-09-30T22:28:07.051 INFO:teuthology.orchestra.run.smithi016.stdout:(32/35): ceph-mgr-rook-18.2.0-501.gfc1e90a3.el8 635 kB/s | 47 kB 00:00 2023-09-30T22:28:07.135 INFO:teuthology.orchestra.run.smithi016.stdout:(33/35): ceph-prometheus-alerts-18.2.0-501.gfc1 175 kB/s | 15 kB 00:00 2023-09-30T22:28:07.227 INFO:teuthology.orchestra.run.smithi016.stdout:(34/35): rbd-mirror-18.2.0-501.gfc1e90a3.el8.x8 4.6 MB/s | 3.4 MB 00:00 2023-09-30T22:28:07.977 INFO:teuthology.orchestra.run.smithi016.stdout:(35/35): ceph-mgr-diskprediction-local-18.2.0-5 5.8 MB/s | 7.4 MB 00:01 2023-09-30T22:28:07.981 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2023-09-30T22:28:07.981 INFO:teuthology.orchestra.run.smithi016.stdout:Total 13 MB/s | 145 MB 00:11 2023-09-30T22:28:07.982 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2023-09-30T22:28:08.061 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2023-09-30T22:28:08.061 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2023-09-30T22:28:08.788 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2023-09-30T22:28:08.792 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2023-09-30T22:28:11.352 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2023-09-30T22:28:11.557 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: thrift-0.13.0-2.el8.x86_64 1/1 2023-09-30T22:28:12.140 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/69 2023-09-30T22:28:12.361 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librados2-2:18.2.0-501.gfc1e90a3.el8.x86_64 2/69 2023-09-30T22:28:12.788 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-2:18.2.0-501.gfc1e90a3.el8.x86_64 2/69 2023-09-30T22:28:12.961 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librbd1-2:18.2.0-501.gfc1e90a3.el8.x86_64 3/69 2023-09-30T22:28:13.080 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-2:18.2.0-501.gfc1e90a3.el8.x86_64 3/69 2023-09-30T22:28:13.261 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : libcephfs2-2:18.2.0-501.gfc1e90a3.el8.x86_64 4/69 2023-09-30T22:28:14.157 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephfs2-2:18.2.0-501.gfc1e90a3.el8.x86_64 4/69 2023-09-30T22:28:14.369 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librgw2-2:18.2.0-501.gfc1e90a3.el8.x86_64 5/69 2023-09-30T22:28:14.513 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librgw2-2:18.2.0-501.gfc1e90a3.el8.x86_64 5/69 2023-09-30T22:28:14.866 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : python3-rados-2:18.2.0-501.gfc1e90a3.el8.x86_64 6/69 2023-09-30T22:28:15.069 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : libcephsqlite-2:18.2.0-501.gfc1e90a3.el8.x86_64 7/69 2023-09-30T22:28:15.185 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephsqlite-2:18.2.0-501.gfc1e90a3.el8.x86_64 7/69 2023-09-30T22:28:15.403 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : libradosstriper1-2:18.2.0-501.gfc1e90a3.el8.x86_64 8/69 2023-09-30T22:28:15.472 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libradosstriper1-2:18.2.0-501.gfc1e90a3.el8.x86_64 8/69 2023-09-30T22:28:15.666 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : python3-ceph-argparse-2:18.2.0-501.gfc1e90a3.el8.x 9/69 2023-09-30T22:28:15.942 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : python3-cephfs-2:18.2.0-501.gfc1e90a3.el8.x86_64 10/69 2023-09-30T22:28:16.174 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : python3-rbd-2:18.2.0-501.gfc1e90a3.el8.x86_64 11/69 2023-09-30T22:28:16.677 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : python3-rgw-2:18.2.0-501.gfc1e90a3.el8.x86_64 12/69 2023-09-30T22:28:16.930 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librados-devel-2:18.2.0-501.gfc1e90a3.el8.x86_64 13/69 2023-09-30T22:28:17.074 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : ceph-prometheus-alerts-2:18.2.0-501.gfc1e90a3.el8. 14/69 2023-09-30T22:28:17.258 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : ceph-grafana-dashboards-2:18.2.0-501.gfc1e90a3.el8 15/69 2023-09-30T22:28:17.444 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : python3-ceph-common-2:18.2.0-501.gfc1e90a3.el8.x86 16/69 2023-09-30T22:28:19.991 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:18.2.0-501.gfc1e90a3.el8.x86_64 17/69 2023-09-30T22:28:20.294 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : ceph-common-2:18.2.0-501.gfc1e90a3.el8.x86_64 17/69 2023-09-30T22:28:21.006 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:18.2.0-501.gfc1e90a3.el8.x86_64 17/69 2023-09-30T22:28:21.293 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : ceph-base-2:18.2.0-501.gfc1e90a3.el8.x86_64 18/69 2023-09-30T22:28:21.398 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-base-2:18.2.0-501.gfc1e90a3.el8.x86_64 18/69 2023-09-30T22:28:21.543 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : ceph-selinux-2:18.2.0-501.gfc1e90a3.el8.x86_64 19/69 2023-09-30T22:28:41.652 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-selinux-2:18.2.0-501.gfc1e90a3.el8.x86_64 19/69 2023-09-30T22:28:41.906 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : ceph-mgr-dashboard-2:18.2.0-501.gfc1e90a3.el8.noar 20/69 2023-09-30T22:28:43.224 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.0-501.gfc1e90a3.el8.noar 20/69 2023-09-30T22:28:43.490 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : ceph-mgr-diskprediction-local-2:18.2.0-501.gfc1e90 21/69 2023-09-30T22:28:43.603 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.0-501.gfc1e90 21/69 2023-09-30T22:28:44.028 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : ceph-mgr-modules-core-2:18.2.0-501.gfc1e90a3.el8.n 22/69 2023-09-30T22:28:44.315 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : ceph-mgr-2:18.2.0-501.gfc1e90a3.el8.x86_64 23/69 2023-09-30T22:28:44.344 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-2:18.2.0-501.gfc1e90a3.el8.x86_64 23/69 2023-09-30T22:28:44.540 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : ceph-mgr-rook-2:18.2.0-501.gfc1e90a3.el8.noarch 24/69 2023-09-30T22:28:44.767 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.0-501.gfc1e90a3.el8.noarch 24/69 2023-09-30T22:28:44.990 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : ceph-mds-2:18.2.0-501.gfc1e90a3.el8.x86_64 25/69 2023-09-30T22:28:45.628 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mds-2:18.2.0-501.gfc1e90a3.el8.x86_64 25/69 2023-09-30T22:28:45.999 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : ceph-mon-2:18.2.0-501.gfc1e90a3.el8.x86_64 26/69 2023-09-30T22:28:48.049 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mon-2:18.2.0-501.gfc1e90a3.el8.x86_64 26/69 2023-09-30T22:28:48.314 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : ceph-osd-2:18.2.0-501.gfc1e90a3.el8.x86_64 27/69 2023-09-30T22:28:48.328 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-osd-2:18.2.0-501.gfc1e90a3.el8.x86_64 27/69 2023-09-30T22:28:49.943 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : ceph-2:18.2.0-501.gfc1e90a3.el8.x86_64 28/69 2023-09-30T22:28:50.231 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : ceph-radosgw-2:18.2.0-501.gfc1e90a3.el8.x86_64 29/69 2023-09-30T22:28:50.679 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-radosgw-2:18.2.0-501.gfc1e90a3.el8.x86_64 29/69 2023-09-30T22:28:50.940 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : rbd-mirror-2:18.2.0-501.gfc1e90a3.el8.x86_64 30/69 2023-09-30T22:28:56.504 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: rbd-mirror-2:18.2.0-501.gfc1e90a3.el8.x86_64 30/69 2023-09-30T22:28:56.811 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : ceph-test-2:18.2.0-501.gfc1e90a3.el8.x86_64 31/69 2023-09-30T22:28:56.990 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : libcephfs-devel-2:18.2.0-501.gfc1e90a3.el8.x86_64 32/69 2023-09-30T22:28:57.207 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : rbd-fuse-2:18.2.0-501.gfc1e90a3.el8.x86_64 33/69 2023-09-30T22:28:57.521 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : rbd-nbd-2:18.2.0-501.gfc1e90a3.el8.x86_64 34/69 2023-09-30T22:28:57.757 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : ceph-fuse-2:18.2.0-501.gfc1e90a3.el8.x86_64 35/69 2023-09-30T22:28:57.868 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : ceph-2:16.2.14-44.geeb13ac9.el8.x86_64 36/69 2023-09-30T22:28:58.115 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : ceph-test-2:16.2.14-44.geeb13ac9.el8.x86_64 37/69 2023-09-30T22:28:58.125 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-radosgw-2:16.2.14-44.geeb13ac9.el8.x86_64 38/69 2023-09-30T22:28:58.125 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : ceph-radosgw-2:16.2.14-44.geeb13ac9.el8.x86_64 38/69 2023-09-30T22:28:58.448 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-radosgw-2:16.2.14-44.geeb13ac9.el8.x86_64 38/69 2023-09-30T22:28:58.598 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : libcephfs-devel-2:16.2.14-44.geeb13ac9.el8.x86_64 39/69 2023-09-30T22:28:58.599 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : ceph-mgr-rook-2:16.2.14-44.geeb13ac9.el8.noarch 40/69 2023-09-30T22:28:58.791 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-rook-2:16.2.14-44.geeb13ac9.el8.noarch 40/69 2023-09-30T22:28:58.798 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-2:16.2.14-44.geeb13ac9.el8.x86_64 41/69 2023-09-30T22:28:58.799 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : ceph-mgr-2:16.2.14-44.geeb13ac9.el8.x86_64 41/69 2023-09-30T22:28:59.011 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-2:16.2.14-44.geeb13ac9.el8.x86_64 41/69 2023-09-30T22:28:59.188 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : ceph-mgr-modules-core-2:16.2.14-44.geeb13ac9.el8.n 42/69 2023-09-30T22:28:59.390 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : rbd-nbd-2:16.2.14-44.geeb13ac9.el8.x86_64 43/69 2023-09-30T22:28:59.397 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: rbd-mirror-2:16.2.14-44.geeb13ac9.el8.x86_64 44/69 2023-09-30T22:28:59.397 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : rbd-mirror-2:16.2.14-44.geeb13ac9.el8.x86_64 44/69 2023-09-30T22:28:59.607 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: rbd-mirror-2:16.2.14-44.geeb13ac9.el8.x86_64 44/69 2023-09-30T22:28:59.756 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : rbd-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 45/69 2023-09-30T22:28:59.756 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : libcephsqlite-2:16.2.14-44.geeb13ac9.el8.x86_64 46/69 2023-09-30T22:28:59.999 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephsqlite-2:16.2.14-44.geeb13ac9.el8.x86_64 46/69 2023-09-30T22:29:00.181 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librados-devel-2:16.2.14-44.geeb13ac9.el8.x86_64 47/69 2023-09-30T22:29:00.189 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-osd-2:16.2.14-44.geeb13ac9.el8.x86_64 48/69 2023-09-30T22:29:00.190 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : ceph-osd-2:16.2.14-44.geeb13ac9.el8.x86_64 48/69 2023-09-30T22:29:00.600 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-osd-2:16.2.14-44.geeb13ac9.el8.x86_64 48/69 2023-09-30T22:29:00.600 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : ceph-mgr-dashboard-2:16.2.14-44.geeb13ac9.el8.noar 49/69 2023-09-30T22:29:00.798 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-dashboard-2:16.2.14-44.geeb13ac9.el8.noar 49/69 2023-09-30T22:29:00.806 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mds-2:16.2.14-44.geeb13ac9.el8.x86_64 50/69 2023-09-30T22:29:00.806 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : ceph-mds-2:16.2.14-44.geeb13ac9.el8.x86_64 50/69 2023-09-30T22:29:01.007 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mds-2:16.2.14-44.geeb13ac9.el8.x86_64 50/69 2023-09-30T22:29:01.015 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mon-2:16.2.14-44.geeb13ac9.el8.x86_64 51/69 2023-09-30T22:29:01.015 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : ceph-mon-2:16.2.14-44.geeb13ac9.el8.x86_64 51/69 2023-09-30T22:29:01.232 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mon-2:16.2.14-44.geeb13ac9.el8.x86_64 51/69 2023-09-30T22:29:01.232 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : ceph-selinux-2:16.2.14-44.geeb13ac9.el8.x86_64 52/69 2023-09-30T22:29:01.382 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-selinux-2:16.2.14-44.geeb13ac9.el8.x86_64 52/69 2023-09-30T22:29:01.413 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-base-2:16.2.14-44.geeb13ac9.el8.x86_64 53/69 2023-09-30T22:29:01.414 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : ceph-base-2:16.2.14-44.geeb13ac9.el8.x86_64 53/69 2023-09-30T22:29:01.768 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-base-2:16.2.14-44.geeb13ac9.el8.x86_64 53/69 2023-09-30T22:29:01.769 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 54/69 2023-09-30T22:29:01.997 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 54/69 2023-09-30T22:29:01.998 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : libradosstriper1-2:16.2.14-44.geeb13ac9.el8.x86_64 55/69 2023-09-30T22:29:02.232 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libradosstriper1-2:16.2.14-44.geeb13ac9.el8.x86_64 55/69 2023-09-30T22:29:02.439 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : python3-cephfs-2:16.2.14-44.geeb13ac9.el8.x86_64 56/69 2023-09-30T22:29:02.665 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : python3-rbd-2:16.2.14-44.geeb13ac9.el8.x86_64 57/69 2023-09-30T22:29:02.665 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librbd1-2:16.2.14-44.geeb13ac9.el8.x86_64 58/69 2023-09-30T22:29:02.940 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-2:16.2.14-44.geeb13ac9.el8.x86_64 58/69 2023-09-30T22:29:03.148 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : python3-rgw-2:16.2.14-44.geeb13ac9.el8.x86_64 59/69 2023-09-30T22:29:03.148 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librgw2-2:16.2.14-44.geeb13ac9.el8.x86_64 60/69 2023-09-30T22:29:03.398 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librgw2-2:16.2.14-44.geeb13ac9.el8.x86_64 60/69 2023-09-30T22:29:03.564 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : python3-rados-2:16.2.14-44.geeb13ac9.el8.x86_64 61/69 2023-09-30T22:29:03.564 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : libcephfs2-2:16.2.14-44.geeb13ac9.el8.x86_64 62/69 2023-09-30T22:29:03.808 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephfs2-2:16.2.14-44.geeb13ac9.el8.x86_64 62/69 2023-09-30T22:29:04.014 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : ceph-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 63/69 2023-09-30T22:29:04.116 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : python3-ceph-argparse-2:16.2.14-44.geeb13ac9.el8.x 64/69 2023-09-30T22:29:04.322 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : python3-ceph-common-2:16.2.14-44.geeb13ac9.el8.x86 65/69 2023-09-30T22:29:04.463 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : ceph-grafana-dashboards-2:16.2.14-44.geeb13ac9.el8 66/69 2023-09-30T22:29:04.566 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : ceph-prometheus-alerts-2:16.2.14-44.geeb13ac9.el8. 67/69 2023-09-30T22:29:04.567 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : ceph-mgr-diskprediction-local-2:16.2.14-44.geeb13a 68/69 2023-09-30T22:29:04.722 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:16.2.14-44.geeb13a 68/69 2023-09-30T22:29:04.722 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librados2-2:16.2.14-44.geeb13ac9.el8.x86_64 69/69 2023-09-30T22:29:05.584 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-2:16.2.14-44.geeb13ac9.el8.x86_64 69/69 2023-09-30T22:29:05.584 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 1/69 2023-09-30T22:29:05.584 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-2:18.2.0-501.gfc1e90a3.el8.x86_64 2/69 2023-09-30T22:29:05.584 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-2:16.2.14-44.geeb13ac9.el8.x86_64 3/69 2023-09-30T22:29:05.584 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-base-2:18.2.0-501.gfc1e90a3.el8.x86_64 4/69 2023-09-30T22:29:05.584 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-base-2:16.2.14-44.geeb13ac9.el8.x86_64 5/69 2023-09-30T22:29:05.584 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-common-2:18.2.0-501.gfc1e90a3.el8.x86_64 6/69 2023-09-30T22:29:05.585 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-common-2:16.2.14-44.geeb13ac9.el8.x86_64 7/69 2023-09-30T22:29:05.585 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-fuse-2:18.2.0-501.gfc1e90a3.el8.x86_64 8/69 2023-09-30T22:29:05.585 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 9/69 2023-09-30T22:29:05.585 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mds-2:18.2.0-501.gfc1e90a3.el8.x86_64 10/69 2023-09-30T22:29:05.585 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mds-2:16.2.14-44.geeb13ac9.el8.x86_64 11/69 2023-09-30T22:29:05.585 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-2:18.2.0-501.gfc1e90a3.el8.x86_64 12/69 2023-09-30T22:29:05.585 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-2:16.2.14-44.geeb13ac9.el8.x86_64 13/69 2023-09-30T22:29:05.585 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mon-2:18.2.0-501.gfc1e90a3.el8.x86_64 14/69 2023-09-30T22:29:05.585 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mon-2:16.2.14-44.geeb13ac9.el8.x86_64 15/69 2023-09-30T22:29:05.585 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-osd-2:18.2.0-501.gfc1e90a3.el8.x86_64 16/69 2023-09-30T22:29:05.585 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-osd-2:16.2.14-44.geeb13ac9.el8.x86_64 17/69 2023-09-30T22:29:05.585 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-radosgw-2:18.2.0-501.gfc1e90a3.el8.x86_64 18/69 2023-09-30T22:29:05.586 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-radosgw-2:16.2.14-44.geeb13ac9.el8.x86_64 19/69 2023-09-30T22:29:05.586 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-selinux-2:18.2.0-501.gfc1e90a3.el8.x86_64 20/69 2023-09-30T22:29:05.586 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-selinux-2:16.2.14-44.geeb13ac9.el8.x86_64 21/69 2023-09-30T22:29:05.586 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-test-2:18.2.0-501.gfc1e90a3.el8.x86_64 22/69 2023-09-30T22:29:05.586 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-test-2:16.2.14-44.geeb13ac9.el8.x86_64 23/69 2023-09-30T22:29:05.586 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs-devel-2:18.2.0-501.gfc1e90a3.el8.x86_64 24/69 2023-09-30T22:29:05.586 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs-devel-2:16.2.14-44.geeb13ac9.el8.x86_64 25/69 2023-09-30T22:29:05.586 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs2-2:18.2.0-501.gfc1e90a3.el8.x86_64 26/69 2023-09-30T22:29:05.586 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs2-2:16.2.14-44.geeb13ac9.el8.x86_64 27/69 2023-09-30T22:29:05.586 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephsqlite-2:18.2.0-501.gfc1e90a3.el8.x86_64 28/69 2023-09-30T22:29:05.586 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephsqlite-2:16.2.14-44.geeb13ac9.el8.x86_64 29/69 2023-09-30T22:29:05.586 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados-devel-2:18.2.0-501.gfc1e90a3.el8.x86_64 30/69 2023-09-30T22:29:05.587 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados-devel-2:16.2.14-44.geeb13ac9.el8.x86_64 31/69 2023-09-30T22:29:05.587 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-2:18.2.0-501.gfc1e90a3.el8.x86_64 32/69 2023-09-30T22:29:05.587 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-2:16.2.14-44.geeb13ac9.el8.x86_64 33/69 2023-09-30T22:29:05.587 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libradosstriper1-2:18.2.0-501.gfc1e90a3.el8.x86_64 34/69 2023-09-30T22:29:05.587 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libradosstriper1-2:16.2.14-44.geeb13ac9.el8.x86_64 35/69 2023-09-30T22:29:05.587 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-2:18.2.0-501.gfc1e90a3.el8.x86_64 36/69 2023-09-30T22:29:05.587 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-2:16.2.14-44.geeb13ac9.el8.x86_64 37/69 2023-09-30T22:29:05.587 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librgw2-2:18.2.0-501.gfc1e90a3.el8.x86_64 38/69 2023-09-30T22:29:05.587 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librgw2-2:16.2.14-44.geeb13ac9.el8.x86_64 39/69 2023-09-30T22:29:05.587 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-argparse-2:18.2.0-501.gfc1e90a3.el8.x 40/69 2023-09-30T22:29:05.587 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-argparse-2:16.2.14-44.geeb13ac9.el8.x 41/69 2023-09-30T22:29:05.587 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-common-2:18.2.0-501.gfc1e90a3.el8.x86 42/69 2023-09-30T22:29:05.588 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-common-2:16.2.14-44.geeb13ac9.el8.x86 43/69 2023-09-30T22:29:05.588 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cephfs-2:18.2.0-501.gfc1e90a3.el8.x86_64 44/69 2023-09-30T22:29:05.588 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cephfs-2:16.2.14-44.geeb13ac9.el8.x86_64 45/69 2023-09-30T22:29:05.588 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rados-2:18.2.0-501.gfc1e90a3.el8.x86_64 46/69 2023-09-30T22:29:05.588 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rados-2:16.2.14-44.geeb13ac9.el8.x86_64 47/69 2023-09-30T22:29:05.588 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rbd-2:18.2.0-501.gfc1e90a3.el8.x86_64 48/69 2023-09-30T22:29:05.588 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rbd-2:16.2.14-44.geeb13ac9.el8.x86_64 49/69 2023-09-30T22:29:05.588 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rgw-2:18.2.0-501.gfc1e90a3.el8.x86_64 50/69 2023-09-30T22:29:05.588 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rgw-2:16.2.14-44.geeb13ac9.el8.x86_64 51/69 2023-09-30T22:29:05.588 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-fuse-2:18.2.0-501.gfc1e90a3.el8.x86_64 52/69 2023-09-30T22:29:05.588 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-fuse-2:16.2.14-44.geeb13ac9.el8.x86_64 53/69 2023-09-30T22:29:05.588 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-mirror-2:18.2.0-501.gfc1e90a3.el8.x86_64 54/69 2023-09-30T22:29:05.589 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-mirror-2:16.2.14-44.geeb13ac9.el8.x86_64 55/69 2023-09-30T22:29:05.589 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-nbd-2:18.2.0-501.gfc1e90a3.el8.x86_64 56/69 2023-09-30T22:29:05.589 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-nbd-2:16.2.14-44.geeb13ac9.el8.x86_64 57/69 2023-09-30T22:29:05.589 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-grafana-dashboards-2:18.2.0-501.gfc1e90a3.el8 58/69 2023-09-30T22:29:05.589 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-grafana-dashboards-2:16.2.14-44.geeb13ac9.el8 59/69 2023-09-30T22:29:05.589 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-dashboard-2:18.2.0-501.gfc1e90a3.el8.noar 60/69 2023-09-30T22:29:05.589 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-dashboard-2:16.2.14-44.geeb13ac9.el8.noar 61/69 2023-09-30T22:29:05.589 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.0-501.gfc1e90 62/69 2023-09-30T22:29:05.589 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-diskprediction-local-2:16.2.14-44.geeb13a 63/69 2023-09-30T22:29:05.589 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-modules-core-2:18.2.0-501.gfc1e90a3.el8.n 64/69 2023-09-30T22:29:05.589 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-modules-core-2:16.2.14-44.geeb13ac9.el8.n 65/69 2023-09-30T22:29:05.589 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-rook-2:18.2.0-501.gfc1e90a3.el8.noarch 66/69 2023-09-30T22:29:05.590 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-rook-2:16.2.14-44.geeb13ac9.el8.noarch 67/69 2023-09-30T22:29:05.590 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-prometheus-alerts-2:18.2.0-501.gfc1e90a3.el8. 68/69 2023-09-30T22:29:07.773 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-prometheus-alerts-2:16.2.14-44.geeb13ac9.el8. 69/69 2023-09-30T22:29:07.773 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:29:07.773 INFO:teuthology.orchestra.run.smithi016.stdout:Upgraded: 2023-09-30T22:29:07.774 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:29:07.774 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:29:07.774 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:29:07.774 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:29:07.774 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards-2:18.2.0-501.gfc1e90a3.el8.noarch 2023-09-30T22:29:07.774 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:29:07.775 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:29:07.775 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard-2:18.2.0-501.gfc1e90a3.el8.noarch 2023-09-30T22:29:07.775 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local-2:18.2.0-501.gfc1e90a3.el8.noarch 2023-09-30T22:29:07.775 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core-2:18.2.0-501.gfc1e90a3.el8.noarch 2023-09-30T22:29:07.775 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook-2:18.2.0-501.gfc1e90a3.el8.noarch 2023-09-30T22:29:07.775 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:29:07.775 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:29:07.775 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts-2:18.2.0-501.gfc1e90a3.el8.noarch 2023-09-30T22:29:07.776 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:29:07.776 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:29:07.776 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:29:07.776 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:29:07.776 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:29:07.776 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:29:07.776 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:29:07.777 INFO:teuthology.orchestra.run.smithi016.stdout: librados2-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:29:07.777 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:29:07.777 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:29:07.777 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:29:07.777 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:29:07.777 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:29:07.777 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:29:07.777 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:29:07.778 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:29:07.778 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:29:07.778 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:29:07.778 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:29:07.778 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd-2:18.2.0-501.gfc1e90a3.el8.x86_64 2023-09-30T22:29:07.778 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2023-09-30T22:29:07.778 INFO:teuthology.orchestra.run.smithi016.stdout: thrift-0.13.0-2.el8.x86_64 2023-09-30T22:29:07.779 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:29:07.779 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-09-30T22:29:07.931 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=reef 2023-09-30T22:29:08.231 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-09-30T22:29:08.261 INFO:teuthology.orchestra.run.smithi016.stdout:18.2.0-501.gfc1e90a3.el8 2023-09-30T22:29:08.261 INFO:teuthology.packaging:The installed version of ceph is 18.2.0-501.gfc1e90a3.el8 2023-09-30T22:29:08.262 INFO:teuthology.task.install:The correct ceph version 18.2.0-501.gfc1e90a3 is installed. 2023-09-30T22:29:08.262 INFO:teuthology.run_tasks:Running task print... 2023-09-30T22:29:08.275 INFO:teuthology.task.print:**** done install.upgrade the host 2023-09-30T22:29:08.275 INFO:teuthology.run_tasks:Running task ceph.restart... 2023-09-30T22:29:08.284 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-09-30T22:29:08.285 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T22:29:08.315 INFO:tasks.ceph.mon.a:Stopped 2023-09-30T22:29:08.315 INFO:tasks.ceph.mon.a:Restarting daemon 2023-09-30T22:29:08.315 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-09-30T22:29:08.358 INFO:tasks.ceph.mon.a:Started 2023-09-30T22:29:08.359 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-09-30T22:29:08.359 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T22:29:08.373 INFO:tasks.ceph.mon.b:Stopped 2023-09-30T22:29:08.373 INFO:tasks.ceph.mon.b:Restarting daemon 2023-09-30T22:29:08.373 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-09-30T22:29:08.415 INFO:tasks.ceph.mon.b:Started 2023-09-30T22:29:08.416 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-09-30T22:29:08.416 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T22:29:08.431 INFO:tasks.ceph.mon.c:Stopped 2023-09-30T22:29:08.431 INFO:tasks.ceph.mon.c:Restarting daemon 2023-09-30T22:29:08.431 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-09-30T22:29:08.473 INFO:tasks.ceph.mon.c:Started 2023-09-30T22:29:08.473 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-09-30T22:29:08.473 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T22:29:08.502 INFO:tasks.ceph.mgr.x:Stopped 2023-09-30T22:29:08.503 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-09-30T22:29:08.503 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-09-30T22:29:08.545 INFO:tasks.ceph.mgr.x:Started 2023-09-30T22:29:08.545 INFO:teuthology.run_tasks:Running task ceph.healthy... 2023-09-30T22:29:08.554 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-09-30T22:29:08.555 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-09-30T22:29:08.555 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-09-30T22:29:08.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.695+0000 7f556c2ff700 1 Processor -- start 2023-09-30T22:29:08.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.695+0000 7f556c2ff700 1 -- start start 2023-09-30T22:29:08.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.696+0000 7f556c2ff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f55641337f0 0x7f5564138880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:08.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.696+0000 7f556c2ff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5564138dc0 0x7f556413b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:08.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.696+0000 7f556c2ff700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f556406b990 con 0x7f5564133420 2023-09-30T22:29:08.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.696+0000 7f556c2ff700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f556409b0e0 con 0x7f55641337f0 2023-09-30T22:29:08.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.696+0000 7f556c2ff700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5564133c20 con 0x7f5564138dc0 2023-09-30T22:29:08.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.696+0000 7f556989a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f55641337f0 0x7f5564138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:08.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.696+0000 7f556989a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f55641337f0 0x7f5564138880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46698/0 (socket says 172.21.15.16:46698) 2023-09-30T22:29:08.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.696+0000 7f556a09b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5564133420 0x7f5564133e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40228/0 (socket says 172.21.15.16:40228) 2023-09-30T22:29:08.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.696+0000 7f556989a700 1 -- 172.21.15.16:0/3118349757 learned_addr learned my addr 172.21.15.16:0/3118349757 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:08.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.696+0000 7f556a89c700 1 --2- 172.21.15.16:0/3118349757 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5564138dc0 0x7f556413b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:08.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.696+0000 7f556989a700 1 -- 172.21.15.16:0/3118349757 >> v2:172.21.15.16:3300/0 conn(0x7f55641337f0 msgr2=0x7f5564138880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-09-30T22:29:08.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.696+0000 7f556989a700 1 -- 172.21.15.16:0/3118349757 >> v2:172.21.15.16:3300/0 conn(0x7f55641337f0 msgr2=0x7f5564138880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:29:08.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.696+0000 7f556989a700 1 --2- 172.21.15.16:0/3118349757 >> v2:172.21.15.16:3300/0 conn(0x7f55641337f0 0x7f5564138880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-09-30T22:29:08.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.696+0000 7f556989a700 1 --2- 172.21.15.16:0/3118349757 >> v2:172.21.15.16:3300/0 conn(0x7f55641337f0 0x7f5564138880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-30T22:29:08.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.696+0000 7f556a89c700 1 -- 172.21.15.16:0/3118349757 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5564138dc0 msgr2=0x7f556413b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-09-30T22:29:08.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.696+0000 7f556a89c700 1 -- 172.21.15.16:0/3118349757 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5564138dc0 msgr2=0x7f556413b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-09-30T22:29:08.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.696+0000 7f556a89c700 1 --2- 172.21.15.16:0/3118349757 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5564138dc0 0x7f556413b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-09-30T22:29:08.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.696+0000 7f556a89c700 1 --2- 172.21.15.16:0/3118349757 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5564138dc0 0x7f556413b1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-09-30T22:29:08.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.729+0000 7f5569099700 1 -- 172.21.15.16:0/3118349757 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 133843553 0 0) 0x7f556406b990 con 0x7f5564133420 2023-09-30T22:29:08.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.729+0000 7f5569099700 1 -- 172.21.15.16:0/3118349757 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55500035b0 con 0x7f5564133420 2023-09-30T22:29:08.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.730+0000 7f5569099700 1 -- 172.21.15.16:0/3118349757 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1703008479 0 0) 0x7f55500035b0 con 0x7f5564133420 2023-09-30T22:29:08.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.730+0000 7f5569099700 1 -- 172.21.15.16:0/3118349757 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5564138dc0 msgr2=0x7f556413b1a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T22:29:08.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.730+0000 7f5569099700 1 --2- 172.21.15.16:0/3118349757 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5564138dc0 0x7f556413b1a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:08.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.730+0000 7f5569099700 1 -- 172.21.15.16:0/3118349757 >> v2:172.21.15.16:3300/0 conn(0x7f55641337f0 msgr2=0x7f5564138880 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T22:29:08.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.730+0000 7f5569099700 1 --2- 172.21.15.16:0/3118349757 >> v2:172.21.15.16:3300/0 conn(0x7f55641337f0 0x7f5564138880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:08.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.730+0000 7f5569099700 1 -- 172.21.15.16:0/3118349757 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55640a0590 con 0x7f5564133420 2023-09-30T22:29:08.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.730+0000 7f5569099700 1 -- 172.21.15.16:0/3118349757 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1074163509 0 0) 0x7f5560001670 con 0x7f5564133420 2023-09-30T22:29:08.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.730+0000 7f5569099700 1 -- 172.21.15.16:0/3118349757 >> v1:172.21.15.16:6789/0 conn(0x7f5564133420 legacy=0x7f5564133e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:08.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.730+0000 7f5569099700 1 --2- 172.21.15.16:0/3118349757 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5550003a00 0x7f5550003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:08.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.730+0000 7f5569099700 1 --2- 172.21.15.16:0/3118349757 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5564138dc0 0x7f5550004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:08.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.730+0000 7f5569099700 1 --2- 172.21.15.16:0/3118349757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5550004d90 0x7f5550005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:08.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.730+0000 7f5569099700 1 -- 172.21.15.16:0/3118349757 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f556409b0e0 con 0x7f5550004d90 2023-09-30T22:29:08.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.730+0000 7f5569099700 1 -- 172.21.15.16:0/3118349757 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5564133c20 con 0x7f5564138dc0 2023-09-30T22:29:08.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.730+0000 7f5569099700 1 -- 172.21.15.16:0/3118349757 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f555000bbc0 con 0x7f5550003a00 2023-09-30T22:29:08.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.730+0000 7f556a09b700 1 --2- 172.21.15.16:0/3118349757 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5550003a00 0x7f5550003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:08.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.730+0000 7f556989a700 1 --2- 172.21.15.16:0/3118349757 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5564138dc0 0x7f5550004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:08.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.730+0000 7f556a89c700 1 --2- 172.21.15.16:0/3118349757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5550004d90 0x7f5550005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:08.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.731+0000 7f556a09b700 1 -- 172.21.15.16:0/3118349757 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5564138dc0 msgr2=0x7f5550004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:08.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.731+0000 7f556a09b700 1 --2- 172.21.15.16:0/3118349757 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5564138dc0 0x7f5550004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:08.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.731+0000 7f556a09b700 1 -- 172.21.15.16:0/3118349757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5550004d90 msgr2=0x7f5550005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:08.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.731+0000 7f556a09b700 1 --2- 172.21.15.16:0/3118349757 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5550004d90 0x7f5550005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:08.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.731+0000 7f556a09b700 1 -- 172.21.15.16:0/3118349757 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55500035b0 con 0x7f5550003a00 2023-09-30T22:29:08.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.731+0000 7f556a09b700 1 --2- 172.21.15.16:0/3118349757 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5550003a00 0x7f5550003e10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f556000c210 tx=0x7f5550005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=395d95d4f8ba0cb6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:08.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.734+0000 7f5569099700 1 -- 172.21.15.16:0/3118349757 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5560007b80 con 0x7f5550003a00 2023-09-30T22:29:08.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.734+0000 7f5569099700 1 -- 172.21.15.16:0/3118349757 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f556000cd50 con 0x7f5550003a00 2023-09-30T22:29:08.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.734+0000 7f5569099700 1 -- 172.21.15.16:0/3118349757 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5560005ae0 con 0x7f5550003a00 2023-09-30T22:29:08.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.734+0000 7f556c2ff700 1 -- 172.21.15.16:0/3118349757 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5550003a00 msgr2=0x7f5550003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:08.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.734+0000 7f556c2ff700 1 --2- 172.21.15.16:0/3118349757 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5550003a00 0x7f5550003e10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f556000c210 tx=0x7f5550005880 comp rx=0 tx=0).stop 2023-09-30T22:29:08.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.734+0000 7f556a89c700 1 -- 172.21.15.16:0/3118349757 reap_dead start 2023-09-30T22:29:08.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.735+0000 7f556c2ff700 1 -- 172.21.15.16:0/3118349757 shutdown_connections 2023-09-30T22:29:08.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.735+0000 7f556c2ff700 1 -- 172.21.15.16:0/3118349757 >> 172.21.15.16:0/3118349757 conn(0x7f55640a4210 msgr2=0x7f55640a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:08.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.735+0000 7f556c2ff700 1 -- 172.21.15.16:0/3118349757 shutdown_connections 2023-09-30T22:29:08.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.735+0000 7f556c2ff700 1 -- 172.21.15.16:0/3118349757 wait complete. 2023-09-30T22:29:08.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.736+0000 7f556c2ff700 1 Processor -- start 2023-09-30T22:29:08.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.736+0000 7f556c2ff700 1 -- start start 2023-09-30T22:29:08.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.736+0000 7f556c2ff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5550003a00 0x7f55641c8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:08.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.736+0000 7f556c2ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5564130410 0x7f55641c86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:08.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.736+0000 7f556c2ff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f55641c8c00 0x7f556412bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:08.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.736+0000 7f556c2ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f556409df20 con 0x7f5564130410 2023-09-30T22:29:08.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.736+0000 7f556c2ff700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5564133190 con 0x7f5550003a00 2023-09-30T22:29:08.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.736+0000 7f556c2ff700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f556409c830 con 0x7f55641c8c00 2023-09-30T22:29:08.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.737+0000 7f556a09b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5550003a00 0x7f55641c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:08.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.737+0000 7f556a09b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5550003a00 0x7f55641c8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:53762/0 (socket says 172.21.15.16:53762) 2023-09-30T22:29:08.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.737+0000 7f556a09b700 1 -- 172.21.15.16:0/689678633 learned_addr learned my addr 172.21.15.16:0/689678633 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:08.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.737+0000 7f556989a700 1 --2- 172.21.15.16:0/689678633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5564130410 0x7f55641c86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:08.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.737+0000 7f556a89c700 1 --2- 172.21.15.16:0/689678633 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f55641c8c00 0x7f556412bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:08.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.737+0000 7f556a09b700 1 -- 172.21.15.16:0/689678633 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f55641c8c00 msgr2=0x7f556412bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:08.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.737+0000 7f556a09b700 1 --2- 172.21.15.16:0/689678633 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f55641c8c00 0x7f556412bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:08.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.737+0000 7f556a09b700 1 -- 172.21.15.16:0/689678633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5564130410 msgr2=0x7f55641c86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:08.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.737+0000 7f556a09b700 1 --2- 172.21.15.16:0/689678633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5564130410 0x7f55641c86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:08.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.737+0000 7f556a09b700 1 -- 172.21.15.16:0/689678633 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f556409b4d0 con 0x7f5550003a00 2023-09-30T22:29:08.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.737+0000 7f556989a700 1 --2- 172.21.15.16:0/689678633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5564130410 0x7f55641c86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:29:08.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.737+0000 7f556a09b700 1 --2- 172.21.15.16:0/689678633 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5550003a00 0x7f55641c8180 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5560001820 tx=0x7f556409f9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:08.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.738+0000 7f5556ffd700 1 -- 172.21.15.16:0/689678633 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556000cec0 con 0x7f5550003a00 2023-09-30T22:29:08.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.738+0000 7f5556ffd700 1 -- 172.21.15.16:0/689678633 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55600070d0 con 0x7f5550003a00 2023-09-30T22:29:08.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.738+0000 7f5556ffd700 1 -- 172.21.15.16:0/689678633 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5560007440 con 0x7f5550003a00 2023-09-30T22:29:08.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.738+0000 7f556c2ff700 1 -- 172.21.15.16:0/689678633 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f556409ca70 con 0x7f5550003a00 2023-09-30T22:29:08.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.738+0000 7f556c2ff700 1 -- 172.21.15.16:0/689678633 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f556412c0a0 con 0x7f5550003a00 2023-09-30T22:29:08.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.739+0000 7f5556ffd700 1 -- 172.21.15.16:0/689678633 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7f55600075a0 con 0x7f5550003a00 2023-09-30T22:29:08.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.739+0000 7f5556ffd700 1 -- 172.21.15.16:0/689678633 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3922+0+0 (secure 0 0 0) 0x7f556003cd20 con 0x7f5550003a00 2023-09-30T22:29:08.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.739+0000 7f556989a700 1 -- 172.21.15.16:0/689678633 >> v1:172.21.15.16:6816/118285 conn(0x7f554802c1b0 legacy=0x7f554802e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/118285 2023-09-30T22:29:08.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.740+0000 7f556c2ff700 1 -- 172.21.15.16:0/689678633 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f556409dd60 con 0x7f5550003a00 2023-09-30T22:29:08.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.745+0000 7f5556ffd700 1 -- 172.21.15.16:0/689678633 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7f5560017020 con 0x7f5550003a00 2023-09-30T22:29:08.869 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:29:08.867+0000 7f90c460e240 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-09-30T22:29:08.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.940+0000 7f556989a700 1 -- 172.21.15.16:0/689678633 >> v1:172.21.15.16:6816/118285 conn(0x7f554802c1b0 legacy=0x7f554802e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/118285 2023-09-30T22:29:08.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.976+0000 7f556c2ff700 1 -- 172.21.15.16:0/689678633 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f556409db40 con 0x7f5550003a00 2023-09-30T22:29:08.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.980+0000 7f5556ffd700 1 -- 172.21.15.16:0/689678633 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+64282 (secure 0 0 0) 0x7f556001e430 con 0x7f5550003a00 2023-09-30T22:29:08.986 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:29:08.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.984+0000 7f556c2ff700 1 -- 172.21.15.16:0/689678633 >> v1:172.21.15.16:6816/118285 conn(0x7f554802c1b0 legacy=0x7f554802e660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:08.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.984+0000 7f556c2ff700 1 -- 172.21.15.16:0/689678633 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5550003a00 msgr2=0x7f55641c8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:08.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.984+0000 7f556c2ff700 1 --2- 172.21.15.16:0/689678633 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5550003a00 0x7f55641c8180 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5560001820 tx=0x7f556409f9f0 comp rx=0 tx=0).stop 2023-09-30T22:29:08.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.984+0000 7f556c2ff700 1 -- 172.21.15.16:0/689678633 shutdown_connections 2023-09-30T22:29:08.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.984+0000 7f556c2ff700 1 --2- 172.21.15.16:0/689678633 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5550003a00 0x7f55641c8180 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:08.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.985+0000 7f556c2ff700 1 --2- 172.21.15.16:0/689678633 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f55641c8c00 0x7f556412bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:08.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.985+0000 7f556c2ff700 1 --2- 172.21.15.16:0/689678633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5564130410 0x7f55641c86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:08.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.985+0000 7f556c2ff700 1 -- 172.21.15.16:0/689678633 >> 172.21.15.16:0/689678633 conn(0x7f55640a4210 msgr2=0x7f556412cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:08.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.985+0000 7f556c2ff700 1 -- 172.21.15.16:0/689678633 shutdown_connections 2023-09-30T22:29:08.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:08.985+0000 7f556c2ff700 1 -- 172.21.15.16:0/689678633 wait complete. 2023-09-30T22:29:09.012 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":5,"active_gid":4098,"active_name":"x","active_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6816","nonce":118285}]},"active_addr":"172.21.15.16:6816/118285","active_change":"2023-09-30T22:26:40.448871+0000","active_mgr_features":4540138314316775423,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":1101648925}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":2249754484}]}]} 2023-09-30T22:29:09.013 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-09-30T22:29:09.014 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-09-30T22:29:09.014 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:29:09.051 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:29:09.050+0000 7f90c460e240 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-09-30T22:29:09.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.164+0000 7fca0290a700 1 Processor -- start 2023-09-30T22:29:09.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.164+0000 7fca0290a700 1 -- start start 2023-09-30T22:29:09.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.164+0000 7fca0290a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc9fc0a0a30 0x7fc9fc0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:09.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.164+0000 7fca0290a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc9fc134170 0x7fc9fc13bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:09.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.164+0000 7fca0290a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9fc09edd0 con 0x7fc9fc0a1340 2023-09-30T22:29:09.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.164+0000 7fca0290a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9fc09e1f0 con 0x7fc9fc0a0a30 2023-09-30T22:29:09.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.164+0000 7fca0290a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc9fc09dc50 con 0x7fc9fc134170 2023-09-30T22:29:09.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.165+0000 7fc9fbfff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc9fc0a0a30 0x7fc9fc0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:09.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.165+0000 7fca00ea7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc9fc134170 0x7fc9fc13bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:09.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.165+0000 7fc9fb7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc9fc0a1340 0x7fc9fc133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40232/0 (socket says 172.21.15.16:40232) 2023-09-30T22:29:09.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.165+0000 7fc9fbfff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc9fc0a0a30 0x7fc9fc0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:53770/0 (socket says 172.21.15.16:53770) 2023-09-30T22:29:09.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.165+0000 7fc9fb7fe700 1 -- 172.21.15.16:0/4098205860 learned_addr learned my addr 172.21.15.16:0/4098205860 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:29:09.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.165+0000 7fc9fbfff700 1 -- 172.21.15.16:0/4098205860 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc9fc134170 msgr2=0x7fc9fc13bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:09.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.165+0000 7fc9fbfff700 1 --2- 172.21.15.16:0/4098205860 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc9fc134170 0x7fc9fc13bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:09.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.165+0000 7fc9fbfff700 1 -- 172.21.15.16:0/4098205860 >> v1:172.21.15.16:6789/0 conn(0x7fc9fc0a1340 legacy=0x7fc9fc133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:09.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.165+0000 7fc9faffd700 1 -- 172.21.15.16:0/4098205860 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1847016355 0 0) 0x7fc9fc09edd0 con 0x7fc9fc0a1340 2023-09-30T22:29:09.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.165+0000 7fc9fbfff700 1 -- 172.21.15.16:0/4098205860 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9fc0a1fe0 con 0x7fc9fc0a0a30 2023-09-30T22:29:09.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.165+0000 7fca00ea7700 1 --2- 172.21.15.16:0/4098205860 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc9fc134170 0x7fc9fc13bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:09.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.165+0000 7fc9fbfff700 1 --2- 172.21.15.16:0/4098205860 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc9fc0a0a30 0x7fc9fc0a0e00 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec009870 tx=0x7fc9ec009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f001dd26d4b8aa99 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:09.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.165+0000 7fc9faffd700 1 -- 172.21.15.16:0/4098205860 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9ec011830 con 0x7fc9fc0a0a30 2023-09-30T22:29:09.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.165+0000 7fc9faffd700 1 -- 172.21.15.16:0/4098205860 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9ec011990 con 0x7fc9fc0a0a30 2023-09-30T22:29:09.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.165+0000 7fc9faffd700 1 -- 172.21.15.16:0/4098205860 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9ec011d20 con 0x7fc9fc0a0a30 2023-09-30T22:29:09.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.166+0000 7fca0290a700 1 -- 172.21.15.16:0/4098205860 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc9fc0a0a30 msgr2=0x7fc9fc0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:09.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.166+0000 7fca0290a700 1 --2- 172.21.15.16:0/4098205860 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc9fc0a0a30 0x7fc9fc0a0e00 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec009870 tx=0x7fc9ec009b80 comp rx=0 tx=0).stop 2023-09-30T22:29:09.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.166+0000 7fca0290a700 1 -- 172.21.15.16:0/4098205860 shutdown_connections 2023-09-30T22:29:09.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.166+0000 7fca0290a700 1 --2- 172.21.15.16:0/4098205860 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc9fc134170 0x7fc9fc13bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:09.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.166+0000 7fca0290a700 1 --2- 172.21.15.16:0/4098205860 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc9fc0a0a30 0x7fc9fc0a0e00 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:09.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.166+0000 7fca0290a700 1 -- 172.21.15.16:0/4098205860 >> 172.21.15.16:0/4098205860 conn(0x7fc9fc0a6550 msgr2=0x7fc9fc0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:09.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.166+0000 7fca0290a700 1 -- 172.21.15.16:0/4098205860 shutdown_connections 2023-09-30T22:29:09.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.166+0000 7fca0290a700 1 -- 172.21.15.16:0/4098205860 wait complete. 2023-09-30T22:29:09.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.167+0000 7fca0290a700 1 Processor -- start 2023-09-30T22:29:09.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.167+0000 7fca0290a700 1 -- start start 2023-09-30T22:29:09.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.168+0000 7fca0290a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc9fc0a0a30 0x7fc9fc1c8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:09.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.168+0000 7fca0290a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc9fc0a1340 0x7fc9fc1c8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:09.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.168+0000 7fca0290a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9fc134170 0x7fc9fc1c8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:09.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.168+0000 7fca0290a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9fc126440 con 0x7fc9fc134170 2023-09-30T22:29:09.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.168+0000 7fca0290a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9fc1246d0 con 0x7fc9fc0a0a30 2023-09-30T22:29:09.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.168+0000 7fca0290a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc9fc1244a0 con 0x7fc9fc0a1340 2023-09-30T22:29:09.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.168+0000 7fc9fbfff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc9fc0a0a30 0x7fc9fc1c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:09.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.168+0000 7fc9fbfff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc9fc0a0a30 0x7fc9fc1c8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:53774/0 (socket says 172.21.15.16:53774) 2023-09-30T22:29:09.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.168+0000 7fc9fbfff700 1 -- 172.21.15.16:0/524956570 learned_addr learned my addr 172.21.15.16:0/524956570 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:09.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.168+0000 7fc9fb7fe700 1 --2- 172.21.15.16:0/524956570 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc9fc0a1340 0x7fc9fc1c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:09.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.168+0000 7fca00ea7700 1 --2- 172.21.15.16:0/524956570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9fc134170 0x7fc9fc1c8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:09.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.168+0000 7fca00ea7700 1 -- 172.21.15.16:0/524956570 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc9fc0a1340 msgr2=0x7fc9fc1c8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:09.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.168+0000 7fca00ea7700 1 --2- 172.21.15.16:0/524956570 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc9fc0a1340 0x7fc9fc1c8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:09.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.168+0000 7fca00ea7700 1 -- 172.21.15.16:0/524956570 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc9fc0a0a30 msgr2=0x7fc9fc1c8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:09.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.168+0000 7fca00ea7700 1 --2- 172.21.15.16:0/524956570 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc9fc0a0a30 0x7fc9fc1c8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:09.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.168+0000 7fca00ea7700 1 -- 172.21.15.16:0/524956570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9f4009a70 con 0x7fc9fc134170 2023-09-30T22:29:09.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.168+0000 7fc9fbfff700 1 --2- 172.21.15.16:0/524956570 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc9fc0a0a30 0x7fc9fc1c8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:09.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.169+0000 7fca00ea7700 1 --2- 172.21.15.16:0/524956570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9fc134170 0x7fc9fc1c8ed0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4009e50 tx=0x7fc9f400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:09.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.169+0000 7fc9f8ff9700 1 -- 172.21.15.16:0/524956570 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f400cb40 con 0x7fc9fc134170 2023-09-30T22:29:09.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.169+0000 7fc9f8ff9700 1 -- 172.21.15.16:0/524956570 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9f400f040 con 0x7fc9fc134170 2023-09-30T22:29:09.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.169+0000 7fca0290a700 1 -- 172.21.15.16:0/524956570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9ec0094a0 con 0x7fc9fc134170 2023-09-30T22:29:09.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.169+0000 7fc9f8ff9700 1 -- 172.21.15.16:0/524956570 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f4003a20 con 0x7fc9fc134170 2023-09-30T22:29:09.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.169+0000 7fca0290a700 1 -- 172.21.15.16:0/524956570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9fc0a25e0 con 0x7fc9fc134170 2023-09-30T22:29:09.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.170+0000 7fc9f8ff9700 1 -- 172.21.15.16:0/524956570 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7fc9f4014480 con 0x7fc9fc134170 2023-09-30T22:29:09.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.170+0000 7fc9f8ff9700 1 -- 172.21.15.16:0/524956570 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3922+0+0 (secure 0 0 0) 0x7fc9f403c0e0 con 0x7fc9fc134170 2023-09-30T22:29:09.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.170+0000 7fc9fbfff700 1 -- 172.21.15.16:0/524956570 >> v1:172.21.15.16:6816/118285 conn(0x7fc9e802c1b0 legacy=0x7fc9e802e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/118285 2023-09-30T22:29:09.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.170+0000 7fca0290a700 1 -- 172.21.15.16:0/524956570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9fc09c7e0 con 0x7fc9fc134170 2023-09-30T22:29:09.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.174+0000 7fc9f8ff9700 1 -- 172.21.15.16:0/524956570 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7fc9f4018030 con 0x7fc9fc134170 2023-09-30T22:29:09.338 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:29:09.337+0000 7f90c460e240 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-09-30T22:29:09.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.357+0000 7fca0290a700 1 -- 172.21.15.16:0/524956570 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc9fc074ea0 con 0x7fc9fc134170 2023-09-30T22:29:09.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.358+0000 7fc9f8ff9700 1 -- 172.21.15.16:0/524956570 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+12738 (secure 0 0 0) 0x7fc9f4012930 con 0x7fc9fc134170 2023-09-30T22:29:09.360 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:29:09.360 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":19,"fsid":"7ede3e14-2bd0-4726-9437-6d13c90cd342","created":"2023-09-30T22:26:35.483697+0000","modified":"2023-09-30T22:27:42.229390+0000","last_up_change":"2023-09-30T22:26:44.227199+0000","last_in_change":"2023-09-30T22:26:37.120906+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"luminous","require_osd_release":"pacific","allow_crimson":false,"pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-09-30T22:26:44.285133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:26:45.132921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:26:49.353683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:26:50.655685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"19946985-0ee2-487f-9347-929670dbe5ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6804","nonce":118824}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6805","nonce":118824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6807","nonce":118824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6806","nonce":118824}]},"public_addr":"172.21.15.16:6804/118824","cluster_addr":"172.21.15.16:6805/118824","heartbeat_back_addr":"172.21.15.16:6807/118824","heartbeat_front_addr":"172.21.15.16:6806/118824","state":["exists","up"]},{"osd":1,"uuid":"8efd4dbc-6ae4-4b82-b530-3b3eac2be445","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6812","nonce":118832}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6813","nonce":118832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6815","nonce":118832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6814","nonce":118832}]},"public_addr":"172.21.15.16:6812/118832","cluster_addr":"172.21.15.16:6813/118832","heartbeat_back_addr":"172.21.15.16:6815/118832","heartbeat_front_addr":"172.21.15.16:6814/118832","state":["exists","up"]},{"osd":2,"uuid":"b47bb1ce-27af-4c0c-868a-b76ebd0d9499","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6808","nonce":118840}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6809","nonce":118840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6811","nonce":118840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6810","nonce":118840}]},"public_addr":"172.21.15.16:6808/118840","cluster_addr":"172.21.15.16:6809/118840","heartbeat_back_addr":"172.21.15.16:6811/118840","heartbeat_front_addr":"172.21.15.16:6810/118840","state":["exists","up"]},{"osd":3,"uuid":"03b53d64-d10d-448a-a081-39ecceab589b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6800","nonce":118823}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6801","nonce":118823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6803","nonce":118823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6802","nonce":118823}]},"public_addr":"172.21.15.16:6800/118823","cluster_addr":"172.21.15.16:6801/118823","heartbeat_back_addr":"172.21.15.16:6803/118823","heartbeat_front_addr":"172.21.15.16:6802/118823","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.305953+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.307488+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.341023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.192071+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.8","mappings":[{"from":3,"to":2}]},{"pgid":"4.d","mappings":[{"from":3,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":0}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]},{"pgid":"4.17","mappings":[{"from":3,"to":2}]},{"pgid":"4.19","mappings":[{"from":3,"to":0}]},{"pgid":"4.3a","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:29:09.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.360+0000 7fca0290a700 1 -- 172.21.15.16:0/524956570 >> v1:172.21.15.16:6816/118285 conn(0x7fc9e802c1b0 legacy=0x7fc9e802e660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:09.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.360+0000 7fca0290a700 1 -- 172.21.15.16:0/524956570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9fc134170 msgr2=0x7fc9fc1c8ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:09.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.360+0000 7fca0290a700 1 --2- 172.21.15.16:0/524956570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9fc134170 0x7fc9fc1c8ed0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4009e50 tx=0x7fc9f400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:29:09.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.360+0000 7fca0290a700 1 -- 172.21.15.16:0/524956570 shutdown_connections 2023-09-30T22:29:09.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.360+0000 7fca0290a700 1 --2- 172.21.15.16:0/524956570 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc9fc0a0a30 0x7fc9fc1c8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:09.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.360+0000 7fca0290a700 1 --2- 172.21.15.16:0/524956570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9fc134170 0x7fc9fc1c8ed0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:09.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.360+0000 7fca0290a700 1 --2- 172.21.15.16:0/524956570 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc9fc0a1340 0x7fc9fc1c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:09.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.360+0000 7fca0290a700 1 -- 172.21.15.16:0/524956570 >> 172.21.15.16:0/524956570 conn(0x7fc9fc0a6550 msgr2=0x7fc9fc09edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:09.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.361+0000 7fca0290a700 1 -- 172.21.15.16:0/524956570 shutdown_connections 2023-09-30T22:29:09.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.361+0000 7fca0290a700 1 -- 172.21.15.16:0/524956570 wait complete. 2023-09-30T22:29:09.377 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-09-30T22:29:09.377 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:29:09.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.555+0000 7faf80f00700 1 Processor -- start 2023-09-30T22:29:09.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.555+0000 7faf80f00700 1 -- start start 2023-09-30T22:29:09.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.556+0000 7faf80f00700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faf7c0a0a30 0x7faf7c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:09.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.556+0000 7faf80f00700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf7c134170 0x7faf7c13bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:09.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.556+0000 7faf80f00700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf7c09edd0 con 0x7faf7c0a1340 2023-09-30T22:29:09.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.556+0000 7faf80f00700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf7c09e1f0 con 0x7faf7c0a0a30 2023-09-30T22:29:09.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.556+0000 7faf80f00700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf7c09dc50 con 0x7faf7c134170 2023-09-30T22:29:09.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.556+0000 7faf7ad9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf7c134170 0x7faf7c13bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:09.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.556+0000 7faf7a59c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faf7c0a0a30 0x7faf7c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:09.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.556+0000 7faf79d9b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7faf7c0a1340 0x7faf7c133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40236/0 (socket says 172.21.15.16:40236) 2023-09-30T22:29:09.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.556+0000 7faf79d9b700 1 -- 172.21.15.16:0/509644838 learned_addr learned my addr 172.21.15.16:0/509644838 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:29:09.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.556+0000 7faf7a59c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faf7c0a0a30 0x7faf7c0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46752/0 (socket says 172.21.15.16:46752) 2023-09-30T22:29:09.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.556+0000 7faf7ad9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf7c134170 0x7faf7c13bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:53780/0 (socket says 172.21.15.16:53780) 2023-09-30T22:29:09.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.557+0000 7faf7ad9d700 1 -- 172.21.15.16:0/509644838 >> v2:172.21.15.16:3300/0 conn(0x7faf7c0a0a30 msgr2=0x7faf7c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:09.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.557+0000 7faf7ad9d700 1 --2- 172.21.15.16:0/509644838 >> v2:172.21.15.16:3300/0 conn(0x7faf7c0a0a30 0x7faf7c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:09.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.557+0000 7faf7ad9d700 1 -- 172.21.15.16:0/509644838 >> v1:172.21.15.16:6789/0 conn(0x7faf7c0a1340 legacy=0x7faf7c133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:09.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.557+0000 7faf7ad9d700 1 -- 172.21.15.16:0/509644838 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf7c0a1fe0 con 0x7faf7c134170 2023-09-30T22:29:09.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.557+0000 7faf7a59c700 1 --2- 172.21.15.16:0/509644838 >> v2:172.21.15.16:3300/0 conn(0x7faf7c0a0a30 0x7faf7c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:09.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.557+0000 7faf7ad9d700 1 --2- 172.21.15.16:0/509644838 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf7c134170 0x7faf7c13bd30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7faf74009840 tx=0x7faf74009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dee7ba59035be48 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:09.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.557+0000 7faf7959a700 1 -- 172.21.15.16:0/509644838 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf74011830 con 0x7faf7c134170 2023-09-30T22:29:09.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.557+0000 7faf7959a700 1 -- 172.21.15.16:0/509644838 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf74011990 con 0x7faf7c134170 2023-09-30T22:29:09.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.557+0000 7faf7959a700 1 -- 172.21.15.16:0/509644838 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf74011d20 con 0x7faf7c134170 2023-09-30T22:29:09.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.558+0000 7faf80f00700 1 -- 172.21.15.16:0/509644838 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf7c134170 msgr2=0x7faf7c13bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:09.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.558+0000 7faf80f00700 1 --2- 172.21.15.16:0/509644838 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf7c134170 0x7faf7c13bd30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7faf74009840 tx=0x7faf74009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:09.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.558+0000 7faf80f00700 1 -- 172.21.15.16:0/509644838 shutdown_connections 2023-09-30T22:29:09.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.558+0000 7faf80f00700 1 --2- 172.21.15.16:0/509644838 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf7c134170 0x7faf7c13bd30 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:09.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.558+0000 7faf80f00700 1 --2- 172.21.15.16:0/509644838 >> v2:172.21.15.16:3300/0 conn(0x7faf7c0a0a30 0x7faf7c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:09.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.558+0000 7faf80f00700 1 -- 172.21.15.16:0/509644838 >> 172.21.15.16:0/509644838 conn(0x7faf7c0a6550 msgr2=0x7faf7c0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:09.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.558+0000 7faf80f00700 1 -- 172.21.15.16:0/509644838 shutdown_connections 2023-09-30T22:29:09.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.558+0000 7faf80f00700 1 -- 172.21.15.16:0/509644838 wait complete. 2023-09-30T22:29:09.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.559+0000 7faf80f00700 1 Processor -- start 2023-09-30T22:29:09.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.559+0000 7faf80f00700 1 -- start start 2023-09-30T22:29:09.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.560+0000 7faf80f00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf7c0a0a30 0x7faf7c11e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:09.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.560+0000 7faf80f00700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf7c0a1340 0x7faf7c11ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:09.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.560+0000 7faf80f00700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf7c134170 0x7faf7c11f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:09.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.560+0000 7faf80f00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf7c11fbe0 con 0x7faf7c0a0a30 2023-09-30T22:29:09.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.560+0000 7faf80f00700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf7c1221a0 con 0x7faf7c0a1340 2023-09-30T22:29:09.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.560+0000 7faf80f00700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faf7c120290 con 0x7faf7c134170 2023-09-30T22:29:09.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.560+0000 7faf7ad9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf7c134170 0x7faf7c11f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:09.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.560+0000 7faf7a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf7c0a0a30 0x7faf7c11e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:09.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.560+0000 7faf7ad9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf7c134170 0x7faf7c11f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:58210/0 (socket says 172.21.15.16:58210) 2023-09-30T22:29:09.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.560+0000 7faf7ad9d700 1 -- 172.21.15.16:0/2575168600 learned_addr learned my addr 172.21.15.16:0/2575168600 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:09.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.560+0000 7faf79d9b700 1 --2- 172.21.15.16:0/2575168600 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf7c0a1340 0x7faf7c11ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:09.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.560+0000 7faf7a59c700 1 -- 172.21.15.16:0/2575168600 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf7c134170 msgr2=0x7faf7c11f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:09.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.560+0000 7faf7a59c700 1 --2- 172.21.15.16:0/2575168600 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf7c134170 0x7faf7c11f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:09.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.560+0000 7faf7a59c700 1 -- 172.21.15.16:0/2575168600 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf7c0a1340 msgr2=0x7faf7c11ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:09.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.560+0000 7faf7a59c700 1 --2- 172.21.15.16:0/2575168600 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf7c0a1340 0x7faf7c11ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:09.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.560+0000 7faf7a59c700 1 -- 172.21.15.16:0/2575168600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf6c009a70 con 0x7faf7c0a0a30 2023-09-30T22:29:09.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.560+0000 7faf7a59c700 1 --2- 172.21.15.16:0/2575168600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf7c0a0a30 0x7faf7c11e710 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7faf6c009e60 tx=0x7faf6c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:09.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.560+0000 7faf72ffd700 1 -- 172.21.15.16:0/2575168600 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6c00cd10 con 0x7faf7c0a0a30 2023-09-30T22:29:09.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.560+0000 7faf72ffd700 1 -- 172.21.15.16:0/2575168600 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf6c00f040 con 0x7faf7c0a0a30 2023-09-30T22:29:09.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.561+0000 7faf80f00700 1 -- 172.21.15.16:0/2575168600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf740094a0 con 0x7faf7c0a0a30 2023-09-30T22:29:09.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.561+0000 7faf72ffd700 1 -- 172.21.15.16:0/2575168600 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf6c0146c0 con 0x7faf7c0a0a30 2023-09-30T22:29:09.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.561+0000 7faf80f00700 1 -- 172.21.15.16:0/2575168600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf7c0a25e0 con 0x7faf7c0a0a30 2023-09-30T22:29:09.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.561+0000 7faf72ffd700 1 -- 172.21.15.16:0/2575168600 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7faf6c014820 con 0x7faf7c0a0a30 2023-09-30T22:29:09.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.561+0000 7faf72ffd700 1 -- 172.21.15.16:0/2575168600 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3922+0+0 (secure 0 0 0) 0x7faf6c03c580 con 0x7faf7c0a0a30 2023-09-30T22:29:09.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.561+0000 7faf79d9b700 1 -- 172.21.15.16:0/2575168600 >> v1:172.21.15.16:6816/118285 conn(0x7faf5c02c1b0 legacy=0x7faf5c02e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/118285 2023-09-30T22:29:09.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.562+0000 7faf80f00700 1 -- 172.21.15.16:0/2575168600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf7c074ea0 con 0x7faf7c0a0a30 2023-09-30T22:29:09.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.566+0000 7faf72ffd700 1 -- 172.21.15.16:0/2575168600 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7faf6c00ce70 con 0x7faf7c0a0a30 2023-09-30T22:29:09.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.762+0000 7faf79d9b700 1 -- 172.21.15.16:0/2575168600 >> v1:172.21.15.16:6816/118285 conn(0x7faf5c02c1b0 legacy=0x7faf5c02e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/118285 2023-09-30T22:29:09.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.768+0000 7faf80f00700 1 -- 172.21.15.16:0/2575168600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faf7c0a28e0 con 0x7faf7c0a0a30 2023-09-30T22:29:09.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.769+0000 7faf72ffd700 1 -- 172.21.15.16:0/2575168600 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+12738 (secure 0 0 0) 0x7faf6c012340 con 0x7faf7c0a0a30 2023-09-30T22:29:09.771 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:29:09.771 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":19,"fsid":"7ede3e14-2bd0-4726-9437-6d13c90cd342","created":"2023-09-30T22:26:35.483697+0000","modified":"2023-09-30T22:27:42.229390+0000","last_up_change":"2023-09-30T22:26:44.227199+0000","last_in_change":"2023-09-30T22:26:37.120906+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"luminous","require_osd_release":"pacific","allow_crimson":false,"pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-09-30T22:26:44.285133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:26:45.132921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:26:49.353683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:26:50.655685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"19946985-0ee2-487f-9347-929670dbe5ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6804","nonce":118824}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6805","nonce":118824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6807","nonce":118824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6806","nonce":118824}]},"public_addr":"172.21.15.16:6804/118824","cluster_addr":"172.21.15.16:6805/118824","heartbeat_back_addr":"172.21.15.16:6807/118824","heartbeat_front_addr":"172.21.15.16:6806/118824","state":["exists","up"]},{"osd":1,"uuid":"8efd4dbc-6ae4-4b82-b530-3b3eac2be445","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6812","nonce":118832}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6813","nonce":118832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6815","nonce":118832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6814","nonce":118832}]},"public_addr":"172.21.15.16:6812/118832","cluster_addr":"172.21.15.16:6813/118832","heartbeat_back_addr":"172.21.15.16:6815/118832","heartbeat_front_addr":"172.21.15.16:6814/118832","state":["exists","up"]},{"osd":2,"uuid":"b47bb1ce-27af-4c0c-868a-b76ebd0d9499","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6808","nonce":118840}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6809","nonce":118840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6811","nonce":118840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6810","nonce":118840}]},"public_addr":"172.21.15.16:6808/118840","cluster_addr":"172.21.15.16:6809/118840","heartbeat_back_addr":"172.21.15.16:6811/118840","heartbeat_front_addr":"172.21.15.16:6810/118840","state":["exists","up"]},{"osd":3,"uuid":"03b53d64-d10d-448a-a081-39ecceab589b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6800","nonce":118823}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6801","nonce":118823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6803","nonce":118823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6802","nonce":118823}]},"public_addr":"172.21.15.16:6800/118823","cluster_addr":"172.21.15.16:6801/118823","heartbeat_back_addr":"172.21.15.16:6803/118823","heartbeat_front_addr":"172.21.15.16:6802/118823","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.305953+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.307488+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.341023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.192071+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.8","mappings":[{"from":3,"to":2}]},{"pgid":"4.d","mappings":[{"from":3,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":0}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]},{"pgid":"4.17","mappings":[{"from":3,"to":2}]},{"pgid":"4.19","mappings":[{"from":3,"to":0}]},{"pgid":"4.3a","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:29:09.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.771+0000 7faf80f00700 1 -- 172.21.15.16:0/2575168600 >> v1:172.21.15.16:6816/118285 conn(0x7faf5c02c1b0 legacy=0x7faf5c02e660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:09.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.771+0000 7faf80f00700 1 -- 172.21.15.16:0/2575168600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf7c0a0a30 msgr2=0x7faf7c11e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:09.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.771+0000 7faf80f00700 1 --2- 172.21.15.16:0/2575168600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf7c0a0a30 0x7faf7c11e710 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7faf6c009e60 tx=0x7faf6c00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:29:09.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.772+0000 7faf80f00700 1 -- 172.21.15.16:0/2575168600 shutdown_connections 2023-09-30T22:29:09.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.772+0000 7faf80f00700 1 --2- 172.21.15.16:0/2575168600 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf7c134170 0x7faf7c11f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:09.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.772+0000 7faf80f00700 1 --2- 172.21.15.16:0/2575168600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf7c0a0a30 0x7faf7c11e710 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:09.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.772+0000 7faf80f00700 1 --2- 172.21.15.16:0/2575168600 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf7c0a1340 0x7faf7c11ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:09.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.772+0000 7faf80f00700 1 -- 172.21.15.16:0/2575168600 >> 172.21.15.16:0/2575168600 conn(0x7faf7c0a6550 msgr2=0x7faf7c09ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:09.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.772+0000 7faf80f00700 1 -- 172.21.15.16:0/2575168600 shutdown_connections 2023-09-30T22:29:09.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.772+0000 7faf80f00700 1 -- 172.21.15.16:0/2575168600 wait complete. 2023-09-30T22:29:09.787 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-09-30T22:29:09.788 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-09-30T22:29:09.788 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-09-30T22:29:09.788 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-09-30T22:29:09.930 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:29:09.929+0000 7f90c460e240 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-09-30T22:29:09.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.938+0000 7fe1cad66700 1 Processor -- start 2023-09-30T22:29:09.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.938+0000 7fe1cad66700 1 -- start start 2023-09-30T22:29:09.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.939+0000 7fe1cad66700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe1c412cf60 0x7fe1c406f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:09.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.939+0000 7fe1cad66700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe1c412f8b0 0x7fe1c412fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:09.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.939+0000 7fe1cad66700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1c412cd90 con 0x7fe1c4130260 2023-09-30T22:29:09.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.939+0000 7fe1cad66700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1c412d330 con 0x7fe1c412f8b0 2023-09-30T22:29:09.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.939+0000 7fe1cad66700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c41306e0 con 0x7fe1c412cf60 2023-09-30T22:29:09.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.939+0000 7fe1c3fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe1c412f8b0 0x7fe1c412fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:09.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.939+0000 7fe1c9303700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe1c4130260 0x7fe1c4136c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40242/0 (socket says 172.21.15.16:40242) 2023-09-30T22:29:09.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.939+0000 7fe1c9303700 1 -- 172.21.15.16:0/3973606610 learned_addr learned my addr 172.21.15.16:0/3973606610 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:29:09.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.939+0000 7fe1c8b02700 1 --2- 172.21.15.16:0/3973606610 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe1c412cf60 0x7fe1c406f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:09.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.939+0000 7fe1c3fff700 1 -- 172.21.15.16:0/3973606610 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe1c412cf60 msgr2=0x7fe1c406f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:09.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.939+0000 7fe1c3fff700 1 --2- 172.21.15.16:0/3973606610 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe1c412cf60 0x7fe1c406f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:09.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.939+0000 7fe1c3fff700 1 -- 172.21.15.16:0/3973606610 >> v1:172.21.15.16:6789/0 conn(0x7fe1c4130260 legacy=0x7fe1c4136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:09.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.939+0000 7fe1c3fff700 1 -- 172.21.15.16:0/3973606610 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c4071780 con 0x7fe1c412f8b0 2023-09-30T22:29:09.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.940+0000 7fe1c3fff700 1 --2- 172.21.15.16:0/3973606610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1c412f8b0 0x7fe1c412fd20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe1b000a9e0 tx=0x7fe1b000acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46c8feb7d072d6bf server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:09.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.940+0000 7fe1c37fe700 1 -- 172.21.15.16:0/3973606610 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b000cb70 con 0x7fe1c412f8b0 2023-09-30T22:29:09.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.940+0000 7fe1c37fe700 1 -- 172.21.15.16:0/3973606610 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1b000ccd0 con 0x7fe1c412f8b0 2023-09-30T22:29:09.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.940+0000 7fe1c37fe700 1 -- 172.21.15.16:0/3973606610 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b0018620 con 0x7fe1c412f8b0 2023-09-30T22:29:09.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.941+0000 7fe1cad66700 1 -- 172.21.15.16:0/3973606610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1c412f8b0 msgr2=0x7fe1c412fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:09.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.941+0000 7fe1cad66700 1 --2- 172.21.15.16:0/3973606610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1c412f8b0 0x7fe1c412fd20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe1b000a9e0 tx=0x7fe1b000acf0 comp rx=0 tx=0).stop 2023-09-30T22:29:09.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.941+0000 7fe1cad66700 1 -- 172.21.15.16:0/3973606610 shutdown_connections 2023-09-30T22:29:09.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.941+0000 7fe1cad66700 1 --2- 172.21.15.16:0/3973606610 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe1c412cf60 0x7fe1c406f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:09.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.941+0000 7fe1cad66700 1 --2- 172.21.15.16:0/3973606610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1c412f8b0 0x7fe1c412fd20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:09.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.941+0000 7fe1cad66700 1 -- 172.21.15.16:0/3973606610 >> 172.21.15.16:0/3973606610 conn(0x7fe1c40a2aa0 msgr2=0x7fe1c40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:09.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.941+0000 7fe1cad66700 1 -- 172.21.15.16:0/3973606610 shutdown_connections 2023-09-30T22:29:09.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.941+0000 7fe1cad66700 1 -- 172.21.15.16:0/3973606610 wait complete. 2023-09-30T22:29:09.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.942+0000 7fe1cad66700 1 Processor -- start 2023-09-30T22:29:09.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.942+0000 7fe1cad66700 1 -- start start 2023-09-30T22:29:09.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.943+0000 7fe1cad66700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe1c412cf60 0x7fe1c409b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:09.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.943+0000 7fe1cad66700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe1c412f8b0 0x7fe1c409bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:09.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.943+0000 7fe1cad66700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1c4130260 0x7fe1c41c3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:09.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.943+0000 7fe1cad66700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c409da00 con 0x7fe1c4130260 2023-09-30T22:29:09.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.943+0000 7fe1cad66700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c4129a00 con 0x7fe1c412cf60 2023-09-30T22:29:09.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.943+0000 7fe1cad66700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c409b740 con 0x7fe1c412f8b0 2023-09-30T22:29:09.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.943+0000 7fe1c9303700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1c4130260 0x7fe1c41c3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:09.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.943+0000 7fe1c9303700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1c4130260 0x7fe1c41c3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46778/0 (socket says 172.21.15.16:46778) 2023-09-30T22:29:09.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.943+0000 7fe1c9303700 1 -- 172.21.15.16:0/3250657168 learned_addr learned my addr 172.21.15.16:0/3250657168 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:09.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.943+0000 7fe1c3fff700 1 --2- 172.21.15.16:0/3250657168 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe1c412f8b0 0x7fe1c409bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:09.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.943+0000 7fe1c3fff700 1 -- 172.21.15.16:0/3250657168 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe1c412cf60 msgr2=0x7fe1c409b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:09.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.943+0000 7fe1c3fff700 1 --2- 172.21.15.16:0/3250657168 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe1c412cf60 0x7fe1c409b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:09.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.943+0000 7fe1c3fff700 1 -- 172.21.15.16:0/3250657168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1c4130260 msgr2=0x7fe1c41c3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:09.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.943+0000 7fe1c3fff700 1 --2- 172.21.15.16:0/3250657168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1c4130260 0x7fe1c41c3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:09.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.943+0000 7fe1c3fff700 1 -- 172.21.15.16:0/3250657168 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b000a640 con 0x7fe1c412f8b0 2023-09-30T22:29:09.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.944+0000 7fe1c9303700 1 --2- 172.21.15.16:0/3250657168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1c4130260 0x7fe1c41c3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:09.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.944+0000 7fe1c3fff700 1 --2- 172.21.15.16:0/3250657168 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe1c412f8b0 0x7fe1c409bf10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0017000 tx=0x7fe1b00047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:09.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.944+0000 7fe1c17fa700 1 -- 172.21.15.16:0/3250657168 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b0003ab0 con 0x7fe1c412f8b0 2023-09-30T22:29:09.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.944+0000 7fe1c17fa700 1 -- 172.21.15.16:0/3250657168 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1b0003c30 con 0x7fe1c412f8b0 2023-09-30T22:29:09.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.944+0000 7fe1c17fa700 1 -- 172.21.15.16:0/3250657168 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b0021480 con 0x7fe1c412f8b0 2023-09-30T22:29:09.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.944+0000 7fe1cad66700 1 -- 172.21.15.16:0/3250657168 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1c409e620 con 0x7fe1c412f8b0 2023-09-30T22:29:09.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.944+0000 7fe1cad66700 1 -- 172.21.15.16:0/3250657168 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1c409c450 con 0x7fe1c412f8b0 2023-09-30T22:29:09.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.945+0000 7fe1cad66700 1 -- 172.21.15.16:0/3250657168 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe1c412c620 con 0x7fe1c412f8b0 2023-09-30T22:29:09.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.946+0000 7fe1c17fa700 1 -- 172.21.15.16:0/3250657168 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7fe1b0018400 con 0x7fe1c412f8b0 2023-09-30T22:29:09.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.946+0000 7fe1c8b02700 1 -- 172.21.15.16:0/3250657168 >> v1:172.21.15.16:6816/118285 conn(0x7fe1a802c1b0 legacy=0x7fe1a802e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/118285 2023-09-30T22:29:09.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.946+0000 7fe1c17fa700 1 -- 172.21.15.16:0/3250657168 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3922+0+0 (secure 0 0 0) 0x7fe1b00186b0 con 0x7fe1c412f8b0 2023-09-30T22:29:09.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.946+0000 7fe1c17fa700 1 -- 172.21.15.16:0/3250657168 --> v1:172.21.15.16:6808/118840 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe1a8038e10 con 0x7fe1a8035110 2023-09-30T22:29:09.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.946+0000 7fe1c17fa700 1 -- 172.21.15.16:0/3250657168 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_get_version_reply(handle=1 version=19) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe1b000f370 con 0x7fe1c412f8b0 2023-09-30T22:29:09.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.946+0000 7f2b1cc8d700 1 Processor -- start 2023-09-30T22:29:09.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.946+0000 7f2b1cc8d700 1 -- start start 2023-09-30T22:29:09.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.947+0000 7f2b1cc8d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2b1806f5d0 0x7f2b1812ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:09.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.947+0000 7f2b1cc8d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2b18135c20 0x7f2b18138000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:09.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.947+0000 7f2b1cc8d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b1806fa40 con 0x7f2b1812cf60 2023-09-30T22:29:09.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.947+0000 7f2b1cc8d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1812cd90 con 0x7f2b18135c20 2023-09-30T22:29:09.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.947+0000 7f2b1cc8d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2b18071780 con 0x7f2b1806f5d0 2023-09-30T22:29:09.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.947+0000 7f2b15d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2b1806f5d0 0x7f2b1812ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:09.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.947+0000 7f2b15d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2b1806f5d0 0x7f2b1812ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:58230/0 (socket says 172.21.15.16:58230) 2023-09-30T22:29:09.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.947+0000 7f2b15d9b700 1 -- 172.21.15.16:0/963062726 learned_addr learned my addr 172.21.15.16:0/963062726 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:09.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.948+0000 7f2b16d9d700 1 --2- 172.21.15.16:0/963062726 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2b18135c20 0x7f2b18138000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:09.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.948+0000 7f2b15d9b700 1 -- 172.21.15.16:0/963062726 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2b18135c20 msgr2=0x7f2b18138000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:09.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.948+0000 7f2b15d9b700 1 --2- 172.21.15.16:0/963062726 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2b18135c20 0x7f2b18138000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:09.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.948+0000 7f2b15d9b700 1 -- 172.21.15.16:0/963062726 >> v1:172.21.15.16:6789/0 conn(0x7f2b1812cf60 legacy=0x7f2b1812f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:09.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.948+0000 7f2b15d9b700 1 -- 172.21.15.16:0/963062726 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b18071a40 con 0x7f2b1806f5d0 2023-09-30T22:29:09.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.948+0000 7f2b15d9b700 1 --2- 172.21.15.16:0/963062726 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2b1806f5d0 0x7f2b1812ffc0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c009950 tx=0x7f2b0c009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54fe21ed9cd3b4ee server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:09.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.948+0000 7f2b1559a700 1 -- 172.21.15.16:0/963062726 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c009ea0 con 0x7f2b1806f5d0 2023-09-30T22:29:09.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.948+0000 7f2b1559a700 1 -- 172.21.15.16:0/963062726 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b0c004230 con 0x7f2b1806f5d0 2023-09-30T22:29:09.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.948+0000 7f2b1559a700 1 -- 172.21.15.16:0/963062726 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c0045a0 con 0x7f2b1806f5d0 2023-09-30T22:29:09.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.949+0000 7f2b1cc8d700 1 -- 172.21.15.16:0/963062726 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2b1806f5d0 msgr2=0x7f2b1812ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:09.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.949+0000 7f2b1cc8d700 1 --2- 172.21.15.16:0/963062726 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2b1806f5d0 0x7f2b1812ffc0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c009950 tx=0x7f2b0c009c60 comp rx=0 tx=0).stop 2023-09-30T22:29:09.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.949+0000 7f2b1cc8d700 1 -- 172.21.15.16:0/963062726 shutdown_connections 2023-09-30T22:29:09.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.949+0000 7f2b1cc8d700 1 --2- 172.21.15.16:0/963062726 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2b18135c20 0x7f2b18138000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:09.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.949+0000 7f2b1cc8d700 1 --2- 172.21.15.16:0/963062726 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2b1806f5d0 0x7f2b1812ffc0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:09.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.949+0000 7f2b1cc8d700 1 -- 172.21.15.16:0/963062726 >> 172.21.15.16:0/963062726 conn(0x7f2b180a2aa0 msgr2=0x7f2b180a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:09.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.949+0000 7f2b1cc8d700 1 -- 172.21.15.16:0/963062726 shutdown_connections 2023-09-30T22:29:09.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.949+0000 7f2b1cc8d700 1 -- 172.21.15.16:0/963062726 wait complete. 2023-09-30T22:29:09.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.950+0000 7f2b1cc8d700 1 Processor -- start 2023-09-30T22:29:09.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.951+0000 7f2b1cc8d700 1 -- start start 2023-09-30T22:29:09.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.951+0000 7f2b1cc8d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b1806f5d0 0x7f2b18137480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:09.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.951+0000 7f2b1cc8d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2b1812cf60 0x7f2b181379c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:09.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.951+0000 7f2b1cc8d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2b18135c20 0x7f2b18137f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:09.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.951+0000 7f2b1cc8d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b18139550 con 0x7f2b1806f5d0 2023-09-30T22:29:09.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.951+0000 7f2b1cc8d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1809a370 con 0x7f2b18135c20 2023-09-30T22:29:09.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.951+0000 7f2b1cc8d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2b18139080 con 0x7f2b1812cf60 2023-09-30T22:29:09.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.951+0000 7f2b15d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2b1812cf60 0x7f2b181379c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:09.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.951+0000 7f2b15d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2b1812cf60 0x7f2b181379c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:58236/0 (socket says 172.21.15.16:58236) 2023-09-30T22:29:09.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.951+0000 7f2b15d9b700 1 -- 172.21.15.16:0/180431037 learned_addr learned my addr 172.21.15.16:0/180431037 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:09.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.951+0000 7f2b1659c700 1 --2- 172.21.15.16:0/180431037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b1806f5d0 0x7f2b18137480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:09.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.952+0000 7f2b16d9d700 1 --2- 172.21.15.16:0/180431037 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2b18135c20 0x7f2b18137f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:09.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.952+0000 7f2b15d9b700 1 -- 172.21.15.16:0/180431037 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2b18135c20 msgr2=0x7f2b18137f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:09.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.952+0000 7f2b15d9b700 1 --2- 172.21.15.16:0/180431037 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2b18135c20 0x7f2b18137f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:09.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.952+0000 7f2b15d9b700 1 -- 172.21.15.16:0/180431037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b1806f5d0 msgr2=0x7f2b18137480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:09.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.952+0000 7f2b15d9b700 1 --2- 172.21.15.16:0/180431037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b1806f5d0 0x7f2b18137480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:09.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.952+0000 7f2b15d9b700 1 -- 172.21.15.16:0/180431037 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b0c009550 con 0x7f2b1812cf60 2023-09-30T22:29:09.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.952+0000 7f2b15d9b700 1 --2- 172.21.15.16:0/180431037 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2b1812cf60 0x7f2b181379c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2b18071910 tx=0x7f2b0c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:09.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.952+0000 7f2b06ffd700 1 -- 172.21.15.16:0/180431037 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c01d070 con 0x7f2b1812cf60 2023-09-30T22:29:09.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.952+0000 7f2b06ffd700 1 -- 172.21.15.16:0/180431037 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b0c004030 con 0x7f2b1812cf60 2023-09-30T22:29:09.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.952+0000 7f2b1cc8d700 1 -- 172.21.15.16:0/180431037 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b18129fa0 con 0x7f2b1812cf60 2023-09-30T22:29:09.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.952+0000 7f2b06ffd700 1 -- 172.21.15.16:0/180431037 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c003af0 con 0x7f2b1812cf60 2023-09-30T22:29:09.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.952+0000 7f2b1cc8d700 1 -- 172.21.15.16:0/180431037 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b18138e30 con 0x7f2b1812cf60 2023-09-30T22:29:09.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.953+0000 7f2b1cc8d700 1 -- 172.21.15.16:0/180431037 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2b18070ec0 con 0x7f2b1812cf60 2023-09-30T22:29:09.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.954+0000 7f2b06ffd700 1 -- 172.21.15.16:0/180431037 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7f2b0c003c50 con 0x7f2b1812cf60 2023-09-30T22:29:09.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.954+0000 7f2b06ffd700 1 -- 172.21.15.16:0/180431037 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3922+0+0 (secure 0 0 0) 0x7f2b0c03c890 con 0x7f2b1812cf60 2023-09-30T22:29:09.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.954+0000 7f2b1659c700 1 -- 172.21.15.16:0/180431037 >> v1:172.21.15.16:6816/118285 conn(0x7f2af802c1b0 legacy=0x7f2af802e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/118285 2023-09-30T22:29:09.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.954+0000 7f2b06ffd700 1 -- 172.21.15.16:0/180431037 --> v1:172.21.15.16:6800/118823 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2af8038e10 con 0x7f2af8035110 2023-09-30T22:29:09.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.954+0000 7f2b06ffd700 1 -- 172.21.15.16:0/180431037 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_get_version_reply(handle=1 version=19) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b0c017d70 con 0x7f2b1812cf60 2023-09-30T22:29:09.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.955+0000 7fe1c17fa700 1 -- 172.21.15.16:0/3250657168 <== osd.2 v1:172.21.15.16:6808/118840 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (unknown 0 0 1859947542) 0x7fe1a8038e10 con 0x7fe1a8035110 2023-09-30T22:29:09.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.955+0000 7f2b06ffd700 1 -- 172.21.15.16:0/180431037 <== osd.3 v1:172.21.15.16:6800/118823 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (unknown 0 0 1859947542) 0x7f2af8038e10 con 0x7f2af8035110 2023-09-30T22:29:09.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.971+0000 7fe1cad66700 1 -- 172.21.15.16:0/3250657168 --> v1:172.21.15.16:6808/118840 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe1c4072220 con 0x7fe1a8035110 2023-09-30T22:29:09.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.972+0000 7fe1c17fa700 1 -- 172.21.15.16:0/3250657168 <== osd.2 v1:172.21.15.16:6808/118840 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 2434951863) 0x7fe1bc00c120 con 0x7fe1a8035110 2023-09-30T22:29:09.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.972+0000 7fe1cad66700 1 -- 172.21.15.16:0/3250657168 >> v1:172.21.15.16:6808/118840 conn(0x7fe1a8035110 legacy=0x7fe1a80375a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:09.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.972+0000 7fe1cad66700 1 -- 172.21.15.16:0/3250657168 >> v1:172.21.15.16:6816/118285 conn(0x7fe1a802c1b0 legacy=0x7fe1a802e660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:09.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.972+0000 7fe1cad66700 1 -- 172.21.15.16:0/3250657168 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe1c412f8b0 msgr2=0x7fe1c409bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:09.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.972+0000 7fe1cad66700 1 --2- 172.21.15.16:0/3250657168 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe1c412f8b0 0x7fe1c409bf10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0017000 tx=0x7fe1b00047c0 comp rx=0 tx=0).stop 2023-09-30T22:29:09.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.972+0000 7fe1c9303700 1 -- 172.21.15.16:0/3250657168 reap_dead start 2023-09-30T22:29:09.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.974+0000 7f2b1cc8d700 1 -- 172.21.15.16:0/180431037 --> v1:172.21.15.16:6800/118823 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2b1812c620 con 0x7f2af8035110 2023-09-30T22:29:09.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.974+0000 7fe1cad66700 1 -- 172.21.15.16:0/3250657168 shutdown_connections 2023-09-30T22:29:09.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.974+0000 7fe1cad66700 1 -- 172.21.15.16:0/3250657168 >> 172.21.15.16:0/3250657168 conn(0x7fe1c40a2aa0 msgr2=0x7fe1c406e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:09.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.974+0000 7fe1cad66700 1 -- 172.21.15.16:0/3250657168 shutdown_connections 2023-09-30T22:29:09.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.974+0000 7fe1cad66700 1 -- 172.21.15.16:0/3250657168 wait complete. 2023-09-30T22:29:09.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.974+0000 7f2b06ffd700 1 -- 172.21.15.16:0/180431037 <== osd.3 v1:172.21.15.16:6800/118823 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 2188548419) 0x7f2b100083a0 con 0x7f2af8035110 2023-09-30T22:29:09.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.975+0000 7f2b1cc8d700 1 -- 172.21.15.16:0/180431037 >> v1:172.21.15.16:6800/118823 conn(0x7f2af8035110 legacy=0x7f2af80375a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:09.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.975+0000 7f2b1cc8d700 1 -- 172.21.15.16:0/180431037 >> v1:172.21.15.16:6816/118285 conn(0x7f2af802c1b0 legacy=0x7f2af802e660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:09.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.975+0000 7f2b1cc8d700 1 -- 172.21.15.16:0/180431037 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2b1812cf60 msgr2=0x7f2b181379c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:09.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.975+0000 7f2b1cc8d700 1 --2- 172.21.15.16:0/180431037 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2b1812cf60 0x7f2b181379c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2b18071910 tx=0x7f2b0c004750 comp rx=0 tx=0).stop 2023-09-30T22:29:09.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.975+0000 7f2b16d9d700 1 -- 172.21.15.16:0/180431037 reap_dead start 2023-09-30T22:29:09.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.975+0000 7f2b1cc8d700 1 -- 172.21.15.16:0/180431037 shutdown_connections 2023-09-30T22:29:09.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.975+0000 7f2b1cc8d700 1 -- 172.21.15.16:0/180431037 >> 172.21.15.16:0/180431037 conn(0x7f2b180a2aa0 msgr2=0x7f2b1806e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:09.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.977+0000 7f2b1cc8d700 1 -- 172.21.15.16:0/180431037 shutdown_connections 2023-09-30T22:29:09.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:09.977+0000 7f2b1cc8d700 1 -- 172.21.15.16:0/180431037 wait complete. 2023-09-30T22:29:09.991 INFO:teuthology.orchestra.run.smithi016.stdout:38654705696 2023-09-30T22:29:09.991 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:29:09.994 INFO:teuthology.orchestra.run.smithi016.stdout:38654705695 2023-09-30T22:29:09.994 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:29:10.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.005+0000 7fbeb6ebd700 1 Processor -- start 2023-09-30T22:29:10.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.005+0000 7fbeb6ebd700 1 -- start start 2023-09-30T22:29:10.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.006+0000 7fbeb6ebd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbeb006a770 0x7fbeb006ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:10.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.006+0000 7fbeb6ebd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbeb0131d10 0x7fbeb0139860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:10.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.006+0000 7fbeb6ebd700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbeb009ece0 con 0x7fbeb006b080 2023-09-30T22:29:10.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.006+0000 7fbeb6ebd700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb009e760 con 0x7fbeb0131d10 2023-09-30T22:29:10.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.006+0000 7fbeb6ebd700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb009e040 con 0x7fbeb006a770 2023-09-30T22:29:10.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.006+0000 7fbeb4c59700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbeb006a770 0x7fbeb006ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:10.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.006+0000 7fbeb4c59700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbeb006a770 0x7fbeb006ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:58246/0 (socket says 172.21.15.16:58246) 2023-09-30T22:29:10.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.006+0000 7fbeb4c59700 1 -- 172.21.15.16:0/2370005178 learned_addr learned my addr 172.21.15.16:0/2370005178 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:10.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.007+0000 7fbeb4c59700 1 -- 172.21.15.16:0/2370005178 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbeb0131d10 msgr2=0x7fbeb0139860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:10.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.007+0000 7fbeb4c59700 1 --2- 172.21.15.16:0/2370005178 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbeb0131d10 0x7fbeb0139860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.007+0000 7fbeb4c59700 1 -- 172.21.15.16:0/2370005178 >> v1:172.21.15.16:6789/0 conn(0x7fbeb006b080 legacy=0x7fbeb0131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:10.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.007+0000 7fbeb4c59700 1 -- 172.21.15.16:0/2370005178 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb009f630 con 0x7fbeb006a770 2023-09-30T22:29:10.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.007+0000 7fbeb4c59700 1 --2- 172.21.15.16:0/2370005178 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbeb006a770 0x7fbeb006ab40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fbea4009920 tx=0x7fbea4009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1141a528147d88ea server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:10.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.007+0000 7fbeaf7fe700 1 -- 172.21.15.16:0/2370005178 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea4004230 con 0x7fbeb006a770 2023-09-30T22:29:10.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.007+0000 7fbeaf7fe700 1 -- 172.21.15.16:0/2370005178 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbea4004390 con 0x7fbeb006a770 2023-09-30T22:29:10.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.007+0000 7fbeaf7fe700 1 -- 172.21.15.16:0/2370005178 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea4009e70 con 0x7fbeb006a770 2023-09-30T22:29:10.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.008+0000 7fbeb6ebd700 1 -- 172.21.15.16:0/2370005178 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbeb006a770 msgr2=0x7fbeb006ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:10.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.008+0000 7fbeb6ebd700 1 --2- 172.21.15.16:0/2370005178 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbeb006a770 0x7fbeb006ab40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fbea4009920 tx=0x7fbea4009c30 comp rx=0 tx=0).stop 2023-09-30T22:29:10.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.008+0000 7f4fddd5a700 1 Processor -- start 2023-09-30T22:29:10.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.009+0000 7f4fddd5a700 1 -- start start 2023-09-30T22:29:10.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.009+0000 7f4fddd5a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4fd809d6f0 0x7f4fd809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:10.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.009+0000 7f4fddd5a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4fd809e000 0x7f4fd8130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:10.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.009+0000 7f4fddd5a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fd809f130 con 0x7f4fd809e410 2023-09-30T22:29:10.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.009+0000 7f4fddd5a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd809f520 con 0x7f4fd809e000 2023-09-30T22:29:10.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.009+0000 7f4fddd5a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd809e930 con 0x7f4fd809d6f0 2023-09-30T22:29:10.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.009+0000 7fbeb6ebd700 1 -- 172.21.15.16:0/2370005178 shutdown_connections 2023-09-30T22:29:10.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.009+0000 7fbeb6ebd700 1 --2- 172.21.15.16:0/2370005178 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbeb0131d10 0x7fbeb0139860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.009+0000 7fbeb6ebd700 1 --2- 172.21.15.16:0/2370005178 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbeb006a770 0x7fbeb006ab40 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.009+0000 7fbeb6ebd700 1 -- 172.21.15.16:0/2370005178 >> 172.21.15.16:0/2370005178 conn(0x7fbeb0066f50 msgr2=0x7fbeb009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:10.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.009+0000 7f4fd7fff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4fd809e410 0x7f4fd8130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40278/0 (socket says 172.21.15.16:40278) 2023-09-30T22:29:10.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.009+0000 7f4fd7fff700 1 -- 172.21.15.16:0/3314489086 learned_addr learned my addr 172.21.15.16:0/3314489086 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:29:10.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.009+0000 7f4fd6ffd700 1 --2- 172.21.15.16:0/3314489086 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4fd809e000 0x7f4fd8130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:10.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.010+0000 7fbeb6ebd700 1 -- 172.21.15.16:0/2370005178 shutdown_connections 2023-09-30T22:29:10.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.010+0000 7f4fd67fc700 1 -- 172.21.15.16:0/3314489086 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1895181790 0 0) 0x7f4fd809f130 con 0x7f4fd809e410 2023-09-30T22:29:10.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.010+0000 7f4fd6ffd700 1 -- 172.21.15.16:0/3314489086 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4fd809d6f0 msgr2=0x7f4fd809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:10.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.010+0000 7f4fd6ffd700 1 --2- 172.21.15.16:0/3314489086 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4fd809d6f0 0x7f4fd809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.010+0000 7f4fd6ffd700 1 -- 172.21.15.16:0/3314489086 >> v1:172.21.15.16:6789/0 conn(0x7f4fd809e410 legacy=0x7f4fd8130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:10.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.010+0000 7f4fd6ffd700 1 -- 172.21.15.16:0/3314489086 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd809b2d0 con 0x7f4fd809e000 2023-09-30T22:29:10.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.010+0000 7f4fd6ffd700 1 --2- 172.21.15.16:0/3314489086 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4fd809e000 0x7f4fd8130660 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc009870 tx=0x7f4fcc009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26827614e4234a39 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:10.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.011+0000 7f4fd67fc700 1 -- 172.21.15.16:0/3314489086 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc011830 con 0x7f4fd809e000 2023-09-30T22:29:10.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.011+0000 7f4fd67fc700 1 -- 172.21.15.16:0/3314489086 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fcc011990 con 0x7f4fd809e000 2023-09-30T22:29:10.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.011+0000 7f4fd67fc700 1 -- 172.21.15.16:0/3314489086 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc011d20 con 0x7f4fd809e000 2023-09-30T22:29:10.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.010+0000 7fbeb6ebd700 1 -- 172.21.15.16:0/2370005178 wait complete. 2023-09-30T22:29:10.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.011+0000 7fbeb6ebd700 1 Processor -- start 2023-09-30T22:29:10.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.011+0000 7fbeb6ebd700 1 -- start start 2023-09-30T22:29:10.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.012+0000 7f4fddd5a700 1 -- 172.21.15.16:0/3314489086 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4fd809e000 msgr2=0x7f4fd8130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:10.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.012+0000 7f4fddd5a700 1 --2- 172.21.15.16:0/3314489086 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4fd809e000 0x7f4fd8130660 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc009870 tx=0x7f4fcc009b80 comp rx=0 tx=0).stop 2023-09-30T22:29:10.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.012+0000 7f4fddd5a700 1 -- 172.21.15.16:0/3314489086 shutdown_connections 2023-09-30T22:29:10.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.012+0000 7f4fddd5a700 1 --2- 172.21.15.16:0/3314489086 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4fd809d6f0 0x7f4fd809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.012+0000 7f4fddd5a700 1 --2- 172.21.15.16:0/3314489086 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4fd809e000 0x7f4fd8130660 secure :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc009870 tx=0x7f4fcc009b80 comp rx=0 tx=0).stop 2023-09-30T22:29:10.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.012+0000 7f4fddd5a700 1 -- 172.21.15.16:0/3314489086 >> 172.21.15.16:0/3314489086 conn(0x7f4fd8066f50 msgr2=0x7f4fd809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:10.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.012+0000 7fbeb6ebd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbeb006a770 0x7fbeb011c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:10.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.012+0000 7fbeb6ebd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbeb006b080 0x7fbeb011c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:10.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.012+0000 7fbeb6ebd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbeb0131d10 0x7fbeb011cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:10.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.012+0000 7fbeb6ebd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb011fc00 con 0x7fbeb006b080 2023-09-30T22:29:10.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.012+0000 7fbeb6ebd700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb011e070 con 0x7fbeb0131d10 2023-09-30T22:29:10.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.012+0000 7fbeb6ebd700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb0131330 con 0x7fbeb006a770 2023-09-30T22:29:10.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.012+0000 7fbeb545a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbeb0131d10 0x7fbeb011cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:10.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.012+0000 7fbeaffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbeb006b080 0x7fbeb011c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:10.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.012+0000 7fbeb545a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbeb0131d10 0x7fbeb011cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:53840/0 (socket says 172.21.15.16:53840) 2023-09-30T22:29:10.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.012+0000 7fbeb545a700 1 -- 172.21.15.16:0/3897529350 learned_addr learned my addr 172.21.15.16:0/3897529350 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:10.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.012+0000 7fbeb4c59700 1 --2- 172.21.15.16:0/3897529350 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbeb006a770 0x7fbeb011c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:10.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.012+0000 7fbeb545a700 1 -- 172.21.15.16:0/3897529350 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbeb006a770 msgr2=0x7fbeb011c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:10.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.013+0000 7fbeb545a700 1 --2- 172.21.15.16:0/3897529350 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbeb006a770 0x7fbeb011c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.013+0000 7fbeb545a700 1 -- 172.21.15.16:0/3897529350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbeb006b080 msgr2=0x7fbeb011c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:10.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.013+0000 7fbeb545a700 1 --2- 172.21.15.16:0/3897529350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbeb006b080 0x7fbeb011c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.013+0000 7fbeb545a700 1 -- 172.21.15.16:0/3897529350 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbea4009550 con 0x7fbeb0131d10 2023-09-30T22:29:10.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.013+0000 7fbeaffff700 1 --2- 172.21.15.16:0/3897529350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbeb006b080 0x7fbeb011c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:10.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.013+0000 7fbeb545a700 1 --2- 172.21.15.16:0/3897529350 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbeb0131d10 0x7fbeb011cf10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbea800b970 tx=0x7fbea800bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:10.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.013+0000 7f4fddd5a700 1 -- 172.21.15.16:0/3314489086 shutdown_connections 2023-09-30T22:29:10.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.013+0000 7f4fddd5a700 1 -- 172.21.15.16:0/3314489086 wait complete. 2023-09-30T22:29:10.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.013+0000 7fbead7fa700 1 -- 172.21.15.16:0/3897529350 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea800c930 con 0x7fbeb0131d10 2023-09-30T22:29:10.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.013+0000 7fbead7fa700 1 -- 172.21.15.16:0/3897529350 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbea800cab0 con 0x7fbeb0131d10 2023-09-30T22:29:10.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.013+0000 7fbeb6ebd700 1 -- 172.21.15.16:0/3897529350 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbeb011f940 con 0x7fbeb0131d10 2023-09-30T22:29:10.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.013+0000 7fbead7fa700 1 -- 172.21.15.16:0/3897529350 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea800ce60 con 0x7fbeb0131d10 2023-09-30T22:29:10.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.013+0000 7fbeb6ebd700 1 -- 172.21.15.16:0/3897529350 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbeb0130e70 con 0x7fbeb0131d10 2023-09-30T22:29:10.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.014+0000 7f4fddd5a700 1 Processor -- start 2023-09-30T22:29:10.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.014+0000 7f4fddd5a700 1 -- start start 2023-09-30T22:29:10.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.015+0000 7f4fddd5a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fd809d6f0 0x7f4fd811e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:10.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.015+0000 7fbead7fa700 1 -- 172.21.15.16:0/3897529350 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7fbea8007370 con 0x7fbeb0131d10 2023-09-30T22:29:10.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.015+0000 7fbead7fa700 1 -- 172.21.15.16:0/3897529350 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3922+0+0 (secure 0 0 0) 0x7fbea8006e40 con 0x7fbeb0131d10 2023-09-30T22:29:10.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.015+0000 7f4fddd5a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4fd809e410 0x7f4fd811e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:10.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.015+0000 7f4fddd5a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4fd811eb00 0x7f4fd81ca490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:10.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.015+0000 7f4fddd5a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd8121880 con 0x7f4fd809d6f0 2023-09-30T22:29:10.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.015+0000 7f4fddd5a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd8120520 con 0x7f4fd811eb00 2023-09-30T22:29:10.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.015+0000 7f4fddd5a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd80a1d00 con 0x7f4fd809e410 2023-09-30T22:29:10.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.015+0000 7f4fd7fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4fd811eb00 0x7f4fd81ca490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:10.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.015+0000 7f4fd7fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4fd811eb00 0x7f4fd81ca490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:53844/0 (socket says 172.21.15.16:53844) 2023-09-30T22:29:10.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.015+0000 7f4fd7fff700 1 -- 172.21.15.16:0/1442437560 learned_addr learned my addr 172.21.15.16:0/1442437560 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:10.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.016+0000 7fbeb4c59700 1 -- 172.21.15.16:0/3897529350 >> v1:172.21.15.16:6816/118285 conn(0x7fbe9402c1b0 legacy=0x7fbe9402e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/118285 2023-09-30T22:29:10.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.016+0000 7f4fd7fff700 1 -- 172.21.15.16:0/1442437560 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4fd809e410 msgr2=0x7f4fd811e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:10.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.016+0000 7f4fd7fff700 1 --2- 172.21.15.16:0/1442437560 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4fd809e410 0x7f4fd811e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.016+0000 7f4fd7fff700 1 -- 172.21.15.16:0/1442437560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fd809d6f0 msgr2=0x7f4fd811e080 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:10.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.016+0000 7f4fd7fff700 1 --2- 172.21.15.16:0/1442437560 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4fd809d6f0 0x7f4fd811e080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.016+0000 7f4fd7fff700 1 -- 172.21.15.16:0/1442437560 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fcc0094a0 con 0x7f4fd811eb00 2023-09-30T22:29:10.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.016+0000 7f4fd7fff700 1 --2- 172.21.15.16:0/1442437560 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4fd811eb00 0x7f4fd81ca490 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4fd000cc70 tx=0x7f4fd000a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:10.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.016+0000 7f4fbffff700 1 -- 172.21.15.16:0/1442437560 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd000ada0 con 0x7f4fd811eb00 2023-09-30T22:29:10.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.016+0000 7f4fbffff700 1 -- 172.21.15.16:0/1442437560 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fd000ce70 con 0x7f4fd811eb00 2023-09-30T22:29:10.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.016+0000 7f4fbffff700 1 -- 172.21.15.16:0/1442437560 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd00072d0 con 0x7f4fd811eb00 2023-09-30T22:29:10.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.016+0000 7f4fddd5a700 1 -- 172.21.15.16:0/1442437560 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fd8121b60 con 0x7f4fd811eb00 2023-09-30T22:29:10.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.016+0000 7f4fddd5a700 1 -- 172.21.15.16:0/1442437560 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fd81212f0 con 0x7f4fd811eb00 2023-09-30T22:29:10.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.017+0000 7f4fddd5a700 1 -- 172.21.15.16:0/1442437560 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4fd809c000 con 0x7f4fd811eb00 2023-09-30T22:29:10.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.018+0000 7fbeb6ebd700 1 -- 172.21.15.16:0/3897529350 --> v1:172.21.15.16:6812/118832 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbeb00621a0 con 0x7fbeb012fd40 2023-09-30T22:29:10.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.018+0000 7f4fbffff700 1 -- 172.21.15.16:0/1442437560 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7f4fd0007430 con 0x7f4fd811eb00 2023-09-30T22:29:10.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.018+0000 7f4fbffff700 1 -- 172.21.15.16:0/1442437560 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3922+0+0 (secure 0 0 0) 0x7f4fd003bce0 con 0x7f4fd811eb00 2023-09-30T22:29:10.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.018+0000 7f4fd77fe700 1 -- 172.21.15.16:0/1442437560 >> v1:172.21.15.16:6816/118285 conn(0x7f4fc402c1b0 legacy=0x7f4fc402e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/118285 2023-09-30T22:29:10.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.018+0000 7f4fbffff700 1 -- 172.21.15.16:0/1442437560 --> v1:172.21.15.16:6804/118824 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4fc4038f00 con 0x7f4fc4035200 2023-09-30T22:29:10.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.018+0000 7f4fbffff700 1 -- 172.21.15.16:0/1442437560 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_get_version_reply(handle=1 version=19) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4fd003c010 con 0x7f4fd811eb00 2023-09-30T22:29:10.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.019+0000 7fbead7fa700 1 -- 172.21.15.16:0/3897529350 <== osd.1 v1:172.21.15.16:6812/118832 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (unknown 0 0 1859947542) 0x7fbeb00621a0 con 0x7fbeb012fd40 2023-09-30T22:29:10.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.020+0000 7f4fbffff700 1 -- 172.21.15.16:0/1442437560 <== osd.0 v1:172.21.15.16:6804/118824 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (unknown 0 0 1859947542) 0x7f4fc4038f00 con 0x7f4fc4035200 2023-09-30T22:29:10.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.038+0000 7f4fddd5a700 1 -- 172.21.15.16:0/1442437560 --> v1:172.21.15.16:6804/118824 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4fd809c450 con 0x7f4fc4035200 2023-09-30T22:29:10.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.039+0000 7f4fbffff700 1 -- 172.21.15.16:0/1442437560 <== osd.0 v1:172.21.15.16:6804/118824 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 2188548419) 0x7f4fcc011630 con 0x7f4fc4035200 2023-09-30T22:29:10.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.039+0000 7f4fddd5a700 1 -- 172.21.15.16:0/1442437560 >> v1:172.21.15.16:6804/118824 conn(0x7f4fc4035200 legacy=0x7f4fc4037690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:10.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.039+0000 7f4fddd5a700 1 -- 172.21.15.16:0/1442437560 >> v1:172.21.15.16:6816/118285 conn(0x7f4fc402c1b0 legacy=0x7f4fc402e660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:10.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.039+0000 7f4fddd5a700 1 -- 172.21.15.16:0/1442437560 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4fd811eb00 msgr2=0x7f4fd81ca490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:10.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.039+0000 7f4fddd5a700 1 --2- 172.21.15.16:0/1442437560 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4fd811eb00 0x7f4fd81ca490 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4fd000cc70 tx=0x7f4fd000a340 comp rx=0 tx=0).stop 2023-09-30T22:29:10.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.040+0000 7f4fd7fff700 1 -- 172.21.15.16:0/1442437560 reap_dead start 2023-09-30T22:29:10.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.040+0000 7f4fddd5a700 1 -- 172.21.15.16:0/1442437560 shutdown_connections 2023-09-30T22:29:10.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.040+0000 7f4fddd5a700 1 -- 172.21.15.16:0/1442437560 >> 172.21.15.16:0/1442437560 conn(0x7f4fd8066f50 msgr2=0x7f4fd8071a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:10.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.040+0000 7f4fddd5a700 1 -- 172.21.15.16:0/1442437560 shutdown_connections 2023-09-30T22:29:10.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.040+0000 7f4fddd5a700 1 -- 172.21.15.16:0/1442437560 wait complete. 2023-09-30T22:29:10.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.047+0000 7fbeb6ebd700 1 -- 172.21.15.16:0/3897529350 --> v1:172.21.15.16:6812/118832 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbeb009c890 con 0x7fbeb012fd40 2023-09-30T22:29:10.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.050+0000 7fbead7fa700 1 -- 172.21.15.16:0/3897529350 <== osd.1 v1:172.21.15.16:6812/118832 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 2188548419) 0x7fbe9c003680 con 0x7fbeb012fd40 2023-09-30T22:29:10.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.054+0000 7fbeb6ebd700 1 -- 172.21.15.16:0/3897529350 >> v1:172.21.15.16:6812/118832 conn(0x7fbeb012fd40 legacy=0x7fbeb0130110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:10.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.054+0000 7fbeb6ebd700 1 -- 172.21.15.16:0/3897529350 >> v1:172.21.15.16:6816/118285 conn(0x7fbe9402c1b0 legacy=0x7fbe9402e660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:10.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.054+0000 7fbeb6ebd700 1 -- 172.21.15.16:0/3897529350 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbeb0131d10 msgr2=0x7fbeb011cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:10.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.054+0000 7fbeb6ebd700 1 --2- 172.21.15.16:0/3897529350 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbeb0131d10 0x7fbeb011cf10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbea800b970 tx=0x7fbea800bc80 comp rx=0 tx=0).stop 2023-09-30T22:29:10.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.054+0000 7fbeb545a700 1 -- 172.21.15.16:0/3897529350 reap_dead start 2023-09-30T22:29:10.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.054+0000 7fbeb6ebd700 1 -- 172.21.15.16:0/3897529350 shutdown_connections 2023-09-30T22:29:10.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.055+0000 7fbeb6ebd700 1 -- 172.21.15.16:0/3897529350 >> 172.21.15.16:0/3897529350 conn(0x7fbeb0066f50 msgr2=0x7fbeb00a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:10.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.055+0000 7fbeb6ebd700 1 -- 172.21.15.16:0/3897529350 shutdown_connections 2023-09-30T22:29:10.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.055+0000 7fbeb6ebd700 1 -- 172.21.15.16:0/3897529350 wait complete. 2023-09-30T22:29:10.058 INFO:teuthology.orchestra.run.smithi016.stdout:38654705695 2023-09-30T22:29:10.058 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:29:10.076 INFO:teuthology.orchestra.run.smithi016.stdout:38654705695 2023-09-30T22:29:10.076 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:29:10.092 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:29:10.091+0000 7f90c460e240 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-09-30T22:29:10.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.164+0000 7f29a976e700 1 Processor -- start 2023-09-30T22:29:10.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.164+0000 7f29a976e700 1 -- start start 2023-09-30T22:29:10.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.164+0000 7f29a976e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f29a409e1d0 0x7f29a4135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:10.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.164+0000 7f29a976e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f29a4135e50 0x7f29a4138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:10.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.164+0000 7f29a976e700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29a409f130 con 0x7f29a409d6f0 2023-09-30T22:29:10.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.164+0000 7f29a976e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29a409f520 con 0x7f29a4135e50 2023-09-30T22:29:10.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.164+0000 7f29a976e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f29a409e930 con 0x7f29a409e1d0 2023-09-30T22:29:10.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.165+0000 7f29a37fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f29a4135e50 0x7f29a4138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:10.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.165+0000 7f29a37fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f29a4135e50 0x7f29a4138230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:53856/0 (socket says 172.21.15.16:53856) 2023-09-30T22:29:10.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.165+0000 7f29a37fe700 1 -- 172.21.15.16:0/2780001275 learned_addr learned my addr 172.21.15.16:0/2780001275 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:10.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.165+0000 7f29a27fc700 1 --2- 172.21.15.16:0/2780001275 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f29a409e1d0 0x7f29a4135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:10.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.165+0000 7f29a37fe700 1 -- 172.21.15.16:0/2780001275 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f29a409e1d0 msgr2=0x7f29a4135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:10.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.165+0000 7f29a37fe700 1 --2- 172.21.15.16:0/2780001275 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f29a409e1d0 0x7f29a4135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.165+0000 7f29a37fe700 1 -- 172.21.15.16:0/2780001275 >> v1:172.21.15.16:6789/0 conn(0x7f29a409d6f0 legacy=0x7f29a409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:10.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.165+0000 7f29a37fe700 1 -- 172.21.15.16:0/2780001275 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a409b2d0 con 0x7f29a4135e50 2023-09-30T22:29:10.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.165+0000 7f29a37fe700 1 --2- 172.21.15.16:0/2780001275 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f29a4135e50 0x7f29a4138230 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f299c00bf20 tx=0x7f299c0093b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab5558b13310f319 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:10.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.166+0000 7f29a1ffb700 1 -- 172.21.15.16:0/2780001275 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f299c00b4e0 con 0x7f29a4135e50 2023-09-30T22:29:10.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.166+0000 7f29a1ffb700 1 -- 172.21.15.16:0/2780001275 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f299c00c950 con 0x7f29a4135e50 2023-09-30T22:29:10.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.166+0000 7f29a1ffb700 1 -- 172.21.15.16:0/2780001275 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f299c00cd00 con 0x7f29a4135e50 2023-09-30T22:29:10.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.167+0000 7f29a976e700 1 -- 172.21.15.16:0/2780001275 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f29a4135e50 msgr2=0x7f29a4138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:10.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.167+0000 7f29a976e700 1 --2- 172.21.15.16:0/2780001275 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f29a4135e50 0x7f29a4138230 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f299c00bf20 tx=0x7f299c0093b0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.167+0000 7f29a976e700 1 -- 172.21.15.16:0/2780001275 shutdown_connections 2023-09-30T22:29:10.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.167+0000 7f29a976e700 1 --2- 172.21.15.16:0/2780001275 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f29a4135e50 0x7f29a4138230 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.167+0000 7f29a976e700 1 --2- 172.21.15.16:0/2780001275 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f29a409e1d0 0x7f29a4135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.167+0000 7f29a976e700 1 -- 172.21.15.16:0/2780001275 >> 172.21.15.16:0/2780001275 conn(0x7f29a4066f50 msgr2=0x7f29a409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:10.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.167+0000 7f29a976e700 1 -- 172.21.15.16:0/2780001275 shutdown_connections 2023-09-30T22:29:10.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.167+0000 7f29a976e700 1 -- 172.21.15.16:0/2780001275 wait complete. 2023-09-30T22:29:10.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.169+0000 7f29a976e700 1 Processor -- start 2023-09-30T22:29:10.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.169+0000 7f29a976e700 1 -- start start 2023-09-30T22:29:10.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.170+0000 7f29a976e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f29a409d6f0 0x7f29a41c43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:10.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.170+0000 7f29a976e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f29a409e1d0 0x7f29a41c4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:10.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.170+0000 7f29a976e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29a4135e50 0x7f29a4119480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:10.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.170+0000 7f29a976e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29a411ca40 con 0x7f29a4135e50 2023-09-30T22:29:10.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.170+0000 7f29a976e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29a40a1d00 con 0x7f29a409e1d0 2023-09-30T22:29:10.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.170+0000 7f29a976e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f29a411b930 con 0x7f29a409d6f0 2023-09-30T22:29:10.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.170+0000 7f29a27fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f29a409e1d0 0x7f29a41c4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:10.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.170+0000 7f29a27fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f29a409e1d0 0x7f29a41c4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:53872/0 (socket says 172.21.15.16:53872) 2023-09-30T22:29:10.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.170+0000 7f29a27fc700 1 -- 172.21.15.16:0/628245425 learned_addr learned my addr 172.21.15.16:0/628245425 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:10.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.170+0000 7f29a37fe700 1 --2- 172.21.15.16:0/628245425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29a4135e50 0x7f29a4119480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:10.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.170+0000 7f29a2ffd700 1 --2- 172.21.15.16:0/628245425 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f29a409d6f0 0x7f29a41c43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:10.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.170+0000 7f29a27fc700 1 -- 172.21.15.16:0/628245425 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f29a409d6f0 msgr2=0x7f29a41c43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:10.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.170+0000 7f29a27fc700 1 --2- 172.21.15.16:0/628245425 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f29a409d6f0 0x7f29a41c43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.170+0000 7f29a27fc700 1 -- 172.21.15.16:0/628245425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29a4135e50 msgr2=0x7f29a4119480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:10.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.170+0000 7f29a27fc700 1 --2- 172.21.15.16:0/628245425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29a4135e50 0x7f29a4119480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.170+0000 7f29a27fc700 1 -- 172.21.15.16:0/628245425 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f299c00bb50 con 0x7f29a409e1d0 2023-09-30T22:29:10.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.171+0000 7f29a37fe700 1 --2- 172.21.15.16:0/628245425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29a4135e50 0x7f29a4119480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:29:10.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.171+0000 7f29a27fc700 1 --2- 172.21.15.16:0/628245425 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f29a409e1d0 0x7f29a41c4930 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f299800b390 tx=0x7f299800b750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:10.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.171+0000 7f298b7fe700 1 -- 172.21.15.16:0/628245425 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f299801e410 con 0x7f29a409e1d0 2023-09-30T22:29:10.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.171+0000 7f298b7fe700 1 -- 172.21.15.16:0/628245425 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f299801e570 con 0x7f29a409e1d0 2023-09-30T22:29:10.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.171+0000 7f298b7fe700 1 -- 172.21.15.16:0/628245425 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f299801e8e0 con 0x7f29a409e1d0 2023-09-30T22:29:10.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.172+0000 7f29a976e700 1 -- 172.21.15.16:0/628245425 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29a41c4e70 con 0x7f29a409e1d0 2023-09-30T22:29:10.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.172+0000 7f29a976e700 1 -- 172.21.15.16:0/628245425 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29a411b170 con 0x7f29a409e1d0 2023-09-30T22:29:10.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.174+0000 7f29a976e700 1 -- 172.21.15.16:0/628245425 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29a4074ea0 con 0x7f29a409e1d0 2023-09-30T22:29:10.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.175+0000 7f298b7fe700 1 -- 172.21.15.16:0/628245425 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7f299801ea40 con 0x7f29a409e1d0 2023-09-30T22:29:10.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.175+0000 7f298b7fe700 1 -- 172.21.15.16:0/628245425 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3922+0+0 (secure 0 0 0) 0x7f2998043ae0 con 0x7f29a409e1d0 2023-09-30T22:29:10.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.175+0000 7f29a2ffd700 1 -- 172.21.15.16:0/628245425 >> v1:172.21.15.16:6816/118285 conn(0x7f299002c200 legacy=0x7f299002e6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/118285 2023-09-30T22:29:10.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.179+0000 7f298b7fe700 1 -- 172.21.15.16:0/628245425 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7f2998025b40 con 0x7f29a409e1d0 2023-09-30T22:29:10.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.184+0000 7fcbf1987700 1 Processor -- start 2023-09-30T22:29:10.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.184+0000 7fcbf1987700 1 -- start start 2023-09-30T22:29:10.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.185+0000 7fcbf1987700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcbec06f5d0 0x7fcbec12ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:10.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.185+0000 7fcbf1987700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcbec135c20 0x7fcbec138000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:10.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.185+0000 7fcbf1987700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbec06fa40 con 0x7fcbec12cf60 2023-09-30T22:29:10.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.185+0000 7fcbf1987700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbec12cd90 con 0x7fcbec135c20 2023-09-30T22:29:10.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.185+0000 7fcbf1987700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcbec071780 con 0x7fcbec06f5d0 2023-09-30T22:29:10.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.185+0000 7fcbea7fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcbec06f5d0 0x7fcbec12ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:10.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.186+0000 7fcbea7fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcbec06f5d0 0x7fcbec12ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:58306/0 (socket says 172.21.15.16:58306) 2023-09-30T22:29:10.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.186+0000 7fcbea7fc700 1 -- 172.21.15.16:0/1170618869 learned_addr learned my addr 172.21.15.16:0/1170618869 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:10.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.186+0000 7fcbea7fc700 1 -- 172.21.15.16:0/1170618869 >> v2:172.21.15.16:3300/0 conn(0x7fcbec135c20 msgr2=0x7fcbec138000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:10.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.186+0000 7fcbea7fc700 1 --2- 172.21.15.16:0/1170618869 >> v2:172.21.15.16:3300/0 conn(0x7fcbec135c20 0x7fcbec138000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.186+0000 7fcbea7fc700 1 -- 172.21.15.16:0/1170618869 >> v1:172.21.15.16:6789/0 conn(0x7fcbec12cf60 legacy=0x7fcbec12f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:10.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.186+0000 7fcbea7fc700 1 -- 172.21.15.16:0/1170618869 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbec071a40 con 0x7fcbec06f5d0 2023-09-30T22:29:10.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.186+0000 7fcbea7fc700 1 --2- 172.21.15.16:0/1170618869 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcbec06f5d0 0x7fcbec12ffc0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcbe00098f0 tx=0x7fcbe0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=333c4a1ba95a364d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:10.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.186+0000 7fcbe9ffb700 1 -- 172.21.15.16:0/1170618869 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe0004230 con 0x7fcbec06f5d0 2023-09-30T22:29:10.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.186+0000 7fcbe9ffb700 1 -- 172.21.15.16:0/1170618869 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbe0004390 con 0x7fcbec06f5d0 2023-09-30T22:29:10.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.186+0000 7fcbe9ffb700 1 -- 172.21.15.16:0/1170618869 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe00036a0 con 0x7fcbec06f5d0 2023-09-30T22:29:10.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.188+0000 7fcbf1987700 1 -- 172.21.15.16:0/1170618869 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcbec06f5d0 msgr2=0x7fcbec12ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:10.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.188+0000 7fcbf1987700 1 --2- 172.21.15.16:0/1170618869 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcbec06f5d0 0x7fcbec12ffc0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcbe00098f0 tx=0x7fcbe0009c00 comp rx=0 tx=0).stop 2023-09-30T22:29:10.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.188+0000 7fcbf1987700 1 -- 172.21.15.16:0/1170618869 shutdown_connections 2023-09-30T22:29:10.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.188+0000 7fcbf1987700 1 --2- 172.21.15.16:0/1170618869 >> v2:172.21.15.16:3300/0 conn(0x7fcbec135c20 0x7fcbec138000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.188+0000 7fcbf1987700 1 --2- 172.21.15.16:0/1170618869 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcbec06f5d0 0x7fcbec12ffc0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.188+0000 7fcbf1987700 1 -- 172.21.15.16:0/1170618869 >> 172.21.15.16:0/1170618869 conn(0x7fcbec0a2aa0 msgr2=0x7fcbec0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:10.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.188+0000 7fcbf1987700 1 -- 172.21.15.16:0/1170618869 shutdown_connections 2023-09-30T22:29:10.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.188+0000 7fcbf1987700 1 -- 172.21.15.16:0/1170618869 wait complete. 2023-09-30T22:29:10.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.190+0000 7fcbf1987700 1 Processor -- start 2023-09-30T22:29:10.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.190+0000 7fcbf1987700 1 -- start start 2023-09-30T22:29:10.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.191+0000 7fcbf1987700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcbec06f5d0 0x7fcbec13a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:10.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.191+0000 7fcbf1987700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbec12cf60 0x7fcbec133e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:10.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.191+0000 7fcbf1987700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcbec135c20 0x7fcbec1343d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:10.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.191+0000 7fcbf1987700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbec13b480 con 0x7fcbec12cf60 2023-09-30T22:29:10.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.191+0000 7fcbf1987700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbec13b290 con 0x7fcbec135c20 2023-09-30T22:29:10.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.191+0000 7fcbf1987700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcbec132730 con 0x7fcbec06f5d0 2023-09-30T22:29:10.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.191+0000 7fcbea7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbec12cf60 0x7fcbec133e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:10.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.191+0000 7fcbea7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbec12cf60 0x7fcbec133e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46844/0 (socket says 172.21.15.16:46844) 2023-09-30T22:29:10.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.191+0000 7fcbea7fc700 1 -- 172.21.15.16:0/408704217 learned_addr learned my addr 172.21.15.16:0/408704217 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:10.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.191+0000 7fcbeaffd700 1 --2- 172.21.15.16:0/408704217 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcbec06f5d0 0x7fcbec13a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:10.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.191+0000 7fcbeaffd700 1 -- 172.21.15.16:0/408704217 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcbec135c20 msgr2=0x7fcbec1343d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:10.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.191+0000 7fcbeaffd700 1 --2- 172.21.15.16:0/408704217 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcbec135c20 0x7fcbec1343d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.191+0000 7fcbeaffd700 1 -- 172.21.15.16:0/408704217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbec12cf60 msgr2=0x7fcbec133e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:10.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.191+0000 7fcbeaffd700 1 --2- 172.21.15.16:0/408704217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbec12cf60 0x7fcbec133e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.191+0000 7fcbeaffd700 1 -- 172.21.15.16:0/408704217 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbe0009550 con 0x7fcbec06f5d0 2023-09-30T22:29:10.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.192+0000 7fcbeaffd700 1 --2- 172.21.15.16:0/408704217 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcbec06f5d0 0x7fcbec13a690 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc00c410 tx=0x7fcbdc00c720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:10.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.192+0000 7fcbd3fff700 1 -- 172.21.15.16:0/408704217 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc0054e0 con 0x7fcbec06f5d0 2023-09-30T22:29:10.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.192+0000 7fcbd3fff700 1 -- 172.21.15.16:0/408704217 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbdc00ce90 con 0x7fcbec06f5d0 2023-09-30T22:29:10.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.192+0000 7fcbd3fff700 1 -- 172.21.15.16:0/408704217 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc0038c0 con 0x7fcbec06f5d0 2023-09-30T22:29:10.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.192+0000 7fcbf1987700 1 -- 172.21.15.16:0/408704217 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbec129fa0 con 0x7fcbec06f5d0 2023-09-30T22:29:10.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.192+0000 7fcbf1987700 1 -- 172.21.15.16:0/408704217 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbec129990 con 0x7fcbec06f5d0 2023-09-30T22:29:10.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.193+0000 7fcbd3fff700 1 -- 172.21.15.16:0/408704217 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7fcbdc003a20 con 0x7fcbec06f5d0 2023-09-30T22:29:10.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.194+0000 7fcbf1987700 1 -- 172.21.15.16:0/408704217 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbec12bb40 con 0x7fcbec06f5d0 2023-09-30T22:29:10.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.194+0000 7fcbea7fc700 1 -- 172.21.15.16:0/408704217 >> v1:172.21.15.16:6816/118285 conn(0x7fcbd802c1b0 legacy=0x7fcbd802e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/118285 2023-09-30T22:29:10.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.194+0000 7fcbd3fff700 1 -- 172.21.15.16:0/408704217 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3922+0+0 (secure 0 0 0) 0x7fcbdc03f170 con 0x7fcbec06f5d0 2023-09-30T22:29:10.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.202+0000 7fcbd3fff700 1 -- 172.21.15.16:0/408704217 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7fcbdc017020 con 0x7fcbec06f5d0 2023-09-30T22:29:10.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.260+0000 7f831fefc700 1 Processor -- start 2023-09-30T22:29:10.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.261+0000 7f831fefc700 1 -- start start 2023-09-30T22:29:10.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.261+0000 7f831fefc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f831809e930 0x7f831809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:10.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.261+0000 7f831fefc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f831809f240 0x7f83181317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:10.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.261+0000 7f831fefc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f831809f650 0x7f8318131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:10.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.261+0000 7f831fefc700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8318070940 con 0x7f831809e930 2023-09-30T22:29:10.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.261+0000 7f831fefc700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f831806fd60 con 0x7f831809f650 2023-09-30T22:29:10.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.261+0000 7f831fefc700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f831806fa80 con 0x7f831809f240 2023-09-30T22:29:10.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.262+0000 7f831d497700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f831809f240 0x7f83181317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:10.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.262+0000 7f831d497700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f831809f240 0x7f83181317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:58314/0 (socket says 172.21.15.16:58314) 2023-09-30T22:29:10.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.262+0000 7f831d497700 1 -- 172.21.15.16:0/4096549932 learned_addr learned my addr 172.21.15.16:0/4096549932 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:10.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.262+0000 7f831e499700 1 --2- 172.21.15.16:0/4096549932 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f831809f650 0x7f8318131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:10.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.262+0000 7f831d497700 1 -- 172.21.15.16:0/4096549932 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f831809f650 msgr2=0x7f8318131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:10.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.262+0000 7f831d497700 1 --2- 172.21.15.16:0/4096549932 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f831809f650 0x7f8318131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.262+0000 7f831d497700 1 -- 172.21.15.16:0/4096549932 >> v2:172.21.15.16:3300/0 conn(0x7f831809e930 msgr2=0x7f831809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:10.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.262+0000 7f831d497700 1 --2- 172.21.15.16:0/4096549932 >> v2:172.21.15.16:3300/0 conn(0x7f831809e930 0x7f831809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.262+0000 7f831d497700 1 -- 172.21.15.16:0/4096549932 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f831809fb60 con 0x7f831809f240 2023-09-30T22:29:10.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.262+0000 7f831d497700 1 --2- 172.21.15.16:0/4096549932 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f831809f240 0x7f83181317f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f83080131e0 tx=0x7f83080134f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1873e1ec5e53960 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:10.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.263+0000 7f831cc96700 1 -- 172.21.15.16:0/4096549932 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8308016070 con 0x7f831809f240 2023-09-30T22:29:10.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.263+0000 7f831cc96700 1 -- 172.21.15.16:0/4096549932 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8308013d80 con 0x7f831809f240 2023-09-30T22:29:10.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.263+0000 7f831cc96700 1 -- 172.21.15.16:0/4096549932 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83080112d0 con 0x7f831809f240 2023-09-30T22:29:10.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.264+0000 7f831fefc700 1 -- 172.21.15.16:0/4096549932 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f831809f240 msgr2=0x7f83181317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:10.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.264+0000 7f831fefc700 1 --2- 172.21.15.16:0/4096549932 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f831809f240 0x7f83181317f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f83080131e0 tx=0x7f83080134f0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.264+0000 7f831fefc700 1 -- 172.21.15.16:0/4096549932 shutdown_connections 2023-09-30T22:29:10.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.264+0000 7f831fefc700 1 --2- 172.21.15.16:0/4096549932 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f831809f650 0x7f8318131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.264+0000 7f831fefc700 1 --2- 172.21.15.16:0/4096549932 >> v2:172.21.15.16:3300/0 conn(0x7f831809e930 0x7f831809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.264+0000 7f831fefc700 1 --2- 172.21.15.16:0/4096549932 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f831809f240 0x7f83181317f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.264+0000 7f831fefc700 1 -- 172.21.15.16:0/4096549932 >> 172.21.15.16:0/4096549932 conn(0x7f83180a4210 msgr2=0x7f83180a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:10.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.264+0000 7f831fefc700 1 -- 172.21.15.16:0/4096549932 shutdown_connections 2023-09-30T22:29:10.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.264+0000 7f831fefc700 1 -- 172.21.15.16:0/4096549932 wait complete. 2023-09-30T22:29:10.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.265+0000 7f831fefc700 1 Processor -- start 2023-09-30T22:29:10.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.265+0000 7f831fefc700 1 -- start start 2023-09-30T22:29:10.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.266+0000 7f831fefc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f831809e930 0x7f83181a1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:10.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.266+0000 7f831fefc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f831809f240 0x7f83181a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:10.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.266+0000 7f831fefc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f831809f650 0x7f83181c4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:10.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.266+0000 7f831fefc700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83181a0ec0 con 0x7f831809f650 2023-09-30T22:29:10.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.266+0000 7f831fefc700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83181b52b0 con 0x7f831809f240 2023-09-30T22:29:10.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.266+0000 7f831fefc700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8318071d20 con 0x7f831809e930 2023-09-30T22:29:10.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.266+0000 7f831dc98700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f831809e930 0x7f83181a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:10.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.266+0000 7f831d497700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f831809f240 0x7f83181a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:10.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.267+0000 7f831dc98700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f831809e930 0x7f83181a1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:58316/0 (socket says 172.21.15.16:58316) 2023-09-30T22:29:10.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.267+0000 7f831dc98700 1 -- 172.21.15.16:0/2605510537 learned_addr learned my addr 172.21.15.16:0/2605510537 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:10.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.267+0000 7f831d497700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f831809f240 0x7f83181a1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:53898/0 (socket says 172.21.15.16:53898) 2023-09-30T22:29:10.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.267+0000 7f831d497700 1 -- 172.21.15.16:0/2605510537 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f831809e930 msgr2=0x7f83181a1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:10.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.267+0000 7f831d497700 1 --2- 172.21.15.16:0/2605510537 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f831809e930 0x7f83181a1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.267+0000 7f831d497700 1 -- 172.21.15.16:0/2605510537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f831809f650 msgr2=0x7f83181c4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:10.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.267+0000 7f831d497700 1 --2- 172.21.15.16:0/2605510537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f831809f650 0x7f83181c4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.267+0000 7f831d497700 1 -- 172.21.15.16:0/2605510537 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8308011df0 con 0x7f831809f240 2023-09-30T22:29:10.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.267+0000 7f831d497700 1 --2- 172.21.15.16:0/2605510537 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f831809f240 0x7f83181a1850 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f830801d000 tx=0x7f8308004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:10.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.269+0000 7f830e7fc700 1 -- 172.21.15.16:0/2605510537 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83080110d0 con 0x7f831809f240 2023-09-30T22:29:10.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.269+0000 7f830e7fc700 1 -- 172.21.15.16:0/2605510537 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8308011230 con 0x7f831809f240 2023-09-30T22:29:10.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.269+0000 7f830e7fc700 1 -- 172.21.15.16:0/2605510537 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83080115e0 con 0x7f831809f240 2023-09-30T22:29:10.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.270+0000 7f831fefc700 1 -- 172.21.15.16:0/2605510537 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83180a0190 con 0x7f831809f240 2023-09-30T22:29:10.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.270+0000 7f831fefc700 1 -- 172.21.15.16:0/2605510537 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83180a3370 con 0x7f831809f240 2023-09-30T22:29:10.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.271+0000 7f830e7fc700 1 -- 172.21.15.16:0/2605510537 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7f83080036a0 con 0x7f831809f240 2023-09-30T22:29:10.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.271+0000 7f831fefc700 1 -- 172.21.15.16:0/2605510537 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f831806b980 con 0x7f831809f240 2023-09-30T22:29:10.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.271+0000 7f830e7fc700 1 -- 172.21.15.16:0/2605510537 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3922+0+0 (secure 0 0 0) 0x7f8308044fe0 con 0x7f831809f240 2023-09-30T22:29:10.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.271+0000 7f831dc98700 1 -- 172.21.15.16:0/2605510537 >> v1:172.21.15.16:6816/118285 conn(0x7f82fc02c200 legacy=0x7f82fc02e6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/118285 2023-09-30T22:29:10.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.280+0000 7f830e7fc700 1 -- 172.21.15.16:0/2605510537 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7f8308019980 con 0x7f831809f240 2023-09-30T22:29:10.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.295+0000 7fc1b74ac700 1 Processor -- start 2023-09-30T22:29:10.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.295+0000 7fc1b74ac700 1 -- start start 2023-09-30T22:29:10.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.296+0000 7fc1b74ac700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc1b006b250 0x7fc1b0136840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:10.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.296+0000 7fc1b74ac700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc1b0136d80 0x7fc1b0139160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:10.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.296+0000 7fc1b74ac700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1b009ece0 con 0x7fc1b006a770 2023-09-30T22:29:10.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.296+0000 7fc1b74ac700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1b009e760 con 0x7fc1b006b250 2023-09-30T22:29:10.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.296+0000 7fc1b74ac700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b009e040 con 0x7fc1b0136d80 2023-09-30T22:29:10.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.296+0000 7fc1b5a49700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc1b0136d80 0x7fc1b0139160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:10.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.297+0000 7fc1b5a49700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc1b0136d80 0x7fc1b0139160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:58326/0 (socket says 172.21.15.16:58326) 2023-09-30T22:29:10.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.297+0000 7fc1b5a49700 1 -- 172.21.15.16:0/3765888938 learned_addr learned my addr 172.21.15.16:0/3765888938 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:10.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.297+0000 7fc1b5a49700 1 -- 172.21.15.16:0/3765888938 >> v2:172.21.15.16:3300/0 conn(0x7fc1b006b250 msgr2=0x7fc1b0136840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:10.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.297+0000 7fc1b5a49700 1 --2- 172.21.15.16:0/3765888938 >> v2:172.21.15.16:3300/0 conn(0x7fc1b006b250 0x7fc1b0136840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.297+0000 7fc1b5a49700 1 -- 172.21.15.16:0/3765888938 >> v1:172.21.15.16:6789/0 conn(0x7fc1b006a770 legacy=0x7fc1b006ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:10.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.297+0000 7fc1b5a49700 1 -- 172.21.15.16:0/3765888938 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b009f630 con 0x7fc1b0136d80 2023-09-30T22:29:10.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.297+0000 7fc1b5a49700 1 --2- 172.21.15.16:0/3765888938 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc1b0136d80 0x7fc1b0139160 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8009e50 tx=0x7fc1a800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea207a60cc0683af server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:10.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.297+0000 7fc1a7fff700 1 -- 172.21.15.16:0/3765888938 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a800e070 con 0x7fc1b0136d80 2023-09-30T22:29:10.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.298+0000 7fc1a7fff700 1 -- 172.21.15.16:0/3765888938 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1a8003a20 con 0x7fc1b0136d80 2023-09-30T22:29:10.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.298+0000 7fc1a7fff700 1 -- 172.21.15.16:0/3765888938 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a8003dd0 con 0x7fc1b0136d80 2023-09-30T22:29:10.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.299+0000 7fc1b74ac700 1 -- 172.21.15.16:0/3765888938 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc1b0136d80 msgr2=0x7fc1b0139160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:10.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.299+0000 7fc1b74ac700 1 --2- 172.21.15.16:0/3765888938 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc1b0136d80 0x7fc1b0139160 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc1a8009e50 tx=0x7fc1a800b330 comp rx=0 tx=0).stop 2023-09-30T22:29:10.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.299+0000 7fc1b74ac700 1 -- 172.21.15.16:0/3765888938 shutdown_connections 2023-09-30T22:29:10.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.299+0000 7fc1b74ac700 1 --2- 172.21.15.16:0/3765888938 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc1b0136d80 0x7fc1b0139160 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.299+0000 7fc1b74ac700 1 --2- 172.21.15.16:0/3765888938 >> v2:172.21.15.16:3300/0 conn(0x7fc1b006b250 0x7fc1b0136840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.299+0000 7fc1b74ac700 1 -- 172.21.15.16:0/3765888938 >> 172.21.15.16:0/3765888938 conn(0x7fc1b0066f50 msgr2=0x7fc1b009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:10.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.299+0000 7fc1b74ac700 1 -- 172.21.15.16:0/3765888938 shutdown_connections 2023-09-30T22:29:10.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.299+0000 7fc1b74ac700 1 -- 172.21.15.16:0/3765888938 wait complete. 2023-09-30T22:29:10.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.301+0000 7fc1b74ac700 1 Processor -- start 2023-09-30T22:29:10.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.301+0000 7fc1b74ac700 1 -- start start 2023-09-30T22:29:10.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.301+0000 7fc1b74ac700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc1b006a770 0x7fc1b011d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:10.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.301+0000 7fc1b74ac700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1b006b250 0x7fc1b011d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:10.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.302+0000 7fc1b74ac700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc1b0136d80 0x7fc1b01c6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:10.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.302+0000 7fc1b74ac700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b011fcd0 con 0x7fc1b006b250 2023-09-30T22:29:10.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.302+0000 7fc1b74ac700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b006ea70 con 0x7fc1b006a770 2023-09-30T22:29:10.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.302+0000 7fc1b74ac700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b0071e60 con 0x7fc1b0136d80 2023-09-30T22:29:10.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.302+0000 7fc1b5248700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc1b006a770 0x7fc1b011d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:10.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.302+0000 7fc1b5248700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc1b006a770 0x7fc1b011d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:53910/0 (socket says 172.21.15.16:53910) 2023-09-30T22:29:10.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.302+0000 7fc1b5248700 1 -- 172.21.15.16:0/78475421 learned_addr learned my addr 172.21.15.16:0/78475421 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:10.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.302+0000 7fc1b5a49700 1 --2- 172.21.15.16:0/78475421 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc1b0136d80 0x7fc1b01c6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:10.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.302+0000 7fc1b4a47700 1 --2- 172.21.15.16:0/78475421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1b006b250 0x7fc1b011d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:10.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.302+0000 7fc1b5248700 1 -- 172.21.15.16:0/78475421 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc1b0136d80 msgr2=0x7fc1b01c6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:10.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.302+0000 7fc1b5248700 1 --2- 172.21.15.16:0/78475421 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc1b0136d80 0x7fc1b01c6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.302+0000 7fc1b5248700 1 -- 172.21.15.16:0/78475421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1b006b250 msgr2=0x7fc1b011d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:10.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.302+0000 7fc1b5248700 1 --2- 172.21.15.16:0/78475421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1b006b250 0x7fc1b011d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.302+0000 7fc1b5248700 1 -- 172.21.15.16:0/78475421 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a8009b30 con 0x7fc1b006a770 2023-09-30T22:29:10.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.303+0000 7fc1b5248700 1 --2- 172.21.15.16:0/78475421 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc1b006a770 0x7fc1b011d040 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac00c4a0 tx=0x7fc1ac00c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:10.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.305+0000 7fc1a5ffb700 1 -- 172.21.15.16:0/78475421 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac0054e0 con 0x7fc1b006a770 2023-09-30T22:29:10.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.305+0000 7fc1a5ffb700 1 -- 172.21.15.16:0/78475421 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1ac003680 con 0x7fc1b006a770 2023-09-30T22:29:10.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.305+0000 7fc1a5ffb700 1 -- 172.21.15.16:0/78475421 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac003a30 con 0x7fc1b006a770 2023-09-30T22:29:10.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.305+0000 7fc1b74ac700 1 -- 172.21.15.16:0/78475421 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1b0120a00 con 0x7fc1b006a770 2023-09-30T22:29:10.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.305+0000 7fc1b74ac700 1 -- 172.21.15.16:0/78475421 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1b0120270 con 0x7fc1b006a770 2023-09-30T22:29:10.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.306+0000 7fc1a5ffb700 1 -- 172.21.15.16:0/78475421 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7fc1ac003b90 con 0x7fc1b006a770 2023-09-30T22:29:10.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.307+0000 7fc1b74ac700 1 -- 172.21.15.16:0/78475421 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1b0074ea0 con 0x7fc1b006a770 2023-09-30T22:29:10.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.307+0000 7fc1a5ffb700 1 -- 172.21.15.16:0/78475421 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3922+0+0 (secure 0 0 0) 0x7fc1ac03aab0 con 0x7fc1b006a770 2023-09-30T22:29:10.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.307+0000 7fc1b4a47700 1 -- 172.21.15.16:0/78475421 >> v1:172.21.15.16:6816/118285 conn(0x7fc19402c1b0 legacy=0x7fc19402e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/118285 2023-09-30T22:29:10.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.315+0000 7fc1a5ffb700 1 -- 172.21.15.16:0/78475421 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7fc1ac003e40 con 0x7fc1b006a770 2023-09-30T22:29:10.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.376+0000 7f29a2ffd700 1 -- 172.21.15.16:0/628245425 >> v1:172.21.15.16:6816/118285 conn(0x7f299002c200 legacy=0x7f299002e6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/118285 2023-09-30T22:29:10.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.384+0000 7f29a976e700 1 -- 172.21.15.16:0/628245425 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f29a406c360 con 0x7f29a409e1d0 2023-09-30T22:29:10.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.384+0000 7f298b7fe700 1 -- 172.21.15.16:0/628245425 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2998023070 con 0x7f29a409e1d0 2023-09-30T22:29:10.386 INFO:teuthology.orchestra.run.smithi016.stdout:38654705694 2023-09-30T22:29:10.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.387+0000 7f29a976e700 1 -- 172.21.15.16:0/628245425 >> v1:172.21.15.16:6816/118285 conn(0x7f299002c200 legacy=0x7f299002e6b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:10.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.387+0000 7f29a976e700 1 -- 172.21.15.16:0/628245425 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f29a409e1d0 msgr2=0x7f29a41c4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:10.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.387+0000 7f29a976e700 1 --2- 172.21.15.16:0/628245425 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f29a409e1d0 0x7f29a41c4930 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f299800b390 tx=0x7f299800b750 comp rx=0 tx=0).stop 2023-09-30T22:29:10.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.388+0000 7f29a976e700 1 -- 172.21.15.16:0/628245425 shutdown_connections 2023-09-30T22:29:10.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.388+0000 7f29a976e700 1 --2- 172.21.15.16:0/628245425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29a4135e50 0x7f29a4119480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.388+0000 7f29a976e700 1 --2- 172.21.15.16:0/628245425 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f29a409d6f0 0x7f29a41c43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.388+0000 7f29a976e700 1 --2- 172.21.15.16:0/628245425 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f29a409e1d0 0x7f29a41c4930 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.388+0000 7f29a976e700 1 -- 172.21.15.16:0/628245425 >> 172.21.15.16:0/628245425 conn(0x7f29a4066f50 msgr2=0x7f29a40a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:10.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.388+0000 7f29a976e700 1 -- 172.21.15.16:0/628245425 shutdown_connections 2023-09-30T22:29:10.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.388+0000 7f29a976e700 1 -- 172.21.15.16:0/628245425 wait complete. 2023-09-30T22:29:10.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.395+0000 7fcbea7fc700 1 -- 172.21.15.16:0/408704217 >> v1:172.21.15.16:6816/118285 conn(0x7fcbd802c1b0 legacy=0x7fcbd802e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/118285 2023-09-30T22:29:10.403 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705695 got 38654705694 for osd.3 2023-09-30T22:29:10.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.430+0000 7fcbf1987700 1 -- 172.21.15.16:0/408704217 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fcbec074ea0 con 0x7fcbec06f5d0 2023-09-30T22:29:10.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.431+0000 7fcbd3fff700 1 -- 172.21.15.16:0/408704217 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fcbdc040060 con 0x7fcbec06f5d0 2023-09-30T22:29:10.432 INFO:teuthology.orchestra.run.smithi016.stdout:38654705695 2023-09-30T22:29:10.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.433+0000 7fcbf1987700 1 -- 172.21.15.16:0/408704217 >> v1:172.21.15.16:6816/118285 conn(0x7fcbd802c1b0 legacy=0x7fcbd802e660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:10.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.433+0000 7fcbf1987700 1 -- 172.21.15.16:0/408704217 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcbec06f5d0 msgr2=0x7fcbec13a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:10.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.433+0000 7fcbf1987700 1 --2- 172.21.15.16:0/408704217 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcbec06f5d0 0x7fcbec13a690 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcbdc00c410 tx=0x7fcbdc00c720 comp rx=0 tx=0).stop 2023-09-30T22:29:10.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.433+0000 7fcbf1987700 1 -- 172.21.15.16:0/408704217 shutdown_connections 2023-09-30T22:29:10.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.433+0000 7fcbf1987700 1 --2- 172.21.15.16:0/408704217 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcbec135c20 0x7fcbec1343d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.433+0000 7fcbf1987700 1 --2- 172.21.15.16:0/408704217 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcbec06f5d0 0x7fcbec13a690 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.433+0000 7fcbf1987700 1 --2- 172.21.15.16:0/408704217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbec12cf60 0x7fcbec133e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.433+0000 7fcbf1987700 1 -- 172.21.15.16:0/408704217 >> 172.21.15.16:0/408704217 conn(0x7fcbec0a2aa0 msgr2=0x7fcbec12f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:10.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.433+0000 7fcbf1987700 1 -- 172.21.15.16:0/408704217 shutdown_connections 2023-09-30T22:29:10.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.433+0000 7fcbf1987700 1 -- 172.21.15.16:0/408704217 wait complete. 2023-09-30T22:29:10.448 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705696 got 38654705695 for osd.2 2023-09-30T22:29:10.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.471+0000 7f831fefc700 1 -- 172.21.15.16:0/2605510537 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f8318074ea0 con 0x7f831809f240 2023-09-30T22:29:10.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.471+0000 7f830e7fc700 1 -- 172.21.15.16:0/2605510537 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8308019390 con 0x7f831809f240 2023-09-30T22:29:10.473 INFO:teuthology.orchestra.run.smithi016.stdout:38654705694 2023-09-30T22:29:10.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.472+0000 7f831dc98700 1 -- 172.21.15.16:0/2605510537 >> v1:172.21.15.16:6816/118285 conn(0x7f82fc02c200 legacy=0x7f82fc02e6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/118285 2023-09-30T22:29:10.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.473+0000 7f831fefc700 1 -- 172.21.15.16:0/2605510537 >> v1:172.21.15.16:6816/118285 conn(0x7f82fc02c200 legacy=0x7f82fc02e6b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:10.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.473+0000 7f831fefc700 1 -- 172.21.15.16:0/2605510537 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f831809f240 msgr2=0x7f83181a1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:10.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.473+0000 7f831fefc700 1 --2- 172.21.15.16:0/2605510537 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f831809f240 0x7f83181a1850 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f830801d000 tx=0x7f8308004750 comp rx=0 tx=0).stop 2023-09-30T22:29:10.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.473+0000 7f831fefc700 1 -- 172.21.15.16:0/2605510537 shutdown_connections 2023-09-30T22:29:10.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.473+0000 7f831fefc700 1 --2- 172.21.15.16:0/2605510537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f831809f650 0x7f83181c4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.473+0000 7f831fefc700 1 --2- 172.21.15.16:0/2605510537 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f831809e930 0x7f83181a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.473+0000 7f831fefc700 1 --2- 172.21.15.16:0/2605510537 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f831809f240 0x7f83181a1850 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.473+0000 7f831fefc700 1 -- 172.21.15.16:0/2605510537 >> 172.21.15.16:0/2605510537 conn(0x7f83180a4210 msgr2=0x7f83181b79e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:10.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.473+0000 7f831fefc700 1 -- 172.21.15.16:0/2605510537 shutdown_connections 2023-09-30T22:29:10.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.473+0000 7f831fefc700 1 -- 172.21.15.16:0/2605510537 wait complete. 2023-09-30T22:29:10.488 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705695 got 38654705694 for osd.0 2023-09-30T22:29:10.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.508+0000 7fc1b4a47700 1 -- 172.21.15.16:0/78475421 >> v1:172.21.15.16:6816/118285 conn(0x7fc19402c1b0 legacy=0x7fc19402e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/118285 2023-09-30T22:29:10.558 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:29:10.557+0000 7f90c460e240 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-09-30T22:29:10.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.576+0000 7fc1b74ac700 1 -- 172.21.15.16:0/78475421 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc1b006c360 con 0x7fc1b006a770 2023-09-30T22:29:10.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.577+0000 7fc1a5ffb700 1 -- 172.21.15.16:0/78475421 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc1ac017020 con 0x7fc1b006a770 2023-09-30T22:29:10.578 INFO:teuthology.orchestra.run.smithi016.stdout:38654705694 2023-09-30T22:29:10.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.579+0000 7fc1b74ac700 1 -- 172.21.15.16:0/78475421 >> v1:172.21.15.16:6816/118285 conn(0x7fc19402c1b0 legacy=0x7fc19402e660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:10.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.579+0000 7fc1b74ac700 1 -- 172.21.15.16:0/78475421 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc1b006a770 msgr2=0x7fc1b011d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:10.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.579+0000 7fc1b74ac700 1 --2- 172.21.15.16:0/78475421 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc1b006a770 0x7fc1b011d040 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac00c4a0 tx=0x7fc1ac00c860 comp rx=0 tx=0).stop 2023-09-30T22:29:10.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.579+0000 7fc1b74ac700 1 -- 172.21.15.16:0/78475421 shutdown_connections 2023-09-30T22:29:10.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.579+0000 7fc1b74ac700 1 --2- 172.21.15.16:0/78475421 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc1b006a770 0x7fc1b011d040 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.579+0000 7fc1b74ac700 1 --2- 172.21.15.16:0/78475421 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc1b0136d80 0x7fc1b01c6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.579+0000 7fc1b74ac700 1 --2- 172.21.15.16:0/78475421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1b006b250 0x7fc1b011d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:10.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.579+0000 7fc1b74ac700 1 -- 172.21.15.16:0/78475421 >> 172.21.15.16:0/78475421 conn(0x7fc1b0066f50 msgr2=0x7fc1b0135ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:10.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.579+0000 7fc1b74ac700 1 -- 172.21.15.16:0/78475421 shutdown_connections 2023-09-30T22:29:10.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:10.579+0000 7fc1b74ac700 1 -- 172.21.15.16:0/78475421 wait complete. 2023-09-30T22:29:10.594 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705695 got 38654705694 for osd.1 2023-09-30T22:29:10.721 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:29:10.719+0000 7f90c460e240 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-09-30T22:29:10.804 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:29:10.803+0000 7f90c460e240 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-09-30T22:29:10.957 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:29:10.956+0000 7f90c460e240 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-09-30T22:29:11.036 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:29:11.035+0000 7f90c460e240 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-09-30T22:29:11.218 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:29:11.217+0000 7f90c460e240 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-09-30T22:29:11.307 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:29:11.306+0000 7f90c460e240 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-09-30T22:29:11.394 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:29:11.393+0000 7f90c460e240 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-09-30T22:29:11.404 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:29:11.449 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:29:11.489 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:29:11.510 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:29:11.509+0000 7f90c460e240 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-09-30T22:29:11.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.551+0000 7f508380d700 1 Processor -- start 2023-09-30T22:29:11.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.551+0000 7f508380d700 1 -- start start 2023-09-30T22:29:11.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.552+0000 7f508380d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f507c09d6f0 0x7f507c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:11.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.552+0000 7f508380d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f507c09e410 0x7f507c130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:11.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.552+0000 7f508380d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f507c09f130 con 0x7f507c09e000 2023-09-30T22:29:11.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.552+0000 7f508380d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f507c09f520 con 0x7f507c09d6f0 2023-09-30T22:29:11.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.552+0000 7f508380d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f507c09e930 con 0x7f507c09e410 2023-09-30T22:29:11.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.552+0000 7f5080da8700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f507c09e000 0x7f507c130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40302/0 (socket says 172.21.15.16:40302) 2023-09-30T22:29:11.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.552+0000 7f5080da8700 1 -- 172.21.15.16:0/1785127533 learned_addr learned my addr 172.21.15.16:0/1785127533 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:29:11.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.552+0000 7f50815a9700 1 --2- 172.21.15.16:0/1785127533 >> v2:172.21.15.16:3300/0 conn(0x7f507c09d6f0 0x7f507c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:11.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.552+0000 7f5081daa700 1 --2- 172.21.15.16:0/1785127533 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f507c09e410 0x7f507c130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:11.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.552+0000 7f50815a9700 1 -- 172.21.15.16:0/1785127533 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f507c09e410 msgr2=0x7f507c130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:11.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.552+0000 7f50815a9700 1 --2- 172.21.15.16:0/1785127533 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f507c09e410 0x7f507c130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.552+0000 7f5073fff700 1 -- 172.21.15.16:0/1785127533 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3199032781 0 0) 0x7f507c09f130 con 0x7f507c09e000 2023-09-30T22:29:11.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.552+0000 7f50815a9700 1 -- 172.21.15.16:0/1785127533 >> v1:172.21.15.16:6789/0 conn(0x7f507c09e000 legacy=0x7f507c130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:11.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.553+0000 7f50815a9700 1 -- 172.21.15.16:0/1785127533 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f507c09b2d0 con 0x7f507c09d6f0 2023-09-30T22:29:11.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.553+0000 7f50815a9700 1 --2- 172.21.15.16:0/1785127533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f507c09d6f0 0x7f507c09dac0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5078009840 tx=0x7f5078009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f17667ffe375d9d1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:11.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.553+0000 7f5073fff700 1 -- 172.21.15.16:0/1785127533 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5078011830 con 0x7f507c09d6f0 2023-09-30T22:29:11.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.553+0000 7f5073fff700 1 -- 172.21.15.16:0/1785127533 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5078011990 con 0x7f507c09d6f0 2023-09-30T22:29:11.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.553+0000 7f5073fff700 1 -- 172.21.15.16:0/1785127533 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5078011d20 con 0x7f507c09d6f0 2023-09-30T22:29:11.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.554+0000 7f508380d700 1 -- 172.21.15.16:0/1785127533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f507c09d6f0 msgr2=0x7f507c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:11.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.554+0000 7f508380d700 1 --2- 172.21.15.16:0/1785127533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f507c09d6f0 0x7f507c09dac0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5078009840 tx=0x7f5078009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:11.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.554+0000 7f508380d700 1 -- 172.21.15.16:0/1785127533 shutdown_connections 2023-09-30T22:29:11.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.554+0000 7f508380d700 1 --2- 172.21.15.16:0/1785127533 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f507c09e410 0x7f507c130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.554+0000 7f508380d700 1 --2- 172.21.15.16:0/1785127533 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f507c09d6f0 0x7f507c09dac0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.554+0000 7f508380d700 1 -- 172.21.15.16:0/1785127533 >> 172.21.15.16:0/1785127533 conn(0x7f507c066f50 msgr2=0x7f507c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:11.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.554+0000 7f508380d700 1 -- 172.21.15.16:0/1785127533 shutdown_connections 2023-09-30T22:29:11.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.554+0000 7f508380d700 1 -- 172.21.15.16:0/1785127533 wait complete. 2023-09-30T22:29:11.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.555+0000 7f508380d700 1 Processor -- start 2023-09-30T22:29:11.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.555+0000 7f508380d700 1 -- start start 2023-09-30T22:29:11.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.556+0000 7f508380d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f507c09d6f0 0x7f507c11e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:11.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.556+0000 7f508380d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f507c09e000 0x7f507c11e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:11.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.556+0000 7f508380d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f507c09e410 0x7f507c11eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:11.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.556+0000 7f508380d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507c121aa0 con 0x7f507c09d6f0 2023-09-30T22:29:11.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.556+0000 7f508380d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f507c120e70 con 0x7f507c09e410 2023-09-30T22:29:11.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.556+0000 7f508380d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f507c120170 con 0x7f507c09e000 2023-09-30T22:29:11.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.556+0000 7f5081daa700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f507c09e410 0x7f507c11eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:11.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.556+0000 7f5080da8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f507c09e000 0x7f507c11e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:11.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.556+0000 7f5081daa700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f507c09e410 0x7f507c11eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:53926/0 (socket says 172.21.15.16:53926) 2023-09-30T22:29:11.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.556+0000 7f5081daa700 1 -- 172.21.15.16:0/998308977 learned_addr learned my addr 172.21.15.16:0/998308977 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:11.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.556+0000 7f50815a9700 1 --2- 172.21.15.16:0/998308977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f507c09d6f0 0x7f507c11e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:11.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.557+0000 7f5081daa700 1 -- 172.21.15.16:0/998308977 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f507c09e000 msgr2=0x7f507c11e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:11.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.557+0000 7f5081daa700 1 --2- 172.21.15.16:0/998308977 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f507c09e000 0x7f507c11e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.557+0000 7f5081daa700 1 -- 172.21.15.16:0/998308977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f507c09d6f0 msgr2=0x7f507c11e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:11.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.557+0000 7f5081daa700 1 --2- 172.21.15.16:0/998308977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f507c09d6f0 0x7f507c11e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.557+0000 7f5081daa700 1 -- 172.21.15.16:0/998308977 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50780094a0 con 0x7f507c09e410 2023-09-30T22:29:11.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.557+0000 7f5080da8700 1 --2- 172.21.15.16:0/998308977 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f507c09e000 0x7f507c11e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:11.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.557+0000 7f5081daa700 1 --2- 172.21.15.16:0/998308977 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f507c09e410 0x7f507c11eaa0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5074009dd0 tx=0x7f507400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:11.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.557+0000 7f5071ffb700 1 -- 172.21.15.16:0/998308977 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5074009c40 con 0x7f507c09e410 2023-09-30T22:29:11.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.557+0000 7f5071ffb700 1 -- 172.21.15.16:0/998308977 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f507400f040 con 0x7f507c09e410 2023-09-30T22:29:11.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.557+0000 7f508380d700 1 -- 172.21.15.16:0/998308977 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f507c121230 con 0x7f507c09e410 2023-09-30T22:29:11.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.557+0000 7f5071ffb700 1 -- 172.21.15.16:0/998308977 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507400ceb0 con 0x7f507c09e410 2023-09-30T22:29:11.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.558+0000 7f508380d700 1 -- 172.21.15.16:0/998308977 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f507c09c750 con 0x7f507c09e410 2023-09-30T22:29:11.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.558+0000 7f5071ffb700 1 -- 172.21.15.16:0/998308977 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7f50740135c0 con 0x7f507c09e410 2023-09-30T22:29:11.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.559+0000 7f5071ffb700 1 -- 172.21.15.16:0/998308977 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3922+0+0 (secure 0 0 0) 0x7f507403fdd0 con 0x7f507c09e410 2023-09-30T22:29:11.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.559+0000 7f50815a9700 1 -- 172.21.15.16:0/998308977 >> v1:172.21.15.16:6816/118285 conn(0x7f506002c1b0 legacy=0x7f506002e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/118285 2023-09-30T22:29:11.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.559+0000 7f508380d700 1 -- 172.21.15.16:0/998308977 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f507c121460 con 0x7f507c09e410 2023-09-30T22:29:11.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.563+0000 7f5071ffb700 1 -- 172.21.15.16:0/998308977 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7f507c121460 con 0x7f507c09e410 2023-09-30T22:29:11.594 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:29:11.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.634+0000 7f2bf1c41700 1 Processor -- start 2023-09-30T22:29:11.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.634+0000 7f2bf1c41700 1 -- start start 2023-09-30T22:29:11.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.635+0000 7f2bf1c41700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2bec06a770 0x7f2bec09d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:11.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.635+0000 7f2bf1c41700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2bec09dc30 0x7f2bec09e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:11.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.635+0000 7f2bf1c41700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bec09a810 con 0x7f2bec13c1e0 2023-09-30T22:29:11.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.635+0000 7f2bf1c41700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bec06ab40 con 0x7f2bec09dc30 2023-09-30T22:29:11.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.635+0000 7f2bf1c41700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec09ebc0 con 0x7f2bec06a770 2023-09-30T22:29:11.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.635+0000 7f2beaffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2bec09dc30 0x7f2bec09e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:11.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.636+0000 7f2beb7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2bec06a770 0x7f2bec09d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:11.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.636+0000 7f2beaffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2bec09dc30 0x7f2bec09e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46918/0 (socket says 172.21.15.16:46918) 2023-09-30T22:29:11.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.636+0000 7f2beaffd700 1 -- 172.21.15.16:0/3711209172 learned_addr learned my addr 172.21.15.16:0/3711209172 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:11.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.636+0000 7f2beb7fe700 1 -- 172.21.15.16:0/3711209172 >> v2:172.21.15.16:3300/0 conn(0x7f2bec09dc30 msgr2=0x7f2bec09e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:11.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.636+0000 7f2beb7fe700 1 --2- 172.21.15.16:0/3711209172 >> v2:172.21.15.16:3300/0 conn(0x7f2bec09dc30 0x7f2bec09e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.636+0000 7f2beb7fe700 1 -- 172.21.15.16:0/3711209172 >> v1:172.21.15.16:6789/0 conn(0x7f2bec13c1e0 legacy=0x7f2bec13e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:11.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.636+0000 7f2beb7fe700 1 -- 172.21.15.16:0/3711209172 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bec09e920 con 0x7f2bec06a770 2023-09-30T22:29:11.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.636+0000 7f2beb7fe700 1 --2- 172.21.15.16:0/3711209172 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2bec06a770 0x7f2bec09d6f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc009870 tx=0x7f2bdc009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c09d371620a5846c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:11.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.637+0000 7f2bea7fc700 1 -- 172.21.15.16:0/3711209172 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc011830 con 0x7f2bec06a770 2023-09-30T22:29:11.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.637+0000 7f2bea7fc700 1 -- 172.21.15.16:0/3711209172 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bdc011990 con 0x7f2bec06a770 2023-09-30T22:29:11.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.637+0000 7f2bea7fc700 1 -- 172.21.15.16:0/3711209172 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc011d00 con 0x7f2bec06a770 2023-09-30T22:29:11.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.638+0000 7f2bf1c41700 1 -- 172.21.15.16:0/3711209172 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2bec06a770 msgr2=0x7f2bec09d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:11.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.638+0000 7f2bf1c41700 1 --2- 172.21.15.16:0/3711209172 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2bec06a770 0x7f2bec09d6f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc009870 tx=0x7f2bdc009b80 comp rx=0 tx=0).stop 2023-09-30T22:29:11.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.638+0000 7f2bf1c41700 1 -- 172.21.15.16:0/3711209172 shutdown_connections 2023-09-30T22:29:11.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.638+0000 7f2bf1c41700 1 --2- 172.21.15.16:0/3711209172 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2bec06a770 0x7f2bec09d6f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.638+0000 7f2bf1c41700 1 --2- 172.21.15.16:0/3711209172 >> v2:172.21.15.16:3300/0 conn(0x7f2bec09dc30 0x7f2bec09e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.638+0000 7f2bf1c41700 1 -- 172.21.15.16:0/3711209172 >> 172.21.15.16:0/3711209172 conn(0x7f2bec066f50 msgr2=0x7f2bec09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:11.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.638+0000 7f2bf1c41700 1 -- 172.21.15.16:0/3711209172 shutdown_connections 2023-09-30T22:29:11.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.638+0000 7f2bf1c41700 1 -- 172.21.15.16:0/3711209172 wait complete. 2023-09-30T22:29:11.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.639+0000 7f2bf1c41700 1 Processor -- start 2023-09-30T22:29:11.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.640+0000 7f2bf1c41700 1 -- start start 2023-09-30T22:29:11.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.640+0000 7f2bf1c41700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bec06a770 0x7f2bec1cc960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:11.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.641+0000 7f2bf1c41700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2bec09dc30 0x7f2bec1ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:11.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.641+0000 7f2bf1c41700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2bec13c1e0 0x7f2bec127110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:11.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.641+0000 7f2bf1c41700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec1c12c0 con 0x7f2bec06a770 2023-09-30T22:29:11.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.641+0000 7f2bf1c41700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec09f1a0 con 0x7f2bec09dc30 2023-09-30T22:29:11.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.641+0000 7f2bf1c41700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2bec12ace0 con 0x7f2bec13c1e0 2023-09-30T22:29:11.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.641+0000 7f2beb7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bec06a770 0x7f2bec1cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:11.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.641+0000 7f2beb7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bec06a770 0x7f2bec1cc960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46924/0 (socket says 172.21.15.16:46924) 2023-09-30T22:29:11.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.641+0000 7f2beb7fe700 1 -- 172.21.15.16:0/1783622489 learned_addr learned my addr 172.21.15.16:0/1783622489 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:11.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.641+0000 7f2beaffd700 1 --2- 172.21.15.16:0/1783622489 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2bec09dc30 0x7f2bec1ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:11.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.641+0000 7f2beb7fe700 1 -- 172.21.15.16:0/1783622489 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2bec13c1e0 msgr2=0x7f2bec127110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:11.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.641+0000 7f2beb7fe700 1 --2- 172.21.15.16:0/1783622489 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2bec13c1e0 0x7f2bec127110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.641+0000 7f2beb7fe700 1 -- 172.21.15.16:0/1783622489 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2bec09dc30 msgr2=0x7f2bec1ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:11.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.641+0000 7f2beb7fe700 1 --2- 172.21.15.16:0/1783622489 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2bec09dc30 0x7f2bec1ccea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.641+0000 7f2beb7fe700 1 -- 172.21.15.16:0/1783622489 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bdc0094a0 con 0x7f2bec06a770 2023-09-30T22:29:11.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.642+0000 7f2beb7fe700 1 --2- 172.21.15.16:0/1783622489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bec06a770 0x7f2bec1cc960 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc000c00 tx=0x7f2bdc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:11.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.642+0000 7f2bd3fff700 1 -- 172.21.15.16:0/1783622489 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc0036a0 con 0x7f2bec06a770 2023-09-30T22:29:11.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.642+0000 7f2bd3fff700 1 -- 172.21.15.16:0/1783622489 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bdc01f070 con 0x7f2bec06a770 2023-09-30T22:29:11.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.642+0000 7f2bd3fff700 1 -- 172.21.15.16:0/1783622489 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc003a20 con 0x7f2bec06a770 2023-09-30T22:29:11.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.642+0000 7f2bf1c41700 1 -- 172.21.15.16:0/1783622489 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bec072500 con 0x7f2bec06a770 2023-09-30T22:29:11.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.642+0000 7f2bf1c41700 1 -- 172.21.15.16:0/1783622489 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bec128cd0 con 0x7f2bec06a770 2023-09-30T22:29:11.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.643+0000 7f2bf1c41700 1 -- 172.21.15.16:0/1783622489 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bec074ea0 con 0x7f2bec06a770 2023-09-30T22:29:11.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.644+0000 7f2bd3fff700 1 -- 172.21.15.16:0/1783622489 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7f2bdc003b80 con 0x7f2bec06a770 2023-09-30T22:29:11.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.644+0000 7f2bd3fff700 1 -- 172.21.15.16:0/1783622489 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3922+0+0 (secure 0 0 0) 0x7f2bdc021070 con 0x7f2bec06a770 2023-09-30T22:29:11.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.644+0000 7f2beaffd700 1 -- 172.21.15.16:0/1783622489 >> v1:172.21.15.16:6816/118285 conn(0x7f2bd802c1b0 legacy=0x7f2bd802e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/118285 2023-09-30T22:29:11.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.651+0000 7f261e5a8700 1 Processor -- start 2023-09-30T22:29:11.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.652+0000 7f2bd3fff700 1 -- 172.21.15.16:0/1783622489 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7f2bdc0253e0 con 0x7f2bec06a770 2023-09-30T22:29:11.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.652+0000 7f261e5a8700 1 -- start start 2023-09-30T22:29:11.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.652+0000 7f261e5a8700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f261809d6f0 0x7f261809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:11.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.652+0000 7f261e5a8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f261809e000 0x7f2618130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:11.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.652+0000 7f261e5a8700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f261809f130 con 0x7f261809e410 2023-09-30T22:29:11.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.652+0000 7f261e5a8700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f261809f520 con 0x7f261809d6f0 2023-09-30T22:29:11.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.652+0000 7f261e5a8700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f261809e930 con 0x7f261809e000 2023-09-30T22:29:11.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.653+0000 7f2617fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f261809d6f0 0x7f261809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:11.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.653+0000 7f2617fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f261809d6f0 0x7f261809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46938/0 (socket says 172.21.15.16:46938) 2023-09-30T22:29:11.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.653+0000 7f2617fff700 1 -- 172.21.15.16:0/3026210098 learned_addr learned my addr 172.21.15.16:0/3026210098 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:11.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.653+0000 7f2617fff700 1 -- 172.21.15.16:0/3026210098 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f261809e000 msgr2=0x7f2618130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:11.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.653+0000 7f2617fff700 1 --2- 172.21.15.16:0/3026210098 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f261809e000 0x7f2618130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.653+0000 7f2617fff700 1 -- 172.21.15.16:0/3026210098 >> v1:172.21.15.16:6789/0 conn(0x7f261809e410 legacy=0x7f2618130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:11.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.653+0000 7f2617fff700 1 -- 172.21.15.16:0/3026210098 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f261809b2d0 con 0x7f261809d6f0 2023-09-30T22:29:11.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.653+0000 7f2617fff700 1 --2- 172.21.15.16:0/3026210098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f261809d6f0 0x7f261809dac0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f260800bb90 tx=0x7f260800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=494aecb506ca5d19 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:11.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.653+0000 7f2616ffd700 1 -- 172.21.15.16:0/3026210098 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f260800dd10 con 0x7f261809d6f0 2023-09-30T22:29:11.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.653+0000 7f2616ffd700 1 -- 172.21.15.16:0/3026210098 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f260800de70 con 0x7f261809d6f0 2023-09-30T22:29:11.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.653+0000 7f2616ffd700 1 -- 172.21.15.16:0/3026210098 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2608004230 con 0x7f261809d6f0 2023-09-30T22:29:11.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.654+0000 7f261e5a8700 1 -- 172.21.15.16:0/3026210098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f261809d6f0 msgr2=0x7f261809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:11.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.654+0000 7f261e5a8700 1 --2- 172.21.15.16:0/3026210098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f261809d6f0 0x7f261809dac0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f260800bb90 tx=0x7f260800bea0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.654+0000 7f261e5a8700 1 -- 172.21.15.16:0/3026210098 shutdown_connections 2023-09-30T22:29:11.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.654+0000 7f261e5a8700 1 --2- 172.21.15.16:0/3026210098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f261809d6f0 0x7f261809dac0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.654+0000 7f261e5a8700 1 --2- 172.21.15.16:0/3026210098 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f261809e000 0x7f2618130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.654+0000 7f261e5a8700 1 -- 172.21.15.16:0/3026210098 >> 172.21.15.16:0/3026210098 conn(0x7f2618066f50 msgr2=0x7f261809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:11.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.654+0000 7f261e5a8700 1 -- 172.21.15.16:0/3026210098 shutdown_connections 2023-09-30T22:29:11.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.654+0000 7f261e5a8700 1 -- 172.21.15.16:0/3026210098 wait complete. 2023-09-30T22:29:11.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.656+0000 7f261e5a8700 1 Processor -- start 2023-09-30T22:29:11.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.656+0000 7f261e5a8700 1 -- start start 2023-09-30T22:29:11.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.657+0000 7f261e5a8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f261809d6f0 0x7f26181c8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:11.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.657+0000 7f261e5a8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f261809e000 0x7f26181c8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:11.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.657+0000 7f261e5a8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f261809e410 0x7f26181c9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:11.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.657+0000 7f261e5a8700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2618124520 con 0x7f261809e410 2023-09-30T22:29:11.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.657+0000 7f261e5a8700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2618124330 con 0x7f261809e000 2023-09-30T22:29:11.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.657+0000 7f261e5a8700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f26180a1c70 con 0x7f261809d6f0 2023-09-30T22:29:11.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.657+0000 7f261cb45700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f261809e410 0x7f26181c9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:11.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.657+0000 7f261cb45700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f261809e410 0x7f26181c9010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46946/0 (socket says 172.21.15.16:46946) 2023-09-30T22:29:11.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.657+0000 7f261cb45700 1 -- 172.21.15.16:0/84756727 learned_addr learned my addr 172.21.15.16:0/84756727 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:11.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.657+0000 7f26177fe700 1 --2- 172.21.15.16:0/84756727 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f261809e000 0x7f26181c8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:11.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.658+0000 7f261cb45700 1 -- 172.21.15.16:0/84756727 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f261809d6f0 msgr2=0x7f26181c8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:11.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.658+0000 7f261cb45700 1 --2- 172.21.15.16:0/84756727 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f261809d6f0 0x7f26181c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.658+0000 7f261cb45700 1 -- 172.21.15.16:0/84756727 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f261809e000 msgr2=0x7f26181c8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:11.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.658+0000 7f261cb45700 1 --2- 172.21.15.16:0/84756727 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f261809e000 0x7f26181c8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.658+0000 7f261cb45700 1 -- 172.21.15.16:0/84756727 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f260800b7f0 con 0x7f261809e410 2023-09-30T22:29:11.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.658+0000 7f261cb45700 1 --2- 172.21.15.16:0/84756727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f261809e410 0x7f26181c9010 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f261000c7a0 tx=0x7f261000cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:11.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.658+0000 7f2614ff9700 1 -- 172.21.15.16:0/84756727 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2610003e10 con 0x7f261809e410 2023-09-30T22:29:11.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.658+0000 7f2614ff9700 1 -- 172.21.15.16:0/84756727 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2610005070 con 0x7f261809e410 2023-09-30T22:29:11.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.658+0000 7f261e5a8700 1 -- 172.21.15.16:0/84756727 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2618126240 con 0x7f261809e410 2023-09-30T22:29:11.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.658+0000 7f2614ff9700 1 -- 172.21.15.16:0/84756727 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2610005400 con 0x7f261809e410 2023-09-30T22:29:11.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.658+0000 7f261e5a8700 1 -- 172.21.15.16:0/84756727 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f261809c750 con 0x7f261809e410 2023-09-30T22:29:11.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.659+0000 7f2614ff9700 1 -- 172.21.15.16:0/84756727 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7f2610005560 con 0x7f261809e410 2023-09-30T22:29:11.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.659+0000 7f2614ff9700 1 -- 172.21.15.16:0/84756727 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3922+0+0 (secure 0 0 0) 0x7f261003b050 con 0x7f261809e410 2023-09-30T22:29:11.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.659+0000 7f2617fff700 1 -- 172.21.15.16:0/84756727 >> v1:172.21.15.16:6816/118285 conn(0x7f260402c1b0 legacy=0x7f260402e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/118285 2023-09-30T22:29:11.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.660+0000 7f261e5a8700 1 -- 172.21.15.16:0/84756727 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f261804ea50 con 0x7f261809e410 2023-09-30T22:29:11.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.668+0000 7f2614ff9700 1 -- 172.21.15.16:0/84756727 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7f2610005810 con 0x7f261809e410 2023-09-30T22:29:11.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.753+0000 7eff13b42700 1 Processor -- start 2023-09-30T22:29:11.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.754+0000 7eff13b42700 1 -- start start 2023-09-30T22:29:11.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.754+0000 7eff13b42700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7eff0c1337f0 0x7eff0c138880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:11.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.754+0000 7eff13b42700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7eff0c138dc0 0x7eff0c13b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:11.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.754+0000 7eff13b42700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff0c06b990 con 0x7eff0c133420 2023-09-30T22:29:11.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.754+0000 7eff13b42700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff0c09b0e0 con 0x7eff0c138dc0 2023-09-30T22:29:11.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.754+0000 7eff13b42700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff0c133c20 con 0x7eff0c1337f0 2023-09-30T22:29:11.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.755+0000 7eff120df700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7eff0c138dc0 0x7eff0c13b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:11.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.755+0000 7eff118de700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7eff0c133420 0x7eff0c133e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40328/0 (socket says 172.21.15.16:40328) 2023-09-30T22:29:11.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.755+0000 7eff118de700 1 -- 172.21.15.16:0/1484513013 learned_addr learned my addr 172.21.15.16:0/1484513013 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:29:11.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.755+0000 7eff110dd700 1 --2- 172.21.15.16:0/1484513013 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7eff0c1337f0 0x7eff0c138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:11.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.755+0000 7eff120df700 1 -- 172.21.15.16:0/1484513013 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7eff0c1337f0 msgr2=0x7eff0c138880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:11.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.755+0000 7eff120df700 1 --2- 172.21.15.16:0/1484513013 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7eff0c1337f0 0x7eff0c138880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.755+0000 7eff120df700 1 -- 172.21.15.16:0/1484513013 >> v1:172.21.15.16:6789/0 conn(0x7eff0c133420 legacy=0x7eff0c133e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:11.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.755+0000 7eff120df700 1 -- 172.21.15.16:0/1484513013 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff0c0a0590 con 0x7eff0c138dc0 2023-09-30T22:29:11.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.755+0000 7eff108dc700 1 -- 172.21.15.16:0/1484513013 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1716910562 0 0) 0x7eff0c06b990 con 0x7eff0c133420 2023-09-30T22:29:11.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.755+0000 7eff110dd700 1 --2- 172.21.15.16:0/1484513013 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7eff0c1337f0 0x7eff0c138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:11.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.755+0000 7eff120df700 1 --2- 172.21.15.16:0/1484513013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff0c138dc0 0x7eff0c13b1a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7eff04009d90 tx=0x7eff0400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=914a12f6671fff54 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:11.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.756+0000 7eff108dc700 1 -- 172.21.15.16:0/1484513013 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0400e070 con 0x7eff0c138dc0 2023-09-30T22:29:11.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.756+0000 7eff108dc700 1 -- 172.21.15.16:0/1484513013 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff0400bb20 con 0x7eff0c138dc0 2023-09-30T22:29:11.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.756+0000 7eff108dc700 1 -- 172.21.15.16:0/1484513013 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff04012430 con 0x7eff0c138dc0 2023-09-30T22:29:11.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.757+0000 7eff13b42700 1 -- 172.21.15.16:0/1484513013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff0c138dc0 msgr2=0x7eff0c13b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:11.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.757+0000 7eff13b42700 1 --2- 172.21.15.16:0/1484513013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff0c138dc0 0x7eff0c13b1a0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7eff04009d90 tx=0x7eff0400b330 comp rx=0 tx=0).stop 2023-09-30T22:29:11.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.757+0000 7eff13b42700 1 -- 172.21.15.16:0/1484513013 shutdown_connections 2023-09-30T22:29:11.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.757+0000 7eff13b42700 1 --2- 172.21.15.16:0/1484513013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff0c138dc0 0x7eff0c13b1a0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.757+0000 7eff13b42700 1 --2- 172.21.15.16:0/1484513013 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7eff0c1337f0 0x7eff0c138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.757+0000 7eff13b42700 1 -- 172.21.15.16:0/1484513013 >> 172.21.15.16:0/1484513013 conn(0x7eff0c0a4210 msgr2=0x7eff0c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:11.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.757+0000 7eff13b42700 1 -- 172.21.15.16:0/1484513013 shutdown_connections 2023-09-30T22:29:11.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.757+0000 7eff13b42700 1 -- 172.21.15.16:0/1484513013 wait complete. 2023-09-30T22:29:11.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.759+0000 7eff13b42700 1 Processor -- start 2023-09-30T22:29:11.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.760+0000 7f50815a9700 1 -- 172.21.15.16:0/998308977 >> v1:172.21.15.16:6816/118285 conn(0x7f506002c1b0 legacy=0x7f506002e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/118285 2023-09-30T22:29:11.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.759+0000 7eff13b42700 1 -- start start 2023-09-30T22:29:11.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.760+0000 7eff13b42700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7eff0c133420 0x7eff0c130220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:11.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.760+0000 7eff13b42700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff0c1337f0 0x7eff0c130760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:11.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.760+0000 7eff13b42700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7eff0c138dc0 0x7eff0c09c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:11.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.760+0000 7eff13b42700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0c12f440 con 0x7eff0c1337f0 2023-09-30T22:29:11.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.760+0000 7eff13b42700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff0c12fce0 con 0x7eff0c138dc0 2023-09-30T22:29:11.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.760+0000 7eff13b42700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7eff0c12d9d0 con 0x7eff0c133420 2023-09-30T22:29:11.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.760+0000 7eff110dd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff0c1337f0 0x7eff0c130760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:11.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.760+0000 7eff110dd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff0c1337f0 0x7eff0c130760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46974/0 (socket says 172.21.15.16:46974) 2023-09-30T22:29:11.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.760+0000 7eff110dd700 1 -- 172.21.15.16:0/1482057184 learned_addr learned my addr 172.21.15.16:0/1482057184 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:11.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.760+0000 7eff118de700 1 --2- 172.21.15.16:0/1482057184 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7eff0c133420 0x7eff0c130220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:11.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.760+0000 7eff120df700 1 --2- 172.21.15.16:0/1482057184 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7eff0c138dc0 0x7eff0c09c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:11.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.761+0000 7eff118de700 1 -- 172.21.15.16:0/1482057184 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7eff0c138dc0 msgr2=0x7eff0c09c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:11.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.761+0000 7eff118de700 1 --2- 172.21.15.16:0/1482057184 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7eff0c138dc0 0x7eff0c09c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.761+0000 7eff118de700 1 -- 172.21.15.16:0/1482057184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff0c1337f0 msgr2=0x7eff0c130760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:11.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.761+0000 7eff118de700 1 --2- 172.21.15.16:0/1482057184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff0c1337f0 0x7eff0c130760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.761+0000 7eff118de700 1 -- 172.21.15.16:0/1482057184 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efefc0094a0 con 0x7eff0c133420 2023-09-30T22:29:11.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.761+0000 7eff110dd700 1 --2- 172.21.15.16:0/1482057184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff0c1337f0 0x7eff0c130760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:11.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.761+0000 7eff118de700 1 --2- 172.21.15.16:0/1482057184 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7eff0c133420 0x7eff0c130220 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7eff0800c470 tx=0x7eff0800c780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:11.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.761+0000 7eff027fc700 1 -- 172.21.15.16:0/1482057184 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff08004c80 con 0x7eff0c133420 2023-09-30T22:29:11.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.761+0000 7eff027fc700 1 -- 172.21.15.16:0/1482057184 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff08004de0 con 0x7eff0c133420 2023-09-30T22:29:11.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.761+0000 7eff027fc700 1 -- 172.21.15.16:0/1482057184 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff08007310 con 0x7eff0c133420 2023-09-30T22:29:11.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.761+0000 7eff13b42700 1 -- 172.21.15.16:0/1482057184 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff04009a70 con 0x7eff0c133420 2023-09-30T22:29:11.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.761+0000 7eff13b42700 1 -- 172.21.15.16:0/1482057184 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff0c12f640 con 0x7eff0c133420 2023-09-30T22:29:11.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.763+0000 7eff13b42700 1 -- 172.21.15.16:0/1482057184 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff0c074ea0 con 0x7eff0c133420 2023-09-30T22:29:11.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.764+0000 7eff027fc700 1 -- 172.21.15.16:0/1482057184 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7eff08007470 con 0x7eff0c133420 2023-09-30T22:29:11.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.764+0000 7eff027fc700 1 -- 172.21.15.16:0/1482057184 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3922+0+0 (secure 0 0 0) 0x7eff0803ba40 con 0x7eff0c133420 2023-09-30T22:29:11.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.768+0000 7eff110dd700 1 -- 172.21.15.16:0/1482057184 >> v1:172.21.15.16:6816/118285 conn(0x7efef002c1b0 legacy=0x7efef002e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/118285 2023-09-30T22:29:11.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.772+0000 7eff027fc700 1 -- 172.21.15.16:0/1482057184 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7eff08007720 con 0x7eff0c133420 2023-09-30T22:29:11.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.845+0000 7f2beaffd700 1 -- 172.21.15.16:0/1783622489 >> v1:172.21.15.16:6816/118285 conn(0x7f2bd802c1b0 legacy=0x7f2bd802e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/118285 2023-09-30T22:29:11.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.853+0000 7f508380d700 1 -- 172.21.15.16:0/998308977 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f507c121460 con 0x7f507c09e410 2023-09-30T22:29:11.854 INFO:teuthology.orchestra.run.smithi016.stdout:38654705694 2023-09-30T22:29:11.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.853+0000 7f5071ffb700 1 -- 172.21.15.16:0/998308977 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f507c121460 con 0x7f507c09e410 2023-09-30T22:29:11.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.855+0000 7f508380d700 1 -- 172.21.15.16:0/998308977 >> v1:172.21.15.16:6816/118285 conn(0x7f506002c1b0 legacy=0x7f506002e660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:11.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.855+0000 7f508380d700 1 -- 172.21.15.16:0/998308977 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f507c09e410 msgr2=0x7f507c11eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:11.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.855+0000 7f508380d700 1 --2- 172.21.15.16:0/998308977 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f507c09e410 0x7f507c11eaa0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5074009dd0 tx=0x7f507400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.855+0000 7f508380d700 1 -- 172.21.15.16:0/998308977 shutdown_connections 2023-09-30T22:29:11.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.855+0000 7f508380d700 1 --2- 172.21.15.16:0/998308977 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f507c09e410 0x7f507c11eaa0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.855+0000 7f508380d700 1 --2- 172.21.15.16:0/998308977 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f507c09d6f0 0x7f507c11e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.855+0000 7f508380d700 1 --2- 172.21.15.16:0/998308977 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f507c09e000 0x7f507c11e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.855+0000 7f508380d700 1 -- 172.21.15.16:0/998308977 >> 172.21.15.16:0/998308977 conn(0x7f507c066f50 msgr2=0x7f507c070e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:11.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.856+0000 7f508380d700 1 -- 172.21.15.16:0/998308977 shutdown_connections 2023-09-30T22:29:11.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.856+0000 7f508380d700 1 -- 172.21.15.16:0/998308977 wait complete. 2023-09-30T22:29:11.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.858+0000 7f2bf1c41700 1 -- 172.21.15.16:0/1783622489 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f2bec09a3d0 con 0x7f2bec06a770 2023-09-30T22:29:11.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.858+0000 7f2bd3fff700 1 -- 172.21.15.16:0/1783622489 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2bdc02b070 con 0x7f2bec06a770 2023-09-30T22:29:11.860 INFO:teuthology.orchestra.run.smithi016.stdout:38654705695 2023-09-30T22:29:11.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.860+0000 7f2bf1c41700 1 -- 172.21.15.16:0/1783622489 >> v1:172.21.15.16:6816/118285 conn(0x7f2bd802c1b0 legacy=0x7f2bd802e660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:11.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.860+0000 7f2bf1c41700 1 -- 172.21.15.16:0/1783622489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bec06a770 msgr2=0x7f2bec1cc960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:11.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.860+0000 7f2bf1c41700 1 --2- 172.21.15.16:0/1783622489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bec06a770 0x7f2bec1cc960 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc000c00 tx=0x7f2bdc004750 comp rx=0 tx=0).stop 2023-09-30T22:29:11.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.860+0000 7f2617fff700 1 -- 172.21.15.16:0/84756727 >> v1:172.21.15.16:6816/118285 conn(0x7f260402c1b0 legacy=0x7f260402e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/118285 2023-09-30T22:29:11.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.861+0000 7f2bf1c41700 1 -- 172.21.15.16:0/1783622489 shutdown_connections 2023-09-30T22:29:11.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.861+0000 7f2bf1c41700 1 --2- 172.21.15.16:0/1783622489 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2bec13c1e0 0x7f2bec127110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.861+0000 7f2bf1c41700 1 --2- 172.21.15.16:0/1783622489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bec06a770 0x7f2bec1cc960 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.861+0000 7f2bf1c41700 1 --2- 172.21.15.16:0/1783622489 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2bec09dc30 0x7f2bec1ccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.861+0000 7f2bf1c41700 1 -- 172.21.15.16:0/1783622489 >> 172.21.15.16:0/1783622489 conn(0x7f2bec066f50 msgr2=0x7f2bec09b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:11.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.866+0000 7f2bf1c41700 1 -- 172.21.15.16:0/1783622489 shutdown_connections 2023-09-30T22:29:11.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.866+0000 7f2bf1c41700 1 -- 172.21.15.16:0/1783622489 wait complete. 2023-09-30T22:29:11.874 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705695 got 38654705694 for osd.3 2023-09-30T22:29:11.883 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705696 got 38654705695 for osd.2 2023-09-30T22:29:11.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.940+0000 7f261e5a8700 1 -- 172.21.15.16:0/84756727 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2618074ea0 con 0x7f261809e410 2023-09-30T22:29:11.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.940+0000 7f2614ff9700 1 -- 172.21.15.16:0/84756727 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2610017020 con 0x7f261809e410 2023-09-30T22:29:11.942 INFO:teuthology.orchestra.run.smithi016.stdout:38654705694 2023-09-30T22:29:11.943 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:29:11.942+0000 7f90c460e240 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-09-30T22:29:11.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.943+0000 7f261e5a8700 1 -- 172.21.15.16:0/84756727 >> v1:172.21.15.16:6816/118285 conn(0x7f260402c1b0 legacy=0x7f260402e660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:11.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.943+0000 7f261e5a8700 1 -- 172.21.15.16:0/84756727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f261809e410 msgr2=0x7f26181c9010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:11.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.943+0000 7f261e5a8700 1 --2- 172.21.15.16:0/84756727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f261809e410 0x7f26181c9010 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f261000c7a0 tx=0x7f261000cab0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.943+0000 7f261e5a8700 1 -- 172.21.15.16:0/84756727 shutdown_connections 2023-09-30T22:29:11.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.943+0000 7f261e5a8700 1 --2- 172.21.15.16:0/84756727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f261809e410 0x7f26181c9010 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.943+0000 7f261e5a8700 1 --2- 172.21.15.16:0/84756727 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f261809d6f0 0x7f26181c8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.943+0000 7f261e5a8700 1 --2- 172.21.15.16:0/84756727 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f261809e000 0x7f26181c8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:11.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.943+0000 7f261e5a8700 1 -- 172.21.15.16:0/84756727 >> 172.21.15.16:0/84756727 conn(0x7f2618066f50 msgr2=0x7f2618071030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:11.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.943+0000 7f261e5a8700 1 -- 172.21.15.16:0/84756727 shutdown_connections 2023-09-30T22:29:11.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.943+0000 7f261e5a8700 1 -- 172.21.15.16:0/84756727 wait complete. 2023-09-30T22:29:11.958 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705695 got 38654705694 for osd.0 2023-09-30T22:29:11.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:11.969+0000 7eff110dd700 1 -- 172.21.15.16:0/1482057184 >> v1:172.21.15.16:6816/118285 conn(0x7efef002c1b0 legacy=0x7efef002e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/118285 2023-09-30T22:29:12.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:12.019+0000 7eff13b42700 1 -- 172.21.15.16:0/1482057184 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7eff0c12db60 con 0x7eff0c133420 2023-09-30T22:29:12.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:12.020+0000 7eff027fc700 1 -- 172.21.15.16:0/1482057184 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7eff08011360 con 0x7eff0c133420 2023-09-30T22:29:12.021 INFO:teuthology.orchestra.run.smithi016.stdout:38654705694 2023-09-30T22:29:12.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:12.022+0000 7eff13b42700 1 -- 172.21.15.16:0/1482057184 >> v1:172.21.15.16:6816/118285 conn(0x7efef002c1b0 legacy=0x7efef002e660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:12.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:12.022+0000 7eff13b42700 1 -- 172.21.15.16:0/1482057184 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7eff0c133420 msgr2=0x7eff0c130220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:12.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:12.022+0000 7eff13b42700 1 --2- 172.21.15.16:0/1482057184 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7eff0c133420 0x7eff0c130220 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7eff0800c470 tx=0x7eff0800c780 comp rx=0 tx=0).stop 2023-09-30T22:29:12.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:12.022+0000 7eff13b42700 1 -- 172.21.15.16:0/1482057184 shutdown_connections 2023-09-30T22:29:12.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:12.022+0000 7eff13b42700 1 --2- 172.21.15.16:0/1482057184 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7eff0c138dc0 0x7eff0c09c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:12.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:12.022+0000 7eff13b42700 1 --2- 172.21.15.16:0/1482057184 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7eff0c133420 0x7eff0c130220 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:12.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:12.022+0000 7eff13b42700 1 --2- 172.21.15.16:0/1482057184 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff0c1337f0 0x7eff0c130760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:12.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:12.022+0000 7eff13b42700 1 -- 172.21.15.16:0/1482057184 >> 172.21.15.16:0/1482057184 conn(0x7eff0c0a4210 msgr2=0x7eff0c0a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:12.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:12.022+0000 7eff13b42700 1 -- 172.21.15.16:0/1482057184 shutdown_connections 2023-09-30T22:29:12.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:12.022+0000 7eff13b42700 1 -- 172.21.15.16:0/1482057184 wait complete. 2023-09-30T22:29:12.037 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705695 got 38654705694 for osd.1 2023-09-30T22:29:12.040 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:29:12.039+0000 7f90c460e240 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-09-30T22:29:12.216 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:29:12.215+0000 7f90c460e240 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-09-30T22:29:12.297 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:29:12.295+0000 7f90c460e240 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-09-30T22:29:12.874 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:29:12.883 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:29:12.959 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:29:13.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.024+0000 7f272c7f6700 1 Processor -- start 2023-09-30T22:29:13.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.024+0000 7f272c7f6700 1 -- start start 2023-09-30T22:29:13.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.025+0000 7f272c7f6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2724130540 0x7f2724130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:13.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.025+0000 7f272c7f6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f272406fcf0 0x7f2724138570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:13.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.025+0000 7f272c7f6700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27240716a0 con 0x7f2724130e50 2023-09-30T22:29:13.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.025+0000 7f272c7f6700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f272409a390 con 0x7f2724130540 2023-09-30T22:29:13.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.025+0000 7f272c7f6700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2724071290 con 0x7f272406fcf0 2023-09-30T22:29:13.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.025+0000 7f272a592700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2724130540 0x7f2724130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:13.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.025+0000 7f272a592700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2724130540 0x7f2724130910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:53966/0 (socket says 172.21.15.16:53966) 2023-09-30T22:29:13.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.025+0000 7f272a592700 1 -- 172.21.15.16:0/578839164 learned_addr learned my addr 172.21.15.16:0/578839164 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:13.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.025+0000 7f272ad93700 1 --2- 172.21.15.16:0/578839164 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f272406fcf0 0x7f2724138570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:13.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.025+0000 7f272ad93700 1 -- 172.21.15.16:0/578839164 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2724130540 msgr2=0x7f2724130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:13.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.025+0000 7f272ad93700 1 --2- 172.21.15.16:0/578839164 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2724130540 0x7f2724130910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.025+0000 7f272ad93700 1 -- 172.21.15.16:0/578839164 >> v1:172.21.15.16:6789/0 conn(0x7f2724130e50 legacy=0x7f272406f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:13.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.025+0000 7f2729590700 1 -- 172.21.15.16:0/578839164 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1879726106 0 0) 0x7f27240716a0 con 0x7f2724130e50 2023-09-30T22:29:13.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.026+0000 7f272ad93700 1 -- 172.21.15.16:0/578839164 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2724130340 con 0x7f272406fcf0 2023-09-30T22:29:13.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.026+0000 7f272ad93700 1 --2- 172.21.15.16:0/578839164 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f272406fcf0 0x7f2724138570 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f271c00bfd0 tx=0x7f271c00d330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7de6c2577a643ba server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:13.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.026+0000 7f2729590700 1 -- 172.21.15.16:0/578839164 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f271c010070 con 0x7f272406fcf0 2023-09-30T22:29:13.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.026+0000 7f2729590700 1 -- 172.21.15.16:0/578839164 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f271c00b2e0 con 0x7f272406fcf0 2023-09-30T22:29:13.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.026+0000 7f2729590700 1 -- 172.21.15.16:0/578839164 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f271c00dc70 con 0x7f272406fcf0 2023-09-30T22:29:13.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.027+0000 7f272c7f6700 1 -- 172.21.15.16:0/578839164 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f272406fcf0 msgr2=0x7f2724138570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:13.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.027+0000 7f272c7f6700 1 --2- 172.21.15.16:0/578839164 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f272406fcf0 0x7f2724138570 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f271c00bfd0 tx=0x7f271c00d330 comp rx=0 tx=0).stop 2023-09-30T22:29:13.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.027+0000 7f272c7f6700 1 -- 172.21.15.16:0/578839164 shutdown_connections 2023-09-30T22:29:13.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.027+0000 7f272c7f6700 1 --2- 172.21.15.16:0/578839164 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f272406fcf0 0x7f2724138570 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.027+0000 7f272c7f6700 1 --2- 172.21.15.16:0/578839164 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2724130540 0x7f2724130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.027+0000 7f272c7f6700 1 -- 172.21.15.16:0/578839164 >> 172.21.15.16:0/578839164 conn(0x7f27240a2aa0 msgr2=0x7f27240a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:13.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.027+0000 7f272c7f6700 1 -- 172.21.15.16:0/578839164 shutdown_connections 2023-09-30T22:29:13.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.027+0000 7f272c7f6700 1 -- 172.21.15.16:0/578839164 wait complete. 2023-09-30T22:29:13.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.028+0000 7f272c7f6700 1 Processor -- start 2023-09-30T22:29:13.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.028+0000 7f272c7f6700 1 -- start start 2023-09-30T22:29:13.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.029+0000 7f272c7f6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f272406fcf0 0x7f272411e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:13.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.029+0000 7f272c7f6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2724130540 0x7f272411ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:13.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.029+0000 7f272c7f6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2724130e50 0x7f272411f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:13.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.029+0000 7f272c7f6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2724121310 con 0x7f2724130e50 2023-09-30T22:29:13.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.029+0000 7f272c7f6700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2724122310 con 0x7f2724130540 2023-09-30T22:29:13.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.029+0000 7f272c7f6700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2724121d70 con 0x7f272406fcf0 2023-09-30T22:29:13.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.029+0000 7f2729d91700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2724130540 0x7f272411ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:13.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.029+0000 7f272ad93700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2724130e50 0x7f272411f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:13.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.029+0000 7f2729d91700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2724130540 0x7f272411ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:53976/0 (socket says 172.21.15.16:53976) 2023-09-30T22:29:13.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.029+0000 7f2729d91700 1 -- 172.21.15.16:0/3535645791 learned_addr learned my addr 172.21.15.16:0/3535645791 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:13.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.029+0000 7f272a592700 1 --2- 172.21.15.16:0/3535645791 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f272406fcf0 0x7f272411e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:13.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.030+0000 7f272a592700 1 -- 172.21.15.16:0/3535645791 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2724130540 msgr2=0x7f272411ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:13.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.030+0000 7f272a592700 1 --2- 172.21.15.16:0/3535645791 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2724130540 0x7f272411ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.030+0000 7f272a592700 1 -- 172.21.15.16:0/3535645791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2724130e50 msgr2=0x7f272411f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:13.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.030+0000 7f272a592700 1 --2- 172.21.15.16:0/3535645791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2724130e50 0x7f272411f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.030+0000 7f272a592700 1 -- 172.21.15.16:0/3535645791 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f271c00bc80 con 0x7f272406fcf0 2023-09-30T22:29:13.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.030+0000 7f272a592700 1 --2- 172.21.15.16:0/3535645791 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f272406fcf0 0x7f272411e840 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f272000e910 tx=0x7f272000ec20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:13.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.030+0000 7f271affd700 1 -- 172.21.15.16:0/3535645791 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2720013680 con 0x7f272406fcf0 2023-09-30T22:29:13.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.030+0000 7f271affd700 1 -- 172.21.15.16:0/3535645791 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27200137e0 con 0x7f272406fcf0 2023-09-30T22:29:13.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.030+0000 7f271affd700 1 -- 172.21.15.16:0/3535645791 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2720013b70 con 0x7f272406fcf0 2023-09-30T22:29:13.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.030+0000 7f272c7f6700 1 -- 172.21.15.16:0/3535645791 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27240a0730 con 0x7f272406fcf0 2023-09-30T22:29:13.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.030+0000 7f272c7f6700 1 -- 172.21.15.16:0/3535645791 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f272412cbb0 con 0x7f272406fcf0 2023-09-30T22:29:13.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.031+0000 7f271affd700 1 -- 172.21.15.16:0/3535645791 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7f2720013cd0 con 0x7f272406fcf0 2023-09-30T22:29:13.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.032+0000 7f272c7f6700 1 -- 172.21.15.16:0/3535645791 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2724074ea0 con 0x7f272406fcf0 2023-09-30T22:29:13.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.032+0000 7f2729d91700 1 -- 172.21.15.16:0/3535645791 >> v1:172.21.15.16:6816/118285 conn(0x7f270802c1b0 legacy=0x7f270802e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/118285 2023-09-30T22:29:13.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.032+0000 7f271affd700 1 -- 172.21.15.16:0/3535645791 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3922+0+0 (secure 0 0 0) 0x7f272003bb10 con 0x7f272406fcf0 2023-09-30T22:29:13.037 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:29:13.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.036+0000 7f271affd700 1 -- 172.21.15.16:0/3535645791 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7f2720011db0 con 0x7f272406fcf0 2023-09-30T22:29:13.040 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:29:13.038+0000 7f90c460e240 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-09-30T22:29:13.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.071+0000 7f609f91b700 1 Processor -- start 2023-09-30T22:29:13.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.071+0000 7f609f91b700 1 -- start start 2023-09-30T22:29:13.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.072+0000 7f609f91b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6098131810 0x7f6098131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:13.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.072+0000 7f609f91b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f609809a580 0x7f6098139660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:13.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.072+0000 7f609f91b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60981313c0 con 0x7f6098099a40 2023-09-30T22:29:13.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.072+0000 7f609f91b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f609806e8c0 con 0x7f609809a580 2023-09-30T22:29:13.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.072+0000 7f609f91b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6098071d20 con 0x7f6098131810 2023-09-30T22:29:13.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.072+0000 7f609ceb6700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6098099a40 0x7f6098099e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40344/0 (socket says 172.21.15.16:40344) 2023-09-30T22:29:13.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.073+0000 7f609ceb6700 1 -- 172.21.15.16:0/4243624497 learned_addr learned my addr 172.21.15.16:0/4243624497 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:29:13.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.073+0000 7f609deb8700 1 --2- 172.21.15.16:0/4243624497 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f609809a580 0x7f6098139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:13.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.073+0000 7f609d6b7700 1 --2- 172.21.15.16:0/4243624497 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6098131810 0x7f6098131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:13.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.073+0000 7f609d6b7700 1 -- 172.21.15.16:0/4243624497 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f609809a580 msgr2=0x7f6098139660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:13.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.073+0000 7f609d6b7700 1 --2- 172.21.15.16:0/4243624497 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f609809a580 0x7f6098139660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.073+0000 7f609d6b7700 1 -- 172.21.15.16:0/4243624497 >> v1:172.21.15.16:6789/0 conn(0x7f6098099a40 legacy=0x7f6098099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:13.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.073+0000 7f609d6b7700 1 -- 172.21.15.16:0/4243624497 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60980721a0 con 0x7f6098131810 2023-09-30T22:29:13.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.073+0000 7f609d6b7700 1 --2- 172.21.15.16:0/4243624497 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6098131810 0x7f6098131be0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6094009870 tx=0x7f6094009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14064169fd487cd5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:13.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.074+0000 7f608ffff700 1 -- 172.21.15.16:0/4243624497 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6094011830 con 0x7f6098131810 2023-09-30T22:29:13.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.074+0000 7f608ffff700 1 -- 172.21.15.16:0/4243624497 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6094011990 con 0x7f6098131810 2023-09-30T22:29:13.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.074+0000 7f608ffff700 1 -- 172.21.15.16:0/4243624497 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6094011d20 con 0x7f6098131810 2023-09-30T22:29:13.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.075+0000 7f609f91b700 1 -- 172.21.15.16:0/4243624497 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6098131810 msgr2=0x7f6098131be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:13.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.075+0000 7f609f91b700 1 --2- 172.21.15.16:0/4243624497 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6098131810 0x7f6098131be0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6094009870 tx=0x7f6094009b80 comp rx=0 tx=0).stop 2023-09-30T22:29:13.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.075+0000 7f609f91b700 1 -- 172.21.15.16:0/4243624497 shutdown_connections 2023-09-30T22:29:13.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.075+0000 7f609f91b700 1 --2- 172.21.15.16:0/4243624497 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f609809a580 0x7f6098139660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.075+0000 7f609f91b700 1 --2- 172.21.15.16:0/4243624497 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6098131810 0x7f6098131be0 secure :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6094009870 tx=0x7f6094009b80 comp rx=0 tx=0).stop 2023-09-30T22:29:13.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.075+0000 7f609f91b700 1 -- 172.21.15.16:0/4243624497 >> 172.21.15.16:0/4243624497 conn(0x7f60980a2aa0 msgr2=0x7f60980a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:13.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.075+0000 7f609f91b700 1 -- 172.21.15.16:0/4243624497 shutdown_connections 2023-09-30T22:29:13.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.075+0000 7f609f91b700 1 -- 172.21.15.16:0/4243624497 wait complete. 2023-09-30T22:29:13.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.077+0000 7f609f91b700 1 Processor -- start 2023-09-30T22:29:13.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.077+0000 7f609f91b700 1 -- start start 2023-09-30T22:29:13.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.077+0000 7f609f91b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6098099a40 0x7f609811f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:13.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.077+0000 7f609f91b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f609809a580 0x7f609811f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:13.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.077+0000 7f609f91b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f609811eb80 0x7f609811fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:13.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.077+0000 7f609f91b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6098070b20 con 0x7f609811eb80 2023-09-30T22:29:13.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.077+0000 7f609f91b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f609811e8c0 con 0x7f609809a580 2023-09-30T22:29:13.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.077+0000 7f609f91b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f609811cfd0 con 0x7f6098099a40 2023-09-30T22:29:13.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.078+0000 7f609ceb6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f609809a580 0x7f609811f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:13.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.078+0000 7f609deb8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f609811eb80 0x7f609811fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:13.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.078+0000 7f609ceb6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f609809a580 0x7f609811f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:53990/0 (socket says 172.21.15.16:53990) 2023-09-30T22:29:13.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.078+0000 7f609deb8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f609811eb80 0x7f609811fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46982/0 (socket says 172.21.15.16:46982) 2023-09-30T22:29:13.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.078+0000 7f609ceb6700 1 -- 172.21.15.16:0/613274600 learned_addr learned my addr 172.21.15.16:0/613274600 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:13.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.078+0000 7f609d6b7700 1 --2- 172.21.15.16:0/613274600 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6098099a40 0x7f609811f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:13.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.078+0000 7f609deb8700 1 -- 172.21.15.16:0/613274600 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6098099a40 msgr2=0x7f609811f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:13.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.078+0000 7f609deb8700 1 --2- 172.21.15.16:0/613274600 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6098099a40 0x7f609811f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.078+0000 7f609deb8700 1 -- 172.21.15.16:0/613274600 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f609809a580 msgr2=0x7f609811f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:13.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.078+0000 7f609deb8700 1 --2- 172.21.15.16:0/613274600 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f609809a580 0x7f609811f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.078+0000 7f609deb8700 1 -- 172.21.15.16:0/613274600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60940094a0 con 0x7f609811eb80 2023-09-30T22:29:13.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.079+0000 7f609ceb6700 1 --2- 172.21.15.16:0/613274600 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f609809a580 0x7f609811f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:13.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.079+0000 7f609deb8700 1 --2- 172.21.15.16:0/613274600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f609811eb80 0x7f609811fce0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6090009d90 tx=0x7f609000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:13.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.079+0000 7f608dffb700 1 -- 172.21.15.16:0/613274600 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6090009c00 con 0x7f609811eb80 2023-09-30T22:29:13.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.079+0000 7f608dffb700 1 -- 172.21.15.16:0/613274600 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f609000f040 con 0x7f609811eb80 2023-09-30T22:29:13.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.079+0000 7f608dffb700 1 -- 172.21.15.16:0/613274600 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609000ce90 con 0x7f609811eb80 2023-09-30T22:29:13.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.079+0000 7f609f91b700 1 -- 172.21.15.16:0/613274600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6098120220 con 0x7f609811eb80 2023-09-30T22:29:13.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.079+0000 7f609f91b700 1 -- 172.21.15.16:0/613274600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f609811e0e0 con 0x7f609811eb80 2023-09-30T22:29:13.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.081+0000 7f609f91b700 1 -- 172.21.15.16:0/613274600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6098074ea0 con 0x7f609811eb80 2023-09-30T22:29:13.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.081+0000 7f608dffb700 1 -- 172.21.15.16:0/613274600 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7f6090014480 con 0x7f609811eb80 2023-09-30T22:29:13.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.081+0000 7f608dffb700 1 -- 172.21.15.16:0/613274600 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3922+0+0 (secure 0 0 0) 0x7f609003c420 con 0x7f609811eb80 2023-09-30T22:29:13.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.081+0000 7f609d6b7700 1 -- 172.21.15.16:0/613274600 >> v1:172.21.15.16:6816/118285 conn(0x7f607c02c200 legacy=0x7f607c02e6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/118285 2023-09-30T22:29:13.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.087+0000 7f608dffb700 1 -- 172.21.15.16:0/613274600 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7f6090007920 con 0x7f609811eb80 2023-09-30T22:29:13.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.126+0000 7f47d9b9a700 1 Processor -- start 2023-09-30T22:29:13.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.126+0000 7f47d9b9a700 1 -- start start 2023-09-30T22:29:13.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.127+0000 7f47d9b9a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f47d409d6f0 0x7f47d409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:13.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.127+0000 7f47d9b9a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f47d409e410 0x7f47d4134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:13.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.127+0000 7f47d9b9a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47d409b120 con 0x7f47d409e000 2023-09-30T22:29:13.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.127+0000 7f47d9b9a700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47d406b460 con 0x7f47d409e410 2023-09-30T22:29:13.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.127+0000 7f47d9b9a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f47d409a7e0 con 0x7f47d409d6f0 2023-09-30T22:29:13.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.127+0000 7f47d2ffd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f47d409e000 0x7f47d41339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40350/0 (socket says 172.21.15.16:40350) 2023-09-30T22:29:13.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.127+0000 7f47d2ffd700 1 -- 172.21.15.16:0/2627313226 learned_addr learned my addr 172.21.15.16:0/2627313226 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:29:13.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.127+0000 7f47d3fff700 1 --2- 172.21.15.16:0/2627313226 >> v2:172.21.15.16:3300/0 conn(0x7f47d409e410 0x7f47d4134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:13.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.128+0000 7f47d37fe700 1 --2- 172.21.15.16:0/2627313226 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f47d409d6f0 0x7f47d409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:13.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.128+0000 7f47d3fff700 1 -- 172.21.15.16:0/2627313226 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f47d409d6f0 msgr2=0x7f47d409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:13.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.128+0000 7f47d3fff700 1 --2- 172.21.15.16:0/2627313226 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f47d409d6f0 0x7f47d409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.128+0000 7f47d3fff700 1 -- 172.21.15.16:0/2627313226 >> v1:172.21.15.16:6789/0 conn(0x7f47d409e000 legacy=0x7f47d41339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:13.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.128+0000 7f47d3fff700 1 -- 172.21.15.16:0/2627313226 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d409a500 con 0x7f47d409e410 2023-09-30T22:29:13.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.128+0000 7f47d3fff700 1 --2- 172.21.15.16:0/2627313226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47d409e410 0x7f47d4134100 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f47cc009d90 tx=0x7f47cc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=538e7de5919e3125 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:13.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.131+0000 7f47d27fc700 1 -- 172.21.15.16:0/2627313226 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47cc00e070 con 0x7f47d409e410 2023-09-30T22:29:13.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.132+0000 7f47d27fc700 1 -- 172.21.15.16:0/2627313226 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47cc00bbd0 con 0x7f47d409e410 2023-09-30T22:29:13.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.132+0000 7f47d27fc700 1 -- 172.21.15.16:0/2627313226 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47cc012430 con 0x7f47d409e410 2023-09-30T22:29:13.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.132+0000 7f47d9b9a700 1 -- 172.21.15.16:0/2627313226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47d409e410 msgr2=0x7f47d4134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:13.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.132+0000 7f47d9b9a700 1 --2- 172.21.15.16:0/2627313226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47d409e410 0x7f47d4134100 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f47cc009d90 tx=0x7f47cc00b330 comp rx=0 tx=0).stop 2023-09-30T22:29:13.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.132+0000 7f47d9b9a700 1 -- 172.21.15.16:0/2627313226 shutdown_connections 2023-09-30T22:29:13.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.132+0000 7f47d9b9a700 1 --2- 172.21.15.16:0/2627313226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47d409e410 0x7f47d4134100 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.132+0000 7f47d9b9a700 1 --2- 172.21.15.16:0/2627313226 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f47d409d6f0 0x7f47d409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.132+0000 7f47d9b9a700 1 -- 172.21.15.16:0/2627313226 >> 172.21.15.16:0/2627313226 conn(0x7f47d4066f50 msgr2=0x7f47d409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:13.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.132+0000 7f47d9b9a700 1 -- 172.21.15.16:0/2627313226 shutdown_connections 2023-09-30T22:29:13.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.132+0000 7f47d9b9a700 1 -- 172.21.15.16:0/2627313226 wait complete. 2023-09-30T22:29:13.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.134+0000 7f47d9b9a700 1 Processor -- start 2023-09-30T22:29:13.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.134+0000 7f47d9b9a700 1 -- start start 2023-09-30T22:29:13.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.135+0000 7f47d9b9a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47d409d6f0 0x7f47d41a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:13.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.135+0000 7f47d9b9a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f47d409e000 0x7f47d41a65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:13.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.135+0000 7f47d9b9a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f47d409e410 0x7f47d41a6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:13.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.135+0000 7f47d9b9a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d41a7540 con 0x7f47d409d6f0 2023-09-30T22:29:13.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.135+0000 7f47d9b9a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47d41a9ab0 con 0x7f47d409e000 2023-09-30T22:29:13.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.135+0000 7f47d9b9a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f47d41a81c0 con 0x7f47d409e410 2023-09-30T22:29:13.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.135+0000 7f47d3fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f47d409e410 0x7f47d41a6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:13.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.135+0000 7f47d3fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f47d409e410 0x7f47d41a6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:58462/0 (socket says 172.21.15.16:58462) 2023-09-30T22:29:13.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.135+0000 7f47d2ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f47d409e000 0x7f47d41a65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:13.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.135+0000 7f47d3fff700 1 -- 172.21.15.16:0/2788130035 learned_addr learned my addr 172.21.15.16:0/2788130035 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:13.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.135+0000 7f47d37fe700 1 --2- 172.21.15.16:0/2788130035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47d409d6f0 0x7f47d41a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:13.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.136+0000 7f47d2ffd700 1 -- 172.21.15.16:0/2788130035 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f47d409e410 msgr2=0x7f47d41a6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:13.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.136+0000 7f47d2ffd700 1 --2- 172.21.15.16:0/2788130035 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f47d409e410 0x7f47d41a6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.136+0000 7f47d2ffd700 1 -- 172.21.15.16:0/2788130035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47d409d6f0 msgr2=0x7f47d41a6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:13.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.136+0000 7f47d2ffd700 1 --2- 172.21.15.16:0/2788130035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47d409d6f0 0x7f47d41a6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.136+0000 7f47d2ffd700 1 -- 172.21.15.16:0/2788130035 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47cc009a70 con 0x7f47d409e000 2023-09-30T22:29:13.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.136+0000 7f47d37fe700 1 --2- 172.21.15.16:0/2788130035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47d409d6f0 0x7f47d41a6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:13.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.136+0000 7f47d3fff700 1 --2- 172.21.15.16:0/2788130035 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f47d409e410 0x7f47d41a6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:13.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.136+0000 7f47d2ffd700 1 --2- 172.21.15.16:0/2788130035 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f47d409e000 0x7f47d41a65b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f47c800c4d0 tx=0x7f47c800c7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:13.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.136+0000 7f47bbfff700 1 -- 172.21.15.16:0/2788130035 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c8005d70 con 0x7f47d409e000 2023-09-30T22:29:13.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.136+0000 7f47bbfff700 1 -- 172.21.15.16:0/2788130035 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47c800ce80 con 0x7f47d409e000 2023-09-30T22:29:13.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.136+0000 7f47d9b9a700 1 -- 172.21.15.16:0/2788130035 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47d41a8e80 con 0x7f47d409e000 2023-09-30T22:29:13.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.137+0000 7f47bbfff700 1 -- 172.21.15.16:0/2788130035 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47c80072d0 con 0x7f47d409e000 2023-09-30T22:29:13.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.137+0000 7f47d9b9a700 1 -- 172.21.15.16:0/2788130035 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47d409c710 con 0x7f47d409e000 2023-09-30T22:29:13.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.137+0000 7f47bbfff700 1 -- 172.21.15.16:0/2788130035 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7f47c8007430 con 0x7f47d409e000 2023-09-30T22:29:13.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.138+0000 7f47bbfff700 1 -- 172.21.15.16:0/2788130035 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3922+0+0 (secure 0 0 0) 0x7f47c803abb0 con 0x7f47d409e000 2023-09-30T22:29:13.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.138+0000 7f47d37fe700 1 -- 172.21.15.16:0/2788130035 >> v1:172.21.15.16:6816/118285 conn(0x7f47c002c200 legacy=0x7f47c002e6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/118285 2023-09-30T22:29:13.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.140+0000 7f47d9b9a700 1 -- 172.21.15.16:0/2788130035 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47d4072370 con 0x7f47d409e000 2023-09-30T22:29:13.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.145+0000 7f47bbfff700 1 -- 172.21.15.16:0/2788130035 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7f47c8017080 con 0x7f47d409e000 2023-09-30T22:29:13.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.232+0000 7f899d746700 1 Processor -- start 2023-09-30T22:29:13.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.232+0000 7f899d746700 1 -- start start 2023-09-30T22:29:13.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.233+0000 7f899d746700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f899812cf60 0x7f899806f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:13.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.233+0000 7f899d746700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f899812f8b0 0x7f899812fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:13.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.233+0000 7f899d746700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f899812cd90 con 0x7f8998130260 2023-09-30T22:29:13.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.233+0000 7f899d746700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f899812d330 con 0x7f899812cf60 2023-09-30T22:29:13.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.233+0000 7f899d746700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89981306d0 con 0x7f899812f8b0 2023-09-30T22:29:13.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.233+0000 7f89977fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8998130260 0x7f8998136c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40352/0 (socket says 172.21.15.16:40352) 2023-09-30T22:29:13.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.233+0000 7f89977fe700 1 -- 172.21.15.16:0/3507988447 learned_addr learned my addr 172.21.15.16:0/3507988447 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:29:13.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.233+0000 7f89967fc700 1 --2- 172.21.15.16:0/3507988447 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f899812f8b0 0x7f899812fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:13.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.233+0000 7f89967fc700 1 -- 172.21.15.16:0/3507988447 >> v2:172.21.15.16:3300/0 conn(0x7f899812cf60 msgr2=0x7f899806f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:13.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.233+0000 7f89967fc700 1 --2- 172.21.15.16:0/3507988447 >> v2:172.21.15.16:3300/0 conn(0x7f899812cf60 0x7f899806f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.233+0000 7f2729d91700 1 -- 172.21.15.16:0/3535645791 >> v1:172.21.15.16:6816/118285 conn(0x7f270802c1b0 legacy=0x7f270802e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/118285 2023-09-30T22:29:13.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.233+0000 7f89967fc700 1 -- 172.21.15.16:0/3507988447 >> v1:172.21.15.16:6789/0 conn(0x7f8998130260 legacy=0x7f8998136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:13.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.234+0000 7f89967fc700 1 -- 172.21.15.16:0/3507988447 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8998071780 con 0x7f899812f8b0 2023-09-30T22:29:13.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.234+0000 7f89967fc700 1 --2- 172.21.15.16:0/3507988447 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f899812f8b0 0x7f899812fd20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f898c00bae0 tx=0x7f898c00bdf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b75a7ced4e186e6b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:13.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.234+0000 7f8995ffb700 1 -- 172.21.15.16:0/3507988447 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f898c00dc20 con 0x7f899812f8b0 2023-09-30T22:29:13.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.234+0000 7f8995ffb700 1 -- 172.21.15.16:0/3507988447 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f898c00dd80 con 0x7f899812f8b0 2023-09-30T22:29:13.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.234+0000 7f8995ffb700 1 -- 172.21.15.16:0/3507988447 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f898c017630 con 0x7f899812f8b0 2023-09-30T22:29:13.238 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:29:13.234+0000 7f90c460e240 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-09-30T22:29:13.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.235+0000 7f899d746700 1 -- 172.21.15.16:0/3507988447 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f899812f8b0 msgr2=0x7f899812fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:13.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.235+0000 7f899d746700 1 --2- 172.21.15.16:0/3507988447 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f899812f8b0 0x7f899812fd20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f898c00bae0 tx=0x7f898c00bdf0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.235+0000 7f899d746700 1 -- 172.21.15.16:0/3507988447 shutdown_connections 2023-09-30T22:29:13.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.235+0000 7f899d746700 1 --2- 172.21.15.16:0/3507988447 >> v2:172.21.15.16:3300/0 conn(0x7f899812cf60 0x7f899806f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.235+0000 7f899d746700 1 --2- 172.21.15.16:0/3507988447 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f899812f8b0 0x7f899812fd20 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.235+0000 7f899d746700 1 -- 172.21.15.16:0/3507988447 >> 172.21.15.16:0/3507988447 conn(0x7f89980a2aa0 msgr2=0x7f89980a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:13.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.235+0000 7f899d746700 1 -- 172.21.15.16:0/3507988447 shutdown_connections 2023-09-30T22:29:13.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.235+0000 7f899d746700 1 -- 172.21.15.16:0/3507988447 wait complete. 2023-09-30T22:29:13.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.236+0000 7f899d746700 1 Processor -- start 2023-09-30T22:29:13.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.236+0000 7f899d746700 1 -- start start 2023-09-30T22:29:13.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.237+0000 7f899d746700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f899812cf60 0x7f899809b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:13.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.237+0000 7f899d746700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f899812f8b0 0x7f899809b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:13.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.237+0000 7f899d746700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8998130260 0x7f899809bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:13.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.237+0000 7f899d746700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f899811d300 con 0x7f8998130260 2023-09-30T22:29:13.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.237+0000 7f899d746700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f899809dd60 con 0x7f899812cf60 2023-09-30T22:29:13.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.237+0000 7f899d746700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f899812d7b0 con 0x7f899812f8b0 2023-09-30T22:29:13.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.237+0000 7f8996ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f899812cf60 0x7f899809b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:13.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.237+0000 7f8996ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f899812cf60 0x7f899809b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54020/0 (socket says 172.21.15.16:54020) 2023-09-30T22:29:13.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.237+0000 7f8996ffd700 1 -- 172.21.15.16:0/1566520503 learned_addr learned my addr 172.21.15.16:0/1566520503 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:13.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.237+0000 7f8996ffd700 1 -- 172.21.15.16:0/1566520503 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f899812f8b0 msgr2=0x7f899809b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:13.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.237+0000 7f8996ffd700 1 --2- 172.21.15.16:0/1566520503 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f899812f8b0 0x7f899809b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.237+0000 7f8996ffd700 1 -- 172.21.15.16:0/1566520503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8998130260 msgr2=0x7f899809bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:13.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.237+0000 7f8996ffd700 1 --2- 172.21.15.16:0/1566520503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8998130260 0x7f899809bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.237+0000 7f8996ffd700 1 -- 172.21.15.16:0/1566520503 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f898c00b740 con 0x7f899812cf60 2023-09-30T22:29:13.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.238+0000 7f8996ffd700 1 --2- 172.21.15.16:0/1566520503 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f899812cf60 0x7f899809b160 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f898800b530 tx=0x7f898800b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:13.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.238+0000 7f897f7fe700 1 -- 172.21.15.16:0/1566520503 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8988011830 con 0x7f899812cf60 2023-09-30T22:29:13.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.238+0000 7f897f7fe700 1 -- 172.21.15.16:0/1566520503 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89880119b0 con 0x7f899812cf60 2023-09-30T22:29:13.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.238+0000 7f897f7fe700 1 -- 172.21.15.16:0/1566520503 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8988011d60 con 0x7f899812cf60 2023-09-30T22:29:13.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.238+0000 7f899d746700 1 -- 172.21.15.16:0/1566520503 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8998129fa0 con 0x7f899812cf60 2023-09-30T22:29:13.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.238+0000 7f899d746700 1 -- 172.21.15.16:0/1566520503 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89981c8460 con 0x7f899812cf60 2023-09-30T22:29:13.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.239+0000 7f899d746700 1 -- 172.21.15.16:0/1566520503 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f899811d850 con 0x7f899812cf60 2023-09-30T22:29:13.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.240+0000 7f897f7fe700 1 -- 172.21.15.16:0/1566520503 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7f898801a400 con 0x7f899812cf60 2023-09-30T22:29:13.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.240+0000 7f897f7fe700 1 -- 172.21.15.16:0/1566520503 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3922+0+0 (secure 0 0 0) 0x7f898803bfe0 con 0x7f899812cf60 2023-09-30T22:29:13.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.240+0000 7f89967fc700 1 -- 172.21.15.16:0/1566520503 >> v1:172.21.15.16:6816/118285 conn(0x7f898402c1b0 legacy=0x7f898402e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/118285 2023-09-30T22:29:13.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.247+0000 7f897f7fe700 1 -- 172.21.15.16:0/1566520503 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7f8988021710 con 0x7f899812cf60 2023-09-30T22:29:13.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.275+0000 7f609f91b700 1 -- 172.21.15.16:0/613274600 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f609809fcb0 con 0x7f609811eb80 2023-09-30T22:29:13.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.276+0000 7f608dffb700 1 -- 172.21.15.16:0/613274600 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6090018070 con 0x7f609811eb80 2023-09-30T22:29:13.278 INFO:teuthology.orchestra.run.smithi016.stdout:38654705695 2023-09-30T22:29:13.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.278+0000 7f609f91b700 1 -- 172.21.15.16:0/613274600 >> v1:172.21.15.16:6816/118285 conn(0x7f607c02c200 legacy=0x7f607c02e6b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:13.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.278+0000 7f609f91b700 1 -- 172.21.15.16:0/613274600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f609811eb80 msgr2=0x7f609811fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:13.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.278+0000 7f609f91b700 1 --2- 172.21.15.16:0/613274600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f609811eb80 0x7f609811fce0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6090009d90 tx=0x7f609000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.278+0000 7f609f91b700 1 -- 172.21.15.16:0/613274600 shutdown_connections 2023-09-30T22:29:13.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.278+0000 7f609f91b700 1 --2- 172.21.15.16:0/613274600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f609811eb80 0x7f609811fce0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.278+0000 7f609f91b700 1 --2- 172.21.15.16:0/613274600 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6098099a40 0x7f609811f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.278+0000 7f609f91b700 1 --2- 172.21.15.16:0/613274600 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f609809a580 0x7f609811f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.278+0000 7f609f91b700 1 -- 172.21.15.16:0/613274600 >> 172.21.15.16:0/613274600 conn(0x7f60980a2aa0 msgr2=0x7f60980a1410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:13.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.278+0000 7f609f91b700 1 -- 172.21.15.16:0/613274600 shutdown_connections 2023-09-30T22:29:13.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.278+0000 7f609f91b700 1 -- 172.21.15.16:0/613274600 wait complete. 2023-09-30T22:29:13.298 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705696 got 38654705695 for osd.2 2023-09-30T22:29:13.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.339+0000 7f47d37fe700 1 -- 172.21.15.16:0/2788130035 >> v1:172.21.15.16:6816/118285 conn(0x7f47c002c200 legacy=0x7f47c002e6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/118285 2023-09-30T22:29:13.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.351+0000 7f272c7f6700 1 -- 172.21.15.16:0/3535645791 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2724121f90 con 0x7f272406fcf0 2023-09-30T22:29:13.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.351+0000 7f271affd700 1 -- 172.21.15.16:0/3535645791 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2724121f90 con 0x7f272406fcf0 2023-09-30T22:29:13.353 INFO:teuthology.orchestra.run.smithi016.stdout:38654705694 2023-09-30T22:29:13.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.354+0000 7f272c7f6700 1 -- 172.21.15.16:0/3535645791 >> v1:172.21.15.16:6816/118285 conn(0x7f270802c1b0 legacy=0x7f270802e660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:13.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.354+0000 7f272c7f6700 1 -- 172.21.15.16:0/3535645791 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f272406fcf0 msgr2=0x7f272411e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:13.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.354+0000 7f272c7f6700 1 --2- 172.21.15.16:0/3535645791 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f272406fcf0 0x7f272411e840 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f272000e910 tx=0x7f272000ec20 comp rx=0 tx=0).stop 2023-09-30T22:29:13.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.354+0000 7f272c7f6700 1 -- 172.21.15.16:0/3535645791 shutdown_connections 2023-09-30T22:29:13.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.354+0000 7f272c7f6700 1 --2- 172.21.15.16:0/3535645791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2724130e50 0x7f272411f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.354+0000 7f272c7f6700 1 --2- 172.21.15.16:0/3535645791 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f272406fcf0 0x7f272411e840 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.354+0000 7f272c7f6700 1 --2- 172.21.15.16:0/3535645791 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2724130540 0x7f272411ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.354+0000 7f272c7f6700 1 -- 172.21.15.16:0/3535645791 >> 172.21.15.16:0/3535645791 conn(0x7f27240a2aa0 msgr2=0x7f272412d340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:13.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.355+0000 7f272c7f6700 1 -- 172.21.15.16:0/3535645791 shutdown_connections 2023-09-30T22:29:13.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.355+0000 7f272c7f6700 1 -- 172.21.15.16:0/3535645791 wait complete. 2023-09-30T22:29:13.375 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705695 got 38654705694 for osd.3 2023-09-30T22:29:13.385 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:29:13.384+0000 7f90c460e240 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-09-30T22:29:13.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.433+0000 7f899d746700 1 -- 172.21.15.16:0/1566520503 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8998074ea0 con 0x7f899812cf60 2023-09-30T22:29:13.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.433+0000 7f897f7fe700 1 -- 172.21.15.16:0/1566520503 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8988028090 con 0x7f899812cf60 2023-09-30T22:29:13.435 INFO:teuthology.orchestra.run.smithi016.stdout:38654705694 2023-09-30T22:29:13.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.436+0000 7f899d746700 1 -- 172.21.15.16:0/1566520503 >> v1:172.21.15.16:6816/118285 conn(0x7f898402c1b0 legacy=0x7f898402e660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:13.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.436+0000 7f899d746700 1 -- 172.21.15.16:0/1566520503 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f899812cf60 msgr2=0x7f899809b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:13.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.436+0000 7f899d746700 1 --2- 172.21.15.16:0/1566520503 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f899812cf60 0x7f899809b160 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f898800b530 tx=0x7f898800b8f0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.436+0000 7f899d746700 1 -- 172.21.15.16:0/1566520503 shutdown_connections 2023-09-30T22:29:13.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.436+0000 7f899d746700 1 --2- 172.21.15.16:0/1566520503 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f899812cf60 0x7f899809b160 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.436+0000 7f899d746700 1 --2- 172.21.15.16:0/1566520503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8998130260 0x7f899809bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.436+0000 7f899d746700 1 --2- 172.21.15.16:0/1566520503 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f899812f8b0 0x7f899809b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.436+0000 7f899d746700 1 -- 172.21.15.16:0/1566520503 >> 172.21.15.16:0/1566520503 conn(0x7f89980a2aa0 msgr2=0x7f899806e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:13.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.436+0000 7f899d746700 1 -- 172.21.15.16:0/1566520503 shutdown_connections 2023-09-30T22:29:13.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.436+0000 7f899d746700 1 -- 172.21.15.16:0/1566520503 wait complete. 2023-09-30T22:29:13.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.443+0000 7f47d9b9a700 1 -- 172.21.15.16:0/2788130035 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f47d4074ea0 con 0x7f47d409e000 2023-09-30T22:29:13.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.444+0000 7f47bbfff700 1 -- 172.21.15.16:0/2788130035 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f47c801b3b0 con 0x7f47d409e000 2023-09-30T22:29:13.445 INFO:teuthology.orchestra.run.smithi016.stdout:38654705694 2023-09-30T22:29:13.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.446+0000 7f47d9b9a700 1 -- 172.21.15.16:0/2788130035 >> v1:172.21.15.16:6816/118285 conn(0x7f47c002c200 legacy=0x7f47c002e6b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:13.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.446+0000 7f47d9b9a700 1 -- 172.21.15.16:0/2788130035 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f47d409e000 msgr2=0x7f47d41a65b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:13.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.446+0000 7f47d9b9a700 1 --2- 172.21.15.16:0/2788130035 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f47d409e000 0x7f47d41a65b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f47c800c4d0 tx=0x7f47c800c7e0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.446+0000 7f47d9b9a700 1 -- 172.21.15.16:0/2788130035 shutdown_connections 2023-09-30T22:29:13.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.446+0000 7f47d9b9a700 1 --2- 172.21.15.16:0/2788130035 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f47d409e410 0x7f47d41a6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.446+0000 7f47d9b9a700 1 --2- 172.21.15.16:0/2788130035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47d409d6f0 0x7f47d41a6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.446+0000 7f47d9b9a700 1 --2- 172.21.15.16:0/2788130035 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f47d409e000 0x7f47d41a65b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:13.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.446+0000 7f47d9b9a700 1 -- 172.21.15.16:0/2788130035 >> 172.21.15.16:0/2788130035 conn(0x7f47d4066f50 msgr2=0x7f47d41bacf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:13.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.446+0000 7f47d9b9a700 1 -- 172.21.15.16:0/2788130035 shutdown_connections 2023-09-30T22:29:13.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:13.446+0000 7f47d9b9a700 1 -- 172.21.15.16:0/2788130035 wait complete. 2023-09-30T22:29:13.452 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705695 got 38654705694 for osd.1 2023-09-30T22:29:13.461 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705695 got 38654705694 for osd.0 2023-09-30T22:29:14.027 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:29:14.026+0000 7f90c460e240 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-09-30T22:29:14.161 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:29:14.160+0000 7f90c460e240 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-09-30T22:29:14.299 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:29:14.336 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:29:14.334+0000 7f90c460e240 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-09-30T22:29:14.375 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:29:14.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.437+0000 7f80638ee700 1 Processor -- start 2023-09-30T22:29:14.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.437+0000 7f80638ee700 1 -- start start 2023-09-30T22:29:14.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.437+0000 7f80638ee700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f805c133420 0x7f805c1337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:14.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.438+0000 7f80638ee700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f805c133e30 0x7f805c12ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:14.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.438+0000 7f80638ee700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f805c134260 0x7f805c1304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:14.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.438+0000 7f80638ee700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f805c06b990 con 0x7f805c133e30 2023-09-30T22:29:14.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.438+0000 7f80638ee700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f805c0a0590 con 0x7f805c133420 2023-09-30T22:29:14.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.438+0000 7f80638ee700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f805c09fb80 con 0x7f805c134260 2023-09-30T22:29:14.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.438+0000 7f806168a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f805c133420 0x7f805c1337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:14.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.438+0000 7f806168a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f805c133420 0x7f805c1337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54034/0 (socket says 172.21.15.16:54034) 2023-09-30T22:29:14.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.438+0000 7f806168a700 1 -- 172.21.15.16:0/3545679276 learned_addr learned my addr 172.21.15.16:0/3545679276 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:14.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.438+0000 7f8060e89700 1 --2- 172.21.15.16:0/3545679276 >> v2:172.21.15.16:3300/0 conn(0x7f805c133e30 0x7f805c12ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:14.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.438+0000 7f8061e8b700 1 --2- 172.21.15.16:0/3545679276 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f805c134260 0x7f805c1304d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:14.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.438+0000 7f806168a700 1 -- 172.21.15.16:0/3545679276 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f805c134260 msgr2=0x7f805c1304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:14.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.438+0000 7f806168a700 1 --2- 172.21.15.16:0/3545679276 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f805c134260 0x7f805c1304d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.438+0000 7f806168a700 1 -- 172.21.15.16:0/3545679276 >> v2:172.21.15.16:3300/0 conn(0x7f805c133e30 msgr2=0x7f805c12ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:14.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.438+0000 7f806168a700 1 --2- 172.21.15.16:0/3545679276 >> v2:172.21.15.16:3300/0 conn(0x7f805c133e30 0x7f805c12ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.438+0000 7f806168a700 1 -- 172.21.15.16:0/3545679276 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f805c0a0030 con 0x7f805c133420 2023-09-30T22:29:14.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.438+0000 7f806168a700 1 --2- 172.21.15.16:0/3545679276 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f805c133420 0x7f805c1337f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f80580098f0 tx=0x7f8058009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a82148aca1d20103 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:14.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.439+0000 7f8053fff700 1 -- 172.21.15.16:0/3545679276 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8058011830 con 0x7f805c133420 2023-09-30T22:29:14.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.439+0000 7f8053fff700 1 -- 172.21.15.16:0/3545679276 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8058011990 con 0x7f805c133420 2023-09-30T22:29:14.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.439+0000 7f8053fff700 1 -- 172.21.15.16:0/3545679276 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8058011d20 con 0x7f805c133420 2023-09-30T22:29:14.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.439+0000 7f80638ee700 1 -- 172.21.15.16:0/3545679276 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f805c133420 msgr2=0x7f805c1337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:14.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.439+0000 7f80638ee700 1 --2- 172.21.15.16:0/3545679276 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f805c133420 0x7f805c1337f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f80580098f0 tx=0x7f8058009c00 comp rx=0 tx=0).stop 2023-09-30T22:29:14.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.439+0000 7f80638ee700 1 -- 172.21.15.16:0/3545679276 shutdown_connections 2023-09-30T22:29:14.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.439+0000 7f80638ee700 1 --2- 172.21.15.16:0/3545679276 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f805c133420 0x7f805c1337f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.439+0000 7f80638ee700 1 --2- 172.21.15.16:0/3545679276 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f805c134260 0x7f805c1304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.439+0000 7f80638ee700 1 --2- 172.21.15.16:0/3545679276 >> v2:172.21.15.16:3300/0 conn(0x7f805c133e30 0x7f805c12ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.439+0000 7f80638ee700 1 -- 172.21.15.16:0/3545679276 >> 172.21.15.16:0/3545679276 conn(0x7f805c0a4210 msgr2=0x7f805c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:14.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.440+0000 7f80638ee700 1 -- 172.21.15.16:0/3545679276 shutdown_connections 2023-09-30T22:29:14.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.440+0000 7f80638ee700 1 -- 172.21.15.16:0/3545679276 wait complete. 2023-09-30T22:29:14.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.441+0000 7f80638ee700 1 Processor -- start 2023-09-30T22:29:14.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.441+0000 7f80638ee700 1 -- start start 2023-09-30T22:29:14.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.441+0000 7f80638ee700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f805c133420 0x7f805c09c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:14.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.441+0000 7f80638ee700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f805c133e30 0x7f805c09cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:14.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.441+0000 7f80638ee700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f805c134260 0x7f805c09d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:14.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.441+0000 7f80638ee700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f805c11ff60 con 0x7f805c134260 2023-09-30T22:29:14.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.441+0000 7f80638ee700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f805c12dc30 con 0x7f805c133e30 2023-09-30T22:29:14.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.441+0000 7f80638ee700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f805c09e050 con 0x7f805c133420 2023-09-30T22:29:14.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.441+0000 7f8060e89700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f805c133e30 0x7f805c09cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:14.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.441+0000 7f8060e89700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f805c133e30 0x7f805c09cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54038/0 (socket says 172.21.15.16:54038) 2023-09-30T22:29:14.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.441+0000 7f8060e89700 1 -- 172.21.15.16:0/1545782054 learned_addr learned my addr 172.21.15.16:0/1545782054 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:14.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.441+0000 7f806168a700 1 --2- 172.21.15.16:0/1545782054 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f805c133420 0x7f805c09c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:14.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.442+0000 7f8061e8b700 1 --2- 172.21.15.16:0/1545782054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f805c134260 0x7f805c09d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:14.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.442+0000 7f8060e89700 1 -- 172.21.15.16:0/1545782054 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f805c133420 msgr2=0x7f805c09c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:14.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.442+0000 7f8060e89700 1 --2- 172.21.15.16:0/1545782054 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f805c133420 0x7f805c09c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.442+0000 7f8060e89700 1 -- 172.21.15.16:0/1545782054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f805c134260 msgr2=0x7f805c09d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:14.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.442+0000 7f8060e89700 1 --2- 172.21.15.16:0/1545782054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f805c134260 0x7f805c09d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.442+0000 7f8060e89700 1 -- 172.21.15.16:0/1545782054 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8058009550 con 0x7f805c133e30 2023-09-30T22:29:14.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.442+0000 7f806168a700 1 --2- 172.21.15.16:0/1545782054 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f805c133420 0x7f805c09c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:14.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.442+0000 7f8061e8b700 1 --2- 172.21.15.16:0/1545782054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f805c134260 0x7f805c09d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:29:14.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.442+0000 7f8060e89700 1 --2- 172.21.15.16:0/1545782054 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f805c133e30 0x7f805c09cc20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f804c00e980 tx=0x7f804c00ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:14.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.442+0000 7f8051ffb700 1 -- 172.21.15.16:0/1545782054 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804c009710 con 0x7f805c133e30 2023-09-30T22:29:14.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.442+0000 7f8051ffb700 1 -- 172.21.15.16:0/1545782054 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f804c00eed0 con 0x7f805c133e30 2023-09-30T22:29:14.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.442+0000 7f8051ffb700 1 -- 172.21.15.16:0/1545782054 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804c00ca70 con 0x7f805c133e30 2023-09-30T22:29:14.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.442+0000 7f80638ee700 1 -- 172.21.15.16:0/1545782054 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f805c11fca0 con 0x7f805c133e30 2023-09-30T22:29:14.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.443+0000 7f80638ee700 1 -- 172.21.15.16:0/1545782054 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f805c11f920 con 0x7f805c133e30 2023-09-30T22:29:14.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.443+0000 7f8051ffb700 1 -- 172.21.15.16:0/1545782054 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 5) v1 ==== 31713+0+0 (secure 0 0 0) 0x7f804c00cbd0 con 0x7f805c133e30 2023-09-30T22:29:14.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.443+0000 7f8051ffb700 1 -- 172.21.15.16:0/1545782054 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3922+0+0 (secure 0 0 0) 0x7f804c03b9b0 con 0x7f805c133e30 2023-09-30T22:29:14.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.443+0000 7f806168a700 1 -- 172.21.15.16:0/1545782054 >> v1:172.21.15.16:6816/118285 conn(0x7f804002c1b0 legacy=0x7f804002e660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/118285 2023-09-30T22:29:14.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.444+0000 7f80638ee700 1 -- 172.21.15.16:0/1545782054 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f805c074ea0 con 0x7f805c133e30 2023-09-30T22:29:14.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.448+0000 7f8051ffb700 1 -- 172.21.15.16:0/1545782054 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7f804c017080 con 0x7f805c133e30 2023-09-30T22:29:14.453 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:29:14.461 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:29:14.460+0000 7f90c460e240 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-09-30T22:29:14.463 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:29:14.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.484+0000 7f8051ffb700 1 -- 172.21.15.16:0/1545782054 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mgrmap(e 6) v1 ==== 35253+0+0 (secure 0 0 0) 0x7f804c013990 con 0x7f805c133e30 2023-09-30T22:29:14.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.484+0000 7f8051ffb700 1 -- 172.21.15.16:0/1545782054 >> v1:172.21.15.16:6816/118285 conn(0x7f804002c1b0 legacy=0x7f804002e660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:14.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.545+0000 7f24f47e8700 1 Processor -- start 2023-09-30T22:29:14.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.545+0000 7f24f47e8700 1 -- start start 2023-09-30T22:29:14.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.546+0000 7f24f47e8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24ec0a0a30 0x7f24ec0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:14.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.546+0000 7f24f47e8700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f24ec134170 0x7f24ec13bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:14.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.546+0000 7f24f47e8700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24ec09edd0 con 0x7f24ec0a1340 2023-09-30T22:29:14.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.546+0000 7f24f47e8700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24ec09e1f0 con 0x7f24ec134170 2023-09-30T22:29:14.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.546+0000 7f24f47e8700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f24ec09dc50 con 0x7f24ec0a0a30 2023-09-30T22:29:14.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.546+0000 7f24f2d85700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f24ec134170 0x7f24ec13bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:14.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.546+0000 7f24f1d83700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f24ec0a1340 0x7f24ec133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40358/0 (socket says 172.21.15.16:40358) 2023-09-30T22:29:14.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.546+0000 7f24f1d83700 1 -- 172.21.15.16:0/821395307 learned_addr learned my addr 172.21.15.16:0/821395307 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:29:14.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.546+0000 7f24f1582700 1 -- 172.21.15.16:0/821395307 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1046259140 0 0) 0x7f24ec09edd0 con 0x7f24ec0a1340 2023-09-30T22:29:14.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.546+0000 7f24f2d85700 1 -- 172.21.15.16:0/821395307 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24ec0a0a30 msgr2=0x7f24ec0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:14.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.546+0000 7f24f2d85700 1 --2- 172.21.15.16:0/821395307 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24ec0a0a30 0x7f24ec0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.547+0000 7f24f2d85700 1 -- 172.21.15.16:0/821395307 >> v1:172.21.15.16:6789/0 conn(0x7f24ec0a1340 legacy=0x7f24ec133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:14.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.547+0000 7f24f2d85700 1 -- 172.21.15.16:0/821395307 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24ec0a1fe0 con 0x7f24ec134170 2023-09-30T22:29:14.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.547+0000 7f24f2d85700 1 --2- 172.21.15.16:0/821395307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24ec134170 0x7f24ec13bd30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f24e4009d90 tx=0x7f24e400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=482be396d2271da1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:14.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.547+0000 7f24f1582700 1 -- 172.21.15.16:0/821395307 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e400e070 con 0x7f24ec134170 2023-09-30T22:29:14.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.547+0000 7f24f1582700 1 -- 172.21.15.16:0/821395307 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24e400bbd0 con 0x7f24ec134170 2023-09-30T22:29:14.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.547+0000 7f24f1582700 1 -- 172.21.15.16:0/821395307 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e4012430 con 0x7f24ec134170 2023-09-30T22:29:14.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.548+0000 7f24f47e8700 1 -- 172.21.15.16:0/821395307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24ec134170 msgr2=0x7f24ec13bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:14.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.548+0000 7f24f47e8700 1 --2- 172.21.15.16:0/821395307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24ec134170 0x7f24ec13bd30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f24e4009d90 tx=0x7f24e400b330 comp rx=0 tx=0).stop 2023-09-30T22:29:14.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.549+0000 7f24f47e8700 1 -- 172.21.15.16:0/821395307 shutdown_connections 2023-09-30T22:29:14.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.549+0000 7f24f47e8700 1 --2- 172.21.15.16:0/821395307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24ec134170 0x7f24ec13bd30 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.549+0000 7f24f47e8700 1 --2- 172.21.15.16:0/821395307 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24ec0a0a30 0x7f24ec0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.549+0000 7f24f47e8700 1 -- 172.21.15.16:0/821395307 >> 172.21.15.16:0/821395307 conn(0x7f24ec0a6550 msgr2=0x7f24ec0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:14.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.549+0000 7f24f47e8700 1 -- 172.21.15.16:0/821395307 shutdown_connections 2023-09-30T22:29:14.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.549+0000 7f24f47e8700 1 -- 172.21.15.16:0/821395307 wait complete. 2023-09-30T22:29:14.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.550+0000 7f24f47e8700 1 Processor -- start 2023-09-30T22:29:14.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.550+0000 7f24f47e8700 1 -- start start 2023-09-30T22:29:14.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.551+0000 7f24f47e8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24ec0a0a30 0x7f24ec06dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:14.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.551+0000 7f24f47e8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24ec0a1340 0x7f24ec06e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:14.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.551+0000 7f24f47e8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f24ec134170 0x7f24ec1cc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:14.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.551+0000 7f24f47e8700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24ec06ef00 con 0x7f24ec0a0a30 2023-09-30T22:29:14.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.551+0000 7f24f47e8700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24ec06f680 con 0x7f24ec134170 2023-09-30T22:29:14.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.551+0000 7f24f47e8700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f24ec071320 con 0x7f24ec0a1340 2023-09-30T22:29:14.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.551+0000 7f24f2584700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24ec0a0a30 0x7f24ec06dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:14.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.552+0000 7f24f2584700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24ec0a0a30 0x7f24ec06dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47092/0 (socket says 172.21.15.16:47092) 2023-09-30T22:29:14.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.552+0000 7f24f2584700 1 -- 172.21.15.16:0/3902400029 learned_addr learned my addr 172.21.15.16:0/3902400029 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:14.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.552+0000 7f24f2584700 1 -- 172.21.15.16:0/3902400029 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24ec0a1340 msgr2=0x7f24ec06e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:14.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.552+0000 7f24f2584700 1 --2- 172.21.15.16:0/3902400029 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24ec0a1340 0x7f24ec06e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.552+0000 7f24f2584700 1 -- 172.21.15.16:0/3902400029 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f24ec134170 msgr2=0x7f24ec1cc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:14.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.552+0000 7f24f2584700 1 --2- 172.21.15.16:0/3902400029 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f24ec134170 0x7f24ec1cc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.552+0000 7f24f2584700 1 -- 172.21.15.16:0/3902400029 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24e4009a70 con 0x7f24ec0a0a30 2023-09-30T22:29:14.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.552+0000 7f24f2584700 1 --2- 172.21.15.16:0/3902400029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24ec0a0a30 0x7f24ec06dd80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f24e800b560 tx=0x7f24e800b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:14.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.553+0000 7f24e2ffd700 1 -- 172.21.15.16:0/3902400029 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e8011830 con 0x7f24ec0a0a30 2023-09-30T22:29:14.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.553+0000 7f24e2ffd700 1 -- 172.21.15.16:0/3902400029 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24e8011990 con 0x7f24ec0a0a30 2023-09-30T22:29:14.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.553+0000 7f24e2ffd700 1 -- 172.21.15.16:0/3902400029 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e8011d40 con 0x7f24ec0a0a30 2023-09-30T22:29:14.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.553+0000 7f24f47e8700 1 -- 172.21.15.16:0/3902400029 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24ec070570 con 0x7f24ec0a0a30 2023-09-30T22:29:14.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.553+0000 7f24f47e8700 1 -- 172.21.15.16:0/3902400029 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24ec0a0220 con 0x7f24ec0a0a30 2023-09-30T22:29:14.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.555+0000 7f24e2ffd700 1 -- 172.21.15.16:0/3902400029 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 6) v1 ==== 35253+0+0 (secure 0 0 0) 0x7f24e801a400 con 0x7f24ec0a0a30 2023-09-30T22:29:14.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.555+0000 7f24e2ffd700 1 -- 172.21.15.16:0/3902400029 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 4055+0+0 (secure 0 0 0) 0x7f24e803f560 con 0x7f24ec0a0a30 2023-09-30T22:29:14.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.556+0000 7f24f47e8700 1 -- 172.21.15.16:0/3902400029 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24d40052e0 con 0x7f24ec0a0a30 2023-09-30T22:29:14.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.563+0000 7f24e2ffd700 1 -- 172.21.15.16:0/3902400029 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7f24e8015080 con 0x7f24ec0a0a30 2023-09-30T22:29:14.607 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:29:14.602+0000 7f903fca0700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-30T22:29:14.607 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:29:14.602+0000 7f903fca0700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-30T22:29:14.607 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:29:14.602+0000 7f903fca0700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-30T22:29:14.607 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:29:14.602+0000 7f903fca0700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-30T22:29:14.607 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:29:14.602+0000 7f903fca0700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-30T22:29:14.610 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:29:14.607+0000 7f90434a7700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-30T22:29:14.610 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:29:14.607+0000 7f90434a7700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-30T22:29:14.610 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:29:14.607+0000 7f90434a7700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-30T22:29:14.610 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:29:14.607+0000 7f90434a7700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-30T22:29:14.611 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:29:14.607+0000 7f90434a7700 -1 client.0 error registering admin socket command: (17) File exists 2023-09-30T22:29:14.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.632+0000 7fe027e91700 1 Processor -- start 2023-09-30T22:29:14.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.632+0000 7fe027e91700 1 -- start start 2023-09-30T22:29:14.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.633+0000 7fe027e91700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe02009d6f0 0x7fe02009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:14.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.633+0000 7fe027e91700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe02009e000 0x7fe020130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:14.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.633+0000 7fe027e91700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe02009f130 con 0x7fe02009e410 2023-09-30T22:29:14.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.633+0000 7fe027e91700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe02009f520 con 0x7fe02009d6f0 2023-09-30T22:29:14.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.633+0000 7fe027e91700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe02009e930 con 0x7fe02009e000 2023-09-30T22:29:14.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.633+0000 7fe025c2d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe02009d6f0 0x7fe02009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:14.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.633+0000 7fe025c2d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe02009d6f0 0x7fe02009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54102/0 (socket says 172.21.15.16:54102) 2023-09-30T22:29:14.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.633+0000 7fe025c2d700 1 -- 172.21.15.16:0/3303086169 learned_addr learned my addr 172.21.15.16:0/3303086169 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:14.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.633+0000 7fe02542c700 1 --2- 172.21.15.16:0/3303086169 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe02009e000 0x7fe020130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:14.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.633+0000 7fe025c2d700 1 -- 172.21.15.16:0/3303086169 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe02009e000 msgr2=0x7fe020130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:14.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.634+0000 7fe025c2d700 1 --2- 172.21.15.16:0/3303086169 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe02009e000 0x7fe020130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.634+0000 7fe025c2d700 1 -- 172.21.15.16:0/3303086169 >> v1:172.21.15.16:6789/0 conn(0x7fe02009e410 legacy=0x7fe020130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:14.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.634+0000 7fe025c2d700 1 -- 172.21.15.16:0/3303086169 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe02009b2d0 con 0x7fe02009d6f0 2023-09-30T22:29:14.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.634+0000 7fe025c2d700 1 --2- 172.21.15.16:0/3303086169 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe02009d6f0 0x7fe02009dac0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe01c00bbc0 tx=0x7fe01c00bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c557931f5b9cbedf server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:14.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.634+0000 7fe024c2b700 1 -- 172.21.15.16:0/3303086169 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01c00dd30 con 0x7fe02009d6f0 2023-09-30T22:29:14.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.634+0000 7fe024c2b700 1 -- 172.21.15.16:0/3303086169 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe01c00de90 con 0x7fe02009d6f0 2023-09-30T22:29:14.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.634+0000 7fe024c2b700 1 -- 172.21.15.16:0/3303086169 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01c010690 con 0x7fe02009d6f0 2023-09-30T22:29:14.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.635+0000 7fe027e91700 1 -- 172.21.15.16:0/3303086169 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe02009d6f0 msgr2=0x7fe02009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:14.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.635+0000 7fe027e91700 1 --2- 172.21.15.16:0/3303086169 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe02009d6f0 0x7fe02009dac0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe01c00bbc0 tx=0x7fe01c00bed0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.635+0000 7fe027e91700 1 -- 172.21.15.16:0/3303086169 shutdown_connections 2023-09-30T22:29:14.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.635+0000 7fe027e91700 1 --2- 172.21.15.16:0/3303086169 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe02009d6f0 0x7fe02009dac0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.635+0000 7fe027e91700 1 --2- 172.21.15.16:0/3303086169 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe02009e000 0x7fe020130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.635+0000 7fe027e91700 1 -- 172.21.15.16:0/3303086169 >> 172.21.15.16:0/3303086169 conn(0x7fe020066f50 msgr2=0x7fe02009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:14.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.635+0000 7fe027e91700 1 -- 172.21.15.16:0/3303086169 shutdown_connections 2023-09-30T22:29:14.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.635+0000 7fe027e91700 1 -- 172.21.15.16:0/3303086169 wait complete. 2023-09-30T22:29:14.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.636+0000 7fe027e91700 1 Processor -- start 2023-09-30T22:29:14.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.636+0000 7fe027e91700 1 -- start start 2023-09-30T22:29:14.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.637+0000 7fe027e91700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe02009d6f0 0x7fe020122430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:14.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.637+0000 7fe027e91700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe02009e000 0x7fe020122970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:14.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.637+0000 7fe027e91700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe02009e410 0x7fe020122eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:14.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.637+0000 7fe027e91700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe020125ec0 con 0x7fe02009e410 2023-09-30T22:29:14.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.637+0000 7fe027e91700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe020123fb0 con 0x7fe02009d6f0 2023-09-30T22:29:14.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.637+0000 7fe027e91700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe0200a1d90 con 0x7fe02009e000 2023-09-30T22:29:14.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.637+0000 7fe025c2d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe02009d6f0 0x7fe020122430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:14.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.637+0000 7fe025c2d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe02009d6f0 0x7fe020122430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54108/0 (socket says 172.21.15.16:54108) 2023-09-30T22:29:14.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.637+0000 7fe025c2d700 1 -- 172.21.15.16:0/1403475988 learned_addr learned my addr 172.21.15.16:0/1403475988 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:14.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.638+0000 7fe025c2d700 1 -- 172.21.15.16:0/1403475988 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe02009e000 msgr2=0x7fe020122970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:14.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.638+0000 7fe025c2d700 1 --2- 172.21.15.16:0/1403475988 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe02009e000 0x7fe020122970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.638+0000 7fe025c2d700 1 -- 172.21.15.16:0/1403475988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe02009e410 msgr2=0x7fe020122eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:14.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.638+0000 7fe025c2d700 1 --2- 172.21.15.16:0/1403475988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe02009e410 0x7fe020122eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.638+0000 7fe025c2d700 1 -- 172.21.15.16:0/1403475988 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe01c00b7f0 con 0x7fe02009d6f0 2023-09-30T22:29:14.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.638+0000 7fe025c2d700 1 --2- 172.21.15.16:0/1403475988 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe02009d6f0 0x7fe020122430 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe01c000c00 tx=0x7fe01c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:14.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.638+0000 7fe0167fc700 1 -- 172.21.15.16:0/1403475988 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01c003c70 con 0x7fe02009d6f0 2023-09-30T22:29:14.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.638+0000 7fe0167fc700 1 -- 172.21.15.16:0/1403475988 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe01c003dd0 con 0x7fe02009d6f0 2023-09-30T22:29:14.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.638+0000 7fe0167fc700 1 -- 172.21.15.16:0/1403475988 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01c010690 con 0x7fe02009d6f0 2023-09-30T22:29:14.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.638+0000 7fe027e91700 1 -- 172.21.15.16:0/1403475988 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0201302c0 con 0x7fe02009d6f0 2023-09-30T22:29:14.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.638+0000 7fe027e91700 1 -- 172.21.15.16:0/1403475988 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe020126650 con 0x7fe02009d6f0 2023-09-30T22:29:14.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.640+0000 7fe0167fc700 1 -- 172.21.15.16:0/1403475988 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 6) v1 ==== 35253+0+0 (secure 0 0 0) 0x7fe01c023440 con 0x7fe02009d6f0 2023-09-30T22:29:14.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.640+0000 7fe0167fc700 1 -- 172.21.15.16:0/1403475988 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 4055+0+0 (secure 0 0 0) 0x7fe01c0410d0 con 0x7fe02009d6f0 2023-09-30T22:29:14.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.640+0000 7fe027e91700 1 -- 172.21.15.16:0/1403475988 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe020125a60 con 0x7fe02009d6f0 2023-09-30T22:29:14.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.647+0000 7fa8e16d2700 1 Processor -- start 2023-09-30T22:29:14.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.647+0000 7fa8e16d2700 1 -- start start 2023-09-30T22:29:14.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.647+0000 7fa8e16d2700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa8dc131830 0x7fa8dc131c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:14.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.647+0000 7fa8e16d2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa8dc09e230 0x7fa8dc06a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:14.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.647+0000 7fa8e16d2700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8dc09dba0 con 0x7fa8dc06acb0 2023-09-30T22:29:14.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.647+0000 7fa8e16d2700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8dc132140 con 0x7fa8dc131830 2023-09-30T22:29:14.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.647+0000 7fa8e16d2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc1313e0 con 0x7fa8dc09e230 2023-09-30T22:29:14.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.648+0000 7fa8daffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa8dc131830 0x7fa8dc131c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:14.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.648+0000 7fa8db7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa8dc06acb0 0x7fa8dc138580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40370/0 (socket says 172.21.15.16:40370) 2023-09-30T22:29:14.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.648+0000 7fa8db7fe700 1 -- 172.21.15.16:0/976305818 learned_addr learned my addr 172.21.15.16:0/976305818 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:29:14.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.648+0000 7fa8d9ffb700 1 -- 172.21.15.16:0/976305818 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2712117600 0 0) 0x7fa8dc09dba0 con 0x7fa8dc06acb0 2023-09-30T22:29:14.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.648+0000 7fa8d9ffb700 1 -- 172.21.15.16:0/976305818 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8c40035b0 con 0x7fa8dc06acb0 2023-09-30T22:29:14.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.648+0000 7fa8daffd700 1 -- 172.21.15.16:0/976305818 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa8dc09e230 msgr2=0x7fa8dc06a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:14.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.648+0000 7fa8daffd700 1 --2- 172.21.15.16:0/976305818 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa8dc09e230 0x7fa8dc06a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.648+0000 7fa8daffd700 1 -- 172.21.15.16:0/976305818 >> v1:172.21.15.16:6789/0 conn(0x7fa8dc06acb0 legacy=0x7fa8dc138580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:14.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.648+0000 7fa8daffd700 1 -- 172.21.15.16:0/976305818 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8dc09d6d0 con 0x7fa8dc131830 2023-09-30T22:29:14.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.648+0000 7fa8daffd700 1 --2- 172.21.15.16:0/976305818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8dc131830 0x7fa8dc131c00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc009840 tx=0x7fa8cc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8d1bfc0c1c50227 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:14.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.648+0000 7fa8d9ffb700 1 -- 172.21.15.16:0/976305818 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc011830 con 0x7fa8dc131830 2023-09-30T22:29:14.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.648+0000 7fa8d9ffb700 1 -- 172.21.15.16:0/976305818 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8cc011990 con 0x7fa8dc131830 2023-09-30T22:29:14.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.649+0000 7fa8d9ffb700 1 -- 172.21.15.16:0/976305818 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc011d20 con 0x7fa8dc131830 2023-09-30T22:29:14.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.649+0000 7fe0167fc700 1 -- 172.21.15.16:0/1403475988 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7fe01c0238d0 con 0x7fe02009d6f0 2023-09-30T22:29:14.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.649+0000 7fa8e16d2700 1 -- 172.21.15.16:0/976305818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8dc131830 msgr2=0x7fa8dc131c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:14.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.649+0000 7fa8e16d2700 1 --2- 172.21.15.16:0/976305818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8dc131830 0x7fa8dc131c00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc009840 tx=0x7fa8cc009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:14.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.649+0000 7fa8e16d2700 1 -- 172.21.15.16:0/976305818 shutdown_connections 2023-09-30T22:29:14.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.649+0000 7fa8e16d2700 1 --2- 172.21.15.16:0/976305818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8dc131830 0x7fa8dc131c00 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.649+0000 7fa8e16d2700 1 --2- 172.21.15.16:0/976305818 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa8dc09e230 0x7fa8dc06a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.649+0000 7fa8e16d2700 1 -- 172.21.15.16:0/976305818 >> 172.21.15.16:0/976305818 conn(0x7fa8dc099c20 msgr2=0x7fa8dc0a2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:14.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.650+0000 7fa8e16d2700 1 -- 172.21.15.16:0/976305818 shutdown_connections 2023-09-30T22:29:14.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.650+0000 7fa8e16d2700 1 -- 172.21.15.16:0/976305818 wait complete. 2023-09-30T22:29:14.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.651+0000 7fa8e16d2700 1 Processor -- start 2023-09-30T22:29:14.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.651+0000 7fa8e16d2700 1 -- start start 2023-09-30T22:29:14.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.651+0000 7fa8e16d2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa8dc06acb0 0x7fa8dc12ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:14.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.651+0000 7fa8e16d2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8dc09e230 0x7fa8dc070f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:14.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.651+0000 7fa8e16d2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa8dc131830 0x7fa8dc071480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:14.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.651+0000 7fa8e16d2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc1b7520 con 0x7fa8dc09e230 2023-09-30T22:29:14.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.651+0000 7fa8e16d2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc1b7330 con 0x7fa8dc131830 2023-09-30T22:29:14.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.651+0000 7fa8e16d2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa8dc1b8ca0 con 0x7fa8dc06acb0 2023-09-30T22:29:14.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.651+0000 7fa8daffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa8dc06acb0 0x7fa8dc12ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:14.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.652+0000 7fa8daffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa8dc06acb0 0x7fa8dc12ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:58630/0 (socket says 172.21.15.16:58630) 2023-09-30T22:29:14.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.652+0000 7fa8daffd700 1 -- 172.21.15.16:0/3515738981 learned_addr learned my addr 172.21.15.16:0/3515738981 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:14.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.652+0000 7fa8db7fe700 1 --2- 172.21.15.16:0/3515738981 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa8dc131830 0x7fa8dc071480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:14.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.652+0000 7fa8daffd700 1 -- 172.21.15.16:0/3515738981 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa8dc131830 msgr2=0x7fa8dc071480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:14.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.652+0000 7fa8daffd700 1 --2- 172.21.15.16:0/3515738981 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa8dc131830 0x7fa8dc071480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.652+0000 7fa8daffd700 1 -- 172.21.15.16:0/3515738981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8dc09e230 msgr2=0x7fa8dc070f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:14.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.652+0000 7fa8daffd700 1 --2- 172.21.15.16:0/3515738981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8dc09e230 0x7fa8dc070f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.652+0000 7fa8daffd700 1 -- 172.21.15.16:0/3515738981 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8cc0094a0 con 0x7fa8dc06acb0 2023-09-30T22:29:14.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.652+0000 7fa8daffd700 1 --2- 172.21.15.16:0/3515738981 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa8dc06acb0 0x7fa8dc12ad20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc009fd0 tx=0x7fa8cc004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:14.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.652+0000 7fa8c37fe700 1 -- 172.21.15.16:0/3515738981 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc004300 con 0x7fa8dc06acb0 2023-09-30T22:29:14.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.653+0000 7fa8c37fe700 1 -- 172.21.15.16:0/3515738981 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8cc01f070 con 0x7fa8dc06acb0 2023-09-30T22:29:14.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.653+0000 7fa8c37fe700 1 -- 172.21.15.16:0/3515738981 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc0036a0 con 0x7fa8dc06acb0 2023-09-30T22:29:14.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.653+0000 7fa8e16d2700 1 -- 172.21.15.16:0/3515738981 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8dc1b9340 con 0x7fa8dc06acb0 2023-09-30T22:29:14.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.653+0000 7fa8e16d2700 1 -- 172.21.15.16:0/3515738981 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8dc1b99d0 con 0x7fa8dc06acb0 2023-09-30T22:29:14.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.654+0000 7fa8c17fa700 1 -- 172.21.15.16:0/3515738981 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8b40052e0 con 0x7fa8dc06acb0 2023-09-30T22:29:14.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.655+0000 7fa8c37fe700 1 -- 172.21.15.16:0/3515738981 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 6) v1 ==== 35253+0+0 (secure 0 0 0) 0x7fa8cc021030 con 0x7fa8dc06acb0 2023-09-30T22:29:14.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.655+0000 7fa8c37fe700 1 -- 172.21.15.16:0/3515738981 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 4055+0+0 (secure 0 0 0) 0x7fa8cc040b30 con 0x7fa8dc06acb0 2023-09-30T22:29:14.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.659+0000 7fa8c37fe700 1 -- 172.21.15.16:0/3515738981 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155592 (secure 0 0 0) 0x7fa8cc0199a0 con 0x7fa8dc06acb0 2023-09-30T22:29:14.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.671+0000 7f80638ee700 1 -- 172.21.15.16:0/1545782054 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f805c06b250 con 0x7f805c133e30 2023-09-30T22:29:14.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.672+0000 7f8051ffb700 1 -- 172.21.15.16:0/1545782054 <== mon.1 v2:172.21.15.16:3301/0 8 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f804c021020 con 0x7f805c133e30 2023-09-30T22:29:14.673 INFO:teuthology.orchestra.run.smithi016.stdout:38654705695 2023-09-30T22:29:14.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.674+0000 7f80638ee700 1 -- 172.21.15.16:0/1545782054 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f805c133e30 msgr2=0x7f805c09cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:14.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.674+0000 7f80638ee700 1 --2- 172.21.15.16:0/1545782054 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f805c133e30 0x7f805c09cc20 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f804c00e980 tx=0x7f804c00ec90 comp rx=0 tx=0).stop 2023-09-30T22:29:14.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.674+0000 7f80638ee700 1 -- 172.21.15.16:0/1545782054 shutdown_connections 2023-09-30T22:29:14.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.674+0000 7f80638ee700 1 --2- 172.21.15.16:0/1545782054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f805c134260 0x7f805c09d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.674+0000 7f80638ee700 1 --2- 172.21.15.16:0/1545782054 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f805c133420 0x7f805c09c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.674+0000 7f80638ee700 1 --2- 172.21.15.16:0/1545782054 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f805c133e30 0x7f805c09cc20 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.674+0000 7f80638ee700 1 -- 172.21.15.16:0/1545782054 >> 172.21.15.16:0/1545782054 conn(0x7f805c0a4210 msgr2=0x7f805c0a2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:14.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.674+0000 7f80638ee700 1 -- 172.21.15.16:0/1545782054 shutdown_connections 2023-09-30T22:29:14.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.674+0000 7f80638ee700 1 -- 172.21.15.16:0/1545782054 wait complete. 2023-09-30T22:29:14.689 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705696 got 38654705695 for osd.2 2023-09-30T22:29:14.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.777+0000 7f24f47e8700 1 -- 172.21.15.16:0/3902400029 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f24d4005150 con 0x7f24ec0a0a30 2023-09-30T22:29:14.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.778+0000 7f24e2ffd700 1 -- 172.21.15.16:0/3902400029 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f24e800f8d0 con 0x7f24ec0a0a30 2023-09-30T22:29:14.780 INFO:teuthology.orchestra.run.smithi016.stdout:38654705694 2023-09-30T22:29:14.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.780+0000 7f24f47e8700 1 -- 172.21.15.16:0/3902400029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24ec0a0a30 msgr2=0x7f24ec06dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:14.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.780+0000 7f24f47e8700 1 --2- 172.21.15.16:0/3902400029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24ec0a0a30 0x7f24ec06dd80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f24e800b560 tx=0x7f24e800b920 comp rx=0 tx=0).stop 2023-09-30T22:29:14.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.780+0000 7f24f47e8700 1 -- 172.21.15.16:0/3902400029 shutdown_connections 2023-09-30T22:29:14.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.780+0000 7f24f47e8700 1 --2- 172.21.15.16:0/3902400029 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f24ec134170 0x7f24ec1cc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.780+0000 7f24f47e8700 1 --2- 172.21.15.16:0/3902400029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24ec0a0a30 0x7f24ec06dd80 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.780+0000 7f24f47e8700 1 --2- 172.21.15.16:0/3902400029 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24ec0a1340 0x7f24ec06e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.780+0000 7f24f47e8700 1 -- 172.21.15.16:0/3902400029 >> 172.21.15.16:0/3902400029 conn(0x7f24ec0a6550 msgr2=0x7f24ec09edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:14.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.780+0000 7f24f47e8700 1 -- 172.21.15.16:0/3902400029 shutdown_connections 2023-09-30T22:29:14.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.780+0000 7f24f47e8700 1 -- 172.21.15.16:0/3902400029 wait complete. 2023-09-30T22:29:14.795 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705695 got 38654705694 for osd.3 2023-09-30T22:29:14.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.833+0000 7fe027e91700 1 -- 172.21.15.16:0/1403475988 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe020074ea0 con 0x7fe02009d6f0 2023-09-30T22:29:14.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.834+0000 7fe0167fc700 1 -- 172.21.15.16:0/1403475988 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe01c00f3d0 con 0x7fe02009d6f0 2023-09-30T22:29:14.836 INFO:teuthology.orchestra.run.smithi016.stdout:38654705694 2023-09-30T22:29:14.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.836+0000 7fe027e91700 1 -- 172.21.15.16:0/1403475988 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe02009d6f0 msgr2=0x7fe020122430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:14.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.836+0000 7fe027e91700 1 --2- 172.21.15.16:0/1403475988 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe02009d6f0 0x7fe020122430 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe01c000c00 tx=0x7fe01c004750 comp rx=0 tx=0).stop 2023-09-30T22:29:14.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.836+0000 7fe027e91700 1 -- 172.21.15.16:0/1403475988 shutdown_connections 2023-09-30T22:29:14.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.836+0000 7fe027e91700 1 --2- 172.21.15.16:0/1403475988 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe02009d6f0 0x7fe020122430 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.836+0000 7fe027e91700 1 --2- 172.21.15.16:0/1403475988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe02009e410 0x7fe020122eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.836+0000 7fe027e91700 1 --2- 172.21.15.16:0/1403475988 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe02009e000 0x7fe020122970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.836+0000 7fe027e91700 1 -- 172.21.15.16:0/1403475988 >> 172.21.15.16:0/1403475988 conn(0x7fe020066f50 msgr2=0x7fe02006f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:14.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.836+0000 7fe027e91700 1 -- 172.21.15.16:0/1403475988 shutdown_connections 2023-09-30T22:29:14.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.836+0000 7fe027e91700 1 -- 172.21.15.16:0/1403475988 wait complete. 2023-09-30T22:29:14.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.843+0000 7fa8c17fa700 1 -- 172.21.15.16:0/3515738981 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa8b4005860 con 0x7fa8dc06acb0 2023-09-30T22:29:14.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.843+0000 7fa8c37fe700 1 -- 172.21.15.16:0/3515738981 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa8cc015030 con 0x7fa8dc06acb0 2023-09-30T22:29:14.845 INFO:teuthology.orchestra.run.smithi016.stdout:38654705694 2023-09-30T22:29:14.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.845+0000 7fa8c17fa700 1 -- 172.21.15.16:0/3515738981 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa8dc06acb0 msgr2=0x7fa8dc12ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:14.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.845+0000 7fa8c17fa700 1 --2- 172.21.15.16:0/3515738981 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa8dc06acb0 0x7fa8dc12ad20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc009fd0 tx=0x7fa8cc004750 comp rx=0 tx=0).stop 2023-09-30T22:29:14.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.845+0000 7fa8c17fa700 1 -- 172.21.15.16:0/3515738981 shutdown_connections 2023-09-30T22:29:14.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.845+0000 7fa8c17fa700 1 --2- 172.21.15.16:0/3515738981 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa8dc131830 0x7fa8dc071480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.845+0000 7fa8c17fa700 1 --2- 172.21.15.16:0/3515738981 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa8dc06acb0 0x7fa8dc12ad20 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.845+0000 7fa8c17fa700 1 --2- 172.21.15.16:0/3515738981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8dc09e230 0x7fa8dc070f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:14.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.845+0000 7fa8c17fa700 1 -- 172.21.15.16:0/3515738981 >> 172.21.15.16:0/3515738981 conn(0x7fa8dc099c20 msgr2=0x7fa8dc12c170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:14.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.845+0000 7fa8c17fa700 1 -- 172.21.15.16:0/3515738981 shutdown_connections 2023-09-30T22:29:14.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:14.845+0000 7fa8c17fa700 1 -- 172.21.15.16:0/3515738981 wait complete. 2023-09-30T22:29:14.851 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705695 got 38654705694 for osd.1 2023-09-30T22:29:14.860 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705695 got 38654705694 for osd.0 2023-09-30T22:29:15.492 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:29:15.490+0000 7f9085a29700 -1 mgr.server handle_report got status from non-daemon mon.a 2023-09-30T22:29:15.492 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:29:15.491+0000 7f9085a29700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-09-30T22:29:15.493 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-09-30T22:29:15.492+0000 7f9085a29700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-09-30T22:29:15.690 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:29:15.796 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:29:15.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.843+0000 7ffa5162e700 1 Processor -- start 2023-09-30T22:29:15.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.843+0000 7ffa5162e700 1 -- start start 2023-09-30T22:29:15.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.843+0000 7ffa5162e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa4c0a0a30 0x7ffa4c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:15.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.843+0000 7ffa5162e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa4c138d80 0x7ffa4c13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:15.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.843+0000 7ffa5162e700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa4c0a2a70 con 0x7ffa4c09d630 2023-09-30T22:29:15.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.843+0000 7ffa5162e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c0a01d0 con 0x7ffa4c138d80 2023-09-30T22:29:15.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.843+0000 7ffa5162e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c09e2b0 con 0x7ffa4c0a0a30 2023-09-30T22:29:15.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.843+0000 7ffa4affd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa4c0a0a30 0x7ffa4c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:15.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.843+0000 7ffa4affd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa4c0a0a30 0x7ffa4c0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:58640/0 (socket says 172.21.15.16:58640) 2023-09-30T22:29:15.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.843+0000 7ffa4affd700 1 -- 172.21.15.16:0/2414236790 learned_addr learned my addr 172.21.15.16:0/2414236790 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:15.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.843+0000 7ffa4a7fc700 1 --1- 172.21.15.16:0/2414236790 >> v1:172.21.15.16:6789/0 conn(0x7ffa4c09d630 0x7ffa4c0a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40378/0 (socket says 172.21.15.16:40378) 2023-09-30T22:29:15.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.843+0000 7ffa4b7fe700 1 --2- 172.21.15.16:0/2414236790 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa4c138d80 0x7ffa4c13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:15.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.844+0000 7ffa49ffb700 1 -- 172.21.15.16:0/2414236790 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4249446848 0 0) 0x7ffa4c0a2a70 con 0x7ffa4c09d630 2023-09-30T22:29:15.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.844+0000 7ffa49ffb700 1 -- 172.21.15.16:0/2414236790 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa340035b0 con 0x7ffa4c09d630 2023-09-30T22:29:15.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.844+0000 7ffa4affd700 1 -- 172.21.15.16:0/2414236790 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa4c138d80 msgr2=0x7ffa4c13b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:15.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.844+0000 7ffa4affd700 1 --2- 172.21.15.16:0/2414236790 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa4c138d80 0x7ffa4c13b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:15.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.844+0000 7ffa4affd700 1 -- 172.21.15.16:0/2414236790 >> v1:172.21.15.16:6789/0 conn(0x7ffa4c09d630 legacy=0x7ffa4c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:15.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.844+0000 7ffa4affd700 1 -- 172.21.15.16:0/2414236790 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa4c09dc50 con 0x7ffa4c0a0a30 2023-09-30T22:29:15.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.844+0000 7ffa4b7fe700 1 --2- 172.21.15.16:0/2414236790 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa4c138d80 0x7ffa4c13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:29:15.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.844+0000 7ffa4affd700 1 --2- 172.21.15.16:0/2414236790 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa4c0a0a30 0x7ffa4c0a0e00 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c009920 tx=0x7ffa3c009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9db9f3b5bf7901f1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:15.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.844+0000 7ffa49ffb700 1 -- 172.21.15.16:0/2414236790 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c011830 con 0x7ffa4c0a0a30 2023-09-30T22:29:15.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.844+0000 7ffa49ffb700 1 -- 172.21.15.16:0/2414236790 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa3c011990 con 0x7ffa4c0a0a30 2023-09-30T22:29:15.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.844+0000 7ffa49ffb700 1 -- 172.21.15.16:0/2414236790 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c011d20 con 0x7ffa4c0a0a30 2023-09-30T22:29:15.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.845+0000 7ffa5162e700 1 -- 172.21.15.16:0/2414236790 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa4c0a0a30 msgr2=0x7ffa4c0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:15.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.845+0000 7ffa5162e700 1 --2- 172.21.15.16:0/2414236790 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa4c0a0a30 0x7ffa4c0a0e00 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c009920 tx=0x7ffa3c009c30 comp rx=0 tx=0).stop 2023-09-30T22:29:15.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.845+0000 7ffa5162e700 1 -- 172.21.15.16:0/2414236790 shutdown_connections 2023-09-30T22:29:15.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.845+0000 7ffa5162e700 1 --2- 172.21.15.16:0/2414236790 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa4c138d80 0x7ffa4c13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:15.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.845+0000 7ffa5162e700 1 --2- 172.21.15.16:0/2414236790 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa4c0a0a30 0x7ffa4c0a0e00 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:15.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.845+0000 7ffa5162e700 1 -- 172.21.15.16:0/2414236790 >> 172.21.15.16:0/2414236790 conn(0x7ffa4c0a3050 msgr2=0x7ffa4c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:15.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.845+0000 7ffa5162e700 1 -- 172.21.15.16:0/2414236790 shutdown_connections 2023-09-30T22:29:15.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.845+0000 7ffa5162e700 1 -- 172.21.15.16:0/2414236790 wait complete. 2023-09-30T22:29:15.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.846+0000 7ffa5162e700 1 Processor -- start 2023-09-30T22:29:15.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.846+0000 7ffa5162e700 1 -- start start 2023-09-30T22:29:15.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.847+0000 7ffa5162e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa4c09d630 0x7ffa4c132790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:15.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.847+0000 7ffa5162e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa4c0a0a30 0x7ffa4c132cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:15.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.847+0000 7ffa5162e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa4c138d80 0x7ffa4c133210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:15.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.847+0000 7ffa5162e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c1343e0 con 0x7ffa4c0a0a30 2023-09-30T22:29:15.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.847+0000 7ffa5162e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c133b60 con 0x7ffa4c138d80 2023-09-30T22:29:15.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.847+0000 7ffa5162e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c0a4d10 con 0x7ffa4c09d630 2023-09-30T22:29:15.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.847+0000 7ffa4affd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa4c09d630 0x7ffa4c132790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:15.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.847+0000 7ffa4a7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa4c0a0a30 0x7ffa4c132cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:15.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.847+0000 7ffa4b7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa4c138d80 0x7ffa4c133210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:15.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.847+0000 7ffa4affd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa4c09d630 0x7ffa4c132790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:58646/0 (socket says 172.21.15.16:58646) 2023-09-30T22:29:15.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.847+0000 7ffa4a7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa4c0a0a30 0x7ffa4c132cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47182/0 (socket says 172.21.15.16:47182) 2023-09-30T22:29:15.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.847+0000 7ffa4affd700 1 -- 172.21.15.16:0/15031433 learned_addr learned my addr 172.21.15.16:0/15031433 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:15.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.847+0000 7ffa4affd700 1 -- 172.21.15.16:0/15031433 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa4c138d80 msgr2=0x7ffa4c133210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:15.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.847+0000 7ffa4affd700 1 --2- 172.21.15.16:0/15031433 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa4c138d80 0x7ffa4c133210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:15.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.847+0000 7ffa4affd700 1 -- 172.21.15.16:0/15031433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa4c0a0a30 msgr2=0x7ffa4c132cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:15.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.847+0000 7ffa4affd700 1 --2- 172.21.15.16:0/15031433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa4c0a0a30 0x7ffa4c132cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:15.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.847+0000 7ffa4affd700 1 -- 172.21.15.16:0/15031433 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa3c009550 con 0x7ffa4c09d630 2023-09-30T22:29:15.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.847+0000 7ffa4b7fe700 1 --2- 172.21.15.16:0/15031433 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa4c138d80 0x7ffa4c133210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:15.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.847+0000 7ffa4a7fc700 1 --2- 172.21.15.16:0/15031433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa4c0a0a30 0x7ffa4c132cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:15.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.847+0000 7ffa4affd700 1 --2- 172.21.15.16:0/15031433 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa4c09d630 0x7ffa4c132790 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c000c00 tx=0x7ffa3c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:15.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.848+0000 7ffa337fe700 1 -- 172.21.15.16:0/15031433 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c004450 con 0x7ffa4c09d630 2023-09-30T22:29:15.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.848+0000 7ffa337fe700 1 -- 172.21.15.16:0/15031433 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa3c0045b0 con 0x7ffa4c09d630 2023-09-30T22:29:15.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.848+0000 7ffa5162e700 1 -- 172.21.15.16:0/15031433 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa4c0a0410 con 0x7ffa4c09d630 2023-09-30T22:29:15.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.848+0000 7ffa337fe700 1 -- 172.21.15.16:0/15031433 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c0038a0 con 0x7ffa4c09d630 2023-09-30T22:29:15.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.848+0000 7ffa5162e700 1 -- 172.21.15.16:0/15031433 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa4c136970 con 0x7ffa4c09d630 2023-09-30T22:29:15.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.848+0000 7ffa337fe700 1 -- 172.21.15.16:0/15031433 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 7) v1 ==== 35345+0+0 (secure 0 0 0) 0x7ffa3c021070 con 0x7ffa4c09d630 2023-09-30T22:29:15.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.849+0000 7ffa337fe700 1 -- 172.21.15.16:0/15031433 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4038+0+0 (secure 0 0 0) 0x7ffa3c040920 con 0x7ffa4c09d630 2023-09-30T22:29:15.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.849+0000 7ffa5162e700 1 -- 172.21.15.16:0/15031433 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa4c0747f0 con 0x7ffa4c09d630 2023-09-30T22:29:15.854 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:29:15.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.854+0000 7ffa337fe700 1 -- 172.21.15.16:0/15031433 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ffa3c015020 con 0x7ffa4c09d630 2023-09-30T22:29:15.861 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:29:15.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.944+0000 7f46e5808700 1 Processor -- start 2023-09-30T22:29:15.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.944+0000 7f46e5808700 1 -- start start 2023-09-30T22:29:15.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.944+0000 7f46e5808700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f46e00a09c0 0x7f46e00a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:15.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.944+0000 7f46e5808700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f46e009d5c0 0x7f46e00a12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:15.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.944+0000 7f46e5808700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f46e0133920 0x7f46e0133d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:15.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.944+0000 7f46e5808700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46e00a1810 con 0x7f46e00a09c0 2023-09-30T22:29:15.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.944+0000 7f46e5808700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46e009ea10 con 0x7f46e009d5c0 2023-09-30T22:29:15.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.944+0000 7f46e5808700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f46e009e210 con 0x7f46e0133920 2023-09-30T22:29:15.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.944+0000 7f46deffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f46e00a09c0 0x7f46e00a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:15.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.945+0000 7f46deffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f46e00a09c0 0x7f46e00a0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47184/0 (socket says 172.21.15.16:47184) 2023-09-30T22:29:15.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.945+0000 7f46deffd700 1 -- 172.21.15.16:0/1837276621 learned_addr learned my addr 172.21.15.16:0/1837276621 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:15.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.945+0000 7f46df7fe700 1 --2- 172.21.15.16:0/1837276621 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f46e0133920 0x7f46e0133d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:15.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.945+0000 7f46de7fc700 1 --2- 172.21.15.16:0/1837276621 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f46e009d5c0 0x7f46e00a12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:15.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.945+0000 7f46df7fe700 1 -- 172.21.15.16:0/1837276621 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f46e009d5c0 msgr2=0x7f46e00a12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:15.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.945+0000 7f46df7fe700 1 --2- 172.21.15.16:0/1837276621 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f46e009d5c0 0x7f46e00a12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:15.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.945+0000 7f46df7fe700 1 -- 172.21.15.16:0/1837276621 >> v2:172.21.15.16:3300/0 conn(0x7f46e00a09c0 msgr2=0x7f46e00a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:15.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.945+0000 7f46df7fe700 1 --2- 172.21.15.16:0/1837276621 >> v2:172.21.15.16:3300/0 conn(0x7f46e00a09c0 0x7f46e00a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:15.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.945+0000 7f46df7fe700 1 -- 172.21.15.16:0/1837276621 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e00a26f0 con 0x7f46e0133920 2023-09-30T22:29:15.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.945+0000 7f46deffd700 1 --2- 172.21.15.16:0/1837276621 >> v2:172.21.15.16:3300/0 conn(0x7f46e00a09c0 0x7f46e00a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:15.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.945+0000 7f46df7fe700 1 --2- 172.21.15.16:0/1837276621 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f46e0133920 0x7f46e0133d70 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f46d8009d90 tx=0x7f46d800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77bf1c58e8c17fcd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:15.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.946+0000 7f46ddffb700 1 -- 172.21.15.16:0/1837276621 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d800e070 con 0x7f46e0133920 2023-09-30T22:29:15.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.946+0000 7f46ddffb700 1 -- 172.21.15.16:0/1837276621 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46d800bb20 con 0x7f46e0133920 2023-09-30T22:29:15.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.946+0000 7f46ddffb700 1 -- 172.21.15.16:0/1837276621 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d8012430 con 0x7f46e0133920 2023-09-30T22:29:15.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.946+0000 7f46e5808700 1 -- 172.21.15.16:0/1837276621 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f46e0133920 msgr2=0x7f46e0133d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:15.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.946+0000 7f46e5808700 1 --2- 172.21.15.16:0/1837276621 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f46e0133920 0x7f46e0133d70 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f46d8009d90 tx=0x7f46d800b330 comp rx=0 tx=0).stop 2023-09-30T22:29:15.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.946+0000 7f46e5808700 1 -- 172.21.15.16:0/1837276621 shutdown_connections 2023-09-30T22:29:15.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.946+0000 7f46e5808700 1 --2- 172.21.15.16:0/1837276621 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f46e0133920 0x7f46e0133d70 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:15.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.946+0000 7f46e5808700 1 --2- 172.21.15.16:0/1837276621 >> v2:172.21.15.16:3300/0 conn(0x7f46e00a09c0 0x7f46e00a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:15.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.946+0000 7f46e5808700 1 --2- 172.21.15.16:0/1837276621 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f46e009d5c0 0x7f46e00a12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:15.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.946+0000 7f46e5808700 1 -- 172.21.15.16:0/1837276621 >> 172.21.15.16:0/1837276621 conn(0x7f46e00a2fe0 msgr2=0x7f46e00a33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:15.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.946+0000 7f46e5808700 1 -- 172.21.15.16:0/1837276621 shutdown_connections 2023-09-30T22:29:15.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.947+0000 7f46e5808700 1 -- 172.21.15.16:0/1837276621 wait complete. 2023-09-30T22:29:15.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.948+0000 7f46e5808700 1 Processor -- start 2023-09-30T22:29:15.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.948+0000 7f46e5808700 1 -- start start 2023-09-30T22:29:15.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.949+0000 7f46e5808700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f46e009d5c0 0x7f46e011f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:15.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.949+0000 7f46e5808700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f46e00a09c0 0x7f46e011fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:15.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.949+0000 7f46e5808700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46e0133920 0x7f46e01c7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:15.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.949+0000 7f46e5808700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e01223e0 con 0x7f46e0133920 2023-09-30T22:29:15.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.949+0000 7f46e5808700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46e011ff70 con 0x7f46e00a09c0 2023-09-30T22:29:15.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.949+0000 7f46e5808700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f46e01225c0 con 0x7f46e009d5c0 2023-09-30T22:29:15.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.949+0000 7f46df7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46e0133920 0x7f46e01c7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:15.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.949+0000 7f46df7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46e0133920 0x7f46e01c7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47192/0 (socket says 172.21.15.16:47192) 2023-09-30T22:29:15.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.949+0000 7f46df7fe700 1 -- 172.21.15.16:0/3422299616 learned_addr learned my addr 172.21.15.16:0/3422299616 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:15.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.949+0000 7f46deffd700 1 --2- 172.21.15.16:0/3422299616 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f46e009d5c0 0x7f46e011f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:15.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.949+0000 7f46de7fc700 1 --2- 172.21.15.16:0/3422299616 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f46e00a09c0 0x7f46e011fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:15.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.950+0000 7f46deffd700 1 -- 172.21.15.16:0/3422299616 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f46e00a09c0 msgr2=0x7f46e011fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:15.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.950+0000 7f46deffd700 1 --2- 172.21.15.16:0/3422299616 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f46e00a09c0 0x7f46e011fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:15.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.950+0000 7f46deffd700 1 -- 172.21.15.16:0/3422299616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46e0133920 msgr2=0x7f46e01c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:15.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.950+0000 7f46deffd700 1 --2- 172.21.15.16:0/3422299616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46e0133920 0x7f46e01c7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:15.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.950+0000 7f46deffd700 1 -- 172.21.15.16:0/3422299616 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46d0009550 con 0x7f46e009d5c0 2023-09-30T22:29:15.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.950+0000 7f46df7fe700 1 --2- 172.21.15.16:0/3422299616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46e0133920 0x7f46e01c7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:15.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.950+0000 7f46de7fc700 1 --2- 172.21.15.16:0/3422299616 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f46e00a09c0 0x7f46e011fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:15.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.950+0000 7f46deffd700 1 --2- 172.21.15.16:0/3422299616 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f46e009d5c0 0x7f46e011f4f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f46d000e9d0 tx=0x7f46d000ece0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:15.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.950+0000 7f46c7fff700 1 -- 172.21.15.16:0/3422299616 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d000ee70 con 0x7f46e009d5c0 2023-09-30T22:29:15.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.950+0000 7f46c7fff700 1 -- 172.21.15.16:0/3422299616 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46d0004030 con 0x7f46e009d5c0 2023-09-30T22:29:15.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.950+0000 7f46e5808700 1 -- 172.21.15.16:0/3422299616 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46d8009a70 con 0x7f46e009d5c0 2023-09-30T22:29:15.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.950+0000 7f46c7fff700 1 -- 172.21.15.16:0/3422299616 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d00043c0 con 0x7f46e009d5c0 2023-09-30T22:29:15.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.950+0000 7f46e5808700 1 -- 172.21.15.16:0/3422299616 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46e009ad70 con 0x7f46e009d5c0 2023-09-30T22:29:15.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.951+0000 7f46e5808700 1 -- 172.21.15.16:0/3422299616 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46e00747f0 con 0x7f46e009d5c0 2023-09-30T22:29:15.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.952+0000 7f46c7fff700 1 -- 172.21.15.16:0/3422299616 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 7) v1 ==== 35345+0+0 (secure 0 0 0) 0x7f46d000c850 con 0x7f46e009d5c0 2023-09-30T22:29:15.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.952+0000 7f46c7fff700 1 -- 172.21.15.16:0/3422299616 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4038+0+0 (secure 0 0 0) 0x7f46d003fcf0 con 0x7f46e009d5c0 2023-09-30T22:29:15.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:15.957+0000 7f46c7fff700 1 -- 172.21.15.16:0/3422299616 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f46d0011350 con 0x7f46e009d5c0 2023-09-30T22:29:16.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.001+0000 7fc9740fb700 1 Processor -- start 2023-09-30T22:29:16.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.001+0000 7fc9740fb700 1 -- start start 2023-09-30T22:29:16.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.001+0000 7fc9740fb700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc96c09d6f0 0x7fc96c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:16.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.001+0000 7fc9740fb700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc96c09e000 0x7fc96c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:16.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.002+0000 7fc9740fb700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc96c09e410 0x7fc96c133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:16.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.002+0000 7fc9740fb700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc96c06b460 con 0x7fc96c09d6f0 2023-09-30T22:29:16.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.002+0000 7fc9740fb700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc96c09b120 con 0x7fc96c09e000 2023-09-30T22:29:16.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.002+0000 7fc9740fb700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc96c09a7e0 con 0x7fc96c09e410 2023-09-30T22:29:16.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.002+0000 7fc971e97700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc96c09d6f0 0x7fc96c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:16.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.002+0000 7fc971e97700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc96c09d6f0 0x7fc96c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47208/0 (socket says 172.21.15.16:47208) 2023-09-30T22:29:16.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.002+0000 7fc971e97700 1 -- 172.21.15.16:0/2374563748 learned_addr learned my addr 172.21.15.16:0/2374563748 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:16.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.002+0000 7fc971696700 1 --2- 172.21.15.16:0/2374563748 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc96c09e000 0x7fc96c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:16.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.002+0000 7fc972698700 1 --2- 172.21.15.16:0/2374563748 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc96c09e410 0x7fc96c133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:16.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.002+0000 7fc971e97700 1 -- 172.21.15.16:0/2374563748 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc96c09e410 msgr2=0x7fc96c133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:16.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.002+0000 7fc971e97700 1 --2- 172.21.15.16:0/2374563748 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc96c09e410 0x7fc96c133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:16.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.002+0000 7fc971e97700 1 -- 172.21.15.16:0/2374563748 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc96c09e000 msgr2=0x7fc96c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:16.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.002+0000 7fc971e97700 1 --2- 172.21.15.16:0/2374563748 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc96c09e000 0x7fc96c1339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:16.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.002+0000 7fc971e97700 1 -- 172.21.15.16:0/2374563748 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc96c09a500 con 0x7fc96c09d6f0 2023-09-30T22:29:16.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.002+0000 7fc971e97700 1 --2- 172.21.15.16:0/2374563748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc96c09d6f0 0x7fc96c09dac0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc9680098f0 tx=0x7fc968009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2524867b6d60bb99 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:16.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.002+0000 7fc970e95700 1 -- 172.21.15.16:0/2374563748 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc968011830 con 0x7fc96c09d6f0 2023-09-30T22:29:16.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.003+0000 7fc970e95700 1 -- 172.21.15.16:0/2374563748 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc968011990 con 0x7fc96c09d6f0 2023-09-30T22:29:16.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.003+0000 7fc970e95700 1 -- 172.21.15.16:0/2374563748 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc968011d20 con 0x7fc96c09d6f0 2023-09-30T22:29:16.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.003+0000 7fc9740fb700 1 -- 172.21.15.16:0/2374563748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc96c09d6f0 msgr2=0x7fc96c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:16.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.003+0000 7fc9740fb700 1 --2- 172.21.15.16:0/2374563748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc96c09d6f0 0x7fc96c09dac0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc9680098f0 tx=0x7fc968009c00 comp rx=0 tx=0).stop 2023-09-30T22:29:16.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.003+0000 7fc9740fb700 1 -- 172.21.15.16:0/2374563748 shutdown_connections 2023-09-30T22:29:16.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.003+0000 7fc9740fb700 1 --2- 172.21.15.16:0/2374563748 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc96c09e410 0x7fc96c133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:16.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.003+0000 7fc9740fb700 1 --2- 172.21.15.16:0/2374563748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc96c09d6f0 0x7fc96c09dac0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:16.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.003+0000 7fc9740fb700 1 --2- 172.21.15.16:0/2374563748 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc96c09e000 0x7fc96c1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:16.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.003+0000 7fc9740fb700 1 -- 172.21.15.16:0/2374563748 >> 172.21.15.16:0/2374563748 conn(0x7fc96c066f50 msgr2=0x7fc96c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:16.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.003+0000 7fc9740fb700 1 -- 172.21.15.16:0/2374563748 shutdown_connections 2023-09-30T22:29:16.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.003+0000 7fc9740fb700 1 -- 172.21.15.16:0/2374563748 wait complete. 2023-09-30T22:29:16.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.005+0000 7fc9740fb700 1 Processor -- start 2023-09-30T22:29:16.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.005+0000 7fc9740fb700 1 -- start start 2023-09-30T22:29:16.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.006+0000 7fc9740fb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc96c09d6f0 0x7fc96c1b8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:16.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.006+0000 7fc9740fb700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc96c09e000 0x7fc96c1b8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:16.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.006+0000 7fc9740fb700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc96c09e410 0x7fc96c1c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:16.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.006+0000 7fc9740fb700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc96c1a80f0 con 0x7fc96c09d6f0 2023-09-30T22:29:16.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.006+0000 7fc9740fb700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc96c1b91a0 con 0x7fc96c09e000 2023-09-30T22:29:16.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.006+0000 7fc9740fb700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc96c1a6430 con 0x7fc96c09e410 2023-09-30T22:29:16.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.006+0000 7fc972698700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc96c09e410 0x7fc96c1c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:16.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.006+0000 7fc971696700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc96c09e000 0x7fc96c1b8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:16.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.006+0000 7fc972698700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc96c09e410 0x7fc96c1c8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:58664/0 (socket says 172.21.15.16:58664) 2023-09-30T22:29:16.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.006+0000 7fc972698700 1 -- 172.21.15.16:0/2180795181 learned_addr learned my addr 172.21.15.16:0/2180795181 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:16.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.006+0000 7fc971e97700 1 --2- 172.21.15.16:0/2180795181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc96c09d6f0 0x7fc96c1b8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:16.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.007+0000 7fc972698700 1 -- 172.21.15.16:0/2180795181 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc96c09e000 msgr2=0x7fc96c1b8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:16.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.007+0000 7fc972698700 1 --2- 172.21.15.16:0/2180795181 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc96c09e000 0x7fc96c1b8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:16.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.007+0000 7fc972698700 1 -- 172.21.15.16:0/2180795181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc96c09d6f0 msgr2=0x7fc96c1b8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:16.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.007+0000 7fc972698700 1 --2- 172.21.15.16:0/2180795181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc96c09d6f0 0x7fc96c1b8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:16.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.007+0000 7fc972698700 1 -- 172.21.15.16:0/2180795181 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc968009550 con 0x7fc96c09e410 2023-09-30T22:29:16.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.007+0000 7fc972698700 1 --2- 172.21.15.16:0/2180795181 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc96c09e410 0x7fc96c1c8040 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc964007e40 tx=0x7fc96400e3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:16.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.007+0000 7fc9627fc700 1 -- 172.21.15.16:0/2180795181 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96400ed30 con 0x7fc96c09e410 2023-09-30T22:29:16.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.007+0000 7fc9627fc700 1 -- 172.21.15.16:0/2180795181 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc964014420 con 0x7fc96c09e410 2023-09-30T22:29:16.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.007+0000 7fc9740fb700 1 -- 172.21.15.16:0/2180795181 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc96c0a1f20 con 0x7fc96c09e410 2023-09-30T22:29:16.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.007+0000 7fc9627fc700 1 -- 172.21.15.16:0/2180795181 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9640147d0 con 0x7fc96c09e410 2023-09-30T22:29:16.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.007+0000 7fc9740fb700 1 -- 172.21.15.16:0/2180795181 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc96c1a6c20 con 0x7fc96c09e410 2023-09-30T22:29:16.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.008+0000 7fc9627fc700 1 -- 172.21.15.16:0/2180795181 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 7) v1 ==== 35345+0+0 (secure 0 0 0) 0x7fc964014930 con 0x7fc96c09e410 2023-09-30T22:29:16.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.008+0000 7fc9740fb700 1 -- 172.21.15.16:0/2180795181 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc96c074ea0 con 0x7fc96c09e410 2023-09-30T22:29:16.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.008+0000 7fc9627fc700 1 -- 172.21.15.16:0/2180795181 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4038+0+0 (secure 0 0 0) 0x7fc96403fd50 con 0x7fc96c09e410 2023-09-30T22:29:16.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.013+0000 7fc9627fc700 1 -- 172.21.15.16:0/2180795181 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc96401bce0 con 0x7fc96c09e410 2023-09-30T22:29:16.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.047+0000 7f0c1a881700 1 Processor -- start 2023-09-30T22:29:16.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.047+0000 7f0c1a881700 1 -- start start 2023-09-30T22:29:16.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.048+0000 7f0c1a881700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0c1409d6f0 0x7f0c1409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:16.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.048+0000 7f0c1a881700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0c1409e000 0x7f0c141339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:16.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.048+0000 7f0c1a881700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c1409b120 con 0x7f0c1409e410 2023-09-30T22:29:16.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.048+0000 7f0c1a881700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c1406b460 con 0x7f0c1409d6f0 2023-09-30T22:29:16.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.048+0000 7f0c1a881700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1409a7e0 con 0x7f0c1409e000 2023-09-30T22:29:16.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.048+0000 7f0c13fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0c1409d6f0 0x7f0c1409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:16.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.048+0000 7f0c137fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0c1409e000 0x7f0c141339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:16.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.048+0000 7f0c18e1e700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0c1409e410 0x7f0c14133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40380/0 (socket says 172.21.15.16:40380) 2023-09-30T22:29:16.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.048+0000 7f0c18e1e700 1 -- 172.21.15.16:0/1168951052 learned_addr learned my addr 172.21.15.16:0/1168951052 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:29:16.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.048+0000 7f0c137fe700 1 -- 172.21.15.16:0/1168951052 >> v2:172.21.15.16:3300/0 conn(0x7f0c1409d6f0 msgr2=0x7f0c1409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:16.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.048+0000 7f0c137fe700 1 --2- 172.21.15.16:0/1168951052 >> v2:172.21.15.16:3300/0 conn(0x7f0c1409d6f0 0x7f0c1409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:16.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.048+0000 7f0c137fe700 1 -- 172.21.15.16:0/1168951052 >> v1:172.21.15.16:6789/0 conn(0x7f0c1409e410 legacy=0x7f0c14133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:16.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.048+0000 7f0c137fe700 1 -- 172.21.15.16:0/1168951052 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c1409a500 con 0x7f0c1409e000 2023-09-30T22:29:16.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.049+0000 7f0c137fe700 1 --2- 172.21.15.16:0/1168951052 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0c1409e000 0x7f0c141339f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0c08009870 tx=0x7f0c08009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=139b5597a7b39a0e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:16.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.049+0000 7f0c12ffd700 1 -- 172.21.15.16:0/1168951052 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c08011830 con 0x7f0c1409e000 2023-09-30T22:29:16.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.049+0000 7f0c12ffd700 1 -- 172.21.15.16:0/1168951052 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c08011990 con 0x7f0c1409e000 2023-09-30T22:29:16.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.049+0000 7f0c12ffd700 1 -- 172.21.15.16:0/1168951052 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c08011d20 con 0x7f0c1409e000 2023-09-30T22:29:16.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.049+0000 7f0c1a881700 1 -- 172.21.15.16:0/1168951052 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0c1409e000 msgr2=0x7f0c141339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:16.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.049+0000 7f0c1a881700 1 --2- 172.21.15.16:0/1168951052 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0c1409e000 0x7f0c141339f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f0c08009870 tx=0x7f0c08009b80 comp rx=0 tx=0).stop 2023-09-30T22:29:16.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.049+0000 7f0c1a881700 1 -- 172.21.15.16:0/1168951052 shutdown_connections 2023-09-30T22:29:16.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.049+0000 7f0c1a881700 1 --2- 172.21.15.16:0/1168951052 >> v2:172.21.15.16:3300/0 conn(0x7f0c1409d6f0 0x7f0c1409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:16.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.050+0000 7f0c1a881700 1 --2- 172.21.15.16:0/1168951052 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0c1409e000 0x7f0c141339f0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:16.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.050+0000 7f0c1a881700 1 -- 172.21.15.16:0/1168951052 >> 172.21.15.16:0/1168951052 conn(0x7f0c14066f50 msgr2=0x7f0c1409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:16.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.050+0000 7f0c1a881700 1 -- 172.21.15.16:0/1168951052 shutdown_connections 2023-09-30T22:29:16.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.050+0000 7f0c1a881700 1 -- 172.21.15.16:0/1168951052 wait complete. 2023-09-30T22:29:16.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.051+0000 7f0c1a881700 1 Processor -- start 2023-09-30T22:29:16.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.051+0000 7f0c1a881700 1 -- start start 2023-09-30T22:29:16.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.051+0000 7f0c1a881700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0c1409d6f0 0x7f0c141a6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:16.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.051+0000 7f0c1a881700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c1409e000 0x7f0c141a6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:16.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.051+0000 7f0c1a881700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0c1409e410 0x7f0c141a6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:16.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.051+0000 7f0c1a881700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c141a8e70 con 0x7f0c1409e000 2023-09-30T22:29:16.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.051+0000 7f0c1a881700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c141a98c0 con 0x7f0c1409e410 2023-09-30T22:29:16.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.051+0000 7f0c1a881700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0c141a7450 con 0x7f0c1409d6f0 2023-09-30T22:29:16.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.051+0000 7f0c18e1e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0c1409e410 0x7f0c141a6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:16.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.051+0000 7f0c18e1e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0c1409e410 0x7f0c141a6ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54200/0 (socket says 172.21.15.16:54200) 2023-09-30T22:29:16.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.051+0000 7f0c18e1e700 1 -- 172.21.15.16:0/3646875058 learned_addr learned my addr 172.21.15.16:0/3646875058 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:16.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.052+0000 7f0c13fff700 1 --2- 172.21.15.16:0/3646875058 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0c1409d6f0 0x7f0c141a6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:16.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.052+0000 7f0c137fe700 1 --2- 172.21.15.16:0/3646875058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c1409e000 0x7f0c141a6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:16.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.052+0000 7f0c18e1e700 1 -- 172.21.15.16:0/3646875058 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0c1409d6f0 msgr2=0x7f0c141a6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:16.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.052+0000 7f0c18e1e700 1 --2- 172.21.15.16:0/3646875058 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0c1409d6f0 0x7f0c141a6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:16.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.052+0000 7f0c18e1e700 1 -- 172.21.15.16:0/3646875058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c1409e000 msgr2=0x7f0c141a6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:16.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.052+0000 7f0c18e1e700 1 --2- 172.21.15.16:0/3646875058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c1409e000 0x7f0c141a6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:16.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.052+0000 7f0c18e1e700 1 -- 172.21.15.16:0/3646875058 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c080094a0 con 0x7f0c1409e410 2023-09-30T22:29:16.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.052+0000 7f0c137fe700 1 --2- 172.21.15.16:0/3646875058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c1409e000 0x7f0c141a6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:29:16.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.052+0000 7f0c18e1e700 1 --2- 172.21.15.16:0/3646875058 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0c1409e410 0x7f0c141a6ab0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c00cbf0 tx=0x7f0c0c00cf00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:16.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.052+0000 7f0c10ff9700 1 -- 172.21.15.16:0/3646875058 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c00ab10 con 0x7f0c1409e410 2023-09-30T22:29:16.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.052+0000 7f0c10ff9700 1 -- 172.21.15.16:0/3646875058 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c0c00ac70 con 0x7f0c1409e410 2023-09-30T22:29:16.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.052+0000 7f0c10ff9700 1 -- 172.21.15.16:0/3646875058 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c012400 con 0x7f0c1409e410 2023-09-30T22:29:16.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.052+0000 7f0c1a881700 1 -- 172.21.15.16:0/3646875058 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c141a89b0 con 0x7f0c1409e410 2023-09-30T22:29:16.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.052+0000 7f0c1a881700 1 -- 172.21.15.16:0/3646875058 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c140a3180 con 0x7f0c1409e410 2023-09-30T22:29:16.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.053+0000 7f0c10ff9700 1 -- 172.21.15.16:0/3646875058 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 7) v1 ==== 35345+0+0 (secure 0 0 0) 0x7f0c0c01b3b0 con 0x7f0c1409e410 2023-09-30T22:29:16.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.054+0000 7f0c10ff9700 1 -- 172.21.15.16:0/3646875058 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 4038+0+0 (secure 0 0 0) 0x7f0c0c03f310 con 0x7f0c1409e410 2023-09-30T22:29:16.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.056+0000 7f0c1a881700 1 -- 172.21.15.16:0/3646875058 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c14074ea0 con 0x7f0c1409e410 2023-09-30T22:29:16.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.061+0000 7f0c10ff9700 1 -- 172.21.15.16:0/3646875058 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0c0c017020 con 0x7f0c1409e410 2023-09-30T22:29:16.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.077+0000 7ffa5162e700 1 -- 172.21.15.16:0/15031433 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ffa4c072980 con 0x7ffa4c09d630 2023-09-30T22:29:16.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.077+0000 7ffa337fe700 1 -- 172.21.15.16:0/15031433 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ffa3c032070 con 0x7ffa4c09d630 2023-09-30T22:29:16.093 INFO:teuthology.orchestra.run.smithi016.stdout:38654705695 2023-09-30T22:29:16.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.080+0000 7ffa5162e700 1 -- 172.21.15.16:0/15031433 >> v1:172.21.15.16:6816/126811 conn(0x7ffa3802f240 legacy=0x7ffa380316f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:16.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.080+0000 7ffa5162e700 1 -- 172.21.15.16:0/15031433 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa4c09d630 msgr2=0x7ffa4c132790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:16.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.080+0000 7ffa5162e700 1 --2- 172.21.15.16:0/15031433 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa4c09d630 0x7ffa4c132790 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c000c00 tx=0x7ffa3c004750 comp rx=0 tx=0).stop 2023-09-30T22:29:16.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.080+0000 7ffa5162e700 1 -- 172.21.15.16:0/15031433 shutdown_connections 2023-09-30T22:29:16.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.080+0000 7ffa5162e700 1 --2- 172.21.15.16:0/15031433 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa4c138d80 0x7ffa4c133210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:16.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.080+0000 7ffa5162e700 1 --2- 172.21.15.16:0/15031433 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa4c09d630 0x7ffa4c132790 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:16.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.080+0000 7ffa5162e700 1 --2- 172.21.15.16:0/15031433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa4c0a0a30 0x7ffa4c132cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:16.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.080+0000 7ffa5162e700 1 -- 172.21.15.16:0/15031433 >> 172.21.15.16:0/15031433 conn(0x7ffa4c0a3050 msgr2=0x7ffa4c1393f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:16.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.080+0000 7ffa5162e700 1 -- 172.21.15.16:0/15031433 shutdown_connections 2023-09-30T22:29:16.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.080+0000 7ffa5162e700 1 -- 172.21.15.16:0/15031433 wait complete. 2023-09-30T22:29:16.096 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705696 got 38654705695 for osd.2 2023-09-30T22:29:16.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.177+0000 7f46e5808700 1 -- 172.21.15.16:0/3422299616 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f46e0122220 con 0x7f46e009d5c0 2023-09-30T22:29:16.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.178+0000 7f46c7fff700 1 -- 172.21.15.16:0/3422299616 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f46e0122220 con 0x7f46e009d5c0 2023-09-30T22:29:16.179 INFO:teuthology.orchestra.run.smithi016.stdout:38654705694 2023-09-30T22:29:16.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.180+0000 7f46e5808700 1 -- 172.21.15.16:0/3422299616 >> v1:172.21.15.16:6816/126811 conn(0x7f46cc02f240 legacy=0x7f46cc0316f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:16.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.180+0000 7f46e5808700 1 -- 172.21.15.16:0/3422299616 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f46e009d5c0 msgr2=0x7f46e011f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:16.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.180+0000 7f46e5808700 1 --2- 172.21.15.16:0/3422299616 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f46e009d5c0 0x7f46e011f4f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f46d000e9d0 tx=0x7f46d000ece0 comp rx=0 tx=0).stop 2023-09-30T22:29:16.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.180+0000 7f46e5808700 1 -- 172.21.15.16:0/3422299616 shutdown_connections 2023-09-30T22:29:16.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.180+0000 7f46e5808700 1 --2- 172.21.15.16:0/3422299616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46e0133920 0x7f46e01c7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:16.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.180+0000 7f46e5808700 1 --2- 172.21.15.16:0/3422299616 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f46e009d5c0 0x7f46e011f4f0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:16.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.180+0000 7f46e5808700 1 --2- 172.21.15.16:0/3422299616 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f46e00a09c0 0x7f46e011fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:16.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.180+0000 7f46e5808700 1 -- 172.21.15.16:0/3422299616 >> 172.21.15.16:0/3422299616 conn(0x7f46e00a2fe0 msgr2=0x7f46e01bba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:16.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.180+0000 7f46e5808700 1 -- 172.21.15.16:0/3422299616 shutdown_connections 2023-09-30T22:29:16.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.180+0000 7f46e5808700 1 -- 172.21.15.16:0/3422299616 wait complete. 2023-09-30T22:29:16.195 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705695 got 38654705694 for osd.3 2023-09-30T22:29:16.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.227+0000 7fc9740fb700 1 -- 172.21.15.16:0/2180795181 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc96c06c360 con 0x7fc96c09e410 2023-09-30T22:29:16.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.227+0000 7fc9627fc700 1 -- 172.21.15.16:0/2180795181 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc96401bb00 con 0x7fc96c09e410 2023-09-30T22:29:16.229 INFO:teuthology.orchestra.run.smithi016.stdout:38654705694 2023-09-30T22:29:16.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.229+0000 7fc9740fb700 1 -- 172.21.15.16:0/2180795181 >> v1:172.21.15.16:6816/126811 conn(0x7fc95002f240 legacy=0x7fc9500316f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:16.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.229+0000 7fc9740fb700 1 -- 172.21.15.16:0/2180795181 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc96c09e410 msgr2=0x7fc96c1c8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:16.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.229+0000 7fc9740fb700 1 --2- 172.21.15.16:0/2180795181 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc96c09e410 0x7fc96c1c8040 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc964007e40 tx=0x7fc96400e3c0 comp rx=0 tx=0).stop 2023-09-30T22:29:16.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.229+0000 7fc9740fb700 1 -- 172.21.15.16:0/2180795181 shutdown_connections 2023-09-30T22:29:16.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.229+0000 7fc9740fb700 1 --2- 172.21.15.16:0/2180795181 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc96c09e410 0x7fc96c1c8040 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:16.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.229+0000 7fc9740fb700 1 --2- 172.21.15.16:0/2180795181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc96c09d6f0 0x7fc96c1b8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:16.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.229+0000 7fc9740fb700 1 --2- 172.21.15.16:0/2180795181 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc96c09e000 0x7fc96c1b8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:16.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.229+0000 7fc9740fb700 1 -- 172.21.15.16:0/2180795181 >> 172.21.15.16:0/2180795181 conn(0x7fc96c066f50 msgr2=0x7fc96c09bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:16.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.229+0000 7fc9740fb700 1 -- 172.21.15.16:0/2180795181 shutdown_connections 2023-09-30T22:29:16.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.229+0000 7fc9740fb700 1 -- 172.21.15.16:0/2180795181 wait complete. 2023-09-30T22:29:16.245 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705695 got 38654705694 for osd.1 2023-09-30T22:29:16.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.273+0000 7f0c1a881700 1 -- 172.21.15.16:0/3646875058 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0c141a96b0 con 0x7f0c1409e410 2023-09-30T22:29:16.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.273+0000 7f0c10ff9700 1 -- 172.21.15.16:0/3646875058 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0c0c0192c0 con 0x7f0c1409e410 2023-09-30T22:29:16.275 INFO:teuthology.orchestra.run.smithi016.stdout:38654705694 2023-09-30T22:29:16.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.276+0000 7f0c1a881700 1 -- 172.21.15.16:0/3646875058 >> v1:172.21.15.16:6816/126811 conn(0x7f0c0002f240 legacy=0x7f0c000316f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:16.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.276+0000 7f0c1a881700 1 -- 172.21.15.16:0/3646875058 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0c1409e410 msgr2=0x7f0c141a6ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:16.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.276+0000 7f0c1a881700 1 --2- 172.21.15.16:0/3646875058 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0c1409e410 0x7f0c141a6ab0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c00cbf0 tx=0x7f0c0c00cf00 comp rx=0 tx=0).stop 2023-09-30T22:29:16.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.276+0000 7f0c1a881700 1 -- 172.21.15.16:0/3646875058 shutdown_connections 2023-09-30T22:29:16.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.276+0000 7f0c1a881700 1 --2- 172.21.15.16:0/3646875058 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0c1409e410 0x7f0c141a6ab0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:16.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.276+0000 7f0c1a881700 1 --2- 172.21.15.16:0/3646875058 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0c1409d6f0 0x7f0c141a6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:16.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.276+0000 7f0c1a881700 1 --2- 172.21.15.16:0/3646875058 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c1409e000 0x7f0c141a6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:16.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.276+0000 7f0c1a881700 1 -- 172.21.15.16:0/3646875058 >> 172.21.15.16:0/3646875058 conn(0x7f0c14066f50 msgr2=0x7f0c140a36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:16.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.276+0000 7f0c1a881700 1 -- 172.21.15.16:0/3646875058 shutdown_connections 2023-09-30T22:29:16.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:16.276+0000 7f0c1a881700 1 -- 172.21.15.16:0/3646875058 wait complete. 2023-09-30T22:29:16.291 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705695 got 38654705694 for osd.0 2023-09-30T22:29:17.097 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:29:17.195 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:29:17.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.239+0000 7f343c3a3700 1 Processor -- start 2023-09-30T22:29:17.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.240+0000 7f343c3a3700 1 -- start start 2023-09-30T22:29:17.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.240+0000 7f343c3a3700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f34340a0a30 0x7f34340a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:17.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.240+0000 7f343c3a3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f34340a1340 0x7f3434133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:17.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.240+0000 7f343c3a3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3434133fa0 0x7f3434134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:17.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.240+0000 7f343c3a3700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f343409edd0 con 0x7f34340a0a30 2023-09-30T22:29:17.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.240+0000 7f343c3a3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f343409e1f0 con 0x7f3434133fa0 2023-09-30T22:29:17.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.240+0000 7f343c3a3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f343409dc50 con 0x7f34340a1340 2023-09-30T22:29:17.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.241+0000 7f343a13f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f34340a0a30 0x7f34340a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:17.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.241+0000 7f343a13f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f34340a0a30 0x7f34340a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47270/0 (socket says 172.21.15.16:47270) 2023-09-30T22:29:17.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.241+0000 7f343a13f700 1 -- 172.21.15.16:0/3871851742 learned_addr learned my addr 172.21.15.16:0/3871851742 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:17.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.241+0000 7f343a940700 1 --2- 172.21.15.16:0/3871851742 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3434133fa0 0x7f3434134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:17.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.241+0000 7f343993e700 1 --2- 172.21.15.16:0/3871851742 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f34340a1340 0x7f3434133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:17.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.241+0000 7f343a13f700 1 -- 172.21.15.16:0/3871851742 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f34340a1340 msgr2=0x7f3434133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:17.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.241+0000 7f343a13f700 1 --2- 172.21.15.16:0/3871851742 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f34340a1340 0x7f3434133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.241+0000 7f343a13f700 1 -- 172.21.15.16:0/3871851742 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3434133fa0 msgr2=0x7f3434134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:17.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.241+0000 7f343a13f700 1 --2- 172.21.15.16:0/3871851742 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3434133fa0 0x7f3434134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.241+0000 7f343a13f700 1 -- 172.21.15.16:0/3871851742 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34340a1fe0 con 0x7f34340a0a30 2023-09-30T22:29:17.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.241+0000 7f343a13f700 1 --2- 172.21.15.16:0/3871851742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34340a0a30 0x7f34340a0e00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f34300098f0 tx=0x7f3430009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17fb421b0175470e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:17.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.241+0000 7f343913d700 1 -- 172.21.15.16:0/3871851742 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343000b8a0 con 0x7f34340a0a30 2023-09-30T22:29:17.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.242+0000 7f343913d700 1 -- 172.21.15.16:0/3871851742 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f343000ba00 con 0x7f34340a0a30 2023-09-30T22:29:17.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.242+0000 7f343913d700 1 -- 172.21.15.16:0/3871851742 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343000bd90 con 0x7f34340a0a30 2023-09-30T22:29:17.246 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:29:17.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.242+0000 7f343c3a3700 1 -- 172.21.15.16:0/3871851742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34340a0a30 msgr2=0x7f34340a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:17.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.242+0000 7f343c3a3700 1 --2- 172.21.15.16:0/3871851742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34340a0a30 0x7f34340a0e00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f34300098f0 tx=0x7f3430009c00 comp rx=0 tx=0).stop 2023-09-30T22:29:17.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.242+0000 7f343c3a3700 1 -- 172.21.15.16:0/3871851742 shutdown_connections 2023-09-30T22:29:17.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.242+0000 7f343c3a3700 1 --2- 172.21.15.16:0/3871851742 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3434133fa0 0x7f3434134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.242+0000 7f343c3a3700 1 --2- 172.21.15.16:0/3871851742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34340a0a30 0x7f34340a0e00 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.242+0000 7f343c3a3700 1 --2- 172.21.15.16:0/3871851742 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f34340a1340 0x7f3434133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.242+0000 7f343c3a3700 1 -- 172.21.15.16:0/3871851742 >> 172.21.15.16:0/3871851742 conn(0x7f34340a6550 msgr2=0x7f34340a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:17.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.242+0000 7f343c3a3700 1 -- 172.21.15.16:0/3871851742 shutdown_connections 2023-09-30T22:29:17.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.242+0000 7f343c3a3700 1 -- 172.21.15.16:0/3871851742 wait complete. 2023-09-30T22:29:17.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.244+0000 7f343c3a3700 1 Processor -- start 2023-09-30T22:29:17.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.244+0000 7f343c3a3700 1 -- start start 2023-09-30T22:29:17.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.245+0000 7f343c3a3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f34340a0a30 0x7f34341203b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:17.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.245+0000 7f343c3a3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f34340a1340 0x7f34341208f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:17.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.245+0000 7f343c3a3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3434133fa0 0x7f3434120e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:17.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.245+0000 7f343c3a3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3434123e30 con 0x7f3434133fa0 2023-09-30T22:29:17.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.245+0000 7f343c3a3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34340a0220 con 0x7f34340a0a30 2023-09-30T22:29:17.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.245+0000 7f343c3a3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3434122500 con 0x7f34340a1340 2023-09-30T22:29:17.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.245+0000 7f343a13f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f34340a0a30 0x7f34341203b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:17.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.245+0000 7f343993e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f34340a1340 0x7f34341208f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:17.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.245+0000 7f343a13f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f34340a0a30 0x7f34341203b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54244/0 (socket says 172.21.15.16:54244) 2023-09-30T22:29:17.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.245+0000 7f343a13f700 1 -- 172.21.15.16:0/3588940891 learned_addr learned my addr 172.21.15.16:0/3588940891 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:17.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.245+0000 7f343a940700 1 --2- 172.21.15.16:0/3588940891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3434133fa0 0x7f3434120e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:17.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.245+0000 7f343a940700 1 -- 172.21.15.16:0/3588940891 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f34340a1340 msgr2=0x7f34341208f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:17.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.245+0000 7f343a940700 1 --2- 172.21.15.16:0/3588940891 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f34340a1340 0x7f34341208f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.245+0000 7f343a940700 1 -- 172.21.15.16:0/3588940891 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f34340a0a30 msgr2=0x7f34341203b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:17.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.245+0000 7f343a940700 1 --2- 172.21.15.16:0/3588940891 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f34340a0a30 0x7f34341203b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.245+0000 7f343a940700 1 -- 172.21.15.16:0/3588940891 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3430009550 con 0x7f3434133fa0 2023-09-30T22:29:17.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.246+0000 7f343a940700 1 --2- 172.21.15.16:0/3588940891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3434133fa0 0x7f3434120e30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f342c007d90 tx=0x7f342c00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:17.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.246+0000 7f342affd700 1 -- 172.21.15.16:0/3588940891 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342c018070 con 0x7f3434133fa0 2023-09-30T22:29:17.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.246+0000 7f342affd700 1 -- 172.21.15.16:0/3588940891 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f342c00eb30 con 0x7f3434133fa0 2023-09-30T22:29:17.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.246+0000 7f343c3a3700 1 -- 172.21.15.16:0/3588940891 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34341235c0 con 0x7f3434133fa0 2023-09-30T22:29:17.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.246+0000 7f342affd700 1 -- 172.21.15.16:0/3588940891 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342c003a20 con 0x7f3434133fa0 2023-09-30T22:29:17.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.246+0000 7f343c3a3700 1 -- 172.21.15.16:0/3588940891 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3434133700 con 0x7f3434133fa0 2023-09-30T22:29:17.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.247+0000 7f343c3a3700 1 -- 172.21.15.16:0/3588940891 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3434074ea0 con 0x7f3434133fa0 2023-09-30T22:29:17.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.247+0000 7f342affd700 1 -- 172.21.15.16:0/3588940891 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 7) v1 ==== 35345+0+0 (secure 0 0 0) 0x7f342c014420 con 0x7f3434133fa0 2023-09-30T22:29:17.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.248+0000 7f342affd700 1 -- 172.21.15.16:0/3588940891 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4049+0+0 (secure 0 0 0) 0x7f342c040450 con 0x7f3434133fa0 2023-09-30T22:29:17.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.257+0000 7f342affd700 1 -- 172.21.15.16:0/3588940891 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f342c008760 con 0x7f3434133fa0 2023-09-30T22:29:17.291 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:29:17.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.362+0000 7fafafef1700 1 Processor -- start 2023-09-30T22:29:17.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.363+0000 7fafafef1700 1 -- start start 2023-09-30T22:29:17.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.363+0000 7fafafef1700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fafa812cf60 0x7fafa806f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:17.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.363+0000 7fafafef1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fafa8130430 0x7fafa8137c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:17.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.363+0000 7fafafef1700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafa812cd90 con 0x7fafa812f8b0 2023-09-30T22:29:17.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.363+0000 7fafafef1700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafa812d330 con 0x7fafa812cf60 2023-09-30T22:29:17.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.363+0000 7fafafef1700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafa8071780 con 0x7fafa8130430 2023-09-30T22:29:17.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.363+0000 7fafad48c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fafa812f8b0 0x7fafa812fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40382/0 (socket says 172.21.15.16:40382) 2023-09-30T22:29:17.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.363+0000 7fafad48c700 1 -- 172.21.15.16:0/2213695001 learned_addr learned my addr 172.21.15.16:0/2213695001 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:29:17.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.363+0000 7fafadc8d700 1 --2- 172.21.15.16:0/2213695001 >> v2:172.21.15.16:3300/0 conn(0x7fafa812cf60 0x7fafa806f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:17.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.363+0000 7fafae48e700 1 --2- 172.21.15.16:0/2213695001 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fafa8130430 0x7fafa8137c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:17.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.364+0000 7fafadc8d700 1 -- 172.21.15.16:0/2213695001 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fafa8130430 msgr2=0x7fafa8137c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:17.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.364+0000 7fafadc8d700 1 --2- 172.21.15.16:0/2213695001 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fafa8130430 0x7fafa8137c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.364+0000 7fafadc8d700 1 -- 172.21.15.16:0/2213695001 >> v1:172.21.15.16:6789/0 conn(0x7fafa812f8b0 legacy=0x7fafa812fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:17.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.364+0000 7fafadc8d700 1 -- 172.21.15.16:0/2213695001 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafa8071a40 con 0x7fafa812cf60 2023-09-30T22:29:17.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.364+0000 7fafae48e700 1 --2- 172.21.15.16:0/2213695001 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fafa8130430 0x7fafa8137c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:17.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.364+0000 7fafadc8d700 1 --2- 172.21.15.16:0/2213695001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafa812cf60 0x7fafa806f5d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fafa400b8f0 tx=0x7fafa400bc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72d3250b3cff55da server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:17.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.364+0000 7fafacc8b700 1 -- 172.21.15.16:0/2213695001 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa400c9f0 con 0x7fafa812cf60 2023-09-30T22:29:17.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.364+0000 7fafacc8b700 1 -- 172.21.15.16:0/2213695001 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafa400cb50 con 0x7fafa812cf60 2023-09-30T22:29:17.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.364+0000 7fafacc8b700 1 -- 172.21.15.16:0/2213695001 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa400f460 con 0x7fafa812cf60 2023-09-30T22:29:17.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.365+0000 7fafafef1700 1 -- 172.21.15.16:0/2213695001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafa812cf60 msgr2=0x7fafa806f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:17.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.365+0000 7fafafef1700 1 --2- 172.21.15.16:0/2213695001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafa812cf60 0x7fafa806f5d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fafa400b8f0 tx=0x7fafa400bc00 comp rx=0 tx=0).stop 2023-09-30T22:29:17.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.365+0000 7fafafef1700 1 -- 172.21.15.16:0/2213695001 shutdown_connections 2023-09-30T22:29:17.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.365+0000 7fafafef1700 1 --2- 172.21.15.16:0/2213695001 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fafa8130430 0x7fafa8137c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.365+0000 7fafafef1700 1 --2- 172.21.15.16:0/2213695001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafa812cf60 0x7fafa806f5d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.365+0000 7fafafef1700 1 -- 172.21.15.16:0/2213695001 >> 172.21.15.16:0/2213695001 conn(0x7fafa80a2aa0 msgr2=0x7fafa80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:17.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.365+0000 7fafafef1700 1 -- 172.21.15.16:0/2213695001 shutdown_connections 2023-09-30T22:29:17.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.365+0000 7fafafef1700 1 -- 172.21.15.16:0/2213695001 wait complete. 2023-09-30T22:29:17.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.366+0000 7fafafef1700 1 Processor -- start 2023-09-30T22:29:17.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.366+0000 7fafafef1700 1 -- start start 2023-09-30T22:29:17.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.367+0000 7fafafef1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafa812cf60 0x7fafa809d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:17.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.367+0000 7fafafef1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fafa812f8b0 0x7fafa809dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:17.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.367+0000 7fafafef1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fafa8130430 0x7fafa809dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:17.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.367+0000 7fafafef1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafa811ae30 con 0x7fafa812cf60 2023-09-30T22:29:17.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.367+0000 7fafafef1700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafa8119150 con 0x7fafa812f8b0 2023-09-30T22:29:17.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.367+0000 7fafafef1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fafa8129fa0 con 0x7fafa8130430 2023-09-30T22:29:17.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.367+0000 7fafae48e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fafa8130430 0x7fafa809dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:17.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.367+0000 7fafae48e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fafa8130430 0x7fafa809dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:58704/0 (socket says 172.21.15.16:58704) 2023-09-30T22:29:17.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.367+0000 7fafae48e700 1 -- 172.21.15.16:0/344630019 learned_addr learned my addr 172.21.15.16:0/344630019 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:17.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.367+0000 7fafadc8d700 1 --2- 172.21.15.16:0/344630019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafa812cf60 0x7fafa809d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:17.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.368+0000 7fafae48e700 1 -- 172.21.15.16:0/344630019 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fafa812f8b0 msgr2=0x7fafa809dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:17.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.368+0000 7fafad48c700 1 --2- 172.21.15.16:0/344630019 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fafa812f8b0 0x7fafa809dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:17.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.368+0000 7fafae48e700 1 --2- 172.21.15.16:0/344630019 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fafa812f8b0 0x7fafa809dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.368+0000 7fafae48e700 1 -- 172.21.15.16:0/344630019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafa812cf60 msgr2=0x7fafa809d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:17.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.368+0000 7fafae48e700 1 --2- 172.21.15.16:0/344630019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafa812cf60 0x7fafa809d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.368+0000 7fafae48e700 1 -- 172.21.15.16:0/344630019 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafa000eb70 con 0x7fafa8130430 2023-09-30T22:29:17.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.368+0000 7fafadc8d700 1 --2- 172.21.15.16:0/344630019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafa812cf60 0x7fafa809d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:29:17.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.368+0000 7fafad48c700 1 --2- 172.21.15.16:0/344630019 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fafa812f8b0 0x7fafa809dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:29:17.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.368+0000 7fafae48e700 1 --2- 172.21.15.16:0/344630019 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fafa8130430 0x7fafa809dff0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fafa00131c0 tx=0x7fafa0013580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:17.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.368+0000 7faf9e7fc700 1 -- 172.21.15.16:0/344630019 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa0013b70 con 0x7fafa8130430 2023-09-30T22:29:17.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.368+0000 7faf9e7fc700 1 -- 172.21.15.16:0/344630019 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafa0014040 con 0x7fafa8130430 2023-09-30T22:29:17.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.368+0000 7fafafef1700 1 -- 172.21.15.16:0/344630019 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafa400b550 con 0x7fafa8130430 2023-09-30T22:29:17.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.368+0000 7faf9e7fc700 1 -- 172.21.15.16:0/344630019 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa0003a20 con 0x7fafa8130430 2023-09-30T22:29:17.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.368+0000 7fafafef1700 1 -- 172.21.15.16:0/344630019 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafa812f570 con 0x7fafa8130430 2023-09-30T22:29:17.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.369+0000 7faf9e7fc700 1 -- 172.21.15.16:0/344630019 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 7) v1 ==== 35345+0+0 (secure 0 0 0) 0x7fafa0003b80 con 0x7fafa8130430 2023-09-30T22:29:17.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.369+0000 7fafafef1700 1 -- 172.21.15.16:0/344630019 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafa8074ea0 con 0x7fafa8130430 2023-09-30T22:29:17.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.370+0000 7faf9e7fc700 1 -- 172.21.15.16:0/344630019 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4049+0+0 (secure 0 0 0) 0x7fafa0043e50 con 0x7fafa8130430 2023-09-30T22:29:17.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.375+0000 7faf9e7fc700 1 -- 172.21.15.16:0/344630019 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fafa0003e30 con 0x7fafa8130430 2023-09-30T22:29:17.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.437+0000 7f45341fe700 1 Processor -- start 2023-09-30T22:29:17.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.437+0000 7f45341fe700 1 -- start start 2023-09-30T22:29:17.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.438+0000 7f45341fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f452c09d6f0 0x7f452c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:17.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.438+0000 7f45341fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f452c09e000 0x7f452c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:17.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.438+0000 7f45341fe700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f452c09b120 con 0x7f452c09e410 2023-09-30T22:29:17.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.438+0000 7f45341fe700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f452c06b460 con 0x7f452c09d6f0 2023-09-30T22:29:17.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.438+0000 7f45341fe700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f452c09a7e0 con 0x7f452c09e000 2023-09-30T22:29:17.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.438+0000 7f4531799700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f452c09e000 0x7f452c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:17.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.438+0000 7f4531f9a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f452c09d6f0 0x7f452c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:17.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.438+0000 7f4531799700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f452c09e000 0x7f452c1339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37006/0 (socket says 172.21.15.16:37006) 2023-09-30T22:29:17.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.438+0000 7f4531f9a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f452c09d6f0 0x7f452c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57572/0 (socket says 172.21.15.16:57572) 2023-09-30T22:29:17.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.438+0000 7f4531799700 1 -- 172.21.15.16:0/3982437807 learned_addr learned my addr 172.21.15.16:0/3982437807 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:17.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.439+0000 7f4531799700 1 -- 172.21.15.16:0/3982437807 >> v2:172.21.15.16:3300/0 conn(0x7f452c09d6f0 msgr2=0x7f452c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:17.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.439+0000 7f4531799700 1 --2- 172.21.15.16:0/3982437807 >> v2:172.21.15.16:3300/0 conn(0x7f452c09d6f0 0x7f452c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.439+0000 7f4530f98700 1 -- 172.21.15.16:0/3982437807 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 926655025 0 0) 0x7f452c09b120 con 0x7f452c09e410 2023-09-30T22:29:17.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.439+0000 7f4531799700 1 -- 172.21.15.16:0/3982437807 >> v1:172.21.15.16:6789/0 conn(0x7f452c09e410 legacy=0x7f452c133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:17.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.439+0000 7f4531799700 1 -- 172.21.15.16:0/3982437807 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f452c09a500 con 0x7f452c09e000 2023-09-30T22:29:17.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.439+0000 7f4531799700 1 --2- 172.21.15.16:0/3982437807 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f452c09e000 0x7f452c1339f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f451c009920 tx=0x7f451c009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b85a47b57ccd6541 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:17.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.439+0000 7f4530f98700 1 -- 172.21.15.16:0/3982437807 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451c011830 con 0x7f452c09e000 2023-09-30T22:29:17.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.439+0000 7f4530f98700 1 -- 172.21.15.16:0/3982437807 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f451c011990 con 0x7f452c09e000 2023-09-30T22:29:17.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.439+0000 7f4530f98700 1 -- 172.21.15.16:0/3982437807 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451c011d20 con 0x7f452c09e000 2023-09-30T22:29:17.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.440+0000 7f45341fe700 1 -- 172.21.15.16:0/3982437807 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f452c09e000 msgr2=0x7f452c1339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:17.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.440+0000 7f45341fe700 1 --2- 172.21.15.16:0/3982437807 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f452c09e000 0x7f452c1339f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f451c009920 tx=0x7f451c009c30 comp rx=0 tx=0).stop 2023-09-30T22:29:17.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.440+0000 7f45341fe700 1 -- 172.21.15.16:0/3982437807 shutdown_connections 2023-09-30T22:29:17.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.440+0000 7f45341fe700 1 --2- 172.21.15.16:0/3982437807 >> v2:172.21.15.16:3300/0 conn(0x7f452c09d6f0 0x7f452c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.440+0000 7f45341fe700 1 --2- 172.21.15.16:0/3982437807 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f452c09e000 0x7f452c1339f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.440+0000 7f45341fe700 1 -- 172.21.15.16:0/3982437807 >> 172.21.15.16:0/3982437807 conn(0x7f452c066f50 msgr2=0x7f452c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:17.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.441+0000 7f45341fe700 1 -- 172.21.15.16:0/3982437807 shutdown_connections 2023-09-30T22:29:17.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.441+0000 7f45341fe700 1 -- 172.21.15.16:0/3982437807 wait complete. 2023-09-30T22:29:17.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.442+0000 7f45341fe700 1 Processor -- start 2023-09-30T22:29:17.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.442+0000 7f45341fe700 1 -- start start 2023-09-30T22:29:17.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.443+0000 7f45341fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f452c09d6f0 0x7f452c1a5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:17.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.443+0000 7f45341fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f452c09e000 0x7f452c1a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:17.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.443+0000 7f45341fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f452c09e410 0x7f452c1a6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:17.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.443+0000 7f45341fe700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452c1a72b0 con 0x7f452c09d6f0 2023-09-30T22:29:17.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.443+0000 7f45341fe700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f452c1aa230 con 0x7f452c09e410 2023-09-30T22:29:17.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.443+0000 7f45341fe700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f452c1a9870 con 0x7f452c09e000 2023-09-30T22:29:17.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.443+0000 7f4531f9a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f452c09d6f0 0x7f452c1a5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:17.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.443+0000 7f4531f9a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f452c09d6f0 0x7f452c1a5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57576/0 (socket says 172.21.15.16:57576) 2023-09-30T22:29:17.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.443+0000 7f4531f9a700 1 -- 172.21.15.16:0/2700450527 learned_addr learned my addr 172.21.15.16:0/2700450527 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:17.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.443+0000 7f453279b700 1 --2- 172.21.15.16:0/2700450527 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f452c09e410 0x7f452c1a6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:17.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.443+0000 7f4531799700 1 --2- 172.21.15.16:0/2700450527 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f452c09e000 0x7f452c1a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:17.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.444+0000 7f4531f9a700 1 -- 172.21.15.16:0/2700450527 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f452c09e000 msgr2=0x7f452c1a6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:17.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.444+0000 7f4531f9a700 1 --2- 172.21.15.16:0/2700450527 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f452c09e000 0x7f452c1a6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.444+0000 7f4531f9a700 1 -- 172.21.15.16:0/2700450527 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f452c09e410 msgr2=0x7f452c1a6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:17.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.444+0000 7f4531f9a700 1 --2- 172.21.15.16:0/2700450527 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f452c09e410 0x7f452c1a6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.444+0000 7f4531f9a700 1 -- 172.21.15.16:0/2700450527 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f451c009550 con 0x7f452c09d6f0 2023-09-30T22:29:17.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.444+0000 7f4531f9a700 1 --2- 172.21.15.16:0/2700450527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f452c09d6f0 0x7f452c1a5de0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f452800e8a0 tx=0x7f452800ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:17.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.444+0000 7f45227fc700 1 -- 172.21.15.16:0/2700450527 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4528009590 con 0x7f452c09d6f0 2023-09-30T22:29:17.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.444+0000 7f45227fc700 1 -- 172.21.15.16:0/2700450527 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4528013680 con 0x7f452c09d6f0 2023-09-30T22:29:17.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.444+0000 7f45341fe700 1 -- 172.21.15.16:0/2700450527 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f452c1a7a00 con 0x7f452c09d6f0 2023-09-30T22:29:17.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.444+0000 7f45227fc700 1 -- 172.21.15.16:0/2700450527 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45280139f0 con 0x7f452c09d6f0 2023-09-30T22:29:17.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.444+0000 7f45341fe700 1 -- 172.21.15.16:0/2700450527 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f452c0a3040 con 0x7f452c09d6f0 2023-09-30T22:29:17.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.445+0000 7f45227fc700 1 -- 172.21.15.16:0/2700450527 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 7) v1 ==== 35345+0+0 (secure 0 0 0) 0x7f4528013b50 con 0x7f452c09d6f0 2023-09-30T22:29:17.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.446+0000 7f45227fc700 1 -- 172.21.15.16:0/2700450527 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4049+0+0 (secure 0 0 0) 0x7f452803f5d0 con 0x7f452c09d6f0 2023-09-30T22:29:17.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.446+0000 7f45341fe700 1 -- 172.21.15.16:0/2700450527 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f452c1a8880 con 0x7f452c09d6f0 2023-09-30T22:29:17.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.455+0000 7f45227fc700 1 -- 172.21.15.16:0/2700450527 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4528021080 con 0x7f452c09d6f0 2023-09-30T22:29:17.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.491+0000 7faf9e7fc700 1 -- 172.21.15.16:0/344630019 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mgrmap(e 8) v1 ==== 35553+0+0 (secure 0 0 0) 0x7fafa00047e0 con 0x7fafa8130430 2023-09-30T22:29:17.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.493+0000 7f342affd700 1 -- 172.21.15.16:0/3588940891 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 8) v1 ==== 35553+0+0 (secure 0 0 0) 0x7f342c01d410 con 0x7f3434133fa0 2023-09-30T22:29:17.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.493+0000 7f45227fc700 1 -- 172.21.15.16:0/2700450527 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 8) v1 ==== 35553+0+0 (secure 0 0 0) 0x7f4528012750 con 0x7f452c09d6f0 2023-09-30T22:29:17.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.495+0000 7f343c3a3700 1 -- 172.21.15.16:0/3588940891 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3434123170 con 0x7f3434133fa0 2023-09-30T22:29:17.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.495+0000 7f342affd700 1 -- 172.21.15.16:0/3588940891 <== mon.0 v2:172.21.15.16:3300/0 8 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f342c0253b0 con 0x7f3434133fa0 2023-09-30T22:29:17.496 INFO:teuthology.orchestra.run.smithi016.stdout:38654705697 2023-09-30T22:29:17.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.498+0000 7f343c3a3700 1 -- 172.21.15.16:0/3588940891 >> v1:172.21.15.16:6816/126811 conn(0x7f341802f240 legacy=0x7f34180316f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:17.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.498+0000 7f343c3a3700 1 -- 172.21.15.16:0/3588940891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3434133fa0 msgr2=0x7f3434120e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:17.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.498+0000 7f343c3a3700 1 --2- 172.21.15.16:0/3588940891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3434133fa0 0x7f3434120e30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f342c007d90 tx=0x7f342c00e3c0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.499+0000 7f343c3a3700 1 -- 172.21.15.16:0/3588940891 shutdown_connections 2023-09-30T22:29:17.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.499+0000 7f343c3a3700 1 --2- 172.21.15.16:0/3588940891 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f34340a0a30 0x7f34341203b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.499+0000 7f343c3a3700 1 --2- 172.21.15.16:0/3588940891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3434133fa0 0x7f3434120e30 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.499+0000 7f343c3a3700 1 --2- 172.21.15.16:0/3588940891 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f34340a1340 0x7f34341208f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.499+0000 7f343c3a3700 1 -- 172.21.15.16:0/3588940891 >> 172.21.15.16:0/3588940891 conn(0x7f34340a6550 msgr2=0x7f34341bb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:17.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.499+0000 7f343c3a3700 1 -- 172.21.15.16:0/3588940891 shutdown_connections 2023-09-30T22:29:17.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.499+0000 7f343c3a3700 1 -- 172.21.15.16:0/3588940891 wait complete. 2023-09-30T22:29:17.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.505+0000 7f557f799700 1 Processor -- start 2023-09-30T22:29:17.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.506+0000 7f557f799700 1 -- start start 2023-09-30T22:29:17.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.506+0000 7f557f799700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f557809d6f0 0x7f557809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:17.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.506+0000 7f557f799700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f557809e000 0x7f5578130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:17.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.506+0000 7f557f799700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f557809f130 con 0x7f557809e410 2023-09-30T22:29:17.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.506+0000 7f557f799700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f557809f520 con 0x7f557809e000 2023-09-30T22:29:17.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.506+0000 7f557f799700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f557809e930 con 0x7f557809d6f0 2023-09-30T22:29:17.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.507+0000 7f557dd36700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f557809e410 0x7f5578130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33762/0 (socket says 172.21.15.16:33762) 2023-09-30T22:29:17.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.507+0000 7f557dd36700 1 -- 172.21.15.16:0/1646307327 learned_addr learned my addr 172.21.15.16:0/1646307327 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:29:17.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.507+0000 7f557d535700 1 --2- 172.21.15.16:0/1646307327 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f557809d6f0 0x7f557809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:17.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.507+0000 7f557cd34700 1 --2- 172.21.15.16:0/1646307327 >> v2:172.21.15.16:3300/0 conn(0x7f557809e000 0x7f5578130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:17.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.507+0000 7f556ffff700 1 -- 172.21.15.16:0/1646307327 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1104419776 0 0) 0x7f557809f130 con 0x7f557809e410 2023-09-30T22:29:17.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.507+0000 7f556ffff700 1 -- 172.21.15.16:0/1646307327 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55640035b0 con 0x7f557809e410 2023-09-30T22:29:17.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.507+0000 7f557d535700 1 -- 172.21.15.16:0/1646307327 >> v2:172.21.15.16:3300/0 conn(0x7f557809e000 msgr2=0x7f5578130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:17.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.507+0000 7f557d535700 1 --2- 172.21.15.16:0/1646307327 >> v2:172.21.15.16:3300/0 conn(0x7f557809e000 0x7f5578130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.507+0000 7f557d535700 1 -- 172.21.15.16:0/1646307327 >> v1:172.21.15.16:6789/0 conn(0x7f557809e410 legacy=0x7f5578130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:17.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.507+0000 7f557d535700 1 -- 172.21.15.16:0/1646307327 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f557809b2d0 con 0x7f557809d6f0 2023-09-30T22:29:17.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.508+0000 7f557d535700 1 --2- 172.21.15.16:0/1646307327 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f557809d6f0 0x7f557809dac0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5574009840 tx=0x7f5574009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a14b148e0b3b4daf server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:17.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.508+0000 7f556ffff700 1 -- 172.21.15.16:0/1646307327 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5574011830 con 0x7f557809d6f0 2023-09-30T22:29:17.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.508+0000 7f556ffff700 1 -- 172.21.15.16:0/1646307327 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5574011990 con 0x7f557809d6f0 2023-09-30T22:29:17.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.508+0000 7f556ffff700 1 -- 172.21.15.16:0/1646307327 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5574011d20 con 0x7f557809d6f0 2023-09-30T22:29:17.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.509+0000 7f557f799700 1 -- 172.21.15.16:0/1646307327 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f557809d6f0 msgr2=0x7f557809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:17.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.509+0000 7f557f799700 1 --2- 172.21.15.16:0/1646307327 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f557809d6f0 0x7f557809dac0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5574009840 tx=0x7f5574009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:17.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.509+0000 7f557f799700 1 -- 172.21.15.16:0/1646307327 shutdown_connections 2023-09-30T22:29:17.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.509+0000 7f557f799700 1 --2- 172.21.15.16:0/1646307327 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f557809d6f0 0x7f557809dac0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.509+0000 7f557f799700 1 --2- 172.21.15.16:0/1646307327 >> v2:172.21.15.16:3300/0 conn(0x7f557809e000 0x7f5578130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.509+0000 7f557f799700 1 -- 172.21.15.16:0/1646307327 >> 172.21.15.16:0/1646307327 conn(0x7f5578066f50 msgr2=0x7f557809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:17.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.509+0000 7f557f799700 1 -- 172.21.15.16:0/1646307327 shutdown_connections 2023-09-30T22:29:17.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.509+0000 7f557f799700 1 -- 172.21.15.16:0/1646307327 wait complete. 2023-09-30T22:29:17.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.510+0000 7f557f799700 1 Processor -- start 2023-09-30T22:29:17.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.510+0000 7f557f799700 1 -- start start 2023-09-30T22:29:17.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.511+0000 7f557f799700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f557809d6f0 0x7f55781301c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:17.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.511+0000 7f557f799700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f557809e000 0x7f5578130700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:17.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.511+0000 7f557f799700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f557809e410 0x7f5578130c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:17.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.511+0000 7f557f799700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f557812e420 con 0x7f557809e000 2023-09-30T22:29:17.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.511+0000 7f557f799700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f557812f3a0 con 0x7f557809e410 2023-09-30T22:29:17.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.511+0000 7f557f799700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f557812f6f0 con 0x7f557809d6f0 2023-09-30T22:29:17.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.511+0000 7f557cd34700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f557809e000 0x7f5578130700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:17.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.511+0000 7f557cd34700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f557809e000 0x7f5578130700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57598/0 (socket says 172.21.15.16:57598) 2023-09-30T22:29:17.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.511+0000 7f557cd34700 1 -- 172.21.15.16:0/2111111629 learned_addr learned my addr 172.21.15.16:0/2111111629 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:17.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.511+0000 7f557d535700 1 --2- 172.21.15.16:0/2111111629 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f557809d6f0 0x7f55781301c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:17.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.511+0000 7f557cd34700 1 -- 172.21.15.16:0/2111111629 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f557809d6f0 msgr2=0x7f55781301c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:17.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.511+0000 7f557cd34700 1 --2- 172.21.15.16:0/2111111629 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f557809d6f0 0x7f55781301c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.511+0000 7f557cd34700 1 -- 172.21.15.16:0/2111111629 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f557809e410 msgr2=0x7f5578130c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:17.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.511+0000 7f557cd34700 1 --2- 172.21.15.16:0/2111111629 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f557809e410 0x7f5578130c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.511+0000 7f557cd34700 1 -- 172.21.15.16:0/2111111629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55740094a0 con 0x7f557809e000 2023-09-30T22:29:17.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.512+0000 7f557cd34700 1 --2- 172.21.15.16:0/2111111629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f557809e000 0x7f5578130700 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f556800e9e0 tx=0x7f556800ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:17.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.512+0000 7f556dffb700 1 -- 172.21.15.16:0/2111111629 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5568009710 con 0x7f557809e000 2023-09-30T22:29:17.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.512+0000 7f556dffb700 1 -- 172.21.15.16:0/2111111629 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f556800c7f0 con 0x7f557809e000 2023-09-30T22:29:17.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.512+0000 7f557f799700 1 -- 172.21.15.16:0/2111111629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f557812f040 con 0x7f557809e000 2023-09-30T22:29:17.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.512+0000 7f556dffb700 1 -- 172.21.15.16:0/2111111629 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f556800cb80 con 0x7f557809e000 2023-09-30T22:29:17.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.512+0000 7f557f799700 1 -- 172.21.15.16:0/2111111629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f557812e580 con 0x7f557809e000 2023-09-30T22:29:17.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.513+0000 7f556dffb700 1 -- 172.21.15.16:0/2111111629 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 35553+0+0 (secure 0 0 0) 0x7f556800cce0 con 0x7f557809e000 2023-09-30T22:29:17.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.513+0000 7f557f799700 1 -- 172.21.15.16:0/2111111629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5578074ea0 con 0x7f557809e000 2023-09-30T22:29:17.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.513+0000 7f556dffb700 1 -- 172.21.15.16:0/2111111629 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4049+0+0 (secure 0 0 0) 0x7f556803ff10 con 0x7f557809e000 2023-09-30T22:29:17.517 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705696 got 38654705697 for osd.2 2023-09-30T22:29:17.517 DEBUG:teuthology.parallel:result is None 2023-09-30T22:29:17.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.519+0000 7f556dffb700 1 -- 172.21.15.16:0/2111111629 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5568011b40 con 0x7f557809e000 2023-09-30T22:29:17.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.596+0000 7fafafef1700 1 -- 172.21.15.16:0/344630019 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fafa80713a0 con 0x7fafa8130430 2023-09-30T22:29:17.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.596+0000 7faf9e7fc700 1 -- 172.21.15.16:0/344630019 <== mon.2 v2:172.21.15.16:3302/0 8 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fafa0003e30 con 0x7fafa8130430 2023-09-30T22:29:17.597 INFO:teuthology.orchestra.run.smithi016.stdout:38654705696 2023-09-30T22:29:17.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.599+0000 7fafafef1700 1 -- 172.21.15.16:0/344630019 >> v1:172.21.15.16:6816/126811 conn(0x7faf8c02f240 legacy=0x7faf8c0316f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:17.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.599+0000 7fafafef1700 1 -- 172.21.15.16:0/344630019 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fafa8130430 msgr2=0x7fafa809dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:17.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.599+0000 7fafafef1700 1 --2- 172.21.15.16:0/344630019 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fafa8130430 0x7fafa809dff0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fafa00131c0 tx=0x7fafa0013580 comp rx=0 tx=0).stop 2023-09-30T22:29:17.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.599+0000 7fafafef1700 1 -- 172.21.15.16:0/344630019 shutdown_connections 2023-09-30T22:29:17.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.599+0000 7fafafef1700 1 --2- 172.21.15.16:0/344630019 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fafa8130430 0x7fafa809dff0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.599+0000 7fafafef1700 1 --2- 172.21.15.16:0/344630019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafa812cf60 0x7fafa809d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.599+0000 7fafafef1700 1 --2- 172.21.15.16:0/344630019 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fafa812f8b0 0x7fafa809dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.599+0000 7fafafef1700 1 -- 172.21.15.16:0/344630019 >> 172.21.15.16:0/344630019 conn(0x7fafa80a2aa0 msgr2=0x7fafa8129650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:17.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.599+0000 7fafafef1700 1 -- 172.21.15.16:0/344630019 shutdown_connections 2023-09-30T22:29:17.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.599+0000 7fafafef1700 1 -- 172.21.15.16:0/344630019 wait complete. 2023-09-30T22:29:17.613 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705695 got 38654705696 for osd.3 2023-09-30T22:29:17.613 DEBUG:teuthology.parallel:result is None 2023-09-30T22:29:17.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.685+0000 7f45341fe700 1 -- 172.21.15.16:0/2700450527 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f452c1a8c20 con 0x7f452c09d6f0 2023-09-30T22:29:17.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.686+0000 7f45227fc700 1 -- 172.21.15.16:0/2700450527 <== mon.0 v2:172.21.15.16:3300/0 8 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4528011c70 con 0x7f452c09d6f0 2023-09-30T22:29:17.686 INFO:teuthology.orchestra.run.smithi016.stdout:38654705696 2023-09-30T22:29:17.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.688+0000 7f45341fe700 1 -- 172.21.15.16:0/2700450527 >> v1:172.21.15.16:6816/126811 conn(0x7f451002f240 legacy=0x7f45100316f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:17.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.688+0000 7f45341fe700 1 -- 172.21.15.16:0/2700450527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f452c09d6f0 msgr2=0x7f452c1a5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:17.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.688+0000 7f45341fe700 1 --2- 172.21.15.16:0/2700450527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f452c09d6f0 0x7f452c1a5de0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f452800e8a0 tx=0x7f452800ebb0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.688+0000 7f45341fe700 1 -- 172.21.15.16:0/2700450527 shutdown_connections 2023-09-30T22:29:17.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.688+0000 7f45341fe700 1 --2- 172.21.15.16:0/2700450527 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f452c09e410 0x7f452c1a6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.688+0000 7f45341fe700 1 --2- 172.21.15.16:0/2700450527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f452c09d6f0 0x7f452c1a5de0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.688+0000 7f45341fe700 1 --2- 172.21.15.16:0/2700450527 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f452c09e000 0x7f452c1a6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.688+0000 7f45341fe700 1 -- 172.21.15.16:0/2700450527 >> 172.21.15.16:0/2700450527 conn(0x7f452c066f50 msgr2=0x7f452c1bad60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:17.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.688+0000 7f45341fe700 1 -- 172.21.15.16:0/2700450527 shutdown_connections 2023-09-30T22:29:17.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.688+0000 7f45341fe700 1 -- 172.21.15.16:0/2700450527 wait complete. 2023-09-30T22:29:17.702 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705695 got 38654705696 for osd.1 2023-09-30T22:29:17.702 DEBUG:teuthology.parallel:result is None 2023-09-30T22:29:17.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.729+0000 7f557f799700 1 -- 172.21.15.16:0/2111111629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f557806c360 con 0x7f557809e000 2023-09-30T22:29:17.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.729+0000 7f556dffb700 1 -- 172.21.15.16:0/2111111629 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5568017070 con 0x7f557809e000 2023-09-30T22:29:17.730 INFO:teuthology.orchestra.run.smithi016.stdout:38654705696 2023-09-30T22:29:17.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.732+0000 7f557f799700 1 -- 172.21.15.16:0/2111111629 >> v1:172.21.15.16:6816/126811 conn(0x7f555c02f480 legacy=0x7f555c031930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:17.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.732+0000 7f557f799700 1 -- 172.21.15.16:0/2111111629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f557809e000 msgr2=0x7f5578130700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:17.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.732+0000 7f557f799700 1 --2- 172.21.15.16:0/2111111629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f557809e000 0x7f5578130700 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f556800e9e0 tx=0x7f556800ecf0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.732+0000 7f557f799700 1 -- 172.21.15.16:0/2111111629 shutdown_connections 2023-09-30T22:29:17.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.732+0000 7f557f799700 1 --2- 172.21.15.16:0/2111111629 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f557809e410 0x7f5578130c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.732+0000 7f557f799700 1 --2- 172.21.15.16:0/2111111629 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f557809d6f0 0x7f55781301c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.732+0000 7f557f799700 1 --2- 172.21.15.16:0/2111111629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f557809e000 0x7f5578130700 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.732+0000 7f557f799700 1 -- 172.21.15.16:0/2111111629 >> 172.21.15.16:0/2111111629 conn(0x7f5578066f50 msgr2=0x7f557812a700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:17.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.732+0000 7f557f799700 1 -- 172.21.15.16:0/2111111629 shutdown_connections 2023-09-30T22:29:17.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.732+0000 7f557f799700 1 -- 172.21.15.16:0/2111111629 wait complete. 2023-09-30T22:29:17.746 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705695 got 38654705696 for osd.0 2023-09-30T22:29:17.746 DEBUG:teuthology.parallel:result is None 2023-09-30T22:29:17.746 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-09-30T22:29:17.746 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-30T22:29:17.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.926+0000 7f5b0326b700 1 Processor -- start 2023-09-30T22:29:17.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.926+0000 7f5b0326b700 1 -- start start 2023-09-30T22:29:17.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.927+0000 7f5b0326b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5afc09d6f0 0x7f5afc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:17.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.927+0000 7f5b0326b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5afc09e000 0x7f5afc130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:17.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.927+0000 7f5b0326b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5afc09e410 0x7f5afc130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:17.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.927+0000 7f5b0326b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5afc09f520 con 0x7f5afc09d6f0 2023-09-30T22:29:17.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.927+0000 7f5b0326b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5afc09f130 con 0x7f5afc09e410 2023-09-30T22:29:17.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.927+0000 7f5b0326b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5afc09e930 con 0x7f5afc09e000 2023-09-30T22:29:17.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.927+0000 7f5b00806700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5afc09e000 0x7f5afc130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:17.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.927+0000 7f5b00806700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5afc09e000 0x7f5afc130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:55752/0 (socket says 172.21.15.16:55752) 2023-09-30T22:29:17.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.927+0000 7f5b00806700 1 -- 172.21.15.16:0/440799864 learned_addr learned my addr 172.21.15.16:0/440799864 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:17.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.927+0000 7f5b01007700 1 --2- 172.21.15.16:0/440799864 >> v2:172.21.15.16:3300/0 conn(0x7f5afc09d6f0 0x7f5afc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:17.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.927+0000 7f5b01808700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5afc09e410 0x7f5afc130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:17.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.927+0000 7f5b00806700 1 -- 172.21.15.16:0/440799864 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5afc09e410 msgr2=0x7f5afc130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:17.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.927+0000 7f5b00806700 1 --2- 172.21.15.16:0/440799864 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5afc09e410 0x7f5afc130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.927+0000 7f5b00806700 1 -- 172.21.15.16:0/440799864 >> v2:172.21.15.16:3300/0 conn(0x7f5afc09d6f0 msgr2=0x7f5afc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:17.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.927+0000 7f5b00806700 1 --2- 172.21.15.16:0/440799864 >> v2:172.21.15.16:3300/0 conn(0x7f5afc09d6f0 0x7f5afc09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.927+0000 7f5b00806700 1 -- 172.21.15.16:0/440799864 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5afc09b2d0 con 0x7f5afc09e000 2023-09-30T22:29:17.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.927+0000 7f5b01007700 1 --2- 172.21.15.16:0/440799864 >> v2:172.21.15.16:3300/0 conn(0x7f5afc09d6f0 0x7f5afc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:17.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.927+0000 7f5b00806700 1 --2- 172.21.15.16:0/440799864 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5afc09e000 0x7f5afc130660 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5af8009920 tx=0x7f5af8009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f8235dabd340adf server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:17.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.928+0000 7f5aeffff700 1 -- 172.21.15.16:0/440799864 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af8011830 con 0x7f5afc09e000 2023-09-30T22:29:17.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.928+0000 7f5aeffff700 1 -- 172.21.15.16:0/440799864 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5af8011990 con 0x7f5afc09e000 2023-09-30T22:29:17.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.928+0000 7f5aeffff700 1 -- 172.21.15.16:0/440799864 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af8011d20 con 0x7f5afc09e000 2023-09-30T22:29:17.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.928+0000 7f5b0326b700 1 -- 172.21.15.16:0/440799864 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5afc09e000 msgr2=0x7f5afc130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:17.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.928+0000 7f5b0326b700 1 --2- 172.21.15.16:0/440799864 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5afc09e000 0x7f5afc130660 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5af8009920 tx=0x7f5af8009c30 comp rx=0 tx=0).stop 2023-09-30T22:29:17.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.928+0000 7f5b0326b700 1 -- 172.21.15.16:0/440799864 shutdown_connections 2023-09-30T22:29:17.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.928+0000 7f5b0326b700 1 --2- 172.21.15.16:0/440799864 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5afc09e410 0x7f5afc130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.928+0000 7f5b0326b700 1 --2- 172.21.15.16:0/440799864 >> v2:172.21.15.16:3300/0 conn(0x7f5afc09d6f0 0x7f5afc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.928+0000 7f5b0326b700 1 --2- 172.21.15.16:0/440799864 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5afc09e000 0x7f5afc130660 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.928+0000 7f5b0326b700 1 -- 172.21.15.16:0/440799864 >> 172.21.15.16:0/440799864 conn(0x7f5afc066f50 msgr2=0x7f5afc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:17.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.928+0000 7f5b0326b700 1 -- 172.21.15.16:0/440799864 shutdown_connections 2023-09-30T22:29:17.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.928+0000 7f5b0326b700 1 -- 172.21.15.16:0/440799864 wait complete. 2023-09-30T22:29:17.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.929+0000 7f5b0326b700 1 Processor -- start 2023-09-30T22:29:17.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.929+0000 7f5b0326b700 1 -- start start 2023-09-30T22:29:17.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.930+0000 7f5b0326b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5afc09d6f0 0x7f5afc1c85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:17.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.930+0000 7f5b0326b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5afc09e000 0x7f5afc1c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:17.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.930+0000 7f5b0326b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5afc09e410 0x7f5afc1c9070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:17.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.930+0000 7f5b0326b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5afc125fd0 con 0x7f5afc09d6f0 2023-09-30T22:29:17.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.930+0000 7f5b0326b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5afc11f350 con 0x7f5afc09e000 2023-09-30T22:29:17.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.930+0000 7f5b0326b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5afc1261d0 con 0x7f5afc09e410 2023-09-30T22:29:17.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.930+0000 7f5b00806700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5afc09e000 0x7f5afc1c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:17.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.930+0000 7f5b01808700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5afc09e410 0x7f5afc1c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:17.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.930+0000 7f5b00806700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5afc09e000 0x7f5afc1c8b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37048/0 (socket says 172.21.15.16:37048) 2023-09-30T22:29:17.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.930+0000 7f5b00806700 1 -- 172.21.15.16:0/1514391743 learned_addr learned my addr 172.21.15.16:0/1514391743 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:17.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.930+0000 7f5b01007700 1 --2- 172.21.15.16:0/1514391743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5afc09d6f0 0x7f5afc1c85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:17.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.930+0000 7f5b01808700 1 -- 172.21.15.16:0/1514391743 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5afc09e000 msgr2=0x7f5afc1c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:17.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.930+0000 7f5b01808700 1 --2- 172.21.15.16:0/1514391743 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5afc09e000 0x7f5afc1c8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.930+0000 7f5b01808700 1 -- 172.21.15.16:0/1514391743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5afc09d6f0 msgr2=0x7f5afc1c85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:17.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.930+0000 7f5b01808700 1 --2- 172.21.15.16:0/1514391743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5afc09d6f0 0x7f5afc1c85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:17.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.930+0000 7f5b01808700 1 -- 172.21.15.16:0/1514391743 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af00094a0 con 0x7f5afc09e410 2023-09-30T22:29:17.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.930+0000 7f5b00806700 1 --2- 172.21.15.16:0/1514391743 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5afc09e000 0x7f5afc1c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:17.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.931+0000 7f5b01808700 1 --2- 172.21.15.16:0/1514391743 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5afc09e410 0x7f5afc1c9070 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5af4009d90 tx=0x7f5af400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:17.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.931+0000 7f5aedffb700 1 -- 172.21.15.16:0/1514391743 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af4009c00 con 0x7f5afc09e410 2023-09-30T22:29:17.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.931+0000 7f5aedffb700 1 -- 172.21.15.16:0/1514391743 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5af400f040 con 0x7f5afc09e410 2023-09-30T22:29:17.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.931+0000 7f5b0326b700 1 -- 172.21.15.16:0/1514391743 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5af8009550 con 0x7f5afc09e410 2023-09-30T22:29:17.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.931+0000 7f5aedffb700 1 -- 172.21.15.16:0/1514391743 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af4014480 con 0x7f5afc09e410 2023-09-30T22:29:17.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.931+0000 7f5b0326b700 1 -- 172.21.15.16:0/1514391743 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5afc0a2ae0 con 0x7f5afc09e410 2023-09-30T22:29:17.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.932+0000 7f5aedffb700 1 -- 172.21.15.16:0/1514391743 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 8) v1 ==== 35553+0+0 (secure 0 0 0) 0x7f5af40145e0 con 0x7f5afc09e410 2023-09-30T22:29:17.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.932+0000 7f5b0326b700 1 -- 172.21.15.16:0/1514391743 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5afc074ea0 con 0x7f5afc09e410 2023-09-30T22:29:17.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.932+0000 7f5aedffb700 1 -- 172.21.15.16:0/1514391743 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4049+0+0 (secure 0 0 0) 0x7f5af40406e0 con 0x7f5afc09e410 2023-09-30T22:29:17.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:17.937+0000 7f5aedffb700 1 -- 172.21.15.16:0/1514391743 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5af4007920 con 0x7f5afc09e410 2023-09-30T22:29:18.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.137+0000 7f5b0326b700 1 -- 172.21.15.16:0/1514391743 --> v1:172.21.15.16:6816/126811 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5afc09c450 con 0x7f5ae002f480 2023-09-30T22:29:18.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.145+0000 7f5aedffb700 1 -- 172.21.15.16:0/1514391743 <== mgr.14103 v1:172.21.15.16:6816/126811 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+270764 (unknown 2965378022 0 3274324205) 0x7f5af00098c0 con 0x7f5ae002f480 2023-09-30T22:29:18.149 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:29:18.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.148+0000 7f5b0326b700 1 -- 172.21.15.16:0/1514391743 >> v1:172.21.15.16:6816/126811 conn(0x7f5ae002f480 legacy=0x7f5ae0031930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:18.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.148+0000 7f5b0326b700 1 -- 172.21.15.16:0/1514391743 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5afc09e410 msgr2=0x7f5afc1c9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:18.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.148+0000 7f5b0326b700 1 --2- 172.21.15.16:0/1514391743 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5afc09e410 0x7f5afc1c9070 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5af4009d90 tx=0x7f5af400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:29:18.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.148+0000 7f5b0326b700 1 -- 172.21.15.16:0/1514391743 shutdown_connections 2023-09-30T22:29:18.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.148+0000 7f5b0326b700 1 --2- 172.21.15.16:0/1514391743 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5afc09e410 0x7f5afc1c9070 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:18.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.148+0000 7f5b0326b700 1 --2- 172.21.15.16:0/1514391743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5afc09d6f0 0x7f5afc1c85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:18.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.148+0000 7f5b0326b700 1 --2- 172.21.15.16:0/1514391743 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5afc09e000 0x7f5afc1c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:18.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.148+0000 7f5b0326b700 1 -- 172.21.15.16:0/1514391743 >> 172.21.15.16:0/1514391743 conn(0x7f5afc066f50 msgr2=0x7f5afc119480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:18.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.148+0000 7f5b0326b700 1 -- 172.21.15.16:0/1514391743 shutdown_connections 2023-09-30T22:29:18.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.148+0000 7f5b0326b700 1 -- 172.21.15.16:0/1514391743 wait complete. 2023-09-30T22:29:18.153 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-09-30T22:29:18.162 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2023-09-30T22:29:16.495420+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4333,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":51,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":45,"ondisk_log_size":45,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1189092,"kb_used_data":708,"kb_used_omap":0,"kb_used_meta":1188352,"kb_avail":376298268,"statfs":{"total":386547056640,"available":385329426432,"internally_reserved":0,"allocated":724992,"data_stored":243006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1216872448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.001767"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:29:14.583604+0000","last_change":"2023-09-30T22:26:51.708966+0000","last_active":"2023-09-30T22:29:14.583604+0000","last_peered":"2023-09-30T22:29:14.583604+0000","last_clean":"2023-09-30T22:29:14.583604+0000","last_became_active":"2023-09-30T22:26:51.708653+0000","last_became_peered":"2023-09-30T22:26:51.708653+0000","last_unstale":"2023-09-30T22:29:14.583604+0000","last_undegraded":"2023-09-30T22:29:14.583604+0000","last_fullsized":"2023-09-30T22:29:14.583604+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:29:14.583358+0000","last_change":"2023-09-30T22:26:51.727077+0000","last_active":"2023-09-30T22:29:14.583358+0000","last_peered":"2023-09-30T22:29:14.583358+0000","last_clean":"2023-09-30T22:29:14.583358+0000","last_became_active":"2023-09-30T22:26:51.726778+0000","last_became_peered":"2023-09-30T22:26:51.726778+0000","last_unstale":"2023-09-30T22:29:14.583358+0000","last_undegraded":"2023-09-30T22:29:14.583358+0000","last_fullsized":"2023-09-30T22:29:14.583358+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.238854+0000","last_change":"2023-09-30T22:26:51.706919+0000","last_active":"2023-09-30T22:27:42.238854+0000","last_peered":"2023-09-30T22:27:42.238854+0000","last_clean":"2023-09-30T22:27:42.238854+0000","last_became_active":"2023-09-30T22:26:51.706676+0000","last_became_peered":"2023-09-30T22:26:51.706676+0000","last_unstale":"2023-09-30T22:27:42.238854+0000","last_undegraded":"2023-09-30T22:27:42.238854+0000","last_fullsized":"2023-09-30T22:27:42.238854+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.238471+0000","last_change":"2023-09-30T22:26:51.727386+0000","last_active":"2023-09-30T22:27:42.238471+0000","last_peered":"2023-09-30T22:27:42.238471+0000","last_clean":"2023-09-30T22:27:42.238471+0000","last_became_active":"2023-09-30T22:26:51.727091+0000","last_became_peered":"2023-09-30T22:26:51.727091+0000","last_unstale":"2023-09-30T22:27:42.238471+0000","last_undegraded":"2023-09-30T22:27:42.238471+0000","last_fullsized":"2023-09-30T22:27:42.238471+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.238005+0000","last_change":"2023-09-30T22:26:51.719993+0000","last_active":"2023-09-30T22:27:42.238005+0000","last_peered":"2023-09-30T22:27:42.238005+0000","last_clean":"2023-09-30T22:27:42.238005+0000","last_became_active":"2023-09-30T22:26:51.719495+0000","last_became_peered":"2023-09-30T22:26:51.719495+0000","last_unstale":"2023-09-30T22:27:42.238005+0000","last_undegraded":"2023-09-30T22:27:42.238005+0000","last_fullsized":"2023-09-30T22:27:42.238005+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.241844+0000","last_change":"2023-09-30T22:27:42.240573+0000","last_active":"2023-09-30T22:27:42.241844+0000","last_peered":"2023-09-30T22:27:42.241844+0000","last_clean":"2023-09-30T22:27:42.241844+0000","last_became_active":"2023-09-30T22:27:42.240236+0000","last_became_peered":"2023-09-30T22:27:42.240236+0000","last_unstale":"2023-09-30T22:27:42.241844+0000","last_undegraded":"2023-09-30T22:27:42.241844+0000","last_fullsized":"2023-09-30T22:27:42.241844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.233985+0000","last_change":"2023-09-30T22:26:51.705826+0000","last_active":"2023-09-30T22:27:42.233985+0000","last_peered":"2023-09-30T22:27:42.233985+0000","last_clean":"2023-09-30T22:27:42.233985+0000","last_became_active":"2023-09-30T22:26:51.705037+0000","last_became_peered":"2023-09-30T22:26:51.705037+0000","last_unstale":"2023-09-30T22:27:42.233985+0000","last_undegraded":"2023-09-30T22:27:42.233985+0000","last_fullsized":"2023-09-30T22:27:42.233985+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.394405+0000","last_change":"2023-09-30T22:26:51.726640+0000","last_active":"2023-09-30T22:27:42.394405+0000","last_peered":"2023-09-30T22:27:42.394405+0000","last_clean":"2023-09-30T22:27:42.394405+0000","last_became_active":"2023-09-30T22:26:51.726278+0000","last_became_peered":"2023-09-30T22:26:51.726278+0000","last_unstale":"2023-09-30T22:27:42.394405+0000","last_undegraded":"2023-09-30T22:27:42.394405+0000","last_fullsized":"2023-09-30T22:27:42.394405+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.239452+0000","last_change":"2023-09-30T22:26:51.721817+0000","last_active":"2023-09-30T22:27:42.239452+0000","last_peered":"2023-09-30T22:27:42.239452+0000","last_clean":"2023-09-30T22:27:42.239452+0000","last_became_active":"2023-09-30T22:26:51.721373+0000","last_became_peered":"2023-09-30T22:26:51.721373+0000","last_unstale":"2023-09-30T22:27:42.239452+0000","last_undegraded":"2023-09-30T22:27:42.239452+0000","last_fullsized":"2023-09-30T22:27:42.239452+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.395173+0000","last_change":"2023-09-30T22:26:51.726348+0000","last_active":"2023-09-30T22:27:42.395173+0000","last_peered":"2023-09-30T22:27:42.395173+0000","last_clean":"2023-09-30T22:27:42.395173+0000","last_became_active":"2023-09-30T22:26:51.726030+0000","last_became_peered":"2023-09-30T22:26:51.726030+0000","last_unstale":"2023-09-30T22:27:42.395173+0000","last_undegraded":"2023-09-30T22:27:42.395173+0000","last_fullsized":"2023-09-30T22:27:42.395173+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.394692+0000","last_change":"2023-09-30T22:26:51.722619+0000","last_active":"2023-09-30T22:27:42.394692+0000","last_peered":"2023-09-30T22:27:42.394692+0000","last_clean":"2023-09-30T22:27:42.394692+0000","last_became_active":"2023-09-30T22:26:51.722317+0000","last_became_peered":"2023-09-30T22:26:51.722317+0000","last_unstale":"2023-09-30T22:27:42.394692+0000","last_undegraded":"2023-09-30T22:27:42.394692+0000","last_fullsized":"2023-09-30T22:27:42.394692+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:29:14.582628+0000","last_change":"2023-09-30T22:26:51.709697+0000","last_active":"2023-09-30T22:29:14.582628+0000","last_peered":"2023-09-30T22:29:14.582628+0000","last_clean":"2023-09-30T22:29:14.582628+0000","last_became_active":"2023-09-30T22:26:51.709342+0000","last_became_peered":"2023-09-30T22:26:51.709342+0000","last_unstale":"2023-09-30T22:29:14.582628+0000","last_undegraded":"2023-09-30T22:29:14.582628+0000","last_fullsized":"2023-09-30T22:29:14.582628+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:29:14.582455+0000","last_change":"2023-09-30T22:26:51.716276+0000","last_active":"2023-09-30T22:29:14.582455+0000","last_peered":"2023-09-30T22:29:14.582455+0000","last_clean":"2023-09-30T22:29:14.582455+0000","last_became_active":"2023-09-30T22:26:51.715634+0000","last_became_peered":"2023-09-30T22:26:51.715634+0000","last_unstale":"2023-09-30T22:29:14.582455+0000","last_undegraded":"2023-09-30T22:29:14.582455+0000","last_fullsized":"2023-09-30T22:29:14.582455+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.394593+0000","last_change":"2023-09-30T22:26:51.695948+0000","last_active":"2023-09-30T22:27:42.394593+0000","last_peered":"2023-09-30T22:27:42.394593+0000","last_clean":"2023-09-30T22:27:42.394593+0000","last_became_active":"2023-09-30T22:26:51.695661+0000","last_became_peered":"2023-09-30T22:26:51.695661+0000","last_unstale":"2023-09-30T22:27:42.394593+0000","last_undegraded":"2023-09-30T22:27:42.394593+0000","last_fullsized":"2023-09-30T22:27:42.394593+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.237389+0000","last_change":"2023-09-30T22:26:51.708532+0000","last_active":"2023-09-30T22:27:42.237389+0000","last_peered":"2023-09-30T22:27:42.237389+0000","last_clean":"2023-09-30T22:27:42.237389+0000","last_became_active":"2023-09-30T22:26:51.708204+0000","last_became_peered":"2023-09-30T22:26:51.708204+0000","last_unstale":"2023-09-30T22:27:42.237389+0000","last_undegraded":"2023-09-30T22:27:42.237389+0000","last_fullsized":"2023-09-30T22:27:42.237389+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.235077+0000","last_change":"2023-09-30T22:26:51.704597+0000","last_active":"2023-09-30T22:27:42.235077+0000","last_peered":"2023-09-30T22:27:42.235077+0000","last_clean":"2023-09-30T22:27:42.235077+0000","last_became_active":"2023-09-30T22:26:51.704307+0000","last_became_peered":"2023-09-30T22:26:51.704307+0000","last_unstale":"2023-09-30T22:27:42.235077+0000","last_undegraded":"2023-09-30T22:27:42.235077+0000","last_fullsized":"2023-09-30T22:27:42.235077+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.239386+0000","last_change":"2023-09-30T22:26:51.721640+0000","last_active":"2023-09-30T22:27:42.239386+0000","last_peered":"2023-09-30T22:27:42.239386+0000","last_clean":"2023-09-30T22:27:42.239386+0000","last_became_active":"2023-09-30T22:26:51.720950+0000","last_became_peered":"2023-09-30T22:26:51.720950+0000","last_unstale":"2023-09-30T22:27:42.239386+0000","last_undegraded":"2023-09-30T22:27:42.239386+0000","last_fullsized":"2023-09-30T22:27:42.239386+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.238988+0000","last_change":"2023-09-30T22:26:51.697513+0000","last_active":"2023-09-30T22:27:42.238988+0000","last_peered":"2023-09-30T22:27:42.238988+0000","last_clean":"2023-09-30T22:27:42.238988+0000","last_became_active":"2023-09-30T22:26:51.697137+0000","last_became_peered":"2023-09-30T22:26:51.697137+0000","last_unstale":"2023-09-30T22:27:42.238988+0000","last_undegraded":"2023-09-30T22:27:42.238988+0000","last_fullsized":"2023-09-30T22:27:42.238988+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:29:14.582991+0000","last_change":"2023-09-30T22:26:51.715476+0000","last_active":"2023-09-30T22:29:14.582991+0000","last_peered":"2023-09-30T22:29:14.582991+0000","last_clean":"2023-09-30T22:29:14.582991+0000","last_became_active":"2023-09-30T22:26:51.715176+0000","last_became_peered":"2023-09-30T22:26:51.715176+0000","last_unstale":"2023-09-30T22:29:14.582991+0000","last_undegraded":"2023-09-30T22:29:14.582991+0000","last_fullsized":"2023-09-30T22:29:14.582991+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.394819+0000","last_change":"2023-09-30T22:26:51.724991+0000","last_active":"2023-09-30T22:27:42.394819+0000","last_peered":"2023-09-30T22:27:42.394819+0000","last_clean":"2023-09-30T22:27:42.394819+0000","last_became_active":"2023-09-30T22:26:51.724240+0000","last_became_peered":"2023-09-30T22:26:51.724240+0000","last_unstale":"2023-09-30T22:27:42.394819+0000","last_undegraded":"2023-09-30T22:27:42.394819+0000","last_fullsized":"2023-09-30T22:27:42.394819+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.394105+0000","last_change":"2023-09-30T22:26:51.717833+0000","last_active":"2023-09-30T22:27:42.394105+0000","last_peered":"2023-09-30T22:27:42.394105+0000","last_clean":"2023-09-30T22:27:42.394105+0000","last_became_active":"2023-09-30T22:26:51.717517+0000","last_became_peered":"2023-09-30T22:26:51.717517+0000","last_unstale":"2023-09-30T22:27:42.394105+0000","last_undegraded":"2023-09-30T22:27:42.394105+0000","last_fullsized":"2023-09-30T22:27:42.394105+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.237155+0000","last_change":"2023-09-30T22:26:51.719030+0000","last_active":"2023-09-30T22:27:42.237155+0000","last_peered":"2023-09-30T22:27:42.237155+0000","last_clean":"2023-09-30T22:27:42.237155+0000","last_became_active":"2023-09-30T22:26:51.715370+0000","last_became_peered":"2023-09-30T22:26:51.715370+0000","last_unstale":"2023-09-30T22:27:42.237155+0000","last_undegraded":"2023-09-30T22:27:42.237155+0000","last_fullsized":"2023-09-30T22:27:42.237155+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.233718+0000","last_change":"2023-09-30T22:26:51.698664+0000","last_active":"2023-09-30T22:27:42.233718+0000","last_peered":"2023-09-30T22:27:42.233718+0000","last_clean":"2023-09-30T22:27:42.233718+0000","last_became_active":"2023-09-30T22:26:51.698321+0000","last_became_peered":"2023-09-30T22:26:51.698321+0000","last_unstale":"2023-09-30T22:27:42.233718+0000","last_undegraded":"2023-09-30T22:27:42.233718+0000","last_fullsized":"2023-09-30T22:27:42.233718+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.394494+0000","last_change":"2023-09-30T22:26:51.727955+0000","last_active":"2023-09-30T22:27:42.394494+0000","last_peered":"2023-09-30T22:27:42.394494+0000","last_clean":"2023-09-30T22:27:42.394494+0000","last_became_active":"2023-09-30T22:26:51.727209+0000","last_became_peered":"2023-09-30T22:26:51.727209+0000","last_unstale":"2023-09-30T22:27:42.394494+0000","last_undegraded":"2023-09-30T22:27:42.394494+0000","last_fullsized":"2023-09-30T22:27:42.394494+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.395261+0000","last_change":"2023-09-30T22:26:51.721154+0000","last_active":"2023-09-30T22:27:42.395261+0000","last_peered":"2023-09-30T22:27:42.395261+0000","last_clean":"2023-09-30T22:27:42.395261+0000","last_became_active":"2023-09-30T22:26:51.720703+0000","last_became_peered":"2023-09-30T22:26:51.720703+0000","last_unstale":"2023-09-30T22:27:42.395261+0000","last_undegraded":"2023-09-30T22:27:42.395261+0000","last_fullsized":"2023-09-30T22:27:42.395261+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.395135+0000","last_change":"2023-09-30T22:26:51.701309+0000","last_active":"2023-09-30T22:27:42.395135+0000","last_peered":"2023-09-30T22:27:42.395135+0000","last_clean":"2023-09-30T22:27:42.395135+0000","last_became_active":"2023-09-30T22:26:51.701012+0000","last_became_peered":"2023-09-30T22:26:51.701012+0000","last_unstale":"2023-09-30T22:27:42.395135+0000","last_undegraded":"2023-09-30T22:27:42.395135+0000","last_fullsized":"2023-09-30T22:27:42.395135+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.233911+0000","last_change":"2023-09-30T22:26:51.715042+0000","last_active":"2023-09-30T22:27:42.233911+0000","last_peered":"2023-09-30T22:27:42.233911+0000","last_clean":"2023-09-30T22:27:42.233911+0000","last_became_active":"2023-09-30T22:26:51.712158+0000","last_became_peered":"2023-09-30T22:26:51.712158+0000","last_unstale":"2023-09-30T22:27:42.233911+0000","last_undegraded":"2023-09-30T22:27:42.233911+0000","last_fullsized":"2023-09-30T22:27:42.233911+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"15'1","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.394384+0000","last_change":"2023-09-30T22:26:51.296397+0000","last_active":"2023-09-30T22:27:42.394384+0000","last_peered":"2023-09-30T22:27:42.394384+0000","last_clean":"2023-09-30T22:27:42.394384+0000","last_became_active":"2023-09-30T22:26:51.295860+0000","last_became_peered":"2023-09-30T22:26:51.295860+0000","last_unstale":"2023-09-30T22:27:42.394384+0000","last_undegraded":"2023-09-30T22:27:42.394384+0000","last_fullsized":"2023-09-30T22:27:42.394384+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.234008+0000","last_change":"2023-09-30T22:26:51.712472+0000","last_active":"2023-09-30T22:27:42.234008+0000","last_peered":"2023-09-30T22:27:42.234008+0000","last_clean":"2023-09-30T22:27:42.234008+0000","last_became_active":"2023-09-30T22:26:51.711527+0000","last_became_peered":"2023-09-30T22:26:51.711527+0000","last_unstale":"2023-09-30T22:27:42.234008+0000","last_undegraded":"2023-09-30T22:27:42.234008+0000","last_fullsized":"2023-09-30T22:27:42.234008+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.394016+0000","last_change":"2023-09-30T22:26:51.298004+0000","last_active":"2023-09-30T22:27:42.394016+0000","last_peered":"2023-09-30T22:27:42.394016+0000","last_clean":"2023-09-30T22:27:42.394016+0000","last_became_active":"2023-09-30T22:26:51.297714+0000","last_became_peered":"2023-09-30T22:26:51.297714+0000","last_unstale":"2023-09-30T22:27:42.394016+0000","last_undegraded":"2023-09-30T22:27:42.394016+0000","last_fullsized":"2023-09-30T22:27:42.394016+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.239079+0000","last_change":"2023-09-30T22:26:51.727608+0000","last_active":"2023-09-30T22:27:42.239079+0000","last_peered":"2023-09-30T22:27:42.239079+0000","last_clean":"2023-09-30T22:27:42.239079+0000","last_became_active":"2023-09-30T22:26:51.727154+0000","last_became_peered":"2023-09-30T22:26:51.727154+0000","last_unstale":"2023-09-30T22:27:42.239079+0000","last_undegraded":"2023-09-30T22:27:42.239079+0000","last_fullsized":"2023-09-30T22:27:42.239079+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.235317+0000","last_change":"2023-09-30T22:26:51.285214+0000","last_active":"2023-09-30T22:27:42.235317+0000","last_peered":"2023-09-30T22:27:42.235317+0000","last_clean":"2023-09-30T22:27:42.235317+0000","last_became_active":"2023-09-30T22:26:51.284749+0000","last_became_peered":"2023-09-30T22:26:51.284749+0000","last_unstale":"2023-09-30T22:27:42.235317+0000","last_undegraded":"2023-09-30T22:27:42.235317+0000","last_fullsized":"2023-09-30T22:27:42.235317+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.235520+0000","last_change":"2023-09-30T22:26:51.706697+0000","last_active":"2023-09-30T22:27:42.235520+0000","last_peered":"2023-09-30T22:27:42.235520+0000","last_clean":"2023-09-30T22:27:42.235520+0000","last_became_active":"2023-09-30T22:26:51.706373+0000","last_became_peered":"2023-09-30T22:26:51.706373+0000","last_unstale":"2023-09-30T22:27:42.235520+0000","last_undegraded":"2023-09-30T22:27:42.235520+0000","last_fullsized":"2023-09-30T22:27:42.235520+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.239279+0000","last_change":"2023-09-30T22:26:51.281642+0000","last_active":"2023-09-30T22:27:42.239279+0000","last_peered":"2023-09-30T22:27:42.239279+0000","last_clean":"2023-09-30T22:27:42.239279+0000","last_became_active":"2023-09-30T22:26:51.281319+0000","last_became_peered":"2023-09-30T22:26:51.281319+0000","last_unstale":"2023-09-30T22:27:42.239279+0000","last_undegraded":"2023-09-30T22:27:42.239279+0000","last_fullsized":"2023-09-30T22:27:42.239279+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.235304+0000","last_change":"2023-09-30T22:26:51.705192+0000","last_active":"2023-09-30T22:27:42.235304+0000","last_peered":"2023-09-30T22:27:42.235304+0000","last_clean":"2023-09-30T22:27:42.235304+0000","last_became_active":"2023-09-30T22:26:51.704870+0000","last_became_peered":"2023-09-30T22:26:51.704870+0000","last_unstale":"2023-09-30T22:27:42.235304+0000","last_undegraded":"2023-09-30T22:27:42.235304+0000","last_fullsized":"2023-09-30T22:27:42.235304+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:29:14.583093+0000","last_change":"2023-09-30T22:26:51.293711+0000","last_active":"2023-09-30T22:29:14.583093+0000","last_peered":"2023-09-30T22:29:14.583093+0000","last_clean":"2023-09-30T22:29:14.583093+0000","last_became_active":"2023-09-30T22:26:51.293410+0000","last_became_peered":"2023-09-30T22:26:51.293410+0000","last_unstale":"2023-09-30T22:29:14.583093+0000","last_undegraded":"2023-09-30T22:29:14.583093+0000","last_fullsized":"2023-09-30T22:29:14.583093+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.234676+0000","last_change":"2023-09-30T22:26:51.706005+0000","last_active":"2023-09-30T22:27:42.234676+0000","last_peered":"2023-09-30T22:27:42.234676+0000","last_clean":"2023-09-30T22:27:42.234676+0000","last_became_active":"2023-09-30T22:26:51.705491+0000","last_became_peered":"2023-09-30T22:26:51.705491+0000","last_unstale":"2023-09-30T22:27:42.234676+0000","last_undegraded":"2023-09-30T22:27:42.234676+0000","last_fullsized":"2023-09-30T22:27:42.234676+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"15'3","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.238196+0000","last_change":"2023-09-30T22:26:51.279976+0000","last_active":"2023-09-30T22:27:42.238196+0000","last_peered":"2023-09-30T22:27:42.238196+0000","last_clean":"2023-09-30T22:27:42.238196+0000","last_became_active":"2023-09-30T22:26:51.279669+0000","last_became_peered":"2023-09-30T22:26:51.279669+0000","last_unstale":"2023-09-30T22:27:42.238196+0000","last_undegraded":"2023-09-30T22:27:42.238196+0000","last_fullsized":"2023-09-30T22:27:42.238196+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.394529+0000","last_change":"2023-09-30T22:26:51.725947+0000","last_active":"2023-09-30T22:27:42.394529+0000","last_peered":"2023-09-30T22:27:42.394529+0000","last_clean":"2023-09-30T22:27:42.394529+0000","last_became_active":"2023-09-30T22:26:51.723294+0000","last_became_peered":"2023-09-30T22:26:51.723294+0000","last_unstale":"2023-09-30T22:27:42.394529+0000","last_undegraded":"2023-09-30T22:27:42.394529+0000","last_fullsized":"2023-09-30T22:27:42.394529+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"20'12","reported_seq":26,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:29:14.617123+0000","last_change":"2023-09-30T22:26:51.297527+0000","last_active":"2023-09-30T22:29:14.617123+0000","last_peered":"2023-09-30T22:29:14.617123+0000","last_clean":"2023-09-30T22:29:14.617123+0000","last_became_active":"2023-09-30T22:26:51.297246+0000","last_became_peered":"2023-09-30T22:26:51.297246+0000","last_unstale":"2023-09-30T22:29:14.617123+0000","last_undegraded":"2023-09-30T22:29:14.617123+0000","last_fullsized":"2023-09-30T22:29:14.617123+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3563,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:29:14.582565+0000","last_change":"2023-09-30T22:26:51.710781+0000","last_active":"2023-09-30T22:29:14.582565+0000","last_peered":"2023-09-30T22:29:14.582565+0000","last_clean":"2023-09-30T22:29:14.582565+0000","last_became_active":"2023-09-30T22:26:51.710107+0000","last_became_peered":"2023-09-30T22:26:51.710107+0000","last_unstale":"2023-09-30T22:29:14.582565+0000","last_undegraded":"2023-09-30T22:29:14.582565+0000","last_fullsized":"2023-09-30T22:29:14.582565+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"15'2","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.234355+0000","last_change":"2023-09-30T22:26:51.290194+0000","last_active":"2023-09-30T22:27:42.234355+0000","last_peered":"2023-09-30T22:27:42.234355+0000","last_clean":"2023-09-30T22:27:42.234355+0000","last_became_active":"2023-09-30T22:26:51.288688+0000","last_became_peered":"2023-09-30T22:26:51.288688+0000","last_unstale":"2023-09-30T22:27:42.234355+0000","last_undegraded":"2023-09-30T22:27:42.234355+0000","last_fullsized":"2023-09-30T22:27:42.234355+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:29:14.582525+0000","last_change":"2023-09-30T22:27:42.396108+0000","last_active":"2023-09-30T22:29:14.582525+0000","last_peered":"2023-09-30T22:29:14.582525+0000","last_clean":"2023-09-30T22:29:14.582525+0000","last_became_active":"2023-09-30T22:27:42.395692+0000","last_became_peered":"2023-09-30T22:27:42.395692+0000","last_unstale":"2023-09-30T22:29:14.582525+0000","last_undegraded":"2023-09-30T22:29:14.582525+0000","last_fullsized":"2023-09-30T22:29:14.582525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.237153+0000","last_change":"2023-09-30T22:26:51.281127+0000","last_active":"2023-09-30T22:27:42.237153+0000","last_peered":"2023-09-30T22:27:42.237153+0000","last_clean":"2023-09-30T22:27:42.237153+0000","last_became_active":"2023-09-30T22:26:51.280507+0000","last_became_peered":"2023-09-30T22:26:51.280507+0000","last_unstale":"2023-09-30T22:27:42.237153+0000","last_undegraded":"2023-09-30T22:27:42.237153+0000","last_fullsized":"2023-09-30T22:27:42.237153+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.234226+0000","last_change":"2023-09-30T22:26:51.712671+0000","last_active":"2023-09-30T22:27:42.234226+0000","last_peered":"2023-09-30T22:27:42.234226+0000","last_clean":"2023-09-30T22:27:42.234226+0000","last_became_active":"2023-09-30T22:26:51.711836+0000","last_became_peered":"2023-09-30T22:26:51.711836+0000","last_unstale":"2023-09-30T22:27:42.234226+0000","last_undegraded":"2023-09-30T22:27:42.234226+0000","last_fullsized":"2023-09-30T22:27:42.234226+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.234330+0000","last_change":"2023-09-30T22:26:51.297467+0000","last_active":"2023-09-30T22:27:42.234330+0000","last_peered":"2023-09-30T22:27:42.234330+0000","last_clean":"2023-09-30T22:27:42.234330+0000","last_became_active":"2023-09-30T22:26:51.296950+0000","last_became_peered":"2023-09-30T22:26:51.296950+0000","last_unstale":"2023-09-30T22:27:42.234330+0000","last_undegraded":"2023-09-30T22:27:42.234330+0000","last_fullsized":"2023-09-30T22:27:42.234330+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.239168+0000","last_change":"2023-09-30T22:26:51.719411+0000","last_active":"2023-09-30T22:27:42.239168+0000","last_peered":"2023-09-30T22:27:42.239168+0000","last_clean":"2023-09-30T22:27:42.239168+0000","last_became_active":"2023-09-30T22:26:51.719111+0000","last_became_peered":"2023-09-30T22:26:51.719111+0000","last_unstale":"2023-09-30T22:27:42.239168+0000","last_undegraded":"2023-09-30T22:27:42.239168+0000","last_fullsized":"2023-09-30T22:27:42.239168+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"15'8","reported_seq":22,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.394756+0000","last_change":"2023-09-30T22:26:51.287702+0000","last_active":"2023-09-30T22:27:42.394756+0000","last_peered":"2023-09-30T22:27:42.394756+0000","last_clean":"2023-09-30T22:27:42.394756+0000","last_became_active":"2023-09-30T22:26:51.287434+0000","last_became_peered":"2023-09-30T22:26:51.287434+0000","last_unstale":"2023-09-30T22:27:42.394756+0000","last_undegraded":"2023-09-30T22:27:42.394756+0000","last_fullsized":"2023-09-30T22:27:42.394756+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:29:14.583520+0000","last_change":"2023-09-30T22:27:42.396215+0000","last_active":"2023-09-30T22:29:14.583520+0000","last_peered":"2023-09-30T22:29:14.583520+0000","last_clean":"2023-09-30T22:29:14.583520+0000","last_became_active":"2023-09-30T22:27:42.395849+0000","last_became_peered":"2023-09-30T22:27:42.395849+0000","last_unstale":"2023-09-30T22:29:14.583520+0000","last_undegraded":"2023-09-30T22:29:14.583520+0000","last_fullsized":"2023-09-30T22:29:14.583520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"15'1","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.239086+0000","last_change":"2023-09-30T22:26:51.291996+0000","last_active":"2023-09-30T22:27:42.239086+0000","last_peered":"2023-09-30T22:27:42.239086+0000","last_clean":"2023-09-30T22:27:42.239086+0000","last_became_active":"2023-09-30T22:26:51.291656+0000","last_became_peered":"2023-09-30T22:26:51.291656+0000","last_unstale":"2023-09-30T22:27:42.239086+0000","last_undegraded":"2023-09-30T22:27:42.239086+0000","last_fullsized":"2023-09-30T22:27:42.239086+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:29:14.581708+0000","last_change":"2023-09-30T22:27:42.396720+0000","last_active":"2023-09-30T22:29:14.581708+0000","last_peered":"2023-09-30T22:29:14.581708+0000","last_clean":"2023-09-30T22:29:14.581708+0000","last_became_active":"2023-09-30T22:27:42.395725+0000","last_became_peered":"2023-09-30T22:27:42.395725+0000","last_unstale":"2023-09-30T22:29:14.581708+0000","last_undegraded":"2023-09-30T22:29:14.581708+0000","last_fullsized":"2023-09-30T22:29:14.581708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"15'2","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:29:14.583186+0000","last_change":"2023-09-30T22:26:51.300700+0000","last_active":"2023-09-30T22:29:14.583186+0000","last_peered":"2023-09-30T22:29:14.583186+0000","last_clean":"2023-09-30T22:29:14.583186+0000","last_became_active":"2023-09-30T22:26:51.300416+0000","last_became_peered":"2023-09-30T22:26:51.300416+0000","last_unstale":"2023-09-30T22:29:14.583186+0000","last_undegraded":"2023-09-30T22:29:14.583186+0000","last_fullsized":"2023-09-30T22:29:14.583186+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.394121+0000","last_change":"2023-09-30T22:26:51.722902+0000","last_active":"2023-09-30T22:27:42.394121+0000","last_peered":"2023-09-30T22:27:42.394121+0000","last_clean":"2023-09-30T22:27:42.394121+0000","last_became_active":"2023-09-30T22:26:51.717919+0000","last_became_peered":"2023-09-30T22:26:51.717919+0000","last_unstale":"2023-09-30T22:27:42.394121+0000","last_undegraded":"2023-09-30T22:27:42.394121+0000","last_fullsized":"2023-09-30T22:27:42.394121+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"15'1","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.235059+0000","last_change":"2023-09-30T22:26:51.292120+0000","last_active":"2023-09-30T22:27:42.235059+0000","last_peered":"2023-09-30T22:27:42.235059+0000","last_clean":"2023-09-30T22:27:42.235059+0000","last_became_active":"2023-09-30T22:26:51.289469+0000","last_became_peered":"2023-09-30T22:26:51.289469+0000","last_unstale":"2023-09-30T22:27:42.235059+0000","last_undegraded":"2023-09-30T22:27:42.235059+0000","last_fullsized":"2023-09-30T22:27:42.235059+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:29:14.582253+0000","last_change":"2023-09-30T22:26:51.711649+0000","last_active":"2023-09-30T22:29:14.582253+0000","last_peered":"2023-09-30T22:29:14.582253+0000","last_clean":"2023-09-30T22:29:14.582253+0000","last_became_active":"2023-09-30T22:26:51.711384+0000","last_became_peered":"2023-09-30T22:26:51.711384+0000","last_unstale":"2023-09-30T22:29:14.582253+0000","last_undegraded":"2023-09-30T22:29:14.582253+0000","last_fullsized":"2023-09-30T22:29:14.582253+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.238674+0000","last_change":"2023-09-30T22:26:51.281627+0000","last_active":"2023-09-30T22:27:42.238674+0000","last_peered":"2023-09-30T22:27:42.238674+0000","last_clean":"2023-09-30T22:27:42.238674+0000","last_became_active":"2023-09-30T22:26:51.281311+0000","last_became_peered":"2023-09-30T22:26:51.281311+0000","last_unstale":"2023-09-30T22:27:42.238674+0000","last_undegraded":"2023-09-30T22:27:42.238674+0000","last_fullsized":"2023-09-30T22:27:42.238674+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.238320+0000","last_change":"2023-09-30T22:26:51.700837+0000","last_active":"2023-09-30T22:27:42.238320+0000","last_peered":"2023-09-30T22:27:42.238320+0000","last_clean":"2023-09-30T22:27:42.238320+0000","last_became_active":"2023-09-30T22:26:51.700523+0000","last_became_peered":"2023-09-30T22:26:51.700523+0000","last_unstale":"2023-09-30T22:27:42.238320+0000","last_undegraded":"2023-09-30T22:27:42.238320+0000","last_fullsized":"2023-09-30T22:27:42.238320+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"15'2","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.234152+0000","last_change":"2023-09-30T22:26:51.290248+0000","last_active":"2023-09-30T22:27:42.234152+0000","last_peered":"2023-09-30T22:27:42.234152+0000","last_clean":"2023-09-30T22:27:42.234152+0000","last_became_active":"2023-09-30T22:26:51.289168+0000","last_became_peered":"2023-09-30T22:26:51.289168+0000","last_unstale":"2023-09-30T22:27:42.234152+0000","last_undegraded":"2023-09-30T22:27:42.234152+0000","last_fullsized":"2023-09-30T22:27:42.234152+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.238595+0000","last_change":"2023-09-30T22:26:51.719224+0000","last_active":"2023-09-30T22:27:42.238595+0000","last_peered":"2023-09-30T22:27:42.238595+0000","last_clean":"2023-09-30T22:27:42.238595+0000","last_became_active":"2023-09-30T22:26:51.718890+0000","last_became_peered":"2023-09-30T22:26:51.718890+0000","last_unstale":"2023-09-30T22:27:42.238595+0000","last_undegraded":"2023-09-30T22:27:42.238595+0000","last_fullsized":"2023-09-30T22:27:42.238595+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.235840+0000","last_change":"2023-09-30T22:26:51.288204+0000","last_active":"2023-09-30T22:27:42.235840+0000","last_peered":"2023-09-30T22:27:42.235840+0000","last_clean":"2023-09-30T22:27:42.235840+0000","last_became_active":"2023-09-30T22:26:51.287903+0000","last_became_peered":"2023-09-30T22:26:51.287903+0000","last_unstale":"2023-09-30T22:27:42.235840+0000","last_undegraded":"2023-09-30T22:27:42.235840+0000","last_fullsized":"2023-09-30T22:27:42.235840+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:29:14.582796+0000","last_change":"2023-09-30T22:27:42.396176+0000","last_active":"2023-09-30T22:29:14.582796+0000","last_peered":"2023-09-30T22:29:14.582796+0000","last_clean":"2023-09-30T22:29:14.582796+0000","last_became_active":"2023-09-30T22:27:42.395667+0000","last_became_peered":"2023-09-30T22:27:42.395667+0000","last_unstale":"2023-09-30T22:29:14.582796+0000","last_undegraded":"2023-09-30T22:29:14.582796+0000","last_fullsized":"2023-09-30T22:29:14.582796+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.234082+0000","last_change":"2023-09-30T22:26:51.297240+0000","last_active":"2023-09-30T22:27:42.234082+0000","last_peered":"2023-09-30T22:27:42.234082+0000","last_clean":"2023-09-30T22:27:42.234082+0000","last_became_active":"2023-09-30T22:26:51.296547+0000","last_became_peered":"2023-09-30T22:26:51.296547+0000","last_unstale":"2023-09-30T22:27:42.234082+0000","last_undegraded":"2023-09-30T22:27:42.234082+0000","last_fullsized":"2023-09-30T22:27:42.234082+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:29:14.583275+0000","last_change":"2023-09-30T22:26:51.711889+0000","last_active":"2023-09-30T22:29:14.583275+0000","last_peered":"2023-09-30T22:29:14.583275+0000","last_clean":"2023-09-30T22:29:14.583275+0000","last_became_active":"2023-09-30T22:26:51.711570+0000","last_became_peered":"2023-09-30T22:26:51.711570+0000","last_unstale":"2023-09-30T22:29:14.583275+0000","last_undegraded":"2023-09-30T22:29:14.583275+0000","last_fullsized":"2023-09-30T22:29:14.583275+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.235076+0000","last_change":"2023-09-30T22:26:51.290699+0000","last_active":"2023-09-30T22:27:42.235076+0000","last_peered":"2023-09-30T22:27:42.235076+0000","last_clean":"2023-09-30T22:27:42.235076+0000","last_became_active":"2023-09-30T22:26:51.290396+0000","last_became_peered":"2023-09-30T22:26:51.290396+0000","last_unstale":"2023-09-30T22:27:42.235076+0000","last_undegraded":"2023-09-30T22:27:42.235076+0000","last_fullsized":"2023-09-30T22:27:42.235076+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.241857+0000","last_change":"2023-09-30T22:27:42.240919+0000","last_active":"2023-09-30T22:27:42.241857+0000","last_peered":"2023-09-30T22:27:42.241857+0000","last_clean":"2023-09-30T22:27:42.241857+0000","last_became_active":"2023-09-30T22:27:42.240544+0000","last_became_peered":"2023-09-30T22:27:42.240544+0000","last_unstale":"2023-09-30T22:27:42.241857+0000","last_undegraded":"2023-09-30T22:27:42.241857+0000","last_fullsized":"2023-09-30T22:27:42.241857+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.393691+0000","last_change":"2023-09-30T22:26:46.273891+0000","last_active":"2023-09-30T22:27:42.393691+0000","last_peered":"2023-09-30T22:27:42.393691+0000","last_clean":"2023-09-30T22:27:42.393691+0000","last_became_active":"2023-09-30T22:26:46.273588+0000","last_became_peered":"2023-09-30T22:26:46.273588+0000","last_unstale":"2023-09-30T22:27:42.393691+0000","last_undegraded":"2023-09-30T22:27:42.393691+0000","last_fullsized":"2023-09-30T22:27:42.393691+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:29:14.582902+0000","last_change":"2023-09-30T22:26:51.294682+0000","last_active":"2023-09-30T22:29:14.582902+0000","last_peered":"2023-09-30T22:29:14.582902+0000","last_clean":"2023-09-30T22:29:14.582902+0000","last_became_active":"2023-09-30T22:26:51.294422+0000","last_became_peered":"2023-09-30T22:26:51.294422+0000","last_unstale":"2023-09-30T22:29:14.582902+0000","last_undegraded":"2023-09-30T22:29:14.582902+0000","last_fullsized":"2023-09-30T22:29:14.582902+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.394478+0000","last_change":"2023-09-30T22:26:51.726954+0000","last_active":"2023-09-30T22:27:42.394478+0000","last_peered":"2023-09-30T22:27:42.394478+0000","last_clean":"2023-09-30T22:27:42.394478+0000","last_became_active":"2023-09-30T22:26:51.726641+0000","last_became_peered":"2023-09-30T22:26:51.726641+0000","last_unstale":"2023-09-30T22:27:42.394478+0000","last_undegraded":"2023-09-30T22:27:42.394478+0000","last_fullsized":"2023-09-30T22:27:42.394478+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.235981+0000","last_change":"2023-09-30T22:26:46.245946+0000","last_active":"2023-09-30T22:27:42.235981+0000","last_peered":"2023-09-30T22:27:42.235981+0000","last_clean":"2023-09-30T22:27:42.235981+0000","last_became_active":"2023-09-30T22:26:46.245602+0000","last_became_peered":"2023-09-30T22:26:46.245602+0000","last_unstale":"2023-09-30T22:27:42.235981+0000","last_undegraded":"2023-09-30T22:27:42.235981+0000","last_fullsized":"2023-09-30T22:27:42.235981+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.238085+0000","last_change":"2023-09-30T22:26:51.294691+0000","last_active":"2023-09-30T22:27:42.238085+0000","last_peered":"2023-09-30T22:27:42.238085+0000","last_clean":"2023-09-30T22:27:42.238085+0000","last_became_active":"2023-09-30T22:26:51.294411+0000","last_became_peered":"2023-09-30T22:26:51.294411+0000","last_unstale":"2023-09-30T22:27:42.238085+0000","last_undegraded":"2023-09-30T22:27:42.238085+0000","last_fullsized":"2023-09-30T22:27:42.238085+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.394893+0000","last_change":"2023-09-30T22:26:51.724759+0000","last_active":"2023-09-30T22:27:42.394893+0000","last_peered":"2023-09-30T22:27:42.394893+0000","last_clean":"2023-09-30T22:27:42.394893+0000","last_became_active":"2023-09-30T22:26:51.723221+0000","last_became_peered":"2023-09-30T22:26:51.723221+0000","last_unstale":"2023-09-30T22:27:42.394893+0000","last_undegraded":"2023-09-30T22:27:42.394893+0000","last_fullsized":"2023-09-30T22:27:42.394893+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:29:14.582369+0000","last_change":"2023-09-30T22:26:46.242607+0000","last_active":"2023-09-30T22:29:14.582369+0000","last_peered":"2023-09-30T22:29:14.582369+0000","last_clean":"2023-09-30T22:29:14.582369+0000","last_became_active":"2023-09-30T22:26:46.242283+0000","last_became_peered":"2023-09-30T22:26:46.242283+0000","last_unstale":"2023-09-30T22:29:14.582369+0000","last_undegraded":"2023-09-30T22:29:14.582369+0000","last_fullsized":"2023-09-30T22:29:14.582369+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.394654+0000","last_change":"2023-09-30T22:26:51.295409+0000","last_active":"2023-09-30T22:27:42.394654+0000","last_peered":"2023-09-30T22:27:42.394654+0000","last_clean":"2023-09-30T22:27:42.394654+0000","last_became_active":"2023-09-30T22:26:51.295140+0000","last_became_peered":"2023-09-30T22:26:51.295140+0000","last_unstale":"2023-09-30T22:27:42.394654+0000","last_undegraded":"2023-09-30T22:27:42.394654+0000","last_fullsized":"2023-09-30T22:27:42.394654+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.395245+0000","last_change":"2023-09-30T22:26:51.715504+0000","last_active":"2023-09-30T22:27:42.395245+0000","last_peered":"2023-09-30T22:27:42.395245+0000","last_clean":"2023-09-30T22:27:42.395245+0000","last_became_active":"2023-09-30T22:26:51.715258+0000","last_became_peered":"2023-09-30T22:26:51.715258+0000","last_unstale":"2023-09-30T22:27:42.395245+0000","last_undegraded":"2023-09-30T22:27:42.395245+0000","last_fullsized":"2023-09-30T22:27:42.395245+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.234249+0000","last_change":"2023-09-30T22:26:46.242752+0000","last_active":"2023-09-30T22:27:42.234249+0000","last_peered":"2023-09-30T22:27:42.234249+0000","last_clean":"2023-09-30T22:27:42.234249+0000","last_became_active":"2023-09-30T22:26:46.242385+0000","last_became_peered":"2023-09-30T22:26:46.242385+0000","last_unstale":"2023-09-30T22:27:42.234249+0000","last_undegraded":"2023-09-30T22:27:42.234249+0000","last_fullsized":"2023-09-30T22:27:42.234249+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.393778+0000","last_change":"2023-09-30T22:26:46.273930+0000","last_active":"2023-09-30T22:27:42.393778+0000","last_peered":"2023-09-30T22:27:42.393778+0000","last_clean":"2023-09-30T22:27:42.393778+0000","last_became_active":"2023-09-30T22:26:46.273621+0000","last_became_peered":"2023-09-30T22:26:46.273621+0000","last_unstale":"2023-09-30T22:27:42.393778+0000","last_undegraded":"2023-09-30T22:27:42.393778+0000","last_fullsized":"2023-09-30T22:27:42.393778+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.235754+0000","last_change":"2023-09-30T22:26:51.298163+0000","last_active":"2023-09-30T22:27:42.235754+0000","last_peered":"2023-09-30T22:27:42.235754+0000","last_clean":"2023-09-30T22:27:42.235754+0000","last_became_active":"2023-09-30T22:26:51.297897+0000","last_became_peered":"2023-09-30T22:26:51.297897+0000","last_unstale":"2023-09-30T22:27:42.235754+0000","last_undegraded":"2023-09-30T22:27:42.235754+0000","last_fullsized":"2023-09-30T22:27:42.235754+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.234722+0000","last_change":"2023-09-30T22:26:51.713608+0000","last_active":"2023-09-30T22:27:42.234722+0000","last_peered":"2023-09-30T22:27:42.234722+0000","last_clean":"2023-09-30T22:27:42.234722+0000","last_became_active":"2023-09-30T22:26:51.712992+0000","last_became_peered":"2023-09-30T22:26:51.712992+0000","last_unstale":"2023-09-30T22:27:42.234722+0000","last_undegraded":"2023-09-30T22:27:42.234722+0000","last_fullsized":"2023-09-30T22:27:42.234722+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.235158+0000","last_change":"2023-09-30T22:26:46.245668+0000","last_active":"2023-09-30T22:27:42.235158+0000","last_peered":"2023-09-30T22:27:42.235158+0000","last_clean":"2023-09-30T22:27:42.235158+0000","last_became_active":"2023-09-30T22:26:46.245319+0000","last_became_peered":"2023-09-30T22:26:46.245319+0000","last_unstale":"2023-09-30T22:27:42.235158+0000","last_undegraded":"2023-09-30T22:27:42.235158+0000","last_fullsized":"2023-09-30T22:27:42.235158+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.237312+0000","last_change":"2023-09-30T22:26:51.294664+0000","last_active":"2023-09-30T22:27:42.237312+0000","last_peered":"2023-09-30T22:27:42.237312+0000","last_clean":"2023-09-30T22:27:42.237312+0000","last_became_active":"2023-09-30T22:26:51.294357+0000","last_became_peered":"2023-09-30T22:26:51.294357+0000","last_unstale":"2023-09-30T22:27:42.237312+0000","last_undegraded":"2023-09-30T22:27:42.237312+0000","last_fullsized":"2023-09-30T22:27:42.237312+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.239247+0000","last_change":"2023-09-30T22:26:51.705784+0000","last_active":"2023-09-30T22:27:42.239247+0000","last_peered":"2023-09-30T22:27:42.239247+0000","last_clean":"2023-09-30T22:27:42.239247+0000","last_became_active":"2023-09-30T22:26:51.705480+0000","last_became_peered":"2023-09-30T22:26:51.705480+0000","last_unstale":"2023-09-30T22:27:42.239247+0000","last_undegraded":"2023-09-30T22:27:42.239247+0000","last_fullsized":"2023-09-30T22:27:42.239247+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.234542+0000","last_change":"2023-09-30T22:26:46.274062+0000","last_active":"2023-09-30T22:27:42.234542+0000","last_peered":"2023-09-30T22:27:42.234542+0000","last_clean":"2023-09-30T22:27:42.234542+0000","last_became_active":"2023-09-30T22:26:46.273752+0000","last_became_peered":"2023-09-30T22:26:46.273752+0000","last_unstale":"2023-09-30T22:27:42.234542+0000","last_undegraded":"2023-09-30T22:27:42.234542+0000","last_fullsized":"2023-09-30T22:27:42.234542+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.394316+0000","last_change":"2023-09-30T22:26:51.295181+0000","last_active":"2023-09-30T22:27:42.394316+0000","last_peered":"2023-09-30T22:27:42.394316+0000","last_clean":"2023-09-30T22:27:42.394316+0000","last_became_active":"2023-09-30T22:26:51.294889+0000","last_became_peered":"2023-09-30T22:26:51.294889+0000","last_unstale":"2023-09-30T22:27:42.394316+0000","last_undegraded":"2023-09-30T22:27:42.394316+0000","last_fullsized":"2023-09-30T22:27:42.394316+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.238245+0000","last_change":"2023-09-30T22:26:51.690467+0000","last_active":"2023-09-30T22:27:42.238245+0000","last_peered":"2023-09-30T22:27:42.238245+0000","last_clean":"2023-09-30T22:27:42.238245+0000","last_became_active":"2023-09-30T22:26:51.690163+0000","last_became_peered":"2023-09-30T22:26:51.690163+0000","last_unstale":"2023-09-30T22:27:42.238245+0000","last_undegraded":"2023-09-30T22:27:42.238245+0000","last_fullsized":"2023-09-30T22:27:42.238245+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"15'2","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.235289+0000","last_change":"2023-09-30T22:26:51.290268+0000","last_active":"2023-09-30T22:27:42.235289+0000","last_peered":"2023-09-30T22:27:42.235289+0000","last_clean":"2023-09-30T22:27:42.235289+0000","last_became_active":"2023-09-30T22:26:51.289409+0000","last_became_peered":"2023-09-30T22:26:51.289409+0000","last_unstale":"2023-09-30T22:27:42.235289+0000","last_undegraded":"2023-09-30T22:27:42.235289+0000","last_fullsized":"2023-09-30T22:27:42.235289+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.393716+0000","last_change":"2023-09-30T22:26:46.273888+0000","last_active":"2023-09-30T22:27:42.393716+0000","last_peered":"2023-09-30T22:27:42.393716+0000","last_clean":"2023-09-30T22:27:42.393716+0000","last_became_active":"2023-09-30T22:26:46.273549+0000","last_became_peered":"2023-09-30T22:26:46.273549+0000","last_unstale":"2023-09-30T22:27:42.393716+0000","last_undegraded":"2023-09-30T22:27:42.393716+0000","last_fullsized":"2023-09-30T22:27:42.393716+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:29:14.581810+0000","last_change":"2023-09-30T22:27:42.396210+0000","last_active":"2023-09-30T22:29:14.581810+0000","last_peered":"2023-09-30T22:29:14.581810+0000","last_clean":"2023-09-30T22:29:14.581810+0000","last_became_active":"2023-09-30T22:27:42.395859+0000","last_became_peered":"2023-09-30T22:27:42.395859+0000","last_unstale":"2023-09-30T22:29:14.581810+0000","last_undegraded":"2023-09-30T22:29:14.581810+0000","last_fullsized":"2023-09-30T22:29:14.581810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.235361+0000","last_change":"2023-09-30T22:26:46.274508+0000","last_active":"2023-09-30T22:27:42.235361+0000","last_peered":"2023-09-30T22:27:42.235361+0000","last_clean":"2023-09-30T22:27:42.235361+0000","last_became_active":"2023-09-30T22:26:46.273567+0000","last_became_peered":"2023-09-30T22:26:46.273567+0000","last_unstale":"2023-09-30T22:27:42.235361+0000","last_undegraded":"2023-09-30T22:27:42.235361+0000","last_fullsized":"2023-09-30T22:27:42.235361+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"16'1","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.239058+0000","last_change":"2023-09-30T22:26:51.288625+0000","last_active":"2023-09-30T22:27:42.239058+0000","last_peered":"2023-09-30T22:27:42.239058+0000","last_clean":"2023-09-30T22:27:42.239058+0000","last_became_active":"2023-09-30T22:26:51.288286+0000","last_became_peered":"2023-09-30T22:26:51.288286+0000","last_unstale":"2023-09-30T22:27:42.239058+0000","last_undegraded":"2023-09-30T22:27:42.239058+0000","last_fullsized":"2023-09-30T22:27:42.239058+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:29:14.582221+0000","last_change":"2023-09-30T22:26:51.708965+0000","last_active":"2023-09-30T22:29:14.582221+0000","last_peered":"2023-09-30T22:29:14.582221+0000","last_clean":"2023-09-30T22:29:14.582221+0000","last_became_active":"2023-09-30T22:26:51.708607+0000","last_became_peered":"2023-09-30T22:26:51.708607+0000","last_unstale":"2023-09-30T22:29:14.582221+0000","last_undegraded":"2023-09-30T22:29:14.582221+0000","last_fullsized":"2023-09-30T22:29:14.582221+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"15'1","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.238166+0000","last_change":"2023-09-30T22:26:51.282125+0000","last_active":"2023-09-30T22:27:42.238166+0000","last_peered":"2023-09-30T22:27:42.238166+0000","last_clean":"2023-09-30T22:27:42.238166+0000","last_became_active":"2023-09-30T22:26:51.281780+0000","last_became_peered":"2023-09-30T22:26:51.281780+0000","last_unstale":"2023-09-30T22:27:42.238166+0000","last_undegraded":"2023-09-30T22:27:42.238166+0000","last_fullsized":"2023-09-30T22:27:42.238166+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:29:14.582806+0000","last_change":"2023-09-30T22:26:51.703281+0000","last_active":"2023-09-30T22:29:14.582806+0000","last_peered":"2023-09-30T22:29:14.582806+0000","last_clean":"2023-09-30T22:29:14.582806+0000","last_became_active":"2023-09-30T22:26:51.703007+0000","last_became_peered":"2023-09-30T22:26:51.703007+0000","last_unstale":"2023-09-30T22:29:14.582806+0000","last_undegraded":"2023-09-30T22:29:14.582806+0000","last_fullsized":"2023-09-30T22:29:14.582806+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"15'2","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.394623+0000","last_change":"2023-09-30T22:26:51.287630+0000","last_active":"2023-09-30T22:27:42.394623+0000","last_peered":"2023-09-30T22:27:42.394623+0000","last_clean":"2023-09-30T22:27:42.394623+0000","last_became_active":"2023-09-30T22:26:51.287337+0000","last_became_peered":"2023-09-30T22:26:51.287337+0000","last_unstale":"2023-09-30T22:27:42.394623+0000","last_undegraded":"2023-09-30T22:27:42.394623+0000","last_fullsized":"2023-09-30T22:27:42.394623+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:29:14.582509+0000","last_change":"2023-09-30T22:26:51.716458+0000","last_active":"2023-09-30T22:29:14.582509+0000","last_peered":"2023-09-30T22:29:14.582509+0000","last_clean":"2023-09-30T22:29:14.582509+0000","last_became_active":"2023-09-30T22:26:51.715957+0000","last_became_peered":"2023-09-30T22:26:51.715957+0000","last_unstale":"2023-09-30T22:29:14.582509+0000","last_undegraded":"2023-09-30T22:29:14.582509+0000","last_fullsized":"2023-09-30T22:29:14.582509+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"15'2","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.234639+0000","last_change":"2023-09-30T22:26:51.285956+0000","last_active":"2023-09-30T22:27:42.234639+0000","last_peered":"2023-09-30T22:27:42.234639+0000","last_clean":"2023-09-30T22:27:42.234639+0000","last_became_active":"2023-09-30T22:26:51.285690+0000","last_became_peered":"2023-09-30T22:26:51.285690+0000","last_unstale":"2023-09-30T22:27:42.234639+0000","last_undegraded":"2023-09-30T22:27:42.234639+0000","last_fullsized":"2023-09-30T22:27:42.234639+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:29:14.582154+0000","last_change":"2023-09-30T22:26:51.716290+0000","last_active":"2023-09-30T22:29:14.582154+0000","last_peered":"2023-09-30T22:29:14.582154+0000","last_clean":"2023-09-30T22:29:14.582154+0000","last_became_active":"2023-09-30T22:26:51.716007+0000","last_became_peered":"2023-09-30T22:26:51.716007+0000","last_unstale":"2023-09-30T22:29:14.582154+0000","last_undegraded":"2023-09-30T22:29:14.582154+0000","last_fullsized":"2023-09-30T22:29:14.582154+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1e","version":"15'1","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.234865+0000","last_change":"2023-09-30T22:26:51.296949+0000","last_active":"2023-09-30T22:27:42.234865+0000","last_peered":"2023-09-30T22:27:42.234865+0000","last_clean":"2023-09-30T22:27:42.234865+0000","last_became_active":"2023-09-30T22:26:51.296602+0000","last_became_peered":"2023-09-30T22:26:51.296602+0000","last_unstale":"2023-09-30T22:27:42.234865+0000","last_undegraded":"2023-09-30T22:27:42.234865+0000","last_fullsized":"2023-09-30T22:27:42.234865+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.241844+0000","last_change":"2023-09-30T22:27:42.240569+0000","last_active":"2023-09-30T22:27:42.241844+0000","last_peered":"2023-09-30T22:27:42.241844+0000","last_clean":"2023-09-30T22:27:42.241844+0000","last_became_active":"2023-09-30T22:27:42.240222+0000","last_became_peered":"2023-09-30T22:27:42.240222+0000","last_unstale":"2023-09-30T22:27:42.241844+0000","last_undegraded":"2023-09-30T22:27:42.241844+0000","last_fullsized":"2023-09-30T22:27:42.241844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"15'4","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.239340+0000","last_change":"2023-09-30T22:26:51.291976+0000","last_active":"2023-09-30T22:27:42.239340+0000","last_peered":"2023-09-30T22:27:42.239340+0000","last_clean":"2023-09-30T22:27:42.239340+0000","last_became_active":"2023-09-30T22:26:51.291255+0000","last_became_peered":"2023-09-30T22:26:51.291255+0000","last_unstale":"2023-09-30T22:27:42.239340+0000","last_undegraded":"2023-09-30T22:27:42.239340+0000","last_fullsized":"2023-09-30T22:27:42.239340+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:29:14.581987+0000","last_change":"2023-09-30T22:26:51.713485+0000","last_active":"2023-09-30T22:29:14.581987+0000","last_peered":"2023-09-30T22:29:14.581987+0000","last_clean":"2023-09-30T22:29:14.581987+0000","last_became_active":"2023-09-30T22:26:51.713188+0000","last_became_peered":"2023-09-30T22:26:51.713188+0000","last_unstale":"2023-09-30T22:29:14.581987+0000","last_undegraded":"2023-09-30T22:29:14.581987+0000","last_fullsized":"2023-09-30T22:29:14.581987+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.394694+0000","last_change":"2023-09-30T22:26:51.704082+0000","last_active":"2023-09-30T22:27:42.394694+0000","last_peered":"2023-09-30T22:27:42.394694+0000","last_clean":"2023-09-30T22:27:42.394694+0000","last_became_active":"2023-09-30T22:26:51.703280+0000","last_became_peered":"2023-09-30T22:26:51.703280+0000","last_unstale":"2023-09-30T22:27:42.394694+0000","last_undegraded":"2023-09-30T22:27:42.394694+0000","last_fullsized":"2023-09-30T22:27:42.394694+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.234177+0000","last_change":"2023-09-30T22:26:51.705877+0000","last_active":"2023-09-30T22:27:42.234177+0000","last_peered":"2023-09-30T22:27:42.234177+0000","last_clean":"2023-09-30T22:27:42.234177+0000","last_became_active":"2023-09-30T22:26:51.705352+0000","last_became_peered":"2023-09-30T22:26:51.705352+0000","last_unstale":"2023-09-30T22:27:42.234177+0000","last_undegraded":"2023-09-30T22:27:42.234177+0000","last_fullsized":"2023-09-30T22:27:42.234177+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.237225+0000","last_change":"2023-09-30T22:26:51.712201+0000","last_active":"2023-09-30T22:27:42.237225+0000","last_peered":"2023-09-30T22:27:42.237225+0000","last_clean":"2023-09-30T22:27:42.237225+0000","last_became_active":"2023-09-30T22:26:51.709083+0000","last_became_peered":"2023-09-30T22:26:51.709083+0000","last_unstale":"2023-09-30T22:27:42.237225+0000","last_undegraded":"2023-09-30T22:27:42.237225+0000","last_fullsized":"2023-09-30T22:27:42.237225+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.235010+0000","last_change":"2023-09-30T22:26:51.706543+0000","last_active":"2023-09-30T22:27:42.235010+0000","last_peered":"2023-09-30T22:27:42.235010+0000","last_clean":"2023-09-30T22:27:42.235010+0000","last_became_active":"2023-09-30T22:26:51.706251+0000","last_became_peered":"2023-09-30T22:26:51.706251+0000","last_unstale":"2023-09-30T22:27:42.235010+0000","last_undegraded":"2023-09-30T22:27:42.235010+0000","last_fullsized":"2023-09-30T22:27:42.235010+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.394716+0000","last_change":"2023-09-30T22:26:51.724591+0000","last_active":"2023-09-30T22:27:42.394716+0000","last_peered":"2023-09-30T22:27:42.394716+0000","last_clean":"2023-09-30T22:27:42.394716+0000","last_became_active":"2023-09-30T22:26:51.722847+0000","last_became_peered":"2023-09-30T22:26:51.722847+0000","last_unstale":"2023-09-30T22:27:42.394716+0000","last_undegraded":"2023-09-30T22:27:42.394716+0000","last_fullsized":"2023-09-30T22:27:42.394716+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4333,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":51,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":45,"ondisk_log_size":45,"up":64,"acting":64,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705696,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297284,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074556,"statfs":{"total":96636764160,"available":96332345344,"internally_reserved":0,"allocated":192512,"data_stored":64640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 22:28:48 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.35799999999999998,"15min":0.36199999999999999},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.46999999999999997,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.307},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.44900000000000001,"15min":0.46700000000000003},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.51500000000000001,"5min":2.5600000000000001,"15min":2.5600000000000001},"last":0.36699999999999999}]},{"osd":1,"last update":"Sat Sep 30 22:28:48 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.45700000000000002,"15min":0.47299999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.5,"5min":2.6040000000000001,"15min":2.6040000000000001},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.375,"15min":0.38},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.57299999999999995,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.38400000000000001}]},{"osd":2,"last update":"Sat Sep 30 22:28:48 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.40000000000000002,"15min":0.40600000000000003},"min":{"1min":0.27400000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.54400000000000004,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.35199999999999998}]}]},{"osd":2,"up_from":9,"seq":38654705697,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297272,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074568,"statfs":{"total":96636764160,"available":96332357632,"internally_reserved":0,"allocated":180224,"data_stored":64446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 22:28:47 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.32900000000000001,"15min":0.33000000000000002},"min":{"1min":0.26700000000000002,"5min":0.223,"15min":0.223},"max":{"1min":0.40600000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"last":0.307},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.35699999999999998,"15min":0.35899999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.49199999999999999,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.36899999999999999}]},{"osd":1,"last update":"Sat Sep 30 22:28:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.35699999999999998,"15min":0.35799999999999998},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.46300000000000002,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.35299999999999998,"15min":0.35699999999999998},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.38200000000000001,"5min":0.52100000000000002,"15min":0.52100000000000002},"last":0.32700000000000001}]},{"osd":3,"last update":"Sat Sep 30 22:28:47 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.39000000000000001,"15min":0.39300000000000002},"min":{"1min":0.309,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.45900000000000002,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.374,"5min":0.39200000000000002,"15min":0.39500000000000002},"min":{"1min":0.316,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.433,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.378}]}]},{"osd":0,"up_from":9,"seq":38654705696,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297268,"kb_used_data":172,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074572,"statfs":{"total":96636764160,"available":96332361728,"internally_reserved":0,"allocated":176128,"data_stored":56402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 30 22:28:50 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.30499999999999999,"15min":0.30099999999999999},"min":{"1min":0.27900000000000003,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.45600000000000002,"5min":0.51200000000000001,"15min":0.51200000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.33600000000000002,"15min":0.33300000000000002},"min":{"1min":0.26100000000000001,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"last":0.41699999999999998}]},{"osd":2,"last update":"Sat Sep 30 22:28:50 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.35399999999999998,"15min":0.35199999999999998},"min":{"1min":0.28000000000000003,"5min":0.25,"15min":0.25},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.34300000000000003,"15min":0.34100000000000003},"min":{"1min":0.28799999999999998,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.51300000000000001,"5min":0.53800000000000003,"15min":0.53800000000000003},"last":0.58599999999999997}]},{"osd":3,"last update":"Sat Sep 30 22:28:50 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.36399999999999999,"15min":0.36299999999999999},"min":{"1min":0.27400000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.626},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.50900000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"last":0.33200000000000002}]}]},{"osd":1,"up_from":9,"seq":38654705696,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297268,"kb_used_data":172,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074572,"statfs":{"total":96636764160,"available":96332361728,"internally_reserved":0,"allocated":176128,"data_stored":57518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 22:28:45 2023","interfaces":[{"interface":"back","average":{"1min":0.29299999999999998,"5min":0.309,"15min":0.312},"min":{"1min":0.17000000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.374,"5min":0.40699999999999997,"15min":0.40699999999999997},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.32000000000000001,"5min":0.33900000000000002,"15min":0.34200000000000003},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.40000000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"last":0.59299999999999997}]},{"osd":2,"last update":"Sat Sep 30 22:28:45 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.34599999999999997,"15min":0.34899999999999998},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.40500000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.41999999999999998,"5min":0.442,"15min":0.442},"last":0.57099999999999995}]},{"osd":3,"last update":"Sat Sep 30 22:28:45 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.38300000000000001,"15min":0.38700000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.45400000000000001,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.60699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-30T22:29:18.169 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-30T22:29:18.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.340+0000 7f53de8c5700 1 Processor -- start 2023-09-30T22:29:18.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.341+0000 7f53de8c5700 1 -- start start 2023-09-30T22:29:18.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.341+0000 7f53de8c5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f53d809d6f0 0x7f53d809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:18.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.341+0000 7f53de8c5700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f53d809e000 0x7f53d8130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:18.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.341+0000 7f53de8c5700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53d809f130 con 0x7f53d809e410 2023-09-30T22:29:18.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.341+0000 7f53de8c5700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53d809f520 con 0x7f53d809e000 2023-09-30T22:29:18.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.341+0000 7f53de8c5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53d809e930 con 0x7f53d809d6f0 2023-09-30T22:29:18.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.341+0000 7f53d77fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f53d809e000 0x7f53d8130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:18.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.341+0000 7f53d7fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f53d809d6f0 0x7f53d809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:18.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.341+0000 7f53dce62700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f53d809e410 0x7f53d8130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33772/0 (socket says 172.21.15.16:33772) 2023-09-30T22:29:18.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.341+0000 7f53dce62700 1 -- 172.21.15.16:0/873959899 learned_addr learned my addr 172.21.15.16:0/873959899 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:29:18.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.342+0000 7f53d7fff700 1 -- 172.21.15.16:0/873959899 >> v2:172.21.15.16:3300/0 conn(0x7f53d809e000 msgr2=0x7f53d8130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:18.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.342+0000 7f53d7fff700 1 --2- 172.21.15.16:0/873959899 >> v2:172.21.15.16:3300/0 conn(0x7f53d809e000 0x7f53d8130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:18.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.342+0000 7f53d7fff700 1 -- 172.21.15.16:0/873959899 >> v1:172.21.15.16:6789/0 conn(0x7f53d809e410 legacy=0x7f53d8130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:18.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.342+0000 7f53d7fff700 1 -- 172.21.15.16:0/873959899 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d809b2d0 con 0x7f53d809d6f0 2023-09-30T22:29:18.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.342+0000 7f53d77fe700 1 --2- 172.21.15.16:0/873959899 >> v2:172.21.15.16:3300/0 conn(0x7f53d809e000 0x7f53d8130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:18.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.342+0000 7f53d7fff700 1 --2- 172.21.15.16:0/873959899 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f53d809d6f0 0x7f53d809dac0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f53c8009840 tx=0x7f53c8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a9e18b55858c464 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:18.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.342+0000 7f53d6ffd700 1 -- 172.21.15.16:0/873959899 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c8011830 con 0x7f53d809d6f0 2023-09-30T22:29:18.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.342+0000 7f53d6ffd700 1 -- 172.21.15.16:0/873959899 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c8011990 con 0x7f53d809d6f0 2023-09-30T22:29:18.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.342+0000 7f53d6ffd700 1 -- 172.21.15.16:0/873959899 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c8011d20 con 0x7f53d809d6f0 2023-09-30T22:29:18.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.343+0000 7f53de8c5700 1 -- 172.21.15.16:0/873959899 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f53d809d6f0 msgr2=0x7f53d809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:18.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.343+0000 7f53de8c5700 1 --2- 172.21.15.16:0/873959899 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f53d809d6f0 0x7f53d809dac0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f53c8009840 tx=0x7f53c8009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:18.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.343+0000 7f53de8c5700 1 -- 172.21.15.16:0/873959899 shutdown_connections 2023-09-30T22:29:18.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.343+0000 7f53de8c5700 1 --2- 172.21.15.16:0/873959899 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f53d809d6f0 0x7f53d809dac0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:18.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.343+0000 7f53de8c5700 1 --2- 172.21.15.16:0/873959899 >> v2:172.21.15.16:3300/0 conn(0x7f53d809e000 0x7f53d8130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:18.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.343+0000 7f53de8c5700 1 -- 172.21.15.16:0/873959899 >> 172.21.15.16:0/873959899 conn(0x7f53d8066f50 msgr2=0x7f53d809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:18.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.343+0000 7f53de8c5700 1 -- 172.21.15.16:0/873959899 shutdown_connections 2023-09-30T22:29:18.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.343+0000 7f53de8c5700 1 -- 172.21.15.16:0/873959899 wait complete. 2023-09-30T22:29:18.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.344+0000 7f53de8c5700 1 Processor -- start 2023-09-30T22:29:18.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.344+0000 7f53de8c5700 1 -- start start 2023-09-30T22:29:18.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.344+0000 7f53de8c5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f53d809d6f0 0x7f53d811e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:18.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.344+0000 7f53de8c5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f53d809e000 0x7f53d811e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:18.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.344+0000 7f53de8c5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53d809e410 0x7f53d811eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:18.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.344+0000 7f53de8c5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d811f4d0 con 0x7f53d809e410 2023-09-30T22:29:18.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.344+0000 7f53de8c5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53d8121af0 con 0x7f53d809d6f0 2023-09-30T22:29:18.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.344+0000 7f53de8c5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f53d8120200 con 0x7f53d809e000 2023-09-30T22:29:18.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.344+0000 7f53d7fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f53d809d6f0 0x7f53d811e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:18.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.344+0000 7f53d7fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f53d809d6f0 0x7f53d811e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37062/0 (socket says 172.21.15.16:37062) 2023-09-30T22:29:18.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.344+0000 7f53d77fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f53d809e000 0x7f53d811e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:18.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.344+0000 7f53d7fff700 1 -- 172.21.15.16:0/3298109182 learned_addr learned my addr 172.21.15.16:0/3298109182 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:18.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.344+0000 7f53dce62700 1 --2- 172.21.15.16:0/3298109182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53d809e410 0x7f53d811eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:18.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.345+0000 7f53d7fff700 1 -- 172.21.15.16:0/3298109182 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f53d809e000 msgr2=0x7f53d811e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:18.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.345+0000 7f53d7fff700 1 --2- 172.21.15.16:0/3298109182 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f53d809e000 0x7f53d811e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:18.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.345+0000 7f53d7fff700 1 -- 172.21.15.16:0/3298109182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53d809e410 msgr2=0x7f53d811eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:18.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.345+0000 7f53d7fff700 1 --2- 172.21.15.16:0/3298109182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53d809e410 0x7f53d811eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:18.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.345+0000 7f53d7fff700 1 -- 172.21.15.16:0/3298109182 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53cc00b740 con 0x7f53d809d6f0 2023-09-30T22:29:18.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.345+0000 7f53d77fe700 1 --2- 172.21.15.16:0/3298109182 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f53d809e000 0x7f53d811e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:18.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.345+0000 7f53dce62700 1 --2- 172.21.15.16:0/3298109182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53d809e410 0x7f53d811eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:18.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.345+0000 7f53d7fff700 1 --2- 172.21.15.16:0/3298109182 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f53d809d6f0 0x7f53d811e020 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f53c8009fd0 tx=0x7f53c8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:18.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.345+0000 7f53d4ff9700 1 -- 172.21.15.16:0/3298109182 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c80036a0 con 0x7f53d809d6f0 2023-09-30T22:29:18.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.345+0000 7f53d4ff9700 1 -- 172.21.15.16:0/3298109182 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c801e070 con 0x7f53d809d6f0 2023-09-30T22:29:18.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.345+0000 7f53de8c5700 1 -- 172.21.15.16:0/3298109182 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53c80094a0 con 0x7f53d809d6f0 2023-09-30T22:29:18.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.345+0000 7f53d4ff9700 1 -- 172.21.15.16:0/3298109182 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c8003a40 con 0x7f53d809d6f0 2023-09-30T22:29:18.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.345+0000 7f53de8c5700 1 -- 172.21.15.16:0/3298109182 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53d80a1d90 con 0x7f53d809d6f0 2023-09-30T22:29:18.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.346+0000 7f53d4ff9700 1 -- 172.21.15.16:0/3298109182 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 8) v1 ==== 35553+0+0 (secure 0 0 0) 0x7f53c8003ba0 con 0x7f53d809d6f0 2023-09-30T22:29:18.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.346+0000 7f53d4ff9700 1 -- 172.21.15.16:0/3298109182 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4049+0+0 (secure 0 0 0) 0x7f53c8040930 con 0x7f53d809d6f0 2023-09-30T22:29:18.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.346+0000 7f53de8c5700 1 -- 172.21.15.16:0/3298109182 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53d809c700 con 0x7f53d809d6f0 2023-09-30T22:29:18.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.352+0000 7f53d4ff9700 1 -- 172.21.15.16:0/3298109182 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f53c8015070 con 0x7f53d809d6f0 2023-09-30T22:29:18.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.553+0000 7f53de8c5700 1 -- 172.21.15.16:0/3298109182 --> v1:172.21.15.16:6816/126811 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f53d809c000 con 0x7f53c402f480 2023-09-30T22:29:18.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.561+0000 7f53d4ff9700 1 -- 172.21.15.16:0/3298109182 <== mgr.14103 v1:172.21.15.16:6816/126811 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+270764 (unknown 2965378022 0 1624831560) 0x7f53cc011450 con 0x7f53c402f480 2023-09-30T22:29:18.565 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:29:18.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.564+0000 7f53de8c5700 1 -- 172.21.15.16:0/3298109182 >> v1:172.21.15.16:6816/126811 conn(0x7f53c402f480 legacy=0x7f53c4031930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:18.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.564+0000 7f53de8c5700 1 -- 172.21.15.16:0/3298109182 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f53d809d6f0 msgr2=0x7f53d811e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:18.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.564+0000 7f53de8c5700 1 --2- 172.21.15.16:0/3298109182 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f53d809d6f0 0x7f53d811e020 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f53c8009fd0 tx=0x7f53c8004750 comp rx=0 tx=0).stop 2023-09-30T22:29:18.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.564+0000 7f53de8c5700 1 -- 172.21.15.16:0/3298109182 shutdown_connections 2023-09-30T22:29:18.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.564+0000 7f53de8c5700 1 --2- 172.21.15.16:0/3298109182 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f53d809d6f0 0x7f53d811e020 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:18.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.564+0000 7f53de8c5700 1 --2- 172.21.15.16:0/3298109182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53d809e410 0x7f53d811eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:18.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.564+0000 7f53de8c5700 1 --2- 172.21.15.16:0/3298109182 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f53d809e000 0x7f53d811e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:18.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.564+0000 7f53de8c5700 1 -- 172.21.15.16:0/3298109182 >> 172.21.15.16:0/3298109182 conn(0x7f53d8066f50 msgr2=0x7f53d80710a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:18.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.564+0000 7f53de8c5700 1 -- 172.21.15.16:0/3298109182 shutdown_connections 2023-09-30T22:29:18.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.564+0000 7f53de8c5700 1 -- 172.21.15.16:0/3298109182 wait complete. 2023-09-30T22:29:18.568 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-09-30T22:29:18.578 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":7,"stamp":"2023-09-30T22:29:18.495790+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4333,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":51,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":45,"ondisk_log_size":45,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":1189092,"kb_used_data":708,"kb_used_omap":0,"kb_used_meta":1188352,"kb_avail":376298268,"statfs":{"total":386547056640,"available":385329426432,"internally_reserved":0,"allocated":724992,"data_stored":243006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":1216872448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.002138"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:29:14.583604+0000","last_change":"2023-09-30T22:26:51.708966+0000","last_active":"2023-09-30T22:29:14.583604+0000","last_peered":"2023-09-30T22:29:14.583604+0000","last_clean":"2023-09-30T22:29:14.583604+0000","last_became_active":"2023-09-30T22:26:51.708653+0000","last_became_peered":"2023-09-30T22:26:51.708653+0000","last_unstale":"2023-09-30T22:29:14.583604+0000","last_undegraded":"2023-09-30T22:29:14.583604+0000","last_fullsized":"2023-09-30T22:29:14.583604+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:29:14.583358+0000","last_change":"2023-09-30T22:26:51.727077+0000","last_active":"2023-09-30T22:29:14.583358+0000","last_peered":"2023-09-30T22:29:14.583358+0000","last_clean":"2023-09-30T22:29:14.583358+0000","last_became_active":"2023-09-30T22:26:51.726778+0000","last_became_peered":"2023-09-30T22:26:51.726778+0000","last_unstale":"2023-09-30T22:29:14.583358+0000","last_undegraded":"2023-09-30T22:29:14.583358+0000","last_fullsized":"2023-09-30T22:29:14.583358+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.238854+0000","last_change":"2023-09-30T22:26:51.706919+0000","last_active":"2023-09-30T22:27:42.238854+0000","last_peered":"2023-09-30T22:27:42.238854+0000","last_clean":"2023-09-30T22:27:42.238854+0000","last_became_active":"2023-09-30T22:26:51.706676+0000","last_became_peered":"2023-09-30T22:26:51.706676+0000","last_unstale":"2023-09-30T22:27:42.238854+0000","last_undegraded":"2023-09-30T22:27:42.238854+0000","last_fullsized":"2023-09-30T22:27:42.238854+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.238471+0000","last_change":"2023-09-30T22:26:51.727386+0000","last_active":"2023-09-30T22:27:42.238471+0000","last_peered":"2023-09-30T22:27:42.238471+0000","last_clean":"2023-09-30T22:27:42.238471+0000","last_became_active":"2023-09-30T22:26:51.727091+0000","last_became_peered":"2023-09-30T22:26:51.727091+0000","last_unstale":"2023-09-30T22:27:42.238471+0000","last_undegraded":"2023-09-30T22:27:42.238471+0000","last_fullsized":"2023-09-30T22:27:42.238471+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.238005+0000","last_change":"2023-09-30T22:26:51.719993+0000","last_active":"2023-09-30T22:27:42.238005+0000","last_peered":"2023-09-30T22:27:42.238005+0000","last_clean":"2023-09-30T22:27:42.238005+0000","last_became_active":"2023-09-30T22:26:51.719495+0000","last_became_peered":"2023-09-30T22:26:51.719495+0000","last_unstale":"2023-09-30T22:27:42.238005+0000","last_undegraded":"2023-09-30T22:27:42.238005+0000","last_fullsized":"2023-09-30T22:27:42.238005+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.241844+0000","last_change":"2023-09-30T22:27:42.240573+0000","last_active":"2023-09-30T22:27:42.241844+0000","last_peered":"2023-09-30T22:27:42.241844+0000","last_clean":"2023-09-30T22:27:42.241844+0000","last_became_active":"2023-09-30T22:27:42.240236+0000","last_became_peered":"2023-09-30T22:27:42.240236+0000","last_unstale":"2023-09-30T22:27:42.241844+0000","last_undegraded":"2023-09-30T22:27:42.241844+0000","last_fullsized":"2023-09-30T22:27:42.241844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.233985+0000","last_change":"2023-09-30T22:26:51.705826+0000","last_active":"2023-09-30T22:27:42.233985+0000","last_peered":"2023-09-30T22:27:42.233985+0000","last_clean":"2023-09-30T22:27:42.233985+0000","last_became_active":"2023-09-30T22:26:51.705037+0000","last_became_peered":"2023-09-30T22:26:51.705037+0000","last_unstale":"2023-09-30T22:27:42.233985+0000","last_undegraded":"2023-09-30T22:27:42.233985+0000","last_fullsized":"2023-09-30T22:27:42.233985+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.394405+0000","last_change":"2023-09-30T22:26:51.726640+0000","last_active":"2023-09-30T22:27:42.394405+0000","last_peered":"2023-09-30T22:27:42.394405+0000","last_clean":"2023-09-30T22:27:42.394405+0000","last_became_active":"2023-09-30T22:26:51.726278+0000","last_became_peered":"2023-09-30T22:26:51.726278+0000","last_unstale":"2023-09-30T22:27:42.394405+0000","last_undegraded":"2023-09-30T22:27:42.394405+0000","last_fullsized":"2023-09-30T22:27:42.394405+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.239452+0000","last_change":"2023-09-30T22:26:51.721817+0000","last_active":"2023-09-30T22:27:42.239452+0000","last_peered":"2023-09-30T22:27:42.239452+0000","last_clean":"2023-09-30T22:27:42.239452+0000","last_became_active":"2023-09-30T22:26:51.721373+0000","last_became_peered":"2023-09-30T22:26:51.721373+0000","last_unstale":"2023-09-30T22:27:42.239452+0000","last_undegraded":"2023-09-30T22:27:42.239452+0000","last_fullsized":"2023-09-30T22:27:42.239452+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.395173+0000","last_change":"2023-09-30T22:26:51.726348+0000","last_active":"2023-09-30T22:27:42.395173+0000","last_peered":"2023-09-30T22:27:42.395173+0000","last_clean":"2023-09-30T22:27:42.395173+0000","last_became_active":"2023-09-30T22:26:51.726030+0000","last_became_peered":"2023-09-30T22:26:51.726030+0000","last_unstale":"2023-09-30T22:27:42.395173+0000","last_undegraded":"2023-09-30T22:27:42.395173+0000","last_fullsized":"2023-09-30T22:27:42.395173+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.394692+0000","last_change":"2023-09-30T22:26:51.722619+0000","last_active":"2023-09-30T22:27:42.394692+0000","last_peered":"2023-09-30T22:27:42.394692+0000","last_clean":"2023-09-30T22:27:42.394692+0000","last_became_active":"2023-09-30T22:26:51.722317+0000","last_became_peered":"2023-09-30T22:26:51.722317+0000","last_unstale":"2023-09-30T22:27:42.394692+0000","last_undegraded":"2023-09-30T22:27:42.394692+0000","last_fullsized":"2023-09-30T22:27:42.394692+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:29:14.582628+0000","last_change":"2023-09-30T22:26:51.709697+0000","last_active":"2023-09-30T22:29:14.582628+0000","last_peered":"2023-09-30T22:29:14.582628+0000","last_clean":"2023-09-30T22:29:14.582628+0000","last_became_active":"2023-09-30T22:26:51.709342+0000","last_became_peered":"2023-09-30T22:26:51.709342+0000","last_unstale":"2023-09-30T22:29:14.582628+0000","last_undegraded":"2023-09-30T22:29:14.582628+0000","last_fullsized":"2023-09-30T22:29:14.582628+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:29:14.582455+0000","last_change":"2023-09-30T22:26:51.716276+0000","last_active":"2023-09-30T22:29:14.582455+0000","last_peered":"2023-09-30T22:29:14.582455+0000","last_clean":"2023-09-30T22:29:14.582455+0000","last_became_active":"2023-09-30T22:26:51.715634+0000","last_became_peered":"2023-09-30T22:26:51.715634+0000","last_unstale":"2023-09-30T22:29:14.582455+0000","last_undegraded":"2023-09-30T22:29:14.582455+0000","last_fullsized":"2023-09-30T22:29:14.582455+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.394593+0000","last_change":"2023-09-30T22:26:51.695948+0000","last_active":"2023-09-30T22:27:42.394593+0000","last_peered":"2023-09-30T22:27:42.394593+0000","last_clean":"2023-09-30T22:27:42.394593+0000","last_became_active":"2023-09-30T22:26:51.695661+0000","last_became_peered":"2023-09-30T22:26:51.695661+0000","last_unstale":"2023-09-30T22:27:42.394593+0000","last_undegraded":"2023-09-30T22:27:42.394593+0000","last_fullsized":"2023-09-30T22:27:42.394593+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.237389+0000","last_change":"2023-09-30T22:26:51.708532+0000","last_active":"2023-09-30T22:27:42.237389+0000","last_peered":"2023-09-30T22:27:42.237389+0000","last_clean":"2023-09-30T22:27:42.237389+0000","last_became_active":"2023-09-30T22:26:51.708204+0000","last_became_peered":"2023-09-30T22:26:51.708204+0000","last_unstale":"2023-09-30T22:27:42.237389+0000","last_undegraded":"2023-09-30T22:27:42.237389+0000","last_fullsized":"2023-09-30T22:27:42.237389+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.235077+0000","last_change":"2023-09-30T22:26:51.704597+0000","last_active":"2023-09-30T22:27:42.235077+0000","last_peered":"2023-09-30T22:27:42.235077+0000","last_clean":"2023-09-30T22:27:42.235077+0000","last_became_active":"2023-09-30T22:26:51.704307+0000","last_became_peered":"2023-09-30T22:26:51.704307+0000","last_unstale":"2023-09-30T22:27:42.235077+0000","last_undegraded":"2023-09-30T22:27:42.235077+0000","last_fullsized":"2023-09-30T22:27:42.235077+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.239386+0000","last_change":"2023-09-30T22:26:51.721640+0000","last_active":"2023-09-30T22:27:42.239386+0000","last_peered":"2023-09-30T22:27:42.239386+0000","last_clean":"2023-09-30T22:27:42.239386+0000","last_became_active":"2023-09-30T22:26:51.720950+0000","last_became_peered":"2023-09-30T22:26:51.720950+0000","last_unstale":"2023-09-30T22:27:42.239386+0000","last_undegraded":"2023-09-30T22:27:42.239386+0000","last_fullsized":"2023-09-30T22:27:42.239386+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.238988+0000","last_change":"2023-09-30T22:26:51.697513+0000","last_active":"2023-09-30T22:27:42.238988+0000","last_peered":"2023-09-30T22:27:42.238988+0000","last_clean":"2023-09-30T22:27:42.238988+0000","last_became_active":"2023-09-30T22:26:51.697137+0000","last_became_peered":"2023-09-30T22:26:51.697137+0000","last_unstale":"2023-09-30T22:27:42.238988+0000","last_undegraded":"2023-09-30T22:27:42.238988+0000","last_fullsized":"2023-09-30T22:27:42.238988+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:29:14.582991+0000","last_change":"2023-09-30T22:26:51.715476+0000","last_active":"2023-09-30T22:29:14.582991+0000","last_peered":"2023-09-30T22:29:14.582991+0000","last_clean":"2023-09-30T22:29:14.582991+0000","last_became_active":"2023-09-30T22:26:51.715176+0000","last_became_peered":"2023-09-30T22:26:51.715176+0000","last_unstale":"2023-09-30T22:29:14.582991+0000","last_undegraded":"2023-09-30T22:29:14.582991+0000","last_fullsized":"2023-09-30T22:29:14.582991+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.394819+0000","last_change":"2023-09-30T22:26:51.724991+0000","last_active":"2023-09-30T22:27:42.394819+0000","last_peered":"2023-09-30T22:27:42.394819+0000","last_clean":"2023-09-30T22:27:42.394819+0000","last_became_active":"2023-09-30T22:26:51.724240+0000","last_became_peered":"2023-09-30T22:26:51.724240+0000","last_unstale":"2023-09-30T22:27:42.394819+0000","last_undegraded":"2023-09-30T22:27:42.394819+0000","last_fullsized":"2023-09-30T22:27:42.394819+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.394105+0000","last_change":"2023-09-30T22:26:51.717833+0000","last_active":"2023-09-30T22:27:42.394105+0000","last_peered":"2023-09-30T22:27:42.394105+0000","last_clean":"2023-09-30T22:27:42.394105+0000","last_became_active":"2023-09-30T22:26:51.717517+0000","last_became_peered":"2023-09-30T22:26:51.717517+0000","last_unstale":"2023-09-30T22:27:42.394105+0000","last_undegraded":"2023-09-30T22:27:42.394105+0000","last_fullsized":"2023-09-30T22:27:42.394105+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.237155+0000","last_change":"2023-09-30T22:26:51.719030+0000","last_active":"2023-09-30T22:27:42.237155+0000","last_peered":"2023-09-30T22:27:42.237155+0000","last_clean":"2023-09-30T22:27:42.237155+0000","last_became_active":"2023-09-30T22:26:51.715370+0000","last_became_peered":"2023-09-30T22:26:51.715370+0000","last_unstale":"2023-09-30T22:27:42.237155+0000","last_undegraded":"2023-09-30T22:27:42.237155+0000","last_fullsized":"2023-09-30T22:27:42.237155+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.233718+0000","last_change":"2023-09-30T22:26:51.698664+0000","last_active":"2023-09-30T22:27:42.233718+0000","last_peered":"2023-09-30T22:27:42.233718+0000","last_clean":"2023-09-30T22:27:42.233718+0000","last_became_active":"2023-09-30T22:26:51.698321+0000","last_became_peered":"2023-09-30T22:26:51.698321+0000","last_unstale":"2023-09-30T22:27:42.233718+0000","last_undegraded":"2023-09-30T22:27:42.233718+0000","last_fullsized":"2023-09-30T22:27:42.233718+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.394494+0000","last_change":"2023-09-30T22:26:51.727955+0000","last_active":"2023-09-30T22:27:42.394494+0000","last_peered":"2023-09-30T22:27:42.394494+0000","last_clean":"2023-09-30T22:27:42.394494+0000","last_became_active":"2023-09-30T22:26:51.727209+0000","last_became_peered":"2023-09-30T22:26:51.727209+0000","last_unstale":"2023-09-30T22:27:42.394494+0000","last_undegraded":"2023-09-30T22:27:42.394494+0000","last_fullsized":"2023-09-30T22:27:42.394494+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.395261+0000","last_change":"2023-09-30T22:26:51.721154+0000","last_active":"2023-09-30T22:27:42.395261+0000","last_peered":"2023-09-30T22:27:42.395261+0000","last_clean":"2023-09-30T22:27:42.395261+0000","last_became_active":"2023-09-30T22:26:51.720703+0000","last_became_peered":"2023-09-30T22:26:51.720703+0000","last_unstale":"2023-09-30T22:27:42.395261+0000","last_undegraded":"2023-09-30T22:27:42.395261+0000","last_fullsized":"2023-09-30T22:27:42.395261+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.395135+0000","last_change":"2023-09-30T22:26:51.701309+0000","last_active":"2023-09-30T22:27:42.395135+0000","last_peered":"2023-09-30T22:27:42.395135+0000","last_clean":"2023-09-30T22:27:42.395135+0000","last_became_active":"2023-09-30T22:26:51.701012+0000","last_became_peered":"2023-09-30T22:26:51.701012+0000","last_unstale":"2023-09-30T22:27:42.395135+0000","last_undegraded":"2023-09-30T22:27:42.395135+0000","last_fullsized":"2023-09-30T22:27:42.395135+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.233911+0000","last_change":"2023-09-30T22:26:51.715042+0000","last_active":"2023-09-30T22:27:42.233911+0000","last_peered":"2023-09-30T22:27:42.233911+0000","last_clean":"2023-09-30T22:27:42.233911+0000","last_became_active":"2023-09-30T22:26:51.712158+0000","last_became_peered":"2023-09-30T22:26:51.712158+0000","last_unstale":"2023-09-30T22:27:42.233911+0000","last_undegraded":"2023-09-30T22:27:42.233911+0000","last_fullsized":"2023-09-30T22:27:42.233911+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"15'1","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.394384+0000","last_change":"2023-09-30T22:26:51.296397+0000","last_active":"2023-09-30T22:27:42.394384+0000","last_peered":"2023-09-30T22:27:42.394384+0000","last_clean":"2023-09-30T22:27:42.394384+0000","last_became_active":"2023-09-30T22:26:51.295860+0000","last_became_peered":"2023-09-30T22:26:51.295860+0000","last_unstale":"2023-09-30T22:27:42.394384+0000","last_undegraded":"2023-09-30T22:27:42.394384+0000","last_fullsized":"2023-09-30T22:27:42.394384+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.234008+0000","last_change":"2023-09-30T22:26:51.712472+0000","last_active":"2023-09-30T22:27:42.234008+0000","last_peered":"2023-09-30T22:27:42.234008+0000","last_clean":"2023-09-30T22:27:42.234008+0000","last_became_active":"2023-09-30T22:26:51.711527+0000","last_became_peered":"2023-09-30T22:26:51.711527+0000","last_unstale":"2023-09-30T22:27:42.234008+0000","last_undegraded":"2023-09-30T22:27:42.234008+0000","last_fullsized":"2023-09-30T22:27:42.234008+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.394016+0000","last_change":"2023-09-30T22:26:51.298004+0000","last_active":"2023-09-30T22:27:42.394016+0000","last_peered":"2023-09-30T22:27:42.394016+0000","last_clean":"2023-09-30T22:27:42.394016+0000","last_became_active":"2023-09-30T22:26:51.297714+0000","last_became_peered":"2023-09-30T22:26:51.297714+0000","last_unstale":"2023-09-30T22:27:42.394016+0000","last_undegraded":"2023-09-30T22:27:42.394016+0000","last_fullsized":"2023-09-30T22:27:42.394016+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.239079+0000","last_change":"2023-09-30T22:26:51.727608+0000","last_active":"2023-09-30T22:27:42.239079+0000","last_peered":"2023-09-30T22:27:42.239079+0000","last_clean":"2023-09-30T22:27:42.239079+0000","last_became_active":"2023-09-30T22:26:51.727154+0000","last_became_peered":"2023-09-30T22:26:51.727154+0000","last_unstale":"2023-09-30T22:27:42.239079+0000","last_undegraded":"2023-09-30T22:27:42.239079+0000","last_fullsized":"2023-09-30T22:27:42.239079+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.235317+0000","last_change":"2023-09-30T22:26:51.285214+0000","last_active":"2023-09-30T22:27:42.235317+0000","last_peered":"2023-09-30T22:27:42.235317+0000","last_clean":"2023-09-30T22:27:42.235317+0000","last_became_active":"2023-09-30T22:26:51.284749+0000","last_became_peered":"2023-09-30T22:26:51.284749+0000","last_unstale":"2023-09-30T22:27:42.235317+0000","last_undegraded":"2023-09-30T22:27:42.235317+0000","last_fullsized":"2023-09-30T22:27:42.235317+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.235520+0000","last_change":"2023-09-30T22:26:51.706697+0000","last_active":"2023-09-30T22:27:42.235520+0000","last_peered":"2023-09-30T22:27:42.235520+0000","last_clean":"2023-09-30T22:27:42.235520+0000","last_became_active":"2023-09-30T22:26:51.706373+0000","last_became_peered":"2023-09-30T22:26:51.706373+0000","last_unstale":"2023-09-30T22:27:42.235520+0000","last_undegraded":"2023-09-30T22:27:42.235520+0000","last_fullsized":"2023-09-30T22:27:42.235520+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.239279+0000","last_change":"2023-09-30T22:26:51.281642+0000","last_active":"2023-09-30T22:27:42.239279+0000","last_peered":"2023-09-30T22:27:42.239279+0000","last_clean":"2023-09-30T22:27:42.239279+0000","last_became_active":"2023-09-30T22:26:51.281319+0000","last_became_peered":"2023-09-30T22:26:51.281319+0000","last_unstale":"2023-09-30T22:27:42.239279+0000","last_undegraded":"2023-09-30T22:27:42.239279+0000","last_fullsized":"2023-09-30T22:27:42.239279+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.235304+0000","last_change":"2023-09-30T22:26:51.705192+0000","last_active":"2023-09-30T22:27:42.235304+0000","last_peered":"2023-09-30T22:27:42.235304+0000","last_clean":"2023-09-30T22:27:42.235304+0000","last_became_active":"2023-09-30T22:26:51.704870+0000","last_became_peered":"2023-09-30T22:26:51.704870+0000","last_unstale":"2023-09-30T22:27:42.235304+0000","last_undegraded":"2023-09-30T22:27:42.235304+0000","last_fullsized":"2023-09-30T22:27:42.235304+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:29:14.583093+0000","last_change":"2023-09-30T22:26:51.293711+0000","last_active":"2023-09-30T22:29:14.583093+0000","last_peered":"2023-09-30T22:29:14.583093+0000","last_clean":"2023-09-30T22:29:14.583093+0000","last_became_active":"2023-09-30T22:26:51.293410+0000","last_became_peered":"2023-09-30T22:26:51.293410+0000","last_unstale":"2023-09-30T22:29:14.583093+0000","last_undegraded":"2023-09-30T22:29:14.583093+0000","last_fullsized":"2023-09-30T22:29:14.583093+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.234676+0000","last_change":"2023-09-30T22:26:51.706005+0000","last_active":"2023-09-30T22:27:42.234676+0000","last_peered":"2023-09-30T22:27:42.234676+0000","last_clean":"2023-09-30T22:27:42.234676+0000","last_became_active":"2023-09-30T22:26:51.705491+0000","last_became_peered":"2023-09-30T22:26:51.705491+0000","last_unstale":"2023-09-30T22:27:42.234676+0000","last_undegraded":"2023-09-30T22:27:42.234676+0000","last_fullsized":"2023-09-30T22:27:42.234676+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"15'3","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.238196+0000","last_change":"2023-09-30T22:26:51.279976+0000","last_active":"2023-09-30T22:27:42.238196+0000","last_peered":"2023-09-30T22:27:42.238196+0000","last_clean":"2023-09-30T22:27:42.238196+0000","last_became_active":"2023-09-30T22:26:51.279669+0000","last_became_peered":"2023-09-30T22:26:51.279669+0000","last_unstale":"2023-09-30T22:27:42.238196+0000","last_undegraded":"2023-09-30T22:27:42.238196+0000","last_fullsized":"2023-09-30T22:27:42.238196+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.394529+0000","last_change":"2023-09-30T22:26:51.725947+0000","last_active":"2023-09-30T22:27:42.394529+0000","last_peered":"2023-09-30T22:27:42.394529+0000","last_clean":"2023-09-30T22:27:42.394529+0000","last_became_active":"2023-09-30T22:26:51.723294+0000","last_became_peered":"2023-09-30T22:26:51.723294+0000","last_unstale":"2023-09-30T22:27:42.394529+0000","last_undegraded":"2023-09-30T22:27:42.394529+0000","last_fullsized":"2023-09-30T22:27:42.394529+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"20'12","reported_seq":26,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:29:14.617123+0000","last_change":"2023-09-30T22:26:51.297527+0000","last_active":"2023-09-30T22:29:14.617123+0000","last_peered":"2023-09-30T22:29:14.617123+0000","last_clean":"2023-09-30T22:29:14.617123+0000","last_became_active":"2023-09-30T22:26:51.297246+0000","last_became_peered":"2023-09-30T22:26:51.297246+0000","last_unstale":"2023-09-30T22:29:14.617123+0000","last_undegraded":"2023-09-30T22:29:14.617123+0000","last_fullsized":"2023-09-30T22:29:14.617123+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3563,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:29:14.582565+0000","last_change":"2023-09-30T22:26:51.710781+0000","last_active":"2023-09-30T22:29:14.582565+0000","last_peered":"2023-09-30T22:29:14.582565+0000","last_clean":"2023-09-30T22:29:14.582565+0000","last_became_active":"2023-09-30T22:26:51.710107+0000","last_became_peered":"2023-09-30T22:26:51.710107+0000","last_unstale":"2023-09-30T22:29:14.582565+0000","last_undegraded":"2023-09-30T22:29:14.582565+0000","last_fullsized":"2023-09-30T22:29:14.582565+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"15'2","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.234355+0000","last_change":"2023-09-30T22:26:51.290194+0000","last_active":"2023-09-30T22:27:42.234355+0000","last_peered":"2023-09-30T22:27:42.234355+0000","last_clean":"2023-09-30T22:27:42.234355+0000","last_became_active":"2023-09-30T22:26:51.288688+0000","last_became_peered":"2023-09-30T22:26:51.288688+0000","last_unstale":"2023-09-30T22:27:42.234355+0000","last_undegraded":"2023-09-30T22:27:42.234355+0000","last_fullsized":"2023-09-30T22:27:42.234355+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:29:14.582525+0000","last_change":"2023-09-30T22:27:42.396108+0000","last_active":"2023-09-30T22:29:14.582525+0000","last_peered":"2023-09-30T22:29:14.582525+0000","last_clean":"2023-09-30T22:29:14.582525+0000","last_became_active":"2023-09-30T22:27:42.395692+0000","last_became_peered":"2023-09-30T22:27:42.395692+0000","last_unstale":"2023-09-30T22:29:14.582525+0000","last_undegraded":"2023-09-30T22:29:14.582525+0000","last_fullsized":"2023-09-30T22:29:14.582525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.237153+0000","last_change":"2023-09-30T22:26:51.281127+0000","last_active":"2023-09-30T22:27:42.237153+0000","last_peered":"2023-09-30T22:27:42.237153+0000","last_clean":"2023-09-30T22:27:42.237153+0000","last_became_active":"2023-09-30T22:26:51.280507+0000","last_became_peered":"2023-09-30T22:26:51.280507+0000","last_unstale":"2023-09-30T22:27:42.237153+0000","last_undegraded":"2023-09-30T22:27:42.237153+0000","last_fullsized":"2023-09-30T22:27:42.237153+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.234226+0000","last_change":"2023-09-30T22:26:51.712671+0000","last_active":"2023-09-30T22:27:42.234226+0000","last_peered":"2023-09-30T22:27:42.234226+0000","last_clean":"2023-09-30T22:27:42.234226+0000","last_became_active":"2023-09-30T22:26:51.711836+0000","last_became_peered":"2023-09-30T22:26:51.711836+0000","last_unstale":"2023-09-30T22:27:42.234226+0000","last_undegraded":"2023-09-30T22:27:42.234226+0000","last_fullsized":"2023-09-30T22:27:42.234226+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.234330+0000","last_change":"2023-09-30T22:26:51.297467+0000","last_active":"2023-09-30T22:27:42.234330+0000","last_peered":"2023-09-30T22:27:42.234330+0000","last_clean":"2023-09-30T22:27:42.234330+0000","last_became_active":"2023-09-30T22:26:51.296950+0000","last_became_peered":"2023-09-30T22:26:51.296950+0000","last_unstale":"2023-09-30T22:27:42.234330+0000","last_undegraded":"2023-09-30T22:27:42.234330+0000","last_fullsized":"2023-09-30T22:27:42.234330+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.239168+0000","last_change":"2023-09-30T22:26:51.719411+0000","last_active":"2023-09-30T22:27:42.239168+0000","last_peered":"2023-09-30T22:27:42.239168+0000","last_clean":"2023-09-30T22:27:42.239168+0000","last_became_active":"2023-09-30T22:26:51.719111+0000","last_became_peered":"2023-09-30T22:26:51.719111+0000","last_unstale":"2023-09-30T22:27:42.239168+0000","last_undegraded":"2023-09-30T22:27:42.239168+0000","last_fullsized":"2023-09-30T22:27:42.239168+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"15'8","reported_seq":22,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.394756+0000","last_change":"2023-09-30T22:26:51.287702+0000","last_active":"2023-09-30T22:27:42.394756+0000","last_peered":"2023-09-30T22:27:42.394756+0000","last_clean":"2023-09-30T22:27:42.394756+0000","last_became_active":"2023-09-30T22:26:51.287434+0000","last_became_peered":"2023-09-30T22:26:51.287434+0000","last_unstale":"2023-09-30T22:27:42.394756+0000","last_undegraded":"2023-09-30T22:27:42.394756+0000","last_fullsized":"2023-09-30T22:27:42.394756+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:29:14.583520+0000","last_change":"2023-09-30T22:27:42.396215+0000","last_active":"2023-09-30T22:29:14.583520+0000","last_peered":"2023-09-30T22:29:14.583520+0000","last_clean":"2023-09-30T22:29:14.583520+0000","last_became_active":"2023-09-30T22:27:42.395849+0000","last_became_peered":"2023-09-30T22:27:42.395849+0000","last_unstale":"2023-09-30T22:29:14.583520+0000","last_undegraded":"2023-09-30T22:29:14.583520+0000","last_fullsized":"2023-09-30T22:29:14.583520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"15'1","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.239086+0000","last_change":"2023-09-30T22:26:51.291996+0000","last_active":"2023-09-30T22:27:42.239086+0000","last_peered":"2023-09-30T22:27:42.239086+0000","last_clean":"2023-09-30T22:27:42.239086+0000","last_became_active":"2023-09-30T22:26:51.291656+0000","last_became_peered":"2023-09-30T22:26:51.291656+0000","last_unstale":"2023-09-30T22:27:42.239086+0000","last_undegraded":"2023-09-30T22:27:42.239086+0000","last_fullsized":"2023-09-30T22:27:42.239086+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:29:14.581708+0000","last_change":"2023-09-30T22:27:42.396720+0000","last_active":"2023-09-30T22:29:14.581708+0000","last_peered":"2023-09-30T22:29:14.581708+0000","last_clean":"2023-09-30T22:29:14.581708+0000","last_became_active":"2023-09-30T22:27:42.395725+0000","last_became_peered":"2023-09-30T22:27:42.395725+0000","last_unstale":"2023-09-30T22:29:14.581708+0000","last_undegraded":"2023-09-30T22:29:14.581708+0000","last_fullsized":"2023-09-30T22:29:14.581708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"15'2","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:29:14.583186+0000","last_change":"2023-09-30T22:26:51.300700+0000","last_active":"2023-09-30T22:29:14.583186+0000","last_peered":"2023-09-30T22:29:14.583186+0000","last_clean":"2023-09-30T22:29:14.583186+0000","last_became_active":"2023-09-30T22:26:51.300416+0000","last_became_peered":"2023-09-30T22:26:51.300416+0000","last_unstale":"2023-09-30T22:29:14.583186+0000","last_undegraded":"2023-09-30T22:29:14.583186+0000","last_fullsized":"2023-09-30T22:29:14.583186+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.394121+0000","last_change":"2023-09-30T22:26:51.722902+0000","last_active":"2023-09-30T22:27:42.394121+0000","last_peered":"2023-09-30T22:27:42.394121+0000","last_clean":"2023-09-30T22:27:42.394121+0000","last_became_active":"2023-09-30T22:26:51.717919+0000","last_became_peered":"2023-09-30T22:26:51.717919+0000","last_unstale":"2023-09-30T22:27:42.394121+0000","last_undegraded":"2023-09-30T22:27:42.394121+0000","last_fullsized":"2023-09-30T22:27:42.394121+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"15'1","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.235059+0000","last_change":"2023-09-30T22:26:51.292120+0000","last_active":"2023-09-30T22:27:42.235059+0000","last_peered":"2023-09-30T22:27:42.235059+0000","last_clean":"2023-09-30T22:27:42.235059+0000","last_became_active":"2023-09-30T22:26:51.289469+0000","last_became_peered":"2023-09-30T22:26:51.289469+0000","last_unstale":"2023-09-30T22:27:42.235059+0000","last_undegraded":"2023-09-30T22:27:42.235059+0000","last_fullsized":"2023-09-30T22:27:42.235059+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:29:14.582253+0000","last_change":"2023-09-30T22:26:51.711649+0000","last_active":"2023-09-30T22:29:14.582253+0000","last_peered":"2023-09-30T22:29:14.582253+0000","last_clean":"2023-09-30T22:29:14.582253+0000","last_became_active":"2023-09-30T22:26:51.711384+0000","last_became_peered":"2023-09-30T22:26:51.711384+0000","last_unstale":"2023-09-30T22:29:14.582253+0000","last_undegraded":"2023-09-30T22:29:14.582253+0000","last_fullsized":"2023-09-30T22:29:14.582253+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.238674+0000","last_change":"2023-09-30T22:26:51.281627+0000","last_active":"2023-09-30T22:27:42.238674+0000","last_peered":"2023-09-30T22:27:42.238674+0000","last_clean":"2023-09-30T22:27:42.238674+0000","last_became_active":"2023-09-30T22:26:51.281311+0000","last_became_peered":"2023-09-30T22:26:51.281311+0000","last_unstale":"2023-09-30T22:27:42.238674+0000","last_undegraded":"2023-09-30T22:27:42.238674+0000","last_fullsized":"2023-09-30T22:27:42.238674+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.238320+0000","last_change":"2023-09-30T22:26:51.700837+0000","last_active":"2023-09-30T22:27:42.238320+0000","last_peered":"2023-09-30T22:27:42.238320+0000","last_clean":"2023-09-30T22:27:42.238320+0000","last_became_active":"2023-09-30T22:26:51.700523+0000","last_became_peered":"2023-09-30T22:26:51.700523+0000","last_unstale":"2023-09-30T22:27:42.238320+0000","last_undegraded":"2023-09-30T22:27:42.238320+0000","last_fullsized":"2023-09-30T22:27:42.238320+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"15'2","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.234152+0000","last_change":"2023-09-30T22:26:51.290248+0000","last_active":"2023-09-30T22:27:42.234152+0000","last_peered":"2023-09-30T22:27:42.234152+0000","last_clean":"2023-09-30T22:27:42.234152+0000","last_became_active":"2023-09-30T22:26:51.289168+0000","last_became_peered":"2023-09-30T22:26:51.289168+0000","last_unstale":"2023-09-30T22:27:42.234152+0000","last_undegraded":"2023-09-30T22:27:42.234152+0000","last_fullsized":"2023-09-30T22:27:42.234152+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.238595+0000","last_change":"2023-09-30T22:26:51.719224+0000","last_active":"2023-09-30T22:27:42.238595+0000","last_peered":"2023-09-30T22:27:42.238595+0000","last_clean":"2023-09-30T22:27:42.238595+0000","last_became_active":"2023-09-30T22:26:51.718890+0000","last_became_peered":"2023-09-30T22:26:51.718890+0000","last_unstale":"2023-09-30T22:27:42.238595+0000","last_undegraded":"2023-09-30T22:27:42.238595+0000","last_fullsized":"2023-09-30T22:27:42.238595+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.235840+0000","last_change":"2023-09-30T22:26:51.288204+0000","last_active":"2023-09-30T22:27:42.235840+0000","last_peered":"2023-09-30T22:27:42.235840+0000","last_clean":"2023-09-30T22:27:42.235840+0000","last_became_active":"2023-09-30T22:26:51.287903+0000","last_became_peered":"2023-09-30T22:26:51.287903+0000","last_unstale":"2023-09-30T22:27:42.235840+0000","last_undegraded":"2023-09-30T22:27:42.235840+0000","last_fullsized":"2023-09-30T22:27:42.235840+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:29:14.582796+0000","last_change":"2023-09-30T22:27:42.396176+0000","last_active":"2023-09-30T22:29:14.582796+0000","last_peered":"2023-09-30T22:29:14.582796+0000","last_clean":"2023-09-30T22:29:14.582796+0000","last_became_active":"2023-09-30T22:27:42.395667+0000","last_became_peered":"2023-09-30T22:27:42.395667+0000","last_unstale":"2023-09-30T22:29:14.582796+0000","last_undegraded":"2023-09-30T22:29:14.582796+0000","last_fullsized":"2023-09-30T22:29:14.582796+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.234082+0000","last_change":"2023-09-30T22:26:51.297240+0000","last_active":"2023-09-30T22:27:42.234082+0000","last_peered":"2023-09-30T22:27:42.234082+0000","last_clean":"2023-09-30T22:27:42.234082+0000","last_became_active":"2023-09-30T22:26:51.296547+0000","last_became_peered":"2023-09-30T22:26:51.296547+0000","last_unstale":"2023-09-30T22:27:42.234082+0000","last_undegraded":"2023-09-30T22:27:42.234082+0000","last_fullsized":"2023-09-30T22:27:42.234082+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:29:14.583275+0000","last_change":"2023-09-30T22:26:51.711889+0000","last_active":"2023-09-30T22:29:14.583275+0000","last_peered":"2023-09-30T22:29:14.583275+0000","last_clean":"2023-09-30T22:29:14.583275+0000","last_became_active":"2023-09-30T22:26:51.711570+0000","last_became_peered":"2023-09-30T22:26:51.711570+0000","last_unstale":"2023-09-30T22:29:14.583275+0000","last_undegraded":"2023-09-30T22:29:14.583275+0000","last_fullsized":"2023-09-30T22:29:14.583275+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.235076+0000","last_change":"2023-09-30T22:26:51.290699+0000","last_active":"2023-09-30T22:27:42.235076+0000","last_peered":"2023-09-30T22:27:42.235076+0000","last_clean":"2023-09-30T22:27:42.235076+0000","last_became_active":"2023-09-30T22:26:51.290396+0000","last_became_peered":"2023-09-30T22:26:51.290396+0000","last_unstale":"2023-09-30T22:27:42.235076+0000","last_undegraded":"2023-09-30T22:27:42.235076+0000","last_fullsized":"2023-09-30T22:27:42.235076+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.241857+0000","last_change":"2023-09-30T22:27:42.240919+0000","last_active":"2023-09-30T22:27:42.241857+0000","last_peered":"2023-09-30T22:27:42.241857+0000","last_clean":"2023-09-30T22:27:42.241857+0000","last_became_active":"2023-09-30T22:27:42.240544+0000","last_became_peered":"2023-09-30T22:27:42.240544+0000","last_unstale":"2023-09-30T22:27:42.241857+0000","last_undegraded":"2023-09-30T22:27:42.241857+0000","last_fullsized":"2023-09-30T22:27:42.241857+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.393691+0000","last_change":"2023-09-30T22:26:46.273891+0000","last_active":"2023-09-30T22:27:42.393691+0000","last_peered":"2023-09-30T22:27:42.393691+0000","last_clean":"2023-09-30T22:27:42.393691+0000","last_became_active":"2023-09-30T22:26:46.273588+0000","last_became_peered":"2023-09-30T22:26:46.273588+0000","last_unstale":"2023-09-30T22:27:42.393691+0000","last_undegraded":"2023-09-30T22:27:42.393691+0000","last_fullsized":"2023-09-30T22:27:42.393691+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:29:14.582902+0000","last_change":"2023-09-30T22:26:51.294682+0000","last_active":"2023-09-30T22:29:14.582902+0000","last_peered":"2023-09-30T22:29:14.582902+0000","last_clean":"2023-09-30T22:29:14.582902+0000","last_became_active":"2023-09-30T22:26:51.294422+0000","last_became_peered":"2023-09-30T22:26:51.294422+0000","last_unstale":"2023-09-30T22:29:14.582902+0000","last_undegraded":"2023-09-30T22:29:14.582902+0000","last_fullsized":"2023-09-30T22:29:14.582902+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.394478+0000","last_change":"2023-09-30T22:26:51.726954+0000","last_active":"2023-09-30T22:27:42.394478+0000","last_peered":"2023-09-30T22:27:42.394478+0000","last_clean":"2023-09-30T22:27:42.394478+0000","last_became_active":"2023-09-30T22:26:51.726641+0000","last_became_peered":"2023-09-30T22:26:51.726641+0000","last_unstale":"2023-09-30T22:27:42.394478+0000","last_undegraded":"2023-09-30T22:27:42.394478+0000","last_fullsized":"2023-09-30T22:27:42.394478+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.235981+0000","last_change":"2023-09-30T22:26:46.245946+0000","last_active":"2023-09-30T22:27:42.235981+0000","last_peered":"2023-09-30T22:27:42.235981+0000","last_clean":"2023-09-30T22:27:42.235981+0000","last_became_active":"2023-09-30T22:26:46.245602+0000","last_became_peered":"2023-09-30T22:26:46.245602+0000","last_unstale":"2023-09-30T22:27:42.235981+0000","last_undegraded":"2023-09-30T22:27:42.235981+0000","last_fullsized":"2023-09-30T22:27:42.235981+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.238085+0000","last_change":"2023-09-30T22:26:51.294691+0000","last_active":"2023-09-30T22:27:42.238085+0000","last_peered":"2023-09-30T22:27:42.238085+0000","last_clean":"2023-09-30T22:27:42.238085+0000","last_became_active":"2023-09-30T22:26:51.294411+0000","last_became_peered":"2023-09-30T22:26:51.294411+0000","last_unstale":"2023-09-30T22:27:42.238085+0000","last_undegraded":"2023-09-30T22:27:42.238085+0000","last_fullsized":"2023-09-30T22:27:42.238085+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.394893+0000","last_change":"2023-09-30T22:26:51.724759+0000","last_active":"2023-09-30T22:27:42.394893+0000","last_peered":"2023-09-30T22:27:42.394893+0000","last_clean":"2023-09-30T22:27:42.394893+0000","last_became_active":"2023-09-30T22:26:51.723221+0000","last_became_peered":"2023-09-30T22:26:51.723221+0000","last_unstale":"2023-09-30T22:27:42.394893+0000","last_undegraded":"2023-09-30T22:27:42.394893+0000","last_fullsized":"2023-09-30T22:27:42.394893+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:29:14.582369+0000","last_change":"2023-09-30T22:26:46.242607+0000","last_active":"2023-09-30T22:29:14.582369+0000","last_peered":"2023-09-30T22:29:14.582369+0000","last_clean":"2023-09-30T22:29:14.582369+0000","last_became_active":"2023-09-30T22:26:46.242283+0000","last_became_peered":"2023-09-30T22:26:46.242283+0000","last_unstale":"2023-09-30T22:29:14.582369+0000","last_undegraded":"2023-09-30T22:29:14.582369+0000","last_fullsized":"2023-09-30T22:29:14.582369+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.394654+0000","last_change":"2023-09-30T22:26:51.295409+0000","last_active":"2023-09-30T22:27:42.394654+0000","last_peered":"2023-09-30T22:27:42.394654+0000","last_clean":"2023-09-30T22:27:42.394654+0000","last_became_active":"2023-09-30T22:26:51.295140+0000","last_became_peered":"2023-09-30T22:26:51.295140+0000","last_unstale":"2023-09-30T22:27:42.394654+0000","last_undegraded":"2023-09-30T22:27:42.394654+0000","last_fullsized":"2023-09-30T22:27:42.394654+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.395245+0000","last_change":"2023-09-30T22:26:51.715504+0000","last_active":"2023-09-30T22:27:42.395245+0000","last_peered":"2023-09-30T22:27:42.395245+0000","last_clean":"2023-09-30T22:27:42.395245+0000","last_became_active":"2023-09-30T22:26:51.715258+0000","last_became_peered":"2023-09-30T22:26:51.715258+0000","last_unstale":"2023-09-30T22:27:42.395245+0000","last_undegraded":"2023-09-30T22:27:42.395245+0000","last_fullsized":"2023-09-30T22:27:42.395245+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.234249+0000","last_change":"2023-09-30T22:26:46.242752+0000","last_active":"2023-09-30T22:27:42.234249+0000","last_peered":"2023-09-30T22:27:42.234249+0000","last_clean":"2023-09-30T22:27:42.234249+0000","last_became_active":"2023-09-30T22:26:46.242385+0000","last_became_peered":"2023-09-30T22:26:46.242385+0000","last_unstale":"2023-09-30T22:27:42.234249+0000","last_undegraded":"2023-09-30T22:27:42.234249+0000","last_fullsized":"2023-09-30T22:27:42.234249+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.393778+0000","last_change":"2023-09-30T22:26:46.273930+0000","last_active":"2023-09-30T22:27:42.393778+0000","last_peered":"2023-09-30T22:27:42.393778+0000","last_clean":"2023-09-30T22:27:42.393778+0000","last_became_active":"2023-09-30T22:26:46.273621+0000","last_became_peered":"2023-09-30T22:26:46.273621+0000","last_unstale":"2023-09-30T22:27:42.393778+0000","last_undegraded":"2023-09-30T22:27:42.393778+0000","last_fullsized":"2023-09-30T22:27:42.393778+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.235754+0000","last_change":"2023-09-30T22:26:51.298163+0000","last_active":"2023-09-30T22:27:42.235754+0000","last_peered":"2023-09-30T22:27:42.235754+0000","last_clean":"2023-09-30T22:27:42.235754+0000","last_became_active":"2023-09-30T22:26:51.297897+0000","last_became_peered":"2023-09-30T22:26:51.297897+0000","last_unstale":"2023-09-30T22:27:42.235754+0000","last_undegraded":"2023-09-30T22:27:42.235754+0000","last_fullsized":"2023-09-30T22:27:42.235754+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.234722+0000","last_change":"2023-09-30T22:26:51.713608+0000","last_active":"2023-09-30T22:27:42.234722+0000","last_peered":"2023-09-30T22:27:42.234722+0000","last_clean":"2023-09-30T22:27:42.234722+0000","last_became_active":"2023-09-30T22:26:51.712992+0000","last_became_peered":"2023-09-30T22:26:51.712992+0000","last_unstale":"2023-09-30T22:27:42.234722+0000","last_undegraded":"2023-09-30T22:27:42.234722+0000","last_fullsized":"2023-09-30T22:27:42.234722+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.235158+0000","last_change":"2023-09-30T22:26:46.245668+0000","last_active":"2023-09-30T22:27:42.235158+0000","last_peered":"2023-09-30T22:27:42.235158+0000","last_clean":"2023-09-30T22:27:42.235158+0000","last_became_active":"2023-09-30T22:26:46.245319+0000","last_became_peered":"2023-09-30T22:26:46.245319+0000","last_unstale":"2023-09-30T22:27:42.235158+0000","last_undegraded":"2023-09-30T22:27:42.235158+0000","last_fullsized":"2023-09-30T22:27:42.235158+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.237312+0000","last_change":"2023-09-30T22:26:51.294664+0000","last_active":"2023-09-30T22:27:42.237312+0000","last_peered":"2023-09-30T22:27:42.237312+0000","last_clean":"2023-09-30T22:27:42.237312+0000","last_became_active":"2023-09-30T22:26:51.294357+0000","last_became_peered":"2023-09-30T22:26:51.294357+0000","last_unstale":"2023-09-30T22:27:42.237312+0000","last_undegraded":"2023-09-30T22:27:42.237312+0000","last_fullsized":"2023-09-30T22:27:42.237312+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.239247+0000","last_change":"2023-09-30T22:26:51.705784+0000","last_active":"2023-09-30T22:27:42.239247+0000","last_peered":"2023-09-30T22:27:42.239247+0000","last_clean":"2023-09-30T22:27:42.239247+0000","last_became_active":"2023-09-30T22:26:51.705480+0000","last_became_peered":"2023-09-30T22:26:51.705480+0000","last_unstale":"2023-09-30T22:27:42.239247+0000","last_undegraded":"2023-09-30T22:27:42.239247+0000","last_fullsized":"2023-09-30T22:27:42.239247+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.234542+0000","last_change":"2023-09-30T22:26:46.274062+0000","last_active":"2023-09-30T22:27:42.234542+0000","last_peered":"2023-09-30T22:27:42.234542+0000","last_clean":"2023-09-30T22:27:42.234542+0000","last_became_active":"2023-09-30T22:26:46.273752+0000","last_became_peered":"2023-09-30T22:26:46.273752+0000","last_unstale":"2023-09-30T22:27:42.234542+0000","last_undegraded":"2023-09-30T22:27:42.234542+0000","last_fullsized":"2023-09-30T22:27:42.234542+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.394316+0000","last_change":"2023-09-30T22:26:51.295181+0000","last_active":"2023-09-30T22:27:42.394316+0000","last_peered":"2023-09-30T22:27:42.394316+0000","last_clean":"2023-09-30T22:27:42.394316+0000","last_became_active":"2023-09-30T22:26:51.294889+0000","last_became_peered":"2023-09-30T22:26:51.294889+0000","last_unstale":"2023-09-30T22:27:42.394316+0000","last_undegraded":"2023-09-30T22:27:42.394316+0000","last_fullsized":"2023-09-30T22:27:42.394316+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.238245+0000","last_change":"2023-09-30T22:26:51.690467+0000","last_active":"2023-09-30T22:27:42.238245+0000","last_peered":"2023-09-30T22:27:42.238245+0000","last_clean":"2023-09-30T22:27:42.238245+0000","last_became_active":"2023-09-30T22:26:51.690163+0000","last_became_peered":"2023-09-30T22:26:51.690163+0000","last_unstale":"2023-09-30T22:27:42.238245+0000","last_undegraded":"2023-09-30T22:27:42.238245+0000","last_fullsized":"2023-09-30T22:27:42.238245+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"15'2","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.235289+0000","last_change":"2023-09-30T22:26:51.290268+0000","last_active":"2023-09-30T22:27:42.235289+0000","last_peered":"2023-09-30T22:27:42.235289+0000","last_clean":"2023-09-30T22:27:42.235289+0000","last_became_active":"2023-09-30T22:26:51.289409+0000","last_became_peered":"2023-09-30T22:26:51.289409+0000","last_unstale":"2023-09-30T22:27:42.235289+0000","last_undegraded":"2023-09-30T22:27:42.235289+0000","last_fullsized":"2023-09-30T22:27:42.235289+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.393716+0000","last_change":"2023-09-30T22:26:46.273888+0000","last_active":"2023-09-30T22:27:42.393716+0000","last_peered":"2023-09-30T22:27:42.393716+0000","last_clean":"2023-09-30T22:27:42.393716+0000","last_became_active":"2023-09-30T22:26:46.273549+0000","last_became_peered":"2023-09-30T22:26:46.273549+0000","last_unstale":"2023-09-30T22:27:42.393716+0000","last_undegraded":"2023-09-30T22:27:42.393716+0000","last_fullsized":"2023-09-30T22:27:42.393716+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:29:14.581810+0000","last_change":"2023-09-30T22:27:42.396210+0000","last_active":"2023-09-30T22:29:14.581810+0000","last_peered":"2023-09-30T22:29:14.581810+0000","last_clean":"2023-09-30T22:29:14.581810+0000","last_became_active":"2023-09-30T22:27:42.395859+0000","last_became_peered":"2023-09-30T22:27:42.395859+0000","last_unstale":"2023-09-30T22:29:14.581810+0000","last_undegraded":"2023-09-30T22:29:14.581810+0000","last_fullsized":"2023-09-30T22:29:14.581810+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.235361+0000","last_change":"2023-09-30T22:26:46.274508+0000","last_active":"2023-09-30T22:27:42.235361+0000","last_peered":"2023-09-30T22:27:42.235361+0000","last_clean":"2023-09-30T22:27:42.235361+0000","last_became_active":"2023-09-30T22:26:46.273567+0000","last_became_peered":"2023-09-30T22:26:46.273567+0000","last_unstale":"2023-09-30T22:27:42.235361+0000","last_undegraded":"2023-09-30T22:27:42.235361+0000","last_fullsized":"2023-09-30T22:27:42.235361+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:45.234457+0000","last_clean_scrub_stamp":"2023-09-30T22:26:45.234457+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"16'1","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.239058+0000","last_change":"2023-09-30T22:26:51.288625+0000","last_active":"2023-09-30T22:27:42.239058+0000","last_peered":"2023-09-30T22:27:42.239058+0000","last_clean":"2023-09-30T22:27:42.239058+0000","last_became_active":"2023-09-30T22:26:51.288286+0000","last_became_peered":"2023-09-30T22:26:51.288286+0000","last_unstale":"2023-09-30T22:27:42.239058+0000","last_undegraded":"2023-09-30T22:27:42.239058+0000","last_fullsized":"2023-09-30T22:27:42.239058+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:29:14.582221+0000","last_change":"2023-09-30T22:26:51.708965+0000","last_active":"2023-09-30T22:29:14.582221+0000","last_peered":"2023-09-30T22:29:14.582221+0000","last_clean":"2023-09-30T22:29:14.582221+0000","last_became_active":"2023-09-30T22:26:51.708607+0000","last_became_peered":"2023-09-30T22:26:51.708607+0000","last_unstale":"2023-09-30T22:29:14.582221+0000","last_undegraded":"2023-09-30T22:29:14.582221+0000","last_fullsized":"2023-09-30T22:29:14.582221+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"15'1","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.238166+0000","last_change":"2023-09-30T22:26:51.282125+0000","last_active":"2023-09-30T22:27:42.238166+0000","last_peered":"2023-09-30T22:27:42.238166+0000","last_clean":"2023-09-30T22:27:42.238166+0000","last_became_active":"2023-09-30T22:26:51.281780+0000","last_became_peered":"2023-09-30T22:26:51.281780+0000","last_unstale":"2023-09-30T22:27:42.238166+0000","last_undegraded":"2023-09-30T22:27:42.238166+0000","last_fullsized":"2023-09-30T22:27:42.238166+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:29:14.582806+0000","last_change":"2023-09-30T22:26:51.703281+0000","last_active":"2023-09-30T22:29:14.582806+0000","last_peered":"2023-09-30T22:29:14.582806+0000","last_clean":"2023-09-30T22:29:14.582806+0000","last_became_active":"2023-09-30T22:26:51.703007+0000","last_became_peered":"2023-09-30T22:26:51.703007+0000","last_unstale":"2023-09-30T22:29:14.582806+0000","last_undegraded":"2023-09-30T22:29:14.582806+0000","last_fullsized":"2023-09-30T22:29:14.582806+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"15'2","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.394623+0000","last_change":"2023-09-30T22:26:51.287630+0000","last_active":"2023-09-30T22:27:42.394623+0000","last_peered":"2023-09-30T22:27:42.394623+0000","last_clean":"2023-09-30T22:27:42.394623+0000","last_became_active":"2023-09-30T22:26:51.287337+0000","last_became_peered":"2023-09-30T22:26:51.287337+0000","last_unstale":"2023-09-30T22:27:42.394623+0000","last_undegraded":"2023-09-30T22:27:42.394623+0000","last_fullsized":"2023-09-30T22:27:42.394623+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:29:14.582509+0000","last_change":"2023-09-30T22:26:51.716458+0000","last_active":"2023-09-30T22:29:14.582509+0000","last_peered":"2023-09-30T22:29:14.582509+0000","last_clean":"2023-09-30T22:29:14.582509+0000","last_became_active":"2023-09-30T22:26:51.715957+0000","last_became_peered":"2023-09-30T22:26:51.715957+0000","last_unstale":"2023-09-30T22:29:14.582509+0000","last_undegraded":"2023-09-30T22:29:14.582509+0000","last_fullsized":"2023-09-30T22:29:14.582509+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"15'2","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.234639+0000","last_change":"2023-09-30T22:26:51.285956+0000","last_active":"2023-09-30T22:27:42.234639+0000","last_peered":"2023-09-30T22:27:42.234639+0000","last_clean":"2023-09-30T22:27:42.234639+0000","last_became_active":"2023-09-30T22:26:51.285690+0000","last_became_peered":"2023-09-30T22:26:51.285690+0000","last_unstale":"2023-09-30T22:27:42.234639+0000","last_undegraded":"2023-09-30T22:27:42.234639+0000","last_fullsized":"2023-09-30T22:27:42.234639+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:29:14.582154+0000","last_change":"2023-09-30T22:26:51.716290+0000","last_active":"2023-09-30T22:29:14.582154+0000","last_peered":"2023-09-30T22:29:14.582154+0000","last_clean":"2023-09-30T22:29:14.582154+0000","last_became_active":"2023-09-30T22:26:51.716007+0000","last_became_peered":"2023-09-30T22:26:51.716007+0000","last_unstale":"2023-09-30T22:29:14.582154+0000","last_undegraded":"2023-09-30T22:29:14.582154+0000","last_fullsized":"2023-09-30T22:29:14.582154+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1e","version":"15'1","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.234865+0000","last_change":"2023-09-30T22:26:51.296949+0000","last_active":"2023-09-30T22:27:42.234865+0000","last_peered":"2023-09-30T22:27:42.234865+0000","last_clean":"2023-09-30T22:27:42.234865+0000","last_became_active":"2023-09-30T22:26:51.296602+0000","last_became_peered":"2023-09-30T22:26:51.296602+0000","last_unstale":"2023-09-30T22:27:42.234865+0000","last_undegraded":"2023-09-30T22:27:42.234865+0000","last_fullsized":"2023-09-30T22:27:42.234865+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.241844+0000","last_change":"2023-09-30T22:27:42.240569+0000","last_active":"2023-09-30T22:27:42.241844+0000","last_peered":"2023-09-30T22:27:42.241844+0000","last_clean":"2023-09-30T22:27:42.241844+0000","last_became_active":"2023-09-30T22:27:42.240222+0000","last_became_peered":"2023-09-30T22:27:42.240222+0000","last_unstale":"2023-09-30T22:27:42.241844+0000","last_undegraded":"2023-09-30T22:27:42.241844+0000","last_fullsized":"2023-09-30T22:27:42.241844+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"15'4","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.239340+0000","last_change":"2023-09-30T22:26:51.291976+0000","last_active":"2023-09-30T22:27:42.239340+0000","last_peered":"2023-09-30T22:27:42.239340+0000","last_clean":"2023-09-30T22:27:42.239340+0000","last_became_active":"2023-09-30T22:26:51.291255+0000","last_became_peered":"2023-09-30T22:26:51.291255+0000","last_unstale":"2023-09-30T22:27:42.239340+0000","last_undegraded":"2023-09-30T22:27:42.239340+0000","last_fullsized":"2023-09-30T22:27:42.239340+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:50.251859+0000","last_clean_scrub_stamp":"2023-09-30T22:26:50.251859+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-09-30T22:29:14.581987+0000","last_change":"2023-09-30T22:26:51.713485+0000","last_active":"2023-09-30T22:29:14.581987+0000","last_peered":"2023-09-30T22:29:14.581987+0000","last_clean":"2023-09-30T22:29:14.581987+0000","last_became_active":"2023-09-30T22:26:51.713188+0000","last_became_peered":"2023-09-30T22:26:51.713188+0000","last_unstale":"2023-09-30T22:29:14.581987+0000","last_undegraded":"2023-09-30T22:29:14.581987+0000","last_fullsized":"2023-09-30T22:29:14.581987+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.394694+0000","last_change":"2023-09-30T22:26:51.704082+0000","last_active":"2023-09-30T22:27:42.394694+0000","last_peered":"2023-09-30T22:27:42.394694+0000","last_clean":"2023-09-30T22:27:42.394694+0000","last_became_active":"2023-09-30T22:26:51.703280+0000","last_became_peered":"2023-09-30T22:26:51.703280+0000","last_unstale":"2023-09-30T22:27:42.394694+0000","last_undegraded":"2023-09-30T22:27:42.394694+0000","last_fullsized":"2023-09-30T22:27:42.394694+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.234177+0000","last_change":"2023-09-30T22:26:51.705877+0000","last_active":"2023-09-30T22:27:42.234177+0000","last_peered":"2023-09-30T22:27:42.234177+0000","last_clean":"2023-09-30T22:27:42.234177+0000","last_became_active":"2023-09-30T22:26:51.705352+0000","last_became_peered":"2023-09-30T22:26:51.705352+0000","last_unstale":"2023-09-30T22:27:42.234177+0000","last_undegraded":"2023-09-30T22:27:42.234177+0000","last_fullsized":"2023-09-30T22:27:42.234177+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.237225+0000","last_change":"2023-09-30T22:26:51.712201+0000","last_active":"2023-09-30T22:27:42.237225+0000","last_peered":"2023-09-30T22:27:42.237225+0000","last_clean":"2023-09-30T22:27:42.237225+0000","last_became_active":"2023-09-30T22:26:51.709083+0000","last_became_peered":"2023-09-30T22:26:51.709083+0000","last_unstale":"2023-09-30T22:27:42.237225+0000","last_undegraded":"2023-09-30T22:27:42.237225+0000","last_fullsized":"2023-09-30T22:27:42.237225+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.235010+0000","last_change":"2023-09-30T22:26:51.706543+0000","last_active":"2023-09-30T22:27:42.235010+0000","last_peered":"2023-09-30T22:27:42.235010+0000","last_clean":"2023-09-30T22:27:42.235010+0000","last_became_active":"2023-09-30T22:26:51.706251+0000","last_became_peered":"2023-09-30T22:26:51.706251+0000","last_unstale":"2023-09-30T22:27:42.235010+0000","last_undegraded":"2023-09-30T22:27:42.235010+0000","last_fullsized":"2023-09-30T22:27:42.235010+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-09-30T22:27:42.394716+0000","last_change":"2023-09-30T22:26:51.724591+0000","last_active":"2023-09-30T22:27:42.394716+0000","last_peered":"2023-09-30T22:27:42.394716+0000","last_clean":"2023-09-30T22:27:42.394716+0000","last_became_active":"2023-09-30T22:26:51.722847+0000","last_became_peered":"2023-09-30T22:26:51.722847+0000","last_unstale":"2023-09-30T22:27:42.394716+0000","last_undegraded":"2023-09-30T22:27:42.394716+0000","last_fullsized":"2023-09-30T22:27:42.394716+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-09-30T22:26:51.255999+0000","last_clean_scrub_stamp":"2023-09-30T22:26:51.255999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4333,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":51,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":45,"ondisk_log_size":45,"up":64,"acting":64,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705696,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297284,"kb_used_data":188,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074556,"statfs":{"total":96636764160,"available":96332345344,"internally_reserved":0,"allocated":192512,"data_stored":64640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 22:28:48 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.35799999999999998,"15min":0.36199999999999999},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.46999999999999997,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.307},{"interface":"front","average":{"1min":0.33900000000000002,"5min":0.44900000000000001,"15min":0.46700000000000003},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.51500000000000001,"5min":2.5600000000000001,"15min":2.5600000000000001},"last":0.36699999999999999}]},{"osd":1,"last update":"Sat Sep 30 22:28:48 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.45700000000000002,"15min":0.47299999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.5,"5min":2.6040000000000001,"15min":2.6040000000000001},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.375,"15min":0.38},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.57299999999999995,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.38400000000000001}]},{"osd":2,"last update":"Sat Sep 30 22:28:48 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.376,"15min":0.376},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.40000000000000002,"15min":0.40600000000000003},"min":{"1min":0.27400000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.54400000000000004,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.35199999999999998}]}]},{"osd":2,"up_from":9,"seq":38654705697,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297272,"kb_used_data":176,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074568,"statfs":{"total":96636764160,"available":96332357632,"internally_reserved":0,"allocated":180224,"data_stored":64446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 22:28:47 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.32900000000000001,"15min":0.33000000000000002},"min":{"1min":0.26700000000000002,"5min":0.223,"15min":0.223},"max":{"1min":0.40600000000000003,"5min":0.45300000000000001,"15min":0.45300000000000001},"last":0.307},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.35699999999999998,"15min":0.35899999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.49199999999999999,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.36899999999999999}]},{"osd":1,"last update":"Sat Sep 30 22:28:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.35699999999999998,"15min":0.35799999999999998},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.46300000000000002,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.35299999999999998,"15min":0.35699999999999998},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.38200000000000001,"5min":0.52100000000000002,"15min":0.52100000000000002},"last":0.32700000000000001}]},{"osd":3,"last update":"Sat Sep 30 22:28:47 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.39000000000000001,"15min":0.39300000000000002},"min":{"1min":0.309,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.45900000000000002,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.374,"5min":0.39200000000000002,"15min":0.39500000000000002},"min":{"1min":0.316,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.433,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.378}]}]},{"osd":0,"up_from":9,"seq":38654705696,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297268,"kb_used_data":172,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074572,"statfs":{"total":96636764160,"available":96332361728,"internally_reserved":0,"allocated":176128,"data_stored":56402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Sep 30 22:28:50 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.30499999999999999,"15min":0.30099999999999999},"min":{"1min":0.27900000000000003,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.45600000000000002,"5min":0.51200000000000001,"15min":0.51200000000000001},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.33600000000000002,"15min":0.33300000000000002},"min":{"1min":0.26100000000000001,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"last":0.41699999999999998}]},{"osd":2,"last update":"Sat Sep 30 22:28:50 2023","interfaces":[{"interface":"back","average":{"1min":0.36599999999999999,"5min":0.35399999999999998,"15min":0.35199999999999998},"min":{"1min":0.28000000000000003,"5min":0.25,"15min":0.25},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.34300000000000003,"15min":0.34100000000000003},"min":{"1min":0.28799999999999998,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.51300000000000001,"5min":0.53800000000000003,"15min":0.53800000000000003},"last":0.58599999999999997}]},{"osd":3,"last update":"Sat Sep 30 22:28:50 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.36399999999999999,"15min":0.36299999999999999},"min":{"1min":0.27400000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.626},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.50900000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"last":0.33200000000000002}]}]},{"osd":1,"up_from":9,"seq":38654705696,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297268,"kb_used_data":172,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074572,"statfs":{"total":96636764160,"available":96332361728,"internally_reserved":0,"allocated":176128,"data_stored":57518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Sep 30 22:28:45 2023","interfaces":[{"interface":"back","average":{"1min":0.29299999999999998,"5min":0.309,"15min":0.312},"min":{"1min":0.17000000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.374,"5min":0.40699999999999997,"15min":0.40699999999999997},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.32000000000000001,"5min":0.33900000000000002,"15min":0.34200000000000003},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.40000000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"last":0.59299999999999997}]},{"osd":2,"last update":"Sat Sep 30 22:28:45 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.34599999999999997,"15min":0.34899999999999998},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.40500000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.41999999999999998,"5min":0.442,"15min":0.442},"last":0.57099999999999995}]},{"osd":3,"last update":"Sat Sep 30 22:28:45 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.38300000000000001,"15min":0.38700000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.45400000000000001,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.60699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-09-30T22:29:18.584 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-09-30T22:29:18.584 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-09-30T22:29:18.584 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-09-30T22:29:18.584 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-09-30T22:29:18.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.756+0000 7f80cda8b700 1 Processor -- start 2023-09-30T22:29:18.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.756+0000 7f80cda8b700 1 -- start start 2023-09-30T22:29:18.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.756+0000 7f80cda8b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f80c806a770 0x7f80c806ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:18.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.757+0000 7f80cda8b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f80c806b080 0x7f80c8131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:18.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.757+0000 7f80cda8b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80c809ece0 con 0x7f80c8131b40 2023-09-30T22:29:18.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.757+0000 7f80cda8b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80c809e760 con 0x7f80c806a770 2023-09-30T22:29:18.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.757+0000 7f80cda8b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80c809e040 con 0x7f80c806b080 2023-09-30T22:29:18.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.757+0000 7f80c6ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f80c806b080 0x7f80c8131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:18.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.757+0000 7f80c77fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f80c806a770 0x7f80c806ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:18.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.757+0000 7f80c7fff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f80c8131b40 0x7f80c8138850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33788/0 (socket says 172.21.15.16:33788) 2023-09-30T22:29:18.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.757+0000 7f80c7fff700 1 -- 172.21.15.16:0/1524997575 learned_addr learned my addr 172.21.15.16:0/1524997575 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:29:18.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.757+0000 7f80c77fe700 1 -- 172.21.15.16:0/1524997575 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f80c806b080 msgr2=0x7f80c8131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:18.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.757+0000 7f80c77fe700 1 --2- 172.21.15.16:0/1524997575 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f80c806b080 0x7f80c8131600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:18.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.757+0000 7f80c77fe700 1 -- 172.21.15.16:0/1524997575 >> v1:172.21.15.16:6789/0 conn(0x7f80c8131b40 legacy=0x7f80c8138850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:18.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.757+0000 7f80c77fe700 1 -- 172.21.15.16:0/1524997575 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80c809f630 con 0x7f80c806a770 2023-09-30T22:29:18.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.757+0000 7f80c67fc700 1 -- 172.21.15.16:0/1524997575 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3456516337 0 0) 0x7f80c809ece0 con 0x7f80c8131b40 2023-09-30T22:29:18.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.757+0000 7f80c6ffd700 1 --2- 172.21.15.16:0/1524997575 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f80c806b080 0x7f80c8131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:18.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.757+0000 7f80c77fe700 1 --2- 172.21.15.16:0/1524997575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80c806a770 0x7f80c806ab40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f80b8009840 tx=0x7f80b8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f880afaeaf1d26a0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:18.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.757+0000 7f80c67fc700 1 -- 172.21.15.16:0/1524997575 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b8011830 con 0x7f80c806a770 2023-09-30T22:29:18.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.757+0000 7f80c67fc700 1 -- 172.21.15.16:0/1524997575 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80b8011990 con 0x7f80c806a770 2023-09-30T22:29:18.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.757+0000 7f80c67fc700 1 -- 172.21.15.16:0/1524997575 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b8011d20 con 0x7f80c806a770 2023-09-30T22:29:18.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.758+0000 7f80cda8b700 1 -- 172.21.15.16:0/1524997575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80c806a770 msgr2=0x7f80c806ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:18.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.758+0000 7f80cda8b700 1 --2- 172.21.15.16:0/1524997575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80c806a770 0x7f80c806ab40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f80b8009840 tx=0x7f80b8009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:18.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.758+0000 7f80cda8b700 1 -- 172.21.15.16:0/1524997575 shutdown_connections 2023-09-30T22:29:18.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.758+0000 7f80cda8b700 1 --2- 172.21.15.16:0/1524997575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80c806a770 0x7f80c806ab40 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:18.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.758+0000 7f80cda8b700 1 --2- 172.21.15.16:0/1524997575 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f80c806b080 0x7f80c8131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:18.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.758+0000 7f80cda8b700 1 -- 172.21.15.16:0/1524997575 >> 172.21.15.16:0/1524997575 conn(0x7f80c8066f50 msgr2=0x7f80c809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:18.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.758+0000 7f80cda8b700 1 -- 172.21.15.16:0/1524997575 shutdown_connections 2023-09-30T22:29:18.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.758+0000 7f80cda8b700 1 -- 172.21.15.16:0/1524997575 wait complete. 2023-09-30T22:29:18.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.759+0000 7f80cda8b700 1 Processor -- start 2023-09-30T22:29:18.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.759+0000 7f80cda8b700 1 -- start start 2023-09-30T22:29:18.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.760+0000 7f80cda8b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f80c806a770 0x7f80c809aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:18.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.760+0000 7f80cda8b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f80c806b080 0x7f80c809b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:18.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.760+0000 7f80cda8b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80c8131b40 0x7f80c81c7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:18.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.760+0000 7f80cda8b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80c809a980 con 0x7f80c8131b40 2023-09-30T22:29:18.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.760+0000 7f80cda8b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80c8131160 con 0x7f80c806b080 2023-09-30T22:29:18.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.760+0000 7f80cda8b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f80c8121b40 con 0x7f80c806a770 2023-09-30T22:29:18.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.760+0000 7f80c6ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f80c806b080 0x7f80c809b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:18.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.760+0000 7f80c7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80c8131b40 0x7f80c81c7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:18.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.760+0000 7f80c6ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f80c806b080 0x7f80c809b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37078/0 (socket says 172.21.15.16:37078) 2023-09-30T22:29:18.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.760+0000 7f80c6ffd700 1 -- 172.21.15.16:0/3683540252 learned_addr learned my addr 172.21.15.16:0/3683540252 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:18.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.760+0000 7f80c77fe700 1 --2- 172.21.15.16:0/3683540252 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f80c806a770 0x7f80c809aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:18.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.760+0000 7f80c77fe700 1 -- 172.21.15.16:0/3683540252 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f80c806b080 msgr2=0x7f80c809b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:18.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.760+0000 7f80c77fe700 1 --2- 172.21.15.16:0/3683540252 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f80c806b080 0x7f80c809b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:18.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.760+0000 7f80c77fe700 1 -- 172.21.15.16:0/3683540252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80c8131b40 msgr2=0x7f80c81c7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:18.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.760+0000 7f80c77fe700 1 --2- 172.21.15.16:0/3683540252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80c8131b40 0x7f80c81c7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:18.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.760+0000 7f80c77fe700 1 -- 172.21.15.16:0/3683540252 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b00094a0 con 0x7f80c806a770 2023-09-30T22:29:18.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.760+0000 7f80c6ffd700 1 --2- 172.21.15.16:0/3683540252 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f80c806b080 0x7f80c809b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:18.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.760+0000 7f80c7fff700 1 --2- 172.21.15.16:0/3683540252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80c8131b40 0x7f80c81c7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:18.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.760+0000 7f80c77fe700 1 --2- 172.21.15.16:0/3683540252 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f80c806a770 0x7f80c809aca0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f80b8016000 tx=0x7f80b8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:18.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.761+0000 7f80bffff700 1 -- 172.21.15.16:0/3683540252 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b80036a0 con 0x7f80c806a770 2023-09-30T22:29:18.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.761+0000 7f80bffff700 1 -- 172.21.15.16:0/3683540252 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80b801f070 con 0x7f80c806a770 2023-09-30T22:29:18.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.761+0000 7f80cda8b700 1 -- 172.21.15.16:0/3683540252 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80b80094a0 con 0x7f80c806a770 2023-09-30T22:29:18.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.761+0000 7f80bffff700 1 -- 172.21.15.16:0/3683540252 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80b8003a20 con 0x7f80c806a770 2023-09-30T22:29:18.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.761+0000 7f80cda8b700 1 -- 172.21.15.16:0/3683540252 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80c809b720 con 0x7f80c806a770 2023-09-30T22:29:18.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.761+0000 7f80bffff700 1 -- 172.21.15.16:0/3683540252 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 8) v1 ==== 35553+0+0 (secure 0 0 0) 0x7f80b8003b80 con 0x7f80c806a770 2023-09-30T22:29:18.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.762+0000 7f80bffff700 1 -- 172.21.15.16:0/3683540252 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4049+0+0 (secure 0 0 0) 0x7f80b8021070 con 0x7f80c806a770 2023-09-30T22:29:18.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.762+0000 7f80cda8b700 1 -- 172.21.15.16:0/3683540252 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80c8074ea0 con 0x7f80c806a770 2023-09-30T22:29:18.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:18.767+0000 7f80bffff700 1 -- 172.21.15.16:0/3683540252 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f80b802b080 con 0x7f80c806a770 2023-09-30T22:29:19.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.015+0000 7f80cda8b700 1 -- 172.21.15.16:0/3683540252 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f80c81315a0 con 0x7f80c806a770 2023-09-30T22:29:19.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.015+0000 7f80bffff700 1 -- 172.21.15.16:0/3683540252 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f80b8015020 con 0x7f80c806a770 2023-09-30T22:29:19.175 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:29:19.176 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-09-30T22:29:19.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.018+0000 7f80cda8b700 1 -- 172.21.15.16:0/3683540252 >> v1:172.21.15.16:6816/126811 conn(0x7f80a802f480 legacy=0x7f80a8031930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:19.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.018+0000 7f80cda8b700 1 -- 172.21.15.16:0/3683540252 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f80c806a770 msgr2=0x7f80c809aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:19.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.018+0000 7f80cda8b700 1 --2- 172.21.15.16:0/3683540252 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f80c806a770 0x7f80c809aca0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f80b8016000 tx=0x7f80b8004750 comp rx=0 tx=0).stop 2023-09-30T22:29:19.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.018+0000 7f80cda8b700 1 -- 172.21.15.16:0/3683540252 shutdown_connections 2023-09-30T22:29:19.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.018+0000 7f80cda8b700 1 --2- 172.21.15.16:0/3683540252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80c8131b40 0x7f80c81c7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:19.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.018+0000 7f80cda8b700 1 --2- 172.21.15.16:0/3683540252 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f80c806a770 0x7f80c809aca0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:19.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.018+0000 7f80cda8b700 1 --2- 172.21.15.16:0/3683540252 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f80c806b080 0x7f80c809b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:19.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.018+0000 7f80cda8b700 1 -- 172.21.15.16:0/3683540252 >> 172.21.15.16:0/3683540252 conn(0x7f80c8066f50 msgr2=0x7f80c806ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:19.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.018+0000 7f80cda8b700 1 -- 172.21.15.16:0/3683540252 shutdown_connections 2023-09-30T22:29:19.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.018+0000 7f80cda8b700 1 -- 172.21.15.16:0/3683540252 wait complete. 2023-09-30T22:29:19.180 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-09-30T22:29:19.180 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:29:19.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.321+0000 7f9104e50700 1 Processor -- start 2023-09-30T22:29:19.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.321+0000 7f9104e50700 1 -- start start 2023-09-30T22:29:19.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.322+0000 7f9104e50700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f910009f410 0x7f9100136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:19.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.322+0000 7f9104e50700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9100136fe0 0x7f91001393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:19.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.322+0000 7f9104e50700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9100070940 con 0x7f910009e930 2023-09-30T22:29:19.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.322+0000 7f9104e50700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f910006fd60 con 0x7f910009f410 2023-09-30T22:29:19.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.322+0000 7f9104e50700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f910006fa80 con 0x7f9100136fe0 2023-09-30T22:29:19.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.322+0000 7f90fdd9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f910009f410 0x7f9100136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:19.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.322+0000 7f90fed9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9100136fe0 0x7f91001393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:19.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.322+0000 7f90fe59c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f910009e930 0x7f910009ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33798/0 (socket says 172.21.15.16:33798) 2023-09-30T22:29:19.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.322+0000 7f90fe59c700 1 -- 172.21.15.16:0/3010132635 learned_addr learned my addr 172.21.15.16:0/3010132635 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:29:19.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.323+0000 7f90fdd9b700 1 -- 172.21.15.16:0/3010132635 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9100136fe0 msgr2=0x7f91001393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:19.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.323+0000 7f90fdd9b700 1 --2- 172.21.15.16:0/3010132635 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9100136fe0 0x7f91001393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:19.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.323+0000 7f90fdd9b700 1 -- 172.21.15.16:0/3010132635 >> v1:172.21.15.16:6789/0 conn(0x7f910009e930 legacy=0x7f910009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:19.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.323+0000 7f90fd59a700 1 -- 172.21.15.16:0/3010132635 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1326091235 0 0) 0x7f9100070940 con 0x7f910009e930 2023-09-30T22:29:19.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.323+0000 7f90fdd9b700 1 -- 172.21.15.16:0/3010132635 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f910009fb60 con 0x7f910009f410 2023-09-30T22:29:19.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.323+0000 7f90fed9d700 1 --2- 172.21.15.16:0/3010132635 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9100136fe0 0x7f91001393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:19.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.323+0000 7f90fdd9b700 1 --2- 172.21.15.16:0/3010132635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f910009f410 0x7f9100136aa0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f90e8009840 tx=0x7f90e8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa0c01b76c5d382a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:19.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.323+0000 7f90fd59a700 1 -- 172.21.15.16:0/3010132635 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e8011830 con 0x7f910009f410 2023-09-30T22:29:19.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.323+0000 7f90fd59a700 1 -- 172.21.15.16:0/3010132635 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90e8011990 con 0x7f910009f410 2023-09-30T22:29:19.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.323+0000 7f90fd59a700 1 -- 172.21.15.16:0/3010132635 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e8011d20 con 0x7f910009f410 2023-09-30T22:29:19.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.324+0000 7f9104e50700 1 -- 172.21.15.16:0/3010132635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f910009f410 msgr2=0x7f9100136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:19.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.324+0000 7f9104e50700 1 --2- 172.21.15.16:0/3010132635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f910009f410 0x7f9100136aa0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f90e8009840 tx=0x7f90e8009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:19.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.324+0000 7f9104e50700 1 -- 172.21.15.16:0/3010132635 shutdown_connections 2023-09-30T22:29:19.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.324+0000 7f9104e50700 1 --2- 172.21.15.16:0/3010132635 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9100136fe0 0x7f91001393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:19.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.324+0000 7f9104e50700 1 --2- 172.21.15.16:0/3010132635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f910009f410 0x7f9100136aa0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:19.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.324+0000 7f9104e50700 1 -- 172.21.15.16:0/3010132635 >> 172.21.15.16:0/3010132635 conn(0x7f91000a4210 msgr2=0x7f91000a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:19.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.324+0000 7f9104e50700 1 -- 172.21.15.16:0/3010132635 shutdown_connections 2023-09-30T22:29:19.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.324+0000 7f9104e50700 1 -- 172.21.15.16:0/3010132635 wait complete. 2023-09-30T22:29:19.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.325+0000 7f9104e50700 1 Processor -- start 2023-09-30T22:29:19.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.325+0000 7f9104e50700 1 -- start start 2023-09-30T22:29:19.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.325+0000 7f9104e50700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f910009e930 0x7f91001c84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:19.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.325+0000 7f9104e50700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f910009f410 0x7f91001c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:19.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.325+0000 7f9104e50700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9100136fe0 0x7f910006b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:19.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.325+0000 7f9104e50700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9100132270 con 0x7f910009f410 2023-09-30T22:29:19.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.325+0000 7f9104e50700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f910009de80 con 0x7f910009e930 2023-09-30T22:29:19.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.326+0000 7f9104e50700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f91001bc160 con 0x7f9100136fe0 2023-09-30T22:29:19.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.326+0000 7f90fdd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f910009f410 0x7f91001c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:19.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.326+0000 7f90fe59c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f910009e930 0x7f91001c84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:19.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.326+0000 7f90fdd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f910009f410 0x7f91001c8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57668/0 (socket says 172.21.15.16:57668) 2023-09-30T22:29:19.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.326+0000 7f90fdd9b700 1 -- 172.21.15.16:0/4167159472 learned_addr learned my addr 172.21.15.16:0/4167159472 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:19.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.326+0000 7f90fed9d700 1 --2- 172.21.15.16:0/4167159472 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9100136fe0 0x7f910006b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:19.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.326+0000 7f90fdd9b700 1 -- 172.21.15.16:0/4167159472 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9100136fe0 msgr2=0x7f910006b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:19.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.326+0000 7f90fdd9b700 1 --2- 172.21.15.16:0/4167159472 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9100136fe0 0x7f910006b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:19.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.326+0000 7f90fdd9b700 1 -- 172.21.15.16:0/4167159472 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f910009e930 msgr2=0x7f91001c84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:19.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.326+0000 7f90fdd9b700 1 --2- 172.21.15.16:0/4167159472 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f910009e930 0x7f91001c84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:19.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.326+0000 7f90fdd9b700 1 -- 172.21.15.16:0/4167159472 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f8009a70 con 0x7f910009f410 2023-09-30T22:29:19.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.326+0000 7f90fed9d700 1 --2- 172.21.15.16:0/4167159472 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9100136fe0 0x7f910006b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:19.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.326+0000 7f90fe59c700 1 --2- 172.21.15.16:0/4167159472 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f910009e930 0x7f91001c84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:19.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.326+0000 7f90fdd9b700 1 --2- 172.21.15.16:0/4167159472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f910009f410 0x7f91001c8a10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f90e8009fd0 tx=0x7f90e8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:19.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.326+0000 7f90f6ffd700 1 -- 172.21.15.16:0/4167159472 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e80045d0 con 0x7f910009f410 2023-09-30T22:29:19.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.326+0000 7f90f6ffd700 1 -- 172.21.15.16:0/4167159472 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90e801f070 con 0x7f910009f410 2023-09-30T22:29:19.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.326+0000 7f9104e50700 1 -- 172.21.15.16:0/4167159472 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90e80094a0 con 0x7f910009f410 2023-09-30T22:29:19.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.326+0000 7f90f6ffd700 1 -- 172.21.15.16:0/4167159472 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e80038a0 con 0x7f910009f410 2023-09-30T22:29:19.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.326+0000 7f9104e50700 1 -- 172.21.15.16:0/4167159472 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91001c8f50 con 0x7f910009f410 2023-09-30T22:29:19.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.327+0000 7f90f6ffd700 1 -- 172.21.15.16:0/4167159472 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 8) v1 ==== 35553+0+0 (secure 0 0 0) 0x7f90e8003a00 con 0x7f910009f410 2023-09-30T22:29:19.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.328+0000 7f90f6ffd700 1 -- 172.21.15.16:0/4167159472 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4049+0+0 (secure 0 0 0) 0x7f90e8021070 con 0x7f910009f410 2023-09-30T22:29:19.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.328+0000 7f9104e50700 1 -- 172.21.15.16:0/4167159472 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9100074ea0 con 0x7f910009f410 2023-09-30T22:29:19.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.333+0000 7f90f6ffd700 1 -- 172.21.15.16:0/4167159472 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f90e8003cb0 con 0x7f910009f410 2023-09-30T22:29:19.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.496+0000 7f90f6ffd700 1 -- 172.21.15.16:0/4167159472 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f90e801a4d0 con 0x7f910009f410 2023-09-30T22:29:19.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.538+0000 7f9104e50700 1 -- 172.21.15.16:0/4167159472 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9100121a10 con 0x7f910009f410 2023-09-30T22:29:19.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.539+0000 7f90f6ffd700 1 -- 172.21.15.16:0/4167159472 <== mon.0 v2:172.21.15.16:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+12914 (secure 0 0 0) 0x7f90e802b020 con 0x7f910009f410 2023-09-30T22:29:19.819 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:29:19.819 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":22,"fsid":"7ede3e14-2bd0-4726-9437-6d13c90cd342","created":"2023-09-30T22:26:35.483697+0000","modified":"2023-09-30T22:29:16.486363+0000","last_up_change":"2023-09-30T22:26:44.227199+0000","last_in_change":"2023-09-30T22:26:37.120906+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"luminous","require_osd_release":"pacific","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T22:26:44.285133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{},"mgr_devicehealth":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:26:45.132921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:26:49.353683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:26:50.655685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"19946985-0ee2-487f-9347-929670dbe5ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6804","nonce":118824}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6805","nonce":118824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6807","nonce":118824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6806","nonce":118824}]},"public_addr":"172.21.15.16:6804/118824","cluster_addr":"172.21.15.16:6805/118824","heartbeat_back_addr":"172.21.15.16:6807/118824","heartbeat_front_addr":"172.21.15.16:6806/118824","state":["exists","up"]},{"osd":1,"uuid":"8efd4dbc-6ae4-4b82-b530-3b3eac2be445","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6812","nonce":118832}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6813","nonce":118832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6815","nonce":118832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6814","nonce":118832}]},"public_addr":"172.21.15.16:6812/118832","cluster_addr":"172.21.15.16:6813/118832","heartbeat_back_addr":"172.21.15.16:6815/118832","heartbeat_front_addr":"172.21.15.16:6814/118832","state":["exists","up"]},{"osd":2,"uuid":"b47bb1ce-27af-4c0c-868a-b76ebd0d9499","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6808","nonce":118840}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6809","nonce":118840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6811","nonce":118840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6810","nonce":118840}]},"public_addr":"172.21.15.16:6808/118840","cluster_addr":"172.21.15.16:6809/118840","heartbeat_back_addr":"172.21.15.16:6811/118840","heartbeat_front_addr":"172.21.15.16:6810/118840","state":["exists","up"]},{"osd":3,"uuid":"03b53d64-d10d-448a-a081-39ecceab589b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6800","nonce":118823}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6801","nonce":118823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6803","nonce":118823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6802","nonce":118823}]},"public_addr":"172.21.15.16:6800/118823","cluster_addr":"172.21.15.16:6801/118823","heartbeat_back_addr":"172.21.15.16:6803/118823","heartbeat_front_addr":"172.21.15.16:6802/118823","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.305953+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.307488+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.341023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.192071+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.8","mappings":[{"from":3,"to":2}]},{"pgid":"4.d","mappings":[{"from":3,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":0}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]},{"pgid":"4.17","mappings":[{"from":3,"to":2}]},{"pgid":"4.19","mappings":[{"from":3,"to":0}]},{"pgid":"4.3a","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6816/118285":"2023-10-01T22:29:14.475978+0000","172.21.15.16:0/2249754484":"2023-10-01T22:29:14.475978+0000","172.21.15.16:0/1101648925":"2023-10-01T22:29:14.475978+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:29:19.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.541+0000 7f9104e50700 1 -- 172.21.15.16:0/4167159472 >> v1:172.21.15.16:6816/126811 conn(0x7f90e002f480 legacy=0x7f90e0031930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:19.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.541+0000 7f9104e50700 1 -- 172.21.15.16:0/4167159472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f910009f410 msgr2=0x7f91001c8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:19.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.541+0000 7f9104e50700 1 --2- 172.21.15.16:0/4167159472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f910009f410 0x7f91001c8a10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f90e8009fd0 tx=0x7f90e8004750 comp rx=0 tx=0).stop 2023-09-30T22:29:19.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.541+0000 7f9104e50700 1 -- 172.21.15.16:0/4167159472 shutdown_connections 2023-09-30T22:29:19.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.541+0000 7f9104e50700 1 --2- 172.21.15.16:0/4167159472 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f910009e930 0x7f91001c84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:19.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.541+0000 7f9104e50700 1 --2- 172.21.15.16:0/4167159472 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9100136fe0 0x7f910006b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:19.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.541+0000 7f9104e50700 1 --2- 172.21.15.16:0/4167159472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f910009f410 0x7f91001c8a10 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:19.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.541+0000 7f9104e50700 1 -- 172.21.15.16:0/4167159472 >> 172.21.15.16:0/4167159472 conn(0x7f91000a4210 msgr2=0x7f9100134b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:19.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.542+0000 7f9104e50700 1 -- 172.21.15.16:0/4167159472 shutdown_connections 2023-09-30T22:29:19.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.542+0000 7f9104e50700 1 -- 172.21.15.16:0/4167159472 wait complete. 2023-09-30T22:29:19.822 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-30T22:26:44.285133+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}, 'mgr_devicehealth': {}}, 'read_balance': {'score_acting': 4, 'score_stable': 4, 'optimal_score': 0.5, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-09-30T22:26:45.132921+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-30T22:26:49.353683+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-09-30T22:26:50.655685+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-30T22:29:19.822 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-30T22:29:19.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.970+0000 7f6681e78700 1 Processor -- start 2023-09-30T22:29:19.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.970+0000 7f6681e78700 1 -- start start 2023-09-30T22:29:19.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.970+0000 7f6681e78700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f667c1303c0 0x7f667c130790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:19.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.970+0000 7f6681e78700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f667c06a770 0x7f667c06ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:19.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.970+0000 7f6681e78700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f667c09f620 con 0x7f667c06b0c0 2023-09-30T22:29:19.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.970+0000 7f6681e78700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f667c130280 con 0x7f667c06a770 2023-09-30T22:29:19.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.970+0000 7f6681e78700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f667c09e1b0 con 0x7f667c1303c0 2023-09-30T22:29:19.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.970+0000 7f667b7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f667c1303c0 0x7f667c130790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:19.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.970+0000 7f667affd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f667c06a770 0x7f667c06ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:19.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.970+0000 7f667b7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f667c1303c0 0x7f667c130790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37098/0 (socket says 172.21.15.16:37098) 2023-09-30T22:29:19.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.970+0000 7f667b7fe700 1 -- 172.21.15.16:0/332151840 learned_addr learned my addr 172.21.15.16:0/332151840 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:19.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.971+0000 7f667b7fe700 1 -- 172.21.15.16:0/332151840 >> v2:172.21.15.16:3300/0 conn(0x7f667c06a770 msgr2=0x7f667c06ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:19.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.971+0000 7f667b7fe700 1 --2- 172.21.15.16:0/332151840 >> v2:172.21.15.16:3300/0 conn(0x7f667c06a770 0x7f667c06ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:19.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.971+0000 7f667b7fe700 1 -- 172.21.15.16:0/332151840 >> v1:172.21.15.16:6789/0 conn(0x7f667c06b0c0 legacy=0x7f667c13a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:19.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.971+0000 7f667b7fe700 1 -- 172.21.15.16:0/332151840 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f667c06b540 con 0x7f667c1303c0 2023-09-30T22:29:19.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.971+0000 7f667affd700 1 --2- 172.21.15.16:0/332151840 >> v2:172.21.15.16:3300/0 conn(0x7f667c06a770 0x7f667c06ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:19.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.971+0000 7f667b7fe700 1 --2- 172.21.15.16:0/332151840 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f667c1303c0 0x7f667c130790 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f666c0098f0 tx=0x7f666c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bca8cb1ee8be62ab server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:19.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.971+0000 7f667a7fc700 1 -- 172.21.15.16:0/332151840 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f666c011830 con 0x7f667c1303c0 2023-09-30T22:29:19.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.971+0000 7f667a7fc700 1 -- 172.21.15.16:0/332151840 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f666c011990 con 0x7f667c1303c0 2023-09-30T22:29:19.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.971+0000 7f667a7fc700 1 -- 172.21.15.16:0/332151840 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f666c011d20 con 0x7f667c1303c0 2023-09-30T22:29:19.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.972+0000 7f6681e78700 1 -- 172.21.15.16:0/332151840 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f667c1303c0 msgr2=0x7f667c130790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:19.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.972+0000 7f6681e78700 1 --2- 172.21.15.16:0/332151840 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f667c1303c0 0x7f667c130790 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f666c0098f0 tx=0x7f666c009c00 comp rx=0 tx=0).stop 2023-09-30T22:29:19.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.972+0000 7f6681e78700 1 -- 172.21.15.16:0/332151840 shutdown_connections 2023-09-30T22:29:19.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.972+0000 7f6681e78700 1 --2- 172.21.15.16:0/332151840 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f667c1303c0 0x7f667c130790 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:19.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.972+0000 7f6681e78700 1 --2- 172.21.15.16:0/332151840 >> v2:172.21.15.16:3300/0 conn(0x7f667c06a770 0x7f667c06ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:19.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.972+0000 7f6681e78700 1 -- 172.21.15.16:0/332151840 >> 172.21.15.16:0/332151840 conn(0x7f667c066d70 msgr2=0x7f667c0a2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:19.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.972+0000 7f6681e78700 1 -- 172.21.15.16:0/332151840 shutdown_connections 2023-09-30T22:29:19.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.972+0000 7f6681e78700 1 -- 172.21.15.16:0/332151840 wait complete. 2023-09-30T22:29:19.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.973+0000 7f6681e78700 1 Processor -- start 2023-09-30T22:29:19.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.973+0000 7f6681e78700 1 -- start start 2023-09-30T22:29:19.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.973+0000 7f6681e78700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f667c06a770 0x7f667c1c8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:19.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.973+0000 7f6681e78700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f667c06b0c0 0x7f667c1c8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:19.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.974+0000 7f6681e78700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f667c11d930 0x7f667c11ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:19.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.974+0000 7f6681e78700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f667c072390 con 0x7f667c11d930 2023-09-30T22:29:19.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.974+0000 7f6681e78700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f667c1c93b0 con 0x7f667c06a770 2023-09-30T22:29:19.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.974+0000 7f6681e78700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f667c12d7f0 con 0x7f667c06b0c0 2023-09-30T22:29:19.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.974+0000 7f667b7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f667c06a770 0x7f667c1c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:19.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.974+0000 7f667b7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f667c06a770 0x7f667c1c8930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37114/0 (socket says 172.21.15.16:37114) 2023-09-30T22:29:19.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.974+0000 7f667b7fe700 1 -- 172.21.15.16:0/3735392164 learned_addr learned my addr 172.21.15.16:0/3735392164 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:19.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.974+0000 7f667affd700 1 --2- 172.21.15.16:0/3735392164 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f667c06b0c0 0x7f667c1c8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:19.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.974+0000 7f667bfff700 1 --2- 172.21.15.16:0/3735392164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f667c11d930 0x7f667c11ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:19.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.974+0000 7f667b7fe700 1 -- 172.21.15.16:0/3735392164 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f667c06b0c0 msgr2=0x7f667c1c8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:19.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.974+0000 7f667b7fe700 1 --2- 172.21.15.16:0/3735392164 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f667c06b0c0 0x7f667c1c8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:19.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.974+0000 7f667b7fe700 1 -- 172.21.15.16:0/3735392164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f667c11d930 msgr2=0x7f667c11ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:19.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.974+0000 7f667b7fe700 1 --2- 172.21.15.16:0/3735392164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f667c11d930 0x7f667c11ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:19.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.974+0000 7f667b7fe700 1 -- 172.21.15.16:0/3735392164 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66700094a0 con 0x7f667c06a770 2023-09-30T22:29:19.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.974+0000 7f667b7fe700 1 --2- 172.21.15.16:0/3735392164 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f667c06a770 0x7f667c1c8930 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f666c000c00 tx=0x7f666c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:19.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.974+0000 7f6663fff700 1 -- 172.21.15.16:0/3735392164 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f666c004340 con 0x7f667c06a770 2023-09-30T22:29:19.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.974+0000 7f6663fff700 1 -- 172.21.15.16:0/3735392164 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f666c01f070 con 0x7f667c06a770 2023-09-30T22:29:19.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.974+0000 7f6681e78700 1 -- 172.21.15.16:0/3735392164 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f666c009550 con 0x7f667c06a770 2023-09-30T22:29:19.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.974+0000 7f6663fff700 1 -- 172.21.15.16:0/3735392164 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f666c0036a0 con 0x7f667c06a770 2023-09-30T22:29:19.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.974+0000 7f6681e78700 1 -- 172.21.15.16:0/3735392164 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f667c11fdb0 con 0x7f667c06a770 2023-09-30T22:29:19.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.975+0000 7f6663fff700 1 -- 172.21.15.16:0/3735392164 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f666c021030 con 0x7f667c06a770 2023-09-30T22:29:19.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.975+0000 7f6663fff700 1 -- 172.21.15.16:0/3735392164 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4049+0+0 (secure 0 0 0) 0x7f666c040c60 con 0x7f667c06a770 2023-09-30T22:29:19.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.975+0000 7f6681e78700 1 -- 172.21.15.16:0/3735392164 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f667c130d00 con 0x7f667c06a770 2023-09-30T22:29:19.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:19.981+0000 7f6663fff700 1 -- 172.21.15.16:0/3735392164 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f666c015020 con 0x7f667c06a770 2023-09-30T22:29:20.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.180+0000 7f6681e78700 1 -- 172.21.15.16:0/3735392164 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f667c04f2a0 con 0x7f667c06a770 2023-09-30T22:29:20.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.181+0000 7f6663fff700 1 -- 172.21.15.16:0/3735392164 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f666c00b9d0 con 0x7f667c06a770 2023-09-30T22:29:20.181 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-09-30T22:29:20.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.182+0000 7f6681e78700 1 -- 172.21.15.16:0/3735392164 >> v1:172.21.15.16:6816/126811 conn(0x7f666802f540 legacy=0x7f66680319f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:20.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.182+0000 7f6681e78700 1 -- 172.21.15.16:0/3735392164 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f667c06a770 msgr2=0x7f667c1c8930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:20.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.182+0000 7f6681e78700 1 --2- 172.21.15.16:0/3735392164 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f667c06a770 0x7f667c1c8930 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f666c000c00 tx=0x7f666c004750 comp rx=0 tx=0).stop 2023-09-30T22:29:20.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.183+0000 7f6681e78700 1 -- 172.21.15.16:0/3735392164 shutdown_connections 2023-09-30T22:29:20.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.183+0000 7f6681e78700 1 --2- 172.21.15.16:0/3735392164 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f667c06a770 0x7f667c1c8930 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:20.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.183+0000 7f6681e78700 1 --2- 172.21.15.16:0/3735392164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f667c11d930 0x7f667c11ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:20.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.183+0000 7f6681e78700 1 --2- 172.21.15.16:0/3735392164 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f667c06b0c0 0x7f667c1c8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:20.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.183+0000 7f6681e78700 1 -- 172.21.15.16:0/3735392164 >> 172.21.15.16:0/3735392164 conn(0x7f667c066d70 msgr2=0x7f667c0a0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:20.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.183+0000 7f6681e78700 1 -- 172.21.15.16:0/3735392164 shutdown_connections 2023-09-30T22:29:20.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.183+0000 7f6681e78700 1 -- 172.21.15.16:0/3735392164 wait complete. 2023-09-30T22:29:20.196 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-09-30T22:29:20.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.335+0000 7fcc267bf700 1 Processor -- start 2023-09-30T22:29:20.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.335+0000 7fcc267bf700 1 -- start start 2023-09-30T22:29:20.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.335+0000 7fcc267bf700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc20131760 0x7fcc20131b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:20.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.335+0000 7fcc267bf700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcc2006b2d0 0x7fcc201395f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:20.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.335+0000 7fcc267bf700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc20131350 con 0x7fcc2006a770 2023-09-30T22:29:20.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.335+0000 7fcc267bf700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc2009ea40 con 0x7fcc2006b2d0 2023-09-30T22:29:20.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.335+0000 7fcc267bf700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2009dfc0 con 0x7fcc20131760 2023-09-30T22:29:20.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.336+0000 7fcc1ffff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc20131760 0x7fcc20131b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:20.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.336+0000 7fcc24d5c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcc2006b2d0 0x7fcc201395f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:20.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.336+0000 7fcc1ffff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc20131760 0x7fcc20131b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:55828/0 (socket says 172.21.15.16:55828) 2023-09-30T22:29:20.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.336+0000 7fcc24d5c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcc2006b2d0 0x7fcc201395f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57692/0 (socket says 172.21.15.16:57692) 2023-09-30T22:29:20.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.336+0000 7fcc1ffff700 1 -- 172.21.15.16:0/3211277301 learned_addr learned my addr 172.21.15.16:0/3211277301 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:20.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.336+0000 7fcc1ffff700 1 -- 172.21.15.16:0/3211277301 >> v2:172.21.15.16:3300/0 conn(0x7fcc2006b2d0 msgr2=0x7fcc201395f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:20.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.336+0000 7fcc1ffff700 1 --2- 172.21.15.16:0/3211277301 >> v2:172.21.15.16:3300/0 conn(0x7fcc2006b2d0 0x7fcc201395f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:20.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.336+0000 7fcc1ffff700 1 -- 172.21.15.16:0/3211277301 >> v1:172.21.15.16:6789/0 conn(0x7fcc2006a770 legacy=0x7fcc2006abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:20.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.336+0000 7fcc1ffff700 1 -- 172.21.15.16:0/3211277301 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc2009e210 con 0x7fcc20131760 2023-09-30T22:29:20.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.336+0000 7fcc24d5c700 1 --2- 172.21.15.16:0/3211277301 >> v2:172.21.15.16:3300/0 conn(0x7fcc2006b2d0 0x7fcc201395f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:20.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.336+0000 7fcc1ffff700 1 --2- 172.21.15.16:0/3211277301 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc20131760 0x7fcc20131b30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcc100098f0 tx=0x7fcc10009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39546537423166a6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:20.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.336+0000 7fcc1effd700 1 -- 172.21.15.16:0/3211277301 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc10011830 con 0x7fcc20131760 2023-09-30T22:29:20.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.336+0000 7fcc1effd700 1 -- 172.21.15.16:0/3211277301 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc10011990 con 0x7fcc20131760 2023-09-30T22:29:20.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.336+0000 7fcc1effd700 1 -- 172.21.15.16:0/3211277301 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc10011d20 con 0x7fcc20131760 2023-09-30T22:29:20.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.337+0000 7fcc267bf700 1 -- 172.21.15.16:0/3211277301 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc20131760 msgr2=0x7fcc20131b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:20.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.337+0000 7fcc267bf700 1 --2- 172.21.15.16:0/3211277301 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc20131760 0x7fcc20131b30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcc100098f0 tx=0x7fcc10009c00 comp rx=0 tx=0).stop 2023-09-30T22:29:20.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.337+0000 7fcc267bf700 1 -- 172.21.15.16:0/3211277301 shutdown_connections 2023-09-30T22:29:20.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.337+0000 7fcc267bf700 1 --2- 172.21.15.16:0/3211277301 >> v2:172.21.15.16:3300/0 conn(0x7fcc2006b2d0 0x7fcc201395f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:20.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.337+0000 7fcc267bf700 1 --2- 172.21.15.16:0/3211277301 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc20131760 0x7fcc20131b30 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:20.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.337+0000 7fcc267bf700 1 -- 172.21.15.16:0/3211277301 >> 172.21.15.16:0/3211277301 conn(0x7fcc20066d70 msgr2=0x7fcc200a2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:20.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.337+0000 7fcc267bf700 1 -- 172.21.15.16:0/3211277301 shutdown_connections 2023-09-30T22:29:20.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.337+0000 7fcc267bf700 1 -- 172.21.15.16:0/3211277301 wait complete. 2023-09-30T22:29:20.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.338+0000 7fcc267bf700 1 Processor -- start 2023-09-30T22:29:20.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.338+0000 7fcc267bf700 1 -- start start 2023-09-30T22:29:20.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.339+0000 7fcc267bf700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc2006a770 0x7fcc20119520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:20.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.339+0000 7fcc267bf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc2006b2d0 0x7fcc20119a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:20.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.339+0000 7fcc267bf700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcc20131760 0x7fcc20119fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:20.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.339+0000 7fcc267bf700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2011b280 con 0x7fcc2006b2d0 2023-09-30T22:29:20.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.339+0000 7fcc267bf700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2011c840 con 0x7fcc20131760 2023-09-30T22:29:20.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.339+0000 7fcc267bf700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2011a820 con 0x7fcc2006a770 2023-09-30T22:29:20.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.339+0000 7fcc1ffff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc2006a770 0x7fcc20119520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:20.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.339+0000 7fcc1f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc2006b2d0 0x7fcc20119a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:20.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.339+0000 7fcc1ffff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc2006a770 0x7fcc20119520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:55834/0 (socket says 172.21.15.16:55834) 2023-09-30T22:29:20.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.339+0000 7fcc1ffff700 1 -- 172.21.15.16:0/2889700956 learned_addr learned my addr 172.21.15.16:0/2889700956 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:20.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.339+0000 7fcc24d5c700 1 --2- 172.21.15.16:0/2889700956 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcc20131760 0x7fcc20119fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:20.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.339+0000 7fcc1ffff700 1 -- 172.21.15.16:0/2889700956 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcc20131760 msgr2=0x7fcc20119fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:20.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.339+0000 7fcc1ffff700 1 --2- 172.21.15.16:0/2889700956 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcc20131760 0x7fcc20119fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.339+0000 7fcc1ffff700 1 -- 172.21.15.16:0/2889700956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc2006b2d0 msgr2=0x7fcc20119a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.339+0000 7fcc1ffff700 1 --2- 172.21.15.16:0/2889700956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc2006b2d0 0x7fcc20119a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.339+0000 7fcc1ffff700 1 -- 172.21.15.16:0/2889700956 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc1800f050 con 0x7fcc2006a770 2023-09-30T22:29:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.339+0000 7fcc1f7fe700 1 --2- 172.21.15.16:0/2889700956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc2006b2d0 0x7fcc20119a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.339+0000 7fcc1ffff700 1 --2- 172.21.15.16:0/2889700956 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc2006a770 0x7fcc20119520 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcc10009c40 tx=0x7fcc10004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.340+0000 7fcc1cff9700 1 -- 172.21.15.16:0/2889700956 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc10004400 con 0x7fcc2006a770 2023-09-30T22:29:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.340+0000 7fcc1cff9700 1 -- 172.21.15.16:0/2889700956 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc1001f070 con 0x7fcc2006a770 2023-09-30T22:29:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.340+0000 7fcc267bf700 1 -- 172.21.15.16:0/2889700956 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc10009550 con 0x7fcc2006a770 2023-09-30T22:29:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.340+0000 7fcc1cff9700 1 -- 172.21.15.16:0/2889700956 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc100038e0 con 0x7fcc2006a770 2023-09-30T22:29:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.340+0000 7fcc267bf700 1 -- 172.21.15.16:0/2889700956 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc2012b9a0 con 0x7fcc2006a770 2023-09-30T22:29:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.340+0000 7fcc1cff9700 1 -- 172.21.15.16:0/2889700956 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fcc10021030 con 0x7fcc2006a770 2023-09-30T22:29:20.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.341+0000 7fcc1cff9700 1 -- 172.21.15.16:0/2889700956 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4049+0+0 (secure 0 0 0) 0x7fcc10040e40 con 0x7fcc2006a770 2023-09-30T22:29:20.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.341+0000 7fcc267bf700 1 -- 172.21.15.16:0/2889700956 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc2004f2a0 con 0x7fcc2006a770 2023-09-30T22:29:20.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.346+0000 7fcc1cff9700 1 -- 172.21.15.16:0/2889700956 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcc10015020 con 0x7fcc2006a770 2023-09-30T22:29:20.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.549+0000 7fcc267bf700 1 -- 172.21.15.16:0/2889700956 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fcc2009e5e0 con 0x7fcc2006a770 2023-09-30T22:29:20.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.549+0000 7fcc1cff9700 1 -- 172.21.15.16:0/2889700956 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7fcc10004560 con 0x7fcc2006a770 2023-09-30T22:29:20.550 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 8 2023-09-30T22:29:20.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.551+0000 7fcc267bf700 1 -- 172.21.15.16:0/2889700956 >> v1:172.21.15.16:6816/126811 conn(0x7fcc0c02f510 legacy=0x7fcc0c0319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:20.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.551+0000 7fcc267bf700 1 -- 172.21.15.16:0/2889700956 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc2006a770 msgr2=0x7fcc20119520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:20.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.551+0000 7fcc267bf700 1 --2- 172.21.15.16:0/2889700956 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc2006a770 0x7fcc20119520 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcc10009c40 tx=0x7fcc10004750 comp rx=0 tx=0).stop 2023-09-30T22:29:20.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.551+0000 7fcc267bf700 1 -- 172.21.15.16:0/2889700956 shutdown_connections 2023-09-30T22:29:20.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.551+0000 7fcc267bf700 1 --2- 172.21.15.16:0/2889700956 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcc20131760 0x7fcc20119fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:20.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.551+0000 7fcc267bf700 1 --2- 172.21.15.16:0/2889700956 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc2006a770 0x7fcc20119520 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:20.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.551+0000 7fcc267bf700 1 --2- 172.21.15.16:0/2889700956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc2006b2d0 0x7fcc20119a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:20.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.551+0000 7fcc267bf700 1 -- 172.21.15.16:0/2889700956 >> 172.21.15.16:0/2889700956 conn(0x7fcc20066d70 msgr2=0x7fcc200a0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:20.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.551+0000 7fcc267bf700 1 -- 172.21.15.16:0/2889700956 shutdown_connections 2023-09-30T22:29:20.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.551+0000 7fcc267bf700 1 -- 172.21.15.16:0/2889700956 wait complete. 2023-09-30T22:29:20.566 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-30T22:29:20.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.706+0000 7fc22434f700 1 Processor -- start 2023-09-30T22:29:20.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.706+0000 7fc22434f700 1 -- start start 2023-09-30T22:29:20.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.707+0000 7fc22434f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc21c12e180 0x7fc21c12e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:20.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.707+0000 7fc22434f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc21c06a770 0x7fc21c06ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:20.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.707+0000 7fc22434f700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc21c12df80 con 0x7fc21c06b0a0 2023-09-30T22:29:20.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.707+0000 7fc22434f700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc21c09e640 con 0x7fc21c12e180 2023-09-30T22:29:20.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.707+0000 7fc22434f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc21c09f7c0 con 0x7fc21c06a770 2023-09-30T22:29:20.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.707+0000 7fc2218ea700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc21c06a770 0x7fc21c06ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:20.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.707+0000 7fc2220eb700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc21c12e180 0x7fc21c12e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:20.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.707+0000 7fc2228ec700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc21c06b0a0 0x7fc21c138580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33830/0 (socket says 172.21.15.16:33830) 2023-09-30T22:29:20.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.707+0000 7fc2228ec700 1 -- 172.21.15.16:0/2608320282 learned_addr learned my addr 172.21.15.16:0/2608320282 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:29:20.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.708+0000 7fc2220eb700 1 -- 172.21.15.16:0/2608320282 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc21c06a770 msgr2=0x7fc21c06ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:20.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.708+0000 7fc2220eb700 1 --2- 172.21.15.16:0/2608320282 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc21c06a770 0x7fc21c06ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:20.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.708+0000 7fc2220eb700 1 -- 172.21.15.16:0/2608320282 >> v1:172.21.15.16:6789/0 conn(0x7fc21c06b0a0 legacy=0x7fc21c138580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:20.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.708+0000 7fc2220eb700 1 -- 172.21.15.16:0/2608320282 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc21c09e900 con 0x7fc21c12e180 2023-09-30T22:29:20.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.708+0000 7fc2210e9700 1 -- 172.21.15.16:0/2608320282 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3969989957 0 0) 0x7fc21c12df80 con 0x7fc21c06b0a0 2023-09-30T22:29:20.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.708+0000 7fc2218ea700 1 --2- 172.21.15.16:0/2608320282 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc21c06a770 0x7fc21c06ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:20.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.708+0000 7fc2220eb700 1 --2- 172.21.15.16:0/2608320282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc21c12e180 0x7fc21c12e550 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc218009840 tx=0x7fc218009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a608d80285215c4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:20.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.708+0000 7fc2210e9700 1 -- 172.21.15.16:0/2608320282 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc218011830 con 0x7fc21c12e180 2023-09-30T22:29:20.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.708+0000 7fc2210e9700 1 -- 172.21.15.16:0/2608320282 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc218011990 con 0x7fc21c12e180 2023-09-30T22:29:20.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.708+0000 7fc2210e9700 1 -- 172.21.15.16:0/2608320282 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc218011d20 con 0x7fc21c12e180 2023-09-30T22:29:20.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.709+0000 7fc22434f700 1 -- 172.21.15.16:0/2608320282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc21c12e180 msgr2=0x7fc21c12e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:20.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.709+0000 7fc22434f700 1 --2- 172.21.15.16:0/2608320282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc21c12e180 0x7fc21c12e550 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc218009840 tx=0x7fc218009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:20.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.709+0000 7fc22434f700 1 -- 172.21.15.16:0/2608320282 shutdown_connections 2023-09-30T22:29:20.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.709+0000 7fc22434f700 1 --2- 172.21.15.16:0/2608320282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc21c12e180 0x7fc21c12e550 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:20.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.709+0000 7fc22434f700 1 --2- 172.21.15.16:0/2608320282 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc21c06a770 0x7fc21c06ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:20.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.709+0000 7fc22434f700 1 -- 172.21.15.16:0/2608320282 >> 172.21.15.16:0/2608320282 conn(0x7fc21c066d70 msgr2=0x7fc21c0a2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:20.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.709+0000 7fc22434f700 1 -- 172.21.15.16:0/2608320282 shutdown_connections 2023-09-30T22:29:20.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.709+0000 7fc22434f700 1 -- 172.21.15.16:0/2608320282 wait complete. 2023-09-30T22:29:20.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.710+0000 7fc22434f700 1 Processor -- start 2023-09-30T22:29:20.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.710+0000 7fc22434f700 1 -- start start 2023-09-30T22:29:20.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.711+0000 7fc22434f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc21c06a770 0x7fc21c11a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:20.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.711+0000 7fc22434f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc21c06b0a0 0x7fc21c11a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:20.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.711+0000 7fc22434f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc21c12e180 0x7fc21c11abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:20.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.711+0000 7fc22434f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc21c11cb50 con 0x7fc21c06b0a0 2023-09-30T22:29:20.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.711+0000 7fc22434f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc21c11c620 con 0x7fc21c06a770 2023-09-30T22:29:20.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.711+0000 7fc22434f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc21c11bd50 con 0x7fc21c12e180 2023-09-30T22:29:20.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.711+0000 7fc2220eb700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc21c06a770 0x7fc21c11a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:20.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.711+0000 7fc2228ec700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc21c12e180 0x7fc21c11abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:20.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.711+0000 7fc2228ec700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc21c12e180 0x7fc21c11abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:55856/0 (socket says 172.21.15.16:55856) 2023-09-30T22:29:20.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.711+0000 7fc2228ec700 1 -- 172.21.15.16:0/3820830129 learned_addr learned my addr 172.21.15.16:0/3820830129 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:20.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.711+0000 7fc2220eb700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc21c06a770 0x7fc21c11a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37136/0 (socket says 172.21.15.16:37136) 2023-09-30T22:29:20.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.711+0000 7fc2218ea700 1 --2- 172.21.15.16:0/3820830129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc21c06b0a0 0x7fc21c11a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:20.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.711+0000 7fc2220eb700 1 -- 172.21.15.16:0/3820830129 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc21c12e180 msgr2=0x7fc21c11abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:20.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.711+0000 7fc2220eb700 1 --2- 172.21.15.16:0/3820830129 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc21c12e180 0x7fc21c11abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:20.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.711+0000 7fc2220eb700 1 -- 172.21.15.16:0/3820830129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc21c06b0a0 msgr2=0x7fc21c11a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:20.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.711+0000 7fc2220eb700 1 --2- 172.21.15.16:0/3820830129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc21c06b0a0 0x7fc21c11a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:20.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.711+0000 7fc2220eb700 1 -- 172.21.15.16:0/3820830129 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc20c0094a0 con 0x7fc21c06a770 2023-09-30T22:29:20.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.711+0000 7fc2228ec700 1 --2- 172.21.15.16:0/3820830129 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc21c12e180 0x7fc21c11abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:20.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.712+0000 7fc2220eb700 1 --2- 172.21.15.16:0/3820830129 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc21c06a770 0x7fc21c11a140 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc218016000 tx=0x7fc218004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:20.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.712+0000 7fc212ffd700 1 -- 172.21.15.16:0/3820830129 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2180036a0 con 0x7fc21c06a770 2023-09-30T22:29:20.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.712+0000 7fc212ffd700 1 -- 172.21.15.16:0/3820830129 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc21801f070 con 0x7fc21c06a770 2023-09-30T22:29:20.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.712+0000 7fc22434f700 1 -- 172.21.15.16:0/3820830129 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2180094a0 con 0x7fc21c06a770 2023-09-30T22:29:20.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.712+0000 7fc212ffd700 1 -- 172.21.15.16:0/3820830129 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc218003a20 con 0x7fc21c06a770 2023-09-30T22:29:20.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.712+0000 7fc22434f700 1 -- 172.21.15.16:0/3820830129 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc21c12b840 con 0x7fc21c06a770 2023-09-30T22:29:20.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.712+0000 7fc212ffd700 1 -- 172.21.15.16:0/3820830129 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fc218003b80 con 0x7fc21c06a770 2023-09-30T22:29:20.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.713+0000 7fc212ffd700 1 -- 172.21.15.16:0/3820830129 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4049+0+0 (secure 0 0 0) 0x7fc218021070 con 0x7fc21c06a770 2023-09-30T22:29:20.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.713+0000 7fc22434f700 1 -- 172.21.15.16:0/3820830129 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc21c04f2a0 con 0x7fc21c06a770 2023-09-30T22:29:20.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.718+0000 7fc212ffd700 1 -- 172.21.15.16:0/3820830129 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc2180253e0 con 0x7fc21c06a770 2023-09-30T22:29:20.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.919+0000 7fc22434f700 1 -- 172.21.15.16:0/3820830129 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc21c09e210 con 0x7fc21c06a770 2023-09-30T22:29:20.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.919+0000 7fc212ffd700 1 -- 172.21.15.16:0/3820830129 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc218010b70 con 0x7fc21c06a770 2023-09-30T22:29:20.920 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-09-30T22:29:20.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.921+0000 7fc22434f700 1 -- 172.21.15.16:0/3820830129 >> v1:172.21.15.16:6816/126811 conn(0x7fc20002f510 legacy=0x7fc2000319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:20.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.921+0000 7fc22434f700 1 -- 172.21.15.16:0/3820830129 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc21c06a770 msgr2=0x7fc21c11a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:20.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.921+0000 7fc22434f700 1 --2- 172.21.15.16:0/3820830129 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc21c06a770 0x7fc21c11a140 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc218016000 tx=0x7fc218004750 comp rx=0 tx=0).stop 2023-09-30T22:29:20.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.921+0000 7fc22434f700 1 -- 172.21.15.16:0/3820830129 shutdown_connections 2023-09-30T22:29:20.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.921+0000 7fc22434f700 1 --2- 172.21.15.16:0/3820830129 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc21c06a770 0x7fc21c11a140 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:20.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.921+0000 7fc22434f700 1 --2- 172.21.15.16:0/3820830129 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc21c12e180 0x7fc21c11abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:20.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.921+0000 7fc22434f700 1 --2- 172.21.15.16:0/3820830129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc21c06b0a0 0x7fc21c11a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:20.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.921+0000 7fc22434f700 1 -- 172.21.15.16:0/3820830129 >> 172.21.15.16:0/3820830129 conn(0x7fc21c066d70 msgr2=0x7fc21c0a1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:20.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.922+0000 7fc22434f700 1 -- 172.21.15.16:0/3820830129 shutdown_connections 2023-09-30T22:29:20.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:20.922+0000 7fc22434f700 1 -- 172.21.15.16:0/3820830129 wait complete. 2023-09-30T22:29:20.935 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-30T22:29:21.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.074+0000 7fe41c118700 1 Processor -- start 2023-09-30T22:29:21.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.074+0000 7fe41c118700 1 -- start start 2023-09-30T22:29:21.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.075+0000 7fe41c118700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe414131c20 0x7fe414131ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:21.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.075+0000 7fe41c118700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe41409bc50 0x7fe41409c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:21.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.075+0000 7fe41c118700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe41409c5a0 0x7fe414138ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:21.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.075+0000 7fe41c118700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe41409ec00 con 0x7fe41409bc50 2023-09-30T22:29:21.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.075+0000 7fe41c118700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe41406ee30 con 0x7fe41409c5a0 2023-09-30T22:29:21.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.075+0000 7fe41c118700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe414131780 con 0x7fe414131c20 2023-09-30T22:29:21.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.075+0000 7fe419eb4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe414131c20 0x7fe414131ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:21.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.075+0000 7fe4196b3700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe41409bc50 0x7fe41409c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:21.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.075+0000 7fe41a6b5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe41409c5a0 0x7fe414138ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:21.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.075+0000 7fe4196b3700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe41409bc50 0x7fe41409c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57718/0 (socket says 172.21.15.16:57718) 2023-09-30T22:29:21.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.075+0000 7fe41a6b5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe41409c5a0 0x7fe414138ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37150/0 (socket says 172.21.15.16:37150) 2023-09-30T22:29:21.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.075+0000 7fe419eb4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe414131c20 0x7fe414131ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:55868/0 (socket says 172.21.15.16:55868) 2023-09-30T22:29:21.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.075+0000 7fe4196b3700 1 -- 172.21.15.16:0/2874866142 learned_addr learned my addr 172.21.15.16:0/2874866142 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:21.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.075+0000 7fe419eb4700 1 -- 172.21.15.16:0/2874866142 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe41409c5a0 msgr2=0x7fe414138ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:21.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.075+0000 7fe419eb4700 1 --2- 172.21.15.16:0/2874866142 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe41409c5a0 0x7fe414138ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:21.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.075+0000 7fe419eb4700 1 -- 172.21.15.16:0/2874866142 >> v2:172.21.15.16:3300/0 conn(0x7fe41409bc50 msgr2=0x7fe41409c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:21.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.075+0000 7fe419eb4700 1 --2- 172.21.15.16:0/2874866142 >> v2:172.21.15.16:3300/0 conn(0x7fe41409bc50 0x7fe41409c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:21.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.075+0000 7fe419eb4700 1 -- 172.21.15.16:0/2874866142 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe414131a20 con 0x7fe414131c20 2023-09-30T22:29:21.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.075+0000 7fe4196b3700 1 --2- 172.21.15.16:0/2874866142 >> v2:172.21.15.16:3300/0 conn(0x7fe41409bc50 0x7fe41409c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:21.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.075+0000 7fe419eb4700 1 --2- 172.21.15.16:0/2874866142 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe414131c20 0x7fe414131ff0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe408009840 tx=0x7fe408009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a171a631d3050be server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:21.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.076+0000 7fe418eb2700 1 -- 172.21.15.16:0/2874866142 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe408011830 con 0x7fe414131c20 2023-09-30T22:29:21.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.076+0000 7fe418eb2700 1 -- 172.21.15.16:0/2874866142 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe408011990 con 0x7fe414131c20 2023-09-30T22:29:21.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.076+0000 7fe418eb2700 1 -- 172.21.15.16:0/2874866142 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe408011d20 con 0x7fe414131c20 2023-09-30T22:29:21.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.076+0000 7fe41c118700 1 -- 172.21.15.16:0/2874866142 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe414131c20 msgr2=0x7fe414131ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:21.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.076+0000 7fe41c118700 1 --2- 172.21.15.16:0/2874866142 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe414131c20 0x7fe414131ff0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe408009840 tx=0x7fe408009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:21.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.076+0000 7fe41c118700 1 -- 172.21.15.16:0/2874866142 shutdown_connections 2023-09-30T22:29:21.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.076+0000 7fe41c118700 1 --2- 172.21.15.16:0/2874866142 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe41409c5a0 0x7fe414138ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:21.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.076+0000 7fe41c118700 1 --2- 172.21.15.16:0/2874866142 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe414131c20 0x7fe414131ff0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:21.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.076+0000 7fe41c118700 1 --2- 172.21.15.16:0/2874866142 >> v2:172.21.15.16:3300/0 conn(0x7fe41409bc50 0x7fe41409c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:21.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.076+0000 7fe41c118700 1 -- 172.21.15.16:0/2874866142 >> 172.21.15.16:0/2874866142 conn(0x7fe4140a2c10 msgr2=0x7fe4140a3010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:21.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.076+0000 7fe41c118700 1 -- 172.21.15.16:0/2874866142 shutdown_connections 2023-09-30T22:29:21.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.076+0000 7fe41c118700 1 -- 172.21.15.16:0/2874866142 wait complete. 2023-09-30T22:29:21.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.077+0000 7fe41c118700 1 Processor -- start 2023-09-30T22:29:21.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.077+0000 7fe41c118700 1 -- start start 2023-09-30T22:29:21.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.078+0000 7fe41c118700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe41409bc50 0x7fe414099a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:21.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.078+0000 7fe41c118700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe41409c5a0 0x7fe414099f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:21.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.078+0000 7fe41c118700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe414131c20 0x7fe41409a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:21.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.078+0000 7fe41c118700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe41411dd50 con 0x7fe414131c20 2023-09-30T22:29:21.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.078+0000 7fe41c118700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe41412c6e0 con 0x7fe41409c5a0 2023-09-30T22:29:21.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.078+0000 7fe41c118700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe41411d680 con 0x7fe41409bc50 2023-09-30T22:29:21.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.078+0000 7fe419eb4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe41409bc50 0x7fe414099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:21.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.078+0000 7fe41a6b5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe414131c20 0x7fe41409a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:21.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.078+0000 7fe419eb4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe41409bc50 0x7fe414099a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:55870/0 (socket says 172.21.15.16:55870) 2023-09-30T22:29:21.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.078+0000 7fe419eb4700 1 -- 172.21.15.16:0/1728336246 learned_addr learned my addr 172.21.15.16:0/1728336246 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:21.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.078+0000 7fe41a6b5700 1 --2- 172.21.15.16:0/1728336246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe414131c20 0x7fe41409a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57730/0 (socket says 172.21.15.16:57730) 2023-09-30T22:29:21.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.078+0000 7fe4196b3700 1 --2- 172.21.15.16:0/1728336246 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe41409c5a0 0x7fe414099f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:21.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.078+0000 7fe41a6b5700 1 -- 172.21.15.16:0/1728336246 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe41409bc50 msgr2=0x7fe414099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:21.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.078+0000 7fe41a6b5700 1 --2- 172.21.15.16:0/1728336246 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe41409bc50 0x7fe414099a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:21.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.078+0000 7fe41a6b5700 1 -- 172.21.15.16:0/1728336246 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe41409c5a0 msgr2=0x7fe414099f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:21.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.078+0000 7fe41a6b5700 1 --2- 172.21.15.16:0/1728336246 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe41409c5a0 0x7fe414099f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:21.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.078+0000 7fe41a6b5700 1 -- 172.21.15.16:0/1728336246 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe410009550 con 0x7fe414131c20 2023-09-30T22:29:21.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.078+0000 7fe419eb4700 1 --2- 172.21.15.16:0/1728336246 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe41409bc50 0x7fe414099a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:21.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.078+0000 7fe4196b3700 1 --2- 172.21.15.16:0/1728336246 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe41409c5a0 0x7fe414099f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:21.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.079+0000 7fe41a6b5700 1 --2- 172.21.15.16:0/1728336246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe414131c20 0x7fe41409a4c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe40c00efc0 tx=0x7fe40c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:21.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.079+0000 7fe4067fc700 1 -- 172.21.15.16:0/1728336246 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe40c00cc90 con 0x7fe414131c20 2023-09-30T22:29:21.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.079+0000 7fe4067fc700 1 -- 172.21.15.16:0/1728336246 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe40c00f040 con 0x7fe414131c20 2023-09-30T22:29:21.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.079+0000 7fe41c118700 1 -- 172.21.15.16:0/1728336246 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4080094a0 con 0x7fe414131c20 2023-09-30T22:29:21.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.079+0000 7fe4067fc700 1 -- 172.21.15.16:0/1728336246 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe40c0146c0 con 0x7fe414131c20 2023-09-30T22:29:21.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.079+0000 7fe41c118700 1 -- 172.21.15.16:0/1728336246 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe41411e010 con 0x7fe414131c20 2023-09-30T22:29:21.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.080+0000 7fe4067fc700 1 -- 172.21.15.16:0/1728336246 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fe40c014820 con 0x7fe414131c20 2023-09-30T22:29:21.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.080+0000 7fe41c118700 1 -- 172.21.15.16:0/1728336246 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe414130e20 con 0x7fe414131c20 2023-09-30T22:29:21.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.080+0000 7fe4067fc700 1 -- 172.21.15.16:0/1728336246 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4049+0+0 (secure 0 0 0) 0x7fe40c040780 con 0x7fe414131c20 2023-09-30T22:29:21.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.085+0000 7fe4067fc700 1 -- 172.21.15.16:0/1728336246 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe40c01d400 con 0x7fe414131c20 2023-09-30T22:29:21.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.287+0000 7fe41c118700 1 -- 172.21.15.16:0/1728336246 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe41409b850 con 0x7fe414131c20 2023-09-30T22:29:21.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.288+0000 7fe4067fc700 1 -- 172.21.15.16:0/1728336246 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe40c007330 con 0x7fe414131c20 2023-09-30T22:29:21.288 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-09-30T22:29:21.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.290+0000 7fe41c118700 1 -- 172.21.15.16:0/1728336246 >> v1:172.21.15.16:6816/126811 conn(0x7fe3f802f510 legacy=0x7fe3f80319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:21.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.290+0000 7fe41c118700 1 -- 172.21.15.16:0/1728336246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe414131c20 msgr2=0x7fe41409a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:21.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.290+0000 7fe41c118700 1 --2- 172.21.15.16:0/1728336246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe414131c20 0x7fe41409a4c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe40c00efc0 tx=0x7fe40c00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:29:21.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.290+0000 7fe41c118700 1 -- 172.21.15.16:0/1728336246 shutdown_connections 2023-09-30T22:29:21.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.290+0000 7fe41c118700 1 --2- 172.21.15.16:0/1728336246 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe414131c20 0x7fe41409a4c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:21.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.290+0000 7fe41c118700 1 --2- 172.21.15.16:0/1728336246 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe41409bc50 0x7fe414099a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:21.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.290+0000 7fe41c118700 1 --2- 172.21.15.16:0/1728336246 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe41409c5a0 0x7fe414099f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:21.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.290+0000 7fe41c118700 1 -- 172.21.15.16:0/1728336246 >> 172.21.15.16:0/1728336246 conn(0x7fe4140a2c10 msgr2=0x7fe4141b8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:21.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.290+0000 7fe41c118700 1 -- 172.21.15.16:0/1728336246 shutdown_connections 2023-09-30T22:29:21.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.290+0000 7fe41c118700 1 -- 172.21.15.16:0/1728336246 wait complete. 2023-09-30T22:29:21.304 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-30T22:29:21.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.439+0000 7f27a8ed5700 1 Processor -- start 2023-09-30T22:29:21.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.439+0000 7f27a8ed5700 1 -- start start 2023-09-30T22:29:21.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.440+0000 7f27a8ed5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f27a409d6f0 0x7f27a409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:21.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.440+0000 7f27a8ed5700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f27a409e410 0x7f27a4134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:21.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.440+0000 7f27a8ed5700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27a409b120 con 0x7f27a409e000 2023-09-30T22:29:21.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.440+0000 7f27a8ed5700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27a406b460 con 0x7f27a409e410 2023-09-30T22:29:21.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.440+0000 7f27a8ed5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f27a409a7e0 con 0x7f27a409d6f0 2023-09-30T22:29:21.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.440+0000 7f27a2d9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f27a409e410 0x7f27a4134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:21.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.440+0000 7f27a259c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f27a409d6f0 0x7f27a409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:21.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.440+0000 7f27a1d9b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f27a409e000 0x7f27a41339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33832/0 (socket says 172.21.15.16:33832) 2023-09-30T22:29:21.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.440+0000 7f27a1d9b700 1 -- 172.21.15.16:0/1124286847 learned_addr learned my addr 172.21.15.16:0/1124286847 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:29:21.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.440+0000 7f27a2d9d700 1 -- 172.21.15.16:0/1124286847 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f27a409d6f0 msgr2=0x7f27a409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:21.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.440+0000 7f27a2d9d700 1 --2- 172.21.15.16:0/1124286847 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f27a409d6f0 0x7f27a409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:21.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.441+0000 7f27a2d9d700 1 -- 172.21.15.16:0/1124286847 >> v1:172.21.15.16:6789/0 conn(0x7f27a409e000 legacy=0x7f27a41339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:21.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.441+0000 7f27a159a700 1 -- 172.21.15.16:0/1124286847 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3903942804 0 0) 0x7f27a409b120 con 0x7f27a409e000 2023-09-30T22:29:21.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.441+0000 7f27a2d9d700 1 -- 172.21.15.16:0/1124286847 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27a409a500 con 0x7f27a409e410 2023-09-30T22:29:21.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.441+0000 7f27a259c700 1 --2- 172.21.15.16:0/1124286847 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f27a409d6f0 0x7f27a409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:21.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.441+0000 7f27a2d9d700 1 --2- 172.21.15.16:0/1124286847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27a409e410 0x7f27a4134100 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f279c009d90 tx=0x7f279c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=236f464daf24a28c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:21.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.441+0000 7f27a159a700 1 -- 172.21.15.16:0/1124286847 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279c00e070 con 0x7f27a409e410 2023-09-30T22:29:21.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.441+0000 7f27a159a700 1 -- 172.21.15.16:0/1124286847 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f279c00bb20 con 0x7f27a409e410 2023-09-30T22:29:21.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.441+0000 7f27a159a700 1 -- 172.21.15.16:0/1124286847 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279c012430 con 0x7f27a409e410 2023-09-30T22:29:21.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.441+0000 7f27a8ed5700 1 -- 172.21.15.16:0/1124286847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27a409e410 msgr2=0x7f27a4134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:21.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.441+0000 7f27a8ed5700 1 --2- 172.21.15.16:0/1124286847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27a409e410 0x7f27a4134100 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f279c009d90 tx=0x7f279c00b330 comp rx=0 tx=0).stop 2023-09-30T22:29:21.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.442+0000 7f27a8ed5700 1 -- 172.21.15.16:0/1124286847 shutdown_connections 2023-09-30T22:29:21.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.442+0000 7f27a8ed5700 1 --2- 172.21.15.16:0/1124286847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27a409e410 0x7f27a4134100 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:21.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.442+0000 7f27a8ed5700 1 --2- 172.21.15.16:0/1124286847 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f27a409d6f0 0x7f27a409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:21.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.442+0000 7f27a8ed5700 1 -- 172.21.15.16:0/1124286847 >> 172.21.15.16:0/1124286847 conn(0x7f27a4066f50 msgr2=0x7f27a409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:21.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.442+0000 7f27a8ed5700 1 -- 172.21.15.16:0/1124286847 shutdown_connections 2023-09-30T22:29:21.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.442+0000 7f27a8ed5700 1 -- 172.21.15.16:0/1124286847 wait complete. 2023-09-30T22:29:21.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.443+0000 7f27a8ed5700 1 Processor -- start 2023-09-30T22:29:21.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.443+0000 7f27a8ed5700 1 -- start start 2023-09-30T22:29:21.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.443+0000 7f27a8ed5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27a409d6f0 0x7f27a41c8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:21.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.443+0000 7f27a8ed5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f27a409e000 0x7f27a41c8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:21.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.443+0000 7f27a8ed5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f27a409e410 0x7f27a41c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:21.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.443+0000 7f27a8ed5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27a41a8a30 con 0x7f27a409d6f0 2023-09-30T22:29:21.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.443+0000 7f27a8ed5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27a41a9780 con 0x7f27a409e410 2023-09-30T22:29:21.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.443+0000 7f27a8ed5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f27a41c8e80 con 0x7f27a409e000 2023-09-30T22:29:21.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.443+0000 7f27a1d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f27a409e000 0x7f27a41c8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:21.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.444+0000 7f27a2d9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f27a409e410 0x7f27a41c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:21.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.444+0000 7f27a1d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f27a409e000 0x7f27a41c8940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:55886/0 (socket says 172.21.15.16:55886) 2023-09-30T22:29:21.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.444+0000 7f27a259c700 1 --2- 172.21.15.16:0/2148035689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27a409d6f0 0x7f27a41c8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:21.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.444+0000 7f27a1d9b700 1 -- 172.21.15.16:0/2148035689 learned_addr learned my addr 172.21.15.16:0/2148035689 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:21.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.444+0000 7f27a1d9b700 1 -- 172.21.15.16:0/2148035689 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f27a409e410 msgr2=0x7f27a41c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:21.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.444+0000 7f27a1d9b700 1 --2- 172.21.15.16:0/2148035689 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f27a409e410 0x7f27a41c49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:21.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.444+0000 7f27a1d9b700 1 -- 172.21.15.16:0/2148035689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27a409d6f0 msgr2=0x7f27a41c8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:21.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.444+0000 7f27a1d9b700 1 --2- 172.21.15.16:0/2148035689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27a409d6f0 0x7f27a41c8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:21.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.444+0000 7f27a1d9b700 1 -- 172.21.15.16:0/2148035689 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27940094a0 con 0x7f27a409e000 2023-09-30T22:29:21.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.444+0000 7f27a2d9d700 1 --2- 172.21.15.16:0/2148035689 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f27a409e410 0x7f27a41c49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:21.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.444+0000 7f27a259c700 1 --2- 172.21.15.16:0/2148035689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27a409d6f0 0x7f27a41c8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:21.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.444+0000 7f27a1d9b700 1 --2- 172.21.15.16:0/2148035689 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f27a409e000 0x7f27a41c8940 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f278c00c520 tx=0x7f278c00c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:21.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.444+0000 7f279affd700 1 -- 172.21.15.16:0/2148035689 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f278c004c80 con 0x7f27a409e000 2023-09-30T22:29:21.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.444+0000 7f279affd700 1 -- 172.21.15.16:0/2148035689 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f278c004de0 con 0x7f27a409e000 2023-09-30T22:29:21.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.444+0000 7f27a8ed5700 1 -- 172.21.15.16:0/2148035689 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f279c009a70 con 0x7f27a409e000 2023-09-30T22:29:21.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.444+0000 7f279affd700 1 -- 172.21.15.16:0/2148035689 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f278c007310 con 0x7f27a409e000 2023-09-30T22:29:21.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.444+0000 7f27a8ed5700 1 -- 172.21.15.16:0/2148035689 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27a41a85e0 con 0x7f27a409e000 2023-09-30T22:29:21.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.445+0000 7f27a8ed5700 1 -- 172.21.15.16:0/2148035689 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27a41a9240 con 0x7f27a409e000 2023-09-30T22:29:21.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.446+0000 7f279affd700 1 -- 172.21.15.16:0/2148035689 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f278c007470 con 0x7f27a409e000 2023-09-30T22:29:21.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.446+0000 7f279affd700 1 -- 172.21.15.16:0/2148035689 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4049+0+0 (secure 0 0 0) 0x7f278c03fb90 con 0x7f27a409e000 2023-09-30T22:29:21.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.450+0000 7f279affd700 1 -- 172.21.15.16:0/2148035689 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f278c0102c0 con 0x7f27a409e000 2023-09-30T22:29:21.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.695+0000 7f27a8ed5700 1 -- 172.21.15.16:0/2148035689 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f27a41a9240 con 0x7f27a409e000 2023-09-30T22:29:21.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.696+0000 7f279affd700 1 -- 172.21.15.16:0/2148035689 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+2315 (secure 0 0 0) 0x7f27a41a9240 con 0x7f27a409e000 2023-09-30T22:29:21.696 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:29:21.696 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-30T22:26:51.674561+0000","modified":"2023-09-30T22:27:18.263912+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4218},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4218":{"gid":4218,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6817/1424366610","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6817","nonce":1424366610}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-30T22:29:21.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.698+0000 7f27a8ed5700 1 -- 172.21.15.16:0/2148035689 >> v1:172.21.15.16:6816/126811 conn(0x7f278402f510 legacy=0x7f27840319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:21.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.698+0000 7f27a8ed5700 1 -- 172.21.15.16:0/2148035689 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f27a409e000 msgr2=0x7f27a41c8940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:21.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.698+0000 7f27a8ed5700 1 --2- 172.21.15.16:0/2148035689 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f27a409e000 0x7f27a41c8940 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f278c00c520 tx=0x7f278c00c830 comp rx=0 tx=0).stop 2023-09-30T22:29:21.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.698+0000 7f27a8ed5700 1 -- 172.21.15.16:0/2148035689 shutdown_connections 2023-09-30T22:29:21.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.698+0000 7f27a8ed5700 1 --2- 172.21.15.16:0/2148035689 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f27a409e410 0x7f27a41c49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:21.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.698+0000 7f27a8ed5700 1 --2- 172.21.15.16:0/2148035689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27a409d6f0 0x7f27a41c8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:21.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.698+0000 7f27a8ed5700 1 --2- 172.21.15.16:0/2148035689 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f27a409e000 0x7f27a41c8940 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:21.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.698+0000 7f27a8ed5700 1 -- 172.21.15.16:0/2148035689 >> 172.21.15.16:0/2148035689 conn(0x7f27a4066f50 msgr2=0x7f27a41bacf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:21.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.698+0000 7f27a8ed5700 1 -- 172.21.15.16:0/2148035689 shutdown_connections 2023-09-30T22:29:21.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.698+0000 7f27a8ed5700 1 -- 172.21.15.16:0/2148035689 wait complete. 2023-09-30T22:29:21.700 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2023-09-30T22:29:21.713 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:29:21.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.849+0000 7f1fa43d6700 1 Processor -- start 2023-09-30T22:29:21.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.849+0000 7f1fa43d6700 1 -- start start 2023-09-30T22:29:21.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.850+0000 7f1fa43d6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f9c09d6f0 0x7f1f9c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:21.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.850+0000 7f1fa43d6700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1f9c09e000 0x7f1f9c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:21.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.850+0000 7f1fa43d6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f9c09e410 0x7f1f9c133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:21.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.850+0000 7f1fa43d6700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f9c06b460 con 0x7f1f9c09e000 2023-09-30T22:29:21.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.850+0000 7f1fa43d6700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9c09b120 con 0x7f1f9c09d6f0 2023-09-30T22:29:21.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.850+0000 7f1fa43d6700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9c09a7e0 con 0x7f1f9c09e410 2023-09-30T22:29:21.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.850+0000 7f1fa2172700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f9c09d6f0 0x7f1f9c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:21.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.850+0000 7f1fa1971700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1f9c09e000 0x7f1f9c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:21.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.850+0000 7f1fa2973700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f9c09e410 0x7f1f9c133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:21.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.850+0000 7f1fa2172700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f9c09d6f0 0x7f1f9c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37182/0 (socket says 172.21.15.16:37182) 2023-09-30T22:29:21.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.850+0000 7f1fa1971700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1f9c09e000 0x7f1f9c1339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57772/0 (socket says 172.21.15.16:57772) 2023-09-30T22:29:21.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.850+0000 7f1fa2172700 1 -- 172.21.15.16:0/1040422301 learned_addr learned my addr 172.21.15.16:0/1040422301 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:21.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.851+0000 7f1fa1971700 1 -- 172.21.15.16:0/1040422301 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f9c09e410 msgr2=0x7f1f9c133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:21.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.851+0000 7f1fa1971700 1 --2- 172.21.15.16:0/1040422301 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f9c09e410 0x7f1f9c133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:21.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.851+0000 7f1fa1971700 1 -- 172.21.15.16:0/1040422301 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f9c09d6f0 msgr2=0x7f1f9c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:21.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.851+0000 7f1fa1971700 1 --2- 172.21.15.16:0/1040422301 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f9c09d6f0 0x7f1f9c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:21.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.851+0000 7f1fa1971700 1 -- 172.21.15.16:0/1040422301 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f9c09a500 con 0x7f1f9c09e000 2023-09-30T22:29:21.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.851+0000 7f1fa2973700 1 --2- 172.21.15.16:0/1040422301 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f9c09e410 0x7f1f9c133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:21.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.851+0000 7f1fa2172700 1 --2- 172.21.15.16:0/1040422301 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f9c09d6f0 0x7f1f9c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:21.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.851+0000 7f1fa1971700 1 --2- 172.21.15.16:0/1040422301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f9c09e000 0x7f1f9c1339f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c009840 tx=0x7f1f8c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f46e0be377e8a71f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:21.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.851+0000 7f1fa1170700 1 -- 172.21.15.16:0/1040422301 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c011830 con 0x7f1f9c09e000 2023-09-30T22:29:21.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.851+0000 7f1fa1170700 1 -- 172.21.15.16:0/1040422301 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f8c011990 con 0x7f1f9c09e000 2023-09-30T22:29:21.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.851+0000 7f1fa1170700 1 -- 172.21.15.16:0/1040422301 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c011d20 con 0x7f1f9c09e000 2023-09-30T22:29:21.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.852+0000 7f1fa43d6700 1 -- 172.21.15.16:0/1040422301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f9c09e000 msgr2=0x7f1f9c1339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:21.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.852+0000 7f1fa43d6700 1 --2- 172.21.15.16:0/1040422301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f9c09e000 0x7f1f9c1339f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c009840 tx=0x7f1f8c009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:21.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.852+0000 7f1fa43d6700 1 -- 172.21.15.16:0/1040422301 shutdown_connections 2023-09-30T22:29:21.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.852+0000 7f1fa43d6700 1 --2- 172.21.15.16:0/1040422301 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f9c09d6f0 0x7f1f9c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:21.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.852+0000 7f1fa43d6700 1 --2- 172.21.15.16:0/1040422301 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f9c09e410 0x7f1f9c133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:21.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.852+0000 7f1fa43d6700 1 --2- 172.21.15.16:0/1040422301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f9c09e000 0x7f1f9c1339f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:21.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.852+0000 7f1fa43d6700 1 -- 172.21.15.16:0/1040422301 >> 172.21.15.16:0/1040422301 conn(0x7f1f9c066f50 msgr2=0x7f1f9c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:21.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.852+0000 7f1fa43d6700 1 -- 172.21.15.16:0/1040422301 shutdown_connections 2023-09-30T22:29:21.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.852+0000 7f1fa43d6700 1 -- 172.21.15.16:0/1040422301 wait complete. 2023-09-30T22:29:21.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.853+0000 7f1fa43d6700 1 Processor -- start 2023-09-30T22:29:21.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.853+0000 7f1fa43d6700 1 -- start start 2023-09-30T22:29:21.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.853+0000 7f1fa43d6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f9c09d6f0 0x7f1f9c1a8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:21.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.853+0000 7f1fa43d6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f9c09e000 0x7f1f9c1a8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:21.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.853+0000 7f1fa43d6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f9c09e410 0x7f1f9c1a8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:21.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.853+0000 7f1fa43d6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9c1a9620 con 0x7f1f9c09e000 2023-09-30T22:29:21.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.853+0000 7f1fa43d6700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9c1bb4b0 con 0x7f1f9c09d6f0 2023-09-30T22:29:21.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.853+0000 7f1fa43d6700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9c1aa2f0 con 0x7f1f9c09e410 2023-09-30T22:29:21.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.853+0000 7f1fa2172700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f9c09d6f0 0x7f1f9c1a8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:21.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.853+0000 7f1fa2973700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f9c09e410 0x7f1f9c1a8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:21.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.853+0000 7f1fa2172700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f9c09d6f0 0x7f1f9c1a8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37192/0 (socket says 172.21.15.16:37192) 2023-09-30T22:29:21.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.853+0000 7f1fa2172700 1 -- 172.21.15.16:0/2386730918 learned_addr learned my addr 172.21.15.16:0/2386730918 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:21.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.853+0000 7f1fa1971700 1 --2- 172.21.15.16:0/2386730918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f9c09e000 0x7f1f9c1a8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:21.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.854+0000 7f1fa2172700 1 -- 172.21.15.16:0/2386730918 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f9c09e410 msgr2=0x7f1f9c1a8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:21.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.854+0000 7f1fa2172700 1 --2- 172.21.15.16:0/2386730918 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f9c09e410 0x7f1f9c1a8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:21.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.854+0000 7f1fa2172700 1 -- 172.21.15.16:0/2386730918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f9c09e000 msgr2=0x7f1f9c1a8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:21.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.854+0000 7f1fa2172700 1 --2- 172.21.15.16:0/2386730918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f9c09e000 0x7f1f9c1a8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:21.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.854+0000 7f1fa2172700 1 -- 172.21.15.16:0/2386730918 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f98009550 con 0x7f1f9c09d6f0 2023-09-30T22:29:21.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.854+0000 7f1fa2973700 1 --2- 172.21.15.16:0/2386730918 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f9c09e410 0x7f1f9c1a8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:21.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.854+0000 7f1fa1971700 1 --2- 172.21.15.16:0/2386730918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f9c09e000 0x7f1f9c1a8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:21.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.854+0000 7f1fa2172700 1 --2- 172.21.15.16:0/2386730918 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f9c09d6f0 0x7f1f9c1a8150 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1f9800ec10 tx=0x7f1f9800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:21.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.854+0000 7f1f92ffd700 1 -- 172.21.15.16:0/2386730918 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9800c850 con 0x7f1f9c09d6f0 2023-09-30T22:29:21.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.854+0000 7f1f92ffd700 1 -- 172.21.15.16:0/2386730918 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f9800c9b0 con 0x7f1f9c09d6f0 2023-09-30T22:29:21.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.854+0000 7f1fa43d6700 1 -- 172.21.15.16:0/2386730918 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f94009a70 con 0x7f1f9c09d6f0 2023-09-30T22:29:21.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.854+0000 7f1f92ffd700 1 -- 172.21.15.16:0/2386730918 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9800cd20 con 0x7f1f9c09d6f0 2023-09-30T22:29:21.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.854+0000 7f1fa43d6700 1 -- 172.21.15.16:0/2386730918 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f8c0094a0 con 0x7f1f9c09d6f0 2023-09-30T22:29:21.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.855+0000 7f1f92ffd700 1 -- 172.21.15.16:0/2386730918 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f1f98013470 con 0x7f1f9c09d6f0 2023-09-30T22:29:21.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.855+0000 7f1f92ffd700 1 -- 172.21.15.16:0/2386730918 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4049+0+0 (secure 0 0 0) 0x7f1f9803fc40 con 0x7f1f9c09d6f0 2023-09-30T22:29:21.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.855+0000 7f1fa43d6700 1 -- 172.21.15.16:0/2386730918 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f9c0a2230 con 0x7f1f9c09d6f0 2023-09-30T22:29:21.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:21.860+0000 7f1f92ffd700 1 -- 172.21.15.16:0/2386730918 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1f98021020 con 0x7f1f9c09d6f0 2023-09-30T22:29:22.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.064+0000 7f1fa43d6700 1 -- 172.21.15.16:0/2386730918 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1f9c09c710 con 0x7f1f9c09d6f0 2023-09-30T22:29:22.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.065+0000 7f1f92ffd700 1 -- 172.21.15.16:0/2386730918 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+12914 (secure 0 0 0) 0x7f1f98013720 con 0x7f1f9c09d6f0 2023-09-30T22:29:22.065 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:29:22.066 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":22,"fsid":"7ede3e14-2bd0-4726-9437-6d13c90cd342","created":"2023-09-30T22:26:35.483697+0000","modified":"2023-09-30T22:29:16.486363+0000","last_up_change":"2023-09-30T22:26:44.227199+0000","last_in_change":"2023-09-30T22:26:37.120906+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"luminous","require_osd_release":"pacific","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T22:26:44.285133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{},"mgr_devicehealth":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:26:45.132921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:26:49.353683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:26:50.655685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"19946985-0ee2-487f-9347-929670dbe5ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6804","nonce":118824}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6805","nonce":118824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6807","nonce":118824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6806","nonce":118824}]},"public_addr":"172.21.15.16:6804/118824","cluster_addr":"172.21.15.16:6805/118824","heartbeat_back_addr":"172.21.15.16:6807/118824","heartbeat_front_addr":"172.21.15.16:6806/118824","state":["exists","up"]},{"osd":1,"uuid":"8efd4dbc-6ae4-4b82-b530-3b3eac2be445","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6812","nonce":118832}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6813","nonce":118832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6815","nonce":118832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6814","nonce":118832}]},"public_addr":"172.21.15.16:6812/118832","cluster_addr":"172.21.15.16:6813/118832","heartbeat_back_addr":"172.21.15.16:6815/118832","heartbeat_front_addr":"172.21.15.16:6814/118832","state":["exists","up"]},{"osd":2,"uuid":"b47bb1ce-27af-4c0c-868a-b76ebd0d9499","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6808","nonce":118840}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6809","nonce":118840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6811","nonce":118840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6810","nonce":118840}]},"public_addr":"172.21.15.16:6808/118840","cluster_addr":"172.21.15.16:6809/118840","heartbeat_back_addr":"172.21.15.16:6811/118840","heartbeat_front_addr":"172.21.15.16:6810/118840","state":["exists","up"]},{"osd":3,"uuid":"03b53d64-d10d-448a-a081-39ecceab589b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6800","nonce":118823}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6801","nonce":118823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6803","nonce":118823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6802","nonce":118823}]},"public_addr":"172.21.15.16:6800/118823","cluster_addr":"172.21.15.16:6801/118823","heartbeat_back_addr":"172.21.15.16:6803/118823","heartbeat_front_addr":"172.21.15.16:6802/118823","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.305953+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.307488+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.341023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.192071+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.8","mappings":[{"from":3,"to":2}]},{"pgid":"4.d","mappings":[{"from":3,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":0}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]},{"pgid":"4.17","mappings":[{"from":3,"to":2}]},{"pgid":"4.19","mappings":[{"from":3,"to":0}]},{"pgid":"4.3a","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6816/118285":"2023-10-01T22:29:14.475978+0000","172.21.15.16:0/2249754484":"2023-10-01T22:29:14.475978+0000","172.21.15.16:0/1101648925":"2023-10-01T22:29:14.475978+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:29:22.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.067+0000 7f1fa43d6700 1 -- 172.21.15.16:0/2386730918 >> v1:172.21.15.16:6816/126811 conn(0x7f1f8002f510 legacy=0x7f1f800319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:22.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.067+0000 7f1fa43d6700 1 -- 172.21.15.16:0/2386730918 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f9c09d6f0 msgr2=0x7f1f9c1a8150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:22.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.067+0000 7f1fa43d6700 1 --2- 172.21.15.16:0/2386730918 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f9c09d6f0 0x7f1f9c1a8150 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1f9800ec10 tx=0x7f1f9800ef20 comp rx=0 tx=0).stop 2023-09-30T22:29:22.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.067+0000 7f1fa43d6700 1 -- 172.21.15.16:0/2386730918 shutdown_connections 2023-09-30T22:29:22.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.067+0000 7f1fa43d6700 1 --2- 172.21.15.16:0/2386730918 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f9c09d6f0 0x7f1f9c1a8150 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:22.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.067+0000 7f1fa43d6700 1 --2- 172.21.15.16:0/2386730918 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f9c09e410 0x7f1f9c1a8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:22.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.067+0000 7f1fa43d6700 1 --2- 172.21.15.16:0/2386730918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f9c09e000 0x7f1f9c1a8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:22.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.067+0000 7f1fa43d6700 1 -- 172.21.15.16:0/2386730918 >> 172.21.15.16:0/2386730918 conn(0x7f1f9c066f50 msgr2=0x7f1f9c06f420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:22.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.067+0000 7f1fa43d6700 1 -- 172.21.15.16:0/2386730918 shutdown_connections 2023-09-30T22:29:22.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.067+0000 7f1fa43d6700 1 -- 172.21.15.16:0/2386730918 wait complete. 2023-09-30T22:29:22.081 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-30T22:26:44.285133+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}, 'mgr_devicehealth': {}}, 'read_balance': {'score_acting': 4, 'score_stable': 4, 'optimal_score': 0.5, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-09-30T22:26:45.132921+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-30T22:26:49.353683+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '15', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-09-30T22:26:50.655685+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-30T22:29:22.082 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-30T22:29:22.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.221+0000 7f5d40ffa700 1 Processor -- start 2023-09-30T22:29:22.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.221+0000 7f5d40ffa700 1 -- start start 2023-09-30T22:29:22.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.222+0000 7f5d40ffa700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d3c134a90 0x7f5d3c136e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:22.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.222+0000 7f5d40ffa700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d3c1373b0 0x7f5d3c139790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:22.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.222+0000 7f5d40ffa700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d3c09ec00 con 0x7f5d3c131c20 2023-09-30T22:29:22.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.222+0000 7f5d40ffa700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c132700 con 0x7f5d3c1373b0 2023-09-30T22:29:22.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.222+0000 7f5d40ffa700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c06ee30 con 0x7f5d3c134a90 2023-09-30T22:29:22.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.222+0000 7f5d3ad9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d3c1373b0 0x7f5d3c139790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:22.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.222+0000 7f5d39d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d3c134a90 0x7f5d3c136e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:22.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.222+0000 7f5d3a59c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5d3c131c20 0x7f5d3c131ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33834/0 (socket says 172.21.15.16:33834) 2023-09-30T22:29:22.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.222+0000 7f5d39d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d3c134a90 0x7f5d3c136e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:55912/0 (socket says 172.21.15.16:55912) 2023-09-30T22:29:22.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.222+0000 7f5d3a59c700 1 -- 172.21.15.16:0/3634440138 learned_addr learned my addr 172.21.15.16:0/3634440138 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:29:22.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.222+0000 7f5d3ad9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d3c1373b0 0x7f5d3c139790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37208/0 (socket says 172.21.15.16:37208) 2023-09-30T22:29:22.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.222+0000 7f5d3ad9d700 1 -- 172.21.15.16:0/3634440138 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d3c134a90 msgr2=0x7f5d3c136e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:22.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.222+0000 7f5d3ad9d700 1 --2- 172.21.15.16:0/3634440138 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d3c134a90 0x7f5d3c136e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:22.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.222+0000 7f5d3ad9d700 1 -- 172.21.15.16:0/3634440138 >> v1:172.21.15.16:6789/0 conn(0x7f5d3c131c20 legacy=0x7f5d3c131ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:22.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.222+0000 7f5d3959a700 1 -- 172.21.15.16:0/3634440138 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3283637413 0 0) 0x7f5d3c09ec00 con 0x7f5d3c131c20 2023-09-30T22:29:22.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.222+0000 7f5d3ad9d700 1 -- 172.21.15.16:0/3634440138 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d3c131780 con 0x7f5d3c1373b0 2023-09-30T22:29:22.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.222+0000 7f5d39d9b700 1 --2- 172.21.15.16:0/3634440138 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d3c134a90 0x7f5d3c136e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:22.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.222+0000 7f5d3ad9d700 1 --2- 172.21.15.16:0/3634440138 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d3c1373b0 0x7f5d3c139790 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5d34009dc0 tx=0x7f5d3400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=875920afbb7f979b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:22.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.223+0000 7f5d3959a700 1 -- 172.21.15.16:0/3634440138 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3400e070 con 0x7f5d3c1373b0 2023-09-30T22:29:22.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.223+0000 7f5d3959a700 1 -- 172.21.15.16:0/3634440138 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d3400bb20 con 0x7f5d3c1373b0 2023-09-30T22:29:22.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.223+0000 7f5d3959a700 1 -- 172.21.15.16:0/3634440138 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d34012430 con 0x7f5d3c1373b0 2023-09-30T22:29:22.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.223+0000 7f5d40ffa700 1 -- 172.21.15.16:0/3634440138 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d3c1373b0 msgr2=0x7f5d3c139790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:22.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.223+0000 7f5d40ffa700 1 --2- 172.21.15.16:0/3634440138 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d3c1373b0 0x7f5d3c139790 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5d34009dc0 tx=0x7f5d3400b330 comp rx=0 tx=0).stop 2023-09-30T22:29:22.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.223+0000 7f5d40ffa700 1 -- 172.21.15.16:0/3634440138 shutdown_connections 2023-09-30T22:29:22.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.223+0000 7f5d40ffa700 1 --2- 172.21.15.16:0/3634440138 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d3c1373b0 0x7f5d3c139790 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:22.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.223+0000 7f5d40ffa700 1 --2- 172.21.15.16:0/3634440138 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d3c134a90 0x7f5d3c136e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:22.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.223+0000 7f5d40ffa700 1 -- 172.21.15.16:0/3634440138 >> 172.21.15.16:0/3634440138 conn(0x7f5d3c0a2c10 msgr2=0x7f5d3c0a3010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:22.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.223+0000 7f5d40ffa700 1 -- 172.21.15.16:0/3634440138 shutdown_connections 2023-09-30T22:29:22.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.223+0000 7f5d40ffa700 1 -- 172.21.15.16:0/3634440138 wait complete. 2023-09-30T22:29:22.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.224+0000 7f5d40ffa700 1 Processor -- start 2023-09-30T22:29:22.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.225+0000 7f5d40ffa700 1 -- start start 2023-09-30T22:29:22.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.225+0000 7f5d40ffa700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d3c131c20 0x7f5d3c12c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:22.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.225+0000 7f5d40ffa700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d3c134a90 0x7f5d3c12c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:22.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.225+0000 7f5d40ffa700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d3c1373b0 0x7f5d3c12ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:22.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.225+0000 7f5d40ffa700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c119510 con 0x7f5d3c134a90 2023-09-30T22:29:22.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.225+0000 7f5d40ffa700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c119a30 con 0x7f5d3c1373b0 2023-09-30T22:29:22.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.225+0000 7f5d40ffa700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3c12e390 con 0x7f5d3c131c20 2023-09-30T22:29:22.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.225+0000 7f5d3a59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d3c131c20 0x7f5d3c12c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:22.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.225+0000 7f5d39d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d3c134a90 0x7f5d3c12c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:22.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.225+0000 7f5d3ad9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d3c1373b0 0x7f5d3c12ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:22.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.225+0000 7f5d3a59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d3c131c20 0x7f5d3c12c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:55924/0 (socket says 172.21.15.16:55924) 2023-09-30T22:29:22.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.225+0000 7f5d39d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d3c134a90 0x7f5d3c12c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57794/0 (socket says 172.21.15.16:57794) 2023-09-30T22:29:22.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.225+0000 7f5d3a59c700 1 -- 172.21.15.16:0/392589582 learned_addr learned my addr 172.21.15.16:0/392589582 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:22.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.225+0000 7f5d3a59c700 1 -- 172.21.15.16:0/392589582 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d3c1373b0 msgr2=0x7f5d3c12ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:22.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.225+0000 7f5d3a59c700 1 --2- 172.21.15.16:0/392589582 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d3c1373b0 0x7f5d3c12ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:22.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.225+0000 7f5d3a59c700 1 -- 172.21.15.16:0/392589582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d3c134a90 msgr2=0x7f5d3c12c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:22.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.225+0000 7f5d3a59c700 1 --2- 172.21.15.16:0/392589582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d3c134a90 0x7f5d3c12c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:22.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.225+0000 7f5d3a59c700 1 -- 172.21.15.16:0/392589582 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d300094a0 con 0x7f5d3c131c20 2023-09-30T22:29:22.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.226+0000 7f5d3ad9d700 1 --2- 172.21.15.16:0/392589582 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d3c1373b0 0x7f5d3c12ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:22.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.226+0000 7f5d3a59c700 1 --2- 172.21.15.16:0/392589582 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d3c131c20 0x7f5d3c12c3e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c00c520 tx=0x7f5d2c00c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:22.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.226+0000 7f5d2affd700 1 -- 172.21.15.16:0/392589582 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d2c004c80 con 0x7f5d3c131c20 2023-09-30T22:29:22.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.226+0000 7f5d2affd700 1 -- 172.21.15.16:0/392589582 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d2c004e00 con 0x7f5d3c131c20 2023-09-30T22:29:22.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.226+0000 7f5d40ffa700 1 -- 172.21.15.16:0/392589582 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d34009a70 con 0x7f5d3c131c20 2023-09-30T22:29:22.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.226+0000 7f5d2affd700 1 -- 172.21.15.16:0/392589582 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d2c007310 con 0x7f5d3c131c20 2023-09-30T22:29:22.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.226+0000 7f5d40ffa700 1 -- 172.21.15.16:0/392589582 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d3c12e5d0 con 0x7f5d3c131c20 2023-09-30T22:29:22.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.227+0000 7f5d2affd700 1 -- 172.21.15.16:0/392589582 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f5d2c007470 con 0x7f5d3c131c20 2023-09-30T22:29:22.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.227+0000 7f5d2affd700 1 -- 172.21.15.16:0/392589582 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4049+0+0 (secure 0 0 0) 0x7f5d2c03f750 con 0x7f5d3c131c20 2023-09-30T22:29:22.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.227+0000 7f5d40ffa700 1 -- 172.21.15.16:0/392589582 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d3c04f2a0 con 0x7f5d3c131c20 2023-09-30T22:29:22.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.232+0000 7f5d2affd700 1 -- 172.21.15.16:0/392589582 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5d2c0123b0 con 0x7f5d3c131c20 2023-09-30T22:29:22.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.433+0000 7f5d40ffa700 1 -- 172.21.15.16:0/392589582 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5d3c072d60 con 0x7f5d3c131c20 2023-09-30T22:29:22.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.433+0000 7f5d2affd700 1 -- 172.21.15.16:0/392589582 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5d2c0102c0 con 0x7f5d3c131c20 2023-09-30T22:29:22.434 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-09-30T22:29:22.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.435+0000 7f5d40ffa700 1 -- 172.21.15.16:0/392589582 >> v1:172.21.15.16:6816/126811 conn(0x7f5d1c02f510 legacy=0x7f5d1c0319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:22.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.435+0000 7f5d40ffa700 1 -- 172.21.15.16:0/392589582 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d3c131c20 msgr2=0x7f5d3c12c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:22.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.435+0000 7f5d40ffa700 1 --2- 172.21.15.16:0/392589582 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d3c131c20 0x7f5d3c12c3e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c00c520 tx=0x7f5d2c00c830 comp rx=0 tx=0).stop 2023-09-30T22:29:22.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.435+0000 7f5d40ffa700 1 -- 172.21.15.16:0/392589582 shutdown_connections 2023-09-30T22:29:22.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.435+0000 7f5d40ffa700 1 --2- 172.21.15.16:0/392589582 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d3c1373b0 0x7f5d3c12ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:22.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.435+0000 7f5d40ffa700 1 --2- 172.21.15.16:0/392589582 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d3c131c20 0x7f5d3c12c3e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:22.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.435+0000 7f5d40ffa700 1 --2- 172.21.15.16:0/392589582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d3c134a90 0x7f5d3c12c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:22.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.435+0000 7f5d40ffa700 1 -- 172.21.15.16:0/392589582 >> 172.21.15.16:0/392589582 conn(0x7f5d3c0a2c10 msgr2=0x7f5d3c12fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:22.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.436+0000 7f5d40ffa700 1 -- 172.21.15.16:0/392589582 shutdown_connections 2023-09-30T22:29:22.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.436+0000 7f5d40ffa700 1 -- 172.21.15.16:0/392589582 wait complete. 2023-09-30T22:29:22.449 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-09-30T22:29:22.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.590+0000 7fa9f8975700 1 Processor -- start 2023-09-30T22:29:22.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.590+0000 7fa9f8975700 1 -- start start 2023-09-30T22:29:22.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.591+0000 7fa9f8975700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa9f409d7d0 0x7fa9f409dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:22.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.591+0000 7fa9f8975700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa9f409e0e0 0x7fa9f4130770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:22.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.591+0000 7fa9f8975700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9f409b800 con 0x7fa9f4130cb0 2023-09-30T22:29:22.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.591+0000 7fa9f8975700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9f409b620 con 0x7fa9f409e0e0 2023-09-30T22:29:22.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.591+0000 7fa9f8975700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f409f370 con 0x7fa9f409d7d0 2023-09-30T22:29:22.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.591+0000 7fa9f259c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa9f409d7d0 0x7fa9f409dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:22.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.591+0000 7fa9f1d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa9f409e0e0 0x7fa9f4130770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:22.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.591+0000 7fa9f2d9d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa9f4130cb0 0x7fa9f4137a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33848/0 (socket says 172.21.15.16:33848) 2023-09-30T22:29:22.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.591+0000 7fa9f259c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa9f409d7d0 0x7fa9f409dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37224/0 (socket says 172.21.15.16:37224) 2023-09-30T22:29:22.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.591+0000 7fa9f1d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa9f409e0e0 0x7fa9f4130770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57804/0 (socket says 172.21.15.16:57804) 2023-09-30T22:29:22.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.591+0000 7fa9f2d9d700 1 -- 172.21.15.16:0/4024795014 learned_addr learned my addr 172.21.15.16:0/4024795014 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:29:22.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.591+0000 7fa9f159a700 1 -- 172.21.15.16:0/4024795014 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 226030942 0 0) 0x7fa9f409b800 con 0x7fa9f4130cb0 2023-09-30T22:29:22.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.591+0000 7fa9f259c700 1 -- 172.21.15.16:0/4024795014 >> v2:172.21.15.16:3300/0 conn(0x7fa9f409e0e0 msgr2=0x7fa9f4130770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:22.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.591+0000 7fa9f259c700 1 --2- 172.21.15.16:0/4024795014 >> v2:172.21.15.16:3300/0 conn(0x7fa9f409e0e0 0x7fa9f4130770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:22.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.591+0000 7fa9f259c700 1 -- 172.21.15.16:0/4024795014 >> v1:172.21.15.16:6789/0 conn(0x7fa9f4130cb0 legacy=0x7fa9f4137a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:22.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.591+0000 7fa9f259c700 1 -- 172.21.15.16:0/4024795014 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f409ef80 con 0x7fa9f409d7d0 2023-09-30T22:29:22.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.591+0000 7fa9f259c700 1 --2- 172.21.15.16:0/4024795014 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa9f409d7d0 0x7fa9f409dba0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4009840 tx=0x7fa9e4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c8cb33920645530 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:22.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.591+0000 7fa9f159a700 1 -- 172.21.15.16:0/4024795014 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e4011830 con 0x7fa9f409d7d0 2023-09-30T22:29:22.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.591+0000 7fa9f159a700 1 -- 172.21.15.16:0/4024795014 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9e4011990 con 0x7fa9f409d7d0 2023-09-30T22:29:22.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.592+0000 7fa9f159a700 1 -- 172.21.15.16:0/4024795014 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e4011d20 con 0x7fa9f409d7d0 2023-09-30T22:29:22.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.592+0000 7fa9f8975700 1 -- 172.21.15.16:0/4024795014 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa9f409d7d0 msgr2=0x7fa9f409dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:22.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.592+0000 7fa9f8975700 1 --2- 172.21.15.16:0/4024795014 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa9f409d7d0 0x7fa9f409dba0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4009840 tx=0x7fa9e4009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:22.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.592+0000 7fa9f8975700 1 -- 172.21.15.16:0/4024795014 shutdown_connections 2023-09-30T22:29:22.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.592+0000 7fa9f8975700 1 --2- 172.21.15.16:0/4024795014 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa9f409d7d0 0x7fa9f409dba0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:22.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.592+0000 7fa9f8975700 1 --2- 172.21.15.16:0/4024795014 >> v2:172.21.15.16:3300/0 conn(0x7fa9f409e0e0 0x7fa9f4130770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:22.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.592+0000 7fa9f8975700 1 -- 172.21.15.16:0/4024795014 >> 172.21.15.16:0/4024795014 conn(0x7fa9f4066d70 msgr2=0x7fa9f40a2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:22.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.592+0000 7fa9f8975700 1 -- 172.21.15.16:0/4024795014 shutdown_connections 2023-09-30T22:29:22.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.592+0000 7fa9f8975700 1 -- 172.21.15.16:0/4024795014 wait complete. 2023-09-30T22:29:22.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.593+0000 7fa9f8975700 1 Processor -- start 2023-09-30T22:29:22.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.593+0000 7fa9f8975700 1 -- start start 2023-09-30T22:29:22.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.594+0000 7fa9f8975700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa9f409d7d0 0x7fa9f411b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:22.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.594+0000 7fa9f8975700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9f409e0e0 0x7fa9f411ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:22.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.594+0000 7fa9f8975700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa9f4130cb0 0x7fa9f411bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:22.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.594+0000 7fa9f8975700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f411ef90 con 0x7fa9f409e0e0 2023-09-30T22:29:22.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.594+0000 7fa9f8975700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f411d080 con 0x7fa9f4130cb0 2023-09-30T22:29:22.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.594+0000 7fa9f8975700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f4071c80 con 0x7fa9f409d7d0 2023-09-30T22:29:22.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.594+0000 7fa9f259c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa9f409d7d0 0x7fa9f411b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:22.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.594+0000 7fa9f2d9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa9f4130cb0 0x7fa9f411bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:22.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.594+0000 7fa9f259c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa9f409d7d0 0x7fa9f411b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:55926/0 (socket says 172.21.15.16:55926) 2023-09-30T22:29:22.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.594+0000 7fa9f2d9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa9f4130cb0 0x7fa9f411bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37230/0 (socket says 172.21.15.16:37230) 2023-09-30T22:29:22.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.594+0000 7fa9f1d9b700 1 --2- 172.21.15.16:0/2170622197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9f409e0e0 0x7fa9f411ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:22.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.594+0000 7fa9f259c700 1 -- 172.21.15.16:0/2170622197 learned_addr learned my addr 172.21.15.16:0/2170622197 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:22.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.594+0000 7fa9f2d9d700 1 -- 172.21.15.16:0/2170622197 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa9f409d7d0 msgr2=0x7fa9f411b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:22.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.594+0000 7fa9f2d9d700 1 --2- 172.21.15.16:0/2170622197 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa9f409d7d0 0x7fa9f411b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:22.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.594+0000 7fa9f2d9d700 1 -- 172.21.15.16:0/2170622197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9f409e0e0 msgr2=0x7fa9f411ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:22.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.594+0000 7fa9f2d9d700 1 --2- 172.21.15.16:0/2170622197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9f409e0e0 0x7fa9f411ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:22.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.594+0000 7fa9f2d9d700 1 -- 172.21.15.16:0/2170622197 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e40094a0 con 0x7fa9f4130cb0 2023-09-30T22:29:22.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.594+0000 7fa9f259c700 1 --2- 172.21.15.16:0/2170622197 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa9f409d7d0 0x7fa9f411b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:22.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.594+0000 7fa9f1d9b700 1 --2- 172.21.15.16:0/2170622197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9f409e0e0 0x7fa9f411ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:22.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.595+0000 7fa9f2d9d700 1 --2- 172.21.15.16:0/2170622197 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa9f4130cb0 0x7fa9f411bf80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec00cc10 tx=0x7fa9ec00cf20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:22.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.595+0000 7fa9e2ffd700 1 -- 172.21.15.16:0/2170622197 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ec00aa00 con 0x7fa9f4130cb0 2023-09-30T22:29:22.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.595+0000 7fa9e2ffd700 1 -- 172.21.15.16:0/2170622197 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9ec00ab80 con 0x7fa9f4130cb0 2023-09-30T22:29:22.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.595+0000 7fa9f8975700 1 -- 172.21.15.16:0/2170622197 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9f409c720 con 0x7fa9f4130cb0 2023-09-30T22:29:22.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.595+0000 7fa9e2ffd700 1 -- 172.21.15.16:0/2170622197 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ec0070d0 con 0x7fa9f4130cb0 2023-09-30T22:29:22.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.595+0000 7fa9f8975700 1 -- 172.21.15.16:0/2170622197 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9f411d830 con 0x7fa9f4130cb0 2023-09-30T22:29:22.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.596+0000 7fa9e2ffd700 1 -- 172.21.15.16:0/2170622197 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fa9ec007230 con 0x7fa9f4130cb0 2023-09-30T22:29:22.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.596+0000 7fa9e2ffd700 1 -- 172.21.15.16:0/2170622197 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4049+0+0 (secure 0 0 0) 0x7fa9ec03f8d0 con 0x7fa9f4130cb0 2023-09-30T22:29:22.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.596+0000 7fa9f8975700 1 -- 172.21.15.16:0/2170622197 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9f404f2a0 con 0x7fa9f4130cb0 2023-09-30T22:29:22.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.601+0000 7fa9e2ffd700 1 -- 172.21.15.16:0/2170622197 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa9ec018080 con 0x7fa9f4130cb0 2023-09-30T22:29:22.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.803+0000 7fa9f8975700 1 -- 172.21.15.16:0/2170622197 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa9f406c380 con 0x7fa9f4130cb0 2023-09-30T22:29:22.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.803+0000 7fa9e2ffd700 1 -- 172.21.15.16:0/2170622197 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa9ec0074e0 con 0x7fa9f4130cb0 2023-09-30T22:29:22.804 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 8 2023-09-30T22:29:22.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.806+0000 7fa9f8975700 1 -- 172.21.15.16:0/2170622197 >> v1:172.21.15.16:6816/126811 conn(0x7fa9d402f510 legacy=0x7fa9d40319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:22.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.806+0000 7fa9f8975700 1 -- 172.21.15.16:0/2170622197 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa9f4130cb0 msgr2=0x7fa9f411bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:22.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.806+0000 7fa9f8975700 1 --2- 172.21.15.16:0/2170622197 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa9f4130cb0 0x7fa9f411bf80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec00cc10 tx=0x7fa9ec00cf20 comp rx=0 tx=0).stop 2023-09-30T22:29:22.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.806+0000 7fa9f8975700 1 -- 172.21.15.16:0/2170622197 shutdown_connections 2023-09-30T22:29:22.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.806+0000 7fa9f8975700 1 --2- 172.21.15.16:0/2170622197 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa9f4130cb0 0x7fa9f411bf80 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:22.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.806+0000 7fa9f8975700 1 --2- 172.21.15.16:0/2170622197 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa9f409d7d0 0x7fa9f411b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:22.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.806+0000 7fa9f8975700 1 --2- 172.21.15.16:0/2170622197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9f409e0e0 0x7fa9f411ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:22.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.806+0000 7fa9f8975700 1 -- 172.21.15.16:0/2170622197 >> 172.21.15.16:0/2170622197 conn(0x7fa9f4066d70 msgr2=0x7fa9f4070a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:22.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.806+0000 7fa9f8975700 1 -- 172.21.15.16:0/2170622197 shutdown_connections 2023-09-30T22:29:22.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.806+0000 7fa9f8975700 1 -- 172.21.15.16:0/2170622197 wait complete. 2023-09-30T22:29:22.819 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-30T22:29:22.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.958+0000 7f570bb6d700 1 Processor -- start 2023-09-30T22:29:22.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.958+0000 7f570bb6d700 1 -- start start 2023-09-30T22:29:22.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.959+0000 7f570bb6d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f570409f9d0 0x7f570413d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:22.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.959+0000 7f570bb6d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f570413d6b0 0x7f570413fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:22.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.959+0000 7f570bb6d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f570409d0b0 con 0x7f570409eef0 2023-09-30T22:29:22.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.959+0000 7f570bb6d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f570409dd70 con 0x7f570413d6b0 2023-09-30T22:29:22.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.959+0000 7f570bb6d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f57040a2820 con 0x7f570409f9d0 2023-09-30T22:29:22.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.959+0000 7f570a10a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f570413d6b0 0x7f570413fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:22.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.959+0000 7f5709108700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f570409f9d0 0x7f570413d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:22.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.959+0000 7f5709909700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f570409eef0 0x7f570409f2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33864/0 (socket says 172.21.15.16:33864) 2023-09-30T22:29:22.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.959+0000 7f5709909700 1 -- 172.21.15.16:0/3934483489 learned_addr learned my addr 172.21.15.16:0/3934483489 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:29:22.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.959+0000 7f5709108700 1 -- 172.21.15.16:0/3934483489 >> v2:172.21.15.16:3300/0 conn(0x7f570413d6b0 msgr2=0x7f570413fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:22.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.959+0000 7f5709108700 1 --2- 172.21.15.16:0/3934483489 >> v2:172.21.15.16:3300/0 conn(0x7f570413d6b0 0x7f570413fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:22.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.959+0000 7f5709108700 1 -- 172.21.15.16:0/3934483489 >> v1:172.21.15.16:6789/0 conn(0x7f570409eef0 legacy=0x7f570409f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:22.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.959+0000 7f5709108700 1 -- 172.21.15.16:0/3934483489 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f570406cf80 con 0x7f570409f9d0 2023-09-30T22:29:22.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.959+0000 7f570a10a700 1 --2- 172.21.15.16:0/3934483489 >> v2:172.21.15.16:3300/0 conn(0x7f570413d6b0 0x7f570413fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:22.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.959+0000 7f5708907700 1 -- 172.21.15.16:0/3934483489 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4219301708 0 0) 0x7f570409d0b0 con 0x7f570409eef0 2023-09-30T22:29:22.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.959+0000 7f5709108700 1 --2- 172.21.15.16:0/3934483489 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f570409f9d0 0x7f570413d170 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f56f4009870 tx=0x7f56f4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=655f522f949dc075 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:22.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.959+0000 7f5708907700 1 -- 172.21.15.16:0/3934483489 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f4011830 con 0x7f570409f9d0 2023-09-30T22:29:22.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.960+0000 7f5708907700 1 -- 172.21.15.16:0/3934483489 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56f4011990 con 0x7f570409f9d0 2023-09-30T22:29:22.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.960+0000 7f5708907700 1 -- 172.21.15.16:0/3934483489 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f4011d20 con 0x7f570409f9d0 2023-09-30T22:29:22.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.960+0000 7f570bb6d700 1 -- 172.21.15.16:0/3934483489 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f570409f9d0 msgr2=0x7f570413d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:22.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.960+0000 7f570bb6d700 1 --2- 172.21.15.16:0/3934483489 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f570409f9d0 0x7f570413d170 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f56f4009870 tx=0x7f56f4009b80 comp rx=0 tx=0).stop 2023-09-30T22:29:22.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.960+0000 7f570bb6d700 1 -- 172.21.15.16:0/3934483489 shutdown_connections 2023-09-30T22:29:22.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.960+0000 7f570bb6d700 1 --2- 172.21.15.16:0/3934483489 >> v2:172.21.15.16:3300/0 conn(0x7f570413d6b0 0x7f570413fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:22.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.960+0000 7f570bb6d700 1 --2- 172.21.15.16:0/3934483489 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f570409f9d0 0x7f570413d170 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:22.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.960+0000 7f570bb6d700 1 -- 172.21.15.16:0/3934483489 >> 172.21.15.16:0/3934483489 conn(0x7f57040a3050 msgr2=0x7f57040a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:22.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.960+0000 7f570bb6d700 1 -- 172.21.15.16:0/3934483489 shutdown_connections 2023-09-30T22:29:22.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.960+0000 7f570bb6d700 1 -- 172.21.15.16:0/3934483489 wait complete. 2023-09-30T22:29:22.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.961+0000 7f570bb6d700 1 Processor -- start 2023-09-30T22:29:22.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.961+0000 7f570bb6d700 1 -- start start 2023-09-30T22:29:22.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.962+0000 7f570bb6d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f570409eef0 0x7f5704125f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:22.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.962+0000 7f570bb6d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f570409f9d0 0x7f5704126460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:22.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.962+0000 7f570bb6d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f570413d6b0 0x7f57041269a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:22.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.962+0000 7f570bb6d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5704129760 con 0x7f570409eef0 2023-09-30T22:29:22.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.962+0000 7f570bb6d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5704129240 con 0x7f570413d6b0 2023-09-30T22:29:22.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.962+0000 7f570bb6d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f570409bc00 con 0x7f570409f9d0 2023-09-30T22:29:22.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.962+0000 7f5709909700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f570409eef0 0x7f5704125f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:22.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.962+0000 7f5709108700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f570409f9d0 0x7f5704126460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:22.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.962+0000 7f5709108700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f570409f9d0 0x7f5704126460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:55942/0 (socket says 172.21.15.16:55942) 2023-09-30T22:29:22.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.962+0000 7f5709909700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f570409eef0 0x7f5704125f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57834/0 (socket says 172.21.15.16:57834) 2023-09-30T22:29:22.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.962+0000 7f5709108700 1 -- 172.21.15.16:0/236368178 learned_addr learned my addr 172.21.15.16:0/236368178 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:22.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.962+0000 7f570a10a700 1 --2- 172.21.15.16:0/236368178 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f570413d6b0 0x7f57041269a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:22.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.962+0000 7f5709909700 1 -- 172.21.15.16:0/236368178 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f570409f9d0 msgr2=0x7f5704126460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:22.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.962+0000 7f5709909700 1 --2- 172.21.15.16:0/236368178 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f570409f9d0 0x7f5704126460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:22.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.962+0000 7f5709909700 1 -- 172.21.15.16:0/236368178 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f570413d6b0 msgr2=0x7f57041269a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:22.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.962+0000 7f5709909700 1 --2- 172.21.15.16:0/236368178 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f570413d6b0 0x7f57041269a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:22.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.962+0000 7f5709909700 1 -- 172.21.15.16:0/236368178 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56fc009a70 con 0x7f570409eef0 2023-09-30T22:29:22.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.962+0000 7f5709108700 1 --2- 172.21.15.16:0/236368178 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f570409f9d0 0x7f5704126460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:22.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.962+0000 7f5709909700 1 --2- 172.21.15.16:0/236368178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f570409eef0 0x7f5704125f20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f570000c470 tx=0x7f570000c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:22.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.963+0000 7f56fa7fc700 1 -- 172.21.15.16:0/236368178 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5700004c80 con 0x7f570409eef0 2023-09-30T22:29:22.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.963+0000 7f56fa7fc700 1 -- 172.21.15.16:0/236368178 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5700004de0 con 0x7f570409eef0 2023-09-30T22:29:22.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.963+0000 7f56fa7fc700 1 -- 172.21.15.16:0/236368178 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5700007310 con 0x7f570409eef0 2023-09-30T22:29:22.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.963+0000 7f570bb6d700 1 -- 172.21.15.16:0/236368178 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56f40094a0 con 0x7f570409eef0 2023-09-30T22:29:22.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.963+0000 7f570bb6d700 1 -- 172.21.15.16:0/236368178 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f570412a1d0 con 0x7f570409eef0 2023-09-30T22:29:22.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.963+0000 7f56fa7fc700 1 -- 172.21.15.16:0/236368178 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f5700007470 con 0x7f570409eef0 2023-09-30T22:29:22.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.964+0000 7f570bb6d700 1 -- 172.21.15.16:0/236368178 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f570404f2a0 con 0x7f570409eef0 2023-09-30T22:29:22.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.964+0000 7f56fa7fc700 1 -- 172.21.15.16:0/236368178 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4049+0+0 (secure 0 0 0) 0x7f570003f700 con 0x7f570409eef0 2023-09-30T22:29:22.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:22.968+0000 7f56fa7fc700 1 -- 172.21.15.16:0/236368178 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5700007720 con 0x7f570409eef0 2023-09-30T22:29:23.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.169+0000 7f570bb6d700 1 -- 172.21.15.16:0/236368178 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f57040729a0 con 0x7f570409eef0 2023-09-30T22:29:23.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.169+0000 7f56fa7fc700 1 -- 172.21.15.16:0/236368178 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5700011360 con 0x7f570409eef0 2023-09-30T22:29:23.170 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-09-30T22:29:23.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.171+0000 7f570bb6d700 1 -- 172.21.15.16:0/236368178 >> v1:172.21.15.16:6816/126811 conn(0x7f56e802f510 legacy=0x7f56e80319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:23.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.171+0000 7f570bb6d700 1 -- 172.21.15.16:0/236368178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f570409eef0 msgr2=0x7f5704125f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:23.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.171+0000 7f570bb6d700 1 --2- 172.21.15.16:0/236368178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f570409eef0 0x7f5704125f20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f570000c470 tx=0x7f570000c780 comp rx=0 tx=0).stop 2023-09-30T22:29:23.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.171+0000 7f570bb6d700 1 -- 172.21.15.16:0/236368178 shutdown_connections 2023-09-30T22:29:23.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.171+0000 7f570bb6d700 1 --2- 172.21.15.16:0/236368178 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f570413d6b0 0x7f57041269a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:23.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.171+0000 7f570bb6d700 1 --2- 172.21.15.16:0/236368178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f570409eef0 0x7f5704125f20 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:23.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.171+0000 7f570bb6d700 1 --2- 172.21.15.16:0/236368178 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f570409f9d0 0x7f5704126460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:23.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.171+0000 7f570bb6d700 1 -- 172.21.15.16:0/236368178 >> 172.21.15.16:0/236368178 conn(0x7f57040a3050 msgr2=0x7f570413c4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:23.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.171+0000 7f570bb6d700 1 -- 172.21.15.16:0/236368178 shutdown_connections 2023-09-30T22:29:23.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.171+0000 7f570bb6d700 1 -- 172.21.15.16:0/236368178 wait complete. 2023-09-30T22:29:23.185 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-30T22:29:23.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.325+0000 7f2c37013700 1 Processor -- start 2023-09-30T22:29:23.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.325+0000 7f2c37013700 1 -- start start 2023-09-30T22:29:23.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.326+0000 7f2c37013700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c30131b30 0x7f2c30131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:23.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.326+0000 7f2c37013700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c3009e850 0x7f2c3009ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:23.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.326+0000 7f2c37013700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c300a01c0 con 0x7f2c3009f1c0 2023-09-30T22:29:23.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.326+0000 7f2c37013700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3009b560 con 0x7f2c3009e850 2023-09-30T22:29:23.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.326+0000 7f2c37013700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3009a260 con 0x7f2c30131b30 2023-09-30T22:29:23.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.326+0000 7f2c34daf700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c30131b30 0x7f2c30131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:23.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.326+0000 7f2c2ffff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c3009e850 0x7f2c3009ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:23.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.326+0000 7f2c355b0700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2c3009f1c0 0x7f2c301389c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33876/0 (socket says 172.21.15.16:33876) 2023-09-30T22:29:23.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.326+0000 7f2c355b0700 1 -- 172.21.15.16:0/2993103777 learned_addr learned my addr 172.21.15.16:0/2993103777 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:29:23.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.327+0000 7f2c2f7fe700 1 -- 172.21.15.16:0/2993103777 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3083774312 0 0) 0x7f2c300a01c0 con 0x7f2c3009f1c0 2023-09-30T22:29:23.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.327+0000 7f2c34daf700 1 -- 172.21.15.16:0/2993103777 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c3009e850 msgr2=0x7f2c3009ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:23.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.327+0000 7f2c34daf700 1 --2- 172.21.15.16:0/2993103777 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c3009e850 0x7f2c3009ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:23.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.327+0000 7f2c34daf700 1 -- 172.21.15.16:0/2993103777 >> v1:172.21.15.16:6789/0 conn(0x7f2c3009f1c0 legacy=0x7f2c301389c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:23.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.327+0000 7f2c34daf700 1 -- 172.21.15.16:0/2993103777 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c300a07a0 con 0x7f2c30131b30 2023-09-30T22:29:23.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.327+0000 7f2c2ffff700 1 --2- 172.21.15.16:0/2993103777 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c3009e850 0x7f2c3009ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:23.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.327+0000 7f2c34daf700 1 --2- 172.21.15.16:0/2993103777 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c30131b30 0x7f2c30131f00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2c20009870 tx=0x7f2c20009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22cdb37ed2504510 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:23.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.327+0000 7f2c2f7fe700 1 -- 172.21.15.16:0/2993103777 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c20011830 con 0x7f2c30131b30 2023-09-30T22:29:23.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.327+0000 7f2c2f7fe700 1 -- 172.21.15.16:0/2993103777 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c20011990 con 0x7f2c30131b30 2023-09-30T22:29:23.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.327+0000 7f2c2f7fe700 1 -- 172.21.15.16:0/2993103777 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c20011d20 con 0x7f2c30131b30 2023-09-30T22:29:23.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.328+0000 7f2c37013700 1 -- 172.21.15.16:0/2993103777 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c30131b30 msgr2=0x7f2c30131f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:23.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.328+0000 7f2c37013700 1 --2- 172.21.15.16:0/2993103777 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c30131b30 0x7f2c30131f00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2c20009870 tx=0x7f2c20009b80 comp rx=0 tx=0).stop 2023-09-30T22:29:23.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.328+0000 7f2c37013700 1 -- 172.21.15.16:0/2993103777 shutdown_connections 2023-09-30T22:29:23.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.328+0000 7f2c37013700 1 --2- 172.21.15.16:0/2993103777 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c30131b30 0x7f2c30131f00 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:23.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.328+0000 7f2c37013700 1 --2- 172.21.15.16:0/2993103777 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c3009e850 0x7f2c3009ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:23.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.328+0000 7f2c37013700 1 -- 172.21.15.16:0/2993103777 >> 172.21.15.16:0/2993103777 conn(0x7f2c300a4130 msgr2=0x7f2c300a4530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:23.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.328+0000 7f2c37013700 1 -- 172.21.15.16:0/2993103777 shutdown_connections 2023-09-30T22:29:23.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.328+0000 7f2c37013700 1 -- 172.21.15.16:0/2993103777 wait complete. 2023-09-30T22:29:23.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.329+0000 7f2c37013700 1 Processor -- start 2023-09-30T22:29:23.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.329+0000 7f2c37013700 1 -- start start 2023-09-30T22:29:23.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.329+0000 7f2c37013700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c3009e850 0x7f2c3011b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:23.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.329+0000 7f2c37013700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c3009f1c0 0x7f2c3011b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:23.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.330+0000 7f2c37013700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c30131b30 0x7f2c3011baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:23.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.330+0000 7f2c37013700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3011d280 con 0x7f2c30131b30 2023-09-30T22:29:23.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.330+0000 7f2c37013700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3011eb40 con 0x7f2c3009f1c0 2023-09-30T22:29:23.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.330+0000 7f2c37013700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3011df10 con 0x7f2c3009e850 2023-09-30T22:29:23.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.330+0000 7f2c2ffff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c3009f1c0 0x7f2c3011b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:23.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.330+0000 7f2c34daf700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c3009e850 0x7f2c3011b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:23.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.330+0000 7f2c2ffff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c3009f1c0 0x7f2c3011b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37256/0 (socket says 172.21.15.16:37256) 2023-09-30T22:29:23.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.330+0000 7f2c2ffff700 1 -- 172.21.15.16:0/1012747587 learned_addr learned my addr 172.21.15.16:0/1012747587 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:23.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.330+0000 7f2c355b0700 1 --2- 172.21.15.16:0/1012747587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c30131b30 0x7f2c3011baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:23.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.330+0000 7f2c2ffff700 1 -- 172.21.15.16:0/1012747587 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c3009e850 msgr2=0x7f2c3011b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:23.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.330+0000 7f2c2ffff700 1 --2- 172.21.15.16:0/1012747587 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c3009e850 0x7f2c3011b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:23.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.330+0000 7f2c2ffff700 1 -- 172.21.15.16:0/1012747587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c30131b30 msgr2=0x7f2c3011baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:23.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.330+0000 7f2c2ffff700 1 --2- 172.21.15.16:0/1012747587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c30131b30 0x7f2c3011baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:23.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.330+0000 7f2c2ffff700 1 -- 172.21.15.16:0/1012747587 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c180094a0 con 0x7f2c3009f1c0 2023-09-30T22:29:23.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.330+0000 7f2c2ffff700 1 --2- 172.21.15.16:0/1012747587 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c3009f1c0 0x7f2c3011b5b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2c1800ebb0 tx=0x7f2c1800eec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:23.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.330+0000 7f2c2d7fa700 1 -- 172.21.15.16:0/1012747587 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c1800c850 con 0x7f2c3009f1c0 2023-09-30T22:29:23.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.330+0000 7f2c2d7fa700 1 -- 172.21.15.16:0/1012747587 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c1800c9b0 con 0x7f2c3009f1c0 2023-09-30T22:29:23.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.330+0000 7f2c37013700 1 -- 172.21.15.16:0/1012747587 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c200094a0 con 0x7f2c3009f1c0 2023-09-30T22:29:23.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.330+0000 7f2c2d7fa700 1 -- 172.21.15.16:0/1012747587 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c1800cd40 con 0x7f2c3009f1c0 2023-09-30T22:29:23.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.330+0000 7f2c37013700 1 -- 172.21.15.16:0/1012747587 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c3011cc30 con 0x7f2c3009f1c0 2023-09-30T22:29:23.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.331+0000 7f2c2d7fa700 1 -- 172.21.15.16:0/1012747587 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f2c18013470 con 0x7f2c3009f1c0 2023-09-30T22:29:23.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.331+0000 7f2c2d7fa700 1 -- 172.21.15.16:0/1012747587 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4049+0+0 (secure 0 0 0) 0x7f2c1803fc50 con 0x7f2c3009f1c0 2023-09-30T22:29:23.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.332+0000 7f2c37013700 1 -- 172.21.15.16:0/1012747587 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c3004f2a0 con 0x7f2c3009f1c0 2023-09-30T22:29:23.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.336+0000 7f2c2d7fa700 1 -- 172.21.15.16:0/1012747587 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2c18017080 con 0x7f2c3009f1c0 2023-09-30T22:29:23.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.541+0000 7f2c37013700 1 -- 172.21.15.16:0/1012747587 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2c300728b0 con 0x7f2c3009f1c0 2023-09-30T22:29:23.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.542+0000 7f2c2d7fa700 1 -- 172.21.15.16:0/1012747587 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2c180232f0 con 0x7f2c3009f1c0 2023-09-30T22:29:23.542 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-09-30T22:29:23.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.544+0000 7f2c37013700 1 -- 172.21.15.16:0/1012747587 >> v1:172.21.15.16:6816/126811 conn(0x7f2c1002f540 legacy=0x7f2c100319f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:23.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.544+0000 7f2c37013700 1 -- 172.21.15.16:0/1012747587 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c3009f1c0 msgr2=0x7f2c3011b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:23.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.544+0000 7f2c37013700 1 --2- 172.21.15.16:0/1012747587 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c3009f1c0 0x7f2c3011b5b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2c1800ebb0 tx=0x7f2c1800eec0 comp rx=0 tx=0).stop 2023-09-30T22:29:23.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.544+0000 7f2c37013700 1 -- 172.21.15.16:0/1012747587 shutdown_connections 2023-09-30T22:29:23.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.544+0000 7f2c37013700 1 --2- 172.21.15.16:0/1012747587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c30131b30 0x7f2c3011baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:23.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.544+0000 7f2c37013700 1 --2- 172.21.15.16:0/1012747587 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c3009e850 0x7f2c3011b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:23.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.544+0000 7f2c37013700 1 --2- 172.21.15.16:0/1012747587 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c3009f1c0 0x7f2c3011b5b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:23.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.544+0000 7f2c37013700 1 -- 172.21.15.16:0/1012747587 >> 172.21.15.16:0/1012747587 conn(0x7f2c300a4130 msgr2=0x7f2c30131050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:23.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.544+0000 7f2c37013700 1 -- 172.21.15.16:0/1012747587 shutdown_connections 2023-09-30T22:29:23.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.544+0000 7f2c37013700 1 -- 172.21.15.16:0/1012747587 wait complete. 2023-09-30T22:29:23.558 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-30T22:29:23.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.694+0000 7fd615e5f700 1 Processor -- start 2023-09-30T22:29:23.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.694+0000 7fd615e5f700 1 -- start start 2023-09-30T22:29:23.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.694+0000 7fd615e5f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6100a0a30 0x7fd6100a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:23.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.694+0000 7fd615e5f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd6100a1340 0x7fd610133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:23.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.694+0000 7fd615e5f700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd61009edd0 con 0x7fd610133fa0 2023-09-30T22:29:23.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.694+0000 7fd615e5f700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd61009e1f0 con 0x7fd6100a1340 2023-09-30T22:29:23.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.694+0000 7fd615e5f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd61009dc50 con 0x7fd6100a0a30 2023-09-30T22:29:23.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.694+0000 7fd60f7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6100a0a30 0x7fd6100a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:23.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.694+0000 7fd60effd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd6100a1340 0x7fd610133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:23.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.694+0000 7fd60f7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6100a0a30 0x7fd6100a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37262/0 (socket says 172.21.15.16:37262) 2023-09-30T22:29:23.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.694+0000 7fd60ffff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd610133fa0 0x7fd61013ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33888/0 (socket says 172.21.15.16:33888) 2023-09-30T22:29:23.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.694+0000 7fd60effd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd6100a1340 0x7fd610133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57846/0 (socket says 172.21.15.16:57846) 2023-09-30T22:29:23.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.694+0000 7fd60f7fe700 1 -- 172.21.15.16:0/1124201695 learned_addr learned my addr 172.21.15.16:0/1124201695 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:23.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.695+0000 7fd60f7fe700 1 -- 172.21.15.16:0/1124201695 >> v2:172.21.15.16:3300/0 conn(0x7fd6100a1340 msgr2=0x7fd610133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:23.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.695+0000 7fd60f7fe700 1 --2- 172.21.15.16:0/1124201695 >> v2:172.21.15.16:3300/0 conn(0x7fd6100a1340 0x7fd610133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:23.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.695+0000 7fd60f7fe700 1 -- 172.21.15.16:0/1124201695 >> v1:172.21.15.16:6789/0 conn(0x7fd610133fa0 legacy=0x7fd61013ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:23.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.695+0000 7fd60f7fe700 1 -- 172.21.15.16:0/1124201695 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6100a1fe0 con 0x7fd6100a0a30 2023-09-30T22:29:23.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.695+0000 7fd60e7fc700 1 -- 172.21.15.16:0/1124201695 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 531311531 0 0) 0x7fd61009edd0 con 0x7fd610133fa0 2023-09-30T22:29:23.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.695+0000 7fd60effd700 1 --2- 172.21.15.16:0/1124201695 >> v2:172.21.15.16:3300/0 conn(0x7fd6100a1340 0x7fd610133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:23.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.695+0000 7fd60f7fe700 1 --2- 172.21.15.16:0/1124201695 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6100a0a30 0x7fd6100a0e00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd6000098f0 tx=0x7fd600009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4be7b56502ce72a7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:23.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.695+0000 7fd60e7fc700 1 -- 172.21.15.16:0/1124201695 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd600011830 con 0x7fd6100a0a30 2023-09-30T22:29:23.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.695+0000 7fd60e7fc700 1 -- 172.21.15.16:0/1124201695 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd600011990 con 0x7fd6100a0a30 2023-09-30T22:29:23.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.695+0000 7fd60e7fc700 1 -- 172.21.15.16:0/1124201695 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd600011d20 con 0x7fd6100a0a30 2023-09-30T22:29:23.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.696+0000 7fd615e5f700 1 -- 172.21.15.16:0/1124201695 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6100a0a30 msgr2=0x7fd6100a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:23.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.696+0000 7fd615e5f700 1 --2- 172.21.15.16:0/1124201695 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6100a0a30 0x7fd6100a0e00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd6000098f0 tx=0x7fd600009c00 comp rx=0 tx=0).stop 2023-09-30T22:29:23.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.696+0000 7fd615e5f700 1 -- 172.21.15.16:0/1124201695 shutdown_connections 2023-09-30T22:29:23.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.696+0000 7fd615e5f700 1 --2- 172.21.15.16:0/1124201695 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6100a0a30 0x7fd6100a0e00 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:23.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.696+0000 7fd615e5f700 1 --2- 172.21.15.16:0/1124201695 >> v2:172.21.15.16:3300/0 conn(0x7fd6100a1340 0x7fd610133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:23.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.696+0000 7fd615e5f700 1 -- 172.21.15.16:0/1124201695 >> 172.21.15.16:0/1124201695 conn(0x7fd6100a6550 msgr2=0x7fd6100a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:23.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.696+0000 7fd615e5f700 1 -- 172.21.15.16:0/1124201695 shutdown_connections 2023-09-30T22:29:23.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.696+0000 7fd615e5f700 1 -- 172.21.15.16:0/1124201695 wait complete. 2023-09-30T22:29:23.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.697+0000 7fd615e5f700 1 Processor -- start 2023-09-30T22:29:23.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.697+0000 7fd615e5f700 1 -- start start 2023-09-30T22:29:23.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.697+0000 7fd615e5f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd6100a0a30 0x7fd610122790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:23.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.697+0000 7fd615e5f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6100a1340 0x7fd610122cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:23.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.697+0000 7fd615e5f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd610133fa0 0x7fd610123210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:23.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.698+0000 7fd615e5f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd610125c80 con 0x7fd610133fa0 2023-09-30T22:29:23.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.698+0000 7fd615e5f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd610123b80 con 0x7fd6100a1340 2023-09-30T22:29:23.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.698+0000 7fd615e5f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd6101243b0 con 0x7fd6100a0a30 2023-09-30T22:29:23.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.698+0000 7fd60effd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6100a1340 0x7fd610122cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:23.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.698+0000 7fd60ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd610133fa0 0x7fd610123210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:23.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.698+0000 7fd60f7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd6100a0a30 0x7fd610122790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:23.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.698+0000 7fd60effd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6100a1340 0x7fd610122cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37264/0 (socket says 172.21.15.16:37264) 2023-09-30T22:29:23.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.698+0000 7fd60effd700 1 -- 172.21.15.16:0/2972553235 learned_addr learned my addr 172.21.15.16:0/2972553235 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:23.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.698+0000 7fd60effd700 1 -- 172.21.15.16:0/2972553235 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd6100a0a30 msgr2=0x7fd610122790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:23.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.698+0000 7fd60effd700 1 --2- 172.21.15.16:0/2972553235 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd6100a0a30 0x7fd610122790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:23.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.698+0000 7fd60effd700 1 -- 172.21.15.16:0/2972553235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd610133fa0 msgr2=0x7fd610123210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:23.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.698+0000 7fd60effd700 1 --2- 172.21.15.16:0/2972553235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd610133fa0 0x7fd610123210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:23.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.698+0000 7fd60effd700 1 -- 172.21.15.16:0/2972553235 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6040094a0 con 0x7fd6100a1340 2023-09-30T22:29:23.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.698+0000 7fd60f7fe700 1 --2- 172.21.15.16:0/2972553235 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd6100a0a30 0x7fd610122790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:23.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.698+0000 7fd60effd700 1 --2- 172.21.15.16:0/2972553235 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6100a1340 0x7fd610122cd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd60400ebe0 tx=0x7fd60400eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:23.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.698+0000 7fd5f7fff700 1 -- 172.21.15.16:0/2972553235 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60400c850 con 0x7fd6100a1340 2023-09-30T22:29:23.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.698+0000 7fd5f7fff700 1 -- 172.21.15.16:0/2972553235 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd60400c9b0 con 0x7fd6100a1340 2023-09-30T22:29:23.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.698+0000 7fd5f7fff700 1 -- 172.21.15.16:0/2972553235 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60400cd40 con 0x7fd6100a1340 2023-09-30T22:29:23.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.698+0000 7fd615e5f700 1 -- 172.21.15.16:0/2972553235 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd600009550 con 0x7fd6100a1340 2023-09-30T22:29:23.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.699+0000 7fd615e5f700 1 -- 172.21.15.16:0/2972553235 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd610133200 con 0x7fd6100a1340 2023-09-30T22:29:23.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.699+0000 7fd5f7fff700 1 -- 172.21.15.16:0/2972553235 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fd604013470 con 0x7fd6100a1340 2023-09-30T22:29:23.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.699+0000 7fd5f7fff700 1 -- 172.21.15.16:0/2972553235 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4049+0+0 (secure 0 0 0) 0x7fd604040080 con 0x7fd6100a1340 2023-09-30T22:29:23.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.700+0000 7fd615e5f700 1 -- 172.21.15.16:0/2972553235 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6100a1d40 con 0x7fd6100a1340 2023-09-30T22:29:23.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.705+0000 7fd5f7fff700 1 -- 172.21.15.16:0/2972553235 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd604021080 con 0x7fd6100a1340 2023-09-30T22:29:23.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.963+0000 7fd615e5f700 1 -- 172.21.15.16:0/2972553235 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd610125df0 con 0x7fd6100a1340 2023-09-30T22:29:23.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.964+0000 7fd5f7fff700 1 -- 172.21.15.16:0/2972553235 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+2315 (secure 0 0 0) 0x7fd604017020 con 0x7fd6100a1340 2023-09-30T22:29:23.964 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:29:23.964 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-30T22:26:51.674561+0000","modified":"2023-09-30T22:27:18.263912+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4218},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4218":{"gid":4218,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6817/1424366610","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6817","nonce":1424366610}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-30T22:29:23.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.966+0000 7fd615e5f700 1 -- 172.21.15.16:0/2972553235 >> v1:172.21.15.16:6816/126811 conn(0x7fd5fc02f510 legacy=0x7fd5fc0319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:23.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.966+0000 7fd615e5f700 1 -- 172.21.15.16:0/2972553235 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6100a1340 msgr2=0x7fd610122cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:23.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.966+0000 7fd615e5f700 1 --2- 172.21.15.16:0/2972553235 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6100a1340 0x7fd610122cd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd60400ebe0 tx=0x7fd60400eef0 comp rx=0 tx=0).stop 2023-09-30T22:29:23.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.966+0000 7fd615e5f700 1 -- 172.21.15.16:0/2972553235 shutdown_connections 2023-09-30T22:29:23.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.966+0000 7fd615e5f700 1 --2- 172.21.15.16:0/2972553235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd610133fa0 0x7fd610123210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:23.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.966+0000 7fd615e5f700 1 --2- 172.21.15.16:0/2972553235 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd6100a0a30 0x7fd610122790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:23.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.966+0000 7fd615e5f700 1 --2- 172.21.15.16:0/2972553235 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6100a1340 0x7fd610122cd0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:23.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.966+0000 7fd615e5f700 1 -- 172.21.15.16:0/2972553235 >> 172.21.15.16:0/2972553235 conn(0x7fd6100a6550 msgr2=0x7fd6100a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:23.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.966+0000 7fd615e5f700 1 -- 172.21.15.16:0/2972553235 shutdown_connections 2023-09-30T22:29:23.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:23.966+0000 7fd615e5f700 1 -- 172.21.15.16:0/2972553235 wait complete. 2023-09-30T22:29:23.970 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2023-09-30T22:29:23.980 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:29:24.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.115+0000 7fd1e8e17700 1 Processor -- start 2023-09-30T22:29:24.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.115+0000 7fd1e8e17700 1 -- start start 2023-09-30T22:29:24.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.115+0000 7fd1e8e17700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd1e4133420 0x7fd1e41337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:24.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.116+0000 7fd1e8e17700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd1e4133e30 0x7fd1e412ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:24.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.116+0000 7fd1e8e17700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1e406b990 con 0x7fd1e4134260 2023-09-30T22:29:24.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.116+0000 7fd1e8e17700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e40a0590 con 0x7fd1e4133e30 2023-09-30T22:29:24.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.116+0000 7fd1e8e17700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e409fb80 con 0x7fd1e4133420 2023-09-30T22:29:24.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.116+0000 7fd1e1d9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd1e4133e30 0x7fd1e412ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:24.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.116+0000 7fd1e259c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd1e4133420 0x7fd1e41337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:24.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.116+0000 7fd1e259c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd1e4133420 0x7fd1e41337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:55984/0 (socket says 172.21.15.16:55984) 2023-09-30T22:29:24.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.116+0000 7fd1e1d9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd1e4133e30 0x7fd1e412ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37268/0 (socket says 172.21.15.16:37268) 2023-09-30T22:29:24.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.116+0000 7fd1e259c700 1 -- 172.21.15.16:0/2446757458 learned_addr learned my addr 172.21.15.16:0/2446757458 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:24.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.116+0000 7fd1e259c700 1 -- 172.21.15.16:0/2446757458 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd1e4133e30 msgr2=0x7fd1e412ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:24.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.116+0000 7fd1e159a700 1 -- 172.21.15.16:0/2446757458 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2183677857 0 0) 0x7fd1e406b990 con 0x7fd1e4134260 2023-09-30T22:29:24.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.116+0000 7fd1e259c700 1 --2- 172.21.15.16:0/2446757458 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd1e4133e30 0x7fd1e412ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:24.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.116+0000 7fd1e259c700 1 -- 172.21.15.16:0/2446757458 >> v1:172.21.15.16:6789/0 conn(0x7fd1e4134260 legacy=0x7fd1e41304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:24.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.116+0000 7fd1e259c700 1 -- 172.21.15.16:0/2446757458 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1e40a0030 con 0x7fd1e4133420 2023-09-30T22:29:24.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.116+0000 7fd1e1d9b700 1 --2- 172.21.15.16:0/2446757458 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd1e4133e30 0x7fd1e412ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:24.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.116+0000 7fd1e259c700 1 --2- 172.21.15.16:0/2446757458 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd1e4133420 0x7fd1e41337f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8009920 tx=0x7fd1d8009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52cb4784f45d16f8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:24.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.116+0000 7fd1e159a700 1 -- 172.21.15.16:0/2446757458 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d8011830 con 0x7fd1e4133420 2023-09-30T22:29:24.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.116+0000 7fd1e159a700 1 -- 172.21.15.16:0/2446757458 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1d8011990 con 0x7fd1e4133420 2023-09-30T22:29:24.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.116+0000 7fd1e159a700 1 -- 172.21.15.16:0/2446757458 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d8011d20 con 0x7fd1e4133420 2023-09-30T22:29:24.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.117+0000 7fd1e8e17700 1 -- 172.21.15.16:0/2446757458 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd1e4133420 msgr2=0x7fd1e41337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:24.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.117+0000 7fd1e8e17700 1 --2- 172.21.15.16:0/2446757458 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd1e4133420 0x7fd1e41337f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8009920 tx=0x7fd1d8009c30 comp rx=0 tx=0).stop 2023-09-30T22:29:24.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.117+0000 7fd1e8e17700 1 -- 172.21.15.16:0/2446757458 shutdown_connections 2023-09-30T22:29:24.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.117+0000 7fd1e8e17700 1 --2- 172.21.15.16:0/2446757458 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd1e4133420 0x7fd1e41337f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:24.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.117+0000 7fd1e8e17700 1 --2- 172.21.15.16:0/2446757458 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd1e4133e30 0x7fd1e412ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:24.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.117+0000 7fd1e8e17700 1 -- 172.21.15.16:0/2446757458 >> 172.21.15.16:0/2446757458 conn(0x7fd1e40a4210 msgr2=0x7fd1e40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:24.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.117+0000 7fd1e8e17700 1 -- 172.21.15.16:0/2446757458 shutdown_connections 2023-09-30T22:29:24.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.117+0000 7fd1e8e17700 1 -- 172.21.15.16:0/2446757458 wait complete. 2023-09-30T22:29:24.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.118+0000 7fd1e8e17700 1 Processor -- start 2023-09-30T22:29:24.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.118+0000 7fd1e8e17700 1 -- start start 2023-09-30T22:29:24.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.119+0000 7fd1e8e17700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd1e4133420 0x7fd1e409c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:24.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.119+0000 7fd1e8e17700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1e4133e30 0x7fd1e409c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:24.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.119+0000 7fd1e8e17700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd1e4134260 0x7fd1e409cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:24.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.119+0000 7fd1e8e17700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e411fcb0 con 0x7fd1e4133e30 2023-09-30T22:29:24.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.119+0000 7fd1e8e17700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e409ded0 con 0x7fd1e4134260 2023-09-30T22:29:24.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.119+0000 7fd1e8e17700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd1e4131570 con 0x7fd1e4133420 2023-09-30T22:29:24.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.119+0000 7fd1e259c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd1e4133420 0x7fd1e409c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:24.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.119+0000 7fd1e2d9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd1e4134260 0x7fd1e409cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:24.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.119+0000 7fd1e259c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd1e4133420 0x7fd1e409c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:55990/0 (socket says 172.21.15.16:55990) 2023-09-30T22:29:24.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.119+0000 7fd1e259c700 1 -- 172.21.15.16:0/1780558702 learned_addr learned my addr 172.21.15.16:0/1780558702 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:24.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.119+0000 7fd1e1d9b700 1 --2- 172.21.15.16:0/1780558702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1e4133e30 0x7fd1e409c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:24.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.119+0000 7fd1e2d9d700 1 -- 172.21.15.16:0/1780558702 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd1e4133420 msgr2=0x7fd1e409c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:24.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.119+0000 7fd1e2d9d700 1 --2- 172.21.15.16:0/1780558702 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd1e4133420 0x7fd1e409c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:24.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.119+0000 7fd1e2d9d700 1 -- 172.21.15.16:0/1780558702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1e4133e30 msgr2=0x7fd1e409c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:24.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.119+0000 7fd1e2d9d700 1 --2- 172.21.15.16:0/1780558702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1e4133e30 0x7fd1e409c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:24.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.119+0000 7fd1e2d9d700 1 -- 172.21.15.16:0/1780558702 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1d40094a0 con 0x7fd1e4134260 2023-09-30T22:29:24.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.119+0000 7fd1e259c700 1 --2- 172.21.15.16:0/1780558702 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd1e4133420 0x7fd1e409c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:24.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.120+0000 7fd1e2d9d700 1 --2- 172.21.15.16:0/1780558702 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd1e4134260 0x7fd1e409cf00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc00c7c0 tx=0x7fd1dc00cad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:24.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.120+0000 7fd1d2ffd700 1 -- 172.21.15.16:0/1780558702 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc003e30 con 0x7fd1e4134260 2023-09-30T22:29:24.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.120+0000 7fd1d2ffd700 1 -- 172.21.15.16:0/1780558702 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1dc005070 con 0x7fd1e4134260 2023-09-30T22:29:24.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.120+0000 7fd1e8e17700 1 -- 172.21.15.16:0/1780558702 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1d8009550 con 0x7fd1e4134260 2023-09-30T22:29:24.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.120+0000 7fd1d2ffd700 1 -- 172.21.15.16:0/1780558702 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1dc005420 con 0x7fd1e4134260 2023-09-30T22:29:24.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.120+0000 7fd1e8e17700 1 -- 172.21.15.16:0/1780558702 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1e411fe90 con 0x7fd1e4134260 2023-09-30T22:29:24.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.121+0000 7fd1d2ffd700 1 -- 172.21.15.16:0/1780558702 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fd1dc005580 con 0x7fd1e4134260 2023-09-30T22:29:24.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.121+0000 7fd1e8e17700 1 -- 172.21.15.16:0/1780558702 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1e4074ea0 con 0x7fd1e4134260 2023-09-30T22:29:24.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.121+0000 7fd1d2ffd700 1 -- 172.21.15.16:0/1780558702 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4049+0+0 (secure 0 0 0) 0x7fd1dc03f1b0 con 0x7fd1e4134260 2023-09-30T22:29:24.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.126+0000 7fd1d2ffd700 1 -- 172.21.15.16:0/1780558702 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd1dc010b80 con 0x7fd1e4134260 2023-09-30T22:29:24.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.330+0000 7fd1e8e17700 1 -- 172.21.15.16:0/1780558702 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd1e411f4d0 con 0x7fd1e4134260 2023-09-30T22:29:24.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.331+0000 7fd1d2ffd700 1 -- 172.21.15.16:0/1780558702 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+12914 (secure 0 0 0) 0x7fd1e411f4d0 con 0x7fd1e4134260 2023-09-30T22:29:24.331 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:29:24.332 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":22,"fsid":"7ede3e14-2bd0-4726-9437-6d13c90cd342","created":"2023-09-30T22:26:35.483697+0000","modified":"2023-09-30T22:29:16.486363+0000","last_up_change":"2023-09-30T22:26:44.227199+0000","last_in_change":"2023-09-30T22:26:37.120906+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"luminous","require_osd_release":"pacific","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T22:26:44.285133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{},"mgr_devicehealth":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:26:45.132921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:26:49.353683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:26:50.655685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"19946985-0ee2-487f-9347-929670dbe5ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6804","nonce":118824}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6805","nonce":118824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6807","nonce":118824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6806","nonce":118824}]},"public_addr":"172.21.15.16:6804/118824","cluster_addr":"172.21.15.16:6805/118824","heartbeat_back_addr":"172.21.15.16:6807/118824","heartbeat_front_addr":"172.21.15.16:6806/118824","state":["exists","up"]},{"osd":1,"uuid":"8efd4dbc-6ae4-4b82-b530-3b3eac2be445","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6812","nonce":118832}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6813","nonce":118832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6815","nonce":118832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6814","nonce":118832}]},"public_addr":"172.21.15.16:6812/118832","cluster_addr":"172.21.15.16:6813/118832","heartbeat_back_addr":"172.21.15.16:6815/118832","heartbeat_front_addr":"172.21.15.16:6814/118832","state":["exists","up"]},{"osd":2,"uuid":"b47bb1ce-27af-4c0c-868a-b76ebd0d9499","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6808","nonce":118840}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6809","nonce":118840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6811","nonce":118840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6810","nonce":118840}]},"public_addr":"172.21.15.16:6808/118840","cluster_addr":"172.21.15.16:6809/118840","heartbeat_back_addr":"172.21.15.16:6811/118840","heartbeat_front_addr":"172.21.15.16:6810/118840","state":["exists","up"]},{"osd":3,"uuid":"03b53d64-d10d-448a-a081-39ecceab589b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6800","nonce":118823}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6801","nonce":118823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6803","nonce":118823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6802","nonce":118823}]},"public_addr":"172.21.15.16:6800/118823","cluster_addr":"172.21.15.16:6801/118823","heartbeat_back_addr":"172.21.15.16:6803/118823","heartbeat_front_addr":"172.21.15.16:6802/118823","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.305953+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.307488+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.341023+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.192071+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.8","mappings":[{"from":3,"to":2}]},{"pgid":"4.d","mappings":[{"from":3,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":0}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]},{"pgid":"4.17","mappings":[{"from":3,"to":2}]},{"pgid":"4.19","mappings":[{"from":3,"to":0}]},{"pgid":"4.3a","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6816/118285":"2023-10-01T22:29:14.475978+0000","172.21.15.16:0/2249754484":"2023-10-01T22:29:14.475978+0000","172.21.15.16:0/1101648925":"2023-10-01T22:29:14.475978+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:29:24.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.333+0000 7fd1e8e17700 1 -- 172.21.15.16:0/1780558702 >> v1:172.21.15.16:6816/126811 conn(0x7fd1c402f510 legacy=0x7fd1c40319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:24.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.333+0000 7fd1e8e17700 1 -- 172.21.15.16:0/1780558702 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd1e4134260 msgr2=0x7fd1e409cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:24.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.333+0000 7fd1e8e17700 1 --2- 172.21.15.16:0/1780558702 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd1e4134260 0x7fd1e409cf00 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc00c7c0 tx=0x7fd1dc00cad0 comp rx=0 tx=0).stop 2023-09-30T22:29:24.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.334+0000 7fd1e8e17700 1 -- 172.21.15.16:0/1780558702 shutdown_connections 2023-09-30T22:29:24.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.334+0000 7fd1e8e17700 1 --2- 172.21.15.16:0/1780558702 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd1e4134260 0x7fd1e409cf00 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:24.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.334+0000 7fd1e8e17700 1 --2- 172.21.15.16:0/1780558702 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd1e4133420 0x7fd1e409c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:24.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.334+0000 7fd1e8e17700 1 --2- 172.21.15.16:0/1780558702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1e4133e30 0x7fd1e409c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:24.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.334+0000 7fd1e8e17700 1 -- 172.21.15.16:0/1780558702 >> 172.21.15.16:0/1780558702 conn(0x7fd1e40a4210 msgr2=0x7fd1e40a3090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:24.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.334+0000 7fd1e8e17700 1 -- 172.21.15.16:0/1780558702 shutdown_connections 2023-09-30T22:29:24.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.334+0000 7fd1e8e17700 1 -- 172.21.15.16:0/1780558702 wait complete. 2023-09-30T22:29:24.347 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-09-30T22:29:24.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.483+0000 7f65ecbbd700 1 Processor -- start 2023-09-30T22:29:24.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.483+0000 7f65ecbbd700 1 -- start start 2023-09-30T22:29:24.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.484+0000 7f65ecbbd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f65e809d6f0 0x7f65e809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:24.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.484+0000 7f65ecbbd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f65e809e000 0x7f65e81339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:24.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.484+0000 7f65ecbbd700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65e809b120 con 0x7f65e809e410 2023-09-30T22:29:24.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.484+0000 7f65ecbbd700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65e806b460 con 0x7f65e809e000 2023-09-30T22:29:24.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.484+0000 7f65ecbbd700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f65e809a7e0 con 0x7f65e809d6f0 2023-09-30T22:29:24.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.484+0000 7f65e5d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f65e809e000 0x7f65e81339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:24.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.484+0000 7f65e659c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f65e809d6f0 0x7f65e809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:24.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.484+0000 7f65e6d9d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f65e809e410 0x7f65e8133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33908/0 (socket says 172.21.15.16:33908) 2023-09-30T22:29:24.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.484+0000 7f65e6d9d700 1 -- 172.21.15.16:0/994116640 learned_addr learned my addr 172.21.15.16:0/994116640 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:29:24.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.485+0000 7f65e5d9b700 1 -- 172.21.15.16:0/994116640 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f65e809d6f0 msgr2=0x7f65e809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:24.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.485+0000 7f65e5d9b700 1 --2- 172.21.15.16:0/994116640 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f65e809d6f0 0x7f65e809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:24.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.485+0000 7f65e5d9b700 1 -- 172.21.15.16:0/994116640 >> v1:172.21.15.16:6789/0 conn(0x7f65e809e410 legacy=0x7f65e8133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:24.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.485+0000 7f65e5d9b700 1 -- 172.21.15.16:0/994116640 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e809a500 con 0x7f65e809e000 2023-09-30T22:29:24.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.485+0000 7f65e559a700 1 -- 172.21.15.16:0/994116640 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1677163432 0 0) 0x7f65e809b120 con 0x7f65e809e410 2023-09-30T22:29:24.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.485+0000 7f65e659c700 1 --2- 172.21.15.16:0/994116640 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f65e809d6f0 0x7f65e809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:24.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.485+0000 7f65e5d9b700 1 --2- 172.21.15.16:0/994116640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65e809e000 0x7f65e81339f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f65d0009840 tx=0x7f65d0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f578b524b99a19a0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:24.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.485+0000 7f65e559a700 1 -- 172.21.15.16:0/994116640 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d0011830 con 0x7f65e809e000 2023-09-30T22:29:24.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.485+0000 7f65e559a700 1 -- 172.21.15.16:0/994116640 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65d0011990 con 0x7f65e809e000 2023-09-30T22:29:24.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.485+0000 7f65e559a700 1 -- 172.21.15.16:0/994116640 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d0011d20 con 0x7f65e809e000 2023-09-30T22:29:24.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.486+0000 7f65ecbbd700 1 -- 172.21.15.16:0/994116640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65e809e000 msgr2=0x7f65e81339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:24.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.486+0000 7f65ecbbd700 1 --2- 172.21.15.16:0/994116640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65e809e000 0x7f65e81339f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f65d0009840 tx=0x7f65d0009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:24.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.486+0000 7f65ecbbd700 1 -- 172.21.15.16:0/994116640 shutdown_connections 2023-09-30T22:29:24.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.486+0000 7f65ecbbd700 1 --2- 172.21.15.16:0/994116640 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f65e809d6f0 0x7f65e809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:24.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.486+0000 7f65ecbbd700 1 --2- 172.21.15.16:0/994116640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65e809e000 0x7f65e81339f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:24.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.486+0000 7f65ecbbd700 1 -- 172.21.15.16:0/994116640 >> 172.21.15.16:0/994116640 conn(0x7f65e8066f50 msgr2=0x7f65e809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:24.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.486+0000 7f65ecbbd700 1 -- 172.21.15.16:0/994116640 shutdown_connections 2023-09-30T22:29:24.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.486+0000 7f65ecbbd700 1 -- 172.21.15.16:0/994116640 wait complete. 2023-09-30T22:29:24.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.487+0000 7f65ecbbd700 1 Processor -- start 2023-09-30T22:29:24.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.487+0000 7f65ecbbd700 1 -- start start 2023-09-30T22:29:24.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.487+0000 7f65ecbbd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f65e809d6f0 0x7f65e81a5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:24.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.488+0000 7f65ecbbd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f65e809e000 0x7f65e81a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:24.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.488+0000 7f65ecbbd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65e809e410 0x7f65e81a6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:24.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.488+0000 7f65ecbbd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e81a9150 con 0x7f65e809e410 2023-09-30T22:29:24.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.488+0000 7f65ecbbd700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65e809c710 con 0x7f65e809e000 2023-09-30T22:29:24.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.488+0000 7f65ecbbd700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f65e80a2e40 con 0x7f65e809d6f0 2023-09-30T22:29:24.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.488+0000 7f65e5d9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f65e809e000 0x7f65e81a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:24.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.488+0000 7f65e6d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65e809e410 0x7f65e81a6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:24.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.488+0000 7f65e5d9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f65e809e000 0x7f65e81a63e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37290/0 (socket says 172.21.15.16:37290) 2023-09-30T22:29:24.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.488+0000 7f65e5d9b700 1 -- 172.21.15.16:0/1192783746 learned_addr learned my addr 172.21.15.16:0/1192783746 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:24.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.488+0000 7f65e659c700 1 --2- 172.21.15.16:0/1192783746 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f65e809d6f0 0x7f65e81a5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:24.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.488+0000 7f65e6d9d700 1 -- 172.21.15.16:0/1192783746 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f65e809d6f0 msgr2=0x7f65e81a5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:24.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.488+0000 7f65e6d9d700 1 --2- 172.21.15.16:0/1192783746 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f65e809d6f0 0x7f65e81a5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:24.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.488+0000 7f65e6d9d700 1 -- 172.21.15.16:0/1192783746 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f65e809e000 msgr2=0x7f65e81a63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:24.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.488+0000 7f65e6d9d700 1 --2- 172.21.15.16:0/1192783746 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f65e809e000 0x7f65e81a63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:24.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.488+0000 7f65e6d9d700 1 -- 172.21.15.16:0/1192783746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65d80094a0 con 0x7f65e809e410 2023-09-30T22:29:24.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.488+0000 7f65e5d9b700 1 --2- 172.21.15.16:0/1192783746 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f65e809e000 0x7f65e81a63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:24.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.488+0000 7f65e6d9d700 1 --2- 172.21.15.16:0/1192783746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65e809e410 0x7f65e81a6920 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f65e000cbe0 tx=0x7f65e000cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:24.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.488+0000 7f65deffd700 1 -- 172.21.15.16:0/1192783746 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e000aa40 con 0x7f65e809e410 2023-09-30T22:29:24.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.488+0000 7f65deffd700 1 -- 172.21.15.16:0/1192783746 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65e000abc0 con 0x7f65e809e410 2023-09-30T22:29:24.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.488+0000 7f65ecbbd700 1 -- 172.21.15.16:0/1192783746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65d00094a0 con 0x7f65e809e410 2023-09-30T22:29:24.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.488+0000 7f65deffd700 1 -- 172.21.15.16:0/1192783746 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e00070d0 con 0x7f65e809e410 2023-09-30T22:29:24.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.488+0000 7f65ecbbd700 1 -- 172.21.15.16:0/1192783746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65e81a8d00 con 0x7f65e809e410 2023-09-30T22:29:24.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.489+0000 7f65ecbbd700 1 -- 172.21.15.16:0/1192783746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65e8074ea0 con 0x7f65e809e410 2023-09-30T22:29:24.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.490+0000 7f65deffd700 1 -- 172.21.15.16:0/1192783746 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f65e0007230 con 0x7f65e809e410 2023-09-30T22:29:24.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.490+0000 7f65deffd700 1 -- 172.21.15.16:0/1192783746 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4049+0+0 (secure 0 0 0) 0x7f65e003f890 con 0x7f65e809e410 2023-09-30T22:29:24.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.495+0000 7f65deffd700 1 -- 172.21.15.16:0/1192783746 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f65e00074e0 con 0x7f65e809e410 2023-09-30T22:29:24.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.747+0000 7f65ecbbd700 1 -- 172.21.15.16:0/1192783746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f65e81a9560 con 0x7f65e809e410 2023-09-30T22:29:24.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.747+0000 7f65deffd700 1 -- 172.21.15.16:0/1192783746 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+2315 (secure 0 0 0) 0x7f65e0018020 con 0x7f65e809e410 2023-09-30T22:29:24.748 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:29:24.748 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-09-30T22:26:51.674561+0000","modified":"2023-09-30T22:27:18.263912+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4218},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4218":{"gid":4218,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":3,"addr":"172.21.15.16:6817/1424366610","addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6817","nonce":1424366610}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-09-30T22:29:24.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.749+0000 7f65ecbbd700 1 -- 172.21.15.16:0/1192783746 >> v1:172.21.15.16:6816/126811 conn(0x7f65c802f510 legacy=0x7f65c80319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:24.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.749+0000 7f65ecbbd700 1 -- 172.21.15.16:0/1192783746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65e809e410 msgr2=0x7f65e81a6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:24.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.749+0000 7f65ecbbd700 1 --2- 172.21.15.16:0/1192783746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65e809e410 0x7f65e81a6920 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f65e000cbe0 tx=0x7f65e000cef0 comp rx=0 tx=0).stop 2023-09-30T22:29:24.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.749+0000 7f65ecbbd700 1 -- 172.21.15.16:0/1192783746 shutdown_connections 2023-09-30T22:29:24.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.749+0000 7f65ecbbd700 1 --2- 172.21.15.16:0/1192783746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65e809e410 0x7f65e81a6920 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:24.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.749+0000 7f65ecbbd700 1 --2- 172.21.15.16:0/1192783746 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f65e809d6f0 0x7f65e81a5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:24.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.749+0000 7f65ecbbd700 1 --2- 172.21.15.16:0/1192783746 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f65e809e000 0x7f65e81a63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:24.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.749+0000 7f65ecbbd700 1 -- 172.21.15.16:0/1192783746 >> 172.21.15.16:0/1192783746 conn(0x7f65e8066f50 msgr2=0x7f65e81bad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:24.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.749+0000 7f65ecbbd700 1 -- 172.21.15.16:0/1192783746 shutdown_connections 2023-09-30T22:29:24.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:24.750+0000 7f65ecbbd700 1 -- 172.21.15.16:0/1192783746 wait complete. 2023-09-30T22:29:24.751 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 8 2023-09-30T22:29:24.864 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-09-30T22:26:51.674561+0000', 'modified': '2023-09-30T22:27:18.263912+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4218}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4218': {'gid': 4218, 'name': 'a', 'rank': 0, 'incarnation': 3, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.16:6817/1424366610', 'addrs': {'addrvec': [{'type': 'v1', 'addr': '172.21.15.16:6817', 'nonce': 1424366610}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-09-30T22:29:24.864 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-09-30T22:29:24.864 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-09-30T22:29:25.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.007+0000 7fc22f3b9700 1 Processor -- start 2023-09-30T22:29:25.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.007+0000 7fc22f3b9700 1 -- start start 2023-09-30T22:29:25.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.008+0000 7fc22f3b9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc22809d6f0 0x7fc22809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:25.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.008+0000 7fc22f3b9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc22809e000 0x7fc228130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:25.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.008+0000 7fc22f3b9700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc22809e410 0x7fc228130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:25.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.008+0000 7fc22f3b9700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc22809f520 con 0x7fc22809e410 2023-09-30T22:29:25.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.008+0000 7fc22f3b9700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc22809f130 con 0x7fc22809e000 2023-09-30T22:29:25.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.008+0000 7fc22f3b9700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc22809e930 con 0x7fc22809d6f0 2023-09-30T22:29:25.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.008+0000 7fc22c954700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc22809e000 0x7fc228130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:25.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.008+0000 7fc22d956700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc22809e410 0x7fc228130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:25.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.008+0000 7fc22d155700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc22809d6f0 0x7fc22809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:25.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.008+0000 7fc22d155700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc22809d6f0 0x7fc22809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:56014/0 (socket says 172.21.15.16:56014) 2023-09-30T22:29:25.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.008+0000 7fc22d956700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc22809e410 0x7fc228130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57884/0 (socket says 172.21.15.16:57884) 2023-09-30T22:29:25.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.008+0000 7fc22c954700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc22809e000 0x7fc228130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37294/0 (socket says 172.21.15.16:37294) 2023-09-30T22:29:25.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.008+0000 7fc22d155700 1 -- 172.21.15.16:0/1502983770 learned_addr learned my addr 172.21.15.16:0/1502983770 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:25.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.008+0000 7fc22d155700 1 -- 172.21.15.16:0/1502983770 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc22809e000 msgr2=0x7fc228130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:25.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.008+0000 7fc22d155700 1 --2- 172.21.15.16:0/1502983770 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc22809e000 0x7fc228130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:25.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.008+0000 7fc22d155700 1 -- 172.21.15.16:0/1502983770 >> v2:172.21.15.16:3300/0 conn(0x7fc22809e410 msgr2=0x7fc228130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:25.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.008+0000 7fc22d155700 1 --2- 172.21.15.16:0/1502983770 >> v2:172.21.15.16:3300/0 conn(0x7fc22809e410 0x7fc228130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:25.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.008+0000 7fc22d155700 1 -- 172.21.15.16:0/1502983770 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc22809b2d0 con 0x7fc22809d6f0 2023-09-30T22:29:25.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.008+0000 7fc22c954700 1 --2- 172.21.15.16:0/1502983770 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc22809e000 0x7fc228130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:25.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.008+0000 7fc22d956700 1 --2- 172.21.15.16:0/1502983770 >> v2:172.21.15.16:3300/0 conn(0x7fc22809e410 0x7fc228130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:25.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.008+0000 7fc22d155700 1 --2- 172.21.15.16:0/1502983770 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc22809d6f0 0x7fc22809dac0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc224009920 tx=0x7fc224009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17b6ab4cd476453c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:25.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.009+0000 7fc21ffff700 1 -- 172.21.15.16:0/1502983770 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc224011830 con 0x7fc22809d6f0 2023-09-30T22:29:25.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.009+0000 7fc21ffff700 1 -- 172.21.15.16:0/1502983770 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc224011990 con 0x7fc22809d6f0 2023-09-30T22:29:25.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.009+0000 7fc21ffff700 1 -- 172.21.15.16:0/1502983770 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc224011d20 con 0x7fc22809d6f0 2023-09-30T22:29:25.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.009+0000 7fc22f3b9700 1 -- 172.21.15.16:0/1502983770 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc22809d6f0 msgr2=0x7fc22809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:25.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.009+0000 7fc22f3b9700 1 --2- 172.21.15.16:0/1502983770 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc22809d6f0 0x7fc22809dac0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc224009920 tx=0x7fc224009c30 comp rx=0 tx=0).stop 2023-09-30T22:29:25.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.009+0000 7fc22f3b9700 1 -- 172.21.15.16:0/1502983770 shutdown_connections 2023-09-30T22:29:25.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.009+0000 7fc22f3b9700 1 --2- 172.21.15.16:0/1502983770 >> v2:172.21.15.16:3300/0 conn(0x7fc22809e410 0x7fc228130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:25.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.009+0000 7fc22f3b9700 1 --2- 172.21.15.16:0/1502983770 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc22809d6f0 0x7fc22809dac0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:25.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.009+0000 7fc22f3b9700 1 --2- 172.21.15.16:0/1502983770 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc22809e000 0x7fc228130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:25.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.009+0000 7fc22f3b9700 1 -- 172.21.15.16:0/1502983770 >> 172.21.15.16:0/1502983770 conn(0x7fc228066f50 msgr2=0x7fc22809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:25.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.009+0000 7fc22f3b9700 1 -- 172.21.15.16:0/1502983770 shutdown_connections 2023-09-30T22:29:25.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.009+0000 7fc22f3b9700 1 -- 172.21.15.16:0/1502983770 wait complete. 2023-09-30T22:29:25.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.010+0000 7fc22f3b9700 1 Processor -- start 2023-09-30T22:29:25.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.010+0000 7fc22f3b9700 1 -- start start 2023-09-30T22:29:25.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.011+0000 7fc22f3b9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc22809d6f0 0x7fc2281223e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:25.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.011+0000 7fc22f3b9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc22809e000 0x7fc228122920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:25.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.011+0000 7fc22f3b9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc22809e410 0x7fc228122e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:25.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.011+0000 7fc22f3b9700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc228123ff0 con 0x7fc22809e000 2023-09-30T22:29:25.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.011+0000 7fc22f3b9700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc228124880 con 0x7fc22809e410 2023-09-30T22:29:25.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.011+0000 7fc22f3b9700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc228123840 con 0x7fc22809d6f0 2023-09-30T22:29:25.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.011+0000 7fc22d155700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc22809d6f0 0x7fc2281223e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:25.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.011+0000 7fc22c954700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc22809e000 0x7fc228122920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:25.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.011+0000 7fc22d956700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc22809e410 0x7fc228122e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:25.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.011+0000 7fc22d155700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc22809d6f0 0x7fc2281223e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:56030/0 (socket says 172.21.15.16:56030) 2023-09-30T22:29:25.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.011+0000 7fc22c954700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc22809e000 0x7fc228122920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57898/0 (socket says 172.21.15.16:57898) 2023-09-30T22:29:25.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.011+0000 7fc22d155700 1 -- 172.21.15.16:0/4193375580 learned_addr learned my addr 172.21.15.16:0/4193375580 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:25.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.011+0000 7fc22c954700 1 -- 172.21.15.16:0/4193375580 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc22809d6f0 msgr2=0x7fc2281223e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:25.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.011+0000 7fc22c954700 1 --2- 172.21.15.16:0/4193375580 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc22809d6f0 0x7fc2281223e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:25.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.011+0000 7fc22c954700 1 -- 172.21.15.16:0/4193375580 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc22809e410 msgr2=0x7fc228122e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:25.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.011+0000 7fc22c954700 1 --2- 172.21.15.16:0/4193375580 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc22809e410 0x7fc228122e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:25.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.011+0000 7fc22c954700 1 -- 172.21.15.16:0/4193375580 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2180094a0 con 0x7fc22809e000 2023-09-30T22:29:25.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.011+0000 7fc22d155700 1 --2- 172.21.15.16:0/4193375580 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc22809d6f0 0x7fc2281223e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:25.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.012+0000 7fc22c954700 1 --2- 172.21.15.16:0/4193375580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc22809e000 0x7fc228122920 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc21800e980 tx=0x7fc21800ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:25.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.012+0000 7fc21dffb700 1 -- 172.21.15.16:0/4193375580 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc218004240 con 0x7fc22809e000 2023-09-30T22:29:25.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.012+0000 7fc21dffb700 1 -- 172.21.15.16:0/4193375580 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2180043a0 con 0x7fc22809e000 2023-09-30T22:29:25.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.012+0000 7fc22f3b9700 1 -- 172.21.15.16:0/4193375580 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc22000bc80 con 0x7fc22809e000 2023-09-30T22:29:25.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.012+0000 7fc21dffb700 1 -- 172.21.15.16:0/4193375580 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2180036a0 con 0x7fc22809e000 2023-09-30T22:29:25.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.012+0000 7fc22f3b9700 1 -- 172.21.15.16:0/4193375580 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc224009550 con 0x7fc22809e000 2023-09-30T22:29:25.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.013+0000 7fc21dffb700 1 -- 172.21.15.16:0/4193375580 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fc218003800 con 0x7fc22809e000 2023-09-30T22:29:25.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.013+0000 7fc21dffb700 1 -- 172.21.15.16:0/4193375580 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4049+0+0 (secure 0 0 0) 0x7fc21803f950 con 0x7fc22809e000 2023-09-30T22:29:25.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.016+0000 7fc23b55cb80 1 Processor -- start 2023-09-30T22:29:25.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.016+0000 7fc23b55cb80 1 -- start start 2023-09-30T22:29:25.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.016+0000 7fc23b55cb80 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x55d103bdbc50 0x55d103bdc020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:25.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.016+0000 7fc23b55cb80 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x55d103bf8100 0x55d103bdc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:25.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.016+0000 7fc23b55cb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d103bf28f0 0x55d103bf2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:25.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.016+0000 7fc23b55cb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d103b3ec60 con 0x55d103bf28f0 2023-09-30T22:29:25.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.016+0000 7fc23b55cb80 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d103b39a10 con 0x55d103bf8100 2023-09-30T22:29:25.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.016+0000 7fc23b55cb80 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x55d103b37210 con 0x55d103bdbc50 2023-09-30T22:29:25.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.016+0000 7fc22d155700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x55d103bdbc50 0x55d103bdc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:25.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.016+0000 7fc22d956700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d103bf28f0 0x55d103bf2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:25.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.016+0000 7fc22d155700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x55d103bdbc50 0x55d103bdc020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:56042/0 (socket says 172.21.15.16:56042) 2023-09-30T22:29:25.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.016+0000 7fc22d155700 1 -- 172.21.15.16:0/1568002467 learned_addr learned my addr 172.21.15.16:0/1568002467 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:25.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.016+0000 7fc22c954700 1 --2- 172.21.15.16:0/1568002467 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x55d103bf8100 0x55d103bdc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:25.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.017+0000 7fc22d155700 1 -- 172.21.15.16:0/1568002467 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x55d103bf8100 msgr2=0x55d103bdc560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:25.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.017+0000 7fc22d155700 1 --2- 172.21.15.16:0/1568002467 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x55d103bf8100 0x55d103bdc560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:25.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.017+0000 7fc22d155700 1 -- 172.21.15.16:0/1568002467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d103bf28f0 msgr2=0x55d103bf2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:25.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.017+0000 7fc22d155700 1 --2- 172.21.15.16:0/1568002467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d103bf28f0 0x55d103bf2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:25.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.017+0000 7fc22d155700 1 -- 172.21.15.16:0/1568002467 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d103926790 con 0x55d103bdbc50 2023-09-30T22:29:25.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.017+0000 7fc22d956700 1 --2- 172.21.15.16:0/1568002467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d103bf28f0 0x55d103bf2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:25.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.017+0000 7fc22d155700 1 --2- 172.21.15.16:0/1568002467 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x55d103bdbc50 0x55d103bdc020 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc224011700 tx=0x7fc224010000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31ee5869e6248328 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:25.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.017+0000 7fc212ffd700 1 -- 172.21.15.16:0/1568002467 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc224022070 con 0x55d103bdbc50 2023-09-30T22:29:25.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.017+0000 7fc212ffd700 1 -- 172.21.15.16:0/1568002467 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc224011bc0 con 0x55d103bdbc50 2023-09-30T22:29:25.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.017+0000 7fc212ffd700 1 -- 172.21.15.16:0/1568002467 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc224004020 con 0x55d103bdbc50 2023-09-30T22:29:25.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.018+0000 7fc23b55cb80 1 -- 172.21.15.16:0/1568002467 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x55d103bdbc50 msgr2=0x55d103bdc020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:25.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.018+0000 7fc23b55cb80 1 --2- 172.21.15.16:0/1568002467 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x55d103bdbc50 0x55d103bdc020 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc224011700 tx=0x7fc224010000 comp rx=0 tx=0).stop 2023-09-30T22:29:25.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.018+0000 7fc23b55cb80 1 -- 172.21.15.16:0/1568002467 shutdown_connections 2023-09-30T22:29:25.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.018+0000 7fc23b55cb80 1 --2- 172.21.15.16:0/1568002467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d103bf28f0 0x55d103bf2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:25.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.018+0000 7fc23b55cb80 1 --2- 172.21.15.16:0/1568002467 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x55d103bdbc50 0x55d103bdc020 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:25.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.018+0000 7fc23b55cb80 1 --2- 172.21.15.16:0/1568002467 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x55d103bf8100 0x55d103bdc560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:25.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.018+0000 7fc23b55cb80 1 -- 172.21.15.16:0/1568002467 >> 172.21.15.16:0/1568002467 conn(0x55d103be00f0 msgr2=0x55d103bdf690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:25.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.018+0000 7fc23b55cb80 1 -- 172.21.15.16:0/1568002467 shutdown_connections 2023-09-30T22:29:25.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.018+0000 7fc23b55cb80 1 -- 172.21.15.16:0/1568002467 wait complete. 2023-09-30T22:29:25.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.018+0000 7fc23b55cb80 1 -- start start 2023-09-30T22:29:25.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.018+0000 7fc23b55cb80 1 Processor -- start 2023-09-30T22:29:25.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.019+0000 7fc23b55cb80 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x55d103bdbc50 0x55d103c01290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:25.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.019+0000 7fc23b55cb80 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x55d103bf28f0 0x55d103c017d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:25.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.019+0000 7fc23b55cb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d103bf8100 0x55d103c01d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:25.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.019+0000 7fc23b55cb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d103b32b00 con 0x55d103bf8100 2023-09-30T22:29:25.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.019+0000 7fc23b55cb80 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d103ace770 con 0x55d103bf28f0 2023-09-30T22:29:25.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.019+0000 7fc23b55cb80 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x55d103af16f0 con 0x55d103bdbc50 2023-09-30T22:29:25.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.019+0000 7fc23b55cb80 10 client.0 _ll_register_callbacks cb 0x55d10393a5a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T22:29:25.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.019+0000 7fc22c954700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x55d103bf28f0 0x55d103c017d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:25.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.019+0000 7fc22c954700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x55d103bf28f0 0x55d103c017d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37310/0 (socket says 172.21.15.16:37310) 2023-09-30T22:29:25.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.019+0000 7fc22c954700 1 -- 172.21.15.16:0/2079802302 learned_addr learned my addr 172.21.15.16:0/2079802302 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:25.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.019+0000 7fc22d155700 1 --2- 172.21.15.16:0/2079802302 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x55d103bdbc50 0x55d103c01290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:25.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.019+0000 7fc22d956700 1 --2- 172.21.15.16:0/2079802302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d103bf8100 0x55d103c01d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:25.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.019+0000 7fc22c954700 1 -- 172.21.15.16:0/2079802302 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x55d103bdbc50 msgr2=0x55d103c01290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:25.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.019+0000 7fc22c954700 1 --2- 172.21.15.16:0/2079802302 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x55d103bdbc50 0x55d103c01290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:25.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.019+0000 7fc22c954700 1 -- 172.21.15.16:0/2079802302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d103bf8100 msgr2=0x55d103c01d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:25.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.019+0000 7fc22c954700 1 --2- 172.21.15.16:0/2079802302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d103bf8100 0x55d103c01d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:25.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.019+0000 7fc22c954700 1 -- 172.21.15.16:0/2079802302 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc22000b0b0 con 0x55d103bf28f0 2023-09-30T22:29:25.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.019+0000 7fc22d155700 1 --2- 172.21.15.16:0/2079802302 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x55d103bdbc50 0x55d103c01290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:25.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.020+0000 7fc22c954700 1 --2- 172.21.15.16:0/2079802302 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x55d103bf28f0 0x55d103c017d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc218000c00 tx=0x7fc218004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=944e60f77a4911b6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:25.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.020+0000 7fc207fff700 10 client.0 ms_handle_connect on v2:172.21.15.16:3301/0 2023-09-30T22:29:25.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.020+0000 7fc207fff700 1 -- 172.21.15.16:0/2079802302 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc218030040 con 0x55d103bf28f0 2023-09-30T22:29:25.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.020+0000 7fc207fff700 1 -- 172.21.15.16:0/2079802302 <== mon.1 v2:172.21.15.16:3301/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 4049+0+0 (secure 0 0 0) 0x7fc2180380b0 con 0x55d103bf28f0 2023-09-30T22:29:25.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.020+0000 7fc23b55cb80 1 -- 172.21.15.16:0/2079802302 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d103bd42b0 con 0x55d103bf28f0 2023-09-30T22:29:25.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.020+0000 7fc207fff700 1 -- 172.21.15.16:0/2079802302 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc218038270 con 0x55d103bf28f0 2023-09-30T22:29:25.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.020+0000 7fc23b55cb80 10 client.14362 fetch_fsmap learned FSMap version 8 2023-09-30T22:29:25.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.020+0000 7fc23b55cb80 1 -- 172.21.15.16:0/2079802302 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,fsmap=8,monmap=3+}) v3 -- 0x7fc22809b2d0 con 0x55d103bf28f0 2023-09-30T22:29:25.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.020+0000 7fc207fff700 1 -- 172.21.15.16:0/2079802302 <== mon.1 v2:172.21.15.16:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d103bd42b0 con 0x55d103bf28f0 2023-09-30T22:29:25.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.020+0000 7fc207fff700 1 -- 172.21.15.16:0/2079802302 <== mon.1 v2:172.21.15.16:3301/0 5 ==== fsmap(e 8) v1 ==== 1436+0+0 (secure 0 0 0) 0x7fc218038be0 con 0x55d103bf28f0 2023-09-30T22:29:25.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.020+0000 7fc23b55cb80 10 client.14362 fetch_fsmap finished waiting for FSMap version 8 2023-09-30T22:29:25.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.020+0000 7fc23b55cb80 10 client.14362 resolve_mds: resolved name 'a' to daemon mds.a 2023-09-30T22:29:25.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.020+0000 7fc23b55cb80 20 client.14362 populate_metadata read hostname 'smithi016' 2023-09-30T22:29:25.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.020+0000 7fc23b55cb80 4 client.14362 mds_command: new command op to 4218 tid=0[{"prefix": "get_command_descriptions"}] 2023-09-30T22:29:25.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.020+0000 7fc23b55cb80 1 -- 172.21.15.16:0/2079802302 --> v1:172.21.15.16:6817/1424366610 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d103bb0f40 con 0x55d103bf2330 2023-09-30T22:29:25.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.021+0000 7fc207fff700 10 client.14362 ms_handle_connect on v1:172.21.15.16:6817/1424366610 2023-09-30T22:29:25.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.021+0000 7fc207fff700 1 -- 172.21.15.16:0/2079802302 <== mds.0 v1:172.21.15.16:6817/1424366610 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (unknown 0 0 3608088055) 0x55d103bb0f40 con 0x55d103bf2330 2023-09-30T22:29:25.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.021+0000 7fc207fff700 10 client.14362 handle_command_reply: tid=0 2023-09-30T22:29:25.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.021+0000 7fc23b55cb80 1 client.14362 shutdown 2023-09-30T22:29:25.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.021+0000 7fc23b55cb80 1 -- 172.21.15.16:0/2079802302 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x55d103bf28f0 msgr2=0x55d103c017d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:25.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.021+0000 7fc23b55cb80 1 --2- 172.21.15.16:0/2079802302 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x55d103bf28f0 0x55d103c017d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc218000c00 tx=0x7fc218004750 comp rx=0 tx=0).stop 2023-09-30T22:29:25.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.021+0000 7fc23b55cb80 1 -- 172.21.15.16:0/2079802302 shutdown_connections 2023-09-30T22:29:25.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.021+0000 7fc23b55cb80 1 --2- 172.21.15.16:0/2079802302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d103bf8100 0x55d103c01d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:25.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.021+0000 7fc207fff700 0 client.14362 ms_handle_reset on v1:172.21.15.16:6817/1424366610 2023-09-30T22:29:25.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.021+0000 7fc23b55cb80 1 --2- 172.21.15.16:0/2079802302 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x55d103bdbc50 0x55d103c01290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:25.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.021+0000 7fc23b55cb80 1 --2- 172.21.15.16:0/2079802302 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x55d103bf28f0 0x55d103c017d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:25.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.021+0000 7fc23b55cb80 1 -- 172.21.15.16:0/2079802302 >> 172.21.15.16:0/2079802302 conn(0x55d103be00f0 msgr2=0x55d103bdc020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:25.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.022+0000 7fc23b55cb80 1 -- 172.21.15.16:0/2079802302 shutdown_connections 2023-09-30T22:29:25.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.022+0000 7fc23b55cb80 1 -- 172.21.15.16:0/2079802302 wait complete. 2023-09-30T22:29:25.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.022+0000 7fc23b55cb80 20 client.14362 trim_cache size 0 max 16384 2023-09-30T22:29:25.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.039+0000 7fc23b55cb80 1 Processor -- start 2023-09-30T22:29:25.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.039+0000 7fc23b55cb80 1 -- start start 2023-09-30T22:29:25.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.040+0000 7fc23b55cb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d103bdbc50 0x55d103c18ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:25.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.040+0000 7fc23b55cb80 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x55d103bf28f0 0x55d103c02220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:25.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.040+0000 7fc23b55cb80 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x55d103bf8100 0x55d103bdf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:25.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.040+0000 7fc23b55cb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d103927390 con 0x55d103bdbc50 2023-09-30T22:29:25.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.040+0000 7fc23b55cb80 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d103bf84d0 con 0x55d103bf28f0 2023-09-30T22:29:25.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.040+0000 7fc23b55cb80 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x55d103bf9f40 con 0x55d103bf8100 2023-09-30T22:29:25.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.040+0000 7fc22d155700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d103bdbc50 0x55d103c18ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:25.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.040+0000 7fc22d155700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d103bdbc50 0x55d103c18ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57910/0 (socket says 172.21.15.16:57910) 2023-09-30T22:29:25.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.040+0000 7fc22c954700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x55d103bf28f0 0x55d103c02220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:25.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.040+0000 7fc22d155700 1 -- 172.21.15.16:0/4136498656 learned_addr learned my addr 172.21.15.16:0/4136498656 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:25.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.040+0000 7fc22d956700 1 --2- 172.21.15.16:0/4136498656 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x55d103bf8100 0x55d103bdf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:25.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.040+0000 7fc22d155700 1 -- 172.21.15.16:0/4136498656 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x55d103bf8100 msgr2=0x55d103bdf690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:25.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.040+0000 7fc22d155700 1 --2- 172.21.15.16:0/4136498656 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x55d103bf8100 0x55d103bdf690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:25.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.040+0000 7fc22d155700 1 -- 172.21.15.16:0/4136498656 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x55d103bf28f0 msgr2=0x55d103c02220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:25.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.040+0000 7fc22d155700 1 --2- 172.21.15.16:0/4136498656 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x55d103bf28f0 0x55d103c02220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:25.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.040+0000 7fc22d155700 1 -- 172.21.15.16:0/4136498656 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d103c01000 con 0x55d103bdbc50 2023-09-30T22:29:25.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.040+0000 7fc22c954700 1 --2- 172.21.15.16:0/4136498656 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x55d103bf28f0 0x55d103c02220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:25.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.040+0000 7fc22d956700 1 --2- 172.21.15.16:0/4136498656 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x55d103bf8100 0x55d103bdf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:25.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.040+0000 7fc22d155700 1 --2- 172.21.15.16:0/4136498656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d103bdbc50 0x55d103c18ac0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc224009000 tx=0x7fc2240092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5518c951c8308597 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:25.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.040+0000 7fc2137fe700 1 -- 172.21.15.16:0/4136498656 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc224022040 con 0x55d103bdbc50 2023-09-30T22:29:25.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.040+0000 7fc2137fe700 1 -- 172.21.15.16:0/4136498656 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc224004030 con 0x55d103bdbc50 2023-09-30T22:29:25.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.040+0000 7fc2137fe700 1 -- 172.21.15.16:0/4136498656 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2240043e0 con 0x55d103bdbc50 2023-09-30T22:29:25.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.041+0000 7fc23b55cb80 1 -- 172.21.15.16:0/4136498656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d103bdbc50 msgr2=0x55d103c18ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:25.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.041+0000 7fc23b55cb80 1 --2- 172.21.15.16:0/4136498656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d103bdbc50 0x55d103c18ac0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc224009000 tx=0x7fc2240092c0 comp rx=0 tx=0).stop 2023-09-30T22:29:25.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.041+0000 7fc23b55cb80 1 -- 172.21.15.16:0/4136498656 shutdown_connections 2023-09-30T22:29:25.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.041+0000 7fc23b55cb80 1 --2- 172.21.15.16:0/4136498656 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x55d103bf8100 0x55d103bdf690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:25.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.041+0000 7fc23b55cb80 1 --2- 172.21.15.16:0/4136498656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d103bdbc50 0x55d103c18ac0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:25.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.041+0000 7fc23b55cb80 1 --2- 172.21.15.16:0/4136498656 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x55d103bf28f0 0x55d103c02220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:25.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.041+0000 7fc23b55cb80 1 -- 172.21.15.16:0/4136498656 >> 172.21.15.16:0/4136498656 conn(0x55d103be00f0 msgr2=0x55d103bf5f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:25.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.041+0000 7fc23b55cb80 1 -- 172.21.15.16:0/4136498656 shutdown_connections 2023-09-30T22:29:25.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.041+0000 7fc23b55cb80 1 -- 172.21.15.16:0/4136498656 wait complete. 2023-09-30T22:29:25.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.042+0000 7fc23b55cb80 1 -- start start 2023-09-30T22:29:25.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.042+0000 7fc23b55cb80 1 Processor -- start 2023-09-30T22:29:25.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.042+0000 7fc23b55cb80 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x55d103bdbc50 0x55d103c01210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:25.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.042+0000 7fc23b55cb80 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x55d103bf28f0 0x55d103c01750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:25.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.042+0000 7fc23b55cb80 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d103bf8100 0x55d103c01c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:25.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.042+0000 7fc23b55cb80 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d103bf38f0 con 0x55d103bf8100 2023-09-30T22:29:25.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.042+0000 7fc23b55cb80 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d103bf3300 con 0x55d103bdbc50 2023-09-30T22:29:25.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.042+0000 7fc23b55cb80 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x55d103c10830 con 0x55d103bf28f0 2023-09-30T22:29:25.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.042+0000 7fc23b55cb80 10 client.0 _ll_register_callbacks cb 0x7fc218011890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-09-30T22:29:25.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.042+0000 7fc22d956700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d103bf8100 0x55d103c01c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:25.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.042+0000 7fc22c954700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x55d103bf28f0 0x55d103c01750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:25.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.042+0000 7fc22d155700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x55d103bdbc50 0x55d103c01210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:25.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.042+0000 7fc22d956700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d103bf8100 0x55d103c01c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57912/0 (socket says 172.21.15.16:57912) 2023-09-30T22:29:25.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.042+0000 7fc22c954700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x55d103bf28f0 0x55d103c01750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:56068/0 (socket says 172.21.15.16:56068) 2023-09-30T22:29:25.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.043+0000 7fc22d956700 1 -- 172.21.15.16:0/1511357391 learned_addr learned my addr 172.21.15.16:0/1511357391 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:25.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.043+0000 7fc22d956700 1 -- 172.21.15.16:0/1511357391 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x55d103bf28f0 msgr2=0x55d103c01750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:25.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.043+0000 7fc22d956700 1 --2- 172.21.15.16:0/1511357391 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x55d103bf28f0 0x55d103c01750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:25.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.043+0000 7fc22d956700 1 -- 172.21.15.16:0/1511357391 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x55d103bdbc50 msgr2=0x55d103c01210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:25.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.043+0000 7fc22d956700 1 --2- 172.21.15.16:0/1511357391 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x55d103bdbc50 0x55d103c01210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:25.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.043+0000 7fc22d956700 1 -- 172.21.15.16:0/1511357391 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc220003a60 con 0x55d103bf8100 2023-09-30T22:29:25.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.043+0000 7fc22d956700 1 --2- 172.21.15.16:0/1511357391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d103bf8100 0x55d103c01c90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x55d103bde5e0 tx=0x7fc220003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75fa64cc3bf789e0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:25.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.043+0000 7fc207fff700 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2023-09-30T22:29:25.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.043+0000 7fc207fff700 1 -- 172.21.15.16:0/1511357391 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc220010040 con 0x55d103bf8100 2023-09-30T22:29:25.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.043+0000 7fc207fff700 1 -- 172.21.15.16:0/1511357391 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 4049+0+0 (secure 0 0 0) 0x7fc220009d40 con 0x55d103bf8100 2023-09-30T22:29:25.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.043+0000 7fc23b55cb80 1 -- 172.21.15.16:0/1511357391 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc224011bc0 con 0x55d103bf8100 2023-09-30T22:29:25.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.043+0000 7fc207fff700 1 -- 172.21.15.16:0/1511357391 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc220018400 con 0x55d103bf8100 2023-09-30T22:29:25.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.043+0000 7fc23b55cb80 10 client.14466 fetch_fsmap learned FSMap version 8 2023-09-30T22:29:25.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.043+0000 7fc23b55cb80 1 -- 172.21.15.16:0/1511357391 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=3+}) v3 -- 0x55d103926790 con 0x55d103bf8100 2023-09-30T22:29:25.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.044+0000 7fc207fff700 1 -- 172.21.15.16:0/1511357391 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc224011bc0 con 0x55d103bf8100 2023-09-30T22:29:25.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.044+0000 7fc207fff700 1 -- 172.21.15.16:0/1511357391 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 8) v1 ==== 1436+0+0 (secure 0 0 0) 0x7fc220018b60 con 0x55d103bf8100 2023-09-30T22:29:25.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.044+0000 7fc23b55cb80 10 client.14466 fetch_fsmap finished waiting for FSMap version 8 2023-09-30T22:29:25.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.044+0000 7fc23b55cb80 10 client.14466 resolve_mds: resolved name 'a' to daemon mds.a 2023-09-30T22:29:25.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.044+0000 7fc23b55cb80 20 client.14466 populate_metadata read hostname 'smithi016' 2023-09-30T22:29:25.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.044+0000 7fc23b55cb80 4 client.14466 mds_command: new command op to 4218 tid=0[{"prefix": "status"}] 2023-09-30T22:29:25.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.044+0000 7fc23b55cb80 1 -- 172.21.15.16:0/1511357391 --> v1:172.21.15.16:6817/1424366610 -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d103b2bce0 con 0x55d103c5e3b0 2023-09-30T22:29:25.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.044+0000 7fc207fff700 10 client.14466 ms_handle_connect on v1:172.21.15.16:6817/1424366610 2023-09-30T22:29:25.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.044+0000 7fc207fff700 1 -- 172.21.15.16:0/1511357391 <== mds.0 v1:172.21.15.16:6817/1424366610 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (unknown 0 0 2253596822) 0x55d103b2bce0 con 0x55d103c5e3b0 2023-09-30T22:29:25.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.044+0000 7fc207fff700 10 client.14466 handle_command_reply: tid=0 2023-09-30T22:29:25.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.044+0000 7fc23b55cb80 1 client.14466 shutdown 2023-09-30T22:29:25.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.044+0000 7fc23b55cb80 1 -- 172.21.15.16:0/1511357391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d103bf8100 msgr2=0x55d103c01c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:25.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.044+0000 7fc23b55cb80 1 --2- 172.21.15.16:0/1511357391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d103bf8100 0x55d103c01c90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x55d103bde5e0 tx=0x7fc220003f90 comp rx=0 tx=0).stop 2023-09-30T22:29:25.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.044+0000 7fc23b55cb80 1 -- 172.21.15.16:0/1511357391 shutdown_connections 2023-09-30T22:29:25.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.045+0000 7fc23b55cb80 1 --2- 172.21.15.16:0/1511357391 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x55d103bdbc50 0x55d103c01210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:25.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.045+0000 7fc23b55cb80 1 --2- 172.21.15.16:0/1511357391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d103bf8100 0x55d103c01c90 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:25.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.045+0000 7fc207fff700 0 client.14466 ms_handle_reset on v1:172.21.15.16:6817/1424366610 2023-09-30T22:29:25.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.045+0000 7fc23b55cb80 1 --2- 172.21.15.16:0/1511357391 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x55d103bf28f0 0x55d103c01750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:25.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.045+0000 7fc23b55cb80 1 -- 172.21.15.16:0/1511357391 >> 172.21.15.16:0/1511357391 conn(0x55d103be00f0 msgr2=0x55d103bda920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:25.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.045+0000 7fc23b55cb80 1 -- 172.21.15.16:0/1511357391 shutdown_connections 2023-09-30T22:29:25.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.045+0000 7fc23b55cb80 1 -- 172.21.15.16:0/1511357391 wait complete. 2023-09-30T22:29:25.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.045+0000 7fc23b55cb80 20 client.14466 trim_cache size 0 max 16384 2023-09-30T22:29:25.055 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-09-30T22:29:25.055 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "7ede3e14-2bd0-4726-9437-6d13c90cd342", 2023-09-30T22:29:25.055 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2023-09-30T22:29:25.055 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 4218, 2023-09-30T22:29:25.055 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2023-09-30T22:29:25.055 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2023-09-30T22:29:25.055 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2023-09-30T22:29:25.055 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 153.36949125300001, 2023-09-30T22:29:25.055 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 8, 2023-09-30T22:29:25.055 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 22, 2023-09-30T22:29:25.055 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 17, 2023-09-30T22:29:25.056 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 158.694082431 2023-09-30T22:29:25.056 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-09-30T22:29:25.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.045+0000 7fc22f3b9700 1 -- 172.21.15.16:0/4193375580 >> v1:172.21.15.16:6816/126811 conn(0x7fc20c02f510 legacy=0x7fc20c0319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:25.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.045+0000 7fc22f3b9700 1 -- 172.21.15.16:0/4193375580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc22809e000 msgr2=0x7fc228122920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:25.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.045+0000 7fc22f3b9700 1 --2- 172.21.15.16:0/4193375580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc22809e000 0x7fc228122920 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc21800e980 tx=0x7fc21800ec90 comp rx=0 tx=0).stop 2023-09-30T22:29:25.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.045+0000 7fc22f3b9700 1 -- 172.21.15.16:0/4193375580 shutdown_connections 2023-09-30T22:29:25.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.045+0000 7fc22f3b9700 1 --2- 172.21.15.16:0/4193375580 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc22809e410 0x7fc228122e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:25.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.045+0000 7fc22f3b9700 1 --2- 172.21.15.16:0/4193375580 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc22809d6f0 0x7fc2281223e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:25.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.045+0000 7fc22f3b9700 1 --2- 172.21.15.16:0/4193375580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc22809e000 0x7fc228122920 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:25.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.045+0000 7fc22f3b9700 1 -- 172.21.15.16:0/4193375580 >> 172.21.15.16:0/4193375580 conn(0x7fc228066f50 msgr2=0x7fc228130890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:25.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.045+0000 7fc22f3b9700 1 -- 172.21.15.16:0/4193375580 shutdown_connections 2023-09-30T22:29:25.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.045+0000 7fc22f3b9700 1 -- 172.21.15.16:0/4193375580 wait complete. 2023-09-30T22:29:25.059 INFO:teuthology.run_tasks:Running task ceph.restart... 2023-09-30T22:29:25.069 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-09-30T22:29:25.070 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T22:29:25.099 INFO:tasks.ceph.osd.0:Stopped 2023-09-30T22:29:25.100 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 0 2023-09-30T22:29:25.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.280+0000 7f3606106700 1 Processor -- start 2023-09-30T22:29:25.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.281+0000 7f3606106700 1 -- start start 2023-09-30T22:29:25.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.281+0000 7f3606106700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f36001316c0 0x7f3600131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:25.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.281+0000 7f3606106700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f360006ae80 0x7f3600139550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:25.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.281+0000 7f3606106700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f360009e130 con 0x7f360009d9f0 2023-09-30T22:29:25.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.281+0000 7f3606106700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36001312b0 con 0x7f360006ae80 2023-09-30T22:29:25.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.281+0000 7f3606106700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f360009f6e0 con 0x7f36001316c0 2023-09-30T22:29:25.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.282+0000 7f35fffff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f360006ae80 0x7f3600139550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:25.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.282+0000 7f35ff7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f36001316c0 0x7f3600131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:25.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.282+0000 7f35fffff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f360006ae80 0x7f3600139550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57928/0 (socket says 172.21.15.16:57928) 2023-09-30T22:29:25.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.282+0000 7f35fffff700 1 -- 172.21.15.16:0/2477248562 learned_addr learned my addr 172.21.15.16:0/2477248562 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:25.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.282+0000 7f35feffd700 1 --1- 172.21.15.16:0/2477248562 >> v1:172.21.15.16:6789/0 conn(0x7f360009d9f0 0x7f360006a770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33924/0 (socket says 172.21.15.16:33924) 2023-09-30T22:29:25.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.282+0000 7f35fe7fc700 1 -- 172.21.15.16:0/2477248562 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 705403135 0 0) 0x7f35f4001110 con 0x7f360009d9f0 2023-09-30T22:29:25.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.282+0000 7f35fe7fc700 1 -- 172.21.15.16:0/2477248562 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35e80035b0 con 0x7f360009d9f0 2023-09-30T22:29:25.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.282+0000 7f35ff7fe700 1 -- 172.21.15.16:0/2477248562 >> v2:172.21.15.16:3300/0 conn(0x7f360006ae80 msgr2=0x7f3600139550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:25.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.282+0000 7f35ff7fe700 1 --2- 172.21.15.16:0/2477248562 >> v2:172.21.15.16:3300/0 conn(0x7f360006ae80 0x7f3600139550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:25.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.282+0000 7f35ff7fe700 1 -- 172.21.15.16:0/2477248562 >> v1:172.21.15.16:6789/0 conn(0x7f360009d9f0 legacy=0x7f360006a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:25.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.282+0000 7f35ff7fe700 1 -- 172.21.15.16:0/2477248562 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f360009e760 con 0x7f36001316c0 2023-09-30T22:29:25.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.282+0000 7f35fffff700 1 --2- 172.21.15.16:0/2477248562 >> v2:172.21.15.16:3300/0 conn(0x7f360006ae80 0x7f3600139550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:25.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.282+0000 7f35ff7fe700 1 --2- 172.21.15.16:0/2477248562 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f36001316c0 0x7f3600131a90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f35f0009840 tx=0x7f35f0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8aad46bcc14b41c7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:25.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.282+0000 7f35fe7fc700 1 -- 172.21.15.16:0/2477248562 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f0011830 con 0x7f36001316c0 2023-09-30T22:29:25.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.282+0000 7f35fe7fc700 1 -- 172.21.15.16:0/2477248562 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35f0011990 con 0x7f36001316c0 2023-09-30T22:29:25.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.282+0000 7f35fe7fc700 1 -- 172.21.15.16:0/2477248562 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f0011d20 con 0x7f36001316c0 2023-09-30T22:29:25.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.283+0000 7f3606106700 1 -- 172.21.15.16:0/2477248562 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f36001316c0 msgr2=0x7f3600131a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:25.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.283+0000 7f3606106700 1 --2- 172.21.15.16:0/2477248562 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f36001316c0 0x7f3600131a90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f35f0009840 tx=0x7f35f0009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:25.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.283+0000 7f3606106700 1 -- 172.21.15.16:0/2477248562 shutdown_connections 2023-09-30T22:29:25.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.283+0000 7f3606106700 1 --2- 172.21.15.16:0/2477248562 >> v2:172.21.15.16:3300/0 conn(0x7f360006ae80 0x7f3600139550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:25.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.283+0000 7f3606106700 1 --2- 172.21.15.16:0/2477248562 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f36001316c0 0x7f3600131a90 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:25.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.283+0000 7f3606106700 1 -- 172.21.15.16:0/2477248562 >> 172.21.15.16:0/2477248562 conn(0x7f3600066f50 msgr2=0x7f360009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:25.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.283+0000 7f3606106700 1 -- 172.21.15.16:0/2477248562 shutdown_connections 2023-09-30T22:29:25.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.283+0000 7f3606106700 1 -- 172.21.15.16:0/2477248562 wait complete. 2023-09-30T22:29:25.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.284+0000 7f3606106700 1 Processor -- start 2023-09-30T22:29:25.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.284+0000 7f3606106700 1 -- start start 2023-09-30T22:29:25.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.285+0000 7f3606106700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f360006ae80 0x7f3600070b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:25.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.285+0000 7f3606106700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f360009d9f0 0x7f3600071040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:25.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.285+0000 7f3606106700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36001316c0 0x7f3600071580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:25.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.285+0000 7f3606106700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f360011ac00 con 0x7f36001316c0 2023-09-30T22:29:25.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.285+0000 7f3606106700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f360011ade0 con 0x7f360006ae80 2023-09-30T22:29:25.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.285+0000 7f3606106700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f360012b900 con 0x7f360009d9f0 2023-09-30T22:29:25.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.285+0000 7f35feffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f360009d9f0 0x7f3600071040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:25.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.285+0000 7f35fffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36001316c0 0x7f3600071580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:25.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.285+0000 7f35ff7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f360006ae80 0x7f3600070b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:25.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.285+0000 7f35feffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f360009d9f0 0x7f3600071040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:56088/0 (socket says 172.21.15.16:56088) 2023-09-30T22:29:25.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.285+0000 7f35feffd700 1 -- 172.21.15.16:0/721516074 learned_addr learned my addr 172.21.15.16:0/721516074 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:25.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.285+0000 7f35fffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36001316c0 0x7f3600071580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57938/0 (socket says 172.21.15.16:57938) 2023-09-30T22:29:25.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.285+0000 7f35feffd700 1 -- 172.21.15.16:0/721516074 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f360006ae80 msgr2=0x7f3600070b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:25.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.285+0000 7f35feffd700 1 --2- 172.21.15.16:0/721516074 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f360006ae80 0x7f3600070b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:25.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.285+0000 7f35feffd700 1 -- 172.21.15.16:0/721516074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36001316c0 msgr2=0x7f3600071580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:25.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.285+0000 7f35feffd700 1 --2- 172.21.15.16:0/721516074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36001316c0 0x7f3600071580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:25.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.285+0000 7f35feffd700 1 -- 172.21.15.16:0/721516074 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f8009b30 con 0x7f360009d9f0 2023-09-30T22:29:25.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.285+0000 7f35fffff700 1 --2- 172.21.15.16:0/721516074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36001316c0 0x7f3600071580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:25.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.285+0000 7f35feffd700 1 --2- 172.21.15.16:0/721516074 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f360009d9f0 0x7f3600071040 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f35f400c760 tx=0x7f35f400ca70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:25.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.286+0000 7f35e7fff700 1 -- 172.21.15.16:0/721516074 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f4005cb0 con 0x7f360009d9f0 2023-09-30T22:29:25.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.286+0000 7f35e7fff700 1 -- 172.21.15.16:0/721516074 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35f4005e10 con 0x7f360009d9f0 2023-09-30T22:29:25.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.286+0000 7f3606106700 1 -- 172.21.15.16:0/721516074 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35f00094a0 con 0x7f360009d9f0 2023-09-30T22:29:25.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.286+0000 7f35e7fff700 1 -- 172.21.15.16:0/721516074 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35f40072f0 con 0x7f360009d9f0 2023-09-30T22:29:25.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.286+0000 7f3606106700 1 -- 172.21.15.16:0/721516074 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36000a36d0 con 0x7f360009d9f0 2023-09-30T22:29:25.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.287+0000 7f35e7fff700 1 -- 172.21.15.16:0/721516074 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f35f4007450 con 0x7f360009d9f0 2023-09-30T22:29:25.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.287+0000 7f35e7fff700 1 -- 172.21.15.16:0/721516074 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 4049+0+0 (secure 0 0 0) 0x7f35f403edc0 con 0x7f360009d9f0 2023-09-30T22:29:25.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.287+0000 7f3606106700 1 -- 172.21.15.16:0/721516074 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f360009e130 con 0x7f360009d9f0 2023-09-30T22:29:25.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.292+0000 7f35e7fff700 1 -- 172.21.15.16:0/721516074 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f360009e130 con 0x7f360009d9f0 2023-09-30T22:29:25.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.513+0000 7f3606106700 1 -- 172.21.15.16:0/721516074 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd down", "ids": ["0"]} v 0) v1 -- 0x7f360009e130 con 0x7f360009d9f0 2023-09-30T22:29:25.515 INFO:tasks.ceph.mon.a.smithi016.stderr:2023-09-30T22:29:25.514+0000 7f081fac5700 -1 mon.a@0(leader).osd e23 definitely_dead 0 2023-09-30T22:29:25.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.515+0000 7f35e7fff700 1 -- 172.21.15.16:0/721516074 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd down", "ids": ["0"]}]=0 osd.0 is already down. v23) v1 ==== 93+0+0 (secure 0 0 0) 0x7f360009e130 con 0x7f360009d9f0 2023-09-30T22:29:25.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.518+0000 7f3606106700 1 -- 172.21.15.16:0/721516074 >> v1:172.21.15.16:6816/126811 conn(0x7f35ec02f510 legacy=0x7f35ec0319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:25.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.518+0000 7f3606106700 1 -- 172.21.15.16:0/721516074 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f360009d9f0 msgr2=0x7f3600071040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:25.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.518+0000 7f3606106700 1 --2- 172.21.15.16:0/721516074 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f360009d9f0 0x7f3600071040 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f35f400c760 tx=0x7f35f400ca70 comp rx=0 tx=0).stop 2023-09-30T22:29:25.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.519+0000 7f3606106700 1 -- 172.21.15.16:0/721516074 shutdown_connections 2023-09-30T22:29:25.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.519+0000 7f3606106700 1 --2- 172.21.15.16:0/721516074 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f360006ae80 0x7f3600070b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:25.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.519+0000 7f3606106700 1 --2- 172.21.15.16:0/721516074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36001316c0 0x7f3600071580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:25.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.519+0000 7f3606106700 1 --2- 172.21.15.16:0/721516074 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f360009d9f0 0x7f3600071040 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:25.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.519+0000 7f3606106700 1 -- 172.21.15.16:0/721516074 >> 172.21.15.16:0/721516074 conn(0x7f3600066f50 msgr2=0x7f36001301b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:25.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.519+0000 7f3606106700 1 -- 172.21.15.16:0/721516074 shutdown_connections 2023-09-30T22:29:25.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.519+0000 7f3606106700 1 -- 172.21.15.16:0/721516074 wait complete. 2023-09-30T22:29:25.520 INFO:teuthology.orchestra.run.smithi016.stderr:osd.0 is already down. 2023-09-30T22:29:25.533 INFO:tasks.ceph.osd.0:Restarting daemon 2023-09-30T22:29:25.534 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-09-30T22:29:25.536 INFO:tasks.ceph.osd.0:Started 2023-09-30T22:29:25.536 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-09-30T22:29:25.536 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T22:29:25.566 INFO:tasks.ceph.osd.1:Stopped 2023-09-30T22:29:25.566 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 1 2023-09-30T22:29:25.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.745+0000 7f4da617c700 1 Processor -- start 2023-09-30T22:29:25.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.745+0000 7f4da617c700 1 -- start start 2023-09-30T22:29:25.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.746+0000 7f4da617c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4da009e1d0 0x7f4da0138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:25.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.746+0000 7f4da617c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4da0139170 0x7f4da013b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:25.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.746+0000 7f4da617c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4da009b120 con 0x7f4da009d6f0 2023-09-30T22:29:25.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.746+0000 7f4da617c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4da006b460 con 0x7f4da009e1d0 2023-09-30T22:29:25.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.746+0000 7f4da617c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4da009a7e0 con 0x7f4da0139170 2023-09-30T22:29:25.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.746+0000 7f4d9f7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4da009d6f0 0x7f4da009dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33936/0 (socket says 172.21.15.16:33936) 2023-09-30T22:29:25.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.746+0000 7f4d9f7fe700 1 -- 172.21.15.16:0/2972082224 learned_addr learned my addr 172.21.15.16:0/2972082224 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:29:25.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.746+0000 7f4d9ffff700 1 --2- 172.21.15.16:0/2972082224 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4da0139170 0x7f4da013b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:25.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.746+0000 7f4d9effd700 1 --2- 172.21.15.16:0/2972082224 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4da009e1d0 0x7f4da0138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:25.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.746+0000 7f4d9e7fc700 1 -- 172.21.15.16:0/2972082224 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4215894209 0 0) 0x7f4da009b120 con 0x7f4da009d6f0 2023-09-30T22:29:25.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.746+0000 7f4d9e7fc700 1 -- 172.21.15.16:0/2972082224 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d8c0035b0 con 0x7f4da009d6f0 2023-09-30T22:29:25.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.746+0000 7f4d9ffff700 1 -- 172.21.15.16:0/2972082224 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4da009e1d0 msgr2=0x7f4da0138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:25.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.746+0000 7f4d9ffff700 1 --2- 172.21.15.16:0/2972082224 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4da009e1d0 0x7f4da0138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:25.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.746+0000 7f4d9ffff700 1 -- 172.21.15.16:0/2972082224 >> v1:172.21.15.16:6789/0 conn(0x7f4da009d6f0 legacy=0x7f4da009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:25.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.746+0000 7f4d9ffff700 1 -- 172.21.15.16:0/2972082224 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da009a500 con 0x7f4da0139170 2023-09-30T22:29:25.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.747+0000 7f4d9effd700 1 --2- 172.21.15.16:0/2972082224 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4da009e1d0 0x7f4da0138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:25.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.747+0000 7f4d9ffff700 1 --2- 172.21.15.16:0/2972082224 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4da0139170 0x7f4da013b550 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4d98009df0 tx=0x7f4d9800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47ad3475d9ea9a95 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:25.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.747+0000 7f4d9e7fc700 1 -- 172.21.15.16:0/2972082224 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9800e070 con 0x7f4da0139170 2023-09-30T22:29:25.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.747+0000 7f4d9e7fc700 1 -- 172.21.15.16:0/2972082224 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d9800bae0 con 0x7f4da0139170 2023-09-30T22:29:25.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.747+0000 7f4d9e7fc700 1 -- 172.21.15.16:0/2972082224 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9800be90 con 0x7f4da0139170 2023-09-30T22:29:25.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.747+0000 7f4da617c700 1 -- 172.21.15.16:0/2972082224 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4da0139170 msgr2=0x7f4da013b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:25.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.747+0000 7f4da617c700 1 --2- 172.21.15.16:0/2972082224 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4da0139170 0x7f4da013b550 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4d98009df0 tx=0x7f4d9800b330 comp rx=0 tx=0).stop 2023-09-30T22:29:25.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.748+0000 7f4da617c700 1 -- 172.21.15.16:0/2972082224 shutdown_connections 2023-09-30T22:29:25.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.748+0000 7f4da617c700 1 --2- 172.21.15.16:0/2972082224 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4da0139170 0x7f4da013b550 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:25.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.748+0000 7f4da617c700 1 --2- 172.21.15.16:0/2972082224 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4da009e1d0 0x7f4da0138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:25.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.748+0000 7f4da617c700 1 -- 172.21.15.16:0/2972082224 >> 172.21.15.16:0/2972082224 conn(0x7f4da0066f50 msgr2=0x7f4da009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:25.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.748+0000 7f4da617c700 1 -- 172.21.15.16:0/2972082224 shutdown_connections 2023-09-30T22:29:25.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.748+0000 7f4da617c700 1 -- 172.21.15.16:0/2972082224 wait complete. 2023-09-30T22:29:25.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.749+0000 7f4da617c700 1 Processor -- start 2023-09-30T22:29:25.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.749+0000 7f4da617c700 1 -- start start 2023-09-30T22:29:25.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.749+0000 7f4da617c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4da009d6f0 0x7f4da0121730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:25.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.749+0000 7f4da617c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4da009e1d0 0x7f4da0121c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:25.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.749+0000 7f4da617c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4da0139170 0x7f4da01221b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:25.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.749+0000 7f4da617c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da0123530 con 0x7f4da0139170 2023-09-30T22:29:25.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.749+0000 7f4da617c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4da0123340 con 0x7f4da009d6f0 2023-09-30T22:29:25.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.749+0000 7f4da617c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4da01376b0 con 0x7f4da009e1d0 2023-09-30T22:29:25.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.749+0000 7f4d9effd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4da009e1d0 0x7f4da0121c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:25.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.749+0000 7f4d9ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4da0139170 0x7f4da01221b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:25.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.749+0000 7f4d9effd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4da009e1d0 0x7f4da0121c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:56116/0 (socket says 172.21.15.16:56116) 2023-09-30T22:29:25.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.749+0000 7f4d9effd700 1 -- 172.21.15.16:0/405519970 learned_addr learned my addr 172.21.15.16:0/405519970 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:25.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.749+0000 7f4d9ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4da0139170 0x7f4da01221b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57954/0 (socket says 172.21.15.16:57954) 2023-09-30T22:29:25.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.749+0000 7f4d9f7fe700 1 --2- 172.21.15.16:0/405519970 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4da009d6f0 0x7f4da0121730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:25.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.750+0000 7f4d9effd700 1 -- 172.21.15.16:0/405519970 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4da009d6f0 msgr2=0x7f4da0121730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:25.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.750+0000 7f4d9effd700 1 --2- 172.21.15.16:0/405519970 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4da009d6f0 0x7f4da0121730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:25.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.750+0000 7f4d9effd700 1 -- 172.21.15.16:0/405519970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4da0139170 msgr2=0x7f4da01221b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:25.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.750+0000 7f4d9effd700 1 --2- 172.21.15.16:0/405519970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4da0139170 0x7f4da01221b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:25.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.750+0000 7f4d9effd700 1 -- 172.21.15.16:0/405519970 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d880094a0 con 0x7f4da009e1d0 2023-09-30T22:29:25.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.750+0000 7f4d9ffff700 1 --2- 172.21.15.16:0/405519970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4da0139170 0x7f4da01221b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:25.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.750+0000 7f4d9effd700 1 --2- 172.21.15.16:0/405519970 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4da009e1d0 0x7f4da0121c70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4d8800ebe0 tx=0x7f4d8800eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:25.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.750+0000 7f4d97fff700 1 -- 172.21.15.16:0/405519970 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8800c850 con 0x7f4da009e1d0 2023-09-30T22:29:25.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.750+0000 7f4d97fff700 1 -- 172.21.15.16:0/405519970 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d8800c9b0 con 0x7f4da009e1d0 2023-09-30T22:29:25.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.750+0000 7f4da617c700 1 -- 172.21.15.16:0/405519970 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d98009a70 con 0x7f4da009e1d0 2023-09-30T22:29:25.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.750+0000 7f4d97fff700 1 -- 172.21.15.16:0/405519970 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8800cd40 con 0x7f4da009e1d0 2023-09-30T22:29:25.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.750+0000 7f4da617c700 1 -- 172.21.15.16:0/405519970 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4da0124f70 con 0x7f4da009e1d0 2023-09-30T22:29:25.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.751+0000 7f4d97fff700 1 -- 172.21.15.16:0/405519970 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f4d88013470 con 0x7f4da009e1d0 2023-09-30T22:29:25.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.751+0000 7f4d97fff700 1 -- 172.21.15.16:0/405519970 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 4049+0+0 (secure 0 0 0) 0x7f4d88040020 con 0x7f4da009e1d0 2023-09-30T22:29:25.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.751+0000 7f4da617c700 1 -- 172.21.15.16:0/405519970 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4da0123b90 con 0x7f4da009e1d0 2023-09-30T22:29:25.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.756+0000 7f4d97fff700 1 -- 172.21.15.16:0/405519970 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4d88021080 con 0x7f4da009e1d0 2023-09-30T22:29:25.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:25.973+0000 7f4da617c700 1 -- 172.21.15.16:0/405519970 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd down", "ids": ["1"]} v 0) v1 -- 0x7f4da0074ea0 con 0x7f4da009e1d0 2023-09-30T22:29:25.973 INFO:tasks.ceph.mon.a.smithi016.stderr:2023-09-30T22:29:25.974+0000 7f081fac5700 -1 mon.a@0(leader).osd e23 definitely_dead 0 2023-09-30T22:29:26.138 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-09-30T22:29:26.138+0000 7f7223727680 -1 Falling back to public interface 2023-09-30T22:29:26.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.515+0000 7f4d97fff700 1 -- 172.21.15.16:0/405519970 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd down", "ids": ["1"]}]=0 marked down osd.1. v24) v1 ==== 89+0+0 (secure 0 0 0) 0x7f4d88017020 con 0x7f4da009e1d0 2023-09-30T22:29:26.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.519+0000 7f4da617c700 1 -- 172.21.15.16:0/405519970 >> v1:172.21.15.16:6816/126811 conn(0x7f4d8002f510 legacy=0x7f4d800319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:26.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.519+0000 7f4da617c700 1 -- 172.21.15.16:0/405519970 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4da009e1d0 msgr2=0x7f4da0121c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:26.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.519+0000 7f4da617c700 1 --2- 172.21.15.16:0/405519970 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4da009e1d0 0x7f4da0121c70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4d8800ebe0 tx=0x7f4d8800eef0 comp rx=0 tx=0).stop 2023-09-30T22:29:26.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.524+0000 7f4da617c700 1 -- 172.21.15.16:0/405519970 shutdown_connections 2023-09-30T22:29:26.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.524+0000 7f4da617c700 1 --2- 172.21.15.16:0/405519970 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4da009d6f0 0x7f4da0121730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:26.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.524+0000 7f4da617c700 1 --2- 172.21.15.16:0/405519970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4da0139170 0x7f4da01221b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:26.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.524+0000 7f4da617c700 1 --2- 172.21.15.16:0/405519970 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4da009e1d0 0x7f4da0121c70 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:26.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.524+0000 7f4da617c700 1 -- 172.21.15.16:0/405519970 >> 172.21.15.16:0/405519970 conn(0x7f4da0066f50 msgr2=0x7f4da009b410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:26.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.525+0000 7f4da617c700 1 -- 172.21.15.16:0/405519970 shutdown_connections 2023-09-30T22:29:26.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.525+0000 7f4da617c700 1 -- 172.21.15.16:0/405519970 wait complete. 2023-09-30T22:29:26.530 INFO:teuthology.orchestra.run.smithi016.stderr:marked down osd.1. 2023-09-30T22:29:26.545 INFO:tasks.ceph.osd.1:Restarting daemon 2023-09-30T22:29:26.545 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-09-30T22:29:26.547 INFO:tasks.ceph.osd.1:Started 2023-09-30T22:29:26.548 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-09-30T22:29:26.548 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T22:29:26.575 INFO:tasks.ceph.osd.2:Stopped 2023-09-30T22:29:26.575 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 2 2023-09-30T22:29:26.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.752+0000 7fd53c66d700 1 Processor -- start 2023-09-30T22:29:26.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.752+0000 7fd53c66d700 1 -- start start 2023-09-30T22:29:26.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.752+0000 7fd53c66d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd5340a0a30 0x7fd5340a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:26.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.752+0000 7fd53c66d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd53409d630 0x7fd5340a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:26.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.752+0000 7fd53c66d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5340a2a70 con 0x7fd534133930 2023-09-30T22:29:26.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.752+0000 7fd53c66d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5340a01d0 con 0x7fd5340a0a30 2023-09-30T22:29:26.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.752+0000 7fd53c66d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd53409e2b0 con 0x7fd53409d630 2023-09-30T22:29:26.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.753+0000 7fd539c08700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd53409d630 0x7fd5340a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:26.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.753+0000 7fd53ac0a700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd534133930 0x7fd534133dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33944/0 (socket says 172.21.15.16:33944) 2023-09-30T22:29:26.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.753+0000 7fd53ac0a700 1 -- 172.21.15.16:0/113621057 learned_addr learned my addr 172.21.15.16:0/113621057 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:29:26.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.753+0000 7fd53a409700 1 --2- 172.21.15.16:0/113621057 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd5340a0a30 0x7fd5340a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:26.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.753+0000 7fd539407700 1 -- 172.21.15.16:0/113621057 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3153518940 0 0) 0x7fd5340a2a70 con 0x7fd534133930 2023-09-30T22:29:26.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.753+0000 7fd53a409700 1 -- 172.21.15.16:0/113621057 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd53409d630 msgr2=0x7fd5340a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:26.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.753+0000 7fd53a409700 1 --2- 172.21.15.16:0/113621057 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd53409d630 0x7fd5340a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:26.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.753+0000 7fd53a409700 1 -- 172.21.15.16:0/113621057 >> v1:172.21.15.16:6789/0 conn(0x7fd534133930 legacy=0x7fd534133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:26.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.753+0000 7fd53a409700 1 -- 172.21.15.16:0/113621057 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd53409dc50 con 0x7fd5340a0a30 2023-09-30T22:29:26.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.753+0000 7fd539c08700 1 --2- 172.21.15.16:0/113621057 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd53409d630 0x7fd5340a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:26.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.753+0000 7fd53a409700 1 --2- 172.21.15.16:0/113621057 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd5340a0a30 0x7fd5340a0e00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd530009870 tx=0x7fd530009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b02e3e00171aade8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:26.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.753+0000 7fd539407700 1 -- 172.21.15.16:0/113621057 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd530011830 con 0x7fd5340a0a30 2023-09-30T22:29:26.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.753+0000 7fd539407700 1 -- 172.21.15.16:0/113621057 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd530011990 con 0x7fd5340a0a30 2023-09-30T22:29:26.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.753+0000 7fd539407700 1 -- 172.21.15.16:0/113621057 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd530011d20 con 0x7fd5340a0a30 2023-09-30T22:29:26.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.754+0000 7fd53c66d700 1 -- 172.21.15.16:0/113621057 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd5340a0a30 msgr2=0x7fd5340a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:26.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.754+0000 7fd53c66d700 1 --2- 172.21.15.16:0/113621057 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd5340a0a30 0x7fd5340a0e00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd530009870 tx=0x7fd530009b80 comp rx=0 tx=0).stop 2023-09-30T22:29:26.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.754+0000 7fd53c66d700 1 -- 172.21.15.16:0/113621057 shutdown_connections 2023-09-30T22:29:26.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.754+0000 7fd53c66d700 1 --2- 172.21.15.16:0/113621057 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd5340a0a30 0x7fd5340a0e00 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:26.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.754+0000 7fd53c66d700 1 --2- 172.21.15.16:0/113621057 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd53409d630 0x7fd5340a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:26.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.754+0000 7fd53c66d700 1 -- 172.21.15.16:0/113621057 >> 172.21.15.16:0/113621057 conn(0x7fd5340a3050 msgr2=0x7fd5340a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:26.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.754+0000 7fd53c66d700 1 -- 172.21.15.16:0/113621057 shutdown_connections 2023-09-30T22:29:26.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.754+0000 7fd53c66d700 1 -- 172.21.15.16:0/113621057 wait complete. 2023-09-30T22:29:26.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.755+0000 7fd53c66d700 1 Processor -- start 2023-09-30T22:29:26.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.756+0000 7fd53c66d700 1 -- start start 2023-09-30T22:29:26.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.756+0000 7fd53c66d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd53409d630 0x7fd53411f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:26.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.756+0000 7fd53c66d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5340a0a30 0x7fd53411f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:26.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.756+0000 7fd53c66d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd534133930 0x7fd5341c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:26.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.756+0000 7fd53c66d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd534121ed0 con 0x7fd5340a0a30 2023-09-30T22:29:26.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.756+0000 7fd53c66d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd534133250 con 0x7fd53409d630 2023-09-30T22:29:26.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.756+0000 7fd53c66d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd53411ee20 con 0x7fd534133930 2023-09-30T22:29:26.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.756+0000 7fd53a409700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd53409d630 0x7fd53411f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:26.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.756+0000 7fd53a409700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd53409d630 0x7fd53411f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37394/0 (socket says 172.21.15.16:37394) 2023-09-30T22:29:26.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.756+0000 7fd53a409700 1 -- 172.21.15.16:0/2839119935 learned_addr learned my addr 172.21.15.16:0/2839119935 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:26.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.756+0000 7fd53ac0a700 1 --2- 172.21.15.16:0/2839119935 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd534133930 0x7fd5341c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:26.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.756+0000 7fd539c08700 1 --2- 172.21.15.16:0/2839119935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5340a0a30 0x7fd53411f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:26.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.757+0000 7fd53a409700 1 -- 172.21.15.16:0/2839119935 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd534133930 msgr2=0x7fd5341c7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:26.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.757+0000 7fd53a409700 1 --2- 172.21.15.16:0/2839119935 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd534133930 0x7fd5341c7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:26.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.757+0000 7fd53a409700 1 -- 172.21.15.16:0/2839119935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5340a0a30 msgr2=0x7fd53411f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:26.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.757+0000 7fd53a409700 1 --2- 172.21.15.16:0/2839119935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5340a0a30 0x7fd53411f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:26.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.757+0000 7fd53a409700 1 -- 172.21.15.16:0/2839119935 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5240094a0 con 0x7fd53409d630 2023-09-30T22:29:26.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.757+0000 7fd53a409700 1 --2- 172.21.15.16:0/2839119935 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd53409d630 0x7fd53411f2c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd5340a2c00 tx=0x7fd530004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:26.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.757+0000 7fd52affd700 1 -- 172.21.15.16:0/2839119935 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd530004300 con 0x7fd53409d630 2023-09-30T22:29:26.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.757+0000 7fd52affd700 1 -- 172.21.15.16:0/2839119935 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd53001f070 con 0x7fd53409d630 2023-09-30T22:29:26.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.757+0000 7fd53c66d700 1 -- 172.21.15.16:0/2839119935 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5300094a0 con 0x7fd53409d630 2023-09-30T22:29:26.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.757+0000 7fd52affd700 1 -- 172.21.15.16:0/2839119935 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5300036a0 con 0x7fd53409d630 2023-09-30T22:29:26.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.757+0000 7fd53c66d700 1 -- 172.21.15.16:0/2839119935 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd534120520 con 0x7fd53409d630 2023-09-30T22:29:26.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.758+0000 7fd52affd700 1 -- 172.21.15.16:0/2839119935 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fd530021030 con 0x7fd53409d630 2023-09-30T22:29:26.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.758+0000 7fd53c66d700 1 -- 172.21.15.16:0/2839119935 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd53411fda0 con 0x7fd53409d630 2023-09-30T22:29:26.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.758+0000 7fd52affd700 1 -- 172.21.15.16:0/2839119935 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 4049+0+0 (secure 0 0 0) 0x7fd530040d50 con 0x7fd53409d630 2023-09-30T22:29:26.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.763+0000 7fd52affd700 1 -- 172.21.15.16:0/2839119935 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd530019770 con 0x7fd53409d630 2023-09-30T22:29:26.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:26.979+0000 7fd53c66d700 1 -- 172.21.15.16:0/2839119935 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd down", "ids": ["2"]} v 0) v1 -- 0x7fd534120a40 con 0x7fd53409d630 2023-09-30T22:29:26.980 INFO:tasks.ceph.mon.a.smithi016.stderr:2023-09-30T22:29:26.980+0000 7f081fac5700 -1 mon.a@0(leader).osd e24 definitely_dead 0 2023-09-30T22:29:27.150 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-09-30T22:29:27.150+0000 7f78f0488680 -1 Falling back to public interface 2023-09-30T22:29:27.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.521+0000 7fd52affd700 1 -- 172.21.15.16:0/2839119935 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd down", "ids": ["2"]}]=0 marked down osd.2. v25) v1 ==== 89+0+0 (secure 0 0 0) 0x7fd53001fae0 con 0x7fd53409d630 2023-09-30T22:29:27.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.525+0000 7fd53c66d700 1 -- 172.21.15.16:0/2839119935 >> v1:172.21.15.16:6816/126811 conn(0x7fd51802f510 legacy=0x7fd5180319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:27.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.525+0000 7fd53c66d700 1 -- 172.21.15.16:0/2839119935 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd53409d630 msgr2=0x7fd53411f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:27.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.525+0000 7fd53c66d700 1 --2- 172.21.15.16:0/2839119935 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd53409d630 0x7fd53411f2c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd5340a2c00 tx=0x7fd530004750 comp rx=0 tx=0).stop 2023-09-30T22:29:27.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.525+0000 7fd53c66d700 1 -- 172.21.15.16:0/2839119935 shutdown_connections 2023-09-30T22:29:27.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.525+0000 7fd53c66d700 1 --2- 172.21.15.16:0/2839119935 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd53409d630 0x7fd53411f2c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:27.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.525+0000 7fd53c66d700 1 --2- 172.21.15.16:0/2839119935 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd534133930 0x7fd5341c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:27.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.525+0000 7fd53c66d700 1 --2- 172.21.15.16:0/2839119935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5340a0a30 0x7fd53411f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:27.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.525+0000 7fd53c66d700 1 -- 172.21.15.16:0/2839119935 >> 172.21.15.16:0/2839119935 conn(0x7fd5340a3050 msgr2=0x7fd5341bbb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:27.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.526+0000 7fd53c66d700 1 -- 172.21.15.16:0/2839119935 shutdown_connections 2023-09-30T22:29:27.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.526+0000 7fd53c66d700 1 -- 172.21.15.16:0/2839119935 wait complete. 2023-09-30T22:29:27.528 INFO:teuthology.orchestra.run.smithi016.stderr:marked down osd.2. 2023-09-30T22:29:27.545 INFO:tasks.ceph.osd.2:Restarting daemon 2023-09-30T22:29:27.545 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-09-30T22:29:27.547 INFO:tasks.ceph.osd.2:Started 2023-09-30T22:29:27.548 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-09-30T22:29:27.548 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T22:29:27.584 INFO:tasks.ceph.osd.3:Stopped 2023-09-30T22:29:27.584 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 3 2023-09-30T22:29:27.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.762+0000 7f0dee953700 1 Processor -- start 2023-09-30T22:29:27.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.762+0000 7f0dee953700 1 -- start start 2023-09-30T22:29:27.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.763+0000 7f0dee953700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0de809d6f0 0x7f0de809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:27.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.763+0000 7f0dee953700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0de809e000 0x7f0de81339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:27.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.763+0000 7f0dee953700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0de809e410 0x7f0de8133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:27.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.763+0000 7f0dee953700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0de806b460 con 0x7f0de809e000 2023-09-30T22:29:27.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.763+0000 7f0dee953700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0de809b120 con 0x7f0de809e410 2023-09-30T22:29:27.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.763+0000 7f0dee953700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0de809a7e0 con 0x7f0de809d6f0 2023-09-30T22:29:27.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.763+0000 7f0de7fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0de809d6f0 0x7f0de809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:27.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.763+0000 7f0de77fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0de809e000 0x7f0de81339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:27.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.763+0000 7f0decef0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0de809e410 0x7f0de8133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:27.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.763+0000 7f0decef0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0de809e410 0x7f0de8133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54214/0 (socket says 172.21.15.16:54214) 2023-09-30T22:29:27.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.763+0000 7f0de7fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0de809d6f0 0x7f0de809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:36106/0 (socket says 172.21.15.16:36106) 2023-09-30T22:29:27.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.763+0000 7f0de77fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0de809e000 0x7f0de81339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37558/0 (socket says 172.21.15.16:37558) 2023-09-30T22:29:27.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.763+0000 7f0decef0700 1 -- 172.21.15.16:0/2031999041 learned_addr learned my addr 172.21.15.16:0/2031999041 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:27.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.763+0000 7f0decef0700 1 -- 172.21.15.16:0/2031999041 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0de809d6f0 msgr2=0x7f0de809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:27.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.763+0000 7f0decef0700 1 --2- 172.21.15.16:0/2031999041 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0de809d6f0 0x7f0de809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:27.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.763+0000 7f0decef0700 1 -- 172.21.15.16:0/2031999041 >> v2:172.21.15.16:3300/0 conn(0x7f0de809e000 msgr2=0x7f0de81339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:27.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.763+0000 7f0decef0700 1 --2- 172.21.15.16:0/2031999041 >> v2:172.21.15.16:3300/0 conn(0x7f0de809e000 0x7f0de81339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:27.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.763+0000 7f0decef0700 1 -- 172.21.15.16:0/2031999041 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0de809a500 con 0x7f0de809e410 2023-09-30T22:29:27.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.763+0000 7f0de7fff700 1 --2- 172.21.15.16:0/2031999041 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0de809d6f0 0x7f0de809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:27.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.763+0000 7f0decef0700 1 --2- 172.21.15.16:0/2031999041 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0de809e410 0x7f0de8133f30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0de0009e80 tx=0x7f0de000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f811216f79138952 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:27.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.763+0000 7f0de6ffd700 1 -- 172.21.15.16:0/2031999041 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de000e070 con 0x7f0de809e410 2023-09-30T22:29:27.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.764+0000 7f0de6ffd700 1 -- 172.21.15.16:0/2031999041 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0de000b8d0 con 0x7f0de809e410 2023-09-30T22:29:27.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.764+0000 7f0de6ffd700 1 -- 172.21.15.16:0/2031999041 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de000bc80 con 0x7f0de809e410 2023-09-30T22:29:27.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.764+0000 7f0dee953700 1 -- 172.21.15.16:0/2031999041 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0de809e410 msgr2=0x7f0de8133f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:27.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.764+0000 7f0dee953700 1 --2- 172.21.15.16:0/2031999041 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0de809e410 0x7f0de8133f30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0de0009e80 tx=0x7f0de000b330 comp rx=0 tx=0).stop 2023-09-30T22:29:27.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.764+0000 7f0dee953700 1 -- 172.21.15.16:0/2031999041 shutdown_connections 2023-09-30T22:29:27.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.764+0000 7f0dee953700 1 --2- 172.21.15.16:0/2031999041 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0de809e410 0x7f0de8133f30 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:27.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.764+0000 7f0dee953700 1 --2- 172.21.15.16:0/2031999041 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0de809d6f0 0x7f0de809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:27.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.764+0000 7f0dee953700 1 --2- 172.21.15.16:0/2031999041 >> v2:172.21.15.16:3300/0 conn(0x7f0de809e000 0x7f0de81339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:27.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.764+0000 7f0dee953700 1 -- 172.21.15.16:0/2031999041 >> 172.21.15.16:0/2031999041 conn(0x7f0de8066f50 msgr2=0x7f0de809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:27.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.764+0000 7f0dee953700 1 -- 172.21.15.16:0/2031999041 shutdown_connections 2023-09-30T22:29:27.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.764+0000 7f0dee953700 1 -- 172.21.15.16:0/2031999041 wait complete. 2023-09-30T22:29:27.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.765+0000 7f0dee953700 1 Processor -- start 2023-09-30T22:29:27.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.765+0000 7f0dee953700 1 -- start start 2023-09-30T22:29:27.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.766+0000 7f0dee953700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0de809d6f0 0x7f0de81c7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:27.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.766+0000 7f0dee953700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0de809e000 0x7f0de81c84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:27.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.766+0000 7f0dee953700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0de809e410 0x7f0de81c8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:27.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.766+0000 7f0dee953700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de81a6420 con 0x7f0de809e000 2023-09-30T22:29:27.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.766+0000 7f0dee953700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0de81a7c10 con 0x7f0de809e410 2023-09-30T22:29:27.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.766+0000 7f0dee953700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0de809c710 con 0x7f0de809d6f0 2023-09-30T22:29:27.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.766+0000 7f0de7fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0de809d6f0 0x7f0de81c7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:27.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.766+0000 7f0de77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0de809e000 0x7f0de81c84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:27.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.766+0000 7f0de7fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0de809d6f0 0x7f0de81c7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:36118/0 (socket says 172.21.15.16:36118) 2023-09-30T22:29:27.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.766+0000 7f0de7fff700 1 -- 172.21.15.16:0/3357790160 learned_addr learned my addr 172.21.15.16:0/3357790160 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:27.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.766+0000 7f0decef0700 1 --2- 172.21.15.16:0/3357790160 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0de809e410 0x7f0de81c8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:27.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.766+0000 7f0de77fe700 1 -- 172.21.15.16:0/3357790160 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0de809d6f0 msgr2=0x7f0de81c7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:27.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.766+0000 7f0de77fe700 1 --2- 172.21.15.16:0/3357790160 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0de809d6f0 0x7f0de81c7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:27.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.766+0000 7f0de77fe700 1 -- 172.21.15.16:0/3357790160 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0de809e410 msgr2=0x7f0de81c8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:27.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.766+0000 7f0de77fe700 1 --2- 172.21.15.16:0/3357790160 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0de809e410 0x7f0de81c8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:27.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.766+0000 7f0de77fe700 1 -- 172.21.15.16:0/3357790160 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dd80094a0 con 0x7f0de809e000 2023-09-30T22:29:27.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.767+0000 7f0de77fe700 1 --2- 172.21.15.16:0/3357790160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0de809e000 0x7f0de81c84f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc00e8d0 tx=0x7f0ddc00ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:27.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.767+0000 7f0de4ff9700 1 -- 172.21.15.16:0/3357790160 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc013680 con 0x7f0de809e000 2023-09-30T22:29:27.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.767+0000 7f0de4ff9700 1 -- 172.21.15.16:0/3357790160 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ddc0137e0 con 0x7f0de809e000 2023-09-30T22:29:27.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.767+0000 7f0de4ff9700 1 -- 172.21.15.16:0/3357790160 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc013b50 con 0x7f0de809e000 2023-09-30T22:29:27.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.767+0000 7f0dee953700 1 -- 172.21.15.16:0/3357790160 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0de0009b30 con 0x7f0de809e000 2023-09-30T22:29:27.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.767+0000 7f0dee953700 1 -- 172.21.15.16:0/3357790160 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0de81a6f10 con 0x7f0de809e000 2023-09-30T22:29:27.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.768+0000 7f0de4ff9700 1 -- 172.21.15.16:0/3357790160 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f0ddc013cb0 con 0x7f0de809e000 2023-09-30T22:29:27.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.768+0000 7f0de4ff9700 1 -- 172.21.15.16:0/3357790160 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 4049+0+0 (secure 0 0 0) 0x7f0ddc03fe40 con 0x7f0de809e000 2023-09-30T22:29:27.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.768+0000 7f0dee953700 1 -- 172.21.15.16:0/3357790160 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0de8074ea0 con 0x7f0de809e000 2023-09-30T22:29:27.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.773+0000 7f0de4ff9700 1 -- 172.21.15.16:0/3357790160 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0ddc021080 con 0x7f0de809e000 2023-09-30T22:29:27.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:27.985+0000 7f0dee953700 1 -- 172.21.15.16:0/3357790160 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd down", "ids": ["3"]} v 0) v1 -- 0x7f0de806c360 con 0x7f0de809e000 2023-09-30T22:29:27.986 INFO:tasks.ceph.mon.a.smithi016.stderr:2023-09-30T22:29:27.986+0000 7f081fac5700 -1 mon.a@0(leader).osd e25 definitely_dead 0 2023-09-30T22:29:28.159 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-09-30T22:29:28.159+0000 7faf73779680 -1 Falling back to public interface 2023-09-30T22:29:28.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.522+0000 7f0de4ff9700 1 -- 172.21.15.16:0/3357790160 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd down", "ids": ["3"]}]=0 marked down osd.3. v26) v1 ==== 89+0+0 (secure 0 0 0) 0x7f0ddc017020 con 0x7f0de809e000 2023-09-30T22:29:28.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.526+0000 7f0dee953700 1 -- 172.21.15.16:0/3357790160 >> v1:172.21.15.16:6816/126811 conn(0x7f0dd402f510 legacy=0x7f0dd40319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:28.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.527+0000 7f0dee953700 1 -- 172.21.15.16:0/3357790160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0de809e000 msgr2=0x7f0de81c84f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:28.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.527+0000 7f0dee953700 1 --2- 172.21.15.16:0/3357790160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0de809e000 0x7f0de81c84f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc00e8d0 tx=0x7f0ddc00ebe0 comp rx=0 tx=0).stop 2023-09-30T22:29:28.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.527+0000 7f0dee953700 1 -- 172.21.15.16:0/3357790160 shutdown_connections 2023-09-30T22:29:28.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.527+0000 7f0dee953700 1 --2- 172.21.15.16:0/3357790160 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0de809e410 0x7f0de81c8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:28.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.527+0000 7f0dee953700 1 --2- 172.21.15.16:0/3357790160 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0de809d6f0 0x7f0de81c7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:28.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.527+0000 7f0dee953700 1 --2- 172.21.15.16:0/3357790160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0de809e000 0x7f0de81c84f0 secure :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc00e8d0 tx=0x7f0ddc00ebe0 comp rx=0 tx=0).stop 2023-09-30T22:29:28.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.527+0000 7f0dee953700 1 -- 172.21.15.16:0/3357790160 >> 172.21.15.16:0/3357790160 conn(0x7f0de8066f50 msgr2=0x7f0de81bbb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:28.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.527+0000 7f0dee953700 1 -- 172.21.15.16:0/3357790160 shutdown_connections 2023-09-30T22:29:28.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.527+0000 7f0dee953700 1 -- 172.21.15.16:0/3357790160 wait complete. 2023-09-30T22:29:28.529 INFO:teuthology.orchestra.run.smithi016.stderr:marked down osd.3. 2023-09-30T22:29:28.544 INFO:tasks.ceph.osd.3:Restarting daemon 2023-09-30T22:29:28.545 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-09-30T22:29:28.547 INFO:tasks.ceph.osd.3:Started 2023-09-30T22:29:28.547 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-09-30T22:29:28.547 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:29:28.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.687+0000 7f9c8de51700 1 Processor -- start 2023-09-30T22:29:28.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.687+0000 7f9c8de51700 1 -- start start 2023-09-30T22:29:28.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.688+0000 7f9c8de51700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9c8809d6f0 0x7f9c8809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:28.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.688+0000 7f9c8de51700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9c8809e000 0x7f9c88130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:28.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.688+0000 7f9c8de51700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c8809f130 con 0x7f9c8809e410 2023-09-30T22:29:28.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.688+0000 7f9c8de51700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c8809f520 con 0x7f9c8809d6f0 2023-09-30T22:29:28.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.688+0000 7f9c8de51700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8809e930 con 0x7f9c8809e000 2023-09-30T22:29:28.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.688+0000 7f9c877fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9c8809d6f0 0x7f9c8809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:28.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.688+0000 7f9c86ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9c8809e000 0x7f9c88130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:28.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.688+0000 7f9c87fff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9c8809e410 0x7f9c88130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40618/0 (socket says 172.21.15.16:40618) 2023-09-30T22:29:28.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.688+0000 7f9c877fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9c8809d6f0 0x7f9c8809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37580/0 (socket says 172.21.15.16:37580) 2023-09-30T22:29:28.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.688+0000 7f9c87fff700 1 -- 172.21.15.16:0/2696909108 learned_addr learned my addr 172.21.15.16:0/2696909108 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:29:28.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.688+0000 7f9c86ffd700 1 -- 172.21.15.16:0/2696909108 >> v2:172.21.15.16:3300/0 conn(0x7f9c8809d6f0 msgr2=0x7f9c8809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:28.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.688+0000 7f9c86ffd700 1 --2- 172.21.15.16:0/2696909108 >> v2:172.21.15.16:3300/0 conn(0x7f9c8809d6f0 0x7f9c8809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:28.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.688+0000 7f9c86ffd700 1 -- 172.21.15.16:0/2696909108 >> v1:172.21.15.16:6789/0 conn(0x7f9c8809e410 legacy=0x7f9c88130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:28.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.688+0000 7f9c86ffd700 1 -- 172.21.15.16:0/2696909108 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c8809b2d0 con 0x7f9c8809e000 2023-09-30T22:29:28.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.688+0000 7f9c867fc700 1 -- 172.21.15.16:0/2696909108 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3078345328 0 0) 0x7f9c8809f130 con 0x7f9c8809e410 2023-09-30T22:29:28.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.688+0000 7f9c877fe700 1 --2- 172.21.15.16:0/2696909108 >> v2:172.21.15.16:3300/0 conn(0x7f9c8809d6f0 0x7f9c8809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:28.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.689+0000 7f9c86ffd700 1 --2- 172.21.15.16:0/2696909108 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9c8809e000 0x7f9c88130660 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c009840 tx=0x7f9c7c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31b3c52b5bdcf75b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:28.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.689+0000 7f9c867fc700 1 -- 172.21.15.16:0/2696909108 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c011830 con 0x7f9c8809e000 2023-09-30T22:29:28.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.689+0000 7f9c867fc700 1 -- 172.21.15.16:0/2696909108 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c7c011990 con 0x7f9c8809e000 2023-09-30T22:29:28.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.689+0000 7f9c867fc700 1 -- 172.21.15.16:0/2696909108 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c011d20 con 0x7f9c8809e000 2023-09-30T22:29:28.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.689+0000 7f9c8de51700 1 -- 172.21.15.16:0/2696909108 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9c8809e000 msgr2=0x7f9c88130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:28.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.689+0000 7f9c8de51700 1 --2- 172.21.15.16:0/2696909108 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9c8809e000 0x7f9c88130660 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c009840 tx=0x7f9c7c009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:28.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.689+0000 7f9c8de51700 1 -- 172.21.15.16:0/2696909108 shutdown_connections 2023-09-30T22:29:28.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.689+0000 7f9c8de51700 1 --2- 172.21.15.16:0/2696909108 >> v2:172.21.15.16:3300/0 conn(0x7f9c8809d6f0 0x7f9c8809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:28.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.689+0000 7f9c8de51700 1 --2- 172.21.15.16:0/2696909108 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9c8809e000 0x7f9c88130660 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:28.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.689+0000 7f9c8de51700 1 -- 172.21.15.16:0/2696909108 >> 172.21.15.16:0/2696909108 conn(0x7f9c88066f50 msgr2=0x7f9c8809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:28.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.690+0000 7f9c8de51700 1 -- 172.21.15.16:0/2696909108 shutdown_connections 2023-09-30T22:29:28.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.690+0000 7f9c8de51700 1 -- 172.21.15.16:0/2696909108 wait complete. 2023-09-30T22:29:28.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.691+0000 7f9c8de51700 1 Processor -- start 2023-09-30T22:29:28.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.691+0000 7f9c8de51700 1 -- start start 2023-09-30T22:29:28.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.691+0000 7f9c8de51700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c8809d6f0 0x7f9c88138960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:28.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.691+0000 7f9c8de51700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9c8809e000 0x7f9c88138ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:28.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.691+0000 7f9c8de51700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9c8809e410 0x7f9c881393e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:28.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.691+0000 7f9c8de51700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c88139e30 con 0x7f9c8809d6f0 2023-09-30T22:29:28.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.691+0000 7f9c8de51700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8812c2e0 con 0x7f9c8809e000 2023-09-30T22:29:28.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.691+0000 7f9c8de51700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9c88136ed0 con 0x7f9c8809e410 2023-09-30T22:29:28.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.691+0000 7f9c877fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c8809d6f0 0x7f9c88138960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:28.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.691+0000 7f9c877fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c8809d6f0 0x7f9c88138960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37582/0 (socket says 172.21.15.16:37582) 2023-09-30T22:29:28.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.691+0000 7f9c877fe700 1 -- 172.21.15.16:0/2559419141 learned_addr learned my addr 172.21.15.16:0/2559419141 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:28.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.691+0000 7f9c87fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9c8809e410 0x7f9c881393e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:28.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.691+0000 7f9c86ffd700 1 --2- 172.21.15.16:0/2559419141 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9c8809e000 0x7f9c88138ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:28.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.692+0000 7f9c877fe700 1 -- 172.21.15.16:0/2559419141 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9c8809e410 msgr2=0x7f9c881393e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:28.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.692+0000 7f9c877fe700 1 --2- 172.21.15.16:0/2559419141 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9c8809e410 0x7f9c881393e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:28.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.692+0000 7f9c877fe700 1 -- 172.21.15.16:0/2559419141 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9c8809e000 msgr2=0x7f9c88138ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:28.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.692+0000 7f9c877fe700 1 --2- 172.21.15.16:0/2559419141 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9c8809e000 0x7f9c88138ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:28.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.692+0000 7f9c877fe700 1 -- 172.21.15.16:0/2559419141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c780094a0 con 0x7f9c8809d6f0 2023-09-30T22:29:28.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.692+0000 7f9c86ffd700 1 --2- 172.21.15.16:0/2559419141 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9c8809e000 0x7f9c88138ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:28.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.692+0000 7f9c87fff700 1 --2- 172.21.15.16:0/2559419141 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9c8809e410 0x7f9c881393e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:28.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.692+0000 7f9c877fe700 1 --2- 172.21.15.16:0/2559419141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c8809d6f0 0x7f9c88138960 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9c7800ec10 tx=0x7f9c7800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:28.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.692+0000 7f9c6ffff700 1 -- 172.21.15.16:0/2559419141 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7800c850 con 0x7f9c8809d6f0 2023-09-30T22:29:28.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.692+0000 7f9c6ffff700 1 -- 172.21.15.16:0/2559419141 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c7800c9b0 con 0x7f9c8809d6f0 2023-09-30T22:29:28.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.692+0000 7f9c8de51700 1 -- 172.21.15.16:0/2559419141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c7c0094a0 con 0x7f9c8809d6f0 2023-09-30T22:29:28.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.692+0000 7f9c6ffff700 1 -- 172.21.15.16:0/2559419141 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7800cd20 con 0x7f9c8809d6f0 2023-09-30T22:29:28.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.692+0000 7f9c8de51700 1 -- 172.21.15.16:0/2559419141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c8809c700 con 0x7f9c8809d6f0 2023-09-30T22:29:28.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.693+0000 7f9c6ffff700 1 -- 172.21.15.16:0/2559419141 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f9c78013470 con 0x7f9c8809d6f0 2023-09-30T22:29:28.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.693+0000 7f9c6ffff700 1 -- 172.21.15.16:0/2559419141 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 4049+0+0 (secure 0 0 0) 0x7f9c7803fce0 con 0x7f9c8809d6f0 2023-09-30T22:29:28.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.693+0000 7f9c8de51700 1 -- 172.21.15.16:0/2559419141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c88074ea0 con 0x7f9c8809d6f0 2023-09-30T22:29:28.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.698+0000 7f9c6ffff700 1 -- 172.21.15.16:0/2559419141 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9c78013720 con 0x7f9c8809d6f0 2023-09-30T22:29:28.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.904+0000 7f9c8de51700 1 -- 172.21.15.16:0/2559419141 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9c88136400 con 0x7f9c8809d6f0 2023-09-30T22:29:28.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.905+0000 7f9c6ffff700 1 -- 172.21.15.16:0/2559419141 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+11946 (secure 0 0 0) 0x7f9c7801bba0 con 0x7f9c8809d6f0 2023-09-30T22:29:28.906 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:29:28.906 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":26,"fsid":"7ede3e14-2bd0-4726-9437-6d13c90cd342","created":"2023-09-30T22:26:35.483697+0000","modified":"2023-09-30T22:29:28.519844+0000","last_up_change":"2023-09-30T22:29:28.519844+0000","last_in_change":"2023-09-30T22:26:37.120906+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"luminous","require_osd_release":"pacific","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T22:26:44.285133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{},"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:26:45.132921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:26:49.353683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:26:50.655685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"19946985-0ee2-487f-9347-929670dbe5ff","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":23,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6804","nonce":118824}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6805","nonce":118824}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6807","nonce":118824}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6806","nonce":118824}]},"public_addr":"172.21.15.16:6804/118824","cluster_addr":"172.21.15.16:6805/118824","heartbeat_back_addr":"172.21.15.16:6807/118824","heartbeat_front_addr":"172.21.15.16:6806/118824","state":["exists"]},{"osd":1,"uuid":"8efd4dbc-6ae4-4b82-b530-3b3eac2be445","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":24,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6812","nonce":118832}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6813","nonce":118832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6815","nonce":118832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6814","nonce":118832}]},"public_addr":"172.21.15.16:6812/118832","cluster_addr":"172.21.15.16:6813/118832","heartbeat_back_addr":"172.21.15.16:6815/118832","heartbeat_front_addr":"172.21.15.16:6814/118832","state":["exists"]},{"osd":2,"uuid":"b47bb1ce-27af-4c0c-868a-b76ebd0d9499","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":24,"down_at":25,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6808","nonce":118840}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6809","nonce":118840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6811","nonce":118840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6810","nonce":118840}]},"public_addr":"172.21.15.16:6808/118840","cluster_addr":"172.21.15.16:6809/118840","heartbeat_back_addr":"172.21.15.16:6811/118840","heartbeat_front_addr":"172.21.15.16:6810/118840","state":["exists"]},{"osd":3,"uuid":"03b53d64-d10d-448a-a081-39ecceab589b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":25,"down_at":26,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6800","nonce":118823}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6801","nonce":118823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6803","nonce":118823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6802","nonce":118823}]},"public_addr":"172.21.15.16:6800/118823","cluster_addr":"172.21.15.16:6801/118823","heartbeat_back_addr":"172.21.15.16:6803/118823","heartbeat_front_addr":"172.21.15.16:6802/118823","state":["exists"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.305953+0000","dead_epoch":23},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.307488+0000","dead_epoch":24},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.341023+0000","dead_epoch":25},{"osd":3,"down_stamp":"2023-09-30T22:29:28.519844+0000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.192071+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.8","mappings":[{"from":3,"to":2}]},{"pgid":"4.d","mappings":[{"from":3,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":0}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]},{"pgid":"4.17","mappings":[{"from":3,"to":2}]},{"pgid":"4.19","mappings":[{"from":3,"to":0}]},{"pgid":"4.3a","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6816/118285":"2023-10-01T22:29:14.475978+0000","172.21.15.16:0/2249754484":"2023-10-01T22:29:14.475978+0000","172.21.15.16:0/1101648925":"2023-10-01T22:29:14.475978+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:29:28.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.908+0000 7f9c8de51700 1 -- 172.21.15.16:0/2559419141 >> v1:172.21.15.16:6816/126811 conn(0x7f9c7402f510 legacy=0x7f9c740319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:28.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.908+0000 7f9c8de51700 1 -- 172.21.15.16:0/2559419141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c8809d6f0 msgr2=0x7f9c88138960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:28.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.908+0000 7f9c8de51700 1 --2- 172.21.15.16:0/2559419141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c8809d6f0 0x7f9c88138960 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9c7800ec10 tx=0x7f9c7800ef20 comp rx=0 tx=0).stop 2023-09-30T22:29:28.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.908+0000 7f9c8de51700 1 -- 172.21.15.16:0/2559419141 shutdown_connections 2023-09-30T22:29:28.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.908+0000 7f9c8de51700 1 --2- 172.21.15.16:0/2559419141 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9c8809e410 0x7f9c881393e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:28.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.908+0000 7f9c8de51700 1 --2- 172.21.15.16:0/2559419141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c8809d6f0 0x7f9c88138960 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:28.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.908+0000 7f9c8de51700 1 --2- 172.21.15.16:0/2559419141 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9c8809e000 0x7f9c88138ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:28.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.908+0000 7f9c8de51700 1 -- 172.21.15.16:0/2559419141 >> 172.21.15.16:0/2559419141 conn(0x7f9c88066f50 msgr2=0x7f9c88070e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:28.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.908+0000 7f9c8de51700 1 -- 172.21.15.16:0/2559419141 shutdown_connections 2023-09-30T22:29:28.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:28.908+0000 7f9c8de51700 1 -- 172.21.15.16:0/2559419141 wait complete. 2023-09-30T22:29:29.152 INFO:tasks.ceph.osd.3.smithi016.stderr:2023-09-30T22:29:29.151+0000 7f6f27514680 -1 Falling back to public interface 2023-09-30T22:29:29.934 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-09-30T22:29:29.933+0000 7f7223727680 -1 osd.0 22 log_to_monitors true 2023-09-30T22:29:30.986 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-09-30T22:29:30.985+0000 7f78f0488680 -1 osd.1 22 log_to_monitors true 2023-09-30T22:29:31.377 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-09-30T22:29:31.375+0000 7f720f6a2700 -1 osd.0 22 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-30T22:29:31.926 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:29:31.955 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-09-30T22:29:31.954+0000 7faf73779680 -1 osd.2 24 log_to_monitors true 2023-09-30T22:29:32.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.126+0000 7f5f298c7700 1 Processor -- start 2023-09-30T22:29:32.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.126+0000 7f5f298c7700 1 -- start start 2023-09-30T22:29:32.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.127+0000 7f5f298c7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5f2409e1d0 0x7f5f24138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:32.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.127+0000 7f5f298c7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5f24139170 0x7f5f2413b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:32.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.127+0000 7f5f298c7700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f2409b120 con 0x7f5f2409d6f0 2023-09-30T22:29:32.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.127+0000 7f5f298c7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2406b460 con 0x7f5f2409e1d0 2023-09-30T22:29:32.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.127+0000 7f5f298c7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2409a7e0 con 0x7f5f24139170 2023-09-30T22:29:32.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.127+0000 7f5f22ffd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5f2409d6f0 0x7f5f2409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40672/0 (socket says 172.21.15.16:40672) 2023-09-30T22:29:32.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.127+0000 7f5f22ffd700 1 -- 172.21.15.16:0/1712235745 learned_addr learned my addr 172.21.15.16:0/1712235745 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:29:32.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.127+0000 7f5f237fe700 1 --2- 172.21.15.16:0/1712235745 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5f24139170 0x7f5f2413b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:32.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.127+0000 7f5f227fc700 1 --2- 172.21.15.16:0/1712235745 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5f2409e1d0 0x7f5f24138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:32.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.127+0000 7f5f21ffb700 1 -- 172.21.15.16:0/1712235745 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2804671070 0 0) 0x7f5f2409b120 con 0x7f5f2409d6f0 2023-09-30T22:29:32.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.127+0000 7f5f21ffb700 1 -- 172.21.15.16:0/1712235745 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f0c0035b0 con 0x7f5f2409d6f0 2023-09-30T22:29:32.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.127+0000 7f5f237fe700 1 -- 172.21.15.16:0/1712235745 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5f2409e1d0 msgr2=0x7f5f24138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:32.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.127+0000 7f5f237fe700 1 --2- 172.21.15.16:0/1712235745 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5f2409e1d0 0x7f5f24138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:32.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.127+0000 7f5f237fe700 1 -- 172.21.15.16:0/1712235745 >> v1:172.21.15.16:6789/0 conn(0x7f5f2409d6f0 legacy=0x7f5f2409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:32.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.127+0000 7f5f237fe700 1 -- 172.21.15.16:0/1712235745 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f2409a500 con 0x7f5f24139170 2023-09-30T22:29:32.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.128+0000 7f5f237fe700 1 --2- 172.21.15.16:0/1712235745 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5f24139170 0x7f5f2413b550 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c009df0 tx=0x7f5f1c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4c5fc7b7170e468 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:32.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.128+0000 7f5f21ffb700 1 -- 172.21.15.16:0/1712235745 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c00e070 con 0x7f5f24139170 2023-09-30T22:29:32.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.128+0000 7f5f21ffb700 1 -- 172.21.15.16:0/1712235745 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f1c00bae0 con 0x7f5f24139170 2023-09-30T22:29:32.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.128+0000 7f5f21ffb700 1 -- 172.21.15.16:0/1712235745 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c00be90 con 0x7f5f24139170 2023-09-30T22:29:32.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.129+0000 7f5f298c7700 1 -- 172.21.15.16:0/1712235745 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5f24139170 msgr2=0x7f5f2413b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:32.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.129+0000 7f5f298c7700 1 --2- 172.21.15.16:0/1712235745 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5f24139170 0x7f5f2413b550 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c009df0 tx=0x7f5f1c00b330 comp rx=0 tx=0).stop 2023-09-30T22:29:32.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.129+0000 7f5f298c7700 1 -- 172.21.15.16:0/1712235745 shutdown_connections 2023-09-30T22:29:32.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.129+0000 7f5f298c7700 1 --2- 172.21.15.16:0/1712235745 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5f24139170 0x7f5f2413b550 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:32.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.129+0000 7f5f298c7700 1 --2- 172.21.15.16:0/1712235745 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5f2409e1d0 0x7f5f24138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:32.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.129+0000 7f5f298c7700 1 -- 172.21.15.16:0/1712235745 >> 172.21.15.16:0/1712235745 conn(0x7f5f24066f50 msgr2=0x7f5f2409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:32.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.129+0000 7f5f298c7700 1 -- 172.21.15.16:0/1712235745 shutdown_connections 2023-09-30T22:29:32.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.129+0000 7f5f298c7700 1 -- 172.21.15.16:0/1712235745 wait complete. 2023-09-30T22:29:32.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.130+0000 7f5f298c7700 1 Processor -- start 2023-09-30T22:29:32.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.131+0000 7f5f298c7700 1 -- start start 2023-09-30T22:29:32.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.131+0000 7f5f298c7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f2409d6f0 0x7f5f24119480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:32.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.131+0000 7f5f298c7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5f2409e1d0 0x7f5f241199c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:32.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.131+0000 7f5f298c7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5f24139170 0x7f5f24119f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:32.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.131+0000 7f5f298c7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2411b1a0 con 0x7f5f2409d6f0 2023-09-30T22:29:32.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.131+0000 7f5f298c7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2411c990 con 0x7f5f2409e1d0 2023-09-30T22:29:32.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.131+0000 7f5f298c7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5f240a1560 con 0x7f5f24139170 2023-09-30T22:29:32.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.131+0000 7f5f22ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f2409d6f0 0x7f5f24119480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:32.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.131+0000 7f5f237fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5f24139170 0x7f5f24119f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:32.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.131+0000 7f5f22ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f2409d6f0 0x7f5f24119480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37590/0 (socket says 172.21.15.16:37590) 2023-09-30T22:29:32.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.131+0000 7f5f22ffd700 1 -- 172.21.15.16:0/421128516 learned_addr learned my addr 172.21.15.16:0/421128516 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:32.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.131+0000 7f5f237fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5f24139170 0x7f5f24119f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:36164/0 (socket says 172.21.15.16:36164) 2023-09-30T22:29:32.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.132+0000 7f5f227fc700 1 --2- 172.21.15.16:0/421128516 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5f2409e1d0 0x7f5f241199c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:32.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.132+0000 7f5f237fe700 1 -- 172.21.15.16:0/421128516 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5f2409e1d0 msgr2=0x7f5f241199c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:32.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.132+0000 7f5f237fe700 1 --2- 172.21.15.16:0/421128516 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5f2409e1d0 0x7f5f241199c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:32.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.132+0000 7f5f237fe700 1 -- 172.21.15.16:0/421128516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f2409d6f0 msgr2=0x7f5f24119480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:32.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.132+0000 7f5f237fe700 1 --2- 172.21.15.16:0/421128516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f2409d6f0 0x7f5f24119480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:32.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.132+0000 7f5f237fe700 1 -- 172.21.15.16:0/421128516 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f1c009a70 con 0x7f5f24139170 2023-09-30T22:29:32.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.132+0000 7f5f22ffd700 1 --2- 172.21.15.16:0/421128516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f2409d6f0 0x7f5f24119480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:32.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.132+0000 7f5f237fe700 1 --2- 172.21.15.16:0/421128516 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5f24139170 0x7f5f24119f00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c016000 tx=0x7f5f1c004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:32.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.132+0000 7f5f0bfff700 1 -- 172.21.15.16:0/421128516 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c004550 con 0x7f5f24139170 2023-09-30T22:29:32.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.132+0000 7f5f0bfff700 1 -- 172.21.15.16:0/421128516 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f1c020070 con 0x7f5f24139170 2023-09-30T22:29:32.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.132+0000 7f5f0bfff700 1 -- 172.21.15.16:0/421128516 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1c004910 con 0x7f5f24139170 2023-09-30T22:29:32.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.132+0000 7f5f298c7700 1 -- 172.21.15.16:0/421128516 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f2409c710 con 0x7f5f24139170 2023-09-30T22:29:32.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.133+0000 7f5f298c7700 1 -- 172.21.15.16:0/421128516 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f241cc410 con 0x7f5f24139170 2023-09-30T22:29:32.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.133+0000 7f5f0bfff700 1 -- 172.21.15.16:0/421128516 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f5f1c012430 con 0x7f5f24139170 2023-09-30T22:29:32.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.134+0000 7f5f298c7700 1 -- 172.21.15.16:0/421128516 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f24074ea0 con 0x7f5f24139170 2023-09-30T22:29:32.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.134+0000 7f5f0bfff700 1 -- 172.21.15.16:0/421128516 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 4049+0+0 (secure 0 0 0) 0x7f5f1c022070 con 0x7f5f24139170 2023-09-30T22:29:32.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.141+0000 7f5f0bfff700 1 -- 172.21.15.16:0/421128516 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5f1c02c020 con 0x7f5f24139170 2023-09-30T22:29:32.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.409+0000 7f5f298c7700 1 -- 172.21.15.16:0/421128516 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5f2406c360 con 0x7f5f24139170 2023-09-30T22:29:32.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.410+0000 7f5f0bfff700 1 -- 172.21.15.16:0/421128516 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v28) v1 ==== 74+0+12734 (secure 0 0 0) 0x7f5f1c0269d0 con 0x7f5f24139170 2023-09-30T22:29:32.413 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:29:32.413 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":28,"fsid":"7ede3e14-2bd0-4726-9437-6d13c90cd342","created":"2023-09-30T22:26:35.483697+0000","modified":"2023-09-30T22:29:31.534147+0000","last_up_change":"2023-09-30T22:29:31.534147+0000","last_in_change":"2023-09-30T22:26:37.120906+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"luminous","require_osd_release":"pacific","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T22:26:44.285133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{},"mgr_devicehealth":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.25,"raw_score_acting":1,"raw_score_stable":1,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:26:45.132921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"error_message":"pool 2 has some PGs where all OSDs are with primary_affinity 0 (at least pg 2.0), read balance score may not be reliable","score_acting":0,"score_stable":0}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:26:49.353683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"error_message":"pool 3 has some PGs where all OSDs are with primary_affinity 0 (at least pg 3.0), read balance score may not be reliable","score_acting":0,"score_stable":0}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:26:50.655685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"error_message":"pool 4 has some PGs where all OSDs are with primary_affinity 0 (at least pg 4.0), read balance score may not be reliable","score_acting":0,"score_stable":0}}],"osds":[{"osd":0,"uuid":"19946985-0ee2-487f-9347-929670dbe5ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":22,"up_from":28,"up_thru":18,"down_at":23,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6804","nonce":129539}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6805","nonce":129539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6807","nonce":129539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6806","nonce":129539}]},"public_addr":"172.21.15.16:6804/129539","cluster_addr":"172.21.15.16:6805/129539","heartbeat_back_addr":"172.21.15.16:6807/129539","heartbeat_front_addr":"172.21.15.16:6806/129539","state":["exists","up"]},{"osd":1,"uuid":"8efd4dbc-6ae4-4b82-b530-3b3eac2be445","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":24,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6812","nonce":118832}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6813","nonce":118832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6815","nonce":118832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6814","nonce":118832}]},"public_addr":"172.21.15.16:6812/118832","cluster_addr":"172.21.15.16:6813/118832","heartbeat_back_addr":"172.21.15.16:6815/118832","heartbeat_front_addr":"172.21.15.16:6814/118832","state":["exists"]},{"osd":2,"uuid":"b47bb1ce-27af-4c0c-868a-b76ebd0d9499","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":24,"down_at":25,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6808","nonce":118840}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6809","nonce":118840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6811","nonce":118840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6810","nonce":118840}]},"public_addr":"172.21.15.16:6808/118840","cluster_addr":"172.21.15.16:6809/118840","heartbeat_back_addr":"172.21.15.16:6811/118840","heartbeat_front_addr":"172.21.15.16:6810/118840","state":["exists"]},{"osd":3,"uuid":"03b53d64-d10d-448a-a081-39ecceab589b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":25,"down_at":26,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6800","nonce":118823}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6801","nonce":118823}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6803","nonce":118823}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6802","nonce":118823}]},"public_addr":"172.21.15.16:6800/118823","cluster_addr":"172.21.15.16:6801/118823","heartbeat_back_addr":"172.21.15.16:6803/118823","heartbeat_front_addr":"172.21.15.16:6802/118823","state":["exists"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.305953+0000","dead_epoch":23},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.307488+0000","dead_epoch":24},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.341023+0000","dead_epoch":25},{"osd":3,"down_stamp":"2023-09-30T22:29:28.519844+0000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.192071+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.8","mappings":[{"from":3,"to":2}]},{"pgid":"4.d","mappings":[{"from":3,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":0}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]},{"pgid":"4.17","mappings":[{"from":3,"to":2}]},{"pgid":"4.19","mappings":[{"from":3,"to":0}]},{"pgid":"4.3a","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6816/118285":"2023-10-01T22:29:14.475978+0000","172.21.15.16:0/2249754484":"2023-10-01T22:29:14.475978+0000","172.21.15.16:0/1101648925":"2023-10-01T22:29:14.475978+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:29:32.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.413+0000 7f5f298c7700 1 -- 172.21.15.16:0/421128516 >> v1:172.21.15.16:6816/126811 conn(0x7f5f1002f510 legacy=0x7f5f100319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:32.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.413+0000 7f5f298c7700 1 -- 172.21.15.16:0/421128516 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5f24139170 msgr2=0x7f5f24119f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:32.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.413+0000 7f5f298c7700 1 --2- 172.21.15.16:0/421128516 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5f24139170 0x7f5f24119f00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c016000 tx=0x7f5f1c004ad0 comp rx=0 tx=0).stop 2023-09-30T22:29:32.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.413+0000 7f5f298c7700 1 -- 172.21.15.16:0/421128516 shutdown_connections 2023-09-30T22:29:32.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.413+0000 7f5f298c7700 1 --2- 172.21.15.16:0/421128516 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5f24139170 0x7f5f24119f00 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:32.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.413+0000 7f5f298c7700 1 --2- 172.21.15.16:0/421128516 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f2409d6f0 0x7f5f24119480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:32.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.413+0000 7f5f298c7700 1 --2- 172.21.15.16:0/421128516 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5f2409e1d0 0x7f5f241199c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:32.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.413+0000 7f5f298c7700 1 -- 172.21.15.16:0/421128516 >> 172.21.15.16:0/421128516 conn(0x7f5f24066f50 msgr2=0x7f5f24136a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:32.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.413+0000 7f5f298c7700 1 -- 172.21.15.16:0/421128516 shutdown_connections 2023-09-30T22:29:32.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:32.413+0000 7f5f298c7700 1 -- 172.21.15.16:0/421128516 wait complete. 2023-09-30T22:29:32.452 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-09-30T22:29:32.452+0000 7f78dc403700 -1 osd.1 22 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-30T22:29:32.954 INFO:tasks.ceph.osd.3.smithi016.stderr:2023-09-30T22:29:32.953+0000 7f6f27514680 -1 osd.3 25 log_to_monitors true 2023-09-30T22:29:33.695 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-09-30T22:29:33.695+0000 7faf5f6f4700 -1 osd.2 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-30T22:29:34.769 INFO:tasks.ceph.osd.3.smithi016.stderr:2023-09-30T22:29:34.768+0000 7f6f1348f700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-09-30T22:29:35.433 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:29:35.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.580+0000 7f5e4da85700 1 Processor -- start 2023-09-30T22:29:35.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.580+0000 7f5e4da85700 1 -- start start 2023-09-30T22:29:35.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.580+0000 7f5e4da85700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5e48133a80 0x7f5e48133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:35.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.580+0000 7f5e4da85700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5e4812ff20 0x7f5e48130330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:35.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.580+0000 7f5e4da85700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5e48130870 0x7f5e4813a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:35.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.580+0000 7f5e4da85700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e4806c570 con 0x7f5e48130870 2023-09-30T22:29:35.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.580+0000 7f5e4da85700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4806bc00 con 0x7f5e4812ff20 2023-09-30T22:29:35.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.580+0000 7f5e4da85700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5e480a0040 con 0x7f5e48133a80 2023-09-30T22:29:35.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.580+0000 7f5e477fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5e48133a80 0x7f5e48133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:35.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.580+0000 7f5e47fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5e48130870 0x7f5e4813a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:35.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.581+0000 7f5e477fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5e48133a80 0x7f5e48133e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:36166/0 (socket says 172.21.15.16:36166) 2023-09-30T22:29:35.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.581+0000 7f5e477fe700 1 -- 172.21.15.16:0/2319480199 learned_addr learned my addr 172.21.15.16:0/2319480199 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:35.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.581+0000 7f5e46ffd700 1 --2- 172.21.15.16:0/2319480199 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5e4812ff20 0x7f5e48130330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:35.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.581+0000 7f5e477fe700 1 -- 172.21.15.16:0/2319480199 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5e4812ff20 msgr2=0x7f5e48130330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:35.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.581+0000 7f5e477fe700 1 --2- 172.21.15.16:0/2319480199 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5e4812ff20 0x7f5e48130330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:35.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.581+0000 7f5e477fe700 1 -- 172.21.15.16:0/2319480199 >> v2:172.21.15.16:3300/0 conn(0x7f5e48130870 msgr2=0x7f5e4813a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:35.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.581+0000 7f5e477fe700 1 --2- 172.21.15.16:0/2319480199 >> v2:172.21.15.16:3300/0 conn(0x7f5e48130870 0x7f5e4813a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:35.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.581+0000 7f5e477fe700 1 -- 172.21.15.16:0/2319480199 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e48133860 con 0x7f5e48133a80 2023-09-30T22:29:35.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.581+0000 7f5e47fff700 1 --2- 172.21.15.16:0/2319480199 >> v2:172.21.15.16:3300/0 conn(0x7f5e48130870 0x7f5e4813a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:35.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.581+0000 7f5e46ffd700 1 --2- 172.21.15.16:0/2319480199 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5e4812ff20 0x7f5e48130330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:35.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.581+0000 7f5e477fe700 1 --2- 172.21.15.16:0/2319480199 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5e48133a80 0x7f5e48133e50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5e380098f0 tx=0x7f5e38009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a6c2cf35e47f26b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:35.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.581+0000 7f5e467fc700 1 -- 172.21.15.16:0/2319480199 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e38011830 con 0x7f5e48133a80 2023-09-30T22:29:35.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.581+0000 7f5e467fc700 1 -- 172.21.15.16:0/2319480199 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e38011990 con 0x7f5e48133a80 2023-09-30T22:29:35.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.581+0000 7f5e467fc700 1 -- 172.21.15.16:0/2319480199 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e38011d20 con 0x7f5e48133a80 2023-09-30T22:29:35.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.582+0000 7f5e4da85700 1 -- 172.21.15.16:0/2319480199 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5e48133a80 msgr2=0x7f5e48133e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:35.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.582+0000 7f5e4da85700 1 --2- 172.21.15.16:0/2319480199 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5e48133a80 0x7f5e48133e50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5e380098f0 tx=0x7f5e38009c00 comp rx=0 tx=0).stop 2023-09-30T22:29:35.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.582+0000 7f5e4da85700 1 -- 172.21.15.16:0/2319480199 shutdown_connections 2023-09-30T22:29:35.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.582+0000 7f5e4da85700 1 --2- 172.21.15.16:0/2319480199 >> v2:172.21.15.16:3300/0 conn(0x7f5e48130870 0x7f5e4813a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:35.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.582+0000 7f5e4da85700 1 --2- 172.21.15.16:0/2319480199 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5e48133a80 0x7f5e48133e50 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:35.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.582+0000 7f5e4da85700 1 --2- 172.21.15.16:0/2319480199 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5e4812ff20 0x7f5e48130330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:35.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.582+0000 7f5e4da85700 1 -- 172.21.15.16:0/2319480199 >> 172.21.15.16:0/2319480199 conn(0x7f5e480a4210 msgr2=0x7f5e480a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:35.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.582+0000 7f5e4da85700 1 -- 172.21.15.16:0/2319480199 shutdown_connections 2023-09-30T22:29:35.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.582+0000 7f5e4da85700 1 -- 172.21.15.16:0/2319480199 wait complete. 2023-09-30T22:29:35.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.583+0000 7f5e4da85700 1 Processor -- start 2023-09-30T22:29:35.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.583+0000 7f5e4da85700 1 -- start start 2023-09-30T22:29:35.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.584+0000 7f5e4da85700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5e4812ff20 0x7f5e4809c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:35.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.584+0000 7f5e4da85700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e48130870 0x7f5e4809c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:35.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.584+0000 7f5e4da85700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5e48133a80 0x7f5e4809cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:35.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.584+0000 7f5e4da85700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4811fa90 con 0x7f5e48130870 2023-09-30T22:29:35.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.584+0000 7f5e4da85700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4809df50 con 0x7f5e4812ff20 2023-09-30T22:29:35.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.584+0000 7f5e4da85700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4811fdf0 con 0x7f5e48133a80 2023-09-30T22:29:35.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.584+0000 7f5e46ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e48130870 0x7f5e4809c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:35.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.584+0000 7f5e477fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5e4812ff20 0x7f5e4809c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:35.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.584+0000 7f5e46ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e48130870 0x7f5e4809c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37602/0 (socket says 172.21.15.16:37602) 2023-09-30T22:29:35.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.584+0000 7f5e47fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5e48133a80 0x7f5e4809cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:35.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.584+0000 7f5e46ffd700 1 -- 172.21.15.16:0/2413960284 learned_addr learned my addr 172.21.15.16:0/2413960284 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:35.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.584+0000 7f5e46ffd700 1 -- 172.21.15.16:0/2413960284 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5e48133a80 msgr2=0x7f5e4809cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:35.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.584+0000 7f5e46ffd700 1 --2- 172.21.15.16:0/2413960284 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5e48133a80 0x7f5e4809cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:35.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.584+0000 7f5e46ffd700 1 -- 172.21.15.16:0/2413960284 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5e4812ff20 msgr2=0x7f5e4809c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:35.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.584+0000 7f5e46ffd700 1 --2- 172.21.15.16:0/2413960284 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5e4812ff20 0x7f5e4809c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:35.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.584+0000 7f5e46ffd700 1 -- 172.21.15.16:0/2413960284 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e3c0094a0 con 0x7f5e48130870 2023-09-30T22:29:35.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.584+0000 7f5e477fe700 1 --2- 172.21.15.16:0/2413960284 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5e4812ff20 0x7f5e4809c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:35.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.584+0000 7f5e47fff700 1 --2- 172.21.15.16:0/2413960284 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5e48133a80 0x7f5e4809cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:35.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.584+0000 7f5e46ffd700 1 --2- 172.21.15.16:0/2413960284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e48130870 0x7f5e4809c9c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c00ebb0 tx=0x7f5e3c00eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:35.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.584+0000 7f5e2ffff700 1 -- 172.21.15.16:0/2413960284 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c00c850 con 0x7f5e48130870 2023-09-30T22:29:35.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.584+0000 7f5e2ffff700 1 -- 172.21.15.16:0/2413960284 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e3c00c9b0 con 0x7f5e48130870 2023-09-30T22:29:35.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.584+0000 7f5e2ffff700 1 -- 172.21.15.16:0/2413960284 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3c00cd40 con 0x7f5e48130870 2023-09-30T22:29:35.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.584+0000 7f5e4da85700 1 -- 172.21.15.16:0/2413960284 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e40009a70 con 0x7f5e48130870 2023-09-30T22:29:35.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.585+0000 7f5e4da85700 1 -- 172.21.15.16:0/2413960284 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e38009550 con 0x7f5e48130870 2023-09-30T22:29:35.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.585+0000 7f5e2ffff700 1 -- 172.21.15.16:0/2413960284 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f5e3c013470 con 0x7f5e48130870 2023-09-30T22:29:35.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.585+0000 7f5e2ffff700 1 -- 172.21.15.16:0/2413960284 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4049+0+0 (secure 0 0 0) 0x7f5e3c03fc80 con 0x7f5e48130870 2023-09-30T22:29:35.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.586+0000 7f5e4da85700 1 -- 172.21.15.16:0/2413960284 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e4804ea50 con 0x7f5e48130870 2023-09-30T22:29:35.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.591+0000 7f5e2ffff700 1 -- 172.21.15.16:0/2413960284 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5e3c0232f0 con 0x7f5e48130870 2023-09-30T22:29:35.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.825+0000 7f5e4da85700 1 -- 172.21.15.16:0/2413960284 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5e48074ea0 con 0x7f5e48130870 2023-09-30T22:29:35.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.827+0000 7f5e2ffff700 1 -- 172.21.15.16:0/2413960284 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+12952 (secure 0 0 0) 0x7f5e3c01bb50 con 0x7f5e48130870 2023-09-30T22:29:35.827 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:29:35.827 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":32,"fsid":"7ede3e14-2bd0-4726-9437-6d13c90cd342","created":"2023-09-30T22:26:35.483697+0000","modified":"2023-09-30T22:29:35.702639+0000","last_up_change":"2023-09-30T22:29:35.702639+0000","last_in_change":"2023-09-30T22:26:37.120906+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"luminous","require_osd_release":"pacific","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T22:26:44.285133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{},"mgr_devicehealth":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:26:45.132921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:26:49.353683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:26:50.655685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"19946985-0ee2-487f-9347-929670dbe5ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":22,"up_from":28,"up_thru":31,"down_at":23,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6804","nonce":129539}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6805","nonce":129539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6807","nonce":129539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6806","nonce":129539}]},"public_addr":"172.21.15.16:6804/129539","cluster_addr":"172.21.15.16:6805/129539","heartbeat_back_addr":"172.21.15.16:6807/129539","heartbeat_front_addr":"172.21.15.16:6806/129539","state":["exists","up"]},{"osd":1,"uuid":"8efd4dbc-6ae4-4b82-b530-3b3eac2be445","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":22,"up_from":29,"up_thru":31,"down_at":24,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6808","nonce":129637}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6809","nonce":129637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6811","nonce":129637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6810","nonce":129637}]},"public_addr":"172.21.15.16:6808/129637","cluster_addr":"172.21.15.16:6809/129637","heartbeat_back_addr":"172.21.15.16:6811/129637","heartbeat_front_addr":"172.21.15.16:6810/129637","state":["exists","up"]},{"osd":2,"uuid":"b47bb1ce-27af-4c0c-868a-b76ebd0d9499","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":24,"up_from":31,"up_thru":31,"down_at":25,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6800","nonce":129747}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6801","nonce":129747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6803","nonce":129747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6802","nonce":129747}]},"public_addr":"172.21.15.16:6800/129747","cluster_addr":"172.21.15.16:6801/129747","heartbeat_back_addr":"172.21.15.16:6803/129747","heartbeat_front_addr":"172.21.15.16:6802/129747","state":["exists","up"]},{"osd":3,"uuid":"03b53d64-d10d-448a-a081-39ecceab589b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":25,"up_from":32,"up_thru":25,"down_at":26,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6812","nonce":130087}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6813","nonce":130087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6815","nonce":130087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6814","nonce":130087}]},"public_addr":"172.21.15.16:6812/130087","cluster_addr":"172.21.15.16:6813/130087","heartbeat_back_addr":"172.21.15.16:6815/130087","heartbeat_front_addr":"172.21.15.16:6814/130087","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.305953+0000","dead_epoch":23},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.307488+0000","dead_epoch":24},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.341023+0000","dead_epoch":25},{"osd":3,"down_stamp":"2023-09-30T22:29:28.519844+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.192071+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.8","mappings":[{"from":3,"to":2}]},{"pgid":"4.d","mappings":[{"from":3,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":0}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]},{"pgid":"4.17","mappings":[{"from":3,"to":2}]},{"pgid":"4.19","mappings":[{"from":3,"to":0}]},{"pgid":"4.3a","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6816/118285":"2023-10-01T22:29:14.475978+0000","172.21.15.16:0/2249754484":"2023-10-01T22:29:14.475978+0000","172.21.15.16:0/1101648925":"2023-10-01T22:29:14.475978+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:29:35.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.829+0000 7f5e4da85700 1 -- 172.21.15.16:0/2413960284 >> v1:172.21.15.16:6816/126811 conn(0x7f5e3402f510 legacy=0x7f5e340319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:35.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.829+0000 7f5e4da85700 1 -- 172.21.15.16:0/2413960284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e48130870 msgr2=0x7f5e4809c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:35.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.829+0000 7f5e4da85700 1 --2- 172.21.15.16:0/2413960284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e48130870 0x7f5e4809c9c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c00ebb0 tx=0x7f5e3c00eec0 comp rx=0 tx=0).stop 2023-09-30T22:29:35.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.829+0000 7f5e4da85700 1 -- 172.21.15.16:0/2413960284 shutdown_connections 2023-09-30T22:29:35.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.829+0000 7f5e4da85700 1 --2- 172.21.15.16:0/2413960284 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5e4812ff20 0x7f5e4809c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:35.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.829+0000 7f5e4da85700 1 --2- 172.21.15.16:0/2413960284 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5e48133a80 0x7f5e4809cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:35.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.829+0000 7f5e4da85700 1 --2- 172.21.15.16:0/2413960284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e48130870 0x7f5e4809c9c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:35.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.829+0000 7f5e4da85700 1 -- 172.21.15.16:0/2413960284 >> 172.21.15.16:0/2413960284 conn(0x7f5e480a4210 msgr2=0x7f5e4809e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:35.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.829+0000 7f5e4da85700 1 -- 172.21.15.16:0/2413960284 shutdown_connections 2023-09-30T22:29:35.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:35.829+0000 7f5e4da85700 1 -- 172.21.15.16:0/2413960284 wait complete. 2023-09-30T22:29:35.843 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-09-30T22:29:35.843 INFO:teuthology.run_tasks:Running task ceph.stop... 2023-09-30T22:29:35.854 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-09-30T22:29:35.854 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T22:29:35.865 INFO:tasks.ceph.mds.a:Stopped 2023-09-30T22:29:40.134 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog finished 2023-09-30T22:29:40.134 INFO:teuthology.run_tasks:Running task ceph.restart... 2023-09-30T22:29:40.145 ERROR:tasks.ceph.mds.a:tried to stop a non-running daemon 2023-09-30T22:29:40.145 INFO:tasks.ceph.mds.a:Restarting daemon 2023-09-30T22:29:40.145 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-09-30T22:29:40.148 INFO:tasks.ceph.mds.a:Started 2023-09-30T22:29:40.148 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-09-30T22:29:40.148 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:29:40.242 INFO:tasks.ceph.mds.a.smithi016.stdout:starting mds.a at 2023-09-30T22:29:40.259 INFO:tasks.ceph.mon.a.smithi016.stderr:2023-09-30T22:29:40.259+0000 7f081fac5700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-09-30T22:29:40.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.301+0000 7f9e82a9a700 1 Processor -- start 2023-09-30T22:29:40.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.301+0000 7f9e82a9a700 1 -- start start 2023-09-30T22:29:40.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.301+0000 7f9e82a9a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e7c09d6f0 0x7f9e7c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:40.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.302+0000 7f9e82a9a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e7c09e410 0x7f9e7c134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:40.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.302+0000 7f9e82a9a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e7c09b120 con 0x7f9e7c09e000 2023-09-30T22:29:40.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.302+0000 7f9e82a9a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7c06b460 con 0x7f9e7c09d6f0 2023-09-30T22:29:40.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.302+0000 7f9e82a9a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7c09a7e0 con 0x7f9e7c09e410 2023-09-30T22:29:40.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.302+0000 7f9e80836700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e7c09d6f0 0x7f9e7c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:40.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.302+0000 7f9e7bfff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9e7c09e000 0x7f9e7c1339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52834/0 (socket says 172.21.15.16:52834) 2023-09-30T22:29:40.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.302+0000 7f9e7bfff700 1 -- 172.21.15.16:0/2367188936 learned_addr learned my addr 172.21.15.16:0/2367188936 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:29:40.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.302+0000 7f9e81037700 1 --2- 172.21.15.16:0/2367188936 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e7c09e410 0x7f9e7c134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:40.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.302+0000 7f9e81037700 1 -- 172.21.15.16:0/2367188936 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e7c09d6f0 msgr2=0x7f9e7c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:40.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.302+0000 7f9e81037700 1 --2- 172.21.15.16:0/2367188936 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e7c09d6f0 0x7f9e7c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:40.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.302+0000 7f9e81037700 1 -- 172.21.15.16:0/2367188936 >> v1:172.21.15.16:6789/0 conn(0x7f9e7c09e000 legacy=0x7f9e7c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:40.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.302+0000 7f9e7b7fe700 1 -- 172.21.15.16:0/2367188936 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1925642307 0 0) 0x7f9e7c09b120 con 0x7f9e7c09e000 2023-09-30T22:29:40.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.302+0000 7f9e81037700 1 -- 172.21.15.16:0/2367188936 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e7c09a500 con 0x7f9e7c09e410 2023-09-30T22:29:40.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.302+0000 7f9e80836700 1 --2- 172.21.15.16:0/2367188936 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e7c09d6f0 0x7f9e7c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:40.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.302+0000 7f9e81037700 1 --2- 172.21.15.16:0/2367188936 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e7c09e410 0x7f9e7c134100 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9e74009df0 tx=0x7f9e7400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22deb7c70c6c9b97 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:40.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.302+0000 7f9e7b7fe700 1 -- 172.21.15.16:0/2367188936 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e7400e070 con 0x7f9e7c09e410 2023-09-30T22:29:40.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.303+0000 7f9e7b7fe700 1 -- 172.21.15.16:0/2367188936 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e7400bae0 con 0x7f9e7c09e410 2023-09-30T22:29:40.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.303+0000 7f9e7b7fe700 1 -- 172.21.15.16:0/2367188936 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e7400be90 con 0x7f9e7c09e410 2023-09-30T22:29:40.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.303+0000 7f9e82a9a700 1 -- 172.21.15.16:0/2367188936 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e7c09e410 msgr2=0x7f9e7c134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:40.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.303+0000 7f9e82a9a700 1 --2- 172.21.15.16:0/2367188936 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e7c09e410 0x7f9e7c134100 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9e74009df0 tx=0x7f9e7400b330 comp rx=0 tx=0).stop 2023-09-30T22:29:40.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.303+0000 7f9e82a9a700 1 -- 172.21.15.16:0/2367188936 shutdown_connections 2023-09-30T22:29:40.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.303+0000 7f9e82a9a700 1 --2- 172.21.15.16:0/2367188936 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e7c09e410 0x7f9e7c134100 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:40.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.303+0000 7f9e82a9a700 1 --2- 172.21.15.16:0/2367188936 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e7c09d6f0 0x7f9e7c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:40.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.303+0000 7f9e82a9a700 1 -- 172.21.15.16:0/2367188936 >> 172.21.15.16:0/2367188936 conn(0x7f9e7c066f50 msgr2=0x7f9e7c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:40.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.303+0000 7f9e82a9a700 1 -- 172.21.15.16:0/2367188936 shutdown_connections 2023-09-30T22:29:40.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.303+0000 7f9e82a9a700 1 -- 172.21.15.16:0/2367188936 wait complete. 2023-09-30T22:29:40.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.304+0000 7f9e82a9a700 1 Processor -- start 2023-09-30T22:29:40.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.304+0000 7f9e82a9a700 1 -- start start 2023-09-30T22:29:40.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.305+0000 7f9e82a9a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e7c09d6f0 0x7f9e7c122320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:40.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.305+0000 7f9e82a9a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e7c09e000 0x7f9e7c122860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:40.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.305+0000 7f9e82a9a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e7c09e410 0x7f9e7c122da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:40.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.305+0000 7f9e82a9a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7c1237f0 con 0x7f9e7c09e410 2023-09-30T22:29:40.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.305+0000 7f9e82a9a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7c124e00 con 0x7f9e7c09d6f0 2023-09-30T22:29:40.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.305+0000 7f9e82a9a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7c125e40 con 0x7f9e7c09e000 2023-09-30T22:29:40.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.305+0000 7f9e81037700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e7c09e410 0x7f9e7c122da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:40.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.305+0000 7f9e80836700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e7c09d6f0 0x7f9e7c122320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:40.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.305+0000 7f9e81037700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e7c09e410 0x7f9e7c122da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47058/0 (socket says 172.21.15.16:47058) 2023-09-30T22:29:40.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.305+0000 7f9e81037700 1 -- 172.21.15.16:0/368480805 learned_addr learned my addr 172.21.15.16:0/368480805 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:40.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.305+0000 7f9e7bfff700 1 --2- 172.21.15.16:0/368480805 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e7c09e000 0x7f9e7c122860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:40.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.305+0000 7f9e80836700 1 -- 172.21.15.16:0/368480805 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e7c09e000 msgr2=0x7f9e7c122860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:40.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.305+0000 7f9e80836700 1 --2- 172.21.15.16:0/368480805 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e7c09e000 0x7f9e7c122860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:40.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.305+0000 7f9e80836700 1 -- 172.21.15.16:0/368480805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e7c09e410 msgr2=0x7f9e7c122da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:40.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.305+0000 7f9e80836700 1 --2- 172.21.15.16:0/368480805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e7c09e410 0x7f9e7c122da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:40.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.305+0000 7f9e80836700 1 -- 172.21.15.16:0/368480805 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e700094a0 con 0x7f9e7c09d6f0 2023-09-30T22:29:40.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.305+0000 7f9e81037700 1 --2- 172.21.15.16:0/368480805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e7c09e410 0x7f9e7c122da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:40.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.305+0000 7f9e7bfff700 1 --2- 172.21.15.16:0/368480805 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e7c09e000 0x7f9e7c122860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:40.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.306+0000 7f9e80836700 1 --2- 172.21.15.16:0/368480805 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e7c09d6f0 0x7f9e7c122320 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9e7000e9e0 tx=0x7f9e7000ecf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:40.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.306+0000 7f9e797fa700 1 -- 172.21.15.16:0/368480805 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e7000ee80 con 0x7f9e7c09d6f0 2023-09-30T22:29:40.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.306+0000 7f9e797fa700 1 -- 172.21.15.16:0/368480805 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e7000c850 con 0x7f9e7c09d6f0 2023-09-30T22:29:40.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.306+0000 7f9e82a9a700 1 -- 172.21.15.16:0/368480805 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e74009a70 con 0x7f9e7c09d6f0 2023-09-30T22:29:40.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.306+0000 7f9e797fa700 1 -- 172.21.15.16:0/368480805 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e7000cc00 con 0x7f9e7c09d6f0 2023-09-30T22:29:40.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.306+0000 7f9e82a9a700 1 -- 172.21.15.16:0/368480805 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e7c0a2020 con 0x7f9e7c09d6f0 2023-09-30T22:29:40.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.306+0000 7f9e797fa700 1 -- 172.21.15.16:0/368480805 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f9e7000cd60 con 0x7f9e7c09d6f0 2023-09-30T22:29:40.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.307+0000 7f9e797fa700 1 -- 172.21.15.16:0/368480805 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4092+0+0 (secure 0 0 0) 0x7f9e7003fc80 con 0x7f9e7c09d6f0 2023-09-30T22:29:40.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.307+0000 7f9e82a9a700 1 -- 172.21.15.16:0/368480805 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e7c11f030 con 0x7f9e7c09d6f0 2023-09-30T22:29:40.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.312+0000 7f9e797fa700 1 -- 172.21.15.16:0/368480805 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9e7001be10 con 0x7f9e7c09d6f0 2023-09-30T22:29:40.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.517+0000 7f9e82a9a700 1 -- 172.21.15.16:0/368480805 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9e7c074ea0 con 0x7f9e7c09d6f0 2023-09-30T22:29:40.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.519+0000 7f9e797fa700 1 -- 172.21.15.16:0/368480805 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+13017 (secure 0 0 0) 0x7f9e7001b3b0 con 0x7f9e7c09d6f0 2023-09-30T22:29:40.519 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:29:40.519 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":34,"fsid":"7ede3e14-2bd0-4726-9437-6d13c90cd342","created":"2023-09-30T22:26:35.483697+0000","modified":"2023-09-30T22:29:40.259258+0000","last_up_change":"2023-09-30T22:29:35.702639+0000","last_in_change":"2023-09-30T22:26:37.120906+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"luminous","require_osd_release":"pacific","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T22:26:44.285133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{},"mgr_devicehealth":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:26:45.132921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:26:49.353683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:26:50.655685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"19946985-0ee2-487f-9347-929670dbe5ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":22,"up_from":28,"up_thru":32,"down_at":23,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6804","nonce":129539}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6805","nonce":129539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6807","nonce":129539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6806","nonce":129539}]},"public_addr":"172.21.15.16:6804/129539","cluster_addr":"172.21.15.16:6805/129539","heartbeat_back_addr":"172.21.15.16:6807/129539","heartbeat_front_addr":"172.21.15.16:6806/129539","state":["exists","up"]},{"osd":1,"uuid":"8efd4dbc-6ae4-4b82-b530-3b3eac2be445","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":22,"up_from":29,"up_thru":32,"down_at":24,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6808","nonce":129637}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6809","nonce":129637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6811","nonce":129637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6810","nonce":129637}]},"public_addr":"172.21.15.16:6808/129637","cluster_addr":"172.21.15.16:6809/129637","heartbeat_back_addr":"172.21.15.16:6811/129637","heartbeat_front_addr":"172.21.15.16:6810/129637","state":["exists","up"]},{"osd":2,"uuid":"b47bb1ce-27af-4c0c-868a-b76ebd0d9499","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":24,"up_from":31,"up_thru":32,"down_at":25,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6800","nonce":129747}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6801","nonce":129747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6803","nonce":129747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6802","nonce":129747}]},"public_addr":"172.21.15.16:6800/129747","cluster_addr":"172.21.15.16:6801/129747","heartbeat_back_addr":"172.21.15.16:6803/129747","heartbeat_front_addr":"172.21.15.16:6802/129747","state":["exists","up"]},{"osd":3,"uuid":"03b53d64-d10d-448a-a081-39ecceab589b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":25,"up_from":32,"up_thru":32,"down_at":26,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6812","nonce":130087}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6813","nonce":130087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6815","nonce":130087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6814","nonce":130087}]},"public_addr":"172.21.15.16:6812/130087","cluster_addr":"172.21.15.16:6813/130087","heartbeat_back_addr":"172.21.15.16:6815/130087","heartbeat_front_addr":"172.21.15.16:6814/130087","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.305953+0000","dead_epoch":23},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.307488+0000","dead_epoch":24},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.341023+0000","dead_epoch":25},{"osd":3,"down_stamp":"2023-09-30T22:29:28.519844+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.192071+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.8","mappings":[{"from":3,"to":2}]},{"pgid":"4.d","mappings":[{"from":3,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":0}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]},{"pgid":"4.17","mappings":[{"from":3,"to":2}]},{"pgid":"4.19","mappings":[{"from":3,"to":0}]},{"pgid":"4.3a","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6817/1424366610":"2023-10-01T22:29:40.259230+0000","172.21.15.16:6816/118285":"2023-10-01T22:29:14.475978+0000","172.21.15.16:0/2249754484":"2023-10-01T22:29:14.475978+0000","172.21.15.16:0/1101648925":"2023-10-01T22:29:14.475978+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:29:40.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.521+0000 7f9e82a9a700 1 -- 172.21.15.16:0/368480805 >> v1:172.21.15.16:6816/126811 conn(0x7f9e6002f510 legacy=0x7f9e600319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:40.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.521+0000 7f9e82a9a700 1 -- 172.21.15.16:0/368480805 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e7c09d6f0 msgr2=0x7f9e7c122320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:40.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.521+0000 7f9e82a9a700 1 --2- 172.21.15.16:0/368480805 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e7c09d6f0 0x7f9e7c122320 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9e7000e9e0 tx=0x7f9e7000ecf0 comp rx=0 tx=0).stop 2023-09-30T22:29:40.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.521+0000 7f9e82a9a700 1 -- 172.21.15.16:0/368480805 shutdown_connections 2023-09-30T22:29:40.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.521+0000 7f9e82a9a700 1 --2- 172.21.15.16:0/368480805 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e7c09d6f0 0x7f9e7c122320 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:40.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.521+0000 7f9e82a9a700 1 --2- 172.21.15.16:0/368480805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e7c09e410 0x7f9e7c122da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:40.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.521+0000 7f9e82a9a700 1 --2- 172.21.15.16:0/368480805 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e7c09e000 0x7f9e7c122860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:40.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.521+0000 7f9e82a9a700 1 -- 172.21.15.16:0/368480805 >> 172.21.15.16:0/368480805 conn(0x7f9e7c066f50 msgr2=0x7f9e7c133de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:40.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.521+0000 7f9e82a9a700 1 -- 172.21.15.16:0/368480805 shutdown_connections 2023-09-30T22:29:40.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.521+0000 7f9e82a9a700 1 -- 172.21.15.16:0/368480805 wait complete. 2023-09-30T22:29:40.535 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-09-30T22:29:40.535 INFO:teuthology.run_tasks:Running task exec... 2023-09-30T22:29:40.545 INFO:teuthology.task.exec:Executing custom commands... 2023-09-30T22:29:40.545 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi016.front.sepia.ceph.com 2023-09-30T22:29:40.545 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph osd dump -f json-pretty' 2023-09-30T22:29:40.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.679+0000 7fd2bf7f0700 1 Processor -- start 2023-09-30T22:29:40.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.679+0000 7fd2bf7f0700 1 -- start start 2023-09-30T22:29:40.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.680+0000 7fd2bf7f0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd2b8133b00 0x7fd2b8133ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:40.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.680+0000 7fd2bf7f0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd2b809e060 0x7fd2b813ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:40.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.680+0000 7fd2bf7f0700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2b809f640 con 0x7fd2b8134410 2023-09-30T22:29:40.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.680+0000 7fd2bf7f0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b809a790 con 0x7fd2b809e060 2023-09-30T22:29:40.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.680+0000 7fd2bf7f0700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b809a4f0 con 0x7fd2b8133b00 2023-09-30T22:29:40.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.680+0000 7fd2bd58c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd2b8133b00 0x7fd2b8133ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:40.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.680+0000 7fd2bcd8b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd2b8134410 0x7fd2b809d950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52844/0 (socket says 172.21.15.16:52844) 2023-09-30T22:29:40.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.680+0000 7fd2bcd8b700 1 -- 172.21.15.16:0/3153123432 learned_addr learned my addr 172.21.15.16:0/3153123432 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:29:40.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.680+0000 7fd2bdd8d700 1 --2- 172.21.15.16:0/3153123432 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd2b809e060 0x7fd2b813ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:40.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.680+0000 7fd2affff700 1 -- 172.21.15.16:0/3153123432 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2261738037 0 0) 0x7fd2b809f640 con 0x7fd2b8134410 2023-09-30T22:29:40.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.680+0000 7fd2bdd8d700 1 -- 172.21.15.16:0/3153123432 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd2b8133b00 msgr2=0x7fd2b8133ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:40.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.680+0000 7fd2bdd8d700 1 --2- 172.21.15.16:0/3153123432 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd2b8133b00 0x7fd2b8133ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:40.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.680+0000 7fd2bdd8d700 1 -- 172.21.15.16:0/3153123432 >> v1:172.21.15.16:6789/0 conn(0x7fd2b8134410 legacy=0x7fd2b809d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:40.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.680+0000 7fd2bdd8d700 1 -- 172.21.15.16:0/3153123432 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b809eb20 con 0x7fd2b809e060 2023-09-30T22:29:40.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.680+0000 7fd2bd58c700 1 --2- 172.21.15.16:0/3153123432 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd2b8133b00 0x7fd2b8133ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:40.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.680+0000 7fd2bdd8d700 1 --2- 172.21.15.16:0/3153123432 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd2b809e060 0x7fd2b813ba80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0009dc0 tx=0x7fd2b000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae396ccf3f4a20bc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:40.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.681+0000 7fd2affff700 1 -- 172.21.15.16:0/3153123432 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b000e070 con 0x7fd2b809e060 2023-09-30T22:29:40.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.681+0000 7fd2affff700 1 -- 172.21.15.16:0/3153123432 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2b000bb20 con 0x7fd2b809e060 2023-09-30T22:29:40.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.681+0000 7fd2affff700 1 -- 172.21.15.16:0/3153123432 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b0012430 con 0x7fd2b809e060 2023-09-30T22:29:40.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.681+0000 7fd2bf7f0700 1 -- 172.21.15.16:0/3153123432 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd2b809e060 msgr2=0x7fd2b813ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:40.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.681+0000 7fd2bf7f0700 1 --2- 172.21.15.16:0/3153123432 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd2b809e060 0x7fd2b813ba80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0009dc0 tx=0x7fd2b000b330 comp rx=0 tx=0).stop 2023-09-30T22:29:40.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.681+0000 7fd2bf7f0700 1 -- 172.21.15.16:0/3153123432 shutdown_connections 2023-09-30T22:29:40.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.681+0000 7fd2bf7f0700 1 --2- 172.21.15.16:0/3153123432 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd2b809e060 0x7fd2b813ba80 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:40.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.681+0000 7fd2bf7f0700 1 --2- 172.21.15.16:0/3153123432 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd2b8133b00 0x7fd2b8133ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:40.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.681+0000 7fd2bf7f0700 1 -- 172.21.15.16:0/3153123432 >> 172.21.15.16:0/3153123432 conn(0x7fd2b80a2f50 msgr2=0x7fd2b80a3350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:40.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.681+0000 7fd2bf7f0700 1 -- 172.21.15.16:0/3153123432 shutdown_connections 2023-09-30T22:29:40.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.681+0000 7fd2bf7f0700 1 -- 172.21.15.16:0/3153123432 wait complete. 2023-09-30T22:29:40.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.682+0000 7fd2bf7f0700 1 Processor -- start 2023-09-30T22:29:40.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.682+0000 7fd2bf7f0700 1 -- start start 2023-09-30T22:29:40.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.683+0000 7fd2bf7f0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2b809e060 0x7fd2b811e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:40.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.683+0000 7fd2bf7f0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd2b8133b00 0x7fd2b811e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:40.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.683+0000 7fd2bf7f0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd2b8134410 0x7fd2b811ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:40.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.683+0000 7fd2bf7f0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b811fcd0 con 0x7fd2b809e060 2023-09-30T22:29:40.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.683+0000 7fd2bf7f0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b809cc60 con 0x7fd2b8134410 2023-09-30T22:29:40.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.683+0000 7fd2bf7f0700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b8121900 con 0x7fd2b8133b00 2023-09-30T22:29:40.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.683+0000 7fd2bd58c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2b809e060 0x7fd2b811e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:40.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.683+0000 7fd2bcd8b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd2b8133b00 0x7fd2b811e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:40.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.683+0000 7fd2bd58c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2b809e060 0x7fd2b811e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47064/0 (socket says 172.21.15.16:47064) 2023-09-30T22:29:40.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.683+0000 7fd2bd58c700 1 -- 172.21.15.16:0/2675460334 learned_addr learned my addr 172.21.15.16:0/2675460334 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:40.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.683+0000 7fd2bdd8d700 1 --2- 172.21.15.16:0/2675460334 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd2b8134410 0x7fd2b811ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:40.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.683+0000 7fd2bcd8b700 1 -- 172.21.15.16:0/2675460334 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd2b8134410 msgr2=0x7fd2b811ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:40.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.683+0000 7fd2bcd8b700 1 --2- 172.21.15.16:0/2675460334 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd2b8134410 0x7fd2b811ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:40.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.683+0000 7fd2bcd8b700 1 -- 172.21.15.16:0/2675460334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2b809e060 msgr2=0x7fd2b811e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:40.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.683+0000 7fd2bcd8b700 1 --2- 172.21.15.16:0/2675460334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2b809e060 0x7fd2b811e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:40.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.683+0000 7fd2bcd8b700 1 -- 172.21.15.16:0/2675460334 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b40094a0 con 0x7fd2b8133b00 2023-09-30T22:29:40.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.683+0000 7fd2bd58c700 1 --2- 172.21.15.16:0/2675460334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2b809e060 0x7fd2b811e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:40.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.683+0000 7fd2bcd8b700 1 --2- 172.21.15.16:0/2675460334 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd2b8133b00 0x7fd2b811e6d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd2b806b720 tx=0x7fd2a800ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:40.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.684+0000 7fd2adffb700 1 -- 172.21.15.16:0/2675460334 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a80074e0 con 0x7fd2b8133b00 2023-09-30T22:29:40.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.684+0000 7fd2adffb700 1 -- 172.21.15.16:0/2675460334 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a8007660 con 0x7fd2b8133b00 2023-09-30T22:29:40.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.684+0000 7fd2bf7f0700 1 -- 172.21.15.16:0/2675460334 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2b0009a70 con 0x7fd2b8133b00 2023-09-30T22:29:40.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.684+0000 7fd2adffb700 1 -- 172.21.15.16:0/2675460334 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a8005a70 con 0x7fd2b8133b00 2023-09-30T22:29:40.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.684+0000 7fd2bf7f0700 1 -- 172.21.15.16:0/2675460334 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2b809f410 con 0x7fd2b8133b00 2023-09-30T22:29:40.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.685+0000 7fd2bf7f0700 1 -- 172.21.15.16:0/2675460334 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2b804f2a0 con 0x7fd2b8133b00 2023-09-30T22:29:40.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.685+0000 7fd2adffb700 1 -- 172.21.15.16:0/2675460334 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fd2a8005bd0 con 0x7fd2b8133b00 2023-09-30T22:29:40.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.685+0000 7fd2adffb700 1 -- 172.21.15.16:0/2675460334 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4092+0+0 (secure 0 0 0) 0x7fd2a803eef0 con 0x7fd2b8133b00 2023-09-30T22:29:40.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.690+0000 7fd2adffb700 1 -- 172.21.15.16:0/2675460334 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd2a8019d20 con 0x7fd2b8133b00 2023-09-30T22:29:40.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.895+0000 7fd2bf7f0700 1 -- 172.21.15.16:0/2675460334 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fd2b8120f20 con 0x7fd2b8133b00 2023-09-30T22:29:40.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.896+0000 7fd2adffb700 1 -- 172.21.15.16:0/2675460334 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v34) v1 ==== 81+0+24344 (secure 0 0 0) 0x7fd2a80192c0 con 0x7fd2b8133b00 2023-09-30T22:29:40.897 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:29:40.897 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-09-30T22:29:40.897 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 34, 2023-09-30T22:29:40.897 INFO:teuthology.orchestra.run.smithi016.stdout: "fsid": "7ede3e14-2bd0-4726-9437-6d13c90cd342", 2023-09-30T22:29:40.898 INFO:teuthology.orchestra.run.smithi016.stdout: "created": "2023-09-30T22:26:35.483697+0000", 2023-09-30T22:29:40.898 INFO:teuthology.orchestra.run.smithi016.stdout: "modified": "2023-09-30T22:29:40.259258+0000", 2023-09-30T22:29:40.898 INFO:teuthology.orchestra.run.smithi016.stdout: "last_up_change": "2023-09-30T22:29:35.702639+0000", 2023-09-30T22:29:40.898 INFO:teuthology.orchestra.run.smithi016.stdout: "last_in_change": "2023-09-30T22:26:37.120906+0000", 2023-09-30T22:29:40.898 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-09-30T22:29:40.898 INFO:teuthology.orchestra.run.smithi016.stdout: "flags_num": 5799936, 2023-09-30T22:29:40.898 INFO:teuthology.orchestra.run.smithi016.stdout: "flags_set": [ 2023-09-30T22:29:40.898 INFO:teuthology.orchestra.run.smithi016.stdout: "pglog_hardlimit", 2023-09-30T22:29:40.898 INFO:teuthology.orchestra.run.smithi016.stdout: "purged_snapdirs", 2023-09-30T22:29:40.898 INFO:teuthology.orchestra.run.smithi016.stdout: "recovery_deletes", 2023-09-30T22:29:40.898 INFO:teuthology.orchestra.run.smithi016.stdout: "sortbitwise" 2023-09-30T22:29:40.898 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-09-30T22:29:40.899 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_version": 8, 2023-09-30T22:29:40.899 INFO:teuthology.orchestra.run.smithi016.stdout: "full_ratio": 0.94999998807907104, 2023-09-30T22:29:40.899 INFO:teuthology.orchestra.run.smithi016.stdout: "backfillfull_ratio": 0.89999997615814209, 2023-09-30T22:29:40.899 INFO:teuthology.orchestra.run.smithi016.stdout: "nearfull_ratio": 0.85000002384185791, 2023-09-30T22:29:40.899 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_snapshot": "", 2023-09-30T22:29:40.899 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_max": 4, 2023-09-30T22:29:40.899 INFO:teuthology.orchestra.run.smithi016.stdout: "max_osd": 4, 2023-09-30T22:29:40.899 INFO:teuthology.orchestra.run.smithi016.stdout: "require_min_compat_client": "pacific", 2023-09-30T22:29:40.899 INFO:teuthology.orchestra.run.smithi016.stdout: "min_compat_client": "luminous", 2023-09-30T22:29:40.899 INFO:teuthology.orchestra.run.smithi016.stdout: "require_osd_release": "pacific", 2023-09-30T22:29:40.899 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_crimson": false, 2023-09-30T22:29:40.900 INFO:teuthology.orchestra.run.smithi016.stdout: "pools": [ 2023-09-30T22:29:40.900 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-09-30T22:29:40.900 INFO:teuthology.orchestra.run.smithi016.stdout: "pool": 1, 2023-09-30T22:29:40.900 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_name": ".mgr", 2023-09-30T22:29:40.900 INFO:teuthology.orchestra.run.smithi016.stdout: "create_time": "2023-09-30T22:26:44.285133+0000", 2023-09-30T22:29:40.900 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-09-30T22:29:40.900 INFO:teuthology.orchestra.run.smithi016.stdout: "flags_names": "hashpspool", 2023-09-30T22:29:40.900 INFO:teuthology.orchestra.run.smithi016.stdout: "type": 1, 2023-09-30T22:29:40.900 INFO:teuthology.orchestra.run.smithi016.stdout: "size": 2, 2023-09-30T22:29:40.900 INFO:teuthology.orchestra.run.smithi016.stdout: "min_size": 1, 2023-09-30T22:29:40.900 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_rule": 0, 2023-09-30T22:29:40.900 INFO:teuthology.orchestra.run.smithi016.stdout: "peering_crush_bucket_count": 0, 2023-09-30T22:29:40.901 INFO:teuthology.orchestra.run.smithi016.stdout: "peering_crush_bucket_target": 0, 2023-09-30T22:29:40.901 INFO:teuthology.orchestra.run.smithi016.stdout: "peering_crush_bucket_barrier": 0, 2023-09-30T22:29:40.901 INFO:teuthology.orchestra.run.smithi016.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-09-30T22:29:40.901 INFO:teuthology.orchestra.run.smithi016.stdout: "object_hash": 2, 2023-09-30T22:29:40.901 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscale_mode": "off", 2023-09-30T22:29:40.901 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_num": 1, 2023-09-30T22:29:40.901 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_placement_num": 1, 2023-09-30T22:29:40.901 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_placement_num_target": 1, 2023-09-30T22:29:40.901 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_num_target": 1, 2023-09-30T22:29:40.901 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_num_pending": 1, 2023-09-30T22:29:40.901 INFO:teuthology.orchestra.run.smithi016.stdout: "last_pg_merge_meta": { 2023-09-30T22:29:40.901 INFO:teuthology.orchestra.run.smithi016.stdout: "source_pgid": "0.0", 2023-09-30T22:29:40.902 INFO:teuthology.orchestra.run.smithi016.stdout: "ready_epoch": 0, 2023-09-30T22:29:40.902 INFO:teuthology.orchestra.run.smithi016.stdout: "last_epoch_started": 0, 2023-09-30T22:29:40.902 INFO:teuthology.orchestra.run.smithi016.stdout: "last_epoch_clean": 0, 2023-09-30T22:29:40.902 INFO:teuthology.orchestra.run.smithi016.stdout: "source_version": "0'0", 2023-09-30T22:29:40.902 INFO:teuthology.orchestra.run.smithi016.stdout: "target_version": "0'0" 2023-09-30T22:29:40.902 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-09-30T22:29:40.902 INFO:teuthology.orchestra.run.smithi016.stdout: "last_change": "22", 2023-09-30T22:29:40.902 INFO:teuthology.orchestra.run.smithi016.stdout: "last_force_op_resend": "0", 2023-09-30T22:29:40.902 INFO:teuthology.orchestra.run.smithi016.stdout: "last_force_op_resend_prenautilus": "0", 2023-09-30T22:29:40.902 INFO:teuthology.orchestra.run.smithi016.stdout: "last_force_op_resend_preluminous": "0", 2023-09-30T22:29:40.902 INFO:teuthology.orchestra.run.smithi016.stdout: "auid": 0, 2023-09-30T22:29:40.902 INFO:teuthology.orchestra.run.smithi016.stdout: "snap_mode": "selfmanaged", 2023-09-30T22:29:40.902 INFO:teuthology.orchestra.run.smithi016.stdout: "snap_seq": 0, 2023-09-30T22:29:40.903 INFO:teuthology.orchestra.run.smithi016.stdout: "snap_epoch": 0, 2023-09-30T22:29:40.903 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_snaps": [], 2023-09-30T22:29:40.903 INFO:teuthology.orchestra.run.smithi016.stdout: "removed_snaps": "[]", 2023-09-30T22:29:40.903 INFO:teuthology.orchestra.run.smithi016.stdout: "quota_max_bytes": 0, 2023-09-30T22:29:40.903 INFO:teuthology.orchestra.run.smithi016.stdout: "quota_max_objects": 0, 2023-09-30T22:29:40.903 INFO:teuthology.orchestra.run.smithi016.stdout: "tiers": [], 2023-09-30T22:29:40.903 INFO:teuthology.orchestra.run.smithi016.stdout: "tier_of": -1, 2023-09-30T22:29:40.903 INFO:teuthology.orchestra.run.smithi016.stdout: "read_tier": -1, 2023-09-30T22:29:40.903 INFO:teuthology.orchestra.run.smithi016.stdout: "write_tier": -1, 2023-09-30T22:29:40.903 INFO:teuthology.orchestra.run.smithi016.stdout: "cache_mode": "none", 2023-09-30T22:29:40.903 INFO:teuthology.orchestra.run.smithi016.stdout: "target_max_bytes": 0, 2023-09-30T22:29:40.904 INFO:teuthology.orchestra.run.smithi016.stdout: "target_max_objects": 0, 2023-09-30T22:29:40.904 INFO:teuthology.orchestra.run.smithi016.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-09-30T22:29:40.904 INFO:teuthology.orchestra.run.smithi016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-09-30T22:29:40.904 INFO:teuthology.orchestra.run.smithi016.stdout: "cache_target_full_ratio_micro": 800000, 2023-09-30T22:29:40.904 INFO:teuthology.orchestra.run.smithi016.stdout: "cache_min_flush_age": 0, 2023-09-30T22:29:40.904 INFO:teuthology.orchestra.run.smithi016.stdout: "cache_min_evict_age": 0, 2023-09-30T22:29:40.904 INFO:teuthology.orchestra.run.smithi016.stdout: "erasure_code_profile": "", 2023-09-30T22:29:40.904 INFO:teuthology.orchestra.run.smithi016.stdout: "hit_set_params": { 2023-09-30T22:29:40.904 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "none" 2023-09-30T22:29:40.904 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-09-30T22:29:40.904 INFO:teuthology.orchestra.run.smithi016.stdout: "hit_set_period": 0, 2023-09-30T22:29:40.904 INFO:teuthology.orchestra.run.smithi016.stdout: "hit_set_count": 0, 2023-09-30T22:29:40.904 INFO:teuthology.orchestra.run.smithi016.stdout: "use_gmt_hitset": true, 2023-09-30T22:29:40.905 INFO:teuthology.orchestra.run.smithi016.stdout: "min_read_recency_for_promote": 0, 2023-09-30T22:29:40.905 INFO:teuthology.orchestra.run.smithi016.stdout: "min_write_recency_for_promote": 0, 2023-09-30T22:29:40.905 INFO:teuthology.orchestra.run.smithi016.stdout: "hit_set_grade_decay_rate": 0, 2023-09-30T22:29:40.905 INFO:teuthology.orchestra.run.smithi016.stdout: "hit_set_search_last_n": 0, 2023-09-30T22:29:40.905 INFO:teuthology.orchestra.run.smithi016.stdout: "grade_table": [], 2023-09-30T22:29:40.905 INFO:teuthology.orchestra.run.smithi016.stdout: "stripe_width": 0, 2023-09-30T22:29:40.905 INFO:teuthology.orchestra.run.smithi016.stdout: "expected_num_objects": 0, 2023-09-30T22:29:40.905 INFO:teuthology.orchestra.run.smithi016.stdout: "fast_read": false, 2023-09-30T22:29:40.905 INFO:teuthology.orchestra.run.smithi016.stdout: "options": { 2023-09-30T22:29:40.905 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_num_max": 32, 2023-09-30T22:29:40.905 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_num_min": 1 2023-09-30T22:29:40.905 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-09-30T22:29:40.906 INFO:teuthology.orchestra.run.smithi016.stdout: "application_metadata": { 2023-09-30T22:29:40.906 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": {}, 2023-09-30T22:29:40.906 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr_devicehealth": {} 2023-09-30T22:29:40.906 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-09-30T22:29:40.906 INFO:teuthology.orchestra.run.smithi016.stdout: "read_balance": { 2023-09-30T22:29:40.906 INFO:teuthology.orchestra.run.smithi016.stdout: "score_acting": 4, 2023-09-30T22:29:40.906 INFO:teuthology.orchestra.run.smithi016.stdout: "score_stable": 4, 2023-09-30T22:29:40.906 INFO:teuthology.orchestra.run.smithi016.stdout: "optimal_score": 0.5, 2023-09-30T22:29:40.906 INFO:teuthology.orchestra.run.smithi016.stdout: "raw_score_acting": 2, 2023-09-30T22:29:40.906 INFO:teuthology.orchestra.run.smithi016.stdout: "raw_score_stable": 2, 2023-09-30T22:29:40.906 INFO:teuthology.orchestra.run.smithi016.stdout: "primary_affinity_weighted": 1, 2023-09-30T22:29:40.906 INFO:teuthology.orchestra.run.smithi016.stdout: "average_primary_affinity": 1, 2023-09-30T22:29:40.907 INFO:teuthology.orchestra.run.smithi016.stdout: "average_primary_affinity_weighted": 1 2023-09-30T22:29:40.907 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-09-30T22:29:40.907 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-09-30T22:29:40.907 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-09-30T22:29:40.907 INFO:teuthology.orchestra.run.smithi016.stdout: "pool": 2, 2023-09-30T22:29:40.907 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_name": "rbd", 2023-09-30T22:29:40.907 INFO:teuthology.orchestra.run.smithi016.stdout: "create_time": "2023-09-30T22:26:45.132921+0000", 2023-09-30T22:29:40.907 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-09-30T22:29:40.907 INFO:teuthology.orchestra.run.smithi016.stdout: "flags_names": "hashpspool", 2023-09-30T22:29:40.907 INFO:teuthology.orchestra.run.smithi016.stdout: "type": 1, 2023-09-30T22:29:40.907 INFO:teuthology.orchestra.run.smithi016.stdout: "size": 2, 2023-09-30T22:29:40.907 INFO:teuthology.orchestra.run.smithi016.stdout: "min_size": 1, 2023-09-30T22:29:40.907 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_rule": 0, 2023-09-30T22:29:40.908 INFO:teuthology.orchestra.run.smithi016.stdout: "peering_crush_bucket_count": 0, 2023-09-30T22:29:40.908 INFO:teuthology.orchestra.run.smithi016.stdout: "peering_crush_bucket_target": 0, 2023-09-30T22:29:40.908 INFO:teuthology.orchestra.run.smithi016.stdout: "peering_crush_bucket_barrier": 0, 2023-09-30T22:29:40.908 INFO:teuthology.orchestra.run.smithi016.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-09-30T22:29:40.908 INFO:teuthology.orchestra.run.smithi016.stdout: "object_hash": 2, 2023-09-30T22:29:40.908 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscale_mode": "off", 2023-09-30T22:29:40.908 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_num": 8, 2023-09-30T22:29:40.908 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_placement_num": 8, 2023-09-30T22:29:40.908 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_placement_num_target": 8, 2023-09-30T22:29:40.908 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_num_target": 8, 2023-09-30T22:29:40.909 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_num_pending": 8, 2023-09-30T22:29:40.909 INFO:teuthology.orchestra.run.smithi016.stdout: "last_pg_merge_meta": { 2023-09-30T22:29:40.909 INFO:teuthology.orchestra.run.smithi016.stdout: "source_pgid": "0.0", 2023-09-30T22:29:40.909 INFO:teuthology.orchestra.run.smithi016.stdout: "ready_epoch": 0, 2023-09-30T22:29:40.909 INFO:teuthology.orchestra.run.smithi016.stdout: "last_epoch_started": 0, 2023-09-30T22:29:40.909 INFO:teuthology.orchestra.run.smithi016.stdout: "last_epoch_clean": 0, 2023-09-30T22:29:40.909 INFO:teuthology.orchestra.run.smithi016.stdout: "source_version": "0'0", 2023-09-30T22:29:40.909 INFO:teuthology.orchestra.run.smithi016.stdout: "target_version": "0'0" 2023-09-30T22:29:40.909 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-09-30T22:29:40.909 INFO:teuthology.orchestra.run.smithi016.stdout: "last_change": "12", 2023-09-30T22:29:40.910 INFO:teuthology.orchestra.run.smithi016.stdout: "last_force_op_resend": "0", 2023-09-30T22:29:40.910 INFO:teuthology.orchestra.run.smithi016.stdout: "last_force_op_resend_prenautilus": "0", 2023-09-30T22:29:40.910 INFO:teuthology.orchestra.run.smithi016.stdout: "last_force_op_resend_preluminous": "0", 2023-09-30T22:29:40.910 INFO:teuthology.orchestra.run.smithi016.stdout: "auid": 0, 2023-09-30T22:29:40.910 INFO:teuthology.orchestra.run.smithi016.stdout: "snap_mode": "selfmanaged", 2023-09-30T22:29:40.910 INFO:teuthology.orchestra.run.smithi016.stdout: "snap_seq": 0, 2023-09-30T22:29:40.910 INFO:teuthology.orchestra.run.smithi016.stdout: "snap_epoch": 0, 2023-09-30T22:29:40.910 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_snaps": [], 2023-09-30T22:29:40.910 INFO:teuthology.orchestra.run.smithi016.stdout: "removed_snaps": "[]", 2023-09-30T22:29:40.911 INFO:teuthology.orchestra.run.smithi016.stdout: "quota_max_bytes": 0, 2023-09-30T22:29:40.911 INFO:teuthology.orchestra.run.smithi016.stdout: "quota_max_objects": 0, 2023-09-30T22:29:40.911 INFO:teuthology.orchestra.run.smithi016.stdout: "tiers": [], 2023-09-30T22:29:40.911 INFO:teuthology.orchestra.run.smithi016.stdout: "tier_of": -1, 2023-09-30T22:29:40.911 INFO:teuthology.orchestra.run.smithi016.stdout: "read_tier": -1, 2023-09-30T22:29:40.911 INFO:teuthology.orchestra.run.smithi016.stdout: "write_tier": -1, 2023-09-30T22:29:40.911 INFO:teuthology.orchestra.run.smithi016.stdout: "cache_mode": "none", 2023-09-30T22:29:40.911 INFO:teuthology.orchestra.run.smithi016.stdout: "target_max_bytes": 0, 2023-09-30T22:29:40.911 INFO:teuthology.orchestra.run.smithi016.stdout: "target_max_objects": 0, 2023-09-30T22:29:40.911 INFO:teuthology.orchestra.run.smithi016.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-09-30T22:29:40.912 INFO:teuthology.orchestra.run.smithi016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-09-30T22:29:40.912 INFO:teuthology.orchestra.run.smithi016.stdout: "cache_target_full_ratio_micro": 800000, 2023-09-30T22:29:40.912 INFO:teuthology.orchestra.run.smithi016.stdout: "cache_min_flush_age": 0, 2023-09-30T22:29:40.912 INFO:teuthology.orchestra.run.smithi016.stdout: "cache_min_evict_age": 0, 2023-09-30T22:29:40.912 INFO:teuthology.orchestra.run.smithi016.stdout: "erasure_code_profile": "", 2023-09-30T22:29:40.912 INFO:teuthology.orchestra.run.smithi016.stdout: "hit_set_params": { 2023-09-30T22:29:40.912 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "none" 2023-09-30T22:29:40.912 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-09-30T22:29:40.912 INFO:teuthology.orchestra.run.smithi016.stdout: "hit_set_period": 0, 2023-09-30T22:29:40.912 INFO:teuthology.orchestra.run.smithi016.stdout: "hit_set_count": 0, 2023-09-30T22:29:40.913 INFO:teuthology.orchestra.run.smithi016.stdout: "use_gmt_hitset": true, 2023-09-30T22:29:40.913 INFO:teuthology.orchestra.run.smithi016.stdout: "min_read_recency_for_promote": 0, 2023-09-30T22:29:40.913 INFO:teuthology.orchestra.run.smithi016.stdout: "min_write_recency_for_promote": 0, 2023-09-30T22:29:40.913 INFO:teuthology.orchestra.run.smithi016.stdout: "hit_set_grade_decay_rate": 0, 2023-09-30T22:29:40.913 INFO:teuthology.orchestra.run.smithi016.stdout: "hit_set_search_last_n": 0, 2023-09-30T22:29:40.913 INFO:teuthology.orchestra.run.smithi016.stdout: "grade_table": [], 2023-09-30T22:29:40.913 INFO:teuthology.orchestra.run.smithi016.stdout: "stripe_width": 0, 2023-09-30T22:29:40.913 INFO:teuthology.orchestra.run.smithi016.stdout: "expected_num_objects": 0, 2023-09-30T22:29:40.913 INFO:teuthology.orchestra.run.smithi016.stdout: "fast_read": false, 2023-09-30T22:29:40.913 INFO:teuthology.orchestra.run.smithi016.stdout: "options": {}, 2023-09-30T22:29:40.913 INFO:teuthology.orchestra.run.smithi016.stdout: "application_metadata": { 2023-09-30T22:29:40.913 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd": {} 2023-09-30T22:29:40.914 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-09-30T22:29:40.914 INFO:teuthology.orchestra.run.smithi016.stdout: "read_balance": { 2023-09-30T22:29:40.914 INFO:teuthology.orchestra.run.smithi016.stdout: "score_acting": 2.5, 2023-09-30T22:29:40.914 INFO:teuthology.orchestra.run.smithi016.stdout: "score_stable": 2.5, 2023-09-30T22:29:40.914 INFO:teuthology.orchestra.run.smithi016.stdout: "optimal_score": 1, 2023-09-30T22:29:40.914 INFO:teuthology.orchestra.run.smithi016.stdout: "raw_score_acting": 2.5, 2023-09-30T22:29:40.914 INFO:teuthology.orchestra.run.smithi016.stdout: "raw_score_stable": 2.5, 2023-09-30T22:29:40.914 INFO:teuthology.orchestra.run.smithi016.stdout: "primary_affinity_weighted": 1, 2023-09-30T22:29:40.914 INFO:teuthology.orchestra.run.smithi016.stdout: "average_primary_affinity": 1, 2023-09-30T22:29:40.914 INFO:teuthology.orchestra.run.smithi016.stdout: "average_primary_affinity_weighted": 1 2023-09-30T22:29:40.914 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-09-30T22:29:40.915 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-09-30T22:29:40.915 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-09-30T22:29:40.915 INFO:teuthology.orchestra.run.smithi016.stdout: "pool": 3, 2023-09-30T22:29:40.915 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_name": "cephfs_metadata", 2023-09-30T22:29:40.915 INFO:teuthology.orchestra.run.smithi016.stdout: "create_time": "2023-09-30T22:26:49.353683+0000", 2023-09-30T22:29:40.915 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-09-30T22:29:40.915 INFO:teuthology.orchestra.run.smithi016.stdout: "flags_names": "hashpspool", 2023-09-30T22:29:40.915 INFO:teuthology.orchestra.run.smithi016.stdout: "type": 1, 2023-09-30T22:29:40.915 INFO:teuthology.orchestra.run.smithi016.stdout: "size": 2, 2023-09-30T22:29:40.915 INFO:teuthology.orchestra.run.smithi016.stdout: "min_size": 1, 2023-09-30T22:29:40.915 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_rule": 0, 2023-09-30T22:29:40.915 INFO:teuthology.orchestra.run.smithi016.stdout: "peering_crush_bucket_count": 0, 2023-09-30T22:29:40.916 INFO:teuthology.orchestra.run.smithi016.stdout: "peering_crush_bucket_target": 0, 2023-09-30T22:29:40.916 INFO:teuthology.orchestra.run.smithi016.stdout: "peering_crush_bucket_barrier": 0, 2023-09-30T22:29:40.916 INFO:teuthology.orchestra.run.smithi016.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-09-30T22:29:40.916 INFO:teuthology.orchestra.run.smithi016.stdout: "object_hash": 2, 2023-09-30T22:29:40.916 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscale_mode": "off", 2023-09-30T22:29:40.916 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_num": 32, 2023-09-30T22:29:40.916 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_placement_num": 32, 2023-09-30T22:29:40.916 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_placement_num_target": 32, 2023-09-30T22:29:40.916 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_num_target": 32, 2023-09-30T22:29:40.916 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_num_pending": 32, 2023-09-30T22:29:40.916 INFO:teuthology.orchestra.run.smithi016.stdout: "last_pg_merge_meta": { 2023-09-30T22:29:40.916 INFO:teuthology.orchestra.run.smithi016.stdout: "source_pgid": "0.0", 2023-09-30T22:29:40.917 INFO:teuthology.orchestra.run.smithi016.stdout: "ready_epoch": 0, 2023-09-30T22:29:40.917 INFO:teuthology.orchestra.run.smithi016.stdout: "last_epoch_started": 0, 2023-09-30T22:29:40.917 INFO:teuthology.orchestra.run.smithi016.stdout: "last_epoch_clean": 0, 2023-09-30T22:29:40.917 INFO:teuthology.orchestra.run.smithi016.stdout: "source_version": "0'0", 2023-09-30T22:29:40.917 INFO:teuthology.orchestra.run.smithi016.stdout: "target_version": "0'0" 2023-09-30T22:29:40.917 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-09-30T22:29:40.917 INFO:teuthology.orchestra.run.smithi016.stdout: "last_change": "15", 2023-09-30T22:29:40.917 INFO:teuthology.orchestra.run.smithi016.stdout: "last_force_op_resend": "0", 2023-09-30T22:29:40.917 INFO:teuthology.orchestra.run.smithi016.stdout: "last_force_op_resend_prenautilus": "0", 2023-09-30T22:29:40.917 INFO:teuthology.orchestra.run.smithi016.stdout: "last_force_op_resend_preluminous": "0", 2023-09-30T22:29:40.917 INFO:teuthology.orchestra.run.smithi016.stdout: "auid": 0, 2023-09-30T22:29:40.918 INFO:teuthology.orchestra.run.smithi016.stdout: "snap_mode": "selfmanaged", 2023-09-30T22:29:40.918 INFO:teuthology.orchestra.run.smithi016.stdout: "snap_seq": 0, 2023-09-30T22:29:40.918 INFO:teuthology.orchestra.run.smithi016.stdout: "snap_epoch": 0, 2023-09-30T22:29:40.918 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_snaps": [], 2023-09-30T22:29:40.918 INFO:teuthology.orchestra.run.smithi016.stdout: "removed_snaps": "[]", 2023-09-30T22:29:40.918 INFO:teuthology.orchestra.run.smithi016.stdout: "quota_max_bytes": 0, 2023-09-30T22:29:40.918 INFO:teuthology.orchestra.run.smithi016.stdout: "quota_max_objects": 0, 2023-09-30T22:29:40.918 INFO:teuthology.orchestra.run.smithi016.stdout: "tiers": [], 2023-09-30T22:29:40.918 INFO:teuthology.orchestra.run.smithi016.stdout: "tier_of": -1, 2023-09-30T22:29:40.918 INFO:teuthology.orchestra.run.smithi016.stdout: "read_tier": -1, 2023-09-30T22:29:40.918 INFO:teuthology.orchestra.run.smithi016.stdout: "write_tier": -1, 2023-09-30T22:29:40.919 INFO:teuthology.orchestra.run.smithi016.stdout: "cache_mode": "none", 2023-09-30T22:29:40.919 INFO:teuthology.orchestra.run.smithi016.stdout: "target_max_bytes": 0, 2023-09-30T22:29:40.919 INFO:teuthology.orchestra.run.smithi016.stdout: "target_max_objects": 0, 2023-09-30T22:29:40.919 INFO:teuthology.orchestra.run.smithi016.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-09-30T22:29:40.919 INFO:teuthology.orchestra.run.smithi016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-09-30T22:29:40.919 INFO:teuthology.orchestra.run.smithi016.stdout: "cache_target_full_ratio_micro": 800000, 2023-09-30T22:29:40.919 INFO:teuthology.orchestra.run.smithi016.stdout: "cache_min_flush_age": 0, 2023-09-30T22:29:40.919 INFO:teuthology.orchestra.run.smithi016.stdout: "cache_min_evict_age": 0, 2023-09-30T22:29:40.919 INFO:teuthology.orchestra.run.smithi016.stdout: "erasure_code_profile": "", 2023-09-30T22:29:40.919 INFO:teuthology.orchestra.run.smithi016.stdout: "hit_set_params": { 2023-09-30T22:29:40.919 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "none" 2023-09-30T22:29:40.919 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-09-30T22:29:40.919 INFO:teuthology.orchestra.run.smithi016.stdout: "hit_set_period": 0, 2023-09-30T22:29:40.920 INFO:teuthology.orchestra.run.smithi016.stdout: "hit_set_count": 0, 2023-09-30T22:29:40.920 INFO:teuthology.orchestra.run.smithi016.stdout: "use_gmt_hitset": true, 2023-09-30T22:29:40.920 INFO:teuthology.orchestra.run.smithi016.stdout: "min_read_recency_for_promote": 0, 2023-09-30T22:29:40.920 INFO:teuthology.orchestra.run.smithi016.stdout: "min_write_recency_for_promote": 0, 2023-09-30T22:29:40.920 INFO:teuthology.orchestra.run.smithi016.stdout: "hit_set_grade_decay_rate": 0, 2023-09-30T22:29:40.920 INFO:teuthology.orchestra.run.smithi016.stdout: "hit_set_search_last_n": 0, 2023-09-30T22:29:40.920 INFO:teuthology.orchestra.run.smithi016.stdout: "grade_table": [], 2023-09-30T22:29:40.920 INFO:teuthology.orchestra.run.smithi016.stdout: "stripe_width": 0, 2023-09-30T22:29:40.920 INFO:teuthology.orchestra.run.smithi016.stdout: "expected_num_objects": 0, 2023-09-30T22:29:40.920 INFO:teuthology.orchestra.run.smithi016.stdout: "fast_read": false, 2023-09-30T22:29:40.920 INFO:teuthology.orchestra.run.smithi016.stdout: "options": { 2023-09-30T22:29:40.920 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscale_bias": 4, 2023-09-30T22:29:40.921 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_num_min": 16, 2023-09-30T22:29:40.921 INFO:teuthology.orchestra.run.smithi016.stdout: "recovery_priority": 5 2023-09-30T22:29:40.921 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-09-30T22:29:40.921 INFO:teuthology.orchestra.run.smithi016.stdout: "application_metadata": { 2023-09-30T22:29:40.921 INFO:teuthology.orchestra.run.smithi016.stdout: "cephfs": { 2023-09-30T22:29:40.921 INFO:teuthology.orchestra.run.smithi016.stdout: "metadata": "cephfs" 2023-09-30T22:29:40.921 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-09-30T22:29:40.921 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-09-30T22:29:40.921 INFO:teuthology.orchestra.run.smithi016.stdout: "read_balance": { 2023-09-30T22:29:40.921 INFO:teuthology.orchestra.run.smithi016.stdout: "score_acting": 1.6299999952316284, 2023-09-30T22:29:40.921 INFO:teuthology.orchestra.run.smithi016.stdout: "score_stable": 1.6299999952316284, 2023-09-30T22:29:40.921 INFO:teuthology.orchestra.run.smithi016.stdout: "optimal_score": 1, 2023-09-30T22:29:40.921 INFO:teuthology.orchestra.run.smithi016.stdout: "raw_score_acting": 1.6299999952316284, 2023-09-30T22:29:40.922 INFO:teuthology.orchestra.run.smithi016.stdout: "raw_score_stable": 1.6299999952316284, 2023-09-30T22:29:40.922 INFO:teuthology.orchestra.run.smithi016.stdout: "primary_affinity_weighted": 1, 2023-09-30T22:29:40.922 INFO:teuthology.orchestra.run.smithi016.stdout: "average_primary_affinity": 1, 2023-09-30T22:29:40.922 INFO:teuthology.orchestra.run.smithi016.stdout: "average_primary_affinity_weighted": 1 2023-09-30T22:29:40.922 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-09-30T22:29:40.922 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-09-30T22:29:40.922 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-09-30T22:29:40.922 INFO:teuthology.orchestra.run.smithi016.stdout: "pool": 4, 2023-09-30T22:29:40.922 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_name": "cephfs_data", 2023-09-30T22:29:40.922 INFO:teuthology.orchestra.run.smithi016.stdout: "create_time": "2023-09-30T22:26:50.655685+0000", 2023-09-30T22:29:40.922 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2023-09-30T22:29:40.923 INFO:teuthology.orchestra.run.smithi016.stdout: "flags_names": "hashpspool", 2023-09-30T22:29:40.923 INFO:teuthology.orchestra.run.smithi016.stdout: "type": 1, 2023-09-30T22:29:40.923 INFO:teuthology.orchestra.run.smithi016.stdout: "size": 2, 2023-09-30T22:29:40.923 INFO:teuthology.orchestra.run.smithi016.stdout: "min_size": 1, 2023-09-30T22:29:40.923 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_rule": 0, 2023-09-30T22:29:40.923 INFO:teuthology.orchestra.run.smithi016.stdout: "peering_crush_bucket_count": 0, 2023-09-30T22:29:40.923 INFO:teuthology.orchestra.run.smithi016.stdout: "peering_crush_bucket_target": 0, 2023-09-30T22:29:40.923 INFO:teuthology.orchestra.run.smithi016.stdout: "peering_crush_bucket_barrier": 0, 2023-09-30T22:29:40.923 INFO:teuthology.orchestra.run.smithi016.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-09-30T22:29:40.923 INFO:teuthology.orchestra.run.smithi016.stdout: "object_hash": 2, 2023-09-30T22:29:40.923 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscale_mode": "off", 2023-09-30T22:29:40.923 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_num": 64, 2023-09-30T22:29:40.923 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_placement_num": 64, 2023-09-30T22:29:40.924 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_placement_num_target": 64, 2023-09-30T22:29:40.924 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_num_target": 64, 2023-09-30T22:29:40.924 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_num_pending": 64, 2023-09-30T22:29:40.924 INFO:teuthology.orchestra.run.smithi016.stdout: "last_pg_merge_meta": { 2023-09-30T22:29:40.924 INFO:teuthology.orchestra.run.smithi016.stdout: "source_pgid": "0.0", 2023-09-30T22:29:40.924 INFO:teuthology.orchestra.run.smithi016.stdout: "ready_epoch": 0, 2023-09-30T22:29:40.924 INFO:teuthology.orchestra.run.smithi016.stdout: "last_epoch_started": 0, 2023-09-30T22:29:40.924 INFO:teuthology.orchestra.run.smithi016.stdout: "last_epoch_clean": 0, 2023-09-30T22:29:40.924 INFO:teuthology.orchestra.run.smithi016.stdout: "source_version": "0'0", 2023-09-30T22:29:40.924 INFO:teuthology.orchestra.run.smithi016.stdout: "target_version": "0'0" 2023-09-30T22:29:40.924 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-09-30T22:29:40.924 INFO:teuthology.orchestra.run.smithi016.stdout: "last_change": "16", 2023-09-30T22:29:40.925 INFO:teuthology.orchestra.run.smithi016.stdout: "last_force_op_resend": "0", 2023-09-30T22:29:40.925 INFO:teuthology.orchestra.run.smithi016.stdout: "last_force_op_resend_prenautilus": "0", 2023-09-30T22:29:40.925 INFO:teuthology.orchestra.run.smithi016.stdout: "last_force_op_resend_preluminous": "0", 2023-09-30T22:29:40.925 INFO:teuthology.orchestra.run.smithi016.stdout: "auid": 0, 2023-09-30T22:29:40.925 INFO:teuthology.orchestra.run.smithi016.stdout: "snap_mode": "selfmanaged", 2023-09-30T22:29:40.925 INFO:teuthology.orchestra.run.smithi016.stdout: "snap_seq": 0, 2023-09-30T22:29:40.925 INFO:teuthology.orchestra.run.smithi016.stdout: "snap_epoch": 0, 2023-09-30T22:29:40.925 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_snaps": [], 2023-09-30T22:29:40.925 INFO:teuthology.orchestra.run.smithi016.stdout: "removed_snaps": "[]", 2023-09-30T22:29:40.925 INFO:teuthology.orchestra.run.smithi016.stdout: "quota_max_bytes": 0, 2023-09-30T22:29:40.925 INFO:teuthology.orchestra.run.smithi016.stdout: "quota_max_objects": 0, 2023-09-30T22:29:40.925 INFO:teuthology.orchestra.run.smithi016.stdout: "tiers": [], 2023-09-30T22:29:40.926 INFO:teuthology.orchestra.run.smithi016.stdout: "tier_of": -1, 2023-09-30T22:29:40.926 INFO:teuthology.orchestra.run.smithi016.stdout: "read_tier": -1, 2023-09-30T22:29:40.926 INFO:teuthology.orchestra.run.smithi016.stdout: "write_tier": -1, 2023-09-30T22:29:40.926 INFO:teuthology.orchestra.run.smithi016.stdout: "cache_mode": "none", 2023-09-30T22:29:40.926 INFO:teuthology.orchestra.run.smithi016.stdout: "target_max_bytes": 0, 2023-09-30T22:29:40.926 INFO:teuthology.orchestra.run.smithi016.stdout: "target_max_objects": 0, 2023-09-30T22:29:40.926 INFO:teuthology.orchestra.run.smithi016.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-09-30T22:29:40.926 INFO:teuthology.orchestra.run.smithi016.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-09-30T22:29:40.926 INFO:teuthology.orchestra.run.smithi016.stdout: "cache_target_full_ratio_micro": 800000, 2023-09-30T22:29:40.926 INFO:teuthology.orchestra.run.smithi016.stdout: "cache_min_flush_age": 0, 2023-09-30T22:29:40.926 INFO:teuthology.orchestra.run.smithi016.stdout: "cache_min_evict_age": 0, 2023-09-30T22:29:40.926 INFO:teuthology.orchestra.run.smithi016.stdout: "erasure_code_profile": "", 2023-09-30T22:29:40.927 INFO:teuthology.orchestra.run.smithi016.stdout: "hit_set_params": { 2023-09-30T22:29:40.927 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "none" 2023-09-30T22:29:40.927 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-09-30T22:29:40.927 INFO:teuthology.orchestra.run.smithi016.stdout: "hit_set_period": 0, 2023-09-30T22:29:40.927 INFO:teuthology.orchestra.run.smithi016.stdout: "hit_set_count": 0, 2023-09-30T22:29:40.927 INFO:teuthology.orchestra.run.smithi016.stdout: "use_gmt_hitset": true, 2023-09-30T22:29:40.927 INFO:teuthology.orchestra.run.smithi016.stdout: "min_read_recency_for_promote": 0, 2023-09-30T22:29:40.927 INFO:teuthology.orchestra.run.smithi016.stdout: "min_write_recency_for_promote": 0, 2023-09-30T22:29:40.927 INFO:teuthology.orchestra.run.smithi016.stdout: "hit_set_grade_decay_rate": 0, 2023-09-30T22:29:40.927 INFO:teuthology.orchestra.run.smithi016.stdout: "hit_set_search_last_n": 0, 2023-09-30T22:29:40.927 INFO:teuthology.orchestra.run.smithi016.stdout: "grade_table": [], 2023-09-30T22:29:40.927 INFO:teuthology.orchestra.run.smithi016.stdout: "stripe_width": 0, 2023-09-30T22:29:40.928 INFO:teuthology.orchestra.run.smithi016.stdout: "expected_num_objects": 0, 2023-09-30T22:29:40.928 INFO:teuthology.orchestra.run.smithi016.stdout: "fast_read": false, 2023-09-30T22:29:40.928 INFO:teuthology.orchestra.run.smithi016.stdout: "options": { 2023-09-30T22:29:40.928 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_num_min": 64, 2023-09-30T22:29:40.928 INFO:teuthology.orchestra.run.smithi016.stdout: "target_size_ratio": 0.89999997615814209 2023-09-30T22:29:40.928 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-09-30T22:29:40.928 INFO:teuthology.orchestra.run.smithi016.stdout: "application_metadata": { 2023-09-30T22:29:40.928 INFO:teuthology.orchestra.run.smithi016.stdout: "cephfs": { 2023-09-30T22:29:40.928 INFO:teuthology.orchestra.run.smithi016.stdout: "data": "cephfs" 2023-09-30T22:29:40.928 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-09-30T22:29:40.928 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-09-30T22:29:40.929 INFO:teuthology.orchestra.run.smithi016.stdout: "read_balance": { 2023-09-30T22:29:40.929 INFO:teuthology.orchestra.run.smithi016.stdout: "score_acting": 1.1299999952316284, 2023-09-30T22:29:40.929 INFO:teuthology.orchestra.run.smithi016.stdout: "score_stable": 1.1299999952316284, 2023-09-30T22:29:40.929 INFO:teuthology.orchestra.run.smithi016.stdout: "optimal_score": 1, 2023-09-30T22:29:40.929 INFO:teuthology.orchestra.run.smithi016.stdout: "raw_score_acting": 1.1299999952316284, 2023-09-30T22:29:40.929 INFO:teuthology.orchestra.run.smithi016.stdout: "raw_score_stable": 1.1299999952316284, 2023-09-30T22:29:40.929 INFO:teuthology.orchestra.run.smithi016.stdout: "primary_affinity_weighted": 1, 2023-09-30T22:29:40.929 INFO:teuthology.orchestra.run.smithi016.stdout: "average_primary_affinity": 1, 2023-09-30T22:29:40.929 INFO:teuthology.orchestra.run.smithi016.stdout: "average_primary_affinity_weighted": 1 2023-09-30T22:29:40.929 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-09-30T22:29:40.929 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-09-30T22:29:40.929 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-09-30T22:29:40.930 INFO:teuthology.orchestra.run.smithi016.stdout: "osds": [ 2023-09-30T22:29:40.930 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-09-30T22:29:40.930 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": 0, 2023-09-30T22:29:40.930 INFO:teuthology.orchestra.run.smithi016.stdout: "uuid": "19946985-0ee2-487f-9347-929670dbe5ff", 2023-09-30T22:29:40.930 INFO:teuthology.orchestra.run.smithi016.stdout: "up": 1, 2023-09-30T22:29:40.930 INFO:teuthology.orchestra.run.smithi016.stdout: "in": 1, 2023-09-30T22:29:40.930 INFO:teuthology.orchestra.run.smithi016.stdout: "weight": 1, 2023-09-30T22:29:40.930 INFO:teuthology.orchestra.run.smithi016.stdout: "primary_affinity": 1, 2023-09-30T22:29:40.930 INFO:teuthology.orchestra.run.smithi016.stdout: "last_clean_begin": 9, 2023-09-30T22:29:40.930 INFO:teuthology.orchestra.run.smithi016.stdout: "last_clean_end": 22, 2023-09-30T22:29:40.930 INFO:teuthology.orchestra.run.smithi016.stdout: "up_from": 28, 2023-09-30T22:29:40.930 INFO:teuthology.orchestra.run.smithi016.stdout: "up_thru": 32, 2023-09-30T22:29:40.931 INFO:teuthology.orchestra.run.smithi016.stdout: "down_at": 23, 2023-09-30T22:29:40.931 INFO:teuthology.orchestra.run.smithi016.stdout: "lost_at": 0, 2023-09-30T22:29:40.931 INFO:teuthology.orchestra.run.smithi016.stdout: "public_addrs": { 2023-09-30T22:29:40.931 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2023-09-30T22:29:40.931 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-09-30T22:29:40.931 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2023-09-30T22:29:40.931 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:6804", 2023-09-30T22:29:40.931 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 129539 2023-09-30T22:29:40.931 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-09-30T22:29:40.931 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2023-09-30T22:29:40.931 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-09-30T22:29:40.931 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_addrs": { 2023-09-30T22:29:40.932 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2023-09-30T22:29:40.932 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-09-30T22:29:40.932 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2023-09-30T22:29:40.932 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:6805", 2023-09-30T22:29:40.932 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 129539 2023-09-30T22:29:40.932 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-09-30T22:29:40.932 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2023-09-30T22:29:40.932 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-09-30T22:29:40.932 INFO:teuthology.orchestra.run.smithi016.stdout: "heartbeat_back_addrs": { 2023-09-30T22:29:40.932 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2023-09-30T22:29:40.932 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-09-30T22:29:40.932 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2023-09-30T22:29:40.933 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:6807", 2023-09-30T22:29:40.933 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 129539 2023-09-30T22:29:40.933 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-09-30T22:29:40.933 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2023-09-30T22:29:40.933 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-09-30T22:29:40.933 INFO:teuthology.orchestra.run.smithi016.stdout: "heartbeat_front_addrs": { 2023-09-30T22:29:40.933 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2023-09-30T22:29:40.933 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-09-30T22:29:40.933 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2023-09-30T22:29:40.933 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:6806", 2023-09-30T22:29:40.933 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 129539 2023-09-30T22:29:40.933 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-09-30T22:29:40.934 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2023-09-30T22:29:40.934 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-09-30T22:29:40.934 INFO:teuthology.orchestra.run.smithi016.stdout: "public_addr": "172.21.15.16:6804/129539", 2023-09-30T22:29:40.934 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_addr": "172.21.15.16:6805/129539", 2023-09-30T22:29:40.934 INFO:teuthology.orchestra.run.smithi016.stdout: "heartbeat_back_addr": "172.21.15.16:6807/129539", 2023-09-30T22:29:40.934 INFO:teuthology.orchestra.run.smithi016.stdout: "heartbeat_front_addr": "172.21.15.16:6806/129539", 2023-09-30T22:29:40.934 INFO:teuthology.orchestra.run.smithi016.stdout: "state": [ 2023-09-30T22:29:40.934 INFO:teuthology.orchestra.run.smithi016.stdout: "exists", 2023-09-30T22:29:40.934 INFO:teuthology.orchestra.run.smithi016.stdout: "up" 2023-09-30T22:29:40.934 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2023-09-30T22:29:40.934 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-09-30T22:29:40.934 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-09-30T22:29:40.935 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": 1, 2023-09-30T22:29:40.935 INFO:teuthology.orchestra.run.smithi016.stdout: "uuid": "8efd4dbc-6ae4-4b82-b530-3b3eac2be445", 2023-09-30T22:29:40.935 INFO:teuthology.orchestra.run.smithi016.stdout: "up": 1, 2023-09-30T22:29:40.935 INFO:teuthology.orchestra.run.smithi016.stdout: "in": 1, 2023-09-30T22:29:40.935 INFO:teuthology.orchestra.run.smithi016.stdout: "weight": 1, 2023-09-30T22:29:40.935 INFO:teuthology.orchestra.run.smithi016.stdout: "primary_affinity": 1, 2023-09-30T22:29:40.935 INFO:teuthology.orchestra.run.smithi016.stdout: "last_clean_begin": 9, 2023-09-30T22:29:40.935 INFO:teuthology.orchestra.run.smithi016.stdout: "last_clean_end": 22, 2023-09-30T22:29:40.935 INFO:teuthology.orchestra.run.smithi016.stdout: "up_from": 29, 2023-09-30T22:29:40.935 INFO:teuthology.orchestra.run.smithi016.stdout: "up_thru": 32, 2023-09-30T22:29:40.935 INFO:teuthology.orchestra.run.smithi016.stdout: "down_at": 24, 2023-09-30T22:29:40.936 INFO:teuthology.orchestra.run.smithi016.stdout: "lost_at": 0, 2023-09-30T22:29:40.936 INFO:teuthology.orchestra.run.smithi016.stdout: "public_addrs": { 2023-09-30T22:29:40.936 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2023-09-30T22:29:40.936 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-09-30T22:29:40.936 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2023-09-30T22:29:40.936 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:6808", 2023-09-30T22:29:40.936 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 129637 2023-09-30T22:29:40.936 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-09-30T22:29:40.936 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2023-09-30T22:29:40.936 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-09-30T22:29:40.936 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_addrs": { 2023-09-30T22:29:40.936 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2023-09-30T22:29:40.936 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-09-30T22:29:40.937 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2023-09-30T22:29:40.937 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:6809", 2023-09-30T22:29:40.937 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 129637 2023-09-30T22:29:40.937 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-09-30T22:29:40.937 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2023-09-30T22:29:40.937 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-09-30T22:29:40.937 INFO:teuthology.orchestra.run.smithi016.stdout: "heartbeat_back_addrs": { 2023-09-30T22:29:40.937 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2023-09-30T22:29:40.937 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-09-30T22:29:40.937 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2023-09-30T22:29:40.937 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:6811", 2023-09-30T22:29:40.937 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 129637 2023-09-30T22:29:40.938 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-09-30T22:29:40.938 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2023-09-30T22:29:40.938 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-09-30T22:29:40.938 INFO:teuthology.orchestra.run.smithi016.stdout: "heartbeat_front_addrs": { 2023-09-30T22:29:40.938 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2023-09-30T22:29:40.938 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-09-30T22:29:40.938 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2023-09-30T22:29:40.938 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:6810", 2023-09-30T22:29:40.938 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 129637 2023-09-30T22:29:40.938 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-09-30T22:29:40.938 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2023-09-30T22:29:40.938 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-09-30T22:29:40.939 INFO:teuthology.orchestra.run.smithi016.stdout: "public_addr": "172.21.15.16:6808/129637", 2023-09-30T22:29:40.939 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_addr": "172.21.15.16:6809/129637", 2023-09-30T22:29:40.939 INFO:teuthology.orchestra.run.smithi016.stdout: "heartbeat_back_addr": "172.21.15.16:6811/129637", 2023-09-30T22:29:40.939 INFO:teuthology.orchestra.run.smithi016.stdout: "heartbeat_front_addr": "172.21.15.16:6810/129637", 2023-09-30T22:29:40.939 INFO:teuthology.orchestra.run.smithi016.stdout: "state": [ 2023-09-30T22:29:40.939 INFO:teuthology.orchestra.run.smithi016.stdout: "exists", 2023-09-30T22:29:40.939 INFO:teuthology.orchestra.run.smithi016.stdout: "up" 2023-09-30T22:29:40.939 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2023-09-30T22:29:40.939 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-09-30T22:29:40.939 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-09-30T22:29:40.939 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": 2, 2023-09-30T22:29:40.939 INFO:teuthology.orchestra.run.smithi016.stdout: "uuid": "b47bb1ce-27af-4c0c-868a-b76ebd0d9499", 2023-09-30T22:29:40.940 INFO:teuthology.orchestra.run.smithi016.stdout: "up": 1, 2023-09-30T22:29:40.940 INFO:teuthology.orchestra.run.smithi016.stdout: "in": 1, 2023-09-30T22:29:40.940 INFO:teuthology.orchestra.run.smithi016.stdout: "weight": 1, 2023-09-30T22:29:40.940 INFO:teuthology.orchestra.run.smithi016.stdout: "primary_affinity": 1, 2023-09-30T22:29:40.940 INFO:teuthology.orchestra.run.smithi016.stdout: "last_clean_begin": 9, 2023-09-30T22:29:40.940 INFO:teuthology.orchestra.run.smithi016.stdout: "last_clean_end": 24, 2023-09-30T22:29:40.940 INFO:teuthology.orchestra.run.smithi016.stdout: "up_from": 31, 2023-09-30T22:29:40.940 INFO:teuthology.orchestra.run.smithi016.stdout: "up_thru": 32, 2023-09-30T22:29:40.940 INFO:teuthology.orchestra.run.smithi016.stdout: "down_at": 25, 2023-09-30T22:29:40.940 INFO:teuthology.orchestra.run.smithi016.stdout: "lost_at": 0, 2023-09-30T22:29:40.940 INFO:teuthology.orchestra.run.smithi016.stdout: "public_addrs": { 2023-09-30T22:29:40.940 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2023-09-30T22:29:40.941 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-09-30T22:29:40.941 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2023-09-30T22:29:40.941 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:6800", 2023-09-30T22:29:40.941 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 129747 2023-09-30T22:29:40.941 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-09-30T22:29:40.941 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2023-09-30T22:29:40.941 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-09-30T22:29:40.941 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_addrs": { 2023-09-30T22:29:40.941 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2023-09-30T22:29:40.941 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-09-30T22:29:40.941 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2023-09-30T22:29:40.941 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:6801", 2023-09-30T22:29:40.942 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 129747 2023-09-30T22:29:40.942 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-09-30T22:29:40.942 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2023-09-30T22:29:40.942 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-09-30T22:29:40.942 INFO:teuthology.orchestra.run.smithi016.stdout: "heartbeat_back_addrs": { 2023-09-30T22:29:40.942 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2023-09-30T22:29:40.942 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-09-30T22:29:40.942 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2023-09-30T22:29:40.942 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:6803", 2023-09-30T22:29:40.942 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 129747 2023-09-30T22:29:40.942 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-09-30T22:29:40.942 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2023-09-30T22:29:40.943 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-09-30T22:29:40.943 INFO:teuthology.orchestra.run.smithi016.stdout: "heartbeat_front_addrs": { 2023-09-30T22:29:40.943 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2023-09-30T22:29:40.943 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-09-30T22:29:40.943 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2023-09-30T22:29:40.943 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:6802", 2023-09-30T22:29:40.943 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 129747 2023-09-30T22:29:40.943 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-09-30T22:29:40.943 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2023-09-30T22:29:40.943 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-09-30T22:29:40.943 INFO:teuthology.orchestra.run.smithi016.stdout: "public_addr": "172.21.15.16:6800/129747", 2023-09-30T22:29:40.943 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_addr": "172.21.15.16:6801/129747", 2023-09-30T22:29:40.944 INFO:teuthology.orchestra.run.smithi016.stdout: "heartbeat_back_addr": "172.21.15.16:6803/129747", 2023-09-30T22:29:40.944 INFO:teuthology.orchestra.run.smithi016.stdout: "heartbeat_front_addr": "172.21.15.16:6802/129747", 2023-09-30T22:29:40.944 INFO:teuthology.orchestra.run.smithi016.stdout: "state": [ 2023-09-30T22:29:40.944 INFO:teuthology.orchestra.run.smithi016.stdout: "exists", 2023-09-30T22:29:40.944 INFO:teuthology.orchestra.run.smithi016.stdout: "up" 2023-09-30T22:29:40.944 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2023-09-30T22:29:40.944 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-09-30T22:29:40.944 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-09-30T22:29:40.944 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": 3, 2023-09-30T22:29:40.944 INFO:teuthology.orchestra.run.smithi016.stdout: "uuid": "03b53d64-d10d-448a-a081-39ecceab589b", 2023-09-30T22:29:40.944 INFO:teuthology.orchestra.run.smithi016.stdout: "up": 1, 2023-09-30T22:29:40.944 INFO:teuthology.orchestra.run.smithi016.stdout: "in": 1, 2023-09-30T22:29:40.944 INFO:teuthology.orchestra.run.smithi016.stdout: "weight": 1, 2023-09-30T22:29:40.945 INFO:teuthology.orchestra.run.smithi016.stdout: "primary_affinity": 1, 2023-09-30T22:29:40.945 INFO:teuthology.orchestra.run.smithi016.stdout: "last_clean_begin": 9, 2023-09-30T22:29:40.945 INFO:teuthology.orchestra.run.smithi016.stdout: "last_clean_end": 25, 2023-09-30T22:29:40.945 INFO:teuthology.orchestra.run.smithi016.stdout: "up_from": 32, 2023-09-30T22:29:40.945 INFO:teuthology.orchestra.run.smithi016.stdout: "up_thru": 32, 2023-09-30T22:29:40.945 INFO:teuthology.orchestra.run.smithi016.stdout: "down_at": 26, 2023-09-30T22:29:40.945 INFO:teuthology.orchestra.run.smithi016.stdout: "lost_at": 0, 2023-09-30T22:29:40.945 INFO:teuthology.orchestra.run.smithi016.stdout: "public_addrs": { 2023-09-30T22:29:40.945 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2023-09-30T22:29:40.945 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-09-30T22:29:40.945 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2023-09-30T22:29:40.945 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:6812", 2023-09-30T22:29:40.946 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 130087 2023-09-30T22:29:40.946 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-09-30T22:29:40.946 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2023-09-30T22:29:40.946 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-09-30T22:29:40.946 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_addrs": { 2023-09-30T22:29:40.946 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2023-09-30T22:29:40.946 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-09-30T22:29:40.946 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2023-09-30T22:29:40.946 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:6813", 2023-09-30T22:29:40.946 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 130087 2023-09-30T22:29:40.946 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-09-30T22:29:40.946 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2023-09-30T22:29:40.947 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-09-30T22:29:40.947 INFO:teuthology.orchestra.run.smithi016.stdout: "heartbeat_back_addrs": { 2023-09-30T22:29:40.947 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2023-09-30T22:29:40.947 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-09-30T22:29:40.947 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2023-09-30T22:29:40.947 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:6815", 2023-09-30T22:29:40.947 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 130087 2023-09-30T22:29:40.947 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-09-30T22:29:40.947 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2023-09-30T22:29:40.947 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-09-30T22:29:40.947 INFO:teuthology.orchestra.run.smithi016.stdout: "heartbeat_front_addrs": { 2023-09-30T22:29:40.947 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2023-09-30T22:29:40.947 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-09-30T22:29:40.948 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2023-09-30T22:29:40.948 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:6814", 2023-09-30T22:29:40.948 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 130087 2023-09-30T22:29:40.948 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-09-30T22:29:40.948 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2023-09-30T22:29:40.948 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-09-30T22:29:40.948 INFO:teuthology.orchestra.run.smithi016.stdout: "public_addr": "172.21.15.16:6812/130087", 2023-09-30T22:29:40.948 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_addr": "172.21.15.16:6813/130087", 2023-09-30T22:29:40.948 INFO:teuthology.orchestra.run.smithi016.stdout: "heartbeat_back_addr": "172.21.15.16:6815/130087", 2023-09-30T22:29:40.948 INFO:teuthology.orchestra.run.smithi016.stdout: "heartbeat_front_addr": "172.21.15.16:6814/130087", 2023-09-30T22:29:40.948 INFO:teuthology.orchestra.run.smithi016.stdout: "state": [ 2023-09-30T22:29:40.948 INFO:teuthology.orchestra.run.smithi016.stdout: "exists", 2023-09-30T22:29:40.949 INFO:teuthology.orchestra.run.smithi016.stdout: "up" 2023-09-30T22:29:40.949 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2023-09-30T22:29:40.949 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-09-30T22:29:40.949 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-09-30T22:29:40.949 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_xinfo": [ 2023-09-30T22:29:40.949 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-09-30T22:29:40.949 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": 0, 2023-09-30T22:29:40.949 INFO:teuthology.orchestra.run.smithi016.stdout: "down_stamp": "0.000000", 2023-09-30T22:29:40.949 INFO:teuthology.orchestra.run.smithi016.stdout: "laggy_probability": 0, 2023-09-30T22:29:40.949 INFO:teuthology.orchestra.run.smithi016.stdout: "laggy_interval": 0, 2023-09-30T22:29:40.949 INFO:teuthology.orchestra.run.smithi016.stdout: "features": 4540138322906710015, 2023-09-30T22:29:40.949 INFO:teuthology.orchestra.run.smithi016.stdout: "old_weight": 0, 2023-09-30T22:29:40.949 INFO:teuthology.orchestra.run.smithi016.stdout: "last_purged_snaps_scrub": "2023-09-30T22:26:42.305953+0000", 2023-09-30T22:29:40.950 INFO:teuthology.orchestra.run.smithi016.stdout: "dead_epoch": 23 2023-09-30T22:29:40.950 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-09-30T22:29:40.950 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-09-30T22:29:40.950 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": 1, 2023-09-30T22:29:40.950 INFO:teuthology.orchestra.run.smithi016.stdout: "down_stamp": "0.000000", 2023-09-30T22:29:40.950 INFO:teuthology.orchestra.run.smithi016.stdout: "laggy_probability": 0, 2023-09-30T22:29:40.950 INFO:teuthology.orchestra.run.smithi016.stdout: "laggy_interval": 0, 2023-09-30T22:29:40.950 INFO:teuthology.orchestra.run.smithi016.stdout: "features": 4540138322906710015, 2023-09-30T22:29:40.950 INFO:teuthology.orchestra.run.smithi016.stdout: "old_weight": 0, 2023-09-30T22:29:40.950 INFO:teuthology.orchestra.run.smithi016.stdout: "last_purged_snaps_scrub": "2023-09-30T22:26:42.307488+0000", 2023-09-30T22:29:40.950 INFO:teuthology.orchestra.run.smithi016.stdout: "dead_epoch": 24 2023-09-30T22:29:40.950 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-09-30T22:29:40.951 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-09-30T22:29:40.951 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": 2, 2023-09-30T22:29:40.951 INFO:teuthology.orchestra.run.smithi016.stdout: "down_stamp": "0.000000", 2023-09-30T22:29:40.951 INFO:teuthology.orchestra.run.smithi016.stdout: "laggy_probability": 0, 2023-09-30T22:29:40.951 INFO:teuthology.orchestra.run.smithi016.stdout: "laggy_interval": 0, 2023-09-30T22:29:40.951 INFO:teuthology.orchestra.run.smithi016.stdout: "features": 4540138322906710015, 2023-09-30T22:29:40.951 INFO:teuthology.orchestra.run.smithi016.stdout: "old_weight": 0, 2023-09-30T22:29:40.951 INFO:teuthology.orchestra.run.smithi016.stdout: "last_purged_snaps_scrub": "2023-09-30T22:26:42.341023+0000", 2023-09-30T22:29:40.951 INFO:teuthology.orchestra.run.smithi016.stdout: "dead_epoch": 25 2023-09-30T22:29:40.951 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-09-30T22:29:40.951 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-09-30T22:29:40.951 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": 3, 2023-09-30T22:29:40.952 INFO:teuthology.orchestra.run.smithi016.stdout: "down_stamp": "2023-09-30T22:29:28.519844+0000", 2023-09-30T22:29:40.952 INFO:teuthology.orchestra.run.smithi016.stdout: "laggy_probability": 0, 2023-09-30T22:29:40.952 INFO:teuthology.orchestra.run.smithi016.stdout: "laggy_interval": 0, 2023-09-30T22:29:40.952 INFO:teuthology.orchestra.run.smithi016.stdout: "features": 4540138322906710015, 2023-09-30T22:29:40.952 INFO:teuthology.orchestra.run.smithi016.stdout: "old_weight": 0, 2023-09-30T22:29:40.952 INFO:teuthology.orchestra.run.smithi016.stdout: "last_purged_snaps_scrub": "2023-09-30T22:26:42.192071+0000", 2023-09-30T22:29:40.952 INFO:teuthology.orchestra.run.smithi016.stdout: "dead_epoch": 0 2023-09-30T22:29:40.952 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-09-30T22:29:40.952 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-09-30T22:29:40.952 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_upmap": [], 2023-09-30T22:29:40.952 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_upmap_items": [ 2023-09-30T22:29:40.952 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-09-30T22:29:40.953 INFO:teuthology.orchestra.run.smithi016.stdout: "pgid": "4.0", 2023-09-30T22:29:40.953 INFO:teuthology.orchestra.run.smithi016.stdout: "mappings": [ 2023-09-30T22:29:40.953 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-09-30T22:29:40.953 INFO:teuthology.orchestra.run.smithi016.stdout: "from": 3, 2023-09-30T22:29:40.953 INFO:teuthology.orchestra.run.smithi016.stdout: "to": 2 2023-09-30T22:29:40.953 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-09-30T22:29:40.953 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2023-09-30T22:29:40.953 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-09-30T22:29:40.953 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-09-30T22:29:40.953 INFO:teuthology.orchestra.run.smithi016.stdout: "pgid": "4.8", 2023-09-30T22:29:40.953 INFO:teuthology.orchestra.run.smithi016.stdout: "mappings": [ 2023-09-30T22:29:40.953 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-09-30T22:29:40.954 INFO:teuthology.orchestra.run.smithi016.stdout: "from": 3, 2023-09-30T22:29:40.954 INFO:teuthology.orchestra.run.smithi016.stdout: "to": 2 2023-09-30T22:29:40.954 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-09-30T22:29:40.954 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2023-09-30T22:29:40.954 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-09-30T22:29:40.954 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-09-30T22:29:40.954 INFO:teuthology.orchestra.run.smithi016.stdout: "pgid": "4.d", 2023-09-30T22:29:40.954 INFO:teuthology.orchestra.run.smithi016.stdout: "mappings": [ 2023-09-30T22:29:40.954 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-09-30T22:29:40.955 INFO:teuthology.orchestra.run.smithi016.stdout: "from": 3, 2023-09-30T22:29:40.955 INFO:teuthology.orchestra.run.smithi016.stdout: "to": 2 2023-09-30T22:29:40.955 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-09-30T22:29:40.955 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2023-09-30T22:29:40.955 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-09-30T22:29:40.955 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-09-30T22:29:40.955 INFO:teuthology.orchestra.run.smithi016.stdout: "pgid": "4.f", 2023-09-30T22:29:40.955 INFO:teuthology.orchestra.run.smithi016.stdout: "mappings": [ 2023-09-30T22:29:40.955 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-09-30T22:29:40.955 INFO:teuthology.orchestra.run.smithi016.stdout: "from": 1, 2023-09-30T22:29:40.955 INFO:teuthology.orchestra.run.smithi016.stdout: "to": 0 2023-09-30T22:29:40.955 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-09-30T22:29:40.956 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2023-09-30T22:29:40.956 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-09-30T22:29:40.956 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-09-30T22:29:40.956 INFO:teuthology.orchestra.run.smithi016.stdout: "pgid": "4.14", 2023-09-30T22:29:40.956 INFO:teuthology.orchestra.run.smithi016.stdout: "mappings": [ 2023-09-30T22:29:40.956 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-09-30T22:29:40.956 INFO:teuthology.orchestra.run.smithi016.stdout: "from": 3, 2023-09-30T22:29:40.956 INFO:teuthology.orchestra.run.smithi016.stdout: "to": 2 2023-09-30T22:29:40.956 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-09-30T22:29:40.956 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2023-09-30T22:29:40.957 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-09-30T22:29:40.957 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-09-30T22:29:40.957 INFO:teuthology.orchestra.run.smithi016.stdout: "pgid": "4.17", 2023-09-30T22:29:40.957 INFO:teuthology.orchestra.run.smithi016.stdout: "mappings": [ 2023-09-30T22:29:40.957 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-09-30T22:29:40.957 INFO:teuthology.orchestra.run.smithi016.stdout: "from": 3, 2023-09-30T22:29:40.957 INFO:teuthology.orchestra.run.smithi016.stdout: "to": 2 2023-09-30T22:29:40.957 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-09-30T22:29:40.957 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2023-09-30T22:29:40.957 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-09-30T22:29:40.957 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-09-30T22:29:40.958 INFO:teuthology.orchestra.run.smithi016.stdout: "pgid": "4.19", 2023-09-30T22:29:40.958 INFO:teuthology.orchestra.run.smithi016.stdout: "mappings": [ 2023-09-30T22:29:40.958 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-09-30T22:29:40.958 INFO:teuthology.orchestra.run.smithi016.stdout: "from": 3, 2023-09-30T22:29:40.958 INFO:teuthology.orchestra.run.smithi016.stdout: "to": 0 2023-09-30T22:29:40.958 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-09-30T22:29:40.958 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2023-09-30T22:29:40.958 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-09-30T22:29:40.958 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-09-30T22:29:40.958 INFO:teuthology.orchestra.run.smithi016.stdout: "pgid": "4.3a", 2023-09-30T22:29:40.958 INFO:teuthology.orchestra.run.smithi016.stdout: "mappings": [ 2023-09-30T22:29:40.959 INFO:teuthology.orchestra.run.smithi016.stdout: { 2023-09-30T22:29:40.959 INFO:teuthology.orchestra.run.smithi016.stdout: "from": 3, 2023-09-30T22:29:40.959 INFO:teuthology.orchestra.run.smithi016.stdout: "to": 0 2023-09-30T22:29:40.959 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-09-30T22:29:40.959 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2023-09-30T22:29:40.959 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-09-30T22:29:40.959 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2023-09-30T22:29:40.959 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_upmap_primaries": [], 2023-09-30T22:29:40.959 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_temp": [], 2023-09-30T22:29:40.959 INFO:teuthology.orchestra.run.smithi016.stdout: "primary_temp": [], 2023-09-30T22:29:40.959 INFO:teuthology.orchestra.run.smithi016.stdout: "blocklist": { 2023-09-30T22:29:40.959 INFO:teuthology.orchestra.run.smithi016.stdout: "172.21.15.16:6817/1424366610": "2023-10-01T22:29:40.259230+0000", 2023-09-30T22:29:40.960 INFO:teuthology.orchestra.run.smithi016.stdout: "172.21.15.16:6816/118285": "2023-10-01T22:29:14.475978+0000", 2023-09-30T22:29:40.960 INFO:teuthology.orchestra.run.smithi016.stdout: "172.21.15.16:0/2249754484": "2023-10-01T22:29:14.475978+0000", 2023-09-30T22:29:40.960 INFO:teuthology.orchestra.run.smithi016.stdout: "172.21.15.16:0/1101648925": "2023-10-01T22:29:14.475978+0000" 2023-09-30T22:29:40.960 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-09-30T22:29:40.960 INFO:teuthology.orchestra.run.smithi016.stdout: "range_blocklist": {}, 2023-09-30T22:29:40.960 INFO:teuthology.orchestra.run.smithi016.stdout: "erasure_code_profiles": { 2023-09-30T22:29:40.960 INFO:teuthology.orchestra.run.smithi016.stdout: "default": { 2023-09-30T22:29:40.960 INFO:teuthology.orchestra.run.smithi016.stdout: "crush-failure-domain": "osd", 2023-09-30T22:29:40.960 INFO:teuthology.orchestra.run.smithi016.stdout: "k": "2", 2023-09-30T22:29:40.960 INFO:teuthology.orchestra.run.smithi016.stdout: "m": "1", 2023-09-30T22:29:40.960 INFO:teuthology.orchestra.run.smithi016.stdout: "plugin": "jerasure", 2023-09-30T22:29:40.961 INFO:teuthology.orchestra.run.smithi016.stdout: "technique": "reed_sol_van" 2023-09-30T22:29:40.961 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-09-30T22:29:40.961 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-09-30T22:29:40.961 INFO:teuthology.orchestra.run.smithi016.stdout: "removed_snaps_queue": [], 2023-09-30T22:29:40.961 INFO:teuthology.orchestra.run.smithi016.stdout: "new_removed_snaps": [], 2023-09-30T22:29:40.961 INFO:teuthology.orchestra.run.smithi016.stdout: "new_purged_snaps": [], 2023-09-30T22:29:40.961 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_node_flags": {}, 2023-09-30T22:29:40.961 INFO:teuthology.orchestra.run.smithi016.stdout: "device_class_flags": {}, 2023-09-30T22:29:40.961 INFO:teuthology.orchestra.run.smithi016.stdout: "stretch_mode": { 2023-09-30T22:29:40.961 INFO:teuthology.orchestra.run.smithi016.stdout: "stretch_mode_enabled": false, 2023-09-30T22:29:40.961 INFO:teuthology.orchestra.run.smithi016.stdout: "stretch_bucket_count": 0, 2023-09-30T22:29:40.962 INFO:teuthology.orchestra.run.smithi016.stdout: "degraded_stretch_mode": 0, 2023-09-30T22:29:40.962 INFO:teuthology.orchestra.run.smithi016.stdout: "recovering_stretch_mode": 0, 2023-09-30T22:29:40.962 INFO:teuthology.orchestra.run.smithi016.stdout: "stretch_mode_bucket": 0 2023-09-30T22:29:40.962 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-09-30T22:29:40.962 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-09-30T22:29:40.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.899+0000 7fd2bf7f0700 1 -- 172.21.15.16:0/2675460334 >> v1:172.21.15.16:6816/126811 conn(0x7fd29c02f510 legacy=0x7fd29c0319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:40.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.899+0000 7fd2bf7f0700 1 -- 172.21.15.16:0/2675460334 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd2b8133b00 msgr2=0x7fd2b811e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:40.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.899+0000 7fd2bf7f0700 1 --2- 172.21.15.16:0/2675460334 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd2b8133b00 0x7fd2b811e6d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd2b806b720 tx=0x7fd2a800ca30 comp rx=0 tx=0).stop 2023-09-30T22:29:40.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.899+0000 7fd2bf7f0700 1 -- 172.21.15.16:0/2675460334 shutdown_connections 2023-09-30T22:29:40.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.899+0000 7fd2bf7f0700 1 --2- 172.21.15.16:0/2675460334 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd2b8134410 0x7fd2b811ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:40.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.899+0000 7fd2bf7f0700 1 --2- 172.21.15.16:0/2675460334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2b809e060 0x7fd2b811e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:40.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.899+0000 7fd2bf7f0700 1 --2- 172.21.15.16:0/2675460334 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd2b8133b00 0x7fd2b811e6d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:40.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.899+0000 7fd2bf7f0700 1 -- 172.21.15.16:0/2675460334 >> 172.21.15.16:0/2675460334 conn(0x7fd2b80a2f50 msgr2=0x7fd2b8131c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:40.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.899+0000 7fd2bf7f0700 1 -- 172.21.15.16:0/2675460334 shutdown_connections 2023-09-30T22:29:40.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:40.899+0000 7fd2bf7f0700 1 -- 172.21.15.16:0/2675460334 wait complete. 2023-09-30T22:29:40.965 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph versions' 2023-09-30T22:29:41.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.099+0000 7fbf481f8700 1 Processor -- start 2023-09-30T22:29:41.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.100+0000 7fbf481f8700 1 -- start start 2023-09-30T22:29:41.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.100+0000 7fbf481f8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbf40131900 0x7fbf40131cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:41.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.100+0000 7fbf481f8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbf40132210 0x7fbf4009bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:41.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.100+0000 7fbf481f8700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf4009e400 con 0x7fbf4009c1a0 2023-09-30T22:29:41.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.100+0000 7fbf481f8700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf40131510 con 0x7fbf40132210 2023-09-30T22:29:41.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.100+0000 7fbf481f8700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbf40131700 con 0x7fbf40131900 2023-09-30T22:29:41.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.100+0000 7fbf46795700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fbf4009c1a0 0x7fbf40138860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52852/0 (socket says 172.21.15.16:52852) 2023-09-30T22:29:41.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.100+0000 7fbf46795700 1 -- 172.21.15.16:0/1311582286 learned_addr learned my addr 172.21.15.16:0/1311582286 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:29:41.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.100+0000 7fbf45793700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbf40132210 0x7fbf4009bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:41.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.100+0000 7fbf45f94700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbf40131900 0x7fbf40131cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:41.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.101+0000 7fbf44f92700 1 -- 172.21.15.16:0/1311582286 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3009835499 0 0) 0x7fbf4009e400 con 0x7fbf4009c1a0 2023-09-30T22:29:41.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.101+0000 7fbf45f94700 1 -- 172.21.15.16:0/1311582286 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbf40132210 msgr2=0x7fbf4009bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:41.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.101+0000 7fbf45f94700 1 --2- 172.21.15.16:0/1311582286 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbf40132210 0x7fbf4009bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:41.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.101+0000 7fbf45f94700 1 -- 172.21.15.16:0/1311582286 >> v1:172.21.15.16:6789/0 conn(0x7fbf4009c1a0 legacy=0x7fbf40138860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:41.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.101+0000 7fbf45f94700 1 -- 172.21.15.16:0/1311582286 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf4009e7c0 con 0x7fbf40131900 2023-09-30T22:29:41.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.101+0000 7fbf45793700 1 --2- 172.21.15.16:0/1311582286 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbf40132210 0x7fbf4009bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:41.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.101+0000 7fbf45f94700 1 --2- 172.21.15.16:0/1311582286 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbf40131900 0x7fbf40131cd0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c009870 tx=0x7fbf3c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aec830d3f6dd59c1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:41.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.101+0000 7fbf44f92700 1 -- 172.21.15.16:0/1311582286 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c011830 con 0x7fbf40131900 2023-09-30T22:29:41.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.101+0000 7fbf44f92700 1 -- 172.21.15.16:0/1311582286 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf3c011990 con 0x7fbf40131900 2023-09-30T22:29:41.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.101+0000 7fbf44f92700 1 -- 172.21.15.16:0/1311582286 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c011d20 con 0x7fbf40131900 2023-09-30T22:29:41.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.102+0000 7fbf481f8700 1 -- 172.21.15.16:0/1311582286 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbf40131900 msgr2=0x7fbf40131cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:41.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.102+0000 7fbf481f8700 1 --2- 172.21.15.16:0/1311582286 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbf40131900 0x7fbf40131cd0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c009870 tx=0x7fbf3c009b80 comp rx=0 tx=0).stop 2023-09-30T22:29:41.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.102+0000 7fbf481f8700 1 -- 172.21.15.16:0/1311582286 shutdown_connections 2023-09-30T22:29:41.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.102+0000 7fbf481f8700 1 --2- 172.21.15.16:0/1311582286 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbf40131900 0x7fbf40131cd0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:41.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.102+0000 7fbf481f8700 1 --2- 172.21.15.16:0/1311582286 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbf40132210 0x7fbf4009bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:41.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.102+0000 7fbf481f8700 1 -- 172.21.15.16:0/1311582286 >> 172.21.15.16:0/1311582286 conn(0x7fbf400a29e0 msgr2=0x7fbf400a2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:41.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.102+0000 7fbf481f8700 1 -- 172.21.15.16:0/1311582286 shutdown_connections 2023-09-30T22:29:41.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.102+0000 7fbf481f8700 1 -- 172.21.15.16:0/1311582286 wait complete. 2023-09-30T22:29:41.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.103+0000 7fbf481f8700 1 Processor -- start 2023-09-30T22:29:41.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.103+0000 7fbf481f8700 1 -- start start 2023-09-30T22:29:41.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.103+0000 7fbf481f8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf4009c1a0 0x7fbf4011b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:41.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.103+0000 7fbf481f8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbf40131900 0x7fbf4011bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:41.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.103+0000 7fbf481f8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbf40132210 0x7fbf4011c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:41.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.103+0000 7fbf481f8700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4011f370 con 0x7fbf4009c1a0 2023-09-30T22:29:41.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.103+0000 7fbf481f8700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4011fbb0 con 0x7fbf40131900 2023-09-30T22:29:41.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.103+0000 7fbf481f8700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4011d2e0 con 0x7fbf40132210 2023-09-30T22:29:41.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.103+0000 7fbf45f94700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf4009c1a0 0x7fbf4011b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:41.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.103+0000 7fbf45793700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbf40131900 0x7fbf4011bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:41.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.103+0000 7fbf45f94700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf4009c1a0 0x7fbf4011b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47070/0 (socket says 172.21.15.16:47070) 2023-09-30T22:29:41.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.103+0000 7fbf46795700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbf40132210 0x7fbf4011c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:41.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.103+0000 7fbf45f94700 1 -- 172.21.15.16:0/1045115168 learned_addr learned my addr 172.21.15.16:0/1045115168 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:41.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.104+0000 7fbf45f94700 1 -- 172.21.15.16:0/1045115168 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbf40132210 msgr2=0x7fbf4011c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:41.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.104+0000 7fbf45f94700 1 --2- 172.21.15.16:0/1045115168 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbf40132210 0x7fbf4011c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:41.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.104+0000 7fbf45f94700 1 -- 172.21.15.16:0/1045115168 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbf40131900 msgr2=0x7fbf4011bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:41.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.104+0000 7fbf45f94700 1 --2- 172.21.15.16:0/1045115168 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbf40131900 0x7fbf4011bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:41.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.104+0000 7fbf45f94700 1 -- 172.21.15.16:0/1045115168 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf300094a0 con 0x7fbf4009c1a0 2023-09-30T22:29:41.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.104+0000 7fbf45793700 1 --2- 172.21.15.16:0/1045115168 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbf40131900 0x7fbf4011bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:41.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.104+0000 7fbf45f94700 1 --2- 172.21.15.16:0/1045115168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf4009c1a0 0x7fbf4011b720 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c009840 tx=0x7fbf3c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:41.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.104+0000 7fbf367fc700 1 -- 172.21.15.16:0/1045115168 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c004340 con 0x7fbf4009c1a0 2023-09-30T22:29:41.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.104+0000 7fbf367fc700 1 -- 172.21.15.16:0/1045115168 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf3c01f070 con 0x7fbf4009c1a0 2023-09-30T22:29:41.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.104+0000 7fbf481f8700 1 -- 172.21.15.16:0/1045115168 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf3c0094a0 con 0x7fbf4009c1a0 2023-09-30T22:29:41.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.104+0000 7fbf367fc700 1 -- 172.21.15.16:0/1045115168 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c0036a0 con 0x7fbf4009c1a0 2023-09-30T22:29:41.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.104+0000 7fbf481f8700 1 -- 172.21.15.16:0/1045115168 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf4009e9e0 con 0x7fbf4009c1a0 2023-09-30T22:29:41.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.105+0000 7fbf367fc700 1 -- 172.21.15.16:0/1045115168 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fbf3c021030 con 0x7fbf4009c1a0 2023-09-30T22:29:41.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.105+0000 7fbf367fc700 1 -- 172.21.15.16:0/1045115168 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4092+0+0 (secure 0 0 0) 0x7fbf3c040e50 con 0x7fbf4009c1a0 2023-09-30T22:29:41.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.105+0000 7fbf481f8700 1 -- 172.21.15.16:0/1045115168 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf4004f2a0 con 0x7fbf4009c1a0 2023-09-30T22:29:41.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.110+0000 7fbf367fc700 1 -- 172.21.15.16:0/1045115168 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbf3c015080 con 0x7fbf4009c1a0 2023-09-30T22:29:41.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.401+0000 7fbf481f8700 1 -- 172.21.15.16:0/1045115168 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fbf400721f0 con 0x7fbf4009c1a0 2023-09-30T22:29:41.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.401+0000 7fbf367fc700 1 -- 172.21.15.16:0/1045115168 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+627 (secure 0 0 0) 0x7fbf3c0199a0 con 0x7fbf4009c1a0 2023-09-30T22:29:41.403 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2023-09-30T22:29:41.403 INFO:teuthology.orchestra.run.smithi016.stdout: "mon": { 2023-09-30T22:29:41.403 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.0-501-gfc1e90a3 (fc1e90a395a6a5631142eb70fd9bf3d22740a9da) reef (stable)": 3 2023-09-30T22:29:41.403 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-09-30T22:29:41.403 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr": { 2023-09-30T22:29:41.403 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.0-501-gfc1e90a3 (fc1e90a395a6a5631142eb70fd9bf3d22740a9da) reef (stable)": 1 2023-09-30T22:29:41.403 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-09-30T22:29:41.403 INFO:teuthology.orchestra.run.smithi016.stdout: "osd": { 2023-09-30T22:29:41.403 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.0-501-gfc1e90a3 (fc1e90a395a6a5631142eb70fd9bf3d22740a9da) reef (stable)": 4 2023-09-30T22:29:41.403 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-09-30T22:29:41.404 INFO:teuthology.orchestra.run.smithi016.stdout: "mds": { 2023-09-30T22:29:41.404 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.0-501-gfc1e90a3 (fc1e90a395a6a5631142eb70fd9bf3d22740a9da) reef (stable)": 1 2023-09-30T22:29:41.404 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2023-09-30T22:29:41.404 INFO:teuthology.orchestra.run.smithi016.stdout: "overall": { 2023-09-30T22:29:41.404 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph version 18.2.0-501-gfc1e90a3 (fc1e90a395a6a5631142eb70fd9bf3d22740a9da) reef (stable)": 9 2023-09-30T22:29:41.404 INFO:teuthology.orchestra.run.smithi016.stdout: } 2023-09-30T22:29:41.404 INFO:teuthology.orchestra.run.smithi016.stdout:} 2023-09-30T22:29:41.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.404+0000 7fbf481f8700 1 -- 172.21.15.16:0/1045115168 >> v1:172.21.15.16:6816/126811 conn(0x7fbf2402f510 legacy=0x7fbf240319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:41.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.404+0000 7fbf481f8700 1 -- 172.21.15.16:0/1045115168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf4009c1a0 msgr2=0x7fbf4011b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:41.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.404+0000 7fbf481f8700 1 --2- 172.21.15.16:0/1045115168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf4009c1a0 0x7fbf4011b720 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c009840 tx=0x7fbf3c004750 comp rx=0 tx=0).stop 2023-09-30T22:29:41.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.404+0000 7fbf481f8700 1 -- 172.21.15.16:0/1045115168 shutdown_connections 2023-09-30T22:29:41.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.404+0000 7fbf481f8700 1 --2- 172.21.15.16:0/1045115168 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbf40132210 0x7fbf4011c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:41.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.404+0000 7fbf481f8700 1 --2- 172.21.15.16:0/1045115168 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf4009c1a0 0x7fbf4011b720 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:41.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.404+0000 7fbf481f8700 1 --2- 172.21.15.16:0/1045115168 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbf40131900 0x7fbf4011bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:41.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.404+0000 7fbf481f8700 1 -- 172.21.15.16:0/1045115168 >> 172.21.15.16:0/1045115168 conn(0x7fbf400a29e0 msgr2=0x7fbf401b8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:41.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.404+0000 7fbf481f8700 1 -- 172.21.15.16:0/1045115168 shutdown_connections 2023-09-30T22:29:41.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.404+0000 7fbf481f8700 1 -- 172.21.15.16:0/1045115168 wait complete. 2023-09-30T22:29:41.417 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'for f in `ceph osd pool ls` ; do ceph osd pool set $f pg_autoscale_mode off ; done' 2023-09-30T22:29:41.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.552+0000 7f1c15f4d700 1 Processor -- start 2023-09-30T22:29:41.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.552+0000 7f1c15f4d700 1 -- start start 2023-09-30T22:29:41.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.553+0000 7f1c15f4d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1c1012d350 0x7f1c1012d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:41.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.553+0000 7f1c15f4d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1c1009a160 0x7f1c101387d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:41.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.553+0000 7f1c15f4d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c1009b4a0 con 0x7f1c1012dc60 2023-09-30T22:29:41.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.553+0000 7f1c15f4d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c1009c230 con 0x7f1c1012d350 2023-09-30T22:29:41.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.553+0000 7f1c15f4d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1009ce10 con 0x7f1c1009a160 2023-09-30T22:29:41.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.553+0000 7f1c0ffff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1c1009a160 0x7f1c101387d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:41.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.554+0000 7f1c0ffff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1c1009a160 0x7f1c101387d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:46000/0 (socket says 172.21.15.16:46000) 2023-09-30T22:29:41.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.554+0000 7f1c0ffff700 1 -- 172.21.15.16:0/4152052425 learned_addr learned my addr 172.21.15.16:0/4152052425 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:41.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.554+0000 7f1c0ffff700 1 -- 172.21.15.16:0/4152052425 >> v2:172.21.15.16:3300/0 conn(0x7f1c1012d350 msgr2=0x7f1c1012d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:41.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.554+0000 7f1c0ffff700 1 --2- 172.21.15.16:0/4152052425 >> v2:172.21.15.16:3300/0 conn(0x7f1c1012d350 0x7f1c1012d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:41.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.554+0000 7f1c0ffff700 1 -- 172.21.15.16:0/4152052425 >> v1:172.21.15.16:6789/0 conn(0x7f1c1012dc60 legacy=0x7f1c10099a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:41.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.554+0000 7f1c0ffff700 1 -- 172.21.15.16:0/4152052425 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c1009bc90 con 0x7f1c1009a160 2023-09-30T22:29:41.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.554+0000 7f1c0ffff700 1 --2- 172.21.15.16:0/4152052425 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1c1009a160 0x7f1c101387d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1c08009e50 tx=0x7f1c0800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=392fa57d6975ccbc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:41.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.555+0000 7f1c0e7fc700 1 -- 172.21.15.16:0/4152052425 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0800e070 con 0x7f1c1009a160 2023-09-30T22:29:41.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.555+0000 7f1c0e7fc700 1 -- 172.21.15.16:0/4152052425 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c0800b9c0 con 0x7f1c1009a160 2023-09-30T22:29:41.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.555+0000 7f1c0e7fc700 1 -- 172.21.15.16:0/4152052425 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0800bd70 con 0x7f1c1009a160 2023-09-30T22:29:41.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.556+0000 7f1c15f4d700 1 -- 172.21.15.16:0/4152052425 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1c1009a160 msgr2=0x7f1c101387d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:41.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.556+0000 7f1c15f4d700 1 --2- 172.21.15.16:0/4152052425 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1c1009a160 0x7f1c101387d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1c08009e50 tx=0x7f1c0800b330 comp rx=0 tx=0).stop 2023-09-30T22:29:41.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.556+0000 7f1c15f4d700 1 -- 172.21.15.16:0/4152052425 shutdown_connections 2023-09-30T22:29:41.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.556+0000 7f1c15f4d700 1 --2- 172.21.15.16:0/4152052425 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1c1009a160 0x7f1c101387d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:41.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.556+0000 7f1c15f4d700 1 --2- 172.21.15.16:0/4152052425 >> v2:172.21.15.16:3300/0 conn(0x7f1c1012d350 0x7f1c1012d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:41.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.556+0000 7f1c15f4d700 1 -- 172.21.15.16:0/4152052425 >> 172.21.15.16:0/4152052425 conn(0x7f1c100a29e0 msgr2=0x7f1c100a2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:41.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.556+0000 7f1c15f4d700 1 -- 172.21.15.16:0/4152052425 shutdown_connections 2023-09-30T22:29:41.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.556+0000 7f1c15f4d700 1 -- 172.21.15.16:0/4152052425 wait complete. 2023-09-30T22:29:41.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.557+0000 7f1c15f4d700 1 Processor -- start 2023-09-30T22:29:41.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.557+0000 7f1c15f4d700 1 -- start start 2023-09-30T22:29:41.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.558+0000 7f1c15f4d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c1009a160 0x7f1c1011cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:41.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.558+0000 7f1c15f4d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1c1012d350 0x7f1c1011d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:41.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.558+0000 7f1c15f4d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1c1012dc60 0x7f1c1011d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:41.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.558+0000 7f1c15f4d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c100a30d0 con 0x7f1c1009a160 2023-09-30T22:29:41.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.558+0000 7f1c15f4d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1011e920 con 0x7f1c1012d350 2023-09-30T22:29:41.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.558+0000 7f1c15f4d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1c10120180 con 0x7f1c1012dc60 2023-09-30T22:29:41.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.558+0000 7f1c0f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c1009a160 0x7f1c1011cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:41.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.558+0000 7f1c0f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c1009a160 0x7f1c1011cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47084/0 (socket says 172.21.15.16:47084) 2023-09-30T22:29:41.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.558+0000 7f1c0f7fe700 1 -- 172.21.15.16:0/109812630 learned_addr learned my addr 172.21.15.16:0/109812630 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:41.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.558+0000 7f1c0ffff700 1 --2- 172.21.15.16:0/109812630 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1c1012dc60 0x7f1c1011d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:41.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.558+0000 7f1c0effd700 1 --2- 172.21.15.16:0/109812630 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1c1012d350 0x7f1c1011d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:41.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.559+0000 7f1c0ffff700 1 -- 172.21.15.16:0/109812630 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1c1012d350 msgr2=0x7f1c1011d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:41.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.559+0000 7f1c0ffff700 1 --2- 172.21.15.16:0/109812630 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1c1012d350 0x7f1c1011d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:41.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.559+0000 7f1c0ffff700 1 -- 172.21.15.16:0/109812630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c1009a160 msgr2=0x7f1c1011cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:41.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.559+0000 7f1c0ffff700 1 --2- 172.21.15.16:0/109812630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c1009a160 0x7f1c1011cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:41.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.559+0000 7f1c0ffff700 1 -- 172.21.15.16:0/109812630 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c08009b30 con 0x7f1c1012dc60 2023-09-30T22:29:41.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.559+0000 7f1c0f7fe700 1 --2- 172.21.15.16:0/109812630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c1009a160 0x7f1c1011cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:41.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.559+0000 7f1c0ffff700 1 --2- 172.21.15.16:0/109812630 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1c1012dc60 0x7f1c1011d680 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1c08015000 tx=0x7f1c08004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:41.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.559+0000 7f1bf7fff700 1 -- 172.21.15.16:0/109812630 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c08004680 con 0x7f1c1012dc60 2023-09-30T22:29:41.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.559+0000 7f1bf7fff700 1 -- 172.21.15.16:0/109812630 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c08004800 con 0x7f1c1012dc60 2023-09-30T22:29:41.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.559+0000 7f1c15f4d700 1 -- 172.21.15.16:0/109812630 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c1009b670 con 0x7f1c1012dc60 2023-09-30T22:29:41.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.559+0000 7f1bf7fff700 1 -- 172.21.15.16:0/109812630 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c08004970 con 0x7f1c1012dc60 2023-09-30T22:29:41.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.559+0000 7f1c15f4d700 1 -- 172.21.15.16:0/109812630 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c10120eb0 con 0x7f1c1012dc60 2023-09-30T22:29:41.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.560+0000 7f1bf7fff700 1 -- 172.21.15.16:0/109812630 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f1c08024070 con 0x7f1c1012dc60 2023-09-30T22:29:41.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.560+0000 7f1bf7fff700 1 -- 172.21.15.16:0/109812630 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4092+0+0 (secure 0 0 0) 0x7f1c08044ff0 con 0x7f1c1012dc60 2023-09-30T22:29:41.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.560+0000 7f1c15f4d700 1 -- 172.21.15.16:0/109812630 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c1004f2a0 con 0x7f1c1012dc60 2023-09-30T22:29:41.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.566+0000 7f1bf7fff700 1 -- 172.21.15.16:0/109812630 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1c0802c020 con 0x7f1c1012dc60 2023-09-30T22:29:41.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.767+0000 7f1c15f4d700 1 -- 172.21.15.16:0/109812630 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd pool ls"} v 0) v1 -- 0x7f1c100721f0 con 0x7f1c1012dc60 2023-09-30T22:29:41.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.768+0000 7f1bf7fff700 1 -- 172.21.15.16:0/109812630 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool ls"}]=0 v34) v1 ==== 59+0+37 (secure 0 0 0) 0x7f1c08003e10 con 0x7f1c1012dc60 2023-09-30T22:29:41.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.770+0000 7f1c15f4d700 1 -- 172.21.15.16:0/109812630 >> v1:172.21.15.16:6816/126811 conn(0x7f1bfc02f540 legacy=0x7f1bfc0319f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:41.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.770+0000 7f1c15f4d700 1 -- 172.21.15.16:0/109812630 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1c1012dc60 msgr2=0x7f1c1011d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:41.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.770+0000 7f1c15f4d700 1 --2- 172.21.15.16:0/109812630 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1c1012dc60 0x7f1c1011d680 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1c08015000 tx=0x7f1c08004ad0 comp rx=0 tx=0).stop 2023-09-30T22:29:41.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.770+0000 7f1c15f4d700 1 -- 172.21.15.16:0/109812630 shutdown_connections 2023-09-30T22:29:41.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.770+0000 7f1c15f4d700 1 --2- 172.21.15.16:0/109812630 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1c1012dc60 0x7f1c1011d680 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:41.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.770+0000 7f1c15f4d700 1 --2- 172.21.15.16:0/109812630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c1009a160 0x7f1c1011cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:41.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.770+0000 7f1c15f4d700 1 --2- 172.21.15.16:0/109812630 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1c1012d350 0x7f1c1011d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:41.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.770+0000 7f1c15f4d700 1 -- 172.21.15.16:0/109812630 >> 172.21.15.16:0/109812630 conn(0x7f1c100a29e0 msgr2=0x7f1c1009d480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:41.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.771+0000 7f1c15f4d700 1 -- 172.21.15.16:0/109812630 shutdown_connections 2023-09-30T22:29:41.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.771+0000 7f1c15f4d700 1 -- 172.21.15.16:0/109812630 wait complete. 2023-09-30T22:29:41.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.890+0000 7f53594e5700 1 Processor -- start 2023-09-30T22:29:41.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.890+0000 7f53594e5700 1 -- start start 2023-09-30T22:29:41.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.891+0000 7f53594e5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f535409de00 0x7f535409feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:41.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.891+0000 7f53594e5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f535409e1d0 0x7f535409e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:41.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.891+0000 7f53594e5700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5354133960 0x7f5354133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:41.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.891+0000 7f53594e5700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f535409f000 con 0x7f5354133960 2023-09-30T22:29:41.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.891+0000 7f53594e5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53540a0500 con 0x7f535409de00 2023-09-30T22:29:41.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.891+0000 7f53594e5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f535409f380 con 0x7f535409e1d0 2023-09-30T22:29:41.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.891+0000 7f53537fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5354133960 0x7f5354133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:41.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.891+0000 7f5352ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f535409de00 0x7f535409feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:41.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.891+0000 7f53527fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f535409e1d0 0x7f535409e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:41.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.891+0000 7f5352ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f535409de00 0x7f535409feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:44996/0 (socket says 172.21.15.16:44996) 2023-09-30T22:29:41.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.891+0000 7f53537fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5354133960 0x7f5354133dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47090/0 (socket says 172.21.15.16:47090) 2023-09-30T22:29:41.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.891+0000 7f5352ffd700 1 -- 172.21.15.16:0/2196714759 learned_addr learned my addr 172.21.15.16:0/2196714759 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:41.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.891+0000 7f53537fe700 1 -- 172.21.15.16:0/2196714759 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f535409e1d0 msgr2=0x7f535409e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:41.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.891+0000 7f53537fe700 1 --2- 172.21.15.16:0/2196714759 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f535409e1d0 0x7f535409e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:41.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.892+0000 7f53537fe700 1 -- 172.21.15.16:0/2196714759 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f535409de00 msgr2=0x7f535409feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:41.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.892+0000 7f53537fe700 1 --2- 172.21.15.16:0/2196714759 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f535409de00 0x7f535409feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:41.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.892+0000 7f53537fe700 1 -- 172.21.15.16:0/2196714759 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f535409d9d0 con 0x7f5354133960 2023-09-30T22:29:41.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.892+0000 7f5352ffd700 1 --2- 172.21.15.16:0/2196714759 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f535409de00 0x7f535409feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:41.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.892+0000 7f53527fc700 1 --2- 172.21.15.16:0/2196714759 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f535409e1d0 0x7f535409e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:41.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.892+0000 7f53537fe700 1 --2- 172.21.15.16:0/2196714759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5354133960 0x7f5354133dd0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f534c009d90 tx=0x7f534c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cd17369a29c10dc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:41.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.892+0000 7f5351ffb700 1 -- 172.21.15.16:0/2196714759 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534c00e070 con 0x7f5354133960 2023-09-30T22:29:41.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.892+0000 7f5351ffb700 1 -- 172.21.15.16:0/2196714759 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f534c00bb20 con 0x7f5354133960 2023-09-30T22:29:41.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.892+0000 7f5351ffb700 1 -- 172.21.15.16:0/2196714759 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534c012430 con 0x7f5354133960 2023-09-30T22:29:41.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.892+0000 7f53594e5700 1 -- 172.21.15.16:0/2196714759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5354133960 msgr2=0x7f5354133dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:41.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.892+0000 7f53594e5700 1 --2- 172.21.15.16:0/2196714759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5354133960 0x7f5354133dd0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f534c009d90 tx=0x7f534c00b330 comp rx=0 tx=0).stop 2023-09-30T22:29:41.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.893+0000 7f53594e5700 1 -- 172.21.15.16:0/2196714759 shutdown_connections 2023-09-30T22:29:41.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.893+0000 7f53594e5700 1 --2- 172.21.15.16:0/2196714759 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f535409de00 0x7f535409feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:41.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.893+0000 7f53594e5700 1 --2- 172.21.15.16:0/2196714759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5354133960 0x7f5354133dd0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:41.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.893+0000 7f53594e5700 1 --2- 172.21.15.16:0/2196714759 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f535409e1d0 0x7f535409e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:41.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.893+0000 7f53594e5700 1 -- 172.21.15.16:0/2196714759 >> 172.21.15.16:0/2196714759 conn(0x7f53540a2f30 msgr2=0x7f53540a3330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:41.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.893+0000 7f53594e5700 1 -- 172.21.15.16:0/2196714759 shutdown_connections 2023-09-30T22:29:41.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.893+0000 7f53594e5700 1 -- 172.21.15.16:0/2196714759 wait complete. 2023-09-30T22:29:41.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.894+0000 7f53594e5700 1 Processor -- start 2023-09-30T22:29:41.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.894+0000 7f53594e5700 1 -- start start 2023-09-30T22:29:41.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.894+0000 7f53594e5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f535409de00 0x7f53541c8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:41.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.894+0000 7f53594e5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f535409e1d0 0x7f53541c8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:41.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.894+0000 7f53594e5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5354133960 0x7f53541c4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:41.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.894+0000 7f53594e5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5354071b50 con 0x7f535409e1d0 2023-09-30T22:29:41.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.894+0000 7f53594e5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f535412c6f0 con 0x7f535409de00 2023-09-30T22:29:41.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.894+0000 7f53594e5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f535412d2d0 con 0x7f5354133960 2023-09-30T22:29:41.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.894+0000 7f53527fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f535409e1d0 0x7f53541c8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:41.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.894+0000 7f53537fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5354133960 0x7f53541c4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:41.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.895+0000 7f53527fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f535409e1d0 0x7f53541c8bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47094/0 (socket says 172.21.15.16:47094) 2023-09-30T22:29:41.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.895+0000 7f53527fc700 1 -- 172.21.15.16:0/1527178357 learned_addr learned my addr 172.21.15.16:0/1527178357 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:41.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.895+0000 7f5352ffd700 1 --2- 172.21.15.16:0/1527178357 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f535409de00 0x7f53541c8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:41.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.895+0000 7f53537fe700 1 -- 172.21.15.16:0/1527178357 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f535409de00 msgr2=0x7f53541c8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:41.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.895+0000 7f53537fe700 1 --2- 172.21.15.16:0/1527178357 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f535409de00 0x7f53541c8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:41.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.895+0000 7f53537fe700 1 -- 172.21.15.16:0/1527178357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f535409e1d0 msgr2=0x7f53541c8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:41.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.895+0000 7f53537fe700 1 --2- 172.21.15.16:0/1527178357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f535409e1d0 0x7f53541c8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:41.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.895+0000 7f53537fe700 1 -- 172.21.15.16:0/1527178357 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5344009550 con 0x7f5354133960 2023-09-30T22:29:41.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.895+0000 7f53527fc700 1 --2- 172.21.15.16:0/1527178357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f535409e1d0 0x7f53541c8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:41.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.895+0000 7f53537fe700 1 --2- 172.21.15.16:0/1527178357 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5354133960 0x7f53541c4c30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f534c016000 tx=0x7f534c004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:41.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.895+0000 7f534b7fe700 1 -- 172.21.15.16:0/1527178357 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534c0048b0 con 0x7f5354133960 2023-09-30T22:29:41.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.895+0000 7f534b7fe700 1 -- 172.21.15.16:0/1527178357 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f534c020070 con 0x7f5354133960 2023-09-30T22:29:41.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.895+0000 7f53594e5700 1 -- 172.21.15.16:0/1527178357 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f533c0094a0 con 0x7f5354133960 2023-09-30T22:29:41.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.895+0000 7f534b7fe700 1 -- 172.21.15.16:0/1527178357 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534c0276c0 con 0x7f5354133960 2023-09-30T22:29:41.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.895+0000 7f53594e5700 1 -- 172.21.15.16:0/1527178357 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f534c009a70 con 0x7f5354133960 2023-09-30T22:29:41.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.896+0000 7f534b7fe700 1 -- 172.21.15.16:0/1527178357 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f534c012430 con 0x7f5354133960 2023-09-30T22:29:41.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.896+0000 7f53594e5700 1 -- 172.21.15.16:0/1527178357 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f535406baa0 con 0x7f5354133960 2023-09-30T22:29:41.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.896+0000 7f534b7fe700 1 -- 172.21.15.16:0/1527178357 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4092+0+0 (secure 0 0 0) 0x7f534c022070 con 0x7f5354133960 2023-09-30T22:29:41.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:41.901+0000 7f534b7fe700 1 -- 172.21.15.16:0/1527178357 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f534c01c400 con 0x7f5354133960 2023-09-30T22:29:42.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.104+0000 7f53594e5700 1 -- 172.21.15.16:0/1527178357 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd pool set", "pool": ".mgr", "var": "pg_autoscale_mode", "val": "off"} v 0) v1 -- 0x7f5354067ae0 con 0x7f5354133960 2023-09-30T22:29:42.277 INFO:tasks.ceph.mgr.x.smithi016.stderr:terminate called recursively 2023-09-30T22:29:42.277 INFO:tasks.ceph.mgr.x.smithi016.stderr:terminate called after throwing an instance of 'ceph::buffer::v15_2_0::end_of_buffer' 2023-09-30T22:29:42.277 INFO:tasks.ceph.mgr.x.smithi016.stderr: what(): End of buffer [buffer:2] 2023-09-30T22:29:42.277 INFO:tasks.ceph.mgr.x.smithi016.stderr:*** Caught signal (Aborted) ** 2023-09-30T22:29:42.277 INFO:tasks.ceph.mgr.x.smithi016.stderr: in thread 7f9034489700 thread_name:ms_dispatch 2023-09-30T22:29:42.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.538+0000 7f5352ffd700 1 -- 172.21.15.16:0/1527178357 >> v1:172.21.15.16:6816/126811 conn(0x7f533402f510 legacy=0x7f53340319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 20 2023-09-30T22:29:42.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.538+0000 7f5352ffd700 1 -- 172.21.15.16:0/1527178357 >> v1:172.21.15.16:6816/126811 conn(0x7f533402f510 legacy=0x7f53340319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-09-30T22:29:42.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.538+0000 7f5352ffd700 1 --1- 172.21.15.16:0/1527178357 >> v1:172.21.15.16:6816/126811 conn(0x7f533402f510 0x7f53340319c0 :-1 s=OPENED pgs=56 cs=1 l=1).handle_message read tag failed 2023-09-30T22:29:42.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.538+0000 7f5352ffd700 1 --1- 172.21.15.16:0/1527178357 >> v1:172.21.15.16:6816/126811 conn(0x7f533402f510 0x7f53340319c0 :-1 s=OPENED pgs=56 cs=1 l=1).fault on lossy channel, failing 2023-09-30T22:29:42.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.539+0000 7f534b7fe700 1 -- 172.21.15.16:0/1527178357 >> v1:172.21.15.16:6816/126811 conn(0x7f533402f510 legacy=0x7f53340319c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2023-09-30T22:29:42.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.558+0000 7f534b7fe700 1 -- 172.21.15.16:0/1527178357 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_autoscale_mode", "val": "off"}]=0 set pool 1 pg_autoscale_mode to off v35) v1 ==== 153+0+0 (secure 0 0 0) 0x7f534c03c020 con 0x7f5354133960 2023-09-30T22:29:42.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.560+0000 7f53594e5700 1 -- 172.21.15.16:0/1527178357 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5354133960 msgr2=0x7f53541c4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:42.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.560+0000 7f53594e5700 1 --2- 172.21.15.16:0/1527178357 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5354133960 0x7f53541c4c30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f534c016000 tx=0x7f534c004ad0 comp rx=0 tx=0).stop 2023-09-30T22:29:42.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.561+0000 7f53594e5700 1 -- 172.21.15.16:0/1527178357 shutdown_connections 2023-09-30T22:29:42.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.561+0000 7f53594e5700 1 --2- 172.21.15.16:0/1527178357 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f535409de00 0x7f53541c8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:42.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.561+0000 7f53594e5700 1 --2- 172.21.15.16:0/1527178357 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5354133960 0x7f53541c4c30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:42.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.561+0000 7f53594e5700 1 --2- 172.21.15.16:0/1527178357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f535409e1d0 0x7f53541c8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:42.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.561+0000 7f53594e5700 1 -- 172.21.15.16:0/1527178357 >> 172.21.15.16:0/1527178357 conn(0x7f53540a2f30 msgr2=0x7f535409c960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:42.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.561+0000 7f53594e5700 1 -- 172.21.15.16:0/1527178357 shutdown_connections 2023-09-30T22:29:42.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.561+0000 7f53594e5700 1 -- 172.21.15.16:0/1527178357 wait complete. 2023-09-30T22:29:42.562 INFO:teuthology.orchestra.run.smithi016.stderr:set pool 1 pg_autoscale_mode to off 2023-09-30T22:29:42.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.682+0000 7f88aade6700 1 Processor -- start 2023-09-30T22:29:42.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.682+0000 7f88aade6700 1 -- start start 2023-09-30T22:29:42.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.682+0000 7f88aade6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f88a409e9d0 0x7f88a409eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:42.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.682+0000 7f88aade6700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f88a41342b0 0x7f88a413be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:42.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.682+0000 7f88aade6700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88a406daf0 con 0x7f88a409f2e0 2023-09-30T22:29:42.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.682+0000 7f88aade6700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88a409ae80 con 0x7f88a41342b0 2023-09-30T22:29:42.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.682+0000 7f88aade6700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f88a409fa50 con 0x7f88a409e9d0 2023-09-30T22:29:42.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.683+0000 7f88a8b82700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f88a409e9d0 0x7f88a409eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:42.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.683+0000 7f88a9383700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f88a41342b0 0x7f88a413be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:42.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.683+0000 7f88a8b82700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f88a409e9d0 0x7f88a409eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:46040/0 (socket says 172.21.15.16:46040) 2023-09-30T22:29:42.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.683+0000 7f88a9383700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f88a41342b0 0x7f88a413be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47096/0 (socket says 172.21.15.16:47096) 2023-09-30T22:29:42.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.683+0000 7f88a8b82700 1 -- 172.21.15.16:0/3216210073 learned_addr learned my addr 172.21.15.16:0/3216210073 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:42.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.683+0000 7f88a3fff700 1 --1- 172.21.15.16:0/3216210073 >> v1:172.21.15.16:6789/0 conn(0x7f88a409f2e0 0x7f88a4133ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52872/0 (socket says 172.21.15.16:52872) 2023-09-30T22:29:42.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.683+0000 7f88a9383700 1 -- 172.21.15.16:0/3216210073 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f88a409e9d0 msgr2=0x7f88a409eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:42.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.683+0000 7f88a9383700 1 --2- 172.21.15.16:0/3216210073 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f88a409e9d0 0x7f88a409eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:42.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.683+0000 7f88a9383700 1 -- 172.21.15.16:0/3216210073 >> v1:172.21.15.16:6789/0 conn(0x7f88a409f2e0 legacy=0x7f88a4133ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:42.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.683+0000 7f88a9383700 1 -- 172.21.15.16:0/3216210073 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88a406cba0 con 0x7f88a41342b0 2023-09-30T22:29:42.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.683+0000 7f88a37fe700 1 -- 172.21.15.16:0/3216210073 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1264282807 0 0) 0x7f88a406daf0 con 0x7f88a409f2e0 2023-09-30T22:29:42.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.683+0000 7f88a8b82700 1 --2- 172.21.15.16:0/3216210073 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f88a409e9d0 0x7f88a409eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:42.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.683+0000 7f88a9383700 1 --2- 172.21.15.16:0/3216210073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88a41342b0 0x7f88a413be00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f889c009d90 tx=0x7f889c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccb8ba5db4c9ec3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:42.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.683+0000 7f88a37fe700 1 -- 172.21.15.16:0/3216210073 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889c00e070 con 0x7f88a41342b0 2023-09-30T22:29:42.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.683+0000 7f88a37fe700 1 -- 172.21.15.16:0/3216210073 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f889c00bb20 con 0x7f88a41342b0 2023-09-30T22:29:42.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.683+0000 7f88a37fe700 1 -- 172.21.15.16:0/3216210073 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889c012430 con 0x7f88a41342b0 2023-09-30T22:29:42.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.684+0000 7f88aade6700 1 -- 172.21.15.16:0/3216210073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88a41342b0 msgr2=0x7f88a413be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:42.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.684+0000 7f88aade6700 1 --2- 172.21.15.16:0/3216210073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88a41342b0 0x7f88a413be00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f889c009d90 tx=0x7f889c00b330 comp rx=0 tx=0).stop 2023-09-30T22:29:42.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.684+0000 7f88aade6700 1 -- 172.21.15.16:0/3216210073 shutdown_connections 2023-09-30T22:29:42.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.684+0000 7f88aade6700 1 --2- 172.21.15.16:0/3216210073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88a41342b0 0x7f88a413be00 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:42.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.684+0000 7f88aade6700 1 --2- 172.21.15.16:0/3216210073 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f88a409e9d0 0x7f88a409eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:42.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.684+0000 7f88aade6700 1 -- 172.21.15.16:0/3216210073 >> 172.21.15.16:0/3216210073 conn(0x7f88a40a43c0 msgr2=0x7f88a40a47c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:42.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.684+0000 7f88aade6700 1 -- 172.21.15.16:0/3216210073 shutdown_connections 2023-09-30T22:29:42.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.684+0000 7f88aade6700 1 -- 172.21.15.16:0/3216210073 wait complete. 2023-09-30T22:29:42.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.685+0000 7f88aade6700 1 Processor -- start 2023-09-30T22:29:42.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.685+0000 7f88aade6700 1 -- start start 2023-09-30T22:29:42.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.686+0000 7f88aade6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f88a409e9d0 0x7f88a41227c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:42.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.686+0000 7f88aade6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f88a409f2e0 0x7f88a4122d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:42.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.686+0000 7f88aade6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88a41342b0 0x7f88a4123240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:42.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.686+0000 7f88aade6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88a4125f30 con 0x7f88a41342b0 2023-09-30T22:29:42.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.686+0000 7f88aade6700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88a40a32c0 con 0x7f88a409e9d0 2023-09-30T22:29:42.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.686+0000 7f88aade6700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f88a4125c70 con 0x7f88a409f2e0 2023-09-30T22:29:42.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.686+0000 7f88a3fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f88a409f2e0 0x7f88a4122d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:42.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.686+0000 7f88a9383700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88a41342b0 0x7f88a4123240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:42.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.686+0000 7f88a3fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f88a409f2e0 0x7f88a4122d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:46048/0 (socket says 172.21.15.16:46048) 2023-09-30T22:29:42.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.686+0000 7f88a9383700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88a41342b0 0x7f88a4123240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47112/0 (socket says 172.21.15.16:47112) 2023-09-30T22:29:42.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.686+0000 7f88a3fff700 1 -- 172.21.15.16:0/2982452078 learned_addr learned my addr 172.21.15.16:0/2982452078 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:42.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.686+0000 7f88a8b82700 1 --2- 172.21.15.16:0/2982452078 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f88a409e9d0 0x7f88a41227c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:42.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.686+0000 7f88a3fff700 1 -- 172.21.15.16:0/2982452078 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f88a409e9d0 msgr2=0x7f88a41227c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:42.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.686+0000 7f88a3fff700 1 --2- 172.21.15.16:0/2982452078 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f88a409e9d0 0x7f88a41227c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:42.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.686+0000 7f88a3fff700 1 -- 172.21.15.16:0/2982452078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88a41342b0 msgr2=0x7f88a4123240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:42.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.686+0000 7f88a3fff700 1 --2- 172.21.15.16:0/2982452078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88a41342b0 0x7f88a4123240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:42.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.686+0000 7f88a3fff700 1 -- 172.21.15.16:0/2982452078 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8894009550 con 0x7f88a409f2e0 2023-09-30T22:29:42.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.686+0000 7f88a9383700 1 --2- 172.21.15.16:0/2982452078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88a41342b0 0x7f88a4123240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:42.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.686+0000 7f88a8b82700 1 --2- 172.21.15.16:0/2982452078 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f88a409e9d0 0x7f88a41227c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:42.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.687+0000 7f88a3fff700 1 --2- 172.21.15.16:0/2982452078 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f88a409f2e0 0x7f88a4122d00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f888c001eb0 tx=0x7f888c00c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:42.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.687+0000 7f88a17fa700 1 -- 172.21.15.16:0/2982452078 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888c00cea0 con 0x7f88a409f2e0 2023-09-30T22:29:42.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.687+0000 7f88a17fa700 1 -- 172.21.15.16:0/2982452078 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f888c004c80 con 0x7f88a409f2e0 2023-09-30T22:29:42.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.687+0000 7f88aade6700 1 -- 172.21.15.16:0/2982452078 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f889c009a70 con 0x7f88a409f2e0 2023-09-30T22:29:42.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.687+0000 7f88a17fa700 1 -- 172.21.15.16:0/2982452078 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888c0070d0 con 0x7f88a409f2e0 2023-09-30T22:29:42.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.687+0000 7f88aade6700 1 -- 172.21.15.16:0/2982452078 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88a4124290 con 0x7f88a409f2e0 2023-09-30T22:29:42.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.688+0000 7f88a17fa700 1 -- 172.21.15.16:0/2982452078 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f888c007230 con 0x7f88a409f2e0 2023-09-30T22:29:42.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.688+0000 7f88aade6700 1 -- 172.21.15.16:0/2982452078 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88a404f2a0 con 0x7f88a409f2e0 2023-09-30T22:29:42.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.688+0000 7f88a17fa700 1 -- 172.21.15.16:0/2982452078 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4092+0+0 (secure 0 0 0) 0x7f888c03f8c0 con 0x7f88a409f2e0 2023-09-30T22:29:42.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.688+0000 7f88a8b82700 1 -- 172.21.15.16:0/2982452078 >> v1:172.21.15.16:6816/126811 conn(0x7f888402f510 legacy=0x7f88840319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:42.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.693+0000 7f88a17fa700 1 -- 172.21.15.16:0/2982452078 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f888c01bda0 con 0x7f88a409f2e0 2023-09-30T22:29:42.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.889+0000 7f88a8b82700 1 -- 172.21.15.16:0/2982452078 >> v1:172.21.15.16:6816/126811 conn(0x7f888402f510 legacy=0x7f88840319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:42.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:42.900+0000 7f88aade6700 1 -- 172.21.15.16:0/2982452078 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd pool set", "pool": "rbd", "var": "pg_autoscale_mode", "val": "off"} v 0) v1 -- 0x7f88a41269a0 con 0x7f88a409f2e0 2023-09-30T22:29:43.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.290+0000 7f88a8b82700 1 -- 172.21.15.16:0/2982452078 >> v1:172.21.15.16:6816/126811 conn(0x7f888402f510 legacy=0x7f88840319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:43.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.568+0000 7f88a17fa700 1 -- 172.21.15.16:0/2982452078 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "rbd", "var": "pg_autoscale_mode", "val": "off"}]=0 set pool 2 pg_autoscale_mode to off v36) v1 ==== 152+0+0 (secure 0 0 0) 0x7f888c01b340 con 0x7f88a409f2e0 2023-09-30T22:29:43.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.572+0000 7f88aade6700 1 -- 172.21.15.16:0/2982452078 >> v1:172.21.15.16:6816/126811 conn(0x7f888402f510 legacy=0x7f88840319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:43.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.572+0000 7f88aade6700 1 -- 172.21.15.16:0/2982452078 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f88a409f2e0 msgr2=0x7f88a4122d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:43.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.572+0000 7f88aade6700 1 --2- 172.21.15.16:0/2982452078 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f88a409f2e0 0x7f88a4122d00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f888c001eb0 tx=0x7f888c00c800 comp rx=0 tx=0).stop 2023-09-30T22:29:43.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.572+0000 7f88aade6700 1 -- 172.21.15.16:0/2982452078 shutdown_connections 2023-09-30T22:29:43.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.572+0000 7f88aade6700 1 --2- 172.21.15.16:0/2982452078 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f88a409e9d0 0x7f88a41227c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:43.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.572+0000 7f88aade6700 1 --2- 172.21.15.16:0/2982452078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88a41342b0 0x7f88a4123240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:43.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.572+0000 7f88aade6700 1 --2- 172.21.15.16:0/2982452078 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f88a409f2e0 0x7f88a4122d00 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:43.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.572+0000 7f88aade6700 1 -- 172.21.15.16:0/2982452078 >> 172.21.15.16:0/2982452078 conn(0x7f88a40a43c0 msgr2=0x7f88a409c600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:43.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.572+0000 7f88aade6700 1 -- 172.21.15.16:0/2982452078 shutdown_connections 2023-09-30T22:29:43.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.572+0000 7f88aade6700 1 -- 172.21.15.16:0/2982452078 wait complete. 2023-09-30T22:29:43.573 INFO:teuthology.orchestra.run.smithi016.stderr:set pool 2 pg_autoscale_mode to off 2023-09-30T22:29:43.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.698+0000 7fef87d80700 1 Processor -- start 2023-09-30T22:29:43.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.698+0000 7fef87d80700 1 -- start start 2023-09-30T22:29:43.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.698+0000 7fef87d80700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fef8009e410 0x7fef80138e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:43.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.698+0000 7fef87d80700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fef80139370 0x7fef8013b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:43.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.698+0000 7fef87d80700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef8006b640 con 0x7fef8009d930 2023-09-30T22:29:43.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.698+0000 7fef87d80700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef8009a790 con 0x7fef80139370 2023-09-30T22:29:43.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.698+0000 7fef87d80700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef8009ef00 con 0x7fef8009e410 2023-09-30T22:29:43.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.699+0000 7fef8531b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fef8009e410 0x7fef80138e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:43.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.699+0000 7fef8531b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fef8009e410 0x7fef80138e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:45006/0 (socket says 172.21.15.16:45006) 2023-09-30T22:29:43.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.699+0000 7fef8531b700 1 -- 172.21.15.16:0/2045389734 learned_addr learned my addr 172.21.15.16:0/2045389734 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:43.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.699+0000 7fef8631d700 1 --2- 172.21.15.16:0/2045389734 >> v2:172.21.15.16:3300/0 conn(0x7fef80139370 0x7fef8013b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:43.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.699+0000 7fef8631d700 1 -- 172.21.15.16:0/2045389734 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fef8009e410 msgr2=0x7fef80138e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:43.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.699+0000 7fef8631d700 1 --2- 172.21.15.16:0/2045389734 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fef8009e410 0x7fef80138e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:43.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.699+0000 7fef8631d700 1 -- 172.21.15.16:0/2045389734 >> v1:172.21.15.16:6789/0 conn(0x7fef8009d930 legacy=0x7fef8009dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:43.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.699+0000 7fef8631d700 1 -- 172.21.15.16:0/2045389734 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef8009f1a0 con 0x7fef80139370 2023-09-30T22:29:43.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.699+0000 7fef84b1a700 1 -- 172.21.15.16:0/2045389734 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1473654173 0 0) 0x7fef8006b640 con 0x7fef8009d930 2023-09-30T22:29:43.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.699+0000 7fef8531b700 1 --2- 172.21.15.16:0/2045389734 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fef8009e410 0x7fef80138e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:43.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.699+0000 7fef8631d700 1 --2- 172.21.15.16:0/2045389734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef80139370 0x7fef8013b750 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fef78009d90 tx=0x7fef7800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=142e0c45659772f1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:43.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.699+0000 7fef84b1a700 1 -- 172.21.15.16:0/2045389734 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7800e070 con 0x7fef80139370 2023-09-30T22:29:43.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.699+0000 7fef84b1a700 1 -- 172.21.15.16:0/2045389734 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef7800bb20 con 0x7fef80139370 2023-09-30T22:29:43.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.699+0000 7fef84b1a700 1 -- 172.21.15.16:0/2045389734 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef78012430 con 0x7fef80139370 2023-09-30T22:29:43.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.700+0000 7fef87d80700 1 -- 172.21.15.16:0/2045389734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef80139370 msgr2=0x7fef8013b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:43.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.700+0000 7fef87d80700 1 --2- 172.21.15.16:0/2045389734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef80139370 0x7fef8013b750 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fef78009d90 tx=0x7fef7800b330 comp rx=0 tx=0).stop 2023-09-30T22:29:43.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.700+0000 7fef87d80700 1 -- 172.21.15.16:0/2045389734 shutdown_connections 2023-09-30T22:29:43.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.700+0000 7fef87d80700 1 --2- 172.21.15.16:0/2045389734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef80139370 0x7fef8013b750 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:43.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.700+0000 7fef87d80700 1 --2- 172.21.15.16:0/2045389734 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fef8009e410 0x7fef80138e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:43.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.700+0000 7fef87d80700 1 -- 172.21.15.16:0/2045389734 >> 172.21.15.16:0/2045389734 conn(0x7fef80099c30 msgr2=0x7fef800a3030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:43.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.700+0000 7fef87d80700 1 -- 172.21.15.16:0/2045389734 shutdown_connections 2023-09-30T22:29:43.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.700+0000 7fef87d80700 1 -- 172.21.15.16:0/2045389734 wait complete. 2023-09-30T22:29:43.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.701+0000 7fef87d80700 1 Processor -- start 2023-09-30T22:29:43.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.701+0000 7fef87d80700 1 -- start start 2023-09-30T22:29:43.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.701+0000 7fef87d80700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fef8009d930 0x7fef80121780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:43.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.702+0000 7fef87d80700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef8009e410 0x7fef80121cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:43.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.702+0000 7fef87d80700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fef80139370 0x7fef80122200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:43.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.702+0000 7fef87d80700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef80123950 con 0x7fef8009e410 2023-09-30T22:29:43.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.702+0000 7fef87d80700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef80071cb0 con 0x7fef8009d930 2023-09-30T22:29:43.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.702+0000 7fef87d80700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fef801249a0 con 0x7fef80139370 2023-09-30T22:29:43.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.702+0000 7fef85b1c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fef8009d930 0x7fef80121780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:43.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.702+0000 7fef8531b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef8009e410 0x7fef80121cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:43.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.702+0000 7fef85b1c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fef8009d930 0x7fef80121780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:45012/0 (socket says 172.21.15.16:45012) 2023-09-30T22:29:43.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.702+0000 7fef85b1c700 1 -- 172.21.15.16:0/848274947 learned_addr learned my addr 172.21.15.16:0/848274947 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:43.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.702+0000 7fef8631d700 1 --2- 172.21.15.16:0/848274947 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fef80139370 0x7fef80122200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:43.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.702+0000 7fef85b1c700 1 -- 172.21.15.16:0/848274947 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fef80139370 msgr2=0x7fef80122200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:43.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.702+0000 7fef85b1c700 1 --2- 172.21.15.16:0/848274947 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fef80139370 0x7fef80122200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:43.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.702+0000 7fef85b1c700 1 -- 172.21.15.16:0/848274947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef8009e410 msgr2=0x7fef80121cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:43.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.702+0000 7fef85b1c700 1 --2- 172.21.15.16:0/848274947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef8009e410 0x7fef80121cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:43.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.702+0000 7fef85b1c700 1 -- 172.21.15.16:0/848274947 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef70009550 con 0x7fef8009d930 2023-09-30T22:29:43.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.702+0000 7fef85b1c700 1 --2- 172.21.15.16:0/848274947 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fef8009d930 0x7fef80121780 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fef7c001eb0 tx=0x7fef7c00c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:43.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.702+0000 7fef767fc700 1 -- 172.21.15.16:0/848274947 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7c0054e0 con 0x7fef8009d930 2023-09-30T22:29:43.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.702+0000 7fef767fc700 1 -- 172.21.15.16:0/848274947 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef7c003680 con 0x7fef8009d930 2023-09-30T22:29:43.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.702+0000 7fef87d80700 1 -- 172.21.15.16:0/848274947 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef78009a70 con 0x7fef8009d930 2023-09-30T22:29:43.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.702+0000 7fef767fc700 1 -- 172.21.15.16:0/848274947 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7c0039f0 con 0x7fef8009d930 2023-09-30T22:29:43.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.702+0000 7fef87d80700 1 -- 172.21.15.16:0/848274947 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef80123af0 con 0x7fef8009d930 2023-09-30T22:29:43.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.703+0000 7fef767fc700 1 -- 172.21.15.16:0/848274947 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fef7c003b50 con 0x7fef8009d930 2023-09-30T22:29:43.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.703+0000 7fef767fc700 1 -- 172.21.15.16:0/848274947 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4092+0+0 (secure 0 0 0) 0x7fef7c03ef40 con 0x7fef8009d930 2023-09-30T22:29:43.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.703+0000 7fef8531b700 1 -- 172.21.15.16:0/848274947 >> v1:172.21.15.16:6816/126811 conn(0x7fef6402f510 legacy=0x7fef640319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:43.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.703+0000 7fef87d80700 1 -- 172.21.15.16:0/848274947 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef80124b70 con 0x7fef8009d930 2023-09-30T22:29:43.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.709+0000 7fef767fc700 1 -- 172.21.15.16:0/848274947 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fef7c017020 con 0x7fef8009d930 2023-09-30T22:29:43.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.904+0000 7fef8531b700 1 -- 172.21.15.16:0/848274947 >> v1:172.21.15.16:6816/126811 conn(0x7fef6402f510 legacy=0x7fef640319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:43.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:43.912+0000 7fef87d80700 1 -- 172.21.15.16:0/848274947 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_metadata", "var": "pg_autoscale_mode", "val": "off"} v 0) v1 -- 0x7fef8004f2a0 con 0x7fef8009d930 2023-09-30T22:29:44.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.305+0000 7fef8531b700 1 -- 172.21.15.16:0/848274947 >> v1:172.21.15.16:6816/126811 conn(0x7fef6402f510 legacy=0x7fef640319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:44.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.574+0000 7fef767fc700 1 -- 172.21.15.16:0/848274947 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_metadata", "var": "pg_autoscale_mode", "val": "off"}]=0 set pool 3 pg_autoscale_mode to off v37) v1 ==== 164+0+0 (secure 0 0 0) 0x7fef7c01b3b0 con 0x7fef8009d930 2023-09-30T22:29:44.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.577+0000 7fef87d80700 1 -- 172.21.15.16:0/848274947 >> v1:172.21.15.16:6816/126811 conn(0x7fef6402f510 legacy=0x7fef640319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:44.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.577+0000 7fef87d80700 1 -- 172.21.15.16:0/848274947 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fef8009d930 msgr2=0x7fef80121780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:44.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.577+0000 7fef87d80700 1 --2- 172.21.15.16:0/848274947 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fef8009d930 0x7fef80121780 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fef7c001eb0 tx=0x7fef7c00c800 comp rx=0 tx=0).stop 2023-09-30T22:29:44.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.577+0000 7fef87d80700 1 -- 172.21.15.16:0/848274947 shutdown_connections 2023-09-30T22:29:44.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.577+0000 7fef87d80700 1 --2- 172.21.15.16:0/848274947 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fef8009d930 0x7fef80121780 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:44.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.577+0000 7fef87d80700 1 --2- 172.21.15.16:0/848274947 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fef80139370 0x7fef80122200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:44.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.577+0000 7fef87d80700 1 --2- 172.21.15.16:0/848274947 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef8009e410 0x7fef80121cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:44.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.577+0000 7fef87d80700 1 -- 172.21.15.16:0/848274947 >> 172.21.15.16:0/848274947 conn(0x7fef80099c30 msgr2=0x7fef80070280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:44.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.577+0000 7fef87d80700 1 -- 172.21.15.16:0/848274947 shutdown_connections 2023-09-30T22:29:44.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.577+0000 7fef87d80700 1 -- 172.21.15.16:0/848274947 wait complete. 2023-09-30T22:29:44.579 INFO:teuthology.orchestra.run.smithi016.stderr:set pool 3 pg_autoscale_mode to off 2023-09-30T22:29:44.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.700+0000 7f1d0fa86700 1 Processor -- start 2023-09-30T22:29:44.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.701+0000 7f1d0fa86700 1 -- start start 2023-09-30T22:29:44.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.701+0000 7f1d0fa86700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d0809e9d0 0x7f1d0809eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:44.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.701+0000 7f1d0fa86700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1d0809f2e0 0x7f1d0809f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:44.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.701+0000 7f1d0fa86700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1d081317b0 0x7f1d08131c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:44.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.701+0000 7f1d0fa86700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d0809abd0 con 0x7f1d0809f2e0 2023-09-30T22:29:44.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.701+0000 7f1d0fa86700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0806fd90 con 0x7f1d0809e9d0 2023-09-30T22:29:44.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.701+0000 7f1d0fa86700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1d08071e80 con 0x7f1d081317b0 2023-09-30T22:29:44.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.701+0000 7f1d0d822700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d0809e9d0 0x7f1d0809eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:44.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.701+0000 7f1d0d021700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1d0809f2e0 0x7f1d0809f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:44.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.701+0000 7f1d0e023700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1d081317b0 0x7f1d08131c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:44.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.701+0000 7f1d0d822700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d0809e9d0 0x7f1d0809eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:45016/0 (socket says 172.21.15.16:45016) 2023-09-30T22:29:44.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.701+0000 7f1d0d021700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1d0809f2e0 0x7f1d0809f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47144/0 (socket says 172.21.15.16:47144) 2023-09-30T22:29:44.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.701+0000 7f1d0d822700 1 -- 172.21.15.16:0/558328250 learned_addr learned my addr 172.21.15.16:0/558328250 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:44.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.701+0000 7f1d0e023700 1 --2- 172.21.15.16:0/558328250 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1d081317b0 0x7f1d08131c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:46062/0 (socket says 172.21.15.16:46062) 2023-09-30T22:29:44.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.702+0000 7f1d0d822700 1 -- 172.21.15.16:0/558328250 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1d081317b0 msgr2=0x7f1d08131c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:44.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.702+0000 7f1d0d822700 1 --2- 172.21.15.16:0/558328250 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1d081317b0 0x7f1d08131c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:44.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.702+0000 7f1d0d822700 1 -- 172.21.15.16:0/558328250 >> v2:172.21.15.16:3300/0 conn(0x7f1d0809f2e0 msgr2=0x7f1d0809f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:44.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.702+0000 7f1d0d822700 1 --2- 172.21.15.16:0/558328250 >> v2:172.21.15.16:3300/0 conn(0x7f1d0809f2e0 0x7f1d0809f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:44.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.702+0000 7f1d0d822700 1 -- 172.21.15.16:0/558328250 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d080a0240 con 0x7f1d0809e9d0 2023-09-30T22:29:44.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.702+0000 7f1d0d021700 1 --2- 172.21.15.16:0/558328250 >> v2:172.21.15.16:3300/0 conn(0x7f1d0809f2e0 0x7f1d0809f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:44.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.702+0000 7f1d0e023700 1 --2- 172.21.15.16:0/558328250 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1d081317b0 0x7f1d08131c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:44.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.702+0000 7f1d0d822700 1 --2- 172.21.15.16:0/558328250 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d0809e9d0 0x7f1d0809eda0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1d040098f0 tx=0x7f1d04009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65e7fd01ab8897b5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:44.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.702+0000 7f1d0c820700 1 -- 172.21.15.16:0/558328250 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d04011830 con 0x7f1d0809e9d0 2023-09-30T22:29:44.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.702+0000 7f1d0c820700 1 -- 172.21.15.16:0/558328250 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d04011990 con 0x7f1d0809e9d0 2023-09-30T22:29:44.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.702+0000 7f1d0c820700 1 -- 172.21.15.16:0/558328250 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d04011d20 con 0x7f1d0809e9d0 2023-09-30T22:29:44.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.703+0000 7f1d0fa86700 1 -- 172.21.15.16:0/558328250 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d0809e9d0 msgr2=0x7f1d0809eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:44.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.703+0000 7f1d0fa86700 1 --2- 172.21.15.16:0/558328250 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d0809e9d0 0x7f1d0809eda0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1d040098f0 tx=0x7f1d04009c00 comp rx=0 tx=0).stop 2023-09-30T22:29:44.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.703+0000 7f1d0fa86700 1 -- 172.21.15.16:0/558328250 shutdown_connections 2023-09-30T22:29:44.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.703+0000 7f1d0fa86700 1 --2- 172.21.15.16:0/558328250 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d0809e9d0 0x7f1d0809eda0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:44.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.703+0000 7f1d0fa86700 1 --2- 172.21.15.16:0/558328250 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1d081317b0 0x7f1d08131c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:44.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.703+0000 7f1d0fa86700 1 --2- 172.21.15.16:0/558328250 >> v2:172.21.15.16:3300/0 conn(0x7f1d0809f2e0 0x7f1d0809f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:44.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.703+0000 7f1d0fa86700 1 -- 172.21.15.16:0/558328250 >> 172.21.15.16:0/558328250 conn(0x7f1d080a43c0 msgr2=0x7f1d080a47c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:44.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.703+0000 7f1d0fa86700 1 -- 172.21.15.16:0/558328250 shutdown_connections 2023-09-30T22:29:44.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.703+0000 7f1d0fa86700 1 -- 172.21.15.16:0/558328250 wait complete. 2023-09-30T22:29:44.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.704+0000 7f1d0fa86700 1 Processor -- start 2023-09-30T22:29:44.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.704+0000 7f1d0fa86700 1 -- start start 2023-09-30T22:29:44.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.704+0000 7f1d0fa86700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d0809e9d0 0x7f1d0806c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:44.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.704+0000 7f1d0fa86700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d0809f2e0 0x7f1d0806cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:44.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.704+0000 7f1d0fa86700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1d081317b0 0x7f1d0806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:44.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.704+0000 7f1d0fa86700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d08121840 con 0x7f1d0809e9d0 2023-09-30T22:29:44.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.704+0000 7f1d0fa86700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0806e5b0 con 0x7f1d0809f2e0 2023-09-30T22:29:44.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.704+0000 7f1d0fa86700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1d081214e0 con 0x7f1d081317b0 2023-09-30T22:29:44.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.704+0000 7f1d0d822700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d0809e9d0 0x7f1d0806c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:44.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.704+0000 7f1d0d822700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d0809e9d0 0x7f1d0806c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47148/0 (socket says 172.21.15.16:47148) 2023-09-30T22:29:44.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.704+0000 7f1d0d822700 1 -- 172.21.15.16:0/1547537753 learned_addr learned my addr 172.21.15.16:0/1547537753 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:44.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.704+0000 7f1d0e023700 1 --2- 172.21.15.16:0/1547537753 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1d081317b0 0x7f1d0806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:44.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.705+0000 7f1d0d021700 1 --2- 172.21.15.16:0/1547537753 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d0809f2e0 0x7f1d0806cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:44.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.705+0000 7f1d0e023700 1 -- 172.21.15.16:0/1547537753 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d0809f2e0 msgr2=0x7f1d0806cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:44.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.705+0000 7f1d0e023700 1 --2- 172.21.15.16:0/1547537753 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d0809f2e0 0x7f1d0806cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:44.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.705+0000 7f1d0e023700 1 -- 172.21.15.16:0/1547537753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d0809e9d0 msgr2=0x7f1d0806c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:44.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.705+0000 7f1d0e023700 1 --2- 172.21.15.16:0/1547537753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d0809e9d0 0x7f1d0806c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:44.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.705+0000 7f1d0e023700 1 -- 172.21.15.16:0/1547537753 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf80094a0 con 0x7f1d081317b0 2023-09-30T22:29:44.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.705+0000 7f1d0d021700 1 --2- 172.21.15.16:0/1547537753 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d0809f2e0 0x7f1d0806cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:44.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.705+0000 7f1d0e023700 1 --2- 172.21.15.16:0/1547537753 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1d081317b0 0x7f1d0806d420 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1d00009e50 tx=0x7f1d0000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:44.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.705+0000 7f1cfe7fc700 1 -- 172.21.15.16:0/1547537753 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d0000ca90 con 0x7f1d081317b0 2023-09-30T22:29:44.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.705+0000 7f1cfe7fc700 1 -- 172.21.15.16:0/1547537753 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d0000f040 con 0x7f1d081317b0 2023-09-30T22:29:44.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.705+0000 7f1d0fa86700 1 -- 172.21.15.16:0/1547537753 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d00009a70 con 0x7f1d081317b0 2023-09-30T22:29:44.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.705+0000 7f1cfe7fc700 1 -- 172.21.15.16:0/1547537753 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d0000ce50 con 0x7f1d081317b0 2023-09-30T22:29:44.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.705+0000 7f1d0fa86700 1 -- 172.21.15.16:0/1547537753 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d04009550 con 0x7f1d081317b0 2023-09-30T22:29:44.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.706+0000 7f1cfe7fc700 1 -- 172.21.15.16:0/1547537753 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f1d00014480 con 0x7f1d081317b0 2023-09-30T22:29:44.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.706+0000 7f1d0d822700 1 -- 172.21.15.16:0/1547537753 >> v1:172.21.15.16:6816/126811 conn(0x7f1cec02f540 legacy=0x7f1cec0319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:44.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.706+0000 7f1cfe7fc700 1 -- 172.21.15.16:0/1547537753 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4092+0+0 (secure 0 0 0) 0x7f1d00040350 con 0x7f1d081317b0 2023-09-30T22:29:44.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.706+0000 7f1d0fa86700 1 -- 172.21.15.16:0/1547537753 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d080a20f0 con 0x7f1d081317b0 2023-09-30T22:29:44.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.711+0000 7f1cfe7fc700 1 -- 172.21.15.16:0/1547537753 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1d00012910 con 0x7f1d081317b0 2023-09-30T22:29:44.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.907+0000 7f1d0d822700 1 -- 172.21.15.16:0/1547537753 >> v1:172.21.15.16:6816/126811 conn(0x7f1cec02f540 legacy=0x7f1cec0319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:44.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:44.921+0000 7f1d0fa86700 1 -- 172.21.15.16:0/1547537753 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data", "var": "pg_autoscale_mode", "val": "off"} v 0) v1 -- 0x7f1d08071450 con 0x7f1d081317b0 2023-09-30T22:29:45.262 INFO:tasks.ceph.mgr.x.smithi016.stderr:daemon-helper: command crashed with signal 6 2023-09-30T22:29:45.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.308+0000 7f1d0d822700 1 -- 172.21.15.16:0/1547537753 >> v1:172.21.15.16:6816/126811 conn(0x7f1cec02f540 legacy=0x7f1cec0319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:45.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.579+0000 7f1cfe7fc700 1 -- 172.21.15.16:0/1547537753 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data", "var": "pg_autoscale_mode", "val": "off"}]=0 set pool 4 pg_autoscale_mode to off v38) v1 ==== 160+0+0 (secure 0 0 0) 0x7f1d00018020 con 0x7f1d081317b0 2023-09-30T22:29:45.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.585+0000 7f1d0fa86700 1 -- 172.21.15.16:0/1547537753 >> v1:172.21.15.16:6816/126811 conn(0x7f1cec02f540 legacy=0x7f1cec0319f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:45.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.585+0000 7f1d0fa86700 1 -- 172.21.15.16:0/1547537753 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1d081317b0 msgr2=0x7f1d0806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:45.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.585+0000 7f1d0fa86700 1 --2- 172.21.15.16:0/1547537753 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1d081317b0 0x7f1d0806d420 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1d00009e50 tx=0x7f1d0000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:29:45.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.585+0000 7f1d0fa86700 1 -- 172.21.15.16:0/1547537753 shutdown_connections 2023-09-30T22:29:45.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.585+0000 7f1d0fa86700 1 --2- 172.21.15.16:0/1547537753 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1d081317b0 0x7f1d0806d420 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:45.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.585+0000 7f1d0fa86700 1 --2- 172.21.15.16:0/1547537753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d0809e9d0 0x7f1d0806c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:45.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.585+0000 7f1d0fa86700 1 --2- 172.21.15.16:0/1547537753 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d0809f2e0 0x7f1d0806cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:45.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.585+0000 7f1d0fa86700 1 -- 172.21.15.16:0/1547537753 >> 172.21.15.16:0/1547537753 conn(0x7f1d080a43c0 msgr2=0x7f1d08130c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:45.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.585+0000 7f1d0fa86700 1 -- 172.21.15.16:0/1547537753 shutdown_connections 2023-09-30T22:29:45.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.585+0000 7f1d0fa86700 1 -- 172.21.15.16:0/1547537753 wait complete. 2023-09-30T22:29:45.587 INFO:teuthology.orchestra.run.smithi016.stderr:set pool 4 pg_autoscale_mode to off 2023-09-30T22:29:45.601 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph osd require-osd-release reef' 2023-09-30T22:29:45.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.784+0000 7f6f19932700 1 Processor -- start 2023-09-30T22:29:45.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.784+0000 7f6f19932700 1 -- start start 2023-09-30T22:29:45.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.784+0000 7f6f19932700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6f1409f500 0x7f6f14138f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:45.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.784+0000 7f6f19932700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6f14139450 0x7f6f1413b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:45.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.784+0000 7f6f19932700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f14074bb0 con 0x7f6f1409ea20 2023-09-30T22:29:45.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.784+0000 7f6f19932700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f1409f8f0 con 0x7f6f1409f500 2023-09-30T22:29:45.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.784+0000 7f6f19932700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f14072210 con 0x7f6f14139450 2023-09-30T22:29:45.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.784+0000 7f6f12ffd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6f1409ea20 0x7f6f1409edf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52892/0 (socket says 172.21.15.16:52892) 2023-09-30T22:29:45.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.784+0000 7f6f127fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6f1409f500 0x7f6f14138f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:45.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.784+0000 7f6f12ffd700 1 -- 172.21.15.16:0/229608493 learned_addr learned my addr 172.21.15.16:0/229608493 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:29:45.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.784+0000 7f6f137fe700 1 --2- 172.21.15.16:0/229608493 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6f14139450 0x7f6f1413b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:45.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.785+0000 7f6f137fe700 1 -- 172.21.15.16:0/229608493 >> v2:172.21.15.16:3300/0 conn(0x7f6f1409f500 msgr2=0x7f6f14138f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:45.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.785+0000 7f6f137fe700 1 --2- 172.21.15.16:0/229608493 >> v2:172.21.15.16:3300/0 conn(0x7f6f1409f500 0x7f6f14138f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:45.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.785+0000 7f6f137fe700 1 -- 172.21.15.16:0/229608493 >> v1:172.21.15.16:6789/0 conn(0x7f6f1409ea20 legacy=0x7f6f1409edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:45.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.785+0000 7f6f137fe700 1 -- 172.21.15.16:0/229608493 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f1409ff40 con 0x7f6f14139450 2023-09-30T22:29:45.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.785+0000 7f6f11ffb700 1 -- 172.21.15.16:0/229608493 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4233881602 0 0) 0x7f6f14074bb0 con 0x7f6f1409ea20 2023-09-30T22:29:45.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.785+0000 7f6f127fc700 1 --2- 172.21.15.16:0/229608493 >> v2:172.21.15.16:3300/0 conn(0x7f6f1409f500 0x7f6f14138f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:45.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.785+0000 7f6f137fe700 1 --2- 172.21.15.16:0/229608493 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6f14139450 0x7f6f1413b830 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c009d90 tx=0x7f6f0c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72a1a451d9d2ff2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:45.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.785+0000 7f6f11ffb700 1 -- 172.21.15.16:0/229608493 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c00e070 con 0x7f6f14139450 2023-09-30T22:29:45.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.785+0000 7f6f11ffb700 1 -- 172.21.15.16:0/229608493 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f0c00bb20 con 0x7f6f14139450 2023-09-30T22:29:45.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.785+0000 7f6f11ffb700 1 -- 172.21.15.16:0/229608493 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f0c012430 con 0x7f6f14139450 2023-09-30T22:29:45.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.786+0000 7f6f19932700 1 -- 172.21.15.16:0/229608493 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6f14139450 msgr2=0x7f6f1413b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:45.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.786+0000 7f6f19932700 1 --2- 172.21.15.16:0/229608493 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6f14139450 0x7f6f1413b830 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c009d90 tx=0x7f6f0c00b330 comp rx=0 tx=0).stop 2023-09-30T22:29:45.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.786+0000 7f6f19932700 1 -- 172.21.15.16:0/229608493 shutdown_connections 2023-09-30T22:29:45.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.786+0000 7f6f19932700 1 --2- 172.21.15.16:0/229608493 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6f14139450 0x7f6f1413b830 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:45.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.786+0000 7f6f19932700 1 --2- 172.21.15.16:0/229608493 >> v2:172.21.15.16:3300/0 conn(0x7f6f1409f500 0x7f6f14138f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:45.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.786+0000 7f6f19932700 1 -- 172.21.15.16:0/229608493 >> 172.21.15.16:0/229608493 conn(0x7f6f140a0c20 msgr2=0x7f6f140a1020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:45.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.786+0000 7f6f19932700 1 -- 172.21.15.16:0/229608493 shutdown_connections 2023-09-30T22:29:45.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.786+0000 7f6f19932700 1 -- 172.21.15.16:0/229608493 wait complete. 2023-09-30T22:29:45.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.787+0000 7f6f19932700 1 Processor -- start 2023-09-30T22:29:45.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.787+0000 7f6f19932700 1 -- start start 2023-09-30T22:29:45.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.788+0000 7f6f19932700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6f1409ea20 0x7f6f14121af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:45.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.788+0000 7f6f19932700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6f1409f500 0x7f6f14122030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:45.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.788+0000 7f6f19932700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f14139450 0x7f6f14122570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:45.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.788+0000 7f6f19932700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f14137400 con 0x7f6f14139450 2023-09-30T22:29:45.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.788+0000 7f6f19932700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f14125610 con 0x7f6f1409ea20 2023-09-30T22:29:45.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.788+0000 7f6f19932700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6f14123d20 con 0x7f6f1409f500 2023-09-30T22:29:45.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.788+0000 7f6f12ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6f1409ea20 0x7f6f14121af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:45.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.788+0000 7f6f137fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f14139450 0x7f6f14122570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:45.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.788+0000 7f6f12ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6f1409ea20 0x7f6f14121af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:45046/0 (socket says 172.21.15.16:45046) 2023-09-30T22:29:45.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.788+0000 7f6f12ffd700 1 -- 172.21.15.16:0/3240793020 learned_addr learned my addr 172.21.15.16:0/3240793020 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:45.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.788+0000 7f6f127fc700 1 --2- 172.21.15.16:0/3240793020 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6f1409f500 0x7f6f14122030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:45.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.788+0000 7f6f12ffd700 1 -- 172.21.15.16:0/3240793020 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6f1409f500 msgr2=0x7f6f14122030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:45.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.788+0000 7f6f12ffd700 1 --2- 172.21.15.16:0/3240793020 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6f1409f500 0x7f6f14122030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:45.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.788+0000 7f6f12ffd700 1 -- 172.21.15.16:0/3240793020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f14139450 msgr2=0x7f6f14122570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:45.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.788+0000 7f6f12ffd700 1 --2- 172.21.15.16:0/3240793020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f14139450 0x7f6f14122570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:45.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.788+0000 7f6f12ffd700 1 -- 172.21.15.16:0/3240793020 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f080094a0 con 0x7f6f1409ea20 2023-09-30T22:29:45.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.788+0000 7f6f137fe700 1 --2- 172.21.15.16:0/3240793020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f14139450 0x7f6f14122570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:45.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.788+0000 7f6f127fc700 1 --2- 172.21.15.16:0/3240793020 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6f1409f500 0x7f6f14122030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:45.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.788+0000 7f6f12ffd700 1 --2- 172.21.15.16:0/3240793020 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6f1409ea20 0x7f6f14121af0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6f0400c520 tx=0x7f6f0400c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:45.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.788+0000 7f6efbfff700 1 -- 172.21.15.16:0/3240793020 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f04004c80 con 0x7f6f1409ea20 2023-09-30T22:29:45.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.788+0000 7f6efbfff700 1 -- 172.21.15.16:0/3240793020 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f04004de0 con 0x7f6f1409ea20 2023-09-30T22:29:45.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.788+0000 7f6f19932700 1 -- 172.21.15.16:0/3240793020 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f0c009a70 con 0x7f6f1409ea20 2023-09-30T22:29:45.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.789+0000 7f6efbfff700 1 -- 172.21.15.16:0/3240793020 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f04007310 con 0x7f6f1409ea20 2023-09-30T22:29:45.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.789+0000 7f6f19932700 1 -- 172.21.15.16:0/3240793020 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f14125070 con 0x7f6f1409ea20 2023-09-30T22:29:45.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.789+0000 7f6efbfff700 1 -- 172.21.15.16:0/3240793020 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f6f04007470 con 0x7f6f1409ea20 2023-09-30T22:29:45.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.789+0000 7f6f19932700 1 -- 172.21.15.16:0/3240793020 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f1409af70 con 0x7f6f1409ea20 2023-09-30T22:29:45.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.790+0000 7f6f127fc700 1 -- 172.21.15.16:0/3240793020 >> v1:172.21.15.16:6816/126811 conn(0x7f6f0002f510 legacy=0x7f6f000319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:45.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.790+0000 7f6efbfff700 1 -- 172.21.15.16:0/3240793020 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4092+0+0 (secure 0 0 0) 0x7f6f0403f840 con 0x7f6f1409ea20 2023-09-30T22:29:45.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.794+0000 7f6efbfff700 1 -- 172.21.15.16:0/3240793020 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6f040133c0 con 0x7f6f1409ea20 2023-09-30T22:29:45.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:45.991+0000 7f6f127fc700 1 -- 172.21.15.16:0/3240793020 >> v1:172.21.15.16:6816/126811 conn(0x7f6f0002f510 legacy=0x7f6f000319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:46.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.015+0000 7f6f19932700 1 -- 172.21.15.16:0/3240793020 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd require-osd-release", "release": "reef"} v 0) v1 -- 0x7f6f1404f2a0 con 0x7f6f1409ea20 2023-09-30T22:29:46.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.392+0000 7f6f127fc700 1 -- 172.21.15.16:0/3240793020 >> v1:172.21.15.16:6816/126811 conn(0x7f6f0002f510 legacy=0x7f6f000319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:46.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.584+0000 7f6efbfff700 1 -- 172.21.15.16:0/3240793020 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd require-osd-release", "release": "reef"}]=0 v39) v1 ==== 90+0+0 (secure 0 0 0) 0x7f6f040102c0 con 0x7f6f1409ea20 2023-09-30T22:29:46.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.589+0000 7f6f19932700 1 -- 172.21.15.16:0/3240793020 >> v1:172.21.15.16:6816/126811 conn(0x7f6f0002f510 legacy=0x7f6f000319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:46.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.589+0000 7f6f19932700 1 -- 172.21.15.16:0/3240793020 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6f1409ea20 msgr2=0x7f6f14121af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:46.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.589+0000 7f6f19932700 1 --2- 172.21.15.16:0/3240793020 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6f1409ea20 0x7f6f14121af0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6f0400c520 tx=0x7f6f0400c830 comp rx=0 tx=0).stop 2023-09-30T22:29:46.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.589+0000 7f6f19932700 1 -- 172.21.15.16:0/3240793020 shutdown_connections 2023-09-30T22:29:46.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.589+0000 7f6f19932700 1 --2- 172.21.15.16:0/3240793020 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6f1409ea20 0x7f6f14121af0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:46.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.589+0000 7f6f19932700 1 --2- 172.21.15.16:0/3240793020 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f14139450 0x7f6f14122570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:46.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.589+0000 7f6f19932700 1 --2- 172.21.15.16:0/3240793020 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6f1409f500 0x7f6f14122030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:46.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.589+0000 7f6f19932700 1 -- 172.21.15.16:0/3240793020 >> 172.21.15.16:0/3240793020 conn(0x7f6f140a0c20 msgr2=0x7f6f14070fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:46.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.589+0000 7f6f19932700 1 -- 172.21.15.16:0/3240793020 shutdown_connections 2023-09-30T22:29:46.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.589+0000 7f6f19932700 1 -- 172.21.15.16:0/3240793020 wait complete. 2023-09-30T22:29:46.605 INFO:teuthology.run_tasks:Running task ceph.healthy... 2023-09-30T22:29:46.616 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-09-30T22:29:46.617 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-09-30T22:29:46.617 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-09-30T22:29:46.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.759+0000 7f76c9cb4700 1 Processor -- start 2023-09-30T22:29:46.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.759+0000 7f76c9cb4700 1 -- start start 2023-09-30T22:29:46.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.759+0000 7f76c9cb4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f76c409d6f0 0x7f76c409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:46.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.759+0000 7f76c9cb4700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f76c409e410 0x7f76c4130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:46.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.759+0000 7f76c9cb4700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76c409f130 con 0x7f76c409e000 2023-09-30T22:29:46.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.759+0000 7f76c9cb4700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76c409f520 con 0x7f76c409e410 2023-09-30T22:29:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.759+0000 7f76c9cb4700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f76c409e930 con 0x7f76c409d6f0 2023-09-30T22:29:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.760+0000 7f76c3fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f76c409e410 0x7f76c4130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.760+0000 7f76c2ffd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f76c409e000 0x7f76c4130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52900/0 (socket says 172.21.15.16:52900) 2023-09-30T22:29:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.760+0000 7f76c2ffd700 1 -- 172.21.15.16:0/439287435 learned_addr learned my addr 172.21.15.16:0/439287435 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:29:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.760+0000 7f76c37fe700 1 --2- 172.21.15.16:0/439287435 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f76c409d6f0 0x7f76c409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.760+0000 7f76c27fc700 1 -- 172.21.15.16:0/439287435 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2504330768 0 0) 0x7f76c409f130 con 0x7f76c409e000 2023-09-30T22:29:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.760+0000 7f76c27fc700 1 -- 172.21.15.16:0/439287435 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76b00035b0 con 0x7f76c409e000 2023-09-30T22:29:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.760+0000 7f76c37fe700 1 -- 172.21.15.16:0/439287435 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f76c409e410 msgr2=0x7f76c4130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.760+0000 7f76c37fe700 1 --2- 172.21.15.16:0/439287435 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f76c409e410 0x7f76c4130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.760+0000 7f76c37fe700 1 -- 172.21.15.16:0/439287435 >> v1:172.21.15.16:6789/0 conn(0x7f76c409e000 legacy=0x7f76c4130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:46.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.760+0000 7f76c37fe700 1 -- 172.21.15.16:0/439287435 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76c409b2d0 con 0x7f76c409d6f0 2023-09-30T22:29:46.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.760+0000 7f76c3fff700 1 --2- 172.21.15.16:0/439287435 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f76c409e410 0x7f76c4130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:46.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.760+0000 7f76c37fe700 1 --2- 172.21.15.16:0/439287435 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f76c409d6f0 0x7f76c409dac0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f76b4009870 tx=0x7f76b4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a681a57e7950f5d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:46.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.760+0000 7f76c27fc700 1 -- 172.21.15.16:0/439287435 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b4011830 con 0x7f76c409d6f0 2023-09-30T22:29:46.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.760+0000 7f76c27fc700 1 -- 172.21.15.16:0/439287435 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76b4011990 con 0x7f76c409d6f0 2023-09-30T22:29:46.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.760+0000 7f76c27fc700 1 -- 172.21.15.16:0/439287435 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b4011d20 con 0x7f76c409d6f0 2023-09-30T22:29:46.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.761+0000 7f76c9cb4700 1 -- 172.21.15.16:0/439287435 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f76c409d6f0 msgr2=0x7f76c409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:46.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.761+0000 7f76c9cb4700 1 --2- 172.21.15.16:0/439287435 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f76c409d6f0 0x7f76c409dac0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f76b4009870 tx=0x7f76b4009b80 comp rx=0 tx=0).stop 2023-09-30T22:29:46.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.761+0000 7f76c9cb4700 1 -- 172.21.15.16:0/439287435 shutdown_connections 2023-09-30T22:29:46.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.761+0000 7f76c9cb4700 1 --2- 172.21.15.16:0/439287435 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f76c409e410 0x7f76c4130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:46.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.761+0000 7f76c9cb4700 1 --2- 172.21.15.16:0/439287435 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f76c409d6f0 0x7f76c409dac0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:46.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.761+0000 7f76c9cb4700 1 -- 172.21.15.16:0/439287435 >> 172.21.15.16:0/439287435 conn(0x7f76c4066f50 msgr2=0x7f76c409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:46.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.761+0000 7f76c9cb4700 1 -- 172.21.15.16:0/439287435 shutdown_connections 2023-09-30T22:29:46.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.761+0000 7f76c9cb4700 1 -- 172.21.15.16:0/439287435 wait complete. 2023-09-30T22:29:46.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.762+0000 7f76c9cb4700 1 Processor -- start 2023-09-30T22:29:46.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.762+0000 7f76c9cb4700 1 -- start start 2023-09-30T22:29:46.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.763+0000 7f76c9cb4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76c409d6f0 0x7f76c4122400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:46.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.763+0000 7f76c9cb4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f76c409e000 0x7f76c4122940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:46.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.763+0000 7f76c9cb4700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f76c409e410 0x7f76c4122e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:46.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.763+0000 7f76c9cb4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c4130390 con 0x7f76c409d6f0 2023-09-30T22:29:46.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.763+0000 7f76c9cb4700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76c41240b0 con 0x7f76c409e410 2023-09-30T22:29:46.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.763+0000 7f76c9cb4700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f76c4125c40 con 0x7f76c409e000 2023-09-30T22:29:46.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.763+0000 7f76c37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76c409d6f0 0x7f76c4122400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:46.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.763+0000 7f76c37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76c409d6f0 0x7f76c4122400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47176/0 (socket says 172.21.15.16:47176) 2023-09-30T22:29:46.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.763+0000 7f76c37fe700 1 -- 172.21.15.16:0/699591737 learned_addr learned my addr 172.21.15.16:0/699591737 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:46.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.763+0000 7f76c3fff700 1 --2- 172.21.15.16:0/699591737 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f76c409e410 0x7f76c4122e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:46.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.763+0000 7f76c2ffd700 1 --2- 172.21.15.16:0/699591737 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f76c409e000 0x7f76c4122940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:46.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.763+0000 7f76c37fe700 1 -- 172.21.15.16:0/699591737 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f76c409e000 msgr2=0x7f76c4122940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:46.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.763+0000 7f76c37fe700 1 --2- 172.21.15.16:0/699591737 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f76c409e000 0x7f76c4122940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:46.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.763+0000 7f76c37fe700 1 -- 172.21.15.16:0/699591737 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f76c409e410 msgr2=0x7f76c4122e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:46.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.763+0000 7f76c37fe700 1 --2- 172.21.15.16:0/699591737 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f76c409e410 0x7f76c4122e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:46.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.763+0000 7f76c37fe700 1 -- 172.21.15.16:0/699591737 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76bc00bc80 con 0x7f76c409d6f0 2023-09-30T22:29:46.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.763+0000 7f76c2ffd700 1 --2- 172.21.15.16:0/699591737 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f76c409e000 0x7f76c4122940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:46.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.763+0000 7f76c3fff700 1 --2- 172.21.15.16:0/699591737 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f76c409e410 0x7f76c4122e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:46.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.763+0000 7f76c37fe700 1 --2- 172.21.15.16:0/699591737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76c409d6f0 0x7f76c4122400 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f76b4000c00 tx=0x7f76b4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:46.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.764+0000 7f76bbfff700 1 -- 172.21.15.16:0/699591737 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b4004450 con 0x7f76c409d6f0 2023-09-30T22:29:46.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.764+0000 7f76bbfff700 1 -- 172.21.15.16:0/699591737 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76b401f070 con 0x7f76c409d6f0 2023-09-30T22:29:46.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.764+0000 7f76c9cb4700 1 -- 172.21.15.16:0/699591737 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76b40094a0 con 0x7f76c409d6f0 2023-09-30T22:29:46.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.764+0000 7f76bbfff700 1 -- 172.21.15.16:0/699591737 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b40038e0 con 0x7f76c409d6f0 2023-09-30T22:29:46.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.764+0000 7f76c9cb4700 1 -- 172.21.15.16:0/699591737 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76c4124800 con 0x7f76c409d6f0 2023-09-30T22:29:46.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.764+0000 7f76bbfff700 1 -- 172.21.15.16:0/699591737 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f76b4003a40 con 0x7f76c409d6f0 2023-09-30T22:29:46.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.765+0000 7f76bbfff700 1 -- 172.21.15.16:0/699591737 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7f76b4021070 con 0x7f76c409d6f0 2023-09-30T22:29:46.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.765+0000 7f76c2ffd700 1 -- 172.21.15.16:0/699591737 >> v1:172.21.15.16:6816/126811 conn(0x7f76a402f510 legacy=0x7f76a40319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:46.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.765+0000 7f76c9cb4700 1 -- 172.21.15.16:0/699591737 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76c41258e0 con 0x7f76c409d6f0 2023-09-30T22:29:46.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.770+0000 7f76bbfff700 1 -- 172.21.15.16:0/699591737 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f76b4024380 con 0x7f76c409d6f0 2023-09-30T22:29:46.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:46.966+0000 7f76c2ffd700 1 -- 172.21.15.16:0/699591737 >> v1:172.21.15.16:6816/126811 conn(0x7f76a402f510 legacy=0x7f76a40319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:47.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.031+0000 7f76c9cb4700 1 -- 172.21.15.16:0/699591737 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f76c4074ea0 con 0x7f76c409d6f0 2023-09-30T22:29:47.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.033+0000 7f76bbfff700 1 -- 172.21.15.16:0/699591737 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v9) v1 ==== 74+0+70484 (secure 0 0 0) 0x7f76b402eb90 con 0x7f76c409d6f0 2023-09-30T22:29:47.034 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:29:47.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.035+0000 7f76c9cb4700 1 -- 172.21.15.16:0/699591737 >> v1:172.21.15.16:6816/126811 conn(0x7f76a402f510 legacy=0x7f76a40319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:47.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.036+0000 7f76c9cb4700 1 -- 172.21.15.16:0/699591737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76c409d6f0 msgr2=0x7f76c4122400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:47.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.036+0000 7f76c9cb4700 1 --2- 172.21.15.16:0/699591737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76c409d6f0 0x7f76c4122400 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f76b4000c00 tx=0x7f76b4004750 comp rx=0 tx=0).stop 2023-09-30T22:29:47.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.036+0000 7f76c9cb4700 1 -- 172.21.15.16:0/699591737 shutdown_connections 2023-09-30T22:29:47.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.036+0000 7f76c9cb4700 1 --2- 172.21.15.16:0/699591737 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f76c409e410 0x7f76c4122e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.036+0000 7f76c9cb4700 1 --2- 172.21.15.16:0/699591737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76c409d6f0 0x7f76c4122400 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.036+0000 7f76c9cb4700 1 --2- 172.21.15.16:0/699591737 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f76c409e000 0x7f76c4122940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.036+0000 7f76c9cb4700 1 -- 172.21.15.16:0/699591737 >> 172.21.15.16:0/699591737 conn(0x7f76c4066f50 msgr2=0x7f76c4070e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:47.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.036+0000 7f76c9cb4700 1 -- 172.21.15.16:0/699591737 shutdown_connections 2023-09-30T22:29:47.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.036+0000 7f76c9cb4700 1 -- 172.21.15.16:0/699591737 wait complete. 2023-09-30T22:29:47.050 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":9,"active_gid":14103,"active_name":"x","active_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6816","nonce":126811}]},"active_addr":"172.21.15.16:6816/126811","active_change":"2023-09-30T22:29:14.476162+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":20,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":1879378585}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":3146063150}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":3403658744}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":3418827610}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":1216430525}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":2178963721}]}]} 2023-09-30T22:29:47.052 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-09-30T22:29:47.052 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-09-30T22:29:47.052 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:29:47.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.188+0000 7ff25dde2700 1 Processor -- start 2023-09-30T22:29:47.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.188+0000 7ff25dde2700 1 -- start start 2023-09-30T22:29:47.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.189+0000 7ff25dde2700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff25809d6f0 0x7ff25809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:47.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.189+0000 7ff25dde2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff25809e440 0x7ff258134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:47.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.189+0000 7ff25dde2700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff25809b120 con 0x7ff25809e000 2023-09-30T22:29:47.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.189+0000 7ff25dde2700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff25806b460 con 0x7ff25809d6f0 2023-09-30T22:29:47.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.189+0000 7ff25dde2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff25809a7e0 con 0x7ff25809e440 2023-09-30T22:29:47.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.189+0000 7ff257fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff25809e440 0x7ff258134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:47.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.189+0000 7ff2577fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff25809d6f0 0x7ff25809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:47.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.189+0000 7ff257fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff25809e440 0x7ff258134100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:45080/0 (socket says 172.21.15.16:45080) 2023-09-30T22:29:47.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.189+0000 7ff2577fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff25809d6f0 0x7ff25809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47186/0 (socket says 172.21.15.16:47186) 2023-09-30T22:29:47.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.189+0000 7ff257fff700 1 -- 172.21.15.16:0/2043248323 learned_addr learned my addr 172.21.15.16:0/2043248323 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:47.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.189+0000 7ff2577fe700 1 -- 172.21.15.16:0/2043248323 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff25809e440 msgr2=0x7ff258134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:47.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.189+0000 7ff2577fe700 1 --2- 172.21.15.16:0/2043248323 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff25809e440 0x7ff258134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.189+0000 7ff2577fe700 1 -- 172.21.15.16:0/2043248323 >> v1:172.21.15.16:6789/0 conn(0x7ff25809e000 legacy=0x7ff2581339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:47.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.189+0000 7ff2577fe700 1 -- 172.21.15.16:0/2043248323 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff25809a500 con 0x7ff25809d6f0 2023-09-30T22:29:47.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.189+0000 7ff2567fc700 1 -- 172.21.15.16:0/2043248323 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1369850421 0 0) 0x7ff25809b120 con 0x7ff25809e000 2023-09-30T22:29:47.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.189+0000 7ff257fff700 1 --2- 172.21.15.16:0/2043248323 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff25809e440 0x7ff258134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:47.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.190+0000 7ff2577fe700 1 --2- 172.21.15.16:0/2043248323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff25809d6f0 0x7ff25809dac0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff248009870 tx=0x7ff248009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93d14f48561567a0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:47.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.190+0000 7ff2567fc700 1 -- 172.21.15.16:0/2043248323 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff248011830 con 0x7ff25809d6f0 2023-09-30T22:29:47.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.190+0000 7ff2567fc700 1 -- 172.21.15.16:0/2043248323 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff248011990 con 0x7ff25809d6f0 2023-09-30T22:29:47.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.190+0000 7ff2567fc700 1 -- 172.21.15.16:0/2043248323 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff248011d20 con 0x7ff25809d6f0 2023-09-30T22:29:47.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.190+0000 7ff25dde2700 1 -- 172.21.15.16:0/2043248323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff25809d6f0 msgr2=0x7ff25809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:47.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.190+0000 7ff25dde2700 1 --2- 172.21.15.16:0/2043248323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff25809d6f0 0x7ff25809dac0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff248009870 tx=0x7ff248009b80 comp rx=0 tx=0).stop 2023-09-30T22:29:47.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.190+0000 7ff25dde2700 1 -- 172.21.15.16:0/2043248323 shutdown_connections 2023-09-30T22:29:47.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.190+0000 7ff25dde2700 1 --2- 172.21.15.16:0/2043248323 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff25809e440 0x7ff258134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.190+0000 7ff25dde2700 1 --2- 172.21.15.16:0/2043248323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff25809d6f0 0x7ff25809dac0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.190+0000 7ff25dde2700 1 -- 172.21.15.16:0/2043248323 >> 172.21.15.16:0/2043248323 conn(0x7ff258066f50 msgr2=0x7ff25809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:47.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.190+0000 7ff25dde2700 1 -- 172.21.15.16:0/2043248323 shutdown_connections 2023-09-30T22:29:47.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.190+0000 7ff25dde2700 1 -- 172.21.15.16:0/2043248323 wait complete. 2023-09-30T22:29:47.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.191+0000 7ff25dde2700 1 Processor -- start 2023-09-30T22:29:47.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.192+0000 7ff25dde2700 1 -- start start 2023-09-30T22:29:47.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.192+0000 7ff25dde2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff25809d6f0 0x7ff2581a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:47.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.192+0000 7ff25dde2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff25809e000 0x7ff2581a58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:47.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.192+0000 7ff25dde2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff25809e440 0x7ff2581c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:47.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.192+0000 7ff25dde2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2581a8110 con 0x7ff25809e440 2023-09-30T22:29:47.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.192+0000 7ff25dde2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2581b9180 con 0x7ff25809d6f0 2023-09-30T22:29:47.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.192+0000 7ff25dde2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff2581a8310 con 0x7ff25809e000 2023-09-30T22:29:47.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.192+0000 7ff2577fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff25809d6f0 0x7ff2581a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:47.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.192+0000 7ff257fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff25809e440 0x7ff2581c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:47.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.192+0000 7ff256ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff25809e000 0x7ff2581a58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:47.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.192+0000 7ff2577fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff25809d6f0 0x7ff2581a5380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:45088/0 (socket says 172.21.15.16:45088) 2023-09-30T22:29:47.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.192+0000 7ff2577fe700 1 -- 172.21.15.16:0/2438766336 learned_addr learned my addr 172.21.15.16:0/2438766336 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:47.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.192+0000 7ff257fff700 1 --2- 172.21.15.16:0/2438766336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff25809e440 0x7ff2581c7fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47196/0 (socket says 172.21.15.16:47196) 2023-09-30T22:29:47.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.193+0000 7ff2577fe700 1 -- 172.21.15.16:0/2438766336 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff25809e000 msgr2=0x7ff2581a58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:47.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.193+0000 7ff2577fe700 1 --2- 172.21.15.16:0/2438766336 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff25809e000 0x7ff2581a58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.193+0000 7ff2577fe700 1 -- 172.21.15.16:0/2438766336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff25809e440 msgr2=0x7ff2581c7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:47.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.193+0000 7ff2577fe700 1 --2- 172.21.15.16:0/2438766336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff25809e440 0x7ff2581c7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.193+0000 7ff2577fe700 1 -- 172.21.15.16:0/2438766336 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff250009b30 con 0x7ff25809d6f0 2023-09-30T22:29:47.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.193+0000 7ff257fff700 1 --2- 172.21.15.16:0/2438766336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff25809e440 0x7ff2581c7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:47.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.193+0000 7ff256ffd700 1 --2- 172.21.15.16:0/2438766336 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff25809e000 0x7ff2581a58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:47.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.193+0000 7ff2577fe700 1 --2- 172.21.15.16:0/2438766336 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff25809d6f0 0x7ff2581a5380 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff248000c00 tx=0x7ff248004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:47.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.193+0000 7ff24ffff700 1 -- 172.21.15.16:0/2438766336 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2480036a0 con 0x7ff25809d6f0 2023-09-30T22:29:47.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.193+0000 7ff24ffff700 1 -- 172.21.15.16:0/2438766336 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff24801f070 con 0x7ff25809d6f0 2023-09-30T22:29:47.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.193+0000 7ff25dde2700 1 -- 172.21.15.16:0/2438766336 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2480094a0 con 0x7ff25809d6f0 2023-09-30T22:29:47.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.193+0000 7ff24ffff700 1 -- 172.21.15.16:0/2438766336 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff248003a20 con 0x7ff25809d6f0 2023-09-30T22:29:47.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.193+0000 7ff25dde2700 1 -- 172.21.15.16:0/2438766336 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2581a4d90 con 0x7ff25809d6f0 2023-09-30T22:29:47.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.194+0000 7ff24ffff700 1 -- 172.21.15.16:0/2438766336 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7ff248003b80 con 0x7ff25809d6f0 2023-09-30T22:29:47.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.194+0000 7ff24ffff700 1 -- 172.21.15.16:0/2438766336 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7ff248021070 con 0x7ff25809d6f0 2023-09-30T22:29:47.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.194+0000 7ff256ffd700 1 -- 172.21.15.16:0/2438766336 >> v1:172.21.15.16:6816/126811 conn(0x7ff23802f510 legacy=0x7ff2380319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:47.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.194+0000 7ff25dde2700 1 -- 172.21.15.16:0/2438766336 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff25809c710 con 0x7ff25809d6f0 2023-09-30T22:29:47.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.199+0000 7ff24ffff700 1 -- 172.21.15.16:0/2438766336 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff248003e30 con 0x7ff25809d6f0 2023-09-30T22:29:47.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.395+0000 7ff256ffd700 1 -- 172.21.15.16:0/2438766336 >> v1:172.21.15.16:6816/126811 conn(0x7ff23802f510 legacy=0x7ff2380319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:47.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.403+0000 7ff25dde2700 1 -- 172.21.15.16:0/2438766336 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff25809f420 con 0x7ff25809d6f0 2023-09-30T22:29:47.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.404+0000 7ff24ffff700 1 -- 172.21.15.16:0/2438766336 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+13014 (secure 0 0 0) 0x7ff248024b70 con 0x7ff25809d6f0 2023-09-30T22:29:47.405 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:29:47.405 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":39,"fsid":"7ede3e14-2bd0-4726-9437-6d13c90cd342","created":"2023-09-30T22:26:35.483697+0000","modified":"2023-09-30T22:29:46.581239+0000","last_up_change":"2023-09-30T22:29:35.702639+0000","last_in_change":"2023-09-30T22:26:37.120906+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T22:26:44.285133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{},"mgr_devicehealth":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:26:45.132921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:26:49.353683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:26:50.655685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"19946985-0ee2-487f-9347-929670dbe5ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":22,"up_from":28,"up_thru":32,"down_at":23,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6804","nonce":129539}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6805","nonce":129539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6807","nonce":129539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6806","nonce":129539}]},"public_addr":"172.21.15.16:6804/129539","cluster_addr":"172.21.15.16:6805/129539","heartbeat_back_addr":"172.21.15.16:6807/129539","heartbeat_front_addr":"172.21.15.16:6806/129539","state":["exists","up"]},{"osd":1,"uuid":"8efd4dbc-6ae4-4b82-b530-3b3eac2be445","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":22,"up_from":29,"up_thru":32,"down_at":24,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6808","nonce":129637}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6809","nonce":129637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6811","nonce":129637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6810","nonce":129637}]},"public_addr":"172.21.15.16:6808/129637","cluster_addr":"172.21.15.16:6809/129637","heartbeat_back_addr":"172.21.15.16:6811/129637","heartbeat_front_addr":"172.21.15.16:6810/129637","state":["exists","up"]},{"osd":2,"uuid":"b47bb1ce-27af-4c0c-868a-b76ebd0d9499","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":24,"up_from":31,"up_thru":32,"down_at":25,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6800","nonce":129747}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6801","nonce":129747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6803","nonce":129747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6802","nonce":129747}]},"public_addr":"172.21.15.16:6800/129747","cluster_addr":"172.21.15.16:6801/129747","heartbeat_back_addr":"172.21.15.16:6803/129747","heartbeat_front_addr":"172.21.15.16:6802/129747","state":["exists","up"]},{"osd":3,"uuid":"03b53d64-d10d-448a-a081-39ecceab589b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":25,"up_from":32,"up_thru":32,"down_at":26,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6812","nonce":130087}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6813","nonce":130087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6815","nonce":130087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6814","nonce":130087}]},"public_addr":"172.21.15.16:6812/130087","cluster_addr":"172.21.15.16:6813/130087","heartbeat_back_addr":"172.21.15.16:6815/130087","heartbeat_front_addr":"172.21.15.16:6814/130087","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.305953+0000","dead_epoch":23},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.307488+0000","dead_epoch":24},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.341023+0000","dead_epoch":25},{"osd":3,"down_stamp":"2023-09-30T22:29:28.519844+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.192071+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.8","mappings":[{"from":3,"to":2}]},{"pgid":"4.d","mappings":[{"from":3,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":0}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]},{"pgid":"4.17","mappings":[{"from":3,"to":2}]},{"pgid":"4.19","mappings":[{"from":3,"to":0}]},{"pgid":"4.3a","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6817/1424366610":"2023-10-01T22:29:40.259230+0000","172.21.15.16:6816/118285":"2023-10-01T22:29:14.475978+0000","172.21.15.16:0/2249754484":"2023-10-01T22:29:14.475978+0000","172.21.15.16:0/1101648925":"2023-10-01T22:29:14.475978+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:29:47.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.407+0000 7ff25dde2700 1 -- 172.21.15.16:0/2438766336 >> v1:172.21.15.16:6816/126811 conn(0x7ff23802f510 legacy=0x7ff2380319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:47.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.407+0000 7ff25dde2700 1 -- 172.21.15.16:0/2438766336 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff25809d6f0 msgr2=0x7ff2581a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:47.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.407+0000 7ff25dde2700 1 --2- 172.21.15.16:0/2438766336 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff25809d6f0 0x7ff2581a5380 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff248000c00 tx=0x7ff248004750 comp rx=0 tx=0).stop 2023-09-30T22:29:47.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.407+0000 7ff25dde2700 1 -- 172.21.15.16:0/2438766336 shutdown_connections 2023-09-30T22:29:47.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.407+0000 7ff25dde2700 1 --2- 172.21.15.16:0/2438766336 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff25809d6f0 0x7ff2581a5380 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.407+0000 7ff25dde2700 1 --2- 172.21.15.16:0/2438766336 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff25809e440 0x7ff2581c7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.407+0000 7ff25dde2700 1 --2- 172.21.15.16:0/2438766336 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff25809e000 0x7ff2581a58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.407+0000 7ff25dde2700 1 -- 172.21.15.16:0/2438766336 >> 172.21.15.16:0/2438766336 conn(0x7ff258066f50 msgr2=0x7ff2581bbab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:47.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.407+0000 7ff25dde2700 1 -- 172.21.15.16:0/2438766336 shutdown_connections 2023-09-30T22:29:47.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.407+0000 7ff25dde2700 1 -- 172.21.15.16:0/2438766336 wait complete. 2023-09-30T22:29:47.421 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-09-30T22:29:47.421 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:29:47.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.558+0000 7f6d21440700 1 Processor -- start 2023-09-30T22:29:47.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.558+0000 7f6d21440700 1 -- start start 2023-09-30T22:29:47.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.558+0000 7f6d21440700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6d1c0a1430 0x7f6d1c0a2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:47.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.558+0000 7f6d21440700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d1c138bd0 0x7f6d1c13afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:47.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.558+0000 7f6d21440700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d1c0a2950 con 0x7f6d1c0a0950 2023-09-30T22:29:47.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.558+0000 7f6d21440700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d1c09e0a0 con 0x7f6d1c0a1430 2023-09-30T22:29:47.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.558+0000 7f6d21440700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c09db00 con 0x7f6d1c138bd0 2023-09-30T22:29:47.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.558+0000 7f6d1b7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d1c138bd0 0x7f6d1c13afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:47.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.559+0000 7f6d1b7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d1c138bd0 0x7f6d1c13afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:49238/0 (socket says 172.21.15.16:49238) 2023-09-30T22:29:47.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.559+0000 7f6d1a7fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6d1c0a1430 0x7f6d1c0a2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:47.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.559+0000 7f6d1b7fe700 1 -- 172.21.15.16:0/2599175325 learned_addr learned my addr 172.21.15.16:0/2599175325 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:47.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.559+0000 7f6d1a7fc700 1 -- 172.21.15.16:0/2599175325 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d1c138bd0 msgr2=0x7f6d1c13afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:47.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.559+0000 7f6d1a7fc700 1 --2- 172.21.15.16:0/2599175325 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d1c138bd0 0x7f6d1c13afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.559+0000 7f6d1a7fc700 1 -- 172.21.15.16:0/2599175325 >> v1:172.21.15.16:6789/0 conn(0x7f6d1c0a0950 legacy=0x7f6d1c0a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:47.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.559+0000 7f6d1a7fc700 1 -- 172.21.15.16:0/2599175325 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d1c09ddc0 con 0x7f6d1c0a1430 2023-09-30T22:29:47.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.559+0000 7f6d19ffb700 1 -- 172.21.15.16:0/2599175325 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 335588894 0 0) 0x7f6d1c0a2950 con 0x7f6d1c0a0950 2023-09-30T22:29:47.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.559+0000 7f6d1b7fe700 1 --2- 172.21.15.16:0/2599175325 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d1c138bd0 0x7f6d1c13afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:47.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.559+0000 7f6d1a7fc700 1 --2- 172.21.15.16:0/2599175325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d1c0a1430 0x7f6d1c0a2330 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6d10009840 tx=0x7f6d10009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5947ad4f7c94804 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:47.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.559+0000 7f6d19ffb700 1 -- 172.21.15.16:0/2599175325 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d10011830 con 0x7f6d1c0a1430 2023-09-30T22:29:47.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.559+0000 7f6d19ffb700 1 -- 172.21.15.16:0/2599175325 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d10011990 con 0x7f6d1c0a1430 2023-09-30T22:29:47.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.559+0000 7f6d19ffb700 1 -- 172.21.15.16:0/2599175325 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d10011d20 con 0x7f6d1c0a1430 2023-09-30T22:29:47.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.560+0000 7f6d21440700 1 -- 172.21.15.16:0/2599175325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d1c0a1430 msgr2=0x7f6d1c0a2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:47.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.560+0000 7f6d21440700 1 --2- 172.21.15.16:0/2599175325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d1c0a1430 0x7f6d1c0a2330 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6d10009840 tx=0x7f6d10009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:47.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.560+0000 7f6d21440700 1 -- 172.21.15.16:0/2599175325 shutdown_connections 2023-09-30T22:29:47.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.560+0000 7f6d21440700 1 --2- 172.21.15.16:0/2599175325 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d1c138bd0 0x7f6d1c13afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.560+0000 7f6d21440700 1 --2- 172.21.15.16:0/2599175325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d1c0a1430 0x7f6d1c0a2330 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.560+0000 7f6d21440700 1 -- 172.21.15.16:0/2599175325 >> 172.21.15.16:0/2599175325 conn(0x7f6d1c0a2f70 msgr2=0x7f6d1c0a3370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:47.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.560+0000 7f6d21440700 1 -- 172.21.15.16:0/2599175325 shutdown_connections 2023-09-30T22:29:47.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.560+0000 7f6d21440700 1 -- 172.21.15.16:0/2599175325 wait complete. 2023-09-30T22:29:47.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.561+0000 7f6d21440700 1 Processor -- start 2023-09-30T22:29:47.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.561+0000 7f6d21440700 1 -- start start 2023-09-30T22:29:47.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.562+0000 7f6d21440700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d1c0a0950 0x7f6d1c133dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:47.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.562+0000 7f6d21440700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d1c0a1430 0x7f6d1c134300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:47.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.562+0000 7f6d21440700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d1c138bd0 0x7f6d1c134840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:47.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.562+0000 7f6d21440700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c13ae40 con 0x7f6d1c0a1430 2023-09-30T22:29:47.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.562+0000 7f6d21440700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c09cdc0 con 0x7f6d1c0a0950 2023-09-30T22:29:47.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.562+0000 7f6d21440700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c135420 con 0x7f6d1c138bd0 2023-09-30T22:29:47.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.562+0000 7f6d1affd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d1c0a0950 0x7f6d1c133dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:47.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.562+0000 7f6d1b7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d1c138bd0 0x7f6d1c134840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:47.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.562+0000 7f6d1affd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d1c0a0950 0x7f6d1c133dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:49242/0 (socket says 172.21.15.16:49242) 2023-09-30T22:29:47.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.562+0000 7f6d1affd700 1 -- 172.21.15.16:0/3861862900 learned_addr learned my addr 172.21.15.16:0/3861862900 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:47.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.562+0000 7f6d1a7fc700 1 --2- 172.21.15.16:0/3861862900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d1c0a1430 0x7f6d1c134300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:47.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.562+0000 7f6d1affd700 1 -- 172.21.15.16:0/3861862900 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d1c138bd0 msgr2=0x7f6d1c134840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:47.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.562+0000 7f6d1affd700 1 --2- 172.21.15.16:0/3861862900 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d1c138bd0 0x7f6d1c134840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.562+0000 7f6d1affd700 1 -- 172.21.15.16:0/3861862900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d1c0a1430 msgr2=0x7f6d1c134300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:47.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.562+0000 7f6d1affd700 1 --2- 172.21.15.16:0/3861862900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d1c0a1430 0x7f6d1c134300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.562+0000 7f6d1affd700 1 -- 172.21.15.16:0/3861862900 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d14009b30 con 0x7f6d1c0a0950 2023-09-30T22:29:47.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.562+0000 7f6d1affd700 1 --2- 172.21.15.16:0/3861862900 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d1c0a0950 0x7f6d1c133dc0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c001eb0 tx=0x7f6d0c00c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:47.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.562+0000 7f6d037fe700 1 -- 172.21.15.16:0/3861862900 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c0054e0 con 0x7f6d1c0a0950 2023-09-30T22:29:47.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.562+0000 7f6d037fe700 1 -- 172.21.15.16:0/3861862900 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d0c003680 con 0x7f6d1c0a0950 2023-09-30T22:29:47.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.562+0000 7f6d21440700 1 -- 172.21.15.16:0/3861862900 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d100094a0 con 0x7f6d1c0a0950 2023-09-30T22:29:47.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.562+0000 7f6d037fe700 1 -- 172.21.15.16:0/3861862900 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c0039f0 con 0x7f6d1c0a0950 2023-09-30T22:29:47.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.563+0000 7f6d21440700 1 -- 172.21.15.16:0/3861862900 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d1c137220 con 0x7f6d1c0a0950 2023-09-30T22:29:47.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.563+0000 7f6d037fe700 1 -- 172.21.15.16:0/3861862900 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f6d0c003b50 con 0x7f6d1c0a0950 2023-09-30T22:29:47.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.563+0000 7f6d21440700 1 -- 172.21.15.16:0/3861862900 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d1c132120 con 0x7f6d1c0a0950 2023-09-30T22:29:47.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.564+0000 7f6d1a7fc700 1 -- 172.21.15.16:0/3861862900 >> v1:172.21.15.16:6816/126811 conn(0x7f6d0802f510 legacy=0x7f6d080319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:47.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.564+0000 7f6d037fe700 1 -- 172.21.15.16:0/3861862900 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7f6d0c03ef40 con 0x7f6d1c0a0950 2023-09-30T22:29:47.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.568+0000 7f6d037fe700 1 -- 172.21.15.16:0/3861862900 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6d0c01b3b0 con 0x7f6d1c0a0950 2023-09-30T22:29:47.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.765+0000 7f6d1a7fc700 1 -- 172.21.15.16:0/3861862900 >> v1:172.21.15.16:6816/126811 conn(0x7f6d0802f510 legacy=0x7f6d080319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:47.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.772+0000 7f6d21440700 1 -- 172.21.15.16:0/3861862900 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6d1c04f2a0 con 0x7f6d1c0a0950 2023-09-30T22:29:47.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.773+0000 7f6d037fe700 1 -- 172.21.15.16:0/3861862900 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+13014 (secure 0 0 0) 0x7f6d0c017030 con 0x7f6d1c0a0950 2023-09-30T22:29:47.773 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:29:47.773 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":39,"fsid":"7ede3e14-2bd0-4726-9437-6d13c90cd342","created":"2023-09-30T22:26:35.483697+0000","modified":"2023-09-30T22:29:46.581239+0000","last_up_change":"2023-09-30T22:29:35.702639+0000","last_in_change":"2023-09-30T22:26:37.120906+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T22:26:44.285133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{},"mgr_devicehealth":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:26:45.132921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:26:49.353683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:26:50.655685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"19946985-0ee2-487f-9347-929670dbe5ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":22,"up_from":28,"up_thru":32,"down_at":23,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6804","nonce":129539}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6805","nonce":129539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6807","nonce":129539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6806","nonce":129539}]},"public_addr":"172.21.15.16:6804/129539","cluster_addr":"172.21.15.16:6805/129539","heartbeat_back_addr":"172.21.15.16:6807/129539","heartbeat_front_addr":"172.21.15.16:6806/129539","state":["exists","up"]},{"osd":1,"uuid":"8efd4dbc-6ae4-4b82-b530-3b3eac2be445","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":22,"up_from":29,"up_thru":32,"down_at":24,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6808","nonce":129637}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6809","nonce":129637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6811","nonce":129637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6810","nonce":129637}]},"public_addr":"172.21.15.16:6808/129637","cluster_addr":"172.21.15.16:6809/129637","heartbeat_back_addr":"172.21.15.16:6811/129637","heartbeat_front_addr":"172.21.15.16:6810/129637","state":["exists","up"]},{"osd":2,"uuid":"b47bb1ce-27af-4c0c-868a-b76ebd0d9499","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":24,"up_from":31,"up_thru":32,"down_at":25,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6800","nonce":129747}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6801","nonce":129747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6803","nonce":129747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6802","nonce":129747}]},"public_addr":"172.21.15.16:6800/129747","cluster_addr":"172.21.15.16:6801/129747","heartbeat_back_addr":"172.21.15.16:6803/129747","heartbeat_front_addr":"172.21.15.16:6802/129747","state":["exists","up"]},{"osd":3,"uuid":"03b53d64-d10d-448a-a081-39ecceab589b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":25,"up_from":32,"up_thru":32,"down_at":26,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6812","nonce":130087}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6813","nonce":130087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6815","nonce":130087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6814","nonce":130087}]},"public_addr":"172.21.15.16:6812/130087","cluster_addr":"172.21.15.16:6813/130087","heartbeat_back_addr":"172.21.15.16:6815/130087","heartbeat_front_addr":"172.21.15.16:6814/130087","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.305953+0000","dead_epoch":23},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.307488+0000","dead_epoch":24},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.341023+0000","dead_epoch":25},{"osd":3,"down_stamp":"2023-09-30T22:29:28.519844+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.192071+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.8","mappings":[{"from":3,"to":2}]},{"pgid":"4.d","mappings":[{"from":3,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":0}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]},{"pgid":"4.17","mappings":[{"from":3,"to":2}]},{"pgid":"4.19","mappings":[{"from":3,"to":0}]},{"pgid":"4.3a","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6817/1424366610":"2023-10-01T22:29:40.259230+0000","172.21.15.16:6816/118285":"2023-10-01T22:29:14.475978+0000","172.21.15.16:0/2249754484":"2023-10-01T22:29:14.475978+0000","172.21.15.16:0/1101648925":"2023-10-01T22:29:14.475978+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:29:47.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.775+0000 7f6d21440700 1 -- 172.21.15.16:0/3861862900 >> v1:172.21.15.16:6816/126811 conn(0x7f6d0802f510 legacy=0x7f6d080319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:47.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.775+0000 7f6d21440700 1 -- 172.21.15.16:0/3861862900 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d1c0a0950 msgr2=0x7f6d1c133dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:47.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.775+0000 7f6d21440700 1 --2- 172.21.15.16:0/3861862900 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d1c0a0950 0x7f6d1c133dc0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c001eb0 tx=0x7f6d0c00c800 comp rx=0 tx=0).stop 2023-09-30T22:29:47.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.775+0000 7f6d21440700 1 -- 172.21.15.16:0/3861862900 shutdown_connections 2023-09-30T22:29:47.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.775+0000 7f6d21440700 1 --2- 172.21.15.16:0/3861862900 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d1c0a0950 0x7f6d1c133dc0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.775+0000 7f6d21440700 1 --2- 172.21.15.16:0/3861862900 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d1c138bd0 0x7f6d1c134840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.775+0000 7f6d21440700 1 --2- 172.21.15.16:0/3861862900 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d1c0a1430 0x7f6d1c134300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.775+0000 7f6d21440700 1 -- 172.21.15.16:0/3861862900 >> 172.21.15.16:0/3861862900 conn(0x7f6d1c0a2f70 msgr2=0x7f6d1c138440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:47.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.775+0000 7f6d21440700 1 -- 172.21.15.16:0/3861862900 shutdown_connections 2023-09-30T22:29:47.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.775+0000 7f6d21440700 1 -- 172.21.15.16:0/3861862900 wait complete. 2023-09-30T22:29:47.790 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-09-30T22:29:47.791 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-09-30T22:29:47.791 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-09-30T22:29:47.792 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-09-30T22:29:47.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.938+0000 7f2810f05700 1 Processor -- start 2023-09-30T22:29:47.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.938+0000 7f2810f05700 1 -- start start 2023-09-30T22:29:47.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.939+0000 7f2810f05700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f280c0a0a30 0x7f280c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:47.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.939+0000 7f2810f05700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f280c138d80 0x7f280c13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:47.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.939+0000 7f2810f05700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f280c0a2a70 con 0x7f280c09d630 2023-09-30T22:29:47.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.939+0000 7f2810f05700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f280c0a01d0 con 0x7f280c138d80 2023-09-30T22:29:47.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.939+0000 7f2810f05700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f280c09e2b0 con 0x7f280c0a0a30 2023-09-30T22:29:47.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.939+0000 7f280a59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f280c0a0a30 0x7f280c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:47.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.940+0000 7f280a59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f280c0a0a30 0x7f280c0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:51864/0 (socket says 172.21.15.16:51864) 2023-09-30T22:29:47.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.940+0000 7f280a59c700 1 -- 172.21.15.16:0/172301913 learned_addr learned my addr 172.21.15.16:0/172301913 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:47.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.940+0000 7f2809d9b700 1 --1- 172.21.15.16:0/172301913 >> v1:172.21.15.16:6789/0 conn(0x7f280c09d630 0x7f280c0a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59090/0 (socket says 172.21.15.16:59090) 2023-09-30T22:29:47.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.938+0000 7f651f0eb700 1 Processor -- start 2023-09-30T22:29:47.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.938+0000 7f651f0eb700 1 -- start start 2023-09-30T22:29:47.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.939+0000 7f651f0eb700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f651809e930 0x7f651809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:47.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.939+0000 7f651f0eb700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f651809f240 0x7f65181317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:47.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.939+0000 7f651f0eb700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f651809f650 0x7f6518131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:47.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.939+0000 7f651f0eb700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6518070940 con 0x7f651809f650 2023-09-30T22:29:47.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.939+0000 7f651f0eb700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f651806fd60 con 0x7f651809e930 2023-09-30T22:29:47.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.939+0000 7f651f0eb700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f651806fa80 con 0x7f651809f240 2023-09-30T22:29:47.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.940+0000 7f651ce87700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f651809e930 0x7f651809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:47.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.940+0000 7f651ce87700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f651809e930 0x7f651809ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:49248/0 (socket says 172.21.15.16:49248) 2023-09-30T22:29:47.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.940+0000 7f651ce87700 1 -- 172.21.15.16:0/3617632567 learned_addr learned my addr 172.21.15.16:0/3617632567 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:47.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.940+0000 7f651d688700 1 --2- 172.21.15.16:0/3617632567 >> v2:172.21.15.16:3300/0 conn(0x7f651809f650 0x7f6518131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:47.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.940+0000 7f6517fff700 1 --2- 172.21.15.16:0/3617632567 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f651809f240 0x7f65181317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:47.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.940+0000 7f651ce87700 1 -- 172.21.15.16:0/3617632567 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f651809f240 msgr2=0x7f65181317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:47.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.940+0000 7f651ce87700 1 --2- 172.21.15.16:0/3617632567 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f651809f240 0x7f65181317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.940+0000 7f651ce87700 1 -- 172.21.15.16:0/3617632567 >> v2:172.21.15.16:3300/0 conn(0x7f651809f650 msgr2=0x7f6518131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:47.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.940+0000 7f651ce87700 1 --2- 172.21.15.16:0/3617632567 >> v2:172.21.15.16:3300/0 conn(0x7f651809f650 0x7f6518131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.940+0000 7f651ce87700 1 -- 172.21.15.16:0/3617632567 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f651809fb60 con 0x7f651809e930 2023-09-30T22:29:47.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.940+0000 7f280ad9d700 1 --2- 172.21.15.16:0/172301913 >> v2:172.21.15.16:3300/0 conn(0x7f280c138d80 0x7f280c13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:47.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.940+0000 7f651d688700 1 --2- 172.21.15.16:0/3617632567 >> v2:172.21.15.16:3300/0 conn(0x7f651809f650 0x7f6518131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:29:47.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.940+0000 7f651ce87700 1 --2- 172.21.15.16:0/3617632567 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f651809e930 0x7f651809ed00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f65100098f0 tx=0x7f6510009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11d336723beaa889 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:47.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.940+0000 7f280a59c700 1 -- 172.21.15.16:0/172301913 >> v2:172.21.15.16:3300/0 conn(0x7f280c138d80 msgr2=0x7f280c13b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:47.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.940+0000 7f280a59c700 1 --2- 172.21.15.16:0/172301913 >> v2:172.21.15.16:3300/0 conn(0x7f280c138d80 0x7f280c13b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.940+0000 7f280a59c700 1 -- 172.21.15.16:0/172301913 >> v1:172.21.15.16:6789/0 conn(0x7f280c09d630 legacy=0x7f280c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:47.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.940+0000 7f280a59c700 1 -- 172.21.15.16:0/172301913 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f280c09dc50 con 0x7f280c0a0a30 2023-09-30T22:29:47.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.941+0000 7f65177fe700 1 -- 172.21.15.16:0/3617632567 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6510010a90 con 0x7f651809e930 2023-09-30T22:29:47.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.941+0000 7f65177fe700 1 -- 172.21.15.16:0/3617632567 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6510010bf0 con 0x7f651809e930 2023-09-30T22:29:47.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.941+0000 7f280a59c700 1 --2- 172.21.15.16:0/172301913 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f280c0a0a30 0x7f280c0a0e00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f27fc0098f0 tx=0x7f27fc009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc43ec84edaddfdf server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:47.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.941+0000 7f65177fe700 1 -- 172.21.15.16:0/3617632567 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6510018440 con 0x7f651809e930 2023-09-30T22:29:47.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.941+0000 7f280959a700 1 -- 172.21.15.16:0/172301913 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27fc004230 con 0x7f280c0a0a30 2023-09-30T22:29:47.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.941+0000 7f280959a700 1 -- 172.21.15.16:0/172301913 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27fc004390 con 0x7f280c0a0a30 2023-09-30T22:29:47.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.941+0000 7f280959a700 1 -- 172.21.15.16:0/172301913 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27fc0036a0 con 0x7f280c0a0a30 2023-09-30T22:29:47.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.942+0000 7f651f0eb700 1 -- 172.21.15.16:0/3617632567 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f651809e930 msgr2=0x7f651809ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:47.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.942+0000 7f651f0eb700 1 --2- 172.21.15.16:0/3617632567 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f651809e930 0x7f651809ed00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f65100098f0 tx=0x7f6510009c00 comp rx=0 tx=0).stop 2023-09-30T22:29:47.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.942+0000 7f651f0eb700 1 -- 172.21.15.16:0/3617632567 shutdown_connections 2023-09-30T22:29:47.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.942+0000 7f651f0eb700 1 --2- 172.21.15.16:0/3617632567 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f651809e930 0x7f651809ed00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.942+0000 7f651f0eb700 1 --2- 172.21.15.16:0/3617632567 >> v2:172.21.15.16:3300/0 conn(0x7f651809f650 0x7f6518131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.942+0000 7f651f0eb700 1 --2- 172.21.15.16:0/3617632567 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f651809f240 0x7f65181317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.942+0000 7f651f0eb700 1 -- 172.21.15.16:0/3617632567 >> 172.21.15.16:0/3617632567 conn(0x7f65180a4210 msgr2=0x7f65180a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:47.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.942+0000 7f651f0eb700 1 -- 172.21.15.16:0/3617632567 shutdown_connections 2023-09-30T22:29:47.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.942+0000 7f651f0eb700 1 -- 172.21.15.16:0/3617632567 wait complete. 2023-09-30T22:29:47.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.942+0000 7f2810f05700 1 -- 172.21.15.16:0/172301913 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f280c0a0a30 msgr2=0x7f280c0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:47.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.942+0000 7f2810f05700 1 --2- 172.21.15.16:0/172301913 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f280c0a0a30 0x7f280c0a0e00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f27fc0098f0 tx=0x7f27fc009c00 comp rx=0 tx=0).stop 2023-09-30T22:29:47.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.942+0000 7f2810f05700 1 -- 172.21.15.16:0/172301913 shutdown_connections 2023-09-30T22:29:47.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.942+0000 7f2810f05700 1 --2- 172.21.15.16:0/172301913 >> v2:172.21.15.16:3300/0 conn(0x7f280c138d80 0x7f280c13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.942+0000 7f2810f05700 1 --2- 172.21.15.16:0/172301913 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f280c0a0a30 0x7f280c0a0e00 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.942+0000 7f2810f05700 1 -- 172.21.15.16:0/172301913 >> 172.21.15.16:0/172301913 conn(0x7f280c0a3050 msgr2=0x7f280c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:47.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.943+0000 7f651f0eb700 1 Processor -- start 2023-09-30T22:29:47.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.943+0000 7f2810f05700 1 -- 172.21.15.16:0/172301913 shutdown_connections 2023-09-30T22:29:47.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.944+0000 7f2810f05700 1 -- 172.21.15.16:0/172301913 wait complete. 2023-09-30T22:29:47.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.943+0000 7f651f0eb700 1 -- start start 2023-09-30T22:29:47.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.944+0000 7f651f0eb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f651809e930 0x7f651811d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:47.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.944+0000 7f651f0eb700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f651809f240 0x7f651811d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:47.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.944+0000 7f651f0eb700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f651809f650 0x7f651811de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:47.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.944+0000 7f651f0eb700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6518071d20 con 0x7f651809e930 2023-09-30T22:29:47.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.944+0000 7f651f0eb700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f651811f410 con 0x7f651809f240 2023-09-30T22:29:47.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.944+0000 7f651f0eb700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f65181200b0 con 0x7f651809f650 2023-09-30T22:29:47.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.945+0000 7f651d688700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f651809f650 0x7f651811de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:47.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.945+0000 7f651d688700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f651809f650 0x7f651811de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:51872/0 (socket says 172.21.15.16:51872) 2023-09-30T22:29:47.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.945+0000 7f651d688700 1 -- 172.21.15.16:0/2371818742 learned_addr learned my addr 172.21.15.16:0/2371818742 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:47.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.945+0000 7f2810f05700 1 Processor -- start 2023-09-30T22:29:47.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.945+0000 7f651ce87700 1 --2- 172.21.15.16:0/2371818742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f651809e930 0x7f651811d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:47.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.945+0000 7f651d688700 1 -- 172.21.15.16:0/2371818742 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f651809f240 msgr2=0x7f651811d8e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:47.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.945+0000 7f651d688700 1 --2- 172.21.15.16:0/2371818742 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f651809f240 0x7f651811d8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.945+0000 7f651d688700 1 -- 172.21.15.16:0/2371818742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f651809e930 msgr2=0x7f651811d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:47.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.945+0000 7f651d688700 1 --2- 172.21.15.16:0/2371818742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f651809e930 0x7f651811d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.945+0000 7f651d688700 1 -- 172.21.15.16:0/2371818742 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6510009550 con 0x7f651809f650 2023-09-30T22:29:47.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.945+0000 7f651ce87700 1 --2- 172.21.15.16:0/2371818742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f651809e930 0x7f651811d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:29:47.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.946+0000 7f651d688700 1 --2- 172.21.15.16:0/2371818742 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f651809f650 0x7f651811de20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f650c00e810 tx=0x7f650c00ebd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:47.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.946+0000 7f65157fa700 1 -- 172.21.15.16:0/2371818742 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f650c013610 con 0x7f651809f650 2023-09-30T22:29:47.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.946+0000 7f65157fa700 1 -- 172.21.15.16:0/2371818742 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f650c013770 con 0x7f651809f650 2023-09-30T22:29:47.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.946+0000 7f65157fa700 1 -- 172.21.15.16:0/2371818742 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f650c013b00 con 0x7f651809f650 2023-09-30T22:29:47.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.946+0000 7f651f0eb700 1 -- 172.21.15.16:0/2371818742 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65180a0190 con 0x7f651809f650 2023-09-30T22:29:47.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.946+0000 7f651f0eb700 1 -- 172.21.15.16:0/2371818742 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6518121500 con 0x7f651809f650 2023-09-30T22:29:47.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.946+0000 7f2810f05700 1 -- start start 2023-09-30T22:29:47.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.946+0000 7f2810f05700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f280c09d630 0x7f280c11c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:47.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.946+0000 7f2810f05700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f280c0a0a30 0x7f280c11cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:47.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.946+0000 7f2810f05700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f280c138d80 0x7f280c11d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:47.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.946+0000 7f2810f05700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f280c11fa80 con 0x7f280c09d630 2023-09-30T22:29:47.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.946+0000 7f2810f05700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f280c11ebc0 con 0x7f280c0a0a30 2023-09-30T22:29:47.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.946+0000 7f2810f05700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f280c0a4230 con 0x7f280c138d80 2023-09-30T22:29:47.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.947+0000 7f280a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f280c09d630 0x7f280c11c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:47.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.947+0000 7f280a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f280c09d630 0x7f280c11c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47848/0 (socket says 172.21.15.16:47848) 2023-09-30T22:29:47.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.947+0000 7f280a59c700 1 -- 172.21.15.16:0/848120298 learned_addr learned my addr 172.21.15.16:0/848120298 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:47.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.947+0000 7f2809d9b700 1 --2- 172.21.15.16:0/848120298 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f280c0a0a30 0x7f280c11cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:47.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.947+0000 7f280a59c700 1 -- 172.21.15.16:0/848120298 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f280c138d80 msgr2=0x7f280c11d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:47.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.947+0000 7f280a59c700 1 --2- 172.21.15.16:0/848120298 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f280c138d80 0x7f280c11d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.947+0000 7f280a59c700 1 -- 172.21.15.16:0/848120298 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f280c0a0a30 msgr2=0x7f280c11cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:47.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.947+0000 7f280a59c700 1 --2- 172.21.15.16:0/848120298 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f280c0a0a30 0x7f280c11cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.947+0000 7f280a59c700 1 -- 172.21.15.16:0/848120298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27fc009550 con 0x7f280c09d630 2023-09-30T22:29:47.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.948+0000 7f65157fa700 1 -- 172.21.15.16:0/2371818742 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f650c013c60 con 0x7f651809f650 2023-09-30T22:29:47.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.948+0000 7f651f0eb700 1 -- 172.21.15.16:0/2371818742 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f65180a0010 con 0x7f651809f650 2023-09-30T22:29:47.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.948+0000 7f280a59c700 1 --2- 172.21.15.16:0/848120298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f280c09d630 0x7f280c11c7c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f280c09e3f0 tx=0x7f27fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:47.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.948+0000 7f651ce87700 1 -- 172.21.15.16:0/2371818742 >> v1:172.21.15.16:6816/126811 conn(0x7f6504037d70 legacy=0x7f650403a220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:47.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.948+0000 7f65157fa700 1 -- 172.21.15.16:0/2371818742 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7f650c03fae0 con 0x7f651809f650 2023-09-30T22:29:47.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.948+0000 7f27faffd700 1 -- 172.21.15.16:0/848120298 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27fc003de0 con 0x7f280c09d630 2023-09-30T22:29:47.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.948+0000 7f27faffd700 1 -- 172.21.15.16:0/848120298 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27fc011630 con 0x7f280c09d630 2023-09-30T22:29:47.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.948+0000 7f27faffd700 1 -- 172.21.15.16:0/848120298 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27fc0119e0 con 0x7f280c09d630 2023-09-30T22:29:47.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.948+0000 7f65157fa700 1 -- 172.21.15.16:0/2371818742 --> v1:172.21.15.16:6808/129637 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6504044c40 con 0x7f6504040f40 2023-09-30T22:29:47.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.948+0000 7f65157fa700 1 -- 172.21.15.16:0/2371818742 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f650c0442a0 con 0x7f651809f650 2023-09-30T22:29:47.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.948+0000 7f2810f05700 1 -- 172.21.15.16:0/848120298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f280c0a0410 con 0x7f280c09d630 2023-09-30T22:29:47.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.948+0000 7f2810f05700 1 -- 172.21.15.16:0/848120298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f280c11eda0 con 0x7f280c09d630 2023-09-30T22:29:47.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.949+0000 7f2810f05700 1 -- 172.21.15.16:0/848120298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f27f0000fb0 con 0x7f280c09d630 2023-09-30T22:29:47.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.949+0000 7f27faffd700 1 -- 172.21.15.16:0/848120298 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f27fc022020 con 0x7f280c09d630 2023-09-30T22:29:47.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.950+0000 7f27faffd700 1 -- 172.21.15.16:0/848120298 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7f27fc0412a0 con 0x7f280c09d630 2023-09-30T22:29:47.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.950+0000 7f2809d9b700 1 -- 172.21.15.16:0/848120298 >> v1:172.21.15.16:6816/126811 conn(0x7f27ec02f560 legacy=0x7f27ec031a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:47.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.950+0000 7f27faffd700 1 -- 172.21.15.16:0/848120298 --> v1:172.21.15.16:6800/129747 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f27ec03c430 con 0x7f27ec038730 2023-09-30T22:29:47.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.950+0000 7f27faffd700 1 -- 172.21.15.16:0/848120298 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27fc043780 con 0x7f280c09d630 2023-09-30T22:29:47.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.951+0000 7f65157fa700 1 -- 172.21.15.16:0/2371818742 <== osd.1 v1:172.21.15.16:6808/129637 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (unknown 0 0 75535769) 0x7f6504044c40 con 0x7f6504040f40 2023-09-30T22:29:47.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.951+0000 7f27faffd700 1 -- 172.21.15.16:0/848120298 <== osd.2 v1:172.21.15.16:6800/129747 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (unknown 0 0 75535769) 0x7f27ec03c430 con 0x7f27ec038730 2023-09-30T22:29:47.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.952+0000 7f249afce700 1 Processor -- start 2023-09-30T22:29:47.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.952+0000 7f249afce700 1 -- start start 2023-09-30T22:29:47.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.952+0000 7f249afce700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f249409d6f0 0x7f249409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:47.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.952+0000 7f249afce700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f249409e000 0x7f2494130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:47.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.952+0000 7f249afce700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f249409f130 con 0x7f249409e410 2023-09-30T22:29:47.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.952+0000 7f249afce700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f249409f520 con 0x7f249409d6f0 2023-09-30T22:29:47.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.952+0000 7f249afce700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f249409e930 con 0x7f249409e000 2023-09-30T22:29:47.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.952+0000 7f2493fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f249409e000 0x7f2494130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:47.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.953+0000 7f2493fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f249409e000 0x7f2494130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:49266/0 (socket says 172.21.15.16:49266) 2023-09-30T22:29:47.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.953+0000 7f2493fff700 1 -- 172.21.15.16:0/4277972666 learned_addr learned my addr 172.21.15.16:0/4277972666 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:47.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.953+0000 7f2498d6a700 1 --2- 172.21.15.16:0/4277972666 >> v2:172.21.15.16:3300/0 conn(0x7f249409d6f0 0x7f249409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:47.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.953+0000 7f2493fff700 1 -- 172.21.15.16:0/4277972666 >> v2:172.21.15.16:3300/0 conn(0x7f249409d6f0 msgr2=0x7f249409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:47.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.953+0000 7f2493fff700 1 --2- 172.21.15.16:0/4277972666 >> v2:172.21.15.16:3300/0 conn(0x7f249409d6f0 0x7f249409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.953+0000 7f2493fff700 1 -- 172.21.15.16:0/4277972666 >> v1:172.21.15.16:6789/0 conn(0x7f249409e410 legacy=0x7f2494130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:47.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.953+0000 7f2493fff700 1 -- 172.21.15.16:0/4277972666 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f249409b2d0 con 0x7f249409e000 2023-09-30T22:29:47.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.953+0000 7f2493fff700 1 --2- 172.21.15.16:0/4277972666 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f249409e000 0x7f2494130660 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f24800098f0 tx=0x7f2480009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=915b10519daeccb3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:47.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.953+0000 7f308b923700 1 Processor -- start 2023-09-30T22:29:47.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.954+0000 7f308b923700 1 -- start start 2023-09-30T22:29:47.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.954+0000 7f308b923700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f30840984f0 0x7f30840988c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:47.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.954+0000 7f308b923700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f308409d6f0 0x7f308409db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:47.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.954+0000 7f308b923700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f308409e040 0x7f30841385c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:47.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.954+0000 7f308b923700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3084099620 con 0x7f30840984f0 2023-09-30T22:29:47.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.954+0000 7f308b923700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f308409aca0 con 0x7f308409d6f0 2023-09-30T22:29:47.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.954+0000 7f308b923700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f308409a9e0 con 0x7f308409e040 2023-09-30T22:29:47.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.955+0000 7f30896bf700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f30840984f0 0x7f30840988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:47.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.955+0000 7f30896bf700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f30840984f0 0x7f30840988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47858/0 (socket says 172.21.15.16:47858) 2023-09-30T22:29:47.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.955+0000 7f30896bf700 1 -- 172.21.15.16:0/290102386 learned_addr learned my addr 172.21.15.16:0/290102386 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:47.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.954+0000 7f24937fe700 1 -- 172.21.15.16:0/4277972666 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2480004230 con 0x7f249409e000 2023-09-30T22:29:47.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.954+0000 7f24937fe700 1 -- 172.21.15.16:0/4277972666 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2480004390 con 0x7f249409e000 2023-09-30T22:29:47.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.954+0000 7f24937fe700 1 -- 172.21.15.16:0/4277972666 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24800036a0 con 0x7f249409e000 2023-09-30T22:29:47.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.955+0000 7f3088ebe700 1 --2- 172.21.15.16:0/290102386 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f308409d6f0 0x7f308409db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:47.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.955+0000 7f30896bf700 1 -- 172.21.15.16:0/290102386 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f308409e040 msgr2=0x7f30841385c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:47.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.955+0000 7f30896bf700 1 --2- 172.21.15.16:0/290102386 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f308409e040 0x7f30841385c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.955+0000 7f30896bf700 1 -- 172.21.15.16:0/290102386 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f308409d6f0 msgr2=0x7f308409db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:47.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.955+0000 7f30896bf700 1 --2- 172.21.15.16:0/290102386 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f308409d6f0 0x7f308409db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.955+0000 7f30896bf700 1 -- 172.21.15.16:0/290102386 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f308409bce0 con 0x7f30840984f0 2023-09-30T22:29:47.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.955+0000 7f3088ebe700 1 --2- 172.21.15.16:0/290102386 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f308409d6f0 0x7f308409db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:29:47.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.955+0000 7f30896bf700 1 --2- 172.21.15.16:0/290102386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30840984f0 0x7f30840988c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f30800098f0 tx=0x7f3080009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1035d43e3b68b03 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:47.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.955+0000 7f307bfff700 1 -- 172.21.15.16:0/290102386 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f308000b8a0 con 0x7f30840984f0 2023-09-30T22:29:47.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.955+0000 7f307bfff700 1 -- 172.21.15.16:0/290102386 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f308000ba00 con 0x7f30840984f0 2023-09-30T22:29:47.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.955+0000 7f307bfff700 1 -- 172.21.15.16:0/290102386 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f308000bd90 con 0x7f30840984f0 2023-09-30T22:29:47.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.956+0000 7f249afce700 1 -- 172.21.15.16:0/4277972666 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f249409e000 msgr2=0x7f2494130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:47.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.956+0000 7f249afce700 1 --2- 172.21.15.16:0/4277972666 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f249409e000 0x7f2494130660 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f24800098f0 tx=0x7f2480009c00 comp rx=0 tx=0).stop 2023-09-30T22:29:47.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.956+0000 7f249afce700 1 -- 172.21.15.16:0/4277972666 shutdown_connections 2023-09-30T22:29:47.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.956+0000 7f249afce700 1 --2- 172.21.15.16:0/4277972666 >> v2:172.21.15.16:3300/0 conn(0x7f249409d6f0 0x7f249409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.956+0000 7f249afce700 1 --2- 172.21.15.16:0/4277972666 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f249409e000 0x7f2494130660 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.956+0000 7f249afce700 1 -- 172.21.15.16:0/4277972666 >> 172.21.15.16:0/4277972666 conn(0x7f2494066f50 msgr2=0x7f249409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:47.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.956+0000 7f249afce700 1 -- 172.21.15.16:0/4277972666 shutdown_connections 2023-09-30T22:29:47.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.956+0000 7f249afce700 1 -- 172.21.15.16:0/4277972666 wait complete. 2023-09-30T22:29:47.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.956+0000 7f308b923700 1 -- 172.21.15.16:0/290102386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30840984f0 msgr2=0x7f30840988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:47.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.956+0000 7f308b923700 1 --2- 172.21.15.16:0/290102386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30840984f0 0x7f30840988c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f30800098f0 tx=0x7f3080009c00 comp rx=0 tx=0).stop 2023-09-30T22:29:47.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.957+0000 7f308b923700 1 -- 172.21.15.16:0/290102386 shutdown_connections 2023-09-30T22:29:47.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.957+0000 7f308b923700 1 --2- 172.21.15.16:0/290102386 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f308409e040 0x7f30841385c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.957+0000 7f308b923700 1 --2- 172.21.15.16:0/290102386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f30840984f0 0x7f30840988c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.957+0000 7f308b923700 1 --2- 172.21.15.16:0/290102386 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f308409d6f0 0x7f308409db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.957+0000 7f308b923700 1 -- 172.21.15.16:0/290102386 >> 172.21.15.16:0/290102386 conn(0x7f308409f470 msgr2=0x7f30840a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:47.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.957+0000 7f308b923700 1 -- 172.21.15.16:0/290102386 shutdown_connections 2023-09-30T22:29:47.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.957+0000 7f249afce700 1 Processor -- start 2023-09-30T22:29:47.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.957+0000 7f249afce700 1 -- start start 2023-09-30T22:29:47.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.958+0000 7f249afce700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f249409d6f0 0x7f24941315f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:47.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.958+0000 7f249afce700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f249409e000 0x7f2494131b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:47.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.958+0000 7f249afce700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f249409e410 0x7f2494132070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:47.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.958+0000 7f249afce700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2494132f40 con 0x7f249409e000 2023-09-30T22:29:47.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.958+0000 7f249afce700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2494132d50 con 0x7f249409e410 2023-09-30T22:29:47.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.958+0000 7f249afce700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2494133320 con 0x7f249409d6f0 2023-09-30T22:29:47.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.958+0000 7f249956b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f249409e410 0x7f2494132070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:47.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.958+0000 7f2493fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f249409e000 0x7f2494131b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:47.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.958+0000 7f249956b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f249409e410 0x7f2494132070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:49290/0 (socket says 172.21.15.16:49290) 2023-09-30T22:29:47.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.958+0000 7f2498d6a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f249409d6f0 0x7f24941315f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:47.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.958+0000 7f249956b700 1 -- 172.21.15.16:0/2792577935 learned_addr learned my addr 172.21.15.16:0/2792577935 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:47.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.958+0000 7f308b923700 1 -- 172.21.15.16:0/290102386 wait complete. 2023-09-30T22:29:47.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.959+0000 7f308b923700 1 Processor -- start 2023-09-30T22:29:47.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.958+0000 7f2493fff700 1 -- 172.21.15.16:0/2792577935 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f249409d6f0 msgr2=0x7f24941315f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:47.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.958+0000 7f2493fff700 1 --2- 172.21.15.16:0/2792577935 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f249409d6f0 0x7f24941315f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.959+0000 7f2493fff700 1 -- 172.21.15.16:0/2792577935 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f249409e410 msgr2=0x7f2494132070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:47.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.959+0000 7f2493fff700 1 --2- 172.21.15.16:0/2792577935 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f249409e410 0x7f2494132070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.959+0000 7f2493fff700 1 -- 172.21.15.16:0/2792577935 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2480009550 con 0x7f249409e000 2023-09-30T22:29:47.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.959+0000 7f2493fff700 1 --2- 172.21.15.16:0/2792577935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f249409e000 0x7f2494131b30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f24940a3e80 tx=0x7f2480004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:47.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.959+0000 7f24917fa700 1 -- 172.21.15.16:0/2792577935 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2480003c30 con 0x7f249409e000 2023-09-30T22:29:47.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.959+0000 7f308b923700 1 -- start start 2023-09-30T22:29:47.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.959+0000 7f24917fa700 1 -- 172.21.15.16:0/2792577935 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2480003d90 con 0x7f249409e000 2023-09-30T22:29:47.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.959+0000 7f24917fa700 1 -- 172.21.15.16:0/2792577935 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2480011850 con 0x7f249409e000 2023-09-30T22:29:47.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.960+0000 7f308b923700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f30840984f0 0x7f308406d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:47.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.960+0000 7f308b923700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f308409d6f0 0x7f308406db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:47.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.960+0000 7f308b923700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f308409e040 0x7f308406e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:47.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.960+0000 7f308b923700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3084095c30 con 0x7f308409d6f0 2023-09-30T22:29:47.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.960+0000 7f308b923700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f308412ece0 con 0x7f308409e040 2023-09-30T22:29:47.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.960+0000 7f308b923700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3084094fb0 con 0x7f30840984f0 2023-09-30T22:29:47.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.959+0000 7f249afce700 1 -- 172.21.15.16:0/2792577935 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f249409c700 con 0x7f249409e000 2023-09-30T22:29:47.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.959+0000 7f249afce700 1 -- 172.21.15.16:0/2792577935 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2494134960 con 0x7f249409e000 2023-09-30T22:29:47.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.960+0000 7f249afce700 1 -- 172.21.15.16:0/2792577935 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f249409c000 con 0x7f249409e000 2023-09-30T22:29:47.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.960+0000 7f3088ebe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f308409d6f0 0x7f308406db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:47.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.960+0000 7f3088ebe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f308409d6f0 0x7f308406db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47878/0 (socket says 172.21.15.16:47878) 2023-09-30T22:29:47.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.960+0000 7f3088ebe700 1 -- 172.21.15.16:0/2763864449 learned_addr learned my addr 172.21.15.16:0/2763864449 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:47.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.961+0000 7f3089ec0700 1 --2- 172.21.15.16:0/2763864449 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f308409e040 0x7f308406e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:47.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.961+0000 7f30896bf700 1 --2- 172.21.15.16:0/2763864449 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f30840984f0 0x7f308406d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:47.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.961+0000 7f3088ebe700 1 -- 172.21.15.16:0/2763864449 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f30840984f0 msgr2=0x7f308406d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:47.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.961+0000 7f3088ebe700 1 --2- 172.21.15.16:0/2763864449 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f30840984f0 0x7f308406d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.961+0000 7f3088ebe700 1 -- 172.21.15.16:0/2763864449 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f308409e040 msgr2=0x7f308406e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:47.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.961+0000 7f3088ebe700 1 --2- 172.21.15.16:0/2763864449 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f308409e040 0x7f308406e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.961+0000 7f3088ebe700 1 -- 172.21.15.16:0/2763864449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3080009550 con 0x7f308409d6f0 2023-09-30T22:29:47.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.961+0000 7f3088ebe700 1 --2- 172.21.15.16:0/2763864449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f308409d6f0 0x7f308406db30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f307400e810 tx=0x7f307400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:47.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.961+0000 7f24917fa700 1 -- 172.21.15.16:0/2792577935 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f2480022070 con 0x7f249409e000 2023-09-30T22:29:47.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.961+0000 7f2498d6a700 1 -- 172.21.15.16:0/2792577935 >> v1:172.21.15.16:6816/126811 conn(0x7f247802f560 legacy=0x7f2478031a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:47.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.961+0000 7f24917fa700 1 -- 172.21.15.16:0/2792577935 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7f2480041510 con 0x7f249409e000 2023-09-30T22:29:47.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.961+0000 7f3079ffb700 1 -- 172.21.15.16:0/2763864449 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3074013610 con 0x7f308409d6f0 2023-09-30T22:29:47.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.962+0000 7f3079ffb700 1 -- 172.21.15.16:0/2763864449 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3074013770 con 0x7f308409d6f0 2023-09-30T22:29:47.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.962+0000 7f308b923700 1 -- 172.21.15.16:0/2763864449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3084094a70 con 0x7f308409d6f0 2023-09-30T22:29:47.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.962+0000 7f3079ffb700 1 -- 172.21.15.16:0/2763864449 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3074013b20 con 0x7f308409d6f0 2023-09-30T22:29:47.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.962+0000 7f24917fa700 1 -- 172.21.15.16:0/2792577935 --> v1:172.21.15.16:6812/130087 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f247803c4b0 con 0x7f24780387b0 2023-09-30T22:29:47.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.962+0000 7f24917fa700 1 -- 172.21.15.16:0/2792577935 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2480043ab0 con 0x7f249409e000 2023-09-30T22:29:47.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.962+0000 7f308b923700 1 -- 172.21.15.16:0/2763864449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30840954b0 con 0x7f308409d6f0 2023-09-30T22:29:47.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.963+0000 7f308b923700 1 -- 172.21.15.16:0/2763864449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f308409e490 con 0x7f308409d6f0 2023-09-30T22:29:47.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.963+0000 7f3079ffb700 1 -- 172.21.15.16:0/2763864449 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f3074013c80 con 0x7f308409d6f0 2023-09-30T22:29:47.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.963+0000 7f30896bf700 1 -- 172.21.15.16:0/2763864449 >> v1:172.21.15.16:6816/126811 conn(0x7f306802f560 legacy=0x7f3068031a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:47.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.963+0000 7f24917fa700 1 -- 172.21.15.16:0/2792577935 <== osd.3 v1:172.21.15.16:6812/130087 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (unknown 0 0 75535769) 0x7f247803c4b0 con 0x7f24780387b0 2023-09-30T22:29:47.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.963+0000 7f3079ffb700 1 -- 172.21.15.16:0/2763864449 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7f307401a760 con 0x7f308409d6f0 2023-09-30T22:29:47.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.964+0000 7f3079ffb700 1 -- 172.21.15.16:0/2763864449 --> v1:172.21.15.16:6804/129539 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f306803c3b0 con 0x7f30680386b0 2023-09-30T22:29:47.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.964+0000 7f3079ffb700 1 -- 172.21.15.16:0/2763864449 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f30740118e0 con 0x7f308409d6f0 2023-09-30T22:29:47.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.969+0000 7f3079ffb700 1 -- 172.21.15.16:0/2763864449 <== osd.0 v1:172.21.15.16:6804/129539 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (unknown 0 0 75535769) 0x7f306803c3b0 con 0x7f30680386b0 2023-09-30T22:29:47.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.973+0000 7f2810f05700 1 -- 172.21.15.16:0/848120298 --> v1:172.21.15.16:6800/129747 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f27f0002d00 con 0x7f27ec038730 2023-09-30T22:29:47.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.973+0000 7f651f0eb700 1 -- 172.21.15.16:0/2371818742 --> v1:172.21.15.16:6808/129637 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f65180704f0 con 0x7f6504040f40 2023-09-30T22:29:47.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.973+0000 7f27faffd700 1 -- 172.21.15.16:0/848120298 <== osd.2 v1:172.21.15.16:6800/129747 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 1985872850) 0x7f27f0002d00 con 0x7f27ec038730 2023-09-30T22:29:47.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.974+0000 7f65157fa700 1 -- 172.21.15.16:0/2371818742 <== osd.1 v1:172.21.15.16:6808/129637 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 277563570) 0x7f6508007330 con 0x7f6504040f40 2023-09-30T22:29:47.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.974+0000 7f2810f05700 1 -- 172.21.15.16:0/848120298 >> v1:172.21.15.16:6800/129747 conn(0x7f27ec038730 legacy=0x7f27ec03abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:47.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.975+0000 7f651f0eb700 1 -- 172.21.15.16:0/2371818742 >> v1:172.21.15.16:6808/129637 conn(0x7f6504040f40 legacy=0x7f65040433d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:47.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.975+0000 7f2810f05700 1 -- 172.21.15.16:0/848120298 >> v1:172.21.15.16:6816/126811 conn(0x7f27ec02f560 legacy=0x7f27ec031a10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:47.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.975+0000 7f2810f05700 1 -- 172.21.15.16:0/848120298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f280c09d630 msgr2=0x7f280c11c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:47.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.975+0000 7f2810f05700 1 --2- 172.21.15.16:0/848120298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f280c09d630 0x7f280c11c7c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f280c09e3f0 tx=0x7f27fc004750 comp rx=0 tx=0).stop 2023-09-30T22:29:47.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.975+0000 7f280ad9d700 1 -- 172.21.15.16:0/848120298 reap_dead start 2023-09-30T22:29:47.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.975+0000 7f2810f05700 1 -- 172.21.15.16:0/848120298 shutdown_connections 2023-09-30T22:29:47.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.975+0000 7f2810f05700 1 -- 172.21.15.16:0/848120298 >> 172.21.15.16:0/848120298 conn(0x7f280c0a3050 msgr2=0x7f280c09ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:47.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.975+0000 7f651f0eb700 1 -- 172.21.15.16:0/2371818742 >> v1:172.21.15.16:6816/126811 conn(0x7f6504037d70 legacy=0x7f650403a220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:47.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.975+0000 7f651f0eb700 1 -- 172.21.15.16:0/2371818742 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f651809f650 msgr2=0x7f651811de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:47.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.975+0000 7f651f0eb700 1 --2- 172.21.15.16:0/2371818742 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f651809f650 0x7f651811de20 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f650c00e810 tx=0x7f650c00ebd0 comp rx=0 tx=0).stop 2023-09-30T22:29:47.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.975+0000 7f651d688700 1 -- 172.21.15.16:0/2371818742 reap_dead start 2023-09-30T22:29:47.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.975+0000 7f651f0eb700 1 -- 172.21.15.16:0/2371818742 shutdown_connections 2023-09-30T22:29:47.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.975+0000 7f651f0eb700 1 -- 172.21.15.16:0/2371818742 >> 172.21.15.16:0/2371818742 conn(0x7f65180a4210 msgr2=0x7f65180a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:47.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.975+0000 7f2810f05700 1 -- 172.21.15.16:0/848120298 shutdown_connections 2023-09-30T22:29:47.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.975+0000 7f2810f05700 1 -- 172.21.15.16:0/848120298 wait complete. 2023-09-30T22:29:47.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.976+0000 7f651f0eb700 1 -- 172.21.15.16:0/2371818742 shutdown_connections 2023-09-30T22:29:47.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.976+0000 7f651f0eb700 1 -- 172.21.15.16:0/2371818742 wait complete. 2023-09-30T22:29:47.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.981+0000 7f249afce700 1 -- 172.21.15.16:0/2792577935 --> v1:172.21.15.16:6812/130087 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f249409c450 con 0x7f24780387b0 2023-09-30T22:29:47.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.982+0000 7f24917fa700 1 -- 172.21.15.16:0/2792577935 <== osd.3 v1:172.21.15.16:6812/130087 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 3191560019) 0x7f248c0056d0 con 0x7f24780387b0 2023-09-30T22:29:47.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.982+0000 7f249afce700 1 -- 172.21.15.16:0/2792577935 >> v1:172.21.15.16:6812/130087 conn(0x7f24780387b0 legacy=0x7f247803ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:47.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.982+0000 7f249afce700 1 -- 172.21.15.16:0/2792577935 >> v1:172.21.15.16:6816/126811 conn(0x7f247802f560 legacy=0x7f2478031a10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:47.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.982+0000 7f249afce700 1 -- 172.21.15.16:0/2792577935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f249409e000 msgr2=0x7f2494131b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:47.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.982+0000 7f249afce700 1 --2- 172.21.15.16:0/2792577935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f249409e000 0x7f2494131b30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f24940a3e80 tx=0x7f2480004750 comp rx=0 tx=0).stop 2023-09-30T22:29:47.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.982+0000 7f249956b700 1 -- 172.21.15.16:0/2792577935 reap_dead start 2023-09-30T22:29:47.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.982+0000 7f249afce700 1 -- 172.21.15.16:0/2792577935 shutdown_connections 2023-09-30T22:29:47.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.982+0000 7f249afce700 1 -- 172.21.15.16:0/2792577935 >> 172.21.15.16:0/2792577935 conn(0x7f2494066f50 msgr2=0x7f24940711b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:47.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.982+0000 7f249afce700 1 -- 172.21.15.16:0/2792577935 shutdown_connections 2023-09-30T22:29:47.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.982+0000 7f249afce700 1 -- 172.21.15.16:0/2792577935 wait complete. 2023-09-30T22:29:47.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.988+0000 7f308b923700 1 -- 172.21.15.16:0/2763864449 --> v1:172.21.15.16:6804/129539 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f30840997f0 con 0x7f30680386b0 2023-09-30T22:29:47.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.989+0000 7f3079ffb700 1 -- 172.21.15.16:0/2763864449 <== osd.0 v1:172.21.15.16:6804/129539 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 3232526981) 0x7f307c009aa0 con 0x7f30680386b0 2023-09-30T22:29:47.991 INFO:teuthology.orchestra.run.smithi016.stdout:133143986179 2023-09-30T22:29:47.992 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:29:47.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.990+0000 7f308b923700 1 -- 172.21.15.16:0/2763864449 >> v1:172.21.15.16:6804/129539 conn(0x7f30680386b0 legacy=0x7f306803ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:47.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.990+0000 7f308b923700 1 -- 172.21.15.16:0/2763864449 >> v1:172.21.15.16:6816/126811 conn(0x7f306802f560 legacy=0x7f3068031a10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:47.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.990+0000 7f308b923700 1 -- 172.21.15.16:0/2763864449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f308409d6f0 msgr2=0x7f308406db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:47.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.990+0000 7f308b923700 1 --2- 172.21.15.16:0/2763864449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f308409d6f0 0x7f308406db30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f307400e810 tx=0x7f307400eb20 comp rx=0 tx=0).stop 2023-09-30T22:29:47.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.990+0000 7f3089ec0700 1 -- 172.21.15.16:0/2763864449 reap_dead start 2023-09-30T22:29:47.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.990+0000 7f308b923700 1 -- 172.21.15.16:0/2763864449 shutdown_connections 2023-09-30T22:29:47.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.990+0000 7f308b923700 1 -- 172.21.15.16:0/2763864449 >> 172.21.15.16:0/2763864449 conn(0x7f308409f470 msgr2=0x7f30841b8b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:47.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.990+0000 7f308b923700 1 -- 172.21.15.16:0/2763864449 shutdown_connections 2023-09-30T22:29:47.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:47.990+0000 7f308b923700 1 -- 172.21.15.16:0/2763864449 wait complete. 2023-09-30T22:29:47.996 INFO:teuthology.orchestra.run.smithi016.stdout:124554051588 2023-09-30T22:29:47.996 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:29:47.999 INFO:teuthology.orchestra.run.smithi016.stdout:137438953475 2023-09-30T22:29:48.000 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:29:48.007 INFO:teuthology.orchestra.run.smithi016.stdout:120259084292 2023-09-30T22:29:48.007 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:29:48.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.140+0000 7fc0a6cdf700 1 Processor -- start 2023-09-30T22:29:48.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.140+0000 7fc0a6cdf700 1 -- start start 2023-09-30T22:29:48.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.141+0000 7fc0a6cdf700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc0a009d6f0 0x7fc0a009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:48.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.141+0000 7fc0a6cdf700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc0a009e410 0x7fc0a0134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:48.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.141+0000 7fc0a6cdf700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0a009b120 con 0x7fc0a009e000 2023-09-30T22:29:48.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.141+0000 7fc0a6cdf700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a006b460 con 0x7fc0a009d6f0 2023-09-30T22:29:48.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.141+0000 7fc0a6cdf700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a009a7e0 con 0x7fc0a009e410 2023-09-30T22:29:48.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.141+0000 7fc0a527c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc0a009e410 0x7fc0a0134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:48.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.141+0000 7fc0a4a7b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc0a009d6f0 0x7fc0a009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:48.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.141+0000 7fc0a4a7b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc0a009d6f0 0x7fc0a009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:49302/0 (socket says 172.21.15.16:49302) 2023-09-30T22:29:48.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.141+0000 7fc0a4a7b700 1 -- 172.21.15.16:0/320724105 learned_addr learned my addr 172.21.15.16:0/320724105 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:48.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.141+0000 7fc0a527c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc0a009e410 0x7fc0a0134100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:51912/0 (socket says 172.21.15.16:51912) 2023-09-30T22:29:48.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.141+0000 7fc0a527c700 1 -- 172.21.15.16:0/320724105 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc0a009d6f0 msgr2=0x7fc0a009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:48.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.141+0000 7fc0a527c700 1 --2- 172.21.15.16:0/320724105 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc0a009d6f0 0x7fc0a009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:48.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.141+0000 7fc0a527c700 1 -- 172.21.15.16:0/320724105 >> v1:172.21.15.16:6789/0 conn(0x7fc0a009e000 legacy=0x7fc0a01339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:48.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.141+0000 7fc0a527c700 1 -- 172.21.15.16:0/320724105 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0a009a500 con 0x7fc0a009e410 2023-09-30T22:29:48.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.142+0000 7fc0a4a7b700 1 --2- 172.21.15.16:0/320724105 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc0a009d6f0 0x7fc0a009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:48.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.142+0000 7fc0a527c700 1 --2- 172.21.15.16:0/320724105 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc0a009e410 0x7fc0a0134100 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc098009df0 tx=0x7fc09800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a14754249eafb2ac server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:48.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.142+0000 7fc09f7fe700 1 -- 172.21.15.16:0/320724105 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc09800e070 con 0x7fc0a009e410 2023-09-30T22:29:48.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.142+0000 7fc09f7fe700 1 -- 172.21.15.16:0/320724105 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc098003a20 con 0x7fc0a009e410 2023-09-30T22:29:48.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.142+0000 7fc09f7fe700 1 -- 172.21.15.16:0/320724105 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc098003db0 con 0x7fc0a009e410 2023-09-30T22:29:48.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.143+0000 7fc0a6cdf700 1 -- 172.21.15.16:0/320724105 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc0a009e410 msgr2=0x7fc0a0134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:48.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.143+0000 7fc0a6cdf700 1 --2- 172.21.15.16:0/320724105 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc0a009e410 0x7fc0a0134100 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc098009df0 tx=0x7fc09800b330 comp rx=0 tx=0).stop 2023-09-30T22:29:48.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.143+0000 7fc0a6cdf700 1 -- 172.21.15.16:0/320724105 shutdown_connections 2023-09-30T22:29:48.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.143+0000 7fc0a6cdf700 1 --2- 172.21.15.16:0/320724105 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc0a009e410 0x7fc0a0134100 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:48.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.143+0000 7fc0a6cdf700 1 --2- 172.21.15.16:0/320724105 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc0a009d6f0 0x7fc0a009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:48.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.143+0000 7fc0a6cdf700 1 -- 172.21.15.16:0/320724105 >> 172.21.15.16:0/320724105 conn(0x7fc0a0066f50 msgr2=0x7fc0a009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:48.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.143+0000 7fc0a6cdf700 1 -- 172.21.15.16:0/320724105 shutdown_connections 2023-09-30T22:29:48.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.143+0000 7fc0a6cdf700 1 -- 172.21.15.16:0/320724105 wait complete. 2023-09-30T22:29:48.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.144+0000 7fc0a6cdf700 1 Processor -- start 2023-09-30T22:29:48.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.144+0000 7fc0a6cdf700 1 -- start start 2023-09-30T22:29:48.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.145+0000 7fc0a6cdf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0a009d6f0 0x7fc0a0122780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:48.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.145+0000 7fc0a6cdf700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc0a009e000 0x7fc0a0122cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:48.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.145+0000 7fc0a6cdf700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc0a009e410 0x7fc0a0123200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:48.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.145+0000 7fc0a6cdf700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a0125910 con 0x7fc0a009d6f0 2023-09-30T22:29:48.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.145+0000 7fc0a6cdf700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a0124300 con 0x7fc0a009e410 2023-09-30T22:29:48.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.145+0000 7fc0a6cdf700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc0a00a1d90 con 0x7fc0a009e000 2023-09-30T22:29:48.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.145+0000 7fc09ffff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc0a009e000 0x7fc0a0122cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:48.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.145+0000 7fc09ffff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc0a009e000 0x7fc0a0122cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:51914/0 (socket says 172.21.15.16:51914) 2023-09-30T22:29:48.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.145+0000 7fc09ffff700 1 -- 172.21.15.16:0/1919401059 learned_addr learned my addr 172.21.15.16:0/1919401059 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:48.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.145+0000 7fc0a527c700 1 --2- 172.21.15.16:0/1919401059 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc0a009e410 0x7fc0a0123200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:48.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.145+0000 7fc0a4a7b700 1 --2- 172.21.15.16:0/1919401059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0a009d6f0 0x7fc0a0122780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:48.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.145+0000 7fc09ffff700 1 -- 172.21.15.16:0/1919401059 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc0a009e410 msgr2=0x7fc0a0123200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:48.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.145+0000 7fc09ffff700 1 --2- 172.21.15.16:0/1919401059 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc0a009e410 0x7fc0a0123200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:48.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.145+0000 7fc09ffff700 1 -- 172.21.15.16:0/1919401059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0a009d6f0 msgr2=0x7fc0a0122780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:48.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.145+0000 7fc09ffff700 1 --2- 172.21.15.16:0/1919401059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0a009d6f0 0x7fc0a0122780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:48.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.145+0000 7fc09ffff700 1 -- 172.21.15.16:0/1919401059 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc094009550 con 0x7fc0a009e000 2023-09-30T22:29:48.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.146+0000 7fc0a527c700 1 --2- 172.21.15.16:0/1919401059 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc0a009e410 0x7fc0a0123200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:48.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.146+0000 7fc09ffff700 1 --2- 172.21.15.16:0/1919401059 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc0a009e000 0x7fc0a0122cc0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc08c00c780 tx=0x7fc08c00ca90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:48.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.146+0000 7fc09d7fa700 1 -- 172.21.15.16:0/1919401059 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08c0054e0 con 0x7fc0a009e000 2023-09-30T22:29:48.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.146+0000 7fc09d7fa700 1 -- 172.21.15.16:0/1919401059 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc08c003680 con 0x7fc0a009e000 2023-09-30T22:29:48.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.146+0000 7fc0a6cdf700 1 -- 172.21.15.16:0/1919401059 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc098009a70 con 0x7fc0a009e000 2023-09-30T22:29:48.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.146+0000 7fc09d7fa700 1 -- 172.21.15.16:0/1919401059 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08c003a30 con 0x7fc0a009e000 2023-09-30T22:29:48.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.146+0000 7fc0a6cdf700 1 -- 172.21.15.16:0/1919401059 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0a0125c70 con 0x7fc0a009e000 2023-09-30T22:29:48.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.147+0000 7fc09d7fa700 1 -- 172.21.15.16:0/1919401059 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fc08c003b90 con 0x7fc0a009e000 2023-09-30T22:29:48.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.147+0000 7fc09d7fa700 1 -- 172.21.15.16:0/1919401059 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7fc08c03ef90 con 0x7fc0a009e000 2023-09-30T22:29:48.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.147+0000 7fc0a4a7b700 1 -- 172.21.15.16:0/1919401059 >> v1:172.21.15.16:6816/126811 conn(0x7fc08402f540 legacy=0x7fc0840319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:48.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.147+0000 7fc0a6cdf700 1 -- 172.21.15.16:0/1919401059 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0a009d1b0 con 0x7fc0a009e000 2023-09-30T22:29:48.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.152+0000 7f06b78ff700 1 Processor -- start 2023-09-30T22:29:48.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.152+0000 7f06b78ff700 1 -- start start 2023-09-30T22:29:48.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.153+0000 7fc09d7fa700 1 -- 172.21.15.16:0/1919401059 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc08c017020 con 0x7fc0a009e000 2023-09-30T22:29:48.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.153+0000 7f06b78ff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f06b009e1d0 0x7f06b0138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:48.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.153+0000 7f06b78ff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f06b0139170 0x7f06b013b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:48.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.153+0000 7f06b78ff700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06b009b120 con 0x7f06b009d6f0 2023-09-30T22:29:48.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.153+0000 7f06b78ff700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06b006b460 con 0x7f06b009e1d0 2023-09-30T22:29:48.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.153+0000 7f06b78ff700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f06b009a7e0 con 0x7f06b0139170 2023-09-30T22:29:48.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.153+0000 7f06b5e9c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f06b0139170 0x7f06b013b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:48.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.153+0000 7f06b5e9c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f06b0139170 0x7f06b013b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:51920/0 (socket says 172.21.15.16:51920) 2023-09-30T22:29:48.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.153+0000 7f06b5e9c700 1 -- 172.21.15.16:0/1340503843 learned_addr learned my addr 172.21.15.16:0/1340503843 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:48.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.153+0000 7f06b4e9a700 1 --2- 172.21.15.16:0/1340503843 >> v2:172.21.15.16:3300/0 conn(0x7f06b009e1d0 0x7f06b0138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:48.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.154+0000 7f06b5e9c700 1 -- 172.21.15.16:0/1340503843 >> v2:172.21.15.16:3300/0 conn(0x7f06b009e1d0 msgr2=0x7f06b0138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:48.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.154+0000 7f06b5e9c700 1 --2- 172.21.15.16:0/1340503843 >> v2:172.21.15.16:3300/0 conn(0x7f06b009e1d0 0x7f06b0138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:48.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.154+0000 7f06b5e9c700 1 -- 172.21.15.16:0/1340503843 >> v1:172.21.15.16:6789/0 conn(0x7f06b009d6f0 legacy=0x7f06b009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:48.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.154+0000 7f06b5e9c700 1 -- 172.21.15.16:0/1340503843 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06b009a500 con 0x7f06b0139170 2023-09-30T22:29:48.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.154+0000 7f06b4e9a700 1 --2- 172.21.15.16:0/1340503843 >> v2:172.21.15.16:3300/0 conn(0x7f06b009e1d0 0x7f06b0138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:29:48.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.154+0000 7f06b5e9c700 1 --2- 172.21.15.16:0/1340503843 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f06b0139170 0x7f06b013b550 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f06a8009e80 tx=0x7f06a800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74bf4dcd241e2441 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:48.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.154+0000 7f06a7fff700 1 -- 172.21.15.16:0/1340503843 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a800e070 con 0x7f06b0139170 2023-09-30T22:29:48.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.154+0000 7f06a7fff700 1 -- 172.21.15.16:0/1340503843 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a800b980 con 0x7f06b0139170 2023-09-30T22:29:48.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.154+0000 7f06a7fff700 1 -- 172.21.15.16:0/1340503843 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a800bd30 con 0x7f06b0139170 2023-09-30T22:29:48.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.155+0000 7f06b78ff700 1 -- 172.21.15.16:0/1340503843 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f06b0139170 msgr2=0x7f06b013b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:48.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.155+0000 7f06b78ff700 1 --2- 172.21.15.16:0/1340503843 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f06b0139170 0x7f06b013b550 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f06a8009e80 tx=0x7f06a800b330 comp rx=0 tx=0).stop 2023-09-30T22:29:48.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.155+0000 7f06b78ff700 1 -- 172.21.15.16:0/1340503843 shutdown_connections 2023-09-30T22:29:48.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.155+0000 7f06b78ff700 1 --2- 172.21.15.16:0/1340503843 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f06b0139170 0x7f06b013b550 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:48.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.155+0000 7f06b78ff700 1 --2- 172.21.15.16:0/1340503843 >> v2:172.21.15.16:3300/0 conn(0x7f06b009e1d0 0x7f06b0138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:48.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.155+0000 7f06b78ff700 1 -- 172.21.15.16:0/1340503843 >> 172.21.15.16:0/1340503843 conn(0x7f06b0066f50 msgr2=0x7f06b009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:48.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.155+0000 7f06b78ff700 1 -- 172.21.15.16:0/1340503843 shutdown_connections 2023-09-30T22:29:48.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.155+0000 7f06b78ff700 1 -- 172.21.15.16:0/1340503843 wait complete. 2023-09-30T22:29:48.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.157+0000 7f06b78ff700 1 Processor -- start 2023-09-30T22:29:48.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.157+0000 7f06b78ff700 1 -- start start 2023-09-30T22:29:48.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.157+0000 7f06b78ff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f06b009d6f0 0x7f06b011e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:48.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.157+0000 7f06b78ff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f06b009e1d0 0x7f06b011eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:48.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.157+0000 7f06b78ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06b0139170 0x7f06b011f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:48.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.157+0000 7f06b78ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b0120a40 con 0x7f06b0139170 2023-09-30T22:29:48.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.157+0000 7f06b78ff700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06b01340e0 con 0x7f06b009d6f0 2023-09-30T22:29:48.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.157+0000 7f06b78ff700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f06b01222f0 con 0x7f06b009e1d0 2023-09-30T22:29:48.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.158+0000 7f06b4e9a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f06b009e1d0 0x7f06b011eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:48.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.158+0000 7f06b4e9a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f06b009e1d0 0x7f06b011eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:51926/0 (socket says 172.21.15.16:51926) 2023-09-30T22:29:48.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.158+0000 7f06b4e9a700 1 -- 172.21.15.16:0/3686724542 learned_addr learned my addr 172.21.15.16:0/3686724542 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:48.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.158+0000 7f06b569b700 1 --2- 172.21.15.16:0/3686724542 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f06b009d6f0 0x7f06b011e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:48.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.158+0000 7f06b5e9c700 1 --2- 172.21.15.16:0/3686724542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06b0139170 0x7f06b011f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:48.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.158+0000 7f06b569b700 1 -- 172.21.15.16:0/3686724542 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f06b009e1d0 msgr2=0x7f06b011eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:48.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.158+0000 7f06b569b700 1 --2- 172.21.15.16:0/3686724542 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f06b009e1d0 0x7f06b011eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:48.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.158+0000 7f06b569b700 1 -- 172.21.15.16:0/3686724542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06b0139170 msgr2=0x7f06b011f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:48.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.158+0000 7f06b569b700 1 --2- 172.21.15.16:0/3686724542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06b0139170 0x7f06b011f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:48.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.158+0000 7f06b569b700 1 -- 172.21.15.16:0/3686724542 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a8009b30 con 0x7f06b009d6f0 2023-09-30T22:29:48.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.158+0000 7f06b569b700 1 --2- 172.21.15.16:0/3686724542 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f06b009d6f0 0x7f06b011e860 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f06ac000f60 tx=0x7f06ac00c730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:48.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.159+0000 7f06a5ffb700 1 -- 172.21.15.16:0/3686724542 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06ac0052d0 con 0x7f06b009d6f0 2023-09-30T22:29:48.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.159+0000 7f06a5ffb700 1 -- 172.21.15.16:0/3686724542 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06ac005430 con 0x7f06b009d6f0 2023-09-30T22:29:48.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.159+0000 7f06b78ff700 1 -- 172.21.15.16:0/3686724542 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06b01203e0 con 0x7f06b009d6f0 2023-09-30T22:29:48.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.159+0000 7f06a5ffb700 1 -- 172.21.15.16:0/3686724542 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06ac0038c0 con 0x7f06b009d6f0 2023-09-30T22:29:48.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.159+0000 7f06b78ff700 1 -- 172.21.15.16:0/3686724542 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06b0120bd0 con 0x7f06b009d6f0 2023-09-30T22:29:48.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.159+0000 7f06a5ffb700 1 -- 172.21.15.16:0/3686724542 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f06ac003a20 con 0x7f06b009d6f0 2023-09-30T22:29:48.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.160+0000 7f06a5ffb700 1 -- 172.21.15.16:0/3686724542 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7f06ac03ee90 con 0x7f06b009d6f0 2023-09-30T22:29:48.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.160+0000 7f06b4e9a700 1 -- 172.21.15.16:0/3686724542 >> v1:172.21.15.16:6816/126811 conn(0x7f069402f510 legacy=0x7f06940319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:48.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.160+0000 7f06b78ff700 1 -- 172.21.15.16:0/3686724542 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06b0074ea0 con 0x7f06b009d6f0 2023-09-30T22:29:48.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.165+0000 7f06a5ffb700 1 -- 172.21.15.16:0/3686724542 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f06ac06a750 con 0x7f06b009d6f0 2023-09-30T22:29:48.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.210+0000 7f19ab797700 1 Processor -- start 2023-09-30T22:29:48.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.210+0000 7f19ab797700 1 -- start start 2023-09-30T22:29:48.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.211+0000 7f19ab797700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f19a406a770 0x7f19a406ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:48.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.211+0000 7f19ab797700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f19a406b080 0x7f19a4131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:48.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.211+0000 7f19ab797700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19a409ece0 con 0x7f19a4131b40 2023-09-30T22:29:48.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.211+0000 7f19ab797700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19a409e760 con 0x7f19a406a770 2023-09-30T22:29:48.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.211+0000 7f19ab797700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19a409e040 con 0x7f19a406b080 2023-09-30T22:29:48.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.211+0000 7f19a8d32700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f19a406b080 0x7f19a4131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:48.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.211+0000 7f19a8d32700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f19a406b080 0x7f19a4131600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:49332/0 (socket says 172.21.15.16:49332) 2023-09-30T22:29:48.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.211+0000 7f19a8d32700 1 -- 172.21.15.16:0/2494151441 learned_addr learned my addr 172.21.15.16:0/2494151441 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:48.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.211+0000 7f19a9533700 1 --2- 172.21.15.16:0/2494151441 >> v2:172.21.15.16:3300/0 conn(0x7f19a406a770 0x7f19a406ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:48.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.212+0000 7f19a8d32700 1 -- 172.21.15.16:0/2494151441 >> v2:172.21.15.16:3300/0 conn(0x7f19a406a770 msgr2=0x7f19a406ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:48.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.212+0000 7f19a8d32700 1 --2- 172.21.15.16:0/2494151441 >> v2:172.21.15.16:3300/0 conn(0x7f19a406a770 0x7f19a406ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:48.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.212+0000 7f19a8d32700 1 -- 172.21.15.16:0/2494151441 >> v1:172.21.15.16:6789/0 conn(0x7f19a4131b40 legacy=0x7f19a4138850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:48.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.212+0000 7f19a8d32700 1 -- 172.21.15.16:0/2494151441 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19a409f630 con 0x7f19a406b080 2023-09-30T22:29:48.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.212+0000 7f19a8d32700 1 --2- 172.21.15.16:0/2494151441 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f19a406b080 0x7f19a4131600 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f19940098f0 tx=0x7f1994009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0a7c5004a50a3fb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:48.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.212+0000 7f199bfff700 1 -- 172.21.15.16:0/2494151441 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1994011830 con 0x7f19a406b080 2023-09-30T22:29:48.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.212+0000 7f199bfff700 1 -- 172.21.15.16:0/2494151441 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1994011990 con 0x7f19a406b080 2023-09-30T22:29:48.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.212+0000 7f199bfff700 1 -- 172.21.15.16:0/2494151441 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1994011d20 con 0x7f19a406b080 2023-09-30T22:29:48.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.213+0000 7f19ab797700 1 -- 172.21.15.16:0/2494151441 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f19a406b080 msgr2=0x7f19a4131600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:48.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.213+0000 7f19ab797700 1 --2- 172.21.15.16:0/2494151441 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f19a406b080 0x7f19a4131600 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f19940098f0 tx=0x7f1994009c00 comp rx=0 tx=0).stop 2023-09-30T22:29:48.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.213+0000 7f19ab797700 1 -- 172.21.15.16:0/2494151441 shutdown_connections 2023-09-30T22:29:48.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.213+0000 7f19ab797700 1 --2- 172.21.15.16:0/2494151441 >> v2:172.21.15.16:3300/0 conn(0x7f19a406a770 0x7f19a406ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:48.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.213+0000 7f19ab797700 1 --2- 172.21.15.16:0/2494151441 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f19a406b080 0x7f19a4131600 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:48.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.213+0000 7f19ab797700 1 -- 172.21.15.16:0/2494151441 >> 172.21.15.16:0/2494151441 conn(0x7f19a4066f50 msgr2=0x7f19a409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:48.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.213+0000 7f19ab797700 1 -- 172.21.15.16:0/2494151441 shutdown_connections 2023-09-30T22:29:48.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.213+0000 7f19ab797700 1 -- 172.21.15.16:0/2494151441 wait complete. 2023-09-30T22:29:48.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.214+0000 7f19ab797700 1 Processor -- start 2023-09-30T22:29:48.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.215+0000 7f19ab797700 1 -- start start 2023-09-30T22:29:48.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.215+0000 7f19ab797700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19a406a770 0x7f19a41c80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:48.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.215+0000 7f19ab797700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f19a406b080 0x7f19a41c8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:48.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.215+0000 7f19ab797700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f19a4131b40 0x7f19a41c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:48.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.215+0000 7f19ab797700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a4131680 con 0x7f19a406a770 2023-09-30T22:29:48.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.215+0000 7f19ab797700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19a4120d90 con 0x7f19a4131b40 2023-09-30T22:29:48.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.215+0000 7f19ab797700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f19a4071e60 con 0x7f19a406b080 2023-09-30T22:29:48.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.215+0000 7f19a9d34700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f19a4131b40 0x7f19a41c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:48.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.215+0000 7f19a9d34700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f19a4131b40 0x7f19a41c8b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:49348/0 (socket says 172.21.15.16:49348) 2023-09-30T22:29:48.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.215+0000 7f19a9d34700 1 -- 172.21.15.16:0/3964107883 learned_addr learned my addr 172.21.15.16:0/3964107883 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:48.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.215+0000 7f19a8d32700 1 --2- 172.21.15.16:0/3964107883 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f19a406b080 0x7f19a41c8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:48.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.216+0000 7fb2e5350700 1 Processor -- start 2023-09-30T22:29:48.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.216+0000 7f19a9533700 1 --2- 172.21.15.16:0/3964107883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19a406a770 0x7f19a41c80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:48.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.216+0000 7f19a9d34700 1 -- 172.21.15.16:0/3964107883 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f19a406b080 msgr2=0x7f19a41c8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:48.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.216+0000 7f19a9d34700 1 --2- 172.21.15.16:0/3964107883 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f19a406b080 0x7f19a41c8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:48.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.216+0000 7f19a9d34700 1 -- 172.21.15.16:0/3964107883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19a406a770 msgr2=0x7f19a41c80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:48.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.216+0000 7f19a9d34700 1 --2- 172.21.15.16:0/3964107883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19a406a770 0x7f19a41c80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:48.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.216+0000 7f19a9d34700 1 -- 172.21.15.16:0/3964107883 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1994009550 con 0x7f19a4131b40 2023-09-30T22:29:48.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.216+0000 7f19a9533700 1 --2- 172.21.15.16:0/3964107883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19a406a770 0x7f19a41c80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:29:48.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.216+0000 7fb2e5350700 1 -- start start 2023-09-30T22:29:48.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.216+0000 7f19a9d34700 1 --2- 172.21.15.16:0/3964107883 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f19a4131b40 0x7f19a41c8b70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f199c00cc50 tx=0x7f199c00a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:48.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.216+0000 7f1999ffb700 1 -- 172.21.15.16:0/3964107883 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f199c00ad80 con 0x7f19a4131b40 2023-09-30T22:29:48.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.216+0000 7f1999ffb700 1 -- 172.21.15.16:0/3964107883 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f199c00ce50 con 0x7f19a4131b40 2023-09-30T22:29:48.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.216+0000 7f1999ffb700 1 -- 172.21.15.16:0/3964107883 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f199c003c40 con 0x7f19a4131b40 2023-09-30T22:29:48.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.216+0000 7fb2e5350700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb2e009d6f0 0x7fb2e009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:48.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.217+0000 7fb2e5350700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb2e009e410 0x7fb2e0130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:48.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.217+0000 7fb2e5350700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2e009f130 con 0x7fb2e009e000 2023-09-30T22:29:48.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.217+0000 7fb2e5350700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e009f520 con 0x7fb2e009d6f0 2023-09-30T22:29:48.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.217+0000 7fb2e5350700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e009e930 con 0x7fb2e009e410 2023-09-30T22:29:48.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.217+0000 7fb2de7fc700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb2e009e000 0x7fb2e0130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59138/0 (socket says 172.21.15.16:59138) 2023-09-30T22:29:48.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.217+0000 7fb2de7fc700 1 -- 172.21.15.16:0/1443999747 learned_addr learned my addr 172.21.15.16:0/1443999747 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:29:48.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.217+0000 7fb2deffd700 1 --2- 172.21.15.16:0/1443999747 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb2e009d6f0 0x7fb2e009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:48.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.217+0000 7fb2df7fe700 1 --2- 172.21.15.16:0/1443999747 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb2e009e410 0x7fb2e0130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:48.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.217+0000 7f19ab797700 1 -- 172.21.15.16:0/3964107883 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19a41313b0 con 0x7f19a4131b40 2023-09-30T22:29:48.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.217+0000 7f19ab797700 1 -- 172.21.15.16:0/3964107883 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19a4122970 con 0x7f19a4131b40 2023-09-30T22:29:48.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.217+0000 7fb2ddffb700 1 -- 172.21.15.16:0/1443999747 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2572980649 0 0) 0x7fb2e009f130 con 0x7fb2e009e000 2023-09-30T22:29:48.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.217+0000 7fb2ddffb700 1 -- 172.21.15.16:0/1443999747 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2c80035b0 con 0x7fb2e009e000 2023-09-30T22:29:48.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.217+0000 7fb2deffd700 1 -- 172.21.15.16:0/1443999747 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb2e009e410 msgr2=0x7fb2e0130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:48.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.217+0000 7fb2deffd700 1 --2- 172.21.15.16:0/1443999747 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb2e009e410 0x7fb2e0130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:48.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.218+0000 7fb2deffd700 1 -- 172.21.15.16:0/1443999747 >> v1:172.21.15.16:6789/0 conn(0x7fb2e009e000 legacy=0x7fb2e0130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:48.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.218+0000 7fb2deffd700 1 -- 172.21.15.16:0/1443999747 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2e009b2d0 con 0x7fb2e009d6f0 2023-09-30T22:29:48.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.218+0000 7fb2deffd700 1 --2- 172.21.15.16:0/1443999747 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb2e009d6f0 0x7fb2e009dac0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0009870 tx=0x7fb2d0009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=898570f40961db23 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:48.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.218+0000 7fb2ddffb700 1 -- 172.21.15.16:0/1443999747 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d0011830 con 0x7fb2e009d6f0 2023-09-30T22:29:48.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.218+0000 7fb2ddffb700 1 -- 172.21.15.16:0/1443999747 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2d0011990 con 0x7fb2e009d6f0 2023-09-30T22:29:48.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.218+0000 7fb2ddffb700 1 -- 172.21.15.16:0/1443999747 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d0011d20 con 0x7fb2e009d6f0 2023-09-30T22:29:48.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.218+0000 7f1999ffb700 1 -- 172.21.15.16:0/3964107883 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f199c0050d0 con 0x7f19a4131b40 2023-09-30T22:29:48.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.219+0000 7f1999ffb700 1 -- 172.21.15.16:0/3964107883 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7f199c03fda0 con 0x7f19a4131b40 2023-09-30T22:29:48.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.219+0000 7f19a9533700 1 -- 172.21.15.16:0/3964107883 >> v1:172.21.15.16:6816/126811 conn(0x7f198802f510 legacy=0x7f19880319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:48.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.219+0000 7f19ab797700 1 -- 172.21.15.16:0/3964107883 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19a4074ea0 con 0x7f19a4131b40 2023-09-30T22:29:48.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.219+0000 7fb2e5350700 1 -- 172.21.15.16:0/1443999747 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb2e009d6f0 msgr2=0x7fb2e009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:48.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.219+0000 7fb2e5350700 1 --2- 172.21.15.16:0/1443999747 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb2e009d6f0 0x7fb2e009dac0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb2d0009870 tx=0x7fb2d0009b80 comp rx=0 tx=0).stop 2023-09-30T22:29:48.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.219+0000 7fb2e5350700 1 -- 172.21.15.16:0/1443999747 shutdown_connections 2023-09-30T22:29:48.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.219+0000 7fb2e5350700 1 --2- 172.21.15.16:0/1443999747 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb2e009e410 0x7fb2e0130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:48.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.219+0000 7fb2e5350700 1 --2- 172.21.15.16:0/1443999747 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb2e009d6f0 0x7fb2e009dac0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:48.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.219+0000 7fb2e5350700 1 -- 172.21.15.16:0/1443999747 >> 172.21.15.16:0/1443999747 conn(0x7fb2e0066f50 msgr2=0x7fb2e009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:48.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.220+0000 7fb2e5350700 1 -- 172.21.15.16:0/1443999747 shutdown_connections 2023-09-30T22:29:48.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.220+0000 7fb2e5350700 1 -- 172.21.15.16:0/1443999747 wait complete. 2023-09-30T22:29:48.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.221+0000 7fb2e5350700 1 Processor -- start 2023-09-30T22:29:48.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.221+0000 7fb2e5350700 1 -- start start 2023-09-30T22:29:48.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.221+0000 7fb2e5350700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb2e009d6f0 0x7fb2e011de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:48.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.221+0000 7fb2e5350700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb2e009e000 0x7fb2e011e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:48.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.221+0000 7fb2e5350700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2e009e410 0x7fb2e011e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:48.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.221+0000 7fb2e5350700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e011fba0 con 0x7fb2e009e410 2023-09-30T22:29:48.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.221+0000 7fb2e5350700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e011f9b0 con 0x7fb2e009e000 2023-09-30T22:29:48.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.221+0000 7fb2e5350700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e0120220 con 0x7fb2e009d6f0 2023-09-30T22:29:48.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.222+0000 7fb2df7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2e009e410 0x7fb2e011e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:48.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.222+0000 7fb2df7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2e009e410 0x7fb2e011e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47932/0 (socket says 172.21.15.16:47932) 2023-09-30T22:29:48.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.222+0000 7fb2df7fe700 1 -- 172.21.15.16:0/3752516436 learned_addr learned my addr 172.21.15.16:0/3752516436 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:48.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.222+0000 7fb2de7fc700 1 --2- 172.21.15.16:0/3752516436 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb2e009e000 0x7fb2e011e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:48.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.222+0000 7fb2df7fe700 1 -- 172.21.15.16:0/3752516436 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb2e009d6f0 msgr2=0x7fb2e011de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:48.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.222+0000 7fb2df7fe700 1 --2- 172.21.15.16:0/3752516436 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb2e009d6f0 0x7fb2e011de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:48.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.222+0000 7fb2df7fe700 1 -- 172.21.15.16:0/3752516436 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb2e009e000 msgr2=0x7fb2e011e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:48.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.222+0000 7fb2df7fe700 1 --2- 172.21.15.16:0/3752516436 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb2e009e000 0x7fb2e011e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:48.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.222+0000 7fb2df7fe700 1 -- 172.21.15.16:0/3752516436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d00094a0 con 0x7fb2e009e410 2023-09-30T22:29:48.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.223+0000 7fb2df7fe700 1 --2- 172.21.15.16:0/3752516436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2e009e410 0x7fb2e011e910 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8007360 tx=0x7fb2d8007670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:48.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.223+0000 7fb2c77fe700 1 -- 172.21.15.16:0/3752516436 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d8012360 con 0x7fb2e009e410 2023-09-30T22:29:48.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.223+0000 7fb2c77fe700 1 -- 172.21.15.16:0/3752516436 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2d800f070 con 0x7fb2e009e410 2023-09-30T22:29:48.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.223+0000 7fb2e5350700 1 -- 172.21.15.16:0/3752516436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2e0072090 con 0x7fb2e009e410 2023-09-30T22:29:48.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.223+0000 7fb2c77fe700 1 -- 172.21.15.16:0/3752516436 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d8011560 con 0x7fb2e009e410 2023-09-30T22:29:48.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.223+0000 7fb2e5350700 1 -- 172.21.15.16:0/3752516436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2e009c750 con 0x7fb2e009e410 2023-09-30T22:29:48.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.224+0000 7f1999ffb700 1 -- 172.21.15.16:0/3964107883 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f199c005380 con 0x7f19a4131b40 2023-09-30T22:29:48.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.224+0000 7fb2c77fe700 1 -- 172.21.15.16:0/3752516436 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fb2d80116c0 con 0x7fb2e009e410 2023-09-30T22:29:48.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.225+0000 7fb2e5350700 1 -- 172.21.15.16:0/3752516436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2e011ac20 con 0x7fb2e009e410 2023-09-30T22:29:48.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.225+0000 7fb2c77fe700 1 -- 172.21.15.16:0/3752516436 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7fb2d803fce0 con 0x7fb2e009e410 2023-09-30T22:29:48.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.229+0000 7fb2deffd700 1 -- 172.21.15.16:0/3752516436 >> v1:172.21.15.16:6816/126811 conn(0x7fb2cc02f510 legacy=0x7fb2cc0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:48.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.234+0000 7fb2c77fe700 1 -- 172.21.15.16:0/3752516436 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb2d8011970 con 0x7fb2e009e410 2023-09-30T22:29:48.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.348+0000 7fc0a4a7b700 1 -- 172.21.15.16:0/1919401059 >> v1:172.21.15.16:6816/126811 conn(0x7fc08402f540 legacy=0x7fc0840319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:48.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.361+0000 7f06b4e9a700 1 -- 172.21.15.16:0/3686724542 >> v1:172.21.15.16:6816/126811 conn(0x7f069402f510 legacy=0x7f06940319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:48.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.394+0000 7fc0a6cdf700 1 -- 172.21.15.16:0/1919401059 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc0a0074ea0 con 0x7fc0a009e000 2023-09-30T22:29:48.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.395+0000 7fc09d7fa700 1 -- 172.21.15.16:0/1919401059 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc08c024070 con 0x7fc0a009e000 2023-09-30T22:29:48.395 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:29:48.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.397+0000 7fc0a6cdf700 1 -- 172.21.15.16:0/1919401059 >> v1:172.21.15.16:6816/126811 conn(0x7fc08402f540 legacy=0x7fc0840319f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:48.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.397+0000 7fc0a6cdf700 1 -- 172.21.15.16:0/1919401059 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc0a009e000 msgr2=0x7fc0a0122cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:48.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.397+0000 7fc0a6cdf700 1 --2- 172.21.15.16:0/1919401059 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc0a009e000 0x7fc0a0122cc0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc08c00c780 tx=0x7fc08c00ca90 comp rx=0 tx=0).stop 2023-09-30T22:29:48.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.397+0000 7fc0a6cdf700 1 -- 172.21.15.16:0/1919401059 shutdown_connections 2023-09-30T22:29:48.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.397+0000 7fc0a6cdf700 1 --2- 172.21.15.16:0/1919401059 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc0a009e410 0x7fc0a0123200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:48.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.397+0000 7fc0a6cdf700 1 --2- 172.21.15.16:0/1919401059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0a009d6f0 0x7fc0a0122780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:48.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.397+0000 7fc0a6cdf700 1 --2- 172.21.15.16:0/1919401059 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc0a009e000 0x7fc0a0122cc0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:48.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.397+0000 7fc0a6cdf700 1 -- 172.21.15.16:0/1919401059 >> 172.21.15.16:0/1919401059 conn(0x7fc0a0066f50 msgr2=0x7fc0a006efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:48.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.398+0000 7fc0a6cdf700 1 -- 172.21.15.16:0/1919401059 shutdown_connections 2023-09-30T22:29:48.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.398+0000 7fc0a6cdf700 1 -- 172.21.15.16:0/1919401059 wait complete. 2023-09-30T22:29:48.412 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:29:48.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.420+0000 7f19a9533700 1 -- 172.21.15.16:0/3964107883 >> v1:172.21.15.16:6816/126811 conn(0x7f198802f510 legacy=0x7f19880319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:48.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.430+0000 7fb2deffd700 1 -- 172.21.15.16:0/3752516436 >> v1:172.21.15.16:6816/126811 conn(0x7fb2cc02f510 legacy=0x7fb2cc0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:48.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.441+0000 7fb2e5350700 1 -- 172.21.15.16:0/3752516436 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb2e0074ea0 con 0x7fb2e009e410 2023-09-30T22:29:48.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.441+0000 7fb2c77fe700 1 -- 172.21.15.16:0/3752516436 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb2d801ae90 con 0x7fb2e009e410 2023-09-30T22:29:48.442 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:29:48.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.443+0000 7fb2e5350700 1 -- 172.21.15.16:0/3752516436 >> v1:172.21.15.16:6816/126811 conn(0x7fb2cc02f510 legacy=0x7fb2cc0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:48.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.443+0000 7fb2e5350700 1 -- 172.21.15.16:0/3752516436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2e009e410 msgr2=0x7fb2e011e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:48.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.443+0000 7fb2e5350700 1 --2- 172.21.15.16:0/3752516436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2e009e410 0x7fb2e011e910 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8007360 tx=0x7fb2d8007670 comp rx=0 tx=0).stop 2023-09-30T22:29:48.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.443+0000 7fb2e5350700 1 -- 172.21.15.16:0/3752516436 shutdown_connections 2023-09-30T22:29:48.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.443+0000 7fb2e5350700 1 --2- 172.21.15.16:0/3752516436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2e009e410 0x7fb2e011e910 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:48.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.443+0000 7fb2e5350700 1 --2- 172.21.15.16:0/3752516436 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb2e009d6f0 0x7fb2e011de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:48.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.443+0000 7fb2e5350700 1 --2- 172.21.15.16:0/3752516436 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb2e009e000 0x7fb2e011e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:48.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.443+0000 7fb2e5350700 1 -- 172.21.15.16:0/3752516436 >> 172.21.15.16:0/3752516436 conn(0x7fb2e0066f50 msgr2=0x7fb2e0130990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:48.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.444+0000 7fb2e5350700 1 -- 172.21.15.16:0/3752516436 shutdown_connections 2023-09-30T22:29:48.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.444+0000 7fb2e5350700 1 -- 172.21.15.16:0/3752516436 wait complete. 2023-09-30T22:29:48.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.450+0000 7f06b78ff700 1 -- 172.21.15.16:0/3686724542 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f06b009f420 con 0x7f06b009d6f0 2023-09-30T22:29:48.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.450+0000 7f06a5ffb700 1 -- 172.21.15.16:0/3686724542 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f06ac017020 con 0x7f06b009d6f0 2023-09-30T22:29:48.450 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:29:48.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.453+0000 7f06b78ff700 1 -- 172.21.15.16:0/3686724542 >> v1:172.21.15.16:6816/126811 conn(0x7f069402f510 legacy=0x7f06940319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:48.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.453+0000 7f06b78ff700 1 -- 172.21.15.16:0/3686724542 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f06b009d6f0 msgr2=0x7f06b011e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:48.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.453+0000 7f06b78ff700 1 --2- 172.21.15.16:0/3686724542 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f06b009d6f0 0x7f06b011e860 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f06ac000f60 tx=0x7f06ac00c730 comp rx=0 tx=0).stop 2023-09-30T22:29:48.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.453+0000 7f06b78ff700 1 -- 172.21.15.16:0/3686724542 shutdown_connections 2023-09-30T22:29:48.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.453+0000 7f06b78ff700 1 --2- 172.21.15.16:0/3686724542 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f06b009d6f0 0x7f06b011e860 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:48.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.453+0000 7f06b78ff700 1 --2- 172.21.15.16:0/3686724542 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f06b0139170 0x7f06b011f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:48.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.453+0000 7f06b78ff700 1 --2- 172.21.15.16:0/3686724542 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f06b009e1d0 0x7f06b011eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:48.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.453+0000 7f06b78ff700 1 -- 172.21.15.16:0/3686724542 >> 172.21.15.16:0/3686724542 conn(0x7f06b0066f50 msgr2=0x7f06b009b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:48.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.453+0000 7f06b78ff700 1 -- 172.21.15.16:0/3686724542 shutdown_connections 2023-09-30T22:29:48.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.453+0000 7f06b78ff700 1 -- 172.21.15.16:0/3686724542 wait complete. 2023-09-30T22:29:48.459 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:29:48.467 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:29:48.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.482+0000 7f19ab797700 1 -- 172.21.15.16:0/3964107883 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f19a4122140 con 0x7f19a4131b40 2023-09-30T22:29:48.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.483+0000 7f1999ffb700 1 -- 172.21.15.16:0/3964107883 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f199c019300 con 0x7f19a4131b40 2023-09-30T22:29:48.483 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:29:48.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.485+0000 7f19ab797700 1 -- 172.21.15.16:0/3964107883 >> v1:172.21.15.16:6816/126811 conn(0x7f198802f510 legacy=0x7f19880319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:48.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.485+0000 7f19ab797700 1 -- 172.21.15.16:0/3964107883 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f19a4131b40 msgr2=0x7f19a41c8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:48.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.485+0000 7f19ab797700 1 --2- 172.21.15.16:0/3964107883 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f19a4131b40 0x7f19a41c8b70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f199c00cc50 tx=0x7f199c00a340 comp rx=0 tx=0).stop 2023-09-30T22:29:48.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.485+0000 7f19ab797700 1 -- 172.21.15.16:0/3964107883 shutdown_connections 2023-09-30T22:29:48.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.485+0000 7f19ab797700 1 --2- 172.21.15.16:0/3964107883 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f19a4131b40 0x7f19a41c8b70 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:48.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.485+0000 7f19ab797700 1 --2- 172.21.15.16:0/3964107883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19a406a770 0x7f19a41c80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:48.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.485+0000 7f19ab797700 1 --2- 172.21.15.16:0/3964107883 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f19a406b080 0x7f19a41c8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:48.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.485+0000 7f19ab797700 1 -- 172.21.15.16:0/3964107883 >> 172.21.15.16:0/3964107883 conn(0x7f19a4066f50 msgr2=0x7f19a40a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:48.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.485+0000 7f19ab797700 1 -- 172.21.15.16:0/3964107883 shutdown_connections 2023-09-30T22:29:48.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:48.485+0000 7f19ab797700 1 -- 172.21.15.16:0/3964107883 wait complete. 2023-09-30T22:29:48.500 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:29:49.414 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:29:49.460 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:29:49.468 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:29:49.501 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:29:49.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.571+0000 7f3c85c51700 1 Processor -- start 2023-09-30T22:29:49.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.571+0000 7f3c85c51700 1 -- start start 2023-09-30T22:29:49.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.572+0000 7f3c85c51700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3c801306e0 0x7f3c80130ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:49.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.572+0000 7f3c85c51700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3c8009de00 0x7f3c801384d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:49.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.572+0000 7f3c85c51700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c80130ff0 con 0x7f3c80072590 2023-09-30T22:29:49.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.572+0000 7f3c85c51700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c80130290 con 0x7f3c8009de00 2023-09-30T22:29:49.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.572+0000 7f3c85c51700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8009bad0 con 0x7f3c801306e0 2023-09-30T22:29:49.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.572+0000 7f3c7f7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3c801306e0 0x7f3c80130ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:49.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.572+0000 7f3c7f7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3c801306e0 0x7f3c80130ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:51960/0 (socket says 172.21.15.16:51960) 2023-09-30T22:29:49.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.572+0000 7f3c7f7fe700 1 -- 172.21.15.16:0/2631184548 learned_addr learned my addr 172.21.15.16:0/2631184548 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:49.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.573+0000 7f3c7ffff700 1 --2- 172.21.15.16:0/2631184548 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3c8009de00 0x7f3c801384d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:49.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.573+0000 7f3c7f7fe700 1 -- 172.21.15.16:0/2631184548 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3c8009de00 msgr2=0x7f3c801384d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:49.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.573+0000 7f3c7f7fe700 1 --2- 172.21.15.16:0/2631184548 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3c8009de00 0x7f3c801384d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.573+0000 7f3c7f7fe700 1 -- 172.21.15.16:0/2631184548 >> v1:172.21.15.16:6789/0 conn(0x7f3c80072590 legacy=0x7f3c8009d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:49.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.573+0000 7f3c7f7fe700 1 -- 172.21.15.16:0/2631184548 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c8009a8e0 con 0x7f3c801306e0 2023-09-30T22:29:49.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.573+0000 7f3c7f7fe700 1 --2- 172.21.15.16:0/2631184548 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3c801306e0 0x7f3c80130ab0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3c70009920 tx=0x7f3c70009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab3150b6b891750b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:49.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.573+0000 7f3c7e7fc700 1 -- 172.21.15.16:0/2631184548 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c70011830 con 0x7f3c801306e0 2023-09-30T22:29:49.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.573+0000 7f3c7e7fc700 1 -- 172.21.15.16:0/2631184548 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c70011990 con 0x7f3c801306e0 2023-09-30T22:29:49.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.573+0000 7f3c7e7fc700 1 -- 172.21.15.16:0/2631184548 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c70011d20 con 0x7f3c801306e0 2023-09-30T22:29:49.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.574+0000 7f3c85c51700 1 -- 172.21.15.16:0/2631184548 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3c801306e0 msgr2=0x7f3c80130ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:49.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.574+0000 7f3c85c51700 1 --2- 172.21.15.16:0/2631184548 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3c801306e0 0x7f3c80130ab0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3c70009920 tx=0x7f3c70009c30 comp rx=0 tx=0).stop 2023-09-30T22:29:49.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.575+0000 7f3c85c51700 1 -- 172.21.15.16:0/2631184548 shutdown_connections 2023-09-30T22:29:49.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.575+0000 7f3c85c51700 1 --2- 172.21.15.16:0/2631184548 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3c8009de00 0x7f3c801384d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.575+0000 7f3c85c51700 1 --2- 172.21.15.16:0/2631184548 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3c801306e0 0x7f3c80130ab0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.575+0000 7f3c85c51700 1 -- 172.21.15.16:0/2631184548 >> 172.21.15.16:0/2631184548 conn(0x7f3c8009f470 msgr2=0x7f3c800a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:49.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.575+0000 7f3c85c51700 1 -- 172.21.15.16:0/2631184548 shutdown_connections 2023-09-30T22:29:49.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.575+0000 7f3c85c51700 1 -- 172.21.15.16:0/2631184548 wait complete. 2023-09-30T22:29:49.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.576+0000 7f3c85c51700 1 Processor -- start 2023-09-30T22:29:49.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.577+0000 7f3c85c51700 1 -- start start 2023-09-30T22:29:49.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.577+0000 7f3c85c51700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3c80072590 0x7f3c8012ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:49.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.577+0000 7f3c85c51700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3c8009de00 0x7f3c8012f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:49.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.578+0000 7f3c85c51700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c801306e0 0x7f3c801c8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:49.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.578+0000 7f3c85c51700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8006f610 con 0x7f3c801306e0 2023-09-30T22:29:49.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.578+0000 7f3c85c51700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8012e770 con 0x7f3c8009de00 2023-09-30T22:29:49.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.578+0000 7f3c85c51700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3c800709f0 con 0x7f3c80072590 2023-09-30T22:29:49.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.578+0000 7f3c7f7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3c80072590 0x7f3c8012ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:49.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.578+0000 7f3c7ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c801306e0 0x7f3c801c8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:49.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.578+0000 7f3c7f7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3c80072590 0x7f3c8012ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:51966/0 (socket says 172.21.15.16:51966) 2023-09-30T22:29:49.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.578+0000 7f3c7f7fe700 1 -- 172.21.15.16:0/3939503618 learned_addr learned my addr 172.21.15.16:0/3939503618 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:49.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.578+0000 7f3c7effd700 1 --2- 172.21.15.16:0/3939503618 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3c8009de00 0x7f3c8012f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:49.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.578+0000 7f3c7f7fe700 1 -- 172.21.15.16:0/3939503618 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3c8009de00 msgr2=0x7f3c8012f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:49.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.578+0000 7f3c7f7fe700 1 --2- 172.21.15.16:0/3939503618 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3c8009de00 0x7f3c8012f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.578+0000 7f3c7f7fe700 1 -- 172.21.15.16:0/3939503618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c801306e0 msgr2=0x7f3c801c8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:49.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.578+0000 7f3c7f7fe700 1 --2- 172.21.15.16:0/3939503618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c801306e0 0x7f3c801c8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.578+0000 7f3c7f7fe700 1 -- 172.21.15.16:0/3939503618 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c70009550 con 0x7f3c80072590 2023-09-30T22:29:49.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.578+0000 7f3c7f7fe700 1 --2- 172.21.15.16:0/3939503618 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3c80072590 0x7f3c8012ebb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3c70000c00 tx=0x7f3c70004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:49.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.579+0000 7f3c67fff700 1 -- 172.21.15.16:0/3939503618 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c70004450 con 0x7f3c80072590 2023-09-30T22:29:49.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.579+0000 7f3c67fff700 1 -- 172.21.15.16:0/3939503618 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c700045b0 con 0x7f3c80072590 2023-09-30T22:29:49.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.579+0000 7f3c67fff700 1 -- 172.21.15.16:0/3939503618 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c700038a0 con 0x7f3c80072590 2023-09-30T22:29:49.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.579+0000 7f3c85c51700 1 -- 172.21.15.16:0/3939503618 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c8006fc50 con 0x7f3c80072590 2023-09-30T22:29:49.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.579+0000 7f3c85c51700 1 -- 172.21.15.16:0/3939503618 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c8008f6e0 con 0x7f3c80072590 2023-09-30T22:29:49.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.580+0000 7f3c67fff700 1 -- 172.21.15.16:0/3939503618 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f3c70003a00 con 0x7f3c80072590 2023-09-30T22:29:49.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.580+0000 7f3c85c51700 1 -- 172.21.15.16:0/3939503618 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c80130500 con 0x7f3c80072590 2023-09-30T22:29:49.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.580+0000 7f3c67fff700 1 -- 172.21.15.16:0/3939503618 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7f3c70021030 con 0x7f3c80072590 2023-09-30T22:29:49.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.580+0000 7f3c7effd700 1 -- 172.21.15.16:0/3939503618 >> v1:172.21.15.16:6816/126811 conn(0x7f3c6c02f510 legacy=0x7f3c6c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:49.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.585+0000 7f3c67fff700 1 -- 172.21.15.16:0/3939503618 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3c70072750 con 0x7f3c80072590 2023-09-30T22:29:49.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.623+0000 7fe041db0700 1 Processor -- start 2023-09-30T22:29:49.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.623+0000 7fe041db0700 1 -- start start 2023-09-30T22:29:49.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.624+0000 7fe041db0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe03c06c0e0 0x7fe03c09ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:49.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.624+0000 7fe041db0700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe03c134ec0 0x7fe03c1379e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:49.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.624+0000 7fe041db0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe03c137f20 0x7fe03c1383c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:49.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.624+0000 7fe041db0700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe03c06c4b0 con 0x7fe03c134ec0 2023-09-30T22:29:49.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.624+0000 7fe041db0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe03c0a09c0 con 0x7fe03c06c0e0 2023-09-30T22:29:49.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.624+0000 7fe041db0700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe03c134980 con 0x7fe03c137f20 2023-09-30T22:29:49.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.624+0000 7fe03bfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe03c137f20 0x7fe03c1383c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:49.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.624+0000 7fe03b7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe03c06c0e0 0x7fe03c09ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:49.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.624+0000 7fe03b7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe03c06c0e0 0x7fe03c09ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:49392/0 (socket says 172.21.15.16:49392) 2023-09-30T22:29:49.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.624+0000 7fe03bfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe03c137f20 0x7fe03c1383c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:51972/0 (socket says 172.21.15.16:51972) 2023-09-30T22:29:49.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.624+0000 7fe03b7fe700 1 -- 172.21.15.16:0/2836487135 learned_addr learned my addr 172.21.15.16:0/2836487135 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:49.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.624+0000 7fe03b7fe700 1 -- 172.21.15.16:0/2836487135 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe03c137f20 msgr2=0x7fe03c1383c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:49.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.624+0000 7fe03b7fe700 1 --2- 172.21.15.16:0/2836487135 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe03c137f20 0x7fe03c1383c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.624+0000 7fe03b7fe700 1 -- 172.21.15.16:0/2836487135 >> v2:172.21.15.16:3300/0 conn(0x7fe03c134ec0 msgr2=0x7fe03c1379e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:49.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.624+0000 7fe03b7fe700 1 --2- 172.21.15.16:0/2836487135 >> v2:172.21.15.16:3300/0 conn(0x7fe03c134ec0 0x7fe03c1379e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.624+0000 7fe03b7fe700 1 -- 172.21.15.16:0/2836487135 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe03c0a0600 con 0x7fe03c06c0e0 2023-09-30T22:29:49.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.624+0000 7fe03bfff700 1 --2- 172.21.15.16:0/2836487135 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe03c137f20 0x7fe03c1383c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:49.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.625+0000 7fe03b7fe700 1 --2- 172.21.15.16:0/2836487135 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe03c06c0e0 0x7fe03c09ac90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe02c0098f0 tx=0x7fe02c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa841d45852ed9b1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:49.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.625+0000 7fe03a7fc700 1 -- 172.21.15.16:0/2836487135 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe02c011830 con 0x7fe03c06c0e0 2023-09-30T22:29:49.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.625+0000 7fe03a7fc700 1 -- 172.21.15.16:0/2836487135 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe02c011990 con 0x7fe03c06c0e0 2023-09-30T22:29:49.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.625+0000 7fe03a7fc700 1 -- 172.21.15.16:0/2836487135 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe02c011d20 con 0x7fe03c06c0e0 2023-09-30T22:29:49.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.626+0000 7fe041db0700 1 -- 172.21.15.16:0/2836487135 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe03c06c0e0 msgr2=0x7fe03c09ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:49.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.626+0000 7fe041db0700 1 --2- 172.21.15.16:0/2836487135 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe03c06c0e0 0x7fe03c09ac90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe02c0098f0 tx=0x7fe02c009c00 comp rx=0 tx=0).stop 2023-09-30T22:29:49.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.626+0000 7fe041db0700 1 -- 172.21.15.16:0/2836487135 shutdown_connections 2023-09-30T22:29:49.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.626+0000 7fe041db0700 1 --2- 172.21.15.16:0/2836487135 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe03c06c0e0 0x7fe03c09ac90 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.626+0000 7fe041db0700 1 --2- 172.21.15.16:0/2836487135 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe03c137f20 0x7fe03c1383c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.626+0000 7fe041db0700 1 --2- 172.21.15.16:0/2836487135 >> v2:172.21.15.16:3300/0 conn(0x7fe03c134ec0 0x7fe03c1379e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.626+0000 7fe041db0700 1 -- 172.21.15.16:0/2836487135 >> 172.21.15.16:0/2836487135 conn(0x7fe03c0a4210 msgr2=0x7fe03c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:49.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.626+0000 7fe041db0700 1 -- 172.21.15.16:0/2836487135 shutdown_connections 2023-09-30T22:29:49.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.626+0000 7fe041db0700 1 -- 172.21.15.16:0/2836487135 wait complete. 2023-09-30T22:29:49.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.627+0000 7fe041db0700 1 Processor -- start 2023-09-30T22:29:49.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.628+0000 7fe041db0700 1 -- start start 2023-09-30T22:29:49.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.628+0000 7fe041db0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe03c06c0e0 0x7fe03c1ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:49.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.628+0000 7fe041db0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe03c134ec0 0x7fe03c1cd290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:49.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.628+0000 7fe041db0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe03c137f20 0x7fe03c11a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:49.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.628+0000 7fe041db0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe03c11c4f0 con 0x7fe03c137f20 2023-09-30T22:29:49.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.628+0000 7fe041db0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe03c133d80 con 0x7fe03c134ec0 2023-09-30T22:29:49.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.628+0000 7fe041db0700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe03c0a03f0 con 0x7fe03c06c0e0 2023-09-30T22:29:49.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.628+0000 7fe03b7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe03c06c0e0 0x7fe03c1ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:49.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.628+0000 7fe03b7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe03c06c0e0 0x7fe03c1ccd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:51978/0 (socket says 172.21.15.16:51978) 2023-09-30T22:29:49.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.628+0000 7fe03b7fe700 1 -- 172.21.15.16:0/3122562684 learned_addr learned my addr 172.21.15.16:0/3122562684 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:49.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.629+0000 7fe03affd700 1 --2- 172.21.15.16:0/3122562684 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe03c134ec0 0x7fe03c1cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:49.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.629+0000 7fe03affd700 1 -- 172.21.15.16:0/3122562684 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe03c06c0e0 msgr2=0x7fe03c1ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:49.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.629+0000 7fe03affd700 1 --2- 172.21.15.16:0/3122562684 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe03c06c0e0 0x7fe03c1ccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.629+0000 7fe03affd700 1 -- 172.21.15.16:0/3122562684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe03c137f20 msgr2=0x7fe03c11a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:49.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.629+0000 7fe03affd700 1 --2- 172.21.15.16:0/3122562684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe03c137f20 0x7fe03c11a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.629+0000 7fe03affd700 1 -- 172.21.15.16:0/3122562684 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe034009a70 con 0x7fe03c134ec0 2023-09-30T22:29:49.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.629+0000 7fe03affd700 1 --2- 172.21.15.16:0/3122562684 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe03c134ec0 0x7fe03c1cd290 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe03c06f500 tx=0x7fe03000b730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:49.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.629+0000 7fe023fff700 1 -- 172.21.15.16:0/3122562684 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe030011830 con 0x7fe03c134ec0 2023-09-30T22:29:49.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.629+0000 7fe041db0700 1 -- 172.21.15.16:0/3122562684 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe02c009550 con 0x7fe03c134ec0 2023-09-30T22:29:49.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.629+0000 7fe023fff700 1 -- 172.21.15.16:0/3122562684 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe030011990 con 0x7fe03c134ec0 2023-09-30T22:29:49.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.630+0000 7fe023fff700 1 -- 172.21.15.16:0/3122562684 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe030011d40 con 0x7fe03c134ec0 2023-09-30T22:29:49.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.630+0000 7fe041db0700 1 -- 172.21.15.16:0/3122562684 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe03c1272a0 con 0x7fe03c134ec0 2023-09-30T22:29:49.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.631+0000 7fe023fff700 1 -- 172.21.15.16:0/3122562684 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fe03001a400 con 0x7fe03c134ec0 2023-09-30T22:29:49.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.631+0000 7fe023fff700 1 -- 172.21.15.16:0/3122562684 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7fe03003fac0 con 0x7fe03c134ec0 2023-09-30T22:29:49.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.631+0000 7fe03b7fe700 1 -- 172.21.15.16:0/3122562684 >> v1:172.21.15.16:6816/126811 conn(0x7fe02802f510 legacy=0x7fe0280319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:49.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.632+0000 7fe041db0700 1 -- 172.21.15.16:0/3122562684 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe03c074ea0 con 0x7fe03c134ec0 2023-09-30T22:29:49.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.642+0000 7fe023fff700 1 -- 172.21.15.16:0/3122562684 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe030009280 con 0x7fe03c134ec0 2023-09-30T22:29:49.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.664+0000 7fe370510700 1 Processor -- start 2023-09-30T22:29:49.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.665+0000 7f1403757700 1 Processor -- start 2023-09-30T22:29:49.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.665+0000 7f1403757700 1 -- start start 2023-09-30T22:29:49.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.665+0000 7f1403757700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f13fc131810 0x7f13fc131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:49.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.665+0000 7f1403757700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f13fc099a40 0x7f13fc099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:49.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.665+0000 7f1403757700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f13fc09a3b0 0x7f13fc138650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:49.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.665+0000 7f1403757700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13fc09b580 con 0x7f13fc09a3b0 2023-09-30T22:29:49.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.665+0000 7f1403757700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13fc1313c0 con 0x7f13fc131810 2023-09-30T22:29:49.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.665+0000 7f1403757700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f13fc06e8c0 con 0x7f13fc099a40 2023-09-30T22:29:49.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.666+0000 7f14014f3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f13fc131810 0x7f13fc131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:49.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.666+0000 7f14014f3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f13fc131810 0x7f13fc131be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:49400/0 (socket says 172.21.15.16:49400) 2023-09-30T22:29:49.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.666+0000 7f14014f3700 1 -- 172.21.15.16:0/301290404 learned_addr learned my addr 172.21.15.16:0/301290404 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:49.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.666+0000 7f1400cf2700 1 --2- 172.21.15.16:0/301290404 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f13fc099a40 0x7f13fc099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:49.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.665+0000 7fe370510700 1 -- start start 2023-09-30T22:29:49.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.666+0000 7f1401cf4700 1 --2- 172.21.15.16:0/301290404 >> v2:172.21.15.16:3300/0 conn(0x7f13fc09a3b0 0x7f13fc138650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:49.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.666+0000 7fe370510700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe36806e570 0x7fe36806e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:49.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.666+0000 7fe370510700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe36809eca0 0x7fe36806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:49.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.666+0000 7fe370510700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe368070ee0 con 0x7fe368134610 2023-09-30T22:29:49.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.666+0000 7fe370510700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe368071480 con 0x7fe36806e570 2023-09-30T22:29:49.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.666+0000 7fe370510700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe36806f3c0 con 0x7fe36809eca0 2023-09-30T22:29:49.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.666+0000 7fe36e2ac700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe36806e570 0x7fe36806e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:49.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.666+0000 7fe36e2ac700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe36806e570 0x7fe36806e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47962/0 (socket says 172.21.15.16:47962) 2023-09-30T22:29:49.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.666+0000 7fe36e2ac700 1 -- 172.21.15.16:0/2687517051 learned_addr learned my addr 172.21.15.16:0/2687517051 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:49.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.666+0000 7fe36daab700 1 --2- 172.21.15.16:0/2687517051 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe36809eca0 0x7fe36806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:49.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.666+0000 7f14014f3700 1 -- 172.21.15.16:0/301290404 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f13fc099a40 msgr2=0x7f13fc099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:49.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.666+0000 7f14014f3700 1 --2- 172.21.15.16:0/301290404 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f13fc099a40 0x7f13fc099e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.666+0000 7f14014f3700 1 -- 172.21.15.16:0/301290404 >> v2:172.21.15.16:3300/0 conn(0x7f13fc09a3b0 msgr2=0x7f13fc138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:49.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.666+0000 7f14014f3700 1 --2- 172.21.15.16:0/301290404 >> v2:172.21.15.16:3300/0 conn(0x7f13fc09a3b0 0x7f13fc138650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.666+0000 7f14014f3700 1 -- 172.21.15.16:0/301290404 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13fc09a820 con 0x7f13fc131810 2023-09-30T22:29:49.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.666+0000 7f14014f3700 1 --2- 172.21.15.16:0/301290404 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f13fc131810 0x7f13fc131be0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f13f80098f0 tx=0x7f13f8009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e275555f009e04e3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:49.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.667+0000 7f13f3fff700 1 -- 172.21.15.16:0/301290404 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f8010a90 con 0x7f13fc131810 2023-09-30T22:29:49.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.667+0000 7f13f3fff700 1 -- 172.21.15.16:0/301290404 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13f8010bf0 con 0x7f13fc131810 2023-09-30T22:29:49.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.667+0000 7f13f3fff700 1 -- 172.21.15.16:0/301290404 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f8018440 con 0x7f13fc131810 2023-09-30T22:29:49.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.666+0000 7fe36daab700 1 -- 172.21.15.16:0/2687517051 >> v2:172.21.15.16:3300/0 conn(0x7fe36806e570 msgr2=0x7fe36806e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:49.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.666+0000 7fe36daab700 1 --2- 172.21.15.16:0/2687517051 >> v2:172.21.15.16:3300/0 conn(0x7fe36806e570 0x7fe36806e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.667+0000 7fe36daab700 1 -- 172.21.15.16:0/2687517051 >> v1:172.21.15.16:6789/0 conn(0x7fe368134610 legacy=0x7fe368134a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:49.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.667+0000 7fe36daab700 1 -- 172.21.15.16:0/2687517051 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe368071b90 con 0x7fe36809eca0 2023-09-30T22:29:49.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.667+0000 7fe36daab700 1 --2- 172.21.15.16:0/2687517051 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe36809eca0 0x7fe36806ee80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe358009840 tx=0x7fe358009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e599a219fc6549d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:49.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.667+0000 7fe36d2aa700 1 -- 172.21.15.16:0/2687517051 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe358011830 con 0x7fe36809eca0 2023-09-30T22:29:49.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.667+0000 7fe36d2aa700 1 -- 172.21.15.16:0/2687517051 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe358011990 con 0x7fe36809eca0 2023-09-30T22:29:49.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.667+0000 7fe36d2aa700 1 -- 172.21.15.16:0/2687517051 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe358011d20 con 0x7fe36809eca0 2023-09-30T22:29:49.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.668+0000 7f1403757700 1 -- 172.21.15.16:0/301290404 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f13fc131810 msgr2=0x7f13fc131be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:49.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.668+0000 7f1403757700 1 --2- 172.21.15.16:0/301290404 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f13fc131810 0x7f13fc131be0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f13f80098f0 tx=0x7f13f8009c00 comp rx=0 tx=0).stop 2023-09-30T22:29:49.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.668+0000 7f1403757700 1 -- 172.21.15.16:0/301290404 shutdown_connections 2023-09-30T22:29:49.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.668+0000 7f1403757700 1 --2- 172.21.15.16:0/301290404 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f13fc131810 0x7f13fc131be0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.668+0000 7f1403757700 1 --2- 172.21.15.16:0/301290404 >> v2:172.21.15.16:3300/0 conn(0x7f13fc09a3b0 0x7f13fc138650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.668+0000 7f1403757700 1 --2- 172.21.15.16:0/301290404 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f13fc099a40 0x7f13fc099e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.668+0000 7f1403757700 1 -- 172.21.15.16:0/301290404 >> 172.21.15.16:0/301290404 conn(0x7f13fc0a2aa0 msgr2=0x7f13fc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:49.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.668+0000 7f1403757700 1 -- 172.21.15.16:0/301290404 shutdown_connections 2023-09-30T22:29:49.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.668+0000 7fe370510700 1 -- 172.21.15.16:0/2687517051 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe36809eca0 msgr2=0x7fe36806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:49.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.668+0000 7fe370510700 1 --2- 172.21.15.16:0/2687517051 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe36809eca0 0x7fe36806ee80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe358009840 tx=0x7fe358009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:49.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.668+0000 7f1403757700 1 -- 172.21.15.16:0/301290404 wait complete. 2023-09-30T22:29:49.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.669+0000 7f1403757700 1 Processor -- start 2023-09-30T22:29:49.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.669+0000 7f1403757700 1 -- start start 2023-09-30T22:29:49.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.669+0000 7f1403757700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f13fc099a40 0x7f13fc11b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:49.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.669+0000 7f1403757700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f13fc09a3b0 0x7f13fc11bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:49.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.669+0000 7f1403757700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13fc131810 0x7f13fc11c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:49.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.669+0000 7f1403757700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13fc11f290 con 0x7f13fc131810 2023-09-30T22:29:49.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.669+0000 7f1403757700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13fc11d960 con 0x7f13fc09a3b0 2023-09-30T22:29:49.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.669+0000 7f1403757700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f13fc11ea20 con 0x7f13fc099a40 2023-09-30T22:29:49.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.670+0000 7f1401cf4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13fc131810 0x7f13fc11c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:49.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.670+0000 7f1401cf4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13fc131810 0x7f13fc11c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47974/0 (socket says 172.21.15.16:47974) 2023-09-30T22:29:49.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.670+0000 7f1401cf4700 1 -- 172.21.15.16:0/595886090 learned_addr learned my addr 172.21.15.16:0/595886090 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:49.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.670+0000 7f14014f3700 1 --2- 172.21.15.16:0/595886090 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f13fc099a40 0x7f13fc11b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:49.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.670+0000 7f1401cf4700 1 -- 172.21.15.16:0/595886090 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f13fc099a40 msgr2=0x7f13fc11b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:49.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.670+0000 7f1401cf4700 1 --2- 172.21.15.16:0/595886090 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f13fc099a40 0x7f13fc11b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.670+0000 7f1401cf4700 1 -- 172.21.15.16:0/595886090 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f13fc09a3b0 msgr2=0x7f13fc11bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:49.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.670+0000 7f1401cf4700 1 --2- 172.21.15.16:0/595886090 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f13fc09a3b0 0x7f13fc11bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.670+0000 7f1401cf4700 1 -- 172.21.15.16:0/595886090 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f8009550 con 0x7f13fc131810 2023-09-30T22:29:49.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.669+0000 7fe370510700 1 -- 172.21.15.16:0/2687517051 shutdown_connections 2023-09-30T22:29:49.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.669+0000 7fe370510700 1 --2- 172.21.15.16:0/2687517051 >> v2:172.21.15.16:3300/0 conn(0x7fe36806e570 0x7fe36806e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.669+0000 7fe370510700 1 --2- 172.21.15.16:0/2687517051 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe36809eca0 0x7fe36806ee80 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.669+0000 7fe370510700 1 -- 172.21.15.16:0/2687517051 >> 172.21.15.16:0/2687517051 conn(0x7fe3680a2aa0 msgr2=0x7fe3680a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:49.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.669+0000 7fe370510700 1 -- 172.21.15.16:0/2687517051 shutdown_connections 2023-09-30T22:29:49.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.670+0000 7fe370510700 1 -- 172.21.15.16:0/2687517051 wait complete. 2023-09-30T22:29:49.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.670+0000 7f1401cf4700 1 --2- 172.21.15.16:0/595886090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13fc131810 0x7f13fc11c280 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f13f4009dd0 tx=0x7f13f400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:49.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.671+0000 7f13f1ffb700 1 -- 172.21.15.16:0/595886090 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f4009c40 con 0x7f13fc131810 2023-09-30T22:29:49.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.671+0000 7f13f1ffb700 1 -- 172.21.15.16:0/595886090 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13f400f040 con 0x7f13fc131810 2023-09-30T22:29:49.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.671+0000 7f13f1ffb700 1 -- 172.21.15.16:0/595886090 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f4014480 con 0x7f13fc131810 2023-09-30T22:29:49.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.671+0000 7f1403757700 1 -- 172.21.15.16:0/595886090 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13fc06eb00 con 0x7f13fc131810 2023-09-30T22:29:49.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.671+0000 7fe370510700 1 Processor -- start 2023-09-30T22:29:49.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.671+0000 7fe370510700 1 -- start start 2023-09-30T22:29:49.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.672+0000 7fe370510700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe36806e570 0x7fe368140280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:49.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.672+0000 7fe370510700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe368134610 0x7fe36813e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:49.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.672+0000 7fe370510700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe36813eeb0 0x7fe36813f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:49.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.672+0000 7fe370510700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36813cc80 con 0x7fe368134610 2023-09-30T22:29:49.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.672+0000 7fe370510700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe368134330 con 0x7fe36806e570 2023-09-30T22:29:49.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.672+0000 7fe370510700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe368070610 con 0x7fe36813eeb0 2023-09-30T22:29:49.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.672+0000 7fe36daab700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe368134610 0x7fe36813e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:49.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.672+0000 7fe36daab700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe368134610 0x7fe36813e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47986/0 (socket says 172.21.15.16:47986) 2023-09-30T22:29:49.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.672+0000 7fe36daab700 1 -- 172.21.15.16:0/1914311414 learned_addr learned my addr 172.21.15.16:0/1914311414 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:49.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.672+0000 7fe36e2ac700 1 --2- 172.21.15.16:0/1914311414 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe36806e570 0x7fe368140280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:49.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.672+0000 7fe36eaad700 1 --2- 172.21.15.16:0/1914311414 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe36813eeb0 0x7fe36813f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:49.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.673+0000 7fe36daab700 1 -- 172.21.15.16:0/1914311414 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe36813eeb0 msgr2=0x7fe36813f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:49.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.673+0000 7fe36daab700 1 --2- 172.21.15.16:0/1914311414 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe36813eeb0 0x7fe36813f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.673+0000 7fe36daab700 1 -- 172.21.15.16:0/1914311414 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe36806e570 msgr2=0x7fe368140280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:49.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.673+0000 7fe36daab700 1 --2- 172.21.15.16:0/1914311414 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe36806e570 0x7fe368140280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.673+0000 7fe36daab700 1 -- 172.21.15.16:0/1914311414 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3580094a0 con 0x7fe368134610 2023-09-30T22:29:49.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.673+0000 7fe36daab700 1 --2- 172.21.15.16:0/1914311414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe368134610 0x7fe36813e970 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe358009b90 tx=0x7fe358004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:49.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.671+0000 7f1403757700 1 -- 172.21.15.16:0/595886090 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13fc12eb60 con 0x7f13fc131810 2023-09-30T22:29:49.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.673+0000 7f1403757700 1 -- 172.21.15.16:0/595886090 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13fc074ea0 con 0x7f13fc131810 2023-09-30T22:29:49.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.673+0000 7f13f1ffb700 1 -- 172.21.15.16:0/595886090 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f13f40145e0 con 0x7f13fc131810 2023-09-30T22:29:49.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.673+0000 7f14014f3700 1 -- 172.21.15.16:0/595886090 >> v1:172.21.15.16:6816/126811 conn(0x7f13e002f510 legacy=0x7f13e00319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:49.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.676+0000 7f13f1ffb700 1 -- 172.21.15.16:0/595886090 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7f13f401c570 con 0x7f13fc131810 2023-09-30T22:29:49.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.678+0000 7fe35effd700 1 -- 172.21.15.16:0/1914311414 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe358004510 con 0x7fe368134610 2023-09-30T22:29:49.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.678+0000 7fe35effd700 1 -- 172.21.15.16:0/1914311414 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3580036a0 con 0x7fe368134610 2023-09-30T22:29:49.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.678+0000 7fe370510700 1 -- 172.21.15.16:0/1914311414 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe36813d140 con 0x7fe368134610 2023-09-30T22:29:49.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.678+0000 7fe35effd700 1 -- 172.21.15.16:0/1914311414 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe358003a30 con 0x7fe368134610 2023-09-30T22:29:49.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.678+0000 7f13f1ffb700 1 -- 172.21.15.16:0/595886090 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f13f4045060 con 0x7f13fc131810 2023-09-30T22:29:49.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.678+0000 7fe370510700 1 -- 172.21.15.16:0/1914311414 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe36813dab0 con 0x7fe368134610 2023-09-30T22:29:49.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.679+0000 7fe35effd700 1 -- 172.21.15.16:0/1914311414 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fe358003b90 con 0x7fe368134610 2023-09-30T22:29:49.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.680+0000 7fe370510700 1 -- 172.21.15.16:0/1914311414 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe368074ea0 con 0x7fe368134610 2023-09-30T22:29:49.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.680+0000 7fe36e2ac700 1 -- 172.21.15.16:0/1914311414 >> v1:172.21.15.16:6816/126811 conn(0x7fe34c02f540 legacy=0x7fe34c0319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:49.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.680+0000 7fe35effd700 1 -- 172.21.15.16:0/1914311414 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7fe358040eb0 con 0x7fe368134610 2023-09-30T22:29:49.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.689+0000 7fe35effd700 1 -- 172.21.15.16:0/1914311414 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe35801a420 con 0x7fe368134610 2023-09-30T22:29:49.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.781+0000 7f3c7effd700 1 -- 172.21.15.16:0/3939503618 >> v1:172.21.15.16:6816/126811 conn(0x7f3c6c02f510 legacy=0x7f3c6c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:49.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.828+0000 7f3c85c51700 1 -- 172.21.15.16:0/3939503618 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3c800747f0 con 0x7f3c80072590 2023-09-30T22:29:49.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.829+0000 7f3c67fff700 1 -- 172.21.15.16:0/3939503618 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3c7002b090 con 0x7f3c80072590 2023-09-30T22:29:49.829 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:29:49.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.831+0000 7f3c85c51700 1 -- 172.21.15.16:0/3939503618 >> v1:172.21.15.16:6816/126811 conn(0x7f3c6c02f510 legacy=0x7f3c6c0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:49.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.831+0000 7f3c85c51700 1 -- 172.21.15.16:0/3939503618 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3c80072590 msgr2=0x7f3c8012ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:49.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.831+0000 7f3c85c51700 1 --2- 172.21.15.16:0/3939503618 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3c80072590 0x7f3c8012ebb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3c70000c00 tx=0x7f3c70004750 comp rx=0 tx=0).stop 2023-09-30T22:29:49.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.831+0000 7f3c85c51700 1 -- 172.21.15.16:0/3939503618 shutdown_connections 2023-09-30T22:29:49.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.831+0000 7f3c85c51700 1 --2- 172.21.15.16:0/3939503618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c801306e0 0x7f3c801c8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.831+0000 7f3c85c51700 1 --2- 172.21.15.16:0/3939503618 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3c80072590 0x7f3c8012ebb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.831+0000 7f3c85c51700 1 --2- 172.21.15.16:0/3939503618 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3c8009de00 0x7f3c8012f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.831+0000 7f3c85c51700 1 -- 172.21.15.16:0/3939503618 >> 172.21.15.16:0/3939503618 conn(0x7f3c8009f470 msgr2=0x7f3c800992c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:49.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.831+0000 7f3c85c51700 1 -- 172.21.15.16:0/3939503618 shutdown_connections 2023-09-30T22:29:49.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.831+0000 7f3c85c51700 1 -- 172.21.15.16:0/3939503618 wait complete. 2023-09-30T22:29:49.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.832+0000 7fe03b7fe700 1 -- 172.21.15.16:0/3122562684 >> v1:172.21.15.16:6816/126811 conn(0x7fe02802f510 legacy=0x7fe0280319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:49.846 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:29:49.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.868+0000 7fe041db0700 1 -- 172.21.15.16:0/3122562684 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe03c1352f0 con 0x7fe03c134ec0 2023-09-30T22:29:49.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.868+0000 7fe023fff700 1 -- 172.21.15.16:0/3122562684 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe03c1352f0 con 0x7fe03c134ec0 2023-09-30T22:29:49.869 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:29:49.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.870+0000 7fe041db0700 1 -- 172.21.15.16:0/3122562684 >> v1:172.21.15.16:6816/126811 conn(0x7fe02802f510 legacy=0x7fe0280319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:49.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.871+0000 7fe041db0700 1 -- 172.21.15.16:0/3122562684 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe03c134ec0 msgr2=0x7fe03c1cd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:49.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.871+0000 7fe041db0700 1 --2- 172.21.15.16:0/3122562684 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe03c134ec0 0x7fe03c1cd290 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe03c06f500 tx=0x7fe03000b730 comp rx=0 tx=0).stop 2023-09-30T22:29:49.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.871+0000 7fe041db0700 1 -- 172.21.15.16:0/3122562684 shutdown_connections 2023-09-30T22:29:49.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.871+0000 7fe041db0700 1 --2- 172.21.15.16:0/3122562684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe03c137f20 0x7fe03c11a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.871+0000 7fe041db0700 1 --2- 172.21.15.16:0/3122562684 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe03c06c0e0 0x7fe03c1ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.871+0000 7fe041db0700 1 --2- 172.21.15.16:0/3122562684 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe03c134ec0 0x7fe03c1cd290 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.871+0000 7fe041db0700 1 -- 172.21.15.16:0/3122562684 >> 172.21.15.16:0/3122562684 conn(0x7fe03c0a4210 msgr2=0x7fe03c099c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:49.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.871+0000 7fe041db0700 1 -- 172.21.15.16:0/3122562684 shutdown_connections 2023-09-30T22:29:49.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.871+0000 7fe041db0700 1 -- 172.21.15.16:0/3122562684 wait complete. 2023-09-30T22:29:49.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.874+0000 7f14014f3700 1 -- 172.21.15.16:0/595886090 >> v1:172.21.15.16:6816/126811 conn(0x7f13e002f510 legacy=0x7f13e00319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:49.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.881+0000 7fe36e2ac700 1 -- 172.21.15.16:0/1914311414 >> v1:172.21.15.16:6816/126811 conn(0x7fe34c02f540 legacy=0x7fe34c0319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:49.886 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:29:49.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.893+0000 7f1403757700 1 -- 172.21.15.16:0/595886090 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f13fc11e5d0 con 0x7f13fc131810 2023-09-30T22:29:49.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.893+0000 7f13f1ffb700 1 -- 172.21.15.16:0/595886090 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f13f4045090 con 0x7f13fc131810 2023-09-30T22:29:49.893 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:29:49.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.895+0000 7f1403757700 1 -- 172.21.15.16:0/595886090 >> v1:172.21.15.16:6816/126811 conn(0x7f13e002f510 legacy=0x7f13e00319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:49.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.895+0000 7f1403757700 1 -- 172.21.15.16:0/595886090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13fc131810 msgr2=0x7f13fc11c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:49.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.895+0000 7f1403757700 1 --2- 172.21.15.16:0/595886090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13fc131810 0x7f13fc11c280 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f13f4009dd0 tx=0x7f13f400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.895+0000 7f1403757700 1 -- 172.21.15.16:0/595886090 shutdown_connections 2023-09-30T22:29:49.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.895+0000 7f1403757700 1 --2- 172.21.15.16:0/595886090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13fc131810 0x7f13fc11c280 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.895+0000 7f1403757700 1 --2- 172.21.15.16:0/595886090 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f13fc099a40 0x7f13fc11b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.895+0000 7f1403757700 1 --2- 172.21.15.16:0/595886090 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f13fc09a3b0 0x7f13fc11bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.895+0000 7f1403757700 1 -- 172.21.15.16:0/595886090 >> 172.21.15.16:0/595886090 conn(0x7f13fc0a2aa0 msgr2=0x7f13fc070ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:49.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.895+0000 7f1403757700 1 -- 172.21.15.16:0/595886090 shutdown_connections 2023-09-30T22:29:49.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.895+0000 7f1403757700 1 -- 172.21.15.16:0/595886090 wait complete. 2023-09-30T22:29:49.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.900+0000 7fe370510700 1 -- 172.21.15.16:0/1914311414 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe368071100 con 0x7fe368134610 2023-09-30T22:29:49.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.900+0000 7fe35effd700 1 -- 172.21.15.16:0/1914311414 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe358015030 con 0x7fe368134610 2023-09-30T22:29:49.901 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:29:49.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.902+0000 7fe370510700 1 -- 172.21.15.16:0/1914311414 >> v1:172.21.15.16:6816/126811 conn(0x7fe34c02f540 legacy=0x7fe34c0319f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:49.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.902+0000 7fe370510700 1 -- 172.21.15.16:0/1914311414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe368134610 msgr2=0x7fe36813e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:49.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.902+0000 7fe370510700 1 --2- 172.21.15.16:0/1914311414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe368134610 0x7fe36813e970 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe358009b90 tx=0x7fe358004750 comp rx=0 tx=0).stop 2023-09-30T22:29:49.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.902+0000 7fe370510700 1 -- 172.21.15.16:0/1914311414 shutdown_connections 2023-09-30T22:29:49.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.902+0000 7fe370510700 1 --2- 172.21.15.16:0/1914311414 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe36806e570 0x7fe368140280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.902+0000 7fe370510700 1 --2- 172.21.15.16:0/1914311414 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe36813eeb0 0x7fe36813f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.902+0000 7fe370510700 1 --2- 172.21.15.16:0/1914311414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe368134610 0x7fe36813e970 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:49.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.902+0000 7fe370510700 1 -- 172.21.15.16:0/1914311414 >> 172.21.15.16:0/1914311414 conn(0x7fe3680a2aa0 msgr2=0x7fe368137f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:49.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.903+0000 7fe370510700 1 -- 172.21.15.16:0/1914311414 shutdown_connections 2023-09-30T22:29:49.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:49.903+0000 7fe370510700 1 -- 172.21.15.16:0/1914311414 wait complete. 2023-09-30T22:29:49.910 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:29:49.917 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:29:50.846 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:29:50.887 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:29:50.911 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:29:50.917 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:29:51.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.011+0000 7f6777791700 1 Processor -- start 2023-09-30T22:29:51.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.011+0000 7f6777791700 1 -- start start 2023-09-30T22:29:51.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.012+0000 7f6777791700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f677009d6f0 0x7f677009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:51.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.012+0000 7f6777791700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f677009e410 0x7f6770130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:51.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.012+0000 7f6777791700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f677009f130 con 0x7f677009e000 2023-09-30T22:29:51.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.012+0000 7f6777791700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f677009f520 con 0x7f677009d6f0 2023-09-30T22:29:51.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.012+0000 7f6777791700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f677009e930 con 0x7f677009e410 2023-09-30T22:29:51.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.012+0000 7f677552d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f677009d6f0 0x7f677009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:51.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.012+0000 7f677552d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f677009d6f0 0x7f677009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48002/0 (socket says 172.21.15.16:48002) 2023-09-30T22:29:51.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.012+0000 7f677552d700 1 -- 172.21.15.16:0/1536062317 learned_addr learned my addr 172.21.15.16:0/1536062317 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:51.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.012+0000 7f6775d2e700 1 --2- 172.21.15.16:0/1536062317 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f677009e410 0x7f6770130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:51.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.013+0000 7f677552d700 1 -- 172.21.15.16:0/1536062317 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f677009e410 msgr2=0x7f6770130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:51.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.013+0000 7f677552d700 1 --2- 172.21.15.16:0/1536062317 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f677009e410 0x7f6770130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:51.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.013+0000 7f677552d700 1 -- 172.21.15.16:0/1536062317 >> v1:172.21.15.16:6789/0 conn(0x7f677009e000 legacy=0x7f6770130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:51.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.013+0000 7f677552d700 1 -- 172.21.15.16:0/1536062317 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f677009b2d0 con 0x7f677009d6f0 2023-09-30T22:29:51.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.013+0000 7f677552d700 1 --2- 172.21.15.16:0/1536062317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f677009d6f0 0x7f677009dac0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f676c0098f0 tx=0x7f676c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bb69b13f4246ddc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:51.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.013+0000 7f6767fff700 1 -- 172.21.15.16:0/1536062317 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676c011830 con 0x7f677009d6f0 2023-09-30T22:29:51.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.013+0000 7f6767fff700 1 -- 172.21.15.16:0/1536062317 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f676c011990 con 0x7f677009d6f0 2023-09-30T22:29:51.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.013+0000 7f6767fff700 1 -- 172.21.15.16:0/1536062317 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676c011d20 con 0x7f677009d6f0 2023-09-30T22:29:51.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.014+0000 7f6777791700 1 -- 172.21.15.16:0/1536062317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f677009d6f0 msgr2=0x7f677009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:51.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.014+0000 7f6777791700 1 --2- 172.21.15.16:0/1536062317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f677009d6f0 0x7f677009dac0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f676c0098f0 tx=0x7f676c009c00 comp rx=0 tx=0).stop 2023-09-30T22:29:51.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.015+0000 7f6777791700 1 -- 172.21.15.16:0/1536062317 shutdown_connections 2023-09-30T22:29:51.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.015+0000 7f6777791700 1 --2- 172.21.15.16:0/1536062317 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f677009e410 0x7f6770130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:51.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.015+0000 7f6777791700 1 --2- 172.21.15.16:0/1536062317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f677009d6f0 0x7f677009dac0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:51.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.015+0000 7f6777791700 1 -- 172.21.15.16:0/1536062317 >> 172.21.15.16:0/1536062317 conn(0x7f6770066f50 msgr2=0x7f677009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:51.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.015+0000 7f6777791700 1 -- 172.21.15.16:0/1536062317 shutdown_connections 2023-09-30T22:29:51.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.015+0000 7f6777791700 1 -- 172.21.15.16:0/1536062317 wait complete. 2023-09-30T22:29:51.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.016+0000 7f6777791700 1 Processor -- start 2023-09-30T22:29:51.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.016+0000 7f6777791700 1 -- start start 2023-09-30T22:29:51.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.017+0000 7f6777791700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f677009d6f0 0x7f6770120360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:51.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.017+0000 7f6777791700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f677009e000 0x7f67701208a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:51.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.017+0000 7f6777791700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f677009e410 0x7f6770120de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:51.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.017+0000 7f6777791700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6770123c80 con 0x7f677009d6f0 2023-09-30T22:29:51.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.017+0000 7f6777791700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6770123e80 con 0x7f677009e410 2023-09-30T22:29:51.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.017+0000 7f6777791700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f67701235f0 con 0x7f677009e000 2023-09-30T22:29:51.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.017+0000 7f677552d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f677009d6f0 0x7f6770120360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:51.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.017+0000 7f6774d2c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f677009e000 0x7f67701208a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:51.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.017+0000 7f677552d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f677009d6f0 0x7f6770120360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48014/0 (socket says 172.21.15.16:48014) 2023-09-30T22:29:51.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.017+0000 7f677552d700 1 -- 172.21.15.16:0/1415614809 learned_addr learned my addr 172.21.15.16:0/1415614809 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:51.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.017+0000 7f6774d2c700 1 --2- 172.21.15.16:0/1415614809 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f677009e000 0x7f67701208a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:52024/0 (socket says 172.21.15.16:52024) 2023-09-30T22:29:51.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.018+0000 7f6775d2e700 1 --2- 172.21.15.16:0/1415614809 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f677009e410 0x7f6770120de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:51.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.018+0000 7f6774d2c700 1 -- 172.21.15.16:0/1415614809 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f677009e410 msgr2=0x7f6770120de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:51.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.018+0000 7f6774d2c700 1 --2- 172.21.15.16:0/1415614809 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f677009e410 0x7f6770120de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:51.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.018+0000 7f6774d2c700 1 -- 172.21.15.16:0/1415614809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f677009d6f0 msgr2=0x7f6770120360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:51.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.018+0000 7f6774d2c700 1 --2- 172.21.15.16:0/1415614809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f677009d6f0 0x7f6770120360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:51.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.018+0000 7f6774d2c700 1 -- 172.21.15.16:0/1415614809 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f676c009550 con 0x7f677009e000 2023-09-30T22:29:51.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.018+0000 7f6774d2c700 1 --2- 172.21.15.16:0/1415614809 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f677009e000 0x7f67701208a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f676000c450 tx=0x7f676000c760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:51.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.018+0000 7f6765ffb700 1 -- 172.21.15.16:0/1415614809 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67600052d0 con 0x7f677009e000 2023-09-30T22:29:51.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.018+0000 7f6765ffb700 1 -- 172.21.15.16:0/1415614809 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6760005430 con 0x7f677009e000 2023-09-30T22:29:51.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.018+0000 7f6765ffb700 1 -- 172.21.15.16:0/1415614809 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67600038c0 con 0x7f677009e000 2023-09-30T22:29:51.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.018+0000 7f6777791700 1 -- 172.21.15.16:0/1415614809 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6770122010 con 0x7f677009e000 2023-09-30T22:29:51.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.019+0000 7f6777791700 1 -- 172.21.15.16:0/1415614809 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67701238e0 con 0x7f677009e000 2023-09-30T22:29:51.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.020+0000 7f6777791700 1 -- 172.21.15.16:0/1415614809 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6770123ad0 con 0x7f677009e000 2023-09-30T22:29:51.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.020+0000 7f6765ffb700 1 -- 172.21.15.16:0/1415614809 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f6760003a20 con 0x7f677009e000 2023-09-30T22:29:51.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.020+0000 7f6765ffb700 1 -- 172.21.15.16:0/1415614809 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7f676003eee0 con 0x7f677009e000 2023-09-30T22:29:51.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.020+0000 7f677552d700 1 -- 172.21.15.16:0/1415614809 >> v1:172.21.15.16:6816/126811 conn(0x7f675402f510 legacy=0x7f67540319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:51.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.025+0000 7f6765ffb700 1 -- 172.21.15.16:0/1415614809 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6760011590 con 0x7f677009e000 2023-09-30T22:29:51.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.049+0000 7f1d5e97b700 1 Processor -- start 2023-09-30T22:29:51.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.049+0000 7f1d5e97b700 1 -- start start 2023-09-30T22:29:51.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.050+0000 7f1d5e97b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1d5806f5d0 0x7f1d5812ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:51.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.050+0000 7f1d5e97b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d58135c20 0x7f1d58138000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:51.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.050+0000 7f1d5e97b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d5806fa40 con 0x7f1d5812cf60 2023-09-30T22:29:51.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.050+0000 7f1d5e97b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d5812cd90 con 0x7f1d5806f5d0 2023-09-30T22:29:51.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.050+0000 7f1d5e97b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d58071780 con 0x7f1d58135c20 2023-09-30T22:29:51.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.050+0000 7f1d577fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1d5806f5d0 0x7f1d5812ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:51.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.050+0000 7f1d57fff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1d5812cf60 0x7f1d5812f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59164/0 (socket says 172.21.15.16:59164) 2023-09-30T22:29:51.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.050+0000 7f1d57fff700 1 -- 172.21.15.16:0/2028192262 learned_addr learned my addr 172.21.15.16:0/2028192262 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:29:51.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.050+0000 7f1d577fe700 1 -- 172.21.15.16:0/2028192262 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d58135c20 msgr2=0x7f1d58138000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:51.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.050+0000 7f1d5cf18700 1 --2- 172.21.15.16:0/2028192262 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d58135c20 0x7f1d58138000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:51.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.050+0000 7f1d577fe700 1 --2- 172.21.15.16:0/2028192262 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d58135c20 0x7f1d58138000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:51.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.050+0000 7f1d577fe700 1 -- 172.21.15.16:0/2028192262 >> v1:172.21.15.16:6789/0 conn(0x7f1d5812cf60 legacy=0x7f1d5812f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:51.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.050+0000 7f1d577fe700 1 -- 172.21.15.16:0/2028192262 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d58071a40 con 0x7f1d5806f5d0 2023-09-30T22:29:51.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.051+0000 7f1d5cf18700 1 --2- 172.21.15.16:0/2028192262 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d58135c20 0x7f1d58138000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:29:51.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.051+0000 7f1d577fe700 1 --2- 172.21.15.16:0/2028192262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d5806f5d0 0x7f1d5812ffc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c009840 tx=0x7f1d4c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=179ebed10f88754b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:51.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.051+0000 7f1d56ffd700 1 -- 172.21.15.16:0/2028192262 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c010800 con 0x7f1d5806f5d0 2023-09-30T22:29:51.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.051+0000 7f1d56ffd700 1 -- 172.21.15.16:0/2028192262 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d4c010960 con 0x7f1d5806f5d0 2023-09-30T22:29:51.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.051+0000 7f1d56ffd700 1 -- 172.21.15.16:0/2028192262 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c010cf0 con 0x7f1d5806f5d0 2023-09-30T22:29:51.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.052+0000 7f1d5e97b700 1 -- 172.21.15.16:0/2028192262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d5806f5d0 msgr2=0x7f1d5812ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:51.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.052+0000 7f1d5e97b700 1 --2- 172.21.15.16:0/2028192262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d5806f5d0 0x7f1d5812ffc0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c009840 tx=0x7f1d4c009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:51.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.052+0000 7f1d5e97b700 1 -- 172.21.15.16:0/2028192262 shutdown_connections 2023-09-30T22:29:51.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.052+0000 7f1d5e97b700 1 --2- 172.21.15.16:0/2028192262 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d58135c20 0x7f1d58138000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:51.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.052+0000 7f1d5e97b700 1 --2- 172.21.15.16:0/2028192262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d5806f5d0 0x7f1d5812ffc0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:51.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.052+0000 7f1d5e97b700 1 -- 172.21.15.16:0/2028192262 >> 172.21.15.16:0/2028192262 conn(0x7f1d580a2aa0 msgr2=0x7f1d580a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:51.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.052+0000 7f1d5e97b700 1 -- 172.21.15.16:0/2028192262 shutdown_connections 2023-09-30T22:29:51.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.052+0000 7f1d5e97b700 1 -- 172.21.15.16:0/2028192262 wait complete. 2023-09-30T22:29:51.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.053+0000 7f1d5e97b700 1 Processor -- start 2023-09-30T22:29:51.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.053+0000 7f1d5e97b700 1 -- start start 2023-09-30T22:29:51.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.054+0000 7f1d5e97b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1d5806f5d0 0x7f1d5809d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:51.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.054+0000 7f1d5e97b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d5812cf60 0x7f1d5809dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:51.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.054+0000 7f1d5e97b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d58135c20 0x7f1d5809e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:51.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.054+0000 7f1d5e97b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5811a420 con 0x7f1d5812cf60 2023-09-30T22:29:51.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.054+0000 7f1d5e97b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5809ebf0 con 0x7f1d58135c20 2023-09-30T22:29:51.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.054+0000 7f1d5e97b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5811ae70 con 0x7f1d5806f5d0 2023-09-30T22:29:51.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.054+0000 7f1d5cf18700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d58135c20 0x7f1d5809e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:51.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.054+0000 7f1d5cf18700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d58135c20 0x7f1d5809e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:49448/0 (socket says 172.21.15.16:49448) 2023-09-30T22:29:51.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.054+0000 7f1d5cf18700 1 -- 172.21.15.16:0/2294266980 learned_addr learned my addr 172.21.15.16:0/2294266980 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:51.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.054+0000 7f1d577fe700 1 --2- 172.21.15.16:0/2294266980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d5812cf60 0x7f1d5809dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:51.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.054+0000 7f1d5cf18700 1 -- 172.21.15.16:0/2294266980 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1d5806f5d0 msgr2=0x7f1d5809d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:51.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.054+0000 7f1d5cf18700 1 --2- 172.21.15.16:0/2294266980 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1d5806f5d0 0x7f1d5809d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:51.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.054+0000 7f1d5cf18700 1 -- 172.21.15.16:0/2294266980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d5812cf60 msgr2=0x7f1d5809dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:51.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.054+0000 7f1d5cf18700 1 --2- 172.21.15.16:0/2294266980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d5812cf60 0x7f1d5809dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:51.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.054+0000 7f1d5cf18700 1 -- 172.21.15.16:0/2294266980 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d4c0094a0 con 0x7f1d58135c20 2023-09-30T22:29:51.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.054+0000 7f1d577fe700 1 --2- 172.21.15.16:0/2294266980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d5812cf60 0x7f1d5809dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:29:51.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.054+0000 7f1d5cf18700 1 --2- 172.21.15.16:0/2294266980 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d58135c20 0x7f1d5809e1c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1d5000ceb0 tx=0x7f1d5000cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:51.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.055+0000 7f1d54ff9700 1 -- 172.21.15.16:0/2294266980 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5000ae10 con 0x7f1d58135c20 2023-09-30T22:29:51.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.055+0000 7f1d54ff9700 1 -- 172.21.15.16:0/2294266980 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d50015440 con 0x7f1d58135c20 2023-09-30T22:29:51.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.055+0000 7f1d5e97b700 1 -- 172.21.15.16:0/2294266980 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d58119f60 con 0x7f1d58135c20 2023-09-30T22:29:51.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.055+0000 7f1d54ff9700 1 -- 172.21.15.16:0/2294266980 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d500157f0 con 0x7f1d58135c20 2023-09-30T22:29:51.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.055+0000 7f1d5e97b700 1 -- 172.21.15.16:0/2294266980 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d580a0580 con 0x7f1d58135c20 2023-09-30T22:29:51.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.056+0000 7f1d54ff9700 1 -- 172.21.15.16:0/2294266980 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f1d5000d430 con 0x7f1d58135c20 2023-09-30T22:29:51.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.056+0000 7f1d5e97b700 1 -- 172.21.15.16:0/2294266980 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d58074ea0 con 0x7f1d58135c20 2023-09-30T22:29:51.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.056+0000 7f1d54ff9700 1 -- 172.21.15.16:0/2294266980 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7f1d50040290 con 0x7f1d58135c20 2023-09-30T22:29:51.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.056+0000 7f1d57fff700 1 -- 172.21.15.16:0/2294266980 >> v1:172.21.15.16:6816/126811 conn(0x7f1d4402f510 legacy=0x7f1d440319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:51.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.065+0000 7f1d54ff9700 1 -- 172.21.15.16:0/2294266980 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1d5001a080 con 0x7f1d58135c20 2023-09-30T22:29:51.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.083+0000 7f117de2d700 1 Processor -- start 2023-09-30T22:29:51.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.083+0000 7f117de2d700 1 -- start start 2023-09-30T22:29:51.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.083+0000 7f117de2d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f117809d6f0 0x7f117809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:51.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.083+0000 7f117de2d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f117809e410 0x7f1178130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:51.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.083+0000 7f117de2d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f117809f130 con 0x7f117809e000 2023-09-30T22:29:51.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.083+0000 7f117de2d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f117809f520 con 0x7f117809e410 2023-09-30T22:29:51.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.083+0000 7f117de2d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f117809e930 con 0x7f117809d6f0 2023-09-30T22:29:51.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.083+0000 7f1177fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f117809e410 0x7f1178130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:51.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.084+0000 7f1176ffd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f117809e000 0x7f1178130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59172/0 (socket says 172.21.15.16:59172) 2023-09-30T22:29:51.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.084+0000 7f1177fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f117809e410 0x7f1178130d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48048/0 (socket says 172.21.15.16:48048) 2023-09-30T22:29:51.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.084+0000 7f1176ffd700 1 -- 172.21.15.16:0/3238954492 learned_addr learned my addr 172.21.15.16:0/3238954492 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:29:51.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.084+0000 7f11777fe700 1 --2- 172.21.15.16:0/3238954492 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f117809d6f0 0x7f117809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:51.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.084+0000 7f11767fc700 1 -- 172.21.15.16:0/3238954492 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1205408559 0 0) 0x7f117809f130 con 0x7f117809e000 2023-09-30T22:29:51.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.084+0000 7f11767fc700 1 -- 172.21.15.16:0/3238954492 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11600035b0 con 0x7f117809e000 2023-09-30T22:29:51.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.084+0000 7f1177fff700 1 -- 172.21.15.16:0/3238954492 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f117809d6f0 msgr2=0x7f117809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:51.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.084+0000 7f1177fff700 1 --2- 172.21.15.16:0/3238954492 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f117809d6f0 0x7f117809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:51.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.084+0000 7f1177fff700 1 -- 172.21.15.16:0/3238954492 >> v1:172.21.15.16:6789/0 conn(0x7f117809e000 legacy=0x7f1178130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:51.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.084+0000 7f1177fff700 1 -- 172.21.15.16:0/3238954492 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f117809b2d0 con 0x7f117809e410 2023-09-30T22:29:51.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.084+0000 7f1177fff700 1 --2- 172.21.15.16:0/3238954492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f117809e410 0x7f1178130d70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f117000bfa0 tx=0x7f117000d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c731a579fc10446 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:51.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.084+0000 7f11767fc700 1 -- 172.21.15.16:0/3238954492 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117000d940 con 0x7f117809e410 2023-09-30T22:29:51.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.084+0000 7f11767fc700 1 -- 172.21.15.16:0/3238954492 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f117000daa0 con 0x7f117809e410 2023-09-30T22:29:51.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.084+0000 7f11767fc700 1 -- 172.21.15.16:0/3238954492 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117000de50 con 0x7f117809e410 2023-09-30T22:29:51.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.085+0000 7f117de2d700 1 -- 172.21.15.16:0/3238954492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f117809e410 msgr2=0x7f1178130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:51.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.085+0000 7f117de2d700 1 --2- 172.21.15.16:0/3238954492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f117809e410 0x7f1178130d70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f117000bfa0 tx=0x7f117000d330 comp rx=0 tx=0).stop 2023-09-30T22:29:51.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.085+0000 7f117de2d700 1 -- 172.21.15.16:0/3238954492 shutdown_connections 2023-09-30T22:29:51.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.085+0000 7f117de2d700 1 --2- 172.21.15.16:0/3238954492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f117809e410 0x7f1178130d70 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:51.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.085+0000 7f117de2d700 1 --2- 172.21.15.16:0/3238954492 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f117809d6f0 0x7f117809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:51.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.085+0000 7f117de2d700 1 -- 172.21.15.16:0/3238954492 >> 172.21.15.16:0/3238954492 conn(0x7f1178066f50 msgr2=0x7f117809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:51.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.085+0000 7f117de2d700 1 -- 172.21.15.16:0/3238954492 shutdown_connections 2023-09-30T22:29:51.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.085+0000 7f117de2d700 1 -- 172.21.15.16:0/3238954492 wait complete. 2023-09-30T22:29:51.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.087+0000 7f117de2d700 1 Processor -- start 2023-09-30T22:29:51.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.087+0000 7f117de2d700 1 -- start start 2023-09-30T22:29:51.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.088+0000 7f117de2d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f117809d6f0 0x7f117811dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:51.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.088+0000 7f117de2d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f117809e000 0x7f117811e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:51.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.088+0000 7f117de2d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f117809e410 0x7f117811e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:51.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.088+0000 7f117de2d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11781210e0 con 0x7f117809e410 2023-09-30T22:29:51.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.088+0000 7f117de2d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f117811fa00 con 0x7f117809d6f0 2023-09-30T22:29:51.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.088+0000 7f117de2d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1178121590 con 0x7f117809e000 2023-09-30T22:29:51.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.088+0000 7f11777fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f117809d6f0 0x7f117811dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:51.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.088+0000 7f11777fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f117809d6f0 0x7f117811dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:49450/0 (socket says 172.21.15.16:49450) 2023-09-30T22:29:51.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.088+0000 7f11777fe700 1 -- 172.21.15.16:0/2032930320 learned_addr learned my addr 172.21.15.16:0/2032930320 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:51.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.088+0000 7f1176ffd700 1 --2- 172.21.15.16:0/2032930320 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f117809e000 0x7f117811e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:51.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.088+0000 7f1177fff700 1 --2- 172.21.15.16:0/2032930320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f117809e410 0x7f117811e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:51.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.088+0000 7f11777fe700 1 -- 172.21.15.16:0/2032930320 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f117809e000 msgr2=0x7f117811e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:51.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.088+0000 7f11777fe700 1 --2- 172.21.15.16:0/2032930320 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f117809e000 0x7f117811e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:51.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.088+0000 7f11777fe700 1 -- 172.21.15.16:0/2032930320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f117809e410 msgr2=0x7f117811e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:51.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.088+0000 7f11777fe700 1 --2- 172.21.15.16:0/2032930320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f117809e410 0x7f117811e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:51.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.088+0000 7f11777fe700 1 -- 172.21.15.16:0/2032930320 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f117000bc80 con 0x7f117809d6f0 2023-09-30T22:29:51.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.089+0000 7f1176ffd700 1 --2- 172.21.15.16:0/2032930320 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f117809e000 0x7f117811e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:51.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.089+0000 7f11777fe700 1 --2- 172.21.15.16:0/2032930320 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f117809d6f0 0x7f117811dd50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f116800d800 tx=0x7f116800db10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:51.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.089+0000 7f115ffff700 1 -- 172.21.15.16:0/2032930320 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11680125f0 con 0x7f117809d6f0 2023-09-30T22:29:51.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.089+0000 7f115ffff700 1 -- 172.21.15.16:0/2032930320 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1168012750 con 0x7f117809d6f0 2023-09-30T22:29:51.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.089+0000 7f117de2d700 1 -- 172.21.15.16:0/2032930320 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11780a3bf0 con 0x7f117809d6f0 2023-09-30T22:29:51.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.089+0000 7f115ffff700 1 -- 172.21.15.16:0/2032930320 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1168012b00 con 0x7f117809d6f0 2023-09-30T22:29:51.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.089+0000 7f117de2d700 1 -- 172.21.15.16:0/2032930320 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1178121870 con 0x7f117809d6f0 2023-09-30T22:29:51.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.090+0000 7f117de2d700 1 -- 172.21.15.16:0/2032930320 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f117811ac20 con 0x7f117809d6f0 2023-09-30T22:29:51.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.089+0000 7f716d0b6700 1 Processor -- start 2023-09-30T22:29:51.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.089+0000 7f716d0b6700 1 -- start start 2023-09-30T22:29:51.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.090+0000 7f716d0b6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f716809d6f0 0x7f716809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:51.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.090+0000 7f716d0b6700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f716809e000 0x7f7168130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:51.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.090+0000 7f716d0b6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f716809e410 0x7f7168130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:51.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.090+0000 7f716d0b6700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f716809f520 con 0x7f716809e000 2023-09-30T22:29:51.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.090+0000 7f716d0b6700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f716809f130 con 0x7f716809d6f0 2023-09-30T22:29:51.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.090+0000 7f716d0b6700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f716809e930 con 0x7f716809e410 2023-09-30T22:29:51.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.090+0000 7f716759e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f716809e410 0x7f7168130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:51.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.090+0000 7f716759e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f716809e410 0x7f7168130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:52068/0 (socket says 172.21.15.16:52068) 2023-09-30T22:29:51.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.090+0000 7f716759e700 1 -- 172.21.15.16:0/1088519230 learned_addr learned my addr 172.21.15.16:0/1088519230 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:51.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.090+0000 7f716659c700 1 --2- 172.21.15.16:0/1088519230 >> v2:172.21.15.16:3300/0 conn(0x7f716809e000 0x7f7168130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:51.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.090+0000 7f716759e700 1 -- 172.21.15.16:0/1088519230 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f716809d6f0 msgr2=0x7f716809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:51.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.090+0000 7f716759e700 1 --2- 172.21.15.16:0/1088519230 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f716809d6f0 0x7f716809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:51.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.090+0000 7f716759e700 1 -- 172.21.15.16:0/1088519230 >> v2:172.21.15.16:3300/0 conn(0x7f716809e000 msgr2=0x7f7168130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:51.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.091+0000 7f716759e700 1 --2- 172.21.15.16:0/1088519230 >> v2:172.21.15.16:3300/0 conn(0x7f716809e000 0x7f7168130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:51.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.091+0000 7f716759e700 1 -- 172.21.15.16:0/1088519230 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f716809b2d0 con 0x7f716809e410 2023-09-30T22:29:51.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.091+0000 7f716659c700 1 --2- 172.21.15.16:0/1088519230 >> v2:172.21.15.16:3300/0 conn(0x7f716809e000 0x7f7168130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:29:51.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.091+0000 7f716759e700 1 --2- 172.21.15.16:0/1088519230 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f716809e410 0x7f7168130ba0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7160009e80 tx=0x7f716000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=566ac12a9bb5f506 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:51.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.091+0000 7f7165d9b700 1 -- 172.21.15.16:0/1088519230 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716000e070 con 0x7f716809e410 2023-09-30T22:29:51.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.091+0000 7f7165d9b700 1 -- 172.21.15.16:0/1088519230 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f716000b980 con 0x7f716809e410 2023-09-30T22:29:51.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.091+0000 7f7165d9b700 1 -- 172.21.15.16:0/1088519230 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716000bd30 con 0x7f716809e410 2023-09-30T22:29:51.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.091+0000 7f115ffff700 1 -- 172.21.15.16:0/2032930320 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f1168012c60 con 0x7f117809d6f0 2023-09-30T22:29:51.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.092+0000 7f115ffff700 1 -- 172.21.15.16:0/2032930320 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7f11680406f0 con 0x7f117809d6f0 2023-09-30T22:29:51.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.092+0000 7f1176ffd700 1 -- 172.21.15.16:0/2032930320 >> v1:172.21.15.16:6816/126811 conn(0x7f116402f510 legacy=0x7f11640319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:51.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.092+0000 7f716d0b6700 1 -- 172.21.15.16:0/1088519230 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f716809e410 msgr2=0x7f7168130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:51.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.092+0000 7f716d0b6700 1 --2- 172.21.15.16:0/1088519230 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f716809e410 0x7f7168130ba0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7160009e80 tx=0x7f716000b330 comp rx=0 tx=0).stop 2023-09-30T22:29:51.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.092+0000 7f716d0b6700 1 -- 172.21.15.16:0/1088519230 shutdown_connections 2023-09-30T22:29:51.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.092+0000 7f716d0b6700 1 --2- 172.21.15.16:0/1088519230 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f716809d6f0 0x7f716809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:51.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.092+0000 7f716d0b6700 1 --2- 172.21.15.16:0/1088519230 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f716809e410 0x7f7168130ba0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:51.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.092+0000 7f716d0b6700 1 --2- 172.21.15.16:0/1088519230 >> v2:172.21.15.16:3300/0 conn(0x7f716809e000 0x7f7168130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:51.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.092+0000 7f716d0b6700 1 -- 172.21.15.16:0/1088519230 >> 172.21.15.16:0/1088519230 conn(0x7f7168066f50 msgr2=0x7f716809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:51.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.092+0000 7f716d0b6700 1 -- 172.21.15.16:0/1088519230 shutdown_connections 2023-09-30T22:29:51.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.092+0000 7f716d0b6700 1 -- 172.21.15.16:0/1088519230 wait complete. 2023-09-30T22:29:51.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.094+0000 7f716d0b6700 1 Processor -- start 2023-09-30T22:29:51.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.094+0000 7f716d0b6700 1 -- start start 2023-09-30T22:29:51.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.095+0000 7f716d0b6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f716809d6f0 0x7f7168088260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:51.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.095+0000 7f716d0b6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f716809e000 0x7f71680887a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:51.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.095+0000 7f716d0b6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f716809e410 0x7f71681bb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:51.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.095+0000 7f716d0b6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71680a2120 con 0x7f716809e000 2023-09-30T22:29:51.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.095+0000 7f716d0b6700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7168119670 con 0x7f716809e410 2023-09-30T22:29:51.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.095+0000 7f716d0b6700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7168119c70 con 0x7f716809d6f0 2023-09-30T22:29:51.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.095+0000 7f7166d9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f716809d6f0 0x7f7168088260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:51.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.095+0000 7f7166d9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f716809d6f0 0x7f7168088260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:52078/0 (socket says 172.21.15.16:52078) 2023-09-30T22:29:51.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.095+0000 7f7166d9d700 1 -- 172.21.15.16:0/403685996 learned_addr learned my addr 172.21.15.16:0/403685996 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:51.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.095+0000 7f716759e700 1 --2- 172.21.15.16:0/403685996 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f716809e410 0x7f71681bb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:51.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.095+0000 7f716659c700 1 --2- 172.21.15.16:0/403685996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f716809e000 0x7f71680887a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:51.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.096+0000 7f7166d9d700 1 -- 172.21.15.16:0/403685996 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f716809e410 msgr2=0x7f71681bb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:51.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.096+0000 7f7166d9d700 1 --2- 172.21.15.16:0/403685996 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f716809e410 0x7f71681bb5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:51.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.096+0000 7f7166d9d700 1 -- 172.21.15.16:0/403685996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f716809e000 msgr2=0x7f71680887a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:51.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.096+0000 7f7166d9d700 1 --2- 172.21.15.16:0/403685996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f716809e000 0x7f71680887a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:51.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.096+0000 7f7166d9d700 1 -- 172.21.15.16:0/403685996 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7160009b30 con 0x7f716809d6f0 2023-09-30T22:29:51.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.096+0000 7f716759e700 1 --2- 172.21.15.16:0/403685996 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f716809e410 0x7f71681bb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:29:51.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.096+0000 7f7166d9d700 1 --2- 172.21.15.16:0/403685996 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f716809d6f0 0x7f7168088260 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f715800b530 tx=0x7f715800b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:51.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.096+0000 7f714f7fe700 1 -- 172.21.15.16:0/403685996 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7158011830 con 0x7f716809d6f0 2023-09-30T22:29:51.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.096+0000 7f716d0b6700 1 -- 172.21.15.16:0/403685996 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7168087cc0 con 0x7f716809d6f0 2023-09-30T22:29:51.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.096+0000 7f714f7fe700 1 -- 172.21.15.16:0/403685996 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7158011990 con 0x7f716809d6f0 2023-09-30T22:29:51.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.096+0000 7f714f7fe700 1 -- 172.21.15.16:0/403685996 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7158011d40 con 0x7f716809d6f0 2023-09-30T22:29:51.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.096+0000 7f716d0b6700 1 -- 172.21.15.16:0/403685996 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7168089410 con 0x7f716809d6f0 2023-09-30T22:29:51.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.098+0000 7f714f7fe700 1 -- 172.21.15.16:0/403685996 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f715801a400 con 0x7f716809d6f0 2023-09-30T22:29:51.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.097+0000 7f115ffff700 1 -- 172.21.15.16:0/2032930320 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f116801b950 con 0x7f117809d6f0 2023-09-30T22:29:51.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.098+0000 7f716d0b6700 1 -- 172.21.15.16:0/403685996 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7168074ea0 con 0x7f716809d6f0 2023-09-30T22:29:51.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.098+0000 7f714f7fe700 1 -- 172.21.15.16:0/403685996 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7f715803fcb0 con 0x7f716809d6f0 2023-09-30T22:29:51.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.098+0000 7f716659c700 1 -- 172.21.15.16:0/403685996 >> v1:172.21.15.16:6816/126811 conn(0x7f715402f540 legacy=0x7f71540319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:51.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.103+0000 7f714f7fe700 1 -- 172.21.15.16:0/403685996 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7158015070 con 0x7f716809d6f0 2023-09-30T22:29:51.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.221+0000 7f677552d700 1 -- 172.21.15.16:0/1415614809 >> v1:172.21.15.16:6816/126811 conn(0x7f675402f510 legacy=0x7f67540319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:51.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.257+0000 7f1d57fff700 1 -- 172.21.15.16:0/2294266980 >> v1:172.21.15.16:6816/126811 conn(0x7f1d4402f510 legacy=0x7f1d440319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:51.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.264+0000 7f6777791700 1 -- 172.21.15.16:0/1415614809 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f6770074ea0 con 0x7f677009e000 2023-09-30T22:29:51.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.265+0000 7f6765ffb700 1 -- 172.21.15.16:0/1415614809 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6760017020 con 0x7f677009e000 2023-09-30T22:29:51.265 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:29:51.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.267+0000 7f6777791700 1 -- 172.21.15.16:0/1415614809 >> v1:172.21.15.16:6816/126811 conn(0x7f675402f510 legacy=0x7f67540319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:51.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.267+0000 7f6777791700 1 -- 172.21.15.16:0/1415614809 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f677009e000 msgr2=0x7f67701208a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:51.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.267+0000 7f6777791700 1 --2- 172.21.15.16:0/1415614809 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f677009e000 0x7f67701208a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f676000c450 tx=0x7f676000c760 comp rx=0 tx=0).stop 2023-09-30T22:29:51.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.267+0000 7f6777791700 1 -- 172.21.15.16:0/1415614809 shutdown_connections 2023-09-30T22:29:51.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.267+0000 7f6777791700 1 --2- 172.21.15.16:0/1415614809 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f677009e410 0x7f6770120de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:51.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.267+0000 7f6777791700 1 --2- 172.21.15.16:0/1415614809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f677009d6f0 0x7f6770120360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:51.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.267+0000 7f6777791700 1 --2- 172.21.15.16:0/1415614809 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f677009e000 0x7f67701208a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:51.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.267+0000 7f6777791700 1 -- 172.21.15.16:0/1415614809 >> 172.21.15.16:0/1415614809 conn(0x7f6770066f50 msgr2=0x7f677006a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:51.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.267+0000 7f6777791700 1 -- 172.21.15.16:0/1415614809 shutdown_connections 2023-09-30T22:29:51.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.267+0000 7f6777791700 1 -- 172.21.15.16:0/1415614809 wait complete. 2023-09-30T22:29:51.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.281+0000 7f1d5e97b700 1 -- 172.21.15.16:0/2294266980 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1d5811ac60 con 0x7f1d58135c20 2023-09-30T22:29:51.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.281+0000 7f1d54ff9700 1 -- 172.21.15.16:0/2294266980 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1d50008910 con 0x7f1d58135c20 2023-09-30T22:29:51.282 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:29:51.283 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:29:51.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.284+0000 7f1d5e97b700 1 -- 172.21.15.16:0/2294266980 >> v1:172.21.15.16:6816/126811 conn(0x7f1d4402f510 legacy=0x7f1d440319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:51.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.284+0000 7f1d5e97b700 1 -- 172.21.15.16:0/2294266980 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d58135c20 msgr2=0x7f1d5809e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:51.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.284+0000 7f1d5e97b700 1 --2- 172.21.15.16:0/2294266980 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d58135c20 0x7f1d5809e1c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1d5000ceb0 tx=0x7f1d5000cf90 comp rx=0 tx=0).stop 2023-09-30T22:29:51.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.284+0000 7f1d5e97b700 1 -- 172.21.15.16:0/2294266980 shutdown_connections 2023-09-30T22:29:51.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.284+0000 7f1d5e97b700 1 --2- 172.21.15.16:0/2294266980 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d58135c20 0x7f1d5809e1c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:51.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.284+0000 7f1d5e97b700 1 --2- 172.21.15.16:0/2294266980 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1d5806f5d0 0x7f1d5809d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:51.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.284+0000 7f1d5e97b700 1 --2- 172.21.15.16:0/2294266980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d5812cf60 0x7f1d5809dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:51.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.284+0000 7f1d5e97b700 1 -- 172.21.15.16:0/2294266980 >> 172.21.15.16:0/2294266980 conn(0x7f1d580a2aa0 msgr2=0x7f1d5812a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:51.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.284+0000 7f1d5e97b700 1 -- 172.21.15.16:0/2294266980 shutdown_connections 2023-09-30T22:29:51.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.284+0000 7f1d5e97b700 1 -- 172.21.15.16:0/2294266980 wait complete. 2023-09-30T22:29:51.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.293+0000 7f1176ffd700 1 -- 172.21.15.16:0/2032930320 >> v1:172.21.15.16:6816/126811 conn(0x7f116402f510 legacy=0x7f11640319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:51.298 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:29:51.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.299+0000 7f716659c700 1 -- 172.21.15.16:0/403685996 >> v1:172.21.15.16:6816/126811 conn(0x7f715402f540 legacy=0x7f71540319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:51.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.305+0000 7f117de2d700 1 -- 172.21.15.16:0/2032930320 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1178074ea0 con 0x7f117809d6f0 2023-09-30T22:29:51.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.306+0000 7f115ffff700 1 -- 172.21.15.16:0/2032930320 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f116801b360 con 0x7f117809d6f0 2023-09-30T22:29:51.306 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:29:51.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.308+0000 7f117de2d700 1 -- 172.21.15.16:0/2032930320 >> v1:172.21.15.16:6816/126811 conn(0x7f116402f510 legacy=0x7f11640319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:51.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.308+0000 7f117de2d700 1 -- 172.21.15.16:0/2032930320 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f117809d6f0 msgr2=0x7f117811dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:51.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.308+0000 7f117de2d700 1 --2- 172.21.15.16:0/2032930320 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f117809d6f0 0x7f117811dd50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f116800d800 tx=0x7f116800db10 comp rx=0 tx=0).stop 2023-09-30T22:29:51.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.308+0000 7f117de2d700 1 -- 172.21.15.16:0/2032930320 shutdown_connections 2023-09-30T22:29:51.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.308+0000 7f117de2d700 1 --2- 172.21.15.16:0/2032930320 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f117809d6f0 0x7f117811dd50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:51.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.308+0000 7f117de2d700 1 --2- 172.21.15.16:0/2032930320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f117809e410 0x7f117811e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:51.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.308+0000 7f117de2d700 1 --2- 172.21.15.16:0/2032930320 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f117809e000 0x7f117811e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:51.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.308+0000 7f117de2d700 1 -- 172.21.15.16:0/2032930320 >> 172.21.15.16:0/2032930320 conn(0x7f1178066f50 msgr2=0x7f1178130990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:51.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.308+0000 7f117de2d700 1 -- 172.21.15.16:0/2032930320 shutdown_connections 2023-09-30T22:29:51.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.308+0000 7f117de2d700 1 -- 172.21.15.16:0/2032930320 wait complete. 2023-09-30T22:29:51.322 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:29:51.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.336+0000 7f716d0b6700 1 -- 172.21.15.16:0/403685996 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7168071e90 con 0x7f716809d6f0 2023-09-30T22:29:51.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.336+0000 7f714f7fe700 1 -- 172.21.15.16:0/403685996 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f71580193b0 con 0x7f716809d6f0 2023-09-30T22:29:51.337 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:29:51.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.338+0000 7f716d0b6700 1 -- 172.21.15.16:0/403685996 >> v1:172.21.15.16:6816/126811 conn(0x7f715402f540 legacy=0x7f71540319f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:51.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.338+0000 7f716d0b6700 1 -- 172.21.15.16:0/403685996 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f716809d6f0 msgr2=0x7f7168088260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:51.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.338+0000 7f716d0b6700 1 --2- 172.21.15.16:0/403685996 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f716809d6f0 0x7f7168088260 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f715800b530 tx=0x7f715800b840 comp rx=0 tx=0).stop 2023-09-30T22:29:51.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.338+0000 7f716d0b6700 1 -- 172.21.15.16:0/403685996 shutdown_connections 2023-09-30T22:29:51.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.338+0000 7f716d0b6700 1 --2- 172.21.15.16:0/403685996 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f716809e410 0x7f71681bb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:51.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.338+0000 7f716d0b6700 1 --2- 172.21.15.16:0/403685996 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f716809d6f0 0x7f7168088260 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:51.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.338+0000 7f716d0b6700 1 --2- 172.21.15.16:0/403685996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f716809e000 0x7f71680887a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:51.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.338+0000 7f716d0b6700 1 -- 172.21.15.16:0/403685996 >> 172.21.15.16:0/403685996 conn(0x7f7168066f50 msgr2=0x7f7168070e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:51.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.338+0000 7f716d0b6700 1 -- 172.21.15.16:0/403685996 shutdown_connections 2023-09-30T22:29:51.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:51.338+0000 7f716d0b6700 1 -- 172.21.15.16:0/403685996 wait complete. 2023-09-30T22:29:51.352 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:29:52.284 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:29:52.299 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:29:52.323 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:29:52.353 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:29:52.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.444+0000 7f39f607d700 1 Processor -- start 2023-09-30T22:29:52.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.444+0000 7f39f607d700 1 -- start start 2023-09-30T22:29:52.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.445+0000 7f39f607d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f39f0130540 0x7f39f0130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:52.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.445+0000 7f39f607d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f39f0130e50 0x7f39f006f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:52.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.445+0000 7f39f607d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f39f006fb20 0x7f39f006ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:52.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.445+0000 7f39f607d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39f009a390 con 0x7f39f006fb20 2023-09-30T22:29:52.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.445+0000 7f39f607d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39f00716a0 con 0x7f39f0130540 2023-09-30T22:29:52.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.445+0000 7f39f607d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f39f0071290 con 0x7f39f0130e50 2023-09-30T22:29:52.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.445+0000 7f39ef7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f39f0130540 0x7f39f0130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:52.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.445+0000 7f39ef7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f39f0130540 0x7f39f0130910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:49474/0 (socket says 172.21.15.16:49474) 2023-09-30T22:29:52.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.445+0000 7f39ef7fe700 1 -- 172.21.15.16:0/3353598369 learned_addr learned my addr 172.21.15.16:0/3353598369 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:52.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.445+0000 7f39eeffd700 1 --2- 172.21.15.16:0/3353598369 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f39f0130e50 0x7f39f006f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:52.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.445+0000 7f39effff700 1 --2- 172.21.15.16:0/3353598369 >> v2:172.21.15.16:3300/0 conn(0x7f39f006fb20 0x7f39f006ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:52.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.446+0000 7f39eeffd700 1 -- 172.21.15.16:0/3353598369 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f39f0130540 msgr2=0x7f39f0130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:52.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.446+0000 7f39eeffd700 1 --2- 172.21.15.16:0/3353598369 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f39f0130540 0x7f39f0130910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.446+0000 7f39eeffd700 1 -- 172.21.15.16:0/3353598369 >> v2:172.21.15.16:3300/0 conn(0x7f39f006fb20 msgr2=0x7f39f006ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:52.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.446+0000 7f39eeffd700 1 --2- 172.21.15.16:0/3353598369 >> v2:172.21.15.16:3300/0 conn(0x7f39f006fb20 0x7f39f006ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.446+0000 7f39eeffd700 1 -- 172.21.15.16:0/3353598369 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f0130340 con 0x7f39f0130e50 2023-09-30T22:29:52.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.446+0000 7f39ef7fe700 1 --2- 172.21.15.16:0/3353598369 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f39f0130540 0x7f39f0130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:52.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.446+0000 7f39effff700 1 --2- 172.21.15.16:0/3353598369 >> v2:172.21.15.16:3300/0 conn(0x7f39f006fb20 0x7f39f006ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:52.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.446+0000 7f39eeffd700 1 --2- 172.21.15.16:0/3353598369 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f39f0130e50 0x7f39f006f5e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f39e4009870 tx=0x7f39e4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db9d4e39fa83bc86 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:52.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.446+0000 7f39ee7fc700 1 -- 172.21.15.16:0/3353598369 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e4011830 con 0x7f39f0130e50 2023-09-30T22:29:52.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.446+0000 7f39ee7fc700 1 -- 172.21.15.16:0/3353598369 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39e4011990 con 0x7f39f0130e50 2023-09-30T22:29:52.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.446+0000 7f39ee7fc700 1 -- 172.21.15.16:0/3353598369 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e4011d20 con 0x7f39f0130e50 2023-09-30T22:29:52.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.447+0000 7f39f607d700 1 -- 172.21.15.16:0/3353598369 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f39f0130e50 msgr2=0x7f39f006f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:52.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.447+0000 7f39f607d700 1 --2- 172.21.15.16:0/3353598369 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f39f0130e50 0x7f39f006f5e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f39e4009870 tx=0x7f39e4009b80 comp rx=0 tx=0).stop 2023-09-30T22:29:52.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.448+0000 7f39f607d700 1 -- 172.21.15.16:0/3353598369 shutdown_connections 2023-09-30T22:29:52.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.448+0000 7f39f607d700 1 --2- 172.21.15.16:0/3353598369 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f39f0130540 0x7f39f0130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.448+0000 7f39f607d700 1 --2- 172.21.15.16:0/3353598369 >> v2:172.21.15.16:3300/0 conn(0x7f39f006fb20 0x7f39f006ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.448+0000 7f39f607d700 1 --2- 172.21.15.16:0/3353598369 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f39f0130e50 0x7f39f006f5e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.448+0000 7f39f607d700 1 -- 172.21.15.16:0/3353598369 >> 172.21.15.16:0/3353598369 conn(0x7f39f00a2aa0 msgr2=0x7f39f00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:52.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.448+0000 7f39f607d700 1 -- 172.21.15.16:0/3353598369 shutdown_connections 2023-09-30T22:29:52.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.448+0000 7f39f607d700 1 -- 172.21.15.16:0/3353598369 wait complete. 2023-09-30T22:29:52.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.449+0000 7f39f607d700 1 Processor -- start 2023-09-30T22:29:52.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.449+0000 7f39f607d700 1 -- start start 2023-09-30T22:29:52.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.450+0000 7f39f607d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f39f006fb20 0x7f39f0132da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:52.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.450+0000 7f39f607d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39f0130540 0x7f39f01332e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:52.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.450+0000 7f39f607d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f39f0130e50 0x7f39f0133820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:52.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.450+0000 7f39f607d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f0119270 con 0x7f39f0130540 2023-09-30T22:29:52.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.450+0000 7f39f607d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39f01357b0 con 0x7f39f006fb20 2023-09-30T22:29:52.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.450+0000 7f39f607d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f39f0134a50 con 0x7f39f0130e50 2023-09-30T22:29:52.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.450+0000 7f39ef7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f39f006fb20 0x7f39f0132da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:52.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.450+0000 7f39ef7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f39f006fb20 0x7f39f0132da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:49486/0 (socket says 172.21.15.16:49486) 2023-09-30T22:29:52.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.450+0000 7f39ef7fe700 1 -- 172.21.15.16:0/163510173 learned_addr learned my addr 172.21.15.16:0/163510173 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:52.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.451+0000 7f39effff700 1 --2- 172.21.15.16:0/163510173 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f39f0130e50 0x7f39f0133820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:52.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.451+0000 7f39eeffd700 1 --2- 172.21.15.16:0/163510173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39f0130540 0x7f39f01332e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:52.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.451+0000 7f39ef7fe700 1 -- 172.21.15.16:0/163510173 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f39f0130e50 msgr2=0x7f39f0133820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:52.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.451+0000 7f39ef7fe700 1 --2- 172.21.15.16:0/163510173 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f39f0130e50 0x7f39f0133820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.451+0000 7f39ef7fe700 1 -- 172.21.15.16:0/163510173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39f0130540 msgr2=0x7f39f01332e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:52.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.451+0000 7f39ef7fe700 1 --2- 172.21.15.16:0/163510173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39f0130540 0x7f39f01332e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.451+0000 7f39ef7fe700 1 -- 172.21.15.16:0/163510173 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e8009a70 con 0x7f39f006fb20 2023-09-30T22:29:52.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.451+0000 7f39ef7fe700 1 --2- 172.21.15.16:0/163510173 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f39f006fb20 0x7f39f0132da0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f39e000ebe0 tx=0x7f39e000eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:52.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.451+0000 7f39d7fff700 1 -- 172.21.15.16:0/163510173 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e000c850 con 0x7f39f006fb20 2023-09-30T22:29:52.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.451+0000 7f39d7fff700 1 -- 172.21.15.16:0/163510173 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39e000c9b0 con 0x7f39f006fb20 2023-09-30T22:29:52.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.451+0000 7f39d7fff700 1 -- 172.21.15.16:0/163510173 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39e000cd20 con 0x7f39f006fb20 2023-09-30T22:29:52.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.452+0000 7f39f607d700 1 -- 172.21.15.16:0/163510173 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39e0009550 con 0x7f39f006fb20 2023-09-30T22:29:52.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.452+0000 7f39f607d700 1 -- 172.21.15.16:0/163510173 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39e40094a0 con 0x7f39f006fb20 2023-09-30T22:29:52.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.453+0000 7f39d7fff700 1 -- 172.21.15.16:0/163510173 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f39e0013470 con 0x7f39f006fb20 2023-09-30T22:29:52.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.453+0000 7f39d7fff700 1 -- 172.21.15.16:0/163510173 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7f39e003ffc0 con 0x7f39f006fb20 2023-09-30T22:29:52.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.453+0000 7f39eeffd700 1 -- 172.21.15.16:0/163510173 >> v1:172.21.15.16:6816/126811 conn(0x7f39dc033920 legacy=0x7f39dc035dd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:52.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.453+0000 7f39f607d700 1 -- 172.21.15.16:0/163510173 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39f0074ea0 con 0x7f39f006fb20 2023-09-30T22:29:52.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.462+0000 7f39d7fff700 1 -- 172.21.15.16:0/163510173 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f39e0017020 con 0x7f39f006fb20 2023-09-30T22:29:52.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.499+0000 7efc372a8700 1 Processor -- start 2023-09-30T22:29:52.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.499+0000 7efc372a8700 1 -- start start 2023-09-30T22:29:52.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.500+0000 7efc372a8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc3009d6f0 0x7efc3009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:52.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.500+0000 7efc372a8700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efc3009e000 0x7efc301339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:52.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.500+0000 7efc372a8700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc3009b120 con 0x7efc3009e410 2023-09-30T22:29:52.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.500+0000 7efc372a8700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc3006b460 con 0x7efc3009e000 2023-09-30T22:29:52.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.500+0000 7efc372a8700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efc3009a7e0 con 0x7efc3009d6f0 2023-09-30T22:29:52.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.500+0000 7efc35044700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc3009d6f0 0x7efc3009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:52.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.500+0000 7efc34843700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efc3009e000 0x7efc301339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:52.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.501+0000 7efc34843700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efc3009e000 0x7efc301339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48102/0 (socket says 172.21.15.16:48102) 2023-09-30T22:29:52.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.501+0000 7efc34843700 1 -- 172.21.15.16:0/2292321683 learned_addr learned my addr 172.21.15.16:0/2292321683 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:52.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.501+0000 7efc35044700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc3009d6f0 0x7efc3009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:52100/0 (socket says 172.21.15.16:52100) 2023-09-30T22:29:52.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.501+0000 7efc35044700 1 -- 172.21.15.16:0/2292321683 >> v2:172.21.15.16:3300/0 conn(0x7efc3009e000 msgr2=0x7efc301339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:52.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.501+0000 7efc35044700 1 --2- 172.21.15.16:0/2292321683 >> v2:172.21.15.16:3300/0 conn(0x7efc3009e000 0x7efc301339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.501+0000 7efc35044700 1 -- 172.21.15.16:0/2292321683 >> v1:172.21.15.16:6789/0 conn(0x7efc3009e410 legacy=0x7efc30133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:52.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.501+0000 7efc35044700 1 -- 172.21.15.16:0/2292321683 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc3009a500 con 0x7efc3009d6f0 2023-09-30T22:29:52.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.501+0000 7efc35044700 1 --2- 172.21.15.16:0/2292321683 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc3009d6f0 0x7efc3009dac0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7efc2c009870 tx=0x7efc2c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62cae73962faaf0e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:52.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.501+0000 7efc27fff700 1 -- 172.21.15.16:0/2292321683 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc2c011830 con 0x7efc3009d6f0 2023-09-30T22:29:52.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.501+0000 7efc27fff700 1 -- 172.21.15.16:0/2292321683 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc2c011990 con 0x7efc3009d6f0 2023-09-30T22:29:52.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.501+0000 7efc27fff700 1 -- 172.21.15.16:0/2292321683 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc2c011d20 con 0x7efc3009d6f0 2023-09-30T22:29:52.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.502+0000 7efc372a8700 1 -- 172.21.15.16:0/2292321683 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc3009d6f0 msgr2=0x7efc3009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:52.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.502+0000 7efc372a8700 1 --2- 172.21.15.16:0/2292321683 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc3009d6f0 0x7efc3009dac0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7efc2c009870 tx=0x7efc2c009b80 comp rx=0 tx=0).stop 2023-09-30T22:29:52.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.502+0000 7efc372a8700 1 -- 172.21.15.16:0/2292321683 shutdown_connections 2023-09-30T22:29:52.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.503+0000 7efc372a8700 1 --2- 172.21.15.16:0/2292321683 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc3009d6f0 0x7efc3009dac0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.503+0000 7efc372a8700 1 --2- 172.21.15.16:0/2292321683 >> v2:172.21.15.16:3300/0 conn(0x7efc3009e000 0x7efc301339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.503+0000 7efc372a8700 1 -- 172.21.15.16:0/2292321683 >> 172.21.15.16:0/2292321683 conn(0x7efc30066f50 msgr2=0x7efc3009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:52.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.503+0000 7ff4c0392700 1 Processor -- start 2023-09-30T22:29:52.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.503+0000 7ff4c0392700 1 -- start start 2023-09-30T22:29:52.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.503+0000 7efc372a8700 1 -- 172.21.15.16:0/2292321683 shutdown_connections 2023-09-30T22:29:52.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.503+0000 7ff4c0392700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff4b809e1d0 0x7ff4b8138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:52.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.503+0000 7efc372a8700 1 -- 172.21.15.16:0/2292321683 wait complete. 2023-09-30T22:29:52.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.503+0000 7ff4c0392700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff4b8139170 0x7ff4b813b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:52.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.503+0000 7ff4c0392700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4b809b120 con 0x7ff4b809d6f0 2023-09-30T22:29:52.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.503+0000 7ff4c0392700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b806b460 con 0x7ff4b8139170 2023-09-30T22:29:52.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.503+0000 7ff4c0392700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b809a7e0 con 0x7ff4b809e1d0 2023-09-30T22:29:52.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.504+0000 7ff4be12e700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff4b809d6f0 0x7ff4b809dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59190/0 (socket says 172.21.15.16:59190) 2023-09-30T22:29:52.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.504+0000 7ff4be12e700 1 -- 172.21.15.16:0/3906626029 learned_addr learned my addr 172.21.15.16:0/3906626029 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:29:52.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.504+0000 7ff4bd92d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff4b809e1d0 0x7ff4b8138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:52.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.504+0000 7ff4bd12c700 1 -- 172.21.15.16:0/3906626029 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 104154939 0 0) 0x7ff4b809b120 con 0x7ff4b809d6f0 2023-09-30T22:29:52.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.504+0000 7ff4bd12c700 1 -- 172.21.15.16:0/3906626029 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4a00035b0 con 0x7ff4b809d6f0 2023-09-30T22:29:52.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.504+0000 7ff4bd92d700 1 -- 172.21.15.16:0/3906626029 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff4b8139170 msgr2=0x7ff4b813b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:52.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.504+0000 7ff4bd92d700 1 --2- 172.21.15.16:0/3906626029 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff4b8139170 0x7ff4b813b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.504+0000 7ff4bd92d700 1 -- 172.21.15.16:0/3906626029 >> v1:172.21.15.16:6789/0 conn(0x7ff4b809d6f0 legacy=0x7ff4b809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:52.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.504+0000 7ff4bd92d700 1 -- 172.21.15.16:0/3906626029 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b809a500 con 0x7ff4b809e1d0 2023-09-30T22:29:52.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.505+0000 7efc372a8700 1 Processor -- start 2023-09-30T22:29:52.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.505+0000 7efc372a8700 1 -- start start 2023-09-30T22:29:52.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.505+0000 7ff4bd92d700 1 --2- 172.21.15.16:0/3906626029 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff4b809e1d0 0x7ff4b8138c30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8009df0 tx=0x7ff4a800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0127ca1db22b538 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:52.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.505+0000 7ff4bd12c700 1 -- 172.21.15.16:0/3906626029 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a800e070 con 0x7ff4b809e1d0 2023-09-30T22:29:52.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.505+0000 7ff4bd12c700 1 -- 172.21.15.16:0/3906626029 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4a800bae0 con 0x7ff4b809e1d0 2023-09-30T22:29:52.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.505+0000 7ff4bd12c700 1 -- 172.21.15.16:0/3906626029 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a800be50 con 0x7ff4b809e1d0 2023-09-30T22:29:52.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.505+0000 7efc372a8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efc3009d6f0 0x7efc301a60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:52.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.505+0000 7efc372a8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc3009e000 0x7efc301a6610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:52.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.505+0000 7efc372a8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc3009e410 0x7efc301a6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:52.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.505+0000 7efc372a8700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc301a74f0 con 0x7efc3009e000 2023-09-30T22:29:52.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.505+0000 7efc372a8700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc301a7b90 con 0x7efc3009d6f0 2023-09-30T22:29:52.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.505+0000 7efc372a8700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efc3009c710 con 0x7efc3009e410 2023-09-30T22:29:52.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.506+0000 7efc35845700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc3009e410 0x7efc301a6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:52.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.506+0000 7efc35845700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc3009e410 0x7efc301a6b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:52122/0 (socket says 172.21.15.16:52122) 2023-09-30T22:29:52.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.506+0000 7efc35845700 1 -- 172.21.15.16:0/600009760 learned_addr learned my addr 172.21.15.16:0/600009760 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:52.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.506+0000 7efc34843700 1 --2- 172.21.15.16:0/600009760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc3009e000 0x7efc301a6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:52.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.506+0000 7efc35845700 1 -- 172.21.15.16:0/600009760 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efc3009d6f0 msgr2=0x7efc301a60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:52.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.506+0000 7efc35845700 1 --2- 172.21.15.16:0/600009760 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efc3009d6f0 0x7efc301a60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.506+0000 7efc35845700 1 -- 172.21.15.16:0/600009760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc3009e000 msgr2=0x7efc301a6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:52.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.506+0000 7efc35845700 1 --2- 172.21.15.16:0/600009760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc3009e000 0x7efc301a6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.506+0000 7efc35845700 1 -- 172.21.15.16:0/600009760 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc2c0094a0 con 0x7efc3009e410 2023-09-30T22:29:52.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.506+0000 7ff4c0392700 1 -- 172.21.15.16:0/3906626029 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff4b809e1d0 msgr2=0x7ff4b8138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:52.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.506+0000 7ff4c0392700 1 --2- 172.21.15.16:0/3906626029 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff4b809e1d0 0x7ff4b8138c30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff4a8009df0 tx=0x7ff4a800b330 comp rx=0 tx=0).stop 2023-09-30T22:29:52.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.506+0000 7efc35845700 1 --2- 172.21.15.16:0/600009760 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc3009e410 0x7efc301a6b50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efc2800cc50 tx=0x7efc2800cf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:52.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.506+0000 7ff4c0392700 1 -- 172.21.15.16:0/3906626029 shutdown_connections 2023-09-30T22:29:52.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.506+0000 7ff4c0392700 1 --2- 172.21.15.16:0/3906626029 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff4b8139170 0x7ff4b813b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.506+0000 7ff4c0392700 1 --2- 172.21.15.16:0/3906626029 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff4b809e1d0 0x7ff4b8138c30 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.506+0000 7ff4c0392700 1 -- 172.21.15.16:0/3906626029 >> 172.21.15.16:0/3906626029 conn(0x7ff4b8066f50 msgr2=0x7ff4b809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:52.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.506+0000 7ff4c0392700 1 -- 172.21.15.16:0/3906626029 shutdown_connections 2023-09-30T22:29:52.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.506+0000 7ff4c0392700 1 -- 172.21.15.16:0/3906626029 wait complete. 2023-09-30T22:29:52.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.506+0000 7efc25ffb700 1 -- 172.21.15.16:0/600009760 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc28003c00 con 0x7efc3009e410 2023-09-30T22:29:52.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.507+0000 7efc25ffb700 1 -- 172.21.15.16:0/600009760 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc28003d60 con 0x7efc3009e410 2023-09-30T22:29:52.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.507+0000 7efc25ffb700 1 -- 172.21.15.16:0/600009760 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc2800ab60 con 0x7efc3009e410 2023-09-30T22:29:52.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.507+0000 7efc372a8700 1 -- 172.21.15.16:0/600009760 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc301a97c0 con 0x7efc3009e410 2023-09-30T22:29:52.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.508+0000 7efc372a8700 1 -- 172.21.15.16:0/600009760 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc301a8de0 con 0x7efc3009e410 2023-09-30T22:29:52.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.508+0000 7efc25ffb700 1 -- 172.21.15.16:0/600009760 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7efc2800acc0 con 0x7efc3009e410 2023-09-30T22:29:52.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.508+0000 7efc35044700 1 -- 172.21.15.16:0/600009760 >> v1:172.21.15.16:6816/126811 conn(0x7efc1402f510 legacy=0x7efc140319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:52.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.508+0000 7ff4c0392700 1 Processor -- start 2023-09-30T22:29:52.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.508+0000 7ff4c0392700 1 -- start start 2023-09-30T22:29:52.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.509+0000 7ff4c0392700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff4b809d6f0 0x7ff4b81c8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:52.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.509+0000 7ff4c0392700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff4b8139170 0x7ff4b81c8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:52.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.509+0000 7ff4c0392700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4b811e8e0 0x7ff4b811ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:52.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.509+0000 7ff4c0392700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b809c710 con 0x7ff4b811e8e0 2023-09-30T22:29:52.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.509+0000 7ff4c0392700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b8122140 con 0x7ff4b8139170 2023-09-30T22:29:52.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.509+0000 7ff4c0392700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b81be360 con 0x7ff4b809d6f0 2023-09-30T22:29:52.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.509+0000 7efc25ffb700 1 -- 172.21.15.16:0/600009760 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7efc28005090 con 0x7efc3009e410 2023-09-30T22:29:52.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.509+0000 7efc372a8700 1 -- 172.21.15.16:0/600009760 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc30074ea0 con 0x7efc3009e410 2023-09-30T22:29:52.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.509+0000 7ff4be12e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff4b809d6f0 0x7ff4b81c8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:52.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.509+0000 7ff4be12e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff4b809d6f0 0x7ff4b81c8650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:52124/0 (socket says 172.21.15.16:52124) 2023-09-30T22:29:52.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.509+0000 7ff4be12e700 1 -- 172.21.15.16:0/2777137983 learned_addr learned my addr 172.21.15.16:0/2777137983 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:52.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.509+0000 7ff4be92f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4b811e8e0 0x7ff4b811ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:52.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.509+0000 7ff4be92f700 1 -- 172.21.15.16:0/2777137983 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff4b809d6f0 msgr2=0x7ff4b81c8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:52.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.509+0000 7ff4be92f700 1 --2- 172.21.15.16:0/2777137983 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff4b809d6f0 0x7ff4b81c8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.509+0000 7ff4be92f700 1 -- 172.21.15.16:0/2777137983 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff4b8139170 msgr2=0x7ff4b81c8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:52.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.509+0000 7ff4be92f700 1 --2- 172.21.15.16:0/2777137983 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff4b8139170 0x7ff4b81c8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.509+0000 7ff4be92f700 1 -- 172.21.15.16:0/2777137983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a8009a70 con 0x7ff4b811e8e0 2023-09-30T22:29:52.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.510+0000 7ff4be92f700 1 --2- 172.21.15.16:0/2777137983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4b811e8e0 0x7ff4b811ed60 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac00b3f0 tx=0x7ff4ac00b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:52.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.510+0000 7ff4b6ffd700 1 -- 172.21.15.16:0/2777137983 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac011830 con 0x7ff4b811e8e0 2023-09-30T22:29:52.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.510+0000 7ff4b6ffd700 1 -- 172.21.15.16:0/2777137983 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4ac011990 con 0x7ff4b811e8e0 2023-09-30T22:29:52.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.510+0000 7ff4b6ffd700 1 -- 172.21.15.16:0/2777137983 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac011d40 con 0x7ff4b811e8e0 2023-09-30T22:29:52.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.510+0000 7ff4c0392700 1 -- 172.21.15.16:0/2777137983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4b8134760 con 0x7ff4b811e8e0 2023-09-30T22:29:52.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.510+0000 7ff4c0392700 1 -- 172.21.15.16:0/2777137983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4b8122b40 con 0x7ff4b811e8e0 2023-09-30T22:29:52.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.512+0000 7ff4c0392700 1 -- 172.21.15.16:0/2777137983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4b8074ea0 con 0x7ff4b811e8e0 2023-09-30T22:29:52.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.512+0000 7ff4b6ffd700 1 -- 172.21.15.16:0/2777137983 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7ff4ac01a400 con 0x7ff4b811e8e0 2023-09-30T22:29:52.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.513+0000 7ff4be12e700 1 -- 172.21.15.16:0/2777137983 >> v1:172.21.15.16:6816/126811 conn(0x7ff4a402f510 legacy=0x7ff4a40319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:52.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.515+0000 7efc25ffb700 1 -- 172.21.15.16:0/600009760 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7efc280198f0 con 0x7efc3009e410 2023-09-30T22:29:52.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.514+0000 7ff4b6ffd700 1 -- 172.21.15.16:0/2777137983 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7ff4ac01a6b0 con 0x7ff4b811e8e0 2023-09-30T22:29:52.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.522+0000 7ff4b6ffd700 1 -- 172.21.15.16:0/2777137983 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff4ac00f8d0 con 0x7ff4b811e8e0 2023-09-30T22:29:52.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.523+0000 7f344c7b5700 1 Processor -- start 2023-09-30T22:29:52.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.523+0000 7f344c7b5700 1 -- start start 2023-09-30T22:29:52.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.524+0000 7f344c7b5700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f344409d6f0 0x7f344409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:52.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.524+0000 7f344c7b5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f344409e000 0x7f3444130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:52.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.524+0000 7f344c7b5700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f344409f130 con 0x7f344409e410 2023-09-30T22:29:52.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.524+0000 7f344c7b5700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f344409f520 con 0x7f344409d6f0 2023-09-30T22:29:52.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.524+0000 7f344c7b5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f344409e930 con 0x7f344409e000 2023-09-30T22:29:52.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.524+0000 7f344ad52700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f344409e410 0x7f3444130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59198/0 (socket says 172.21.15.16:59198) 2023-09-30T22:29:52.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.524+0000 7f344ad52700 1 -- 172.21.15.16:0/3417208259 learned_addr learned my addr 172.21.15.16:0/3417208259 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:29:52.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.524+0000 7f3449d50700 1 --2- 172.21.15.16:0/3417208259 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f344409e000 0x7f3444130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:52.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.524+0000 7f344a551700 1 --2- 172.21.15.16:0/3417208259 >> v2:172.21.15.16:3300/0 conn(0x7f344409d6f0 0x7f344409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:52.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.524+0000 7f3449d50700 1 -- 172.21.15.16:0/3417208259 >> v2:172.21.15.16:3300/0 conn(0x7f344409d6f0 msgr2=0x7f344409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:52.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.524+0000 7f3449d50700 1 --2- 172.21.15.16:0/3417208259 >> v2:172.21.15.16:3300/0 conn(0x7f344409d6f0 0x7f344409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.524+0000 7f3449d50700 1 -- 172.21.15.16:0/3417208259 >> v1:172.21.15.16:6789/0 conn(0x7f344409e410 legacy=0x7f3444130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:52.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.524+0000 7f3449d50700 1 -- 172.21.15.16:0/3417208259 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f344409b2d0 con 0x7f344409e000 2023-09-30T22:29:52.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.525+0000 7f3449d50700 1 --2- 172.21.15.16:0/3417208259 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f344409e000 0x7f3444130660 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3434009840 tx=0x7f3434009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2a4013110083630 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:52.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.525+0000 7f344954f700 1 -- 172.21.15.16:0/3417208259 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3434004230 con 0x7f344409e000 2023-09-30T22:29:52.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.525+0000 7f344954f700 1 -- 172.21.15.16:0/3417208259 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3434004390 con 0x7f344409e000 2023-09-30T22:29:52.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.525+0000 7f344954f700 1 -- 172.21.15.16:0/3417208259 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34340036a0 con 0x7f344409e000 2023-09-30T22:29:52.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.526+0000 7f344c7b5700 1 -- 172.21.15.16:0/3417208259 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f344409e000 msgr2=0x7f3444130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:52.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.526+0000 7f344c7b5700 1 --2- 172.21.15.16:0/3417208259 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f344409e000 0x7f3444130660 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3434009840 tx=0x7f3434009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:52.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.526+0000 7f344c7b5700 1 -- 172.21.15.16:0/3417208259 shutdown_connections 2023-09-30T22:29:52.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.526+0000 7f344c7b5700 1 --2- 172.21.15.16:0/3417208259 >> v2:172.21.15.16:3300/0 conn(0x7f344409d6f0 0x7f344409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.526+0000 7f344c7b5700 1 --2- 172.21.15.16:0/3417208259 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f344409e000 0x7f3444130660 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.526+0000 7f344c7b5700 1 -- 172.21.15.16:0/3417208259 >> 172.21.15.16:0/3417208259 conn(0x7f3444066f50 msgr2=0x7f344409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:52.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.526+0000 7f344c7b5700 1 -- 172.21.15.16:0/3417208259 shutdown_connections 2023-09-30T22:29:52.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.526+0000 7f344c7b5700 1 -- 172.21.15.16:0/3417208259 wait complete. 2023-09-30T22:29:52.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.527+0000 7f344c7b5700 1 Processor -- start 2023-09-30T22:29:52.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.527+0000 7f344c7b5700 1 -- start start 2023-09-30T22:29:52.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.527+0000 7f344c7b5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f344409d6f0 0x7f34441224b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:52.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.527+0000 7f344c7b5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f344409e000 0x7f34441229f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:52.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.527+0000 7f344c7b5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f344409e410 0x7f3444122f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:52.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.527+0000 7f344c7b5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3444124140 con 0x7f344409e410 2023-09-30T22:29:52.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.527+0000 7f344c7b5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3444125c60 con 0x7f344409e000 2023-09-30T22:29:52.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.527+0000 7f344c7b5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f34440a1d90 con 0x7f344409d6f0 2023-09-30T22:29:52.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.528+0000 7f344ad52700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f344409e410 0x7f3444122f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:52.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.528+0000 7f344ad52700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f344409e410 0x7f3444122f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48134/0 (socket says 172.21.15.16:48134) 2023-09-30T22:29:52.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.528+0000 7f344ad52700 1 -- 172.21.15.16:0/3134334851 learned_addr learned my addr 172.21.15.16:0/3134334851 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:52.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.528+0000 7f344a551700 1 --2- 172.21.15.16:0/3134334851 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f344409d6f0 0x7f34441224b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:52.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.528+0000 7f344ad52700 1 -- 172.21.15.16:0/3134334851 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f344409d6f0 msgr2=0x7f34441224b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:52.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.528+0000 7f344ad52700 1 --2- 172.21.15.16:0/3134334851 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f344409d6f0 0x7f34441224b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.528+0000 7f344ad52700 1 -- 172.21.15.16:0/3134334851 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f344409e000 msgr2=0x7f34441229f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:52.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.528+0000 7f344ad52700 1 --2- 172.21.15.16:0/3134334851 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f344409e000 0x7f34441229f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.528+0000 7f344ad52700 1 -- 172.21.15.16:0/3134334851 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34340094a0 con 0x7f344409e410 2023-09-30T22:29:52.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.528+0000 7f344ad52700 1 --2- 172.21.15.16:0/3134334851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f344409e410 0x7f3444122f30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f343c002000 tx=0x7f343c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:52.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.528+0000 7f343affd700 1 -- 172.21.15.16:0/3134334851 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343c00ad40 con 0x7f344409e410 2023-09-30T22:29:52.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.528+0000 7f343affd700 1 -- 172.21.15.16:0/3134334851 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f343c0070d0 con 0x7f344409e410 2023-09-30T22:29:52.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.528+0000 7f344c7b5700 1 -- 172.21.15.16:0/3134334851 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3444124820 con 0x7f344409e410 2023-09-30T22:29:52.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.528+0000 7f343affd700 1 -- 172.21.15.16:0/3134334851 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343c00aea0 con 0x7f344409e410 2023-09-30T22:29:52.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.529+0000 7f344c7b5700 1 -- 172.21.15.16:0/3134334851 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3444126640 con 0x7f344409e410 2023-09-30T22:29:52.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.529+0000 7f344c7b5700 1 -- 172.21.15.16:0/3134334851 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3444074ea0 con 0x7f344409e410 2023-09-30T22:29:52.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.530+0000 7f343affd700 1 -- 172.21.15.16:0/3134334851 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f343c007480 con 0x7f344409e410 2023-09-30T22:29:52.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.530+0000 7f343affd700 1 -- 172.21.15.16:0/3134334851 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7f343c03ffc0 con 0x7f344409e410 2023-09-30T22:29:52.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.530+0000 7f344a551700 1 -- 172.21.15.16:0/3134334851 >> v1:172.21.15.16:6816/126811 conn(0x7f342802f510 legacy=0x7f34280319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:52.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.534+0000 7f343affd700 1 -- 172.21.15.16:0/3134334851 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f343c01c3b0 con 0x7f344409e410 2023-09-30T22:29:52.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.654+0000 7f39eeffd700 1 -- 172.21.15.16:0/163510173 >> v1:172.21.15.16:6816/126811 conn(0x7f39dc033920 legacy=0x7f39dc035dd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:52.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.699+0000 7f39f607d700 1 -- 172.21.15.16:0/163510173 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f39f0135c70 con 0x7f39f006fb20 2023-09-30T22:29:52.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.700+0000 7f39d7fff700 1 -- 172.21.15.16:0/163510173 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f39e002b090 con 0x7f39f006fb20 2023-09-30T22:29:52.700 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:29:52.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.703+0000 7f39f607d700 1 -- 172.21.15.16:0/163510173 >> v1:172.21.15.16:6816/126811 conn(0x7f39dc033920 legacy=0x7f39dc035dd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:52.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.703+0000 7f39f607d700 1 -- 172.21.15.16:0/163510173 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f39f006fb20 msgr2=0x7f39f0132da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:52.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.703+0000 7f39f607d700 1 --2- 172.21.15.16:0/163510173 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f39f006fb20 0x7f39f0132da0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f39e000ebe0 tx=0x7f39e000eef0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.703+0000 7f39f607d700 1 -- 172.21.15.16:0/163510173 shutdown_connections 2023-09-30T22:29:52.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.703+0000 7f39f607d700 1 --2- 172.21.15.16:0/163510173 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f39f006fb20 0x7f39f0132da0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.703+0000 7f39f607d700 1 --2- 172.21.15.16:0/163510173 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f39f0130e50 0x7f39f0133820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.703+0000 7f39f607d700 1 --2- 172.21.15.16:0/163510173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39f0130540 0x7f39f01332e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.703+0000 7f39f607d700 1 -- 172.21.15.16:0/163510173 >> 172.21.15.16:0/163510173 conn(0x7f39f00a2aa0 msgr2=0x7f39f012f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:52.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.703+0000 7f39f607d700 1 -- 172.21.15.16:0/163510173 shutdown_connections 2023-09-30T22:29:52.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.703+0000 7f39f607d700 1 -- 172.21.15.16:0/163510173 wait complete. 2023-09-30T22:29:52.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.709+0000 7efc35044700 1 -- 172.21.15.16:0/600009760 >> v1:172.21.15.16:6816/126811 conn(0x7efc1402f510 legacy=0x7efc140319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:52.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.714+0000 7ff4be12e700 1 -- 172.21.15.16:0/2777137983 >> v1:172.21.15.16:6816/126811 conn(0x7ff4a402f510 legacy=0x7ff4a40319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:52.718 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:29:52.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.726+0000 7efc372a8700 1 -- 172.21.15.16:0/600009760 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7efc3006c360 con 0x7efc3009e410 2023-09-30T22:29:52.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.726+0000 7efc25ffb700 1 -- 172.21.15.16:0/600009760 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7efc2802a090 con 0x7efc3009e410 2023-09-30T22:29:52.727 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:29:52.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.730+0000 7efc372a8700 1 -- 172.21.15.16:0/600009760 >> v1:172.21.15.16:6816/126811 conn(0x7efc1402f510 legacy=0x7efc140319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:52.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.730+0000 7efc372a8700 1 -- 172.21.15.16:0/600009760 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc3009e410 msgr2=0x7efc301a6b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:52.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.730+0000 7efc372a8700 1 --2- 172.21.15.16:0/600009760 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc3009e410 0x7efc301a6b50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efc2800cc50 tx=0x7efc2800cf60 comp rx=0 tx=0).stop 2023-09-30T22:29:52.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.730+0000 7efc372a8700 1 -- 172.21.15.16:0/600009760 shutdown_connections 2023-09-30T22:29:52.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.730+0000 7efc372a8700 1 --2- 172.21.15.16:0/600009760 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efc3009d6f0 0x7efc301a60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.730+0000 7efc372a8700 1 --2- 172.21.15.16:0/600009760 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc3009e410 0x7efc301a6b50 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.730+0000 7efc372a8700 1 --2- 172.21.15.16:0/600009760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc3009e000 0x7efc301a6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.730+0000 7efc372a8700 1 -- 172.21.15.16:0/600009760 >> 172.21.15.16:0/600009760 conn(0x7efc30066f50 msgr2=0x7efc301baef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:52.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.731+0000 7efc372a8700 1 -- 172.21.15.16:0/600009760 shutdown_connections 2023-09-30T22:29:52.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.731+0000 7f344a551700 1 -- 172.21.15.16:0/3134334851 >> v1:172.21.15.16:6816/126811 conn(0x7f342802f510 legacy=0x7f34280319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:52.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.733+0000 7efc372a8700 1 -- 172.21.15.16:0/600009760 wait complete. 2023-09-30T22:29:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.749+0000 7f344c7b5700 1 -- 172.21.15.16:0/3134334851 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f344406c360 con 0x7f344409e410 2023-09-30T22:29:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.749+0000 7f343affd700 1 -- 172.21.15.16:0/3134334851 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f343c018020 con 0x7f344409e410 2023-09-30T22:29:52.750 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:29:52.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.751+0000 7f344c7b5700 1 -- 172.21.15.16:0/3134334851 >> v1:172.21.15.16:6816/126811 conn(0x7f342802f510 legacy=0x7f34280319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:52.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.751+0000 7f344c7b5700 1 -- 172.21.15.16:0/3134334851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f344409e410 msgr2=0x7f3444122f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:52.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.751+0000 7f344c7b5700 1 --2- 172.21.15.16:0/3134334851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f344409e410 0x7f3444122f30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f343c002000 tx=0x7f343c00a340 comp rx=0 tx=0).stop 2023-09-30T22:29:52.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.751+0000 7f344c7b5700 1 -- 172.21.15.16:0/3134334851 shutdown_connections 2023-09-30T22:29:52.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.751+0000 7f344c7b5700 1 --2- 172.21.15.16:0/3134334851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f344409e410 0x7f3444122f30 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.751+0000 7f344c7b5700 1 --2- 172.21.15.16:0/3134334851 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f344409d6f0 0x7f34441224b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.751+0000 7f344c7b5700 1 --2- 172.21.15.16:0/3134334851 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f344409e000 0x7f34441229f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.751+0000 7f344c7b5700 1 -- 172.21.15.16:0/3134334851 >> 172.21.15.16:0/3134334851 conn(0x7f3444066f50 msgr2=0x7f344406f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:52.755 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:29:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.752+0000 7f344c7b5700 1 -- 172.21.15.16:0/3134334851 shutdown_connections 2023-09-30T22:29:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.752+0000 7f344c7b5700 1 -- 172.21.15.16:0/3134334851 wait complete. 2023-09-30T22:29:52.766 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:29:52.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.768+0000 7ff4c0392700 1 -- 172.21.15.16:0/2777137983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff4b806c360 con 0x7ff4b811e8e0 2023-09-30T22:29:52.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.769+0000 7ff4b6ffd700 1 -- 172.21.15.16:0/2777137983 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff4ac015070 con 0x7ff4b811e8e0 2023-09-30T22:29:52.769 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:29:52.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.771+0000 7ff4c0392700 1 -- 172.21.15.16:0/2777137983 >> v1:172.21.15.16:6816/126811 conn(0x7ff4a402f510 legacy=0x7ff4a40319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:52.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.771+0000 7ff4c0392700 1 -- 172.21.15.16:0/2777137983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4b811e8e0 msgr2=0x7ff4b811ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:52.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.771+0000 7ff4c0392700 1 --2- 172.21.15.16:0/2777137983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4b811e8e0 0x7ff4b811ed60 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac00b3f0 tx=0x7ff4ac00b700 comp rx=0 tx=0).stop 2023-09-30T22:29:52.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.771+0000 7ff4c0392700 1 -- 172.21.15.16:0/2777137983 shutdown_connections 2023-09-30T22:29:52.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.771+0000 7ff4c0392700 1 --2- 172.21.15.16:0/2777137983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4b811e8e0 0x7ff4b811ed60 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.771+0000 7ff4c0392700 1 --2- 172.21.15.16:0/2777137983 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff4b809d6f0 0x7ff4b81c8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.771+0000 7ff4c0392700 1 --2- 172.21.15.16:0/2777137983 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff4b8139170 0x7ff4b81c8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:52.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.771+0000 7ff4c0392700 1 -- 172.21.15.16:0/2777137983 >> 172.21.15.16:0/2777137983 conn(0x7ff4b8066f50 msgr2=0x7ff4b8136c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:52.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.771+0000 7ff4c0392700 1 -- 172.21.15.16:0/2777137983 shutdown_connections 2023-09-30T22:29:52.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:52.771+0000 7ff4c0392700 1 -- 172.21.15.16:0/2777137983 wait complete. 2023-09-30T22:29:52.785 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:29:53.719 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:29:53.756 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:29:53.766 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:29:53.786 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:29:53.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.887+0000 7f988ae3a700 1 Processor -- start 2023-09-30T22:29:53.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.887+0000 7f988ae3a700 1 -- start start 2023-09-30T22:29:53.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.888+0000 7f988ae3a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f98840a0a30 0x7f98840a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:53.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.888+0000 7f988ae3a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f988413d820 0x7f988413fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:53.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.888+0000 7f988ae3a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f988409d920 con 0x7f98840a1340 2023-09-30T22:29:53.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.888+0000 7f988ae3a700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f988409d020 con 0x7f988413d820 2023-09-30T22:29:53.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.888+0000 7f988ae3a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98840a2010 con 0x7f98840a0a30 2023-09-30T22:29:53.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.888+0000 7f9883fff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f98840a1340 0x7f988413c100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59208/0 (socket says 172.21.15.16:59208) 2023-09-30T22:29:53.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.888+0000 7f9883fff700 1 -- 172.21.15.16:0/3865533032 learned_addr learned my addr 172.21.15.16:0/3865533032 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:29:53.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.888+0000 7f9888bd6700 1 --2- 172.21.15.16:0/3865533032 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f98840a0a30 0x7f98840a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:53.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.888+0000 7f98893d7700 1 --2- 172.21.15.16:0/3865533032 >> v2:172.21.15.16:3300/0 conn(0x7f988413d820 0x7f988413fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:53.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.889+0000 7f98837fe700 1 -- 172.21.15.16:0/3865533032 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 596333962 0 0) 0x7f988409d920 con 0x7f98840a1340 2023-09-30T22:29:53.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.889+0000 7f98837fe700 1 -- 172.21.15.16:0/3865533032 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98740035b0 con 0x7f98840a1340 2023-09-30T22:29:53.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.889+0000 7f98893d7700 1 -- 172.21.15.16:0/3865533032 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f98840a0a30 msgr2=0x7f98840a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:53.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.889+0000 7f98893d7700 1 --2- 172.21.15.16:0/3865533032 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f98840a0a30 0x7f98840a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:53.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.889+0000 7f98893d7700 1 -- 172.21.15.16:0/3865533032 >> v1:172.21.15.16:6789/0 conn(0x7f98840a1340 legacy=0x7f988413c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:53.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.889+0000 7f98893d7700 1 -- 172.21.15.16:0/3865533032 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f988409dbe0 con 0x7f988413d820 2023-09-30T22:29:53.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.889+0000 7f98893d7700 1 --2- 172.21.15.16:0/3865533032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f988413d820 0x7f988413fc00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f987c009d90 tx=0x7f987c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2094884b6cae6b65 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:53.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.889+0000 7f98837fe700 1 -- 172.21.15.16:0/3865533032 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987c00e070 con 0x7f988413d820 2023-09-30T22:29:53.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.889+0000 7f98837fe700 1 -- 172.21.15.16:0/3865533032 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f987c00bae0 con 0x7f988413d820 2023-09-30T22:29:53.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.889+0000 7f98837fe700 1 -- 172.21.15.16:0/3865533032 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987c00be90 con 0x7f988413d820 2023-09-30T22:29:53.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.890+0000 7f988ae3a700 1 -- 172.21.15.16:0/3865533032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f988413d820 msgr2=0x7f988413fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:53.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.890+0000 7f988ae3a700 1 --2- 172.21.15.16:0/3865533032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f988413d820 0x7f988413fc00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f987c009d90 tx=0x7f987c00b330 comp rx=0 tx=0).stop 2023-09-30T22:29:53.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.890+0000 7f988ae3a700 1 -- 172.21.15.16:0/3865533032 shutdown_connections 2023-09-30T22:29:53.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.890+0000 7f988ae3a700 1 --2- 172.21.15.16:0/3865533032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f988413d820 0x7f988413fc00 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:53.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.890+0000 7f988ae3a700 1 --2- 172.21.15.16:0/3865533032 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f98840a0a30 0x7f98840a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:53.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.890+0000 7f988ae3a700 1 -- 172.21.15.16:0/3865533032 >> 172.21.15.16:0/3865533032 conn(0x7f98840a6550 msgr2=0x7f98840a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:53.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.890+0000 7f988ae3a700 1 -- 172.21.15.16:0/3865533032 shutdown_connections 2023-09-30T22:29:53.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.890+0000 7f988ae3a700 1 -- 172.21.15.16:0/3865533032 wait complete. 2023-09-30T22:29:53.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.892+0000 7f988ae3a700 1 Processor -- start 2023-09-30T22:29:53.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.892+0000 7f988ae3a700 1 -- start start 2023-09-30T22:29:53.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.893+0000 7f988ae3a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98840a0a30 0x7f98841274d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:53.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.893+0000 7f988ae3a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f98840a1340 0x7f9884127a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:53.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.893+0000 7f988ae3a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f988413d820 0x7f9884127f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:53.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.893+0000 7f988ae3a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f988413d330 con 0x7f988413d820 2023-09-30T22:29:53.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.893+0000 7f988ae3a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f988412ad60 con 0x7f98840a1340 2023-09-30T22:29:53.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.893+0000 7f988ae3a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9884129050 con 0x7f98840a0a30 2023-09-30T22:29:53.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.893+0000 7f9883fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f98840a1340 0x7f9884127a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:53.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.893+0000 7f9883fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f98840a1340 0x7f9884127a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:49568/0 (socket says 172.21.15.16:49568) 2023-09-30T22:29:53.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.893+0000 7f9883fff700 1 -- 172.21.15.16:0/166392186 learned_addr learned my addr 172.21.15.16:0/166392186 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:53.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.893+0000 7f98893d7700 1 --2- 172.21.15.16:0/166392186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f988413d820 0x7f9884127f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:53.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.894+0000 7f9888bd6700 1 --2- 172.21.15.16:0/166392186 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98840a0a30 0x7f98841274d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:53.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.894+0000 7f9883fff700 1 -- 172.21.15.16:0/166392186 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98840a0a30 msgr2=0x7f98841274d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:53.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.894+0000 7f9883fff700 1 --2- 172.21.15.16:0/166392186 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98840a0a30 0x7f98841274d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:53.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.894+0000 7f9883fff700 1 -- 172.21.15.16:0/166392186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f988413d820 msgr2=0x7f9884127f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:53.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.894+0000 7f9883fff700 1 --2- 172.21.15.16:0/166392186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f988413d820 0x7f9884127f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:53.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.894+0000 7f9883fff700 1 -- 172.21.15.16:0/166392186 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f987c009a70 con 0x7f98840a1340 2023-09-30T22:29:53.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.894+0000 7f9888bd6700 1 --2- 172.21.15.16:0/166392186 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98840a0a30 0x7f98841274d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:29:53.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.894+0000 7f9883fff700 1 --2- 172.21.15.16:0/166392186 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f98840a1340 0x7f9884127a10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f987000c540 tx=0x7f987000c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:53.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.894+0000 7f98817fa700 1 -- 172.21.15.16:0/166392186 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9870004c80 con 0x7f98840a1340 2023-09-30T22:29:53.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.894+0000 7f98817fa700 1 -- 172.21.15.16:0/166392186 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9870004de0 con 0x7f98840a1340 2023-09-30T22:29:53.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.894+0000 7f98817fa700 1 -- 172.21.15.16:0/166392186 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9870007310 con 0x7f98840a1340 2023-09-30T22:29:53.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.894+0000 7f988ae3a700 1 -- 172.21.15.16:0/166392186 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f988412af60 con 0x7f98840a1340 2023-09-30T22:29:53.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.894+0000 7f988ae3a700 1 -- 172.21.15.16:0/166392186 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9884129840 con 0x7f98840a1340 2023-09-30T22:29:53.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.895+0000 7f98817fa700 1 -- 172.21.15.16:0/166392186 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f9870007470 con 0x7f98840a1340 2023-09-30T22:29:53.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.896+0000 7f9888bd6700 1 -- 172.21.15.16:0/166392186 >> v1:172.21.15.16:6816/126811 conn(0x7f986802f510 legacy=0x7f98680319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:53.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.896+0000 7f988ae3a700 1 -- 172.21.15.16:0/166392186 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9884071e20 con 0x7f98840a1340 2023-09-30T22:29:53.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.896+0000 7f98817fa700 1 -- 172.21.15.16:0/166392186 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7f9870013da0 con 0x7f98840a1340 2023-09-30T22:29:53.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.905+0000 7f98817fa700 1 -- 172.21.15.16:0/166392186 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f98700123b0 con 0x7f98840a1340 2023-09-30T22:29:53.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.914+0000 7f684f62d700 1 Processor -- start 2023-09-30T22:29:53.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.914+0000 7f684f62d700 1 -- start start 2023-09-30T22:29:53.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.915+0000 7f684f62d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f684809d6f0 0x7f684809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:53.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.915+0000 7f684f62d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f684809e000 0x7f6848130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:53.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.915+0000 7f684f62d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f684809e410 0x7f6848130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:53.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.915+0000 7f684f62d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f684809f520 con 0x7f684809d6f0 2023-09-30T22:29:53.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.915+0000 7f684f62d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f684809f130 con 0x7f684809e410 2023-09-30T22:29:53.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.915+0000 7f684f62d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f684809e930 con 0x7f684809e000 2023-09-30T22:29:53.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.915+0000 7f684d3c9700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f684809d6f0 0x7f684809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:53.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.915+0000 7f684d3c9700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f684809d6f0 0x7f684809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48164/0 (socket says 172.21.15.16:48164) 2023-09-30T22:29:53.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.915+0000 7f684d3c9700 1 -- 172.21.15.16:0/86607245 learned_addr learned my addr 172.21.15.16:0/86607245 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:53.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.915+0000 7f684dbca700 1 --2- 172.21.15.16:0/86607245 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f684809e410 0x7f6848130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:53.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.915+0000 7f684cbc8700 1 --2- 172.21.15.16:0/86607245 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f684809e000 0x7f6848130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:53.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.916+0000 7f684dbca700 1 -- 172.21.15.16:0/86607245 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f684809e000 msgr2=0x7f6848130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:53.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.916+0000 7f684dbca700 1 --2- 172.21.15.16:0/86607245 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f684809e000 0x7f6848130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:53.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.916+0000 7f684dbca700 1 -- 172.21.15.16:0/86607245 >> v2:172.21.15.16:3300/0 conn(0x7f684809d6f0 msgr2=0x7f684809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:53.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.916+0000 7f684dbca700 1 --2- 172.21.15.16:0/86607245 >> v2:172.21.15.16:3300/0 conn(0x7f684809d6f0 0x7f684809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:53.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.916+0000 7f684dbca700 1 -- 172.21.15.16:0/86607245 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f684809b2d0 con 0x7f684809e410 2023-09-30T22:29:53.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.916+0000 7f684d3c9700 1 --2- 172.21.15.16:0/86607245 >> v2:172.21.15.16:3300/0 conn(0x7f684809d6f0 0x7f684809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:53.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.916+0000 7f684dbca700 1 --2- 172.21.15.16:0/86607245 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f684809e410 0x7f6848130ba0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6840009d90 tx=0x7f684000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4628808e817f6341 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:53.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.916+0000 7f683ffff700 1 -- 172.21.15.16:0/86607245 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684000bd20 con 0x7f684809e410 2023-09-30T22:29:53.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.916+0000 7f683ffff700 1 -- 172.21.15.16:0/86607245 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f684000be80 con 0x7f684809e410 2023-09-30T22:29:53.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.916+0000 7f683ffff700 1 -- 172.21.15.16:0/86607245 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6840010710 con 0x7f684809e410 2023-09-30T22:29:53.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.917+0000 7f684f62d700 1 -- 172.21.15.16:0/86607245 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f684809e410 msgr2=0x7f6848130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:53.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.917+0000 7f684f62d700 1 --2- 172.21.15.16:0/86607245 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f684809e410 0x7f6848130ba0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6840009d90 tx=0x7f684000b330 comp rx=0 tx=0).stop 2023-09-30T22:29:53.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.917+0000 7f684f62d700 1 -- 172.21.15.16:0/86607245 shutdown_connections 2023-09-30T22:29:53.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.917+0000 7f684f62d700 1 --2- 172.21.15.16:0/86607245 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f684809e410 0x7f6848130ba0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:53.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.917+0000 7f684f62d700 1 --2- 172.21.15.16:0/86607245 >> v2:172.21.15.16:3300/0 conn(0x7f684809d6f0 0x7f684809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:53.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.917+0000 7f684f62d700 1 --2- 172.21.15.16:0/86607245 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f684809e000 0x7f6848130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:53.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.917+0000 7f684f62d700 1 -- 172.21.15.16:0/86607245 >> 172.21.15.16:0/86607245 conn(0x7f6848066f50 msgr2=0x7f684809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:53.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.917+0000 7f684f62d700 1 -- 172.21.15.16:0/86607245 shutdown_connections 2023-09-30T22:29:53.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.917+0000 7f684f62d700 1 -- 172.21.15.16:0/86607245 wait complete. 2023-09-30T22:29:53.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.918+0000 7f684f62d700 1 Processor -- start 2023-09-30T22:29:53.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.918+0000 7f684f62d700 1 -- start start 2023-09-30T22:29:53.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.919+0000 7f684f62d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f684809d6f0 0x7f684811e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:53.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.919+0000 7f684f62d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f684809e000 0x7f684811e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:53.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.919+0000 7f684f62d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f684809e410 0x7f684811ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:53.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.919+0000 7f684f62d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6848130340 con 0x7f684809e000 2023-09-30T22:29:53.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.919+0000 7f684f62d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6848121e70 con 0x7f684809d6f0 2023-09-30T22:29:53.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.919+0000 7f684f62d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f68480a1990 con 0x7f684809e410 2023-09-30T22:29:53.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.919+0000 7f684dbca700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f684809e410 0x7f684811ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:53.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.919+0000 7f684dbca700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f684809e410 0x7f684811ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:52172/0 (socket says 172.21.15.16:52172) 2023-09-30T22:29:53.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.919+0000 7f684dbca700 1 -- 172.21.15.16:0/3616737573 learned_addr learned my addr 172.21.15.16:0/3616737573 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:53.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.919+0000 7f684cbc8700 1 --2- 172.21.15.16:0/3616737573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f684809e000 0x7f684811e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:53.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.919+0000 7f684d3c9700 1 --2- 172.21.15.16:0/3616737573 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f684809d6f0 0x7f684811e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:53.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.919+0000 7f684dbca700 1 -- 172.21.15.16:0/3616737573 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f684809d6f0 msgr2=0x7f684811e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:53.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.919+0000 7f684dbca700 1 --2- 172.21.15.16:0/3616737573 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f684809d6f0 0x7f684811e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:53.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.919+0000 7f684dbca700 1 -- 172.21.15.16:0/3616737573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f684809e000 msgr2=0x7f684811e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:53.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.919+0000 7f684dbca700 1 --2- 172.21.15.16:0/3616737573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f684809e000 0x7f684811e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:53.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.919+0000 7f684dbca700 1 -- 172.21.15.16:0/3616737573 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f684400b7f0 con 0x7f684809e410 2023-09-30T22:29:53.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.919+0000 7f684cbc8700 1 --2- 172.21.15.16:0/3616737573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f684809e000 0x7f684811e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:53.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.920+0000 7f684dbca700 1 --2- 172.21.15.16:0/3616737573 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f684809e410 0x7f684811ef00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f684000b680 tx=0x7f6840004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:53.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.920+0000 7f683dffb700 1 -- 172.21.15.16:0/3616737573 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6840004960 con 0x7f684809e410 2023-09-30T22:29:53.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.920+0000 7f683dffb700 1 -- 172.21.15.16:0/3616737573 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6840003e10 con 0x7f684809e410 2023-09-30T22:29:53.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.920+0000 7f683dffb700 1 -- 172.21.15.16:0/3616737573 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6840010710 con 0x7f684809e410 2023-09-30T22:29:53.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.920+0000 7f684f62d700 1 -- 172.21.15.16:0/3616737573 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6840009a70 con 0x7f684809e410 2023-09-30T22:29:53.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.920+0000 7f684f62d700 1 -- 172.21.15.16:0/3616737573 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f684811ff60 con 0x7f684809e410 2023-09-30T22:29:53.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.921+0000 7f684f62d700 1 -- 172.21.15.16:0/3616737573 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6848074ea0 con 0x7f684809e410 2023-09-30T22:29:53.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.921+0000 7f683dffb700 1 -- 172.21.15.16:0/3616737573 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f6840010870 con 0x7f684809e410 2023-09-30T22:29:53.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.922+0000 7f683dffb700 1 -- 172.21.15.16:0/3616737573 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7f6840045740 con 0x7f684809e410 2023-09-30T22:29:53.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.923+0000 7f684d3c9700 1 -- 172.21.15.16:0/3616737573 >> v1:172.21.15.16:6816/126811 conn(0x7f682c02f510 legacy=0x7f682c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:53.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.929+0000 7fbd35206700 1 Processor -- start 2023-09-30T22:29:53.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.929+0000 7fbd35206700 1 -- start start 2023-09-30T22:29:53.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.930+0000 7fbd35206700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbd3009d6f0 0x7fbd3009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:53.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.930+0000 7fbd35206700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbd3009e410 0x7fbd30134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:53.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.930+0000 7fbd35206700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd3009b120 con 0x7fbd3009e000 2023-09-30T22:29:53.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.930+0000 7fbd35206700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3006b460 con 0x7fbd3009e410 2023-09-30T22:29:53.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.930+0000 7fbd35206700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbd3009a7e0 con 0x7fbd3009d6f0 2023-09-30T22:29:53.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.930+0000 7fbd2f59e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbd3009e410 0x7fbd30134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:53.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.930+0000 7fbd2e59c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fbd3009e000 0x7fbd301339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59212/0 (socket says 172.21.15.16:59212) 2023-09-30T22:29:53.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.930+0000 7fbd2e59c700 1 -- 172.21.15.16:0/3670298005 learned_addr learned my addr 172.21.15.16:0/3670298005 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:29:53.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.930+0000 7f683dffb700 1 -- 172.21.15.16:0/3616737573 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6840016930 con 0x7f684809e410 2023-09-30T22:29:53.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.930+0000 7fbd2ed9d700 1 --2- 172.21.15.16:0/3670298005 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbd3009d6f0 0x7fbd3009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:53.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.930+0000 7fbd2dd9b700 1 -- 172.21.15.16:0/3670298005 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1099584988 0 0) 0x7fbd3009b120 con 0x7fbd3009e000 2023-09-30T22:29:53.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.930+0000 7fbd2dd9b700 1 -- 172.21.15.16:0/3670298005 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd180035b0 con 0x7fbd3009e000 2023-09-30T22:29:53.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.930+0000 7fbd2f59e700 1 -- 172.21.15.16:0/3670298005 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbd3009d6f0 msgr2=0x7fbd3009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:53.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.930+0000 7fbd2f59e700 1 --2- 172.21.15.16:0/3670298005 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbd3009d6f0 0x7fbd3009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:53.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.931+0000 7fbd2f59e700 1 -- 172.21.15.16:0/3670298005 >> v1:172.21.15.16:6789/0 conn(0x7fbd3009e000 legacy=0x7fbd301339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:53.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.931+0000 7fbd2f59e700 1 -- 172.21.15.16:0/3670298005 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd3009a500 con 0x7fbd3009e410 2023-09-30T22:29:53.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.931+0000 7fbd2f59e700 1 --2- 172.21.15.16:0/3670298005 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbd3009e410 0x7fbd30134100 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbd28009dc0 tx=0x7fbd2800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26c1679dabee3714 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:53.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.931+0000 7fbd2dd9b700 1 -- 172.21.15.16:0/3670298005 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd2800e070 con 0x7fbd3009e410 2023-09-30T22:29:53.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.931+0000 7fbd2dd9b700 1 -- 172.21.15.16:0/3670298005 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd2800bae0 con 0x7fbd3009e410 2023-09-30T22:29:53.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.931+0000 7fbd2dd9b700 1 -- 172.21.15.16:0/3670298005 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd2800be90 con 0x7fbd3009e410 2023-09-30T22:29:53.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.932+0000 7fbd35206700 1 -- 172.21.15.16:0/3670298005 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbd3009e410 msgr2=0x7fbd30134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:53.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.932+0000 7fbd35206700 1 --2- 172.21.15.16:0/3670298005 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbd3009e410 0x7fbd30134100 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbd28009dc0 tx=0x7fbd2800b330 comp rx=0 tx=0).stop 2023-09-30T22:29:53.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.932+0000 7fbd35206700 1 -- 172.21.15.16:0/3670298005 shutdown_connections 2023-09-30T22:29:53.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.932+0000 7fbd35206700 1 --2- 172.21.15.16:0/3670298005 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbd3009e410 0x7fbd30134100 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:53.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.932+0000 7fbd35206700 1 --2- 172.21.15.16:0/3670298005 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbd3009d6f0 0x7fbd3009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:53.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.932+0000 7fbd35206700 1 -- 172.21.15.16:0/3670298005 >> 172.21.15.16:0/3670298005 conn(0x7fbd30066f50 msgr2=0x7fbd3009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:53.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.932+0000 7fbd35206700 1 -- 172.21.15.16:0/3670298005 shutdown_connections 2023-09-30T22:29:53.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.932+0000 7fbd35206700 1 -- 172.21.15.16:0/3670298005 wait complete. 2023-09-30T22:29:53.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.933+0000 7fbd35206700 1 Processor -- start 2023-09-30T22:29:53.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.933+0000 7fbd35206700 1 -- start start 2023-09-30T22:29:53.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.934+0000 7fbd35206700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd3009d6f0 0x7fbd301222e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:53.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.934+0000 7fbd35206700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbd3009e000 0x7fbd30122820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:53.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.934+0000 7fbd35206700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbd3009e410 0x7fbd30122d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:53.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.934+0000 7fbd35206700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd30125b70 con 0x7fbd3009d6f0 2023-09-30T22:29:53.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.934+0000 7fbd35206700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd301257d0 con 0x7fbd3009e410 2023-09-30T22:29:53.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.934+0000 7fbd35206700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbd30124c60 con 0x7fbd3009e000 2023-09-30T22:29:53.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.934+0000 7fbd2ed9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd3009d6f0 0x7fbd301222e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:53.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.934+0000 7fbd2f59e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbd3009e410 0x7fbd30122d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:53.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.934+0000 7fbd2ed9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd3009d6f0 0x7fbd301222e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48186/0 (socket says 172.21.15.16:48186) 2023-09-30T22:29:53.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.934+0000 7fbd2ed9d700 1 -- 172.21.15.16:0/1138440188 learned_addr learned my addr 172.21.15.16:0/1138440188 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:53.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.934+0000 7fbd2e59c700 1 --2- 172.21.15.16:0/1138440188 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbd3009e000 0x7fbd30122820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:53.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.935+0000 7fbd2f59e700 1 -- 172.21.15.16:0/1138440188 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbd3009e000 msgr2=0x7fbd30122820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:53.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.935+0000 7fbd2f59e700 1 --2- 172.21.15.16:0/1138440188 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbd3009e000 0x7fbd30122820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:53.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.935+0000 7fbd2f59e700 1 -- 172.21.15.16:0/1138440188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd3009d6f0 msgr2=0x7fbd301222e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:53.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.935+0000 7fbd2f59e700 1 --2- 172.21.15.16:0/1138440188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd3009d6f0 0x7fbd301222e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:53.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.935+0000 7fbd2f59e700 1 -- 172.21.15.16:0/1138440188 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd28009a70 con 0x7fbd3009e410 2023-09-30T22:29:53.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.935+0000 7fbd2e59c700 1 --2- 172.21.15.16:0/1138440188 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbd3009e000 0x7fbd30122820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:29:53.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.935+0000 7fbd2ed9d700 1 --2- 172.21.15.16:0/1138440188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd3009d6f0 0x7fbd301222e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:53.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.935+0000 7fbd2f59e700 1 --2- 172.21.15.16:0/1138440188 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbd3009e410 0x7fbd30122d60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbd28009dc0 tx=0x7fbd28004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:53.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.935+0000 7fbd177fe700 1 -- 172.21.15.16:0/1138440188 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd280045c0 con 0x7fbd3009e410 2023-09-30T22:29:53.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.935+0000 7fbd177fe700 1 -- 172.21.15.16:0/1138440188 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd28004740 con 0x7fbd3009e410 2023-09-30T22:29:53.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.935+0000 7fbd177fe700 1 -- 172.21.15.16:0/1138440188 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd28012430 con 0x7fbd3009e410 2023-09-30T22:29:53.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.935+0000 7fbd35206700 1 -- 172.21.15.16:0/1138440188 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd30123f00 con 0x7fbd3009e410 2023-09-30T22:29:53.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.935+0000 7fbd35206700 1 -- 172.21.15.16:0/1138440188 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd3009c710 con 0x7fbd3009e410 2023-09-30T22:29:53.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.936+0000 7fbd177fe700 1 -- 172.21.15.16:0/1138440188 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fbd28012590 con 0x7fbd3009e410 2023-09-30T22:29:53.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.937+0000 7fbd177fe700 1 -- 172.21.15.16:0/1138440188 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7fbd280418e0 con 0x7fbd3009e410 2023-09-30T22:29:53.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.937+0000 7fbd2ed9d700 1 -- 172.21.15.16:0/1138440188 >> v1:172.21.15.16:6816/126811 conn(0x7fbd1c02f540 legacy=0x7fbd1c0319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:53.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.937+0000 7fbd35206700 1 -- 172.21.15.16:0/1138440188 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd30074ea0 con 0x7fbd3009e410 2023-09-30T22:29:53.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.942+0000 7fbd177fe700 1 -- 172.21.15.16:0/1138440188 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbd2802d940 con 0x7fbd3009e410 2023-09-30T22:29:53.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.968+0000 7f2824a3d700 1 Processor -- start 2023-09-30T22:29:53.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.968+0000 7f2824a3d700 1 -- start start 2023-09-30T22:29:53.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.969+0000 7f2824a3d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2820133a80 0x7f2820133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:53.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.969+0000 7f2824a3d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f282012ff20 0x7f2820130330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:53.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.969+0000 7f2824a3d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2820130870 0x7f282013a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:53.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.969+0000 7f2824a3d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f282006c570 con 0x7f2820133a80 2023-09-30T22:29:53.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.969+0000 7f2824a3d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f282006bc00 con 0x7f2820130870 2023-09-30T22:29:53.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.969+0000 7f2824a3d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f28200a0040 con 0x7f282012ff20 2023-09-30T22:29:53.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.969+0000 7f281e59c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2820133a80 0x7f2820133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:53.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.969+0000 7f281e59c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2820133a80 0x7f2820133e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48200/0 (socket says 172.21.15.16:48200) 2023-09-30T22:29:53.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.969+0000 7f281e59c700 1 -- 172.21.15.16:0/2990544597 learned_addr learned my addr 172.21.15.16:0/2990544597 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:53.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.969+0000 7f281dd9b700 1 --2- 172.21.15.16:0/2990544597 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f282012ff20 0x7f2820130330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:53.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.969+0000 7f281ed9d700 1 --2- 172.21.15.16:0/2990544597 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2820130870 0x7f282013a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:53.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.969+0000 7f281e59c700 1 -- 172.21.15.16:0/2990544597 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f282012ff20 msgr2=0x7f2820130330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:53.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.969+0000 7f281e59c700 1 --2- 172.21.15.16:0/2990544597 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f282012ff20 0x7f2820130330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:53.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.969+0000 7f281e59c700 1 -- 172.21.15.16:0/2990544597 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2820130870 msgr2=0x7f282013a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:53.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.969+0000 7f281e59c700 1 --2- 172.21.15.16:0/2990544597 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2820130870 0x7f282013a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:53.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.969+0000 7f281e59c700 1 -- 172.21.15.16:0/2990544597 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2820133860 con 0x7f2820133a80 2023-09-30T22:29:53.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.970+0000 7f281e59c700 1 --2- 172.21.15.16:0/2990544597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2820133a80 0x7f2820133e50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f28100098f0 tx=0x7f2810009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4bd8a069c6def8d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:53.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.970+0000 7f281d59a700 1 -- 172.21.15.16:0/2990544597 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2810011830 con 0x7f2820133a80 2023-09-30T22:29:53.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.970+0000 7f281d59a700 1 -- 172.21.15.16:0/2990544597 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2810011990 con 0x7f2820133a80 2023-09-30T22:29:53.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.970+0000 7f281d59a700 1 -- 172.21.15.16:0/2990544597 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2810011d20 con 0x7f2820133a80 2023-09-30T22:29:53.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.970+0000 7f2824a3d700 1 -- 172.21.15.16:0/2990544597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2820133a80 msgr2=0x7f2820133e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:53.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.970+0000 7f2824a3d700 1 --2- 172.21.15.16:0/2990544597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2820133a80 0x7f2820133e50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f28100098f0 tx=0x7f2810009c00 comp rx=0 tx=0).stop 2023-09-30T22:29:53.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.971+0000 7f2824a3d700 1 -- 172.21.15.16:0/2990544597 shutdown_connections 2023-09-30T22:29:53.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.971+0000 7f2824a3d700 1 --2- 172.21.15.16:0/2990544597 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2820130870 0x7f282013a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:53.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.971+0000 7f2824a3d700 1 --2- 172.21.15.16:0/2990544597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2820133a80 0x7f2820133e50 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:53.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.971+0000 7f2824a3d700 1 --2- 172.21.15.16:0/2990544597 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f282012ff20 0x7f2820130330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:53.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.971+0000 7f2824a3d700 1 -- 172.21.15.16:0/2990544597 >> 172.21.15.16:0/2990544597 conn(0x7f28200a4210 msgr2=0x7f28200a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:53.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.971+0000 7f2824a3d700 1 -- 172.21.15.16:0/2990544597 shutdown_connections 2023-09-30T22:29:53.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.971+0000 7f2824a3d700 1 -- 172.21.15.16:0/2990544597 wait complete. 2023-09-30T22:29:53.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.972+0000 7f2824a3d700 1 Processor -- start 2023-09-30T22:29:53.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.972+0000 7f2824a3d700 1 -- start start 2023-09-30T22:29:53.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.972+0000 7f2824a3d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f282012ff20 0x7f282009c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:53.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.972+0000 7f2824a3d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2820130870 0x7f282009cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:53.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.972+0000 7f2824a3d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2820133a80 0x7f282009d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:53.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.973+0000 7f2824a3d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2820120250 con 0x7f2820130870 2023-09-30T22:29:53.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.973+0000 7f2824a3d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f282009db20 con 0x7f2820133a80 2023-09-30T22:29:53.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.973+0000 7f2824a3d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f28200a2050 con 0x7f282012ff20 2023-09-30T22:29:53.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.973+0000 7f281ed9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2820133a80 0x7f282009d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:53.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.973+0000 7f281ed9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2820133a80 0x7f282009d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:49612/0 (socket says 172.21.15.16:49612) 2023-09-30T22:29:53.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.973+0000 7f281ed9d700 1 -- 172.21.15.16:0/2574290826 learned_addr learned my addr 172.21.15.16:0/2574290826 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:53.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.973+0000 7f281e59c700 1 --2- 172.21.15.16:0/2574290826 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f282012ff20 0x7f282009c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:53.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.973+0000 7f281dd9b700 1 --2- 172.21.15.16:0/2574290826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2820130870 0x7f282009cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:53.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.973+0000 7f281ed9d700 1 -- 172.21.15.16:0/2574290826 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f282012ff20 msgr2=0x7f282009c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:53.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.973+0000 7f281ed9d700 1 --2- 172.21.15.16:0/2574290826 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f282012ff20 0x7f282009c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:53.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.973+0000 7f281ed9d700 1 -- 172.21.15.16:0/2574290826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2820130870 msgr2=0x7f282009cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:53.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.973+0000 7f281ed9d700 1 --2- 172.21.15.16:0/2574290826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2820130870 0x7f282009cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:53.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.973+0000 7f281ed9d700 1 -- 172.21.15.16:0/2574290826 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2810009550 con 0x7f2820133a80 2023-09-30T22:29:53.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.973+0000 7f281ed9d700 1 --2- 172.21.15.16:0/2574290826 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2820133a80 0x7f282009d160 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f281800efc0 tx=0x7f281800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:53.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.974+0000 7f280effd700 1 -- 172.21.15.16:0/2574290826 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281800cd40 con 0x7f2820133a80 2023-09-30T22:29:53.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.974+0000 7f280effd700 1 -- 172.21.15.16:0/2574290826 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f281800f040 con 0x7f2820133a80 2023-09-30T22:29:53.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.974+0000 7f280effd700 1 -- 172.21.15.16:0/2574290826 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28180146c0 con 0x7f2820133a80 2023-09-30T22:29:53.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.974+0000 7f2824a3d700 1 -- 172.21.15.16:0/2574290826 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2820120490 con 0x7f2820133a80 2023-09-30T22:29:53.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.974+0000 7f2824a3d700 1 -- 172.21.15.16:0/2574290826 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f282011fe90 con 0x7f2820133a80 2023-09-30T22:29:53.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.975+0000 7f2824a3d700 1 -- 172.21.15.16:0/2574290826 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2820074ea0 con 0x7f2820133a80 2023-09-30T22:29:53.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.976+0000 7f280effd700 1 -- 172.21.15.16:0/2574290826 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f2818014820 con 0x7f2820133a80 2023-09-30T22:29:53.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.976+0000 7f280effd700 1 -- 172.21.15.16:0/2574290826 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7f2818040cb0 con 0x7f2820133a80 2023-09-30T22:29:53.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.976+0000 7f281e59c700 1 -- 172.21.15.16:0/2574290826 >> v1:172.21.15.16:6816/126811 conn(0x7f280002f510 legacy=0x7f28000319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:53.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:53.980+0000 7f280effd700 1 -- 172.21.15.16:0/2574290826 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2818012930 con 0x7f2820133a80 2023-09-30T22:29:54.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:54.097+0000 7f9888bd6700 1 -- 172.21.15.16:0/166392186 >> v1:172.21.15.16:6816/126811 conn(0x7f986802f510 legacy=0x7f98680319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:54.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:54.124+0000 7f684d3c9700 1 -- 172.21.15.16:0/3616737573 >> v1:172.21.15.16:6816/126811 conn(0x7f682c02f510 legacy=0x7f682c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:54.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:54.131+0000 7f988ae3a700 1 -- 172.21.15.16:0/166392186 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9884074ea0 con 0x7f98840a1340 2023-09-30T22:29:54.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:54.131+0000 7f98817fa700 1 -- 172.21.15.16:0/166392186 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f987001a2b0 con 0x7f98840a1340 2023-09-30T22:29:54.132 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:29:54.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:54.134+0000 7f988ae3a700 1 -- 172.21.15.16:0/166392186 >> v1:172.21.15.16:6816/126811 conn(0x7f986802f510 legacy=0x7f98680319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:54.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:54.134+0000 7f988ae3a700 1 -- 172.21.15.16:0/166392186 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f98840a1340 msgr2=0x7f9884127a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:54.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:54.134+0000 7f988ae3a700 1 --2- 172.21.15.16:0/166392186 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f98840a1340 0x7f9884127a10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f987000c540 tx=0x7f987000c850 comp rx=0 tx=0).stop 2023-09-30T22:29:54.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:54.135+0000 7f988ae3a700 1 -- 172.21.15.16:0/166392186 shutdown_connections 2023-09-30T22:29:54.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:54.135+0000 7f988ae3a700 1 --2- 172.21.15.16:0/166392186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f988413d820 0x7f9884127f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:54.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:54.135+0000 7f988ae3a700 1 --2- 172.21.15.16:0/166392186 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98840a0a30 0x7f98841274d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:54.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:54.135+0000 7f988ae3a700 1 --2- 172.21.15.16:0/166392186 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f98840a1340 0x7f9884127a10 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:54.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:54.135+0000 7f988ae3a700 1 -- 172.21.15.16:0/166392186 >> 172.21.15.16:0/166392186 conn(0x7f98840a6550 msgr2=0x7f988409c6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:54.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:54.135+0000 7f988ae3a700 1 -- 172.21.15.16:0/166392186 shutdown_connections 2023-09-30T22:29:54.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:54.135+0000 7f988ae3a700 1 -- 172.21.15.16:0/166392186 wait complete. 2023-09-30T22:29:54.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:54.138+0000 7fbd2ed9d700 1 -- 172.21.15.16:0/1138440188 >> v1:172.21.15.16:6816/126811 conn(0x7fbd1c02f540 legacy=0x7fbd1c0319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:54.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:54.141+0000 7f684f62d700 1 -- 172.21.15.16:0/3616737573 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6848121c50 con 0x7f684809e410 2023-09-30T22:29:54.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:54.141+0000 7f683dffb700 1 -- 172.21.15.16:0/3616737573 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6840026020 con 0x7f684809e410 2023-09-30T22:29:54.142 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:29:54.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:54.143+0000 7f684f62d700 1 -- 172.21.15.16:0/3616737573 >> v1:172.21.15.16:6816/126811 conn(0x7f682c02f510 legacy=0x7f682c0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:54.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:54.144+0000 7f684f62d700 1 -- 172.21.15.16:0/3616737573 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f684809e410 msgr2=0x7f684811ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:54.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:54.144+0000 7f684f62d700 1 --2- 172.21.15.16:0/3616737573 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f684809e410 0x7f684811ef00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f684000b680 tx=0x7f6840004ad0 comp rx=0 tx=0).stop 2023-09-30T22:29:54.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:54.144+0000 7f684f62d700 1 -- 172.21.15.16:0/3616737573 shutdown_connections 2023-09-30T22:29:54.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:54.144+0000 7f684f62d700 1 --2- 172.21.15.16:0/3616737573 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f684809d6f0 0x7f684811e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:54.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:54.144+0000 7f684f62d700 1 --2- 172.21.15.16:0/3616737573 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f684809e410 0x7f684811ef00 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:54.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:54.144+0000 7f684f62d700 1 --2- 172.21.15.16:0/3616737573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f684809e000 0x7f684811e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:54.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:54.144+0000 7f684f62d700 1 -- 172.21.15.16:0/3616737573 >> 172.21.15.16:0/3616737573 conn(0x7f6848066f50 msgr2=0x7f684806a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:54.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:54.144+0000 7f684f62d700 1 -- 172.21.15.16:0/3616737573 shutdown_connections 2023-09-30T22:29:54.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:54.144+0000 7f684f62d700 1 -- 172.21.15.16:0/3616737573 wait complete. 2023-09-30T22:29:54.156 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:29:54.158 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:29:54.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:54.167+0000 7fbd35206700 1 -- 172.21.15.16:0/1138440188 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fbd3006c360 con 0x7fbd3009e410 2023-09-30T22:29:54.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:54.167+0000 7fbd177fe700 1 -- 172.21.15.16:0/1138440188 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbd2802d350 con 0x7fbd3009e410 2023-09-30T22:29:54.167 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:29:54.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:54.169+0000 7fbd35206700 1 -- 172.21.15.16:0/1138440188 >> v1:172.21.15.16:6816/126811 conn(0x7fbd1c02f540 legacy=0x7fbd1c0319f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:54.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:54.169+0000 7fbd35206700 1 -- 172.21.15.16:0/1138440188 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbd3009e410 msgr2=0x7fbd30122d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:54.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:54.169+0000 7fbd35206700 1 --2- 172.21.15.16:0/1138440188 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbd3009e410 0x7fbd30122d60 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbd28009dc0 tx=0x7fbd28004ad0 comp rx=0 tx=0).stop 2023-09-30T22:29:54.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:54.169+0000 7fbd35206700 1 -- 172.21.15.16:0/1138440188 shutdown_connections 2023-09-30T22:29:54.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:54.169+0000 7fbd35206700 1 --2- 172.21.15.16:0/1138440188 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbd3009e410 0x7fbd30122d60 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:54.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:54.169+0000 7fbd35206700 1 --2- 172.21.15.16:0/1138440188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd3009d6f0 0x7fbd301222e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:54.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:54.169+0000 7fbd35206700 1 --2- 172.21.15.16:0/1138440188 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbd3009e000 0x7fbd30122820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:54.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:54.169+0000 7fbd35206700 1 -- 172.21.15.16:0/1138440188 >> 172.21.15.16:0/1138440188 conn(0x7fbd30066f50 msgr2=0x7fbd30070e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:54.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:54.170+0000 7fbd35206700 1 -- 172.21.15.16:0/1138440188 shutdown_connections 2023-09-30T22:29:54.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:54.170+0000 7fbd35206700 1 -- 172.21.15.16:0/1138440188 wait complete. 2023-09-30T22:29:54.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:54.177+0000 7f281e59c700 1 -- 172.21.15.16:0/2574290826 >> v1:172.21.15.16:6816/126811 conn(0x7f280002f510 legacy=0x7f28000319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:54.184 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:29:54.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:54.187+0000 7f2824a3d700 1 -- 172.21.15.16:0/2574290826 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f282006c240 con 0x7f2820133a80 2023-09-30T22:29:54.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:54.187+0000 7f280effd700 1 -- 172.21.15.16:0/2574290826 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2818018070 con 0x7f2820133a80 2023-09-30T22:29:54.188 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:29:54.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:54.189+0000 7f2824a3d700 1 -- 172.21.15.16:0/2574290826 >> v1:172.21.15.16:6816/126811 conn(0x7f280002f510 legacy=0x7f28000319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:54.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:54.189+0000 7f2824a3d700 1 -- 172.21.15.16:0/2574290826 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2820133a80 msgr2=0x7f282009d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:54.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:54.189+0000 7f2824a3d700 1 --2- 172.21.15.16:0/2574290826 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2820133a80 0x7f282009d160 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f281800efc0 tx=0x7f281800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:29:54.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:54.189+0000 7f2824a3d700 1 -- 172.21.15.16:0/2574290826 shutdown_connections 2023-09-30T22:29:54.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:54.189+0000 7f2824a3d700 1 --2- 172.21.15.16:0/2574290826 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2820133a80 0x7f282009d160 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:54.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:54.189+0000 7f2824a3d700 1 --2- 172.21.15.16:0/2574290826 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f282012ff20 0x7f282009c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:54.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:54.189+0000 7f2824a3d700 1 --2- 172.21.15.16:0/2574290826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2820130870 0x7f282009cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:54.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:54.189+0000 7f2824a3d700 1 -- 172.21.15.16:0/2574290826 >> 172.21.15.16:0/2574290826 conn(0x7f28200a4210 msgr2=0x7f282009e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:54.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:54.189+0000 7f2824a3d700 1 -- 172.21.15.16:0/2574290826 shutdown_connections 2023-09-30T22:29:54.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:54.189+0000 7f2824a3d700 1 -- 172.21.15.16:0/2574290826 wait complete. 2023-09-30T22:29:54.203 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:29:55.157 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:29:55.159 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:29:55.185 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:29:55.203 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:29:55.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.308+0000 7fe70e07a700 1 Processor -- start 2023-09-30T22:29:55.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.309+0000 7fe70e07a700 1 -- start start 2023-09-30T22:29:55.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.309+0000 7fe70e07a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe70809e930 0x7fe70809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:55.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.309+0000 7fe70e07a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe70809f650 0x7fe708131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:55.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.309+0000 7fe70e07a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe708070940 con 0x7fe70809f240 2023-09-30T22:29:55.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.309+0000 7fe70e07a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe70806fd60 con 0x7fe70809e930 2023-09-30T22:29:55.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.309+0000 7fe70e07a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe70806fa80 con 0x7fe70809f650 2023-09-30T22:29:55.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.309+0000 7fe707fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe70809f650 0x7fe708131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:55.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.309+0000 7fe706ffd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe70809f240 0x7fe7081317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59226/0 (socket says 172.21.15.16:59226) 2023-09-30T22:29:55.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.309+0000 7fe707fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe70809f650 0x7fe708131f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:52192/0 (socket says 172.21.15.16:52192) 2023-09-30T22:29:55.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.309+0000 7fe706ffd700 1 -- 172.21.15.16:0/347907713 learned_addr learned my addr 172.21.15.16:0/347907713 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:29:55.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.309+0000 7fe7077fe700 1 --2- 172.21.15.16:0/347907713 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe70809e930 0x7fe70809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:55.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.310+0000 7fe707fff700 1 -- 172.21.15.16:0/347907713 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe70809e930 msgr2=0x7fe70809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:55.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.310+0000 7fe707fff700 1 --2- 172.21.15.16:0/347907713 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe70809e930 0x7fe70809ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:55.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.310+0000 7fe707fff700 1 -- 172.21.15.16:0/347907713 >> v1:172.21.15.16:6789/0 conn(0x7fe70809f240 legacy=0x7fe7081317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:55.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.310+0000 7fe707fff700 1 -- 172.21.15.16:0/347907713 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe70809fb60 con 0x7fe70809f650 2023-09-30T22:29:55.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.310+0000 7fe7077fe700 1 --2- 172.21.15.16:0/347907713 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe70809e930 0x7fe70809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:55.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.310+0000 7fe707fff700 1 --2- 172.21.15.16:0/347907713 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe70809f650 0x7fe708131f00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe700009dc0 tx=0x7fe70000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e25c5417adbc0f9e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:55.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.310+0000 7fe7067fc700 1 -- 172.21.15.16:0/347907713 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70000bd20 con 0x7fe70809f650 2023-09-30T22:29:55.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.310+0000 7fe7067fc700 1 -- 172.21.15.16:0/347907713 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe70000be80 con 0x7fe70809f650 2023-09-30T22:29:55.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.310+0000 7fe7067fc700 1 -- 172.21.15.16:0/347907713 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe700010710 con 0x7fe70809f650 2023-09-30T22:29:55.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.311+0000 7fe70e07a700 1 -- 172.21.15.16:0/347907713 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe70809f650 msgr2=0x7fe708131f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:55.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.311+0000 7fe70e07a700 1 --2- 172.21.15.16:0/347907713 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe70809f650 0x7fe708131f00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe700009dc0 tx=0x7fe70000b330 comp rx=0 tx=0).stop 2023-09-30T22:29:55.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.311+0000 7fe70e07a700 1 -- 172.21.15.16:0/347907713 shutdown_connections 2023-09-30T22:29:55.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.311+0000 7fe70e07a700 1 --2- 172.21.15.16:0/347907713 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe70809f650 0x7fe708131f00 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:55.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.311+0000 7fe70e07a700 1 --2- 172.21.15.16:0/347907713 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe70809e930 0x7fe70809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:55.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.311+0000 7fe70e07a700 1 -- 172.21.15.16:0/347907713 >> 172.21.15.16:0/347907713 conn(0x7fe7080a4210 msgr2=0x7fe7080a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:55.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.311+0000 7fe70e07a700 1 -- 172.21.15.16:0/347907713 shutdown_connections 2023-09-30T22:29:55.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.311+0000 7fe70e07a700 1 -- 172.21.15.16:0/347907713 wait complete. 2023-09-30T22:29:55.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.312+0000 7fe70e07a700 1 Processor -- start 2023-09-30T22:29:55.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.313+0000 7fe70e07a700 1 -- start start 2023-09-30T22:29:55.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.313+0000 7fe70e07a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe70809e930 0x7fe70811d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:55.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.313+0000 7fe70e07a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe70809f240 0x7fe70811d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:55.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.313+0000 7fe70e07a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe70809f650 0x7fe70811dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:55.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.313+0000 7fe70e07a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe70811f720 con 0x7fe70809f650 2023-09-30T22:29:55.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.313+0000 7fe70e07a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7080a3060 con 0x7fe70809e930 2023-09-30T22:29:55.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.313+0000 7fe70e07a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe708120d60 con 0x7fe70809f240 2023-09-30T22:29:55.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.313+0000 7fe7077fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe70809e930 0x7fe70811d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:55.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.313+0000 7fe7077fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe70809e930 0x7fe70811d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:49622/0 (socket says 172.21.15.16:49622) 2023-09-30T22:29:55.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.313+0000 7fe7077fe700 1 -- 172.21.15.16:0/3860049337 learned_addr learned my addr 172.21.15.16:0/3860049337 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:55.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.313+0000 7fe706ffd700 1 --2- 172.21.15.16:0/3860049337 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe70809f240 0x7fe70811d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:55.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.313+0000 7fe707fff700 1 --2- 172.21.15.16:0/3860049337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe70809f650 0x7fe70811dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:55.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.314+0000 7fe7077fe700 1 -- 172.21.15.16:0/3860049337 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe70809f240 msgr2=0x7fe70811d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:55.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.314+0000 7fe7077fe700 1 --2- 172.21.15.16:0/3860049337 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe70809f240 0x7fe70811d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:55.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.314+0000 7fe7077fe700 1 -- 172.21.15.16:0/3860049337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe70809f650 msgr2=0x7fe70811dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:55.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.314+0000 7fe7077fe700 1 --2- 172.21.15.16:0/3860049337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe70809f650 0x7fe70811dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:55.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.314+0000 7fe7077fe700 1 -- 172.21.15.16:0/3860049337 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f80094a0 con 0x7fe70809e930 2023-09-30T22:29:55.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.314+0000 7fe7077fe700 1 --2- 172.21.15.16:0/3860049337 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe70809e930 0x7fe70811d2e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe6f800ebe0 tx=0x7fe6f800eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:55.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.314+0000 7fe6effff700 1 -- 172.21.15.16:0/3860049337 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f800c850 con 0x7fe70809e930 2023-09-30T22:29:55.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.314+0000 7fe6effff700 1 -- 172.21.15.16:0/3860049337 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6f800c9b0 con 0x7fe70809e930 2023-09-30T22:29:55.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.314+0000 7fe70e07a700 1 -- 172.21.15.16:0/3860049337 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe700009a70 con 0x7fe70809e930 2023-09-30T22:29:55.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.314+0000 7fe6effff700 1 -- 172.21.15.16:0/3860049337 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f800cd40 con 0x7fe70809e930 2023-09-30T22:29:55.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.314+0000 7fe70e07a700 1 -- 172.21.15.16:0/3860049337 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7080a0190 con 0x7fe70809e930 2023-09-30T22:29:55.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.316+0000 7fe70e07a700 1 -- 172.21.15.16:0/3860049337 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe708120610 con 0x7fe70809e930 2023-09-30T22:29:55.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.316+0000 7fe6effff700 1 -- 172.21.15.16:0/3860049337 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fe6f8013470 con 0x7fe70809e930 2023-09-30T22:29:55.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.316+0000 7fe6effff700 1 -- 172.21.15.16:0/3860049337 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7fe6f803fd10 con 0x7fe70809e930 2023-09-30T22:29:55.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.316+0000 7fe706ffd700 1 -- 172.21.15.16:0/3860049337 >> v1:172.21.15.16:6816/126811 conn(0x7fe6f402f510 legacy=0x7fe6f40319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:55.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.321+0000 7fe6effff700 1 -- 172.21.15.16:0/3860049337 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe6f8013720 con 0x7fe70809e930 2023-09-30T22:29:55.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.326+0000 7f8bcc6c7700 1 Processor -- start 2023-09-30T22:29:55.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.326+0000 7f8bcc6c7700 1 -- start start 2023-09-30T22:29:55.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.326+0000 7f8bcc6c7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8bc4133420 0x7f8bc41337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:55.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.327+0000 7f8bcc6c7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8bc4133e30 0x7f8bc412ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:55.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.327+0000 7f8bcc6c7700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8bc4134260 0x7f8bc41304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:55.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.327+0000 7f8bcc6c7700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bc406b990 con 0x7f8bc4134260 2023-09-30T22:29:55.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.327+0000 7f8bcc6c7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc40a0590 con 0x7f8bc4133420 2023-09-30T22:29:55.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.327+0000 7f8bcc6c7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc409fb80 con 0x7f8bc4133e30 2023-09-30T22:29:55.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.327+0000 7f8bca463700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8bc4133420 0x7f8bc41337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:55.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.327+0000 7f8bca463700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8bc4133420 0x7f8bc41337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:49638/0 (socket says 172.21.15.16:49638) 2023-09-30T22:29:55.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.327+0000 7f8bca463700 1 -- 172.21.15.16:0/1183054024 learned_addr learned my addr 172.21.15.16:0/1183054024 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:55.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.327+0000 7f8bcac64700 1 --2- 172.21.15.16:0/1183054024 >> v2:172.21.15.16:3300/0 conn(0x7f8bc4134260 0x7f8bc41304d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:55.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.327+0000 7f8bca463700 1 -- 172.21.15.16:0/1183054024 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8bc4133e30 msgr2=0x7f8bc412ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:55.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.327+0000 7f8bca463700 1 --2- 172.21.15.16:0/1183054024 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8bc4133e30 0x7f8bc412ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:55.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.327+0000 7f8bca463700 1 -- 172.21.15.16:0/1183054024 >> v2:172.21.15.16:3300/0 conn(0x7f8bc4134260 msgr2=0x7f8bc41304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:55.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.327+0000 7f8bca463700 1 --2- 172.21.15.16:0/1183054024 >> v2:172.21.15.16:3300/0 conn(0x7f8bc4134260 0x7f8bc41304d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:55.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.327+0000 7f8bca463700 1 -- 172.21.15.16:0/1183054024 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc40a0030 con 0x7f8bc4133420 2023-09-30T22:29:55.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.327+0000 7f8bca463700 1 --2- 172.21.15.16:0/1183054024 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8bc4133420 0x7f8bc41337f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8bc00098f0 tx=0x7f8bc0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6e5226ffeddfcc2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:55.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.328+0000 7f8bc9461700 1 -- 172.21.15.16:0/1183054024 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc0011830 con 0x7f8bc4133420 2023-09-30T22:29:55.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.328+0000 7f8bc9461700 1 -- 172.21.15.16:0/1183054024 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bc0011990 con 0x7f8bc4133420 2023-09-30T22:29:55.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.328+0000 7f8bc9461700 1 -- 172.21.15.16:0/1183054024 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc0011d20 con 0x7f8bc4133420 2023-09-30T22:29:55.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.328+0000 7f8bcc6c7700 1 -- 172.21.15.16:0/1183054024 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8bc4133420 msgr2=0x7f8bc41337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:55.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.328+0000 7f8bcc6c7700 1 --2- 172.21.15.16:0/1183054024 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8bc4133420 0x7f8bc41337f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8bc00098f0 tx=0x7f8bc0009c00 comp rx=0 tx=0).stop 2023-09-30T22:29:55.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.329+0000 7f8bcc6c7700 1 -- 172.21.15.16:0/1183054024 shutdown_connections 2023-09-30T22:29:55.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.329+0000 7f8bcc6c7700 1 --2- 172.21.15.16:0/1183054024 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8bc4133420 0x7f8bc41337f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:55.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.329+0000 7f8bcc6c7700 1 --2- 172.21.15.16:0/1183054024 >> v2:172.21.15.16:3300/0 conn(0x7f8bc4134260 0x7f8bc41304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:55.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.329+0000 7f8bcc6c7700 1 --2- 172.21.15.16:0/1183054024 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8bc4133e30 0x7f8bc412ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:55.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.329+0000 7f8bcc6c7700 1 -- 172.21.15.16:0/1183054024 >> 172.21.15.16:0/1183054024 conn(0x7f8bc40a4210 msgr2=0x7f8bc40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:55.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.329+0000 7f8bcc6c7700 1 -- 172.21.15.16:0/1183054024 shutdown_connections 2023-09-30T22:29:55.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.329+0000 7f8bcc6c7700 1 -- 172.21.15.16:0/1183054024 wait complete. 2023-09-30T22:29:55.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.330+0000 7f8bcc6c7700 1 Processor -- start 2023-09-30T22:29:55.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.330+0000 7f8bcc6c7700 1 -- start start 2023-09-30T22:29:55.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.330+0000 7f8bcc6c7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8bc4133420 0x7f8bc409c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:55.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.330+0000 7f8bcc6c7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bc4133e30 0x7f8bc409ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:55.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.330+0000 7f8bcc6c7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8bc4134260 0x7f8bc409d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:55.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.330+0000 7f8bcc6c7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc41201f0 con 0x7f8bc4133e30 2023-09-30T22:29:55.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.330+0000 7f8bcc6c7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc4120c60 con 0x7f8bc4133420 2023-09-30T22:29:55.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.330+0000 7f8bcc6c7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc409e150 con 0x7f8bc4134260 2023-09-30T22:29:55.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.330+0000 7f8bca463700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8bc4133420 0x7f8bc409c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:55.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.330+0000 7f8bca463700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8bc4133420 0x7f8bc409c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:49650/0 (socket says 172.21.15.16:49650) 2023-09-30T22:29:55.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.330+0000 7f8bca463700 1 -- 172.21.15.16:0/907520901 learned_addr learned my addr 172.21.15.16:0/907520901 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:55.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.331+0000 7f8bcac64700 1 --2- 172.21.15.16:0/907520901 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8bc4134260 0x7f8bc409d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:55.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.331+0000 7f8bca463700 1 -- 172.21.15.16:0/907520901 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8bc4134260 msgr2=0x7f8bc409d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:55.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.331+0000 7f8bca463700 1 --2- 172.21.15.16:0/907520901 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8bc4134260 0x7f8bc409d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:55.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.331+0000 7f8bca463700 1 -- 172.21.15.16:0/907520901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bc4133e30 msgr2=0x7f8bc409ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:55.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.331+0000 7f8bca463700 1 --2- 172.21.15.16:0/907520901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bc4133e30 0x7f8bc409ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:55.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.331+0000 7f8bca463700 1 -- 172.21.15.16:0/907520901 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc0009550 con 0x7f8bc4133420 2023-09-30T22:29:55.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.331+0000 7f8bca463700 1 --2- 172.21.15.16:0/907520901 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8bc4133420 0x7f8bc409c8c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0000c00 tx=0x7f8bc0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:55.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.331+0000 7f8bbaffd700 1 -- 172.21.15.16:0/907520901 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc0004590 con 0x7f8bc4133420 2023-09-30T22:29:55.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.331+0000 7f8bbaffd700 1 -- 172.21.15.16:0/907520901 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bc00036a0 con 0x7f8bc4133420 2023-09-30T22:29:55.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.331+0000 7f8bbaffd700 1 -- 172.21.15.16:0/907520901 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc0003a50 con 0x7f8bc4133420 2023-09-30T22:29:55.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.331+0000 7f8bcc6c7700 1 -- 172.21.15.16:0/907520901 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bc412db20 con 0x7f8bc4133420 2023-09-30T22:29:55.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.332+0000 7f8bcc6c7700 1 -- 172.21.15.16:0/907520901 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bc41c8660 con 0x7f8bc4133420 2023-09-30T22:29:55.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.333+0000 7f8bbaffd700 1 -- 172.21.15.16:0/907520901 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f8bc0021070 con 0x7f8bc4133420 2023-09-30T22:29:55.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.333+0000 7f8bcc6c7700 1 -- 172.21.15.16:0/907520901 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bc4074ea0 con 0x7f8bc4133420 2023-09-30T22:29:55.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.333+0000 7f8bbaffd700 1 -- 172.21.15.16:0/907520901 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7f8bc0040f20 con 0x7f8bc4133420 2023-09-30T22:29:55.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.333+0000 7f8bc9c62700 1 -- 172.21.15.16:0/907520901 >> v1:172.21.15.16:6816/126811 conn(0x7f8ba802f510 legacy=0x7f8ba80319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:55.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.342+0000 7f8bbaffd700 1 -- 172.21.15.16:0/907520901 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8bc0019770 con 0x7f8bc4133420 2023-09-30T22:29:55.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.355+0000 7ff48e071700 1 Processor -- start 2023-09-30T22:29:55.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.355+0000 7ff48e071700 1 -- start start 2023-09-30T22:29:55.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.356+0000 7ff48e071700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff488133420 0x7ff4881337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:55.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.356+0000 7ff48e071700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff488133e30 0x7ff48812ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:55.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.356+0000 7ff48e071700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff48806b990 con 0x7ff488134260 2023-09-30T22:29:55.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.356+0000 7ff48e071700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4880a0590 con 0x7ff488133e30 2023-09-30T22:29:55.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.356+0000 7ff48e071700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff48809fb80 con 0x7ff488133420 2023-09-30T22:29:55.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.356+0000 7ff4877fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff488133420 0x7ff4881337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:55.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.356+0000 7ff486ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff488133e30 0x7ff48812ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:55.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.356+0000 7ff4877fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff488133420 0x7ff4881337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:49664/0 (socket says 172.21.15.16:49664) 2023-09-30T22:29:55.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.356+0000 7ff4877fe700 1 -- 172.21.15.16:0/749910089 learned_addr learned my addr 172.21.15.16:0/749910089 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:55.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.356+0000 7ff486ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff488133e30 0x7ff48812ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48262/0 (socket says 172.21.15.16:48262) 2023-09-30T22:29:55.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.357+0000 7ff4877fe700 1 -- 172.21.15.16:0/749910089 >> v2:172.21.15.16:3300/0 conn(0x7ff488133e30 msgr2=0x7ff48812ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:55.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.357+0000 7ff4877fe700 1 --2- 172.21.15.16:0/749910089 >> v2:172.21.15.16:3300/0 conn(0x7ff488133e30 0x7ff48812ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:55.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.357+0000 7ff4877fe700 1 -- 172.21.15.16:0/749910089 >> v1:172.21.15.16:6789/0 conn(0x7ff488134260 legacy=0x7ff4881304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:55.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.357+0000 7ff4877fe700 1 -- 172.21.15.16:0/749910089 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4880a0030 con 0x7ff488133420 2023-09-30T22:29:55.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.357+0000 7ff4867fc700 1 -- 172.21.15.16:0/749910089 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3403025484 0 0) 0x7ff48806b990 con 0x7ff488134260 2023-09-30T22:29:55.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.357+0000 7ff486ffd700 1 --2- 172.21.15.16:0/749910089 >> v2:172.21.15.16:3300/0 conn(0x7ff488133e30 0x7ff48812ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:55.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.357+0000 7ff4877fe700 1 --2- 172.21.15.16:0/749910089 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff488133420 0x7ff4881337f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff4780098f0 tx=0x7ff478009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bce812795cae6de server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:55.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.357+0000 7ff4867fc700 1 -- 172.21.15.16:0/749910089 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff478011830 con 0x7ff488133420 2023-09-30T22:29:55.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.357+0000 7ff4867fc700 1 -- 172.21.15.16:0/749910089 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff478011990 con 0x7ff488133420 2023-09-30T22:29:55.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.357+0000 7ff4867fc700 1 -- 172.21.15.16:0/749910089 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff478011d20 con 0x7ff488133420 2023-09-30T22:29:55.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.358+0000 7ff48e071700 1 -- 172.21.15.16:0/749910089 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff488133420 msgr2=0x7ff4881337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:55.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.358+0000 7ff48e071700 1 --2- 172.21.15.16:0/749910089 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff488133420 0x7ff4881337f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff4780098f0 tx=0x7ff478009c00 comp rx=0 tx=0).stop 2023-09-30T22:29:55.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.358+0000 7ff48e071700 1 -- 172.21.15.16:0/749910089 shutdown_connections 2023-09-30T22:29:55.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.358+0000 7ff48e071700 1 --2- 172.21.15.16:0/749910089 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff488133420 0x7ff4881337f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:55.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.358+0000 7ff48e071700 1 --2- 172.21.15.16:0/749910089 >> v2:172.21.15.16:3300/0 conn(0x7ff488133e30 0x7ff48812ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:55.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.358+0000 7ff48e071700 1 -- 172.21.15.16:0/749910089 >> 172.21.15.16:0/749910089 conn(0x7ff4880a4210 msgr2=0x7ff4880a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:55.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.358+0000 7ff48e071700 1 -- 172.21.15.16:0/749910089 shutdown_connections 2023-09-30T22:29:55.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.358+0000 7ff48e071700 1 -- 172.21.15.16:0/749910089 wait complete. 2023-09-30T22:29:55.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.359+0000 7ff48e071700 1 Processor -- start 2023-09-30T22:29:55.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.359+0000 7ff48e071700 1 -- start start 2023-09-30T22:29:55.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.360+0000 7ff48e071700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff488133420 0x7ff48809c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:55.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.360+0000 7ff48e071700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff488133e30 0x7ff48809c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:55.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.360+0000 7ff48e071700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff488134260 0x7ff48809cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:55.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.360+0000 7ff48e071700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48811fad0 con 0x7ff488133420 2023-09-30T22:29:55.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.360+0000 7ff48e071700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff48812d850 con 0x7ff488133e30 2023-09-30T22:29:55.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.360+0000 7ff48e071700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff48811ee40 con 0x7ff488134260 2023-09-30T22:29:55.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.360+0000 7ff4877fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff488133420 0x7ff48809c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:55.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.360+0000 7ff4877fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff488133420 0x7ff48809c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48264/0 (socket says 172.21.15.16:48264) 2023-09-30T22:29:55.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.360+0000 7ff4877fe700 1 -- 172.21.15.16:0/176246587 learned_addr learned my addr 172.21.15.16:0/176246587 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:55.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.360+0000 7ff487fff700 1 --2- 172.21.15.16:0/176246587 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff488134260 0x7ff48809cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:55.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.360+0000 7ff486ffd700 1 --2- 172.21.15.16:0/176246587 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff488133e30 0x7ff48809c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:55.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.361+0000 7ff4877fe700 1 -- 172.21.15.16:0/176246587 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff488134260 msgr2=0x7ff48809cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:55.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.361+0000 7ff4877fe700 1 --2- 172.21.15.16:0/176246587 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff488134260 0x7ff48809cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:55.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.361+0000 7ff4877fe700 1 -- 172.21.15.16:0/176246587 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff488133e30 msgr2=0x7ff48809c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:55.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.361+0000 7ff4877fe700 1 --2- 172.21.15.16:0/176246587 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff488133e30 0x7ff48809c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:55.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.361+0000 7ff4877fe700 1 -- 172.21.15.16:0/176246587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff47c0094a0 con 0x7ff488133420 2023-09-30T22:29:55.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.361+0000 7ff486ffd700 1 --2- 172.21.15.16:0/176246587 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff488133e30 0x7ff48809c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:29:55.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.361+0000 7ff4877fe700 1 --2- 172.21.15.16:0/176246587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff488133420 0x7ff48809c480 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff478009c40 tx=0x7ff478004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:55.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.361+0000 7ff46ffff700 1 -- 172.21.15.16:0/176246587 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4780036a0 con 0x7ff488133420 2023-09-30T22:29:55.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.361+0000 7ff46ffff700 1 -- 172.21.15.16:0/176246587 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff478003800 con 0x7ff488133420 2023-09-30T22:29:55.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.361+0000 7ff48e071700 1 -- 172.21.15.16:0/176246587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff478009550 con 0x7ff488133420 2023-09-30T22:29:55.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.361+0000 7ff46ffff700 1 -- 172.21.15.16:0/176246587 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff478003bb0 con 0x7ff488133420 2023-09-30T22:29:55.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.361+0000 7ff48e071700 1 -- 172.21.15.16:0/176246587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff48811fd20 con 0x7ff488133420 2023-09-30T22:29:55.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.362+0000 7ff46ffff700 1 -- 172.21.15.16:0/176246587 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7ff478021070 con 0x7ff488133420 2023-09-30T22:29:55.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.362+0000 7ff48e071700 1 -- 172.21.15.16:0/176246587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff488120080 con 0x7ff488133420 2023-09-30T22:29:55.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.362+0000 7ff46ffff700 1 -- 172.21.15.16:0/176246587 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7ff478041170 con 0x7ff488133420 2023-09-30T22:29:55.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.362+0000 7ff486ffd700 1 -- 172.21.15.16:0/176246587 >> v1:172.21.15.16:6816/126811 conn(0x7ff47402f510 legacy=0x7ff4740319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:55.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.367+0000 7ff46ffff700 1 -- 172.21.15.16:0/176246587 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff47800b3e0 con 0x7ff488133420 2023-09-30T22:29:55.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.374+0000 7f13b5d17700 1 Processor -- start 2023-09-30T22:29:55.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.374+0000 7f13b5d17700 1 -- start start 2023-09-30T22:29:55.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.375+0000 7f13b5d17700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f13b009d6f0 0x7f13b009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:55.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.375+0000 7f13b5d17700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f13b009e000 0x7f13b01339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:55.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.375+0000 7f13b5d17700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13b009b120 con 0x7f13b009e410 2023-09-30T22:29:55.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.375+0000 7f13b5d17700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13b006b460 con 0x7f13b009e000 2023-09-30T22:29:55.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.375+0000 7f13b5d17700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f13b009a7e0 con 0x7f13b009d6f0 2023-09-30T22:29:55.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.375+0000 7f13af7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f13b009d6f0 0x7f13b009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:55.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.375+0000 7f13af7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f13b009d6f0 0x7f13b009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:52242/0 (socket says 172.21.15.16:52242) 2023-09-30T22:29:55.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.375+0000 7f13af7fe700 1 -- 172.21.15.16:0/2992636022 learned_addr learned my addr 172.21.15.16:0/2992636022 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:55.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.375+0000 7f13aeffd700 1 --2- 172.21.15.16:0/2992636022 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f13b009e000 0x7f13b01339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:55.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.376+0000 7f13aeffd700 1 -- 172.21.15.16:0/2992636022 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f13b009d6f0 msgr2=0x7f13b009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:55.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.376+0000 7f13aeffd700 1 --2- 172.21.15.16:0/2992636022 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f13b009d6f0 0x7f13b009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:55.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.376+0000 7f13aeffd700 1 -- 172.21.15.16:0/2992636022 >> v1:172.21.15.16:6789/0 conn(0x7f13b009e410 legacy=0x7f13b0133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:55.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.376+0000 7f13aeffd700 1 -- 172.21.15.16:0/2992636022 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13b009a500 con 0x7f13b009e000 2023-09-30T22:29:55.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.376+0000 7f13af7fe700 1 --2- 172.21.15.16:0/2992636022 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f13b009d6f0 0x7f13b009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:55.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.376+0000 7f13aeffd700 1 --2- 172.21.15.16:0/2992636022 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f13b009e000 0x7f13b01339f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f13a4009870 tx=0x7f13a4009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d678fe97e957c67 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:55.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.376+0000 7f13ae7fc700 1 -- 172.21.15.16:0/2992636022 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a4011830 con 0x7f13b009e000 2023-09-30T22:29:55.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.376+0000 7f13ae7fc700 1 -- 172.21.15.16:0/2992636022 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13a4011990 con 0x7f13b009e000 2023-09-30T22:29:55.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.376+0000 7f13ae7fc700 1 -- 172.21.15.16:0/2992636022 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a4011d20 con 0x7f13b009e000 2023-09-30T22:29:55.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.377+0000 7f13b5d17700 1 -- 172.21.15.16:0/2992636022 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f13b009e000 msgr2=0x7f13b01339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:55.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.377+0000 7f13b5d17700 1 --2- 172.21.15.16:0/2992636022 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f13b009e000 0x7f13b01339f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f13a4009870 tx=0x7f13a4009b80 comp rx=0 tx=0).stop 2023-09-30T22:29:55.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.377+0000 7f13b5d17700 1 -- 172.21.15.16:0/2992636022 shutdown_connections 2023-09-30T22:29:55.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.377+0000 7f13b5d17700 1 --2- 172.21.15.16:0/2992636022 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f13b009d6f0 0x7f13b009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:55.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.377+0000 7f13b5d17700 1 --2- 172.21.15.16:0/2992636022 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f13b009e000 0x7f13b01339f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:55.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.377+0000 7f13b5d17700 1 -- 172.21.15.16:0/2992636022 >> 172.21.15.16:0/2992636022 conn(0x7f13b0066f50 msgr2=0x7f13b009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:55.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.377+0000 7f13b5d17700 1 -- 172.21.15.16:0/2992636022 shutdown_connections 2023-09-30T22:29:55.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.377+0000 7f13b5d17700 1 -- 172.21.15.16:0/2992636022 wait complete. 2023-09-30T22:29:55.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.378+0000 7f13b5d17700 1 Processor -- start 2023-09-30T22:29:55.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.379+0000 7f13b5d17700 1 -- start start 2023-09-30T22:29:55.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.379+0000 7f13b5d17700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f13b009d6f0 0x7f13b0122540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:55.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.379+0000 7f13b5d17700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f13b009e000 0x7f13b0122a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:55.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.380+0000 7f13b5d17700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13b009e410 0x7f13b0122fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:55.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.380+0000 7f13b5d17700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b0126060 con 0x7f13b009e410 2023-09-30T22:29:55.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.380+0000 7f13b5d17700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13b009c710 con 0x7f13b009e000 2023-09-30T22:29:55.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.380+0000 7f13b5d17700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f13b0124150 con 0x7f13b009d6f0 2023-09-30T22:29:55.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.380+0000 7f13aeffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f13b009e000 0x7f13b0122a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:55.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.380+0000 7f13aeffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f13b009e000 0x7f13b0122a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:49696/0 (socket says 172.21.15.16:49696) 2023-09-30T22:29:55.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.380+0000 7f13aeffd700 1 -- 172.21.15.16:0/2752414142 learned_addr learned my addr 172.21.15.16:0/2752414142 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:55.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.380+0000 7f13affff700 1 --2- 172.21.15.16:0/2752414142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13b009e410 0x7f13b0122fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:55.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.380+0000 7f13af7fe700 1 --2- 172.21.15.16:0/2752414142 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f13b009d6f0 0x7f13b0122540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:55.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.380+0000 7f13aeffd700 1 -- 172.21.15.16:0/2752414142 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f13b009d6f0 msgr2=0x7f13b0122540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:55.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.380+0000 7f13aeffd700 1 --2- 172.21.15.16:0/2752414142 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f13b009d6f0 0x7f13b0122540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:55.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.380+0000 7f13aeffd700 1 -- 172.21.15.16:0/2752414142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13b009e410 msgr2=0x7f13b0122fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:55.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.380+0000 7f13aeffd700 1 --2- 172.21.15.16:0/2752414142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13b009e410 0x7f13b0122fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:55.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.380+0000 7f13aeffd700 1 -- 172.21.15.16:0/2752414142 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a0009550 con 0x7f13b009e000 2023-09-30T22:29:55.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.380+0000 7f13affff700 1 --2- 172.21.15.16:0/2752414142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13b009e410 0x7f13b0122fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:55.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.380+0000 7f13aeffd700 1 --2- 172.21.15.16:0/2752414142 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f13b009e000 0x7f13b0122a80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f13a40090b0 tx=0x7f13a4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:55.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.381+0000 7f1397fff700 1 -- 172.21.15.16:0/2752414142 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a4004510 con 0x7f13b009e000 2023-09-30T22:29:55.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.381+0000 7f1397fff700 1 -- 172.21.15.16:0/2752414142 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13a401f070 con 0x7f13b009e000 2023-09-30T22:29:55.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.381+0000 7f1397fff700 1 -- 172.21.15.16:0/2752414142 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a40038a0 con 0x7f13b009e000 2023-09-30T22:29:55.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.381+0000 7f13b5d17700 1 -- 172.21.15.16:0/2752414142 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13a40094a0 con 0x7f13b009e000 2023-09-30T22:29:55.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.381+0000 7f13b5d17700 1 -- 172.21.15.16:0/2752414142 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13b01249d0 con 0x7f13b009e000 2023-09-30T22:29:55.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.382+0000 7f1397fff700 1 -- 172.21.15.16:0/2752414142 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f13a4003a00 con 0x7f13b009e000 2023-09-30T22:29:55.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.382+0000 7f1397fff700 1 -- 172.21.15.16:0/2752414142 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7f13a4021070 con 0x7f13b009e000 2023-09-30T22:29:55.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.382+0000 7f13af7fe700 1 -- 172.21.15.16:0/2752414142 >> v1:172.21.15.16:6816/126811 conn(0x7f139c02f510 legacy=0x7f139c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:55.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.382+0000 7f13b5d17700 1 -- 172.21.15.16:0/2752414142 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13b0125a20 con 0x7f13b009e000 2023-09-30T22:29:55.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.392+0000 7f1397fff700 1 -- 172.21.15.16:0/2752414142 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f13b0125a20 con 0x7f13b009e000 2023-09-30T22:29:55.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.518+0000 7fe706ffd700 1 -- 172.21.15.16:0/3860049337 >> v1:172.21.15.16:6816/126811 conn(0x7fe6f402f510 legacy=0x7fe6f40319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:55.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.534+0000 7f8bc9c62700 1 -- 172.21.15.16:0/907520901 >> v1:172.21.15.16:6816/126811 conn(0x7f8ba802f510 legacy=0x7f8ba80319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:55.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.543+0000 7fe70e07a700 1 -- 172.21.15.16:0/3860049337 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe70809ffe0 con 0x7fe70809e930 2023-09-30T22:29:55.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.543+0000 7fe6effff700 1 -- 172.21.15.16:0/3860049337 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe6f801b3b0 con 0x7fe70809e930 2023-09-30T22:29:55.544 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:29:55.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.546+0000 7fe70e07a700 1 -- 172.21.15.16:0/3860049337 >> v1:172.21.15.16:6816/126811 conn(0x7fe6f402f510 legacy=0x7fe6f40319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:55.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.546+0000 7fe70e07a700 1 -- 172.21.15.16:0/3860049337 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe70809e930 msgr2=0x7fe70811d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:55.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.546+0000 7fe70e07a700 1 --2- 172.21.15.16:0/3860049337 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe70809e930 0x7fe70811d2e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe6f800ebe0 tx=0x7fe6f800eef0 comp rx=0 tx=0).stop 2023-09-30T22:29:55.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.547+0000 7fe70e07a700 1 -- 172.21.15.16:0/3860049337 shutdown_connections 2023-09-30T22:29:55.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.547+0000 7fe70e07a700 1 --2- 172.21.15.16:0/3860049337 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe70809e930 0x7fe70811d2e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:55.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.547+0000 7fe70e07a700 1 --2- 172.21.15.16:0/3860049337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe70809f650 0x7fe70811dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:55.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.547+0000 7fe70e07a700 1 --2- 172.21.15.16:0/3860049337 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe70809f240 0x7fe70811d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:55.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.547+0000 7fe70e07a700 1 -- 172.21.15.16:0/3860049337 >> 172.21.15.16:0/3860049337 conn(0x7fe7080a4210 msgr2=0x7fe708131c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:55.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.547+0000 7fe70e07a700 1 -- 172.21.15.16:0/3860049337 shutdown_connections 2023-09-30T22:29:55.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.547+0000 7fe70e07a700 1 -- 172.21.15.16:0/3860049337 wait complete. 2023-09-30T22:29:55.561 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:29:55.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.563+0000 7ff486ffd700 1 -- 172.21.15.16:0/176246587 >> v1:172.21.15.16:6816/126811 conn(0x7ff47402f510 legacy=0x7ff4740319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:55.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.567+0000 7f8bcc6c7700 1 -- 172.21.15.16:0/907520901 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8bc409fe10 con 0x7f8bc4133420 2023-09-30T22:29:55.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.568+0000 7f8bbaffd700 1 -- 172.21.15.16:0/907520901 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8bc0029070 con 0x7f8bc4133420 2023-09-30T22:29:55.568 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:29:55.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.571+0000 7f8bcc6c7700 1 -- 172.21.15.16:0/907520901 >> v1:172.21.15.16:6816/126811 conn(0x7f8ba802f510 legacy=0x7f8ba80319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:55.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.571+0000 7f8bcc6c7700 1 -- 172.21.15.16:0/907520901 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8bc4133420 msgr2=0x7f8bc409c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:55.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.571+0000 7f8bcc6c7700 1 --2- 172.21.15.16:0/907520901 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8bc4133420 0x7f8bc409c8c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0000c00 tx=0x7f8bc0004750 comp rx=0 tx=0).stop 2023-09-30T22:29:55.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.571+0000 7f8bcc6c7700 1 -- 172.21.15.16:0/907520901 shutdown_connections 2023-09-30T22:29:55.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.571+0000 7f8bcc6c7700 1 --2- 172.21.15.16:0/907520901 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8bc4133420 0x7f8bc409c8c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:55.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.571+0000 7f8bcc6c7700 1 --2- 172.21.15.16:0/907520901 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8bc4134260 0x7f8bc409d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:55.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.571+0000 7f8bcc6c7700 1 --2- 172.21.15.16:0/907520901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bc4133e30 0x7f8bc409ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:55.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.571+0000 7f8bcc6c7700 1 -- 172.21.15.16:0/907520901 >> 172.21.15.16:0/907520901 conn(0x7f8bc40a4210 msgr2=0x7f8bc40a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:55.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.571+0000 7f8bcc6c7700 1 -- 172.21.15.16:0/907520901 shutdown_connections 2023-09-30T22:29:55.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.571+0000 7f8bcc6c7700 1 -- 172.21.15.16:0/907520901 wait complete. 2023-09-30T22:29:55.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.583+0000 7f13af7fe700 1 -- 172.21.15.16:0/2752414142 >> v1:172.21.15.16:6816/126811 conn(0x7f139c02f510 legacy=0x7f139c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:55.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.583+0000 7ff48e071700 1 -- 172.21.15.16:0/176246587 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff488074ea0 con 0x7ff488133420 2023-09-30T22:29:55.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.583+0000 7ff46ffff700 1 -- 172.21.15.16:0/176246587 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff478015020 con 0x7ff488133420 2023-09-30T22:29:55.584 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:29:55.586 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:29:55.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.586+0000 7ff48e071700 1 -- 172.21.15.16:0/176246587 >> v1:172.21.15.16:6816/126811 conn(0x7ff47402f510 legacy=0x7ff4740319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:55.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.586+0000 7ff48e071700 1 -- 172.21.15.16:0/176246587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff488133420 msgr2=0x7ff48809c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:55.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.586+0000 7ff48e071700 1 --2- 172.21.15.16:0/176246587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff488133420 0x7ff48809c480 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff478009c40 tx=0x7ff478004750 comp rx=0 tx=0).stop 2023-09-30T22:29:55.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.586+0000 7ff48e071700 1 -- 172.21.15.16:0/176246587 shutdown_connections 2023-09-30T22:29:55.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.586+0000 7ff48e071700 1 --2- 172.21.15.16:0/176246587 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff488134260 0x7ff48809cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:55.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.586+0000 7ff48e071700 1 --2- 172.21.15.16:0/176246587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff488133420 0x7ff48809c480 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:55.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.586+0000 7ff48e071700 1 --2- 172.21.15.16:0/176246587 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff488133e30 0x7ff48809c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:55.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.586+0000 7ff48e071700 1 -- 172.21.15.16:0/176246587 >> 172.21.15.16:0/176246587 conn(0x7ff4880a4210 msgr2=0x7ff4880a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:55.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.586+0000 7ff48e071700 1 -- 172.21.15.16:0/176246587 shutdown_connections 2023-09-30T22:29:55.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.586+0000 7ff48e071700 1 -- 172.21.15.16:0/176246587 wait complete. 2023-09-30T22:29:55.601 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:29:55.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.605+0000 7f13b5d17700 1 -- 172.21.15.16:0/2752414142 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f13b0125a20 con 0x7f13b009e000 2023-09-30T22:29:55.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.606+0000 7f1397fff700 1 -- 172.21.15.16:0/2752414142 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f13b0125a20 con 0x7f13b009e000 2023-09-30T22:29:55.606 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:29:55.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.608+0000 7f13b5d17700 1 -- 172.21.15.16:0/2752414142 >> v1:172.21.15.16:6816/126811 conn(0x7f139c02f510 legacy=0x7f139c0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:55.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.608+0000 7f13b5d17700 1 -- 172.21.15.16:0/2752414142 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f13b009e000 msgr2=0x7f13b0122a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:55.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.608+0000 7f13b5d17700 1 --2- 172.21.15.16:0/2752414142 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f13b009e000 0x7f13b0122a80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f13a40090b0 tx=0x7f13a4004750 comp rx=0 tx=0).stop 2023-09-30T22:29:55.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.608+0000 7f13b5d17700 1 -- 172.21.15.16:0/2752414142 shutdown_connections 2023-09-30T22:29:55.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.608+0000 7f13b5d17700 1 --2- 172.21.15.16:0/2752414142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13b009e410 0x7f13b0122fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:55.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.608+0000 7f13b5d17700 1 --2- 172.21.15.16:0/2752414142 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f13b009d6f0 0x7f13b0122540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:55.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.608+0000 7f13b5d17700 1 --2- 172.21.15.16:0/2752414142 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f13b009e000 0x7f13b0122a80 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:55.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.608+0000 7f13b5d17700 1 -- 172.21.15.16:0/2752414142 >> 172.21.15.16:0/2752414142 conn(0x7f13b0066f50 msgr2=0x7f13b006f000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:55.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.608+0000 7f13b5d17700 1 -- 172.21.15.16:0/2752414142 shutdown_connections 2023-09-30T22:29:55.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:55.608+0000 7f13b5d17700 1 -- 172.21.15.16:0/2752414142 wait complete. 2023-09-30T22:29:55.622 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:29:56.562 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:29:56.587 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:29:56.602 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:29:56.623 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:29:56.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.718+0000 7f394f410700 1 Processor -- start 2023-09-30T22:29:56.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.719+0000 7f394f410700 1 -- start start 2023-09-30T22:29:56.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.719+0000 7f394f410700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f394809d6f0 0x7f394809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:56.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.719+0000 7f394f410700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f394809e000 0x7f39481339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:56.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.719+0000 7f394f410700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394809b120 con 0x7f394809e410 2023-09-30T22:29:56.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.719+0000 7f394f410700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f394806b460 con 0x7f394809d6f0 2023-09-30T22:29:56.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.719+0000 7f394f410700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f394809a7e0 con 0x7f394809e000 2023-09-30T22:29:56.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.719+0000 7f394d1ac700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f394809d6f0 0x7f394809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:56.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.719+0000 7f394d1ac700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f394809d6f0 0x7f394809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:49698/0 (socket says 172.21.15.16:49698) 2023-09-30T22:29:56.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.719+0000 7f394d1ac700 1 -- 172.21.15.16:0/4211304430 learned_addr learned my addr 172.21.15.16:0/4211304430 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:56.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.720+0000 7f394c9ab700 1 --2- 172.21.15.16:0/4211304430 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f394809e000 0x7f39481339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:56.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.720+0000 7f394d1ac700 1 -- 172.21.15.16:0/4211304430 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f394809e000 msgr2=0x7f39481339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:56.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.720+0000 7f394d1ac700 1 --2- 172.21.15.16:0/4211304430 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f394809e000 0x7f39481339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.720+0000 7f394d1ac700 1 -- 172.21.15.16:0/4211304430 >> v1:172.21.15.16:6789/0 conn(0x7f394809e410 legacy=0x7f3948133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:56.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.720+0000 7f394d1ac700 1 -- 172.21.15.16:0/4211304430 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f394809a500 con 0x7f394809d6f0 2023-09-30T22:29:56.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.720+0000 7f394d1ac700 1 --2- 172.21.15.16:0/4211304430 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f394809d6f0 0x7f394809dac0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3944009920 tx=0x7f3944009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6e4d913b93f68b3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:56.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.720+0000 7f393ffff700 1 -- 172.21.15.16:0/4211304430 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394400ba60 con 0x7f394809d6f0 2023-09-30T22:29:56.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.720+0000 7f393ffff700 1 -- 172.21.15.16:0/4211304430 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f394400bbc0 con 0x7f394809d6f0 2023-09-30T22:29:56.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.720+0000 7f393ffff700 1 -- 172.21.15.16:0/4211304430 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3944009e70 con 0x7f394809d6f0 2023-09-30T22:29:56.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.721+0000 7f394f410700 1 -- 172.21.15.16:0/4211304430 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f394809d6f0 msgr2=0x7f394809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:56.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.721+0000 7f394f410700 1 --2- 172.21.15.16:0/4211304430 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f394809d6f0 0x7f394809dac0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3944009920 tx=0x7f3944009c30 comp rx=0 tx=0).stop 2023-09-30T22:29:56.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.721+0000 7f394f410700 1 -- 172.21.15.16:0/4211304430 shutdown_connections 2023-09-30T22:29:56.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.721+0000 7f394f410700 1 --2- 172.21.15.16:0/4211304430 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f394809d6f0 0x7f394809dac0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.721+0000 7f394f410700 1 --2- 172.21.15.16:0/4211304430 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f394809e000 0x7f39481339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.721+0000 7f394f410700 1 -- 172.21.15.16:0/4211304430 >> 172.21.15.16:0/4211304430 conn(0x7f3948066f50 msgr2=0x7f394809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:56.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.722+0000 7f394f410700 1 -- 172.21.15.16:0/4211304430 shutdown_connections 2023-09-30T22:29:56.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.722+0000 7f394f410700 1 -- 172.21.15.16:0/4211304430 wait complete. 2023-09-30T22:29:56.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.723+0000 7f394f410700 1 Processor -- start 2023-09-30T22:29:56.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.723+0000 7f394f410700 1 -- start start 2023-09-30T22:29:56.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.723+0000 7f394f410700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f394809d6f0 0x7f3948122490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:56.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.723+0000 7f394f410700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f394809e000 0x7f39481229d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:56.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.723+0000 7f394f410700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f394809e410 0x7f3948122f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:56.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.723+0000 7f394f410700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3948125a10 con 0x7f394809e410 2023-09-30T22:29:56.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.723+0000 7f394f410700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39481238f0 con 0x7f394809d6f0 2023-09-30T22:29:56.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.723+0000 7f394f410700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3948124900 con 0x7f394809e000 2023-09-30T22:29:56.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.723+0000 7f394d1ac700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f394809d6f0 0x7f3948122490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:56.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.723+0000 7f394d1ac700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f394809d6f0 0x7f3948122490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:49712/0 (socket says 172.21.15.16:49712) 2023-09-30T22:29:56.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.723+0000 7f394d1ac700 1 -- 172.21.15.16:0/3488065112 learned_addr learned my addr 172.21.15.16:0/3488065112 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:56.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.723+0000 7f394d9ad700 1 --2- 172.21.15.16:0/3488065112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f394809e410 0x7f3948122f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:56.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.724+0000 7f394d1ac700 1 -- 172.21.15.16:0/3488065112 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f394809e000 msgr2=0x7f39481229d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:56.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.724+0000 7f394d1ac700 1 --2- 172.21.15.16:0/3488065112 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f394809e000 0x7f39481229d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.724+0000 7f394d1ac700 1 -- 172.21.15.16:0/3488065112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f394809e410 msgr2=0x7f3948122f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:56.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.724+0000 7f394d1ac700 1 --2- 172.21.15.16:0/3488065112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f394809e410 0x7f3948122f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.724+0000 7f394d1ac700 1 -- 172.21.15.16:0/3488065112 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3944009550 con 0x7f394809d6f0 2023-09-30T22:29:56.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.724+0000 7f394d1ac700 1 --2- 172.21.15.16:0/3488065112 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f394809d6f0 0x7f3948122490 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3944009c70 tx=0x7f3944004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:56.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.724+0000 7f393dffb700 1 -- 172.21.15.16:0/3488065112 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3944004380 con 0x7f394809d6f0 2023-09-30T22:29:56.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.724+0000 7f393dffb700 1 -- 172.21.15.16:0/3488065112 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f394401f070 con 0x7f394809d6f0 2023-09-30T22:29:56.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.724+0000 7f394f410700 1 -- 172.21.15.16:0/3488065112 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39480a2be0 con 0x7f394809d6f0 2023-09-30T22:29:56.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.724+0000 7f393dffb700 1 -- 172.21.15.16:0/3488065112 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39440036a0 con 0x7f394809d6f0 2023-09-30T22:29:56.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.724+0000 7f394f410700 1 -- 172.21.15.16:0/3488065112 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3948126740 con 0x7f394809d6f0 2023-09-30T22:29:56.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.725+0000 7f393dffb700 1 -- 172.21.15.16:0/3488065112 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f3944021030 con 0x7f394809d6f0 2023-09-30T22:29:56.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.726+0000 7f394f410700 1 -- 172.21.15.16:0/3488065112 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3948125360 con 0x7f394809d6f0 2023-09-30T22:29:56.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.726+0000 7f393dffb700 1 -- 172.21.15.16:0/3488065112 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7f39440410c0 con 0x7f394809d6f0 2023-09-30T22:29:56.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.726+0000 7f394c9ab700 1 -- 172.21.15.16:0/3488065112 >> v1:172.21.15.16:6816/126811 conn(0x7f392c02f510 legacy=0x7f392c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:56.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.735+0000 7f393dffb700 1 -- 172.21.15.16:0/3488065112 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3944023990 con 0x7f394809d6f0 2023-09-30T22:29:56.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.756+0000 7f9a70c1a700 1 Processor -- start 2023-09-30T22:29:56.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.756+0000 7f9a70c1a700 1 -- start start 2023-09-30T22:29:56.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.757+0000 7f9a70c1a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9a6c06e570 0x7f9a6c06e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:56.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.757+0000 7f9a70c1a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9a6c09eca0 0x7f9a6c06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:56.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.757+0000 7f9a70c1a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9a6c134610 0x7f9a6c134a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:56.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.757+0000 7f9a70c1a700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a6c070ee0 con 0x7f9a6c09eca0 2023-09-30T22:29:56.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.757+0000 7f9a70c1a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6c071480 con 0x7f9a6c134610 2023-09-30T22:29:56.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.757+0000 7f9a70c1a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6c06f3c0 con 0x7f9a6c06e570 2023-09-30T22:29:56.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.757+0000 7f9a69d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9a6c09eca0 0x7f9a6c06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:56.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.757+0000 7f9a6a59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9a6c06e570 0x7f9a6c06e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:56.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.757+0000 7f9a69d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9a6c09eca0 0x7f9a6c06ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48286/0 (socket says 172.21.15.16:48286) 2023-09-30T22:29:56.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.757+0000 7f9a69d9b700 1 -- 172.21.15.16:0/3027468107 learned_addr learned my addr 172.21.15.16:0/3027468107 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:56.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.757+0000 7f9a6ad9d700 1 --2- 172.21.15.16:0/3027468107 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9a6c134610 0x7f9a6c134a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:56.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.758+0000 7f9a6a59c700 1 -- 172.21.15.16:0/3027468107 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9a6c134610 msgr2=0x7f9a6c134a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:56.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.758+0000 7f9a6a59c700 1 --2- 172.21.15.16:0/3027468107 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9a6c134610 0x7f9a6c134a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.758+0000 7f9a6a59c700 1 -- 172.21.15.16:0/3027468107 >> v2:172.21.15.16:3300/0 conn(0x7f9a6c09eca0 msgr2=0x7f9a6c06ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:56.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.758+0000 7f9a6a59c700 1 --2- 172.21.15.16:0/3027468107 >> v2:172.21.15.16:3300/0 conn(0x7f9a6c09eca0 0x7f9a6c06ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.758+0000 7f9a6a59c700 1 -- 172.21.15.16:0/3027468107 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a6c071b90 con 0x7f9a6c06e570 2023-09-30T22:29:56.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.758+0000 7f9a6a59c700 1 --2- 172.21.15.16:0/3027468107 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9a6c06e570 0x7f9a6c06e940 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c009840 tx=0x7f9a5c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=631ec9f86841f79a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:56.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.758+0000 7f9a6959a700 1 -- 172.21.15.16:0/3027468107 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a5c011830 con 0x7f9a6c06e570 2023-09-30T22:29:56.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.758+0000 7f9a6959a700 1 -- 172.21.15.16:0/3027468107 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a5c011990 con 0x7f9a6c06e570 2023-09-30T22:29:56.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.758+0000 7f9a6959a700 1 -- 172.21.15.16:0/3027468107 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a5c011d20 con 0x7f9a6c06e570 2023-09-30T22:29:56.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.759+0000 7f9a70c1a700 1 -- 172.21.15.16:0/3027468107 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9a6c06e570 msgr2=0x7f9a6c06e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:56.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.759+0000 7f9a70c1a700 1 --2- 172.21.15.16:0/3027468107 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9a6c06e570 0x7f9a6c06e940 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c009840 tx=0x7f9a5c009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:56.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.759+0000 7f9a70c1a700 1 -- 172.21.15.16:0/3027468107 shutdown_connections 2023-09-30T22:29:56.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.759+0000 7f9a70c1a700 1 --2- 172.21.15.16:0/3027468107 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9a6c134610 0x7f9a6c134a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.759+0000 7f9a70c1a700 1 --2- 172.21.15.16:0/3027468107 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9a6c06e570 0x7f9a6c06e940 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.759+0000 7f9a70c1a700 1 --2- 172.21.15.16:0/3027468107 >> v2:172.21.15.16:3300/0 conn(0x7f9a6c09eca0 0x7f9a6c06ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.759+0000 7f9a70c1a700 1 -- 172.21.15.16:0/3027468107 >> 172.21.15.16:0/3027468107 conn(0x7f9a6c0a2aa0 msgr2=0x7f9a6c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:56.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.760+0000 7f9a70c1a700 1 -- 172.21.15.16:0/3027468107 shutdown_connections 2023-09-30T22:29:56.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.760+0000 7f9a70c1a700 1 -- 172.21.15.16:0/3027468107 wait complete. 2023-09-30T22:29:56.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.761+0000 7f9a70c1a700 1 Processor -- start 2023-09-30T22:29:56.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.761+0000 7f9a70c1a700 1 -- start start 2023-09-30T22:29:56.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.762+0000 7f9a70c1a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a6c06e570 0x7f9a6c1214e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:56.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.762+0000 7f9a70c1a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9a6c134610 0x7f9a6c121a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:56.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.762+0000 7f9a70c1a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9a6c121f60 0x7f9a6c1d2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:56.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.762+0000 7f9a70c1a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6c1228b0 con 0x7f9a6c06e570 2023-09-30T22:29:56.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.762+0000 7f9a70c1a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6c134330 con 0x7f9a6c134610 2023-09-30T22:29:56.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.762+0000 7f9a70c1a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6c123170 con 0x7f9a6c121f60 2023-09-30T22:29:56.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.762+0000 7f9a6ad9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9a6c121f60 0x7f9a6c1d2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:56.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.762+0000 7f9a6ad9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9a6c121f60 0x7f9a6c1d2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:52292/0 (socket says 172.21.15.16:52292) 2023-09-30T22:29:56.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.762+0000 7f9a6ad9d700 1 -- 172.21.15.16:0/291381487 learned_addr learned my addr 172.21.15.16:0/291381487 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:56.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.763+0000 7f9a69d9b700 1 --2- 172.21.15.16:0/291381487 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9a6c134610 0x7f9a6c121a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:56.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.763+0000 7f9a6a59c700 1 --2- 172.21.15.16:0/291381487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a6c06e570 0x7f9a6c1214e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:56.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.763+0000 7f9a6ad9d700 1 -- 172.21.15.16:0/291381487 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9a6c134610 msgr2=0x7f9a6c121a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:56.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.763+0000 7f9a6ad9d700 1 --2- 172.21.15.16:0/291381487 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9a6c134610 0x7f9a6c121a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.763+0000 7f9a6ad9d700 1 -- 172.21.15.16:0/291381487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a6c06e570 msgr2=0x7f9a6c1214e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:56.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.763+0000 7f9a6ad9d700 1 --2- 172.21.15.16:0/291381487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a6c06e570 0x7f9a6c1214e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.763+0000 7f9a6ad9d700 1 -- 172.21.15.16:0/291381487 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a5c0094a0 con 0x7f9a6c121f60 2023-09-30T22:29:56.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.763+0000 7f9a6ad9d700 1 --2- 172.21.15.16:0/291381487 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9a6c121f60 0x7f9a6c1d2d90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9a64009dd0 tx=0x7f9a6400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:56.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.763+0000 7fb52b16d700 1 Processor -- start 2023-09-30T22:29:56.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.763+0000 7f9a5affd700 1 -- 172.21.15.16:0/291381487 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a64009c40 con 0x7f9a6c121f60 2023-09-30T22:29:56.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.763+0000 7f9a5affd700 1 -- 172.21.15.16:0/291381487 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a6400f040 con 0x7f9a6c121f60 2023-09-30T22:29:56.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.764+0000 7f9a70c1a700 1 -- 172.21.15.16:0/291381487 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a6c123da0 con 0x7f9a6c121f60 2023-09-30T22:29:56.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.764+0000 7fb52b16d700 1 -- start start 2023-09-30T22:29:56.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.764+0000 7f9a70c1a700 1 -- 172.21.15.16:0/291381487 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a6c0a22a0 con 0x7f9a6c121f60 2023-09-30T22:29:56.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.764+0000 7fb52b16d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb5240a0a30 0x7fb5240a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:56.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.764+0000 7fb52b16d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb5240a1340 0x7fb524133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:56.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.764+0000 7fb52b16d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb52409edd0 con 0x7fb524133fa0 2023-09-30T22:29:56.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.764+0000 7fb52b16d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb52409e1f0 con 0x7fb5240a1340 2023-09-30T22:29:56.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.764+0000 7fb52b16d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb52409dc50 con 0x7fb5240a0a30 2023-09-30T22:29:56.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.764+0000 7fb52970a700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb524133fa0 0x7fb52413ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59270/0 (socket says 172.21.15.16:59270) 2023-09-30T22:29:56.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.764+0000 7fb52970a700 1 -- 172.21.15.16:0/956008128 learned_addr learned my addr 172.21.15.16:0/956008128 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:29:56.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.765+0000 7fb5237fe700 1 -- 172.21.15.16:0/956008128 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3179469863 0 0) 0x7fb52409edd0 con 0x7fb524133fa0 2023-09-30T22:29:56.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.765+0000 7fb5237fe700 1 -- 172.21.15.16:0/956008128 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5140035b0 con 0x7fb524133fa0 2023-09-30T22:29:56.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.765+0000 7f9a70c1a700 1 -- 172.21.15.16:0/291381487 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a6c09b1e0 con 0x7f9a6c121f60 2023-09-30T22:29:56.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.765+0000 7fb523fff700 1 --2- 172.21.15.16:0/956008128 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb5240a1340 0x7fb524133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:56.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.765+0000 7fb528f09700 1 --2- 172.21.15.16:0/956008128 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb5240a0a30 0x7fb5240a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:56.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.765+0000 7fb5237fe700 1 -- 172.21.15.16:0/956008128 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 832038028 0 0) 0x7fb5140035b0 con 0x7fb524133fa0 2023-09-30T22:29:56.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.765+0000 7fb5237fe700 1 -- 172.21.15.16:0/956008128 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb5240a0a30 msgr2=0x7fb5240a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:56.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.765+0000 7fb5237fe700 1 --2- 172.21.15.16:0/956008128 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb5240a0a30 0x7fb5240a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.765+0000 7fb5237fe700 1 -- 172.21.15.16:0/956008128 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb5240a1340 msgr2=0x7fb524133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:56.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.765+0000 7fb5237fe700 1 --2- 172.21.15.16:0/956008128 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb5240a1340 0x7fb524133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.765+0000 7fb5237fe700 1 -- 172.21.15.16:0/956008128 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5240a1fe0 con 0x7fb524133fa0 2023-09-30T22:29:56.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.765+0000 7fb5237fe700 1 -- 172.21.15.16:0/956008128 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1074163509 0 0) 0x7fb51c0019f0 con 0x7fb524133fa0 2023-09-30T22:29:56.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.765+0000 7fb523fff700 1 --2- 172.21.15.16:0/956008128 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb5240a1340 0x7fb524133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:29:56.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.765+0000 7fb5237fe700 1 -- 172.21.15.16:0/956008128 >> v1:172.21.15.16:6789/0 conn(0x7fb524133fa0 legacy=0x7fb52413ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:56.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.765+0000 7fb5237fe700 1 --2- 172.21.15.16:0/956008128 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb5240a1340 0x7fb514003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:56.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.765+0000 7fb5237fe700 1 --2- 172.21.15.16:0/956008128 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb5240a0a30 0x7fb514004050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:56.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.765+0000 7fb5237fe700 1 --2- 172.21.15.16:0/956008128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb514004850 0x7fb514004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:56.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.765+0000 7fb5237fe700 1 -- 172.21.15.16:0/956008128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52409e1f0 con 0x7fb514004850 2023-09-30T22:29:56.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.765+0000 7fb5237fe700 1 -- 172.21.15.16:0/956008128 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb52409dc50 con 0x7fb5240a1340 2023-09-30T22:29:56.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.765+0000 7fb5237fe700 1 -- 172.21.15.16:0/956008128 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb514005430 con 0x7fb5240a0a30 2023-09-30T22:29:56.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.766+0000 7fb52970a700 1 --2- 172.21.15.16:0/956008128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb514004850 0x7fb514004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:56.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.766+0000 7fb528f09700 1 --2- 172.21.15.16:0/956008128 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb5240a1340 0x7fb514003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:56.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.766+0000 7fb52970a700 1 -- 172.21.15.16:0/956008128 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb5240a0a30 msgr2=0x7fb514004050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:56.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.766+0000 7fb52970a700 1 --2- 172.21.15.16:0/956008128 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb5240a0a30 0x7fb514004050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.766+0000 7fb52970a700 1 -- 172.21.15.16:0/956008128 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb5240a1340 msgr2=0x7fb514003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:56.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.766+0000 7fb52970a700 1 --2- 172.21.15.16:0/956008128 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb5240a1340 0x7fb514003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.766+0000 7fb52970a700 1 -- 172.21.15.16:0/956008128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5140035b0 con 0x7fb514004850 2023-09-30T22:29:56.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.766+0000 7fb52970a700 1 --2- 172.21.15.16:0/956008128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb514004850 0x7fb514004cc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb51c00a780 tx=0x7fb514005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62e6ae27a1ce3b9c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:56.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.766+0000 7fb5237fe700 1 -- 172.21.15.16:0/956008128 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51c0072d0 con 0x7fb514004850 2023-09-30T22:29:56.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.766+0000 7fb5237fe700 1 -- 172.21.15.16:0/956008128 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb51c007430 con 0x7fb514004850 2023-09-30T22:29:56.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.766+0000 7fb5237fe700 1 -- 172.21.15.16:0/956008128 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51c0077e0 con 0x7fb514004850 2023-09-30T22:29:56.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.767+0000 7fb52b16d700 1 -- 172.21.15.16:0/956008128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb514004850 msgr2=0x7fb514004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:56.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.767+0000 7fb52b16d700 1 --2- 172.21.15.16:0/956008128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb514004850 0x7fb514004cc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb51c00a780 tx=0x7fb514005880 comp rx=0 tx=0).stop 2023-09-30T22:29:56.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.767+0000 7fb52b16d700 1 -- 172.21.15.16:0/956008128 shutdown_connections 2023-09-30T22:29:56.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.767+0000 7fb52b16d700 1 --2- 172.21.15.16:0/956008128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb514004850 0x7fb514004cc0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.767+0000 7fb52b16d700 1 --2- 172.21.15.16:0/956008128 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb5240a0a30 0x7fb514004050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.767+0000 7fb52b16d700 1 --2- 172.21.15.16:0/956008128 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb5240a1340 0x7fb514003a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.767+0000 7fb52b16d700 1 -- 172.21.15.16:0/956008128 >> 172.21.15.16:0/956008128 conn(0x7fb5240a6550 msgr2=0x7fb5240a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:56.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.767+0000 7fb52b16d700 1 -- 172.21.15.16:0/956008128 shutdown_connections 2023-09-30T22:29:56.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.767+0000 7fb52b16d700 1 -- 172.21.15.16:0/956008128 wait complete. 2023-09-30T22:29:56.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.768+0000 7fb52b16d700 1 Processor -- start 2023-09-30T22:29:56.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.769+0000 7fb52b16d700 1 -- start start 2023-09-30T22:29:56.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.769+0000 7fb52b16d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb514004850 0x7fb52406da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:56.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.769+0000 7fb52b16d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb5240a0a30 0x7fb52406dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:56.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.769+0000 7fb52b16d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb5240a1340 0x7fb52406e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:56.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.769+0000 7fb52b16d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb524071300 con 0x7fb514004850 2023-09-30T22:29:56.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.769+0000 7fb52b16d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb52406f5f0 con 0x7fb5240a1340 2023-09-30T22:29:56.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.769+0000 7fb52b16d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb524070670 con 0x7fb5240a0a30 2023-09-30T22:29:56.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.769+0000 7fb52970a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb5240a1340 0x7fb52406e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:56.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.770+0000 7fb52970a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb5240a1340 0x7fb52406e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:49764/0 (socket says 172.21.15.16:49764) 2023-09-30T22:29:56.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.770+0000 7fb52970a700 1 -- 172.21.15.16:0/1622727508 learned_addr learned my addr 172.21.15.16:0/1622727508 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:56.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.770+0000 7fb528f09700 1 --2- 172.21.15.16:0/1622727508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb514004850 0x7fb52406da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:56.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.770+0000 7fb52970a700 1 -- 172.21.15.16:0/1622727508 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb5240a0a30 msgr2=0x7fb52406dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:56.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.770+0000 7fb52970a700 1 --2- 172.21.15.16:0/1622727508 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb5240a0a30 0x7fb52406dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.770+0000 7fb52970a700 1 -- 172.21.15.16:0/1622727508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb514004850 msgr2=0x7fb52406da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:56.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.770+0000 7fb52970a700 1 --2- 172.21.15.16:0/1622727508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb514004850 0x7fb52406da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.770+0000 7fb52970a700 1 -- 172.21.15.16:0/1622727508 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5240a25e0 con 0x7fb5240a1340 2023-09-30T22:29:56.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.770+0000 7fb52970a700 1 --2- 172.21.15.16:0/1622727508 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb5240a1340 0x7fb52406e510 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb51c00ad70 tx=0x7fb51c001360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:56.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.770+0000 7fb5217fa700 1 -- 172.21.15.16:0/1622727508 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51c01f630 con 0x7fb5240a1340 2023-09-30T22:29:56.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.770+0000 7fb5217fa700 1 -- 172.21.15.16:0/1622727508 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb51c007b90 con 0x7fb5240a1340 2023-09-30T22:29:56.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.770+0000 7fb5217fa700 1 -- 172.21.15.16:0/1622727508 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51c01fa10 con 0x7fb5240a1340 2023-09-30T22:29:56.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.770+0000 7fb52b16d700 1 -- 172.21.15.16:0/1622727508 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5240a0220 con 0x7fb5240a1340 2023-09-30T22:29:56.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.770+0000 7fb52b16d700 1 -- 172.21.15.16:0/1622727508 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb52409bd90 con 0x7fb5240a1340 2023-09-30T22:29:56.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.771+0000 7fb5217fa700 1 -- 172.21.15.16:0/1622727508 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fb51c01fb70 con 0x7fb5240a1340 2023-09-30T22:29:56.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.772+0000 7fb528f09700 1 -- 172.21.15.16:0/1622727508 >> v1:172.21.15.16:6816/126811 conn(0x7fb50802f510 legacy=0x7fb5080319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:56.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.772+0000 7fb5217fa700 1 -- 172.21.15.16:0/1622727508 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7fb51c0420f0 con 0x7fb5240a1340 2023-09-30T22:29:56.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.772+0000 7fb52b16d700 1 -- 172.21.15.16:0/1622727508 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb52406f820 con 0x7fb5240a1340 2023-09-30T22:29:56.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.775+0000 7f9a5affd700 1 -- 172.21.15.16:0/291381487 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6400ceb0 con 0x7f9a6c121f60 2023-09-30T22:29:56.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.775+0000 7f9a5affd700 1 -- 172.21.15.16:0/291381487 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f9a64014480 con 0x7f9a6c121f60 2023-09-30T22:29:56.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.775+0000 7f9a5affd700 1 -- 172.21.15.16:0/291381487 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7f9a64043220 con 0x7f9a6c121f60 2023-09-30T22:29:56.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.775+0000 7f9a6a59c700 1 -- 172.21.15.16:0/291381487 >> v1:172.21.15.16:6816/126811 conn(0x7f9a4c02f590 legacy=0x7f9a4c031a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:56.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.775+0000 7f9a5affd700 1 -- 172.21.15.16:0/291381487 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9a640434c0 con 0x7f9a6c121f60 2023-09-30T22:29:56.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.781+0000 7fb5217fa700 1 -- 172.21.15.16:0/1622727508 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb51c023080 con 0x7fb5240a1340 2023-09-30T22:29:56.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.809+0000 7fde88785700 1 Processor -- start 2023-09-30T22:29:56.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.809+0000 7fde88785700 1 -- start start 2023-09-30T22:29:56.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.810+0000 7fde88785700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fde8006a770 0x7fde8006ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:56.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.810+0000 7fde88785700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fde80131d10 0x7fde80139860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:56.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.810+0000 7fde88785700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde8009ece0 con 0x7fde8006b080 2023-09-30T22:29:56.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.810+0000 7fde88785700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde8009e760 con 0x7fde8006a770 2023-09-30T22:29:56.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.810+0000 7fde88785700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde8009e040 con 0x7fde80131d10 2023-09-30T22:29:56.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.810+0000 7fde85d20700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fde8006b080 0x7fde80131600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59278/0 (socket says 172.21.15.16:59278) 2023-09-30T22:29:56.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.810+0000 7fde85d20700 1 -- 172.21.15.16:0/167913267 learned_addr learned my addr 172.21.15.16:0/167913267 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:29:56.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.810+0000 7fde86d22700 1 --2- 172.21.15.16:0/167913267 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fde80131d10 0x7fde80139860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:56.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.810+0000 7fde86521700 1 --2- 172.21.15.16:0/167913267 >> v2:172.21.15.16:3300/0 conn(0x7fde8006a770 0x7fde8006ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:56.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.810+0000 7fde8551f700 1 -- 172.21.15.16:0/167913267 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4090424099 0 0) 0x7fde8009ece0 con 0x7fde8006b080 2023-09-30T22:29:56.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.810+0000 7fde8551f700 1 -- 172.21.15.16:0/167913267 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde6c0035b0 con 0x7fde8006b080 2023-09-30T22:29:56.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.810+0000 7fde86d22700 1 -- 172.21.15.16:0/167913267 >> v2:172.21.15.16:3300/0 conn(0x7fde8006a770 msgr2=0x7fde8006ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:56.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.810+0000 7fde86d22700 1 --2- 172.21.15.16:0/167913267 >> v2:172.21.15.16:3300/0 conn(0x7fde8006a770 0x7fde8006ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.811+0000 7fde86d22700 1 -- 172.21.15.16:0/167913267 >> v1:172.21.15.16:6789/0 conn(0x7fde8006b080 legacy=0x7fde80131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:56.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.811+0000 7fde86d22700 1 -- 172.21.15.16:0/167913267 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde8009f630 con 0x7fde80131d10 2023-09-30T22:29:56.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.811+0000 7fde86521700 1 --2- 172.21.15.16:0/167913267 >> v2:172.21.15.16:3300/0 conn(0x7fde8006a770 0x7fde8006ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:56.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.811+0000 7fde86d22700 1 --2- 172.21.15.16:0/167913267 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fde80131d10 0x7fde80139860 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fde78009d90 tx=0x7fde7800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=167bb3334c75801a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:56.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.811+0000 7fde8551f700 1 -- 172.21.15.16:0/167913267 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7800e070 con 0x7fde80131d10 2023-09-30T22:29:56.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.811+0000 7fde8551f700 1 -- 172.21.15.16:0/167913267 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde7800bb20 con 0x7fde80131d10 2023-09-30T22:29:56.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.811+0000 7fde8551f700 1 -- 172.21.15.16:0/167913267 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde78012430 con 0x7fde80131d10 2023-09-30T22:29:56.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.812+0000 7fde88785700 1 -- 172.21.15.16:0/167913267 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fde80131d10 msgr2=0x7fde80139860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:56.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.812+0000 7fde88785700 1 --2- 172.21.15.16:0/167913267 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fde80131d10 0x7fde80139860 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fde78009d90 tx=0x7fde7800b330 comp rx=0 tx=0).stop 2023-09-30T22:29:56.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.812+0000 7fde88785700 1 -- 172.21.15.16:0/167913267 shutdown_connections 2023-09-30T22:29:56.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.812+0000 7fde88785700 1 --2- 172.21.15.16:0/167913267 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fde80131d10 0x7fde80139860 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.812+0000 7fde88785700 1 --2- 172.21.15.16:0/167913267 >> v2:172.21.15.16:3300/0 conn(0x7fde8006a770 0x7fde8006ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.812+0000 7fde88785700 1 -- 172.21.15.16:0/167913267 >> 172.21.15.16:0/167913267 conn(0x7fde80066f50 msgr2=0x7fde8009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:56.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.812+0000 7fde88785700 1 -- 172.21.15.16:0/167913267 shutdown_connections 2023-09-30T22:29:56.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.812+0000 7fde88785700 1 -- 172.21.15.16:0/167913267 wait complete. 2023-09-30T22:29:56.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.813+0000 7fde88785700 1 Processor -- start 2023-09-30T22:29:56.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.813+0000 7fde88785700 1 -- start start 2023-09-30T22:29:56.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.814+0000 7fde88785700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fde8006a770 0x7fde80120150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:56.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.814+0000 7fde88785700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde8006b080 0x7fde80120690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:56.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.814+0000 7fde88785700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fde80131d10 0x7fde80120bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:56.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.814+0000 7fde88785700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde80071e60 con 0x7fde8006b080 2023-09-30T22:29:56.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.814+0000 7fde88785700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde80131110 con 0x7fde8006a770 2023-09-30T22:29:56.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.814+0000 7fde88785700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fde80121e70 con 0x7fde80131d10 2023-09-30T22:29:56.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.814+0000 7fde86521700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fde8006a770 0x7fde80120150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:56.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.814+0000 7fde86521700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fde8006a770 0x7fde80120150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:49782/0 (socket says 172.21.15.16:49782) 2023-09-30T22:29:56.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.814+0000 7fde86521700 1 -- 172.21.15.16:0/1828957544 learned_addr learned my addr 172.21.15.16:0/1828957544 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:56.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.814+0000 7fde86d22700 1 --2- 172.21.15.16:0/1828957544 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fde80131d10 0x7fde80120bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:56.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.814+0000 7fde85d20700 1 --2- 172.21.15.16:0/1828957544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde8006b080 0x7fde80120690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:56.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.814+0000 7fde86521700 1 -- 172.21.15.16:0/1828957544 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fde80131d10 msgr2=0x7fde80120bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:56.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.814+0000 7fde86521700 1 --2- 172.21.15.16:0/1828957544 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fde80131d10 0x7fde80120bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.814+0000 7fde86521700 1 -- 172.21.15.16:0/1828957544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde8006b080 msgr2=0x7fde80120690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:56.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.814+0000 7fde86521700 1 --2- 172.21.15.16:0/1828957544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde8006b080 0x7fde80120690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.814+0000 7fde86521700 1 -- 172.21.15.16:0/1828957544 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde7c0094a0 con 0x7fde8006a770 2023-09-30T22:29:56.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.815+0000 7fde86521700 1 --2- 172.21.15.16:0/1828957544 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fde8006a770 0x7fde80120150 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fde7c00ebe0 tx=0x7fde7c00eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:56.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.815+0000 7fde76ffd700 1 -- 172.21.15.16:0/1828957544 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7c00c850 con 0x7fde8006a770 2023-09-30T22:29:56.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.815+0000 7fde76ffd700 1 -- 172.21.15.16:0/1828957544 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde7c00c9b0 con 0x7fde8006a770 2023-09-30T22:29:56.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.815+0000 7fde88785700 1 -- 172.21.15.16:0/1828957544 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde78009a70 con 0x7fde8006a770 2023-09-30T22:29:56.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.815+0000 7fde76ffd700 1 -- 172.21.15.16:0/1828957544 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde7c00cd40 con 0x7fde8006a770 2023-09-30T22:29:56.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.815+0000 7fde88785700 1 -- 172.21.15.16:0/1828957544 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde80124400 con 0x7fde8006a770 2023-09-30T22:29:56.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.816+0000 7fde88785700 1 -- 172.21.15.16:0/1828957544 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde80074ea0 con 0x7fde8006a770 2023-09-30T22:29:56.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.816+0000 7fde76ffd700 1 -- 172.21.15.16:0/1828957544 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fde7c013470 con 0x7fde8006a770 2023-09-30T22:29:56.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.816+0000 7fde85d20700 1 -- 172.21.15.16:0/1828957544 >> v1:172.21.15.16:6816/126811 conn(0x7fde6402f510 legacy=0x7fde640319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:56.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.816+0000 7fde76ffd700 1 -- 172.21.15.16:0/1828957544 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7fde7c040040 con 0x7fde8006a770 2023-09-30T22:29:56.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.821+0000 7fde76ffd700 1 -- 172.21.15.16:0/1828957544 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fde7c013720 con 0x7fde8006a770 2023-09-30T22:29:56.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.927+0000 7f394c9ab700 1 -- 172.21.15.16:0/3488065112 >> v1:172.21.15.16:6816/126811 conn(0x7f392c02f510 legacy=0x7f392c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:56.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.958+0000 7f394f410700 1 -- 172.21.15.16:0/3488065112 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3948074ea0 con 0x7f394809d6f0 2023-09-30T22:29:56.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.959+0000 7f393dffb700 1 -- 172.21.15.16:0/3488065112 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f394401b430 con 0x7f394809d6f0 2023-09-30T22:29:56.961 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:29:56.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.961+0000 7f394f410700 1 -- 172.21.15.16:0/3488065112 >> v1:172.21.15.16:6816/126811 conn(0x7f392c02f510 legacy=0x7f392c0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:56.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.961+0000 7f394f410700 1 -- 172.21.15.16:0/3488065112 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f394809d6f0 msgr2=0x7f3948122490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:56.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.961+0000 7f394f410700 1 --2- 172.21.15.16:0/3488065112 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f394809d6f0 0x7f3948122490 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3944009c70 tx=0x7f3944004750 comp rx=0 tx=0).stop 2023-09-30T22:29:56.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.961+0000 7f394f410700 1 -- 172.21.15.16:0/3488065112 shutdown_connections 2023-09-30T22:29:56.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.961+0000 7f394f410700 1 --2- 172.21.15.16:0/3488065112 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f394809d6f0 0x7f3948122490 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.961+0000 7f394f410700 1 --2- 172.21.15.16:0/3488065112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f394809e410 0x7f3948122f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.961+0000 7f394f410700 1 --2- 172.21.15.16:0/3488065112 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f394809e000 0x7f39481229d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:56.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.961+0000 7f394f410700 1 -- 172.21.15.16:0/3488065112 >> 172.21.15.16:0/3488065112 conn(0x7f3948066f50 msgr2=0x7f39480a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:56.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.961+0000 7f394f410700 1 -- 172.21.15.16:0/3488065112 shutdown_connections 2023-09-30T22:29:56.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.961+0000 7f394f410700 1 -- 172.21.15.16:0/3488065112 wait complete. 2023-09-30T22:29:56.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.973+0000 7fb528f09700 1 -- 172.21.15.16:0/1622727508 >> v1:172.21.15.16:6816/126811 conn(0x7fb50802f510 legacy=0x7fb5080319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:56.975 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:29:56.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:56.976+0000 7f9a6a59c700 1 -- 172.21.15.16:0/291381487 >> v1:172.21.15.16:6816/126811 conn(0x7f9a4c02f590 legacy=0x7f9a4c031a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:57.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:57.017+0000 7fde85d20700 1 -- 172.21.15.16:0/1828957544 >> v1:172.21.15.16:6816/126811 conn(0x7fde6402f510 legacy=0x7fde640319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:57.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:57.028+0000 7fde88785700 1 -- 172.21.15.16:0/1828957544 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fde8006c360 con 0x7fde8006a770 2023-09-30T22:29:57.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:57.028+0000 7fde76ffd700 1 -- 172.21.15.16:0/1828957544 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fde7c021020 con 0x7fde8006a770 2023-09-30T22:29:57.029 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:29:57.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:57.030+0000 7fde88785700 1 -- 172.21.15.16:0/1828957544 >> v1:172.21.15.16:6816/126811 conn(0x7fde6402f510 legacy=0x7fde640319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:57.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:57.030+0000 7fde88785700 1 -- 172.21.15.16:0/1828957544 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fde8006a770 msgr2=0x7fde80120150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:57.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:57.030+0000 7fde88785700 1 --2- 172.21.15.16:0/1828957544 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fde8006a770 0x7fde80120150 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fde7c00ebe0 tx=0x7fde7c00eef0 comp rx=0 tx=0).stop 2023-09-30T22:29:57.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:57.030+0000 7fde88785700 1 -- 172.21.15.16:0/1828957544 shutdown_connections 2023-09-30T22:29:57.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:57.030+0000 7fde88785700 1 --2- 172.21.15.16:0/1828957544 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fde8006a770 0x7fde80120150 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:57.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:57.030+0000 7fde88785700 1 --2- 172.21.15.16:0/1828957544 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fde80131d10 0x7fde80120bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:57.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:57.030+0000 7fde88785700 1 --2- 172.21.15.16:0/1828957544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde8006b080 0x7fde80120690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:57.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:57.030+0000 7fde88785700 1 -- 172.21.15.16:0/1828957544 >> 172.21.15.16:0/1828957544 conn(0x7fde80066f50 msgr2=0x7fde8006feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:57.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:57.030+0000 7fde88785700 1 -- 172.21.15.16:0/1828957544 shutdown_connections 2023-09-30T22:29:57.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:57.030+0000 7fde88785700 1 -- 172.21.15.16:0/1828957544 wait complete. 2023-09-30T22:29:57.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:57.042+0000 7f9a70c1a700 1 -- 172.21.15.16:0/291381487 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9a6c0a1b20 con 0x7f9a6c121f60 2023-09-30T22:29:57.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:57.042+0000 7f9a5affd700 1 -- 172.21.15.16:0/291381487 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9a64018030 con 0x7f9a6c121f60 2023-09-30T22:29:57.043 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:29:57.045 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:29:57.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:57.044+0000 7f9a70c1a700 1 -- 172.21.15.16:0/291381487 >> v1:172.21.15.16:6816/126811 conn(0x7f9a4c02f590 legacy=0x7f9a4c031a40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:57.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:57.044+0000 7f9a70c1a700 1 -- 172.21.15.16:0/291381487 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9a6c121f60 msgr2=0x7f9a6c1d2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:57.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:57.044+0000 7f9a70c1a700 1 --2- 172.21.15.16:0/291381487 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9a6c121f60 0x7f9a6c1d2d90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9a64009dd0 tx=0x7f9a6400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:29:57.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:57.045+0000 7f9a70c1a700 1 -- 172.21.15.16:0/291381487 shutdown_connections 2023-09-30T22:29:57.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:57.045+0000 7f9a70c1a700 1 --2- 172.21.15.16:0/291381487 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9a6c121f60 0x7f9a6c1d2d90 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:57.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:57.045+0000 7f9a70c1a700 1 --2- 172.21.15.16:0/291381487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a6c06e570 0x7f9a6c1214e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:57.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:57.045+0000 7f9a70c1a700 1 --2- 172.21.15.16:0/291381487 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9a6c134610 0x7f9a6c121a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:57.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:57.045+0000 7f9a70c1a700 1 -- 172.21.15.16:0/291381487 >> 172.21.15.16:0/291381487 conn(0x7f9a6c0a2aa0 msgr2=0x7f9a6c1bf2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:57.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:57.045+0000 7f9a70c1a700 1 -- 172.21.15.16:0/291381487 shutdown_connections 2023-09-30T22:29:57.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:57.045+0000 7f9a70c1a700 1 -- 172.21.15.16:0/291381487 wait complete. 2023-09-30T22:29:57.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:57.045+0000 7fb52b16d700 1 -- 172.21.15.16:0/1622727508 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb524074ea0 con 0x7fb5240a1340 2023-09-30T22:29:57.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:57.046+0000 7fb5217fa700 1 -- 172.21.15.16:0/1622727508 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb51c018070 con 0x7fb5240a1340 2023-09-30T22:29:57.047 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:29:57.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:57.048+0000 7fb52b16d700 1 -- 172.21.15.16:0/1622727508 >> v1:172.21.15.16:6816/126811 conn(0x7fb50802f510 legacy=0x7fb5080319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:57.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:57.048+0000 7fb52b16d700 1 -- 172.21.15.16:0/1622727508 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb5240a1340 msgr2=0x7fb52406e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:57.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:57.048+0000 7fb52b16d700 1 --2- 172.21.15.16:0/1622727508 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb5240a1340 0x7fb52406e510 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb51c00ad70 tx=0x7fb51c001360 comp rx=0 tx=0).stop 2023-09-30T22:29:57.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:57.048+0000 7fb52b16d700 1 -- 172.21.15.16:0/1622727508 shutdown_connections 2023-09-30T22:29:57.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:57.048+0000 7fb52b16d700 1 --2- 172.21.15.16:0/1622727508 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb5240a1340 0x7fb52406e510 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:57.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:57.048+0000 7fb52b16d700 1 --2- 172.21.15.16:0/1622727508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb514004850 0x7fb52406da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:57.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:57.048+0000 7fb52b16d700 1 --2- 172.21.15.16:0/1622727508 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb5240a0a30 0x7fb52406dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:57.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:57.048+0000 7fb52b16d700 1 -- 172.21.15.16:0/1622727508 >> 172.21.15.16:0/1622727508 conn(0x7fb5240a6550 msgr2=0x7fb52409a650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:57.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:57.048+0000 7fb52b16d700 1 -- 172.21.15.16:0/1622727508 shutdown_connections 2023-09-30T22:29:57.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:57.048+0000 7fb52b16d700 1 -- 172.21.15.16:0/1622727508 wait complete. 2023-09-30T22:29:57.061 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:29:57.063 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:29:57.977 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:29:58.045 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:29:58.061 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:29:58.064 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:29:58.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.130+0000 7f3954d89700 1 Processor -- start 2023-09-30T22:29:58.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.130+0000 7f3954d89700 1 -- start start 2023-09-30T22:29:58.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.131+0000 7f3954d89700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f39501316c0 0x7f3950131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:58.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.131+0000 7f3954d89700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f395009d9f0 0x7f395006a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:58.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.131+0000 7f3954d89700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f395006acb0 0x7f395006b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:58.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.131+0000 7f3954d89700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f395006b630 con 0x7f395009d9f0 2023-09-30T22:29:58.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.131+0000 7f3954d89700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f395009e130 con 0x7f395006acb0 2023-09-30T22:29:58.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.131+0000 7f3954d89700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f39501312b0 con 0x7f39501316c0 2023-09-30T22:29:58.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.131+0000 7f394e59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f39501316c0 0x7f3950131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:58.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.131+0000 7f394e59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f39501316c0 0x7f3950131a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:45016/0 (socket says 172.21.15.16:45016) 2023-09-30T22:29:58.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.131+0000 7f394e59c700 1 -- 172.21.15.16:0/3518170313 learned_addr learned my addr 172.21.15.16:0/3518170313 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:58.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.131+0000 7f394ed9d700 1 --2- 172.21.15.16:0/3518170313 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f395006acb0 0x7f395006b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:58.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.132+0000 7f394dd9b700 1 --2- 172.21.15.16:0/3518170313 >> v2:172.21.15.16:3300/0 conn(0x7f395009d9f0 0x7f395006a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:58.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.132+0000 7f394e59c700 1 -- 172.21.15.16:0/3518170313 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f395006acb0 msgr2=0x7f395006b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:58.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.132+0000 7f394e59c700 1 --2- 172.21.15.16:0/3518170313 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f395006acb0 0x7f395006b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:58.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.132+0000 7f394e59c700 1 -- 172.21.15.16:0/3518170313 >> v2:172.21.15.16:3300/0 conn(0x7f395009d9f0 msgr2=0x7f395006a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:58.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.132+0000 7f394e59c700 1 --2- 172.21.15.16:0/3518170313 >> v2:172.21.15.16:3300/0 conn(0x7f395009d9f0 0x7f395006a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:58.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.132+0000 7f394e59c700 1 -- 172.21.15.16:0/3518170313 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f395009f6e0 con 0x7f39501316c0 2023-09-30T22:29:58.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.132+0000 7f394dd9b700 1 --2- 172.21.15.16:0/3518170313 >> v2:172.21.15.16:3300/0 conn(0x7f395009d9f0 0x7f395006a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:29:58.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.132+0000 7f394e59c700 1 --2- 172.21.15.16:0/3518170313 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f39501316c0 0x7f3950131a90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3940009920 tx=0x7f3940009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f73a1956b1c6a7e1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:58.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.132+0000 7f394d59a700 1 -- 172.21.15.16:0/3518170313 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3940011830 con 0x7f39501316c0 2023-09-30T22:29:58.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.132+0000 7f394d59a700 1 -- 172.21.15.16:0/3518170313 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3940011990 con 0x7f39501316c0 2023-09-30T22:29:58.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.132+0000 7f394d59a700 1 -- 172.21.15.16:0/3518170313 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3940011d20 con 0x7f39501316c0 2023-09-30T22:29:58.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.133+0000 7f3954d89700 1 -- 172.21.15.16:0/3518170313 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f39501316c0 msgr2=0x7f3950131a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:58.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.133+0000 7f3954d89700 1 --2- 172.21.15.16:0/3518170313 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f39501316c0 0x7f3950131a90 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3940009920 tx=0x7f3940009c30 comp rx=0 tx=0).stop 2023-09-30T22:29:58.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.133+0000 7f3954d89700 1 -- 172.21.15.16:0/3518170313 shutdown_connections 2023-09-30T22:29:58.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.133+0000 7f3954d89700 1 --2- 172.21.15.16:0/3518170313 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f395006acb0 0x7f395006b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:58.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.133+0000 7f3954d89700 1 --2- 172.21.15.16:0/3518170313 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f39501316c0 0x7f3950131a90 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:58.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.133+0000 7f3954d89700 1 --2- 172.21.15.16:0/3518170313 >> v2:172.21.15.16:3300/0 conn(0x7f395009d9f0 0x7f395006a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:58.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.133+0000 7f3954d89700 1 -- 172.21.15.16:0/3518170313 >> 172.21.15.16:0/3518170313 conn(0x7f3950066f50 msgr2=0x7f395009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:58.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.133+0000 7f3954d89700 1 -- 172.21.15.16:0/3518170313 shutdown_connections 2023-09-30T22:29:58.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.133+0000 7f3954d89700 1 -- 172.21.15.16:0/3518170313 wait complete. 2023-09-30T22:29:58.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.134+0000 7f3954d89700 1 Processor -- start 2023-09-30T22:29:58.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.134+0000 7f3954d89700 1 -- start start 2023-09-30T22:29:58.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.135+0000 7f3954d89700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f395006acb0 0x7f395006e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:58.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.135+0000 7f3954d89700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f395009d9f0 0x7f395006ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:58.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.135+0000 7f3954d89700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f39501316c0 0x7f395006f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:58.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.135+0000 7f3954d89700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39500721a0 con 0x7f395006acb0 2023-09-30T22:29:58.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.135+0000 7f3954d89700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39500711d0 con 0x7f39501316c0 2023-09-30T22:29:58.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.135+0000 7f3954d89700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3950071db0 con 0x7f395009d9f0 2023-09-30T22:29:58.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.135+0000 7f394dd9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f395009d9f0 0x7f395006ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:58.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.135+0000 7f394dd9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f395009d9f0 0x7f395006ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:45018/0 (socket says 172.21.15.16:45018) 2023-09-30T22:29:58.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.135+0000 7f394e59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f395006acb0 0x7f395006e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:58.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.135+0000 7f394dd9b700 1 -- 172.21.15.16:0/816377192 learned_addr learned my addr 172.21.15.16:0/816377192 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:58.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.135+0000 7f394ed9d700 1 --2- 172.21.15.16:0/816377192 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f39501316c0 0x7f395006f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:58.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.136+0000 7f394dd9b700 1 -- 172.21.15.16:0/816377192 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f39501316c0 msgr2=0x7f395006f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:58.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.136+0000 7f394dd9b700 1 --2- 172.21.15.16:0/816377192 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f39501316c0 0x7f395006f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:58.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.136+0000 7f394dd9b700 1 -- 172.21.15.16:0/816377192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f395006acb0 msgr2=0x7f395006e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:58.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.136+0000 7f394dd9b700 1 --2- 172.21.15.16:0/816377192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f395006acb0 0x7f395006e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:58.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.136+0000 7f394dd9b700 1 -- 172.21.15.16:0/816377192 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3940009550 con 0x7f395009d9f0 2023-09-30T22:29:58.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.136+0000 7f394e59c700 1 --2- 172.21.15.16:0/816377192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f395006acb0 0x7f395006e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:58.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.136+0000 7f394dd9b700 1 --2- 172.21.15.16:0/816377192 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f395009d9f0 0x7f395006ee20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f394400c820 tx=0x7f394400cb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:58.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.136+0000 7f393effd700 1 -- 172.21.15.16:0/816377192 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3944011800 con 0x7f395009d9f0 2023-09-30T22:29:58.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.136+0000 7f393effd700 1 -- 172.21.15.16:0/816377192 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3944011960 con 0x7f395009d9f0 2023-09-30T22:29:58.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.136+0000 7f3954d89700 1 -- 172.21.15.16:0/816377192 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39500a3540 con 0x7f395009d9f0 2023-09-30T22:29:58.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.136+0000 7f393effd700 1 -- 172.21.15.16:0/816377192 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3944011cf0 con 0x7f395009d9f0 2023-09-30T22:29:58.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.136+0000 7f3954d89700 1 -- 172.21.15.16:0/816377192 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3950072340 con 0x7f395009d9f0 2023-09-30T22:29:58.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.137+0000 7f393effd700 1 -- 172.21.15.16:0/816377192 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f3944018360 con 0x7f395009d9f0 2023-09-30T22:29:58.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.138+0000 7f3954d89700 1 -- 172.21.15.16:0/816377192 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3950074ea0 con 0x7f395009d9f0 2023-09-30T22:29:58.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.138+0000 7f393effd700 1 -- 172.21.15.16:0/816377192 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7f39440402b0 con 0x7f395009d9f0 2023-09-30T22:29:58.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.138+0000 7f394e59c700 1 -- 172.21.15.16:0/816377192 >> v1:172.21.15.16:6816/126811 conn(0x7f393002f510 legacy=0x7f39300319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:58.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.147+0000 7f393effd700 1 -- 172.21.15.16:0/816377192 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3944016260 con 0x7f395009d9f0 2023-09-30T22:29:58.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.201+0000 7f21fb343700 1 Processor -- start 2023-09-30T22:29:58.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.201+0000 7f21fb343700 1 -- start start 2023-09-30T22:29:58.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.202+0000 7f21fb343700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f21f409e1d0 0x7f21f4138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:58.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.202+0000 7f21fb343700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f21f4139170 0x7f21f413b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:58.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.202+0000 7f21fb343700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21f409b120 con 0x7f21f409d6f0 2023-09-30T22:29:58.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.202+0000 7f21fb343700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21f406b460 con 0x7f21f4139170 2023-09-30T22:29:58.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.202+0000 7f21fb343700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21f409a7e0 con 0x7f21f409e1d0 2023-09-30T22:29:58.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.202+0000 7f21f90df700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f21f409d6f0 0x7f21f409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57476/0 (socket says 172.21.15.16:57476) 2023-09-30T22:29:58.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.202+0000 7f21f90df700 1 -- 172.21.15.16:0/2415672910 learned_addr learned my addr 172.21.15.16:0/2415672910 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:29:58.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.202+0000 7f21f98e0700 1 --2- 172.21.15.16:0/2415672910 >> v2:172.21.15.16:3300/0 conn(0x7f21f4139170 0x7f21f413b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:58.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.202+0000 7f21f88de700 1 --2- 172.21.15.16:0/2415672910 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f21f409e1d0 0x7f21f4138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:58.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.203+0000 7f21ebfff700 1 -- 172.21.15.16:0/2415672910 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3152046625 0 0) 0x7f21f409b120 con 0x7f21f409d6f0 2023-09-30T22:29:58.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.203+0000 7f21ebfff700 1 -- 172.21.15.16:0/2415672910 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21e00035b0 con 0x7f21f409d6f0 2023-09-30T22:29:58.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.203+0000 7f21f88de700 1 -- 172.21.15.16:0/2415672910 >> v2:172.21.15.16:3300/0 conn(0x7f21f4139170 msgr2=0x7f21f413b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:58.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.203+0000 7f21f88de700 1 --2- 172.21.15.16:0/2415672910 >> v2:172.21.15.16:3300/0 conn(0x7f21f4139170 0x7f21f413b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:58.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.203+0000 7f21f88de700 1 -- 172.21.15.16:0/2415672910 >> v1:172.21.15.16:6789/0 conn(0x7f21f409d6f0 legacy=0x7f21f409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:58.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.203+0000 7f21f88de700 1 -- 172.21.15.16:0/2415672910 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21f409a500 con 0x7f21f409e1d0 2023-09-30T22:29:58.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.203+0000 7f21f98e0700 1 --2- 172.21.15.16:0/2415672910 >> v2:172.21.15.16:3300/0 conn(0x7f21f4139170 0x7f21f413b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:58.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.203+0000 7f21f88de700 1 --2- 172.21.15.16:0/2415672910 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f21f409e1d0 0x7f21f4138c30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f21e4009870 tx=0x7f21e4009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2bf82589b0f0e6a5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:58.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.203+0000 7f21ebfff700 1 -- 172.21.15.16:0/2415672910 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e4011830 con 0x7f21f409e1d0 2023-09-30T22:29:58.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.203+0000 7f21ebfff700 1 -- 172.21.15.16:0/2415672910 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21e4011990 con 0x7f21f409e1d0 2023-09-30T22:29:58.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.204+0000 7f21ebfff700 1 -- 172.21.15.16:0/2415672910 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e4011d00 con 0x7f21f409e1d0 2023-09-30T22:29:58.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.204+0000 7f21fb343700 1 -- 172.21.15.16:0/2415672910 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f21f409e1d0 msgr2=0x7f21f4138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:58.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.204+0000 7f21fb343700 1 --2- 172.21.15.16:0/2415672910 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f21f409e1d0 0x7f21f4138c30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f21e4009870 tx=0x7f21e4009b80 comp rx=0 tx=0).stop 2023-09-30T22:29:58.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.205+0000 7f21fb343700 1 -- 172.21.15.16:0/2415672910 shutdown_connections 2023-09-30T22:29:58.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.205+0000 7f21fb343700 1 --2- 172.21.15.16:0/2415672910 >> v2:172.21.15.16:3300/0 conn(0x7f21f4139170 0x7f21f413b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:58.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.205+0000 7f21fb343700 1 --2- 172.21.15.16:0/2415672910 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f21f409e1d0 0x7f21f4138c30 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:58.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.205+0000 7f21fb343700 1 -- 172.21.15.16:0/2415672910 >> 172.21.15.16:0/2415672910 conn(0x7f21f4066f50 msgr2=0x7f21f409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:58.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.205+0000 7f21fb343700 1 -- 172.21.15.16:0/2415672910 shutdown_connections 2023-09-30T22:29:58.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.205+0000 7f21fb343700 1 -- 172.21.15.16:0/2415672910 wait complete. 2023-09-30T22:29:58.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.206+0000 7f21fb343700 1 Processor -- start 2023-09-30T22:29:58.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.206+0000 7f21fb343700 1 -- start start 2023-09-30T22:29:58.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.206+0000 7f21fb343700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f21f409d6f0 0x7f21f4120690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:58.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.206+0000 7f21fb343700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f21f409e1d0 0x7f21f4120bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:58.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.206+0000 7f21fb343700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21f4139170 0x7f21f4121110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:58.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.206+0000 7f21fb343700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f4135fc0 con 0x7f21f4139170 2023-09-30T22:29:58.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.206+0000 7f21fb343700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21f4136580 con 0x7f21f409e1d0 2023-09-30T22:29:58.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.206+0000 7f21fb343700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f21f41241b0 con 0x7f21f409d6f0 2023-09-30T22:29:58.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.207+0000 7f21f98e0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21f4139170 0x7f21f4121110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:58.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.207+0000 7f21f98e0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21f4139170 0x7f21f4121110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52812/0 (socket says 172.21.15.16:52812) 2023-09-30T22:29:58.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.207+0000 7f21f98e0700 1 -- 172.21.15.16:0/2601778525 learned_addr learned my addr 172.21.15.16:0/2601778525 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:58.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.207+0000 7f21f90df700 1 --2- 172.21.15.16:0/2601778525 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f21f409d6f0 0x7f21f4120690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:58.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.207+0000 7f21f88de700 1 --2- 172.21.15.16:0/2601778525 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f21f409e1d0 0x7f21f4120bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:58.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.207+0000 7f21f98e0700 1 -- 172.21.15.16:0/2601778525 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f21f409d6f0 msgr2=0x7f21f4120690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:58.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.207+0000 7f21f98e0700 1 --2- 172.21.15.16:0/2601778525 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f21f409d6f0 0x7f21f4120690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:58.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.207+0000 7f21f98e0700 1 -- 172.21.15.16:0/2601778525 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f21f409e1d0 msgr2=0x7f21f4120bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:58.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.207+0000 7f21f98e0700 1 --2- 172.21.15.16:0/2601778525 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f21f409e1d0 0x7f21f4120bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:58.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.207+0000 7f21f98e0700 1 -- 172.21.15.16:0/2601778525 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21ec009a70 con 0x7f21f4139170 2023-09-30T22:29:58.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.207+0000 7f21f90df700 1 --2- 172.21.15.16:0/2601778525 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f21f409d6f0 0x7f21f4120690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:29:58.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.207+0000 7f21f98e0700 1 --2- 172.21.15.16:0/2601778525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21f4139170 0x7f21f4121110 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f21ec00efc0 tx=0x7f21ec00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:58.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.207+0000 7f21e9ffb700 1 -- 172.21.15.16:0/2601778525 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ec00cb40 con 0x7f21f4139170 2023-09-30T22:29:58.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.207+0000 7f21e9ffb700 1 -- 172.21.15.16:0/2601778525 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21ec00f040 con 0x7f21f4139170 2023-09-30T22:29:58.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.207+0000 7f21fb343700 1 -- 172.21.15.16:0/2601778525 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21e40094a0 con 0x7f21f4139170 2023-09-30T22:29:58.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.208+0000 7f21e9ffb700 1 -- 172.21.15.16:0/2601778525 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ec003a20 con 0x7f21f4139170 2023-09-30T22:29:58.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.208+0000 7f21fb343700 1 -- 172.21.15.16:0/2601778525 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21f409c710 con 0x7f21f4139170 2023-09-30T22:29:58.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.209+0000 7f21fb343700 1 -- 172.21.15.16:0/2601778525 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21f4074ea0 con 0x7f21f4139170 2023-09-30T22:29:58.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.210+0000 7f21e9ffb700 1 -- 172.21.15.16:0/2601778525 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f21ec014480 con 0x7f21f4139170 2023-09-30T22:29:58.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.210+0000 7f21e9ffb700 1 -- 172.21.15.16:0/2601778525 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7f21ec040d00 con 0x7f21f4139170 2023-09-30T22:29:58.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.210+0000 7f21f90df700 1 -- 172.21.15.16:0/2601778525 >> v1:172.21.15.16:6816/126811 conn(0x7f21d802f540 legacy=0x7f21d80319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:58.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.214+0000 7f21e9ffb700 1 -- 172.21.15.16:0/2601778525 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f21ec01b950 con 0x7f21f4139170 2023-09-30T22:29:58.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.236+0000 7f102ece9700 1 Processor -- start 2023-09-30T22:29:58.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.237+0000 7f102ece9700 1 -- start start 2023-09-30T22:29:58.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.237+0000 7f102ece9700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f102809e1d0 0x7f1028138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:58.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.237+0000 7f102ece9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1028139170 0x7f102813b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:58.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.237+0000 7f102ece9700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f102809b120 con 0x7f102809d6f0 2023-09-30T22:29:58.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.237+0000 7f102ece9700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f102806b460 con 0x7f102809e1d0 2023-09-30T22:29:58.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.237+0000 7f102ece9700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f102809a7e0 con 0x7f1028139170 2023-09-30T22:29:58.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.237+0000 7f102d286700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1028139170 0x7f102813b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:58.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.237+0000 7f102ca85700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f102809d6f0 0x7f102809dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57488/0 (socket says 172.21.15.16:57488) 2023-09-30T22:29:58.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.237+0000 7f102ca85700 1 -- 172.21.15.16:0/2820001767 learned_addr learned my addr 172.21.15.16:0/2820001767 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:29:58.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.238+0000 7f1027fff700 1 --2- 172.21.15.16:0/2820001767 >> v2:172.21.15.16:3300/0 conn(0x7f102809e1d0 0x7f1028138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:58.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.238+0000 7f10277fe700 1 -- 172.21.15.16:0/2820001767 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4192674898 0 0) 0x7f102809b120 con 0x7f102809d6f0 2023-09-30T22:29:58.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.238+0000 7f10277fe700 1 -- 172.21.15.16:0/2820001767 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10180035b0 con 0x7f102809d6f0 2023-09-30T22:29:58.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.238+0000 7f1027fff700 1 -- 172.21.15.16:0/2820001767 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1028139170 msgr2=0x7f102813b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:58.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.238+0000 7f1027fff700 1 --2- 172.21.15.16:0/2820001767 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1028139170 0x7f102813b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:58.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.238+0000 7f1027fff700 1 -- 172.21.15.16:0/2820001767 >> v1:172.21.15.16:6789/0 conn(0x7f102809d6f0 legacy=0x7f102809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:58.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.238+0000 7f1027fff700 1 -- 172.21.15.16:0/2820001767 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f102809a500 con 0x7f102809e1d0 2023-09-30T22:29:58.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.238+0000 7f1027fff700 1 --2- 172.21.15.16:0/2820001767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f102809e1d0 0x7f1028138c30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1014009870 tx=0x7f1014009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bea434a7226f152 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:58.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.238+0000 7f10277fe700 1 -- 172.21.15.16:0/2820001767 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1014011830 con 0x7f102809e1d0 2023-09-30T22:29:58.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.238+0000 7f10277fe700 1 -- 172.21.15.16:0/2820001767 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1014011990 con 0x7f102809e1d0 2023-09-30T22:29:58.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.238+0000 7f10277fe700 1 -- 172.21.15.16:0/2820001767 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1014011d20 con 0x7f102809e1d0 2023-09-30T22:29:58.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.239+0000 7f102ece9700 1 -- 172.21.15.16:0/2820001767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f102809e1d0 msgr2=0x7f1028138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:58.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.239+0000 7f102ece9700 1 --2- 172.21.15.16:0/2820001767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f102809e1d0 0x7f1028138c30 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f1014009870 tx=0x7f1014009b80 comp rx=0 tx=0).stop 2023-09-30T22:29:58.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.239+0000 7f102ece9700 1 -- 172.21.15.16:0/2820001767 shutdown_connections 2023-09-30T22:29:58.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.239+0000 7f102ece9700 1 --2- 172.21.15.16:0/2820001767 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1028139170 0x7f102813b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:58.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.239+0000 7f102ece9700 1 --2- 172.21.15.16:0/2820001767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f102809e1d0 0x7f1028138c30 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:58.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.239+0000 7f102ece9700 1 -- 172.21.15.16:0/2820001767 >> 172.21.15.16:0/2820001767 conn(0x7f1028066f50 msgr2=0x7f102809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:58.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.239+0000 7f102ece9700 1 -- 172.21.15.16:0/2820001767 shutdown_connections 2023-09-30T22:29:58.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.239+0000 7f102ece9700 1 -- 172.21.15.16:0/2820001767 wait complete. 2023-09-30T22:29:58.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.240+0000 7f102ece9700 1 Processor -- start 2023-09-30T22:29:58.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.240+0000 7f102ece9700 1 -- start start 2023-09-30T22:29:58.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.241+0000 7f102ece9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f102809d6f0 0x7f10281c8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:58.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.241+0000 7f102ece9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f102809e1d0 0x7f10281c8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:58.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.241+0000 7f102ece9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1028139170 0x7f102811e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:58.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.241+0000 7f102ece9700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10281221f0 con 0x7f102809e1d0 2023-09-30T22:29:58.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.241+0000 7f102ece9700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10281c9110 con 0x7f102809d6f0 2023-09-30T22:29:58.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.241+0000 7f102ece9700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1028120470 con 0x7f1028139170 2023-09-30T22:29:58.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.241+0000 7f102ca85700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f102809d6f0 0x7f10281c8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:58.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.241+0000 7f102d286700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1028139170 0x7f102811e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:58.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.241+0000 7f102ca85700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f102809d6f0 0x7f10281c8690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54794/0 (socket says 172.21.15.16:54794) 2023-09-30T22:29:58.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.241+0000 7f102ca85700 1 -- 172.21.15.16:0/4102804349 learned_addr learned my addr 172.21.15.16:0/4102804349 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:58.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.242+0000 7f1027fff700 1 --2- 172.21.15.16:0/4102804349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f102809e1d0 0x7f10281c8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:58.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.242+0000 7f102ca85700 1 -- 172.21.15.16:0/4102804349 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1028139170 msgr2=0x7f102811e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:58.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.242+0000 7f102ca85700 1 --2- 172.21.15.16:0/4102804349 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1028139170 0x7f102811e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:58.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.242+0000 7f102ca85700 1 -- 172.21.15.16:0/4102804349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f102809e1d0 msgr2=0x7f10281c8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:58.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.242+0000 7f102ca85700 1 --2- 172.21.15.16:0/4102804349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f102809e1d0 0x7f10281c8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:58.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.242+0000 7f102ca85700 1 -- 172.21.15.16:0/4102804349 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10140094a0 con 0x7f102809d6f0 2023-09-30T22:29:58.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.242+0000 7f1027fff700 1 --2- 172.21.15.16:0/4102804349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f102809e1d0 0x7f10281c8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:29:58.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.242+0000 7f102ca85700 1 --2- 172.21.15.16:0/4102804349 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f102809d6f0 0x7f10281c8690 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f101c00c730 tx=0x7f101c00ca40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:58.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.242+0000 7f10257fa700 1 -- 172.21.15.16:0/4102804349 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101c005d70 con 0x7f102809d6f0 2023-09-30T22:29:58.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.242+0000 7f10257fa700 1 -- 172.21.15.16:0/4102804349 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f101c0070d0 con 0x7f102809d6f0 2023-09-30T22:29:58.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.242+0000 7f102ece9700 1 -- 172.21.15.16:0/4102804349 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10280a3a40 con 0x7f102809d6f0 2023-09-30T22:29:58.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.242+0000 7f10257fa700 1 -- 172.21.15.16:0/4102804349 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101c007460 con 0x7f102809d6f0 2023-09-30T22:29:58.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.242+0000 7f102ece9700 1 -- 172.21.15.16:0/4102804349 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1028120c20 con 0x7f102809d6f0 2023-09-30T22:29:58.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.243+0000 7f10257fa700 1 -- 172.21.15.16:0/4102804349 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f101c0075c0 con 0x7f102809d6f0 2023-09-30T22:29:58.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.244+0000 7f102ece9700 1 -- 172.21.15.16:0/4102804349 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1028074ea0 con 0x7f102809d6f0 2023-09-30T22:29:58.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.244+0000 7f1027fff700 1 -- 172.21.15.16:0/4102804349 >> v1:172.21.15.16:6816/126811 conn(0x7f100c02f510 legacy=0x7f100c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:58.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.244+0000 7f10257fa700 1 -- 172.21.15.16:0/4102804349 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7f101c03f100 con 0x7f102809d6f0 2023-09-30T22:29:58.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.245+0000 7fc85ebfb700 1 Processor -- start 2023-09-30T22:29:58.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.245+0000 7fc85ebfb700 1 -- start start 2023-09-30T22:29:58.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.246+0000 7fc85ebfb700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc85806f5d0 0x7fc85812ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:58.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.246+0000 7fc85ebfb700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc858135c20 0x7fc858138000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:58.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.246+0000 7fc85ebfb700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc85806fa40 con 0x7fc85812cf60 2023-09-30T22:29:58.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.246+0000 7fc85ebfb700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc85812cd90 con 0x7fc858135c20 2023-09-30T22:29:58.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.246+0000 7fc85ebfb700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc858071780 con 0x7fc85806f5d0 2023-09-30T22:29:58.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.246+0000 7fc85d198700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc858135c20 0x7fc858138000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:58.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.246+0000 7fc85d198700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc858135c20 0x7fc858138000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52836/0 (socket says 172.21.15.16:52836) 2023-09-30T22:29:58.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.246+0000 7fc85d198700 1 -- 172.21.15.16:0/33013827 learned_addr learned my addr 172.21.15.16:0/33013827 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:58.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.246+0000 7fc857fff700 1 --2- 172.21.15.16:0/33013827 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc85806f5d0 0x7fc85812ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:58.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.247+0000 7fc85d198700 1 -- 172.21.15.16:0/33013827 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc85806f5d0 msgr2=0x7fc85812ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:58.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.247+0000 7fc85d198700 1 --2- 172.21.15.16:0/33013827 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc85806f5d0 0x7fc85812ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:58.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.247+0000 7fc85d198700 1 -- 172.21.15.16:0/33013827 >> v1:172.21.15.16:6789/0 conn(0x7fc85812cf60 legacy=0x7fc85812f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:58.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.247+0000 7fc85d198700 1 -- 172.21.15.16:0/33013827 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc858071a40 con 0x7fc858135c20 2023-09-30T22:29:58.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.247+0000 7fc85d198700 1 --2- 172.21.15.16:0/33013827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc858135c20 0x7fc858138000 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc850009e80 tx=0x7fc85000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a925e8ec4b28d2e9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:58.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.247+0000 7fc8577fe700 1 -- 172.21.15.16:0/33013827 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85000e070 con 0x7fc858135c20 2023-09-30T22:29:58.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.247+0000 7fc8577fe700 1 -- 172.21.15.16:0/33013827 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc850003a20 con 0x7fc858135c20 2023-09-30T22:29:58.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.247+0000 7fc8577fe700 1 -- 172.21.15.16:0/33013827 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc850003dd0 con 0x7fc858135c20 2023-09-30T22:29:58.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.248+0000 7fc85ebfb700 1 -- 172.21.15.16:0/33013827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc858135c20 msgr2=0x7fc858138000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:58.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.248+0000 7fc85ebfb700 1 --2- 172.21.15.16:0/33013827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc858135c20 0x7fc858138000 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc850009e80 tx=0x7fc85000b330 comp rx=0 tx=0).stop 2023-09-30T22:29:58.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.248+0000 7fc85ebfb700 1 -- 172.21.15.16:0/33013827 shutdown_connections 2023-09-30T22:29:58.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.248+0000 7fc85ebfb700 1 --2- 172.21.15.16:0/33013827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc858135c20 0x7fc858138000 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:58.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.248+0000 7fc85ebfb700 1 --2- 172.21.15.16:0/33013827 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc85806f5d0 0x7fc85812ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:58.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.248+0000 7fc85ebfb700 1 -- 172.21.15.16:0/33013827 >> 172.21.15.16:0/33013827 conn(0x7fc8580a2aa0 msgr2=0x7fc8580a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:58.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.248+0000 7fc85ebfb700 1 -- 172.21.15.16:0/33013827 shutdown_connections 2023-09-30T22:29:58.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.248+0000 7fc85ebfb700 1 -- 172.21.15.16:0/33013827 wait complete. 2023-09-30T22:29:58.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.249+0000 7f10257fa700 1 -- 172.21.15.16:0/4102804349 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f101c00e280 con 0x7f102809d6f0 2023-09-30T22:29:58.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.250+0000 7fc85ebfb700 1 Processor -- start 2023-09-30T22:29:58.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.250+0000 7fc85ebfb700 1 -- start start 2023-09-30T22:29:58.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.251+0000 7fc85ebfb700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc85806f5d0 0x7fc8581308e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:58.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.251+0000 7fc85ebfb700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc85812cf60 0x7fc858130e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:58.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.251+0000 7fc85ebfb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc858135c20 0x7fc858131360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:58.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.251+0000 7fc85ebfb700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc858134dd0 con 0x7fc858135c20 2023-09-30T22:29:58.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.251+0000 7fc85ebfb700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8580a0ff0 con 0x7fc85812cf60 2023-09-30T22:29:58.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.251+0000 7fc85ebfb700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc858129fa0 con 0x7fc85806f5d0 2023-09-30T22:29:58.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.251+0000 7fc85d198700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc858135c20 0x7fc858131360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:58.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.251+0000 7fc85d198700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc858135c20 0x7fc858131360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52850/0 (socket says 172.21.15.16:52850) 2023-09-30T22:29:58.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.251+0000 7fc85d198700 1 -- 172.21.15.16:0/1033410360 learned_addr learned my addr 172.21.15.16:0/1033410360 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:58.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.251+0000 7fc85c997700 1 --2- 172.21.15.16:0/1033410360 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc85806f5d0 0x7fc8581308e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:58.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.251+0000 7fc85d198700 1 -- 172.21.15.16:0/1033410360 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc85806f5d0 msgr2=0x7fc8581308e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:58.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.251+0000 7fc85d198700 1 --2- 172.21.15.16:0/1033410360 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc85806f5d0 0x7fc8581308e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:58.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.251+0000 7fc85d198700 1 -- 172.21.15.16:0/1033410360 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc85812cf60 msgr2=0x7fc858130e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:58.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.251+0000 7fc85d198700 1 --2- 172.21.15.16:0/1033410360 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc85812cf60 0x7fc858130e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:58.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.252+0000 7fc85d198700 1 -- 172.21.15.16:0/1033410360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc850009b30 con 0x7fc858135c20 2023-09-30T22:29:58.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.252+0000 7fc85d198700 1 --2- 172.21.15.16:0/1033410360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc858135c20 0x7fc858131360 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc850015040 tx=0x7fc850004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:58.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.252+0000 7fc8557fa700 1 -- 172.21.15.16:0/1033410360 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85001ba30 con 0x7fc858135c20 2023-09-30T22:29:58.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.252+0000 7fc8557fa700 1 -- 172.21.15.16:0/1033410360 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8500092e0 con 0x7fc858135c20 2023-09-30T22:29:58.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.252+0000 7fc85ebfb700 1 -- 172.21.15.16:0/1033410360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc858132cd0 con 0x7fc858135c20 2023-09-30T22:29:58.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.252+0000 7fc8557fa700 1 -- 172.21.15.16:0/1033410360 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85001bdd0 con 0x7fc858135c20 2023-09-30T22:29:58.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.252+0000 7fc85ebfb700 1 -- 172.21.15.16:0/1033410360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc858132300 con 0x7fc858135c20 2023-09-30T22:29:58.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.253+0000 7fc8557fa700 1 -- 172.21.15.16:0/1033410360 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fc850004030 con 0x7fc858135c20 2023-09-30T22:29:58.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.253+0000 7fc8557fa700 1 -- 172.21.15.16:0/1033410360 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7fc850044890 con 0x7fc858135c20 2023-09-30T22:29:58.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.253+0000 7fc85c997700 1 -- 172.21.15.16:0/1033410360 >> v1:172.21.15.16:6816/126811 conn(0x7fc83c02f560 legacy=0x7fc83c031a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:58.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.254+0000 7fc842ffd700 1 -- 172.21.15.16:0/1033410360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8300052e0 con 0x7fc858135c20 2023-09-30T22:29:58.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.263+0000 7fc8557fa700 1 -- 172.21.15.16:0/1033410360 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc850017070 con 0x7fc858135c20 2023-09-30T22:29:58.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.339+0000 7f394e59c700 1 -- 172.21.15.16:0/816377192 >> v1:172.21.15.16:6816/126811 conn(0x7f393002f510 legacy=0x7f39300319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:58.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.365+0000 7f3954d89700 1 -- 172.21.15.16:0/816377192 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f395009e980 con 0x7f395009d9f0 2023-09-30T22:29:58.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.366+0000 7f393effd700 1 -- 172.21.15.16:0/816377192 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3944022020 con 0x7f395009d9f0 2023-09-30T22:29:58.366 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:29:58.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.368+0000 7f3954d89700 1 -- 172.21.15.16:0/816377192 >> v1:172.21.15.16:6816/126811 conn(0x7f393002f510 legacy=0x7f39300319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:58.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.368+0000 7f3954d89700 1 -- 172.21.15.16:0/816377192 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f395009d9f0 msgr2=0x7f395006ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:58.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.368+0000 7f3954d89700 1 --2- 172.21.15.16:0/816377192 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f395009d9f0 0x7f395006ee20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f394400c820 tx=0x7f394400cb30 comp rx=0 tx=0).stop 2023-09-30T22:29:58.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.368+0000 7f3954d89700 1 -- 172.21.15.16:0/816377192 shutdown_connections 2023-09-30T22:29:58.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.368+0000 7f3954d89700 1 --2- 172.21.15.16:0/816377192 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f39501316c0 0x7f395006f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:58.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.368+0000 7f3954d89700 1 --2- 172.21.15.16:0/816377192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f395006acb0 0x7f395006e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:58.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.368+0000 7f3954d89700 1 --2- 172.21.15.16:0/816377192 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f395009d9f0 0x7f395006ee20 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:58.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.368+0000 7f3954d89700 1 -- 172.21.15.16:0/816377192 >> 172.21.15.16:0/816377192 conn(0x7f3950066f50 msgr2=0x7f3950130150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:58.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.368+0000 7f3954d89700 1 -- 172.21.15.16:0/816377192 shutdown_connections 2023-09-30T22:29:58.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.368+0000 7f3954d89700 1 -- 172.21.15.16:0/816377192 wait complete. 2023-09-30T22:29:58.383 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:29:58.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.411+0000 7f21f90df700 1 -- 172.21.15.16:0/2601778525 >> v1:172.21.15.16:6816/126811 conn(0x7f21d802f540 legacy=0x7f21d80319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:58.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.445+0000 7f1027fff700 1 -- 172.21.15.16:0/4102804349 >> v1:172.21.15.16:6816/126811 conn(0x7f100c02f510 legacy=0x7f100c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:58.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.454+0000 7fc85c997700 1 -- 172.21.15.16:0/1033410360 >> v1:172.21.15.16:6816/126811 conn(0x7fc83c02f560 legacy=0x7fc83c031a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:58.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.460+0000 7f102ece9700 1 -- 172.21.15.16:0/4102804349 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f102809f420 con 0x7f102809d6f0 2023-09-30T22:29:58.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.461+0000 7f10257fa700 1 -- 172.21.15.16:0/4102804349 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f101c017020 con 0x7f102809d6f0 2023-09-30T22:29:58.461 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:29:58.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.463+0000 7f102ece9700 1 -- 172.21.15.16:0/4102804349 >> v1:172.21.15.16:6816/126811 conn(0x7f100c02f510 legacy=0x7f100c0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:58.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.463+0000 7f102ece9700 1 -- 172.21.15.16:0/4102804349 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f102809d6f0 msgr2=0x7f10281c8690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:58.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.463+0000 7f102ece9700 1 --2- 172.21.15.16:0/4102804349 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f102809d6f0 0x7f10281c8690 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f101c00c730 tx=0x7f101c00ca40 comp rx=0 tx=0).stop 2023-09-30T22:29:58.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.463+0000 7f102ece9700 1 -- 172.21.15.16:0/4102804349 shutdown_connections 2023-09-30T22:29:58.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.463+0000 7f102ece9700 1 --2- 172.21.15.16:0/4102804349 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f102809d6f0 0x7f10281c8690 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:58.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.463+0000 7f102ece9700 1 --2- 172.21.15.16:0/4102804349 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1028139170 0x7f102811e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:58.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.463+0000 7f102ece9700 1 --2- 172.21.15.16:0/4102804349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f102809e1d0 0x7f10281c8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:58.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.463+0000 7f102ece9700 1 -- 172.21.15.16:0/4102804349 >> 172.21.15.16:0/4102804349 conn(0x7f1028066f50 msgr2=0x7f1028136b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:58.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.463+0000 7f102ece9700 1 -- 172.21.15.16:0/4102804349 shutdown_connections 2023-09-30T22:29:58.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.463+0000 7f102ece9700 1 -- 172.21.15.16:0/4102804349 wait complete. 2023-09-30T22:29:58.479 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:29:58.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.480+0000 7f21fb343700 1 -- 172.21.15.16:0/2601778525 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f21f409f420 con 0x7f21f4139170 2023-09-30T22:29:58.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.480+0000 7f21e9ffb700 1 -- 172.21.15.16:0/2601778525 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f21ec018070 con 0x7f21f4139170 2023-09-30T22:29:58.481 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:29:58.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.482+0000 7f21fb343700 1 -- 172.21.15.16:0/2601778525 >> v1:172.21.15.16:6816/126811 conn(0x7f21d802f540 legacy=0x7f21d80319f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:58.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.482+0000 7f21fb343700 1 -- 172.21.15.16:0/2601778525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21f4139170 msgr2=0x7f21f4121110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:58.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.482+0000 7f21fb343700 1 --2- 172.21.15.16:0/2601778525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21f4139170 0x7f21f4121110 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f21ec00efc0 tx=0x7f21ec00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:29:58.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.482+0000 7f21fb343700 1 -- 172.21.15.16:0/2601778525 shutdown_connections 2023-09-30T22:29:58.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.482+0000 7f21fb343700 1 --2- 172.21.15.16:0/2601778525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21f4139170 0x7f21f4121110 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:58.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.482+0000 7f21fb343700 1 --2- 172.21.15.16:0/2601778525 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f21f409d6f0 0x7f21f4120690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:58.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.482+0000 7f21fb343700 1 --2- 172.21.15.16:0/2601778525 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f21f409e1d0 0x7f21f4120bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:58.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.482+0000 7f21fb343700 1 -- 172.21.15.16:0/2601778525 >> 172.21.15.16:0/2601778525 conn(0x7f21f4066f50 msgr2=0x7f21f40a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:58.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.483+0000 7f21fb343700 1 -- 172.21.15.16:0/2601778525 shutdown_connections 2023-09-30T22:29:58.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.483+0000 7f21fb343700 1 -- 172.21.15.16:0/2601778525 wait complete. 2023-09-30T22:29:58.497 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:29:58.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.522+0000 7fc842ffd700 1 -- 172.21.15.16:0/1033410360 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc830005860 con 0x7fc858135c20 2023-09-30T22:29:58.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.522+0000 7fc8557fa700 1 -- 172.21.15.16:0/1033410360 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc850012430 con 0x7fc858135c20 2023-09-30T22:29:58.522 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:29:58.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.524+0000 7fc842ffd700 1 -- 172.21.15.16:0/1033410360 >> v1:172.21.15.16:6816/126811 conn(0x7fc83c02f560 legacy=0x7fc83c031a10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:58.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.524+0000 7fc842ffd700 1 -- 172.21.15.16:0/1033410360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc858135c20 msgr2=0x7fc858131360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:58.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.524+0000 7fc842ffd700 1 --2- 172.21.15.16:0/1033410360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc858135c20 0x7fc858131360 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc850015040 tx=0x7fc850004ad0 comp rx=0 tx=0).stop 2023-09-30T22:29:58.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.524+0000 7fc842ffd700 1 -- 172.21.15.16:0/1033410360 shutdown_connections 2023-09-30T22:29:58.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.524+0000 7fc842ffd700 1 --2- 172.21.15.16:0/1033410360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc858135c20 0x7fc858131360 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:58.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.524+0000 7fc842ffd700 1 --2- 172.21.15.16:0/1033410360 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc85806f5d0 0x7fc8581308e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:58.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.524+0000 7fc842ffd700 1 --2- 172.21.15.16:0/1033410360 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc85812cf60 0x7fc858130e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:58.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.524+0000 7fc842ffd700 1 -- 172.21.15.16:0/1033410360 >> 172.21.15.16:0/1033410360 conn(0x7fc8580a2aa0 msgr2=0x7fc85812f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:58.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.524+0000 7fc842ffd700 1 -- 172.21.15.16:0/1033410360 shutdown_connections 2023-09-30T22:29:58.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:58.524+0000 7fc842ffd700 1 -- 172.21.15.16:0/1033410360 wait complete. 2023-09-30T22:29:58.538 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:29:59.384 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:29:59.480 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:29:59.498 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:29:59.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.532+0000 7fa2adebb700 1 Processor -- start 2023-09-30T22:29:59.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.532+0000 7fa2adebb700 1 -- start start 2023-09-30T22:29:59.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.533+0000 7fa2adebb700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa2a81338a0 0x7fa2a8133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.533+0000 7fa2adebb700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa2a809e930 0x7fa2a813b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.533+0000 7fa2adebb700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2a806ef30 con 0x7fa2a809de50 2023-09-30T22:29:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.533+0000 7fa2adebb700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2a806fef0 con 0x7fa2a81338a0 2023-09-30T22:29:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.533+0000 7fa2adebb700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a81334b0 con 0x7fa2a809e930 2023-09-30T22:29:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.533+0000 7fa2a77fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa2a81338a0 0x7fa2a8133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.533+0000 7fa2a6ffd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa2a809de50 0x7fa2a809e220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57506/0 (socket says 172.21.15.16:57506) 2023-09-30T22:29:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.533+0000 7fa2a6ffd700 1 -- 172.21.15.16:0/913660340 learned_addr learned my addr 172.21.15.16:0/913660340 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:29:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.533+0000 7fa2a7fff700 1 --2- 172.21.15.16:0/913660340 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa2a809e930 0x7fa2a813b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.534+0000 7fa2a67fc700 1 -- 172.21.15.16:0/913660340 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3667472627 0 0) 0x7fa2a806ef30 con 0x7fa2a809de50 2023-09-30T22:29:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.534+0000 7fa2a77fe700 1 -- 172.21.15.16:0/913660340 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa2a809e930 msgr2=0x7fa2a813b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.534+0000 7fa2a77fe700 1 --2- 172.21.15.16:0/913660340 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa2a809e930 0x7fa2a813b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.534+0000 7fa2a77fe700 1 -- 172.21.15.16:0/913660340 >> v1:172.21.15.16:6789/0 conn(0x7fa2a809de50 legacy=0x7fa2a809e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:59.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.534+0000 7fa2a77fe700 1 -- 172.21.15.16:0/913660340 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a806fc30 con 0x7fa2a81338a0 2023-09-30T22:29:59.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.534+0000 7fa2a7fff700 1 --2- 172.21.15.16:0/913660340 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa2a809e930 0x7fa2a813b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:59.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.534+0000 7fa2a77fe700 1 --2- 172.21.15.16:0/913660340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2a81338a0 0x7fa2a8133c70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa298009840 tx=0x7fa298009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9fdc7c580988b26 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:59.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.534+0000 7fa2a67fc700 1 -- 172.21.15.16:0/913660340 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa298011830 con 0x7fa2a81338a0 2023-09-30T22:29:59.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.534+0000 7fa2a67fc700 1 -- 172.21.15.16:0/913660340 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa298011990 con 0x7fa2a81338a0 2023-09-30T22:29:59.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.534+0000 7fa2a67fc700 1 -- 172.21.15.16:0/913660340 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa298011d20 con 0x7fa2a81338a0 2023-09-30T22:29:59.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.535+0000 7fa2adebb700 1 -- 172.21.15.16:0/913660340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2a81338a0 msgr2=0x7fa2a8133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:59.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.535+0000 7fa2adebb700 1 --2- 172.21.15.16:0/913660340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2a81338a0 0x7fa2a8133c70 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa298009840 tx=0x7fa298009b50 comp rx=0 tx=0).stop 2023-09-30T22:29:59.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.535+0000 7fa2adebb700 1 -- 172.21.15.16:0/913660340 shutdown_connections 2023-09-30T22:29:59.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.535+0000 7fa2adebb700 1 --2- 172.21.15.16:0/913660340 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa2a809e930 0x7fa2a813b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.535+0000 7fa2adebb700 1 --2- 172.21.15.16:0/913660340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2a81338a0 0x7fa2a8133c70 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.535+0000 7fa2adebb700 1 -- 172.21.15.16:0/913660340 >> 172.21.15.16:0/913660340 conn(0x7fa2a80a2aa0 msgr2=0x7fa2a80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:59.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.535+0000 7fa2adebb700 1 -- 172.21.15.16:0/913660340 shutdown_connections 2023-09-30T22:29:59.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.535+0000 7fa2adebb700 1 -- 172.21.15.16:0/913660340 wait complete. 2023-09-30T22:29:59.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.536+0000 7fa2adebb700 1 Processor -- start 2023-09-30T22:29:59.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.537+0000 7fa2adebb700 1 -- start start 2023-09-30T22:29:59.538 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:29:59.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.537+0000 7fa2adebb700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa2a809de50 0x7fa2a811d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:59.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.537+0000 7fa2adebb700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa2a809e930 0x7fa2a811dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:59.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.537+0000 7fa2adebb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2a81338a0 0x7fa2a811e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:59.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.538+0000 7fa2adebb700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a81211c0 con 0x7fa2a81338a0 2023-09-30T22:29:59.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.538+0000 7fa2adebb700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a811f350 con 0x7fa2a809de50 2023-09-30T22:29:59.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.538+0000 7fa2adebb700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a80a0d70 con 0x7fa2a809e930 2023-09-30T22:29:59.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.538+0000 7fa2a6ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa2a809e930 0x7fa2a811dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:59.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.538+0000 7fa2a77fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa2a809de50 0x7fa2a811d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:59.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.538+0000 7fa2a77fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa2a809de50 0x7fa2a811d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54816/0 (socket says 172.21.15.16:54816) 2023-09-30T22:29:59.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.538+0000 7fa2a6ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa2a809e930 0x7fa2a811dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:45054/0 (socket says 172.21.15.16:45054) 2023-09-30T22:29:59.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.538+0000 7fa2a77fe700 1 -- 172.21.15.16:0/104994075 learned_addr learned my addr 172.21.15.16:0/104994075 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:59.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.538+0000 7fa2a7fff700 1 --2- 172.21.15.16:0/104994075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2a81338a0 0x7fa2a811e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:59.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.538+0000 7fa2a6ffd700 1 -- 172.21.15.16:0/104994075 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa2a809de50 msgr2=0x7fa2a811d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:59.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.538+0000 7fa2a6ffd700 1 --2- 172.21.15.16:0/104994075 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa2a809de50 0x7fa2a811d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.538+0000 7fa2a6ffd700 1 -- 172.21.15.16:0/104994075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2a81338a0 msgr2=0x7fa2a811e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:59.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.538+0000 7fa2a6ffd700 1 --2- 172.21.15.16:0/104994075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2a81338a0 0x7fa2a811e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.538+0000 7fa2a6ffd700 1 -- 172.21.15.16:0/104994075 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a0009a70 con 0x7fa2a809e930 2023-09-30T22:29:59.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.538+0000 7fa2a6ffd700 1 --2- 172.21.15.16:0/104994075 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa2a809e930 0x7fa2a811dc70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa29c00c3c0 tx=0x7fa29c00c6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:59.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.539+0000 7fa28ffff700 1 -- 172.21.15.16:0/104994075 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa29c004a70 con 0x7fa2a809e930 2023-09-30T22:29:59.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.539+0000 7fa28ffff700 1 -- 172.21.15.16:0/104994075 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa29c004c10 con 0x7fa2a809e930 2023-09-30T22:29:59.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.539+0000 7fa2adebb700 1 -- 172.21.15.16:0/104994075 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2980094a0 con 0x7fa2a809e930 2023-09-30T22:29:59.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.539+0000 7fa28ffff700 1 -- 172.21.15.16:0/104994075 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa29c0070d0 con 0x7fa2a809e930 2023-09-30T22:29:59.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.539+0000 7fa2adebb700 1 -- 172.21.15.16:0/104994075 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2a81200b0 con 0x7fa2a809e930 2023-09-30T22:29:59.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.540+0000 7fa28ffff700 1 -- 172.21.15.16:0/104994075 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fa29c007230 con 0x7fa2a809e930 2023-09-30T22:29:59.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.540+0000 7fa28ffff700 1 -- 172.21.15.16:0/104994075 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7fa29c03f6d0 con 0x7fa2a809e930 2023-09-30T22:29:59.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.540+0000 7fa2a77fe700 1 -- 172.21.15.16:0/104994075 >> v1:172.21.15.16:6816/126811 conn(0x7fa29402f510 legacy=0x7fa2940319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:59.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.540+0000 7fa2adebb700 1 -- 172.21.15.16:0/104994075 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2a8074ea0 con 0x7fa2a809e930 2023-09-30T22:29:59.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.546+0000 7fa28ffff700 1 -- 172.21.15.16:0/104994075 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa29c01bda0 con 0x7fa2a809e930 2023-09-30T22:29:59.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.627+0000 7ff8ac71e700 1 Processor -- start 2023-09-30T22:29:59.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.627+0000 7ff8ac71e700 1 -- start start 2023-09-30T22:29:59.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.628+0000 7ff8ac71e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff8a409d6f0 0x7ff8a409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:59.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.628+0000 7ff8ac71e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff8a409e410 0x7ff8a4130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:59.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.628+0000 7ff8ac71e700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8a409f130 con 0x7ff8a409e000 2023-09-30T22:29:59.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.628+0000 7ff8ac71e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a409f520 con 0x7ff8a409e410 2023-09-30T22:29:59.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.628+0000 7ff8ac71e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a409e930 con 0x7ff8a409d6f0 2023-09-30T22:29:59.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.628+0000 7ff8a9cb9700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff8a409e000 0x7ff8a4130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57510/0 (socket says 172.21.15.16:57510) 2023-09-30T22:29:59.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.628+0000 7ff8a9cb9700 1 -- 172.21.15.16:0/1588083688 learned_addr learned my addr 172.21.15.16:0/1588083688 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:29:59.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.628+0000 7ff8aa4ba700 1 --2- 172.21.15.16:0/1588083688 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff8a409d6f0 0x7ff8a409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:59.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.628+0000 7ff8aacbb700 1 --2- 172.21.15.16:0/1588083688 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff8a409e410 0x7ff8a4130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:59.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.629+0000 7ff8aa4ba700 1 -- 172.21.15.16:0/1588083688 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff8a409e410 msgr2=0x7ff8a4130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:59.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.629+0000 7ff8aa4ba700 1 --2- 172.21.15.16:0/1588083688 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff8a409e410 0x7ff8a4130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.629+0000 7ff8aa4ba700 1 -- 172.21.15.16:0/1588083688 >> v1:172.21.15.16:6789/0 conn(0x7ff8a409e000 legacy=0x7ff8a4130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:59.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.629+0000 7ff8a94b8700 1 -- 172.21.15.16:0/1588083688 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 897961057 0 0) 0x7ff8a409f130 con 0x7ff8a409e000 2023-09-30T22:29:59.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.629+0000 7ff8aa4ba700 1 -- 172.21.15.16:0/1588083688 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8a409b2d0 con 0x7ff8a409d6f0 2023-09-30T22:29:59.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.629+0000 7ff8aa4ba700 1 --2- 172.21.15.16:0/1588083688 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff8a409d6f0 0x7ff8a409dac0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff8a000bb10 tx=0x7ff8a000be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e12f92c86fe07eee server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:59.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.629+0000 7ff8a94b8700 1 -- 172.21.15.16:0/1588083688 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a000dc70 con 0x7ff8a409d6f0 2023-09-30T22:29:59.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.629+0000 7ff8a94b8700 1 -- 172.21.15.16:0/1588083688 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8a000ddd0 con 0x7ff8a409d6f0 2023-09-30T22:29:59.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.629+0000 7ff8a94b8700 1 -- 172.21.15.16:0/1588083688 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a0010690 con 0x7ff8a409d6f0 2023-09-30T22:29:59.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.630+0000 7ff8ac71e700 1 -- 172.21.15.16:0/1588083688 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff8a409d6f0 msgr2=0x7ff8a409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:59.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.630+0000 7ff8ac71e700 1 --2- 172.21.15.16:0/1588083688 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff8a409d6f0 0x7ff8a409dac0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff8a000bb10 tx=0x7ff8a000be20 comp rx=0 tx=0).stop 2023-09-30T22:29:59.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.630+0000 7ff8ac71e700 1 -- 172.21.15.16:0/1588083688 shutdown_connections 2023-09-30T22:29:59.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.630+0000 7ff8ac71e700 1 --2- 172.21.15.16:0/1588083688 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff8a409e410 0x7ff8a4130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.630+0000 7ff8ac71e700 1 --2- 172.21.15.16:0/1588083688 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff8a409d6f0 0x7ff8a409dac0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.630+0000 7ff8ac71e700 1 -- 172.21.15.16:0/1588083688 >> 172.21.15.16:0/1588083688 conn(0x7ff8a4066f50 msgr2=0x7ff8a409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:59.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.630+0000 7ff8ac71e700 1 -- 172.21.15.16:0/1588083688 shutdown_connections 2023-09-30T22:29:59.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.630+0000 7ff8ac71e700 1 -- 172.21.15.16:0/1588083688 wait complete. 2023-09-30T22:29:59.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.631+0000 7ff8ac71e700 1 Processor -- start 2023-09-30T22:29:59.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.631+0000 7ff8ac71e700 1 -- start start 2023-09-30T22:29:59.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.632+0000 7ff8ac71e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff8a409d6f0 0x7ff8a411e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:59.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.632+0000 7ff8ac71e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8a409e000 0x7ff8a411e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:59.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.632+0000 7ff8ac71e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff8a409e410 0x7ff8a411eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:59.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.632+0000 7ff8ac71e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a4120230 con 0x7ff8a409e000 2023-09-30T22:29:59.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.632+0000 7ff8ac71e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a40a1d00 con 0x7ff8a409d6f0 2023-09-30T22:29:59.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.632+0000 7ff8ac71e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a4121320 con 0x7ff8a409e410 2023-09-30T22:29:59.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.632+0000 7ff8aa4ba700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff8a409d6f0 0x7ff8a411e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:59.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.632+0000 7ff8aa4ba700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff8a409d6f0 0x7ff8a411e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54822/0 (socket says 172.21.15.16:54822) 2023-09-30T22:29:59.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.632+0000 7ff8aa4ba700 1 -- 172.21.15.16:0/2026083664 learned_addr learned my addr 172.21.15.16:0/2026083664 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:59.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.632+0000 7ff8a9cb9700 1 --2- 172.21.15.16:0/2026083664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8a409e000 0x7ff8a411e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:59.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.632+0000 7ff8aacbb700 1 --2- 172.21.15.16:0/2026083664 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff8a409e410 0x7ff8a411eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:59.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.633+0000 7ff8aa4ba700 1 -- 172.21.15.16:0/2026083664 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff8a409e410 msgr2=0x7ff8a411eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:59.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.633+0000 7ff8aa4ba700 1 --2- 172.21.15.16:0/2026083664 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff8a409e410 0x7ff8a411eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.633+0000 7ff8aa4ba700 1 -- 172.21.15.16:0/2026083664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8a409e000 msgr2=0x7ff8a411e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:59.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.633+0000 7ff8aa4ba700 1 --2- 172.21.15.16:0/2026083664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8a409e000 0x7ff8a411e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.633+0000 7ff8aa4ba700 1 -- 172.21.15.16:0/2026083664 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8a000b740 con 0x7ff8a409d6f0 2023-09-30T22:29:59.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.633+0000 7ff8aa4ba700 1 --2- 172.21.15.16:0/2026083664 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff8a409d6f0 0x7ff8a411e070 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff8a000bae0 tx=0x7ff8a0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:59.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.633+0000 7ff89affd700 1 -- 172.21.15.16:0/2026083664 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a0003b70 con 0x7ff8a409d6f0 2023-09-30T22:29:59.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.633+0000 7ff89affd700 1 -- 172.21.15.16:0/2026083664 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8a0003cd0 con 0x7ff8a409d6f0 2023-09-30T22:29:59.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.633+0000 7ff89affd700 1 -- 172.21.15.16:0/2026083664 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8a0010670 con 0x7ff8a409d6f0 2023-09-30T22:29:59.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.633+0000 7ff8ac71e700 1 -- 172.21.15.16:0/2026083664 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8a41218b0 con 0x7ff8a409d6f0 2023-09-30T22:29:59.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.633+0000 7ff8ac71e700 1 -- 172.21.15.16:0/2026083664 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8a41ca420 con 0x7ff8a409d6f0 2023-09-30T22:29:59.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.634+0000 7ff89affd700 1 -- 172.21.15.16:0/2026083664 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7ff8a002a3f0 con 0x7ff8a409d6f0 2023-09-30T22:29:59.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.635+0000 7ff89affd700 1 -- 172.21.15.16:0/2026083664 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7ff8a0049f50 con 0x7ff8a409d6f0 2023-09-30T22:29:59.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.635+0000 7ff8a9cb9700 1 -- 172.21.15.16:0/2026083664 >> v1:172.21.15.16:6816/126811 conn(0x7ff88802f510 legacy=0x7ff8880319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:59.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.635+0000 7ff8ac71e700 1 -- 172.21.15.16:0/2026083664 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8a411ac20 con 0x7ff8a409d6f0 2023-09-30T22:29:59.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.644+0000 7ff89affd700 1 -- 172.21.15.16:0/2026083664 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff8a002f080 con 0x7ff8a409d6f0 2023-09-30T22:29:59.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.676+0000 7f3ca4a62700 1 Processor -- start 2023-09-30T22:29:59.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.676+0000 7f3ca4a62700 1 -- start start 2023-09-30T22:29:59.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.677+0000 7f3ca4a62700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3ca009d6f0 0x7f3ca009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:59.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.677+0000 7f3ca4a62700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3ca009e000 0x7f3ca0130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:59.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.677+0000 7f3ca4a62700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3ca009e410 0x7f3ca0130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:59.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.677+0000 7f3ca4a62700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ca009f520 con 0x7f3ca009e410 2023-09-30T22:29:59.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.677+0000 7f3ca4a62700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca009f130 con 0x7f3ca009e000 2023-09-30T22:29:59.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.677+0000 7f3ca4a62700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca009e930 con 0x7f3ca009d6f0 2023-09-30T22:29:59.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.677+0000 7f3c9ed9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3ca009e410 0x7f3ca0130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:59.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.677+0000 7f3c9ed9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3ca009e410 0x7f3ca0130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52888/0 (socket says 172.21.15.16:52888) 2023-09-30T22:29:59.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.677+0000 7f3c9ed9d700 1 -- 172.21.15.16:0/2493446233 learned_addr learned my addr 172.21.15.16:0/2493446233 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:59.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.677+0000 7f3c9e59c700 1 --2- 172.21.15.16:0/2493446233 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3ca009d6f0 0x7f3ca009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:59.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.677+0000 7f3c9dd9b700 1 --2- 172.21.15.16:0/2493446233 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3ca009e000 0x7f3ca0130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:59.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.677+0000 7f3c9e59c700 1 -- 172.21.15.16:0/2493446233 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3ca009e000 msgr2=0x7f3ca0130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:59.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.677+0000 7f3c9e59c700 1 --2- 172.21.15.16:0/2493446233 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3ca009e000 0x7f3ca0130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.677+0000 7f3c9e59c700 1 -- 172.21.15.16:0/2493446233 >> v2:172.21.15.16:3300/0 conn(0x7f3ca009e410 msgr2=0x7f3ca0130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:59.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.677+0000 7f3c9e59c700 1 --2- 172.21.15.16:0/2493446233 >> v2:172.21.15.16:3300/0 conn(0x7f3ca009e410 0x7f3ca0130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.677+0000 7f3c9e59c700 1 -- 172.21.15.16:0/2493446233 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ca009b2d0 con 0x7f3ca009d6f0 2023-09-30T22:29:59.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.678+0000 7f3c9e59c700 1 --2- 172.21.15.16:0/2493446233 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3ca009d6f0 0x7f3ca009dac0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3c900098d0 tx=0x7f3c90009be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad24195285674823 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:59.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.678+0000 7f3c9d59a700 1 -- 172.21.15.16:0/2493446233 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c90004230 con 0x7f3ca009d6f0 2023-09-30T22:29:59.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.678+0000 7f3c9d59a700 1 -- 172.21.15.16:0/2493446233 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c90004390 con 0x7f3ca009d6f0 2023-09-30T22:29:59.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.678+0000 7f3c9d59a700 1 -- 172.21.15.16:0/2493446233 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c90010890 con 0x7f3ca009d6f0 2023-09-30T22:29:59.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.679+0000 7f3ca4a62700 1 -- 172.21.15.16:0/2493446233 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3ca009d6f0 msgr2=0x7f3ca009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:59.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.679+0000 7f3ca4a62700 1 --2- 172.21.15.16:0/2493446233 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3ca009d6f0 0x7f3ca009dac0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3c900098d0 tx=0x7f3c90009be0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.679+0000 7f3ca4a62700 1 -- 172.21.15.16:0/2493446233 shutdown_connections 2023-09-30T22:29:59.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.679+0000 7f3ca4a62700 1 --2- 172.21.15.16:0/2493446233 >> v2:172.21.15.16:3300/0 conn(0x7f3ca009e410 0x7f3ca0130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.679+0000 7f3ca4a62700 1 --2- 172.21.15.16:0/2493446233 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3ca009d6f0 0x7f3ca009dac0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.679+0000 7f3ca4a62700 1 --2- 172.21.15.16:0/2493446233 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3ca009e000 0x7f3ca0130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.679+0000 7f3ca4a62700 1 -- 172.21.15.16:0/2493446233 >> 172.21.15.16:0/2493446233 conn(0x7f3ca0066f50 msgr2=0x7f3ca009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:59.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.679+0000 7f3ca4a62700 1 -- 172.21.15.16:0/2493446233 shutdown_connections 2023-09-30T22:29:59.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.679+0000 7f3ca4a62700 1 -- 172.21.15.16:0/2493446233 wait complete. 2023-09-30T22:29:59.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.680+0000 7f3ca4a62700 1 Processor -- start 2023-09-30T22:29:59.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.680+0000 7f3ca4a62700 1 -- start start 2023-09-30T22:29:59.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.681+0000 7f3ca4a62700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3ca009d6f0 0x7f3ca011ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:59.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.681+0000 7f3ca4a62700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3ca009e000 0x7f3ca011e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:59.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.681+0000 7f3ca4a62700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ca009e410 0x7f3ca011e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:59.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.681+0000 7f3ca4a62700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca011fae0 con 0x7f3ca009e410 2023-09-30T22:29:59.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.681+0000 7f3ca4a62700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca0120100 con 0x7f3ca009d6f0 2023-09-30T22:29:59.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.681+0000 7f3ca4a62700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ca0120d30 con 0x7f3ca009e000 2023-09-30T22:29:59.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.681+0000 7f3c9dd9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3ca009e000 0x7f3ca011e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:59.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.681+0000 7f3c9dd9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3ca009e000 0x7f3ca011e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:45086/0 (socket says 172.21.15.16:45086) 2023-09-30T22:29:59.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.681+0000 7f3c9dd9b700 1 -- 172.21.15.16:0/418100101 learned_addr learned my addr 172.21.15.16:0/418100101 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:59.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.681+0000 7f3c9ed9d700 1 --2- 172.21.15.16:0/418100101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ca009e410 0x7f3ca011e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:59.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.681+0000 7f3c9e59c700 1 --2- 172.21.15.16:0/418100101 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3ca009d6f0 0x7f3ca011ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:59.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.681+0000 7f3c9dd9b700 1 -- 172.21.15.16:0/418100101 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3ca009d6f0 msgr2=0x7f3ca011ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:59.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.681+0000 7f3c9dd9b700 1 --2- 172.21.15.16:0/418100101 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3ca009d6f0 0x7f3ca011ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.681+0000 7f3c9dd9b700 1 -- 172.21.15.16:0/418100101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ca009e410 msgr2=0x7f3ca011e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:59.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.681+0000 7f3c9dd9b700 1 --2- 172.21.15.16:0/418100101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ca009e410 0x7f3ca011e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.681+0000 7f3c9dd9b700 1 -- 172.21.15.16:0/418100101 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c900094a0 con 0x7f3ca009e000 2023-09-30T22:29:59.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.681+0000 7f3c9ed9d700 1 --2- 172.21.15.16:0/418100101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ca009e410 0x7f3ca011e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:29:59.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.682+0000 7f3c9dd9b700 1 --2- 172.21.15.16:0/418100101 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3ca009e000 0x7f3ca011e410 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3c9400e810 tx=0x7f3c9400eb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:59.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.682+0000 7f3c8effd700 1 -- 172.21.15.16:0/418100101 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c94004230 con 0x7f3ca009e000 2023-09-30T22:29:59.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.682+0000 7f3c8effd700 1 -- 172.21.15.16:0/418100101 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c94004390 con 0x7f3ca009e000 2023-09-30T22:29:59.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.682+0000 7f3ca4a62700 1 -- 172.21.15.16:0/418100101 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ca0121710 con 0x7f3ca009e000 2023-09-30T22:29:59.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.682+0000 7f3c8effd700 1 -- 172.21.15.16:0/418100101 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c940036a0 con 0x7f3ca009e000 2023-09-30T22:29:59.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.682+0000 7f3ca4a62700 1 -- 172.21.15.16:0/418100101 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ca009c700 con 0x7f3ca009e000 2023-09-30T22:29:59.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.683+0000 7f3c8effd700 1 -- 172.21.15.16:0/418100101 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f3c94003800 con 0x7f3ca009e000 2023-09-30T22:29:59.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.683+0000 7f3ca4a62700 1 -- 172.21.15.16:0/418100101 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ca011ac20 con 0x7f3ca009e000 2023-09-30T22:29:59.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.683+0000 7f3c9e59c700 1 -- 172.21.15.16:0/418100101 >> v1:172.21.15.16:6816/126811 conn(0x7f3c8002f540 legacy=0x7f3c800319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:59.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.683+0000 7f3c8effd700 1 -- 172.21.15.16:0/418100101 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7f3c9403f560 con 0x7f3ca009e000 2023-09-30T22:29:59.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.693+0000 7f3c8effd700 1 -- 172.21.15.16:0/418100101 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3c9401b720 con 0x7f3ca009e000 2023-09-30T22:29:59.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.697+0000 7f59e0ea0700 1 Processor -- start 2023-09-30T22:29:59.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.697+0000 7f59e0ea0700 1 -- start start 2023-09-30T22:29:59.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.698+0000 7f59e0ea0700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f59dc1338a0 0x7f59dc133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:59.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.698+0000 7f59e0ea0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f59dc09de50 0x7f59dc09e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:59.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.698+0000 7f59e0ea0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f59dc09e760 0x7f59dc13a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:59.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.698+0000 7f59e0ea0700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59dc06fef0 con 0x7f59dc1338a0 2023-09-30T22:29:59.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.698+0000 7f59e0ea0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59dc06ef30 con 0x7f59dc09de50 2023-09-30T22:29:59.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.698+0000 7f59e0ea0700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f59dc1334b0 con 0x7f59dc09e760 2023-09-30T22:29:59.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.698+0000 7f59da59c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f59dc1338a0 0x7f59dc133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:59.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.698+0000 7f59da59c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f59dc1338a0 0x7f59dc133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52898/0 (socket says 172.21.15.16:52898) 2023-09-30T22:29:59.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.698+0000 7f59da59c700 1 -- 172.21.15.16:0/2738308992 learned_addr learned my addr 172.21.15.16:0/2738308992 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:59.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.698+0000 7f59d9d9b700 1 --2- 172.21.15.16:0/2738308992 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f59dc09de50 0x7f59dc09e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:59.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.698+0000 7f59dad9d700 1 --2- 172.21.15.16:0/2738308992 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f59dc09e760 0x7f59dc13a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:59.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.698+0000 7f59da59c700 1 -- 172.21.15.16:0/2738308992 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f59dc09e760 msgr2=0x7f59dc13a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:59.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.698+0000 7f59da59c700 1 --2- 172.21.15.16:0/2738308992 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f59dc09e760 0x7f59dc13a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.698+0000 7f59da59c700 1 -- 172.21.15.16:0/2738308992 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f59dc09de50 msgr2=0x7f59dc09e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:29:59.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.698+0000 7f59da59c700 1 --2- 172.21.15.16:0/2738308992 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f59dc09de50 0x7f59dc09e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.698+0000 7f59da59c700 1 -- 172.21.15.16:0/2738308992 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59dc06fc30 con 0x7f59dc1338a0 2023-09-30T22:29:59.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.699+0000 7f59da59c700 1 --2- 172.21.15.16:0/2738308992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59dc1338a0 0x7f59dc133c70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f59cc0098f0 tx=0x7f59cc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3adf1f2f31a0ebd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:59.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.699+0000 7f59d959a700 1 -- 172.21.15.16:0/2738308992 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59cc011830 con 0x7f59dc1338a0 2023-09-30T22:29:59.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.699+0000 7f59d959a700 1 -- 172.21.15.16:0/2738308992 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59cc011990 con 0x7f59dc1338a0 2023-09-30T22:29:59.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.699+0000 7f59d959a700 1 -- 172.21.15.16:0/2738308992 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59cc011d20 con 0x7f59dc1338a0 2023-09-30T22:29:59.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.700+0000 7f59e0ea0700 1 -- 172.21.15.16:0/2738308992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59dc1338a0 msgr2=0x7f59dc133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:59.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.700+0000 7f59e0ea0700 1 --2- 172.21.15.16:0/2738308992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59dc1338a0 0x7f59dc133c70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f59cc0098f0 tx=0x7f59cc009c00 comp rx=0 tx=0).stop 2023-09-30T22:29:59.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.700+0000 7f59e0ea0700 1 -- 172.21.15.16:0/2738308992 shutdown_connections 2023-09-30T22:29:59.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.700+0000 7f59e0ea0700 1 --2- 172.21.15.16:0/2738308992 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f59dc09e760 0x7f59dc13a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.700+0000 7f59e0ea0700 1 --2- 172.21.15.16:0/2738308992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59dc1338a0 0x7f59dc133c70 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.700+0000 7f59e0ea0700 1 --2- 172.21.15.16:0/2738308992 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f59dc09de50 0x7f59dc09e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.700+0000 7f59e0ea0700 1 -- 172.21.15.16:0/2738308992 >> 172.21.15.16:0/2738308992 conn(0x7f59dc0a2aa0 msgr2=0x7f59dc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:59.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.700+0000 7f59e0ea0700 1 -- 172.21.15.16:0/2738308992 shutdown_connections 2023-09-30T22:29:59.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.700+0000 7f59e0ea0700 1 -- 172.21.15.16:0/2738308992 wait complete. 2023-09-30T22:29:59.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.701+0000 7f59e0ea0700 1 Processor -- start 2023-09-30T22:29:59.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.701+0000 7f59e0ea0700 1 -- start start 2023-09-30T22:29:59.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.701+0000 7f59e0ea0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59dc09de50 0x7f59dc11b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:59.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.701+0000 7f59e0ea0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f59dc09e760 0x7f59dc11bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:59.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.702+0000 7f59e0ea0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f59dc1338a0 0x7f59dc11c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:29:59.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.702+0000 7f59e0ea0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59dc11cbe0 con 0x7f59dc09de50 2023-09-30T22:29:59.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.702+0000 7f59e0ea0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59dc11d2f0 con 0x7f59dc1338a0 2023-09-30T22:29:59.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.702+0000 7f59e0ea0700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f59dc11d910 con 0x7f59dc09e760 2023-09-30T22:29:59.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.702+0000 7f59dad9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f59dc1338a0 0x7f59dc11c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:59.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.702+0000 7f59dad9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f59dc1338a0 0x7f59dc11c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54852/0 (socket says 172.21.15.16:54852) 2023-09-30T22:29:59.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.702+0000 7f59dad9d700 1 -- 172.21.15.16:0/3417166517 learned_addr learned my addr 172.21.15.16:0/3417166517 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:29:59.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.702+0000 7f59da59c700 1 --2- 172.21.15.16:0/3417166517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59dc09de50 0x7f59dc11b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:59.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.702+0000 7f59d9d9b700 1 --2- 172.21.15.16:0/3417166517 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f59dc09e760 0x7f59dc11bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:29:59.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.702+0000 7f59da59c700 1 -- 172.21.15.16:0/3417166517 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f59dc09e760 msgr2=0x7f59dc11bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:59.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.702+0000 7f59da59c700 1 --2- 172.21.15.16:0/3417166517 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f59dc09e760 0x7f59dc11bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.702+0000 7f59da59c700 1 -- 172.21.15.16:0/3417166517 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f59dc1338a0 msgr2=0x7f59dc11c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:59.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.702+0000 7f59da59c700 1 --2- 172.21.15.16:0/3417166517 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f59dc1338a0 0x7f59dc11c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.702+0000 7f59da59c700 1 -- 172.21.15.16:0/3417166517 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59cc009550 con 0x7f59dc09de50 2023-09-30T22:29:59.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.703+0000 7f59da59c700 1 --2- 172.21.15.16:0/3417166517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59dc09de50 0x7f59dc11b730 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f59dc06e680 tx=0x7f59cc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:29:59.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.703+0000 7f59caffd700 1 -- 172.21.15.16:0/3417166517 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59cc004240 con 0x7f59dc09de50 2023-09-30T22:29:59.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.703+0000 7f59e0ea0700 1 -- 172.21.15.16:0/3417166517 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59dc11ef20 con 0x7f59dc09de50 2023-09-30T22:29:59.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.703+0000 7f59caffd700 1 -- 172.21.15.16:0/3417166517 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59cc01f070 con 0x7f59dc09de50 2023-09-30T22:29:59.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.703+0000 7f59caffd700 1 -- 172.21.15.16:0/3417166517 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59cc0045e0 con 0x7f59dc09de50 2023-09-30T22:29:59.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.703+0000 7f59e0ea0700 1 -- 172.21.15.16:0/3417166517 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59dc0a0f90 con 0x7f59dc09de50 2023-09-30T22:29:59.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.704+0000 7f59caffd700 1 -- 172.21.15.16:0/3417166517 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f59cc021030 con 0x7f59dc09de50 2023-09-30T22:29:59.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.704+0000 7f59e0ea0700 1 -- 172.21.15.16:0/3417166517 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59dc06f6a0 con 0x7f59dc09de50 2023-09-30T22:29:59.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.704+0000 7f59caffd700 1 -- 172.21.15.16:0/3417166517 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7f59cc040d10 con 0x7f59dc09de50 2023-09-30T22:29:59.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.705+0000 7f59d9d9b700 1 -- 172.21.15.16:0/3417166517 >> v1:172.21.15.16:6816/126811 conn(0x7f59bc02f510 legacy=0x7f59bc0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:59.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.714+0000 7f59caffd700 1 -- 172.21.15.16:0/3417166517 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f59cc0197c0 con 0x7f59dc09de50 2023-09-30T22:29:59.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.741+0000 7fa2a77fe700 1 -- 172.21.15.16:0/104994075 >> v1:172.21.15.16:6816/126811 conn(0x7fa29402f510 legacy=0x7fa2940319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:59.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.775+0000 7fa2adebb700 1 -- 172.21.15.16:0/104994075 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa2a806b9e0 con 0x7fa2a809e930 2023-09-30T22:29:59.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.776+0000 7fa28ffff700 1 -- 172.21.15.16:0/104994075 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa29c01b340 con 0x7fa2a809e930 2023-09-30T22:29:59.776 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:29:59.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.778+0000 7fa2adebb700 1 -- 172.21.15.16:0/104994075 >> v1:172.21.15.16:6816/126811 conn(0x7fa29402f510 legacy=0x7fa2940319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:59.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.778+0000 7fa2adebb700 1 -- 172.21.15.16:0/104994075 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa2a809e930 msgr2=0x7fa2a811dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:59.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.778+0000 7fa2adebb700 1 --2- 172.21.15.16:0/104994075 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa2a809e930 0x7fa2a811dc70 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa29c00c3c0 tx=0x7fa29c00c6d0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.779+0000 7fa2adebb700 1 -- 172.21.15.16:0/104994075 shutdown_connections 2023-09-30T22:29:59.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.779+0000 7fa2adebb700 1 --2- 172.21.15.16:0/104994075 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa2a809de50 0x7fa2a811d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.779+0000 7fa2adebb700 1 --2- 172.21.15.16:0/104994075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2a81338a0 0x7fa2a811e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.779+0000 7fa2adebb700 1 --2- 172.21.15.16:0/104994075 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa2a809e930 0x7fa2a811dc70 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.779+0000 7fa2adebb700 1 -- 172.21.15.16:0/104994075 >> 172.21.15.16:0/104994075 conn(0x7fa2a80a2aa0 msgr2=0x7fa2a81b9410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:59.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.779+0000 7fa2adebb700 1 -- 172.21.15.16:0/104994075 shutdown_connections 2023-09-30T22:29:59.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.779+0000 7fa2adebb700 1 -- 172.21.15.16:0/104994075 wait complete. 2023-09-30T22:29:59.794 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:29:59.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.836+0000 7ff8a9cb9700 1 -- 172.21.15.16:0/2026083664 >> v1:172.21.15.16:6816/126811 conn(0x7ff88802f510 legacy=0x7ff8880319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:59.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.876+0000 7ff8ac71e700 1 -- 172.21.15.16:0/2026083664 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff8a41214f0 con 0x7ff8a409d6f0 2023-09-30T22:29:59.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.877+0000 7ff89affd700 1 -- 172.21.15.16:0/2026083664 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff8a0003e40 con 0x7ff8a409d6f0 2023-09-30T22:29:59.878 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:29:59.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.879+0000 7ff8ac71e700 1 -- 172.21.15.16:0/2026083664 >> v1:172.21.15.16:6816/126811 conn(0x7ff88802f510 legacy=0x7ff8880319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:59.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.879+0000 7ff8ac71e700 1 -- 172.21.15.16:0/2026083664 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff8a409d6f0 msgr2=0x7ff8a411e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:59.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.879+0000 7ff8ac71e700 1 --2- 172.21.15.16:0/2026083664 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff8a409d6f0 0x7ff8a411e070 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff8a000bae0 tx=0x7ff8a0004750 comp rx=0 tx=0).stop 2023-09-30T22:29:59.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.879+0000 7ff8ac71e700 1 -- 172.21.15.16:0/2026083664 shutdown_connections 2023-09-30T22:29:59.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.879+0000 7ff8ac71e700 1 --2- 172.21.15.16:0/2026083664 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff8a409d6f0 0x7ff8a411e070 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.879+0000 7ff8ac71e700 1 --2- 172.21.15.16:0/2026083664 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff8a409e410 0x7ff8a411eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.879+0000 7ff8ac71e700 1 --2- 172.21.15.16:0/2026083664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8a409e000 0x7ff8a411e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.879+0000 7ff8ac71e700 1 -- 172.21.15.16:0/2026083664 >> 172.21.15.16:0/2026083664 conn(0x7ff8a4066f50 msgr2=0x7ff8a4130b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:59.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.879+0000 7ff8ac71e700 1 -- 172.21.15.16:0/2026083664 shutdown_connections 2023-09-30T22:29:59.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.879+0000 7ff8ac71e700 1 -- 172.21.15.16:0/2026083664 wait complete. 2023-09-30T22:29:59.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.884+0000 7f3c9e59c700 1 -- 172.21.15.16:0/418100101 >> v1:172.21.15.16:6816/126811 conn(0x7f3c8002f540 legacy=0x7f3c800319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:59.893 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:29:59.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.906+0000 7f59d9d9b700 1 -- 172.21.15.16:0/3417166517 >> v1:172.21.15.16:6816/126811 conn(0x7f59bc02f510 legacy=0x7f59bc0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:29:59.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.909+0000 7f3ca4a62700 1 -- 172.21.15.16:0/418100101 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3ca0074ea0 con 0x7f3ca009e000 2023-09-30T22:29:59.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.910+0000 7f3c8effd700 1 -- 172.21.15.16:0/418100101 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3c9401b360 con 0x7f3ca009e000 2023-09-30T22:29:59.910 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:29:59.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.912+0000 7f3ca4a62700 1 -- 172.21.15.16:0/418100101 >> v1:172.21.15.16:6816/126811 conn(0x7f3c8002f540 legacy=0x7f3c800319f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:59.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.912+0000 7f3ca4a62700 1 -- 172.21.15.16:0/418100101 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3ca009e000 msgr2=0x7f3ca011e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:59.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.912+0000 7f3ca4a62700 1 --2- 172.21.15.16:0/418100101 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3ca009e000 0x7f3ca011e410 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3c9400e810 tx=0x7f3c9400eb20 comp rx=0 tx=0).stop 2023-09-30T22:29:59.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.912+0000 7f3ca4a62700 1 -- 172.21.15.16:0/418100101 shutdown_connections 2023-09-30T22:29:59.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.912+0000 7f3ca4a62700 1 --2- 172.21.15.16:0/418100101 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3ca009d6f0 0x7f3ca011ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.912+0000 7f3ca4a62700 1 --2- 172.21.15.16:0/418100101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ca009e410 0x7f3ca011e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.912+0000 7f3ca4a62700 1 --2- 172.21.15.16:0/418100101 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3ca009e000 0x7f3ca011e410 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.912+0000 7f3ca4a62700 1 -- 172.21.15.16:0/418100101 >> 172.21.15.16:0/418100101 conn(0x7f3ca0066f50 msgr2=0x7f3ca0130890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:59.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.912+0000 7f3ca4a62700 1 -- 172.21.15.16:0/418100101 shutdown_connections 2023-09-30T22:29:59.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.912+0000 7f3ca4a62700 1 -- 172.21.15.16:0/418100101 wait complete. 2023-09-30T22:29:59.926 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:29:59.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.926+0000 7f59e0ea0700 1 -- 172.21.15.16:0/3417166517 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f59dc11e540 con 0x7f59dc09de50 2023-09-30T22:29:59.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.926+0000 7f59caffd700 1 -- 172.21.15.16:0/3417166517 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f59cc015030 con 0x7f59dc09de50 2023-09-30T22:29:59.927 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:29:59.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.928+0000 7f59e0ea0700 1 -- 172.21.15.16:0/3417166517 >> v1:172.21.15.16:6816/126811 conn(0x7f59bc02f510 legacy=0x7f59bc0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:29:59.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.928+0000 7f59e0ea0700 1 -- 172.21.15.16:0/3417166517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59dc09de50 msgr2=0x7f59dc11b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:29:59.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.928+0000 7f59e0ea0700 1 --2- 172.21.15.16:0/3417166517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59dc09de50 0x7f59dc11b730 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f59dc06e680 tx=0x7f59cc004750 comp rx=0 tx=0).stop 2023-09-30T22:29:59.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.929+0000 7f59e0ea0700 1 -- 172.21.15.16:0/3417166517 shutdown_connections 2023-09-30T22:29:59.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.929+0000 7f59e0ea0700 1 --2- 172.21.15.16:0/3417166517 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f59dc1338a0 0x7f59dc11c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.929+0000 7f59e0ea0700 1 --2- 172.21.15.16:0/3417166517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59dc09de50 0x7f59dc11b730 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.929+0000 7f59e0ea0700 1 --2- 172.21.15.16:0/3417166517 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f59dc09e760 0x7f59dc11bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:29:59.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.929+0000 7f59e0ea0700 1 -- 172.21.15.16:0/3417166517 >> 172.21.15.16:0/3417166517 conn(0x7f59dc0a2aa0 msgr2=0x7f59dc1306c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:29:59.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.929+0000 7f59e0ea0700 1 -- 172.21.15.16:0/3417166517 shutdown_connections 2023-09-30T22:29:59.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:29:59.929+0000 7f59e0ea0700 1 -- 172.21.15.16:0/3417166517 wait complete. 2023-09-30T22:29:59.943 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:30:00.794 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:00.894 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:00.927 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:00.944 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:00.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:00.944+0000 7f9d992c5700 1 Processor -- start 2023-09-30T22:30:00.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:00.945+0000 7f9d992c5700 1 -- start start 2023-09-30T22:30:00.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:00.945+0000 7f9d992c5700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9d9412cc20 0x7f9d9412f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:00.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:00.945+0000 7f9d992c5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9d9412cff0 0x7f9d94137cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:00.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:00.945+0000 7f9d992c5700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d9409a510 con 0x7f9d9412c2f0 2023-09-30T22:30:00.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:00.945+0000 7f9d992c5700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d9412c7d0 con 0x7f9d9412cc20 2023-09-30T22:30:00.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:00.945+0000 7f9d992c5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9409c190 con 0x7f9d9412cff0 2023-09-30T22:30:00.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:00.945+0000 7f9d937fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9d9412cff0 0x7f9d94137cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:00.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:00.945+0000 7f9d92ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9d9412cc20 0x7f9d9412f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:00.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:00.945+0000 7f9d937fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9d9412cff0 0x7f9d94137cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:45120/0 (socket says 172.21.15.16:45120) 2023-09-30T22:30:00.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:00.945+0000 7f9d937fe700 1 -- 172.21.15.16:0/2692993530 learned_addr learned my addr 172.21.15.16:0/2692993530 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:00.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:00.946+0000 7f9d92ffd700 1 -- 172.21.15.16:0/2692993530 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9d9412cff0 msgr2=0x7f9d94137cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:00.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:00.946+0000 7f9d92ffd700 1 --2- 172.21.15.16:0/2692993530 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9d9412cff0 0x7f9d94137cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:00.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:00.946+0000 7f9d92ffd700 1 -- 172.21.15.16:0/2692993530 >> v1:172.21.15.16:6789/0 conn(0x7f9d9412c2f0 legacy=0x7f9d9412fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:00.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:00.946+0000 7f9d92ffd700 1 -- 172.21.15.16:0/2692993530 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d9409b540 con 0x7f9d9412cc20 2023-09-30T22:30:00.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:00.946+0000 7f9d91ffb700 1 -- 172.21.15.16:0/2692993530 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2987433409 0 0) 0x7f9d9409a510 con 0x7f9d9412c2f0 2023-09-30T22:30:00.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:00.946+0000 7f9d937fe700 1 --2- 172.21.15.16:0/2692993530 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9d9412cff0 0x7f9d94137cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:00.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:00.946+0000 7f9d92ffd700 1 --2- 172.21.15.16:0/2692993530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d9412cc20 0x7f9d9412f930 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9d8400a9e0 tx=0x7f9d8400acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2421d9d3c440d9ff server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:00.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:00.946+0000 7f9d91ffb700 1 -- 172.21.15.16:0/2692993530 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8400cb70 con 0x7f9d9412cc20 2023-09-30T22:30:00.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:00.946+0000 7f9d91ffb700 1 -- 172.21.15.16:0/2692993530 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d8400ccd0 con 0x7f9d9412cc20 2023-09-30T22:30:00.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:00.946+0000 7f9d91ffb700 1 -- 172.21.15.16:0/2692993530 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d84018620 con 0x7f9d9412cc20 2023-09-30T22:30:00.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:00.947+0000 7f9d992c5700 1 -- 172.21.15.16:0/2692993530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d9412cc20 msgr2=0x7f9d9412f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:00.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:00.947+0000 7f9d992c5700 1 --2- 172.21.15.16:0/2692993530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d9412cc20 0x7f9d9412f930 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9d8400a9e0 tx=0x7f9d8400acf0 comp rx=0 tx=0).stop 2023-09-30T22:30:00.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:00.947+0000 7f9d992c5700 1 -- 172.21.15.16:0/2692993530 shutdown_connections 2023-09-30T22:30:00.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:00.947+0000 7f9d992c5700 1 --2- 172.21.15.16:0/2692993530 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9d9412cff0 0x7f9d94137cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:00.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:00.947+0000 7f9d992c5700 1 --2- 172.21.15.16:0/2692993530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d9412cc20 0x7f9d9412f930 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:00.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:00.947+0000 7f9d992c5700 1 -- 172.21.15.16:0/2692993530 >> 172.21.15.16:0/2692993530 conn(0x7f9d9409f470 msgr2=0x7f9d940a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:00.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:00.947+0000 7f9d992c5700 1 -- 172.21.15.16:0/2692993530 shutdown_connections 2023-09-30T22:30:00.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:00.947+0000 7f9d992c5700 1 -- 172.21.15.16:0/2692993530 wait complete. 2023-09-30T22:30:00.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:00.948+0000 7f9d992c5700 1 Processor -- start 2023-09-30T22:30:00.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:00.948+0000 7f9d992c5700 1 -- start start 2023-09-30T22:30:00.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:00.949+0000 7f9d992c5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9d9412c2f0 0x7f9d941c8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:00.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:00.949+0000 7f9d992c5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9d9412cc20 0x7f9d941c8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:00.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:00.949+0000 7f9d992c5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d9412cff0 0x7f9d941c8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:00.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:00.949+0000 7f9d992c5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9408bc90 con 0x7f9d9412cff0 2023-09-30T22:30:00.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:00.949+0000 7f9d992c5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d94128090 con 0x7f9d9412cc20 2023-09-30T22:30:00.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:00.949+0000 7f9d992c5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9408b9f0 con 0x7f9d9412c2f0 2023-09-30T22:30:00.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:00.949+0000 7f9d92ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9d9412c2f0 0x7f9d941c8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:00.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:00.949+0000 7f9d937fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d9412cff0 0x7f9d941c8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:00.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:00.949+0000 7f9d92ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9d9412c2f0 0x7f9d941c8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:45128/0 (socket says 172.21.15.16:45128) 2023-09-30T22:30:00.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:00.949+0000 7f9d92ffd700 1 -- 172.21.15.16:0/3517522201 learned_addr learned my addr 172.21.15.16:0/3517522201 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:00.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:00.950+0000 7f9d92ffd700 1 -- 172.21.15.16:0/3517522201 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9d9412cc20 msgr2=0x7f9d941c8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:00.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:00.950+0000 7f9d92ffd700 1 --2- 172.21.15.16:0/3517522201 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9d9412cc20 0x7f9d941c8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:00.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:00.950+0000 7f9d92ffd700 1 -- 172.21.15.16:0/3517522201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d9412cff0 msgr2=0x7f9d941c8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:00.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:00.950+0000 7f9d92ffd700 1 --2- 172.21.15.16:0/3517522201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d9412cff0 0x7f9d941c8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:00.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:00.950+0000 7f9d92ffd700 1 -- 172.21.15.16:0/3517522201 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d8c009b30 con 0x7f9d9412c2f0 2023-09-30T22:30:00.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:00.950+0000 7f9d92ffd700 1 --2- 172.21.15.16:0/3517522201 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9d9412c2f0 0x7f9d941c8130 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9d84000c00 tx=0x7f9d84004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:00.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:00.950+0000 7f9d7b7fe700 1 -- 172.21.15.16:0/3517522201 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8401c040 con 0x7f9d9412c2f0 2023-09-30T22:30:00.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:00.950+0000 7f9d7b7fe700 1 -- 172.21.15.16:0/3517522201 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d84003ad0 con 0x7f9d9412c2f0 2023-09-30T22:30:00.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:00.950+0000 7f9d992c5700 1 -- 172.21.15.16:0/3517522201 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d8400a640 con 0x7f9d9412c2f0 2023-09-30T22:30:00.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:00.950+0000 7f9d7b7fe700 1 -- 172.21.15.16:0/3517522201 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d84003e80 con 0x7f9d9412c2f0 2023-09-30T22:30:00.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:00.950+0000 7f9d992c5700 1 -- 172.21.15.16:0/3517522201 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d94129780 con 0x7f9d9412c2f0 2023-09-30T22:30:00.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:00.952+0000 7f9d992c5700 1 -- 172.21.15.16:0/3517522201 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d940747f0 con 0x7f9d9412c2f0 2023-09-30T22:30:00.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:00.952+0000 7f9d7b7fe700 1 -- 172.21.15.16:0/3517522201 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f9d84018400 con 0x7f9d9412c2f0 2023-09-30T22:30:00.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:00.952+0000 7f9d7b7fe700 1 -- 172.21.15.16:0/3517522201 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7f9d840413c0 con 0x7f9d9412c2f0 2023-09-30T22:30:00.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:00.952+0000 7f9d927fc700 1 -- 172.21.15.16:0/3517522201 >> v1:172.21.15.16:6816/126811 conn(0x7f9d8002f510 legacy=0x7f9d800319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:00.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:00.957+0000 7f9d7b7fe700 1 -- 172.21.15.16:0/3517522201 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9d8400f370 con 0x7f9d9412c2f0 2023-09-30T22:30:01.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.042+0000 7f487c063700 1 Processor -- start 2023-09-30T22:30:01.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.043+0000 7f487c063700 1 -- start start 2023-09-30T22:30:01.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.043+0000 7f487c063700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f487412cf60 0x7f487406f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:01.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.043+0000 7f487c063700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4874130430 0x7f4874137c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:01.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.043+0000 7f487c063700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f487412cd90 con 0x7f487412f8b0 2023-09-30T22:30:01.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.043+0000 7f487c063700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f487412d330 con 0x7f4874130430 2023-09-30T22:30:01.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.043+0000 7f487c063700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4874071780 con 0x7f487412cf60 2023-09-30T22:30:01.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.043+0000 7f4879dff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f487412cf60 0x7f487406f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:01.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.044+0000 7f4879dff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f487412cf60 0x7f487406f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:45144/0 (socket says 172.21.15.16:45144) 2023-09-30T22:30:01.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.044+0000 7f4879dff700 1 -- 172.21.15.16:0/1104847707 learned_addr learned my addr 172.21.15.16:0/1104847707 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:01.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.044+0000 7f48795fe700 1 --1- 172.21.15.16:0/1104847707 >> v1:172.21.15.16:6789/0 conn(0x7f487412f8b0 0x7f487412fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57532/0 (socket says 172.21.15.16:57532) 2023-09-30T22:30:01.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.044+0000 7f487a600700 1 --2- 172.21.15.16:0/1104847707 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4874130430 0x7f4874137c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:01.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.044+0000 7f4879dff700 1 -- 172.21.15.16:0/1104847707 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4874130430 msgr2=0x7f4874137c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:01.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.044+0000 7f4879dff700 1 --2- 172.21.15.16:0/1104847707 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4874130430 0x7f4874137c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:01.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.044+0000 7f4879dff700 1 -- 172.21.15.16:0/1104847707 >> v1:172.21.15.16:6789/0 conn(0x7f487412f8b0 legacy=0x7f487412fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:01.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.044+0000 7f4879dff700 1 -- 172.21.15.16:0/1104847707 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4874071a40 con 0x7f487412cf60 2023-09-30T22:30:01.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.044+0000 7f4878dfd700 1 -- 172.21.15.16:0/1104847707 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2188823860 0 0) 0x7f487412cd90 con 0x7f487412f8b0 2023-09-30T22:30:01.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.044+0000 7f4879dff700 1 --2- 172.21.15.16:0/1104847707 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f487412cf60 0x7f487406f5d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4870009920 tx=0x7f4870009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1ee4a8adf0a777a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:01.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.044+0000 7f4878dfd700 1 -- 172.21.15.16:0/1104847707 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f487000ba60 con 0x7f487412cf60 2023-09-30T22:30:01.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.044+0000 7f4878dfd700 1 -- 172.21.15.16:0/1104847707 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f487000bbc0 con 0x7f487412cf60 2023-09-30T22:30:01.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.044+0000 7f4878dfd700 1 -- 172.21.15.16:0/1104847707 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4870009e70 con 0x7f487412cf60 2023-09-30T22:30:01.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.045+0000 7f487c063700 1 -- 172.21.15.16:0/1104847707 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f487412cf60 msgr2=0x7f487406f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:01.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.045+0000 7f487c063700 1 --2- 172.21.15.16:0/1104847707 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f487412cf60 0x7f487406f5d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f4870009920 tx=0x7f4870009c30 comp rx=0 tx=0).stop 2023-09-30T22:30:01.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.045+0000 7f487c063700 1 -- 172.21.15.16:0/1104847707 shutdown_connections 2023-09-30T22:30:01.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.045+0000 7f487c063700 1 --2- 172.21.15.16:0/1104847707 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4874130430 0x7f4874137c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:01.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.045+0000 7f487c063700 1 --2- 172.21.15.16:0/1104847707 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f487412cf60 0x7f487406f5d0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:01.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.045+0000 7f487c063700 1 -- 172.21.15.16:0/1104847707 >> 172.21.15.16:0/1104847707 conn(0x7f48740a2aa0 msgr2=0x7f48740a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:01.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.045+0000 7f487c063700 1 -- 172.21.15.16:0/1104847707 shutdown_connections 2023-09-30T22:30:01.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.046+0000 7f487c063700 1 -- 172.21.15.16:0/1104847707 wait complete. 2023-09-30T22:30:01.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.047+0000 7f487c063700 1 Processor -- start 2023-09-30T22:30:01.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.047+0000 7f487c063700 1 -- start start 2023-09-30T22:30:01.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.048+0000 7f487c063700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f487412cf60 0x7f487413a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:01.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.048+0000 7f487c063700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f487412f8b0 0x7f487413a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:01.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.048+0000 7f487c063700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4874130430 0x7f487413ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:01.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.048+0000 7f487c063700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487413c140 con 0x7f487412cf60 2023-09-30T22:30:01.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.048+0000 7f487c063700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f487413b7f0 con 0x7f487412f8b0 2023-09-30T22:30:01.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.048+0000 7f487c063700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f487409ed10 con 0x7f4874130430 2023-09-30T22:30:01.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.048+0000 7f4879dff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f487412cf60 0x7f487413a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:01.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.048+0000 7f4879dff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f487412cf60 0x7f487413a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52942/0 (socket says 172.21.15.16:52942) 2023-09-30T22:30:01.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.048+0000 7f4879dff700 1 -- 172.21.15.16:0/194447645 learned_addr learned my addr 172.21.15.16:0/194447645 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:01.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.048+0000 7f48795fe700 1 --2- 172.21.15.16:0/194447645 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f487412f8b0 0x7f487413a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:01.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.048+0000 7f487a600700 1 --2- 172.21.15.16:0/194447645 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4874130430 0x7f487413ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:01.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.048+0000 7f4879dff700 1 -- 172.21.15.16:0/194447645 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4874130430 msgr2=0x7f487413ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:01.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.048+0000 7f4879dff700 1 --2- 172.21.15.16:0/194447645 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4874130430 0x7f487413ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:01.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.048+0000 7f4879dff700 1 -- 172.21.15.16:0/194447645 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f487412f8b0 msgr2=0x7f487413a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:01.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.048+0000 7f4879dff700 1 --2- 172.21.15.16:0/194447645 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f487412f8b0 0x7f487413a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:01.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.048+0000 7f4879dff700 1 -- 172.21.15.16:0/194447645 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4870009550 con 0x7f487412cf60 2023-09-30T22:30:01.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.048+0000 7f4879dff700 1 --2- 172.21.15.16:0/194447645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f487412cf60 0x7f487413a290 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f48700098f0 tx=0x7f4870004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:01.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.049+0000 7f486a7fc700 1 -- 172.21.15.16:0/194447645 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48700045d0 con 0x7f487412cf60 2023-09-30T22:30:01.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.049+0000 7f486a7fc700 1 -- 172.21.15.16:0/194447645 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48700036a0 con 0x7f487412cf60 2023-09-30T22:30:01.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.049+0000 7f487c063700 1 -- 172.21.15.16:0/194447645 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4874067cd0 con 0x7f487412cf60 2023-09-30T22:30:01.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.049+0000 7f486a7fc700 1 -- 172.21.15.16:0/194447645 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4870003a10 con 0x7f487412cf60 2023-09-30T22:30:01.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.049+0000 7f487c063700 1 -- 172.21.15.16:0/194447645 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f487409e2c0 con 0x7f487412cf60 2023-09-30T22:30:01.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.050+0000 7f486a7fc700 1 -- 172.21.15.16:0/194447645 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f4870021070 con 0x7f487412cf60 2023-09-30T22:30:01.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.050+0000 7f487c063700 1 -- 172.21.15.16:0/194447645 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4874074ea0 con 0x7f487412cf60 2023-09-30T22:30:01.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.050+0000 7f486a7fc700 1 -- 172.21.15.16:0/194447645 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7f4870041030 con 0x7f487412cf60 2023-09-30T22:30:01.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.050+0000 7f48795fe700 1 -- 172.21.15.16:0/194447645 >> v1:172.21.15.16:6816/126811 conn(0x7f485802f510 legacy=0x7f48580319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:01.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.055+0000 7f486a7fc700 1 -- 172.21.15.16:0/194447645 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f48700199a0 con 0x7f487412cf60 2023-09-30T22:30:01.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.082+0000 7f79d2958700 1 Processor -- start 2023-09-30T22:30:01.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.082+0000 7f79d2958700 1 -- start start 2023-09-30T22:30:01.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.083+0000 7f79d2958700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f79cc06a770 0x7f79cc09d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:01.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.083+0000 7f79d2958700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f79cc13d1f0 0x7f79cc13f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:01.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.083+0000 7f79d2958700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79cc09a810 con 0x7f79cc09dc30 2023-09-30T22:30:01.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.083+0000 7f79d2958700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79cc06ab40 con 0x7f79cc06a770 2023-09-30T22:30:01.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.083+0000 7f79d2958700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79cc09ebc0 con 0x7f79cc13d1f0 2023-09-30T22:30:01.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.083+0000 7f79cbfff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f79cc06a770 0x7f79cc09d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:01.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.083+0000 7f79cb7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f79cc09dc30 0x7f79cc09e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57544/0 (socket says 172.21.15.16:57544) 2023-09-30T22:30:01.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.083+0000 7f79cb7fe700 1 -- 172.21.15.16:0/1636264244 learned_addr learned my addr 172.21.15.16:0/1636264244 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:30:01.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.083+0000 7f79d0ef5700 1 --2- 172.21.15.16:0/1636264244 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f79cc13d1f0 0x7f79cc13f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:01.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.083+0000 7f79d0ef5700 1 -- 172.21.15.16:0/1636264244 >> v2:172.21.15.16:3300/0 conn(0x7f79cc06a770 msgr2=0x7f79cc09d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:01.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.083+0000 7f79d0ef5700 1 --2- 172.21.15.16:0/1636264244 >> v2:172.21.15.16:3300/0 conn(0x7f79cc06a770 0x7f79cc09d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:01.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.083+0000 7f79caffd700 1 -- 172.21.15.16:0/1636264244 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4145143428 0 0) 0x7f79cc09a810 con 0x7f79cc09dc30 2023-09-30T22:30:01.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.083+0000 7f79d0ef5700 1 -- 172.21.15.16:0/1636264244 >> v1:172.21.15.16:6789/0 conn(0x7f79cc09dc30 legacy=0x7f79cc09e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:01.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.083+0000 7f79d0ef5700 1 -- 172.21.15.16:0/1636264244 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79cc09e920 con 0x7f79cc13d1f0 2023-09-30T22:30:01.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.084+0000 7f79d0ef5700 1 --2- 172.21.15.16:0/1636264244 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f79cc13d1f0 0x7f79cc13f5d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f79c4009d90 tx=0x7f79c400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adb63e78f8a8b95a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:01.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.084+0000 7f79caffd700 1 -- 172.21.15.16:0/1636264244 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c400e070 con 0x7f79cc13d1f0 2023-09-30T22:30:01.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.084+0000 7f79caffd700 1 -- 172.21.15.16:0/1636264244 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79c400bb20 con 0x7f79cc13d1f0 2023-09-30T22:30:01.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.084+0000 7f79caffd700 1 -- 172.21.15.16:0/1636264244 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c4012430 con 0x7f79cc13d1f0 2023-09-30T22:30:01.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.085+0000 7f79d2958700 1 -- 172.21.15.16:0/1636264244 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f79cc13d1f0 msgr2=0x7f79cc13f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:01.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.085+0000 7f79d2958700 1 --2- 172.21.15.16:0/1636264244 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f79cc13d1f0 0x7f79cc13f5d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f79c4009d90 tx=0x7f79c400b330 comp rx=0 tx=0).stop 2023-09-30T22:30:01.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.085+0000 7f79d2958700 1 -- 172.21.15.16:0/1636264244 shutdown_connections 2023-09-30T22:30:01.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.085+0000 7f79d2958700 1 --2- 172.21.15.16:0/1636264244 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f79cc13d1f0 0x7f79cc13f5d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:01.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.085+0000 7f79d2958700 1 --2- 172.21.15.16:0/1636264244 >> v2:172.21.15.16:3300/0 conn(0x7f79cc06a770 0x7f79cc09d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:01.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.085+0000 7f79d2958700 1 -- 172.21.15.16:0/1636264244 >> 172.21.15.16:0/1636264244 conn(0x7f79cc066f50 msgr2=0x7f79cc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:01.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.085+0000 7f79d2958700 1 -- 172.21.15.16:0/1636264244 shutdown_connections 2023-09-30T22:30:01.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.085+0000 7f79d2958700 1 -- 172.21.15.16:0/1636264244 wait complete. 2023-09-30T22:30:01.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.086+0000 7f79d2958700 1 Processor -- start 2023-09-30T22:30:01.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.086+0000 7f79d2958700 1 -- start start 2023-09-30T22:30:01.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.087+0000 7f79d2958700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f79cc06a770 0x7f79cc071660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:01.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.087+0000 7f79d2958700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f79cc09dc30 0x7f79cc13b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:01.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.087+0000 7f79d2958700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79cc13d1f0 0x7f79cc13baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:01.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.087+0000 7f79d2958700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79cc0723e0 con 0x7f79cc13d1f0 2023-09-30T22:30:01.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.087+0000 7f79d2958700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79cc09f1a0 con 0x7f79cc09dc30 2023-09-30T22:30:01.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.087+0000 7f79d2958700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f79cc11de50 con 0x7f79cc06a770 2023-09-30T22:30:01.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.087+0000 7f79cbfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f79cc06a770 0x7f79cc071660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:01.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.087+0000 7f79cbfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f79cc06a770 0x7f79cc071660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:45166/0 (socket says 172.21.15.16:45166) 2023-09-30T22:30:01.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.087+0000 7f79cbfff700 1 -- 172.21.15.16:0/36542659 learned_addr learned my addr 172.21.15.16:0/36542659 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:01.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.087+0000 7f79cb7fe700 1 --2- 172.21.15.16:0/36542659 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f79cc09dc30 0x7f79cc13b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:01.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.087+0000 7f79d0ef5700 1 --2- 172.21.15.16:0/36542659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79cc13d1f0 0x7f79cc13baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:01.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.088+0000 7f79cbfff700 1 -- 172.21.15.16:0/36542659 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f79cc09dc30 msgr2=0x7f79cc13b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:01.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.088+0000 7f79cbfff700 1 --2- 172.21.15.16:0/36542659 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f79cc09dc30 0x7f79cc13b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:01.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.088+0000 7f79cbfff700 1 -- 172.21.15.16:0/36542659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79cc13d1f0 msgr2=0x7f79cc13baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:01.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.088+0000 7f79cbfff700 1 --2- 172.21.15.16:0/36542659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79cc13d1f0 0x7f79cc13baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:01.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.088+0000 7f79cbfff700 1 -- 172.21.15.16:0/36542659 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79c4009a70 con 0x7f79cc06a770 2023-09-30T22:30:01.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.088+0000 7f79d0ef5700 1 --2- 172.21.15.16:0/36542659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79cc13d1f0 0x7f79cc13baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:01.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.088+0000 7f79cbfff700 1 --2- 172.21.15.16:0/36542659 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f79cc06a770 0x7f79cc071660 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f79bc00e9e0 tx=0x7f79bc00ecf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:01.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.088+0000 7f79c8ff9700 1 -- 172.21.15.16:0/36542659 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79bc009710 con 0x7f79cc06a770 2023-09-30T22:30:01.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.088+0000 7f79c8ff9700 1 -- 172.21.15.16:0/36542659 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79bc00ee80 con 0x7f79cc06a770 2023-09-30T22:30:01.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.088+0000 7f79d2958700 1 -- 172.21.15.16:0/36542659 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79cc11e930 con 0x7f79cc06a770 2023-09-30T22:30:01.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.088+0000 7f79c8ff9700 1 -- 172.21.15.16:0/36542659 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79bc00ca50 con 0x7f79cc06a770 2023-09-30T22:30:01.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.088+0000 7f79d2958700 1 -- 172.21.15.16:0/36542659 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79cc072760 con 0x7f79cc06a770 2023-09-30T22:30:01.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.089+0000 7f79c8ff9700 1 -- 172.21.15.16:0/36542659 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f79bc00cbb0 con 0x7f79cc06a770 2023-09-30T22:30:01.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.090+0000 7f79c8ff9700 1 -- 172.21.15.16:0/36542659 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7f79bc03ff80 con 0x7f79cc06a770 2023-09-30T22:30:01.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.090+0000 7f79cb7fe700 1 -- 172.21.15.16:0/36542659 >> v1:172.21.15.16:6816/126811 conn(0x7f79b802f510 legacy=0x7f79b80319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:01.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.091+0000 7f79d2958700 1 -- 172.21.15.16:0/36542659 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79cc074ea0 con 0x7f79cc06a770 2023-09-30T22:30:01.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.095+0000 7fd620637700 1 Processor -- start 2023-09-30T22:30:01.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.095+0000 7fd620637700 1 -- start start 2023-09-30T22:30:01.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.096+0000 7f79c8ff9700 1 -- 172.21.15.16:0/36542659 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f79bc017020 con 0x7f79cc06a770 2023-09-30T22:30:01.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.096+0000 7fd620637700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd61809d6f0 0x7fd61809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:01.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.096+0000 7fd620637700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd61809e000 0x7fd6181339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:01.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.096+0000 7fd620637700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd61809b120 con 0x7fd61809e410 2023-09-30T22:30:01.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.096+0000 7fd620637700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd61806b460 con 0x7fd61809e000 2023-09-30T22:30:01.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.096+0000 7fd620637700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd61809a7e0 con 0x7fd61809d6f0 2023-09-30T22:30:01.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.096+0000 7fd61dbd2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd61809e000 0x7fd6181339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:01.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.096+0000 7fd61dbd2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd61809e000 0x7fd6181339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54910/0 (socket says 172.21.15.16:54910) 2023-09-30T22:30:01.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.096+0000 7fd61dbd2700 1 -- 172.21.15.16:0/2366719872 learned_addr learned my addr 172.21.15.16:0/2366719872 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:01.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.097+0000 7fd61e3d3700 1 --2- 172.21.15.16:0/2366719872 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd61809d6f0 0x7fd61809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:01.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.097+0000 7fd61dbd2700 1 -- 172.21.15.16:0/2366719872 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd61809d6f0 msgr2=0x7fd61809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:01.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.097+0000 7fd61dbd2700 1 --2- 172.21.15.16:0/2366719872 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd61809d6f0 0x7fd61809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:01.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.097+0000 7fd61dbd2700 1 -- 172.21.15.16:0/2366719872 >> v1:172.21.15.16:6789/0 conn(0x7fd61809e410 legacy=0x7fd618133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:01.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.097+0000 7fd61dbd2700 1 -- 172.21.15.16:0/2366719872 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd61809a500 con 0x7fd61809e000 2023-09-30T22:30:01.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.097+0000 7fd61dbd2700 1 --2- 172.21.15.16:0/2366719872 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd61809e000 0x7fd6181339f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd608009920 tx=0x7fd608009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40654f3ef4d72c37 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:01.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.097+0000 7fd61d3d1700 1 -- 172.21.15.16:0/2366719872 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd608004230 con 0x7fd61809e000 2023-09-30T22:30:01.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.097+0000 7fd61d3d1700 1 -- 172.21.15.16:0/2366719872 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd608004390 con 0x7fd61809e000 2023-09-30T22:30:01.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.097+0000 7fd61d3d1700 1 -- 172.21.15.16:0/2366719872 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6080036a0 con 0x7fd61809e000 2023-09-30T22:30:01.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.098+0000 7fd620637700 1 -- 172.21.15.16:0/2366719872 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd61809e000 msgr2=0x7fd6181339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:01.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.098+0000 7fd620637700 1 --2- 172.21.15.16:0/2366719872 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd61809e000 0x7fd6181339f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd608009920 tx=0x7fd608009c30 comp rx=0 tx=0).stop 2023-09-30T22:30:01.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.098+0000 7fd620637700 1 -- 172.21.15.16:0/2366719872 shutdown_connections 2023-09-30T22:30:01.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.098+0000 7fd620637700 1 --2- 172.21.15.16:0/2366719872 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd61809d6f0 0x7fd61809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:01.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.098+0000 7fd620637700 1 --2- 172.21.15.16:0/2366719872 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd61809e000 0x7fd6181339f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:01.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.098+0000 7fd620637700 1 -- 172.21.15.16:0/2366719872 >> 172.21.15.16:0/2366719872 conn(0x7fd618066f50 msgr2=0x7fd61809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:01.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.098+0000 7fd620637700 1 -- 172.21.15.16:0/2366719872 shutdown_connections 2023-09-30T22:30:01.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.098+0000 7fd620637700 1 -- 172.21.15.16:0/2366719872 wait complete. 2023-09-30T22:30:01.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.099+0000 7fd620637700 1 Processor -- start 2023-09-30T22:30:01.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.099+0000 7fd620637700 1 -- start start 2023-09-30T22:30:01.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.100+0000 7fd620637700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd61809d6f0 0x7fd6181225f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:01.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.100+0000 7fd620637700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd61809e000 0x7fd618122b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:01.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.100+0000 7fd620637700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd61809e410 0x7fd618123070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:01.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.100+0000 7fd620637700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd618125410 con 0x7fd61809e410 2023-09-30T22:30:01.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.100+0000 7fd620637700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd618125050 con 0x7fd61809d6f0 2023-09-30T22:30:01.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.100+0000 7fd620637700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd6181260d0 con 0x7fd61809e000 2023-09-30T22:30:01.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.100+0000 7fd61e3d3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd61809d6f0 0x7fd6181225f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:01.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.100+0000 7fd61e3d3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd61809d6f0 0x7fd6181225f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54914/0 (socket says 172.21.15.16:54914) 2023-09-30T22:30:01.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.100+0000 7fd61e3d3700 1 -- 172.21.15.16:0/2445362273 learned_addr learned my addr 172.21.15.16:0/2445362273 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:01.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.100+0000 7fd61dbd2700 1 --2- 172.21.15.16:0/2445362273 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd61809e000 0x7fd618122b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:01.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.100+0000 7fd61e3d3700 1 -- 172.21.15.16:0/2445362273 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd61809e000 msgr2=0x7fd618122b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:01.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.100+0000 7fd61e3d3700 1 --2- 172.21.15.16:0/2445362273 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd61809e000 0x7fd618122b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:01.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.100+0000 7fd61e3d3700 1 -- 172.21.15.16:0/2445362273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd61809e410 msgr2=0x7fd618123070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:01.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.100+0000 7fd61ebd4700 1 --2- 172.21.15.16:0/2445362273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd61809e410 0x7fd618123070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:01.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.100+0000 7fd61e3d3700 1 --2- 172.21.15.16:0/2445362273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd61809e410 0x7fd618123070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:01.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.100+0000 7fd61e3d3700 1 -- 172.21.15.16:0/2445362273 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd608009550 con 0x7fd61809d6f0 2023-09-30T22:30:01.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.100+0000 7fd61ebd4700 1 --2- 172.21.15.16:0/2445362273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd61809e410 0x7fd618123070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:30:01.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.101+0000 7fd61e3d3700 1 --2- 172.21.15.16:0/2445362273 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd61809d6f0 0x7fd6181225f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd61400e9b0 tx=0x7fd61400ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:01.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.101+0000 7fd60effd700 1 -- 172.21.15.16:0/2445362273 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd614009710 con 0x7fd61809d6f0 2023-09-30T22:30:01.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.101+0000 7fd60effd700 1 -- 172.21.15.16:0/2445362273 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd61400c850 con 0x7fd61809d6f0 2023-09-30T22:30:01.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.101+0000 7fd620637700 1 -- 172.21.15.16:0/2445362273 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6180a2d20 con 0x7fd61809d6f0 2023-09-30T22:30:01.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.101+0000 7fd60effd700 1 -- 172.21.15.16:0/2445362273 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61400cbc0 con 0x7fd61809d6f0 2023-09-30T22:30:01.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.101+0000 7fd620637700 1 -- 172.21.15.16:0/2445362273 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd618125df0 con 0x7fd61809d6f0 2023-09-30T22:30:01.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.102+0000 7fd60effd700 1 -- 172.21.15.16:0/2445362273 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fd61400cd20 con 0x7fd61809d6f0 2023-09-30T22:30:01.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.102+0000 7fd61dbd2700 1 -- 172.21.15.16:0/2445362273 >> v1:172.21.15.16:6816/126811 conn(0x7fd5fc02f510 legacy=0x7fd5fc0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:01.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.102+0000 7fd60effd700 1 -- 172.21.15.16:0/2445362273 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7fd614040090 con 0x7fd61809d6f0 2023-09-30T22:30:01.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.102+0000 7fd620637700 1 -- 172.21.15.16:0/2445362273 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd618126920 con 0x7fd61809d6f0 2023-09-30T22:30:01.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.107+0000 7fd60effd700 1 -- 172.21.15.16:0/2445362273 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd614021020 con 0x7fd61809d6f0 2023-09-30T22:30:01.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.153+0000 7f9d927fc700 1 -- 172.21.15.16:0/3517522201 >> v1:172.21.15.16:6816/126811 conn(0x7f9d8002f510 legacy=0x7f9d800319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:01.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.182+0000 7f9d992c5700 1 -- 172.21.15.16:0/3517522201 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9d94071dd0 con 0x7f9d9412c2f0 2023-09-30T22:30:01.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.183+0000 7f9d7b7fe700 1 -- 172.21.15.16:0/3517522201 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9d84026070 con 0x7f9d9412c2f0 2023-09-30T22:30:01.183 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:30:01.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.185+0000 7f9d992c5700 1 -- 172.21.15.16:0/3517522201 >> v1:172.21.15.16:6816/126811 conn(0x7f9d8002f510 legacy=0x7f9d800319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:30:01.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.185+0000 7f9d992c5700 1 -- 172.21.15.16:0/3517522201 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9d9412c2f0 msgr2=0x7f9d941c8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:01.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.185+0000 7f9d992c5700 1 --2- 172.21.15.16:0/3517522201 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9d9412c2f0 0x7f9d941c8130 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9d84000c00 tx=0x7f9d84004750 comp rx=0 tx=0).stop 2023-09-30T22:30:01.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.185+0000 7f9d992c5700 1 -- 172.21.15.16:0/3517522201 shutdown_connections 2023-09-30T22:30:01.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.185+0000 7f9d992c5700 1 --2- 172.21.15.16:0/3517522201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9d9412cff0 0x7f9d941c8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:01.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.185+0000 7f9d992c5700 1 --2- 172.21.15.16:0/3517522201 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9d9412c2f0 0x7f9d941c8130 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:01.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.185+0000 7f9d992c5700 1 --2- 172.21.15.16:0/3517522201 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9d9412cc20 0x7f9d941c8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:01.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.185+0000 7f9d992c5700 1 -- 172.21.15.16:0/3517522201 >> 172.21.15.16:0/3517522201 conn(0x7f9d9409f470 msgr2=0x7f9d940a0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:01.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.185+0000 7f9d992c5700 1 -- 172.21.15.16:0/3517522201 shutdown_connections 2023-09-30T22:30:01.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.185+0000 7f9d992c5700 1 -- 172.21.15.16:0/3517522201 wait complete. 2023-09-30T22:30:01.206 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:30:01.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.251+0000 7f48795fe700 1 -- 172.21.15.16:0/194447645 >> v1:172.21.15.16:6816/126811 conn(0x7f485802f510 legacy=0x7f48580319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:01.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.268+0000 7f487c063700 1 -- 172.21.15.16:0/194447645 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4874071370 con 0x7f487412cf60 2023-09-30T22:30:01.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.269+0000 7f486a7fc700 1 -- 172.21.15.16:0/194447645 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4870029020 con 0x7f487412cf60 2023-09-30T22:30:01.269 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:30:01.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.271+0000 7f487c063700 1 -- 172.21.15.16:0/194447645 >> v1:172.21.15.16:6816/126811 conn(0x7f485802f510 legacy=0x7f48580319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:30:01.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.271+0000 7f487c063700 1 -- 172.21.15.16:0/194447645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f487412cf60 msgr2=0x7f487413a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:01.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.271+0000 7f487c063700 1 --2- 172.21.15.16:0/194447645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f487412cf60 0x7f487413a290 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f48700098f0 tx=0x7f4870004750 comp rx=0 tx=0).stop 2023-09-30T22:30:01.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.271+0000 7f487c063700 1 -- 172.21.15.16:0/194447645 shutdown_connections 2023-09-30T22:30:01.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.271+0000 7f487c063700 1 --2- 172.21.15.16:0/194447645 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4874130430 0x7f487413ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:01.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.271+0000 7f487c063700 1 --2- 172.21.15.16:0/194447645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f487412cf60 0x7f487413a290 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:01.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.271+0000 7f487c063700 1 --2- 172.21.15.16:0/194447645 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f487412f8b0 0x7f487413a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:01.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.271+0000 7f487c063700 1 -- 172.21.15.16:0/194447645 >> 172.21.15.16:0/194447645 conn(0x7f48740a2aa0 msgr2=0x7f487406e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:01.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.271+0000 7f487c063700 1 -- 172.21.15.16:0/194447645 shutdown_connections 2023-09-30T22:30:01.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.271+0000 7f487c063700 1 -- 172.21.15.16:0/194447645 wait complete. 2023-09-30T22:30:01.286 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:30:01.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.291+0000 7f79cb7fe700 1 -- 172.21.15.16:0/36542659 >> v1:172.21.15.16:6816/126811 conn(0x7f79b802f510 legacy=0x7f79b80319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:01.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.303+0000 7fd61dbd2700 1 -- 172.21.15.16:0/2445362273 >> v1:172.21.15.16:6816/126811 conn(0x7fd5fc02f510 legacy=0x7fd5fc0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:01.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.335+0000 7fd620637700 1 -- 172.21.15.16:0/2445362273 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd618074ea0 con 0x7fd61809d6f0 2023-09-30T22:30:01.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.336+0000 7fd60effd700 1 -- 172.21.15.16:0/2445362273 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd61401b3b0 con 0x7fd61809d6f0 2023-09-30T22:30:01.336 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:30:01.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.338+0000 7fd620637700 1 -- 172.21.15.16:0/2445362273 >> v1:172.21.15.16:6816/126811 conn(0x7fd5fc02f510 legacy=0x7fd5fc0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:30:01.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.338+0000 7fd620637700 1 -- 172.21.15.16:0/2445362273 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd61809d6f0 msgr2=0x7fd6181225f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:01.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.338+0000 7fd620637700 1 --2- 172.21.15.16:0/2445362273 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd61809d6f0 0x7fd6181225f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd61400e9b0 tx=0x7fd61400ed70 comp rx=0 tx=0).stop 2023-09-30T22:30:01.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.338+0000 7fd620637700 1 -- 172.21.15.16:0/2445362273 shutdown_connections 2023-09-30T22:30:01.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.338+0000 7fd620637700 1 --2- 172.21.15.16:0/2445362273 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd61809d6f0 0x7fd6181225f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:01.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.338+0000 7fd620637700 1 --2- 172.21.15.16:0/2445362273 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd61809e410 0x7fd618123070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:01.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.338+0000 7fd620637700 1 --2- 172.21.15.16:0/2445362273 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd61809e000 0x7fd618122b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:01.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.338+0000 7fd620637700 1 -- 172.21.15.16:0/2445362273 >> 172.21.15.16:0/2445362273 conn(0x7fd618066f50 msgr2=0x7fd6180710c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:01.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.338+0000 7fd620637700 1 -- 172.21.15.16:0/2445362273 shutdown_connections 2023-09-30T22:30:01.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.338+0000 7fd620637700 1 -- 172.21.15.16:0/2445362273 wait complete. 2023-09-30T22:30:01.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.340+0000 7f79d2958700 1 -- 172.21.15.16:0/36542659 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f79cc11eb30 con 0x7f79cc06a770 2023-09-30T22:30:01.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.340+0000 7f79c8ff9700 1 -- 172.21.15.16:0/36542659 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f79cc11eb30 con 0x7f79cc06a770 2023-09-30T22:30:01.341 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:30:01.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.343+0000 7f79d2958700 1 -- 172.21.15.16:0/36542659 >> v1:172.21.15.16:6816/126811 conn(0x7f79b802f510 legacy=0x7f79b80319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:30:01.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.343+0000 7f79d2958700 1 -- 172.21.15.16:0/36542659 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f79cc06a770 msgr2=0x7f79cc071660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:01.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.343+0000 7f79d2958700 1 --2- 172.21.15.16:0/36542659 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f79cc06a770 0x7f79cc071660 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f79bc00e9e0 tx=0x7f79bc00ecf0 comp rx=0 tx=0).stop 2023-09-30T22:30:01.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.343+0000 7f79d2958700 1 -- 172.21.15.16:0/36542659 shutdown_connections 2023-09-30T22:30:01.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.343+0000 7f79d2958700 1 --2- 172.21.15.16:0/36542659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79cc13d1f0 0x7f79cc13baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:01.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.343+0000 7f79d2958700 1 --2- 172.21.15.16:0/36542659 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f79cc06a770 0x7f79cc071660 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:01.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.343+0000 7f79d2958700 1 --2- 172.21.15.16:0/36542659 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f79cc09dc30 0x7f79cc13b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:01.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.343+0000 7f79d2958700 1 -- 172.21.15.16:0/36542659 >> 172.21.15.16:0/36542659 conn(0x7f79cc066f50 msgr2=0x7f79cc09b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:01.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.343+0000 7f79d2958700 1 -- 172.21.15.16:0/36542659 shutdown_connections 2023-09-30T22:30:01.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:01.343+0000 7f79d2958700 1 -- 172.21.15.16:0/36542659 wait complete. 2023-09-30T22:30:01.353 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:30:01.357 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:30:02.208 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:02.286 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:02.354 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:02.359 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:02.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.359+0000 7fb33a688700 1 Processor -- start 2023-09-30T22:30:02.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.359+0000 7fb33a688700 1 -- start start 2023-09-30T22:30:02.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.359+0000 7fb33a688700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb334130750 0x7fb334130b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:02.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.359+0000 7fb33a688700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb33409e250 0x7fb3341384c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:02.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.359+0000 7fb33a688700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb33409c190 con 0x7fb33409d6f0 2023-09-30T22:30:02.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.359+0000 7fb33a688700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb33409b090 con 0x7fb33409e250 2023-09-30T22:30:02.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.359+0000 7fb33a688700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb33409a6c0 con 0x7fb334130750 2023-09-30T22:30:02.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.360+0000 7fb3337fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb33409d6f0 0x7fb33409db40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57556/0 (socket says 172.21.15.16:57556) 2023-09-30T22:30:02.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.360+0000 7fb338c25700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb33409e250 0x7fb3341384c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:02.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.360+0000 7fb3337fe700 1 -- 172.21.15.16:0/3105616306 learned_addr learned my addr 172.21.15.16:0/3105616306 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:30:02.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.360+0000 7fb333fff700 1 --2- 172.21.15.16:0/3105616306 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb334130750 0x7fb334130b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:02.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.360+0000 7fb338c25700 1 -- 172.21.15.16:0/3105616306 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb334130750 msgr2=0x7fb334130b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:02.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.360+0000 7fb338c25700 1 --2- 172.21.15.16:0/3105616306 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb334130750 0x7fb334130b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.360+0000 7fb338c25700 1 -- 172.21.15.16:0/3105616306 >> v1:172.21.15.16:6789/0 conn(0x7fb33409d6f0 legacy=0x7fb33409db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:02.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.360+0000 7fb338c25700 1 -- 172.21.15.16:0/3105616306 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb334131060 con 0x7fb33409e250 2023-09-30T22:30:02.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.360+0000 7fb338c25700 1 --2- 172.21.15.16:0/3105616306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb33409e250 0x7fb3341384c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb32c009d90 tx=0x7fb32c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=779d183776cf297b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:02.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.360+0000 7fb332ffd700 1 -- 172.21.15.16:0/3105616306 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32c00e070 con 0x7fb33409e250 2023-09-30T22:30:02.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.361+0000 7fb332ffd700 1 -- 172.21.15.16:0/3105616306 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb32c00bb20 con 0x7fb33409e250 2023-09-30T22:30:02.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.361+0000 7fb332ffd700 1 -- 172.21.15.16:0/3105616306 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32c012430 con 0x7fb33409e250 2023-09-30T22:30:02.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.362+0000 7fb33a688700 1 -- 172.21.15.16:0/3105616306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb33409e250 msgr2=0x7fb3341384c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:02.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.362+0000 7fb33a688700 1 --2- 172.21.15.16:0/3105616306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb33409e250 0x7fb3341384c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb32c009d90 tx=0x7fb32c00b330 comp rx=0 tx=0).stop 2023-09-30T22:30:02.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.362+0000 7fb33a688700 1 -- 172.21.15.16:0/3105616306 shutdown_connections 2023-09-30T22:30:02.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.362+0000 7fb33a688700 1 --2- 172.21.15.16:0/3105616306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb33409e250 0x7fb3341384c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.362+0000 7fb33a688700 1 --2- 172.21.15.16:0/3105616306 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb334130750 0x7fb334130b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.362+0000 7fb33a688700 1 -- 172.21.15.16:0/3105616306 >> 172.21.15.16:0/3105616306 conn(0x7fb33409f470 msgr2=0x7fb3340a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:02.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.362+0000 7fb33a688700 1 -- 172.21.15.16:0/3105616306 shutdown_connections 2023-09-30T22:30:02.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.363+0000 7fb33a688700 1 -- 172.21.15.16:0/3105616306 wait complete. 2023-09-30T22:30:02.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.364+0000 7fb33a688700 1 Processor -- start 2023-09-30T22:30:02.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.365+0000 7fb33a688700 1 -- start start 2023-09-30T22:30:02.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.365+0000 7fb33a688700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb33409d6f0 0x7fb334137690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:02.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.365+0000 7fb33a688700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb33409e250 0x7fb334137bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:02.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.365+0000 7fb33a688700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb334130750 0x7fb334138110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:02.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.365+0000 7fb33a688700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb334134d20 con 0x7fb33409e250 2023-09-30T22:30:02.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.365+0000 7fb33a688700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb334134b30 con 0x7fb33409d6f0 2023-09-30T22:30:02.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.365+0000 7fb33a688700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb33409b2b0 con 0x7fb334130750 2023-09-30T22:30:02.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.365+0000 7fb3337fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb33409e250 0x7fb334137bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:02.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.365+0000 7fb3337fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb33409e250 0x7fb334137bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53000/0 (socket says 172.21.15.16:53000) 2023-09-30T22:30:02.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.365+0000 7fb3337fe700 1 -- 172.21.15.16:0/4194520216 learned_addr learned my addr 172.21.15.16:0/4194520216 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:02.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.366+0000 7fb333fff700 1 --2- 172.21.15.16:0/4194520216 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb33409d6f0 0x7fb334137690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:02.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.366+0000 7fb3337fe700 1 -- 172.21.15.16:0/4194520216 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb334130750 msgr2=0x7fb334138110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:02.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.366+0000 7fb3337fe700 1 --2- 172.21.15.16:0/4194520216 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb334130750 0x7fb334138110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.366+0000 7fb3337fe700 1 -- 172.21.15.16:0/4194520216 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb33409d6f0 msgr2=0x7fb334137690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:02.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.366+0000 7fb3337fe700 1 --2- 172.21.15.16:0/4194520216 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb33409d6f0 0x7fb334137690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.366+0000 7fb3337fe700 1 -- 172.21.15.16:0/4194520216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb32c009a70 con 0x7fb33409e250 2023-09-30T22:30:02.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.366+0000 7fb333fff700 1 --2- 172.21.15.16:0/4194520216 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb33409d6f0 0x7fb334137690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:02.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.366+0000 7fb3337fe700 1 --2- 172.21.15.16:0/4194520216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb33409e250 0x7fb334137bd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb32800c480 tx=0x7fb32800c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:02.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.366+0000 7fb330ff9700 1 -- 172.21.15.16:0/4194520216 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb328005d70 con 0x7fb33409e250 2023-09-30T22:30:02.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.366+0000 7fb330ff9700 1 -- 172.21.15.16:0/4194520216 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3280070d0 con 0x7fb33409e250 2023-09-30T22:30:02.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.366+0000 7fb33a688700 1 -- 172.21.15.16:0/4194520216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb334135410 con 0x7fb33409e250 2023-09-30T22:30:02.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.366+0000 7fb330ff9700 1 -- 172.21.15.16:0/4194520216 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb328007440 con 0x7fb33409e250 2023-09-30T22:30:02.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.366+0000 7fb33a688700 1 -- 172.21.15.16:0/4194520216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb334136560 con 0x7fb33409e250 2023-09-30T22:30:02.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.368+0000 7fb33a688700 1 -- 172.21.15.16:0/4194520216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3340747f0 con 0x7fb33409e250 2023-09-30T22:30:02.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.369+0000 7fb330ff9700 1 -- 172.21.15.16:0/4194520216 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fb3280075a0 con 0x7fb33409e250 2023-09-30T22:30:02.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.369+0000 7fb330ff9700 1 -- 172.21.15.16:0/4194520216 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7fb32803f420 con 0x7fb33409e250 2023-09-30T22:30:02.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.369+0000 7fb333fff700 1 -- 172.21.15.16:0/4194520216 >> v1:172.21.15.16:6816/126811 conn(0x7fb32002f510 legacy=0x7fb3200319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:02.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.373+0000 7fb330ff9700 1 -- 172.21.15.16:0/4194520216 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb328019b40 con 0x7fb33409e250 2023-09-30T22:30:02.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.435+0000 7faddf4a9700 1 Processor -- start 2023-09-30T22:30:02.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.435+0000 7faddf4a9700 1 -- start start 2023-09-30T22:30:02.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.436+0000 7faddf4a9700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fadd809d6f0 0x7fadd809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:02.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.436+0000 7faddf4a9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fadd809e000 0x7fadd8130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:02.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.436+0000 7faddf4a9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fadd809e410 0x7fadd8130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:02.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.436+0000 7faddf4a9700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadd809f520 con 0x7fadd809d6f0 2023-09-30T22:30:02.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.436+0000 7faddf4a9700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadd809f130 con 0x7fadd809e000 2023-09-30T22:30:02.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.436+0000 7faddf4a9700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fadd809e930 con 0x7fadd809e410 2023-09-30T22:30:02.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.436+0000 7faddda46700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fadd809e410 0x7fadd8130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:02.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.436+0000 7faddda46700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fadd809e410 0x7fadd8130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:45216/0 (socket says 172.21.15.16:45216) 2023-09-30T22:30:02.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.436+0000 7faddda46700 1 -- 172.21.15.16:0/4259621313 learned_addr learned my addr 172.21.15.16:0/4259621313 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:02.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.436+0000 7faddd245700 1 --2- 172.21.15.16:0/4259621313 >> v2:172.21.15.16:3300/0 conn(0x7fadd809d6f0 0x7fadd809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:02.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.436+0000 7faddca44700 1 --2- 172.21.15.16:0/4259621313 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fadd809e000 0x7fadd8130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:02.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.436+0000 7faddda46700 1 -- 172.21.15.16:0/4259621313 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fadd809e000 msgr2=0x7fadd8130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:02.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.437+0000 7faddda46700 1 --2- 172.21.15.16:0/4259621313 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fadd809e000 0x7fadd8130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.437+0000 7faddda46700 1 -- 172.21.15.16:0/4259621313 >> v2:172.21.15.16:3300/0 conn(0x7fadd809d6f0 msgr2=0x7fadd809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:02.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.437+0000 7faddda46700 1 --2- 172.21.15.16:0/4259621313 >> v2:172.21.15.16:3300/0 conn(0x7fadd809d6f0 0x7fadd809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.437+0000 7faddda46700 1 -- 172.21.15.16:0/4259621313 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd809b2d0 con 0x7fadd809e410 2023-09-30T22:30:02.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.437+0000 7faddd245700 1 --2- 172.21.15.16:0/4259621313 >> v2:172.21.15.16:3300/0 conn(0x7fadd809d6f0 0x7fadd809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:02.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.437+0000 7faddda46700 1 --2- 172.21.15.16:0/4259621313 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fadd809e410 0x7fadd8130ba0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fadd0009e80 tx=0x7fadd000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3fb799d95ef756c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:02.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.437+0000 7fadcffff700 1 -- 172.21.15.16:0/4259621313 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd000e070 con 0x7fadd809e410 2023-09-30T22:30:02.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.437+0000 7fadcffff700 1 -- 172.21.15.16:0/4259621313 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadd000b8d0 con 0x7fadd809e410 2023-09-30T22:30:02.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.437+0000 7fadcffff700 1 -- 172.21.15.16:0/4259621313 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd000bc80 con 0x7fadd809e410 2023-09-30T22:30:02.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.438+0000 7faddf4a9700 1 -- 172.21.15.16:0/4259621313 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fadd809e410 msgr2=0x7fadd8130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:02.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.438+0000 7faddf4a9700 1 --2- 172.21.15.16:0/4259621313 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fadd809e410 0x7fadd8130ba0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fadd0009e80 tx=0x7fadd000b330 comp rx=0 tx=0).stop 2023-09-30T22:30:02.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.438+0000 7faddf4a9700 1 -- 172.21.15.16:0/4259621313 shutdown_connections 2023-09-30T22:30:02.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.438+0000 7faddf4a9700 1 --2- 172.21.15.16:0/4259621313 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fadd809e410 0x7fadd8130ba0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.438+0000 7faddf4a9700 1 --2- 172.21.15.16:0/4259621313 >> v2:172.21.15.16:3300/0 conn(0x7fadd809d6f0 0x7fadd809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.438+0000 7faddf4a9700 1 --2- 172.21.15.16:0/4259621313 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fadd809e000 0x7fadd8130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.438+0000 7faddf4a9700 1 -- 172.21.15.16:0/4259621313 >> 172.21.15.16:0/4259621313 conn(0x7fadd8066f50 msgr2=0x7fadd809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:02.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.438+0000 7faddf4a9700 1 -- 172.21.15.16:0/4259621313 shutdown_connections 2023-09-30T22:30:02.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.438+0000 7faddf4a9700 1 -- 172.21.15.16:0/4259621313 wait complete. 2023-09-30T22:30:02.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.440+0000 7faddf4a9700 1 Processor -- start 2023-09-30T22:30:02.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.440+0000 7faddf4a9700 1 -- start start 2023-09-30T22:30:02.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.440+0000 7faddf4a9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd809d6f0 0x7fadd806e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:02.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.440+0000 7faddf4a9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fadd809e000 0x7fadd806ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:02.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.440+0000 7faddf4a9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fadd809e410 0x7fadd806f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:02.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.440+0000 7faddf4a9700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd80a1e10 con 0x7fadd809d6f0 2023-09-30T22:30:02.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.440+0000 7faddf4a9700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadd809c700 con 0x7fadd809e000 2023-09-30T22:30:02.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.440+0000 7faddf4a9700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fadd8070400 con 0x7fadd809e410 2023-09-30T22:30:02.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.441+0000 7faddd245700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd809d6f0 0x7fadd806e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:02.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.441+0000 7faddd245700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd809d6f0 0x7fadd806e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53014/0 (socket says 172.21.15.16:53014) 2023-09-30T22:30:02.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.441+0000 7faddd245700 1 -- 172.21.15.16:0/2896865729 learned_addr learned my addr 172.21.15.16:0/2896865729 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:02.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.441+0000 7faddca44700 1 --2- 172.21.15.16:0/2896865729 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fadd809e000 0x7fadd806ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:02.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.441+0000 7faddda46700 1 --2- 172.21.15.16:0/2896865729 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fadd809e410 0x7fadd806f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:02.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.441+0000 7faddca44700 1 -- 172.21.15.16:0/2896865729 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fadd809e410 msgr2=0x7fadd806f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:02.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.441+0000 7faddca44700 1 --2- 172.21.15.16:0/2896865729 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fadd809e410 0x7fadd806f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.441+0000 7faddca44700 1 -- 172.21.15.16:0/2896865729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd809d6f0 msgr2=0x7fadd806e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:02.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.441+0000 7faddca44700 1 --2- 172.21.15.16:0/2896865729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd809d6f0 0x7fadd806e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.441+0000 7faddca44700 1 -- 172.21.15.16:0/2896865729 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd40094a0 con 0x7fadd809e000 2023-09-30T22:30:02.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.441+0000 7faddd245700 1 --2- 172.21.15.16:0/2896865729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd809d6f0 0x7fadd806e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:02.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.441+0000 7faddca44700 1 --2- 172.21.15.16:0/2896865729 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fadd809e000 0x7fadd806ed80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fadc800e900 tx=0x7fadc800ec10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:02.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.441+0000 7fadcdffb700 1 -- 172.21.15.16:0/2896865729 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc800ee50 con 0x7fadd809e000 2023-09-30T22:30:02.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.442+0000 7fadcdffb700 1 -- 172.21.15.16:0/2896865729 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadc800c850 con 0x7fadd809e000 2023-09-30T22:30:02.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.442+0000 7faddf4a9700 1 -- 172.21.15.16:0/2896865729 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadd0009b30 con 0x7fadd809e000 2023-09-30T22:30:02.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.442+0000 7fadcdffb700 1 -- 172.21.15.16:0/2896865729 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc800cbc0 con 0x7fadd809e000 2023-09-30T22:30:02.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.442+0000 7faddf4a9700 1 -- 172.21.15.16:0/2896865729 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadd81264d0 con 0x7fadd809e000 2023-09-30T22:30:02.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.443+0000 7fadcdffb700 1 -- 172.21.15.16:0/2896865729 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fadc800cd20 con 0x7fadd809e000 2023-09-30T22:30:02.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.443+0000 7faddf4a9700 1 -- 172.21.15.16:0/2896865729 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadd8125700 con 0x7fadd809e000 2023-09-30T22:30:02.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.443+0000 7fadcdffb700 1 -- 172.21.15.16:0/2896865729 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7fadc803ffc0 con 0x7fadd809e000 2023-09-30T22:30:02.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.443+0000 7faddd245700 1 -- 172.21.15.16:0/2896865729 >> v1:172.21.15.16:6816/126811 conn(0x7fadbc02f510 legacy=0x7fadbc0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:02.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.448+0000 7fadcdffb700 1 -- 172.21.15.16:0/2896865729 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fadc8071750 con 0x7fadd809e000 2023-09-30T22:30:02.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.511+0000 7fc2d4bdf700 1 Processor -- start 2023-09-30T22:30:02.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.511+0000 7fc2d4bdf700 1 -- start start 2023-09-30T22:30:02.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.512+0000 7fc2d4bdf700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2d00a0a30 0x7fc2d00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:02.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.512+0000 7fc2d4bdf700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2d009d630 0x7fc2d00a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:02.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.512+0000 7fc2d4bdf700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2d00a2a70 con 0x7fc2d0133930 2023-09-30T22:30:02.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.512+0000 7fc2d4bdf700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d00a01d0 con 0x7fc2d009d630 2023-09-30T22:30:02.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.512+0000 7fc2d4bdf700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d009e2b0 con 0x7fc2d00a0a30 2023-09-30T22:30:02.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.512+0000 7fc2cdd9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2d009d630 0x7fc2d00a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:02.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.512+0000 7fc2ce59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2d00a0a30 0x7fc2d00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:02.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.512+0000 7fc2cdd9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2d009d630 0x7fc2d00a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54948/0 (socket says 172.21.15.16:54948) 2023-09-30T22:30:02.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.512+0000 7fc2ce59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2d00a0a30 0x7fc2d00a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:45238/0 (socket says 172.21.15.16:45238) 2023-09-30T22:30:02.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.512+0000 7fc2cdd9b700 1 -- 172.21.15.16:0/2206369404 learned_addr learned my addr 172.21.15.16:0/2206369404 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:02.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.513+0000 7fc2ce59c700 1 -- 172.21.15.16:0/2206369404 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2d009d630 msgr2=0x7fc2d00a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:02.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.513+0000 7fc2ce59c700 1 --2- 172.21.15.16:0/2206369404 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2d009d630 0x7fc2d00a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.513+0000 7fc2ce59c700 1 -- 172.21.15.16:0/2206369404 >> v1:172.21.15.16:6789/0 conn(0x7fc2d0133930 legacy=0x7fc2d0133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:02.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.513+0000 7fc2ce59c700 1 -- 172.21.15.16:0/2206369404 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d009dc50 con 0x7fc2d00a0a30 2023-09-30T22:30:02.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.513+0000 7fc2ce59c700 1 --2- 172.21.15.16:0/2206369404 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2d00a0a30 0x7fc2d00a0e00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0009870 tx=0x7fc2c0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b1d3d9f6cfe24b9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:02.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.513+0000 7fc2cd59a700 1 -- 172.21.15.16:0/2206369404 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0011830 con 0x7fc2d00a0a30 2023-09-30T22:30:02.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.513+0000 7fc2cd59a700 1 -- 172.21.15.16:0/2206369404 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2c0011990 con 0x7fc2d00a0a30 2023-09-30T22:30:02.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.513+0000 7fc2cd59a700 1 -- 172.21.15.16:0/2206369404 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0011d20 con 0x7fc2d00a0a30 2023-09-30T22:30:02.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.514+0000 7fc2d4bdf700 1 -- 172.21.15.16:0/2206369404 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2d00a0a30 msgr2=0x7fc2d00a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:02.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.514+0000 7fc2d4bdf700 1 --2- 172.21.15.16:0/2206369404 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2d00a0a30 0x7fc2d00a0e00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0009870 tx=0x7fc2c0009b80 comp rx=0 tx=0).stop 2023-09-30T22:30:02.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.516+0000 7fc2d4bdf700 1 -- 172.21.15.16:0/2206369404 shutdown_connections 2023-09-30T22:30:02.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.516+0000 7fc2d4bdf700 1 --2- 172.21.15.16:0/2206369404 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2d00a0a30 0x7fc2d00a0e00 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.516+0000 7fc2d4bdf700 1 --2- 172.21.15.16:0/2206369404 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2d009d630 0x7fc2d00a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.516+0000 7fc2d4bdf700 1 -- 172.21.15.16:0/2206369404 >> 172.21.15.16:0/2206369404 conn(0x7fc2d00a3050 msgr2=0x7fc2d00a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:02.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.516+0000 7fc2d4bdf700 1 -- 172.21.15.16:0/2206369404 shutdown_connections 2023-09-30T22:30:02.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.517+0000 7fc2d4bdf700 1 -- 172.21.15.16:0/2206369404 wait complete. 2023-09-30T22:30:02.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.517+0000 7fae24005700 1 Processor -- start 2023-09-30T22:30:02.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.517+0000 7fae24005700 1 -- start start 2023-09-30T22:30:02.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.518+0000 7fae24005700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fae1c09d6f0 0x7fae1c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:02.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.518+0000 7fae24005700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fae1c13d650 0x7fae1c13fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:02.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.518+0000 7fae24005700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae1c0992c0 con 0x7fae1c09e000 2023-09-30T22:30:02.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.518+0000 7fae24005700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae1c099520 con 0x7fae1c09d6f0 2023-09-30T22:30:02.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.518+0000 7fae24005700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fae1c0997e0 con 0x7fae1c13d650 2023-09-30T22:30:02.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.518+0000 7fc2d4bdf700 1 Processor -- start 2023-09-30T22:30:02.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.518+0000 7fc2d4bdf700 1 -- start start 2023-09-30T22:30:02.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.518+0000 7fae225a2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fae1c13d650 0x7fae1c13fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:02.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.518+0000 7fae225a2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fae1c13d650 0x7fae1c13fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:45240/0 (socket says 172.21.15.16:45240) 2023-09-30T22:30:02.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.518+0000 7fae225a2700 1 -- 172.21.15.16:0/2337081725 learned_addr learned my addr 172.21.15.16:0/2337081725 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:02.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.519+0000 7fae225a2700 1 -- 172.21.15.16:0/2337081725 >> v2:172.21.15.16:3300/0 conn(0x7fae1c09d6f0 msgr2=0x7fae1c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:02.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.519+0000 7fae225a2700 1 --2- 172.21.15.16:0/2337081725 >> v2:172.21.15.16:3300/0 conn(0x7fae1c09d6f0 0x7fae1c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.519+0000 7fae225a2700 1 -- 172.21.15.16:0/2337081725 >> v1:172.21.15.16:6789/0 conn(0x7fae1c09e000 legacy=0x7fae1c13bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:02.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.519+0000 7fae225a2700 1 -- 172.21.15.16:0/2337081725 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae1c099b30 con 0x7fae1c13d650 2023-09-30T22:30:02.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.519+0000 7fae225a2700 1 --2- 172.21.15.16:0/2337081725 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fae1c13d650 0x7fae1c13fa30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fae14009e50 tx=0x7fae1400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=840c61da01a92a02 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:02.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.519+0000 7fc2d4bdf700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2d009d630 0x7fc2d011f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:02.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.519+0000 7fc2d4bdf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2d00a0a30 0x7fc2d011fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:02.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.519+0000 7fc2d4bdf700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2d0133930 0x7fc2d01c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:02.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.519+0000 7fc2d4bdf700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d0121330 con 0x7fc2d00a0a30 2023-09-30T22:30:02.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.519+0000 7fc2d4bdf700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d011ef60 con 0x7fc2d0133930 2023-09-30T22:30:02.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.519+0000 7fc2d4bdf700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d01221f0 con 0x7fc2d009d630 2023-09-30T22:30:02.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.519+0000 7fae20d9f700 1 -- 172.21.15.16:0/2337081725 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae1400e070 con 0x7fae1c13d650 2023-09-30T22:30:02.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.519+0000 7fae20d9f700 1 -- 172.21.15.16:0/2337081725 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae14003a20 con 0x7fae1c13d650 2023-09-30T22:30:02.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.519+0000 7fae20d9f700 1 -- 172.21.15.16:0/2337081725 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae14003dd0 con 0x7fae1c13d650 2023-09-30T22:30:02.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.519+0000 7fc2ced9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2d0133930 0x7fc2d01c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:02.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.519+0000 7fc2ced9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2d0133930 0x7fc2d01c80b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54960/0 (socket says 172.21.15.16:54960) 2023-09-30T22:30:02.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.519+0000 7fc2ced9d700 1 -- 172.21.15.16:0/3621961936 learned_addr learned my addr 172.21.15.16:0/3621961936 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:02.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.520+0000 7fc2ced9d700 1 -- 172.21.15.16:0/3621961936 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2d009d630 msgr2=0x7fc2d011f540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:30:02.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.520+0000 7fc2ced9d700 1 --2- 172.21.15.16:0/3621961936 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2d009d630 0x7fc2d011f540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.520+0000 7fc2ced9d700 1 -- 172.21.15.16:0/3621961936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2d00a0a30 msgr2=0x7fc2d011fa80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:30:02.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.520+0000 7fc2ced9d700 1 --2- 172.21.15.16:0/3621961936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2d00a0a30 0x7fc2d011fa80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.520+0000 7fc2ced9d700 1 -- 172.21.15.16:0/3621961936 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2c00094a0 con 0x7fc2d0133930 2023-09-30T22:30:02.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.520+0000 7fae24005700 1 -- 172.21.15.16:0/2337081725 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fae1c13d650 msgr2=0x7fae1c13fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:02.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.520+0000 7fae24005700 1 --2- 172.21.15.16:0/2337081725 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fae1c13d650 0x7fae1c13fa30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fae14009e50 tx=0x7fae1400b330 comp rx=0 tx=0).stop 2023-09-30T22:30:02.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.520+0000 7fae24005700 1 -- 172.21.15.16:0/2337081725 shutdown_connections 2023-09-30T22:30:02.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.520+0000 7fae24005700 1 --2- 172.21.15.16:0/2337081725 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fae1c13d650 0x7fae1c13fa30 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.520+0000 7fae24005700 1 --2- 172.21.15.16:0/2337081725 >> v2:172.21.15.16:3300/0 conn(0x7fae1c09d6f0 0x7fae1c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.520+0000 7fae24005700 1 -- 172.21.15.16:0/2337081725 >> 172.21.15.16:0/2337081725 conn(0x7fae1c09f470 msgr2=0x7fae1c0a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:02.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.520+0000 7fc2ced9d700 1 --2- 172.21.15.16:0/3621961936 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2d0133930 0x7fc2d01c80b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc2c800cc50 tx=0x7fc2c800a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:02.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.520+0000 7fae24005700 1 -- 172.21.15.16:0/2337081725 shutdown_connections 2023-09-30T22:30:02.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.520+0000 7fc2beffd700 1 -- 172.21.15.16:0/3621961936 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c800ad80 con 0x7fc2d0133930 2023-09-30T22:30:02.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.520+0000 7fc2beffd700 1 -- 172.21.15.16:0/3621961936 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2c800ce50 con 0x7fc2d0133930 2023-09-30T22:30:02.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.520+0000 7fc2beffd700 1 -- 172.21.15.16:0/3621961936 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c8003c20 con 0x7fc2d0133930 2023-09-30T22:30:02.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.520+0000 7fae24005700 1 -- 172.21.15.16:0/2337081725 wait complete. 2023-09-30T22:30:02.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.521+0000 7fae24005700 1 Processor -- start 2023-09-30T22:30:02.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.521+0000 7fc2d4bdf700 1 -- 172.21.15.16:0/3621961936 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2d0122c60 con 0x7fc2d0133930 2023-09-30T22:30:02.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.521+0000 7fae24005700 1 -- start start 2023-09-30T22:30:02.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.522+0000 7fc2d4bdf700 1 -- 172.21.15.16:0/3621961936 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2d01205c0 con 0x7fc2d0133930 2023-09-30T22:30:02.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.522+0000 7fae24005700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fae1c09d6f0 0x7fae1c0708e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:02.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.522+0000 7fae24005700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fae1c09e000 0x7fae1c070e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:02.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.522+0000 7fae24005700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae1c13d650 0x7fae1c071360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:02.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.522+0000 7fae24005700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1c098c70 con 0x7fae1c13d650 2023-09-30T22:30:02.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.522+0000 7fae24005700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae1c1278d0 con 0x7fae1c09d6f0 2023-09-30T22:30:02.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.522+0000 7fae24005700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fae1c0a0e00 con 0x7fae1c09e000 2023-09-30T22:30:02.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.522+0000 7fae21da1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fae1c09d6f0 0x7fae1c0708e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:02.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.522+0000 7fae21da1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fae1c09d6f0 0x7fae1c0708e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54974/0 (socket says 172.21.15.16:54974) 2023-09-30T22:30:02.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.522+0000 7fae21da1700 1 -- 172.21.15.16:0/2703028703 learned_addr learned my addr 172.21.15.16:0/2703028703 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:02.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.522+0000 7fae215a0700 1 --2- 172.21.15.16:0/2703028703 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fae1c09e000 0x7fae1c070e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:02.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.522+0000 7fae225a2700 1 --2- 172.21.15.16:0/2703028703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae1c13d650 0x7fae1c071360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:02.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.523+0000 7fc2beffd700 1 -- 172.21.15.16:0/3621961936 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fc2c80050d0 con 0x7fc2d0133930 2023-09-30T22:30:02.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.523+0000 7fc2d4bdf700 1 -- 172.21.15.16:0/3621961936 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2d0122060 con 0x7fc2d0133930 2023-09-30T22:30:02.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.523+0000 7fae225a2700 1 -- 172.21.15.16:0/2703028703 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fae1c09e000 msgr2=0x7fae1c070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:02.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.523+0000 7fae225a2700 1 --2- 172.21.15.16:0/2703028703 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fae1c09e000 0x7fae1c070e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.523+0000 7fae225a2700 1 -- 172.21.15.16:0/2703028703 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fae1c09d6f0 msgr2=0x7fae1c0708e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:02.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.523+0000 7fae225a2700 1 --2- 172.21.15.16:0/2703028703 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fae1c09d6f0 0x7fae1c0708e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.523+0000 7fae225a2700 1 -- 172.21.15.16:0/2703028703 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae14009b30 con 0x7fae1c13d650 2023-09-30T22:30:02.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.523+0000 7fae215a0700 1 --2- 172.21.15.16:0/2703028703 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fae1c09e000 0x7fae1c070e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:02.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.523+0000 7fae225a2700 1 --2- 172.21.15.16:0/2703028703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae1c13d650 0x7fae1c071360 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fae14014000 tx=0x7fae14004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:02.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.524+0000 7fae127fc700 1 -- 172.21.15.16:0/2703028703 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae140092e0 con 0x7fae1c13d650 2023-09-30T22:30:02.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.524+0000 7fae127fc700 1 -- 172.21.15.16:0/2703028703 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae14004240 con 0x7fae1c13d650 2023-09-30T22:30:02.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.524+0000 7fae127fc700 1 -- 172.21.15.16:0/2703028703 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae140045f0 con 0x7fae1c13d650 2023-09-30T22:30:02.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.524+0000 7fae24005700 1 -- 172.21.15.16:0/2703028703 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae1c128770 con 0x7fae1c13d650 2023-09-30T22:30:02.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.524+0000 7fae24005700 1 -- 172.21.15.16:0/2703028703 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae1c13d320 con 0x7fae1c13d650 2023-09-30T22:30:02.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.525+0000 7fae127fc700 1 -- 172.21.15.16:0/2703028703 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fae14019070 con 0x7fae1c13d650 2023-09-30T22:30:02.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.525+0000 7fae127fc700 1 -- 172.21.15.16:0/2703028703 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7fae14014030 con 0x7fae1c13d650 2023-09-30T22:30:02.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.525+0000 7fae24005700 1 -- 172.21.15.16:0/2703028703 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae1c128a40 con 0x7fae1c13d650 2023-09-30T22:30:02.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.523+0000 7fc2beffd700 1 -- 172.21.15.16:0/3621961936 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7fc2c803fcf0 con 0x7fc2d0133930 2023-09-30T22:30:02.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.523+0000 7fc2ce59c700 1 -- 172.21.15.16:0/3621961936 >> v1:172.21.15.16:6816/126811 conn(0x7fc2b002f510 legacy=0x7fc2b00319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:02.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.529+0000 7fae21da1700 1 -- 172.21.15.16:0/2703028703 >> v1:172.21.15.16:6816/126811 conn(0x7fae0002f510 legacy=0x7fae000319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:02.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.530+0000 7fae127fc700 1 -- 172.21.15.16:0/2703028703 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fae14017080 con 0x7fae1c13d650 2023-09-30T22:30:02.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.532+0000 7fc2beffd700 1 -- 172.21.15.16:0/3621961936 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc2c8003d80 con 0x7fc2d0133930 2023-09-30T22:30:02.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.570+0000 7fb333fff700 1 -- 172.21.15.16:0/4194520216 >> v1:172.21.15.16:6816/126811 conn(0x7fb32002f510 legacy=0x7fb3200319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:02.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.609+0000 7fb33a688700 1 -- 172.21.15.16:0/4194520216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb33406b250 con 0x7fb33409e250 2023-09-30T22:30:02.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.609+0000 7fb330ff9700 1 -- 172.21.15.16:0/4194520216 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb328019730 con 0x7fb33409e250 2023-09-30T22:30:02.610 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:30:02.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.612+0000 7fb33a688700 1 -- 172.21.15.16:0/4194520216 >> v1:172.21.15.16:6816/126811 conn(0x7fb32002f510 legacy=0x7fb3200319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:30:02.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.612+0000 7fb33a688700 1 -- 172.21.15.16:0/4194520216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb33409e250 msgr2=0x7fb334137bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:02.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.612+0000 7fb33a688700 1 --2- 172.21.15.16:0/4194520216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb33409e250 0x7fb334137bd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb32800c480 tx=0x7fb32800c790 comp rx=0 tx=0).stop 2023-09-30T22:30:02.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.613+0000 7fb33a688700 1 -- 172.21.15.16:0/4194520216 shutdown_connections 2023-09-30T22:30:02.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.613+0000 7fb33a688700 1 --2- 172.21.15.16:0/4194520216 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb33409d6f0 0x7fb334137690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.613+0000 7fb33a688700 1 --2- 172.21.15.16:0/4194520216 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb334130750 0x7fb334138110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.613+0000 7fb33a688700 1 --2- 172.21.15.16:0/4194520216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb33409e250 0x7fb334137bd0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.613+0000 7fb33a688700 1 -- 172.21.15.16:0/4194520216 >> 172.21.15.16:0/4194520216 conn(0x7fb33409f470 msgr2=0x7fb3340992c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:02.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.613+0000 7fb33a688700 1 -- 172.21.15.16:0/4194520216 shutdown_connections 2023-09-30T22:30:02.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.613+0000 7fb33a688700 1 -- 172.21.15.16:0/4194520216 wait complete. 2023-09-30T22:30:02.633 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:30:02.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.644+0000 7faddd245700 1 -- 172.21.15.16:0/2896865729 >> v1:172.21.15.16:6816/126811 conn(0x7fadbc02f510 legacy=0x7fadbc0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:02.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.657+0000 7faddf4a9700 1 -- 172.21.15.16:0/2896865729 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fadd8074ea0 con 0x7fadd809e000 2023-09-30T22:30:02.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.657+0000 7fadcdffb700 1 -- 172.21.15.16:0/2896865729 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fadc8017030 con 0x7fadd809e000 2023-09-30T22:30:02.658 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:30:02.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.659+0000 7faddf4a9700 1 -- 172.21.15.16:0/2896865729 >> v1:172.21.15.16:6816/126811 conn(0x7fadbc02f510 legacy=0x7fadbc0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:30:02.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.659+0000 7faddf4a9700 1 -- 172.21.15.16:0/2896865729 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fadd809e000 msgr2=0x7fadd806ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:02.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.659+0000 7faddf4a9700 1 --2- 172.21.15.16:0/2896865729 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fadd809e000 0x7fadd806ed80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fadc800e900 tx=0x7fadc800ec10 comp rx=0 tx=0).stop 2023-09-30T22:30:02.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.659+0000 7faddf4a9700 1 -- 172.21.15.16:0/2896865729 shutdown_connections 2023-09-30T22:30:02.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.659+0000 7faddf4a9700 1 --2- 172.21.15.16:0/2896865729 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fadd809e410 0x7fadd806f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.659+0000 7faddf4a9700 1 --2- 172.21.15.16:0/2896865729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd809d6f0 0x7fadd806e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.659+0000 7faddf4a9700 1 --2- 172.21.15.16:0/2896865729 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fadd809e000 0x7fadd806ed80 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.659+0000 7faddf4a9700 1 -- 172.21.15.16:0/2896865729 >> 172.21.15.16:0/2896865729 conn(0x7fadd8066f50 msgr2=0x7fadd8070f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:02.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.660+0000 7faddf4a9700 1 -- 172.21.15.16:0/2896865729 shutdown_connections 2023-09-30T22:30:02.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.660+0000 7faddf4a9700 1 -- 172.21.15.16:0/2896865729 wait complete. 2023-09-30T22:30:02.674 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:30:02.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.724+0000 7fc2ce59c700 1 -- 172.21.15.16:0/3621961936 >> v1:172.21.15.16:6816/126811 conn(0x7fc2b002f510 legacy=0x7fc2b00319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:02.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.730+0000 7fae21da1700 1 -- 172.21.15.16:0/2703028703 >> v1:172.21.15.16:6816/126811 conn(0x7fae0002f510 legacy=0x7fae000319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:02.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.760+0000 7fc2d4bdf700 1 -- 172.21.15.16:0/3621961936 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc2d0133710 con 0x7fc2d0133930 2023-09-30T22:30:02.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.761+0000 7fc2beffd700 1 -- 172.21.15.16:0/3621961936 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc2c8019300 con 0x7fc2d0133930 2023-09-30T22:30:02.761 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:30:02.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.763+0000 7fc2d4bdf700 1 -- 172.21.15.16:0/3621961936 >> v1:172.21.15.16:6816/126811 conn(0x7fc2b002f510 legacy=0x7fc2b00319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:30:02.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.763+0000 7fc2d4bdf700 1 -- 172.21.15.16:0/3621961936 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2d0133930 msgr2=0x7fc2d01c80b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:02.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.763+0000 7fc2d4bdf700 1 --2- 172.21.15.16:0/3621961936 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2d0133930 0x7fc2d01c80b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fc2c800cc50 tx=0x7fc2c800a340 comp rx=0 tx=0).stop 2023-09-30T22:30:02.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.763+0000 7fc2d4bdf700 1 -- 172.21.15.16:0/3621961936 shutdown_connections 2023-09-30T22:30:02.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.763+0000 7fc2d4bdf700 1 --2- 172.21.15.16:0/3621961936 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2d0133930 0x7fc2d01c80b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.763+0000 7fc2d4bdf700 1 --2- 172.21.15.16:0/3621961936 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2d009d630 0x7fc2d011f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.763+0000 7fc2d4bdf700 1 --2- 172.21.15.16:0/3621961936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2d00a0a30 0x7fc2d011fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.763+0000 7fc2d4bdf700 1 -- 172.21.15.16:0/3621961936 >> 172.21.15.16:0/3621961936 conn(0x7fc2d00a3050 msgr2=0x7fc2d01bbb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:02.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.763+0000 7fae24005700 1 -- 172.21.15.16:0/2703028703 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fae1c1283a0 con 0x7fae1c13d650 2023-09-30T22:30:02.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.763+0000 7fc2d4bdf700 1 -- 172.21.15.16:0/3621961936 shutdown_connections 2023-09-30T22:30:02.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.763+0000 7fc2d4bdf700 1 -- 172.21.15.16:0/3621961936 wait complete. 2023-09-30T22:30:02.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.764+0000 7fae127fc700 1 -- 172.21.15.16:0/2703028703 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fae14020020 con 0x7fae1c13d650 2023-09-30T22:30:02.768 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:30:02.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.767+0000 7fae24005700 1 -- 172.21.15.16:0/2703028703 >> v1:172.21.15.16:6816/126811 conn(0x7fae0002f510 legacy=0x7fae000319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:30:02.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.767+0000 7fae24005700 1 -- 172.21.15.16:0/2703028703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae1c13d650 msgr2=0x7fae1c071360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:02.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.767+0000 7fae24005700 1 --2- 172.21.15.16:0/2703028703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae1c13d650 0x7fae1c071360 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fae14014000 tx=0x7fae14004ad0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.767+0000 7fae24005700 1 -- 172.21.15.16:0/2703028703 shutdown_connections 2023-09-30T22:30:02.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.767+0000 7fae24005700 1 --2- 172.21.15.16:0/2703028703 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fae1c09d6f0 0x7fae1c0708e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.767+0000 7fae24005700 1 --2- 172.21.15.16:0/2703028703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae1c13d650 0x7fae1c071360 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.767+0000 7fae24005700 1 --2- 172.21.15.16:0/2703028703 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fae1c09e000 0x7fae1c070e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:02.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.767+0000 7fae24005700 1 -- 172.21.15.16:0/2703028703 >> 172.21.15.16:0/2703028703 conn(0x7fae1c09f470 msgr2=0x7fae1c13e130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:02.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.767+0000 7fae24005700 1 -- 172.21.15.16:0/2703028703 shutdown_connections 2023-09-30T22:30:02.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:02.767+0000 7fae24005700 1 -- 172.21.15.16:0/2703028703 wait complete. 2023-09-30T22:30:02.783 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:30:02.785 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:30:03.634 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:03.675 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:03.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.783+0000 7f0ab1d02700 1 Processor -- start 2023-09-30T22:30:03.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.783+0000 7f0ab1d02700 1 -- start start 2023-09-30T22:30:03.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.783+0000 7f0ab1d02700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0aac0a0a30 0x7f0aac0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:03.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.783+0000 7f0ab1d02700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0aac0a1340 0x7f0aac133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:03.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.783+0000 7f0ab1d02700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0aac09edd0 con 0x7f0aac133fa0 2023-09-30T22:30:03.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.783+0000 7f0ab1d02700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0aac09e1f0 con 0x7f0aac0a1340 2023-09-30T22:30:03.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.783+0000 7f0ab1d02700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0aac09dc50 con 0x7f0aac0a0a30 2023-09-30T22:30:03.784 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:03.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.783+0000 7f0aab7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0aac0a0a30 0x7f0aac0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:03.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.783+0000 7f0aaaffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0aac0a1340 0x7f0aac133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:03.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.783+0000 7f0aab7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0aac0a0a30 0x7f0aac0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:45260/0 (socket says 172.21.15.16:45260) 2023-09-30T22:30:03.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.783+0000 7f0aabfff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0aac133fa0 0x7f0aac13ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57586/0 (socket says 172.21.15.16:57586) 2023-09-30T22:30:03.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.783+0000 7f0aab7fe700 1 -- 172.21.15.16:0/1889747849 learned_addr learned my addr 172.21.15.16:0/1889747849 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:03.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.783+0000 7f0aaaffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0aac0a1340 0x7f0aac133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53030/0 (socket says 172.21.15.16:53030) 2023-09-30T22:30:03.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.784+0000 7f0aaaffd700 1 -- 172.21.15.16:0/1889747849 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0aac0a0a30 msgr2=0x7f0aac0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:03.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.784+0000 7f0aaaffd700 1 --2- 172.21.15.16:0/1889747849 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0aac0a0a30 0x7f0aac0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:03.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.784+0000 7f0aaaffd700 1 -- 172.21.15.16:0/1889747849 >> v1:172.21.15.16:6789/0 conn(0x7f0aac133fa0 legacy=0x7f0aac13ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:03.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.784+0000 7f0aaaffd700 1 -- 172.21.15.16:0/1889747849 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aac0a1fe0 con 0x7f0aac0a1340 2023-09-30T22:30:03.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.784+0000 7f0aab7fe700 1 --2- 172.21.15.16:0/1889747849 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0aac0a0a30 0x7f0aac0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:03.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.784+0000 7f0aaa7fc700 1 -- 172.21.15.16:0/1889747849 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2097578867 0 0) 0x7f0aac09edd0 con 0x7f0aac133fa0 2023-09-30T22:30:03.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.784+0000 7f0aaaffd700 1 --2- 172.21.15.16:0/1889747849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0aac0a1340 0x7f0aac133a60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0009840 tx=0x7f0aa0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73d5ada8d3a36cea server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:03.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.784+0000 7f0aaa7fc700 1 -- 172.21.15.16:0/1889747849 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa0011830 con 0x7f0aac0a1340 2023-09-30T22:30:03.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.784+0000 7f0aaa7fc700 1 -- 172.21.15.16:0/1889747849 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aa0011990 con 0x7f0aac0a1340 2023-09-30T22:30:03.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.784+0000 7f0aaa7fc700 1 -- 172.21.15.16:0/1889747849 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa0011d20 con 0x7f0aac0a1340 2023-09-30T22:30:03.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.785+0000 7f0ab1d02700 1 -- 172.21.15.16:0/1889747849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0aac0a1340 msgr2=0x7f0aac133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:03.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.785+0000 7f0ab1d02700 1 --2- 172.21.15.16:0/1889747849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0aac0a1340 0x7f0aac133a60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0009840 tx=0x7f0aa0009b50 comp rx=0 tx=0).stop 2023-09-30T22:30:03.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.785+0000 7f0ab1d02700 1 -- 172.21.15.16:0/1889747849 shutdown_connections 2023-09-30T22:30:03.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.785+0000 7f0ab1d02700 1 --2- 172.21.15.16:0/1889747849 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0aac0a0a30 0x7f0aac0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:03.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.785+0000 7f0ab1d02700 1 --2- 172.21.15.16:0/1889747849 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0aac0a1340 0x7f0aac133a60 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:03.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.785+0000 7f0ab1d02700 1 -- 172.21.15.16:0/1889747849 >> 172.21.15.16:0/1889747849 conn(0x7f0aac0a6550 msgr2=0x7f0aac0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:03.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.785+0000 7f0ab1d02700 1 -- 172.21.15.16:0/1889747849 shutdown_connections 2023-09-30T22:30:03.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.785+0000 7f0ab1d02700 1 -- 172.21.15.16:0/1889747849 wait complete. 2023-09-30T22:30:03.788 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:03.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.786+0000 7f0ab1d02700 1 Processor -- start 2023-09-30T22:30:03.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.787+0000 7f0ab1d02700 1 -- start start 2023-09-30T22:30:03.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.787+0000 7f0ab1d02700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0aac0a0a30 0x7f0aac122820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:03.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.787+0000 7f0ab1d02700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0aac0a1340 0x7f0aac122d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:03.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.787+0000 7f0ab1d02700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0aac133fa0 0x7f0aac1232a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:03.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.787+0000 7f0ab1d02700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aac126f80 con 0x7f0aac133fa0 2023-09-30T22:30:03.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.787+0000 7f0ab1d02700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aac125d60 con 0x7f0aac0a1340 2023-09-30T22:30:03.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.787+0000 7f0ab1d02700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0aac0a3580 con 0x7f0aac0a0a30 2023-09-30T22:30:03.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.787+0000 7f0aab7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0aac0a0a30 0x7f0aac122820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:03.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.787+0000 7f0aab7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0aac0a0a30 0x7f0aac122820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:45268/0 (socket says 172.21.15.16:45268) 2023-09-30T22:30:03.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.788+0000 7f0aab7fe700 1 -- 172.21.15.16:0/3768753881 learned_addr learned my addr 172.21.15.16:0/3768753881 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:03.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.788+0000 7f0aaaffd700 1 --2- 172.21.15.16:0/3768753881 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0aac0a1340 0x7f0aac122d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:03.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.788+0000 7f0aabfff700 1 --2- 172.21.15.16:0/3768753881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0aac133fa0 0x7f0aac1232a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:03.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.788+0000 7f0aab7fe700 1 -- 172.21.15.16:0/3768753881 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0aac0a1340 msgr2=0x7f0aac122d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:03.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.788+0000 7f0aab7fe700 1 --2- 172.21.15.16:0/3768753881 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0aac0a1340 0x7f0aac122d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:03.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.788+0000 7f0aab7fe700 1 -- 172.21.15.16:0/3768753881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0aac133fa0 msgr2=0x7f0aac1232a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:03.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.788+0000 7f0aab7fe700 1 --2- 172.21.15.16:0/3768753881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0aac133fa0 0x7f0aac1232a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:03.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.788+0000 7f0aab7fe700 1 -- 172.21.15.16:0/3768753881 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a9c009550 con 0x7f0aac0a0a30 2023-09-30T22:30:03.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.788+0000 7f0aab7fe700 1 --2- 172.21.15.16:0/3768753881 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0aac0a0a30 0x7f0aac122820 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c00ec10 tx=0x7f0a9c00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:03.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.788+0000 7f0a93fff700 1 -- 172.21.15.16:0/3768753881 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c00c850 con 0x7f0aac0a0a30 2023-09-30T22:30:03.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.788+0000 7f0a93fff700 1 -- 172.21.15.16:0/3768753881 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a9c00c9b0 con 0x7f0aac0a0a30 2023-09-30T22:30:03.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.788+0000 7f0a93fff700 1 -- 172.21.15.16:0/3768753881 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c00cd40 con 0x7f0aac0a0a30 2023-09-30T22:30:03.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.788+0000 7f0ab1d02700 1 -- 172.21.15.16:0/3768753881 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0aa00094a0 con 0x7f0aac0a0a30 2023-09-30T22:30:03.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.789+0000 7f0ab1d02700 1 -- 172.21.15.16:0/3768753881 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0aac124bf0 con 0x7f0aac0a0a30 2023-09-30T22:30:03.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.789+0000 7f0ab1d02700 1 -- 172.21.15.16:0/3768753881 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0aac074ea0 con 0x7f0aac0a0a30 2023-09-30T22:30:03.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.790+0000 7f0a93fff700 1 -- 172.21.15.16:0/3768753881 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f0a9c013470 con 0x7f0aac0a0a30 2023-09-30T22:30:03.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.790+0000 7f0a93fff700 1 -- 172.21.15.16:0/3768753881 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7f0a9c040500 con 0x7f0aac0a0a30 2023-09-30T22:30:03.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.790+0000 7f0aaaffd700 1 -- 172.21.15.16:0/3768753881 >> v1:172.21.15.16:6816/126811 conn(0x7f0a9802f510 legacy=0x7f0a980319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:03.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.795+0000 7f0a93fff700 1 -- 172.21.15.16:0/3768753881 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0a9c013720 con 0x7f0aac0a0a30 2023-09-30T22:30:03.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.827+0000 7f336294c700 1 Processor -- start 2023-09-30T22:30:03.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.827+0000 7f336294c700 1 -- start start 2023-09-30T22:30:03.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.828+0000 7f336294c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f335c136710 0x7f335c138af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:03.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.828+0000 7f336294c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f335c139030 0x7f335c13b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:03.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.828+0000 7f336294c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f335c06ef30 con 0x7f335c1338a0 2023-09-30T22:30:03.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.828+0000 7f336294c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f335c06fef0 con 0x7f335c136710 2023-09-30T22:30:03.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.828+0000 7f336294c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f335c1334b0 con 0x7f335c139030 2023-09-30T22:30:03.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.828+0000 7f335b7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f335c136710 0x7f335c138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:03.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.828+0000 7f335bfff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f335c1338a0 0x7f335c133c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57594/0 (socket says 172.21.15.16:57594) 2023-09-30T22:30:03.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.828+0000 7f335bfff700 1 -- 172.21.15.16:0/26094905 learned_addr learned my addr 172.21.15.16:0/26094905 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:30:03.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.828+0000 7f335affd700 1 -- 172.21.15.16:0/26094905 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 600042582 0 0) 0x7f335c06ef30 con 0x7f335c1338a0 2023-09-30T22:30:03.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.828+0000 7f335b7fe700 1 -- 172.21.15.16:0/26094905 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f335c139030 msgr2=0x7f335c13b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:03.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.828+0000 7f335b7fe700 1 --2- 172.21.15.16:0/26094905 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f335c139030 0x7f335c13b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:03.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.828+0000 7f335b7fe700 1 -- 172.21.15.16:0/26094905 >> v1:172.21.15.16:6789/0 conn(0x7f335c1338a0 legacy=0x7f335c133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:03.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.828+0000 7f335b7fe700 1 -- 172.21.15.16:0/26094905 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f335c06fc30 con 0x7f335c136710 2023-09-30T22:30:03.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.829+0000 7f335b7fe700 1 --2- 172.21.15.16:0/26094905 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f335c136710 0x7f335c138af0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3350009870 tx=0x7f3350009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8560de591fca2515 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:03.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.829+0000 7f335affd700 1 -- 172.21.15.16:0/26094905 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3350011830 con 0x7f335c136710 2023-09-30T22:30:03.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.829+0000 7f335affd700 1 -- 172.21.15.16:0/26094905 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3350011990 con 0x7f335c136710 2023-09-30T22:30:03.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.829+0000 7f335affd700 1 -- 172.21.15.16:0/26094905 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3350011d20 con 0x7f335c136710 2023-09-30T22:30:03.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.829+0000 7f336294c700 1 -- 172.21.15.16:0/26094905 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f335c136710 msgr2=0x7f335c138af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:03.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.829+0000 7f336294c700 1 --2- 172.21.15.16:0/26094905 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f335c136710 0x7f335c138af0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3350009870 tx=0x7f3350009b80 comp rx=0 tx=0).stop 2023-09-30T22:30:03.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.830+0000 7f336294c700 1 -- 172.21.15.16:0/26094905 shutdown_connections 2023-09-30T22:30:03.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.830+0000 7f336294c700 1 --2- 172.21.15.16:0/26094905 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f335c139030 0x7f335c13b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:03.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.830+0000 7f336294c700 1 --2- 172.21.15.16:0/26094905 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f335c136710 0x7f335c138af0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:03.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.830+0000 7f336294c700 1 -- 172.21.15.16:0/26094905 >> 172.21.15.16:0/26094905 conn(0x7f335c0a2aa0 msgr2=0x7f335c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:03.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.830+0000 7f336294c700 1 -- 172.21.15.16:0/26094905 shutdown_connections 2023-09-30T22:30:03.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.830+0000 7f336294c700 1 -- 172.21.15.16:0/26094905 wait complete. 2023-09-30T22:30:03.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.831+0000 7f336294c700 1 Processor -- start 2023-09-30T22:30:03.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.831+0000 7f336294c700 1 -- start start 2023-09-30T22:30:03.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.831+0000 7f336294c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f335c1338a0 0x7f335c1c8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:03.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.831+0000 7f336294c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f335c136710 0x7f335c1c8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:03.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.831+0000 7f336294c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f335c139030 0x7f335c11acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:03.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.831+0000 7f336294c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f335c11e590 con 0x7f335c1338a0 2023-09-30T22:30:03.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.831+0000 7f336294c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f335c11c880 con 0x7f335c136710 2023-09-30T22:30:03.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.831+0000 7f336294c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f335c0a2380 con 0x7f335c139030 2023-09-30T22:30:03.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.832+0000 7f335b7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f335c136710 0x7f335c1c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:03.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.832+0000 7f335b7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f335c136710 0x7f335c1c8960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:55002/0 (socket says 172.21.15.16:55002) 2023-09-30T22:30:03.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.832+0000 7f335b7fe700 1 -- 172.21.15.16:0/257076626 learned_addr learned my addr 172.21.15.16:0/257076626 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:03.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.832+0000 7f3360ee9700 1 --2- 172.21.15.16:0/257076626 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f335c139030 0x7f335c11acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:03.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.832+0000 7f335bfff700 1 --2- 172.21.15.16:0/257076626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f335c1338a0 0x7f335c1c8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:03.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.832+0000 7f335b7fe700 1 -- 172.21.15.16:0/257076626 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f335c139030 msgr2=0x7f335c11acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:03.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.832+0000 7f335b7fe700 1 --2- 172.21.15.16:0/257076626 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f335c139030 0x7f335c11acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:03.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.832+0000 7f335b7fe700 1 -- 172.21.15.16:0/257076626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f335c1338a0 msgr2=0x7f335c1c8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:03.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.832+0000 7f335b7fe700 1 --2- 172.21.15.16:0/257076626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f335c1338a0 0x7f335c1c8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:03.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.832+0000 7f335b7fe700 1 -- 172.21.15.16:0/257076626 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33500094a0 con 0x7f335c136710 2023-09-30T22:30:03.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.832+0000 7f335b7fe700 1 --2- 172.21.15.16:0/257076626 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f335c136710 0x7f335c1c8960 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3350000c00 tx=0x7f3350004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:03.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.833+0000 7f3358ff9700 1 -- 172.21.15.16:0/257076626 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3350004590 con 0x7f335c136710 2023-09-30T22:30:03.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.833+0000 7f3358ff9700 1 -- 172.21.15.16:0/257076626 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f335001f070 con 0x7f335c136710 2023-09-30T22:30:03.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.833+0000 7f336294c700 1 -- 172.21.15.16:0/257076626 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f335c06f640 con 0x7f335c136710 2023-09-30T22:30:03.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.833+0000 7f3358ff9700 1 -- 172.21.15.16:0/257076626 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33500038e0 con 0x7f335c136710 2023-09-30T22:30:03.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.833+0000 7f336294c700 1 -- 172.21.15.16:0/257076626 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f335c11ef20 con 0x7f335c136710 2023-09-30T22:30:03.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.834+0000 7f336294c700 1 -- 172.21.15.16:0/257076626 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f335c074ea0 con 0x7f335c136710 2023-09-30T22:30:03.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.835+0000 7f3358ff9700 1 -- 172.21.15.16:0/257076626 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f3350003a40 con 0x7f335c136710 2023-09-30T22:30:03.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.835+0000 7f335bfff700 1 -- 172.21.15.16:0/257076626 >> v1:172.21.15.16:6816/126811 conn(0x7f334802f540 legacy=0x7f33480319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:03.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.835+0000 7f3358ff9700 1 -- 172.21.15.16:0/257076626 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7f3350021070 con 0x7f335c136710 2023-09-30T22:30:03.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.844+0000 7f3358ff9700 1 -- 172.21.15.16:0/257076626 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f335002f3a0 con 0x7f335c136710 2023-09-30T22:30:03.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.942+0000 7f18679df700 1 Processor -- start 2023-09-30T22:30:03.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.942+0000 7f18679df700 1 -- start start 2023-09-30T22:30:03.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.943+0000 7f18679df700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f186009d630 0x7f18600a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:03.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.943+0000 7f18679df700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1860138d80 0x7f186013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:03.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.943+0000 7f18679df700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18600a2a70 con 0x7f18600a0a30 2023-09-30T22:30:03.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.943+0000 7f18679df700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18600a01d0 con 0x7f186009d630 2023-09-30T22:30:03.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.943+0000 7f18679df700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f186009e2b0 con 0x7f1860138d80 2023-09-30T22:30:03.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.943+0000 7f1864f7a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f186009d630 0x7f18600a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:03.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.943+0000 7f1864f7a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f186009d630 0x7f18600a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53052/0 (socket says 172.21.15.16:53052) 2023-09-30T22:30:03.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.943+0000 7f1864f7a700 1 -- 172.21.15.16:0/3200918529 learned_addr learned my addr 172.21.15.16:0/3200918529 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:03.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.943+0000 7f1864f7a700 1 -- 172.21.15.16:0/3200918529 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1860138d80 msgr2=0x7f186013b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:03.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.944+0000 7f1864f7a700 1 --2- 172.21.15.16:0/3200918529 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1860138d80 0x7f186013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:03.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.944+0000 7f1864f7a700 1 -- 172.21.15.16:0/3200918529 >> v1:172.21.15.16:6789/0 conn(0x7f18600a0a30 legacy=0x7f18600a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:03.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.944+0000 7f1857fff700 1 -- 172.21.15.16:0/3200918529 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3321336308 0 0) 0x7f18600a2a70 con 0x7f18600a0a30 2023-09-30T22:30:03.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.944+0000 7f1864f7a700 1 -- 172.21.15.16:0/3200918529 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f186009dc50 con 0x7f186009d630 2023-09-30T22:30:03.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.944+0000 7f1864f7a700 1 --2- 172.21.15.16:0/3200918529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f186009d630 0x7f18600a1510 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f18500098f0 tx=0x7f1850009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fe5d72db58dd434 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:03.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.944+0000 7f1857fff700 1 -- 172.21.15.16:0/3200918529 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185000b8a0 con 0x7f186009d630 2023-09-30T22:30:03.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.944+0000 7f1857fff700 1 -- 172.21.15.16:0/3200918529 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f185000ba00 con 0x7f186009d630 2023-09-30T22:30:03.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.944+0000 7f1857fff700 1 -- 172.21.15.16:0/3200918529 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185000bd90 con 0x7f186009d630 2023-09-30T22:30:03.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.944+0000 7fd24b39e700 1 Processor -- start 2023-09-30T22:30:03.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.944+0000 7fd24b39e700 1 -- start start 2023-09-30T22:30:03.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.945+0000 7f18679df700 1 -- 172.21.15.16:0/3200918529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f186009d630 msgr2=0x7f18600a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:03.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.945+0000 7f18679df700 1 --2- 172.21.15.16:0/3200918529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f186009d630 0x7f18600a1510 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f18500098f0 tx=0x7f1850009c00 comp rx=0 tx=0).stop 2023-09-30T22:30:03.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.946+0000 7f18679df700 1 -- 172.21.15.16:0/3200918529 shutdown_connections 2023-09-30T22:30:03.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.946+0000 7f18679df700 1 --2- 172.21.15.16:0/3200918529 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1860138d80 0x7f186013b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:03.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.946+0000 7f18679df700 1 --2- 172.21.15.16:0/3200918529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f186009d630 0x7f18600a1510 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:03.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.946+0000 7f18679df700 1 -- 172.21.15.16:0/3200918529 >> 172.21.15.16:0/3200918529 conn(0x7f18600a3050 msgr2=0x7f18600a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:03.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.945+0000 7fd24b39e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd2440a12e0 0x7fd244138bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:03.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.945+0000 7fd24b39e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd244139110 0x7fd24413b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:03.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.945+0000 7fd24b39e700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd24409e770 con 0x7fd2440a0800 2023-09-30T22:30:03.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.945+0000 7fd24b39e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2440a1aa0 con 0x7fd244139110 2023-09-30T22:30:03.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.945+0000 7fd24b39e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd24409cc80 con 0x7fd2440a12e0 2023-09-30T22:30:03.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.945+0000 7fd24a39c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd2440a0800 0x7fd2440a0bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57614/0 (socket says 172.21.15.16:57614) 2023-09-30T22:30:03.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.945+0000 7fd24a39c700 1 -- 172.21.15.16:0/2896649302 learned_addr learned my addr 172.21.15.16:0/2896649302 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:30:03.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.945+0000 7fd249b9b700 1 --2- 172.21.15.16:0/2896649302 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd2440a12e0 0x7fd244138bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:03.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.946+0000 7fd24939a700 1 -- 172.21.15.16:0/2896649302 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1378935493 0 0) 0x7fd24409e770 con 0x7fd2440a0800 2023-09-30T22:30:03.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.946+0000 7fd24939a700 1 -- 172.21.15.16:0/2896649302 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2300035b0 con 0x7fd2440a0800 2023-09-30T22:30:03.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.946+0000 7fd24ab9d700 1 --2- 172.21.15.16:0/2896649302 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd244139110 0x7fd24413b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:03.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.946+0000 7fd249b9b700 1 -- 172.21.15.16:0/2896649302 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd244139110 msgr2=0x7fd24413b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:03.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.946+0000 7fd249b9b700 1 --2- 172.21.15.16:0/2896649302 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd244139110 0x7fd24413b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:03.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.946+0000 7fd249b9b700 1 -- 172.21.15.16:0/2896649302 >> v1:172.21.15.16:6789/0 conn(0x7fd2440a0800 legacy=0x7fd2440a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:03.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.946+0000 7fd249b9b700 1 -- 172.21.15.16:0/2896649302 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2440a1c70 con 0x7fd2440a12e0 2023-09-30T22:30:03.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.946+0000 7f18679df700 1 -- 172.21.15.16:0/3200918529 shutdown_connections 2023-09-30T22:30:03.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.946+0000 7f18679df700 1 -- 172.21.15.16:0/3200918529 wait complete. 2023-09-30T22:30:03.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.946+0000 7fd249b9b700 1 --2- 172.21.15.16:0/2896649302 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd2440a12e0 0x7fd244138bd0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd2380098a0 tx=0x7fd238009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf24cb27149d101 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:03.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.946+0000 7fd24939a700 1 -- 172.21.15.16:0/2896649302 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd238011830 con 0x7fd2440a12e0 2023-09-30T22:30:03.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.946+0000 7fd24939a700 1 -- 172.21.15.16:0/2896649302 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd238011990 con 0x7fd2440a12e0 2023-09-30T22:30:03.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.946+0000 7fd24939a700 1 -- 172.21.15.16:0/2896649302 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd238011d00 con 0x7fd2440a12e0 2023-09-30T22:30:03.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.947+0000 7f18679df700 1 Processor -- start 2023-09-30T22:30:03.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.947+0000 7fd24b39e700 1 -- 172.21.15.16:0/2896649302 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd2440a12e0 msgr2=0x7fd244138bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:03.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.947+0000 7fd24b39e700 1 --2- 172.21.15.16:0/2896649302 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd2440a12e0 0x7fd244138bd0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd2380098a0 tx=0x7fd238009bb0 comp rx=0 tx=0).stop 2023-09-30T22:30:03.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.947+0000 7f18679df700 1 -- start start 2023-09-30T22:30:03.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.947+0000 7fd24b39e700 1 -- 172.21.15.16:0/2896649302 shutdown_connections 2023-09-30T22:30:03.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.947+0000 7fd24b39e700 1 --2- 172.21.15.16:0/2896649302 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd244139110 0x7fd24413b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:03.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.947+0000 7fd24b39e700 1 --2- 172.21.15.16:0/2896649302 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd2440a12e0 0x7fd244138bd0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:03.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.947+0000 7fd24b39e700 1 -- 172.21.15.16:0/2896649302 >> 172.21.15.16:0/2896649302 conn(0x7fd2440a6140 msgr2=0x7fd2440a6540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:03.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.948+0000 7f18679df700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f186009d630 0x7f186012c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:03.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.948+0000 7f18679df700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f18600a0a30 0x7f186012c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:03.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.948+0000 7f18679df700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1860138d80 0x7f186012ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:03.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.948+0000 7f18679df700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f186012d620 con 0x7f186009d630 2023-09-30T22:30:03.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.948+0000 7f18679df700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f186012ef90 con 0x7f18600a0a30 2023-09-30T22:30:03.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.948+0000 7f18679df700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f186012d8e0 con 0x7f1860138d80 2023-09-30T22:30:03.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.948+0000 7f1865f7c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1860138d80 0x7f186012ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:03.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.948+0000 7f1865f7c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1860138d80 0x7f186012ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:45302/0 (socket says 172.21.15.16:45302) 2023-09-30T22:30:03.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.948+0000 7f1865f7c700 1 -- 172.21.15.16:0/720291047 learned_addr learned my addr 172.21.15.16:0/720291047 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:03.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.948+0000 7f1864f7a700 1 --2- 172.21.15.16:0/720291047 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f18600a0a30 0x7f186012c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:03.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.948+0000 7f186577b700 1 --2- 172.21.15.16:0/720291047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f186009d630 0x7f186012c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:03.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.948+0000 7f1865f7c700 1 -- 172.21.15.16:0/720291047 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f18600a0a30 msgr2=0x7f186012c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:03.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.948+0000 7f1865f7c700 1 --2- 172.21.15.16:0/720291047 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f18600a0a30 0x7f186012c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:03.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.948+0000 7f1865f7c700 1 -- 172.21.15.16:0/720291047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f186009d630 msgr2=0x7f186012c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:03.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.948+0000 7f1865f7c700 1 --2- 172.21.15.16:0/720291047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f186009d630 0x7f186012c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:03.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.948+0000 7f1865f7c700 1 -- 172.21.15.16:0/720291047 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1850009550 con 0x7f1860138d80 2023-09-30T22:30:03.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.948+0000 7f186577b700 1 --2- 172.21.15.16:0/720291047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f186009d630 0x7f186012c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:30:03.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.948+0000 7fd24b39e700 1 -- 172.21.15.16:0/2896649302 shutdown_connections 2023-09-30T22:30:03.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.949+0000 7f1865f7c700 1 --2- 172.21.15.16:0/720291047 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1860138d80 0x7f186012ccb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f185800b9e0 tx=0x7f185800bcf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:03.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.949+0000 7fd24b39e700 1 -- 172.21.15.16:0/2896649302 wait complete. 2023-09-30T22:30:03.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.949+0000 7f1855ffb700 1 -- 172.21.15.16:0/720291047 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185800c930 con 0x7f1860138d80 2023-09-30T22:30:03.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.949+0000 7f1855ffb700 1 -- 172.21.15.16:0/720291047 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f185800cad0 con 0x7f1860138d80 2023-09-30T22:30:03.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.949+0000 7f1855ffb700 1 -- 172.21.15.16:0/720291047 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185800ce80 con 0x7f1860138d80 2023-09-30T22:30:03.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.949+0000 7f18679df700 1 -- 172.21.15.16:0/720291047 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f186012e9f0 con 0x7f1860138d80 2023-09-30T22:30:03.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.949+0000 7f18679df700 1 -- 172.21.15.16:0/720291047 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f186012e2b0 con 0x7f1860138d80 2023-09-30T22:30:03.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.950+0000 7f1855ffb700 1 -- 172.21.15.16:0/720291047 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f1858014440 con 0x7f1860138d80 2023-09-30T22:30:03.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.951+0000 7f186577b700 1 -- 172.21.15.16:0/720291047 >> v1:172.21.15.16:6816/126811 conn(0x7f184402f510 legacy=0x7f18440319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:03.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.950+0000 7fd24b39e700 1 Processor -- start 2023-09-30T22:30:03.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.951+0000 7f1855ffb700 1 -- 172.21.15.16:0/720291047 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7f1858006e40 con 0x7f1860138d80 2023-09-30T22:30:03.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.951+0000 7f18679df700 1 -- 172.21.15.16:0/720291047 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18400052e0 con 0x7f1860138d80 2023-09-30T22:30:03.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.951+0000 7fd24b39e700 1 -- start start 2023-09-30T22:30:03.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.952+0000 7fd24b39e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd2440a0800 0x7fd2441260d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:03.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.952+0000 7fd24b39e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2440a12e0 0x7fd244126610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:03.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.952+0000 7fd24b39e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd244139110 0x7fd244126b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:03.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.952+0000 7fd24b39e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd244129240 con 0x7fd2440a12e0 2023-09-30T22:30:03.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.952+0000 7fd24b39e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd24412a670 con 0x7fd2440a0800 2023-09-30T22:30:03.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.952+0000 7fd24b39e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd2441389e0 con 0x7fd244139110 2023-09-30T22:30:03.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.952+0000 7fd24ab9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd244139110 0x7fd244126b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:03.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.952+0000 7fd24ab9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd244139110 0x7fd244126b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:45316/0 (socket says 172.21.15.16:45316) 2023-09-30T22:30:03.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.952+0000 7fd24a39c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd2440a0800 0x7fd2441260d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:03.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.952+0000 7fd24ab9d700 1 -- 172.21.15.16:0/1743906116 learned_addr learned my addr 172.21.15.16:0/1743906116 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:03.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.952+0000 7fd24a39c700 1 -- 172.21.15.16:0/1743906116 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd244139110 msgr2=0x7fd244126b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:03.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.952+0000 7fd24a39c700 1 --2- 172.21.15.16:0/1743906116 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd244139110 0x7fd244126b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:03.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.952+0000 7fd24a39c700 1 -- 172.21.15.16:0/1743906116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2440a12e0 msgr2=0x7fd244126610 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:30:03.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.952+0000 7fd24a39c700 1 --2- 172.21.15.16:0/1743906116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2440a12e0 0x7fd244126610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:03.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.952+0000 7fd24a39c700 1 -- 172.21.15.16:0/1743906116 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2380094a0 con 0x7fd2440a0800 2023-09-30T22:30:03.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.952+0000 7fd24ab9d700 1 --2- 172.21.15.16:0/1743906116 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd244139110 0x7fd244126b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:03.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.953+0000 7fd24a39c700 1 --2- 172.21.15.16:0/1743906116 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd2440a0800 0x7fd2441260d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd24000c490 tx=0x7fd24000c7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:03.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.953+0000 7fd236ffd700 1 -- 172.21.15.16:0/1743906116 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd240005e30 con 0x7fd2440a0800 2023-09-30T22:30:03.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.953+0000 7fd236ffd700 1 -- 172.21.15.16:0/1743906116 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2400070d0 con 0x7fd2440a0800 2023-09-30T22:30:03.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.953+0000 7fd236ffd700 1 -- 172.21.15.16:0/1743906116 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd240007460 con 0x7fd2440a0800 2023-09-30T22:30:03.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.953+0000 7fd24b39e700 1 -- 172.21.15.16:0/1743906116 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd244129450 con 0x7fd2440a0800 2023-09-30T22:30:03.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.953+0000 7fd24b39e700 1 -- 172.21.15.16:0/1743906116 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2441299f0 con 0x7fd2440a0800 2023-09-30T22:30:03.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.954+0000 7fd24b39e700 1 -- 172.21.15.16:0/1743906116 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd244129830 con 0x7fd2440a0800 2023-09-30T22:30:03.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.955+0000 7fd236ffd700 1 -- 172.21.15.16:0/1743906116 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fd2400075c0 con 0x7fd2440a0800 2023-09-30T22:30:03.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.955+0000 7fd236ffd700 1 -- 172.21.15.16:0/1743906116 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7fd240040c30 con 0x7fd2440a0800 2023-09-30T22:30:03.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.955+0000 7fd249b9b700 1 -- 172.21.15.16:0/1743906116 >> v1:172.21.15.16:6816/126811 conn(0x7fd22802f540 legacy=0x7fd2280319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:03.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.957+0000 7f1855ffb700 1 -- 172.21.15.16:0/720291047 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1858019030 con 0x7f1860138d80 2023-09-30T22:30:03.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.964+0000 7fd236ffd700 1 -- 172.21.15.16:0/1743906116 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd240044060 con 0x7fd2440a0800 2023-09-30T22:30:03.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:03.991+0000 7f0aaaffd700 1 -- 172.21.15.16:0/3768753881 >> v1:172.21.15.16:6816/126811 conn(0x7f0a9802f510 legacy=0x7f0a980319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:04.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:04.036+0000 7f335bfff700 1 -- 172.21.15.16:0/257076626 >> v1:172.21.15.16:6816/126811 conn(0x7f334802f540 legacy=0x7f33480319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:04.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:04.053+0000 7f0ab1d02700 1 -- 172.21.15.16:0/3768753881 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f0aac0a28e0 con 0x7f0aac0a0a30 2023-09-30T22:30:04.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:04.053+0000 7f0a93fff700 1 -- 172.21.15.16:0/3768753881 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0a9c017020 con 0x7f0aac0a0a30 2023-09-30T22:30:04.054 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:30:04.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:04.057+0000 7f0ab1d02700 1 -- 172.21.15.16:0/3768753881 >> v1:172.21.15.16:6816/126811 conn(0x7f0a9802f510 legacy=0x7f0a980319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:30:04.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:04.057+0000 7f0ab1d02700 1 -- 172.21.15.16:0/3768753881 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0aac0a0a30 msgr2=0x7f0aac122820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:04.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:04.057+0000 7f0ab1d02700 1 --2- 172.21.15.16:0/3768753881 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0aac0a0a30 0x7f0aac122820 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c00ec10 tx=0x7f0a9c00ef20 comp rx=0 tx=0).stop 2023-09-30T22:30:04.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:04.058+0000 7f0ab1d02700 1 -- 172.21.15.16:0/3768753881 shutdown_connections 2023-09-30T22:30:04.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:04.058+0000 7f0ab1d02700 1 --2- 172.21.15.16:0/3768753881 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0aac133fa0 0x7f0aac1232a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:04.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:04.058+0000 7f0ab1d02700 1 --2- 172.21.15.16:0/3768753881 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0aac0a0a30 0x7f0aac122820 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:04.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:04.058+0000 7f0ab1d02700 1 --2- 172.21.15.16:0/3768753881 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0aac0a1340 0x7f0aac122d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:04.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:04.058+0000 7f0ab1d02700 1 -- 172.21.15.16:0/3768753881 >> 172.21.15.16:0/3768753881 conn(0x7f0aac0a6550 msgr2=0x7f0aac11c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:04.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:04.058+0000 7f0ab1d02700 1 -- 172.21.15.16:0/3768753881 shutdown_connections 2023-09-30T22:30:04.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:04.058+0000 7f0ab1d02700 1 -- 172.21.15.16:0/3768753881 wait complete. 2023-09-30T22:30:04.073 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:30:04.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:04.075+0000 7f336294c700 1 -- 172.21.15.16:0/257076626 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f335c11e7f0 con 0x7f335c136710 2023-09-30T22:30:04.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:04.076+0000 7f3358ff9700 1 -- 172.21.15.16:0/257076626 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f335002b020 con 0x7f335c136710 2023-09-30T22:30:04.076 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:30:04.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:04.078+0000 7f336294c700 1 -- 172.21.15.16:0/257076626 >> v1:172.21.15.16:6816/126811 conn(0x7f334802f540 legacy=0x7f33480319f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:30:04.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:04.078+0000 7f336294c700 1 -- 172.21.15.16:0/257076626 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f335c136710 msgr2=0x7f335c1c8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:04.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:04.078+0000 7f336294c700 1 --2- 172.21.15.16:0/257076626 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f335c136710 0x7f335c1c8960 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3350000c00 tx=0x7f3350004750 comp rx=0 tx=0).stop 2023-09-30T22:30:04.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:04.078+0000 7f336294c700 1 -- 172.21.15.16:0/257076626 shutdown_connections 2023-09-30T22:30:04.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:04.078+0000 7f336294c700 1 --2- 172.21.15.16:0/257076626 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f335c139030 0x7f335c11acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:04.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:04.078+0000 7f336294c700 1 --2- 172.21.15.16:0/257076626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f335c1338a0 0x7f335c1c8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:04.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:04.078+0000 7f336294c700 1 --2- 172.21.15.16:0/257076626 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f335c136710 0x7f335c1c8960 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:04.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:04.078+0000 7f336294c700 1 -- 172.21.15.16:0/257076626 >> 172.21.15.16:0/257076626 conn(0x7f335c0a2aa0 msgr2=0x7f335c1344f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:04.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:04.078+0000 7f336294c700 1 -- 172.21.15.16:0/257076626 shutdown_connections 2023-09-30T22:30:04.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:04.078+0000 7f336294c700 1 -- 172.21.15.16:0/257076626 wait complete. 2023-09-30T22:30:04.092 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:30:04.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:04.152+0000 7f186577b700 1 -- 172.21.15.16:0/720291047 >> v1:172.21.15.16:6816/126811 conn(0x7f184402f510 legacy=0x7f18440319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:04.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:04.156+0000 7fd249b9b700 1 -- 172.21.15.16:0/1743906116 >> v1:172.21.15.16:6816/126811 conn(0x7fd22802f540 legacy=0x7fd2280319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:04.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:04.173+0000 7fd24b39e700 1 -- 172.21.15.16:0/1743906116 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd244129670 con 0x7fd2440a0800 2023-09-30T22:30:04.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:04.173+0000 7fd236ffd700 1 -- 172.21.15.16:0/1743906116 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd244129670 con 0x7fd2440a0800 2023-09-30T22:30:04.174 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:30:04.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:04.176+0000 7fd24b39e700 1 -- 172.21.15.16:0/1743906116 >> v1:172.21.15.16:6816/126811 conn(0x7fd22802f540 legacy=0x7fd2280319f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:30:04.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:04.176+0000 7fd24b39e700 1 -- 172.21.15.16:0/1743906116 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd2440a0800 msgr2=0x7fd2441260d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:04.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:04.176+0000 7fd24b39e700 1 --2- 172.21.15.16:0/1743906116 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd2440a0800 0x7fd2441260d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd24000c490 tx=0x7fd24000c7a0 comp rx=0 tx=0).stop 2023-09-30T22:30:04.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:04.176+0000 7fd24b39e700 1 -- 172.21.15.16:0/1743906116 shutdown_connections 2023-09-30T22:30:04.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:04.176+0000 7fd24b39e700 1 --2- 172.21.15.16:0/1743906116 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd2440a0800 0x7fd2441260d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:04.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:04.176+0000 7fd24b39e700 1 --2- 172.21.15.16:0/1743906116 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd244139110 0x7fd244126b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:04.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:04.176+0000 7fd24b39e700 1 --2- 172.21.15.16:0/1743906116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2440a12e0 0x7fd244126610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:04.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:04.176+0000 7fd24b39e700 1 -- 172.21.15.16:0/1743906116 >> 172.21.15.16:0/1743906116 conn(0x7fd2440a6140 msgr2=0x7fd24409be70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:04.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:04.176+0000 7fd24b39e700 1 -- 172.21.15.16:0/1743906116 shutdown_connections 2023-09-30T22:30:04.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:04.176+0000 7fd24b39e700 1 -- 172.21.15.16:0/1743906116 wait complete. 2023-09-30T22:30:04.190 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:30:04.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:04.192+0000 7f18679df700 1 -- 172.21.15.16:0/720291047 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1840005860 con 0x7f1860138d80 2023-09-30T22:30:04.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:04.192+0000 7f1855ffb700 1 -- 172.21.15.16:0/720291047 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1858019030 con 0x7f1860138d80 2023-09-30T22:30:04.193 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:30:04.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:04.194+0000 7f18679df700 1 -- 172.21.15.16:0/720291047 >> v1:172.21.15.16:6816/126811 conn(0x7f184402f510 legacy=0x7f18440319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:30:04.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:04.194+0000 7f18679df700 1 -- 172.21.15.16:0/720291047 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1860138d80 msgr2=0x7f186012ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:04.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:04.194+0000 7f18679df700 1 --2- 172.21.15.16:0/720291047 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1860138d80 0x7f186012ccb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f185800b9e0 tx=0x7f185800bcf0 comp rx=0 tx=0).stop 2023-09-30T22:30:04.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:04.194+0000 7f18679df700 1 -- 172.21.15.16:0/720291047 shutdown_connections 2023-09-30T22:30:04.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:04.194+0000 7f18679df700 1 --2- 172.21.15.16:0/720291047 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1860138d80 0x7f186012ccb0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:04.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:04.194+0000 7f18679df700 1 --2- 172.21.15.16:0/720291047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f186009d630 0x7f186012c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:04.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:04.194+0000 7f18679df700 1 --2- 172.21.15.16:0/720291047 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f18600a0a30 0x7f186012c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:04.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:04.195+0000 7f18679df700 1 -- 172.21.15.16:0/720291047 >> 172.21.15.16:0/720291047 conn(0x7f18600a3050 msgr2=0x7f186009ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:04.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:04.195+0000 7f18679df700 1 -- 172.21.15.16:0/720291047 shutdown_connections 2023-09-30T22:30:04.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:04.195+0000 7f18679df700 1 -- 172.21.15.16:0/720291047 wait complete. 2023-09-30T22:30:04.208 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:30:05.074 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:05.093 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:05.190 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:05.210 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:05.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.224+0000 7f402722c700 1 Processor -- start 2023-09-30T22:30:05.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.224+0000 7f402722c700 1 -- start start 2023-09-30T22:30:05.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.224+0000 7f402722c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f402009d6f0 0x7f402009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:05.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.224+0000 7f402722c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f402009e410 0x7f4020134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:05.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.224+0000 7f402722c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f402009b120 con 0x7f402009e000 2023-09-30T22:30:05.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.224+0000 7f402722c700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f402006b460 con 0x7f402009e410 2023-09-30T22:30:05.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.224+0000 7f402722c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f402009a7e0 con 0x7f402009d6f0 2023-09-30T22:30:05.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.224+0000 7f4024fc8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f402009d6f0 0x7f402009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:05.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.224+0000 7f40257c9700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f402009e410 0x7f4020134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:05.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.225+0000 7f4024fc8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f402009d6f0 0x7f402009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:55052/0 (socket says 172.21.15.16:55052) 2023-09-30T22:30:05.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.225+0000 7f4024fc8700 1 -- 172.21.15.16:0/2400277891 learned_addr learned my addr 172.21.15.16:0/2400277891 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:05.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.225+0000 7f40257c9700 1 -- 172.21.15.16:0/2400277891 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f402009d6f0 msgr2=0x7f402009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:05.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.225+0000 7f40257c9700 1 --2- 172.21.15.16:0/2400277891 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f402009d6f0 0x7f402009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:05.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.225+0000 7f40257c9700 1 -- 172.21.15.16:0/2400277891 >> v1:172.21.15.16:6789/0 conn(0x7f402009e000 legacy=0x7f40201339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:05.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.225+0000 7f40257c9700 1 -- 172.21.15.16:0/2400277891 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f402009a500 con 0x7f402009e410 2023-09-30T22:30:05.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.225+0000 7f401f7fe700 1 -- 172.21.15.16:0/2400277891 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1696585986 0 0) 0x7f402009b120 con 0x7f402009e000 2023-09-30T22:30:05.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.225+0000 7f40257c9700 1 --2- 172.21.15.16:0/2400277891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f402009e410 0x7f4020134100 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4018009d90 tx=0x7f401800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f6b4e3bbe0f1c4a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:05.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.225+0000 7f401f7fe700 1 -- 172.21.15.16:0/2400277891 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401800e070 con 0x7f402009e410 2023-09-30T22:30:05.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.225+0000 7f401f7fe700 1 -- 172.21.15.16:0/2400277891 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f401800bb20 con 0x7f402009e410 2023-09-30T22:30:05.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.225+0000 7f401f7fe700 1 -- 172.21.15.16:0/2400277891 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4018012430 con 0x7f402009e410 2023-09-30T22:30:05.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.226+0000 7f402722c700 1 -- 172.21.15.16:0/2400277891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f402009e410 msgr2=0x7f4020134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:05.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.226+0000 7f402722c700 1 --2- 172.21.15.16:0/2400277891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f402009e410 0x7f4020134100 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4018009d90 tx=0x7f401800b330 comp rx=0 tx=0).stop 2023-09-30T22:30:05.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.227+0000 7f402722c700 1 -- 172.21.15.16:0/2400277891 shutdown_connections 2023-09-30T22:30:05.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.227+0000 7f402722c700 1 --2- 172.21.15.16:0/2400277891 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f402009e410 0x7f4020134100 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:05.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.227+0000 7f402722c700 1 --2- 172.21.15.16:0/2400277891 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f402009d6f0 0x7f402009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:05.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.227+0000 7f402722c700 1 -- 172.21.15.16:0/2400277891 >> 172.21.15.16:0/2400277891 conn(0x7f4020066f50 msgr2=0x7f402009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:05.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.227+0000 7f402722c700 1 -- 172.21.15.16:0/2400277891 shutdown_connections 2023-09-30T22:30:05.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.227+0000 7f402722c700 1 -- 172.21.15.16:0/2400277891 wait complete. 2023-09-30T22:30:05.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.228+0000 7f402722c700 1 Processor -- start 2023-09-30T22:30:05.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.228+0000 7f402722c700 1 -- start start 2023-09-30T22:30:05.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.229+0000 7f402722c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f402009d6f0 0x7f40201a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:05.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.229+0000 7f402722c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f402009e000 0x7f40201a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:05.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.229+0000 7f402722c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f402009e410 0x7f40201c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:05.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.229+0000 7f402722c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40201a8120 con 0x7f402009d6f0 2023-09-30T22:30:05.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.229+0000 7f402722c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40201a8300 con 0x7f402009e000 2023-09-30T22:30:05.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.229+0000 7f402722c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f40201b9170 con 0x7f402009e410 2023-09-30T22:30:05.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.229+0000 7f4024fc8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f402009d6f0 0x7f40201a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:05.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.230+0000 7f4024fc8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f402009d6f0 0x7f40201a5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53082/0 (socket says 172.21.15.16:53082) 2023-09-30T22:30:05.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.230+0000 7f4024fc8700 1 -- 172.21.15.16:0/3501722541 learned_addr learned my addr 172.21.15.16:0/3501722541 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:05.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.230+0000 7f40257c9700 1 --2- 172.21.15.16:0/3501722541 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f402009e410 0x7f40201c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:05.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.230+0000 7f401ffff700 1 --2- 172.21.15.16:0/3501722541 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f402009e000 0x7f40201a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:05.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.230+0000 7f4024fc8700 1 -- 172.21.15.16:0/3501722541 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f402009e410 msgr2=0x7f40201c8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:05.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.230+0000 7f4024fc8700 1 --2- 172.21.15.16:0/3501722541 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f402009e410 0x7f40201c8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:05.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.230+0000 7f4024fc8700 1 -- 172.21.15.16:0/3501722541 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f402009e000 msgr2=0x7f40201a5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:05.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.230+0000 7f4024fc8700 1 --2- 172.21.15.16:0/3501722541 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f402009e000 0x7f40201a5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:05.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.230+0000 7f4024fc8700 1 -- 172.21.15.16:0/3501722541 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4018009a70 con 0x7f402009d6f0 2023-09-30T22:30:05.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.230+0000 7f40257c9700 1 --2- 172.21.15.16:0/3501722541 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f402009e410 0x7f40201c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:05.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.230+0000 7f401ffff700 1 --2- 172.21.15.16:0/3501722541 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f402009e000 0x7f40201a5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:05.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.230+0000 7f4024fc8700 1 --2- 172.21.15.16:0/3501722541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f402009d6f0 0x7f40201a5240 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f401400e940 tx=0x7f401400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:05.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.230+0000 7f401d7fa700 1 -- 172.21.15.16:0/3501722541 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40140096a0 con 0x7f402009d6f0 2023-09-30T22:30:05.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.231+0000 7f401d7fa700 1 -- 172.21.15.16:0/3501722541 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4014013680 con 0x7f402009d6f0 2023-09-30T22:30:05.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.231+0000 7f402722c700 1 -- 172.21.15.16:0/3501722541 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40201a4ca0 con 0x7f402009d6f0 2023-09-30T22:30:05.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.231+0000 7f401d7fa700 1 -- 172.21.15.16:0/3501722541 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4014013a30 con 0x7f402009d6f0 2023-09-30T22:30:05.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.231+0000 7f402722c700 1 -- 172.21.15.16:0/3501722541 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40201a7b00 con 0x7f402009d6f0 2023-09-30T22:30:05.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.232+0000 7f401d7fa700 1 -- 172.21.15.16:0/3501722541 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f4014013b90 con 0x7f402009d6f0 2023-09-30T22:30:05.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.232+0000 7f401ffff700 1 -- 172.21.15.16:0/3501722541 >> v1:172.21.15.16:6816/126811 conn(0x7f400402f510 legacy=0x7f40040319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:05.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.232+0000 7f401d7fa700 1 -- 172.21.15.16:0/3501722541 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7f401403fb40 con 0x7f402009d6f0 2023-09-30T22:30:05.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.233+0000 7f402722c700 1 -- 172.21.15.16:0/3501722541 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40201a5cc0 con 0x7f402009d6f0 2023-09-30T22:30:05.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.239+0000 7f401d7fa700 1 -- 172.21.15.16:0/3501722541 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4014017080 con 0x7f402009d6f0 2023-09-30T22:30:05.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.274+0000 7fbf70f6b700 1 Processor -- start 2023-09-30T22:30:05.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.274+0000 7fbf70f6b700 1 -- start start 2023-09-30T22:30:05.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.275+0000 7fbf70f6b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbf6c0a0a30 0x7fbf6c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:05.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.275+0000 7fbf70f6b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbf6c138d80 0x7fbf6c13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:05.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.275+0000 7fbf70f6b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf6c0a2a70 con 0x7fbf6c09d630 2023-09-30T22:30:05.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.275+0000 7fbf70f6b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6c0a01d0 con 0x7fbf6c138d80 2023-09-30T22:30:05.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.275+0000 7fbf70f6b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6c09e2b0 con 0x7fbf6c0a0a30 2023-09-30T22:30:05.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.276+0000 7fbf69d9b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fbf6c09d630 0x7fbf6c0a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57624/0 (socket says 172.21.15.16:57624) 2023-09-30T22:30:05.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.276+0000 7fbf69d9b700 1 -- 172.21.15.16:0/1063338316 learned_addr learned my addr 172.21.15.16:0/1063338316 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:30:05.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.276+0000 7fbf6a59c700 1 --2- 172.21.15.16:0/1063338316 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbf6c0a0a30 0x7fbf6c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:05.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.276+0000 7fbf6ad9d700 1 --2- 172.21.15.16:0/1063338316 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbf6c138d80 0x7fbf6c13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:05.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.276+0000 7fbf6ad9d700 1 -- 172.21.15.16:0/1063338316 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbf6c0a0a30 msgr2=0x7fbf6c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:05.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.276+0000 7fbf6ad9d700 1 --2- 172.21.15.16:0/1063338316 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbf6c0a0a30 0x7fbf6c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:05.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.276+0000 7fbf6959a700 1 -- 172.21.15.16:0/1063338316 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3082062959 0 0) 0x7fbf6c0a2a70 con 0x7fbf6c09d630 2023-09-30T22:30:05.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.276+0000 7fbf6ad9d700 1 -- 172.21.15.16:0/1063338316 >> v1:172.21.15.16:6789/0 conn(0x7fbf6c09d630 legacy=0x7fbf6c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:05.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.276+0000 7fbf6ad9d700 1 -- 172.21.15.16:0/1063338316 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf6c09dc50 con 0x7fbf6c138d80 2023-09-30T22:30:05.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.276+0000 7fbf6a59c700 1 --2- 172.21.15.16:0/1063338316 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbf6c0a0a30 0x7fbf6c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:05.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.276+0000 7fbf6ad9d700 1 --2- 172.21.15.16:0/1063338316 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbf6c138d80 0x7fbf6c13b160 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbf64009dc0 tx=0x7fbf6400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a6a9e5a91dc6f2f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:05.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.276+0000 7fbf6959a700 1 -- 172.21.15.16:0/1063338316 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf6400e070 con 0x7fbf6c138d80 2023-09-30T22:30:05.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.277+0000 7fbf6959a700 1 -- 172.21.15.16:0/1063338316 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf6400bb20 con 0x7fbf6c138d80 2023-09-30T22:30:05.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.277+0000 7fbf6959a700 1 -- 172.21.15.16:0/1063338316 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf64012430 con 0x7fbf6c138d80 2023-09-30T22:30:05.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.277+0000 7fbf70f6b700 1 -- 172.21.15.16:0/1063338316 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbf6c138d80 msgr2=0x7fbf6c13b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:05.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.277+0000 7fbf70f6b700 1 --2- 172.21.15.16:0/1063338316 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbf6c138d80 0x7fbf6c13b160 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fbf64009dc0 tx=0x7fbf6400b330 comp rx=0 tx=0).stop 2023-09-30T22:30:05.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.277+0000 7fbf70f6b700 1 -- 172.21.15.16:0/1063338316 shutdown_connections 2023-09-30T22:30:05.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.277+0000 7fbf70f6b700 1 --2- 172.21.15.16:0/1063338316 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbf6c138d80 0x7fbf6c13b160 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:05.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.277+0000 7fbf70f6b700 1 --2- 172.21.15.16:0/1063338316 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbf6c0a0a30 0x7fbf6c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:05.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.277+0000 7fbf70f6b700 1 -- 172.21.15.16:0/1063338316 >> 172.21.15.16:0/1063338316 conn(0x7fbf6c0a3050 msgr2=0x7fbf6c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:05.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.277+0000 7fbf70f6b700 1 -- 172.21.15.16:0/1063338316 shutdown_connections 2023-09-30T22:30:05.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.278+0000 7fbf70f6b700 1 -- 172.21.15.16:0/1063338316 wait complete. 2023-09-30T22:30:05.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.279+0000 7fbf70f6b700 1 Processor -- start 2023-09-30T22:30:05.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.279+0000 7fbf70f6b700 1 -- start start 2023-09-30T22:30:05.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.280+0000 7fbf70f6b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbf6c09d630 0x7fbf6c1204f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:05.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.280+0000 7fbf70f6b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbf6c0a0a30 0x7fbf6c120a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:05.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.280+0000 7fbf70f6b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf6c138d80 0x7fbf6c120f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:05.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.280+0000 7fbf70f6b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6c123e10 con 0x7fbf6c138d80 2023-09-30T22:30:05.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.280+0000 7fbf70f6b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6c1233c0 con 0x7fbf6c09d630 2023-09-30T22:30:05.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.280+0000 7fbf70f6b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6c122f00 con 0x7fbf6c0a0a30 2023-09-30T22:30:05.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.280+0000 7fbf6a59c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbf6c09d630 0x7fbf6c1204f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:05.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.280+0000 7fbf6a59c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbf6c09d630 0x7fbf6c1204f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:55084/0 (socket says 172.21.15.16:55084) 2023-09-30T22:30:05.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.280+0000 7fbf6a59c700 1 -- 172.21.15.16:0/183561416 learned_addr learned my addr 172.21.15.16:0/183561416 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:05.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.280+0000 7fbf69d9b700 1 --2- 172.21.15.16:0/183561416 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbf6c0a0a30 0x7fbf6c120a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:05.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.280+0000 7fbf6ad9d700 1 --2- 172.21.15.16:0/183561416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf6c138d80 0x7fbf6c120f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:05.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.281+0000 7fbf6a59c700 1 -- 172.21.15.16:0/183561416 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbf6c0a0a30 msgr2=0x7fbf6c120a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:05.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.281+0000 7fbf6a59c700 1 --2- 172.21.15.16:0/183561416 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbf6c0a0a30 0x7fbf6c120a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:05.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.281+0000 7fbf6a59c700 1 -- 172.21.15.16:0/183561416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf6c138d80 msgr2=0x7fbf6c120f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:05.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.281+0000 7fbf6a59c700 1 --2- 172.21.15.16:0/183561416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf6c138d80 0x7fbf6c120f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:05.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.281+0000 7fbf6a59c700 1 -- 172.21.15.16:0/183561416 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf5c0094a0 con 0x7fbf6c09d630 2023-09-30T22:30:05.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.281+0000 7fbf69d9b700 1 --2- 172.21.15.16:0/183561416 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbf6c0a0a30 0x7fbf6c120a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:05.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.281+0000 7fbf6a59c700 1 --2- 172.21.15.16:0/183561416 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbf6c09d630 0x7fbf6c1204f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c00ebe0 tx=0x7fbf5c00eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:05.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.281+0000 7fbf5affd700 1 -- 172.21.15.16:0/183561416 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c00c850 con 0x7fbf6c09d630 2023-09-30T22:30:05.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.281+0000 7fbf5affd700 1 -- 172.21.15.16:0/183561416 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf5c00c9b0 con 0x7fbf6c09d630 2023-09-30T22:30:05.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.281+0000 7fbf5affd700 1 -- 172.21.15.16:0/183561416 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c00cd20 con 0x7fbf6c09d630 2023-09-30T22:30:05.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.281+0000 7fbf70f6b700 1 -- 172.21.15.16:0/183561416 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf64009a70 con 0x7fbf6c09d630 2023-09-30T22:30:05.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.281+0000 7fbf70f6b700 1 -- 172.21.15.16:0/183561416 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf6c13aa00 con 0x7fbf6c09d630 2023-09-30T22:30:05.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.282+0000 7fbf70f6b700 1 -- 172.21.15.16:0/183561416 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf6c13b620 con 0x7fbf6c09d630 2023-09-30T22:30:05.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.282+0000 7fbf5affd700 1 -- 172.21.15.16:0/183561416 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fbf5c013470 con 0x7fbf6c09d630 2023-09-30T22:30:05.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.283+0000 7fbf5affd700 1 -- 172.21.15.16:0/183561416 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7fbf5c040070 con 0x7fbf6c09d630 2023-09-30T22:30:05.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.283+0000 7fbf69d9b700 1 -- 172.21.15.16:0/183561416 >> v1:172.21.15.16:6816/126811 conn(0x7fbf4c02f510 legacy=0x7fbf4c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:05.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.288+0000 7fbf5affd700 1 -- 172.21.15.16:0/183561416 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbf5c013720 con 0x7fbf6c09d630 2023-09-30T22:30:05.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.350+0000 7fa9a41ab700 1 Processor -- start 2023-09-30T22:30:05.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.351+0000 7fa9a41ab700 1 -- start start 2023-09-30T22:30:05.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.352+0000 7fa9a41ab700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa99c09f410 0x7fa99c136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:05.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.352+0000 7fa9a41ab700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa99c136fe0 0x7fa99c1393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:05.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.352+0000 7fa9a41ab700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa99c070940 con 0x7fa99c09e930 2023-09-30T22:30:05.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.352+0000 7fa9a41ab700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa99c06fd60 con 0x7fa99c09f410 2023-09-30T22:30:05.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.352+0000 7fa9a41ab700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa99c06fa80 con 0x7fa99c136fe0 2023-09-30T22:30:05.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.352+0000 7fa9a2748700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa99c136fe0 0x7fa99c1393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:05.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.352+0000 7fa9a2748700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa99c136fe0 0x7fa99c1393c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:55092/0 (socket says 172.21.15.16:55092) 2023-09-30T22:30:05.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.352+0000 7fa9a2748700 1 -- 172.21.15.16:0/4241357378 learned_addr learned my addr 172.21.15.16:0/4241357378 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:05.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.352+0000 7fa9a1f47700 1 --1- 172.21.15.16:0/4241357378 >> v1:172.21.15.16:6789/0 conn(0x7fa99c09e930 0x7fa99c09ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57630/0 (socket says 172.21.15.16:57630) 2023-09-30T22:30:05.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.352+0000 7fa9a1746700 1 --2- 172.21.15.16:0/4241357378 >> v2:172.21.15.16:3300/0 conn(0x7fa99c09f410 0x7fa99c136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:05.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.352+0000 7fa9a1746700 1 -- 172.21.15.16:0/4241357378 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa99c136fe0 msgr2=0x7fa99c1393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:05.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.352+0000 7fa9a1746700 1 --2- 172.21.15.16:0/4241357378 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa99c136fe0 0x7fa99c1393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:05.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.352+0000 7fa9a1746700 1 -- 172.21.15.16:0/4241357378 >> v1:172.21.15.16:6789/0 conn(0x7fa99c09e930 legacy=0x7fa99c09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:05.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.352+0000 7fa9a1746700 1 -- 172.21.15.16:0/4241357378 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa99c09fb60 con 0x7fa99c09f410 2023-09-30T22:30:05.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.353+0000 7fa9a1746700 1 --2- 172.21.15.16:0/4241357378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa99c09f410 0x7fa99c136aa0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa98c009840 tx=0x7fa98c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f621ab80609edbe server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:05.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.353+0000 7fa9a0f45700 1 -- 172.21.15.16:0/4241357378 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98c011830 con 0x7fa99c09f410 2023-09-30T22:30:05.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.353+0000 7fa9a0f45700 1 -- 172.21.15.16:0/4241357378 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa98c011990 con 0x7fa99c09f410 2023-09-30T22:30:05.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.353+0000 7fa9a0f45700 1 -- 172.21.15.16:0/4241357378 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98c011d20 con 0x7fa99c09f410 2023-09-30T22:30:05.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.353+0000 7fa9a41ab700 1 -- 172.21.15.16:0/4241357378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa99c09f410 msgr2=0x7fa99c136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:05.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.353+0000 7fa9a41ab700 1 --2- 172.21.15.16:0/4241357378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa99c09f410 0x7fa99c136aa0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa98c009840 tx=0x7fa98c009b50 comp rx=0 tx=0).stop 2023-09-30T22:30:05.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.354+0000 7fa9a41ab700 1 -- 172.21.15.16:0/4241357378 shutdown_connections 2023-09-30T22:30:05.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.354+0000 7fa9a41ab700 1 --2- 172.21.15.16:0/4241357378 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa99c136fe0 0x7fa99c1393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:05.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.354+0000 7fa9a41ab700 1 --2- 172.21.15.16:0/4241357378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa99c09f410 0x7fa99c136aa0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:05.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.354+0000 7fa9a41ab700 1 -- 172.21.15.16:0/4241357378 >> 172.21.15.16:0/4241357378 conn(0x7fa99c0a4210 msgr2=0x7fa99c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:05.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.354+0000 7fa9a41ab700 1 -- 172.21.15.16:0/4241357378 shutdown_connections 2023-09-30T22:30:05.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.354+0000 7fa9a41ab700 1 -- 172.21.15.16:0/4241357378 wait complete. 2023-09-30T22:30:05.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.355+0000 7fa9a41ab700 1 Processor -- start 2023-09-30T22:30:05.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.355+0000 7fa9a41ab700 1 -- start start 2023-09-30T22:30:05.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.355+0000 7fa9a41ab700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa99c09e930 0x7fa99c06c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:05.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.355+0000 7fa9a41ab700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa99c09f410 0x7fa99c06c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:05.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.355+0000 7fa9a41ab700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa99c136fe0 0x7fa99c1c8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:05.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.355+0000 7fa9a41ab700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa99c1222e0 con 0x7fa99c09f410 2023-09-30T22:30:05.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.355+0000 7fa9a41ab700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa99c132340 con 0x7fa99c136fe0 2023-09-30T22:30:05.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.355+0000 7fa9a41ab700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa99c06bd70 con 0x7fa99c09e930 2023-09-30T22:30:05.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.355+0000 7fa9a1746700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa99c09f410 0x7fa99c06c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:05.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.356+0000 7fa9a1746700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa99c09f410 0x7fa99c06c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53110/0 (socket says 172.21.15.16:53110) 2023-09-30T22:30:05.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.356+0000 7fa9a1746700 1 -- 172.21.15.16:0/2198406080 learned_addr learned my addr 172.21.15.16:0/2198406080 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:05.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.356+0000 7fa9a2748700 1 --2- 172.21.15.16:0/2198406080 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa99c136fe0 0x7fa99c1c8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:05.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.356+0000 7fa9a1746700 1 -- 172.21.15.16:0/2198406080 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa99c09e930 msgr2=0x7fa99c06c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:05.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.356+0000 7fa9a1746700 1 --2- 172.21.15.16:0/2198406080 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa99c09e930 0x7fa99c06c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:05.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.356+0000 7fa9a1746700 1 -- 172.21.15.16:0/2198406080 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa99c136fe0 msgr2=0x7fa99c1c8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:05.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.356+0000 7fa9a1746700 1 --2- 172.21.15.16:0/2198406080 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa99c136fe0 0x7fa99c1c8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:05.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.356+0000 7fa9a1746700 1 -- 172.21.15.16:0/2198406080 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa98c0094a0 con 0x7fa99c09f410 2023-09-30T22:30:05.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.356+0000 7fa9a1746700 1 --2- 172.21.15.16:0/2198406080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa99c09f410 0x7fa99c06c850 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa98c0090b0 tx=0x7fa98c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:05.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.356+0000 7fa9927fc700 1 -- 172.21.15.16:0/2198406080 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98c0036a0 con 0x7fa99c09f410 2023-09-30T22:30:05.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.356+0000 7fa9927fc700 1 -- 172.21.15.16:0/2198406080 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa98c01f070 con 0x7fa99c09f410 2023-09-30T22:30:05.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.356+0000 7fa9a41ab700 1 -- 172.21.15.16:0/2198406080 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa99c1224e0 con 0x7fa99c09f410 2023-09-30T22:30:05.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.356+0000 7fa9a41ab700 1 -- 172.21.15.16:0/2198406080 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa99c0a2f40 con 0x7fa99c09f410 2023-09-30T22:30:05.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.357+0000 7fa9927fc700 1 -- 172.21.15.16:0/2198406080 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98c003a20 con 0x7fa99c09f410 2023-09-30T22:30:05.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.358+0000 7fa9a41ab700 1 -- 172.21.15.16:0/2198406080 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9840052e0 con 0x7fa99c09f410 2023-09-30T22:30:05.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.358+0000 7fa9927fc700 1 -- 172.21.15.16:0/2198406080 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fa98c003b80 con 0x7fa99c09f410 2023-09-30T22:30:05.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.358+0000 7fa9927fc700 1 -- 172.21.15.16:0/2198406080 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7fa98c021070 con 0x7fa99c09f410 2023-09-30T22:30:05.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.358+0000 7fa9a1f47700 1 -- 172.21.15.16:0/2198406080 >> v1:172.21.15.16:6816/126811 conn(0x7fa98002f540 legacy=0x7fa9800319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:05.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.367+0000 7fa9927fc700 1 -- 172.21.15.16:0/2198406080 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa98c010380 con 0x7fa99c09f410 2023-09-30T22:30:05.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.387+0000 7fb6c910f700 1 Processor -- start 2023-09-30T22:30:05.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.387+0000 7fb6c910f700 1 -- start start 2023-09-30T22:30:05.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.388+0000 7fb6c910f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb6c409e930 0x7fb6c409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:05.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.388+0000 7fb6c910f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb6c409f240 0x7fb6c41317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:05.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.388+0000 7fb6c910f700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6c4070940 con 0x7fb6c409f650 2023-09-30T22:30:05.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.388+0000 7fb6c910f700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6c406fd60 con 0x7fb6c409f240 2023-09-30T22:30:05.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.388+0000 7fb6c910f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c406fa80 con 0x7fb6c409e930 2023-09-30T22:30:05.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.388+0000 7fb6c2d9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb6c409e930 0x7fb6c409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:05.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.388+0000 7fb6c359e700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb6c409f650 0x7fb6c4131d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57644/0 (socket says 172.21.15.16:57644) 2023-09-30T22:30:05.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.388+0000 7fb6c2d9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb6c409e930 0x7fb6c409ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:45352/0 (socket says 172.21.15.16:45352) 2023-09-30T22:30:05.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.388+0000 7fb6c359e700 1 -- 172.21.15.16:0/2199664764 learned_addr learned my addr 172.21.15.16:0/2199664764 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:30:05.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.388+0000 7fb6bbfff700 1 --2- 172.21.15.16:0/2199664764 >> v2:172.21.15.16:3300/0 conn(0x7fb6c409f240 0x7fb6c41317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:05.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.388+0000 7fb6c2d9d700 1 -- 172.21.15.16:0/2199664764 >> v2:172.21.15.16:3300/0 conn(0x7fb6c409f240 msgr2=0x7fb6c41317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:05.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.388+0000 7fb6c2d9d700 1 --2- 172.21.15.16:0/2199664764 >> v2:172.21.15.16:3300/0 conn(0x7fb6c409f240 0x7fb6c41317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:05.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.388+0000 7fb6c259c700 1 -- 172.21.15.16:0/2199664764 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2765475684 0 0) 0x7fb6c4070940 con 0x7fb6c409f650 2023-09-30T22:30:05.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.388+0000 7fb6c2d9d700 1 -- 172.21.15.16:0/2199664764 >> v1:172.21.15.16:6789/0 conn(0x7fb6c409f650 legacy=0x7fb6c4131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:05.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.388+0000 7fb6c2d9d700 1 -- 172.21.15.16:0/2199664764 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c409fb60 con 0x7fb6c409e930 2023-09-30T22:30:05.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.389+0000 7fb6c2d9d700 1 --2- 172.21.15.16:0/2199664764 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb6c409e930 0x7fb6c409ed00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb6b4009840 tx=0x7fb6b4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5ac8800cd7ba16d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:05.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.389+0000 7fb6c259c700 1 -- 172.21.15.16:0/2199664764 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b4010a90 con 0x7fb6c409e930 2023-09-30T22:30:05.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.389+0000 7fb6c259c700 1 -- 172.21.15.16:0/2199664764 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6b4010bf0 con 0x7fb6c409e930 2023-09-30T22:30:05.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.389+0000 7fb6c259c700 1 -- 172.21.15.16:0/2199664764 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b4018440 con 0x7fb6c409e930 2023-09-30T22:30:05.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.390+0000 7fb6c910f700 1 -- 172.21.15.16:0/2199664764 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb6c409e930 msgr2=0x7fb6c409ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:05.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.390+0000 7fb6c910f700 1 --2- 172.21.15.16:0/2199664764 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb6c409e930 0x7fb6c409ed00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb6b4009840 tx=0x7fb6b4009b50 comp rx=0 tx=0).stop 2023-09-30T22:30:05.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.390+0000 7fb6c910f700 1 -- 172.21.15.16:0/2199664764 shutdown_connections 2023-09-30T22:30:05.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.390+0000 7fb6c910f700 1 --2- 172.21.15.16:0/2199664764 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb6c409e930 0x7fb6c409ed00 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:05.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.390+0000 7fb6c910f700 1 --2- 172.21.15.16:0/2199664764 >> v2:172.21.15.16:3300/0 conn(0x7fb6c409f240 0x7fb6c41317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:05.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.390+0000 7fb6c910f700 1 -- 172.21.15.16:0/2199664764 >> 172.21.15.16:0/2199664764 conn(0x7fb6c40a4210 msgr2=0x7fb6c40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:05.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.390+0000 7fb6c910f700 1 -- 172.21.15.16:0/2199664764 shutdown_connections 2023-09-30T22:30:05.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.390+0000 7fb6c910f700 1 -- 172.21.15.16:0/2199664764 wait complete. 2023-09-30T22:30:05.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.392+0000 7fb6c910f700 1 Processor -- start 2023-09-30T22:30:05.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.392+0000 7fb6c910f700 1 -- start start 2023-09-30T22:30:05.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.392+0000 7fb6c910f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6c409e930 0x7fb6c41c3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:05.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.392+0000 7fb6c910f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb6c409f240 0x7fb6c41c42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:05.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.392+0000 7fb6c910f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb6c409f650 0x7fb6c41c4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:05.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.392+0000 7fb6c910f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c41a2640 con 0x7fb6c409e930 2023-09-30T22:30:05.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.393+0000 7fb6c910f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c41a2450 con 0x7fb6c409f650 2023-09-30T22:30:05.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.393+0000 7fb6c910f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb6c41a34f0 con 0x7fb6c409f240 2023-09-30T22:30:05.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.393+0000 7fb6c359e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb6c409f650 0x7fb6c41c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:05.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.393+0000 7fb6c359e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb6c409f650 0x7fb6c41c4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:55110/0 (socket says 172.21.15.16:55110) 2023-09-30T22:30:05.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.393+0000 7fb6c359e700 1 -- 172.21.15.16:0/2458454779 learned_addr learned my addr 172.21.15.16:0/2458454779 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:05.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.393+0000 7fb6c2d9d700 1 --2- 172.21.15.16:0/2458454779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6c409e930 0x7fb6c41c3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:05.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.393+0000 7fb6bbfff700 1 --2- 172.21.15.16:0/2458454779 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb6c409f240 0x7fb6c41c42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:05.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.393+0000 7fb6c2d9d700 1 -- 172.21.15.16:0/2458454779 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb6c409f240 msgr2=0x7fb6c41c42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:05.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.393+0000 7fb6c2d9d700 1 --2- 172.21.15.16:0/2458454779 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb6c409f240 0x7fb6c41c42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:05.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.393+0000 7fb6c2d9d700 1 -- 172.21.15.16:0/2458454779 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb6c409f650 msgr2=0x7fb6c41c4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:05.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.393+0000 7fb6c2d9d700 1 --2- 172.21.15.16:0/2458454779 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb6c409f650 0x7fb6c41c4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:05.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.393+0000 7fb6c2d9d700 1 -- 172.21.15.16:0/2458454779 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6b40094a0 con 0x7fb6c409e930 2023-09-30T22:30:05.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.394+0000 7fb6c2d9d700 1 --2- 172.21.15.16:0/2458454779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6c409e930 0x7fb6c41c3db0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb6c406ef10 tx=0x7fb6b4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:05.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.394+0000 7fb6bb7fe700 1 -- 172.21.15.16:0/2458454779 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b4003ab0 con 0x7fb6c409e930 2023-09-30T22:30:05.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.394+0000 7fb6bb7fe700 1 -- 172.21.15.16:0/2458454779 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6b4003c10 con 0x7fb6c409e930 2023-09-30T22:30:05.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.394+0000 7fb6c910f700 1 -- 172.21.15.16:0/2458454779 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6c40a0190 con 0x7fb6c409e930 2023-09-30T22:30:05.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.394+0000 7fb6bb7fe700 1 -- 172.21.15.16:0/2458454779 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b40185f0 con 0x7fb6c409e930 2023-09-30T22:30:05.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.394+0000 7fb6c910f700 1 -- 172.21.15.16:0/2458454779 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6c41a4080 con 0x7fb6c409e930 2023-09-30T22:30:05.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.395+0000 7fb6bb7fe700 1 -- 172.21.15.16:0/2458454779 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fb6b401f040 con 0x7fb6c409e930 2023-09-30T22:30:05.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.396+0000 7fb6c910f700 1 -- 172.21.15.16:0/2458454779 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6c4074ea0 con 0x7fb6c409e930 2023-09-30T22:30:05.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.396+0000 7fb6bb7fe700 1 -- 172.21.15.16:0/2458454779 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7fb6b4044fc0 con 0x7fb6c409e930 2023-09-30T22:30:05.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.396+0000 7fb6bbfff700 1 -- 172.21.15.16:0/2458454779 >> v1:172.21.15.16:6816/126811 conn(0x7fb6a402f560 legacy=0x7fb6a4031a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:05.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.405+0000 7fb6bb7fe700 1 -- 172.21.15.16:0/2458454779 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb6b4076750 con 0x7fb6c409e930 2023-09-30T22:30:05.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.433+0000 7f401ffff700 1 -- 172.21.15.16:0/3501722541 >> v1:172.21.15.16:6816/126811 conn(0x7f400402f510 legacy=0x7f40040319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:05.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.467+0000 7f402722c700 1 -- 172.21.15.16:0/3501722541 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f4020074ea0 con 0x7f402009d6f0 2023-09-30T22:30:05.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.468+0000 7f401d7fa700 1 -- 172.21.15.16:0/3501722541 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f401401bc00 con 0x7f402009d6f0 2023-09-30T22:30:05.468 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:30:05.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.470+0000 7f402722c700 1 -- 172.21.15.16:0/3501722541 >> v1:172.21.15.16:6816/126811 conn(0x7f400402f510 legacy=0x7f40040319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:30:05.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.470+0000 7f402722c700 1 -- 172.21.15.16:0/3501722541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f402009d6f0 msgr2=0x7f40201a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:05.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.470+0000 7f402722c700 1 --2- 172.21.15.16:0/3501722541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f402009d6f0 0x7f40201a5240 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f401400e940 tx=0x7f401400ec50 comp rx=0 tx=0).stop 2023-09-30T22:30:05.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.470+0000 7f402722c700 1 -- 172.21.15.16:0/3501722541 shutdown_connections 2023-09-30T22:30:05.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.470+0000 7f402722c700 1 --2- 172.21.15.16:0/3501722541 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f402009e410 0x7f40201c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:05.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.471+0000 7f402722c700 1 --2- 172.21.15.16:0/3501722541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f402009d6f0 0x7f40201a5240 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:05.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.471+0000 7f402722c700 1 --2- 172.21.15.16:0/3501722541 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f402009e000 0x7f40201a5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:05.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.471+0000 7f402722c700 1 -- 172.21.15.16:0/3501722541 >> 172.21.15.16:0/3501722541 conn(0x7f4020066f50 msgr2=0x7f402006f070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:05.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.471+0000 7f402722c700 1 -- 172.21.15.16:0/3501722541 shutdown_connections 2023-09-30T22:30:05.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.471+0000 7f402722c700 1 -- 172.21.15.16:0/3501722541 wait complete. 2023-09-30T22:30:05.485 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:30:05.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.509+0000 7fbf70f6b700 1 -- 172.21.15.16:0/183561416 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fbf6c0747f0 con 0x7fbf6c09d630 2023-09-30T22:30:05.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.509+0000 7fbf5affd700 1 -- 172.21.15.16:0/183561416 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbf5c021070 con 0x7fbf6c09d630 2023-09-30T22:30:05.510 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:30:05.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.512+0000 7fbf70f6b700 1 -- 172.21.15.16:0/183561416 >> v1:172.21.15.16:6816/126811 conn(0x7fbf4c02f510 legacy=0x7fbf4c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-09-30T22:30:05.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.512+0000 7fbf70f6b700 1 -- 172.21.15.16:0/183561416 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbf6c09d630 msgr2=0x7fbf6c1204f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:05.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.512+0000 7fbf70f6b700 1 --2- 172.21.15.16:0/183561416 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbf6c09d630 0x7fbf6c1204f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c00ebe0 tx=0x7fbf5c00eef0 comp rx=0 tx=0).stop 2023-09-30T22:30:05.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.512+0000 7fbf70f6b700 1 -- 172.21.15.16:0/183561416 shutdown_connections 2023-09-30T22:30:05.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.512+0000 7fbf70f6b700 1 --2- 172.21.15.16:0/183561416 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbf6c09d630 0x7fbf6c1204f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:05.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.512+0000 7fbf70f6b700 1 --2- 172.21.15.16:0/183561416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf6c138d80 0x7fbf6c120f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:05.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.512+0000 7fbf70f6b700 1 --2- 172.21.15.16:0/183561416 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbf6c0a0a30 0x7fbf6c120a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:05.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.512+0000 7fbf70f6b700 1 -- 172.21.15.16:0/183561416 >> 172.21.15.16:0/183561416 conn(0x7fbf6c0a3050 msgr2=0x7fbf6c09ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:05.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.512+0000 7fbf70f6b700 1 -- 172.21.15.16:0/183561416 shutdown_connections 2023-09-30T22:30:05.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.512+0000 7fbf70f6b700 1 -- 172.21.15.16:0/183561416 wait complete. 2023-09-30T22:30:05.526 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:30:05.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.559+0000 7fa9a1f47700 1 -- 172.21.15.16:0/2198406080 >> v1:172.21.15.16:6816/126811 conn(0x7fa98002f540 legacy=0x7fa9800319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:05.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.585+0000 7fa9a41ab700 1 -- 172.21.15.16:0/2198406080 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa984005860 con 0x7fa99c09f410 2023-09-30T22:30:05.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.585+0000 7fa9927fc700 1 -- 172.21.15.16:0/2198406080 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa98c015070 con 0x7fa99c09f410 2023-09-30T22:30:05.585 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:30:05.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.587+0000 7fa9a41ab700 1 -- 172.21.15.16:0/2198406080 >> v1:172.21.15.16:6816/126811 conn(0x7fa98002f540 legacy=0x7fa9800319f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:30:05.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.587+0000 7fa9a41ab700 1 -- 172.21.15.16:0/2198406080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa99c09f410 msgr2=0x7fa99c06c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:05.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.587+0000 7fa9a41ab700 1 --2- 172.21.15.16:0/2198406080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa99c09f410 0x7fa99c06c850 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa98c0090b0 tx=0x7fa98c004750 comp rx=0 tx=0).stop 2023-09-30T22:30:05.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.587+0000 7fa9a41ab700 1 -- 172.21.15.16:0/2198406080 shutdown_connections 2023-09-30T22:30:05.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.587+0000 7fa9a41ab700 1 --2- 172.21.15.16:0/2198406080 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa99c136fe0 0x7fa99c1c8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:05.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.587+0000 7fa9a41ab700 1 --2- 172.21.15.16:0/2198406080 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa99c09e930 0x7fa99c06c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:05.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.587+0000 7fa9a41ab700 1 --2- 172.21.15.16:0/2198406080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa99c09f410 0x7fa99c06c850 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:05.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.587+0000 7fa9a41ab700 1 -- 172.21.15.16:0/2198406080 >> 172.21.15.16:0/2198406080 conn(0x7fa99c0a4210 msgr2=0x7fa99c09be30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:05.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.587+0000 7fa9a41ab700 1 -- 172.21.15.16:0/2198406080 shutdown_connections 2023-09-30T22:30:05.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.587+0000 7fa9a41ab700 1 -- 172.21.15.16:0/2198406080 wait complete. 2023-09-30T22:30:05.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.597+0000 7fb6bbfff700 1 -- 172.21.15.16:0/2458454779 >> v1:172.21.15.16:6816/126811 conn(0x7fb6a402f560 legacy=0x7fb6a4031a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:05.601 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:30:05.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.614+0000 7fb6c910f700 1 -- 172.21.15.16:0/2458454779 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb6c409ffe0 con 0x7fb6c409e930 2023-09-30T22:30:05.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.615+0000 7fb6bb7fe700 1 -- 172.21.15.16:0/2458454779 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb6b402f020 con 0x7fb6c409e930 2023-09-30T22:30:05.615 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:30:05.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.617+0000 7fb6c910f700 1 -- 172.21.15.16:0/2458454779 >> v1:172.21.15.16:6816/126811 conn(0x7fb6a402f560 legacy=0x7fb6a4031a10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:30:05.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.617+0000 7fb6c910f700 1 -- 172.21.15.16:0/2458454779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6c409e930 msgr2=0x7fb6c41c3db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:05.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.617+0000 7fb6c910f700 1 --2- 172.21.15.16:0/2458454779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6c409e930 0x7fb6c41c3db0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb6c406ef10 tx=0x7fb6b4004750 comp rx=0 tx=0).stop 2023-09-30T22:30:05.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.617+0000 7fb6c910f700 1 -- 172.21.15.16:0/2458454779 shutdown_connections 2023-09-30T22:30:05.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.617+0000 7fb6c910f700 1 --2- 172.21.15.16:0/2458454779 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb6c409f650 0x7fb6c41c4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:05.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.617+0000 7fb6c910f700 1 --2- 172.21.15.16:0/2458454779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6c409e930 0x7fb6c41c3db0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:05.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.617+0000 7fb6c910f700 1 --2- 172.21.15.16:0/2458454779 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb6c409f240 0x7fb6c41c42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:05.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.617+0000 7fb6c910f700 1 -- 172.21.15.16:0/2458454779 >> 172.21.15.16:0/2458454779 conn(0x7fb6c40a4210 msgr2=0x7fb6c40a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:05.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.617+0000 7fb6c910f700 1 -- 172.21.15.16:0/2458454779 shutdown_connections 2023-09-30T22:30:05.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:05.617+0000 7fb6c910f700 1 -- 172.21.15.16:0/2458454779 wait complete. 2023-09-30T22:30:05.631 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:30:06.486 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:06.528 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:06.602 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:06.632 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:06.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.637+0000 7fb03aee8700 1 Processor -- start 2023-09-30T22:30:06.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.637+0000 7fb03aee8700 1 -- start start 2023-09-30T22:30:06.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.637+0000 7fb03aee8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb03409e1d0 0x7fb034138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:06.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.637+0000 7fb03aee8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb034139170 0x7fb03413b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:06.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.637+0000 7fb03aee8700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb03409b120 con 0x7fb03409d6f0 2023-09-30T22:30:06.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.637+0000 7fb03aee8700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb03406b460 con 0x7fb03409e1d0 2023-09-30T22:30:06.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.637+0000 7fb03aee8700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb03409a7e0 con 0x7fb034139170 2023-09-30T22:30:06.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.638+0000 7fb038c84700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb03409d6f0 0x7fb03409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57660/0 (socket says 172.21.15.16:57660) 2023-09-30T22:30:06.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.638+0000 7fb038c84700 1 -- 172.21.15.16:0/3321710131 learned_addr learned my addr 172.21.15.16:0/3321710131 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:30:06.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.638+0000 7fb033fff700 1 --2- 172.21.15.16:0/3321710131 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb03409e1d0 0x7fb034138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:06.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.638+0000 7fb039485700 1 --2- 172.21.15.16:0/3321710131 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb034139170 0x7fb03413b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:06.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.638+0000 7fb0337fe700 1 -- 172.21.15.16:0/3321710131 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1715538270 0 0) 0x7fb03409b120 con 0x7fb03409d6f0 2023-09-30T22:30:06.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.638+0000 7fb0337fe700 1 -- 172.21.15.16:0/3321710131 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0240035b0 con 0x7fb03409d6f0 2023-09-30T22:30:06.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.638+0000 7fb033fff700 1 -- 172.21.15.16:0/3321710131 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb034139170 msgr2=0x7fb03413b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:06.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.638+0000 7fb033fff700 1 --2- 172.21.15.16:0/3321710131 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb034139170 0x7fb03413b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:06.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.638+0000 7fb033fff700 1 -- 172.21.15.16:0/3321710131 >> v1:172.21.15.16:6789/0 conn(0x7fb03409d6f0 legacy=0x7fb03409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:06.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.638+0000 7fb033fff700 1 -- 172.21.15.16:0/3321710131 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb03409a500 con 0x7fb03409e1d0 2023-09-30T22:30:06.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.638+0000 7fb033fff700 1 --2- 172.21.15.16:0/3321710131 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb03409e1d0 0x7fb034138c30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb020009870 tx=0x7fb020009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7147d244ff5dd363 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:06.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.639+0000 7fb0337fe700 1 -- 172.21.15.16:0/3321710131 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb020011830 con 0x7fb03409e1d0 2023-09-30T22:30:06.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.639+0000 7fb0337fe700 1 -- 172.21.15.16:0/3321710131 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb020011990 con 0x7fb03409e1d0 2023-09-30T22:30:06.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.639+0000 7fb0337fe700 1 -- 172.21.15.16:0/3321710131 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb020011d20 con 0x7fb03409e1d0 2023-09-30T22:30:06.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.639+0000 7fb03aee8700 1 -- 172.21.15.16:0/3321710131 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb03409e1d0 msgr2=0x7fb034138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:06.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.639+0000 7fb03aee8700 1 --2- 172.21.15.16:0/3321710131 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb03409e1d0 0x7fb034138c30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb020009870 tx=0x7fb020009b80 comp rx=0 tx=0).stop 2023-09-30T22:30:06.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.639+0000 7fb03aee8700 1 -- 172.21.15.16:0/3321710131 shutdown_connections 2023-09-30T22:30:06.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.639+0000 7fb03aee8700 1 --2- 172.21.15.16:0/3321710131 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb034139170 0x7fb03413b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:06.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.640+0000 7fb03aee8700 1 --2- 172.21.15.16:0/3321710131 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb03409e1d0 0x7fb034138c30 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:06.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.640+0000 7fb03aee8700 1 -- 172.21.15.16:0/3321710131 >> 172.21.15.16:0/3321710131 conn(0x7fb034066f50 msgr2=0x7fb03409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:06.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.640+0000 7fb03aee8700 1 -- 172.21.15.16:0/3321710131 shutdown_connections 2023-09-30T22:30:06.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.640+0000 7fb03aee8700 1 -- 172.21.15.16:0/3321710131 wait complete. 2023-09-30T22:30:06.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.641+0000 7fb03aee8700 1 Processor -- start 2023-09-30T22:30:06.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.641+0000 7fb03aee8700 1 -- start start 2023-09-30T22:30:06.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.642+0000 7fb03aee8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb03409d6f0 0x7fb034120710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:06.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.642+0000 7fb03aee8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb03409e1d0 0x7fb034120c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:06.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.642+0000 7fb03aee8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb034139170 0x7fb034121190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:06.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.642+0000 7fb03aee8700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb034123ce0 con 0x7fb03409d6f0 2023-09-30T22:30:06.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.642+0000 7fb03aee8700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0341239a0 con 0x7fb03409e1d0 2023-09-30T22:30:06.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.642+0000 7fb03aee8700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb034135f90 con 0x7fb034139170 2023-09-30T22:30:06.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.642+0000 7fb033fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb03409e1d0 0x7fb034120c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:06.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.643+0000 7fb033fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb03409e1d0 0x7fb034120c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:55128/0 (socket says 172.21.15.16:55128) 2023-09-30T22:30:06.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.643+0000 7fb033fff700 1 -- 172.21.15.16:0/2564710778 learned_addr learned my addr 172.21.15.16:0/2564710778 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:06.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.643+0000 7fb039485700 1 --2- 172.21.15.16:0/2564710778 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb034139170 0x7fb034121190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:06.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.643+0000 7fb033fff700 1 -- 172.21.15.16:0/2564710778 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb034139170 msgr2=0x7fb034121190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:06.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.643+0000 7fb033fff700 1 --2- 172.21.15.16:0/2564710778 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb034139170 0x7fb034121190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:06.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.643+0000 7fb033fff700 1 -- 172.21.15.16:0/2564710778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb03409d6f0 msgr2=0x7fb034120710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:06.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.643+0000 7fb033fff700 1 --2- 172.21.15.16:0/2564710778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb03409d6f0 0x7fb034120710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:06.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.643+0000 7fb033fff700 1 -- 172.21.15.16:0/2564710778 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0200094a0 con 0x7fb03409e1d0 2023-09-30T22:30:06.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.644+0000 7fb033fff700 1 --2- 172.21.15.16:0/2564710778 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb03409e1d0 0x7fb034120c50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb0200090b0 tx=0x7fb020004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:06.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.644+0000 7fb0317fa700 1 -- 172.21.15.16:0/2564710778 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb020004510 con 0x7fb03409e1d0 2023-09-30T22:30:06.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.644+0000 7fb0317fa700 1 -- 172.21.15.16:0/2564710778 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb02001f070 con 0x7fb03409e1d0 2023-09-30T22:30:06.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.644+0000 7fb03aee8700 1 -- 172.21.15.16:0/2564710778 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0340a1770 con 0x7fb03409e1d0 2023-09-30T22:30:06.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.644+0000 7fb0317fa700 1 -- 172.21.15.16:0/2564710778 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0200038e0 con 0x7fb03409e1d0 2023-09-30T22:30:06.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.644+0000 7fb03aee8700 1 -- 172.21.15.16:0/2564710778 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb03409c770 con 0x7fb03409e1d0 2023-09-30T22:30:06.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.645+0000 7fb0317fa700 1 -- 172.21.15.16:0/2564710778 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fb020003a40 con 0x7fb03409e1d0 2023-09-30T22:30:06.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.645+0000 7fb0317fa700 1 -- 172.21.15.16:0/2564710778 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7fb020021070 con 0x7fb03409e1d0 2023-09-30T22:30:06.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.646+0000 7fb038c84700 1 -- 172.21.15.16:0/2564710778 >> v1:172.21.15.16:6816/126811 conn(0x7fb01802f510 legacy=0x7fb0180319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:06.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.646+0000 7fb03aee8700 1 -- 172.21.15.16:0/2564710778 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0341249f0 con 0x7fb03409e1d0 2023-09-30T22:30:06.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.651+0000 7fb0317fa700 1 -- 172.21.15.16:0/2564710778 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb02001a430 con 0x7fb03409e1d0 2023-09-30T22:30:06.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.679+0000 7f14fe3a4700 1 Processor -- start 2023-09-30T22:30:06.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.679+0000 7f14fe3a4700 1 -- start start 2023-09-30T22:30:06.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.679+0000 7f14fe3a4700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f14f812cf60 0x7f14f806f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:06.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.679+0000 7f14fe3a4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f14f812f8b0 0x7f14f812fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:06.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.679+0000 7f14fe3a4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f14f8130260 0x7f14f8136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:06.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.679+0000 7f14fe3a4700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14f812d330 con 0x7f14f812f8b0 2023-09-30T22:30:06.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.679+0000 7f14fe3a4700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14f812cd90 con 0x7f14f812cf60 2023-09-30T22:30:06.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.679+0000 7f14fe3a4700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f14f81306d0 con 0x7f14f8130260 2023-09-30T22:30:06.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.680+0000 7f14fc941700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f14f8130260 0x7f14f8136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:06.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.680+0000 7f14fc941700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f14f8130260 0x7f14f8136c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:45372/0 (socket says 172.21.15.16:45372) 2023-09-30T22:30:06.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.680+0000 7f14fc941700 1 -- 172.21.15.16:0/2178605365 learned_addr learned my addr 172.21.15.16:0/2178605365 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:06.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.680+0000 7f14f77fe700 1 --2- 172.21.15.16:0/2178605365 >> v2:172.21.15.16:3300/0 conn(0x7f14f812f8b0 0x7f14f812fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:06.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.680+0000 7f14f7fff700 1 --2- 172.21.15.16:0/2178605365 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f14f812cf60 0x7f14f806f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:06.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.680+0000 7f14f77fe700 1 -- 172.21.15.16:0/2178605365 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f14f8130260 msgr2=0x7f14f8136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:06.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.680+0000 7f14f77fe700 1 --2- 172.21.15.16:0/2178605365 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f14f8130260 0x7f14f8136c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:06.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.680+0000 7f14f77fe700 1 -- 172.21.15.16:0/2178605365 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f14f812cf60 msgr2=0x7f14f806f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:06.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.680+0000 7f14f77fe700 1 --2- 172.21.15.16:0/2178605365 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f14f812cf60 0x7f14f806f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:06.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.680+0000 7f14f77fe700 1 -- 172.21.15.16:0/2178605365 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14f8071780 con 0x7f14f812f8b0 2023-09-30T22:30:06.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.680+0000 7f14fc941700 1 --2- 172.21.15.16:0/2178605365 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f14f8130260 0x7f14f8136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:06.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.680+0000 7f14f7fff700 1 --2- 172.21.15.16:0/2178605365 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f14f812cf60 0x7f14f806f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:30:06.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.680+0000 7f14f77fe700 1 --2- 172.21.15.16:0/2178605365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14f812f8b0 0x7f14f812fd20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f14ec00aa10 tx=0x7f14ec00ad20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=469dcfedab05f22f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:06.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.681+0000 7f14f6ffd700 1 -- 172.21.15.16:0/2178605365 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ec00cb30 con 0x7f14f812f8b0 2023-09-30T22:30:06.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.681+0000 7f14f6ffd700 1 -- 172.21.15.16:0/2178605365 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14ec00cc90 con 0x7f14f812f8b0 2023-09-30T22:30:06.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.681+0000 7f14f6ffd700 1 -- 172.21.15.16:0/2178605365 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ec018400 con 0x7f14f812f8b0 2023-09-30T22:30:06.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.682+0000 7f14fe3a4700 1 -- 172.21.15.16:0/2178605365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14f812f8b0 msgr2=0x7f14f812fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:06.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.682+0000 7f14fe3a4700 1 --2- 172.21.15.16:0/2178605365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14f812f8b0 0x7f14f812fd20 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f14ec00aa10 tx=0x7f14ec00ad20 comp rx=0 tx=0).stop 2023-09-30T22:30:06.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.682+0000 7f14fe3a4700 1 -- 172.21.15.16:0/2178605365 shutdown_connections 2023-09-30T22:30:06.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.682+0000 7f14fe3a4700 1 --2- 172.21.15.16:0/2178605365 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f14f812cf60 0x7f14f806f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:06.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.682+0000 7f14fe3a4700 1 --2- 172.21.15.16:0/2178605365 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f14f8130260 0x7f14f8136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:06.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.682+0000 7f14fe3a4700 1 --2- 172.21.15.16:0/2178605365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14f812f8b0 0x7f14f812fd20 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:06.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.682+0000 7f14fe3a4700 1 -- 172.21.15.16:0/2178605365 >> 172.21.15.16:0/2178605365 conn(0x7f14f80a2aa0 msgr2=0x7f14f80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:06.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.682+0000 7f14fe3a4700 1 -- 172.21.15.16:0/2178605365 shutdown_connections 2023-09-30T22:30:06.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.682+0000 7f14fe3a4700 1 -- 172.21.15.16:0/2178605365 wait complete. 2023-09-30T22:30:06.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.683+0000 7f14fe3a4700 1 Processor -- start 2023-09-30T22:30:06.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.684+0000 7f14fe3a4700 1 -- start start 2023-09-30T22:30:06.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.684+0000 7f14fe3a4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14f812cf60 0x7f14f81c3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:06.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.684+0000 7f14fe3a4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f14f812f8b0 0x7f14f81c4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:06.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.684+0000 7f14fe3a4700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f14f8130260 0x7f14f81c49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:06.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.684+0000 7f14fe3a4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14f809e530 con 0x7f14f812cf60 2023-09-30T22:30:06.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.684+0000 7f14fe3a4700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14f809d930 con 0x7f14f8130260 2023-09-30T22:30:06.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.684+0000 7f14fe3a4700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f14f809cd40 con 0x7f14f812f8b0 2023-09-30T22:30:06.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.685+0000 7f14f77fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f14f812f8b0 0x7f14f81c4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:06.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.685+0000 7f14f77fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f14f812f8b0 0x7f14f81c4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:45378/0 (socket says 172.21.15.16:45378) 2023-09-30T22:30:06.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.685+0000 7f14f77fe700 1 -- 172.21.15.16:0/3199400598 learned_addr learned my addr 172.21.15.16:0/3199400598 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:06.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.685+0000 7f14fc941700 1 --2- 172.21.15.16:0/3199400598 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f14f8130260 0x7f14f81c49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:06.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.685+0000 7f14f7fff700 1 --2- 172.21.15.16:0/3199400598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14f812cf60 0x7f14f81c3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:06.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.685+0000 7f14f77fe700 1 -- 172.21.15.16:0/3199400598 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f14f8130260 msgr2=0x7f14f81c49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:06.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.685+0000 7f14f77fe700 1 --2- 172.21.15.16:0/3199400598 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f14f8130260 0x7f14f81c49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:06.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.685+0000 7f14f77fe700 1 -- 172.21.15.16:0/3199400598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14f812cf60 msgr2=0x7f14f81c3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:06.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.685+0000 7f14f77fe700 1 --2- 172.21.15.16:0/3199400598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14f812cf60 0x7f14f81c3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:06.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.685+0000 7f14f77fe700 1 -- 172.21.15.16:0/3199400598 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14f0009b30 con 0x7f14f812f8b0 2023-09-30T22:30:06.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.685+0000 7f14fc941700 1 --2- 172.21.15.16:0/3199400598 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f14f8130260 0x7f14f81c49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:06.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.686+0000 7f14f77fe700 1 --2- 172.21.15.16:0/3199400598 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f14f812f8b0 0x7f14f81c4460 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f14ec00a1c0 tx=0x7f14ec004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:06.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.686+0000 7f14f4ff9700 1 -- 172.21.15.16:0/3199400598 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ec01e070 con 0x7f14f812f8b0 2023-09-30T22:30:06.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.686+0000 7f14f4ff9700 1 -- 172.21.15.16:0/3199400598 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14ec0036a0 con 0x7f14f812f8b0 2023-09-30T22:30:06.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.686+0000 7f14fe3a4700 1 -- 172.21.15.16:0/3199400598 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14ec00a640 con 0x7f14f812f8b0 2023-09-30T22:30:06.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.686+0000 7f14f4ff9700 1 -- 172.21.15.16:0/3199400598 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ec003a30 con 0x7f14f812f8b0 2023-09-30T22:30:06.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.686+0000 7f14fe3a4700 1 -- 172.21.15.16:0/3199400598 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14f8119220 con 0x7f14f812f8b0 2023-09-30T22:30:06.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.687+0000 7f14f4ff9700 1 -- 172.21.15.16:0/3199400598 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f14ec003b90 con 0x7f14f812f8b0 2023-09-30T22:30:06.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.687+0000 7f14f4ff9700 1 -- 172.21.15.16:0/3199400598 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7f14ec040b70 con 0x7f14f812f8b0 2023-09-30T22:30:06.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.687+0000 7f14f7fff700 1 -- 172.21.15.16:0/3199400598 >> v1:172.21.15.16:6816/126811 conn(0x7f14e402f510 legacy=0x7f14e40319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:06.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.687+0000 7f14fe3a4700 1 -- 172.21.15.16:0/3199400598 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14f8074ea0 con 0x7f14f812f8b0 2023-09-30T22:30:06.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.697+0000 7f14f4ff9700 1 -- 172.21.15.16:0/3199400598 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f14ec018400 con 0x7f14f812f8b0 2023-09-30T22:30:06.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.783+0000 7f3a16d6a700 1 Processor -- start 2023-09-30T22:30:06.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.783+0000 7f3a16d6a700 1 -- start start 2023-09-30T22:30:06.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.783+0000 7f98b380f700 1 Processor -- start 2023-09-30T22:30:06.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.783+0000 7f3a16d6a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3a1006b250 0x7f3a10136840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:06.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.783+0000 7f3a16d6a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3a10136d80 0x7f3a10139160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:06.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.783+0000 7f3a16d6a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a1009ece0 con 0x7f3a1006a770 2023-09-30T22:30:06.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.783+0000 7f3a16d6a700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a1009e760 con 0x7f3a1006b250 2023-09-30T22:30:06.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.783+0000 7f3a16d6a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1009e040 con 0x7f3a10136d80 2023-09-30T22:30:06.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.784+0000 7f98b380f700 1 -- start start 2023-09-30T22:30:06.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.783+0000 7f3a15307700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3a10136d80 0x7f3a10139160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:06.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.784+0000 7f3a15307700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3a10136d80 0x7f3a10139160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:55148/0 (socket says 172.21.15.16:55148) 2023-09-30T22:30:06.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.784+0000 7f3a15307700 1 -- 172.21.15.16:0/4005138985 learned_addr learned my addr 172.21.15.16:0/4005138985 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:06.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.784+0000 7f3a0ffff700 1 --2- 172.21.15.16:0/4005138985 >> v2:172.21.15.16:3300/0 conn(0x7f3a1006b250 0x7f3a10136840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:06.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.784+0000 7f3a0f7fe700 1 -- 172.21.15.16:0/4005138985 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3956228478 0 0) 0x7f3a1009ece0 con 0x7f3a1006a770 2023-09-30T22:30:06.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.784+0000 7f3a0f7fe700 1 -- 172.21.15.16:0/4005138985 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a000035b0 con 0x7f3a1006a770 2023-09-30T22:30:06.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.784+0000 7f3a15307700 1 -- 172.21.15.16:0/4005138985 >> v2:172.21.15.16:3300/0 conn(0x7f3a1006b250 msgr2=0x7f3a10136840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:06.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.784+0000 7f3a15307700 1 --2- 172.21.15.16:0/4005138985 >> v2:172.21.15.16:3300/0 conn(0x7f3a1006b250 0x7f3a10136840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:06.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.784+0000 7f3a15307700 1 -- 172.21.15.16:0/4005138985 >> v1:172.21.15.16:6789/0 conn(0x7f3a1006a770 legacy=0x7f3a1006ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:06.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.784+0000 7f3a15307700 1 -- 172.21.15.16:0/4005138985 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a1009f630 con 0x7f3a10136d80 2023-09-30T22:30:06.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.784+0000 7f98b380f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98ac09e930 0x7f98ac09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:06.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.784+0000 7f98b380f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f98ac1342f0 0x7f98ac13be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:06.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.784+0000 7f98b380f700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98ac06c9f0 con 0x7f98ac09f240 2023-09-30T22:30:06.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.784+0000 7f98b380f700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98ac06c670 con 0x7f98ac1342f0 2023-09-30T22:30:06.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.784+0000 7f98b380f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac06bc90 con 0x7f98ac09e930 2023-09-30T22:30:06.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.784+0000 7f98b0daa700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f98ac09f240 0x7f98ac133be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57684/0 (socket says 172.21.15.16:57684) 2023-09-30T22:30:06.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.784+0000 7f98b0daa700 1 -- 172.21.15.16:0/1888370815 learned_addr learned my addr 172.21.15.16:0/1888370815 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:30:06.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.784+0000 7f98b15ab700 1 --2- 172.21.15.16:0/1888370815 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98ac09e930 0x7f98ac09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:06.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.784+0000 7f3a15307700 1 --2- 172.21.15.16:0/4005138985 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3a10136d80 0x7f3a10139160 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3a08009e50 tx=0x7f3a0800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7320234cb9c436 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:06.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.785+0000 7f98b1dac700 1 --2- 172.21.15.16:0/1888370815 >> v2:172.21.15.16:3300/0 conn(0x7f98ac1342f0 0x7f98ac13be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:06.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.785+0000 7f98b15ab700 1 -- 172.21.15.16:0/1888370815 >> v2:172.21.15.16:3300/0 conn(0x7f98ac1342f0 msgr2=0x7f98ac13be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:06.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.785+0000 7f98b15ab700 1 --2- 172.21.15.16:0/1888370815 >> v2:172.21.15.16:3300/0 conn(0x7f98ac1342f0 0x7f98ac13be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:06.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.785+0000 7f98b15ab700 1 -- 172.21.15.16:0/1888370815 >> v1:172.21.15.16:6789/0 conn(0x7f98ac09f240 legacy=0x7f98ac133be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:06.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.785+0000 7f98b15ab700 1 -- 172.21.15.16:0/1888370815 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98ac06d860 con 0x7f98ac09e930 2023-09-30T22:30:06.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.785+0000 7f98b15ab700 1 --2- 172.21.15.16:0/1888370815 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98ac09e930 0x7f98ac09ed00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f98a8009840 tx=0x7f98a8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49fd10f1e2d40e72 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:06.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.785+0000 7f3a0f7fe700 1 -- 172.21.15.16:0/4005138985 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0800e070 con 0x7f3a10136d80 2023-09-30T22:30:06.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.785+0000 7f3a0f7fe700 1 -- 172.21.15.16:0/4005138985 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a08003a20 con 0x7f3a10136d80 2023-09-30T22:30:06.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.785+0000 7f3a0f7fe700 1 -- 172.21.15.16:0/4005138985 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a08003dd0 con 0x7f3a10136d80 2023-09-30T22:30:06.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.785+0000 7f98a3fff700 1 -- 172.21.15.16:0/1888370815 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a8004230 con 0x7f98ac09e930 2023-09-30T22:30:06.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.785+0000 7f98a3fff700 1 -- 172.21.15.16:0/1888370815 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98a8004390 con 0x7f98ac09e930 2023-09-30T22:30:06.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.786+0000 7f98a3fff700 1 -- 172.21.15.16:0/1888370815 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a80036a0 con 0x7f98ac09e930 2023-09-30T22:30:06.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.786+0000 7f3a16d6a700 1 -- 172.21.15.16:0/4005138985 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3a10136d80 msgr2=0x7f3a10139160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:06.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.786+0000 7f3a16d6a700 1 --2- 172.21.15.16:0/4005138985 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3a10136d80 0x7f3a10139160 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3a08009e50 tx=0x7f3a0800b330 comp rx=0 tx=0).stop 2023-09-30T22:30:06.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.786+0000 7f3a16d6a700 1 -- 172.21.15.16:0/4005138985 shutdown_connections 2023-09-30T22:30:06.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.786+0000 7f3a16d6a700 1 --2- 172.21.15.16:0/4005138985 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3a10136d80 0x7f3a10139160 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:06.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.786+0000 7f3a16d6a700 1 --2- 172.21.15.16:0/4005138985 >> v2:172.21.15.16:3300/0 conn(0x7f3a1006b250 0x7f3a10136840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:06.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.786+0000 7f3a16d6a700 1 -- 172.21.15.16:0/4005138985 >> 172.21.15.16:0/4005138985 conn(0x7f3a10066f50 msgr2=0x7f3a1009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:06.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.786+0000 7f3a16d6a700 1 -- 172.21.15.16:0/4005138985 shutdown_connections 2023-09-30T22:30:06.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.786+0000 7f3a16d6a700 1 -- 172.21.15.16:0/4005138985 wait complete. 2023-09-30T22:30:06.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.786+0000 7f98b380f700 1 -- 172.21.15.16:0/1888370815 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98ac09e930 msgr2=0x7f98ac09ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:06.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.786+0000 7f98b380f700 1 --2- 172.21.15.16:0/1888370815 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98ac09e930 0x7f98ac09ed00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f98a8009840 tx=0x7f98a8009b50 comp rx=0 tx=0).stop 2023-09-30T22:30:06.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.787+0000 7f3a16d6a700 1 Processor -- start 2023-09-30T22:30:06.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.787+0000 7f3a16d6a700 1 -- start start 2023-09-30T22:30:06.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.788+0000 7f3a16d6a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3a1006a770 0x7f3a10120100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:06.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.788+0000 7f3a16d6a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3a1006b250 0x7f3a1011e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:06.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.788+0000 7f3a16d6a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a10136d80 0x7f3a1011edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:06.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.788+0000 7f3a16d6a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a10071e60 con 0x7f3a10136d80 2023-09-30T22:30:06.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.788+0000 7f3a16d6a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1009c070 con 0x7f3a1006a770 2023-09-30T22:30:06.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.788+0000 7f3a16d6a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3a100711a0 con 0x7f3a1006b250 2023-09-30T22:30:06.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.788+0000 7f3a14b06700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3a1006a770 0x7f3a10120100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:06.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.788+0000 7f3a15307700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a10136d80 0x7f3a1011edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:06.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.788+0000 7f3a14b06700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3a1006a770 0x7f3a10120100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:55150/0 (socket says 172.21.15.16:55150) 2023-09-30T22:30:06.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.788+0000 7f3a14b06700 1 -- 172.21.15.16:0/374504116 learned_addr learned my addr 172.21.15.16:0/374504116 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:06.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.788+0000 7f3a0ffff700 1 --2- 172.21.15.16:0/374504116 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3a1006b250 0x7f3a1011e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:06.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.788+0000 7f98b380f700 1 -- 172.21.15.16:0/1888370815 shutdown_connections 2023-09-30T22:30:06.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.788+0000 7f98b380f700 1 --2- 172.21.15.16:0/1888370815 >> v2:172.21.15.16:3300/0 conn(0x7f98ac1342f0 0x7f98ac13be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:06.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.788+0000 7f98b380f700 1 --2- 172.21.15.16:0/1888370815 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98ac09e930 0x7f98ac09ed00 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:06.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.788+0000 7f98b380f700 1 -- 172.21.15.16:0/1888370815 >> 172.21.15.16:0/1888370815 conn(0x7f98ac0a4210 msgr2=0x7f98ac0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:06.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.788+0000 7f3a15307700 1 -- 172.21.15.16:0/374504116 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3a1006b250 msgr2=0x7f3a1011e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:06.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.788+0000 7f3a15307700 1 --2- 172.21.15.16:0/374504116 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3a1006b250 0x7f3a1011e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:06.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.788+0000 7f3a15307700 1 -- 172.21.15.16:0/374504116 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3a1006a770 msgr2=0x7f3a10120100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:06.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.788+0000 7f3a15307700 1 --2- 172.21.15.16:0/374504116 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3a1006a770 0x7f3a10120100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:06.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.788+0000 7f3a15307700 1 -- 172.21.15.16:0/374504116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a08009b30 con 0x7f3a10136d80 2023-09-30T22:30:06.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.789+0000 7f3a15307700 1 --2- 172.21.15.16:0/374504116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a10136d80 0x7f3a1011edd0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3a08014000 tx=0x7f3a08004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:06.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.789+0000 7f98b380f700 1 -- 172.21.15.16:0/1888370815 shutdown_connections 2023-09-30T22:30:06.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.789+0000 7f98b380f700 1 -- 172.21.15.16:0/1888370815 wait complete. 2023-09-30T22:30:06.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.789+0000 7f3a0d7fa700 1 -- 172.21.15.16:0/374504116 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a08004440 con 0x7f3a10136d80 2023-09-30T22:30:06.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.789+0000 7f3a0d7fa700 1 -- 172.21.15.16:0/374504116 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a080045c0 con 0x7f3a10136d80 2023-09-30T22:30:06.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.789+0000 7f3a16d6a700 1 -- 172.21.15.16:0/374504116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a1011d710 con 0x7f3a10136d80 2023-09-30T22:30:06.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.789+0000 7f3a0d7fa700 1 -- 172.21.15.16:0/374504116 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a08004970 con 0x7f3a10136d80 2023-09-30T22:30:06.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.789+0000 7f3a16d6a700 1 -- 172.21.15.16:0/374504116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a1011dcb0 con 0x7f3a10136d80 2023-09-30T22:30:06.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.790+0000 7f98b380f700 1 Processor -- start 2023-09-30T22:30:06.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.790+0000 7f98b380f700 1 -- start start 2023-09-30T22:30:06.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.790+0000 7f3a0d7fa700 1 -- 172.21.15.16:0/374504116 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f3a08019070 con 0x7f3a10136d80 2023-09-30T22:30:06.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.790+0000 7f3a0d7fa700 1 -- 172.21.15.16:0/374504116 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7f3a08040af0 con 0x7f3a10136d80 2023-09-30T22:30:06.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.790+0000 7f3a14b06700 1 -- 172.21.15.16:0/374504116 >> v1:172.21.15.16:6816/126811 conn(0x7f39f402f510 legacy=0x7f39f40319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:06.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.790+0000 7f3a16d6a700 1 -- 172.21.15.16:0/374504116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a10074ea0 con 0x7f3a10136d80 2023-09-30T22:30:06.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.791+0000 7f98b380f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98ac09e930 0x7f98ac1cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:06.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.791+0000 7f98b380f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f98ac09f240 0x7f98ac1cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:06.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.791+0000 7f98b380f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98ac1342f0 0x7f98ac1cd230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:06.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.791+0000 7f98b380f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac128950 con 0x7f98ac09e930 2023-09-30T22:30:06.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.791+0000 7f98b380f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac129d90 con 0x7f98ac09f240 2023-09-30T22:30:06.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.791+0000 7f98b380f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f98ac1273b0 con 0x7f98ac1342f0 2023-09-30T22:30:06.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.791+0000 7f98b15ab700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98ac09e930 0x7f98ac1cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:06.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.791+0000 7f98b15ab700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98ac09e930 0x7f98ac1cc7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53194/0 (socket says 172.21.15.16:53194) 2023-09-30T22:30:06.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.791+0000 7f98b15ab700 1 -- 172.21.15.16:0/2869397132 learned_addr learned my addr 172.21.15.16:0/2869397132 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:06.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.792+0000 7f98b15ab700 1 -- 172.21.15.16:0/2869397132 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98ac1342f0 msgr2=0x7f98ac1cd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:06.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.792+0000 7f98b15ab700 1 --2- 172.21.15.16:0/2869397132 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98ac1342f0 0x7f98ac1cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:06.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.792+0000 7f98b15ab700 1 -- 172.21.15.16:0/2869397132 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f98ac09f240 msgr2=0x7f98ac1cccf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:30:06.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.792+0000 7f98b15ab700 1 --2- 172.21.15.16:0/2869397132 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f98ac09f240 0x7f98ac1cccf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:06.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.792+0000 7f98b15ab700 1 -- 172.21.15.16:0/2869397132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a80094a0 con 0x7f98ac09e930 2023-09-30T22:30:06.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.792+0000 7f98b15ab700 1 --2- 172.21.15.16:0/2869397132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98ac09e930 0x7f98ac1cc7b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f98a80090b0 tx=0x7f98a8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:06.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.796+0000 7f98a1ffb700 1 -- 172.21.15.16:0/2869397132 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a8003e30 con 0x7f98ac09e930 2023-09-30T22:30:06.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.796+0000 7f98a1ffb700 1 -- 172.21.15.16:0/2869397132 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98a801f070 con 0x7f98ac09e930 2023-09-30T22:30:06.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.796+0000 7f98a1ffb700 1 -- 172.21.15.16:0/2869397132 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a8011870 con 0x7f98ac09e930 2023-09-30T22:30:06.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.796+0000 7f98b380f700 1 -- 172.21.15.16:0/2869397132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98ac1294c0 con 0x7f98ac09e930 2023-09-30T22:30:06.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.796+0000 7f3a0d7fa700 1 -- 172.21.15.16:0/374504116 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3a08017070 con 0x7f3a10136d80 2023-09-30T22:30:06.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.797+0000 7f98b380f700 1 -- 172.21.15.16:0/2869397132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98ac129b30 con 0x7f98ac09e930 2023-09-30T22:30:06.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.798+0000 7f98b380f700 1 -- 172.21.15.16:0/2869397132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98ac11dd30 con 0x7f98ac09e930 2023-09-30T22:30:06.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.798+0000 7f98a1ffb700 1 -- 172.21.15.16:0/2869397132 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f98a8021030 con 0x7f98ac09e930 2023-09-30T22:30:06.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.798+0000 7f98a1ffb700 1 -- 172.21.15.16:0/2869397132 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7f98a8040fb0 con 0x7f98ac09e930 2023-09-30T22:30:06.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.798+0000 7f98b0daa700 1 -- 172.21.15.16:0/2869397132 >> v1:172.21.15.16:6816/126811 conn(0x7f989002f560 legacy=0x7f9890031a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:06.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.803+0000 7f98a1ffb700 1 -- 172.21.15.16:0/2869397132 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f98a8006bd0 con 0x7f98ac09e930 2023-09-30T22:30:06.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.847+0000 7fb038c84700 1 -- 172.21.15.16:0/2564710778 >> v1:172.21.15.16:6816/126811 conn(0x7fb01802f510 legacy=0x7fb0180319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:06.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.877+0000 7fb03aee8700 1 -- 172.21.15.16:0/2564710778 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb034074ea0 con 0x7fb03409e1d0 2023-09-30T22:30:06.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.877+0000 7fb0317fa700 1 -- 172.21.15.16:0/2564710778 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb020015070 con 0x7fb03409e1d0 2023-09-30T22:30:06.878 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:30:06.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.881+0000 7fb03aee8700 1 -- 172.21.15.16:0/2564710778 >> v1:172.21.15.16:6816/126811 conn(0x7fb01802f510 legacy=0x7fb0180319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:30:06.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.881+0000 7fb03aee8700 1 -- 172.21.15.16:0/2564710778 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb03409e1d0 msgr2=0x7fb034120c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:06.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.881+0000 7fb03aee8700 1 --2- 172.21.15.16:0/2564710778 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb03409e1d0 0x7fb034120c50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb0200090b0 tx=0x7fb020004750 comp rx=0 tx=0).stop 2023-09-30T22:30:06.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.881+0000 7fb03aee8700 1 -- 172.21.15.16:0/2564710778 shutdown_connections 2023-09-30T22:30:06.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.881+0000 7fb03aee8700 1 --2- 172.21.15.16:0/2564710778 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb034139170 0x7fb034121190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:06.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.881+0000 7fb03aee8700 1 --2- 172.21.15.16:0/2564710778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb03409d6f0 0x7fb034120710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:06.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.881+0000 7fb03aee8700 1 --2- 172.21.15.16:0/2564710778 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb03409e1d0 0x7fb034120c50 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:06.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.881+0000 7fb03aee8700 1 -- 172.21.15.16:0/2564710778 >> 172.21.15.16:0/2564710778 conn(0x7fb034066f50 msgr2=0x7fb03409b410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:06.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.881+0000 7fb03aee8700 1 -- 172.21.15.16:0/2564710778 shutdown_connections 2023-09-30T22:30:06.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.881+0000 7fb03aee8700 1 -- 172.21.15.16:0/2564710778 wait complete. 2023-09-30T22:30:06.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.888+0000 7f14f7fff700 1 -- 172.21.15.16:0/3199400598 >> v1:172.21.15.16:6816/126811 conn(0x7f14e402f510 legacy=0x7f14e40319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:06.901 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:30:06.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.969+0000 7f14fe3a4700 1 -- 172.21.15.16:0/3199400598 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f14f80713a0 con 0x7f14f812f8b0 2023-09-30T22:30:06.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.970+0000 7f14f4ff9700 1 -- 172.21.15.16:0/3199400598 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f14ec027070 con 0x7f14f812f8b0 2023-09-30T22:30:06.971 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:30:06.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.972+0000 7f14fe3a4700 1 -- 172.21.15.16:0/3199400598 >> v1:172.21.15.16:6816/126811 conn(0x7f14e402f510 legacy=0x7f14e40319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:30:06.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.972+0000 7f14fe3a4700 1 -- 172.21.15.16:0/3199400598 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f14f812f8b0 msgr2=0x7f14f81c4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:06.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.972+0000 7f14fe3a4700 1 --2- 172.21.15.16:0/3199400598 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f14f812f8b0 0x7f14f81c4460 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f14ec00a1c0 tx=0x7f14ec004750 comp rx=0 tx=0).stop 2023-09-30T22:30:06.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.972+0000 7f14fe3a4700 1 -- 172.21.15.16:0/3199400598 shutdown_connections 2023-09-30T22:30:06.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.972+0000 7f14fe3a4700 1 --2- 172.21.15.16:0/3199400598 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f14f8130260 0x7f14f81c49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:06.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.972+0000 7f14fe3a4700 1 --2- 172.21.15.16:0/3199400598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14f812cf60 0x7f14f81c3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:06.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.972+0000 7f14fe3a4700 1 --2- 172.21.15.16:0/3199400598 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f14f812f8b0 0x7f14f81c4460 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:06.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.972+0000 7f14fe3a4700 1 -- 172.21.15.16:0/3199400598 >> 172.21.15.16:0/3199400598 conn(0x7f14f80a2aa0 msgr2=0x7f14f806e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:06.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.973+0000 7f14fe3a4700 1 -- 172.21.15.16:0/3199400598 shutdown_connections 2023-09-30T22:30:06.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.973+0000 7f14fe3a4700 1 -- 172.21.15.16:0/3199400598 wait complete. 2023-09-30T22:30:06.987 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:30:06.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.991+0000 7f3a14b06700 1 -- 172.21.15.16:0/374504116 >> v1:172.21.15.16:6816/126811 conn(0x7f39f402f510 legacy=0x7f39f40319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:06.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:06.999+0000 7f98b0daa700 1 -- 172.21.15.16:0/2869397132 >> v1:172.21.15.16:6816/126811 conn(0x7f989002f560 legacy=0x7f9890031a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:07.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:07.006+0000 7f3a16d6a700 1 -- 172.21.15.16:0/374504116 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f3a1011d8a0 con 0x7f3a10136d80 2023-09-30T22:30:07.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:07.006+0000 7f3a0d7fa700 1 -- 172.21.15.16:0/374504116 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3a08011390 con 0x7f3a10136d80 2023-09-30T22:30:07.007 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:30:07.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:07.009+0000 7f3a16d6a700 1 -- 172.21.15.16:0/374504116 >> v1:172.21.15.16:6816/126811 conn(0x7f39f402f510 legacy=0x7f39f40319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:30:07.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:07.009+0000 7f3a16d6a700 1 -- 172.21.15.16:0/374504116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a10136d80 msgr2=0x7f3a1011edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:07.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:07.009+0000 7f3a16d6a700 1 --2- 172.21.15.16:0/374504116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a10136d80 0x7f3a1011edd0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3a08014000 tx=0x7f3a08004ad0 comp rx=0 tx=0).stop 2023-09-30T22:30:07.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:07.009+0000 7f3a16d6a700 1 -- 172.21.15.16:0/374504116 shutdown_connections 2023-09-30T22:30:07.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:07.009+0000 7f3a16d6a700 1 --2- 172.21.15.16:0/374504116 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3a1006a770 0x7f3a10120100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:07.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:07.009+0000 7f3a16d6a700 1 --2- 172.21.15.16:0/374504116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a10136d80 0x7f3a1011edd0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:07.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:07.009+0000 7f3a16d6a700 1 --2- 172.21.15.16:0/374504116 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3a1006b250 0x7f3a1011e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:07.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:07.009+0000 7f3a16d6a700 1 -- 172.21.15.16:0/374504116 >> 172.21.15.16:0/374504116 conn(0x7f3a10066f50 msgr2=0x7f3a100a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:07.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:07.009+0000 7f3a16d6a700 1 -- 172.21.15.16:0/374504116 shutdown_connections 2023-09-30T22:30:07.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:07.009+0000 7f3a16d6a700 1 -- 172.21.15.16:0/374504116 wait complete. 2023-09-30T22:30:07.023 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:30:07.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:07.024+0000 7f98b380f700 1 -- 172.21.15.16:0/2869397132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f98ac11dd30 con 0x7f98ac09e930 2023-09-30T22:30:07.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:07.025+0000 7f98a1ffb700 1 -- 172.21.15.16:0/2869397132 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f98a80233d0 con 0x7f98ac09e930 2023-09-30T22:30:07.025 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:30:07.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:07.027+0000 7f98b380f700 1 -- 172.21.15.16:0/2869397132 >> v1:172.21.15.16:6816/126811 conn(0x7f989002f560 legacy=0x7f9890031a10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:30:07.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:07.027+0000 7f98b380f700 1 -- 172.21.15.16:0/2869397132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98ac09e930 msgr2=0x7f98ac1cc7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:07.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:07.027+0000 7f98b380f700 1 --2- 172.21.15.16:0/2869397132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98ac09e930 0x7f98ac1cc7b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f98a80090b0 tx=0x7f98a8004750 comp rx=0 tx=0).stop 2023-09-30T22:30:07.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:07.027+0000 7f98b380f700 1 -- 172.21.15.16:0/2869397132 shutdown_connections 2023-09-30T22:30:07.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:07.027+0000 7f98b380f700 1 --2- 172.21.15.16:0/2869397132 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98ac1342f0 0x7f98ac1cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:07.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:07.027+0000 7f98b380f700 1 --2- 172.21.15.16:0/2869397132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98ac09e930 0x7f98ac1cc7b0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:07.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:07.027+0000 7f98b380f700 1 --2- 172.21.15.16:0/2869397132 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f98ac09f240 0x7f98ac1cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:07.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:07.027+0000 7f98b380f700 1 -- 172.21.15.16:0/2869397132 >> 172.21.15.16:0/2869397132 conn(0x7f98ac0a4210 msgr2=0x7f98ac071e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:07.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:07.027+0000 7f98b380f700 1 -- 172.21.15.16:0/2869397132 shutdown_connections 2023-09-30T22:30:07.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:07.027+0000 7f98b380f700 1 -- 172.21.15.16:0/2869397132 wait complete. 2023-09-30T22:30:07.041 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:30:07.901 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:07.987 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:08.025 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:08.041 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:08.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.049+0000 7f3efc395700 1 Processor -- start 2023-09-30T22:30:08.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.049+0000 7f3efc395700 1 -- start start 2023-09-30T22:30:08.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.050+0000 7f3efc395700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3ef4130540 0x7f3ef4130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:08.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.050+0000 7f3efc395700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3ef406fcf0 0x7f3ef4138570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:08.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.050+0000 7f3efc395700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ef40716a0 con 0x7f3ef4130e50 2023-09-30T22:30:08.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.050+0000 7f3efc395700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef409a390 con 0x7f3ef406fcf0 2023-09-30T22:30:08.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.050+0000 7f3efc395700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef4071290 con 0x7f3ef4130540 2023-09-30T22:30:08.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.050+0000 7f3efa932700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3ef406fcf0 0x7f3ef4138570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:08.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.050+0000 7f3ef9930700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3ef4130e50 0x7f3ef406f5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47900/0 (socket says 172.21.15.16:47900) 2023-09-30T22:30:08.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.050+0000 7f3ef9930700 1 -- 172.21.15.16:0/1241871527 learned_addr learned my addr 172.21.15.16:0/1241871527 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:30:08.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.050+0000 7f3efa131700 1 --2- 172.21.15.16:0/1241871527 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3ef4130540 0x7f3ef4130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:08.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.051+0000 7f3efa131700 1 -- 172.21.15.16:0/1241871527 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3ef406fcf0 msgr2=0x7f3ef4138570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:08.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.051+0000 7f3ef912f700 1 -- 172.21.15.16:0/1241871527 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2086417904 0 0) 0x7f3ef40716a0 con 0x7f3ef4130e50 2023-09-30T22:30:08.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.051+0000 7f3efa131700 1 --2- 172.21.15.16:0/1241871527 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3ef406fcf0 0x7f3ef4138570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:08.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.051+0000 7f3efa131700 1 -- 172.21.15.16:0/1241871527 >> v1:172.21.15.16:6789/0 conn(0x7f3ef4130e50 legacy=0x7f3ef406f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:08.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.051+0000 7f3efa131700 1 -- 172.21.15.16:0/1241871527 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef4130340 con 0x7f3ef4130540 2023-09-30T22:30:08.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.051+0000 7f3efa932700 1 --2- 172.21.15.16:0/1241871527 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3ef406fcf0 0x7f3ef4138570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:08.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.051+0000 7f3efa131700 1 --2- 172.21.15.16:0/1241871527 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3ef4130540 0x7f3ef4130910 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3ef000bdc0 tx=0x7f3ef000d330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19890da083976dbd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:08.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.051+0000 7f3ef912f700 1 -- 172.21.15.16:0/1241871527 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0010070 con 0x7f3ef4130540 2023-09-30T22:30:08.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.051+0000 7f3ef912f700 1 -- 172.21.15.16:0/1241871527 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ef000db20 con 0x7f3ef4130540 2023-09-30T22:30:08.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.051+0000 7f3ef912f700 1 -- 172.21.15.16:0/1241871527 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0014430 con 0x7f3ef4130540 2023-09-30T22:30:08.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.052+0000 7f3efc395700 1 -- 172.21.15.16:0/1241871527 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3ef4130540 msgr2=0x7f3ef4130910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:08.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.052+0000 7f3efc395700 1 --2- 172.21.15.16:0/1241871527 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3ef4130540 0x7f3ef4130910 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3ef000bdc0 tx=0x7f3ef000d330 comp rx=0 tx=0).stop 2023-09-30T22:30:08.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.052+0000 7f3efc395700 1 -- 172.21.15.16:0/1241871527 shutdown_connections 2023-09-30T22:30:08.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.052+0000 7f3efc395700 1 --2- 172.21.15.16:0/1241871527 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3ef406fcf0 0x7f3ef4138570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:08.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.052+0000 7f3efc395700 1 --2- 172.21.15.16:0/1241871527 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3ef4130540 0x7f3ef4130910 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:08.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.052+0000 7f3efc395700 1 -- 172.21.15.16:0/1241871527 >> 172.21.15.16:0/1241871527 conn(0x7f3ef40a2aa0 msgr2=0x7f3ef40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:08.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.052+0000 7f3efc395700 1 -- 172.21.15.16:0/1241871527 shutdown_connections 2023-09-30T22:30:08.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.052+0000 7f3efc395700 1 -- 172.21.15.16:0/1241871527 wait complete. 2023-09-30T22:30:08.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.053+0000 7f3efc395700 1 Processor -- start 2023-09-30T22:30:08.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.053+0000 7f3efc395700 1 -- start start 2023-09-30T22:30:08.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.054+0000 7f3efc395700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3ef406fcf0 0x7f3ef411ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:08.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.054+0000 7f3efc395700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ef4130540 0x7f3ef411f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:08.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.054+0000 7f3efc395700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3ef4130e50 0x7f3ef411f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:08.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.054+0000 7f3efc395700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef4122800 con 0x7f3ef4130540 2023-09-30T22:30:08.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.054+0000 7f3efc395700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef4120990 con 0x7f3ef406fcf0 2023-09-30T22:30:08.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.054+0000 7f3efc395700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef40a0950 con 0x7f3ef4130e50 2023-09-30T22:30:08.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.054+0000 7f3efa932700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3ef4130e50 0x7f3ef411f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:08.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.054+0000 7f3efa932700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3ef4130e50 0x7f3ef411f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:50834/0 (socket says 172.21.15.16:50834) 2023-09-30T22:30:08.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.054+0000 7f3efa932700 1 -- 172.21.15.16:0/1610700729 learned_addr learned my addr 172.21.15.16:0/1610700729 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:08.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.054+0000 7f3efa131700 1 --2- 172.21.15.16:0/1610700729 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3ef406fcf0 0x7f3ef411ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:08.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.054+0000 7f3ef9930700 1 --2- 172.21.15.16:0/1610700729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ef4130540 0x7f3ef411f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:08.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.055+0000 7f3ef9930700 1 -- 172.21.15.16:0/1610700729 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3ef4130e50 msgr2=0x7f3ef411f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:08.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.055+0000 7f3ef9930700 1 --2- 172.21.15.16:0/1610700729 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3ef4130e50 0x7f3ef411f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:08.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.055+0000 7f3ef9930700 1 -- 172.21.15.16:0/1610700729 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3ef406fcf0 msgr2=0x7f3ef411ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:08.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.055+0000 7f3ef9930700 1 --2- 172.21.15.16:0/1610700729 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3ef406fcf0 0x7f3ef411ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:08.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.055+0000 7f3ef9930700 1 -- 172.21.15.16:0/1610700729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eec009a70 con 0x7f3ef4130540 2023-09-30T22:30:08.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.055+0000 7f3efa932700 1 --2- 172.21.15.16:0/1610700729 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3ef4130e50 0x7f3ef411f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:08.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.055+0000 7f3ef9930700 1 --2- 172.21.15.16:0/1610700729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ef4130540 0x7f3ef411f2b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3ef40a35d0 tx=0x7f3ee401d760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:08.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.055+0000 7f3eeaffd700 1 -- 172.21.15.16:0/1610700729 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee401de60 con 0x7f3ef4130540 2023-09-30T22:30:08.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.055+0000 7f3eeaffd700 1 -- 172.21.15.16:0/1610700729 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ee4004a80 con 0x7f3ef4130540 2023-09-30T22:30:08.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.055+0000 7f3efc395700 1 -- 172.21.15.16:0/1610700729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ef000ba70 con 0x7f3ef4130540 2023-09-30T22:30:08.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.055+0000 7f3eeaffd700 1 -- 172.21.15.16:0/1610700729 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4004e30 con 0x7f3ef4130540 2023-09-30T22:30:08.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.055+0000 7f3efc395700 1 -- 172.21.15.16:0/1610700729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ef4122f90 con 0x7f3ef4130540 2023-09-30T22:30:08.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.057+0000 7f3efc395700 1 -- 172.21.15.16:0/1610700729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ef4122440 con 0x7f3ef4130540 2023-09-30T22:30:08.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.057+0000 7f3eeaffd700 1 -- 172.21.15.16:0/1610700729 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f3ee40070d0 con 0x7f3ef4130540 2023-09-30T22:30:08.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.058+0000 7f3eeaffd700 1 -- 172.21.15.16:0/1610700729 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7f3ee4050aa0 con 0x7f3ef4130540 2023-09-30T22:30:08.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.058+0000 7f3efa131700 1 -- 172.21.15.16:0/1610700729 >> v1:172.21.15.16:6816/126811 conn(0x7f3ed802f510 legacy=0x7f3ed80319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:08.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.066+0000 7f3eeaffd700 1 -- 172.21.15.16:0/1610700729 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3ee4007380 con 0x7f3ef4130540 2023-09-30T22:30:08.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.137+0000 7f97fdb20700 1 Processor -- start 2023-09-30T22:30:08.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.137+0000 7f97fdb20700 1 -- start start 2023-09-30T22:30:08.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.138+0000 7f97fdb20700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f97f809d6f0 0x7f97f809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:08.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.138+0000 7f97fdb20700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f97f809e000 0x7f97f81339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:08.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.138+0000 7f97fdb20700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97f809b120 con 0x7f97f809e410 2023-09-30T22:30:08.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.138+0000 7f97fdb20700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97f806b460 con 0x7f97f809e000 2023-09-30T22:30:08.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.138+0000 7f97fdb20700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97f809a7e0 con 0x7f97f809d6f0 2023-09-30T22:30:08.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.138+0000 7f97f6ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f97f809e000 0x7f97f81339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:08.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.138+0000 7f97f6ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f97f809e000 0x7f97f81339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38522/0 (socket says 172.21.15.16:38522) 2023-09-30T22:30:08.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.138+0000 7f97f6ffd700 1 -- 172.21.15.16:0/794262857 learned_addr learned my addr 172.21.15.16:0/794262857 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:08.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.138+0000 7f97f7fff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f97f809e410 0x7f97f8133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47906/0 (socket says 172.21.15.16:47906) 2023-09-30T22:30:08.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.138+0000 7f97f77fe700 1 --2- 172.21.15.16:0/794262857 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f97f809d6f0 0x7f97f809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:08.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.138+0000 7f97f6ffd700 1 -- 172.21.15.16:0/794262857 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f97f809d6f0 msgr2=0x7f97f809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:08.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.138+0000 7f97f6ffd700 1 --2- 172.21.15.16:0/794262857 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f97f809d6f0 0x7f97f809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:08.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.138+0000 7f97f6ffd700 1 -- 172.21.15.16:0/794262857 >> v1:172.21.15.16:6789/0 conn(0x7f97f809e410 legacy=0x7f97f8133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:08.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.138+0000 7f97f6ffd700 1 -- 172.21.15.16:0/794262857 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f809a500 con 0x7f97f809e000 2023-09-30T22:30:08.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.138+0000 7f97f67fc700 1 -- 172.21.15.16:0/794262857 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 469180122 0 0) 0x7f97f809b120 con 0x7f97f809e410 2023-09-30T22:30:08.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.138+0000 7f97f77fe700 1 --2- 172.21.15.16:0/794262857 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f97f809d6f0 0x7f97f809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:30:08.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.139+0000 7f97f6ffd700 1 --2- 172.21.15.16:0/794262857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97f809e000 0x7f97f81339f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f97ec0098f0 tx=0x7f97ec009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=782112ce2630dea9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:08.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.139+0000 7f97f67fc700 1 -- 172.21.15.16:0/794262857 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ec00b8a0 con 0x7f97f809e000 2023-09-30T22:30:08.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.139+0000 7f97f67fc700 1 -- 172.21.15.16:0/794262857 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97ec00ba00 con 0x7f97f809e000 2023-09-30T22:30:08.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.139+0000 7f97f67fc700 1 -- 172.21.15.16:0/794262857 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ec00bd90 con 0x7f97f809e000 2023-09-30T22:30:08.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.140+0000 7f97fdb20700 1 -- 172.21.15.16:0/794262857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97f809e000 msgr2=0x7f97f81339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:08.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.140+0000 7f97fdb20700 1 --2- 172.21.15.16:0/794262857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97f809e000 0x7f97f81339f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f97ec0098f0 tx=0x7f97ec009c00 comp rx=0 tx=0).stop 2023-09-30T22:30:08.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.140+0000 7f97fdb20700 1 -- 172.21.15.16:0/794262857 shutdown_connections 2023-09-30T22:30:08.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.140+0000 7f97fdb20700 1 --2- 172.21.15.16:0/794262857 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f97f809d6f0 0x7f97f809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:08.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.140+0000 7f97fdb20700 1 --2- 172.21.15.16:0/794262857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97f809e000 0x7f97f81339f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:08.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.140+0000 7f97fdb20700 1 -- 172.21.15.16:0/794262857 >> 172.21.15.16:0/794262857 conn(0x7f97f8066f50 msgr2=0x7f97f809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:08.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.140+0000 7f97fdb20700 1 -- 172.21.15.16:0/794262857 shutdown_connections 2023-09-30T22:30:08.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.140+0000 7f97fdb20700 1 -- 172.21.15.16:0/794262857 wait complete. 2023-09-30T22:30:08.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.141+0000 7f97fdb20700 1 Processor -- start 2023-09-30T22:30:08.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.141+0000 7f97fdb20700 1 -- start start 2023-09-30T22:30:08.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.142+0000 7f97fdb20700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97f809d6f0 0x7f97f81a5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:08.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.142+0000 7f97fdb20700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f97f809e000 0x7f97f81a5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:08.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.142+0000 7f97fdb20700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f97f809e410 0x7f97f81c7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:08.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.142+0000 7f97fdb20700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f81a8250 con 0x7f97f809d6f0 2023-09-30T22:30:08.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.142+0000 7f97fdb20700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97f81b90c0 con 0x7f97f809e000 2023-09-30T22:30:08.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.142+0000 7f97fdb20700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f97f81a4c10 con 0x7f97f809e410 2023-09-30T22:30:08.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.142+0000 7f97f7fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f97f809e410 0x7f97f81c7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:08.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.142+0000 7f97f7fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f97f809e410 0x7f97f81c7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:50844/0 (socket says 172.21.15.16:50844) 2023-09-30T22:30:08.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.142+0000 7f97f7fff700 1 -- 172.21.15.16:0/2465662161 learned_addr learned my addr 172.21.15.16:0/2465662161 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:08.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.142+0000 7f97f6ffd700 1 --2- 172.21.15.16:0/2465662161 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f97f809e000 0x7f97f81a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:08.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.142+0000 7f97f77fe700 1 --2- 172.21.15.16:0/2465662161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97f809d6f0 0x7f97f81a5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:08.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.142+0000 7f97f7fff700 1 -- 172.21.15.16:0/2465662161 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f97f809e000 msgr2=0x7f97f81a5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:08.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.142+0000 7f97f7fff700 1 --2- 172.21.15.16:0/2465662161 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f97f809e000 0x7f97f81a5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:08.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.142+0000 7f97f7fff700 1 -- 172.21.15.16:0/2465662161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97f809d6f0 msgr2=0x7f97f81a5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:08.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.142+0000 7f97f7fff700 1 --2- 172.21.15.16:0/2465662161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97f809d6f0 0x7f97f81a5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:08.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.142+0000 7f97f7fff700 1 -- 172.21.15.16:0/2465662161 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97ec009550 con 0x7f97f809e410 2023-09-30T22:30:08.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.142+0000 7f97f77fe700 1 --2- 172.21.15.16:0/2465662161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97f809d6f0 0x7f97f81a5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:30:08.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.143+0000 7f97f7fff700 1 --2- 172.21.15.16:0/2465662161 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f97f809e410 0x7f97f81c7fb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f97f0002230 tx=0x7f97f000a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:08.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.143+0000 7f97dffff700 1 -- 172.21.15.16:0/2465662161 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f000ac20 con 0x7f97f809e410 2023-09-30T22:30:08.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.143+0000 7f97dffff700 1 -- 172.21.15.16:0/2465662161 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97f000ada0 con 0x7f97f809e410 2023-09-30T22:30:08.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.143+0000 7f97fdb20700 1 -- 172.21.15.16:0/2465662161 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97f80a2e80 con 0x7f97f809e410 2023-09-30T22:30:08.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.143+0000 7f97dffff700 1 -- 172.21.15.16:0/2465662161 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f0007310 con 0x7f97f809e410 2023-09-30T22:30:08.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.143+0000 7f97fdb20700 1 -- 172.21.15.16:0/2465662161 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97f81a6340 con 0x7f97f809e410 2023-09-30T22:30:08.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.144+0000 7f97dffff700 1 -- 172.21.15.16:0/2465662161 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f97f0007470 con 0x7f97f809e410 2023-09-30T22:30:08.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.144+0000 7f97f77fe700 1 -- 172.21.15.16:0/2465662161 >> v1:172.21.15.16:6816/126811 conn(0x7f97e402f510 legacy=0x7f97e40319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:08.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.144+0000 7f97dffff700 1 -- 172.21.15.16:0/2465662161 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7f97f0041840 con 0x7f97f809e410 2023-09-30T22:30:08.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.145+0000 7f97fdb20700 1 -- 172.21.15.16:0/2465662161 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97f8074ea0 con 0x7f97f809e410 2023-09-30T22:30:08.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.150+0000 7f97dffff700 1 -- 172.21.15.16:0/2465662161 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f97f0045060 con 0x7f97f809e410 2023-09-30T22:30:08.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.189+0000 7fb43d1a3700 1 Processor -- start 2023-09-30T22:30:08.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.189+0000 7fb43d1a3700 1 -- start start 2023-09-30T22:30:08.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.190+0000 7fb43d1a3700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb43809e930 0x7fb43809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:08.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.190+0000 7fb43d1a3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb43809f240 0x7fb4381317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:08.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.190+0000 7fb43d1a3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb43809f650 0x7fb438131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:08.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.190+0000 7fb43d1a3700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb438070940 con 0x7fb43809e930 2023-09-30T22:30:08.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.190+0000 7fb43d1a3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb43806fd60 con 0x7fb43809f240 2023-09-30T22:30:08.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.190+0000 7fb43d1a3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb43806fa80 con 0x7fb43809f650 2023-09-30T22:30:08.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.190+0000 7fb436d9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb43809e930 0x7fb43809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:08.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.190+0000 7fb436d9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb43809e930 0x7fb43809ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38546/0 (socket says 172.21.15.16:38546) 2023-09-30T22:30:08.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.190+0000 7fb436d9d700 1 -- 172.21.15.16:0/2055334150 learned_addr learned my addr 172.21.15.16:0/2055334150 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:08.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.190+0000 7fb42ffff700 1 --2- 172.21.15.16:0/2055334150 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb43809f240 0x7fb4381317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:08.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.190+0000 7fb43759e700 1 --2- 172.21.15.16:0/2055334150 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb43809f650 0x7fb438131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:08.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.190+0000 7fb42ffff700 1 -- 172.21.15.16:0/2055334150 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb43809f650 msgr2=0x7fb438131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:08.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.190+0000 7fb42ffff700 1 --2- 172.21.15.16:0/2055334150 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb43809f650 0x7fb438131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:08.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.190+0000 7fb42ffff700 1 -- 172.21.15.16:0/2055334150 >> v2:172.21.15.16:3300/0 conn(0x7fb43809e930 msgr2=0x7fb43809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:08.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.190+0000 7fb42ffff700 1 --2- 172.21.15.16:0/2055334150 >> v2:172.21.15.16:3300/0 conn(0x7fb43809e930 0x7fb43809ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:08.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.190+0000 7fb42ffff700 1 -- 172.21.15.16:0/2055334150 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb43809fb60 con 0x7fb43809f240 2023-09-30T22:30:08.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.190+0000 7fb42ffff700 1 --2- 172.21.15.16:0/2055334150 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb43809f240 0x7fb4381317f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb420009840 tx=0x7fb420009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0ceb477e21d5bd5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:08.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.191+0000 7fb43659c700 1 -- 172.21.15.16:0/2055334150 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb420011830 con 0x7fb43809f240 2023-09-30T22:30:08.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.191+0000 7fb43659c700 1 -- 172.21.15.16:0/2055334150 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb420011990 con 0x7fb43809f240 2023-09-30T22:30:08.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.191+0000 7fb43659c700 1 -- 172.21.15.16:0/2055334150 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb420011d20 con 0x7fb43809f240 2023-09-30T22:30:08.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.192+0000 7fb43d1a3700 1 -- 172.21.15.16:0/2055334150 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb43809f240 msgr2=0x7fb4381317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:08.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.192+0000 7fb43d1a3700 1 --2- 172.21.15.16:0/2055334150 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb43809f240 0x7fb4381317f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb420009840 tx=0x7fb420009b50 comp rx=0 tx=0).stop 2023-09-30T22:30:08.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.192+0000 7fb43d1a3700 1 -- 172.21.15.16:0/2055334150 shutdown_connections 2023-09-30T22:30:08.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.192+0000 7fb43d1a3700 1 --2- 172.21.15.16:0/2055334150 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb43809f650 0x7fb438131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:08.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.192+0000 7fb43d1a3700 1 --2- 172.21.15.16:0/2055334150 >> v2:172.21.15.16:3300/0 conn(0x7fb43809e930 0x7fb43809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:08.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.192+0000 7fb43d1a3700 1 --2- 172.21.15.16:0/2055334150 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb43809f240 0x7fb4381317f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:08.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.192+0000 7fb43d1a3700 1 -- 172.21.15.16:0/2055334150 >> 172.21.15.16:0/2055334150 conn(0x7fb4380a4210 msgr2=0x7fb4380a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:08.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.192+0000 7fb43d1a3700 1 -- 172.21.15.16:0/2055334150 shutdown_connections 2023-09-30T22:30:08.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.192+0000 7fb43d1a3700 1 -- 172.21.15.16:0/2055334150 wait complete. 2023-09-30T22:30:08.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.194+0000 7fb43d1a3700 1 Processor -- start 2023-09-30T22:30:08.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.194+0000 7fb43d1a3700 1 -- start start 2023-09-30T22:30:08.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.194+0000 7fb43d1a3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb43809e930 0x7fb4381a3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:08.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.194+0000 7fb43d1a3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb43809f240 0x7fb4381a4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:08.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.194+0000 7fb43d1a3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb43809f650 0x7fb4381a4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:08.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.194+0000 7fb43d1a3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4381a5f30 con 0x7fb43809e930 2023-09-30T22:30:08.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.194+0000 7fb43d1a3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4381a6ff0 con 0x7fb43809f240 2023-09-30T22:30:08.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.195+0000 7fb43d1a3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb438071d20 con 0x7fb43809f650 2023-09-30T22:30:08.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.195+0000 7fb42ffff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb43809f240 0x7fb4381a4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:08.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.195+0000 7fb42ffff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb43809f240 0x7fb4381a4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41282/0 (socket says 172.21.15.16:41282) 2023-09-30T22:30:08.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.195+0000 7fb42ffff700 1 -- 172.21.15.16:0/3989564204 learned_addr learned my addr 172.21.15.16:0/3989564204 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:08.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.195+0000 7fb43759e700 1 --2- 172.21.15.16:0/3989564204 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb43809f650 0x7fb4381a4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:08.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.195+0000 7fb436d9d700 1 --2- 172.21.15.16:0/3989564204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb43809e930 0x7fb4381a3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:08.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.195+0000 7fb43759e700 1 -- 172.21.15.16:0/3989564204 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb43809f240 msgr2=0x7fb4381a4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:08.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.195+0000 7fb43759e700 1 --2- 172.21.15.16:0/3989564204 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb43809f240 0x7fb4381a4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:08.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.195+0000 7fb43759e700 1 -- 172.21.15.16:0/3989564204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb43809e930 msgr2=0x7fb4381a3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:08.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.195+0000 7fb43759e700 1 --2- 172.21.15.16:0/3989564204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb43809e930 0x7fb4381a3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:08.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.195+0000 7fb43759e700 1 -- 172.21.15.16:0/3989564204 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4200094a0 con 0x7fb43809f650 2023-09-30T22:30:08.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.195+0000 7fb436d9d700 1 --2- 172.21.15.16:0/3989564204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb43809e930 0x7fb4381a3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:08.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.195+0000 7fb42ffff700 1 --2- 172.21.15.16:0/3989564204 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb43809f240 0x7fb4381a4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:08.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.195+0000 7fb43759e700 1 --2- 172.21.15.16:0/3989564204 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb43809f650 0x7fb4381a4850 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb43000efc0 tx=0x7fb43000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:08.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.196+0000 7fb42f7fe700 1 -- 172.21.15.16:0/3989564204 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb43000cc90 con 0x7fb43809f650 2023-09-30T22:30:08.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.196+0000 7fb42f7fe700 1 -- 172.21.15.16:0/3989564204 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb43000f040 con 0x7fb43809f650 2023-09-30T22:30:08.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.196+0000 7fb42f7fe700 1 -- 172.21.15.16:0/3989564204 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4300146c0 con 0x7fb43809f650 2023-09-30T22:30:08.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.197+0000 7fb43d1a3700 1 -- 172.21.15.16:0/3989564204 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4380a0190 con 0x7fb43809f650 2023-09-30T22:30:08.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.197+0000 7fb43d1a3700 1 -- 172.21.15.16:0/3989564204 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4381a5950 con 0x7fb43809f650 2023-09-30T22:30:08.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.198+0000 7fb42f7fe700 1 -- 172.21.15.16:0/3989564204 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fb430014820 con 0x7fb43809f650 2023-09-30T22:30:08.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.198+0000 7fb42f7fe700 1 -- 172.21.15.16:0/3989564204 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7fb4300407e0 con 0x7fb43809f650 2023-09-30T22:30:08.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.198+0000 7fb436d9d700 1 -- 172.21.15.16:0/3989564204 >> v1:172.21.15.16:6816/126811 conn(0x7fb41802f510 legacy=0x7fb4180319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:08.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.198+0000 7fb43d1a3700 1 -- 172.21.15.16:0/3989564204 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb438074ea0 con 0x7fb43809f650 2023-09-30T22:30:08.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.198+0000 7f37b5a56700 1 Processor -- start 2023-09-30T22:30:08.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.198+0000 7f37b5a56700 1 -- start start 2023-09-30T22:30:08.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.199+0000 7f37b5a56700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f37b00a1510 0x7f37b0138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:08.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.199+0000 7f37b5a56700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f37b0139250 0x7f37b013b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:08.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.199+0000 7f37b5a56700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37b009edd0 con 0x7f37b00a0a30 2023-09-30T22:30:08.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.199+0000 7f37b5a56700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37b00a18e0 con 0x7f37b0139250 2023-09-30T22:30:08.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.199+0000 7f37b5a56700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f37b009e1f0 con 0x7f37b00a1510 2023-09-30T22:30:08.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.199+0000 7f37aeffd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f37b00a0a30 0x7f37b00a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47920/0 (socket says 172.21.15.16:47920) 2023-09-30T22:30:08.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.199+0000 7f37aeffd700 1 -- 172.21.15.16:0/2784832097 learned_addr learned my addr 172.21.15.16:0/2784832097 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:30:08.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.200+0000 7f37ae7fc700 1 --2- 172.21.15.16:0/2784832097 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f37b00a1510 0x7f37b0138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:08.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.200+0000 7f37ae7fc700 1 -- 172.21.15.16:0/2784832097 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f37b0139250 msgr2=0x7f37b013b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:08.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.200+0000 7f37af7fe700 1 --2- 172.21.15.16:0/2784832097 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f37b0139250 0x7f37b013b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:08.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.200+0000 7f37adffb700 1 -- 172.21.15.16:0/2784832097 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2475878077 0 0) 0x7f37b009edd0 con 0x7f37b00a0a30 2023-09-30T22:30:08.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.200+0000 7f37ae7fc700 1 --2- 172.21.15.16:0/2784832097 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f37b0139250 0x7f37b013b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:08.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.200+0000 7f37ae7fc700 1 -- 172.21.15.16:0/2784832097 >> v1:172.21.15.16:6789/0 conn(0x7f37b00a0a30 legacy=0x7f37b00a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:08.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.200+0000 7f37ae7fc700 1 -- 172.21.15.16:0/2784832097 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b009dc50 con 0x7f37b00a1510 2023-09-30T22:30:08.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.200+0000 7f37af7fe700 1 --2- 172.21.15.16:0/2784832097 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f37b0139250 0x7f37b013b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:30:08.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.200+0000 7f37ae7fc700 1 --2- 172.21.15.16:0/2784832097 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f37b00a1510 0x7f37b0138d10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f37a4009870 tx=0x7f37a4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81e5006abb6aef11 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:08.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.203+0000 7f37adffb700 1 -- 172.21.15.16:0/2784832097 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a4011830 con 0x7f37b00a1510 2023-09-30T22:30:08.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.203+0000 7f37adffb700 1 -- 172.21.15.16:0/2784832097 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37a4011990 con 0x7f37b00a1510 2023-09-30T22:30:08.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.203+0000 7f37adffb700 1 -- 172.21.15.16:0/2784832097 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a4011d20 con 0x7f37b00a1510 2023-09-30T22:30:08.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.204+0000 7f37b5a56700 1 -- 172.21.15.16:0/2784832097 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f37b00a1510 msgr2=0x7f37b0138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:08.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.204+0000 7f37b5a56700 1 --2- 172.21.15.16:0/2784832097 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f37b00a1510 0x7f37b0138d10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f37a4009870 tx=0x7f37a4009b80 comp rx=0 tx=0).stop 2023-09-30T22:30:08.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.204+0000 7fb42f7fe700 1 -- 172.21.15.16:0/3989564204 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb430009e60 con 0x7fb43809f650 2023-09-30T22:30:08.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.205+0000 7f37b5a56700 1 -- 172.21.15.16:0/2784832097 shutdown_connections 2023-09-30T22:30:08.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.205+0000 7f37b5a56700 1 --2- 172.21.15.16:0/2784832097 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f37b0139250 0x7f37b013b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:08.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.205+0000 7f37b5a56700 1 --2- 172.21.15.16:0/2784832097 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f37b00a1510 0x7f37b0138d10 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:08.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.205+0000 7f37b5a56700 1 -- 172.21.15.16:0/2784832097 >> 172.21.15.16:0/2784832097 conn(0x7f37b00a6550 msgr2=0x7f37b00a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:08.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.205+0000 7f37b5a56700 1 -- 172.21.15.16:0/2784832097 shutdown_connections 2023-09-30T22:30:08.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.205+0000 7f37b5a56700 1 -- 172.21.15.16:0/2784832097 wait complete. 2023-09-30T22:30:08.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.206+0000 7f37b5a56700 1 Processor -- start 2023-09-30T22:30:08.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.206+0000 7f37b5a56700 1 -- start start 2023-09-30T22:30:08.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.206+0000 7f37b5a56700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f37b00a0a30 0x7f37b01227a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:08.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.206+0000 7f37b5a56700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f37b00a1510 0x7f37b0122ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:08.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.206+0000 7f37b5a56700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37b0139250 0x7f37b0123220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:08.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.206+0000 7f37b5a56700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b0124420 con 0x7f37b0139250 2023-09-30T22:30:08.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.206+0000 7f37b5a56700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37b01385d0 con 0x7f37b00a0a30 2023-09-30T22:30:08.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.206+0000 7f37b5a56700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f37b0125c80 con 0x7f37b00a1510 2023-09-30T22:30:08.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.207+0000 7f37aeffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f37b00a0a30 0x7f37b01227a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:08.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.207+0000 7f37aeffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f37b00a0a30 0x7f37b01227a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41300/0 (socket says 172.21.15.16:41300) 2023-09-30T22:30:08.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.207+0000 7f37aeffd700 1 -- 172.21.15.16:0/2428929351 learned_addr learned my addr 172.21.15.16:0/2428929351 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:08.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.207+0000 7f37af7fe700 1 --2- 172.21.15.16:0/2428929351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37b0139250 0x7f37b0123220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:08.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.207+0000 7f37ae7fc700 1 --2- 172.21.15.16:0/2428929351 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f37b00a1510 0x7f37b0122ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:08.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.207+0000 7f37aeffd700 1 -- 172.21.15.16:0/2428929351 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f37b00a1510 msgr2=0x7f37b0122ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:08.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.207+0000 7f37aeffd700 1 --2- 172.21.15.16:0/2428929351 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f37b00a1510 0x7f37b0122ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:08.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.207+0000 7f37aeffd700 1 -- 172.21.15.16:0/2428929351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37b0139250 msgr2=0x7f37b0123220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:08.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.207+0000 7f37aeffd700 1 --2- 172.21.15.16:0/2428929351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37b0139250 0x7f37b0123220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:08.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.207+0000 7f37aeffd700 1 -- 172.21.15.16:0/2428929351 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a40094a0 con 0x7f37b00a0a30 2023-09-30T22:30:08.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.207+0000 7f37aeffd700 1 --2- 172.21.15.16:0/2428929351 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f37b00a0a30 0x7f37b01227a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f37a00011e0 tx=0x7f37a000c9b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:08.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.208+0000 7f3797fff700 1 -- 172.21.15.16:0/2428929351 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a0015d70 con 0x7f37b00a0a30 2023-09-30T22:30:08.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.208+0000 7f3797fff700 1 -- 172.21.15.16:0/2428929351 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37a0019070 con 0x7f37b00a0a30 2023-09-30T22:30:08.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.208+0000 7f37b5a56700 1 -- 172.21.15.16:0/2428929351 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37b01262c0 con 0x7f37b00a0a30 2023-09-30T22:30:08.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.208+0000 7f3797fff700 1 -- 172.21.15.16:0/2428929351 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a0004ab0 con 0x7f37b00a0a30 2023-09-30T22:30:08.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.208+0000 7f37b5a56700 1 -- 172.21.15.16:0/2428929351 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37b0126720 con 0x7f37b00a0a30 2023-09-30T22:30:08.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.209+0000 7f3797fff700 1 -- 172.21.15.16:0/2428929351 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f37a0004c10 con 0x7f37b00a0a30 2023-09-30T22:30:08.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.209+0000 7f3797fff700 1 -- 172.21.15.16:0/2428929351 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7f37a00475a0 con 0x7f37b00a0a30 2023-09-30T22:30:08.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.209+0000 7f37ae7fc700 1 -- 172.21.15.16:0/2428929351 >> v1:172.21.15.16:6816/126811 conn(0x7f379c02f510 legacy=0x7f379c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:08.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.209+0000 7f37b5a56700 1 -- 172.21.15.16:0/2428929351 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37b00a1d00 con 0x7f37b00a0a30 2023-09-30T22:30:08.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.219+0000 7f3797fff700 1 -- 172.21.15.16:0/2428929351 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f37a001a820 con 0x7f37b00a0a30 2023-09-30T22:30:08.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.259+0000 7f3efa131700 1 -- 172.21.15.16:0/1610700729 >> v1:172.21.15.16:6816/126811 conn(0x7f3ed802f510 legacy=0x7f3ed80319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:08.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.284+0000 7f3efc395700 1 -- 172.21.15.16:0/1610700729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3ef4074ea0 con 0x7f3ef4130540 2023-09-30T22:30:08.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.284+0000 7f3eeaffd700 1 -- 172.21.15.16:0/1610700729 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3ee402c340 con 0x7f3ef4130540 2023-09-30T22:30:08.721 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:30:08.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.286+0000 7f3efc395700 1 -- 172.21.15.16:0/1610700729 >> v1:172.21.15.16:6816/126811 conn(0x7f3ed802f510 legacy=0x7f3ed80319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:30:08.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.286+0000 7f3efc395700 1 -- 172.21.15.16:0/1610700729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ef4130540 msgr2=0x7f3ef411f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:08.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.286+0000 7f3efc395700 1 --2- 172.21.15.16:0/1610700729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ef4130540 0x7f3ef411f2b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3ef40a35d0 tx=0x7f3ee401d760 comp rx=0 tx=0).stop 2023-09-30T22:30:08.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.286+0000 7f3efc395700 1 -- 172.21.15.16:0/1610700729 shutdown_connections 2023-09-30T22:30:08.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.286+0000 7f3efc395700 1 --2- 172.21.15.16:0/1610700729 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3ef406fcf0 0x7f3ef411ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:08.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.286+0000 7f3efc395700 1 --2- 172.21.15.16:0/1610700729 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3ef4130e50 0x7f3ef411f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:08.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.286+0000 7f3efc395700 1 --2- 172.21.15.16:0/1610700729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ef4130540 0x7f3ef411f2b0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:08.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.286+0000 7f3efc395700 1 -- 172.21.15.16:0/1610700729 >> 172.21.15.16:0/1610700729 conn(0x7f3ef40a2aa0 msgr2=0x7f3ef412e240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:08.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.286+0000 7f3efc395700 1 -- 172.21.15.16:0/1610700729 shutdown_connections 2023-09-30T22:30:08.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.287+0000 7f3efc395700 1 -- 172.21.15.16:0/1610700729 wait complete. 2023-09-30T22:30:08.756 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:30:08.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.345+0000 7f97f77fe700 1 -- 172.21.15.16:0/2465662161 >> v1:172.21.15.16:6816/126811 conn(0x7f97e402f510 legacy=0x7f97e40319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:08.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.376+0000 7f97fdb20700 1 -- 172.21.15.16:0/2465662161 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f97f806c360 con 0x7f97f809e410 2023-09-30T22:30:08.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.377+0000 7f97dffff700 1 -- 172.21.15.16:0/2465662161 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f97f001b360 con 0x7f97f809e410 2023-09-30T22:30:08.757 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:30:08.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.379+0000 7f97fdb20700 1 -- 172.21.15.16:0/2465662161 >> v1:172.21.15.16:6816/126811 conn(0x7f97e402f510 legacy=0x7f97e40319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:30:08.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.379+0000 7f97fdb20700 1 -- 172.21.15.16:0/2465662161 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f97f809e410 msgr2=0x7f97f81c7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:08.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.379+0000 7f97fdb20700 1 --2- 172.21.15.16:0/2465662161 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f97f809e410 0x7f97f81c7fb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f97f0002230 tx=0x7f97f000a060 comp rx=0 tx=0).stop 2023-09-30T22:30:08.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.379+0000 7f97fdb20700 1 -- 172.21.15.16:0/2465662161 shutdown_connections 2023-09-30T22:30:08.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.379+0000 7f97fdb20700 1 --2- 172.21.15.16:0/2465662161 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f97f809e410 0x7f97f81c7fb0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:08.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.379+0000 7f97fdb20700 1 --2- 172.21.15.16:0/2465662161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97f809d6f0 0x7f97f81a5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:08.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.379+0000 7f97fdb20700 1 --2- 172.21.15.16:0/2465662161 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f97f809e000 0x7f97f81a5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:08.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.379+0000 7f97fdb20700 1 -- 172.21.15.16:0/2465662161 >> 172.21.15.16:0/2465662161 conn(0x7f97f8066f50 msgr2=0x7f97f81bbb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:08.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.379+0000 7f97fdb20700 1 -- 172.21.15.16:0/2465662161 shutdown_connections 2023-09-30T22:30:08.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.379+0000 7f97fdb20700 1 -- 172.21.15.16:0/2465662161 wait complete. 2023-09-30T22:30:08.760 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:30:08.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.399+0000 7fb436d9d700 1 -- 172.21.15.16:0/3989564204 >> v1:172.21.15.16:6816/126811 conn(0x7fb41802f510 legacy=0x7fb4180319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:08.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.410+0000 7f37ae7fc700 1 -- 172.21.15.16:0/2428929351 >> v1:172.21.15.16:6816/126811 conn(0x7f379c02f510 legacy=0x7f379c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:08.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.413+0000 7fb43d1a3700 1 -- 172.21.15.16:0/3989564204 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb4381a6ba0 con 0x7fb43809f650 2023-09-30T22:30:08.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.414+0000 7fb42f7fe700 1 -- 172.21.15.16:0/3989564204 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb43002c090 con 0x7fb43809f650 2023-09-30T22:30:08.761 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:30:08.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.416+0000 7fb43d1a3700 1 -- 172.21.15.16:0/3989564204 >> v1:172.21.15.16:6816/126811 conn(0x7fb41802f510 legacy=0x7fb4180319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:30:08.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.416+0000 7fb43d1a3700 1 -- 172.21.15.16:0/3989564204 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb43809f650 msgr2=0x7fb4381a4850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:08.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.416+0000 7fb43d1a3700 1 --2- 172.21.15.16:0/3989564204 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb43809f650 0x7fb4381a4850 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb43000efc0 tx=0x7fb43000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:30:08.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.416+0000 7fb43d1a3700 1 -- 172.21.15.16:0/3989564204 shutdown_connections 2023-09-30T22:30:08.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.416+0000 7fb43d1a3700 1 --2- 172.21.15.16:0/3989564204 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb43809f650 0x7fb4381a4850 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:08.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.416+0000 7fb43d1a3700 1 --2- 172.21.15.16:0/3989564204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb43809e930 0x7fb4381a3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:08.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.416+0000 7fb43d1a3700 1 --2- 172.21.15.16:0/3989564204 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb43809f240 0x7fb4381a4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:08.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.416+0000 7fb43d1a3700 1 -- 172.21.15.16:0/3989564204 >> 172.21.15.16:0/3989564204 conn(0x7fb4380a4210 msgr2=0x7fb4381b8d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:08.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.416+0000 7fb43d1a3700 1 -- 172.21.15.16:0/3989564204 shutdown_connections 2023-09-30T22:30:08.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.416+0000 7fb43d1a3700 1 -- 172.21.15.16:0/3989564204 wait complete. 2023-09-30T22:30:08.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.428+0000 7f37b5a56700 1 -- 172.21.15.16:0/2428929351 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f37b0074ea0 con 0x7f37b00a0a30 2023-09-30T22:30:08.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.428+0000 7f3797fff700 1 -- 172.21.15.16:0/2428929351 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f37a001d380 con 0x7f37b00a0a30 2023-09-30T22:30:08.764 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:30:08.764 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:30:08.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.430+0000 7f37b5a56700 1 -- 172.21.15.16:0/2428929351 >> v1:172.21.15.16:6816/126811 conn(0x7f379c02f510 legacy=0x7f379c0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:30:08.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.430+0000 7f37b5a56700 1 -- 172.21.15.16:0/2428929351 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f37b00a0a30 msgr2=0x7f37b01227a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:08.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.430+0000 7f37b5a56700 1 --2- 172.21.15.16:0/2428929351 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f37b00a0a30 0x7f37b01227a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f37a00011e0 tx=0x7f37a000c9b0 comp rx=0 tx=0).stop 2023-09-30T22:30:08.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.430+0000 7f37b5a56700 1 -- 172.21.15.16:0/2428929351 shutdown_connections 2023-09-30T22:30:08.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.430+0000 7f37b5a56700 1 --2- 172.21.15.16:0/2428929351 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f37b00a0a30 0x7f37b01227a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:08.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.430+0000 7f37b5a56700 1 --2- 172.21.15.16:0/2428929351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37b0139250 0x7f37b0123220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:08.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.430+0000 7f37b5a56700 1 --2- 172.21.15.16:0/2428929351 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f37b00a1510 0x7f37b0122ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:08.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.430+0000 7f37b5a56700 1 -- 172.21.15.16:0/2428929351 >> 172.21.15.16:0/2428929351 conn(0x7f37b00a6550 msgr2=0x7f37b009aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:08.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.431+0000 7f37b5a56700 1 -- 172.21.15.16:0/2428929351 shutdown_connections 2023-09-30T22:30:08.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:08.431+0000 7f37b5a56700 1 -- 172.21.15.16:0/2428929351 wait complete. 2023-09-30T22:30:08.768 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:30:09.757 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:09.760 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:09.767 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:09.769 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:09.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.913+0000 7f10b8cdb700 1 Processor -- start 2023-09-30T22:30:09.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.913+0000 7f10b8cdb700 1 -- start start 2023-09-30T22:30:09.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.914+0000 7f10b8cdb700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f10b4130540 0x7f10b4130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:09.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.914+0000 7f10b8cdb700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f10b4130e50 0x7f10b406f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:09.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.914+0000 7f10b8cdb700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f10b406fb20 0x7f10b406ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:09.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.914+0000 7f10b8cdb700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10b409a390 con 0x7f10b406fb20 2023-09-30T22:30:09.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.914+0000 7f10b8cdb700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10b40716a0 con 0x7f10b4130e50 2023-09-30T22:30:09.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.914+0000 7f10b8cdb700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f10b4071290 con 0x7f10b4130540 2023-09-30T22:30:09.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.914+0000 7f10b259c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f10b4130540 0x7f10b4130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:09.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.914+0000 7f10b259c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f10b4130540 0x7f10b4130910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:50894/0 (socket says 172.21.15.16:50894) 2023-09-30T22:30:09.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.914+0000 7f10b259c700 1 -- 172.21.15.16:0/4099636858 learned_addr learned my addr 172.21.15.16:0/4099636858 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:09.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.914+0000 7f10b2d9d700 1 --2- 172.21.15.16:0/4099636858 >> v2:172.21.15.16:3300/0 conn(0x7f10b406fb20 0x7f10b406ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:09.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.915+0000 7f10b1d9b700 1 --2- 172.21.15.16:0/4099636858 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f10b4130e50 0x7f10b406f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:09.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.915+0000 7f10b2d9d700 1 -- 172.21.15.16:0/4099636858 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f10b4130540 msgr2=0x7f10b4130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:09.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.915+0000 7f10b2d9d700 1 --2- 172.21.15.16:0/4099636858 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f10b4130540 0x7f10b4130910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:09.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.915+0000 7f10b2d9d700 1 -- 172.21.15.16:0/4099636858 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f10b4130e50 msgr2=0x7f10b406f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:09.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.915+0000 7f10b2d9d700 1 --2- 172.21.15.16:0/4099636858 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f10b4130e50 0x7f10b406f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:09.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.915+0000 7f10b2d9d700 1 -- 172.21.15.16:0/4099636858 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10b4130340 con 0x7f10b406fb20 2023-09-30T22:30:09.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.915+0000 7f10b1d9b700 1 --2- 172.21.15.16:0/4099636858 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f10b4130e50 0x7f10b406f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:30:09.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.915+0000 7f10b2d9d700 1 --2- 172.21.15.16:0/4099636858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10b406fb20 0x7f10b406ffc0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f10ac009dc0 tx=0x7f10ac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7afd76b3413633f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:09.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.915+0000 7f10b159a700 1 -- 172.21.15.16:0/4099636858 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ac00e070 con 0x7f10b406fb20 2023-09-30T22:30:09.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.915+0000 7f10b159a700 1 -- 172.21.15.16:0/4099636858 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10ac00bae0 con 0x7f10b406fb20 2023-09-30T22:30:09.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.915+0000 7f10b159a700 1 -- 172.21.15.16:0/4099636858 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ac00be90 con 0x7f10b406fb20 2023-09-30T22:30:09.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.916+0000 7f10b8cdb700 1 -- 172.21.15.16:0/4099636858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10b406fb20 msgr2=0x7f10b406ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:09.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.916+0000 7f10b8cdb700 1 --2- 172.21.15.16:0/4099636858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10b406fb20 0x7f10b406ffc0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f10ac009dc0 tx=0x7f10ac00b330 comp rx=0 tx=0).stop 2023-09-30T22:30:09.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.916+0000 7f10b8cdb700 1 -- 172.21.15.16:0/4099636858 shutdown_connections 2023-09-30T22:30:09.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.916+0000 7f10b8cdb700 1 --2- 172.21.15.16:0/4099636858 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10b406fb20 0x7f10b406ffc0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:09.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.916+0000 7f10b8cdb700 1 --2- 172.21.15.16:0/4099636858 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f10b4130540 0x7f10b4130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:09.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.916+0000 7f10b8cdb700 1 --2- 172.21.15.16:0/4099636858 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f10b4130e50 0x7f10b406f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:09.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.916+0000 7f10b8cdb700 1 -- 172.21.15.16:0/4099636858 >> 172.21.15.16:0/4099636858 conn(0x7f10b40a2aa0 msgr2=0x7f10b40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:09.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.916+0000 7f10b8cdb700 1 -- 172.21.15.16:0/4099636858 shutdown_connections 2023-09-30T22:30:09.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.916+0000 7f10b8cdb700 1 -- 172.21.15.16:0/4099636858 wait complete. 2023-09-30T22:30:09.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.917+0000 7f10b8cdb700 1 Processor -- start 2023-09-30T22:30:09.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.917+0000 7f10b8cdb700 1 -- start start 2023-09-30T22:30:09.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.918+0000 7f10b8cdb700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f10b406fb20 0x7f10b4118cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:09.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.918+0000 7f10b8cdb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10b4130540 0x7f10b4119210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:09.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.918+0000 7f10b8cdb700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f10b4130e50 0x7f10b4119750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:09.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.918+0000 7f10b8cdb700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b411c8f0 con 0x7f10b4130540 2023-09-30T22:30:09.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.918+0000 7f10b8cdb700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10b411bf50 con 0x7f10b406fb20 2023-09-30T22:30:09.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.918+0000 7f10b8cdb700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f10b411ae40 con 0x7f10b4130e50 2023-09-30T22:30:09.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.918+0000 7f10b2d9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f10b4130e50 0x7f10b4119750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:09.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.918+0000 7f10b2d9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f10b4130e50 0x7f10b4119750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:50904/0 (socket says 172.21.15.16:50904) 2023-09-30T22:30:09.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.918+0000 7f10b2d9d700 1 -- 172.21.15.16:0/4089796319 learned_addr learned my addr 172.21.15.16:0/4089796319 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:09.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.918+0000 7f10b1d9b700 1 --2- 172.21.15.16:0/4089796319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10b4130540 0x7f10b4119210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:09.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.918+0000 7f10b259c700 1 --2- 172.21.15.16:0/4089796319 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f10b406fb20 0x7f10b4118cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:09.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.918+0000 7f10b2d9d700 1 -- 172.21.15.16:0/4089796319 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f10b406fb20 msgr2=0x7f10b4118cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:09.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.918+0000 7f10b2d9d700 1 --2- 172.21.15.16:0/4089796319 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f10b406fb20 0x7f10b4118cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:09.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.918+0000 7f10b2d9d700 1 -- 172.21.15.16:0/4089796319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10b4130540 msgr2=0x7f10b4119210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:09.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.918+0000 7f10b2d9d700 1 --2- 172.21.15.16:0/4089796319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10b4130540 0x7f10b4119210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:09.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.919+0000 7f10b2d9d700 1 -- 172.21.15.16:0/4089796319 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10ac009a70 con 0x7f10b4130e50 2023-09-30T22:30:09.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.919+0000 7f10b2d9d700 1 --2- 172.21.15.16:0/4089796319 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f10b4130e50 0x7f10b4119750 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f10ac014040 tx=0x7f10ac004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:09.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.919+0000 7f10a2ffd700 1 -- 172.21.15.16:0/4089796319 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ac004680 con 0x7f10b4130e50 2023-09-30T22:30:09.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.919+0000 7f10a2ffd700 1 -- 172.21.15.16:0/4089796319 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10ac028070 con 0x7f10b4130e50 2023-09-30T22:30:09.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.919+0000 7f10b8cdb700 1 -- 172.21.15.16:0/4089796319 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10b409cc70 con 0x7f10b4130e50 2023-09-30T22:30:09.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.919+0000 7f10a2ffd700 1 -- 172.21.15.16:0/4089796319 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ac02f480 con 0x7f10b4130e50 2023-09-30T22:30:09.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.919+0000 7f10b8cdb700 1 -- 172.21.15.16:0/4089796319 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10b411a100 con 0x7f10b4130e50 2023-09-30T22:30:09.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.920+0000 7f10a2ffd700 1 -- 172.21.15.16:0/4089796319 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f10ac012430 con 0x7f10b4130e50 2023-09-30T22:30:09.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.920+0000 7f10a2ffd700 1 -- 172.21.15.16:0/4089796319 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7f10ac02a070 con 0x7f10b4130e50 2023-09-30T22:30:09.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.920+0000 7f10b259c700 1 -- 172.21.15.16:0/4089796319 >> v1:172.21.15.16:6816/126811 conn(0x7f109402f510 legacy=0x7f10940319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:09.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.920+0000 7f10b8cdb700 1 -- 172.21.15.16:0/4089796319 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10b411b360 con 0x7f10b4130e50 2023-09-30T22:30:09.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.926+0000 7f10a2ffd700 1 -- 172.21.15.16:0/4089796319 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f10ac02e9d0 con 0x7f10b4130e50 2023-09-30T22:30:09.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.951+0000 7f6e51a97700 1 Processor -- start 2023-09-30T22:30:09.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.951+0000 7f6e51a97700 1 -- start start 2023-09-30T22:30:09.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.952+0000 7f6e51a97700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e4c09e1d0 0x7f6e4c135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:09.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.952+0000 7f6e51a97700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6e4c135e50 0x7f6e4c138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:09.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.952+0000 7f6e51a97700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e4c09f130 con 0x7f6e4c09d6f0 2023-09-30T22:30:09.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.952+0000 7f6e51a97700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e4c09f520 con 0x7f6e4c135e50 2023-09-30T22:30:09.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.952+0000 7f6e51a97700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c09e930 con 0x7f6e4c09e1d0 2023-09-30T22:30:09.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.952+0000 7f6e4affd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e4c09e1d0 0x7f6e4c135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:09.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.952+0000 7f6e4affd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e4c09e1d0 0x7f6e4c135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:50914/0 (socket says 172.21.15.16:50914) 2023-09-30T22:30:09.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.952+0000 7f6e4affd700 1 -- 172.21.15.16:0/53479599 learned_addr learned my addr 172.21.15.16:0/53479599 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:09.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.952+0000 7f6e4bfff700 1 --2- 172.21.15.16:0/53479599 >> v2:172.21.15.16:3300/0 conn(0x7f6e4c135e50 0x7f6e4c138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:09.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.952+0000 7f6e4affd700 1 -- 172.21.15.16:0/53479599 >> v2:172.21.15.16:3300/0 conn(0x7f6e4c135e50 msgr2=0x7f6e4c138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:09.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.952+0000 7f6e4affd700 1 --2- 172.21.15.16:0/53479599 >> v2:172.21.15.16:3300/0 conn(0x7f6e4c135e50 0x7f6e4c138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:09.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.952+0000 7f6e4affd700 1 -- 172.21.15.16:0/53479599 >> v1:172.21.15.16:6789/0 conn(0x7f6e4c09d6f0 legacy=0x7f6e4c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:09.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.952+0000 7f6e4affd700 1 -- 172.21.15.16:0/53479599 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e4c09b2d0 con 0x7f6e4c09e1d0 2023-09-30T22:30:09.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.953+0000 7f6e4affd700 1 --2- 172.21.15.16:0/53479599 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e4c09e1d0 0x7f6e4c135910 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6e4000b9d0 tx=0x7f6e4000bce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74a856795ceaeb87 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:09.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.953+0000 7f6e4a7fc700 1 -- 172.21.15.16:0/53479599 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4000cc20 con 0x7f6e4c09e1d0 2023-09-30T22:30:09.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.953+0000 7f6e4a7fc700 1 -- 172.21.15.16:0/53479599 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e4000cd80 con 0x7f6e4c09e1d0 2023-09-30T22:30:09.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.953+0000 7f6e4a7fc700 1 -- 172.21.15.16:0/53479599 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4000f680 con 0x7f6e4c09e1d0 2023-09-30T22:30:09.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.953+0000 7f6e51a97700 1 -- 172.21.15.16:0/53479599 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e4c09e1d0 msgr2=0x7f6e4c135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:09.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.954+0000 7f6e51a97700 1 --2- 172.21.15.16:0/53479599 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e4c09e1d0 0x7f6e4c135910 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6e4000b9d0 tx=0x7f6e4000bce0 comp rx=0 tx=0).stop 2023-09-30T22:30:09.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.954+0000 7f6e51a97700 1 -- 172.21.15.16:0/53479599 shutdown_connections 2023-09-30T22:30:09.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.954+0000 7f6e51a97700 1 --2- 172.21.15.16:0/53479599 >> v2:172.21.15.16:3300/0 conn(0x7f6e4c135e50 0x7f6e4c138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:09.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.954+0000 7f6e51a97700 1 --2- 172.21.15.16:0/53479599 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e4c09e1d0 0x7f6e4c135910 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:09.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.954+0000 7f6e51a97700 1 -- 172.21.15.16:0/53479599 >> 172.21.15.16:0/53479599 conn(0x7f6e4c066f50 msgr2=0x7f6e4c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:09.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.954+0000 7f6e51a97700 1 -- 172.21.15.16:0/53479599 shutdown_connections 2023-09-30T22:30:09.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.954+0000 7f6e51a97700 1 -- 172.21.15.16:0/53479599 wait complete. 2023-09-30T22:30:09.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.955+0000 7f6e51a97700 1 Processor -- start 2023-09-30T22:30:09.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.955+0000 7f6e51a97700 1 -- start start 2023-09-30T22:30:09.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.955+0000 7f6e51a97700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6e4c09d6f0 0x7f6e4c11a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:09.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.955+0000 7f6e51a97700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e4c09e1d0 0x7f6e4c11a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:09.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.955+0000 7f6e51a97700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e4c135e50 0x7f6e4c1c3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:09.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.955+0000 7f6e51a97700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c09c700 con 0x7f6e4c135e50 2023-09-30T22:30:09.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.955+0000 7f6e51a97700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c11ce30 con 0x7f6e4c09d6f0 2023-09-30T22:30:09.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.955+0000 7f6e51a97700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c119c00 con 0x7f6e4c09e1d0 2023-09-30T22:30:09.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.956+0000 7f6e4bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e4c135e50 0x7f6e4c1c3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:09.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.956+0000 7f6e4bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e4c135e50 0x7f6e4c1c3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38594/0 (socket says 172.21.15.16:38594) 2023-09-30T22:30:09.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.956+0000 7f6e4bfff700 1 -- 172.21.15.16:0/1884740860 learned_addr learned my addr 172.21.15.16:0/1884740860 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:09.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.956+0000 7f6e4b7fe700 1 --2- 172.21.15.16:0/1884740860 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6e4c09d6f0 0x7f6e4c11a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:09.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.956+0000 7f6e4bfff700 1 -- 172.21.15.16:0/1884740860 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e4c09e1d0 msgr2=0x7f6e4c11a6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:30:09.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.956+0000 7f6e4bfff700 1 --2- 172.21.15.16:0/1884740860 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e4c09e1d0 0x7f6e4c11a6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:09.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.956+0000 7f6e4bfff700 1 -- 172.21.15.16:0/1884740860 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6e4c09d6f0 msgr2=0x7f6e4c11a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:09.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.956+0000 7f6e4bfff700 1 --2- 172.21.15.16:0/1884740860 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6e4c09d6f0 0x7f6e4c11a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:09.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.956+0000 7f6e4bfff700 1 -- 172.21.15.16:0/1884740860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e4000b600 con 0x7f6e4c135e50 2023-09-30T22:30:09.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.956+0000 7f6e4b7fe700 1 --2- 172.21.15.16:0/1884740860 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6e4c09d6f0 0x7f6e4c11a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:09.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.956+0000 7f6e4bfff700 1 --2- 172.21.15.16:0/1884740860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e4c135e50 0x7f6e4c1c3fe0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6e440123f0 tx=0x7f6e44012700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:09.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.956+0000 7f6e33fff700 1 -- 172.21.15.16:0/1884740860 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4400d2d0 con 0x7f6e4c135e50 2023-09-30T22:30:09.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.957+0000 7f6e33fff700 1 -- 172.21.15.16:0/1884740860 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e44013040 con 0x7f6e4c135e50 2023-09-30T22:30:09.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.957+0000 7f6e33fff700 1 -- 172.21.15.16:0/1884740860 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4400b740 con 0x7f6e4c135e50 2023-09-30T22:30:09.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.957+0000 7f6e51a97700 1 -- 172.21.15.16:0/1884740860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e4c0a1e20 con 0x7f6e4c135e50 2023-09-30T22:30:09.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.957+0000 7f6e51a97700 1 -- 172.21.15.16:0/1884740860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e4c11b220 con 0x7f6e4c135e50 2023-09-30T22:30:09.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.958+0000 7f6e33fff700 1 -- 172.21.15.16:0/1884740860 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f6e4400b8a0 con 0x7f6e4c135e50 2023-09-30T22:30:09.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.958+0000 7f6e51a97700 1 -- 172.21.15.16:0/1884740860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e4c09ef50 con 0x7f6e4c135e50 2023-09-30T22:30:09.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.958+0000 7f6e4b7fe700 1 -- 172.21.15.16:0/1884740860 >> v1:172.21.15.16:6816/126811 conn(0x7f6e3802f510 legacy=0x7f6e380319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:09.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.958+0000 7f6e33fff700 1 -- 172.21.15.16:0/1884740860 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7f6e440446e0 con 0x7f6e4c135e50 2023-09-30T22:30:09.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.962+0000 7f0eeb5ab700 1 Processor -- start 2023-09-30T22:30:09.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.962+0000 7f0eeb5ab700 1 -- start start 2023-09-30T22:30:09.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.962+0000 7f0eeb5ab700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0ee41316c0 0x7f0ee4131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:09.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.962+0000 7f0eeb5ab700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0ee409d9f0 0x7f0ee406a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:09.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.962+0000 7f0eeb5ab700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ee409e130 con 0x7f0ee406acb0 2023-09-30T22:30:09.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.962+0000 7f0eeb5ab700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ee41312b0 con 0x7f0ee409d9f0 2023-09-30T22:30:09.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.962+0000 7f0eeb5ab700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee409f6e0 con 0x7f0ee41316c0 2023-09-30T22:30:09.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.963+0000 7f0ee8b46700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0ee409d9f0 0x7f0ee406a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:09.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.963+0000 7f0ee8b46700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0ee409d9f0 0x7f0ee406a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38598/0 (socket says 172.21.15.16:38598) 2023-09-30T22:30:09.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.963+0000 7f0ee8b46700 1 -- 172.21.15.16:0/2079877277 learned_addr learned my addr 172.21.15.16:0/2079877277 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:09.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.963+0000 7f0ee9347700 1 --2- 172.21.15.16:0/2079877277 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0ee41316c0 0x7f0ee4131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:09.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.963+0000 7f0ee8b46700 1 -- 172.21.15.16:0/2079877277 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0ee41316c0 msgr2=0x7f0ee4131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:09.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.963+0000 7f0ee8b46700 1 --2- 172.21.15.16:0/2079877277 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0ee41316c0 0x7f0ee4131a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:09.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.963+0000 7f0ee8b46700 1 -- 172.21.15.16:0/2079877277 >> v1:172.21.15.16:6789/0 conn(0x7f0ee406acb0 legacy=0x7f0ee4138540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:09.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.963+0000 7f0ee8b46700 1 -- 172.21.15.16:0/2079877277 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ee409e760 con 0x7f0ee409d9f0 2023-09-30T22:30:09.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.963+0000 7f0ee8b46700 1 --2- 172.21.15.16:0/2079877277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ee409d9f0 0x7f0ee406a770 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0ed400de50 tx=0x7f0ed400f330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2225f95b4d9658e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:09.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.964+0000 7f0edbfff700 1 -- 172.21.15.16:0/2079877277 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed4012070 con 0x7f0ee409d9f0 2023-09-30T22:30:09.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.964+0000 7f0edbfff700 1 -- 172.21.15.16:0/2079877277 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ed400f9c0 con 0x7f0ee409d9f0 2023-09-30T22:30:09.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.964+0000 7f0edbfff700 1 -- 172.21.15.16:0/2079877277 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ed400fd70 con 0x7f0ee409d9f0 2023-09-30T22:30:09.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.964+0000 7f6e33fff700 1 -- 172.21.15.16:0/1884740860 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6e4400bb50 con 0x7f6e4c135e50 2023-09-30T22:30:09.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.964+0000 7fb5b7c92700 1 Processor -- start 2023-09-30T22:30:09.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.964+0000 7fb5b7c92700 1 -- start start 2023-09-30T22:30:09.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.965+0000 7f0eeb5ab700 1 -- 172.21.15.16:0/2079877277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ee409d9f0 msgr2=0x7f0ee406a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:09.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.965+0000 7f0eeb5ab700 1 --2- 172.21.15.16:0/2079877277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ee409d9f0 0x7f0ee406a770 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0ed400de50 tx=0x7f0ed400f330 comp rx=0 tx=0).stop 2023-09-30T22:30:09.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.965+0000 7f0eeb5ab700 1 -- 172.21.15.16:0/2079877277 shutdown_connections 2023-09-30T22:30:09.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.965+0000 7f0eeb5ab700 1 --2- 172.21.15.16:0/2079877277 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0ee41316c0 0x7f0ee4131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:09.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.965+0000 7f0eeb5ab700 1 --2- 172.21.15.16:0/2079877277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ee409d9f0 0x7f0ee406a770 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:09.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.965+0000 7f0eeb5ab700 1 -- 172.21.15.16:0/2079877277 >> 172.21.15.16:0/2079877277 conn(0x7f0ee4066f50 msgr2=0x7f0ee409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:09.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.965+0000 7f0eeb5ab700 1 -- 172.21.15.16:0/2079877277 shutdown_connections 2023-09-30T22:30:09.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.965+0000 7f0eeb5ab700 1 -- 172.21.15.16:0/2079877277 wait complete. 2023-09-30T22:30:09.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.965+0000 7fb5b7c92700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb5b0130560 0x7fb5b0130930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:09.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.965+0000 7fb5b7c92700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb5b009eca0 0x7fb5b006f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:09.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.965+0000 7fb5b7c92700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5b0070a50 con 0x7fb5b0130e70 2023-09-30T22:30:09.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.965+0000 7fb5b7c92700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5b00717d0 con 0x7fb5b009eca0 2023-09-30T22:30:09.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.965+0000 7fb5b7c92700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b0130340 con 0x7fb5b0130560 2023-09-30T22:30:09.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.966+0000 7fb5b622f700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb5b0130e70 0x7fb5b006fb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47946/0 (socket says 172.21.15.16:47946) 2023-09-30T22:30:09.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.966+0000 7fb5b622f700 1 -- 172.21.15.16:0/1624627216 learned_addr learned my addr 172.21.15.16:0/1624627216 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:30:09.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.966+0000 7fb5b4a2c700 1 -- 172.21.15.16:0/1624627216 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1238515877 0 0) 0x7fb5b0070a50 con 0x7fb5b0130e70 2023-09-30T22:30:09.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.966+0000 7fb5b4a2c700 1 -- 172.21.15.16:0/1624627216 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb59c0057c0 con 0x7fb5b0130e70 2023-09-30T22:30:09.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.966+0000 7fb5b4a2c700 1 -- 172.21.15.16:0/1624627216 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1759055143 0 0) 0x7fb59c0057c0 con 0x7fb5b0130e70 2023-09-30T22:30:09.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.966+0000 7fb5b4a2c700 1 -- 172.21.15.16:0/1624627216 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb5b0130560 msgr2=0x7fb5b0130930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:09.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.966+0000 7fb5b4a2c700 1 --2- 172.21.15.16:0/1624627216 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb5b0130560 0x7fb5b0130930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:09.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.966+0000 7fb5b4a2c700 1 -- 172.21.15.16:0/1624627216 >> v2:172.21.15.16:3300/0 conn(0x7fb5b009eca0 msgr2=0x7fb5b006f620 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T22:30:09.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.966+0000 7fb5b4a2c700 1 --2- 172.21.15.16:0/1624627216 >> v2:172.21.15.16:3300/0 conn(0x7fb5b009eca0 0x7fb5b006f620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:09.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.966+0000 7fb5b4a2c700 1 -- 172.21.15.16:0/1624627216 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5b0071060 con 0x7fb5b0130e70 2023-09-30T22:30:09.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.966+0000 7f0eeb5ab700 1 Processor -- start 2023-09-30T22:30:09.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.966+0000 7fb5b4a2c700 1 -- 172.21.15.16:0/1624627216 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1074163509 0 0) 0x7fb5ac0019f0 con 0x7fb5b0130e70 2023-09-30T22:30:09.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.966+0000 7fb5b4a2c700 1 -- 172.21.15.16:0/1624627216 >> v1:172.21.15.16:6789/0 conn(0x7fb5b0130e70 legacy=0x7fb5b006fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:09.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.966+0000 7fb5b4a2c700 1 --2- 172.21.15.16:0/1624627216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb59c005c10 0x7fb59c006020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:09.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.966+0000 7fb5b4a2c700 1 --2- 172.21.15.16:0/1624627216 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb59c006a60 0x7fb59c006ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:09.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.966+0000 7fb5b4a2c700 1 --2- 172.21.15.16:0/1624627216 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb5b0130560 0x7fb59c007410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:09.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.966+0000 7fb5b4a2c700 1 -- 172.21.15.16:0/1624627216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b00717d0 con 0x7fb59c005c10 2023-09-30T22:30:09.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.966+0000 7fb5b4a2c700 1 -- 172.21.15.16:0/1624627216 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b0130340 con 0x7fb5b0130560 2023-09-30T22:30:09.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.966+0000 7fb5b4a2c700 1 -- 172.21.15.16:0/1624627216 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb59c009c80 con 0x7fb59c006a60 2023-09-30T22:30:09.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.966+0000 7fb5b5a2e700 1 --2- 172.21.15.16:0/1624627216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb59c005c10 0x7fb59c006020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:09.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.967+0000 7f0eeb5ab700 1 -- start start 2023-09-30T22:30:09.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.967+0000 7fb5b622f700 1 --2- 172.21.15.16:0/1624627216 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb5b0130560 0x7fb59c007410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:09.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.967+0000 7fb5b5a2e700 1 -- 172.21.15.16:0/1624627216 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb59c006a60 msgr2=0x7fb59c006ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:09.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.967+0000 7fb5b5a2e700 1 --2- 172.21.15.16:0/1624627216 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb59c006a60 0x7fb59c006ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:09.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.967+0000 7fb5b5a2e700 1 -- 172.21.15.16:0/1624627216 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb5b0130560 msgr2=0x7fb59c007410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:09.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.967+0000 7fb5b5a2e700 1 --2- 172.21.15.16:0/1624627216 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb5b0130560 0x7fb59c007410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:09.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.967+0000 7fb5b5a2e700 1 -- 172.21.15.16:0/1624627216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb59c0057c0 con 0x7fb59c005c10 2023-09-30T22:30:09.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.967+0000 7fb5b5a2e700 1 --2- 172.21.15.16:0/1624627216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb59c005c10 0x7fb59c006020 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8009910 tx=0x7fb59c007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a57fa73e297dc8df server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:09.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.967+0000 7f0eeb5ab700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0ee406acb0 0x7f0ee4119480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:09.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.967+0000 7f0eeb5ab700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ee409d9f0 0x7f0ee41199c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:09.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.967+0000 7f0eeb5ab700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0ee41316c0 0x7f0ee4119f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:09.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.967+0000 7f0eeb5ab700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee40a35b0 con 0x7f0ee409d9f0 2023-09-30T22:30:09.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.967+0000 7f0eeb5ab700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee411b680 con 0x7f0ee41316c0 2023-09-30T22:30:09.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.967+0000 7f0eeb5ab700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee412b900 con 0x7f0ee406acb0 2023-09-30T22:30:09.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.968+0000 7f0ee9b48700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0ee41316c0 0x7f0ee4119f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:09.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.968+0000 7f0ee8b46700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ee409d9f0 0x7f0ee41199c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:09.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.968+0000 7f0ee9b48700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0ee41316c0 0x7f0ee4119f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41368/0 (socket says 172.21.15.16:41368) 2023-09-30T22:30:09.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.968+0000 7f0ee9b48700 1 -- 172.21.15.16:0/1459501921 learned_addr learned my addr 172.21.15.16:0/1459501921 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:09.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.968+0000 7f0ee9347700 1 --2- 172.21.15.16:0/1459501921 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0ee406acb0 0x7f0ee4119480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:09.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.968+0000 7fb5b4a2c700 1 -- 172.21.15.16:0/1624627216 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a8004230 con 0x7fb59c005c10 2023-09-30T22:30:09.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.968+0000 7fb5b4a2c700 1 -- 172.21.15.16:0/1624627216 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5a80043b0 con 0x7fb59c005c10 2023-09-30T22:30:09.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.968+0000 7fb5b4a2c700 1 -- 172.21.15.16:0/1624627216 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a80038c0 con 0x7fb59c005c10 2023-09-30T22:30:09.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.968+0000 7f0ee9b48700 1 -- 172.21.15.16:0/1459501921 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0ee406acb0 msgr2=0x7f0ee4119480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:09.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.968+0000 7f0ee9b48700 1 --2- 172.21.15.16:0/1459501921 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0ee406acb0 0x7f0ee4119480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:09.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.968+0000 7f0ee9b48700 1 -- 172.21.15.16:0/1459501921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ee409d9f0 msgr2=0x7f0ee41199c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:09.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.968+0000 7f0ee9b48700 1 --2- 172.21.15.16:0/1459501921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ee409d9f0 0x7f0ee41199c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:09.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.968+0000 7f0ee9b48700 1 -- 172.21.15.16:0/1459501921 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ed400db30 con 0x7f0ee41316c0 2023-09-30T22:30:09.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.968+0000 7f0ee9b48700 1 --2- 172.21.15.16:0/1459501921 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0ee41316c0 0x7f0ee4119f00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0edc00cc80 tx=0x7f0edc00cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:09.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.969+0000 7f0ed9ffb700 1 -- 172.21.15.16:0/1459501921 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0edc003c00 con 0x7f0ee41316c0 2023-09-30T22:30:09.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.969+0000 7f0ed9ffb700 1 -- 172.21.15.16:0/1459501921 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0edc003d60 con 0x7f0ee41316c0 2023-09-30T22:30:09.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.969+0000 7f0ed9ffb700 1 -- 172.21.15.16:0/1459501921 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0edc00aaa0 con 0x7f0ee41316c0 2023-09-30T22:30:09.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.969+0000 7f0eeb5ab700 1 -- 172.21.15.16:0/1459501921 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ee412b620 con 0x7f0ee41316c0 2023-09-30T22:30:09.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.969+0000 7f0eeb5ab700 1 -- 172.21.15.16:0/1459501921 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ee411bc20 con 0x7f0ee41316c0 2023-09-30T22:30:09.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.969+0000 7fb5b7c92700 1 -- 172.21.15.16:0/1624627216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb59c005c10 msgr2=0x7fb59c006020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:09.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.969+0000 7fb5b7c92700 1 --2- 172.21.15.16:0/1624627216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb59c005c10 0x7fb59c006020 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8009910 tx=0x7fb59c007a90 comp rx=0 tx=0).stop 2023-09-30T22:30:09.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.969+0000 7fb5b622f700 1 -- 172.21.15.16:0/1624627216 reap_dead start 2023-09-30T22:30:09.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.969+0000 7fb5b7c92700 1 -- 172.21.15.16:0/1624627216 shutdown_connections 2023-09-30T22:30:09.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.969+0000 7fb5b7c92700 1 -- 172.21.15.16:0/1624627216 >> 172.21.15.16:0/1624627216 conn(0x7fb5b00a2aa0 msgr2=0x7fb5b00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:09.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.970+0000 7f0eeb5ab700 1 -- 172.21.15.16:0/1459501921 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ee4074ea0 con 0x7f0ee41316c0 2023-09-30T22:30:09.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.970+0000 7f0ed9ffb700 1 -- 172.21.15.16:0/1459501921 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f0edc00ac00 con 0x7f0ee41316c0 2023-09-30T22:30:09.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.970+0000 7fb5b7c92700 1 -- 172.21.15.16:0/1624627216 shutdown_connections 2023-09-30T22:30:09.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.970+0000 7fb5b7c92700 1 -- 172.21.15.16:0/1624627216 wait complete. 2023-09-30T22:30:09.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.970+0000 7f0ed9ffb700 1 -- 172.21.15.16:0/1459501921 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7f0edc03faa0 con 0x7f0ee41316c0 2023-09-30T22:30:09.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.971+0000 7f0ee9347700 1 -- 172.21.15.16:0/1459501921 >> v1:172.21.15.16:6816/126811 conn(0x7f0ec802f510 legacy=0x7f0ec80319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:09.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.972+0000 7fb5b7c92700 1 Processor -- start 2023-09-30T22:30:09.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.972+0000 7fb5b7c92700 1 -- start start 2023-09-30T22:30:09.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.973+0000 7fb5b7c92700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb59c005c10 0x7fb5b0118cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:09.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.973+0000 7fb5b7c92700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5b012b830 0x7fb5b0119210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:09.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.973+0000 7fb5b7c92700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb5b0119750 0x7fb5b01ca560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:09.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.973+0000 7fb5b7c92700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b011bd40 con 0x7fb5b012b830 2023-09-30T22:30:09.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.973+0000 7fb5b7c92700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b011bf40 con 0x7fb5b0119750 2023-09-30T22:30:09.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.973+0000 7fb5b7c92700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b011ae30 con 0x7fb59c005c10 2023-09-30T22:30:09.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.973+0000 7fb5b5a2e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb59c005c10 0x7fb5b0118cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:09.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.973+0000 7fb5b5a2e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb59c005c10 0x7fb5b0118cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:50920/0 (socket says 172.21.15.16:50920) 2023-09-30T22:30:09.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.973+0000 7fb5b5a2e700 1 -- 172.21.15.16:0/985411341 learned_addr learned my addr 172.21.15.16:0/985411341 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:09.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.973+0000 7fb5b5a2e700 1 -- 172.21.15.16:0/985411341 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb5b0119750 msgr2=0x7fb5b01ca560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:09.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.973+0000 7fb5b5a2e700 1 --2- 172.21.15.16:0/985411341 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb5b0119750 0x7fb5b01ca560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:09.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.973+0000 7fb5b5a2e700 1 -- 172.21.15.16:0/985411341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5b012b830 msgr2=0x7fb5b0119210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:09.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.973+0000 7fb5b5a2e700 1 --2- 172.21.15.16:0/985411341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5b012b830 0x7fb5b0119210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:09.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.973+0000 7fb5b5a2e700 1 -- 172.21.15.16:0/985411341 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5b011b2f0 con 0x7fb59c005c10 2023-09-30T22:30:09.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.974+0000 7fb5b5a2e700 1 --2- 172.21.15.16:0/985411341 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb59c005c10 0x7fb5b0118cd0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8013040 tx=0x7fb5a8004860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:09.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.976+0000 7fb5a67fc700 1 -- 172.21.15.16:0/985411341 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a8003c10 con 0x7fb59c005c10 2023-09-30T22:30:09.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.976+0000 7fb5a67fc700 1 -- 172.21.15.16:0/985411341 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5a8003db0 con 0x7fb59c005c10 2023-09-30T22:30:09.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.976+0000 7fb5b7c92700 1 -- 172.21.15.16:0/985411341 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5b012e7b0 con 0x7fb59c005c10 2023-09-30T22:30:09.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.976+0000 7fb5a67fc700 1 -- 172.21.15.16:0/985411341 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a8026690 con 0x7fb59c005c10 2023-09-30T22:30:09.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.976+0000 7fb5b7c92700 1 -- 172.21.15.16:0/985411341 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5b011a0d0 con 0x7fb59c005c10 2023-09-30T22:30:09.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.978+0000 7fb593fff700 1 -- 172.21.15.16:0/985411341 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb58c0052e0 con 0x7fb59c005c10 2023-09-30T22:30:09.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.978+0000 7fb5a67fc700 1 -- 172.21.15.16:0/985411341 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fb5a8011420 con 0x7fb59c005c10 2023-09-30T22:30:09.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.980+0000 7f0ed9ffb700 1 -- 172.21.15.16:0/1459501921 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0edc019710 con 0x7f0ee41316c0 2023-09-30T22:30:09.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.979+0000 7fb5a67fc700 1 -- 172.21.15.16:0/985411341 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7fb5a8040c90 con 0x7fb59c005c10 2023-09-30T22:30:09.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.979+0000 7fb5b522d700 1 -- 172.21.15.16:0/985411341 >> v1:172.21.15.16:6816/126811 conn(0x7fb59402f510 legacy=0x7fb5940319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:09.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:09.984+0000 7fb5a67fc700 1 -- 172.21.15.16:0/985411341 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb5a8019790 con 0x7fb59c005c10 2023-09-30T22:30:10.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:10.121+0000 7f10b259c700 1 -- 172.21.15.16:0/4089796319 >> v1:172.21.15.16:6816/126811 conn(0x7f109402f510 legacy=0x7f10940319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:10.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:10.159+0000 7f6e4b7fe700 1 -- 172.21.15.16:0/1884740860 >> v1:172.21.15.16:6816/126811 conn(0x7f6e3802f510 legacy=0x7f6e380319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:10.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:10.172+0000 7f0ee9347700 1 -- 172.21.15.16:0/1459501921 >> v1:172.21.15.16:6816/126811 conn(0x7f0ec802f510 legacy=0x7f0ec80319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:10.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:10.177+0000 7f10b8cdb700 1 -- 172.21.15.16:0/4089796319 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f10b4074ea0 con 0x7f10b4130e50 2023-09-30T22:30:10.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:10.177+0000 7f6e51a97700 1 -- 172.21.15.16:0/1884740860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f6e4c11ca50 con 0x7f6e4c135e50 2023-09-30T22:30:10.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:10.177+0000 7f10a2ffd700 1 -- 172.21.15.16:0/4089796319 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f10ac02e3e0 con 0x7f10b4130e50 2023-09-30T22:30:10.179 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:30:10.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:10.178+0000 7f6e33fff700 1 -- 172.21.15.16:0/1884740860 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6e44016ae0 con 0x7f6e4c135e50 2023-09-30T22:30:10.179 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:30:10.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:10.179+0000 7f10b8cdb700 1 -- 172.21.15.16:0/4089796319 >> v1:172.21.15.16:6816/126811 conn(0x7f109402f510 legacy=0x7f10940319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:30:10.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:10.180+0000 7f10b8cdb700 1 -- 172.21.15.16:0/4089796319 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f10b4130e50 msgr2=0x7f10b4119750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:10.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:10.180+0000 7f10b8cdb700 1 --2- 172.21.15.16:0/4089796319 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f10b4130e50 0x7f10b4119750 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f10ac014040 tx=0x7f10ac004ad0 comp rx=0 tx=0).stop 2023-09-30T22:30:10.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:10.180+0000 7fb5b522d700 1 -- 172.21.15.16:0/985411341 >> v1:172.21.15.16:6816/126811 conn(0x7fb59402f510 legacy=0x7fb5940319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:10.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:10.180+0000 7f10b8cdb700 1 -- 172.21.15.16:0/4089796319 shutdown_connections 2023-09-30T22:30:10.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:10.180+0000 7f10b8cdb700 1 --2- 172.21.15.16:0/4089796319 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f10b406fb20 0x7f10b4118cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:10.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:10.180+0000 7f10b8cdb700 1 --2- 172.21.15.16:0/4089796319 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f10b4130e50 0x7f10b4119750 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:10.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:10.180+0000 7f10b8cdb700 1 --2- 172.21.15.16:0/4089796319 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10b4130540 0x7f10b4119210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:10.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:10.180+0000 7f10b8cdb700 1 -- 172.21.15.16:0/4089796319 >> 172.21.15.16:0/4089796319 conn(0x7f10b40a2aa0 msgr2=0x7f10b412d450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:10.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:10.180+0000 7f10b8cdb700 1 -- 172.21.15.16:0/4089796319 shutdown_connections 2023-09-30T22:30:10.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:10.180+0000 7f10b8cdb700 1 -- 172.21.15.16:0/4089796319 wait complete. 2023-09-30T22:30:10.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:10.180+0000 7f6e51a97700 1 -- 172.21.15.16:0/1884740860 >> v1:172.21.15.16:6816/126811 conn(0x7f6e3802f510 legacy=0x7f6e380319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:30:10.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:10.180+0000 7f6e51a97700 1 -- 172.21.15.16:0/1884740860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e4c135e50 msgr2=0x7f6e4c1c3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:10.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:10.180+0000 7f6e51a97700 1 --2- 172.21.15.16:0/1884740860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e4c135e50 0x7f6e4c1c3fe0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6e440123f0 tx=0x7f6e44012700 comp rx=0 tx=0).stop 2023-09-30T22:30:10.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:10.180+0000 7f6e51a97700 1 -- 172.21.15.16:0/1884740860 shutdown_connections 2023-09-30T22:30:10.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:10.180+0000 7f6e51a97700 1 --2- 172.21.15.16:0/1884740860 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6e4c09d6f0 0x7f6e4c11a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:10.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:10.180+0000 7f6e51a97700 1 --2- 172.21.15.16:0/1884740860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e4c135e50 0x7f6e4c1c3fe0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:10.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:10.180+0000 7f6e51a97700 1 --2- 172.21.15.16:0/1884740860 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e4c09e1d0 0x7f6e4c11a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:10.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:10.180+0000 7f6e51a97700 1 -- 172.21.15.16:0/1884740860 >> 172.21.15.16:0/1884740860 conn(0x7f6e4c066f50 msgr2=0x7f6e4c06f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:10.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:10.181+0000 7f6e51a97700 1 -- 172.21.15.16:0/1884740860 shutdown_connections 2023-09-30T22:30:10.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:10.181+0000 7f6e51a97700 1 -- 172.21.15.16:0/1884740860 wait complete. 2023-09-30T22:30:10.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:10.187+0000 7f0eeb5ab700 1 -- 172.21.15.16:0/1459501921 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f0ee409e130 con 0x7f0ee41316c0 2023-09-30T22:30:10.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:10.187+0000 7f0ed9ffb700 1 -- 172.21.15.16:0/1459501921 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0ee409e130 con 0x7f0ee41316c0 2023-09-30T22:30:10.189 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:30:10.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:10.190+0000 7f0eeb5ab700 1 -- 172.21.15.16:0/1459501921 >> v1:172.21.15.16:6816/126811 conn(0x7f0ec802f510 legacy=0x7f0ec80319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:30:10.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:10.190+0000 7f0eeb5ab700 1 -- 172.21.15.16:0/1459501921 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0ee41316c0 msgr2=0x7f0ee4119f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:10.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:10.190+0000 7f0eeb5ab700 1 --2- 172.21.15.16:0/1459501921 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0ee41316c0 0x7f0ee4119f00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0edc00cc80 tx=0x7f0edc00cf90 comp rx=0 tx=0).stop 2023-09-30T22:30:10.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:10.190+0000 7f0eeb5ab700 1 -- 172.21.15.16:0/1459501921 shutdown_connections 2023-09-30T22:30:10.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:10.190+0000 7f0eeb5ab700 1 --2- 172.21.15.16:0/1459501921 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0ee41316c0 0x7f0ee4119f00 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:10.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:10.190+0000 7f0eeb5ab700 1 --2- 172.21.15.16:0/1459501921 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0ee406acb0 0x7f0ee4119480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:10.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:10.190+0000 7f0eeb5ab700 1 --2- 172.21.15.16:0/1459501921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ee409d9f0 0x7f0ee41199c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:10.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:10.190+0000 7f0eeb5ab700 1 -- 172.21.15.16:0/1459501921 >> 172.21.15.16:0/1459501921 conn(0x7f0ee4066f50 msgr2=0x7f0ee412f110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:10.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:10.190+0000 7f0eeb5ab700 1 -- 172.21.15.16:0/1459501921 shutdown_connections 2023-09-30T22:30:10.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:10.190+0000 7f0eeb5ab700 1 -- 172.21.15.16:0/1459501921 wait complete. 2023-09-30T22:30:10.198 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:30:10.200 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:30:10.208 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:30:10.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:10.228+0000 7fb593fff700 1 -- 172.21.15.16:0/985411341 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb58c005860 con 0x7fb59c005c10 2023-09-30T22:30:10.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:10.231+0000 7fb5a67fc700 1 -- 172.21.15.16:0/985411341 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb5a800b3e0 con 0x7fb59c005c10 2023-09-30T22:30:10.232 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:30:10.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:10.233+0000 7fb593fff700 1 -- 172.21.15.16:0/985411341 >> v1:172.21.15.16:6816/126811 conn(0x7fb59402f510 legacy=0x7fb5940319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:30:10.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:10.233+0000 7fb593fff700 1 -- 172.21.15.16:0/985411341 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb59c005c10 msgr2=0x7fb5b0118cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:10.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:10.233+0000 7fb593fff700 1 --2- 172.21.15.16:0/985411341 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb59c005c10 0x7fb5b0118cd0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8013040 tx=0x7fb5a8004860 comp rx=0 tx=0).stop 2023-09-30T22:30:10.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:10.233+0000 7fb593fff700 1 -- 172.21.15.16:0/985411341 shutdown_connections 2023-09-30T22:30:10.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:10.233+0000 7fb593fff700 1 --2- 172.21.15.16:0/985411341 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb5b0119750 0x7fb5b01ca560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:10.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:10.233+0000 7fb593fff700 1 --2- 172.21.15.16:0/985411341 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb59c005c10 0x7fb5b0118cd0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:10.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:10.233+0000 7fb593fff700 1 --2- 172.21.15.16:0/985411341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5b012b830 0x7fb5b0119210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:10.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:10.233+0000 7fb593fff700 1 -- 172.21.15.16:0/985411341 >> 172.21.15.16:0/985411341 conn(0x7fb5b00a2aa0 msgr2=0x7fb5b012d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:10.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:10.233+0000 7fb593fff700 1 -- 172.21.15.16:0/985411341 shutdown_connections 2023-09-30T22:30:10.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:10.233+0000 7fb593fff700 1 -- 172.21.15.16:0/985411341 wait complete. 2023-09-30T22:30:10.248 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:30:11.200 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:11.200 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:11.209 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:11.249 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:11.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.352+0000 7fb4e7b90700 1 Processor -- start 2023-09-30T22:30:11.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.353+0000 7fb4e7b90700 1 -- start start 2023-09-30T22:30:11.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.353+0000 7fb4e7b90700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb4e009e1d0 0x7fb4e0135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:11.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.353+0000 7fb4e7b90700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb4e0135e50 0x7fb4e0138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:11.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.353+0000 7fb4e7b90700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4e009f130 con 0x7fb4e009d6f0 2023-09-30T22:30:11.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.353+0000 7fb4e7b90700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4e009f520 con 0x7fb4e009e1d0 2023-09-30T22:30:11.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.353+0000 7fb4e7b90700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e009e930 con 0x7fb4e0135e50 2023-09-30T22:30:11.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.353+0000 7fb4e512b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb4e009e1d0 0x7fb4e0135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:11.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.354+0000 7fb4e512b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb4e009e1d0 0x7fb4e0135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38636/0 (socket says 172.21.15.16:38636) 2023-09-30T22:30:11.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.354+0000 7fb4e592c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb4e009d6f0 0x7fb4e009dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47962/0 (socket says 172.21.15.16:47962) 2023-09-30T22:30:11.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.354+0000 7fb4e512b700 1 -- 172.21.15.16:0/1603553282 learned_addr learned my addr 172.21.15.16:0/1603553282 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:11.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.354+0000 7fb4e612d700 1 --2- 172.21.15.16:0/1603553282 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb4e0135e50 0x7fb4e0138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:11.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.354+0000 7fb4e512b700 1 -- 172.21.15.16:0/1603553282 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb4e0135e50 msgr2=0x7fb4e0138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:11.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.354+0000 7fb4e512b700 1 --2- 172.21.15.16:0/1603553282 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb4e0135e50 0x7fb4e0138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.354+0000 7fb4e512b700 1 -- 172.21.15.16:0/1603553282 >> v1:172.21.15.16:6789/0 conn(0x7fb4e009d6f0 legacy=0x7fb4e009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:11.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.354+0000 7fb4e512b700 1 -- 172.21.15.16:0/1603553282 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4e009b2d0 con 0x7fb4e009e1d0 2023-09-30T22:30:11.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.354+0000 7fb4e492a700 1 -- 172.21.15.16:0/1603553282 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 446226310 0 0) 0x7fb4e009f130 con 0x7fb4e009d6f0 2023-09-30T22:30:11.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.354+0000 7fb4e512b700 1 --2- 172.21.15.16:0/1603553282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e009e1d0 0x7fb4e0135910 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb4d00098f0 tx=0x7fb4d0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9225bc628e513d6d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:11.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.354+0000 7fb4e492a700 1 -- 172.21.15.16:0/1603553282 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d0011830 con 0x7fb4e009e1d0 2023-09-30T22:30:11.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.355+0000 7fb4e492a700 1 -- 172.21.15.16:0/1603553282 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4d0011990 con 0x7fb4e009e1d0 2023-09-30T22:30:11.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.355+0000 7fb4e492a700 1 -- 172.21.15.16:0/1603553282 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d0011d20 con 0x7fb4e009e1d0 2023-09-30T22:30:11.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.355+0000 7fb4e7b90700 1 -- 172.21.15.16:0/1603553282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e009e1d0 msgr2=0x7fb4e0135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:11.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.355+0000 7fb4e7b90700 1 --2- 172.21.15.16:0/1603553282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e009e1d0 0x7fb4e0135910 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb4d00098f0 tx=0x7fb4d0009c00 comp rx=0 tx=0).stop 2023-09-30T22:30:11.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.355+0000 7fb4e7b90700 1 -- 172.21.15.16:0/1603553282 shutdown_connections 2023-09-30T22:30:11.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.355+0000 7fb4e7b90700 1 --2- 172.21.15.16:0/1603553282 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb4e0135e50 0x7fb4e0138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.355+0000 7fb4e7b90700 1 --2- 172.21.15.16:0/1603553282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e009e1d0 0x7fb4e0135910 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.355+0000 7fb4e7b90700 1 -- 172.21.15.16:0/1603553282 >> 172.21.15.16:0/1603553282 conn(0x7fb4e0066f50 msgr2=0x7fb4e009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:11.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.355+0000 7fb4e7b90700 1 -- 172.21.15.16:0/1603553282 shutdown_connections 2023-09-30T22:30:11.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.356+0000 7fb4e7b90700 1 -- 172.21.15.16:0/1603553282 wait complete. 2023-09-30T22:30:11.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.357+0000 7fb4e7b90700 1 Processor -- start 2023-09-30T22:30:11.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.357+0000 7fb4e7b90700 1 -- start start 2023-09-30T22:30:11.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.357+0000 7fb4e7b90700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e009d6f0 0x7fb4e011b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:11.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.357+0000 7fb4e7b90700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb4e009e1d0 0x7fb4e011b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:11.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.357+0000 7fb4e7b90700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb4e0135e50 0x7fb4e011bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:11.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.357+0000 7fb4e7b90700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e006b5b0 con 0x7fb4e009d6f0 2023-09-30T22:30:11.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.357+0000 7fb4e7b90700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e006b3c0 con 0x7fb4e0135e50 2023-09-30T22:30:11.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.357+0000 7fb4e7b90700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e011c8a0 con 0x7fb4e009e1d0 2023-09-30T22:30:11.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.357+0000 7fb4e612d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb4e0135e50 0x7fb4e011bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:11.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.357+0000 7fb4e612d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb4e0135e50 0x7fb4e011bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41388/0 (socket says 172.21.15.16:41388) 2023-09-30T22:30:11.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.357+0000 7fb4e612d700 1 -- 172.21.15.16:0/76658748 learned_addr learned my addr 172.21.15.16:0/76658748 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:11.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.357+0000 7fb4e512b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb4e009e1d0 0x7fb4e011b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:11.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.358+0000 7fb4e592c700 1 --2- 172.21.15.16:0/76658748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e009d6f0 0x7fb4e011b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:11.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.358+0000 7fb4e512b700 1 -- 172.21.15.16:0/76658748 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb4e0135e50 msgr2=0x7fb4e011bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:11.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.358+0000 7fb4e512b700 1 --2- 172.21.15.16:0/76658748 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb4e0135e50 0x7fb4e011bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.358+0000 7fb4e512b700 1 -- 172.21.15.16:0/76658748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e009d6f0 msgr2=0x7fb4e011b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:11.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.358+0000 7fb4e512b700 1 --2- 172.21.15.16:0/76658748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e009d6f0 0x7fb4e011b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.358+0000 7fb4e512b700 1 -- 172.21.15.16:0/76658748 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4d0009550 con 0x7fb4e009e1d0 2023-09-30T22:30:11.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.358+0000 7fb4e512b700 1 --2- 172.21.15.16:0/76658748 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb4e009e1d0 0x7fb4e011b990 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0016000 tx=0x7fb4d0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:11.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.358+0000 7fb4d67fc700 1 -- 172.21.15.16:0/76658748 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d0004550 con 0x7fb4e009e1d0 2023-09-30T22:30:11.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.358+0000 7f3620aba700 1 Processor -- start 2023-09-30T22:30:11.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.358+0000 7fb4d67fc700 1 -- 172.21.15.16:0/76658748 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4d001f070 con 0x7fb4e009e1d0 2023-09-30T22:30:11.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.358+0000 7fb4d67fc700 1 -- 172.21.15.16:0/76658748 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d00038e0 con 0x7fb4e009e1d0 2023-09-30T22:30:11.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.358+0000 7f3620aba700 1 -- start start 2023-09-30T22:30:11.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.359+0000 7fb4e7b90700 1 -- 172.21.15.16:0/76658748 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4e009c700 con 0x7fb4e009e1d0 2023-09-30T22:30:11.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.359+0000 7f3620aba700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f361c09e420 0x7f361c1388a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:11.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.359+0000 7f3620aba700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f361c09dcd0 0x7f361c13adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:11.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.359+0000 7fb4e7b90700 1 -- 172.21.15.16:0/76658748 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4e011dc00 con 0x7fb4e009e1d0 2023-09-30T22:30:11.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.360+0000 7fb4d67fc700 1 -- 172.21.15.16:0/76658748 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fb4d0003a40 con 0x7fb4e009e1d0 2023-09-30T22:30:11.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.359+0000 7f3620aba700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f361c09cdc0 con 0x7f361c09d1f0 2023-09-30T22:30:11.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.359+0000 7f3620aba700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f361c099e50 con 0x7f361c09dcd0 2023-09-30T22:30:11.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.359+0000 7f3620aba700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f361c09cff0 con 0x7f361c09e420 2023-09-30T22:30:11.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.360+0000 7f361ad9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f361c09dcd0 0x7f361c13adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:11.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.360+0000 7f361ad9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f361c09dcd0 0x7f361c13adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41402/0 (socket says 172.21.15.16:41402) 2023-09-30T22:30:11.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.360+0000 7f361ad9d700 1 -- 172.21.15.16:0/4080951966 learned_addr learned my addr 172.21.15.16:0/4080951966 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:11.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.360+0000 7f3619d9b700 1 --2- 172.21.15.16:0/4080951966 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f361c09e420 0x7f361c1388a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:11.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.360+0000 7f361ad9d700 1 -- 172.21.15.16:0/4080951966 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f361c09e420 msgr2=0x7f361c1388a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:11.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.360+0000 7f361ad9d700 1 --2- 172.21.15.16:0/4080951966 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f361c09e420 0x7f361c1388a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.360+0000 7f361ad9d700 1 -- 172.21.15.16:0/4080951966 >> v1:172.21.15.16:6789/0 conn(0x7f361c09d1f0 legacy=0x7f361c09d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:11.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.360+0000 7f361ad9d700 1 -- 172.21.15.16:0/4080951966 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f361c09e250 con 0x7f361c09dcd0 2023-09-30T22:30:11.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.360+0000 7f3619d9b700 1 --2- 172.21.15.16:0/4080951966 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f361c09e420 0x7f361c1388a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:30:11.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.360+0000 7f361ad9d700 1 --2- 172.21.15.16:0/4080951966 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f361c09dcd0 0x7f361c13adf0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3614009e80 tx=0x7f361400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f78ca774885fe91 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:11.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.360+0000 7f361959a700 1 -- 172.21.15.16:0/4080951966 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361400e070 con 0x7f361c09dcd0 2023-09-30T22:30:11.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.361+0000 7f361959a700 1 -- 172.21.15.16:0/4080951966 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f361400b9c0 con 0x7f361c09dcd0 2023-09-30T22:30:11.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.361+0000 7f361959a700 1 -- 172.21.15.16:0/4080951966 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361400bd70 con 0x7f361c09dcd0 2023-09-30T22:30:11.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.361+0000 7fb4e7b90700 1 -- 172.21.15.16:0/76658748 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4e0074ea0 con 0x7fb4e009e1d0 2023-09-30T22:30:11.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.361+0000 7fb4e592c700 1 -- 172.21.15.16:0/76658748 >> v1:172.21.15.16:6816/126811 conn(0x7fb4c402f510 legacy=0x7fb4c40319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:11.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.361+0000 7fb4d67fc700 1 -- 172.21.15.16:0/76658748 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7fb4d0021070 con 0x7fb4e009e1d0 2023-09-30T22:30:11.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.361+0000 7f3620aba700 1 -- 172.21.15.16:0/4080951966 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f361c09dcd0 msgr2=0x7f361c13adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:11.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.361+0000 7f3620aba700 1 --2- 172.21.15.16:0/4080951966 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f361c09dcd0 0x7f361c13adf0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3614009e80 tx=0x7f361400b330 comp rx=0 tx=0).stop 2023-09-30T22:30:11.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.362+0000 7f3620aba700 1 -- 172.21.15.16:0/4080951966 shutdown_connections 2023-09-30T22:30:11.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.362+0000 7f3620aba700 1 --2- 172.21.15.16:0/4080951966 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f361c09dcd0 0x7f361c13adf0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.362+0000 7f3620aba700 1 --2- 172.21.15.16:0/4080951966 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f361c09e420 0x7f361c1388a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.362+0000 7f3620aba700 1 -- 172.21.15.16:0/4080951966 >> 172.21.15.16:0/4080951966 conn(0x7f361c0a2360 msgr2=0x7f361c0a2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:11.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.362+0000 7f3620aba700 1 -- 172.21.15.16:0/4080951966 shutdown_connections 2023-09-30T22:30:11.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.362+0000 7f3620aba700 1 -- 172.21.15.16:0/4080951966 wait complete. 2023-09-30T22:30:11.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.363+0000 7f3620aba700 1 Processor -- start 2023-09-30T22:30:11.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.363+0000 7f3620aba700 1 -- start start 2023-09-30T22:30:11.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.364+0000 7f3620aba700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f361c09d1f0 0x7f361c12a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:11.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.364+0000 7f3620aba700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f361c09dcd0 0x7f361c12a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:11.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.364+0000 7f3620aba700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f361c09e420 0x7f361c12ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:11.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.364+0000 7f3620aba700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361c12bd60 con 0x7f361c09dcd0 2023-09-30T22:30:11.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.364+0000 7f3620aba700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f361c12d880 con 0x7f361c09e420 2023-09-30T22:30:11.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.364+0000 7f3620aba700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f361c12c440 con 0x7f361c09d1f0 2023-09-30T22:30:11.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.364+0000 7f361ad9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f361c09e420 0x7f361c12ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:11.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.364+0000 7f361ad9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f361c09e420 0x7f361c12ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41416/0 (socket says 172.21.15.16:41416) 2023-09-30T22:30:11.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.364+0000 7f361ad9d700 1 -- 172.21.15.16:0/90621940 learned_addr learned my addr 172.21.15.16:0/90621940 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:11.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.365+0000 7f361a59c700 1 --2- 172.21.15.16:0/90621940 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f361c09d1f0 0x7f361c12a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:11.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.365+0000 7f361a59c700 1 -- 172.21.15.16:0/90621940 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f361c09e420 msgr2=0x7f361c12ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:11.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.365+0000 7f361a59c700 1 --2- 172.21.15.16:0/90621940 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f361c09e420 0x7f361c12ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.365+0000 7f361a59c700 1 -- 172.21.15.16:0/90621940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f361c09dcd0 msgr2=0x7f361c12a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:11.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.365+0000 7f361a59c700 1 --2- 172.21.15.16:0/90621940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f361c09dcd0 0x7f361c12a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.365+0000 7f361a59c700 1 -- 172.21.15.16:0/90621940 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3614009b30 con 0x7f361c09d1f0 2023-09-30T22:30:11.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.365+0000 7f361ad9d700 1 --2- 172.21.15.16:0/90621940 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f361c09e420 0x7f361c12ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:11.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.365+0000 7f361a59c700 1 --2- 172.21.15.16:0/90621940 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f361c09d1f0 0x7f361c12a080 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3610000f60 tx=0x7f361000c730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:11.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.366+0000 7fb4d67fc700 1 -- 172.21.15.16:0/76658748 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb4d002b020 con 0x7fb4e009e1d0 2023-09-30T22:30:11.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.366+0000 7f360affd700 1 -- 172.21.15.16:0/90621940 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36100052d0 con 0x7f361c09d1f0 2023-09-30T22:30:11.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.366+0000 7f360affd700 1 -- 172.21.15.16:0/90621940 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3610005430 con 0x7f361c09d1f0 2023-09-30T22:30:11.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.366+0000 7f360affd700 1 -- 172.21.15.16:0/90621940 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36100038c0 con 0x7f361c09d1f0 2023-09-30T22:30:11.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.367+0000 7f3620aba700 1 -- 172.21.15.16:0/90621940 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f361c12c180 con 0x7f361c09d1f0 2023-09-30T22:30:11.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.367+0000 7f3620aba700 1 -- 172.21.15.16:0/90621940 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f361c12db60 con 0x7f361c09d1f0 2023-09-30T22:30:11.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.368+0000 7f3620aba700 1 -- 172.21.15.16:0/90621940 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f361c074ea0 con 0x7f361c09d1f0 2023-09-30T22:30:11.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.374+0000 7f360affd700 1 -- 172.21.15.16:0/90621940 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f3610003a20 con 0x7f361c09d1f0 2023-09-30T22:30:11.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.374+0000 7f360affd700 1 -- 172.21.15.16:0/90621940 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7f361003f4b0 con 0x7f361c09d1f0 2023-09-30T22:30:11.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.374+0000 7f3619d9b700 1 -- 172.21.15.16:0/90621940 >> v1:172.21.15.16:6816/126811 conn(0x7f35fc02f510 legacy=0x7f35fc0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:11.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.377+0000 7f360affd700 1 -- 172.21.15.16:0/90621940 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3610019d00 con 0x7f361c09d1f0 2023-09-30T22:30:11.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.397+0000 7f5ae2b6d700 1 Processor -- start 2023-09-30T22:30:11.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.397+0000 7f5ae2b6d700 1 -- start start 2023-09-30T22:30:11.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.398+0000 7f5ae2b6d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5adc1316d0 0x7f5adc131aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:11.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.398+0000 7f5ae2b6d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5adc06ec60 0x7f5adc139740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:11.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.398+0000 7f5ae2b6d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5adc131510 con 0x7f5adc131fe0 2023-09-30T22:30:11.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.398+0000 7f5ae2b6d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5adc099e70 con 0x7f5adc06ec60 2023-09-30T22:30:11.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.398+0000 7f5ae2b6d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5adc066680 con 0x7f5adc1316d0 2023-09-30T22:30:11.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.399+0000 7f5adbfff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5adc131fe0 0x7f5adc06e550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47984/0 (socket says 172.21.15.16:47984) 2023-09-30T22:30:11.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.399+0000 7f5adbfff700 1 -- 172.21.15.16:0/1107935078 learned_addr learned my addr 172.21.15.16:0/1107935078 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:30:11.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.399+0000 7f5ae110a700 1 --2- 172.21.15.16:0/1107935078 >> v2:172.21.15.16:3300/0 conn(0x7f5adc06ec60 0x7f5adc139740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:11.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.399+0000 7f5ae0909700 1 --2- 172.21.15.16:0/1107935078 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5adc1316d0 0x7f5adc131aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:11.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.399+0000 7f5adb7fe700 1 -- 172.21.15.16:0/1107935078 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 396353344 0 0) 0x7f5adc131510 con 0x7f5adc131fe0 2023-09-30T22:30:11.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.399+0000 7f5adb7fe700 1 -- 172.21.15.16:0/1107935078 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5acc0035b0 con 0x7f5adc131fe0 2023-09-30T22:30:11.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.399+0000 7f5ae110a700 1 -- 172.21.15.16:0/1107935078 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5adc1316d0 msgr2=0x7f5adc131aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:11.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.399+0000 7f5ae110a700 1 --2- 172.21.15.16:0/1107935078 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5adc1316d0 0x7f5adc131aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.399+0000 7f5ae110a700 1 -- 172.21.15.16:0/1107935078 >> v1:172.21.15.16:6789/0 conn(0x7f5adc131fe0 legacy=0x7f5adc06e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:11.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.399+0000 7f5ae110a700 1 -- 172.21.15.16:0/1107935078 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5adc09ee00 con 0x7f5adc06ec60 2023-09-30T22:30:11.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.399+0000 7f5ae110a700 1 --2- 172.21.15.16:0/1107935078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5adc06ec60 0x7f5adc139740 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4009d90 tx=0x7f5ad400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16255d13857047c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:11.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.400+0000 7f5adb7fe700 1 -- 172.21.15.16:0/1107935078 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad400e070 con 0x7f5adc06ec60 2023-09-30T22:30:11.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.400+0000 7f5adb7fe700 1 -- 172.21.15.16:0/1107935078 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ad400bb20 con 0x7f5adc06ec60 2023-09-30T22:30:11.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.400+0000 7f5adb7fe700 1 -- 172.21.15.16:0/1107935078 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad4012430 con 0x7f5adc06ec60 2023-09-30T22:30:11.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.401+0000 7f5ae2b6d700 1 -- 172.21.15.16:0/1107935078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5adc06ec60 msgr2=0x7f5adc139740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:11.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.401+0000 7f5ae2b6d700 1 --2- 172.21.15.16:0/1107935078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5adc06ec60 0x7f5adc139740 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5ad4009d90 tx=0x7f5ad400b330 comp rx=0 tx=0).stop 2023-09-30T22:30:11.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.401+0000 7f5ae2b6d700 1 -- 172.21.15.16:0/1107935078 shutdown_connections 2023-09-30T22:30:11.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.401+0000 7f5ae2b6d700 1 --2- 172.21.15.16:0/1107935078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5adc06ec60 0x7f5adc139740 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.401+0000 7f5ae2b6d700 1 --2- 172.21.15.16:0/1107935078 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5adc1316d0 0x7f5adc131aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.401+0000 7f5ae2b6d700 1 -- 172.21.15.16:0/1107935078 >> 172.21.15.16:0/1107935078 conn(0x7f5adc0a2aa0 msgr2=0x7f5adc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:11.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.401+0000 7f5ae2b6d700 1 -- 172.21.15.16:0/1107935078 shutdown_connections 2023-09-30T22:30:11.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.401+0000 7f5ae2b6d700 1 -- 172.21.15.16:0/1107935078 wait complete. 2023-09-30T22:30:11.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.402+0000 7f5ae2b6d700 1 Processor -- start 2023-09-30T22:30:11.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.402+0000 7f5ae2b6d700 1 -- start start 2023-09-30T22:30:11.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.403+0000 7f5ae2b6d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5adc06ec60 0x7f5adc1c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:11.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.403+0000 7f5ae2b6d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5adc1316d0 0x7f5adc1c8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:11.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.403+0000 7f5ae2b6d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5adc131fe0 0x7f5adc1c8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:11.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.403+0000 7f5ae2b6d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5adc12c7a0 con 0x7f5adc1316d0 2023-09-30T22:30:11.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.403+0000 7f5ae2b6d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5adc13a930 con 0x7f5adc06ec60 2023-09-30T22:30:11.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.403+0000 7f5ae2b6d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5adc09a0b0 con 0x7f5adc131fe0 2023-09-30T22:30:11.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.403+0000 7f5ae0909700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5adc06ec60 0x7f5adc1c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:11.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.404+0000 7f5ae0909700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5adc06ec60 0x7f5adc1c7fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41434/0 (socket says 172.21.15.16:41434) 2023-09-30T22:30:11.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.404+0000 7f5ae0909700 1 -- 172.21.15.16:0/1777609543 learned_addr learned my addr 172.21.15.16:0/1777609543 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:11.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.404+0000 7f5adbfff700 1 --2- 172.21.15.16:0/1777609543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5adc1316d0 0x7f5adc1c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:11.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.404+0000 7f5ae110a700 1 --2- 172.21.15.16:0/1777609543 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5adc131fe0 0x7f5adc1c8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:11.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.404+0000 7f5ae0909700 1 -- 172.21.15.16:0/1777609543 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5adc131fe0 msgr2=0x7f5adc1c8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:11.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.404+0000 7f5ae0909700 1 --2- 172.21.15.16:0/1777609543 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5adc131fe0 0x7f5adc1c8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.404+0000 7f5ae0909700 1 -- 172.21.15.16:0/1777609543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5adc1316d0 msgr2=0x7f5adc1c8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:11.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.404+0000 7f5ae0909700 1 --2- 172.21.15.16:0/1777609543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5adc1316d0 0x7f5adc1c8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.404+0000 7f5ae0909700 1 -- 172.21.15.16:0/1777609543 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ad4009a70 con 0x7f5adc06ec60 2023-09-30T22:30:11.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.404+0000 7f5adbfff700 1 --2- 172.21.15.16:0/1777609543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5adc1316d0 0x7f5adc1c8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:11.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.404+0000 7f5ae0909700 1 --2- 172.21.15.16:0/1777609543 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5adc06ec60 0x7f5adc1c7fd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5ad000e9b0 tx=0x7f5ad000ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:11.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.404+0000 7f5ad97fa700 1 -- 172.21.15.16:0/1777609543 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad0009710 con 0x7f5adc06ec60 2023-09-30T22:30:11.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.404+0000 7f5ad97fa700 1 -- 172.21.15.16:0/1777609543 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ad000ee50 con 0x7f5adc06ec60 2023-09-30T22:30:11.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.405+0000 7f5ae2b6d700 1 -- 172.21.15.16:0/1777609543 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5adc12bfd0 con 0x7f5adc06ec60 2023-09-30T22:30:11.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.405+0000 7f5ad97fa700 1 -- 172.21.15.16:0/1777609543 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad000ca50 con 0x7f5adc06ec60 2023-09-30T22:30:11.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.405+0000 7f5ae2b6d700 1 -- 172.21.15.16:0/1777609543 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5adc12e9c0 con 0x7f5adc06ec60 2023-09-30T22:30:11.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.406+0000 7f5ad97fa700 1 -- 172.21.15.16:0/1777609543 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f5ad000cbb0 con 0x7f5adc06ec60 2023-09-30T22:30:11.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.406+0000 7f5ae2b6d700 1 -- 172.21.15.16:0/1777609543 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5abc0052e0 con 0x7f5adc06ec60 2023-09-30T22:30:11.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.406+0000 7f5adbfff700 1 -- 172.21.15.16:0/1777609543 >> v1:172.21.15.16:6816/126811 conn(0x7f5ac002f510 legacy=0x7f5ac00319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:11.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.406+0000 7f5ad97fa700 1 -- 172.21.15.16:0/1777609543 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7f5ad003fba0 con 0x7f5adc06ec60 2023-09-30T22:30:11.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.415+0000 7f5ad97fa700 1 -- 172.21.15.16:0/1777609543 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5ad00232f0 con 0x7f5adc06ec60 2023-09-30T22:30:11.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.448+0000 7ff1458f3700 1 Processor -- start 2023-09-30T22:30:11.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.448+0000 7ff1458f3700 1 -- start start 2023-09-30T22:30:11.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.449+0000 7ff1458f3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff14009d9f0 0x7ff140136530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:11.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.449+0000 7ff1458f3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff140136a70 0x7ff140138e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:11.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.449+0000 7ff1458f3700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff14009e130 con 0x7ff1401316c0 2023-09-30T22:30:11.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.449+0000 7ff1458f3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1401321a0 con 0x7ff14009d9f0 2023-09-30T22:30:11.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.449+0000 7ff1458f3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff1401312b0 con 0x7ff140136a70 2023-09-30T22:30:11.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.449+0000 7ff13effd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff1401316c0 0x7ff140131a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47990/0 (socket says 172.21.15.16:47990) 2023-09-30T22:30:11.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.449+0000 7ff13effd700 1 -- 172.21.15.16:0/3771453573 learned_addr learned my addr 172.21.15.16:0/3771453573 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:30:11.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.449+0000 7ff13f7fe700 1 --2- 172.21.15.16:0/3771453573 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff140136a70 0x7ff140138e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:11.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.449+0000 7ff13e7fc700 1 --2- 172.21.15.16:0/3771453573 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff14009d9f0 0x7ff140136530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:11.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.449+0000 7ff13dffb700 1 -- 172.21.15.16:0/3771453573 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 835642751 0 0) 0x7ff130001110 con 0x7ff1401316c0 2023-09-30T22:30:11.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.449+0000 7ff13dffb700 1 -- 172.21.15.16:0/3771453573 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1280035b0 con 0x7ff1401316c0 2023-09-30T22:30:11.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.449+0000 7ff13f7fe700 1 -- 172.21.15.16:0/3771453573 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff14009d9f0 msgr2=0x7ff140136530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:11.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.449+0000 7ff13f7fe700 1 --2- 172.21.15.16:0/3771453573 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff14009d9f0 0x7ff140136530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.449+0000 7ff13f7fe700 1 -- 172.21.15.16:0/3771453573 >> v1:172.21.15.16:6789/0 conn(0x7ff1401316c0 legacy=0x7ff140131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:11.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.449+0000 7ff13f7fe700 1 -- 172.21.15.16:0/3771453573 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff14009f6e0 con 0x7ff140136a70 2023-09-30T22:30:11.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.450+0000 7ff13f7fe700 1 --2- 172.21.15.16:0/3771453573 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff140136a70 0x7ff140138e50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff138009df0 tx=0x7ff13800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb5c38d139bf22f8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:11.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.450+0000 7ff13dffb700 1 -- 172.21.15.16:0/3771453573 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13800e070 con 0x7ff140136a70 2023-09-30T22:30:11.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.450+0000 7ff13dffb700 1 -- 172.21.15.16:0/3771453573 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff13800bae0 con 0x7ff140136a70 2023-09-30T22:30:11.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.450+0000 7ff13dffb700 1 -- 172.21.15.16:0/3771453573 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13800be90 con 0x7ff140136a70 2023-09-30T22:30:11.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.451+0000 7ff1458f3700 1 -- 172.21.15.16:0/3771453573 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff140136a70 msgr2=0x7ff140138e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:11.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.451+0000 7ff1458f3700 1 --2- 172.21.15.16:0/3771453573 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff140136a70 0x7ff140138e50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff138009df0 tx=0x7ff13800b330 comp rx=0 tx=0).stop 2023-09-30T22:30:11.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.451+0000 7ff1458f3700 1 -- 172.21.15.16:0/3771453573 shutdown_connections 2023-09-30T22:30:11.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.451+0000 7ff1458f3700 1 --2- 172.21.15.16:0/3771453573 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff140136a70 0x7ff140138e50 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.451+0000 7ff1458f3700 1 --2- 172.21.15.16:0/3771453573 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff14009d9f0 0x7ff140136530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.451+0000 7ff1458f3700 1 -- 172.21.15.16:0/3771453573 >> 172.21.15.16:0/3771453573 conn(0x7ff140066f50 msgr2=0x7ff14009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:11.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.451+0000 7ff1458f3700 1 -- 172.21.15.16:0/3771453573 shutdown_connections 2023-09-30T22:30:11.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.451+0000 7ff1458f3700 1 -- 172.21.15.16:0/3771453573 wait complete. 2023-09-30T22:30:11.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.452+0000 7ff1458f3700 1 Processor -- start 2023-09-30T22:30:11.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.453+0000 7ff1458f3700 1 -- start start 2023-09-30T22:30:11.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.453+0000 7ff1458f3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff14009d9f0 0x7ff140134a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:11.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.453+0000 7ff1458f3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1401316c0 0x7ff140134f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:11.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.453+0000 7ff1458f3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff140136a70 0x7ff140135490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:11.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.453+0000 7ff1458f3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff140133940 con 0x7ff1401316c0 2023-09-30T22:30:11.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.454+0000 7ff1458f3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1400725f0 con 0x7ff140136a70 2023-09-30T22:30:11.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.454+0000 7ff1458f3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff140133580 con 0x7ff14009d9f0 2023-09-30T22:30:11.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.454+0000 7ff13f7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff140136a70 0x7ff140135490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:11.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.454+0000 7ff13f7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff140136a70 0x7ff140135490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41458/0 (socket says 172.21.15.16:41458) 2023-09-30T22:30:11.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.454+0000 7ff13f7fe700 1 -- 172.21.15.16:0/3698520975 learned_addr learned my addr 172.21.15.16:0/3698520975 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:11.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.454+0000 7ff13e7fc700 1 --2- 172.21.15.16:0/3698520975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1401316c0 0x7ff140134f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:11.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.454+0000 7ff13effd700 1 --2- 172.21.15.16:0/3698520975 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff14009d9f0 0x7ff140134a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:11.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.454+0000 7ff13f7fe700 1 -- 172.21.15.16:0/3698520975 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff14009d9f0 msgr2=0x7ff140134a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:11.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.454+0000 7ff13f7fe700 1 --2- 172.21.15.16:0/3698520975 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff14009d9f0 0x7ff140134a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.454+0000 7ff13f7fe700 1 -- 172.21.15.16:0/3698520975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1401316c0 msgr2=0x7ff140134f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:11.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.454+0000 7ff13f7fe700 1 --2- 172.21.15.16:0/3698520975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1401316c0 0x7ff140134f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.454+0000 7ff13f7fe700 1 -- 172.21.15.16:0/3698520975 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff138009a70 con 0x7ff140136a70 2023-09-30T22:30:11.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.454+0000 7ff13e7fc700 1 --2- 172.21.15.16:0/3698520975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1401316c0 0x7ff140134f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:30:11.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.455+0000 7ff13f7fe700 1 --2- 172.21.15.16:0/3698520975 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff140136a70 0x7ff140135490 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff138013000 tx=0x7ff138004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:11.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.455+0000 7ff127fff700 1 -- 172.21.15.16:0/3698520975 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff138004730 con 0x7ff140136a70 2023-09-30T22:30:11.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.455+0000 7ff127fff700 1 -- 172.21.15.16:0/3698520975 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff138020070 con 0x7ff140136a70 2023-09-30T22:30:11.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.455+0000 7ff1458f3700 1 -- 172.21.15.16:0/3698520975 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff140133e30 con 0x7ff140136a70 2023-09-30T22:30:11.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.455+0000 7ff127fff700 1 -- 172.21.15.16:0/3698520975 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff138027480 con 0x7ff140136a70 2023-09-30T22:30:11.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.455+0000 7ff1458f3700 1 -- 172.21.15.16:0/3698520975 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1401345c0 con 0x7ff140136a70 2023-09-30T22:30:11.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.456+0000 7ff127fff700 1 -- 172.21.15.16:0/3698520975 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7ff138012430 con 0x7ff140136a70 2023-09-30T22:30:11.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.456+0000 7ff127fff700 1 -- 172.21.15.16:0/3698520975 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7ff138022070 con 0x7ff140136a70 2023-09-30T22:30:11.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.456+0000 7ff13effd700 1 -- 172.21.15.16:0/3698520975 >> v1:172.21.15.16:6816/126811 conn(0x7ff12c02f510 legacy=0x7ff12c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:11.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.456+0000 7ff1458f3700 1 -- 172.21.15.16:0/3698520975 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff140074ea0 con 0x7ff140136a70 2023-09-30T22:30:11.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.466+0000 7ff127fff700 1 -- 172.21.15.16:0/3698520975 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff138046060 con 0x7ff140136a70 2023-09-30T22:30:11.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.562+0000 7fb4e592c700 1 -- 172.21.15.16:0/76658748 >> v1:172.21.15.16:6816/126811 conn(0x7fb4c402f510 legacy=0x7fb4c40319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:11.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.575+0000 7f3619d9b700 1 -- 172.21.15.16:0/90621940 >> v1:172.21.15.16:6816/126811 conn(0x7f35fc02f510 legacy=0x7f35fc0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:11.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.588+0000 7fb4e7b90700 1 -- 172.21.15.16:0/76658748 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb4e00a3ab0 con 0x7fb4e009e1d0 2023-09-30T22:30:11.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.588+0000 7fb4d67fc700 1 -- 172.21.15.16:0/76658748 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb4d0015070 con 0x7fb4e009e1d0 2023-09-30T22:30:11.590 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:30:11.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.591+0000 7fb4e7b90700 1 -- 172.21.15.16:0/76658748 >> v1:172.21.15.16:6816/126811 conn(0x7fb4c402f510 legacy=0x7fb4c40319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:30:11.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.591+0000 7fb4e7b90700 1 -- 172.21.15.16:0/76658748 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb4e009e1d0 msgr2=0x7fb4e011b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:11.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.591+0000 7fb4e7b90700 1 --2- 172.21.15.16:0/76658748 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb4e009e1d0 0x7fb4e011b990 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb4d0016000 tx=0x7fb4d0004750 comp rx=0 tx=0).stop 2023-09-30T22:30:11.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.591+0000 7fb4e7b90700 1 -- 172.21.15.16:0/76658748 shutdown_connections 2023-09-30T22:30:11.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.591+0000 7fb4e7b90700 1 --2- 172.21.15.16:0/76658748 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb4e0135e50 0x7fb4e011bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.591+0000 7fb4e7b90700 1 --2- 172.21.15.16:0/76658748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e009d6f0 0x7fb4e011b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.591+0000 7fb4e7b90700 1 --2- 172.21.15.16:0/76658748 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb4e009e1d0 0x7fb4e011b990 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.591+0000 7fb4e7b90700 1 -- 172.21.15.16:0/76658748 >> 172.21.15.16:0/76658748 conn(0x7fb4e0066f50 msgr2=0x7fb4e0133790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:11.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.591+0000 7fb4e7b90700 1 -- 172.21.15.16:0/76658748 shutdown_connections 2023-09-30T22:30:11.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.591+0000 7fb4e7b90700 1 -- 172.21.15.16:0/76658748 wait complete. 2023-09-30T22:30:11.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.593+0000 7f3620aba700 1 -- 172.21.15.16:0/90621940 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f361c09e830 con 0x7f361c09d1f0 2023-09-30T22:30:11.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.593+0000 7f360affd700 1 -- 172.21.15.16:0/90621940 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3610017030 con 0x7f361c09d1f0 2023-09-30T22:30:11.594 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:30:11.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.597+0000 7f3620aba700 1 -- 172.21.15.16:0/90621940 >> v1:172.21.15.16:6816/126811 conn(0x7f35fc02f510 legacy=0x7f35fc0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:30:11.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.597+0000 7f3620aba700 1 -- 172.21.15.16:0/90621940 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f361c09d1f0 msgr2=0x7f361c12a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:11.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.597+0000 7f3620aba700 1 --2- 172.21.15.16:0/90621940 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f361c09d1f0 0x7f361c12a080 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3610000f60 tx=0x7f361000c730 comp rx=0 tx=0).stop 2023-09-30T22:30:11.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.597+0000 7f3620aba700 1 -- 172.21.15.16:0/90621940 shutdown_connections 2023-09-30T22:30:11.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.597+0000 7f3620aba700 1 --2- 172.21.15.16:0/90621940 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f361c09e420 0x7f361c12ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.597+0000 7f3620aba700 1 --2- 172.21.15.16:0/90621940 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f361c09d1f0 0x7f361c12a080 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.597+0000 7f3620aba700 1 --2- 172.21.15.16:0/90621940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f361c09dcd0 0x7f361c12a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.597+0000 7f3620aba700 1 -- 172.21.15.16:0/90621940 >> 172.21.15.16:0/90621940 conn(0x7f361c0a2360 msgr2=0x7f361c0a0080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:11.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.597+0000 7f3620aba700 1 -- 172.21.15.16:0/90621940 shutdown_connections 2023-09-30T22:30:11.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.597+0000 7f3620aba700 1 -- 172.21.15.16:0/90621940 wait complete. 2023-09-30T22:30:11.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.607+0000 7f5adbfff700 1 -- 172.21.15.16:0/1777609543 >> v1:172.21.15.16:6816/126811 conn(0x7f5ac002f510 legacy=0x7f5ac00319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:11.613 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:30:11.616 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:30:11.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.657+0000 7ff13effd700 1 -- 172.21.15.16:0/3698520975 >> v1:172.21.15.16:6816/126811 conn(0x7ff12c02f510 legacy=0x7ff12c0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:11.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.660+0000 7f5ae2b6d700 1 -- 172.21.15.16:0/1777609543 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5abc005860 con 0x7f5adc06ec60 2023-09-30T22:30:11.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.660+0000 7f5ad97fa700 1 -- 172.21.15.16:0/1777609543 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5ad001ae80 con 0x7f5adc06ec60 2023-09-30T22:30:11.661 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:30:11.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.662+0000 7f5ae2b6d700 1 -- 172.21.15.16:0/1777609543 >> v1:172.21.15.16:6816/126811 conn(0x7f5ac002f510 legacy=0x7f5ac00319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:30:11.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.662+0000 7f5ae2b6d700 1 -- 172.21.15.16:0/1777609543 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5adc06ec60 msgr2=0x7f5adc1c7fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:11.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.662+0000 7f5ae2b6d700 1 --2- 172.21.15.16:0/1777609543 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5adc06ec60 0x7f5adc1c7fd0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5ad000e9b0 tx=0x7f5ad000ecc0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.662+0000 7f5ae2b6d700 1 -- 172.21.15.16:0/1777609543 shutdown_connections 2023-09-30T22:30:11.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.663+0000 7f5ae2b6d700 1 --2- 172.21.15.16:0/1777609543 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5adc06ec60 0x7f5adc1c7fd0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.663+0000 7f5ae2b6d700 1 --2- 172.21.15.16:0/1777609543 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5adc131fe0 0x7f5adc1c8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.663+0000 7f5ae2b6d700 1 --2- 172.21.15.16:0/1777609543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5adc1316d0 0x7f5adc1c8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.663+0000 7f5ae2b6d700 1 -- 172.21.15.16:0/1777609543 >> 172.21.15.16:0/1777609543 conn(0x7f5adc0a2aa0 msgr2=0x7f5adc12f300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:11.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.663+0000 7f5ae2b6d700 1 -- 172.21.15.16:0/1777609543 shutdown_connections 2023-09-30T22:30:11.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.663+0000 7f5ae2b6d700 1 -- 172.21.15.16:0/1777609543 wait complete. 2023-09-30T22:30:11.678 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:30:11.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.696+0000 7ff1458f3700 1 -- 172.21.15.16:0/3698520975 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff14009e980 con 0x7ff140136a70 2023-09-30T22:30:11.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.696+0000 7ff127fff700 1 -- 172.21.15.16:0/3698520975 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff13801a970 con 0x7ff140136a70 2023-09-30T22:30:11.698 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:30:11.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.699+0000 7ff1458f3700 1 -- 172.21.15.16:0/3698520975 >> v1:172.21.15.16:6816/126811 conn(0x7ff12c02f510 legacy=0x7ff12c0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:30:11.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.699+0000 7ff1458f3700 1 -- 172.21.15.16:0/3698520975 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff140136a70 msgr2=0x7ff140135490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:11.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.699+0000 7ff1458f3700 1 --2- 172.21.15.16:0/3698520975 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff140136a70 0x7ff140135490 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff138013000 tx=0x7ff138004ad0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.699+0000 7ff1458f3700 1 -- 172.21.15.16:0/3698520975 shutdown_connections 2023-09-30T22:30:11.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.699+0000 7ff1458f3700 1 --2- 172.21.15.16:0/3698520975 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff140136a70 0x7ff140135490 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.699+0000 7ff1458f3700 1 --2- 172.21.15.16:0/3698520975 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff14009d9f0 0x7ff140134a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.699+0000 7ff1458f3700 1 --2- 172.21.15.16:0/3698520975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1401316c0 0x7ff140134f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:11.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.699+0000 7ff1458f3700 1 -- 172.21.15.16:0/3698520975 >> 172.21.15.16:0/3698520975 conn(0x7ff140066f50 msgr2=0x7ff140136e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:11.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.699+0000 7ff1458f3700 1 -- 172.21.15.16:0/3698520975 shutdown_connections 2023-09-30T22:30:11.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:11.699+0000 7ff1458f3700 1 -- 172.21.15.16:0/3698520975 wait complete. 2023-09-30T22:30:11.714 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:30:12.613 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:12.618 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:12.679 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:12.715 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:12.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.766+0000 7f25a9f44700 1 Processor -- start 2023-09-30T22:30:12.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.767+0000 7f25a9f44700 1 -- start start 2023-09-30T22:30:12.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.767+0000 7f25a9f44700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f25a409e930 0x7f25a409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:12.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.767+0000 7f25a9f44700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f25a409f240 0x7f25a41317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:12.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.767+0000 7f25a9f44700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25a4070940 con 0x7f25a409f650 2023-09-30T22:30:12.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.767+0000 7f25a9f44700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25a406fd60 con 0x7f25a409f240 2023-09-30T22:30:12.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.767+0000 7f25a9f44700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25a406fa80 con 0x7f25a409e930 2023-09-30T22:30:12.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.768+0000 7f25a3fff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f25a409f650 0x7f25a4131d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48000/0 (socket says 172.21.15.16:48000) 2023-09-30T22:30:12.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.768+0000 7f25a3fff700 1 -- 172.21.15.16:0/1209196043 learned_addr learned my addr 172.21.15.16:0/1209196043 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:30:12.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.768+0000 7f25a2ffd700 1 --2- 172.21.15.16:0/1209196043 >> v2:172.21.15.16:3300/0 conn(0x7f25a409f240 0x7f25a41317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:12.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.768+0000 7f25a37fe700 1 --2- 172.21.15.16:0/1209196043 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f25a409e930 0x7f25a409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:12.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.768+0000 7f25a2ffd700 1 -- 172.21.15.16:0/1209196043 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f25a409e930 msgr2=0x7f25a409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:12.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.768+0000 7f25a2ffd700 1 --2- 172.21.15.16:0/1209196043 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f25a409e930 0x7f25a409ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:12.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.768+0000 7f25a2ffd700 1 -- 172.21.15.16:0/1209196043 >> v1:172.21.15.16:6789/0 conn(0x7f25a409f650 legacy=0x7f25a4131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:12.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.768+0000 7f25a2ffd700 1 -- 172.21.15.16:0/1209196043 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a409fb60 con 0x7f25a409f240 2023-09-30T22:30:12.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.768+0000 7f25a27fc700 1 -- 172.21.15.16:0/1209196043 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3336593183 0 0) 0x7f25a4070940 con 0x7f25a409f650 2023-09-30T22:30:12.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.768+0000 7f25a2ffd700 1 --2- 172.21.15.16:0/1209196043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25a409f240 0x7f25a41317f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2598009840 tx=0x7f2598009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=229af42acbbcfa94 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:12.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.768+0000 7f25a27fc700 1 -- 172.21.15.16:0/1209196043 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2598011830 con 0x7f25a409f240 2023-09-30T22:30:12.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.769+0000 7f25a27fc700 1 -- 172.21.15.16:0/1209196043 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2598011990 con 0x7f25a409f240 2023-09-30T22:30:12.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.769+0000 7f25a27fc700 1 -- 172.21.15.16:0/1209196043 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2598011d20 con 0x7f25a409f240 2023-09-30T22:30:12.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.770+0000 7f25a9f44700 1 -- 172.21.15.16:0/1209196043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25a409f240 msgr2=0x7f25a41317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:12.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.770+0000 7f25a9f44700 1 --2- 172.21.15.16:0/1209196043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25a409f240 0x7f25a41317f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2598009840 tx=0x7f2598009b50 comp rx=0 tx=0).stop 2023-09-30T22:30:12.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.770+0000 7f25a9f44700 1 -- 172.21.15.16:0/1209196043 shutdown_connections 2023-09-30T22:30:12.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.770+0000 7f25a9f44700 1 --2- 172.21.15.16:0/1209196043 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f25a409e930 0x7f25a409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:12.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.770+0000 7f25a9f44700 1 --2- 172.21.15.16:0/1209196043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25a409f240 0x7f25a41317f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:12.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.770+0000 7f25a9f44700 1 -- 172.21.15.16:0/1209196043 >> 172.21.15.16:0/1209196043 conn(0x7f25a40a4210 msgr2=0x7f25a40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:12.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.770+0000 7f25a9f44700 1 -- 172.21.15.16:0/1209196043 shutdown_connections 2023-09-30T22:30:12.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.770+0000 7f25a9f44700 1 -- 172.21.15.16:0/1209196043 wait complete. 2023-09-30T22:30:12.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.771+0000 7f25a9f44700 1 Processor -- start 2023-09-30T22:30:12.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.772+0000 7f25a9f44700 1 -- start start 2023-09-30T22:30:12.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.772+0000 7f25a9f44700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f25a409e930 0x7f25a41c6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:12.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.772+0000 7f25a9f44700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f25a409f240 0x7f25a41c6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:12.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.772+0000 7f25a9f44700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25a409f650 0x7f25a41c27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:12.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.772+0000 7f25a9f44700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a41a72d0 con 0x7f25a409f650 2023-09-30T22:30:12.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.772+0000 7f25a9f44700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25a40a3260 con 0x7f25a409f240 2023-09-30T22:30:12.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.772+0000 7f25a9f44700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f25a41c6ca0 con 0x7f25a409e930 2023-09-30T22:30:12.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.772+0000 7f25a2ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f25a409f240 0x7f25a41c6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:12.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.773+0000 7f25a2ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f25a409f240 0x7f25a41c6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41466/0 (socket says 172.21.15.16:41466) 2023-09-30T22:30:12.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.773+0000 7f25a2ffd700 1 -- 172.21.15.16:0/288497948 learned_addr learned my addr 172.21.15.16:0/288497948 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:12.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.773+0000 7f25a3fff700 1 --2- 172.21.15.16:0/288497948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25a409f650 0x7f25a41c27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:12.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.773+0000 7f25a37fe700 1 --2- 172.21.15.16:0/288497948 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f25a409e930 0x7f25a41c6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:12.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.773+0000 7f25a2ffd700 1 -- 172.21.15.16:0/288497948 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f25a409e930 msgr2=0x7f25a41c6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:12.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.773+0000 7f25a2ffd700 1 --2- 172.21.15.16:0/288497948 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f25a409e930 0x7f25a41c6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:12.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.773+0000 7f25a2ffd700 1 -- 172.21.15.16:0/288497948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25a409f650 msgr2=0x7f25a41c27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:12.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.773+0000 7f25a2ffd700 1 --2- 172.21.15.16:0/288497948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25a409f650 0x7f25a41c27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:12.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.773+0000 7f25a2ffd700 1 -- 172.21.15.16:0/288497948 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25980094a0 con 0x7f25a409f240 2023-09-30T22:30:12.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.773+0000 7f25a3fff700 1 --2- 172.21.15.16:0/288497948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25a409f650 0x7f25a41c27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:12.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.773+0000 7f25a2ffd700 1 --2- 172.21.15.16:0/288497948 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f25a409f240 0x7f25a41c6760 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f25980090b0 tx=0x7f2598004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:12.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.773+0000 7f258bfff700 1 -- 172.21.15.16:0/288497948 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2598004550 con 0x7f25a409f240 2023-09-30T22:30:12.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.774+0000 7f258bfff700 1 -- 172.21.15.16:0/288497948 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f259801f070 con 0x7f25a409f240 2023-09-30T22:30:12.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.774+0000 7f258bfff700 1 -- 172.21.15.16:0/288497948 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25980038e0 con 0x7f25a409f240 2023-09-30T22:30:12.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.774+0000 7f25a9f44700 1 -- 172.21.15.16:0/288497948 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25a40a0190 con 0x7f25a409f240 2023-09-30T22:30:12.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.774+0000 7f25a9f44700 1 -- 172.21.15.16:0/288497948 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25a4071d20 con 0x7f25a409f240 2023-09-30T22:30:12.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.775+0000 7f258bfff700 1 -- 172.21.15.16:0/288497948 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7f2598003a40 con 0x7f25a409f240 2023-09-30T22:30:12.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.775+0000 7f25a9f44700 1 -- 172.21.15.16:0/288497948 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25a41a5700 con 0x7f25a409f240 2023-09-30T22:30:12.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.775+0000 7f258bfff700 1 -- 172.21.15.16:0/288497948 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7f2598021070 con 0x7f25a409f240 2023-09-30T22:30:12.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.775+0000 7f25a37fe700 1 -- 172.21.15.16:0/288497948 >> v1:172.21.15.16:6816/126811 conn(0x7f259002f510 legacy=0x7f25900319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:12.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.784+0000 7f258bfff700 1 -- 172.21.15.16:0/288497948 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f259802b080 con 0x7f25a409f240 2023-09-30T22:30:12.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.821+0000 7fc3e9dae700 1 Processor -- start 2023-09-30T22:30:12.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.821+0000 7fc3e9dae700 1 -- start start 2023-09-30T22:30:12.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.822+0000 7fc3e9dae700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc3e409e930 0x7fc3e409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:12.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.822+0000 7fc3e9dae700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc3e409f240 0x7fc3e41317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:12.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.822+0000 7fc3e9dae700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3e4070940 con 0x7fc3e409f650 2023-09-30T22:30:12.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.822+0000 7fc3e9dae700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3e406fd60 con 0x7fc3e409f240 2023-09-30T22:30:12.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.822+0000 7fc3e9dae700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e406fa80 con 0x7fc3e409e930 2023-09-30T22:30:12.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.822+0000 7fc3e37fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc3e409e930 0x7fc3e409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:12.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.822+0000 7fc3e2ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc3e409f240 0x7fc3e41317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:12.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.822+0000 7fc3e37fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc3e409e930 0x7fc3e409ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:50982/0 (socket says 172.21.15.16:50982) 2023-09-30T22:30:12.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.822+0000 7fc3e37fe700 1 -- 172.21.15.16:0/37840736 learned_addr learned my addr 172.21.15.16:0/37840736 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:12.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.822+0000 7fc3e37fe700 1 -- 172.21.15.16:0/37840736 >> v2:172.21.15.16:3300/0 conn(0x7fc3e409f240 msgr2=0x7fc3e41317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:12.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.822+0000 7fc3e37fe700 1 --2- 172.21.15.16:0/37840736 >> v2:172.21.15.16:3300/0 conn(0x7fc3e409f240 0x7fc3e41317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:12.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.822+0000 7fc3e37fe700 1 -- 172.21.15.16:0/37840736 >> v1:172.21.15.16:6789/0 conn(0x7fc3e409f650 legacy=0x7fc3e4131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:12.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.822+0000 7fc3e37fe700 1 -- 172.21.15.16:0/37840736 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3e409fb60 con 0x7fc3e409e930 2023-09-30T22:30:12.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.823+0000 7fc3e37fe700 1 --2- 172.21.15.16:0/37840736 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc3e409e930 0x7fc3e409ed00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc3d400dd10 tx=0x7fc3d400f330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53668b3d94e6146c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:12.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.823+0000 7fc3e27fc700 1 -- 172.21.15.16:0/37840736 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d4012070 con 0x7fc3e409e930 2023-09-30T22:30:12.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.823+0000 7fc3e27fc700 1 -- 172.21.15.16:0/37840736 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3d400d2e0 con 0x7fc3e409e930 2023-09-30T22:30:12.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.823+0000 7fc3e27fc700 1 -- 172.21.15.16:0/37840736 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d400fb10 con 0x7fc3e409e930 2023-09-30T22:30:12.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.824+0000 7fc3e9dae700 1 -- 172.21.15.16:0/37840736 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc3e409e930 msgr2=0x7fc3e409ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:12.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.824+0000 7fc3e9dae700 1 --2- 172.21.15.16:0/37840736 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc3e409e930 0x7fc3e409ed00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc3d400dd10 tx=0x7fc3d400f330 comp rx=0 tx=0).stop 2023-09-30T22:30:12.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.824+0000 7fc3e9dae700 1 -- 172.21.15.16:0/37840736 shutdown_connections 2023-09-30T22:30:12.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.824+0000 7fc3e9dae700 1 --2- 172.21.15.16:0/37840736 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc3e409e930 0x7fc3e409ed00 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:12.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.824+0000 7fc3e9dae700 1 --2- 172.21.15.16:0/37840736 >> v2:172.21.15.16:3300/0 conn(0x7fc3e409f240 0x7fc3e41317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:12.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.824+0000 7fc3e9dae700 1 -- 172.21.15.16:0/37840736 >> 172.21.15.16:0/37840736 conn(0x7fc3e40a4210 msgr2=0x7fc3e40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:12.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.824+0000 7fc3e9dae700 1 -- 172.21.15.16:0/37840736 shutdown_connections 2023-09-30T22:30:12.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.824+0000 7fc3e9dae700 1 -- 172.21.15.16:0/37840736 wait complete. 2023-09-30T22:30:12.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.825+0000 7fc3e9dae700 1 Processor -- start 2023-09-30T22:30:12.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.826+0000 7fc3e9dae700 1 -- start start 2023-09-30T22:30:12.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.826+0000 7fc3e9dae700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e409e930 0x7fc3e411d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:12.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.826+0000 7fc3e9dae700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc3e409f240 0x7fc3e411d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:12.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.826+0000 7fc3e9dae700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc3e409f650 0x7fc3e411db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:12.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.826+0000 7fc3e9dae700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e41209b0 con 0x7fc3e409e930 2023-09-30T22:30:12.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.826+0000 7fc3e9dae700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e41304c0 con 0x7fc3e409f240 2023-09-30T22:30:12.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.826+0000 7fc3e9dae700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e411ff30 con 0x7fc3e409f650 2023-09-30T22:30:12.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.826+0000 7fc3e3fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc3e409f650 0x7fc3e411db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:12.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.827+0000 7fc3e3fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc3e409f650 0x7fc3e411db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:50988/0 (socket says 172.21.15.16:50988) 2023-09-30T22:30:12.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.827+0000 7fc3e3fff700 1 -- 172.21.15.16:0/2064427510 learned_addr learned my addr 172.21.15.16:0/2064427510 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:12.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.827+0000 7fc3e37fe700 1 --2- 172.21.15.16:0/2064427510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e409e930 0x7fc3e411d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:12.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.827+0000 7fc3e2ffd700 1 --2- 172.21.15.16:0/2064427510 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc3e409f240 0x7fc3e411d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:12.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.827+0000 7fc3e3fff700 1 -- 172.21.15.16:0/2064427510 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc3e409f240 msgr2=0x7fc3e411d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:12.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.827+0000 7fc3e3fff700 1 --2- 172.21.15.16:0/2064427510 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc3e409f240 0x7fc3e411d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:12.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.827+0000 7fc3e3fff700 1 -- 172.21.15.16:0/2064427510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e409e930 msgr2=0x7fc3e411d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:12.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.827+0000 7fc3e3fff700 1 --2- 172.21.15.16:0/2064427510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e409e930 0x7fc3e411d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:12.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.827+0000 7fc3e3fff700 1 -- 172.21.15.16:0/2064427510 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3d400d9f0 con 0x7fc3e409f650 2023-09-30T22:30:12.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.827+0000 7fc3e37fe700 1 --2- 172.21.15.16:0/2064427510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e409e930 0x7fc3e411d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:12.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.827+0000 7fc3e3fff700 1 --2- 172.21.15.16:0/2064427510 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc3e409f650 0x7fc3e411db50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc00cc80 tx=0x7fc3dc00cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:12.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.827+0000 7fc3cbfff700 1 -- 172.21.15.16:0/2064427510 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc003c00 con 0x7fc3e409f650 2023-09-30T22:30:12.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.827+0000 7fc3cbfff700 1 -- 172.21.15.16:0/2064427510 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3dc003d60 con 0x7fc3e409f650 2023-09-30T22:30:12.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.828+0000 7fc3cbfff700 1 -- 172.21.15.16:0/2064427510 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc00aaa0 con 0x7fc3e409f650 2023-09-30T22:30:12.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.828+0000 7fc3e9dae700 1 -- 172.21.15.16:0/2064427510 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3e40a0190 con 0x7fc3e409f650 2023-09-30T22:30:12.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.828+0000 7fc3e9dae700 1 -- 172.21.15.16:0/2064427510 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3e40a3040 con 0x7fc3e409f650 2023-09-30T22:30:12.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.829+0000 7fc3cbfff700 1 -- 172.21.15.16:0/2064427510 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fc3dc00ac00 con 0x7fc3e409f650 2023-09-30T22:30:12.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.829+0000 7fc3e9dae700 1 -- 172.21.15.16:0/2064427510 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3e4074ea0 con 0x7fc3e409f650 2023-09-30T22:30:12.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.830+0000 7fc3cbfff700 1 -- 172.21.15.16:0/2064427510 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7fc3dc03fb20 con 0x7fc3e409f650 2023-09-30T22:30:12.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.830+0000 7fc3e37fe700 1 -- 172.21.15.16:0/2064427510 >> v1:172.21.15.16:6816/126811 conn(0x7fc3d002f510 legacy=0x7fc3d00319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:12.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.835+0000 7fc3cbfff700 1 -- 172.21.15.16:0/2064427510 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc3dc0198f0 con 0x7fc3e409f650 2023-09-30T22:30:12.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.859+0000 7faccf0e2700 1 Processor -- start 2023-09-30T22:30:12.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.859+0000 7faccf0e2700 1 -- start start 2023-09-30T22:30:12.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.859+0000 7faccf0e2700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7facc809d6f0 0x7facc809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:12.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.859+0000 7faccf0e2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7facc809e000 0x7facc8130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:12.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.859+0000 7faccf0e2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7facc809e410 0x7facc8130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:12.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.859+0000 7faccf0e2700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facc809f520 con 0x7facc809d6f0 2023-09-30T22:30:12.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.859+0000 7faccf0e2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facc809f130 con 0x7facc809e000 2023-09-30T22:30:12.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.859+0000 7faccf0e2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7facc809e930 con 0x7facc809e410 2023-09-30T22:30:12.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.859+0000 7facc7fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7facc809e000 0x7facc8130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:12.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.859+0000 7facc7fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7facc809e000 0x7facc8130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41490/0 (socket says 172.21.15.16:41490) 2023-09-30T22:30:12.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.859+0000 7facc7fff700 1 -- 172.21.15.16:0/4100818284 learned_addr learned my addr 172.21.15.16:0/4100818284 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:12.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.859+0000 7faccce7e700 1 --2- 172.21.15.16:0/4100818284 >> v2:172.21.15.16:3300/0 conn(0x7facc809d6f0 0x7facc809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:12.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.859+0000 7faccd67f700 1 --2- 172.21.15.16:0/4100818284 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7facc809e410 0x7facc8130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:12.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.860+0000 7facc7fff700 1 -- 172.21.15.16:0/4100818284 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7facc809e410 msgr2=0x7facc8130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:12.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.860+0000 7facc7fff700 1 --2- 172.21.15.16:0/4100818284 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7facc809e410 0x7facc8130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:12.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.860+0000 7facc7fff700 1 -- 172.21.15.16:0/4100818284 >> v2:172.21.15.16:3300/0 conn(0x7facc809d6f0 msgr2=0x7facc809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:12.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.860+0000 7facc7fff700 1 --2- 172.21.15.16:0/4100818284 >> v2:172.21.15.16:3300/0 conn(0x7facc809d6f0 0x7facc809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:12.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.860+0000 7facc7fff700 1 -- 172.21.15.16:0/4100818284 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facc809b2d0 con 0x7facc809e000 2023-09-30T22:30:12.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.860+0000 7faccce7e700 1 --2- 172.21.15.16:0/4100818284 >> v2:172.21.15.16:3300/0 conn(0x7facc809d6f0 0x7facc809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:12.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.860+0000 7facc7fff700 1 --2- 172.21.15.16:0/4100818284 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7facc809e000 0x7facc8130660 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7facb40098f0 tx=0x7facb4009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9110b2497a35f1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:12.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.860+0000 7facc77fe700 1 -- 172.21.15.16:0/4100818284 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb4011830 con 0x7facc809e000 2023-09-30T22:30:12.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.860+0000 7facc77fe700 1 -- 172.21.15.16:0/4100818284 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facb4011990 con 0x7facc809e000 2023-09-30T22:30:12.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.860+0000 7facc77fe700 1 -- 172.21.15.16:0/4100818284 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facb4011d20 con 0x7facc809e000 2023-09-30T22:30:12.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.861+0000 7faccf0e2700 1 -- 172.21.15.16:0/4100818284 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7facc809e000 msgr2=0x7facc8130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:12.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.861+0000 7faccf0e2700 1 --2- 172.21.15.16:0/4100818284 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7facc809e000 0x7facc8130660 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7facb40098f0 tx=0x7facb4009c00 comp rx=0 tx=0).stop 2023-09-30T22:30:12.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.862+0000 7faccf0e2700 1 -- 172.21.15.16:0/4100818284 shutdown_connections 2023-09-30T22:30:12.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.862+0000 7faccf0e2700 1 --2- 172.21.15.16:0/4100818284 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7facc809e410 0x7facc8130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:12.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.862+0000 7faccf0e2700 1 --2- 172.21.15.16:0/4100818284 >> v2:172.21.15.16:3300/0 conn(0x7facc809d6f0 0x7facc809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:12.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.862+0000 7faccf0e2700 1 --2- 172.21.15.16:0/4100818284 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7facc809e000 0x7facc8130660 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:12.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.862+0000 7faccf0e2700 1 -- 172.21.15.16:0/4100818284 >> 172.21.15.16:0/4100818284 conn(0x7facc8066f50 msgr2=0x7facc809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:12.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.862+0000 7faccf0e2700 1 -- 172.21.15.16:0/4100818284 shutdown_connections 2023-09-30T22:30:12.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.862+0000 7faccf0e2700 1 -- 172.21.15.16:0/4100818284 wait complete. 2023-09-30T22:30:12.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.863+0000 7faccf0e2700 1 Processor -- start 2023-09-30T22:30:12.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.863+0000 7faccf0e2700 1 -- start start 2023-09-30T22:30:12.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.863+0000 7faccf0e2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7facc809d6f0 0x7facc806e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:12.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.863+0000 7faccf0e2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7facc809e000 0x7facc806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:12.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.863+0000 7faccf0e2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facc809e410 0x7facc806f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:12.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.863+0000 7faccf0e2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facc806fd40 con 0x7facc809e410 2023-09-30T22:30:12.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.863+0000 7faccf0e2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facc8125a50 con 0x7facc809d6f0 2023-09-30T22:30:12.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.863+0000 7faccf0e2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7facc809c700 con 0x7facc809e000 2023-09-30T22:30:12.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.864+0000 7faccce7e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7facc809d6f0 0x7facc806e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:12.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.864+0000 7faccce7e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7facc809d6f0 0x7facc806e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41496/0 (socket says 172.21.15.16:41496) 2023-09-30T22:30:12.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.864+0000 7faccce7e700 1 -- 172.21.15.16:0/978964827 learned_addr learned my addr 172.21.15.16:0/978964827 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:12.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.864+0000 7facc7fff700 1 --2- 172.21.15.16:0/978964827 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7facc809e000 0x7facc806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:12.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.864+0000 7faccce7e700 1 -- 172.21.15.16:0/978964827 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7facc809e000 msgr2=0x7facc806edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:12.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.864+0000 7faccce7e700 1 --2- 172.21.15.16:0/978964827 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7facc809e000 0x7facc806edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:12.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.864+0000 7faccce7e700 1 -- 172.21.15.16:0/978964827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facc809e410 msgr2=0x7facc806f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:12.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.864+0000 7faccce7e700 1 --2- 172.21.15.16:0/978964827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facc809e410 0x7facc806f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:12.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.864+0000 7faccce7e700 1 -- 172.21.15.16:0/978964827 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facbc0094a0 con 0x7facc809d6f0 2023-09-30T22:30:12.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.864+0000 7faccce7e700 1 --2- 172.21.15.16:0/978964827 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7facc809d6f0 0x7facc806e870 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7facbc00ebe0 tx=0x7facbc00eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:12.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.864+0000 7facc57fa700 1 -- 172.21.15.16:0/978964827 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facbc00c850 con 0x7facc809d6f0 2023-09-30T22:30:12.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.865+0000 7facc57fa700 1 -- 172.21.15.16:0/978964827 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facbc00c9b0 con 0x7facc809d6f0 2023-09-30T22:30:12.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.865+0000 7facc57fa700 1 -- 172.21.15.16:0/978964827 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facbc00cd20 con 0x7facc809d6f0 2023-09-30T22:30:12.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.865+0000 7faccf0e2700 1 -- 172.21.15.16:0/978964827 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facb4009550 con 0x7facc809d6f0 2023-09-30T22:30:12.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.865+0000 7faccf0e2700 1 -- 172.21.15.16:0/978964827 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facc8124610 con 0x7facc809d6f0 2023-09-30T22:30:12.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.866+0000 7facc57fa700 1 -- 172.21.15.16:0/978964827 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7facbc013470 con 0x7facc809d6f0 2023-09-30T22:30:12.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.866+0000 7faccf0e2700 1 -- 172.21.15.16:0/978964827 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facc8074ea0 con 0x7facc809d6f0 2023-09-30T22:30:12.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.866+0000 7facc57fa700 1 -- 172.21.15.16:0/978964827 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7facbc03fc00 con 0x7facc809d6f0 2023-09-30T22:30:12.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.866+0000 7facc7fff700 1 -- 172.21.15.16:0/978964827 >> v1:172.21.15.16:6816/126811 conn(0x7facac02f510 legacy=0x7facac0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:12.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.876+0000 7facc57fa700 1 -- 172.21.15.16:0/978964827 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7facbc013720 con 0x7facc809d6f0 2023-09-30T22:30:12.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.905+0000 7fbb3890c700 1 Processor -- start 2023-09-30T22:30:12.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.905+0000 7fbb3890c700 1 -- start start 2023-09-30T22:30:12.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.906+0000 7fbb3890c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbb3409d6f0 0x7fbb3409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:12.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.906+0000 7fbb3890c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbb3409e000 0x7fbb34130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:12.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.906+0000 7fbb3890c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbb3409e410 0x7fbb34130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:12.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.906+0000 7fbb3890c700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb3409f520 con 0x7fbb3409e410 2023-09-30T22:30:12.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.906+0000 7fbb3890c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3409f130 con 0x7fbb3409e000 2023-09-30T22:30:12.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.906+0000 7fbb3890c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3409e930 con 0x7fbb3409d6f0 2023-09-30T22:30:12.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.906+0000 7fbb3259c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbb3409d6f0 0x7fbb3409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:12.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.906+0000 7fbb31d9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbb3409e000 0x7fbb34130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:12.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.906+0000 7fbb31d9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbb3409e000 0x7fbb34130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41510/0 (socket says 172.21.15.16:41510) 2023-09-30T22:30:12.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.906+0000 7fbb3259c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbb3409d6f0 0x7fbb3409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:51006/0 (socket says 172.21.15.16:51006) 2023-09-30T22:30:12.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.907+0000 7fbb31d9b700 1 -- 172.21.15.16:0/1488562999 learned_addr learned my addr 172.21.15.16:0/1488562999 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:12.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.907+0000 7fbb32d9d700 1 --2- 172.21.15.16:0/1488562999 >> v2:172.21.15.16:3300/0 conn(0x7fbb3409e410 0x7fbb34130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:12.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.907+0000 7fbb32d9d700 1 -- 172.21.15.16:0/1488562999 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbb3409d6f0 msgr2=0x7fbb3409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:12.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.907+0000 7fbb32d9d700 1 --2- 172.21.15.16:0/1488562999 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbb3409d6f0 0x7fbb3409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:12.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.907+0000 7fbb32d9d700 1 -- 172.21.15.16:0/1488562999 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbb3409e000 msgr2=0x7fbb34130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:12.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.907+0000 7fbb32d9d700 1 --2- 172.21.15.16:0/1488562999 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbb3409e000 0x7fbb34130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:12.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.907+0000 7fbb32d9d700 1 -- 172.21.15.16:0/1488562999 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb3409b2d0 con 0x7fbb3409e410 2023-09-30T22:30:12.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.907+0000 7fbb31d9b700 1 --2- 172.21.15.16:0/1488562999 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbb3409e000 0x7fbb34130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:12.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.907+0000 7fbb32d9d700 1 --2- 172.21.15.16:0/1488562999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb3409e410 0x7fbb34130ba0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c009d90 tx=0x7fbb2c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e93b8ba9e3b21e84 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:12.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.908+0000 7fbb3159a700 1 -- 172.21.15.16:0/1488562999 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c00e070 con 0x7fbb3409e410 2023-09-30T22:30:12.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.908+0000 7fbb3159a700 1 -- 172.21.15.16:0/1488562999 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb2c00bb20 con 0x7fbb3409e410 2023-09-30T22:30:12.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.908+0000 7fbb3159a700 1 -- 172.21.15.16:0/1488562999 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c012430 con 0x7fbb3409e410 2023-09-30T22:30:12.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.909+0000 7fbb3890c700 1 -- 172.21.15.16:0/1488562999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb3409e410 msgr2=0x7fbb34130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:12.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.909+0000 7fbb3890c700 1 --2- 172.21.15.16:0/1488562999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb3409e410 0x7fbb34130ba0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c009d90 tx=0x7fbb2c00b330 comp rx=0 tx=0).stop 2023-09-30T22:30:12.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.909+0000 7fbb3890c700 1 -- 172.21.15.16:0/1488562999 shutdown_connections 2023-09-30T22:30:12.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.909+0000 7fbb3890c700 1 --2- 172.21.15.16:0/1488562999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb3409e410 0x7fbb34130ba0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:12.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.909+0000 7fbb3890c700 1 --2- 172.21.15.16:0/1488562999 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbb3409d6f0 0x7fbb3409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:12.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.909+0000 7fbb3890c700 1 --2- 172.21.15.16:0/1488562999 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbb3409e000 0x7fbb34130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:12.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.909+0000 7fbb3890c700 1 -- 172.21.15.16:0/1488562999 >> 172.21.15.16:0/1488562999 conn(0x7fbb34066f50 msgr2=0x7fbb3409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:12.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.909+0000 7fbb3890c700 1 -- 172.21.15.16:0/1488562999 shutdown_connections 2023-09-30T22:30:12.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.909+0000 7fbb3890c700 1 -- 172.21.15.16:0/1488562999 wait complete. 2023-09-30T22:30:12.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.910+0000 7fbb3890c700 1 Processor -- start 2023-09-30T22:30:12.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.910+0000 7fbb3890c700 1 -- start start 2023-09-30T22:30:12.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.911+0000 7fbb3890c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbb3409d6f0 0x7fbb3411de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:12.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.911+0000 7fbb3890c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbb3409e000 0x7fbb3411e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:12.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.911+0000 7fbb3890c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb3409e410 0x7fbb3411e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:12.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.911+0000 7fbb3890c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3411fa10 con 0x7fbb3409e410 2023-09-30T22:30:12.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.911+0000 7fbb3890c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb34120030 con 0x7fbb3409d6f0 2023-09-30T22:30:12.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.911+0000 7fbb3890c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbb34120c60 con 0x7fbb3409e000 2023-09-30T22:30:12.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.911+0000 7fbb32d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb3409e410 0x7fbb3411e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:12.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.911+0000 7fbb32d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb3409e410 0x7fbb3411e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38764/0 (socket says 172.21.15.16:38764) 2023-09-30T22:30:12.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.911+0000 7fbb32d9d700 1 -- 172.21.15.16:0/4055577457 learned_addr learned my addr 172.21.15.16:0/4055577457 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:12.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.911+0000 7fbb31d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbb3409e000 0x7fbb3411e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:12.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.911+0000 7fbb3259c700 1 --2- 172.21.15.16:0/4055577457 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbb3409d6f0 0x7fbb3411de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:12.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.911+0000 7fbb32d9d700 1 -- 172.21.15.16:0/4055577457 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbb3409e000 msgr2=0x7fbb3411e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:12.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.911+0000 7fbb32d9d700 1 --2- 172.21.15.16:0/4055577457 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbb3409e000 0x7fbb3411e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:12.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.911+0000 7fbb32d9d700 1 -- 172.21.15.16:0/4055577457 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbb3409d6f0 msgr2=0x7fbb3411de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:12.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.911+0000 7fbb32d9d700 1 --2- 172.21.15.16:0/4055577457 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbb3409d6f0 0x7fbb3411de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:12.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.911+0000 7fbb32d9d700 1 -- 172.21.15.16:0/4055577457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb28014c10 con 0x7fbb3409e410 2023-09-30T22:30:12.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.912+0000 7fbb31d9b700 1 --2- 172.21.15.16:0/4055577457 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbb3409e000 0x7fbb3411e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:12.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.912+0000 7fbb32d9d700 1 --2- 172.21.15.16:0/4055577457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb3409e410 0x7fbb3411e8d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c004270 tx=0x7fbb2c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:12.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.912+0000 7fbb22ffd700 1 -- 172.21.15.16:0/4055577457 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c0048a0 con 0x7fbb3409e410 2023-09-30T22:30:12.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.912+0000 7fbb22ffd700 1 -- 172.21.15.16:0/4055577457 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb2c020070 con 0x7fbb3409e410 2023-09-30T22:30:12.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.912+0000 7fbb22ffd700 1 -- 172.21.15.16:0/4055577457 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c0276c0 con 0x7fbb3409e410 2023-09-30T22:30:12.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.912+0000 7fbb3890c700 1 -- 172.21.15.16:0/4055577457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb2c009a70 con 0x7fbb3409e410 2023-09-30T22:30:12.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.912+0000 7fbb3890c700 1 -- 172.21.15.16:0/4055577457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb340a1640 con 0x7fbb3409e410 2023-09-30T22:30:12.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.913+0000 7fbb3890c700 1 -- 172.21.15.16:0/4055577457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb3409c700 con 0x7fbb3409e410 2023-09-30T22:30:12.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.914+0000 7fbb22ffd700 1 -- 172.21.15.16:0/4055577457 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 9) v1 ==== 35609+0+0 (secure 0 0 0) 0x7fbb2c027820 con 0x7fbb3409e410 2023-09-30T22:30:12.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.914+0000 7fbb22ffd700 1 -- 172.21.15.16:0/4055577457 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4092+0+0 (secure 0 0 0) 0x7fbb2c022070 con 0x7fbb3409e410 2023-09-30T22:30:12.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.914+0000 7fbb3259c700 1 -- 172.21.15.16:0/4055577457 >> v1:172.21.15.16:6816/126811 conn(0x7fbb1402f540 legacy=0x7fbb140319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:12.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.923+0000 7fbb22ffd700 1 -- 172.21.15.16:0/4055577457 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbb2c003e10 con 0x7fbb3409e410 2023-09-30T22:30:12.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:12.976+0000 7f25a37fe700 1 -- 172.21.15.16:0/288497948 >> v1:172.21.15.16:6816/126811 conn(0x7f259002f510 legacy=0x7f25900319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:13.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:13.016+0000 7f25a9f44700 1 -- 172.21.15.16:0/288497948 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f25a4074ea0 con 0x7f25a409f240 2023-09-30T22:30:13.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:13.016+0000 7f258bfff700 1 -- 172.21.15.16:0/288497948 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2598015020 con 0x7f25a409f240 2023-09-30T22:30:13.018 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:30:13.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:13.019+0000 7f25a9f44700 1 -- 172.21.15.16:0/288497948 >> v1:172.21.15.16:6816/126811 conn(0x7f259002f510 legacy=0x7f25900319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:30:13.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:13.019+0000 7f25a9f44700 1 -- 172.21.15.16:0/288497948 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f25a409f240 msgr2=0x7f25a41c6760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:13.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:13.019+0000 7f25a9f44700 1 --2- 172.21.15.16:0/288497948 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f25a409f240 0x7f25a41c6760 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f25980090b0 tx=0x7f2598004750 comp rx=0 tx=0).stop 2023-09-30T22:30:13.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:13.019+0000 7f25a9f44700 1 -- 172.21.15.16:0/288497948 shutdown_connections 2023-09-30T22:30:13.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:13.019+0000 7f25a9f44700 1 --2- 172.21.15.16:0/288497948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25a409f650 0x7f25a41c27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:13.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:13.019+0000 7f25a9f44700 1 --2- 172.21.15.16:0/288497948 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f25a409e930 0x7f25a41c6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:13.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:13.019+0000 7f25a9f44700 1 --2- 172.21.15.16:0/288497948 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f25a409f240 0x7f25a41c6760 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:13.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:13.019+0000 7f25a9f44700 1 -- 172.21.15.16:0/288497948 >> 172.21.15.16:0/288497948 conn(0x7f25a40a4210 msgr2=0x7f25a41b8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:13.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:13.019+0000 7f25a9f44700 1 -- 172.21.15.16:0/288497948 shutdown_connections 2023-09-30T22:30:13.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:13.019+0000 7f25a9f44700 1 -- 172.21.15.16:0/288497948 wait complete. 2023-09-30T22:30:13.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:13.031+0000 7fc3e37fe700 1 -- 172.21.15.16:0/2064427510 >> v1:172.21.15.16:6816/126811 conn(0x7fc3d002f510 legacy=0x7fc3d00319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:13.039 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:30:13.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:13.053+0000 7fc3e9dae700 1 -- 172.21.15.16:0/2064427510 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc3e406b250 con 0x7fc3e409f650 2023-09-30T22:30:13.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:13.053+0000 7fc3cbfff700 1 -- 172.21.15.16:0/2064427510 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc3dc019300 con 0x7fc3e409f650 2023-09-30T22:30:13.055 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:30:13.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:13.056+0000 7fc3e9dae700 1 -- 172.21.15.16:0/2064427510 >> v1:172.21.15.16:6816/126811 conn(0x7fc3d002f510 legacy=0x7fc3d00319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:30:13.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:13.056+0000 7fc3e9dae700 1 -- 172.21.15.16:0/2064427510 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc3e409f650 msgr2=0x7fc3e411db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:13.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:13.056+0000 7fc3e9dae700 1 --2- 172.21.15.16:0/2064427510 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc3e409f650 0x7fc3e411db50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc00cc80 tx=0x7fc3dc00cf90 comp rx=0 tx=0).stop 2023-09-30T22:30:13.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:13.056+0000 7fc3e9dae700 1 -- 172.21.15.16:0/2064427510 shutdown_connections 2023-09-30T22:30:13.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:13.056+0000 7fc3e9dae700 1 --2- 172.21.15.16:0/2064427510 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc3e409f650 0x7fc3e411db50 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:13.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:13.056+0000 7fc3e9dae700 1 --2- 172.21.15.16:0/2064427510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3e409e930 0x7fc3e411d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:13.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:13.056+0000 7fc3e9dae700 1 --2- 172.21.15.16:0/2064427510 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc3e409f240 0x7fc3e411d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:13.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:13.056+0000 7fc3e9dae700 1 -- 172.21.15.16:0/2064427510 >> 172.21.15.16:0/2064427510 conn(0x7fc3e40a4210 msgr2=0x7fc3e4131d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:13.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:13.056+0000 7fc3e9dae700 1 -- 172.21.15.16:0/2064427510 shutdown_connections 2023-09-30T22:30:13.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:13.056+0000 7fc3e9dae700 1 -- 172.21.15.16:0/2064427510 wait complete. 2023-09-30T22:30:13.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:13.067+0000 7facc7fff700 1 -- 172.21.15.16:0/978964827 >> v1:172.21.15.16:6816/126811 conn(0x7facac02f510 legacy=0x7facac0319c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:13.071 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:30:13.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:13.089+0000 7faccf0e2700 1 -- 172.21.15.16:0/978964827 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7facc8125770 con 0x7facc809d6f0 2023-09-30T22:30:13.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:13.090+0000 7facc57fa700 1 -- 172.21.15.16:0/978964827 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7facbc021090 con 0x7facc809d6f0 2023-09-30T22:30:13.091 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:30:13.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:13.092+0000 7faccf0e2700 1 -- 172.21.15.16:0/978964827 >> v1:172.21.15.16:6816/126811 conn(0x7facac02f510 legacy=0x7facac0319c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:30:13.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:13.092+0000 7faccf0e2700 1 -- 172.21.15.16:0/978964827 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7facc809d6f0 msgr2=0x7facc806e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:13.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:13.092+0000 7faccf0e2700 1 --2- 172.21.15.16:0/978964827 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7facc809d6f0 0x7facc806e870 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7facbc00ebe0 tx=0x7facbc00eef0 comp rx=0 tx=0).stop 2023-09-30T22:30:13.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:13.092+0000 7faccf0e2700 1 -- 172.21.15.16:0/978964827 shutdown_connections 2023-09-30T22:30:13.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:13.092+0000 7faccf0e2700 1 --2- 172.21.15.16:0/978964827 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7facc809d6f0 0x7facc806e870 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:13.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:13.092+0000 7faccf0e2700 1 --2- 172.21.15.16:0/978964827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facc809e410 0x7facc806f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:13.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:13.092+0000 7faccf0e2700 1 --2- 172.21.15.16:0/978964827 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7facc809e000 0x7facc806edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:13.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:13.092+0000 7faccf0e2700 1 -- 172.21.15.16:0/978964827 >> 172.21.15.16:0/978964827 conn(0x7facc8066f50 msgr2=0x7facc806a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:13.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:13.092+0000 7faccf0e2700 1 -- 172.21.15.16:0/978964827 shutdown_connections 2023-09-30T22:30:13.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:13.092+0000 7faccf0e2700 1 -- 172.21.15.16:0/978964827 wait complete. 2023-09-30T22:30:13.107 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:30:13.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:13.115+0000 7fbb3259c700 1 -- 172.21.15.16:0/4055577457 >> v1:172.21.15.16:6816/126811 conn(0x7fbb1402f540 legacy=0x7fbb140319f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.16:6816/126811 2023-09-30T22:30:13.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:13.140+0000 7fbb3890c700 1 -- 172.21.15.16:0/4055577457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fbb34074ea0 con 0x7fbb3409e410 2023-09-30T22:30:13.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:13.140+0000 7fbb22ffd700 1 -- 172.21.15.16:0/4055577457 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbb2c02c020 con 0x7fbb3409e410 2023-09-30T22:30:13.141 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:30:13.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:13.142+0000 7fbb3890c700 1 -- 172.21.15.16:0/4055577457 >> v1:172.21.15.16:6816/126811 conn(0x7fbb1402f540 legacy=0x7fbb140319f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:30:13.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:13.142+0000 7fbb3890c700 1 -- 172.21.15.16:0/4055577457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb3409e410 msgr2=0x7fbb3411e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:13.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:13.142+0000 7fbb3890c700 1 --2- 172.21.15.16:0/4055577457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb3409e410 0x7fbb3411e8d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c004270 tx=0x7fbb2c004ad0 comp rx=0 tx=0).stop 2023-09-30T22:30:13.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:13.142+0000 7fbb3890c700 1 -- 172.21.15.16:0/4055577457 shutdown_connections 2023-09-30T22:30:13.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:13.142+0000 7fbb3890c700 1 --2- 172.21.15.16:0/4055577457 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbb3409d6f0 0x7fbb3411de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:13.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:13.142+0000 7fbb3890c700 1 --2- 172.21.15.16:0/4055577457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb3409e410 0x7fbb3411e8d0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:13.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:13.142+0000 7fbb3890c700 1 --2- 172.21.15.16:0/4055577457 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbb3409e000 0x7fbb3411e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:13.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:13.142+0000 7fbb3890c700 1 -- 172.21.15.16:0/4055577457 >> 172.21.15.16:0/4055577457 conn(0x7fbb34066f50 msgr2=0x7fbb3406a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:13.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:13.142+0000 7fbb3890c700 1 -- 172.21.15.16:0/4055577457 shutdown_connections 2023-09-30T22:30:13.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:13.142+0000 7fbb3890c700 1 -- 172.21.15.16:0/4055577457 wait complete. 2023-09-30T22:30:13.157 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:30:14.041 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:14.072 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:14.108 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:14.158 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:14.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.193+0000 7fe7a1f17700 1 Processor -- start 2023-09-30T22:30:14.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.194+0000 7fe7a1f17700 1 -- start start 2023-09-30T22:30:14.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.194+0000 7fe7a1f17700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe79c130540 0x7fe79c130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:14.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.194+0000 7fe7a1f17700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe79c06fcf0 0x7fe79c138570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:14.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.194+0000 7fe7a1f17700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe79c0716a0 con 0x7fe79c130e50 2023-09-30T22:30:14.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.194+0000 7fe7a1f17700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe79c09a390 con 0x7fe79c06fcf0 2023-09-30T22:30:14.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.194+0000 7fe7a1f17700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe79c071290 con 0x7fe79c130540 2023-09-30T22:30:14.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.194+0000 7fe79bfff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe79c06fcf0 0x7fe79c138570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:14.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.194+0000 7fe79b7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe79c130540 0x7fe79c130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:14.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.194+0000 7fe79bfff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe79c06fcf0 0x7fe79c138570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41524/0 (socket says 172.21.15.16:41524) 2023-09-30T22:30:14.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.194+0000 7fe79bfff700 1 -- 172.21.15.16:0/964904909 learned_addr learned my addr 172.21.15.16:0/964904909 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:14.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.194+0000 7fe79b7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe79c130540 0x7fe79c130910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:51028/0 (socket says 172.21.15.16:51028) 2023-09-30T22:30:14.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.195+0000 7fe79bfff700 1 -- 172.21.15.16:0/964904909 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe79c130540 msgr2=0x7fe79c130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:14.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.195+0000 7fe79bfff700 1 --2- 172.21.15.16:0/964904909 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe79c130540 0x7fe79c130910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.195+0000 7fe79bfff700 1 -- 172.21.15.16:0/964904909 >> v1:172.21.15.16:6789/0 conn(0x7fe79c130e50 legacy=0x7fe79c06f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:14.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.195+0000 7fe79bfff700 1 -- 172.21.15.16:0/964904909 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe79c130340 con 0x7fe79c06fcf0 2023-09-30T22:30:14.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.195+0000 7fe79bfff700 1 --2- 172.21.15.16:0/964904909 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe79c06fcf0 0x7fe79c138570 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe794009950 tx=0x7fe794009c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ad0e2a6245d14ca server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:14.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.195+0000 7fe79affd700 1 -- 172.21.15.16:0/964904909 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe794004230 con 0x7fe79c06fcf0 2023-09-30T22:30:14.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.195+0000 7fe79affd700 1 -- 172.21.15.16:0/964904909 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe794004390 con 0x7fe79c06fcf0 2023-09-30T22:30:14.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.195+0000 7fe79affd700 1 -- 172.21.15.16:0/964904909 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7940036a0 con 0x7fe79c06fcf0 2023-09-30T22:30:14.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.196+0000 7fe7a1f17700 1 -- 172.21.15.16:0/964904909 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe79c06fcf0 msgr2=0x7fe79c138570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:14.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.196+0000 7fe7a1f17700 1 --2- 172.21.15.16:0/964904909 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe79c06fcf0 0x7fe79c138570 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe794009950 tx=0x7fe794009c60 comp rx=0 tx=0).stop 2023-09-30T22:30:14.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.196+0000 7fe7a1f17700 1 -- 172.21.15.16:0/964904909 shutdown_connections 2023-09-30T22:30:14.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.196+0000 7fe7a1f17700 1 --2- 172.21.15.16:0/964904909 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe79c06fcf0 0x7fe79c138570 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.196+0000 7fe7a1f17700 1 --2- 172.21.15.16:0/964904909 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe79c130540 0x7fe79c130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.196+0000 7fe7a1f17700 1 -- 172.21.15.16:0/964904909 >> 172.21.15.16:0/964904909 conn(0x7fe79c0a2aa0 msgr2=0x7fe79c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:14.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.196+0000 7fe7a1f17700 1 -- 172.21.15.16:0/964904909 shutdown_connections 2023-09-30T22:30:14.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.196+0000 7fe7a1f17700 1 -- 172.21.15.16:0/964904909 wait complete. 2023-09-30T22:30:14.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.198+0000 7fe7a1f17700 1 Processor -- start 2023-09-30T22:30:14.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.198+0000 7fe7a1f17700 1 -- start start 2023-09-30T22:30:14.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.199+0000 7fe7a1f17700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe79c06fcf0 0x7fe79c1260f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:14.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.199+0000 7fe7a1f17700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe79c130540 0x7fe79c126630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:14.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.199+0000 7fe7a1f17700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe79c130e50 0x7fe79c1c5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:14.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.199+0000 7fe7a1f17700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79c12c790 con 0x7fe79c06fcf0 2023-09-30T22:30:14.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.199+0000 7fe7a1f17700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe79c0718e0 con 0x7fe79c130540 2023-09-30T22:30:14.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.199+0000 7fe7a1f17700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe79c1297a0 con 0x7fe79c130e50 2023-09-30T22:30:14.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.199+0000 7fe792ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe79c130540 0x7fe79c126630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:14.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.199+0000 7fe792ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe79c130540 0x7fe79c126630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41534/0 (socket says 172.21.15.16:41534) 2023-09-30T22:30:14.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.199+0000 7fe792ffd700 1 -- 172.21.15.16:0/207148409 learned_addr learned my addr 172.21.15.16:0/207148409 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:14.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.199+0000 7fe79bfff700 1 --2- 172.21.15.16:0/207148409 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe79c130e50 0x7fe79c1c5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:14.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.199+0000 7fe792ffd700 1 -- 172.21.15.16:0/207148409 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe79c130e50 msgr2=0x7fe79c1c5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:14.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.199+0000 7fe792ffd700 1 --2- 172.21.15.16:0/207148409 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe79c130e50 0x7fe79c1c5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.199+0000 7fe792ffd700 1 -- 172.21.15.16:0/207148409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe79c06fcf0 msgr2=0x7fe79c1260f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:14.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.199+0000 7fe792ffd700 1 --2- 172.21.15.16:0/207148409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe79c06fcf0 0x7fe79c1260f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.199+0000 7fe792ffd700 1 -- 172.21.15.16:0/207148409 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe794009550 con 0x7fe79c130540 2023-09-30T22:30:14.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.200+0000 7fe792ffd700 1 --2- 172.21.15.16:0/207148409 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe79c130540 0x7fe79c126630 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe78400c4a0 tx=0x7fe78400c7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:14.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.200+0000 7fe798ff9700 1 -- 172.21.15.16:0/207148409 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe784011790 con 0x7fe79c130540 2023-09-30T22:30:14.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.200+0000 7fe798ff9700 1 -- 172.21.15.16:0/207148409 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7840118f0 con 0x7fe79c130540 2023-09-30T22:30:14.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.200+0000 7fe7a1f17700 1 -- 172.21.15.16:0/207148409 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe79c12c0b0 con 0x7fe79c130540 2023-09-30T22:30:14.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.200+0000 7fe798ff9700 1 -- 172.21.15.16:0/207148409 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe784011ca0 con 0x7fe79c130540 2023-09-30T22:30:14.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.200+0000 7fe7a1f17700 1 -- 172.21.15.16:0/207148409 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe79c126b70 con 0x7fe79c130540 2023-09-30T22:30:14.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.202+0000 7fe798ff9700 1 -- 172.21.15.16:0/207148409 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe78401c460 con 0x7fe79c130540 2023-09-30T22:30:14.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.202+0000 7fe7a1f17700 1 -- 172.21.15.16:0/207148409 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe79c074ea0 con 0x7fe79c130540 2023-09-30T22:30:14.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.202+0000 7fe798ff9700 1 -- 172.21.15.16:0/207148409 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fe78403f3d0 con 0x7fe79c130540 2023-09-30T22:30:14.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.211+0000 7fe798ff9700 1 -- 172.21.15.16:0/207148409 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe7840108d0 con 0x7fe79c130540 2023-09-30T22:30:14.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.246+0000 7f749ed31700 1 Processor -- start 2023-09-30T22:30:14.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.246+0000 7f749ed31700 1 -- start start 2023-09-30T22:30:14.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.246+0000 7f749ed31700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7498130750 0x7f7498130b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:14.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.246+0000 7f749ed31700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f749809e250 0x7f74981384c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:14.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.247+0000 7f749ed31700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f749809c190 con 0x7f749809d6f0 2023-09-30T22:30:14.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.247+0000 7f749ed31700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f749809b090 con 0x7f7498130750 2023-09-30T22:30:14.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.247+0000 7f749ed31700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f749809a6c0 con 0x7f749809e250 2023-09-30T22:30:14.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.247+0000 7f749cacd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7498130750 0x7f7498130b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:14.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.247+0000 7f749cacd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7498130750 0x7f7498130b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41536/0 (socket says 172.21.15.16:41536) 2023-09-30T22:30:14.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.247+0000 7f749cacd700 1 -- 172.21.15.16:0/3110798086 learned_addr learned my addr 172.21.15.16:0/3110798086 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:14.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.247+0000 7f749cacd700 1 -- 172.21.15.16:0/3110798086 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f749809e250 msgr2=0x7f74981384c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:14.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.247+0000 7f749cacd700 1 --2- 172.21.15.16:0/3110798086 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f749809e250 0x7f74981384c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.247+0000 7f749cacd700 1 -- 172.21.15.16:0/3110798086 >> v1:172.21.15.16:6789/0 conn(0x7f749809d6f0 legacy=0x7f749809db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:14.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.247+0000 7f749cacd700 1 -- 172.21.15.16:0/3110798086 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7498131060 con 0x7f7498130750 2023-09-30T22:30:14.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.247+0000 7f749cacd700 1 --2- 172.21.15.16:0/3110798086 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7498130750 0x7f7498130b20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f748c009920 tx=0x7f748c009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d81e1635de137bc8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:14.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.248+0000 7f74977fe700 1 -- 172.21.15.16:0/3110798086 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748c00ba60 con 0x7f7498130750 2023-09-30T22:30:14.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.248+0000 7f74977fe700 1 -- 172.21.15.16:0/3110798086 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f748c00bbc0 con 0x7f7498130750 2023-09-30T22:30:14.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.248+0000 7f74977fe700 1 -- 172.21.15.16:0/3110798086 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748c009e70 con 0x7f7498130750 2023-09-30T22:30:14.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.248+0000 7f749ed31700 1 -- 172.21.15.16:0/3110798086 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7498130750 msgr2=0x7f7498130b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:14.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.248+0000 7f749ed31700 1 --2- 172.21.15.16:0/3110798086 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7498130750 0x7f7498130b20 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f748c009920 tx=0x7f748c009c30 comp rx=0 tx=0).stop 2023-09-30T22:30:14.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.249+0000 7f749ed31700 1 -- 172.21.15.16:0/3110798086 shutdown_connections 2023-09-30T22:30:14.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.249+0000 7f749ed31700 1 --2- 172.21.15.16:0/3110798086 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f749809e250 0x7f74981384c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.249+0000 7f749ed31700 1 --2- 172.21.15.16:0/3110798086 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7498130750 0x7f7498130b20 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.249+0000 7f749ed31700 1 -- 172.21.15.16:0/3110798086 >> 172.21.15.16:0/3110798086 conn(0x7f749809f470 msgr2=0x7f74980a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:14.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.249+0000 7f749ed31700 1 -- 172.21.15.16:0/3110798086 shutdown_connections 2023-09-30T22:30:14.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.249+0000 7f749ed31700 1 -- 172.21.15.16:0/3110798086 wait complete. 2023-09-30T22:30:14.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.250+0000 7f749ed31700 1 Processor -- start 2023-09-30T22:30:14.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.250+0000 7f749ed31700 1 -- start start 2023-09-30T22:30:14.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.251+0000 7f749ed31700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f749809d6f0 0x7f74981c8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:14.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.251+0000 7f749ed31700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f749809e250 0x7f74981c4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:14.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.251+0000 7f749ed31700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7498130750 0x7f74981c5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:14.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.251+0000 7f749ed31700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7498092ad0 con 0x7f7498130750 2023-09-30T22:30:14.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.251+0000 7f749ed31700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f749809b2b0 con 0x7f749809e250 2023-09-30T22:30:14.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.251+0000 7f749ed31700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f74980920e0 con 0x7f749809d6f0 2023-09-30T22:30:14.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.251+0000 7f749cacd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f749809d6f0 0x7f74981c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:14.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.251+0000 7f749cacd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f749809d6f0 0x7f74981c8780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:51060/0 (socket says 172.21.15.16:51060) 2023-09-30T22:30:14.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.251+0000 7f749cacd700 1 -- 172.21.15.16:0/3697199878 learned_addr learned my addr 172.21.15.16:0/3697199878 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:14.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.251+0000 7f7497fff700 1 --2- 172.21.15.16:0/3697199878 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f749809e250 0x7f74981c4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:14.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.251+0000 7f749d2ce700 1 --2- 172.21.15.16:0/3697199878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7498130750 0x7f74981c5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:14.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.252+0000 7f7497fff700 1 -- 172.21.15.16:0/3697199878 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f749809d6f0 msgr2=0x7f74981c8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:14.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.252+0000 7f7497fff700 1 --2- 172.21.15.16:0/3697199878 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f749809d6f0 0x7f74981c8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.252+0000 7f7497fff700 1 -- 172.21.15.16:0/3697199878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7498130750 msgr2=0x7f74981c5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:14.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.252+0000 7f7497fff700 1 --2- 172.21.15.16:0/3697199878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7498130750 0x7f74981c5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.252+0000 7f7497fff700 1 -- 172.21.15.16:0/3697199878 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f748c009550 con 0x7f749809e250 2023-09-30T22:30:14.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.252+0000 7f749cacd700 1 --2- 172.21.15.16:0/3697199878 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f749809d6f0 0x7f74981c8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:14.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.252+0000 7f7497fff700 1 --2- 172.21.15.16:0/3697199878 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f749809e250 0x7f74981c4d40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7484000f60 tx=0x7f748400c6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:14.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.252+0000 7f74957fa700 1 -- 172.21.15.16:0/3697199878 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74840052d0 con 0x7f749809e250 2023-09-30T22:30:14.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.252+0000 7f74957fa700 1 -- 172.21.15.16:0/3697199878 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7484005430 con 0x7f749809e250 2023-09-30T22:30:14.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.252+0000 7f74957fa700 1 -- 172.21.15.16:0/3697199878 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74840038c0 con 0x7f749809e250 2023-09-30T22:30:14.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.252+0000 7f749ed31700 1 -- 172.21.15.16:0/3697199878 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7498092d90 con 0x7f749809e250 2023-09-30T22:30:14.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.252+0000 7f749ed31700 1 -- 172.21.15.16:0/3697199878 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7498093070 con 0x7f749809e250 2023-09-30T22:30:14.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.254+0000 7f74957fa700 1 -- 172.21.15.16:0/3697199878 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7484003a20 con 0x7f749809e250 2023-09-30T22:30:14.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.254+0000 7f74957fa700 1 -- 172.21.15.16:0/3697199878 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f748403e8d0 con 0x7f749809e250 2023-09-30T22:30:14.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.254+0000 7f749ed31700 1 -- 172.21.15.16:0/3697199878 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74980747f0 con 0x7f749809e250 2023-09-30T22:30:14.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.263+0000 7f74957fa700 1 -- 172.21.15.16:0/3697199878 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7484003cd0 con 0x7f749809e250 2023-09-30T22:30:14.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.294+0000 7f1077658700 1 Processor -- start 2023-09-30T22:30:14.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.294+0000 7f1077658700 1 -- start start 2023-09-30T22:30:14.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.295+0000 7f1077658700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f10700a1510 0x7f1070138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:14.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.295+0000 7f1077658700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1070139250 0x7f107013b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:14.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.295+0000 7f1077658700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f107009edd0 con 0x7f10700a0a30 2023-09-30T22:30:14.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.295+0000 7f1077658700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10700a18e0 con 0x7f10700a1510 2023-09-30T22:30:14.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.295+0000 7f1077658700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f107009e1f0 con 0x7f1070139250 2023-09-30T22:30:14.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.295+0000 7f10753f4700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f10700a0a30 0x7f10700a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48028/0 (socket says 172.21.15.16:48028) 2023-09-30T22:30:14.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.295+0000 7f10753f4700 1 -- 172.21.15.16:0/2627523197 learned_addr learned my addr 172.21.15.16:0/2627523197 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:30:14.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.295+0000 7f1074bf3700 1 --2- 172.21.15.16:0/2627523197 >> v2:172.21.15.16:3300/0 conn(0x7f10700a1510 0x7f1070138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:14.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.295+0000 7f1075bf5700 1 --2- 172.21.15.16:0/2627523197 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1070139250 0x7f107013b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:14.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.296+0000 7f1074bf3700 1 -- 172.21.15.16:0/2627523197 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1070139250 msgr2=0x7f107013b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:14.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.296+0000 7f1074bf3700 1 --2- 172.21.15.16:0/2627523197 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1070139250 0x7f107013b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.296+0000 7f1074bf3700 1 -- 172.21.15.16:0/2627523197 >> v1:172.21.15.16:6789/0 conn(0x7f10700a0a30 legacy=0x7f10700a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:14.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.296+0000 7f1074bf3700 1 -- 172.21.15.16:0/2627523197 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f107009dc50 con 0x7f10700a1510 2023-09-30T22:30:14.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.296+0000 7f1074bf3700 1 --2- 172.21.15.16:0/2627523197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10700a1510 0x7f1070138d10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1060009840 tx=0x7f1060009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a9b9173ac677903 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:14.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.296+0000 7f1067fff700 1 -- 172.21.15.16:0/2627523197 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1060011830 con 0x7f10700a1510 2023-09-30T22:30:14.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.296+0000 7f1067fff700 1 -- 172.21.15.16:0/2627523197 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1060011990 con 0x7f10700a1510 2023-09-30T22:30:14.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.296+0000 7f1067fff700 1 -- 172.21.15.16:0/2627523197 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1060011d20 con 0x7f10700a1510 2023-09-30T22:30:14.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.298+0000 7f1077658700 1 -- 172.21.15.16:0/2627523197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10700a1510 msgr2=0x7f1070138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:14.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.298+0000 7f1077658700 1 --2- 172.21.15.16:0/2627523197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10700a1510 0x7f1070138d10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1060009840 tx=0x7f1060009b50 comp rx=0 tx=0).stop 2023-09-30T22:30:14.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.298+0000 7f1077658700 1 -- 172.21.15.16:0/2627523197 shutdown_connections 2023-09-30T22:30:14.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.298+0000 7f1077658700 1 --2- 172.21.15.16:0/2627523197 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1070139250 0x7f107013b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.298+0000 7f1077658700 1 --2- 172.21.15.16:0/2627523197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10700a1510 0x7f1070138d10 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.298+0000 7f1077658700 1 -- 172.21.15.16:0/2627523197 >> 172.21.15.16:0/2627523197 conn(0x7f10700a6550 msgr2=0x7f10700a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:14.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.298+0000 7f1077658700 1 -- 172.21.15.16:0/2627523197 shutdown_connections 2023-09-30T22:30:14.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.298+0000 7f1077658700 1 -- 172.21.15.16:0/2627523197 wait complete. 2023-09-30T22:30:14.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.299+0000 7f1077658700 1 Processor -- start 2023-09-30T22:30:14.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.299+0000 7f1077658700 1 -- start start 2023-09-30T22:30:14.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.300+0000 7f1077658700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10700a0a30 0x7f107011c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:14.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.300+0000 7f1077658700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f10700a1510 0x7f107011cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:14.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.300+0000 7f1077658700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1070139250 0x7f107011d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:14.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.300+0000 7f1077658700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f107011e290 con 0x7f10700a0a30 2023-09-30T22:30:14.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.300+0000 7f1077658700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f107011f7e0 con 0x7f10700a1510 2023-09-30T22:30:14.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.300+0000 7f1077658700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1070138cb0 con 0x7f1070139250 2023-09-30T22:30:14.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.300+0000 7f1074bf3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f10700a1510 0x7f107011cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:14.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.300+0000 7f1074bf3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f10700a1510 0x7f107011cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41570/0 (socket says 172.21.15.16:41570) 2023-09-30T22:30:14.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.300+0000 7f1074bf3700 1 -- 172.21.15.16:0/2814227563 learned_addr learned my addr 172.21.15.16:0/2814227563 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:14.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.300+0000 7f10753f4700 1 --2- 172.21.15.16:0/2814227563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10700a0a30 0x7f107011c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:14.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.301+0000 7f1075bf5700 1 --2- 172.21.15.16:0/2814227563 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1070139250 0x7f107011d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:14.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.301+0000 7f10753f4700 1 -- 172.21.15.16:0/2814227563 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1070139250 msgr2=0x7f107011d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:14.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.301+0000 7f10753f4700 1 --2- 172.21.15.16:0/2814227563 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1070139250 0x7f107011d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.301+0000 7f10753f4700 1 -- 172.21.15.16:0/2814227563 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f10700a1510 msgr2=0x7f107011cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:14.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.301+0000 7f10753f4700 1 --2- 172.21.15.16:0/2814227563 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f10700a1510 0x7f107011cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.301+0000 7f10753f4700 1 -- 172.21.15.16:0/2814227563 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10600094a0 con 0x7f10700a0a30 2023-09-30T22:30:14.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.301+0000 7f1074bf3700 1 --2- 172.21.15.16:0/2814227563 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f10700a1510 0x7f107011cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:14.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.301+0000 7f10753f4700 1 --2- 172.21.15.16:0/2814227563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10700a0a30 0x7f107011c7a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f106c00c530 tx=0x7f106c00c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:14.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.301+0000 7f1065ffb700 1 -- 172.21.15.16:0/2814227563 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106c00cea0 con 0x7f10700a0a30 2023-09-30T22:30:14.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.301+0000 7f1065ffb700 1 -- 172.21.15.16:0/2814227563 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f106c005d70 con 0x7f10700a0a30 2023-09-30T22:30:14.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.301+0000 7f1065ffb700 1 -- 172.21.15.16:0/2814227563 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106c0072d0 con 0x7f10700a0a30 2023-09-30T22:30:14.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.301+0000 7f1077658700 1 -- 172.21.15.16:0/2814227563 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10700a0220 con 0x7f10700a0a30 2023-09-30T22:30:14.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.302+0000 7f1077658700 1 -- 172.21.15.16:0/2814227563 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f107011e600 con 0x7f10700a0a30 2023-09-30T22:30:14.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.303+0000 7f1065ffb700 1 -- 172.21.15.16:0/2814227563 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f106c007430 con 0x7f10700a0a30 2023-09-30T22:30:14.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.303+0000 7f1065ffb700 1 -- 172.21.15.16:0/2814227563 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f106c01a980 con 0x7f10700a0a30 2023-09-30T22:30:14.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.304+0000 7f1077658700 1 -- 172.21.15.16:0/2814227563 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10500052e0 con 0x7f10700a0a30 2023-09-30T22:30:14.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.309+0000 7f1065ffb700 1 -- 172.21.15.16:0/2814227563 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f106c017030 con 0x7f10700a0a30 2023-09-30T22:30:14.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.321+0000 7f691769c700 1 Processor -- start 2023-09-30T22:30:14.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.321+0000 7f691769c700 1 -- start start 2023-09-30T22:30:14.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.322+0000 7f691769c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f691009d6f0 0x7f691009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:14.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.322+0000 7f691769c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f691009e000 0x7f69101339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:14.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.322+0000 7f691769c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f691009b120 con 0x7f691009e410 2023-09-30T22:30:14.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.322+0000 7f691769c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f691006b460 con 0x7f691009d6f0 2023-09-30T22:30:14.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.322+0000 7f691769c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f691009a7e0 con 0x7f691009e000 2023-09-30T22:30:14.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.322+0000 7f6915438700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f691009d6f0 0x7f691009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:14.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.322+0000 7f6915c39700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f691009e410 0x7f6910133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48032/0 (socket says 172.21.15.16:48032) 2023-09-30T22:30:14.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.322+0000 7f6915c39700 1 -- 172.21.15.16:0/2184728660 learned_addr learned my addr 172.21.15.16:0/2184728660 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:30:14.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.322+0000 7f6914c37700 1 --2- 172.21.15.16:0/2184728660 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f691009e000 0x7f69101339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:14.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.323+0000 7f6915438700 1 -- 172.21.15.16:0/2184728660 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f691009e000 msgr2=0x7f69101339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:14.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.323+0000 7f6915438700 1 --2- 172.21.15.16:0/2184728660 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f691009e000 0x7f69101339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.323+0000 7f6915438700 1 -- 172.21.15.16:0/2184728660 >> v1:172.21.15.16:6789/0 conn(0x7f691009e410 legacy=0x7f6910133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:14.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.323+0000 7f6907fff700 1 -- 172.21.15.16:0/2184728660 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1697227009 0 0) 0x7f691009b120 con 0x7f691009e410 2023-09-30T22:30:14.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.323+0000 7f6915438700 1 -- 172.21.15.16:0/2184728660 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f691009a500 con 0x7f691009d6f0 2023-09-30T22:30:14.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.323+0000 7f6915438700 1 --2- 172.21.15.16:0/2184728660 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f691009d6f0 0x7f691009dac0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f690c009870 tx=0x7f690c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49eeb2e8048dc17f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:14.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.323+0000 7f6907fff700 1 -- 172.21.15.16:0/2184728660 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690c011830 con 0x7f691009d6f0 2023-09-30T22:30:14.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.324+0000 7f6907fff700 1 -- 172.21.15.16:0/2184728660 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f690c011990 con 0x7f691009d6f0 2023-09-30T22:30:14.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.324+0000 7f6907fff700 1 -- 172.21.15.16:0/2184728660 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690c011d20 con 0x7f691009d6f0 2023-09-30T22:30:14.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.324+0000 7f691769c700 1 -- 172.21.15.16:0/2184728660 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f691009d6f0 msgr2=0x7f691009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:14.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.324+0000 7f691769c700 1 --2- 172.21.15.16:0/2184728660 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f691009d6f0 0x7f691009dac0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f690c009870 tx=0x7f690c009b80 comp rx=0 tx=0).stop 2023-09-30T22:30:14.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.324+0000 7f691769c700 1 -- 172.21.15.16:0/2184728660 shutdown_connections 2023-09-30T22:30:14.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.324+0000 7f691769c700 1 --2- 172.21.15.16:0/2184728660 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f691009d6f0 0x7f691009dac0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.324+0000 7f691769c700 1 --2- 172.21.15.16:0/2184728660 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f691009e000 0x7f69101339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.324+0000 7f691769c700 1 -- 172.21.15.16:0/2184728660 >> 172.21.15.16:0/2184728660 conn(0x7f6910066f50 msgr2=0x7f691009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:14.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.324+0000 7f691769c700 1 -- 172.21.15.16:0/2184728660 shutdown_connections 2023-09-30T22:30:14.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.324+0000 7f691769c700 1 -- 172.21.15.16:0/2184728660 wait complete. 2023-09-30T22:30:14.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.325+0000 7f691769c700 1 Processor -- start 2023-09-30T22:30:14.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.326+0000 7f691769c700 1 -- start start 2023-09-30T22:30:14.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.326+0000 7f691769c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f691009d6f0 0x7f6910122320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:14.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.326+0000 7f691769c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f691009e000 0x7f6910122860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:14.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.326+0000 7f691769c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f691009e410 0x7f6910122da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:14.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.326+0000 7f691769c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6910124d20 con 0x7f691009d6f0 2023-09-30T22:30:14.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.326+0000 7f691769c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f691009c710 con 0x7f691009e410 2023-09-30T22:30:14.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.326+0000 7f691769c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6910125530 con 0x7f691009e000 2023-09-30T22:30:14.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.326+0000 7f6915438700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f691009d6f0 0x7f6910122320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:14.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.326+0000 7f6915438700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f691009d6f0 0x7f6910122320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38800/0 (socket says 172.21.15.16:38800) 2023-09-30T22:30:14.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.326+0000 7f6915438700 1 -- 172.21.15.16:0/3776964069 learned_addr learned my addr 172.21.15.16:0/3776964069 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:14.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.326+0000 7f6915c39700 1 --2- 172.21.15.16:0/3776964069 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f691009e410 0x7f6910122da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:14.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.326+0000 7f6914c37700 1 --2- 172.21.15.16:0/3776964069 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f691009e000 0x7f6910122860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:14.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.327+0000 7f6915c39700 1 -- 172.21.15.16:0/3776964069 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f691009e000 msgr2=0x7f6910122860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:14.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.327+0000 7f6915c39700 1 --2- 172.21.15.16:0/3776964069 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f691009e000 0x7f6910122860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.327+0000 7f6915c39700 1 -- 172.21.15.16:0/3776964069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f691009d6f0 msgr2=0x7f6910122320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:14.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.327+0000 7f6915c39700 1 --2- 172.21.15.16:0/3776964069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f691009d6f0 0x7f6910122320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.327+0000 7f6915c39700 1 -- 172.21.15.16:0/3776964069 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f690c0094a0 con 0x7f691009e410 2023-09-30T22:30:14.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.327+0000 7f6915c39700 1 --2- 172.21.15.16:0/3776964069 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f691009e410 0x7f6910122da0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f691009e8b0 tx=0x7f690800cec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:14.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.327+0000 7f6905ffb700 1 -- 172.21.15.16:0/3776964069 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690800aa40 con 0x7f691009e410 2023-09-30T22:30:14.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.327+0000 7f6905ffb700 1 -- 172.21.15.16:0/3776964069 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f690800abc0 con 0x7f691009e410 2023-09-30T22:30:14.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.327+0000 7f691769c700 1 -- 172.21.15.16:0/3776964069 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69100a29f0 con 0x7f691009e410 2023-09-30T22:30:14.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.327+0000 7f6905ffb700 1 -- 172.21.15.16:0/3776964069 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69080070d0 con 0x7f691009e410 2023-09-30T22:30:14.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.328+0000 7f691769c700 1 -- 172.21.15.16:0/3776964069 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6910125800 con 0x7f691009e410 2023-09-30T22:30:14.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.329+0000 7f691769c700 1 -- 172.21.15.16:0/3776964069 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f691011f030 con 0x7f691009e410 2023-09-30T22:30:14.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.329+0000 7f6905ffb700 1 -- 172.21.15.16:0/3776964069 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6908007230 con 0x7f691009e410 2023-09-30T22:30:14.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.330+0000 7f6905ffb700 1 -- 172.21.15.16:0/3776964069 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f690803f560 con 0x7f691009e410 2023-09-30T22:30:14.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.339+0000 7f6905ffb700 1 -- 172.21.15.16:0/3776964069 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f69080074e0 con 0x7f691009e410 2023-09-30T22:30:14.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.444+0000 7fe7a1f17700 1 -- 172.21.15.16:0/207148409 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe79c06b9e0 con 0x7fe79c130540 2023-09-30T22:30:14.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.445+0000 7fe798ff9700 1 -- 172.21.15.16:0/207148409 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe78401a360 con 0x7fe79c130540 2023-09-30T22:30:14.446 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:30:14.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.447+0000 7fe7a1f17700 1 -- 172.21.15.16:0/207148409 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe79c130540 msgr2=0x7fe79c126630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:14.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.447+0000 7fe7a1f17700 1 --2- 172.21.15.16:0/207148409 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe79c130540 0x7fe79c126630 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe78400c4a0 tx=0x7fe78400c7b0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.447+0000 7fe7a1f17700 1 -- 172.21.15.16:0/207148409 shutdown_connections 2023-09-30T22:30:14.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.447+0000 7fe7a1f17700 1 --2- 172.21.15.16:0/207148409 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe79c130e50 0x7fe79c1c5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.447+0000 7fe7a1f17700 1 --2- 172.21.15.16:0/207148409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe79c06fcf0 0x7fe79c1260f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.447+0000 7fe7a1f17700 1 --2- 172.21.15.16:0/207148409 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe79c130540 0x7fe79c126630 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.447+0000 7fe7a1f17700 1 -- 172.21.15.16:0/207148409 >> 172.21.15.16:0/207148409 conn(0x7fe79c0a2aa0 msgr2=0x7fe79c06e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:14.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.447+0000 7fe7a1f17700 1 -- 172.21.15.16:0/207148409 shutdown_connections 2023-09-30T22:30:14.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.447+0000 7fe7a1f17700 1 -- 172.21.15.16:0/207148409 wait complete. 2023-09-30T22:30:14.462 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:30:14.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.474+0000 7f749ed31700 1 -- 172.21.15.16:0/3697199878 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f749806b250 con 0x7f749809e250 2023-09-30T22:30:14.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.475+0000 7f74957fa700 1 -- 172.21.15.16:0/3697199878 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f74840192c0 con 0x7f749809e250 2023-09-30T22:30:14.476 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:30:14.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.477+0000 7f749ed31700 1 -- 172.21.15.16:0/3697199878 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f749809e250 msgr2=0x7f74981c4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:14.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.477+0000 7f749ed31700 1 --2- 172.21.15.16:0/3697199878 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f749809e250 0x7f74981c4d40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7484000f60 tx=0x7f748400c6e0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.477+0000 7f749ed31700 1 -- 172.21.15.16:0/3697199878 shutdown_connections 2023-09-30T22:30:14.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.477+0000 7f749ed31700 1 --2- 172.21.15.16:0/3697199878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7498130750 0x7f74981c5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.477+0000 7f749ed31700 1 --2- 172.21.15.16:0/3697199878 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f749809d6f0 0x7f74981c8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.477+0000 7f749ed31700 1 --2- 172.21.15.16:0/3697199878 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f749809e250 0x7f74981c4d40 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.477+0000 7f749ed31700 1 -- 172.21.15.16:0/3697199878 >> 172.21.15.16:0/3697199878 conn(0x7f749809f470 msgr2=0x7f74980992c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:14.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.477+0000 7f749ed31700 1 -- 172.21.15.16:0/3697199878 shutdown_connections 2023-09-30T22:30:14.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.477+0000 7f749ed31700 1 -- 172.21.15.16:0/3697199878 wait complete. 2023-09-30T22:30:14.492 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:30:14.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.519+0000 7f1077658700 1 -- 172.21.15.16:0/2814227563 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1050005590 con 0x7f10700a0a30 2023-09-30T22:30:14.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.519+0000 7f1065ffb700 1 -- 172.21.15.16:0/2814227563 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f106c044060 con 0x7f10700a0a30 2023-09-30T22:30:14.521 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:30:14.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.522+0000 7f1077658700 1 -- 172.21.15.16:0/2814227563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10700a0a30 msgr2=0x7f107011c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:14.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.522+0000 7f1077658700 1 --2- 172.21.15.16:0/2814227563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10700a0a30 0x7f107011c7a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f106c00c530 tx=0x7f106c00c840 comp rx=0 tx=0).stop 2023-09-30T22:30:14.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.522+0000 7f1077658700 1 -- 172.21.15.16:0/2814227563 shutdown_connections 2023-09-30T22:30:14.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.522+0000 7f1077658700 1 --2- 172.21.15.16:0/2814227563 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1070139250 0x7f107011d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.522+0000 7f1077658700 1 --2- 172.21.15.16:0/2814227563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10700a0a30 0x7f107011c7a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.522+0000 7f1077658700 1 --2- 172.21.15.16:0/2814227563 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f10700a1510 0x7f107011cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.522+0000 7f1077658700 1 -- 172.21.15.16:0/2814227563 >> 172.21.15.16:0/2814227563 conn(0x7f10700a6550 msgr2=0x7f1070137fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:14.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.522+0000 7f1077658700 1 -- 172.21.15.16:0/2814227563 shutdown_connections 2023-09-30T22:30:14.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.522+0000 7f1077658700 1 -- 172.21.15.16:0/2814227563 wait complete. 2023-09-30T22:30:14.537 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:30:14.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.546+0000 7f691769c700 1 -- 172.21.15.16:0/3776964069 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6910074ea0 con 0x7f691009e410 2023-09-30T22:30:14.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.546+0000 7f6905ffb700 1 -- 172.21.15.16:0/3776964069 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f690801a2c0 con 0x7f691009e410 2023-09-30T22:30:14.547 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:30:14.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.548+0000 7f691769c700 1 -- 172.21.15.16:0/3776964069 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f691009e410 msgr2=0x7f6910122da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:14.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.548+0000 7f691769c700 1 --2- 172.21.15.16:0/3776964069 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f691009e410 0x7f6910122da0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f691009e8b0 tx=0x7f690800cec0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.548+0000 7f691769c700 1 -- 172.21.15.16:0/3776964069 shutdown_connections 2023-09-30T22:30:14.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.548+0000 7f691769c700 1 --2- 172.21.15.16:0/3776964069 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f691009e410 0x7f6910122da0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.548+0000 7f691769c700 1 --2- 172.21.15.16:0/3776964069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f691009d6f0 0x7f6910122320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.548+0000 7f691769c700 1 --2- 172.21.15.16:0/3776964069 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f691009e000 0x7f6910122860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:14.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.548+0000 7f691769c700 1 -- 172.21.15.16:0/3776964069 >> 172.21.15.16:0/3776964069 conn(0x7f6910066f50 msgr2=0x7f6910133f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:14.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.549+0000 7f691769c700 1 -- 172.21.15.16:0/3776964069 shutdown_connections 2023-09-30T22:30:14.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:14.549+0000 7f691769c700 1 -- 172.21.15.16:0/3776964069 wait complete. 2023-09-30T22:30:14.563 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:30:15.464 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:15.494 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:15.538 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:15.564 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:15.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.613+0000 7f91460fc700 1 Processor -- start 2023-09-30T22:30:15.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.613+0000 7f91460fc700 1 -- start start 2023-09-30T22:30:15.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.614+0000 7f91460fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9140130540 0x7f9140130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:15.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.614+0000 7f91460fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9140130e50 0x7f914006f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:15.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.614+0000 7f91460fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f914006fb20 0x7f914006ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:15.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.614+0000 7f91460fc700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f914009a390 con 0x7f9140130540 2023-09-30T22:30:15.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.614+0000 7f91460fc700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91400716a0 con 0x7f9140130e50 2023-09-30T22:30:15.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.614+0000 7f91460fc700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9140071290 con 0x7f914006fb20 2023-09-30T22:30:15.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.614+0000 7f913effd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9140130e50 0x7f914006f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:15.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.614+0000 7f913effd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9140130e50 0x7f914006f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41594/0 (socket says 172.21.15.16:41594) 2023-09-30T22:30:15.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.614+0000 7f913effd700 1 -- 172.21.15.16:0/2402072202 learned_addr learned my addr 172.21.15.16:0/2402072202 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:15.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.614+0000 7f913ffff700 1 --2- 172.21.15.16:0/2402072202 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f914006fb20 0x7f914006ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:15.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.614+0000 7f913f7fe700 1 --2- 172.21.15.16:0/2402072202 >> v2:172.21.15.16:3300/0 conn(0x7f9140130540 0x7f9140130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:15.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.615+0000 7f913ffff700 1 -- 172.21.15.16:0/2402072202 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9140130e50 msgr2=0x7f914006f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:15.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.615+0000 7f913ffff700 1 --2- 172.21.15.16:0/2402072202 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9140130e50 0x7f914006f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:15.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.615+0000 7f913ffff700 1 -- 172.21.15.16:0/2402072202 >> v2:172.21.15.16:3300/0 conn(0x7f9140130540 msgr2=0x7f9140130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:15.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.615+0000 7f913ffff700 1 --2- 172.21.15.16:0/2402072202 >> v2:172.21.15.16:3300/0 conn(0x7f9140130540 0x7f9140130910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:15.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.615+0000 7f913ffff700 1 -- 172.21.15.16:0/2402072202 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9140130340 con 0x7f914006fb20 2023-09-30T22:30:15.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.615+0000 7f913ffff700 1 --2- 172.21.15.16:0/2402072202 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f914006fb20 0x7f914006ffa0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9138009d90 tx=0x7f913800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1aa3526ac662f0e8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:15.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.615+0000 7f913e7fc700 1 -- 172.21.15.16:0/2402072202 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913800e070 con 0x7f914006fb20 2023-09-30T22:30:15.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.615+0000 7f913e7fc700 1 -- 172.21.15.16:0/2402072202 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f913800bb20 con 0x7f914006fb20 2023-09-30T22:30:15.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.615+0000 7f913e7fc700 1 -- 172.21.15.16:0/2402072202 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9138012430 con 0x7f914006fb20 2023-09-30T22:30:15.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.616+0000 7f91460fc700 1 -- 172.21.15.16:0/2402072202 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f914006fb20 msgr2=0x7f914006ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:15.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.616+0000 7f91460fc700 1 --2- 172.21.15.16:0/2402072202 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f914006fb20 0x7f914006ffa0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9138009d90 tx=0x7f913800b330 comp rx=0 tx=0).stop 2023-09-30T22:30:15.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.616+0000 7f91460fc700 1 -- 172.21.15.16:0/2402072202 shutdown_connections 2023-09-30T22:30:15.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.616+0000 7f91460fc700 1 --2- 172.21.15.16:0/2402072202 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f914006fb20 0x7f914006ffa0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:15.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.616+0000 7f91460fc700 1 --2- 172.21.15.16:0/2402072202 >> v2:172.21.15.16:3300/0 conn(0x7f9140130540 0x7f9140130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:15.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.616+0000 7f91460fc700 1 --2- 172.21.15.16:0/2402072202 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9140130e50 0x7f914006f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:15.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.616+0000 7f91460fc700 1 -- 172.21.15.16:0/2402072202 >> 172.21.15.16:0/2402072202 conn(0x7f91400a2aa0 msgr2=0x7f91400a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:15.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.616+0000 7f91460fc700 1 -- 172.21.15.16:0/2402072202 shutdown_connections 2023-09-30T22:30:15.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.616+0000 7f91460fc700 1 -- 172.21.15.16:0/2402072202 wait complete. 2023-09-30T22:30:15.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.617+0000 7f91460fc700 1 Processor -- start 2023-09-30T22:30:15.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.618+0000 7f91460fc700 1 -- start start 2023-09-30T22:30:15.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.618+0000 7f91460fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f914006fb20 0x7f914011ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:15.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.618+0000 7f91460fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9140130540 0x7f914011ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:15.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.618+0000 7f91460fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9140130e50 0x7f914011f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:15.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.618+0000 7f91460fc700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91401224d0 con 0x7f9140130540 2023-09-30T22:30:15.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.618+0000 7f91460fc700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9140120be0 con 0x7f914006fb20 2023-09-30T22:30:15.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.618+0000 7f91460fc700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f91400718e0 con 0x7f9140130e50 2023-09-30T22:30:15.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.618+0000 7f913ffff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9140130e50 0x7f914011f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:15.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.618+0000 7f913effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9140130540 0x7f914011ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:15.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.618+0000 7f913ffff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9140130e50 0x7f914011f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:51098/0 (socket says 172.21.15.16:51098) 2023-09-30T22:30:15.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.618+0000 7f913ffff700 1 -- 172.21.15.16:0/1228123826 learned_addr learned my addr 172.21.15.16:0/1228123826 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:15.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.619+0000 7f913f7fe700 1 --2- 172.21.15.16:0/1228123826 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f914006fb20 0x7f914011ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:15.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.619+0000 7f913f7fe700 1 -- 172.21.15.16:0/1228123826 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9140130e50 msgr2=0x7f914011f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:15.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.619+0000 7f913f7fe700 1 --2- 172.21.15.16:0/1228123826 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9140130e50 0x7f914011f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:15.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.619+0000 7f913f7fe700 1 -- 172.21.15.16:0/1228123826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9140130540 msgr2=0x7f914011ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:15.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.619+0000 7f913f7fe700 1 --2- 172.21.15.16:0/1228123826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9140130540 0x7f914011ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:15.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.619+0000 7f913f7fe700 1 -- 172.21.15.16:0/1228123826 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9138009a70 con 0x7f914006fb20 2023-09-30T22:30:15.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.619+0000 7f913ffff700 1 --2- 172.21.15.16:0/1228123826 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9140130e50 0x7f914011f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:15.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.619+0000 7f913effd700 1 --2- 172.21.15.16:0/1228123826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9140130540 0x7f914011ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:15.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.619+0000 7f913f7fe700 1 --2- 172.21.15.16:0/1228123826 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f914006fb20 0x7f914011ea40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f913000e7e0 tx=0x7f913000eaf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:15.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.619+0000 7f9127fff700 1 -- 172.21.15.16:0/1228123826 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913000c070 con 0x7f914006fb20 2023-09-30T22:30:15.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.619+0000 7f9127fff700 1 -- 172.21.15.16:0/1228123826 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9130013610 con 0x7f914006fb20 2023-09-30T22:30:15.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.620+0000 7f91460fc700 1 -- 172.21.15.16:0/1228123826 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91401205c0 con 0x7f914006fb20 2023-09-30T22:30:15.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.620+0000 7f9127fff700 1 -- 172.21.15.16:0/1228123826 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91300139a0 con 0x7f914006fb20 2023-09-30T22:30:15.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.620+0000 7f91460fc700 1 -- 172.21.15.16:0/1228123826 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9140122c60 con 0x7f914006fb20 2023-09-30T22:30:15.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.621+0000 7f91460fc700 1 -- 172.21.15.16:0/1228123826 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9140074ea0 con 0x7f914006fb20 2023-09-30T22:30:15.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.621+0000 7f9127fff700 1 -- 172.21.15.16:0/1228123826 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9130013b00 con 0x7f914006fb20 2023-09-30T22:30:15.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.621+0000 7f9127fff700 1 -- 172.21.15.16:0/1228123826 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f91300408a0 con 0x7f914006fb20 2023-09-30T22:30:15.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.626+0000 7f9127fff700 1 -- 172.21.15.16:0/1228123826 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9130044060 con 0x7f914006fb20 2023-09-30T22:30:15.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.651+0000 7fcc1fe43700 1 Processor -- start 2023-09-30T22:30:15.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.652+0000 7fcc1fe43700 1 -- start start 2023-09-30T22:30:15.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.652+0000 7fcc1fe43700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc181338a0 0x7fcc18133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:15.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.652+0000 7fcc1fe43700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcc1809de50 0x7fcc1809e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:15.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.652+0000 7fcc1fe43700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcc1809e760 0x7fcc1813a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:15.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.652+0000 7fcc1fe43700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc1806fef0 con 0x7fcc1809e760 2023-09-30T22:30:15.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.652+0000 7fcc1fe43700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1806ef30 con 0x7fcc1809de50 2023-09-30T22:30:15.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.652+0000 7fcc1fe43700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcc181334b0 con 0x7fcc181338a0 2023-09-30T22:30:15.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.652+0000 7fcc1dbdf700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc181338a0 0x7fcc18133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:15.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.653+0000 7fcc1dbdf700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc181338a0 0x7fcc18133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:51108/0 (socket says 172.21.15.16:51108) 2023-09-30T22:30:15.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.653+0000 7fcc1dbdf700 1 -- 172.21.15.16:0/3941197119 learned_addr learned my addr 172.21.15.16:0/3941197119 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:15.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.653+0000 7fcc1e3e0700 1 --2- 172.21.15.16:0/3941197119 >> v2:172.21.15.16:3300/0 conn(0x7fcc1809e760 0x7fcc1813a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:15.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.653+0000 7fcc1d3de700 1 --2- 172.21.15.16:0/3941197119 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcc1809de50 0x7fcc1809e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:15.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.653+0000 7fcc1e3e0700 1 -- 172.21.15.16:0/3941197119 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc181338a0 msgr2=0x7fcc18133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:15.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.653+0000 7fcc1e3e0700 1 --2- 172.21.15.16:0/3941197119 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc181338a0 0x7fcc18133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:15.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.653+0000 7fcc1e3e0700 1 -- 172.21.15.16:0/3941197119 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcc1809de50 msgr2=0x7fcc1809e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:15.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.653+0000 7fcc1e3e0700 1 --2- 172.21.15.16:0/3941197119 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcc1809de50 0x7fcc1809e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:15.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.653+0000 7fcc1e3e0700 1 -- 172.21.15.16:0/3941197119 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc1806fc30 con 0x7fcc1809e760 2023-09-30T22:30:15.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.653+0000 7fcc1e3e0700 1 --2- 172.21.15.16:0/3941197119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc1809e760 0x7fcc1813a730 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fcc10009d90 tx=0x7fcc1000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfb03a2c16b8682e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:15.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.653+0000 7fcc1cbdd700 1 -- 172.21.15.16:0/3941197119 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1000e070 con 0x7fcc1809e760 2023-09-30T22:30:15.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.654+0000 7fcc1cbdd700 1 -- 172.21.15.16:0/3941197119 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc1000bb20 con 0x7fcc1809e760 2023-09-30T22:30:15.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.654+0000 7fcc1cbdd700 1 -- 172.21.15.16:0/3941197119 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc10012430 con 0x7fcc1809e760 2023-09-30T22:30:15.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.654+0000 7fcc1fe43700 1 -- 172.21.15.16:0/3941197119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc1809e760 msgr2=0x7fcc1813a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:15.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.654+0000 7fcc1fe43700 1 --2- 172.21.15.16:0/3941197119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc1809e760 0x7fcc1813a730 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fcc10009d90 tx=0x7fcc1000b330 comp rx=0 tx=0).stop 2023-09-30T22:30:15.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.654+0000 7fcc1fe43700 1 -- 172.21.15.16:0/3941197119 shutdown_connections 2023-09-30T22:30:15.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.654+0000 7fcc1fe43700 1 --2- 172.21.15.16:0/3941197119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc1809e760 0x7fcc1813a730 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:15.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.654+0000 7fcc1fe43700 1 --2- 172.21.15.16:0/3941197119 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc181338a0 0x7fcc18133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:15.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.654+0000 7fcc1fe43700 1 --2- 172.21.15.16:0/3941197119 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcc1809de50 0x7fcc1809e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:15.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.654+0000 7fcc1fe43700 1 -- 172.21.15.16:0/3941197119 >> 172.21.15.16:0/3941197119 conn(0x7fcc180a2aa0 msgr2=0x7fcc180a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:15.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.655+0000 7fcc1fe43700 1 -- 172.21.15.16:0/3941197119 shutdown_connections 2023-09-30T22:30:15.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.655+0000 7fcc1fe43700 1 -- 172.21.15.16:0/3941197119 wait complete. 2023-09-30T22:30:15.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.656+0000 7fcc1fe43700 1 Processor -- start 2023-09-30T22:30:15.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.656+0000 7fcc1fe43700 1 -- start start 2023-09-30T22:30:15.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.656+0000 7fcc1fe43700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcc1809de50 0x7fcc1811b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:15.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.656+0000 7fcc1fe43700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc1809e760 0x7fcc1811bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:15.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.656+0000 7fcc1fe43700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc181338a0 0x7fcc1811c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:15.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.656+0000 7fcc1fe43700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1811f100 con 0x7fcc1809e760 2023-09-30T22:30:15.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.656+0000 7fcc1fe43700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1811d810 con 0x7fcc1809de50 2023-09-30T22:30:15.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.656+0000 7fcc1fe43700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1811faa0 con 0x7fcc181338a0 2023-09-30T22:30:15.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.656+0000 7fcc1dbdf700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcc1809de50 0x7fcc1811b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:15.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.656+0000 7fcc1dbdf700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcc1809de50 0x7fcc1811b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41618/0 (socket says 172.21.15.16:41618) 2023-09-30T22:30:15.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.656+0000 7fcc1dbdf700 1 -- 172.21.15.16:0/1421966080 learned_addr learned my addr 172.21.15.16:0/1421966080 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:15.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.656+0000 7fcc1d3de700 1 --2- 172.21.15.16:0/1421966080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc1809e760 0x7fcc1811bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:15.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.657+0000 7fcc1e3e0700 1 --2- 172.21.15.16:0/1421966080 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc181338a0 0x7fcc1811c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:15.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.657+0000 7fcc1dbdf700 1 -- 172.21.15.16:0/1421966080 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc181338a0 msgr2=0x7fcc1811c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:15.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.657+0000 7fcc1dbdf700 1 --2- 172.21.15.16:0/1421966080 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc181338a0 0x7fcc1811c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:15.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.657+0000 7fcc1dbdf700 1 -- 172.21.15.16:0/1421966080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc1809e760 msgr2=0x7fcc1811bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:15.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.657+0000 7fcc1dbdf700 1 --2- 172.21.15.16:0/1421966080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc1809e760 0x7fcc1811bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:15.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.657+0000 7fcc1dbdf700 1 -- 172.21.15.16:0/1421966080 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc10009a70 con 0x7fcc1809de50 2023-09-30T22:30:15.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.657+0000 7fcc1d3de700 1 --2- 172.21.15.16:0/1421966080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc1809e760 0x7fcc1811bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:15.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.657+0000 7fcc1dbdf700 1 --2- 172.21.15.16:0/1421966080 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcc1809de50 0x7fcc1811b5e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fcc1400e940 tx=0x7fcc1400ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:15.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.657+0000 7fcc0e7fc700 1 -- 172.21.15.16:0/1421966080 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc140096a0 con 0x7fcc1809de50 2023-09-30T22:30:15.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.657+0000 7fcc0e7fc700 1 -- 172.21.15.16:0/1421966080 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc14013680 con 0x7fcc1809de50 2023-09-30T22:30:15.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.657+0000 7fcc1fe43700 1 -- 172.21.15.16:0/1421966080 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc1806f640 con 0x7fcc1809de50 2023-09-30T22:30:15.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.657+0000 7fcc0e7fc700 1 -- 172.21.15.16:0/1421966080 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc14013a30 con 0x7fcc1809de50 2023-09-30T22:30:15.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.657+0000 7fcc1fe43700 1 -- 172.21.15.16:0/1421966080 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc1811e8e0 con 0x7fcc1809de50 2023-09-30T22:30:15.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.658+0000 7fcc1fe43700 1 -- 172.21.15.16:0/1421966080 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc18074ea0 con 0x7fcc1809de50 2023-09-30T22:30:15.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.659+0000 7fcc0e7fc700 1 -- 172.21.15.16:0/1421966080 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcc14013b90 con 0x7fcc1809de50 2023-09-30T22:30:15.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.660+0000 7fcc0e7fc700 1 -- 172.21.15.16:0/1421966080 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fcc1403fd90 con 0x7fcc1809de50 2023-09-30T22:30:15.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.664+0000 7fcc0e7fc700 1 -- 172.21.15.16:0/1421966080 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcc140253d0 con 0x7fcc1809de50 2023-09-30T22:30:15.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.721+0000 7fa78f41c700 1 Processor -- start 2023-09-30T22:30:15.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.721+0000 7fa78f41c700 1 -- start start 2023-09-30T22:30:15.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.722+0000 7fa78f41c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa788131630 0x7fa788131a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:15.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.722+0000 7fa78f41c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa788131f40 0x7fa78806e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:15.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.722+0000 7fa78f41c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa788071a00 con 0x7fa78806e9d0 2023-09-30T22:30:15.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.722+0000 7fa78f41c700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa788131430 con 0x7fa788131630 2023-09-30T22:30:15.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.722+0000 7fa78f41c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa78809a510 con 0x7fa788131f40 2023-09-30T22:30:15.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.722+0000 7fa78c9b7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa788131f40 0x7fa78806e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:15.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.722+0000 7fa78c9b7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa788131f40 0x7fa78806e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41628/0 (socket says 172.21.15.16:41628) 2023-09-30T22:30:15.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.722+0000 7fa78c9b7700 1 -- 172.21.15.16:0/4001838889 learned_addr learned my addr 172.21.15.16:0/4001838889 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:15.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.722+0000 7fa78c9b7700 1 -- 172.21.15.16:0/4001838889 >> v2:172.21.15.16:3300/0 conn(0x7fa788131630 msgr2=0x7fa788131a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:15.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.722+0000 7fa78c9b7700 1 --2- 172.21.15.16:0/4001838889 >> v2:172.21.15.16:3300/0 conn(0x7fa788131630 0x7fa788131a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:15.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.722+0000 7fa78c9b7700 1 -- 172.21.15.16:0/4001838889 >> v1:172.21.15.16:6789/0 conn(0x7fa78806e9d0 legacy=0x7fa788138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:15.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.722+0000 7fa78c9b7700 1 -- 172.21.15.16:0/4001838889 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7880721c0 con 0x7fa788131f40 2023-09-30T22:30:15.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.723+0000 7fa78c9b7700 1 --2- 172.21.15.16:0/4001838889 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa788131f40 0x7fa78806e490 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa7780098f0 tx=0x7fa778009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2df0735152eb0f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:15.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.723+0000 7fa77ffff700 1 -- 172.21.15.16:0/4001838889 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa778004230 con 0x7fa788131f40 2023-09-30T22:30:15.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.723+0000 7fa77ffff700 1 -- 172.21.15.16:0/4001838889 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa778004390 con 0x7fa788131f40 2023-09-30T22:30:15.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.723+0000 7fa77ffff700 1 -- 172.21.15.16:0/4001838889 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7780036a0 con 0x7fa788131f40 2023-09-30T22:30:15.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.724+0000 7fa78f41c700 1 -- 172.21.15.16:0/4001838889 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa788131f40 msgr2=0x7fa78806e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:15.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.724+0000 7fa78f41c700 1 --2- 172.21.15.16:0/4001838889 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa788131f40 0x7fa78806e490 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa7780098f0 tx=0x7fa778009c00 comp rx=0 tx=0).stop 2023-09-30T22:30:15.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.725+0000 7fa78f41c700 1 -- 172.21.15.16:0/4001838889 shutdown_connections 2023-09-30T22:30:15.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.725+0000 7fa78f41c700 1 --2- 172.21.15.16:0/4001838889 >> v2:172.21.15.16:3300/0 conn(0x7fa788131630 0x7fa788131a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:15.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.725+0000 7fa78f41c700 1 --2- 172.21.15.16:0/4001838889 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa788131f40 0x7fa78806e490 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:15.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.725+0000 7fa78f41c700 1 -- 172.21.15.16:0/4001838889 >> 172.21.15.16:0/4001838889 conn(0x7fa7880a2aa0 msgr2=0x7fa7880a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:15.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.725+0000 7fa78f41c700 1 -- 172.21.15.16:0/4001838889 shutdown_connections 2023-09-30T22:30:15.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.725+0000 7fa78f41c700 1 -- 172.21.15.16:0/4001838889 wait complete. 2023-09-30T22:30:15.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.726+0000 7fa78f41c700 1 Processor -- start 2023-09-30T22:30:15.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.726+0000 7fa78f41c700 1 -- start start 2023-09-30T22:30:15.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.727+0000 7fa78f41c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa78806e9d0 0x7fa78806f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:15.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.727+0000 7fa78f41c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa788131630 0x7fa78806fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:15.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.727+0000 7fa78f41c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa788131f40 0x7fa788070060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:15.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.727+0000 7fa78f41c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa78811e110 con 0x7fa788131630 2023-09-30T22:30:15.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.727+0000 7fa78f41c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa78811d2a0 con 0x7fa788131f40 2023-09-30T22:30:15.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.727+0000 7fa78f41c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa78811e310 con 0x7fa78806e9d0 2023-09-30T22:30:15.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.727+0000 7fa78d9b9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa788131f40 0x7fa788070060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:15.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.727+0000 7fa78c9b7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa788131630 0x7fa78806fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:15.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.727+0000 7fa78d9b9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa788131f40 0x7fa788070060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41634/0 (socket says 172.21.15.16:41634) 2023-09-30T22:30:15.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.727+0000 7fa78d9b9700 1 -- 172.21.15.16:0/2159882493 learned_addr learned my addr 172.21.15.16:0/2159882493 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:15.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.727+0000 7fa78d1b8700 1 --2- 172.21.15.16:0/2159882493 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa78806e9d0 0x7fa78806f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:15.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.727+0000 7fa78d9b9700 1 -- 172.21.15.16:0/2159882493 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa78806e9d0 msgr2=0x7fa78806f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:15.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.727+0000 7fa78d9b9700 1 --2- 172.21.15.16:0/2159882493 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa78806e9d0 0x7fa78806f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:15.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.727+0000 7fa78d9b9700 1 -- 172.21.15.16:0/2159882493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa788131630 msgr2=0x7fa78806fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:15.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.727+0000 7fa78d9b9700 1 --2- 172.21.15.16:0/2159882493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa788131630 0x7fa78806fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:15.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.727+0000 7fa78d9b9700 1 -- 172.21.15.16:0/2159882493 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa778009550 con 0x7fa788131f40 2023-09-30T22:30:15.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.727+0000 7fa78d9b9700 1 --2- 172.21.15.16:0/2159882493 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa788131f40 0x7fa788070060 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa78000c7a0 tx=0x7fa78000cab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:15.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.728+0000 7fa77dffb700 1 -- 172.21.15.16:0/2159882493 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa780003e10 con 0x7fa788131f40 2023-09-30T22:30:15.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.728+0000 7fa77dffb700 1 -- 172.21.15.16:0/2159882493 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa780005070 con 0x7fa788131f40 2023-09-30T22:30:15.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.728+0000 7fa78f41c700 1 -- 172.21.15.16:0/2159882493 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7880a1d10 con 0x7fa788131f40 2023-09-30T22:30:15.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.728+0000 7fa78f41c700 1 -- 172.21.15.16:0/2159882493 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa78812bbe0 con 0x7fa788131f40 2023-09-30T22:30:15.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.728+0000 7fa77dffb700 1 -- 172.21.15.16:0/2159882493 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa780005420 con 0x7fa788131f40 2023-09-30T22:30:15.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.729+0000 7fa77dffb700 1 -- 172.21.15.16:0/2159882493 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa780005580 con 0x7fa788131f40 2023-09-30T22:30:15.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.730+0000 7fa77dffb700 1 -- 172.21.15.16:0/2159882493 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fa78003ecd0 con 0x7fa788131f40 2023-09-30T22:30:15.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.730+0000 7fa78f41c700 1 -- 172.21.15.16:0/2159882493 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7680052e0 con 0x7fa788131f40 2023-09-30T22:30:15.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.739+0000 7fa77dffb700 1 -- 172.21.15.16:0/2159882493 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa780017070 con 0x7fa788131f40 2023-09-30T22:30:15.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.750+0000 7fd182bfc700 1 Processor -- start 2023-09-30T22:30:15.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.750+0000 7fd182bfc700 1 -- start start 2023-09-30T22:30:15.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.750+0000 7fd182bfc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd17c06a770 0x7fd17c09d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:15.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.750+0000 7fd182bfc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd17c13d1f0 0x7fd17c13f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:15.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.750+0000 7fd182bfc700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd17c09a810 con 0x7fd17c09dc30 2023-09-30T22:30:15.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.750+0000 7fd182bfc700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd17c06ab40 con 0x7fd17c06a770 2023-09-30T22:30:15.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.750+0000 7fd182bfc700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c09ebc0 con 0x7fd17c13d1f0 2023-09-30T22:30:15.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.751+0000 7fd181199700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd17c13d1f0 0x7fd17c13f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:15.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.751+0000 7fd181199700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd17c13d1f0 0x7fd17c13f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41642/0 (socket says 172.21.15.16:41642) 2023-09-30T22:30:15.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.751+0000 7fd181199700 1 -- 172.21.15.16:0/3802920598 learned_addr learned my addr 172.21.15.16:0/3802920598 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:15.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.751+0000 7fd180998700 1 --2- 172.21.15.16:0/3802920598 >> v2:172.21.15.16:3300/0 conn(0x7fd17c06a770 0x7fd17c09d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:15.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.751+0000 7fd181199700 1 -- 172.21.15.16:0/3802920598 >> v2:172.21.15.16:3300/0 conn(0x7fd17c06a770 msgr2=0x7fd17c09d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:15.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.751+0000 7fd181199700 1 --2- 172.21.15.16:0/3802920598 >> v2:172.21.15.16:3300/0 conn(0x7fd17c06a770 0x7fd17c09d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:15.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.751+0000 7fd181199700 1 -- 172.21.15.16:0/3802920598 >> v1:172.21.15.16:6789/0 conn(0x7fd17c09dc30 legacy=0x7fd17c09e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:15.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.751+0000 7fd181199700 1 -- 172.21.15.16:0/3802920598 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd17c09e920 con 0x7fd17c13d1f0 2023-09-30T22:30:15.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.751+0000 7fd180998700 1 --2- 172.21.15.16:0/3802920598 >> v2:172.21.15.16:3300/0 conn(0x7fd17c06a770 0x7fd17c09d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:30:15.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.751+0000 7fd181199700 1 --2- 172.21.15.16:0/3802920598 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd17c13d1f0 0x7fd17c13f5d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd174009e50 tx=0x7fd17400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d06fd8917c2a74da server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:15.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.751+0000 7fd17b7fe700 1 -- 172.21.15.16:0/3802920598 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17400e070 con 0x7fd17c13d1f0 2023-09-30T22:30:15.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.752+0000 7fd17b7fe700 1 -- 172.21.15.16:0/3802920598 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd17400b9c0 con 0x7fd17c13d1f0 2023-09-30T22:30:15.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.752+0000 7fd17b7fe700 1 -- 172.21.15.16:0/3802920598 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd17400bd70 con 0x7fd17c13d1f0 2023-09-30T22:30:15.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.752+0000 7fd182bfc700 1 -- 172.21.15.16:0/3802920598 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd17c13d1f0 msgr2=0x7fd17c13f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:15.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.752+0000 7fd182bfc700 1 --2- 172.21.15.16:0/3802920598 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd17c13d1f0 0x7fd17c13f5d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd174009e50 tx=0x7fd17400b330 comp rx=0 tx=0).stop 2023-09-30T22:30:15.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.752+0000 7fd182bfc700 1 -- 172.21.15.16:0/3802920598 shutdown_connections 2023-09-30T22:30:15.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.752+0000 7fd182bfc700 1 --2- 172.21.15.16:0/3802920598 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd17c13d1f0 0x7fd17c13f5d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:15.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.752+0000 7fd182bfc700 1 --2- 172.21.15.16:0/3802920598 >> v2:172.21.15.16:3300/0 conn(0x7fd17c06a770 0x7fd17c09d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:15.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.752+0000 7fd182bfc700 1 -- 172.21.15.16:0/3802920598 >> 172.21.15.16:0/3802920598 conn(0x7fd17c066f50 msgr2=0x7fd17c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:15.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.752+0000 7fd182bfc700 1 -- 172.21.15.16:0/3802920598 shutdown_connections 2023-09-30T22:30:15.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.752+0000 7fd182bfc700 1 -- 172.21.15.16:0/3802920598 wait complete. 2023-09-30T22:30:15.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.753+0000 7fd182bfc700 1 Processor -- start 2023-09-30T22:30:15.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.754+0000 7fd182bfc700 1 -- start start 2023-09-30T22:30:15.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.754+0000 7fd182bfc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd17c06a770 0x7fd17c11c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:15.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.754+0000 7fd182bfc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd17c09dc30 0x7fd17c13b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:15.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.754+0000 7fd182bfc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd17c13d1f0 0x7fd17c13bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:15.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.754+0000 7fd182bfc700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c13d070 con 0x7fd17c06a770 2023-09-30T22:30:15.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.754+0000 7fd182bfc700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c13f6e0 con 0x7fd17c13d1f0 2023-09-30T22:30:15.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.754+0000 7fd182bfc700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c11ea90 con 0x7fd17c09dc30 2023-09-30T22:30:15.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.755+0000 7fd17bfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd17c09dc30 0x7fd17c13b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:15.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.755+0000 7fd17bfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd17c09dc30 0x7fd17c13b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:51132/0 (socket says 172.21.15.16:51132) 2023-09-30T22:30:15.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.755+0000 7fd17bfff700 1 -- 172.21.15.16:0/3371589587 learned_addr learned my addr 172.21.15.16:0/3371589587 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:15.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.755+0000 7fd181199700 1 --2- 172.21.15.16:0/3371589587 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd17c13d1f0 0x7fd17c13bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:15.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.755+0000 7fd180998700 1 --2- 172.21.15.16:0/3371589587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd17c06a770 0x7fd17c11c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:15.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.755+0000 7fd17bfff700 1 -- 172.21.15.16:0/3371589587 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd17c13d1f0 msgr2=0x7fd17c13bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:15.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.755+0000 7fd17bfff700 1 --2- 172.21.15.16:0/3371589587 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd17c13d1f0 0x7fd17c13bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:15.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.755+0000 7fd17bfff700 1 -- 172.21.15.16:0/3371589587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd17c06a770 msgr2=0x7fd17c11c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:15.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.755+0000 7fd17bfff700 1 --2- 172.21.15.16:0/3371589587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd17c06a770 0x7fd17c11c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:15.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.755+0000 7fd17bfff700 1 -- 172.21.15.16:0/3371589587 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd174009b30 con 0x7fd17c09dc30 2023-09-30T22:30:15.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.755+0000 7fd17bfff700 1 --2- 172.21.15.16:0/3371589587 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd17c09dc30 0x7fd17c13b6a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd16800c4e0 tx=0x7fd16800c7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:15.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.756+0000 7fd1797fa700 1 -- 172.21.15.16:0/3371589587 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd168011790 con 0x7fd17c09dc30 2023-09-30T22:30:15.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.756+0000 7fd1797fa700 1 -- 172.21.15.16:0/3371589587 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1680118f0 con 0x7fd17c09dc30 2023-09-30T22:30:15.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.756+0000 7fd1797fa700 1 -- 172.21.15.16:0/3371589587 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd168011c60 con 0x7fd17c09dc30 2023-09-30T22:30:15.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.756+0000 7fd182bfc700 1 -- 172.21.15.16:0/3371589587 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd17c11f030 con 0x7fd17c09dc30 2023-09-30T22:30:15.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.756+0000 7fd182bfc700 1 -- 172.21.15.16:0/3371589587 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd17c11e370 con 0x7fd17c09dc30 2023-09-30T22:30:15.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.757+0000 7fd1797fa700 1 -- 172.21.15.16:0/3371589587 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd16801c460 con 0x7fd17c09dc30 2023-09-30T22:30:15.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.757+0000 7fd182bfc700 1 -- 172.21.15.16:0/3371589587 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd17c074ea0 con 0x7fd17c09dc30 2023-09-30T22:30:15.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.757+0000 7fd1797fa700 1 -- 172.21.15.16:0/3371589587 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd16803f300 con 0x7fd17c09dc30 2023-09-30T22:30:15.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.766+0000 7fd1797fa700 1 -- 172.21.15.16:0/3371589587 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd1680108d0 con 0x7fd17c09dc30 2023-09-30T22:30:15.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.876+0000 7fcc1fe43700 1 -- 172.21.15.16:0/1421966080 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fcc1811ec60 con 0x7fcc1809de50 2023-09-30T22:30:15.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.877+0000 7fcc0e7fc700 1 -- 172.21.15.16:0/1421966080 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcc14017070 con 0x7fcc1809de50 2023-09-30T22:30:15.878 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:30:15.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.878+0000 7f91460fc700 1 -- 172.21.15.16:0/1228123826 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9140070b80 con 0x7f914006fb20 2023-09-30T22:30:15.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.879+0000 7f9127fff700 1 -- 172.21.15.16:0/1228123826 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9130044070 con 0x7f914006fb20 2023-09-30T22:30:15.880 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:30:15.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.880+0000 7fcc1fe43700 1 -- 172.21.15.16:0/1421966080 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcc1809de50 msgr2=0x7fcc1811b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:15.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.880+0000 7fcc1fe43700 1 --2- 172.21.15.16:0/1421966080 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcc1809de50 0x7fcc1811b5e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fcc1400e940 tx=0x7fcc1400ec50 comp rx=0 tx=0).stop 2023-09-30T22:30:15.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.880+0000 7fcc1fe43700 1 -- 172.21.15.16:0/1421966080 shutdown_connections 2023-09-30T22:30:15.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.880+0000 7fcc1fe43700 1 --2- 172.21.15.16:0/1421966080 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcc1809de50 0x7fcc1811b5e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:15.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.880+0000 7fcc1fe43700 1 --2- 172.21.15.16:0/1421966080 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc181338a0 0x7fcc1811c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:15.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.880+0000 7fcc1fe43700 1 --2- 172.21.15.16:0/1421966080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc1809e760 0x7fcc1811bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:15.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.880+0000 7fcc1fe43700 1 -- 172.21.15.16:0/1421966080 >> 172.21.15.16:0/1421966080 conn(0x7fcc180a2aa0 msgr2=0x7fcc181306c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:15.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.880+0000 7fcc1fe43700 1 -- 172.21.15.16:0/1421966080 shutdown_connections 2023-09-30T22:30:15.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.880+0000 7fcc1fe43700 1 -- 172.21.15.16:0/1421966080 wait complete. 2023-09-30T22:30:15.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.882+0000 7f91460fc700 1 -- 172.21.15.16:0/1228123826 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f914006fb20 msgr2=0x7f914011ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:15.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.882+0000 7f91460fc700 1 --2- 172.21.15.16:0/1228123826 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f914006fb20 0x7f914011ea40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f913000e7e0 tx=0x7f913000eaf0 comp rx=0 tx=0).stop 2023-09-30T22:30:15.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.883+0000 7f91460fc700 1 -- 172.21.15.16:0/1228123826 shutdown_connections 2023-09-30T22:30:15.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.883+0000 7f91460fc700 1 --2- 172.21.15.16:0/1228123826 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f914006fb20 0x7f914011ea40 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:15.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.883+0000 7f91460fc700 1 --2- 172.21.15.16:0/1228123826 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9140130e50 0x7f914011f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:15.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.883+0000 7f91460fc700 1 --2- 172.21.15.16:0/1228123826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9140130540 0x7f914011ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:15.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.883+0000 7f91460fc700 1 -- 172.21.15.16:0/1228123826 >> 172.21.15.16:0/1228123826 conn(0x7f91400a2aa0 msgr2=0x7f914012d470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:15.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.883+0000 7f91460fc700 1 -- 172.21.15.16:0/1228123826 shutdown_connections 2023-09-30T22:30:15.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.883+0000 7f91460fc700 1 -- 172.21.15.16:0/1228123826 wait complete. 2023-09-30T22:30:15.896 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:30:15.905 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:30:15.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.970+0000 7fa78f41c700 1 -- 172.21.15.16:0/2159882493 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa768005590 con 0x7fa788131f40 2023-09-30T22:30:15.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.970+0000 7fa77dffb700 1 -- 172.21.15.16:0/2159882493 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa780019cd0 con 0x7fa788131f40 2023-09-30T22:30:15.972 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:30:15.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.972+0000 7fa78f41c700 1 -- 172.21.15.16:0/2159882493 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa788131f40 msgr2=0x7fa788070060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:15.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.972+0000 7fa78f41c700 1 --2- 172.21.15.16:0/2159882493 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa788131f40 0x7fa788070060 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa78000c7a0 tx=0x7fa78000cab0 comp rx=0 tx=0).stop 2023-09-30T22:30:15.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.972+0000 7fa78f41c700 1 -- 172.21.15.16:0/2159882493 shutdown_connections 2023-09-30T22:30:15.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.972+0000 7fa78f41c700 1 --2- 172.21.15.16:0/2159882493 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa788131f40 0x7fa788070060 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:15.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.972+0000 7fa78f41c700 1 --2- 172.21.15.16:0/2159882493 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa78806e9d0 0x7fa78806f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:15.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.972+0000 7fa78f41c700 1 --2- 172.21.15.16:0/2159882493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa788131630 0x7fa78806fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:15.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.972+0000 7fa78f41c700 1 -- 172.21.15.16:0/2159882493 >> 172.21.15.16:0/2159882493 conn(0x7fa7880a2aa0 msgr2=0x7fa78812e710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:15.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.972+0000 7fa78f41c700 1 -- 172.21.15.16:0/2159882493 shutdown_connections 2023-09-30T22:30:15.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:15.972+0000 7fa78f41c700 1 -- 172.21.15.16:0/2159882493 wait complete. 2023-09-30T22:30:15.988 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:30:16.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:16.004+0000 7fd182bfc700 1 -- 172.21.15.16:0/3371589587 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd17c09a3d0 con 0x7fd17c09dc30 2023-09-30T22:30:16.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:16.005+0000 7fd1797fa700 1 -- 172.21.15.16:0/3371589587 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd1680102e0 con 0x7fd17c09dc30 2023-09-30T22:30:16.007 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:30:16.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:16.007+0000 7fd182bfc700 1 -- 172.21.15.16:0/3371589587 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd17c09dc30 msgr2=0x7fd17c13b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:16.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:16.007+0000 7fd182bfc700 1 --2- 172.21.15.16:0/3371589587 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd17c09dc30 0x7fd17c13b6a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd16800c4e0 tx=0x7fd16800c7f0 comp rx=0 tx=0).stop 2023-09-30T22:30:16.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:16.007+0000 7fd182bfc700 1 -- 172.21.15.16:0/3371589587 shutdown_connections 2023-09-30T22:30:16.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:16.007+0000 7fd182bfc700 1 --2- 172.21.15.16:0/3371589587 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd17c13d1f0 0x7fd17c13bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:16.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:16.007+0000 7fd182bfc700 1 --2- 172.21.15.16:0/3371589587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd17c06a770 0x7fd17c11c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:16.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:16.007+0000 7fd182bfc700 1 --2- 172.21.15.16:0/3371589587 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd17c09dc30 0x7fd17c13b6a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:16.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:16.007+0000 7fd182bfc700 1 -- 172.21.15.16:0/3371589587 >> 172.21.15.16:0/3371589587 conn(0x7fd17c066f50 msgr2=0x7fd17c09b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:16.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:16.007+0000 7fd182bfc700 1 -- 172.21.15.16:0/3371589587 shutdown_connections 2023-09-30T22:30:16.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:16.007+0000 7fd182bfc700 1 -- 172.21.15.16:0/3371589587 wait complete. 2023-09-30T22:30:16.022 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:30:16.897 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:16.906 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:16.988 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:17.023 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:17.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.059+0000 7fac03a26700 1 Processor -- start 2023-09-30T22:30:17.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.060+0000 7fac03a26700 1 -- start start 2023-09-30T22:30:17.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.060+0000 7fac03a26700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fabfc09d6f0 0x7fabfc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:17.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.060+0000 7fac03a26700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fabfc09e000 0x7fabfc1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:17.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.060+0000 7fac03a26700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fabfc09e410 0x7fabfc133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:17.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.060+0000 7fac03a26700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabfc06b460 con 0x7fabfc09d6f0 2023-09-30T22:30:17.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.060+0000 7fac03a26700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabfc09b120 con 0x7fabfc09e000 2023-09-30T22:30:17.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.060+0000 7fac03a26700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fabfc09a7e0 con 0x7fabfc09e410 2023-09-30T22:30:17.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.061+0000 7fac01fc3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fabfc09e410 0x7fabfc133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:17.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.061+0000 7fac00fc1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fabfc09e000 0x7fabfc1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:17.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.061+0000 7fac01fc3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fabfc09e410 0x7fabfc133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:51136/0 (socket says 172.21.15.16:51136) 2023-09-30T22:30:17.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.061+0000 7fac01fc3700 1 -- 172.21.15.16:0/2024268498 learned_addr learned my addr 172.21.15.16:0/2024268498 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:17.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.061+0000 7fac017c2700 1 --2- 172.21.15.16:0/2024268498 >> v2:172.21.15.16:3300/0 conn(0x7fabfc09d6f0 0x7fabfc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:17.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.061+0000 7fac00fc1700 1 -- 172.21.15.16:0/2024268498 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fabfc09e410 msgr2=0x7fabfc133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:17.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.061+0000 7fac00fc1700 1 --2- 172.21.15.16:0/2024268498 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fabfc09e410 0x7fabfc133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.061+0000 7fac00fc1700 1 -- 172.21.15.16:0/2024268498 >> v2:172.21.15.16:3300/0 conn(0x7fabfc09d6f0 msgr2=0x7fabfc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:17.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.061+0000 7fac00fc1700 1 --2- 172.21.15.16:0/2024268498 >> v2:172.21.15.16:3300/0 conn(0x7fabfc09d6f0 0x7fabfc09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.061+0000 7fac00fc1700 1 -- 172.21.15.16:0/2024268498 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabfc09a500 con 0x7fabfc09e000 2023-09-30T22:30:17.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.061+0000 7fac00fc1700 1 --2- 172.21.15.16:0/2024268498 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fabfc09e000 0x7fabfc1339f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fabec009870 tx=0x7fabec009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=463f4d79b2f2e046 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:17.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.062+0000 7fabf3fff700 1 -- 172.21.15.16:0/2024268498 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabec011830 con 0x7fabfc09e000 2023-09-30T22:30:17.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.062+0000 7fabf3fff700 1 -- 172.21.15.16:0/2024268498 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabec011990 con 0x7fabfc09e000 2023-09-30T22:30:17.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.062+0000 7fabf3fff700 1 -- 172.21.15.16:0/2024268498 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabec011d20 con 0x7fabfc09e000 2023-09-30T22:30:17.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.062+0000 7fac03a26700 1 -- 172.21.15.16:0/2024268498 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fabfc09e000 msgr2=0x7fabfc1339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:17.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.062+0000 7fac03a26700 1 --2- 172.21.15.16:0/2024268498 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fabfc09e000 0x7fabfc1339f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fabec009870 tx=0x7fabec009b80 comp rx=0 tx=0).stop 2023-09-30T22:30:17.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.063+0000 7fac03a26700 1 -- 172.21.15.16:0/2024268498 shutdown_connections 2023-09-30T22:30:17.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.063+0000 7fac03a26700 1 --2- 172.21.15.16:0/2024268498 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fabfc09e410 0x7fabfc133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.063+0000 7fac03a26700 1 --2- 172.21.15.16:0/2024268498 >> v2:172.21.15.16:3300/0 conn(0x7fabfc09d6f0 0x7fabfc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.063+0000 7fac03a26700 1 --2- 172.21.15.16:0/2024268498 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fabfc09e000 0x7fabfc1339f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.063+0000 7fac03a26700 1 -- 172.21.15.16:0/2024268498 >> 172.21.15.16:0/2024268498 conn(0x7fabfc066f50 msgr2=0x7fabfc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:17.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.063+0000 7fac03a26700 1 -- 172.21.15.16:0/2024268498 shutdown_connections 2023-09-30T22:30:17.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.063+0000 7fac03a26700 1 -- 172.21.15.16:0/2024268498 wait complete. 2023-09-30T22:30:17.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.064+0000 7fac03a26700 1 Processor -- start 2023-09-30T22:30:17.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.064+0000 7fac03a26700 1 -- start start 2023-09-30T22:30:17.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.065+0000 7fac03a26700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabfc09d6f0 0x7fabfc1a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:17.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.065+0000 7fac03a26700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fabfc09e410 0x7fabfc1a5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:17.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.065+0000 7fac03a26700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fabfc1a5ab0 0x7fabfc1ceb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:17.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.065+0000 7fac03a26700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabfc1a6500 con 0x7fabfc09d6f0 2023-09-30T22:30:17.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.065+0000 7fac03a26700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabfc1a5ee0 con 0x7fabfc1a5ab0 2023-09-30T22:30:17.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.065+0000 7fac03a26700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fabfc1a8b50 con 0x7fabfc09e410 2023-09-30T22:30:17.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.065+0000 7fac00fc1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fabfc09e410 0x7fabfc1a5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:17.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.065+0000 7fac01fc3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fabfc1a5ab0 0x7fabfc1ceb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:17.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.065+0000 7fac00fc1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fabfc09e410 0x7fabfc1a5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:51146/0 (socket says 172.21.15.16:51146) 2023-09-30T22:30:17.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.065+0000 7fac00fc1700 1 -- 172.21.15.16:0/2950062729 learned_addr learned my addr 172.21.15.16:0/2950062729 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:17.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.065+0000 7fac017c2700 1 --2- 172.21.15.16:0/2950062729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabfc09d6f0 0x7fabfc1a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:17.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.066+0000 7fac01fc3700 1 -- 172.21.15.16:0/2950062729 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fabfc09e410 msgr2=0x7fabfc1a5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:17.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.066+0000 7fac01fc3700 1 --2- 172.21.15.16:0/2950062729 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fabfc09e410 0x7fabfc1a5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.066+0000 7fac01fc3700 1 -- 172.21.15.16:0/2950062729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabfc09d6f0 msgr2=0x7fabfc1a5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:17.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.066+0000 7fac01fc3700 1 --2- 172.21.15.16:0/2950062729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabfc09d6f0 0x7fabfc1a5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.066+0000 7fac01fc3700 1 -- 172.21.15.16:0/2950062729 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabec0094a0 con 0x7fabfc1a5ab0 2023-09-30T22:30:17.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.066+0000 7fac017c2700 1 --2- 172.21.15.16:0/2950062729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabfc09d6f0 0x7fabfc1a5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:17.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.066+0000 7fac01fc3700 1 --2- 172.21.15.16:0/2950062729 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fabfc1a5ab0 0x7fabfc1ceb20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fabf400efc0 tx=0x7fabf400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:17.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.066+0000 7fabf1ffb700 1 -- 172.21.15.16:0/2950062729 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf4003c20 con 0x7fabfc1a5ab0 2023-09-30T22:30:17.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.066+0000 7fabf1ffb700 1 -- 172.21.15.16:0/2950062729 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabf400f040 con 0x7fabfc1a5ab0 2023-09-30T22:30:17.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.066+0000 7fac03a26700 1 -- 172.21.15.16:0/2950062729 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabfc1a7f20 con 0x7fabfc1a5ab0 2023-09-30T22:30:17.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.066+0000 7fabf1ffb700 1 -- 172.21.15.16:0/2950062729 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf4009e60 con 0x7fabfc1a5ab0 2023-09-30T22:30:17.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.067+0000 7fac03a26700 1 -- 172.21.15.16:0/2950062729 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabfc09c710 con 0x7fabfc1a5ab0 2023-09-30T22:30:17.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.067+0000 7fabf1ffb700 1 -- 172.21.15.16:0/2950062729 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fabf400c880 con 0x7fabfc1a5ab0 2023-09-30T22:30:17.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.068+0000 7fabf1ffb700 1 -- 172.21.15.16:0/2950062729 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fabf403f8c0 con 0x7fabfc1a5ab0 2023-09-30T22:30:17.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.068+0000 7fac03a26700 1 -- 172.21.15.16:0/2950062729 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabfc1a8510 con 0x7fabfc1a5ab0 2023-09-30T22:30:17.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.077+0000 7fabf1ffb700 1 -- 172.21.15.16:0/2950062729 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fabfc1a8510 con 0x7fabfc1a5ab0 2023-09-30T22:30:17.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.101+0000 7fd2a2623700 1 Processor -- start 2023-09-30T22:30:17.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.101+0000 7fd2a2623700 1 -- start start 2023-09-30T22:30:17.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.102+0000 7fd2a2623700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd29c09e930 0x7fd29c09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:17.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.102+0000 7fd2a2623700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd29c09f650 0x7fd29c131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:17.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.102+0000 7fd2a2623700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd29c070940 con 0x7fd29c09f240 2023-09-30T22:30:17.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.102+0000 7fd2a2623700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd29c06fd60 con 0x7fd29c09e930 2023-09-30T22:30:17.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.102+0000 7fd2a2623700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd29c06fa80 con 0x7fd29c09f650 2023-09-30T22:30:17.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.102+0000 7fd29b7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd29c09f240 0x7fd29c1317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48064/0 (socket says 172.21.15.16:48064) 2023-09-30T22:30:17.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.102+0000 7fd29bfff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd29c09e930 0x7fd29c09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:17.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.102+0000 7fd29b7fe700 1 -- 172.21.15.16:0/1356375289 learned_addr learned my addr 172.21.15.16:0/1356375289 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:30:17.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.102+0000 7fd2a0bc0700 1 --2- 172.21.15.16:0/1356375289 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd29c09f650 0x7fd29c131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:17.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.103+0000 7fd2a0bc0700 1 -- 172.21.15.16:0/1356375289 >> v2:172.21.15.16:3300/0 conn(0x7fd29c09e930 msgr2=0x7fd29c09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:17.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.103+0000 7fd2a0bc0700 1 --2- 172.21.15.16:0/1356375289 >> v2:172.21.15.16:3300/0 conn(0x7fd29c09e930 0x7fd29c09ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.103+0000 7fd2a0bc0700 1 -- 172.21.15.16:0/1356375289 >> v1:172.21.15.16:6789/0 conn(0x7fd29c09f240 legacy=0x7fd29c1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:17.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.103+0000 7fd29affd700 1 -- 172.21.15.16:0/1356375289 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 233367323 0 0) 0x7fd29c070940 con 0x7fd29c09f240 2023-09-30T22:30:17.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.103+0000 7fd2a0bc0700 1 -- 172.21.15.16:0/1356375289 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd29c09fb60 con 0x7fd29c09f650 2023-09-30T22:30:17.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.103+0000 7fd29bfff700 1 --2- 172.21.15.16:0/1356375289 >> v2:172.21.15.16:3300/0 conn(0x7fd29c09e930 0x7fd29c09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:17.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.103+0000 7fd2a0bc0700 1 --2- 172.21.15.16:0/1356375289 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd29c09f650 0x7fd29c131f00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd294009d90 tx=0x7fd29400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fe126dfedfb45b2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:17.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.103+0000 7fd29affd700 1 -- 172.21.15.16:0/1356375289 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29400e070 con 0x7fd29c09f650 2023-09-30T22:30:17.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.103+0000 7fd29affd700 1 -- 172.21.15.16:0/1356375289 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd29400bb20 con 0x7fd29c09f650 2023-09-30T22:30:17.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.103+0000 7fd29affd700 1 -- 172.21.15.16:0/1356375289 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd294012430 con 0x7fd29c09f650 2023-09-30T22:30:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.104+0000 7fd2a2623700 1 -- 172.21.15.16:0/1356375289 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd29c09f650 msgr2=0x7fd29c131f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.104+0000 7fd2a2623700 1 --2- 172.21.15.16:0/1356375289 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd29c09f650 0x7fd29c131f00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd294009d90 tx=0x7fd29400b330 comp rx=0 tx=0).stop 2023-09-30T22:30:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.104+0000 7fd2a2623700 1 -- 172.21.15.16:0/1356375289 shutdown_connections 2023-09-30T22:30:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.104+0000 7fd2a2623700 1 --2- 172.21.15.16:0/1356375289 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd29c09f650 0x7fd29c131f00 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.104+0000 7fd2a2623700 1 --2- 172.21.15.16:0/1356375289 >> v2:172.21.15.16:3300/0 conn(0x7fd29c09e930 0x7fd29c09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.104+0000 7fd2a2623700 1 -- 172.21.15.16:0/1356375289 >> 172.21.15.16:0/1356375289 conn(0x7fd29c0a4210 msgr2=0x7fd29c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.105+0000 7fd2a2623700 1 -- 172.21.15.16:0/1356375289 shutdown_connections 2023-09-30T22:30:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.105+0000 7fd2a2623700 1 -- 172.21.15.16:0/1356375289 wait complete. 2023-09-30T22:30:17.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.106+0000 7fd2a2623700 1 Processor -- start 2023-09-30T22:30:17.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.106+0000 7fd2a2623700 1 -- start start 2023-09-30T22:30:17.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.107+0000 7fd2a2623700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd29c09e930 0x7fd29c139ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:17.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.107+0000 7fd2a2623700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd29c09f240 0x7fd29c139ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:17.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.107+0000 7fd2a2623700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd29c09f650 0x7fd29c1c7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:17.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.107+0000 7fd2a2623700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd29c0a30d0 con 0x7fd29c09f240 2023-09-30T22:30:17.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.107+0000 7fd2a2623700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd29c138730 con 0x7fd29c09f650 2023-09-30T22:30:17.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.107+0000 7fd2a2623700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd29c1372f0 con 0x7fd29c09e930 2023-09-30T22:30:17.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.107+0000 7fd29b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd29c09f240 0x7fd29c139ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:17.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.107+0000 7fd29bfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd29c09e930 0x7fd29c139ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:17.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.107+0000 7fd29b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd29c09f240 0x7fd29c139ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38950/0 (socket says 172.21.15.16:38950) 2023-09-30T22:30:17.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.107+0000 7fd29b7fe700 1 -- 172.21.15.16:0/1564644511 learned_addr learned my addr 172.21.15.16:0/1564644511 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:17.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.108+0000 7fd2a0bc0700 1 --2- 172.21.15.16:0/1564644511 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd29c09f650 0x7fd29c1c7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:17.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.108+0000 7fd29bfff700 1 -- 172.21.15.16:0/1564644511 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd29c09f650 msgr2=0x7fd29c1c7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:17.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.108+0000 7fd29bfff700 1 --2- 172.21.15.16:0/1564644511 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd29c09f650 0x7fd29c1c7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.108+0000 7fd29bfff700 1 -- 172.21.15.16:0/1564644511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd29c09f240 msgr2=0x7fd29c139ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:17.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.108+0000 7fd29bfff700 1 --2- 172.21.15.16:0/1564644511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd29c09f240 0x7fd29c139ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.108+0000 7fd29bfff700 1 -- 172.21.15.16:0/1564644511 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd28c0094a0 con 0x7fd29c09e930 2023-09-30T22:30:17.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.108+0000 7fd29b7fe700 1 --2- 172.21.15.16:0/1564644511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd29c09f240 0x7fd29c139ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:17.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.108+0000 7fd29bfff700 1 --2- 172.21.15.16:0/1564644511 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd29c09e930 0x7fd29c139ab0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd28c00e9b0 tx=0x7fd28c00ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:17.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.108+0000 7fd298ff9700 1 -- 172.21.15.16:0/1564644511 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28c00ee50 con 0x7fd29c09e930 2023-09-30T22:30:17.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.108+0000 7fd298ff9700 1 -- 172.21.15.16:0/1564644511 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd28c00c850 con 0x7fd29c09e930 2023-09-30T22:30:17.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.108+0000 7fd2a2623700 1 -- 172.21.15.16:0/1564644511 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd294009a70 con 0x7fd29c09e930 2023-09-30T22:30:17.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.108+0000 7fd298ff9700 1 -- 172.21.15.16:0/1564644511 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28c00cc00 con 0x7fd29c09e930 2023-09-30T22:30:17.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.108+0000 7fd2a2623700 1 -- 172.21.15.16:0/1564644511 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd29c0a0190 con 0x7fd29c09e930 2023-09-30T22:30:17.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.109+0000 7fd298ff9700 1 -- 172.21.15.16:0/1564644511 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd28c013470 con 0x7fd29c09e930 2023-09-30T22:30:17.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.109+0000 7fd2a2623700 1 -- 172.21.15.16:0/1564644511 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd29c074ea0 con 0x7fd29c09e930 2023-09-30T22:30:17.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.110+0000 7fd298ff9700 1 -- 172.21.15.16:0/1564644511 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd28c03f780 con 0x7fd29c09e930 2023-09-30T22:30:17.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.119+0000 7fd298ff9700 1 -- 172.21.15.16:0/1564644511 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd28c013720 con 0x7fd29c09e930 2023-09-30T22:30:17.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.149+0000 7faf327c0700 1 Processor -- start 2023-09-30T22:30:17.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.149+0000 7faf327c0700 1 -- start start 2023-09-30T22:30:17.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.150+0000 7faf327c0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf2c1338a0 0x7faf2c133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:17.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.150+0000 7faf327c0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf2c09de50 0x7faf2c09e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:17.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.150+0000 7faf327c0700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf2c06ef30 con 0x7faf2c09e760 2023-09-30T22:30:17.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.150+0000 7faf327c0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf2c06fef0 con 0x7faf2c09de50 2023-09-30T22:30:17.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.150+0000 7faf327c0700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faf2c1334b0 con 0x7faf2c1338a0 2023-09-30T22:30:17.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.150+0000 7faf2bfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf2c1338a0 0x7faf2c133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:17.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.150+0000 7faf2bfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf2c1338a0 0x7faf2c133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:51170/0 (socket says 172.21.15.16:51170) 2023-09-30T22:30:17.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.150+0000 7faf2bfff700 1 -- 172.21.15.16:0/2960767777 learned_addr learned my addr 172.21.15.16:0/2960767777 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:17.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.150+0000 7faf2b7fe700 1 --2- 172.21.15.16:0/2960767777 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf2c09de50 0x7faf2c09e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:17.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.150+0000 7faf2affd700 1 -- 172.21.15.16:0/2960767777 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3342832775 0 0) 0x7faf2c06ef30 con 0x7faf2c09e760 2023-09-30T22:30:17.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.150+0000 7faf2affd700 1 -- 172.21.15.16:0/2960767777 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf140035b0 con 0x7faf2c09e760 2023-09-30T22:30:17.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.151+0000 7faf2bfff700 1 -- 172.21.15.16:0/2960767777 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf2c09de50 msgr2=0x7faf2c09e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:17.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.151+0000 7faf2bfff700 1 --2- 172.21.15.16:0/2960767777 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf2c09de50 0x7faf2c09e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.151+0000 7faf2bfff700 1 -- 172.21.15.16:0/2960767777 >> v1:172.21.15.16:6789/0 conn(0x7faf2c09e760 legacy=0x7faf2c13a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:17.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.151+0000 7faf2bfff700 1 -- 172.21.15.16:0/2960767777 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf2c06fc30 con 0x7faf2c1338a0 2023-09-30T22:30:17.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.151+0000 7faf2b7fe700 1 --2- 172.21.15.16:0/2960767777 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf2c09de50 0x7faf2c09e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:17.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.151+0000 7faf2bfff700 1 --2- 172.21.15.16:0/2960767777 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf2c1338a0 0x7faf2c133c70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7faf1c009950 tx=0x7faf1c009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2507099ecdf58c98 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:17.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.151+0000 7faf2affd700 1 -- 172.21.15.16:0/2960767777 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1c011830 con 0x7faf2c1338a0 2023-09-30T22:30:17.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.151+0000 7faf2affd700 1 -- 172.21.15.16:0/2960767777 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf1c011990 con 0x7faf2c1338a0 2023-09-30T22:30:17.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.151+0000 7faf2affd700 1 -- 172.21.15.16:0/2960767777 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1c011d40 con 0x7faf2c1338a0 2023-09-30T22:30:17.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.152+0000 7faf327c0700 1 -- 172.21.15.16:0/2960767777 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf2c1338a0 msgr2=0x7faf2c133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:17.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.152+0000 7faf327c0700 1 --2- 172.21.15.16:0/2960767777 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf2c1338a0 0x7faf2c133c70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7faf1c009950 tx=0x7faf1c009c60 comp rx=0 tx=0).stop 2023-09-30T22:30:17.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.152+0000 7faf327c0700 1 -- 172.21.15.16:0/2960767777 shutdown_connections 2023-09-30T22:30:17.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.152+0000 7faf327c0700 1 --2- 172.21.15.16:0/2960767777 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf2c1338a0 0x7faf2c133c70 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.152+0000 7faf327c0700 1 --2- 172.21.15.16:0/2960767777 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf2c09de50 0x7faf2c09e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.152+0000 7faf327c0700 1 -- 172.21.15.16:0/2960767777 >> 172.21.15.16:0/2960767777 conn(0x7faf2c0a2aa0 msgr2=0x7faf2c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:17.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.152+0000 7faf327c0700 1 -- 172.21.15.16:0/2960767777 shutdown_connections 2023-09-30T22:30:17.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.152+0000 7faf327c0700 1 -- 172.21.15.16:0/2960767777 wait complete. 2023-09-30T22:30:17.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.153+0000 7faf327c0700 1 Processor -- start 2023-09-30T22:30:17.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.154+0000 7faf327c0700 1 -- start start 2023-09-30T22:30:17.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.154+0000 7faf327c0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf2c09de50 0x7faf2c11d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:17.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.154+0000 7faf327c0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf2c09e760 0x7faf2c11dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:17.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.154+0000 7faf327c0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf2c1338a0 0x7faf2c11e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:17.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.154+0000 7faf327c0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf2c120f20 con 0x7faf2c09de50 2023-09-30T22:30:17.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.154+0000 7faf327c0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf2c121170 con 0x7faf2c1338a0 2023-09-30T22:30:17.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.154+0000 7faf327c0700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faf2c120060 con 0x7faf2c09e760 2023-09-30T22:30:17.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.155+0000 7faf2b7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf2c09e760 0x7faf2c11dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:17.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.155+0000 7faf2b7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf2c09e760 0x7faf2c11dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:51178/0 (socket says 172.21.15.16:51178) 2023-09-30T22:30:17.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.155+0000 7faf2b7fe700 1 -- 172.21.15.16:0/3759551211 learned_addr learned my addr 172.21.15.16:0/3759551211 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:17.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.155+0000 7faf2bfff700 1 --2- 172.21.15.16:0/3759551211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf2c09de50 0x7faf2c11d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:17.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.155+0000 7faf30d5d700 1 --2- 172.21.15.16:0/3759551211 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf2c1338a0 0x7faf2c11e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:17.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.155+0000 7faf2b7fe700 1 -- 172.21.15.16:0/3759551211 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf2c1338a0 msgr2=0x7faf2c11e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:17.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.155+0000 7faf2b7fe700 1 --2- 172.21.15.16:0/3759551211 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf2c1338a0 0x7faf2c11e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.155+0000 7faf2b7fe700 1 -- 172.21.15.16:0/3759551211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf2c09de50 msgr2=0x7faf2c11d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:17.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.155+0000 7faf2b7fe700 1 --2- 172.21.15.16:0/3759551211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf2c09de50 0x7faf2c11d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.155+0000 7faf2b7fe700 1 -- 172.21.15.16:0/3759551211 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf200094a0 con 0x7faf2c09e760 2023-09-30T22:30:17.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.155+0000 7faf30d5d700 1 --2- 172.21.15.16:0/3759551211 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf2c1338a0 0x7faf2c11e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:30:17.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.155+0000 7faf2bfff700 1 --2- 172.21.15.16:0/3759551211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf2c09de50 0x7faf2c11d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:30:17.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.155+0000 7faf2b7fe700 1 --2- 172.21.15.16:0/3759551211 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf2c09e760 0x7faf2c11dd20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7faf2000ebb0 tx=0x7faf2000ef70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:17.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.155+0000 7faf28ff9700 1 -- 172.21.15.16:0/3759551211 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2000c850 con 0x7faf2c09e760 2023-09-30T22:30:17.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.155+0000 7faf28ff9700 1 -- 172.21.15.16:0/3759551211 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf2000c9b0 con 0x7faf2c09e760 2023-09-30T22:30:17.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.155+0000 7faf327c0700 1 -- 172.21.15.16:0/3759551211 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf1c009550 con 0x7faf2c09e760 2023-09-30T22:30:17.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.156+0000 7faf28ff9700 1 -- 172.21.15.16:0/3759551211 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2000cd40 con 0x7faf2c09e760 2023-09-30T22:30:17.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.157+0000 7faf327c0700 1 -- 172.21.15.16:0/3759551211 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf2c11f300 con 0x7faf2c09e760 2023-09-30T22:30:17.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.157+0000 7faf28ff9700 1 -- 172.21.15.16:0/3759551211 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7faf20013470 con 0x7faf2c09e760 2023-09-30T22:30:17.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.157+0000 7faf28ff9700 1 -- 172.21.15.16:0/3759551211 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7faf20043260 con 0x7faf2c09e760 2023-09-30T22:30:17.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.158+0000 7faf327c0700 1 -- 172.21.15.16:0/3759551211 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf2c120d60 con 0x7faf2c09e760 2023-09-30T22:30:17.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.163+0000 7faf28ff9700 1 -- 172.21.15.16:0/3759551211 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7faf2c120d60 con 0x7faf2c09e760 2023-09-30T22:30:17.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.178+0000 7f4871efd700 1 Processor -- start 2023-09-30T22:30:17.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.178+0000 7f4871efd700 1 -- start start 2023-09-30T22:30:17.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.179+0000 7f4871efd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f486c133420 0x7f486c1337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:17.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.179+0000 7f4871efd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f486c133e30 0x7f486c12ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:17.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.179+0000 7f4871efd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f486c134260 0x7f486c1304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:17.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.179+0000 7f4871efd700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f486c06b990 con 0x7f486c133420 2023-09-30T22:30:17.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.179+0000 7f4871efd700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f486c0a0590 con 0x7f486c134260 2023-09-30T22:30:17.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.179+0000 7f4871efd700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f486c09fb80 con 0x7f486c133e30 2023-09-30T22:30:17.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.179+0000 7f486b7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f486c133420 0x7f486c1337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:17.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.179+0000 7f486b7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f486c133420 0x7f486c1337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38974/0 (socket says 172.21.15.16:38974) 2023-09-30T22:30:17.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.179+0000 7f486b7fe700 1 -- 172.21.15.16:0/3915154148 learned_addr learned my addr 172.21.15.16:0/3915154148 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:17.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.179+0000 7f486bfff700 1 --2- 172.21.15.16:0/3915154148 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f486c134260 0x7f486c1304d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:17.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.180+0000 7f486affd700 1 --2- 172.21.15.16:0/3915154148 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f486c133e30 0x7f486c12ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:17.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.180+0000 7f486b7fe700 1 -- 172.21.15.16:0/3915154148 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f486c133e30 msgr2=0x7f486c12ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:17.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.180+0000 7f486b7fe700 1 --2- 172.21.15.16:0/3915154148 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f486c133e30 0x7f486c12ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.180+0000 7f486b7fe700 1 -- 172.21.15.16:0/3915154148 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f486c134260 msgr2=0x7f486c1304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:17.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.180+0000 7f486b7fe700 1 --2- 172.21.15.16:0/3915154148 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f486c134260 0x7f486c1304d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.180+0000 7f486b7fe700 1 -- 172.21.15.16:0/3915154148 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f486c0a0030 con 0x7f486c133420 2023-09-30T22:30:17.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.180+0000 7f486b7fe700 1 --2- 172.21.15.16:0/3915154148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f486c133420 0x7f486c1337f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f485c0098f0 tx=0x7f485c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ea63c3cece2cd5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:17.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.180+0000 7f486a7fc700 1 -- 172.21.15.16:0/3915154148 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f485c011830 con 0x7f486c133420 2023-09-30T22:30:17.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.180+0000 7f486a7fc700 1 -- 172.21.15.16:0/3915154148 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f485c011990 con 0x7f486c133420 2023-09-30T22:30:17.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.180+0000 7f486a7fc700 1 -- 172.21.15.16:0/3915154148 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f485c011d20 con 0x7f486c133420 2023-09-30T22:30:17.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.181+0000 7f4871efd700 1 -- 172.21.15.16:0/3915154148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f486c133420 msgr2=0x7f486c1337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:17.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.181+0000 7f4871efd700 1 --2- 172.21.15.16:0/3915154148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f486c133420 0x7f486c1337f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f485c0098f0 tx=0x7f485c009c00 comp rx=0 tx=0).stop 2023-09-30T22:30:17.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.181+0000 7f4871efd700 1 -- 172.21.15.16:0/3915154148 shutdown_connections 2023-09-30T22:30:17.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.181+0000 7f4871efd700 1 --2- 172.21.15.16:0/3915154148 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f486c134260 0x7f486c1304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.181+0000 7f4871efd700 1 --2- 172.21.15.16:0/3915154148 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f486c133420 0x7f486c1337f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.181+0000 7f4871efd700 1 --2- 172.21.15.16:0/3915154148 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f486c133e30 0x7f486c12ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.181+0000 7f4871efd700 1 -- 172.21.15.16:0/3915154148 >> 172.21.15.16:0/3915154148 conn(0x7f486c0a4210 msgr2=0x7f486c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:17.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.181+0000 7f4871efd700 1 -- 172.21.15.16:0/3915154148 shutdown_connections 2023-09-30T22:30:17.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.181+0000 7f4871efd700 1 -- 172.21.15.16:0/3915154148 wait complete. 2023-09-30T22:30:17.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.182+0000 7f4871efd700 1 Processor -- start 2023-09-30T22:30:17.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.182+0000 7f4871efd700 1 -- start start 2023-09-30T22:30:17.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.183+0000 7f4871efd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f486c133420 0x7f486c09c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:17.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.183+0000 7f4871efd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f486c133e30 0x7f486c09c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:17.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.183+0000 7f4871efd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f486c134260 0x7f486c09cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:17.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.183+0000 7f4871efd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f486c11fbd0 con 0x7f486c134260 2023-09-30T22:30:17.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.183+0000 7f4871efd700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f486c11f990 con 0x7f486c133e30 2023-09-30T22:30:17.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.183+0000 7f4871efd700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f486c09df10 con 0x7f486c133420 2023-09-30T22:30:17.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.183+0000 7f486b7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f486c133420 0x7f486c09c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:17.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.183+0000 7f486affd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f486c133e30 0x7f486c09c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:17.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.183+0000 7f486affd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f486c133e30 0x7f486c09c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41706/0 (socket says 172.21.15.16:41706) 2023-09-30T22:30:17.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.183+0000 7f486affd700 1 -- 172.21.15.16:0/377838676 learned_addr learned my addr 172.21.15.16:0/377838676 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:17.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.183+0000 7f486b7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f486c133420 0x7f486c09c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:51186/0 (socket says 172.21.15.16:51186) 2023-09-30T22:30:17.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.183+0000 7f486bfff700 1 --2- 172.21.15.16:0/377838676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f486c134260 0x7f486c09cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:17.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.184+0000 7f486affd700 1 -- 172.21.15.16:0/377838676 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f486c133420 msgr2=0x7f486c09c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:17.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.184+0000 7f486affd700 1 --2- 172.21.15.16:0/377838676 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f486c133420 0x7f486c09c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.184+0000 7f486affd700 1 -- 172.21.15.16:0/377838676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f486c134260 msgr2=0x7f486c09cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:17.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.184+0000 7f486affd700 1 --2- 172.21.15.16:0/377838676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f486c134260 0x7f486c09cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.184+0000 7f486affd700 1 -- 172.21.15.16:0/377838676 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f485c009550 con 0x7f486c133e30 2023-09-30T22:30:17.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.184+0000 7f486affd700 1 --2- 172.21.15.16:0/377838676 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f486c133e30 0x7f486c09c9c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f486000e840 tx=0x7f486000eb50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:17.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.184+0000 7f4853fff700 1 -- 172.21.15.16:0/377838676 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4860013610 con 0x7f486c133e30 2023-09-30T22:30:17.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.184+0000 7f4853fff700 1 -- 172.21.15.16:0/377838676 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4860013770 con 0x7f486c133e30 2023-09-30T22:30:17.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.184+0000 7f4871efd700 1 -- 172.21.15.16:0/377838676 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f486c120170 con 0x7f486c133e30 2023-09-30T22:30:17.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.184+0000 7f4853fff700 1 -- 172.21.15.16:0/377838676 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4860013b20 con 0x7f486c133e30 2023-09-30T22:30:17.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.184+0000 7f4871efd700 1 -- 172.21.15.16:0/377838676 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f486c11fe90 con 0x7f486c133e30 2023-09-30T22:30:17.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.185+0000 7f4853fff700 1 -- 172.21.15.16:0/377838676 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4860013c80 con 0x7f486c133e30 2023-09-30T22:30:17.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.185+0000 7f4853fff700 1 -- 172.21.15.16:0/377838676 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f486003f410 con 0x7f486c133e30 2023-09-30T22:30:17.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.186+0000 7f4871efd700 1 -- 172.21.15.16:0/377838676 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f486c074ea0 con 0x7f486c133e30 2023-09-30T22:30:17.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.191+0000 7f4853fff700 1 -- 172.21.15.16:0/377838676 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4860012370 con 0x7f486c133e30 2023-09-30T22:30:17.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.291+0000 7fac03a26700 1 -- 172.21.15.16:0/2950062729 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fabfc1a86d0 con 0x7fabfc1a5ab0 2023-09-30T22:30:17.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.292+0000 7fabf1ffb700 1 -- 172.21.15.16:0/2950062729 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fabfc1a86d0 con 0x7fabfc1a5ab0 2023-09-30T22:30:17.293 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:30:17.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.294+0000 7fac03a26700 1 -- 172.21.15.16:0/2950062729 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fabfc1a5ab0 msgr2=0x7fabfc1ceb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:17.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.294+0000 7fac03a26700 1 --2- 172.21.15.16:0/2950062729 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fabfc1a5ab0 0x7fabfc1ceb20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fabf400efc0 tx=0x7fabf400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.294+0000 7fac03a26700 1 -- 172.21.15.16:0/2950062729 shutdown_connections 2023-09-30T22:30:17.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.294+0000 7fac03a26700 1 --2- 172.21.15.16:0/2950062729 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fabfc1a5ab0 0x7fabfc1ceb20 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.294+0000 7fac03a26700 1 --2- 172.21.15.16:0/2950062729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabfc09d6f0 0x7fabfc1a5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.294+0000 7fac03a26700 1 --2- 172.21.15.16:0/2950062729 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fabfc09e410 0x7fabfc1a5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.294+0000 7fac03a26700 1 -- 172.21.15.16:0/2950062729 >> 172.21.15.16:0/2950062729 conn(0x7fabfc066f50 msgr2=0x7fabfc1bb010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:17.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.294+0000 7fac03a26700 1 -- 172.21.15.16:0/2950062729 shutdown_connections 2023-09-30T22:30:17.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.294+0000 7fac03a26700 1 -- 172.21.15.16:0/2950062729 wait complete. 2023-09-30T22:30:17.309 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:30:17.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.353+0000 7fd2a2623700 1 -- 172.21.15.16:0/1564644511 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd29c136b00 con 0x7fd29c09e930 2023-09-30T22:30:17.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.353+0000 7fd298ff9700 1 -- 172.21.15.16:0/1564644511 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd28c017070 con 0x7fd29c09e930 2023-09-30T22:30:17.355 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:30:17.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.355+0000 7fd2a2623700 1 -- 172.21.15.16:0/1564644511 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd29c09e930 msgr2=0x7fd29c139ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:17.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.355+0000 7fd2a2623700 1 --2- 172.21.15.16:0/1564644511 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd29c09e930 0x7fd29c139ab0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd28c00e9b0 tx=0x7fd28c00ecc0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.355+0000 7fd2a2623700 1 -- 172.21.15.16:0/1564644511 shutdown_connections 2023-09-30T22:30:17.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.355+0000 7fd2a2623700 1 --2- 172.21.15.16:0/1564644511 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd29c09f650 0x7fd29c1c7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.355+0000 7fd2a2623700 1 --2- 172.21.15.16:0/1564644511 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd29c09e930 0x7fd29c139ab0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.355+0000 7fd2a2623700 1 --2- 172.21.15.16:0/1564644511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd29c09f240 0x7fd29c139ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.355+0000 7fd2a2623700 1 -- 172.21.15.16:0/1564644511 >> 172.21.15.16:0/1564644511 conn(0x7fd29c0a4210 msgr2=0x7fd29c06c320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:17.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.355+0000 7fd2a2623700 1 -- 172.21.15.16:0/1564644511 shutdown_connections 2023-09-30T22:30:17.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.355+0000 7fd2a2623700 1 -- 172.21.15.16:0/1564644511 wait complete. 2023-09-30T22:30:17.371 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:30:17.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.374+0000 7faf327c0700 1 -- 172.21.15.16:0/3759551211 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7faf2c120d60 con 0x7faf2c09e760 2023-09-30T22:30:17.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.375+0000 7faf28ff9700 1 -- 172.21.15.16:0/3759551211 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7faf2c120d60 con 0x7faf2c09e760 2023-09-30T22:30:17.376 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:30:17.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.377+0000 7faf327c0700 1 -- 172.21.15.16:0/3759551211 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf2c09e760 msgr2=0x7faf2c11dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:17.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.377+0000 7faf327c0700 1 --2- 172.21.15.16:0/3759551211 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf2c09e760 0x7faf2c11dd20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7faf2000ebb0 tx=0x7faf2000ef70 comp rx=0 tx=0).stop 2023-09-30T22:30:17.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.377+0000 7faf327c0700 1 -- 172.21.15.16:0/3759551211 shutdown_connections 2023-09-30T22:30:17.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.377+0000 7faf327c0700 1 --2- 172.21.15.16:0/3759551211 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf2c1338a0 0x7faf2c11e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.377+0000 7faf327c0700 1 --2- 172.21.15.16:0/3759551211 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf2c09de50 0x7faf2c11d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.377+0000 7faf327c0700 1 --2- 172.21.15.16:0/3759551211 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf2c09e760 0x7faf2c11dd20 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.377+0000 7faf327c0700 1 -- 172.21.15.16:0/3759551211 >> 172.21.15.16:0/3759551211 conn(0x7faf2c0a2aa0 msgr2=0x7faf2c1b9410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:17.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.377+0000 7faf327c0700 1 -- 172.21.15.16:0/3759551211 shutdown_connections 2023-09-30T22:30:17.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.377+0000 7faf327c0700 1 -- 172.21.15.16:0/3759551211 wait complete. 2023-09-30T22:30:17.392 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:30:17.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.417+0000 7f4871efd700 1 -- 172.21.15.16:0/377838676 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f486c06b250 con 0x7f486c133e30 2023-09-30T22:30:17.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.417+0000 7f4853fff700 1 -- 172.21.15.16:0/377838676 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4860021020 con 0x7f486c133e30 2023-09-30T22:30:17.418 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:30:17.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.419+0000 7f4871efd700 1 -- 172.21.15.16:0/377838676 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f486c133e30 msgr2=0x7f486c09c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:17.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.419+0000 7f4871efd700 1 --2- 172.21.15.16:0/377838676 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f486c133e30 0x7f486c09c9c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f486000e840 tx=0x7f486000eb50 comp rx=0 tx=0).stop 2023-09-30T22:30:17.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.419+0000 7f4871efd700 1 -- 172.21.15.16:0/377838676 shutdown_connections 2023-09-30T22:30:17.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.419+0000 7f4871efd700 1 --2- 172.21.15.16:0/377838676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f486c134260 0x7f486c09cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.419+0000 7f4871efd700 1 --2- 172.21.15.16:0/377838676 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f486c133420 0x7f486c09c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.419+0000 7f4871efd700 1 --2- 172.21.15.16:0/377838676 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f486c133e30 0x7f486c09c9c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:17.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.419+0000 7f4871efd700 1 -- 172.21.15.16:0/377838676 >> 172.21.15.16:0/377838676 conn(0x7f486c0a4210 msgr2=0x7f486c0a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:17.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.419+0000 7f4871efd700 1 -- 172.21.15.16:0/377838676 shutdown_connections 2023-09-30T22:30:17.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:17.419+0000 7f4871efd700 1 -- 172.21.15.16:0/377838676 wait complete. 2023-09-30T22:30:17.434 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:30:18.310 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:18.372 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:18.393 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:18.435 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:18.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.464+0000 7f85f987c700 1 Processor -- start 2023-09-30T22:30:18.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.464+0000 7f85f987c700 1 -- start start 2023-09-30T22:30:18.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.465+0000 7f85f987c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85f409d6f0 0x7f85f409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:18.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.465+0000 7f85f987c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85f409e000 0x7f85f41339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:18.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.465+0000 7f85f987c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85f409b120 con 0x7f85f409e410 2023-09-30T22:30:18.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.465+0000 7f85f987c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85f406b460 con 0x7f85f409d6f0 2023-09-30T22:30:18.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.465+0000 7f85f987c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f85f409a7e0 con 0x7f85f409e000 2023-09-30T22:30:18.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.465+0000 7f85f2ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85f409d6f0 0x7f85f409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:18.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.465+0000 7f85f2ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85f409d6f0 0x7f85f409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:34136/0 (socket says 172.21.15.16:34136) 2023-09-30T22:30:18.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.465+0000 7f85f2ffd700 1 -- 172.21.15.16:0/1161576350 learned_addr learned my addr 172.21.15.16:0/1161576350 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:18.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.466+0000 7f85f27fc700 1 --2- 172.21.15.16:0/1161576350 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85f409e000 0x7f85f41339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:18.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.466+0000 7f85f2ffd700 1 -- 172.21.15.16:0/1161576350 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85f409e000 msgr2=0x7f85f41339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:18.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.466+0000 7f85f2ffd700 1 --2- 172.21.15.16:0/1161576350 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85f409e000 0x7f85f41339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.466+0000 7f85f2ffd700 1 -- 172.21.15.16:0/1161576350 >> v1:172.21.15.16:6789/0 conn(0x7f85f409e410 legacy=0x7f85f4133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:18.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.466+0000 7f85f2ffd700 1 -- 172.21.15.16:0/1161576350 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f409a500 con 0x7f85f409d6f0 2023-09-30T22:30:18.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.466+0000 7f85f1ffb700 1 -- 172.21.15.16:0/1161576350 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4089654801 0 0) 0x7f85f409b120 con 0x7f85f409e410 2023-09-30T22:30:18.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.466+0000 7f85f2ffd700 1 --2- 172.21.15.16:0/1161576350 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85f409d6f0 0x7f85f409dac0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f85e4009920 tx=0x7f85e4009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b267e19d3a7d9438 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:18.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.466+0000 7f85f1ffb700 1 -- 172.21.15.16:0/1161576350 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e400ba60 con 0x7f85f409d6f0 2023-09-30T22:30:18.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.466+0000 7f85f1ffb700 1 -- 172.21.15.16:0/1161576350 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85e400bbc0 con 0x7f85f409d6f0 2023-09-30T22:30:18.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.466+0000 7f85f1ffb700 1 -- 172.21.15.16:0/1161576350 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e4009e70 con 0x7f85f409d6f0 2023-09-30T22:30:18.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.467+0000 7f85f987c700 1 -- 172.21.15.16:0/1161576350 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85f409d6f0 msgr2=0x7f85f409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:18.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.467+0000 7f85f987c700 1 --2- 172.21.15.16:0/1161576350 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85f409d6f0 0x7f85f409dac0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f85e4009920 tx=0x7f85e4009c30 comp rx=0 tx=0).stop 2023-09-30T22:30:18.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.467+0000 7f85f987c700 1 -- 172.21.15.16:0/1161576350 shutdown_connections 2023-09-30T22:30:18.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.467+0000 7f85f987c700 1 --2- 172.21.15.16:0/1161576350 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85f409d6f0 0x7f85f409dac0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.467+0000 7f85f987c700 1 --2- 172.21.15.16:0/1161576350 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85f409e000 0x7f85f41339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.467+0000 7f85f987c700 1 -- 172.21.15.16:0/1161576350 >> 172.21.15.16:0/1161576350 conn(0x7f85f4066f50 msgr2=0x7f85f409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:18.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.467+0000 7f85f987c700 1 -- 172.21.15.16:0/1161576350 shutdown_connections 2023-09-30T22:30:18.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.467+0000 7f85f987c700 1 -- 172.21.15.16:0/1161576350 wait complete. 2023-09-30T22:30:18.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.468+0000 7f85f987c700 1 Processor -- start 2023-09-30T22:30:18.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.468+0000 7f85f987c700 1 -- start start 2023-09-30T22:30:18.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.469+0000 7f85f987c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85f409d6f0 0x7f85f4122310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:18.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.469+0000 7f85f987c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85f409e000 0x7f85f4122850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:18.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.469+0000 7f85f987c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85f409e410 0x7f85f4122d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:18.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.469+0000 7f85f987c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85f4125e30 con 0x7f85f409e410 2023-09-30T22:30:18.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.469+0000 7f85f987c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85f4125920 con 0x7f85f409e000 2023-09-30T22:30:18.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.469+0000 7f85f987c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f85f4124db0 con 0x7f85f409d6f0 2023-09-30T22:30:18.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.469+0000 7f85f2ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85f409d6f0 0x7f85f4122310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:18.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.469+0000 7f85f37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85f409e410 0x7f85f4122d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:18.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.469+0000 7f85f27fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85f409e000 0x7f85f4122850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:18.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.469+0000 7f85f2ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85f409d6f0 0x7f85f4122310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:58732/0 (socket says 172.21.15.16:58732) 2023-09-30T22:30:18.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.469+0000 7f85f2ffd700 1 -- 172.21.15.16:0/3521502670 learned_addr learned my addr 172.21.15.16:0/3521502670 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:18.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.469+0000 7f85f37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85f409e410 0x7f85f4122d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57864/0 (socket says 172.21.15.16:57864) 2023-09-30T22:30:18.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.469+0000 7f85f27fc700 1 --2- 172.21.15.16:0/3521502670 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85f409e000 0x7f85f4122850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:34140/0 (socket says 172.21.15.16:34140) 2023-09-30T22:30:18.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.470+0000 7f85f37fe700 1 -- 172.21.15.16:0/3521502670 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85f409d6f0 msgr2=0x7f85f4122310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:18.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.470+0000 7f85f37fe700 1 --2- 172.21.15.16:0/3521502670 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85f409d6f0 0x7f85f4122310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.470+0000 7f85f37fe700 1 -- 172.21.15.16:0/3521502670 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85f409e000 msgr2=0x7f85f4122850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:18.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.470+0000 7f85f37fe700 1 --2- 172.21.15.16:0/3521502670 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85f409e000 0x7f85f4122850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.470+0000 7f85f37fe700 1 -- 172.21.15.16:0/3521502670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85e4009550 con 0x7f85f409e410 2023-09-30T22:30:18.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.470+0000 7f85f27fc700 1 --2- 172.21.15.16:0/3521502670 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85f409e000 0x7f85f4122850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:18.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.470+0000 7f85f37fe700 1 --2- 172.21.15.16:0/3521502670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85f409e410 0x7f85f4122d90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f85f409e8b0 tx=0x7f85ec00ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:18.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.470+0000 7f85dbfff700 1 -- 172.21.15.16:0/3521502670 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ec003e30 con 0x7f85f409e410 2023-09-30T22:30:18.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.470+0000 7f85dbfff700 1 -- 172.21.15.16:0/3521502670 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85ec005070 con 0x7f85f409e410 2023-09-30T22:30:18.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.470+0000 7f85dbfff700 1 -- 172.21.15.16:0/3521502670 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ec005420 con 0x7f85f409e410 2023-09-30T22:30:18.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.471+0000 7f85f987c700 1 -- 172.21.15.16:0/3521502670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85f409c710 con 0x7f85f409e410 2023-09-30T22:30:18.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.471+0000 7f85f987c700 1 -- 172.21.15.16:0/3521502670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85f40a2970 con 0x7f85f409e410 2023-09-30T22:30:18.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.472+0000 7f85dbfff700 1 -- 172.21.15.16:0/3521502670 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f85ec005580 con 0x7f85f409e410 2023-09-30T22:30:18.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.472+0000 7f85dbfff700 1 -- 172.21.15.16:0/3521502670 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f85ec03f270 con 0x7f85f409e410 2023-09-30T22:30:18.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.473+0000 7f85f987c700 1 -- 172.21.15.16:0/3521502670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85f411f030 con 0x7f85f409e410 2023-09-30T22:30:18.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.478+0000 7f85dbfff700 1 -- 172.21.15.16:0/3521502670 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f85ec019af0 con 0x7f85f409e410 2023-09-30T22:30:18.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.522+0000 7f3808422700 1 Processor -- start 2023-09-30T22:30:18.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.522+0000 7f3808422700 1 -- start start 2023-09-30T22:30:18.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.522+0000 7f3808422700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38000a1510 0x7f3800138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:18.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.522+0000 7f3808422700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3800139250 0x7f380013b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:18.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.522+0000 7f3808422700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f380009edd0 con 0x7f38000a0a30 2023-09-30T22:30:18.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.522+0000 7f3808422700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38000a18e0 con 0x7f38000a1510 2023-09-30T22:30:18.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.523+0000 7f3808422700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f380009e1f0 con 0x7f3800139250 2023-09-30T22:30:18.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.523+0000 7f38061be700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f38000a0a30 0x7f38000a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56908/0 (socket says 172.21.15.16:56908) 2023-09-30T22:30:18.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.523+0000 7f38059bd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38000a1510 0x7f3800138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:18.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.523+0000 7f38061be700 1 -- 172.21.15.16:0/2335209119 learned_addr learned my addr 172.21.15.16:0/2335209119 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:30:18.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.523+0000 7f38069bf700 1 --2- 172.21.15.16:0/2335209119 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3800139250 0x7f380013b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:18.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.523+0000 7f38051bc700 1 -- 172.21.15.16:0/2335209119 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1687652589 0 0) 0x7f380009edd0 con 0x7f38000a0a30 2023-09-30T22:30:18.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.523+0000 7f38051bc700 1 -- 172.21.15.16:0/2335209119 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37ec0035b0 con 0x7f38000a0a30 2023-09-30T22:30:18.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.523+0000 7f38059bd700 1 -- 172.21.15.16:0/2335209119 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3800139250 msgr2=0x7f380013b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:18.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.523+0000 7f38059bd700 1 --2- 172.21.15.16:0/2335209119 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3800139250 0x7f380013b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.523+0000 7f38059bd700 1 -- 172.21.15.16:0/2335209119 >> v1:172.21.15.16:6789/0 conn(0x7f38000a0a30 legacy=0x7f38000a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:18.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.523+0000 7f38059bd700 1 -- 172.21.15.16:0/2335209119 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f380009dc50 con 0x7f38000a1510 2023-09-30T22:30:18.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.523+0000 7f38069bf700 1 --2- 172.21.15.16:0/2335209119 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3800139250 0x7f380013b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:18.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.524+0000 7f38059bd700 1 --2- 172.21.15.16:0/2335209119 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38000a1510 0x7f3800138d10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f37f0009870 tx=0x7f37f0009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f887bddd4d690b98 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:18.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.524+0000 7f38051bc700 1 -- 172.21.15.16:0/2335209119 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f0011830 con 0x7f38000a1510 2023-09-30T22:30:18.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.524+0000 7f38051bc700 1 -- 172.21.15.16:0/2335209119 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37f0011990 con 0x7f38000a1510 2023-09-30T22:30:18.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.524+0000 7f38051bc700 1 -- 172.21.15.16:0/2335209119 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f0011d00 con 0x7f38000a1510 2023-09-30T22:30:18.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.524+0000 7f3808422700 1 -- 172.21.15.16:0/2335209119 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38000a1510 msgr2=0x7f3800138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:18.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.524+0000 7f3808422700 1 --2- 172.21.15.16:0/2335209119 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38000a1510 0x7f3800138d10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f37f0009870 tx=0x7f37f0009b80 comp rx=0 tx=0).stop 2023-09-30T22:30:18.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.525+0000 7f3808422700 1 -- 172.21.15.16:0/2335209119 shutdown_connections 2023-09-30T22:30:18.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.525+0000 7f3808422700 1 --2- 172.21.15.16:0/2335209119 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3800139250 0x7f380013b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.525+0000 7f3808422700 1 --2- 172.21.15.16:0/2335209119 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38000a1510 0x7f3800138d10 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.525+0000 7f3808422700 1 -- 172.21.15.16:0/2335209119 >> 172.21.15.16:0/2335209119 conn(0x7f38000a6550 msgr2=0x7f38000a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:18.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.525+0000 7f3808422700 1 -- 172.21.15.16:0/2335209119 shutdown_connections 2023-09-30T22:30:18.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.525+0000 7f3808422700 1 -- 172.21.15.16:0/2335209119 wait complete. 2023-09-30T22:30:18.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.526+0000 7f3808422700 1 Processor -- start 2023-09-30T22:30:18.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.526+0000 7f3808422700 1 -- start start 2023-09-30T22:30:18.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.527+0000 7f3808422700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38000a0a30 0x7f380011e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:18.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.527+0000 7f3808422700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38000a1510 0x7f380011ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:18.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.527+0000 7f3808422700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3800139250 0x7f380011f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:18.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.527+0000 7f3808422700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3800134110 con 0x7f38000a1510 2023-09-30T22:30:18.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.527+0000 7f3808422700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3800121b60 con 0x7f38000a0a30 2023-09-30T22:30:18.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.527+0000 7f3808422700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3800120300 con 0x7f3800139250 2023-09-30T22:30:18.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.527+0000 7f38059bd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38000a1510 0x7f380011ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:18.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.527+0000 7f38059bd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38000a1510 0x7f380011ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57874/0 (socket says 172.21.15.16:57874) 2023-09-30T22:30:18.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.527+0000 7f38059bd700 1 -- 172.21.15.16:0/3195367035 learned_addr learned my addr 172.21.15.16:0/3195367035 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:18.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.527+0000 7f38061be700 1 --2- 172.21.15.16:0/3195367035 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38000a0a30 0x7f380011e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:18.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.527+0000 7f38069bf700 1 --2- 172.21.15.16:0/3195367035 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3800139250 0x7f380011f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:18.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.527+0000 7f38059bd700 1 -- 172.21.15.16:0/3195367035 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3800139250 msgr2=0x7f380011f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:18.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.527+0000 7f38059bd700 1 --2- 172.21.15.16:0/3195367035 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3800139250 0x7f380011f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.527+0000 7f38059bd700 1 -- 172.21.15.16:0/3195367035 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38000a0a30 msgr2=0x7f380011e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:18.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.527+0000 7f38059bd700 1 --2- 172.21.15.16:0/3195367035 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38000a0a30 0x7f380011e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.527+0000 7f38059bd700 1 -- 172.21.15.16:0/3195367035 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f8009a70 con 0x7f38000a1510 2023-09-30T22:30:18.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.527+0000 7f38059bd700 1 --2- 172.21.15.16:0/3195367035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38000a1510 0x7f380011ec50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f37f00090b0 tx=0x7f37f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:18.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.528+0000 7f37f6ffd700 1 -- 172.21.15.16:0/3195367035 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f00045d0 con 0x7f38000a1510 2023-09-30T22:30:18.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.528+0000 7f37f6ffd700 1 -- 172.21.15.16:0/3195367035 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37f001f070 con 0x7f38000a1510 2023-09-30T22:30:18.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.528+0000 7f3808422700 1 -- 172.21.15.16:0/3195367035 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37f00094a0 con 0x7f38000a1510 2023-09-30T22:30:18.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.528+0000 7f37f6ffd700 1 -- 172.21.15.16:0/3195367035 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f00038c0 con 0x7f38000a1510 2023-09-30T22:30:18.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.528+0000 7f3808422700 1 -- 172.21.15.16:0/3195367035 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3800121810 con 0x7f38000a1510 2023-09-30T22:30:18.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.529+0000 7f3808422700 1 -- 172.21.15.16:0/3195367035 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3800121cf0 con 0x7f38000a1510 2023-09-30T22:30:18.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.530+0000 7f37f6ffd700 1 -- 172.21.15.16:0/3195367035 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f37f0003a20 con 0x7f38000a1510 2023-09-30T22:30:18.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.530+0000 7f37f6ffd700 1 -- 172.21.15.16:0/3195367035 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f37f0021070 con 0x7f38000a1510 2023-09-30T22:30:18.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.535+0000 7f37f6ffd700 1 -- 172.21.15.16:0/3195367035 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f37f002f990 con 0x7f38000a1510 2023-09-30T22:30:18.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.608+0000 7f85de653700 1 Processor -- start 2023-09-30T22:30:18.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.608+0000 7f85de653700 1 -- start start 2023-09-30T22:30:18.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.608+0000 7f19f4463700 1 Processor -- start 2023-09-30T22:30:18.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.608+0000 7f19f4463700 1 -- start start 2023-09-30T22:30:18.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.608+0000 7f85de653700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f85d80a0a30 0x7f85d80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:18.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.608+0000 7f85de653700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85d809d630 0x7f85d80a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:18.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.608+0000 7f85de653700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85d8133930 0x7f85d8133db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:18.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.609+0000 7f85de653700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85d80a2a70 con 0x7f85d80a0a30 2023-09-30T22:30:18.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.609+0000 7f85de653700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85d80a01d0 con 0x7f85d8133930 2023-09-30T22:30:18.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.609+0000 7f85de653700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f85d809e2b0 con 0x7f85d809d630 2023-09-30T22:30:18.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.609+0000 7f85d7fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f85d80a0a30 0x7f85d80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:18.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.609+0000 7f85d7fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f85d80a0a30 0x7f85d80a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57884/0 (socket says 172.21.15.16:57884) 2023-09-30T22:30:18.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.609+0000 7f85d7fff700 1 -- 172.21.15.16:0/4225118475 learned_addr learned my addr 172.21.15.16:0/4225118475 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:18.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.609+0000 7f85d77fe700 1 --2- 172.21.15.16:0/4225118475 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85d809d630 0x7f85d80a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:18.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.609+0000 7f85dcbf0700 1 --2- 172.21.15.16:0/4225118475 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85d8133930 0x7f85d8133db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:18.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.609+0000 7f85d7fff700 1 -- 172.21.15.16:0/4225118475 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85d809d630 msgr2=0x7f85d80a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:18.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.609+0000 7f85d7fff700 1 --2- 172.21.15.16:0/4225118475 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85d809d630 0x7f85d80a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.609+0000 7f85d7fff700 1 -- 172.21.15.16:0/4225118475 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85d8133930 msgr2=0x7f85d8133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:18.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.609+0000 7f85d7fff700 1 --2- 172.21.15.16:0/4225118475 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85d8133930 0x7f85d8133db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.609+0000 7f85d7fff700 1 -- 172.21.15.16:0/4225118475 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85d809dc50 con 0x7f85d80a0a30 2023-09-30T22:30:18.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.609+0000 7f85d7fff700 1 --2- 172.21.15.16:0/4225118475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85d80a0a30 0x7f85d80a0e00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f85c80098f0 tx=0x7f85c8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20c71c035ce446c9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:18.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.610+0000 7f85d6ffd700 1 -- 172.21.15.16:0/4225118475 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c800b8a0 con 0x7f85d80a0a30 2023-09-30T22:30:18.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.610+0000 7f85d6ffd700 1 -- 172.21.15.16:0/4225118475 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85c800ba00 con 0x7f85d80a0a30 2023-09-30T22:30:18.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.610+0000 7f85d6ffd700 1 -- 172.21.15.16:0/4225118475 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c800bd90 con 0x7f85d80a0a30 2023-09-30T22:30:18.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.610+0000 7f19f4463700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f19ec09d6f0 0x7f19ec09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:18.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.610+0000 7f19f4463700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f19ec09e000 0x7f19ec130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:18.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.610+0000 7f19f4463700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19ec09f130 con 0x7f19ec09e410 2023-09-30T22:30:18.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.610+0000 7f19f4463700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19ec09f520 con 0x7f19ec09e000 2023-09-30T22:30:18.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.610+0000 7f19f4463700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec09e930 con 0x7f19ec09d6f0 2023-09-30T22:30:18.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.610+0000 7f19f21ff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f19ec09d6f0 0x7f19ec09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:18.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.610+0000 7f19f2a00700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f19ec09e410 0x7f19ec130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56920/0 (socket says 172.21.15.16:56920) 2023-09-30T22:30:18.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.610+0000 7f19f2a00700 1 -- 172.21.15.16:0/2875388394 learned_addr learned my addr 172.21.15.16:0/2875388394 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:30:18.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.610+0000 7f19f21ff700 1 -- 172.21.15.16:0/2875388394 >> v2:172.21.15.16:3300/0 conn(0x7f19ec09e000 msgr2=0x7f19ec130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:18.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.610+0000 7f19f21ff700 1 --2- 172.21.15.16:0/2875388394 >> v2:172.21.15.16:3300/0 conn(0x7f19ec09e000 0x7f19ec130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.610+0000 7f19f21ff700 1 -- 172.21.15.16:0/2875388394 >> v1:172.21.15.16:6789/0 conn(0x7f19ec09e410 legacy=0x7f19ec130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:18.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.611+0000 7f19f21ff700 1 -- 172.21.15.16:0/2875388394 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19ec09b2d0 con 0x7f19ec09d6f0 2023-09-30T22:30:18.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.611+0000 7f19f21ff700 1 --2- 172.21.15.16:0/2875388394 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f19ec09d6f0 0x7f19ec09dac0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f19e8009840 tx=0x7f19e8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dea7794c0f7eaa7e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:18.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.611+0000 7f85de653700 1 -- 172.21.15.16:0/4225118475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85d80a0a30 msgr2=0x7f85d80a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:18.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.611+0000 7f85de653700 1 --2- 172.21.15.16:0/4225118475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85d80a0a30 0x7f85d80a0e00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f85c80098f0 tx=0x7f85c8009c00 comp rx=0 tx=0).stop 2023-09-30T22:30:18.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.611+0000 7f85de653700 1 -- 172.21.15.16:0/4225118475 shutdown_connections 2023-09-30T22:30:18.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.611+0000 7f85de653700 1 --2- 172.21.15.16:0/4225118475 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85d8133930 0x7f85d8133db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.611+0000 7f85de653700 1 --2- 172.21.15.16:0/4225118475 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85d80a0a30 0x7f85d80a0e00 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.611+0000 7f85de653700 1 --2- 172.21.15.16:0/4225118475 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85d809d630 0x7f85d80a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.611+0000 7f85de653700 1 -- 172.21.15.16:0/4225118475 >> 172.21.15.16:0/4225118475 conn(0x7f85d80a3050 msgr2=0x7f85d80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:18.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.611+0000 7f19f11fd700 1 -- 172.21.15.16:0/2875388394 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e8011830 con 0x7f19ec09d6f0 2023-09-30T22:30:18.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.611+0000 7f85de653700 1 -- 172.21.15.16:0/4225118475 shutdown_connections 2023-09-30T22:30:18.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.611+0000 7f19f11fd700 1 -- 172.21.15.16:0/2875388394 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19e8011990 con 0x7f19ec09d6f0 2023-09-30T22:30:18.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.611+0000 7f19f11fd700 1 -- 172.21.15.16:0/2875388394 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e8011d20 con 0x7f19ec09d6f0 2023-09-30T22:30:18.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.611+0000 7f85de653700 1 -- 172.21.15.16:0/4225118475 wait complete. 2023-09-30T22:30:18.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.612+0000 7f19f4463700 1 -- 172.21.15.16:0/2875388394 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f19ec09d6f0 msgr2=0x7f19ec09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:18.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.612+0000 7f19f4463700 1 --2- 172.21.15.16:0/2875388394 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f19ec09d6f0 0x7f19ec09dac0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f19e8009840 tx=0x7f19e8009b50 comp rx=0 tx=0).stop 2023-09-30T22:30:18.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.612+0000 7f19f4463700 1 -- 172.21.15.16:0/2875388394 shutdown_connections 2023-09-30T22:30:18.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.612+0000 7f19f4463700 1 --2- 172.21.15.16:0/2875388394 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f19ec09d6f0 0x7f19ec09dac0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.612+0000 7f19f4463700 1 --2- 172.21.15.16:0/2875388394 >> v2:172.21.15.16:3300/0 conn(0x7f19ec09e000 0x7f19ec130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.612+0000 7f19f4463700 1 -- 172.21.15.16:0/2875388394 >> 172.21.15.16:0/2875388394 conn(0x7f19ec066f50 msgr2=0x7f19ec09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:18.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.612+0000 7f19f4463700 1 -- 172.21.15.16:0/2875388394 shutdown_connections 2023-09-30T22:30:18.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.612+0000 7f19f4463700 1 -- 172.21.15.16:0/2875388394 wait complete. 2023-09-30T22:30:18.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.612+0000 7f85de653700 1 Processor -- start 2023-09-30T22:30:18.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.613+0000 7f19f4463700 1 Processor -- start 2023-09-30T22:30:18.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.613+0000 7f85de653700 1 -- start start 2023-09-30T22:30:18.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.613+0000 7f85de653700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85d809d630 0x7f85d811e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:18.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.613+0000 7f85de653700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85d80a0a30 0x7f85d811ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:18.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.613+0000 7f85de653700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85d8133930 0x7f85d811f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:18.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.613+0000 7f85de653700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d81205c0 con 0x7f85d8133930 2023-09-30T22:30:18.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.613+0000 7f85de653700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85d8120a40 con 0x7f85d809d630 2023-09-30T22:30:18.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.613+0000 7f85de653700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f85d8122350 con 0x7f85d80a0a30 2023-09-30T22:30:18.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.613+0000 7f85d7fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85d809d630 0x7f85d811e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:18.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.613+0000 7f85d7fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85d809d630 0x7f85d811e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:34178/0 (socket says 172.21.15.16:34178) 2023-09-30T22:30:18.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.614+0000 7f85d7fff700 1 -- 172.21.15.16:0/698630523 learned_addr learned my addr 172.21.15.16:0/698630523 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:18.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.614+0000 7f85d7fff700 1 -- 172.21.15.16:0/698630523 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85d80a0a30 msgr2=0x7f85d811ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:18.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.614+0000 7f85d7fff700 1 --2- 172.21.15.16:0/698630523 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85d80a0a30 0x7f85d811ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.614+0000 7f85d7fff700 1 -- 172.21.15.16:0/698630523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85d8133930 msgr2=0x7f85d811f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:18.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.614+0000 7f85d7fff700 1 --2- 172.21.15.16:0/698630523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85d8133930 0x7f85d811f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.614+0000 7f85d7fff700 1 -- 172.21.15.16:0/698630523 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c8009550 con 0x7f85d809d630 2023-09-30T22:30:18.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.613+0000 7f19f4463700 1 -- start start 2023-09-30T22:30:18.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.614+0000 7f19f4463700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f19ec09d6f0 0x7f19ec12deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:18.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.614+0000 7f19f4463700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19ec09e000 0x7f19ec12e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:18.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.614+0000 7f19f4463700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f19ec09e410 0x7f19ec12e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:18.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.614+0000 7f19f4463700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec1319f0 con 0x7f19ec09e000 2023-09-30T22:30:18.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.614+0000 7f19f4463700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec0a1d00 con 0x7f19ec09e410 2023-09-30T22:30:18.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.614+0000 7f19f4463700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f19ec12fae0 con 0x7f19ec09d6f0 2023-09-30T22:30:18.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.614+0000 7f19f19fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19ec09e000 0x7f19ec12e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:18.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.614+0000 7f19f19fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19ec09e000 0x7f19ec12e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57908/0 (socket says 172.21.15.16:57908) 2023-09-30T22:30:18.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.614+0000 7f19f19fe700 1 -- 172.21.15.16:0/3901903227 learned_addr learned my addr 172.21.15.16:0/3901903227 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:18.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.614+0000 7f85d7fff700 1 --2- 172.21.15.16:0/698630523 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85d809d630 0x7f85d811e8c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f85c8000c00 tx=0x7f85c8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:18.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.614+0000 7f19f2a00700 1 --2- 172.21.15.16:0/3901903227 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f19ec09e410 0x7f19ec12e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:18.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.614+0000 7f19f19fe700 1 -- 172.21.15.16:0/3901903227 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f19ec09d6f0 msgr2=0x7f19ec12deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:18.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.614+0000 7f19f19fe700 1 --2- 172.21.15.16:0/3901903227 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f19ec09d6f0 0x7f19ec12deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.614+0000 7f19f19fe700 1 -- 172.21.15.16:0/3901903227 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f19ec09e410 msgr2=0x7f19ec12e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:18.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.614+0000 7f19f19fe700 1 --2- 172.21.15.16:0/3901903227 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f19ec09e410 0x7f19ec12e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.614+0000 7f19f19fe700 1 -- 172.21.15.16:0/3901903227 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e80094a0 con 0x7f19ec09e000 2023-09-30T22:30:18.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.615+0000 7f19f19fe700 1 --2- 172.21.15.16:0/3901903227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19ec09e000 0x7f19ec12e3f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f19dc00b500 tx=0x7f19dc00b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:18.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.615+0000 7f85d4ff9700 1 -- 172.21.15.16:0/698630523 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c8004400 con 0x7f85d809d630 2023-09-30T22:30:18.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.615+0000 7f85d4ff9700 1 -- 172.21.15.16:0/698630523 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85c801f070 con 0x7f85d809d630 2023-09-30T22:30:18.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.615+0000 7f85d4ff9700 1 -- 172.21.15.16:0/698630523 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c80038e0 con 0x7f85d809d630 2023-09-30T22:30:18.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.615+0000 7f85de653700 1 -- 172.21.15.16:0/698630523 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85d80a50d0 con 0x7f85d809d630 2023-09-30T22:30:18.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.615+0000 7f85de653700 1 -- 172.21.15.16:0/698630523 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85d8120c30 con 0x7f85d809d630 2023-09-30T22:30:18.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.616+0000 7f85d4ff9700 1 -- 172.21.15.16:0/698630523 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f85c8021030 con 0x7f85d809d630 2023-09-30T22:30:18.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.616+0000 7f85d4ff9700 1 -- 172.21.15.16:0/698630523 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f85c8040940 con 0x7f85d809d630 2023-09-30T22:30:18.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.616+0000 7f19e2ffd700 1 -- 172.21.15.16:0/3901903227 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19dc011830 con 0x7f19ec09e000 2023-09-30T22:30:18.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.616+0000 7f19e2ffd700 1 -- 172.21.15.16:0/3901903227 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19dc0119b0 con 0x7f19ec09e000 2023-09-30T22:30:18.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.616+0000 7f19f4463700 1 -- 172.21.15.16:0/3901903227 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19ec09c700 con 0x7f19ec09e000 2023-09-30T22:30:18.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.616+0000 7f19e2ffd700 1 -- 172.21.15.16:0/3901903227 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19dc011d60 con 0x7f19ec09e000 2023-09-30T22:30:18.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.617+0000 7f85de653700 1 -- 172.21.15.16:0/698630523 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85b80052e0 con 0x7f85d809d630 2023-09-30T22:30:18.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.616+0000 7f19f4463700 1 -- 172.21.15.16:0/3901903227 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19ec131cd0 con 0x7f19ec09e000 2023-09-30T22:30:18.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.617+0000 7f19e2ffd700 1 -- 172.21.15.16:0/3901903227 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f19dc01a400 con 0x7f19ec09e000 2023-09-30T22:30:18.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.617+0000 7f19e2ffd700 1 -- 172.21.15.16:0/3901903227 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f19dc03f630 con 0x7f19ec09e000 2023-09-30T22:30:18.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.618+0000 7f19f4463700 1 -- 172.21.15.16:0/3901903227 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19ec131b30 con 0x7f19ec09e000 2023-09-30T22:30:18.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.622+0000 7f85d4ff9700 1 -- 172.21.15.16:0/698630523 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f85c8015080 con 0x7f85d809d630 2023-09-30T22:30:18.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.623+0000 7f19e2ffd700 1 -- 172.21.15.16:0/3901903227 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f19ec131b30 con 0x7f19ec09e000 2023-09-30T22:30:18.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.749+0000 7f3808422700 1 -- 172.21.15.16:0/3195367035 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f38000a1d10 con 0x7f38000a1510 2023-09-30T22:30:18.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.749+0000 7f37f6ffd700 1 -- 172.21.15.16:0/3195367035 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f37f0015020 con 0x7f38000a1510 2023-09-30T22:30:18.751 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:30:18.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.752+0000 7f3808422700 1 -- 172.21.15.16:0/3195367035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38000a1510 msgr2=0x7f380011ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:18.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.752+0000 7f3808422700 1 --2- 172.21.15.16:0/3195367035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38000a1510 0x7f380011ec50 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f37f00090b0 tx=0x7f37f0004750 comp rx=0 tx=0).stop 2023-09-30T22:30:18.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.752+0000 7f3808422700 1 -- 172.21.15.16:0/3195367035 shutdown_connections 2023-09-30T22:30:18.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.752+0000 7f3808422700 1 --2- 172.21.15.16:0/3195367035 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38000a0a30 0x7f380011e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.752+0000 7f3808422700 1 --2- 172.21.15.16:0/3195367035 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3800139250 0x7f380011f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.752+0000 7f3808422700 1 --2- 172.21.15.16:0/3195367035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38000a1510 0x7f380011ec50 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.752+0000 7f3808422700 1 -- 172.21.15.16:0/3195367035 >> 172.21.15.16:0/3195367035 conn(0x7f38000a6550 msgr2=0x7f38000a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:18.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.752+0000 7f3808422700 1 -- 172.21.15.16:0/3195367035 shutdown_connections 2023-09-30T22:30:18.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.752+0000 7f3808422700 1 -- 172.21.15.16:0/3195367035 wait complete. 2023-09-30T22:30:18.767 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:30:18.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.779+0000 7f85f987c700 1 -- 172.21.15.16:0/3521502670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f85f4074ea0 con 0x7f85f409e410 2023-09-30T22:30:18.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.779+0000 7f85dbfff700 1 -- 172.21.15.16:0/3521502670 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f85ec019af0 con 0x7f85f409e410 2023-09-30T22:30:18.781 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:30:18.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.782+0000 7f85f987c700 1 -- 172.21.15.16:0/3521502670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85f409e410 msgr2=0x7f85f4122d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:18.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.782+0000 7f85f987c700 1 --2- 172.21.15.16:0/3521502670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85f409e410 0x7f85f4122d90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f85f409e8b0 tx=0x7f85ec00ca10 comp rx=0 tx=0).stop 2023-09-30T22:30:18.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.782+0000 7f85f987c700 1 -- 172.21.15.16:0/3521502670 shutdown_connections 2023-09-30T22:30:18.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.782+0000 7f85f987c700 1 --2- 172.21.15.16:0/3521502670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85f409e410 0x7f85f4122d90 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.782+0000 7f85f987c700 1 --2- 172.21.15.16:0/3521502670 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85f409d6f0 0x7f85f4122310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.782+0000 7f85f987c700 1 --2- 172.21.15.16:0/3521502670 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85f409e000 0x7f85f4122850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.782+0000 7f85f987c700 1 -- 172.21.15.16:0/3521502670 >> 172.21.15.16:0/3521502670 conn(0x7f85f4066f50 msgr2=0x7f85f4133f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:18.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.782+0000 7f85f987c700 1 -- 172.21.15.16:0/3521502670 shutdown_connections 2023-09-30T22:30:18.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.782+0000 7f85f987c700 1 -- 172.21.15.16:0/3521502670 wait complete. 2023-09-30T22:30:18.797 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:30:18.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.831+0000 7f85de653700 1 -- 172.21.15.16:0/698630523 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f85b8005590 con 0x7f85d809d630 2023-09-30T22:30:18.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.831+0000 7f85d4ff9700 1 -- 172.21.15.16:0/698630523 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f85c8004560 con 0x7f85d809d630 2023-09-30T22:30:18.833 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:30:18.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.833+0000 7f85de653700 1 -- 172.21.15.16:0/698630523 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85d809d630 msgr2=0x7f85d811e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:18.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.833+0000 7f85de653700 1 --2- 172.21.15.16:0/698630523 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85d809d630 0x7f85d811e8c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f85c8000c00 tx=0x7f85c8004750 comp rx=0 tx=0).stop 2023-09-30T22:30:18.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.833+0000 7f85de653700 1 -- 172.21.15.16:0/698630523 shutdown_connections 2023-09-30T22:30:18.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.833+0000 7f85de653700 1 --2- 172.21.15.16:0/698630523 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85d809d630 0x7f85d811e8c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.833+0000 7f85de653700 1 --2- 172.21.15.16:0/698630523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85d8133930 0x7f85d811f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.833+0000 7f85de653700 1 --2- 172.21.15.16:0/698630523 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85d80a0a30 0x7f85d811ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.833+0000 7f85de653700 1 -- 172.21.15.16:0/698630523 >> 172.21.15.16:0/698630523 conn(0x7f85d80a3050 msgr2=0x7f85d80a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:18.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.834+0000 7f85de653700 1 -- 172.21.15.16:0/698630523 shutdown_connections 2023-09-30T22:30:18.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.834+0000 7f85de653700 1 -- 172.21.15.16:0/698630523 wait complete. 2023-09-30T22:30:18.848 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:30:18.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.860+0000 7f19f4463700 1 -- 172.21.15.16:0/3901903227 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f19ec131b30 con 0x7f19ec09e000 2023-09-30T22:30:18.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.860+0000 7f19e2ffd700 1 -- 172.21.15.16:0/3901903227 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f19ec131b30 con 0x7f19ec09e000 2023-09-30T22:30:18.862 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:30:18.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.862+0000 7f19f4463700 1 -- 172.21.15.16:0/3901903227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19ec09e000 msgr2=0x7f19ec12e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:18.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.862+0000 7f19f4463700 1 --2- 172.21.15.16:0/3901903227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19ec09e000 0x7f19ec12e3f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f19dc00b500 tx=0x7f19dc00b810 comp rx=0 tx=0).stop 2023-09-30T22:30:18.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.862+0000 7f19f4463700 1 -- 172.21.15.16:0/3901903227 shutdown_connections 2023-09-30T22:30:18.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.862+0000 7f19f4463700 1 --2- 172.21.15.16:0/3901903227 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f19ec09e410 0x7f19ec12e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.862+0000 7f19f4463700 1 --2- 172.21.15.16:0/3901903227 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f19ec09d6f0 0x7f19ec12deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.862+0000 7f19f4463700 1 --2- 172.21.15.16:0/3901903227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19ec09e000 0x7f19ec12e3f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:18.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.862+0000 7f19f4463700 1 -- 172.21.15.16:0/3901903227 >> 172.21.15.16:0/3901903227 conn(0x7f19ec066f50 msgr2=0x7f19ec0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:18.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.863+0000 7f19f4463700 1 -- 172.21.15.16:0/3901903227 shutdown_connections 2023-09-30T22:30:18.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:18.863+0000 7f19f4463700 1 -- 172.21.15.16:0/3901903227 wait complete. 2023-09-30T22:30:18.877 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:30:19.768 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:19.798 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:19.850 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:19.879 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:19.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.919+0000 7f7d1cab3700 1 Processor -- start 2023-09-30T22:30:19.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.919+0000 7f7d1cab3700 1 -- start start 2023-09-30T22:30:19.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.919+0000 7f7d1cab3700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7d18133540 0x7f7d18135920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:19.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.919+0000 7f7d1cab3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7d18135e60 0x7f7d18138240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:19.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.919+0000 7f7d1cab3700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d180716a0 con 0x7f7d18130540 2023-09-30T22:30:19.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.919+0000 7f7d1cab3700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d18071290 con 0x7f7d18133540 2023-09-30T22:30:19.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.919+0000 7f7d1cab3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7d18130340 con 0x7f7d18135e60 2023-09-30T22:30:19.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.920+0000 7f7d16d9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7d18135e60 0x7f7d18138240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:19.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.920+0000 7f7d16d9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7d18135e60 0x7f7d18138240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:58804/0 (socket says 172.21.15.16:58804) 2023-09-30T22:30:19.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.920+0000 7f7d16d9d700 1 -- 172.21.15.16:0/2779925641 learned_addr learned my addr 172.21.15.16:0/2779925641 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:19.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.920+0000 7f7d15d9b700 1 --2- 172.21.15.16:0/2779925641 >> v2:172.21.15.16:3300/0 conn(0x7f7d18133540 0x7f7d18135920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:19.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.920+0000 7f7d16d9d700 1 -- 172.21.15.16:0/2779925641 >> v2:172.21.15.16:3300/0 conn(0x7f7d18133540 msgr2=0x7f7d18135920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:19.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.920+0000 7f7d16d9d700 1 --2- 172.21.15.16:0/2779925641 >> v2:172.21.15.16:3300/0 conn(0x7f7d18133540 0x7f7d18135920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:19.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.920+0000 7f7d16d9d700 1 -- 172.21.15.16:0/2779925641 >> v1:172.21.15.16:6789/0 conn(0x7f7d18130540 legacy=0x7f7d18130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:19.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.920+0000 7f7d1559a700 1 -- 172.21.15.16:0/2779925641 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 336138807 0 0) 0x7f7d180716a0 con 0x7f7d18130540 2023-09-30T22:30:19.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.920+0000 7f7d16d9d700 1 -- 172.21.15.16:0/2779925641 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d18070b80 con 0x7f7d18135e60 2023-09-30T22:30:19.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.920+0000 7f7d16d9d700 1 --2- 172.21.15.16:0/2779925641 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7d18135e60 0x7f7d18138240 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7d10009e50 tx=0x7f7d1000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c54965f716683941 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:19.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.921+0000 7f7d1559a700 1 -- 172.21.15.16:0/2779925641 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1000bb10 con 0x7f7d18135e60 2023-09-30T22:30:19.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.921+0000 7f7d1559a700 1 -- 172.21.15.16:0/2779925641 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d1000bc70 con 0x7f7d18135e60 2023-09-30T22:30:19.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.921+0000 7f7d1559a700 1 -- 172.21.15.16:0/2779925641 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d10011460 con 0x7f7d18135e60 2023-09-30T22:30:19.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.921+0000 7f7d1cab3700 1 -- 172.21.15.16:0/2779925641 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7d18135e60 msgr2=0x7f7d18138240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:19.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.921+0000 7f7d1cab3700 1 --2- 172.21.15.16:0/2779925641 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7d18135e60 0x7f7d18138240 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7d10009e50 tx=0x7f7d1000b330 comp rx=0 tx=0).stop 2023-09-30T22:30:19.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.922+0000 7f7d1cab3700 1 -- 172.21.15.16:0/2779925641 shutdown_connections 2023-09-30T22:30:19.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.922+0000 7f7d1cab3700 1 --2- 172.21.15.16:0/2779925641 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7d18135e60 0x7f7d18138240 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:19.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.922+0000 7f7d1cab3700 1 --2- 172.21.15.16:0/2779925641 >> v2:172.21.15.16:3300/0 conn(0x7f7d18133540 0x7f7d18135920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:19.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.922+0000 7f7d1cab3700 1 -- 172.21.15.16:0/2779925641 >> 172.21.15.16:0/2779925641 conn(0x7f7d180a2aa0 msgr2=0x7f7d180a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:19.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.922+0000 7f7d1cab3700 1 -- 172.21.15.16:0/2779925641 shutdown_connections 2023-09-30T22:30:19.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.922+0000 7f7d1cab3700 1 -- 172.21.15.16:0/2779925641 wait complete. 2023-09-30T22:30:19.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.923+0000 7f7d1cab3700 1 Processor -- start 2023-09-30T22:30:19.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.923+0000 7f7d1cab3700 1 -- start start 2023-09-30T22:30:19.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.924+0000 7f7d1cab3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d18130540 0x7f7d1809c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:19.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.924+0000 7f7d1cab3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7d18133540 0x7f7d1809cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:19.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.924+0000 7f7d1cab3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7d18135e60 0x7f7d1809d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:19.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.924+0000 7f7d1cab3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1809e3a0 con 0x7f7d18130540 2023-09-30T22:30:19.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.924+0000 7f7d1cab3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d1811a5f0 con 0x7f7d18135e60 2023-09-30T22:30:19.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.924+0000 7f7d1cab3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7d18135ac0 con 0x7f7d18133540 2023-09-30T22:30:19.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.924+0000 7f7d15d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7d18133540 0x7f7d1809cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:19.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.924+0000 7f7d15d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7d18133540 0x7f7d1809cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:58812/0 (socket says 172.21.15.16:58812) 2023-09-30T22:30:19.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.924+0000 7f7d15d9b700 1 -- 172.21.15.16:0/145094232 learned_addr learned my addr 172.21.15.16:0/145094232 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:19.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.925+0000 7f7d16d9d700 1 --2- 172.21.15.16:0/145094232 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7d18135e60 0x7f7d1809d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:19.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.925+0000 7f7d15d9b700 1 -- 172.21.15.16:0/145094232 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7d18135e60 msgr2=0x7f7d1809d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:19.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.925+0000 7f7d15d9b700 1 --2- 172.21.15.16:0/145094232 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7d18135e60 0x7f7d1809d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:19.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.925+0000 7f7d15d9b700 1 -- 172.21.15.16:0/145094232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d18130540 msgr2=0x7f7d1809c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:19.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.925+0000 7f7d15d9b700 1 --2- 172.21.15.16:0/145094232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d18130540 0x7f7d1809c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:19.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.925+0000 7f7d15d9b700 1 -- 172.21.15.16:0/145094232 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d10009b30 con 0x7f7d18133540 2023-09-30T22:30:19.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.925+0000 7f7d15d9b700 1 --2- 172.21.15.16:0/145094232 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7d18133540 0x7f7d1809cbc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c00e980 tx=0x7f7d0c00ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:19.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.925+0000 7f7d06ffd700 1 -- 172.21.15.16:0/145094232 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c009710 con 0x7f7d18133540 2023-09-30T22:30:19.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.925+0000 7f7d06ffd700 1 -- 172.21.15.16:0/145094232 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d0c00eed0 con 0x7f7d18133540 2023-09-30T22:30:19.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.925+0000 7f7d1cab3700 1 -- 172.21.15.16:0/145094232 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d180718e0 con 0x7f7d18133540 2023-09-30T22:30:19.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.925+0000 7f7d06ffd700 1 -- 172.21.15.16:0/145094232 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c00ca90 con 0x7f7d18133540 2023-09-30T22:30:19.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.926+0000 7f7d1cab3700 1 -- 172.21.15.16:0/145094232 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d18119460 con 0x7f7d18133540 2023-09-30T22:30:19.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.927+0000 7f7d1cab3700 1 -- 172.21.15.16:0/145094232 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d18074ea0 con 0x7f7d18133540 2023-09-30T22:30:19.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.927+0000 7f7d06ffd700 1 -- 172.21.15.16:0/145094232 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7d0c00cbf0 con 0x7f7d18133540 2023-09-30T22:30:19.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.927+0000 7f7d06ffd700 1 -- 172.21.15.16:0/145094232 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f7d0c047350 con 0x7f7d18133540 2023-09-30T22:30:19.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.933+0000 7f7d06ffd700 1 -- 172.21.15.16:0/145094232 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7d0c044060 con 0x7f7d18133540 2023-09-30T22:30:19.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.962+0000 7f283ae9b700 1 Processor -- start 2023-09-30T22:30:19.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.962+0000 7f283ae9b700 1 -- start start 2023-09-30T22:30:19.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.962+0000 7f283ae9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f283409e930 0x7f283409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:19.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.962+0000 7f283ae9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f283409f240 0x7f28341317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:19.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.962+0000 7f283ae9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f283409f650 0x7f2834131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:19.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.962+0000 7f283ae9b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2834070940 con 0x7f283409f650 2023-09-30T22:30:19.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.962+0000 7f283ae9b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f283406fd60 con 0x7f283409e930 2023-09-30T22:30:19.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.962+0000 7f283ae9b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f283406fa80 con 0x7f283409f240 2023-09-30T22:30:19.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.963+0000 7f2833fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f283409f240 0x7f28341317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:19.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.963+0000 7f2833fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f283409f240 0x7f28341317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:58826/0 (socket says 172.21.15.16:58826) 2023-09-30T22:30:19.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.963+0000 7f2833fff700 1 -- 172.21.15.16:0/3806328587 learned_addr learned my addr 172.21.15.16:0/3806328587 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:19.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.963+0000 7f2838c37700 1 --2- 172.21.15.16:0/3806328587 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f283409e930 0x7f283409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:19.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.963+0000 7f2839438700 1 --2- 172.21.15.16:0/3806328587 >> v2:172.21.15.16:3300/0 conn(0x7f283409f650 0x7f2834131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:19.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.963+0000 7f2833fff700 1 -- 172.21.15.16:0/3806328587 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f283409e930 msgr2=0x7f283409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:19.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.963+0000 7f2833fff700 1 --2- 172.21.15.16:0/3806328587 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f283409e930 0x7f283409ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:19.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.963+0000 7f2833fff700 1 -- 172.21.15.16:0/3806328587 >> v2:172.21.15.16:3300/0 conn(0x7f283409f650 msgr2=0x7f2834131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:19.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.963+0000 7f2833fff700 1 --2- 172.21.15.16:0/3806328587 >> v2:172.21.15.16:3300/0 conn(0x7f283409f650 0x7f2834131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:19.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.963+0000 7f2833fff700 1 -- 172.21.15.16:0/3806328587 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f283409fb60 con 0x7f283409f240 2023-09-30T22:30:19.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.963+0000 7f2833fff700 1 --2- 172.21.15.16:0/3806328587 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f283409f240 0x7f28341317f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f28200098f0 tx=0x7f2820009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39be340a859d2910 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:19.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.964+0000 7f28337fe700 1 -- 172.21.15.16:0/3806328587 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2820011830 con 0x7f283409f240 2023-09-30T22:30:19.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.964+0000 7f28337fe700 1 -- 172.21.15.16:0/3806328587 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2820011990 con 0x7f283409f240 2023-09-30T22:30:19.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.964+0000 7f28337fe700 1 -- 172.21.15.16:0/3806328587 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2820011d20 con 0x7f283409f240 2023-09-30T22:30:19.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.964+0000 7f283ae9b700 1 -- 172.21.15.16:0/3806328587 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f283409f240 msgr2=0x7f28341317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:19.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.964+0000 7f283ae9b700 1 --2- 172.21.15.16:0/3806328587 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f283409f240 0x7f28341317f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f28200098f0 tx=0x7f2820009c00 comp rx=0 tx=0).stop 2023-09-30T22:30:19.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.964+0000 7f283ae9b700 1 -- 172.21.15.16:0/3806328587 shutdown_connections 2023-09-30T22:30:19.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.964+0000 7f283ae9b700 1 --2- 172.21.15.16:0/3806328587 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f283409e930 0x7f283409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:19.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.964+0000 7f283ae9b700 1 --2- 172.21.15.16:0/3806328587 >> v2:172.21.15.16:3300/0 conn(0x7f283409f650 0x7f2834131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:19.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.964+0000 7f283ae9b700 1 --2- 172.21.15.16:0/3806328587 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f283409f240 0x7f28341317f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:19.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.964+0000 7f283ae9b700 1 -- 172.21.15.16:0/3806328587 >> 172.21.15.16:0/3806328587 conn(0x7f28340a4210 msgr2=0x7f28340a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:19.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.965+0000 7f283ae9b700 1 -- 172.21.15.16:0/3806328587 shutdown_connections 2023-09-30T22:30:19.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.965+0000 7f283ae9b700 1 -- 172.21.15.16:0/3806328587 wait complete. 2023-09-30T22:30:19.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.966+0000 7f283ae9b700 1 Processor -- start 2023-09-30T22:30:19.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.966+0000 7f283ae9b700 1 -- start start 2023-09-30T22:30:19.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.966+0000 7f283ae9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f283409e930 0x7f2834123e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:19.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.966+0000 7f283ae9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f283409f240 0x7f28341229b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:19.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.966+0000 7f283ae9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f283409f650 0x7f2834122ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:19.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.966+0000 7f283ae9b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2834131670 con 0x7f283409f650 2023-09-30T22:30:19.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.966+0000 7f283ae9b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f283406d480 con 0x7f283409e930 2023-09-30T22:30:19.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.966+0000 7f283ae9b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f28340a31d0 con 0x7f283409f240 2023-09-30T22:30:19.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.967+0000 7f2838c37700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f283409e930 0x7f2834123e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:19.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.967+0000 7f2838c37700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f283409e930 0x7f2834123e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:34212/0 (socket says 172.21.15.16:34212) 2023-09-30T22:30:19.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.967+0000 7f2838c37700 1 -- 172.21.15.16:0/725590217 learned_addr learned my addr 172.21.15.16:0/725590217 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:19.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.967+0000 7f2833fff700 1 --2- 172.21.15.16:0/725590217 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f283409f240 0x7f28341229b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:19.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.967+0000 7f2839438700 1 --2- 172.21.15.16:0/725590217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f283409f650 0x7f2834122ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:19.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.967+0000 7f2838c37700 1 -- 172.21.15.16:0/725590217 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f283409f240 msgr2=0x7f28341229b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:19.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.967+0000 7f2838c37700 1 --2- 172.21.15.16:0/725590217 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f283409f240 0x7f28341229b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:19.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.967+0000 7f2838c37700 1 -- 172.21.15.16:0/725590217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f283409f650 msgr2=0x7f2834122ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:19.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.967+0000 7f2838c37700 1 --2- 172.21.15.16:0/725590217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f283409f650 0x7f2834122ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:19.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.967+0000 7f2838c37700 1 -- 172.21.15.16:0/725590217 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2820009550 con 0x7f283409e930 2023-09-30T22:30:19.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.967+0000 7f2838c37700 1 --2- 172.21.15.16:0/725590217 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f283409e930 0x7f2834123e80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f282800e960 tx=0x7f282800ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:19.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.968+0000 7f28317fa700 1 -- 172.21.15.16:0/725590217 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f282800eeb0 con 0x7f283409e930 2023-09-30T22:30:19.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.968+0000 7f28317fa700 1 -- 172.21.15.16:0/725590217 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f282800c850 con 0x7f283409e930 2023-09-30T22:30:19.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.968+0000 7f28317fa700 1 -- 172.21.15.16:0/725590217 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f282800cbe0 con 0x7f283409e930 2023-09-30T22:30:19.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.968+0000 7f283ae9b700 1 -- 172.21.15.16:0/725590217 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28340a0190 con 0x7f283409e930 2023-09-30T22:30:19.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.968+0000 7f283ae9b700 1 -- 172.21.15.16:0/725590217 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2834121830 con 0x7f283409e930 2023-09-30T22:30:19.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.969+0000 7f28317fa700 1 -- 172.21.15.16:0/725590217 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f282800cd40 con 0x7f283409e930 2023-09-30T22:30:19.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.969+0000 7f28317fa700 1 -- 172.21.15.16:0/725590217 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f282803fa80 con 0x7f283409e930 2023-09-30T22:30:19.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.970+0000 7f283ae9b700 1 -- 172.21.15.16:0/725590217 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f283411d540 con 0x7f283409e930 2023-09-30T22:30:19.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:19.979+0000 7f28317fa700 1 -- 172.21.15.16:0/725590217 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2828017020 con 0x7f283409e930 2023-09-30T22:30:20.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.004+0000 7f1cc4906700 1 Processor -- start 2023-09-30T22:30:20.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.005+0000 7f1cc4906700 1 -- start start 2023-09-30T22:30:20.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.005+0000 7f1cc4906700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1cc006a770 0x7f1cc009d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:20.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.005+0000 7f1cc4906700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1cc013d1f0 0x7f1cc013f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:20.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.005+0000 7f1cc4906700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cc009a810 con 0x7f1cc009dc30 2023-09-30T22:30:20.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.005+0000 7f1cc4906700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cc006ab40 con 0x7f1cc006a770 2023-09-30T22:30:20.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.005+0000 7f1cc4906700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc009ebc0 con 0x7f1cc013d1f0 2023-09-30T22:30:20.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.006+0000 7f1cbdd9b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1cc009dc30 0x7f1cc009e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56942/0 (socket says 172.21.15.16:56942) 2023-09-30T22:30:20.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.006+0000 7f1cbe59c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1cc006a770 0x7f1cc009d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:20.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.006+0000 7f1cbdd9b700 1 -- 172.21.15.16:0/2704507596 learned_addr learned my addr 172.21.15.16:0/2704507596 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:30:20.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.006+0000 7f1cbed9d700 1 --2- 172.21.15.16:0/2704507596 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1cc013d1f0 0x7f1cc013f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:20.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.006+0000 7f1cbd59a700 1 -- 172.21.15.16:0/2704507596 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1139033103 0 0) 0x7f1cc009a810 con 0x7f1cc009dc30 2023-09-30T22:30:20.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.006+0000 7f1cbd59a700 1 -- 172.21.15.16:0/2704507596 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ca80035b0 con 0x7f1cc009dc30 2023-09-30T22:30:20.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.006+0000 7f1cbed9d700 1 -- 172.21.15.16:0/2704507596 >> v2:172.21.15.16:3300/0 conn(0x7f1cc006a770 msgr2=0x7f1cc009d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:20.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.006+0000 7f1cbed9d700 1 --2- 172.21.15.16:0/2704507596 >> v2:172.21.15.16:3300/0 conn(0x7f1cc006a770 0x7f1cc009d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:20.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.006+0000 7f1cbed9d700 1 -- 172.21.15.16:0/2704507596 >> v1:172.21.15.16:6789/0 conn(0x7f1cc009dc30 legacy=0x7f1cc009e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:20.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.006+0000 7f1cbed9d700 1 -- 172.21.15.16:0/2704507596 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cc009e920 con 0x7f1cc013d1f0 2023-09-30T22:30:20.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.007+0000 7f1cbed9d700 1 --2- 172.21.15.16:0/2704507596 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1cc013d1f0 0x7f1cc013f5d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8009d90 tx=0x7f1cb800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddc71ea0803e6fdf server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:20.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.007+0000 7f1cbd59a700 1 -- 172.21.15.16:0/2704507596 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb800e070 con 0x7f1cc013d1f0 2023-09-30T22:30:20.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.007+0000 7f1cbd59a700 1 -- 172.21.15.16:0/2704507596 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cb800bb20 con 0x7f1cc013d1f0 2023-09-30T22:30:20.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.007+0000 7f1cbd59a700 1 -- 172.21.15.16:0/2704507596 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb8012430 con 0x7f1cc013d1f0 2023-09-30T22:30:20.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.007+0000 7f1cc4906700 1 -- 172.21.15.16:0/2704507596 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1cc013d1f0 msgr2=0x7f1cc013f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:20.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.007+0000 7f1cc4906700 1 --2- 172.21.15.16:0/2704507596 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1cc013d1f0 0x7f1cc013f5d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8009d90 tx=0x7f1cb800b330 comp rx=0 tx=0).stop 2023-09-30T22:30:20.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.008+0000 7f1cc4906700 1 -- 172.21.15.16:0/2704507596 shutdown_connections 2023-09-30T22:30:20.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.008+0000 7f1cc4906700 1 --2- 172.21.15.16:0/2704507596 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1cc013d1f0 0x7f1cc013f5d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:20.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.008+0000 7f1cc4906700 1 --2- 172.21.15.16:0/2704507596 >> v2:172.21.15.16:3300/0 conn(0x7f1cc006a770 0x7f1cc009d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:20.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.008+0000 7f1cc4906700 1 -- 172.21.15.16:0/2704507596 >> 172.21.15.16:0/2704507596 conn(0x7f1cc0066f50 msgr2=0x7f1cc009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:20.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.008+0000 7f1cc4906700 1 -- 172.21.15.16:0/2704507596 shutdown_connections 2023-09-30T22:30:20.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.008+0000 7f1cc4906700 1 -- 172.21.15.16:0/2704507596 wait complete. 2023-09-30T22:30:20.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.009+0000 7f1cc4906700 1 Processor -- start 2023-09-30T22:30:20.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.009+0000 7f1cc4906700 1 -- start start 2023-09-30T22:30:20.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.009+0000 7f1cc4906700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cc006a770 0x7f1cc0071670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:20.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.009+0000 7f1cc4906700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1cc009dc30 0x7f1cc013b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:20.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.009+0000 7f1cc4906700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1cc013d1f0 0x7f1cc013bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:20.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.009+0000 7f1cc4906700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc0135cf0 con 0x7f1cc006a770 2023-09-30T22:30:20.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.009+0000 7f1cc4906700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc011e260 con 0x7f1cc009dc30 2023-09-30T22:30:20.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.009+0000 7f1cc4906700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1cc011e980 con 0x7f1cc013d1f0 2023-09-30T22:30:20.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.010+0000 7f1cbed9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1cc013d1f0 0x7f1cc013bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:20.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.010+0000 7f1cbed9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1cc013d1f0 0x7f1cc013bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:58842/0 (socket says 172.21.15.16:58842) 2023-09-30T22:30:20.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.010+0000 7f1cbed9d700 1 -- 172.21.15.16:0/2406072741 learned_addr learned my addr 172.21.15.16:0/2406072741 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:20.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.010+0000 7f1cbe59c700 1 --2- 172.21.15.16:0/2406072741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cc006a770 0x7f1cc0071670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:20.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.010+0000 7f1cbdd9b700 1 --2- 172.21.15.16:0/2406072741 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1cc009dc30 0x7f1cc013b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:20.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.010+0000 7f1cbed9d700 1 -- 172.21.15.16:0/2406072741 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1cc009dc30 msgr2=0x7f1cc013b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:20.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.010+0000 7f1cbed9d700 1 --2- 172.21.15.16:0/2406072741 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1cc009dc30 0x7f1cc013b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:20.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.010+0000 7f1cbed9d700 1 -- 172.21.15.16:0/2406072741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cc006a770 msgr2=0x7f1cc0071670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:20.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.010+0000 7f1cbed9d700 1 --2- 172.21.15.16:0/2406072741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cc006a770 0x7f1cc0071670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:20.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.010+0000 7f1cbed9d700 1 -- 172.21.15.16:0/2406072741 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb8009a70 con 0x7f1cc013d1f0 2023-09-30T22:30:20.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.010+0000 7f1cbe59c700 1 --2- 172.21.15.16:0/2406072741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cc006a770 0x7f1cc0071670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:20.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.010+0000 7f1cbed9d700 1 --2- 172.21.15.16:0/2406072741 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1cc013d1f0 0x7f1cc013bad0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8013000 tx=0x7f1cb8004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:20.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.011+0000 7f1caeffd700 1 -- 172.21.15.16:0/2406072741 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb80129f0 con 0x7f1cc013d1f0 2023-09-30T22:30:20.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.011+0000 7f1caeffd700 1 -- 172.21.15.16:0/2406072741 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cb8020070 con 0x7f1cc013d1f0 2023-09-30T22:30:20.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.011+0000 7f1cc4906700 1 -- 172.21.15.16:0/2406072741 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cc011e6b0 con 0x7f1cc013d1f0 2023-09-30T22:30:20.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.011+0000 7f1caeffd700 1 -- 172.21.15.16:0/2406072741 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb8012db0 con 0x7f1cc013d1f0 2023-09-30T22:30:20.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.011+0000 7f1cc4906700 1 -- 172.21.15.16:0/2406072741 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cc011f6b0 con 0x7f1cc013d1f0 2023-09-30T22:30:20.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.012+0000 7f1caeffd700 1 -- 172.21.15.16:0/2406072741 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1cb8004040 con 0x7f1cc013d1f0 2023-09-30T22:30:20.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.012+0000 7f1cc4906700 1 -- 172.21.15.16:0/2406072741 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cc0074ea0 con 0x7f1cc013d1f0 2023-09-30T22:30:20.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.012+0000 7f1caeffd700 1 -- 172.21.15.16:0/2406072741 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f1cb8022070 con 0x7f1cc013d1f0 2023-09-30T22:30:20.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.021+0000 7f1caeffd700 1 -- 172.21.15.16:0/2406072741 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1cb8025970 con 0x7f1cc013d1f0 2023-09-30T22:30:20.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.052+0000 7ffa10e3f700 1 Processor -- start 2023-09-30T22:30:20.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.052+0000 7ffa10e3f700 1 -- start start 2023-09-30T22:30:20.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.053+0000 7ffa10e3f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa0c09f410 0x7ffa0c136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:20.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.053+0000 7ffa10e3f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ffa0c136fe0 0x7ffa0c1393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:20.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.053+0000 7ffa10e3f700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa0c070940 con 0x7ffa0c09e930 2023-09-30T22:30:20.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.053+0000 7ffa10e3f700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa0c06fd60 con 0x7ffa0c136fe0 2023-09-30T22:30:20.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.053+0000 7ffa10e3f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0c06fa80 con 0x7ffa0c09f410 2023-09-30T22:30:20.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.053+0000 7ffa09d9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa0c09f410 0x7ffa0c136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:20.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.053+0000 7ffa09d9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa0c09f410 0x7ffa0c136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:34242/0 (socket says 172.21.15.16:34242) 2023-09-30T22:30:20.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.053+0000 7ffa09d9b700 1 -- 172.21.15.16:0/3541081459 learned_addr learned my addr 172.21.15.16:0/3541081459 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:20.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.054+0000 7ffa0ad9d700 1 --2- 172.21.15.16:0/3541081459 >> v2:172.21.15.16:3300/0 conn(0x7ffa0c136fe0 0x7ffa0c1393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:20.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.054+0000 7ffa09d9b700 1 -- 172.21.15.16:0/3541081459 >> v2:172.21.15.16:3300/0 conn(0x7ffa0c136fe0 msgr2=0x7ffa0c1393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:20.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.054+0000 7ffa09d9b700 1 --2- 172.21.15.16:0/3541081459 >> v2:172.21.15.16:3300/0 conn(0x7ffa0c136fe0 0x7ffa0c1393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:20.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.054+0000 7ffa09d9b700 1 -- 172.21.15.16:0/3541081459 >> v1:172.21.15.16:6789/0 conn(0x7ffa0c09e930 legacy=0x7ffa0c09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:20.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.054+0000 7ffa09d9b700 1 -- 172.21.15.16:0/3541081459 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa0c09fb60 con 0x7ffa0c09f410 2023-09-30T22:30:20.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.054+0000 7ffa09d9b700 1 --2- 172.21.15.16:0/3541081459 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa0c09f410 0x7ffa0c136aa0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ffa000098f0 tx=0x7ffa00009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=735eabe4cb559dc8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:20.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.054+0000 7ffa0959a700 1 -- 172.21.15.16:0/3541081459 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa00011830 con 0x7ffa0c09f410 2023-09-30T22:30:20.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.054+0000 7ffa0959a700 1 -- 172.21.15.16:0/3541081459 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa00011990 con 0x7ffa0c09f410 2023-09-30T22:30:20.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.054+0000 7ffa0959a700 1 -- 172.21.15.16:0/3541081459 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa00011d20 con 0x7ffa0c09f410 2023-09-30T22:30:20.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.055+0000 7ffa10e3f700 1 -- 172.21.15.16:0/3541081459 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa0c09f410 msgr2=0x7ffa0c136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:20.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.055+0000 7ffa10e3f700 1 --2- 172.21.15.16:0/3541081459 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa0c09f410 0x7ffa0c136aa0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ffa000098f0 tx=0x7ffa00009c00 comp rx=0 tx=0).stop 2023-09-30T22:30:20.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.055+0000 7ffa10e3f700 1 -- 172.21.15.16:0/3541081459 shutdown_connections 2023-09-30T22:30:20.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.055+0000 7ffa10e3f700 1 --2- 172.21.15.16:0/3541081459 >> v2:172.21.15.16:3300/0 conn(0x7ffa0c136fe0 0x7ffa0c1393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:20.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.055+0000 7ffa10e3f700 1 --2- 172.21.15.16:0/3541081459 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa0c09f410 0x7ffa0c136aa0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:20.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.055+0000 7ffa10e3f700 1 -- 172.21.15.16:0/3541081459 >> 172.21.15.16:0/3541081459 conn(0x7ffa0c0a4210 msgr2=0x7ffa0c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:20.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.055+0000 7ffa10e3f700 1 -- 172.21.15.16:0/3541081459 shutdown_connections 2023-09-30T22:30:20.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.055+0000 7ffa10e3f700 1 -- 172.21.15.16:0/3541081459 wait complete. 2023-09-30T22:30:20.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.056+0000 7ffa10e3f700 1 Processor -- start 2023-09-30T22:30:20.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.056+0000 7ffa10e3f700 1 -- start start 2023-09-30T22:30:20.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.057+0000 7ffa10e3f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa0c09e930 0x7ffa0c11a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:20.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.057+0000 7ffa10e3f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa0c09f410 0x7ffa0c11abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:20.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.057+0000 7ffa10e3f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa0c136fe0 0x7ffa0c11b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:20.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.057+0000 7ffa10e3f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0c11d670 con 0x7ffa0c09e930 2023-09-30T22:30:20.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.057+0000 7ffa10e3f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0c071d20 con 0x7ffa0c136fe0 2023-09-30T22:30:20.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.057+0000 7ffa10e3f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0c06d780 con 0x7ffa0c09f410 2023-09-30T22:30:20.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.057+0000 7ffa0ad9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa0c136fe0 0x7ffa0c11b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:20.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.057+0000 7ffa0ad9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa0c136fe0 0x7ffa0c11b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:34258/0 (socket says 172.21.15.16:34258) 2023-09-30T22:30:20.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.057+0000 7ffa0ad9d700 1 -- 172.21.15.16:0/1030228185 learned_addr learned my addr 172.21.15.16:0/1030228185 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:20.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.057+0000 7ffa0a59c700 1 --2- 172.21.15.16:0/1030228185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa0c09e930 0x7ffa0c11a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:20.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.057+0000 7ffa09d9b700 1 --2- 172.21.15.16:0/1030228185 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa0c09f410 0x7ffa0c11abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:20.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.057+0000 7ffa0ad9d700 1 -- 172.21.15.16:0/1030228185 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa0c09f410 msgr2=0x7ffa0c11abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:20.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.057+0000 7ffa0ad9d700 1 --2- 172.21.15.16:0/1030228185 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa0c09f410 0x7ffa0c11abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:20.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.057+0000 7ffa0ad9d700 1 -- 172.21.15.16:0/1030228185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa0c09e930 msgr2=0x7ffa0c11a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:20.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.057+0000 7ffa0ad9d700 1 --2- 172.21.15.16:0/1030228185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa0c09e930 0x7ffa0c11a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:20.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.058+0000 7ffa0ad9d700 1 -- 172.21.15.16:0/1030228185 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa00009550 con 0x7ffa0c136fe0 2023-09-30T22:30:20.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.058+0000 7ffa0ad9d700 1 --2- 172.21.15.16:0/1030228185 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa0c136fe0 0x7ffa0c11b100 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ffa04012320 tx=0x7ffa04012630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:20.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.058+0000 7ff9faffd700 1 -- 172.21.15.16:0/1030228185 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa04014040 con 0x7ffa0c136fe0 2023-09-30T22:30:20.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.058+0000 7ff9faffd700 1 -- 172.21.15.16:0/1030228185 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa04018620 con 0x7ffa0c136fe0 2023-09-30T22:30:20.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.058+0000 7ffa10e3f700 1 -- 172.21.15.16:0/1030228185 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa0c11bfd0 con 0x7ffa0c136fe0 2023-09-30T22:30:20.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.058+0000 7ff9faffd700 1 -- 172.21.15.16:0/1030228185 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa040189d0 con 0x7ffa0c136fe0 2023-09-30T22:30:20.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.058+0000 7ffa10e3f700 1 -- 172.21.15.16:0/1030228185 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa0c11c7d0 con 0x7ffa0c136fe0 2023-09-30T22:30:20.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.059+0000 7ffa10e3f700 1 -- 172.21.15.16:0/1030228185 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa0c11d8a0 con 0x7ffa0c136fe0 2023-09-30T22:30:20.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.059+0000 7ff9faffd700 1 -- 172.21.15.16:0/1030228185 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ffa04018b30 con 0x7ffa0c136fe0 2023-09-30T22:30:20.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.060+0000 7ff9faffd700 1 -- 172.21.15.16:0/1030228185 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7ffa04044300 con 0x7ffa0c136fe0 2023-09-30T22:30:20.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.069+0000 7ff9faffd700 1 -- 172.21.15.16:0/1030228185 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ffa0c11d8a0 con 0x7ffa0c136fe0 2023-09-30T22:30:20.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.151+0000 7f7d1cab3700 1 -- 172.21.15.16:0/145094232 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7d1811a930 con 0x7f7d18133540 2023-09-30T22:30:20.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.152+0000 7f7d06ffd700 1 -- 172.21.15.16:0/145094232 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7d0c017020 con 0x7f7d18133540 2023-09-30T22:30:20.153 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:30:20.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.154+0000 7f7d1cab3700 1 -- 172.21.15.16:0/145094232 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7d18133540 msgr2=0x7f7d1809cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:20.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.154+0000 7f7d1cab3700 1 --2- 172.21.15.16:0/145094232 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7d18133540 0x7f7d1809cbc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c00e980 tx=0x7f7d0c00ec90 comp rx=0 tx=0).stop 2023-09-30T22:30:20.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.154+0000 7f7d1cab3700 1 -- 172.21.15.16:0/145094232 shutdown_connections 2023-09-30T22:30:20.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.154+0000 7f7d1cab3700 1 --2- 172.21.15.16:0/145094232 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7d18135e60 0x7f7d1809d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:20.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.154+0000 7f7d1cab3700 1 --2- 172.21.15.16:0/145094232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d18130540 0x7f7d1809c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:20.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.154+0000 7f7d1cab3700 1 --2- 172.21.15.16:0/145094232 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7d18133540 0x7f7d1809cbc0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:20.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.154+0000 7f7d1cab3700 1 -- 172.21.15.16:0/145094232 >> 172.21.15.16:0/145094232 conn(0x7f7d180a2aa0 msgr2=0x7f7d18131300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:20.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.154+0000 7f7d1cab3700 1 -- 172.21.15.16:0/145094232 shutdown_connections 2023-09-30T22:30:20.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.154+0000 7f7d1cab3700 1 -- 172.21.15.16:0/145094232 wait complete. 2023-09-30T22:30:20.170 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:30:20.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.191+0000 7f283ae9b700 1 -- 172.21.15.16:0/725590217 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f2834121a20 con 0x7f283409e930 2023-09-30T22:30:20.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.191+0000 7f28317fa700 1 -- 172.21.15.16:0/725590217 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f282802b090 con 0x7f283409e930 2023-09-30T22:30:20.192 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:30:20.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.194+0000 7f283ae9b700 1 -- 172.21.15.16:0/725590217 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f283409e930 msgr2=0x7f2834123e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:20.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.194+0000 7f283ae9b700 1 --2- 172.21.15.16:0/725590217 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f283409e930 0x7f2834123e80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f282800e960 tx=0x7f282800ec70 comp rx=0 tx=0).stop 2023-09-30T22:30:20.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.195+0000 7f283ae9b700 1 -- 172.21.15.16:0/725590217 shutdown_connections 2023-09-30T22:30:20.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.195+0000 7f283ae9b700 1 --2- 172.21.15.16:0/725590217 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f283409e930 0x7f2834123e80 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:20.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.195+0000 7f283ae9b700 1 --2- 172.21.15.16:0/725590217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f283409f650 0x7f2834122ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:20.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.195+0000 7f283ae9b700 1 --2- 172.21.15.16:0/725590217 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f283409f240 0x7f28341229b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:20.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.195+0000 7f283ae9b700 1 -- 172.21.15.16:0/725590217 >> 172.21.15.16:0/725590217 conn(0x7f28340a4210 msgr2=0x7f283411a680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:20.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.195+0000 7f283ae9b700 1 -- 172.21.15.16:0/725590217 shutdown_connections 2023-09-30T22:30:20.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.195+0000 7f283ae9b700 1 -- 172.21.15.16:0/725590217 wait complete. 2023-09-30T22:30:20.215 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:30:20.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.235+0000 7f1cc4906700 1 -- 172.21.15.16:0/2406072741 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1cc009a3d0 con 0x7f1cc013d1f0 2023-09-30T22:30:20.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.236+0000 7f1caeffd700 1 -- 172.21.15.16:0/2406072741 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1cb8036090 con 0x7f1cc013d1f0 2023-09-30T22:30:20.237 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:30:20.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.238+0000 7f1cc4906700 1 -- 172.21.15.16:0/2406072741 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1cc013d1f0 msgr2=0x7f1cc013bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:20.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.238+0000 7f1cc4906700 1 --2- 172.21.15.16:0/2406072741 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1cc013d1f0 0x7f1cc013bad0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8013000 tx=0x7f1cb8004ad0 comp rx=0 tx=0).stop 2023-09-30T22:30:20.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.238+0000 7f1cc4906700 1 -- 172.21.15.16:0/2406072741 shutdown_connections 2023-09-30T22:30:20.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.238+0000 7f1cc4906700 1 --2- 172.21.15.16:0/2406072741 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1cc013d1f0 0x7f1cc013bad0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:20.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.238+0000 7f1cc4906700 1 --2- 172.21.15.16:0/2406072741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cc006a770 0x7f1cc0071670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:20.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.238+0000 7f1cc4906700 1 --2- 172.21.15.16:0/2406072741 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1cc009dc30 0x7f1cc013b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:20.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.238+0000 7f1cc4906700 1 -- 172.21.15.16:0/2406072741 >> 172.21.15.16:0/2406072741 conn(0x7f1cc0066f50 msgr2=0x7f1cc009b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:20.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.238+0000 7f1cc4906700 1 -- 172.21.15.16:0/2406072741 shutdown_connections 2023-09-30T22:30:20.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.238+0000 7f1cc4906700 1 -- 172.21.15.16:0/2406072741 wait complete. 2023-09-30T22:30:20.253 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:30:20.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.288+0000 7ffa10e3f700 1 -- 172.21.15.16:0/1030228185 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ffa0c11d8a0 con 0x7ffa0c136fe0 2023-09-30T22:30:20.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.289+0000 7ff9faffd700 1 -- 172.21.15.16:0/1030228185 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ffa0c11d8a0 con 0x7ffa0c136fe0 2023-09-30T22:30:20.290 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:30:20.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.291+0000 7ffa10e3f700 1 -- 172.21.15.16:0/1030228185 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa0c136fe0 msgr2=0x7ffa0c11b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:20.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.291+0000 7ffa10e3f700 1 --2- 172.21.15.16:0/1030228185 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa0c136fe0 0x7ffa0c11b100 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ffa04012320 tx=0x7ffa04012630 comp rx=0 tx=0).stop 2023-09-30T22:30:20.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.291+0000 7ffa10e3f700 1 -- 172.21.15.16:0/1030228185 shutdown_connections 2023-09-30T22:30:20.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.291+0000 7ffa10e3f700 1 --2- 172.21.15.16:0/1030228185 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa0c136fe0 0x7ffa0c11b100 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:20.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.291+0000 7ffa10e3f700 1 --2- 172.21.15.16:0/1030228185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa0c09e930 0x7ffa0c11a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:20.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.291+0000 7ffa10e3f700 1 --2- 172.21.15.16:0/1030228185 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa0c09f410 0x7ffa0c11abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:20.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.291+0000 7ffa10e3f700 1 -- 172.21.15.16:0/1030228185 >> 172.21.15.16:0/1030228185 conn(0x7ffa0c0a4210 msgr2=0x7ffa0c0a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:20.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.291+0000 7ffa10e3f700 1 -- 172.21.15.16:0/1030228185 shutdown_connections 2023-09-30T22:30:20.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:20.291+0000 7ffa10e3f700 1 -- 172.21.15.16:0/1030228185 wait complete. 2023-09-30T22:30:20.306 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:30:21.171 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:21.216 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:21.254 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:21.308 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:21.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.326+0000 7f7e5d56a700 1 Processor -- start 2023-09-30T22:30:21.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.326+0000 7f7e5d56a700 1 -- start start 2023-09-30T22:30:21.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.327+0000 7f7e5d56a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7e5809d6f0 0x7f7e5809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:21.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.327+0000 7f7e5d56a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7e5809e410 0x7f7e58130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:21.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.327+0000 7f7e5d56a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e5809f130 con 0x7f7e5809e000 2023-09-30T22:30:21.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.327+0000 7f7e5d56a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5809f520 con 0x7f7e5809d6f0 2023-09-30T22:30:21.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.327+0000 7f7e5d56a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5809e930 con 0x7f7e5809e410 2023-09-30T22:30:21.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.327+0000 7f7e56ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7e5809d6f0 0x7f7e5809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:21.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.327+0000 7f7e56ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7e5809d6f0 0x7f7e5809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:34262/0 (socket says 172.21.15.16:34262) 2023-09-30T22:30:21.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.327+0000 7f7e56ffd700 1 -- 172.21.15.16:0/3703042377 learned_addr learned my addr 172.21.15.16:0/3703042377 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:21.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.327+0000 7f7e577fe700 1 --2- 172.21.15.16:0/3703042377 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7e5809e410 0x7f7e58130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:21.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.327+0000 7f7e55ffb700 1 -- 172.21.15.16:0/3703042377 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2521313303 0 0) 0x7f7e5809f130 con 0x7f7e5809e000 2023-09-30T22:30:21.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.327+0000 7f7e55ffb700 1 -- 172.21.15.16:0/3703042377 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e400035b0 con 0x7f7e5809e000 2023-09-30T22:30:21.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.327+0000 7f7e56ffd700 1 -- 172.21.15.16:0/3703042377 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7e5809e410 msgr2=0x7f7e58130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:21.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.327+0000 7f7e56ffd700 1 --2- 172.21.15.16:0/3703042377 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7e5809e410 0x7f7e58130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.327+0000 7f7e56ffd700 1 -- 172.21.15.16:0/3703042377 >> v1:172.21.15.16:6789/0 conn(0x7f7e5809e000 legacy=0x7f7e58130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:21.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.327+0000 7f7e56ffd700 1 -- 172.21.15.16:0/3703042377 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e5809b2d0 con 0x7f7e5809d6f0 2023-09-30T22:30:21.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.328+0000 7f7e56ffd700 1 --2- 172.21.15.16:0/3703042377 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7e5809d6f0 0x7f7e5809dac0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7e48009920 tx=0x7f7e48009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd58d0e94e97431f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:21.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.328+0000 7f7e55ffb700 1 -- 172.21.15.16:0/3703042377 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e48011830 con 0x7f7e5809d6f0 2023-09-30T22:30:21.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.328+0000 7f7e55ffb700 1 -- 172.21.15.16:0/3703042377 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e48011990 con 0x7f7e5809d6f0 2023-09-30T22:30:21.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.328+0000 7f7e55ffb700 1 -- 172.21.15.16:0/3703042377 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e48011d20 con 0x7f7e5809d6f0 2023-09-30T22:30:21.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.329+0000 7f7e5d56a700 1 -- 172.21.15.16:0/3703042377 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7e5809d6f0 msgr2=0x7f7e5809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:21.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.329+0000 7f7e5d56a700 1 --2- 172.21.15.16:0/3703042377 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7e5809d6f0 0x7f7e5809dac0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7e48009920 tx=0x7f7e48009c30 comp rx=0 tx=0).stop 2023-09-30T22:30:21.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.329+0000 7f7e5d56a700 1 -- 172.21.15.16:0/3703042377 shutdown_connections 2023-09-30T22:30:21.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.329+0000 7f7e5d56a700 1 --2- 172.21.15.16:0/3703042377 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7e5809e410 0x7f7e58130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.329+0000 7f7e5d56a700 1 --2- 172.21.15.16:0/3703042377 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7e5809d6f0 0x7f7e5809dac0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.329+0000 7f7e5d56a700 1 -- 172.21.15.16:0/3703042377 >> 172.21.15.16:0/3703042377 conn(0x7f7e58066f50 msgr2=0x7f7e5809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:21.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.329+0000 7f7e5d56a700 1 -- 172.21.15.16:0/3703042377 shutdown_connections 2023-09-30T22:30:21.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.329+0000 7f7e5d56a700 1 -- 172.21.15.16:0/3703042377 wait complete. 2023-09-30T22:30:21.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.331+0000 7f7e5d56a700 1 Processor -- start 2023-09-30T22:30:21.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.331+0000 7f7e5d56a700 1 -- start start 2023-09-30T22:30:21.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.331+0000 7f7e5d56a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e5809d6f0 0x7f7e58136a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:21.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.331+0000 7f7e5d56a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7e5809e000 0x7f7e58136fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:21.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.331+0000 7f7e5d56a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7e5809e410 0x7f7e581374f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:21.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.331+0000 7f7e5d56a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5813a010 con 0x7f7e5809d6f0 2023-09-30T22:30:21.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.331+0000 7f7e5d56a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e58138e00 con 0x7f7e5809e000 2023-09-30T22:30:21.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.331+0000 7f7e5d56a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5809c700 con 0x7f7e5809e410 2023-09-30T22:30:21.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.332+0000 7f7e56ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e5809d6f0 0x7f7e58136a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:21.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.332+0000 7f7e577fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7e5809e410 0x7f7e581374f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:21.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.332+0000 7f7e56ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e5809d6f0 0x7f7e58136a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57996/0 (socket says 172.21.15.16:57996) 2023-09-30T22:30:21.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.332+0000 7f7e56ffd700 1 -- 172.21.15.16:0/2808227502 learned_addr learned my addr 172.21.15.16:0/2808227502 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:21.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.332+0000 7f7e577fe700 1 --2- 172.21.15.16:0/2808227502 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7e5809e410 0x7f7e581374f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:58874/0 (socket says 172.21.15.16:58874) 2023-09-30T22:30:21.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.332+0000 7f7e567fc700 1 --2- 172.21.15.16:0/2808227502 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7e5809e000 0x7f7e58136fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:21.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.332+0000 7f7e577fe700 1 -- 172.21.15.16:0/2808227502 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7e5809e000 msgr2=0x7f7e58136fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:21.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.332+0000 7f7e577fe700 1 --2- 172.21.15.16:0/2808227502 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7e5809e000 0x7f7e58136fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.332+0000 7f7e577fe700 1 -- 172.21.15.16:0/2808227502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e5809d6f0 msgr2=0x7f7e58136a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:21.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.332+0000 7f7e577fe700 1 --2- 172.21.15.16:0/2808227502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e5809d6f0 0x7f7e58136a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.332+0000 7f7e577fe700 1 -- 172.21.15.16:0/2808227502 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e48009550 con 0x7f7e5809e410 2023-09-30T22:30:21.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.332+0000 7f7e56ffd700 1 --2- 172.21.15.16:0/2808227502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e5809d6f0 0x7f7e58136a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:21.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.332+0000 7f7e577fe700 1 --2- 172.21.15.16:0/2808227502 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7e5809e410 0x7f7e581374f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7e5000efc0 tx=0x7f7e5000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:21.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.333+0000 7f7e3f7fe700 1 -- 172.21.15.16:0/2808227502 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5000cc90 con 0x7f7e5809e410 2023-09-30T22:30:21.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.333+0000 7f7e3f7fe700 1 -- 172.21.15.16:0/2808227502 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e5000f040 con 0x7f7e5809e410 2023-09-30T22:30:21.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.333+0000 7f7e5d56a700 1 -- 172.21.15.16:0/2808227502 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e581385e0 con 0x7f7e5809e410 2023-09-30T22:30:21.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.333+0000 7f7e5d56a700 1 -- 172.21.15.16:0/2808227502 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e58139250 con 0x7f7e5809e410 2023-09-30T22:30:21.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.333+0000 7f7e3f7fe700 1 -- 172.21.15.16:0/2808227502 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e500146c0 con 0x7f7e5809e410 2023-09-30T22:30:21.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.334+0000 7f7e3f7fe700 1 -- 172.21.15.16:0/2808227502 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7e50014820 con 0x7f7e5809e410 2023-09-30T22:30:21.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.334+0000 7f7e3f7fe700 1 -- 172.21.15.16:0/2808227502 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f7e500401d0 con 0x7f7e5809e410 2023-09-30T22:30:21.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.334+0000 7f7e5d56a700 1 -- 172.21.15.16:0/2808227502 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e5813a450 con 0x7f7e5809e410 2023-09-30T22:30:21.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.339+0000 7f7e3f7fe700 1 -- 172.21.15.16:0/2808227502 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7e50018020 con 0x7f7e5809e410 2023-09-30T22:30:21.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.375+0000 7f5a76cd5700 1 Processor -- start 2023-09-30T22:30:21.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.375+0000 7f5a76cd5700 1 -- start start 2023-09-30T22:30:21.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.376+0000 7f5a76cd5700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5a701316c0 0x7f5a70131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:21.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.376+0000 7f5a76cd5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a7009d9f0 0x7f5a7006a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:21.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.376+0000 7f5a76cd5700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a7009e130 con 0x7f5a7006acb0 2023-09-30T22:30:21.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.376+0000 7f5a76cd5700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a701312b0 con 0x7f5a701316c0 2023-09-30T22:30:21.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.376+0000 7f5a76cd5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7009f6e0 con 0x7f5a7009d9f0 2023-09-30T22:30:21.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.376+0000 7f5a74a71700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5a701316c0 0x7f5a70131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:21.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.376+0000 7f5a74a71700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5a701316c0 0x7f5a70131a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58002/0 (socket says 172.21.15.16:58002) 2023-09-30T22:30:21.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.376+0000 7f5a74a71700 1 -- 172.21.15.16:0/3793476241 learned_addr learned my addr 172.21.15.16:0/3793476241 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:21.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.376+0000 7f5a6ffff700 1 --2- 172.21.15.16:0/3793476241 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a7009d9f0 0x7f5a7006a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:21.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.377+0000 7f5a74a71700 1 -- 172.21.15.16:0/3793476241 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a7009d9f0 msgr2=0x7f5a7006a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:21.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.377+0000 7f5a74a71700 1 --2- 172.21.15.16:0/3793476241 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a7009d9f0 0x7f5a7006a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.377+0000 7f5a74a71700 1 -- 172.21.15.16:0/3793476241 >> v1:172.21.15.16:6789/0 conn(0x7f5a7006acb0 legacy=0x7f5a70138540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:21.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.377+0000 7f5a74a71700 1 -- 172.21.15.16:0/3793476241 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a7009e760 con 0x7f5a701316c0 2023-09-30T22:30:21.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.377+0000 7f5a6f7fe700 1 -- 172.21.15.16:0/3793476241 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1550860923 0 0) 0x7f5a68001490 con 0x7f5a7006acb0 2023-09-30T22:30:21.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.377+0000 7f5a74a71700 1 --2- 172.21.15.16:0/3793476241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a701316c0 0x7f5a70131a90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5a6400de50 tx=0x7f5a6400f330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c19f63c988264bd8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:21.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.377+0000 7f5a6f7fe700 1 -- 172.21.15.16:0/3793476241 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a64012070 con 0x7f5a701316c0 2023-09-30T22:30:21.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.377+0000 7f5a6f7fe700 1 -- 172.21.15.16:0/3793476241 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a6400f910 con 0x7f5a701316c0 2023-09-30T22:30:21.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.377+0000 7f5a6f7fe700 1 -- 172.21.15.16:0/3793476241 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6400fcc0 con 0x7f5a701316c0 2023-09-30T22:30:21.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.378+0000 7f5a76cd5700 1 -- 172.21.15.16:0/3793476241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a701316c0 msgr2=0x7f5a70131a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:21.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.378+0000 7f5a76cd5700 1 --2- 172.21.15.16:0/3793476241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a701316c0 0x7f5a70131a90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5a6400de50 tx=0x7f5a6400f330 comp rx=0 tx=0).stop 2023-09-30T22:30:21.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.378+0000 7f5a76cd5700 1 -- 172.21.15.16:0/3793476241 shutdown_connections 2023-09-30T22:30:21.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.378+0000 7f5a76cd5700 1 --2- 172.21.15.16:0/3793476241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a701316c0 0x7f5a70131a90 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.378+0000 7f5a76cd5700 1 --2- 172.21.15.16:0/3793476241 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a7009d9f0 0x7f5a7006a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.378+0000 7f5a76cd5700 1 -- 172.21.15.16:0/3793476241 >> 172.21.15.16:0/3793476241 conn(0x7f5a70066f50 msgr2=0x7f5a7009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:21.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.378+0000 7f5a76cd5700 1 -- 172.21.15.16:0/3793476241 shutdown_connections 2023-09-30T22:30:21.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.378+0000 7f5a76cd5700 1 -- 172.21.15.16:0/3793476241 wait complete. 2023-09-30T22:30:21.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.379+0000 7f5a76cd5700 1 Processor -- start 2023-09-30T22:30:21.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.379+0000 7f5a76cd5700 1 -- start start 2023-09-30T22:30:21.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.380+0000 7f5a76cd5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a7006acb0 0x7f5a70119480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:21.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.380+0000 7f5a76cd5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5a7009d9f0 0x7f5a701199c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:21.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.380+0000 7f5a76cd5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a701316c0 0x7f5a70119f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:21.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.380+0000 7f5a76cd5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7011a740 con 0x7f5a7006acb0 2023-09-30T22:30:21.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.380+0000 7f5a76cd5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7011c550 con 0x7f5a7009d9f0 2023-09-30T22:30:21.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.380+0000 7f5a76cd5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7011bb70 con 0x7f5a701316c0 2023-09-30T22:30:21.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.380+0000 7f5a74a71700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a7006acb0 0x7f5a70119480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:21.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.380+0000 7f5a75272700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a701316c0 0x7f5a70119f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:21.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.380+0000 7f5a74a71700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a7006acb0 0x7f5a70119480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58008/0 (socket says 172.21.15.16:58008) 2023-09-30T22:30:21.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.380+0000 7f5a74a71700 1 -- 172.21.15.16:0/17931429 learned_addr learned my addr 172.21.15.16:0/17931429 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:21.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.381+0000 7f5a74a71700 1 -- 172.21.15.16:0/17931429 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a701316c0 msgr2=0x7f5a70119f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:21.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.381+0000 7f5a74a71700 1 --2- 172.21.15.16:0/17931429 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a701316c0 0x7f5a70119f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.381+0000 7f5a74a71700 1 -- 172.21.15.16:0/17931429 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5a7009d9f0 msgr2=0x7f5a701199c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:21.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.381+0000 7f5a74a71700 1 --2- 172.21.15.16:0/17931429 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5a7009d9f0 0x7f5a701199c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.381+0000 7f5a74a71700 1 -- 172.21.15.16:0/17931429 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a6400db30 con 0x7f5a7006acb0 2023-09-30T22:30:21.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.381+0000 7f5a74a71700 1 --2- 172.21.15.16:0/17931429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a7006acb0 0x7f5a70119480 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5a6400f680 tx=0x7f5a64004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:21.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.381+0000 7f5a6d7fa700 1 -- 172.21.15.16:0/17931429 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a64003ab0 con 0x7f5a7006acb0 2023-09-30T22:30:21.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.381+0000 7f5a6d7fa700 1 -- 172.21.15.16:0/17931429 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a64003c10 con 0x7f5a7006acb0 2023-09-30T22:30:21.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.381+0000 7f5a76cd5700 1 -- 172.21.15.16:0/17931429 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a7009ead0 con 0x7f5a7006acb0 2023-09-30T22:30:21.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.381+0000 7f5a6d7fa700 1 -- 172.21.15.16:0/17931429 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a64020440 con 0x7f5a7006acb0 2023-09-30T22:30:21.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.381+0000 7f5a76cd5700 1 -- 172.21.15.16:0/17931429 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a7012b900 con 0x7f5a7006acb0 2023-09-30T22:30:21.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.382+0000 7f5a6d7fa700 1 -- 172.21.15.16:0/17931429 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5a640205a0 con 0x7f5a7006acb0 2023-09-30T22:30:21.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.383+0000 7f5a6d7fa700 1 -- 172.21.15.16:0/17931429 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f5a64045040 con 0x7f5a7006acb0 2023-09-30T22:30:21.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.384+0000 7f5a76cd5700 1 -- 172.21.15.16:0/17931429 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a7011b6f0 con 0x7f5a7006acb0 2023-09-30T22:30:21.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.392+0000 7f5a6d7fa700 1 -- 172.21.15.16:0/17931429 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5a7011b6f0 con 0x7f5a7006acb0 2023-09-30T22:30:21.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.413+0000 7fd07860a700 1 Processor -- start 2023-09-30T22:30:21.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.413+0000 7fd07860a700 1 -- start start 2023-09-30T22:30:21.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.414+0000 7fd07860a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd07013ac00 0x7fd07013cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:21.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.414+0000 7fd07860a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd07013d520 0x7fd07013f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:21.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.414+0000 7fd07860a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd07009e800 con 0x7fd070095690 2023-09-30T22:30:21.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.414+0000 7fd07860a700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd070095a60 con 0x7fd07013d520 2023-09-30T22:30:21.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.414+0000 7fd07860a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd070094bd0 con 0x7fd07013ac00 2023-09-30T22:30:21.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.414+0000 7fd0763a6700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd070095690 0x7fd070134cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56970/0 (socket says 172.21.15.16:56970) 2023-09-30T22:30:21.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.414+0000 7fd0763a6700 1 -- 172.21.15.16:0/22574116 learned_addr learned my addr 172.21.15.16:0/22574116 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:30:21.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.414+0000 7fd075ba5700 1 --2- 172.21.15.16:0/22574116 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd07013ac00 0x7fd07013cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:21.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.414+0000 7fd076ba7700 1 --2- 172.21.15.16:0/22574116 >> v2:172.21.15.16:3300/0 conn(0x7fd07013d520 0x7fd07013f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:21.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.414+0000 7fd0753a4700 1 -- 172.21.15.16:0/22574116 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 793281483 0 0) 0x7fd07009e800 con 0x7fd070095690 2023-09-30T22:30:21.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.414+0000 7fd0753a4700 1 -- 172.21.15.16:0/22574116 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd05c0035b0 con 0x7fd070095690 2023-09-30T22:30:21.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.415+0000 7fd075ba5700 1 -- 172.21.15.16:0/22574116 >> v2:172.21.15.16:3300/0 conn(0x7fd07013d520 msgr2=0x7fd07013f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:21.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.415+0000 7fd075ba5700 1 --2- 172.21.15.16:0/22574116 >> v2:172.21.15.16:3300/0 conn(0x7fd07013d520 0x7fd07013f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.415+0000 7fd075ba5700 1 -- 172.21.15.16:0/22574116 >> v1:172.21.15.16:6789/0 conn(0x7fd070095690 legacy=0x7fd070134cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:21.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.415+0000 7fd075ba5700 1 -- 172.21.15.16:0/22574116 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0700940e0 con 0x7fd07013ac00 2023-09-30T22:30:21.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.415+0000 7fd076ba7700 1 --2- 172.21.15.16:0/22574116 >> v2:172.21.15.16:3300/0 conn(0x7fd07013d520 0x7fd07013f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:21.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.415+0000 7fd075ba5700 1 --2- 172.21.15.16:0/22574116 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd07013ac00 0x7fd07013cfe0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd060009870 tx=0x7fd060009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef8d17d97200f93a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:21.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.415+0000 7fd0753a4700 1 -- 172.21.15.16:0/22574116 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd060011830 con 0x7fd07013ac00 2023-09-30T22:30:21.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.415+0000 7fd0753a4700 1 -- 172.21.15.16:0/22574116 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd060011990 con 0x7fd07013ac00 2023-09-30T22:30:21.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.415+0000 7fd0753a4700 1 -- 172.21.15.16:0/22574116 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd060011d20 con 0x7fd07013ac00 2023-09-30T22:30:21.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.416+0000 7fd07860a700 1 -- 172.21.15.16:0/22574116 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd07013ac00 msgr2=0x7fd07013cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:21.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.416+0000 7fd07860a700 1 --2- 172.21.15.16:0/22574116 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd07013ac00 0x7fd07013cfe0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd060009870 tx=0x7fd060009b80 comp rx=0 tx=0).stop 2023-09-30T22:30:21.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.416+0000 7fd07860a700 1 -- 172.21.15.16:0/22574116 shutdown_connections 2023-09-30T22:30:21.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.416+0000 7fd07860a700 1 --2- 172.21.15.16:0/22574116 >> v2:172.21.15.16:3300/0 conn(0x7fd07013d520 0x7fd07013f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.416+0000 7fd07860a700 1 --2- 172.21.15.16:0/22574116 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd07013ac00 0x7fd07013cfe0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.416+0000 7fd07860a700 1 -- 172.21.15.16:0/22574116 >> 172.21.15.16:0/22574116 conn(0x7fd0700a2360 msgr2=0x7fd0700a2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:21.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.416+0000 7fd07860a700 1 -- 172.21.15.16:0/22574116 shutdown_connections 2023-09-30T22:30:21.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.416+0000 7fd07860a700 1 -- 172.21.15.16:0/22574116 wait complete. 2023-09-30T22:30:21.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.417+0000 7fd07860a700 1 Processor -- start 2023-09-30T22:30:21.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.417+0000 7fd07860a700 1 -- start start 2023-09-30T22:30:21.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.418+0000 7fd07860a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd070095690 0x7fd0700968e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:21.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.418+0000 7fd07860a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd07013ac00 0x7fd070096e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:21.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.418+0000 7fd07860a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd07013d520 0x7fd0701cc780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:21.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.418+0000 7fd07860a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd070098940 con 0x7fd07013d520 2023-09-30T22:30:21.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.418+0000 7fd07860a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0700946d0 con 0x7fd070095690 2023-09-30T22:30:21.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.418+0000 7fd07860a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd070096400 con 0x7fd07013ac00 2023-09-30T22:30:21.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.418+0000 7fd0763a6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd070095690 0x7fd0700968e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:21.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.418+0000 7fd0763a6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd070095690 0x7fd0700968e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:34288/0 (socket says 172.21.15.16:34288) 2023-09-30T22:30:21.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.418+0000 7fd0763a6700 1 -- 172.21.15.16:0/2060726447 learned_addr learned my addr 172.21.15.16:0/2060726447 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:21.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.418+0000 7fd076ba7700 1 --2- 172.21.15.16:0/2060726447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd07013d520 0x7fd0701cc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:21.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.418+0000 7fd075ba5700 1 --2- 172.21.15.16:0/2060726447 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd07013ac00 0x7fd070096e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:21.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.419+0000 7fd076ba7700 1 -- 172.21.15.16:0/2060726447 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd07013ac00 msgr2=0x7fd070096e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:21.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.419+0000 7fd076ba7700 1 --2- 172.21.15.16:0/2060726447 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd07013ac00 0x7fd070096e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.419+0000 7fd076ba7700 1 -- 172.21.15.16:0/2060726447 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd070095690 msgr2=0x7fd0700968e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:21.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.419+0000 7fd076ba7700 1 --2- 172.21.15.16:0/2060726447 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd070095690 0x7fd0700968e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.419+0000 7fd076ba7700 1 -- 172.21.15.16:0/2060726447 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd068009a70 con 0x7fd07013d520 2023-09-30T22:30:21.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.419+0000 7fd0763a6700 1 --2- 172.21.15.16:0/2060726447 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd070095690 0x7fd0700968e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:21.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.419+0000 7fd076ba7700 1 --2- 172.21.15.16:0/2060726447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd07013d520 0x7fd0701cc780 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd070074230 tx=0x7fd06800ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:21.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.419+0000 7fd066ffd700 1 -- 172.21.15.16:0/2060726447 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06800cbf0 con 0x7fd07013d520 2023-09-30T22:30:21.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.419+0000 7fd066ffd700 1 -- 172.21.15.16:0/2060726447 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd06800f040 con 0x7fd07013d520 2023-09-30T22:30:21.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.419+0000 7fd066ffd700 1 -- 172.21.15.16:0/2060726447 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd068014480 con 0x7fd07013d520 2023-09-30T22:30:21.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.419+0000 7fd07860a700 1 -- 172.21.15.16:0/2060726447 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0600094a0 con 0x7fd07013d520 2023-09-30T22:30:21.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.419+0000 7fd07860a700 1 -- 172.21.15.16:0/2060726447 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd070098ee0 con 0x7fd07013d520 2023-09-30T22:30:21.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.421+0000 7fd07860a700 1 -- 172.21.15.16:0/2060726447 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd070074ea0 con 0x7fd07013d520 2023-09-30T22:30:21.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.421+0000 7fd066ffd700 1 -- 172.21.15.16:0/2060726447 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd0680145e0 con 0x7fd07013d520 2023-09-30T22:30:21.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.422+0000 7fd066ffd700 1 -- 172.21.15.16:0/2060726447 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd068040700 con 0x7fd07013d520 2023-09-30T22:30:21.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.426+0000 7fd066ffd700 1 -- 172.21.15.16:0/2060726447 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd068012930 con 0x7fd07013d520 2023-09-30T22:30:21.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.469+0000 7fe5f9baf700 1 Processor -- start 2023-09-30T22:30:21.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.469+0000 7fe5f9baf700 1 -- start start 2023-09-30T22:30:21.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.470+0000 7fe5f9baf700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe5f409d6f0 0x7fe5f409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:21.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.470+0000 7fe5f9baf700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe5f409e410 0x7fe5f4130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:21.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.470+0000 7fe5f9baf700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5f409f130 con 0x7fe5f409e000 2023-09-30T22:30:21.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.470+0000 7fe5f9baf700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5f409f520 con 0x7fe5f409d6f0 2023-09-30T22:30:21.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.470+0000 7fe5f9baf700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f409e930 con 0x7fe5f409e410 2023-09-30T22:30:21.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.470+0000 7fe5f37fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe5f409d6f0 0x7fe5f409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:21.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.470+0000 7fe5f3fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe5f409e410 0x7fe5f4130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:21.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.470+0000 7fe5f2ffd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe5f409e000 0x7fe5f4130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56982/0 (socket says 172.21.15.16:56982) 2023-09-30T22:30:21.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.470+0000 7fe5f37fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe5f409d6f0 0x7fe5f409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58018/0 (socket says 172.21.15.16:58018) 2023-09-30T22:30:21.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.470+0000 7fe5f2ffd700 1 -- 172.21.15.16:0/339039069 learned_addr learned my addr 172.21.15.16:0/339039069 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:30:21.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.470+0000 7fe5f3fff700 1 -- 172.21.15.16:0/339039069 >> v2:172.21.15.16:3300/0 conn(0x7fe5f409d6f0 msgr2=0x7fe5f409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:21.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.470+0000 7fe5f3fff700 1 --2- 172.21.15.16:0/339039069 >> v2:172.21.15.16:3300/0 conn(0x7fe5f409d6f0 0x7fe5f409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.470+0000 7fe5f3fff700 1 -- 172.21.15.16:0/339039069 >> v1:172.21.15.16:6789/0 conn(0x7fe5f409e000 legacy=0x7fe5f4130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:21.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.470+0000 7fe5f3fff700 1 -- 172.21.15.16:0/339039069 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f409b2d0 con 0x7fe5f409e410 2023-09-30T22:30:21.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.471+0000 7fe5f3fff700 1 --2- 172.21.15.16:0/339039069 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe5f409e410 0x7fe5f4130d70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec009d90 tx=0x7fe5ec00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bda618a2d0a1db3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:21.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.471+0000 7fe5f27fc700 1 -- 172.21.15.16:0/339039069 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec00bd20 con 0x7fe5f409e410 2023-09-30T22:30:21.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.471+0000 7fe5f27fc700 1 -- 172.21.15.16:0/339039069 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5ec00be80 con 0x7fe5f409e410 2023-09-30T22:30:21.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.471+0000 7fe5f27fc700 1 -- 172.21.15.16:0/339039069 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec0116a0 con 0x7fe5f409e410 2023-09-30T22:30:21.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.472+0000 7fe5f9baf700 1 -- 172.21.15.16:0/339039069 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe5f409e410 msgr2=0x7fe5f4130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:21.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.472+0000 7fe5f9baf700 1 --2- 172.21.15.16:0/339039069 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe5f409e410 0x7fe5f4130d70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec009d90 tx=0x7fe5ec00b330 comp rx=0 tx=0).stop 2023-09-30T22:30:21.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.472+0000 7fe5f9baf700 1 -- 172.21.15.16:0/339039069 shutdown_connections 2023-09-30T22:30:21.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.472+0000 7fe5f9baf700 1 --2- 172.21.15.16:0/339039069 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe5f409e410 0x7fe5f4130d70 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.472+0000 7fe5f9baf700 1 --2- 172.21.15.16:0/339039069 >> v2:172.21.15.16:3300/0 conn(0x7fe5f409d6f0 0x7fe5f409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.472+0000 7fe5f9baf700 1 -- 172.21.15.16:0/339039069 >> 172.21.15.16:0/339039069 conn(0x7fe5f4066f50 msgr2=0x7fe5f409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:21.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.472+0000 7fe5f9baf700 1 -- 172.21.15.16:0/339039069 shutdown_connections 2023-09-30T22:30:21.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.472+0000 7fe5f9baf700 1 -- 172.21.15.16:0/339039069 wait complete. 2023-09-30T22:30:21.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.473+0000 7fe5f9baf700 1 Processor -- start 2023-09-30T22:30:21.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.474+0000 7fe5f9baf700 1 -- start start 2023-09-30T22:30:21.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.474+0000 7fe5f9baf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5f409d6f0 0x7fe5f411e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:21.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.474+0000 7fe5f9baf700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe5f409e000 0x7fe5f411e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:21.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.474+0000 7fe5f9baf700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe5f409e410 0x7fe5f411ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:21.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.474+0000 7fe5f9baf700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f4121490 con 0x7fe5f409d6f0 2023-09-30T22:30:21.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.474+0000 7fe5f9baf700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f409c700 con 0x7fe5f409e000 2023-09-30T22:30:21.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.474+0000 7fe5f9baf700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f40a1e20 con 0x7fe5f409e410 2023-09-30T22:30:21.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.475+0000 7fe5f37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5f409d6f0 0x7fe5f411e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:21.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.475+0000 7fe5f37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5f409d6f0 0x7fe5f411e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58022/0 (socket says 172.21.15.16:58022) 2023-09-30T22:30:21.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.475+0000 7fe5f37fe700 1 -- 172.21.15.16:0/457078537 learned_addr learned my addr 172.21.15.16:0/457078537 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:21.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.475+0000 7fe5f3fff700 1 --2- 172.21.15.16:0/457078537 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe5f409e410 0x7fe5f411ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:21.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.475+0000 7fe5f2ffd700 1 --2- 172.21.15.16:0/457078537 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe5f409e000 0x7fe5f411e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:21.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.475+0000 7fe5f37fe700 1 -- 172.21.15.16:0/457078537 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe5f409e410 msgr2=0x7fe5f411ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:21.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.475+0000 7fe5f37fe700 1 --2- 172.21.15.16:0/457078537 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe5f409e410 0x7fe5f411ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.475+0000 7fe5f37fe700 1 -- 172.21.15.16:0/457078537 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe5f409e000 msgr2=0x7fe5f411e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:21.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.475+0000 7fe5f37fe700 1 --2- 172.21.15.16:0/457078537 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe5f409e000 0x7fe5f411e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.475+0000 7fe5f37fe700 1 -- 172.21.15.16:0/457078537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5ec009a70 con 0x7fe5f409d6f0 2023-09-30T22:30:21.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.475+0000 7fe5f3fff700 1 --2- 172.21.15.16:0/457078537 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe5f409e410 0x7fe5f411ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:21.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.475+0000 7fe5f37fe700 1 --2- 172.21.15.16:0/457078537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5f409d6f0 0x7fe5f411e140 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe5e400e870 tx=0x7fe5e400eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:21.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.475+0000 7fe5dbfff700 1 -- 172.21.15.16:0/457078537 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e4009590 con 0x7fe5f409d6f0 2023-09-30T22:30:21.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.476+0000 7fe5dbfff700 1 -- 172.21.15.16:0/457078537 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5e4013680 con 0x7fe5f409d6f0 2023-09-30T22:30:21.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.476+0000 7fe5dbfff700 1 -- 172.21.15.16:0/457078537 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e40139f0 con 0x7fe5f409d6f0 2023-09-30T22:30:21.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.477+0000 7fe5f9baf700 1 -- 172.21.15.16:0/457078537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5f411fd50 con 0x7fe5f409d6f0 2023-09-30T22:30:21.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.477+0000 7fe5f9baf700 1 -- 172.21.15.16:0/457078537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5f41216c0 con 0x7fe5f409d6f0 2023-09-30T22:30:21.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.478+0000 7fe5f9baf700 1 -- 172.21.15.16:0/457078537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5f4074ea0 con 0x7fe5f409d6f0 2023-09-30T22:30:21.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.478+0000 7fe5dbfff700 1 -- 172.21.15.16:0/457078537 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe5e4013b50 con 0x7fe5f409d6f0 2023-09-30T22:30:21.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.478+0000 7fe5dbfff700 1 -- 172.21.15.16:0/457078537 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fe5e403faa0 con 0x7fe5f409d6f0 2023-09-30T22:30:21.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.487+0000 7fe5dbfff700 1 -- 172.21.15.16:0/457078537 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe5e400c970 con 0x7fe5f409d6f0 2023-09-30T22:30:21.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.572+0000 7f7e5d56a700 1 -- 172.21.15.16:0/2808227502 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7e58074ea0 con 0x7f7e5809e410 2023-09-30T22:30:21.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.573+0000 7f7e3f7fe700 1 -- 172.21.15.16:0/2808227502 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7e5002b090 con 0x7f7e5809e410 2023-09-30T22:30:21.573 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:30:21.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.575+0000 7f7e5d56a700 1 -- 172.21.15.16:0/2808227502 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7e5809e410 msgr2=0x7f7e581374f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:21.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.575+0000 7f7e5d56a700 1 --2- 172.21.15.16:0/2808227502 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7e5809e410 0x7f7e581374f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7e5000efc0 tx=0x7f7e5000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.575+0000 7f7e5d56a700 1 -- 172.21.15.16:0/2808227502 shutdown_connections 2023-09-30T22:30:21.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.575+0000 7f7e5d56a700 1 --2- 172.21.15.16:0/2808227502 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7e5809e410 0x7f7e581374f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.575+0000 7f7e5d56a700 1 --2- 172.21.15.16:0/2808227502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e5809d6f0 0x7f7e58136a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.575+0000 7f7e5d56a700 1 --2- 172.21.15.16:0/2808227502 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7e5809e000 0x7f7e58136fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.575+0000 7f7e5d56a700 1 -- 172.21.15.16:0/2808227502 >> 172.21.15.16:0/2808227502 conn(0x7f7e58066f50 msgr2=0x7f7e58070f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:21.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.576+0000 7f7e5d56a700 1 -- 172.21.15.16:0/2808227502 shutdown_connections 2023-09-30T22:30:21.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.576+0000 7f7e5d56a700 1 -- 172.21.15.16:0/2808227502 wait complete. 2023-09-30T22:30:21.590 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:30:21.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.620+0000 7f5a76cd5700 1 -- 172.21.15.16:0/17931429 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f5a7011b6f0 con 0x7f5a7006acb0 2023-09-30T22:30:21.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.620+0000 7f5a6d7fa700 1 -- 172.21.15.16:0/17931429 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5a7011b6f0 con 0x7f5a7006acb0 2023-09-30T22:30:21.621 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:30:21.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.623+0000 7f5a76cd5700 1 -- 172.21.15.16:0/17931429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a7006acb0 msgr2=0x7f5a70119480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:21.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.623+0000 7f5a76cd5700 1 --2- 172.21.15.16:0/17931429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a7006acb0 0x7f5a70119480 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5a6400f680 tx=0x7f5a64004750 comp rx=0 tx=0).stop 2023-09-30T22:30:21.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.623+0000 7f5a76cd5700 1 -- 172.21.15.16:0/17931429 shutdown_connections 2023-09-30T22:30:21.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.623+0000 7f5a76cd5700 1 --2- 172.21.15.16:0/17931429 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a701316c0 0x7f5a70119f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.623+0000 7f5a76cd5700 1 --2- 172.21.15.16:0/17931429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a7006acb0 0x7f5a70119480 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.623+0000 7f5a76cd5700 1 --2- 172.21.15.16:0/17931429 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5a7009d9f0 0x7f5a701199c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.623+0000 7f5a76cd5700 1 -- 172.21.15.16:0/17931429 >> 172.21.15.16:0/17931429 conn(0x7f5a70066f50 msgr2=0x7f5a700a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:21.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.623+0000 7f5a76cd5700 1 -- 172.21.15.16:0/17931429 shutdown_connections 2023-09-30T22:30:21.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.623+0000 7f5a76cd5700 1 -- 172.21.15.16:0/17931429 wait complete. 2023-09-30T22:30:21.639 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:30:21.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.643+0000 7fd07860a700 1 -- 172.21.15.16:0/2060726447 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd07006b700 con 0x7fd07013d520 2023-09-30T22:30:21.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.644+0000 7fd066ffd700 1 -- 172.21.15.16:0/2060726447 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd06802a070 con 0x7fd07013d520 2023-09-30T22:30:21.644 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:30:21.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.646+0000 7fd07860a700 1 -- 172.21.15.16:0/2060726447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd07013d520 msgr2=0x7fd0701cc780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:21.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.646+0000 7fd07860a700 1 --2- 172.21.15.16:0/2060726447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd07013d520 0x7fd0701cc780 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd070074230 tx=0x7fd06800ef90 comp rx=0 tx=0).stop 2023-09-30T22:30:21.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.646+0000 7fd07860a700 1 -- 172.21.15.16:0/2060726447 shutdown_connections 2023-09-30T22:30:21.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.646+0000 7fd07860a700 1 --2- 172.21.15.16:0/2060726447 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd070095690 0x7fd0700968e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.646+0000 7fd07860a700 1 --2- 172.21.15.16:0/2060726447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd07013d520 0x7fd0701cc780 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.646+0000 7fd07860a700 1 --2- 172.21.15.16:0/2060726447 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd07013ac00 0x7fd070096e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.646+0000 7fd07860a700 1 -- 172.21.15.16:0/2060726447 >> 172.21.15.16:0/2060726447 conn(0x7fd0700a2360 msgr2=0x7fd07013afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:21.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.646+0000 7fd07860a700 1 -- 172.21.15.16:0/2060726447 shutdown_connections 2023-09-30T22:30:21.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.646+0000 7fd07860a700 1 -- 172.21.15.16:0/2060726447 wait complete. 2023-09-30T22:30:21.660 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:30:21.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.697+0000 7fe5f9baf700 1 -- 172.21.15.16:0/457078537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe5f4122450 con 0x7fe5f409d6f0 2023-09-30T22:30:21.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.697+0000 7fe5dbfff700 1 -- 172.21.15.16:0/457078537 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe5e4021020 con 0x7fe5f409d6f0 2023-09-30T22:30:21.698 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:30:21.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.700+0000 7fe5f9baf700 1 -- 172.21.15.16:0/457078537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5f409d6f0 msgr2=0x7fe5f411e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:21.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.700+0000 7fe5f9baf700 1 --2- 172.21.15.16:0/457078537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5f409d6f0 0x7fe5f411e140 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe5e400e870 tx=0x7fe5e400eb80 comp rx=0 tx=0).stop 2023-09-30T22:30:21.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.700+0000 7fe5f9baf700 1 -- 172.21.15.16:0/457078537 shutdown_connections 2023-09-30T22:30:21.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.700+0000 7fe5f9baf700 1 --2- 172.21.15.16:0/457078537 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe5f409e410 0x7fe5f411ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.700+0000 7fe5f9baf700 1 --2- 172.21.15.16:0/457078537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5f409d6f0 0x7fe5f411e140 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.700+0000 7fe5f9baf700 1 --2- 172.21.15.16:0/457078537 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe5f409e000 0x7fe5f411e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:21.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.700+0000 7fe5f9baf700 1 -- 172.21.15.16:0/457078537 >> 172.21.15.16:0/457078537 conn(0x7fe5f4066f50 msgr2=0x7fe5f40a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:21.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.700+0000 7fe5f9baf700 1 -- 172.21.15.16:0/457078537 shutdown_connections 2023-09-30T22:30:21.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:21.700+0000 7fe5f9baf700 1 -- 172.21.15.16:0/457078537 wait complete. 2023-09-30T22:30:21.714 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:30:22.593 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:22.640 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:22.661 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:22.715 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:22.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.746+0000 7f03d190e700 1 Processor -- start 2023-09-30T22:30:22.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.746+0000 7f03d190e700 1 -- start start 2023-09-30T22:30:22.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.746+0000 7f03d190e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f03cc09d6f0 0x7f03cc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:22.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.747+0000 7f03d190e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f03cc09e000 0x7f03cc1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:22.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.747+0000 7f03d190e700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03cc09b120 con 0x7f03cc09e410 2023-09-30T22:30:22.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.747+0000 7f03d190e700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03cc06b460 con 0x7f03cc09e000 2023-09-30T22:30:22.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.747+0000 7f03d190e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f03cc09a7e0 con 0x7f03cc09d6f0 2023-09-30T22:30:22.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.747+0000 7f03cb7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f03cc09e410 0x7f03cc133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56992/0 (socket says 172.21.15.16:56992) 2023-09-30T22:30:22.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.747+0000 7f03cb7fe700 1 -- 172.21.15.16:0/102429252 learned_addr learned my addr 172.21.15.16:0/102429252 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:30:22.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.747+0000 7f03caffd700 1 --2- 172.21.15.16:0/102429252 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f03cc09d6f0 0x7f03cc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:22.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.747+0000 7f03ca7fc700 1 --2- 172.21.15.16:0/102429252 >> v2:172.21.15.16:3300/0 conn(0x7f03cc09e000 0x7f03cc1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:22.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.747+0000 7f03c9ffb700 1 -- 172.21.15.16:0/102429252 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2486288115 0 0) 0x7f03cc09b120 con 0x7f03cc09e410 2023-09-30T22:30:22.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.747+0000 7f03c9ffb700 1 -- 172.21.15.16:0/102429252 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03b40035b0 con 0x7f03cc09e410 2023-09-30T22:30:22.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.747+0000 7f03ca7fc700 1 -- 172.21.15.16:0/102429252 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f03cc09d6f0 msgr2=0x7f03cc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:22.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.747+0000 7f03ca7fc700 1 --2- 172.21.15.16:0/102429252 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f03cc09d6f0 0x7f03cc09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:22.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.747+0000 7f03ca7fc700 1 -- 172.21.15.16:0/102429252 >> v1:172.21.15.16:6789/0 conn(0x7f03cc09e410 legacy=0x7f03cc133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:22.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.747+0000 7f03ca7fc700 1 -- 172.21.15.16:0/102429252 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03cc09a500 con 0x7f03cc09e000 2023-09-30T22:30:22.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.748+0000 7f03ca7fc700 1 --2- 172.21.15.16:0/102429252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03cc09e000 0x7f03cc1339f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f03c0009840 tx=0x7f03c0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43808f7bcc96f809 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:22.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.748+0000 7f03c9ffb700 1 -- 172.21.15.16:0/102429252 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c0011830 con 0x7f03cc09e000 2023-09-30T22:30:22.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.748+0000 7f03c9ffb700 1 -- 172.21.15.16:0/102429252 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03c0011990 con 0x7f03cc09e000 2023-09-30T22:30:22.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.748+0000 7f03c9ffb700 1 -- 172.21.15.16:0/102429252 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c0011d00 con 0x7f03cc09e000 2023-09-30T22:30:22.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.749+0000 7f03d190e700 1 -- 172.21.15.16:0/102429252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03cc09e000 msgr2=0x7f03cc1339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:22.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.749+0000 7f03d190e700 1 --2- 172.21.15.16:0/102429252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03cc09e000 0x7f03cc1339f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f03c0009840 tx=0x7f03c0009b50 comp rx=0 tx=0).stop 2023-09-30T22:30:22.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.749+0000 7f03d190e700 1 -- 172.21.15.16:0/102429252 shutdown_connections 2023-09-30T22:30:22.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.749+0000 7f03d190e700 1 --2- 172.21.15.16:0/102429252 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f03cc09d6f0 0x7f03cc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:22.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.749+0000 7f03d190e700 1 --2- 172.21.15.16:0/102429252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03cc09e000 0x7f03cc1339f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:22.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.749+0000 7f03d190e700 1 -- 172.21.15.16:0/102429252 >> 172.21.15.16:0/102429252 conn(0x7f03cc066f50 msgr2=0x7f03cc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:22.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.749+0000 7f03d190e700 1 -- 172.21.15.16:0/102429252 shutdown_connections 2023-09-30T22:30:22.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.749+0000 7f03d190e700 1 -- 172.21.15.16:0/102429252 wait complete. 2023-09-30T22:30:22.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.750+0000 7f03d190e700 1 Processor -- start 2023-09-30T22:30:22.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.750+0000 7f03d190e700 1 -- start start 2023-09-30T22:30:22.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.751+0000 7f03d190e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f03cc09d6f0 0x7f03cc1a25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:22.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.751+0000 7f03d190e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03cc09e000 0x7f03cc1a2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:22.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.751+0000 7f03d190e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f03cc09e410 0x7f03cc1a3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:22.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.751+0000 7f03d190e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03cc0a2c20 con 0x7f03cc09e000 2023-09-30T22:30:22.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.751+0000 7f03d190e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03cc1a4f20 con 0x7f03cc09d6f0 2023-09-30T22:30:22.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.751+0000 7f03d190e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f03cc1a38a0 con 0x7f03cc09e410 2023-09-30T22:30:22.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.751+0000 7f03caffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f03cc09d6f0 0x7f03cc1a25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:22.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.751+0000 7f03caffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f03cc09d6f0 0x7f03cc1a25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:34298/0 (socket says 172.21.15.16:34298) 2023-09-30T22:30:22.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.751+0000 7f03caffd700 1 -- 172.21.15.16:0/818581565 learned_addr learned my addr 172.21.15.16:0/818581565 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:22.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.751+0000 7f03cb7fe700 1 --2- 172.21.15.16:0/818581565 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f03cc09e410 0x7f03cc1a3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:22.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.751+0000 7f03ca7fc700 1 --2- 172.21.15.16:0/818581565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03cc09e000 0x7f03cc1a2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:22.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.751+0000 7f03caffd700 1 -- 172.21.15.16:0/818581565 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f03cc09e410 msgr2=0x7f03cc1a3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:22.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.751+0000 7f03caffd700 1 --2- 172.21.15.16:0/818581565 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f03cc09e410 0x7f03cc1a3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:22.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.751+0000 7f03caffd700 1 -- 172.21.15.16:0/818581565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03cc09e000 msgr2=0x7f03cc1a2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:22.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.751+0000 7f03caffd700 1 --2- 172.21.15.16:0/818581565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03cc09e000 0x7f03cc1a2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:22.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.751+0000 7f03caffd700 1 -- 172.21.15.16:0/818581565 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03c00094a0 con 0x7f03cc09d6f0 2023-09-30T22:30:22.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.751+0000 7f03ca7fc700 1 --2- 172.21.15.16:0/818581565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03cc09e000 0x7f03cc1a2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:30:22.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.752+0000 7f03caffd700 1 --2- 172.21.15.16:0/818581565 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f03cc09d6f0 0x7f03cc1a25a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f03bc00e9e0 tx=0x7f03bc00ecf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:22.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.752+0000 7f03b3fff700 1 -- 172.21.15.16:0/818581565 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03bc009710 con 0x7f03cc09d6f0 2023-09-30T22:30:22.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.752+0000 7f03b3fff700 1 -- 172.21.15.16:0/818581565 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03bc00c850 con 0x7f03cc09d6f0 2023-09-30T22:30:22.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.752+0000 7f03b3fff700 1 -- 172.21.15.16:0/818581565 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03bc00cbc0 con 0x7f03cc09d6f0 2023-09-30T22:30:22.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.753+0000 7f03d190e700 1 -- 172.21.15.16:0/818581565 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03cc1a3ae0 con 0x7f03cc09d6f0 2023-09-30T22:30:22.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.753+0000 7f03d190e700 1 -- 172.21.15.16:0/818581565 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03cc071800 con 0x7f03cc09d6f0 2023-09-30T22:30:22.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.754+0000 7f03d190e700 1 -- 172.21.15.16:0/818581565 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03cc09e9f0 con 0x7f03cc09d6f0 2023-09-30T22:30:22.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.755+0000 7f03b3fff700 1 -- 172.21.15.16:0/818581565 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f03bc00cd20 con 0x7f03cc09d6f0 2023-09-30T22:30:22.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.755+0000 7f03b3fff700 1 -- 172.21.15.16:0/818581565 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f03bc040020 con 0x7f03cc09d6f0 2023-09-30T22:30:22.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.760+0000 7f03b3fff700 1 -- 172.21.15.16:0/818581565 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f03bc01a360 con 0x7f03cc09d6f0 2023-09-30T22:30:22.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.790+0000 7fe325b1c700 1 Processor -- start 2023-09-30T22:30:22.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.790+0000 7fe325b1c700 1 -- start start 2023-09-30T22:30:22.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.791+0000 7fe325b1c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe320131810 0x7fe320131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:22.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.791+0000 7fe325b1c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe320099a40 0x7fe320099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:22.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.791+0000 7fe325b1c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3201313c0 con 0x7fe32009a3b0 2023-09-30T22:30:22.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.791+0000 7fe325b1c700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe32009b580 con 0x7fe320131810 2023-09-30T22:30:22.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.791+0000 7fe325b1c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe32006e8c0 con 0x7fe320099a40 2023-09-30T22:30:22.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.791+0000 7fe31ffff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe32009a3b0 0x7fe320138650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57002/0 (socket says 172.21.15.16:57002) 2023-09-30T22:30:22.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.791+0000 7fe31f7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe320131810 0x7fe320131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:22.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.791+0000 7fe31ffff700 1 -- 172.21.15.16:0/1970873647 learned_addr learned my addr 172.21.15.16:0/1970873647 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:30:22.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.791+0000 7fe31effd700 1 --2- 172.21.15.16:0/1970873647 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe320099a40 0x7fe320099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:22.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.792+0000 7fe31e7fc700 1 -- 172.21.15.16:0/1970873647 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2804538868 0 0) 0x7fe3201313c0 con 0x7fe32009a3b0 2023-09-30T22:30:22.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.792+0000 7fe31e7fc700 1 -- 172.21.15.16:0/1970873647 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3080035b0 con 0x7fe32009a3b0 2023-09-30T22:30:22.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.792+0000 7fe31f7fe700 1 -- 172.21.15.16:0/1970873647 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe320099a40 msgr2=0x7fe320099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:22.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.792+0000 7fe31f7fe700 1 --2- 172.21.15.16:0/1970873647 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe320099a40 0x7fe320099e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:22.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.792+0000 7fe31f7fe700 1 -- 172.21.15.16:0/1970873647 >> v1:172.21.15.16:6789/0 conn(0x7fe32009a3b0 legacy=0x7fe320138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:22.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.792+0000 7fe31f7fe700 1 -- 172.21.15.16:0/1970873647 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe32009a830 con 0x7fe320131810 2023-09-30T22:30:22.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.792+0000 7fe31f7fe700 1 --2- 172.21.15.16:0/1970873647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe320131810 0x7fe320131be0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe310009840 tx=0x7fe310009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f5b589ecec15cc7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:22.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.792+0000 7fe31e7fc700 1 -- 172.21.15.16:0/1970873647 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe310011830 con 0x7fe320131810 2023-09-30T22:30:22.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.792+0000 7fe31e7fc700 1 -- 172.21.15.16:0/1970873647 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe310011990 con 0x7fe320131810 2023-09-30T22:30:22.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.792+0000 7fe31e7fc700 1 -- 172.21.15.16:0/1970873647 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe310011d20 con 0x7fe320131810 2023-09-30T22:30:22.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.793+0000 7fe325b1c700 1 -- 172.21.15.16:0/1970873647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe320131810 msgr2=0x7fe320131be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:22.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.793+0000 7fe325b1c700 1 --2- 172.21.15.16:0/1970873647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe320131810 0x7fe320131be0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe310009840 tx=0x7fe310009b50 comp rx=0 tx=0).stop 2023-09-30T22:30:22.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.793+0000 7fe325b1c700 1 -- 172.21.15.16:0/1970873647 shutdown_connections 2023-09-30T22:30:22.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.793+0000 7fe325b1c700 1 --2- 172.21.15.16:0/1970873647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe320131810 0x7fe320131be0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:22.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.793+0000 7fe325b1c700 1 --2- 172.21.15.16:0/1970873647 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe320099a40 0x7fe320099e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:22.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.793+0000 7fe325b1c700 1 -- 172.21.15.16:0/1970873647 >> 172.21.15.16:0/1970873647 conn(0x7fe3200a2aa0 msgr2=0x7fe3200a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:22.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.793+0000 7fe325b1c700 1 -- 172.21.15.16:0/1970873647 shutdown_connections 2023-09-30T22:30:22.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.793+0000 7fe325b1c700 1 -- 172.21.15.16:0/1970873647 wait complete. 2023-09-30T22:30:22.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.794+0000 7fe325b1c700 1 Processor -- start 2023-09-30T22:30:22.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.795+0000 7fe325b1c700 1 -- start start 2023-09-30T22:30:22.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.795+0000 7fe325b1c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe320099a40 0x7fe320119520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:22.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.795+0000 7fe325b1c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe32009a3b0 0x7fe320119a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:22.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.795+0000 7fe325b1c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe320131810 0x7fe320119fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:22.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.795+0000 7fe325b1c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32011d970 con 0x7fe320099a40 2023-09-30T22:30:22.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.795+0000 7fe325b1c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe32011b0a0 con 0x7fe32009a3b0 2023-09-30T22:30:22.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.795+0000 7fe325b1c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe32011c380 con 0x7fe320131810 2023-09-30T22:30:22.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.795+0000 7fe31effd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe32009a3b0 0x7fe320119a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:22.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.795+0000 7fe31f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe320099a40 0x7fe320119520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:22.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.795+0000 7fe31effd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe32009a3b0 0x7fe320119a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:34324/0 (socket says 172.21.15.16:34324) 2023-09-30T22:30:22.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.795+0000 7fe31f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe320099a40 0x7fe320119520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58056/0 (socket says 172.21.15.16:58056) 2023-09-30T22:30:22.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.795+0000 7fe31effd700 1 -- 172.21.15.16:0/3678758363 learned_addr learned my addr 172.21.15.16:0/3678758363 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:22.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.796+0000 7fe31ffff700 1 --2- 172.21.15.16:0/3678758363 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe320131810 0x7fe320119fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:22.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.796+0000 7fe31effd700 1 -- 172.21.15.16:0/3678758363 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe320131810 msgr2=0x7fe320119fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:22.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.796+0000 7fe31effd700 1 --2- 172.21.15.16:0/3678758363 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe320131810 0x7fe320119fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:22.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.796+0000 7fe31effd700 1 -- 172.21.15.16:0/3678758363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe320099a40 msgr2=0x7fe320119520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:22.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.796+0000 7fe31effd700 1 --2- 172.21.15.16:0/3678758363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe320099a40 0x7fe320119520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:22.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.796+0000 7fe31effd700 1 -- 172.21.15.16:0/3678758363 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3100094a0 con 0x7fe32009a3b0 2023-09-30T22:30:22.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.796+0000 7fe31effd700 1 --2- 172.21.15.16:0/3678758363 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe32009a3b0 0x7fe320119a60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe31400e9e0 tx=0x7fe31400ecf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:22.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.796+0000 7fe307fff700 1 -- 172.21.15.16:0/3678758363 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe314009710 con 0x7fe32009a3b0 2023-09-30T22:30:22.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.796+0000 7fe307fff700 1 -- 172.21.15.16:0/3678758363 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe31400c850 con 0x7fe32009a3b0 2023-09-30T22:30:22.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.796+0000 7fe325b1c700 1 -- 172.21.15.16:0/3678758363 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3200a1790 con 0x7fe32009a3b0 2023-09-30T22:30:22.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.796+0000 7fe307fff700 1 -- 172.21.15.16:0/3678758363 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31400cbc0 con 0x7fe32009a3b0 2023-09-30T22:30:22.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.796+0000 7fe325b1c700 1 -- 172.21.15.16:0/3678758363 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe32011c740 con 0x7fe32009a3b0 2023-09-30T22:30:22.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.797+0000 7fe307fff700 1 -- 172.21.15.16:0/3678758363 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe31400cd20 con 0x7fe32009a3b0 2023-09-30T22:30:22.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.798+0000 7fe325b1c700 1 -- 172.21.15.16:0/3678758363 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe320074ea0 con 0x7fe32009a3b0 2023-09-30T22:30:22.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.798+0000 7fe307fff700 1 -- 172.21.15.16:0/3678758363 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fe31403f7a0 con 0x7fe32009a3b0 2023-09-30T22:30:22.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.803+0000 7fe307fff700 1 -- 172.21.15.16:0/3678758363 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe3140232f0 con 0x7fe32009a3b0 2023-09-30T22:30:22.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.844+0000 7fd2c2a7e700 1 Processor -- start 2023-09-30T22:30:22.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.844+0000 7fd2c2a7e700 1 -- start start 2023-09-30T22:30:22.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.845+0000 7fd2c2a7e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd2bc12cf60 0x7fd2bc06f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:22.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.845+0000 7fd2c2a7e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd2bc12f8b0 0x7fd2bc12fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:22.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.845+0000 7fd2c2a7e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd2bc130260 0x7fd2bc136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:22.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.845+0000 7fd2c2a7e700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2bc12d330 con 0x7fd2bc12cf60 2023-09-30T22:30:22.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.845+0000 7fd2c2a7e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2bc12cd90 con 0x7fd2bc130260 2023-09-30T22:30:22.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.845+0000 7fd2c2a7e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd2bc1306d0 con 0x7fd2bc12f8b0 2023-09-30T22:30:22.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.845+0000 7fd2bbfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd2bc12f8b0 0x7fd2bc12fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:22.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.845+0000 7fd2bbfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd2bc12f8b0 0x7fd2bc12fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:58938/0 (socket says 172.21.15.16:58938) 2023-09-30T22:30:22.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.845+0000 7fd2bbfff700 1 -- 172.21.15.16:0/3853804757 learned_addr learned my addr 172.21.15.16:0/3853804757 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:22.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.845+0000 7fd2c081a700 1 --2- 172.21.15.16:0/3853804757 >> v2:172.21.15.16:3300/0 conn(0x7fd2bc12cf60 0x7fd2bc06f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:22.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.845+0000 7fd2c101b700 1 --2- 172.21.15.16:0/3853804757 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd2bc130260 0x7fd2bc136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:22.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.846+0000 7fd2bbfff700 1 -- 172.21.15.16:0/3853804757 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd2bc130260 msgr2=0x7fd2bc136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:22.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.846+0000 7fd2bbfff700 1 --2- 172.21.15.16:0/3853804757 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd2bc130260 0x7fd2bc136c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:22.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.846+0000 7fd2bbfff700 1 -- 172.21.15.16:0/3853804757 >> v2:172.21.15.16:3300/0 conn(0x7fd2bc12cf60 msgr2=0x7fd2bc06f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:22.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.846+0000 7fd2bbfff700 1 --2- 172.21.15.16:0/3853804757 >> v2:172.21.15.16:3300/0 conn(0x7fd2bc12cf60 0x7fd2bc06f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:22.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.846+0000 7fd2bbfff700 1 -- 172.21.15.16:0/3853804757 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2bc071780 con 0x7fd2bc12f8b0 2023-09-30T22:30:22.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.846+0000 7fd2c081a700 1 --2- 172.21.15.16:0/3853804757 >> v2:172.21.15.16:3300/0 conn(0x7fd2bc12cf60 0x7fd2bc06f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:22.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.846+0000 7fd2bbfff700 1 --2- 172.21.15.16:0/3853804757 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd2bc12f8b0 0x7fd2bc12fd20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8009920 tx=0x7fd2a8009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e92e4b3617e46fd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:22.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.846+0000 7fd2bb7fe700 1 -- 172.21.15.16:0/3853804757 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a8011830 con 0x7fd2bc12f8b0 2023-09-30T22:30:22.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.846+0000 7fd2bb7fe700 1 -- 172.21.15.16:0/3853804757 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a8011990 con 0x7fd2bc12f8b0 2023-09-30T22:30:22.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.846+0000 7fd2bb7fe700 1 -- 172.21.15.16:0/3853804757 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a8011d20 con 0x7fd2bc12f8b0 2023-09-30T22:30:22.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.847+0000 7fd2c2a7e700 1 -- 172.21.15.16:0/3853804757 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd2bc12f8b0 msgr2=0x7fd2bc12fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:22.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.847+0000 7fd2c2a7e700 1 --2- 172.21.15.16:0/3853804757 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd2bc12f8b0 0x7fd2bc12fd20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8009920 tx=0x7fd2a8009c30 comp rx=0 tx=0).stop 2023-09-30T22:30:22.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.847+0000 7fd2c2a7e700 1 -- 172.21.15.16:0/3853804757 shutdown_connections 2023-09-30T22:30:22.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.847+0000 7fd2c2a7e700 1 --2- 172.21.15.16:0/3853804757 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd2bc130260 0x7fd2bc136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:22.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.847+0000 7fd2c2a7e700 1 --2- 172.21.15.16:0/3853804757 >> v2:172.21.15.16:3300/0 conn(0x7fd2bc12cf60 0x7fd2bc06f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:22.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.847+0000 7fd2c2a7e700 1 --2- 172.21.15.16:0/3853804757 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd2bc12f8b0 0x7fd2bc12fd20 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:22.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.847+0000 7fd2c2a7e700 1 -- 172.21.15.16:0/3853804757 >> 172.21.15.16:0/3853804757 conn(0x7fd2bc0a2aa0 msgr2=0x7fd2bc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:22.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.847+0000 7fd2c2a7e700 1 -- 172.21.15.16:0/3853804757 shutdown_connections 2023-09-30T22:30:22.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.847+0000 7fd2c2a7e700 1 -- 172.21.15.16:0/3853804757 wait complete. 2023-09-30T22:30:22.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.848+0000 7fd2c2a7e700 1 Processor -- start 2023-09-30T22:30:22.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.848+0000 7fd2c2a7e700 1 -- start start 2023-09-30T22:30:22.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.849+0000 7fd2c2a7e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd2bc12cf60 0x7fd2bc136b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:22.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.849+0000 7fd2c2a7e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd2bc12f8b0 0x7fd2bc137080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:22.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.849+0000 7fd2c2a7e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2bc130260 0x7fd2bc1c6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:22.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.849+0000 7fd2c2a7e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2bc1337e0 con 0x7fd2bc130260 2023-09-30T22:30:22.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.849+0000 7fd2c2a7e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2bc135820 con 0x7fd2bc12cf60 2023-09-30T22:30:22.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.849+0000 7fd2c2a7e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd2bc137f80 con 0x7fd2bc12f8b0 2023-09-30T22:30:22.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.849+0000 7fd2bbfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd2bc12f8b0 0x7fd2bc137080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:22.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.849+0000 7fd2bbfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd2bc12f8b0 0x7fd2bc137080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:58946/0 (socket says 172.21.15.16:58946) 2023-09-30T22:30:22.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.849+0000 7fd2bbfff700 1 -- 172.21.15.16:0/1767208902 learned_addr learned my addr 172.21.15.16:0/1767208902 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:22.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.849+0000 7fd2c101b700 1 --2- 172.21.15.16:0/1767208902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2bc130260 0x7fd2bc1c6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:22.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.849+0000 7fd2c081a700 1 --2- 172.21.15.16:0/1767208902 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd2bc12cf60 0x7fd2bc136b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:22.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.849+0000 7fd2bbfff700 1 -- 172.21.15.16:0/1767208902 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd2bc12cf60 msgr2=0x7fd2bc136b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:22.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.849+0000 7fd2bbfff700 1 --2- 172.21.15.16:0/1767208902 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd2bc12cf60 0x7fd2bc136b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:22.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.849+0000 7fd2bbfff700 1 -- 172.21.15.16:0/1767208902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2bc130260 msgr2=0x7fd2bc1c6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:22.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.849+0000 7fd2bbfff700 1 --2- 172.21.15.16:0/1767208902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2bc130260 0x7fd2bc1c6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:22.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.849+0000 7fd2bbfff700 1 -- 172.21.15.16:0/1767208902 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b00094a0 con 0x7fd2bc12f8b0 2023-09-30T22:30:22.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.850+0000 7fd2bbfff700 1 --2- 172.21.15.16:0/1767208902 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd2bc12f8b0 0x7fd2bc137080 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8000c00 tx=0x7fd2a8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:22.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.850+0000 7fd2b97fa700 1 -- 172.21.15.16:0/1767208902 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a8004300 con 0x7fd2bc12f8b0 2023-09-30T22:30:22.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.850+0000 7fd2c2a7e700 1 -- 172.21.15.16:0/1767208902 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2a8009550 con 0x7fd2bc12f8b0 2023-09-30T22:30:22.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.850+0000 7fd2b97fa700 1 -- 172.21.15.16:0/1767208902 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a801f070 con 0x7fd2bc12f8b0 2023-09-30T22:30:22.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.850+0000 7fd2b97fa700 1 -- 172.21.15.16:0/1767208902 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a80036a0 con 0x7fd2bc12f8b0 2023-09-30T22:30:22.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.850+0000 7fd2c2a7e700 1 -- 172.21.15.16:0/1767208902 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2bc1351a0 con 0x7fd2bc12f8b0 2023-09-30T22:30:22.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.851+0000 7fd2b97fa700 1 -- 172.21.15.16:0/1767208902 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd2a8021030 con 0x7fd2bc12f8b0 2023-09-30T22:30:22.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.851+0000 7fd2c2a7e700 1 -- 172.21.15.16:0/1767208902 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2bc074ea0 con 0x7fd2bc12f8b0 2023-09-30T22:30:22.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.851+0000 7fd2b97fa700 1 -- 172.21.15.16:0/1767208902 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd2a8040850 con 0x7fd2bc12f8b0 2023-09-30T22:30:22.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.857+0000 7fd2b97fa700 1 -- 172.21.15.16:0/1767208902 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd2a8015070 con 0x7fd2bc12f8b0 2023-09-30T22:30:22.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.880+0000 7ff2fdb52700 1 Processor -- start 2023-09-30T22:30:22.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.880+0000 7ff2fdb52700 1 -- start start 2023-09-30T22:30:22.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.880+0000 7ff2fdb52700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff2f806a770 0x7ff2f806ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:22.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.880+0000 7ff2fdb52700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff2f8131d10 0x7ff2f8139860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:22.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.880+0000 7ff2fdb52700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2f809ece0 con 0x7ff2f806b080 2023-09-30T22:30:22.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.880+0000 7ff2fdb52700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f809e760 con 0x7ff2f806a770 2023-09-30T22:30:22.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.880+0000 7ff2fdb52700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f809e040 con 0x7ff2f8131d10 2023-09-30T22:30:22.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.881+0000 7ff2f7fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff2f8131d10 0x7ff2f8139860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:22.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.881+0000 7ff2f7fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff2f8131d10 0x7ff2f8139860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:58948/0 (socket says 172.21.15.16:58948) 2023-09-30T22:30:22.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.881+0000 7ff2f7fff700 1 -- 172.21.15.16:0/2888422403 learned_addr learned my addr 172.21.15.16:0/2888422403 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:22.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.881+0000 7ff2f77fe700 1 --2- 172.21.15.16:0/2888422403 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff2f806a770 0x7ff2f806ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:22.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.881+0000 7ff2f7fff700 1 -- 172.21.15.16:0/2888422403 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff2f806a770 msgr2=0x7ff2f806ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:22.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.881+0000 7ff2f7fff700 1 --2- 172.21.15.16:0/2888422403 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff2f806a770 0x7ff2f806ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:22.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.881+0000 7ff2f7fff700 1 -- 172.21.15.16:0/2888422403 >> v1:172.21.15.16:6789/0 conn(0x7ff2f806b080 legacy=0x7ff2f8131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:22.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.881+0000 7ff2f7fff700 1 -- 172.21.15.16:0/2888422403 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f809f630 con 0x7ff2f8131d10 2023-09-30T22:30:22.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.881+0000 7ff2f7fff700 1 --2- 172.21.15.16:0/2888422403 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff2f8131d10 0x7ff2f8139860 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0009e80 tx=0x7ff2f000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=232b7e326c26b7d3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:22.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.881+0000 7ff2f67fc700 1 -- 172.21.15.16:0/2888422403 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f000e070 con 0x7ff2f8131d10 2023-09-30T22:30:22.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.881+0000 7ff2f67fc700 1 -- 172.21.15.16:0/2888422403 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2f000b9c0 con 0x7ff2f8131d10 2023-09-30T22:30:22.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.881+0000 7ff2f67fc700 1 -- 172.21.15.16:0/2888422403 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f000bd70 con 0x7ff2f8131d10 2023-09-30T22:30:22.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.882+0000 7ff2fdb52700 1 -- 172.21.15.16:0/2888422403 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff2f8131d10 msgr2=0x7ff2f8139860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:22.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.882+0000 7ff2fdb52700 1 --2- 172.21.15.16:0/2888422403 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff2f8131d10 0x7ff2f8139860 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0009e80 tx=0x7ff2f000b330 comp rx=0 tx=0).stop 2023-09-30T22:30:22.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.882+0000 7ff2fdb52700 1 -- 172.21.15.16:0/2888422403 shutdown_connections 2023-09-30T22:30:22.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.882+0000 7ff2fdb52700 1 --2- 172.21.15.16:0/2888422403 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff2f8131d10 0x7ff2f8139860 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:22.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.882+0000 7ff2fdb52700 1 --2- 172.21.15.16:0/2888422403 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff2f806a770 0x7ff2f806ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:22.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.882+0000 7ff2fdb52700 1 -- 172.21.15.16:0/2888422403 >> 172.21.15.16:0/2888422403 conn(0x7ff2f8066f50 msgr2=0x7ff2f809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:22.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.882+0000 7ff2fdb52700 1 -- 172.21.15.16:0/2888422403 shutdown_connections 2023-09-30T22:30:22.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.882+0000 7ff2fdb52700 1 -- 172.21.15.16:0/2888422403 wait complete. 2023-09-30T22:30:22.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.883+0000 7ff2fdb52700 1 Processor -- start 2023-09-30T22:30:22.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.883+0000 7ff2fdb52700 1 -- start start 2023-09-30T22:30:22.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.884+0000 7ff2fdb52700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff2f806a770 0x7ff2f81c4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:22.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.884+0000 7ff2fdb52700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2f806b080 0x7ff2f81c46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:22.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.884+0000 7ff2fdb52700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff2f8131d10 0x7ff2f811e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:22.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.884+0000 7ff2fdb52700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f8130d70 con 0x7ff2f806b080 2023-09-30T22:30:22.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.884+0000 7ff2fdb52700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f8120190 con 0x7ff2f806a770 2023-09-30T22:30:22.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.884+0000 7ff2fdb52700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f8071e60 con 0x7ff2f8131d10 2023-09-30T22:30:22.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.884+0000 7ff2f77fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff2f806a770 0x7ff2f81c4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:22.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.884+0000 7ff2f77fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff2f806a770 0x7ff2f81c4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:34366/0 (socket says 172.21.15.16:34366) 2023-09-30T22:30:22.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.884+0000 7ff2f77fe700 1 -- 172.21.15.16:0/2431361846 learned_addr learned my addr 172.21.15.16:0/2431361846 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:22.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.884+0000 7ff2f7fff700 1 --2- 172.21.15.16:0/2431361846 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff2f8131d10 0x7ff2f811e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:22.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.884+0000 7ff2f6ffd700 1 --2- 172.21.15.16:0/2431361846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2f806b080 0x7ff2f81c46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:22.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.885+0000 7ff2f77fe700 1 -- 172.21.15.16:0/2431361846 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff2f8131d10 msgr2=0x7ff2f811e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:22.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.885+0000 7ff2f77fe700 1 --2- 172.21.15.16:0/2431361846 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff2f8131d10 0x7ff2f811e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:22.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.885+0000 7ff2f77fe700 1 -- 172.21.15.16:0/2431361846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2f806b080 msgr2=0x7ff2f81c46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:22.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.885+0000 7ff2f77fe700 1 --2- 172.21.15.16:0/2431361846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2f806b080 0x7ff2f81c46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:22.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.885+0000 7ff2f77fe700 1 -- 172.21.15.16:0/2431361846 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f0009b30 con 0x7ff2f806a770 2023-09-30T22:30:22.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.885+0000 7ff2f7fff700 1 --2- 172.21.15.16:0/2431361846 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff2f8131d10 0x7ff2f811e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:22.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.885+0000 7ff2f77fe700 1 --2- 172.21.15.16:0/2431361846 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff2f806a770 0x7ff2f81c4190 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff2e800bb00 tx=0x7ff2e800be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:22.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.885+0000 7ff2dffff700 1 -- 172.21.15.16:0/2431361846 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e8011a90 con 0x7ff2f806a770 2023-09-30T22:30:22.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.885+0000 7ff2dffff700 1 -- 172.21.15.16:0/2431361846 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2e8013040 con 0x7ff2f806a770 2023-09-30T22:30:22.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.885+0000 7ff2fdb52700 1 -- 172.21.15.16:0/2431361846 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2f81313e0 con 0x7ff2f806a770 2023-09-30T22:30:22.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.885+0000 7ff2dffff700 1 -- 172.21.15.16:0/2431361846 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e8011e30 con 0x7ff2f806a770 2023-09-30T22:30:22.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.885+0000 7ff2fdb52700 1 -- 172.21.15.16:0/2431361846 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2f8121f20 con 0x7ff2f806a770 2023-09-30T22:30:22.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.886+0000 7ff2dffff700 1 -- 172.21.15.16:0/2431361846 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff2e8021460 con 0x7ff2f806a770 2023-09-30T22:30:22.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.886+0000 7ff2dffff700 1 -- 172.21.15.16:0/2431361846 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7ff2e8043710 con 0x7ff2f806a770 2023-09-30T22:30:22.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.886+0000 7ff2fdb52700 1 -- 172.21.15.16:0/2431361846 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2f8121b80 con 0x7ff2f806a770 2023-09-30T22:30:22.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.891+0000 7ff2dffff700 1 -- 172.21.15.16:0/2431361846 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff2f8121b80 con 0x7ff2f806a770 2023-09-30T22:30:22.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.978+0000 7f03d190e700 1 -- 172.21.15.16:0/818581565 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f03cc074ea0 con 0x7f03cc09d6f0 2023-09-30T22:30:22.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.979+0000 7f03b3fff700 1 -- 172.21.15.16:0/818581565 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f03bc017070 con 0x7f03cc09d6f0 2023-09-30T22:30:22.979 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:30:22.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.981+0000 7f03d190e700 1 -- 172.21.15.16:0/818581565 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f03cc09d6f0 msgr2=0x7f03cc1a25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:22.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.981+0000 7f03d190e700 1 --2- 172.21.15.16:0/818581565 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f03cc09d6f0 0x7f03cc1a25a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f03bc00e9e0 tx=0x7f03bc00ecf0 comp rx=0 tx=0).stop 2023-09-30T22:30:22.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.981+0000 7f03d190e700 1 -- 172.21.15.16:0/818581565 shutdown_connections 2023-09-30T22:30:22.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.981+0000 7f03d190e700 1 --2- 172.21.15.16:0/818581565 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f03cc09d6f0 0x7f03cc1a25a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:22.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.981+0000 7f03d190e700 1 --2- 172.21.15.16:0/818581565 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f03cc09e410 0x7f03cc1a3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:22.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.981+0000 7f03d190e700 1 --2- 172.21.15.16:0/818581565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03cc09e000 0x7f03cc1a2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:22.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.981+0000 7f03d190e700 1 -- 172.21.15.16:0/818581565 >> 172.21.15.16:0/818581565 conn(0x7f03cc066f50 msgr2=0x7f03cc0a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:22.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.982+0000 7f03d190e700 1 -- 172.21.15.16:0/818581565 shutdown_connections 2023-09-30T22:30:22.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:22.984+0000 7f03d190e700 1 -- 172.21.15.16:0/818581565 wait complete. 2023-09-30T22:30:22.998 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:30:23.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:23.064+0000 7fe325b1c700 1 -- 172.21.15.16:0/3678758363 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe32011c930 con 0x7fe32009a3b0 2023-09-30T22:30:23.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:23.064+0000 7fe307fff700 1 -- 172.21.15.16:0/3678758363 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe31401ae80 con 0x7fe32009a3b0 2023-09-30T22:30:23.065 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:30:23.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:23.066+0000 7fe325b1c700 1 -- 172.21.15.16:0/3678758363 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe32009a3b0 msgr2=0x7fe320119a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:23.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:23.066+0000 7fe325b1c700 1 --2- 172.21.15.16:0/3678758363 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe32009a3b0 0x7fe320119a60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe31400e9e0 tx=0x7fe31400ecf0 comp rx=0 tx=0).stop 2023-09-30T22:30:23.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:23.066+0000 7fe325b1c700 1 -- 172.21.15.16:0/3678758363 shutdown_connections 2023-09-30T22:30:23.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:23.066+0000 7fe325b1c700 1 --2- 172.21.15.16:0/3678758363 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe320131810 0x7fe320119fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:23.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:23.066+0000 7fe325b1c700 1 --2- 172.21.15.16:0/3678758363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe320099a40 0x7fe320119520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:23.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:23.066+0000 7fe325b1c700 1 --2- 172.21.15.16:0/3678758363 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe32009a3b0 0x7fe320119a60 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:23.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:23.066+0000 7fe325b1c700 1 -- 172.21.15.16:0/3678758363 >> 172.21.15.16:0/3678758363 conn(0x7fe3200a2aa0 msgr2=0x7fe32012fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:23.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:23.067+0000 7fe325b1c700 1 -- 172.21.15.16:0/3678758363 shutdown_connections 2023-09-30T22:30:23.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:23.067+0000 7fe325b1c700 1 -- 172.21.15.16:0/3678758363 wait complete. 2023-09-30T22:30:23.085 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:30:23.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:23.088+0000 7fd2c2a7e700 1 -- 172.21.15.16:0/1767208902 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd2bc135640 con 0x7fd2bc12f8b0 2023-09-30T22:30:23.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:23.088+0000 7fd2b97fa700 1 -- 172.21.15.16:0/1767208902 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd2a80239c0 con 0x7fd2bc12f8b0 2023-09-30T22:30:23.089 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:30:23.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:23.090+0000 7fd2c2a7e700 1 -- 172.21.15.16:0/1767208902 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd2bc12f8b0 msgr2=0x7fd2bc137080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:23.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:23.090+0000 7fd2c2a7e700 1 --2- 172.21.15.16:0/1767208902 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd2bc12f8b0 0x7fd2bc137080 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8000c00 tx=0x7fd2a8004750 comp rx=0 tx=0).stop 2023-09-30T22:30:23.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:23.091+0000 7fd2c2a7e700 1 -- 172.21.15.16:0/1767208902 shutdown_connections 2023-09-30T22:30:23.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:23.091+0000 7fd2c2a7e700 1 --2- 172.21.15.16:0/1767208902 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd2bc12cf60 0x7fd2bc136b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:23.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:23.091+0000 7fd2c2a7e700 1 --2- 172.21.15.16:0/1767208902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2bc130260 0x7fd2bc1c6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:23.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:23.091+0000 7fd2c2a7e700 1 --2- 172.21.15.16:0/1767208902 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd2bc12f8b0 0x7fd2bc137080 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:23.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:23.091+0000 7fd2c2a7e700 1 -- 172.21.15.16:0/1767208902 >> 172.21.15.16:0/1767208902 conn(0x7fd2bc0a2aa0 msgr2=0x7fd2bc06e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:23.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:23.091+0000 7fd2c2a7e700 1 -- 172.21.15.16:0/1767208902 shutdown_connections 2023-09-30T22:30:23.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:23.091+0000 7fd2c2a7e700 1 -- 172.21.15.16:0/1767208902 wait complete. 2023-09-30T22:30:23.104 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:30:23.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:23.115+0000 7ff2fdb52700 1 -- 172.21.15.16:0/2431361846 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff2f8121b80 con 0x7ff2f806a770 2023-09-30T22:30:23.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:23.115+0000 7ff2dffff700 1 -- 172.21.15.16:0/2431361846 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff2f8121b80 con 0x7ff2f806a770 2023-09-30T22:30:23.116 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:30:23.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:23.118+0000 7ff2fdb52700 1 -- 172.21.15.16:0/2431361846 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff2f806a770 msgr2=0x7ff2f81c4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:23.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:23.118+0000 7ff2fdb52700 1 --2- 172.21.15.16:0/2431361846 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff2f806a770 0x7ff2f81c4190 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff2e800bb00 tx=0x7ff2e800be10 comp rx=0 tx=0).stop 2023-09-30T22:30:23.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:23.118+0000 7ff2fdb52700 1 -- 172.21.15.16:0/2431361846 shutdown_connections 2023-09-30T22:30:23.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:23.118+0000 7ff2fdb52700 1 --2- 172.21.15.16:0/2431361846 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff2f806a770 0x7ff2f81c4190 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:23.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:23.118+0000 7ff2fdb52700 1 --2- 172.21.15.16:0/2431361846 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff2f8131d10 0x7ff2f811e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:23.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:23.118+0000 7ff2fdb52700 1 --2- 172.21.15.16:0/2431361846 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2f806b080 0x7ff2f81c46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:23.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:23.118+0000 7ff2fdb52700 1 -- 172.21.15.16:0/2431361846 >> 172.21.15.16:0/2431361846 conn(0x7ff2f8066f50 msgr2=0x7ff2f80a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:23.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:23.118+0000 7ff2fdb52700 1 -- 172.21.15.16:0/2431361846 shutdown_connections 2023-09-30T22:30:23.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:23.118+0000 7ff2fdb52700 1 -- 172.21.15.16:0/2431361846 wait complete. 2023-09-30T22:30:23.131 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:30:23.999 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:24.086 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:24.106 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:24.132 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:24.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.149+0000 7f8ff4483700 1 Processor -- start 2023-09-30T22:30:24.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.149+0000 7f8ff4483700 1 -- start start 2023-09-30T22:30:24.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.150+0000 7f8ff4483700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8fec09d6f0 0x7f8fec09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:24.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.150+0000 7f8ff4483700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8fec09e440 0x7f8fec134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:24.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.150+0000 7f8ff4483700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fec09b120 con 0x7f8fec09e000 2023-09-30T22:30:24.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.150+0000 7f8ff4483700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fec06b460 con 0x7f8fec09d6f0 2023-09-30T22:30:24.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.150+0000 7f8ff4483700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec09a7e0 con 0x7f8fec09e440 2023-09-30T22:30:24.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.150+0000 7f8ff221f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8fec09d6f0 0x7f8fec09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:24.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.150+0000 7f8ff1a1e700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8fec09e000 0x7f8fec1339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57012/0 (socket says 172.21.15.16:57012) 2023-09-30T22:30:24.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.150+0000 7f8ff1a1e700 1 -- 172.21.15.16:0/3925210032 learned_addr learned my addr 172.21.15.16:0/3925210032 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:30:24.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.150+0000 7f8ff2a20700 1 --2- 172.21.15.16:0/3925210032 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8fec09e440 0x7f8fec134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:24.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.150+0000 7f8ff121d700 1 -- 172.21.15.16:0/3925210032 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 957549805 0 0) 0x7f8fec09b120 con 0x7f8fec09e000 2023-09-30T22:30:24.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.150+0000 7f8ff121d700 1 -- 172.21.15.16:0/3925210032 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fd80035b0 con 0x7f8fec09e000 2023-09-30T22:30:24.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.150+0000 7f8ff221f700 1 -- 172.21.15.16:0/3925210032 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8fec09e440 msgr2=0x7f8fec134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:24.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.150+0000 7f8ff221f700 1 --2- 172.21.15.16:0/3925210032 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8fec09e440 0x7f8fec134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.150+0000 7f8ff221f700 1 -- 172.21.15.16:0/3925210032 >> v1:172.21.15.16:6789/0 conn(0x7f8fec09e000 legacy=0x7f8fec1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:24.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.150+0000 7f8ff221f700 1 -- 172.21.15.16:0/3925210032 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fec09a500 con 0x7f8fec09d6f0 2023-09-30T22:30:24.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.151+0000 7f8ff221f700 1 --2- 172.21.15.16:0/3925210032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fec09d6f0 0x7f8fec09dac0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8009870 tx=0x7f8fe8009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f1f5cf2eba39598 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:24.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.151+0000 7f8ff121d700 1 -- 172.21.15.16:0/3925210032 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe8011830 con 0x7f8fec09d6f0 2023-09-30T22:30:24.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.151+0000 7f8ff121d700 1 -- 172.21.15.16:0/3925210032 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe8011990 con 0x7f8fec09d6f0 2023-09-30T22:30:24.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.151+0000 7f8ff121d700 1 -- 172.21.15.16:0/3925210032 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe8011d20 con 0x7f8fec09d6f0 2023-09-30T22:30:24.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.152+0000 7f8ff4483700 1 -- 172.21.15.16:0/3925210032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fec09d6f0 msgr2=0x7f8fec09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:24.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.152+0000 7f8ff4483700 1 --2- 172.21.15.16:0/3925210032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fec09d6f0 0x7f8fec09dac0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8009870 tx=0x7f8fe8009b80 comp rx=0 tx=0).stop 2023-09-30T22:30:24.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.153+0000 7f8ff4483700 1 -- 172.21.15.16:0/3925210032 shutdown_connections 2023-09-30T22:30:24.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.153+0000 7f8ff4483700 1 --2- 172.21.15.16:0/3925210032 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8fec09e440 0x7f8fec134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.153+0000 7f8ff4483700 1 --2- 172.21.15.16:0/3925210032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fec09d6f0 0x7f8fec09dac0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.153+0000 7f8ff4483700 1 -- 172.21.15.16:0/3925210032 >> 172.21.15.16:0/3925210032 conn(0x7f8fec066f50 msgr2=0x7f8fec09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:24.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.153+0000 7f8ff4483700 1 -- 172.21.15.16:0/3925210032 shutdown_connections 2023-09-30T22:30:24.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.153+0000 7f8ff4483700 1 -- 172.21.15.16:0/3925210032 wait complete. 2023-09-30T22:30:24.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.154+0000 7f8ff4483700 1 Processor -- start 2023-09-30T22:30:24.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.154+0000 7f8ff4483700 1 -- start start 2023-09-30T22:30:24.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.155+0000 7f8ff4483700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fec09d6f0 0x7f8fec1a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:24.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.155+0000 7f8ff4483700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8fec09e000 0x7f8fec1c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:24.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.155+0000 7f8ff4483700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8fec09e440 0x7f8fec1c8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:24.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.155+0000 7f8ff4483700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec1a4e20 con 0x7f8fec09d6f0 2023-09-30T22:30:24.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.155+0000 7f8ff4483700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec1a8460 con 0x7f8fec09e000 2023-09-30T22:30:24.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.155+0000 7f8ff4483700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec1a6450 con 0x7f8fec09e440 2023-09-30T22:30:24.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.155+0000 7f8ff221f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fec09d6f0 0x7f8fec1a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:24.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.155+0000 7f8ff221f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fec09d6f0 0x7f8fec1a54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58106/0 (socket says 172.21.15.16:58106) 2023-09-30T22:30:24.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.155+0000 7f8ff221f700 1 -- 172.21.15.16:0/1242226630 learned_addr learned my addr 172.21.15.16:0/1242226630 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:24.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.155+0000 7f8ff1a1e700 1 --2- 172.21.15.16:0/1242226630 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8fec09e000 0x7f8fec1c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:24.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.156+0000 7f8ff221f700 1 -- 172.21.15.16:0/1242226630 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8fec09e440 msgr2=0x7f8fec1c8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:24.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.156+0000 7f8ff221f700 1 --2- 172.21.15.16:0/1242226630 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8fec09e440 0x7f8fec1c8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.156+0000 7f8ff221f700 1 -- 172.21.15.16:0/1242226630 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8fec09e000 msgr2=0x7f8fec1c7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:24.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.156+0000 7f8ff221f700 1 --2- 172.21.15.16:0/1242226630 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8fec09e000 0x7f8fec1c7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.156+0000 7f8ff221f700 1 -- 172.21.15.16:0/1242226630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fe80094a0 con 0x7f8fec09d6f0 2023-09-30T22:30:24.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.156+0000 7f8ff1a1e700 1 --2- 172.21.15.16:0/1242226630 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8fec09e000 0x7f8fec1c7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:24.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.156+0000 7f8ff221f700 1 --2- 172.21.15.16:0/1242226630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fec09d6f0 0x7f8fec1a54e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8fe80090b0 tx=0x7f8fe8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:24.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.156+0000 7f8fe2ffd700 1 -- 172.21.15.16:0/1242226630 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe80036a0 con 0x7f8fec09d6f0 2023-09-30T22:30:24.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.156+0000 7f8fe2ffd700 1 -- 172.21.15.16:0/1242226630 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe801f070 con 0x7f8fec09d6f0 2023-09-30T22:30:24.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.156+0000 7f8fe2ffd700 1 -- 172.21.15.16:0/1242226630 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe8003a20 con 0x7f8fec09d6f0 2023-09-30T22:30:24.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.156+0000 7f8ff4483700 1 -- 172.21.15.16:0/1242226630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fec0a2e00 con 0x7f8fec09d6f0 2023-09-30T22:30:24.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.156+0000 7f8ff4483700 1 -- 172.21.15.16:0/1242226630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fec1a7dc0 con 0x7f8fec09d6f0 2023-09-30T22:30:24.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.157+0000 7f8fe2ffd700 1 -- 172.21.15.16:0/1242226630 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8fe8003b80 con 0x7f8fec09d6f0 2023-09-30T22:30:24.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.158+0000 7f8fe2ffd700 1 -- 172.21.15.16:0/1242226630 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f8fe8021070 con 0x7f8fec09d6f0 2023-09-30T22:30:24.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.158+0000 7f8ff4483700 1 -- 172.21.15.16:0/1242226630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fec074ea0 con 0x7f8fec09d6f0 2023-09-30T22:30:24.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.168+0000 7f8fe2ffd700 1 -- 172.21.15.16:0/1242226630 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8fe802b080 con 0x7f8fec09d6f0 2023-09-30T22:30:24.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.238+0000 7f173e086700 1 Processor -- start 2023-09-30T22:30:24.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.238+0000 7f173e086700 1 -- start start 2023-09-30T22:30:24.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.238+0000 7f173e086700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f173806a770 0x7f173809d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:24.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.238+0000 7f173e086700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f173809dc30 0x7f173809e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:24.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.238+0000 7f173e086700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f173813c1e0 0x7f173813e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:24.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.238+0000 7f173e086700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f173806ab40 con 0x7f173806a770 2023-09-30T22:30:24.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.238+0000 7f173e086700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f173809ebc0 con 0x7f173809dc30 2023-09-30T22:30:24.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.238+0000 7f173e086700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f173809a810 con 0x7f173813c1e0 2023-09-30T22:30:24.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.239+0000 7f1736ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f173809dc30 0x7f173809e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:24.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.239+0000 7f17377fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f173806a770 0x7f173809d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:24.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.239+0000 7f1736ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f173809dc30 0x7f173809e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:34384/0 (socket says 172.21.15.16:34384) 2023-09-30T22:30:24.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.239+0000 7f1736ffd700 1 -- 172.21.15.16:0/1611333686 learned_addr learned my addr 172.21.15.16:0/1611333686 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:24.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.239+0000 7f1737fff700 1 --2- 172.21.15.16:0/1611333686 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f173813c1e0 0x7f173813e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:24.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.239+0000 7f17377fe700 1 -- 172.21.15.16:0/1611333686 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f173813c1e0 msgr2=0x7f173813e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:24.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.239+0000 7f17377fe700 1 --2- 172.21.15.16:0/1611333686 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f173813c1e0 0x7f173813e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.239+0000 7f17377fe700 1 -- 172.21.15.16:0/1611333686 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f173809dc30 msgr2=0x7f173809e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:24.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.239+0000 7f17377fe700 1 --2- 172.21.15.16:0/1611333686 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f173809dc30 0x7f173809e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.239+0000 7f17377fe700 1 -- 172.21.15.16:0/1611333686 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f173809e920 con 0x7f173806a770 2023-09-30T22:30:24.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.239+0000 7f1736ffd700 1 --2- 172.21.15.16:0/1611333686 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f173809dc30 0x7f173809e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:24.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.239+0000 7f17377fe700 1 --2- 172.21.15.16:0/1611333686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f173806a770 0x7f173809d6f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1728009870 tx=0x7f1728009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=865247b8ee7ebbcd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:24.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.240+0000 7f17367fc700 1 -- 172.21.15.16:0/1611333686 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1728011830 con 0x7f173806a770 2023-09-30T22:30:24.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.240+0000 7f17367fc700 1 -- 172.21.15.16:0/1611333686 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1728011990 con 0x7f173806a770 2023-09-30T22:30:24.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.240+0000 7f17367fc700 1 -- 172.21.15.16:0/1611333686 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1728011d20 con 0x7f173806a770 2023-09-30T22:30:24.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.241+0000 7f173e086700 1 -- 172.21.15.16:0/1611333686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f173806a770 msgr2=0x7f173809d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:24.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.241+0000 7f173e086700 1 --2- 172.21.15.16:0/1611333686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f173806a770 0x7f173809d6f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1728009870 tx=0x7f1728009b80 comp rx=0 tx=0).stop 2023-09-30T22:30:24.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.241+0000 7f173e086700 1 -- 172.21.15.16:0/1611333686 shutdown_connections 2023-09-30T22:30:24.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.241+0000 7f173e086700 1 --2- 172.21.15.16:0/1611333686 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f173813c1e0 0x7f173813e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.241+0000 7f173e086700 1 --2- 172.21.15.16:0/1611333686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f173806a770 0x7f173809d6f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.241+0000 7f173e086700 1 --2- 172.21.15.16:0/1611333686 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f173809dc30 0x7f173809e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.241+0000 7f173e086700 1 -- 172.21.15.16:0/1611333686 >> 172.21.15.16:0/1611333686 conn(0x7f1738066f50 msgr2=0x7f173809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:24.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.241+0000 7f173e086700 1 -- 172.21.15.16:0/1611333686 shutdown_connections 2023-09-30T22:30:24.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.241+0000 7f173e086700 1 -- 172.21.15.16:0/1611333686 wait complete. 2023-09-30T22:30:24.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.242+0000 7f173e086700 1 Processor -- start 2023-09-30T22:30:24.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.243+0000 7f173e086700 1 -- start start 2023-09-30T22:30:24.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.243+0000 7f173e086700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f173806a770 0x7f173813ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:24.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.243+0000 7f173e086700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f173809dc30 0x7f173813a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:24.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.243+0000 7f173e086700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f173813c1e0 0x7f173813ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:24.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.243+0000 7f173e086700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17380a3960 con 0x7f173806a770 2023-09-30T22:30:24.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.243+0000 7f173e086700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f173811ed80 con 0x7f173809dc30 2023-09-30T22:30:24.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.243+0000 7f173e086700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f173811cfc0 con 0x7f173813c1e0 2023-09-30T22:30:24.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.244+0000 7f1736ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f173809dc30 0x7f173813a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:24.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.244+0000 7f1736ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f173809dc30 0x7f173813a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:34396/0 (socket says 172.21.15.16:34396) 2023-09-30T22:30:24.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.244+0000 7f1736ffd700 1 -- 172.21.15.16:0/1665484266 learned_addr learned my addr 172.21.15.16:0/1665484266 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:24.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.244+0000 7f17377fe700 1 --2- 172.21.15.16:0/1665484266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f173806a770 0x7f173813ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:24.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.244+0000 7f1737fff700 1 --2- 172.21.15.16:0/1665484266 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f173813c1e0 0x7f173813ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:24.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.244+0000 7f1736ffd700 1 -- 172.21.15.16:0/1665484266 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f173813c1e0 msgr2=0x7f173813ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:24.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.244+0000 7f1736ffd700 1 --2- 172.21.15.16:0/1665484266 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f173813c1e0 0x7f173813ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.244+0000 7f1736ffd700 1 -- 172.21.15.16:0/1665484266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f173806a770 msgr2=0x7f173813ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:24.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.244+0000 7f1736ffd700 1 --2- 172.21.15.16:0/1665484266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f173806a770 0x7f173813ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.244+0000 7f1736ffd700 1 -- 172.21.15.16:0/1665484266 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f172c009550 con 0x7f173809dc30 2023-09-30T22:30:24.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.244+0000 7f1737fff700 1 --2- 172.21.15.16:0/1665484266 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f173813c1e0 0x7f173813ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:30:24.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.244+0000 7f1736ffd700 1 --2- 172.21.15.16:0/1665484266 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f173809dc30 0x7f173813a600 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f172c00ec10 tx=0x7f172c00ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:24.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.244+0000 7f171ffff700 1 -- 172.21.15.16:0/1665484266 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172c00c850 con 0x7f173809dc30 2023-09-30T22:30:24.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.245+0000 7f171ffff700 1 -- 172.21.15.16:0/1665484266 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f172c00c9b0 con 0x7f173809dc30 2023-09-30T22:30:24.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.245+0000 7f173e086700 1 -- 172.21.15.16:0/1665484266 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17280094a0 con 0x7f173809dc30 2023-09-30T22:30:24.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.245+0000 7f171ffff700 1 -- 172.21.15.16:0/1665484266 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172c00cd20 con 0x7f173809dc30 2023-09-30T22:30:24.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.245+0000 7f173e086700 1 -- 172.21.15.16:0/1665484266 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f173811fd40 con 0x7f173809dc30 2023-09-30T22:30:24.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.246+0000 7f171ffff700 1 -- 172.21.15.16:0/1665484266 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f172c013470 con 0x7f173809dc30 2023-09-30T22:30:24.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.246+0000 7f173e086700 1 -- 172.21.15.16:0/1665484266 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1738074ea0 con 0x7f173809dc30 2023-09-30T22:30:24.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.246+0000 7f171ffff700 1 -- 172.21.15.16:0/1665484266 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f172c03f870 con 0x7f173809dc30 2023-09-30T22:30:24.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.251+0000 7f171ffff700 1 -- 172.21.15.16:0/1665484266 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f172c013720 con 0x7f173809dc30 2023-09-30T22:30:24.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.277+0000 7fcb93bb2700 1 Processor -- start 2023-09-30T22:30:24.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.278+0000 7fcb93bb2700 1 -- start start 2023-09-30T22:30:24.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.278+0000 7fcb93bb2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcb8c09e1d0 0x7fcb8c138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:24.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.278+0000 7fcb93bb2700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcb8c139170 0x7fcb8c13b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:24.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.278+0000 7fcb93bb2700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb8c09b120 con 0x7fcb8c09d6f0 2023-09-30T22:30:24.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.278+0000 7fcb93bb2700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb8c06b460 con 0x7fcb8c139170 2023-09-30T22:30:24.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.278+0000 7fcb93bb2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8c09a7e0 con 0x7fcb8c09e1d0 2023-09-30T22:30:24.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.278+0000 7fcb9114d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcb8c09e1d0 0x7fcb8c138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:24.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.278+0000 7fcb9194e700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fcb8c09d6f0 0x7fcb8c09dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57022/0 (socket says 172.21.15.16:57022) 2023-09-30T22:30:24.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.278+0000 7fcb9194e700 1 -- 172.21.15.16:0/2391177171 learned_addr learned my addr 172.21.15.16:0/2391177171 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:30:24.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.279+0000 7fcb9214f700 1 --2- 172.21.15.16:0/2391177171 >> v2:172.21.15.16:3300/0 conn(0x7fcb8c139170 0x7fcb8c13b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:24.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.279+0000 7fcb9114d700 1 -- 172.21.15.16:0/2391177171 >> v2:172.21.15.16:3300/0 conn(0x7fcb8c139170 msgr2=0x7fcb8c13b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:24.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.279+0000 7fcb9114d700 1 --2- 172.21.15.16:0/2391177171 >> v2:172.21.15.16:3300/0 conn(0x7fcb8c139170 0x7fcb8c13b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.279+0000 7fcb9114d700 1 -- 172.21.15.16:0/2391177171 >> v1:172.21.15.16:6789/0 conn(0x7fcb8c09d6f0 legacy=0x7fcb8c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:24.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.279+0000 7fcb9114d700 1 -- 172.21.15.16:0/2391177171 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb8c09a500 con 0x7fcb8c09e1d0 2023-09-30T22:30:24.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.279+0000 7fcb9114d700 1 --2- 172.21.15.16:0/2391177171 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcb8c09e1d0 0x7fcb8c138c30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c009870 tx=0x7fcb7c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd47d4c1f8a5956c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:24.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.279+0000 7fcb9094c700 1 -- 172.21.15.16:0/2391177171 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c011830 con 0x7fcb8c09e1d0 2023-09-30T22:30:24.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.279+0000 7fcb9094c700 1 -- 172.21.15.16:0/2391177171 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb7c011990 con 0x7fcb8c09e1d0 2023-09-30T22:30:24.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.279+0000 7fcb9094c700 1 -- 172.21.15.16:0/2391177171 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c011d20 con 0x7fcb8c09e1d0 2023-09-30T22:30:24.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.280+0000 7fcb93bb2700 1 -- 172.21.15.16:0/2391177171 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcb8c09e1d0 msgr2=0x7fcb8c138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:24.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.280+0000 7fcb93bb2700 1 --2- 172.21.15.16:0/2391177171 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcb8c09e1d0 0x7fcb8c138c30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c009870 tx=0x7fcb7c009b80 comp rx=0 tx=0).stop 2023-09-30T22:30:24.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.280+0000 7fcb93bb2700 1 -- 172.21.15.16:0/2391177171 shutdown_connections 2023-09-30T22:30:24.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.280+0000 7fcb93bb2700 1 --2- 172.21.15.16:0/2391177171 >> v2:172.21.15.16:3300/0 conn(0x7fcb8c139170 0x7fcb8c13b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.280+0000 7fcb93bb2700 1 --2- 172.21.15.16:0/2391177171 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcb8c09e1d0 0x7fcb8c138c30 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.280+0000 7fcb93bb2700 1 -- 172.21.15.16:0/2391177171 >> 172.21.15.16:0/2391177171 conn(0x7fcb8c066f50 msgr2=0x7fcb8c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:24.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.280+0000 7fcb93bb2700 1 -- 172.21.15.16:0/2391177171 shutdown_connections 2023-09-30T22:30:24.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.280+0000 7fcb93bb2700 1 -- 172.21.15.16:0/2391177171 wait complete. 2023-09-30T22:30:24.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.281+0000 7fcb93bb2700 1 Processor -- start 2023-09-30T22:30:24.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.281+0000 7fcb93bb2700 1 -- start start 2023-09-30T22:30:24.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.282+0000 7fcb93bb2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcb8c09d6f0 0x7fcb8c11e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:24.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.282+0000 7fcb93bb2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb8c09e1d0 0x7fcb8c11eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:24.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.282+0000 7fcb93bb2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcb8c139170 0x7fcb8c11f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:24.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.282+0000 7fcb93bb2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8c1220a0 con 0x7fcb8c09e1d0 2023-09-30T22:30:24.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.282+0000 7fcb93bb2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8c120580 con 0x7fcb8c09d6f0 2023-09-30T22:30:24.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.282+0000 7fcb93bb2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8c0a20b0 con 0x7fcb8c139170 2023-09-30T22:30:24.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.282+0000 7fcb9214f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcb8c139170 0x7fcb8c11f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:24.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.282+0000 7fcb9114d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb8c09e1d0 0x7fcb8c11eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:24.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.282+0000 7fcb9214f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcb8c139170 0x7fcb8c11f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:59006/0 (socket says 172.21.15.16:59006) 2023-09-30T22:30:24.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.282+0000 7fcb9114d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb8c09e1d0 0x7fcb8c11eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58162/0 (socket says 172.21.15.16:58162) 2023-09-30T22:30:24.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.282+0000 7fcb9214f700 1 -- 172.21.15.16:0/1399298624 learned_addr learned my addr 172.21.15.16:0/1399298624 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:24.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.282+0000 7fcb9214f700 1 -- 172.21.15.16:0/1399298624 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcb8c09d6f0 msgr2=0x7fcb8c11e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:24.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.282+0000 7fcb9214f700 1 --2- 172.21.15.16:0/1399298624 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcb8c09d6f0 0x7fcb8c11e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.282+0000 7fcb9214f700 1 -- 172.21.15.16:0/1399298624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb8c09e1d0 msgr2=0x7fcb8c11eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:24.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.282+0000 7fcb9214f700 1 --2- 172.21.15.16:0/1399298624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb8c09e1d0 0x7fcb8c11eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.282+0000 7fcb9214f700 1 -- 172.21.15.16:0/1399298624 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb7c0094a0 con 0x7fcb8c139170 2023-09-30T22:30:24.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.282+0000 7fcb9114d700 1 --2- 172.21.15.16:0/1399298624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb8c09e1d0 0x7fcb8c11eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:24.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.283+0000 7fcb9214f700 1 --2- 172.21.15.16:0/1399298624 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcb8c139170 0x7fcb8c11f2e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fcb84007e40 tx=0x7fcb8400e3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:24.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.283+0000 7fcb827fc700 1 -- 172.21.15.16:0/1399298624 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb8400ede0 con 0x7fcb8c139170 2023-09-30T22:30:24.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.283+0000 7fcb827fc700 1 -- 172.21.15.16:0/1399298624 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb84014420 con 0x7fcb8c139170 2023-09-30T22:30:24.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.283+0000 7fcb827fc700 1 -- 172.21.15.16:0/1399298624 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb840147d0 con 0x7fcb8c139170 2023-09-30T22:30:24.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.283+0000 7fcb93bb2700 1 -- 172.21.15.16:0/1399298624 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb8c09c710 con 0x7fcb8c139170 2023-09-30T22:30:24.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.283+0000 7fcb93bb2700 1 -- 172.21.15.16:0/1399298624 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb8c122b10 con 0x7fcb8c139170 2023-09-30T22:30:24.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.284+0000 7fcb93bb2700 1 -- 172.21.15.16:0/1399298624 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb8c074ea0 con 0x7fcb8c139170 2023-09-30T22:30:24.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.285+0000 7fb9c66d2700 1 Processor -- start 2023-09-30T22:30:24.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.286+0000 7fb9c66d2700 1 -- start start 2023-09-30T22:30:24.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.286+0000 7fcb827fc700 1 -- 172.21.15.16:0/1399298624 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcb84014930 con 0x7fcb8c139170 2023-09-30T22:30:24.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.286+0000 7fcb827fc700 1 -- 172.21.15.16:0/1399298624 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fcb84040a50 con 0x7fcb8c139170 2023-09-30T22:30:24.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.286+0000 7fb9c66d2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb9c009de00 0x7fb9c009e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:24.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.286+0000 7fb9c66d2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb9c013d1f0 0x7fb9c013f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:24.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.286+0000 7fb9c66d2700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9c009a810 con 0x7fb9c006a770 2023-09-30T22:30:24.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.286+0000 7fb9c66d2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c006ab40 con 0x7fb9c009de00 2023-09-30T22:30:24.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.286+0000 7fb9c66d2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c009ebc0 con 0x7fb9c013d1f0 2023-09-30T22:30:24.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.287+0000 7fb9c4c6f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb9c013d1f0 0x7fb9c013f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:24.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.287+0000 7fb9c4c6f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb9c013d1f0 0x7fb9c013f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:59016/0 (socket says 172.21.15.16:59016) 2023-09-30T22:30:24.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.287+0000 7fb9bf7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb9c009de00 0x7fb9c009e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:24.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.287+0000 7fb9c4c6f700 1 -- 172.21.15.16:0/2434468916 learned_addr learned my addr 172.21.15.16:0/2434468916 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:24.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.287+0000 7fb9bf7fe700 1 -- 172.21.15.16:0/2434468916 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb9c013d1f0 msgr2=0x7fb9c013f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:24.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.287+0000 7fb9bf7fe700 1 --2- 172.21.15.16:0/2434468916 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb9c013d1f0 0x7fb9c013f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.287+0000 7fb9bf7fe700 1 -- 172.21.15.16:0/2434468916 >> v1:172.21.15.16:6789/0 conn(0x7fb9c006a770 legacy=0x7fb9c009d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:24.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.287+0000 7fb9bf7fe700 1 -- 172.21.15.16:0/2434468916 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c009e920 con 0x7fb9c009de00 2023-09-30T22:30:24.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.287+0000 7fb9bf7fe700 1 --2- 172.21.15.16:0/2434468916 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb9c009de00 0x7fb9c009e250 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4009870 tx=0x7fb9b4009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ef5c20e75594e2a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:24.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.287+0000 7fb9beffd700 1 -- 172.21.15.16:0/2434468916 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b4011830 con 0x7fb9c009de00 2023-09-30T22:30:24.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.288+0000 7fb9beffd700 1 -- 172.21.15.16:0/2434468916 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9b4011990 con 0x7fb9c009de00 2023-09-30T22:30:24.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.288+0000 7fb9beffd700 1 -- 172.21.15.16:0/2434468916 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b4011d20 con 0x7fb9c009de00 2023-09-30T22:30:24.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.288+0000 7fb9c66d2700 1 -- 172.21.15.16:0/2434468916 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb9c009de00 msgr2=0x7fb9c009e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:24.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.288+0000 7fb9c66d2700 1 --2- 172.21.15.16:0/2434468916 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb9c009de00 0x7fb9c009e250 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4009870 tx=0x7fb9b4009b80 comp rx=0 tx=0).stop 2023-09-30T22:30:24.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.288+0000 7fb9c66d2700 1 -- 172.21.15.16:0/2434468916 shutdown_connections 2023-09-30T22:30:24.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.288+0000 7fb9c66d2700 1 --2- 172.21.15.16:0/2434468916 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb9c013d1f0 0x7fb9c013f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.288+0000 7fb9c66d2700 1 --2- 172.21.15.16:0/2434468916 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb9c009de00 0x7fb9c009e250 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.288+0000 7fb9c66d2700 1 -- 172.21.15.16:0/2434468916 >> 172.21.15.16:0/2434468916 conn(0x7fb9c0066f50 msgr2=0x7fb9c009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:24.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.288+0000 7fb9c66d2700 1 -- 172.21.15.16:0/2434468916 shutdown_connections 2023-09-30T22:30:24.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.289+0000 7fb9c66d2700 1 -- 172.21.15.16:0/2434468916 wait complete. 2023-09-30T22:30:24.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.290+0000 7fb9c66d2700 1 Processor -- start 2023-09-30T22:30:24.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.290+0000 7fcb827fc700 1 -- 172.21.15.16:0/1399298624 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcb84013d50 con 0x7fcb8c139170 2023-09-30T22:30:24.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.290+0000 7fb9c66d2700 1 -- start start 2023-09-30T22:30:24.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.291+0000 7fb9c66d2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb9c006a770 0x7fb9c01cc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:24.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.291+0000 7fb9c66d2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb9c009de00 0x7fb9c01ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:24.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.291+0000 7fb9c66d2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9c013d1f0 0x7fb9c01cd440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:24.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.291+0000 7fb9c66d2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c0071fc0 con 0x7fb9c013d1f0 2023-09-30T22:30:24.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.291+0000 7fb9c66d2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c0072180 con 0x7fb9c006a770 2023-09-30T22:30:24.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.291+0000 7fb9c66d2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c006eba0 con 0x7fb9c009de00 2023-09-30T22:30:24.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.291+0000 7fb9bf7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb9c009de00 0x7fb9c01ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:24.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.291+0000 7fb9c4c6f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9c013d1f0 0x7fb9c01cd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:24.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.291+0000 7fb9bf7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb9c009de00 0x7fb9c01ccf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:59028/0 (socket says 172.21.15.16:59028) 2023-09-30T22:30:24.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.291+0000 7fb9bf7fe700 1 -- 172.21.15.16:0/149247478 learned_addr learned my addr 172.21.15.16:0/149247478 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:24.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.291+0000 7fb9bffff700 1 --2- 172.21.15.16:0/149247478 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb9c006a770 0x7fb9c01cc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:24.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.292+0000 7fb9bf7fe700 1 -- 172.21.15.16:0/149247478 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb9c006a770 msgr2=0x7fb9c01cc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:24.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.292+0000 7fb9bf7fe700 1 --2- 172.21.15.16:0/149247478 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb9c006a770 0x7fb9c01cc9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.292+0000 7fb9bf7fe700 1 -- 172.21.15.16:0/149247478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9c013d1f0 msgr2=0x7fb9c01cd440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:24.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.292+0000 7fb9bf7fe700 1 --2- 172.21.15.16:0/149247478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9c013d1f0 0x7fb9c01cd440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.292+0000 7fb9bf7fe700 1 -- 172.21.15.16:0/149247478 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9b40094a0 con 0x7fb9c009de00 2023-09-30T22:30:24.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.292+0000 7fb9bf7fe700 1 --2- 172.21.15.16:0/149247478 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb9c009de00 0x7fb9c01ccf00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb9b40090b0 tx=0x7fb9b4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:24.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.292+0000 7fb9bcff9700 1 -- 172.21.15.16:0/149247478 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b4004510 con 0x7fb9c009de00 2023-09-30T22:30:24.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.292+0000 7fb9bcff9700 1 -- 172.21.15.16:0/149247478 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9b401f070 con 0x7fb9c009de00 2023-09-30T22:30:24.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.292+0000 7fb9bcff9700 1 -- 172.21.15.16:0/149247478 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b40038e0 con 0x7fb9c009de00 2023-09-30T22:30:24.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.292+0000 7fb9c66d2700 1 -- 172.21.15.16:0/149247478 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9c0070250 con 0x7fb9c009de00 2023-09-30T22:30:24.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.292+0000 7fb9c66d2700 1 -- 172.21.15.16:0/149247478 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9c00a38a0 con 0x7fb9c009de00 2023-09-30T22:30:24.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.293+0000 7fb9c66d2700 1 -- 172.21.15.16:0/149247478 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9c0074ea0 con 0x7fb9c009de00 2023-09-30T22:30:24.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.294+0000 7fb9bcff9700 1 -- 172.21.15.16:0/149247478 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb9b4003a40 con 0x7fb9c009de00 2023-09-30T22:30:24.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.294+0000 7fb9bcff9700 1 -- 172.21.15.16:0/149247478 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fb9b4021070 con 0x7fb9c009de00 2023-09-30T22:30:24.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.299+0000 7fb9bcff9700 1 -- 172.21.15.16:0/149247478 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb9b4010380 con 0x7fb9c009de00 2023-09-30T22:30:24.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.399+0000 7f8ff4483700 1 -- 172.21.15.16:0/1242226630 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8fec1b92f0 con 0x7f8fec09d6f0 2023-09-30T22:30:24.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.400+0000 7f8fe2ffd700 1 -- 172.21.15.16:0/1242226630 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8fe8035090 con 0x7f8fec09d6f0 2023-09-30T22:30:24.400 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:30:24.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.403+0000 7f8ff4483700 1 -- 172.21.15.16:0/1242226630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fec09d6f0 msgr2=0x7f8fec1a54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:24.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.403+0000 7f8ff4483700 1 --2- 172.21.15.16:0/1242226630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fec09d6f0 0x7f8fec1a54e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8fe80090b0 tx=0x7f8fe8004750 comp rx=0 tx=0).stop 2023-09-30T22:30:24.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.403+0000 7f8ff4483700 1 -- 172.21.15.16:0/1242226630 shutdown_connections 2023-09-30T22:30:24.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.403+0000 7f8ff4483700 1 --2- 172.21.15.16:0/1242226630 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8fec09e440 0x7f8fec1c8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.403+0000 7f8ff4483700 1 --2- 172.21.15.16:0/1242226630 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fec09d6f0 0x7f8fec1a54e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.403+0000 7f8ff4483700 1 --2- 172.21.15.16:0/1242226630 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8fec09e000 0x7f8fec1c7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.403+0000 7f8ff4483700 1 -- 172.21.15.16:0/1242226630 >> 172.21.15.16:0/1242226630 conn(0x7f8fec066f50 msgr2=0x7f8fec1bb9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:24.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.404+0000 7f8ff4483700 1 -- 172.21.15.16:0/1242226630 shutdown_connections 2023-09-30T22:30:24.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.404+0000 7f8ff4483700 1 -- 172.21.15.16:0/1242226630 wait complete. 2023-09-30T22:30:24.418 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:30:24.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.465+0000 7f173e086700 1 -- 172.21.15.16:0/1665484266 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f173811ff60 con 0x7f173809dc30 2023-09-30T22:30:24.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.465+0000 7f171ffff700 1 -- 172.21.15.16:0/1665484266 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f172c01b3b0 con 0x7f173809dc30 2023-09-30T22:30:24.466 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:30:24.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.467+0000 7f173e086700 1 -- 172.21.15.16:0/1665484266 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f173809dc30 msgr2=0x7f173813a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:24.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.467+0000 7f173e086700 1 --2- 172.21.15.16:0/1665484266 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f173809dc30 0x7f173813a600 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f172c00ec10 tx=0x7f172c00ef20 comp rx=0 tx=0).stop 2023-09-30T22:30:24.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.467+0000 7f173e086700 1 -- 172.21.15.16:0/1665484266 shutdown_connections 2023-09-30T22:30:24.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.467+0000 7f173e086700 1 --2- 172.21.15.16:0/1665484266 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f173813c1e0 0x7f173813ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.467+0000 7f173e086700 1 --2- 172.21.15.16:0/1665484266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f173806a770 0x7f173813ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.467+0000 7f173e086700 1 --2- 172.21.15.16:0/1665484266 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f173809dc30 0x7f173813a600 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.467+0000 7f173e086700 1 -- 172.21.15.16:0/1665484266 >> 172.21.15.16:0/1665484266 conn(0x7f1738066f50 msgr2=0x7f17380a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:24.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.468+0000 7f173e086700 1 -- 172.21.15.16:0/1665484266 shutdown_connections 2023-09-30T22:30:24.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.468+0000 7f173e086700 1 -- 172.21.15.16:0/1665484266 wait complete. 2023-09-30T22:30:24.482 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:30:24.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.508+0000 7fb9c66d2700 1 -- 172.21.15.16:0/149247478 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb9c009a3d0 con 0x7fb9c009de00 2023-09-30T22:30:24.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.509+0000 7fb9bcff9700 1 -- 172.21.15.16:0/149247478 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb9b4015070 con 0x7fb9c009de00 2023-09-30T22:30:24.509 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:30:24.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.510+0000 7fcb93bb2700 1 -- 172.21.15.16:0/1399298624 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fcb8c1223e0 con 0x7fcb8c139170 2023-09-30T22:30:24.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.510+0000 7fcb827fc700 1 -- 172.21.15.16:0/1399298624 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcb84013b70 con 0x7fcb8c139170 2023-09-30T22:30:24.511 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:30:24.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.511+0000 7fb9c66d2700 1 -- 172.21.15.16:0/149247478 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb9c009de00 msgr2=0x7fb9c01ccf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:24.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.511+0000 7fb9c66d2700 1 --2- 172.21.15.16:0/149247478 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb9c009de00 0x7fb9c01ccf00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb9b40090b0 tx=0x7fb9b4004750 comp rx=0 tx=0).stop 2023-09-30T22:30:24.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.511+0000 7fb9c66d2700 1 -- 172.21.15.16:0/149247478 shutdown_connections 2023-09-30T22:30:24.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.511+0000 7fb9c66d2700 1 --2- 172.21.15.16:0/149247478 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb9c006a770 0x7fb9c01cc9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.511+0000 7fb9c66d2700 1 --2- 172.21.15.16:0/149247478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9c013d1f0 0x7fb9c01cd440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.511+0000 7fb9c66d2700 1 --2- 172.21.15.16:0/149247478 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb9c009de00 0x7fb9c01ccf00 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.511+0000 7fb9c66d2700 1 -- 172.21.15.16:0/149247478 >> 172.21.15.16:0/149247478 conn(0x7fb9c0066f50 msgr2=0x7fb9c01384d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:24.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.511+0000 7fb9c66d2700 1 -- 172.21.15.16:0/149247478 shutdown_connections 2023-09-30T22:30:24.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.512+0000 7fb9c66d2700 1 -- 172.21.15.16:0/149247478 wait complete. 2023-09-30T22:30:24.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.513+0000 7fcb93bb2700 1 -- 172.21.15.16:0/1399298624 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcb8c139170 msgr2=0x7fcb8c11f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:24.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.513+0000 7fcb93bb2700 1 --2- 172.21.15.16:0/1399298624 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcb8c139170 0x7fcb8c11f2e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fcb84007e40 tx=0x7fcb8400e3c0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.513+0000 7fcb93bb2700 1 -- 172.21.15.16:0/1399298624 shutdown_connections 2023-09-30T22:30:24.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.513+0000 7fcb93bb2700 1 --2- 172.21.15.16:0/1399298624 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcb8c09d6f0 0x7fcb8c11e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.513+0000 7fcb93bb2700 1 --2- 172.21.15.16:0/1399298624 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcb8c139170 0x7fcb8c11f2e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.513+0000 7fcb93bb2700 1 --2- 172.21.15.16:0/1399298624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb8c09e1d0 0x7fcb8c11eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:24.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.513+0000 7fcb93bb2700 1 -- 172.21.15.16:0/1399298624 >> 172.21.15.16:0/1399298624 conn(0x7fcb8c066f50 msgr2=0x7fcb8c136c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:24.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.513+0000 7fcb93bb2700 1 -- 172.21.15.16:0/1399298624 shutdown_connections 2023-09-30T22:30:24.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:24.513+0000 7fcb93bb2700 1 -- 172.21.15.16:0/1399298624 wait complete. 2023-09-30T22:30:24.527 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:30:24.527 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:30:25.419 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:25.483 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:25.527 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:25.528 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:25.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.571+0000 7fa451444700 1 Processor -- start 2023-09-30T22:30:25.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.571+0000 7fa451444700 1 -- start start 2023-09-30T22:30:25.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.572+0000 7fa451444700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa44c09eca0 0x7fa44c13cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:25.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.572+0000 7fa451444700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa44c06f050 0x7fa44c13f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:25.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.572+0000 7fa451444700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa44c070ee0 con 0x7fa44c06e570 2023-09-30T22:30:25.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.572+0000 7fa451444700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa44c071980 con 0x7fa44c09eca0 2023-09-30T22:30:25.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.572+0000 7fa451444700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa44c071480 con 0x7fa44c06f050 2023-09-30T22:30:25.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.572+0000 7fa44b7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa44c06f050 0x7fa44c13f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:25.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.572+0000 7fa44b7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa44c06f050 0x7fa44c13f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:34432/0 (socket says 172.21.15.16:34432) 2023-09-30T22:30:25.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.572+0000 7fa44b7fe700 1 -- 172.21.15.16:0/2604470685 learned_addr learned my addr 172.21.15.16:0/2604470685 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:25.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.572+0000 7fa44b7fe700 1 -- 172.21.15.16:0/2604470685 >> v2:172.21.15.16:3300/0 conn(0x7fa44c09eca0 msgr2=0x7fa44c13cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:25.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.572+0000 7fa44b7fe700 1 --2- 172.21.15.16:0/2604470685 >> v2:172.21.15.16:3300/0 conn(0x7fa44c09eca0 0x7fa44c13cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.572+0000 7fa44b7fe700 1 -- 172.21.15.16:0/2604470685 >> v1:172.21.15.16:6789/0 conn(0x7fa44c06e570 legacy=0x7fa44c06e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:25.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.572+0000 7fa44b7fe700 1 -- 172.21.15.16:0/2604470685 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa44c071b90 con 0x7fa44c06f050 2023-09-30T22:30:25.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.573+0000 7fa44b7fe700 1 --2- 172.21.15.16:0/2604470685 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa44c06f050 0x7fa44c13f3f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa444009e50 tx=0x7fa44400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d9b81a9224d9785 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:25.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.573+0000 7fa449ffb700 1 -- 172.21.15.16:0/2604470685 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa44400e070 con 0x7fa44c06f050 2023-09-30T22:30:25.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.573+0000 7fa449ffb700 1 -- 172.21.15.16:0/2604470685 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa444003a20 con 0x7fa44c06f050 2023-09-30T22:30:25.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.573+0000 7fa449ffb700 1 -- 172.21.15.16:0/2604470685 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa444003dd0 con 0x7fa44c06f050 2023-09-30T22:30:25.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.574+0000 7fa451444700 1 -- 172.21.15.16:0/2604470685 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa44c06f050 msgr2=0x7fa44c13f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:25.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.574+0000 7fa451444700 1 --2- 172.21.15.16:0/2604470685 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa44c06f050 0x7fa44c13f3f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa444009e50 tx=0x7fa44400b330 comp rx=0 tx=0).stop 2023-09-30T22:30:25.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.574+0000 7fa451444700 1 -- 172.21.15.16:0/2604470685 shutdown_connections 2023-09-30T22:30:25.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.574+0000 7fa451444700 1 --2- 172.21.15.16:0/2604470685 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa44c06f050 0x7fa44c13f3f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.574+0000 7fa451444700 1 --2- 172.21.15.16:0/2604470685 >> v2:172.21.15.16:3300/0 conn(0x7fa44c09eca0 0x7fa44c13cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.574+0000 7fa451444700 1 -- 172.21.15.16:0/2604470685 >> 172.21.15.16:0/2604470685 conn(0x7fa44c0a2aa0 msgr2=0x7fa44c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:25.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.574+0000 7fa451444700 1 -- 172.21.15.16:0/2604470685 shutdown_connections 2023-09-30T22:30:25.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.574+0000 7fa451444700 1 -- 172.21.15.16:0/2604470685 wait complete. 2023-09-30T22:30:25.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.575+0000 7fa451444700 1 Processor -- start 2023-09-30T22:30:25.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.575+0000 7fa451444700 1 -- start start 2023-09-30T22:30:25.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.575+0000 7fa451444700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa44c06e570 0x7fa44c13cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:25.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.575+0000 7fa451444700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa44c06f050 0x7fa44c13d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:25.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.575+0000 7fa451444700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa44c12e210 0x7fa44c13d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:25.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.575+0000 7fa451444700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa44c13ea50 con 0x7fa44c06e570 2023-09-30T22:30:25.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.575+0000 7fa451444700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa44c12dd10 con 0x7fa44c12e210 2023-09-30T22:30:25.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.575+0000 7fa451444700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa44c070680 con 0x7fa44c06f050 2023-09-30T22:30:25.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.576+0000 7fa44b7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa44c12e210 0x7fa44c13d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:25.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.576+0000 7fa44b7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa44c12e210 0x7fa44c13d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:34446/0 (socket says 172.21.15.16:34446) 2023-09-30T22:30:25.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.576+0000 7fa44b7fe700 1 -- 172.21.15.16:0/1616763264 learned_addr learned my addr 172.21.15.16:0/1616763264 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:25.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.576+0000 7fa44affd700 1 --2- 172.21.15.16:0/1616763264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa44c06e570 0x7fa44c13cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:25.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.576+0000 7fa44a7fc700 1 --2- 172.21.15.16:0/1616763264 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa44c06f050 0x7fa44c13d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:25.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.576+0000 7fa44b7fe700 1 -- 172.21.15.16:0/1616763264 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa44c06f050 msgr2=0x7fa44c13d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:25.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.576+0000 7fa44b7fe700 1 --2- 172.21.15.16:0/1616763264 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa44c06f050 0x7fa44c13d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.576+0000 7fa44b7fe700 1 -- 172.21.15.16:0/1616763264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa44c06e570 msgr2=0x7fa44c13cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:25.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.576+0000 7fa44b7fe700 1 --2- 172.21.15.16:0/1616763264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa44c06e570 0x7fa44c13cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.576+0000 7fa44b7fe700 1 -- 172.21.15.16:0/1616763264 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa444009b30 con 0x7fa44c12e210 2023-09-30T22:30:25.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.577+0000 7fa44b7fe700 1 --2- 172.21.15.16:0/1616763264 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa44c12e210 0x7fa44c13d820 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa44c071ac0 tx=0x7fa444004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:25.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.577+0000 7fa4337fe700 1 -- 172.21.15.16:0/1616763264 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa444004240 con 0x7fa44c12e210 2023-09-30T22:30:25.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.577+0000 7fa4337fe700 1 -- 172.21.15.16:0/1616763264 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4440043c0 con 0x7fa44c12e210 2023-09-30T22:30:25.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.577+0000 7fa4337fe700 1 -- 172.21.15.16:0/1616763264 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa444004770 con 0x7fa44c12e210 2023-09-30T22:30:25.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.577+0000 7fa451444700 1 -- 172.21.15.16:0/1616763264 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa44c12c3f0 con 0x7fa44c12e210 2023-09-30T22:30:25.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.577+0000 7fa451444700 1 -- 172.21.15.16:0/1616763264 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa44c13ec60 con 0x7fa44c12e210 2023-09-30T22:30:25.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.578+0000 7fa451444700 1 -- 172.21.15.16:0/1616763264 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa44c12c710 con 0x7fa44c12e210 2023-09-30T22:30:25.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.578+0000 7fa4337fe700 1 -- 172.21.15.16:0/1616763264 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa444019070 con 0x7fa44c12e210 2023-09-30T22:30:25.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.578+0000 7fa4337fe700 1 -- 172.21.15.16:0/1616763264 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fa444040590 con 0x7fa44c12e210 2023-09-30T22:30:25.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.588+0000 7fa4337fe700 1 -- 172.21.15.16:0/1616763264 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa444007310 con 0x7fa44c12e210 2023-09-30T22:30:25.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.640+0000 7f191203d700 1 Processor -- start 2023-09-30T22:30:25.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.640+0000 7f191203d700 1 -- start start 2023-09-30T22:30:25.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.641+0000 7f191203d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f190c09e930 0x7f190c09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:25.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.641+0000 7f191203d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f190c09f240 0x7f190c1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:25.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.641+0000 7f191203d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f190c070940 con 0x7f190c09f650 2023-09-30T22:30:25.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.641+0000 7f191203d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f190c06fd60 con 0x7f190c09e930 2023-09-30T22:30:25.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.641+0000 7f191203d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f190c06fa80 con 0x7f190c09f240 2023-09-30T22:30:25.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.641+0000 7f190affd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f190c09f240 0x7f190c1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:25.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.641+0000 7f190affd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f190c09f240 0x7f190c1317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:59050/0 (socket says 172.21.15.16:59050) 2023-09-30T22:30:25.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.641+0000 7f190b7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f190c09e930 0x7f190c09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:25.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.641+0000 7f190affd700 1 -- 172.21.15.16:0/2586393851 learned_addr learned my addr 172.21.15.16:0/2586393851 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:25.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.641+0000 7f190bfff700 1 --1- 172.21.15.16:0/2586393851 >> v1:172.21.15.16:6789/0 conn(0x7f190c09f650 0x7f190c131d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57054/0 (socket says 172.21.15.16:57054) 2023-09-30T22:30:25.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.642+0000 7f190affd700 1 -- 172.21.15.16:0/2586393851 >> v2:172.21.15.16:3300/0 conn(0x7f190c09e930 msgr2=0x7f190c09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:25.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.642+0000 7f190affd700 1 --2- 172.21.15.16:0/2586393851 >> v2:172.21.15.16:3300/0 conn(0x7f190c09e930 0x7f190c09ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.642+0000 7f190affd700 1 -- 172.21.15.16:0/2586393851 >> v1:172.21.15.16:6789/0 conn(0x7f190c09f650 legacy=0x7f190c131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:25.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.642+0000 7f190affd700 1 -- 172.21.15.16:0/2586393851 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f190c09fb60 con 0x7f190c09f240 2023-09-30T22:30:25.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.642+0000 7f190affd700 1 --2- 172.21.15.16:0/2586393851 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f190c09f240 0x7f190c1317f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f19000098f0 tx=0x7f1900009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8d431ebe2e5f32e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:25.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.642+0000 7f190a7fc700 1 -- 172.21.15.16:0/2586393851 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1900011830 con 0x7f190c09f240 2023-09-30T22:30:25.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.642+0000 7f190a7fc700 1 -- 172.21.15.16:0/2586393851 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1900011990 con 0x7f190c09f240 2023-09-30T22:30:25.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.642+0000 7f190a7fc700 1 -- 172.21.15.16:0/2586393851 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1900011d20 con 0x7f190c09f240 2023-09-30T22:30:25.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.643+0000 7f191203d700 1 -- 172.21.15.16:0/2586393851 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f190c09f240 msgr2=0x7f190c1317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:25.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.643+0000 7f191203d700 1 --2- 172.21.15.16:0/2586393851 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f190c09f240 0x7f190c1317f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f19000098f0 tx=0x7f1900009c00 comp rx=0 tx=0).stop 2023-09-30T22:30:25.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.643+0000 7f191203d700 1 -- 172.21.15.16:0/2586393851 shutdown_connections 2023-09-30T22:30:25.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.643+0000 7f191203d700 1 --2- 172.21.15.16:0/2586393851 >> v2:172.21.15.16:3300/0 conn(0x7f190c09e930 0x7f190c09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.643+0000 7f191203d700 1 --2- 172.21.15.16:0/2586393851 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f190c09f240 0x7f190c1317f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.643+0000 7f191203d700 1 -- 172.21.15.16:0/2586393851 >> 172.21.15.16:0/2586393851 conn(0x7f190c0a4210 msgr2=0x7f190c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:25.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.643+0000 7f191203d700 1 -- 172.21.15.16:0/2586393851 shutdown_connections 2023-09-30T22:30:25.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.643+0000 7f191203d700 1 -- 172.21.15.16:0/2586393851 wait complete. 2023-09-30T22:30:25.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.644+0000 7f191203d700 1 Processor -- start 2023-09-30T22:30:25.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.644+0000 7f191203d700 1 -- start start 2023-09-30T22:30:25.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.645+0000 7f191203d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f190c09e930 0x7f190c06b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:25.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.645+0000 7f191203d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f190c09f240 0x7f190c06bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:25.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.645+0000 7f191203d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f190c09f650 0x7f190c06c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:25.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.645+0000 7f191203d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f190c1b7280 con 0x7f190c09f650 2023-09-30T22:30:25.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.645+0000 7f191203d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f190c1a9430 con 0x7f190c09e930 2023-09-30T22:30:25.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.645+0000 7f191203d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f190c1a8950 con 0x7f190c09f240 2023-09-30T22:30:25.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.645+0000 7f190b7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f190c09e930 0x7f190c06b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:25.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.645+0000 7f190bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f190c09f650 0x7f190c06c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:25.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.645+0000 7f190b7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f190c09e930 0x7f190c06b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:34458/0 (socket says 172.21.15.16:34458) 2023-09-30T22:30:25.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.645+0000 7f190b7fe700 1 -- 172.21.15.16:0/1281826812 learned_addr learned my addr 172.21.15.16:0/1281826812 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:25.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.645+0000 7f190affd700 1 --2- 172.21.15.16:0/1281826812 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f190c09f240 0x7f190c06bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:25.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.646+0000 7f190b7fe700 1 -- 172.21.15.16:0/1281826812 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f190c09f240 msgr2=0x7f190c06bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:25.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.646+0000 7f190b7fe700 1 --2- 172.21.15.16:0/1281826812 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f190c09f240 0x7f190c06bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.646+0000 7f190b7fe700 1 -- 172.21.15.16:0/1281826812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f190c09f650 msgr2=0x7f190c06c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:25.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.646+0000 7f190b7fe700 1 --2- 172.21.15.16:0/1281826812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f190c09f650 0x7f190c06c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.646+0000 7f190b7fe700 1 -- 172.21.15.16:0/1281826812 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1900009550 con 0x7f190c09e930 2023-09-30T22:30:25.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.646+0000 7f190b7fe700 1 --2- 172.21.15.16:0/1281826812 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f190c09e930 0x7f190c06b8b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f18fc00e9e0 tx=0x7f18fc00ecf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:25.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.646+0000 7f18f3fff700 1 -- 172.21.15.16:0/1281826812 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18fc009710 con 0x7f190c09e930 2023-09-30T22:30:25.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.646+0000 7f18f3fff700 1 -- 172.21.15.16:0/1281826812 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18fc00c850 con 0x7f190c09e930 2023-09-30T22:30:25.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.646+0000 7f191203d700 1 -- 172.21.15.16:0/1281826812 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f190c0a0190 con 0x7f190c09e930 2023-09-30T22:30:25.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.646+0000 7f18f3fff700 1 -- 172.21.15.16:0/1281826812 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18fc00cbc0 con 0x7f190c09e930 2023-09-30T22:30:25.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.646+0000 7f191203d700 1 -- 172.21.15.16:0/1281826812 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f190c071d20 con 0x7f190c09e930 2023-09-30T22:30:25.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.648+0000 7f191203d700 1 -- 172.21.15.16:0/1281826812 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f190c1a0b30 con 0x7f190c09e930 2023-09-30T22:30:25.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.648+0000 7f18f3fff700 1 -- 172.21.15.16:0/1281826812 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f18fc00cd20 con 0x7f190c09e930 2023-09-30T22:30:25.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.648+0000 7f18f3fff700 1 -- 172.21.15.16:0/1281826812 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f18fc03f860 con 0x7f190c09e930 2023-09-30T22:30:25.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.657+0000 7f18f3fff700 1 -- 172.21.15.16:0/1281826812 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f18fc01be10 con 0x7f190c09e930 2023-09-30T22:30:25.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.681+0000 7f85dec69700 1 Processor -- start 2023-09-30T22:30:25.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.681+0000 7f85dec69700 1 -- start start 2023-09-30T22:30:25.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.682+0000 7f85dec69700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f85d8133a80 0x7f85d8133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:25.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.682+0000 7f85dec69700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85d8130a70 0x7f85d813b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:25.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.682+0000 7f85dec69700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85d806bc00 con 0x7f85d812ff20 2023-09-30T22:30:25.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.682+0000 7f85dec69700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85d806c570 con 0x7f85d8133a80 2023-09-30T22:30:25.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.682+0000 7f85dec69700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85d80a0040 con 0x7f85d8130a70 2023-09-30T22:30:25.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.682+0000 7f85dd206700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85d8130a70 0x7f85d813b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:25.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.682+0000 7f85d7fff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f85d812ff20 0x7f85d8130360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57058/0 (socket says 172.21.15.16:57058) 2023-09-30T22:30:25.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.682+0000 7f85d7fff700 1 -- 172.21.15.16:0/3607514443 learned_addr learned my addr 172.21.15.16:0/3607514443 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:30:25.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.682+0000 7f85d77fe700 1 -- 172.21.15.16:0/3607514443 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1087912026 0 0) 0x7f85d806bc00 con 0x7f85d812ff20 2023-09-30T22:30:25.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.682+0000 7f85dd206700 1 -- 172.21.15.16:0/3607514443 >> v2:172.21.15.16:3300/0 conn(0x7f85d8133a80 msgr2=0x7f85d8133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:25.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.682+0000 7f85dd206700 1 --2- 172.21.15.16:0/3607514443 >> v2:172.21.15.16:3300/0 conn(0x7f85d8133a80 0x7f85d8133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.682+0000 7f85dd206700 1 -- 172.21.15.16:0/3607514443 >> v1:172.21.15.16:6789/0 conn(0x7f85d812ff20 legacy=0x7f85d8130360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:25.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.682+0000 7f85dd206700 1 -- 172.21.15.16:0/3607514443 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85d8133860 con 0x7f85d8130a70 2023-09-30T22:30:25.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.683+0000 7f85dd206700 1 --2- 172.21.15.16:0/3607514443 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85d8130a70 0x7f85d813b850 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f85d0009d90 tx=0x7f85d000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e69f2c261dc1159 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:25.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.683+0000 7f85d77fe700 1 -- 172.21.15.16:0/3607514443 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d000e070 con 0x7f85d8130a70 2023-09-30T22:30:25.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.683+0000 7f85d77fe700 1 -- 172.21.15.16:0/3607514443 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85d000bbd0 con 0x7f85d8130a70 2023-09-30T22:30:25.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.683+0000 7f85d77fe700 1 -- 172.21.15.16:0/3607514443 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85d0012430 con 0x7f85d8130a70 2023-09-30T22:30:25.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.684+0000 7f85dec69700 1 -- 172.21.15.16:0/3607514443 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85d8130a70 msgr2=0x7f85d813b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:25.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.684+0000 7f85dec69700 1 --2- 172.21.15.16:0/3607514443 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85d8130a70 0x7f85d813b850 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f85d0009d90 tx=0x7f85d000b330 comp rx=0 tx=0).stop 2023-09-30T22:30:25.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.684+0000 7f85dec69700 1 -- 172.21.15.16:0/3607514443 shutdown_connections 2023-09-30T22:30:25.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.684+0000 7f85dec69700 1 --2- 172.21.15.16:0/3607514443 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85d8130a70 0x7f85d813b850 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.684+0000 7f85dec69700 1 --2- 172.21.15.16:0/3607514443 >> v2:172.21.15.16:3300/0 conn(0x7f85d8133a80 0x7f85d8133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.684+0000 7f85dec69700 1 -- 172.21.15.16:0/3607514443 >> 172.21.15.16:0/3607514443 conn(0x7f85d80a4210 msgr2=0x7f85d80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:25.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.684+0000 7f85dec69700 1 -- 172.21.15.16:0/3607514443 shutdown_connections 2023-09-30T22:30:25.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.684+0000 7f85dec69700 1 -- 172.21.15.16:0/3607514443 wait complete. 2023-09-30T22:30:25.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.685+0000 7f70abe81700 1 Processor -- start 2023-09-30T22:30:25.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.685+0000 7f85dec69700 1 Processor -- start 2023-09-30T22:30:25.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.685+0000 7f85dec69700 1 -- start start 2023-09-30T22:30:25.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.686+0000 7f85dec69700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85d812ff20 0x7f85d809c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:25.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.686+0000 7f85dec69700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85d8130a70 0x7f85d809c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:25.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.686+0000 7f85dec69700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85d8133a80 0x7f85d809cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:25.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.686+0000 7f85dec69700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d8120210 con 0x7f85d812ff20 2023-09-30T22:30:25.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.686+0000 7f85dec69700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85d80a20c0 con 0x7f85d8130a70 2023-09-30T22:30:25.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.686+0000 7f85dec69700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f85d811f0a0 con 0x7f85d8133a80 2023-09-30T22:30:25.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.685+0000 7f70abe81700 1 -- start start 2023-09-30T22:30:25.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.686+0000 7f70abe81700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f70a409d6f0 0x7f70a409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:25.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.686+0000 7f70abe81700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f70a409e410 0x7f70a4130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:25.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.686+0000 7f70abe81700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70a409f130 con 0x7f70a409e000 2023-09-30T22:30:25.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.686+0000 7f70abe81700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70a409f520 con 0x7f70a409d6f0 2023-09-30T22:30:25.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.686+0000 7f70abe81700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f70a409e930 con 0x7f70a409e410 2023-09-30T22:30:25.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.686+0000 7f70aa41e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f70a409e410 0x7f70a4130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:25.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.686+0000 7f70aa41e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f70a409e410 0x7f70a4130d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:59070/0 (socket says 172.21.15.16:59070) 2023-09-30T22:30:25.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.686+0000 7f70aa41e700 1 -- 172.21.15.16:0/797421907 learned_addr learned my addr 172.21.15.16:0/797421907 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:25.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.686+0000 7f85dca05700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85d812ff20 0x7f85d809c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:25.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.686+0000 7f85dca05700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85d812ff20 0x7f85d809c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58224/0 (socket says 172.21.15.16:58224) 2023-09-30T22:30:25.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.686+0000 7f85dca05700 1 -- 172.21.15.16:0/1486425902 learned_addr learned my addr 172.21.15.16:0/1486425902 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:25.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.686+0000 7f85d7fff700 1 --2- 172.21.15.16:0/1486425902 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85d8130a70 0x7f85d809c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:25.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.686+0000 7f85dd206700 1 --2- 172.21.15.16:0/1486425902 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85d8133a80 0x7f85d809cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:25.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.686+0000 7f70a9c1d700 1 --2- 172.21.15.16:0/797421907 >> v2:172.21.15.16:3300/0 conn(0x7f70a409d6f0 0x7f70a409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:25.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.687+0000 7f70a9c1d700 1 -- 172.21.15.16:0/797421907 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f70a409e410 msgr2=0x7f70a4130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:25.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.687+0000 7f70a9c1d700 1 --2- 172.21.15.16:0/797421907 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f70a409e410 0x7f70a4130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.687+0000 7f70a9c1d700 1 -- 172.21.15.16:0/797421907 >> v1:172.21.15.16:6789/0 conn(0x7f70a409e000 legacy=0x7f70a4130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:25.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.687+0000 7f70a9c1d700 1 -- 172.21.15.16:0/797421907 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a409b2d0 con 0x7f70a409d6f0 2023-09-30T22:30:25.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.687+0000 7f85dca05700 1 -- 172.21.15.16:0/1486425902 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85d8133a80 msgr2=0x7f85d809cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:25.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.687+0000 7f85dca05700 1 --2- 172.21.15.16:0/1486425902 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85d8133a80 0x7f85d809cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.687+0000 7f85dca05700 1 -- 172.21.15.16:0/1486425902 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85d8130a70 msgr2=0x7f85d809c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:25.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.687+0000 7f85dca05700 1 --2- 172.21.15.16:0/1486425902 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85d8130a70 0x7f85d809c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.687+0000 7f85dca05700 1 -- 172.21.15.16:0/1486425902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85d0009a70 con 0x7f85d812ff20 2023-09-30T22:30:25.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.687+0000 7f85dd206700 1 --2- 172.21.15.16:0/1486425902 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85d8133a80 0x7f85d809cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:30:25.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.687+0000 7f70a9c1d700 1 --2- 172.21.15.16:0/797421907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70a409d6f0 0x7f70a409dac0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f70a000bae0 tx=0x7f70a000bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c93ec4584aa1285a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:25.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.687+0000 7f85dca05700 1 --2- 172.21.15.16:0/1486425902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85d812ff20 0x7f85d809c480 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f85cc00b560 tx=0x7f85cc00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:25.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.687+0000 7f85d57fa700 1 -- 172.21.15.16:0/1486425902 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85cc011830 con 0x7f85d812ff20 2023-09-30T22:30:25.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.687+0000 7f85dec69700 1 -- 172.21.15.16:0/1486425902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85d809df50 con 0x7f85d812ff20 2023-09-30T22:30:25.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.687+0000 7f70a8c1b700 1 -- 172.21.15.16:0/797421907 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a000dc70 con 0x7f70a409d6f0 2023-09-30T22:30:25.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.687+0000 7f70a8c1b700 1 -- 172.21.15.16:0/797421907 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70a000ddd0 con 0x7f70a409d6f0 2023-09-30T22:30:25.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.687+0000 7f70a8c1b700 1 -- 172.21.15.16:0/797421907 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a0010670 con 0x7f70a409d6f0 2023-09-30T22:30:25.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.687+0000 7f85d57fa700 1 -- 172.21.15.16:0/1486425902 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85cc0119b0 con 0x7f85d812ff20 2023-09-30T22:30:25.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.687+0000 7f85d57fa700 1 -- 172.21.15.16:0/1486425902 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85cc011d60 con 0x7f85d812ff20 2023-09-30T22:30:25.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.687+0000 7f85dec69700 1 -- 172.21.15.16:0/1486425902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85d811fe90 con 0x7f85d812ff20 2023-09-30T22:30:25.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.688+0000 7f70abe81700 1 -- 172.21.15.16:0/797421907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70a409d6f0 msgr2=0x7f70a409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:25.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.688+0000 7f70abe81700 1 --2- 172.21.15.16:0/797421907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70a409d6f0 0x7f70a409dac0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f70a000bae0 tx=0x7f70a000bdf0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.688+0000 7f85dec69700 1 -- 172.21.15.16:0/1486425902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85d8074ea0 con 0x7f85d812ff20 2023-09-30T22:30:25.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.689+0000 7f70abe81700 1 -- 172.21.15.16:0/797421907 shutdown_connections 2023-09-30T22:30:25.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.689+0000 7f70abe81700 1 --2- 172.21.15.16:0/797421907 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f70a409e410 0x7f70a4130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.689+0000 7f70abe81700 1 --2- 172.21.15.16:0/797421907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70a409d6f0 0x7f70a409dac0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.689+0000 7f70abe81700 1 -- 172.21.15.16:0/797421907 >> 172.21.15.16:0/797421907 conn(0x7f70a4066f50 msgr2=0x7f70a409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:25.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.689+0000 7f85d57fa700 1 -- 172.21.15.16:0/1486425902 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f85cc01a400 con 0x7f85d812ff20 2023-09-30T22:30:25.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.689+0000 7f85d57fa700 1 -- 172.21.15.16:0/1486425902 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f85cc03f7e0 con 0x7f85d812ff20 2023-09-30T22:30:25.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.690+0000 7f70abe81700 1 -- 172.21.15.16:0/797421907 shutdown_connections 2023-09-30T22:30:25.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.690+0000 7f70abe81700 1 -- 172.21.15.16:0/797421907 wait complete. 2023-09-30T22:30:25.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.691+0000 7f70abe81700 1 Processor -- start 2023-09-30T22:30:25.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.691+0000 7f70abe81700 1 -- start start 2023-09-30T22:30:25.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.692+0000 7f70abe81700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f70a409d6f0 0x7f70a41223c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:25.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.692+0000 7f70abe81700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f70a409e000 0x7f70a4122900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:25.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.692+0000 7f70abe81700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70a409e410 0x7f70a4122e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:25.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.692+0000 7f70abe81700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a4125ee0 con 0x7f70a409e410 2023-09-30T22:30:25.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.692+0000 7f70abe81700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70a41302e0 con 0x7f70a409d6f0 2023-09-30T22:30:25.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.692+0000 7f70abe81700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f70a4123fd0 con 0x7f70a409e000 2023-09-30T22:30:25.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.692+0000 7f70aa41e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70a409e410 0x7f70a4122e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:25.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.692+0000 7f70a9c1d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f70a409d6f0 0x7f70a41223c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:25.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.692+0000 7f70a9c1d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f70a409d6f0 0x7f70a41223c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:34472/0 (socket says 172.21.15.16:34472) 2023-09-30T22:30:25.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.692+0000 7f70aa41e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70a409e410 0x7f70a4122e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58242/0 (socket says 172.21.15.16:58242) 2023-09-30T22:30:25.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.692+0000 7f70a9c1d700 1 -- 172.21.15.16:0/2595250095 learned_addr learned my addr 172.21.15.16:0/2595250095 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:25.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.692+0000 7f70a941c700 1 --2- 172.21.15.16:0/2595250095 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f70a409e000 0x7f70a4122900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:25.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.692+0000 7f70aa41e700 1 -- 172.21.15.16:0/2595250095 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f70a409e000 msgr2=0x7f70a4122900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:25.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.692+0000 7f70aa41e700 1 --2- 172.21.15.16:0/2595250095 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f70a409e000 0x7f70a4122900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.692+0000 7f70aa41e700 1 -- 172.21.15.16:0/2595250095 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f70a409d6f0 msgr2=0x7f70a41223c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:25.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.693+0000 7f70aa41e700 1 --2- 172.21.15.16:0/2595250095 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f70a409d6f0 0x7f70a41223c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.693+0000 7f70aa41e700 1 -- 172.21.15.16:0/2595250095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a000b740 con 0x7f70a409e410 2023-09-30T22:30:25.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.693+0000 7f70a941c700 1 --2- 172.21.15.16:0/2595250095 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f70a409e000 0x7f70a4122900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:30:25.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.693+0000 7f70a9c1d700 1 --2- 172.21.15.16:0/2595250095 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f70a409d6f0 0x7f70a41223c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:25.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.693+0000 7f70aa41e700 1 --2- 172.21.15.16:0/2595250095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70a409e410 0x7f70a4122e40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f709c00efc0 tx=0x7f709c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:25.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.698+0000 7f709a7fc700 1 -- 172.21.15.16:0/2595250095 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709c003c20 con 0x7f70a409e410 2023-09-30T22:30:25.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.698+0000 7f709a7fc700 1 -- 172.21.15.16:0/2595250095 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f709c00f040 con 0x7f70a409e410 2023-09-30T22:30:25.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.698+0000 7f70abe81700 1 -- 172.21.15.16:0/2595250095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70a40a1d90 con 0x7f70a409e410 2023-09-30T22:30:25.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.698+0000 7f709a7fc700 1 -- 172.21.15.16:0/2595250095 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709c009e60 con 0x7f70a409e410 2023-09-30T22:30:25.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.698+0000 7f85d57fa700 1 -- 172.21.15.16:0/1486425902 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f85cc01ae60 con 0x7f85d812ff20 2023-09-30T22:30:25.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.698+0000 7f70abe81700 1 -- 172.21.15.16:0/2595250095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70a4126670 con 0x7f70a409e410 2023-09-30T22:30:25.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.699+0000 7f70abe81700 1 -- 172.21.15.16:0/2595250095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70a4125a60 con 0x7f70a409e410 2023-09-30T22:30:25.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.699+0000 7f709a7fc700 1 -- 172.21.15.16:0/2595250095 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f709c00c930 con 0x7f70a409e410 2023-09-30T22:30:25.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.699+0000 7f709a7fc700 1 -- 172.21.15.16:0/2595250095 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f709c03f9c0 con 0x7f70a409e410 2023-09-30T22:30:25.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.704+0000 7f709a7fc700 1 -- 172.21.15.16:0/2595250095 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f70a4125a60 con 0x7f70a409e410 2023-09-30T22:30:25.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.820+0000 7fa451444700 1 -- 172.21.15.16:0/1616763264 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa44c074ea0 con 0x7fa44c12e210 2023-09-30T22:30:25.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.821+0000 7fa4337fe700 1 -- 172.21.15.16:0/1616763264 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa444017020 con 0x7fa44c12e210 2023-09-30T22:30:25.821 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:30:25.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.823+0000 7fa451444700 1 -- 172.21.15.16:0/1616763264 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa44c12e210 msgr2=0x7fa44c13d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:25.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.823+0000 7fa451444700 1 --2- 172.21.15.16:0/1616763264 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa44c12e210 0x7fa44c13d820 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa44c071ac0 tx=0x7fa444004ad0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.823+0000 7fa451444700 1 -- 172.21.15.16:0/1616763264 shutdown_connections 2023-09-30T22:30:25.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.823+0000 7fa451444700 1 --2- 172.21.15.16:0/1616763264 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa44c12e210 0x7fa44c13d820 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.823+0000 7fa451444700 1 --2- 172.21.15.16:0/1616763264 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa44c06e570 0x7fa44c13cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.823+0000 7fa451444700 1 --2- 172.21.15.16:0/1616763264 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa44c06f050 0x7fa44c13d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.823+0000 7fa451444700 1 -- 172.21.15.16:0/1616763264 >> 172.21.15.16:0/1616763264 conn(0x7fa44c0a2aa0 msgr2=0x7fa44c09cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:25.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.823+0000 7fa451444700 1 -- 172.21.15.16:0/1616763264 shutdown_connections 2023-09-30T22:30:25.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.823+0000 7fa451444700 1 -- 172.21.15.16:0/1616763264 wait complete. 2023-09-30T22:30:25.838 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:30:25.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.876+0000 7f191203d700 1 -- 172.21.15.16:0/1281826812 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f190c0a3460 con 0x7f190c09e930 2023-09-30T22:30:25.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.876+0000 7f18f3fff700 1 -- 172.21.15.16:0/1281826812 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f18fc01b3b0 con 0x7f190c09e930 2023-09-30T22:30:25.876 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:30:25.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.878+0000 7f191203d700 1 -- 172.21.15.16:0/1281826812 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f190c09e930 msgr2=0x7f190c06b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:25.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.878+0000 7f191203d700 1 --2- 172.21.15.16:0/1281826812 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f190c09e930 0x7f190c06b8b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f18fc00e9e0 tx=0x7f18fc00ecf0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.878+0000 7f191203d700 1 -- 172.21.15.16:0/1281826812 shutdown_connections 2023-09-30T22:30:25.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.878+0000 7f191203d700 1 --2- 172.21.15.16:0/1281826812 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f190c09e930 0x7f190c06b8b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.878+0000 7f191203d700 1 --2- 172.21.15.16:0/1281826812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f190c09f650 0x7f190c06c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.878+0000 7f191203d700 1 --2- 172.21.15.16:0/1281826812 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f190c09f240 0x7f190c06bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.878+0000 7f191203d700 1 -- 172.21.15.16:0/1281826812 >> 172.21.15.16:0/1281826812 conn(0x7f190c0a4210 msgr2=0x7f190c1b8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:25.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.879+0000 7f191203d700 1 -- 172.21.15.16:0/1281826812 shutdown_connections 2023-09-30T22:30:25.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.880+0000 7f191203d700 1 -- 172.21.15.16:0/1281826812 wait complete. 2023-09-30T22:30:25.894 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:30:25.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.907+0000 7f85dec69700 1 -- 172.21.15.16:0/1486425902 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f85d806b250 con 0x7f85d812ff20 2023-09-30T22:30:25.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.907+0000 7f85d57fa700 1 -- 172.21.15.16:0/1486425902 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f85cc015020 con 0x7f85d812ff20 2023-09-30T22:30:25.908 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:30:25.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.909+0000 7f85dec69700 1 -- 172.21.15.16:0/1486425902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85d812ff20 msgr2=0x7f85d809c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:25.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.909+0000 7f85dec69700 1 --2- 172.21.15.16:0/1486425902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85d812ff20 0x7f85d809c480 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f85cc00b560 tx=0x7f85cc00b870 comp rx=0 tx=0).stop 2023-09-30T22:30:25.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.910+0000 7f85dec69700 1 -- 172.21.15.16:0/1486425902 shutdown_connections 2023-09-30T22:30:25.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.910+0000 7f85dec69700 1 --2- 172.21.15.16:0/1486425902 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85d8133a80 0x7f85d809cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.910+0000 7f85dec69700 1 --2- 172.21.15.16:0/1486425902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85d812ff20 0x7f85d809c480 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.910+0000 7f85dec69700 1 --2- 172.21.15.16:0/1486425902 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85d8130a70 0x7f85d809c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.910+0000 7f85dec69700 1 -- 172.21.15.16:0/1486425902 >> 172.21.15.16:0/1486425902 conn(0x7f85d80a4210 msgr2=0x7f85d809e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:25.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.910+0000 7f85dec69700 1 -- 172.21.15.16:0/1486425902 shutdown_connections 2023-09-30T22:30:25.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.910+0000 7f85dec69700 1 -- 172.21.15.16:0/1486425902 wait complete. 2023-09-30T22:30:25.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.916+0000 7f70abe81700 1 -- 172.21.15.16:0/2595250095 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f70a4125a60 con 0x7f70a409e410 2023-09-30T22:30:25.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.916+0000 7f709a7fc700 1 -- 172.21.15.16:0/2595250095 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f70a4125a60 con 0x7f70a409e410 2023-09-30T22:30:25.917 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:30:25.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.919+0000 7f70abe81700 1 -- 172.21.15.16:0/2595250095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70a409e410 msgr2=0x7f70a4122e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:25.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.919+0000 7f70abe81700 1 --2- 172.21.15.16:0/2595250095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70a409e410 0x7f70a4122e40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f709c00efc0 tx=0x7f709c00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.919+0000 7f70abe81700 1 -- 172.21.15.16:0/2595250095 shutdown_connections 2023-09-30T22:30:25.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.919+0000 7f70abe81700 1 --2- 172.21.15.16:0/2595250095 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f70a409d6f0 0x7f70a41223c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.919+0000 7f70abe81700 1 --2- 172.21.15.16:0/2595250095 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70a409e410 0x7f70a4122e40 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.919+0000 7f70abe81700 1 --2- 172.21.15.16:0/2595250095 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f70a409e000 0x7f70a4122900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:25.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.919+0000 7f70abe81700 1 -- 172.21.15.16:0/2595250095 >> 172.21.15.16:0/2595250095 conn(0x7f70a4066f50 msgr2=0x7f70a406f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:25.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.919+0000 7f70abe81700 1 -- 172.21.15.16:0/2595250095 shutdown_connections 2023-09-30T22:30:25.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:25.919+0000 7f70abe81700 1 -- 172.21.15.16:0/2595250095 wait complete. 2023-09-30T22:30:25.924 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:30:25.933 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:30:26.839 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:26.895 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:26.925 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:26.933 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:26.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:26.994+0000 7fc676b63700 1 Processor -- start 2023-09-30T22:30:26.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:26.994+0000 7fc676b63700 1 -- start start 2023-09-30T22:30:26.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:26.995+0000 7fc676b63700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc67009e1d0 0x7fc670135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:26.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:26.995+0000 7fc676b63700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc670135e50 0x7fc670138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:26.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:26.995+0000 7fc676b63700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc67009f130 con 0x7fc67009d6f0 2023-09-30T22:30:26.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:26.995+0000 7fc676b63700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc67009f520 con 0x7fc67009e1d0 2023-09-30T22:30:26.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:26.995+0000 7fc676b63700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc67009e930 con 0x7fc670135e50 2023-09-30T22:30:26.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:26.995+0000 7fc66ffff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc67009e1d0 0x7fc670135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:26.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:26.995+0000 7fc675100700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc670135e50 0x7fc670138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:26.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:26.995+0000 7fc66ffff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc67009e1d0 0x7fc670135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58256/0 (socket says 172.21.15.16:58256) 2023-09-30T22:30:26.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:26.995+0000 7fc675100700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc670135e50 0x7fc670138230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:34474/0 (socket says 172.21.15.16:34474) 2023-09-30T22:30:26.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:26.995+0000 7fc66ffff700 1 -- 172.21.15.16:0/4192096181 learned_addr learned my addr 172.21.15.16:0/4192096181 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:26.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:26.996+0000 7fc675100700 1 -- 172.21.15.16:0/4192096181 >> v2:172.21.15.16:3300/0 conn(0x7fc67009e1d0 msgr2=0x7fc670135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:26.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:26.996+0000 7fc675100700 1 --2- 172.21.15.16:0/4192096181 >> v2:172.21.15.16:3300/0 conn(0x7fc67009e1d0 0x7fc670135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:26.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:26.996+0000 7fc675100700 1 -- 172.21.15.16:0/4192096181 >> v1:172.21.15.16:6789/0 conn(0x7fc67009d6f0 legacy=0x7fc67009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:26.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:26.996+0000 7fc66f7fe700 1 -- 172.21.15.16:0/4192096181 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 403895860 0 0) 0x7fc67009f130 con 0x7fc67009d6f0 2023-09-30T22:30:26.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:26.996+0000 7fc675100700 1 -- 172.21.15.16:0/4192096181 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc67009b2d0 con 0x7fc670135e50 2023-09-30T22:30:26.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:26.996+0000 7fc66ffff700 1 --2- 172.21.15.16:0/4192096181 >> v2:172.21.15.16:3300/0 conn(0x7fc67009e1d0 0x7fc670135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:26.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:26.996+0000 7fc675100700 1 --2- 172.21.15.16:0/4192096181 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc670135e50 0x7fc670138230 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc668009d90 tx=0x7fc66800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0d8d851f562ee3c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:26.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:26.996+0000 7fc66f7fe700 1 -- 172.21.15.16:0/4192096181 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66800e070 con 0x7fc670135e50 2023-09-30T22:30:26.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:26.996+0000 7fc66f7fe700 1 -- 172.21.15.16:0/4192096181 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc66800bb20 con 0x7fc670135e50 2023-09-30T22:30:26.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:26.996+0000 7fc66f7fe700 1 -- 172.21.15.16:0/4192096181 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc668012430 con 0x7fc670135e50 2023-09-30T22:30:26.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:26.997+0000 7fc676b63700 1 -- 172.21.15.16:0/4192096181 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc670135e50 msgr2=0x7fc670138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:26.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:26.997+0000 7fc676b63700 1 --2- 172.21.15.16:0/4192096181 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc670135e50 0x7fc670138230 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fc668009d90 tx=0x7fc66800b330 comp rx=0 tx=0).stop 2023-09-30T22:30:26.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:26.997+0000 7fc676b63700 1 -- 172.21.15.16:0/4192096181 shutdown_connections 2023-09-30T22:30:26.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:26.997+0000 7fc676b63700 1 --2- 172.21.15.16:0/4192096181 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc670135e50 0x7fc670138230 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:26.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:26.997+0000 7fc676b63700 1 --2- 172.21.15.16:0/4192096181 >> v2:172.21.15.16:3300/0 conn(0x7fc67009e1d0 0x7fc670135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:26.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:26.997+0000 7fc676b63700 1 -- 172.21.15.16:0/4192096181 >> 172.21.15.16:0/4192096181 conn(0x7fc670066f50 msgr2=0x7fc67009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:27.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:26.997+0000 7fc676b63700 1 -- 172.21.15.16:0/4192096181 shutdown_connections 2023-09-30T22:30:27.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:26.997+0000 7fc676b63700 1 -- 172.21.15.16:0/4192096181 wait complete. 2023-09-30T22:30:27.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:26.999+0000 7fc676b63700 1 Processor -- start 2023-09-30T22:30:27.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:26.999+0000 7fc676b63700 1 -- start start 2023-09-30T22:30:27.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.000+0000 7fc676b63700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc67009d6f0 0x7fc67011b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:27.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.000+0000 7fc676b63700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc67009e1d0 0x7fc67011b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:27.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.000+0000 7fc676b63700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc670135e50 0x7fc67011bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:27.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.000+0000 7fc676b63700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc67011d4f0 con 0x7fc67009d6f0 2023-09-30T22:30:27.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.000+0000 7fc676b63700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc67011ee00 con 0x7fc67009e1d0 2023-09-30T22:30:27.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.000+0000 7fc676b63700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc670072630 con 0x7fc670135e50 2023-09-30T22:30:27.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.000+0000 7fc6748ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc67009d6f0 0x7fc67011b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:27.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.000+0000 7fc6748ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc67009d6f0 0x7fc67011b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58258/0 (socket says 172.21.15.16:58258) 2023-09-30T22:30:27.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.000+0000 7fc6748ff700 1 -- 172.21.15.16:0/489274251 learned_addr learned my addr 172.21.15.16:0/489274251 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:27.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.000+0000 7fc675100700 1 --2- 172.21.15.16:0/489274251 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc670135e50 0x7fc67011bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:27.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.000+0000 7fc66ffff700 1 --2- 172.21.15.16:0/489274251 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc67009e1d0 0x7fc67011b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:27.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.000+0000 7fc675100700 1 -- 172.21.15.16:0/489274251 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc67009e1d0 msgr2=0x7fc67011b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:27.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.000+0000 7fc675100700 1 --2- 172.21.15.16:0/489274251 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc67009e1d0 0x7fc67011b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.000+0000 7fc675100700 1 -- 172.21.15.16:0/489274251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc67009d6f0 msgr2=0x7fc67011b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:27.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.000+0000 7fc675100700 1 --2- 172.21.15.16:0/489274251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc67009d6f0 0x7fc67011b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.000+0000 7fc675100700 1 -- 172.21.15.16:0/489274251 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc65c009550 con 0x7fc670135e50 2023-09-30T22:30:27.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.001+0000 7fc675100700 1 --2- 172.21.15.16:0/489274251 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc670135e50 0x7fc67011bd60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc67009f6f0 tx=0x7fc668004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:27.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.001+0000 7fc66d7fa700 1 -- 172.21.15.16:0/489274251 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc668012430 con 0x7fc670135e50 2023-09-30T22:30:27.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.001+0000 7fc66d7fa700 1 -- 172.21.15.16:0/489274251 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc668020070 con 0x7fc670135e50 2023-09-30T22:30:27.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.001+0000 7fc66d7fa700 1 -- 172.21.15.16:0/489274251 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc668004040 con 0x7fc670135e50 2023-09-30T22:30:27.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.001+0000 7fc676b63700 1 -- 172.21.15.16:0/489274251 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc668009a70 con 0x7fc670135e50 2023-09-30T22:30:27.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.001+0000 7fc676b63700 1 -- 172.21.15.16:0/489274251 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc67011e860 con 0x7fc670135e50 2023-09-30T22:30:27.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.002+0000 7fc676b63700 1 -- 172.21.15.16:0/489274251 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc670074ea0 con 0x7fc670135e50 2023-09-30T22:30:27.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.003+0000 7fc66d7fa700 1 -- 172.21.15.16:0/489274251 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc6680041a0 con 0x7fc670135e50 2023-09-30T22:30:27.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.003+0000 7fc66d7fa700 1 -- 172.21.15.16:0/489274251 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fc668022070 con 0x7fc670135e50 2023-09-30T22:30:27.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.012+0000 7fc66d7fa700 1 -- 172.21.15.16:0/489274251 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc668004450 con 0x7fc670135e50 2023-09-30T22:30:27.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.052+0000 7fd3688ab700 1 Processor -- start 2023-09-30T22:30:27.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.052+0000 7fd3688ab700 1 -- start start 2023-09-30T22:30:27.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.053+0000 7fd3688ab700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd36409e1d0 0x7fd364135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:27.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.053+0000 7fd3688ab700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd364135e50 0x7fd364138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:27.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.054+0000 7fd3688ab700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd36409f130 con 0x7fd36409d6f0 2023-09-30T22:30:27.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.054+0000 7fd3688ab700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd36409f520 con 0x7fd364135e50 2023-09-30T22:30:27.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.054+0000 7fd3688ab700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd36409e930 con 0x7fd36409e1d0 2023-09-30T22:30:27.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.054+0000 7fd36259c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd36409d6f0 0x7fd36409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57082/0 (socket says 172.21.15.16:57082) 2023-09-30T22:30:27.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.054+0000 7fd36259c700 1 -- 172.21.15.16:0/987837065 learned_addr learned my addr 172.21.15.16:0/987837065 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:30:27.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.054+0000 7fd361d9b700 1 --2- 172.21.15.16:0/987837065 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd36409e1d0 0x7fd364135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:27.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.054+0000 7fd362d9d700 1 --2- 172.21.15.16:0/987837065 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd364135e50 0x7fd364138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:27.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.054+0000 7fd361d9b700 1 -- 172.21.15.16:0/987837065 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd364135e50 msgr2=0x7fd364138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:27.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.054+0000 7fd36159a700 1 -- 172.21.15.16:0/987837065 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2748360324 0 0) 0x7fd36409f130 con 0x7fd36409d6f0 2023-09-30T22:30:27.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.054+0000 7fd361d9b700 1 --2- 172.21.15.16:0/987837065 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd364135e50 0x7fd364138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.054+0000 7fd361d9b700 1 -- 172.21.15.16:0/987837065 >> v1:172.21.15.16:6789/0 conn(0x7fd36409d6f0 legacy=0x7fd36409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:27.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.054+0000 7fd361d9b700 1 -- 172.21.15.16:0/987837065 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd36409b2d0 con 0x7fd36409e1d0 2023-09-30T22:30:27.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.054+0000 7fd361d9b700 1 --2- 172.21.15.16:0/987837065 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd36409e1d0 0x7fd364135910 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd358009870 tx=0x7fd358009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3592b09b8cff82c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:27.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.055+0000 7fd36159a700 1 -- 172.21.15.16:0/987837065 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd358011830 con 0x7fd36409e1d0 2023-09-30T22:30:27.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.055+0000 7fd36159a700 1 -- 172.21.15.16:0/987837065 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd358011990 con 0x7fd36409e1d0 2023-09-30T22:30:27.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.055+0000 7fd36159a700 1 -- 172.21.15.16:0/987837065 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd358011d20 con 0x7fd36409e1d0 2023-09-30T22:30:27.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.055+0000 7fd3688ab700 1 -- 172.21.15.16:0/987837065 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd36409e1d0 msgr2=0x7fd364135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:27.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.055+0000 7fd3688ab700 1 --2- 172.21.15.16:0/987837065 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd36409e1d0 0x7fd364135910 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd358009870 tx=0x7fd358009b80 comp rx=0 tx=0).stop 2023-09-30T22:30:27.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.056+0000 7fd3688ab700 1 -- 172.21.15.16:0/987837065 shutdown_connections 2023-09-30T22:30:27.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.056+0000 7fd3688ab700 1 --2- 172.21.15.16:0/987837065 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd364135e50 0x7fd364138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.056+0000 7fd3688ab700 1 --2- 172.21.15.16:0/987837065 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd36409e1d0 0x7fd364135910 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.056+0000 7fd3688ab700 1 -- 172.21.15.16:0/987837065 >> 172.21.15.16:0/987837065 conn(0x7fd364066f50 msgr2=0x7fd36409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:27.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.056+0000 7fd3688ab700 1 -- 172.21.15.16:0/987837065 shutdown_connections 2023-09-30T22:30:27.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.056+0000 7fd3688ab700 1 -- 172.21.15.16:0/987837065 wait complete. 2023-09-30T22:30:27.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.057+0000 7fd3688ab700 1 Processor -- start 2023-09-30T22:30:27.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.057+0000 7fd3688ab700 1 -- start start 2023-09-30T22:30:27.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.057+0000 7fd3688ab700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd36409d6f0 0x7fd36411bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:27.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.057+0000 7fd3688ab700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd36409e1d0 0x7fd36411c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:27.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.058+0000 7fd3688ab700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd364135e50 0x7fd36411c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:27.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.058+0000 7fd3688ab700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd36411f110 con 0x7fd364135e50 2023-09-30T22:30:27.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.058+0000 7fd3688ab700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd36409a0c0 con 0x7fd36409e1d0 2023-09-30T22:30:27.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.058+0000 7fd3688ab700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd36411ed60 con 0x7fd36409d6f0 2023-09-30T22:30:27.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.058+0000 7fd362d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd364135e50 0x7fd36411c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:27.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.058+0000 7fd362d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd364135e50 0x7fd36411c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58264/0 (socket says 172.21.15.16:58264) 2023-09-30T22:30:27.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.058+0000 7fd361d9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd36409e1d0 0x7fd36411c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:27.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.058+0000 7fd362d9d700 1 -- 172.21.15.16:0/2570637725 learned_addr learned my addr 172.21.15.16:0/2570637725 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:27.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.058+0000 7fd36259c700 1 --2- 172.21.15.16:0/2570637725 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd36409d6f0 0x7fd36411bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:27.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.058+0000 7fd361d9b700 1 -- 172.21.15.16:0/2570637725 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd36409d6f0 msgr2=0x7fd36411bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:27.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.058+0000 7fd361d9b700 1 --2- 172.21.15.16:0/2570637725 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd36409d6f0 0x7fd36411bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.058+0000 7fd361d9b700 1 -- 172.21.15.16:0/2570637725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd364135e50 msgr2=0x7fd36411c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:27.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.058+0000 7fd361d9b700 1 --2- 172.21.15.16:0/2570637725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd364135e50 0x7fd36411c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.058+0000 7fd361d9b700 1 -- 172.21.15.16:0/2570637725 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3580094a0 con 0x7fd36409e1d0 2023-09-30T22:30:27.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.059+0000 7fd361d9b700 1 --2- 172.21.15.16:0/2570637725 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd36409e1d0 0x7fd36411c070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd358009870 tx=0x7fd358004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:27.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.059+0000 7fd352ffd700 1 -- 172.21.15.16:0/2570637725 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd358024620 con 0x7fd36409e1d0 2023-09-30T22:30:27.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.059+0000 7fd352ffd700 1 -- 172.21.15.16:0/2570637725 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3580247a0 con 0x7fd36409e1d0 2023-09-30T22:30:27.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.059+0000 7fd3688ab700 1 -- 172.21.15.16:0/2570637725 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd36411f2f0 con 0x7fd36409e1d0 2023-09-30T22:30:27.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.059+0000 7fd352ffd700 1 -- 172.21.15.16:0/2570637725 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd358024b50 con 0x7fd36409e1d0 2023-09-30T22:30:27.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.059+0000 7fd3688ab700 1 -- 172.21.15.16:0/2570637725 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd36411fd60 con 0x7fd36409e1d0 2023-09-30T22:30:27.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.060+0000 7fd352ffd700 1 -- 172.21.15.16:0/2570637725 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd358022070 con 0x7fd36409e1d0 2023-09-30T22:30:27.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.060+0000 7fd3688ab700 1 -- 172.21.15.16:0/2570637725 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3640a35d0 con 0x7fd36409e1d0 2023-09-30T22:30:27.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.060+0000 7fd352ffd700 1 -- 172.21.15.16:0/2570637725 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd3580407c0 con 0x7fd36409e1d0 2023-09-30T22:30:27.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.069+0000 7fd352ffd700 1 -- 172.21.15.16:0/2570637725 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd35800b360 con 0x7fd36409e1d0 2023-09-30T22:30:27.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.087+0000 7fb31df5e700 1 Processor -- start 2023-09-30T22:30:27.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.087+0000 7fb31df5e700 1 -- start start 2023-09-30T22:30:27.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.088+0000 7fb31df5e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb31809e850 0x7fb31809ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:27.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.088+0000 7fb31df5e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb31809f160 0x7fb31813c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:27.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.088+0000 7fb31df5e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb31809f590 0x7fb31813e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:27.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.088+0000 7fb31df5e700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb31806c930 con 0x7fb31809e850 2023-09-30T22:30:27.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.088+0000 7fb31df5e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb31809aa30 con 0x7fb31809f590 2023-09-30T22:30:27.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.088+0000 7fb31df5e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb31806bf30 con 0x7fb31809f160 2023-09-30T22:30:27.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.088+0000 7fb316ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb31809f160 0x7fb31813c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:27.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.088+0000 7fb316ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb31809f160 0x7fb31813c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:59094/0 (socket says 172.21.15.16:59094) 2023-09-30T22:30:27.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.088+0000 7fb316ffd700 1 -- 172.21.15.16:0/50765054 learned_addr learned my addr 172.21.15.16:0/50765054 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:27.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.088+0000 7fb3177fe700 1 --2- 172.21.15.16:0/50765054 >> v2:172.21.15.16:3300/0 conn(0x7fb31809e850 0x7fb31809ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:27.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.088+0000 7fb317fff700 1 --2- 172.21.15.16:0/50765054 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb31809f590 0x7fb31813e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:27.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.088+0000 7fb316ffd700 1 -- 172.21.15.16:0/50765054 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb31809f590 msgr2=0x7fb31813e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:27.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.088+0000 7fb316ffd700 1 --2- 172.21.15.16:0/50765054 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb31809f590 0x7fb31813e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.088+0000 7fb316ffd700 1 -- 172.21.15.16:0/50765054 >> v2:172.21.15.16:3300/0 conn(0x7fb31809e850 msgr2=0x7fb31809ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:27.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.088+0000 7fb316ffd700 1 --2- 172.21.15.16:0/50765054 >> v2:172.21.15.16:3300/0 conn(0x7fb31809e850 0x7fb31809ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.088+0000 7fb316ffd700 1 -- 172.21.15.16:0/50765054 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3180a0250 con 0x7fb31809f160 2023-09-30T22:30:27.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.088+0000 7fb3177fe700 1 --2- 172.21.15.16:0/50765054 >> v2:172.21.15.16:3300/0 conn(0x7fb31809e850 0x7fb31809ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:30:27.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.089+0000 7fb316ffd700 1 --2- 172.21.15.16:0/50765054 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb31809f160 0x7fb31813c370 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb30c0098f0 tx=0x7fb30c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9af9296dcc3bae4e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:27.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.089+0000 7fb3167fc700 1 -- 172.21.15.16:0/50765054 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30c011830 con 0x7fb31809f160 2023-09-30T22:30:27.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.089+0000 7fb3167fc700 1 -- 172.21.15.16:0/50765054 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb30c011990 con 0x7fb31809f160 2023-09-30T22:30:27.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.089+0000 7fb3167fc700 1 -- 172.21.15.16:0/50765054 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30c011d20 con 0x7fb31809f160 2023-09-30T22:30:27.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.090+0000 7fb31df5e700 1 -- 172.21.15.16:0/50765054 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb31809f160 msgr2=0x7fb31813c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:27.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.090+0000 7fb31df5e700 1 --2- 172.21.15.16:0/50765054 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb31809f160 0x7fb31813c370 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb30c0098f0 tx=0x7fb30c009c00 comp rx=0 tx=0).stop 2023-09-30T22:30:27.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.090+0000 7fb31df5e700 1 -- 172.21.15.16:0/50765054 shutdown_connections 2023-09-30T22:30:27.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.090+0000 7fb31df5e700 1 --2- 172.21.15.16:0/50765054 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb31809f590 0x7fb31813e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.090+0000 7fb31df5e700 1 --2- 172.21.15.16:0/50765054 >> v2:172.21.15.16:3300/0 conn(0x7fb31809e850 0x7fb31809ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.090+0000 7fb31df5e700 1 --2- 172.21.15.16:0/50765054 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb31809f160 0x7fb31813c370 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.090+0000 7fb31df5e700 1 -- 172.21.15.16:0/50765054 >> 172.21.15.16:0/50765054 conn(0x7fb3180a3f70 msgr2=0x7fb3180a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:27.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.090+0000 7fb31df5e700 1 -- 172.21.15.16:0/50765054 shutdown_connections 2023-09-30T22:30:27.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.090+0000 7fb31df5e700 1 -- 172.21.15.16:0/50765054 wait complete. 2023-09-30T22:30:27.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.091+0000 7fb31df5e700 1 Processor -- start 2023-09-30T22:30:27.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.092+0000 7fb31df5e700 1 -- start start 2023-09-30T22:30:27.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.092+0000 7f21d692b700 1 Processor -- start 2023-09-30T22:30:27.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.092+0000 7fb31df5e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb31809e850 0x7fb31811b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:27.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.092+0000 7fb31df5e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb31809f160 0x7fb31811b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:27.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.092+0000 7fb3177fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb31809e850 0x7fb31811b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:27.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.092+0000 7fb3177fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb31809e850 0x7fb31811b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:59104/0 (socket says 172.21.15.16:59104) 2023-09-30T22:30:27.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.093+0000 7fb316ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb31809f160 0x7fb31811b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:27.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.093+0000 7fb316ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb31809f160 0x7fb31811b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:34530/0 (socket says 172.21.15.16:34530) 2023-09-30T22:30:27.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.093+0000 7fb31df5e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb31809f590 0x7fb31811be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:27.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.093+0000 7fb3177fe700 1 -- 172.21.15.16:0/94258117 learned_addr learned my addr 172.21.15.16:0/94258117 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:27.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.093+0000 7fb31df5e700 1 -- 172.21.15.16:0/94258117 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb31811e2a0 con 0x7fb31809f590 2023-09-30T22:30:27.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.093+0000 7fb31df5e700 1 -- 172.21.15.16:0/94258117 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb318127b20 con 0x7fb31809f160 2023-09-30T22:30:27.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.093+0000 7fb31df5e700 1 -- 172.21.15.16:0/94258117 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb31811d520 con 0x7fb31809e850 2023-09-30T22:30:27.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.093+0000 7fb317fff700 1 --2- 172.21.15.16:0/94258117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb31809f590 0x7fb31811be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:27.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.094+0000 7fb316ffd700 1 -- 172.21.15.16:0/94258117 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb31809e850 msgr2=0x7fb31811b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:27.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.094+0000 7fb316ffd700 1 --2- 172.21.15.16:0/94258117 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb31809e850 0x7fb31811b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.094+0000 7fb316ffd700 1 -- 172.21.15.16:0/94258117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb31809f590 msgr2=0x7fb31811be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:27.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.094+0000 7fb316ffd700 1 --2- 172.21.15.16:0/94258117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb31809f590 0x7fb31811be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.094+0000 7fb316ffd700 1 -- 172.21.15.16:0/94258117 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb30c009550 con 0x7fb31809f160 2023-09-30T22:30:27.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.094+0000 7fb317fff700 1 --2- 172.21.15.16:0/94258117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb31809f590 0x7fb31811be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:30:27.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.092+0000 7f21d692b700 1 -- start start 2023-09-30T22:30:27.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.093+0000 7f21d692b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f21d006c0e0 0x7f21d009ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:27.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.093+0000 7f21d692b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f21d01380f0 0x7f21d013fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:27.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.093+0000 7f21d692b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d0134980 con 0x7f21d0134ec0 2023-09-30T22:30:27.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.093+0000 7f21d692b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21d006c4b0 con 0x7f21d01380f0 2023-09-30T22:30:27.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.093+0000 7f21d692b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f21d00a09c0 con 0x7f21d006c0e0 2023-09-30T22:30:27.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.093+0000 7f21d4ec8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f21d01380f0 0x7f21d013fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:27.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.093+0000 7f21d4ec8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f21d01380f0 0x7f21d013fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:34540/0 (socket says 172.21.15.16:34540) 2023-09-30T22:30:27.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.093+0000 7f21d4ec8700 1 -- 172.21.15.16:0/2168309763 learned_addr learned my addr 172.21.15.16:0/2168309763 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:27.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.093+0000 7f21cffff700 1 --2- 172.21.15.16:0/2168309763 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f21d006c0e0 0x7f21d009ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:27.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.094+0000 7f21cffff700 1 -- 172.21.15.16:0/2168309763 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f21d01380f0 msgr2=0x7f21d013fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:27.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.094+0000 7f21cffff700 1 --2- 172.21.15.16:0/2168309763 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f21d01380f0 0x7f21d013fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.094+0000 7f21cffff700 1 -- 172.21.15.16:0/2168309763 >> v1:172.21.15.16:6789/0 conn(0x7f21d0134ec0 legacy=0x7f21d01379e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:27.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.094+0000 7f21cffff700 1 -- 172.21.15.16:0/2168309763 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d00a0600 con 0x7f21d006c0e0 2023-09-30T22:30:27.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.094+0000 7f21d4ec8700 1 --2- 172.21.15.16:0/2168309763 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f21d01380f0 0x7f21d013fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:27.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.094+0000 7f21cffff700 1 --2- 172.21.15.16:0/2168309763 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f21d006c0e0 0x7f21d009ac90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f21c0009870 tx=0x7f21c0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b958bc627c0fa748 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:27.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.094+0000 7fb316ffd700 1 --2- 172.21.15.16:0/94258117 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb31809f160 0x7fb31811b8f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb3180a09b0 tx=0x7fb30c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:27.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.094+0000 7f21ceffd700 1 -- 172.21.15.16:0/2168309763 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c0011830 con 0x7f21d006c0e0 2023-09-30T22:30:27.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.094+0000 7f21ceffd700 1 -- 172.21.15.16:0/2168309763 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21c0011990 con 0x7f21d006c0e0 2023-09-30T22:30:27.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.094+0000 7f21ceffd700 1 -- 172.21.15.16:0/2168309763 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c0011d20 con 0x7f21d006c0e0 2023-09-30T22:30:27.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.095+0000 7fb2fffff700 1 -- 172.21.15.16:0/94258117 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30c004300 con 0x7fb31809f160 2023-09-30T22:30:27.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.095+0000 7fb2fffff700 1 -- 172.21.15.16:0/94258117 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb30c01f070 con 0x7fb31809f160 2023-09-30T22:30:27.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.095+0000 7fb31df5e700 1 -- 172.21.15.16:0/94258117 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb318127470 con 0x7fb31809f160 2023-09-30T22:30:27.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.095+0000 7fb2fffff700 1 -- 172.21.15.16:0/94258117 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30c0036a0 con 0x7fb31809f160 2023-09-30T22:30:27.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.095+0000 7fb31df5e700 1 -- 172.21.15.16:0/94258117 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb31811d760 con 0x7fb31809f160 2023-09-30T22:30:27.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.095+0000 7f21d692b700 1 -- 172.21.15.16:0/2168309763 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f21d006c0e0 msgr2=0x7f21d009ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:27.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.095+0000 7f21d692b700 1 --2- 172.21.15.16:0/2168309763 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f21d006c0e0 0x7f21d009ac90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f21c0009870 tx=0x7f21c0009b80 comp rx=0 tx=0).stop 2023-09-30T22:30:27.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.095+0000 7f21d692b700 1 -- 172.21.15.16:0/2168309763 shutdown_connections 2023-09-30T22:30:27.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.095+0000 7f21d692b700 1 --2- 172.21.15.16:0/2168309763 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f21d01380f0 0x7f21d013fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.095+0000 7f21d692b700 1 --2- 172.21.15.16:0/2168309763 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f21d006c0e0 0x7f21d009ac90 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.095+0000 7f21d692b700 1 -- 172.21.15.16:0/2168309763 >> 172.21.15.16:0/2168309763 conn(0x7f21d00a4210 msgr2=0x7f21d00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:27.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.096+0000 7f21d692b700 1 -- 172.21.15.16:0/2168309763 shutdown_connections 2023-09-30T22:30:27.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.096+0000 7f21d692b700 1 -- 172.21.15.16:0/2168309763 wait complete. 2023-09-30T22:30:27.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.096+0000 7fb31df5e700 1 -- 172.21.15.16:0/94258117 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3180747f0 con 0x7fb31809f160 2023-09-30T22:30:27.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.097+0000 7fb2fffff700 1 -- 172.21.15.16:0/94258117 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb30c021030 con 0x7fb31809f160 2023-09-30T22:30:27.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.097+0000 7fb2fffff700 1 -- 172.21.15.16:0/94258117 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fb30c040d50 con 0x7fb31809f160 2023-09-30T22:30:27.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.097+0000 7f21d692b700 1 Processor -- start 2023-09-30T22:30:27.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.097+0000 7f21d692b700 1 -- start start 2023-09-30T22:30:27.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.097+0000 7f21d692b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21d006c0e0 0x7f21d011a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:27.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.097+0000 7f21d692b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f21d0134ec0 0x7f21d011ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:27.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.097+0000 7f21d692b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f21d01380f0 0x7f21d011b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:27.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.097+0000 7f21d692b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d00a03f0 con 0x7f21d006c0e0 2023-09-30T22:30:27.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.097+0000 7f21d692b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21d0141040 con 0x7f21d0134ec0 2023-09-30T22:30:27.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.097+0000 7f21d692b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f21d011d5f0 con 0x7f21d01380f0 2023-09-30T22:30:27.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.097+0000 7f21cf7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f21d0134ec0 0x7f21d011ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:27.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.098+0000 7f21cf7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f21d0134ec0 0x7f21d011ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:34544/0 (socket says 172.21.15.16:34544) 2023-09-30T22:30:27.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.098+0000 7f21cf7fe700 1 -- 172.21.15.16:0/1243480002 learned_addr learned my addr 172.21.15.16:0/1243480002 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:27.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.098+0000 7f21d4ec8700 1 --2- 172.21.15.16:0/1243480002 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f21d01380f0 0x7f21d011b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:27.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.098+0000 7f21cffff700 1 --2- 172.21.15.16:0/1243480002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21d006c0e0 0x7f21d011a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:27.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.098+0000 7f21cf7fe700 1 -- 172.21.15.16:0/1243480002 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f21d01380f0 msgr2=0x7f21d011b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:27.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.098+0000 7f21cf7fe700 1 --2- 172.21.15.16:0/1243480002 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f21d01380f0 0x7f21d011b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.098+0000 7f21cf7fe700 1 -- 172.21.15.16:0/1243480002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21d006c0e0 msgr2=0x7f21d011a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:27.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.098+0000 7f21cf7fe700 1 --2- 172.21.15.16:0/1243480002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21d006c0e0 0x7f21d011a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.098+0000 7f21cf7fe700 1 -- 172.21.15.16:0/1243480002 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c8009b30 con 0x7f21d0134ec0 2023-09-30T22:30:27.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.098+0000 7f21cf7fe700 1 --2- 172.21.15.16:0/1243480002 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f21d0134ec0 0x7f21d011ae30 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f21c400c4b0 tx=0x7f21c400c7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:27.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.098+0000 7f21cffff700 1 --2- 172.21.15.16:0/1243480002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21d006c0e0 0x7f21d011a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:30:27.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.102+0000 7f21ccff9700 1 -- 172.21.15.16:0/1243480002 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c40052d0 con 0x7f21d0134ec0 2023-09-30T22:30:27.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.102+0000 7f21ccff9700 1 -- 172.21.15.16:0/1243480002 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21c4005430 con 0x7f21d0134ec0 2023-09-30T22:30:27.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.102+0000 7f21ccff9700 1 -- 172.21.15.16:0/1243480002 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c4003880 con 0x7f21d0134ec0 2023-09-30T22:30:27.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.102+0000 7fb2fffff700 1 -- 172.21.15.16:0/94258117 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb30c0199a0 con 0x7fb31809f160 2023-09-30T22:30:27.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.102+0000 7f21d692b700 1 -- 172.21.15.16:0/1243480002 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21c00094a0 con 0x7f21d0134ec0 2023-09-30T22:30:27.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.102+0000 7f21d692b700 1 -- 172.21.15.16:0/1243480002 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21d011c740 con 0x7f21d0134ec0 2023-09-30T22:30:27.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.103+0000 7f21d692b700 1 -- 172.21.15.16:0/1243480002 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21d011d1e0 con 0x7f21d0134ec0 2023-09-30T22:30:27.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.103+0000 7f21ccff9700 1 -- 172.21.15.16:0/1243480002 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f21c40039e0 con 0x7f21d0134ec0 2023-09-30T22:30:27.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.104+0000 7f21ccff9700 1 -- 172.21.15.16:0/1243480002 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f21c403ead0 con 0x7f21d0134ec0 2023-09-30T22:30:27.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.108+0000 7f21ccff9700 1 -- 172.21.15.16:0/1243480002 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f21c4010b60 con 0x7f21d0134ec0 2023-09-30T22:30:27.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.270+0000 7fc676b63700 1 -- 172.21.15.16:0/489274251 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc67011d6e0 con 0x7fc670135e50 2023-09-30T22:30:27.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.271+0000 7fc66d7fa700 1 -- 172.21.15.16:0/489274251 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc66802c020 con 0x7fc670135e50 2023-09-30T22:30:27.271 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:30:27.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.273+0000 7fc676b63700 1 -- 172.21.15.16:0/489274251 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc670135e50 msgr2=0x7fc67011bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:27.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.273+0000 7fc676b63700 1 --2- 172.21.15.16:0/489274251 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc670135e50 0x7fc67011bd60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc67009f6f0 tx=0x7fc668004ad0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.273+0000 7fc676b63700 1 -- 172.21.15.16:0/489274251 shutdown_connections 2023-09-30T22:30:27.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.273+0000 7fc676b63700 1 --2- 172.21.15.16:0/489274251 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc670135e50 0x7fc67011bd60 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.273+0000 7fc676b63700 1 --2- 172.21.15.16:0/489274251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc67009d6f0 0x7fc67011b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.273+0000 7fc676b63700 1 --2- 172.21.15.16:0/489274251 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc67009e1d0 0x7fc67011b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.273+0000 7fc676b63700 1 -- 172.21.15.16:0/489274251 >> 172.21.15.16:0/489274251 conn(0x7fc670066f50 msgr2=0x7fc6700a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:27.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.274+0000 7fc676b63700 1 -- 172.21.15.16:0/489274251 shutdown_connections 2023-09-30T22:30:27.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.274+0000 7fc676b63700 1 -- 172.21.15.16:0/489274251 wait complete. 2023-09-30T22:30:27.288 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:30:27.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.310+0000 7fb31df5e700 1 -- 172.21.15.16:0/94258117 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb318072840 con 0x7fb31809f160 2023-09-30T22:30:27.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.311+0000 7fb2fffff700 1 -- 172.21.15.16:0/94258117 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb30c015030 con 0x7fb31809f160 2023-09-30T22:30:27.311 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:30:27.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.313+0000 7fb31df5e700 1 -- 172.21.15.16:0/94258117 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb31809f160 msgr2=0x7fb31811b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:27.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.313+0000 7fb31df5e700 1 --2- 172.21.15.16:0/94258117 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb31809f160 0x7fb31811b8f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb3180a09b0 tx=0x7fb30c004750 comp rx=0 tx=0).stop 2023-09-30T22:30:27.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.313+0000 7fb31df5e700 1 -- 172.21.15.16:0/94258117 shutdown_connections 2023-09-30T22:30:27.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.313+0000 7fb31df5e700 1 --2- 172.21.15.16:0/94258117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb31809f590 0x7fb31811be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.313+0000 7fb31df5e700 1 --2- 172.21.15.16:0/94258117 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb31809e850 0x7fb31811b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.313+0000 7fb31df5e700 1 --2- 172.21.15.16:0/94258117 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb31809f160 0x7fb31811b8f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.313+0000 7fb31df5e700 1 -- 172.21.15.16:0/94258117 >> 172.21.15.16:0/94258117 conn(0x7fb3180a3f70 msgr2=0x7fb3180714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:27.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.313+0000 7fb31df5e700 1 -- 172.21.15.16:0/94258117 shutdown_connections 2023-09-30T22:30:27.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.313+0000 7fb31df5e700 1 -- 172.21.15.16:0/94258117 wait complete. 2023-09-30T22:30:27.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.315+0000 7fd3688ab700 1 -- 172.21.15.16:0/2570637725 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd364074ea0 con 0x7fd36409e1d0 2023-09-30T22:30:27.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.315+0000 7fd352ffd700 1 -- 172.21.15.16:0/2570637725 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd358015070 con 0x7fd36409e1d0 2023-09-30T22:30:27.316 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:30:27.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.317+0000 7fd3688ab700 1 -- 172.21.15.16:0/2570637725 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd36409e1d0 msgr2=0x7fd36411c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:27.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.317+0000 7fd3688ab700 1 --2- 172.21.15.16:0/2570637725 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd36409e1d0 0x7fd36411c070 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fd358009870 tx=0x7fd358004750 comp rx=0 tx=0).stop 2023-09-30T22:30:27.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.318+0000 7fd3688ab700 1 -- 172.21.15.16:0/2570637725 shutdown_connections 2023-09-30T22:30:27.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.318+0000 7fd3688ab700 1 --2- 172.21.15.16:0/2570637725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd364135e50 0x7fd36411c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.318+0000 7fd3688ab700 1 --2- 172.21.15.16:0/2570637725 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd36409d6f0 0x7fd36411bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.318+0000 7fd3688ab700 1 --2- 172.21.15.16:0/2570637725 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd36409e1d0 0x7fd36411c070 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.318+0000 7fd3688ab700 1 -- 172.21.15.16:0/2570637725 >> 172.21.15.16:0/2570637725 conn(0x7fd364066f50 msgr2=0x7fd36406e840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:27.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.318+0000 7fd3688ab700 1 -- 172.21.15.16:0/2570637725 shutdown_connections 2023-09-30T22:30:27.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.318+0000 7fd3688ab700 1 -- 172.21.15.16:0/2570637725 wait complete. 2023-09-30T22:30:27.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.318+0000 7f21d692b700 1 -- 172.21.15.16:0/1243480002 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f21d0074ea0 con 0x7f21d0134ec0 2023-09-30T22:30:27.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.319+0000 7f21ccff9700 1 -- 172.21.15.16:0/1243480002 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f21c4017020 con 0x7f21d0134ec0 2023-09-30T22:30:27.320 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:30:27.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.321+0000 7f21d692b700 1 -- 172.21.15.16:0/1243480002 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f21d0134ec0 msgr2=0x7f21d011ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:27.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.321+0000 7f21d692b700 1 --2- 172.21.15.16:0/1243480002 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f21d0134ec0 0x7f21d011ae30 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f21c400c4b0 tx=0x7f21c400c7c0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.322+0000 7f21d692b700 1 -- 172.21.15.16:0/1243480002 shutdown_connections 2023-09-30T22:30:27.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.322+0000 7f21d692b700 1 --2- 172.21.15.16:0/1243480002 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f21d01380f0 0x7f21d011b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.322+0000 7f21d692b700 1 --2- 172.21.15.16:0/1243480002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21d006c0e0 0x7f21d011a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.322+0000 7f21d692b700 1 --2- 172.21.15.16:0/1243480002 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f21d0134ec0 0x7f21d011ae30 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:27.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.322+0000 7f21d692b700 1 -- 172.21.15.16:0/1243480002 >> 172.21.15.16:0/1243480002 conn(0x7f21d00a4210 msgr2=0x7f21d00a4810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:27.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.322+0000 7f21d692b700 1 -- 172.21.15.16:0/1243480002 shutdown_connections 2023-09-30T22:30:27.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:27.322+0000 7f21d692b700 1 -- 172.21.15.16:0/1243480002 wait complete. 2023-09-30T22:30:27.330 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:30:27.335 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:30:27.337 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:30:28.289 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:28.331 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:28.336 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:28.338 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:28.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.451+0000 7f1f1a66e700 1 Processor -- start 2023-09-30T22:30:28.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.451+0000 7f1f1a66e700 1 -- start start 2023-09-30T22:30:28.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.452+0000 7f1f1a66e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f1409d630 0x7f1f140a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:28.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.452+0000 7f1f1a66e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1f14138d80 0x7f1f1413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:28.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.452+0000 7f1f1a66e700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f140a2a70 con 0x7f1f140a0a30 2023-09-30T22:30:28.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.452+0000 7f1f1a66e700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f140a01d0 con 0x7f1f14138d80 2023-09-30T22:30:28.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.452+0000 7f1f1a66e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1409e2b0 con 0x7f1f1409d630 2023-09-30T22:30:28.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.452+0000 7f1f137fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f1409d630 0x7f1f140a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:28.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.452+0000 7f1f137fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f1409d630 0x7f1f140a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38100/0 (socket says 172.21.15.16:38100) 2023-09-30T22:30:28.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.453+0000 7f1f137fe700 1 -- 172.21.15.16:0/648221078 learned_addr learned my addr 172.21.15.16:0/648221078 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:28.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.453+0000 7f1f18c0b700 1 --2- 172.21.15.16:0/648221078 >> v2:172.21.15.16:3300/0 conn(0x7f1f14138d80 0x7f1f1413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:28.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.453+0000 7f1f137fe700 1 -- 172.21.15.16:0/648221078 >> v2:172.21.15.16:3300/0 conn(0x7f1f14138d80 msgr2=0x7f1f1413b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:28.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.453+0000 7f1f137fe700 1 --2- 172.21.15.16:0/648221078 >> v2:172.21.15.16:3300/0 conn(0x7f1f14138d80 0x7f1f1413b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.453+0000 7f1f137fe700 1 -- 172.21.15.16:0/648221078 >> v1:172.21.15.16:6789/0 conn(0x7f1f140a0a30 legacy=0x7f1f140a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:28.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.453+0000 7f1f137fe700 1 -- 172.21.15.16:0/648221078 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f1409dc50 con 0x7f1f1409d630 2023-09-30T22:30:28.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.453+0000 7f1f137fe700 1 --2- 172.21.15.16:0/648221078 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f1409d630 0x7f1f140a1510 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1f080098f0 tx=0x7f1f08009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2ba10d4e068a117 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:28.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.454+0000 7f1f12ffd700 1 -- 172.21.15.16:0/648221078 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0800b8a0 con 0x7f1f1409d630 2023-09-30T22:30:28.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.454+0000 7f1f12ffd700 1 -- 172.21.15.16:0/648221078 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f0800ba00 con 0x7f1f1409d630 2023-09-30T22:30:28.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.454+0000 7f1f12ffd700 1 -- 172.21.15.16:0/648221078 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0800bd90 con 0x7f1f1409d630 2023-09-30T22:30:28.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.454+0000 7f1f1a66e700 1 -- 172.21.15.16:0/648221078 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f1409d630 msgr2=0x7f1f140a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:28.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.454+0000 7f1f1a66e700 1 --2- 172.21.15.16:0/648221078 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f1409d630 0x7f1f140a1510 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1f080098f0 tx=0x7f1f08009c00 comp rx=0 tx=0).stop 2023-09-30T22:30:28.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.454+0000 7f1f1a66e700 1 -- 172.21.15.16:0/648221078 shutdown_connections 2023-09-30T22:30:28.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.454+0000 7f1f1a66e700 1 --2- 172.21.15.16:0/648221078 >> v2:172.21.15.16:3300/0 conn(0x7f1f14138d80 0x7f1f1413b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.454+0000 7f1f1a66e700 1 --2- 172.21.15.16:0/648221078 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f1409d630 0x7f1f140a1510 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.454+0000 7f1f1a66e700 1 -- 172.21.15.16:0/648221078 >> 172.21.15.16:0/648221078 conn(0x7f1f140a3050 msgr2=0x7f1f140a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:28.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.455+0000 7f1f1a66e700 1 -- 172.21.15.16:0/648221078 shutdown_connections 2023-09-30T22:30:28.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.455+0000 7f1f1a66e700 1 -- 172.21.15.16:0/648221078 wait complete. 2023-09-30T22:30:28.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.456+0000 7f1f1a66e700 1 Processor -- start 2023-09-30T22:30:28.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.456+0000 7f1f1a66e700 1 -- start start 2023-09-30T22:30:28.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.457+0000 7f1f1a66e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f1409d630 0x7f1f14133ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:28.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.457+0000 7f1f1a66e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f140a0a30 0x7f1f14134010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:28.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.457+0000 7f1f1a66e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f14138d80 0x7f1f14134550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:28.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.457+0000 7f1f1a66e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f14137520 con 0x7f1f14138d80 2023-09-30T22:30:28.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.457+0000 7f1f1a66e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1413af80 con 0x7f1f140a0a30 2023-09-30T22:30:28.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.457+0000 7f1f1a66e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1f140a0410 con 0x7f1f1409d630 2023-09-30T22:30:28.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.457+0000 7f1f137fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f140a0a30 0x7f1f14134010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:28.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.457+0000 7f1f137fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f140a0a30 0x7f1f14134010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38102/0 (socket says 172.21.15.16:38102) 2023-09-30T22:30:28.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.457+0000 7f1f137fe700 1 -- 172.21.15.16:0/2018094638 learned_addr learned my addr 172.21.15.16:0/2018094638 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:28.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.457+0000 7f1f18c0b700 1 --2- 172.21.15.16:0/2018094638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f14138d80 0x7f1f14134550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:28.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.458+0000 7f1f13fff700 1 --2- 172.21.15.16:0/2018094638 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f1409d630 0x7f1f14133ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:28.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.458+0000 7f1f18c0b700 1 -- 172.21.15.16:0/2018094638 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f1409d630 msgr2=0x7f1f14133ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:28.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.458+0000 7f1f18c0b700 1 --2- 172.21.15.16:0/2018094638 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f1409d630 0x7f1f14133ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.458+0000 7f1f18c0b700 1 -- 172.21.15.16:0/2018094638 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f140a0a30 msgr2=0x7f1f14134010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:28.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.458+0000 7f1f18c0b700 1 --2- 172.21.15.16:0/2018094638 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f140a0a30 0x7f1f14134010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.458+0000 7f1f18c0b700 1 -- 172.21.15.16:0/2018094638 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f08009550 con 0x7f1f14138d80 2023-09-30T22:30:28.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.458+0000 7f1f18c0b700 1 --2- 172.21.15.16:0/2018094638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f14138d80 0x7f1f14134550 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c007d60 tx=0x7f1f0c00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:28.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.458+0000 7f1f10ff9700 1 -- 172.21.15.16:0/2018094638 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c018070 con 0x7f1f14138d80 2023-09-30T22:30:28.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.458+0000 7f1f10ff9700 1 -- 172.21.15.16:0/2018094638 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f0c00ebe0 con 0x7f1f14138d80 2023-09-30T22:30:28.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.458+0000 7f1f1a66e700 1 -- 172.21.15.16:0/2018094638 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f141381a0 con 0x7f1f14138d80 2023-09-30T22:30:28.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.458+0000 7f1f10ff9700 1 -- 172.21.15.16:0/2018094638 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c003a20 con 0x7f1f14138d80 2023-09-30T22:30:28.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.458+0000 7f1f1a66e700 1 -- 172.21.15.16:0/2018094638 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f14136c70 con 0x7f1f14138d80 2023-09-30T22:30:28.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.461+0000 7f1f10ff9700 1 -- 172.21.15.16:0/2018094638 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1f0c014420 con 0x7f1f14138d80 2023-09-30T22:30:28.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.461+0000 7f1f1a66e700 1 -- 172.21.15.16:0/2018094638 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f140747f0 con 0x7f1f14138d80 2023-09-30T22:30:28.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.461+0000 7f1f10ff9700 1 -- 172.21.15.16:0/2018094638 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f1f0c03f7a0 con 0x7f1f14138d80 2023-09-30T22:30:28.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.466+0000 7f1f10ff9700 1 -- 172.21.15.16:0/2018094638 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1f0c0146d0 con 0x7f1f14138d80 2023-09-30T22:30:28.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.485+0000 7fbce1620700 1 Processor -- start 2023-09-30T22:30:28.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.486+0000 7fbce1620700 1 -- start start 2023-09-30T22:30:28.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.486+0000 7fbce1620700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbcdc06e570 0x7fbcdc06e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:28.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.486+0000 7fbce1620700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbcdc09eca0 0x7fbcdc06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:28.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.486+0000 7fbce1620700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbcdc134610 0x7fbcdc134a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:28.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.486+0000 7fbce1620700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcdc070ee0 con 0x7fbcdc09eca0 2023-09-30T22:30:28.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.486+0000 7fbce1620700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcdc071480 con 0x7fbcdc06e570 2023-09-30T22:30:28.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.486+0000 7fbce1620700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbcdc06f3c0 con 0x7fbcdc134610 2023-09-30T22:30:28.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.486+0000 7fbcdb7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbcdc134610 0x7fbcdc134a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:28.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.486+0000 7fbcdb7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbcdc134610 0x7fbcdc134a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:54576/0 (socket says 172.21.15.16:54576) 2023-09-30T22:30:28.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.486+0000 7fbcdb7fe700 1 -- 172.21.15.16:0/201019974 learned_addr learned my addr 172.21.15.16:0/201019974 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:28.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.486+0000 7fbcdaffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbcdc06e570 0x7fbcdc06e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:28.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.487+0000 7fbcda7fc700 1 --2- 172.21.15.16:0/201019974 >> v2:172.21.15.16:3300/0 conn(0x7fbcdc09eca0 0x7fbcdc06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:28.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.487+0000 7fbcdb7fe700 1 -- 172.21.15.16:0/201019974 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbcdc06e570 msgr2=0x7fbcdc06e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:28.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.487+0000 7fbcdb7fe700 1 --2- 172.21.15.16:0/201019974 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbcdc06e570 0x7fbcdc06e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.487+0000 7fbcdb7fe700 1 -- 172.21.15.16:0/201019974 >> v2:172.21.15.16:3300/0 conn(0x7fbcdc09eca0 msgr2=0x7fbcdc06ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:28.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.487+0000 7fbcdb7fe700 1 --2- 172.21.15.16:0/201019974 >> v2:172.21.15.16:3300/0 conn(0x7fbcdc09eca0 0x7fbcdc06ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.487+0000 7fbcdb7fe700 1 -- 172.21.15.16:0/201019974 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcdc071b90 con 0x7fbcdc134610 2023-09-30T22:30:28.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.487+0000 7fbcdb7fe700 1 --2- 172.21.15.16:0/201019974 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbcdc134610 0x7fbcdc134a40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4009e50 tx=0x7fbcd400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6455aa7a5701646d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:28.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.487+0000 7fbcd9ffb700 1 -- 172.21.15.16:0/201019974 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd400e070 con 0x7fbcdc134610 2023-09-30T22:30:28.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.487+0000 7fbcd9ffb700 1 -- 172.21.15.16:0/201019974 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcd400b910 con 0x7fbcdc134610 2023-09-30T22:30:28.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.487+0000 7fbcd9ffb700 1 -- 172.21.15.16:0/201019974 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd400bcc0 con 0x7fbcdc134610 2023-09-30T22:30:28.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.488+0000 7fbce1620700 1 -- 172.21.15.16:0/201019974 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbcdc134610 msgr2=0x7fbcdc134a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:28.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.488+0000 7fbce1620700 1 --2- 172.21.15.16:0/201019974 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbcdc134610 0x7fbcdc134a40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbcd4009e50 tx=0x7fbcd400b330 comp rx=0 tx=0).stop 2023-09-30T22:30:28.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.489+0000 7fbce1620700 1 -- 172.21.15.16:0/201019974 shutdown_connections 2023-09-30T22:30:28.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.489+0000 7fbce1620700 1 --2- 172.21.15.16:0/201019974 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbcdc06e570 0x7fbcdc06e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.489+0000 7fbce1620700 1 --2- 172.21.15.16:0/201019974 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbcdc134610 0x7fbcdc134a40 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.489+0000 7fbce1620700 1 --2- 172.21.15.16:0/201019974 >> v2:172.21.15.16:3300/0 conn(0x7fbcdc09eca0 0x7fbcdc06ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.489+0000 7fbce1620700 1 -- 172.21.15.16:0/201019974 >> 172.21.15.16:0/201019974 conn(0x7fbcdc0a2aa0 msgr2=0x7fbcdc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:28.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.489+0000 7fbce1620700 1 -- 172.21.15.16:0/201019974 shutdown_connections 2023-09-30T22:30:28.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.489+0000 7fbce1620700 1 -- 172.21.15.16:0/201019974 wait complete. 2023-09-30T22:30:28.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.490+0000 7fbce1620700 1 Processor -- start 2023-09-30T22:30:28.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.490+0000 7fbce1620700 1 -- start start 2023-09-30T22:30:28.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.490+0000 7fbce1620700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbcdc06e570 0x7fbcdc127410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:28.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.490+0000 7fbce1620700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbcdc134610 0x7fbcdc125fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:28.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.490+0000 7fbce1620700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcdc127950 0x7fbcdc1264f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:28.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.490+0000 7fbce1620700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcdc13e5b0 con 0x7fbcdc127950 2023-09-30T22:30:28.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.490+0000 7fbce1620700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcdc070610 con 0x7fbcdc134610 2023-09-30T22:30:28.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.490+0000 7fbce1620700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbcdc13d2b0 con 0x7fbcdc06e570 2023-09-30T22:30:28.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.491+0000 7fbcdaffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbcdc06e570 0x7fbcdc127410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:28.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.491+0000 7fbcdaffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbcdc06e570 0x7fbcdc127410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:54578/0 (socket says 172.21.15.16:54578) 2023-09-30T22:30:28.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.491+0000 7fbcdaffd700 1 -- 172.21.15.16:0/1867562072 learned_addr learned my addr 172.21.15.16:0/1867562072 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:28.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.491+0000 7fbcdb7fe700 1 --2- 172.21.15.16:0/1867562072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcdc127950 0x7fbcdc1264f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:28.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.491+0000 7fbcdaffd700 1 -- 172.21.15.16:0/1867562072 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbcdc134610 msgr2=0x7fbcdc125fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:28.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.491+0000 7fbcdaffd700 1 --2- 172.21.15.16:0/1867562072 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbcdc134610 0x7fbcdc125fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.491+0000 7fbcdaffd700 1 -- 172.21.15.16:0/1867562072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcdc127950 msgr2=0x7fbcdc1264f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:28.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.491+0000 7fbcdaffd700 1 --2- 172.21.15.16:0/1867562072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcdc127950 0x7fbcdc1264f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.491+0000 7fbcdaffd700 1 -- 172.21.15.16:0/1867562072 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd4009b30 con 0x7fbcdc06e570 2023-09-30T22:30:28.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.491+0000 7fbcdb7fe700 1 --2- 172.21.15.16:0/1867562072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcdc127950 0x7fbcdc1264f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:30:28.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.491+0000 7fbcdaffd700 1 --2- 172.21.15.16:0/1867562072 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbcdc06e570 0x7fbcdc127410 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbccc00e9b0 tx=0x7fbccc00ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:28.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.491+0000 7fbcc37fe700 1 -- 172.21.15.16:0/1867562072 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbccc009710 con 0x7fbcdc06e570 2023-09-30T22:30:28.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.492+0000 7fbcc37fe700 1 -- 172.21.15.16:0/1867562072 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbccc00c850 con 0x7fbcdc06e570 2023-09-30T22:30:28.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.492+0000 7fbce1620700 1 -- 172.21.15.16:0/1867562072 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcdc13c830 con 0x7fbcdc06e570 2023-09-30T22:30:28.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.492+0000 7fbcc37fe700 1 -- 172.21.15.16:0/1867562072 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbccc00cbc0 con 0x7fbcdc06e570 2023-09-30T22:30:28.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.492+0000 7fbce1620700 1 -- 172.21.15.16:0/1867562072 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcdc13d850 con 0x7fbcdc06e570 2023-09-30T22:30:28.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.493+0000 7fbce1620700 1 -- 172.21.15.16:0/1867562072 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcdc074ea0 con 0x7fbcdc06e570 2023-09-30T22:30:28.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.493+0000 7fbcc37fe700 1 -- 172.21.15.16:0/1867562072 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbccc00cd20 con 0x7fbcdc06e570 2023-09-30T22:30:28.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.494+0000 7fbcc37fe700 1 -- 172.21.15.16:0/1867562072 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fbccc040000 con 0x7fbcdc06e570 2023-09-30T22:30:28.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.498+0000 7fbcc37fe700 1 -- 172.21.15.16:0/1867562072 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbccc01a360 con 0x7fbcdc06e570 2023-09-30T22:30:28.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.525+0000 7f6054ca9700 1 Processor -- start 2023-09-30T22:30:28.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.525+0000 7f6054ca9700 1 -- start start 2023-09-30T22:30:28.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.525+0000 7f6054ca9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f605009d6f0 0x7f605009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:28.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.525+0000 7f6054ca9700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f605009e410 0x7f6050134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:28.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.526+0000 7f6054ca9700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f605009b120 con 0x7f605009e000 2023-09-30T22:30:28.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.526+0000 7f6054ca9700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f605006b460 con 0x7f605009e410 2023-09-30T22:30:28.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.526+0000 7f6054ca9700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f605009a7e0 con 0x7f605009d6f0 2023-09-30T22:30:28.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.526+0000 7f604ed9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f605009e410 0x7f6050134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:28.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.526+0000 7f604dd9b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f605009e000 0x7f60501339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34454/0 (socket says 172.21.15.16:34454) 2023-09-30T22:30:28.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.526+0000 7f604dd9b700 1 -- 172.21.15.16:0/2867658568 learned_addr learned my addr 172.21.15.16:0/2867658568 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:30:28.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.526+0000 7f604e59c700 1 --2- 172.21.15.16:0/2867658568 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f605009d6f0 0x7f605009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:28.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.526+0000 7f604e59c700 1 -- 172.21.15.16:0/2867658568 >> v2:172.21.15.16:3300/0 conn(0x7f605009e410 msgr2=0x7f6050134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:28.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.526+0000 7f604e59c700 1 --2- 172.21.15.16:0/2867658568 >> v2:172.21.15.16:3300/0 conn(0x7f605009e410 0x7f6050134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.526+0000 7f604e59c700 1 -- 172.21.15.16:0/2867658568 >> v1:172.21.15.16:6789/0 conn(0x7f605009e000 legacy=0x7f60501339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:28.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.526+0000 7f604e59c700 1 -- 172.21.15.16:0/2867658568 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f605009a500 con 0x7f605009d6f0 2023-09-30T22:30:28.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.526+0000 7f604ed9d700 1 --2- 172.21.15.16:0/2867658568 >> v2:172.21.15.16:3300/0 conn(0x7f605009e410 0x7f6050134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:28.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.527+0000 7f604e59c700 1 --2- 172.21.15.16:0/2867658568 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f605009d6f0 0x7f605009dac0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6040009870 tx=0x7f6040009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1227d6caa26e0a4f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:28.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.527+0000 7f604d59a700 1 -- 172.21.15.16:0/2867658568 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6040004230 con 0x7f605009d6f0 2023-09-30T22:30:28.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.527+0000 7f604d59a700 1 -- 172.21.15.16:0/2867658568 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6040004390 con 0x7f605009d6f0 2023-09-30T22:30:28.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.527+0000 7f604d59a700 1 -- 172.21.15.16:0/2867658568 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60400036a0 con 0x7f605009d6f0 2023-09-30T22:30:28.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.528+0000 7f6054ca9700 1 -- 172.21.15.16:0/2867658568 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f605009d6f0 msgr2=0x7f605009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:28.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.528+0000 7f6054ca9700 1 --2- 172.21.15.16:0/2867658568 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f605009d6f0 0x7f605009dac0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6040009870 tx=0x7f6040009b80 comp rx=0 tx=0).stop 2023-09-30T22:30:28.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.528+0000 7f6054ca9700 1 -- 172.21.15.16:0/2867658568 shutdown_connections 2023-09-30T22:30:28.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.528+0000 7f6054ca9700 1 --2- 172.21.15.16:0/2867658568 >> v2:172.21.15.16:3300/0 conn(0x7f605009e410 0x7f6050134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.528+0000 7f6054ca9700 1 --2- 172.21.15.16:0/2867658568 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f605009d6f0 0x7f605009dac0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.528+0000 7f6054ca9700 1 -- 172.21.15.16:0/2867658568 >> 172.21.15.16:0/2867658568 conn(0x7f6050066f50 msgr2=0x7f605009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:28.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.529+0000 7f6054ca9700 1 -- 172.21.15.16:0/2867658568 shutdown_connections 2023-09-30T22:30:28.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.529+0000 7f6054ca9700 1 -- 172.21.15.16:0/2867658568 wait complete. 2023-09-30T22:30:28.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.530+0000 7f6054ca9700 1 Processor -- start 2023-09-30T22:30:28.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.530+0000 7f6054ca9700 1 -- start start 2023-09-30T22:30:28.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.531+0000 7f6054ca9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f605009d6f0 0x7f60501a5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:28.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.531+0000 7f6054ca9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f605009e000 0x7f60501a6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:28.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.531+0000 7f6054ca9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f605009e410 0x7f60501a69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:28.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.531+0000 7f6054ca9700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60501a85c0 con 0x7f605009d6f0 2023-09-30T22:30:28.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.531+0000 7f6054ca9700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60501a7b00 con 0x7f605009e000 2023-09-30T22:30:28.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.531+0000 7f6054ca9700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f60501a8de0 con 0x7f605009e410 2023-09-30T22:30:28.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.531+0000 7f604ed9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f605009e410 0x7f60501a69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:28.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.531+0000 7f604e59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f605009d6f0 0x7f60501a5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:28.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.531+0000 7f604ed9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f605009e410 0x7f60501a69b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:54592/0 (socket says 172.21.15.16:54592) 2023-09-30T22:30:28.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.531+0000 7f604ed9d700 1 -- 172.21.15.16:0/1471790104 learned_addr learned my addr 172.21.15.16:0/1471790104 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:28.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.531+0000 7f604dd9b700 1 --2- 172.21.15.16:0/1471790104 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f605009e000 0x7f60501a6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:28.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.532+0000 7f604ed9d700 1 -- 172.21.15.16:0/1471790104 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f605009e000 msgr2=0x7f60501a6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:28.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.532+0000 7f604ed9d700 1 --2- 172.21.15.16:0/1471790104 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f605009e000 0x7f60501a6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.532+0000 7f604ed9d700 1 -- 172.21.15.16:0/1471790104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f605009d6f0 msgr2=0x7f60501a5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:28.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.532+0000 7f604ed9d700 1 --2- 172.21.15.16:0/1471790104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f605009d6f0 0x7f60501a5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.532+0000 7f604ed9d700 1 -- 172.21.15.16:0/1471790104 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6048009a70 con 0x7f605009e410 2023-09-30T22:30:28.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.532+0000 7f604e59c700 1 --2- 172.21.15.16:0/1471790104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f605009d6f0 0x7f60501a5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:28.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.532+0000 7f604ed9d700 1 --2- 172.21.15.16:0/1471790104 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f605009e410 0x7f60501a69b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f604800efc0 tx=0x7f604800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:28.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.532+0000 7f603effd700 1 -- 172.21.15.16:0/1471790104 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f604800ca90 con 0x7f605009e410 2023-09-30T22:30:28.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.532+0000 7f603effd700 1 -- 172.21.15.16:0/1471790104 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f604800f040 con 0x7f605009e410 2023-09-30T22:30:28.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.532+0000 7f6054ca9700 1 -- 172.21.15.16:0/1471790104 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60400094a0 con 0x7f605009e410 2023-09-30T22:30:28.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.532+0000 7f603effd700 1 -- 172.21.15.16:0/1471790104 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f604800ce10 con 0x7f605009e410 2023-09-30T22:30:28.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.532+0000 7f6054ca9700 1 -- 172.21.15.16:0/1471790104 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60501a9780 con 0x7f605009e410 2023-09-30T22:30:28.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.534+0000 7f6054ca9700 1 -- 172.21.15.16:0/1471790104 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6050074ea0 con 0x7f605009e410 2023-09-30T22:30:28.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.535+0000 7f603effd700 1 -- 172.21.15.16:0/1471790104 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6048003a20 con 0x7f605009e410 2023-09-30T22:30:28.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.535+0000 7f603effd700 1 -- 172.21.15.16:0/1471790104 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f6048040660 con 0x7f605009e410 2023-09-30T22:30:28.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.539+0000 7f907f10c700 1 Processor -- start 2023-09-30T22:30:28.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.539+0000 7f907f10c700 1 -- start start 2023-09-30T22:30:28.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.539+0000 7f907f10c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f907809d6f0 0x7f907809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:28.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.540+0000 7f907f10c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f907809e000 0x7f9078130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:28.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.540+0000 7f907f10c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f907809f130 con 0x7f907809e410 2023-09-30T22:30:28.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.540+0000 7f907f10c700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f907809f520 con 0x7f907809d6f0 2023-09-30T22:30:28.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.540+0000 7f907f10c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f907809e930 con 0x7f907809e000 2023-09-30T22:30:28.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.540+0000 7f907cea8700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f907809d6f0 0x7f907809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:28.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.540+0000 7f9077fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f907809e000 0x7f9078130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:28.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.540+0000 7f907cea8700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f907809d6f0 0x7f907809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55220/0 (socket says 172.21.15.16:55220) 2023-09-30T22:30:28.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.540+0000 7f907cea8700 1 -- 172.21.15.16:0/287812270 learned_addr learned my addr 172.21.15.16:0/287812270 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:28.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.540+0000 7f9077fff700 1 -- 172.21.15.16:0/287812270 >> v2:172.21.15.16:3300/0 conn(0x7f907809d6f0 msgr2=0x7f907809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:28.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.540+0000 7f9077fff700 1 --2- 172.21.15.16:0/287812270 >> v2:172.21.15.16:3300/0 conn(0x7f907809d6f0 0x7f907809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.540+0000 7f9077fff700 1 -- 172.21.15.16:0/287812270 >> v1:172.21.15.16:6789/0 conn(0x7f907809e410 legacy=0x7f9078130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:28.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.540+0000 7f9077fff700 1 -- 172.21.15.16:0/287812270 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f907809b2d0 con 0x7f907809e000 2023-09-30T22:30:28.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.540+0000 7f9077fff700 1 --2- 172.21.15.16:0/287812270 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f907809e000 0x7f9078130660 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9064009840 tx=0x7f9064009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40e31e99fa50ae06 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:28.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.541+0000 7f90777fe700 1 -- 172.21.15.16:0/287812270 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9064011830 con 0x7f907809e000 2023-09-30T22:30:28.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.541+0000 7f90777fe700 1 -- 172.21.15.16:0/287812270 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9064011990 con 0x7f907809e000 2023-09-30T22:30:28.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.541+0000 7f90777fe700 1 -- 172.21.15.16:0/287812270 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9064011d20 con 0x7f907809e000 2023-09-30T22:30:28.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.541+0000 7f907f10c700 1 -- 172.21.15.16:0/287812270 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f907809e000 msgr2=0x7f9078130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:28.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.541+0000 7f907f10c700 1 --2- 172.21.15.16:0/287812270 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f907809e000 0x7f9078130660 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9064009840 tx=0x7f9064009b50 comp rx=0 tx=0).stop 2023-09-30T22:30:28.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.541+0000 7f907f10c700 1 -- 172.21.15.16:0/287812270 shutdown_connections 2023-09-30T22:30:28.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.541+0000 7f907f10c700 1 --2- 172.21.15.16:0/287812270 >> v2:172.21.15.16:3300/0 conn(0x7f907809d6f0 0x7f907809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.541+0000 7f907f10c700 1 --2- 172.21.15.16:0/287812270 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f907809e000 0x7f9078130660 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.541+0000 7f907f10c700 1 -- 172.21.15.16:0/287812270 >> 172.21.15.16:0/287812270 conn(0x7f9078066f50 msgr2=0x7f907809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:28.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.541+0000 7f907f10c700 1 -- 172.21.15.16:0/287812270 shutdown_connections 2023-09-30T22:30:28.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.541+0000 7f907f10c700 1 -- 172.21.15.16:0/287812270 wait complete. 2023-09-30T22:30:28.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.542+0000 7f907f10c700 1 Processor -- start 2023-09-30T22:30:28.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.543+0000 7f907f10c700 1 -- start start 2023-09-30T22:30:28.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.543+0000 7f907f10c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f907809d6f0 0x7f9078122770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:28.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.543+0000 7f907f10c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f907809e000 0x7f9078122cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:28.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.543+0000 7f907f10c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f907809e410 0x7f90781231f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:28.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.543+0000 7f907f10c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90781242b0 con 0x7f907809e000 2023-09-30T22:30:28.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.543+0000 7f907f10c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90781258a0 con 0x7f907809e410 2023-09-30T22:30:28.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.543+0000 7f907f10c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f90780a1d00 con 0x7f907809d6f0 2023-09-30T22:30:28.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.543+0000 7f907d6a9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f907809e410 0x7f90781231f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:28.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.543+0000 7f9077fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f907809e000 0x7f9078122cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:28.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.543+0000 7f907d6a9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f907809e410 0x7f90781231f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38158/0 (socket says 172.21.15.16:38158) 2023-09-30T22:30:28.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.543+0000 7f907d6a9700 1 -- 172.21.15.16:0/3832446402 learned_addr learned my addr 172.21.15.16:0/3832446402 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:28.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.543+0000 7f603effd700 1 -- 172.21.15.16:0/1471790104 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f604801c3b0 con 0x7f605009e410 2023-09-30T22:30:28.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.544+0000 7f907cea8700 1 --2- 172.21.15.16:0/3832446402 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f907809d6f0 0x7f9078122770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:28.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.544+0000 7f907d6a9700 1 -- 172.21.15.16:0/3832446402 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f907809d6f0 msgr2=0x7f9078122770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:28.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.544+0000 7f907d6a9700 1 --2- 172.21.15.16:0/3832446402 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f907809d6f0 0x7f9078122770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.544+0000 7f907d6a9700 1 -- 172.21.15.16:0/3832446402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f907809e000 msgr2=0x7f9078122cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:28.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.544+0000 7f907d6a9700 1 --2- 172.21.15.16:0/3832446402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f907809e000 0x7f9078122cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.544+0000 7f907d6a9700 1 -- 172.21.15.16:0/3832446402 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90640094a0 con 0x7f907809e410 2023-09-30T22:30:28.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.544+0000 7f9077fff700 1 --2- 172.21.15.16:0/3832446402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f907809e000 0x7f9078122cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:28.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.544+0000 7f907d6a9700 1 --2- 172.21.15.16:0/3832446402 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f907809e410 0x7f90781231f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f907000cc50 tx=0x7f907000cf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:28.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.544+0000 7f90757fa700 1 -- 172.21.15.16:0/3832446402 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9070003c00 con 0x7f907809e410 2023-09-30T22:30:28.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.544+0000 7f90757fa700 1 -- 172.21.15.16:0/3832446402 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9070003d60 con 0x7f907809e410 2023-09-30T22:30:28.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.544+0000 7f907f10c700 1 -- 172.21.15.16:0/3832446402 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f907809c700 con 0x7f907809e410 2023-09-30T22:30:28.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.545+0000 7f90757fa700 1 -- 172.21.15.16:0/3832446402 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907000ab60 con 0x7f907809e410 2023-09-30T22:30:28.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.545+0000 7f907f10c700 1 -- 172.21.15.16:0/3832446402 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9078125c20 con 0x7f907809e410 2023-09-30T22:30:28.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.545+0000 7f90757fa700 1 -- 172.21.15.16:0/3832446402 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f907000acc0 con 0x7f907809e410 2023-09-30T22:30:28.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.546+0000 7f90757fa700 1 -- 172.21.15.16:0/3832446402 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f907003f720 con 0x7f907809e410 2023-09-30T22:30:28.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.546+0000 7f907f10c700 1 -- 172.21.15.16:0/3832446402 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9078074ea0 con 0x7f907809e410 2023-09-30T22:30:28.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.551+0000 7f90757fa700 1 -- 172.21.15.16:0/3832446402 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f90700198f0 con 0x7f907809e410 2023-09-30T22:30:28.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.685+0000 7f1f1a66e700 1 -- 172.21.15.16:0/2018094638 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1f141371a0 con 0x7f1f14138d80 2023-09-30T22:30:28.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.685+0000 7f1f10ff9700 1 -- 172.21.15.16:0/2018094638 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1f141371a0 con 0x7f1f14138d80 2023-09-30T22:30:28.686 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:30:28.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.687+0000 7f1f1a66e700 1 -- 172.21.15.16:0/2018094638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f14138d80 msgr2=0x7f1f14134550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:28.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.687+0000 7f1f1a66e700 1 --2- 172.21.15.16:0/2018094638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f14138d80 0x7f1f14134550 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c007d60 tx=0x7f1f0c00e3c0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.688+0000 7f1f1a66e700 1 -- 172.21.15.16:0/2018094638 shutdown_connections 2023-09-30T22:30:28.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.688+0000 7f1f1a66e700 1 --2- 172.21.15.16:0/2018094638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f14138d80 0x7f1f14134550 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.688+0000 7f1f1a66e700 1 --2- 172.21.15.16:0/2018094638 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f1409d630 0x7f1f14133ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.688+0000 7f1f1a66e700 1 --2- 172.21.15.16:0/2018094638 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f140a0a30 0x7f1f14134010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.688+0000 7f1f1a66e700 1 -- 172.21.15.16:0/2018094638 >> 172.21.15.16:0/2018094638 conn(0x7f1f140a3050 msgr2=0x7f1f141399e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:28.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.688+0000 7f1f1a66e700 1 -- 172.21.15.16:0/2018094638 shutdown_connections 2023-09-30T22:30:28.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.688+0000 7f1f1a66e700 1 -- 172.21.15.16:0/2018094638 wait complete. 2023-09-30T22:30:28.702 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:30:28.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.724+0000 7fbce1620700 1 -- 172.21.15.16:0/1867562072 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fbcdc13d470 con 0x7fbcdc06e570 2023-09-30T22:30:28.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.725+0000 7fbcc37fe700 1 -- 172.21.15.16:0/1867562072 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbccc021020 con 0x7fbcdc06e570 2023-09-30T22:30:28.725 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:30:28.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.727+0000 7fbce1620700 1 -- 172.21.15.16:0/1867562072 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbcdc06e570 msgr2=0x7fbcdc127410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:28.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.727+0000 7fbce1620700 1 --2- 172.21.15.16:0/1867562072 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbcdc06e570 0x7fbcdc127410 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbccc00e9b0 tx=0x7fbccc00ed70 comp rx=0 tx=0).stop 2023-09-30T22:30:28.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.727+0000 7fbce1620700 1 -- 172.21.15.16:0/1867562072 shutdown_connections 2023-09-30T22:30:28.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.727+0000 7fbce1620700 1 --2- 172.21.15.16:0/1867562072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcdc127950 0x7fbcdc1264f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.727+0000 7fbce1620700 1 --2- 172.21.15.16:0/1867562072 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbcdc06e570 0x7fbcdc127410 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.727+0000 7fbce1620700 1 --2- 172.21.15.16:0/1867562072 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbcdc134610 0x7fbcdc125fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.727+0000 7fbce1620700 1 -- 172.21.15.16:0/1867562072 >> 172.21.15.16:0/1867562072 conn(0x7fbcdc0a2aa0 msgr2=0x7fbcdc136640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:28.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.727+0000 7fbce1620700 1 -- 172.21.15.16:0/1867562072 shutdown_connections 2023-09-30T22:30:28.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.727+0000 7fbce1620700 1 -- 172.21.15.16:0/1867562072 wait complete. 2023-09-30T22:30:28.741 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:30:28.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.770+0000 7f6054ca9700 1 -- 172.21.15.16:0/1471790104 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f60501a9780 con 0x7f605009e410 2023-09-30T22:30:28.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.770+0000 7f603effd700 1 -- 172.21.15.16:0/1471790104 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f60501a9780 con 0x7f605009e410 2023-09-30T22:30:28.771 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:30:28.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.773+0000 7f6054ca9700 1 -- 172.21.15.16:0/1471790104 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f605009e410 msgr2=0x7f60501a69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:28.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.773+0000 7f6054ca9700 1 --2- 172.21.15.16:0/1471790104 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f605009e410 0x7f60501a69b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f604800efc0 tx=0x7f604800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.773+0000 7f6054ca9700 1 -- 172.21.15.16:0/1471790104 shutdown_connections 2023-09-30T22:30:28.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.773+0000 7f6054ca9700 1 --2- 172.21.15.16:0/1471790104 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f605009e410 0x7f60501a69b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.773+0000 7f6054ca9700 1 --2- 172.21.15.16:0/1471790104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f605009d6f0 0x7f60501a5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.773+0000 7f6054ca9700 1 --2- 172.21.15.16:0/1471790104 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f605009e000 0x7f60501a6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.773+0000 7f6054ca9700 1 -- 172.21.15.16:0/1471790104 >> 172.21.15.16:0/1471790104 conn(0x7f6050066f50 msgr2=0x7f60501baf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:28.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.773+0000 7f6054ca9700 1 -- 172.21.15.16:0/1471790104 shutdown_connections 2023-09-30T22:30:28.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.773+0000 7f6054ca9700 1 -- 172.21.15.16:0/1471790104 wait complete. 2023-09-30T22:30:28.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.777+0000 7f907f10c700 1 -- 172.21.15.16:0/3832446402 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f907806c360 con 0x7f907809e410 2023-09-30T22:30:28.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.777+0000 7f90757fa700 1 -- 172.21.15.16:0/3832446402 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9070011380 con 0x7f907809e410 2023-09-30T22:30:28.777 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:30:28.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.779+0000 7f907f10c700 1 -- 172.21.15.16:0/3832446402 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f907809e410 msgr2=0x7f90781231f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:28.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.779+0000 7f907f10c700 1 --2- 172.21.15.16:0/3832446402 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f907809e410 0x7f90781231f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f907000cc50 tx=0x7f907000cf60 comp rx=0 tx=0).stop 2023-09-30T22:30:28.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.779+0000 7f907f10c700 1 -- 172.21.15.16:0/3832446402 shutdown_connections 2023-09-30T22:30:28.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.779+0000 7f907f10c700 1 --2- 172.21.15.16:0/3832446402 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f907809e410 0x7f90781231f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.779+0000 7f907f10c700 1 --2- 172.21.15.16:0/3832446402 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f907809d6f0 0x7f9078122770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.779+0000 7f907f10c700 1 --2- 172.21.15.16:0/3832446402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f907809e000 0x7f9078122cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:28.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.779+0000 7f907f10c700 1 -- 172.21.15.16:0/3832446402 >> 172.21.15.16:0/3832446402 conn(0x7f9078066f50 msgr2=0x7f907806f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:28.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.779+0000 7f907f10c700 1 -- 172.21.15.16:0/3832446402 shutdown_connections 2023-09-30T22:30:28.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:28.779+0000 7f907f10c700 1 -- 172.21.15.16:0/3832446402 wait complete. 2023-09-30T22:30:28.788 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:30:28.794 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:30:29.703 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:29.742 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:29.789 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:29.795 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:29.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.856+0000 7fd2620fc700 1 Processor -- start 2023-09-30T22:30:29.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.856+0000 7fd2620fc700 1 -- start start 2023-09-30T22:30:29.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.857+0000 7fd2620fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd25c136830 0x7fd25c138c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:29.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.857+0000 7fd2620fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd25c139150 0x7fd25c13b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:29.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.857+0000 7fd2620fc700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd25c06f720 con 0x7fd25c1338c0 2023-09-30T22:30:29.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.857+0000 7fd2620fc700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c070c80 con 0x7fd25c136830 2023-09-30T22:30:29.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.857+0000 7fd2620fc700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c06fce0 con 0x7fd25c139150 2023-09-30T22:30:29.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.857+0000 7fd25bfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd25c139150 0x7fd25c13b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:29.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.857+0000 7fd25bfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd25c139150 0x7fd25c13b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:54608/0 (socket says 172.21.15.16:54608) 2023-09-30T22:30:29.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.857+0000 7fd25bfff700 1 -- 172.21.15.16:0/95702341 learned_addr learned my addr 172.21.15.16:0/95702341 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:29.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.857+0000 7fd25affd700 1 --2- 172.21.15.16:0/95702341 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd25c136830 0x7fd25c138c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:29.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.857+0000 7fd25affd700 1 -- 172.21.15.16:0/95702341 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd25c139150 msgr2=0x7fd25c13b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:29.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.857+0000 7fd25affd700 1 --2- 172.21.15.16:0/95702341 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd25c139150 0x7fd25c13b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:29.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.857+0000 7fd25affd700 1 -- 172.21.15.16:0/95702341 >> v1:172.21.15.16:6789/0 conn(0x7fd25c1338c0 legacy=0x7fd25c133c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:29.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.857+0000 7fd25affd700 1 -- 172.21.15.16:0/95702341 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd25c1336a0 con 0x7fd25c136830 2023-09-30T22:30:29.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.858+0000 7fd25affd700 1 --2- 172.21.15.16:0/95702341 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd25c136830 0x7fd25c138c10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd250009870 tx=0x7fd250009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b22fa90b654a83ca server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:29.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.858+0000 7fd25a7fc700 1 -- 172.21.15.16:0/95702341 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd250011830 con 0x7fd25c136830 2023-09-30T22:30:29.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.858+0000 7fd25a7fc700 1 -- 172.21.15.16:0/95702341 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd250011990 con 0x7fd25c136830 2023-09-30T22:30:29.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.858+0000 7fd25a7fc700 1 -- 172.21.15.16:0/95702341 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd250011d20 con 0x7fd25c136830 2023-09-30T22:30:29.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.859+0000 7fd2620fc700 1 -- 172.21.15.16:0/95702341 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd25c136830 msgr2=0x7fd25c138c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:29.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.859+0000 7fd2620fc700 1 --2- 172.21.15.16:0/95702341 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd25c136830 0x7fd25c138c10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd250009870 tx=0x7fd250009b80 comp rx=0 tx=0).stop 2023-09-30T22:30:29.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.859+0000 7fd2620fc700 1 -- 172.21.15.16:0/95702341 shutdown_connections 2023-09-30T22:30:29.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.859+0000 7fd2620fc700 1 --2- 172.21.15.16:0/95702341 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd25c139150 0x7fd25c13b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:29.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.859+0000 7fd2620fc700 1 --2- 172.21.15.16:0/95702341 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd25c136830 0x7fd25c138c10 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:29.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.859+0000 7fd2620fc700 1 -- 172.21.15.16:0/95702341 >> 172.21.15.16:0/95702341 conn(0x7fd25c0a2aa0 msgr2=0x7fd25c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:29.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.859+0000 7fd2620fc700 1 -- 172.21.15.16:0/95702341 shutdown_connections 2023-09-30T22:30:29.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.859+0000 7fd2620fc700 1 -- 172.21.15.16:0/95702341 wait complete. 2023-09-30T22:30:29.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.860+0000 7fd2620fc700 1 Processor -- start 2023-09-30T22:30:29.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.860+0000 7fd2620fc700 1 -- start start 2023-09-30T22:30:29.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.861+0000 7fd2620fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd25c1338c0 0x7fd25c099ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:29.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.861+0000 7fd2620fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd25c136830 0x7fd25c09a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:29.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.861+0000 7fd2620fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd25c139150 0x7fd25c1cc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:29.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.861+0000 7fd2620fc700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c09d390 con 0x7fd25c136830 2023-09-30T22:30:29.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.861+0000 7fd2620fc700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c09b680 con 0x7fd25c139150 2023-09-30T22:30:29.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.861+0000 7fd2620fc700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c122710 con 0x7fd25c1338c0 2023-09-30T22:30:29.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.861+0000 7fd25b7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd25c1338c0 0x7fd25c099ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:29.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.861+0000 7fd25b7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd25c1338c0 0x7fd25c099ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:54616/0 (socket says 172.21.15.16:54616) 2023-09-30T22:30:29.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.861+0000 7fd25b7fe700 1 -- 172.21.15.16:0/3591649239 learned_addr learned my addr 172.21.15.16:0/3591649239 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:29.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.861+0000 7fd25affd700 1 --2- 172.21.15.16:0/3591649239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd25c136830 0x7fd25c09a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:29.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.861+0000 7fd25bfff700 1 --2- 172.21.15.16:0/3591649239 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd25c139150 0x7fd25c1cc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:29.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.862+0000 7fd25affd700 1 -- 172.21.15.16:0/3591649239 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd25c1338c0 msgr2=0x7fd25c099ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:29.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.862+0000 7fd25affd700 1 --2- 172.21.15.16:0/3591649239 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd25c1338c0 0x7fd25c099ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:29.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.862+0000 7fd25affd700 1 -- 172.21.15.16:0/3591649239 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd25c139150 msgr2=0x7fd25c1cc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:29.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.862+0000 7fd25affd700 1 --2- 172.21.15.16:0/3591649239 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd25c139150 0x7fd25c1cc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:29.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.862+0000 7fd25affd700 1 -- 172.21.15.16:0/3591649239 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2500094a0 con 0x7fd25c136830 2023-09-30T22:30:29.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.862+0000 7fd25affd700 1 --2- 172.21.15.16:0/3591649239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd25c136830 0x7fd25c09a220 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd250016000 tx=0x7fd250004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:29.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.862+0000 7fd243fff700 1 -- 172.21.15.16:0/3591649239 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd250004450 con 0x7fd25c136830 2023-09-30T22:30:29.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.862+0000 7fd243fff700 1 -- 172.21.15.16:0/3591649239 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd25001f070 con 0x7fd25c136830 2023-09-30T22:30:29.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.862+0000 7fd2620fc700 1 -- 172.21.15.16:0/3591649239 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd25c1363a0 con 0x7fd25c136830 2023-09-30T22:30:29.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.862+0000 7fd243fff700 1 -- 172.21.15.16:0/3591649239 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2500038e0 con 0x7fd25c136830 2023-09-30T22:30:29.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.862+0000 7fd2620fc700 1 -- 172.21.15.16:0/3591649239 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd25c09c600 con 0x7fd25c136830 2023-09-30T22:30:29.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.863+0000 7fd243fff700 1 -- 172.21.15.16:0/3591649239 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd250003a40 con 0x7fd25c136830 2023-09-30T22:30:29.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.864+0000 7fd243fff700 1 -- 172.21.15.16:0/3591649239 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd250021070 con 0x7fd25c136830 2023-09-30T22:30:29.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.864+0000 7fd2620fc700 1 -- 172.21.15.16:0/3591649239 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd25c074ea0 con 0x7fd25c136830 2023-09-30T22:30:29.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.873+0000 7fd243fff700 1 -- 172.21.15.16:0/3591649239 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd25002b070 con 0x7fd25c136830 2023-09-30T22:30:29.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.907+0000 7febcdefb700 1 Processor -- start 2023-09-30T22:30:29.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.907+0000 7febcdefb700 1 -- start start 2023-09-30T22:30:29.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.908+0000 7febcdefb700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7febc809d6f0 0x7febc809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:29.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.908+0000 7febcdefb700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7febc809e000 0x7febc81339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:29.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.908+0000 7febcdefb700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7febc8133f30 0x7febc81343b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:29.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.908+0000 7febcdefb700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febc809b0f0 con 0x7febc809e000 2023-09-30T22:30:29.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.908+0000 7febcdefb700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febc809d2f0 con 0x7febc809d6f0 2023-09-30T22:30:29.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.908+0000 7febcdefb700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7febc809a810 con 0x7febc8133f30 2023-09-30T22:30:29.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.908+0000 7febc6ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7febc809e000 0x7febc81339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:29.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.908+0000 7febc6ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7febc809e000 0x7febc81339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55252/0 (socket says 172.21.15.16:55252) 2023-09-30T22:30:29.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.908+0000 7febc6ffd700 1 -- 172.21.15.16:0/1660617329 learned_addr learned my addr 172.21.15.16:0/1660617329 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:29.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.908+0000 7febc7fff700 1 --2- 172.21.15.16:0/1660617329 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7febc8133f30 0x7febc81343b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:29.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.908+0000 7febc77fe700 1 --2- 172.21.15.16:0/1660617329 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7febc809d6f0 0x7febc809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:29.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.908+0000 7febc6ffd700 1 -- 172.21.15.16:0/1660617329 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7febc8133f30 msgr2=0x7febc81343b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:29.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.908+0000 7febc6ffd700 1 --2- 172.21.15.16:0/1660617329 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7febc8133f30 0x7febc81343b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:29.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.908+0000 7febc6ffd700 1 -- 172.21.15.16:0/1660617329 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7febc809d6f0 msgr2=0x7febc809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:29.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.908+0000 7febc6ffd700 1 --2- 172.21.15.16:0/1660617329 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7febc809d6f0 0x7febc809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:29.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.908+0000 7febc6ffd700 1 -- 172.21.15.16:0/1660617329 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc809a470 con 0x7febc809e000 2023-09-30T22:30:29.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.909+0000 7febc6ffd700 1 --2- 172.21.15.16:0/1660617329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febc809e000 0x7febc81339f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7febbc0098f0 tx=0x7febbc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0f412424be8b58c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:29.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.909+0000 7febc67fc700 1 -- 172.21.15.16:0/1660617329 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc014e90 con 0x7febc809e000 2023-09-30T22:30:29.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.909+0000 7febc67fc700 1 -- 172.21.15.16:0/1660617329 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febbc01c440 con 0x7febc809e000 2023-09-30T22:30:29.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.909+0000 7febc67fc700 1 -- 172.21.15.16:0/1660617329 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febbc01c7d0 con 0x7febc809e000 2023-09-30T22:30:29.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.910+0000 7febcdefb700 1 -- 172.21.15.16:0/1660617329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febc809e000 msgr2=0x7febc81339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:29.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.910+0000 7febcdefb700 1 --2- 172.21.15.16:0/1660617329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febc809e000 0x7febc81339f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7febbc0098f0 tx=0x7febbc009c00 comp rx=0 tx=0).stop 2023-09-30T22:30:29.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.910+0000 7febcdefb700 1 -- 172.21.15.16:0/1660617329 shutdown_connections 2023-09-30T22:30:29.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.910+0000 7febcdefb700 1 --2- 172.21.15.16:0/1660617329 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7febc809d6f0 0x7febc809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:29.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.910+0000 7febcdefb700 1 --2- 172.21.15.16:0/1660617329 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7febc8133f30 0x7febc81343b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:29.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.910+0000 7febcdefb700 1 --2- 172.21.15.16:0/1660617329 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febc809e000 0x7febc81339f0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:29.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.910+0000 7febcdefb700 1 -- 172.21.15.16:0/1660617329 >> 172.21.15.16:0/1660617329 conn(0x7febc8066f50 msgr2=0x7febc809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:29.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.910+0000 7febcdefb700 1 -- 172.21.15.16:0/1660617329 shutdown_connections 2023-09-30T22:30:29.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.910+0000 7febcdefb700 1 -- 172.21.15.16:0/1660617329 wait complete. 2023-09-30T22:30:29.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.912+0000 7febcdefb700 1 Processor -- start 2023-09-30T22:30:29.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.912+0000 7febcdefb700 1 -- start start 2023-09-30T22:30:29.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.913+0000 7febcdefb700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7febc809d6f0 0x7febc8070740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:29.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.913+0000 7febcdefb700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7febc809e000 0x7febc8070c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:29.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.913+0000 7febcdefb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febc8133f30 0x7febc81c3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:29.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.913+0000 7febcdefb700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc80704f0 con 0x7febc8133f30 2023-09-30T22:30:29.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.913+0000 7febcdefb700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febc8119ee0 con 0x7febc809d6f0 2023-09-30T22:30:29.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.913+0000 7febcdefb700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7febc80a1730 con 0x7febc809e000 2023-09-30T22:30:29.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.913+0000 7febc77fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7febc809d6f0 0x7febc8070740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:29.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.913+0000 7febc77fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7febc809d6f0 0x7febc8070740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38208/0 (socket says 172.21.15.16:38208) 2023-09-30T22:30:29.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.913+0000 7febc77fe700 1 -- 172.21.15.16:0/2516429098 learned_addr learned my addr 172.21.15.16:0/2516429098 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:29.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.913+0000 7febc6ffd700 1 --2- 172.21.15.16:0/2516429098 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7febc809e000 0x7febc8070c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:29.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.913+0000 7febc7fff700 1 --2- 172.21.15.16:0/2516429098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febc8133f30 0x7febc81c3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:29.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.913+0000 7febc77fe700 1 -- 172.21.15.16:0/2516429098 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7febc809e000 msgr2=0x7febc8070c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:29.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.913+0000 7febc77fe700 1 --2- 172.21.15.16:0/2516429098 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7febc809e000 0x7febc8070c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:29.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.913+0000 7febc77fe700 1 -- 172.21.15.16:0/2516429098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febc8133f30 msgr2=0x7febc81c3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:29.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.913+0000 7febc77fe700 1 --2- 172.21.15.16:0/2516429098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febc8133f30 0x7febc81c3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:29.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.913+0000 7febc77fe700 1 -- 172.21.15.16:0/2516429098 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febbc009550 con 0x7febc809d6f0 2023-09-30T22:30:29.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.913+0000 7febc7fff700 1 --2- 172.21.15.16:0/2516429098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febc8133f30 0x7febc81c3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:30:29.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.914+0000 7febc77fe700 1 --2- 172.21.15.16:0/2516429098 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7febc809d6f0 0x7febc8070740 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7febb800e9b0 tx=0x7febb800ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:29.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.914+0000 7febaffff700 1 -- 172.21.15.16:0/2516429098 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb8009710 con 0x7febc809d6f0 2023-09-30T22:30:29.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.914+0000 7febaffff700 1 -- 172.21.15.16:0/2516429098 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febb800c850 con 0x7febc809d6f0 2023-09-30T22:30:29.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.914+0000 7febcdefb700 1 -- 172.21.15.16:0/2516429098 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febc809c640 con 0x7febc809d6f0 2023-09-30T22:30:29.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.914+0000 7febaffff700 1 -- 172.21.15.16:0/2516429098 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb800cbc0 con 0x7febc809d6f0 2023-09-30T22:30:29.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.914+0000 7febcdefb700 1 -- 172.21.15.16:0/2516429098 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febc811ad00 con 0x7febc809d6f0 2023-09-30T22:30:29.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.915+0000 7febcdefb700 1 -- 172.21.15.16:0/2516429098 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febc8074ea0 con 0x7febc809d6f0 2023-09-30T22:30:29.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.916+0000 7febaffff700 1 -- 172.21.15.16:0/2516429098 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7febb800cd20 con 0x7febc809d6f0 2023-09-30T22:30:29.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.916+0000 7febaffff700 1 -- 172.21.15.16:0/2516429098 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7febb803ffd0 con 0x7febc809d6f0 2023-09-30T22:30:29.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.921+0000 7febaffff700 1 -- 172.21.15.16:0/2516429098 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7febb8011350 con 0x7febc809d6f0 2023-09-30T22:30:29.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.944+0000 7f1edad5d700 1 Processor -- start 2023-09-30T22:30:29.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.944+0000 7f1edad5d700 1 -- start start 2023-09-30T22:30:29.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.945+0000 7f1edad5d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1ed406a770 0x7f1ed409d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:29.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.945+0000 7f1edad5d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ed409dc30 0x7f1ed409e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:29.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.945+0000 7f1edad5d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ed409a810 con 0x7f1ed413c1e0 2023-09-30T22:30:29.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.945+0000 7f1edad5d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ed406ab40 con 0x7f1ed406a770 2023-09-30T22:30:29.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.945+0000 7f1edad5d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed409ebc0 con 0x7f1ed409dc30 2023-09-30T22:30:29.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.945+0000 7f1ed8af9700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1ed406a770 0x7f1ed409d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:29.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.945+0000 7f1ed8af9700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1ed406a770 0x7f1ed409d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55268/0 (socket says 172.21.15.16:55268) 2023-09-30T22:30:29.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.945+0000 7f1ed8af9700 1 -- 172.21.15.16:0/1689311543 learned_addr learned my addr 172.21.15.16:0/1689311543 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:29.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.945+0000 7f1ed3fff700 1 --2- 172.21.15.16:0/1689311543 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ed409dc30 0x7f1ed409e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:29.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.945+0000 7f1ed37fe700 1 -- 172.21.15.16:0/1689311543 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2857709285 0 0) 0x7f1ed409a810 con 0x7f1ed413c1e0 2023-09-30T22:30:29.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.945+0000 7f1ed37fe700 1 -- 172.21.15.16:0/1689311543 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ec40035b0 con 0x7f1ed413c1e0 2023-09-30T22:30:29.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.946+0000 7f1ed8af9700 1 -- 172.21.15.16:0/1689311543 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ed409dc30 msgr2=0x7f1ed409e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:29.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.946+0000 7f1ed8af9700 1 --2- 172.21.15.16:0/1689311543 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ed409dc30 0x7f1ed409e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:29.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.946+0000 7f1ed8af9700 1 -- 172.21.15.16:0/1689311543 >> v1:172.21.15.16:6789/0 conn(0x7f1ed413c1e0 legacy=0x7f1ed413e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:29.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.946+0000 7f1ed8af9700 1 -- 172.21.15.16:0/1689311543 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ed409e920 con 0x7f1ed406a770 2023-09-30T22:30:29.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.946+0000 7f1ed3fff700 1 --2- 172.21.15.16:0/1689311543 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ed409dc30 0x7f1ed409e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:29.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.946+0000 7f1ed8af9700 1 --2- 172.21.15.16:0/1689311543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ed406a770 0x7f1ed409d6f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1ec80098f0 tx=0x7f1ec8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac0ad92b48468517 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:29.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.946+0000 7f1ed37fe700 1 -- 172.21.15.16:0/1689311543 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec8011830 con 0x7f1ed406a770 2023-09-30T22:30:29.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.946+0000 7f1ed37fe700 1 -- 172.21.15.16:0/1689311543 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ec8011990 con 0x7f1ed406a770 2023-09-30T22:30:29.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.946+0000 7f1ed37fe700 1 -- 172.21.15.16:0/1689311543 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec8011d20 con 0x7f1ed406a770 2023-09-30T22:30:29.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.947+0000 7f1edad5d700 1 -- 172.21.15.16:0/1689311543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ed406a770 msgr2=0x7f1ed409d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:29.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.947+0000 7f1edad5d700 1 --2- 172.21.15.16:0/1689311543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ed406a770 0x7f1ed409d6f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1ec80098f0 tx=0x7f1ec8009c00 comp rx=0 tx=0).stop 2023-09-30T22:30:29.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.947+0000 7f1edad5d700 1 -- 172.21.15.16:0/1689311543 shutdown_connections 2023-09-30T22:30:29.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.947+0000 7f1edad5d700 1 --2- 172.21.15.16:0/1689311543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ed406a770 0x7f1ed409d6f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:29.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.947+0000 7f1edad5d700 1 --2- 172.21.15.16:0/1689311543 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ed409dc30 0x7f1ed409e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:29.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.947+0000 7f1edad5d700 1 -- 172.21.15.16:0/1689311543 >> 172.21.15.16:0/1689311543 conn(0x7f1ed4066f50 msgr2=0x7f1ed409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:29.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.947+0000 7f1edad5d700 1 -- 172.21.15.16:0/1689311543 shutdown_connections 2023-09-30T22:30:29.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.947+0000 7f1edad5d700 1 -- 172.21.15.16:0/1689311543 wait complete. 2023-09-30T22:30:29.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.948+0000 7f1edad5d700 1 Processor -- start 2023-09-30T22:30:29.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.949+0000 7f1edad5d700 1 -- start start 2023-09-30T22:30:29.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.949+0000 7f1edad5d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ed406a770 0x7f1ed413e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:29.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.949+0000 7f1edad5d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ed409dc30 0x7f1ed413a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:29.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.949+0000 7f1edad5d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ed413c1e0 0x7f1ed413a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:29.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.949+0000 7f1edad5d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed41349d0 con 0x7f1ed413c1e0 2023-09-30T22:30:29.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.949+0000 7f1edad5d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed413bb80 con 0x7f1ed409dc30 2023-09-30T22:30:29.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.949+0000 7f1edad5d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed411d660 con 0x7f1ed406a770 2023-09-30T22:30:29.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.950+0000 7f1ed3fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ed409dc30 0x7f1ed413a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:29.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.950+0000 7f1ed3fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ed409dc30 0x7f1ed413a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38226/0 (socket says 172.21.15.16:38226) 2023-09-30T22:30:29.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.950+0000 7f1ed3fff700 1 -- 172.21.15.16:0/1961585738 learned_addr learned my addr 172.21.15.16:0/1961585738 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:29.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.950+0000 7f1ed92fa700 1 --2- 172.21.15.16:0/1961585738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ed413c1e0 0x7f1ed413a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:29.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.950+0000 7f1ed8af9700 1 --2- 172.21.15.16:0/1961585738 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ed406a770 0x7f1ed413e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:29.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.950+0000 7f1ed3fff700 1 -- 172.21.15.16:0/1961585738 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ed406a770 msgr2=0x7f1ed413e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:29.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.950+0000 7f1ed3fff700 1 --2- 172.21.15.16:0/1961585738 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ed406a770 0x7f1ed413e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:29.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.950+0000 7f1ed3fff700 1 -- 172.21.15.16:0/1961585738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ed413c1e0 msgr2=0x7f1ed413a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:29.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.950+0000 7f1ed3fff700 1 --2- 172.21.15.16:0/1961585738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ed413c1e0 0x7f1ed413a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:29.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.950+0000 7f1ed3fff700 1 -- 172.21.15.16:0/1961585738 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec00094a0 con 0x7f1ed409dc30 2023-09-30T22:30:29.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.950+0000 7f1ed92fa700 1 --2- 172.21.15.16:0/1961585738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ed413c1e0 0x7f1ed413a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:29.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.950+0000 7f1ed3fff700 1 --2- 172.21.15.16:0/1961585738 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ed409dc30 0x7f1ed413a290 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1ec000ec10 tx=0x7f1ec000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:29.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.951+0000 7f1ed17fa700 1 -- 172.21.15.16:0/1961585738 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec000c850 con 0x7f1ed409dc30 2023-09-30T22:30:29.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.951+0000 7f1ed17fa700 1 -- 172.21.15.16:0/1961585738 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ec000c9b0 con 0x7f1ed409dc30 2023-09-30T22:30:29.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.951+0000 7f1edad5d700 1 -- 172.21.15.16:0/1961585738 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ec8009550 con 0x7f1ed409dc30 2023-09-30T22:30:29.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.951+0000 7f1ed17fa700 1 -- 172.21.15.16:0/1961585738 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec000cd20 con 0x7f1ed409dc30 2023-09-30T22:30:29.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.951+0000 7f1edad5d700 1 -- 172.21.15.16:0/1961585738 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ed411d920 con 0x7f1ed409dc30 2023-09-30T22:30:29.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.952+0000 7f1ed17fa700 1 -- 172.21.15.16:0/1961585738 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1ec0013470 con 0x7f1ed409dc30 2023-09-30T22:30:29.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.952+0000 7f1edad5d700 1 -- 172.21.15.16:0/1961585738 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ed4074ea0 con 0x7f1ed409dc30 2023-09-30T22:30:29.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.952+0000 7f1ed17fa700 1 -- 172.21.15.16:0/1961585738 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f1ec003f7f0 con 0x7f1ed409dc30 2023-09-30T22:30:29.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.961+0000 7f1ed17fa700 1 -- 172.21.15.16:0/1961585738 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1ec0013720 con 0x7f1ed409dc30 2023-09-30T22:30:29.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.997+0000 7efeed17a700 1 Processor -- start 2023-09-30T22:30:29.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.997+0000 7efeed17a700 1 -- start start 2023-09-30T22:30:29.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.997+0000 7efeed17a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efee80a0a30 0x7efee80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:29.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.997+0000 7efeed17a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efee8134170 0x7efee813bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:29.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.997+0000 7efeed17a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efee809edd0 con 0x7efee80a1340 2023-09-30T22:30:29.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.997+0000 7efeed17a700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efee809e1f0 con 0x7efee8134170 2023-09-30T22:30:29.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.997+0000 7efeed17a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efee809dc50 con 0x7efee80a0a30 2023-09-30T22:30:29.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.998+0000 7efee659c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7efee80a1340 0x7efee8133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34476/0 (socket says 172.21.15.16:34476) 2023-09-30T22:30:29.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.998+0000 7efee759e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efee8134170 0x7efee813bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:29.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.998+0000 7efee659c700 1 -- 172.21.15.16:0/2666831492 learned_addr learned my addr 172.21.15.16:0/2666831492 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:30:29.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.998+0000 7efee6d9d700 1 --2- 172.21.15.16:0/2666831492 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efee80a0a30 0x7efee80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:29.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.998+0000 7efee759e700 1 -- 172.21.15.16:0/2666831492 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efee80a0a30 msgr2=0x7efee80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:29.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.998+0000 7efee759e700 1 --2- 172.21.15.16:0/2666831492 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efee80a0a30 0x7efee80a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:29.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.998+0000 7efee759e700 1 -- 172.21.15.16:0/2666831492 >> v1:172.21.15.16:6789/0 conn(0x7efee80a1340 legacy=0x7efee8133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:29.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.998+0000 7efee759e700 1 -- 172.21.15.16:0/2666831492 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee80a1fe0 con 0x7efee8134170 2023-09-30T22:30:29.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.998+0000 7efee6d9d700 1 --2- 172.21.15.16:0/2666831492 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efee80a0a30 0x7efee80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:30.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.998+0000 7efee759e700 1 --2- 172.21.15.16:0/2666831492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efee8134170 0x7efee813bd30 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7efee0009d90 tx=0x7efee000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49b6092f75534870 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:30.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.999+0000 7efee5d9b700 1 -- 172.21.15.16:0/2666831492 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee000e070 con 0x7efee8134170 2023-09-30T22:30:30.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.999+0000 7efee5d9b700 1 -- 172.21.15.16:0/2666831492 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efee000bb20 con 0x7efee8134170 2023-09-30T22:30:30.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.999+0000 7efee5d9b700 1 -- 172.21.15.16:0/2666831492 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee0012430 con 0x7efee8134170 2023-09-30T22:30:30.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.999+0000 7efeed17a700 1 -- 172.21.15.16:0/2666831492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efee8134170 msgr2=0x7efee813bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:30.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.999+0000 7efeed17a700 1 --2- 172.21.15.16:0/2666831492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efee8134170 0x7efee813bd30 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7efee0009d90 tx=0x7efee000b330 comp rx=0 tx=0).stop 2023-09-30T22:30:30.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.999+0000 7efeed17a700 1 -- 172.21.15.16:0/2666831492 shutdown_connections 2023-09-30T22:30:30.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.999+0000 7efeed17a700 1 --2- 172.21.15.16:0/2666831492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efee8134170 0x7efee813bd30 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:30.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.999+0000 7efeed17a700 1 --2- 172.21.15.16:0/2666831492 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efee80a0a30 0x7efee80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:30.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:29.999+0000 7efeed17a700 1 -- 172.21.15.16:0/2666831492 >> 172.21.15.16:0/2666831492 conn(0x7efee80a6550 msgr2=0x7efee80a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:30.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.000+0000 7efeed17a700 1 -- 172.21.15.16:0/2666831492 shutdown_connections 2023-09-30T22:30:30.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.000+0000 7efeed17a700 1 -- 172.21.15.16:0/2666831492 wait complete. 2023-09-30T22:30:30.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.001+0000 7efeed17a700 1 Processor -- start 2023-09-30T22:30:30.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.001+0000 7efeed17a700 1 -- start start 2023-09-30T22:30:30.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.002+0000 7efeed17a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efee80a0a30 0x7efee81228a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:30.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.002+0000 7efeed17a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efee80a1340 0x7efee8122de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:30.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.002+0000 7efeed17a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efee8134170 0x7efee8123320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:30.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.002+0000 7efeed17a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee8124650 con 0x7efee80a0a30 2023-09-30T22:30:30.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.002+0000 7efeed17a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efee8124420 con 0x7efee8134170 2023-09-30T22:30:30.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.002+0000 7efeed17a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efee8125d90 con 0x7efee80a1340 2023-09-30T22:30:30.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.002+0000 7efee6d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efee80a0a30 0x7efee81228a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:30.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.002+0000 7efee6d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efee80a0a30 0x7efee81228a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55292/0 (socket says 172.21.15.16:55292) 2023-09-30T22:30:30.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.002+0000 7efee6d9d700 1 -- 172.21.15.16:0/373762374 learned_addr learned my addr 172.21.15.16:0/373762374 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:30.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.003+0000 7efee759e700 1 --2- 172.21.15.16:0/373762374 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efee8134170 0x7efee8123320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:30.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.003+0000 7efee659c700 1 --2- 172.21.15.16:0/373762374 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efee80a1340 0x7efee8122de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:30.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.003+0000 7efee6d9d700 1 -- 172.21.15.16:0/373762374 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efee80a1340 msgr2=0x7efee8122de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:30.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.003+0000 7efee6d9d700 1 --2- 172.21.15.16:0/373762374 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efee80a1340 0x7efee8122de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:30.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.003+0000 7efee6d9d700 1 -- 172.21.15.16:0/373762374 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efee8134170 msgr2=0x7efee8123320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:30.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.003+0000 7efee6d9d700 1 --2- 172.21.15.16:0/373762374 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efee8134170 0x7efee8123320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:30.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.003+0000 7efee6d9d700 1 -- 172.21.15.16:0/373762374 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed80094a0 con 0x7efee80a0a30 2023-09-30T22:30:30.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.003+0000 7efee6d9d700 1 --2- 172.21.15.16:0/373762374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efee80a0a30 0x7efee81228a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7efed800ec10 tx=0x7efed800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:30.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.004+0000 7efecf7fe700 1 -- 172.21.15.16:0/373762374 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed800c850 con 0x7efee80a0a30 2023-09-30T22:30:30.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.004+0000 7efecf7fe700 1 -- 172.21.15.16:0/373762374 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efed800c9b0 con 0x7efee80a0a30 2023-09-30T22:30:30.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.004+0000 7efeed17a700 1 -- 172.21.15.16:0/373762374 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efee0009a70 con 0x7efee80a0a30 2023-09-30T22:30:30.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.004+0000 7efecf7fe700 1 -- 172.21.15.16:0/373762374 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed800cd20 con 0x7efee80a0a30 2023-09-30T22:30:30.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.004+0000 7efeed17a700 1 -- 172.21.15.16:0/373762374 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efee80a0220 con 0x7efee80a0a30 2023-09-30T22:30:30.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.005+0000 7efeed17a700 1 -- 172.21.15.16:0/373762374 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efee80a1d00 con 0x7efee80a0a30 2023-09-30T22:30:30.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.006+0000 7efecf7fe700 1 -- 172.21.15.16:0/373762374 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7efed8013470 con 0x7efee80a0a30 2023-09-30T22:30:30.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.006+0000 7efecf7fe700 1 -- 172.21.15.16:0/373762374 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7efed803fc60 con 0x7efee80a0a30 2023-09-30T22:30:30.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.015+0000 7efecf7fe700 1 -- 172.21.15.16:0/373762374 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7efed8013720 con 0x7efee80a0a30 2023-09-30T22:30:30.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.098+0000 7fd2620fc700 1 -- 172.21.15.16:0/3591649239 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd25c09d650 con 0x7fd25c136830 2023-09-30T22:30:30.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.098+0000 7fd243fff700 1 -- 172.21.15.16:0/3591649239 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd250026480 con 0x7fd25c136830 2023-09-30T22:30:30.099 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:30:30.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.100+0000 7fd2620fc700 1 -- 172.21.15.16:0/3591649239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd25c136830 msgr2=0x7fd25c09a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:30.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.100+0000 7fd2620fc700 1 --2- 172.21.15.16:0/3591649239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd25c136830 0x7fd25c09a220 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd250016000 tx=0x7fd250004750 comp rx=0 tx=0).stop 2023-09-30T22:30:30.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.101+0000 7fd2620fc700 1 -- 172.21.15.16:0/3591649239 shutdown_connections 2023-09-30T22:30:30.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.101+0000 7fd2620fc700 1 --2- 172.21.15.16:0/3591649239 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd25c139150 0x7fd25c1cc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:30.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.101+0000 7fd2620fc700 1 --2- 172.21.15.16:0/3591649239 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd25c1338c0 0x7fd25c099ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:30.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.101+0000 7fd2620fc700 1 --2- 172.21.15.16:0/3591649239 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd25c136830 0x7fd25c09a220 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:30.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.101+0000 7fd2620fc700 1 -- 172.21.15.16:0/3591649239 >> 172.21.15.16:0/3591649239 conn(0x7fd25c0a2aa0 msgr2=0x7fd25c09f180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:30.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.101+0000 7fd2620fc700 1 -- 172.21.15.16:0/3591649239 shutdown_connections 2023-09-30T22:30:30.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.101+0000 7fd2620fc700 1 -- 172.21.15.16:0/3591649239 wait complete. 2023-09-30T22:30:30.115 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:30:30.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.165+0000 7febcdefb700 1 -- 172.21.15.16:0/2516429098 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7febc806c360 con 0x7febc809d6f0 2023-09-30T22:30:30.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.165+0000 7febaffff700 1 -- 172.21.15.16:0/2516429098 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7febb8024b00 con 0x7febc809d6f0 2023-09-30T22:30:30.166 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:30:30.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.168+0000 7febcdefb700 1 -- 172.21.15.16:0/2516429098 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7febc809d6f0 msgr2=0x7febc8070740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:30.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.168+0000 7febcdefb700 1 --2- 172.21.15.16:0/2516429098 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7febc809d6f0 0x7febc8070740 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7febb800e9b0 tx=0x7febb800ecc0 comp rx=0 tx=0).stop 2023-09-30T22:30:30.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.168+0000 7febcdefb700 1 -- 172.21.15.16:0/2516429098 shutdown_connections 2023-09-30T22:30:30.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.168+0000 7febcdefb700 1 --2- 172.21.15.16:0/2516429098 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7febc809d6f0 0x7febc8070740 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:30.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.168+0000 7febcdefb700 1 --2- 172.21.15.16:0/2516429098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febc8133f30 0x7febc81c3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:30.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.168+0000 7febcdefb700 1 --2- 172.21.15.16:0/2516429098 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7febc809e000 0x7febc8070c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:30.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.168+0000 7febcdefb700 1 -- 172.21.15.16:0/2516429098 >> 172.21.15.16:0/2516429098 conn(0x7febc8066f50 msgr2=0x7febc812faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:30.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.168+0000 7febcdefb700 1 -- 172.21.15.16:0/2516429098 shutdown_connections 2023-09-30T22:30:30.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.168+0000 7febcdefb700 1 -- 172.21.15.16:0/2516429098 wait complete. 2023-09-30T22:30:30.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.173+0000 7f1edad5d700 1 -- 172.21.15.16:0/1961585738 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1ed406c360 con 0x7f1ed409dc30 2023-09-30T22:30:30.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.174+0000 7f1ed17fa700 1 -- 172.21.15.16:0/1961585738 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1ec00232f0 con 0x7f1ed409dc30 2023-09-30T22:30:30.174 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:30:30.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.176+0000 7f1edad5d700 1 -- 172.21.15.16:0/1961585738 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ed409dc30 msgr2=0x7f1ed413a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:30.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.176+0000 7f1edad5d700 1 --2- 172.21.15.16:0/1961585738 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ed409dc30 0x7f1ed413a290 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1ec000ec10 tx=0x7f1ec000ef20 comp rx=0 tx=0).stop 2023-09-30T22:30:30.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.176+0000 7f1edad5d700 1 -- 172.21.15.16:0/1961585738 shutdown_connections 2023-09-30T22:30:30.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.176+0000 7f1edad5d700 1 --2- 172.21.15.16:0/1961585738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ed413c1e0 0x7f1ed413a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:30.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.176+0000 7f1edad5d700 1 --2- 172.21.15.16:0/1961585738 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ed406a770 0x7f1ed413e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:30.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.176+0000 7f1edad5d700 1 --2- 172.21.15.16:0/1961585738 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ed409dc30 0x7f1ed413a290 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:30.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.176+0000 7f1edad5d700 1 -- 172.21.15.16:0/1961585738 >> 172.21.15.16:0/1961585738 conn(0x7f1ed4066f50 msgr2=0x7f1ed4070af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:30.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.176+0000 7f1edad5d700 1 -- 172.21.15.16:0/1961585738 shutdown_connections 2023-09-30T22:30:30.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.176+0000 7f1edad5d700 1 -- 172.21.15.16:0/1961585738 wait complete. 2023-09-30T22:30:30.184 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:30:30.191 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:30:30.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.243+0000 7efeed17a700 1 -- 172.21.15.16:0/373762374 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7efee8074ea0 con 0x7efee80a0a30 2023-09-30T22:30:30.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.244+0000 7efecf7fe700 1 -- 172.21.15.16:0/373762374 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7efed8021070 con 0x7efee80a0a30 2023-09-30T22:30:30.244 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:30:30.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.246+0000 7efeed17a700 1 -- 172.21.15.16:0/373762374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efee80a0a30 msgr2=0x7efee81228a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:30.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.246+0000 7efeed17a700 1 --2- 172.21.15.16:0/373762374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efee80a0a30 0x7efee81228a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7efed800ec10 tx=0x7efed800ef20 comp rx=0 tx=0).stop 2023-09-30T22:30:30.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.246+0000 7efeed17a700 1 -- 172.21.15.16:0/373762374 shutdown_connections 2023-09-30T22:30:30.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.246+0000 7efeed17a700 1 --2- 172.21.15.16:0/373762374 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efee8134170 0x7efee8123320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:30.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.246+0000 7efeed17a700 1 --2- 172.21.15.16:0/373762374 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efee80a0a30 0x7efee81228a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:30.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.246+0000 7efeed17a700 1 --2- 172.21.15.16:0/373762374 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efee80a1340 0x7efee8122de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:30.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.246+0000 7efeed17a700 1 -- 172.21.15.16:0/373762374 >> 172.21.15.16:0/373762374 conn(0x7efee80a6550 msgr2=0x7efee809ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:30.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.246+0000 7efeed17a700 1 -- 172.21.15.16:0/373762374 shutdown_connections 2023-09-30T22:30:30.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:30.246+0000 7efeed17a700 1 -- 172.21.15.16:0/373762374 wait complete. 2023-09-30T22:30:30.260 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:30:31.116 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:31.185 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:31.192 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:31.261 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:31.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.272+0000 7f3634384700 1 Processor -- start 2023-09-30T22:30:31.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.272+0000 7f3634384700 1 -- start start 2023-09-30T22:30:31.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.273+0000 7f3634384700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f362c09d6f0 0x7f362c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:31.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.273+0000 7f3634384700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f362c09e410 0x7f362c130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:31.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.273+0000 7f3634384700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f362c09f130 con 0x7f362c09e000 2023-09-30T22:30:31.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.273+0000 7f3634384700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f362c09f520 con 0x7f362c09d6f0 2023-09-30T22:30:31.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.273+0000 7f3634384700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f362c09e930 con 0x7f362c09e410 2023-09-30T22:30:31.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.273+0000 7f363191f700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f362c09e000 0x7f362c130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34488/0 (socket says 172.21.15.16:34488) 2023-09-30T22:30:31.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.273+0000 7f363191f700 1 -- 172.21.15.16:0/2424684416 learned_addr learned my addr 172.21.15.16:0/2424684416 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:30:31.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.273+0000 7f3632120700 1 --2- 172.21.15.16:0/2424684416 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f362c09d6f0 0x7f362c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:31.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.273+0000 7f3632921700 1 --2- 172.21.15.16:0/2424684416 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f362c09e410 0x7f362c130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:31.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.274+0000 7f3632120700 1 -- 172.21.15.16:0/2424684416 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f362c09e410 msgr2=0x7f362c130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:31.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.274+0000 7f3632120700 1 --2- 172.21.15.16:0/2424684416 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f362c09e410 0x7f362c130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.274+0000 7f3632120700 1 -- 172.21.15.16:0/2424684416 >> v1:172.21.15.16:6789/0 conn(0x7f362c09e000 legacy=0x7f362c130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:31.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.274+0000 7f3632120700 1 -- 172.21.15.16:0/2424684416 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f362c09b2d0 con 0x7f362c09d6f0 2023-09-30T22:30:31.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.274+0000 7f3632120700 1 --2- 172.21.15.16:0/2424684416 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f362c09d6f0 0x7f362c09dac0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3628009870 tx=0x7f3628009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67c4d797b93a8afa server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:31.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.274+0000 7f363111e700 1 -- 172.21.15.16:0/2424684416 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3628011830 con 0x7f362c09d6f0 2023-09-30T22:30:31.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.274+0000 7f363111e700 1 -- 172.21.15.16:0/2424684416 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3628011990 con 0x7f362c09d6f0 2023-09-30T22:30:31.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.274+0000 7f363111e700 1 -- 172.21.15.16:0/2424684416 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3628011d20 con 0x7f362c09d6f0 2023-09-30T22:30:31.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.275+0000 7f3634384700 1 -- 172.21.15.16:0/2424684416 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f362c09d6f0 msgr2=0x7f362c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:31.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.275+0000 7f3634384700 1 --2- 172.21.15.16:0/2424684416 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f362c09d6f0 0x7f362c09dac0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3628009870 tx=0x7f3628009b80 comp rx=0 tx=0).stop 2023-09-30T22:30:31.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.275+0000 7f3634384700 1 -- 172.21.15.16:0/2424684416 shutdown_connections 2023-09-30T22:30:31.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.275+0000 7f3634384700 1 --2- 172.21.15.16:0/2424684416 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f362c09e410 0x7f362c130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.275+0000 7f3634384700 1 --2- 172.21.15.16:0/2424684416 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f362c09d6f0 0x7f362c09dac0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.276+0000 7f3634384700 1 -- 172.21.15.16:0/2424684416 >> 172.21.15.16:0/2424684416 conn(0x7f362c066f50 msgr2=0x7f362c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:31.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.276+0000 7f3634384700 1 -- 172.21.15.16:0/2424684416 shutdown_connections 2023-09-30T22:30:31.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.276+0000 7f3634384700 1 -- 172.21.15.16:0/2424684416 wait complete. 2023-09-30T22:30:31.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.277+0000 7f3634384700 1 Processor -- start 2023-09-30T22:30:31.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.277+0000 7f3634384700 1 -- start start 2023-09-30T22:30:31.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.278+0000 7f3634384700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f362c09d6f0 0x7f362c120190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:31.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.278+0000 7f3634384700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f362c09e000 0x7f362c1206d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:31.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.278+0000 7f3634384700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f362c09e410 0x7f362c120c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:31.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.278+0000 7f3634384700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362c1223f0 con 0x7f362c09e000 2023-09-30T22:30:31.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.278+0000 7f3634384700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f362c09c700 con 0x7f362c09d6f0 2023-09-30T22:30:31.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.278+0000 7f3634384700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f362c121eb0 con 0x7f362c09e410 2023-09-30T22:30:31.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.278+0000 7f3632120700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f362c09d6f0 0x7f362c120190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:31.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.278+0000 7f3632120700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f362c09d6f0 0x7f362c120190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38276/0 (socket says 172.21.15.16:38276) 2023-09-30T22:30:31.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.278+0000 7f3632120700 1 -- 172.21.15.16:0/179784600 learned_addr learned my addr 172.21.15.16:0/179784600 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:31.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.278+0000 7f363191f700 1 --2- 172.21.15.16:0/179784600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f362c09e000 0x7f362c1206d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:31.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.278+0000 7f3632921700 1 --2- 172.21.15.16:0/179784600 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f362c09e410 0x7f362c120c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:31.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.279+0000 7f3632120700 1 -- 172.21.15.16:0/179784600 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f362c09e410 msgr2=0x7f362c120c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:31.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.279+0000 7f3632120700 1 --2- 172.21.15.16:0/179784600 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f362c09e410 0x7f362c120c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.279+0000 7f3632120700 1 -- 172.21.15.16:0/179784600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f362c09e000 msgr2=0x7f362c1206d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:31.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.279+0000 7f3632120700 1 --2- 172.21.15.16:0/179784600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f362c09e000 0x7f362c1206d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.279+0000 7f3632120700 1 -- 172.21.15.16:0/179784600 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36280094a0 con 0x7f362c09d6f0 2023-09-30T22:30:31.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.279+0000 7f3632921700 1 --2- 172.21.15.16:0/179784600 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f362c09e410 0x7f362c120c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:31.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.279+0000 7f3632120700 1 --2- 172.21.15.16:0/179784600 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f362c09d6f0 0x7f362c120190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3628009840 tx=0x7f3628004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:31.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.280+0000 7f3622ffd700 1 -- 172.21.15.16:0/179784600 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3628004450 con 0x7f362c09d6f0 2023-09-30T22:30:31.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.280+0000 7f3622ffd700 1 -- 172.21.15.16:0/179784600 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f362801f070 con 0x7f362c09d6f0 2023-09-30T22:30:31.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.280+0000 7f3634384700 1 -- 172.21.15.16:0/179784600 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f362c1239d0 con 0x7f362c09d6f0 2023-09-30T22:30:31.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.280+0000 7f3634384700 1 -- 172.21.15.16:0/179784600 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f362c124440 con 0x7f362c09d6f0 2023-09-30T22:30:31.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.280+0000 7f3622ffd700 1 -- 172.21.15.16:0/179784600 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36280038e0 con 0x7f362c09d6f0 2023-09-30T22:30:31.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.281+0000 7f3622ffd700 1 -- 172.21.15.16:0/179784600 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3628003a40 con 0x7f362c09d6f0 2023-09-30T22:30:31.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.281+0000 7f3622ffd700 1 -- 172.21.15.16:0/179784600 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f3628021070 con 0x7f362c09d6f0 2023-09-30T22:30:31.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.285+0000 7f3634384700 1 -- 172.21.15.16:0/179784600 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f362c074ea0 con 0x7f362c09d6f0 2023-09-30T22:30:31.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.291+0000 7f3622ffd700 1 -- 172.21.15.16:0/179784600 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3628015030 con 0x7f362c09d6f0 2023-09-30T22:30:31.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.349+0000 7ff4d7e9e700 1 Processor -- start 2023-09-30T22:30:31.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.350+0000 7ff4d7e9e700 1 -- start start 2023-09-30T22:30:31.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.350+0000 7ff4d7e9e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff4d009d6f0 0x7ff4d009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:31.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.350+0000 7ff4d7e9e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff4d009e000 0x7ff4d01339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:31.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.350+0000 7ff4d7e9e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff4d009e410 0x7ff4d0133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:31.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.350+0000 7ff4d7e9e700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4d006b460 con 0x7ff4d009e000 2023-09-30T22:30:31.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.350+0000 7ff4d7e9e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d009b120 con 0x7ff4d009e410 2023-09-30T22:30:31.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.350+0000 7ff4d7e9e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d009a7e0 con 0x7ff4d009d6f0 2023-09-30T22:30:31.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.350+0000 7ff4d5c3a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff4d009d6f0 0x7ff4d009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:31.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.350+0000 7ff4d5c3a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff4d009d6f0 0x7ff4d009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:54678/0 (socket says 172.21.15.16:54678) 2023-09-30T22:30:31.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.350+0000 7ff4d5c3a700 1 -- 172.21.15.16:0/3028449440 learned_addr learned my addr 172.21.15.16:0/3028449440 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:31.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.351+0000 7ff4d5439700 1 --2- 172.21.15.16:0/3028449440 >> v2:172.21.15.16:3300/0 conn(0x7ff4d009e000 0x7ff4d01339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:31.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.351+0000 7ff4d643b700 1 --2- 172.21.15.16:0/3028449440 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff4d009e410 0x7ff4d0133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:31.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.351+0000 7ff4d5c3a700 1 -- 172.21.15.16:0/3028449440 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff4d009e410 msgr2=0x7ff4d0133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:31.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.351+0000 7ff4d5c3a700 1 --2- 172.21.15.16:0/3028449440 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff4d009e410 0x7ff4d0133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.351+0000 7ff4d5c3a700 1 -- 172.21.15.16:0/3028449440 >> v2:172.21.15.16:3300/0 conn(0x7ff4d009e000 msgr2=0x7ff4d01339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:31.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.351+0000 7ff4d5c3a700 1 --2- 172.21.15.16:0/3028449440 >> v2:172.21.15.16:3300/0 conn(0x7ff4d009e000 0x7ff4d01339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.351+0000 7ff4d5c3a700 1 -- 172.21.15.16:0/3028449440 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4d009a500 con 0x7ff4d009d6f0 2023-09-30T22:30:31.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.351+0000 7ff4d5c3a700 1 --2- 172.21.15.16:0/3028449440 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff4d009d6f0 0x7ff4d009dac0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc009920 tx=0x7ff4cc009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47659b3421cbc8dd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:31.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.351+0000 7ff4d4c38700 1 -- 172.21.15.16:0/3028449440 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc011830 con 0x7ff4d009d6f0 2023-09-30T22:30:31.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.351+0000 7ff4d4c38700 1 -- 172.21.15.16:0/3028449440 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4cc011990 con 0x7ff4d009d6f0 2023-09-30T22:30:31.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.351+0000 7ff4d4c38700 1 -- 172.21.15.16:0/3028449440 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc011d20 con 0x7ff4d009d6f0 2023-09-30T22:30:31.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.352+0000 7ff4d7e9e700 1 -- 172.21.15.16:0/3028449440 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff4d009d6f0 msgr2=0x7ff4d009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:31.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.352+0000 7ff4d7e9e700 1 --2- 172.21.15.16:0/3028449440 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff4d009d6f0 0x7ff4d009dac0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc009920 tx=0x7ff4cc009c30 comp rx=0 tx=0).stop 2023-09-30T22:30:31.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.352+0000 7ff4d7e9e700 1 -- 172.21.15.16:0/3028449440 shutdown_connections 2023-09-30T22:30:31.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.352+0000 7ff4d7e9e700 1 --2- 172.21.15.16:0/3028449440 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff4d009e410 0x7ff4d0133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.352+0000 7ff4d7e9e700 1 --2- 172.21.15.16:0/3028449440 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff4d009d6f0 0x7ff4d009dac0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.352+0000 7ff4d7e9e700 1 --2- 172.21.15.16:0/3028449440 >> v2:172.21.15.16:3300/0 conn(0x7ff4d009e000 0x7ff4d01339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.352+0000 7ff4d7e9e700 1 -- 172.21.15.16:0/3028449440 >> 172.21.15.16:0/3028449440 conn(0x7ff4d0066f50 msgr2=0x7ff4d009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:31.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.353+0000 7ff4d7e9e700 1 -- 172.21.15.16:0/3028449440 shutdown_connections 2023-09-30T22:30:31.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.353+0000 7ff4d7e9e700 1 -- 172.21.15.16:0/3028449440 wait complete. 2023-09-30T22:30:31.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.354+0000 7ff4d7e9e700 1 Processor -- start 2023-09-30T22:30:31.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.354+0000 7ff4d7e9e700 1 -- start start 2023-09-30T22:30:31.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.354+0000 7ff4d7e9e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4d009d6f0 0x7ff4d01a6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:31.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.354+0000 7ff4d7e9e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff4d009e000 0x7ff4d01a6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:31.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.354+0000 7ff4d7e9e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff4d009e410 0x7ff4d01a6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:31.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.354+0000 7ff4d7e9e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d01a9a90 con 0x7ff4d009d6f0 2023-09-30T22:30:31.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.354+0000 7ff4d7e9e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d01a8160 con 0x7ff4d009e410 2023-09-30T22:30:31.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.354+0000 7ff4d7e9e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff4d01a9220 con 0x7ff4d009e000 2023-09-30T22:30:31.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.354+0000 7ff4d5c3a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4d009d6f0 0x7ff4d01a6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:31.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.354+0000 7ff4d5c3a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4d009d6f0 0x7ff4d01a6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55326/0 (socket says 172.21.15.16:55326) 2023-09-30T22:30:31.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.354+0000 7ff4d5c3a700 1 -- 172.21.15.16:0/2757569664 learned_addr learned my addr 172.21.15.16:0/2757569664 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:31.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.354+0000 7ff4d5439700 1 --2- 172.21.15.16:0/2757569664 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff4d009e000 0x7ff4d01a6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:31.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.355+0000 7ff4d5439700 1 -- 172.21.15.16:0/2757569664 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff4d009e410 msgr2=0x7ff4d01a6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:31.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.355+0000 7ff4d5439700 1 --2- 172.21.15.16:0/2757569664 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff4d009e410 0x7ff4d01a6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.355+0000 7ff4d5439700 1 -- 172.21.15.16:0/2757569664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4d009d6f0 msgr2=0x7ff4d01a6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:31.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.355+0000 7ff4d5439700 1 --2- 172.21.15.16:0/2757569664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4d009d6f0 0x7ff4d01a6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.355+0000 7ff4d5439700 1 -- 172.21.15.16:0/2757569664 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4cc009550 con 0x7ff4d009e000 2023-09-30T22:30:31.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.355+0000 7ff4d5439700 1 --2- 172.21.15.16:0/2757569664 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff4d009e000 0x7ff4d01a6540 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff4c000e900 tx=0x7ff4c000ec10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:31.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.355+0000 7ff4c67fc700 1 -- 172.21.15.16:0/2757569664 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c000ee50 con 0x7ff4d009e000 2023-09-30T22:30:31.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.355+0000 7ff4c67fc700 1 -- 172.21.15.16:0/2757569664 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4c000c850 con 0x7ff4d009e000 2023-09-30T22:30:31.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.355+0000 7ff4d7e9e700 1 -- 172.21.15.16:0/2757569664 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4d009c710 con 0x7ff4d009e000 2023-09-30T22:30:31.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.356+0000 7ff4d7e9e700 1 -- 172.21.15.16:0/2757569664 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4d01a7b80 con 0x7ff4d009e000 2023-09-30T22:30:31.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.356+0000 7ff4c67fc700 1 -- 172.21.15.16:0/2757569664 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4c000cbe0 con 0x7ff4d009e000 2023-09-30T22:30:31.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.356+0000 7ff4c67fc700 1 -- 172.21.15.16:0/2757569664 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff4c000cd40 con 0x7ff4d009e000 2023-09-30T22:30:31.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.357+0000 7ff4c67fc700 1 -- 172.21.15.16:0/2757569664 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7ff4c003f620 con 0x7ff4d009e000 2023-09-30T22:30:31.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.357+0000 7ff4d7e9e700 1 -- 172.21.15.16:0/2757569664 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4b80052e0 con 0x7ff4d009e000 2023-09-30T22:30:31.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.362+0000 7ff4c67fc700 1 -- 172.21.15.16:0/2757569664 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff4c0021080 con 0x7ff4d009e000 2023-09-30T22:30:31.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.386+0000 7fc05d04d700 1 Processor -- start 2023-09-30T22:30:31.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.386+0000 7fc05d04d700 1 -- start start 2023-09-30T22:30:31.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.386+0000 7fc05d04d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc05809e1d0 0x7fc058135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:31.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.386+0000 7fc05d04d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc058135e50 0x7fc058138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:31.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.386+0000 7fc05d04d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc05809f130 con 0x7fc05809d6f0 2023-09-30T22:30:31.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.386+0000 7fc05d04d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc05809f520 con 0x7fc05809e1d0 2023-09-30T22:30:31.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.386+0000 7fc05d04d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc05809e930 con 0x7fc058135e50 2023-09-30T22:30:31.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.386+0000 7fc056d9d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc05809d6f0 0x7fc05809dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34492/0 (socket says 172.21.15.16:34492) 2023-09-30T22:30:31.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.386+0000 7fc056d9d700 1 -- 172.21.15.16:0/2598620070 learned_addr learned my addr 172.21.15.16:0/2598620070 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:30:31.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.386+0000 7fc05659c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc05809e1d0 0x7fc058135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:31.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.387+0000 7fc05759e700 1 --2- 172.21.15.16:0/2598620070 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc058135e50 0x7fc058138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:31.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.387+0000 7fc05659c700 1 -- 172.21.15.16:0/2598620070 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc058135e50 msgr2=0x7fc058138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:31.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.387+0000 7fc05659c700 1 --2- 172.21.15.16:0/2598620070 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc058135e50 0x7fc058138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.387+0000 7fc05659c700 1 -- 172.21.15.16:0/2598620070 >> v1:172.21.15.16:6789/0 conn(0x7fc05809d6f0 legacy=0x7fc05809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:31.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.387+0000 7fc05659c700 1 -- 172.21.15.16:0/2598620070 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc05809b2d0 con 0x7fc05809e1d0 2023-09-30T22:30:31.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.387+0000 7fc05659c700 1 --2- 172.21.15.16:0/2598620070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc05809e1d0 0x7fc058135910 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc04c009840 tx=0x7fc04c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e6cb00a8981fc0c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:31.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.387+0000 7fc055d9b700 1 -- 172.21.15.16:0/2598620070 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04c011830 con 0x7fc05809e1d0 2023-09-30T22:30:31.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.387+0000 7fc055d9b700 1 -- 172.21.15.16:0/2598620070 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc04c011990 con 0x7fc05809e1d0 2023-09-30T22:30:31.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.387+0000 7fc055d9b700 1 -- 172.21.15.16:0/2598620070 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc04c011d20 con 0x7fc05809e1d0 2023-09-30T22:30:31.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.388+0000 7fc05d04d700 1 -- 172.21.15.16:0/2598620070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc05809e1d0 msgr2=0x7fc058135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:31.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.388+0000 7fc05d04d700 1 --2- 172.21.15.16:0/2598620070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc05809e1d0 0x7fc058135910 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc04c009840 tx=0x7fc04c009b50 comp rx=0 tx=0).stop 2023-09-30T22:30:31.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.388+0000 7fc05d04d700 1 -- 172.21.15.16:0/2598620070 shutdown_connections 2023-09-30T22:30:31.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.388+0000 7fc05d04d700 1 --2- 172.21.15.16:0/2598620070 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc058135e50 0x7fc058138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.388+0000 7fc05d04d700 1 --2- 172.21.15.16:0/2598620070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc05809e1d0 0x7fc058135910 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.388+0000 7fc05d04d700 1 -- 172.21.15.16:0/2598620070 >> 172.21.15.16:0/2598620070 conn(0x7fc058066f50 msgr2=0x7fc05809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:31.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.389+0000 7fc05d04d700 1 -- 172.21.15.16:0/2598620070 shutdown_connections 2023-09-30T22:30:31.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.389+0000 7fc05d04d700 1 -- 172.21.15.16:0/2598620070 wait complete. 2023-09-30T22:30:31.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.390+0000 7fc05d04d700 1 Processor -- start 2023-09-30T22:30:31.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.391+0000 7fc05d04d700 1 -- start start 2023-09-30T22:30:31.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.391+0000 7fc05d04d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc05809d6f0 0x7fc0581c4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:31.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.391+0000 7fc05d04d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc05809e1d0 0x7fc0581c4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:31.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.391+0000 7fc05d04d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc058135e50 0x7fc0581194f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:31.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.392+0000 7fc05d04d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0581baf30 con 0x7fc05809d6f0 2023-09-30T22:30:31.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.392+0000 7fc05d04d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc05811cd50 con 0x7fc058135e50 2023-09-30T22:30:31.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.392+0000 7fc05d04d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc058130260 con 0x7fc05809e1d0 2023-09-30T22:30:31.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.392+0000 7fc056d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc05809d6f0 0x7fc0581c4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:31.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.392+0000 7fc056d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc05809d6f0 0x7fc0581c4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55336/0 (socket says 172.21.15.16:55336) 2023-09-30T22:30:31.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.392+0000 7fc056d9d700 1 -- 172.21.15.16:0/3599945571 learned_addr learned my addr 172.21.15.16:0/3599945571 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:31.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.392+0000 7fc05659c700 1 --2- 172.21.15.16:0/3599945571 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc05809e1d0 0x7fc0581c4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:31.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.392+0000 7fc05759e700 1 --2- 172.21.15.16:0/3599945571 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc058135e50 0x7fc0581194f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:31.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.392+0000 7fc056d9d700 1 -- 172.21.15.16:0/3599945571 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc05809e1d0 msgr2=0x7fc0581c4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:31.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.392+0000 7fc056d9d700 1 --2- 172.21.15.16:0/3599945571 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc05809e1d0 0x7fc0581c4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.392+0000 7fc056d9d700 1 -- 172.21.15.16:0/3599945571 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc058135e50 msgr2=0x7fc0581194f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:31.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.392+0000 7fc056d9d700 1 --2- 172.21.15.16:0/3599945571 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc058135e50 0x7fc0581194f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.392+0000 7fc056d9d700 1 -- 172.21.15.16:0/3599945571 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc04c0094a0 con 0x7fc05809d6f0 2023-09-30T22:30:31.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.392+0000 7fc056d9d700 1 --2- 172.21.15.16:0/3599945571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc05809d6f0 0x7fc0581c4150 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc04800c480 tx=0x7fc04800c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:31.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.393+0000 7fc03f7fe700 1 -- 172.21.15.16:0/3599945571 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc048005d70 con 0x7fc05809d6f0 2023-09-30T22:30:31.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.393+0000 7fc03f7fe700 1 -- 172.21.15.16:0/3599945571 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc04800ce90 con 0x7fc05809d6f0 2023-09-30T22:30:31.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.393+0000 7fc03f7fe700 1 -- 172.21.15.16:0/3599945571 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0480072d0 con 0x7fc05809d6f0 2023-09-30T22:30:31.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.393+0000 7fc05d04d700 1 -- 172.21.15.16:0/3599945571 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc05811b0f0 con 0x7fc05809d6f0 2023-09-30T22:30:31.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.393+0000 7fc05d04d700 1 -- 172.21.15.16:0/3599945571 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc05811b880 con 0x7fc05809d6f0 2023-09-30T22:30:31.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.394+0000 7fc03f7fe700 1 -- 172.21.15.16:0/3599945571 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc048007430 con 0x7fc05809d6f0 2023-09-30T22:30:31.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.394+0000 7fc03f7fe700 1 -- 172.21.15.16:0/3599945571 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fc048012780 con 0x7fc05809d6f0 2023-09-30T22:30:31.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.394+0000 7fc05d04d700 1 -- 172.21.15.16:0/3599945571 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc05811b090 con 0x7fc05809d6f0 2023-09-30T22:30:31.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.399+0000 7fc03f7fe700 1 -- 172.21.15.16:0/3599945571 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc048044060 con 0x7fc05809d6f0 2023-09-30T22:30:31.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.425+0000 7f0df09c3700 1 Processor -- start 2023-09-30T22:30:31.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.425+0000 7f0df09c3700 1 -- start start 2023-09-30T22:30:31.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.426+0000 7f0df09c3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0dec09d6f0 0x7f0dec09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:31.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.426+0000 7f0df09c3700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0dec09e000 0x7f0dec1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:31.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.426+0000 7f0df09c3700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dec09b120 con 0x7f0dec09e410 2023-09-30T22:30:31.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.426+0000 7f0df09c3700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dec06b460 con 0x7f0dec09e000 2023-09-30T22:30:31.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.426+0000 7f0df09c3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dec09a7e0 con 0x7f0dec09d6f0 2023-09-30T22:30:31.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.426+0000 7f0dead9d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0dec09e410 0x7f0dec133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34506/0 (socket says 172.21.15.16:34506) 2023-09-30T22:30:31.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.426+0000 7f0dead9d700 1 -- 172.21.15.16:0/1214818451 learned_addr learned my addr 172.21.15.16:0/1214818451 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:30:31.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.427+0000 7f0dea59c700 1 --2- 172.21.15.16:0/1214818451 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0dec09d6f0 0x7f0dec09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:31.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.427+0000 7f0de959a700 1 -- 172.21.15.16:0/1214818451 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2716990768 0 0) 0x7f0dec09b120 con 0x7f0dec09e410 2023-09-30T22:30:31.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.427+0000 7f0de959a700 1 -- 172.21.15.16:0/1214818451 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0dd40035b0 con 0x7f0dec09e410 2023-09-30T22:30:31.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.427+0000 7f0de9d9b700 1 --2- 172.21.15.16:0/1214818451 >> v2:172.21.15.16:3300/0 conn(0x7f0dec09e000 0x7f0dec1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:31.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.427+0000 7f0de959a700 1 -- 172.21.15.16:0/1214818451 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1988732940 0 0) 0x7f0dd40035b0 con 0x7f0dec09e410 2023-09-30T22:30:31.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.427+0000 7f0de959a700 1 -- 172.21.15.16:0/1214818451 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0dec09d6f0 msgr2=0x7f0dec09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:31.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.427+0000 7f0de959a700 1 --2- 172.21.15.16:0/1214818451 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0dec09d6f0 0x7f0dec09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.427+0000 7f0de959a700 1 -- 172.21.15.16:0/1214818451 >> v2:172.21.15.16:3300/0 conn(0x7f0dec09e000 msgr2=0x7f0dec1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:31.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.427+0000 7f0de959a700 1 --2- 172.21.15.16:0/1214818451 >> v2:172.21.15.16:3300/0 conn(0x7f0dec09e000 0x7f0dec1339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.427+0000 7f0de959a700 1 -- 172.21.15.16:0/1214818451 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dec09a500 con 0x7f0dec09e410 2023-09-30T22:30:31.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.427+0000 7f0de959a700 1 -- 172.21.15.16:0/1214818451 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1074163509 0 0) 0x7f0de40019f0 con 0x7f0dec09e410 2023-09-30T22:30:31.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.427+0000 7f0de959a700 1 -- 172.21.15.16:0/1214818451 >> v1:172.21.15.16:6789/0 conn(0x7f0dec09e410 legacy=0x7f0dec133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:31.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.427+0000 7f0de959a700 1 --2- 172.21.15.16:0/1214818451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dd4003a30 0x7f0dd4003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:31.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.427+0000 7f0de959a700 1 --2- 172.21.15.16:0/1214818451 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0dec09d6f0 0x7f0dd4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:31.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.427+0000 7f0de959a700 1 --2- 172.21.15.16:0/1214818451 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0dd4004d90 0x7f0dd4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:31.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.427+0000 7f0de959a700 1 -- 172.21.15.16:0/1214818451 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dec06b460 con 0x7f0dd4003a30 2023-09-30T22:30:31.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.427+0000 7f0de959a700 1 -- 172.21.15.16:0/1214818451 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dec09a7e0 con 0x7f0dec09d6f0 2023-09-30T22:30:31.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.427+0000 7f0de959a700 1 -- 172.21.15.16:0/1214818451 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd400ba10 con 0x7f0dd4004d90 2023-09-30T22:30:31.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.427+0000 7f0de9d9b700 1 --2- 172.21.15.16:0/1214818451 >> v2:172.21.15.16:3300/0 conn(0x7f0dec09e000 0x7f0dec1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:30:31.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.427+0000 7f0dea59c700 1 --2- 172.21.15.16:0/1214818451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dd4003a30 0x7f0dd4003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:31.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.427+0000 7f0dead9d700 1 --2- 172.21.15.16:0/1214818451 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0dd4004d90 0x7f0dd4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:31.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.427+0000 7f0dea59c700 1 -- 172.21.15.16:0/1214818451 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0dd4004d90 msgr2=0x7f0dd4005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:31.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.427+0000 7f0dea59c700 1 --2- 172.21.15.16:0/1214818451 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0dd4004d90 0x7f0dd4005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.427+0000 7f0dea59c700 1 -- 172.21.15.16:0/1214818451 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0dec09d6f0 msgr2=0x7f0dd4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:31.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.427+0000 7f0dea59c700 1 --2- 172.21.15.16:0/1214818451 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0dec09d6f0 0x7f0dd4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.427+0000 7f0dea59c700 1 -- 172.21.15.16:0/1214818451 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dec09a500 con 0x7f0dd4003a30 2023-09-30T22:30:31.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.428+0000 7f0dea59c700 1 --2- 172.21.15.16:0/1214818451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dd4003a30 0x7f0dd4003e40 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc009b90 tx=0x7f0dd4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9a8029c83e991f4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:31.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.428+0000 7f0de959a700 1 -- 172.21.15.16:0/1214818451 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc00e660 con 0x7f0dd4003a30 2023-09-30T22:30:31.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.428+0000 7f0de959a700 1 -- 172.21.15.16:0/1214818451 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ddc00e7c0 con 0x7f0dd4003a30 2023-09-30T22:30:31.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.428+0000 7f0de959a700 1 -- 172.21.15.16:0/1214818451 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc00eb30 con 0x7f0dd4003a30 2023-09-30T22:30:31.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.429+0000 7f0df09c3700 1 -- 172.21.15.16:0/1214818451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dd4003a30 msgr2=0x7f0dd4003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:31.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.429+0000 7f0df09c3700 1 --2- 172.21.15.16:0/1214818451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dd4003a30 0x7f0dd4003e40 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc009b90 tx=0x7f0dd4005880 comp rx=0 tx=0).stop 2023-09-30T22:30:31.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.429+0000 7f0dead9d700 1 -- 172.21.15.16:0/1214818451 reap_dead start 2023-09-30T22:30:31.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.429+0000 7f0df09c3700 1 -- 172.21.15.16:0/1214818451 shutdown_connections 2023-09-30T22:30:31.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.429+0000 7f0df09c3700 1 -- 172.21.15.16:0/1214818451 >> 172.21.15.16:0/1214818451 conn(0x7f0dec066f50 msgr2=0x7f0dec09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:31.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.429+0000 7f0df09c3700 1 -- 172.21.15.16:0/1214818451 shutdown_connections 2023-09-30T22:30:31.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.429+0000 7f0df09c3700 1 -- 172.21.15.16:0/1214818451 wait complete. 2023-09-30T22:30:31.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.430+0000 7f0df09c3700 1 Processor -- start 2023-09-30T22:30:31.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.430+0000 7f0df09c3700 1 -- start start 2023-09-30T22:30:31.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.431+0000 7f0df09c3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0dd4003a30 0x7f0dec1c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:31.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.431+0000 7f0df09c3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0dec1c8580 0x7f0dec1c8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:31.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.431+0000 7f0df09c3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dec1b8230 0x7f0dec1b86a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:31.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.431+0000 7f0df09c3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dec1c8ed0 con 0x7f0dec1b8230 2023-09-30T22:30:31.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.431+0000 7f0df09c3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dec1a6540 con 0x7f0dec1c8580 2023-09-30T22:30:31.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.431+0000 7f0df09c3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0dec1a7d30 con 0x7f0dd4003a30 2023-09-30T22:30:31.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.431+0000 7f0dead9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dec1b8230 0x7f0dec1b86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:31.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.431+0000 7f0dead9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dec1b8230 0x7f0dec1b86a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55352/0 (socket says 172.21.15.16:55352) 2023-09-30T22:30:31.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.431+0000 7f0dead9d700 1 -- 172.21.15.16:0/1069128343 learned_addr learned my addr 172.21.15.16:0/1069128343 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:31.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.431+0000 7f0dea59c700 1 --2- 172.21.15.16:0/1069128343 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0dd4003a30 0x7f0dec1c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:31.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.431+0000 7f0dead9d700 1 -- 172.21.15.16:0/1069128343 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0dd4003a30 msgr2=0x7f0dec1c8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:31.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.431+0000 7f0dead9d700 1 --2- 172.21.15.16:0/1069128343 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0dd4003a30 0x7f0dec1c8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.431+0000 7f0dead9d700 1 -- 172.21.15.16:0/1069128343 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0dec1c8580 msgr2=0x7f0dec1c8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:31.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.431+0000 7f0de9d9b700 1 --2- 172.21.15.16:0/1069128343 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0dec1c8580 0x7f0dec1c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:31.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.431+0000 7f0dead9d700 1 --2- 172.21.15.16:0/1069128343 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0dec1c8580 0x7f0dec1c8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.431+0000 7f0dead9d700 1 -- 172.21.15.16:0/1069128343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dec0a1e90 con 0x7f0dec1b8230 2023-09-30T22:30:31.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.431+0000 7f0de9d9b700 1 --2- 172.21.15.16:0/1069128343 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0dec1c8580 0x7f0dec1c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:30:31.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.432+0000 7f0dead9d700 1 --2- 172.21.15.16:0/1069128343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dec1b8230 0x7f0dec1b86a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0dec06dfd0 tx=0x7f0de4007c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:31.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.432+0000 7f0ddaffd700 1 -- 172.21.15.16:0/1069128343 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de400ed30 con 0x7f0dec1b8230 2023-09-30T22:30:31.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.432+0000 7f0ddaffd700 1 -- 172.21.15.16:0/1069128343 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0de400ee90 con 0x7f0dec1b8230 2023-09-30T22:30:31.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.432+0000 7f0df09c3700 1 -- 172.21.15.16:0/1069128343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dec09c710 con 0x7f0dec1b8230 2023-09-30T22:30:31.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.432+0000 7f0ddaffd700 1 -- 172.21.15.16:0/1069128343 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de4013660 con 0x7f0dec1b8230 2023-09-30T22:30:31.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.432+0000 7f0df09c3700 1 -- 172.21.15.16:0/1069128343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dec1b9260 con 0x7f0dec1b8230 2023-09-30T22:30:31.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.433+0000 7f0ddaffd700 1 -- 172.21.15.16:0/1069128343 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0de40137c0 con 0x7f0dec1b8230 2023-09-30T22:30:31.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.433+0000 7f0ddaffd700 1 -- 172.21.15.16:0/1069128343 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f0de40416e0 con 0x7f0dec1b8230 2023-09-30T22:30:31.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.434+0000 7f0df09c3700 1 -- 172.21.15.16:0/1069128343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dec1a7ef0 con 0x7f0dec1b8230 2023-09-30T22:30:31.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.439+0000 7f0ddaffd700 1 -- 172.21.15.16:0/1069128343 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0de4045060 con 0x7f0dec1b8230 2023-09-30T22:30:31.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.579+0000 7ff4d7e9e700 1 -- 172.21.15.16:0/2757569664 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff4b8005590 con 0x7ff4d009e000 2023-09-30T22:30:31.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.579+0000 7ff4c67fc700 1 -- 172.21.15.16:0/2757569664 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff4c001bdc0 con 0x7ff4d009e000 2023-09-30T22:30:31.580 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:30:31.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.583+0000 7ff4d7e9e700 1 -- 172.21.15.16:0/2757569664 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff4d009e000 msgr2=0x7ff4d01a6540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:31.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.583+0000 7ff4d7e9e700 1 --2- 172.21.15.16:0/2757569664 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff4d009e000 0x7ff4d01a6540 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff4c000e900 tx=0x7ff4c000ec10 comp rx=0 tx=0).stop 2023-09-30T22:30:31.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.583+0000 7ff4d7e9e700 1 -- 172.21.15.16:0/2757569664 shutdown_connections 2023-09-30T22:30:31.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.583+0000 7ff4d7e9e700 1 --2- 172.21.15.16:0/2757569664 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff4d009e410 0x7ff4d01a6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.583+0000 7ff4d7e9e700 1 --2- 172.21.15.16:0/2757569664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4d009d6f0 0x7ff4d01a6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.583+0000 7ff4d7e9e700 1 --2- 172.21.15.16:0/2757569664 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff4d009e000 0x7ff4d01a6540 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.583+0000 7ff4d7e9e700 1 -- 172.21.15.16:0/2757569664 >> 172.21.15.16:0/2757569664 conn(0x7ff4d0066f50 msgr2=0x7ff4d01bafa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:31.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.583+0000 7ff4d7e9e700 1 -- 172.21.15.16:0/2757569664 shutdown_connections 2023-09-30T22:30:31.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.583+0000 7ff4d7e9e700 1 -- 172.21.15.16:0/2757569664 wait complete. 2023-09-30T22:30:31.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.598+0000 7f3634384700 1 -- 172.21.15.16:0/179784600 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f362c06c360 con 0x7f362c09d6f0 2023-09-30T22:30:31.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.598+0000 7f3622ffd700 1 -- 172.21.15.16:0/179784600 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3628045060 con 0x7f362c09d6f0 2023-09-30T22:30:31.599 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:30:31.602 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:30:31.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.602+0000 7f3634384700 1 -- 172.21.15.16:0/179784600 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f362c09d6f0 msgr2=0x7f362c120190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:31.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.602+0000 7f3634384700 1 --2- 172.21.15.16:0/179784600 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f362c09d6f0 0x7f362c120190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3628009840 tx=0x7f3628004750 comp rx=0 tx=0).stop 2023-09-30T22:30:31.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.602+0000 7f3634384700 1 -- 172.21.15.16:0/179784600 shutdown_connections 2023-09-30T22:30:31.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.602+0000 7f3634384700 1 --2- 172.21.15.16:0/179784600 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f362c09d6f0 0x7f362c120190 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.602+0000 7f3634384700 1 --2- 172.21.15.16:0/179784600 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f362c09e410 0x7f362c120c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.602+0000 7f3634384700 1 --2- 172.21.15.16:0/179784600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f362c09e000 0x7f362c1206d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.602+0000 7f3634384700 1 -- 172.21.15.16:0/179784600 >> 172.21.15.16:0/179784600 conn(0x7f362c066f50 msgr2=0x7f362c06a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:31.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.603+0000 7f3634384700 1 -- 172.21.15.16:0/179784600 shutdown_connections 2023-09-30T22:30:31.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.603+0000 7f3634384700 1 -- 172.21.15.16:0/179784600 wait complete. 2023-09-30T22:30:31.621 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:30:31.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.656+0000 7f0df09c3700 1 -- 172.21.15.16:0/1069128343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f0dec074ea0 con 0x7f0dec1b8230 2023-09-30T22:30:31.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.656+0000 7f0ddaffd700 1 -- 172.21.15.16:0/1069128343 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0de401a900 con 0x7f0dec1b8230 2023-09-30T22:30:31.657 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:30:31.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.658+0000 7f0df09c3700 1 -- 172.21.15.16:0/1069128343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dec1b8230 msgr2=0x7f0dec1b86a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:31.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.658+0000 7f0df09c3700 1 --2- 172.21.15.16:0/1069128343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dec1b8230 0x7f0dec1b86a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0dec06dfd0 tx=0x7f0de4007c90 comp rx=0 tx=0).stop 2023-09-30T22:30:31.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.658+0000 7f0df09c3700 1 -- 172.21.15.16:0/1069128343 shutdown_connections 2023-09-30T22:30:31.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.658+0000 7f0df09c3700 1 --2- 172.21.15.16:0/1069128343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dec1b8230 0x7f0dec1b86a0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.658+0000 7f0df09c3700 1 --2- 172.21.15.16:0/1069128343 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0dd4003a30 0x7f0dec1c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.658+0000 7f0df09c3700 1 --2- 172.21.15.16:0/1069128343 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0dec1c8580 0x7f0dec1c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.658+0000 7f0df09c3700 1 -- 172.21.15.16:0/1069128343 >> 172.21.15.16:0/1069128343 conn(0x7f0dec066f50 msgr2=0x7f0dec1bbb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:31.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.658+0000 7f0df09c3700 1 -- 172.21.15.16:0/1069128343 shutdown_connections 2023-09-30T22:30:31.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.658+0000 7f0df09c3700 1 -- 172.21.15.16:0/1069128343 wait complete. 2023-09-30T22:30:31.672 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:30:31.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.688+0000 7fc05d04d700 1 -- 172.21.15.16:0/3599945571 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc058074ea0 con 0x7fc05809d6f0 2023-09-30T22:30:31.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.688+0000 7fc03f7fe700 1 -- 172.21.15.16:0/3599945571 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc048017020 con 0x7fc05809d6f0 2023-09-30T22:30:31.689 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:30:31.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.690+0000 7fc05d04d700 1 -- 172.21.15.16:0/3599945571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc05809d6f0 msgr2=0x7fc0581c4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:31.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.690+0000 7fc05d04d700 1 --2- 172.21.15.16:0/3599945571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc05809d6f0 0x7fc0581c4150 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc04800c480 tx=0x7fc04800c790 comp rx=0 tx=0).stop 2023-09-30T22:30:31.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.691+0000 7fc05d04d700 1 -- 172.21.15.16:0/3599945571 shutdown_connections 2023-09-30T22:30:31.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.691+0000 7fc05d04d700 1 --2- 172.21.15.16:0/3599945571 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc058135e50 0x7fc0581194f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.691+0000 7fc05d04d700 1 --2- 172.21.15.16:0/3599945571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc05809d6f0 0x7fc0581c4150 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.691+0000 7fc05d04d700 1 --2- 172.21.15.16:0/3599945571 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc05809e1d0 0x7fc0581c4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:31.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.691+0000 7fc05d04d700 1 -- 172.21.15.16:0/3599945571 >> 172.21.15.16:0/3599945571 conn(0x7fc058066f50 msgr2=0x7fc05806f140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:31.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.691+0000 7fc05d04d700 1 -- 172.21.15.16:0/3599945571 shutdown_connections 2023-09-30T22:30:31.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:31.691+0000 7fc05d04d700 1 -- 172.21.15.16:0/3599945571 wait complete. 2023-09-30T22:30:31.705 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:30:32.603 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:32.623 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:32.673 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:32.707 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:32.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.762+0000 7fd0500e7700 1 Processor -- start 2023-09-30T22:30:32.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.762+0000 7fd0500e7700 1 -- start start 2023-09-30T22:30:32.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.763+0000 7fd0500e7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd04806a770 0x7fd04809d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:32.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.763+0000 7fd0500e7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd04809dc30 0x7fd04809e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:32.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.763+0000 7fd0500e7700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd04809a810 con 0x7fd04813c1e0 2023-09-30T22:30:32.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.763+0000 7fd0500e7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd04806ab40 con 0x7fd04806a770 2023-09-30T22:30:32.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.763+0000 7fd0500e7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd04809ebc0 con 0x7fd04809dc30 2023-09-30T22:30:32.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.763+0000 7fd04d682700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd04809dc30 0x7fd04809e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:32.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.763+0000 7fd04d682700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd04809dc30 0x7fd04809e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:54718/0 (socket says 172.21.15.16:54718) 2023-09-30T22:30:32.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.763+0000 7fd04d682700 1 -- 172.21.15.16:0/2368923418 learned_addr learned my addr 172.21.15.16:0/2368923418 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:32.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.764+0000 7fd04de83700 1 --2- 172.21.15.16:0/2368923418 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd04806a770 0x7fd04809d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:32.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.764+0000 7fd04d682700 1 -- 172.21.15.16:0/2368923418 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd04806a770 msgr2=0x7fd04809d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:32.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.764+0000 7fd04d682700 1 --2- 172.21.15.16:0/2368923418 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd04806a770 0x7fd04809d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:32.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.764+0000 7fd04d682700 1 -- 172.21.15.16:0/2368923418 >> v1:172.21.15.16:6789/0 conn(0x7fd04813c1e0 legacy=0x7fd04813e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:32.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.764+0000 7fd04d682700 1 -- 172.21.15.16:0/2368923418 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd04809e920 con 0x7fd04809dc30 2023-09-30T22:30:32.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.764+0000 7fd04d682700 1 --2- 172.21.15.16:0/2368923418 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd04809dc30 0x7fd04809e080 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd038009950 tx=0x7fd038009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef795648503dd23c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:32.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.765+0000 7fd04ce81700 1 -- 172.21.15.16:0/2368923418 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd038011830 con 0x7fd04809dc30 2023-09-30T22:30:32.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.765+0000 7fd04ce81700 1 -- 172.21.15.16:0/2368923418 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd038011990 con 0x7fd04809dc30 2023-09-30T22:30:32.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.765+0000 7fd04ce81700 1 -- 172.21.15.16:0/2368923418 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd038011d40 con 0x7fd04809dc30 2023-09-30T22:30:32.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.765+0000 7fd0500e7700 1 -- 172.21.15.16:0/2368923418 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd04809dc30 msgr2=0x7fd04809e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:32.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.765+0000 7fd0500e7700 1 --2- 172.21.15.16:0/2368923418 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd04809dc30 0x7fd04809e080 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd038009950 tx=0x7fd038009c60 comp rx=0 tx=0).stop 2023-09-30T22:30:32.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.766+0000 7fd0500e7700 1 -- 172.21.15.16:0/2368923418 shutdown_connections 2023-09-30T22:30:32.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.766+0000 7fd0500e7700 1 --2- 172.21.15.16:0/2368923418 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd04806a770 0x7fd04809d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:32.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.766+0000 7fd0500e7700 1 --2- 172.21.15.16:0/2368923418 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd04809dc30 0x7fd04809e080 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:32.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.766+0000 7fd0500e7700 1 -- 172.21.15.16:0/2368923418 >> 172.21.15.16:0/2368923418 conn(0x7fd048066f50 msgr2=0x7fd04809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:32.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.766+0000 7fd0500e7700 1 -- 172.21.15.16:0/2368923418 shutdown_connections 2023-09-30T22:30:32.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.766+0000 7fd0500e7700 1 -- 172.21.15.16:0/2368923418 wait complete. 2023-09-30T22:30:32.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.768+0000 7fd0500e7700 1 Processor -- start 2023-09-30T22:30:32.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.768+0000 7fd0500e7700 1 -- start start 2023-09-30T22:30:32.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.769+0000 7fd0500e7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd04806a770 0x7fd04813e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:32.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.769+0000 7fd0500e7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd04809dc30 0x7fd04813a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:32.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.769+0000 7fd0500e7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd04813c1e0 0x7fd04813ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:32.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.769+0000 7fd0500e7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd048140510 con 0x7fd04809dc30 2023-09-30T22:30:32.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.769+0000 7fd0500e7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd048071eb0 con 0x7fd04813c1e0 2023-09-30T22:30:32.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.769+0000 7fd0500e7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd04813c040 con 0x7fd04806a770 2023-09-30T22:30:32.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.769+0000 7fd04d682700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd04809dc30 0x7fd04813a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:32.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.769+0000 7fd04d682700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd04809dc30 0x7fd04813a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55362/0 (socket says 172.21.15.16:55362) 2023-09-30T22:30:32.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.769+0000 7fd04d682700 1 -- 172.21.15.16:0/2676695369 learned_addr learned my addr 172.21.15.16:0/2676695369 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:32.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.769+0000 7fd04e684700 1 --2- 172.21.15.16:0/2676695369 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd04813c1e0 0x7fd04813ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:32.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.769+0000 7fd04d682700 1 -- 172.21.15.16:0/2676695369 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd04806a770 msgr2=0x7fd04813e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:32.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.769+0000 7fd04d682700 1 --2- 172.21.15.16:0/2676695369 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd04806a770 0x7fd04813e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:32.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.769+0000 7fd04d682700 1 -- 172.21.15.16:0/2676695369 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd04813c1e0 msgr2=0x7fd04813ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:32.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.769+0000 7fd04d682700 1 --2- 172.21.15.16:0/2676695369 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd04813c1e0 0x7fd04813ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:32.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.769+0000 7fd04d682700 1 -- 172.21.15.16:0/2676695369 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd038009550 con 0x7fd04809dc30 2023-09-30T22:30:32.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.769+0000 7fd04e684700 1 --2- 172.21.15.16:0/2676695369 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd04813c1e0 0x7fd04813ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:32.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.770+0000 7fd04d682700 1 --2- 172.21.15.16:0/2676695369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd04809dc30 0x7fd04813a820 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd04809eff0 tx=0x7fd038004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:32.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.770+0000 7fd03e7fc700 1 -- 172.21.15.16:0/2676695369 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd038004450 con 0x7fd04809dc30 2023-09-30T22:30:32.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.770+0000 7fd03e7fc700 1 -- 172.21.15.16:0/2676695369 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0380045b0 con 0x7fd04809dc30 2023-09-30T22:30:32.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.770+0000 7fd03e7fc700 1 -- 172.21.15.16:0/2676695369 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0380038e0 con 0x7fd04809dc30 2023-09-30T22:30:32.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.770+0000 7fd0500e7700 1 -- 172.21.15.16:0/2676695369 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd04813f0c0 con 0x7fd04809dc30 2023-09-30T22:30:32.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.770+0000 7fd0500e7700 1 -- 172.21.15.16:0/2676695369 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd04811e8b0 con 0x7fd04809dc30 2023-09-30T22:30:32.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.771+0000 7fd03e7fc700 1 -- 172.21.15.16:0/2676695369 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd038003a40 con 0x7fd04809dc30 2023-09-30T22:30:32.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.771+0000 7fd03e7fc700 1 -- 172.21.15.16:0/2676695369 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd038040a30 con 0x7fd04809dc30 2023-09-30T22:30:32.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.771+0000 7fd0500e7700 1 -- 172.21.15.16:0/2676695369 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd048140680 con 0x7fd04809dc30 2023-09-30T22:30:32.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.777+0000 7fd03e7fc700 1 -- 172.21.15.16:0/2676695369 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd03801a420 con 0x7fd04809dc30 2023-09-30T22:30:32.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.812+0000 7f1edba66700 1 Processor -- start 2023-09-30T22:30:32.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.812+0000 7f1edba66700 1 -- start start 2023-09-30T22:30:32.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.813+0000 7f1edba66700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ed409e930 0x7f1ed409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:32.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.813+0000 7f1edba66700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1ed409f240 0x7f1ed41317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:32.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.813+0000 7f1edba66700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ed409f650 0x7f1ed4131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:32.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.813+0000 7f1edba66700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ed4070940 con 0x7f1ed409f240 2023-09-30T22:30:32.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.813+0000 7f1edba66700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed406fd60 con 0x7f1ed409f650 2023-09-30T22:30:32.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.813+0000 7f1edba66700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed406fa80 con 0x7f1ed409e930 2023-09-30T22:30:32.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.814+0000 7f1ed9802700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ed409e930 0x7f1ed409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:32.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.814+0000 7f1ed9001700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1ed409f240 0x7f1ed41317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:32.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.814+0000 7f1ed9802700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ed409e930 0x7f1ed409ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:54730/0 (socket says 172.21.15.16:54730) 2023-09-30T22:30:32.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.814+0000 7f1ed9802700 1 -- 172.21.15.16:0/1135055767 learned_addr learned my addr 172.21.15.16:0/1135055767 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:32.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.814+0000 7f1eda003700 1 --2- 172.21.15.16:0/1135055767 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ed409f650 0x7f1ed4131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:32.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.814+0000 7f1ed9802700 1 -- 172.21.15.16:0/1135055767 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ed409f650 msgr2=0x7f1ed4131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:32.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.814+0000 7f1ed9802700 1 --2- 172.21.15.16:0/1135055767 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ed409f650 0x7f1ed4131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:32.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.814+0000 7f1ed9802700 1 -- 172.21.15.16:0/1135055767 >> v2:172.21.15.16:3300/0 conn(0x7f1ed409f240 msgr2=0x7f1ed41317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:32.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.814+0000 7f1ed9802700 1 --2- 172.21.15.16:0/1135055767 >> v2:172.21.15.16:3300/0 conn(0x7f1ed409f240 0x7f1ed41317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:32.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.814+0000 7f1ed9802700 1 -- 172.21.15.16:0/1135055767 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ed409fb60 con 0x7f1ed409e930 2023-09-30T22:30:32.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.814+0000 7f1ed9802700 1 --2- 172.21.15.16:0/1135055767 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ed409e930 0x7f1ed409ed00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1ed00098f0 tx=0x7f1ed0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21c1840fc884c0e7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:32.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.814+0000 7f1ed8800700 1 -- 172.21.15.16:0/1135055767 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed0011830 con 0x7f1ed409e930 2023-09-30T22:30:32.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.814+0000 7f1ed8800700 1 -- 172.21.15.16:0/1135055767 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ed0011990 con 0x7f1ed409e930 2023-09-30T22:30:32.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.814+0000 7f1ed8800700 1 -- 172.21.15.16:0/1135055767 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed0011d20 con 0x7f1ed409e930 2023-09-30T22:30:32.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.815+0000 7f1edba66700 1 -- 172.21.15.16:0/1135055767 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ed409e930 msgr2=0x7f1ed409ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:32.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.815+0000 7f1edba66700 1 --2- 172.21.15.16:0/1135055767 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ed409e930 0x7f1ed409ed00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1ed00098f0 tx=0x7f1ed0009c00 comp rx=0 tx=0).stop 2023-09-30T22:30:32.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.816+0000 7f1edba66700 1 -- 172.21.15.16:0/1135055767 shutdown_connections 2023-09-30T22:30:32.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.816+0000 7f1edba66700 1 --2- 172.21.15.16:0/1135055767 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ed409f650 0x7f1ed4131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:32.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.816+0000 7f1edba66700 1 --2- 172.21.15.16:0/1135055767 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ed409e930 0x7f1ed409ed00 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:32.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.816+0000 7f1edba66700 1 --2- 172.21.15.16:0/1135055767 >> v2:172.21.15.16:3300/0 conn(0x7f1ed409f240 0x7f1ed41317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:32.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.816+0000 7f1edba66700 1 -- 172.21.15.16:0/1135055767 >> 172.21.15.16:0/1135055767 conn(0x7f1ed40a4210 msgr2=0x7f1ed40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:32.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.816+0000 7f1edba66700 1 -- 172.21.15.16:0/1135055767 shutdown_connections 2023-09-30T22:30:32.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.816+0000 7f1edba66700 1 -- 172.21.15.16:0/1135055767 wait complete. 2023-09-30T22:30:32.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.817+0000 7f1edba66700 1 Processor -- start 2023-09-30T22:30:32.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.817+0000 7f1edba66700 1 -- start start 2023-09-30T22:30:32.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.818+0000 7f1edba66700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ed409e930 0x7f1ed411e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:32.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.818+0000 7f1edba66700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ed409f240 0x7f1ed411e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:32.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.818+0000 7f1edba66700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ed409f650 0x7f1ed411ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:32.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.818+0000 7f1edba66700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed411ffc0 con 0x7f1ed409f240 2023-09-30T22:30:32.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.818+0000 7f1edba66700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed4121700 con 0x7f1ed409e930 2023-09-30T22:30:32.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.818+0000 7f1edba66700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed4071d90 con 0x7f1ed409f650 2023-09-30T22:30:32.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.818+0000 7f1ed9802700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ed409e930 0x7f1ed411e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:32.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.818+0000 7f1eda003700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ed409f650 0x7f1ed411ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:32.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.818+0000 7f1ed9802700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ed409e930 0x7f1ed411e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38352/0 (socket says 172.21.15.16:38352) 2023-09-30T22:30:32.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.818+0000 7f1ed9802700 1 -- 172.21.15.16:0/4294395285 learned_addr learned my addr 172.21.15.16:0/4294395285 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:32.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.818+0000 7f1eda003700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ed409f650 0x7f1ed411ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:54740/0 (socket says 172.21.15.16:54740) 2023-09-30T22:30:32.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.819+0000 7f1ed9001700 1 --2- 172.21.15.16:0/4294395285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ed409f240 0x7f1ed411e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:32.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.819+0000 7f1eda003700 1 -- 172.21.15.16:0/4294395285 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ed409e930 msgr2=0x7f1ed411e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:32.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.819+0000 7f1eda003700 1 --2- 172.21.15.16:0/4294395285 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ed409e930 0x7f1ed411e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:32.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.819+0000 7f1eda003700 1 -- 172.21.15.16:0/4294395285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ed409f240 msgr2=0x7f1ed411e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:32.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.819+0000 7f1eda003700 1 --2- 172.21.15.16:0/4294395285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ed409f240 0x7f1ed411e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:32.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.819+0000 7f1eda003700 1 -- 172.21.15.16:0/4294395285 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ed0009550 con 0x7f1ed409f650 2023-09-30T22:30:32.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.819+0000 7f1ed9001700 1 --2- 172.21.15.16:0/4294395285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ed409f240 0x7f1ed411e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:30:32.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.819+0000 7f1eda003700 1 --2- 172.21.15.16:0/4294395285 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ed409f650 0x7f1ed411ee30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc009f40 tx=0x7f1ecc008540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:32.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.819+0000 7f1eca7fc700 1 -- 172.21.15.16:0/4294395285 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc003c20 con 0x7f1ed409f650 2023-09-30T22:30:32.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.819+0000 7f1eca7fc700 1 -- 172.21.15.16:0/4294395285 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ecc003d80 con 0x7f1ed409f650 2023-09-30T22:30:32.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.819+0000 7f1eca7fc700 1 -- 172.21.15.16:0/4294395285 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc00d310 con 0x7f1ed409f650 2023-09-30T22:30:32.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.819+0000 7f1edba66700 1 -- 172.21.15.16:0/4294395285 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ed40a0190 con 0x7f1ed409f650 2023-09-30T22:30:32.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.819+0000 7f1edba66700 1 -- 172.21.15.16:0/4294395285 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ed4120570 con 0x7f1ed409f650 2023-09-30T22:30:32.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.820+0000 7f1eca7fc700 1 -- 172.21.15.16:0/4294395285 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1ecc00d470 con 0x7f1ed409f650 2023-09-30T22:30:32.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.820+0000 7f1eca7fc700 1 -- 172.21.15.16:0/4294395285 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f1ecc03fe70 con 0x7f1ed409f650 2023-09-30T22:30:32.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.821+0000 7f1edba66700 1 -- 172.21.15.16:0/4294395285 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ed40a2f40 con 0x7f1ed409f650 2023-09-30T22:30:32.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.826+0000 7f1eca7fc700 1 -- 172.21.15.16:0/4294395285 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1ecc01ec20 con 0x7f1ed409f650 2023-09-30T22:30:32.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.844+0000 7f81aeb7e700 1 Processor -- start 2023-09-30T22:30:32.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.844+0000 7f81aeb7e700 1 -- start start 2023-09-30T22:30:32.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.844+0000 7f81aeb7e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f81a809d6f0 0x7f81a809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:32.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.844+0000 7f81aeb7e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f81a809e000 0x7f81a81339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:32.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.844+0000 7f81aeb7e700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81a809b120 con 0x7f81a809e410 2023-09-30T22:30:32.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.844+0000 7f81aeb7e700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81a806b460 con 0x7f81a809e000 2023-09-30T22:30:32.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.844+0000 7f81aeb7e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f81a809a7e0 con 0x7f81a809d6f0 2023-09-30T22:30:32.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.845+0000 7f81ac91a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f81a809d6f0 0x7f81a809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:32.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.845+0000 7f81ac91a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f81a809d6f0 0x7f81a809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:54742/0 (socket says 172.21.15.16:54742) 2023-09-30T22:30:32.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.845+0000 7f81ac91a700 1 -- 172.21.15.16:0/1617485164 learned_addr learned my addr 172.21.15.16:0/1617485164 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:32.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.845+0000 7f81a7fff700 1 --2- 172.21.15.16:0/1617485164 >> v2:172.21.15.16:3300/0 conn(0x7f81a809e000 0x7f81a81339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:32.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.845+0000 7f81ac91a700 1 -- 172.21.15.16:0/1617485164 >> v2:172.21.15.16:3300/0 conn(0x7f81a809e000 msgr2=0x7f81a81339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:32.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.845+0000 7f81ac91a700 1 --2- 172.21.15.16:0/1617485164 >> v2:172.21.15.16:3300/0 conn(0x7f81a809e000 0x7f81a81339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:32.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.845+0000 7f81ac91a700 1 -- 172.21.15.16:0/1617485164 >> v1:172.21.15.16:6789/0 conn(0x7f81a809e410 legacy=0x7f81a8133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:32.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.845+0000 7f81ac91a700 1 -- 172.21.15.16:0/1617485164 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a809a500 con 0x7f81a809d6f0 2023-09-30T22:30:32.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.845+0000 7f81ac91a700 1 --2- 172.21.15.16:0/1617485164 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f81a809d6f0 0x7f81a809dac0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f819c009920 tx=0x7f819c009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72e10737dbbf7c45 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:32.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.845+0000 7f81a77fe700 1 -- 172.21.15.16:0/1617485164 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819c011830 con 0x7f81a809d6f0 2023-09-30T22:30:32.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.846+0000 7f81a77fe700 1 -- 172.21.15.16:0/1617485164 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f819c011990 con 0x7f81a809d6f0 2023-09-30T22:30:32.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.846+0000 7f81a77fe700 1 -- 172.21.15.16:0/1617485164 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819c011d20 con 0x7f81a809d6f0 2023-09-30T22:30:32.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.846+0000 7f81aeb7e700 1 -- 172.21.15.16:0/1617485164 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f81a809d6f0 msgr2=0x7f81a809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:32.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.846+0000 7f81aeb7e700 1 --2- 172.21.15.16:0/1617485164 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f81a809d6f0 0x7f81a809dac0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f819c009920 tx=0x7f819c009c30 comp rx=0 tx=0).stop 2023-09-30T22:30:32.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.846+0000 7f81aeb7e700 1 -- 172.21.15.16:0/1617485164 shutdown_connections 2023-09-30T22:30:32.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.846+0000 7f81aeb7e700 1 --2- 172.21.15.16:0/1617485164 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f81a809d6f0 0x7f81a809dac0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:32.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.846+0000 7f81aeb7e700 1 --2- 172.21.15.16:0/1617485164 >> v2:172.21.15.16:3300/0 conn(0x7f81a809e000 0x7f81a81339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:32.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.846+0000 7f81aeb7e700 1 -- 172.21.15.16:0/1617485164 >> 172.21.15.16:0/1617485164 conn(0x7f81a8066f50 msgr2=0x7f81a809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:32.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.846+0000 7f81aeb7e700 1 -- 172.21.15.16:0/1617485164 shutdown_connections 2023-09-30T22:30:32.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.846+0000 7f81aeb7e700 1 -- 172.21.15.16:0/1617485164 wait complete. 2023-09-30T22:30:32.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.847+0000 7f81aeb7e700 1 Processor -- start 2023-09-30T22:30:32.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.847+0000 7f81aeb7e700 1 -- start start 2023-09-30T22:30:32.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.848+0000 7f81aeb7e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81a809d6f0 0x7f81a81a5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:32.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.848+0000 7f81aeb7e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f81a809e000 0x7f81a81a6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:32.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.848+0000 7f81aeb7e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f81a809e410 0x7f81a81a6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:32.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.848+0000 7f81aeb7e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a81a79a0 con 0x7f81a809d6f0 2023-09-30T22:30:32.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.848+0000 7f81aeb7e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81a81a7320 con 0x7f81a809e000 2023-09-30T22:30:32.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.848+0000 7f81aeb7e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f81a81a9840 con 0x7f81a809e410 2023-09-30T22:30:32.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.848+0000 7f81ac91a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81a809d6f0 0x7f81a81a5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:32.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.848+0000 7f81ac91a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81a809d6f0 0x7f81a81a5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55412/0 (socket says 172.21.15.16:55412) 2023-09-30T22:30:32.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.848+0000 7f81ac91a700 1 -- 172.21.15.16:0/4201066921 learned_addr learned my addr 172.21.15.16:0/4201066921 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:32.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.848+0000 7f81ad11b700 1 --2- 172.21.15.16:0/4201066921 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f81a809e410 0x7f81a81a6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:32.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.849+0000 7f81ac91a700 1 -- 172.21.15.16:0/4201066921 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f81a809e410 msgr2=0x7f81a81a6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:32.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.849+0000 7f81ac91a700 1 --2- 172.21.15.16:0/4201066921 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f81a809e410 0x7f81a81a6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:32.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.849+0000 7f81ac91a700 1 -- 172.21.15.16:0/4201066921 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f81a809e000 msgr2=0x7f81a81a6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:32.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.849+0000 7f81ac91a700 1 --2- 172.21.15.16:0/4201066921 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f81a809e000 0x7f81a81a6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:32.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.849+0000 7f81ac91a700 1 -- 172.21.15.16:0/4201066921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f819c009550 con 0x7f81a809d6f0 2023-09-30T22:30:32.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.849+0000 7f81ac91a700 1 --2- 172.21.15.16:0/4201066921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81a809d6f0 0x7f81a81a5ec0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f819c009870 tx=0x7f819c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:32.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.849+0000 7f81a57fa700 1 -- 172.21.15.16:0/4201066921 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819c0045d0 con 0x7f81a809d6f0 2023-09-30T22:30:32.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.849+0000 7f81a57fa700 1 -- 172.21.15.16:0/4201066921 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f819c01f070 con 0x7f81a809d6f0 2023-09-30T22:30:32.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.849+0000 7f81a57fa700 1 -- 172.21.15.16:0/4201066921 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f819c0038e0 con 0x7f81a809d6f0 2023-09-30T22:30:32.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.849+0000 7f81aeb7e700 1 -- 172.21.15.16:0/4201066921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81a81a8bf0 con 0x7f81a809d6f0 2023-09-30T22:30:32.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.849+0000 7f81aeb7e700 1 -- 172.21.15.16:0/4201066921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81a809c710 con 0x7f81a809d6f0 2023-09-30T22:30:32.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.850+0000 7f81a57fa700 1 -- 172.21.15.16:0/4201066921 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f819c003a40 con 0x7f81a809d6f0 2023-09-30T22:30:32.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.850+0000 7f81a57fa700 1 -- 172.21.15.16:0/4201066921 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f819c021070 con 0x7f81a809d6f0 2023-09-30T22:30:32.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.851+0000 7f81aeb7e700 1 -- 172.21.15.16:0/4201066921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81a81a9200 con 0x7f81a809d6f0 2023-09-30T22:30:32.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.856+0000 7f81a57fa700 1 -- 172.21.15.16:0/4201066921 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f81a81a9200 con 0x7f81a809d6f0 2023-09-30T22:30:32.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.911+0000 7f5bb2d7b700 1 Processor -- start 2023-09-30T22:30:32.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.911+0000 7f5bb2d7b700 1 -- start start 2023-09-30T22:30:32.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.911+0000 7f5bb2d7b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5bac06a770 0x7f5bac09d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:32.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.911+0000 7f5bb2d7b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5bac09dc30 0x7f5bac09e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:32.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.911+0000 7f5bb2d7b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bac09a810 con 0x7f5bac13c1e0 2023-09-30T22:30:32.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.912+0000 7f5bb2d7b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bac06ab40 con 0x7f5bac09dc30 2023-09-30T22:30:32.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.912+0000 7f5bb2d7b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bac09ebc0 con 0x7f5bac06a770 2023-09-30T22:30:32.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.912+0000 7f5babfff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5bac09dc30 0x7f5bac09e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:32.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.912+0000 7f5bb1318700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5bac13c1e0 0x7f5bac13e5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34530/0 (socket says 172.21.15.16:34530) 2023-09-30T22:30:32.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.912+0000 7f5bb1318700 1 -- 172.21.15.16:0/1466499565 learned_addr learned my addr 172.21.15.16:0/1466499565 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:30:32.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.912+0000 7f5bb0b17700 1 --2- 172.21.15.16:0/1466499565 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5bac06a770 0x7f5bac09d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:32.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.912+0000 7f5babfff700 1 -- 172.21.15.16:0/1466499565 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5bac06a770 msgr2=0x7f5bac09d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:32.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.912+0000 7f5babfff700 1 --2- 172.21.15.16:0/1466499565 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5bac06a770 0x7f5bac09d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:32.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.912+0000 7f5babfff700 1 -- 172.21.15.16:0/1466499565 >> v1:172.21.15.16:6789/0 conn(0x7f5bac13c1e0 legacy=0x7f5bac13e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:32.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.912+0000 7f5bab7fe700 1 -- 172.21.15.16:0/1466499565 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 925330674 0 0) 0x7f5bac09a810 con 0x7f5bac13c1e0 2023-09-30T22:30:32.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.912+0000 7f5babfff700 1 -- 172.21.15.16:0/1466499565 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bac09e920 con 0x7f5bac09dc30 2023-09-30T22:30:32.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.912+0000 7f5babfff700 1 --2- 172.21.15.16:0/1466499565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bac09dc30 0x7f5bac09e080 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5b98009870 tx=0x7f5b98009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edf917a3411d045a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:32.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.913+0000 7f5bab7fe700 1 -- 172.21.15.16:0/1466499565 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b98011830 con 0x7f5bac09dc30 2023-09-30T22:30:32.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.913+0000 7f5bab7fe700 1 -- 172.21.15.16:0/1466499565 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b98011990 con 0x7f5bac09dc30 2023-09-30T22:30:32.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.913+0000 7f5bab7fe700 1 -- 172.21.15.16:0/1466499565 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b98011d20 con 0x7f5bac09dc30 2023-09-30T22:30:32.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.913+0000 7f5bb2d7b700 1 -- 172.21.15.16:0/1466499565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bac09dc30 msgr2=0x7f5bac09e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:32.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.913+0000 7f5bb2d7b700 1 --2- 172.21.15.16:0/1466499565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bac09dc30 0x7f5bac09e080 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5b98009870 tx=0x7f5b98009b80 comp rx=0 tx=0).stop 2023-09-30T22:30:32.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.913+0000 7f5bb2d7b700 1 -- 172.21.15.16:0/1466499565 shutdown_connections 2023-09-30T22:30:32.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.913+0000 7f5bb2d7b700 1 --2- 172.21.15.16:0/1466499565 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5bac06a770 0x7f5bac09d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:32.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.913+0000 7f5bb2d7b700 1 --2- 172.21.15.16:0/1466499565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bac09dc30 0x7f5bac09e080 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:32.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.913+0000 7f5bb2d7b700 1 -- 172.21.15.16:0/1466499565 >> 172.21.15.16:0/1466499565 conn(0x7f5bac066f50 msgr2=0x7f5bac09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:32.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.914+0000 7f5bb2d7b700 1 -- 172.21.15.16:0/1466499565 shutdown_connections 2023-09-30T22:30:32.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.914+0000 7f5bb2d7b700 1 -- 172.21.15.16:0/1466499565 wait complete. 2023-09-30T22:30:32.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.915+0000 7f5bb2d7b700 1 Processor -- start 2023-09-30T22:30:32.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.915+0000 7f5bb2d7b700 1 -- start start 2023-09-30T22:30:32.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.915+0000 7f5bb2d7b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5bac06a770 0x7f5bac13e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:32.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.915+0000 7f5bb2d7b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5bac09dc30 0x7f5bac13a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:32.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.915+0000 7f5bb2d7b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bac13c1e0 0x7f5bac13a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:32.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.915+0000 7f5bb2d7b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bac13ff50 con 0x7f5bac13c1e0 2023-09-30T22:30:32.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.915+0000 7f5bb2d7b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bac134b20 con 0x7f5bac06a770 2023-09-30T22:30:32.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.915+0000 7f5bb2d7b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5bac11dd50 con 0x7f5bac09dc30 2023-09-30T22:30:32.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.915+0000 7f5babfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5bac09dc30 0x7f5bac13a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:32.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.915+0000 7f5bb0b17700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5bac06a770 0x7f5bac13e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:32.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.916+0000 7f5babfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5bac09dc30 0x7f5bac13a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:54758/0 (socket says 172.21.15.16:54758) 2023-09-30T22:30:32.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.916+0000 7f5babfff700 1 -- 172.21.15.16:0/4244761792 learned_addr learned my addr 172.21.15.16:0/4244761792 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:32.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.916+0000 7f5bb1318700 1 --2- 172.21.15.16:0/4244761792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bac13c1e0 0x7f5bac13a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:32.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.916+0000 7f5babfff700 1 -- 172.21.15.16:0/4244761792 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5bac06a770 msgr2=0x7f5bac13e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:32.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.916+0000 7f5babfff700 1 --2- 172.21.15.16:0/4244761792 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5bac06a770 0x7f5bac13e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:32.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.916+0000 7f5babfff700 1 -- 172.21.15.16:0/4244761792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bac13c1e0 msgr2=0x7f5bac13a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:32.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.916+0000 7f5babfff700 1 --2- 172.21.15.16:0/4244761792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bac13c1e0 0x7f5bac13a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:32.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.916+0000 7f5babfff700 1 -- 172.21.15.16:0/4244761792 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b980094a0 con 0x7f5bac09dc30 2023-09-30T22:30:32.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.916+0000 7f5babfff700 1 --2- 172.21.15.16:0/4244761792 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5bac09dc30 0x7f5bac13a450 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5b98000c00 tx=0x7f5b98004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:32.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.916+0000 7f5ba97fa700 1 -- 172.21.15.16:0/4244761792 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b9801a8b0 con 0x7f5bac09dc30 2023-09-30T22:30:32.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.916+0000 7f5ba97fa700 1 -- 172.21.15.16:0/4244761792 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b9801aa10 con 0x7f5bac09dc30 2023-09-30T22:30:32.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.917+0000 7f5bb2d7b700 1 -- 172.21.15.16:0/4244761792 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bac0a13d0 con 0x7f5bac09dc30 2023-09-30T22:30:32.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.917+0000 7f5ba97fa700 1 -- 172.21.15.16:0/4244761792 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b9801ada0 con 0x7f5bac09dc30 2023-09-30T22:30:32.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.917+0000 7f5bb2d7b700 1 -- 172.21.15.16:0/4244761792 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bac09f1a0 con 0x7f5bac09dc30 2023-09-30T22:30:32.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.918+0000 7f5ba97fa700 1 -- 172.21.15.16:0/4244761792 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5b980243f0 con 0x7f5bac09dc30 2023-09-30T22:30:32.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.918+0000 7f5bb2d7b700 1 -- 172.21.15.16:0/4244761792 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bac11d9d0 con 0x7f5bac09dc30 2023-09-30T22:30:32.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.918+0000 7f5ba97fa700 1 -- 172.21.15.16:0/4244761792 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f5b98040cf0 con 0x7f5bac09dc30 2023-09-30T22:30:32.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:32.927+0000 7f5ba97fa700 1 -- 172.21.15.16:0/4244761792 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5b9800b360 con 0x7f5bac09dc30 2023-09-30T22:30:33.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:33.007+0000 7fd0500e7700 1 -- 172.21.15.16:0/2676695369 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd048074ea0 con 0x7fd04809dc30 2023-09-30T22:30:33.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:33.007+0000 7fd03e7fc700 1 -- 172.21.15.16:0/2676695369 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd038015020 con 0x7fd04809dc30 2023-09-30T22:30:33.008 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:30:33.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:33.010+0000 7fd0500e7700 1 -- 172.21.15.16:0/2676695369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd04809dc30 msgr2=0x7fd04813a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:33.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:33.010+0000 7fd0500e7700 1 --2- 172.21.15.16:0/2676695369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd04809dc30 0x7fd04813a820 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd04809eff0 tx=0x7fd038004750 comp rx=0 tx=0).stop 2023-09-30T22:30:33.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:33.010+0000 7fd0500e7700 1 -- 172.21.15.16:0/2676695369 shutdown_connections 2023-09-30T22:30:33.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:33.010+0000 7fd0500e7700 1 --2- 172.21.15.16:0/2676695369 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd04813c1e0 0x7fd04813ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:33.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:33.010+0000 7fd0500e7700 1 --2- 172.21.15.16:0/2676695369 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd04806a770 0x7fd04813e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:33.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:33.010+0000 7fd0500e7700 1 --2- 172.21.15.16:0/2676695369 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd04809dc30 0x7fd04813a820 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:33.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:33.010+0000 7fd0500e7700 1 -- 172.21.15.16:0/2676695369 >> 172.21.15.16:0/2676695369 conn(0x7fd048066f50 msgr2=0x7fd0480a1590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:33.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:33.010+0000 7fd0500e7700 1 -- 172.21.15.16:0/2676695369 shutdown_connections 2023-09-30T22:30:33.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:33.010+0000 7fd0500e7700 1 -- 172.21.15.16:0/2676695369 wait complete. 2023-09-30T22:30:33.025 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:30:33.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:33.069+0000 7f81aeb7e700 1 -- 172.21.15.16:0/4201066921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f81a81a9200 con 0x7f81a809d6f0 2023-09-30T22:30:33.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:33.069+0000 7f81a57fa700 1 -- 172.21.15.16:0/4201066921 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f81a81a9200 con 0x7f81a809d6f0 2023-09-30T22:30:33.069 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:30:33.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:33.071+0000 7f81aeb7e700 1 -- 172.21.15.16:0/4201066921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81a809d6f0 msgr2=0x7f81a81a5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:33.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:33.071+0000 7f81aeb7e700 1 --2- 172.21.15.16:0/4201066921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81a809d6f0 0x7f81a81a5ec0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f819c009870 tx=0x7f819c004750 comp rx=0 tx=0).stop 2023-09-30T22:30:33.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:33.071+0000 7f81aeb7e700 1 -- 172.21.15.16:0/4201066921 shutdown_connections 2023-09-30T22:30:33.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:33.071+0000 7f81aeb7e700 1 --2- 172.21.15.16:0/4201066921 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f81a809e410 0x7f81a81a6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:33.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:33.071+0000 7f81aeb7e700 1 --2- 172.21.15.16:0/4201066921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81a809d6f0 0x7f81a81a5ec0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:33.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:33.071+0000 7f81aeb7e700 1 --2- 172.21.15.16:0/4201066921 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f81a809e000 0x7f81a81a6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:33.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:33.071+0000 7f81aeb7e700 1 -- 172.21.15.16:0/4201066921 >> 172.21.15.16:0/4201066921 conn(0x7f81a8066f50 msgr2=0x7f81a81bad60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:33.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:33.071+0000 7f81aeb7e700 1 -- 172.21.15.16:0/4201066921 shutdown_connections 2023-09-30T22:30:33.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:33.071+0000 7f81aeb7e700 1 -- 172.21.15.16:0/4201066921 wait complete. 2023-09-30T22:30:33.085 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:30:33.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:33.091+0000 7f1edba66700 1 -- 172.21.15.16:0/4294395285 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1ed4074ea0 con 0x7f1ed409f650 2023-09-30T22:30:33.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:33.092+0000 7f1eca7fc700 1 -- 172.21.15.16:0/4294395285 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1ecc01e810 con 0x7f1ed409f650 2023-09-30T22:30:33.092 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:30:33.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:33.094+0000 7f1edba66700 1 -- 172.21.15.16:0/4294395285 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ed409f650 msgr2=0x7f1ed411ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:33.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:33.094+0000 7f1edba66700 1 --2- 172.21.15.16:0/4294395285 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ed409f650 0x7f1ed411ee30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc009f40 tx=0x7f1ecc008540 comp rx=0 tx=0).stop 2023-09-30T22:30:33.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:33.094+0000 7f1edba66700 1 -- 172.21.15.16:0/4294395285 shutdown_connections 2023-09-30T22:30:33.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:33.094+0000 7f1edba66700 1 --2- 172.21.15.16:0/4294395285 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ed409e930 0x7f1ed411e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:33.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:33.094+0000 7f1edba66700 1 --2- 172.21.15.16:0/4294395285 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ed409f650 0x7f1ed411ee30 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:33.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:33.094+0000 7f1edba66700 1 --2- 172.21.15.16:0/4294395285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ed409f240 0x7f1ed411e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:33.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:33.094+0000 7f1edba66700 1 -- 172.21.15.16:0/4294395285 >> 172.21.15.16:0/4294395285 conn(0x7f1ed40a4210 msgr2=0x7f1ed40a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:33.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:33.094+0000 7f1edba66700 1 -- 172.21.15.16:0/4294395285 shutdown_connections 2023-09-30T22:30:33.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:33.094+0000 7f1edba66700 1 -- 172.21.15.16:0/4294395285 wait complete. 2023-09-30T22:30:33.108 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:30:33.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:33.155+0000 7f5bb2d7b700 1 -- 172.21.15.16:0/4244761792 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5bac074ea0 con 0x7f5bac09dc30 2023-09-30T22:30:33.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:33.156+0000 7f5ba97fa700 1 -- 172.21.15.16:0/4244761792 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5b98015030 con 0x7f5bac09dc30 2023-09-30T22:30:33.156 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:30:33.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:33.158+0000 7f5bb2d7b700 1 -- 172.21.15.16:0/4244761792 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5bac09dc30 msgr2=0x7f5bac13a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:33.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:33.158+0000 7f5bb2d7b700 1 --2- 172.21.15.16:0/4244761792 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5bac09dc30 0x7f5bac13a450 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5b98000c00 tx=0x7f5b98004750 comp rx=0 tx=0).stop 2023-09-30T22:30:33.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:33.158+0000 7f5bb2d7b700 1 -- 172.21.15.16:0/4244761792 shutdown_connections 2023-09-30T22:30:33.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:33.158+0000 7f5bb2d7b700 1 --2- 172.21.15.16:0/4244761792 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5bac06a770 0x7f5bac13e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:33.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:33.158+0000 7f5bb2d7b700 1 --2- 172.21.15.16:0/4244761792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bac13c1e0 0x7f5bac13a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:33.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:33.158+0000 7f5bb2d7b700 1 --2- 172.21.15.16:0/4244761792 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5bac09dc30 0x7f5bac13a450 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:33.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:33.158+0000 7f5bb2d7b700 1 -- 172.21.15.16:0/4244761792 >> 172.21.15.16:0/4244761792 conn(0x7f5bac066f50 msgr2=0x7f5bac072200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:33.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:33.158+0000 7f5bb2d7b700 1 -- 172.21.15.16:0/4244761792 shutdown_connections 2023-09-30T22:30:33.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:33.158+0000 7f5bb2d7b700 1 -- 172.21.15.16:0/4244761792 wait complete. 2023-09-30T22:30:33.172 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:30:34.025 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:34.086 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:34.109 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:34.173 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:34.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.175+0000 7f921555d700 1 Processor -- start 2023-09-30T22:30:34.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.175+0000 7f921555d700 1 -- start start 2023-09-30T22:30:34.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.176+0000 7f921555d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f921009d6f0 0x7f921009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:34.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.176+0000 7f921555d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f921009e000 0x7f92101339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:34.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.176+0000 7f921555d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f921009e410 0x7f9210133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:34.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.176+0000 7f921555d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f921006b460 con 0x7f921009d6f0 2023-09-30T22:30:34.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.176+0000 7f921555d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f921009b120 con 0x7f921009e000 2023-09-30T22:30:34.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.176+0000 7f921555d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f921009a7e0 con 0x7f921009e410 2023-09-30T22:30:34.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.176+0000 7f920effd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f921009d6f0 0x7f921009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:34.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.176+0000 7f920effd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f921009d6f0 0x7f921009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55444/0 (socket says 172.21.15.16:55444) 2023-09-30T22:30:34.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.176+0000 7f920effd700 1 -- 172.21.15.16:0/591848169 learned_addr learned my addr 172.21.15.16:0/591848169 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:34.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.176+0000 7f920f7fe700 1 --2- 172.21.15.16:0/591848169 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f921009e410 0x7f9210133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:34.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.176+0000 7f920e7fc700 1 --2- 172.21.15.16:0/591848169 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f921009e000 0x7f92101339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:34.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.177+0000 7f920effd700 1 -- 172.21.15.16:0/591848169 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f921009e410 msgr2=0x7f9210133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:34.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.177+0000 7f920effd700 1 --2- 172.21.15.16:0/591848169 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f921009e410 0x7f9210133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.177+0000 7f920effd700 1 -- 172.21.15.16:0/591848169 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f921009e000 msgr2=0x7f92101339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:34.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.177+0000 7f920effd700 1 --2- 172.21.15.16:0/591848169 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f921009e000 0x7f92101339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.177+0000 7f920effd700 1 -- 172.21.15.16:0/591848169 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f921009a500 con 0x7f921009d6f0 2023-09-30T22:30:34.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.177+0000 7f920f7fe700 1 --2- 172.21.15.16:0/591848169 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f921009e410 0x7f9210133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:30:34.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.177+0000 7f920effd700 1 --2- 172.21.15.16:0/591848169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f921009d6f0 0x7f921009dac0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f92000098f0 tx=0x7f9200009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bf150e5b02889ce server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:34.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.177+0000 7f920dffb700 1 -- 172.21.15.16:0/591848169 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9200011830 con 0x7f921009d6f0 2023-09-30T22:30:34.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.177+0000 7f920dffb700 1 -- 172.21.15.16:0/591848169 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9200011990 con 0x7f921009d6f0 2023-09-30T22:30:34.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.177+0000 7f920dffb700 1 -- 172.21.15.16:0/591848169 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9200011d20 con 0x7f921009d6f0 2023-09-30T22:30:34.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.178+0000 7f921555d700 1 -- 172.21.15.16:0/591848169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f921009d6f0 msgr2=0x7f921009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:34.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.178+0000 7f921555d700 1 --2- 172.21.15.16:0/591848169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f921009d6f0 0x7f921009dac0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f92000098f0 tx=0x7f9200009c00 comp rx=0 tx=0).stop 2023-09-30T22:30:34.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.178+0000 7f921555d700 1 -- 172.21.15.16:0/591848169 shutdown_connections 2023-09-30T22:30:34.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.178+0000 7f921555d700 1 --2- 172.21.15.16:0/591848169 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f921009e410 0x7f9210133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.178+0000 7f921555d700 1 --2- 172.21.15.16:0/591848169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f921009d6f0 0x7f921009dac0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.178+0000 7f921555d700 1 --2- 172.21.15.16:0/591848169 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f921009e000 0x7f92101339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.178+0000 7f921555d700 1 -- 172.21.15.16:0/591848169 >> 172.21.15.16:0/591848169 conn(0x7f9210066f50 msgr2=0x7f921009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:34.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.178+0000 7f921555d700 1 -- 172.21.15.16:0/591848169 shutdown_connections 2023-09-30T22:30:34.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.178+0000 7f921555d700 1 -- 172.21.15.16:0/591848169 wait complete. 2023-09-30T22:30:34.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.180+0000 7f921555d700 1 Processor -- start 2023-09-30T22:30:34.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.180+0000 7f921555d700 1 -- start start 2023-09-30T22:30:34.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.181+0000 7f921555d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f921009d6f0 0x7f92101c8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:34.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.181+0000 7f921555d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f921009e000 0x7f92101c92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:34.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.181+0000 7f921555d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f921009e410 0x7f92101b4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:34.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.181+0000 7f921555d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92101b8300 con 0x7f921009e000 2023-09-30T22:30:34.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.181+0000 7f921555d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92100a20b0 con 0x7f921009e410 2023-09-30T22:30:34.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.181+0000 7f921555d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f92101b7c60 con 0x7f921009d6f0 2023-09-30T22:30:34.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.181+0000 7f920e7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f921009e000 0x7f92101c92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:34.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.181+0000 7f920e7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f921009e000 0x7f92101c92a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55460/0 (socket says 172.21.15.16:55460) 2023-09-30T22:30:34.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.181+0000 7f920e7fc700 1 -- 172.21.15.16:0/1968329469 learned_addr learned my addr 172.21.15.16:0/1968329469 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:34.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.181+0000 7f920f7fe700 1 --2- 172.21.15.16:0/1968329469 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f921009e410 0x7f92101b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:34.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.181+0000 7f920effd700 1 --2- 172.21.15.16:0/1968329469 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f921009d6f0 0x7f92101c8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:34.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.182+0000 7f920effd700 1 -- 172.21.15.16:0/1968329469 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f921009e410 msgr2=0x7f92101b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:34.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.182+0000 7f920effd700 1 --2- 172.21.15.16:0/1968329469 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f921009e410 0x7f92101b4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.182+0000 7f920effd700 1 -- 172.21.15.16:0/1968329469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f921009e000 msgr2=0x7f92101c92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:34.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.182+0000 7f920effd700 1 --2- 172.21.15.16:0/1968329469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f921009e000 0x7f92101c92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.182+0000 7f920effd700 1 -- 172.21.15.16:0/1968329469 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9200009550 con 0x7f921009d6f0 2023-09-30T22:30:34.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.182+0000 7f920e7fc700 1 --2- 172.21.15.16:0/1968329469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f921009e000 0x7f92101c92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:34.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.182+0000 7f920effd700 1 --2- 172.21.15.16:0/1968329469 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f921009d6f0 0x7f92101c8d60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f92000098f0 tx=0x7f9200004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:34.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.182+0000 7f91f77fe700 1 -- 172.21.15.16:0/1968329469 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9200004450 con 0x7f921009d6f0 2023-09-30T22:30:34.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.182+0000 7f91f77fe700 1 -- 172.21.15.16:0/1968329469 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92000045b0 con 0x7f921009d6f0 2023-09-30T22:30:34.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.182+0000 7f91f77fe700 1 -- 172.21.15.16:0/1968329469 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92000038e0 con 0x7f921009d6f0 2023-09-30T22:30:34.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.182+0000 7f921555d700 1 -- 172.21.15.16:0/1968329469 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92101b6870 con 0x7f921009d6f0 2023-09-30T22:30:34.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.182+0000 7f921555d700 1 -- 172.21.15.16:0/1968329469 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92101b62f0 con 0x7f921009d6f0 2023-09-30T22:30:34.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.184+0000 7f921555d700 1 -- 172.21.15.16:0/1968329469 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92101b8140 con 0x7f921009d6f0 2023-09-30T22:30:34.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.184+0000 7f91f77fe700 1 -- 172.21.15.16:0/1968329469 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9200003a40 con 0x7f921009d6f0 2023-09-30T22:30:34.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.185+0000 7f91f77fe700 1 -- 172.21.15.16:0/1968329469 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f9200040c60 con 0x7f921009d6f0 2023-09-30T22:30:34.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.189+0000 7f91f77fe700 1 -- 172.21.15.16:0/1968329469 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f920002f990 con 0x7f921009d6f0 2023-09-30T22:30:34.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.233+0000 7febfba59700 1 Processor -- start 2023-09-30T22:30:34.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.233+0000 7febfba59700 1 -- start start 2023-09-30T22:30:34.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.234+0000 7febfba59700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7febf40a0a30 0x7febf40a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:34.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.234+0000 7febfba59700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7febf4134170 0x7febf413bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:34.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.234+0000 7febfba59700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febf409edd0 con 0x7febf40a1340 2023-09-30T22:30:34.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.234+0000 7febfba59700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febf409e1f0 con 0x7febf4134170 2023-09-30T22:30:34.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.234+0000 7febfba59700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febf409dc50 con 0x7febf40a0a30 2023-09-30T22:30:34.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.234+0000 7febf8ff4700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7febf40a1340 0x7febf4133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34534/0 (socket says 172.21.15.16:34534) 2023-09-30T22:30:34.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.234+0000 7febf8ff4700 1 -- 172.21.15.16:0/3772521284 learned_addr learned my addr 172.21.15.16:0/3772521284 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:30:34.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.234+0000 7febf97f5700 1 --2- 172.21.15.16:0/3772521284 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7febf40a0a30 0x7febf40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:34.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.234+0000 7febf9ff6700 1 --2- 172.21.15.16:0/3772521284 >> v2:172.21.15.16:3300/0 conn(0x7febf4134170 0x7febf413bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:34.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.235+0000 7febebfff700 1 -- 172.21.15.16:0/3772521284 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3113053162 0 0) 0x7febf409edd0 con 0x7febf40a1340 2023-09-30T22:30:34.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.235+0000 7febebfff700 1 -- 172.21.15.16:0/3772521284 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7febe00035b0 con 0x7febf40a1340 2023-09-30T22:30:34.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.235+0000 7febf9ff6700 1 -- 172.21.15.16:0/3772521284 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7febf40a0a30 msgr2=0x7febf40a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:34.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.235+0000 7febf9ff6700 1 --2- 172.21.15.16:0/3772521284 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7febf40a0a30 0x7febf40a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.235+0000 7febf9ff6700 1 -- 172.21.15.16:0/3772521284 >> v1:172.21.15.16:6789/0 conn(0x7febf40a1340 legacy=0x7febf4133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:34.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.235+0000 7febf9ff6700 1 -- 172.21.15.16:0/3772521284 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febf40a1fe0 con 0x7febf4134170 2023-09-30T22:30:34.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.235+0000 7febf97f5700 1 --2- 172.21.15.16:0/3772521284 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7febf40a0a30 0x7febf40a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:34.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.235+0000 7febf9ff6700 1 --2- 172.21.15.16:0/3772521284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febf4134170 0x7febf413bd30 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7febec009d90 tx=0x7febec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aff3220ec6b84702 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:34.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.235+0000 7febebfff700 1 -- 172.21.15.16:0/3772521284 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febec00e070 con 0x7febf4134170 2023-09-30T22:30:34.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.235+0000 7febebfff700 1 -- 172.21.15.16:0/3772521284 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febec00bb20 con 0x7febf4134170 2023-09-30T22:30:34.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.235+0000 7febebfff700 1 -- 172.21.15.16:0/3772521284 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febec00be90 con 0x7febf4134170 2023-09-30T22:30:34.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.236+0000 7febfba59700 1 -- 172.21.15.16:0/3772521284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febf4134170 msgr2=0x7febf413bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:34.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.236+0000 7febfba59700 1 --2- 172.21.15.16:0/3772521284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febf4134170 0x7febf413bd30 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7febec009d90 tx=0x7febec00b330 comp rx=0 tx=0).stop 2023-09-30T22:30:34.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.236+0000 7febfba59700 1 -- 172.21.15.16:0/3772521284 shutdown_connections 2023-09-30T22:30:34.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.236+0000 7febfba59700 1 --2- 172.21.15.16:0/3772521284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febf4134170 0x7febf413bd30 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.236+0000 7febfba59700 1 --2- 172.21.15.16:0/3772521284 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7febf40a0a30 0x7febf40a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.236+0000 7febfba59700 1 -- 172.21.15.16:0/3772521284 >> 172.21.15.16:0/3772521284 conn(0x7febf40a6550 msgr2=0x7febf40a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:34.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.236+0000 7febfba59700 1 -- 172.21.15.16:0/3772521284 shutdown_connections 2023-09-30T22:30:34.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.236+0000 7febfba59700 1 -- 172.21.15.16:0/3772521284 wait complete. 2023-09-30T22:30:34.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.237+0000 7febfba59700 1 Processor -- start 2023-09-30T22:30:34.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.238+0000 7febfba59700 1 -- start start 2023-09-30T22:30:34.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.238+0000 7febfba59700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7febf40a0a30 0x7febf41228d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:34.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.238+0000 7febfba59700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febf40a1340 0x7febf4122e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:34.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.238+0000 7febfba59700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7febf4134170 0x7febf4123350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:34.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.238+0000 7febfba59700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf4124720 con 0x7febf40a1340 2023-09-30T22:30:34.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.238+0000 7febfba59700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febf40a4cc0 con 0x7febf4134170 2023-09-30T22:30:34.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.238+0000 7febfba59700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7febf40a0220 con 0x7febf40a0a30 2023-09-30T22:30:34.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.238+0000 7febf9ff6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7febf4134170 0x7febf4123350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:34.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.238+0000 7febf9ff6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7febf4134170 0x7febf4123350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38420/0 (socket says 172.21.15.16:38420) 2023-09-30T22:30:34.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.238+0000 7febf9ff6700 1 -- 172.21.15.16:0/2137182590 learned_addr learned my addr 172.21.15.16:0/2137182590 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:34.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.238+0000 7febf8ff4700 1 --2- 172.21.15.16:0/2137182590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febf40a1340 0x7febf4122e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:34.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.239+0000 7febf8ff4700 1 -- 172.21.15.16:0/2137182590 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7febf40a0a30 msgr2=0x7febf41228d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:34.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.239+0000 7febf8ff4700 1 --2- 172.21.15.16:0/2137182590 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7febf40a0a30 0x7febf41228d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.239+0000 7febf8ff4700 1 -- 172.21.15.16:0/2137182590 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7febf4134170 msgr2=0x7febf4123350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:34.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.239+0000 7febf8ff4700 1 --2- 172.21.15.16:0/2137182590 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7febf4134170 0x7febf4123350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.239+0000 7febf8ff4700 1 -- 172.21.15.16:0/2137182590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febf00094a0 con 0x7febf40a1340 2023-09-30T22:30:34.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.239+0000 7febf9ff6700 1 --2- 172.21.15.16:0/2137182590 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7febf4134170 0x7febf4123350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:34.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.239+0000 7febf8ff4700 1 --2- 172.21.15.16:0/2137182590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febf40a1340 0x7febf4122e10 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7febe400c490 tx=0x7febe400c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:34.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.239+0000 7febe9ffb700 1 -- 172.21.15.16:0/2137182590 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe4005e30 con 0x7febf40a1340 2023-09-30T22:30:34.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.239+0000 7febe9ffb700 1 -- 172.21.15.16:0/2137182590 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febe40070d0 con 0x7febf40a1340 2023-09-30T22:30:34.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.239+0000 7febfba59700 1 -- 172.21.15.16:0/2137182590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febec009a70 con 0x7febf40a1340 2023-09-30T22:30:34.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.239+0000 7febe9ffb700 1 -- 172.21.15.16:0/2137182590 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe4007440 con 0x7febf40a1340 2023-09-30T22:30:34.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.240+0000 7febfba59700 1 -- 172.21.15.16:0/2137182590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febf41333d0 con 0x7febf40a1340 2023-09-30T22:30:34.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.240+0000 7febe9ffb700 1 -- 172.21.15.16:0/2137182590 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7febe40075a0 con 0x7febf40a1340 2023-09-30T22:30:34.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.241+0000 7febe9ffb700 1 -- 172.21.15.16:0/2137182590 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7febe403eb10 con 0x7febf40a1340 2023-09-30T22:30:34.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.241+0000 7febfba59700 1 -- 172.21.15.16:0/2137182590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febf4074ea0 con 0x7febf40a1340 2023-09-30T22:30:34.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.250+0000 7febe9ffb700 1 -- 172.21.15.16:0/2137182590 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7febe4019b40 con 0x7febf40a1340 2023-09-30T22:30:34.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.279+0000 7fcaa0b2b700 1 Processor -- start 2023-09-30T22:30:34.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.279+0000 7fcaa0b2b700 1 -- start start 2023-09-30T22:30:34.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.280+0000 7fcaa0b2b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fca9c130540 0x7fca9c130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:34.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.280+0000 7fcaa0b2b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca9c130e50 0x7fca9c06f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:34.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.280+0000 7fcaa0b2b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fca9c06fb20 0x7fca9c06ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:34.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.280+0000 7fcaa0b2b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca9c09a390 con 0x7fca9c130540 2023-09-30T22:30:34.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.280+0000 7fcaa0b2b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c0716a0 con 0x7fca9c06fb20 2023-09-30T22:30:34.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.280+0000 7fcaa0b2b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c071290 con 0x7fca9c130e50 2023-09-30T22:30:34.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.280+0000 7fca9a59c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fca9c130540 0x7fca9c130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:34.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.280+0000 7fca9a59c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fca9c130540 0x7fca9c130910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55494/0 (socket says 172.21.15.16:55494) 2023-09-30T22:30:34.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.280+0000 7fca9a59c700 1 -- 172.21.15.16:0/1785829928 learned_addr learned my addr 172.21.15.16:0/1785829928 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:34.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.281+0000 7fca99d9b700 1 --2- 172.21.15.16:0/1785829928 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca9c130e50 0x7fca9c06f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:34.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.281+0000 7fca9ad9d700 1 --2- 172.21.15.16:0/1785829928 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fca9c06fb20 0x7fca9c06ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:34.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.281+0000 7fca99d9b700 1 -- 172.21.15.16:0/1785829928 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fca9c06fb20 msgr2=0x7fca9c06ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:34.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.281+0000 7fca99d9b700 1 --2- 172.21.15.16:0/1785829928 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fca9c06fb20 0x7fca9c06ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.281+0000 7fca99d9b700 1 -- 172.21.15.16:0/1785829928 >> v2:172.21.15.16:3300/0 conn(0x7fca9c130540 msgr2=0x7fca9c130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:34.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.281+0000 7fca99d9b700 1 --2- 172.21.15.16:0/1785829928 >> v2:172.21.15.16:3300/0 conn(0x7fca9c130540 0x7fca9c130910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.281+0000 7fca99d9b700 1 -- 172.21.15.16:0/1785829928 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca9c130340 con 0x7fca9c130e50 2023-09-30T22:30:34.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.281+0000 7fca9ad9d700 1 --2- 172.21.15.16:0/1785829928 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fca9c06fb20 0x7fca9c06ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:34.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.281+0000 7fca99d9b700 1 --2- 172.21.15.16:0/1785829928 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca9c130e50 0x7fca9c06f5e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fca90009840 tx=0x7fca90009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39bd2d02feb84a64 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:34.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.282+0000 7fca9959a700 1 -- 172.21.15.16:0/1785829928 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca90011830 con 0x7fca9c130e50 2023-09-30T22:30:34.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.282+0000 7fca9959a700 1 -- 172.21.15.16:0/1785829928 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca90011990 con 0x7fca9c130e50 2023-09-30T22:30:34.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.282+0000 7fca9959a700 1 -- 172.21.15.16:0/1785829928 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca90011d20 con 0x7fca9c130e50 2023-09-30T22:30:34.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.282+0000 7fcaa0b2b700 1 -- 172.21.15.16:0/1785829928 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca9c130e50 msgr2=0x7fca9c06f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:34.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.282+0000 7fcaa0b2b700 1 --2- 172.21.15.16:0/1785829928 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca9c130e50 0x7fca9c06f5e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fca90009840 tx=0x7fca90009b50 comp rx=0 tx=0).stop 2023-09-30T22:30:34.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.283+0000 7fcaa0b2b700 1 -- 172.21.15.16:0/1785829928 shutdown_connections 2023-09-30T22:30:34.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.283+0000 7fcaa0b2b700 1 --2- 172.21.15.16:0/1785829928 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fca9c06fb20 0x7fca9c06ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.283+0000 7fcaa0b2b700 1 --2- 172.21.15.16:0/1785829928 >> v2:172.21.15.16:3300/0 conn(0x7fca9c130540 0x7fca9c130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.283+0000 7fcaa0b2b700 1 --2- 172.21.15.16:0/1785829928 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca9c130e50 0x7fca9c06f5e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.283+0000 7fcaa0b2b700 1 -- 172.21.15.16:0/1785829928 >> 172.21.15.16:0/1785829928 conn(0x7fca9c0a2aa0 msgr2=0x7fca9c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:34.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.283+0000 7fcaa0b2b700 1 -- 172.21.15.16:0/1785829928 shutdown_connections 2023-09-30T22:30:34.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.283+0000 7fcaa0b2b700 1 -- 172.21.15.16:0/1785829928 wait complete. 2023-09-30T22:30:34.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.284+0000 7fcaa0b2b700 1 Processor -- start 2023-09-30T22:30:34.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.284+0000 7fcaa0b2b700 1 -- start start 2023-09-30T22:30:34.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.285+0000 7fcaa0b2b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca9c06fb20 0x7fca9c1330d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:34.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.285+0000 7fcaa0b2b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca9c130540 0x7fca9c133610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:34.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.285+0000 7fcaa0b2b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fca9c130e50 0x7fca9c133b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:34.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.285+0000 7fcaa0b2b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c119140 con 0x7fca9c06fb20 2023-09-30T22:30:34.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.285+0000 7fcaa0b2b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c12dd30 con 0x7fca9c130e50 2023-09-30T22:30:34.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.285+0000 7fcaa0b2b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c135f30 con 0x7fca9c130540 2023-09-30T22:30:34.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.285+0000 7fca9a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca9c06fb20 0x7fca9c1330d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:34.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.285+0000 7fca9a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca9c06fb20 0x7fca9c1330d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55504/0 (socket says 172.21.15.16:55504) 2023-09-30T22:30:34.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.285+0000 7fca9a59c700 1 -- 172.21.15.16:0/1353411530 learned_addr learned my addr 172.21.15.16:0/1353411530 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:34.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.285+0000 7fca99d9b700 1 --2- 172.21.15.16:0/1353411530 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca9c130540 0x7fca9c133610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:34.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.285+0000 7fca9ad9d700 1 --2- 172.21.15.16:0/1353411530 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fca9c130e50 0x7fca9c133b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:34.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.285+0000 7fca9a59c700 1 -- 172.21.15.16:0/1353411530 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca9c130540 msgr2=0x7fca9c133610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:34.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.285+0000 7fca9a59c700 1 --2- 172.21.15.16:0/1353411530 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca9c130540 0x7fca9c133610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.285+0000 7fca9a59c700 1 -- 172.21.15.16:0/1353411530 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fca9c130e50 msgr2=0x7fca9c133b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:34.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.285+0000 7fca9a59c700 1 --2- 172.21.15.16:0/1353411530 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fca9c130e50 0x7fca9c133b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.285+0000 7fca9a59c700 1 -- 172.21.15.16:0/1353411530 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca94009a70 con 0x7fca9c06fb20 2023-09-30T22:30:34.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.286+0000 7fca9a59c700 1 --2- 172.21.15.16:0/1353411530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca9c06fb20 0x7fca9c1330d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fca8c00e940 tx=0x7fca8c00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:34.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.286+0000 7fca8affd700 1 -- 172.21.15.16:0/1353411530 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8c0096a0 con 0x7fca9c06fb20 2023-09-30T22:30:34.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.286+0000 7fca8affd700 1 -- 172.21.15.16:0/1353411530 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca8c00c640 con 0x7fca9c06fb20 2023-09-30T22:30:34.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.286+0000 7fcaa0b2b700 1 -- 172.21.15.16:0/1353411530 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca900094a0 con 0x7fca9c06fb20 2023-09-30T22:30:34.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.286+0000 7fca8affd700 1 -- 172.21.15.16:0/1353411530 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8c00c9f0 con 0x7fca9c06fb20 2023-09-30T22:30:34.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.286+0000 7fcaa0b2b700 1 -- 172.21.15.16:0/1353411530 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca9c0718e0 con 0x7fca9c06fb20 2023-09-30T22:30:34.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.287+0000 7fca8affd700 1 -- 172.21.15.16:0/1353411530 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fca8c00cb50 con 0x7fca9c06fb20 2023-09-30T22:30:34.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.287+0000 7fcaa0b2b700 1 -- 172.21.15.16:0/1353411530 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca9c135510 con 0x7fca9c06fb20 2023-09-30T22:30:34.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.287+0000 7fca8affd700 1 -- 172.21.15.16:0/1353411530 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fca8c03faa0 con 0x7fca9c06fb20 2023-09-30T22:30:34.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.296+0000 7fca8affd700 1 -- 172.21.15.16:0/1353411530 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fca8c013470 con 0x7fca9c06fb20 2023-09-30T22:30:34.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.336+0000 7f7b42757700 1 Processor -- start 2023-09-30T22:30:34.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.336+0000 7f7b42757700 1 -- start start 2023-09-30T22:30:34.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.337+0000 7f7b42757700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7b3c09d6f0 0x7f7b3c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:34.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.337+0000 7f7b42757700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7b3c09e000 0x7f7b3c130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:34.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.337+0000 7f7b42757700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7b3c09e410 0x7f7b3c130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:34.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.337+0000 7f7b42757700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b3c09f520 con 0x7f7b3c09d6f0 2023-09-30T22:30:34.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.337+0000 7f7b42757700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3c09f130 con 0x7f7b3c09e000 2023-09-30T22:30:34.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.337+0000 7f7b42757700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3c09e930 con 0x7f7b3c09e410 2023-09-30T22:30:34.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.337+0000 7f7b40cf4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7b3c09e410 0x7f7b3c130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:34.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.337+0000 7f7b40cf4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7b3c09e410 0x7f7b3c130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:54820/0 (socket says 172.21.15.16:54820) 2023-09-30T22:30:34.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.337+0000 7f7b40cf4700 1 -- 172.21.15.16:0/1219655053 learned_addr learned my addr 172.21.15.16:0/1219655053 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:34.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.337+0000 7f7b3bfff700 1 --2- 172.21.15.16:0/1219655053 >> v2:172.21.15.16:3300/0 conn(0x7f7b3c09d6f0 0x7f7b3c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:34.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.338+0000 7f7b3b7fe700 1 --2- 172.21.15.16:0/1219655053 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7b3c09e000 0x7f7b3c130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:34.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.338+0000 7f7b3bfff700 1 -- 172.21.15.16:0/1219655053 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7b3c09e410 msgr2=0x7f7b3c130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:34.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.338+0000 7f7b3bfff700 1 --2- 172.21.15.16:0/1219655053 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7b3c09e410 0x7f7b3c130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.338+0000 7f7b3bfff700 1 -- 172.21.15.16:0/1219655053 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7b3c09e000 msgr2=0x7f7b3c130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:34.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.338+0000 7f7b3bfff700 1 --2- 172.21.15.16:0/1219655053 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7b3c09e000 0x7f7b3c130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.338+0000 7f7b3bfff700 1 -- 172.21.15.16:0/1219655053 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b3c09b2d0 con 0x7f7b3c09d6f0 2023-09-30T22:30:34.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.338+0000 7f7b3b7fe700 1 --2- 172.21.15.16:0/1219655053 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7b3c09e000 0x7f7b3c130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:34.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.338+0000 7f7b3bfff700 1 --2- 172.21.15.16:0/1219655053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b3c09d6f0 0x7f7b3c09dac0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c009870 tx=0x7f7b2c009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f721a700a83398ec server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:34.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.338+0000 7f7b3affd700 1 -- 172.21.15.16:0/1219655053 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c011830 con 0x7f7b3c09d6f0 2023-09-30T22:30:34.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.338+0000 7f7b3affd700 1 -- 172.21.15.16:0/1219655053 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b2c011990 con 0x7f7b3c09d6f0 2023-09-30T22:30:34.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.338+0000 7f7b3affd700 1 -- 172.21.15.16:0/1219655053 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c011d20 con 0x7f7b3c09d6f0 2023-09-30T22:30:34.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.339+0000 7f7b42757700 1 -- 172.21.15.16:0/1219655053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b3c09d6f0 msgr2=0x7f7b3c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:34.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.339+0000 7f7b42757700 1 --2- 172.21.15.16:0/1219655053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b3c09d6f0 0x7f7b3c09dac0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c009870 tx=0x7f7b2c009b80 comp rx=0 tx=0).stop 2023-09-30T22:30:34.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.339+0000 7f7b42757700 1 -- 172.21.15.16:0/1219655053 shutdown_connections 2023-09-30T22:30:34.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.339+0000 7f7b42757700 1 --2- 172.21.15.16:0/1219655053 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7b3c09e410 0x7f7b3c130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.339+0000 7f7b42757700 1 --2- 172.21.15.16:0/1219655053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b3c09d6f0 0x7f7b3c09dac0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.339+0000 7f7b42757700 1 --2- 172.21.15.16:0/1219655053 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7b3c09e000 0x7f7b3c130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.339+0000 7f7b42757700 1 -- 172.21.15.16:0/1219655053 >> 172.21.15.16:0/1219655053 conn(0x7f7b3c066f50 msgr2=0x7f7b3c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:34.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.339+0000 7f7b42757700 1 -- 172.21.15.16:0/1219655053 shutdown_connections 2023-09-30T22:30:34.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.339+0000 7f7b42757700 1 -- 172.21.15.16:0/1219655053 wait complete. 2023-09-30T22:30:34.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.340+0000 7f7b42757700 1 Processor -- start 2023-09-30T22:30:34.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.340+0000 7f7b42757700 1 -- start start 2023-09-30T22:30:34.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.341+0000 7f7b42757700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7b3c09d6f0 0x7f7b3c1c85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:34.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.341+0000 7f7b42757700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b3c09e000 0x7f7b3c1c8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:34.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.341+0000 7f7b42757700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7b3c09e410 0x7f7b3c131650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:34.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.341+0000 7f7b42757700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3c134170 con 0x7f7b3c09e000 2023-09-30T22:30:34.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.341+0000 7f7b42757700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3c0a37f0 con 0x7f7b3c09e410 2023-09-30T22:30:34.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.341+0000 7f7b42757700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3c09c700 con 0x7f7b3c09d6f0 2023-09-30T22:30:34.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.341+0000 7f7b3bfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7b3c09d6f0 0x7f7b3c1c85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:34.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.341+0000 7f7b3bfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7b3c09d6f0 0x7f7b3c1c85a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:54832/0 (socket says 172.21.15.16:54832) 2023-09-30T22:30:34.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.341+0000 7f7b3bfff700 1 -- 172.21.15.16:0/2146684951 learned_addr learned my addr 172.21.15.16:0/2146684951 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:34.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.341+0000 7f7b3b7fe700 1 --2- 172.21.15.16:0/2146684951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b3c09e000 0x7f7b3c1c8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:34.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.341+0000 7f7b40cf4700 1 --2- 172.21.15.16:0/2146684951 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7b3c09e410 0x7f7b3c131650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:34.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.341+0000 7f7b3b7fe700 1 -- 172.21.15.16:0/2146684951 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7b3c09d6f0 msgr2=0x7f7b3c1c85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:34.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.341+0000 7f7b3b7fe700 1 --2- 172.21.15.16:0/2146684951 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7b3c09d6f0 0x7f7b3c1c85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.341+0000 7f7b3b7fe700 1 -- 172.21.15.16:0/2146684951 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7b3c09e410 msgr2=0x7f7b3c131650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:34.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.341+0000 7f7b3b7fe700 1 --2- 172.21.15.16:0/2146684951 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7b3c09e410 0x7f7b3c131650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.342+0000 7f7b3b7fe700 1 -- 172.21.15.16:0/2146684951 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b300094a0 con 0x7f7b3c09e000 2023-09-30T22:30:34.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.342+0000 7f7b40cf4700 1 --2- 172.21.15.16:0/2146684951 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7b3c09e410 0x7f7b3c131650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:34.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.342+0000 7f7b3bfff700 1 --2- 172.21.15.16:0/2146684951 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7b3c09d6f0 0x7f7b3c1c85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:34.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.342+0000 7f7b3b7fe700 1 --2- 172.21.15.16:0/2146684951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b3c09e000 0x7f7b3c1c8ae0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7b3000e980 tx=0x7f7b3000ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:34.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.342+0000 7f7b38ff9700 1 -- 172.21.15.16:0/2146684951 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b30004030 con 0x7f7b3c09e000 2023-09-30T22:30:34.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.342+0000 7f7b38ff9700 1 -- 172.21.15.16:0/2146684951 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b300041b0 con 0x7f7b3c09e000 2023-09-30T22:30:34.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.342+0000 7f7b42757700 1 -- 172.21.15.16:0/2146684951 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b2c0094a0 con 0x7f7b3c09e000 2023-09-30T22:30:34.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.342+0000 7f7b38ff9700 1 -- 172.21.15.16:0/2146684951 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b30004560 con 0x7f7b3c09e000 2023-09-30T22:30:34.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.342+0000 7f7b42757700 1 -- 172.21.15.16:0/2146684951 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b3c1308e0 con 0x7f7b3c09e000 2023-09-30T22:30:34.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.343+0000 7f7b38ff9700 1 -- 172.21.15.16:0/2146684951 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7b300036a0 con 0x7f7b3c09e000 2023-09-30T22:30:34.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.343+0000 7f7b38ff9700 1 -- 172.21.15.16:0/2146684951 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f7b300421c0 con 0x7f7b3c09e000 2023-09-30T22:30:34.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.343+0000 7f7b42757700 1 -- 172.21.15.16:0/2146684951 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b3c074ea0 con 0x7f7b3c09e000 2023-09-30T22:30:34.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.348+0000 7f7b38ff9700 1 -- 172.21.15.16:0/2146684951 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7b30044060 con 0x7f7b3c09e000 2023-09-30T22:30:34.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.408+0000 7f921555d700 1 -- 172.21.15.16:0/1968329469 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f92101b8140 con 0x7f921009d6f0 2023-09-30T22:30:34.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.408+0000 7f91f77fe700 1 -- 172.21.15.16:0/1968329469 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f92101b8140 con 0x7f921009d6f0 2023-09-30T22:30:34.409 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:30:34.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.410+0000 7f921555d700 1 -- 172.21.15.16:0/1968329469 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f921009d6f0 msgr2=0x7f92101c8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:34.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.410+0000 7f921555d700 1 --2- 172.21.15.16:0/1968329469 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f921009d6f0 0x7f92101c8d60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f92000098f0 tx=0x7f9200004750 comp rx=0 tx=0).stop 2023-09-30T22:30:34.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.411+0000 7f921555d700 1 -- 172.21.15.16:0/1968329469 shutdown_connections 2023-09-30T22:30:34.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.411+0000 7f921555d700 1 --2- 172.21.15.16:0/1968329469 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f921009e410 0x7f92101b4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.411+0000 7f921555d700 1 --2- 172.21.15.16:0/1968329469 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f921009d6f0 0x7f92101c8d60 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.411+0000 7f921555d700 1 --2- 172.21.15.16:0/1968329469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f921009e000 0x7f92101c92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.411+0000 7f921555d700 1 -- 172.21.15.16:0/1968329469 >> 172.21.15.16:0/1968329469 conn(0x7f9210066f50 msgr2=0x7f9210071330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:34.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.411+0000 7f921555d700 1 -- 172.21.15.16:0/1968329469 shutdown_connections 2023-09-30T22:30:34.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.411+0000 7f921555d700 1 -- 172.21.15.16:0/1968329469 wait complete. 2023-09-30T22:30:34.425 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:30:34.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.463+0000 7febfba59700 1 -- 172.21.15.16:0/2137182590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7febf40a47d0 con 0x7febf40a1340 2023-09-30T22:30:34.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.463+0000 7febe9ffb700 1 -- 172.21.15.16:0/2137182590 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7febe4017020 con 0x7febf40a1340 2023-09-30T22:30:34.464 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:30:34.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.465+0000 7febfba59700 1 -- 172.21.15.16:0/2137182590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febf40a1340 msgr2=0x7febf4122e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:34.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.465+0000 7febfba59700 1 --2- 172.21.15.16:0/2137182590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febf40a1340 0x7febf4122e10 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7febe400c490 tx=0x7febe400c7a0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.466+0000 7febfba59700 1 -- 172.21.15.16:0/2137182590 shutdown_connections 2023-09-30T22:30:34.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.466+0000 7febfba59700 1 --2- 172.21.15.16:0/2137182590 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7febf4134170 0x7febf4123350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.466+0000 7febfba59700 1 --2- 172.21.15.16:0/2137182590 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7febf40a0a30 0x7febf41228d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.466+0000 7febfba59700 1 --2- 172.21.15.16:0/2137182590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febf40a1340 0x7febf4122e10 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.466+0000 7febfba59700 1 -- 172.21.15.16:0/2137182590 >> 172.21.15.16:0/2137182590 conn(0x7febf40a6550 msgr2=0x7febf409edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:34.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.466+0000 7febfba59700 1 -- 172.21.15.16:0/2137182590 shutdown_connections 2023-09-30T22:30:34.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.466+0000 7febfba59700 1 -- 172.21.15.16:0/2137182590 wait complete. 2023-09-30T22:30:34.480 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:30:34.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.537+0000 7fcaa0b2b700 1 -- 172.21.15.16:0/1353411530 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fca9c074ea0 con 0x7fca9c06fb20 2023-09-30T22:30:34.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.538+0000 7fca8affd700 1 -- 172.21.15.16:0/1353411530 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fca8c021090 con 0x7fca9c06fb20 2023-09-30T22:30:34.538 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:30:34.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.540+0000 7fcaa0b2b700 1 -- 172.21.15.16:0/1353411530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca9c06fb20 msgr2=0x7fca9c1330d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:34.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.540+0000 7fcaa0b2b700 1 --2- 172.21.15.16:0/1353411530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca9c06fb20 0x7fca9c1330d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fca8c00e940 tx=0x7fca8c00ec50 comp rx=0 tx=0).stop 2023-09-30T22:30:34.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.540+0000 7fcaa0b2b700 1 -- 172.21.15.16:0/1353411530 shutdown_connections 2023-09-30T22:30:34.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.540+0000 7fcaa0b2b700 1 --2- 172.21.15.16:0/1353411530 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fca9c130e50 0x7fca9c133b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.540+0000 7fcaa0b2b700 1 --2- 172.21.15.16:0/1353411530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca9c06fb20 0x7fca9c1330d0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.540+0000 7fcaa0b2b700 1 --2- 172.21.15.16:0/1353411530 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca9c130540 0x7fca9c133610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.540+0000 7fcaa0b2b700 1 -- 172.21.15.16:0/1353411530 >> 172.21.15.16:0/1353411530 conn(0x7fca9c0a2aa0 msgr2=0x7fca9c12fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:34.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.540+0000 7fcaa0b2b700 1 -- 172.21.15.16:0/1353411530 shutdown_connections 2023-09-30T22:30:34.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.540+0000 7fcaa0b2b700 1 -- 172.21.15.16:0/1353411530 wait complete. 2023-09-30T22:30:34.554 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:30:34.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.575+0000 7f7b42757700 1 -- 172.21.15.16:0/2146684951 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7b3c06c360 con 0x7f7b3c09e000 2023-09-30T22:30:34.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.575+0000 7f7b38ff9700 1 -- 172.21.15.16:0/2146684951 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7b30003950 con 0x7f7b3c09e000 2023-09-30T22:30:34.576 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:30:34.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.578+0000 7f7b42757700 1 -- 172.21.15.16:0/2146684951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b3c09e000 msgr2=0x7f7b3c1c8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:34.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.578+0000 7f7b42757700 1 --2- 172.21.15.16:0/2146684951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b3c09e000 0x7f7b3c1c8ae0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7b3000e980 tx=0x7f7b3000ec90 comp rx=0 tx=0).stop 2023-09-30T22:30:34.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.578+0000 7f7b42757700 1 -- 172.21.15.16:0/2146684951 shutdown_connections 2023-09-30T22:30:34.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.578+0000 7f7b42757700 1 --2- 172.21.15.16:0/2146684951 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7b3c09e410 0x7f7b3c131650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.578+0000 7f7b42757700 1 --2- 172.21.15.16:0/2146684951 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7b3c09d6f0 0x7f7b3c1c85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.578+0000 7f7b42757700 1 --2- 172.21.15.16:0/2146684951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b3c09e000 0x7f7b3c1c8ae0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:34.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.578+0000 7f7b42757700 1 -- 172.21.15.16:0/2146684951 >> 172.21.15.16:0/2146684951 conn(0x7f7b3c066f50 msgr2=0x7f7b3c070d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:34.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.578+0000 7f7b42757700 1 -- 172.21.15.16:0/2146684951 shutdown_connections 2023-09-30T22:30:34.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:34.578+0000 7f7b42757700 1 -- 172.21.15.16:0/2146684951 wait complete. 2023-09-30T22:30:34.592 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:30:35.426 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:35.481 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:35.554 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:35.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.574+0000 7fb7c4e22700 1 Processor -- start 2023-09-30T22:30:35.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.574+0000 7fb7c4e22700 1 -- start start 2023-09-30T22:30:35.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.575+0000 7fb7c4e22700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb7c0131810 0x7fb7c0131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:35.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.575+0000 7fb7c4e22700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb7c009a580 0x7fb7c0139660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:35.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.575+0000 7fb7c4e22700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7c01313c0 con 0x7fb7c0099a40 2023-09-30T22:30:35.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.575+0000 7fb7c4e22700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c006e8c0 con 0x7fb7c0131810 2023-09-30T22:30:35.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.575+0000 7fb7c4e22700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c0071d20 con 0x7fb7c009a580 2023-09-30T22:30:35.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.575+0000 7fb7bed9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb7c009a580 0x7fb7c0139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:35.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.575+0000 7fb7bed9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb7c009a580 0x7fb7c0139660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:54838/0 (socket says 172.21.15.16:54838) 2023-09-30T22:30:35.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.575+0000 7fb7bdd9b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb7c0099a40 0x7fb7c0099e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34544/0 (socket says 172.21.15.16:34544) 2023-09-30T22:30:35.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.575+0000 7fb7bed9d700 1 -- 172.21.15.16:0/3748004944 learned_addr learned my addr 172.21.15.16:0/3748004944 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:35.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.575+0000 7fb7be59c700 1 --2- 172.21.15.16:0/3748004944 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb7c0131810 0x7fb7c0131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:35.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.576+0000 7fb7bed9d700 1 -- 172.21.15.16:0/3748004944 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb7c0131810 msgr2=0x7fb7c0131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:35.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.576+0000 7fb7bed9d700 1 --2- 172.21.15.16:0/3748004944 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb7c0131810 0x7fb7c0131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.576+0000 7fb7bed9d700 1 -- 172.21.15.16:0/3748004944 >> v1:172.21.15.16:6789/0 conn(0x7fb7c0099a40 legacy=0x7fb7c0099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:35.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.576+0000 7fb7bed9d700 1 -- 172.21.15.16:0/3748004944 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7c00721a0 con 0x7fb7c009a580 2023-09-30T22:30:35.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.576+0000 7fb7bed9d700 1 --2- 172.21.15.16:0/3748004944 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb7c009a580 0x7fb7c0139660 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8009e80 tx=0x7fb7b800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4aa8e4ac81d7b28b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:35.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.576+0000 7fb7bd59a700 1 -- 172.21.15.16:0/3748004944 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b800e070 con 0x7fb7c009a580 2023-09-30T22:30:35.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.576+0000 7fb7bd59a700 1 -- 172.21.15.16:0/3748004944 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7b800b9c0 con 0x7fb7c009a580 2023-09-30T22:30:35.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.576+0000 7fb7bd59a700 1 -- 172.21.15.16:0/3748004944 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b800bd70 con 0x7fb7c009a580 2023-09-30T22:30:35.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.577+0000 7fb7c4e22700 1 -- 172.21.15.16:0/3748004944 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb7c009a580 msgr2=0x7fb7c0139660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:35.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.577+0000 7fb7c4e22700 1 --2- 172.21.15.16:0/3748004944 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb7c009a580 0x7fb7c0139660 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8009e80 tx=0x7fb7b800b330 comp rx=0 tx=0).stop 2023-09-30T22:30:35.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.577+0000 7fb7c4e22700 1 -- 172.21.15.16:0/3748004944 shutdown_connections 2023-09-30T22:30:35.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.577+0000 7fb7c4e22700 1 --2- 172.21.15.16:0/3748004944 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb7c009a580 0x7fb7c0139660 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.577+0000 7fb7c4e22700 1 --2- 172.21.15.16:0/3748004944 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb7c0131810 0x7fb7c0131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.577+0000 7fb7c4e22700 1 -- 172.21.15.16:0/3748004944 >> 172.21.15.16:0/3748004944 conn(0x7fb7c00a2aa0 msgr2=0x7fb7c00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:35.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.577+0000 7fb7c4e22700 1 -- 172.21.15.16:0/3748004944 shutdown_connections 2023-09-30T22:30:35.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.577+0000 7fb7c4e22700 1 -- 172.21.15.16:0/3748004944 wait complete. 2023-09-30T22:30:35.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.578+0000 7fb7c4e22700 1 Processor -- start 2023-09-30T22:30:35.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.578+0000 7fb7c4e22700 1 -- start start 2023-09-30T22:30:35.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.579+0000 7fb7c4e22700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb7c0099a40 0x7fb7c006f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:35.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.579+0000 7fb7c4e22700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7c009a580 0x7fb7c006fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:35.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.579+0000 7fb7c4e22700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb7c0131810 0x7fb7c0070090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:35.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.579+0000 7fb7c4e22700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c011ee10 con 0x7fb7c009a580 2023-09-30T22:30:35.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.579+0000 7fb7c4e22700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c011dd00 con 0x7fb7c0131810 2023-09-30T22:30:35.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.579+0000 7fb7c4e22700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c011f3b0 con 0x7fb7c0099a40 2023-09-30T22:30:35.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.579+0000 7fb7bdd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7c009a580 0x7fb7c006fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:35.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.579+0000 7fb7bdd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7c009a580 0x7fb7c006fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55530/0 (socket says 172.21.15.16:55530) 2023-09-30T22:30:35.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.579+0000 7fb7bdd9b700 1 -- 172.21.15.16:0/2742929719 learned_addr learned my addr 172.21.15.16:0/2742929719 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:35.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.579+0000 7fb7bed9d700 1 --2- 172.21.15.16:0/2742929719 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb7c0131810 0x7fb7c0070090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:35.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.579+0000 7fb7be59c700 1 --2- 172.21.15.16:0/2742929719 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb7c0099a40 0x7fb7c006f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:35.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.579+0000 7fb7bed9d700 1 -- 172.21.15.16:0/2742929719 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb7c0099a40 msgr2=0x7fb7c006f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:35.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.579+0000 7fb7bed9d700 1 --2- 172.21.15.16:0/2742929719 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb7c0099a40 0x7fb7c006f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.579+0000 7fb7bed9d700 1 -- 172.21.15.16:0/2742929719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7c009a580 msgr2=0x7fb7c006fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:35.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.579+0000 7fb7bed9d700 1 --2- 172.21.15.16:0/2742929719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7c009a580 0x7fb7c006fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.579+0000 7fb7bed9d700 1 -- 172.21.15.16:0/2742929719 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b8009b30 con 0x7fb7c0131810 2023-09-30T22:30:35.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.579+0000 7fb7bdd9b700 1 --2- 172.21.15.16:0/2742929719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7c009a580 0x7fb7c006fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:35.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.580+0000 7fb7bed9d700 1 --2- 172.21.15.16:0/2742929719 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb7c0131810 0x7fb7c0070090 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8015000 tx=0x7fb7b8004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:35.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.580+0000 7fb7aeffd700 1 -- 172.21.15.16:0/2742929719 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b80099b0 con 0x7fb7c0131810 2023-09-30T22:30:35.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.580+0000 7fb7aeffd700 1 -- 172.21.15.16:0/2742929719 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7b8004250 con 0x7fb7c0131810 2023-09-30T22:30:35.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.580+0000 7fb7c4e22700 1 -- 172.21.15.16:0/2742929719 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7c00a1d10 con 0x7fb7c0131810 2023-09-30T22:30:35.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.580+0000 7fb7aeffd700 1 -- 172.21.15.16:0/2742929719 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b8004620 con 0x7fb7c0131810 2023-09-30T22:30:35.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.580+0000 7fb7c4e22700 1 -- 172.21.15.16:0/2742929719 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7c011f0d0 con 0x7fb7c0131810 2023-09-30T22:30:35.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.581+0000 7fb7aeffd700 1 -- 172.21.15.16:0/2742929719 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb7b8004780 con 0x7fb7c0131810 2023-09-30T22:30:35.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.581+0000 7fb7aeffd700 1 -- 172.21.15.16:0/2742929719 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fb7b8040af0 con 0x7fb7c0131810 2023-09-30T22:30:35.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.581+0000 7fb7c4e22700 1 -- 172.21.15.16:0/2742929719 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7c0074ea0 con 0x7fb7c0131810 2023-09-30T22:30:35.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.587+0000 7fb7aeffd700 1 -- 172.21.15.16:0/2742929719 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb7b8019350 con 0x7fb7c0131810 2023-09-30T22:30:35.592 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:35.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.628+0000 7fc5e5678700 1 Processor -- start 2023-09-30T22:30:35.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.628+0000 7fc5e5678700 1 -- start start 2023-09-30T22:30:35.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.628+0000 7fc5e5678700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc5e0131810 0x7fc5e0131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:35.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.628+0000 7fc5e5678700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc5e0099a40 0x7fc5e0099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:35.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.628+0000 7fc5e5678700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5e01313c0 con 0x7fc5e009a3b0 2023-09-30T22:30:35.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.628+0000 7fc5e5678700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e009b580 con 0x7fc5e0099a40 2023-09-30T22:30:35.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.628+0000 7fc5e5678700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e006e8c0 con 0x7fc5e0131810 2023-09-30T22:30:35.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.629+0000 7fc5de7fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc5e0099a40 0x7fc5e0099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:35.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.629+0000 7fc5de7fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc5e0099a40 0x7fc5e0099e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38464/0 (socket says 172.21.15.16:38464) 2023-09-30T22:30:35.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.629+0000 7fc5de7fc700 1 -- 172.21.15.16:0/2956197572 learned_addr learned my addr 172.21.15.16:0/2956197572 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:35.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.629+0000 7fc5de7fc700 1 -- 172.21.15.16:0/2956197572 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc5e0131810 msgr2=0x7fc5e0131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:35.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.629+0000 7fc5de7fc700 1 --2- 172.21.15.16:0/2956197572 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc5e0131810 0x7fc5e0131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.629+0000 7fc5de7fc700 1 -- 172.21.15.16:0/2956197572 >> v1:172.21.15.16:6789/0 conn(0x7fc5e009a3b0 legacy=0x7fc5e0138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:35.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.629+0000 7fc5de7fc700 1 -- 172.21.15.16:0/2956197572 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5e009a820 con 0x7fc5e0099a40 2023-09-30T22:30:35.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.629+0000 7fc5de7fc700 1 --2- 172.21.15.16:0/2956197572 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc5e0099a40 0x7fc5e0099e70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc5d4009920 tx=0x7fc5d4009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a01eb1b5fdd1f065 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:35.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.630+0000 7fc5ddffb700 1 -- 172.21.15.16:0/2956197572 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d4004230 con 0x7fc5e0099a40 2023-09-30T22:30:35.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.630+0000 7fc5ddffb700 1 -- 172.21.15.16:0/2956197572 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5d4004390 con 0x7fc5e0099a40 2023-09-30T22:30:35.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.630+0000 7fc5ddffb700 1 -- 172.21.15.16:0/2956197572 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d4009e70 con 0x7fc5e0099a40 2023-09-30T22:30:35.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.631+0000 7fc5e5678700 1 -- 172.21.15.16:0/2956197572 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc5e0099a40 msgr2=0x7fc5e0099e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:35.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.631+0000 7fc5e5678700 1 --2- 172.21.15.16:0/2956197572 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc5e0099a40 0x7fc5e0099e70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc5d4009920 tx=0x7fc5d4009c30 comp rx=0 tx=0).stop 2023-09-30T22:30:35.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.631+0000 7fc5e5678700 1 -- 172.21.15.16:0/2956197572 shutdown_connections 2023-09-30T22:30:35.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.631+0000 7fc5e5678700 1 --2- 172.21.15.16:0/2956197572 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc5e0131810 0x7fc5e0131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.631+0000 7fc5e5678700 1 --2- 172.21.15.16:0/2956197572 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc5e0099a40 0x7fc5e0099e70 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.631+0000 7fc5e5678700 1 -- 172.21.15.16:0/2956197572 >> 172.21.15.16:0/2956197572 conn(0x7fc5e00a2aa0 msgr2=0x7fc5e00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:35.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.631+0000 7fc5e5678700 1 -- 172.21.15.16:0/2956197572 shutdown_connections 2023-09-30T22:30:35.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.631+0000 7fc5e5678700 1 -- 172.21.15.16:0/2956197572 wait complete. 2023-09-30T22:30:35.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.632+0000 7fc5e5678700 1 Processor -- start 2023-09-30T22:30:35.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.633+0000 7fc5e5678700 1 -- start start 2023-09-30T22:30:35.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.633+0000 7fc5e5678700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc5e0099a40 0x7fc5e006f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:35.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.633+0000 7fc5e5678700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc5e009a3b0 0x7fc5e006fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:35.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.633+0000 7fc5e5678700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5e0131810 0x7fc5e0070090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:35.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.633+0000 7fc5e5678700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e0070ae0 con 0x7fc5e0131810 2023-09-30T22:30:35.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.633+0000 7fc5e5678700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e011fe80 con 0x7fc5e0099a40 2023-09-30T22:30:35.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.633+0000 7fc5e5678700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e0120f50 con 0x7fc5e009a3b0 2023-09-30T22:30:35.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.633+0000 7fc5deffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc5e0099a40 0x7fc5e006f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:35.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.634+0000 7fc5df7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5e0131810 0x7fc5e0070090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:35.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.634+0000 7fc5deffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc5e0099a40 0x7fc5e006f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38480/0 (socket says 172.21.15.16:38480) 2023-09-30T22:30:35.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.634+0000 7fc5deffd700 1 -- 172.21.15.16:0/2903450074 learned_addr learned my addr 172.21.15.16:0/2903450074 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:35.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.634+0000 7fc5de7fc700 1 --2- 172.21.15.16:0/2903450074 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc5e009a3b0 0x7fc5e006fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:35.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.634+0000 7fc5df7fe700 1 -- 172.21.15.16:0/2903450074 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc5e009a3b0 msgr2=0x7fc5e006fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:35.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.634+0000 7fc5df7fe700 1 --2- 172.21.15.16:0/2903450074 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc5e009a3b0 0x7fc5e006fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.634+0000 7fc5df7fe700 1 -- 172.21.15.16:0/2903450074 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc5e0099a40 msgr2=0x7fc5e006f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:35.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.634+0000 7fc5df7fe700 1 --2- 172.21.15.16:0/2903450074 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc5e0099a40 0x7fc5e006f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.634+0000 7fc5df7fe700 1 -- 172.21.15.16:0/2903450074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d4009550 con 0x7fc5e0131810 2023-09-30T22:30:35.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.634+0000 7fc5deffd700 1 --2- 172.21.15.16:0/2903450074 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc5e0099a40 0x7fc5e006f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:35.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.634+0000 7fc5df7fe700 1 --2- 172.21.15.16:0/2903450074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5e0131810 0x7fc5e0070090 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc5d800c7b0 tx=0x7fc5d800cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:35.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.634+0000 7fc5c77fe700 1 -- 172.21.15.16:0/2903450074 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d8003e10 con 0x7fc5e0131810 2023-09-30T22:30:35.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.635+0000 7fc5c77fe700 1 -- 172.21.15.16:0/2903450074 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5d80050d0 con 0x7fc5e0131810 2023-09-30T22:30:35.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.635+0000 7fc5c77fe700 1 -- 172.21.15.16:0/2903450074 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d8005480 con 0x7fc5e0131810 2023-09-30T22:30:35.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.635+0000 7fc5e5678700 1 -- 172.21.15.16:0/2903450074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5e01214f0 con 0x7fc5e0131810 2023-09-30T22:30:35.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.635+0000 7fc5e5678700 1 -- 172.21.15.16:0/2903450074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5e006eb00 con 0x7fc5e0131810 2023-09-30T22:30:35.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.636+0000 7fc5e5678700 1 -- 172.21.15.16:0/2903450074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5e01208a0 con 0x7fc5e0131810 2023-09-30T22:30:35.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.636+0000 7fc5c77fe700 1 -- 172.21.15.16:0/2903450074 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc5d80055e0 con 0x7fc5e0131810 2023-09-30T22:30:35.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.636+0000 7fc5c77fe700 1 -- 172.21.15.16:0/2903450074 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fc5d803e9d0 con 0x7fc5e0131810 2023-09-30T22:30:35.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.645+0000 7fc5c77fe700 1 -- 172.21.15.16:0/2903450074 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc5d801ab10 con 0x7fc5e0131810 2023-09-30T22:30:35.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.703+0000 7fe44ab69700 1 Processor -- start 2023-09-30T22:30:35.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.704+0000 7fe44ab69700 1 -- start start 2023-09-30T22:30:35.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.704+0000 7fe44ab69700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe44409d6f0 0x7fe44409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:35.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.704+0000 7fe44ab69700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe44409e000 0x7fe444130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:35.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.704+0000 7fe44ab69700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe44409f130 con 0x7fe44409e410 2023-09-30T22:30:35.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.704+0000 7fe44ab69700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe44409f520 con 0x7fe44409d6f0 2023-09-30T22:30:35.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.704+0000 7fe44ab69700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe44409e930 con 0x7fe44409e000 2023-09-30T22:30:35.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.704+0000 7fe449106700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe44409e410 0x7fe444130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34560/0 (socket says 172.21.15.16:34560) 2023-09-30T22:30:35.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.704+0000 7fe448905700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe44409d6f0 0x7fe44409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:35.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.704+0000 7fe449106700 1 -- 172.21.15.16:0/1482469893 learned_addr learned my addr 172.21.15.16:0/1482469893 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:30:35.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.704+0000 7fe443fff700 1 --2- 172.21.15.16:0/1482469893 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe44409e000 0x7fe444130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:35.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.705+0000 7fe443fff700 1 -- 172.21.15.16:0/1482469893 >> v2:172.21.15.16:3300/0 conn(0x7fe44409d6f0 msgr2=0x7fe44409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:35.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.705+0000 7fe443fff700 1 --2- 172.21.15.16:0/1482469893 >> v2:172.21.15.16:3300/0 conn(0x7fe44409d6f0 0x7fe44409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.705+0000 7fe443fff700 1 -- 172.21.15.16:0/1482469893 >> v1:172.21.15.16:6789/0 conn(0x7fe44409e410 legacy=0x7fe444130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:35.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.705+0000 7fe443fff700 1 -- 172.21.15.16:0/1482469893 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe44409b2d0 con 0x7fe44409e000 2023-09-30T22:30:35.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.705+0000 7fe443fff700 1 --2- 172.21.15.16:0/1482469893 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe44409e000 0x7fe444130660 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe43000dc60 tx=0x7fe43000df70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73d0733fa53c8de5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:35.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.705+0000 7fe4437fe700 1 -- 172.21.15.16:0/1482469893 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe430012070 con 0x7fe44409e000 2023-09-30T22:30:35.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.705+0000 7fe4437fe700 1 -- 172.21.15.16:0/1482469893 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe43000d2e0 con 0x7fe44409e000 2023-09-30T22:30:35.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.705+0000 7fe4437fe700 1 -- 172.21.15.16:0/1482469893 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43000f900 con 0x7fe44409e000 2023-09-30T22:30:35.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.706+0000 7fe44ab69700 1 -- 172.21.15.16:0/1482469893 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe44409e000 msgr2=0x7fe444130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:35.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.706+0000 7fe44ab69700 1 --2- 172.21.15.16:0/1482469893 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe44409e000 0x7fe444130660 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe43000dc60 tx=0x7fe43000df70 comp rx=0 tx=0).stop 2023-09-30T22:30:35.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.706+0000 7fe44ab69700 1 -- 172.21.15.16:0/1482469893 shutdown_connections 2023-09-30T22:30:35.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.706+0000 7fe44ab69700 1 --2- 172.21.15.16:0/1482469893 >> v2:172.21.15.16:3300/0 conn(0x7fe44409d6f0 0x7fe44409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.706+0000 7fe44ab69700 1 --2- 172.21.15.16:0/1482469893 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe44409e000 0x7fe444130660 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.706+0000 7fe44ab69700 1 -- 172.21.15.16:0/1482469893 >> 172.21.15.16:0/1482469893 conn(0x7fe444066f50 msgr2=0x7fe44409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:35.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.706+0000 7fe44ab69700 1 -- 172.21.15.16:0/1482469893 shutdown_connections 2023-09-30T22:30:35.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.706+0000 7fe44ab69700 1 -- 172.21.15.16:0/1482469893 wait complete. 2023-09-30T22:30:35.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.708+0000 7fe44ab69700 1 Processor -- start 2023-09-30T22:30:35.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.708+0000 7fe44ab69700 1 -- start start 2023-09-30T22:30:35.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.709+0000 7fe44ab69700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe44409d6f0 0x7fe44411ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:35.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.709+0000 7fe44ab69700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe44409e000 0x7fe44411e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:35.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.709+0000 7fe44ab69700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe44409e410 0x7fe44411e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:35.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.709+0000 7fe44ab69700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4441302b0 con 0x7fe44409e000 2023-09-30T22:30:35.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.709+0000 7fe44ab69700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe444120d30 con 0x7fe44409e410 2023-09-30T22:30:35.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.709+0000 7fe44ab69700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe44411f330 con 0x7fe44409d6f0 2023-09-30T22:30:35.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.709+0000 7fe449106700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe44409e410 0x7fe44411e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:35.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.709+0000 7fe449106700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe44409e410 0x7fe44411e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38496/0 (socket says 172.21.15.16:38496) 2023-09-30T22:30:35.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.709+0000 7fe449106700 1 -- 172.21.15.16:0/1360411120 learned_addr learned my addr 172.21.15.16:0/1360411120 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:35.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.709+0000 7fe443fff700 1 --2- 172.21.15.16:0/1360411120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe44409e000 0x7fe44411e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:35.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.709+0000 7fe448905700 1 --2- 172.21.15.16:0/1360411120 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe44409d6f0 0x7fe44411ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:35.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.710+0000 7fe443fff700 1 -- 172.21.15.16:0/1360411120 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe44409d6f0 msgr2=0x7fe44411ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:35.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.710+0000 7fe443fff700 1 --2- 172.21.15.16:0/1360411120 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe44409d6f0 0x7fe44411ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.710+0000 7fe443fff700 1 -- 172.21.15.16:0/1360411120 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe44409e410 msgr2=0x7fe44411e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:35.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.710+0000 7fe443fff700 1 --2- 172.21.15.16:0/1360411120 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe44409e410 0x7fe44411e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.710+0000 7fe443fff700 1 -- 172.21.15.16:0/1360411120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe43000d940 con 0x7fe44409e000 2023-09-30T22:30:35.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.710+0000 7fe449106700 1 --2- 172.21.15.16:0/1360411120 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe44409e410 0x7fe44411e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:35.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.710+0000 7fe443fff700 1 --2- 172.21.15.16:0/1360411120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe44409e000 0x7fe44411e410 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe430019000 tx=0x7fe430004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:35.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.710+0000 7fe4417fa700 1 -- 172.21.15.16:0/1360411120 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4300036a0 con 0x7fe44409e000 2023-09-30T22:30:35.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.710+0000 7fe4417fa700 1 -- 172.21.15.16:0/1360411120 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe430003820 con 0x7fe44409e000 2023-09-30T22:30:35.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.710+0000 7fe4417fa700 1 -- 172.21.15.16:0/1360411120 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe430003bd0 con 0x7fe44409e000 2023-09-30T22:30:35.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.711+0000 7fe44ab69700 1 -- 172.21.15.16:0/1360411120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe444121710 con 0x7fe44409e000 2023-09-30T22:30:35.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.711+0000 7fe44ab69700 1 -- 172.21.15.16:0/1360411120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe444121180 con 0x7fe44409e000 2023-09-30T22:30:35.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.712+0000 7fe4417fa700 1 -- 172.21.15.16:0/1360411120 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe430003d30 con 0x7fe44409e000 2023-09-30T22:30:35.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.712+0000 7fe44ab69700 1 -- 172.21.15.16:0/1360411120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe44409c700 con 0x7fe44409e000 2023-09-30T22:30:35.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.712+0000 7fe4417fa700 1 -- 172.21.15.16:0/1360411120 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fe430044d30 con 0x7fe44409e000 2023-09-30T22:30:35.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.721+0000 7fe4417fa700 1 -- 172.21.15.16:0/1360411120 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe430015390 con 0x7fe44409e000 2023-09-30T22:30:35.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.757+0000 7f800c1c7700 1 Processor -- start 2023-09-30T22:30:35.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.757+0000 7f800c1c7700 1 -- start start 2023-09-30T22:30:35.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.758+0000 7f800c1c7700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f800409e1d0 0x7f8004138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:35.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.758+0000 7f800c1c7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8004139170 0x7f800413b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:35.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.758+0000 7f800c1c7700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f800409b120 con 0x7f800409d6f0 2023-09-30T22:30:35.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.758+0000 7f800c1c7700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f800406b460 con 0x7f800409e1d0 2023-09-30T22:30:35.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.758+0000 7f800c1c7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f800409a7e0 con 0x7f8004139170 2023-09-30T22:30:35.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.758+0000 7f8009762700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f800409e1d0 0x7f8004138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:35.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.758+0000 7f8009f63700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f800409d6f0 0x7f800409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34572/0 (socket says 172.21.15.16:34572) 2023-09-30T22:30:35.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.758+0000 7f8009f63700 1 -- 172.21.15.16:0/4153194333 learned_addr learned my addr 172.21.15.16:0/4153194333 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:30:35.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.758+0000 7f800a764700 1 --2- 172.21.15.16:0/4153194333 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8004139170 0x7f800413b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:35.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.759+0000 7f8009762700 1 -- 172.21.15.16:0/4153194333 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8004139170 msgr2=0x7f800413b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:35.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.759+0000 7f8009762700 1 --2- 172.21.15.16:0/4153194333 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8004139170 0x7f800413b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.759+0000 7f8009762700 1 -- 172.21.15.16:0/4153194333 >> v1:172.21.15.16:6789/0 conn(0x7f800409d6f0 legacy=0x7f800409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:35.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.759+0000 7f8008f61700 1 -- 172.21.15.16:0/4153194333 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2386449029 0 0) 0x7f800409b120 con 0x7f800409d6f0 2023-09-30T22:30:35.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.759+0000 7f8009762700 1 -- 172.21.15.16:0/4153194333 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f800409a500 con 0x7f800409e1d0 2023-09-30T22:30:35.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.759+0000 7f800a764700 1 --2- 172.21.15.16:0/4153194333 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8004139170 0x7f800413b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:35.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.759+0000 7f8009762700 1 --2- 172.21.15.16:0/4153194333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f800409e1d0 0x7f8004138c30 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4009870 tx=0x7f7ff4009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea12d2d84b593cac server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:35.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.759+0000 7f8008f61700 1 -- 172.21.15.16:0/4153194333 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4011830 con 0x7f800409e1d0 2023-09-30T22:30:35.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.759+0000 7f8008f61700 1 -- 172.21.15.16:0/4153194333 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ff4011990 con 0x7f800409e1d0 2023-09-30T22:30:35.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.759+0000 7f8008f61700 1 -- 172.21.15.16:0/4153194333 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4011d20 con 0x7f800409e1d0 2023-09-30T22:30:35.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.760+0000 7f800c1c7700 1 -- 172.21.15.16:0/4153194333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f800409e1d0 msgr2=0x7f8004138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:35.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.760+0000 7f800c1c7700 1 --2- 172.21.15.16:0/4153194333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f800409e1d0 0x7f8004138c30 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4009870 tx=0x7f7ff4009b80 comp rx=0 tx=0).stop 2023-09-30T22:30:35.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.760+0000 7f800c1c7700 1 -- 172.21.15.16:0/4153194333 shutdown_connections 2023-09-30T22:30:35.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.760+0000 7f800c1c7700 1 --2- 172.21.15.16:0/4153194333 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8004139170 0x7f800413b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.760+0000 7f800c1c7700 1 --2- 172.21.15.16:0/4153194333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f800409e1d0 0x7f8004138c30 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.760+0000 7f800c1c7700 1 -- 172.21.15.16:0/4153194333 >> 172.21.15.16:0/4153194333 conn(0x7f8004066f50 msgr2=0x7f800409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:35.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.760+0000 7f800c1c7700 1 -- 172.21.15.16:0/4153194333 shutdown_connections 2023-09-30T22:30:35.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.760+0000 7f800c1c7700 1 -- 172.21.15.16:0/4153194333 wait complete. 2023-09-30T22:30:35.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.761+0000 7f800c1c7700 1 Processor -- start 2023-09-30T22:30:35.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.762+0000 7f800c1c7700 1 -- start start 2023-09-30T22:30:35.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.762+0000 7f800c1c7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f800409d6f0 0x7f8004120990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:35.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.762+0000 7f800c1c7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f800409e1d0 0x7f8004120ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:35.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.762+0000 7f800c1c7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8004139170 0x7f8004121410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:35.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.762+0000 7f800c1c7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8004122720 con 0x7f800409e1d0 2023-09-30T22:30:35.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.762+0000 7f800c1c7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8004136880 con 0x7f800409d6f0 2023-09-30T22:30:35.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.762+0000 7f800c1c7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f80041244b0 con 0x7f8004139170 2023-09-30T22:30:35.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.762+0000 7f8009f63700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f800409d6f0 0x7f8004120990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:35.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.762+0000 7f8009f63700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f800409d6f0 0x7f8004120990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38520/0 (socket says 172.21.15.16:38520) 2023-09-30T22:30:35.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.762+0000 7f8009f63700 1 -- 172.21.15.16:0/3990529606 learned_addr learned my addr 172.21.15.16:0/3990529606 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:35.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.762+0000 7f800a764700 1 --2- 172.21.15.16:0/3990529606 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8004139170 0x7f8004121410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.762+0000 7f8009762700 1 --2- 172.21.15.16:0/3990529606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f800409e1d0 0x7f8004120ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.763+0000 7f8009f63700 1 -- 172.21.15.16:0/3990529606 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8004139170 msgr2=0x7f8004121410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.763+0000 7f8009f63700 1 --2- 172.21.15.16:0/3990529606 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8004139170 0x7f8004121410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.763+0000 7f8009f63700 1 -- 172.21.15.16:0/3990529606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f800409e1d0 msgr2=0x7f8004120ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.763+0000 7f8009f63700 1 --2- 172.21.15.16:0/3990529606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f800409e1d0 0x7f8004120ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.763+0000 7f8009f63700 1 -- 172.21.15.16:0/3990529606 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ffc009a70 con 0x7f800409d6f0 2023-09-30T22:30:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.763+0000 7f8009762700 1 --2- 172.21.15.16:0/3990529606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f800409e1d0 0x7f8004120ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:30:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.763+0000 7f8009f63700 1 --2- 172.21.15.16:0/3990529606 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f800409d6f0 0x7f8004120990 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f800000c520 tx=0x7f800000c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.763+0000 7f7ffa7fc700 1 -- 172.21.15.16:0/3990529606 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8000004c80 con 0x7f800409d6f0 2023-09-30T22:30:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.763+0000 7f7ffa7fc700 1 -- 172.21.15.16:0/3990529606 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8000004de0 con 0x7f800409d6f0 2023-09-30T22:30:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.763+0000 7f800c1c7700 1 -- 172.21.15.16:0/3990529606 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ff40094a0 con 0x7f800409d6f0 2023-09-30T22:30:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.763+0000 7f7ffa7fc700 1 -- 172.21.15.16:0/3990529606 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8000007310 con 0x7f800409d6f0 2023-09-30T22:30:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.763+0000 7f800c1c7700 1 -- 172.21.15.16:0/3990529606 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8004072110 con 0x7f800409d6f0 2023-09-30T22:30:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.764+0000 7f7ffa7fc700 1 -- 172.21.15.16:0/3990529606 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8000007470 con 0x7f800409d6f0 2023-09-30T22:30:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.764+0000 7f800c1c7700 1 -- 172.21.15.16:0/3990529606 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8004074ea0 con 0x7f800409d6f0 2023-09-30T22:30:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.765+0000 7f7ffa7fc700 1 -- 172.21.15.16:0/3990529606 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f800003f3c0 con 0x7f800409d6f0 2023-09-30T22:30:35.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.774+0000 7f7ffa7fc700 1 -- 172.21.15.16:0/3990529606 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f80000133c0 con 0x7f800409d6f0 2023-09-30T22:30:35.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.811+0000 7fb7c4e22700 1 -- 172.21.15.16:0/2742929719 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb7c006b9e0 con 0x7fb7c0131810 2023-09-30T22:30:35.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.811+0000 7fb7aeffd700 1 -- 172.21.15.16:0/2742929719 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb7b8011b80 con 0x7fb7c0131810 2023-09-30T22:30:35.812 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:30:35.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.814+0000 7fb7c4e22700 1 -- 172.21.15.16:0/2742929719 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb7c0131810 msgr2=0x7fb7c0070090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:35.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.814+0000 7fb7c4e22700 1 --2- 172.21.15.16:0/2742929719 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb7c0131810 0x7fb7c0070090 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb7b8015000 tx=0x7fb7b8004ad0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.814+0000 7fb7c4e22700 1 -- 172.21.15.16:0/2742929719 shutdown_connections 2023-09-30T22:30:35.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.814+0000 7fb7c4e22700 1 --2- 172.21.15.16:0/2742929719 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb7c0131810 0x7fb7c0070090 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.814+0000 7fb7c4e22700 1 --2- 172.21.15.16:0/2742929719 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb7c0099a40 0x7fb7c006f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.814+0000 7fb7c4e22700 1 --2- 172.21.15.16:0/2742929719 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7c009a580 0x7fb7c006fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.814+0000 7fb7c4e22700 1 -- 172.21.15.16:0/2742929719 >> 172.21.15.16:0/2742929719 conn(0x7fb7c00a2aa0 msgr2=0x7fb7c012e480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:35.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.815+0000 7fb7c4e22700 1 -- 172.21.15.16:0/2742929719 shutdown_connections 2023-09-30T22:30:35.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.815+0000 7fb7c4e22700 1 -- 172.21.15.16:0/2742929719 wait complete. 2023-09-30T22:30:35.830 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:30:35.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.879+0000 7fc5e5678700 1 -- 172.21.15.16:0/2903450074 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc5e0074ea0 con 0x7fc5e0131810 2023-09-30T22:30:35.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.879+0000 7fc5c77fe700 1 -- 172.21.15.16:0/2903450074 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc5d8017030 con 0x7fc5e0131810 2023-09-30T22:30:35.880 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:30:35.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.882+0000 7fc5e5678700 1 -- 172.21.15.16:0/2903450074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5e0131810 msgr2=0x7fc5e0070090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:35.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.882+0000 7fc5e5678700 1 --2- 172.21.15.16:0/2903450074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5e0131810 0x7fc5e0070090 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc5d800c7b0 tx=0x7fc5d800cac0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.882+0000 7fc5e5678700 1 -- 172.21.15.16:0/2903450074 shutdown_connections 2023-09-30T22:30:35.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.882+0000 7fc5e5678700 1 --2- 172.21.15.16:0/2903450074 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc5e0099a40 0x7fc5e006f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.882+0000 7fc5e5678700 1 --2- 172.21.15.16:0/2903450074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5e0131810 0x7fc5e0070090 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.882+0000 7fc5e5678700 1 --2- 172.21.15.16:0/2903450074 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc5e009a3b0 0x7fc5e006fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.882+0000 7fc5e5678700 1 -- 172.21.15.16:0/2903450074 >> 172.21.15.16:0/2903450074 conn(0x7fc5e00a2aa0 msgr2=0x7fc5e00a1380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:35.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.882+0000 7fc5e5678700 1 -- 172.21.15.16:0/2903450074 shutdown_connections 2023-09-30T22:30:35.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.882+0000 7fc5e5678700 1 -- 172.21.15.16:0/2903450074 wait complete. 2023-09-30T22:30:35.897 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:30:35.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.938+0000 7fe44ab69700 1 -- 172.21.15.16:0/1360411120 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe444074ea0 con 0x7fe44409e000 2023-09-30T22:30:35.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.938+0000 7fe4417fa700 1 -- 172.21.15.16:0/1360411120 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe430024070 con 0x7fe44409e000 2023-09-30T22:30:35.939 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:30:35.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.941+0000 7fe44ab69700 1 -- 172.21.15.16:0/1360411120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe44409e000 msgr2=0x7fe44411e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:35.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.941+0000 7fe44ab69700 1 --2- 172.21.15.16:0/1360411120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe44409e000 0x7fe44411e410 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe430019000 tx=0x7fe430004750 comp rx=0 tx=0).stop 2023-09-30T22:30:35.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.941+0000 7fe44ab69700 1 -- 172.21.15.16:0/1360411120 shutdown_connections 2023-09-30T22:30:35.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.941+0000 7fe44ab69700 1 --2- 172.21.15.16:0/1360411120 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe44409e410 0x7fe44411e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.941+0000 7fe44ab69700 1 --2- 172.21.15.16:0/1360411120 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe44409d6f0 0x7fe44411ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.941+0000 7fe44ab69700 1 --2- 172.21.15.16:0/1360411120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe44409e000 0x7fe44411e410 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.941+0000 7fe44ab69700 1 -- 172.21.15.16:0/1360411120 >> 172.21.15.16:0/1360411120 conn(0x7fe444066f50 msgr2=0x7fe444070ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:35.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.941+0000 7fe44ab69700 1 -- 172.21.15.16:0/1360411120 shutdown_connections 2023-09-30T22:30:35.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.941+0000 7fe44ab69700 1 -- 172.21.15.16:0/1360411120 wait complete. 2023-09-30T22:30:35.955 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:30:35.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.984+0000 7f800c1c7700 1 -- 172.21.15.16:0/3990529606 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8004124290 con 0x7f800409d6f0 2023-09-30T22:30:35.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.985+0000 7f7ffa7fc700 1 -- 172.21.15.16:0/3990529606 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8000011360 con 0x7f800409d6f0 2023-09-30T22:30:35.985 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:30:35.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.987+0000 7f800c1c7700 1 -- 172.21.15.16:0/3990529606 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f800409d6f0 msgr2=0x7f8004120990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:35.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.987+0000 7f800c1c7700 1 --2- 172.21.15.16:0/3990529606 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f800409d6f0 0x7f8004120990 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f800000c520 tx=0x7f800000c830 comp rx=0 tx=0).stop 2023-09-30T22:30:35.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.987+0000 7f800c1c7700 1 -- 172.21.15.16:0/3990529606 shutdown_connections 2023-09-30T22:30:35.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.987+0000 7f800c1c7700 1 --2- 172.21.15.16:0/3990529606 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f800409d6f0 0x7f8004120990 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.987+0000 7f800c1c7700 1 --2- 172.21.15.16:0/3990529606 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8004139170 0x7f8004121410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.987+0000 7f800c1c7700 1 --2- 172.21.15.16:0/3990529606 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f800409e1d0 0x7f8004120ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:35.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.987+0000 7f800c1c7700 1 -- 172.21.15.16:0/3990529606 >> 172.21.15.16:0/3990529606 conn(0x7f8004066f50 msgr2=0x7f8004070ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:35.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.987+0000 7f800c1c7700 1 -- 172.21.15.16:0/3990529606 shutdown_connections 2023-09-30T22:30:35.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:35.987+0000 7f800c1c7700 1 -- 172.21.15.16:0/3990529606 wait complete. 2023-09-30T22:30:36.001 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:30:36.831 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:36.898 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:36.956 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:36.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:36.980+0000 7f71cf13e700 1 Processor -- start 2023-09-30T22:30:36.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:36.980+0000 7f71cf13e700 1 -- start start 2023-09-30T22:30:36.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:36.980+0000 7f71cf13e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f71c8133420 0x7f71c81337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:36.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:36.980+0000 7f71cf13e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f71c8133e30 0x7f71c812ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:36.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:36.980+0000 7f71cf13e700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71c806b990 con 0x7f71c8134260 2023-09-30T22:30:36.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:36.980+0000 7f71cf13e700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71c80a0590 con 0x7f71c8133e30 2023-09-30T22:30:36.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:36.980+0000 7f71cf13e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71c809fb80 con 0x7f71c8133420 2023-09-30T22:30:36.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:36.980+0000 7f71c7fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f71c8133e30 0x7f71c812ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:36.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:36.980+0000 7f71c7fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f71c8133e30 0x7f71c812ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55556/0 (socket says 172.21.15.16:55556) 2023-09-30T22:30:36.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:36.980+0000 7f71c7fff700 1 -- 172.21.15.16:0/2877930093 learned_addr learned my addr 172.21.15.16:0/2877930093 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:36.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:36.980+0000 7f71cceda700 1 --2- 172.21.15.16:0/2877930093 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f71c8133420 0x7f71c81337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:36.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:36.981+0000 7f71c7fff700 1 -- 172.21.15.16:0/2877930093 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f71c8133420 msgr2=0x7f71c81337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:36.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:36.981+0000 7f71c7fff700 1 --2- 172.21.15.16:0/2877930093 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f71c8133420 0x7f71c81337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:36.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:36.981+0000 7f71c7fff700 1 -- 172.21.15.16:0/2877930093 >> v1:172.21.15.16:6789/0 conn(0x7f71c8134260 legacy=0x7f71c81304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:36.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:36.981+0000 7f71c7fff700 1 -- 172.21.15.16:0/2877930093 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c80a0030 con 0x7f71c8133e30 2023-09-30T22:30:36.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:36.981+0000 7f71c7fff700 1 --2- 172.21.15.16:0/2877930093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71c8133e30 0x7f71c812ff90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f71b40098f0 tx=0x7f71b4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c1c20a9c4d93019 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:36.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:36.981+0000 7f71c77fe700 1 -- 172.21.15.16:0/2877930093 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b4011830 con 0x7f71c8133e30 2023-09-30T22:30:36.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:36.981+0000 7f71c77fe700 1 -- 172.21.15.16:0/2877930093 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71b4011990 con 0x7f71c8133e30 2023-09-30T22:30:36.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:36.981+0000 7f71c77fe700 1 -- 172.21.15.16:0/2877930093 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b4011d20 con 0x7f71c8133e30 2023-09-30T22:30:36.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:36.982+0000 7f71cf13e700 1 -- 172.21.15.16:0/2877930093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71c8133e30 msgr2=0x7f71c812ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:36.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:36.982+0000 7f71cf13e700 1 --2- 172.21.15.16:0/2877930093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71c8133e30 0x7f71c812ff90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f71b40098f0 tx=0x7f71b4009c00 comp rx=0 tx=0).stop 2023-09-30T22:30:36.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:36.982+0000 7f71cf13e700 1 -- 172.21.15.16:0/2877930093 shutdown_connections 2023-09-30T22:30:36.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:36.982+0000 7f71cf13e700 1 --2- 172.21.15.16:0/2877930093 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f71c8133420 0x7f71c81337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:36.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:36.982+0000 7f71cf13e700 1 --2- 172.21.15.16:0/2877930093 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71c8133e30 0x7f71c812ff90 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:36.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:36.982+0000 7f71cf13e700 1 -- 172.21.15.16:0/2877930093 >> 172.21.15.16:0/2877930093 conn(0x7f71c80a4210 msgr2=0x7f71c80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:36.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:36.982+0000 7f71cf13e700 1 -- 172.21.15.16:0/2877930093 shutdown_connections 2023-09-30T22:30:36.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:36.982+0000 7f71cf13e700 1 -- 172.21.15.16:0/2877930093 wait complete. 2023-09-30T22:30:36.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:36.983+0000 7f71cf13e700 1 Processor -- start 2023-09-30T22:30:36.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:36.983+0000 7f71cf13e700 1 -- start start 2023-09-30T22:30:36.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:36.984+0000 7f71cf13e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f71c8133420 0x7f71c809c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:36.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:36.984+0000 7f71cf13e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f71c8133e30 0x7f71c809ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:36.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:36.984+0000 7f71cf13e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71c8134260 0x7f71c809cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:36.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:36.984+0000 7f71cf13e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c812dc00 con 0x7f71c8134260 2023-09-30T22:30:36.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:36.984+0000 7f71cf13e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71c8120fc0 con 0x7f71c8133e30 2023-09-30T22:30:36.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:36.984+0000 7f71cf13e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f71c8120060 con 0x7f71c8133420 2023-09-30T22:30:36.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:36.984+0000 7f71cd6db700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71c8134260 0x7f71c809cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:36.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:36.984+0000 7f71cceda700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f71c8133420 0x7f71c809c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:36.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:36.984+0000 7f71cceda700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f71c8133420 0x7f71c809c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:54894/0 (socket says 172.21.15.16:54894) 2023-09-30T22:30:36.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:36.984+0000 7f71cd6db700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71c8134260 0x7f71c809cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55562/0 (socket says 172.21.15.16:55562) 2023-09-30T22:30:36.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:36.984+0000 7f71cceda700 1 -- 172.21.15.16:0/1952558661 learned_addr learned my addr 172.21.15.16:0/1952558661 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:36.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:36.984+0000 7f71c7fff700 1 --2- 172.21.15.16:0/1952558661 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f71c8133e30 0x7f71c809ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:36.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:36.985+0000 7f71cceda700 1 -- 172.21.15.16:0/1952558661 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f71c8133e30 msgr2=0x7f71c809ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:36.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:36.985+0000 7f71cceda700 1 --2- 172.21.15.16:0/1952558661 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f71c8133e30 0x7f71c809ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:36.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:36.985+0000 7f71cceda700 1 -- 172.21.15.16:0/1952558661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71c8134260 msgr2=0x7f71c809cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:36.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:36.985+0000 7f71cceda700 1 --2- 172.21.15.16:0/1952558661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71c8134260 0x7f71c809cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:36.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:36.985+0000 7f71cceda700 1 -- 172.21.15.16:0/1952558661 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b4009550 con 0x7f71c8133420 2023-09-30T22:30:36.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:36.985+0000 7f71cceda700 1 --2- 172.21.15.16:0/1952558661 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f71c8133420 0x7f71c809c540 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f71bc00e9b0 tx=0x7f71bc00ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:36.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:36.985+0000 7f71c57fa700 1 -- 172.21.15.16:0/1952558661 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71bc009710 con 0x7f71c8133420 2023-09-30T22:30:36.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:36.985+0000 7f71c57fa700 1 -- 172.21.15.16:0/1952558661 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71bc00ee50 con 0x7f71c8133420 2023-09-30T22:30:36.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:36.985+0000 7f71c57fa700 1 -- 172.21.15.16:0/1952558661 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71bc00ca50 con 0x7f71c8133420 2023-09-30T22:30:36.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:36.986+0000 7f71cf13e700 1 -- 172.21.15.16:0/1952558661 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71c8120ad0 con 0x7f71c8133420 2023-09-30T22:30:36.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:36.986+0000 7f71cf13e700 1 -- 172.21.15.16:0/1952558661 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71c809dfc0 con 0x7f71c8133420 2023-09-30T22:30:36.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:36.987+0000 7f71c57fa700 1 -- 172.21.15.16:0/1952558661 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f71bc00cbb0 con 0x7f71c8133420 2023-09-30T22:30:36.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:36.987+0000 7f71cf13e700 1 -- 172.21.15.16:0/1952558661 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71c811fed0 con 0x7f71c8133420 2023-09-30T22:30:36.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:36.987+0000 7f71c57fa700 1 -- 172.21.15.16:0/1952558661 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f71bc03f770 con 0x7f71c8133420 2023-09-30T22:30:36.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:36.992+0000 7f71c57fa700 1 -- 172.21.15.16:0/1952558661 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f71bc01be10 con 0x7f71c8133420 2023-09-30T22:30:37.002 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:37.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.052+0000 7f3fc1d27700 1 Processor -- start 2023-09-30T22:30:37.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.052+0000 7f3fc1d27700 1 -- start start 2023-09-30T22:30:37.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.053+0000 7f3fc1d27700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3fbc09f410 0x7f3fbc136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:37.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.053+0000 7f3fc1d27700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3fbc136fe0 0x7f3fbc1393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:37.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.053+0000 7f3fc1d27700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fbc070940 con 0x7f3fbc09e930 2023-09-30T22:30:37.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.053+0000 7f3fc1d27700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fbc06fd60 con 0x7f3fbc09f410 2023-09-30T22:30:37.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.053+0000 7f3fc1d27700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3fbc06fa80 con 0x7f3fbc136fe0 2023-09-30T22:30:37.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.053+0000 7f3fbaffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3fbc09f410 0x7f3fbc136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:37.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.054+0000 7f3fbb7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3fbc09e930 0x7f3fbc09ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34590/0 (socket says 172.21.15.16:34590) 2023-09-30T22:30:37.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.054+0000 7f3fbb7fe700 1 -- 172.21.15.16:0/1973426223 learned_addr learned my addr 172.21.15.16:0/1973426223 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:30:37.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.054+0000 7f3fbbfff700 1 --2- 172.21.15.16:0/1973426223 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3fbc136fe0 0x7f3fbc1393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:37.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.054+0000 7f3fbaffd700 1 -- 172.21.15.16:0/1973426223 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3fbc136fe0 msgr2=0x7f3fbc1393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:37.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.054+0000 7f3fbaffd700 1 --2- 172.21.15.16:0/1973426223 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3fbc136fe0 0x7f3fbc1393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:37.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.054+0000 7f3fbaffd700 1 -- 172.21.15.16:0/1973426223 >> v1:172.21.15.16:6789/0 conn(0x7f3fbc09e930 legacy=0x7f3fbc09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:37.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.054+0000 7f3fba7fc700 1 -- 172.21.15.16:0/1973426223 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2330075746 0 0) 0x7f3fbc070940 con 0x7f3fbc09e930 2023-09-30T22:30:37.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.054+0000 7f3fbaffd700 1 -- 172.21.15.16:0/1973426223 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fbc09fb60 con 0x7f3fbc09f410 2023-09-30T22:30:37.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.054+0000 7f3fbbfff700 1 --2- 172.21.15.16:0/1973426223 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3fbc136fe0 0x7f3fbc1393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:30:37.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.054+0000 7f3fbaffd700 1 --2- 172.21.15.16:0/1973426223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fbc09f410 0x7f3fbc136aa0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0009840 tx=0x7f3fb0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e83ffc2be09ccdb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:37.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.054+0000 7f3fba7fc700 1 -- 172.21.15.16:0/1973426223 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb0011830 con 0x7f3fbc09f410 2023-09-30T22:30:37.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.054+0000 7f3fba7fc700 1 -- 172.21.15.16:0/1973426223 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fb0011990 con 0x7f3fbc09f410 2023-09-30T22:30:37.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.054+0000 7f3fba7fc700 1 -- 172.21.15.16:0/1973426223 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb0011d20 con 0x7f3fbc09f410 2023-09-30T22:30:37.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.055+0000 7f3fc1d27700 1 -- 172.21.15.16:0/1973426223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fbc09f410 msgr2=0x7f3fbc136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:37.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.055+0000 7f3fc1d27700 1 --2- 172.21.15.16:0/1973426223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fbc09f410 0x7f3fbc136aa0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0009840 tx=0x7f3fb0009b50 comp rx=0 tx=0).stop 2023-09-30T22:30:37.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.056+0000 7f3fc1d27700 1 -- 172.21.15.16:0/1973426223 shutdown_connections 2023-09-30T22:30:37.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.056+0000 7f3fc1d27700 1 --2- 172.21.15.16:0/1973426223 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3fbc136fe0 0x7f3fbc1393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:37.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.056+0000 7f3fc1d27700 1 --2- 172.21.15.16:0/1973426223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fbc09f410 0x7f3fbc136aa0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:37.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.056+0000 7f3fc1d27700 1 -- 172.21.15.16:0/1973426223 >> 172.21.15.16:0/1973426223 conn(0x7f3fbc0a4210 msgr2=0x7f3fbc0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:37.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.056+0000 7f3fc1d27700 1 -- 172.21.15.16:0/1973426223 shutdown_connections 2023-09-30T22:30:37.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.056+0000 7f3fc1d27700 1 -- 172.21.15.16:0/1973426223 wait complete. 2023-09-30T22:30:37.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.057+0000 7f3fc1d27700 1 Processor -- start 2023-09-30T22:30:37.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.057+0000 7f3fc1d27700 1 -- start start 2023-09-30T22:30:37.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.058+0000 7f3fc1d27700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fbc09e930 0x7f3fbc06c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:37.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.058+0000 7f3fc1d27700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3fbc09f410 0x7f3fbc06c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:37.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.058+0000 7f3fc1d27700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3fbc136fe0 0x7f3fbc1c81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:37.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.058+0000 7f3fc1d27700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fbc121d00 con 0x7f3fbc09e930 2023-09-30T22:30:37.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.058+0000 7f3fc1d27700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fbc06bd90 con 0x7f3fbc09f410 2023-09-30T22:30:37.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.058+0000 7f3fc1d27700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3fbc06d470 con 0x7f3fbc136fe0 2023-09-30T22:30:37.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.058+0000 7f3fbbfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3fbc136fe0 0x7f3fbc1c81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:37.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.058+0000 7f3fbaffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3fbc09f410 0x7f3fbc06c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:37.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.058+0000 7f3fbbfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3fbc136fe0 0x7f3fbc1c81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:54916/0 (socket says 172.21.15.16:54916) 2023-09-30T22:30:37.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.058+0000 7f3fbbfff700 1 -- 172.21.15.16:0/1699146031 learned_addr learned my addr 172.21.15.16:0/1699146031 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:37.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.058+0000 7f3fbaffd700 1 --2- 172.21.15.16:0/1699146031 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3fbc09f410 0x7f3fbc06c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38548/0 (socket says 172.21.15.16:38548) 2023-09-30T22:30:37.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.059+0000 7f3fbb7fe700 1 --2- 172.21.15.16:0/1699146031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fbc09e930 0x7f3fbc06c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:37.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.059+0000 7f3fbaffd700 1 -- 172.21.15.16:0/1699146031 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3fbc136fe0 msgr2=0x7f3fbc1c81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:37.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.059+0000 7f3fbaffd700 1 --2- 172.21.15.16:0/1699146031 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3fbc136fe0 0x7f3fbc1c81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:37.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.059+0000 7f3fbaffd700 1 -- 172.21.15.16:0/1699146031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fbc09e930 msgr2=0x7f3fbc06c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:37.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.059+0000 7f3fbaffd700 1 --2- 172.21.15.16:0/1699146031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fbc09e930 0x7f3fbc06c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:37.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.059+0000 7f3fbaffd700 1 -- 172.21.15.16:0/1699146031 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fb00094a0 con 0x7f3fbc09f410 2023-09-30T22:30:37.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.059+0000 7f3fbbfff700 1 --2- 172.21.15.16:0/1699146031 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3fbc136fe0 0x7f3fbc1c81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:37.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.059+0000 7f3fbb7fe700 1 --2- 172.21.15.16:0/1699146031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fbc09e930 0x7f3fbc06c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:37.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.059+0000 7f3fbaffd700 1 --2- 172.21.15.16:0/1699146031 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3fbc09f410 0x7f3fbc06c620 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0016000 tx=0x7f3fb0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:37.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.060+0000 7f3fa3fff700 1 -- 172.21.15.16:0/1699146031 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb00036a0 con 0x7f3fbc09f410 2023-09-30T22:30:37.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.060+0000 7f3fa3fff700 1 -- 172.21.15.16:0/1699146031 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fb001f070 con 0x7f3fbc09f410 2023-09-30T22:30:37.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.060+0000 7f3fa3fff700 1 -- 172.21.15.16:0/1699146031 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb0003a40 con 0x7f3fbc09f410 2023-09-30T22:30:37.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.062+0000 7f3fc1d27700 1 -- 172.21.15.16:0/1699146031 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fbc1322d0 con 0x7f3fbc09f410 2023-09-30T22:30:37.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.062+0000 7f3fc1d27700 1 -- 172.21.15.16:0/1699146031 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fbc120af0 con 0x7f3fbc09f410 2023-09-30T22:30:37.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.063+0000 7f3fc1d27700 1 -- 172.21.15.16:0/1699146031 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fbc074ea0 con 0x7f3fbc09f410 2023-09-30T22:30:37.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.063+0000 7f3fa3fff700 1 -- 172.21.15.16:0/1699146031 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3fb0003ba0 con 0x7f3fbc09f410 2023-09-30T22:30:37.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.063+0000 7f3fa3fff700 1 -- 172.21.15.16:0/1699146031 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f3fb0021070 con 0x7f3fbc09f410 2023-09-30T22:30:37.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.068+0000 7f3fa3fff700 1 -- 172.21.15.16:0/1699146031 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3fb002f3a0 con 0x7f3fbc09f410 2023-09-30T22:30:37.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.101+0000 7f397f5a3700 1 Processor -- start 2023-09-30T22:30:37.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.101+0000 7f397f5a3700 1 -- start start 2023-09-30T22:30:37.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.102+0000 7f397f5a3700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f39780a0a30 0x7f39780a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:37.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.102+0000 7f397f5a3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3978134170 0x7f397813bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:37.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.102+0000 7f397f5a3700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f397809edd0 con 0x7f39780a1340 2023-09-30T22:30:37.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.102+0000 7f397f5a3700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f397809e1f0 con 0x7f39780a0a30 2023-09-30T22:30:37.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.102+0000 7f397f5a3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f397809dc50 con 0x7f3978134170 2023-09-30T22:30:37.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.102+0000 7f397cb3e700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f39780a1340 0x7f3978133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34606/0 (socket says 172.21.15.16:34606) 2023-09-30T22:30:37.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.102+0000 7f397cb3e700 1 -- 172.21.15.16:0/1846351808 learned_addr learned my addr 172.21.15.16:0/1846351808 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:30:37.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.102+0000 7f397d33f700 1 --2- 172.21.15.16:0/1846351808 >> v2:172.21.15.16:3300/0 conn(0x7f39780a0a30 0x7f39780a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:37.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.102+0000 7f397db40700 1 --2- 172.21.15.16:0/1846351808 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3978134170 0x7f397813bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:37.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.102+0000 7f396ffff700 1 -- 172.21.15.16:0/1846351808 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 555741621 0 0) 0x7f397809edd0 con 0x7f39780a1340 2023-09-30T22:30:37.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.102+0000 7f396ffff700 1 -- 172.21.15.16:0/1846351808 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39640035b0 con 0x7f39780a1340 2023-09-30T22:30:37.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.102+0000 7f397d33f700 1 -- 172.21.15.16:0/1846351808 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3978134170 msgr2=0x7f397813bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:37.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.102+0000 7f397d33f700 1 --2- 172.21.15.16:0/1846351808 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3978134170 0x7f397813bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:37.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.102+0000 7f397d33f700 1 -- 172.21.15.16:0/1846351808 >> v1:172.21.15.16:6789/0 conn(0x7f39780a1340 legacy=0x7f3978133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:37.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.102+0000 7f397d33f700 1 -- 172.21.15.16:0/1846351808 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39780a1fe0 con 0x7f39780a0a30 2023-09-30T22:30:37.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.103+0000 7f397d33f700 1 --2- 172.21.15.16:0/1846351808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39780a0a30 0x7f39780a0e00 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3974009840 tx=0x7f3974009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bad94580b161e724 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:37.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.103+0000 7f396ffff700 1 -- 172.21.15.16:0/1846351808 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3974011830 con 0x7f39780a0a30 2023-09-30T22:30:37.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.103+0000 7f396ffff700 1 -- 172.21.15.16:0/1846351808 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3974011990 con 0x7f39780a0a30 2023-09-30T22:30:37.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.103+0000 7f396ffff700 1 -- 172.21.15.16:0/1846351808 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3974011d20 con 0x7f39780a0a30 2023-09-30T22:30:37.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.104+0000 7f397f5a3700 1 -- 172.21.15.16:0/1846351808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39780a0a30 msgr2=0x7f39780a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:37.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.104+0000 7f397f5a3700 1 --2- 172.21.15.16:0/1846351808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39780a0a30 0x7f39780a0e00 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3974009840 tx=0x7f3974009b50 comp rx=0 tx=0).stop 2023-09-30T22:30:37.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.104+0000 7f397f5a3700 1 -- 172.21.15.16:0/1846351808 shutdown_connections 2023-09-30T22:30:37.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.104+0000 7f397f5a3700 1 --2- 172.21.15.16:0/1846351808 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3978134170 0x7f397813bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:37.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.104+0000 7f397f5a3700 1 --2- 172.21.15.16:0/1846351808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39780a0a30 0x7f39780a0e00 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:37.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.104+0000 7f397f5a3700 1 -- 172.21.15.16:0/1846351808 >> 172.21.15.16:0/1846351808 conn(0x7f39780a6550 msgr2=0x7f39780a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:37.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.104+0000 7f397f5a3700 1 -- 172.21.15.16:0/1846351808 shutdown_connections 2023-09-30T22:30:37.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.104+0000 7f397f5a3700 1 -- 172.21.15.16:0/1846351808 wait complete. 2023-09-30T22:30:37.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.105+0000 7f397f5a3700 1 Processor -- start 2023-09-30T22:30:37.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.105+0000 7f397f5a3700 1 -- start start 2023-09-30T22:30:37.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.105+0000 7f397f5a3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f39780a0a30 0x7f39781232c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:37.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.105+0000 7f397f5a3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39780a1340 0x7f3978123800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:37.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.105+0000 7f397f5a3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3978134170 0x7f39781c8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:37.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.105+0000 7f397f5a3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3978124e10 con 0x7f39780a1340 2023-09-30T22:30:37.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.105+0000 7f397f5a3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3978126430 con 0x7f3978134170 2023-09-30T22:30:37.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.105+0000 7f397f5a3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3978122ed0 con 0x7f39780a0a30 2023-09-30T22:30:37.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.105+0000 7f397d33f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f39780a0a30 0x7f39781232c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:37.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.106+0000 7f397d33f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f39780a0a30 0x7f39781232c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:54926/0 (socket says 172.21.15.16:54926) 2023-09-30T22:30:37.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.106+0000 7f397d33f700 1 -- 172.21.15.16:0/554075442 learned_addr learned my addr 172.21.15.16:0/554075442 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:37.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.106+0000 7f397cb3e700 1 --2- 172.21.15.16:0/554075442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39780a1340 0x7f3978123800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:37.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.106+0000 7f397d33f700 1 -- 172.21.15.16:0/554075442 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3978134170 msgr2=0x7f39781c8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:37.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.106+0000 7f397d33f700 1 --2- 172.21.15.16:0/554075442 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3978134170 0x7f39781c8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:37.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.106+0000 7f397d33f700 1 -- 172.21.15.16:0/554075442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39780a1340 msgr2=0x7f3978123800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:37.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.106+0000 7f397d33f700 1 --2- 172.21.15.16:0/554075442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39780a1340 0x7f3978123800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:37.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.106+0000 7f397d33f700 1 -- 172.21.15.16:0/554075442 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39740094a0 con 0x7f39780a0a30 2023-09-30T22:30:37.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.106+0000 7f397d33f700 1 --2- 172.21.15.16:0/554075442 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f39780a0a30 0x7f39781232c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3974000c00 tx=0x7f3974004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:37.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.106+0000 7f396dffb700 1 -- 172.21.15.16:0/554075442 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3974004550 con 0x7f39780a0a30 2023-09-30T22:30:37.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.106+0000 7f396dffb700 1 -- 172.21.15.16:0/554075442 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f397401f070 con 0x7f39780a0a30 2023-09-30T22:30:37.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.106+0000 7f396dffb700 1 -- 172.21.15.16:0/554075442 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39740038e0 con 0x7f39780a0a30 2023-09-30T22:30:37.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.106+0000 7f397f5a3700 1 -- 172.21.15.16:0/554075442 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39780a25e0 con 0x7f39780a0a30 2023-09-30T22:30:37.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.107+0000 7f397f5a3700 1 -- 172.21.15.16:0/554075442 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3978124a70 con 0x7f39780a0a30 2023-09-30T22:30:37.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.108+0000 7f396dffb700 1 -- 172.21.15.16:0/554075442 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3974003a40 con 0x7f39780a0a30 2023-09-30T22:30:37.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.108+0000 7f396dffb700 1 -- 172.21.15.16:0/554075442 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f3974021070 con 0x7f39780a0a30 2023-09-30T22:30:37.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.108+0000 7f397f5a3700 1 -- 172.21.15.16:0/554075442 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3978074ea0 con 0x7f39780a0a30 2023-09-30T22:30:37.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.113+0000 7f396dffb700 1 -- 172.21.15.16:0/554075442 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3974015080 con 0x7f39780a0a30 2023-09-30T22:30:37.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.187+0000 7f0e7833f700 1 Processor -- start 2023-09-30T22:30:37.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.187+0000 7f0e7833f700 1 -- start start 2023-09-30T22:30:37.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.188+0000 7f0e7833f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0e7006a770 0x7f0e7006ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:37.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.188+0000 7f0e7833f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0e7006b080 0x7f0e70131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:37.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.188+0000 7f0e7833f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0e70131b40 0x7f0e70131fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:37.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.188+0000 7f0e7833f700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e7009ece0 con 0x7f0e7006b080 2023-09-30T22:30:37.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.188+0000 7f0e7833f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7009e760 con 0x7f0e70131b40 2023-09-30T22:30:37.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.188+0000 7f0e7833f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7009e040 con 0x7f0e7006a770 2023-09-30T22:30:37.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.188+0000 7f0e760db700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0e7006a770 0x7f0e7006ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:37.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.188+0000 7f0e760db700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0e7006a770 0x7f0e7006ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:54928/0 (socket says 172.21.15.16:54928) 2023-09-30T22:30:37.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.188+0000 7f0e760db700 1 -- 172.21.15.16:0/1377014411 learned_addr learned my addr 172.21.15.16:0/1377014411 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:37.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.188+0000 7f0e758da700 1 --2- 172.21.15.16:0/1377014411 >> v2:172.21.15.16:3300/0 conn(0x7f0e7006b080 0x7f0e70131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:37.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.188+0000 7f0e768dc700 1 --2- 172.21.15.16:0/1377014411 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0e70131b40 0x7f0e70131fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:37.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.188+0000 7f0e758da700 1 -- 172.21.15.16:0/1377014411 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0e7006a770 msgr2=0x7f0e7006ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:37.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.188+0000 7f0e758da700 1 --2- 172.21.15.16:0/1377014411 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0e7006a770 0x7f0e7006ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:37.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.188+0000 7f0e758da700 1 -- 172.21.15.16:0/1377014411 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0e70131b40 msgr2=0x7f0e70131fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:37.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.188+0000 7f0e758da700 1 --2- 172.21.15.16:0/1377014411 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0e70131b40 0x7f0e70131fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:37.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.189+0000 7f0e758da700 1 -- 172.21.15.16:0/1377014411 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e7009f630 con 0x7f0e7006b080 2023-09-30T22:30:37.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.189+0000 7f0e760db700 1 --2- 172.21.15.16:0/1377014411 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0e7006a770 0x7f0e7006ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:37.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.189+0000 7f0e758da700 1 --2- 172.21.15.16:0/1377014411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e7006b080 0x7f0e70131600 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0e60009840 tx=0x7f0e60009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=899d34cfb979cb2e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:37.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.189+0000 7f0e750d9700 1 -- 172.21.15.16:0/1377014411 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e60011830 con 0x7f0e7006b080 2023-09-30T22:30:37.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.189+0000 7f0e750d9700 1 -- 172.21.15.16:0/1377014411 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e60011990 con 0x7f0e7006b080 2023-09-30T22:30:37.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.189+0000 7f0e750d9700 1 -- 172.21.15.16:0/1377014411 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e60011d20 con 0x7f0e7006b080 2023-09-30T22:30:37.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.190+0000 7f0e7833f700 1 -- 172.21.15.16:0/1377014411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e7006b080 msgr2=0x7f0e70131600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:37.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.190+0000 7f0e7833f700 1 --2- 172.21.15.16:0/1377014411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e7006b080 0x7f0e70131600 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0e60009840 tx=0x7f0e60009b50 comp rx=0 tx=0).stop 2023-09-30T22:30:37.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.190+0000 7f0e7833f700 1 -- 172.21.15.16:0/1377014411 shutdown_connections 2023-09-30T22:30:37.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.190+0000 7f0e7833f700 1 --2- 172.21.15.16:0/1377014411 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0e70131b40 0x7f0e70131fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:37.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.190+0000 7f0e7833f700 1 --2- 172.21.15.16:0/1377014411 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0e7006a770 0x7f0e7006ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:37.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.190+0000 7f0e7833f700 1 --2- 172.21.15.16:0/1377014411 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e7006b080 0x7f0e70131600 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:37.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.190+0000 7f0e7833f700 1 -- 172.21.15.16:0/1377014411 >> 172.21.15.16:0/1377014411 conn(0x7f0e70066f50 msgr2=0x7f0e7009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:37.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.190+0000 7f0e7833f700 1 -- 172.21.15.16:0/1377014411 shutdown_connections 2023-09-30T22:30:37.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.190+0000 7f0e7833f700 1 -- 172.21.15.16:0/1377014411 wait complete. 2023-09-30T22:30:37.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.191+0000 7f0e7833f700 1 Processor -- start 2023-09-30T22:30:37.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.191+0000 7f0e7833f700 1 -- start start 2023-09-30T22:30:37.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.191+0000 7f0e7833f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e7006a770 0x7f0e70120000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:37.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.191+0000 7f0e7833f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0e7006b080 0x7f0e70120540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:37.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.191+0000 7f0e7833f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0e70131b40 0x7f0e70120a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:37.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.191+0000 7f0e7833f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e70121e40 con 0x7f0e7006a770 2023-09-30T22:30:37.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.191+0000 7f0e7833f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e70121c10 con 0x7f0e70131b40 2023-09-30T22:30:37.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.191+0000 7f0e7833f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0e70122ed0 con 0x7f0e7006b080 2023-09-30T22:30:37.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.192+0000 7f0e768dc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0e70131b40 0x7f0e70120a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:37.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.192+0000 7f0e758da700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0e7006b080 0x7f0e70120540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:37.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.192+0000 7f0e768dc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0e70131b40 0x7f0e70120a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38578/0 (socket says 172.21.15.16:38578) 2023-09-30T22:30:37.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.192+0000 7f0e768dc700 1 -- 172.21.15.16:0/766177232 learned_addr learned my addr 172.21.15.16:0/766177232 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:37.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.192+0000 7f0e760db700 1 --2- 172.21.15.16:0/766177232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e7006a770 0x7f0e70120000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:37.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.192+0000 7f0e758da700 1 -- 172.21.15.16:0/766177232 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0e70131b40 msgr2=0x7f0e70120a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:37.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.192+0000 7f0e758da700 1 --2- 172.21.15.16:0/766177232 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0e70131b40 0x7f0e70120a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:37.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.192+0000 7f0e758da700 1 -- 172.21.15.16:0/766177232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e7006a770 msgr2=0x7f0e70120000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:37.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.192+0000 7f0e758da700 1 --2- 172.21.15.16:0/766177232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e7006a770 0x7f0e70120000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:37.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.192+0000 7f0e758da700 1 -- 172.21.15.16:0/766177232 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e6c009550 con 0x7f0e7006b080 2023-09-30T22:30:37.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.192+0000 7f0e768dc700 1 --2- 172.21.15.16:0/766177232 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0e70131b40 0x7f0e70120a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:37.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.192+0000 7f0e758da700 1 --2- 172.21.15.16:0/766177232 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0e7006b080 0x7f0e70120540 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0e60016000 tx=0x7f0e60004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:37.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.193+0000 7f0e66ffd700 1 -- 172.21.15.16:0/766177232 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e60004450 con 0x7f0e7006b080 2023-09-30T22:30:37.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.193+0000 7f0e66ffd700 1 -- 172.21.15.16:0/766177232 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e6001f070 con 0x7f0e7006b080 2023-09-30T22:30:37.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.193+0000 7f0e7833f700 1 -- 172.21.15.16:0/766177232 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e600094a0 con 0x7f0e7006b080 2023-09-30T22:30:37.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.193+0000 7f0e66ffd700 1 -- 172.21.15.16:0/766177232 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e600038e0 con 0x7f0e7006b080 2023-09-30T22:30:37.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.193+0000 7f0e7833f700 1 -- 172.21.15.16:0/766177232 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e70071e60 con 0x7f0e7006b080 2023-09-30T22:30:37.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.194+0000 7f0e66ffd700 1 -- 172.21.15.16:0/766177232 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0e60003a40 con 0x7f0e7006b080 2023-09-30T22:30:37.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.194+0000 7f0e7833f700 1 -- 172.21.15.16:0/766177232 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e70074ea0 con 0x7f0e7006b080 2023-09-30T22:30:37.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.194+0000 7f0e66ffd700 1 -- 172.21.15.16:0/766177232 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f0e60021070 con 0x7f0e7006b080 2023-09-30T22:30:37.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.203+0000 7f0e66ffd700 1 -- 172.21.15.16:0/766177232 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0e60010380 con 0x7f0e7006b080 2023-09-30T22:30:37.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.219+0000 7f71cf13e700 1 -- 172.21.15.16:0/1952558661 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f71c8074ea0 con 0x7f71c8133420 2023-09-30T22:30:37.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.220+0000 7f71c57fa700 1 -- 172.21.15.16:0/1952558661 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f71bc021020 con 0x7f71c8133420 2023-09-30T22:30:37.220 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:30:37.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.222+0000 7f71cf13e700 1 -- 172.21.15.16:0/1952558661 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f71c8133420 msgr2=0x7f71c809c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:37.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.222+0000 7f71cf13e700 1 --2- 172.21.15.16:0/1952558661 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f71c8133420 0x7f71c809c540 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f71bc00e9b0 tx=0x7f71bc00ecc0 comp rx=0 tx=0).stop 2023-09-30T22:30:37.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.222+0000 7f71cf13e700 1 -- 172.21.15.16:0/1952558661 shutdown_connections 2023-09-30T22:30:37.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.222+0000 7f71cf13e700 1 --2- 172.21.15.16:0/1952558661 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71c8134260 0x7f71c809cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:37.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.222+0000 7f71cf13e700 1 --2- 172.21.15.16:0/1952558661 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f71c8133420 0x7f71c809c540 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:37.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.222+0000 7f71cf13e700 1 --2- 172.21.15.16:0/1952558661 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f71c8133e30 0x7f71c809ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:37.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.222+0000 7f71cf13e700 1 -- 172.21.15.16:0/1952558661 >> 172.21.15.16:0/1952558661 conn(0x7f71c80a4210 msgr2=0x7f71c80a0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:37.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.222+0000 7f71cf13e700 1 -- 172.21.15.16:0/1952558661 shutdown_connections 2023-09-30T22:30:37.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.222+0000 7f71cf13e700 1 -- 172.21.15.16:0/1952558661 wait complete. 2023-09-30T22:30:37.237 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:30:37.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.323+0000 7f397f5a3700 1 -- 172.21.15.16:0/554075442 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f39780a4a20 con 0x7f39780a0a30 2023-09-30T22:30:37.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.323+0000 7f396dffb700 1 -- 172.21.15.16:0/554075442 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f397401a430 con 0x7f39780a0a30 2023-09-30T22:30:37.323 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:30:37.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.325+0000 7f397f5a3700 1 -- 172.21.15.16:0/554075442 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f39780a0a30 msgr2=0x7f39781232c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:37.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.325+0000 7f397f5a3700 1 --2- 172.21.15.16:0/554075442 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f39780a0a30 0x7f39781232c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3974000c00 tx=0x7f3974004750 comp rx=0 tx=0).stop 2023-09-30T22:30:37.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.325+0000 7f397f5a3700 1 -- 172.21.15.16:0/554075442 shutdown_connections 2023-09-30T22:30:37.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.325+0000 7f397f5a3700 1 --2- 172.21.15.16:0/554075442 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3978134170 0x7f39781c8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:37.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.325+0000 7f397f5a3700 1 --2- 172.21.15.16:0/554075442 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f39780a0a30 0x7f39781232c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:37.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.325+0000 7f397f5a3700 1 --2- 172.21.15.16:0/554075442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39780a1340 0x7f3978123800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:37.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.325+0000 7f397f5a3700 1 -- 172.21.15.16:0/554075442 >> 172.21.15.16:0/554075442 conn(0x7f39780a6550 msgr2=0x7f397809edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:37.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.325+0000 7f397f5a3700 1 -- 172.21.15.16:0/554075442 shutdown_connections 2023-09-30T22:30:37.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.326+0000 7f397f5a3700 1 -- 172.21.15.16:0/554075442 wait complete. 2023-09-30T22:30:37.340 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:30:37.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.368+0000 7f3fc1d27700 1 -- 172.21.15.16:0/1699146031 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3fbc06b250 con 0x7f3fbc09f410 2023-09-30T22:30:37.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.369+0000 7f3fa3fff700 1 -- 172.21.15.16:0/1699146031 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3fb002b030 con 0x7f3fbc09f410 2023-09-30T22:30:37.369 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:30:37.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.371+0000 7f3fc1d27700 1 -- 172.21.15.16:0/1699146031 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3fbc09f410 msgr2=0x7f3fbc06c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:37.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.371+0000 7f3fc1d27700 1 --2- 172.21.15.16:0/1699146031 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3fbc09f410 0x7f3fbc06c620 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3fb0016000 tx=0x7f3fb0004750 comp rx=0 tx=0).stop 2023-09-30T22:30:37.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.371+0000 7f3fc1d27700 1 -- 172.21.15.16:0/1699146031 shutdown_connections 2023-09-30T22:30:37.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.371+0000 7f3fc1d27700 1 --2- 172.21.15.16:0/1699146031 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3fbc136fe0 0x7f3fbc1c81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:37.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.371+0000 7f3fc1d27700 1 --2- 172.21.15.16:0/1699146031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fbc09e930 0x7f3fbc06c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:37.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.371+0000 7f3fc1d27700 1 --2- 172.21.15.16:0/1699146031 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3fbc09f410 0x7f3fbc06c620 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:37.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.371+0000 7f3fc1d27700 1 -- 172.21.15.16:0/1699146031 >> 172.21.15.16:0/1699146031 conn(0x7f3fbc0a4210 msgr2=0x7f3fbc134b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:37.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.371+0000 7f3fc1d27700 1 -- 172.21.15.16:0/1699146031 shutdown_connections 2023-09-30T22:30:37.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.371+0000 7f3fc1d27700 1 -- 172.21.15.16:0/1699146031 wait complete. 2023-09-30T22:30:37.385 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:30:37.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.448+0000 7f0e7833f700 1 -- 172.21.15.16:0/766177232 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f0e70122a10 con 0x7f0e7006b080 2023-09-30T22:30:37.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.448+0000 7f0e66ffd700 1 -- 172.21.15.16:0/766177232 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0e6002b070 con 0x7f0e7006b080 2023-09-30T22:30:37.448 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:30:37.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.450+0000 7f0e7833f700 1 -- 172.21.15.16:0/766177232 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0e7006b080 msgr2=0x7f0e70120540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:37.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.450+0000 7f0e7833f700 1 --2- 172.21.15.16:0/766177232 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0e7006b080 0x7f0e70120540 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0e60016000 tx=0x7f0e60004750 comp rx=0 tx=0).stop 2023-09-30T22:30:37.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.450+0000 7f0e7833f700 1 -- 172.21.15.16:0/766177232 shutdown_connections 2023-09-30T22:30:37.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.450+0000 7f0e7833f700 1 --2- 172.21.15.16:0/766177232 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0e70131b40 0x7f0e70120a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:37.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.450+0000 7f0e7833f700 1 --2- 172.21.15.16:0/766177232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e7006a770 0x7f0e70120000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:37.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.450+0000 7f0e7833f700 1 --2- 172.21.15.16:0/766177232 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0e7006b080 0x7f0e70120540 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:37.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.450+0000 7f0e7833f700 1 -- 172.21.15.16:0/766177232 >> 172.21.15.16:0/766177232 conn(0x7f0e70066f50 msgr2=0x7f0e700a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:37.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.450+0000 7f0e7833f700 1 -- 172.21.15.16:0/766177232 shutdown_connections 2023-09-30T22:30:37.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:37.450+0000 7f0e7833f700 1 -- 172.21.15.16:0/766177232 wait complete. 2023-09-30T22:30:37.464 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:30:38.238 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:38.341 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:38.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.385+0000 7f048ff51700 1 Processor -- start 2023-09-30T22:30:38.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.386+0000 7f048ff51700 1 -- start start 2023-09-30T22:30:38.386 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:38.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.386+0000 7f048ff51700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f04880a0a30 0x7f04880a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:38.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.386+0000 7f048ff51700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f04880a1340 0x7f0488133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:38.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.386+0000 7f048ff51700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f048809edd0 con 0x7f0488133fa0 2023-09-30T22:30:38.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.386+0000 7f048ff51700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f048809e1f0 con 0x7f04880a0a30 2023-09-30T22:30:38.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.386+0000 7f048ff51700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f048809dc50 con 0x7f04880a1340 2023-09-30T22:30:38.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.386+0000 7f048dced700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f04880a0a30 0x7f04880a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:38.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.386+0000 7f048dced700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f04880a0a30 0x7f04880a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51794/0 (socket says 172.21.15.16:51794) 2023-09-30T22:30:38.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.386+0000 7f048e4ee700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0488133fa0 0x7f048813ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58238/0 (socket says 172.21.15.16:58238) 2023-09-30T22:30:38.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.386+0000 7f048dced700 1 -- 172.21.15.16:0/796705191 learned_addr learned my addr 172.21.15.16:0/796705191 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:38.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.386+0000 7f048d4ec700 1 --2- 172.21.15.16:0/796705191 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f04880a1340 0x7f0488133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:38.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.387+0000 7f048dced700 1 -- 172.21.15.16:0/796705191 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f04880a1340 msgr2=0x7f0488133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:38.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.387+0000 7f048dced700 1 --2- 172.21.15.16:0/796705191 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f04880a1340 0x7f0488133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.387+0000 7f048dced700 1 -- 172.21.15.16:0/796705191 >> v1:172.21.15.16:6789/0 conn(0x7f0488133fa0 legacy=0x7f048813ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:38.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.387+0000 7f048dced700 1 -- 172.21.15.16:0/796705191 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04880a1fe0 con 0x7f04880a0a30 2023-09-30T22:30:38.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.387+0000 7f048cceb700 1 -- 172.21.15.16:0/796705191 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 57901732 0 0) 0x7f048809edd0 con 0x7f0488133fa0 2023-09-30T22:30:38.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.387+0000 7f048dced700 1 --2- 172.21.15.16:0/796705191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04880a0a30 0x7f04880a0e00 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f04800098f0 tx=0x7f0480009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3b7926162fc00d0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:38.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.387+0000 7f048cceb700 1 -- 172.21.15.16:0/796705191 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0480011830 con 0x7f04880a0a30 2023-09-30T22:30:38.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.387+0000 7f048cceb700 1 -- 172.21.15.16:0/796705191 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0480011990 con 0x7f04880a0a30 2023-09-30T22:30:38.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.387+0000 7f048cceb700 1 -- 172.21.15.16:0/796705191 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0480011d20 con 0x7f04880a0a30 2023-09-30T22:30:38.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.388+0000 7f048ff51700 1 -- 172.21.15.16:0/796705191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04880a0a30 msgr2=0x7f04880a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:38.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.388+0000 7f048ff51700 1 --2- 172.21.15.16:0/796705191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04880a0a30 0x7f04880a0e00 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f04800098f0 tx=0x7f0480009c00 comp rx=0 tx=0).stop 2023-09-30T22:30:38.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.388+0000 7f048ff51700 1 -- 172.21.15.16:0/796705191 shutdown_connections 2023-09-30T22:30:38.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.388+0000 7f048ff51700 1 --2- 172.21.15.16:0/796705191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04880a0a30 0x7f04880a0e00 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.388+0000 7f048ff51700 1 --2- 172.21.15.16:0/796705191 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f04880a1340 0x7f0488133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.388+0000 7f048ff51700 1 -- 172.21.15.16:0/796705191 >> 172.21.15.16:0/796705191 conn(0x7f04880a6550 msgr2=0x7f04880a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:38.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.388+0000 7f048ff51700 1 -- 172.21.15.16:0/796705191 shutdown_connections 2023-09-30T22:30:38.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.388+0000 7f048ff51700 1 -- 172.21.15.16:0/796705191 wait complete. 2023-09-30T22:30:38.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.389+0000 7f048ff51700 1 Processor -- start 2023-09-30T22:30:38.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.389+0000 7f048ff51700 1 -- start start 2023-09-30T22:30:38.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.390+0000 7f048ff51700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04880a0a30 0x7f0488122930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:38.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.390+0000 7f048ff51700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f04880a1340 0x7f0488122e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:38.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.390+0000 7f048ff51700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0488133fa0 0x7f04881233b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:38.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.390+0000 7f048ff51700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04881244b0 con 0x7f04880a0a30 2023-09-30T22:30:38.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.390+0000 7f048ff51700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0488124d80 con 0x7f0488133fa0 2023-09-30T22:30:38.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.390+0000 7f048ff51700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f04881263c0 con 0x7f04880a1340 2023-09-30T22:30:38.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.390+0000 7f048d4ec700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f04880a1340 0x7f0488122e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:38.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.390+0000 7f048d4ec700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f04880a1340 0x7f0488122e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:43512/0 (socket says 172.21.15.16:43512) 2023-09-30T22:30:38.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.390+0000 7f048d4ec700 1 -- 172.21.15.16:0/3831980914 learned_addr learned my addr 172.21.15.16:0/3831980914 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:38.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.390+0000 7f048e4ee700 1 --2- 172.21.15.16:0/3831980914 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0488133fa0 0x7f04881233b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:38.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.390+0000 7f048dced700 1 --2- 172.21.15.16:0/3831980914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04880a0a30 0x7f0488122930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:38.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.390+0000 7f048d4ec700 1 -- 172.21.15.16:0/3831980914 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0488133fa0 msgr2=0x7f04881233b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:38.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.390+0000 7f048d4ec700 1 --2- 172.21.15.16:0/3831980914 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0488133fa0 0x7f04881233b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.390+0000 7f048d4ec700 1 -- 172.21.15.16:0/3831980914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04880a0a30 msgr2=0x7f0488122930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:38.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.390+0000 7f048d4ec700 1 --2- 172.21.15.16:0/3831980914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04880a0a30 0x7f0488122930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.390+0000 7f048d4ec700 1 -- 172.21.15.16:0/3831980914 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0480009550 con 0x7f04880a1340 2023-09-30T22:30:38.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.391+0000 7f048d4ec700 1 --2- 172.21.15.16:0/3831980914 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f04880a1340 0x7f0488122e70 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f047c00e9e0 tx=0x7f047c00ecf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:38.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.391+0000 7f04867fc700 1 -- 172.21.15.16:0/3831980914 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047c009710 con 0x7f04880a1340 2023-09-30T22:30:38.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.391+0000 7f04867fc700 1 -- 172.21.15.16:0/3831980914 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f047c00ee80 con 0x7f04880a1340 2023-09-30T22:30:38.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.391+0000 7f048ff51700 1 -- 172.21.15.16:0/3831980914 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0488124a60 con 0x7f04880a1340 2023-09-30T22:30:38.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.391+0000 7f04867fc700 1 -- 172.21.15.16:0/3831980914 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f047c00ca50 con 0x7f04880a1340 2023-09-30T22:30:38.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.391+0000 7f048ff51700 1 -- 172.21.15.16:0/3831980914 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04881260e0 con 0x7f04880a1340 2023-09-30T22:30:38.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.392+0000 7f04867fc700 1 -- 172.21.15.16:0/3831980914 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f047c00cbb0 con 0x7f04880a1340 2023-09-30T22:30:38.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.392+0000 7f04867fc700 1 -- 172.21.15.16:0/3831980914 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f047c03fab0 con 0x7f04880a1340 2023-09-30T22:30:38.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.392+0000 7f048ff51700 1 -- 172.21.15.16:0/3831980914 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04880a1d40 con 0x7f04880a1340 2023-09-30T22:30:38.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.397+0000 7f04867fc700 1 -- 172.21.15.16:0/3831980914 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f047c017020 con 0x7f04880a1340 2023-09-30T22:30:38.465 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:38.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.495+0000 7f366e3cf700 1 Processor -- start 2023-09-30T22:30:38.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.495+0000 7f366e3cf700 1 -- start start 2023-09-30T22:30:38.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.495+0000 7f366e3cf700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3668130540 0x7f3668130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:38.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.495+0000 7f366e3cf700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f366806fcf0 0x7f3668138570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:38.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.495+0000 7f366e3cf700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36680716a0 con 0x7f3668130e50 2023-09-30T22:30:38.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.495+0000 7f366e3cf700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f366809a390 con 0x7f366806fcf0 2023-09-30T22:30:38.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.495+0000 7f366e3cf700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3668071290 con 0x7f3668130540 2023-09-30T22:30:38.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.496+0000 7f3667fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3668130540 0x7f3668130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:38.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.496+0000 7f36677fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3668130e50 0x7f366806f5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58248/0 (socket says 172.21.15.16:58248) 2023-09-30T22:30:38.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.496+0000 7f36677fe700 1 -- 172.21.15.16:0/4107015603 learned_addr learned my addr 172.21.15.16:0/4107015603 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:30:38.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.496+0000 7f3667fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3668130540 0x7f3668130910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:43528/0 (socket says 172.21.15.16:43528) 2023-09-30T22:30:38.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.496+0000 7f366c96c700 1 --2- 172.21.15.16:0/4107015603 >> v2:172.21.15.16:3300/0 conn(0x7f366806fcf0 0x7f3668138570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:38.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.496+0000 7f3667fff700 1 -- 172.21.15.16:0/4107015603 >> v2:172.21.15.16:3300/0 conn(0x7f366806fcf0 msgr2=0x7f3668138570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:38.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.496+0000 7f3667fff700 1 --2- 172.21.15.16:0/4107015603 >> v2:172.21.15.16:3300/0 conn(0x7f366806fcf0 0x7f3668138570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.496+0000 7f3666ffd700 1 -- 172.21.15.16:0/4107015603 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3609757105 0 0) 0x7f36680716a0 con 0x7f3668130e50 2023-09-30T22:30:38.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.496+0000 7f3667fff700 1 -- 172.21.15.16:0/4107015603 >> v1:172.21.15.16:6789/0 conn(0x7f3668130e50 legacy=0x7f366806f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:38.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.496+0000 7f3667fff700 1 -- 172.21.15.16:0/4107015603 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3668130340 con 0x7f3668130540 2023-09-30T22:30:38.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.496+0000 7f3667fff700 1 --2- 172.21.15.16:0/4107015603 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3668130540 0x7f3668130910 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f365800bb10 tx=0x7f365800be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=906b20c010bc172e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:38.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.497+0000 7f3666ffd700 1 -- 172.21.15.16:0/4107015603 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365800dc30 con 0x7f3668130540 2023-09-30T22:30:38.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.497+0000 7f3666ffd700 1 -- 172.21.15.16:0/4107015603 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f365800dd90 con 0x7f3668130540 2023-09-30T22:30:38.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.497+0000 7f3666ffd700 1 -- 172.21.15.16:0/4107015603 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3658010690 con 0x7f3668130540 2023-09-30T22:30:38.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.497+0000 7f366e3cf700 1 -- 172.21.15.16:0/4107015603 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3668130540 msgr2=0x7f3668130910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:38.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.497+0000 7f366e3cf700 1 --2- 172.21.15.16:0/4107015603 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3668130540 0x7f3668130910 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f365800bb10 tx=0x7f365800be20 comp rx=0 tx=0).stop 2023-09-30T22:30:38.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.498+0000 7f366e3cf700 1 -- 172.21.15.16:0/4107015603 shutdown_connections 2023-09-30T22:30:38.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.498+0000 7f366e3cf700 1 --2- 172.21.15.16:0/4107015603 >> v2:172.21.15.16:3300/0 conn(0x7f366806fcf0 0x7f3668138570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.498+0000 7f366e3cf700 1 --2- 172.21.15.16:0/4107015603 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3668130540 0x7f3668130910 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.498+0000 7f366e3cf700 1 -- 172.21.15.16:0/4107015603 >> 172.21.15.16:0/4107015603 conn(0x7f36680a2aa0 msgr2=0x7f36680a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:38.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.499+0000 7f366e3cf700 1 -- 172.21.15.16:0/4107015603 shutdown_connections 2023-09-30T22:30:38.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.499+0000 7f366e3cf700 1 -- 172.21.15.16:0/4107015603 wait complete. 2023-09-30T22:30:38.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.500+0000 7f366e3cf700 1 Processor -- start 2023-09-30T22:30:38.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.501+0000 7f366e3cf700 1 -- start start 2023-09-30T22:30:38.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.501+0000 7f366e3cf700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f366806fcf0 0x7f366812ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:38.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.501+0000 7f366e3cf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3668130540 0x7f366812b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:38.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.501+0000 7f366e3cf700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3668130e50 0x7f36681c7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:38.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.501+0000 7f366e3cf700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3668071950 con 0x7f3668130540 2023-09-30T22:30:38.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.501+0000 7f366e3cf700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36681381c0 con 0x7f366806fcf0 2023-09-30T22:30:38.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.501+0000 7f366e3cf700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f366812d810 con 0x7f3668130e50 2023-09-30T22:30:38.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.502+0000 7f366c96c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3668130e50 0x7f36681c7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:38.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.502+0000 7f366c96c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3668130e50 0x7f36681c7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:43542/0 (socket says 172.21.15.16:43542) 2023-09-30T22:30:38.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.502+0000 7f366c96c700 1 -- 172.21.15.16:0/3502038690 learned_addr learned my addr 172.21.15.16:0/3502038690 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:38.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.502+0000 7f36677fe700 1 --2- 172.21.15.16:0/3502038690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3668130540 0x7f366812b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:38.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.502+0000 7f366c96c700 1 -- 172.21.15.16:0/3502038690 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f366806fcf0 msgr2=0x7f366812ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:38.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.502+0000 7f366c96c700 1 --2- 172.21.15.16:0/3502038690 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f366806fcf0 0x7f366812ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.502+0000 7f366c96c700 1 -- 172.21.15.16:0/3502038690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3668130540 msgr2=0x7f366812b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:38.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.502+0000 7f366c96c700 1 --2- 172.21.15.16:0/3502038690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3668130540 0x7f366812b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.502+0000 7f366c96c700 1 -- 172.21.15.16:0/3502038690 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f365800b740 con 0x7f3668130e50 2023-09-30T22:30:38.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.502+0000 7f366c96c700 1 --2- 172.21.15.16:0/3502038690 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3668130e50 0x7f36681c7e10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f366000efc0 tx=0x7f366000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:38.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.503+0000 7f3664ff9700 1 -- 172.21.15.16:0/3502038690 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3660009ea0 con 0x7f3668130e50 2023-09-30T22:30:38.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.503+0000 7f3664ff9700 1 -- 172.21.15.16:0/3502038690 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f366000f040 con 0x7f3668130e50 2023-09-30T22:30:38.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.503+0000 7f366e3cf700 1 -- 172.21.15.16:0/3502038690 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f366812dad0 con 0x7f3668130e50 2023-09-30T22:30:38.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.503+0000 7f3664ff9700 1 -- 172.21.15.16:0/3502038690 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3660014480 con 0x7f3668130e50 2023-09-30T22:30:38.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.503+0000 7f366e3cf700 1 -- 172.21.15.16:0/3502038690 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f366812bfa0 con 0x7f3668130e50 2023-09-30T22:30:38.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.505+0000 7f366e3cf700 1 -- 172.21.15.16:0/3502038690 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3668074ea0 con 0x7f3668130e50 2023-09-30T22:30:38.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.505+0000 7f3664ff9700 1 -- 172.21.15.16:0/3502038690 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f36600145e0 con 0x7f3668130e50 2023-09-30T22:30:38.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.506+0000 7f3664ff9700 1 -- 172.21.15.16:0/3502038690 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f3660040850 con 0x7f3668130e50 2023-09-30T22:30:38.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.510+0000 7f3664ff9700 1 -- 172.21.15.16:0/3502038690 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3660012930 con 0x7f3668130e50 2023-09-30T22:30:38.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.534+0000 7fa229af2700 1 Processor -- start 2023-09-30T22:30:38.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.534+0000 7fa229af2700 1 -- start start 2023-09-30T22:30:38.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.535+0000 7fa229af2700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa22409e930 0x7fa22409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:38.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.535+0000 7fa229af2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa22409f240 0x7fa2241317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:38.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.535+0000 7fa229af2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa22409f650 0x7fa224131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:38.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.535+0000 7fa229af2700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa224070940 con 0x7fa22409e930 2023-09-30T22:30:38.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.535+0000 7fa229af2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa22406fd60 con 0x7fa22409f650 2023-09-30T22:30:38.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.535+0000 7fa229af2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa22406fa80 con 0x7fa22409f240 2023-09-30T22:30:38.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.535+0000 7fa2237fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa22409e930 0x7fa22409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:38.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.535+0000 7fa223fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa22409f650 0x7fa224131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:38.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.535+0000 7fa2237fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa22409e930 0x7fa22409ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51828/0 (socket says 172.21.15.16:51828) 2023-09-30T22:30:38.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.535+0000 7fa222ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa22409f240 0x7fa2241317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:38.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.535+0000 7fa223fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa22409f650 0x7fa224131d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36750/0 (socket says 172.21.15.16:36750) 2023-09-30T22:30:38.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.535+0000 7fa2237fe700 1 -- 172.21.15.16:0/4136061048 learned_addr learned my addr 172.21.15.16:0/4136061048 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:38.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.536+0000 7fa2237fe700 1 -- 172.21.15.16:0/4136061048 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa22409f240 msgr2=0x7fa2241317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:38.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.536+0000 7fa2237fe700 1 --2- 172.21.15.16:0/4136061048 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa22409f240 0x7fa2241317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.536+0000 7fa2237fe700 1 -- 172.21.15.16:0/4136061048 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa22409f650 msgr2=0x7fa224131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:38.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.536+0000 7fa2237fe700 1 --2- 172.21.15.16:0/4136061048 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa22409f650 0x7fa224131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.536+0000 7fa2237fe700 1 -- 172.21.15.16:0/4136061048 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa22409fb60 con 0x7fa22409e930 2023-09-30T22:30:38.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.536+0000 7fa222ffd700 1 --2- 172.21.15.16:0/4136061048 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa22409f240 0x7fa2241317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:38.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.536+0000 7fa2237fe700 1 --2- 172.21.15.16:0/4136061048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa22409e930 0x7fa22409ed00 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa2140098f0 tx=0x7fa214009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4166a120ec1f7e93 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:38.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.536+0000 7fa2227fc700 1 -- 172.21.15.16:0/4136061048 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa214011830 con 0x7fa22409e930 2023-09-30T22:30:38.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.536+0000 7fa2227fc700 1 -- 172.21.15.16:0/4136061048 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa214011990 con 0x7fa22409e930 2023-09-30T22:30:38.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.536+0000 7fa2227fc700 1 -- 172.21.15.16:0/4136061048 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa214011d20 con 0x7fa22409e930 2023-09-30T22:30:38.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.537+0000 7fa229af2700 1 -- 172.21.15.16:0/4136061048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa22409e930 msgr2=0x7fa22409ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:38.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.537+0000 7fa229af2700 1 --2- 172.21.15.16:0/4136061048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa22409e930 0x7fa22409ed00 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa2140098f0 tx=0x7fa214009c00 comp rx=0 tx=0).stop 2023-09-30T22:30:38.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.537+0000 7fa229af2700 1 -- 172.21.15.16:0/4136061048 shutdown_connections 2023-09-30T22:30:38.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.537+0000 7fa229af2700 1 --2- 172.21.15.16:0/4136061048 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa22409f650 0x7fa224131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.537+0000 7fa229af2700 1 --2- 172.21.15.16:0/4136061048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa22409e930 0x7fa22409ed00 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.537+0000 7fa229af2700 1 --2- 172.21.15.16:0/4136061048 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa22409f240 0x7fa2241317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.537+0000 7fa229af2700 1 -- 172.21.15.16:0/4136061048 >> 172.21.15.16:0/4136061048 conn(0x7fa2240a4210 msgr2=0x7fa2240a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:38.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.537+0000 7fa229af2700 1 -- 172.21.15.16:0/4136061048 shutdown_connections 2023-09-30T22:30:38.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.537+0000 7fa229af2700 1 -- 172.21.15.16:0/4136061048 wait complete. 2023-09-30T22:30:38.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.538+0000 7fa229af2700 1 Processor -- start 2023-09-30T22:30:38.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.538+0000 7fa229af2700 1 -- start start 2023-09-30T22:30:38.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.539+0000 7fa229af2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa22409e930 0x7fa22406b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:38.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.539+0000 7fa229af2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa22409f240 0x7fa22406bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:38.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.539+0000 7fa229af2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa22409f650 0x7fa22406c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:38.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.539+0000 7fa229af2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2241a8930 con 0x7fa22409f240 2023-09-30T22:30:38.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.539+0000 7fa229af2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2241a9970 con 0x7fa22409f650 2023-09-30T22:30:38.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.539+0000 7fa229af2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa2241a9460 con 0x7fa22409e930 2023-09-30T22:30:38.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.539+0000 7fa2237fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa22409e930 0x7fa22406b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:38.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.539+0000 7fa2237fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa22409e930 0x7fa22406b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:43568/0 (socket says 172.21.15.16:43568) 2023-09-30T22:30:38.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.539+0000 7fa2237fe700 1 -- 172.21.15.16:0/1128611814 learned_addr learned my addr 172.21.15.16:0/1128611814 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:38.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.539+0000 7fa223fff700 1 --2- 172.21.15.16:0/1128611814 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa22409f650 0x7fa22406c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:38.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.539+0000 7fa2237fe700 1 -- 172.21.15.16:0/1128611814 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa22409f650 msgr2=0x7fa22406c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:38.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.539+0000 7fa2237fe700 1 --2- 172.21.15.16:0/1128611814 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa22409f650 0x7fa22406c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.539+0000 7fa2237fe700 1 -- 172.21.15.16:0/1128611814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa22409f240 msgr2=0x7fa22406bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:38.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.539+0000 7fa2237fe700 1 --2- 172.21.15.16:0/1128611814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa22409f240 0x7fa22406bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.539+0000 7fa2237fe700 1 -- 172.21.15.16:0/1128611814 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2180094a0 con 0x7fa22409e930 2023-09-30T22:30:38.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.539+0000 7fa223fff700 1 --2- 172.21.15.16:0/1128611814 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa22409f650 0x7fa22406c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:30:38.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.540+0000 7fa2237fe700 1 --2- 172.21.15.16:0/1128611814 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa22409e930 0x7fa22406b8b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa214000c00 tx=0x7fa214004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:38.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.540+0000 7fa20bfff700 1 -- 172.21.15.16:0/1128611814 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa214004300 con 0x7fa22409e930 2023-09-30T22:30:38.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.540+0000 7fa20bfff700 1 -- 172.21.15.16:0/1128611814 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa21401f070 con 0x7fa22409e930 2023-09-30T22:30:38.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.540+0000 7fa20bfff700 1 -- 172.21.15.16:0/1128611814 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2140036a0 con 0x7fa22409e930 2023-09-30T22:30:38.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.541+0000 7fa229af2700 1 -- 172.21.15.16:0/1128611814 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa214009550 con 0x7fa22409e930 2023-09-30T22:30:38.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.541+0000 7fa229af2700 1 -- 172.21.15.16:0/1128611814 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2240a0190 con 0x7fa22409e930 2023-09-30T22:30:38.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.541+0000 7fa20bfff700 1 -- 172.21.15.16:0/1128611814 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa214021030 con 0x7fa22409e930 2023-09-30T22:30:38.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.542+0000 7fa20bfff700 1 -- 172.21.15.16:0/1128611814 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fa214040950 con 0x7fa22409e930 2023-09-30T22:30:38.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.542+0000 7fa229af2700 1 -- 172.21.15.16:0/1128611814 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2241a0b30 con 0x7fa22409e930 2023-09-30T22:30:38.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.546+0000 7fa20bfff700 1 -- 172.21.15.16:0/1128611814 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa214015070 con 0x7fa22409e930 2023-09-30T22:30:38.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.622+0000 7f651aea3700 1 Processor -- start 2023-09-30T22:30:38.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.622+0000 7f651aea3700 1 -- start start 2023-09-30T22:30:38.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.623+0000 7f651aea3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f651409d6f0 0x7f651409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:38.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.623+0000 7f651aea3700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f651409e000 0x7f65141339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:38.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.623+0000 7f651aea3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6514133f30 0x7f65141343b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:38.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.623+0000 7f651aea3700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f651409b0f0 con 0x7f651409e000 2023-09-30T22:30:38.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.623+0000 7f651aea3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f651409d2f0 con 0x7f651409d6f0 2023-09-30T22:30:38.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.623+0000 7f651aea3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f651409a810 con 0x7f6514133f30 2023-09-30T22:30:38.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.623+0000 7f6519440700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6514133f30 0x7f65141343b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:38.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.623+0000 7f6513fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f651409e000 0x7f65141339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:38.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.623+0000 7f6519440700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6514133f30 0x7f65141343b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:43574/0 (socket says 172.21.15.16:43574) 2023-09-30T22:30:38.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.623+0000 7f6513fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f651409e000 0x7f65141339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51848/0 (socket says 172.21.15.16:51848) 2023-09-30T22:30:38.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.623+0000 7f6519440700 1 -- 172.21.15.16:0/3684742304 learned_addr learned my addr 172.21.15.16:0/3684742304 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:38.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.623+0000 7f6518c3f700 1 --2- 172.21.15.16:0/3684742304 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f651409d6f0 0x7f651409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:38.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.623+0000 7f6518c3f700 1 -- 172.21.15.16:0/3684742304 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6514133f30 msgr2=0x7f65141343b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:38.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.623+0000 7f6518c3f700 1 --2- 172.21.15.16:0/3684742304 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6514133f30 0x7f65141343b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.623+0000 7f6518c3f700 1 -- 172.21.15.16:0/3684742304 >> v2:172.21.15.16:3300/0 conn(0x7f651409e000 msgr2=0x7f65141339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:38.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.623+0000 7f6518c3f700 1 --2- 172.21.15.16:0/3684742304 >> v2:172.21.15.16:3300/0 conn(0x7f651409e000 0x7f65141339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.623+0000 7f6518c3f700 1 -- 172.21.15.16:0/3684742304 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f651409a470 con 0x7f651409d6f0 2023-09-30T22:30:38.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.623+0000 7f6519440700 1 --2- 172.21.15.16:0/3684742304 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6514133f30 0x7f65141343b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:38.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.623+0000 7f6513fff700 1 --2- 172.21.15.16:0/3684742304 >> v2:172.21.15.16:3300/0 conn(0x7f651409e000 0x7f65141339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:38.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.624+0000 7f6518c3f700 1 --2- 172.21.15.16:0/3684742304 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f651409d6f0 0x7f651409dac0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6508009840 tx=0x7f6508009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e07740e6e8b26e82 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:38.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.624+0000 7f65137fe700 1 -- 172.21.15.16:0/3684742304 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6508011830 con 0x7f651409d6f0 2023-09-30T22:30:38.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.624+0000 7f65137fe700 1 -- 172.21.15.16:0/3684742304 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6508011990 con 0x7f651409d6f0 2023-09-30T22:30:38.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.624+0000 7f65137fe700 1 -- 172.21.15.16:0/3684742304 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6508011d20 con 0x7f651409d6f0 2023-09-30T22:30:38.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.625+0000 7f651aea3700 1 -- 172.21.15.16:0/3684742304 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f651409d6f0 msgr2=0x7f651409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:38.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.625+0000 7f651aea3700 1 --2- 172.21.15.16:0/3684742304 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f651409d6f0 0x7f651409dac0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6508009840 tx=0x7f6508009b50 comp rx=0 tx=0).stop 2023-09-30T22:30:38.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.625+0000 7f651aea3700 1 -- 172.21.15.16:0/3684742304 shutdown_connections 2023-09-30T22:30:38.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.625+0000 7f651aea3700 1 --2- 172.21.15.16:0/3684742304 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f651409d6f0 0x7f651409dac0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.625+0000 7f651aea3700 1 --2- 172.21.15.16:0/3684742304 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6514133f30 0x7f65141343b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.625+0000 7f651aea3700 1 --2- 172.21.15.16:0/3684742304 >> v2:172.21.15.16:3300/0 conn(0x7f651409e000 0x7f65141339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.625+0000 7f651aea3700 1 -- 172.21.15.16:0/3684742304 >> 172.21.15.16:0/3684742304 conn(0x7f6514066f50 msgr2=0x7f651409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:38.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.625+0000 7f651aea3700 1 -- 172.21.15.16:0/3684742304 shutdown_connections 2023-09-30T22:30:38.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.625+0000 7f651aea3700 1 -- 172.21.15.16:0/3684742304 wait complete. 2023-09-30T22:30:38.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.626+0000 7f651aea3700 1 Processor -- start 2023-09-30T22:30:38.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.626+0000 7f651aea3700 1 -- start start 2023-09-30T22:30:38.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.627+0000 7f651aea3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f651409d6f0 0x7f651411e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:38.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.627+0000 7f651aea3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f651409e000 0x7f651411eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:38.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.627+0000 7f651aea3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6514133f30 0x7f65141c7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:38.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.627+0000 7f651aea3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6514121060 con 0x7f651409e000 2023-09-30T22:30:38.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.627+0000 7f651aea3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6514121700 con 0x7f6514133f30 2023-09-30T22:30:38.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.627+0000 7f651aea3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f651411dfe0 con 0x7f651409d6f0 2023-09-30T22:30:38.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.627+0000 7f6518c3f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f651409d6f0 0x7f651411e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:38.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.627+0000 7f6518c3f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f651409d6f0 0x7f651411e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:43584/0 (socket says 172.21.15.16:43584) 2023-09-30T22:30:38.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.627+0000 7f6518c3f700 1 -- 172.21.15.16:0/1663755431 learned_addr learned my addr 172.21.15.16:0/1663755431 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:38.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.627+0000 7f6519440700 1 --2- 172.21.15.16:0/1663755431 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6514133f30 0x7f65141c7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:38.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.627+0000 7f6513fff700 1 --2- 172.21.15.16:0/1663755431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f651409e000 0x7f651411eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:38.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.627+0000 7f6518c3f700 1 -- 172.21.15.16:0/1663755431 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6514133f30 msgr2=0x7f65141c7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:38.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.627+0000 7f6518c3f700 1 --2- 172.21.15.16:0/1663755431 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6514133f30 0x7f65141c7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.627+0000 7f6518c3f700 1 -- 172.21.15.16:0/1663755431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f651409e000 msgr2=0x7f651411eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:38.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.627+0000 7f6518c3f700 1 --2- 172.21.15.16:0/1663755431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f651409e000 0x7f651411eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.627+0000 7f6518c3f700 1 -- 172.21.15.16:0/1663755431 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65000094a0 con 0x7f651409d6f0 2023-09-30T22:30:38.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.627+0000 7f6513fff700 1 --2- 172.21.15.16:0/1663755431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f651409e000 0x7f651411eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:30:38.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.627+0000 7f6519440700 1 --2- 172.21.15.16:0/1663755431 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6514133f30 0x7f65141c7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:38.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.628+0000 7f6518c3f700 1 --2- 172.21.15.16:0/1663755431 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f651409d6f0 0x7f651411e630 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f65080090b0 tx=0x7f6508004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:38.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.628+0000 7f65117fa700 1 -- 172.21.15.16:0/1663755431 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6508004300 con 0x7f651409d6f0 2023-09-30T22:30:38.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.628+0000 7f65117fa700 1 -- 172.21.15.16:0/1663755431 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f650801f070 con 0x7f651409d6f0 2023-09-30T22:30:38.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.628+0000 7f651aea3700 1 -- 172.21.15.16:0/1663755431 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f650c009b30 con 0x7f651409d6f0 2023-09-30T22:30:38.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.628+0000 7f651aea3700 1 -- 172.21.15.16:0/1663755431 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65080094a0 con 0x7f651409d6f0 2023-09-30T22:30:38.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.628+0000 7f65117fa700 1 -- 172.21.15.16:0/1663755431 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65080036a0 con 0x7f651409d6f0 2023-09-30T22:30:38.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.629+0000 7f651aea3700 1 -- 172.21.15.16:0/1663755431 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64f40052e0 con 0x7f651409d6f0 2023-09-30T22:30:38.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.629+0000 7f65117fa700 1 -- 172.21.15.16:0/1663755431 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6508021030 con 0x7f651409d6f0 2023-09-30T22:30:38.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.630+0000 7f65117fa700 1 -- 172.21.15.16:0/1663755431 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f65080406c0 con 0x7f651409d6f0 2023-09-30T22:30:38.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.639+0000 7f65117fa700 1 -- 172.21.15.16:0/1663755431 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6508006bd0 con 0x7f651409d6f0 2023-09-30T22:30:38.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.670+0000 7f048ff51700 1 -- 172.21.15.16:0/3831980914 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0488074ea0 con 0x7f04880a1340 2023-09-30T22:30:38.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.671+0000 7f04867fc700 1 -- 172.21.15.16:0/3831980914 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f047c02b090 con 0x7f04880a1340 2023-09-30T22:30:38.671 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:30:38.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.673+0000 7f048ff51700 1 -- 172.21.15.16:0/3831980914 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f04880a1340 msgr2=0x7f0488122e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:38.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.673+0000 7f048ff51700 1 --2- 172.21.15.16:0/3831980914 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f04880a1340 0x7f0488122e70 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f047c00e9e0 tx=0x7f047c00ecf0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.674+0000 7f048ff51700 1 -- 172.21.15.16:0/3831980914 shutdown_connections 2023-09-30T22:30:38.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.674+0000 7f048ff51700 1 --2- 172.21.15.16:0/3831980914 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0488133fa0 0x7f04881233b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.674+0000 7f048ff51700 1 --2- 172.21.15.16:0/3831980914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04880a0a30 0x7f0488122930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.674+0000 7f048ff51700 1 --2- 172.21.15.16:0/3831980914 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f04880a1340 0x7f0488122e70 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.674+0000 7f048ff51700 1 -- 172.21.15.16:0/3831980914 >> 172.21.15.16:0/3831980914 conn(0x7f04880a6550 msgr2=0x7f048811c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:38.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.674+0000 7f048ff51700 1 -- 172.21.15.16:0/3831980914 shutdown_connections 2023-09-30T22:30:38.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.674+0000 7f048ff51700 1 -- 172.21.15.16:0/3831980914 wait complete. 2023-09-30T22:30:38.688 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:30:38.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.756+0000 7fa229af2700 1 -- 172.21.15.16:0/1128611814 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa224071d20 con 0x7fa22409e930 2023-09-30T22:30:38.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.756+0000 7fa20bfff700 1 -- 172.21.15.16:0/1128611814 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa2140199a0 con 0x7fa22409e930 2023-09-30T22:30:38.757 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:30:38.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.759+0000 7fa229af2700 1 -- 172.21.15.16:0/1128611814 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa22409e930 msgr2=0x7fa22406b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:38.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.759+0000 7fa229af2700 1 --2- 172.21.15.16:0/1128611814 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa22409e930 0x7fa22406b8b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa214000c00 tx=0x7fa214004750 comp rx=0 tx=0).stop 2023-09-30T22:30:38.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.759+0000 7fa229af2700 1 -- 172.21.15.16:0/1128611814 shutdown_connections 2023-09-30T22:30:38.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.759+0000 7fa229af2700 1 --2- 172.21.15.16:0/1128611814 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa22409f650 0x7fa22406c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.759+0000 7fa229af2700 1 --2- 172.21.15.16:0/1128611814 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa22409e930 0x7fa22406b8b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.759+0000 7fa229af2700 1 --2- 172.21.15.16:0/1128611814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa22409f240 0x7fa22406bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.759+0000 7fa229af2700 1 -- 172.21.15.16:0/1128611814 >> 172.21.15.16:0/1128611814 conn(0x7fa2240a4210 msgr2=0x7fa2241b8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:38.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.759+0000 7fa229af2700 1 -- 172.21.15.16:0/1128611814 shutdown_connections 2023-09-30T22:30:38.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.759+0000 7fa229af2700 1 -- 172.21.15.16:0/1128611814 wait complete. 2023-09-30T22:30:38.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.772+0000 7f366e3cf700 1 -- 172.21.15.16:0/3502038690 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3668131890 con 0x7f3668130e50 2023-09-30T22:30:38.773 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:30:38.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.772+0000 7f3664ff9700 1 -- 172.21.15.16:0/3502038690 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f366002a030 con 0x7f3668130e50 2023-09-30T22:30:38.773 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:30:38.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.775+0000 7f366e3cf700 1 -- 172.21.15.16:0/3502038690 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3668130e50 msgr2=0x7f36681c7e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:38.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.775+0000 7f366e3cf700 1 --2- 172.21.15.16:0/3502038690 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3668130e50 0x7f36681c7e10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f366000efc0 tx=0x7f366000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.775+0000 7f366e3cf700 1 -- 172.21.15.16:0/3502038690 shutdown_connections 2023-09-30T22:30:38.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.775+0000 7f366e3cf700 1 --2- 172.21.15.16:0/3502038690 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f366806fcf0 0x7f366812ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.775+0000 7f366e3cf700 1 --2- 172.21.15.16:0/3502038690 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3668130e50 0x7f36681c7e10 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.775+0000 7f366e3cf700 1 --2- 172.21.15.16:0/3502038690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3668130540 0x7f366812b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.775+0000 7f366e3cf700 1 -- 172.21.15.16:0/3502038690 >> 172.21.15.16:0/3502038690 conn(0x7f36680a2aa0 msgr2=0x7f366806e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:38.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.775+0000 7f366e3cf700 1 -- 172.21.15.16:0/3502038690 shutdown_connections 2023-09-30T22:30:38.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.775+0000 7f366e3cf700 1 -- 172.21.15.16:0/3502038690 wait complete. 2023-09-30T22:30:38.789 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:30:38.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.849+0000 7f651aea3700 1 -- 172.21.15.16:0/1663755431 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f64f4005150 con 0x7f651409d6f0 2023-09-30T22:30:38.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.849+0000 7f65117fa700 1 -- 172.21.15.16:0/1663755431 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f65080233d0 con 0x7f651409d6f0 2023-09-30T22:30:38.850 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:30:38.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.851+0000 7f651aea3700 1 -- 172.21.15.16:0/1663755431 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f651409d6f0 msgr2=0x7f651411e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:38.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.851+0000 7f651aea3700 1 --2- 172.21.15.16:0/1663755431 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f651409d6f0 0x7f651411e630 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f65080090b0 tx=0x7f6508004750 comp rx=0 tx=0).stop 2023-09-30T22:30:38.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.851+0000 7f651aea3700 1 -- 172.21.15.16:0/1663755431 shutdown_connections 2023-09-30T22:30:38.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.851+0000 7f651aea3700 1 --2- 172.21.15.16:0/1663755431 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6514133f30 0x7f65141c7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.851+0000 7f651aea3700 1 --2- 172.21.15.16:0/1663755431 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f651409d6f0 0x7f651411e630 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.852+0000 7f651aea3700 1 --2- 172.21.15.16:0/1663755431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f651409e000 0x7f651411eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:38.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.852+0000 7f651aea3700 1 -- 172.21.15.16:0/1663755431 >> 172.21.15.16:0/1663755431 conn(0x7f6514066f50 msgr2=0x7f6514132ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:38.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.852+0000 7f651aea3700 1 -- 172.21.15.16:0/1663755431 shutdown_connections 2023-09-30T22:30:38.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:38.852+0000 7f651aea3700 1 -- 172.21.15.16:0/1663755431 wait complete. 2023-09-30T22:30:38.865 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:30:39.689 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:39.774 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:39.790 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:39.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.845+0000 7f8fee8fb700 1 Processor -- start 2023-09-30T22:30:39.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.845+0000 7f8fee8fb700 1 -- start start 2023-09-30T22:30:39.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.846+0000 7f8fee8fb700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8fe8130540 0x7f8fe8130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:39.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.846+0000 7f8fee8fb700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8fe8130e50 0x7f8fe806f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:39.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.846+0000 7f8fee8fb700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fe80716a0 con 0x7f8fe806fb20 2023-09-30T22:30:39.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.846+0000 7f8fee8fb700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe809a390 con 0x7f8fe8130540 2023-09-30T22:30:39.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.846+0000 7f8fee8fb700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe8071290 con 0x7f8fe8130e50 2023-09-30T22:30:39.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.846+0000 7f8fe77fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8fe8130e50 0x7f8fe806f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:39.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.846+0000 7f8fe7fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8fe8130540 0x7f8fe8130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:39.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.846+0000 7f8fe77fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8fe8130e50 0x7f8fe806f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:43588/0 (socket says 172.21.15.16:43588) 2023-09-30T22:30:39.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.846+0000 7f8fe7fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8fe8130540 0x7f8fe8130910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36786/0 (socket says 172.21.15.16:36786) 2023-09-30T22:30:39.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.846+0000 7f8fe77fe700 1 -- 172.21.15.16:0/1067564612 learned_addr learned my addr 172.21.15.16:0/1067564612 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:39.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.847+0000 7f8fe7fff700 1 -- 172.21.15.16:0/1067564612 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8fe8130e50 msgr2=0x7f8fe806f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:39.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.847+0000 7f8fe7fff700 1 --2- 172.21.15.16:0/1067564612 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8fe8130e50 0x7f8fe806f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:39.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.847+0000 7f8fe7fff700 1 -- 172.21.15.16:0/1067564612 >> v1:172.21.15.16:6789/0 conn(0x7f8fe806fb20 legacy=0x7f8fe8137560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:39.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.847+0000 7f8fe6ffd700 1 -- 172.21.15.16:0/1067564612 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 407240550 0 0) 0x7f8fe80716a0 con 0x7f8fe806fb20 2023-09-30T22:30:39.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.847+0000 7f8fe7fff700 1 -- 172.21.15.16:0/1067564612 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fe8130340 con 0x7f8fe8130540 2023-09-30T22:30:39.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.847+0000 7f8fe7fff700 1 --2- 172.21.15.16:0/1067564612 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8fe8130540 0x7f8fe8130910 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8fd800bb40 tx=0x7f8fd800be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b5c3cbd1bbee4ff server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:39.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.847+0000 7f8fe6ffd700 1 -- 172.21.15.16:0/1067564612 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd800dc30 con 0x7f8fe8130540 2023-09-30T22:30:39.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.847+0000 7f8fe6ffd700 1 -- 172.21.15.16:0/1067564612 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fd800dd90 con 0x7f8fe8130540 2023-09-30T22:30:39.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.847+0000 7f8fe6ffd700 1 -- 172.21.15.16:0/1067564612 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd8010690 con 0x7f8fe8130540 2023-09-30T22:30:39.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.848+0000 7f8fee8fb700 1 -- 172.21.15.16:0/1067564612 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8fe8130540 msgr2=0x7f8fe8130910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:39.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.848+0000 7f8fee8fb700 1 --2- 172.21.15.16:0/1067564612 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8fe8130540 0x7f8fe8130910 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f8fd800bb40 tx=0x7f8fd800be50 comp rx=0 tx=0).stop 2023-09-30T22:30:39.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.849+0000 7f8fee8fb700 1 -- 172.21.15.16:0/1067564612 shutdown_connections 2023-09-30T22:30:39.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.849+0000 7f8fee8fb700 1 --2- 172.21.15.16:0/1067564612 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8fe8130540 0x7f8fe8130910 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:39.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.849+0000 7f8fee8fb700 1 --2- 172.21.15.16:0/1067564612 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8fe8130e50 0x7f8fe806f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:39.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.849+0000 7f8fee8fb700 1 -- 172.21.15.16:0/1067564612 >> 172.21.15.16:0/1067564612 conn(0x7f8fe80a2aa0 msgr2=0x7f8fe80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:39.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.849+0000 7f8fee8fb700 1 -- 172.21.15.16:0/1067564612 shutdown_connections 2023-09-30T22:30:39.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.849+0000 7f8fee8fb700 1 -- 172.21.15.16:0/1067564612 wait complete. 2023-09-30T22:30:39.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.850+0000 7f8fee8fb700 1 Processor -- start 2023-09-30T22:30:39.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.851+0000 7f8fee8fb700 1 -- start start 2023-09-30T22:30:39.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.851+0000 7f8fee8fb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fe806fb20 0x7f8fe811a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:39.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.851+0000 7f8fee8fb700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8fe8130540 0x7f8fe811ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:39.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.851+0000 7f8fee8fb700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8fe8130e50 0x7f8fe811b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:39.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.851+0000 7f8fee8fb700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe811d570 con 0x7f8fe806fb20 2023-09-30T22:30:39.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.851+0000 7f8fee8fb700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe811bbe0 con 0x7f8fe8130e50 2023-09-30T22:30:39.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.851+0000 7f8fee8fb700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe811e1c0 con 0x7f8fe8130540 2023-09-30T22:30:39.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.851+0000 7f8fe7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fe806fb20 0x7f8fe811a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:39.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.851+0000 7f8fece98700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8fe8130e50 0x7f8fe811b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:39.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.851+0000 7f8fe7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fe806fb20 0x7f8fe811a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51874/0 (socket says 172.21.15.16:51874) 2023-09-30T22:30:39.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.852+0000 7f8fe7fff700 1 -- 172.21.15.16:0/1205741384 learned_addr learned my addr 172.21.15.16:0/1205741384 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:39.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.852+0000 7f8fece98700 1 -- 172.21.15.16:0/1205741384 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8fe8130540 msgr2=0x7f8fe811ac70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:30:39.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.852+0000 7f8fece98700 1 --2- 172.21.15.16:0/1205741384 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8fe8130540 0x7f8fe811ac70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:39.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.852+0000 7f8fece98700 1 -- 172.21.15.16:0/1205741384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fe806fb20 msgr2=0x7f8fe811a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:39.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.852+0000 7f8fece98700 1 --2- 172.21.15.16:0/1205741384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fe806fb20 0x7f8fe811a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:39.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.852+0000 7f8fece98700 1 -- 172.21.15.16:0/1205741384 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd800b740 con 0x7f8fe8130e50 2023-09-30T22:30:39.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.852+0000 7f8fe7fff700 1 --2- 172.21.15.16:0/1205741384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fe806fb20 0x7f8fe811a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:39.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.852+0000 7f8fece98700 1 --2- 172.21.15.16:0/1205741384 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8fe8130e50 0x7f8fe811b1b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0001d60 tx=0x7f8fe0002070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:39.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.852+0000 7f8fe4ff9700 1 -- 172.21.15.16:0/1205741384 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe0013070 con 0x7f8fe8130e50 2023-09-30T22:30:39.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.852+0000 7f8fe4ff9700 1 -- 172.21.15.16:0/1205741384 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe000e2e0 con 0x7f8fe8130e50 2023-09-30T22:30:39.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.852+0000 7f8fee8fb700 1 -- 172.21.15.16:0/1205741384 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fe811d0b0 con 0x7f8fe8130e50 2023-09-30T22:30:39.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.852+0000 7f8fe4ff9700 1 -- 172.21.15.16:0/1205741384 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe0003c00 con 0x7f8fe8130e50 2023-09-30T22:30:39.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.852+0000 7f8fee8fb700 1 -- 172.21.15.16:0/1205741384 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fe80718e0 con 0x7f8fe8130e50 2023-09-30T22:30:39.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.853+0000 7f8fe4ff9700 1 -- 172.21.15.16:0/1205741384 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8fe0010640 con 0x7f8fe8130e50 2023-09-30T22:30:39.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.854+0000 7f8fee8fb700 1 -- 172.21.15.16:0/1205741384 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fe809cd40 con 0x7f8fe8130e50 2023-09-30T22:30:39.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.854+0000 7f8fe4ff9700 1 -- 172.21.15.16:0/1205741384 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f8fe0043450 con 0x7f8fe8130e50 2023-09-30T22:30:39.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.863+0000 7f8fe4ff9700 1 -- 172.21.15.16:0/1205741384 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8fe0005070 con 0x7f8fe8130e50 2023-09-30T22:30:39.866 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:39.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.935+0000 7fc2a9259700 1 Processor -- start 2023-09-30T22:30:39.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.935+0000 7fc2a9259700 1 -- start start 2023-09-30T22:30:39.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.936+0000 7fc2a9259700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2a409d6f0 0x7fc2a409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:39.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.936+0000 7fc2a9259700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2a409e000 0x7fc2a41339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:39.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.936+0000 7fc2a9259700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2a409b120 con 0x7fc2a409e410 2023-09-30T22:30:39.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.936+0000 7fc2a9259700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a406b460 con 0x7fc2a409e000 2023-09-30T22:30:39.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.936+0000 7fc2a9259700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a409a7e0 con 0x7fc2a409d6f0 2023-09-30T22:30:39.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.936+0000 7fc2a359e700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc2a409e410 0x7fc2a4133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58266/0 (socket says 172.21.15.16:58266) 2023-09-30T22:30:39.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.936+0000 7fc2a359e700 1 -- 172.21.15.16:0/4189291199 learned_addr learned my addr 172.21.15.16:0/4189291199 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:30:39.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.936+0000 7fc2a2d9d700 1 --2- 172.21.15.16:0/4189291199 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2a409d6f0 0x7fc2a409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:39.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.936+0000 7fc29bfff700 1 --2- 172.21.15.16:0/4189291199 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2a409e000 0x7fc2a41339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:39.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.937+0000 7fc2a259c700 1 -- 172.21.15.16:0/4189291199 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1026869031 0 0) 0x7fc2a409b120 con 0x7fc2a409e410 2023-09-30T22:30:39.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.937+0000 7fc2a259c700 1 -- 172.21.15.16:0/4189291199 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2900035b0 con 0x7fc2a409e410 2023-09-30T22:30:39.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.937+0000 7fc29bfff700 1 -- 172.21.15.16:0/4189291199 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2a409d6f0 msgr2=0x7fc2a409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:39.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.937+0000 7fc29bfff700 1 --2- 172.21.15.16:0/4189291199 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2a409d6f0 0x7fc2a409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:39.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.937+0000 7fc29bfff700 1 -- 172.21.15.16:0/4189291199 >> v1:172.21.15.16:6789/0 conn(0x7fc2a409e410 legacy=0x7fc2a4133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:39.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.937+0000 7fc29bfff700 1 -- 172.21.15.16:0/4189291199 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a409a500 con 0x7fc2a409e000 2023-09-30T22:30:39.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.937+0000 7fc2a2d9d700 1 --2- 172.21.15.16:0/4189291199 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2a409d6f0 0x7fc2a409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:30:39.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.937+0000 7fc29bfff700 1 --2- 172.21.15.16:0/4189291199 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2a409e000 0x7fc2a41339f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc28c009870 tx=0x7fc28c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91fb2d793ae2b2e8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:39.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.937+0000 7fc2a259c700 1 -- 172.21.15.16:0/4189291199 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28c011830 con 0x7fc2a409e000 2023-09-30T22:30:39.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.937+0000 7fc2a259c700 1 -- 172.21.15.16:0/4189291199 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc28c011990 con 0x7fc2a409e000 2023-09-30T22:30:39.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.937+0000 7fc2a259c700 1 -- 172.21.15.16:0/4189291199 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc28c011d20 con 0x7fc2a409e000 2023-09-30T22:30:39.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.938+0000 7fc2a9259700 1 -- 172.21.15.16:0/4189291199 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2a409e000 msgr2=0x7fc2a41339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:39.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.938+0000 7fc2a9259700 1 --2- 172.21.15.16:0/4189291199 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2a409e000 0x7fc2a41339f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc28c009870 tx=0x7fc28c009b80 comp rx=0 tx=0).stop 2023-09-30T22:30:39.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.938+0000 7fc2a9259700 1 -- 172.21.15.16:0/4189291199 shutdown_connections 2023-09-30T22:30:39.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.938+0000 7fc2a9259700 1 --2- 172.21.15.16:0/4189291199 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2a409d6f0 0x7fc2a409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:39.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.938+0000 7fc2a9259700 1 --2- 172.21.15.16:0/4189291199 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2a409e000 0x7fc2a41339f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:39.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.938+0000 7fc2a9259700 1 -- 172.21.15.16:0/4189291199 >> 172.21.15.16:0/4189291199 conn(0x7fc2a4066f50 msgr2=0x7fc2a409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:39.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.938+0000 7fc2a9259700 1 -- 172.21.15.16:0/4189291199 shutdown_connections 2023-09-30T22:30:39.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.938+0000 7fc2a9259700 1 -- 172.21.15.16:0/4189291199 wait complete. 2023-09-30T22:30:39.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.940+0000 7fc2a9259700 1 Processor -- start 2023-09-30T22:30:39.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.940+0000 7fc2a9259700 1 -- start start 2023-09-30T22:30:39.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.941+0000 7fc2a9259700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2a409d6f0 0x7fc2a41223a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:39.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.941+0000 7fc2a9259700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2a409e000 0x7fc2a41228e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:39.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.941+0000 7fc2a9259700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2a409e410 0x7fc2a4122e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:39.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.941+0000 7fc2a9259700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a4124de0 con 0x7fc2a409e000 2023-09-30T22:30:39.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.941+0000 7fc2a9259700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a4125e20 con 0x7fc2a409e410 2023-09-30T22:30:39.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.941+0000 7fc2a9259700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a4125910 con 0x7fc2a409d6f0 2023-09-30T22:30:39.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.941+0000 7fc2a2d9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2a409d6f0 0x7fc2a41223a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:39.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.941+0000 7fc2a2d9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2a409d6f0 0x7fc2a41223a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:43610/0 (socket says 172.21.15.16:43610) 2023-09-30T22:30:39.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.941+0000 7fc2a2d9d700 1 -- 172.21.15.16:0/4128314189 learned_addr learned my addr 172.21.15.16:0/4128314189 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:39.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.941+0000 7fc29bfff700 1 --2- 172.21.15.16:0/4128314189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2a409e000 0x7fc2a41228e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:39.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.941+0000 7fc2a359e700 1 --2- 172.21.15.16:0/4128314189 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2a409e410 0x7fc2a4122e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:39.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.942+0000 7fc2a2d9d700 1 -- 172.21.15.16:0/4128314189 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2a409e410 msgr2=0x7fc2a4122e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:39.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.942+0000 7fc2a2d9d700 1 --2- 172.21.15.16:0/4128314189 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2a409e410 0x7fc2a4122e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:39.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.942+0000 7fc2a2d9d700 1 -- 172.21.15.16:0/4128314189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2a409e000 msgr2=0x7fc2a41228e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:39.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.942+0000 7fc2a2d9d700 1 --2- 172.21.15.16:0/4128314189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2a409e000 0x7fc2a41228e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:39.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.942+0000 7fc2a2d9d700 1 -- 172.21.15.16:0/4128314189 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc28c0094a0 con 0x7fc2a409d6f0 2023-09-30T22:30:39.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.942+0000 7fc2a2d9d700 1 --2- 172.21.15.16:0/4128314189 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2a409d6f0 0x7fc2a41223a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc29400e810 tx=0x7fc29400eb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:39.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.942+0000 7fc29b7fe700 1 -- 172.21.15.16:0/4128314189 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc294013610 con 0x7fc2a409d6f0 2023-09-30T22:30:39.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.942+0000 7fc29b7fe700 1 -- 172.21.15.16:0/4128314189 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc294013770 con 0x7fc2a409d6f0 2023-09-30T22:30:39.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.942+0000 7fc29b7fe700 1 -- 172.21.15.16:0/4128314189 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc294013b20 con 0x7fc2a409d6f0 2023-09-30T22:30:39.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.942+0000 7fc2a9259700 1 -- 172.21.15.16:0/4128314189 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2a4123800 con 0x7fc2a409d6f0 2023-09-30T22:30:39.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.942+0000 7fc2a9259700 1 -- 172.21.15.16:0/4128314189 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2a409c710 con 0x7fc2a409d6f0 2023-09-30T22:30:39.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.944+0000 7fc29b7fe700 1 -- 172.21.15.16:0/4128314189 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc294013c80 con 0x7fc2a409d6f0 2023-09-30T22:30:39.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.944+0000 7fc2a9259700 1 -- 172.21.15.16:0/4128314189 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2a411f030 con 0x7fc2a409d6f0 2023-09-30T22:30:39.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.944+0000 7fc29b7fe700 1 -- 172.21.15.16:0/4128314189 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fc29403f320 con 0x7fc2a409d6f0 2023-09-30T22:30:39.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.953+0000 7fc29b7fe700 1 -- 172.21.15.16:0/4128314189 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc294012370 con 0x7fc2a409d6f0 2023-09-30T22:30:39.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.971+0000 7f2f433bd700 1 Processor -- start 2023-09-30T22:30:39.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.971+0000 7f2f433bd700 1 -- start start 2023-09-30T22:30:39.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.972+0000 7f2f433bd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2f3c133420 0x7f2f3c1337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:39.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.972+0000 7f2f433bd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f3c134260 0x7f2f3c1306a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:39.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.972+0000 7f2f433bd700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f3c06b990 con 0x7f2f3c133e30 2023-09-30T22:30:39.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.972+0000 7f2f433bd700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f3c0a0590 con 0x7f2f3c133420 2023-09-30T22:30:39.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.972+0000 7f2f433bd700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3c09fb80 con 0x7f2f3c134260 2023-09-30T22:30:39.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.972+0000 7f2f40958700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2f3c133e30 0x7f2f3c12ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58268/0 (socket says 172.21.15.16:58268) 2023-09-30T22:30:39.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.972+0000 7f2f4195a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f3c134260 0x7f2f3c1306a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:39.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.972+0000 7f2f40958700 1 -- 172.21.15.16:0/2227250355 learned_addr learned my addr 172.21.15.16:0/2227250355 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:30:39.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.972+0000 7f2f41159700 1 --2- 172.21.15.16:0/2227250355 >> v2:172.21.15.16:3300/0 conn(0x7f2f3c133420 0x7f2f3c1337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:39.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.973+0000 7f2f4195a700 1 -- 172.21.15.16:0/2227250355 >> v2:172.21.15.16:3300/0 conn(0x7f2f3c133420 msgr2=0x7f2f3c1337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:39.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.973+0000 7f2f4195a700 1 --2- 172.21.15.16:0/2227250355 >> v2:172.21.15.16:3300/0 conn(0x7f2f3c133420 0x7f2f3c1337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:39.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.973+0000 7f2f4195a700 1 -- 172.21.15.16:0/2227250355 >> v1:172.21.15.16:6789/0 conn(0x7f2f3c133e30 legacy=0x7f2f3c12ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:39.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.973+0000 7f2f4195a700 1 -- 172.21.15.16:0/2227250355 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f3c0a0030 con 0x7f2f3c134260 2023-09-30T22:30:39.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.973+0000 7f2f33fff700 1 -- 172.21.15.16:0/2227250355 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2471849418 0 0) 0x7f2f3c06b990 con 0x7f2f3c133e30 2023-09-30T22:30:39.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.973+0000 7f2f41159700 1 --2- 172.21.15.16:0/2227250355 >> v2:172.21.15.16:3300/0 conn(0x7f2f3c133420 0x7f2f3c1337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:39.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.973+0000 7f2f4195a700 1 --2- 172.21.15.16:0/2227250355 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f3c134260 0x7f2f3c1306a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2f34009d90 tx=0x7f2f3400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5720e0ef02a0493 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:39.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.973+0000 7f2f33fff700 1 -- 172.21.15.16:0/2227250355 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3400e070 con 0x7f2f3c134260 2023-09-30T22:30:39.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.973+0000 7f2f33fff700 1 -- 172.21.15.16:0/2227250355 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f3400bb20 con 0x7f2f3c134260 2023-09-30T22:30:39.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.973+0000 7f2f33fff700 1 -- 172.21.15.16:0/2227250355 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f34012430 con 0x7f2f3c134260 2023-09-30T22:30:39.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.974+0000 7f2f433bd700 1 -- 172.21.15.16:0/2227250355 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f3c134260 msgr2=0x7f2f3c1306a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:39.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.974+0000 7f2f433bd700 1 --2- 172.21.15.16:0/2227250355 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f3c134260 0x7f2f3c1306a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2f34009d90 tx=0x7f2f3400b330 comp rx=0 tx=0).stop 2023-09-30T22:30:39.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.974+0000 7f2f433bd700 1 -- 172.21.15.16:0/2227250355 shutdown_connections 2023-09-30T22:30:39.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.974+0000 7f2f433bd700 1 --2- 172.21.15.16:0/2227250355 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f3c134260 0x7f2f3c1306a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:39.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.974+0000 7f2f433bd700 1 --2- 172.21.15.16:0/2227250355 >> v2:172.21.15.16:3300/0 conn(0x7f2f3c133420 0x7f2f3c1337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:39.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.974+0000 7f2f433bd700 1 -- 172.21.15.16:0/2227250355 >> 172.21.15.16:0/2227250355 conn(0x7f2f3c0a4210 msgr2=0x7f2f3c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:39.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.975+0000 7f2f433bd700 1 -- 172.21.15.16:0/2227250355 shutdown_connections 2023-09-30T22:30:39.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.975+0000 7f2f433bd700 1 -- 172.21.15.16:0/2227250355 wait complete. 2023-09-30T22:30:39.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.976+0000 7f2f433bd700 1 Processor -- start 2023-09-30T22:30:39.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.976+0000 7f2f433bd700 1 -- start start 2023-09-30T22:30:39.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.977+0000 7f2f433bd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f3c133420 0x7f2f3c09c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:39.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.977+0000 7f2f433bd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f3c133e30 0x7f2f3c09c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:39.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.977+0000 7f2f433bd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f3c134260 0x7f2f3c09cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:39.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.977+0000 7f2f433bd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3c120600 con 0x7f2f3c134260 2023-09-30T22:30:39.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.977+0000 7f2f433bd700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3c11fb60 con 0x7f2f3c133e30 2023-09-30T22:30:39.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.977+0000 7f2f433bd700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3c131590 con 0x7f2f3c133420 2023-09-30T22:30:39.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.977+0000 7f2f4195a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f3c134260 0x7f2f3c09cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:39.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.977+0000 7f2f4195a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f3c134260 0x7f2f3c09cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51896/0 (socket says 172.21.15.16:51896) 2023-09-30T22:30:39.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.977+0000 7f2f41159700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f3c133420 0x7f2f3c09c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:39.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.977+0000 7f2f4195a700 1 -- 172.21.15.16:0/1273648293 learned_addr learned my addr 172.21.15.16:0/1273648293 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:39.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.977+0000 7f2f40958700 1 --2- 172.21.15.16:0/1273648293 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f3c133e30 0x7f2f3c09c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:39.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.977+0000 7f2f4195a700 1 -- 172.21.15.16:0/1273648293 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f3c133420 msgr2=0x7f2f3c09c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:39.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.977+0000 7f2f4195a700 1 --2- 172.21.15.16:0/1273648293 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f3c133420 0x7f2f3c09c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:39.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.977+0000 7f2f4195a700 1 -- 172.21.15.16:0/1273648293 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f3c133e30 msgr2=0x7f2f3c09c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:39.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.977+0000 7f2f4195a700 1 --2- 172.21.15.16:0/1273648293 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f3c133e30 0x7f2f3c09c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:39.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.977+0000 7f2f4195a700 1 -- 172.21.15.16:0/1273648293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f380094a0 con 0x7f2f3c134260 2023-09-30T22:30:39.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.977+0000 7f2f41159700 1 --2- 172.21.15.16:0/1273648293 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f3c133420 0x7f2f3c09c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:39.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.978+0000 7f2f4195a700 1 --2- 172.21.15.16:0/1273648293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f3c134260 0x7f2f3c09cf00 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2f34013000 tx=0x7f2f34004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:39.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.978+0000 7f2f31ffb700 1 -- 172.21.15.16:0/1273648293 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f34012430 con 0x7f2f3c134260 2023-09-30T22:30:39.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.978+0000 7f2f31ffb700 1 -- 172.21.15.16:0/1273648293 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f34020070 con 0x7f2f3c134260 2023-09-30T22:30:39.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.978+0000 7f2f433bd700 1 -- 172.21.15.16:0/1273648293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f34009a70 con 0x7f2f3c134260 2023-09-30T22:30:39.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.978+0000 7f2f31ffb700 1 -- 172.21.15.16:0/1273648293 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f34004040 con 0x7f2f3c134260 2023-09-30T22:30:39.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.978+0000 7f2f433bd700 1 -- 172.21.15.16:0/1273648293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f3c120170 con 0x7f2f3c134260 2023-09-30T22:30:39.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.980+0000 7f2f433bd700 1 -- 172.21.15.16:0/1273648293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f3c11fd80 con 0x7f2f3c134260 2023-09-30T22:30:39.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.981+0000 7f2f31ffb700 1 -- 172.21.15.16:0/1273648293 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2f340041a0 con 0x7f2f3c134260 2023-09-30T22:30:39.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.981+0000 7f2f31ffb700 1 -- 172.21.15.16:0/1273648293 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f2f34022070 con 0x7f2f3c134260 2023-09-30T22:30:39.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:39.985+0000 7f2f31ffb700 1 -- 172.21.15.16:0/1273648293 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2f34004450 con 0x7f2f3c134260 2023-09-30T22:30:40.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.018+0000 7fcf3b087700 1 Processor -- start 2023-09-30T22:30:40.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.018+0000 7fcf3b087700 1 -- start start 2023-09-30T22:30:40.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.018+0000 7fcf3b087700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcf3412cf60 0x7fcf3406f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:40.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.018+0000 7fcf3b087700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcf34130430 0x7fcf34137c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:40.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.018+0000 7fcf3b087700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf3412cd90 con 0x7fcf3412f8b0 2023-09-30T22:30:40.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.018+0000 7fcf3b087700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf3412d330 con 0x7fcf3412cf60 2023-09-30T22:30:40.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.018+0000 7fcf3b087700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf34071780 con 0x7fcf34130430 2023-09-30T22:30:40.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.018+0000 7fcf38e23700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcf3412cf60 0x7fcf3406f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:40.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.018+0000 7fcf38e23700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcf3412cf60 0x7fcf3406f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51900/0 (socket says 172.21.15.16:51900) 2023-09-30T22:30:40.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.018+0000 7fcf38e23700 1 -- 172.21.15.16:0/521522089 learned_addr learned my addr 172.21.15.16:0/521522089 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:40.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.018+0000 7fcf33fff700 1 --1- 172.21.15.16:0/521522089 >> v1:172.21.15.16:6789/0 conn(0x7fcf3412f8b0 0x7fcf3412fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58280/0 (socket says 172.21.15.16:58280) 2023-09-30T22:30:40.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.019+0000 7fcf39624700 1 --2- 172.21.15.16:0/521522089 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcf34130430 0x7fcf34137c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:40.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.019+0000 7fcf39624700 1 -- 172.21.15.16:0/521522089 >> v2:172.21.15.16:3300/0 conn(0x7fcf3412cf60 msgr2=0x7fcf3406f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:40.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.019+0000 7fcf39624700 1 --2- 172.21.15.16:0/521522089 >> v2:172.21.15.16:3300/0 conn(0x7fcf3412cf60 0x7fcf3406f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:40.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.019+0000 7fcf39624700 1 -- 172.21.15.16:0/521522089 >> v1:172.21.15.16:6789/0 conn(0x7fcf3412f8b0 legacy=0x7fcf3412fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:40.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.019+0000 7fcf39624700 1 -- 172.21.15.16:0/521522089 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf34071a40 con 0x7fcf34130430 2023-09-30T22:30:40.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.019+0000 7fcf38e23700 1 --2- 172.21.15.16:0/521522089 >> v2:172.21.15.16:3300/0 conn(0x7fcf3412cf60 0x7fcf3406f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:40.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.019+0000 7fcf39624700 1 --2- 172.21.15.16:0/521522089 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcf34130430 0x7fcf34137c30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c009d90 tx=0x7fcf2c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce10f499be14a4bb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:40.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.019+0000 7fcf337fe700 1 -- 172.21.15.16:0/521522089 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c00e070 con 0x7fcf34130430 2023-09-30T22:30:40.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.019+0000 7fcf337fe700 1 -- 172.21.15.16:0/521522089 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf2c00bb20 con 0x7fcf34130430 2023-09-30T22:30:40.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.019+0000 7fcf337fe700 1 -- 172.21.15.16:0/521522089 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c012430 con 0x7fcf34130430 2023-09-30T22:30:40.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.020+0000 7fcf3b087700 1 -- 172.21.15.16:0/521522089 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcf34130430 msgr2=0x7fcf34137c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:40.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.020+0000 7fcf3b087700 1 --2- 172.21.15.16:0/521522089 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcf34130430 0x7fcf34137c30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c009d90 tx=0x7fcf2c00b330 comp rx=0 tx=0).stop 2023-09-30T22:30:40.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.020+0000 7fcf3b087700 1 -- 172.21.15.16:0/521522089 shutdown_connections 2023-09-30T22:30:40.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.020+0000 7fcf3b087700 1 --2- 172.21.15.16:0/521522089 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcf34130430 0x7fcf34137c30 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:40.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.020+0000 7fcf3b087700 1 --2- 172.21.15.16:0/521522089 >> v2:172.21.15.16:3300/0 conn(0x7fcf3412cf60 0x7fcf3406f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:40.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.020+0000 7fcf3b087700 1 -- 172.21.15.16:0/521522089 >> 172.21.15.16:0/521522089 conn(0x7fcf340a2aa0 msgr2=0x7fcf340a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:40.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.020+0000 7fcf3b087700 1 -- 172.21.15.16:0/521522089 shutdown_connections 2023-09-30T22:30:40.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.020+0000 7fcf3b087700 1 -- 172.21.15.16:0/521522089 wait complete. 2023-09-30T22:30:40.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.021+0000 7fcf3b087700 1 Processor -- start 2023-09-30T22:30:40.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.022+0000 7fcf3b087700 1 -- start start 2023-09-30T22:30:40.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.022+0000 7fcf3b087700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcf3412cf60 0x7fcf3409b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:40.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.022+0000 7fcf3b087700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf3412f8b0 0x7fcf3409b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:40.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.022+0000 7fcf3b087700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcf34130430 0x7fcf3409bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:40.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.023+0000 7fcf3b087700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3409c790 con 0x7fcf3412f8b0 2023-09-30T22:30:40.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.023+0000 7fcf3b087700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3412d9d0 con 0x7fcf34130430 2023-09-30T22:30:40.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.023+0000 7fcf3b087700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcf34129fa0 con 0x7fcf3412cf60 2023-09-30T22:30:40.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.023+0000 7fcf33fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf3412f8b0 0x7fcf3409b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:40.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.023+0000 7fcf39624700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcf34130430 0x7fcf3409bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:40.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.023+0000 7fcf38e23700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcf3412cf60 0x7fcf3409b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:40.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.023+0000 7fcf39624700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcf34130430 0x7fcf3409bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36840/0 (socket says 172.21.15.16:36840) 2023-09-30T22:30:40.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.023+0000 7fcf39624700 1 -- 172.21.15.16:0/4107711162 learned_addr learned my addr 172.21.15.16:0/4107711162 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:40.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.023+0000 7fcf38e23700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcf3412cf60 0x7fcf3409b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:43640/0 (socket says 172.21.15.16:43640) 2023-09-30T22:30:40.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.023+0000 7fcf33fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf3412f8b0 0x7fcf3409b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51914/0 (socket says 172.21.15.16:51914) 2023-09-30T22:30:40.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.023+0000 7fcf39624700 1 -- 172.21.15.16:0/4107711162 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcf3412cf60 msgr2=0x7fcf3409b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:40.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.023+0000 7fcf39624700 1 --2- 172.21.15.16:0/4107711162 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcf3412cf60 0x7fcf3409b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:40.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.023+0000 7fcf39624700 1 -- 172.21.15.16:0/4107711162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf3412f8b0 msgr2=0x7fcf3409b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:40.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.023+0000 7fcf39624700 1 --2- 172.21.15.16:0/4107711162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf3412f8b0 0x7fcf3409b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:40.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.023+0000 7fcf39624700 1 -- 172.21.15.16:0/4107711162 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf2800b7f0 con 0x7fcf34130430 2023-09-30T22:30:40.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.023+0000 7fcf33fff700 1 --2- 172.21.15.16:0/4107711162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf3412f8b0 0x7fcf3409b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:40.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.023+0000 7fcf39624700 1 --2- 172.21.15.16:0/4107711162 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcf34130430 0x7fcf3409bdb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c014040 tx=0x7fcf2c004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:40.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.024+0000 7fcf317fa700 1 -- 172.21.15.16:0/4107711162 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c012430 con 0x7fcf34130430 2023-09-30T22:30:40.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.024+0000 7fcf317fa700 1 -- 172.21.15.16:0/4107711162 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf2c020070 con 0x7fcf34130430 2023-09-30T22:30:40.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.024+0000 7fcf3b087700 1 -- 172.21.15.16:0/4107711162 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf2c009a70 con 0x7fcf34130430 2023-09-30T22:30:40.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.024+0000 7fcf317fa700 1 -- 172.21.15.16:0/4107711162 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c003ea0 con 0x7fcf34130430 2023-09-30T22:30:40.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.024+0000 7fcf3b087700 1 -- 172.21.15.16:0/4107711162 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf3409eb70 con 0x7fcf34130430 2023-09-30T22:30:40.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.025+0000 7fcf317fa700 1 -- 172.21.15.16:0/4107711162 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcf2c004280 con 0x7fcf34130430 2023-09-30T22:30:40.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.025+0000 7fcf3b087700 1 -- 172.21.15.16:0/4107711162 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf3409e9b0 con 0x7fcf34130430 2023-09-30T22:30:40.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.025+0000 7fcf317fa700 1 -- 172.21.15.16:0/4107711162 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fcf2c022070 con 0x7fcf34130430 2023-09-30T22:30:40.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.030+0000 7fcf317fa700 1 -- 172.21.15.16:0/4107711162 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcf2c012e60 con 0x7fcf34130430 2023-09-30T22:30:40.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.084+0000 7f8fee8fb700 1 -- 172.21.15.16:0/1205741384 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f8fe8074ea0 con 0x7f8fe8130e50 2023-09-30T22:30:40.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.084+0000 7f8fe4ff9700 1 -- 172.21.15.16:0/1205741384 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8fe00213e0 con 0x7f8fe8130e50 2023-09-30T22:30:40.084 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:30:40.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.086+0000 7f8fee8fb700 1 -- 172.21.15.16:0/1205741384 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8fe8130e50 msgr2=0x7f8fe811b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:40.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.086+0000 7f8fee8fb700 1 --2- 172.21.15.16:0/1205741384 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8fe8130e50 0x7f8fe811b1b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0001d60 tx=0x7f8fe0002070 comp rx=0 tx=0).stop 2023-09-30T22:30:40.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.086+0000 7f8fee8fb700 1 -- 172.21.15.16:0/1205741384 shutdown_connections 2023-09-30T22:30:40.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.086+0000 7f8fee8fb700 1 --2- 172.21.15.16:0/1205741384 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8fe8130e50 0x7f8fe811b1b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:40.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.086+0000 7f8fee8fb700 1 --2- 172.21.15.16:0/1205741384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8fe806fb20 0x7f8fe811a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:40.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.086+0000 7f8fee8fb700 1 --2- 172.21.15.16:0/1205741384 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8fe8130540 0x7f8fe811ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:40.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.086+0000 7f8fee8fb700 1 -- 172.21.15.16:0/1205741384 >> 172.21.15.16:0/1205741384 conn(0x7f8fe80a2aa0 msgr2=0x7f8fe812d600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:40.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.087+0000 7f8fee8fb700 1 -- 172.21.15.16:0/1205741384 shutdown_connections 2023-09-30T22:30:40.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.087+0000 7f8fee8fb700 1 -- 172.21.15.16:0/1205741384 wait complete. 2023-09-30T22:30:40.102 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:30:40.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.209+0000 7fc2a9259700 1 -- 172.21.15.16:0/4128314189 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc2a4123f70 con 0x7fc2a409d6f0 2023-09-30T22:30:40.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.210+0000 7fc29b7fe700 1 -- 172.21.15.16:0/4128314189 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc294010310 con 0x7fc2a409d6f0 2023-09-30T22:30:40.210 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:30:40.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.212+0000 7fc2a9259700 1 -- 172.21.15.16:0/4128314189 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2a409d6f0 msgr2=0x7fc2a41223a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:40.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.212+0000 7fc2a9259700 1 --2- 172.21.15.16:0/4128314189 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2a409d6f0 0x7fc2a41223a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc29400e810 tx=0x7fc29400eb20 comp rx=0 tx=0).stop 2023-09-30T22:30:40.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.212+0000 7fc2a9259700 1 -- 172.21.15.16:0/4128314189 shutdown_connections 2023-09-30T22:30:40.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.212+0000 7fc2a9259700 1 --2- 172.21.15.16:0/4128314189 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2a409e410 0x7fc2a4122e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:40.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.212+0000 7fc2a9259700 1 --2- 172.21.15.16:0/4128314189 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2a409d6f0 0x7fc2a41223a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:40.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.212+0000 7fc2a9259700 1 --2- 172.21.15.16:0/4128314189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2a409e000 0x7fc2a41228e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:40.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.212+0000 7fc2a9259700 1 -- 172.21.15.16:0/4128314189 >> 172.21.15.16:0/4128314189 conn(0x7fc2a4066f50 msgr2=0x7fc2a4133e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:40.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.212+0000 7fc2a9259700 1 -- 172.21.15.16:0/4128314189 shutdown_connections 2023-09-30T22:30:40.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.212+0000 7fc2a9259700 1 -- 172.21.15.16:0/4128314189 wait complete. 2023-09-30T22:30:40.227 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:30:40.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.238+0000 7fcf3b087700 1 -- 172.21.15.16:0/4107711162 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fcf34074ea0 con 0x7fcf34130430 2023-09-30T22:30:40.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.238+0000 7fcf317fa700 1 -- 172.21.15.16:0/4107711162 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcf2c01a380 con 0x7fcf34130430 2023-09-30T22:30:40.239 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:30:40.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.240+0000 7fcf3b087700 1 -- 172.21.15.16:0/4107711162 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcf34130430 msgr2=0x7fcf3409bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:40.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.240+0000 7fcf3b087700 1 --2- 172.21.15.16:0/4107711162 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcf34130430 0x7fcf3409bdb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c014040 tx=0x7fcf2c004ad0 comp rx=0 tx=0).stop 2023-09-30T22:30:40.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.240+0000 7fcf3b087700 1 -- 172.21.15.16:0/4107711162 shutdown_connections 2023-09-30T22:30:40.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.240+0000 7fcf3b087700 1 --2- 172.21.15.16:0/4107711162 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcf34130430 0x7fcf3409bdb0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:40.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.240+0000 7fcf3b087700 1 --2- 172.21.15.16:0/4107711162 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcf3412cf60 0x7fcf3409b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:40.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.240+0000 7fcf3b087700 1 --2- 172.21.15.16:0/4107711162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf3412f8b0 0x7fcf3409b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:40.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.240+0000 7fcf3b087700 1 -- 172.21.15.16:0/4107711162 >> 172.21.15.16:0/4107711162 conn(0x7fcf340a2aa0 msgr2=0x7fcf3412a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:40.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.240+0000 7fcf3b087700 1 -- 172.21.15.16:0/4107711162 shutdown_connections 2023-09-30T22:30:40.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.240+0000 7fcf3b087700 1 -- 172.21.15.16:0/4107711162 wait complete. 2023-09-30T22:30:40.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.241+0000 7f2f433bd700 1 -- 172.21.15.16:0/1273648293 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f2f3c074ea0 con 0x7f2f3c134260 2023-09-30T22:30:40.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.241+0000 7f2f31ffb700 1 -- 172.21.15.16:0/1273648293 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2f3402c020 con 0x7f2f3c134260 2023-09-30T22:30:40.242 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:30:40.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.243+0000 7f2f433bd700 1 -- 172.21.15.16:0/1273648293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f3c134260 msgr2=0x7f2f3c09cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:40.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.243+0000 7f2f433bd700 1 --2- 172.21.15.16:0/1273648293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f3c134260 0x7f2f3c09cf00 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f2f34013000 tx=0x7f2f34004ad0 comp rx=0 tx=0).stop 2023-09-30T22:30:40.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.244+0000 7f2f433bd700 1 -- 172.21.15.16:0/1273648293 shutdown_connections 2023-09-30T22:30:40.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.244+0000 7f2f433bd700 1 --2- 172.21.15.16:0/1273648293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f3c134260 0x7f2f3c09cf00 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:40.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.244+0000 7f2f433bd700 1 --2- 172.21.15.16:0/1273648293 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f3c133420 0x7f2f3c09c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:40.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.244+0000 7f2f433bd700 1 --2- 172.21.15.16:0/1273648293 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f3c133e30 0x7f2f3c09c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:40.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.244+0000 7f2f433bd700 1 -- 172.21.15.16:0/1273648293 >> 172.21.15.16:0/1273648293 conn(0x7f2f3c0a4210 msgr2=0x7f2f3c130480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:40.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.244+0000 7f2f433bd700 1 -- 172.21.15.16:0/1273648293 shutdown_connections 2023-09-30T22:30:40.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:40.244+0000 7f2f433bd700 1 -- 172.21.15.16:0/1273648293 wait complete. 2023-09-30T22:30:40.255 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:30:40.258 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:30:41.103 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:41.228 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:41.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.248+0000 7f18db8a1700 1 Processor -- start 2023-09-30T22:30:41.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.249+0000 7f18db8a1700 1 -- start start 2023-09-30T22:30:41.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.249+0000 7f18db8a1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f18d4136820 0x7f18d4138c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:41.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.249+0000 7f18db8a1700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f18d4139140 0x7f18d413b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:41.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.249+0000 7f18db8a1700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18d406bc00 con 0x7f18d4133a80 2023-09-30T22:30:41.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.249+0000 7f18db8a1700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18d406c570 con 0x7f18d4139140 2023-09-30T22:30:41.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.249+0000 7f18db8a1700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18d40a0040 con 0x7f18d4136820 2023-09-30T22:30:41.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.249+0000 7f18d963d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f18d4133a80 0x7f18d4133e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58290/0 (socket says 172.21.15.16:58290) 2023-09-30T22:30:41.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.249+0000 7f18d963d700 1 -- 172.21.15.16:0/2228006526 learned_addr learned my addr 172.21.15.16:0/2228006526 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:30:41.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.249+0000 7f18d9e3e700 1 --2- 172.21.15.16:0/2228006526 >> v2:172.21.15.16:3300/0 conn(0x7f18d4139140 0x7f18d413b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:41.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.249+0000 7f18d8e3c700 1 --2- 172.21.15.16:0/2228006526 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f18d4136820 0x7f18d4138c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:41.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.250+0000 7f18d9e3e700 1 -- 172.21.15.16:0/2228006526 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f18d4136820 msgr2=0x7f18d4138c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:41.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.250+0000 7f18d9e3e700 1 --2- 172.21.15.16:0/2228006526 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f18d4136820 0x7f18d4138c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.250+0000 7f18d9e3e700 1 -- 172.21.15.16:0/2228006526 >> v1:172.21.15.16:6789/0 conn(0x7f18d4133a80 legacy=0x7f18d4133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:41.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.250+0000 7f18d9e3e700 1 -- 172.21.15.16:0/2228006526 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d4133860 con 0x7f18d4139140 2023-09-30T22:30:41.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.250+0000 7f18cbfff700 1 -- 172.21.15.16:0/2228006526 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 87628173 0 0) 0x7f18d406bc00 con 0x7f18d4133a80 2023-09-30T22:30:41.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.250+0000 7f18d8e3c700 1 --2- 172.21.15.16:0/2228006526 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f18d4136820 0x7f18d4138c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:41.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.250+0000 7f18d9e3e700 1 --2- 172.21.15.16:0/2228006526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18d4139140 0x7f18d413b520 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f18cc009840 tx=0x7f18cc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a85f0f5ad058798e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:41.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.250+0000 7f18cbfff700 1 -- 172.21.15.16:0/2228006526 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18cc011830 con 0x7f18d4139140 2023-09-30T22:30:41.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.250+0000 7f18cbfff700 1 -- 172.21.15.16:0/2228006526 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18cc011990 con 0x7f18d4139140 2023-09-30T22:30:41.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.250+0000 7f18cbfff700 1 -- 172.21.15.16:0/2228006526 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18cc011d20 con 0x7f18d4139140 2023-09-30T22:30:41.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.251+0000 7f18db8a1700 1 -- 172.21.15.16:0/2228006526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18d4139140 msgr2=0x7f18d413b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:41.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.251+0000 7f18db8a1700 1 --2- 172.21.15.16:0/2228006526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18d4139140 0x7f18d413b520 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f18cc009840 tx=0x7f18cc009b50 comp rx=0 tx=0).stop 2023-09-30T22:30:41.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.251+0000 7f18db8a1700 1 -- 172.21.15.16:0/2228006526 shutdown_connections 2023-09-30T22:30:41.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.251+0000 7f18db8a1700 1 --2- 172.21.15.16:0/2228006526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18d4139140 0x7f18d413b520 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.251+0000 7f18db8a1700 1 --2- 172.21.15.16:0/2228006526 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f18d4136820 0x7f18d4138c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.251+0000 7f18db8a1700 1 -- 172.21.15.16:0/2228006526 >> 172.21.15.16:0/2228006526 conn(0x7f18d40a4210 msgr2=0x7f18d40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:41.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.251+0000 7f18db8a1700 1 -- 172.21.15.16:0/2228006526 shutdown_connections 2023-09-30T22:30:41.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.251+0000 7f18db8a1700 1 -- 172.21.15.16:0/2228006526 wait complete. 2023-09-30T22:30:41.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.252+0000 7f18db8a1700 1 Processor -- start 2023-09-30T22:30:41.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.252+0000 7f18db8a1700 1 -- start start 2023-09-30T22:30:41.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.253+0000 7f18db8a1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18d4133a80 0x7f18d412de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:41.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.253+0000 7f18db8a1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f18d4136820 0x7f18d412e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:41.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.253+0000 7f18db8a1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f18d4139140 0x7f18d412e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:41.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.253+0000 7f18db8a1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d412f190 con 0x7f18d4133a80 2023-09-30T22:30:41.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.253+0000 7f18db8a1700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18d409e3a0 con 0x7f18d4136820 2023-09-30T22:30:41.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.253+0000 7f18db8a1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f18d409cf10 con 0x7f18d4139140 2023-09-30T22:30:41.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.253+0000 7f18d963d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18d4133a80 0x7f18d412de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:41.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.253+0000 7f18d8e3c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f18d4136820 0x7f18d412e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:41.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.253+0000 7f18d963d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18d4133a80 0x7f18d412de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51944/0 (socket says 172.21.15.16:51944) 2023-09-30T22:30:41.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.253+0000 7f18d963d700 1 -- 172.21.15.16:0/756756425 learned_addr learned my addr 172.21.15.16:0/756756425 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:41.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.253+0000 7f18d9e3e700 1 --2- 172.21.15.16:0/756756425 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f18d4139140 0x7f18d412e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:41.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.253+0000 7f18d963d700 1 -- 172.21.15.16:0/756756425 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f18d4139140 msgr2=0x7f18d412e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:41.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.253+0000 7f18d963d700 1 --2- 172.21.15.16:0/756756425 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f18d4139140 0x7f18d412e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.253+0000 7f18d963d700 1 -- 172.21.15.16:0/756756425 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f18d4136820 msgr2=0x7f18d412e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:41.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.253+0000 7f18d963d700 1 --2- 172.21.15.16:0/756756425 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f18d4136820 0x7f18d412e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.253+0000 7f18d963d700 1 -- 172.21.15.16:0/756756425 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c40094a0 con 0x7f18d4133a80 2023-09-30T22:30:41.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.253+0000 7f18d8e3c700 1 --2- 172.21.15.16:0/756756425 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f18d4136820 0x7f18d412e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:41.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.254+0000 7f18d963d700 1 --2- 172.21.15.16:0/756756425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18d4133a80 0x7f18d412de10 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f18d000cc80 tx=0x7f18d000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:41.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.254+0000 7f18c9ffb700 1 -- 172.21.15.16:0/756756425 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d000ab00 con 0x7f18d4133a80 2023-09-30T22:30:41.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.254+0000 7f18c9ffb700 1 -- 172.21.15.16:0/756756425 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18d000ac60 con 0x7f18d4133a80 2023-09-30T22:30:41.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.254+0000 7f18db8a1700 1 -- 172.21.15.16:0/756756425 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18cc0094a0 con 0x7f18d4133a80 2023-09-30T22:30:41.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.254+0000 7f18c9ffb700 1 -- 172.21.15.16:0/756756425 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d00070d0 con 0x7f18d4133a80 2023-09-30T22:30:41.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.254+0000 7f18db8a1700 1 -- 172.21.15.16:0/756756425 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18d40726f0 con 0x7f18d4133a80 2023-09-30T22:30:41.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.255+0000 7f18c9ffb700 1 -- 172.21.15.16:0/756756425 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f18d0007230 con 0x7f18d4133a80 2023-09-30T22:30:41.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.255+0000 7f18c9ffb700 1 -- 172.21.15.16:0/756756425 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f18d003f610 con 0x7f18d4133a80 2023-09-30T22:30:41.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.255+0000 7f18db8a1700 1 -- 172.21.15.16:0/756756425 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18d4074ea0 con 0x7f18d4133a80 2023-09-30T22:30:41.260 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:41.260 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:41.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.260+0000 7f18c9ffb700 1 -- 172.21.15.16:0/756756425 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f18d0018020 con 0x7f18d4133a80 2023-09-30T22:30:41.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.384+0000 7f74f8fee700 1 Processor -- start 2023-09-30T22:30:41.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.384+0000 7f74f8fee700 1 -- start start 2023-09-30T22:30:41.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.385+0000 7f74f8fee700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f74f409e1d0 0x7f74f4138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:41.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.385+0000 7f74f8fee700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f74f4139170 0x7f74f413b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:41.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.385+0000 7f74f8fee700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74f409b120 con 0x7f74f409d6f0 2023-09-30T22:30:41.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.385+0000 7f74f8fee700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74f406b460 con 0x7f74f409e1d0 2023-09-30T22:30:41.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.385+0000 7f74f8fee700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74f409a7e0 con 0x7f74f4139170 2023-09-30T22:30:41.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.385+0000 7f74f259c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f74f409d6f0 0x7f74f409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58300/0 (socket says 172.21.15.16:58300) 2023-09-30T22:30:41.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.385+0000 7f74f259c700 1 -- 172.21.15.16:0/2596946326 learned_addr learned my addr 172.21.15.16:0/2596946326 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:30:41.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.385+0000 7f74f2d9d700 1 --2- 172.21.15.16:0/2596946326 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f74f4139170 0x7f74f413b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:41.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.385+0000 7f74f1d9b700 1 --2- 172.21.15.16:0/2596946326 >> v2:172.21.15.16:3300/0 conn(0x7f74f409e1d0 0x7f74f4138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:41.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.386+0000 7f74f159a700 1 -- 172.21.15.16:0/2596946326 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2648457610 0 0) 0x7f74f409b120 con 0x7f74f409d6f0 2023-09-30T22:30:41.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.386+0000 7f74f159a700 1 -- 172.21.15.16:0/2596946326 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74dc0035b0 con 0x7f74f409d6f0 2023-09-30T22:30:41.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.386+0000 7f74f1d9b700 1 -- 172.21.15.16:0/2596946326 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f74f4139170 msgr2=0x7f74f413b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:41.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.386+0000 7f74f1d9b700 1 --2- 172.21.15.16:0/2596946326 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f74f4139170 0x7f74f413b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.386+0000 7f74f1d9b700 1 -- 172.21.15.16:0/2596946326 >> v1:172.21.15.16:6789/0 conn(0x7f74f409d6f0 legacy=0x7f74f409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:41.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.386+0000 7f74f1d9b700 1 -- 172.21.15.16:0/2596946326 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74f409a500 con 0x7f74f409e1d0 2023-09-30T22:30:41.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.386+0000 7f74f2d9d700 1 --2- 172.21.15.16:0/2596946326 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f74f4139170 0x7f74f413b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:41.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.386+0000 7f74f1d9b700 1 --2- 172.21.15.16:0/2596946326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74f409e1d0 0x7f74f4138c30 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f74e8013180 tx=0x7f74e8013490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdecbdbb0b889ab5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:41.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.386+0000 7f74f159a700 1 -- 172.21.15.16:0/2596946326 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e8016070 con 0x7f74f409e1d0 2023-09-30T22:30:41.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.386+0000 7f74f159a700 1 -- 172.21.15.16:0/2596946326 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74e8013d00 con 0x7f74f409e1d0 2023-09-30T22:30:41.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.386+0000 7f74f159a700 1 -- 172.21.15.16:0/2596946326 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e80112d0 con 0x7f74f409e1d0 2023-09-30T22:30:41.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.387+0000 7f74f8fee700 1 -- 172.21.15.16:0/2596946326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74f409e1d0 msgr2=0x7f74f4138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:41.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.387+0000 7f74f8fee700 1 --2- 172.21.15.16:0/2596946326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74f409e1d0 0x7f74f4138c30 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f74e8013180 tx=0x7f74e8013490 comp rx=0 tx=0).stop 2023-09-30T22:30:41.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.388+0000 7f74f8fee700 1 -- 172.21.15.16:0/2596946326 shutdown_connections 2023-09-30T22:30:41.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.388+0000 7f74f8fee700 1 --2- 172.21.15.16:0/2596946326 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f74f4139170 0x7f74f413b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.388+0000 7f74f8fee700 1 --2- 172.21.15.16:0/2596946326 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74f409e1d0 0x7f74f4138c30 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.388+0000 7f74f8fee700 1 -- 172.21.15.16:0/2596946326 >> 172.21.15.16:0/2596946326 conn(0x7f74f4066f50 msgr2=0x7f74f409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:41.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.388+0000 7f74f8fee700 1 -- 172.21.15.16:0/2596946326 shutdown_connections 2023-09-30T22:30:41.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.388+0000 7f74f8fee700 1 -- 172.21.15.16:0/2596946326 wait complete. 2023-09-30T22:30:41.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.389+0000 7f74f8fee700 1 Processor -- start 2023-09-30T22:30:41.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.389+0000 7f74f8fee700 1 -- start start 2023-09-30T22:30:41.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.390+0000 7f74f8fee700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f74f409d6f0 0x7f74f4119480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:41.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.390+0000 7f74f8fee700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74f409e1d0 0x7f74f41199c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:41.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.390+0000 7f74f8fee700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f74f4139170 0x7f74f4119f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:41.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.390+0000 7f74f8fee700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74f411ad20 con 0x7f74f409e1d0 2023-09-30T22:30:41.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.390+0000 7f74f8fee700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74f411ab30 con 0x7f74f409d6f0 2023-09-30T22:30:41.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.390+0000 7f74f8fee700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f74f40a17e0 con 0x7f74f4139170 2023-09-30T22:30:41.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.390+0000 7f74f1d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74f409e1d0 0x7f74f41199c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:41.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.390+0000 7f74f1d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74f409e1d0 0x7f74f41199c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51964/0 (socket says 172.21.15.16:51964) 2023-09-30T22:30:41.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.390+0000 7f74f1d9b700 1 -- 172.21.15.16:0/1818501751 learned_addr learned my addr 172.21.15.16:0/1818501751 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:41.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.390+0000 7f74f2d9d700 1 --2- 172.21.15.16:0/1818501751 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f74f4139170 0x7f74f4119f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:41.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.391+0000 7f74f259c700 1 --2- 172.21.15.16:0/1818501751 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f74f409d6f0 0x7f74f4119480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:41.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.391+0000 7f74f1d9b700 1 -- 172.21.15.16:0/1818501751 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f74f4139170 msgr2=0x7f74f4119f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:41.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.391+0000 7f74f1d9b700 1 --2- 172.21.15.16:0/1818501751 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f74f4139170 0x7f74f4119f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.391+0000 7f74f1d9b700 1 -- 172.21.15.16:0/1818501751 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f74f409d6f0 msgr2=0x7f74f4119480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:41.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.391+0000 7f74f1d9b700 1 --2- 172.21.15.16:0/1818501751 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f74f409d6f0 0x7f74f4119480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.391+0000 7f74f1d9b700 1 -- 172.21.15.16:0/1818501751 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74ec009a70 con 0x7f74f409e1d0 2023-09-30T22:30:41.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.391+0000 7f74f1d9b700 1 --2- 172.21.15.16:0/1818501751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74f409e1d0 0x7f74f41199c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f74e801a000 tx=0x7f74e8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:41.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.392+0000 7f74e2ffd700 1 -- 172.21.15.16:0/1818501751 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e800fc60 con 0x7f74f409e1d0 2023-09-30T22:30:41.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.392+0000 7f74e2ffd700 1 -- 172.21.15.16:0/1818501751 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74e800fdc0 con 0x7f74f409e1d0 2023-09-30T22:30:41.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.392+0000 7f74e2ffd700 1 -- 172.21.15.16:0/1818501751 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e80038a0 con 0x7f74f409e1d0 2023-09-30T22:30:41.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.392+0000 7f74f8fee700 1 -- 172.21.15.16:0/1818501751 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74e8011d40 con 0x7f74f409e1d0 2023-09-30T22:30:41.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.392+0000 7f74f8fee700 1 -- 172.21.15.16:0/1818501751 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74f411b930 con 0x7f74f409e1d0 2023-09-30T22:30:41.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.393+0000 7f74e2ffd700 1 -- 172.21.15.16:0/1818501751 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f74e8003a00 con 0x7f74f409e1d0 2023-09-30T22:30:41.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.393+0000 7f74e2ffd700 1 -- 172.21.15.16:0/1818501751 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f74e80493e0 con 0x7f74f409e1d0 2023-09-30T22:30:41.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.395+0000 7f74f8fee700 1 -- 172.21.15.16:0/1818501751 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74f4074ea0 con 0x7f74f409e1d0 2023-09-30T22:30:41.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.400+0000 7f74e2ffd700 1 -- 172.21.15.16:0/1818501751 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f74e8003cb0 con 0x7f74f409e1d0 2023-09-30T22:30:41.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.406+0000 7fd02041c700 1 Processor -- start 2023-09-30T22:30:41.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.406+0000 7fd02041c700 1 -- start start 2023-09-30T22:30:41.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.407+0000 7fd02041c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd01806b250 0x7fd018136840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:41.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.407+0000 7fd02041c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd018136d80 0x7fd018139160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:41.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.407+0000 7fd02041c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd01809ece0 con 0x7fd01806a770 2023-09-30T22:30:41.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.407+0000 7fd02041c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd01806b640 con 0x7fd01806b250 2023-09-30T22:30:41.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.407+0000 7fd02041c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd01809e760 con 0x7fd018136d80 2023-09-30T22:30:41.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.407+0000 7fd01e1b8700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd01806a770 0x7fd01806ab40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58302/0 (socket says 172.21.15.16:58302) 2023-09-30T22:30:41.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.407+0000 7fd01e1b8700 1 -- 172.21.15.16:0/4014028294 learned_addr learned my addr 172.21.15.16:0/4014028294 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:30:41.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.407+0000 7fd01e9b9700 1 --2- 172.21.15.16:0/4014028294 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd018136d80 0x7fd018139160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:41.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.407+0000 7fd01d9b7700 1 --2- 172.21.15.16:0/4014028294 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd01806b250 0x7fd018136840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:41.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.408+0000 7fd01d1b6700 1 -- 172.21.15.16:0/4014028294 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2432301828 0 0) 0x7fd01809ece0 con 0x7fd01806a770 2023-09-30T22:30:41.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.408+0000 7fd01d1b6700 1 -- 172.21.15.16:0/4014028294 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0040035b0 con 0x7fd01806a770 2023-09-30T22:30:41.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.408+0000 7fd01e9b9700 1 -- 172.21.15.16:0/4014028294 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd01806b250 msgr2=0x7fd018136840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:41.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.408+0000 7fd01e9b9700 1 --2- 172.21.15.16:0/4014028294 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd01806b250 0x7fd018136840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.408+0000 7fd01e9b9700 1 -- 172.21.15.16:0/4014028294 >> v1:172.21.15.16:6789/0 conn(0x7fd01806a770 legacy=0x7fd01806ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:41.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.408+0000 7fd01e9b9700 1 -- 172.21.15.16:0/4014028294 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd01809e040 con 0x7fd018136d80 2023-09-30T22:30:41.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.408+0000 7fd01e9b9700 1 --2- 172.21.15.16:0/4014028294 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd018136d80 0x7fd018139160 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd010009dc0 tx=0x7fd01000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8a7817fb7b4d74 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:41.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.408+0000 7fd01d1b6700 1 -- 172.21.15.16:0/4014028294 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd01000e070 con 0x7fd018136d80 2023-09-30T22:30:41.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.408+0000 7fd01d1b6700 1 -- 172.21.15.16:0/4014028294 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd01000bb20 con 0x7fd018136d80 2023-09-30T22:30:41.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.408+0000 7fd01d1b6700 1 -- 172.21.15.16:0/4014028294 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd010012430 con 0x7fd018136d80 2023-09-30T22:30:41.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.409+0000 7fd02041c700 1 -- 172.21.15.16:0/4014028294 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd018136d80 msgr2=0x7fd018139160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:41.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.409+0000 7fd02041c700 1 --2- 172.21.15.16:0/4014028294 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd018136d80 0x7fd018139160 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd010009dc0 tx=0x7fd01000b330 comp rx=0 tx=0).stop 2023-09-30T22:30:41.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.409+0000 7fd02041c700 1 -- 172.21.15.16:0/4014028294 shutdown_connections 2023-09-30T22:30:41.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.409+0000 7fd02041c700 1 --2- 172.21.15.16:0/4014028294 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd018136d80 0x7fd018139160 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.409+0000 7fd02041c700 1 --2- 172.21.15.16:0/4014028294 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd01806b250 0x7fd018136840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.409+0000 7fd02041c700 1 -- 172.21.15.16:0/4014028294 >> 172.21.15.16:0/4014028294 conn(0x7fd018066f50 msgr2=0x7fd01809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:41.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.409+0000 7fd02041c700 1 -- 172.21.15.16:0/4014028294 shutdown_connections 2023-09-30T22:30:41.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.409+0000 7fd02041c700 1 -- 172.21.15.16:0/4014028294 wait complete. 2023-09-30T22:30:41.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.410+0000 7fd02041c700 1 Processor -- start 2023-09-30T22:30:41.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.410+0000 7fd02041c700 1 -- start start 2023-09-30T22:30:41.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.411+0000 7fd02041c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd01806a770 0x7fd01809b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:41.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.411+0000 7fd02041c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd01806b250 0x7fd0181c8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:41.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.411+0000 7fd02041c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd018136d80 0x7fd0181c8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:41.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.411+0000 7fd02041c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd018121ec0 con 0x7fd018136d80 2023-09-30T22:30:41.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.411+0000 7fd02041c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd018120f90 con 0x7fd01806b250 2023-09-30T22:30:41.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.411+0000 7fd02041c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd018132230 con 0x7fd01806a770 2023-09-30T22:30:41.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.411+0000 7fd01e9b9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd018136d80 0x7fd0181c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:41.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.411+0000 7fd01e9b9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd018136d80 0x7fd0181c8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51966/0 (socket says 172.21.15.16:51966) 2023-09-30T22:30:41.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.411+0000 7fd01e9b9700 1 -- 172.21.15.16:0/2884085548 learned_addr learned my addr 172.21.15.16:0/2884085548 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:41.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.411+0000 7fd01d9b7700 1 --2- 172.21.15.16:0/2884085548 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd01806b250 0x7fd0181c8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:41.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.411+0000 7fd01e9b9700 1 -- 172.21.15.16:0/2884085548 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd01806a770 msgr2=0x7fd01809b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:41.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.411+0000 7fd01e1b8700 1 --2- 172.21.15.16:0/2884085548 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd01806a770 0x7fd01809b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:41.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.411+0000 7fd01e9b9700 1 --2- 172.21.15.16:0/2884085548 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd01806a770 0x7fd01809b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.411+0000 7fd01e9b9700 1 -- 172.21.15.16:0/2884085548 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd01806b250 msgr2=0x7fd0181c8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:41.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.411+0000 7fd01e9b9700 1 --2- 172.21.15.16:0/2884085548 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd01806b250 0x7fd0181c8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.411+0000 7fd01e9b9700 1 -- 172.21.15.16:0/2884085548 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd010009a70 con 0x7fd018136d80 2023-09-30T22:30:41.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.412+0000 7fd01e1b8700 1 --2- 172.21.15.16:0/2884085548 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd01806a770 0x7fd01809b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:30:41.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.412+0000 7fd01e9b9700 1 --2- 172.21.15.16:0/2884085548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd018136d80 0x7fd0181c8540 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd01000b370 tx=0x7fd010004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:41.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.412+0000 7fd00effd700 1 -- 172.21.15.16:0/2884085548 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0100048b0 con 0x7fd018136d80 2023-09-30T22:30:41.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.412+0000 7fd00effd700 1 -- 172.21.15.16:0/2884085548 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd010020070 con 0x7fd018136d80 2023-09-30T22:30:41.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.412+0000 7fd02041c700 1 -- 172.21.15.16:0/2884085548 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd018071e60 con 0x7fd018136d80 2023-09-30T22:30:41.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.412+0000 7fd00effd700 1 -- 172.21.15.16:0/2884085548 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0100276c0 con 0x7fd018136d80 2023-09-30T22:30:41.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.412+0000 7fd02041c700 1 -- 172.21.15.16:0/2884085548 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd018122930 con 0x7fd018136d80 2023-09-30T22:30:41.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.413+0000 7fd00effd700 1 -- 172.21.15.16:0/2884085548 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd010012430 con 0x7fd018136d80 2023-09-30T22:30:41.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.413+0000 7fd02041c700 1 -- 172.21.15.16:0/2884085548 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd018074ea0 con 0x7fd018136d80 2023-09-30T22:30:41.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.413+0000 7fd00effd700 1 -- 172.21.15.16:0/2884085548 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd010022070 con 0x7fd018136d80 2023-09-30T22:30:41.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.419+0000 7fd00effd700 1 -- 172.21.15.16:0/2884085548 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd010003e30 con 0x7fd018136d80 2023-09-30T22:30:41.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.420+0000 7fb7a803d700 1 Processor -- start 2023-09-30T22:30:41.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.420+0000 7fb7a803d700 1 -- start start 2023-09-30T22:30:41.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.421+0000 7fb7a803d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb7a006f5d0 0x7fb7a012ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:41.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.421+0000 7fb7a803d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb7a0135c20 0x7fb7a0138000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:41.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.421+0000 7fb7a803d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7a006fa40 con 0x7fb7a012cf60 2023-09-30T22:30:41.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.421+0000 7fb7a803d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7a012cd90 con 0x7fb7a0135c20 2023-09-30T22:30:41.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.421+0000 7fb7a803d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a0071780 con 0x7fb7a006f5d0 2023-09-30T22:30:41.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.421+0000 7fb7a5dd9700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb7a012cf60 0x7fb7a012f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58318/0 (socket says 172.21.15.16:58318) 2023-09-30T22:30:41.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.421+0000 7fb7a5dd9700 1 -- 172.21.15.16:0/3339146732 learned_addr learned my addr 172.21.15.16:0/3339146732 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:30:41.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.421+0000 7fb7a65da700 1 --2- 172.21.15.16:0/3339146732 >> v2:172.21.15.16:3300/0 conn(0x7fb7a0135c20 0x7fb7a0138000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:41.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.422+0000 7fb7a55d8700 1 --2- 172.21.15.16:0/3339146732 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb7a006f5d0 0x7fb7a012ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:41.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.422+0000 7fb7a4dd7700 1 -- 172.21.15.16:0/3339146732 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4111255451 0 0) 0x7fb7a006fa40 con 0x7fb7a012cf60 2023-09-30T22:30:41.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.422+0000 7fb7a4dd7700 1 -- 172.21.15.16:0/3339146732 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb78c0035b0 con 0x7fb7a012cf60 2023-09-30T22:30:41.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.422+0000 7fb7a65da700 1 -- 172.21.15.16:0/3339146732 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb7a006f5d0 msgr2=0x7fb7a012ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:41.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.422+0000 7fb7a65da700 1 --2- 172.21.15.16:0/3339146732 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb7a006f5d0 0x7fb7a012ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.422+0000 7fb7a65da700 1 -- 172.21.15.16:0/3339146732 >> v1:172.21.15.16:6789/0 conn(0x7fb7a012cf60 legacy=0x7fb7a012f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:41.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.422+0000 7fb7a65da700 1 -- 172.21.15.16:0/3339146732 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a0071a40 con 0x7fb7a0135c20 2023-09-30T22:30:41.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.422+0000 7fb7a65da700 1 --2- 172.21.15.16:0/3339146732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7a0135c20 0x7fb7a0138000 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb79800bfa0 tx=0x7fb79800d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a7dea5757551be server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:41.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.422+0000 7fb7a4dd7700 1 -- 172.21.15.16:0/3339146732 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79800dd50 con 0x7fb7a0135c20 2023-09-30T22:30:41.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.422+0000 7fb7a4dd7700 1 -- 172.21.15.16:0/3339146732 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb79800b020 con 0x7fb7a0135c20 2023-09-30T22:30:41.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.422+0000 7fb7a4dd7700 1 -- 172.21.15.16:0/3339146732 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79800b3d0 con 0x7fb7a0135c20 2023-09-30T22:30:41.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.423+0000 7fb7a803d700 1 -- 172.21.15.16:0/3339146732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7a0135c20 msgr2=0x7fb7a0138000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:41.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.423+0000 7fb7a803d700 1 --2- 172.21.15.16:0/3339146732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7a0135c20 0x7fb7a0138000 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb79800bfa0 tx=0x7fb79800d330 comp rx=0 tx=0).stop 2023-09-30T22:30:41.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.423+0000 7fb7a803d700 1 -- 172.21.15.16:0/3339146732 shutdown_connections 2023-09-30T22:30:41.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.423+0000 7fb7a803d700 1 --2- 172.21.15.16:0/3339146732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7a0135c20 0x7fb7a0138000 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.423+0000 7fb7a803d700 1 --2- 172.21.15.16:0/3339146732 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb7a006f5d0 0x7fb7a012ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.423+0000 7fb7a803d700 1 -- 172.21.15.16:0/3339146732 >> 172.21.15.16:0/3339146732 conn(0x7fb7a00a2aa0 msgr2=0x7fb7a00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:41.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.423+0000 7fb7a803d700 1 -- 172.21.15.16:0/3339146732 shutdown_connections 2023-09-30T22:30:41.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.423+0000 7fb7a803d700 1 -- 172.21.15.16:0/3339146732 wait complete. 2023-09-30T22:30:41.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.424+0000 7fb7a803d700 1 Processor -- start 2023-09-30T22:30:41.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.424+0000 7fb7a803d700 1 -- start start 2023-09-30T22:30:41.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.425+0000 7fb7a803d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb7a006f5d0 0x7fb7a0129190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:41.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.425+0000 7fb7a803d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb7a012cf60 0x7fb7a01296d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:41.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.425+0000 7fb7a803d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7a0135c20 0x7fb7a0099d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:41.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.425+0000 7fb7a803d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a009d6c0 con 0x7fb7a0135c20 2023-09-30T22:30:41.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.425+0000 7fb7a803d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a00a1330 con 0x7fb7a012cf60 2023-09-30T22:30:41.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.425+0000 7fb7a803d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a009c760 con 0x7fb7a006f5d0 2023-09-30T22:30:41.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.425+0000 7fb7a65da700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7a0135c20 0x7fb7a0099d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:41.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.425+0000 7fb7a65da700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7a0135c20 0x7fb7a0099d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51984/0 (socket says 172.21.15.16:51984) 2023-09-30T22:30:41.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.425+0000 7fb7a65da700 1 -- 172.21.15.16:0/2147730639 learned_addr learned my addr 172.21.15.16:0/2147730639 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:41.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.425+0000 7fb7a5dd9700 1 --2- 172.21.15.16:0/2147730639 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb7a006f5d0 0x7fb7a0129190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:41.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.425+0000 7fb7a55d8700 1 --2- 172.21.15.16:0/2147730639 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb7a012cf60 0x7fb7a01296d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:41.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.426+0000 7fb7a65da700 1 -- 172.21.15.16:0/2147730639 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb7a006f5d0 msgr2=0x7fb7a0129190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:41.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.426+0000 7fb7a65da700 1 --2- 172.21.15.16:0/2147730639 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb7a006f5d0 0x7fb7a0129190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.426+0000 7fb7a65da700 1 -- 172.21.15.16:0/2147730639 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb7a012cf60 msgr2=0x7fb7a01296d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:41.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.426+0000 7fb7a65da700 1 --2- 172.21.15.16:0/2147730639 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb7a012cf60 0x7fb7a01296d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.426+0000 7fb7a65da700 1 -- 172.21.15.16:0/2147730639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb79800bc80 con 0x7fb7a0135c20 2023-09-30T22:30:41.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.426+0000 7fb7a65da700 1 --2- 172.21.15.16:0/2147730639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7a0135c20 0x7fb7a0099d50 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb798008000 tx=0x7fb798004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:41.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.426+0000 7fb7967fc700 1 -- 172.21.15.16:0/2147730639 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb798004450 con 0x7fb7a0135c20 2023-09-30T22:30:41.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.426+0000 7fb7a803d700 1 -- 172.21.15.16:0/2147730639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7a009b0a0 con 0x7fb7a0135c20 2023-09-30T22:30:41.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.426+0000 7fb7a803d700 1 -- 172.21.15.16:0/2147730639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7a0129c10 con 0x7fb7a0135c20 2023-09-30T22:30:41.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.426+0000 7fb7967fc700 1 -- 172.21.15.16:0/2147730639 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7980045b0 con 0x7fb7a0135c20 2023-09-30T22:30:41.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.426+0000 7fb7967fc700 1 -- 172.21.15.16:0/2147730639 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb798004960 con 0x7fb7a0135c20 2023-09-30T22:30:41.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.427+0000 7fb7967fc700 1 -- 172.21.15.16:0/2147730639 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb79800a530 con 0x7fb7a0135c20 2023-09-30T22:30:41.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.428+0000 7fb7a803d700 1 -- 172.21.15.16:0/2147730639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7880052e0 con 0x7fb7a0135c20 2023-09-30T22:30:41.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.428+0000 7fb7967fc700 1 -- 172.21.15.16:0/2147730639 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fb798041280 con 0x7fb7a0135c20 2023-09-30T22:30:41.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.437+0000 7fb7967fc700 1 -- 172.21.15.16:0/2147730639 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb798022020 con 0x7fb7a0135c20 2023-09-30T22:30:41.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.486+0000 7f18db8a1700 1 -- 172.21.15.16:0/756756425 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f18d406b250 con 0x7f18d4133a80 2023-09-30T22:30:41.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.486+0000 7f18c9ffb700 1 -- 172.21.15.16:0/756756425 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f18d001a8b0 con 0x7f18d4133a80 2023-09-30T22:30:41.487 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:30:41.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.489+0000 7f18db8a1700 1 -- 172.21.15.16:0/756756425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18d4133a80 msgr2=0x7f18d412de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:41.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.489+0000 7f18db8a1700 1 --2- 172.21.15.16:0/756756425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18d4133a80 0x7f18d412de10 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f18d000cc80 tx=0x7f18d000cf90 comp rx=0 tx=0).stop 2023-09-30T22:30:41.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.489+0000 7f18db8a1700 1 -- 172.21.15.16:0/756756425 shutdown_connections 2023-09-30T22:30:41.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.489+0000 7f18db8a1700 1 --2- 172.21.15.16:0/756756425 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f18d4139140 0x7f18d412e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.489+0000 7f18db8a1700 1 --2- 172.21.15.16:0/756756425 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18d4133a80 0x7f18d412de10 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.489+0000 7f18db8a1700 1 --2- 172.21.15.16:0/756756425 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f18d4136820 0x7f18d412e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.489+0000 7f18db8a1700 1 -- 172.21.15.16:0/756756425 >> 172.21.15.16:0/756756425 conn(0x7f18d40a4210 msgr2=0x7f18d4137310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:41.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.489+0000 7f18db8a1700 1 -- 172.21.15.16:0/756756425 shutdown_connections 2023-09-30T22:30:41.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.489+0000 7f18db8a1700 1 -- 172.21.15.16:0/756756425 wait complete. 2023-09-30T22:30:41.504 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:30:41.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.615+0000 7f74f8fee700 1 -- 172.21.15.16:0/1818501751 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f74f409f420 con 0x7f74f409e1d0 2023-09-30T22:30:41.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.616+0000 7f74e2ffd700 1 -- 172.21.15.16:0/1818501751 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f74e803b090 con 0x7f74f409e1d0 2023-09-30T22:30:41.616 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:30:41.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.618+0000 7f74f8fee700 1 -- 172.21.15.16:0/1818501751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74f409e1d0 msgr2=0x7f74f41199c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:41.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.618+0000 7f74f8fee700 1 --2- 172.21.15.16:0/1818501751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74f409e1d0 0x7f74f41199c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f74e801a000 tx=0x7f74e8004750 comp rx=0 tx=0).stop 2023-09-30T22:30:41.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.618+0000 7f74f8fee700 1 -- 172.21.15.16:0/1818501751 shutdown_connections 2023-09-30T22:30:41.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.618+0000 7f74f8fee700 1 --2- 172.21.15.16:0/1818501751 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f74f409d6f0 0x7f74f4119480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.618+0000 7f74f8fee700 1 --2- 172.21.15.16:0/1818501751 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f74f4139170 0x7f74f4119f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.618+0000 7f74f8fee700 1 --2- 172.21.15.16:0/1818501751 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74f409e1d0 0x7f74f41199c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.618+0000 7f74f8fee700 1 -- 172.21.15.16:0/1818501751 >> 172.21.15.16:0/1818501751 conn(0x7f74f4066f50 msgr2=0x7f74f409b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:41.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.618+0000 7f74f8fee700 1 -- 172.21.15.16:0/1818501751 shutdown_connections 2023-09-30T22:30:41.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.618+0000 7f74f8fee700 1 -- 172.21.15.16:0/1818501751 wait complete. 2023-09-30T22:30:41.633 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:30:41.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.651+0000 7fd02041c700 1 -- 172.21.15.16:0/2884085548 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd018122260 con 0x7fd018136d80 2023-09-30T22:30:41.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.652+0000 7fd00effd700 1 -- 172.21.15.16:0/2884085548 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd01002c020 con 0x7fd018136d80 2023-09-30T22:30:41.653 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:30:41.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.654+0000 7fd02041c700 1 -- 172.21.15.16:0/2884085548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd018136d80 msgr2=0x7fd0181c8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:41.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.654+0000 7fd02041c700 1 --2- 172.21.15.16:0/2884085548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd018136d80 0x7fd0181c8540 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd01000b370 tx=0x7fd010004ad0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.654+0000 7fd02041c700 1 -- 172.21.15.16:0/2884085548 shutdown_connections 2023-09-30T22:30:41.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.654+0000 7fd02041c700 1 --2- 172.21.15.16:0/2884085548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd018136d80 0x7fd0181c8540 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.654+0000 7fd02041c700 1 --2- 172.21.15.16:0/2884085548 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd01806a770 0x7fd01809b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.654+0000 7fd02041c700 1 --2- 172.21.15.16:0/2884085548 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd01806b250 0x7fd0181c8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.654+0000 7fd02041c700 1 -- 172.21.15.16:0/2884085548 >> 172.21.15.16:0/2884085548 conn(0x7fd018066f50 msgr2=0x7fd018070040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:41.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.654+0000 7fd02041c700 1 -- 172.21.15.16:0/2884085548 shutdown_connections 2023-09-30T22:30:41.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.654+0000 7fd02041c700 1 -- 172.21.15.16:0/2884085548 wait complete. 2023-09-30T22:30:41.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.665+0000 7fb7a803d700 1 -- 172.21.15.16:0/2147730639 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb788005150 con 0x7fb7a0135c20 2023-09-30T22:30:41.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.665+0000 7fb7967fc700 1 -- 172.21.15.16:0/2147730639 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb79801e420 con 0x7fb7a0135c20 2023-09-30T22:30:41.666 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:30:41.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.667+0000 7fb7a803d700 1 -- 172.21.15.16:0/2147730639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7a0135c20 msgr2=0x7fb7a0099d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:41.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.667+0000 7fb7a803d700 1 --2- 172.21.15.16:0/2147730639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7a0135c20 0x7fb7a0099d50 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb798008000 tx=0x7fb798004ad0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.668+0000 7fb7a803d700 1 -- 172.21.15.16:0/2147730639 shutdown_connections 2023-09-30T22:30:41.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.668+0000 7fb7a803d700 1 --2- 172.21.15.16:0/2147730639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb7a0135c20 0x7fb7a0099d50 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.668+0000 7fb7a803d700 1 --2- 172.21.15.16:0/2147730639 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb7a006f5d0 0x7fb7a0129190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.668+0000 7fb7a803d700 1 --2- 172.21.15.16:0/2147730639 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb7a012cf60 0x7fb7a01296d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:41.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.668+0000 7fb7a803d700 1 -- 172.21.15.16:0/2147730639 >> 172.21.15.16:0/2147730639 conn(0x7fb7a00a2aa0 msgr2=0x7fb7a006e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:41.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.668+0000 7fb7a803d700 1 -- 172.21.15.16:0/2147730639 shutdown_connections 2023-09-30T22:30:41.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:41.668+0000 7fb7a803d700 1 -- 172.21.15.16:0/2147730639 wait complete. 2023-09-30T22:30:41.670 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:30:41.682 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:30:42.505 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:42.634 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:42.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.651+0000 7fcd1d43c700 1 Processor -- start 2023-09-30T22:30:42.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.651+0000 7fcd1d43c700 1 -- start start 2023-09-30T22:30:42.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.652+0000 7fcd1d43c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcd1809e1d0 0x7fcd18135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:42.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.652+0000 7fcd1d43c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd18135e50 0x7fcd18138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:42.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.652+0000 7fcd1d43c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd1809f130 con 0x7fcd1809d6f0 2023-09-30T22:30:42.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.652+0000 7fcd1d43c700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd1809f520 con 0x7fcd1809e1d0 2023-09-30T22:30:42.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.652+0000 7fcd1d43c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1809e930 con 0x7fcd18135e50 2023-09-30T22:30:42.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.652+0000 7fcd167fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcd1809e1d0 0x7fcd18135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:42.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.652+0000 7fcd177fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd18135e50 0x7fcd18138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:42.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.652+0000 7fcd177fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd18135e50 0x7fcd18138230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36920/0 (socket says 172.21.15.16:36920) 2023-09-30T22:30:42.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.652+0000 7fcd167fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcd1809e1d0 0x7fcd18135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51992/0 (socket says 172.21.15.16:51992) 2023-09-30T22:30:42.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.652+0000 7fcd16ffd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fcd1809d6f0 0x7fcd1809dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58324/0 (socket says 172.21.15.16:58324) 2023-09-30T22:30:42.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.652+0000 7fcd177fe700 1 -- 172.21.15.16:0/3274435513 learned_addr learned my addr 172.21.15.16:0/3274435513 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:42.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.652+0000 7fcd15ffb700 1 -- 172.21.15.16:0/3274435513 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 506373010 0 0) 0x7fcd1809f130 con 0x7fcd1809d6f0 2023-09-30T22:30:42.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.652+0000 7fcd177fe700 1 -- 172.21.15.16:0/3274435513 >> v2:172.21.15.16:3300/0 conn(0x7fcd1809e1d0 msgr2=0x7fcd18135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:42.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.652+0000 7fcd177fe700 1 --2- 172.21.15.16:0/3274435513 >> v2:172.21.15.16:3300/0 conn(0x7fcd1809e1d0 0x7fcd18135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:42.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.652+0000 7fcd177fe700 1 -- 172.21.15.16:0/3274435513 >> v1:172.21.15.16:6789/0 conn(0x7fcd1809d6f0 legacy=0x7fcd1809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:42.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.653+0000 7fcd177fe700 1 -- 172.21.15.16:0/3274435513 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd1809b2d0 con 0x7fcd18135e50 2023-09-30T22:30:42.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.653+0000 7fcd167fc700 1 --2- 172.21.15.16:0/3274435513 >> v2:172.21.15.16:3300/0 conn(0x7fcd1809e1d0 0x7fcd18135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:42.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.653+0000 7fcd177fe700 1 --2- 172.21.15.16:0/3274435513 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd18135e50 0x7fcd18138230 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fcd10009e50 tx=0x7fcd1000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0791501b8c33d43 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:42.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.653+0000 7fcd15ffb700 1 -- 172.21.15.16:0/3274435513 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1000e070 con 0x7fcd18135e50 2023-09-30T22:30:42.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.653+0000 7fcd15ffb700 1 -- 172.21.15.16:0/3274435513 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd1000b910 con 0x7fcd18135e50 2023-09-30T22:30:42.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.653+0000 7fcd15ffb700 1 -- 172.21.15.16:0/3274435513 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1000bcc0 con 0x7fcd18135e50 2023-09-30T22:30:42.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.654+0000 7fcd1d43c700 1 -- 172.21.15.16:0/3274435513 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd18135e50 msgr2=0x7fcd18138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:42.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.654+0000 7fcd1d43c700 1 --2- 172.21.15.16:0/3274435513 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd18135e50 0x7fcd18138230 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fcd10009e50 tx=0x7fcd1000b330 comp rx=0 tx=0).stop 2023-09-30T22:30:42.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.654+0000 7fcd1d43c700 1 -- 172.21.15.16:0/3274435513 shutdown_connections 2023-09-30T22:30:42.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.654+0000 7fcd1d43c700 1 --2- 172.21.15.16:0/3274435513 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd18135e50 0x7fcd18138230 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:42.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.654+0000 7fcd1d43c700 1 --2- 172.21.15.16:0/3274435513 >> v2:172.21.15.16:3300/0 conn(0x7fcd1809e1d0 0x7fcd18135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:42.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.654+0000 7fcd1d43c700 1 -- 172.21.15.16:0/3274435513 >> 172.21.15.16:0/3274435513 conn(0x7fcd18066f50 msgr2=0x7fcd1809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:42.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.654+0000 7fcd1d43c700 1 -- 172.21.15.16:0/3274435513 shutdown_connections 2023-09-30T22:30:42.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.654+0000 7fcd1d43c700 1 -- 172.21.15.16:0/3274435513 wait complete. 2023-09-30T22:30:42.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.655+0000 7fcd1d43c700 1 Processor -- start 2023-09-30T22:30:42.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.655+0000 7fcd1d43c700 1 -- start start 2023-09-30T22:30:42.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.655+0000 7fcd1d43c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd1809d6f0 0x7fcd181c80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:42.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.655+0000 7fcd1d43c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd1809e1d0 0x7fcd181c8620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:42.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.655+0000 7fcd1d43c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd18135e50 0x7fcd181c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:42.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.655+0000 7fcd1d43c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd181214f0 con 0x7fcd18135e50 2023-09-30T22:30:42.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.655+0000 7fcd1d43c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd18121300 con 0x7fcd1809e1d0 2023-09-30T22:30:42.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.655+0000 7fcd1d43c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcd18132060 con 0x7fcd1809d6f0 2023-09-30T22:30:42.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.656+0000 7fcd167fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd1809e1d0 0x7fcd181c8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:42.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.656+0000 7fcd16ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd1809d6f0 0x7fcd181c80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:42.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.656+0000 7fcd167fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd1809e1d0 0x7fcd181c8620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36926/0 (socket says 172.21.15.16:36926) 2023-09-30T22:30:42.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.656+0000 7fcd167fc700 1 -- 172.21.15.16:0/2783167303 learned_addr learned my addr 172.21.15.16:0/2783167303 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:42.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.656+0000 7fcd177fe700 1 --2- 172.21.15.16:0/2783167303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd18135e50 0x7fcd181c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:42.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.656+0000 7fcd16ffd700 1 -- 172.21.15.16:0/2783167303 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd1809e1d0 msgr2=0x7fcd181c8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:42.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.656+0000 7fcd16ffd700 1 --2- 172.21.15.16:0/2783167303 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd1809e1d0 0x7fcd181c8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:42.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.656+0000 7fcd16ffd700 1 -- 172.21.15.16:0/2783167303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd18135e50 msgr2=0x7fcd181c8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:42.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.656+0000 7fcd16ffd700 1 --2- 172.21.15.16:0/2783167303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd18135e50 0x7fcd181c8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:42.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.656+0000 7fcd16ffd700 1 -- 172.21.15.16:0/2783167303 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd0c0094a0 con 0x7fcd1809d6f0 2023-09-30T22:30:42.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.656+0000 7fcd167fc700 1 --2- 172.21.15.16:0/2783167303 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd1809e1d0 0x7fcd181c8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:42.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.656+0000 7fcd16ffd700 1 --2- 172.21.15.16:0/2783167303 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd1809d6f0 0x7fcd181c80e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcd0800c3c0 tx=0x7fcd0800c6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:42.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.656+0000 7fccff7fe700 1 -- 172.21.15.16:0/2783167303 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd080062d0 con 0x7fcd1809d6f0 2023-09-30T22:30:42.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.656+0000 7fccff7fe700 1 -- 172.21.15.16:0/2783167303 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd08006430 con 0x7fcd1809d6f0 2023-09-30T22:30:42.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.656+0000 7fcd1d43c700 1 -- 172.21.15.16:0/2783167303 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd10009b30 con 0x7fcd1809d6f0 2023-09-30T22:30:42.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.656+0000 7fccff7fe700 1 -- 172.21.15.16:0/2783167303 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd08004850 con 0x7fcd1809d6f0 2023-09-30T22:30:42.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.656+0000 7fcd1d43c700 1 -- 172.21.15.16:0/2783167303 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd1809c700 con 0x7fcd1809d6f0 2023-09-30T22:30:42.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.657+0000 7fccff7fe700 1 -- 172.21.15.16:0/2783167303 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcd080049b0 con 0x7fcd1809d6f0 2023-09-30T22:30:42.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.657+0000 7fccff7fe700 1 -- 172.21.15.16:0/2783167303 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fcd0803e960 con 0x7fcd1809d6f0 2023-09-30T22:30:42.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.657+0000 7fcd1d43c700 1 -- 172.21.15.16:0/2783167303 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd18074ea0 con 0x7fcd1809d6f0 2023-09-30T22:30:42.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.663+0000 7fccff7fe700 1 -- 172.21.15.16:0/2783167303 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcd08019b40 con 0x7fcd1809d6f0 2023-09-30T22:30:42.671 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:42.683 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:42.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.788+0000 7ff04aaf5700 1 Processor -- start 2023-09-30T22:30:42.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.788+0000 7ff04aaf5700 1 -- start start 2023-09-30T22:30:42.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.789+0000 7ff04aaf5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff04409d6f0 0x7ff04409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:42.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.789+0000 7ff04aaf5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff04409e000 0x7ff0441339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:42.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.789+0000 7ff04aaf5700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff04409e410 0x7ff044133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:42.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.789+0000 7ff04aaf5700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff04406b460 con 0x7ff04409e410 2023-09-30T22:30:42.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.789+0000 7ff04aaf5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff04409b120 con 0x7ff04409d6f0 2023-09-30T22:30:42.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.789+0000 7ff04aaf5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff04409a7e0 con 0x7ff04409e000 2023-09-30T22:30:42.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.789+0000 7ff043fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff04409e000 0x7ff0441339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:42.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.789+0000 7ff043fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff04409e000 0x7ff0441339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:43688/0 (socket says 172.21.15.16:43688) 2023-09-30T22:30:42.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.789+0000 7ff043fff700 1 -- 172.21.15.16:0/1276048056 learned_addr learned my addr 172.21.15.16:0/1276048056 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:42.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.789+0000 7ff049092700 1 --2- 172.21.15.16:0/1276048056 >> v2:172.21.15.16:3300/0 conn(0x7ff04409e410 0x7ff044133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:42.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.789+0000 7ff048891700 1 --2- 172.21.15.16:0/1276048056 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff04409d6f0 0x7ff04409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:42.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.790+0000 7ff048891700 1 -- 172.21.15.16:0/1276048056 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff04409e000 msgr2=0x7ff0441339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:42.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.790+0000 7ff048891700 1 --2- 172.21.15.16:0/1276048056 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff04409e000 0x7ff0441339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:42.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.790+0000 7ff048891700 1 -- 172.21.15.16:0/1276048056 >> v2:172.21.15.16:3300/0 conn(0x7ff04409e410 msgr2=0x7ff044133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:42.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.790+0000 7ff048891700 1 --2- 172.21.15.16:0/1276048056 >> v2:172.21.15.16:3300/0 conn(0x7ff04409e410 0x7ff044133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:42.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.790+0000 7ff048891700 1 -- 172.21.15.16:0/1276048056 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff04409a500 con 0x7ff04409d6f0 2023-09-30T22:30:42.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.790+0000 7ff049092700 1 --2- 172.21.15.16:0/1276048056 >> v2:172.21.15.16:3300/0 conn(0x7ff04409e410 0x7ff044133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:42.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.790+0000 7ff048891700 1 --2- 172.21.15.16:0/1276048056 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff04409d6f0 0x7ff04409dac0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff038009870 tx=0x7ff038009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fabf560c646453d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:42.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.790+0000 7ff0437fe700 1 -- 172.21.15.16:0/1276048056 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff038011830 con 0x7ff04409d6f0 2023-09-30T22:30:42.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.790+0000 7ff0437fe700 1 -- 172.21.15.16:0/1276048056 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff038011990 con 0x7ff04409d6f0 2023-09-30T22:30:42.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.790+0000 7ff0437fe700 1 -- 172.21.15.16:0/1276048056 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff038011d20 con 0x7ff04409d6f0 2023-09-30T22:30:42.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.791+0000 7ff04aaf5700 1 -- 172.21.15.16:0/1276048056 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff04409d6f0 msgr2=0x7ff04409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:42.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.791+0000 7ff04aaf5700 1 --2- 172.21.15.16:0/1276048056 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff04409d6f0 0x7ff04409dac0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff038009870 tx=0x7ff038009b80 comp rx=0 tx=0).stop 2023-09-30T22:30:42.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.791+0000 7ff04aaf5700 1 -- 172.21.15.16:0/1276048056 shutdown_connections 2023-09-30T22:30:42.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.791+0000 7ff04aaf5700 1 --2- 172.21.15.16:0/1276048056 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff04409d6f0 0x7ff04409dac0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:42.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.791+0000 7ff04aaf5700 1 --2- 172.21.15.16:0/1276048056 >> v2:172.21.15.16:3300/0 conn(0x7ff04409e410 0x7ff044133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:42.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.791+0000 7ff04aaf5700 1 --2- 172.21.15.16:0/1276048056 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff04409e000 0x7ff0441339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:42.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.791+0000 7ff04aaf5700 1 -- 172.21.15.16:0/1276048056 >> 172.21.15.16:0/1276048056 conn(0x7ff044066f50 msgr2=0x7ff04409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:42.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.791+0000 7ff04aaf5700 1 -- 172.21.15.16:0/1276048056 shutdown_connections 2023-09-30T22:30:42.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.791+0000 7ff04aaf5700 1 -- 172.21.15.16:0/1276048056 wait complete. 2023-09-30T22:30:42.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.792+0000 7ff04aaf5700 1 Processor -- start 2023-09-30T22:30:42.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.793+0000 7ff04aaf5700 1 -- start start 2023-09-30T22:30:42.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.793+0000 7ff04aaf5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff04409d6f0 0x7ff0441a5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:42.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.793+0000 7ff04aaf5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff04409e000 0x7ff0441a62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:42.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.793+0000 7ff04aaf5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff04409e410 0x7ff0441a6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:42.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.793+0000 7ff04aaf5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0441aa270 con 0x7ff04409e000 2023-09-30T22:30:42.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.793+0000 7ff04aaf5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0441a79a0 con 0x7ff04409e410 2023-09-30T22:30:42.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.793+0000 7ff04aaf5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff0441a7fc0 con 0x7ff04409d6f0 2023-09-30T22:30:42.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.793+0000 7ff048891700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff04409d6f0 0x7ff0441a5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:42.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.793+0000 7ff048891700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff04409d6f0 0x7ff0441a5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:43694/0 (socket says 172.21.15.16:43694) 2023-09-30T22:30:42.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.793+0000 7ff048891700 1 -- 172.21.15.16:0/3628770424 learned_addr learned my addr 172.21.15.16:0/3628770424 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:42.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.793+0000 7ff049092700 1 --2- 172.21.15.16:0/3628770424 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff04409e410 0x7ff0441a6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:42.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.794+0000 7ff048891700 1 -- 172.21.15.16:0/3628770424 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff04409e410 msgr2=0x7ff0441a6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:42.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.794+0000 7ff048891700 1 --2- 172.21.15.16:0/3628770424 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff04409e410 0x7ff0441a6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:42.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.794+0000 7ff048891700 1 -- 172.21.15.16:0/3628770424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff04409e000 msgr2=0x7ff0441a62d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:30:42.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.794+0000 7ff048891700 1 --2- 172.21.15.16:0/3628770424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff04409e000 0x7ff0441a62d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:42.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.794+0000 7ff048891700 1 -- 172.21.15.16:0/3628770424 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff03c009a70 con 0x7ff04409d6f0 2023-09-30T22:30:42.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.794+0000 7ff049092700 1 --2- 172.21.15.16:0/3628770424 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff04409e410 0x7ff0441a6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:42.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.794+0000 7ff048891700 1 --2- 172.21.15.16:0/3628770424 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff04409d6f0 0x7ff0441a5d90 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff0380090b0 tx=0x7ff038004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:42.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.794+0000 7ff0417fa700 1 -- 172.21.15.16:0/3628770424 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff038004510 con 0x7ff04409d6f0 2023-09-30T22:30:42.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.794+0000 7ff0417fa700 1 -- 172.21.15.16:0/3628770424 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff03801f070 con 0x7ff04409d6f0 2023-09-30T22:30:42.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.794+0000 7ff0417fa700 1 -- 172.21.15.16:0/3628770424 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0380038e0 con 0x7ff04409d6f0 2023-09-30T22:30:42.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.794+0000 7ff04aaf5700 1 -- 172.21.15.16:0/3628770424 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0380094a0 con 0x7ff04409d6f0 2023-09-30T22:30:42.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.794+0000 7ff04aaf5700 1 -- 172.21.15.16:0/3628770424 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0440a21d0 con 0x7ff04409d6f0 2023-09-30T22:30:42.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.796+0000 7ff04aaf5700 1 -- 172.21.15.16:0/3628770424 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff04409c770 con 0x7ff04409d6f0 2023-09-30T22:30:42.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.796+0000 7ff0417fa700 1 -- 172.21.15.16:0/3628770424 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff038003a40 con 0x7ff04409d6f0 2023-09-30T22:30:42.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.796+0000 7ff0417fa700 1 -- 172.21.15.16:0/3628770424 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7ff038021070 con 0x7ff04409d6f0 2023-09-30T22:30:42.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.801+0000 7ff0417fa700 1 -- 172.21.15.16:0/3628770424 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff03802f3a0 con 0x7ff04409d6f0 2023-09-30T22:30:42.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.821+0000 7ff31f6cf700 1 Processor -- start 2023-09-30T22:30:42.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.822+0000 7ff31f6cf700 1 -- start start 2023-09-30T22:30:42.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.823+0000 7ff31f6cf700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff31812cf60 0x7ff31806f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:42.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.823+0000 7ff31f6cf700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff31812f8b0 0x7ff31812fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:42.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.823+0000 7ff31f6cf700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff31812cd90 con 0x7ff318130260 2023-09-30T22:30:42.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.823+0000 7ff31f6cf700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff31812d330 con 0x7ff31812f8b0 2023-09-30T22:30:42.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.823+0000 7ff31f6cf700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff3181306d0 con 0x7ff31812cf60 2023-09-30T22:30:42.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.823+0000 7ff31d46b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff31812cf60 0x7ff31806f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:42.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.823+0000 7ff31cc6a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff31812f8b0 0x7ff31812fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:42.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.823+0000 7ff31d46b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff31812cf60 0x7ff31806f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:43706/0 (socket says 172.21.15.16:43706) 2023-09-30T22:30:42.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.823+0000 7ff31d46b700 1 -- 172.21.15.16:0/4293661156 learned_addr learned my addr 172.21.15.16:0/4293661156 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:42.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.823+0000 7ff31d46b700 1 -- 172.21.15.16:0/4293661156 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff31812f8b0 msgr2=0x7ff31812fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:42.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.823+0000 7ff31d46b700 1 --2- 172.21.15.16:0/4293661156 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff31812f8b0 0x7ff31812fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:42.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.823+0000 7ff31d46b700 1 -- 172.21.15.16:0/4293661156 >> v1:172.21.15.16:6789/0 conn(0x7ff318130260 legacy=0x7ff318136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:42.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.824+0000 7ff31d46b700 1 -- 172.21.15.16:0/4293661156 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff318071780 con 0x7ff31812cf60 2023-09-30T22:30:42.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.824+0000 7ff31d46b700 1 --2- 172.21.15.16:0/4293661156 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff31812cf60 0x7ff31806f5d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff314009920 tx=0x7ff314009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6159c2071272efb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:42.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.824+0000 7ff30ffff700 1 -- 172.21.15.16:0/4293661156 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff314004230 con 0x7ff31812cf60 2023-09-30T22:30:42.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.824+0000 7ff30ffff700 1 -- 172.21.15.16:0/4293661156 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff314004390 con 0x7ff31812cf60 2023-09-30T22:30:42.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.824+0000 7ff30ffff700 1 -- 172.21.15.16:0/4293661156 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3140036a0 con 0x7ff31812cf60 2023-09-30T22:30:42.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.825+0000 7ff31f6cf700 1 -- 172.21.15.16:0/4293661156 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff31812cf60 msgr2=0x7ff31806f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:42.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.825+0000 7ff31f6cf700 1 --2- 172.21.15.16:0/4293661156 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff31812cf60 0x7ff31806f5d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff314009920 tx=0x7ff314009c30 comp rx=0 tx=0).stop 2023-09-30T22:30:42.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.825+0000 7ff31f6cf700 1 -- 172.21.15.16:0/4293661156 shutdown_connections 2023-09-30T22:30:42.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.825+0000 7ff31f6cf700 1 --2- 172.21.15.16:0/4293661156 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff31812cf60 0x7ff31806f5d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:42.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.825+0000 7ff31f6cf700 1 --2- 172.21.15.16:0/4293661156 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff31812f8b0 0x7ff31812fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:42.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.825+0000 7ff31f6cf700 1 -- 172.21.15.16:0/4293661156 >> 172.21.15.16:0/4293661156 conn(0x7ff3180a2aa0 msgr2=0x7ff3180a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:42.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.825+0000 7ff31f6cf700 1 -- 172.21.15.16:0/4293661156 shutdown_connections 2023-09-30T22:30:42.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.825+0000 7ff31f6cf700 1 -- 172.21.15.16:0/4293661156 wait complete. 2023-09-30T22:30:42.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.826+0000 7ff31f6cf700 1 Processor -- start 2023-09-30T22:30:42.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.827+0000 7ff31f6cf700 1 -- start start 2023-09-30T22:30:42.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.827+0000 7ff31f6cf700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff31812cf60 0x7ff3181214e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:42.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.827+0000 7ff31f6cf700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff31812f8b0 0x7ff318121a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:42.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.827+0000 7ff31f6cf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff318130260 0x7ff318121f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:42.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.827+0000 7ff31f6cf700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff31809e930 con 0x7ff318130260 2023-09-30T22:30:42.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.827+0000 7ff31f6cf700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff318123920 con 0x7ff31812f8b0 2023-09-30T22:30:42.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.827+0000 7ff31f6cf700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff31812f540 con 0x7ff31812cf60 2023-09-30T22:30:42.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.827+0000 7ff31d46b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff31812cf60 0x7ff3181214e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:42.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.827+0000 7ff31d46b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff31812cf60 0x7ff3181214e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:43718/0 (socket says 172.21.15.16:43718) 2023-09-30T22:30:42.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.827+0000 7ff31d46b700 1 -- 172.21.15.16:0/232819354 learned_addr learned my addr 172.21.15.16:0/232819354 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:42.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.828+0000 7ff31cc6a700 1 --2- 172.21.15.16:0/232819354 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff31812f8b0 0x7ff318121a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:42.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.828+0000 7ff31d46b700 1 -- 172.21.15.16:0/232819354 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff31812f8b0 msgr2=0x7ff318121a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:42.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.828+0000 7ff31d46b700 1 --2- 172.21.15.16:0/232819354 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff31812f8b0 0x7ff318121a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:42.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.828+0000 7ff31d46b700 1 -- 172.21.15.16:0/232819354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff318130260 msgr2=0x7ff318121f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:42.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.828+0000 7ff31d46b700 1 --2- 172.21.15.16:0/232819354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff318130260 0x7ff318121f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:42.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.828+0000 7ff31d46b700 1 -- 172.21.15.16:0/232819354 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff314009550 con 0x7ff31812cf60 2023-09-30T22:30:42.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.828+0000 7ff31d46b700 1 --2- 172.21.15.16:0/232819354 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff31812cf60 0x7ff3181214e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff314009130 tx=0x7ff314004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:42.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.828+0000 7ff30dffb700 1 -- 172.21.15.16:0/232819354 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31401e070 con 0x7ff31812cf60 2023-09-30T22:30:42.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.828+0000 7ff30dffb700 1 -- 172.21.15.16:0/232819354 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff314010600 con 0x7ff31812cf60 2023-09-30T22:30:42.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.828+0000 7ff31f6cf700 1 -- 172.21.15.16:0/232819354 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff318129fa0 con 0x7ff31812cf60 2023-09-30T22:30:42.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.829+0000 7ff31f6cf700 1 -- 172.21.15.16:0/232819354 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff31809e150 con 0x7ff31812cf60 2023-09-30T22:30:42.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.829+0000 7ff30dffb700 1 -- 172.21.15.16:0/232819354 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff314010990 con 0x7ff31812cf60 2023-09-30T22:30:42.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.829+0000 7ff30dffb700 1 -- 172.21.15.16:0/232819354 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff314010af0 con 0x7ff31812cf60 2023-09-30T22:30:42.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.830+0000 7ff30dffb700 1 -- 172.21.15.16:0/232819354 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7ff314040580 con 0x7ff31812cf60 2023-09-30T22:30:42.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.830+0000 7ff31f6cf700 1 -- 172.21.15.16:0/232819354 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2f80052e0 con 0x7ff31812cf60 2023-09-30T22:30:42.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.839+0000 7ff30dffb700 1 -- 172.21.15.16:0/232819354 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff31400f370 con 0x7ff31812cf60 2023-09-30T22:30:42.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.868+0000 7fe579c14700 1 Processor -- start 2023-09-30T22:30:42.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.868+0000 7fe579c14700 1 -- start start 2023-09-30T22:30:42.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.869+0000 7fe579c14700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe57409d630 0x7fe5740a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:42.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.869+0000 7fe579c14700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe574138d80 0x7fe57413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:42.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.869+0000 7fe579c14700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5740a2a70 con 0x7fe5740a0a30 2023-09-30T22:30:42.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.869+0000 7fe579c14700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5740a01d0 con 0x7fe57409d630 2023-09-30T22:30:42.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.869+0000 7fe579c14700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe57409e2b0 con 0x7fe574138d80 2023-09-30T22:30:42.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.869+0000 7fe5737fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe5740a0a30 0x7fe5740a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58346/0 (socket says 172.21.15.16:58346) 2023-09-30T22:30:42.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.869+0000 7fe5737fe700 1 -- 172.21.15.16:0/4019573210 learned_addr learned my addr 172.21.15.16:0/4019573210 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:30:42.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.869+0000 7fe572ffd700 1 --2- 172.21.15.16:0/4019573210 >> v2:172.21.15.16:3300/0 conn(0x7fe57409d630 0x7fe5740a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:42.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.869+0000 7fe573fff700 1 --2- 172.21.15.16:0/4019573210 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe574138d80 0x7fe57413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:42.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.869+0000 7fe5727fc700 1 -- 172.21.15.16:0/4019573210 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4074082480 0 0) 0x7fe5740a2a70 con 0x7fe5740a0a30 2023-09-30T22:30:42.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.869+0000 7fe5727fc700 1 -- 172.21.15.16:0/4019573210 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe55c0035b0 con 0x7fe5740a0a30 2023-09-30T22:30:42.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.870+0000 7fe573fff700 1 -- 172.21.15.16:0/4019573210 >> v2:172.21.15.16:3300/0 conn(0x7fe57409d630 msgr2=0x7fe5740a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:42.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.870+0000 7fe573fff700 1 --2- 172.21.15.16:0/4019573210 >> v2:172.21.15.16:3300/0 conn(0x7fe57409d630 0x7fe5740a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:42.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.870+0000 7fe573fff700 1 -- 172.21.15.16:0/4019573210 >> v1:172.21.15.16:6789/0 conn(0x7fe5740a0a30 legacy=0x7fe5740a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:42.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.870+0000 7fe573fff700 1 -- 172.21.15.16:0/4019573210 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe57409dc50 con 0x7fe574138d80 2023-09-30T22:30:42.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.870+0000 7fe573fff700 1 --2- 172.21.15.16:0/4019573210 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe574138d80 0x7fe57413b160 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe56c009d90 tx=0x7fe56c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7ba3cd1e8f7d45a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:42.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.870+0000 7fe5727fc700 1 -- 172.21.15.16:0/4019573210 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe56c00e070 con 0x7fe574138d80 2023-09-30T22:30:42.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.870+0000 7fe5727fc700 1 -- 172.21.15.16:0/4019573210 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe56c00bb20 con 0x7fe574138d80 2023-09-30T22:30:42.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.870+0000 7fe5727fc700 1 -- 172.21.15.16:0/4019573210 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe56c00be90 con 0x7fe574138d80 2023-09-30T22:30:42.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.871+0000 7fe579c14700 1 -- 172.21.15.16:0/4019573210 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe574138d80 msgr2=0x7fe57413b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:42.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.871+0000 7fe579c14700 1 --2- 172.21.15.16:0/4019573210 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe574138d80 0x7fe57413b160 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe56c009d90 tx=0x7fe56c00b330 comp rx=0 tx=0).stop 2023-09-30T22:30:42.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.871+0000 7fe579c14700 1 -- 172.21.15.16:0/4019573210 shutdown_connections 2023-09-30T22:30:42.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.871+0000 7fe579c14700 1 --2- 172.21.15.16:0/4019573210 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe574138d80 0x7fe57413b160 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:42.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.871+0000 7fe579c14700 1 --2- 172.21.15.16:0/4019573210 >> v2:172.21.15.16:3300/0 conn(0x7fe57409d630 0x7fe5740a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:42.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.871+0000 7fe579c14700 1 -- 172.21.15.16:0/4019573210 >> 172.21.15.16:0/4019573210 conn(0x7fe5740a3050 msgr2=0x7fe5740a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:42.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.871+0000 7fe579c14700 1 -- 172.21.15.16:0/4019573210 shutdown_connections 2023-09-30T22:30:42.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.871+0000 7fe579c14700 1 -- 172.21.15.16:0/4019573210 wait complete. 2023-09-30T22:30:42.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.872+0000 7fe579c14700 1 Processor -- start 2023-09-30T22:30:42.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.872+0000 7fe579c14700 1 -- start start 2023-09-30T22:30:42.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.873+0000 7fe579c14700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe57409d630 0x7fe5741204e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:42.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.873+0000 7fe579c14700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe5740a0a30 0x7fe574120a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:42.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.873+0000 7fe579c14700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe574138d80 0x7fe574120f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:42.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.873+0000 7fe579c14700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5741220f0 con 0x7fe57409d630 2023-09-30T22:30:42.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.873+0000 7fe579c14700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe574123340 con 0x7fe574138d80 2023-09-30T22:30:42.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.873+0000 7fe579c14700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe574123d20 con 0x7fe5740a0a30 2023-09-30T22:30:42.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.873+0000 7fe572ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe5740a0a30 0x7fe574120a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:42.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.873+0000 7fe572ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe5740a0a30 0x7fe574120a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:43726/0 (socket says 172.21.15.16:43726) 2023-09-30T22:30:42.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.873+0000 7fe572ffd700 1 -- 172.21.15.16:0/824056860 learned_addr learned my addr 172.21.15.16:0/824056860 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:42.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.873+0000 7fe5737fe700 1 --2- 172.21.15.16:0/824056860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe57409d630 0x7fe5741204e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:42.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.873+0000 7fe573fff700 1 --2- 172.21.15.16:0/824056860 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe574138d80 0x7fe574120f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:42.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.873+0000 7fe5737fe700 1 -- 172.21.15.16:0/824056860 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe5740a0a30 msgr2=0x7fe574120a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:42.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.873+0000 7fe5737fe700 1 --2- 172.21.15.16:0/824056860 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe5740a0a30 0x7fe574120a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:42.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.873+0000 7fe5737fe700 1 -- 172.21.15.16:0/824056860 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe574138d80 msgr2=0x7fe574120f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:42.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.873+0000 7fe5737fe700 1 --2- 172.21.15.16:0/824056860 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe574138d80 0x7fe574120f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:42.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.873+0000 7fe5737fe700 1 -- 172.21.15.16:0/824056860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe56c009a70 con 0x7fe57409d630 2023-09-30T22:30:42.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.873+0000 7fe573fff700 1 --2- 172.21.15.16:0/824056860 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe574138d80 0x7fe574120f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:42.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.874+0000 7fe5737fe700 1 --2- 172.21.15.16:0/824056860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe57409d630 0x7fe5741204e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe56400c490 tx=0x7fe56400c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:42.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.874+0000 7fe55bfff700 1 -- 172.21.15.16:0/824056860 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe564005d70 con 0x7fe57409d630 2023-09-30T22:30:42.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.874+0000 7fe55bfff700 1 -- 172.21.15.16:0/824056860 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5640070d0 con 0x7fe57409d630 2023-09-30T22:30:42.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.874+0000 7fe55bfff700 1 -- 172.21.15.16:0/824056860 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe564007440 con 0x7fe57409d630 2023-09-30T22:30:42.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.875+0000 7fe579c14700 1 -- 172.21.15.16:0/824056860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe574123790 con 0x7fe57409d630 2023-09-30T22:30:42.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.875+0000 7fe579c14700 1 -- 172.21.15.16:0/824056860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5740a0410 con 0x7fe57409d630 2023-09-30T22:30:42.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.876+0000 7fe55bfff700 1 -- 172.21.15.16:0/824056860 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe5640075a0 con 0x7fe57409d630 2023-09-30T22:30:42.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.876+0000 7fe579c14700 1 -- 172.21.15.16:0/824056860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5740747f0 con 0x7fe57409d630 2023-09-30T22:30:42.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.876+0000 7fe55bfff700 1 -- 172.21.15.16:0/824056860 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fe56403e6b0 con 0x7fe57409d630 2023-09-30T22:30:42.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.885+0000 7fe55bfff700 1 -- 172.21.15.16:0/824056860 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe564018af0 con 0x7fe57409d630 2023-09-30T22:30:42.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.920+0000 7fcd1d43c700 1 -- 172.21.15.16:0/2783167303 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fcd18121f70 con 0x7fcd1809d6f0 2023-09-30T22:30:42.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.920+0000 7fccff7fe700 1 -- 172.21.15.16:0/2783167303 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcd08017020 con 0x7fcd1809d6f0 2023-09-30T22:30:42.921 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:30:42.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.924+0000 7fcd1d43c700 1 -- 172.21.15.16:0/2783167303 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd1809d6f0 msgr2=0x7fcd181c80e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:42.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.924+0000 7fcd1d43c700 1 --2- 172.21.15.16:0/2783167303 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd1809d6f0 0x7fcd181c80e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcd0800c3c0 tx=0x7fcd0800c6d0 comp rx=0 tx=0).stop 2023-09-30T22:30:42.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.925+0000 7fcd1d43c700 1 -- 172.21.15.16:0/2783167303 shutdown_connections 2023-09-30T22:30:42.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.925+0000 7fcd1d43c700 1 --2- 172.21.15.16:0/2783167303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd18135e50 0x7fcd181c8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:42.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.925+0000 7fcd1d43c700 1 --2- 172.21.15.16:0/2783167303 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd1809d6f0 0x7fcd181c80e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:42.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.925+0000 7fcd1d43c700 1 --2- 172.21.15.16:0/2783167303 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd1809e1d0 0x7fcd181c8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:42.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.925+0000 7fcd1d43c700 1 -- 172.21.15.16:0/2783167303 >> 172.21.15.16:0/2783167303 conn(0x7fcd18066f50 msgr2=0x7fcd180a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:42.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.925+0000 7fcd1d43c700 1 -- 172.21.15.16:0/2783167303 shutdown_connections 2023-09-30T22:30:42.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:42.925+0000 7fcd1d43c700 1 -- 172.21.15.16:0/2783167303 wait complete. 2023-09-30T22:30:42.940 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:30:43.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:43.021+0000 7ff04aaf5700 1 -- 172.21.15.16:0/3628770424 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff0441a8bf0 con 0x7ff04409d6f0 2023-09-30T22:30:43.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:43.021+0000 7ff0417fa700 1 -- 172.21.15.16:0/3628770424 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff03802b020 con 0x7ff04409d6f0 2023-09-30T22:30:43.022 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:30:43.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:43.023+0000 7ff04aaf5700 1 -- 172.21.15.16:0/3628770424 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff04409d6f0 msgr2=0x7ff0441a5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:43.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:43.023+0000 7ff04aaf5700 1 --2- 172.21.15.16:0/3628770424 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff04409d6f0 0x7ff0441a5d90 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff0380090b0 tx=0x7ff038004750 comp rx=0 tx=0).stop 2023-09-30T22:30:43.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:43.024+0000 7ff04aaf5700 1 -- 172.21.15.16:0/3628770424 shutdown_connections 2023-09-30T22:30:43.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:43.024+0000 7ff04aaf5700 1 --2- 172.21.15.16:0/3628770424 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff04409e410 0x7ff0441a6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:43.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:43.024+0000 7ff04aaf5700 1 --2- 172.21.15.16:0/3628770424 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff04409d6f0 0x7ff0441a5d90 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:43.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:43.024+0000 7ff04aaf5700 1 --2- 172.21.15.16:0/3628770424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff04409e000 0x7ff0441a62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:43.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:43.024+0000 7ff04aaf5700 1 -- 172.21.15.16:0/3628770424 >> 172.21.15.16:0/3628770424 conn(0x7ff044066f50 msgr2=0x7ff0441bacf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:43.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:43.024+0000 7ff04aaf5700 1 -- 172.21.15.16:0/3628770424 shutdown_connections 2023-09-30T22:30:43.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:43.024+0000 7ff04aaf5700 1 -- 172.21.15.16:0/3628770424 wait complete. 2023-09-30T22:30:43.038 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:30:43.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:43.080+0000 7ff31f6cf700 1 -- 172.21.15.16:0/232819354 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff2f8005150 con 0x7ff31812cf60 2023-09-30T22:30:43.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:43.081+0000 7ff30dffb700 1 -- 172.21.15.16:0/232819354 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff31402ab50 con 0x7ff31812cf60 2023-09-30T22:30:43.081 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:30:43.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:43.083+0000 7ff31f6cf700 1 -- 172.21.15.16:0/232819354 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff31812cf60 msgr2=0x7ff3181214e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:43.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:43.083+0000 7ff31f6cf700 1 --2- 172.21.15.16:0/232819354 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff31812cf60 0x7ff3181214e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff314009130 tx=0x7ff314004750 comp rx=0 tx=0).stop 2023-09-30T22:30:43.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:43.083+0000 7ff31f6cf700 1 -- 172.21.15.16:0/232819354 shutdown_connections 2023-09-30T22:30:43.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:43.083+0000 7ff31f6cf700 1 --2- 172.21.15.16:0/232819354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff318130260 0x7ff318121f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:43.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:43.083+0000 7ff31f6cf700 1 --2- 172.21.15.16:0/232819354 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff31812cf60 0x7ff3181214e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:43.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:43.083+0000 7ff31f6cf700 1 --2- 172.21.15.16:0/232819354 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff31812f8b0 0x7ff318121a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:43.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:43.083+0000 7ff31f6cf700 1 -- 172.21.15.16:0/232819354 >> 172.21.15.16:0/232819354 conn(0x7ff3180a2aa0 msgr2=0x7ff31806e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:43.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:43.083+0000 7ff31f6cf700 1 -- 172.21.15.16:0/232819354 shutdown_connections 2023-09-30T22:30:43.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:43.083+0000 7ff31f6cf700 1 -- 172.21.15.16:0/232819354 wait complete. 2023-09-30T22:30:43.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:43.095+0000 7fe579c14700 1 -- 172.21.15.16:0/824056860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe5741239c0 con 0x7fe57409d630 2023-09-30T22:30:43.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:43.096+0000 7fe55bfff700 1 -- 172.21.15.16:0/824056860 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe564017020 con 0x7fe57409d630 2023-09-30T22:30:43.096 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:30:43.097 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:30:43.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:43.098+0000 7fe579c14700 1 -- 172.21.15.16:0/824056860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe57409d630 msgr2=0x7fe5741204e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:43.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:43.098+0000 7fe579c14700 1 --2- 172.21.15.16:0/824056860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe57409d630 0x7fe5741204e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe56400c490 tx=0x7fe56400c7a0 comp rx=0 tx=0).stop 2023-09-30T22:30:43.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:43.098+0000 7fe579c14700 1 -- 172.21.15.16:0/824056860 shutdown_connections 2023-09-30T22:30:43.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:43.098+0000 7fe579c14700 1 --2- 172.21.15.16:0/824056860 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe574138d80 0x7fe574120f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:43.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:43.098+0000 7fe579c14700 1 --2- 172.21.15.16:0/824056860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe57409d630 0x7fe5741204e0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:43.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:43.098+0000 7fe579c14700 1 --2- 172.21.15.16:0/824056860 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe5740a0a30 0x7fe574120a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:43.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:43.098+0000 7fe579c14700 1 -- 172.21.15.16:0/824056860 >> 172.21.15.16:0/824056860 conn(0x7fe5740a3050 msgr2=0x7fe57409ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:43.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:43.098+0000 7fe579c14700 1 -- 172.21.15.16:0/824056860 shutdown_connections 2023-09-30T22:30:43.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:43.098+0000 7fe579c14700 1 -- 172.21.15.16:0/824056860 wait complete. 2023-09-30T22:30:43.112 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:30:43.941 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:44.039 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:44.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.088+0000 7f7c5eda8700 1 Processor -- start 2023-09-30T22:30:44.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.088+0000 7f7c5eda8700 1 -- start start 2023-09-30T22:30:44.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.089+0000 7f7c5eda8700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7c580a1510 0x7f7c58138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:44.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.089+0000 7f7c5eda8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7c58139250 0x7f7c5813b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:44.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.089+0000 7f7c5eda8700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c5809edd0 con 0x7f7c580a0a30 2023-09-30T22:30:44.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.089+0000 7f7c5eda8700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c580a18e0 con 0x7f7c580a1510 2023-09-30T22:30:44.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.089+0000 7f7c5eda8700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5809e1f0 con 0x7f7c58139250 2023-09-30T22:30:44.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.089+0000 7f7c5d345700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7c58139250 0x7f7c5813b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:44.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.089+0000 7f7c57fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7c580a1510 0x7f7c58138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:44.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.089+0000 7f7c5d345700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7c58139250 0x7f7c5813b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:43730/0 (socket says 172.21.15.16:43730) 2023-09-30T22:30:44.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.089+0000 7f7c57fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7c580a1510 0x7f7c58138d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52024/0 (socket says 172.21.15.16:52024) 2023-09-30T22:30:44.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.089+0000 7f7c5d345700 1 -- 172.21.15.16:0/1725824480 learned_addr learned my addr 172.21.15.16:0/1725824480 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:44.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.089+0000 7f7c5d345700 1 -- 172.21.15.16:0/1725824480 >> v2:172.21.15.16:3300/0 conn(0x7f7c580a1510 msgr2=0x7f7c58138d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:44.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.089+0000 7f7c577fe700 1 -- 172.21.15.16:0/1725824480 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3760810001 0 0) 0x7f7c5809edd0 con 0x7f7c580a0a30 2023-09-30T22:30:44.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.089+0000 7f7c5d345700 1 --2- 172.21.15.16:0/1725824480 >> v2:172.21.15.16:3300/0 conn(0x7f7c580a1510 0x7f7c58138d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:44.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.089+0000 7f7c5d345700 1 -- 172.21.15.16:0/1725824480 >> v1:172.21.15.16:6789/0 conn(0x7f7c580a0a30 legacy=0x7f7c580a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:44.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.089+0000 7f7c5d345700 1 -- 172.21.15.16:0/1725824480 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c5809dc50 con 0x7f7c58139250 2023-09-30T22:30:44.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.090+0000 7f7c5d345700 1 --2- 172.21.15.16:0/1725824480 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7c58139250 0x7f7c5813b630 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7c480098f0 tx=0x7f7c48009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4439a5b738adb2b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:44.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.090+0000 7f7c577fe700 1 -- 172.21.15.16:0/1725824480 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c48011830 con 0x7f7c58139250 2023-09-30T22:30:44.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.090+0000 7f7c577fe700 1 -- 172.21.15.16:0/1725824480 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c48011990 con 0x7f7c58139250 2023-09-30T22:30:44.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.090+0000 7f7c577fe700 1 -- 172.21.15.16:0/1725824480 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c48011d20 con 0x7f7c58139250 2023-09-30T22:30:44.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.090+0000 7f7c5eda8700 1 -- 172.21.15.16:0/1725824480 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7c58139250 msgr2=0x7f7c5813b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:44.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.090+0000 7f7c5eda8700 1 --2- 172.21.15.16:0/1725824480 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7c58139250 0x7f7c5813b630 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7c480098f0 tx=0x7f7c48009c00 comp rx=0 tx=0).stop 2023-09-30T22:30:44.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.090+0000 7f7c5eda8700 1 -- 172.21.15.16:0/1725824480 shutdown_connections 2023-09-30T22:30:44.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.090+0000 7f7c5eda8700 1 --2- 172.21.15.16:0/1725824480 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7c58139250 0x7f7c5813b630 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:44.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.090+0000 7f7c5eda8700 1 --2- 172.21.15.16:0/1725824480 >> v2:172.21.15.16:3300/0 conn(0x7f7c580a1510 0x7f7c58138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:44.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.090+0000 7f7c5eda8700 1 -- 172.21.15.16:0/1725824480 >> 172.21.15.16:0/1725824480 conn(0x7f7c580a6550 msgr2=0x7f7c580a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:44.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.090+0000 7f7c5eda8700 1 -- 172.21.15.16:0/1725824480 shutdown_connections 2023-09-30T22:30:44.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.090+0000 7f7c5eda8700 1 -- 172.21.15.16:0/1725824480 wait complete. 2023-09-30T22:30:44.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.091+0000 7f7c5eda8700 1 Processor -- start 2023-09-30T22:30:44.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.091+0000 7f7c5eda8700 1 -- start start 2023-09-30T22:30:44.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.092+0000 7f7c5eda8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c580a0a30 0x7f7c5811e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:44.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.092+0000 7f7c5eda8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7c580a1510 0x7f7c5811eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:44.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.092+0000 7f7c5eda8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7c58139250 0x7f7c5811f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:44.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.092+0000 7f7c5eda8700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c58133ed0 con 0x7f7c580a0a30 2023-09-30T22:30:44.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.092+0000 7f7c5eda8700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c581344b0 con 0x7f7c580a1510 2023-09-30T22:30:44.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.092+0000 7f7c5eda8700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5809c4f0 con 0x7f7c58139250 2023-09-30T22:30:44.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.092+0000 7f7c57fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7c580a1510 0x7f7c5811eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:44.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.092+0000 7f7c5d345700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7c58139250 0x7f7c5811f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:44.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.092+0000 7f7c57fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7c580a1510 0x7f7c5811eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36988/0 (socket says 172.21.15.16:36988) 2023-09-30T22:30:44.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.092+0000 7f7c57fff700 1 -- 172.21.15.16:0/1565180029 learned_addr learned my addr 172.21.15.16:0/1565180029 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:44.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.092+0000 7f7c5cb44700 1 --2- 172.21.15.16:0/1565180029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c580a0a30 0x7f7c5811e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:44.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.093+0000 7f7c5d345700 1 -- 172.21.15.16:0/1565180029 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7c580a1510 msgr2=0x7f7c5811eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:44.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.093+0000 7f7c5d345700 1 --2- 172.21.15.16:0/1565180029 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7c580a1510 0x7f7c5811eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:44.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.093+0000 7f7c5d345700 1 -- 172.21.15.16:0/1565180029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c580a0a30 msgr2=0x7f7c5811e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:44.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.093+0000 7f7c5d345700 1 --2- 172.21.15.16:0/1565180029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c580a0a30 0x7f7c5811e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:44.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.093+0000 7f7c5d345700 1 -- 172.21.15.16:0/1565180029 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c48009550 con 0x7f7c58139250 2023-09-30T22:30:44.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.093+0000 7f7c57fff700 1 --2- 172.21.15.16:0/1565180029 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7c580a1510 0x7f7c5811eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:44.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.093+0000 7f7c5d345700 1 --2- 172.21.15.16:0/1565180029 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7c58139250 0x7f7c5811f090 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7c48016000 tx=0x7f7c48004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:44.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.093+0000 7f7c557fa700 1 -- 172.21.15.16:0/1565180029 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c48004510 con 0x7f7c58139250 2023-09-30T22:30:44.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.093+0000 7f7c557fa700 1 -- 172.21.15.16:0/1565180029 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c480036a0 con 0x7f7c58139250 2023-09-30T22:30:44.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.093+0000 7f7c557fa700 1 -- 172.21.15.16:0/1565180029 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c48003a30 con 0x7f7c58139250 2023-09-30T22:30:44.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.093+0000 7f7c5eda8700 1 -- 172.21.15.16:0/1565180029 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c58120a30 con 0x7f7c58139250 2023-09-30T22:30:44.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.093+0000 7f7c5eda8700 1 -- 172.21.15.16:0/1565180029 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c580a0220 con 0x7f7c58139250 2023-09-30T22:30:44.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.094+0000 7f7c557fa700 1 -- 172.21.15.16:0/1565180029 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7c48003b90 con 0x7f7c58139250 2023-09-30T22:30:44.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.094+0000 7f7c557fa700 1 -- 172.21.15.16:0/1565180029 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f7c480405c0 con 0x7f7c58139250 2023-09-30T22:30:44.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.094+0000 7f7c4effd700 1 -- 172.21.15.16:0/1565180029 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c300052e0 con 0x7f7c58139250 2023-09-30T22:30:44.102 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:44.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.099+0000 7f7c557fa700 1 -- 172.21.15.16:0/1565180029 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7c48015020 con 0x7f7c58139250 2023-09-30T22:30:44.114 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:44.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.184+0000 7f82721cd700 1 Processor -- start 2023-09-30T22:30:44.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.184+0000 7f82721cd700 1 -- start start 2023-09-30T22:30:44.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.184+0000 7f82721cd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f826c06a770 0x7f826c06ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:44.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.184+0000 7f82721cd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f826c06b080 0x7f826c131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:44.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.184+0000 7f82721cd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f826c131b40 0x7f826c131fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:44.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.184+0000 7f82721cd700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f826c09ece0 con 0x7f826c06b080 2023-09-30T22:30:44.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.184+0000 7f82721cd700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f826c09e760 con 0x7f826c131b40 2023-09-30T22:30:44.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.184+0000 7f82721cd700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f826c09e040 con 0x7f826c06a770 2023-09-30T22:30:44.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.185+0000 7f826b7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f826c06a770 0x7f826c06ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:44.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.185+0000 7f826b7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f826c06a770 0x7f826c06ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:43756/0 (socket says 172.21.15.16:43756) 2023-09-30T22:30:44.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.185+0000 7f826b7fe700 1 -- 172.21.15.16:0/3506391286 learned_addr learned my addr 172.21.15.16:0/3506391286 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:44.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.185+0000 7f826affd700 1 --2- 172.21.15.16:0/3506391286 >> v2:172.21.15.16:3300/0 conn(0x7f826c06b080 0x7f826c131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:44.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.185+0000 7f826affd700 1 -- 172.21.15.16:0/3506391286 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f826c06a770 msgr2=0x7f826c06ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:44.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.185+0000 7f826affd700 1 --2- 172.21.15.16:0/3506391286 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f826c06a770 0x7f826c06ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:44.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.185+0000 7f826affd700 1 -- 172.21.15.16:0/3506391286 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f826c131b40 msgr2=0x7f826c131fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:44.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.185+0000 7f826affd700 1 --2- 172.21.15.16:0/3506391286 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f826c131b40 0x7f826c131fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:44.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.185+0000 7f826affd700 1 -- 172.21.15.16:0/3506391286 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f826c09f630 con 0x7f826c06b080 2023-09-30T22:30:44.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.185+0000 7f826b7fe700 1 --2- 172.21.15.16:0/3506391286 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f826c06a770 0x7f826c06ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:44.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.185+0000 7f826affd700 1 --2- 172.21.15.16:0/3506391286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f826c06b080 0x7f826c131600 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8260009840 tx=0x7f8260009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c1810706f5fb715 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:44.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.185+0000 7f826a7fc700 1 -- 172.21.15.16:0/3506391286 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8260011830 con 0x7f826c06b080 2023-09-30T22:30:44.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.186+0000 7f826a7fc700 1 -- 172.21.15.16:0/3506391286 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8260011990 con 0x7f826c06b080 2023-09-30T22:30:44.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.186+0000 7f826a7fc700 1 -- 172.21.15.16:0/3506391286 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8260011d20 con 0x7f826c06b080 2023-09-30T22:30:44.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.186+0000 7f82721cd700 1 -- 172.21.15.16:0/3506391286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f826c06b080 msgr2=0x7f826c131600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:44.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.186+0000 7f82721cd700 1 --2- 172.21.15.16:0/3506391286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f826c06b080 0x7f826c131600 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8260009840 tx=0x7f8260009b50 comp rx=0 tx=0).stop 2023-09-30T22:30:44.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.186+0000 7f82721cd700 1 -- 172.21.15.16:0/3506391286 shutdown_connections 2023-09-30T22:30:44.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.186+0000 7f82721cd700 1 --2- 172.21.15.16:0/3506391286 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f826c131b40 0x7f826c131fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:44.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.186+0000 7f82721cd700 1 --2- 172.21.15.16:0/3506391286 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f826c06a770 0x7f826c06ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:44.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.186+0000 7f82721cd700 1 --2- 172.21.15.16:0/3506391286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f826c06b080 0x7f826c131600 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:44.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.186+0000 7f82721cd700 1 -- 172.21.15.16:0/3506391286 >> 172.21.15.16:0/3506391286 conn(0x7f826c066f50 msgr2=0x7f826c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:44.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.186+0000 7f82721cd700 1 -- 172.21.15.16:0/3506391286 shutdown_connections 2023-09-30T22:30:44.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.186+0000 7f82721cd700 1 -- 172.21.15.16:0/3506391286 wait complete. 2023-09-30T22:30:44.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.188+0000 7f82721cd700 1 Processor -- start 2023-09-30T22:30:44.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.188+0000 7f82721cd700 1 -- start start 2023-09-30T22:30:44.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.189+0000 7f82721cd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f826c06a770 0x7f826c1c8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:44.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.189+0000 7f82721cd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f826c06b080 0x7f826c1c8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:44.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.189+0000 7f82721cd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f826c131b40 0x7f826c09a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:44.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.189+0000 7f82721cd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826c1268e0 con 0x7f826c06b080 2023-09-30T22:30:44.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.189+0000 7f82721cd700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f826c131380 con 0x7f826c131b40 2023-09-30T22:30:44.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.189+0000 7f82721cd700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f826c125940 con 0x7f826c06a770 2023-09-30T22:30:44.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.189+0000 7f826b7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f826c06a770 0x7f826c1c8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:44.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.189+0000 7f826b7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f826c06a770 0x7f826c1c8700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:43762/0 (socket says 172.21.15.16:43762) 2023-09-30T22:30:44.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.189+0000 7f826b7fe700 1 -- 172.21.15.16:0/2909935531 learned_addr learned my addr 172.21.15.16:0/2909935531 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:44.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.189+0000 7f826affd700 1 --2- 172.21.15.16:0/2909935531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f826c06b080 0x7f826c1c8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:44.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.189+0000 7f826affd700 1 -- 172.21.15.16:0/2909935531 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f826c06a770 msgr2=0x7f826c1c8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:44.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.190+0000 7f826affd700 1 --2- 172.21.15.16:0/2909935531 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f826c06a770 0x7f826c1c8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:44.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.190+0000 7f826affd700 1 -- 172.21.15.16:0/2909935531 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f826c131b40 msgr2=0x7f826c09a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:44.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.190+0000 7f826affd700 1 --2- 172.21.15.16:0/2909935531 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f826c131b40 0x7f826c09a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:44.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.190+0000 7f826affd700 1 -- 172.21.15.16:0/2909935531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f825c009550 con 0x7f826c06b080 2023-09-30T22:30:44.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.190+0000 7f826b7fe700 1 --2- 172.21.15.16:0/2909935531 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f826c06a770 0x7f826c1c8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:44.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.190+0000 7f826affd700 1 --2- 172.21.15.16:0/2909935531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f826c06b080 0x7f826c1c8c40 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8260016000 tx=0x7f8260004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:44.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.190+0000 7f8253fff700 1 -- 172.21.15.16:0/2909935531 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8260004510 con 0x7f826c06b080 2023-09-30T22:30:44.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.190+0000 7f8253fff700 1 -- 172.21.15.16:0/2909935531 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f826001f070 con 0x7f826c06b080 2023-09-30T22:30:44.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.190+0000 7f8253fff700 1 -- 172.21.15.16:0/2909935531 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82600038a0 con 0x7f826c06b080 2023-09-30T22:30:44.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.190+0000 7f82721cd700 1 -- 172.21.15.16:0/2909935531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82600094a0 con 0x7f826c06b080 2023-09-30T22:30:44.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.190+0000 7f82721cd700 1 -- 172.21.15.16:0/2909935531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f826c1c9180 con 0x7f826c06b080 2023-09-30T22:30:44.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.191+0000 7f8253fff700 1 -- 172.21.15.16:0/2909935531 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8260003a00 con 0x7f826c06b080 2023-09-30T22:30:44.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.191+0000 7f8253fff700 1 -- 172.21.15.16:0/2909935531 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f8260021070 con 0x7f826c06b080 2023-09-30T22:30:44.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.191+0000 7f82721cd700 1 -- 172.21.15.16:0/2909935531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f826c074ea0 con 0x7f826c06b080 2023-09-30T22:30:44.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.201+0000 7f8253fff700 1 -- 172.21.15.16:0/2909935531 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8260015080 con 0x7f826c06b080 2023-09-30T22:30:44.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.254+0000 7fd2ae143700 1 Processor -- start 2023-09-30T22:30:44.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.254+0000 7fd2ae143700 1 -- start start 2023-09-30T22:30:44.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.255+0000 7fd2ae143700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd2a809d630 0x7fd2a80a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:44.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.255+0000 7fd2ae143700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd2a8138d80 0x7fd2a813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:44.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.255+0000 7fd2ae143700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2a80a2a70 con 0x7fd2a80a0a30 2023-09-30T22:30:44.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.255+0000 7fd2ae143700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2a80a01d0 con 0x7fd2a809d630 2023-09-30T22:30:44.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.255+0000 7fd2ae143700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a809e2b0 con 0x7fd2a8138d80 2023-09-30T22:30:44.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.255+0000 7fd2a77fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd2a80a0a30 0x7fd2a80a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58358/0 (socket says 172.21.15.16:58358) 2023-09-30T22:30:44.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.255+0000 7fd2a77fe700 1 -- 172.21.15.16:0/2198753946 learned_addr learned my addr 172.21.15.16:0/2198753946 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:30:44.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.255+0000 7fd2a7fff700 1 --2- 172.21.15.16:0/2198753946 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd2a8138d80 0x7fd2a813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:44.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.256+0000 7fd2a6ffd700 1 --2- 172.21.15.16:0/2198753946 >> v2:172.21.15.16:3300/0 conn(0x7fd2a809d630 0x7fd2a80a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:44.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.256+0000 7fd2a67fc700 1 -- 172.21.15.16:0/2198753946 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1718460225 0 0) 0x7fd2a80a2a70 con 0x7fd2a80a0a30 2023-09-30T22:30:44.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.256+0000 7fd2a67fc700 1 -- 172.21.15.16:0/2198753946 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2900035b0 con 0x7fd2a80a0a30 2023-09-30T22:30:44.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.256+0000 7fd2a7fff700 1 -- 172.21.15.16:0/2198753946 >> v2:172.21.15.16:3300/0 conn(0x7fd2a809d630 msgr2=0x7fd2a80a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:44.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.256+0000 7fd2a7fff700 1 --2- 172.21.15.16:0/2198753946 >> v2:172.21.15.16:3300/0 conn(0x7fd2a809d630 0x7fd2a80a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:44.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.256+0000 7fd2a7fff700 1 -- 172.21.15.16:0/2198753946 >> v1:172.21.15.16:6789/0 conn(0x7fd2a80a0a30 legacy=0x7fd2a80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:44.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.256+0000 7fd2a7fff700 1 -- 172.21.15.16:0/2198753946 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a809dc50 con 0x7fd2a8138d80 2023-09-30T22:30:44.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.256+0000 7fd2a7fff700 1 --2- 172.21.15.16:0/2198753946 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd2a8138d80 0x7fd2a813b160 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0009d90 tx=0x7fd2a000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e63be223320ac3d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:44.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.256+0000 7fd2a67fc700 1 -- 172.21.15.16:0/2198753946 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a000e070 con 0x7fd2a8138d80 2023-09-30T22:30:44.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.256+0000 7fd2a67fc700 1 -- 172.21.15.16:0/2198753946 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a000bb20 con 0x7fd2a8138d80 2023-09-30T22:30:44.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.256+0000 7fd2a67fc700 1 -- 172.21.15.16:0/2198753946 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a0012430 con 0x7fd2a8138d80 2023-09-30T22:30:44.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.257+0000 7fd2ae143700 1 -- 172.21.15.16:0/2198753946 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd2a8138d80 msgr2=0x7fd2a813b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:44.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.257+0000 7fd2ae143700 1 --2- 172.21.15.16:0/2198753946 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd2a8138d80 0x7fd2a813b160 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0009d90 tx=0x7fd2a000b330 comp rx=0 tx=0).stop 2023-09-30T22:30:44.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.257+0000 7fd2ae143700 1 -- 172.21.15.16:0/2198753946 shutdown_connections 2023-09-30T22:30:44.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.257+0000 7fd2ae143700 1 --2- 172.21.15.16:0/2198753946 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd2a8138d80 0x7fd2a813b160 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:44.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.257+0000 7fd2ae143700 1 --2- 172.21.15.16:0/2198753946 >> v2:172.21.15.16:3300/0 conn(0x7fd2a809d630 0x7fd2a80a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:44.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.257+0000 7fd2ae143700 1 -- 172.21.15.16:0/2198753946 >> 172.21.15.16:0/2198753946 conn(0x7fd2a80a3050 msgr2=0x7fd2a80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:44.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.257+0000 7fd2ae143700 1 -- 172.21.15.16:0/2198753946 shutdown_connections 2023-09-30T22:30:44.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.257+0000 7fd2ae143700 1 -- 172.21.15.16:0/2198753946 wait complete. 2023-09-30T22:30:44.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.258+0000 7fd2ae143700 1 Processor -- start 2023-09-30T22:30:44.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.259+0000 7fd2ae143700 1 -- start start 2023-09-30T22:30:44.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.259+0000 7fd2ae143700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2a809d630 0x7fd2a8133540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:44.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.259+0000 7fd2ae143700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd2a80a0a30 0x7fd2a8133a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:44.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.259+0000 7fd2ae143700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd2a8138d80 0x7fd2a8133fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:44.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.259+0000 7fd2ae143700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a8136d80 con 0x7fd2a809d630 2023-09-30T22:30:44.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.259+0000 7fd2ae143700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a8136060 con 0x7fd2a8138d80 2023-09-30T22:30:44.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.259+0000 7fd2ae143700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a8136410 con 0x7fd2a80a0a30 2023-09-30T22:30:44.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.259+0000 7fd2a77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2a809d630 0x7fd2a8133540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:44.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.259+0000 7fd2a6ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd2a80a0a30 0x7fd2a8133a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:44.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.259+0000 7fd2a77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2a809d630 0x7fd2a8133540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52076/0 (socket says 172.21.15.16:52076) 2023-09-30T22:30:44.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.259+0000 7fd2a77fe700 1 -- 172.21.15.16:0/1037819039 learned_addr learned my addr 172.21.15.16:0/1037819039 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:44.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.259+0000 7fd2a6ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd2a80a0a30 0x7fd2a8133a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:43790/0 (socket says 172.21.15.16:43790) 2023-09-30T22:30:44.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.259+0000 7fd2a7fff700 1 --2- 172.21.15.16:0/1037819039 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd2a8138d80 0x7fd2a8133fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:44.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.260+0000 7fd2a77fe700 1 -- 172.21.15.16:0/1037819039 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd2a80a0a30 msgr2=0x7fd2a8133a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:44.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.260+0000 7fd2a77fe700 1 --2- 172.21.15.16:0/1037819039 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd2a80a0a30 0x7fd2a8133a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:44.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.260+0000 7fd2a77fe700 1 -- 172.21.15.16:0/1037819039 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd2a8138d80 msgr2=0x7fd2a8133fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:44.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.260+0000 7fd2a77fe700 1 --2- 172.21.15.16:0/1037819039 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd2a8138d80 0x7fd2a8133fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:44.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.260+0000 7fd2a77fe700 1 -- 172.21.15.16:0/1037819039 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a0009a70 con 0x7fd2a809d630 2023-09-30T22:30:44.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.260+0000 7fd2a77fe700 1 --2- 172.21.15.16:0/1037819039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2a809d630 0x7fd2a8133540 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd29800c700 tx=0x7fd29800ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:44.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.260+0000 7fd28ffff700 1 -- 172.21.15.16:0/1037819039 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd298005d70 con 0x7fd2a809d630 2023-09-30T22:30:44.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.260+0000 7fd2ae143700 1 -- 172.21.15.16:0/1037819039 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2a813acc0 con 0x7fd2a809d630 2023-09-30T22:30:44.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.260+0000 7fd28ffff700 1 -- 172.21.15.16:0/1037819039 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2980070d0 con 0x7fd2a809d630 2023-09-30T22:30:44.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.260+0000 7fd28ffff700 1 -- 172.21.15.16:0/1037819039 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd298007460 con 0x7fd2a809d630 2023-09-30T22:30:44.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.260+0000 7fd2ae143700 1 -- 172.21.15.16:0/1037819039 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2a8137560 con 0x7fd2a809d630 2023-09-30T22:30:44.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.261+0000 7fd28ffff700 1 -- 172.21.15.16:0/1037819039 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd2980075c0 con 0x7fd2a809d630 2023-09-30T22:30:44.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.261+0000 7fd2ae143700 1 -- 172.21.15.16:0/1037819039 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2a80747f0 con 0x7fd2a809d630 2023-09-30T22:30:44.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.261+0000 7fd28ffff700 1 -- 172.21.15.16:0/1037819039 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd29803eac0 con 0x7fd2a809d630 2023-09-30T22:30:44.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.266+0000 7fd28ffff700 1 -- 172.21.15.16:0/1037819039 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd298018260 con 0x7fd2a809d630 2023-09-30T22:30:44.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.312+0000 7f7c4effd700 1 -- 172.21.15.16:0/1565180029 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f7c30005590 con 0x7f7c58139250 2023-09-30T22:30:44.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.312+0000 7f7c557fa700 1 -- 172.21.15.16:0/1565180029 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7c4801a420 con 0x7f7c58139250 2023-09-30T22:30:44.313 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:30:44.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.315+0000 7f7c4effd700 1 -- 172.21.15.16:0/1565180029 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7c58139250 msgr2=0x7f7c5811f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:44.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.315+0000 7f7c4effd700 1 --2- 172.21.15.16:0/1565180029 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7c58139250 0x7f7c5811f090 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7c48016000 tx=0x7f7c48004750 comp rx=0 tx=0).stop 2023-09-30T22:30:44.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.315+0000 7f7c4effd700 1 -- 172.21.15.16:0/1565180029 shutdown_connections 2023-09-30T22:30:44.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.315+0000 7f7c4effd700 1 --2- 172.21.15.16:0/1565180029 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7c58139250 0x7f7c5811f090 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:44.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.315+0000 7f7c4effd700 1 --2- 172.21.15.16:0/1565180029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c580a0a30 0x7f7c5811e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:44.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.315+0000 7f7c4effd700 1 --2- 172.21.15.16:0/1565180029 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7c580a1510 0x7f7c5811eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:44.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.315+0000 7f7c4effd700 1 -- 172.21.15.16:0/1565180029 >> 172.21.15.16:0/1565180029 conn(0x7f7c580a6550 msgr2=0x7f7c58136c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:44.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.316+0000 7f7c4effd700 1 -- 172.21.15.16:0/1565180029 shutdown_connections 2023-09-30T22:30:44.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.316+0000 7f7c4effd700 1 -- 172.21.15.16:0/1565180029 wait complete. 2023-09-30T22:30:44.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.330+0000 7fa0ff6b4700 1 Processor -- start 2023-09-30T22:30:44.331 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:30:44.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.330+0000 7fa0ff6b4700 1 -- start start 2023-09-30T22:30:44.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.331+0000 7fa0ff6b4700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa0f806a770 0x7fa0f806ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:44.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.331+0000 7fa0ff6b4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa0f806b080 0x7fa0f8131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:44.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.331+0000 7fa0ff6b4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa0f8131b40 0x7fa0f8131fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:44.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.331+0000 7fa0ff6b4700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0f809ece0 con 0x7fa0f8131b40 2023-09-30T22:30:44.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.331+0000 7fa0ff6b4700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f809e760 con 0x7fa0f806a770 2023-09-30T22:30:44.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.331+0000 7fa0ff6b4700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f809e040 con 0x7fa0f806b080 2023-09-30T22:30:44.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.331+0000 7fa0fcc4f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa0f806b080 0x7fa0f8131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:44.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.331+0000 7fa0fcc4f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa0f806b080 0x7fa0f8131600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:43800/0 (socket says 172.21.15.16:43800) 2023-09-30T22:30:44.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.331+0000 7fa0fcc4f700 1 -- 172.21.15.16:0/2707559663 learned_addr learned my addr 172.21.15.16:0/2707559663 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:44.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.331+0000 7fa0fd450700 1 --2- 172.21.15.16:0/2707559663 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa0f806a770 0x7fa0f806ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:44.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.331+0000 7fa0fdc51700 1 --2- 172.21.15.16:0/2707559663 >> v2:172.21.15.16:3300/0 conn(0x7fa0f8131b40 0x7fa0f8131fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:44.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.331+0000 7fa0fd450700 1 -- 172.21.15.16:0/2707559663 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa0f806b080 msgr2=0x7fa0f8131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:44.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.331+0000 7fa0fd450700 1 --2- 172.21.15.16:0/2707559663 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa0f806b080 0x7fa0f8131600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:44.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.331+0000 7fa0fd450700 1 -- 172.21.15.16:0/2707559663 >> v2:172.21.15.16:3300/0 conn(0x7fa0f8131b40 msgr2=0x7fa0f8131fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:44.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.331+0000 7fa0fd450700 1 --2- 172.21.15.16:0/2707559663 >> v2:172.21.15.16:3300/0 conn(0x7fa0f8131b40 0x7fa0f8131fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:44.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.331+0000 7fa0fd450700 1 -- 172.21.15.16:0/2707559663 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f809f630 con 0x7fa0f806a770 2023-09-30T22:30:44.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.331+0000 7fa0fcc4f700 1 --2- 172.21.15.16:0/2707559663 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa0f806b080 0x7fa0f8131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:44.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.332+0000 7fa0fd450700 1 --2- 172.21.15.16:0/2707559663 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa0f806a770 0x7fa0f806ab40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4009840 tx=0x7fa0f4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=455662883f656108 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:44.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.332+0000 7fa0effff700 1 -- 172.21.15.16:0/2707559663 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f4011830 con 0x7fa0f806a770 2023-09-30T22:30:44.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.332+0000 7fa0effff700 1 -- 172.21.15.16:0/2707559663 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0f4011990 con 0x7fa0f806a770 2023-09-30T22:30:44.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.332+0000 7fa0effff700 1 -- 172.21.15.16:0/2707559663 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f4011d20 con 0x7fa0f806a770 2023-09-30T22:30:44.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.333+0000 7fa0ff6b4700 1 -- 172.21.15.16:0/2707559663 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa0f806a770 msgr2=0x7fa0f806ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:44.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.333+0000 7fa0ff6b4700 1 --2- 172.21.15.16:0/2707559663 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa0f806a770 0x7fa0f806ab40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4009840 tx=0x7fa0f4009b50 comp rx=0 tx=0).stop 2023-09-30T22:30:44.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.333+0000 7fa0ff6b4700 1 -- 172.21.15.16:0/2707559663 shutdown_connections 2023-09-30T22:30:44.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.333+0000 7fa0ff6b4700 1 --2- 172.21.15.16:0/2707559663 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa0f806a770 0x7fa0f806ab40 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:44.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.333+0000 7fa0ff6b4700 1 --2- 172.21.15.16:0/2707559663 >> v2:172.21.15.16:3300/0 conn(0x7fa0f8131b40 0x7fa0f8131fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:44.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.333+0000 7fa0ff6b4700 1 --2- 172.21.15.16:0/2707559663 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa0f806b080 0x7fa0f8131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:44.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.333+0000 7fa0ff6b4700 1 -- 172.21.15.16:0/2707559663 >> 172.21.15.16:0/2707559663 conn(0x7fa0f8066f50 msgr2=0x7fa0f809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:44.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.333+0000 7fa0ff6b4700 1 -- 172.21.15.16:0/2707559663 shutdown_connections 2023-09-30T22:30:44.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.333+0000 7fa0ff6b4700 1 -- 172.21.15.16:0/2707559663 wait complete. 2023-09-30T22:30:44.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.334+0000 7fa0ff6b4700 1 Processor -- start 2023-09-30T22:30:44.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.334+0000 7fa0ff6b4700 1 -- start start 2023-09-30T22:30:44.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.334+0000 7fa0ff6b4700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa0f806a770 0x7fa0f81226f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:44.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.334+0000 7fa0ff6b4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0f806b080 0x7fa0f8122c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:44.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.334+0000 7fa0ff6b4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa0f8131b40 0x7fa0f8123170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:44.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.335+0000 7fa0ff6b4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f8125f00 con 0x7fa0f806b080 2023-09-30T22:30:44.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.335+0000 7fa0ff6b4700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f81260e0 con 0x7fa0f806a770 2023-09-30T22:30:44.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.335+0000 7fa0ff6b4700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f8124270 con 0x7fa0f8131b40 2023-09-30T22:30:44.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.335+0000 7fa0fd450700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa0f806a770 0x7fa0f81226f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:44.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.335+0000 7fa0fd450700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa0f806a770 0x7fa0f81226f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37020/0 (socket says 172.21.15.16:37020) 2023-09-30T22:30:44.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.335+0000 7fa0fd450700 1 -- 172.21.15.16:0/502840455 learned_addr learned my addr 172.21.15.16:0/502840455 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:44.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.335+0000 7fa0fdc51700 1 --2- 172.21.15.16:0/502840455 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa0f8131b40 0x7fa0f8123170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:44.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.335+0000 7fa0fcc4f700 1 --2- 172.21.15.16:0/502840455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0f806b080 0x7fa0f8122c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:44.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.335+0000 7fa0fd450700 1 -- 172.21.15.16:0/502840455 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa0f8131b40 msgr2=0x7fa0f8123170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:44.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.335+0000 7fa0fd450700 1 --2- 172.21.15.16:0/502840455 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa0f8131b40 0x7fa0f8123170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:44.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.335+0000 7fa0fd450700 1 -- 172.21.15.16:0/502840455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0f806b080 msgr2=0x7fa0f8122c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:44.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.335+0000 7fa0fd450700 1 --2- 172.21.15.16:0/502840455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0f806b080 0x7fa0f8122c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:44.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.335+0000 7fa0fd450700 1 -- 172.21.15.16:0/502840455 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0e8009550 con 0x7fa0f806a770 2023-09-30T22:30:44.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.335+0000 7fa0fcc4f700 1 --2- 172.21.15.16:0/502840455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0f806b080 0x7fa0f8122c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:30:44.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.335+0000 7fa0fd450700 1 --2- 172.21.15.16:0/502840455 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa0f806a770 0x7fa0f81226f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa0f40090b0 tx=0x7fa0f4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:44.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.335+0000 7fa0edffb700 1 -- 172.21.15.16:0/502840455 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f4004510 con 0x7fa0f806a770 2023-09-30T22:30:44.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.335+0000 7fa0edffb700 1 -- 172.21.15.16:0/502840455 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0f401f070 con 0x7fa0f806a770 2023-09-30T22:30:44.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.335+0000 7fa0edffb700 1 -- 172.21.15.16:0/502840455 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f40038e0 con 0x7fa0f806a770 2023-09-30T22:30:44.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.336+0000 7fa0ff6b4700 1 -- 172.21.15.16:0/502840455 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0f40094a0 con 0x7fa0f806a770 2023-09-30T22:30:44.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.336+0000 7fa0ff6b4700 1 -- 172.21.15.16:0/502840455 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0f8125b40 con 0x7fa0f806a770 2023-09-30T22:30:44.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.337+0000 7fa0edffb700 1 -- 172.21.15.16:0/502840455 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa0f4003a40 con 0x7fa0f806a770 2023-09-30T22:30:44.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.337+0000 7fa0edffb700 1 -- 172.21.15.16:0/502840455 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fa0f4021070 con 0x7fa0f806a770 2023-09-30T22:30:44.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.337+0000 7fa0ff6b4700 1 -- 172.21.15.16:0/502840455 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0f8074ea0 con 0x7fa0f806a770 2023-09-30T22:30:44.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.342+0000 7fa0edffb700 1 -- 172.21.15.16:0/502840455 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa0f4015080 con 0x7fa0f806a770 2023-09-30T22:30:44.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.463+0000 7f82721cd700 1 -- 172.21.15.16:0/2909935531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f826c1315c0 con 0x7f826c06b080 2023-09-30T22:30:44.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.463+0000 7f8253fff700 1 -- 172.21.15.16:0/2909935531 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f826002b020 con 0x7f826c06b080 2023-09-30T22:30:44.465 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:30:44.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.466+0000 7f82721cd700 1 -- 172.21.15.16:0/2909935531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f826c06b080 msgr2=0x7f826c1c8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:44.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.466+0000 7f82721cd700 1 --2- 172.21.15.16:0/2909935531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f826c06b080 0x7f826c1c8c40 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8260016000 tx=0x7f8260004750 comp rx=0 tx=0).stop 2023-09-30T22:30:44.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.466+0000 7f82721cd700 1 -- 172.21.15.16:0/2909935531 shutdown_connections 2023-09-30T22:30:44.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.466+0000 7f82721cd700 1 --2- 172.21.15.16:0/2909935531 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f826c131b40 0x7f826c09a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:44.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.466+0000 7f82721cd700 1 --2- 172.21.15.16:0/2909935531 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f826c06a770 0x7f826c1c8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:44.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.466+0000 7f82721cd700 1 --2- 172.21.15.16:0/2909935531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f826c06b080 0x7f826c1c8c40 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:44.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.466+0000 7f82721cd700 1 -- 172.21.15.16:0/2909935531 >> 172.21.15.16:0/2909935531 conn(0x7f826c066f50 msgr2=0x7f826c0a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:44.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.466+0000 7f82721cd700 1 -- 172.21.15.16:0/2909935531 shutdown_connections 2023-09-30T22:30:44.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.466+0000 7f82721cd700 1 -- 172.21.15.16:0/2909935531 wait complete. 2023-09-30T22:30:44.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.475+0000 7fd2ae143700 1 -- 172.21.15.16:0/1037819039 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd2a8072980 con 0x7fd2a809d630 2023-09-30T22:30:44.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.475+0000 7fd28ffff700 1 -- 172.21.15.16:0/1037819039 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd298017070 con 0x7fd2a809d630 2023-09-30T22:30:44.476 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:30:44.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.478+0000 7fd2ae143700 1 -- 172.21.15.16:0/1037819039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2a809d630 msgr2=0x7fd2a8133540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:44.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.478+0000 7fd2ae143700 1 --2- 172.21.15.16:0/1037819039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2a809d630 0x7fd2a8133540 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd29800c700 tx=0x7fd29800ca10 comp rx=0 tx=0).stop 2023-09-30T22:30:44.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.478+0000 7fd2ae143700 1 -- 172.21.15.16:0/1037819039 shutdown_connections 2023-09-30T22:30:44.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.478+0000 7fd2ae143700 1 --2- 172.21.15.16:0/1037819039 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd2a8138d80 0x7fd2a8133fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:44.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.478+0000 7fd2ae143700 1 --2- 172.21.15.16:0/1037819039 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd2a809d630 0x7fd2a8133540 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:44.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.478+0000 7fd2ae143700 1 --2- 172.21.15.16:0/1037819039 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd2a80a0a30 0x7fd2a8133a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:44.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.478+0000 7fd2ae143700 1 -- 172.21.15.16:0/1037819039 >> 172.21.15.16:0/1037819039 conn(0x7fd2a80a3050 msgr2=0x7fd2a80a6800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:44.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.478+0000 7fd2ae143700 1 -- 172.21.15.16:0/1037819039 shutdown_connections 2023-09-30T22:30:44.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.478+0000 7fd2ae143700 1 -- 172.21.15.16:0/1037819039 wait complete. 2023-09-30T22:30:44.480 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:30:44.492 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:30:44.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.555+0000 7fa0ff6b4700 1 -- 172.21.15.16:0/502840455 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa0f8125cd0 con 0x7fa0f806a770 2023-09-30T22:30:44.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.555+0000 7fa0edffb700 1 -- 172.21.15.16:0/502840455 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa0f401a430 con 0x7fa0f806a770 2023-09-30T22:30:44.556 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:30:44.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.557+0000 7fa0ff6b4700 1 -- 172.21.15.16:0/502840455 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa0f806a770 msgr2=0x7fa0f81226f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:44.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.557+0000 7fa0ff6b4700 1 --2- 172.21.15.16:0/502840455 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa0f806a770 0x7fa0f81226f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa0f40090b0 tx=0x7fa0f4004750 comp rx=0 tx=0).stop 2023-09-30T22:30:44.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.558+0000 7fa0ff6b4700 1 -- 172.21.15.16:0/502840455 shutdown_connections 2023-09-30T22:30:44.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.558+0000 7fa0ff6b4700 1 --2- 172.21.15.16:0/502840455 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa0f806a770 0x7fa0f81226f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:44.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.558+0000 7fa0ff6b4700 1 --2- 172.21.15.16:0/502840455 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa0f8131b40 0x7fa0f8123170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:44.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.558+0000 7fa0ff6b4700 1 --2- 172.21.15.16:0/502840455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0f806b080 0x7fa0f8122c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:44.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.558+0000 7fa0ff6b4700 1 -- 172.21.15.16:0/502840455 >> 172.21.15.16:0/502840455 conn(0x7fa0f8066f50 msgr2=0x7fa0f80a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:44.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.558+0000 7fa0ff6b4700 1 -- 172.21.15.16:0/502840455 shutdown_connections 2023-09-30T22:30:44.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:44.558+0000 7fa0ff6b4700 1 -- 172.21.15.16:0/502840455 wait complete. 2023-09-30T22:30:44.572 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:30:45.332 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:45.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.478+0000 7efe5b556700 1 Processor -- start 2023-09-30T22:30:45.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.478+0000 7efe5b556700 1 -- start start 2023-09-30T22:30:45.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.478+0000 7efe5b556700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efe5409d6f0 0x7efe5409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:45.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.478+0000 7efe5b556700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efe5409e000 0x7efe54130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:45.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.478+0000 7efe5b556700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efe5409e410 0x7efe54130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:45.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.478+0000 7efe5b556700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe5409f520 con 0x7efe5409d6f0 2023-09-30T22:30:45.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.478+0000 7efe5b556700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe5409f130 con 0x7efe5409e410 2023-09-30T22:30:45.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.478+0000 7efe5b556700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efe5409e930 con 0x7efe5409e000 2023-09-30T22:30:45.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.479+0000 7efe592f2700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efe5409d6f0 0x7efe5409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:45.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.479+0000 7efe59af3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efe5409e410 0x7efe54130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:45.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.479+0000 7efe58af1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efe5409e000 0x7efe54130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:45.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.479+0000 7efe592f2700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efe5409d6f0 0x7efe5409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52094/0 (socket says 172.21.15.16:52094) 2023-09-30T22:30:45.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.479+0000 7efe58af1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efe5409e000 0x7efe54130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:43822/0 (socket says 172.21.15.16:43822) 2023-09-30T22:30:45.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.479+0000 7efe59af3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efe5409e410 0x7efe54130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37034/0 (socket says 172.21.15.16:37034) 2023-09-30T22:30:45.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.479+0000 7efe592f2700 1 -- 172.21.15.16:0/2379681507 learned_addr learned my addr 172.21.15.16:0/2379681507 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:45.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.479+0000 7efe58af1700 1 -- 172.21.15.16:0/2379681507 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efe5409e410 msgr2=0x7efe54130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:45.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.479+0000 7efe58af1700 1 --2- 172.21.15.16:0/2379681507 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efe5409e410 0x7efe54130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.479+0000 7efe58af1700 1 -- 172.21.15.16:0/2379681507 >> v2:172.21.15.16:3300/0 conn(0x7efe5409d6f0 msgr2=0x7efe5409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:45.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.479+0000 7efe58af1700 1 --2- 172.21.15.16:0/2379681507 >> v2:172.21.15.16:3300/0 conn(0x7efe5409d6f0 0x7efe5409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.479+0000 7efe58af1700 1 -- 172.21.15.16:0/2379681507 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe5409b2d0 con 0x7efe5409e000 2023-09-30T22:30:45.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.479+0000 7efe592f2700 1 --2- 172.21.15.16:0/2379681507 >> v2:172.21.15.16:3300/0 conn(0x7efe5409d6f0 0x7efe5409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:45.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.479+0000 7efe58af1700 1 --2- 172.21.15.16:0/2379681507 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efe5409e000 0x7efe54130660 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7efe44009870 tx=0x7efe44009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ea6b2b549a442b6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:45.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.479+0000 7efe4bfff700 1 -- 172.21.15.16:0/2379681507 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe44011830 con 0x7efe5409e000 2023-09-30T22:30:45.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.479+0000 7efe4bfff700 1 -- 172.21.15.16:0/2379681507 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe44011990 con 0x7efe5409e000 2023-09-30T22:30:45.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.479+0000 7efe4bfff700 1 -- 172.21.15.16:0/2379681507 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe44011d20 con 0x7efe5409e000 2023-09-30T22:30:45.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.480+0000 7efe5b556700 1 -- 172.21.15.16:0/2379681507 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efe5409e000 msgr2=0x7efe54130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:45.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.480+0000 7efe5b556700 1 --2- 172.21.15.16:0/2379681507 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efe5409e000 0x7efe54130660 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7efe44009870 tx=0x7efe44009b80 comp rx=0 tx=0).stop 2023-09-30T22:30:45.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.480+0000 7efe5b556700 1 -- 172.21.15.16:0/2379681507 shutdown_connections 2023-09-30T22:30:45.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.480+0000 7efe5b556700 1 --2- 172.21.15.16:0/2379681507 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efe5409e410 0x7efe54130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.480+0000 7efe5b556700 1 --2- 172.21.15.16:0/2379681507 >> v2:172.21.15.16:3300/0 conn(0x7efe5409d6f0 0x7efe5409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.480+0000 7efe5b556700 1 --2- 172.21.15.16:0/2379681507 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efe5409e000 0x7efe54130660 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.480+0000 7efe5b556700 1 -- 172.21.15.16:0/2379681507 >> 172.21.15.16:0/2379681507 conn(0x7efe54066f50 msgr2=0x7efe5409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:45.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.480+0000 7efe5b556700 1 -- 172.21.15.16:0/2379681507 shutdown_connections 2023-09-30T22:30:45.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.480+0000 7efe5b556700 1 -- 172.21.15.16:0/2379681507 wait complete. 2023-09-30T22:30:45.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.481+0000 7efe5b556700 1 Processor -- start 2023-09-30T22:30:45.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.481+0000 7efe5b556700 1 -- start start 2023-09-30T22:30:45.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.482+0000 7efe5b556700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efe5409d6f0 0x7efe54122670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:45.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.482+0000 7efe5b556700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efe5409e000 0x7efe54122bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:45.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.482+0000 7efe5b556700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe5409e410 0x7efe541230f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:45.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.482+0000 7efe5b556700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe54130a50 con 0x7efe5409e410 2023-09-30T22:30:45.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.482+0000 7efe5b556700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe54126220 con 0x7efe5409d6f0 2023-09-30T22:30:45.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.482+0000 7efe5b556700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efe54130590 con 0x7efe5409e000 2023-09-30T22:30:45.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.482+0000 7efe58af1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efe5409e000 0x7efe54122bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:45.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.482+0000 7efe59af3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe5409e410 0x7efe541230f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:45.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.482+0000 7efe592f2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efe5409d6f0 0x7efe54122670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:45.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.482+0000 7efe59af3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe5409e410 0x7efe541230f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52108/0 (socket says 172.21.15.16:52108) 2023-09-30T22:30:45.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.482+0000 7efe592f2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efe5409d6f0 0x7efe54122670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37044/0 (socket says 172.21.15.16:37044) 2023-09-30T22:30:45.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.482+0000 7efe59af3700 1 -- 172.21.15.16:0/3187332565 learned_addr learned my addr 172.21.15.16:0/3187332565 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:45.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.482+0000 7efe58af1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efe5409e000 0x7efe54122bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:43826/0 (socket says 172.21.15.16:43826) 2023-09-30T22:30:45.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.482+0000 7efe58af1700 1 -- 172.21.15.16:0/3187332565 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efe5409d6f0 msgr2=0x7efe54122670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:45.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.482+0000 7efe58af1700 1 --2- 172.21.15.16:0/3187332565 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efe5409d6f0 0x7efe54122670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.482+0000 7efe58af1700 1 -- 172.21.15.16:0/3187332565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe5409e410 msgr2=0x7efe541230f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:45.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.482+0000 7efe58af1700 1 --2- 172.21.15.16:0/3187332565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe5409e410 0x7efe541230f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.482+0000 7efe58af1700 1 -- 172.21.15.16:0/3187332565 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe50009550 con 0x7efe5409e000 2023-09-30T22:30:45.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.482+0000 7efe59af3700 1 --2- 172.21.15.16:0/3187332565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe5409e410 0x7efe541230f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:45.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.482+0000 7efe58af1700 1 --2- 172.21.15.16:0/3187332565 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efe5409e000 0x7efe54122bb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efe5409f6f0 tx=0x7efe44004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:45.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.482+0000 7efe49ffb700 1 -- 172.21.15.16:0/3187332565 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe44004450 con 0x7efe5409e000 2023-09-30T22:30:45.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.482+0000 7efe49ffb700 1 -- 172.21.15.16:0/3187332565 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe4401f070 con 0x7efe5409e000 2023-09-30T22:30:45.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.482+0000 7efe49ffb700 1 -- 172.21.15.16:0/3187332565 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe440038c0 con 0x7efe5409e000 2023-09-30T22:30:45.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.482+0000 7efe5b556700 1 -- 172.21.15.16:0/3187332565 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe440094a0 con 0x7efe5409e000 2023-09-30T22:30:45.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.483+0000 7efe5b556700 1 -- 172.21.15.16:0/3187332565 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe5409c750 con 0x7efe5409e000 2023-09-30T22:30:45.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.483+0000 7efe49ffb700 1 -- 172.21.15.16:0/3187332565 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7efe44003a20 con 0x7efe5409e000 2023-09-30T22:30:45.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.484+0000 7efe49ffb700 1 -- 172.21.15.16:0/3187332565 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7efe44021070 con 0x7efe5409e000 2023-09-30T22:30:45.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.484+0000 7efe5b556700 1 -- 172.21.15.16:0/3187332565 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe54074ea0 con 0x7efe5409e000 2023-09-30T22:30:45.495 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:45.495 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:45.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.489+0000 7efe49ffb700 1 -- 172.21.15.16:0/3187332565 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7efe4402b020 con 0x7efe5409e000 2023-09-30T22:30:45.572 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:45.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.643+0000 7f113969f700 1 Processor -- start 2023-09-30T22:30:45.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.643+0000 7f113969f700 1 -- start start 2023-09-30T22:30:45.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.644+0000 7f113969f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f113409d6f0 0x7f113409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:45.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.644+0000 7f113969f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f113409e000 0x7f1134130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:45.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.644+0000 7f113969f700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f113409f130 con 0x7f113409e410 2023-09-30T22:30:45.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.644+0000 7f113969f700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f113409f520 con 0x7f113409e000 2023-09-30T22:30:45.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.644+0000 7f113969f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f113409e930 con 0x7f113409d6f0 2023-09-30T22:30:45.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.644+0000 7f1132ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f113409d6f0 0x7f113409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:45.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.644+0000 7f11327fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f113409e000 0x7f1134130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:45.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.644+0000 7f1132ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f113409d6f0 0x7f113409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37060/0 (socket says 172.21.15.16:37060) 2023-09-30T22:30:45.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.644+0000 7f1132ffd700 1 -- 172.21.15.16:0/3544508299 learned_addr learned my addr 172.21.15.16:0/3544508299 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:45.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.644+0000 7f11327fc700 1 -- 172.21.15.16:0/3544508299 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f113409d6f0 msgr2=0x7f113409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:45.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.644+0000 7f11327fc700 1 --2- 172.21.15.16:0/3544508299 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f113409d6f0 0x7f113409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.644+0000 7f11327fc700 1 -- 172.21.15.16:0/3544508299 >> v1:172.21.15.16:6789/0 conn(0x7f113409e410 legacy=0x7f1134130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:45.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.644+0000 7f1131ffb700 1 -- 172.21.15.16:0/3544508299 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3412837856 0 0) 0x7f113409f130 con 0x7f113409e410 2023-09-30T22:30:45.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.644+0000 7f11327fc700 1 -- 172.21.15.16:0/3544508299 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f113409b2d0 con 0x7f113409e000 2023-09-30T22:30:45.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.644+0000 7f1132ffd700 1 --2- 172.21.15.16:0/3544508299 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f113409d6f0 0x7f113409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:45.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.645+0000 7f11327fc700 1 --2- 172.21.15.16:0/3544508299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f113409e000 0x7f1134130660 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f112800dc60 tx=0x7f112800df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=724d7fe6b2df40c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:45.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.645+0000 7f1131ffb700 1 -- 172.21.15.16:0/3544508299 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1128012070 con 0x7f113409e000 2023-09-30T22:30:45.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.645+0000 7f1131ffb700 1 -- 172.21.15.16:0/3544508299 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f112800d2e0 con 0x7f113409e000 2023-09-30T22:30:45.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.645+0000 7f1131ffb700 1 -- 172.21.15.16:0/3544508299 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112800f900 con 0x7f113409e000 2023-09-30T22:30:45.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.645+0000 7f113969f700 1 -- 172.21.15.16:0/3544508299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f113409e000 msgr2=0x7f1134130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:45.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.645+0000 7f113969f700 1 --2- 172.21.15.16:0/3544508299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f113409e000 0x7f1134130660 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f112800dc60 tx=0x7f112800df70 comp rx=0 tx=0).stop 2023-09-30T22:30:45.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.645+0000 7f113969f700 1 -- 172.21.15.16:0/3544508299 shutdown_connections 2023-09-30T22:30:45.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.645+0000 7f113969f700 1 --2- 172.21.15.16:0/3544508299 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f113409d6f0 0x7f113409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.645+0000 7f113969f700 1 --2- 172.21.15.16:0/3544508299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f113409e000 0x7f1134130660 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.645+0000 7f113969f700 1 -- 172.21.15.16:0/3544508299 >> 172.21.15.16:0/3544508299 conn(0x7f1134066f50 msgr2=0x7f113409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:45.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.646+0000 7f113969f700 1 -- 172.21.15.16:0/3544508299 shutdown_connections 2023-09-30T22:30:45.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.646+0000 7f113969f700 1 -- 172.21.15.16:0/3544508299 wait complete. 2023-09-30T22:30:45.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.647+0000 7f113969f700 1 Processor -- start 2023-09-30T22:30:45.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.647+0000 7f113969f700 1 -- start start 2023-09-30T22:30:45.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.647+0000 7f113969f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f113409d6f0 0x7f113411ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:45.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.647+0000 7f113969f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f113409e000 0x7f113411e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:45.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.647+0000 7f113969f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f113409e410 0x7f113411e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:45.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.647+0000 7f113969f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f113411fa00 con 0x7f113409e000 2023-09-30T22:30:45.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.647+0000 7f113969f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1134120c50 con 0x7f113409e410 2023-09-30T22:30:45.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.647+0000 7f113969f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f113411ffe0 con 0x7f113409d6f0 2023-09-30T22:30:45.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.647+0000 7f11327fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f113409e000 0x7f113411e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:45.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.647+0000 7f11327fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f113409e000 0x7f113411e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52112/0 (socket says 172.21.15.16:52112) 2023-09-30T22:30:45.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.647+0000 7f11327fc700 1 -- 172.21.15.16:0/387833801 learned_addr learned my addr 172.21.15.16:0/387833801 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:45.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.648+0000 7f11337fe700 1 --2- 172.21.15.16:0/387833801 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f113409e410 0x7f113411e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:45.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.648+0000 7f1132ffd700 1 --2- 172.21.15.16:0/387833801 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f113409d6f0 0x7f113411ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:45.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.648+0000 7f11337fe700 1 -- 172.21.15.16:0/387833801 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f113409d6f0 msgr2=0x7f113411ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:45.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.648+0000 7f11337fe700 1 --2- 172.21.15.16:0/387833801 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f113409d6f0 0x7f113411ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.648+0000 7f11337fe700 1 -- 172.21.15.16:0/387833801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f113409e000 msgr2=0x7f113411e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:45.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.648+0000 7f11337fe700 1 --2- 172.21.15.16:0/387833801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f113409e000 0x7f113411e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.648+0000 7f11337fe700 1 -- 172.21.15.16:0/387833801 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f112400b600 con 0x7f113409e410 2023-09-30T22:30:45.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.648+0000 7f11327fc700 1 --2- 172.21.15.16:0/387833801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f113409e000 0x7f113411e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:45.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.648+0000 7f11337fe700 1 --2- 172.21.15.16:0/387833801 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f113409e410 0x7f113411e870 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f112c00c7c0 tx=0x7f112c00cad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:45.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.648+0000 7f111b7fe700 1 -- 172.21.15.16:0/387833801 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112c003e30 con 0x7f113409e410 2023-09-30T22:30:45.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.648+0000 7f111b7fe700 1 -- 172.21.15.16:0/387833801 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f112c005070 con 0x7f113409e410 2023-09-30T22:30:45.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.648+0000 7f113969f700 1 -- 172.21.15.16:0/387833801 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f112800d940 con 0x7f113409e410 2023-09-30T22:30:45.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.649+0000 7f113969f700 1 -- 172.21.15.16:0/387833801 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11340a2120 con 0x7f113409e410 2023-09-30T22:30:45.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.649+0000 7f111b7fe700 1 -- 172.21.15.16:0/387833801 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112c005420 con 0x7f113409e410 2023-09-30T22:30:45.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.649+0000 7f111b7fe700 1 -- 172.21.15.16:0/387833801 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f112c0055c0 con 0x7f113409e410 2023-09-30T22:30:45.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.650+0000 7f111b7fe700 1 -- 172.21.15.16:0/387833801 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f112c03eee0 con 0x7f113409e410 2023-09-30T22:30:45.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.650+0000 7f113969f700 1 -- 172.21.15.16:0/387833801 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1134121590 con 0x7f113409e410 2023-09-30T22:30:45.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.650+0000 7fa0f7974700 1 Processor -- start 2023-09-30T22:30:45.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.651+0000 7fa0f7974700 1 -- start start 2023-09-30T22:30:45.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.651+0000 7fa0f7974700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa0f01337f0 0x7fa0f0138880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:45.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.651+0000 7fa0f7974700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa0f0138dc0 0x7fa0f013b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:45.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.651+0000 7fa0f7974700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0f006b990 con 0x7fa0f0133420 2023-09-30T22:30:45.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.651+0000 7fa0f7974700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f009b0e0 con 0x7fa0f01337f0 2023-09-30T22:30:45.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.651+0000 7fa0f7974700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f0133c20 con 0x7fa0f0138dc0 2023-09-30T22:30:45.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.651+0000 7fa0f5f11700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa0f0138dc0 0x7fa0f013b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:45.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.651+0000 7fa0f5f11700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa0f0138dc0 0x7fa0f013b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:43850/0 (socket says 172.21.15.16:43850) 2023-09-30T22:30:45.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.652+0000 7fa0f5f11700 1 -- 172.21.15.16:0/3851994456 learned_addr learned my addr 172.21.15.16:0/3851994456 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:45.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.651+0000 7fa0f4f0f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa0f01337f0 0x7fa0f0138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:45.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.652+0000 7fa0f4f0f700 1 -- 172.21.15.16:0/3851994456 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa0f0138dc0 msgr2=0x7fa0f013b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:45.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.652+0000 7fa0f4f0f700 1 --2- 172.21.15.16:0/3851994456 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa0f0138dc0 0x7fa0f013b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.652+0000 7fa0f4f0f700 1 -- 172.21.15.16:0/3851994456 >> v1:172.21.15.16:6789/0 conn(0x7fa0f0133420 legacy=0x7fa0f0133e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:45.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.652+0000 7fa0f4f0f700 1 -- 172.21.15.16:0/3851994456 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f00a0590 con 0x7fa0f01337f0 2023-09-30T22:30:45.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.652+0000 7fa0f4f0f700 1 --2- 172.21.15.16:0/3851994456 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa0f01337f0 0x7fa0f0138880 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec009870 tx=0x7fa0ec009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9cb2f77ae09f30d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:45.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.656+0000 7fa0e3fff700 1 -- 172.21.15.16:0/3851994456 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec011830 con 0x7fa0f01337f0 2023-09-30T22:30:45.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.656+0000 7fa0e3fff700 1 -- 172.21.15.16:0/3851994456 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0ec011990 con 0x7fa0f01337f0 2023-09-30T22:30:45.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.656+0000 7fa0e3fff700 1 -- 172.21.15.16:0/3851994456 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec011d20 con 0x7fa0f01337f0 2023-09-30T22:30:45.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.656+0000 7f111b7fe700 1 -- 172.21.15.16:0/387833801 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f112c017070 con 0x7f113409e410 2023-09-30T22:30:45.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.656+0000 7fa0f7974700 1 -- 172.21.15.16:0/3851994456 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa0f01337f0 msgr2=0x7fa0f0138880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:45.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.656+0000 7fa0f7974700 1 --2- 172.21.15.16:0/3851994456 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa0f01337f0 0x7fa0f0138880 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec009870 tx=0x7fa0ec009b80 comp rx=0 tx=0).stop 2023-09-30T22:30:45.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.657+0000 7fa0f7974700 1 -- 172.21.15.16:0/3851994456 shutdown_connections 2023-09-30T22:30:45.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.657+0000 7fa0f7974700 1 --2- 172.21.15.16:0/3851994456 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa0f0138dc0 0x7fa0f013b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.657+0000 7fa0f7974700 1 --2- 172.21.15.16:0/3851994456 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa0f01337f0 0x7fa0f0138880 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.657+0000 7fa0f7974700 1 -- 172.21.15.16:0/3851994456 >> 172.21.15.16:0/3851994456 conn(0x7fa0f00a4210 msgr2=0x7fa0f00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:45.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.657+0000 7fa0f7974700 1 -- 172.21.15.16:0/3851994456 shutdown_connections 2023-09-30T22:30:45.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.657+0000 7fa0f7974700 1 -- 172.21.15.16:0/3851994456 wait complete. 2023-09-30T22:30:45.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.658+0000 7fa0f7974700 1 Processor -- start 2023-09-30T22:30:45.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.658+0000 7fa0f7974700 1 -- start start 2023-09-30T22:30:45.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.658+0000 7fa0f7974700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa0f0133420 0x7fa0f012f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:45.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.658+0000 7fa0f7974700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa0f01337f0 0x7fa0f012fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:45.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.658+0000 7fa0f7974700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0f0138dc0 0x7fa0f012ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:45.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.658+0000 7fa0f7974700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f01388a0 con 0x7fa0f0138dc0 2023-09-30T22:30:45.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.658+0000 7fa0f7974700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f009b740 con 0x7fa0f01337f0 2023-09-30T22:30:45.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.658+0000 7fa0f7974700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f009cc20 con 0x7fa0f0133420 2023-09-30T22:30:45.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.658+0000 7fa0f5710700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa0f0133420 0x7fa0f012f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:45.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.658+0000 7fa0f5710700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa0f0133420 0x7fa0f012f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:43852/0 (socket says 172.21.15.16:43852) 2023-09-30T22:30:45.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.659+0000 7fa0f5710700 1 -- 172.21.15.16:0/1567930409 learned_addr learned my addr 172.21.15.16:0/1567930409 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:45.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.659+0000 7fa0f4f0f700 1 --2- 172.21.15.16:0/1567930409 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa0f01337f0 0x7fa0f012fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:45.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.659+0000 7fa0f5f11700 1 --2- 172.21.15.16:0/1567930409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0f0138dc0 0x7fa0f012ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:45.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.659+0000 7fa0f5710700 1 -- 172.21.15.16:0/1567930409 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa0f01337f0 msgr2=0x7fa0f012fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:45.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.659+0000 7fa0f5710700 1 --2- 172.21.15.16:0/1567930409 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa0f01337f0 0x7fa0f012fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.659+0000 7fa0f5710700 1 -- 172.21.15.16:0/1567930409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0f0138dc0 msgr2=0x7fa0f012ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:45.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.659+0000 7fa0f5710700 1 --2- 172.21.15.16:0/1567930409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0f0138dc0 0x7fa0f012ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.659+0000 7fa0f5710700 1 -- 172.21.15.16:0/1567930409 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0ec0094a0 con 0x7fa0f0133420 2023-09-30T22:30:45.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.659+0000 7fa0f4f0f700 1 --2- 172.21.15.16:0/1567930409 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa0f01337f0 0x7fa0f012fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:45.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.659+0000 7fa0f5f11700 1 --2- 172.21.15.16:0/1567930409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0f0138dc0 0x7fa0f012ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:45.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.659+0000 7fa0f5710700 1 --2- 172.21.15.16:0/1567930409 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa0f0133420 0x7fa0f012f4c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa0e400c4e0 tx=0x7fa0e400c7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:45.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.659+0000 7fa0e1ffb700 1 -- 172.21.15.16:0/1567930409 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e40052d0 con 0x7fa0f0133420 2023-09-30T22:30:45.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.660+0000 7fa0e1ffb700 1 -- 172.21.15.16:0/1567930409 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0e4005430 con 0x7fa0f0133420 2023-09-30T22:30:45.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.660+0000 7fa0f7974700 1 -- 172.21.15.16:0/1567930409 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0f009baa0 con 0x7fa0f0133420 2023-09-30T22:30:45.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.660+0000 7fa0e1ffb700 1 -- 172.21.15.16:0/1567930409 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e40038c0 con 0x7fa0f0133420 2023-09-30T22:30:45.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.660+0000 7fa0f7974700 1 -- 172.21.15.16:0/1567930409 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0f009b4f0 con 0x7fa0f0133420 2023-09-30T22:30:45.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.661+0000 7fa0e1ffb700 1 -- 172.21.15.16:0/1567930409 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa0e4003a20 con 0x7fa0f0133420 2023-09-30T22:30:45.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.661+0000 7fa0e1ffb700 1 -- 172.21.15.16:0/1567930409 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fa0e403eb50 con 0x7fa0f0133420 2023-09-30T22:30:45.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.661+0000 7fa0f7974700 1 -- 172.21.15.16:0/1567930409 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0f0074ea0 con 0x7fa0f0133420 2023-09-30T22:30:45.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.666+0000 7fa0e1ffb700 1 -- 172.21.15.16:0/1567930409 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa0e4017020 con 0x7fa0f0133420 2023-09-30T22:30:45.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.721+0000 7f91e24aa700 1 Processor -- start 2023-09-30T22:30:45.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.721+0000 7f91e24aa700 1 -- start start 2023-09-30T22:30:45.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.722+0000 7f91e24aa700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f91dc09d6f0 0x7f91dc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:45.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.722+0000 7f91e24aa700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f91dc09e410 0x7f91dc130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:45.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.722+0000 7f91e24aa700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91dc09f130 con 0x7f91dc09e000 2023-09-30T22:30:45.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.722+0000 7f91e24aa700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91dc09f520 con 0x7f91dc09d6f0 2023-09-30T22:30:45.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.722+0000 7f91e24aa700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91dc09e930 con 0x7f91dc09e410 2023-09-30T22:30:45.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.722+0000 7f91db7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f91dc09e000 0x7f91dc130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58378/0 (socket says 172.21.15.16:58378) 2023-09-30T22:30:45.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.722+0000 7f91db7fe700 1 -- 172.21.15.16:0/376124612 learned_addr learned my addr 172.21.15.16:0/376124612 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:30:45.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.722+0000 7f91dbfff700 1 --2- 172.21.15.16:0/376124612 >> v2:172.21.15.16:3300/0 conn(0x7f91dc09d6f0 0x7f91dc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:45.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.722+0000 7f91e0a47700 1 --2- 172.21.15.16:0/376124612 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f91dc09e410 0x7f91dc130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:45.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.722+0000 7f91daffd700 1 -- 172.21.15.16:0/376124612 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3092660347 0 0) 0x7f91dc09f130 con 0x7f91dc09e000 2023-09-30T22:30:45.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.722+0000 7f91daffd700 1 -- 172.21.15.16:0/376124612 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91c40057c0 con 0x7f91dc09e000 2023-09-30T22:30:45.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.722+0000 7f91e0a47700 1 -- 172.21.15.16:0/376124612 >> v2:172.21.15.16:3300/0 conn(0x7f91dc09d6f0 msgr2=0x7f91dc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:45.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.722+0000 7f91e0a47700 1 --2- 172.21.15.16:0/376124612 >> v2:172.21.15.16:3300/0 conn(0x7f91dc09d6f0 0x7f91dc09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.722+0000 7f91e0a47700 1 -- 172.21.15.16:0/376124612 >> v1:172.21.15.16:6789/0 conn(0x7f91dc09e000 legacy=0x7f91dc130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:45.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.722+0000 7f91e0a47700 1 -- 172.21.15.16:0/376124612 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91dc09b2d0 con 0x7f91dc09e410 2023-09-30T22:30:45.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.723+0000 7f91e0a47700 1 --2- 172.21.15.16:0/376124612 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f91dc09e410 0x7f91dc130d70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f91d4009d90 tx=0x7f91d400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c38b65b0db580f96 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:45.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.723+0000 7f91daffd700 1 -- 172.21.15.16:0/376124612 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d400e070 con 0x7f91dc09e410 2023-09-30T22:30:45.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.723+0000 7f91daffd700 1 -- 172.21.15.16:0/376124612 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d400bb20 con 0x7f91dc09e410 2023-09-30T22:30:45.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.723+0000 7f91daffd700 1 -- 172.21.15.16:0/376124612 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d4012430 con 0x7f91dc09e410 2023-09-30T22:30:45.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.723+0000 7f91e24aa700 1 -- 172.21.15.16:0/376124612 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f91dc09e410 msgr2=0x7f91dc130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:45.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.723+0000 7f91e24aa700 1 --2- 172.21.15.16:0/376124612 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f91dc09e410 0x7f91dc130d70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f91d4009d90 tx=0x7f91d400b330 comp rx=0 tx=0).stop 2023-09-30T22:30:45.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.724+0000 7f91e24aa700 1 -- 172.21.15.16:0/376124612 shutdown_connections 2023-09-30T22:30:45.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.724+0000 7f91e24aa700 1 --2- 172.21.15.16:0/376124612 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f91dc09e410 0x7f91dc130d70 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.724+0000 7f91e24aa700 1 --2- 172.21.15.16:0/376124612 >> v2:172.21.15.16:3300/0 conn(0x7f91dc09d6f0 0x7f91dc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.724+0000 7f91e24aa700 1 -- 172.21.15.16:0/376124612 >> 172.21.15.16:0/376124612 conn(0x7f91dc066f50 msgr2=0x7f91dc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:45.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.724+0000 7f91e24aa700 1 -- 172.21.15.16:0/376124612 shutdown_connections 2023-09-30T22:30:45.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.724+0000 7f91e24aa700 1 -- 172.21.15.16:0/376124612 wait complete. 2023-09-30T22:30:45.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.725+0000 7f91e24aa700 1 Processor -- start 2023-09-30T22:30:45.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.725+0000 7f91e24aa700 1 -- start start 2023-09-30T22:30:45.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.725+0000 7f91e24aa700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91dc09d6f0 0x7f91dc1220a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:45.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.725+0000 7f91e24aa700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f91dc09e000 0x7f91dc1225e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:45.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.726+0000 7f91e24aa700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f91dc09e410 0x7f91dc122b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:45.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.726+0000 7f91e24aa700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91dc124300 con 0x7f91dc09d6f0 2023-09-30T22:30:45.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.726+0000 7f91e24aa700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91dc0a3c70 con 0x7f91dc09e000 2023-09-30T22:30:45.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.726+0000 7f91e24aa700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f91dc125350 con 0x7f91dc09e410 2023-09-30T22:30:45.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.726+0000 7f91db7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f91dc09e000 0x7f91dc1225e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:45.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.726+0000 7f91db7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f91dc09e000 0x7f91dc1225e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37102/0 (socket says 172.21.15.16:37102) 2023-09-30T22:30:45.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.726+0000 7f91db7fe700 1 -- 172.21.15.16:0/1001705502 learned_addr learned my addr 172.21.15.16:0/1001705502 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:45.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.726+0000 7f91dbfff700 1 --2- 172.21.15.16:0/1001705502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91dc09d6f0 0x7f91dc1220a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:45.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.726+0000 7f91e0a47700 1 --2- 172.21.15.16:0/1001705502 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f91dc09e410 0x7f91dc122b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:45.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.726+0000 7f91dbfff700 1 -- 172.21.15.16:0/1001705502 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f91dc09e410 msgr2=0x7f91dc122b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:45.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.726+0000 7f91dbfff700 1 --2- 172.21.15.16:0/1001705502 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f91dc09e410 0x7f91dc122b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.726+0000 7f91dbfff700 1 -- 172.21.15.16:0/1001705502 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f91dc09e000 msgr2=0x7f91dc1225e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:45.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.726+0000 7f91dbfff700 1 --2- 172.21.15.16:0/1001705502 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f91dc09e000 0x7f91dc1225e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.726+0000 7f91dbfff700 1 -- 172.21.15.16:0/1001705502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d4009a70 con 0x7f91dc09d6f0 2023-09-30T22:30:45.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.726+0000 7f91db7fe700 1 --2- 172.21.15.16:0/1001705502 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f91dc09e000 0x7f91dc1225e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:45.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.726+0000 7f91dbfff700 1 --2- 172.21.15.16:0/1001705502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91dc09d6f0 0x7f91dc1220a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f91cc00e900 tx=0x7f91cc00ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:45.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.727+0000 7f91d8ff9700 1 -- 172.21.15.16:0/1001705502 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc013680 con 0x7f91dc09d6f0 2023-09-30T22:30:45.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.727+0000 7f91d8ff9700 1 -- 172.21.15.16:0/1001705502 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91cc0137e0 con 0x7f91dc09d6f0 2023-09-30T22:30:45.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.727+0000 7f91e24aa700 1 -- 172.21.15.16:0/1001705502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91dc125c60 con 0x7f91dc09d6f0 2023-09-30T22:30:45.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.727+0000 7f91d8ff9700 1 -- 172.21.15.16:0/1001705502 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc013b50 con 0x7f91dc09d6f0 2023-09-30T22:30:45.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.727+0000 7f91e24aa700 1 -- 172.21.15.16:0/1001705502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91dc1258e0 con 0x7f91dc09d6f0 2023-09-30T22:30:45.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.728+0000 7f91d8ff9700 1 -- 172.21.15.16:0/1001705502 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f91cc013cb0 con 0x7f91dc09d6f0 2023-09-30T22:30:45.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.728+0000 7f91d8ff9700 1 -- 172.21.15.16:0/1001705502 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f91cc03f8f0 con 0x7f91dc09d6f0 2023-09-30T22:30:45.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.728+0000 7f91e24aa700 1 -- 172.21.15.16:0/1001705502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91dc11ac20 con 0x7f91dc09d6f0 2023-09-30T22:30:45.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.732+0000 7efe5b556700 1 -- 172.21.15.16:0/3187332565 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7efe54126980 con 0x7efe5409e000 2023-09-30T22:30:45.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.733+0000 7efe49ffb700 1 -- 172.21.15.16:0/3187332565 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7efe44015070 con 0x7efe5409e000 2023-09-30T22:30:45.733 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:30:45.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.736+0000 7efe5b556700 1 -- 172.21.15.16:0/3187332565 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efe5409e000 msgr2=0x7efe54122bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:45.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.736+0000 7efe5b556700 1 --2- 172.21.15.16:0/3187332565 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efe5409e000 0x7efe54122bb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7efe5409f6f0 tx=0x7efe44004750 comp rx=0 tx=0).stop 2023-09-30T22:30:45.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.736+0000 7efe5b556700 1 -- 172.21.15.16:0/3187332565 shutdown_connections 2023-09-30T22:30:45.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.736+0000 7efe5b556700 1 --2- 172.21.15.16:0/3187332565 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efe5409d6f0 0x7efe54122670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.736+0000 7efe5b556700 1 --2- 172.21.15.16:0/3187332565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efe5409e410 0x7efe541230f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.736+0000 7efe5b556700 1 --2- 172.21.15.16:0/3187332565 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efe5409e000 0x7efe54122bb0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.736+0000 7efe5b556700 1 -- 172.21.15.16:0/3187332565 >> 172.21.15.16:0/3187332565 conn(0x7efe54066f50 msgr2=0x7efe5406a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:45.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.736+0000 7efe5b556700 1 -- 172.21.15.16:0/3187332565 shutdown_connections 2023-09-30T22:30:45.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.736+0000 7efe5b556700 1 -- 172.21.15.16:0/3187332565 wait complete. 2023-09-30T22:30:45.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.738+0000 7f91d8ff9700 1 -- 172.21.15.16:0/1001705502 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f91cc01b590 con 0x7f91dc09d6f0 2023-09-30T22:30:45.758 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:30:45.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.880+0000 7fa0f7974700 1 -- 172.21.15.16:0/1567930409 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa0f0132e80 con 0x7fa0f0133420 2023-09-30T22:30:45.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.880+0000 7fa0e1ffb700 1 -- 172.21.15.16:0/1567930409 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa0e4003cd0 con 0x7fa0f0133420 2023-09-30T22:30:45.881 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:30:45.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.882+0000 7fa0f7974700 1 -- 172.21.15.16:0/1567930409 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa0f0133420 msgr2=0x7fa0f012f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:45.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.882+0000 7fa0f7974700 1 --2- 172.21.15.16:0/1567930409 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa0f0133420 0x7fa0f012f4c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa0e400c4e0 tx=0x7fa0e400c7f0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.882+0000 7fa0f7974700 1 -- 172.21.15.16:0/1567930409 shutdown_connections 2023-09-30T22:30:45.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.882+0000 7fa0f7974700 1 --2- 172.21.15.16:0/1567930409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0f0138dc0 0x7fa0f012ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.882+0000 7fa0f7974700 1 --2- 172.21.15.16:0/1567930409 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa0f0133420 0x7fa0f012f4c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.882+0000 7fa0f7974700 1 --2- 172.21.15.16:0/1567930409 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa0f01337f0 0x7fa0f012fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.882+0000 7fa0f7974700 1 -- 172.21.15.16:0/1567930409 >> 172.21.15.16:0/1567930409 conn(0x7fa0f00a4210 msgr2=0x7fa0f0131400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:45.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.883+0000 7fa0f7974700 1 -- 172.21.15.16:0/1567930409 shutdown_connections 2023-09-30T22:30:45.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.883+0000 7fa0f7974700 1 -- 172.21.15.16:0/1567930409 wait complete. 2023-09-30T22:30:45.898 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:30:45.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.898+0000 7f113969f700 1 -- 172.21.15.16:0/387833801 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f113411ac20 con 0x7f113409e410 2023-09-30T22:30:45.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.899+0000 7f111b7fe700 1 -- 172.21.15.16:0/387833801 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f112c017030 con 0x7f113409e410 2023-09-30T22:30:45.899 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:30:45.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.901+0000 7f113969f700 1 -- 172.21.15.16:0/387833801 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f113409e410 msgr2=0x7f113411e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:45.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.901+0000 7f113969f700 1 --2- 172.21.15.16:0/387833801 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f113409e410 0x7f113411e870 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f112c00c7c0 tx=0x7f112c00cad0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.901+0000 7f113969f700 1 -- 172.21.15.16:0/387833801 shutdown_connections 2023-09-30T22:30:45.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.901+0000 7f113969f700 1 --2- 172.21.15.16:0/387833801 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f113409e410 0x7f113411e870 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.901+0000 7f113969f700 1 --2- 172.21.15.16:0/387833801 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f113409d6f0 0x7f113411ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.901+0000 7f113969f700 1 --2- 172.21.15.16:0/387833801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f113409e000 0x7f113411e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.901+0000 7f113969f700 1 -- 172.21.15.16:0/387833801 >> 172.21.15.16:0/387833801 conn(0x7f1134066f50 msgr2=0x7f1134130a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:45.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.901+0000 7f113969f700 1 -- 172.21.15.16:0/387833801 shutdown_connections 2023-09-30T22:30:45.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.901+0000 7f113969f700 1 -- 172.21.15.16:0/387833801 wait complete. 2023-09-30T22:30:45.915 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:30:45.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.947+0000 7f91e24aa700 1 -- 172.21.15.16:0/1001705502 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f91dc074ea0 con 0x7f91dc09d6f0 2023-09-30T22:30:45.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.948+0000 7f91d8ff9700 1 -- 172.21.15.16:0/1001705502 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f91cc017020 con 0x7f91dc09d6f0 2023-09-30T22:30:45.948 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:30:45.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.950+0000 7f91e24aa700 1 -- 172.21.15.16:0/1001705502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91dc09d6f0 msgr2=0x7f91dc1220a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:45.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.950+0000 7f91e24aa700 1 --2- 172.21.15.16:0/1001705502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91dc09d6f0 0x7f91dc1220a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f91cc00e900 tx=0x7f91cc00ec10 comp rx=0 tx=0).stop 2023-09-30T22:30:45.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.950+0000 7f91e24aa700 1 -- 172.21.15.16:0/1001705502 shutdown_connections 2023-09-30T22:30:45.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.950+0000 7f91e24aa700 1 --2- 172.21.15.16:0/1001705502 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f91dc09e410 0x7f91dc122b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.950+0000 7f91e24aa700 1 --2- 172.21.15.16:0/1001705502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91dc09d6f0 0x7f91dc1220a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.950+0000 7f91e24aa700 1 --2- 172.21.15.16:0/1001705502 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f91dc09e000 0x7f91dc1225e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:45.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.950+0000 7f91e24aa700 1 -- 172.21.15.16:0/1001705502 >> 172.21.15.16:0/1001705502 conn(0x7f91dc066f50 msgr2=0x7f91dc130800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:45.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.950+0000 7f91e24aa700 1 -- 172.21.15.16:0/1001705502 shutdown_connections 2023-09-30T22:30:45.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:45.950+0000 7f91e24aa700 1 -- 172.21.15.16:0/1001705502 wait complete. 2023-09-30T22:30:45.964 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:30:46.759 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:46.899 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:46.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.905+0000 7f98f1645700 1 Processor -- start 2023-09-30T22:30:46.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.905+0000 7f98f1645700 1 -- start start 2023-09-30T22:30:46.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.906+0000 7f98f1645700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98ec09d6f0 0x7f98ec09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:46.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.906+0000 7f98f1645700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f98ec09e000 0x7f98ec1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:46.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.906+0000 7f98f1645700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f98ec09e410 0x7f98ec133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:46.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.906+0000 7f98f1645700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98ec06b460 con 0x7f98ec09e000 2023-09-30T22:30:46.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.906+0000 7f98f1645700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98ec09b120 con 0x7f98ec09e410 2023-09-30T22:30:46.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.906+0000 7f98f1645700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f98ec09a7e0 con 0x7f98ec09d6f0 2023-09-30T22:30:46.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.906+0000 7f98ea7fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f98ec09e000 0x7f98ec1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:46.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.906+0000 7f98ea7fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f98ec09e000 0x7f98ec1339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52144/0 (socket says 172.21.15.16:52144) 2023-09-30T22:30:46.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.906+0000 7f98ea7fc700 1 -- 172.21.15.16:0/4291824136 learned_addr learned my addr 172.21.15.16:0/4291824136 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:46.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.906+0000 7f98eaffd700 1 --2- 172.21.15.16:0/4291824136 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98ec09d6f0 0x7f98ec09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:46.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.906+0000 7f98eb7fe700 1 --2- 172.21.15.16:0/4291824136 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f98ec09e410 0x7f98ec133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:46.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.906+0000 7f98ea7fc700 1 -- 172.21.15.16:0/4291824136 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98ec09d6f0 msgr2=0x7f98ec09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:46.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.906+0000 7f98ea7fc700 1 --2- 172.21.15.16:0/4291824136 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98ec09d6f0 0x7f98ec09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:46.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.906+0000 7f98ea7fc700 1 -- 172.21.15.16:0/4291824136 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f98ec09e410 msgr2=0x7f98ec133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:46.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.906+0000 7f98ea7fc700 1 --2- 172.21.15.16:0/4291824136 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f98ec09e410 0x7f98ec133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:46.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.906+0000 7f98ea7fc700 1 -- 172.21.15.16:0/4291824136 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98ec09a500 con 0x7f98ec09e000 2023-09-30T22:30:46.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.906+0000 7f98eaffd700 1 --2- 172.21.15.16:0/4291824136 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98ec09d6f0 0x7f98ec09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:46.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.906+0000 7f98ea7fc700 1 --2- 172.21.15.16:0/4291824136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98ec09e000 0x7f98ec1339f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f98e00098f0 tx=0x7f98e0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b62b37f3c622d095 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:46.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.907+0000 7f98e9ffb700 1 -- 172.21.15.16:0/4291824136 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e0011830 con 0x7f98ec09e000 2023-09-30T22:30:46.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.907+0000 7f98e9ffb700 1 -- 172.21.15.16:0/4291824136 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98e0011990 con 0x7f98ec09e000 2023-09-30T22:30:46.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.907+0000 7f98e9ffb700 1 -- 172.21.15.16:0/4291824136 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e0011d20 con 0x7f98ec09e000 2023-09-30T22:30:46.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.907+0000 7f98f1645700 1 -- 172.21.15.16:0/4291824136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98ec09e000 msgr2=0x7f98ec1339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:46.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.907+0000 7f98f1645700 1 --2- 172.21.15.16:0/4291824136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98ec09e000 0x7f98ec1339f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f98e00098f0 tx=0x7f98e0009c00 comp rx=0 tx=0).stop 2023-09-30T22:30:46.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.907+0000 7f98f1645700 1 -- 172.21.15.16:0/4291824136 shutdown_connections 2023-09-30T22:30:46.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.907+0000 7f98f1645700 1 --2- 172.21.15.16:0/4291824136 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f98ec09e410 0x7f98ec133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:46.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.907+0000 7f98f1645700 1 --2- 172.21.15.16:0/4291824136 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98ec09d6f0 0x7f98ec09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:46.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.907+0000 7f98f1645700 1 --2- 172.21.15.16:0/4291824136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98ec09e000 0x7f98ec1339f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:46.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.907+0000 7f98f1645700 1 -- 172.21.15.16:0/4291824136 >> 172.21.15.16:0/4291824136 conn(0x7f98ec066f50 msgr2=0x7f98ec09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:46.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.907+0000 7f98f1645700 1 -- 172.21.15.16:0/4291824136 shutdown_connections 2023-09-30T22:30:46.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.907+0000 7f98f1645700 1 -- 172.21.15.16:0/4291824136 wait complete. 2023-09-30T22:30:46.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.909+0000 7f98f1645700 1 Processor -- start 2023-09-30T22:30:46.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.909+0000 7f98f1645700 1 -- start start 2023-09-30T22:30:46.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.909+0000 7f98f1645700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98ec09d6f0 0x7f98ec1a1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:46.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.909+0000 7f98f1645700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f98ec09e000 0x7f98ec1a2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:46.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.909+0000 7f98f1645700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98ec09e410 0x7f98ec1a2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:46.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.909+0000 7f98f1645700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98ec1a3740 con 0x7f98ec09e410 2023-09-30T22:30:46.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.909+0000 7f98f1645700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98ec1a3d60 con 0x7f98ec09e000 2023-09-30T22:30:46.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.909+0000 7f98f1645700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f98ec1a4990 con 0x7f98ec09d6f0 2023-09-30T22:30:46.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.909+0000 7f98ea7fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f98ec09e000 0x7f98ec1a2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:46.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.909+0000 7f98eb7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98ec09e410 0x7f98ec1a2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:46.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.909+0000 7f98ea7fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f98ec09e000 0x7f98ec1a2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37128/0 (socket says 172.21.15.16:37128) 2023-09-30T22:30:46.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.909+0000 7f98ea7fc700 1 -- 172.21.15.16:0/41739011 learned_addr learned my addr 172.21.15.16:0/41739011 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:46.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.909+0000 7f98eaffd700 1 --2- 172.21.15.16:0/41739011 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98ec09d6f0 0x7f98ec1a1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:46.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.910+0000 7f98ea7fc700 1 -- 172.21.15.16:0/41739011 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98ec09d6f0 msgr2=0x7f98ec1a1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:46.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.910+0000 7f98ea7fc700 1 --2- 172.21.15.16:0/41739011 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98ec09d6f0 0x7f98ec1a1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:46.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.910+0000 7f98ea7fc700 1 -- 172.21.15.16:0/41739011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98ec09e410 msgr2=0x7f98ec1a2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:46.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.910+0000 7f98ea7fc700 1 --2- 172.21.15.16:0/41739011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98ec09e410 0x7f98ec1a2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:46.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.910+0000 7f98ea7fc700 1 -- 172.21.15.16:0/41739011 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98dc0094a0 con 0x7f98ec09e000 2023-09-30T22:30:46.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.910+0000 7f98eb7fe700 1 --2- 172.21.15.16:0/41739011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98ec09e410 0x7f98ec1a2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:46.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.910+0000 7f98ea7fc700 1 --2- 172.21.15.16:0/41739011 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f98ec09e000 0x7f98ec1a2100 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f98e0009f50 tx=0x7f98e0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:46.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.910+0000 7f98d37fe700 1 -- 172.21.15.16:0/41739011 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e0004240 con 0x7f98ec09e000 2023-09-30T22:30:46.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.910+0000 7f98d37fe700 1 -- 172.21.15.16:0/41739011 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98e001f070 con 0x7f98ec09e000 2023-09-30T22:30:46.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.910+0000 7f98f1645700 1 -- 172.21.15.16:0/41739011 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98e0009550 con 0x7f98ec09e000 2023-09-30T22:30:46.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.910+0000 7f98d37fe700 1 -- 172.21.15.16:0/41739011 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e00045a0 con 0x7f98ec09e000 2023-09-30T22:30:46.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.910+0000 7f98f1645700 1 -- 172.21.15.16:0/41739011 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98ec0a2000 con 0x7f98ec09e000 2023-09-30T22:30:46.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.911+0000 7f98d37fe700 1 -- 172.21.15.16:0/41739011 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f98e0021030 con 0x7f98ec09e000 2023-09-30T22:30:46.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.911+0000 7f98d37fe700 1 -- 172.21.15.16:0/41739011 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f98e00406f0 con 0x7f98ec09e000 2023-09-30T22:30:46.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.911+0000 7f98f1645700 1 -- 172.21.15.16:0/41739011 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98ec09dd30 con 0x7f98ec09e000 2023-09-30T22:30:46.916 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:46.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:46.917+0000 7f98d37fe700 1 -- 172.21.15.16:0/41739011 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f98e001a450 con 0x7f98ec09e000 2023-09-30T22:30:46.965 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:47.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.045+0000 7feaea993700 1 Processor -- start 2023-09-30T22:30:47.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.045+0000 7feaea993700 1 -- start start 2023-09-30T22:30:47.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.046+0000 7feaea993700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7feae409d6f0 0x7feae409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:47.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.046+0000 7feaea993700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7feae409e000 0x7feae41339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:47.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.046+0000 7feaea993700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7feae409e410 0x7feae4133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:47.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.046+0000 7feaea993700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feae406b460 con 0x7feae409e000 2023-09-30T22:30:47.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.046+0000 7feaea993700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feae409b120 con 0x7feae409e410 2023-09-30T22:30:47.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.046+0000 7feaea993700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feae409a7e0 con 0x7feae409d6f0 2023-09-30T22:30:47.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.046+0000 7feae37fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7feae409e000 0x7feae41339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:47.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.046+0000 7feae8f30700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7feae409e410 0x7feae4133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:47.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.046+0000 7feae37fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7feae409e000 0x7feae41339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52160/0 (socket says 172.21.15.16:52160) 2023-09-30T22:30:47.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.046+0000 7feae37fe700 1 -- 172.21.15.16:0/154708681 learned_addr learned my addr 172.21.15.16:0/154708681 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:47.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.046+0000 7feae3fff700 1 --2- 172.21.15.16:0/154708681 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7feae409d6f0 0x7feae409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:47.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.047+0000 7feae37fe700 1 -- 172.21.15.16:0/154708681 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7feae409d6f0 msgr2=0x7feae409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:47.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.047+0000 7feae37fe700 1 --2- 172.21.15.16:0/154708681 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7feae409d6f0 0x7feae409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:47.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.047+0000 7feae37fe700 1 -- 172.21.15.16:0/154708681 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7feae409e410 msgr2=0x7feae4133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:47.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.047+0000 7feae37fe700 1 --2- 172.21.15.16:0/154708681 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7feae409e410 0x7feae4133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:47.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.047+0000 7feae37fe700 1 -- 172.21.15.16:0/154708681 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae409a500 con 0x7feae409e000 2023-09-30T22:30:47.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.047+0000 7feae8f30700 1 --2- 172.21.15.16:0/154708681 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7feae409e410 0x7feae4133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:47.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.047+0000 7feae37fe700 1 --2- 172.21.15.16:0/154708681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feae409e000 0x7feae41339f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fead80098f0 tx=0x7fead8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43d6fdac1ff5a3e7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:47.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.047+0000 7feae2ffd700 1 -- 172.21.15.16:0/154708681 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead8011830 con 0x7feae409e000 2023-09-30T22:30:47.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.047+0000 7feae2ffd700 1 -- 172.21.15.16:0/154708681 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fead8011990 con 0x7feae409e000 2023-09-30T22:30:47.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.047+0000 7feae2ffd700 1 -- 172.21.15.16:0/154708681 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead8011d20 con 0x7feae409e000 2023-09-30T22:30:47.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.048+0000 7feaea993700 1 -- 172.21.15.16:0/154708681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feae409e000 msgr2=0x7feae41339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:47.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.048+0000 7feaea993700 1 --2- 172.21.15.16:0/154708681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feae409e000 0x7feae41339f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fead80098f0 tx=0x7fead8009c00 comp rx=0 tx=0).stop 2023-09-30T22:30:47.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.048+0000 7feaea993700 1 -- 172.21.15.16:0/154708681 shutdown_connections 2023-09-30T22:30:47.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.048+0000 7feaea993700 1 --2- 172.21.15.16:0/154708681 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7feae409e410 0x7feae4133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:47.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.048+0000 7feaea993700 1 --2- 172.21.15.16:0/154708681 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7feae409d6f0 0x7feae409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:47.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.048+0000 7feaea993700 1 --2- 172.21.15.16:0/154708681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feae409e000 0x7feae41339f0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:47.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.048+0000 7feaea993700 1 -- 172.21.15.16:0/154708681 >> 172.21.15.16:0/154708681 conn(0x7feae4066f50 msgr2=0x7feae409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:47.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.048+0000 7feaea993700 1 -- 172.21.15.16:0/154708681 shutdown_connections 2023-09-30T22:30:47.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.048+0000 7feaea993700 1 -- 172.21.15.16:0/154708681 wait complete. 2023-09-30T22:30:47.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.049+0000 7feaea993700 1 Processor -- start 2023-09-30T22:30:47.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.050+0000 7feaea993700 1 -- start start 2023-09-30T22:30:47.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.050+0000 7feaea993700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feae409d6f0 0x7feae41a5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:47.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.050+0000 7feaea993700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7feae409e000 0x7feae41a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:47.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.050+0000 7feaea993700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7feae409e410 0x7feae41a67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:47.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.050+0000 7feaea993700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae41aa240 con 0x7feae409d6f0 2023-09-30T22:30:47.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.050+0000 7feaea993700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feae41a7970 con 0x7feae409e410 2023-09-30T22:30:47.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.050+0000 7feaea993700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feae41a7f90 con 0x7feae409e000 2023-09-30T22:30:47.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.051+0000 7feae3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feae409d6f0 0x7feae41a5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:47.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.051+0000 7feae3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feae409d6f0 0x7feae41a5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52168/0 (socket says 172.21.15.16:52168) 2023-09-30T22:30:47.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.051+0000 7feae3fff700 1 -- 172.21.15.16:0/2407993245 learned_addr learned my addr 172.21.15.16:0/2407993245 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:47.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.051+0000 7feae8f30700 1 --2- 172.21.15.16:0/2407993245 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7feae409e410 0x7feae41a67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:47.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.051+0000 7feae37fe700 1 --2- 172.21.15.16:0/2407993245 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7feae409e000 0x7feae41a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:47.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.051+0000 7feae3fff700 1 -- 172.21.15.16:0/2407993245 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7feae409e000 msgr2=0x7feae41a62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:47.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.051+0000 7feae3fff700 1 --2- 172.21.15.16:0/2407993245 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7feae409e000 0x7feae41a62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:47.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.051+0000 7feae3fff700 1 -- 172.21.15.16:0/2407993245 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7feae409e410 msgr2=0x7feae41a67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:47.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.051+0000 7feae3fff700 1 --2- 172.21.15.16:0/2407993245 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7feae409e410 0x7feae41a67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:47.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.051+0000 7feae3fff700 1 -- 172.21.15.16:0/2407993245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feadc009a70 con 0x7feae409d6f0 2023-09-30T22:30:47.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.051+0000 7feae3fff700 1 --2- 172.21.15.16:0/2407993245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feae409d6f0 0x7feae41a5d60 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fead400e840 tx=0x7fead400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:47.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.052+0000 7feae0ff9700 1 -- 172.21.15.16:0/2407993245 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead4013610 con 0x7feae409d6f0 2023-09-30T22:30:47.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.052+0000 7feae0ff9700 1 -- 172.21.15.16:0/2407993245 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fead4013770 con 0x7feae409d6f0 2023-09-30T22:30:47.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.052+0000 7feaea993700 1 -- 172.21.15.16:0/2407993245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fead8009550 con 0x7feae409d6f0 2023-09-30T22:30:47.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.052+0000 7feae0ff9700 1 -- 172.21.15.16:0/2407993245 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead4013b00 con 0x7feae409d6f0 2023-09-30T22:30:47.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.052+0000 7feaea993700 1 -- 172.21.15.16:0/2407993245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feae40a21d0 con 0x7feae409d6f0 2023-09-30T22:30:47.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.053+0000 7feaea993700 1 -- 172.21.15.16:0/2407993245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feae409c710 con 0x7feae409d6f0 2023-09-30T22:30:47.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.053+0000 7feae0ff9700 1 -- 172.21.15.16:0/2407993245 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fead4013c60 con 0x7feae409d6f0 2023-09-30T22:30:47.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.053+0000 7feae0ff9700 1 -- 172.21.15.16:0/2407993245 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fead4040b30 con 0x7feae409d6f0 2023-09-30T22:30:47.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.058+0000 7feae0ff9700 1 -- 172.21.15.16:0/2407993245 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fead4044060 con 0x7feae409d6f0 2023-09-30T22:30:47.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.061+0000 7fbf3f67d700 1 Processor -- start 2023-09-30T22:30:47.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.061+0000 7fbf3f67d700 1 -- start start 2023-09-30T22:30:47.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.062+0000 7fbf3f67d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbf38133540 0x7fbf38135920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:47.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.062+0000 7fbf3f67d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbf38135e60 0x7fbf38138240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:47.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.062+0000 7fbf3f67d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf380716a0 con 0x7fbf38130540 2023-09-30T22:30:47.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.062+0000 7fbf3f67d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf38071290 con 0x7fbf38135e60 2023-09-30T22:30:47.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.062+0000 7fbf3f67d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf38130340 con 0x7fbf38133540 2023-09-30T22:30:47.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.062+0000 7fbf3cc18700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbf38133540 0x7fbf38135920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:47.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.062+0000 7fbf3cc18700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbf38133540 0x7fbf38135920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37146/0 (socket says 172.21.15.16:37146) 2023-09-30T22:30:47.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.062+0000 7fbf3cc18700 1 -- 172.21.15.16:0/3827867695 learned_addr learned my addr 172.21.15.16:0/3827867695 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:47.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.062+0000 7fbf3dc1a700 1 --2- 172.21.15.16:0/3827867695 >> v2:172.21.15.16:3300/0 conn(0x7fbf38135e60 0x7fbf38138240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:47.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.062+0000 7fbf3cc18700 1 -- 172.21.15.16:0/3827867695 >> v2:172.21.15.16:3300/0 conn(0x7fbf38135e60 msgr2=0x7fbf38138240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:47.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.062+0000 7fbf3cc18700 1 --2- 172.21.15.16:0/3827867695 >> v2:172.21.15.16:3300/0 conn(0x7fbf38135e60 0x7fbf38138240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:47.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.062+0000 7fbf3cc18700 1 -- 172.21.15.16:0/3827867695 >> v1:172.21.15.16:6789/0 conn(0x7fbf38130540 legacy=0x7fbf38130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:47.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.063+0000 7fbf3cc18700 1 -- 172.21.15.16:0/3827867695 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf38070b80 con 0x7fbf38133540 2023-09-30T22:30:47.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.063+0000 7fbf3cc18700 1 --2- 172.21.15.16:0/3827867695 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbf38133540 0x7fbf38135920 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbf280098f0 tx=0x7fbf28009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99653f6b26bb43d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:47.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.063+0000 7fbf2ffff700 1 -- 172.21.15.16:0/3827867695 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf28004230 con 0x7fbf38133540 2023-09-30T22:30:47.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.063+0000 7fbf2ffff700 1 -- 172.21.15.16:0/3827867695 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf28004390 con 0x7fbf38133540 2023-09-30T22:30:47.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.063+0000 7fbf2ffff700 1 -- 172.21.15.16:0/3827867695 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf280036a0 con 0x7fbf38133540 2023-09-30T22:30:47.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.064+0000 7fbf3f67d700 1 -- 172.21.15.16:0/3827867695 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbf38133540 msgr2=0x7fbf38135920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:47.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.064+0000 7fbf3f67d700 1 --2- 172.21.15.16:0/3827867695 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbf38133540 0x7fbf38135920 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbf280098f0 tx=0x7fbf28009c00 comp rx=0 tx=0).stop 2023-09-30T22:30:47.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.064+0000 7fbf3f67d700 1 -- 172.21.15.16:0/3827867695 shutdown_connections 2023-09-30T22:30:47.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.064+0000 7fbf3f67d700 1 --2- 172.21.15.16:0/3827867695 >> v2:172.21.15.16:3300/0 conn(0x7fbf38135e60 0x7fbf38138240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:47.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.064+0000 7fbf3f67d700 1 --2- 172.21.15.16:0/3827867695 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbf38133540 0x7fbf38135920 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:47.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.064+0000 7fbf3f67d700 1 -- 172.21.15.16:0/3827867695 >> 172.21.15.16:0/3827867695 conn(0x7fbf380a2aa0 msgr2=0x7fbf380a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:47.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.064+0000 7fbf3f67d700 1 -- 172.21.15.16:0/3827867695 shutdown_connections 2023-09-30T22:30:47.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.064+0000 7fbf3f67d700 1 -- 172.21.15.16:0/3827867695 wait complete. 2023-09-30T22:30:47.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.065+0000 7fbf3f67d700 1 Processor -- start 2023-09-30T22:30:47.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.065+0000 7fbf3f67d700 1 -- start start 2023-09-30T22:30:47.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.066+0000 7fbf3f67d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf38130540 0x7fbf381c8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:47.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.066+0000 7fbf3f67d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbf38133540 0x7fbf381c86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:47.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.066+0000 7fbf3f67d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbf38135e60 0x7fbf381c8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:47.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.066+0000 7fbf3f67d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3812dd00 con 0x7fbf38130540 2023-09-30T22:30:47.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.066+0000 7fbf3f67d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3811eb90 con 0x7fbf38135e60 2023-09-30T22:30:47.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.066+0000 7fbf3f67d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3809e270 con 0x7fbf38133540 2023-09-30T22:30:47.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.066+0000 7fbf3dc1a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbf38135e60 0x7fbf381c8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:47.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.066+0000 7fbf3dc1a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbf38135e60 0x7fbf381c8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37160/0 (socket says 172.21.15.16:37160) 2023-09-30T22:30:47.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.066+0000 7fbf3dc1a700 1 -- 172.21.15.16:0/2052117345 learned_addr learned my addr 172.21.15.16:0/2052117345 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:47.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.066+0000 7fbf3dc1a700 1 -- 172.21.15.16:0/2052117345 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbf38133540 msgr2=0x7fbf381c86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:47.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.066+0000 7fbf3dc1a700 1 --2- 172.21.15.16:0/2052117345 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbf38133540 0x7fbf381c86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:47.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.066+0000 7fbf3dc1a700 1 -- 172.21.15.16:0/2052117345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf38130540 msgr2=0x7fbf381c8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:47.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.066+0000 7fbf3dc1a700 1 --2- 172.21.15.16:0/2052117345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf38130540 0x7fbf381c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:47.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.066+0000 7fbf3dc1a700 1 -- 172.21.15.16:0/2052117345 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf28009550 con 0x7fbf38135e60 2023-09-30T22:30:47.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.066+0000 7fbf3dc1a700 1 --2- 172.21.15.16:0/2052117345 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbf38135e60 0x7fbf381c8c00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbf3000efc0 tx=0x7fbf3000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:47.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.067+0000 7fbf2dffb700 1 -- 172.21.15.16:0/2052117345 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3000cd40 con 0x7fbf38135e60 2023-09-30T22:30:47.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.067+0000 7fbf2dffb700 1 -- 172.21.15.16:0/2052117345 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf3000f040 con 0x7fbf38135e60 2023-09-30T22:30:47.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.067+0000 7fbf3f67d700 1 -- 172.21.15.16:0/2052117345 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf3811e5f0 con 0x7fbf38135e60 2023-09-30T22:30:47.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.067+0000 7fbf2dffb700 1 -- 172.21.15.16:0/2052117345 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf300146c0 con 0x7fbf38135e60 2023-09-30T22:30:47.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.067+0000 7fbf3f67d700 1 -- 172.21.15.16:0/2052117345 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf3811f320 con 0x7fbf38135e60 2023-09-30T22:30:47.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.068+0000 7fbf2dffb700 1 -- 172.21.15.16:0/2052117345 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbf30014820 con 0x7fbf38135e60 2023-09-30T22:30:47.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.068+0000 7fbf3f67d700 1 -- 172.21.15.16:0/2052117345 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf38074ea0 con 0x7fbf38135e60 2023-09-30T22:30:47.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.068+0000 7fbf2dffb700 1 -- 172.21.15.16:0/2052117345 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fbf30040040 con 0x7fbf38135e60 2023-09-30T22:30:47.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.073+0000 7fbf2dffb700 1 -- 172.21.15.16:0/2052117345 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbf3001a8b0 con 0x7fbf38135e60 2023-09-30T22:30:47.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.111+0000 7f8dcda63700 1 Processor -- start 2023-09-30T22:30:47.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.112+0000 7f8dcda63700 1 -- start start 2023-09-30T22:30:47.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.113+0000 7f8dcda63700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8dc809f410 0x7f8dc8136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:47.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.113+0000 7f8dcda63700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8dc8136fe0 0x7f8dc81393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:47.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.113+0000 7f8dcda63700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dc8070940 con 0x7f8dc809e930 2023-09-30T22:30:47.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.113+0000 7f8dcda63700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc806fd60 con 0x7f8dc809f410 2023-09-30T22:30:47.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.113+0000 7f8dcda63700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc806fa80 con 0x7f8dc8136fe0 2023-09-30T22:30:47.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.113+0000 7f8dc77fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8dc809e930 0x7f8dc809ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58392/0 (socket says 172.21.15.16:58392) 2023-09-30T22:30:47.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.113+0000 7f8dc77fe700 1 -- 172.21.15.16:0/3450283989 learned_addr learned my addr 172.21.15.16:0/3450283989 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:30:47.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.113+0000 7f8dc6ffd700 1 --2- 172.21.15.16:0/3450283989 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8dc809f410 0x7f8dc8136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:47.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.113+0000 7f8dc7fff700 1 --2- 172.21.15.16:0/3450283989 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8dc8136fe0 0x7f8dc81393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:47.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.113+0000 7f8dc67fc700 1 -- 172.21.15.16:0/3450283989 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3625876319 0 0) 0x7f8dc8070940 con 0x7f8dc809e930 2023-09-30T22:30:47.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.113+0000 7f8dc67fc700 1 -- 172.21.15.16:0/3450283989 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8db00035b0 con 0x7f8dc809e930 2023-09-30T22:30:47.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.113+0000 7f8dc6ffd700 1 -- 172.21.15.16:0/3450283989 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8dc8136fe0 msgr2=0x7f8dc81393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:47.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.113+0000 7f8dc6ffd700 1 --2- 172.21.15.16:0/3450283989 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8dc8136fe0 0x7f8dc81393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:47.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.113+0000 7f8dc6ffd700 1 -- 172.21.15.16:0/3450283989 >> v1:172.21.15.16:6789/0 conn(0x7f8dc809e930 legacy=0x7f8dc809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:47.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.113+0000 7f8dc6ffd700 1 -- 172.21.15.16:0/3450283989 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc809fb60 con 0x7f8dc809f410 2023-09-30T22:30:47.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.114+0000 7f8dc6ffd700 1 --2- 172.21.15.16:0/3450283989 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8dc809f410 0x7f8dc8136aa0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc009870 tx=0x7f8dbc009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26565c614640158 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:47.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.114+0000 7f8dc67fc700 1 -- 172.21.15.16:0/3450283989 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc011830 con 0x7f8dc809f410 2023-09-30T22:30:47.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.114+0000 7f8dc67fc700 1 -- 172.21.15.16:0/3450283989 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dbc011990 con 0x7f8dc809f410 2023-09-30T22:30:47.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.114+0000 7f8dc67fc700 1 -- 172.21.15.16:0/3450283989 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc011d20 con 0x7f8dc809f410 2023-09-30T22:30:47.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.115+0000 7f8dcda63700 1 -- 172.21.15.16:0/3450283989 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8dc809f410 msgr2=0x7f8dc8136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:47.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.115+0000 7f8dcda63700 1 --2- 172.21.15.16:0/3450283989 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8dc809f410 0x7f8dc8136aa0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc009870 tx=0x7f8dbc009b80 comp rx=0 tx=0).stop 2023-09-30T22:30:47.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.115+0000 7f8dcda63700 1 -- 172.21.15.16:0/3450283989 shutdown_connections 2023-09-30T22:30:47.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.115+0000 7f8dcda63700 1 --2- 172.21.15.16:0/3450283989 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8dc8136fe0 0x7f8dc81393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:47.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.115+0000 7f8dcda63700 1 --2- 172.21.15.16:0/3450283989 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8dc809f410 0x7f8dc8136aa0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:47.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.115+0000 7f8dcda63700 1 -- 172.21.15.16:0/3450283989 >> 172.21.15.16:0/3450283989 conn(0x7f8dc80a4210 msgr2=0x7f8dc80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:47.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.115+0000 7f8dcda63700 1 -- 172.21.15.16:0/3450283989 shutdown_connections 2023-09-30T22:30:47.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.115+0000 7f8dcda63700 1 -- 172.21.15.16:0/3450283989 wait complete. 2023-09-30T22:30:47.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.116+0000 7f8dcda63700 1 Processor -- start 2023-09-30T22:30:47.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.117+0000 7f8dcda63700 1 -- start start 2023-09-30T22:30:47.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.117+0000 7f8dcda63700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8dc809e930 0x7f8dc811b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:47.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.117+0000 7f8dcda63700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8dc809f410 0x7f8dc811bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:47.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.117+0000 7f8dcda63700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8dc8136fe0 0x7f8dc811bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:47.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.118+0000 7f8dcda63700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc811da10 con 0x7f8dc809e930 2023-09-30T22:30:47.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.118+0000 7f8dcda63700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc811d140 con 0x7f8dc809f410 2023-09-30T22:30:47.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.118+0000 7f8dcda63700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc8136e40 con 0x7f8dc8136fe0 2023-09-30T22:30:47.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.118+0000 7f8dc6ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8dc809f410 0x7f8dc811bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:47.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.118+0000 7f8dc6ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8dc809f410 0x7f8dc811bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37180/0 (socket says 172.21.15.16:37180) 2023-09-30T22:30:47.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.118+0000 7f8dc6ffd700 1 -- 172.21.15.16:0/226442739 learned_addr learned my addr 172.21.15.16:0/226442739 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:47.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.118+0000 7f8dc7fff700 1 --2- 172.21.15.16:0/226442739 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8dc8136fe0 0x7f8dc811bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:47.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.118+0000 7f8dc6ffd700 1 -- 172.21.15.16:0/226442739 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8dc8136fe0 msgr2=0x7f8dc811bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:47.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.118+0000 7f8dc6ffd700 1 --2- 172.21.15.16:0/226442739 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8dc8136fe0 0x7f8dc811bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:47.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.118+0000 7f8dc6ffd700 1 -- 172.21.15.16:0/226442739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8dc809e930 msgr2=0x7f8dc811b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:47.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.118+0000 7f8dc6ffd700 1 --2- 172.21.15.16:0/226442739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8dc809e930 0x7f8dc811b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:47.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.118+0000 7f8dc6ffd700 1 -- 172.21.15.16:0/226442739 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dbc0094a0 con 0x7f8dc809f410 2023-09-30T22:30:47.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.118+0000 7f8dc6ffd700 1 --2- 172.21.15.16:0/226442739 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8dc809f410 0x7f8dc811bab0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc0090b0 tx=0x7f8dbc004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:47.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.119+0000 7f8daffff700 1 -- 172.21.15.16:0/226442739 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc004510 con 0x7f8dc809f410 2023-09-30T22:30:47.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.119+0000 7f8daffff700 1 -- 172.21.15.16:0/226442739 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dbc01f070 con 0x7f8dc809f410 2023-09-30T22:30:47.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.119+0000 7f8dcda63700 1 -- 172.21.15.16:0/226442739 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dc8071d20 con 0x7f8dc809f410 2023-09-30T22:30:47.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.119+0000 7f8daffff700 1 -- 172.21.15.16:0/226442739 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc0038e0 con 0x7f8dc809f410 2023-09-30T22:30:47.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.119+0000 7f8dcda63700 1 -- 172.21.15.16:0/226442739 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dc811edd0 con 0x7f8dc809f410 2023-09-30T22:30:47.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.120+0000 7f8daffff700 1 -- 172.21.15.16:0/226442739 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8dbc003a40 con 0x7f8dc809f410 2023-09-30T22:30:47.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.120+0000 7f8dcda63700 1 -- 172.21.15.16:0/226442739 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dc8074ea0 con 0x7f8dc809f410 2023-09-30T22:30:47.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.120+0000 7f8daffff700 1 -- 172.21.15.16:0/226442739 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f8dbc021070 con 0x7f8dc809f410 2023-09-30T22:30:47.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.130+0000 7f8daffff700 1 -- 172.21.15.16:0/226442739 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8dbc01a7b0 con 0x7f8dc809f410 2023-09-30T22:30:47.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.147+0000 7f98f1645700 1 -- 172.21.15.16:0/41739011 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f98ec09c710 con 0x7f98ec09e000 2023-09-30T22:30:47.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.147+0000 7f98d37fe700 1 -- 172.21.15.16:0/41739011 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f98e0015020 con 0x7f98ec09e000 2023-09-30T22:30:47.148 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:30:47.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.150+0000 7f98f1645700 1 -- 172.21.15.16:0/41739011 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f98ec09e000 msgr2=0x7f98ec1a2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:47.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.150+0000 7f98f1645700 1 --2- 172.21.15.16:0/41739011 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f98ec09e000 0x7f98ec1a2100 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f98e0009f50 tx=0x7f98e0004750 comp rx=0 tx=0).stop 2023-09-30T22:30:47.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.150+0000 7f98f1645700 1 -- 172.21.15.16:0/41739011 shutdown_connections 2023-09-30T22:30:47.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.150+0000 7f98f1645700 1 --2- 172.21.15.16:0/41739011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98ec09e410 0x7f98ec1a2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:47.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.150+0000 7f98f1645700 1 --2- 172.21.15.16:0/41739011 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98ec09d6f0 0x7f98ec1a1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:47.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.150+0000 7f98f1645700 1 --2- 172.21.15.16:0/41739011 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f98ec09e000 0x7f98ec1a2100 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:47.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.150+0000 7f98f1645700 1 -- 172.21.15.16:0/41739011 >> 172.21.15.16:0/41739011 conn(0x7f98ec066f50 msgr2=0x7f98ec1b6970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:47.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.150+0000 7f98f1645700 1 -- 172.21.15.16:0/41739011 shutdown_connections 2023-09-30T22:30:47.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.150+0000 7f98f1645700 1 -- 172.21.15.16:0/41739011 wait complete. 2023-09-30T22:30:47.164 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:30:47.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.273+0000 7feaea993700 1 -- 172.21.15.16:0/2407993245 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7feae4074ea0 con 0x7feae409d6f0 2023-09-30T22:30:47.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.274+0000 7feae0ff9700 1 -- 172.21.15.16:0/2407993245 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fead4044020 con 0x7feae409d6f0 2023-09-30T22:30:47.274 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:30:47.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.276+0000 7feaea993700 1 -- 172.21.15.16:0/2407993245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feae409d6f0 msgr2=0x7feae41a5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:47.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.276+0000 7feaea993700 1 --2- 172.21.15.16:0/2407993245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feae409d6f0 0x7feae41a5d60 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fead400e840 tx=0x7fead400ec00 comp rx=0 tx=0).stop 2023-09-30T22:30:47.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.276+0000 7feaea993700 1 -- 172.21.15.16:0/2407993245 shutdown_connections 2023-09-30T22:30:47.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.276+0000 7feaea993700 1 --2- 172.21.15.16:0/2407993245 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7feae409e410 0x7feae41a67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:47.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.276+0000 7feaea993700 1 --2- 172.21.15.16:0/2407993245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feae409d6f0 0x7feae41a5d60 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:47.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.276+0000 7feaea993700 1 --2- 172.21.15.16:0/2407993245 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7feae409e000 0x7feae41a62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:47.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.276+0000 7feaea993700 1 -- 172.21.15.16:0/2407993245 >> 172.21.15.16:0/2407993245 conn(0x7feae4066f50 msgr2=0x7feae41bae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:47.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.276+0000 7feaea993700 1 -- 172.21.15.16:0/2407993245 shutdown_connections 2023-09-30T22:30:47.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.276+0000 7feaea993700 1 -- 172.21.15.16:0/2407993245 wait complete. 2023-09-30T22:30:47.290 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:30:47.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.306+0000 7fbf3f67d700 1 -- 172.21.15.16:0/2052117345 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fbf3806b9e0 con 0x7fbf38135e60 2023-09-30T22:30:47.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.306+0000 7fbf2dffb700 1 -- 172.21.15.16:0/2052117345 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbf3001a2c0 con 0x7fbf38135e60 2023-09-30T22:30:47.307 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:30:47.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.309+0000 7fbf3f67d700 1 -- 172.21.15.16:0/2052117345 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbf38135e60 msgr2=0x7fbf381c8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:47.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.309+0000 7fbf3f67d700 1 --2- 172.21.15.16:0/2052117345 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbf38135e60 0x7fbf381c8c00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fbf3000efc0 tx=0x7fbf3000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:30:47.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.309+0000 7fbf3f67d700 1 -- 172.21.15.16:0/2052117345 shutdown_connections 2023-09-30T22:30:47.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.309+0000 7fbf3f67d700 1 --2- 172.21.15.16:0/2052117345 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbf38135e60 0x7fbf381c8c00 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:47.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.309+0000 7fbf3f67d700 1 --2- 172.21.15.16:0/2052117345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf38130540 0x7fbf381c8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:47.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.309+0000 7fbf3f67d700 1 --2- 172.21.15.16:0/2052117345 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbf38133540 0x7fbf381c86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:47.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.309+0000 7fbf3f67d700 1 -- 172.21.15.16:0/2052117345 >> 172.21.15.16:0/2052117345 conn(0x7fbf380a2aa0 msgr2=0x7fbf38131300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:47.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.309+0000 7fbf3f67d700 1 -- 172.21.15.16:0/2052117345 shutdown_connections 2023-09-30T22:30:47.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.309+0000 7fbf3f67d700 1 -- 172.21.15.16:0/2052117345 wait complete. 2023-09-30T22:30:47.323 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:30:47.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.358+0000 7f8dcda63700 1 -- 172.21.15.16:0/226442739 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8dc806b250 con 0x7f8dc809f410 2023-09-30T22:30:47.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.358+0000 7f8daffff700 1 -- 172.21.15.16:0/226442739 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8dbc02b070 con 0x7f8dc809f410 2023-09-30T22:30:47.359 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:30:47.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.360+0000 7f8dcda63700 1 -- 172.21.15.16:0/226442739 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8dc809f410 msgr2=0x7f8dc811bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:47.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.360+0000 7f8dcda63700 1 --2- 172.21.15.16:0/226442739 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8dc809f410 0x7f8dc811bab0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc0090b0 tx=0x7f8dbc004750 comp rx=0 tx=0).stop 2023-09-30T22:30:47.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.360+0000 7f8dcda63700 1 -- 172.21.15.16:0/226442739 shutdown_connections 2023-09-30T22:30:47.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.360+0000 7f8dcda63700 1 --2- 172.21.15.16:0/226442739 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8dc8136fe0 0x7f8dc811bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:47.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.360+0000 7f8dcda63700 1 --2- 172.21.15.16:0/226442739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8dc809e930 0x7f8dc811b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:47.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.360+0000 7f8dcda63700 1 --2- 172.21.15.16:0/226442739 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8dc809f410 0x7f8dc811bab0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:47.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.360+0000 7f8dcda63700 1 -- 172.21.15.16:0/226442739 >> 172.21.15.16:0/226442739 conn(0x7f8dc80a4210 msgr2=0x7f8dc81393c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:47.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.361+0000 7f8dcda63700 1 -- 172.21.15.16:0/226442739 shutdown_connections 2023-09-30T22:30:47.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:47.361+0000 7f8dcda63700 1 -- 172.21.15.16:0/226442739 wait complete. 2023-09-30T22:30:47.374 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:30:48.165 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:48.291 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:48.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.311+0000 7ffaeb247700 1 Processor -- start 2023-09-30T22:30:48.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.311+0000 7ffaeb247700 1 -- start start 2023-09-30T22:30:48.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.311+0000 7ffaeb247700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffae409d6f0 0x7ffae409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:48.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.311+0000 7ffaeb247700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffae409e000 0x7ffae41339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:48.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.311+0000 7ffaeb247700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffae409b120 con 0x7ffae409e410 2023-09-30T22:30:48.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.311+0000 7ffaeb247700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffae406b460 con 0x7ffae409e000 2023-09-30T22:30:48.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.311+0000 7ffaeb247700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffae409a7e0 con 0x7ffae409d6f0 2023-09-30T22:30:48.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.312+0000 7ffae8fe3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffae409d6f0 0x7ffae409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:48.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.312+0000 7ffae3fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffae409e000 0x7ffae41339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:48.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.312+0000 7ffae97e4700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ffae409e410 0x7ffae4133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40796/0 (socket says 172.21.15.16:40796) 2023-09-30T22:30:48.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.312+0000 7ffae97e4700 1 -- 172.21.15.16:0/2258546225 learned_addr learned my addr 172.21.15.16:0/2258546225 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:30:48.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.312+0000 7ffae8fe3700 1 -- 172.21.15.16:0/2258546225 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffae409e000 msgr2=0x7ffae41339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:48.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.312+0000 7ffae8fe3700 1 --2- 172.21.15.16:0/2258546225 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffae409e000 0x7ffae41339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.312+0000 7ffae8fe3700 1 -- 172.21.15.16:0/2258546225 >> v1:172.21.15.16:6789/0 conn(0x7ffae409e410 legacy=0x7ffae4133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:48.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.312+0000 7ffae8fe3700 1 -- 172.21.15.16:0/2258546225 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffae409a500 con 0x7ffae409d6f0 2023-09-30T22:30:48.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.312+0000 7ffae3fff700 1 --2- 172.21.15.16:0/2258546225 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffae409e000 0x7ffae41339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:48.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.312+0000 7ffae8fe3700 1 --2- 172.21.15.16:0/2258546225 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffae409d6f0 0x7ffae409dac0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ffad80098a0 tx=0x7ffad8009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f8a1d86ab0abb28 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:48.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.312+0000 7ffae37fe700 1 -- 172.21.15.16:0/2258546225 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad8011830 con 0x7ffae409d6f0 2023-09-30T22:30:48.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.313+0000 7ffae37fe700 1 -- 172.21.15.16:0/2258546225 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffad8011990 con 0x7ffae409d6f0 2023-09-30T22:30:48.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.313+0000 7ffae37fe700 1 -- 172.21.15.16:0/2258546225 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad8011d20 con 0x7ffae409d6f0 2023-09-30T22:30:48.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.313+0000 7ffaeb247700 1 -- 172.21.15.16:0/2258546225 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffae409d6f0 msgr2=0x7ffae409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:48.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.313+0000 7ffaeb247700 1 --2- 172.21.15.16:0/2258546225 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffae409d6f0 0x7ffae409dac0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ffad80098a0 tx=0x7ffad8009bb0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.313+0000 7ffaeb247700 1 -- 172.21.15.16:0/2258546225 shutdown_connections 2023-09-30T22:30:48.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.313+0000 7ffaeb247700 1 --2- 172.21.15.16:0/2258546225 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffae409d6f0 0x7ffae409dac0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.313+0000 7ffaeb247700 1 --2- 172.21.15.16:0/2258546225 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffae409e000 0x7ffae41339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.313+0000 7ffaeb247700 1 -- 172.21.15.16:0/2258546225 >> 172.21.15.16:0/2258546225 conn(0x7ffae4066f50 msgr2=0x7ffae409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:48.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.313+0000 7ffaeb247700 1 -- 172.21.15.16:0/2258546225 shutdown_connections 2023-09-30T22:30:48.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.313+0000 7ffaeb247700 1 -- 172.21.15.16:0/2258546225 wait complete. 2023-09-30T22:30:48.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.314+0000 7ffaeb247700 1 Processor -- start 2023-09-30T22:30:48.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.314+0000 7ffaeb247700 1 -- start start 2023-09-30T22:30:48.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.315+0000 7ffaeb247700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffae409d6f0 0x7ffae41a5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:48.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.315+0000 7ffaeb247700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffae409e000 0x7ffae41a63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:48.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.315+0000 7ffaeb247700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffae409e410 0x7ffae41a6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:48.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.315+0000 7ffaeb247700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae41a93c0 con 0x7ffae409e410 2023-09-30T22:30:48.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.315+0000 7ffaeb247700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffae409c710 con 0x7ffae409e000 2023-09-30T22:30:48.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.315+0000 7ffaeb247700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffae41a8d10 con 0x7ffae409d6f0 2023-09-30T22:30:48.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.315+0000 7ffae8fe3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffae409d6f0 0x7ffae41a5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:48.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.315+0000 7ffae8fe3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffae409d6f0 0x7ffae41a5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35344/0 (socket says 172.21.15.16:35344) 2023-09-30T22:30:48.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.315+0000 7ffae8fe3700 1 -- 172.21.15.16:0/1598294242 learned_addr learned my addr 172.21.15.16:0/1598294242 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:48.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.315+0000 7ffae97e4700 1 --2- 172.21.15.16:0/1598294242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffae409e410 0x7ffae41a6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:48.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.315+0000 7ffae3fff700 1 --2- 172.21.15.16:0/1598294242 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffae409e000 0x7ffae41a63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:48.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.315+0000 7ffae8fe3700 1 -- 172.21.15.16:0/1598294242 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffae409e000 msgr2=0x7ffae41a63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:48.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.316+0000 7ffae8fe3700 1 --2- 172.21.15.16:0/1598294242 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffae409e000 0x7ffae41a63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.316+0000 7ffae8fe3700 1 -- 172.21.15.16:0/1598294242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffae409e410 msgr2=0x7ffae41a6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:48.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.316+0000 7ffae8fe3700 1 --2- 172.21.15.16:0/1598294242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffae409e410 0x7ffae41a6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.316+0000 7ffae8fe3700 1 -- 172.21.15.16:0/1598294242 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad00094a0 con 0x7ffae409d6f0 2023-09-30T22:30:48.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.316+0000 7ffae8fe3700 1 --2- 172.21.15.16:0/1598294242 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffae409d6f0 0x7ffae41a5e80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ffad80090b0 tx=0x7ffad8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:48.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.316+0000 7ffae17fa700 1 -- 172.21.15.16:0/1598294242 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad8004450 con 0x7ffae409d6f0 2023-09-30T22:30:48.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.316+0000 7ffae17fa700 1 -- 172.21.15.16:0/1598294242 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffad801f070 con 0x7ffae409d6f0 2023-09-30T22:30:48.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.316+0000 7ffae17fa700 1 -- 172.21.15.16:0/1598294242 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad80038e0 con 0x7ffae409d6f0 2023-09-30T22:30:48.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.317+0000 7ffaeb247700 1 -- 172.21.15.16:0/1598294242 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffad80094a0 con 0x7ffae409d6f0 2023-09-30T22:30:48.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.317+0000 7ffaeb247700 1 -- 172.21.15.16:0/1598294242 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffae41a8850 con 0x7ffae409d6f0 2023-09-30T22:30:48.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.317+0000 7ffae17fa700 1 -- 172.21.15.16:0/1598294242 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ffad8003a40 con 0x7ffae409d6f0 2023-09-30T22:30:48.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.318+0000 7ffaeb247700 1 -- 172.21.15.16:0/1598294242 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffae4074ea0 con 0x7ffae409d6f0 2023-09-30T22:30:48.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.318+0000 7ffae17fa700 1 -- 172.21.15.16:0/1598294242 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7ffad8021070 con 0x7ffae409d6f0 2023-09-30T22:30:48.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.323+0000 7ffae17fa700 1 -- 172.21.15.16:0/1598294242 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ffad8010970 con 0x7ffae409d6f0 2023-09-30T22:30:48.324 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:48.376 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:48.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.435+0000 7f66b861d700 1 Processor -- start 2023-09-30T22:30:48.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.435+0000 7f66b861d700 1 -- start start 2023-09-30T22:30:48.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.436+0000 7f66b861d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f66b01338a0 0x7f66b0133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:48.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.436+0000 7f66b861d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f66b009de50 0x7f66b009e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:48.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.436+0000 7f66b861d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f66b009e760 0x7f66b013a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:48.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.436+0000 7f66b861d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66b006fef0 con 0x7f66b01338a0 2023-09-30T22:30:48.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.436+0000 7f66b861d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66b006ef30 con 0x7f66b009de50 2023-09-30T22:30:48.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.436+0000 7f66b861d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f66b01334b0 con 0x7f66b009e760 2023-09-30T22:30:48.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.436+0000 7f66b6bba700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f66b009e760 0x7f66b013a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:48.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.436+0000 7f66b6bba700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f66b009e760 0x7f66b013a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35352/0 (socket says 172.21.15.16:35352) 2023-09-30T22:30:48.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.436+0000 7f66b6bba700 1 -- 172.21.15.16:0/693059268 learned_addr learned my addr 172.21.15.16:0/693059268 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:48.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.436+0000 7f66b5bb8700 1 --2- 172.21.15.16:0/693059268 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f66b009de50 0x7f66b009e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:48.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.436+0000 7f66b63b9700 1 --2- 172.21.15.16:0/693059268 >> v2:172.21.15.16:3300/0 conn(0x7f66b01338a0 0x7f66b0133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:48.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.437+0000 7f66b5bb8700 1 -- 172.21.15.16:0/693059268 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f66b009e760 msgr2=0x7f66b013a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:48.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.437+0000 7f66b5bb8700 1 --2- 172.21.15.16:0/693059268 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f66b009e760 0x7f66b013a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.437+0000 7f66b5bb8700 1 -- 172.21.15.16:0/693059268 >> v2:172.21.15.16:3300/0 conn(0x7f66b01338a0 msgr2=0x7f66b0133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:48.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.437+0000 7f66b5bb8700 1 --2- 172.21.15.16:0/693059268 >> v2:172.21.15.16:3300/0 conn(0x7f66b01338a0 0x7f66b0133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.437+0000 7f66b5bb8700 1 -- 172.21.15.16:0/693059268 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b006fc30 con 0x7f66b009de50 2023-09-30T22:30:48.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.437+0000 7f66b63b9700 1 --2- 172.21.15.16:0/693059268 >> v2:172.21.15.16:3300/0 conn(0x7f66b01338a0 0x7f66b0133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:48.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.437+0000 7f66b5bb8700 1 --2- 172.21.15.16:0/693059268 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f66b009de50 0x7f66b009e220 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f66a0009840 tx=0x7f66a0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82944e4022ebaffc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:48.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.437+0000 7f66b53b7700 1 -- 172.21.15.16:0/693059268 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a0011830 con 0x7f66b009de50 2023-09-30T22:30:48.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.437+0000 7f66b53b7700 1 -- 172.21.15.16:0/693059268 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66a0011990 con 0x7f66b009de50 2023-09-30T22:30:48.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.437+0000 7f66b53b7700 1 -- 172.21.15.16:0/693059268 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a0011d20 con 0x7f66b009de50 2023-09-30T22:30:48.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.438+0000 7f66b861d700 1 -- 172.21.15.16:0/693059268 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f66b009de50 msgr2=0x7f66b009e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:48.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.438+0000 7f66b861d700 1 --2- 172.21.15.16:0/693059268 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f66b009de50 0x7f66b009e220 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f66a0009840 tx=0x7f66a0009b50 comp rx=0 tx=0).stop 2023-09-30T22:30:48.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.438+0000 7f66b861d700 1 -- 172.21.15.16:0/693059268 shutdown_connections 2023-09-30T22:30:48.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.438+0000 7f66b861d700 1 --2- 172.21.15.16:0/693059268 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f66b009e760 0x7f66b013a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.438+0000 7f66b861d700 1 --2- 172.21.15.16:0/693059268 >> v2:172.21.15.16:3300/0 conn(0x7f66b01338a0 0x7f66b0133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.438+0000 7f66b861d700 1 --2- 172.21.15.16:0/693059268 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f66b009de50 0x7f66b009e220 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.438+0000 7f66b861d700 1 -- 172.21.15.16:0/693059268 >> 172.21.15.16:0/693059268 conn(0x7f66b00a2aa0 msgr2=0x7f66b00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:48.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.438+0000 7f66b861d700 1 -- 172.21.15.16:0/693059268 shutdown_connections 2023-09-30T22:30:48.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.438+0000 7f66b861d700 1 -- 172.21.15.16:0/693059268 wait complete. 2023-09-30T22:30:48.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.439+0000 7f66b861d700 1 Processor -- start 2023-09-30T22:30:48.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.439+0000 7f66b861d700 1 -- start start 2023-09-30T22:30:48.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.440+0000 7f66b861d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f66b009de50 0x7f66b011d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:48.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.440+0000 7f66b861d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66b009e760 0x7f66b011de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:48.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.440+0000 7f66b861d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f66b01338a0 0x7f66b011e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:48.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.440+0000 7f66b861d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b011fae0 con 0x7f66b009e760 2023-09-30T22:30:48.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.440+0000 7f66b861d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66b01213d0 con 0x7f66b01338a0 2023-09-30T22:30:48.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.440+0000 7f66b861d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f66b006f640 con 0x7f66b009de50 2023-09-30T22:30:48.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.440+0000 7f66b63b9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f66b009de50 0x7f66b011d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:48.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.440+0000 7f66b63b9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f66b009de50 0x7f66b011d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35358/0 (socket says 172.21.15.16:35358) 2023-09-30T22:30:48.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.440+0000 7f66b63b9700 1 -- 172.21.15.16:0/531690844 learned_addr learned my addr 172.21.15.16:0/531690844 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:48.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.440+0000 7f66b6bba700 1 --2- 172.21.15.16:0/531690844 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f66b01338a0 0x7f66b011e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:48.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.440+0000 7f66b5bb8700 1 --2- 172.21.15.16:0/531690844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66b009e760 0x7f66b011de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:48.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.440+0000 7f66b6bba700 1 -- 172.21.15.16:0/531690844 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f66b009de50 msgr2=0x7f66b011d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:48.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.440+0000 7f66b6bba700 1 --2- 172.21.15.16:0/531690844 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f66b009de50 0x7f66b011d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.440+0000 7f66b6bba700 1 -- 172.21.15.16:0/531690844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66b009e760 msgr2=0x7f66b011de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:48.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.440+0000 7f66b6bba700 1 --2- 172.21.15.16:0/531690844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66b009e760 0x7f66b011de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.440+0000 7f66b6bba700 1 -- 172.21.15.16:0/531690844 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66ac0094a0 con 0x7f66b01338a0 2023-09-30T22:30:48.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.441+0000 7f66b6bba700 1 --2- 172.21.15.16:0/531690844 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f66b01338a0 0x7f66b011e3c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f66a800efc0 tx=0x7f66a800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:48.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.441+0000 7f66a6ffd700 1 -- 172.21.15.16:0/531690844 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a8003c20 con 0x7f66b01338a0 2023-09-30T22:30:48.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.441+0000 7f66a6ffd700 1 -- 172.21.15.16:0/531690844 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66a800f040 con 0x7f66b01338a0 2023-09-30T22:30:48.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.441+0000 7f66a6ffd700 1 -- 172.21.15.16:0/531690844 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a8009e60 con 0x7f66b01338a0 2023-09-30T22:30:48.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.441+0000 7f66b861d700 1 -- 172.21.15.16:0/531690844 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66a00094a0 con 0x7f66b01338a0 2023-09-30T22:30:48.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.441+0000 7f66b861d700 1 -- 172.21.15.16:0/531690844 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66b0130ca0 con 0x7f66b01338a0 2023-09-30T22:30:48.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.442+0000 7f66a6ffd700 1 -- 172.21.15.16:0/531690844 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f66a800c880 con 0x7f66b01338a0 2023-09-30T22:30:48.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.442+0000 7f66a6ffd700 1 -- 172.21.15.16:0/531690844 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f66a803f960 con 0x7f66b01338a0 2023-09-30T22:30:48.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.442+0000 7f66b861d700 1 -- 172.21.15.16:0/531690844 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66b00a0780 con 0x7f66b01338a0 2023-09-30T22:30:48.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.451+0000 7f66a6ffd700 1 -- 172.21.15.16:0/531690844 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f66a800cb30 con 0x7f66b01338a0 2023-09-30T22:30:48.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.470+0000 7fd4d0c6c700 1 Processor -- start 2023-09-30T22:30:48.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.470+0000 7fd4d0c6c700 1 -- start start 2023-09-30T22:30:48.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.471+0000 7fd4d0c6c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4cc09d6f0 0x7fd4cc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:48.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.471+0000 7fd4d0c6c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd4cc09e000 0x7fd4cc1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:48.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.471+0000 7fd4d0c6c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd4cc09e410 0x7fd4cc133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:48.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.471+0000 7fd4d0c6c700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4cc06b460 con 0x7fd4cc09e000 2023-09-30T22:30:48.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.471+0000 7fd4d0c6c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4cc09b120 con 0x7fd4cc09d6f0 2023-09-30T22:30:48.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.471+0000 7fd4d0c6c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd4cc09a7e0 con 0x7fd4cc09e410 2023-09-30T22:30:48.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.471+0000 7fd4c9d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd4cc09e000 0x7fd4cc1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:48.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.471+0000 7fd4c9d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd4cc09e000 0x7fd4cc1339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43694/0 (socket says 172.21.15.16:43694) 2023-09-30T22:30:48.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.471+0000 7fd4c9d9b700 1 -- 172.21.15.16:0/3797440648 learned_addr learned my addr 172.21.15.16:0/3797440648 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:48.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.471+0000 7fd4cad9d700 1 --2- 172.21.15.16:0/3797440648 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd4cc09e410 0x7fd4cc133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:48.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.471+0000 7fd4ca59c700 1 --2- 172.21.15.16:0/3797440648 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4cc09d6f0 0x7fd4cc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:48.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.471+0000 7fd4c9d9b700 1 -- 172.21.15.16:0/3797440648 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd4cc09e410 msgr2=0x7fd4cc133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:48.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.471+0000 7fd4c9d9b700 1 --2- 172.21.15.16:0/3797440648 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd4cc09e410 0x7fd4cc133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.471+0000 7fd4c9d9b700 1 -- 172.21.15.16:0/3797440648 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4cc09d6f0 msgr2=0x7fd4cc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:48.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.471+0000 7fd4c9d9b700 1 --2- 172.21.15.16:0/3797440648 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4cc09d6f0 0x7fd4cc09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.471+0000 7fd4c9d9b700 1 -- 172.21.15.16:0/3797440648 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4cc09a500 con 0x7fd4cc09e000 2023-09-30T22:30:48.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.472+0000 7fd4c9d9b700 1 --2- 172.21.15.16:0/3797440648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4cc09e000 0x7fd4cc1339f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd4c00098f0 tx=0x7fd4c0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3be032e108d898c1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:48.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.472+0000 7fd4c959a700 1 -- 172.21.15.16:0/3797440648 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c000b8a0 con 0x7fd4cc09e000 2023-09-30T22:30:48.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.472+0000 7fd4c959a700 1 -- 172.21.15.16:0/3797440648 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4c000ba00 con 0x7fd4cc09e000 2023-09-30T22:30:48.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.472+0000 7fd4c959a700 1 -- 172.21.15.16:0/3797440648 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c000bd90 con 0x7fd4cc09e000 2023-09-30T22:30:48.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.472+0000 7fd4d0c6c700 1 -- 172.21.15.16:0/3797440648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4cc09e000 msgr2=0x7fd4cc1339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:48.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.472+0000 7fd4d0c6c700 1 --2- 172.21.15.16:0/3797440648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4cc09e000 0x7fd4cc1339f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd4c00098f0 tx=0x7fd4c0009c00 comp rx=0 tx=0).stop 2023-09-30T22:30:48.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.473+0000 7fd4d0c6c700 1 -- 172.21.15.16:0/3797440648 shutdown_connections 2023-09-30T22:30:48.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.473+0000 7fd4d0c6c700 1 --2- 172.21.15.16:0/3797440648 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4cc09d6f0 0x7fd4cc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.473+0000 7fd4d0c6c700 1 --2- 172.21.15.16:0/3797440648 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd4cc09e410 0x7fd4cc133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.473+0000 7fd4d0c6c700 1 --2- 172.21.15.16:0/3797440648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4cc09e000 0x7fd4cc1339f0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.473+0000 7fd4d0c6c700 1 -- 172.21.15.16:0/3797440648 >> 172.21.15.16:0/3797440648 conn(0x7fd4cc066f50 msgr2=0x7fd4cc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:48.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.473+0000 7fd4d0c6c700 1 -- 172.21.15.16:0/3797440648 shutdown_connections 2023-09-30T22:30:48.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.473+0000 7fd4d0c6c700 1 -- 172.21.15.16:0/3797440648 wait complete. 2023-09-30T22:30:48.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.474+0000 7fd4d0c6c700 1 Processor -- start 2023-09-30T22:30:48.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.474+0000 7fd4d0c6c700 1 -- start start 2023-09-30T22:30:48.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.474+0000 7fd4d0c6c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd4cc09d6f0 0x7fd4cc1a6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:48.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.475+0000 7fd4d0c6c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4cc09e000 0x7fd4cc1a6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:48.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.475+0000 7fd4d0c6c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4cc09e410 0x7fd4cc1a6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:48.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.475+0000 7fd4d0c6c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4cc1a7520 con 0x7fd4cc09e410 2023-09-30T22:30:48.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.475+0000 7fd4d0c6c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4cc1a9b20 con 0x7fd4cc09e000 2023-09-30T22:30:48.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.475+0000 7fd4d0c6c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd4cc1a8230 con 0x7fd4cc09d6f0 2023-09-30T22:30:48.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.475+0000 7fd4ca59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd4cc09d6f0 0x7fd4cc1a6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:48.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.475+0000 7fd4ca59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd4cc09d6f0 0x7fd4cc1a6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35362/0 (socket says 172.21.15.16:35362) 2023-09-30T22:30:48.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.475+0000 7fd4ca59c700 1 -- 172.21.15.16:0/127385530 learned_addr learned my addr 172.21.15.16:0/127385530 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:48.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.475+0000 7fd4cad9d700 1 --2- 172.21.15.16:0/127385530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4cc09e410 0x7fd4cc1a6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:48.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.475+0000 7fd4c9d9b700 1 --2- 172.21.15.16:0/127385530 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4cc09e000 0x7fd4cc1a6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:48.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.475+0000 7fd4cad9d700 1 -- 172.21.15.16:0/127385530 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd4cc09d6f0 msgr2=0x7fd4cc1a6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:48.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.475+0000 7fd4cad9d700 1 --2- 172.21.15.16:0/127385530 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd4cc09d6f0 0x7fd4cc1a6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.475+0000 7fd4cad9d700 1 -- 172.21.15.16:0/127385530 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4cc09e000 msgr2=0x7fd4cc1a6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:48.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.475+0000 7fd4cad9d700 1 --2- 172.21.15.16:0/127385530 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4cc09e000 0x7fd4cc1a6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.475+0000 7fd4cad9d700 1 -- 172.21.15.16:0/127385530 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4c0009550 con 0x7fd4cc09e410 2023-09-30T22:30:48.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.475+0000 7fd4cad9d700 1 --2- 172.21.15.16:0/127385530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4cc09e410 0x7fd4cc1a6ad0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4007dc0 tx=0x7fd4c400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:48.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.476+0000 7fd4baffd700 1 -- 172.21.15.16:0/127385530 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4018070 con 0x7fd4cc09e410 2023-09-30T22:30:48.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.476+0000 7fd4baffd700 1 -- 172.21.15.16:0/127385530 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4c400eb30 con 0x7fd4cc09e410 2023-09-30T22:30:48.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.476+0000 7fd4baffd700 1 -- 172.21.15.16:0/127385530 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c400eec0 con 0x7fd4cc09e410 2023-09-30T22:30:48.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.476+0000 7fd4d0c6c700 1 -- 172.21.15.16:0/127385530 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4cc1a8ef0 con 0x7fd4cc09e410 2023-09-30T22:30:48.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.476+0000 7fd4d0c6c700 1 -- 172.21.15.16:0/127385530 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4cc09c710 con 0x7fd4cc09e410 2023-09-30T22:30:48.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.477+0000 7fd4d0c6c700 1 -- 172.21.15.16:0/127385530 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4cc074ea0 con 0x7fd4cc09e410 2023-09-30T22:30:48.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.477+0000 7fd4baffd700 1 -- 172.21.15.16:0/127385530 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd4c4003a20 con 0x7fd4cc09e410 2023-09-30T22:30:48.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.478+0000 7fd4baffd700 1 -- 172.21.15.16:0/127385530 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd4c401cd00 con 0x7fd4cc09e410 2023-09-30T22:30:48.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.487+0000 7fd4baffd700 1 -- 172.21.15.16:0/127385530 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd4c4009950 con 0x7fd4cc09e410 2023-09-30T22:30:48.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.531+0000 7f4523c33700 1 Processor -- start 2023-09-30T22:30:48.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.532+0000 7f4523c33700 1 -- start start 2023-09-30T22:30:48.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.533+0000 7f4523c33700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f451c0a0a30 0x7f451c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:48.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.533+0000 7f4523c33700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f451c09d630 0x7f451c0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:48.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.533+0000 7f4523c33700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f451c133930 0x7f451c133db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:48.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.533+0000 7f4523c33700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f451c0a2a70 con 0x7f451c0a0a30 2023-09-30T22:30:48.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.533+0000 7f4523c33700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f451c0a01d0 con 0x7f451c133930 2023-09-30T22:30:48.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.533+0000 7f4523c33700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f451c09e2b0 con 0x7f451c09d630 2023-09-30T22:30:48.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.533+0000 7f45211ce700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f451c09d630 0x7f451c0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:48.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.533+0000 7f45211ce700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f451c09d630 0x7f451c0a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35372/0 (socket says 172.21.15.16:35372) 2023-09-30T22:30:48.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.533+0000 7f45211ce700 1 -- 172.21.15.16:0/2054144095 learned_addr learned my addr 172.21.15.16:0/2054144095 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:48.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.533+0000 7f45219cf700 1 --2- 172.21.15.16:0/2054144095 >> v2:172.21.15.16:3300/0 conn(0x7f451c0a0a30 0x7f451c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:48.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.533+0000 7f45221d0700 1 --2- 172.21.15.16:0/2054144095 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f451c133930 0x7f451c133db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:48.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.533+0000 7f45211ce700 1 -- 172.21.15.16:0/2054144095 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f451c133930 msgr2=0x7f451c133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:48.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.533+0000 7f45211ce700 1 --2- 172.21.15.16:0/2054144095 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f451c133930 0x7f451c133db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.533+0000 7f45211ce700 1 -- 172.21.15.16:0/2054144095 >> v2:172.21.15.16:3300/0 conn(0x7f451c0a0a30 msgr2=0x7f451c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:48.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.533+0000 7f45211ce700 1 --2- 172.21.15.16:0/2054144095 >> v2:172.21.15.16:3300/0 conn(0x7f451c0a0a30 0x7f451c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.533+0000 7f45211ce700 1 -- 172.21.15.16:0/2054144095 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f451c09dc50 con 0x7f451c09d630 2023-09-30T22:30:48.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.534+0000 7f45211ce700 1 --2- 172.21.15.16:0/2054144095 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f451c09d630 0x7f451c0a1340 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f450c0098f0 tx=0x7f450c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4f0983deb77894e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:48.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.534+0000 7f45209cd700 1 -- 172.21.15.16:0/2054144095 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450c011830 con 0x7f451c09d630 2023-09-30T22:30:48.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.534+0000 7f45209cd700 1 -- 172.21.15.16:0/2054144095 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f450c011990 con 0x7f451c09d630 2023-09-30T22:30:48.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.534+0000 7f45209cd700 1 -- 172.21.15.16:0/2054144095 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450c011d20 con 0x7f451c09d630 2023-09-30T22:30:48.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.535+0000 7f4523c33700 1 -- 172.21.15.16:0/2054144095 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f451c09d630 msgr2=0x7f451c0a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:48.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.535+0000 7f4523c33700 1 --2- 172.21.15.16:0/2054144095 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f451c09d630 0x7f451c0a1340 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f450c0098f0 tx=0x7f450c009c00 comp rx=0 tx=0).stop 2023-09-30T22:30:48.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.535+0000 7f4523c33700 1 -- 172.21.15.16:0/2054144095 shutdown_connections 2023-09-30T22:30:48.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.535+0000 7f4523c33700 1 --2- 172.21.15.16:0/2054144095 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f451c133930 0x7f451c133db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.535+0000 7f4523c33700 1 --2- 172.21.15.16:0/2054144095 >> v2:172.21.15.16:3300/0 conn(0x7f451c0a0a30 0x7f451c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.535+0000 7f4523c33700 1 --2- 172.21.15.16:0/2054144095 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f451c09d630 0x7f451c0a1340 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.535+0000 7f4523c33700 1 -- 172.21.15.16:0/2054144095 >> 172.21.15.16:0/2054144095 conn(0x7f451c0a3050 msgr2=0x7f451c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:48.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.535+0000 7f4523c33700 1 -- 172.21.15.16:0/2054144095 shutdown_connections 2023-09-30T22:30:48.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.535+0000 7f4523c33700 1 -- 172.21.15.16:0/2054144095 wait complete. 2023-09-30T22:30:48.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.536+0000 7f4523c33700 1 Processor -- start 2023-09-30T22:30:48.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.537+0000 7f4523c33700 1 -- start start 2023-09-30T22:30:48.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.537+0000 7f4523c33700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f451c09d630 0x7f451c12e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:48.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.537+0000 7f4523c33700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f451c0a0a30 0x7f451c12eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:48.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.537+0000 7f4523c33700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f451c133930 0x7f451c12f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:48.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.537+0000 7f4523c33700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f451c0a40c0 con 0x7f451c09d630 2023-09-30T22:30:48.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.537+0000 7f4523c33700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f451c130590 con 0x7f451c0a0a30 2023-09-30T22:30:48.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.537+0000 7f4523c33700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f451c131be0 con 0x7f451c133930 2023-09-30T22:30:48.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.537+0000 7f45211ce700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f451c0a0a30 0x7f451c12eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:48.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.538+0000 7f45211ce700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f451c0a0a30 0x7f451c12eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:40934/0 (socket says 172.21.15.16:40934) 2023-09-30T22:30:48.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.538+0000 7f45211ce700 1 -- 172.21.15.16:0/962636795 learned_addr learned my addr 172.21.15.16:0/962636795 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:48.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.538+0000 7f45219cf700 1 --2- 172.21.15.16:0/962636795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f451c09d630 0x7f451c12e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:48.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.538+0000 7f45221d0700 1 --2- 172.21.15.16:0/962636795 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f451c133930 0x7f451c12f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:48.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.538+0000 7f45219cf700 1 -- 172.21.15.16:0/962636795 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f451c133930 msgr2=0x7f451c12f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:48.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.538+0000 7f45219cf700 1 --2- 172.21.15.16:0/962636795 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f451c133930 0x7f451c12f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.538+0000 7f45219cf700 1 -- 172.21.15.16:0/962636795 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f451c0a0a30 msgr2=0x7f451c12eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:48.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.538+0000 7f45219cf700 1 --2- 172.21.15.16:0/962636795 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f451c0a0a30 0x7f451c12eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.538+0000 7f45219cf700 1 -- 172.21.15.16:0/962636795 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f450c009550 con 0x7f451c09d630 2023-09-30T22:30:48.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.538+0000 7f45211ce700 1 --2- 172.21.15.16:0/962636795 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f451c0a0a30 0x7f451c12eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:48.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.538+0000 7f45219cf700 1 --2- 172.21.15.16:0/962636795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f451c09d630 0x7f451c12e970 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f451800e900 tx=0x7f451800ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:48.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.539+0000 7f45127fc700 1 -- 172.21.15.16:0/962636795 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4518013680 con 0x7f451c09d630 2023-09-30T22:30:48.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.539+0000 7f45127fc700 1 -- 172.21.15.16:0/962636795 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45180137e0 con 0x7f451c09d630 2023-09-30T22:30:48.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.539+0000 7f45127fc700 1 -- 172.21.15.16:0/962636795 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4518013b50 con 0x7f451c09d630 2023-09-30T22:30:48.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.539+0000 7f4523c33700 1 -- 172.21.15.16:0/962636795 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f451c132120 con 0x7f451c09d630 2023-09-30T22:30:48.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.539+0000 7f4523c33700 1 -- 172.21.15.16:0/962636795 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f451c132b90 con 0x7f451c09d630 2023-09-30T22:30:48.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.540+0000 7f45127fc700 1 -- 172.21.15.16:0/962636795 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4518013cb0 con 0x7f451c09d630 2023-09-30T22:30:48.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.540+0000 7f4523c33700 1 -- 172.21.15.16:0/962636795 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f451c131f60 con 0x7f451c09d630 2023-09-30T22:30:48.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.540+0000 7f45127fc700 1 -- 172.21.15.16:0/962636795 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f451803f780 con 0x7f451c09d630 2023-09-30T22:30:48.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.545+0000 7f45127fc700 1 -- 172.21.15.16:0/962636795 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f451801bc10 con 0x7f451c09d630 2023-09-30T22:30:48.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.553+0000 7ffaeb247700 1 -- 172.21.15.16:0/1598294242 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ffae406c360 con 0x7ffae409d6f0 2023-09-30T22:30:48.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.554+0000 7ffae17fa700 1 -- 172.21.15.16:0/1598294242 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ffad802b070 con 0x7ffae409d6f0 2023-09-30T22:30:48.554 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:30:48.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.557+0000 7ffaeb247700 1 -- 172.21.15.16:0/1598294242 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffae409d6f0 msgr2=0x7ffae41a5e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:48.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.557+0000 7ffaeb247700 1 --2- 172.21.15.16:0/1598294242 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffae409d6f0 0x7ffae41a5e80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ffad80090b0 tx=0x7ffad8004750 comp rx=0 tx=0).stop 2023-09-30T22:30:48.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.557+0000 7ffaeb247700 1 -- 172.21.15.16:0/1598294242 shutdown_connections 2023-09-30T22:30:48.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.557+0000 7ffaeb247700 1 --2- 172.21.15.16:0/1598294242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffae409e410 0x7ffae41a6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.557+0000 7ffaeb247700 1 --2- 172.21.15.16:0/1598294242 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffae409d6f0 0x7ffae41a5e80 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.557+0000 7ffaeb247700 1 --2- 172.21.15.16:0/1598294242 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffae409e000 0x7ffae41a63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:48.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.557+0000 7ffaeb247700 1 -- 172.21.15.16:0/1598294242 >> 172.21.15.16:0/1598294242 conn(0x7ffae4066f50 msgr2=0x7ffae41baed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:48.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.557+0000 7ffaeb247700 1 -- 172.21.15.16:0/1598294242 shutdown_connections 2023-09-30T22:30:48.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.557+0000 7ffaeb247700 1 -- 172.21.15.16:0/1598294242 wait complete. 2023-09-30T22:30:48.572 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:30:48.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.694+0000 7f66b861d700 1 -- 172.21.15.16:0/531690844 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f66b0074ea0 con 0x7f66b01338a0 2023-09-30T22:30:49.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.695+0000 7f66a6ffd700 1 -- 172.21.15.16:0/531690844 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f66a8014e00 con 0x7f66b01338a0 2023-09-30T22:30:49.056 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:30:49.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.698+0000 7f66b861d700 1 -- 172.21.15.16:0/531690844 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f66b01338a0 msgr2=0x7f66b011e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:49.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.698+0000 7f66b861d700 1 --2- 172.21.15.16:0/531690844 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f66b01338a0 0x7f66b011e3c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f66a800efc0 tx=0x7f66a800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:30:49.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.698+0000 7f66b861d700 1 -- 172.21.15.16:0/531690844 shutdown_connections 2023-09-30T22:30:49.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.698+0000 7f66b861d700 1 --2- 172.21.15.16:0/531690844 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f66b01338a0 0x7f66b011e3c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:49.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.698+0000 7f66b861d700 1 --2- 172.21.15.16:0/531690844 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f66b009de50 0x7f66b011d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:49.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.698+0000 7f66b861d700 1 --2- 172.21.15.16:0/531690844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f66b009e760 0x7f66b011de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:49.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.698+0000 7f66b861d700 1 -- 172.21.15.16:0/531690844 >> 172.21.15.16:0/531690844 conn(0x7f66b00a2aa0 msgr2=0x7f66b009ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:49.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.698+0000 7f66b861d700 1 -- 172.21.15.16:0/531690844 shutdown_connections 2023-09-30T22:30:49.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.698+0000 7f66b861d700 1 -- 172.21.15.16:0/531690844 wait complete. 2023-09-30T22:30:49.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.699+0000 7fd4d0c6c700 1 -- 172.21.15.16:0/127385530 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd4cc1b93f0 con 0x7fd4cc09e410 2023-09-30T22:30:49.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.700+0000 7fd4baffd700 1 -- 172.21.15.16:0/127385530 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd4c401cb20 con 0x7fd4cc09e410 2023-09-30T22:30:49.058 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:30:49.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.702+0000 7fd4d0c6c700 1 -- 172.21.15.16:0/127385530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4cc09e410 msgr2=0x7fd4cc1a6ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:49.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.702+0000 7fd4d0c6c700 1 --2- 172.21.15.16:0/127385530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4cc09e410 0x7fd4cc1a6ad0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd4c4007dc0 tx=0x7fd4c400e3c0 comp rx=0 tx=0).stop 2023-09-30T22:30:49.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.702+0000 7fd4d0c6c700 1 -- 172.21.15.16:0/127385530 shutdown_connections 2023-09-30T22:30:49.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.702+0000 7fd4d0c6c700 1 --2- 172.21.15.16:0/127385530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4cc09e410 0x7fd4cc1a6ad0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:49.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.702+0000 7fd4d0c6c700 1 --2- 172.21.15.16:0/127385530 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd4cc09d6f0 0x7fd4cc1a6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:49.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.702+0000 7fd4d0c6c700 1 --2- 172.21.15.16:0/127385530 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4cc09e000 0x7fd4cc1a6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:49.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.702+0000 7fd4d0c6c700 1 -- 172.21.15.16:0/127385530 >> 172.21.15.16:0/127385530 conn(0x7fd4cc066f50 msgr2=0x7fd4cc1bae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:49.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.703+0000 7fd4d0c6c700 1 -- 172.21.15.16:0/127385530 shutdown_connections 2023-09-30T22:30:49.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.703+0000 7fd4d0c6c700 1 -- 172.21.15.16:0/127385530 wait complete. 2023-09-30T22:30:49.061 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:30:49.061 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:30:49.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.781+0000 7f4523c33700 1 -- 172.21.15.16:0/962636795 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f451c0747f0 con 0x7f451c09d630 2023-09-30T22:30:49.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.781+0000 7f45127fc700 1 -- 172.21.15.16:0/962636795 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4518021090 con 0x7f451c09d630 2023-09-30T22:30:49.062 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:30:49.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.784+0000 7f4523c33700 1 -- 172.21.15.16:0/962636795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f451c09d630 msgr2=0x7f451c12e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:49.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.784+0000 7f4523c33700 1 --2- 172.21.15.16:0/962636795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f451c09d630 0x7f451c12e970 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f451800e900 tx=0x7f451800ec10 comp rx=0 tx=0).stop 2023-09-30T22:30:49.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.784+0000 7f4523c33700 1 -- 172.21.15.16:0/962636795 shutdown_connections 2023-09-30T22:30:49.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.784+0000 7f4523c33700 1 --2- 172.21.15.16:0/962636795 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f451c133930 0x7f451c12f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:49.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.784+0000 7f4523c33700 1 --2- 172.21.15.16:0/962636795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f451c09d630 0x7f451c12e970 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:49.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.784+0000 7f4523c33700 1 --2- 172.21.15.16:0/962636795 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f451c0a0a30 0x7f451c12eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:49.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.784+0000 7f4523c33700 1 -- 172.21.15.16:0/962636795 >> 172.21.15.16:0/962636795 conn(0x7f451c0a3050 msgr2=0x7f451c09ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:49.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.784+0000 7f4523c33700 1 -- 172.21.15.16:0/962636795 shutdown_connections 2023-09-30T22:30:49.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:48.784+0000 7f4523c33700 1 -- 172.21.15.16:0/962636795 wait complete. 2023-09-30T22:30:49.064 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:30:49.573 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:49.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.738+0000 7fcc3f51b700 1 Processor -- start 2023-09-30T22:30:49.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.739+0000 7fcc3f51b700 1 -- start start 2023-09-30T22:30:49.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.739+0000 7fcc3f51b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc3809d6f0 0x7fcc3809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:49.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.739+0000 7fcc3f51b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcc3809e000 0x7fcc38130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:49.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.739+0000 7fcc3f51b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcc3809e410 0x7fcc38130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:49.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.739+0000 7fcc3f51b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc3809f520 con 0x7fcc3809e410 2023-09-30T22:30:49.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.739+0000 7fcc3f51b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc3809f130 con 0x7fcc3809e000 2023-09-30T22:30:49.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.739+0000 7fcc3f51b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcc3809e930 con 0x7fcc3809d6f0 2023-09-30T22:30:49.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.739+0000 7fcc3cab6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcc3809e000 0x7fcc38130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:49.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.739+0000 7fcc3d2b7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc3809d6f0 0x7fcc3809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:49.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.739+0000 7fcc3dab8700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcc3809e410 0x7fcc38130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:49.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.739+0000 7fcc3cab6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcc3809e000 0x7fcc38130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:40946/0 (socket says 172.21.15.16:40946) 2023-09-30T22:30:49.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.739+0000 7fcc3cab6700 1 -- 172.21.15.16:0/2840499314 learned_addr learned my addr 172.21.15.16:0/2840499314 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:49.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.739+0000 7fcc3d2b7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc3809d6f0 0x7fcc3809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35384/0 (socket says 172.21.15.16:35384) 2023-09-30T22:30:49.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.740+0000 7fcc3d2b7700 1 -- 172.21.15.16:0/2840499314 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcc3809e000 msgr2=0x7fcc38130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:49.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.740+0000 7fcc3d2b7700 1 --2- 172.21.15.16:0/2840499314 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcc3809e000 0x7fcc38130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:49.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.740+0000 7fcc3d2b7700 1 -- 172.21.15.16:0/2840499314 >> v2:172.21.15.16:3300/0 conn(0x7fcc3809e410 msgr2=0x7fcc38130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:49.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.740+0000 7fcc3d2b7700 1 --2- 172.21.15.16:0/2840499314 >> v2:172.21.15.16:3300/0 conn(0x7fcc3809e410 0x7fcc38130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:49.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.740+0000 7fcc3d2b7700 1 -- 172.21.15.16:0/2840499314 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc3809b2d0 con 0x7fcc3809d6f0 2023-09-30T22:30:49.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.740+0000 7fcc3dab8700 1 --2- 172.21.15.16:0/2840499314 >> v2:172.21.15.16:3300/0 conn(0x7fcc3809e410 0x7fcc38130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:49.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.740+0000 7fcc3cab6700 1 --2- 172.21.15.16:0/2840499314 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcc3809e000 0x7fcc38130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:49.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.740+0000 7fcc3d2b7700 1 --2- 172.21.15.16:0/2840499314 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc3809d6f0 0x7fcc3809dac0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fcc34009870 tx=0x7fcc34009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2623e673e54e624e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:49.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.740+0000 7fcc2ffff700 1 -- 172.21.15.16:0/2840499314 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc34011830 con 0x7fcc3809d6f0 2023-09-30T22:30:49.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.740+0000 7fcc2ffff700 1 -- 172.21.15.16:0/2840499314 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc34011990 con 0x7fcc3809d6f0 2023-09-30T22:30:49.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.740+0000 7fcc2ffff700 1 -- 172.21.15.16:0/2840499314 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc34011d20 con 0x7fcc3809d6f0 2023-09-30T22:30:49.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.740+0000 7fcc3f51b700 1 -- 172.21.15.16:0/2840499314 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc3809d6f0 msgr2=0x7fcc3809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:49.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.740+0000 7fcc3f51b700 1 --2- 172.21.15.16:0/2840499314 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc3809d6f0 0x7fcc3809dac0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fcc34009870 tx=0x7fcc34009b80 comp rx=0 tx=0).stop 2023-09-30T22:30:49.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.741+0000 7fcc3f51b700 1 -- 172.21.15.16:0/2840499314 shutdown_connections 2023-09-30T22:30:49.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.741+0000 7fcc3f51b700 1 --2- 172.21.15.16:0/2840499314 >> v2:172.21.15.16:3300/0 conn(0x7fcc3809e410 0x7fcc38130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:49.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.741+0000 7fcc3f51b700 1 --2- 172.21.15.16:0/2840499314 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc3809d6f0 0x7fcc3809dac0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:49.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.741+0000 7fcc3f51b700 1 --2- 172.21.15.16:0/2840499314 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcc3809e000 0x7fcc38130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:49.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.741+0000 7fcc3f51b700 1 -- 172.21.15.16:0/2840499314 >> 172.21.15.16:0/2840499314 conn(0x7fcc38066f50 msgr2=0x7fcc3809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:49.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.741+0000 7fcc3f51b700 1 -- 172.21.15.16:0/2840499314 shutdown_connections 2023-09-30T22:30:49.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.741+0000 7fcc3f51b700 1 -- 172.21.15.16:0/2840499314 wait complete. 2023-09-30T22:30:49.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.742+0000 7fcc3f51b700 1 Processor -- start 2023-09-30T22:30:49.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.742+0000 7fcc3f51b700 1 -- start start 2023-09-30T22:30:49.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.742+0000 7fcc3f51b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc3809d6f0 0x7fcc38120100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:49.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.742+0000 7fcc3f51b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc3809e000 0x7fcc38120640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:49.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.742+0000 7fcc3f51b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcc3809e410 0x7fcc38120b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:49.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.742+0000 7fcc3f51b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc381238f0 con 0x7fcc3809d6f0 2023-09-30T22:30:49.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.742+0000 7fcc3f51b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc380715c0 con 0x7fcc3809e410 2023-09-30T22:30:49.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.742+0000 7fcc3f51b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcc38123bd0 con 0x7fcc3809e000 2023-09-30T22:30:49.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.742+0000 7fcc3d2b7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc3809d6f0 0x7fcc38120100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:49.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.742+0000 7fcc3d2b7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc3809d6f0 0x7fcc38120100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43744/0 (socket says 172.21.15.16:43744) 2023-09-30T22:30:49.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.742+0000 7fcc3d2b7700 1 -- 172.21.15.16:0/3851914864 learned_addr learned my addr 172.21.15.16:0/3851914864 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:49.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.742+0000 7fcc3cab6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc3809e000 0x7fcc38120640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:49.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.742+0000 7fcc3dab8700 1 --2- 172.21.15.16:0/3851914864 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcc3809e410 0x7fcc38120b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:49.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.743+0000 7fcc3d2b7700 1 -- 172.21.15.16:0/3851914864 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc3809e000 msgr2=0x7fcc38120640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:49.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.743+0000 7fcc3d2b7700 1 --2- 172.21.15.16:0/3851914864 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc3809e000 0x7fcc38120640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:49.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.743+0000 7fcc3d2b7700 1 -- 172.21.15.16:0/3851914864 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcc3809e410 msgr2=0x7fcc38120b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:49.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.743+0000 7fcc3d2b7700 1 --2- 172.21.15.16:0/3851914864 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcc3809e410 0x7fcc38120b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:49.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.743+0000 7fcc3d2b7700 1 -- 172.21.15.16:0/3851914864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc28009550 con 0x7fcc3809d6f0 2023-09-30T22:30:49.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.743+0000 7fcc3cab6700 1 --2- 172.21.15.16:0/3851914864 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc3809e000 0x7fcc38120640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:49.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.743+0000 7fcc3d2b7700 1 --2- 172.21.15.16:0/3851914864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc3809d6f0 0x7fcc38120100 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fcc34000c00 tx=0x7fcc34004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:49.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.743+0000 7fcc2dffb700 1 -- 172.21.15.16:0/3851914864 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc34004450 con 0x7fcc3809d6f0 2023-09-30T22:30:49.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.743+0000 7fcc2dffb700 1 -- 172.21.15.16:0/3851914864 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc3401f070 con 0x7fcc3809d6f0 2023-09-30T22:30:49.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.743+0000 7fcc3f51b700 1 -- 172.21.15.16:0/3851914864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc30009a70 con 0x7fcc3809d6f0 2023-09-30T22:30:49.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.743+0000 7fcc2dffb700 1 -- 172.21.15.16:0/3851914864 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc340038c0 con 0x7fcc3809d6f0 2023-09-30T22:30:49.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.743+0000 7fcc3f51b700 1 -- 172.21.15.16:0/3851914864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc340094a0 con 0x7fcc3809d6f0 2023-09-30T22:30:49.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.744+0000 7fcc2dffb700 1 -- 172.21.15.16:0/3851914864 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcc34003a20 con 0x7fcc3809d6f0 2023-09-30T22:30:49.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.744+0000 7fcc3f51b700 1 -- 172.21.15.16:0/3851914864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc38123750 con 0x7fcc3809d6f0 2023-09-30T22:30:49.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.744+0000 7fcc2dffb700 1 -- 172.21.15.16:0/3851914864 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fcc34021070 con 0x7fcc3809d6f0 2023-09-30T22:30:49.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.749+0000 7fcc2dffb700 1 -- 172.21.15.16:0/3851914864 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcc38123750 con 0x7fcc3809d6f0 2023-09-30T22:30:49.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.954+0000 7fcc3f51b700 1 -- 172.21.15.16:0/3851914864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fcc38123750 con 0x7fcc3809d6f0 2023-09-30T22:30:50.224 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:50.225 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:50.225 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:50.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.955+0000 7fcc2dffb700 1 -- 172.21.15.16:0/3851914864 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcc38123750 con 0x7fcc3809d6f0 2023-09-30T22:30:50.226 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:30:50.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.957+0000 7fcc3f51b700 1 -- 172.21.15.16:0/3851914864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc3809d6f0 msgr2=0x7fcc38120100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:50.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.957+0000 7fcc3f51b700 1 --2- 172.21.15.16:0/3851914864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc3809d6f0 0x7fcc38120100 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fcc34000c00 tx=0x7fcc34004750 comp rx=0 tx=0).stop 2023-09-30T22:30:50.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.957+0000 7fcc3f51b700 1 -- 172.21.15.16:0/3851914864 shutdown_connections 2023-09-30T22:30:50.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.957+0000 7fcc3f51b700 1 --2- 172.21.15.16:0/3851914864 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcc3809e410 0x7fcc38120b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:50.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.957+0000 7fcc3f51b700 1 --2- 172.21.15.16:0/3851914864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc3809d6f0 0x7fcc38120100 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:50.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.957+0000 7fcc3f51b700 1 --2- 172.21.15.16:0/3851914864 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc3809e000 0x7fcc38120640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:50.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.957+0000 7fcc3f51b700 1 -- 172.21.15.16:0/3851914864 >> 172.21.15.16:0/3851914864 conn(0x7fcc38066f50 msgr2=0x7fcc3806a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:50.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.957+0000 7fcc3f51b700 1 -- 172.21.15.16:0/3851914864 shutdown_connections 2023-09-30T22:30:50.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:49.957+0000 7fcc3f51b700 1 -- 172.21.15.16:0/3851914864 wait complete. 2023-09-30T22:30:50.229 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:30:50.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.417+0000 7f8759917700 1 Processor -- start 2023-09-30T22:30:50.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.417+0000 7fbfefdbe700 1 Processor -- start 2023-09-30T22:30:51.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.417+0000 7f8759917700 1 -- start start 2023-09-30T22:30:51.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.418+0000 7f808fcc6700 1 Processor -- start 2023-09-30T22:30:51.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.418+0000 7fbfefdbe700 1 -- start start 2023-09-30T22:30:51.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.418+0000 7f808fcc6700 1 -- start start 2023-09-30T22:30:51.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.418+0000 7f8759917700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f875409ace0 0x7f8754138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:51.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.418+0000 7f8759917700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f875409e1d0 0x7f875413b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:51.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.418+0000 7f8759917700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f875409b150 con 0x7f875409d6f0 2023-09-30T22:30:51.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.418+0000 7f8759917700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f875409a500 con 0x7f875409ace0 2023-09-30T22:30:51.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.418+0000 7f8759917700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f875409f720 con 0x7f875409e1d0 2023-09-30T22:30:51.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.418+0000 7fbfefdbe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbfe809e850 0x7fbfe809ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:51.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.418+0000 7fbfefdbe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbfe809f160 0x7fbfe8133bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:51.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.418+0000 7f808fcc6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f808812cf60 0x7f808806f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:51.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.419+0000 7f808fcc6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8088130430 0x7f8088137c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:51.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.419+0000 7f808fcc6700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f808812cd90 con 0x7f808812f8b0 2023-09-30T22:30:51.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.419+0000 7f808fcc6700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f808812d330 con 0x7f8088130430 2023-09-30T22:30:51.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.419+0000 7f808fcc6700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8088071780 con 0x7f808812cf60 2023-09-30T22:30:51.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.419+0000 7f808d261700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f808812f8b0 0x7f808812fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40828/0 (socket says 172.21.15.16:40828) 2023-09-30T22:30:51.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.419+0000 7f808d261700 1 -- 172.21.15.16:0/1157132552 learned_addr learned my addr 172.21.15.16:0/1157132552 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:30:51.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.419+0000 7f808da62700 1 --2- 172.21.15.16:0/1157132552 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f808812cf60 0x7f808806f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:51.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.419+0000 7f808e263700 1 --2- 172.21.15.16:0/1157132552 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8088130430 0x7f8088137c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:51.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.418+0000 7f87537fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f875409e1d0 0x7f875413b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:51.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.418+0000 7f87537fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f875409e1d0 0x7f875413b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35402/0 (socket says 172.21.15.16:35402) 2023-09-30T22:30:51.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.418+0000 7f87537fe700 1 -- 172.21.15.16:0/2062202762 learned_addr learned my addr 172.21.15.16:0/2062202762 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:51.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.419+0000 7f87537fe700 1 -- 172.21.15.16:0/2062202762 >> v2:172.21.15.16:3300/0 conn(0x7f875409ace0 msgr2=0x7f8754138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:51.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.419+0000 7f87527fc700 1 --2- 172.21.15.16:0/2062202762 >> v2:172.21.15.16:3300/0 conn(0x7f875409ace0 0x7f8754138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:51.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.419+0000 7f87537fe700 1 --2- 172.21.15.16:0/2062202762 >> v2:172.21.15.16:3300/0 conn(0x7f875409ace0 0x7f8754138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.419+0000 7f87537fe700 1 -- 172.21.15.16:0/2062202762 >> v1:172.21.15.16:6789/0 conn(0x7f875409d6f0 legacy=0x7f875409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:51.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.419+0000 7f87537fe700 1 -- 172.21.15.16:0/2062202762 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f875406b490 con 0x7f875409e1d0 2023-09-30T22:30:51.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.419+0000 7f87527fc700 1 --2- 172.21.15.16:0/2062202762 >> v2:172.21.15.16:3300/0 conn(0x7f875409ace0 0x7f8754138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:30:51.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.418+0000 7fbfefdbe700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfe809f960 con 0x7fbfe8134130 2023-09-30T22:30:51.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.418+0000 7fbfefdbe700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe809ae60 con 0x7fbfe809f160 2023-09-30T22:30:51.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.418+0000 7fbfefdbe700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe806cb10 con 0x7fbfe809e850 2023-09-30T22:30:51.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.419+0000 7fbfedb5a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbfe809e850 0x7fbfe809ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:51.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.419+0000 7fbfedb5a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbfe809e850 0x7fbfe809ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35404/0 (socket says 172.21.15.16:35404) 2023-09-30T22:30:51.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.419+0000 7fbfedb5a700 1 -- 172.21.15.16:0/766274166 learned_addr learned my addr 172.21.15.16:0/766274166 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:51.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.419+0000 7fbfed359700 1 --2- 172.21.15.16:0/766274166 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbfe809f160 0x7fbfe8133bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:51.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.419+0000 7f808da62700 1 -- 172.21.15.16:0/1157132552 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8088130430 msgr2=0x7f8088137c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:51.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.419+0000 7f808da62700 1 --2- 172.21.15.16:0/1157132552 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8088130430 0x7f8088137c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.419+0000 7f808da62700 1 -- 172.21.15.16:0/1157132552 >> v1:172.21.15.16:6789/0 conn(0x7f808812f8b0 legacy=0x7f808812fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:51.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.419+0000 7f808da62700 1 -- 172.21.15.16:0/1157132552 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8088071a40 con 0x7f808812cf60 2023-09-30T22:30:51.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.419+0000 7fbfedb5a700 1 -- 172.21.15.16:0/766274166 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbfe809f160 msgr2=0x7fbfe8133bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:51.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.419+0000 7fbfedb5a700 1 --2- 172.21.15.16:0/766274166 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbfe809f160 0x7fbfe8133bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.419+0000 7fbfedb5a700 1 -- 172.21.15.16:0/766274166 >> v1:172.21.15.16:6789/0 conn(0x7fbfe8134130 legacy=0x7fbfe813aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:51.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.419+0000 7fbfedb5a700 1 -- 172.21.15.16:0/766274166 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfe806b990 con 0x7fbfe809e850 2023-09-30T22:30:51.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.419+0000 7fbfed359700 1 --2- 172.21.15.16:0/766274166 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbfe809f160 0x7fbfe8133bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:51.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.419+0000 7f87537fe700 1 --2- 172.21.15.16:0/2062202762 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f875409e1d0 0x7f875413b180 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f874c009e50 tx=0x7f874c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=450187653abb1f93 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:51.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.419+0000 7fbfedb5a700 1 --2- 172.21.15.16:0/766274166 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbfe809e850 0x7fbfe809ec20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4009920 tx=0x7fbfe4009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff989e36fb17b936 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:51.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.420+0000 7f8751ffb700 1 -- 172.21.15.16:0/2062202762 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874c00e070 con 0x7f875409e1d0 2023-09-30T22:30:51.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.420+0000 7f808da62700 1 --2- 172.21.15.16:0/1157132552 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f808812cf60 0x7f808806f5d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8084009870 tx=0x7f8084009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aadb13d77e7dbf3d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:51.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.420+0000 7f8751ffb700 1 -- 172.21.15.16:0/2062202762 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f874c003a20 con 0x7f875409e1d0 2023-09-30T22:30:51.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.420+0000 7f8751ffb700 1 -- 172.21.15.16:0/2062202762 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874c003db0 con 0x7f875409e1d0 2023-09-30T22:30:51.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.420+0000 7fbfecb58700 1 -- 172.21.15.16:0/766274166 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe4011830 con 0x7fbfe809e850 2023-09-30T22:30:51.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.420+0000 7fbfecb58700 1 -- 172.21.15.16:0/766274166 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfe4011990 con 0x7fbfe809e850 2023-09-30T22:30:51.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.420+0000 7f808ca60700 1 -- 172.21.15.16:0/1157132552 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808400c940 con 0x7f808812cf60 2023-09-30T22:30:51.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.420+0000 7fbfecb58700 1 -- 172.21.15.16:0/766274166 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe4011d20 con 0x7fbfe809e850 2023-09-30T22:30:51.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.420+0000 7f808ca60700 1 -- 172.21.15.16:0/1157132552 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f808400caa0 con 0x7f808812cf60 2023-09-30T22:30:51.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.420+0000 7f808ca60700 1 -- 172.21.15.16:0/1157132552 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808400ce30 con 0x7f808812cf60 2023-09-30T22:30:51.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.420+0000 7f8759917700 1 -- 172.21.15.16:0/2062202762 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f875409e1d0 msgr2=0x7f875413b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:51.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.420+0000 7f8759917700 1 --2- 172.21.15.16:0/2062202762 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f875409e1d0 0x7f875413b180 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f874c009e50 tx=0x7f874c00b330 comp rx=0 tx=0).stop 2023-09-30T22:30:51.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.421+0000 7f8759917700 1 -- 172.21.15.16:0/2062202762 shutdown_connections 2023-09-30T22:30:51.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.421+0000 7f8759917700 1 --2- 172.21.15.16:0/2062202762 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f875409e1d0 0x7f875413b180 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.421+0000 7f8759917700 1 --2- 172.21.15.16:0/2062202762 >> v2:172.21.15.16:3300/0 conn(0x7f875409ace0 0x7f8754138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.421+0000 7f8759917700 1 -- 172.21.15.16:0/2062202762 >> 172.21.15.16:0/2062202762 conn(0x7f8754066f50 msgr2=0x7f875409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:51.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.421+0000 7f8759917700 1 -- 172.21.15.16:0/2062202762 shutdown_connections 2023-09-30T22:30:51.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.421+0000 7f8759917700 1 -- 172.21.15.16:0/2062202762 wait complete. 2023-09-30T22:30:51.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.421+0000 7f808fcc6700 1 -- 172.21.15.16:0/1157132552 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f808812cf60 msgr2=0x7f808806f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:51.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.421+0000 7f808fcc6700 1 --2- 172.21.15.16:0/1157132552 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f808812cf60 0x7f808806f5d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8084009870 tx=0x7f8084009b80 comp rx=0 tx=0).stop 2023-09-30T22:30:51.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.421+0000 7fbfefdbe700 1 -- 172.21.15.16:0/766274166 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbfe809e850 msgr2=0x7fbfe809ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:51.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.421+0000 7fbfefdbe700 1 --2- 172.21.15.16:0/766274166 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbfe809e850 0x7fbfe809ec20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbfe4009920 tx=0x7fbfe4009c30 comp rx=0 tx=0).stop 2023-09-30T22:30:51.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.421+0000 7f808fcc6700 1 -- 172.21.15.16:0/1157132552 shutdown_connections 2023-09-30T22:30:51.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.421+0000 7f808fcc6700 1 --2- 172.21.15.16:0/1157132552 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8088130430 0x7f8088137c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.421+0000 7f808fcc6700 1 --2- 172.21.15.16:0/1157132552 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f808812cf60 0x7f808806f5d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.421+0000 7f808fcc6700 1 -- 172.21.15.16:0/1157132552 >> 172.21.15.16:0/1157132552 conn(0x7f80880a2aa0 msgr2=0x7f80880a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:51.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.421+0000 7fbfefdbe700 1 -- 172.21.15.16:0/766274166 shutdown_connections 2023-09-30T22:30:51.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.421+0000 7fbfefdbe700 1 --2- 172.21.15.16:0/766274166 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbfe809e850 0x7fbfe809ec20 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.421+0000 7fbfefdbe700 1 --2- 172.21.15.16:0/766274166 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbfe809f160 0x7fbfe8133bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.421+0000 7fbfefdbe700 1 -- 172.21.15.16:0/766274166 >> 172.21.15.16:0/766274166 conn(0x7fbfe80a3f70 msgr2=0x7fbfe80a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:51.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.421+0000 7f808fcc6700 1 -- 172.21.15.16:0/1157132552 shutdown_connections 2023-09-30T22:30:51.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.421+0000 7f808fcc6700 1 -- 172.21.15.16:0/1157132552 wait complete. 2023-09-30T22:30:51.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.421+0000 7fbfefdbe700 1 -- 172.21.15.16:0/766274166 shutdown_connections 2023-09-30T22:30:51.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.421+0000 7fbfefdbe700 1 -- 172.21.15.16:0/766274166 wait complete. 2023-09-30T22:30:51.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.422+0000 7f8759917700 1 Processor -- start 2023-09-30T22:30:51.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.422+0000 7f808fcc6700 1 Processor -- start 2023-09-30T22:30:51.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.422+0000 7f8759917700 1 -- start start 2023-09-30T22:30:51.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.422+0000 7fbfefdbe700 1 Processor -- start 2023-09-30T22:30:51.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.422+0000 7fbfefdbe700 1 -- start start 2023-09-30T22:30:51.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.423+0000 7f8759917700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f875409ace0 0x7f875411e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:51.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.423+0000 7f8759917700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f875409d6f0 0x7f875411ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:51.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.423+0000 7f8759917700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f875409e1d0 0x7f875411ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:51.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.423+0000 7f8759917700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8754120650 con 0x7f875409d6f0 2023-09-30T22:30:51.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.423+0000 7f8759917700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f875409c760 con 0x7f875409ace0 2023-09-30T22:30:51.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.423+0000 7f8759917700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f87540a1e90 con 0x7f875409e1d0 2023-09-30T22:30:51.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.422+0000 7f808fcc6700 1 -- start start 2023-09-30T22:30:51.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.423+0000 7fbfefdbe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfe809e850 0x7fbfe81c9810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:51.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.423+0000 7fbfefdbe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbfe809f160 0x7fbfe81c9d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:51.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.423+0000 7f808fcc6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f808812cf60 0x7f80881cce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:51.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.423+0000 7f808fcc6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f808812f8b0 0x7f80881cd390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:51.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.423+0000 7f808fcc6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8088130430 0x7f8088119d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:51.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.423+0000 7f808fcc6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808809a6d0 con 0x7f8088130430 2023-09-30T22:30:51.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.423+0000 7f808fcc6700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f808811bc60 con 0x7f808812f8b0 2023-09-30T22:30:51.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.423+0000 7f808fcc6700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f808811c4a0 con 0x7f808812cf60 2023-09-30T22:30:51.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.423+0000 7fbfefdbe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbfe8134130 0x7fbfe8127240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:51.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.423+0000 7fbfefdbe700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe8129f00 con 0x7fbfe809e850 2023-09-30T22:30:51.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.423+0000 7fbfefdbe700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe812a0e0 con 0x7fbfe8134130 2023-09-30T22:30:51.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.423+0000 7fbfefdbe700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbfe81c93e0 con 0x7fbfe809f160 2023-09-30T22:30:51.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.423+0000 7f87527fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f875409d6f0 0x7f875411ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:51.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.423+0000 7f87527fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f875409d6f0 0x7f875411ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43762/0 (socket says 172.21.15.16:43762) 2023-09-30T22:30:51.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.423+0000 7f87527fc700 1 -- 172.21.15.16:0/2787179674 learned_addr learned my addr 172.21.15.16:0/2787179674 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:51.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.423+0000 7f808d261700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f808812f8b0 0x7f80881cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:51.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.423+0000 7f808d261700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f808812f8b0 0x7f80881cd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:40992/0 (socket says 172.21.15.16:40992) 2023-09-30T22:30:51.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.423+0000 7f808d261700 1 -- 172.21.15.16:0/4258886934 learned_addr learned my addr 172.21.15.16:0/4258886934 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:51.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.423+0000 7fbfee35b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbfe8134130 0x7fbfe8127240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:51.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.423+0000 7fbfee35b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbfe8134130 0x7fbfe8127240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41006/0 (socket says 172.21.15.16:41006) 2023-09-30T22:30:51.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.423+0000 7fbfee35b700 1 -- 172.21.15.16:0/1146986391 learned_addr learned my addr 172.21.15.16:0/1146986391 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:51.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.424+0000 7f808e263700 1 --2- 172.21.15.16:0/4258886934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8088130430 0x7f8088119d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:51.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.424+0000 7fbfee35b700 1 -- 172.21.15.16:0/1146986391 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbfe809f160 msgr2=0x7fbfe81c9d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:51.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.424+0000 7fbfee35b700 1 --2- 172.21.15.16:0/1146986391 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbfe809f160 0x7fbfe81c9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.424+0000 7fbfee35b700 1 -- 172.21.15.16:0/1146986391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfe809e850 msgr2=0x7fbfe81c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:51.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.424+0000 7fbfee35b700 1 --2- 172.21.15.16:0/1146986391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfe809e850 0x7fbfe81c9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.424+0000 7fbfee35b700 1 -- 172.21.15.16:0/1146986391 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd80094a0 con 0x7fbfe8134130 2023-09-30T22:30:51.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.424+0000 7fbfee35b700 1 --2- 172.21.15.16:0/1146986391 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbfe8134130 0x7fbfe8127240 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbfe000cc80 tx=0x7fbfe000a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:51.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.424+0000 7f8752ffd700 1 --2- 172.21.15.16:0/2787179674 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f875409ace0 0x7f875411e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:51.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.424+0000 7f87527fc700 1 -- 172.21.15.16:0/2787179674 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f875409e1d0 msgr2=0x7f875411ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:51.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.424+0000 7f87527fc700 1 --2- 172.21.15.16:0/2787179674 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f875409e1d0 0x7f875411ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.424+0000 7f87527fc700 1 -- 172.21.15.16:0/2787179674 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f875409ace0 msgr2=0x7f875411e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:51.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.424+0000 7f87527fc700 1 --2- 172.21.15.16:0/2787179674 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f875409ace0 0x7f875411e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.424+0000 7f87527fc700 1 -- 172.21.15.16:0/2787179674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f874c009b30 con 0x7f875409d6f0 2023-09-30T22:30:51.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.424+0000 7f87527fc700 1 --2- 172.21.15.16:0/2787179674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f875409d6f0 0x7f875411ea00 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f874800c750 tx=0x7f874800cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:51.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.424+0000 7fbfde7fc700 1 -- 172.21.15.16:0/1146986391 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe000ad40 con 0x7fbfe8134130 2023-09-30T22:30:51.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.424+0000 7fbfde7fc700 1 -- 172.21.15.16:0/1146986391 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfe000ce80 con 0x7fbfe8134130 2023-09-30T22:30:51.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.425+0000 7f808d261700 1 -- 172.21.15.16:0/4258886934 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f808812cf60 msgr2=0x7f80881cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:51.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.425+0000 7f808d261700 1 --2- 172.21.15.16:0/4258886934 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f808812cf60 0x7f80881cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.425+0000 7f808d261700 1 -- 172.21.15.16:0/4258886934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8088130430 msgr2=0x7f8088119d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:51.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.425+0000 7f808d261700 1 --2- 172.21.15.16:0/4258886934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8088130430 0x7f8088119d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.425+0000 7f808d261700 1 -- 172.21.15.16:0/4258886934 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80840094a0 con 0x7f808812f8b0 2023-09-30T22:30:51.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.425+0000 7f808d261700 1 --2- 172.21.15.16:0/4258886934 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f808812f8b0 0x7f80881cd390 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f807800c500 tx=0x7f807800c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:51.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.425+0000 7f807e7fc700 1 -- 172.21.15.16:0/4258886934 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8078005d70 con 0x7f808812f8b0 2023-09-30T22:30:51.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.424+0000 7f873bfff700 1 -- 172.21.15.16:0/2787179674 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8748011800 con 0x7f875409d6f0 2023-09-30T22:30:51.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.424+0000 7f873bfff700 1 -- 172.21.15.16:0/2787179674 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8748011960 con 0x7f875409d6f0 2023-09-30T22:30:51.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.424+0000 7f8759917700 1 -- 172.21.15.16:0/2787179674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87541219a0 con 0x7f875409d6f0 2023-09-30T22:30:51.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.424+0000 7f873bfff700 1 -- 172.21.15.16:0/2787179674 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8748011cd0 con 0x7f875409d6f0 2023-09-30T22:30:51.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.424+0000 7f8759917700 1 -- 172.21.15.16:0/2787179674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8754120820 con 0x7f875409d6f0 2023-09-30T22:30:51.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.424+0000 7fbfde7fc700 1 -- 172.21.15.16:0/1146986391 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfe000aea0 con 0x7fbfe8134130 2023-09-30T22:30:51.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.425+0000 7f807e7fc700 1 -- 172.21.15.16:0/4258886934 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80780070d0 con 0x7f808812f8b0 2023-09-30T22:30:51.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.425+0000 7f808fcc6700 1 -- 172.21.15.16:0/4258886934 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f808811c760 con 0x7f808812f8b0 2023-09-30T22:30:51.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.425+0000 7f808fcc6700 1 -- 172.21.15.16:0/4258886934 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f808809a0e0 con 0x7f808812f8b0 2023-09-30T22:30:51.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.425+0000 7fbfefdbe700 1 -- 172.21.15.16:0/1146986391 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfe4009550 con 0x7fbfe8134130 2023-09-30T22:30:51.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.425+0000 7f873bfff700 1 -- 172.21.15.16:0/2787179674 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8748021350 con 0x7f875409d6f0 2023-09-30T22:30:51.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.426+0000 7fbfefdbe700 1 -- 172.21.15.16:0/1146986391 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfe81284b0 con 0x7fbfe8134130 2023-09-30T22:30:51.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.426+0000 7f873bfff700 1 -- 172.21.15.16:0/2787179674 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f874803f860 con 0x7f875409d6f0 2023-09-30T22:30:51.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.426+0000 7f8759917700 1 -- 172.21.15.16:0/2787179674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87540747f0 con 0x7f875409d6f0 2023-09-30T22:30:51.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.426+0000 7f807e7fc700 1 -- 172.21.15.16:0/4258886934 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8078007440 con 0x7f808812f8b0 2023-09-30T22:30:51.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.426+0000 7f807e7fc700 1 -- 172.21.15.16:0/4258886934 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f80780075e0 con 0x7f808812f8b0 2023-09-30T22:30:51.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.427+0000 7f807e7fc700 1 -- 172.21.15.16:0/4258886934 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f807803ecf0 con 0x7f808812f8b0 2023-09-30T22:30:51.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.426+0000 7fbfde7fc700 1 -- 172.21.15.16:0/1146986391 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbfe0003c40 con 0x7fbfe8134130 2023-09-30T22:30:51.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.427+0000 7fbfde7fc700 1 -- 172.21.15.16:0/1146986391 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fbfe003f780 con 0x7fbfe8134130 2023-09-30T22:30:51.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.427+0000 7fbfefdbe700 1 -- 172.21.15.16:0/1146986391 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfe80747f0 con 0x7fbfe8134130 2023-09-30T22:30:51.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.428+0000 7f808fcc6700 1 -- 172.21.15.16:0/4258886934 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80700052e0 con 0x7f808812f8b0 2023-09-30T22:30:51.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.432+0000 7f873bfff700 1 -- 172.21.15.16:0/2787179674 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8748011e30 con 0x7f875409d6f0 2023-09-30T22:30:51.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.432+0000 7fbfde7fc700 1 -- 172.21.15.16:0/1146986391 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbfe00198f0 con 0x7fbfe8134130 2023-09-30T22:30:51.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.441+0000 7f807e7fc700 1 -- 172.21.15.16:0/4258886934 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8078011d70 con 0x7f808812f8b0 2023-09-30T22:30:51.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.638+0000 7f8759917700 1 -- 172.21.15.16:0/2787179674 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f875406c360 con 0x7f875409d6f0 2023-09-30T22:30:51.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.639+0000 7f873bfff700 1 -- 172.21.15.16:0/2787179674 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f874800eae0 con 0x7f875409d6f0 2023-09-30T22:30:51.091 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:30:51.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.641+0000 7f8759917700 1 -- 172.21.15.16:0/2787179674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f875409d6f0 msgr2=0x7f875411ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:51.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.641+0000 7f8759917700 1 --2- 172.21.15.16:0/2787179674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f875409d6f0 0x7f875411ea00 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f874800c750 tx=0x7f874800cb10 comp rx=0 tx=0).stop 2023-09-30T22:30:51.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.641+0000 7f8759917700 1 -- 172.21.15.16:0/2787179674 shutdown_connections 2023-09-30T22:30:51.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.641+0000 7f8759917700 1 --2- 172.21.15.16:0/2787179674 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f875409ace0 0x7f875411e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.641+0000 7f8759917700 1 --2- 172.21.15.16:0/2787179674 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f875409e1d0 0x7f875411ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.641+0000 7f8759917700 1 --2- 172.21.15.16:0/2787179674 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f875409d6f0 0x7f875411ea00 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.641+0000 7f8759917700 1 -- 172.21.15.16:0/2787179674 >> 172.21.15.16:0/2787179674 conn(0x7f8754066f50 msgr2=0x7f87540a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:51.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.641+0000 7f8759917700 1 -- 172.21.15.16:0/2787179674 shutdown_connections 2023-09-30T22:30:51.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.641+0000 7f8759917700 1 -- 172.21.15.16:0/2787179674 wait complete. 2023-09-30T22:30:51.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.644+0000 7fbfefdbe700 1 -- 172.21.15.16:0/1146986391 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fbfe806bbb0 con 0x7fbfe8134130 2023-09-30T22:30:51.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.644+0000 7fbfde7fc700 1 -- 172.21.15.16:0/1146986391 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbfe0011380 con 0x7fbfe8134130 2023-09-30T22:30:51.093 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:30:51.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.646+0000 7fbfefdbe700 1 -- 172.21.15.16:0/1146986391 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbfe8134130 msgr2=0x7fbfe8127240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:51.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.646+0000 7fbfefdbe700 1 --2- 172.21.15.16:0/1146986391 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbfe8134130 0x7fbfe8127240 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbfe000cc80 tx=0x7fbfe000a340 comp rx=0 tx=0).stop 2023-09-30T22:30:51.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.646+0000 7fbfefdbe700 1 -- 172.21.15.16:0/1146986391 shutdown_connections 2023-09-30T22:30:51.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.646+0000 7fbfefdbe700 1 --2- 172.21.15.16:0/1146986391 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbfe8134130 0x7fbfe8127240 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.646+0000 7fbfefdbe700 1 --2- 172.21.15.16:0/1146986391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbfe809e850 0x7fbfe81c9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.646+0000 7fbfefdbe700 1 --2- 172.21.15.16:0/1146986391 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbfe809f160 0x7fbfe81c9d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.646+0000 7fbfefdbe700 1 -- 172.21.15.16:0/1146986391 >> 172.21.15.16:0/1146986391 conn(0x7fbfe80a3f70 msgr2=0x7fbfe8071b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:51.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.647+0000 7fbfefdbe700 1 -- 172.21.15.16:0/1146986391 shutdown_connections 2023-09-30T22:30:51.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.647+0000 7fbfefdbe700 1 -- 172.21.15.16:0/1146986391 wait complete. 2023-09-30T22:30:51.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.647+0000 7f808fcc6700 1 -- 172.21.15.16:0/4258886934 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8070005590 con 0x7f808812f8b0 2023-09-30T22:30:51.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.647+0000 7f807e7fc700 1 -- 172.21.15.16:0/4258886934 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8078011360 con 0x7f808812f8b0 2023-09-30T22:30:51.104 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:30:51.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.650+0000 7f808fcc6700 1 -- 172.21.15.16:0/4258886934 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f808812f8b0 msgr2=0x7f80881cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:51.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.650+0000 7f808fcc6700 1 --2- 172.21.15.16:0/4258886934 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f808812f8b0 0x7f80881cd390 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f807800c500 tx=0x7f807800c8c0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.654+0000 7f808fcc6700 1 -- 172.21.15.16:0/4258886934 shutdown_connections 2023-09-30T22:30:51.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.654+0000 7f808fcc6700 1 --2- 172.21.15.16:0/4258886934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8088130430 0x7f8088119d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.654+0000 7f808fcc6700 1 --2- 172.21.15.16:0/4258886934 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f808812cf60 0x7f80881cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.654+0000 7f808fcc6700 1 --2- 172.21.15.16:0/4258886934 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f808812f8b0 0x7f80881cd390 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.654+0000 7f808fcc6700 1 -- 172.21.15.16:0/4258886934 >> 172.21.15.16:0/4258886934 conn(0x7f80880a2aa0 msgr2=0x7f80880a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:51.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.654+0000 7f808fcc6700 1 -- 172.21.15.16:0/4258886934 shutdown_connections 2023-09-30T22:30:51.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:50.654+0000 7f808fcc6700 1 -- 172.21.15.16:0/4258886934 wait complete. 2023-09-30T22:30:51.107 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:30:51.108 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:30:51.108 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:30:51.230 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:51.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.375+0000 7f976726b700 1 Processor -- start 2023-09-30T22:30:51.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.375+0000 7f976726b700 1 -- start start 2023-09-30T22:30:51.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.376+0000 7f976726b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f976009d6f0 0x7f976009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:51.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.376+0000 7f976726b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f976009e440 0x7f9760134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:51.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.376+0000 7f976726b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f976009b120 con 0x7f976009e000 2023-09-30T22:30:51.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.376+0000 7f976726b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f976006b460 con 0x7f976009d6f0 2023-09-30T22:30:51.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.376+0000 7f976726b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f976009a7e0 con 0x7f976009e440 2023-09-30T22:30:51.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.376+0000 7f9765007700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f976009d6f0 0x7f976009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:51.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.376+0000 7f9765808700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f976009e440 0x7f9760134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:51.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.376+0000 7f9764806700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f976009e000 0x7f97601339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40836/0 (socket says 172.21.15.16:40836) 2023-09-30T22:30:51.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.376+0000 7f9764806700 1 -- 172.21.15.16:0/710753944 learned_addr learned my addr 172.21.15.16:0/710753944 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:30:51.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.376+0000 7f9765007700 1 -- 172.21.15.16:0/710753944 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f976009e440 msgr2=0x7f9760134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:51.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.376+0000 7f9765007700 1 --2- 172.21.15.16:0/710753944 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f976009e440 0x7f9760134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.376+0000 7f9765007700 1 -- 172.21.15.16:0/710753944 >> v1:172.21.15.16:6789/0 conn(0x7f976009e000 legacy=0x7f97601339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:51.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.376+0000 7f9765007700 1 -- 172.21.15.16:0/710753944 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f976009a500 con 0x7f976009d6f0 2023-09-30T22:30:51.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.376+0000 7f9753fff700 1 -- 172.21.15.16:0/710753944 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3276471412 0 0) 0x7f976009b120 con 0x7f976009e000 2023-09-30T22:30:51.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.376+0000 7f9765808700 1 --2- 172.21.15.16:0/710753944 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f976009e440 0x7f9760134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:51.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.376+0000 7f9765007700 1 --2- 172.21.15.16:0/710753944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f976009d6f0 0x7f976009dac0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9754009870 tx=0x7f9754009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6ee395628364d47 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:51.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.376+0000 7f9753fff700 1 -- 172.21.15.16:0/710753944 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9754011830 con 0x7f976009d6f0 2023-09-30T22:30:51.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.377+0000 7f9753fff700 1 -- 172.21.15.16:0/710753944 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9754011990 con 0x7f976009d6f0 2023-09-30T22:30:51.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.377+0000 7f9753fff700 1 -- 172.21.15.16:0/710753944 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9754011d20 con 0x7f976009d6f0 2023-09-30T22:30:51.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.377+0000 7f976726b700 1 -- 172.21.15.16:0/710753944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f976009d6f0 msgr2=0x7f976009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:51.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.377+0000 7f976726b700 1 --2- 172.21.15.16:0/710753944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f976009d6f0 0x7f976009dac0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9754009870 tx=0x7f9754009b80 comp rx=0 tx=0).stop 2023-09-30T22:30:51.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.377+0000 7f976726b700 1 -- 172.21.15.16:0/710753944 shutdown_connections 2023-09-30T22:30:51.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.377+0000 7f976726b700 1 --2- 172.21.15.16:0/710753944 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f976009e440 0x7f9760134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.377+0000 7f976726b700 1 --2- 172.21.15.16:0/710753944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f976009d6f0 0x7f976009dac0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.377+0000 7f976726b700 1 -- 172.21.15.16:0/710753944 >> 172.21.15.16:0/710753944 conn(0x7f9760066f50 msgr2=0x7f976009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:51.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.377+0000 7f976726b700 1 -- 172.21.15.16:0/710753944 shutdown_connections 2023-09-30T22:30:51.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.377+0000 7f976726b700 1 -- 172.21.15.16:0/710753944 wait complete. 2023-09-30T22:30:51.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.378+0000 7f976726b700 1 Processor -- start 2023-09-30T22:30:51.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.378+0000 7f976726b700 1 -- start start 2023-09-30T22:30:51.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.379+0000 7f976726b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f976009d6f0 0x7f97601a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:51.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.379+0000 7f976726b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f976009e000 0x7f97601a58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:51.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.379+0000 7f976726b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f976009e440 0x7f97601c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:51.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.379+0000 7f976726b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97601a8110 con 0x7f976009e000 2023-09-30T22:30:51.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.379+0000 7f976726b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97601b9180 con 0x7f976009e440 2023-09-30T22:30:51.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.379+0000 7f976726b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f97601a8310 con 0x7f976009d6f0 2023-09-30T22:30:51.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.379+0000 7f9765007700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f976009d6f0 0x7f97601a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:51.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.379+0000 7f9764806700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f976009e000 0x7f97601a58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:51.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.379+0000 7f9765007700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f976009d6f0 0x7f97601a5380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35468/0 (socket says 172.21.15.16:35468) 2023-09-30T22:30:51.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.379+0000 7f9765007700 1 -- 172.21.15.16:0/599225452 learned_addr learned my addr 172.21.15.16:0/599225452 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:51.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.379+0000 7f9765808700 1 --2- 172.21.15.16:0/599225452 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f976009e440 0x7f97601c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:51.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.379+0000 7f9765007700 1 -- 172.21.15.16:0/599225452 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f976009e440 msgr2=0x7f97601c7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:51.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.379+0000 7f9765007700 1 --2- 172.21.15.16:0/599225452 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f976009e440 0x7f97601c7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.379+0000 7f9765007700 1 -- 172.21.15.16:0/599225452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f976009e000 msgr2=0x7f97601a58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:51.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.379+0000 7f9765007700 1 --2- 172.21.15.16:0/599225452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f976009e000 0x7f97601a58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.379+0000 7f9765007700 1 -- 172.21.15.16:0/599225452 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9758009a70 con 0x7f976009d6f0 2023-09-30T22:30:51.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.379+0000 7f9764806700 1 --2- 172.21.15.16:0/599225452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f976009e000 0x7f97601a58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:51.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.379+0000 7f9765007700 1 --2- 172.21.15.16:0/599225452 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f976009d6f0 0x7f97601a5380 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9754000c00 tx=0x7f9754004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:51.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.380+0000 7f9751ffb700 1 -- 172.21.15.16:0/599225452 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97540045d0 con 0x7f976009d6f0 2023-09-30T22:30:51.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.380+0000 7f9751ffb700 1 -- 172.21.15.16:0/599225452 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f975401f070 con 0x7f976009d6f0 2023-09-30T22:30:51.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.380+0000 7f976726b700 1 -- 172.21.15.16:0/599225452 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97540094a0 con 0x7f976009d6f0 2023-09-30T22:30:51.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.380+0000 7f9751ffb700 1 -- 172.21.15.16:0/599225452 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97540038a0 con 0x7f976009d6f0 2023-09-30T22:30:51.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.380+0000 7f976726b700 1 -- 172.21.15.16:0/599225452 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97601a4d90 con 0x7f976009d6f0 2023-09-30T22:30:51.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.381+0000 7f9751ffb700 1 -- 172.21.15.16:0/599225452 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9754003a00 con 0x7f976009d6f0 2023-09-30T22:30:51.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.381+0000 7f9751ffb700 1 -- 172.21.15.16:0/599225452 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f9754021070 con 0x7f976009d6f0 2023-09-30T22:30:51.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.381+0000 7f976726b700 1 -- 172.21.15.16:0/599225452 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f976009c710 con 0x7f976009d6f0 2023-09-30T22:30:51.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.386+0000 7f9751ffb700 1 -- 172.21.15.16:0/599225452 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9754015020 con 0x7f976009d6f0 2023-09-30T22:30:51.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.589+0000 7f976726b700 1 -- 172.21.15.16:0/599225452 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f976009f420 con 0x7f976009d6f0 2023-09-30T22:30:51.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.589+0000 7f9751ffb700 1 -- 172.21.15.16:0/599225452 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f975402b020 con 0x7f976009d6f0 2023-09-30T22:30:51.589 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:30:51.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.591+0000 7f976726b700 1 -- 172.21.15.16:0/599225452 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f976009d6f0 msgr2=0x7f97601a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:51.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.591+0000 7f976726b700 1 --2- 172.21.15.16:0/599225452 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f976009d6f0 0x7f97601a5380 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9754000c00 tx=0x7f9754004750 comp rx=0 tx=0).stop 2023-09-30T22:30:51.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.592+0000 7f976726b700 1 -- 172.21.15.16:0/599225452 shutdown_connections 2023-09-30T22:30:51.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.592+0000 7f976726b700 1 --2- 172.21.15.16:0/599225452 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f976009e440 0x7f97601c7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.592+0000 7f976726b700 1 --2- 172.21.15.16:0/599225452 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f976009d6f0 0x7f97601a5380 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.592+0000 7f976726b700 1 --2- 172.21.15.16:0/599225452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f976009e000 0x7f97601a58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:51.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.592+0000 7f976726b700 1 -- 172.21.15.16:0/599225452 >> 172.21.15.16:0/599225452 conn(0x7f9760066f50 msgr2=0x7f97601bbab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:51.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.592+0000 7f976726b700 1 -- 172.21.15.16:0/599225452 shutdown_connections 2023-09-30T22:30:51.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:51.592+0000 7f976726b700 1 -- 172.21.15.16:0/599225452 wait complete. 2023-09-30T22:30:51.605 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:30:52.108 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:52.109 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:52.109 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:52.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.260+0000 7f7bc2b39700 1 Processor -- start 2023-09-30T22:30:52.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.260+0000 7f7bc2b39700 1 -- start start 2023-09-30T22:30:52.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.260+0000 7f9e1099a700 1 Processor -- start 2023-09-30T22:30:52.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.260+0000 7f9e1099a700 1 -- start start 2023-09-30T22:30:52.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.260+0000 7faaa31d4700 1 Processor -- start 2023-09-30T22:30:52.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.260+0000 7faaa31d4700 1 -- start start 2023-09-30T22:30:52.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.261+0000 7f7bc2b39700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7bbc09d6f0 0x7f7bbc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:52.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.261+0000 7f7bc2b39700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7bbc09e000 0x7f7bbc1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:52.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.261+0000 7f7bc2b39700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bbc09b120 con 0x7f7bbc09e410 2023-09-30T22:30:52.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.261+0000 7f7bc2b39700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bbc06b460 con 0x7f7bbc09e000 2023-09-30T22:30:52.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.261+0000 7f7bc2b39700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bbc09a7e0 con 0x7f7bbc09d6f0 2023-09-30T22:30:52.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.261+0000 7f7bc10d6700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7bbc09e410 0x7f7bbc133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40840/0 (socket says 172.21.15.16:40840) 2023-09-30T22:30:52.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.261+0000 7f7bc10d6700 1 -- 172.21.15.16:0/1222987724 learned_addr learned my addr 172.21.15.16:0/1222987724 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:30:52.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.261+0000 7f7bbb7fe700 1 -- 172.21.15.16:0/1222987724 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2571131149 0 0) 0x7f7bbc09b120 con 0x7f7bbc09e410 2023-09-30T22:30:52.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.261+0000 7f7bbb7fe700 1 -- 172.21.15.16:0/1222987724 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bac0035b0 con 0x7f7bbc09e410 2023-09-30T22:30:52.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.261+0000 7faaa31d4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faa9c1338a0 0x7faa9c133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:52.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.261+0000 7faaa31d4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faa9c09de50 0x7faa9c09e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:52.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.261+0000 7faaa31d4700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa9c06ef30 con 0x7faa9c09e760 2023-09-30T22:30:52.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.261+0000 7faaa31d4700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa9c06fef0 con 0x7faa9c09de50 2023-09-30T22:30:52.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.261+0000 7faaa31d4700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faa9c1334b0 con 0x7faa9c1338a0 2023-09-30T22:30:52.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.262+0000 7faa9bfff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faa9c09de50 0x7faa9c09e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:52.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.262+0000 7f7bc08d5700 1 --2- 172.21.15.16:0/1222987724 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7bbc09d6f0 0x7f7bbc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:52.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.261+0000 7f9e1099a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e0c09d6f0 0x7f9e0c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:52.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.261+0000 7f9e1099a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9e0c09e410 0x7f9e0c134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:52.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.261+0000 7f9e1099a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e0c09b120 con 0x7f9e0c09e000 2023-09-30T22:30:52.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.261+0000 7f9e1099a700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e0c06b460 con 0x7f9e0c09e410 2023-09-30T22:30:52.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.261+0000 7f9e1099a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0c09a7e0 con 0x7f9e0c09d6f0 2023-09-30T22:30:52.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.262+0000 7f7bbb7fe700 1 -- 172.21.15.16:0/1222987724 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4039120732 0 0) 0x7f7bac0035b0 con 0x7f7bbc09e410 2023-09-30T22:30:52.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.262+0000 7f7bbb7fe700 1 -- 172.21.15.16:0/1222987724 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7bbc09d6f0 msgr2=0x7f7bbc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:52.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.262+0000 7f7bbb7fe700 1 --2- 172.21.15.16:0/1222987724 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7bbc09d6f0 0x7f7bbc09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.262+0000 7f7bbb7fe700 1 -- 172.21.15.16:0/1222987724 >> v2:172.21.15.16:3300/0 conn(0x7f7bbc09e000 msgr2=0x7f7bbc1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:52.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.262+0000 7f7bbb7fe700 1 --2- 172.21.15.16:0/1222987724 >> v2:172.21.15.16:3300/0 conn(0x7f7bbc09e000 0x7f7bbc1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.262+0000 7f7bbb7fe700 1 -- 172.21.15.16:0/1222987724 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bbc09a500 con 0x7f7bbc09e410 2023-09-30T22:30:52.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.262+0000 7faa9bfff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faa9c09de50 0x7faa9c09e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43808/0 (socket says 172.21.15.16:43808) 2023-09-30T22:30:52.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.262+0000 7faa9bfff700 1 -- 172.21.15.16:0/4106999334 learned_addr learned my addr 172.21.15.16:0/4106999334 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:52.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.262+0000 7faaa0f70700 1 --2- 172.21.15.16:0/4106999334 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faa9c1338a0 0x7faa9c133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:52.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.262+0000 7f7bbb7fe700 1 -- 172.21.15.16:0/1222987724 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1074163509 0 0) 0x7f7bb40019f0 con 0x7f7bbc09e410 2023-09-30T22:30:52.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.262+0000 7f7bbb7fe700 1 -- 172.21.15.16:0/1222987724 >> v1:172.21.15.16:6789/0 conn(0x7f7bbc09e410 legacy=0x7f7bbc133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:52.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.262+0000 7f7bbb7fe700 1 --2- 172.21.15.16:0/1222987724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bac003a30 0x7f7bac003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:52.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.262+0000 7f7bbb7fe700 1 --2- 172.21.15.16:0/1222987724 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7bbc09d6f0 0x7f7bac004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:52.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.262+0000 7f7bbb7fe700 1 --2- 172.21.15.16:0/1222987724 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7bac004d90 0x7f7bac005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:52.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.262+0000 7f7bbb7fe700 1 -- 172.21.15.16:0/1222987724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bbc06b460 con 0x7f7bac003a30 2023-09-30T22:30:52.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.262+0000 7f7bbb7fe700 1 -- 172.21.15.16:0/1222987724 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bbc09a7e0 con 0x7f7bbc09d6f0 2023-09-30T22:30:52.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.262+0000 7f7bbb7fe700 1 -- 172.21.15.16:0/1222987724 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7bac00bbd0 con 0x7f7bac004d90 2023-09-30T22:30:52.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.262+0000 7faa9bfff700 1 -- 172.21.15.16:0/4106999334 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faa9c1338a0 msgr2=0x7faa9c133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:52.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.262+0000 7faa9bfff700 1 --2- 172.21.15.16:0/4106999334 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faa9c1338a0 0x7faa9c133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.262+0000 7faa9bfff700 1 -- 172.21.15.16:0/4106999334 >> v1:172.21.15.16:6789/0 conn(0x7faa9c09e760 legacy=0x7faa9c13a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:52.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.262+0000 7faa9bfff700 1 -- 172.21.15.16:0/4106999334 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa9c06fc30 con 0x7faa9c09de50 2023-09-30T22:30:52.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.262+0000 7f7bbbfff700 1 --2- 172.21.15.16:0/1222987724 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7bbc09d6f0 0x7f7bac004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:52.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.262+0000 7f9e09d9b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9e0c09e000 0x7f9e0c1339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40854/0 (socket says 172.21.15.16:40854) 2023-09-30T22:30:52.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.262+0000 7f9e09d9b700 1 -- 172.21.15.16:0/1141847751 learned_addr learned my addr 172.21.15.16:0/1141847751 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:30:52.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.262+0000 7f7bc10d6700 1 --2- 172.21.15.16:0/1222987724 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7bac004d90 0x7f7bac005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:52.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.262+0000 7f9e0a59c700 1 --2- 172.21.15.16:0/1141847751 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e0c09d6f0 0x7f9e0c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:52.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.262+0000 7f7bbbfff700 1 -- 172.21.15.16:0/1222987724 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7bac004d90 msgr2=0x7f7bac005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:52.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.262+0000 7f7bbbfff700 1 --2- 172.21.15.16:0/1222987724 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7bac004d90 0x7f7bac005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.262+0000 7f7bbbfff700 1 -- 172.21.15.16:0/1222987724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bac003a30 msgr2=0x7f7bac003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:52.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.262+0000 7f7bbbfff700 1 --2- 172.21.15.16:0/1222987724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bac003a30 0x7f7bac003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.262+0000 7f7bbbfff700 1 -- 172.21.15.16:0/1222987724 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bac0035b0 con 0x7f7bbc09d6f0 2023-09-30T22:30:52.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.263+0000 7faa9bfff700 1 --2- 172.21.15.16:0/4106999334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa9c09de50 0x7faa9c09e220 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7faa880098f0 tx=0x7faa88009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=149f4621f6ee02a2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:52.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.263+0000 7f9e0a59c700 1 -- 172.21.15.16:0/1141847751 >> v2:172.21.15.16:3300/0 conn(0x7f9e0c09e410 msgr2=0x7f9e0c134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:52.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.263+0000 7f9e0a59c700 1 --2- 172.21.15.16:0/1141847751 >> v2:172.21.15.16:3300/0 conn(0x7f9e0c09e410 0x7f9e0c134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.263+0000 7f9e0a59c700 1 -- 172.21.15.16:0/1141847751 >> v1:172.21.15.16:6789/0 conn(0x7f9e0c09e000 legacy=0x7f9e0c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:52.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.263+0000 7f9e0a59c700 1 -- 172.21.15.16:0/1141847751 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e0c09a500 con 0x7f9e0c09d6f0 2023-09-30T22:30:52.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.263+0000 7faa9b7fe700 1 -- 172.21.15.16:0/4106999334 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8800b8a0 con 0x7faa9c09de50 2023-09-30T22:30:52.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.263+0000 7f7bbbfff700 1 --2- 172.21.15.16:0/1222987724 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7bbc09d6f0 0x7f7bac004850 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7ba80098e0 tx=0x7f7bac005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b61be8b06c265c82 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:52.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.263+0000 7f7bbb7fe700 1 -- 172.21.15.16:0/1222987724 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba8011620 con 0x7f7bbc09d6f0 2023-09-30T22:30:52.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.263+0000 7faa9b7fe700 1 -- 172.21.15.16:0/4106999334 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa8800ba00 con 0x7faa9c09de50 2023-09-30T22:30:52.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.263+0000 7faa9b7fe700 1 -- 172.21.15.16:0/4106999334 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8800bd90 con 0x7faa9c09de50 2023-09-30T22:30:52.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.263+0000 7f7bbb7fe700 1 -- 172.21.15.16:0/1222987724 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba80117a0 con 0x7f7bbc09d6f0 2023-09-30T22:30:52.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.263+0000 7f7bbb7fe700 1 -- 172.21.15.16:0/1222987724 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba8011b50 con 0x7f7bbc09d6f0 2023-09-30T22:30:52.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.263+0000 7f9e0a59c700 1 --2- 172.21.15.16:0/1141847751 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e0c09d6f0 0x7f9e0c09dac0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc009870 tx=0x7f9dfc009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e3dc49a9977ef8b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:52.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.263+0000 7f9e0959a700 1 -- 172.21.15.16:0/1141847751 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc004230 con 0x7f9e0c09d6f0 2023-09-30T22:30:52.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.263+0000 7f9e0959a700 1 -- 172.21.15.16:0/1141847751 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dfc004390 con 0x7f9e0c09d6f0 2023-09-30T22:30:52.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.263+0000 7f9e0959a700 1 -- 172.21.15.16:0/1141847751 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc0036a0 con 0x7f9e0c09d6f0 2023-09-30T22:30:52.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.264+0000 7faaa31d4700 1 -- 172.21.15.16:0/4106999334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa9c09de50 msgr2=0x7faa9c09e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:52.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.264+0000 7faaa31d4700 1 --2- 172.21.15.16:0/4106999334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa9c09de50 0x7faa9c09e220 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7faa880098f0 tx=0x7faa88009c00 comp rx=0 tx=0).stop 2023-09-30T22:30:52.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.264+0000 7f7bc2b39700 1 -- 172.21.15.16:0/1222987724 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7bbc09d6f0 msgr2=0x7f7bac004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:52.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.264+0000 7f7bc2b39700 1 --2- 172.21.15.16:0/1222987724 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7bbc09d6f0 0x7f7bac004850 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7ba80098e0 tx=0x7f7bac005880 comp rx=0 tx=0).stop 2023-09-30T22:30:52.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.264+0000 7f7bc10d6700 1 -- 172.21.15.16:0/1222987724 reap_dead start 2023-09-30T22:30:52.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.264+0000 7f7bc2b39700 1 -- 172.21.15.16:0/1222987724 shutdown_connections 2023-09-30T22:30:52.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.264+0000 7f7bc2b39700 1 -- 172.21.15.16:0/1222987724 >> 172.21.15.16:0/1222987724 conn(0x7f7bbc066f50 msgr2=0x7f7bbc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:52.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.264+0000 7faaa31d4700 1 -- 172.21.15.16:0/4106999334 shutdown_connections 2023-09-30T22:30:52.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.264+0000 7faaa31d4700 1 --2- 172.21.15.16:0/4106999334 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faa9c1338a0 0x7faa9c133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.264+0000 7faaa31d4700 1 --2- 172.21.15.16:0/4106999334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa9c09de50 0x7faa9c09e220 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.264+0000 7faaa31d4700 1 -- 172.21.15.16:0/4106999334 >> 172.21.15.16:0/4106999334 conn(0x7faa9c0a2aa0 msgr2=0x7faa9c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:52.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.264+0000 7f7bc2b39700 1 -- 172.21.15.16:0/1222987724 shutdown_connections 2023-09-30T22:30:52.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.264+0000 7f7bc2b39700 1 -- 172.21.15.16:0/1222987724 wait complete. 2023-09-30T22:30:52.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.264+0000 7faaa31d4700 1 -- 172.21.15.16:0/4106999334 shutdown_connections 2023-09-30T22:30:52.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.264+0000 7faaa31d4700 1 -- 172.21.15.16:0/4106999334 wait complete. 2023-09-30T22:30:52.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.264+0000 7f9e1099a700 1 -- 172.21.15.16:0/1141847751 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e0c09d6f0 msgr2=0x7f9e0c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:52.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.264+0000 7f9e1099a700 1 --2- 172.21.15.16:0/1141847751 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e0c09d6f0 0x7f9e0c09dac0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc009870 tx=0x7f9dfc009b80 comp rx=0 tx=0).stop 2023-09-30T22:30:52.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.264+0000 7f9e1099a700 1 -- 172.21.15.16:0/1141847751 shutdown_connections 2023-09-30T22:30:52.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.264+0000 7f9e1099a700 1 --2- 172.21.15.16:0/1141847751 >> v2:172.21.15.16:3300/0 conn(0x7f9e0c09e410 0x7f9e0c134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.264+0000 7f9e1099a700 1 --2- 172.21.15.16:0/1141847751 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e0c09d6f0 0x7f9e0c09dac0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.264+0000 7f9e1099a700 1 -- 172.21.15.16:0/1141847751 >> 172.21.15.16:0/1141847751 conn(0x7f9e0c066f50 msgr2=0x7f9e0c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:52.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.264+0000 7f9e1099a700 1 -- 172.21.15.16:0/1141847751 shutdown_connections 2023-09-30T22:30:52.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.264+0000 7f9e1099a700 1 -- 172.21.15.16:0/1141847751 wait complete. 2023-09-30T22:30:52.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.265+0000 7f7bc2b39700 1 Processor -- start 2023-09-30T22:30:52.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.265+0000 7f7bc2b39700 1 -- start start 2023-09-30T22:30:52.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.265+0000 7faaa31d4700 1 Processor -- start 2023-09-30T22:30:52.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.265+0000 7faaa31d4700 1 -- start start 2023-09-30T22:30:52.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.266+0000 7f7bc2b39700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bbc09d6f0 0x7f7bbc1a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:52.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.266+0000 7f7bc2b39700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7bbc1a56e0 0x7f7bbc1a5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:52.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.266+0000 7f7bc2b39700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7bbc1cca30 0x7f7bbc1cee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:52.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.266+0000 7f7bc2b39700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bbc1a6030 con 0x7f7bbc09d6f0 2023-09-30T22:30:52.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.266+0000 7f7bc2b39700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bbc1b8500 con 0x7f7bbc1cca30 2023-09-30T22:30:52.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.266+0000 7f7bc2b39700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7bbc1a7340 con 0x7f7bbc1a56e0 2023-09-30T22:30:52.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.266+0000 7f9e1099a700 1 Processor -- start 2023-09-30T22:30:52.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.266+0000 7f9e1099a700 1 -- start start 2023-09-30T22:30:52.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.266+0000 7faaa31d4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faa9c09de50 0x7faa9c11d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:52.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.266+0000 7faaa31d4700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faa9c09e760 0x7faa9c11dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:52.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.266+0000 7faaa31d4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa9c1338a0 0x7faa9c11e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:52.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.266+0000 7faaa31d4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9c1211b0 con 0x7faa9c1338a0 2023-09-30T22:30:52.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.266+0000 7faaa31d4700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa9c11f340 con 0x7faa9c09e760 2023-09-30T22:30:52.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.266+0000 7faaa31d4700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faa9c0a1040 con 0x7faa9c09de50 2023-09-30T22:30:52.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.267+0000 7faa9bfff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faa9c09e760 0x7faa9c11dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:52.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.267+0000 7faa9bfff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faa9c09e760 0x7faa9c11dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41062/0 (socket says 172.21.15.16:41062) 2023-09-30T22:30:52.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.267+0000 7faa9bfff700 1 -- 172.21.15.16:0/3624473616 learned_addr learned my addr 172.21.15.16:0/3624473616 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:52.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.267+0000 7faaa1771700 1 --2- 172.21.15.16:0/3624473616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa9c1338a0 0x7faa9c11e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:52.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.267+0000 7faaa0f70700 1 --2- 172.21.15.16:0/3624473616 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faa9c09de50 0x7faa9c11d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:52.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.267+0000 7f9e1099a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e0c09d6f0 0x7f9e0c1a5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:52.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.267+0000 7faa9bfff700 1 -- 172.21.15.16:0/3624473616 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faa9c09de50 msgr2=0x7faa9c11d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:52.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.267+0000 7faa9bfff700 1 --2- 172.21.15.16:0/3624473616 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faa9c09de50 0x7faa9c11d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.267+0000 7faa9bfff700 1 -- 172.21.15.16:0/3624473616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa9c1338a0 msgr2=0x7faa9c11e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:52.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.267+0000 7faa9bfff700 1 --2- 172.21.15.16:0/3624473616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa9c1338a0 0x7faa9c11e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.267+0000 7faa9bfff700 1 -- 172.21.15.16:0/3624473616 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa88009550 con 0x7faa9c09e760 2023-09-30T22:30:52.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.267+0000 7faaa1771700 1 --2- 172.21.15.16:0/3624473616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa9c1338a0 0x7faa9c11e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:30:52.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.266+0000 7f7bbbfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7bbc1a56e0 0x7f7bbc1a5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:52.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.266+0000 7f7bbbfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7bbc1a56e0 0x7f7bbc1a5af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35510/0 (socket says 172.21.15.16:35510) 2023-09-30T22:30:52.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.266+0000 7f7bbbfff700 1 -- 172.21.15.16:0/3534189360 learned_addr learned my addr 172.21.15.16:0/3534189360 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:52.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.266+0000 7f7bc10d6700 1 --2- 172.21.15.16:0/3534189360 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7bbc1cca30 0x7f7bbc1cee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:52.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.267+0000 7f7bbbfff700 1 -- 172.21.15.16:0/3534189360 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7bbc1cca30 msgr2=0x7f7bbc1cee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:52.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.267+0000 7f7bbbfff700 1 --2- 172.21.15.16:0/3534189360 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7bbc1cca30 0x7f7bbc1cee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.267+0000 7f7bbbfff700 1 -- 172.21.15.16:0/3534189360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bbc09d6f0 msgr2=0x7f7bbc1a51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:52.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.267+0000 7f7bbbfff700 1 --2- 172.21.15.16:0/3534189360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bbc09d6f0 0x7f7bbc1a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.267+0000 7f7bbbfff700 1 -- 172.21.15.16:0/3534189360 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bbc1a7f70 con 0x7f7bbc1a56e0 2023-09-30T22:30:52.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.267+0000 7f7bc10d6700 1 --2- 172.21.15.16:0/3534189360 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7bbc1cca30 0x7f7bbc1cee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:52.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.267+0000 7f7bbbfff700 1 --2- 172.21.15.16:0/3534189360 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7bbc1a56e0 0x7f7bbc1a5af0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7bac00bba0 tx=0x7f7ba8004860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:52.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.267+0000 7f9e1099a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e0c09e000 0x7f9e0c1a64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:52.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.267+0000 7f9e1099a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e0c09e410 0x7f9e0c1a6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:52.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.267+0000 7f9e1099a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0c1a8da0 con 0x7f9e0c09e410 2023-09-30T22:30:52.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.267+0000 7f9e1099a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0c1a7460 con 0x7f9e0c09e000 2023-09-30T22:30:52.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.267+0000 7f9e1099a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0c1a97f0 con 0x7f9e0c09d6f0 2023-09-30T22:30:52.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.267+0000 7faa9bfff700 1 --2- 172.21.15.16:0/3624473616 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faa9c09e760 0x7faa9c11dbd0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7faa88009c40 tx=0x7faa88004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:52.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.267+0000 7f7bb97fa700 1 -- 172.21.15.16:0/3534189360 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba801a420 con 0x7f7bbc1a56e0 2023-09-30T22:30:52.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.267+0000 7f7bb97fa700 1 -- 172.21.15.16:0/3534189360 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba801a580 con 0x7f7bbc1a56e0 2023-09-30T22:30:52.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.267+0000 7f7bb97fa700 1 -- 172.21.15.16:0/3534189360 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba801a930 con 0x7f7bbc1a56e0 2023-09-30T22:30:52.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.267+0000 7f9e09d9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e0c09e000 0x7f9e0c1a64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:52.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.267+0000 7f9e09d9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e0c09e000 0x7f9e0c1a64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41066/0 (socket says 172.21.15.16:41066) 2023-09-30T22:30:52.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.267+0000 7f9e09d9b700 1 -- 172.21.15.16:0/2081255007 learned_addr learned my addr 172.21.15.16:0/2081255007 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:52.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.267+0000 7f9e0a59c700 1 --2- 172.21.15.16:0/2081255007 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e0c09d6f0 0x7f9e0c1a5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:52.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.268+0000 7f9e09d9b700 1 -- 172.21.15.16:0/2081255007 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e0c09d6f0 msgr2=0x7f9e0c1a5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:52.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.268+0000 7f9e09d9b700 1 --2- 172.21.15.16:0/2081255007 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e0c09d6f0 0x7f9e0c1a5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.268+0000 7f9e09d9b700 1 -- 172.21.15.16:0/2081255007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e0c09e410 msgr2=0x7f9e0c1a6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:52.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.268+0000 7f9e09d9b700 1 --2- 172.21.15.16:0/2081255007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e0c09e410 0x7f9e0c1a6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.268+0000 7f9e09d9b700 1 -- 172.21.15.16:0/2081255007 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dfc0094a0 con 0x7f9e0c09e000 2023-09-30T22:30:52.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.268+0000 7f9e09d9b700 1 --2- 172.21.15.16:0/2081255007 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e0c09e000 0x7f9e0c1a64f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9e0000c550 tx=0x7f9e0000c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:52.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.268+0000 7faa997fa700 1 -- 172.21.15.16:0/3624473616 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa88004590 con 0x7faa9c09e760 2023-09-30T22:30:52.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.268+0000 7f7bc2b39700 1 -- 172.21.15.16:0/3534189360 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bbc1a83c0 con 0x7f7bbc1a56e0 2023-09-30T22:30:52.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.268+0000 7faa997fa700 1 -- 172.21.15.16:0/3624473616 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa8801e070 con 0x7faa9c09e760 2023-09-30T22:30:52.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.268+0000 7faaa31d4700 1 -- 172.21.15.16:0/3624473616 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa9c06f640 con 0x7faa9c09e760 2023-09-30T22:30:52.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.268+0000 7faa997fa700 1 -- 172.21.15.16:0/3624473616 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa880038e0 con 0x7faa9c09e760 2023-09-30T22:30:52.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.268+0000 7f7bc2b39700 1 -- 172.21.15.16:0/3534189360 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bbc09c710 con 0x7f7bbc1a56e0 2023-09-30T22:30:52.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.268+0000 7faaa31d4700 1 -- 172.21.15.16:0/3624473616 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa9c1216b0 con 0x7faa9c09e760 2023-09-30T22:30:52.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.269+0000 7faaa31d4700 1 -- 172.21.15.16:0/3624473616 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa9c074ea0 con 0x7faa9c09e760 2023-09-30T22:30:52.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.269+0000 7f9dfaffd700 1 -- 172.21.15.16:0/2081255007 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e00005d70 con 0x7f9e0c09e000 2023-09-30T22:30:52.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.269+0000 7f9dfaffd700 1 -- 172.21.15.16:0/2081255007 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e000070d0 con 0x7f9e0c09e000 2023-09-30T22:30:52.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.269+0000 7f9e1099a700 1 -- 172.21.15.16:0/2081255007 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e0c1a88e0 con 0x7f9e0c09e000 2023-09-30T22:30:52.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.269+0000 7f9e1099a700 1 -- 172.21.15.16:0/2081255007 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e0c1a7bd0 con 0x7f9e0c09e000 2023-09-30T22:30:52.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.269+0000 7f7bc2b39700 1 -- 172.21.15.16:0/3534189360 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bbc09dcd0 con 0x7f7bbc1a56e0 2023-09-30T22:30:52.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.269+0000 7faa997fa700 1 -- 172.21.15.16:0/3624473616 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7faa88003a40 con 0x7faa9c09e760 2023-09-30T22:30:52.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.270+0000 7f9dfaffd700 1 -- 172.21.15.16:0/2081255007 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e00007480 con 0x7f9e0c09e000 2023-09-30T22:30:52.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.270+0000 7faa997fa700 1 -- 172.21.15.16:0/3624473616 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7faa88020070 con 0x7faa9c09e760 2023-09-30T22:30:52.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.270+0000 7f7bb97fa700 1 -- 172.21.15.16:0/3534189360 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7ba801aa90 con 0x7f7bbc1a56e0 2023-09-30T22:30:52.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.270+0000 7f7bb97fa700 1 -- 172.21.15.16:0/3534189360 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f7ba8040180 con 0x7f7bbc1a56e0 2023-09-30T22:30:52.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.270+0000 7f9e1099a700 1 -- 172.21.15.16:0/2081255007 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9df00052e0 con 0x7f9e0c09e000 2023-09-30T22:30:52.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.271+0000 7f9dfaffd700 1 -- 172.21.15.16:0/2081255007 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9e000075e0 con 0x7f9e0c09e000 2023-09-30T22:30:52.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.275+0000 7f7bb97fa700 1 -- 172.21.15.16:0/3534189360 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7ba801ad40 con 0x7f7bbc1a56e0 2023-09-30T22:30:52.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.275+0000 7f9dfaffd700 1 -- 172.21.15.16:0/2081255007 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f9e0001b3b0 con 0x7f9e0c09e000 2023-09-30T22:30:52.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.276+0000 7faa997fa700 1 -- 172.21.15.16:0/3624473616 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7faa880243e0 con 0x7faa9c09e760 2023-09-30T22:30:52.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.280+0000 7f9dfaffd700 1 -- 172.21.15.16:0/2081255007 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9e0001bb10 con 0x7f9e0c09e000 2023-09-30T22:30:52.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.482+0000 7faaa31d4700 1 -- 172.21.15.16:0/3624473616 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7faa9c121960 con 0x7faa9c09e760 2023-09-30T22:30:52.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.483+0000 7faa997fa700 1 -- 172.21.15.16:0/3624473616 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7faa88015020 con 0x7faa9c09e760 2023-09-30T22:30:52.483 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:30:52.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.486+0000 7faaa31d4700 1 -- 172.21.15.16:0/3624473616 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faa9c09e760 msgr2=0x7faa9c11dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:52.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.486+0000 7faaa31d4700 1 --2- 172.21.15.16:0/3624473616 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faa9c09e760 0x7faa9c11dbd0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7faa88009c40 tx=0x7faa88004750 comp rx=0 tx=0).stop 2023-09-30T22:30:52.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.486+0000 7faaa31d4700 1 -- 172.21.15.16:0/3624473616 shutdown_connections 2023-09-30T22:30:52.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.486+0000 7faaa31d4700 1 --2- 172.21.15.16:0/3624473616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa9c1338a0 0x7faa9c11e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.486+0000 7faaa31d4700 1 --2- 172.21.15.16:0/3624473616 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faa9c09de50 0x7faa9c11d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.486+0000 7faaa31d4700 1 --2- 172.21.15.16:0/3624473616 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faa9c09e760 0x7faa9c11dbd0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.486+0000 7faaa31d4700 1 -- 172.21.15.16:0/3624473616 >> 172.21.15.16:0/3624473616 conn(0x7faa9c0a2aa0 msgr2=0x7faa9c1b9410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:52.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.486+0000 7faaa31d4700 1 -- 172.21.15.16:0/3624473616 shutdown_connections 2023-09-30T22:30:52.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.486+0000 7faaa31d4700 1 -- 172.21.15.16:0/3624473616 wait complete. 2023-09-30T22:30:52.501 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:30:52.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.506+0000 7f7bc2b39700 1 -- 172.21.15.16:0/3534189360 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7bbc074ea0 con 0x7f7bbc1a56e0 2023-09-30T22:30:52.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.506+0000 7f7bb97fa700 1 -- 172.21.15.16:0/3534189360 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7ba8028090 con 0x7f7bbc1a56e0 2023-09-30T22:30:52.507 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:30:52.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.509+0000 7f7bc2b39700 1 -- 172.21.15.16:0/3534189360 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7bbc1a56e0 msgr2=0x7f7bbc1a5af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:52.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.509+0000 7f7bc2b39700 1 --2- 172.21.15.16:0/3534189360 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7bbc1a56e0 0x7f7bbc1a5af0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7bac00bba0 tx=0x7f7ba8004860 comp rx=0 tx=0).stop 2023-09-30T22:30:52.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.510+0000 7f7bc2b39700 1 -- 172.21.15.16:0/3534189360 shutdown_connections 2023-09-30T22:30:52.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.510+0000 7f7bc2b39700 1 --2- 172.21.15.16:0/3534189360 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7bbc1cca30 0x7f7bbc1cee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.510+0000 7f7bc2b39700 1 --2- 172.21.15.16:0/3534189360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bbc09d6f0 0x7f7bbc1a51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.510+0000 7f7bc2b39700 1 --2- 172.21.15.16:0/3534189360 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7bbc1a56e0 0x7f7bbc1a5af0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.510+0000 7f7bc2b39700 1 -- 172.21.15.16:0/3534189360 >> 172.21.15.16:0/3534189360 conn(0x7f7bbc066f50 msgr2=0x7f7bbc1baff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:52.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.511+0000 7f7bc2b39700 1 -- 172.21.15.16:0/3534189360 shutdown_connections 2023-09-30T22:30:52.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.511+0000 7f7bc2b39700 1 -- 172.21.15.16:0/3534189360 wait complete. 2023-09-30T22:30:52.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.514+0000 7f9e1099a700 1 -- 172.21.15.16:0/2081255007 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9df0005860 con 0x7f9e0c09e000 2023-09-30T22:30:52.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.514+0000 7f9dfaffd700 1 -- 172.21.15.16:0/2081255007 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9e000192c0 con 0x7f9e0c09e000 2023-09-30T22:30:52.515 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:30:52.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.518+0000 7f9e1099a700 1 -- 172.21.15.16:0/2081255007 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e0c09e000 msgr2=0x7f9e0c1a64f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:52.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.518+0000 7f9e1099a700 1 --2- 172.21.15.16:0/2081255007 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e0c09e000 0x7f9e0c1a64f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9e0000c550 tx=0x7f9e0000c860 comp rx=0 tx=0).stop 2023-09-30T22:30:52.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.518+0000 7f9e1099a700 1 -- 172.21.15.16:0/2081255007 shutdown_connections 2023-09-30T22:30:52.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.518+0000 7f9e1099a700 1 --2- 172.21.15.16:0/2081255007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e0c09e410 0x7f9e0c1a6a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.518+0000 7f9e1099a700 1 --2- 172.21.15.16:0/2081255007 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e0c09d6f0 0x7f9e0c1a5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.518+0000 7f9e1099a700 1 --2- 172.21.15.16:0/2081255007 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e0c09e000 0x7f9e0c1a64f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.518+0000 7f9e1099a700 1 -- 172.21.15.16:0/2081255007 >> 172.21.15.16:0/2081255007 conn(0x7f9e0c066f50 msgr2=0x7f9e0c1baf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:52.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.518+0000 7f9e1099a700 1 -- 172.21.15.16:0/2081255007 shutdown_connections 2023-09-30T22:30:52.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.519+0000 7f9e1099a700 1 -- 172.21.15.16:0/2081255007 wait complete. 2023-09-30T22:30:52.531 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:30:52.535 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:30:52.606 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:52.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.745+0000 7f9973935700 1 Processor -- start 2023-09-30T22:30:52.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.746+0000 7f9973935700 1 -- start start 2023-09-30T22:30:52.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.746+0000 7f9973935700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f996c133420 0x7f996c1337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:52.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.746+0000 7f9973935700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f996c133e30 0x7f996c12ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.746+0000 7f9973935700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f996c06b990 con 0x7f996c134260 2023-09-30T22:30:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.746+0000 7f9973935700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f996c0a0590 con 0x7f996c133e30 2023-09-30T22:30:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.746+0000 7f9973935700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f996c09fb80 con 0x7f996c133420 2023-09-30T22:30:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.746+0000 7f99716d1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f996c133420 0x7f996c1337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.746+0000 7f9970ed0700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f996c133e30 0x7f996c12ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:52.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.746+0000 7f9971ed2700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f996c134260 0x7f996c1304d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40866/0 (socket says 172.21.15.16:40866) 2023-09-30T22:30:52.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.746+0000 7f9971ed2700 1 -- 172.21.15.16:0/3174715453 learned_addr learned my addr 172.21.15.16:0/3174715453 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:30:52.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.747+0000 7f9970ed0700 1 -- 172.21.15.16:0/3174715453 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f996c133420 msgr2=0x7f996c1337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:52.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.747+0000 7f9970ed0700 1 --2- 172.21.15.16:0/3174715453 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f996c133420 0x7f996c1337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.747+0000 7f9970ed0700 1 -- 172.21.15.16:0/3174715453 >> v1:172.21.15.16:6789/0 conn(0x7f996c134260 legacy=0x7f996c1304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:52.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.747+0000 7f9970ed0700 1 -- 172.21.15.16:0/3174715453 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f996c0a0030 con 0x7f996c133e30 2023-09-30T22:30:52.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.747+0000 7f99716d1700 1 --2- 172.21.15.16:0/3174715453 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f996c133420 0x7f996c1337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:52.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.747+0000 7f9970ed0700 1 --2- 172.21.15.16:0/3174715453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f996c133e30 0x7f996c12ff90 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f995c009840 tx=0x7f995c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c4c16de7db4d345 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:52.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.747+0000 7f9963fff700 1 -- 172.21.15.16:0/3174715453 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995c011830 con 0x7f996c133e30 2023-09-30T22:30:52.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.747+0000 7f9963fff700 1 -- 172.21.15.16:0/3174715453 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f995c011990 con 0x7f996c133e30 2023-09-30T22:30:52.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.747+0000 7f9963fff700 1 -- 172.21.15.16:0/3174715453 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995c011d20 con 0x7f996c133e30 2023-09-30T22:30:52.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.747+0000 7f9973935700 1 -- 172.21.15.16:0/3174715453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f996c133e30 msgr2=0x7f996c12ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:52.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.747+0000 7f9973935700 1 --2- 172.21.15.16:0/3174715453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f996c133e30 0x7f996c12ff90 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f995c009840 tx=0x7f995c009b50 comp rx=0 tx=0).stop 2023-09-30T22:30:52.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.748+0000 7f9973935700 1 -- 172.21.15.16:0/3174715453 shutdown_connections 2023-09-30T22:30:52.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.748+0000 7f9973935700 1 --2- 172.21.15.16:0/3174715453 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f996c133420 0x7f996c1337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.748+0000 7f9973935700 1 --2- 172.21.15.16:0/3174715453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f996c133e30 0x7f996c12ff90 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.748+0000 7f9973935700 1 -- 172.21.15.16:0/3174715453 >> 172.21.15.16:0/3174715453 conn(0x7f996c0a4210 msgr2=0x7f996c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:52.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.748+0000 7f9973935700 1 -- 172.21.15.16:0/3174715453 shutdown_connections 2023-09-30T22:30:52.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.748+0000 7f9973935700 1 -- 172.21.15.16:0/3174715453 wait complete. 2023-09-30T22:30:52.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.749+0000 7f9973935700 1 Processor -- start 2023-09-30T22:30:52.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.749+0000 7f9973935700 1 -- start start 2023-09-30T22:30:52.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.750+0000 7f9973935700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f996c133420 0x7f996c09c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:52.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.750+0000 7f9973935700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f996c133e30 0x7f996c09c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:52.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.750+0000 7f9973935700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f996c134260 0x7f996c09cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.750+0000 7f9973935700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f996c120010 con 0x7f996c133420 2023-09-30T22:30:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.750+0000 7f9973935700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f996c1201f0 con 0x7f996c134260 2023-09-30T22:30:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.750+0000 7f9973935700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f996c12da70 con 0x7f996c133e30 2023-09-30T22:30:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.750+0000 7f99716d1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f996c133420 0x7f996c09c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.750+0000 7f99716d1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f996c133420 0x7f996c09c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43876/0 (socket says 172.21.15.16:43876) 2023-09-30T22:30:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.750+0000 7f9971ed2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f996c134260 0x7f996c09cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.750+0000 7f99716d1700 1 -- 172.21.15.16:0/2594125749 learned_addr learned my addr 172.21.15.16:0/2594125749 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.750+0000 7f9970ed0700 1 --2- 172.21.15.16:0/2594125749 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f996c133e30 0x7f996c09c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.750+0000 7f99716d1700 1 -- 172.21.15.16:0/2594125749 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f996c133e30 msgr2=0x7f996c09c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.750+0000 7f99716d1700 1 --2- 172.21.15.16:0/2594125749 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f996c133e30 0x7f996c09c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.750+0000 7f99716d1700 1 -- 172.21.15.16:0/2594125749 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f996c134260 msgr2=0x7f996c09cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:52.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.750+0000 7f99716d1700 1 --2- 172.21.15.16:0/2594125749 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f996c134260 0x7f996c09cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.750+0000 7f99716d1700 1 -- 172.21.15.16:0/2594125749 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99680094a0 con 0x7f996c133420 2023-09-30T22:30:52.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.750+0000 7f9970ed0700 1 --2- 172.21.15.16:0/2594125749 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f996c133e30 0x7f996c09c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:52.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.750+0000 7f9971ed2700 1 --2- 172.21.15.16:0/2594125749 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f996c134260 0x7f996c09cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:52.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.750+0000 7f99716d1700 1 --2- 172.21.15.16:0/2594125749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f996c133420 0x7f996c09c480 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f996800ec10 tx=0x7f996800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:52.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.751+0000 7f9961ffb700 1 -- 172.21.15.16:0/2594125749 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996800c850 con 0x7f996c133420 2023-09-30T22:30:52.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.751+0000 7f9961ffb700 1 -- 172.21.15.16:0/2594125749 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f996800c9b0 con 0x7f996c133420 2023-09-30T22:30:52.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.751+0000 7f9973935700 1 -- 172.21.15.16:0/2594125749 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f995c0094a0 con 0x7f996c133420 2023-09-30T22:30:52.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.751+0000 7f9961ffb700 1 -- 172.21.15.16:0/2594125749 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996800cd20 con 0x7f996c133420 2023-09-30T22:30:52.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.751+0000 7f9973935700 1 -- 172.21.15.16:0/2594125749 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f996c12d620 con 0x7f996c133420 2023-09-30T22:30:52.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.752+0000 7f9961ffb700 1 -- 172.21.15.16:0/2594125749 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9968013470 con 0x7f996c133420 2023-09-30T22:30:52.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.752+0000 7f9961ffb700 1 -- 172.21.15.16:0/2594125749 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f996803f7e0 con 0x7f996c133420 2023-09-30T22:30:52.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.752+0000 7f9973935700 1 -- 172.21.15.16:0/2594125749 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f996c074ea0 con 0x7f996c133420 2023-09-30T22:30:52.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.757+0000 7f9961ffb700 1 -- 172.21.15.16:0/2594125749 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9968013720 con 0x7f996c133420 2023-09-30T22:30:52.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.959+0000 7f9973935700 1 -- 172.21.15.16:0/2594125749 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f996c11fe20 con 0x7f996c133420 2023-09-30T22:30:52.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.959+0000 7f9961ffb700 1 -- 172.21.15.16:0/2594125749 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f996802a090 con 0x7f996c133420 2023-09-30T22:30:52.960 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:30:52.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.961+0000 7f9973935700 1 -- 172.21.15.16:0/2594125749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f996c133420 msgr2=0x7f996c09c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:52.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.961+0000 7f9973935700 1 --2- 172.21.15.16:0/2594125749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f996c133420 0x7f996c09c480 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f996800ec10 tx=0x7f996800ef20 comp rx=0 tx=0).stop 2023-09-30T22:30:52.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.961+0000 7f9973935700 1 -- 172.21.15.16:0/2594125749 shutdown_connections 2023-09-30T22:30:52.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.961+0000 7f9973935700 1 --2- 172.21.15.16:0/2594125749 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f996c134260 0x7f996c09cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.961+0000 7f9973935700 1 --2- 172.21.15.16:0/2594125749 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f996c133420 0x7f996c09c480 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.961+0000 7f9973935700 1 --2- 172.21.15.16:0/2594125749 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f996c133e30 0x7f996c09c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:52.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.961+0000 7f9973935700 1 -- 172.21.15.16:0/2594125749 >> 172.21.15.16:0/2594125749 conn(0x7f996c0a4210 msgr2=0x7f996c0a0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:52.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.962+0000 7f9973935700 1 -- 172.21.15.16:0/2594125749 shutdown_connections 2023-09-30T22:30:52.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:52.962+0000 7f9973935700 1 -- 172.21.15.16:0/2594125749 wait complete. 2023-09-30T22:30:52.976 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:30:53.502 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:53.532 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:53.537 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:53.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.655+0000 7f413fe52700 1 Processor -- start 2023-09-30T22:30:53.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.655+0000 7f413fe52700 1 -- start start 2023-09-30T22:30:53.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.656+0000 7f413fe52700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4138131630 0x7f4138131a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:53.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.656+0000 7f413fe52700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4138131f40 0x7f413806e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:53.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.656+0000 7f413fe52700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f413806e9d0 0x7f413806ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:53.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.656+0000 7f413fe52700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4138071a00 con 0x7f4138131630 2023-09-30T22:30:53.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.656+0000 7f413fe52700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4138131430 con 0x7f413806e9d0 2023-09-30T22:30:53.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.656+0000 7f413fe52700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f413809a510 con 0x7f4138131f40 2023-09-30T22:30:53.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.656+0000 7f413d3ed700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4138131f40 0x7f413806e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:53.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.656+0000 7f413d3ed700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4138131f40 0x7f413806e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35540/0 (socket says 172.21.15.16:35540) 2023-09-30T22:30:53.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.656+0000 7f413d3ed700 1 -- 172.21.15.16:0/3130779938 learned_addr learned my addr 172.21.15.16:0/3130779938 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:53.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.656+0000 7f413e3ef700 1 --2- 172.21.15.16:0/3130779938 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f413806e9d0 0x7f413806ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:53.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.656+0000 7f413dbee700 1 --2- 172.21.15.16:0/3130779938 >> v2:172.21.15.16:3300/0 conn(0x7f4138131630 0x7f4138131a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:53.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.656+0000 7f413d3ed700 1 -- 172.21.15.16:0/3130779938 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f413806e9d0 msgr2=0x7f413806ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:53.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.656+0000 7f413d3ed700 1 --2- 172.21.15.16:0/3130779938 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f413806e9d0 0x7f413806ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:53.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.656+0000 7f413d3ed700 1 -- 172.21.15.16:0/3130779938 >> v2:172.21.15.16:3300/0 conn(0x7f4138131630 msgr2=0x7f4138131a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:53.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.656+0000 7f413d3ed700 1 --2- 172.21.15.16:0/3130779938 >> v2:172.21.15.16:3300/0 conn(0x7f4138131630 0x7f4138131a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:53.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.656+0000 7f413d3ed700 1 -- 172.21.15.16:0/3130779938 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41380721c0 con 0x7f4138131f40 2023-09-30T22:30:53.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.656+0000 7f413dbee700 1 --2- 172.21.15.16:0/3130779938 >> v2:172.21.15.16:3300/0 conn(0x7f4138131630 0x7f4138131a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:53.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.656+0000 7f413e3ef700 1 --2- 172.21.15.16:0/3130779938 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f413806e9d0 0x7f413806ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:53.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.657+0000 7f413d3ed700 1 --2- 172.21.15.16:0/3130779938 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4138131f40 0x7f413806e490 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f41280098f0 tx=0x7f4128009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=221d21dfc4c1cf65 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:53.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.657+0000 7f413cbec700 1 -- 172.21.15.16:0/3130779938 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4128011830 con 0x7f4138131f40 2023-09-30T22:30:53.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.657+0000 7f413cbec700 1 -- 172.21.15.16:0/3130779938 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4128011990 con 0x7f4138131f40 2023-09-30T22:30:53.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.657+0000 7f413cbec700 1 -- 172.21.15.16:0/3130779938 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4128011d20 con 0x7f4138131f40 2023-09-30T22:30:53.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.658+0000 7f413fe52700 1 -- 172.21.15.16:0/3130779938 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4138131f40 msgr2=0x7f413806e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:53.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.658+0000 7f413fe52700 1 --2- 172.21.15.16:0/3130779938 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4138131f40 0x7f413806e490 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f41280098f0 tx=0x7f4128009c00 comp rx=0 tx=0).stop 2023-09-30T22:30:53.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.658+0000 7f413fe52700 1 -- 172.21.15.16:0/3130779938 shutdown_connections 2023-09-30T22:30:53.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.658+0000 7f413fe52700 1 --2- 172.21.15.16:0/3130779938 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f413806e9d0 0x7f413806ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:53.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.658+0000 7f413fe52700 1 --2- 172.21.15.16:0/3130779938 >> v2:172.21.15.16:3300/0 conn(0x7f4138131630 0x7f4138131a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:53.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.658+0000 7f413fe52700 1 --2- 172.21.15.16:0/3130779938 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4138131f40 0x7f413806e490 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:53.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.658+0000 7f413fe52700 1 -- 172.21.15.16:0/3130779938 >> 172.21.15.16:0/3130779938 conn(0x7f41380a2aa0 msgr2=0x7f41380a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:53.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.658+0000 7f413fe52700 1 -- 172.21.15.16:0/3130779938 shutdown_connections 2023-09-30T22:30:53.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.658+0000 7f413fe52700 1 -- 172.21.15.16:0/3130779938 wait complete. 2023-09-30T22:30:53.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.659+0000 7f413fe52700 1 Processor -- start 2023-09-30T22:30:53.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.659+0000 7f413fe52700 1 -- start start 2023-09-30T22:30:53.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.660+0000 7f413fe52700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f413806e9d0 0x7f413811b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:53.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.660+0000 7f413fe52700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4138131630 0x7f413811ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:53.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.660+0000 7f413fe52700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4138131f40 0x7f413811bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:53.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.660+0000 7f413fe52700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f413811f150 con 0x7f4138131f40 2023-09-30T22:30:53.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.660+0000 7f413fe52700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f413811f990 con 0x7f4138131630 2023-09-30T22:30:53.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.660+0000 7f413fe52700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f413811d0c0 con 0x7f413806e9d0 2023-09-30T22:30:53.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.660+0000 7f413dbee700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f413806e9d0 0x7f413811b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:53.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.660+0000 7f413dbee700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f413806e9d0 0x7f413811b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35544/0 (socket says 172.21.15.16:35544) 2023-09-30T22:30:53.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.660+0000 7f413dbee700 1 -- 172.21.15.16:0/3189010855 learned_addr learned my addr 172.21.15.16:0/3189010855 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:53.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.660+0000 7f413d3ed700 1 --2- 172.21.15.16:0/3189010855 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4138131630 0x7f413811ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:53.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.660+0000 7f413e3ef700 1 --2- 172.21.15.16:0/3189010855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4138131f40 0x7f413811bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:53.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.660+0000 7f413dbee700 1 -- 172.21.15.16:0/3189010855 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4138131630 msgr2=0x7f413811ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:53.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.660+0000 7f413dbee700 1 --2- 172.21.15.16:0/3189010855 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4138131630 0x7f413811ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:53.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.660+0000 7f413dbee700 1 -- 172.21.15.16:0/3189010855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4138131f40 msgr2=0x7f413811bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:53.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.660+0000 7f413dbee700 1 --2- 172.21.15.16:0/3189010855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4138131f40 0x7f413811bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:53.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.661+0000 7f413dbee700 1 -- 172.21.15.16:0/3189010855 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4130009a70 con 0x7f413806e9d0 2023-09-30T22:30:53.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.661+0000 7f413dbee700 1 --2- 172.21.15.16:0/3189010855 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f413806e9d0 0x7f413811b540 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4134009ec0 tx=0x7f413400eec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:53.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.661+0000 7f412e7fc700 1 -- 172.21.15.16:0/3189010855 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4134013860 con 0x7f413806e9d0 2023-09-30T22:30:53.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.661+0000 7f412e7fc700 1 -- 172.21.15.16:0/3189010855 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41340139c0 con 0x7f413806e9d0 2023-09-30T22:30:53.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.661+0000 7f412e7fc700 1 -- 172.21.15.16:0/3189010855 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4134013d50 con 0x7f413806e9d0 2023-09-30T22:30:53.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.661+0000 7f413fe52700 1 -- 172.21.15.16:0/3189010855 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41340094a0 con 0x7f413806e9d0 2023-09-30T22:30:53.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.661+0000 7f413fe52700 1 -- 172.21.15.16:0/3189010855 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4128009550 con 0x7f413806e9d0 2023-09-30T22:30:53.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.662+0000 7f412e7fc700 1 -- 172.21.15.16:0/3189010855 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f413400c640 con 0x7f413806e9d0 2023-09-30T22:30:53.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.662+0000 7f412e7fc700 1 -- 172.21.15.16:0/3189010855 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f4134043500 con 0x7f413806e9d0 2023-09-30T22:30:53.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.662+0000 7f413fe52700 1 -- 172.21.15.16:0/3189010855 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4138074ea0 con 0x7f413806e9d0 2023-09-30T22:30:53.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.668+0000 7f412e7fc700 1 -- 172.21.15.16:0/3189010855 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f413401b440 con 0x7f413806e9d0 2023-09-30T22:30:53.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.692+0000 7f1b200bd700 1 Processor -- start 2023-09-30T22:30:53.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.692+0000 7f1b200bd700 1 -- start start 2023-09-30T22:30:53.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.692+0000 7f1b200bd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1b18133420 0x7f1b181337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:53.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.692+0000 7f1b200bd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1b18133e30 0x7f1b1812ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:53.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.693+0000 7f1b200bd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1b18134260 0x7f1b181304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:53.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.693+0000 7f1b200bd700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b1806b990 con 0x7f1b18133420 2023-09-30T22:30:53.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.693+0000 7f1b200bd700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b180a0590 con 0x7f1b18134260 2023-09-30T22:30:53.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.693+0000 7f1b200bd700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1809fb80 con 0x7f1b18133e30 2023-09-30T22:30:53.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.693+0000 7f1b1d658700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1b18133e30 0x7f1b1812ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:53.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.693+0000 7f1b1d658700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1b18133e30 0x7f1b1812ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35560/0 (socket says 172.21.15.16:35560) 2023-09-30T22:30:53.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.693+0000 7f1b1d658700 1 -- 172.21.15.16:0/113030955 learned_addr learned my addr 172.21.15.16:0/113030955 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:53.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.693+0000 7f1b1e65a700 1 --2- 172.21.15.16:0/113030955 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1b18134260 0x7f1b181304d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:53.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.693+0000 7f1b1de59700 1 --2- 172.21.15.16:0/113030955 >> v2:172.21.15.16:3300/0 conn(0x7f1b18133420 0x7f1b181337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:53.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.693+0000 7f1b1d658700 1 -- 172.21.15.16:0/113030955 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1b18134260 msgr2=0x7f1b181304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:53.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.693+0000 7f1b1d658700 1 --2- 172.21.15.16:0/113030955 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1b18134260 0x7f1b181304d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:53.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.693+0000 7f1b1d658700 1 -- 172.21.15.16:0/113030955 >> v2:172.21.15.16:3300/0 conn(0x7f1b18133420 msgr2=0x7f1b181337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:53.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.693+0000 7f1b1d658700 1 --2- 172.21.15.16:0/113030955 >> v2:172.21.15.16:3300/0 conn(0x7f1b18133420 0x7f1b181337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:53.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.693+0000 7f1b1d658700 1 -- 172.21.15.16:0/113030955 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b180a0030 con 0x7f1b18133e30 2023-09-30T22:30:53.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.693+0000 7f1b1e65a700 1 --2- 172.21.15.16:0/113030955 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1b18134260 0x7f1b181304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:53.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.693+0000 7f1b1d658700 1 --2- 172.21.15.16:0/113030955 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1b18133e30 0x7f1b1812ff90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1b080098f0 tx=0x7f1b08009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff1e097b1d8cc806 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:53.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.694+0000 7f1b1ce57700 1 -- 172.21.15.16:0/113030955 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b08011830 con 0x7f1b18133e30 2023-09-30T22:30:53.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.694+0000 7f1b1ce57700 1 -- 172.21.15.16:0/113030955 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b08011990 con 0x7f1b18133e30 2023-09-30T22:30:53.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.694+0000 7f1b1ce57700 1 -- 172.21.15.16:0/113030955 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b08011d20 con 0x7f1b18133e30 2023-09-30T22:30:53.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.694+0000 7f1b200bd700 1 -- 172.21.15.16:0/113030955 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1b18133e30 msgr2=0x7f1b1812ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:53.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.694+0000 7f1b200bd700 1 --2- 172.21.15.16:0/113030955 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1b18133e30 0x7f1b1812ff90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1b080098f0 tx=0x7f1b08009c00 comp rx=0 tx=0).stop 2023-09-30T22:30:53.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.694+0000 7f1b200bd700 1 -- 172.21.15.16:0/113030955 shutdown_connections 2023-09-30T22:30:53.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.694+0000 7f1b200bd700 1 --2- 172.21.15.16:0/113030955 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1b18134260 0x7f1b181304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:53.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.694+0000 7f1b200bd700 1 --2- 172.21.15.16:0/113030955 >> v2:172.21.15.16:3300/0 conn(0x7f1b18133420 0x7f1b181337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:53.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.694+0000 7f1b200bd700 1 --2- 172.21.15.16:0/113030955 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1b18133e30 0x7f1b1812ff90 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:53.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.694+0000 7f1b200bd700 1 -- 172.21.15.16:0/113030955 >> 172.21.15.16:0/113030955 conn(0x7f1b180a4210 msgr2=0x7f1b180a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:53.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.694+0000 7f1b200bd700 1 -- 172.21.15.16:0/113030955 shutdown_connections 2023-09-30T22:30:53.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.694+0000 7f1b200bd700 1 -- 172.21.15.16:0/113030955 wait complete. 2023-09-30T22:30:53.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.696+0000 7f1b200bd700 1 Processor -- start 2023-09-30T22:30:53.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.696+0000 7f1b200bd700 1 -- start start 2023-09-30T22:30:53.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.696+0000 7f1b200bd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1b18133420 0x7f1b1809c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:53.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.696+0000 7f1b200bd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b18133e30 0x7f1b1809c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:53.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.696+0000 7f1b200bd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1b18134260 0x7f1b1809cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:53.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.696+0000 7f1b200bd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1811fb60 con 0x7f1b18133e30 2023-09-30T22:30:53.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.696+0000 7f1b200bd700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1811f540 con 0x7f1b18133420 2023-09-30T22:30:53.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.696+0000 7f1b200bd700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1809de30 con 0x7f1b18134260 2023-09-30T22:30:53.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.696+0000 7f1b1e65a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1b18134260 0x7f1b1809cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:53.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.696+0000 7f1b1e65a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1b18134260 0x7f1b1809cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35572/0 (socket says 172.21.15.16:35572) 2023-09-30T22:30:53.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.696+0000 7f1b1e65a700 1 -- 172.21.15.16:0/747148468 learned_addr learned my addr 172.21.15.16:0/747148468 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:53.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.696+0000 7f1b1d658700 1 --2- 172.21.15.16:0/747148468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b18133e30 0x7f1b1809c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:53.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.697+0000 7fb8f68ec700 1 Processor -- start 2023-09-30T22:30:53.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.697+0000 7fb8f68ec700 1 -- start start 2023-09-30T22:30:53.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.697+0000 7f1b1de59700 1 --2- 172.21.15.16:0/747148468 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1b18133420 0x7f1b1809c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:53.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.697+0000 7f1b1e65a700 1 -- 172.21.15.16:0/747148468 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1b18133420 msgr2=0x7f1b1809c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:53.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.697+0000 7f1b1e65a700 1 --2- 172.21.15.16:0/747148468 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1b18133420 0x7f1b1809c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:53.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.697+0000 7f1b1e65a700 1 -- 172.21.15.16:0/747148468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b18133e30 msgr2=0x7f1b1809c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:53.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.697+0000 7f1b1e65a700 1 --2- 172.21.15.16:0/747148468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b18133e30 0x7f1b1809c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:53.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.697+0000 7f1b1e65a700 1 -- 172.21.15.16:0/747148468 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b10009a70 con 0x7f1b18134260 2023-09-30T22:30:53.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.697+0000 7f1b1e65a700 1 --2- 172.21.15.16:0/747148468 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1b18134260 0x7f1b1809cf00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1b1000ef90 tx=0x7f1b1000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:53.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.697+0000 7f1b0e7fc700 1 -- 172.21.15.16:0/747148468 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b1000cd40 con 0x7f1b18134260 2023-09-30T22:30:53.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.697+0000 7f1b0e7fc700 1 -- 172.21.15.16:0/747148468 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b1000f040 con 0x7f1b18134260 2023-09-30T22:30:53.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.697+0000 7f1b0e7fc700 1 -- 172.21.15.16:0/747148468 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b1000cea0 con 0x7f1b18134260 2023-09-30T22:30:53.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.697+0000 7f1b200bd700 1 -- 172.21.15.16:0/747148468 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b08009550 con 0x7f1b18134260 2023-09-30T22:30:53.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.697+0000 7f1b200bd700 1 -- 172.21.15.16:0/747148468 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b18131400 con 0x7f1b18134260 2023-09-30T22:30:53.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.697+0000 7fb8f68ec700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb8f009f410 0x7fb8f0136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:53.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.697+0000 7fb8f68ec700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb8f0136fe0 0x7fb8f01393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:53.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.697+0000 7fb8f68ec700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8f0070940 con 0x7fb8f009e930 2023-09-30T22:30:53.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.697+0000 7fb8f68ec700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8f006fd60 con 0x7fb8f009f410 2023-09-30T22:30:53.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.697+0000 7fb8f68ec700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f006fa80 con 0x7fb8f0136fe0 2023-09-30T22:30:53.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.698+0000 7fb8f4e89700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb8f0136fe0 0x7fb8f01393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:53.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.698+0000 7fb8f4e89700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb8f0136fe0 0x7fb8f01393c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41156/0 (socket says 172.21.15.16:41156) 2023-09-30T22:30:53.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.698+0000 7fb8f4e89700 1 -- 172.21.15.16:0/240373079 learned_addr learned my addr 172.21.15.16:0/240373079 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:53.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.698+0000 7fb8ef7fe700 1 --2- 172.21.15.16:0/240373079 >> v2:172.21.15.16:3300/0 conn(0x7fb8f009f410 0x7fb8f0136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:53.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.698+0000 7fb8f4e89700 1 -- 172.21.15.16:0/240373079 >> v2:172.21.15.16:3300/0 conn(0x7fb8f009f410 msgr2=0x7fb8f0136aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:53.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.698+0000 7fb8f4e89700 1 --2- 172.21.15.16:0/240373079 >> v2:172.21.15.16:3300/0 conn(0x7fb8f009f410 0x7fb8f0136aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:53.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.698+0000 7fb8f4e89700 1 -- 172.21.15.16:0/240373079 >> v1:172.21.15.16:6789/0 conn(0x7fb8f009e930 legacy=0x7fb8f009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:53.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.698+0000 7fb8f4e89700 1 -- 172.21.15.16:0/240373079 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8f009fb60 con 0x7fb8f0136fe0 2023-09-30T22:30:53.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.698+0000 7fb8f4e89700 1 --2- 172.21.15.16:0/240373079 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb8f0136fe0 0x7fb8f01393c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb8e80131e0 tx=0x7fb8e80134f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2474c26cc93e618e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:53.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.698+0000 7fb8eeffd700 1 -- 172.21.15.16:0/240373079 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e8016070 con 0x7fb8f0136fe0 2023-09-30T22:30:53.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.698+0000 7fb8eeffd700 1 -- 172.21.15.16:0/240373079 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8e8013df0 con 0x7fb8f0136fe0 2023-09-30T22:30:53.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.698+0000 7fb8eeffd700 1 -- 172.21.15.16:0/240373079 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e8004250 con 0x7fb8f0136fe0 2023-09-30T22:30:53.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.699+0000 7f1b0e7fc700 1 -- 172.21.15.16:0/747148468 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1b100146c0 con 0x7f1b18134260 2023-09-30T22:30:53.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.699+0000 7f1b200bd700 1 -- 172.21.15.16:0/747148468 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b18074ea0 con 0x7f1b18134260 2023-09-30T22:30:53.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.699+0000 7f1b0e7fc700 1 -- 172.21.15.16:0/747148468 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f1b10040350 con 0x7f1b18134260 2023-09-30T22:30:53.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.699+0000 7fb8f68ec700 1 -- 172.21.15.16:0/240373079 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb8f0136fe0 msgr2=0x7fb8f01393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:53.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.699+0000 7fb8f68ec700 1 --2- 172.21.15.16:0/240373079 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb8f0136fe0 0x7fb8f01393c0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb8e80131e0 tx=0x7fb8e80134f0 comp rx=0 tx=0).stop 2023-09-30T22:30:53.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.699+0000 7fb8f68ec700 1 -- 172.21.15.16:0/240373079 shutdown_connections 2023-09-30T22:30:53.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.699+0000 7fb8f68ec700 1 --2- 172.21.15.16:0/240373079 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb8f0136fe0 0x7fb8f01393c0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:53.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.699+0000 7fb8f68ec700 1 --2- 172.21.15.16:0/240373079 >> v2:172.21.15.16:3300/0 conn(0x7fb8f009f410 0x7fb8f0136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:53.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.699+0000 7fb8f68ec700 1 -- 172.21.15.16:0/240373079 >> 172.21.15.16:0/240373079 conn(0x7fb8f00a4210 msgr2=0x7fb8f00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:53.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.699+0000 7fb8f68ec700 1 -- 172.21.15.16:0/240373079 shutdown_connections 2023-09-30T22:30:53.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.699+0000 7fb8f68ec700 1 -- 172.21.15.16:0/240373079 wait complete. 2023-09-30T22:30:53.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.700+0000 7fb8f68ec700 1 Processor -- start 2023-09-30T22:30:53.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.701+0000 7fb8f68ec700 1 -- start start 2023-09-30T22:30:53.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.701+0000 7fb8f68ec700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb8f009e930 0x7fb8f011a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:53.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.701+0000 7fb8f68ec700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb8f009f410 0x7fb8f011abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:53.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.701+0000 7fb8f68ec700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8f0136fe0 0x7fb8f011b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:53.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.701+0000 7fb8f68ec700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f011dd50 con 0x7fb8f0136fe0 2023-09-30T22:30:53.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.701+0000 7fb8f68ec700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f011df30 con 0x7fb8f009e930 2023-09-30T22:30:53.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.701+0000 7fb8f68ec700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f006d810 con 0x7fb8f009f410 2023-09-30T22:30:53.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.701+0000 7fb8effff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb8f009e930 0x7fb8f011a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:53.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.701+0000 7fb8effff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb8f009e930 0x7fb8f011a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41166/0 (socket says 172.21.15.16:41166) 2023-09-30T22:30:53.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.701+0000 7fb8effff700 1 -- 172.21.15.16:0/630493755 learned_addr learned my addr 172.21.15.16:0/630493755 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:53.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.701+0000 7fb8ef7fe700 1 --2- 172.21.15.16:0/630493755 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb8f009f410 0x7fb8f011abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:53.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.701+0000 7fb8f4e89700 1 --2- 172.21.15.16:0/630493755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8f0136fe0 0x7fb8f011b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:53.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.702+0000 7fb8effff700 1 -- 172.21.15.16:0/630493755 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb8f009f410 msgr2=0x7fb8f011abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:53.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.702+0000 7fb8effff700 1 --2- 172.21.15.16:0/630493755 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb8f009f410 0x7fb8f011abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:53.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.702+0000 7fb8effff700 1 -- 172.21.15.16:0/630493755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8f0136fe0 msgr2=0x7fb8f011b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:53.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.702+0000 7fb8effff700 1 --2- 172.21.15.16:0/630493755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8f0136fe0 0x7fb8f011b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:53.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.702+0000 7fb8effff700 1 -- 172.21.15.16:0/630493755 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e8011df0 con 0x7fb8f009e930 2023-09-30T22:30:53.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.702+0000 7fb8f4e89700 1 --2- 172.21.15.16:0/630493755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8f0136fe0 0x7fb8f011b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:53.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.702+0000 7fb8ef7fe700 1 --2- 172.21.15.16:0/630493755 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb8f009f410 0x7fb8f011abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:53.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.702+0000 7fb8effff700 1 --2- 172.21.15.16:0/630493755 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb8f009e930 0x7fb8f011a680 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb8e000cc80 tx=0x7fb8e000cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:53.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.702+0000 7fb8ecff9700 1 -- 172.21.15.16:0/630493755 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e000aac0 con 0x7fb8f009e930 2023-09-30T22:30:53.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.702+0000 7fb8ecff9700 1 -- 172.21.15.16:0/630493755 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8e000ac20 con 0x7fb8f009e930 2023-09-30T22:30:53.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.702+0000 7fb8ecff9700 1 -- 172.21.15.16:0/630493755 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e0012420 con 0x7fb8f009e930 2023-09-30T22:30:53.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.702+0000 7fb8f68ec700 1 -- 172.21.15.16:0/630493755 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8f011c100 con 0x7fb8f009e930 2023-09-30T22:30:53.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.702+0000 7fb8f68ec700 1 -- 172.21.15.16:0/630493755 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8f011cd60 con 0x7fb8f009e930 2023-09-30T22:30:53.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.703+0000 7fb8ecff9700 1 -- 172.21.15.16:0/630493755 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb8e0012580 con 0x7fb8f009e930 2023-09-30T22:30:53.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.703+0000 7fb8ecff9700 1 -- 172.21.15.16:0/630493755 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fb8e003fd30 con 0x7fb8f009e930 2023-09-30T22:30:53.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.703+0000 7fb8f68ec700 1 -- 172.21.15.16:0/630493755 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8f011dbb0 con 0x7fb8f009e930 2023-09-30T22:30:53.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.708+0000 7fb8ecff9700 1 -- 172.21.15.16:0/630493755 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb8f011dbb0 con 0x7fb8f009e930 2023-09-30T22:30:53.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.708+0000 7f1b0e7fc700 1 -- 172.21.15.16:0/747148468 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1b10007900 con 0x7f1b18134260 2023-09-30T22:30:53.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.889+0000 7f413fe52700 1 -- 172.21.15.16:0/3189010855 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f413806b9e0 con 0x7f413806e9d0 2023-09-30T22:30:53.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.889+0000 7f412e7fc700 1 -- 172.21.15.16:0/3189010855 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4134020020 con 0x7f413806e9d0 2023-09-30T22:30:53.890 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:30:53.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.892+0000 7f413fe52700 1 -- 172.21.15.16:0/3189010855 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f413806e9d0 msgr2=0x7f413811b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:53.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.892+0000 7f413fe52700 1 --2- 172.21.15.16:0/3189010855 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f413806e9d0 0x7f413811b540 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4134009ec0 tx=0x7f413400eec0 comp rx=0 tx=0).stop 2023-09-30T22:30:53.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.892+0000 7f413fe52700 1 -- 172.21.15.16:0/3189010855 shutdown_connections 2023-09-30T22:30:53.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.892+0000 7f413fe52700 1 --2- 172.21.15.16:0/3189010855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4138131f40 0x7f413811bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:53.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.892+0000 7f413fe52700 1 --2- 172.21.15.16:0/3189010855 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f413806e9d0 0x7f413811b540 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:53.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.892+0000 7f413fe52700 1 --2- 172.21.15.16:0/3189010855 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4138131630 0x7f413811ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:53.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.892+0000 7f413fe52700 1 -- 172.21.15.16:0/3189010855 >> 172.21.15.16:0/3189010855 conn(0x7f41380a2aa0 msgr2=0x7f41381b8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:53.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.892+0000 7f413fe52700 1 -- 172.21.15.16:0/3189010855 shutdown_connections 2023-09-30T22:30:53.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.892+0000 7f413fe52700 1 -- 172.21.15.16:0/3189010855 wait complete. 2023-09-30T22:30:53.906 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:30:53.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.915+0000 7f1b200bd700 1 -- 172.21.15.16:0/747148468 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1b1806b250 con 0x7f1b18134260 2023-09-30T22:30:53.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.915+0000 7f1b0e7fc700 1 -- 172.21.15.16:0/747148468 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1b10007330 con 0x7f1b18134260 2023-09-30T22:30:53.915 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:30:53.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.917+0000 7f1b200bd700 1 -- 172.21.15.16:0/747148468 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1b18134260 msgr2=0x7f1b1809cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:53.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.917+0000 7f1b200bd700 1 --2- 172.21.15.16:0/747148468 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1b18134260 0x7f1b1809cf00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1b1000ef90 tx=0x7f1b1000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:30:53.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.917+0000 7f1b200bd700 1 -- 172.21.15.16:0/747148468 shutdown_connections 2023-09-30T22:30:53.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.917+0000 7f1b200bd700 1 --2- 172.21.15.16:0/747148468 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1b18133420 0x7f1b1809c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:53.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.917+0000 7f1b200bd700 1 --2- 172.21.15.16:0/747148468 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1b18134260 0x7f1b1809cf00 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:53.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.917+0000 7f1b200bd700 1 --2- 172.21.15.16:0/747148468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b18133e30 0x7f1b1809c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:53.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.917+0000 7f1b200bd700 1 -- 172.21.15.16:0/747148468 >> 172.21.15.16:0/747148468 conn(0x7f1b180a4210 msgr2=0x7f1b181302c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:53.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.917+0000 7f1b200bd700 1 -- 172.21.15.16:0/747148468 shutdown_connections 2023-09-30T22:30:53.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.917+0000 7f1b200bd700 1 -- 172.21.15.16:0/747148468 wait complete. 2023-09-30T22:30:53.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.918+0000 7fb8f68ec700 1 -- 172.21.15.16:0/630493755 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb8f011dbb0 con 0x7fb8f009e930 2023-09-30T22:30:53.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.919+0000 7fb8ecff9700 1 -- 172.21.15.16:0/630493755 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb8f011dbb0 con 0x7fb8f009e930 2023-09-30T22:30:53.919 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:30:53.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.921+0000 7fb8f68ec700 1 -- 172.21.15.16:0/630493755 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb8f009e930 msgr2=0x7fb8f011a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:53.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.921+0000 7fb8f68ec700 1 --2- 172.21.15.16:0/630493755 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb8f009e930 0x7fb8f011a680 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb8e000cc80 tx=0x7fb8e000cf90 comp rx=0 tx=0).stop 2023-09-30T22:30:53.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.921+0000 7fb8f68ec700 1 -- 172.21.15.16:0/630493755 shutdown_connections 2023-09-30T22:30:53.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.921+0000 7fb8f68ec700 1 --2- 172.21.15.16:0/630493755 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb8f009e930 0x7fb8f011a680 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:53.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.921+0000 7fb8f68ec700 1 --2- 172.21.15.16:0/630493755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8f0136fe0 0x7fb8f011b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:53.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.921+0000 7fb8f68ec700 1 --2- 172.21.15.16:0/630493755 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb8f009f410 0x7fb8f011abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:53.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.921+0000 7fb8f68ec700 1 -- 172.21.15.16:0/630493755 >> 172.21.15.16:0/630493755 conn(0x7fb8f00a4210 msgr2=0x7fb8f00a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:53.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.921+0000 7fb8f68ec700 1 -- 172.21.15.16:0/630493755 shutdown_connections 2023-09-30T22:30:53.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:53.921+0000 7fb8f68ec700 1 -- 172.21.15.16:0/630493755 wait complete. 2023-09-30T22:30:53.933 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:30:53.936 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:30:53.976 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:54.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.116+0000 7f5d4554f700 1 Processor -- start 2023-09-30T22:30:54.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.116+0000 7f5d4554f700 1 -- start start 2023-09-30T22:30:54.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.117+0000 7f5d4554f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d400a0a30 0x7f5d400a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:54.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.117+0000 7f5d4554f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d4009d630 0x7f5d400a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:54.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.117+0000 7f5d4554f700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d400a2a70 con 0x7f5d40133930 2023-09-30T22:30:54.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.117+0000 7f5d4554f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d400a01d0 con 0x7f5d4009d630 2023-09-30T22:30:54.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.117+0000 7f5d4554f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4009e2b0 con 0x7f5d400a0a30 2023-09-30T22:30:54.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.117+0000 7f5d3effd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d400a0a30 0x7f5d400a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:54.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.117+0000 7f5d3e7fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d4009d630 0x7f5d400a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:54.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.117+0000 7f5d3f7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5d40133930 0x7f5d40133dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40878/0 (socket says 172.21.15.16:40878) 2023-09-30T22:30:54.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.117+0000 7f5d3f7fe700 1 -- 172.21.15.16:0/108364809 learned_addr learned my addr 172.21.15.16:0/108364809 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:30:54.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.117+0000 7f5d3e7fc700 1 -- 172.21.15.16:0/108364809 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d400a0a30 msgr2=0x7f5d400a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:54.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.117+0000 7f5d3e7fc700 1 --2- 172.21.15.16:0/108364809 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d400a0a30 0x7f5d400a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:54.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.117+0000 7f5d3e7fc700 1 -- 172.21.15.16:0/108364809 >> v1:172.21.15.16:6789/0 conn(0x7f5d40133930 legacy=0x7f5d40133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:54.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.117+0000 7f5d3e7fc700 1 -- 172.21.15.16:0/108364809 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d4009dc50 con 0x7f5d4009d630 2023-09-30T22:30:54.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.117+0000 7f5d3dffb700 1 -- 172.21.15.16:0/108364809 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 472014207 0 0) 0x7f5d400a2a70 con 0x7f5d40133930 2023-09-30T22:30:54.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.117+0000 7f5d3effd700 1 --2- 172.21.15.16:0/108364809 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d400a0a30 0x7f5d400a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:54.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.117+0000 7f5d3e7fc700 1 --2- 172.21.15.16:0/108364809 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d4009d630 0x7f5d400a1340 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5d28009870 tx=0x7f5d28009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88486fd8f62eb710 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:54.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.118+0000 7f5d3dffb700 1 -- 172.21.15.16:0/108364809 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d28011830 con 0x7f5d4009d630 2023-09-30T22:30:54.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.118+0000 7f5d3dffb700 1 -- 172.21.15.16:0/108364809 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d28011990 con 0x7f5d4009d630 2023-09-30T22:30:54.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.118+0000 7f5d3dffb700 1 -- 172.21.15.16:0/108364809 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d28011d20 con 0x7f5d4009d630 2023-09-30T22:30:54.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.118+0000 7f5d4554f700 1 -- 172.21.15.16:0/108364809 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d4009d630 msgr2=0x7f5d400a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:54.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.118+0000 7f5d4554f700 1 --2- 172.21.15.16:0/108364809 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d4009d630 0x7f5d400a1340 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5d28009870 tx=0x7f5d28009b80 comp rx=0 tx=0).stop 2023-09-30T22:30:54.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.118+0000 7f5d4554f700 1 -- 172.21.15.16:0/108364809 shutdown_connections 2023-09-30T22:30:54.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.118+0000 7f5d4554f700 1 --2- 172.21.15.16:0/108364809 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d400a0a30 0x7f5d400a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:54.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.118+0000 7f5d4554f700 1 --2- 172.21.15.16:0/108364809 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d4009d630 0x7f5d400a1340 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:54.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.118+0000 7f5d4554f700 1 -- 172.21.15.16:0/108364809 >> 172.21.15.16:0/108364809 conn(0x7f5d400a3050 msgr2=0x7f5d400a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:54.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.118+0000 7f5d4554f700 1 -- 172.21.15.16:0/108364809 shutdown_connections 2023-09-30T22:30:54.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.118+0000 7f5d4554f700 1 -- 172.21.15.16:0/108364809 wait complete. 2023-09-30T22:30:54.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.119+0000 7f5d4554f700 1 Processor -- start 2023-09-30T22:30:54.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.119+0000 7f5d4554f700 1 -- start start 2023-09-30T22:30:54.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.120+0000 7f5d4554f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d4009d630 0x7f5d401c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:54.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.120+0000 7f5d4554f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d400a0a30 0x7f5d401c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:54.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.120+0000 7f5d4554f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d40133930 0x7f5d401c8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:54.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.120+0000 7f5d4554f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d401334e0 con 0x7f5d400a0a30 2023-09-30T22:30:54.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.120+0000 7f5d4554f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d401206e0 con 0x7f5d4009d630 2023-09-30T22:30:54.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.120+0000 7f5d4554f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5d400a0410 con 0x7f5d40133930 2023-09-30T22:30:54.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.120+0000 7f5d3effd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d4009d630 0x7f5d401c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:54.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.120+0000 7f5d3e7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d400a0a30 0x7f5d401c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:54.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.120+0000 7f5d3f7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d40133930 0x7f5d401c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:54.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.120+0000 7f5d3effd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d4009d630 0x7f5d401c7e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41188/0 (socket says 172.21.15.16:41188) 2023-09-30T22:30:54.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.120+0000 7f5d3e7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d400a0a30 0x7f5d401c83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43952/0 (socket says 172.21.15.16:43952) 2023-09-30T22:30:54.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.120+0000 7f5d3effd700 1 -- 172.21.15.16:0/1280284762 learned_addr learned my addr 172.21.15.16:0/1280284762 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:54.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.120+0000 7f5d3f7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d40133930 0x7f5d401c8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35606/0 (socket says 172.21.15.16:35606) 2023-09-30T22:30:54.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.120+0000 7f5d3f7fe700 1 -- 172.21.15.16:0/1280284762 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d4009d630 msgr2=0x7f5d401c7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:54.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.120+0000 7f5d3f7fe700 1 --2- 172.21.15.16:0/1280284762 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d4009d630 0x7f5d401c7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:54.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.120+0000 7f5d3f7fe700 1 -- 172.21.15.16:0/1280284762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d400a0a30 msgr2=0x7f5d401c83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:54.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.120+0000 7f5d3f7fe700 1 --2- 172.21.15.16:0/1280284762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d400a0a30 0x7f5d401c83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:54.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.120+0000 7f5d3f7fe700 1 -- 172.21.15.16:0/1280284762 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d300094a0 con 0x7f5d40133930 2023-09-30T22:30:54.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.121+0000 7f5d3f7fe700 1 --2- 172.21.15.16:0/1280284762 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d40133930 0x7f5d401c8900 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5d3800cc10 tx=0x7f5d3800cf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:54.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.121+0000 7f5d377fe700 1 -- 172.21.15.16:0/1280284762 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3800aa00 con 0x7f5d40133930 2023-09-30T22:30:54.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.121+0000 7f5d377fe700 1 -- 172.21.15.16:0/1280284762 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d3800ab80 con 0x7f5d40133930 2023-09-30T22:30:54.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.121+0000 7f5d4554f700 1 -- 172.21.15.16:0/1280284762 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d280094a0 con 0x7f5d40133930 2023-09-30T22:30:54.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.121+0000 7f5d377fe700 1 -- 172.21.15.16:0/1280284762 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d380070d0 con 0x7f5d40133930 2023-09-30T22:30:54.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.121+0000 7f5d4554f700 1 -- 172.21.15.16:0/1280284762 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d40122c00 con 0x7f5d40133930 2023-09-30T22:30:54.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.122+0000 7f5d377fe700 1 -- 172.21.15.16:0/1280284762 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5d38007230 con 0x7f5d40133930 2023-09-30T22:30:54.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.122+0000 7f5d377fe700 1 -- 172.21.15.16:0/1280284762 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f5d3803f070 con 0x7f5d40133930 2023-09-30T22:30:54.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.122+0000 7f5d4554f700 1 -- 172.21.15.16:0/1280284762 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d400747f0 con 0x7f5d40133930 2023-09-30T22:30:54.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.127+0000 7f5d377fe700 1 -- 172.21.15.16:0/1280284762 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5d38018020 con 0x7f5d40133930 2023-09-30T22:30:54.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.334+0000 7f5d4554f700 1 -- 172.21.15.16:0/1280284762 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5d40072980 con 0x7f5d40133930 2023-09-30T22:30:54.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.335+0000 7f5d377fe700 1 -- 172.21.15.16:0/1280284762 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5d380074e0 con 0x7f5d40133930 2023-09-30T22:30:54.335 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:30:54.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.337+0000 7f5d4554f700 1 -- 172.21.15.16:0/1280284762 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d40133930 msgr2=0x7f5d401c8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:54.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.337+0000 7f5d4554f700 1 --2- 172.21.15.16:0/1280284762 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d40133930 0x7f5d401c8900 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f5d3800cc10 tx=0x7f5d3800cf20 comp rx=0 tx=0).stop 2023-09-30T22:30:54.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.337+0000 7f5d4554f700 1 -- 172.21.15.16:0/1280284762 shutdown_connections 2023-09-30T22:30:54.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.337+0000 7f5d4554f700 1 --2- 172.21.15.16:0/1280284762 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d4009d630 0x7f5d401c7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:54.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.337+0000 7f5d4554f700 1 --2- 172.21.15.16:0/1280284762 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d40133930 0x7f5d401c8900 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:54.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.337+0000 7f5d4554f700 1 --2- 172.21.15.16:0/1280284762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d400a0a30 0x7f5d401c83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:54.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.337+0000 7f5d4554f700 1 -- 172.21.15.16:0/1280284762 >> 172.21.15.16:0/1280284762 conn(0x7f5d400a3050 msgr2=0x7f5d401bb960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:54.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.337+0000 7f5d4554f700 1 -- 172.21.15.16:0/1280284762 shutdown_connections 2023-09-30T22:30:54.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:54.337+0000 7f5d4554f700 1 -- 172.21.15.16:0/1280284762 wait complete. 2023-09-30T22:30:54.351 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:30:54.909 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:54.934 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:54.937 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:55.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.058+0000 7f18f085a700 1 Processor -- start 2023-09-30T22:30:55.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.059+0000 7f18f085a700 1 -- start start 2023-09-30T22:30:55.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.059+0000 7f18f085a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f18ec1337f0 0x7f18ec138880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:55.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.059+0000 7f18f085a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f18ec138dc0 0x7f18ec13b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:55.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.059+0000 7f18f085a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18ec06b990 con 0x7f18ec133420 2023-09-30T22:30:55.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.059+0000 7f18f085a700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18ec09b0e0 con 0x7f18ec1337f0 2023-09-30T22:30:55.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.059+0000 7f18f085a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f18ec133c20 con 0x7f18ec138dc0 2023-09-30T22:30:55.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.059+0000 7f18e1d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f18ec1337f0 0x7f18ec138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:55.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.059+0000 7f18ead9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f18ec138dc0 0x7f18ec13b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:55.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.059+0000 7f18e1d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f18ec1337f0 0x7f18ec138880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43964/0 (socket says 172.21.15.16:43964) 2023-09-30T22:30:55.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.059+0000 7f18e1d9b700 1 -- 172.21.15.16:0/691429234 learned_addr learned my addr 172.21.15.16:0/691429234 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:55.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.059+0000 7f18ead9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f18ec138dc0 0x7f18ec13b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35614/0 (socket says 172.21.15.16:35614) 2023-09-30T22:30:55.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.059+0000 7f18ea59c700 1 --1- 172.21.15.16:0/691429234 >> v1:172.21.15.16:6789/0 conn(0x7f18ec133420 0x7f18ec133e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40894/0 (socket says 172.21.15.16:40894) 2023-09-30T22:30:55.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.060+0000 7f18e1d9b700 1 -- 172.21.15.16:0/691429234 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f18ec138dc0 msgr2=0x7f18ec13b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:55.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.060+0000 7f18e1d9b700 1 --2- 172.21.15.16:0/691429234 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f18ec138dc0 0x7f18ec13b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.060+0000 7f18e1d9b700 1 -- 172.21.15.16:0/691429234 >> v1:172.21.15.16:6789/0 conn(0x7f18ec133420 legacy=0x7f18ec133e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:55.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.060+0000 7f18e1d9b700 1 -- 172.21.15.16:0/691429234 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18ec0a0590 con 0x7f18ec1337f0 2023-09-30T22:30:55.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.060+0000 7f18e9d9b700 1 -- 172.21.15.16:0/691429234 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3981700188 0 0) 0x7f18ec06b990 con 0x7f18ec133420 2023-09-30T22:30:55.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.060+0000 7f18e1d9b700 1 --2- 172.21.15.16:0/691429234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18ec1337f0 0x7f18ec138880 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f18d40098f0 tx=0x7f18d4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=422dca300fb44c5b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:55.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.060+0000 7f18e9d9b700 1 -- 172.21.15.16:0/691429234 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d4011830 con 0x7f18ec1337f0 2023-09-30T22:30:55.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.060+0000 7f18e9d9b700 1 -- 172.21.15.16:0/691429234 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18d4011990 con 0x7f18ec1337f0 2023-09-30T22:30:55.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.060+0000 7f18e9d9b700 1 -- 172.21.15.16:0/691429234 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d4011d20 con 0x7f18ec1337f0 2023-09-30T22:30:55.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.061+0000 7f18f085a700 1 -- 172.21.15.16:0/691429234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18ec1337f0 msgr2=0x7f18ec138880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:55.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.061+0000 7f18f085a700 1 --2- 172.21.15.16:0/691429234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18ec1337f0 0x7f18ec138880 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f18d40098f0 tx=0x7f18d4009c00 comp rx=0 tx=0).stop 2023-09-30T22:30:55.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.061+0000 7f18f085a700 1 -- 172.21.15.16:0/691429234 shutdown_connections 2023-09-30T22:30:55.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.061+0000 7f18f085a700 1 --2- 172.21.15.16:0/691429234 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f18ec138dc0 0x7f18ec13b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.061+0000 7f18f085a700 1 --2- 172.21.15.16:0/691429234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18ec1337f0 0x7f18ec138880 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.061+0000 7f18f085a700 1 -- 172.21.15.16:0/691429234 >> 172.21.15.16:0/691429234 conn(0x7f18ec0a4210 msgr2=0x7f18ec0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:55.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.061+0000 7f18f085a700 1 -- 172.21.15.16:0/691429234 shutdown_connections 2023-09-30T22:30:55.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.061+0000 7f18f085a700 1 -- 172.21.15.16:0/691429234 wait complete. 2023-09-30T22:30:55.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.062+0000 7f18f085a700 1 Processor -- start 2023-09-30T22:30:55.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.063+0000 7f18f085a700 1 -- start start 2023-09-30T22:30:55.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.063+0000 7f18f085a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f18ec133420 0x7f18ec12ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:55.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.063+0000 7f18f085a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18ec1337f0 0x7f18ec12f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:55.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.063+0000 7f18f085a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f18ec138dc0 0x7f18ec12f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:55.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.063+0000 7f18f085a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18ec09b750 con 0x7f18ec1337f0 2023-09-30T22:30:55.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.063+0000 7f18f085a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18ec09e490 con 0x7f18ec138dc0 2023-09-30T22:30:55.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.063+0000 7f18f085a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f18ec1388a0 con 0x7f18ec133420 2023-09-30T22:30:55.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.063+0000 7f18ead9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f18ec138dc0 0x7f18ec12f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:55.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.063+0000 7f18e1d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18ec1337f0 0x7f18ec12f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:55.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.063+0000 7f18ea59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f18ec133420 0x7f18ec12ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:55.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.063+0000 7f18ead9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f18ec138dc0 0x7f18ec12f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41196/0 (socket says 172.21.15.16:41196) 2023-09-30T22:30:55.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.063+0000 7f18ead9d700 1 -- 172.21.15.16:0/2906851943 learned_addr learned my addr 172.21.15.16:0/2906851943 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:55.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.063+0000 7f18e1d9b700 1 --2- 172.21.15.16:0/2906851943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18ec1337f0 0x7f18ec12f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43978/0 (socket says 172.21.15.16:43978) 2023-09-30T22:30:55.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.064+0000 7f18ea59c700 1 -- 172.21.15.16:0/2906851943 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f18ec138dc0 msgr2=0x7f18ec12f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:55.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.064+0000 7f18ea59c700 1 --2- 172.21.15.16:0/2906851943 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f18ec138dc0 0x7f18ec12f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.064+0000 7f18ea59c700 1 -- 172.21.15.16:0/2906851943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18ec1337f0 msgr2=0x7f18ec12f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:55.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.064+0000 7f18ea59c700 1 --2- 172.21.15.16:0/2906851943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18ec1337f0 0x7f18ec12f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.064+0000 7f18ea59c700 1 -- 172.21.15.16:0/2906851943 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d4009550 con 0x7f18ec133420 2023-09-30T22:30:55.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.064+0000 7f18ead9d700 1 --2- 172.21.15.16:0/2906851943 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f18ec138dc0 0x7f18ec12f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:55.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.064+0000 7f18ea59c700 1 --2- 172.21.15.16:0/2906851943 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f18ec133420 0x7f18ec12ee50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f18dc00c470 tx=0x7f18dc00c780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:55.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.064+0000 7f18e37fe700 1 -- 172.21.15.16:0/2906851943 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18dc004c80 con 0x7f18ec133420 2023-09-30T22:30:55.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.064+0000 7f18e37fe700 1 -- 172.21.15.16:0/2906851943 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18dc004de0 con 0x7f18ec133420 2023-09-30T22:30:55.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.064+0000 7f18f085a700 1 -- 172.21.15.16:0/2906851943 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18ec09baf0 con 0x7f18ec133420 2023-09-30T22:30:55.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.064+0000 7f18e37fe700 1 -- 172.21.15.16:0/2906851943 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18dc007310 con 0x7f18ec133420 2023-09-30T22:30:55.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.064+0000 7f18f085a700 1 -- 172.21.15.16:0/2906851943 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18ec09d520 con 0x7f18ec133420 2023-09-30T22:30:55.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.065+0000 7f18e37fe700 1 -- 172.21.15.16:0/2906851943 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f18dc007470 con 0x7f18ec133420 2023-09-30T22:30:55.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.065+0000 7f18e37fe700 1 -- 172.21.15.16:0/2906851943 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f18dc03f390 con 0x7f18ec133420 2023-09-30T22:30:55.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.065+0000 7f18f085a700 1 -- 172.21.15.16:0/2906851943 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18ec074ea0 con 0x7f18ec133420 2023-09-30T22:30:55.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.070+0000 7f18e37fe700 1 -- 172.21.15.16:0/2906851943 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f18dc007720 con 0x7f18ec133420 2023-09-30T22:30:55.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.098+0000 7f53f9a77700 1 Processor -- start 2023-09-30T22:30:55.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.098+0000 7f53f9a77700 1 -- start start 2023-09-30T22:30:55.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.098+0000 7fe0df195700 1 Processor -- start 2023-09-30T22:30:55.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.099+0000 7fe0df195700 1 -- start start 2023-09-30T22:30:55.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.099+0000 7f53f9a77700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f53f409f410 0x7f53f4136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:55.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.099+0000 7f53f9a77700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f53f4136fe0 0x7f53f41393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:55.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.099+0000 7f53f9a77700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53f4070940 con 0x7f53f409e930 2023-09-30T22:30:55.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.099+0000 7f53f9a77700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53f406fd60 con 0x7f53f4136fe0 2023-09-30T22:30:55.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.099+0000 7f53f9a77700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f53f406fa80 con 0x7f53f409f410 2023-09-30T22:30:55.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.099+0000 7fe0df195700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe0d809d6f0 0x7fe0d809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:55.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.099+0000 7fe0df195700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe0d809e000 0x7fe0d8130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:55.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.099+0000 7fe0df195700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe0d809e410 0x7fe0d8130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:55.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.099+0000 7fe0df195700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0d809f520 con 0x7fe0d809d6f0 2023-09-30T22:30:55.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.099+0000 7fe0df195700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d809f130 con 0x7fe0d809e410 2023-09-30T22:30:55.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.099+0000 7fe0df195700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d809e930 con 0x7fe0d809e000 2023-09-30T22:30:55.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.100+0000 7fe0d7fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe0d809e000 0x7fe0d8130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:55.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.100+0000 7f53f2ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f53f409f410 0x7f53f4136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:55.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.100+0000 7f53f2ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f53f409f410 0x7f53f4136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35640/0 (socket says 172.21.15.16:35640) 2023-09-30T22:30:55.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.100+0000 7f53f2ffd700 1 -- 172.21.15.16:0/3325007879 learned_addr learned my addr 172.21.15.16:0/3325007879 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:55.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.100+0000 7fe0d7fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe0d809e000 0x7fe0d8130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35642/0 (socket says 172.21.15.16:35642) 2023-09-30T22:30:55.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.100+0000 7fe0d7fff700 1 -- 172.21.15.16:0/2965843731 learned_addr learned my addr 172.21.15.16:0/2965843731 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:55.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.100+0000 7fe0dd732700 1 --2- 172.21.15.16:0/2965843731 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe0d809e410 0x7fe0d8130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:55.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.100+0000 7fe0dcf31700 1 --2- 172.21.15.16:0/2965843731 >> v2:172.21.15.16:3300/0 conn(0x7fe0d809d6f0 0x7fe0d809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:55.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.100+0000 7f53f3fff700 1 --2- 172.21.15.16:0/3325007879 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f53f4136fe0 0x7f53f41393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:55.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.100+0000 7f53f2ffd700 1 -- 172.21.15.16:0/3325007879 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f53f4136fe0 msgr2=0x7f53f41393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:55.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.100+0000 7f53f2ffd700 1 --2- 172.21.15.16:0/3325007879 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f53f4136fe0 0x7f53f41393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.100+0000 7f53f2ffd700 1 -- 172.21.15.16:0/3325007879 >> v1:172.21.15.16:6789/0 conn(0x7f53f409e930 legacy=0x7f53f409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:55.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.100+0000 7f53f2ffd700 1 -- 172.21.15.16:0/3325007879 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53f409fb60 con 0x7f53f409f410 2023-09-30T22:30:55.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.100+0000 7fe0d7fff700 1 -- 172.21.15.16:0/2965843731 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe0d809e410 msgr2=0x7fe0d8130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:55.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.100+0000 7fe0d7fff700 1 --2- 172.21.15.16:0/2965843731 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe0d809e410 0x7fe0d8130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.100+0000 7fe0d7fff700 1 -- 172.21.15.16:0/2965843731 >> v2:172.21.15.16:3300/0 conn(0x7fe0d809d6f0 msgr2=0x7fe0d809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:55.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.100+0000 7fe0d7fff700 1 --2- 172.21.15.16:0/2965843731 >> v2:172.21.15.16:3300/0 conn(0x7fe0d809d6f0 0x7fe0d809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.100+0000 7fe0d7fff700 1 -- 172.21.15.16:0/2965843731 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0d809b2d0 con 0x7fe0d809e000 2023-09-30T22:30:55.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.100+0000 7fe0dcf31700 1 --2- 172.21.15.16:0/2965843731 >> v2:172.21.15.16:3300/0 conn(0x7fe0d809d6f0 0x7fe0d809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:55.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.100+0000 7f53f2ffd700 1 --2- 172.21.15.16:0/3325007879 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f53f409f410 0x7f53f4136aa0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f53e8009920 tx=0x7f53e8009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba0c79d7a9f78c78 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:55.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.101+0000 7fe0d7fff700 1 --2- 172.21.15.16:0/2965843731 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe0d809e000 0x7fe0d8130660 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe0c4009920 tx=0x7fe0c4009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26c83ce7ad0cbc7f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:55.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.101+0000 7f53f27fc700 1 -- 172.21.15.16:0/3325007879 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e800ba60 con 0x7f53f409f410 2023-09-30T22:30:55.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.101+0000 7f53f27fc700 1 -- 172.21.15.16:0/3325007879 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53e800bbc0 con 0x7f53f409f410 2023-09-30T22:30:55.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.101+0000 7f53f27fc700 1 -- 172.21.15.16:0/3325007879 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e8009e70 con 0x7f53f409f410 2023-09-30T22:30:55.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.101+0000 7fe0d77fe700 1 -- 172.21.15.16:0/2965843731 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c4011830 con 0x7fe0d809e000 2023-09-30T22:30:55.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.101+0000 7fe0d77fe700 1 -- 172.21.15.16:0/2965843731 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0c4011990 con 0x7fe0d809e000 2023-09-30T22:30:55.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.101+0000 7fe0d77fe700 1 -- 172.21.15.16:0/2965843731 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c4011d20 con 0x7fe0d809e000 2023-09-30T22:30:55.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.102+0000 7fe0df195700 1 -- 172.21.15.16:0/2965843731 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe0d809e000 msgr2=0x7fe0d8130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:55.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.102+0000 7fe0df195700 1 --2- 172.21.15.16:0/2965843731 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe0d809e000 0x7fe0d8130660 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe0c4009920 tx=0x7fe0c4009c30 comp rx=0 tx=0).stop 2023-09-30T22:30:55.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.102+0000 7fe0df195700 1 -- 172.21.15.16:0/2965843731 shutdown_connections 2023-09-30T22:30:55.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.102+0000 7fe0df195700 1 --2- 172.21.15.16:0/2965843731 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe0d809e410 0x7fe0d8130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.102+0000 7fe0df195700 1 --2- 172.21.15.16:0/2965843731 >> v2:172.21.15.16:3300/0 conn(0x7fe0d809d6f0 0x7fe0d809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.102+0000 7fe0df195700 1 --2- 172.21.15.16:0/2965843731 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe0d809e000 0x7fe0d8130660 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.102+0000 7fe0df195700 1 -- 172.21.15.16:0/2965843731 >> 172.21.15.16:0/2965843731 conn(0x7fe0d8066f50 msgr2=0x7fe0d809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:55.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.102+0000 7fe0df195700 1 -- 172.21.15.16:0/2965843731 shutdown_connections 2023-09-30T22:30:55.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.102+0000 7f53f9a77700 1 -- 172.21.15.16:0/3325007879 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f53f409f410 msgr2=0x7f53f4136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:55.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.102+0000 7f53f9a77700 1 --2- 172.21.15.16:0/3325007879 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f53f409f410 0x7f53f4136aa0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f53e8009920 tx=0x7f53e8009c30 comp rx=0 tx=0).stop 2023-09-30T22:30:55.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.102+0000 7fe0df195700 1 -- 172.21.15.16:0/2965843731 wait complete. 2023-09-30T22:30:55.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.102+0000 7f53f9a77700 1 -- 172.21.15.16:0/3325007879 shutdown_connections 2023-09-30T22:30:55.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.102+0000 7f53f9a77700 1 --2- 172.21.15.16:0/3325007879 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f53f4136fe0 0x7f53f41393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.102+0000 7f53f9a77700 1 --2- 172.21.15.16:0/3325007879 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f53f409f410 0x7f53f4136aa0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.102+0000 7f53f9a77700 1 -- 172.21.15.16:0/3325007879 >> 172.21.15.16:0/3325007879 conn(0x7f53f40a4210 msgr2=0x7f53f40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:55.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.102+0000 7f53f9a77700 1 -- 172.21.15.16:0/3325007879 shutdown_connections 2023-09-30T22:30:55.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.102+0000 7f53f9a77700 1 -- 172.21.15.16:0/3325007879 wait complete. 2023-09-30T22:30:55.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.103+0000 7fe0df195700 1 Processor -- start 2023-09-30T22:30:55.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.103+0000 7f53f9a77700 1 Processor -- start 2023-09-30T22:30:55.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.103+0000 7f53f9a77700 1 -- start start 2023-09-30T22:30:55.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.103+0000 7fe0df195700 1 -- start start 2023-09-30T22:30:55.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.104+0000 7fe0df195700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0d809d6f0 0x7fe0d811d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:55.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.104+0000 7fe0df195700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe0d809e000 0x7fe0d811ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:55.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.104+0000 7fe0df195700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe0d809e410 0x7fe0d811e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:55.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.104+0000 7fe0df195700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d811eee0 con 0x7fe0d809d6f0 2023-09-30T22:30:55.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.104+0000 7fe0df195700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d811ebf0 con 0x7fe0d809e000 2023-09-30T22:30:55.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.104+0000 7fe0df195700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d809c700 con 0x7fe0d809e410 2023-09-30T22:30:55.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.104+0000 7fe0dd732700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe0d809e410 0x7fe0d811e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:55.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.104+0000 7fe0dd732700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe0d809e410 0x7fe0d811e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35644/0 (socket says 172.21.15.16:35644) 2023-09-30T22:30:55.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.104+0000 7fe0dd732700 1 -- 172.21.15.16:0/2110859176 learned_addr learned my addr 172.21.15.16:0/2110859176 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:55.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.104+0000 7fe0dcf31700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0d809d6f0 0x7fe0d811d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:55.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.104+0000 7f53f9a77700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f53f409e930 0x7f53f41c87a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:55.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.104+0000 7f53f9a77700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53f409f410 0x7f53f41c8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:55.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.104+0000 7f53f9a77700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f53f4136fe0 0x7f53f4121ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:55.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.104+0000 7f53f9a77700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f4123670 con 0x7f53f409f410 2023-09-30T22:30:55.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.104+0000 7f53f9a77700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53f4123f40 con 0x7f53f409e930 2023-09-30T22:30:55.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.104+0000 7f53f9a77700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f53f41c9220 con 0x7f53f4136fe0 2023-09-30T22:30:55.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.104+0000 7f53f2ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53f409f410 0x7f53f41c8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:55.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.104+0000 7f53f2ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53f409f410 0x7f53f41c8ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44004/0 (socket says 172.21.15.16:44004) 2023-09-30T22:30:55.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.104+0000 7f53f2ffd700 1 -- 172.21.15.16:0/3456986761 learned_addr learned my addr 172.21.15.16:0/3456986761 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:55.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.104+0000 7f53f3fff700 1 --2- 172.21.15.16:0/3456986761 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f53f4136fe0 0x7f53f4121ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:55.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.105+0000 7f53f37fe700 1 --2- 172.21.15.16:0/3456986761 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f53f409e930 0x7f53f41c87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:55.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.105+0000 7f53f2ffd700 1 -- 172.21.15.16:0/3456986761 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f53f4136fe0 msgr2=0x7f53f4121ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:55.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.105+0000 7f53f2ffd700 1 --2- 172.21.15.16:0/3456986761 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f53f4136fe0 0x7f53f4121ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.105+0000 7f53f2ffd700 1 -- 172.21.15.16:0/3456986761 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f53f409e930 msgr2=0x7f53f41c87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:55.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.105+0000 7f53f2ffd700 1 --2- 172.21.15.16:0/3456986761 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f53f409e930 0x7f53f41c87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.105+0000 7f53f2ffd700 1 -- 172.21.15.16:0/3456986761 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e8009550 con 0x7f53f409f410 2023-09-30T22:30:55.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.105+0000 7f53f37fe700 1 --2- 172.21.15.16:0/3456986761 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f53f409e930 0x7f53f41c87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:30:55.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.104+0000 7fe0dcf31700 1 -- 172.21.15.16:0/2110859176 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe0d809e410 msgr2=0x7fe0d811e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:55.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.104+0000 7fe0dcf31700 1 --2- 172.21.15.16:0/2110859176 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe0d809e410 0x7fe0d811e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.104+0000 7fe0dcf31700 1 -- 172.21.15.16:0/2110859176 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe0d809e000 msgr2=0x7fe0d811ddd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:30:55.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.104+0000 7fe0dcf31700 1 --2- 172.21.15.16:0/2110859176 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe0d809e000 0x7fe0d811ddd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.104+0000 7fe0dcf31700 1 -- 172.21.15.16:0/2110859176 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0cc0094a0 con 0x7fe0d809d6f0 2023-09-30T22:30:55.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.105+0000 7fe0dd732700 1 --2- 172.21.15.16:0/2110859176 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe0d809e410 0x7fe0d811e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:55.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.105+0000 7f53f2ffd700 1 --2- 172.21.15.16:0/3456986761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53f409f410 0x7f53f41c8ce0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f53e8009c70 tx=0x7f53e8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:55.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.105+0000 7fe0dcf31700 1 --2- 172.21.15.16:0/2110859176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0d809d6f0 0x7fe0d811d890 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc00e980 tx=0x7fe0cc00ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:55.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.105+0000 7fe0d57fa700 1 -- 172.21.15.16:0/2110859176 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0cc00eed0 con 0x7fe0d809d6f0 2023-09-30T22:30:55.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.105+0000 7fe0df195700 1 -- 172.21.15.16:0/2110859176 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0c4009550 con 0x7fe0d809d6f0 2023-09-30T22:30:55.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.105+0000 7f53dbfff700 1 -- 172.21.15.16:0/3456986761 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e8004510 con 0x7f53f409f410 2023-09-30T22:30:55.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.105+0000 7fe0df195700 1 -- 172.21.15.16:0/2110859176 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0d8120110 con 0x7fe0d809d6f0 2023-09-30T22:30:55.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.105+0000 7f53dbfff700 1 -- 172.21.15.16:0/3456986761 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53e80036a0 con 0x7f53f409f410 2023-09-30T22:30:55.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.105+0000 7f53f9a77700 1 -- 172.21.15.16:0/3456986761 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53f40a2f40 con 0x7f53f409f410 2023-09-30T22:30:55.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.105+0000 7f53f9a77700 1 -- 172.21.15.16:0/3456986761 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53f4071d70 con 0x7f53f409f410 2023-09-30T22:30:55.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.106+0000 7fe0d57fa700 1 -- 172.21.15.16:0/2110859176 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0cc005a20 con 0x7fe0d809d6f0 2023-09-30T22:30:55.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.106+0000 7fe0d57fa700 1 -- 172.21.15.16:0/2110859176 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0cc005d90 con 0x7fe0d809d6f0 2023-09-30T22:30:55.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.106+0000 7f53dbfff700 1 -- 172.21.15.16:0/3456986761 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e8003a50 con 0x7f53f409f410 2023-09-30T22:30:55.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.106+0000 7fe0d57fa700 1 -- 172.21.15.16:0/2110859176 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe0cc004030 con 0x7fe0d809d6f0 2023-09-30T22:30:55.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.106+0000 7fe0d57fa700 1 -- 172.21.15.16:0/2110859176 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fe0cc03f5e0 con 0x7fe0d809d6f0 2023-09-30T22:30:55.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.107+0000 7f53dbfff700 1 -- 172.21.15.16:0/3456986761 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f53e8003bb0 con 0x7f53f409f410 2023-09-30T22:30:55.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.107+0000 7f53f9a77700 1 -- 172.21.15.16:0/3456986761 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53d40052e0 con 0x7f53f409f410 2023-09-30T22:30:55.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.107+0000 7fe0df195700 1 -- 172.21.15.16:0/2110859176 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0b80052e0 con 0x7fe0d809d6f0 2023-09-30T22:30:55.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.107+0000 7f53dbfff700 1 -- 172.21.15.16:0/3456986761 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f53e8040b80 con 0x7f53f409f410 2023-09-30T22:30:55.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.112+0000 7fe0d57fa700 1 -- 172.21.15.16:0/2110859176 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe0cc020020 con 0x7fe0d809d6f0 2023-09-30T22:30:55.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.116+0000 7f53dbfff700 1 -- 172.21.15.16:0/3456986761 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f53e8015080 con 0x7f53f409f410 2023-09-30T22:30:55.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.282+0000 7f18f085a700 1 -- 172.21.15.16:0/2906851943 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f18ec06b250 con 0x7f18ec133420 2023-09-30T22:30:55.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.282+0000 7f18e37fe700 1 -- 172.21.15.16:0/2906851943 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f18dc011360 con 0x7f18ec133420 2023-09-30T22:30:55.283 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:30:55.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.285+0000 7f18f085a700 1 -- 172.21.15.16:0/2906851943 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f18ec133420 msgr2=0x7f18ec12ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:55.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.285+0000 7f18f085a700 1 --2- 172.21.15.16:0/2906851943 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f18ec133420 0x7f18ec12ee50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f18dc00c470 tx=0x7f18dc00c780 comp rx=0 tx=0).stop 2023-09-30T22:30:55.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.285+0000 7f18f085a700 1 -- 172.21.15.16:0/2906851943 shutdown_connections 2023-09-30T22:30:55.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.285+0000 7f18f085a700 1 --2- 172.21.15.16:0/2906851943 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f18ec138dc0 0x7f18ec12f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.285+0000 7f18f085a700 1 --2- 172.21.15.16:0/2906851943 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f18ec133420 0x7f18ec12ee50 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.285+0000 7f18f085a700 1 --2- 172.21.15.16:0/2906851943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18ec1337f0 0x7f18ec12f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.285+0000 7f18f085a700 1 -- 172.21.15.16:0/2906851943 >> 172.21.15.16:0/2906851943 conn(0x7f18ec0a4210 msgr2=0x7f18ec0a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:55.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.285+0000 7f18f085a700 1 -- 172.21.15.16:0/2906851943 shutdown_connections 2023-09-30T22:30:55.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.285+0000 7f18f085a700 1 -- 172.21.15.16:0/2906851943 wait complete. 2023-09-30T22:30:55.300 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:30:55.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.317+0000 7fe0df195700 1 -- 172.21.15.16:0/2110859176 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe0b8005590 con 0x7fe0d809d6f0 2023-09-30T22:30:55.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.317+0000 7fe0d57fa700 1 -- 172.21.15.16:0/2110859176 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe0cc00c7f0 con 0x7fe0d809d6f0 2023-09-30T22:30:55.320 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:30:55.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.320+0000 7fe0df195700 1 -- 172.21.15.16:0/2110859176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0d809d6f0 msgr2=0x7fe0d811d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:55.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.320+0000 7fe0df195700 1 --2- 172.21.15.16:0/2110859176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0d809d6f0 0x7fe0d811d890 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc00e980 tx=0x7fe0cc00ec90 comp rx=0 tx=0).stop 2023-09-30T22:30:55.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.320+0000 7fe0df195700 1 -- 172.21.15.16:0/2110859176 shutdown_connections 2023-09-30T22:30:55.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.320+0000 7fe0df195700 1 --2- 172.21.15.16:0/2110859176 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe0d809e410 0x7fe0d811e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.320+0000 7fe0df195700 1 --2- 172.21.15.16:0/2110859176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0d809d6f0 0x7fe0d811d890 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.320+0000 7fe0df195700 1 --2- 172.21.15.16:0/2110859176 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe0d809e000 0x7fe0d811ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.320+0000 7fe0df195700 1 -- 172.21.15.16:0/2110859176 >> 172.21.15.16:0/2110859176 conn(0x7fe0d8066f50 msgr2=0x7fe0d81301a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:55.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.320+0000 7fe0df195700 1 -- 172.21.15.16:0/2110859176 shutdown_connections 2023-09-30T22:30:55.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.320+0000 7fe0df195700 1 -- 172.21.15.16:0/2110859176 wait complete. 2023-09-30T22:30:55.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.323+0000 7f53f9a77700 1 -- 172.21.15.16:0/3456986761 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f53d4005590 con 0x7f53f409f410 2023-09-30T22:30:55.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.323+0000 7f53dbfff700 1 -- 172.21.15.16:0/3456986761 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f53e8010380 con 0x7f53f409f410 2023-09-30T22:30:55.324 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:30:55.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.326+0000 7f53f9a77700 1 -- 172.21.15.16:0/3456986761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53f409f410 msgr2=0x7f53f41c8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:55.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.326+0000 7f53f9a77700 1 --2- 172.21.15.16:0/3456986761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53f409f410 0x7f53f41c8ce0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f53e8009c70 tx=0x7f53e8004750 comp rx=0 tx=0).stop 2023-09-30T22:30:55.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.326+0000 7f53f9a77700 1 -- 172.21.15.16:0/3456986761 shutdown_connections 2023-09-30T22:30:55.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.326+0000 7f53f9a77700 1 --2- 172.21.15.16:0/3456986761 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f53f409e930 0x7f53f41c87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.326+0000 7f53f9a77700 1 --2- 172.21.15.16:0/3456986761 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f53f4136fe0 0x7f53f4121ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.326+0000 7f53f9a77700 1 --2- 172.21.15.16:0/3456986761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53f409f410 0x7f53f41c8ce0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.326+0000 7f53f9a77700 1 -- 172.21.15.16:0/3456986761 >> 172.21.15.16:0/3456986761 conn(0x7f53f40a4210 msgr2=0x7f53f40a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:55.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.326+0000 7f53f9a77700 1 -- 172.21.15.16:0/3456986761 shutdown_connections 2023-09-30T22:30:55.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.326+0000 7f53f9a77700 1 -- 172.21.15.16:0/3456986761 wait complete. 2023-09-30T22:30:55.336 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:30:55.341 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:30:55.353 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:55.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.522+0000 7f1a0d09a700 1 Processor -- start 2023-09-30T22:30:55.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.522+0000 7f1a0d09a700 1 -- start start 2023-09-30T22:30:55.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.523+0000 7f1a0d09a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1a0809d6f0 0x7f1a0809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:55.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.523+0000 7f1a0d09a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1a0809e000 0x7f1a08130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:55.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.523+0000 7f1a0d09a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1a0809e410 0x7f1a08130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:55.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.523+0000 7f1a0d09a700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a0809f520 con 0x7f1a0809e410 2023-09-30T22:30:55.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.523+0000 7f1a0d09a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0809f130 con 0x7f1a0809d6f0 2023-09-30T22:30:55.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.523+0000 7f1a0d09a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0809e930 con 0x7f1a0809e000 2023-09-30T22:30:55.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.523+0000 7f1a06d9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1a0809d6f0 0x7f1a0809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:55.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.523+0000 7f1a0659c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1a0809e000 0x7f1a08130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:55.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.523+0000 7f1a06d9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1a0809d6f0 0x7f1a0809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41222/0 (socket says 172.21.15.16:41222) 2023-09-30T22:30:55.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.523+0000 7f1a0659c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1a0809e000 0x7f1a08130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35658/0 (socket says 172.21.15.16:35658) 2023-09-30T22:30:55.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.523+0000 7f1a06d9d700 1 -- 172.21.15.16:0/4140637050 learned_addr learned my addr 172.21.15.16:0/4140637050 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:55.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.523+0000 7f1a0759e700 1 --2- 172.21.15.16:0/4140637050 >> v2:172.21.15.16:3300/0 conn(0x7f1a0809e410 0x7f1a08130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:55.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.523+0000 7f1a06d9d700 1 -- 172.21.15.16:0/4140637050 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1a0809e000 msgr2=0x7f1a08130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:55.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.523+0000 7f1a06d9d700 1 --2- 172.21.15.16:0/4140637050 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1a0809e000 0x7f1a08130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.523+0000 7f1a06d9d700 1 -- 172.21.15.16:0/4140637050 >> v2:172.21.15.16:3300/0 conn(0x7f1a0809e410 msgr2=0x7f1a08130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:55.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.523+0000 7f1a06d9d700 1 --2- 172.21.15.16:0/4140637050 >> v2:172.21.15.16:3300/0 conn(0x7f1a0809e410 0x7f1a08130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.523+0000 7f1a06d9d700 1 -- 172.21.15.16:0/4140637050 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a0809b2d0 con 0x7f1a0809d6f0 2023-09-30T22:30:55.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.523+0000 7f1a0659c700 1 --2- 172.21.15.16:0/4140637050 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1a0809e000 0x7f1a08130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:55.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.523+0000 7f1a06d9d700 1 --2- 172.21.15.16:0/4140637050 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1a0809d6f0 0x7f1a0809dac0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1a00009e80 tx=0x7f1a0000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a8d9e81a50dbb10 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:55.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.524+0000 7f1a05d9b700 1 -- 172.21.15.16:0/4140637050 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0000e070 con 0x7f1a0809d6f0 2023-09-30T22:30:55.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.524+0000 7f1a05d9b700 1 -- 172.21.15.16:0/4140637050 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a0000b8d0 con 0x7f1a0809d6f0 2023-09-30T22:30:55.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.524+0000 7f1a05d9b700 1 -- 172.21.15.16:0/4140637050 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a0000bc80 con 0x7f1a0809d6f0 2023-09-30T22:30:55.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.524+0000 7f1a0d09a700 1 -- 172.21.15.16:0/4140637050 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1a0809d6f0 msgr2=0x7f1a0809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:55.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.524+0000 7f1a0d09a700 1 --2- 172.21.15.16:0/4140637050 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1a0809d6f0 0x7f1a0809dac0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f1a00009e80 tx=0x7f1a0000b330 comp rx=0 tx=0).stop 2023-09-30T22:30:55.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.524+0000 7f1a0d09a700 1 -- 172.21.15.16:0/4140637050 shutdown_connections 2023-09-30T22:30:55.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.524+0000 7f1a0d09a700 1 --2- 172.21.15.16:0/4140637050 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1a0809d6f0 0x7f1a0809dac0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.524+0000 7f1a0d09a700 1 --2- 172.21.15.16:0/4140637050 >> v2:172.21.15.16:3300/0 conn(0x7f1a0809e410 0x7f1a08130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.524+0000 7f1a0d09a700 1 --2- 172.21.15.16:0/4140637050 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1a0809e000 0x7f1a08130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.524+0000 7f1a0d09a700 1 -- 172.21.15.16:0/4140637050 >> 172.21.15.16:0/4140637050 conn(0x7f1a08066f50 msgr2=0x7f1a0809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:55.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.524+0000 7f1a0d09a700 1 -- 172.21.15.16:0/4140637050 shutdown_connections 2023-09-30T22:30:55.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.524+0000 7f1a0d09a700 1 -- 172.21.15.16:0/4140637050 wait complete. 2023-09-30T22:30:55.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.526+0000 7f1a0d09a700 1 Processor -- start 2023-09-30T22:30:55.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.526+0000 7f1a0d09a700 1 -- start start 2023-09-30T22:30:55.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.526+0000 7f1a0d09a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1a0809d6f0 0x7f1a08122640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:55.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.526+0000 7f1a0d09a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a0809e000 0x7f1a08122b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:55.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.526+0000 7f1a0d09a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1a0809e410 0x7f1a081230c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:55.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.526+0000 7f1a0d09a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a08125f60 con 0x7f1a0809e000 2023-09-30T22:30:55.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.526+0000 7f1a0d09a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a08124250 con 0x7f1a0809e410 2023-09-30T22:30:55.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.526+0000 7f1a0d09a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0809c700 con 0x7f1a0809d6f0 2023-09-30T22:30:55.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.526+0000 7f1a06d9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1a0809d6f0 0x7f1a08122640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:55.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.526+0000 7f1a0659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a0809e000 0x7f1a08122b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:55.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.526+0000 7f1a06d9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1a0809d6f0 0x7f1a08122640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35662/0 (socket says 172.21.15.16:35662) 2023-09-30T22:30:55.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.526+0000 7f1a0659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a0809e000 0x7f1a08122b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44026/0 (socket says 172.21.15.16:44026) 2023-09-30T22:30:55.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.526+0000 7f1a06d9d700 1 -- 172.21.15.16:0/4279582940 learned_addr learned my addr 172.21.15.16:0/4279582940 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:55.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.526+0000 7f1a0759e700 1 --2- 172.21.15.16:0/4279582940 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1a0809e410 0x7f1a081230c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:55.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.527+0000 7f1a06d9d700 1 -- 172.21.15.16:0/4279582940 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1a0809e410 msgr2=0x7f1a081230c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:55.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.527+0000 7f1a06d9d700 1 --2- 172.21.15.16:0/4279582940 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1a0809e410 0x7f1a081230c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.527+0000 7f1a06d9d700 1 -- 172.21.15.16:0/4279582940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a0809e000 msgr2=0x7f1a08122b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:55.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.527+0000 7f1a06d9d700 1 --2- 172.21.15.16:0/4279582940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a0809e000 0x7f1a08122b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.527+0000 7f1a06d9d700 1 -- 172.21.15.16:0/4279582940 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f00094a0 con 0x7f1a0809d6f0 2023-09-30T22:30:55.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.527+0000 7f1a0659c700 1 --2- 172.21.15.16:0/4279582940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a0809e000 0x7f1a08122b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:55.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.527+0000 7f1a06d9d700 1 --2- 172.21.15.16:0/4279582940 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1a0809d6f0 0x7f1a08122640 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1a0000b370 tx=0x7f1a00004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:55.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.527+0000 7f19ff7fe700 1 -- 172.21.15.16:0/4279582940 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a00003e10 con 0x7f1a0809d6f0 2023-09-30T22:30:55.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.527+0000 7f19ff7fe700 1 -- 172.21.15.16:0/4279582940 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a00004250 con 0x7f1a0809d6f0 2023-09-30T22:30:55.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.527+0000 7f1a0d09a700 1 -- 172.21.15.16:0/4279582940 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a00009b30 con 0x7f1a0809d6f0 2023-09-30T22:30:55.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.527+0000 7f19ff7fe700 1 -- 172.21.15.16:0/4279582940 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a00004600 con 0x7f1a0809d6f0 2023-09-30T22:30:55.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.527+0000 7f1a0d09a700 1 -- 172.21.15.16:0/4279582940 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a081268f0 con 0x7f1a0809d6f0 2023-09-30T22:30:55.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.528+0000 7f19ff7fe700 1 -- 172.21.15.16:0/4279582940 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1a00004760 con 0x7f1a0809d6f0 2023-09-30T22:30:55.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.528+0000 7f19ff7fe700 1 -- 172.21.15.16:0/4279582940 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f1a00040960 con 0x7f1a0809d6f0 2023-09-30T22:30:55.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.528+0000 7f1a0d09a700 1 -- 172.21.15.16:0/4279582940 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a08074ea0 con 0x7f1a0809d6f0 2023-09-30T22:30:55.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.533+0000 7f19ff7fe700 1 -- 172.21.15.16:0/4279582940 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1a0001a720 con 0x7f1a0809d6f0 2023-09-30T22:30:55.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.738+0000 7f1a0d09a700 1 -- 172.21.15.16:0/4279582940 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1a081307a0 con 0x7f1a0809d6f0 2023-09-30T22:30:55.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.738+0000 7f19ff7fe700 1 -- 172.21.15.16:0/4279582940 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1a00033090 con 0x7f1a0809d6f0 2023-09-30T22:30:55.739 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:30:55.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.741+0000 7f1a0d09a700 1 -- 172.21.15.16:0/4279582940 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1a0809d6f0 msgr2=0x7f1a08122640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:55.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.741+0000 7f1a0d09a700 1 --2- 172.21.15.16:0/4279582940 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1a0809d6f0 0x7f1a08122640 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1a0000b370 tx=0x7f1a00004ad0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.741+0000 7f1a0d09a700 1 -- 172.21.15.16:0/4279582940 shutdown_connections 2023-09-30T22:30:55.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.741+0000 7f1a0d09a700 1 --2- 172.21.15.16:0/4279582940 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1a0809e410 0x7f1a081230c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.741+0000 7f1a0d09a700 1 --2- 172.21.15.16:0/4279582940 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1a0809d6f0 0x7f1a08122640 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.741+0000 7f1a0d09a700 1 --2- 172.21.15.16:0/4279582940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a0809e000 0x7f1a08122b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:55.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.741+0000 7f1a0d09a700 1 -- 172.21.15.16:0/4279582940 >> 172.21.15.16:0/4279582940 conn(0x7f1a08066f50 msgr2=0x7f1a080a18c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:55.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.741+0000 7f1a0d09a700 1 -- 172.21.15.16:0/4279582940 shutdown_connections 2023-09-30T22:30:55.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:55.741+0000 7f1a0d09a700 1 -- 172.21.15.16:0/4279582940 wait complete. 2023-09-30T22:30:55.755 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:30:56.300 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:56.337 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:56.342 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:56.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.451+0000 7f65e426b700 1 Processor -- start 2023-09-30T22:30:56.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.451+0000 7f65e426b700 1 -- start start 2023-09-30T22:30:56.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.452+0000 7f65e426b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f65dc09e930 0x7f65dc09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:56.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.452+0000 7f65e426b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f65dc09f240 0x7f65dc1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:56.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.452+0000 7f65e426b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f65dc09f650 0x7f65dc131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:56.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.452+0000 7f65e426b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65dc070940 con 0x7f65dc09f240 2023-09-30T22:30:56.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.452+0000 7f65e426b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65dc06fd60 con 0x7f65dc09e930 2023-09-30T22:30:56.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.452+0000 7f65e426b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f65dc06fa80 con 0x7f65dc09f650 2023-09-30T22:30:56.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.452+0000 7f65e2007700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f65dc09e930 0x7f65dc09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:56.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.452+0000 7f65e2808700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f65dc09f650 0x7f65dc131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:56.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.452+0000 7f65e2007700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f65dc09e930 0x7f65dc09ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41240/0 (socket says 172.21.15.16:41240) 2023-09-30T22:30:56.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.452+0000 7f65e2808700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f65dc09f650 0x7f65dc131d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35666/0 (socket says 172.21.15.16:35666) 2023-09-30T22:30:56.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.452+0000 7f65e2007700 1 -- 172.21.15.16:0/168669423 learned_addr learned my addr 172.21.15.16:0/168669423 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:56.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.452+0000 7f65e1806700 1 --2- 172.21.15.16:0/168669423 >> v2:172.21.15.16:3300/0 conn(0x7f65dc09f240 0x7f65dc1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:56.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.452+0000 7f65e2007700 1 -- 172.21.15.16:0/168669423 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f65dc09f650 msgr2=0x7f65dc131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:56.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.452+0000 7f65e2007700 1 --2- 172.21.15.16:0/168669423 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f65dc09f650 0x7f65dc131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:56.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.452+0000 7f65e2007700 1 -- 172.21.15.16:0/168669423 >> v2:172.21.15.16:3300/0 conn(0x7f65dc09f240 msgr2=0x7f65dc1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:56.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.452+0000 7f65e2007700 1 --2- 172.21.15.16:0/168669423 >> v2:172.21.15.16:3300/0 conn(0x7f65dc09f240 0x7f65dc1317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:56.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.452+0000 7f65e2007700 1 -- 172.21.15.16:0/168669423 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65dc09fb60 con 0x7f65dc09e930 2023-09-30T22:30:56.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.452+0000 7f65e1806700 1 --2- 172.21.15.16:0/168669423 >> v2:172.21.15.16:3300/0 conn(0x7f65dc09f240 0x7f65dc1317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:56.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.453+0000 7f65e2007700 1 --2- 172.21.15.16:0/168669423 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f65dc09e930 0x7f65dc09ed00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f65d80098f0 tx=0x7f65d8009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1cd985892636557e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:56.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.453+0000 7f65e1005700 1 -- 172.21.15.16:0/168669423 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d8011830 con 0x7f65dc09e930 2023-09-30T22:30:56.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.453+0000 7f65e1005700 1 -- 172.21.15.16:0/168669423 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65d8011990 con 0x7f65dc09e930 2023-09-30T22:30:56.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.453+0000 7f65e1005700 1 -- 172.21.15.16:0/168669423 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d8011d20 con 0x7f65dc09e930 2023-09-30T22:30:56.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.453+0000 7f65e426b700 1 -- 172.21.15.16:0/168669423 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f65dc09e930 msgr2=0x7f65dc09ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:56.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.453+0000 7f65e426b700 1 --2- 172.21.15.16:0/168669423 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f65dc09e930 0x7f65dc09ed00 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f65d80098f0 tx=0x7f65d8009c00 comp rx=0 tx=0).stop 2023-09-30T22:30:56.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.454+0000 7f65e426b700 1 -- 172.21.15.16:0/168669423 shutdown_connections 2023-09-30T22:30:56.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.454+0000 7f65e426b700 1 --2- 172.21.15.16:0/168669423 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f65dc09e930 0x7f65dc09ed00 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:56.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.454+0000 7f65e426b700 1 --2- 172.21.15.16:0/168669423 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f65dc09f650 0x7f65dc131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:56.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.454+0000 7f65e426b700 1 --2- 172.21.15.16:0/168669423 >> v2:172.21.15.16:3300/0 conn(0x7f65dc09f240 0x7f65dc1317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:56.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.454+0000 7f65e426b700 1 -- 172.21.15.16:0/168669423 >> 172.21.15.16:0/168669423 conn(0x7f65dc0a4210 msgr2=0x7f65dc0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:56.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.454+0000 7f65e426b700 1 -- 172.21.15.16:0/168669423 shutdown_connections 2023-09-30T22:30:56.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.454+0000 7f65e426b700 1 -- 172.21.15.16:0/168669423 wait complete. 2023-09-30T22:30:56.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.455+0000 7f65e426b700 1 Processor -- start 2023-09-30T22:30:56.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.455+0000 7f65e426b700 1 -- start start 2023-09-30T22:30:56.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.455+0000 7f65e426b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f65dc09e930 0x7f65dc06b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:56.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.455+0000 7f65e426b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65dc09f240 0x7f65dc06bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:56.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.455+0000 7f65e426b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f65dc09f650 0x7f65dc06c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:56.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.455+0000 7f65e426b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65dc1a9a10 con 0x7f65dc09f240 2023-09-30T22:30:56.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.455+0000 7f65e426b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65dc1a89b0 con 0x7f65dc09e930 2023-09-30T22:30:56.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.455+0000 7f65e426b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f65dc0a35c0 con 0x7f65dc09f650 2023-09-30T22:30:56.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.455+0000 7f65e2007700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f65dc09e930 0x7f65dc06b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:56.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.455+0000 7f65e2007700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f65dc09e930 0x7f65dc06b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41252/0 (socket says 172.21.15.16:41252) 2023-09-30T22:30:56.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.455+0000 7f65e2007700 1 -- 172.21.15.16:0/427681482 learned_addr learned my addr 172.21.15.16:0/427681482 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:56.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.456+0000 7f65e1806700 1 --2- 172.21.15.16:0/427681482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65dc09f240 0x7f65dc06bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:56.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.456+0000 7f65e2808700 1 --2- 172.21.15.16:0/427681482 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f65dc09f650 0x7f65dc06c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:56.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.456+0000 7f65e2007700 1 -- 172.21.15.16:0/427681482 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f65dc09f650 msgr2=0x7f65dc06c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:56.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.456+0000 7f65e2007700 1 --2- 172.21.15.16:0/427681482 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f65dc09f650 0x7f65dc06c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:56.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.456+0000 7f65e2007700 1 -- 172.21.15.16:0/427681482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65dc09f240 msgr2=0x7f65dc06bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:56.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.456+0000 7f65e2007700 1 --2- 172.21.15.16:0/427681482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65dc09f240 0x7f65dc06bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:56.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.456+0000 7f65e2007700 1 -- 172.21.15.16:0/427681482 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65cc0094a0 con 0x7f65dc09e930 2023-09-30T22:30:56.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.456+0000 7f65e2007700 1 --2- 172.21.15.16:0/427681482 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f65dc09e930 0x7f65dc06b880 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f65d8000c00 tx=0x7f65d8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:56.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.456+0000 7f65d2ffd700 1 -- 172.21.15.16:0/427681482 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d8004240 con 0x7f65dc09e930 2023-09-30T22:30:56.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.456+0000 7f65d2ffd700 1 -- 172.21.15.16:0/427681482 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65d801f070 con 0x7f65dc09e930 2023-09-30T22:30:56.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.456+0000 7f65d2ffd700 1 -- 172.21.15.16:0/427681482 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65d80045e0 con 0x7f65dc09e930 2023-09-30T22:30:56.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.456+0000 7f65e426b700 1 -- 172.21.15.16:0/427681482 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65d8009550 con 0x7f65dc09e930 2023-09-30T22:30:56.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.456+0000 7f65e426b700 1 -- 172.21.15.16:0/427681482 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65dc0a0190 con 0x7f65dc09e930 2023-09-30T22:30:56.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.457+0000 7f65d2ffd700 1 -- 172.21.15.16:0/427681482 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f65d8021030 con 0x7f65dc09e930 2023-09-30T22:30:56.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.457+0000 7f65d2ffd700 1 -- 172.21.15.16:0/427681482 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f65d8040930 con 0x7f65dc09e930 2023-09-30T22:30:56.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.457+0000 7f65e426b700 1 -- 172.21.15.16:0/427681482 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65dc1a0b30 con 0x7f65dc09e930 2023-09-30T22:30:56.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.467+0000 7f65d2ffd700 1 -- 172.21.15.16:0/427681482 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f65d8015030 con 0x7f65dc09e930 2023-09-30T22:30:56.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.501+0000 7f750ca33700 1 Processor -- start 2023-09-30T22:30:56.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.501+0000 7f750ca33700 1 -- start start 2023-09-30T22:30:56.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.502+0000 7f750ca33700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f750809d6f0 0x7f750809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:56.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.502+0000 7f750ca33700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f750809e410 0x7f7508130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:56.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.502+0000 7f750ca33700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f750809f130 con 0x7f750809e000 2023-09-30T22:30:56.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.502+0000 7f750ca33700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f750809f520 con 0x7f750809e410 2023-09-30T22:30:56.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.502+0000 7f750ca33700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f750809e930 con 0x7f750809d6f0 2023-09-30T22:30:56.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.502+0000 7f7506d9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f750809e410 0x7f7508130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:56.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.502+0000 7f7505d9b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f750809e000 0x7f7508130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40926/0 (socket says 172.21.15.16:40926) 2023-09-30T22:30:56.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.502+0000 7f7505d9b700 1 -- 172.21.15.16:0/3262370358 learned_addr learned my addr 172.21.15.16:0/3262370358 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:30:56.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.502+0000 7f750659c700 1 --2- 172.21.15.16:0/3262370358 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f750809d6f0 0x7f750809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:56.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.503+0000 7f750559a700 1 -- 172.21.15.16:0/3262370358 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 325130374 0 0) 0x7f750809f130 con 0x7f750809e000 2023-09-30T22:30:56.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.503+0000 7f7506d9d700 1 -- 172.21.15.16:0/3262370358 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f750809d6f0 msgr2=0x7f750809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:56.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.503+0000 7f7506d9d700 1 --2- 172.21.15.16:0/3262370358 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f750809d6f0 0x7f750809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:56.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.503+0000 7f7506d9d700 1 -- 172.21.15.16:0/3262370358 >> v1:172.21.15.16:6789/0 conn(0x7f750809e000 legacy=0x7f7508130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:56.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.503+0000 7f7506d9d700 1 -- 172.21.15.16:0/3262370358 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f750809b2d0 con 0x7f750809e410 2023-09-30T22:30:56.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.503+0000 7f7506d9d700 1 --2- 172.21.15.16:0/3262370358 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f750809e410 0x7f7508130d70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7500009dc0 tx=0x7f750000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ebca5640b2aae25 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:56.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.503+0000 7f750559a700 1 -- 172.21.15.16:0/3262370358 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750000e070 con 0x7f750809e410 2023-09-30T22:30:56.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.503+0000 7f750559a700 1 -- 172.21.15.16:0/3262370358 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f750000bb20 con 0x7f750809e410 2023-09-30T22:30:56.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.503+0000 7f750559a700 1 -- 172.21.15.16:0/3262370358 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7500012430 con 0x7f750809e410 2023-09-30T22:30:56.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.504+0000 7f25ce613700 1 Processor -- start 2023-09-30T22:30:56.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.504+0000 7f25ce613700 1 -- start start 2023-09-30T22:30:56.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.504+0000 7f750ca33700 1 -- 172.21.15.16:0/3262370358 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f750809e410 msgr2=0x7f7508130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:56.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.504+0000 7f750ca33700 1 --2- 172.21.15.16:0/3262370358 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f750809e410 0x7f7508130d70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7500009dc0 tx=0x7f750000b330 comp rx=0 tx=0).stop 2023-09-30T22:30:56.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.504+0000 7f750ca33700 1 -- 172.21.15.16:0/3262370358 shutdown_connections 2023-09-30T22:30:56.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.504+0000 7f750ca33700 1 --2- 172.21.15.16:0/3262370358 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f750809e410 0x7f7508130d70 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:56.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.504+0000 7f750ca33700 1 --2- 172.21.15.16:0/3262370358 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f750809d6f0 0x7f750809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:56.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.504+0000 7f750ca33700 1 -- 172.21.15.16:0/3262370358 >> 172.21.15.16:0/3262370358 conn(0x7f7508066f50 msgr2=0x7f750809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:56.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.504+0000 7f750ca33700 1 -- 172.21.15.16:0/3262370358 shutdown_connections 2023-09-30T22:30:56.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.504+0000 7f750ca33700 1 -- 172.21.15.16:0/3262370358 wait complete. 2023-09-30T22:30:56.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.505+0000 7f25ce613700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f25c8130750 0x7f25c8130b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:56.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.505+0000 7f25ce613700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f25c809e250 0x7f25c81384c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:56.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.505+0000 7f25ce613700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25c809c190 con 0x7f25c809d6f0 2023-09-30T22:30:56.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.505+0000 7f25ce613700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25c809b090 con 0x7f25c809e250 2023-09-30T22:30:56.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.505+0000 7f25ce613700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f25c809a6c0 con 0x7f25c8130750 2023-09-30T22:30:56.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.505+0000 7f25c7fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f25c8130750 0x7f25c8130b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:56.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.505+0000 7f25c7fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f25c8130750 0x7f25c8130b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35712/0 (socket says 172.21.15.16:35712) 2023-09-30T22:30:56.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.505+0000 7f25c7fff700 1 -- 172.21.15.16:0/1908633153 learned_addr learned my addr 172.21.15.16:0/1908633153 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:56.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.505+0000 7f25ccbb0700 1 --2- 172.21.15.16:0/1908633153 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f25c809e250 0x7f25c81384c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:56.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.505+0000 7f25c7fff700 1 -- 172.21.15.16:0/1908633153 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f25c809e250 msgr2=0x7f25c81384c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:56.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.505+0000 7f25c7fff700 1 --2- 172.21.15.16:0/1908633153 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f25c809e250 0x7f25c81384c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:56.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.505+0000 7f25c7fff700 1 -- 172.21.15.16:0/1908633153 >> v1:172.21.15.16:6789/0 conn(0x7f25c809d6f0 legacy=0x7f25c809db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:56.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.505+0000 7f25c7fff700 1 -- 172.21.15.16:0/1908633153 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c8131060 con 0x7f25c8130750 2023-09-30T22:30:56.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.505+0000 7f25ccbb0700 1 --2- 172.21.15.16:0/1908633153 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f25c809e250 0x7f25c81384c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:56.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.505+0000 7f25c7fff700 1 --2- 172.21.15.16:0/1908633153 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f25c8130750 0x7f25c8130b20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f25b8009920 tx=0x7f25b8009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e61c41c0f9e0907 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:56.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.506+0000 7f25c6ffd700 1 -- 172.21.15.16:0/1908633153 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b8004230 con 0x7f25c8130750 2023-09-30T22:30:56.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.506+0000 7f25c6ffd700 1 -- 172.21.15.16:0/1908633153 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25b8004390 con 0x7f25c8130750 2023-09-30T22:30:56.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.506+0000 7f25c6ffd700 1 -- 172.21.15.16:0/1908633153 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b80036a0 con 0x7f25c8130750 2023-09-30T22:30:56.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.506+0000 7f750ca33700 1 Processor -- start 2023-09-30T22:30:56.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.506+0000 7f750ca33700 1 -- start start 2023-09-30T22:30:56.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.507+0000 7f750ca33700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f750809d6f0 0x7f750811dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:56.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.507+0000 7f750ca33700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f750809e000 0x7f750811e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:56.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.507+0000 7f750ca33700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f750809e410 0x7f750811e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:56.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.507+0000 7f750ca33700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7508122280 con 0x7f750809d6f0 2023-09-30T22:30:56.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.507+0000 7f750ca33700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75081215e0 con 0x7f750809e410 2023-09-30T22:30:56.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.507+0000 7f750ca33700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f750811fac0 con 0x7f750809e000 2023-09-30T22:30:56.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.507+0000 7f25ce613700 1 -- 172.21.15.16:0/1908633153 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f25c8130750 msgr2=0x7f25c8130b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:56.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.507+0000 7f25ce613700 1 --2- 172.21.15.16:0/1908633153 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f25c8130750 0x7f25c8130b20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f25b8009920 tx=0x7f25b8009c30 comp rx=0 tx=0).stop 2023-09-30T22:30:56.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.507+0000 7f7505d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f750809e000 0x7f750811e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:56.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.507+0000 7f7505d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f750809e000 0x7f750811e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35718/0 (socket says 172.21.15.16:35718) 2023-09-30T22:30:56.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.507+0000 7f7505d9b700 1 -- 172.21.15.16:0/3025486696 learned_addr learned my addr 172.21.15.16:0/3025486696 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:56.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.507+0000 7f7506d9d700 1 --2- 172.21.15.16:0/3025486696 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f750809e410 0x7f750811e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:56.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.507+0000 7f750659c700 1 --2- 172.21.15.16:0/3025486696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f750809d6f0 0x7f750811dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:56.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.507+0000 7f25ce613700 1 -- 172.21.15.16:0/1908633153 shutdown_connections 2023-09-30T22:30:56.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.507+0000 7f25ce613700 1 --2- 172.21.15.16:0/1908633153 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f25c809e250 0x7f25c81384c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:56.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.507+0000 7f25ce613700 1 --2- 172.21.15.16:0/1908633153 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f25c8130750 0x7f25c8130b20 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:56.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.507+0000 7f25ce613700 1 -- 172.21.15.16:0/1908633153 >> 172.21.15.16:0/1908633153 conn(0x7f25c809f470 msgr2=0x7f25c80a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:56.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.507+0000 7f25ce613700 1 -- 172.21.15.16:0/1908633153 shutdown_connections 2023-09-30T22:30:56.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.507+0000 7f25ce613700 1 -- 172.21.15.16:0/1908633153 wait complete. 2023-09-30T22:30:56.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.507+0000 7f7505d9b700 1 -- 172.21.15.16:0/3025486696 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f750809e410 msgr2=0x7f750811e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:56.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.507+0000 7f7505d9b700 1 --2- 172.21.15.16:0/3025486696 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f750809e410 0x7f750811e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:56.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.507+0000 7f7505d9b700 1 -- 172.21.15.16:0/3025486696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f750809d6f0 msgr2=0x7f750811dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:56.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.507+0000 7f7505d9b700 1 --2- 172.21.15.16:0/3025486696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f750809d6f0 0x7f750811dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:56.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.507+0000 7f7505d9b700 1 -- 172.21.15.16:0/3025486696 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7500009a70 con 0x7f750809e000 2023-09-30T22:30:56.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.507+0000 7f7505d9b700 1 --2- 172.21.15.16:0/3025486696 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f750809e000 0x7f750811e2e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f74fc00c520 tx=0x7f74fc00c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:56.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.508+0000 7f74f6ffd700 1 -- 172.21.15.16:0/3025486696 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74fc004c80 con 0x7f750809e000 2023-09-30T22:30:56.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.508+0000 7f74f6ffd700 1 -- 172.21.15.16:0/3025486696 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74fc004de0 con 0x7f750809e000 2023-09-30T22:30:56.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.508+0000 7f750ca33700 1 -- 172.21.15.16:0/3025486696 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7508072040 con 0x7f750809e000 2023-09-30T22:30:56.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.508+0000 7f74f6ffd700 1 -- 172.21.15.16:0/3025486696 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74fc007310 con 0x7f750809e000 2023-09-30T22:30:56.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.508+0000 7f750ca33700 1 -- 172.21.15.16:0/3025486696 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75081218c0 con 0x7f750809e000 2023-09-30T22:30:56.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.508+0000 7f25ce613700 1 Processor -- start 2023-09-30T22:30:56.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.509+0000 7f25ce613700 1 -- start start 2023-09-30T22:30:56.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.509+0000 7f25ce613700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f25c809d6f0 0x7f25c812b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:56.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.509+0000 7f25ce613700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f25c809e250 0x7f25c812b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:56.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.509+0000 7f25ce613700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25c8130750 0x7f25c812baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:56.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.509+0000 7f25ce613700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c80a08a0 con 0x7f25c8130750 2023-09-30T22:30:56.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.509+0000 7f25ce613700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25c809b2b0 con 0x7f25c809e250 2023-09-30T22:30:56.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.509+0000 7f25ce613700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f25c808b690 con 0x7f25c809d6f0 2023-09-30T22:30:56.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.509+0000 7f25c7fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f25c809d6f0 0x7f25c812b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:56.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.509+0000 7f25c77fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f25c809e250 0x7f25c812b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:56.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.509+0000 7f25c7fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f25c809d6f0 0x7f25c812b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35720/0 (socket says 172.21.15.16:35720) 2023-09-30T22:30:56.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.509+0000 7f74f6ffd700 1 -- 172.21.15.16:0/3025486696 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f74fc007470 con 0x7f750809e000 2023-09-30T22:30:56.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.509+0000 7f74f6ffd700 1 -- 172.21.15.16:0/3025486696 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f74fc03f260 con 0x7f750809e000 2023-09-30T22:30:56.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.509+0000 7f25c77fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f25c809e250 0x7f25c812b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41292/0 (socket says 172.21.15.16:41292) 2023-09-30T22:30:56.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.509+0000 7f25c7fff700 1 -- 172.21.15.16:0/4227609169 learned_addr learned my addr 172.21.15.16:0/4227609169 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:56.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.509+0000 7f750ca33700 1 -- 172.21.15.16:0/3025486696 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7508121720 con 0x7f750809e000 2023-09-30T22:30:56.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.510+0000 7f25c77fe700 1 -- 172.21.15.16:0/4227609169 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f25c809d6f0 msgr2=0x7f25c812b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:56.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.510+0000 7f25c77fe700 1 --2- 172.21.15.16:0/4227609169 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f25c809d6f0 0x7f25c812b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.510+0000 7f25c77fe700 1 -- 172.21.15.16:0/4227609169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25c8130750 msgr2=0x7f25c812baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.510+0000 7f25c77fe700 1 --2- 172.21.15.16:0/4227609169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25c8130750 0x7f25c812baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.510+0000 7f25c77fe700 1 -- 172.21.15.16:0/4227609169 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c0009a70 con 0x7f25c809e250 2023-09-30T22:30:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.510+0000 7f25c7fff700 1 --2- 172.21.15.16:0/4227609169 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f25c809d6f0 0x7f25c812b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.510+0000 7f25c77fe700 1 --2- 172.21.15.16:0/4227609169 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f25c809e250 0x7f25c812b560 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f25bc00b390 tx=0x7f25bc00b6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.510+0000 7f25c4ff9700 1 -- 172.21.15.16:0/4227609169 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25bc011620 con 0x7f25c809e250 2023-09-30T22:30:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.510+0000 7f25c4ff9700 1 -- 172.21.15.16:0/4227609169 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25bc011780 con 0x7f25c809e250 2023-09-30T22:30:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.510+0000 7f25ce613700 1 -- 172.21.15.16:0/4227609169 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25b8009550 con 0x7f25c809e250 2023-09-30T22:30:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.510+0000 7f25c4ff9700 1 -- 172.21.15.16:0/4227609169 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25bc011b30 con 0x7f25c809e250 2023-09-30T22:30:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.510+0000 7f25ce613700 1 -- 172.21.15.16:0/4227609169 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25c812d300 con 0x7f25c809e250 2023-09-30T22:30:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.511+0000 7f25c4ff9700 1 -- 172.21.15.16:0/4227609169 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f25bc011c90 con 0x7f25c809e250 2023-09-30T22:30:56.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.511+0000 7f25c4ff9700 1 -- 172.21.15.16:0/4227609169 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f25bc03f510 con 0x7f25c809e250 2023-09-30T22:30:56.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.511+0000 7f25ce613700 1 -- 172.21.15.16:0/4227609169 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25c808b300 con 0x7f25c809e250 2023-09-30T22:30:56.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.514+0000 7f74f6ffd700 1 -- 172.21.15.16:0/3025486696 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7508121720 con 0x7f750809e000 2023-09-30T22:30:56.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.516+0000 7f25c4ff9700 1 -- 172.21.15.16:0/4227609169 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f25bc00f2e0 con 0x7f25c809e250 2023-09-30T22:30:56.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.676+0000 7f65e426b700 1 -- 172.21.15.16:0/427681482 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f65dc071d20 con 0x7f65dc09e930 2023-09-30T22:30:56.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.676+0000 7f65d2ffd700 1 -- 172.21.15.16:0/427681482 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f65d801a450 con 0x7f65dc09e930 2023-09-30T22:30:56.677 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:30:56.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.678+0000 7f65e426b700 1 -- 172.21.15.16:0/427681482 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f65dc09e930 msgr2=0x7f65dc06b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:56.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.678+0000 7f65e426b700 1 --2- 172.21.15.16:0/427681482 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f65dc09e930 0x7f65dc06b880 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f65d8000c00 tx=0x7f65d8004750 comp rx=0 tx=0).stop 2023-09-30T22:30:56.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.678+0000 7f65e426b700 1 -- 172.21.15.16:0/427681482 shutdown_connections 2023-09-30T22:30:56.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.678+0000 7f65e426b700 1 --2- 172.21.15.16:0/427681482 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f65dc09e930 0x7f65dc06b880 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:56.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.678+0000 7f65e426b700 1 --2- 172.21.15.16:0/427681482 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f65dc09f650 0x7f65dc06c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:56.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.678+0000 7f65e426b700 1 --2- 172.21.15.16:0/427681482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65dc09f240 0x7f65dc06bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:56.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.678+0000 7f65e426b700 1 -- 172.21.15.16:0/427681482 >> 172.21.15.16:0/427681482 conn(0x7f65dc0a4210 msgr2=0x7f65dc1b8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:56.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.678+0000 7f65e426b700 1 -- 172.21.15.16:0/427681482 shutdown_connections 2023-09-30T22:30:56.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.678+0000 7f65e426b700 1 -- 172.21.15.16:0/427681482 wait complete. 2023-09-30T22:30:56.692 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:30:56.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.722+0000 7f750ca33700 1 -- 172.21.15.16:0/3025486696 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7508121720 con 0x7f750809e000 2023-09-30T22:30:56.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.722+0000 7f74f6ffd700 1 -- 172.21.15.16:0/3025486696 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7508121720 con 0x7f750809e000 2023-09-30T22:30:56.723 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:30:56.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.725+0000 7f750ca33700 1 -- 172.21.15.16:0/3025486696 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f750809e000 msgr2=0x7f750811e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:56.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.725+0000 7f750ca33700 1 --2- 172.21.15.16:0/3025486696 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f750809e000 0x7f750811e2e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f74fc00c520 tx=0x7f74fc00c830 comp rx=0 tx=0).stop 2023-09-30T22:30:56.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.725+0000 7f750ca33700 1 -- 172.21.15.16:0/3025486696 shutdown_connections 2023-09-30T22:30:56.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.725+0000 7f750ca33700 1 --2- 172.21.15.16:0/3025486696 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f750809e410 0x7f750811e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:56.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.725+0000 7f750ca33700 1 --2- 172.21.15.16:0/3025486696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f750809d6f0 0x7f750811dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:56.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.725+0000 7f750ca33700 1 --2- 172.21.15.16:0/3025486696 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f750809e000 0x7f750811e2e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:56.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.725+0000 7f750ca33700 1 -- 172.21.15.16:0/3025486696 >> 172.21.15.16:0/3025486696 conn(0x7f7508066f50 msgr2=0x7f7508130a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:56.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.725+0000 7f750ca33700 1 -- 172.21.15.16:0/3025486696 shutdown_connections 2023-09-30T22:30:56.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.725+0000 7f750ca33700 1 -- 172.21.15.16:0/3025486696 wait complete. 2023-09-30T22:30:56.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.726+0000 7f25ce613700 1 -- 172.21.15.16:0/4227609169 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f25c80747f0 con 0x7f25c809e250 2023-09-30T22:30:56.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.726+0000 7f25c4ff9700 1 -- 172.21.15.16:0/4227609169 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f25bc029090 con 0x7f25c809e250 2023-09-30T22:30:56.729 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:30:56.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.730+0000 7f25ce613700 1 -- 172.21.15.16:0/4227609169 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f25c809e250 msgr2=0x7f25c812b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:56.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.730+0000 7f25ce613700 1 --2- 172.21.15.16:0/4227609169 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f25c809e250 0x7f25c812b560 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f25bc00b390 tx=0x7f25bc00b6a0 comp rx=0 tx=0).stop 2023-09-30T22:30:56.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.731+0000 7f25ce613700 1 -- 172.21.15.16:0/4227609169 shutdown_connections 2023-09-30T22:30:56.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.731+0000 7f25ce613700 1 --2- 172.21.15.16:0/4227609169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25c8130750 0x7f25c812baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:56.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.731+0000 7f25ce613700 1 --2- 172.21.15.16:0/4227609169 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f25c809d6f0 0x7f25c812b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:56.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.731+0000 7f25ce613700 1 --2- 172.21.15.16:0/4227609169 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f25c809e250 0x7f25c812b560 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:56.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.731+0000 7f25ce613700 1 -- 172.21.15.16:0/4227609169 >> 172.21.15.16:0/4227609169 conn(0x7f25c809f470 msgr2=0x7f25c80992c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:56.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.731+0000 7f25ce613700 1 -- 172.21.15.16:0/4227609169 shutdown_connections 2023-09-30T22:30:56.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.731+0000 7f25ce613700 1 -- 172.21.15.16:0/4227609169 wait complete. 2023-09-30T22:30:56.746 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:30:56.750 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:30:56.756 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:56.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.930+0000 7ff31a148700 1 Processor -- start 2023-09-30T22:30:56.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.930+0000 7ff31a148700 1 -- start start 2023-09-30T22:30:56.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.930+0000 7ff31a148700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff31409d6f0 0x7ff31409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:56.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.930+0000 7ff31a148700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff31409e410 0x7ff314130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:56.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.931+0000 7ff31a148700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff31409f130 con 0x7ff31409e000 2023-09-30T22:30:56.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.931+0000 7ff31a148700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff31409f520 con 0x7ff31409e410 2023-09-30T22:30:56.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.931+0000 7ff31a148700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff31409e930 con 0x7ff31409d6f0 2023-09-30T22:30:56.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.931+0000 7ff3137fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff31409d6f0 0x7ff31409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:56.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.931+0000 7ff313fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff31409e410 0x7ff314130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:56.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.931+0000 7ff313fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff31409e410 0x7ff314130d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44056/0 (socket says 172.21.15.16:44056) 2023-09-30T22:30:56.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.931+0000 7ff3137fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff31409d6f0 0x7ff31409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35732/0 (socket says 172.21.15.16:35732) 2023-09-30T22:30:56.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.931+0000 7ff313fff700 1 -- 172.21.15.16:0/3113023319 learned_addr learned my addr 172.21.15.16:0/3113023319 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:56.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.931+0000 7ff3137fe700 1 -- 172.21.15.16:0/3113023319 >> v2:172.21.15.16:3300/0 conn(0x7ff31409e410 msgr2=0x7ff314130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:56.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.931+0000 7ff3137fe700 1 --2- 172.21.15.16:0/3113023319 >> v2:172.21.15.16:3300/0 conn(0x7ff31409e410 0x7ff314130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:56.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.931+0000 7ff3137fe700 1 -- 172.21.15.16:0/3113023319 >> v1:172.21.15.16:6789/0 conn(0x7ff31409e000 legacy=0x7ff314130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:56.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.931+0000 7ff3137fe700 1 -- 172.21.15.16:0/3113023319 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff31409b2d0 con 0x7ff31409d6f0 2023-09-30T22:30:56.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.931+0000 7ff313fff700 1 --2- 172.21.15.16:0/3113023319 >> v2:172.21.15.16:3300/0 conn(0x7ff31409e410 0x7ff314130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:56.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.931+0000 7ff3137fe700 1 --2- 172.21.15.16:0/3113023319 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff31409d6f0 0x7ff31409dac0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff304009840 tx=0x7ff304009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=becb6f6740d2fb51 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:56.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.931+0000 7ff3127fc700 1 -- 172.21.15.16:0/3113023319 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff304011830 con 0x7ff31409d6f0 2023-09-30T22:30:56.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.931+0000 7ff3127fc700 1 -- 172.21.15.16:0/3113023319 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff304011990 con 0x7ff31409d6f0 2023-09-30T22:30:56.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.931+0000 7ff3127fc700 1 -- 172.21.15.16:0/3113023319 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff304011d20 con 0x7ff31409d6f0 2023-09-30T22:30:56.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.932+0000 7ff31a148700 1 -- 172.21.15.16:0/3113023319 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff31409d6f0 msgr2=0x7ff31409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:56.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.932+0000 7ff31a148700 1 --2- 172.21.15.16:0/3113023319 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff31409d6f0 0x7ff31409dac0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff304009840 tx=0x7ff304009b50 comp rx=0 tx=0).stop 2023-09-30T22:30:56.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.932+0000 7ff31a148700 1 -- 172.21.15.16:0/3113023319 shutdown_connections 2023-09-30T22:30:56.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.932+0000 7ff31a148700 1 --2- 172.21.15.16:0/3113023319 >> v2:172.21.15.16:3300/0 conn(0x7ff31409e410 0x7ff314130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:56.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.932+0000 7ff31a148700 1 --2- 172.21.15.16:0/3113023319 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff31409d6f0 0x7ff31409dac0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:56.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.932+0000 7ff31a148700 1 -- 172.21.15.16:0/3113023319 >> 172.21.15.16:0/3113023319 conn(0x7ff314066f50 msgr2=0x7ff31409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:56.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.932+0000 7ff31a148700 1 -- 172.21.15.16:0/3113023319 shutdown_connections 2023-09-30T22:30:56.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.932+0000 7ff31a148700 1 -- 172.21.15.16:0/3113023319 wait complete. 2023-09-30T22:30:56.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.933+0000 7ff31a148700 1 Processor -- start 2023-09-30T22:30:56.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.933+0000 7ff31a148700 1 -- start start 2023-09-30T22:30:56.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.934+0000 7ff31a148700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff31409d6f0 0x7ff31411e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:56.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.934+0000 7ff31a148700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff31409e000 0x7ff31411e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:56.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.934+0000 7ff31a148700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff31409e410 0x7ff31411ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:56.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.934+0000 7ff31a148700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3141205c0 con 0x7ff31409e410 2023-09-30T22:30:56.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.934+0000 7ff31a148700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3140a1d90 con 0x7ff31409e000 2023-09-30T22:30:56.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.934+0000 7ff31a148700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff314121c00 con 0x7ff31409d6f0 2023-09-30T22:30:56.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.934+0000 7ff3137fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff31409d6f0 0x7ff31411e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:56.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.934+0000 7ff313fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff31409e410 0x7ff31411ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:56.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.934+0000 7ff3137fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff31409d6f0 0x7ff31411e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35738/0 (socket says 172.21.15.16:35738) 2023-09-30T22:30:56.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.934+0000 7ff313fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff31409e410 0x7ff31411ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44068/0 (socket says 172.21.15.16:44068) 2023-09-30T22:30:56.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.934+0000 7ff3137fe700 1 -- 172.21.15.16:0/1426808433 learned_addr learned my addr 172.21.15.16:0/1426808433 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:56.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.934+0000 7ff312ffd700 1 --2- 172.21.15.16:0/1426808433 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff31409e000 0x7ff31411e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:56.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.934+0000 7ff3137fe700 1 -- 172.21.15.16:0/1426808433 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff31409e000 msgr2=0x7ff31411e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:56.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.934+0000 7ff3137fe700 1 --2- 172.21.15.16:0/1426808433 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff31409e000 0x7ff31411e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:56.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.934+0000 7ff3137fe700 1 -- 172.21.15.16:0/1426808433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff31409e410 msgr2=0x7ff31411ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:56.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.934+0000 7ff3137fe700 1 --2- 172.21.15.16:0/1426808433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff31409e410 0x7ff31411ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:56.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.934+0000 7ff3137fe700 1 -- 172.21.15.16:0/1426808433 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff30c00bd40 con 0x7ff31409d6f0 2023-09-30T22:30:56.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.934+0000 7ff313fff700 1 --2- 172.21.15.16:0/1426808433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff31409e410 0x7ff31411ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:56.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.934+0000 7ff3137fe700 1 --2- 172.21.15.16:0/1426808433 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff31409d6f0 0x7ff31411e170 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff304009fd0 tx=0x7ff304004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:56.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.935+0000 7ff2fbfff700 1 -- 172.21.15.16:0/1426808433 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3040045d0 con 0x7ff31409d6f0 2023-09-30T22:30:56.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.935+0000 7ff2fbfff700 1 -- 172.21.15.16:0/1426808433 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff30401f070 con 0x7ff31409d6f0 2023-09-30T22:30:56.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.935+0000 7ff31a148700 1 -- 172.21.15.16:0/1426808433 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3040094a0 con 0x7ff31409d6f0 2023-09-30T22:30:56.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.935+0000 7ff2fbfff700 1 -- 172.21.15.16:0/1426808433 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3040038a0 con 0x7ff31409d6f0 2023-09-30T22:30:56.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.935+0000 7ff31a148700 1 -- 172.21.15.16:0/1426808433 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff314121390 con 0x7ff31409d6f0 2023-09-30T22:30:56.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.935+0000 7ff2fbfff700 1 -- 172.21.15.16:0/1426808433 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff304003a00 con 0x7ff31409d6f0 2023-09-30T22:30:56.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.936+0000 7ff2fbfff700 1 -- 172.21.15.16:0/1426808433 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7ff304021070 con 0x7ff31409d6f0 2023-09-30T22:30:56.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.936+0000 7ff31a148700 1 -- 172.21.15.16:0/1426808433 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff314120300 con 0x7ff31409d6f0 2023-09-30T22:30:56.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:56.941+0000 7ff2fbfff700 1 -- 172.21.15.16:0/1426808433 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff314120300 con 0x7ff31409d6f0 2023-09-30T22:30:57.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.144+0000 7ff31a148700 1 -- 172.21.15.16:0/1426808433 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff314120300 con 0x7ff31409d6f0 2023-09-30T22:30:57.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.145+0000 7ff2fbfff700 1 -- 172.21.15.16:0/1426808433 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff314120300 con 0x7ff31409d6f0 2023-09-30T22:30:57.145 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:30:57.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.147+0000 7ff31a148700 1 -- 172.21.15.16:0/1426808433 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff31409d6f0 msgr2=0x7ff31411e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:57.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.147+0000 7ff31a148700 1 --2- 172.21.15.16:0/1426808433 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff31409d6f0 0x7ff31411e170 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff304009fd0 tx=0x7ff304004750 comp rx=0 tx=0).stop 2023-09-30T22:30:57.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.147+0000 7ff31a148700 1 -- 172.21.15.16:0/1426808433 shutdown_connections 2023-09-30T22:30:57.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.147+0000 7ff31a148700 1 --2- 172.21.15.16:0/1426808433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff31409e410 0x7ff31411ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:57.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.147+0000 7ff31a148700 1 --2- 172.21.15.16:0/1426808433 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff31409d6f0 0x7ff31411e170 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:57.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.147+0000 7ff31a148700 1 --2- 172.21.15.16:0/1426808433 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff31409e000 0x7ff31411e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:57.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.147+0000 7ff31a148700 1 -- 172.21.15.16:0/1426808433 >> 172.21.15.16:0/1426808433 conn(0x7ff314066f50 msgr2=0x7ff31406f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:57.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.147+0000 7ff31a148700 1 -- 172.21.15.16:0/1426808433 shutdown_connections 2023-09-30T22:30:57.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.147+0000 7ff31a148700 1 -- 172.21.15.16:0/1426808433 wait complete. 2023-09-30T22:30:57.161 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:30:57.693 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:57.747 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:57.751 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:57.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.843+0000 7fc234f6f700 1 Processor -- start 2023-09-30T22:30:57.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.843+0000 7fc234f6f700 1 -- start start 2023-09-30T22:30:57.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.844+0000 7fc234f6f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc23009e1d0 0x7fc230135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:57.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.844+0000 7fc234f6f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc230135e50 0x7fc230138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:57.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.844+0000 7fc234f6f700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc23009f130 con 0x7fc23009d6f0 2023-09-30T22:30:57.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.844+0000 7fc234f6f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc23009f520 con 0x7fc230135e50 2023-09-30T22:30:57.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.844+0000 7fc234f6f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc23009e930 con 0x7fc23009e1d0 2023-09-30T22:30:57.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.844+0000 7fc22dd9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc23009e1d0 0x7fc230135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:57.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.844+0000 7fc22dd9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc23009e1d0 0x7fc230135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:57720/0 (socket says 172.21.15.16:57720) 2023-09-30T22:30:57.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.844+0000 7fc22dd9b700 1 -- 172.21.15.16:0/1152961323 learned_addr learned my addr 172.21.15.16:0/1152961323 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:57.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.844+0000 7fc22ed9d700 1 --2- 172.21.15.16:0/1152961323 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc230135e50 0x7fc230138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:57.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.844+0000 7fc22dd9b700 1 -- 172.21.15.16:0/1152961323 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc230135e50 msgr2=0x7fc230138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:57.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.844+0000 7fc22dd9b700 1 --2- 172.21.15.16:0/1152961323 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc230135e50 0x7fc230138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:57.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.844+0000 7fc22d59a700 1 -- 172.21.15.16:0/1152961323 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 878952388 0 0) 0x7fc23009f130 con 0x7fc23009d6f0 2023-09-30T22:30:57.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.844+0000 7fc22dd9b700 1 -- 172.21.15.16:0/1152961323 >> v1:172.21.15.16:6789/0 conn(0x7fc23009d6f0 legacy=0x7fc23009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:57.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.844+0000 7fc22dd9b700 1 -- 172.21.15.16:0/1152961323 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc23009b2d0 con 0x7fc23009e1d0 2023-09-30T22:30:57.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.844+0000 7fc22ed9d700 1 --2- 172.21.15.16:0/1152961323 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc230135e50 0x7fc230138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:57.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.845+0000 7fc22dd9b700 1 --2- 172.21.15.16:0/1152961323 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc23009e1d0 0x7fc230135910 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc224009920 tx=0x7fc224009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a210bae76890efd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:57.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.845+0000 7fc22d59a700 1 -- 172.21.15.16:0/1152961323 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc224011830 con 0x7fc23009e1d0 2023-09-30T22:30:57.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.845+0000 7fc22d59a700 1 -- 172.21.15.16:0/1152961323 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc224011990 con 0x7fc23009e1d0 2023-09-30T22:30:57.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.845+0000 7fc22d59a700 1 -- 172.21.15.16:0/1152961323 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc224011d20 con 0x7fc23009e1d0 2023-09-30T22:30:57.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.845+0000 7fc234f6f700 1 -- 172.21.15.16:0/1152961323 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc23009e1d0 msgr2=0x7fc230135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:57.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.846+0000 7fc234f6f700 1 --2- 172.21.15.16:0/1152961323 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc23009e1d0 0x7fc230135910 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc224009920 tx=0x7fc224009c30 comp rx=0 tx=0).stop 2023-09-30T22:30:57.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.846+0000 7fc234f6f700 1 -- 172.21.15.16:0/1152961323 shutdown_connections 2023-09-30T22:30:57.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.846+0000 7fc234f6f700 1 --2- 172.21.15.16:0/1152961323 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc230135e50 0x7fc230138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:57.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.846+0000 7fc234f6f700 1 --2- 172.21.15.16:0/1152961323 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc23009e1d0 0x7fc230135910 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:57.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.846+0000 7fc234f6f700 1 -- 172.21.15.16:0/1152961323 >> 172.21.15.16:0/1152961323 conn(0x7fc230066f50 msgr2=0x7fc23009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:57.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.846+0000 7fc234f6f700 1 -- 172.21.15.16:0/1152961323 shutdown_connections 2023-09-30T22:30:57.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.846+0000 7fc234f6f700 1 -- 172.21.15.16:0/1152961323 wait complete. 2023-09-30T22:30:57.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.847+0000 7fc234f6f700 1 Processor -- start 2023-09-30T22:30:57.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.847+0000 7fc234f6f700 1 -- start start 2023-09-30T22:30:57.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.847+0000 7fc234f6f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc23009d6f0 0x7fc230137ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:57.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.847+0000 7fc234f6f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc23009e1d0 0x7fc230138410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:57.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.847+0000 7fc234f6f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc230135e50 0x7fc230138950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:57.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.847+0000 7fc234f6f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc23013a900 con 0x7fc23009e1d0 2023-09-30T22:30:57.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.848+0000 7fc234f6f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc23011e060 con 0x7fc23009d6f0 2023-09-30T22:30:57.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.848+0000 7fc234f6f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc23009c700 con 0x7fc230135e50 2023-09-30T22:30:57.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.848+0000 7fc22dd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc23009e1d0 0x7fc230138410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:57.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.848+0000 7fc22e59c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc23009d6f0 0x7fc230137ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:57.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.848+0000 7fc22dd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc23009e1d0 0x7fc230138410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55354/0 (socket says 172.21.15.16:55354) 2023-09-30T22:30:57.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.848+0000 7fc22dd9b700 1 -- 172.21.15.16:0/4093707625 learned_addr learned my addr 172.21.15.16:0/4093707625 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:57.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.848+0000 7fc22ed9d700 1 --2- 172.21.15.16:0/4093707625 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc230135e50 0x7fc230138950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:57.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.848+0000 7fc22ed9d700 1 -- 172.21.15.16:0/4093707625 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc23009d6f0 msgr2=0x7fc230137ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:57.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.848+0000 7fc22ed9d700 1 --2- 172.21.15.16:0/4093707625 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc23009d6f0 0x7fc230137ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:57.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.848+0000 7fc22ed9d700 1 -- 172.21.15.16:0/4093707625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc23009e1d0 msgr2=0x7fc230138410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:57.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.848+0000 7fc22ed9d700 1 --2- 172.21.15.16:0/4093707625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc23009e1d0 0x7fc230138410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:57.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.848+0000 7fc22ed9d700 1 -- 172.21.15.16:0/4093707625 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc228009a70 con 0x7fc230135e50 2023-09-30T22:30:57.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.848+0000 7fc22e59c700 1 --2- 172.21.15.16:0/4093707625 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc23009d6f0 0x7fc230137ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:57.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.848+0000 7fc22dd9b700 1 --2- 172.21.15.16:0/4093707625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc23009e1d0 0x7fc230138410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:57.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.848+0000 7fc22ed9d700 1 --2- 172.21.15.16:0/4093707625 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc230135e50 0x7fc230138950 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc22800ef90 tx=0x7fc22800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:57.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.849+0000 7fc21effd700 1 -- 172.21.15.16:0/4093707625 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc22800ca90 con 0x7fc230135e50 2023-09-30T22:30:57.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.849+0000 7fc21effd700 1 -- 172.21.15.16:0/4093707625 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc22800f040 con 0x7fc230135e50 2023-09-30T22:30:57.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.849+0000 7fc234f6f700 1 -- 172.21.15.16:0/4093707625 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc224009550 con 0x7fc230135e50 2023-09-30T22:30:57.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.849+0000 7fc21effd700 1 -- 172.21.15.16:0/4093707625 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc22800ce30 con 0x7fc230135e50 2023-09-30T22:30:57.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.849+0000 7fc234f6f700 1 -- 172.21.15.16:0/4093707625 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc23009a0c0 con 0x7fc230135e50 2023-09-30T22:30:57.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.850+0000 7fc234f6f700 1 -- 172.21.15.16:0/4093707625 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc23011dbe0 con 0x7fc230135e50 2023-09-30T22:30:57.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.851+0000 7fc21effd700 1 -- 172.21.15.16:0/4093707625 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc228003a20 con 0x7fc230135e50 2023-09-30T22:30:57.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.851+0000 7fc21effd700 1 -- 172.21.15.16:0/4093707625 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fc228040590 con 0x7fc230135e50 2023-09-30T22:30:57.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.855+0000 7fc21effd700 1 -- 172.21.15.16:0/4093707625 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc228025bc0 con 0x7fc230135e50 2023-09-30T22:30:57.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.891+0000 7faa21f57700 1 Processor -- start 2023-09-30T22:30:57.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.891+0000 7faa21f57700 1 -- start start 2023-09-30T22:30:57.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.892+0000 7faa21f57700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faa1c09f410 0x7faa1c136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:57.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.892+0000 7faa21f57700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faa1c136fe0 0x7faa1c1393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:57.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.892+0000 7faa21f57700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa1c070940 con 0x7faa1c09e930 2023-09-30T22:30:57.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.892+0000 7faa21f57700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa1c06fd60 con 0x7faa1c09f410 2023-09-30T22:30:57.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.892+0000 7faa21f57700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c06fa80 con 0x7faa1c136fe0 2023-09-30T22:30:57.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.892+0000 7faa1bfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faa1c136fe0 0x7faa1c1393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:57.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.892+0000 7faa1bfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faa1c136fe0 0x7faa1c1393c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:57746/0 (socket says 172.21.15.16:57746) 2023-09-30T22:30:57.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.892+0000 7faa1bfff700 1 -- 172.21.15.16:0/386585753 learned_addr learned my addr 172.21.15.16:0/386585753 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:57.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.892+0000 7faa12ffd700 1 --2- 172.21.15.16:0/386585753 >> v2:172.21.15.16:3300/0 conn(0x7faa1c09f410 0x7faa1c136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:57.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.892+0000 7faa1bfff700 1 -- 172.21.15.16:0/386585753 >> v2:172.21.15.16:3300/0 conn(0x7faa1c09f410 msgr2=0x7faa1c136aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:57.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.892+0000 7faa1bfff700 1 --2- 172.21.15.16:0/386585753 >> v2:172.21.15.16:3300/0 conn(0x7faa1c09f410 0x7faa1c136aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:57.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.892+0000 7faa1bfff700 1 -- 172.21.15.16:0/386585753 >> v1:172.21.15.16:6789/0 conn(0x7faa1c09e930 legacy=0x7faa1c09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:57.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.892+0000 7faa1bfff700 1 -- 172.21.15.16:0/386585753 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa1c09fb60 con 0x7faa1c136fe0 2023-09-30T22:30:57.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.893+0000 7faa1bfff700 1 --2- 172.21.15.16:0/386585753 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faa1c136fe0 0x7faa1c1393c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7faa14009e50 tx=0x7faa1400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ffae97ec404303a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:57.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.893+0000 7faa1affd700 1 -- 172.21.15.16:0/386585753 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1400e070 con 0x7faa1c136fe0 2023-09-30T22:30:57.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.893+0000 7faa1affd700 1 -- 172.21.15.16:0/386585753 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa14003a20 con 0x7faa1c136fe0 2023-09-30T22:30:57.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.893+0000 7faa1affd700 1 -- 172.21.15.16:0/386585753 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa14003dd0 con 0x7faa1c136fe0 2023-09-30T22:30:57.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.893+0000 7faa21f57700 1 -- 172.21.15.16:0/386585753 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faa1c136fe0 msgr2=0x7faa1c1393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:57.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.893+0000 7faa21f57700 1 --2- 172.21.15.16:0/386585753 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faa1c136fe0 0x7faa1c1393c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7faa14009e50 tx=0x7faa1400b330 comp rx=0 tx=0).stop 2023-09-30T22:30:57.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.894+0000 7faa21f57700 1 -- 172.21.15.16:0/386585753 shutdown_connections 2023-09-30T22:30:57.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.894+0000 7faa21f57700 1 --2- 172.21.15.16:0/386585753 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faa1c136fe0 0x7faa1c1393c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:57.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.894+0000 7faa21f57700 1 --2- 172.21.15.16:0/386585753 >> v2:172.21.15.16:3300/0 conn(0x7faa1c09f410 0x7faa1c136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:57.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.894+0000 7faa21f57700 1 -- 172.21.15.16:0/386585753 >> 172.21.15.16:0/386585753 conn(0x7faa1c0a4210 msgr2=0x7faa1c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:57.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.894+0000 7faa21f57700 1 -- 172.21.15.16:0/386585753 shutdown_connections 2023-09-30T22:30:57.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.894+0000 7faa21f57700 1 -- 172.21.15.16:0/386585753 wait complete. 2023-09-30T22:30:57.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.895+0000 7faa21f57700 1 Processor -- start 2023-09-30T22:30:57.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.895+0000 7faa21f57700 1 -- start start 2023-09-30T22:30:57.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.896+0000 7faa21f57700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa1c09e930 0x7faa1c11c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:57.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.896+0000 7faa21f57700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faa1c09f410 0x7faa1c11cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:57.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.896+0000 7faa21f57700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faa1c136fe0 0x7faa1c11d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:57.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.896+0000 7faa21f57700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c0a2f40 con 0x7faa1c09e930 2023-09-30T22:30:57.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.896+0000 7faa21f57700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c11fe30 con 0x7faa1c09f410 2023-09-30T22:30:57.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.896+0000 7faa21f57700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faa1c11e200 con 0x7faa1c136fe0 2023-09-30T22:30:57.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.896+0000 7faa1b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa1c09e930 0x7faa1c11c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:57.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.896+0000 7faa12ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faa1c09f410 0x7faa1c11cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:57.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.896+0000 7faa12ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faa1c09f410 0x7faa1c11cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38278/0 (socket says 172.21.15.16:38278) 2023-09-30T22:30:57.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.896+0000 7faa12ffd700 1 -- 172.21.15.16:0/3593916803 learned_addr learned my addr 172.21.15.16:0/3593916803 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:57.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.896+0000 7faa1b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa1c09e930 0x7faa1c11c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55366/0 (socket says 172.21.15.16:55366) 2023-09-30T22:30:57.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.896+0000 7faa1bfff700 1 --2- 172.21.15.16:0/3593916803 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faa1c136fe0 0x7faa1c11d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:57.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.896+0000 7faa12ffd700 1 -- 172.21.15.16:0/3593916803 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faa1c136fe0 msgr2=0x7faa1c11d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:57.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.896+0000 7faa12ffd700 1 --2- 172.21.15.16:0/3593916803 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faa1c136fe0 0x7faa1c11d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:57.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.896+0000 7faa12ffd700 1 -- 172.21.15.16:0/3593916803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa1c09e930 msgr2=0x7faa1c11c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:57.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.897+0000 7faa12ffd700 1 --2- 172.21.15.16:0/3593916803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa1c09e930 0x7faa1c11c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:57.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.897+0000 7faa12ffd700 1 -- 172.21.15.16:0/3593916803 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa14009b30 con 0x7faa1c09f410 2023-09-30T22:30:57.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.897+0000 7faa12ffd700 1 --2- 172.21.15.16:0/3593916803 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faa1c09f410 0x7faa1c11cb30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7faa04010cc0 tx=0x7faa0400b140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:57.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.897+0000 7faa18ff9700 1 -- 172.21.15.16:0/3593916803 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa04015070 con 0x7faa1c09f410 2023-09-30T22:30:57.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.897+0000 7faa18ff9700 1 -- 172.21.15.16:0/3593916803 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa0400b9a0 con 0x7faa1c09f410 2023-09-30T22:30:57.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.897+0000 7faa21f57700 1 -- 172.21.15.16:0/3593916803 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa1c071d20 con 0x7faa1c09f410 2023-09-30T22:30:57.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.897+0000 7faa18ff9700 1 -- 172.21.15.16:0/3593916803 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa04009710 con 0x7faa1c09f410 2023-09-30T22:30:57.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.897+0000 7faa21f57700 1 -- 172.21.15.16:0/3593916803 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa1c11e9f0 con 0x7faa1c09f410 2023-09-30T22:30:57.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.898+0000 7faa18ff9700 1 -- 172.21.15.16:0/3593916803 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7faa0400bd30 con 0x7faa1c09f410 2023-09-30T22:30:57.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.898+0000 7faa18ff9700 1 -- 172.21.15.16:0/3593916803 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7faa04042fa0 con 0x7faa1c09f410 2023-09-30T22:30:57.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.898+0000 7faa21f57700 1 -- 172.21.15.16:0/3593916803 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa1c074ea0 con 0x7faa1c09f410 2023-09-30T22:30:57.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.903+0000 7faa18ff9700 1 -- 172.21.15.16:0/3593916803 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7faa0401e020 con 0x7faa1c09f410 2023-09-30T22:30:57.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.933+0000 7fd31a89f700 1 Processor -- start 2023-09-30T22:30:57.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.933+0000 7fd31a89f700 1 -- start start 2023-09-30T22:30:57.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.934+0000 7fd31a89f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd31409e1d0 0x7fd314135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:57.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.934+0000 7fd31a89f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd314135e50 0x7fd314138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:57.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.934+0000 7fd31a89f700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd31409f130 con 0x7fd31409d6f0 2023-09-30T22:30:57.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.934+0000 7fd31a89f700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd31409f520 con 0x7fd314135e50 2023-09-30T22:30:57.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.934+0000 7fd31a89f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd31409e930 con 0x7fd31409e1d0 2023-09-30T22:30:57.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.934+0000 7fd3137fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd31409e1d0 0x7fd314135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:57.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.934+0000 7fd3137fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd31409e1d0 0x7fd314135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38282/0 (socket says 172.21.15.16:38282) 2023-09-30T22:30:57.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.934+0000 7fd3137fe700 1 -- 172.21.15.16:0/1889131200 learned_addr learned my addr 172.21.15.16:0/1889131200 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:57.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.934+0000 7fd318e3c700 1 --2- 172.21.15.16:0/1889131200 >> v2:172.21.15.16:3300/0 conn(0x7fd314135e50 0x7fd314138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:57.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.934+0000 7fd3137fe700 1 -- 172.21.15.16:0/1889131200 >> v2:172.21.15.16:3300/0 conn(0x7fd314135e50 msgr2=0x7fd314138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:57.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.934+0000 7fd3137fe700 1 --2- 172.21.15.16:0/1889131200 >> v2:172.21.15.16:3300/0 conn(0x7fd314135e50 0x7fd314138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:57.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.934+0000 7fd3137fe700 1 -- 172.21.15.16:0/1889131200 >> v1:172.21.15.16:6789/0 conn(0x7fd31409d6f0 legacy=0x7fd31409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:57.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.934+0000 7fd312ffd700 1 -- 172.21.15.16:0/1889131200 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1139424044 0 0) 0x7fd31409f130 con 0x7fd31409d6f0 2023-09-30T22:30:57.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.934+0000 7fd3137fe700 1 -- 172.21.15.16:0/1889131200 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd31409b2d0 con 0x7fd31409e1d0 2023-09-30T22:30:57.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.934+0000 7fd318e3c700 1 --2- 172.21.15.16:0/1889131200 >> v2:172.21.15.16:3300/0 conn(0x7fd314135e50 0x7fd314138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:57.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.935+0000 7fd3137fe700 1 --2- 172.21.15.16:0/1889131200 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd31409e1d0 0x7fd314135910 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd308009920 tx=0x7fd308009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2445d9b43a98b289 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:57.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.935+0000 7fd312ffd700 1 -- 172.21.15.16:0/1889131200 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd30800f8a0 con 0x7fd31409e1d0 2023-09-30T22:30:57.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.935+0000 7fd312ffd700 1 -- 172.21.15.16:0/1889131200 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd30800fa00 con 0x7fd31409e1d0 2023-09-30T22:30:57.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.935+0000 7fd312ffd700 1 -- 172.21.15.16:0/1889131200 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd30800fd90 con 0x7fd31409e1d0 2023-09-30T22:30:57.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.935+0000 7fd31a89f700 1 -- 172.21.15.16:0/1889131200 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd31409e1d0 msgr2=0x7fd314135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:57.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.935+0000 7fd31a89f700 1 --2- 172.21.15.16:0/1889131200 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd31409e1d0 0x7fd314135910 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd308009920 tx=0x7fd308009c30 comp rx=0 tx=0).stop 2023-09-30T22:30:57.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.936+0000 7fd31a89f700 1 -- 172.21.15.16:0/1889131200 shutdown_connections 2023-09-30T22:30:57.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.936+0000 7fd31a89f700 1 --2- 172.21.15.16:0/1889131200 >> v2:172.21.15.16:3300/0 conn(0x7fd314135e50 0x7fd314138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:57.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.936+0000 7fd31a89f700 1 --2- 172.21.15.16:0/1889131200 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd31409e1d0 0x7fd314135910 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:57.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.936+0000 7fd31a89f700 1 -- 172.21.15.16:0/1889131200 >> 172.21.15.16:0/1889131200 conn(0x7fd314066f50 msgr2=0x7fd31409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:57.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.936+0000 7fd31a89f700 1 -- 172.21.15.16:0/1889131200 shutdown_connections 2023-09-30T22:30:57.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.936+0000 7fd31a89f700 1 -- 172.21.15.16:0/1889131200 wait complete. 2023-09-30T22:30:57.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.937+0000 7fd31a89f700 1 Processor -- start 2023-09-30T22:30:57.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.937+0000 7fd31a89f700 1 -- start start 2023-09-30T22:30:57.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.938+0000 7fd31a89f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd31409d6f0 0x7fd31411b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:57.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.938+0000 7fd31a89f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd31409e1d0 0x7fd31411b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:57.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.938+0000 7fd31a89f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd314135e50 0x7fd31411bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:57.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.938+0000 7fd31a89f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd31411d790 con 0x7fd31409d6f0 2023-09-30T22:30:57.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.938+0000 7fd31a89f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd314072650 con 0x7fd314135e50 2023-09-30T22:30:57.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.938+0000 7fd31a89f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd31411edd0 con 0x7fd31409e1d0 2023-09-30T22:30:57.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.938+0000 7fd313fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd31409d6f0 0x7fd31411b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:57.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.938+0000 7fd313fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd31409d6f0 0x7fd31411b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55392/0 (socket says 172.21.15.16:55392) 2023-09-30T22:30:57.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.938+0000 7fd313fff700 1 -- 172.21.15.16:0/3673506837 learned_addr learned my addr 172.21.15.16:0/3673506837 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:57.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.938+0000 7fd318e3c700 1 --2- 172.21.15.16:0/3673506837 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd314135e50 0x7fd31411bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:57.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.938+0000 7fd313fff700 1 -- 172.21.15.16:0/3673506837 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd31409e1d0 msgr2=0x7fd31411b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:57.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.938+0000 7fd313fff700 1 --2- 172.21.15.16:0/3673506837 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd31409e1d0 0x7fd31411b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:57.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.938+0000 7fd313fff700 1 -- 172.21.15.16:0/3673506837 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd314135e50 msgr2=0x7fd31411bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:57.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.938+0000 7fd313fff700 1 --2- 172.21.15.16:0/3673506837 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd314135e50 0x7fd31411bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:57.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.938+0000 7fd313fff700 1 -- 172.21.15.16:0/3673506837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd30c009a70 con 0x7fd31409d6f0 2023-09-30T22:30:57.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.939+0000 7fd313fff700 1 --2- 172.21.15.16:0/3673506837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd31409d6f0 0x7fd31411b2b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd304001eb0 tx=0x7fd30400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:57.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.939+0000 7fd310ff9700 1 -- 172.21.15.16:0/3673506837 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3040054e0 con 0x7fd31409d6f0 2023-09-30T22:30:57.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.939+0000 7fd310ff9700 1 -- 172.21.15.16:0/3673506837 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd304003680 con 0x7fd31409d6f0 2023-09-30T22:30:57.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.939+0000 7fd31a89f700 1 -- 172.21.15.16:0/3673506837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd308009550 con 0x7fd31409d6f0 2023-09-30T22:30:57.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.939+0000 7fd310ff9700 1 -- 172.21.15.16:0/3673506837 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3040039f0 con 0x7fd31409d6f0 2023-09-30T22:30:57.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.939+0000 7fd31a89f700 1 -- 172.21.15.16:0/3673506837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd31411e110 con 0x7fd31409d6f0 2023-09-30T22:30:57.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.940+0000 7fd310ff9700 1 -- 172.21.15.16:0/3673506837 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd304003b50 con 0x7fd31409d6f0 2023-09-30T22:30:57.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.940+0000 7fd31a89f700 1 -- 172.21.15.16:0/3673506837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd314132560 con 0x7fd31409d6f0 2023-09-30T22:30:57.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.940+0000 7fd310ff9700 1 -- 172.21.15.16:0/3673506837 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd30403eb40 con 0x7fd31409d6f0 2023-09-30T22:30:57.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:57.949+0000 7fd310ff9700 1 -- 172.21.15.16:0/3673506837 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd30401b3b0 con 0x7fd31409d6f0 2023-09-30T22:30:58.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.072+0000 7fc234f6f700 1 -- 172.21.15.16:0/4093707625 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc23011dbe0 con 0x7fc230135e50 2023-09-30T22:30:58.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.072+0000 7fc21effd700 1 -- 172.21.15.16:0/4093707625 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc23011dbe0 con 0x7fc230135e50 2023-09-30T22:30:58.073 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:30:58.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.075+0000 7fc234f6f700 1 -- 172.21.15.16:0/4093707625 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc230135e50 msgr2=0x7fc230138950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:58.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.075+0000 7fc234f6f700 1 --2- 172.21.15.16:0/4093707625 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc230135e50 0x7fc230138950 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc22800ef90 tx=0x7fc22800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:30:58.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.075+0000 7fc234f6f700 1 -- 172.21.15.16:0/4093707625 shutdown_connections 2023-09-30T22:30:58.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.075+0000 7fc234f6f700 1 --2- 172.21.15.16:0/4093707625 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc23009d6f0 0x7fc230137ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:58.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.075+0000 7fc234f6f700 1 --2- 172.21.15.16:0/4093707625 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc230135e50 0x7fc230138950 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:58.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.075+0000 7fc234f6f700 1 --2- 172.21.15.16:0/4093707625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc23009e1d0 0x7fc230138410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:58.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.075+0000 7fc234f6f700 1 -- 172.21.15.16:0/4093707625 >> 172.21.15.16:0/4093707625 conn(0x7fc230066f50 msgr2=0x7fc23006ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:58.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.075+0000 7fc234f6f700 1 -- 172.21.15.16:0/4093707625 shutdown_connections 2023-09-30T22:30:58.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.075+0000 7fc234f6f700 1 -- 172.21.15.16:0/4093707625 wait complete. 2023-09-30T22:30:58.089 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:30:58.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.113+0000 7faa21f57700 1 -- 172.21.15.16:0/3593916803 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7faa1c1324e0 con 0x7faa1c09f410 2023-09-30T22:30:58.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.113+0000 7faa18ff9700 1 -- 172.21.15.16:0/3593916803 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7faa0402b070 con 0x7faa1c09f410 2023-09-30T22:30:58.114 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:30:58.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.115+0000 7faa21f57700 1 -- 172.21.15.16:0/3593916803 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faa1c09f410 msgr2=0x7faa1c11cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:58.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.115+0000 7faa21f57700 1 --2- 172.21.15.16:0/3593916803 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faa1c09f410 0x7faa1c11cb30 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7faa04010cc0 tx=0x7faa0400b140 comp rx=0 tx=0).stop 2023-09-30T22:30:58.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.115+0000 7faa21f57700 1 -- 172.21.15.16:0/3593916803 shutdown_connections 2023-09-30T22:30:58.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.115+0000 7faa21f57700 1 --2- 172.21.15.16:0/3593916803 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faa1c136fe0 0x7faa1c11d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:58.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.115+0000 7faa21f57700 1 --2- 172.21.15.16:0/3593916803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa1c09e930 0x7faa1c11c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:58.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.115+0000 7faa21f57700 1 --2- 172.21.15.16:0/3593916803 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faa1c09f410 0x7faa1c11cb30 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:58.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.115+0000 7faa21f57700 1 -- 172.21.15.16:0/3593916803 >> 172.21.15.16:0/3593916803 conn(0x7faa1c0a4210 msgr2=0x7faa1c0a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:58.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.116+0000 7faa21f57700 1 -- 172.21.15.16:0/3593916803 shutdown_connections 2023-09-30T22:30:58.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.116+0000 7faa21f57700 1 -- 172.21.15.16:0/3593916803 wait complete. 2023-09-30T22:30:58.130 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:30:58.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.154+0000 7fd31a89f700 1 -- 172.21.15.16:0/3673506837 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd314074ea0 con 0x7fd31409d6f0 2023-09-30T22:30:58.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.155+0000 7fd310ff9700 1 -- 172.21.15.16:0/3673506837 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd304017030 con 0x7fd31409d6f0 2023-09-30T22:30:58.155 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:30:58.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.157+0000 7fd31a89f700 1 -- 172.21.15.16:0/3673506837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd31409d6f0 msgr2=0x7fd31411b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:58.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.157+0000 7fd31a89f700 1 --2- 172.21.15.16:0/3673506837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd31409d6f0 0x7fd31411b2b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd304001eb0 tx=0x7fd30400c800 comp rx=0 tx=0).stop 2023-09-30T22:30:58.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.157+0000 7fd31a89f700 1 -- 172.21.15.16:0/3673506837 shutdown_connections 2023-09-30T22:30:58.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.157+0000 7fd31a89f700 1 --2- 172.21.15.16:0/3673506837 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd314135e50 0x7fd31411bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:58.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.157+0000 7fd31a89f700 1 --2- 172.21.15.16:0/3673506837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd31409d6f0 0x7fd31411b2b0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:58.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.157+0000 7fd31a89f700 1 --2- 172.21.15.16:0/3673506837 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd31409e1d0 0x7fd31411b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:58.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.157+0000 7fd31a89f700 1 -- 172.21.15.16:0/3673506837 >> 172.21.15.16:0/3673506837 conn(0x7fd314066f50 msgr2=0x7fd3140a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:58.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.157+0000 7fd31a89f700 1 -- 172.21.15.16:0/3673506837 shutdown_connections 2023-09-30T22:30:58.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.157+0000 7fd31a89f700 1 -- 172.21.15.16:0/3673506837 wait complete. 2023-09-30T22:30:58.162 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:58.171 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:30:58.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.299+0000 7fbbd335a700 1 Processor -- start 2023-09-30T22:30:58.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.299+0000 7fbbd335a700 1 -- start start 2023-09-30T22:30:58.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.299+0000 7fbbd335a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbbcc09e930 0x7fbbcc09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:58.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.299+0000 7fbbd335a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbbcc09f240 0x7fbbcc1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:58.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.299+0000 7fbbd335a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbbcc09f650 0x7fbbcc131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:58.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.299+0000 7fbbd335a700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbcc070940 con 0x7fbbcc09e930 2023-09-30T22:30:58.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.299+0000 7fbbd335a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbcc06fd60 con 0x7fbbcc09f240 2023-09-30T22:30:58.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.299+0000 7fbbd335a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbbcc06fa80 con 0x7fbbcc09f650 2023-09-30T22:30:58.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.300+0000 7fbbd08f5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbbcc09f240 0x7fbbcc1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:58.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.300+0000 7fbbd18f7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbbcc09f650 0x7fbbcc131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:58.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.300+0000 7fbbd08f5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbbcc09f240 0x7fbbcc1317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38298/0 (socket says 172.21.15.16:38298) 2023-09-30T22:30:58.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.300+0000 7fbbd18f7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbbcc09f650 0x7fbbcc131d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:57758/0 (socket says 172.21.15.16:57758) 2023-09-30T22:30:58.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.300+0000 7fbbd10f6700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbbcc09e930 0x7fbbcc09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:58.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.300+0000 7fbbd08f5700 1 -- 172.21.15.16:0/1050884907 learned_addr learned my addr 172.21.15.16:0/1050884907 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:58.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.300+0000 7fbbd08f5700 1 -- 172.21.15.16:0/1050884907 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbbcc09f650 msgr2=0x7fbbcc131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:58.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.300+0000 7fbbd08f5700 1 --2- 172.21.15.16:0/1050884907 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbbcc09f650 0x7fbbcc131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:58.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.300+0000 7fbbd08f5700 1 -- 172.21.15.16:0/1050884907 >> v2:172.21.15.16:3300/0 conn(0x7fbbcc09e930 msgr2=0x7fbbcc09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:58.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.300+0000 7fbbd08f5700 1 --2- 172.21.15.16:0/1050884907 >> v2:172.21.15.16:3300/0 conn(0x7fbbcc09e930 0x7fbbcc09ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:58.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.300+0000 7fbbd08f5700 1 -- 172.21.15.16:0/1050884907 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbcc09fb60 con 0x7fbbcc09f240 2023-09-30T22:30:58.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.300+0000 7fbbd08f5700 1 --2- 172.21.15.16:0/1050884907 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbbcc09f240 0x7fbbcc1317f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbbc00098f0 tx=0x7fbbc0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2f6d5e68ad2937c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:58.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.300+0000 7fbbcbfff700 1 -- 172.21.15.16:0/1050884907 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0011830 con 0x7fbbcc09f240 2023-09-30T22:30:58.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.300+0000 7fbbcbfff700 1 -- 172.21.15.16:0/1050884907 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbc0011990 con 0x7fbbcc09f240 2023-09-30T22:30:58.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.300+0000 7fbbcbfff700 1 -- 172.21.15.16:0/1050884907 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0011d20 con 0x7fbbcc09f240 2023-09-30T22:30:58.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.301+0000 7fbbd335a700 1 -- 172.21.15.16:0/1050884907 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbbcc09f240 msgr2=0x7fbbcc1317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:58.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.301+0000 7fbbd335a700 1 --2- 172.21.15.16:0/1050884907 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbbcc09f240 0x7fbbcc1317f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbbc00098f0 tx=0x7fbbc0009c00 comp rx=0 tx=0).stop 2023-09-30T22:30:58.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.301+0000 7fbbd335a700 1 -- 172.21.15.16:0/1050884907 shutdown_connections 2023-09-30T22:30:58.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.301+0000 7fbbd335a700 1 --2- 172.21.15.16:0/1050884907 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbbcc09f650 0x7fbbcc131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:58.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.301+0000 7fbbd335a700 1 --2- 172.21.15.16:0/1050884907 >> v2:172.21.15.16:3300/0 conn(0x7fbbcc09e930 0x7fbbcc09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:58.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.301+0000 7fbbd335a700 1 --2- 172.21.15.16:0/1050884907 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbbcc09f240 0x7fbbcc1317f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:58.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.301+0000 7fbbd335a700 1 -- 172.21.15.16:0/1050884907 >> 172.21.15.16:0/1050884907 conn(0x7fbbcc0a4210 msgr2=0x7fbbcc0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:58.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.301+0000 7fbbd335a700 1 -- 172.21.15.16:0/1050884907 shutdown_connections 2023-09-30T22:30:58.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.301+0000 7fbbd335a700 1 -- 172.21.15.16:0/1050884907 wait complete. 2023-09-30T22:30:58.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.302+0000 7fbbd335a700 1 Processor -- start 2023-09-30T22:30:58.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.302+0000 7fbbd335a700 1 -- start start 2023-09-30T22:30:58.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.303+0000 7fbbd335a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbbcc09e930 0x7fbbcc1a3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:58.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.303+0000 7fbbd335a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbbcc09f240 0x7fbbcc1a4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:58.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.303+0000 7fbbd335a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbcc09f650 0x7fbbcc1a4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:58.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.303+0000 7fbbd335a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbcc1a8160 con 0x7fbbcc09f650 2023-09-30T22:30:58.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.303+0000 7fbbd335a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbcc1a77a0 con 0x7fbbcc09e930 2023-09-30T22:30:58.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.303+0000 7fbbd335a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbbcc1a6ae0 con 0x7fbbcc09f240 2023-09-30T22:30:58.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.303+0000 7fbbd10f6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbbcc09e930 0x7fbbcc1a3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:58.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.303+0000 7fbbd08f5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbbcc09f240 0x7fbbcc1a4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:58.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.303+0000 7fbbd10f6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbbcc09e930 0x7fbbcc1a3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38302/0 (socket says 172.21.15.16:38302) 2023-09-30T22:30:58.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.303+0000 7fbbd18f7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbcc09f650 0x7fbbcc1a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:58.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.303+0000 7fbbd10f6700 1 -- 172.21.15.16:0/3123290082 learned_addr learned my addr 172.21.15.16:0/3123290082 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:58.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.303+0000 7fbbd10f6700 1 -- 172.21.15.16:0/3123290082 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbbcc09f240 msgr2=0x7fbbcc1a4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:58.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.303+0000 7fbbd10f6700 1 --2- 172.21.15.16:0/3123290082 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbbcc09f240 0x7fbbcc1a4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:58.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.303+0000 7fbbd10f6700 1 -- 172.21.15.16:0/3123290082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbcc09f650 msgr2=0x7fbbcc1a4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:58.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.303+0000 7fbbd10f6700 1 --2- 172.21.15.16:0/3123290082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbcc09f650 0x7fbbcc1a4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:58.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.303+0000 7fbbd10f6700 1 -- 172.21.15.16:0/3123290082 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbc0009550 con 0x7fbbcc09e930 2023-09-30T22:30:58.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.303+0000 7fbbd08f5700 1 --2- 172.21.15.16:0/3123290082 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbbcc09f240 0x7fbbcc1a4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:58.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.303+0000 7fbbd18f7700 1 --2- 172.21.15.16:0/3123290082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbcc09f650 0x7fbbcc1a4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:58.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.303+0000 7fbbd10f6700 1 --2- 172.21.15.16:0/3123290082 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbbcc09e930 0x7fbbcc1a3cd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc00e960 tx=0x7fbbbc00ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:58.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.303+0000 7fbbc9ffb700 1 -- 172.21.15.16:0/3123290082 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc013680 con 0x7fbbcc09e930 2023-09-30T22:30:58.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.303+0000 7fbbc9ffb700 1 -- 172.21.15.16:0/3123290082 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbbc0137e0 con 0x7fbbcc09e930 2023-09-30T22:30:58.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.303+0000 7fbbd335a700 1 -- 172.21.15.16:0/3123290082 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbcc0a0190 con 0x7fbbcc09e930 2023-09-30T22:30:58.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.303+0000 7fbbc9ffb700 1 -- 172.21.15.16:0/3123290082 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc013b70 con 0x7fbbcc09e930 2023-09-30T22:30:58.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.304+0000 7fbbd335a700 1 -- 172.21.15.16:0/3123290082 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbcc1a5890 con 0x7fbbcc09e930 2023-09-30T22:30:58.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.304+0000 7fbbc9ffb700 1 -- 172.21.15.16:0/3123290082 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbbbc013cd0 con 0x7fbbcc09e930 2023-09-30T22:30:58.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.305+0000 7fbbc9ffb700 1 -- 172.21.15.16:0/3123290082 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fbbbc03fb50 con 0x7fbbcc09e930 2023-09-30T22:30:58.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.305+0000 7fbbd335a700 1 -- 172.21.15.16:0/3123290082 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbcc1a0b30 con 0x7fbbcc09e930 2023-09-30T22:30:58.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.310+0000 7fbbc9ffb700 1 -- 172.21.15.16:0/3123290082 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbbbc021080 con 0x7fbbcc09e930 2023-09-30T22:30:58.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.515+0000 7fbbd335a700 1 -- 172.21.15.16:0/3123290082 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fbbcc1a7340 con 0x7fbbcc09e930 2023-09-30T22:30:58.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.516+0000 7fbbc9ffb700 1 -- 172.21.15.16:0/3123290082 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbbbc017020 con 0x7fbbcc09e930 2023-09-30T22:30:58.516 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:30:58.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.518+0000 7fbbd335a700 1 -- 172.21.15.16:0/3123290082 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbbcc09e930 msgr2=0x7fbbcc1a3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:58.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.518+0000 7fbbd335a700 1 --2- 172.21.15.16:0/3123290082 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbbcc09e930 0x7fbbcc1a3cd0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc00e960 tx=0x7fbbbc00ec70 comp rx=0 tx=0).stop 2023-09-30T22:30:58.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.518+0000 7fbbd335a700 1 -- 172.21.15.16:0/3123290082 shutdown_connections 2023-09-30T22:30:58.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.518+0000 7fbbd335a700 1 --2- 172.21.15.16:0/3123290082 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbbcc09e930 0x7fbbcc1a3cd0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:58.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.518+0000 7fbbd335a700 1 --2- 172.21.15.16:0/3123290082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbcc09f650 0x7fbbcc1a4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:58.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.518+0000 7fbbd335a700 1 --2- 172.21.15.16:0/3123290082 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbbcc09f240 0x7fbbcc1a4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:58.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.518+0000 7fbbd335a700 1 -- 172.21.15.16:0/3123290082 >> 172.21.15.16:0/3123290082 conn(0x7fbbcc0a4210 msgr2=0x7fbbcc1b8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:58.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.519+0000 7fbbd335a700 1 -- 172.21.15.16:0/3123290082 shutdown_connections 2023-09-30T22:30:58.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:58.519+0000 7fbbd335a700 1 -- 172.21.15.16:0/3123290082 wait complete. 2023-09-30T22:30:58.533 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:30:59.090 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:30:59.130 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:30:59.172 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:30:59.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.240+0000 7f9e60d55700 1 Processor -- start 2023-09-30T22:30:59.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.240+0000 7f9e60d55700 1 -- start start 2023-09-30T22:30:59.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.241+0000 7f9e60d55700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e5c136830 0x7f9e5c138c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:59.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.241+0000 7f9e60d55700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e5c139150 0x7f9e5c13b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:59.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.241+0000 7f9e60d55700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e5c06f720 con 0x7f9e5c1338c0 2023-09-30T22:30:59.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.241+0000 7f9e60d55700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5c070c80 con 0x7f9e5c139150 2023-09-30T22:30:59.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.241+0000 7f9e60d55700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5c06fce0 con 0x7f9e5c136830 2023-09-30T22:30:59.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.241+0000 7f9e59d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e5c136830 0x7f9e5c138c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:59.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.241+0000 7f9e59d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e5c136830 0x7f9e5c138c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:57776/0 (socket says 172.21.15.16:57776) 2023-09-30T22:30:59.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.241+0000 7f9e59d9b700 1 -- 172.21.15.16:0/1841690293 learned_addr learned my addr 172.21.15.16:0/1841690293 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:59.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.241+0000 7f9e5ad9d700 1 --2- 172.21.15.16:0/1841690293 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e5c139150 0x7f9e5c13b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:59.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.241+0000 7f9e59d9b700 1 -- 172.21.15.16:0/1841690293 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e5c139150 msgr2=0x7f9e5c13b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:59.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.241+0000 7f9e59d9b700 1 --2- 172.21.15.16:0/1841690293 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e5c139150 0x7f9e5c13b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.241+0000 7f9e59d9b700 1 -- 172.21.15.16:0/1841690293 >> v1:172.21.15.16:6789/0 conn(0x7f9e5c1338c0 legacy=0x7f9e5c133c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:59.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.242+0000 7f9e59d9b700 1 -- 172.21.15.16:0/1841690293 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e5c1336a0 con 0x7f9e5c136830 2023-09-30T22:30:59.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.242+0000 7f9e5ad9d700 1 --2- 172.21.15.16:0/1841690293 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e5c139150 0x7f9e5c13b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:59.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.242+0000 7f9e59d9b700 1 --2- 172.21.15.16:0/1841690293 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e5c136830 0x7f9e5c138c10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9e50009920 tx=0x7f9e50009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89d377df0d7cee9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:59.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.242+0000 7f9e5959a700 1 -- 172.21.15.16:0/1841690293 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e50011830 con 0x7f9e5c136830 2023-09-30T22:30:59.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.242+0000 7f9e5959a700 1 -- 172.21.15.16:0/1841690293 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e50011990 con 0x7f9e5c136830 2023-09-30T22:30:59.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.242+0000 7f9e5959a700 1 -- 172.21.15.16:0/1841690293 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e50011d20 con 0x7f9e5c136830 2023-09-30T22:30:59.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.243+0000 7f9e60d55700 1 -- 172.21.15.16:0/1841690293 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e5c136830 msgr2=0x7f9e5c138c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:59.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.243+0000 7f9e60d55700 1 --2- 172.21.15.16:0/1841690293 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e5c136830 0x7f9e5c138c10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9e50009920 tx=0x7f9e50009c30 comp rx=0 tx=0).stop 2023-09-30T22:30:59.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.243+0000 7f9e60d55700 1 -- 172.21.15.16:0/1841690293 shutdown_connections 2023-09-30T22:30:59.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.243+0000 7f9e60d55700 1 --2- 172.21.15.16:0/1841690293 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e5c139150 0x7f9e5c13b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.243+0000 7f9e60d55700 1 --2- 172.21.15.16:0/1841690293 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e5c136830 0x7f9e5c138c10 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.243+0000 7f9e60d55700 1 -- 172.21.15.16:0/1841690293 >> 172.21.15.16:0/1841690293 conn(0x7f9e5c0a2aa0 msgr2=0x7f9e5c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:59.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.243+0000 7f9e60d55700 1 -- 172.21.15.16:0/1841690293 shutdown_connections 2023-09-30T22:30:59.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.243+0000 7f9e60d55700 1 -- 172.21.15.16:0/1841690293 wait complete. 2023-09-30T22:30:59.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.244+0000 7f9e60d55700 1 Processor -- start 2023-09-30T22:30:59.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.244+0000 7f9e60d55700 1 -- start start 2023-09-30T22:30:59.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.245+0000 7f9e60d55700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e5c1338c0 0x7f9e5c1c8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:59.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.245+0000 7f9e60d55700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e5c136830 0x7f9e5c1c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:59.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.245+0000 7f9e60d55700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e5c139150 0x7f9e5c118f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:59.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.245+0000 7f9e60d55700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5c11a4d0 con 0x7f9e5c1338c0 2023-09-30T22:30:59.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.245+0000 7f9e60d55700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5c11c6d0 con 0x7f9e5c139150 2023-09-30T22:30:59.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.245+0000 7f9e60d55700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5c1be230 con 0x7f9e5c136830 2023-09-30T22:30:59.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.245+0000 7f9e5a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e5c1338c0 0x7f9e5c1c8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:59.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.245+0000 7f9e59d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e5c136830 0x7f9e5c1c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:59.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.245+0000 7f9e5a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e5c1338c0 0x7f9e5c1c8790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55410/0 (socket says 172.21.15.16:55410) 2023-09-30T22:30:59.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.245+0000 7f9e59d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e5c136830 0x7f9e5c1c8cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:57790/0 (socket says 172.21.15.16:57790) 2023-09-30T22:30:59.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.245+0000 7f9e5a59c700 1 -- 172.21.15.16:0/3681902603 learned_addr learned my addr 172.21.15.16:0/3681902603 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:59.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.245+0000 7f9e5ad9d700 1 --2- 172.21.15.16:0/3681902603 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e5c139150 0x7f9e5c118f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:59.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.245+0000 7f9e59d9b700 1 -- 172.21.15.16:0/3681902603 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e5c139150 msgr2=0x7f9e5c118f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:59.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.245+0000 7f9e59d9b700 1 --2- 172.21.15.16:0/3681902603 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e5c139150 0x7f9e5c118f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.245+0000 7f9e59d9b700 1 -- 172.21.15.16:0/3681902603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e5c1338c0 msgr2=0x7f9e5c1c8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:59.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.245+0000 7f9e59d9b700 1 --2- 172.21.15.16:0/3681902603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e5c1338c0 0x7f9e5c1c8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.245+0000 7f9e59d9b700 1 -- 172.21.15.16:0/3681902603 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e54009a70 con 0x7f9e5c136830 2023-09-30T22:30:59.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.245+0000 7f9e5a59c700 1 --2- 172.21.15.16:0/3681902603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e5c1338c0 0x7f9e5c1c8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:59.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.246+0000 7f9e59d9b700 1 --2- 172.21.15.16:0/3681902603 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e5c136830 0x7f9e5c1c8cd0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c066c70 tx=0x7f9e50004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:59.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.246+0000 7f9e4affd700 1 -- 172.21.15.16:0/3681902603 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e50004450 con 0x7f9e5c136830 2023-09-30T22:30:59.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.246+0000 7f9e4affd700 1 -- 172.21.15.16:0/3681902603 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e5001f070 con 0x7f9e5c136830 2023-09-30T22:30:59.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.246+0000 7f9e60d55700 1 -- 172.21.15.16:0/3681902603 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e50009550 con 0x7f9e5c136830 2023-09-30T22:30:59.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.246+0000 7f9e4affd700 1 -- 172.21.15.16:0/3681902603 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e500038e0 con 0x7f9e5c136830 2023-09-30T22:30:59.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.246+0000 7f9e60d55700 1 -- 172.21.15.16:0/3681902603 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e5c1363f0 con 0x7f9e5c136830 2023-09-30T22:30:59.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.247+0000 7f9e60d55700 1 -- 172.21.15.16:0/3681902603 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e5c11aa20 con 0x7f9e5c136830 2023-09-30T22:30:59.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.247+0000 7f9e4affd700 1 -- 172.21.15.16:0/3681902603 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9e50003a40 con 0x7f9e5c136830 2023-09-30T22:30:59.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.247+0000 7f9e4affd700 1 -- 172.21.15.16:0/3681902603 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f9e50021070 con 0x7f9e5c136830 2023-09-30T22:30:59.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.252+0000 7f9e4affd700 1 -- 172.21.15.16:0/3681902603 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9e500045b0 con 0x7f9e5c136830 2023-09-30T22:30:59.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.279+0000 7f72bb55e700 1 Processor -- start 2023-09-30T22:30:59.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.279+0000 7f72bb55e700 1 -- start start 2023-09-30T22:30:59.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.279+0000 7f72bb55e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72b409f410 0x7f72b4136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:59.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.279+0000 7f72bb55e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72b4136fe0 0x7f72b41393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:59.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.280+0000 7f72bb55e700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72b4070940 con 0x7f72b409e930 2023-09-30T22:30:59.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.280+0000 7f72bb55e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72b406fd60 con 0x7f72b4136fe0 2023-09-30T22:30:59.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.280+0000 7f72bb55e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f72b406fa80 con 0x7f72b409f410 2023-09-30T22:30:59.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.280+0000 7f72b8af9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72b409f410 0x7f72b4136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:59.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.280+0000 7f72b8af9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72b409f410 0x7f72b4136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:57794/0 (socket says 172.21.15.16:57794) 2023-09-30T22:30:59.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.280+0000 7f72b8af9700 1 -- 172.21.15.16:0/1693853972 learned_addr learned my addr 172.21.15.16:0/1693853972 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:59.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.280+0000 7f72b9afb700 1 --2- 172.21.15.16:0/1693853972 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72b4136fe0 0x7f72b41393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:59.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.280+0000 7f72b8af9700 1 -- 172.21.15.16:0/1693853972 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72b4136fe0 msgr2=0x7f72b41393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:59.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.280+0000 7f72b8af9700 1 --2- 172.21.15.16:0/1693853972 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72b4136fe0 0x7f72b41393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.280+0000 7f72b8af9700 1 -- 172.21.15.16:0/1693853972 >> v1:172.21.15.16:6789/0 conn(0x7f72b409e930 legacy=0x7f72b409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:59.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.280+0000 7f72b8af9700 1 -- 172.21.15.16:0/1693853972 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b409fb60 con 0x7f72b409f410 2023-09-30T22:30:59.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.280+0000 7f72b8af9700 1 --2- 172.21.15.16:0/1693853972 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72b409f410 0x7f72b4136aa0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f72a4009920 tx=0x7f72a4009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cea5004f2adde84a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:59.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.281+0000 7f72abfff700 1 -- 172.21.15.16:0/1693853972 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a4011830 con 0x7f72b409f410 2023-09-30T22:30:59.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.281+0000 7f72abfff700 1 -- 172.21.15.16:0/1693853972 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72a4011990 con 0x7f72b409f410 2023-09-30T22:30:59.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.281+0000 7f72abfff700 1 -- 172.21.15.16:0/1693853972 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a4011d20 con 0x7f72b409f410 2023-09-30T22:30:59.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.281+0000 7f72bb55e700 1 -- 172.21.15.16:0/1693853972 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72b409f410 msgr2=0x7f72b4136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:59.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.281+0000 7f72bb55e700 1 --2- 172.21.15.16:0/1693853972 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72b409f410 0x7f72b4136aa0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f72a4009920 tx=0x7f72a4009c30 comp rx=0 tx=0).stop 2023-09-30T22:30:59.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.281+0000 7f72bb55e700 1 -- 172.21.15.16:0/1693853972 shutdown_connections 2023-09-30T22:30:59.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.281+0000 7f72bb55e700 1 --2- 172.21.15.16:0/1693853972 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72b4136fe0 0x7f72b41393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.281+0000 7f72bb55e700 1 --2- 172.21.15.16:0/1693853972 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72b409f410 0x7f72b4136aa0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.281+0000 7f72bb55e700 1 -- 172.21.15.16:0/1693853972 >> 172.21.15.16:0/1693853972 conn(0x7f72b40a4210 msgr2=0x7f72b40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:59.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.282+0000 7f72bb55e700 1 -- 172.21.15.16:0/1693853972 shutdown_connections 2023-09-30T22:30:59.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.282+0000 7f72bb55e700 1 -- 172.21.15.16:0/1693853972 wait complete. 2023-09-30T22:30:59.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.283+0000 7f72bb55e700 1 Processor -- start 2023-09-30T22:30:59.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.283+0000 7f72bb55e700 1 -- start start 2023-09-30T22:30:59.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.283+0000 7f72bb55e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72b409e930 0x7f72b41c6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:59.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.283+0000 7f72bb55e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72b409f410 0x7f72b41c69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:59.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.283+0000 7f72bb55e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72b4136fe0 0x7f72b411c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:59.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.283+0000 7f72bb55e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b411ff60 con 0x7f72b4136fe0 2023-09-30T22:30:59.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.283+0000 7f72bb55e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72b411e250 con 0x7f72b409e930 2023-09-30T22:30:59.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.283+0000 7f72bb55e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f72b40a3020 con 0x7f72b409f410 2023-09-30T22:30:59.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.283+0000 7f72b8af9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72b409f410 0x7f72b41c69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:59.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.283+0000 7f72b8af9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72b409f410 0x7f72b41c69b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:57798/0 (socket says 172.21.15.16:57798) 2023-09-30T22:30:59.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.283+0000 7f72b9afb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72b4136fe0 0x7f72b411c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:59.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.283+0000 7f72b8af9700 1 -- 172.21.15.16:0/2094139169 learned_addr learned my addr 172.21.15.16:0/2094139169 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:59.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.283+0000 7f72b92fa700 1 --2- 172.21.15.16:0/2094139169 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72b409e930 0x7f72b41c6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:59.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.284+0000 7f72b9afb700 1 -- 172.21.15.16:0/2094139169 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72b409f410 msgr2=0x7f72b41c69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:59.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.284+0000 7f72b9afb700 1 --2- 172.21.15.16:0/2094139169 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72b409f410 0x7f72b41c69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.284+0000 7f72b9afb700 1 -- 172.21.15.16:0/2094139169 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72b409e930 msgr2=0x7f72b41c6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:59.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.284+0000 7f72b9afb700 1 --2- 172.21.15.16:0/2094139169 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72b409e930 0x7f72b41c6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.284+0000 7f72b9afb700 1 -- 172.21.15.16:0/2094139169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72a4009550 con 0x7f72b4136fe0 2023-09-30T22:30:59.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.284+0000 7f72b8af9700 1 --2- 172.21.15.16:0/2094139169 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72b409f410 0x7f72b41c69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:59.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.284+0000 7f72b9afb700 1 --2- 172.21.15.16:0/2094139169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72b4136fe0 0x7f72b411c6e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f72b40a07c0 tx=0x7f72ac00ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:59.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.284+0000 7f72a9ffb700 1 -- 172.21.15.16:0/2094139169 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ac00cc20 con 0x7f72b4136fe0 2023-09-30T22:30:59.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.284+0000 7f72a9ffb700 1 -- 172.21.15.16:0/2094139169 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72ac00f040 con 0x7f72b4136fe0 2023-09-30T22:30:59.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.284+0000 7f72a9ffb700 1 -- 172.21.15.16:0/2094139169 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ac009e60 con 0x7f72b4136fe0 2023-09-30T22:30:59.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.284+0000 7f72bb55e700 1 -- 172.21.15.16:0/2094139169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72b4071d20 con 0x7f72b4136fe0 2023-09-30T22:30:59.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.284+0000 7f72bb55e700 1 -- 172.21.15.16:0/2094139169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72b41208d0 con 0x7f72b4136fe0 2023-09-30T22:30:59.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.285+0000 7f72a9ffb700 1 -- 172.21.15.16:0/2094139169 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f72ac014480 con 0x7f72b4136fe0 2023-09-30T22:30:59.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.286+0000 7f72a9ffb700 1 -- 172.21.15.16:0/2094139169 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f72ac040180 con 0x7f72b4136fe0 2023-09-30T22:30:59.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.286+0000 7f72bb55e700 1 -- 172.21.15.16:0/2094139169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72b4074ea0 con 0x7f72b4136fe0 2023-09-30T22:30:59.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.295+0000 7f72a9ffb700 1 -- 172.21.15.16:0/2094139169 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f72ac018030 con 0x7f72b4136fe0 2023-09-30T22:30:59.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.325+0000 7f448ee81700 1 Processor -- start 2023-09-30T22:30:59.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.325+0000 7f448ee81700 1 -- start start 2023-09-30T22:30:59.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.325+0000 7f448ee81700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f448809e930 0x7f448809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:59.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.325+0000 7f448ee81700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f448809f240 0x7f44881317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:59.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.325+0000 7f448ee81700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f448809f650 0x7f4488131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:59.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.325+0000 7f448ee81700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4488070940 con 0x7f448809e930 2023-09-30T22:30:59.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.325+0000 7f448ee81700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f448806fd60 con 0x7f448809f650 2023-09-30T22:30:59.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.326+0000 7f448ee81700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f448806fa80 con 0x7f448809f240 2023-09-30T22:30:59.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.326+0000 7f4487fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f448809f240 0x7f44881317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:59.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.326+0000 7f448cc1d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f448809e930 0x7f448809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:59.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.326+0000 7f4487fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f448809f240 0x7f44881317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:57814/0 (socket says 172.21.15.16:57814) 2023-09-30T22:30:59.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.326+0000 7f448cc1d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f448809e930 0x7f448809ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55424/0 (socket says 172.21.15.16:55424) 2023-09-30T22:30:59.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.326+0000 7f4487fff700 1 -- 172.21.15.16:0/2253451569 learned_addr learned my addr 172.21.15.16:0/2253451569 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:59.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.326+0000 7f448d41e700 1 --2- 172.21.15.16:0/2253451569 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f448809f650 0x7f4488131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:59.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.326+0000 7f448cc1d700 1 -- 172.21.15.16:0/2253451569 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f448809f240 msgr2=0x7f44881317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:59.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.326+0000 7f448cc1d700 1 --2- 172.21.15.16:0/2253451569 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f448809f240 0x7f44881317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.326+0000 7f448cc1d700 1 -- 172.21.15.16:0/2253451569 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f448809f650 msgr2=0x7f4488131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:59.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.326+0000 7f448cc1d700 1 --2- 172.21.15.16:0/2253451569 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f448809f650 0x7f4488131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.326+0000 7f448cc1d700 1 -- 172.21.15.16:0/2253451569 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f448809fb60 con 0x7f448809e930 2023-09-30T22:30:59.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.326+0000 7f4487fff700 1 --2- 172.21.15.16:0/2253451569 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f448809f240 0x7f44881317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:59.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.326+0000 7f448d41e700 1 --2- 172.21.15.16:0/2253451569 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f448809f650 0x7f4488131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:59.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.326+0000 7f448cc1d700 1 --2- 172.21.15.16:0/2253451569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f448809e930 0x7f448809ed00 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f447c009840 tx=0x7f447c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ac31fefd5bfcaf3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:59.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.326+0000 7f44877fe700 1 -- 172.21.15.16:0/2253451569 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447c011830 con 0x7f448809e930 2023-09-30T22:30:59.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.327+0000 7f44877fe700 1 -- 172.21.15.16:0/2253451569 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f447c011990 con 0x7f448809e930 2023-09-30T22:30:59.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.327+0000 7f44877fe700 1 -- 172.21.15.16:0/2253451569 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447c011d20 con 0x7f448809e930 2023-09-30T22:30:59.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.327+0000 7f448ee81700 1 -- 172.21.15.16:0/2253451569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f448809e930 msgr2=0x7f448809ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:59.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.327+0000 7f448ee81700 1 --2- 172.21.15.16:0/2253451569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f448809e930 0x7f448809ed00 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f447c009840 tx=0x7f447c009b50 comp rx=0 tx=0).stop 2023-09-30T22:30:59.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.327+0000 7f448ee81700 1 -- 172.21.15.16:0/2253451569 shutdown_connections 2023-09-30T22:30:59.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.327+0000 7f448ee81700 1 --2- 172.21.15.16:0/2253451569 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f448809f650 0x7f4488131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.327+0000 7f448ee81700 1 --2- 172.21.15.16:0/2253451569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f448809e930 0x7f448809ed00 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.327+0000 7f448ee81700 1 --2- 172.21.15.16:0/2253451569 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f448809f240 0x7f44881317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.327+0000 7f448ee81700 1 -- 172.21.15.16:0/2253451569 >> 172.21.15.16:0/2253451569 conn(0x7f44880a4210 msgr2=0x7f44880a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:59.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.327+0000 7f448ee81700 1 -- 172.21.15.16:0/2253451569 shutdown_connections 2023-09-30T22:30:59.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.327+0000 7f448ee81700 1 -- 172.21.15.16:0/2253451569 wait complete. 2023-09-30T22:30:59.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.329+0000 7f448ee81700 1 Processor -- start 2023-09-30T22:30:59.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.329+0000 7f448ee81700 1 -- start start 2023-09-30T22:30:59.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.330+0000 7f448ee81700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f448809e930 0x7f448806b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:59.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.330+0000 7f448ee81700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f448809f240 0x7f448806bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:59.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.330+0000 7f448ee81700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f448809f650 0x7f448806c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:59.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.330+0000 7f448ee81700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44881b7320 con 0x7f448809f240 2023-09-30T22:30:59.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.330+0000 7f448ee81700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44881a96f0 con 0x7f448809f650 2023-09-30T22:30:59.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.330+0000 7f448ee81700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f44881a9430 con 0x7f448809e930 2023-09-30T22:30:59.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.330+0000 7f4487fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f448809f240 0x7f448806bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:59.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.330+0000 7f448cc1d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f448809e930 0x7f448806b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:59.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.330+0000 7f4487fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f448809f240 0x7f448806bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55432/0 (socket says 172.21.15.16:55432) 2023-09-30T22:30:59.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.330+0000 7f4487fff700 1 -- 172.21.15.16:0/1241958920 learned_addr learned my addr 172.21.15.16:0/1241958920 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:59.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.330+0000 7f448d41e700 1 --2- 172.21.15.16:0/1241958920 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f448809f650 0x7f448806c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:59.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.330+0000 7f4487fff700 1 -- 172.21.15.16:0/1241958920 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f448809e930 msgr2=0x7f448806b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:59.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.330+0000 7f4487fff700 1 --2- 172.21.15.16:0/1241958920 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f448809e930 0x7f448806b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.330+0000 7f4487fff700 1 -- 172.21.15.16:0/1241958920 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f448809f650 msgr2=0x7f448806c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:59.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.330+0000 7f4487fff700 1 --2- 172.21.15.16:0/1241958920 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f448809f650 0x7f448806c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.330+0000 7f4487fff700 1 -- 172.21.15.16:0/1241958920 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4474009550 con 0x7f448809f240 2023-09-30T22:30:59.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.330+0000 7f448d41e700 1 --2- 172.21.15.16:0/1241958920 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f448809f650 0x7f448806c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:59.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.330+0000 7f448cc1d700 1 --2- 172.21.15.16:0/1241958920 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f448809e930 0x7f448806b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:59.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.331+0000 7f4487fff700 1 --2- 172.21.15.16:0/1241958920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f448809f240 0x7f448806bdc0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f447400ebb0 tx=0x7f447400eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:59.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.331+0000 7f44857fa700 1 -- 172.21.15.16:0/1241958920 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4474009e80 con 0x7f448809f240 2023-09-30T22:30:59.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.331+0000 7f44857fa700 1 -- 172.21.15.16:0/1241958920 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4474004030 con 0x7f448809f240 2023-09-30T22:30:59.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.331+0000 7f448ee81700 1 -- 172.21.15.16:0/1241958920 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4480009a70 con 0x7f448809f240 2023-09-30T22:30:59.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.331+0000 7f448ee81700 1 -- 172.21.15.16:0/1241958920 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f447c0094a0 con 0x7f448809f240 2023-09-30T22:30:59.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.331+0000 7f44857fa700 1 -- 172.21.15.16:0/1241958920 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44740043a0 con 0x7f448809f240 2023-09-30T22:30:59.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.332+0000 7f44857fa700 1 -- 172.21.15.16:0/1241958920 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f447400c850 con 0x7f448809f240 2023-09-30T22:30:59.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.332+0000 7f44857fa700 1 -- 172.21.15.16:0/1241958920 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f447403f540 con 0x7f448809f240 2023-09-30T22:30:59.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.332+0000 7f448ee81700 1 -- 172.21.15.16:0/1241958920 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4488071d20 con 0x7f448809f240 2023-09-30T22:30:59.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.337+0000 7f44857fa700 1 -- 172.21.15.16:0/1241958920 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4474004500 con 0x7f448809f240 2023-09-30T22:30:59.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.462+0000 7f9e60d55700 1 -- 172.21.15.16:0/3681902603 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9e5c11aa20 con 0x7f9e5c136830 2023-09-30T22:30:59.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.463+0000 7f9e4affd700 1 -- 172.21.15.16:0/3681902603 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9e5c11aa20 con 0x7f9e5c136830 2023-09-30T22:30:59.464 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:30:59.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.465+0000 7f9e60d55700 1 -- 172.21.15.16:0/3681902603 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e5c136830 msgr2=0x7f9e5c1c8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:59.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.465+0000 7f9e60d55700 1 --2- 172.21.15.16:0/3681902603 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e5c136830 0x7f9e5c1c8cd0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9e5c066c70 tx=0x7f9e50004750 comp rx=0 tx=0).stop 2023-09-30T22:30:59.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.465+0000 7f9e60d55700 1 -- 172.21.15.16:0/3681902603 shutdown_connections 2023-09-30T22:30:59.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.465+0000 7f9e60d55700 1 --2- 172.21.15.16:0/3681902603 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e5c139150 0x7f9e5c118f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.465+0000 7f9e60d55700 1 --2- 172.21.15.16:0/3681902603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e5c1338c0 0x7f9e5c1c8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.465+0000 7f9e60d55700 1 --2- 172.21.15.16:0/3681902603 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e5c136830 0x7f9e5c1c8cd0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.465+0000 7f9e60d55700 1 -- 172.21.15.16:0/3681902603 >> 172.21.15.16:0/3681902603 conn(0x7f9e5c0a2aa0 msgr2=0x7f9e5c12f040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:59.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.465+0000 7f9e60d55700 1 -- 172.21.15.16:0/3681902603 shutdown_connections 2023-09-30T22:30:59.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.465+0000 7f9e60d55700 1 -- 172.21.15.16:0/3681902603 wait complete. 2023-09-30T22:30:59.480 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:30:59.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.504+0000 7f72bb55e700 1 -- 172.21.15.16:0/2094139169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f72b406b250 con 0x7f72b4136fe0 2023-09-30T22:30:59.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.504+0000 7f72a9ffb700 1 -- 172.21.15.16:0/2094139169 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f72ac02b090 con 0x7f72b4136fe0 2023-09-30T22:30:59.505 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:30:59.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.507+0000 7f72bb55e700 1 -- 172.21.15.16:0/2094139169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72b4136fe0 msgr2=0x7f72b411c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:59.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.507+0000 7f72bb55e700 1 --2- 172.21.15.16:0/2094139169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72b4136fe0 0x7f72b411c6e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f72b40a07c0 tx=0x7f72ac00ef90 comp rx=0 tx=0).stop 2023-09-30T22:30:59.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.507+0000 7f72bb55e700 1 -- 172.21.15.16:0/2094139169 shutdown_connections 2023-09-30T22:30:59.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.507+0000 7f72bb55e700 1 --2- 172.21.15.16:0/2094139169 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72b409e930 0x7f72b41c6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.507+0000 7f72bb55e700 1 --2- 172.21.15.16:0/2094139169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72b4136fe0 0x7f72b411c6e0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.507+0000 7f72bb55e700 1 --2- 172.21.15.16:0/2094139169 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72b409f410 0x7f72b41c69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.507+0000 7f72bb55e700 1 -- 172.21.15.16:0/2094139169 >> 172.21.15.16:0/2094139169 conn(0x7f72b40a4210 msgr2=0x7f72b40a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:59.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.507+0000 7f72bb55e700 1 -- 172.21.15.16:0/2094139169 shutdown_connections 2023-09-30T22:30:59.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.507+0000 7f72bb55e700 1 -- 172.21.15.16:0/2094139169 wait complete. 2023-09-30T22:30:59.521 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:30:59.534 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:30:59.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.545+0000 7f448ee81700 1 -- 172.21.15.16:0/1241958920 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4488074ea0 con 0x7f448809f240 2023-09-30T22:30:59.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.545+0000 7f44857fa700 1 -- 172.21.15.16:0/1241958920 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4474019300 con 0x7f448809f240 2023-09-30T22:30:59.546 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:30:59.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.547+0000 7f448ee81700 1 -- 172.21.15.16:0/1241958920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f448809f240 msgr2=0x7f448806bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:59.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.547+0000 7f448ee81700 1 --2- 172.21.15.16:0/1241958920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f448809f240 0x7f448806bdc0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f447400ebb0 tx=0x7f447400eec0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.548+0000 7f448ee81700 1 -- 172.21.15.16:0/1241958920 shutdown_connections 2023-09-30T22:30:59.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.548+0000 7f448ee81700 1 --2- 172.21.15.16:0/1241958920 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f448809f650 0x7f448806c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.548+0000 7f448ee81700 1 --2- 172.21.15.16:0/1241958920 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f448809e930 0x7f448806b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.548+0000 7f448ee81700 1 --2- 172.21.15.16:0/1241958920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f448809f240 0x7f448806bdc0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.548+0000 7f448ee81700 1 -- 172.21.15.16:0/1241958920 >> 172.21.15.16:0/1241958920 conn(0x7f44880a4210 msgr2=0x7f44881b8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:59.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.548+0000 7f448ee81700 1 -- 172.21.15.16:0/1241958920 shutdown_connections 2023-09-30T22:30:59.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.548+0000 7f448ee81700 1 -- 172.21.15.16:0/1241958920 wait complete. 2023-09-30T22:30:59.562 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:30:59.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.681+0000 7fb73fd7b700 1 Processor -- start 2023-09-30T22:30:59.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.681+0000 7fb73fd7b700 1 -- start start 2023-09-30T22:30:59.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.681+0000 7fb73fd7b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb73809e1d0 0x7fb738138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:59.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.681+0000 7fb73fd7b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb738139170 0x7fb73813b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:59.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.681+0000 7fb73fd7b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb73809d2f0 con 0x7fb73809d6f0 2023-09-30T22:30:59.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.681+0000 7fb73fd7b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb73809b0f0 con 0x7fb73809e1d0 2023-09-30T22:30:59.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.681+0000 7fb73fd7b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb73809a810 con 0x7fb738139170 2023-09-30T22:30:59.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.682+0000 7fb73d316700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb73809e1d0 0x7fb738138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:59.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.682+0000 7fb73e318700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb738139170 0x7fb73813b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:59.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.682+0000 7fb73d316700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb73809e1d0 0x7fb738138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38344/0 (socket says 172.21.15.16:38344) 2023-09-30T22:30:59.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.682+0000 7fb73d316700 1 -- 172.21.15.16:0/3443759727 learned_addr learned my addr 172.21.15.16:0/3443759727 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:59.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.682+0000 7fb73db17700 1 --1- 172.21.15.16:0/3443759727 >> v1:172.21.15.16:6789/0 conn(0x7fb73809d6f0 0x7fb73809dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38622/0 (socket says 172.21.15.16:38622) 2023-09-30T22:30:59.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.682+0000 7fb73d316700 1 -- 172.21.15.16:0/3443759727 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb738139170 msgr2=0x7fb73813b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:30:59.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.682+0000 7fb73d316700 1 --2- 172.21.15.16:0/3443759727 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb738139170 0x7fb73813b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.682+0000 7fb73d316700 1 -- 172.21.15.16:0/3443759727 >> v1:172.21.15.16:6789/0 conn(0x7fb73809d6f0 legacy=0x7fb73809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:59.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.682+0000 7fb73cb15700 1 -- 172.21.15.16:0/3443759727 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 318418417 0 0) 0x7fb73809d2f0 con 0x7fb73809d6f0 2023-09-30T22:30:59.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.682+0000 7fb73d316700 1 -- 172.21.15.16:0/3443759727 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb73809a470 con 0x7fb73809e1d0 2023-09-30T22:30:59.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.682+0000 7fb73e318700 1 --2- 172.21.15.16:0/3443759727 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb738139170 0x7fb73813b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:59.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.682+0000 7fb73d316700 1 --2- 172.21.15.16:0/3443759727 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb73809e1d0 0x7fb738138c30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb728009950 tx=0x7fb728009c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6e1d7b7ceadf9e9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:59.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.682+0000 7fb73cb15700 1 -- 172.21.15.16:0/3443759727 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb728011830 con 0x7fb73809e1d0 2023-09-30T22:30:59.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.682+0000 7fb73cb15700 1 -- 172.21.15.16:0/3443759727 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb728011990 con 0x7fb73809e1d0 2023-09-30T22:30:59.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.682+0000 7fb73cb15700 1 -- 172.21.15.16:0/3443759727 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb728011d40 con 0x7fb73809e1d0 2023-09-30T22:30:59.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.683+0000 7fb73fd7b700 1 -- 172.21.15.16:0/3443759727 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb73809e1d0 msgr2=0x7fb738138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:59.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.683+0000 7fb73fd7b700 1 --2- 172.21.15.16:0/3443759727 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb73809e1d0 0x7fb738138c30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb728009950 tx=0x7fb728009c60 comp rx=0 tx=0).stop 2023-09-30T22:30:59.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.683+0000 7fb73fd7b700 1 -- 172.21.15.16:0/3443759727 shutdown_connections 2023-09-30T22:30:59.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.683+0000 7fb73fd7b700 1 --2- 172.21.15.16:0/3443759727 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb738139170 0x7fb73813b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.683+0000 7fb73fd7b700 1 --2- 172.21.15.16:0/3443759727 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb73809e1d0 0x7fb738138c30 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.683+0000 7fb73fd7b700 1 -- 172.21.15.16:0/3443759727 >> 172.21.15.16:0/3443759727 conn(0x7fb738066f50 msgr2=0x7fb73809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:59.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.683+0000 7fb73fd7b700 1 -- 172.21.15.16:0/3443759727 shutdown_connections 2023-09-30T22:30:59.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.683+0000 7fb73fd7b700 1 -- 172.21.15.16:0/3443759727 wait complete. 2023-09-30T22:30:59.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.684+0000 7fb73fd7b700 1 Processor -- start 2023-09-30T22:30:59.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.684+0000 7fb73fd7b700 1 -- start start 2023-09-30T22:30:59.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.685+0000 7fb73fd7b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb73809d6f0 0x7fb738121840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:59.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.685+0000 7fb73fd7b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb73809e1d0 0x7fb738121d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:59.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.685+0000 7fb73fd7b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb738139170 0x7fb7381222c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:30:59.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.685+0000 7fb73fd7b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb738123ab0 con 0x7fb738139170 2023-09-30T22:30:59.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.685+0000 7fb73fd7b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb738137150 con 0x7fb73809d6f0 2023-09-30T22:30:59.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.685+0000 7fb73fd7b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb738125360 con 0x7fb73809e1d0 2023-09-30T22:30:59.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.685+0000 7fb73e318700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb738139170 0x7fb7381222c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:59.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.685+0000 7fb73d316700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb73809e1d0 0x7fb738121d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:59.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.685+0000 7fb73e318700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb738139170 0x7fb7381222c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55446/0 (socket says 172.21.15.16:55446) 2023-09-30T22:30:59.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.685+0000 7fb73d316700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb73809e1d0 0x7fb738121d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:57840/0 (socket says 172.21.15.16:57840) 2023-09-30T22:30:59.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.685+0000 7fb73e318700 1 -- 172.21.15.16:0/1654541931 learned_addr learned my addr 172.21.15.16:0/1654541931 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:30:59.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.685+0000 7fb73db17700 1 --2- 172.21.15.16:0/1654541931 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb73809d6f0 0x7fb738121840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:30:59.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.685+0000 7fb73d316700 1 -- 172.21.15.16:0/1654541931 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb73809d6f0 msgr2=0x7fb738121840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:59.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.685+0000 7fb73d316700 1 --2- 172.21.15.16:0/1654541931 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb73809d6f0 0x7fb738121840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.685+0000 7fb73d316700 1 -- 172.21.15.16:0/1654541931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb738139170 msgr2=0x7fb7381222c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:59.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.685+0000 7fb73d316700 1 --2- 172.21.15.16:0/1654541931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb738139170 0x7fb7381222c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.685+0000 7fb73d316700 1 -- 172.21.15.16:0/1654541931 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb730009a70 con 0x7fb73809e1d0 2023-09-30T22:30:59.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.685+0000 7fb73e318700 1 --2- 172.21.15.16:0/1654541931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb738139170 0x7fb7381222c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:30:59.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.685+0000 7fb73d316700 1 --2- 172.21.15.16:0/1654541931 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb73809e1d0 0x7fb738121d80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb728016000 tx=0x7fb728004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:30:59.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.685+0000 7fb72e7fc700 1 -- 172.21.15.16:0/1654541931 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72801a840 con 0x7fb73809e1d0 2023-09-30T22:30:59.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.686+0000 7fb72e7fc700 1 -- 172.21.15.16:0/1654541931 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb72801f070 con 0x7fb73809e1d0 2023-09-30T22:30:59.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.686+0000 7fb73fd7b700 1 -- 172.21.15.16:0/1654541931 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb728009550 con 0x7fb73809e1d0 2023-09-30T22:30:59.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.686+0000 7fb72e7fc700 1 -- 172.21.15.16:0/1654541931 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72801ac00 con 0x7fb73809e1d0 2023-09-30T22:30:59.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.686+0000 7fb73fd7b700 1 -- 172.21.15.16:0/1654541931 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb738124dc0 con 0x7fb73809e1d0 2023-09-30T22:30:59.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.686+0000 7fb72e7fc700 1 -- 172.21.15.16:0/1654541931 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb72801b430 con 0x7fb73809e1d0 2023-09-30T22:30:59.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.686+0000 7fb73fd7b700 1 -- 172.21.15.16:0/1654541931 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb738074ea0 con 0x7fb73809e1d0 2023-09-30T22:30:59.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.687+0000 7fb72e7fc700 1 -- 172.21.15.16:0/1654541931 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fb728021070 con 0x7fb73809e1d0 2023-09-30T22:30:59.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.691+0000 7fb72e7fc700 1 -- 172.21.15.16:0/1654541931 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb72806e750 con 0x7fb73809e1d0 2023-09-30T22:30:59.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.895+0000 7fb73fd7b700 1 -- 172.21.15.16:0/1654541931 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb738123c40 con 0x7fb73809e1d0 2023-09-30T22:30:59.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.895+0000 7fb72e7fc700 1 -- 172.21.15.16:0/1654541931 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb728015030 con 0x7fb73809e1d0 2023-09-30T22:30:59.896 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:30:59.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.897+0000 7fb73fd7b700 1 -- 172.21.15.16:0/1654541931 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb73809e1d0 msgr2=0x7fb738121d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:30:59.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.897+0000 7fb73fd7b700 1 --2- 172.21.15.16:0/1654541931 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb73809e1d0 0x7fb738121d80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb728016000 tx=0x7fb728004750 comp rx=0 tx=0).stop 2023-09-30T22:30:59.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.897+0000 7fb73fd7b700 1 -- 172.21.15.16:0/1654541931 shutdown_connections 2023-09-30T22:30:59.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.898+0000 7fb73fd7b700 1 --2- 172.21.15.16:0/1654541931 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb73809d6f0 0x7fb738121840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.898+0000 7fb73fd7b700 1 --2- 172.21.15.16:0/1654541931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb738139170 0x7fb7381222c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.898+0000 7fb73fd7b700 1 --2- 172.21.15.16:0/1654541931 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb73809e1d0 0x7fb738121d80 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:30:59.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.898+0000 7fb73fd7b700 1 -- 172.21.15.16:0/1654541931 >> 172.21.15.16:0/1654541931 conn(0x7fb738066f50 msgr2=0x7fb738070d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:30:59.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.898+0000 7fb73fd7b700 1 -- 172.21.15.16:0/1654541931 shutdown_connections 2023-09-30T22:30:59.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:30:59.898+0000 7fb73fd7b700 1 -- 172.21.15.16:0/1654541931 wait complete. 2023-09-30T22:30:59.911 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:31:00.481 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:00.522 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:00.563 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:00.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.633+0000 7ff7f9166700 1 Processor -- start 2023-09-30T22:31:00.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.633+0000 7ff7f9166700 1 -- start start 2023-09-30T22:31:00.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.633+0000 7ff7f9166700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff7f4131810 0x7ff7f4131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:00.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.633+0000 7ff7f9166700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff7f409a580 0x7ff7f4139660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:00.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.633+0000 7ff7f9166700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7f41313c0 con 0x7ff7f4099a40 2023-09-30T22:31:00.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.633+0000 7ff7f9166700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7f409b580 con 0x7ff7f409a580 2023-09-30T22:31:00.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.633+0000 7ff7f9166700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f406e8c0 con 0x7ff7f4131810 2023-09-30T22:31:00.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.634+0000 7ff7f359e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff7f409a580 0x7ff7f4139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:00.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.634+0000 7ff7f259c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff7f4099a40 0x7ff7f4099e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38636/0 (socket says 172.21.15.16:38636) 2023-09-30T22:31:00.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.634+0000 7ff7f259c700 1 -- 172.21.15.16:0/3936336611 learned_addr learned my addr 172.21.15.16:0/3936336611 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:31:00.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.634+0000 7ff7f2d9d700 1 --2- 172.21.15.16:0/3936336611 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff7f4131810 0x7ff7f4131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:00.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.634+0000 7ff7f1d9b700 1 -- 172.21.15.16:0/3936336611 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3666608776 0 0) 0x7ff7f41313c0 con 0x7ff7f4099a40 2023-09-30T22:31:00.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.634+0000 7ff7f1d9b700 1 -- 172.21.15.16:0/3936336611 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7dc0035b0 con 0x7ff7f4099a40 2023-09-30T22:31:00.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.634+0000 7ff7f359e700 1 -- 172.21.15.16:0/3936336611 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff7f4131810 msgr2=0x7ff7f4131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:00.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.634+0000 7ff7f359e700 1 --2- 172.21.15.16:0/3936336611 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff7f4131810 0x7ff7f4131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:00.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.634+0000 7ff7f359e700 1 -- 172.21.15.16:0/3936336611 >> v1:172.21.15.16:6789/0 conn(0x7ff7f4099a40 legacy=0x7ff7f4099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:00.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.634+0000 7ff7f359e700 1 -- 172.21.15.16:0/3936336611 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f4071d20 con 0x7ff7f409a580 2023-09-30T22:31:00.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.634+0000 7ff7f2d9d700 1 --2- 172.21.15.16:0/3936336611 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff7f4131810 0x7ff7f4131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:00.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.634+0000 7ff7f359e700 1 --2- 172.21.15.16:0/3936336611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7f409a580 0x7ff7f4139660 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec009d90 tx=0x7ff7ec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4199c2db2529ae31 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:00.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.634+0000 7ff7f1d9b700 1 -- 172.21.15.16:0/3936336611 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec00e070 con 0x7ff7f409a580 2023-09-30T22:31:00.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.634+0000 7ff7f1d9b700 1 -- 172.21.15.16:0/3936336611 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7ec00bb20 con 0x7ff7f409a580 2023-09-30T22:31:00.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.634+0000 7ff7f1d9b700 1 -- 172.21.15.16:0/3936336611 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec00be90 con 0x7ff7f409a580 2023-09-30T22:31:00.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.635+0000 7ff7f9166700 1 -- 172.21.15.16:0/3936336611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7f409a580 msgr2=0x7ff7f4139660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:00.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.635+0000 7ff7f9166700 1 --2- 172.21.15.16:0/3936336611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7f409a580 0x7ff7f4139660 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec009d90 tx=0x7ff7ec00b330 comp rx=0 tx=0).stop 2023-09-30T22:31:00.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.635+0000 7ff7f9166700 1 -- 172.21.15.16:0/3936336611 shutdown_connections 2023-09-30T22:31:00.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.635+0000 7ff7f9166700 1 --2- 172.21.15.16:0/3936336611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7f409a580 0x7ff7f4139660 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:00.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.635+0000 7ff7f9166700 1 --2- 172.21.15.16:0/3936336611 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff7f4131810 0x7ff7f4131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:00.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.635+0000 7ff7f9166700 1 -- 172.21.15.16:0/3936336611 >> 172.21.15.16:0/3936336611 conn(0x7ff7f40a2aa0 msgr2=0x7ff7f40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:00.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.636+0000 7ff7f9166700 1 -- 172.21.15.16:0/3936336611 shutdown_connections 2023-09-30T22:31:00.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.636+0000 7ff7f9166700 1 -- 172.21.15.16:0/3936336611 wait complete. 2023-09-30T22:31:00.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.637+0000 7ff7f9166700 1 Processor -- start 2023-09-30T22:31:00.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.638+0000 7ff7f9166700 1 -- start start 2023-09-30T22:31:00.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.638+0000 7ff7f9166700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff7f4099a40 0x7ff7f406f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:00.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.638+0000 7ff7f9166700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7f409a580 0x7ff7f406fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:00.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.638+0000 7ff7f9166700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff7f4131810 0x7ff7f4070090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:00.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.638+0000 7ff7f9166700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f4070ae0 con 0x7ff7f409a580 2023-09-30T22:31:00.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.638+0000 7ff7f9166700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f41212e0 con 0x7ff7f4099a40 2023-09-30T22:31:00.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.638+0000 7ff7f9166700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f4120280 con 0x7ff7f4131810 2023-09-30T22:31:00.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.638+0000 7ff7f2d9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff7f4099a40 0x7ff7f406f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:00.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.639+0000 7ff7f2d9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff7f4099a40 0x7ff7f406f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38386/0 (socket says 172.21.15.16:38386) 2023-09-30T22:31:00.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.639+0000 7ff7f2d9d700 1 -- 172.21.15.16:0/2668409080 learned_addr learned my addr 172.21.15.16:0/2668409080 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:00.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.639+0000 7ff7f359e700 1 --2- 172.21.15.16:0/2668409080 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff7f4131810 0x7ff7f4070090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:00.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.639+0000 7ff7f259c700 1 --2- 172.21.15.16:0/2668409080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7f409a580 0x7ff7f406fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:00.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.639+0000 7ff7f2d9d700 1 -- 172.21.15.16:0/2668409080 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff7f4131810 msgr2=0x7ff7f4070090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:00.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.639+0000 7ff7f2d9d700 1 --2- 172.21.15.16:0/2668409080 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff7f4131810 0x7ff7f4070090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:00.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.639+0000 7ff7f2d9d700 1 -- 172.21.15.16:0/2668409080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7f409a580 msgr2=0x7ff7f406fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:00.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.639+0000 7ff7f2d9d700 1 --2- 172.21.15.16:0/2668409080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7f409a580 0x7ff7f406fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:00.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.639+0000 7ff7f2d9d700 1 -- 172.21.15.16:0/2668409080 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e40094a0 con 0x7ff7f4099a40 2023-09-30T22:31:00.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.639+0000 7ff7f359e700 1 --2- 172.21.15.16:0/2668409080 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff7f4131810 0x7ff7f4070090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:00.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.639+0000 7ff7f2d9d700 1 --2- 172.21.15.16:0/2668409080 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff7f4099a40 0x7ff7f406f610 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff7e400ec10 tx=0x7ff7e400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:00.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.639+0000 7ff7db7fe700 1 -- 172.21.15.16:0/2668409080 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e400c850 con 0x7ff7f4099a40 2023-09-30T22:31:00.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.639+0000 7ff7db7fe700 1 -- 172.21.15.16:0/2668409080 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7e400c9b0 con 0x7ff7f4099a40 2023-09-30T22:31:00.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.639+0000 7ff7f9166700 1 -- 172.21.15.16:0/2668409080 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7ec009a70 con 0x7ff7f4099a40 2023-09-30T22:31:00.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.639+0000 7ff7db7fe700 1 -- 172.21.15.16:0/2668409080 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e400cd20 con 0x7ff7f4099a40 2023-09-30T22:31:00.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.640+0000 7ff7f9166700 1 -- 172.21.15.16:0/2668409080 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7f40a19f0 con 0x7ff7f4099a40 2023-09-30T22:31:00.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.640+0000 7ff7db7fe700 1 -- 172.21.15.16:0/2668409080 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff7e4013470 con 0x7ff7f4099a40 2023-09-30T22:31:00.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.641+0000 7ff7f9166700 1 -- 172.21.15.16:0/2668409080 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7f4120620 con 0x7ff7f4099a40 2023-09-30T22:31:00.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.641+0000 7ff7db7fe700 1 -- 172.21.15.16:0/2668409080 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7ff7e40473b0 con 0x7ff7f4099a40 2023-09-30T22:31:00.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.646+0000 7ff7db7fe700 1 -- 172.21.15.16:0/2668409080 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff7e4013720 con 0x7ff7f4099a40 2023-09-30T22:31:00.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.674+0000 7ff68f1da700 1 Processor -- start 2023-09-30T22:31:00.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.674+0000 7ff68f1da700 1 -- start start 2023-09-30T22:31:00.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.674+0000 7ff68f1da700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff68809d6f0 0x7ff68809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:00.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.674+0000 7ff68f1da700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff68809e410 0x7ff688130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:00.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.674+0000 7ff68f1da700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff68809f130 con 0x7ff68809e000 2023-09-30T22:31:00.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.674+0000 7ff68f1da700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff68809f520 con 0x7ff68809e410 2023-09-30T22:31:00.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.674+0000 7ff68f1da700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff68809e930 con 0x7ff68809d6f0 2023-09-30T22:31:00.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.675+0000 7ff68d777700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff68809e410 0x7ff688130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:00.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.675+0000 7ff68cf76700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff68809d6f0 0x7ff68809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:00.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.675+0000 7ff687fff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff68809e000 0x7ff688130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38638/0 (socket says 172.21.15.16:38638) 2023-09-30T22:31:00.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.675+0000 7ff687fff700 1 -- 172.21.15.16:0/2197125096 learned_addr learned my addr 172.21.15.16:0/2197125096 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:31:00.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.675+0000 7ff68d777700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff68809e410 0x7ff688130d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55478/0 (socket says 172.21.15.16:55478) 2023-09-30T22:31:00.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.675+0000 7ff68cf76700 1 -- 172.21.15.16:0/2197125096 >> v2:172.21.15.16:3300/0 conn(0x7ff68809e410 msgr2=0x7ff688130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:00.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.675+0000 7ff68cf76700 1 --2- 172.21.15.16:0/2197125096 >> v2:172.21.15.16:3300/0 conn(0x7ff68809e410 0x7ff688130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:00.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.675+0000 7ff68cf76700 1 -- 172.21.15.16:0/2197125096 >> v1:172.21.15.16:6789/0 conn(0x7ff68809e000 legacy=0x7ff688130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:00.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.675+0000 7ff68cf76700 1 -- 172.21.15.16:0/2197125096 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff68809b2d0 con 0x7ff68809d6f0 2023-09-30T22:31:00.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.675+0000 7ff68d777700 1 --2- 172.21.15.16:0/2197125096 >> v2:172.21.15.16:3300/0 conn(0x7ff68809e410 0x7ff688130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:00.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.675+0000 7ff68cf76700 1 --2- 172.21.15.16:0/2197125096 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff68809d6f0 0x7ff68809dac0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff67c00dc60 tx=0x7ff67c00df70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d206b8e626928c68 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:00.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.675+0000 7ff6877fe700 1 -- 172.21.15.16:0/2197125096 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67c012070 con 0x7ff68809d6f0 2023-09-30T22:31:00.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.675+0000 7ff6877fe700 1 -- 172.21.15.16:0/2197125096 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff67c00d2e0 con 0x7ff68809d6f0 2023-09-30T22:31:00.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.675+0000 7ff6877fe700 1 -- 172.21.15.16:0/2197125096 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67c00f900 con 0x7ff68809d6f0 2023-09-30T22:31:00.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.676+0000 7ff68f1da700 1 -- 172.21.15.16:0/2197125096 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff68809d6f0 msgr2=0x7ff68809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:00.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.676+0000 7ff68f1da700 1 --2- 172.21.15.16:0/2197125096 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff68809d6f0 0x7ff68809dac0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff67c00dc60 tx=0x7ff67c00df70 comp rx=0 tx=0).stop 2023-09-30T22:31:00.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.676+0000 7ff68f1da700 1 -- 172.21.15.16:0/2197125096 shutdown_connections 2023-09-30T22:31:00.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.676+0000 7ff68f1da700 1 --2- 172.21.15.16:0/2197125096 >> v2:172.21.15.16:3300/0 conn(0x7ff68809e410 0x7ff688130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:00.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.676+0000 7ff68f1da700 1 --2- 172.21.15.16:0/2197125096 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff68809d6f0 0x7ff68809dac0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:00.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.676+0000 7ff68f1da700 1 -- 172.21.15.16:0/2197125096 >> 172.21.15.16:0/2197125096 conn(0x7ff688066f50 msgr2=0x7ff68809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:00.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.676+0000 7ff68f1da700 1 -- 172.21.15.16:0/2197125096 shutdown_connections 2023-09-30T22:31:00.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.676+0000 7ff68f1da700 1 -- 172.21.15.16:0/2197125096 wait complete. 2023-09-30T22:31:00.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.677+0000 7ff68f1da700 1 Processor -- start 2023-09-30T22:31:00.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.677+0000 7ff68f1da700 1 -- start start 2023-09-30T22:31:00.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.678+0000 7ff68f1da700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff68809d6f0 0x7ff68811e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:00.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.678+0000 7ff68f1da700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff68809e000 0x7ff68811e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:00.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.678+0000 7ff68f1da700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff68809e410 0x7ff68811eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:00.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.678+0000 7ff68f1da700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff688121b40 con 0x7ff68809e000 2023-09-30T22:31:00.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.678+0000 7ff68f1da700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff68809c770 con 0x7ff68809d6f0 2023-09-30T22:31:00.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.678+0000 7ff68f1da700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff6880a1e90 con 0x7ff68809e410 2023-09-30T22:31:00.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.678+0000 7ff68cf76700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff68809d6f0 0x7ff68811e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:00.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.678+0000 7ff687fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff68809e000 0x7ff68811e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:00.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.678+0000 7ff68cf76700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff68809d6f0 0x7ff68811e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38412/0 (socket says 172.21.15.16:38412) 2023-09-30T22:31:00.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.678+0000 7ff68cf76700 1 -- 172.21.15.16:0/3242313951 learned_addr learned my addr 172.21.15.16:0/3242313951 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:00.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.679+0000 7ff68cf76700 1 -- 172.21.15.16:0/3242313951 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff68809e410 msgr2=0x7ff68811eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:00.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.679+0000 7ff68cf76700 1 --2- 172.21.15.16:0/3242313951 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff68809e410 0x7ff68811eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:00.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.679+0000 7ff68cf76700 1 -- 172.21.15.16:0/3242313951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff68809e000 msgr2=0x7ff68811e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:00.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.679+0000 7ff68cf76700 1 --2- 172.21.15.16:0/3242313951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff68809e000 0x7ff68811e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:00.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.679+0000 7ff68cf76700 1 -- 172.21.15.16:0/3242313951 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff680009a70 con 0x7ff68809d6f0 2023-09-30T22:31:00.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.679+0000 7ff687fff700 1 --2- 172.21.15.16:0/3242313951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff68809e000 0x7ff68811e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:00.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.679+0000 7ff68cf76700 1 --2- 172.21.15.16:0/3242313951 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff68809d6f0 0x7ff68811e0b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff68809ea70 tx=0x7ff67c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:00.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.679+0000 7ff6857fa700 1 -- 172.21.15.16:0/3242313951 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67c0036a0 con 0x7ff68809d6f0 2023-09-30T22:31:00.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.679+0000 7ff6857fa700 1 -- 172.21.15.16:0/3242313951 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff67c003820 con 0x7ff68809d6f0 2023-09-30T22:31:00.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.679+0000 7ff6857fa700 1 -- 172.21.15.16:0/3242313951 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67c003bd0 con 0x7ff68809d6f0 2023-09-30T22:31:00.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.679+0000 7ff68f1da700 1 -- 172.21.15.16:0/3242313951 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff67c00d940 con 0x7ff68809d6f0 2023-09-30T22:31:00.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.679+0000 7ff68f1da700 1 -- 172.21.15.16:0/3242313951 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6881215a0 con 0x7ff68809d6f0 2023-09-30T22:31:00.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.680+0000 7ff6857fa700 1 -- 172.21.15.16:0/3242313951 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff67c003d30 con 0x7ff68809d6f0 2023-09-30T22:31:00.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.680+0000 7ff6857fa700 1 -- 172.21.15.16:0/3242313951 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7ff67c044bc0 con 0x7ff68809d6f0 2023-09-30T22:31:00.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.681+0000 7ff672ffd700 1 -- 172.21.15.16:0/3242313951 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6600052e0 con 0x7ff68809d6f0 2023-09-30T22:31:00.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.685+0000 7ff6857fa700 1 -- 172.21.15.16:0/3242313951 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff67c01e390 con 0x7ff68809d6f0 2023-09-30T22:31:00.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.713+0000 7fdbd0004700 1 Processor -- start 2023-09-30T22:31:00.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.713+0000 7fdbd0004700 1 -- start start 2023-09-30T22:31:00.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.713+0000 7fdbd0004700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdbc809d630 0x7fdbc80a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:00.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.713+0000 7fdbd0004700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdbc8138d80 0x7fdbc813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:00.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.713+0000 7fdbd0004700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbc80a2a70 con 0x7fdbc80a0a30 2023-09-30T22:31:00.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.713+0000 7fdbd0004700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbc80a01d0 con 0x7fdbc809d630 2023-09-30T22:31:00.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.713+0000 7fdbd0004700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc809e2b0 con 0x7fdbc8138d80 2023-09-30T22:31:00.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.714+0000 7fdbcdda0700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fdbc80a0a30 0x7fdbc80a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38640/0 (socket says 172.21.15.16:38640) 2023-09-30T22:31:00.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.714+0000 7fdbcdda0700 1 -- 172.21.15.16:0/3208308992 learned_addr learned my addr 172.21.15.16:0/3208308992 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:31:00.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.714+0000 7fdbcd59f700 1 --2- 172.21.15.16:0/3208308992 >> v2:172.21.15.16:3300/0 conn(0x7fdbc809d630 0x7fdbc80a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:00.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.714+0000 7fdbce5a1700 1 --2- 172.21.15.16:0/3208308992 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdbc8138d80 0x7fdbc813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:00.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.714+0000 7fdbcd59f700 1 -- 172.21.15.16:0/3208308992 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdbc8138d80 msgr2=0x7fdbc813b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:00.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.714+0000 7fdbcd59f700 1 --2- 172.21.15.16:0/3208308992 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdbc8138d80 0x7fdbc813b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:00.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.714+0000 7fdbcd59f700 1 -- 172.21.15.16:0/3208308992 >> v1:172.21.15.16:6789/0 conn(0x7fdbc80a0a30 legacy=0x7fdbc80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:00.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.714+0000 7fdbccd9e700 1 -- 172.21.15.16:0/3208308992 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 235290223 0 0) 0x7fdbc80a2a70 con 0x7fdbc80a0a30 2023-09-30T22:31:00.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.714+0000 7fdbcd59f700 1 -- 172.21.15.16:0/3208308992 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbc809dc50 con 0x7fdbc809d630 2023-09-30T22:31:00.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.714+0000 7fdbcd59f700 1 --2- 172.21.15.16:0/3208308992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbc809d630 0x7fdbc80a1510 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8009840 tx=0x7fdbb8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a0924a9549c544 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:00.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.715+0000 7fdbccd9e700 1 -- 172.21.15.16:0/3208308992 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb8011830 con 0x7fdbc809d630 2023-09-30T22:31:00.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.715+0000 7fdbccd9e700 1 -- 172.21.15.16:0/3208308992 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbb8011990 con 0x7fdbc809d630 2023-09-30T22:31:00.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.715+0000 7fdbccd9e700 1 -- 172.21.15.16:0/3208308992 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb8011d20 con 0x7fdbc809d630 2023-09-30T22:31:00.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.715+0000 7fdbd0004700 1 -- 172.21.15.16:0/3208308992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbc809d630 msgr2=0x7fdbc80a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:00.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.715+0000 7fdbd0004700 1 --2- 172.21.15.16:0/3208308992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbc809d630 0x7fdbc80a1510 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8009840 tx=0x7fdbb8009b50 comp rx=0 tx=0).stop 2023-09-30T22:31:00.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.715+0000 7fdbd0004700 1 -- 172.21.15.16:0/3208308992 shutdown_connections 2023-09-30T22:31:00.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.715+0000 7fdbd0004700 1 --2- 172.21.15.16:0/3208308992 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdbc8138d80 0x7fdbc813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:00.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.715+0000 7fdbd0004700 1 --2- 172.21.15.16:0/3208308992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbc809d630 0x7fdbc80a1510 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:00.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.715+0000 7fdbd0004700 1 -- 172.21.15.16:0/3208308992 >> 172.21.15.16:0/3208308992 conn(0x7fdbc80a3050 msgr2=0x7fdbc80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:00.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.715+0000 7fdbd0004700 1 -- 172.21.15.16:0/3208308992 shutdown_connections 2023-09-30T22:31:00.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.715+0000 7fdbd0004700 1 -- 172.21.15.16:0/3208308992 wait complete. 2023-09-30T22:31:00.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.716+0000 7fdbd0004700 1 Processor -- start 2023-09-30T22:31:00.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.717+0000 7fdbd0004700 1 -- start start 2023-09-30T22:31:00.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.717+0000 7fdbd0004700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbc809d630 0x7fdbc81337a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:00.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.717+0000 7fdbd0004700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdbc80a0a30 0x7fdbc8133ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:00.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.717+0000 7fdbd0004700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdbc8138d80 0x7fdbc8134220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:00.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.717+0000 7fdbd0004700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc8136270 con 0x7fdbc809d630 2023-09-30T22:31:00.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.717+0000 7fdbd0004700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc8136f90 con 0x7fdbc8138d80 2023-09-30T22:31:00.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.717+0000 7fdbd0004700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdbc80a4b10 con 0x7fdbc80a0a30 2023-09-30T22:31:00.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.717+0000 7fdbcdda0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbc809d630 0x7fdbc81337a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:00.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.717+0000 7fdbcdda0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbc809d630 0x7fdbc81337a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55506/0 (socket says 172.21.15.16:55506) 2023-09-30T22:31:00.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.717+0000 7fdbcdda0700 1 -- 172.21.15.16:0/1147010574 learned_addr learned my addr 172.21.15.16:0/1147010574 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:00.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.717+0000 7fdbcd59f700 1 --2- 172.21.15.16:0/1147010574 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdbc80a0a30 0x7fdbc8133ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:00.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.717+0000 7fdbcdda0700 1 -- 172.21.15.16:0/1147010574 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdbc80a0a30 msgr2=0x7fdbc8133ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:00.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.717+0000 7fdbcdda0700 1 --2- 172.21.15.16:0/1147010574 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdbc80a0a30 0x7fdbc8133ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:00.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.717+0000 7fdbce5a1700 1 --2- 172.21.15.16:0/1147010574 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdbc8138d80 0x7fdbc8134220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:00.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.717+0000 7fdbcdda0700 1 -- 172.21.15.16:0/1147010574 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdbc8138d80 msgr2=0x7fdbc8134220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:00.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.717+0000 7fdbcdda0700 1 --2- 172.21.15.16:0/1147010574 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdbc8138d80 0x7fdbc8134220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:00.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.718+0000 7fdbcdda0700 1 -- 172.21.15.16:0/1147010574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb80094a0 con 0x7fdbc809d630 2023-09-30T22:31:00.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.718+0000 7fdbcd59f700 1 --2- 172.21.15.16:0/1147010574 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdbc80a0a30 0x7fdbc8133ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:31:00.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.718+0000 7fdbce5a1700 1 --2- 172.21.15.16:0/1147010574 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdbc8138d80 0x7fdbc8134220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:31:00.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.718+0000 7fdbcdda0700 1 --2- 172.21.15.16:0/1147010574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbc809d630 0x7fdbc81337a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fdbc400c520 tx=0x7fdbc400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:00.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.718+0000 7fdbbe7fc700 1 -- 172.21.15.16:0/1147010574 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc4004c80 con 0x7fdbc809d630 2023-09-30T22:31:00.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.718+0000 7fdbbe7fc700 1 -- 172.21.15.16:0/1147010574 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbc4004e00 con 0x7fdbc809d630 2023-09-30T22:31:00.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.718+0000 7fdbd0004700 1 -- 172.21.15.16:0/1147010574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbc80a0410 con 0x7fdbc809d630 2023-09-30T22:31:00.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.718+0000 7fdbbe7fc700 1 -- 172.21.15.16:0/1147010574 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc4007310 con 0x7fdbc809d630 2023-09-30T22:31:00.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.718+0000 7fdbd0004700 1 -- 172.21.15.16:0/1147010574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbc8137270 con 0x7fdbc809d630 2023-09-30T22:31:00.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.719+0000 7fdbbe7fc700 1 -- 172.21.15.16:0/1147010574 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdbc4007470 con 0x7fdbc809d630 2023-09-30T22:31:00.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.719+0000 7fdbbe7fc700 1 -- 172.21.15.16:0/1147010574 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fdbc403f3c0 con 0x7fdbc809d630 2023-09-30T22:31:00.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.719+0000 7fdbd0004700 1 -- 172.21.15.16:0/1147010574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbc8136dd0 con 0x7fdbc809d630 2023-09-30T22:31:00.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.724+0000 7fdbbe7fc700 1 -- 172.21.15.16:0/1147010574 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdbc40123b0 con 0x7fdbc809d630 2023-09-30T22:31:00.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.860+0000 7ff7f9166700 1 -- 172.21.15.16:0/2668409080 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff7f4120a70 con 0x7ff7f4099a40 2023-09-30T22:31:00.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.860+0000 7ff7db7fe700 1 -- 172.21.15.16:0/2668409080 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff7f4120a70 con 0x7ff7f4099a40 2023-09-30T22:31:00.861 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:31:00.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.862+0000 7ff7f9166700 1 -- 172.21.15.16:0/2668409080 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff7f4099a40 msgr2=0x7ff7f406f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:00.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.862+0000 7ff7f9166700 1 --2- 172.21.15.16:0/2668409080 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff7f4099a40 0x7ff7f406f610 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff7e400ec10 tx=0x7ff7e400ef20 comp rx=0 tx=0).stop 2023-09-30T22:31:00.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.862+0000 7ff7f9166700 1 -- 172.21.15.16:0/2668409080 shutdown_connections 2023-09-30T22:31:00.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.862+0000 7ff7f9166700 1 --2- 172.21.15.16:0/2668409080 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff7f4099a40 0x7ff7f406f610 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:00.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.862+0000 7ff7f9166700 1 --2- 172.21.15.16:0/2668409080 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff7f4131810 0x7ff7f4070090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:00.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.862+0000 7ff7f9166700 1 --2- 172.21.15.16:0/2668409080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7f409a580 0x7ff7f406fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:00.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.862+0000 7ff7f9166700 1 -- 172.21.15.16:0/2668409080 >> 172.21.15.16:0/2668409080 conn(0x7ff7f40a2aa0 msgr2=0x7ff7f41b8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:00.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.863+0000 7ff7f9166700 1 -- 172.21.15.16:0/2668409080 shutdown_connections 2023-09-30T22:31:00.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.863+0000 7ff7f9166700 1 -- 172.21.15.16:0/2668409080 wait complete. 2023-09-30T22:31:00.877 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:31:00.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.895+0000 7ff672ffd700 1 -- 172.21.15.16:0/3242313951 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff660005590 con 0x7ff68809d6f0 2023-09-30T22:31:00.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.896+0000 7ff6857fa700 1 -- 172.21.15.16:0/3242313951 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff67c027b70 con 0x7ff68809d6f0 2023-09-30T22:31:00.896 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:31:00.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.898+0000 7ff672ffd700 1 -- 172.21.15.16:0/3242313951 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff68809d6f0 msgr2=0x7ff68811e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:00.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.898+0000 7ff672ffd700 1 --2- 172.21.15.16:0/3242313951 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff68809d6f0 0x7ff68811e0b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff68809ea70 tx=0x7ff67c004750 comp rx=0 tx=0).stop 2023-09-30T22:31:00.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.898+0000 7ff672ffd700 1 -- 172.21.15.16:0/3242313951 shutdown_connections 2023-09-30T22:31:00.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.898+0000 7ff672ffd700 1 --2- 172.21.15.16:0/3242313951 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff68809d6f0 0x7ff68811e0b0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:00.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.898+0000 7ff672ffd700 1 --2- 172.21.15.16:0/3242313951 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff68809e410 0x7ff68811eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:00.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.898+0000 7ff672ffd700 1 --2- 172.21.15.16:0/3242313951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff68809e000 0x7ff68811e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:00.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.898+0000 7ff672ffd700 1 -- 172.21.15.16:0/3242313951 >> 172.21.15.16:0/3242313951 conn(0x7ff688066f50 msgr2=0x7ff6880a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:00.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.898+0000 7ff672ffd700 1 -- 172.21.15.16:0/3242313951 shutdown_connections 2023-09-30T22:31:00.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.898+0000 7ff672ffd700 1 -- 172.21.15.16:0/3242313951 wait complete. 2023-09-30T22:31:00.913 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:00.916 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:31:00.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.942+0000 7fdbd0004700 1 -- 172.21.15.16:0/1147010574 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fdbc80747f0 con 0x7fdbc809d630 2023-09-30T22:31:00.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.943+0000 7fdbbe7fc700 1 -- 172.21.15.16:0/1147010574 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdbc4011360 con 0x7fdbc809d630 2023-09-30T22:31:00.943 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:31:00.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.945+0000 7fdbd0004700 1 -- 172.21.15.16:0/1147010574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbc809d630 msgr2=0x7fdbc81337a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:00.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.945+0000 7fdbd0004700 1 --2- 172.21.15.16:0/1147010574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbc809d630 0x7fdbc81337a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fdbc400c520 tx=0x7fdbc400c8e0 comp rx=0 tx=0).stop 2023-09-30T22:31:00.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.945+0000 7fdbd0004700 1 -- 172.21.15.16:0/1147010574 shutdown_connections 2023-09-30T22:31:00.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.945+0000 7fdbd0004700 1 --2- 172.21.15.16:0/1147010574 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdbc8138d80 0x7fdbc8134220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:00.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.945+0000 7fdbd0004700 1 --2- 172.21.15.16:0/1147010574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbc809d630 0x7fdbc81337a0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:00.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.945+0000 7fdbd0004700 1 --2- 172.21.15.16:0/1147010574 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdbc80a0a30 0x7fdbc8133ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:00.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.945+0000 7fdbd0004700 1 -- 172.21.15.16:0/1147010574 >> 172.21.15.16:0/1147010574 conn(0x7fdbc80a3050 msgr2=0x7fdbc809ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:00.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.945+0000 7fdbd0004700 1 -- 172.21.15.16:0/1147010574 shutdown_connections 2023-09-30T22:31:00.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:00.945+0000 7fdbd0004700 1 -- 172.21.15.16:0/1147010574 wait complete. 2023-09-30T22:31:00.959 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:31:01.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.050+0000 7fc795fac700 1 Processor -- start 2023-09-30T22:31:01.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.050+0000 7fc795fac700 1 -- start start 2023-09-30T22:31:01.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.050+0000 7fc795fac700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc79009d6f0 0x7fc79009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:01.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.050+0000 7fc795fac700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc79009e000 0x7fc7901339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:01.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.050+0000 7fc795fac700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc79009e410 0x7fc790133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:01.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.050+0000 7fc795fac700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc79006b460 con 0x7fc79009d6f0 2023-09-30T22:31:01.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.050+0000 7fc795fac700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc79009b120 con 0x7fc79009e410 2023-09-30T22:31:01.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.050+0000 7fc795fac700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc79009a7e0 con 0x7fc79009e000 2023-09-30T22:31:01.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.050+0000 7fc78effd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc79009e000 0x7fc7901339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:01.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.050+0000 7fc78f7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc79009d6f0 0x7fc79009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:01.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.050+0000 7fc78ffff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc79009e410 0x7fc790133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:01.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.051+0000 7fc78effd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc79009e000 0x7fc7901339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:57888/0 (socket says 172.21.15.16:57888) 2023-09-30T22:31:01.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.051+0000 7fc78effd700 1 -- 172.21.15.16:0/3936439473 learned_addr learned my addr 172.21.15.16:0/3936439473 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:01.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.051+0000 7fc78f7fe700 1 -- 172.21.15.16:0/3936439473 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc79009e000 msgr2=0x7fc7901339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:01.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.051+0000 7fc78f7fe700 1 --2- 172.21.15.16:0/3936439473 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc79009e000 0x7fc7901339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:01.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.051+0000 7fc78f7fe700 1 -- 172.21.15.16:0/3936439473 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc79009e410 msgr2=0x7fc790133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:01.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.051+0000 7fc78f7fe700 1 --2- 172.21.15.16:0/3936439473 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc79009e410 0x7fc790133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:01.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.051+0000 7fc78f7fe700 1 -- 172.21.15.16:0/3936439473 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc79009a500 con 0x7fc79009d6f0 2023-09-30T22:31:01.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.051+0000 7fc78effd700 1 --2- 172.21.15.16:0/3936439473 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc79009e000 0x7fc7901339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:01.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.051+0000 7fc78ffff700 1 --2- 172.21.15.16:0/3936439473 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc79009e410 0x7fc790133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:01.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.051+0000 7fc78f7fe700 1 --2- 172.21.15.16:0/3936439473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc79009d6f0 0x7fc79009dac0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc780009840 tx=0x7fc780009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6470b81d36cdbea7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:01.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.051+0000 7fc78e7fc700 1 -- 172.21.15.16:0/3936439473 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc780011830 con 0x7fc79009d6f0 2023-09-30T22:31:01.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.051+0000 7fc78e7fc700 1 -- 172.21.15.16:0/3936439473 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc780011990 con 0x7fc79009d6f0 2023-09-30T22:31:01.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.051+0000 7fc78e7fc700 1 -- 172.21.15.16:0/3936439473 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc780011d20 con 0x7fc79009d6f0 2023-09-30T22:31:01.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.052+0000 7fc795fac700 1 -- 172.21.15.16:0/3936439473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc79009d6f0 msgr2=0x7fc79009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:01.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.052+0000 7fc795fac700 1 --2- 172.21.15.16:0/3936439473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc79009d6f0 0x7fc79009dac0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc780009840 tx=0x7fc780009b50 comp rx=0 tx=0).stop 2023-09-30T22:31:01.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.052+0000 7fc795fac700 1 -- 172.21.15.16:0/3936439473 shutdown_connections 2023-09-30T22:31:01.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.052+0000 7fc795fac700 1 --2- 172.21.15.16:0/3936439473 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc79009e410 0x7fc790133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:01.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.052+0000 7fc795fac700 1 --2- 172.21.15.16:0/3936439473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc79009d6f0 0x7fc79009dac0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:01.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.052+0000 7fc795fac700 1 --2- 172.21.15.16:0/3936439473 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc79009e000 0x7fc7901339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:01.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.052+0000 7fc795fac700 1 -- 172.21.15.16:0/3936439473 >> 172.21.15.16:0/3936439473 conn(0x7fc790066f50 msgr2=0x7fc79009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:01.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.052+0000 7fc795fac700 1 -- 172.21.15.16:0/3936439473 shutdown_connections 2023-09-30T22:31:01.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.052+0000 7fc795fac700 1 -- 172.21.15.16:0/3936439473 wait complete. 2023-09-30T22:31:01.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.053+0000 7fc795fac700 1 Processor -- start 2023-09-30T22:31:01.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.053+0000 7fc795fac700 1 -- start start 2023-09-30T22:31:01.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.054+0000 7fc795fac700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc79009d6f0 0x7fc7901a5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:01.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.054+0000 7fc795fac700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc79009e000 0x7fc7901a63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:01.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.054+0000 7fc795fac700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc79009e410 0x7fc7901a6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:01.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.054+0000 7fc795fac700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7901a7f70 con 0x7fc79009e410 2023-09-30T22:31:01.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.054+0000 7fc795fac700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7901a9830 con 0x7fc79009d6f0 2023-09-30T22:31:01.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.054+0000 7fc795fac700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc7901a8fc0 con 0x7fc79009e000 2023-09-30T22:31:01.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.054+0000 7fc78f7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc79009d6f0 0x7fc7901a5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:01.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.054+0000 7fc78effd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc79009e000 0x7fc7901a63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:01.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.054+0000 7fc78ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc79009e410 0x7fc7901a6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:01.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.054+0000 7fc78f7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc79009d6f0 0x7fc7901a5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38448/0 (socket says 172.21.15.16:38448) 2023-09-30T22:31:01.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.054+0000 7fc78f7fe700 1 -- 172.21.15.16:0/2068739656 learned_addr learned my addr 172.21.15.16:0/2068739656 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:01.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.054+0000 7fc78f7fe700 1 -- 172.21.15.16:0/2068739656 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc79009e000 msgr2=0x7fc7901a63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:01.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.054+0000 7fc78f7fe700 1 --2- 172.21.15.16:0/2068739656 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc79009e000 0x7fc7901a63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:01.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.054+0000 7fc78f7fe700 1 -- 172.21.15.16:0/2068739656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc79009e410 msgr2=0x7fc7901a6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:01.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.054+0000 7fc78f7fe700 1 --2- 172.21.15.16:0/2068739656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc79009e410 0x7fc7901a6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:01.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.054+0000 7fc78f7fe700 1 -- 172.21.15.16:0/2068739656 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc784009550 con 0x7fc79009d6f0 2023-09-30T22:31:01.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.054+0000 7fc78ffff700 1 --2- 172.21.15.16:0/2068739656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc79009e410 0x7fc7901a6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:01.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.054+0000 7fc78f7fe700 1 --2- 172.21.15.16:0/2068739656 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc79009d6f0 0x7fc7901a5eb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc7800090b0 tx=0x7fc780004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:01.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.055+0000 7fc777fff700 1 -- 172.21.15.16:0/2068739656 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc780004450 con 0x7fc79009d6f0 2023-09-30T22:31:01.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.055+0000 7fc777fff700 1 -- 172.21.15.16:0/2068739656 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc78001f070 con 0x7fc79009d6f0 2023-09-30T22:31:01.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.055+0000 7fc777fff700 1 -- 172.21.15.16:0/2068739656 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7800038e0 con 0x7fc79009d6f0 2023-09-30T22:31:01.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.055+0000 7fc795fac700 1 -- 172.21.15.16:0/2068739656 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc788009a70 con 0x7fc79009d6f0 2023-09-30T22:31:01.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.055+0000 7fc795fac700 1 -- 172.21.15.16:0/2068739656 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7800094a0 con 0x7fc79009d6f0 2023-09-30T22:31:01.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.055+0000 7fc777fff700 1 -- 172.21.15.16:0/2068739656 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc780003a40 con 0x7fc79009d6f0 2023-09-30T22:31:01.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.056+0000 7fc777fff700 1 -- 172.21.15.16:0/2068739656 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fc780021070 con 0x7fc79009d6f0 2023-09-30T22:31:01.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.056+0000 7fc795fac700 1 -- 172.21.15.16:0/2068739656 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc790074ea0 con 0x7fc79009d6f0 2023-09-30T22:31:01.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.061+0000 7fc777fff700 1 -- 172.21.15.16:0/2068739656 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc78002b080 con 0x7fc79009d6f0 2023-09-30T22:31:01.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.264+0000 7fc795fac700 1 -- 172.21.15.16:0/2068739656 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc7901a93b0 con 0x7fc79009d6f0 2023-09-30T22:31:01.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.265+0000 7fc777fff700 1 -- 172.21.15.16:0/2068739656 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc7901a93b0 con 0x7fc79009d6f0 2023-09-30T22:31:01.265 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:31:01.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.267+0000 7fc795fac700 1 -- 172.21.15.16:0/2068739656 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc79009d6f0 msgr2=0x7fc7901a5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:01.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.267+0000 7fc795fac700 1 --2- 172.21.15.16:0/2068739656 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc79009d6f0 0x7fc7901a5eb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc7800090b0 tx=0x7fc780004750 comp rx=0 tx=0).stop 2023-09-30T22:31:01.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.267+0000 7fc795fac700 1 -- 172.21.15.16:0/2068739656 shutdown_connections 2023-09-30T22:31:01.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.267+0000 7fc795fac700 1 --2- 172.21.15.16:0/2068739656 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc79009d6f0 0x7fc7901a5eb0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:01.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.267+0000 7fc795fac700 1 --2- 172.21.15.16:0/2068739656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc79009e410 0x7fc7901a6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:01.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.267+0000 7fc795fac700 1 --2- 172.21.15.16:0/2068739656 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc79009e000 0x7fc7901a63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:01.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.267+0000 7fc795fac700 1 -- 172.21.15.16:0/2068739656 >> 172.21.15.16:0/2068739656 conn(0x7fc790066f50 msgr2=0x7fc7900a2690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:01.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.267+0000 7fc795fac700 1 -- 172.21.15.16:0/2068739656 shutdown_connections 2023-09-30T22:31:01.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:01.267+0000 7fc795fac700 1 -- 172.21.15.16:0/2068739656 wait complete. 2023-09-30T22:31:01.281 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:31:01.878 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:01.917 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:01.960 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:02.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.035+0000 7f3f205ab700 1 Processor -- start 2023-09-30T22:31:02.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.035+0000 7f3f205ab700 1 -- start start 2023-09-30T22:31:02.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.036+0000 7f3f205ab700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3f180a0a30 0x7f3f180a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:02.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.036+0000 7f3f205ab700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3f180a1340 0x7f3f18133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:02.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.036+0000 7f3f205ab700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3f18133fa0 0x7f3f18134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:02.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.036+0000 7f3f205ab700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f1809edd0 con 0x7f3f180a0a30 2023-09-30T22:31:02.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.036+0000 7f3f205ab700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1809e1f0 con 0x7f3f18133fa0 2023-09-30T22:31:02.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.036+0000 7f3f205ab700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1809dc50 con 0x7f3f180a1340 2023-09-30T22:31:02.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.036+0000 7f3f1e347700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3f180a0a30 0x7f3f180a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:02.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.036+0000 7f3f1e347700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3f180a0a30 0x7f3f180a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55526/0 (socket says 172.21.15.16:55526) 2023-09-30T22:31:02.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.036+0000 7f3f1e347700 1 -- 172.21.15.16:0/939830240 learned_addr learned my addr 172.21.15.16:0/939830240 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:02.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.036+0000 7f3f1eb48700 1 --2- 172.21.15.16:0/939830240 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3f18133fa0 0x7f3f18134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:02.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.037+0000 7f3f1db46700 1 --2- 172.21.15.16:0/939830240 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3f180a1340 0x7f3f18133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:02.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.037+0000 7f3f1eb48700 1 -- 172.21.15.16:0/939830240 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3f180a1340 msgr2=0x7f3f18133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:02.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.037+0000 7f3f1eb48700 1 --2- 172.21.15.16:0/939830240 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3f180a1340 0x7f3f18133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.037+0000 7f3f1eb48700 1 -- 172.21.15.16:0/939830240 >> v2:172.21.15.16:3300/0 conn(0x7f3f180a0a30 msgr2=0x7f3f180a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:02.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.037+0000 7f3f1eb48700 1 --2- 172.21.15.16:0/939830240 >> v2:172.21.15.16:3300/0 conn(0x7f3f180a0a30 0x7f3f180a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.037+0000 7f3f1eb48700 1 -- 172.21.15.16:0/939830240 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f180a1fe0 con 0x7f3f18133fa0 2023-09-30T22:31:02.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.037+0000 7f3f1e347700 1 --2- 172.21.15.16:0/939830240 >> v2:172.21.15.16:3300/0 conn(0x7f3f180a0a30 0x7f3f180a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:02.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.037+0000 7f3f1eb48700 1 --2- 172.21.15.16:0/939830240 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3f18133fa0 0x7f3f18134410 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3f10009dc0 tx=0x7f3f1000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a246b7c31892a8d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:02.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.037+0000 7f3f1d345700 1 -- 172.21.15.16:0/939830240 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f1000e070 con 0x7f3f18133fa0 2023-09-30T22:31:02.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.037+0000 7f3f1d345700 1 -- 172.21.15.16:0/939830240 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f1000bae0 con 0x7f3f18133fa0 2023-09-30T22:31:02.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.037+0000 7f3f1d345700 1 -- 172.21.15.16:0/939830240 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f1000be90 con 0x7f3f18133fa0 2023-09-30T22:31:02.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.038+0000 7f3f205ab700 1 -- 172.21.15.16:0/939830240 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3f18133fa0 msgr2=0x7f3f18134410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:02.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.038+0000 7f3f205ab700 1 --2- 172.21.15.16:0/939830240 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3f18133fa0 0x7f3f18134410 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f3f10009dc0 tx=0x7f3f1000b330 comp rx=0 tx=0).stop 2023-09-30T22:31:02.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.038+0000 7f3f205ab700 1 -- 172.21.15.16:0/939830240 shutdown_connections 2023-09-30T22:31:02.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.038+0000 7f3f205ab700 1 --2- 172.21.15.16:0/939830240 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3f18133fa0 0x7f3f18134410 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.038+0000 7f3f205ab700 1 --2- 172.21.15.16:0/939830240 >> v2:172.21.15.16:3300/0 conn(0x7f3f180a0a30 0x7f3f180a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.038+0000 7f3f205ab700 1 --2- 172.21.15.16:0/939830240 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3f180a1340 0x7f3f18133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.038+0000 7f3f205ab700 1 -- 172.21.15.16:0/939830240 >> 172.21.15.16:0/939830240 conn(0x7f3f180a6550 msgr2=0x7f3f180a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:02.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.038+0000 7f3f205ab700 1 -- 172.21.15.16:0/939830240 shutdown_connections 2023-09-30T22:31:02.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.039+0000 7f3f205ab700 1 -- 172.21.15.16:0/939830240 wait complete. 2023-09-30T22:31:02.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.040+0000 7f3f205ab700 1 Processor -- start 2023-09-30T22:31:02.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.040+0000 7f3f205ab700 1 -- start start 2023-09-30T22:31:02.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.041+0000 7f3f205ab700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3f180a0a30 0x7f3f181200e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:02.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.041+0000 7f3f205ab700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f180a1340 0x7f3f18120620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:02.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.041+0000 7f3f205ab700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3f18133fa0 0x7f3f18120b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:02.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.041+0000 7f3f205ab700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f18124530 con 0x7f3f180a1340 2023-09-30T22:31:02.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.041+0000 7f3f205ab700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f18121d00 con 0x7f3f18133fa0 2023-09-30T22:31:02.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.041+0000 7f3f205ab700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3f18123890 con 0x7f3f180a0a30 2023-09-30T22:31:02.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.041+0000 7f3f1db46700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f180a1340 0x7f3f18120620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:02.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.041+0000 7f3f1e347700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3f180a0a30 0x7f3f181200e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:02.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.041+0000 7f3f1db46700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f180a1340 0x7f3f18120620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55534/0 (socket says 172.21.15.16:55534) 2023-09-30T22:31:02.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.041+0000 7f3f1db46700 1 -- 172.21.15.16:0/1177315150 learned_addr learned my addr 172.21.15.16:0/1177315150 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:02.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.041+0000 7f3f1eb48700 1 --2- 172.21.15.16:0/1177315150 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3f18133fa0 0x7f3f18120b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:02.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.041+0000 7f3f1db46700 1 -- 172.21.15.16:0/1177315150 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3f180a0a30 msgr2=0x7f3f181200e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:02.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.041+0000 7f3f1db46700 1 --2- 172.21.15.16:0/1177315150 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3f180a0a30 0x7f3f181200e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.041+0000 7f3f1db46700 1 -- 172.21.15.16:0/1177315150 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3f18133fa0 msgr2=0x7f3f18120b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:02.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.041+0000 7f3f1db46700 1 --2- 172.21.15.16:0/1177315150 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3f18133fa0 0x7f3f18120b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.041+0000 7f3f1db46700 1 -- 172.21.15.16:0/1177315150 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f14009550 con 0x7f3f180a1340 2023-09-30T22:31:02.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.042+0000 7f3f1db46700 1 --2- 172.21.15.16:0/1177315150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f180a1340 0x7f3f18120620 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3f0800e810 tx=0x7f3f0800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:02.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.042+0000 7f3f0effd700 1 -- 172.21.15.16:0/1177315150 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0800c070 con 0x7f3f180a1340 2023-09-30T22:31:02.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.042+0000 7f3f0effd700 1 -- 172.21.15.16:0/1177315150 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f08013610 con 0x7f3f180a1340 2023-09-30T22:31:02.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.042+0000 7f3f205ab700 1 -- 172.21.15.16:0/1177315150 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f10009a70 con 0x7f3f180a1340 2023-09-30T22:31:02.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.042+0000 7f3f0effd700 1 -- 172.21.15.16:0/1177315150 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f080139c0 con 0x7f3f180a1340 2023-09-30T22:31:02.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.042+0000 7f3f205ab700 1 -- 172.21.15.16:0/1177315150 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f180a0220 con 0x7f3f180a1340 2023-09-30T22:31:02.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.043+0000 7f3f205ab700 1 -- 172.21.15.16:0/1177315150 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f18074ea0 con 0x7f3f180a1340 2023-09-30T22:31:02.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.043+0000 7f3f0effd700 1 -- 172.21.15.16:0/1177315150 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3f08013b20 con 0x7f3f180a1340 2023-09-30T22:31:02.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.044+0000 7f3f0effd700 1 -- 172.21.15.16:0/1177315150 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f3f0803f7a0 con 0x7f3f180a1340 2023-09-30T22:31:02.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.048+0000 7f3f0effd700 1 -- 172.21.15.16:0/1177315150 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3f0800f040 con 0x7f3f180a1340 2023-09-30T22:31:02.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.085+0000 7fcc1b5b0700 1 Processor -- start 2023-09-30T22:31:02.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.085+0000 7fcc1b5b0700 1 -- start start 2023-09-30T22:31:02.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.086+0000 7fcc1b5b0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcc1409d6f0 0x7fcc1409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:02.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.086+0000 7fcc1b5b0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc1409e000 0x7fcc14130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:02.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.086+0000 7fcc1b5b0700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcc1409e410 0x7fcc14130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:02.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.086+0000 7fcc1b5b0700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc1409f520 con 0x7fcc1409e410 2023-09-30T22:31:02.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.086+0000 7fcc1b5b0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1409f130 con 0x7fcc1409d6f0 2023-09-30T22:31:02.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.086+0000 7fcc1b5b0700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1409e930 con 0x7fcc1409e000 2023-09-30T22:31:02.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.086+0000 7fcc18b4b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc1409e000 0x7fcc14130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:02.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.086+0000 7fcc18b4b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc1409e000 0x7fcc14130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:57922/0 (socket says 172.21.15.16:57922) 2023-09-30T22:31:02.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.086+0000 7fcc18b4b700 1 -- 172.21.15.16:0/4237851563 learned_addr learned my addr 172.21.15.16:0/4237851563 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:02.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.086+0000 7fcc19b4d700 1 --2- 172.21.15.16:0/4237851563 >> v2:172.21.15.16:3300/0 conn(0x7fcc1409e410 0x7fcc14130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:02.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.086+0000 7fcc1934c700 1 --2- 172.21.15.16:0/4237851563 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcc1409d6f0 0x7fcc1409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:02.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.086+0000 7fcc18b4b700 1 -- 172.21.15.16:0/4237851563 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcc1409d6f0 msgr2=0x7fcc1409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:02.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.086+0000 7fcc18b4b700 1 --2- 172.21.15.16:0/4237851563 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcc1409d6f0 0x7fcc1409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.086+0000 7fcc18b4b700 1 -- 172.21.15.16:0/4237851563 >> v2:172.21.15.16:3300/0 conn(0x7fcc1409e410 msgr2=0x7fcc14130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:02.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.086+0000 7fcc18b4b700 1 --2- 172.21.15.16:0/4237851563 >> v2:172.21.15.16:3300/0 conn(0x7fcc1409e410 0x7fcc14130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.086+0000 7fcc18b4b700 1 -- 172.21.15.16:0/4237851563 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc1409b2d0 con 0x7fcc1409e000 2023-09-30T22:31:02.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.086+0000 7fcc19b4d700 1 --2- 172.21.15.16:0/4237851563 >> v2:172.21.15.16:3300/0 conn(0x7fcc1409e410 0x7fcc14130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:02.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.087+0000 7fcc18b4b700 1 --2- 172.21.15.16:0/4237851563 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc1409e000 0x7fcc14130660 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcc0400bbc0 tx=0x7fcc0400bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=127becb9c1f5f061 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:02.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.087+0000 7fcc0bfff700 1 -- 172.21.15.16:0/4237851563 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0400daf0 con 0x7fcc1409e000 2023-09-30T22:31:02.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.087+0000 7fcc0bfff700 1 -- 172.21.15.16:0/4237851563 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc0400dc50 con 0x7fcc1409e000 2023-09-30T22:31:02.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.087+0000 7fcc0bfff700 1 -- 172.21.15.16:0/4237851563 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc04010470 con 0x7fcc1409e000 2023-09-30T22:31:02.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.087+0000 7fcc1b5b0700 1 -- 172.21.15.16:0/4237851563 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc1409e000 msgr2=0x7fcc14130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:02.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.087+0000 7fcc1b5b0700 1 --2- 172.21.15.16:0/4237851563 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc1409e000 0x7fcc14130660 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fcc0400bbc0 tx=0x7fcc0400bed0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.088+0000 7fcc1b5b0700 1 -- 172.21.15.16:0/4237851563 shutdown_connections 2023-09-30T22:31:02.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.088+0000 7fcc1b5b0700 1 --2- 172.21.15.16:0/4237851563 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcc1409d6f0 0x7fcc1409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.088+0000 7fcc1b5b0700 1 --2- 172.21.15.16:0/4237851563 >> v2:172.21.15.16:3300/0 conn(0x7fcc1409e410 0x7fcc14130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.088+0000 7fcc1b5b0700 1 --2- 172.21.15.16:0/4237851563 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc1409e000 0x7fcc14130660 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.088+0000 7fcc1b5b0700 1 -- 172.21.15.16:0/4237851563 >> 172.21.15.16:0/4237851563 conn(0x7fcc14066f50 msgr2=0x7fcc1409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:02.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.088+0000 7fcc1b5b0700 1 -- 172.21.15.16:0/4237851563 shutdown_connections 2023-09-30T22:31:02.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.088+0000 7fcc1b5b0700 1 -- 172.21.15.16:0/4237851563 wait complete. 2023-09-30T22:31:02.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.089+0000 7fcc1b5b0700 1 Processor -- start 2023-09-30T22:31:02.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.089+0000 7fcc1b5b0700 1 -- start start 2023-09-30T22:31:02.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.089+0000 7fcc1b5b0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc1409d6f0 0x7fcc1411df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:02.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.089+0000 7fcc1b5b0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc1409e000 0x7fcc1411e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:02.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.089+0000 7fcc1b5b0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcc1409e410 0x7fcc1411ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:02.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.089+0000 7fcc1b5b0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc14120190 con 0x7fcc1409d6f0 2023-09-30T22:31:02.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.089+0000 7fcc1b5b0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc14121a50 con 0x7fcc1409e410 2023-09-30T22:31:02.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.089+0000 7fcc1b5b0700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcc14120e20 con 0x7fcc1409e000 2023-09-30T22:31:02.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.089+0000 7fcc1934c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc1409d6f0 0x7fcc1411df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:02.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.089+0000 7fcc1934c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc1409d6f0 0x7fcc1411df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55558/0 (socket says 172.21.15.16:55558) 2023-09-30T22:31:02.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.089+0000 7fcc1934c700 1 -- 172.21.15.16:0/3501466889 learned_addr learned my addr 172.21.15.16:0/3501466889 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:02.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.089+0000 7fcc18b4b700 1 --2- 172.21.15.16:0/3501466889 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc1409e000 0x7fcc1411e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:02.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.089+0000 7fcc19b4d700 1 --2- 172.21.15.16:0/3501466889 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcc1409e410 0x7fcc1411ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:02.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.090+0000 7fcc18b4b700 1 -- 172.21.15.16:0/3501466889 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcc1409e410 msgr2=0x7fcc1411ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:02.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.090+0000 7fcc18b4b700 1 --2- 172.21.15.16:0/3501466889 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcc1409e410 0x7fcc1411ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.090+0000 7fcc18b4b700 1 -- 172.21.15.16:0/3501466889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc1409d6f0 msgr2=0x7fcc1411df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:02.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.090+0000 7fcc18b4b700 1 --2- 172.21.15.16:0/3501466889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc1409d6f0 0x7fcc1411df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.090+0000 7fcc18b4b700 1 -- 172.21.15.16:0/3501466889 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc0c009a70 con 0x7fcc1409e000 2023-09-30T22:31:02.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.090+0000 7fcc18b4b700 1 --2- 172.21.15.16:0/3501466889 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc1409e000 0x7fcc1411e4c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fcc04015000 tx=0x7fcc04004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:02.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.090+0000 7fcc09ffb700 1 -- 172.21.15.16:0/3501466889 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc040038a0 con 0x7fcc1409e000 2023-09-30T22:31:02.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.090+0000 7fcc09ffb700 1 -- 172.21.15.16:0/3501466889 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc04003a00 con 0x7fcc1409e000 2023-09-30T22:31:02.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.090+0000 7fcc1b5b0700 1 -- 172.21.15.16:0/3501466889 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc0400b7f0 con 0x7fcc1409e000 2023-09-30T22:31:02.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.090+0000 7fcc09ffb700 1 -- 172.21.15.16:0/3501466889 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc04003d90 con 0x7fcc1409e000 2023-09-30T22:31:02.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.090+0000 7fcc1b5b0700 1 -- 172.21.15.16:0/3501466889 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc140a1720 con 0x7fcc1409e000 2023-09-30T22:31:02.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.091+0000 7fcc09ffb700 1 -- 172.21.15.16:0/3501466889 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcc04010470 con 0x7fcc1409e000 2023-09-30T22:31:02.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.092+0000 7fcc09ffb700 1 -- 172.21.15.16:0/3501466889 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fcc04040d70 con 0x7fcc1409e000 2023-09-30T22:31:02.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.092+0000 7fcc1b5b0700 1 -- 172.21.15.16:0/3501466889 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc14121410 con 0x7fcc1409e000 2023-09-30T22:31:02.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.097+0000 7fcc09ffb700 1 -- 172.21.15.16:0/3501466889 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcc14121410 con 0x7fcc1409e000 2023-09-30T22:31:02.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.108+0000 7f908825a700 1 Processor -- start 2023-09-30T22:31:02.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.109+0000 7f908825a700 1 -- start start 2023-09-30T22:31:02.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.109+0000 7f908825a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f90800a0a30 0x7f90800a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:02.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.109+0000 7f908825a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f90800a1340 0x7f9080133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:02.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.109+0000 7f908825a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9080133fa0 0x7f9080134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:02.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.109+0000 7f908825a700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f908009edd0 con 0x7f90800a0a30 2023-09-30T22:31:02.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.109+0000 7f908825a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f908009e1f0 con 0x7f90800a1340 2023-09-30T22:31:02.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.109+0000 7f908825a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f908009dc50 con 0x7f9080133fa0 2023-09-30T22:31:02.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.109+0000 7f90857f5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f90800a1340 0x7f9080133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:02.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.109+0000 7f9085ff6700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f90800a0a30 0x7f90800a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:02.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.110+0000 7f90857f5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f90800a1340 0x7f9080133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38466/0 (socket says 172.21.15.16:38466) 2023-09-30T22:31:02.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.110+0000 7f90867f7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9080133fa0 0x7f9080134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:02.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.110+0000 7f90857f5700 1 -- 172.21.15.16:0/1552184396 learned_addr learned my addr 172.21.15.16:0/1552184396 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:02.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.110+0000 7f9085ff6700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f90800a0a30 0x7f90800a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55568/0 (socket says 172.21.15.16:55568) 2023-09-30T22:31:02.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.110+0000 7f9085ff6700 1 -- 172.21.15.16:0/1552184396 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9080133fa0 msgr2=0x7f9080134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:02.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.110+0000 7f9085ff6700 1 --2- 172.21.15.16:0/1552184396 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9080133fa0 0x7f9080134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.110+0000 7f9085ff6700 1 -- 172.21.15.16:0/1552184396 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f90800a1340 msgr2=0x7f9080133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:02.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.110+0000 7f9085ff6700 1 --2- 172.21.15.16:0/1552184396 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f90800a1340 0x7f9080133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.110+0000 7f9085ff6700 1 -- 172.21.15.16:0/1552184396 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90800a1fe0 con 0x7f90800a0a30 2023-09-30T22:31:02.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.110+0000 7f90857f5700 1 --2- 172.21.15.16:0/1552184396 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f90800a1340 0x7f9080133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:02.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.110+0000 7f9085ff6700 1 --2- 172.21.15.16:0/1552184396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90800a0a30 0x7f90800a0e00 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f907c009840 tx=0x7f907c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa1519767623bb68 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:02.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.110+0000 7f9084ff4700 1 -- 172.21.15.16:0/1552184396 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907c011830 con 0x7f90800a0a30 2023-09-30T22:31:02.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.110+0000 7f9084ff4700 1 -- 172.21.15.16:0/1552184396 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f907c011990 con 0x7f90800a0a30 2023-09-30T22:31:02.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.110+0000 7f9084ff4700 1 -- 172.21.15.16:0/1552184396 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907c011d20 con 0x7f90800a0a30 2023-09-30T22:31:02.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.111+0000 7f908825a700 1 -- 172.21.15.16:0/1552184396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90800a0a30 msgr2=0x7f90800a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:02.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.111+0000 7f908825a700 1 --2- 172.21.15.16:0/1552184396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90800a0a30 0x7f90800a0e00 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f907c009840 tx=0x7f907c009b50 comp rx=0 tx=0).stop 2023-09-30T22:31:02.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.111+0000 7f908825a700 1 -- 172.21.15.16:0/1552184396 shutdown_connections 2023-09-30T22:31:02.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.111+0000 7f908825a700 1 --2- 172.21.15.16:0/1552184396 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9080133fa0 0x7f9080134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.111+0000 7f908825a700 1 --2- 172.21.15.16:0/1552184396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90800a0a30 0x7f90800a0e00 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.111+0000 7f908825a700 1 --2- 172.21.15.16:0/1552184396 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f90800a1340 0x7f9080133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.111+0000 7f908825a700 1 -- 172.21.15.16:0/1552184396 >> 172.21.15.16:0/1552184396 conn(0x7f90800a6550 msgr2=0x7f90800a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:02.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.111+0000 7f908825a700 1 -- 172.21.15.16:0/1552184396 shutdown_connections 2023-09-30T22:31:02.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.111+0000 7f908825a700 1 -- 172.21.15.16:0/1552184396 wait complete. 2023-09-30T22:31:02.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.112+0000 7f908825a700 1 Processor -- start 2023-09-30T22:31:02.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.112+0000 7f908825a700 1 -- start start 2023-09-30T22:31:02.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.113+0000 7f908825a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f90800a0a30 0x7f90801212d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:02.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.113+0000 7f908825a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f90800a1340 0x7f9080121810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:02.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.113+0000 7f908825a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9080133fa0 0x7f9080121d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:02.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.113+0000 7f908825a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90801326a0 con 0x7f9080133fa0 2023-09-30T22:31:02.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.113+0000 7f908825a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9080123d70 con 0x7f90800a1340 2023-09-30T22:31:02.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.113+0000 7f908825a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9080124850 con 0x7f90800a0a30 2023-09-30T22:31:02.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.113+0000 7f90857f5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f90800a1340 0x7f9080121810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:02.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.113+0000 7f9085ff6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f90800a0a30 0x7f90801212d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:02.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.113+0000 7f90857f5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f90800a1340 0x7f9080121810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38476/0 (socket says 172.21.15.16:38476) 2023-09-30T22:31:02.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.113+0000 7f90857f5700 1 -- 172.21.15.16:0/240627942 learned_addr learned my addr 172.21.15.16:0/240627942 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:02.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.113+0000 7f90867f7700 1 --2- 172.21.15.16:0/240627942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9080133fa0 0x7f9080121d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:02.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.113+0000 7f9085ff6700 1 -- 172.21.15.16:0/240627942 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f90800a1340 msgr2=0x7f9080121810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:02.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.113+0000 7f9085ff6700 1 --2- 172.21.15.16:0/240627942 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f90800a1340 0x7f9080121810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.113+0000 7f9085ff6700 1 -- 172.21.15.16:0/240627942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9080133fa0 msgr2=0x7f9080121d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:02.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.113+0000 7f9085ff6700 1 --2- 172.21.15.16:0/240627942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9080133fa0 0x7f9080121d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.113+0000 7f9085ff6700 1 -- 172.21.15.16:0/240627942 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9070009550 con 0x7f90800a0a30 2023-09-30T22:31:02.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.114+0000 7f9085ff6700 1 --2- 172.21.15.16:0/240627942 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f90800a0a30 0x7f90801212d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f907c009840 tx=0x7f907c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:02.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.114+0000 7f90767fc700 1 -- 172.21.15.16:0/240627942 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907c004450 con 0x7f90800a0a30 2023-09-30T22:31:02.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.114+0000 7f90767fc700 1 -- 172.21.15.16:0/240627942 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f907c01f070 con 0x7f90800a0a30 2023-09-30T22:31:02.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.114+0000 7f90767fc700 1 -- 172.21.15.16:0/240627942 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907c0038e0 con 0x7f90800a0a30 2023-09-30T22:31:02.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.114+0000 7f908825a700 1 -- 172.21.15.16:0/240627942 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f907c0094a0 con 0x7f90800a0a30 2023-09-30T22:31:02.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.114+0000 7f908825a700 1 -- 172.21.15.16:0/240627942 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90800a36e0 con 0x7f90800a0a30 2023-09-30T22:31:02.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.115+0000 7f90767fc700 1 -- 172.21.15.16:0/240627942 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f907c003a40 con 0x7f90800a0a30 2023-09-30T22:31:02.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.115+0000 7f90767fc700 1 -- 172.21.15.16:0/240627942 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f907c021070 con 0x7f90800a0a30 2023-09-30T22:31:02.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.115+0000 7f908825a700 1 -- 172.21.15.16:0/240627942 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90800a0220 con 0x7f90800a0a30 2023-09-30T22:31:02.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.120+0000 7f90767fc700 1 -- 172.21.15.16:0/240627942 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f907c01a430 con 0x7f90800a0a30 2023-09-30T22:31:02.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.269+0000 7f3f205ab700 1 -- 172.21.15.16:0/1177315150 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3f180a28e0 con 0x7f3f180a1340 2023-09-30T22:31:02.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.270+0000 7f3f0effd700 1 -- 172.21.15.16:0/1177315150 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3f0802b090 con 0x7f3f180a1340 2023-09-30T22:31:02.270 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:31:02.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.273+0000 7f3f205ab700 1 -- 172.21.15.16:0/1177315150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f180a1340 msgr2=0x7f3f18120620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:02.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.273+0000 7f3f205ab700 1 --2- 172.21.15.16:0/1177315150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f180a1340 0x7f3f18120620 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3f0800e810 tx=0x7f3f0800eb20 comp rx=0 tx=0).stop 2023-09-30T22:31:02.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.273+0000 7f3f205ab700 1 -- 172.21.15.16:0/1177315150 shutdown_connections 2023-09-30T22:31:02.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.273+0000 7f3f205ab700 1 --2- 172.21.15.16:0/1177315150 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3f18133fa0 0x7f3f18120b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.273+0000 7f3f205ab700 1 --2- 172.21.15.16:0/1177315150 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3f180a0a30 0x7f3f181200e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.273+0000 7f3f205ab700 1 --2- 172.21.15.16:0/1177315150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f180a1340 0x7f3f18120620 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.273+0000 7f3f205ab700 1 -- 172.21.15.16:0/1177315150 >> 172.21.15.16:0/1177315150 conn(0x7f3f180a6550 msgr2=0x7f3f1809ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:02.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.275+0000 7f3f205ab700 1 -- 172.21.15.16:0/1177315150 shutdown_connections 2023-09-30T22:31:02.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.275+0000 7f3f205ab700 1 -- 172.21.15.16:0/1177315150 wait complete. 2023-09-30T22:31:02.281 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:02.294 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:31:02.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.307+0000 7fcc1b5b0700 1 -- 172.21.15.16:0/3501466889 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fcc14121410 con 0x7fcc1409e000 2023-09-30T22:31:02.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.307+0000 7fcc09ffb700 1 -- 172.21.15.16:0/3501466889 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcc14121410 con 0x7fcc1409e000 2023-09-30T22:31:02.308 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:31:02.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.310+0000 7fcc1b5b0700 1 -- 172.21.15.16:0/3501466889 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc1409e000 msgr2=0x7fcc1411e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:02.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.310+0000 7fcc1b5b0700 1 --2- 172.21.15.16:0/3501466889 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc1409e000 0x7fcc1411e4c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fcc04015000 tx=0x7fcc04004750 comp rx=0 tx=0).stop 2023-09-30T22:31:02.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.310+0000 7fcc1b5b0700 1 -- 172.21.15.16:0/3501466889 shutdown_connections 2023-09-30T22:31:02.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.310+0000 7fcc1b5b0700 1 --2- 172.21.15.16:0/3501466889 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcc1409e410 0x7fcc1411ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.310+0000 7fcc1b5b0700 1 --2- 172.21.15.16:0/3501466889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc1409d6f0 0x7fcc1411df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.310+0000 7fcc1b5b0700 1 --2- 172.21.15.16:0/3501466889 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc1409e000 0x7fcc1411e4c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.310+0000 7fcc1b5b0700 1 -- 172.21.15.16:0/3501466889 >> 172.21.15.16:0/3501466889 conn(0x7fcc14066f50 msgr2=0x7fcc1406a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:02.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.310+0000 7fcc1b5b0700 1 -- 172.21.15.16:0/3501466889 shutdown_connections 2023-09-30T22:31:02.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.310+0000 7fcc1b5b0700 1 -- 172.21.15.16:0/3501466889 wait complete. 2023-09-30T22:31:02.325 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:31:02.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.340+0000 7f908825a700 1 -- 172.21.15.16:0/240627942 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9080074ea0 con 0x7f90800a0a30 2023-09-30T22:31:02.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.340+0000 7f90767fc700 1 -- 172.21.15.16:0/240627942 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f907c015030 con 0x7f90800a0a30 2023-09-30T22:31:02.340 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:31:02.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.343+0000 7f908825a700 1 -- 172.21.15.16:0/240627942 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f90800a0a30 msgr2=0x7f90801212d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:02.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.343+0000 7f908825a700 1 --2- 172.21.15.16:0/240627942 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f90800a0a30 0x7f90801212d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f907c009840 tx=0x7f907c004750 comp rx=0 tx=0).stop 2023-09-30T22:31:02.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.344+0000 7f908825a700 1 -- 172.21.15.16:0/240627942 shutdown_connections 2023-09-30T22:31:02.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.344+0000 7f908825a700 1 --2- 172.21.15.16:0/240627942 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9080133fa0 0x7f9080121d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.344+0000 7f908825a700 1 --2- 172.21.15.16:0/240627942 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f90800a0a30 0x7f90801212d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.344+0000 7f908825a700 1 --2- 172.21.15.16:0/240627942 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f90800a1340 0x7f9080121810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.344+0000 7f908825a700 1 -- 172.21.15.16:0/240627942 >> 172.21.15.16:0/240627942 conn(0x7f90800a6550 msgr2=0x7f90800a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:02.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.344+0000 7f908825a700 1 -- 172.21.15.16:0/240627942 shutdown_connections 2023-09-30T22:31:02.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.344+0000 7f908825a700 1 -- 172.21.15.16:0/240627942 wait complete. 2023-09-30T22:31:02.362 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:31:02.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.428+0000 7fcf30e3c700 1 Processor -- start 2023-09-30T22:31:02.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.428+0000 7fcf30e3c700 1 -- start start 2023-09-30T22:31:02.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.429+0000 7fcf30e3c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcf2c09d6f0 0x7fcf2c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:02.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.429+0000 7fcf30e3c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcf2c09e410 0x7fcf2c130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:02.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.429+0000 7fcf30e3c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf2c09f130 con 0x7fcf2c09e000 2023-09-30T22:31:02.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.429+0000 7fcf30e3c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2c09f520 con 0x7fcf2c09e410 2023-09-30T22:31:02.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.429+0000 7fcf30e3c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2c09e930 con 0x7fcf2c09d6f0 2023-09-30T22:31:02.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.429+0000 7fcf2a59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcf2c09d6f0 0x7fcf2c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:02.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.429+0000 7fcf2ad9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcf2c09e410 0x7fcf2c130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:02.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.429+0000 7fcf29d9b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fcf2c09e000 0x7fcf2c130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38642/0 (socket says 172.21.15.16:38642) 2023-09-30T22:31:02.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.429+0000 7fcf29d9b700 1 -- 172.21.15.16:0/1413398229 learned_addr learned my addr 172.21.15.16:0/1413398229 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:31:02.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.430+0000 7fcf2959a700 1 -- 172.21.15.16:0/1413398229 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 270694236 0 0) 0x7fcf2c09f130 con 0x7fcf2c09e000 2023-09-30T22:31:02.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.430+0000 7fcf2ad9d700 1 -- 172.21.15.16:0/1413398229 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcf2c09d6f0 msgr2=0x7fcf2c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:02.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.430+0000 7fcf2ad9d700 1 --2- 172.21.15.16:0/1413398229 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcf2c09d6f0 0x7fcf2c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.430+0000 7fcf2ad9d700 1 -- 172.21.15.16:0/1413398229 >> v1:172.21.15.16:6789/0 conn(0x7fcf2c09e000 legacy=0x7fcf2c130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:02.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.430+0000 7fcf2ad9d700 1 -- 172.21.15.16:0/1413398229 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf2c09b2d0 con 0x7fcf2c09e410 2023-09-30T22:31:02.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.430+0000 7fcf2a59c700 1 --2- 172.21.15.16:0/1413398229 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcf2c09d6f0 0x7fcf2c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:02.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.430+0000 7fcf2ad9d700 1 --2- 172.21.15.16:0/1413398229 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcf2c09e410 0x7fcf2c130d70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fcf24009dc0 tx=0x7fcf2400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6f64a1f6b235b10 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:02.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.430+0000 7fcf2959a700 1 -- 172.21.15.16:0/1413398229 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2400e070 con 0x7fcf2c09e410 2023-09-30T22:31:02.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.430+0000 7fcf2959a700 1 -- 172.21.15.16:0/1413398229 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf2400bb20 con 0x7fcf2c09e410 2023-09-30T22:31:02.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.430+0000 7fcf2959a700 1 -- 172.21.15.16:0/1413398229 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf24012430 con 0x7fcf2c09e410 2023-09-30T22:31:02.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.431+0000 7fcf30e3c700 1 -- 172.21.15.16:0/1413398229 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcf2c09e410 msgr2=0x7fcf2c130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:02.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.431+0000 7fcf30e3c700 1 --2- 172.21.15.16:0/1413398229 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcf2c09e410 0x7fcf2c130d70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fcf24009dc0 tx=0x7fcf2400b330 comp rx=0 tx=0).stop 2023-09-30T22:31:02.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.431+0000 7fcf30e3c700 1 -- 172.21.15.16:0/1413398229 shutdown_connections 2023-09-30T22:31:02.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.431+0000 7fcf30e3c700 1 --2- 172.21.15.16:0/1413398229 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcf2c09e410 0x7fcf2c130d70 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.431+0000 7fcf30e3c700 1 --2- 172.21.15.16:0/1413398229 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcf2c09d6f0 0x7fcf2c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.431+0000 7fcf30e3c700 1 -- 172.21.15.16:0/1413398229 >> 172.21.15.16:0/1413398229 conn(0x7fcf2c066f50 msgr2=0x7fcf2c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:02.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.431+0000 7fcf30e3c700 1 -- 172.21.15.16:0/1413398229 shutdown_connections 2023-09-30T22:31:02.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.431+0000 7fcf30e3c700 1 -- 172.21.15.16:0/1413398229 wait complete. 2023-09-30T22:31:02.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.432+0000 7fcf30e3c700 1 Processor -- start 2023-09-30T22:31:02.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.432+0000 7fcf30e3c700 1 -- start start 2023-09-30T22:31:02.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.432+0000 7fcf30e3c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf2c09d6f0 0x7fcf2c120150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:02.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.432+0000 7fcf30e3c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcf2c09e000 0x7fcf2c120690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:02.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.433+0000 7fcf30e3c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcf2c09e410 0x7fcf2c120bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:02.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.433+0000 7fcf30e3c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2c0a1b80 con 0x7fcf2c09d6f0 2023-09-30T22:31:02.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.433+0000 7fcf30e3c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2c09c700 con 0x7fcf2c09e410 2023-09-30T22:31:02.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.433+0000 7fcf30e3c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2c1303f0 con 0x7fcf2c09e000 2023-09-30T22:31:02.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.433+0000 7fcf2a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf2c09d6f0 0x7fcf2c120150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:02.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.433+0000 7fcf29d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcf2c09e000 0x7fcf2c120690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:02.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.433+0000 7fcf2a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf2c09d6f0 0x7fcf2c120150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55584/0 (socket says 172.21.15.16:55584) 2023-09-30T22:31:02.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.433+0000 7fcf2a59c700 1 -- 172.21.15.16:0/3511049543 learned_addr learned my addr 172.21.15.16:0/3511049543 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:02.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.433+0000 7fcf2ad9d700 1 --2- 172.21.15.16:0/3511049543 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcf2c09e410 0x7fcf2c120bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:02.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.433+0000 7fcf29d9b700 1 -- 172.21.15.16:0/3511049543 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcf2c09e410 msgr2=0x7fcf2c120bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:02.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.433+0000 7fcf29d9b700 1 --2- 172.21.15.16:0/3511049543 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcf2c09e410 0x7fcf2c120bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.433+0000 7fcf29d9b700 1 -- 172.21.15.16:0/3511049543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf2c09d6f0 msgr2=0x7fcf2c120150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:02.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.433+0000 7fcf29d9b700 1 --2- 172.21.15.16:0/3511049543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf2c09d6f0 0x7fcf2c120150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.433+0000 7fcf29d9b700 1 -- 172.21.15.16:0/3511049543 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf1c0094a0 con 0x7fcf2c09e000 2023-09-30T22:31:02.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.433+0000 7fcf29d9b700 1 --2- 172.21.15.16:0/3511049543 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcf2c09e000 0x7fcf2c120690 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c0a3e20 tx=0x7fcf2000c750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:02.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.433+0000 7fcf1affd700 1 -- 172.21.15.16:0/3511049543 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf20004c80 con 0x7fcf2c09e000 2023-09-30T22:31:02.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.433+0000 7fcf1affd700 1 -- 172.21.15.16:0/3511049543 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf20004de0 con 0x7fcf2c09e000 2023-09-30T22:31:02.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.433+0000 7fcf30e3c700 1 -- 172.21.15.16:0/3511049543 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf24009a70 con 0x7fcf2c09e000 2023-09-30T22:31:02.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.433+0000 7fcf1affd700 1 -- 172.21.15.16:0/3511049543 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf20007310 con 0x7fcf2c09e000 2023-09-30T22:31:02.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.433+0000 7fcf30e3c700 1 -- 172.21.15.16:0/3511049543 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf2c122110 con 0x7fcf2c09e000 2023-09-30T22:31:02.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.434+0000 7fcf1affd700 1 -- 172.21.15.16:0/3511049543 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcf20007470 con 0x7fcf2c09e000 2023-09-30T22:31:02.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.434+0000 7fcf30e3c700 1 -- 172.21.15.16:0/3511049543 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf2c11cc30 con 0x7fcf2c09e000 2023-09-30T22:31:02.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.434+0000 7fcf1affd700 1 -- 172.21.15.16:0/3511049543 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fcf2003f2e0 con 0x7fcf2c09e000 2023-09-30T22:31:02.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.439+0000 7fcf1affd700 1 -- 172.21.15.16:0/3511049543 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcf20007720 con 0x7fcf2c09e000 2023-09-30T22:31:02.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.645+0000 7fcf30e3c700 1 -- 172.21.15.16:0/3511049543 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fcf2c074ea0 con 0x7fcf2c09e000 2023-09-30T22:31:02.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.646+0000 7fcf1affd700 1 -- 172.21.15.16:0/3511049543 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcf2001b340 con 0x7fcf2c09e000 2023-09-30T22:31:02.646 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:31:02.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.648+0000 7fcf30e3c700 1 -- 172.21.15.16:0/3511049543 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcf2c09e000 msgr2=0x7fcf2c120690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:02.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.648+0000 7fcf30e3c700 1 --2- 172.21.15.16:0/3511049543 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcf2c09e000 0x7fcf2c120690 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c0a3e20 tx=0x7fcf2000c750 comp rx=0 tx=0).stop 2023-09-30T22:31:02.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.648+0000 7fcf30e3c700 1 -- 172.21.15.16:0/3511049543 shutdown_connections 2023-09-30T22:31:02.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.648+0000 7fcf30e3c700 1 --2- 172.21.15.16:0/3511049543 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcf2c09e410 0x7fcf2c120bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.648+0000 7fcf30e3c700 1 --2- 172.21.15.16:0/3511049543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf2c09d6f0 0x7fcf2c120150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.648+0000 7fcf30e3c700 1 --2- 172.21.15.16:0/3511049543 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcf2c09e000 0x7fcf2c120690 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:02.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.648+0000 7fcf30e3c700 1 -- 172.21.15.16:0/3511049543 >> 172.21.15.16:0/3511049543 conn(0x7fcf2c066f50 msgr2=0x7fcf2c130b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:02.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.648+0000 7fcf30e3c700 1 -- 172.21.15.16:0/3511049543 shutdown_connections 2023-09-30T22:31:02.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:02.648+0000 7fcf30e3c700 1 -- 172.21.15.16:0/3511049543 wait complete. 2023-09-30T22:31:02.662 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:31:03.295 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:03.325 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:03.364 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:03.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.444+0000 7fd7f2a24700 1 Processor -- start 2023-09-30T22:31:03.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.444+0000 7fd7f2a24700 1 -- start start 2023-09-30T22:31:03.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.444+0000 7fd7f2a24700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd7ec09d6f0 0x7fd7ec09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:03.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.444+0000 7fd7f2a24700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd7ec09e000 0x7fd7ec1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:03.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.444+0000 7fd7f2a24700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd7ec09e410 0x7fd7ec133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:03.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.444+0000 7fd7f2a24700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7ec06b460 con 0x7fd7ec09e000 2023-09-30T22:31:03.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.444+0000 7fd7f2a24700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ec09b120 con 0x7fd7ec09d6f0 2023-09-30T22:31:03.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.444+0000 7fd7f2a24700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ec09a7e0 con 0x7fd7ec09e410 2023-09-30T22:31:03.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.444+0000 7fd7eb7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd7ec09e000 0x7fd7ec1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:03.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.445+0000 7fd7eb7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd7ec09e000 0x7fd7ec1339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55588/0 (socket says 172.21.15.16:55588) 2023-09-30T22:31:03.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.445+0000 7fd7eb7fe700 1 -- 172.21.15.16:0/2319573224 learned_addr learned my addr 172.21.15.16:0/2319573224 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:03.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.445+0000 7fd7f0fc1700 1 --2- 172.21.15.16:0/2319573224 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd7ec09e410 0x7fd7ec133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:03.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.445+0000 7fd7ebfff700 1 --2- 172.21.15.16:0/2319573224 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd7ec09d6f0 0x7fd7ec09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:03.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.445+0000 7fd7eb7fe700 1 -- 172.21.15.16:0/2319573224 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd7ec09e410 msgr2=0x7fd7ec133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:03.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.445+0000 7fd7eb7fe700 1 --2- 172.21.15.16:0/2319573224 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd7ec09e410 0x7fd7ec133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.445+0000 7fd7eb7fe700 1 -- 172.21.15.16:0/2319573224 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd7ec09d6f0 msgr2=0x7fd7ec09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:03.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.445+0000 7fd7eb7fe700 1 --2- 172.21.15.16:0/2319573224 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd7ec09d6f0 0x7fd7ec09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.445+0000 7fd7eb7fe700 1 -- 172.21.15.16:0/2319573224 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7ec09a500 con 0x7fd7ec09e000 2023-09-30T22:31:03.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.445+0000 7fd7f0fc1700 1 --2- 172.21.15.16:0/2319573224 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd7ec09e410 0x7fd7ec133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:03.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.445+0000 7fd7eb7fe700 1 --2- 172.21.15.16:0/2319573224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7ec09e000 0x7fd7ec1339f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd7e00098f0 tx=0x7fd7e0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee413b31c0293755 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:03.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.445+0000 7fd7eaffd700 1 -- 172.21.15.16:0/2319573224 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e0011830 con 0x7fd7ec09e000 2023-09-30T22:31:03.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.445+0000 7fd7eaffd700 1 -- 172.21.15.16:0/2319573224 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7e0011990 con 0x7fd7ec09e000 2023-09-30T22:31:03.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.446+0000 7fd7eaffd700 1 -- 172.21.15.16:0/2319573224 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e0011d20 con 0x7fd7ec09e000 2023-09-30T22:31:03.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.446+0000 7fd7f2a24700 1 -- 172.21.15.16:0/2319573224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7ec09e000 msgr2=0x7fd7ec1339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:03.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.446+0000 7fd7f2a24700 1 --2- 172.21.15.16:0/2319573224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7ec09e000 0x7fd7ec1339f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd7e00098f0 tx=0x7fd7e0009c00 comp rx=0 tx=0).stop 2023-09-30T22:31:03.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.446+0000 7fd7f2a24700 1 -- 172.21.15.16:0/2319573224 shutdown_connections 2023-09-30T22:31:03.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.446+0000 7fd7f2a24700 1 --2- 172.21.15.16:0/2319573224 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd7ec09d6f0 0x7fd7ec09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.446+0000 7fd7f2a24700 1 --2- 172.21.15.16:0/2319573224 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd7ec09e410 0x7fd7ec133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.446+0000 7fd7f2a24700 1 --2- 172.21.15.16:0/2319573224 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7ec09e000 0x7fd7ec1339f0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.446+0000 7fd7f2a24700 1 -- 172.21.15.16:0/2319573224 >> 172.21.15.16:0/2319573224 conn(0x7fd7ec066f50 msgr2=0x7fd7ec09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:03.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.446+0000 7fd7f2a24700 1 -- 172.21.15.16:0/2319573224 shutdown_connections 2023-09-30T22:31:03.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.446+0000 7fd7f2a24700 1 -- 172.21.15.16:0/2319573224 wait complete. 2023-09-30T22:31:03.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.447+0000 7fd7f2a24700 1 Processor -- start 2023-09-30T22:31:03.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.448+0000 7fd7f2a24700 1 -- start start 2023-09-30T22:31:03.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.448+0000 7fd7f2a24700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd7ec09d6f0 0x7fd7ec1a8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:03.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.448+0000 7fd7f2a24700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7ec09e000 0x7fd7ec1a8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:03.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.448+0000 7fd7f2a24700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd7ec09e410 0x7fd7ec1a8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:03.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.448+0000 7fd7f2a24700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ec1a9620 con 0x7fd7ec09e000 2023-09-30T22:31:03.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.448+0000 7fd7f2a24700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ec1bb4b0 con 0x7fd7ec09e410 2023-09-30T22:31:03.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.448+0000 7fd7f2a24700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ec1aa2f0 con 0x7fd7ec09d6f0 2023-09-30T22:31:03.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.448+0000 7fd7eb7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7ec09e000 0x7fd7ec1a8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:03.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.448+0000 7fd7f0fc1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd7ec09e410 0x7fd7ec1a8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:03.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.448+0000 7fd7eb7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7ec09e000 0x7fd7ec1a8690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55602/0 (socket says 172.21.15.16:55602) 2023-09-30T22:31:03.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.448+0000 7fd7eb7fe700 1 -- 172.21.15.16:0/18631602 learned_addr learned my addr 172.21.15.16:0/18631602 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:03.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.448+0000 7fd7f0fc1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd7ec09e410 0x7fd7ec1a8bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38518/0 (socket says 172.21.15.16:38518) 2023-09-30T22:31:03.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.448+0000 7fd7ebfff700 1 --2- 172.21.15.16:0/18631602 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd7ec09d6f0 0x7fd7ec1a8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:03.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.449+0000 7fd7f0fc1700 1 -- 172.21.15.16:0/18631602 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd7ec09d6f0 msgr2=0x7fd7ec1a8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:03.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.449+0000 7fd7f0fc1700 1 --2- 172.21.15.16:0/18631602 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd7ec09d6f0 0x7fd7ec1a8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.449+0000 7fd7f0fc1700 1 -- 172.21.15.16:0/18631602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7ec09e000 msgr2=0x7fd7ec1a8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:03.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.449+0000 7fd7f0fc1700 1 --2- 172.21.15.16:0/18631602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7ec09e000 0x7fd7ec1a8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.449+0000 7fd7f0fc1700 1 -- 172.21.15.16:0/18631602 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7e4009a70 con 0x7fd7ec09e410 2023-09-30T22:31:03.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.449+0000 7fd7eb7fe700 1 --2- 172.21.15.16:0/18631602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7ec09e000 0x7fd7ec1a8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:03.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.449+0000 7fd7f0fc1700 1 --2- 172.21.15.16:0/18631602 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd7ec09e410 0x7fd7ec1a8bd0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd7e4009e50 tx=0x7fd7e400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:03.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.449+0000 7fd7e8ff9700 1 -- 172.21.15.16:0/18631602 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e400ca90 con 0x7fd7ec09e410 2023-09-30T22:31:03.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.449+0000 7fd7e8ff9700 1 -- 172.21.15.16:0/18631602 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7e400f040 con 0x7fd7ec09e410 2023-09-30T22:31:03.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.449+0000 7fd7f2a24700 1 -- 172.21.15.16:0/18631602 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7e0009550 con 0x7fd7ec09e410 2023-09-30T22:31:03.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.449+0000 7fd7e8ff9700 1 -- 172.21.15.16:0/18631602 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e400ce50 con 0x7fd7ec09e410 2023-09-30T22:31:03.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.449+0000 7fd7f2a24700 1 -- 172.21.15.16:0/18631602 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7ec0a21d0 con 0x7fd7ec09e410 2023-09-30T22:31:03.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.450+0000 7fd7e8ff9700 1 -- 172.21.15.16:0/18631602 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd7e4014480 con 0x7fd7ec09e410 2023-09-30T22:31:03.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.450+0000 7fd7f2a24700 1 -- 172.21.15.16:0/18631602 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7ec09c710 con 0x7fd7ec09e410 2023-09-30T22:31:03.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.450+0000 7fd7e8ff9700 1 -- 172.21.15.16:0/18631602 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd7e403fcb0 con 0x7fd7ec09e410 2023-09-30T22:31:03.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.455+0000 7fd7e8ff9700 1 -- 172.21.15.16:0/18631602 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd7e4012930 con 0x7fd7ec09e410 2023-09-30T22:31:03.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.482+0000 7f571ef35700 1 Processor -- start 2023-09-30T22:31:03.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.482+0000 7f571ef35700 1 -- start start 2023-09-30T22:31:03.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.483+0000 7f571ef35700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f57181305f0 0x7f57181309c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:03.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.483+0000 7f571ef35700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5718070190 0x7f5718138540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:03.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.483+0000 7f571ef35700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57181301c0 con 0x7f571806f660 2023-09-30T22:31:03.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.483+0000 7f571ef35700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f571809a2e0 con 0x7f57181305f0 2023-09-30T22:31:03.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.483+0000 7f571ef35700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5718071ff0 con 0x7f5718070190 2023-09-30T22:31:03.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.483+0000 7f5717fff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f571806f660 0x7f571806fa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38656/0 (socket says 172.21.15.16:38656) 2023-09-30T22:31:03.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.483+0000 7f5717fff700 1 -- 172.21.15.16:0/3361093595 learned_addr learned my addr 172.21.15.16:0/3361093595 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:31:03.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.483+0000 7f571d4d2700 1 --2- 172.21.15.16:0/3361093595 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5718070190 0x7f5718138540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:03.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.483+0000 7f571ccd1700 1 --2- 172.21.15.16:0/3361093595 >> v2:172.21.15.16:3300/0 conn(0x7f57181305f0 0x7f57181309c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:03.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.484+0000 7f571d4d2700 1 -- 172.21.15.16:0/3361093595 >> v2:172.21.15.16:3300/0 conn(0x7f57181305f0 msgr2=0x7f57181309c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:03.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.484+0000 7f571d4d2700 1 --2- 172.21.15.16:0/3361093595 >> v2:172.21.15.16:3300/0 conn(0x7f57181305f0 0x7f57181309c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.484+0000 7f571d4d2700 1 -- 172.21.15.16:0/3361093595 >> v1:172.21.15.16:6789/0 conn(0x7f571806f660 legacy=0x7f571806fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:03.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.484+0000 7f571d4d2700 1 -- 172.21.15.16:0/3361093595 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5718070e70 con 0x7f5718070190 2023-09-30T22:31:03.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.484+0000 7f57177fe700 1 -- 172.21.15.16:0/3361093595 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3963850927 0 0) 0x7f57181301c0 con 0x7f571806f660 2023-09-30T22:31:03.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.484+0000 7f571d4d2700 1 --2- 172.21.15.16:0/3361093595 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5718070190 0x7f5718138540 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5710009dc0 tx=0x7f571000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c18ee8d41487069 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:03.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.484+0000 7f57177fe700 1 -- 172.21.15.16:0/3361093595 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f571000e070 con 0x7f5718070190 2023-09-30T22:31:03.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.484+0000 7f57177fe700 1 -- 172.21.15.16:0/3361093595 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f571000bae0 con 0x7f5718070190 2023-09-30T22:31:03.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.484+0000 7f57177fe700 1 -- 172.21.15.16:0/3361093595 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f571000be90 con 0x7f5718070190 2023-09-30T22:31:03.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.485+0000 7f571ef35700 1 -- 172.21.15.16:0/3361093595 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5718070190 msgr2=0x7f5718138540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:03.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.485+0000 7f571ef35700 1 --2- 172.21.15.16:0/3361093595 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5718070190 0x7f5718138540 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5710009dc0 tx=0x7f571000b330 comp rx=0 tx=0).stop 2023-09-30T22:31:03.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.485+0000 7f571ef35700 1 -- 172.21.15.16:0/3361093595 shutdown_connections 2023-09-30T22:31:03.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.485+0000 7f571ef35700 1 --2- 172.21.15.16:0/3361093595 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5718070190 0x7f5718138540 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.485+0000 7f571ef35700 1 --2- 172.21.15.16:0/3361093595 >> v2:172.21.15.16:3300/0 conn(0x7f57181305f0 0x7f57181309c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.485+0000 7f571ef35700 1 -- 172.21.15.16:0/3361093595 >> 172.21.15.16:0/3361093595 conn(0x7f57180a2aa0 msgr2=0x7f57180a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:03.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.485+0000 7f571ef35700 1 -- 172.21.15.16:0/3361093595 shutdown_connections 2023-09-30T22:31:03.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.485+0000 7f571ef35700 1 -- 172.21.15.16:0/3361093595 wait complete. 2023-09-30T22:31:03.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.486+0000 7f571ef35700 1 Processor -- start 2023-09-30T22:31:03.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.486+0000 7f571ef35700 1 -- start start 2023-09-30T22:31:03.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.487+0000 7f571ef35700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f571806f660 0x7f5718138570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:03.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.487+0000 7f571ef35700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5718070190 0x7f5718138ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:03.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.487+0000 7f571ef35700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f57181305f0 0x7f57181c5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:03.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.487+0000 7f571ef35700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57181372d0 con 0x7f571806f660 2023-09-30T22:31:03.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.487+0000 7f571ef35700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57180a0d70 con 0x7f5718070190 2023-09-30T22:31:03.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.487+0000 7f571ef35700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f57181374d0 con 0x7f57181305f0 2023-09-30T22:31:03.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.488+0000 7f571ccd1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f571806f660 0x7f5718138570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:03.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.488+0000 7f5717fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5718070190 0x7f5718138ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:03.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.488+0000 7f571ccd1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f571806f660 0x7f5718138570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55622/0 (socket says 172.21.15.16:55622) 2023-09-30T22:31:03.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.488+0000 7f571ccd1700 1 -- 172.21.15.16:0/3196315238 learned_addr learned my addr 172.21.15.16:0/3196315238 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:03.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.488+0000 7f571d4d2700 1 --2- 172.21.15.16:0/3196315238 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f57181305f0 0x7f57181c5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:03.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.488+0000 7f571ccd1700 1 -- 172.21.15.16:0/3196315238 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f57181305f0 msgr2=0x7f57181c5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:03.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.488+0000 7f571ccd1700 1 --2- 172.21.15.16:0/3196315238 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f57181305f0 0x7f57181c5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.488+0000 7f571ccd1700 1 -- 172.21.15.16:0/3196315238 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5718070190 msgr2=0x7f5718138ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:03.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.488+0000 7f571ccd1700 1 --2- 172.21.15.16:0/3196315238 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5718070190 0x7f5718138ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.488+0000 7f571ccd1700 1 -- 172.21.15.16:0/3196315238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5710009a70 con 0x7f571806f660 2023-09-30T22:31:03.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.488+0000 7f571ccd1700 1 --2- 172.21.15.16:0/3196315238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f571806f660 0x7f5718138570 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f570c00e9b0 tx=0x7f570c00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:03.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.488+0000 7f57157fa700 1 -- 172.21.15.16:0/3196315238 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f570c009710 con 0x7f571806f660 2023-09-30T22:31:03.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.488+0000 7f57157fa700 1 -- 172.21.15.16:0/3196315238 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f570c00ee50 con 0x7f571806f660 2023-09-30T22:31:03.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.488+0000 7f571ef35700 1 -- 172.21.15.16:0/3196315238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57181355c0 con 0x7f571806f660 2023-09-30T22:31:03.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.488+0000 7f57157fa700 1 -- 172.21.15.16:0/3196315238 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f570c013a60 con 0x7f571806f660 2023-09-30T22:31:03.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.489+0000 7f571ef35700 1 -- 172.21.15.16:0/3196315238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5718135e90 con 0x7f571806f660 2023-09-30T22:31:03.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.489+0000 7f57157fa700 1 -- 172.21.15.16:0/3196315238 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f570c013bc0 con 0x7f571806f660 2023-09-30T22:31:03.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.490+0000 7f57157fa700 1 -- 172.21.15.16:0/3196315238 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f570c043660 con 0x7f571806f660 2023-09-30T22:31:03.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.490+0000 7f571ef35700 1 -- 172.21.15.16:0/3196315238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5718074ea0 con 0x7f571806f660 2023-09-30T22:31:03.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.499+0000 7f57157fa700 1 -- 172.21.15.16:0/3196315238 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f570c020020 con 0x7f571806f660 2023-09-30T22:31:03.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.531+0000 7f2b362c9700 1 Processor -- start 2023-09-30T22:31:03.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.531+0000 7f2b362c9700 1 -- start start 2023-09-30T22:31:03.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.531+0000 7f2b362c9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2b3009d6f0 0x7f2b3009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:03.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.531+0000 7f2b362c9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2b3009e000 0x7f2b30130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:03.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.531+0000 7f2b362c9700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2b3009e410 0x7f2b30130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:03.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.531+0000 7f2b362c9700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b3009f520 con 0x7f2b3009e410 2023-09-30T22:31:03.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.531+0000 7f2b362c9700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3009f130 con 0x7f2b3009d6f0 2023-09-30T22:31:03.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.531+0000 7f2b362c9700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3009e930 con 0x7f2b3009e000 2023-09-30T22:31:03.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.531+0000 7f2b2ffff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2b3009d6f0 0x7f2b3009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:03.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.531+0000 7f2b2ffff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2b3009d6f0 0x7f2b3009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38540/0 (socket says 172.21.15.16:38540) 2023-09-30T22:31:03.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.531+0000 7f2b2ffff700 1 -- 172.21.15.16:0/2449313373 learned_addr learned my addr 172.21.15.16:0/2449313373 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:03.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.531+0000 7f2b34866700 1 --2- 172.21.15.16:0/2449313373 >> v2:172.21.15.16:3300/0 conn(0x7f2b3009e410 0x7f2b30130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:03.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.532+0000 7f2b2ffff700 1 -- 172.21.15.16:0/2449313373 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2b3009e000 msgr2=0x7f2b30130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:03.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.532+0000 7f2b2ffff700 1 --2- 172.21.15.16:0/2449313373 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2b3009e000 0x7f2b30130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.532+0000 7f2b2ffff700 1 -- 172.21.15.16:0/2449313373 >> v2:172.21.15.16:3300/0 conn(0x7f2b3009e410 msgr2=0x7f2b30130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:03.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.532+0000 7f2b2ffff700 1 --2- 172.21.15.16:0/2449313373 >> v2:172.21.15.16:3300/0 conn(0x7f2b3009e410 0x7f2b30130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.532+0000 7f2b2ffff700 1 -- 172.21.15.16:0/2449313373 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b3009b2d0 con 0x7f2b3009d6f0 2023-09-30T22:31:03.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.532+0000 7f2b34866700 1 --2- 172.21.15.16:0/2449313373 >> v2:172.21.15.16:3300/0 conn(0x7f2b3009e410 0x7f2b30130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:03.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.532+0000 7f2b2ffff700 1 --2- 172.21.15.16:0/2449313373 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2b3009d6f0 0x7f2b3009dac0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2b20009920 tx=0x7f2b20009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1bed65a88de0fc4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:03.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.532+0000 7f2b2effd700 1 -- 172.21.15.16:0/2449313373 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b20011830 con 0x7f2b3009d6f0 2023-09-30T22:31:03.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.532+0000 7f2b2effd700 1 -- 172.21.15.16:0/2449313373 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b20011990 con 0x7f2b3009d6f0 2023-09-30T22:31:03.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.532+0000 7f2b2effd700 1 -- 172.21.15.16:0/2449313373 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b20011d20 con 0x7f2b3009d6f0 2023-09-30T22:31:03.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.533+0000 7f2b362c9700 1 -- 172.21.15.16:0/2449313373 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2b3009d6f0 msgr2=0x7f2b3009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:03.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.533+0000 7f2b362c9700 1 --2- 172.21.15.16:0/2449313373 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2b3009d6f0 0x7f2b3009dac0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2b20009920 tx=0x7f2b20009c30 comp rx=0 tx=0).stop 2023-09-30T22:31:03.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.533+0000 7f2b362c9700 1 -- 172.21.15.16:0/2449313373 shutdown_connections 2023-09-30T22:31:03.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.533+0000 7f2b362c9700 1 --2- 172.21.15.16:0/2449313373 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2b3009d6f0 0x7f2b3009dac0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.533+0000 7f2b362c9700 1 --2- 172.21.15.16:0/2449313373 >> v2:172.21.15.16:3300/0 conn(0x7f2b3009e410 0x7f2b30130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.533+0000 7f2b362c9700 1 --2- 172.21.15.16:0/2449313373 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2b3009e000 0x7f2b30130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.533+0000 7f2b362c9700 1 -- 172.21.15.16:0/2449313373 >> 172.21.15.16:0/2449313373 conn(0x7f2b30066f50 msgr2=0x7f2b3009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:03.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.533+0000 7f2b362c9700 1 -- 172.21.15.16:0/2449313373 shutdown_connections 2023-09-30T22:31:03.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.533+0000 7f2b362c9700 1 -- 172.21.15.16:0/2449313373 wait complete. 2023-09-30T22:31:03.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.534+0000 7f2b362c9700 1 Processor -- start 2023-09-30T22:31:03.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.535+0000 7f2b362c9700 1 -- start start 2023-09-30T22:31:03.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.535+0000 7f2b362c9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2b3009d6f0 0x7f2b3006e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:03.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.535+0000 7f2b362c9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2b3009e000 0x7f2b3006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:03.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.535+0000 7f2b362c9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b3009e410 0x7f2b3006f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:03.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.535+0000 7f2b362c9700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b30124870 con 0x7f2b3009e410 2023-09-30T22:31:03.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.535+0000 7f2b362c9700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b30125f20 con 0x7f2b3009d6f0 2023-09-30T22:31:03.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.535+0000 7f2b362c9700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2b30070480 con 0x7f2b3009e000 2023-09-30T22:31:03.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.535+0000 7f2b2ffff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2b3009d6f0 0x7f2b3006e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:03.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.535+0000 7f2b2f7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2b3009e000 0x7f2b3006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:03.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.535+0000 7f2b2ffff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2b3009d6f0 0x7f2b3006e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38556/0 (socket says 172.21.15.16:38556) 2023-09-30T22:31:03.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.535+0000 7f2b2f7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2b3009e000 0x7f2b3006edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:58006/0 (socket says 172.21.15.16:58006) 2023-09-30T22:31:03.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.535+0000 7f2b2ffff700 1 -- 172.21.15.16:0/2208018561 learned_addr learned my addr 172.21.15.16:0/2208018561 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:03.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.535+0000 7f2b34866700 1 --2- 172.21.15.16:0/2208018561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b3009e410 0x7f2b3006f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:03.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.536+0000 7f2b2ffff700 1 -- 172.21.15.16:0/2208018561 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2b3009e000 msgr2=0x7f2b3006edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:03.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.536+0000 7f2b2ffff700 1 --2- 172.21.15.16:0/2208018561 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2b3009e000 0x7f2b3006edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.536+0000 7f2b2ffff700 1 -- 172.21.15.16:0/2208018561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b3009e410 msgr2=0x7f2b3006f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:03.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.536+0000 7f2b2ffff700 1 --2- 172.21.15.16:0/2208018561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b3009e410 0x7f2b3006f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.536+0000 7f2b2ffff700 1 -- 172.21.15.16:0/2208018561 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b28009a70 con 0x7f2b3009d6f0 2023-09-30T22:31:03.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.536+0000 7f2b34866700 1 --2- 172.21.15.16:0/2208018561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b3009e410 0x7f2b3006f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:31:03.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.536+0000 7f2b2ffff700 1 --2- 172.21.15.16:0/2208018561 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2b3009d6f0 0x7f2b3006e870 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2b20000c00 tx=0x7f2b20004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:03.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.536+0000 7f2b2cff9700 1 -- 172.21.15.16:0/2208018561 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b20004380 con 0x7f2b3009d6f0 2023-09-30T22:31:03.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.536+0000 7f2b2cff9700 1 -- 172.21.15.16:0/2208018561 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b2001f070 con 0x7f2b3009d6f0 2023-09-30T22:31:03.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.536+0000 7f2b362c9700 1 -- 172.21.15.16:0/2208018561 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b20009550 con 0x7f2b3009d6f0 2023-09-30T22:31:03.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.536+0000 7f2b2cff9700 1 -- 172.21.15.16:0/2208018561 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b200036a0 con 0x7f2b3009d6f0 2023-09-30T22:31:03.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.536+0000 7f2b362c9700 1 -- 172.21.15.16:0/2208018561 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b301259d0 con 0x7f2b3009d6f0 2023-09-30T22:31:03.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.537+0000 7f2b2cff9700 1 -- 172.21.15.16:0/2208018561 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2b20021030 con 0x7f2b3009d6f0 2023-09-30T22:31:03.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.537+0000 7f2b2cff9700 1 -- 172.21.15.16:0/2208018561 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f2b20040840 con 0x7f2b3009d6f0 2023-09-30T22:31:03.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.537+0000 7f2b362c9700 1 -- 172.21.15.16:0/2208018561 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b30074ea0 con 0x7f2b3009d6f0 2023-09-30T22:31:03.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.542+0000 7f2b2cff9700 1 -- 172.21.15.16:0/2208018561 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2b20015080 con 0x7f2b3009d6f0 2023-09-30T22:31:03.663 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:03.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.669+0000 7fd7f2a24700 1 -- 172.21.15.16:0/18631602 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd7ec074ea0 con 0x7fd7ec09e410 2023-09-30T22:31:03.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.669+0000 7fd7e8ff9700 1 -- 172.21.15.16:0/18631602 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd7e4012340 con 0x7fd7ec09e410 2023-09-30T22:31:03.670 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:31:03.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.672+0000 7fd7f2a24700 1 -- 172.21.15.16:0/18631602 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd7ec09e410 msgr2=0x7fd7ec1a8bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:03.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.672+0000 7fd7f2a24700 1 --2- 172.21.15.16:0/18631602 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd7ec09e410 0x7fd7ec1a8bd0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd7e4009e50 tx=0x7fd7e400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.672+0000 7fd7f2a24700 1 -- 172.21.15.16:0/18631602 shutdown_connections 2023-09-30T22:31:03.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.672+0000 7fd7f2a24700 1 --2- 172.21.15.16:0/18631602 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd7ec09e410 0x7fd7ec1a8bd0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.672+0000 7fd7f2a24700 1 --2- 172.21.15.16:0/18631602 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd7ec09d6f0 0x7fd7ec1a8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.672+0000 7fd7f2a24700 1 --2- 172.21.15.16:0/18631602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7ec09e000 0x7fd7ec1a8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.672+0000 7fd7f2a24700 1 -- 172.21.15.16:0/18631602 >> 172.21.15.16:0/18631602 conn(0x7fd7ec066f50 msgr2=0x7fd7ec06f420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:03.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.672+0000 7fd7f2a24700 1 -- 172.21.15.16:0/18631602 shutdown_connections 2023-09-30T22:31:03.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.672+0000 7fd7f2a24700 1 -- 172.21.15.16:0/18631602 wait complete. 2023-09-30T22:31:03.687 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:31:03.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.713+0000 7f571ef35700 1 -- 172.21.15.16:0/3196315238 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f571806b9e0 con 0x7f571806f660 2023-09-30T22:31:03.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.714+0000 7f57157fa700 1 -- 172.21.15.16:0/3196315238 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f570c01b440 con 0x7f571806f660 2023-09-30T22:31:03.714 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:31:03.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.716+0000 7f571ef35700 1 -- 172.21.15.16:0/3196315238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f571806f660 msgr2=0x7f5718138570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:03.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.716+0000 7f571ef35700 1 --2- 172.21.15.16:0/3196315238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f571806f660 0x7f5718138570 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f570c00e9b0 tx=0x7f570c00ecc0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.716+0000 7f571ef35700 1 -- 172.21.15.16:0/3196315238 shutdown_connections 2023-09-30T22:31:03.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.716+0000 7f571ef35700 1 --2- 172.21.15.16:0/3196315238 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f57181305f0 0x7f57181c5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.716+0000 7f571ef35700 1 --2- 172.21.15.16:0/3196315238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f571806f660 0x7f5718138570 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.716+0000 7f571ef35700 1 --2- 172.21.15.16:0/3196315238 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5718070190 0x7f5718138ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.716+0000 7f571ef35700 1 -- 172.21.15.16:0/3196315238 >> 172.21.15.16:0/3196315238 conn(0x7f57180a2aa0 msgr2=0x7f571812fad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:03.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.716+0000 7f571ef35700 1 -- 172.21.15.16:0/3196315238 shutdown_connections 2023-09-30T22:31:03.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.716+0000 7f571ef35700 1 -- 172.21.15.16:0/3196315238 wait complete. 2023-09-30T22:31:03.730 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:31:03.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.753+0000 7f2b362c9700 1 -- 172.21.15.16:0/2208018561 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f2b3006c360 con 0x7f2b3009d6f0 2023-09-30T22:31:03.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.754+0000 7f2b2cff9700 1 -- 172.21.15.16:0/2208018561 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2b20003800 con 0x7f2b3009d6f0 2023-09-30T22:31:03.754 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:31:03.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.756+0000 7f2b362c9700 1 -- 172.21.15.16:0/2208018561 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2b3009d6f0 msgr2=0x7f2b3006e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:03.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.756+0000 7f2b362c9700 1 --2- 172.21.15.16:0/2208018561 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2b3009d6f0 0x7f2b3006e870 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2b20000c00 tx=0x7f2b20004750 comp rx=0 tx=0).stop 2023-09-30T22:31:03.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.756+0000 7f2b362c9700 1 -- 172.21.15.16:0/2208018561 shutdown_connections 2023-09-30T22:31:03.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.756+0000 7f2b362c9700 1 --2- 172.21.15.16:0/2208018561 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2b3009d6f0 0x7f2b3006e870 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.756+0000 7f2b362c9700 1 --2- 172.21.15.16:0/2208018561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b3009e410 0x7f2b3006f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.756+0000 7f2b362c9700 1 --2- 172.21.15.16:0/2208018561 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2b3009e000 0x7f2b3006edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.756+0000 7f2b362c9700 1 -- 172.21.15.16:0/2208018561 >> 172.21.15.16:0/2208018561 conn(0x7f2b30066f50 msgr2=0x7f2b3006a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:03.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.756+0000 7f2b362c9700 1 -- 172.21.15.16:0/2208018561 shutdown_connections 2023-09-30T22:31:03.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.756+0000 7f2b362c9700 1 -- 172.21.15.16:0/2208018561 wait complete. 2023-09-30T22:31:03.771 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:31:03.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.808+0000 7f4b041ab700 1 Processor -- start 2023-09-30T22:31:03.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.809+0000 7f4b041ab700 1 -- start start 2023-09-30T22:31:03.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.809+0000 7f4b041ab700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4afc09fa10 0x7f4afc136950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:03.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.809+0000 7f4b041ab700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4afc09f2c0 0x7f4afc138ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:03.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.809+0000 7f4b041ab700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4afc09b610 con 0x7f4afc09e7e0 2023-09-30T22:31:03.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.809+0000 7f4b041ab700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4afc09bbb0 con 0x7f4afc09f2c0 2023-09-30T22:31:03.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.809+0000 7f4b041ab700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4afc0a0010 con 0x7f4afc09fa10 2023-09-30T22:31:03.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.809+0000 7f4b01f47700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4afc09e7e0 0x7f4afc09ebb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38660/0 (socket says 172.21.15.16:38660) 2023-09-30T22:31:03.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.809+0000 7f4b01f47700 1 -- 172.21.15.16:0/1974898625 learned_addr learned my addr 172.21.15.16:0/1974898625 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:31:03.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.809+0000 7f4b01746700 1 --2- 172.21.15.16:0/1974898625 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4afc09fa10 0x7f4afc136950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:03.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.809+0000 7f4b02748700 1 --2- 172.21.15.16:0/1974898625 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4afc09f2c0 0x7f4afc138ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:03.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.810+0000 7f4b01746700 1 -- 172.21.15.16:0/1974898625 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4afc09f2c0 msgr2=0x7f4afc138ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:03.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.810+0000 7f4b00f45700 1 -- 172.21.15.16:0/1974898625 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3289878489 0 0) 0x7f4afc09b610 con 0x7f4afc09e7e0 2023-09-30T22:31:03.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.810+0000 7f4b01746700 1 --2- 172.21.15.16:0/1974898625 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4afc09f2c0 0x7f4afc138ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.810+0000 7f4b01746700 1 -- 172.21.15.16:0/1974898625 >> v1:172.21.15.16:6789/0 conn(0x7f4afc09e7e0 legacy=0x7f4afc09ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:03.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.810+0000 7f4b01746700 1 -- 172.21.15.16:0/1974898625 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4afc09b8d0 con 0x7f4afc09fa10 2023-09-30T22:31:03.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.810+0000 7f4b02748700 1 --2- 172.21.15.16:0/1974898625 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4afc09f2c0 0x7f4afc138ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:03.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.810+0000 7f4b01746700 1 --2- 172.21.15.16:0/1974898625 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4afc09fa10 0x7f4afc136950 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4aec009870 tx=0x7f4aec009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92d0c8f74733202f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:03.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.810+0000 7f4b00f45700 1 -- 172.21.15.16:0/1974898625 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aec011830 con 0x7f4afc09fa10 2023-09-30T22:31:03.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.810+0000 7f4b00f45700 1 -- 172.21.15.16:0/1974898625 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aec011990 con 0x7f4afc09fa10 2023-09-30T22:31:03.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.810+0000 7f4b00f45700 1 -- 172.21.15.16:0/1974898625 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aec011d20 con 0x7f4afc09fa10 2023-09-30T22:31:03.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.811+0000 7f4b041ab700 1 -- 172.21.15.16:0/1974898625 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4afc09fa10 msgr2=0x7f4afc136950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:03.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.811+0000 7f4b041ab700 1 --2- 172.21.15.16:0/1974898625 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4afc09fa10 0x7f4afc136950 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4aec009870 tx=0x7f4aec009b80 comp rx=0 tx=0).stop 2023-09-30T22:31:03.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.811+0000 7f4b041ab700 1 -- 172.21.15.16:0/1974898625 shutdown_connections 2023-09-30T22:31:03.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.811+0000 7f4b041ab700 1 --2- 172.21.15.16:0/1974898625 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4afc09f2c0 0x7f4afc138ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.811+0000 7f4b041ab700 1 --2- 172.21.15.16:0/1974898625 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4afc09fa10 0x7f4afc136950 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.811+0000 7f4b041ab700 1 -- 172.21.15.16:0/1974898625 >> 172.21.15.16:0/1974898625 conn(0x7f4afc0a4300 msgr2=0x7f4afc0a4700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:03.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.811+0000 7f4b041ab700 1 -- 172.21.15.16:0/1974898625 shutdown_connections 2023-09-30T22:31:03.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.811+0000 7f4b041ab700 1 -- 172.21.15.16:0/1974898625 wait complete. 2023-09-30T22:31:03.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.812+0000 7f4b041ab700 1 Processor -- start 2023-09-30T22:31:03.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.812+0000 7f4b041ab700 1 -- start start 2023-09-30T22:31:03.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.812+0000 7f4b041ab700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4afc09e7e0 0x7f4afc11a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:03.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.812+0000 7f4b041ab700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4afc09f2c0 0x7f4afc11aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:03.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.812+0000 7f4b041ab700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4afc09fa10 0x7f4afc11afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:03.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.812+0000 7f4b041ab700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4afc11db30 con 0x7f4afc09fa10 2023-09-30T22:31:03.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.812+0000 7f4b041ab700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4afc11b890 con 0x7f4afc09e7e0 2023-09-30T22:31:03.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.812+0000 7f4b041ab700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4afc09d9c0 con 0x7f4afc09f2c0 2023-09-30T22:31:03.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.812+0000 7f4b01f47700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4afc09e7e0 0x7f4afc11a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:03.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.812+0000 7f4b02748700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4afc09fa10 0x7f4afc11afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:03.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.813+0000 7f4b01f47700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4afc09e7e0 0x7f4afc11a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38580/0 (socket says 172.21.15.16:38580) 2023-09-30T22:31:03.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.813+0000 7f4b01746700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4afc09f2c0 0x7f4afc11aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:03.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.813+0000 7f4b01f47700 1 -- 172.21.15.16:0/1146744584 learned_addr learned my addr 172.21.15.16:0/1146744584 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:03.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.813+0000 7f4b01746700 1 -- 172.21.15.16:0/1146744584 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4afc09e7e0 msgr2=0x7f4afc11a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:03.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.813+0000 7f4b01746700 1 --2- 172.21.15.16:0/1146744584 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4afc09e7e0 0x7f4afc11a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.813+0000 7f4b01746700 1 -- 172.21.15.16:0/1146744584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4afc09fa10 msgr2=0x7f4afc11afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:03.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.813+0000 7f4b01746700 1 --2- 172.21.15.16:0/1146744584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4afc09fa10 0x7f4afc11afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:03.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.813+0000 7f4b01746700 1 -- 172.21.15.16:0/1146744584 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af4009a70 con 0x7f4afc09f2c0 2023-09-30T22:31:03.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.813+0000 7f4b01f47700 1 --2- 172.21.15.16:0/1146744584 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4afc09e7e0 0x7f4afc11a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:03.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.813+0000 7f4b01746700 1 --2- 172.21.15.16:0/1146744584 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4afc09f2c0 0x7f4afc11aa70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4aec016000 tx=0x7f4aec004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:03.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.813+0000 7f4af27fc700 1 -- 172.21.15.16:0/1146744584 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aec0044d0 con 0x7f4afc09f2c0 2023-09-30T22:31:03.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.813+0000 7f4af27fc700 1 -- 172.21.15.16:0/1146744584 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aec0036a0 con 0x7f4afc09f2c0 2023-09-30T22:31:03.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.813+0000 7f4b041ab700 1 -- 172.21.15.16:0/1146744584 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4aec0094a0 con 0x7f4afc09f2c0 2023-09-30T22:31:03.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.813+0000 7f4af27fc700 1 -- 172.21.15.16:0/1146744584 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aec003a30 con 0x7f4afc09f2c0 2023-09-30T22:31:03.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.813+0000 7f4b041ab700 1 -- 172.21.15.16:0/1146744584 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4afc11d220 con 0x7f4afc09f2c0 2023-09-30T22:31:03.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.814+0000 7f4af27fc700 1 -- 172.21.15.16:0/1146744584 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4aec003b90 con 0x7f4afc09f2c0 2023-09-30T22:31:03.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.814+0000 7f4af27fc700 1 -- 172.21.15.16:0/1146744584 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f4aec040af0 con 0x7f4afc09f2c0 2023-09-30T22:31:03.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.814+0000 7f4b041ab700 1 -- 172.21.15.16:0/1146744584 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4afc11d410 con 0x7f4afc09f2c0 2023-09-30T22:31:03.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:03.819+0000 7f4af27fc700 1 -- 172.21.15.16:0/1146744584 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4aec003e40 con 0x7f4afc09f2c0 2023-09-30T22:31:04.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.026+0000 7f4b041ab700 1 -- 172.21.15.16:0/1146744584 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4afc04f2a0 con 0x7f4afc09f2c0 2023-09-30T22:31:04.026 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:31:04.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.026+0000 7f4af27fc700 1 -- 172.21.15.16:0/1146744584 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4aec015020 con 0x7f4afc09f2c0 2023-09-30T22:31:04.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.028+0000 7f4b041ab700 1 -- 172.21.15.16:0/1146744584 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4afc09f2c0 msgr2=0x7f4afc11aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:04.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.028+0000 7f4b041ab700 1 --2- 172.21.15.16:0/1146744584 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4afc09f2c0 0x7f4afc11aa70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4aec016000 tx=0x7f4aec004750 comp rx=0 tx=0).stop 2023-09-30T22:31:04.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.028+0000 7f4b041ab700 1 -- 172.21.15.16:0/1146744584 shutdown_connections 2023-09-30T22:31:04.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.028+0000 7f4b041ab700 1 --2- 172.21.15.16:0/1146744584 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4afc09e7e0 0x7f4afc11a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:04.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.028+0000 7f4b041ab700 1 --2- 172.21.15.16:0/1146744584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4afc09fa10 0x7f4afc11afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:04.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.028+0000 7f4b041ab700 1 --2- 172.21.15.16:0/1146744584 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4afc09f2c0 0x7f4afc11aa70 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:04.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.028+0000 7f4b041ab700 1 -- 172.21.15.16:0/1146744584 >> 172.21.15.16:0/1146744584 conn(0x7f4afc0a4300 msgr2=0x7f4afc1349e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:04.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.028+0000 7f4b041ab700 1 -- 172.21.15.16:0/1146744584 shutdown_connections 2023-09-30T22:31:04.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.028+0000 7f4b041ab700 1 -- 172.21.15.16:0/1146744584 wait complete. 2023-09-30T22:31:04.042 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:31:04.689 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:04.731 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:04.772 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:04.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.841+0000 7f5e35638700 1 Processor -- start 2023-09-30T22:31:04.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.841+0000 7f5e35638700 1 -- start start 2023-09-30T22:31:04.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.842+0000 7f5e35638700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5e3009e930 0x7f5e3009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:04.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.842+0000 7f5e35638700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5e3009f650 0x7f5e30131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:04.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.842+0000 7f5e35638700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e30070940 con 0x7f5e3009f240 2023-09-30T22:31:04.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.842+0000 7f5e35638700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e3006fd60 con 0x7f5e3009e930 2023-09-30T22:31:04.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.842+0000 7f5e35638700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3006fa80 con 0x7f5e3009f650 2023-09-30T22:31:04.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.842+0000 7f5e2f7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5e3009f650 0x7f5e30131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:04.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.842+0000 7f5e267fc700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5e3009f240 0x7f5e301317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38670/0 (socket says 172.21.15.16:38670) 2023-09-30T22:31:04.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.842+0000 7f5e267fc700 1 -- 172.21.15.16:0/2491316984 learned_addr learned my addr 172.21.15.16:0/2491316984 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:31:04.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.842+0000 7f5e2effd700 1 --2- 172.21.15.16:0/2491316984 >> v2:172.21.15.16:3300/0 conn(0x7f5e3009e930 0x7f5e3009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:04.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.842+0000 7f5e2e7fc700 1 -- 172.21.15.16:0/2491316984 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2542040912 0 0) 0x7f5e30070940 con 0x7f5e3009f240 2023-09-30T22:31:04.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.842+0000 7f5e2e7fc700 1 -- 172.21.15.16:0/2491316984 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e1c0035b0 con 0x7f5e3009f240 2023-09-30T22:31:04.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.843+0000 7f5e2f7fe700 1 -- 172.21.15.16:0/2491316984 >> v2:172.21.15.16:3300/0 conn(0x7f5e3009e930 msgr2=0x7f5e3009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:04.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.843+0000 7f5e2f7fe700 1 --2- 172.21.15.16:0/2491316984 >> v2:172.21.15.16:3300/0 conn(0x7f5e3009e930 0x7f5e3009ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:04.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.843+0000 7f5e2f7fe700 1 -- 172.21.15.16:0/2491316984 >> v1:172.21.15.16:6789/0 conn(0x7f5e3009f240 legacy=0x7f5e301317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:04.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.843+0000 7f5e2f7fe700 1 -- 172.21.15.16:0/2491316984 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e3009fb60 con 0x7f5e3009f650 2023-09-30T22:31:04.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.843+0000 7f5e2effd700 1 --2- 172.21.15.16:0/2491316984 >> v2:172.21.15.16:3300/0 conn(0x7f5e3009e930 0x7f5e3009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:04.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.843+0000 7f5e2f7fe700 1 --2- 172.21.15.16:0/2491316984 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5e3009f650 0x7f5e30131f00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5e28009d90 tx=0x7f5e2800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29edfe7be805b227 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:04.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.843+0000 7f5e2e7fc700 1 -- 172.21.15.16:0/2491316984 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2800e070 con 0x7f5e3009f650 2023-09-30T22:31:04.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.843+0000 7f5e2e7fc700 1 -- 172.21.15.16:0/2491316984 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e2800bb20 con 0x7f5e3009f650 2023-09-30T22:31:04.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.843+0000 7f5e2e7fc700 1 -- 172.21.15.16:0/2491316984 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2800be90 con 0x7f5e3009f650 2023-09-30T22:31:04.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.844+0000 7f5e35638700 1 -- 172.21.15.16:0/2491316984 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5e3009f650 msgr2=0x7f5e30131f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:04.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.844+0000 7f5e35638700 1 --2- 172.21.15.16:0/2491316984 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5e3009f650 0x7f5e30131f00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5e28009d90 tx=0x7f5e2800b330 comp rx=0 tx=0).stop 2023-09-30T22:31:04.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.844+0000 7f5e35638700 1 -- 172.21.15.16:0/2491316984 shutdown_connections 2023-09-30T22:31:04.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.844+0000 7f5e35638700 1 --2- 172.21.15.16:0/2491316984 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5e3009f650 0x7f5e30131f00 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:04.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.844+0000 7f5e35638700 1 --2- 172.21.15.16:0/2491316984 >> v2:172.21.15.16:3300/0 conn(0x7f5e3009e930 0x7f5e3009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:04.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.844+0000 7f5e35638700 1 -- 172.21.15.16:0/2491316984 >> 172.21.15.16:0/2491316984 conn(0x7f5e300a4210 msgr2=0x7f5e300a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:04.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.844+0000 7f5e35638700 1 -- 172.21.15.16:0/2491316984 shutdown_connections 2023-09-30T22:31:04.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.844+0000 7f5e35638700 1 -- 172.21.15.16:0/2491316984 wait complete. 2023-09-30T22:31:04.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.845+0000 7f5e35638700 1 Processor -- start 2023-09-30T22:31:04.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.845+0000 7f5e35638700 1 -- start start 2023-09-30T22:31:04.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.846+0000 7f5e35638700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e3009e930 0x7f5e301c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:04.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.846+0000 7f5e35638700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5e3009f240 0x7f5e301c8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:04.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.846+0000 7f5e35638700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5e3009f650 0x7f5e301c89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:04.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.846+0000 7f5e35638700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e30071d20 con 0x7f5e3009e930 2023-09-30T22:31:04.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.846+0000 7f5e35638700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e301a8040 con 0x7f5e3009f240 2023-09-30T22:31:04.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.846+0000 7f5e35638700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3006d4c0 con 0x7f5e3009f650 2023-09-30T22:31:04.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.846+0000 7f5e267fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5e3009f240 0x7f5e301c8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:04.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.846+0000 7f5e2effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e3009e930 0x7f5e301c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:04.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.846+0000 7f5e2f7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5e3009f650 0x7f5e301c89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:04.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.846+0000 7f5e2effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e3009e930 0x7f5e301c7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55648/0 (socket says 172.21.15.16:55648) 2023-09-30T22:31:04.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.846+0000 7f5e2f7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5e3009f650 0x7f5e301c89c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:58030/0 (socket says 172.21.15.16:58030) 2023-09-30T22:31:04.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.846+0000 7f5e267fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5e3009f240 0x7f5e301c8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38588/0 (socket says 172.21.15.16:38588) 2023-09-30T22:31:04.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.846+0000 7f5e2effd700 1 -- 172.21.15.16:0/2305581171 learned_addr learned my addr 172.21.15.16:0/2305581171 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:04.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.846+0000 7f5e2f7fe700 1 -- 172.21.15.16:0/2305581171 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5e3009f240 msgr2=0x7f5e301c8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:04.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.846+0000 7f5e2f7fe700 1 --2- 172.21.15.16:0/2305581171 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5e3009f240 0x7f5e301c8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:04.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.846+0000 7f5e2f7fe700 1 -- 172.21.15.16:0/2305581171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e3009e930 msgr2=0x7f5e301c7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:04.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.846+0000 7f5e2f7fe700 1 --2- 172.21.15.16:0/2305581171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e3009e930 0x7f5e301c7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:04.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.846+0000 7f5e2f7fe700 1 -- 172.21.15.16:0/2305581171 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e200094a0 con 0x7f5e3009f650 2023-09-30T22:31:04.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.846+0000 7f5e267fc700 1 --2- 172.21.15.16:0/2305581171 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5e3009f240 0x7f5e301c8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:04.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.846+0000 7f5e2effd700 1 --2- 172.21.15.16:0/2305581171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e3009e930 0x7f5e301c7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:04.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.847+0000 7f5e2f7fe700 1 --2- 172.21.15.16:0/2305581171 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5e3009f650 0x7f5e301c89c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5e28016000 tx=0x7f5e28004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:04.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.847+0000 7f5e27fff700 1 -- 172.21.15.16:0/2305581171 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e28004880 con 0x7f5e3009f650 2023-09-30T22:31:04.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.847+0000 7f5e27fff700 1 -- 172.21.15.16:0/2305581171 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e28020070 con 0x7f5e3009f650 2023-09-30T22:31:04.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.847+0000 7f5e27fff700 1 -- 172.21.15.16:0/2305581171 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e280276c0 con 0x7f5e3009f650 2023-09-30T22:31:04.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.847+0000 7f5e35638700 1 -- 172.21.15.16:0/2305581171 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e28009a70 con 0x7f5e3009f650 2023-09-30T22:31:04.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.847+0000 7f5e35638700 1 -- 172.21.15.16:0/2305581171 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e300a0190 con 0x7f5e3009f650 2023-09-30T22:31:04.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.848+0000 7f5e27fff700 1 -- 172.21.15.16:0/2305581171 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5e28012430 con 0x7f5e3009f650 2023-09-30T22:31:04.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.848+0000 7f5e35638700 1 -- 172.21.15.16:0/2305581171 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e30074ea0 con 0x7f5e3009f650 2023-09-30T22:31:04.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.848+0000 7f5e27fff700 1 -- 172.21.15.16:0/2305581171 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f5e28022070 con 0x7f5e3009f650 2023-09-30T22:31:04.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.853+0000 7f5e27fff700 1 -- 172.21.15.16:0/2305581171 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5e2801a790 con 0x7f5e3009f650 2023-09-30T22:31:04.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.878+0000 7f4aea451700 1 Processor -- start 2023-09-30T22:31:04.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.878+0000 7f4aea451700 1 -- start start 2023-09-30T22:31:04.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.879+0000 7f4aea451700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4ae40a0a30 0x7f4ae40a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:04.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.879+0000 7f4aea451700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4ae4134170 0x7f4ae413bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:04.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.879+0000 7f4aea451700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ae409edd0 con 0x7f4ae40a1340 2023-09-30T22:31:04.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.879+0000 7f4aea451700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ae409e1f0 con 0x7f4ae4134170 2023-09-30T22:31:04.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.879+0000 7f4aea451700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae409dc50 con 0x7f4ae40a0a30 2023-09-30T22:31:04.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.879+0000 7f4ae37fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4ae40a1340 0x7f4ae4133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38686/0 (socket says 172.21.15.16:38686) 2023-09-30T22:31:04.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.879+0000 7f4ae37fe700 1 -- 172.21.15.16:0/4199796494 learned_addr learned my addr 172.21.15.16:0/4199796494 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:31:04.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.879+0000 7f4ae3fff700 1 --2- 172.21.15.16:0/4199796494 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4ae40a0a30 0x7f4ae40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:04.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.879+0000 7f4ae89ee700 1 --2- 172.21.15.16:0/4199796494 >> v2:172.21.15.16:3300/0 conn(0x7f4ae4134170 0x7f4ae413bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:04.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.879+0000 7f4ae3fff700 1 -- 172.21.15.16:0/4199796494 >> v2:172.21.15.16:3300/0 conn(0x7f4ae4134170 msgr2=0x7f4ae413bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:04.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.879+0000 7f4ae3fff700 1 --2- 172.21.15.16:0/4199796494 >> v2:172.21.15.16:3300/0 conn(0x7f4ae4134170 0x7f4ae413bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:04.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.879+0000 7f4ae3fff700 1 -- 172.21.15.16:0/4199796494 >> v1:172.21.15.16:6789/0 conn(0x7f4ae40a1340 legacy=0x7f4ae4133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:04.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.879+0000 7f4ae3fff700 1 -- 172.21.15.16:0/4199796494 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ae40a1fe0 con 0x7f4ae40a0a30 2023-09-30T22:31:04.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.880+0000 7f4ae3fff700 1 --2- 172.21.15.16:0/4199796494 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4ae40a0a30 0x7f4ae40a0e00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4009840 tx=0x7f4ad4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29324b57a5183b59 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:04.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.880+0000 7f4ae2ffd700 1 -- 172.21.15.16:0/4199796494 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad4011830 con 0x7f4ae40a0a30 2023-09-30T22:31:04.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.880+0000 7f4ae2ffd700 1 -- 172.21.15.16:0/4199796494 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ad4011990 con 0x7f4ae40a0a30 2023-09-30T22:31:04.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.880+0000 7f4ae2ffd700 1 -- 172.21.15.16:0/4199796494 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad4011d20 con 0x7f4ae40a0a30 2023-09-30T22:31:04.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.881+0000 7f4aea451700 1 -- 172.21.15.16:0/4199796494 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4ae40a0a30 msgr2=0x7f4ae40a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:04.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.881+0000 7f4aea451700 1 --2- 172.21.15.16:0/4199796494 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4ae40a0a30 0x7f4ae40a0e00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4009840 tx=0x7f4ad4009b50 comp rx=0 tx=0).stop 2023-09-30T22:31:04.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.881+0000 7f4aea451700 1 -- 172.21.15.16:0/4199796494 shutdown_connections 2023-09-30T22:31:04.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.881+0000 7f4aea451700 1 --2- 172.21.15.16:0/4199796494 >> v2:172.21.15.16:3300/0 conn(0x7f4ae4134170 0x7f4ae413bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:04.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.881+0000 7f4aea451700 1 --2- 172.21.15.16:0/4199796494 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4ae40a0a30 0x7f4ae40a0e00 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:04.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.881+0000 7f4aea451700 1 -- 172.21.15.16:0/4199796494 >> 172.21.15.16:0/4199796494 conn(0x7f4ae40a6550 msgr2=0x7f4ae40a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:04.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.881+0000 7f4aea451700 1 -- 172.21.15.16:0/4199796494 shutdown_connections 2023-09-30T22:31:04.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.881+0000 7f4aea451700 1 -- 172.21.15.16:0/4199796494 wait complete. 2023-09-30T22:31:04.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.882+0000 7f4aea451700 1 Processor -- start 2023-09-30T22:31:04.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.882+0000 7f4aea451700 1 -- start start 2023-09-30T22:31:04.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.883+0000 7f4aea451700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4ae40a0a30 0x7f4ae41c8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:04.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.883+0000 7f4aea451700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ae40a1340 0x7f4ae41c89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:04.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.883+0000 7f4aea451700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4ae4134170 0x7f4ae41c8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:04.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.883+0000 7f4aea451700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae41244d0 con 0x7f4ae40a1340 2023-09-30T22:31:04.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.883+0000 7f4aea451700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae4124da0 con 0x7f4ae40a0a30 2023-09-30T22:31:04.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.883+0000 7f4aea451700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4ae41263e0 con 0x7f4ae4134170 2023-09-30T22:31:04.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.883+0000 7f4ae89ee700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4ae4134170 0x7f4ae41c8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:04.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.883+0000 7f4ae37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ae40a1340 0x7f4ae41c89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:04.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.883+0000 7f4ae3fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4ae40a0a30 0x7f4ae41c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:04.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.883+0000 7f4ae89ee700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4ae4134170 0x7f4ae41c8ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:58038/0 (socket says 172.21.15.16:58038) 2023-09-30T22:31:04.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.883+0000 7f4ae89ee700 1 -- 172.21.15.16:0/877219332 learned_addr learned my addr 172.21.15.16:0/877219332 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:04.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.883+0000 7f4ae37fe700 1 -- 172.21.15.16:0/877219332 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4ae4134170 msgr2=0x7f4ae41c8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:04.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.883+0000 7f4ae37fe700 1 --2- 172.21.15.16:0/877219332 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4ae4134170 0x7f4ae41c8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:04.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.883+0000 7f4ae37fe700 1 -- 172.21.15.16:0/877219332 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4ae40a0a30 msgr2=0x7f4ae41c8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:04.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.883+0000 7f4ae37fe700 1 --2- 172.21.15.16:0/877219332 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4ae40a0a30 0x7f4ae41c8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:04.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.883+0000 7f4ae37fe700 1 -- 172.21.15.16:0/877219332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad40094a0 con 0x7f4ae40a1340 2023-09-30T22:31:04.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.883+0000 7f4ae3fff700 1 --2- 172.21.15.16:0/877219332 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4ae40a0a30 0x7f4ae41c8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:04.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.883+0000 7f4ae37fe700 1 --2- 172.21.15.16:0/877219332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ae40a1340 0x7f4ae41c89a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f4ad800c500 tx=0x7f4ad800c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:04.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.884+0000 7f4ae0ff9700 1 -- 172.21.15.16:0/877219332 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad8005d70 con 0x7f4ae40a1340 2023-09-30T22:31:04.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.884+0000 7f4ae0ff9700 1 -- 172.21.15.16:0/877219332 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ad80070d0 con 0x7f4ae40a1340 2023-09-30T22:31:04.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.884+0000 7f4aea451700 1 -- 172.21.15.16:0/877219332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ae4124a80 con 0x7f4ae40a1340 2023-09-30T22:31:04.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.884+0000 7f4ae0ff9700 1 -- 172.21.15.16:0/877219332 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ad8007440 con 0x7f4ae40a1340 2023-09-30T22:31:04.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.884+0000 7f4aea451700 1 -- 172.21.15.16:0/877219332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ae4126100 con 0x7f4ae40a1340 2023-09-30T22:31:04.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.885+0000 7f4ae0ff9700 1 -- 172.21.15.16:0/877219332 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4ad80075a0 con 0x7f4ae40a1340 2023-09-30T22:31:04.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.885+0000 7f4aea451700 1 -- 172.21.15.16:0/877219332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ae4074ea0 con 0x7f4ae40a1340 2023-09-30T22:31:04.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.885+0000 7f4ae0ff9700 1 -- 172.21.15.16:0/877219332 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f4ad801a950 con 0x7f4ae40a1340 2023-09-30T22:31:04.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.894+0000 7f4ae0ff9700 1 -- 172.21.15.16:0/877219332 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4ad8017080 con 0x7f4ae40a1340 2023-09-30T22:31:04.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.923+0000 7f0622d7f700 1 Processor -- start 2023-09-30T22:31:04.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.923+0000 7f0622d7f700 1 -- start start 2023-09-30T22:31:04.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.923+0000 7f0622d7f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f061c09d6f0 0x7f061c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:04.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.923+0000 7f0622d7f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f061c09e000 0x7f061c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:04.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.923+0000 7f0622d7f700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f061c09b120 con 0x7f061c09e410 2023-09-30T22:31:04.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.923+0000 7f0622d7f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f061c06b460 con 0x7f061c09d6f0 2023-09-30T22:31:04.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.923+0000 7f0622d7f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f061c09a7e0 con 0x7f061c09e000 2023-09-30T22:31:04.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.923+0000 7f0620b1b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f061c09d6f0 0x7f061c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:04.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.923+0000 7f061bfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f061c09e000 0x7f061c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:04.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.924+0000 7f061bfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f061c09e000 0x7f061c1339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:58046/0 (socket says 172.21.15.16:58046) 2023-09-30T22:31:04.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.924+0000 7f0620b1b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f061c09d6f0 0x7f061c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38606/0 (socket says 172.21.15.16:38606) 2023-09-30T22:31:04.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.924+0000 7f061bfff700 1 -- 172.21.15.16:0/367588644 learned_addr learned my addr 172.21.15.16:0/367588644 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:04.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.924+0000 7f061b7fe700 1 -- 172.21.15.16:0/367588644 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3055576061 0 0) 0x7f061c09b120 con 0x7f061c09e410 2023-09-30T22:31:04.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.924+0000 7f061bfff700 1 -- 172.21.15.16:0/367588644 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f061c09d6f0 msgr2=0x7f061c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:04.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.924+0000 7f061bfff700 1 --2- 172.21.15.16:0/367588644 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f061c09d6f0 0x7f061c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:04.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.924+0000 7f061bfff700 1 -- 172.21.15.16:0/367588644 >> v1:172.21.15.16:6789/0 conn(0x7f061c09e410 legacy=0x7f061c133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:04.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.924+0000 7f061bfff700 1 -- 172.21.15.16:0/367588644 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f061c09a500 con 0x7f061c09e000 2023-09-30T22:31:04.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.924+0000 7f0620b1b700 1 --2- 172.21.15.16:0/367588644 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f061c09d6f0 0x7f061c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:04.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.924+0000 7f061bfff700 1 --2- 172.21.15.16:0/367588644 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f061c09e000 0x7f061c1339f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0608009950 tx=0x7f0608009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbae1759d6489bb8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:04.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.924+0000 7f061b7fe700 1 -- 172.21.15.16:0/367588644 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0608011830 con 0x7f061c09e000 2023-09-30T22:31:04.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.924+0000 7f061b7fe700 1 -- 172.21.15.16:0/367588644 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0608011990 con 0x7f061c09e000 2023-09-30T22:31:04.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.924+0000 7f061b7fe700 1 -- 172.21.15.16:0/367588644 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0608011d40 con 0x7f061c09e000 2023-09-30T22:31:04.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.925+0000 7f0622d7f700 1 -- 172.21.15.16:0/367588644 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f061c09e000 msgr2=0x7f061c1339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:04.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.925+0000 7f0622d7f700 1 --2- 172.21.15.16:0/367588644 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f061c09e000 0x7f061c1339f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0608009950 tx=0x7f0608009c60 comp rx=0 tx=0).stop 2023-09-30T22:31:04.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.925+0000 7f0622d7f700 1 -- 172.21.15.16:0/367588644 shutdown_connections 2023-09-30T22:31:04.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.925+0000 7f0622d7f700 1 --2- 172.21.15.16:0/367588644 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f061c09d6f0 0x7f061c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:04.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.925+0000 7f0622d7f700 1 --2- 172.21.15.16:0/367588644 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f061c09e000 0x7f061c1339f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:04.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.925+0000 7f0622d7f700 1 -- 172.21.15.16:0/367588644 >> 172.21.15.16:0/367588644 conn(0x7f061c066f50 msgr2=0x7f061c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:04.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.925+0000 7f0622d7f700 1 -- 172.21.15.16:0/367588644 shutdown_connections 2023-09-30T22:31:04.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.925+0000 7f0622d7f700 1 -- 172.21.15.16:0/367588644 wait complete. 2023-09-30T22:31:04.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.926+0000 7f0622d7f700 1 Processor -- start 2023-09-30T22:31:04.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.926+0000 7f0622d7f700 1 -- start start 2023-09-30T22:31:04.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.927+0000 7f0622d7f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f061c09d6f0 0x7f061c1a5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:04.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.927+0000 7f0622d7f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f061c09e000 0x7f061c1a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:04.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.927+0000 7f0622d7f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f061c09e410 0x7f061c1c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:04.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.927+0000 7f0622d7f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061c1a4f70 con 0x7f061c09e410 2023-09-30T22:31:04.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.927+0000 7f0622d7f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f061c1b93f0 con 0x7f061c09e000 2023-09-30T22:31:04.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.927+0000 7f0622d7f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f061c09c710 con 0x7f061c09d6f0 2023-09-30T22:31:04.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.927+0000 7f0620b1b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f061c09d6f0 0x7f061c1a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:04.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.927+0000 7f0620b1b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f061c09d6f0 0x7f061c1a5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:58062/0 (socket says 172.21.15.16:58062) 2023-09-30T22:31:04.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.927+0000 7f061bfff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f061c09e000 0x7f061c1a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:04.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.927+0000 7f0620b1b700 1 -- 172.21.15.16:0/3259964791 learned_addr learned my addr 172.21.15.16:0/3259964791 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:04.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.927+0000 7f062131c700 1 --2- 172.21.15.16:0/3259964791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f061c09e410 0x7f061c1c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:04.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.927+0000 7f0620b1b700 1 -- 172.21.15.16:0/3259964791 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f061c09e000 msgr2=0x7f061c1a5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:04.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.927+0000 7f0620b1b700 1 --2- 172.21.15.16:0/3259964791 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f061c09e000 0x7f061c1a5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:04.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.927+0000 7f0620b1b700 1 -- 172.21.15.16:0/3259964791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f061c09e410 msgr2=0x7f061c1c82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:04.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.927+0000 7f0620b1b700 1 --2- 172.21.15.16:0/3259964791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f061c09e410 0x7f061c1c82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:04.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.927+0000 7f0620b1b700 1 -- 172.21.15.16:0/3259964791 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06100094a0 con 0x7f061c09d6f0 2023-09-30T22:31:04.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.927+0000 7f062131c700 1 --2- 172.21.15.16:0/3259964791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f061c09e410 0x7f061c1c82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:04.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.928+0000 7f0620b1b700 1 --2- 172.21.15.16:0/3259964791 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f061c09d6f0 0x7f061c1a5330 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f061000ebe0 tx=0x7f061000eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:04.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.928+0000 7f06197fa700 1 -- 172.21.15.16:0/3259964791 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061000c850 con 0x7f061c09d6f0 2023-09-30T22:31:04.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.928+0000 7f06197fa700 1 -- 172.21.15.16:0/3259964791 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f061000c9b0 con 0x7f061c09d6f0 2023-09-30T22:31:04.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.928+0000 7f0622d7f700 1 -- 172.21.15.16:0/3259964791 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0608009550 con 0x7f061c09d6f0 2023-09-30T22:31:04.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.928+0000 7f06197fa700 1 -- 172.21.15.16:0/3259964791 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061000cd40 con 0x7f061c09d6f0 2023-09-30T22:31:04.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.928+0000 7f0622d7f700 1 -- 172.21.15.16:0/3259964791 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f061c1a6670 con 0x7f061c09d6f0 2023-09-30T22:31:04.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.929+0000 7f06197fa700 1 -- 172.21.15.16:0/3259964791 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0610013470 con 0x7f061c09d6f0 2023-09-30T22:31:04.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.929+0000 7f06197fa700 1 -- 172.21.15.16:0/3259964791 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f061003f8a0 con 0x7f061c09d6f0 2023-09-30T22:31:04.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.929+0000 7f0622d7f700 1 -- 172.21.15.16:0/3259964791 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f061c074ea0 con 0x7f061c09d6f0 2023-09-30T22:31:04.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:04.939+0000 7f06197fa700 1 -- 172.21.15.16:0/3259964791 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0610021080 con 0x7f061c09d6f0 2023-09-30T22:31:05.043 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:05.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.065+0000 7f5e35638700 1 -- 172.21.15.16:0/2305581171 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f5e3006b250 con 0x7f5e3009f650 2023-09-30T22:31:05.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.065+0000 7f5e27fff700 1 -- 172.21.15.16:0/2305581171 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5e2802c070 con 0x7f5e3009f650 2023-09-30T22:31:05.066 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:31:05.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.068+0000 7f5e35638700 1 -- 172.21.15.16:0/2305581171 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5e3009f650 msgr2=0x7f5e301c89c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:05.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.068+0000 7f5e35638700 1 --2- 172.21.15.16:0/2305581171 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5e3009f650 0x7f5e301c89c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5e28016000 tx=0x7f5e28004ad0 comp rx=0 tx=0).stop 2023-09-30T22:31:05.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.069+0000 7f5e35638700 1 -- 172.21.15.16:0/2305581171 shutdown_connections 2023-09-30T22:31:05.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.069+0000 7f5e35638700 1 --2- 172.21.15.16:0/2305581171 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5e3009f650 0x7f5e301c89c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:05.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.069+0000 7f5e35638700 1 --2- 172.21.15.16:0/2305581171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e3009e930 0x7f5e301c7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:05.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.069+0000 7f5e35638700 1 --2- 172.21.15.16:0/2305581171 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5e3009f240 0x7f5e301c8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:05.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.069+0000 7f5e35638700 1 -- 172.21.15.16:0/2305581171 >> 172.21.15.16:0/2305581171 conn(0x7f5e300a4210 msgr2=0x7f5e3009bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:05.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.069+0000 7f5e35638700 1 -- 172.21.15.16:0/2305581171 shutdown_connections 2023-09-30T22:31:05.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.069+0000 7f5e35638700 1 -- 172.21.15.16:0/2305581171 wait complete. 2023-09-30T22:31:05.084 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:31:05.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.114+0000 7f4aea451700 1 -- 172.21.15.16:0/877219332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4ae40a28e0 con 0x7f4ae40a1340 2023-09-30T22:31:05.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.114+0000 7f4ae0ff9700 1 -- 172.21.15.16:0/877219332 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4ad8044060 con 0x7f4ae40a1340 2023-09-30T22:31:05.115 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:31:05.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.117+0000 7f4aea451700 1 -- 172.21.15.16:0/877219332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ae40a1340 msgr2=0x7f4ae41c89a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:05.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.117+0000 7f4aea451700 1 --2- 172.21.15.16:0/877219332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ae40a1340 0x7f4ae41c89a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f4ad800c500 tx=0x7f4ad800c810 comp rx=0 tx=0).stop 2023-09-30T22:31:05.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.117+0000 7f4aea451700 1 -- 172.21.15.16:0/877219332 shutdown_connections 2023-09-30T22:31:05.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.117+0000 7f4aea451700 1 --2- 172.21.15.16:0/877219332 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4ae40a0a30 0x7f4ae41c8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:05.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.117+0000 7f4aea451700 1 --2- 172.21.15.16:0/877219332 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4ae4134170 0x7f4ae41c8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:05.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.117+0000 7f4aea451700 1 --2- 172.21.15.16:0/877219332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ae40a1340 0x7f4ae41c89a0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:05.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.117+0000 7f4aea451700 1 -- 172.21.15.16:0/877219332 >> 172.21.15.16:0/877219332 conn(0x7f4ae40a6550 msgr2=0x7f4ae409ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:05.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.117+0000 7f4aea451700 1 -- 172.21.15.16:0/877219332 shutdown_connections 2023-09-30T22:31:05.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.117+0000 7f4aea451700 1 -- 172.21.15.16:0/877219332 wait complete. 2023-09-30T22:31:05.133 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:31:05.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.153+0000 7f0622d7f700 1 -- 172.21.15.16:0/3259964791 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f061c072600 con 0x7f061c09d6f0 2023-09-30T22:31:05.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.154+0000 7f06197fa700 1 -- 172.21.15.16:0/3259964791 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f061000cea0 con 0x7f061c09d6f0 2023-09-30T22:31:05.154 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:31:05.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.157+0000 7f0622d7f700 1 -- 172.21.15.16:0/3259964791 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f061c09d6f0 msgr2=0x7f061c1a5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:05.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.157+0000 7f0622d7f700 1 --2- 172.21.15.16:0/3259964791 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f061c09d6f0 0x7f061c1a5330 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f061000ebe0 tx=0x7f061000eef0 comp rx=0 tx=0).stop 2023-09-30T22:31:05.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.158+0000 7f0622d7f700 1 -- 172.21.15.16:0/3259964791 shutdown_connections 2023-09-30T22:31:05.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.158+0000 7f0622d7f700 1 --2- 172.21.15.16:0/3259964791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f061c09e410 0x7f061c1c82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:05.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.158+0000 7f0622d7f700 1 --2- 172.21.15.16:0/3259964791 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f061c09d6f0 0x7f061c1a5330 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:05.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.158+0000 7f0622d7f700 1 --2- 172.21.15.16:0/3259964791 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f061c09e000 0x7f061c1a5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:05.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.158+0000 7f0622d7f700 1 -- 172.21.15.16:0/3259964791 >> 172.21.15.16:0/3259964791 conn(0x7f061c066f50 msgr2=0x7f061c06f120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:05.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.158+0000 7f0622d7f700 1 -- 172.21.15.16:0/3259964791 shutdown_connections 2023-09-30T22:31:05.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.158+0000 7f0622d7f700 1 -- 172.21.15.16:0/3259964791 wait complete. 2023-09-30T22:31:05.172 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:31:05.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.189+0000 7fd6179c7700 1 Processor -- start 2023-09-30T22:31:05.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.189+0000 7fd6179c7700 1 -- start start 2023-09-30T22:31:05.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.190+0000 7fd6179c7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd61009ac90 0x7fd61013cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:05.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.190+0000 7fd6179c7700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd61013d240 0x7fd61013f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:05.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.190+0000 7fd6179c7700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd610134980 con 0x7fd61006c0e0 2023-09-30T22:31:05.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.190+0000 7fd6179c7700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd61006c4b0 con 0x7fd61013d240 2023-09-30T22:31:05.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.190+0000 7fd6179c7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd61009b0e0 con 0x7fd61009ac90 2023-09-30T22:31:05.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.190+0000 7fd614f62700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd61009ac90 0x7fd61013cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:05.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.190+0000 7fd615f64700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd61013d240 0x7fd61013f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:05.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.190+0000 7fd615763700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd61006c0e0 0x7fd610134ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38702/0 (socket says 172.21.15.16:38702) 2023-09-30T22:31:05.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.190+0000 7fd615763700 1 -- 172.21.15.16:0/2996998137 learned_addr learned my addr 172.21.15.16:0/2996998137 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:31:05.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.190+0000 7fd615f64700 1 -- 172.21.15.16:0/2996998137 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd61009ac90 msgr2=0x7fd61013cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:05.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.190+0000 7fd615f64700 1 --2- 172.21.15.16:0/2996998137 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd61009ac90 0x7fd61013cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:05.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.190+0000 7fd615f64700 1 -- 172.21.15.16:0/2996998137 >> v1:172.21.15.16:6789/0 conn(0x7fd61006c0e0 legacy=0x7fd610134ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:05.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.190+0000 7fd615f64700 1 -- 172.21.15.16:0/2996998137 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6100a0600 con 0x7fd61013d240 2023-09-30T22:31:05.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.190+0000 7fd603fff700 1 -- 172.21.15.16:0/2996998137 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 197805754 0 0) 0x7fd610134980 con 0x7fd61006c0e0 2023-09-30T22:31:05.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.190+0000 7fd614f62700 1 --2- 172.21.15.16:0/2996998137 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd61009ac90 0x7fd61013cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:05.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.191+0000 7fd615f64700 1 --2- 172.21.15.16:0/2996998137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd61013d240 0x7fd61013f620 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd608009d90 tx=0x7fd60800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4c1b69cc07394ed server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:05.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.191+0000 7fd603fff700 1 -- 172.21.15.16:0/2996998137 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60800e070 con 0x7fd61013d240 2023-09-30T22:31:05.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.191+0000 7fd603fff700 1 -- 172.21.15.16:0/2996998137 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd60800bb20 con 0x7fd61013d240 2023-09-30T22:31:05.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.191+0000 7fd603fff700 1 -- 172.21.15.16:0/2996998137 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd608012430 con 0x7fd61013d240 2023-09-30T22:31:05.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.191+0000 7fd6179c7700 1 -- 172.21.15.16:0/2996998137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd61013d240 msgr2=0x7fd61013f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:05.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.191+0000 7fd6179c7700 1 --2- 172.21.15.16:0/2996998137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd61013d240 0x7fd61013f620 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd608009d90 tx=0x7fd60800b330 comp rx=0 tx=0).stop 2023-09-30T22:31:05.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.191+0000 7fd6179c7700 1 -- 172.21.15.16:0/2996998137 shutdown_connections 2023-09-30T22:31:05.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.191+0000 7fd6179c7700 1 --2- 172.21.15.16:0/2996998137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd61013d240 0x7fd61013f620 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:05.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.191+0000 7fd6179c7700 1 --2- 172.21.15.16:0/2996998137 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd61009ac90 0x7fd61013cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:05.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.191+0000 7fd6179c7700 1 -- 172.21.15.16:0/2996998137 >> 172.21.15.16:0/2996998137 conn(0x7fd6100a4210 msgr2=0x7fd6100a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:05.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.191+0000 7fd6179c7700 1 -- 172.21.15.16:0/2996998137 shutdown_connections 2023-09-30T22:31:05.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.191+0000 7fd6179c7700 1 -- 172.21.15.16:0/2996998137 wait complete. 2023-09-30T22:31:05.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.192+0000 7fd6179c7700 1 Processor -- start 2023-09-30T22:31:05.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.192+0000 7fd6179c7700 1 -- start start 2023-09-30T22:31:05.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.193+0000 7fd6179c7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd61006c0e0 0x7fd61009b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:05.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.193+0000 7fd6179c7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd61009ac90 0x7fd61009bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:05.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.193+0000 7fd6179c7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd61013d240 0x7fd61009c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:05.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.193+0000 7fd6179c7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd61009d8b0 con 0x7fd61013d240 2023-09-30T22:31:05.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.193+0000 7fd6179c7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6100a03f0 con 0x7fd61009ac90 2023-09-30T22:31:05.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.193+0000 7fd6179c7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd610139180 con 0x7fd61006c0e0 2023-09-30T22:31:05.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.193+0000 7fd615763700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd61006c0e0 0x7fd61009b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:05.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.193+0000 7fd615f64700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd61013d240 0x7fd61009c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:05.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.193+0000 7fd615f64700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd61013d240 0x7fd61009c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55692/0 (socket says 172.21.15.16:55692) 2023-09-30T22:31:05.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.193+0000 7fd615763700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd61006c0e0 0x7fd61009b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:58066/0 (socket says 172.21.15.16:58066) 2023-09-30T22:31:05.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.193+0000 7fd614f62700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd61009ac90 0x7fd61009bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:05.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.193+0000 7fd615f64700 1 -- 172.21.15.16:0/3514490301 learned_addr learned my addr 172.21.15.16:0/3514490301 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:05.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.193+0000 7fd615f64700 1 -- 172.21.15.16:0/3514490301 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd61006c0e0 msgr2=0x7fd61009b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:05.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.193+0000 7fd615f64700 1 --2- 172.21.15.16:0/3514490301 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd61006c0e0 0x7fd61009b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:05.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.193+0000 7fd615f64700 1 -- 172.21.15.16:0/3514490301 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd61009ac90 msgr2=0x7fd61009bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:05.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.193+0000 7fd615f64700 1 --2- 172.21.15.16:0/3514490301 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd61009ac90 0x7fd61009bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:05.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.193+0000 7fd615f64700 1 -- 172.21.15.16:0/3514490301 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6040094a0 con 0x7fd61013d240 2023-09-30T22:31:05.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.194+0000 7fd615f64700 1 --2- 172.21.15.16:0/3514490301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd61013d240 0x7fd61009c2f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd608013000 tx=0x7fd608004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:05.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.194+0000 7fd601ffb700 1 -- 172.21.15.16:0/3514490301 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd608012430 con 0x7fd61013d240 2023-09-30T22:31:05.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.194+0000 7fd601ffb700 1 -- 172.21.15.16:0/3514490301 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd608020070 con 0x7fd61013d240 2023-09-30T22:31:05.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.194+0000 7fd6179c7700 1 -- 172.21.15.16:0/3514490301 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd608009a70 con 0x7fd61013d240 2023-09-30T22:31:05.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.194+0000 7fd601ffb700 1 -- 172.21.15.16:0/3514490301 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd608004280 con 0x7fd61013d240 2023-09-30T22:31:05.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.194+0000 7fd6179c7700 1 -- 172.21.15.16:0/3514490301 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd610139bf0 con 0x7fd61013d240 2023-09-30T22:31:05.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.195+0000 7fd601ffb700 1 -- 172.21.15.16:0/3514490301 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd6080043e0 con 0x7fd61013d240 2023-09-30T22:31:05.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.195+0000 7fd601ffb700 1 -- 172.21.15.16:0/3514490301 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd608022070 con 0x7fd61013d240 2023-09-30T22:31:05.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.195+0000 7fd6179c7700 1 -- 172.21.15.16:0/3514490301 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd610074ea0 con 0x7fd61013d240 2023-09-30T22:31:05.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.200+0000 7fd601ffb700 1 -- 172.21.15.16:0/3514490301 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd60801a380 con 0x7fd61013d240 2023-09-30T22:31:05.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.405+0000 7fd6179c7700 1 -- 172.21.15.16:0/3514490301 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd61006b250 con 0x7fd61013d240 2023-09-30T22:31:05.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.405+0000 7fd601ffb700 1 -- 172.21.15.16:0/3514490301 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd608025b70 con 0x7fd61013d240 2023-09-30T22:31:05.406 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:31:05.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.407+0000 7fd6179c7700 1 -- 172.21.15.16:0/3514490301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd61013d240 msgr2=0x7fd61009c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:05.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.407+0000 7fd6179c7700 1 --2- 172.21.15.16:0/3514490301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd61013d240 0x7fd61009c2f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fd608013000 tx=0x7fd608004ad0 comp rx=0 tx=0).stop 2023-09-30T22:31:05.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.407+0000 7fd6179c7700 1 -- 172.21.15.16:0/3514490301 shutdown_connections 2023-09-30T22:31:05.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.407+0000 7fd6179c7700 1 --2- 172.21.15.16:0/3514490301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd61013d240 0x7fd61009c2f0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:05.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.407+0000 7fd6179c7700 1 --2- 172.21.15.16:0/3514490301 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd61006c0e0 0x7fd61009b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:05.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.407+0000 7fd6179c7700 1 --2- 172.21.15.16:0/3514490301 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd61009ac90 0x7fd61009bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:05.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.407+0000 7fd6179c7700 1 -- 172.21.15.16:0/3514490301 >> 172.21.15.16:0/3514490301 conn(0x7fd6100a4210 msgr2=0x7fd61006c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:05.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.407+0000 7fd6179c7700 1 -- 172.21.15.16:0/3514490301 shutdown_connections 2023-09-30T22:31:05.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:05.408+0000 7fd6179c7700 1 -- 172.21.15.16:0/3514490301 wait complete. 2023-09-30T22:31:05.422 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:31:06.085 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:06.134 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:06.174 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:06.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.244+0000 7fce6b85d700 1 Processor -- start 2023-09-30T22:31:06.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.244+0000 7fce6b85d700 1 -- start start 2023-09-30T22:31:06.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.244+0000 7fce6b85d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fce640a0a30 0x7fce640a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:06.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.244+0000 7fce6b85d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fce64134170 0x7fce6413bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:06.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.244+0000 7fce6b85d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce6409edd0 con 0x7fce640a1340 2023-09-30T22:31:06.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.244+0000 7fce6b85d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce6409e1f0 con 0x7fce640a0a30 2023-09-30T22:31:06.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.244+0000 7fce6b85d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce6409dc50 con 0x7fce64134170 2023-09-30T22:31:06.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.245+0000 7fce695f9700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fce640a0a30 0x7fce640a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:06.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.245+0000 7fce68df8700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fce640a1340 0x7fce64133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38716/0 (socket says 172.21.15.16:38716) 2023-09-30T22:31:06.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.245+0000 7fce68df8700 1 -- 172.21.15.16:0/3749769504 learned_addr learned my addr 172.21.15.16:0/3749769504 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:31:06.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.245+0000 7fce69dfa700 1 --2- 172.21.15.16:0/3749769504 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fce64134170 0x7fce6413bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:06.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.245+0000 7fce695f9700 1 -- 172.21.15.16:0/3749769504 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fce64134170 msgr2=0x7fce6413bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:06.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.245+0000 7fce695f9700 1 --2- 172.21.15.16:0/3749769504 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fce64134170 0x7fce6413bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.245+0000 7fce695f9700 1 -- 172.21.15.16:0/3749769504 >> v1:172.21.15.16:6789/0 conn(0x7fce640a1340 legacy=0x7fce64133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:06.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.245+0000 7fce695f9700 1 -- 172.21.15.16:0/3749769504 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce640a1fe0 con 0x7fce640a0a30 2023-09-30T22:31:06.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.245+0000 7fce695f9700 1 --2- 172.21.15.16:0/3749769504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce640a0a30 0x7fce640a0e00 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fce60009840 tx=0x7fce60009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5803e2fc6e48335e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:06.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.245+0000 7fce5bfff700 1 -- 172.21.15.16:0/3749769504 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce60011830 con 0x7fce640a0a30 2023-09-30T22:31:06.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.245+0000 7fce5bfff700 1 -- 172.21.15.16:0/3749769504 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce60011990 con 0x7fce640a0a30 2023-09-30T22:31:06.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.245+0000 7fce5bfff700 1 -- 172.21.15.16:0/3749769504 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce60011d20 con 0x7fce640a0a30 2023-09-30T22:31:06.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.246+0000 7fce6b85d700 1 -- 172.21.15.16:0/3749769504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce640a0a30 msgr2=0x7fce640a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:06.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.246+0000 7fce6b85d700 1 --2- 172.21.15.16:0/3749769504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce640a0a30 0x7fce640a0e00 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fce60009840 tx=0x7fce60009b50 comp rx=0 tx=0).stop 2023-09-30T22:31:06.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.246+0000 7fce6b85d700 1 -- 172.21.15.16:0/3749769504 shutdown_connections 2023-09-30T22:31:06.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.246+0000 7fce6b85d700 1 --2- 172.21.15.16:0/3749769504 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fce64134170 0x7fce6413bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.246+0000 7fce6b85d700 1 --2- 172.21.15.16:0/3749769504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce640a0a30 0x7fce640a0e00 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.246+0000 7fce6b85d700 1 -- 172.21.15.16:0/3749769504 >> 172.21.15.16:0/3749769504 conn(0x7fce640a6550 msgr2=0x7fce640a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:06.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.246+0000 7fce6b85d700 1 -- 172.21.15.16:0/3749769504 shutdown_connections 2023-09-30T22:31:06.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.246+0000 7fce6b85d700 1 -- 172.21.15.16:0/3749769504 wait complete. 2023-09-30T22:31:06.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.247+0000 7fce6b85d700 1 Processor -- start 2023-09-30T22:31:06.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.247+0000 7fce6b85d700 1 -- start start 2023-09-30T22:31:06.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.248+0000 7fce6b85d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce640a0a30 0x7fce64122930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:06.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.248+0000 7fce6b85d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fce640a1340 0x7fce64122e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:06.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.248+0000 7fce6b85d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fce64134170 0x7fce641233b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:06.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.248+0000 7fce6b85d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce641246a0 con 0x7fce640a0a30 2023-09-30T22:31:06.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.248+0000 7fce6b85d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce641244b0 con 0x7fce64134170 2023-09-30T22:31:06.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.248+0000 7fce6b85d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fce641263c0 con 0x7fce640a1340 2023-09-30T22:31:06.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.248+0000 7fce695f9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce640a0a30 0x7fce64122930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:06.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.248+0000 7fce68df8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fce640a1340 0x7fce64122e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:06.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.248+0000 7fce695f9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce640a0a30 0x7fce64122930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55710/0 (socket says 172.21.15.16:55710) 2023-09-30T22:31:06.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.248+0000 7fce695f9700 1 -- 172.21.15.16:0/2545354758 learned_addr learned my addr 172.21.15.16:0/2545354758 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:06.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.248+0000 7fce69dfa700 1 --2- 172.21.15.16:0/2545354758 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fce64134170 0x7fce641233b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:06.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.249+0000 7fce695f9700 1 -- 172.21.15.16:0/2545354758 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fce640a1340 msgr2=0x7fce64122e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:06.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.249+0000 7fce695f9700 1 --2- 172.21.15.16:0/2545354758 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fce640a1340 0x7fce64122e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.249+0000 7fce695f9700 1 -- 172.21.15.16:0/2545354758 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fce64134170 msgr2=0x7fce641233b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:06.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.249+0000 7fce695f9700 1 --2- 172.21.15.16:0/2545354758 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fce64134170 0x7fce641233b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.249+0000 7fce695f9700 1 -- 172.21.15.16:0/2545354758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce600094a0 con 0x7fce640a0a30 2023-09-30T22:31:06.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.249+0000 7fce68df8700 1 --2- 172.21.15.16:0/2545354758 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fce640a1340 0x7fce64122e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:06.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.249+0000 7fce69dfa700 1 --2- 172.21.15.16:0/2545354758 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fce64134170 0x7fce641233b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:06.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.249+0000 7fce695f9700 1 --2- 172.21.15.16:0/2545354758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce640a0a30 0x7fce64122930 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fce60009fd0 tx=0x7fce60004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:06.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.249+0000 7fce59ffb700 1 -- 172.21.15.16:0/2545354758 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce600036a0 con 0x7fce640a0a30 2023-09-30T22:31:06.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.249+0000 7fce59ffb700 1 -- 172.21.15.16:0/2545354758 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce6001f070 con 0x7fce640a0a30 2023-09-30T22:31:06.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.249+0000 7fce6b85d700 1 -- 172.21.15.16:0/2545354758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce640a25e0 con 0x7fce640a0a30 2023-09-30T22:31:06.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.249+0000 7fce59ffb700 1 -- 172.21.15.16:0/2545354758 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce60003a60 con 0x7fce640a0a30 2023-09-30T22:31:06.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.249+0000 7fce6b85d700 1 -- 172.21.15.16:0/2545354758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce641260e0 con 0x7fce640a0a30 2023-09-30T22:31:06.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.250+0000 7fce59ffb700 1 -- 172.21.15.16:0/2545354758 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fce60003bc0 con 0x7fce640a0a30 2023-09-30T22:31:06.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.250+0000 7fce6b85d700 1 -- 172.21.15.16:0/2545354758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce64074ea0 con 0x7fce640a0a30 2023-09-30T22:31:06.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.251+0000 7fce59ffb700 1 -- 172.21.15.16:0/2545354758 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fce60021070 con 0x7fce640a0a30 2023-09-30T22:31:06.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.259+0000 7fce59ffb700 1 -- 172.21.15.16:0/2545354758 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fce600257f0 con 0x7fce640a0a30 2023-09-30T22:31:06.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.292+0000 7f13b26e1700 1 Processor -- start 2023-09-30T22:31:06.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.292+0000 7f13b26e1700 1 -- start start 2023-09-30T22:31:06.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.292+0000 7f13b26e1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f13ac0a0a30 0x7f13ac0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:06.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.293+0000 7f13b26e1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f13ac134170 0x7f13ac13bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:06.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.293+0000 7f13b26e1700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13ac09edd0 con 0x7f13ac0a1340 2023-09-30T22:31:06.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.293+0000 7f13b26e1700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13ac09e1f0 con 0x7f13ac0a0a30 2023-09-30T22:31:06.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.293+0000 7f13b26e1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f13ac09dc50 con 0x7f13ac134170 2023-09-30T22:31:06.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.293+0000 7f13b0c7e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f13ac134170 0x7f13ac13bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:06.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.293+0000 7f13b0c7e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f13ac134170 0x7f13ac13bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:58080/0 (socket says 172.21.15.16:58080) 2023-09-30T22:31:06.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.293+0000 7f13ab7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f13ac0a1340 0x7f13ac133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38732/0 (socket says 172.21.15.16:38732) 2023-09-30T22:31:06.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.293+0000 7f13abfff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f13ac0a0a30 0x7f13ac0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:06.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.293+0000 7f13b0c7e700 1 -- 172.21.15.16:0/543949214 learned_addr learned my addr 172.21.15.16:0/543949214 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:06.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.293+0000 7f13aaffd700 1 -- 172.21.15.16:0/543949214 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2012536529 0 0) 0x7f13ac09edd0 con 0x7f13ac0a1340 2023-09-30T22:31:06.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.293+0000 7f13abfff700 1 -- 172.21.15.16:0/543949214 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f13ac134170 msgr2=0x7f13ac13bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:06.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.293+0000 7f13abfff700 1 --2- 172.21.15.16:0/543949214 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f13ac134170 0x7f13ac13bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.293+0000 7f13abfff700 1 -- 172.21.15.16:0/543949214 >> v1:172.21.15.16:6789/0 conn(0x7f13ac0a1340 legacy=0x7f13ac133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:06.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.293+0000 7f13abfff700 1 -- 172.21.15.16:0/543949214 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13ac0a1fe0 con 0x7f13ac0a0a30 2023-09-30T22:31:06.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.293+0000 7f13abfff700 1 --2- 172.21.15.16:0/543949214 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f13ac0a0a30 0x7f13ac0a0e00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f139c009870 tx=0x7f139c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12a55f3903880641 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:06.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.294+0000 7f13aaffd700 1 -- 172.21.15.16:0/543949214 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f139c011830 con 0x7f13ac0a0a30 2023-09-30T22:31:06.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.294+0000 7f13aaffd700 1 -- 172.21.15.16:0/543949214 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f139c011990 con 0x7f13ac0a0a30 2023-09-30T22:31:06.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.294+0000 7f13aaffd700 1 -- 172.21.15.16:0/543949214 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f139c011d20 con 0x7f13ac0a0a30 2023-09-30T22:31:06.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.294+0000 7f13b26e1700 1 -- 172.21.15.16:0/543949214 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f13ac0a0a30 msgr2=0x7f13ac0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:06.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.294+0000 7f13b26e1700 1 --2- 172.21.15.16:0/543949214 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f13ac0a0a30 0x7f13ac0a0e00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f139c009870 tx=0x7f139c009b80 comp rx=0 tx=0).stop 2023-09-30T22:31:06.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.294+0000 7f13b26e1700 1 -- 172.21.15.16:0/543949214 shutdown_connections 2023-09-30T22:31:06.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.294+0000 7f13b26e1700 1 --2- 172.21.15.16:0/543949214 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f13ac134170 0x7f13ac13bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.294+0000 7f13b26e1700 1 --2- 172.21.15.16:0/543949214 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f13ac0a0a30 0x7f13ac0a0e00 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.294+0000 7f13b26e1700 1 -- 172.21.15.16:0/543949214 >> 172.21.15.16:0/543949214 conn(0x7f13ac0a6550 msgr2=0x7f13ac0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:06.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.294+0000 7f13b26e1700 1 -- 172.21.15.16:0/543949214 shutdown_connections 2023-09-30T22:31:06.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.294+0000 7f13b26e1700 1 -- 172.21.15.16:0/543949214 wait complete. 2023-09-30T22:31:06.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.295+0000 7f13b26e1700 1 Processor -- start 2023-09-30T22:31:06.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.296+0000 7f13b26e1700 1 -- start start 2023-09-30T22:31:06.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.296+0000 7f13b26e1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f13ac0a0a30 0x7f13ac122690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:06.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.296+0000 7f13b26e1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f13ac0a1340 0x7f13ac122bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:06.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.296+0000 7f13b26e1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13ac134170 0x7f13ac123110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:06.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.296+0000 7f13b26e1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13ac133650 con 0x7f13ac134170 2023-09-30T22:31:06.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.296+0000 7f13b26e1700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13ac1243b0 con 0x7f13ac0a1340 2023-09-30T22:31:06.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.296+0000 7f13b26e1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f13ac0a0220 con 0x7f13ac0a0a30 2023-09-30T22:31:06.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.296+0000 7f13ab7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f13ac0a1340 0x7f13ac122bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:06.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.296+0000 7f13ab7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f13ac0a1340 0x7f13ac122bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38664/0 (socket says 172.21.15.16:38664) 2023-09-30T22:31:06.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.296+0000 7f13ab7fe700 1 -- 172.21.15.16:0/3272186481 learned_addr learned my addr 172.21.15.16:0/3272186481 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:06.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.296+0000 7f13abfff700 1 --2- 172.21.15.16:0/3272186481 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f13ac0a0a30 0x7f13ac122690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:06.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.296+0000 7f13b0c7e700 1 --2- 172.21.15.16:0/3272186481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13ac134170 0x7f13ac123110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:06.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.297+0000 7f13ab7fe700 1 -- 172.21.15.16:0/3272186481 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f13ac0a0a30 msgr2=0x7f13ac122690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:06.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.297+0000 7f13ab7fe700 1 --2- 172.21.15.16:0/3272186481 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f13ac0a0a30 0x7f13ac122690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.297+0000 7f13ab7fe700 1 -- 172.21.15.16:0/3272186481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13ac134170 msgr2=0x7f13ac123110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:06.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.297+0000 7f13ab7fe700 1 --2- 172.21.15.16:0/3272186481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13ac134170 0x7f13ac123110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.297+0000 7f13ab7fe700 1 -- 172.21.15.16:0/3272186481 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f139c0094a0 con 0x7f13ac0a1340 2023-09-30T22:31:06.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.297+0000 7f13abfff700 1 --2- 172.21.15.16:0/3272186481 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f13ac0a0a30 0x7f13ac122690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:06.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.297+0000 7f13b0c7e700 1 --2- 172.21.15.16:0/3272186481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13ac134170 0x7f13ac123110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:31:06.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.297+0000 7f13ab7fe700 1 --2- 172.21.15.16:0/3272186481 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f13ac0a1340 0x7f13ac122bd0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f13a000c520 tx=0x7f13a000c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:06.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.297+0000 7f13a8ff9700 1 -- 172.21.15.16:0/3272186481 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a0004c80 con 0x7f13ac0a1340 2023-09-30T22:31:06.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.297+0000 7f13a8ff9700 1 -- 172.21.15.16:0/3272186481 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13a0004de0 con 0x7f13ac0a1340 2023-09-30T22:31:06.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.297+0000 7f13b26e1700 1 -- 172.21.15.16:0/3272186481 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13ac0a25e0 con 0x7f13ac0a1340 2023-09-30T22:31:06.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.297+0000 7f13a8ff9700 1 -- 172.21.15.16:0/3272186481 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a0007310 con 0x7f13ac0a1340 2023-09-30T22:31:06.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.297+0000 7f13b26e1700 1 -- 172.21.15.16:0/3272186481 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13ac125c60 con 0x7f13ac0a1340 2023-09-30T22:31:06.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.298+0000 7f13a8ff9700 1 -- 172.21.15.16:0/3272186481 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f13a0007470 con 0x7f13ac0a1340 2023-09-30T22:31:06.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.298+0000 7f13b26e1700 1 -- 172.21.15.16:0/3272186481 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13ac126a00 con 0x7f13ac0a1340 2023-09-30T22:31:06.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.299+0000 7f13a8ff9700 1 -- 172.21.15.16:0/3272186481 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f13a003f3d0 con 0x7f13ac0a1340 2023-09-30T22:31:06.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.304+0000 7f13a8ff9700 1 -- 172.21.15.16:0/3272186481 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f13a00123b0 con 0x7f13ac0a1340 2023-09-30T22:31:06.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.321+0000 7f73e1169700 1 Processor -- start 2023-09-30T22:31:06.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.321+0000 7f73e1169700 1 -- start start 2023-09-30T22:31:06.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.321+0000 7f73e1169700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f73dc06a770 0x7f73dc06ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:06.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.321+0000 7f73e1169700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f73dc06b080 0x7f73dc131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:06.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.321+0000 7f73e1169700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73dc09ece0 con 0x7f73dc131b40 2023-09-30T22:31:06.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.321+0000 7f73e1169700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73dc09e760 con 0x7f73dc06a770 2023-09-30T22:31:06.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.321+0000 7f73e1169700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73dc09e040 con 0x7f73dc06b080 2023-09-30T22:31:06.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.321+0000 7f73da59c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f73dc06b080 0x7f73dc131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:06.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.321+0000 7f73db59e700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f73dc131b40 0x7f73dc138850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38734/0 (socket says 172.21.15.16:38734) 2023-09-30T22:31:06.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.321+0000 7f73db59e700 1 -- 172.21.15.16:0/924388170 learned_addr learned my addr 172.21.15.16:0/924388170 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:31:06.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.321+0000 7f73dad9d700 1 --2- 172.21.15.16:0/924388170 >> v2:172.21.15.16:3300/0 conn(0x7f73dc06a770 0x7f73dc06ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:06.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.322+0000 7f73d9d9b700 1 -- 172.21.15.16:0/924388170 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1326440637 0 0) 0x7f73dc09ece0 con 0x7f73dc131b40 2023-09-30T22:31:06.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.322+0000 7f73d9d9b700 1 -- 172.21.15.16:0/924388170 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73c40035b0 con 0x7f73dc131b40 2023-09-30T22:31:06.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.322+0000 7f73da59c700 1 -- 172.21.15.16:0/924388170 >> v2:172.21.15.16:3300/0 conn(0x7f73dc06a770 msgr2=0x7f73dc06ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:06.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.322+0000 7f73da59c700 1 --2- 172.21.15.16:0/924388170 >> v2:172.21.15.16:3300/0 conn(0x7f73dc06a770 0x7f73dc06ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.322+0000 7f73da59c700 1 -- 172.21.15.16:0/924388170 >> v1:172.21.15.16:6789/0 conn(0x7f73dc131b40 legacy=0x7f73dc138850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:06.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.322+0000 7f73da59c700 1 -- 172.21.15.16:0/924388170 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73dc09f630 con 0x7f73dc06b080 2023-09-30T22:31:06.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.322+0000 7f73dad9d700 1 --2- 172.21.15.16:0/924388170 >> v2:172.21.15.16:3300/0 conn(0x7f73dc06a770 0x7f73dc06ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:06.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.322+0000 7f73da59c700 1 --2- 172.21.15.16:0/924388170 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f73dc06b080 0x7f73dc131600 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f73d0009840 tx=0x7f73d0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30a9036c8d9d806 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:06.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.322+0000 7f73d9d9b700 1 -- 172.21.15.16:0/924388170 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d0011830 con 0x7f73dc06b080 2023-09-30T22:31:06.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.322+0000 7f73d9d9b700 1 -- 172.21.15.16:0/924388170 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73d0011990 con 0x7f73dc06b080 2023-09-30T22:31:06.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.322+0000 7f73d9d9b700 1 -- 172.21.15.16:0/924388170 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d0011d20 con 0x7f73dc06b080 2023-09-30T22:31:06.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.323+0000 7f73e1169700 1 -- 172.21.15.16:0/924388170 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f73dc06b080 msgr2=0x7f73dc131600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:06.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.323+0000 7f73e1169700 1 --2- 172.21.15.16:0/924388170 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f73dc06b080 0x7f73dc131600 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f73d0009840 tx=0x7f73d0009b50 comp rx=0 tx=0).stop 2023-09-30T22:31:06.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.323+0000 7f73e1169700 1 -- 172.21.15.16:0/924388170 shutdown_connections 2023-09-30T22:31:06.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.323+0000 7f73e1169700 1 --2- 172.21.15.16:0/924388170 >> v2:172.21.15.16:3300/0 conn(0x7f73dc06a770 0x7f73dc06ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.323+0000 7f73e1169700 1 --2- 172.21.15.16:0/924388170 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f73dc06b080 0x7f73dc131600 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.323+0000 7f73e1169700 1 -- 172.21.15.16:0/924388170 >> 172.21.15.16:0/924388170 conn(0x7f73dc066f50 msgr2=0x7f73dc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:06.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.323+0000 7f73e1169700 1 -- 172.21.15.16:0/924388170 shutdown_connections 2023-09-30T22:31:06.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.323+0000 7f73e1169700 1 -- 172.21.15.16:0/924388170 wait complete. 2023-09-30T22:31:06.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.325+0000 7f73e1169700 1 Processor -- start 2023-09-30T22:31:06.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.325+0000 7f73e1169700 1 -- start start 2023-09-30T22:31:06.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.325+0000 7f73e1169700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73dc06a770 0x7f73dc1c5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:06.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.325+0000 7f73e1169700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f73dc06b080 0x7f73dc1c6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:06.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.325+0000 7f73e1169700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f73dc131b40 0x7f73dc1c6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:06.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.325+0000 7f73e1169700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73dc11e400 con 0x7f73dc06a770 2023-09-30T22:31:06.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.325+0000 7f73e1169700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73dc11fb30 con 0x7f73dc06b080 2023-09-30T22:31:06.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.325+0000 7f73e1169700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f73dc071e60 con 0x7f73dc131b40 2023-09-30T22:31:06.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.325+0000 7f73dad9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73dc06a770 0x7f73dc1c5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:06.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.325+0000 7f73da59c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f73dc06b080 0x7f73dc1c6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:06.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.325+0000 7f73dad9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73dc06a770 0x7f73dc1c5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55742/0 (socket says 172.21.15.16:55742) 2023-09-30T22:31:06.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.325+0000 7f73da59c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f73dc06b080 0x7f73dc1c6510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38688/0 (socket says 172.21.15.16:38688) 2023-09-30T22:31:06.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.325+0000 7f73dad9d700 1 -- 172.21.15.16:0/2583976802 learned_addr learned my addr 172.21.15.16:0/2583976802 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:06.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.325+0000 7f73db59e700 1 --2- 172.21.15.16:0/2583976802 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f73dc131b40 0x7f73dc1c6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:06.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.326+0000 7f73da59c700 1 -- 172.21.15.16:0/2583976802 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f73dc131b40 msgr2=0x7f73dc1c6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:06.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.326+0000 7f73da59c700 1 --2- 172.21.15.16:0/2583976802 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f73dc131b40 0x7f73dc1c6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.326+0000 7f73da59c700 1 -- 172.21.15.16:0/2583976802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73dc06a770 msgr2=0x7f73dc1c5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:06.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.326+0000 7f73da59c700 1 --2- 172.21.15.16:0/2583976802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73dc06a770 0x7f73dc1c5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.326+0000 7f73da59c700 1 -- 172.21.15.16:0/2583976802 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73cc0094a0 con 0x7f73dc06b080 2023-09-30T22:31:06.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.326+0000 7f73dad9d700 1 --2- 172.21.15.16:0/2583976802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73dc06a770 0x7f73dc1c5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:06.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.326+0000 7f73da59c700 1 --2- 172.21.15.16:0/2583976802 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f73dc06b080 0x7f73dc1c6510 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f73d0016000 tx=0x7f73d0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:06.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.326+0000 7f73c37fe700 1 -- 172.21.15.16:0/2583976802 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d0004590 con 0x7f73dc06b080 2023-09-30T22:31:06.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.326+0000 7f73c37fe700 1 -- 172.21.15.16:0/2583976802 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73d0023070 con 0x7f73dc06b080 2023-09-30T22:31:06.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.326+0000 7f73e1169700 1 -- 172.21.15.16:0/2583976802 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73d00094a0 con 0x7f73dc06b080 2023-09-30T22:31:06.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.326+0000 7f73c37fe700 1 -- 172.21.15.16:0/2583976802 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d001fa80 con 0x7f73dc06b080 2023-09-30T22:31:06.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.326+0000 7f73e1169700 1 -- 172.21.15.16:0/2583976802 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73dc120160 con 0x7f73dc06b080 2023-09-30T22:31:06.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.327+0000 7f73c37fe700 1 -- 172.21.15.16:0/2583976802 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f73d001fbe0 con 0x7f73dc06b080 2023-09-30T22:31:06.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.327+0000 7f73e1169700 1 -- 172.21.15.16:0/2583976802 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73dc09dce0 con 0x7f73dc06b080 2023-09-30T22:31:06.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.328+0000 7f73c37fe700 1 -- 172.21.15.16:0/2583976802 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f73d0025070 con 0x7f73dc06b080 2023-09-30T22:31:06.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.332+0000 7f73c37fe700 1 -- 172.21.15.16:0/2583976802 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f73d00036a0 con 0x7f73dc06b080 2023-09-30T22:31:06.422 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:06.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.477+0000 7fce6b85d700 1 -- 172.21.15.16:0/2545354758 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fce640a28e0 con 0x7fce640a0a30 2023-09-30T22:31:06.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.477+0000 7fce59ffb700 1 -- 172.21.15.16:0/2545354758 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fce60015020 con 0x7fce640a0a30 2023-09-30T22:31:06.478 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:31:06.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.480+0000 7fce6b85d700 1 -- 172.21.15.16:0/2545354758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce640a0a30 msgr2=0x7fce64122930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:06.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.480+0000 7fce6b85d700 1 --2- 172.21.15.16:0/2545354758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce640a0a30 0x7fce64122930 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fce60009fd0 tx=0x7fce60004750 comp rx=0 tx=0).stop 2023-09-30T22:31:06.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.480+0000 7fce6b85d700 1 -- 172.21.15.16:0/2545354758 shutdown_connections 2023-09-30T22:31:06.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.480+0000 7fce6b85d700 1 --2- 172.21.15.16:0/2545354758 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fce64134170 0x7fce641233b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.480+0000 7fce6b85d700 1 --2- 172.21.15.16:0/2545354758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce640a0a30 0x7fce64122930 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.480+0000 7fce6b85d700 1 --2- 172.21.15.16:0/2545354758 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fce640a1340 0x7fce64122e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.480+0000 7fce6b85d700 1 -- 172.21.15.16:0/2545354758 >> 172.21.15.16:0/2545354758 conn(0x7fce640a6550 msgr2=0x7fce6409ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:06.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.480+0000 7fce6b85d700 1 -- 172.21.15.16:0/2545354758 shutdown_connections 2023-09-30T22:31:06.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.480+0000 7fce6b85d700 1 -- 172.21.15.16:0/2545354758 wait complete. 2023-09-30T22:31:06.499 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:31:06.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.517+0000 7f13b26e1700 1 -- 172.21.15.16:0/3272186481 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f13ac074ea0 con 0x7f13ac0a1340 2023-09-30T22:31:06.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.518+0000 7f13a8ff9700 1 -- 172.21.15.16:0/3272186481 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f13a0011360 con 0x7f13ac0a1340 2023-09-30T22:31:06.518 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:31:06.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.520+0000 7f13b26e1700 1 -- 172.21.15.16:0/3272186481 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f13ac0a1340 msgr2=0x7f13ac122bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:06.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.520+0000 7f13b26e1700 1 --2- 172.21.15.16:0/3272186481 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f13ac0a1340 0x7f13ac122bd0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f13a000c520 tx=0x7f13a000c830 comp rx=0 tx=0).stop 2023-09-30T22:31:06.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.520+0000 7f13b26e1700 1 -- 172.21.15.16:0/3272186481 shutdown_connections 2023-09-30T22:31:06.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.520+0000 7f13b26e1700 1 --2- 172.21.15.16:0/3272186481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13ac134170 0x7f13ac123110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.520+0000 7f13b26e1700 1 --2- 172.21.15.16:0/3272186481 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f13ac0a0a30 0x7f13ac122690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.520+0000 7f13b26e1700 1 --2- 172.21.15.16:0/3272186481 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f13ac0a1340 0x7f13ac122bd0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.520+0000 7f13b26e1700 1 -- 172.21.15.16:0/3272186481 >> 172.21.15.16:0/3272186481 conn(0x7f13ac0a6550 msgr2=0x7f13ac09edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:06.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.521+0000 7f13b26e1700 1 -- 172.21.15.16:0/3272186481 shutdown_connections 2023-09-30T22:31:06.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.521+0000 7f13b26e1700 1 -- 172.21.15.16:0/3272186481 wait complete. 2023-09-30T22:31:06.537 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:31:06.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.542+0000 7f73e1169700 1 -- 172.21.15.16:0/2583976802 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f73dc074ea0 con 0x7f73dc06b080 2023-09-30T22:31:06.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.543+0000 7f73c37fe700 1 -- 172.21.15.16:0/2583976802 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f73d0038020 con 0x7f73dc06b080 2023-09-30T22:31:06.543 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:31:06.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.545+0000 7f73e1169700 1 -- 172.21.15.16:0/2583976802 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f73dc06b080 msgr2=0x7f73dc1c6510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:06.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.545+0000 7f73e1169700 1 --2- 172.21.15.16:0/2583976802 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f73dc06b080 0x7f73dc1c6510 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f73d0016000 tx=0x7f73d0004750 comp rx=0 tx=0).stop 2023-09-30T22:31:06.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.545+0000 7f73e1169700 1 -- 172.21.15.16:0/2583976802 shutdown_connections 2023-09-30T22:31:06.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.545+0000 7f73e1169700 1 --2- 172.21.15.16:0/2583976802 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f73dc131b40 0x7f73dc1c6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.545+0000 7f73e1169700 1 --2- 172.21.15.16:0/2583976802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73dc06a770 0x7f73dc1c5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.545+0000 7f73e1169700 1 --2- 172.21.15.16:0/2583976802 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f73dc06b080 0x7f73dc1c6510 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.545+0000 7f73e1169700 1 -- 172.21.15.16:0/2583976802 >> 172.21.15.16:0/2583976802 conn(0x7f73dc066f50 msgr2=0x7f73dc06ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:06.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.545+0000 7f73e1169700 1 -- 172.21.15.16:0/2583976802 shutdown_connections 2023-09-30T22:31:06.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.545+0000 7f73e1169700 1 -- 172.21.15.16:0/2583976802 wait complete. 2023-09-30T22:31:06.559 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:31:06.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.575+0000 7f0ddeccc700 1 Processor -- start 2023-09-30T22:31:06.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.575+0000 7f0ddeccc700 1 -- start start 2023-09-30T22:31:06.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.576+0000 7f0ddeccc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0dd80a0a30 0x7f0dd80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:06.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.576+0000 7f0ddeccc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0dd8138d80 0x7f0dd813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:06.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.576+0000 7f0ddeccc700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dd80a2a70 con 0x7f0dd809d630 2023-09-30T22:31:06.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.576+0000 7f0ddeccc700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dd80a01d0 con 0x7f0dd80a0a30 2023-09-30T22:31:06.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.576+0000 7f0ddeccc700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd809e2b0 con 0x7f0dd8138d80 2023-09-30T22:31:06.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.576+0000 7f0ddca68700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0dd80a0a30 0x7f0dd80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:06.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.576+0000 7f0ddd269700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0dd8138d80 0x7f0dd813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:06.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.576+0000 7f0dd7fff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0dd809d630 0x7f0dd80a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38740/0 (socket says 172.21.15.16:38740) 2023-09-30T22:31:06.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.576+0000 7f0ddca68700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0dd80a0a30 0x7f0dd80a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55750/0 (socket says 172.21.15.16:55750) 2023-09-30T22:31:06.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.576+0000 7f0dd7fff700 1 -- 172.21.15.16:0/1160542804 learned_addr learned my addr 172.21.15.16:0/1160542804 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:31:06.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.576+0000 7f0ddd269700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0dd8138d80 0x7f0dd813b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38694/0 (socket says 172.21.15.16:38694) 2023-09-30T22:31:06.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.576+0000 7f0ddca68700 1 -- 172.21.15.16:0/1160542804 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0dd8138d80 msgr2=0x7f0dd813b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:06.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.576+0000 7f0ddca68700 1 --2- 172.21.15.16:0/1160542804 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0dd8138d80 0x7f0dd813b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.576+0000 7f0ddca68700 1 -- 172.21.15.16:0/1160542804 >> v1:172.21.15.16:6789/0 conn(0x7f0dd809d630 legacy=0x7f0dd80a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:06.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.576+0000 7f0ddca68700 1 -- 172.21.15.16:0/1160542804 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dd809dc50 con 0x7f0dd80a0a30 2023-09-30T22:31:06.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.576+0000 7f0dd77fe700 1 -- 172.21.15.16:0/1160542804 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2623277092 0 0) 0x7f0dd80a2a70 con 0x7f0dd809d630 2023-09-30T22:31:06.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.576+0000 7f0ddd269700 1 --2- 172.21.15.16:0/1160542804 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0dd8138d80 0x7f0dd813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:06.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.576+0000 7f0ddca68700 1 --2- 172.21.15.16:0/1160542804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dd80a0a30 0x7f0dd80a0e00 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8009840 tx=0x7f0dc8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1560f30aca8e833 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:06.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.577+0000 7f0dd77fe700 1 -- 172.21.15.16:0/1160542804 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc8011830 con 0x7f0dd80a0a30 2023-09-30T22:31:06.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.577+0000 7f0dd77fe700 1 -- 172.21.15.16:0/1160542804 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dc8011990 con 0x7f0dd80a0a30 2023-09-30T22:31:06.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.577+0000 7f0dd77fe700 1 -- 172.21.15.16:0/1160542804 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc8011d20 con 0x7f0dd80a0a30 2023-09-30T22:31:06.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.577+0000 7f0ddeccc700 1 -- 172.21.15.16:0/1160542804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dd80a0a30 msgr2=0x7f0dd80a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:06.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.577+0000 7f0ddeccc700 1 --2- 172.21.15.16:0/1160542804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dd80a0a30 0x7f0dd80a0e00 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8009840 tx=0x7f0dc8009b50 comp rx=0 tx=0).stop 2023-09-30T22:31:06.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.577+0000 7f0ddeccc700 1 -- 172.21.15.16:0/1160542804 shutdown_connections 2023-09-30T22:31:06.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.577+0000 7f0ddeccc700 1 --2- 172.21.15.16:0/1160542804 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0dd8138d80 0x7f0dd813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.577+0000 7f0ddeccc700 1 --2- 172.21.15.16:0/1160542804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dd80a0a30 0x7f0dd80a0e00 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.577+0000 7f0ddeccc700 1 -- 172.21.15.16:0/1160542804 >> 172.21.15.16:0/1160542804 conn(0x7f0dd80a3050 msgr2=0x7f0dd80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:06.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.577+0000 7f0ddeccc700 1 -- 172.21.15.16:0/1160542804 shutdown_connections 2023-09-30T22:31:06.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.577+0000 7f0ddeccc700 1 -- 172.21.15.16:0/1160542804 wait complete. 2023-09-30T22:31:06.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.578+0000 7f0ddeccc700 1 Processor -- start 2023-09-30T22:31:06.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.578+0000 7f0ddeccc700 1 -- start start 2023-09-30T22:31:06.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.579+0000 7f0ddeccc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0dd809d630 0x7f0dd8133760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:06.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.579+0000 7f0ddeccc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dd80a0a30 0x7f0dd8133ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:06.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.579+0000 7f0ddeccc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0dd8138d80 0x7f0dd81341e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:06.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.579+0000 7f0ddeccc700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd8136fa0 con 0x7f0dd80a0a30 2023-09-30T22:31:06.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.579+0000 7f0ddeccc700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd8136630 con 0x7f0dd809d630 2023-09-30T22:31:06.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.579+0000 7f0ddeccc700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0dd80a4b50 con 0x7f0dd8138d80 2023-09-30T22:31:06.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.579+0000 7f0dd7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dd80a0a30 0x7f0dd8133ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:06.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.579+0000 7f0ddca68700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0dd809d630 0x7f0dd8133760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:06.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.579+0000 7f0ddca68700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0dd809d630 0x7f0dd8133760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38698/0 (socket says 172.21.15.16:38698) 2023-09-30T22:31:06.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.579+0000 7f0ddca68700 1 -- 172.21.15.16:0/2203079333 learned_addr learned my addr 172.21.15.16:0/2203079333 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:06.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.579+0000 7f0dd7fff700 1 --2- 172.21.15.16:0/2203079333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dd80a0a30 0x7f0dd8133ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55754/0 (socket says 172.21.15.16:55754) 2023-09-30T22:31:06.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.579+0000 7f0ddd269700 1 --2- 172.21.15.16:0/2203079333 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0dd8138d80 0x7f0dd81341e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:06.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.579+0000 7f0ddca68700 1 -- 172.21.15.16:0/2203079333 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0dd8138d80 msgr2=0x7f0dd81341e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:06.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.579+0000 7f0ddca68700 1 --2- 172.21.15.16:0/2203079333 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0dd8138d80 0x7f0dd81341e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.579+0000 7f0ddca68700 1 -- 172.21.15.16:0/2203079333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dd80a0a30 msgr2=0x7f0dd8133ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:06.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.579+0000 7f0ddca68700 1 --2- 172.21.15.16:0/2203079333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dd80a0a30 0x7f0dd8133ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.579+0000 7f0ddca68700 1 -- 172.21.15.16:0/2203079333 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dd0009a70 con 0x7f0dd809d630 2023-09-30T22:31:06.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.579+0000 7f0dd7fff700 1 --2- 172.21.15.16:0/2203079333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dd80a0a30 0x7f0dd8133ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:06.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.580+0000 7f0ddca68700 1 --2- 172.21.15.16:0/2203079333 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0dd809d630 0x7f0dd8133760 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8009fd0 tx=0x7f0dc8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:06.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.580+0000 7f0dd57fa700 1 -- 172.21.15.16:0/2203079333 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc8004400 con 0x7f0dd809d630 2023-09-30T22:31:06.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.580+0000 7f0dd57fa700 1 -- 172.21.15.16:0/2203079333 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dc801f070 con 0x7f0dd809d630 2023-09-30T22:31:06.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.580+0000 7f0ddeccc700 1 -- 172.21.15.16:0/2203079333 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dc80094a0 con 0x7f0dd809d630 2023-09-30T22:31:06.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.580+0000 7f0dd57fa700 1 -- 172.21.15.16:0/2203079333 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc80038e0 con 0x7f0dd809d630 2023-09-30T22:31:06.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.580+0000 7f0ddeccc700 1 -- 172.21.15.16:0/2203079333 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dd8137a10 con 0x7f0dd809d630 2023-09-30T22:31:06.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.581+0000 7f0dd57fa700 1 -- 172.21.15.16:0/2203079333 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0dc8021030 con 0x7f0dd809d630 2023-09-30T22:31:06.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.581+0000 7f0ddeccc700 1 -- 172.21.15.16:0/2203079333 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dd8136c40 con 0x7f0dd809d630 2023-09-30T22:31:06.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.581+0000 7f0dd57fa700 1 -- 172.21.15.16:0/2203079333 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f0dc8040700 con 0x7f0dd809d630 2023-09-30T22:31:06.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.586+0000 7f0dd57fa700 1 -- 172.21.15.16:0/2203079333 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0dc80239c0 con 0x7f0dd809d630 2023-09-30T22:31:06.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.790+0000 7f0ddeccc700 1 -- 172.21.15.16:0/2203079333 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0dd80747f0 con 0x7f0dd809d630 2023-09-30T22:31:06.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.790+0000 7f0dd57fa700 1 -- 172.21.15.16:0/2203079333 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0dc8015020 con 0x7f0dd809d630 2023-09-30T22:31:06.791 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:31:06.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.792+0000 7f0ddeccc700 1 -- 172.21.15.16:0/2203079333 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0dd809d630 msgr2=0x7f0dd8133760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:06.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.792+0000 7f0ddeccc700 1 --2- 172.21.15.16:0/2203079333 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0dd809d630 0x7f0dd8133760 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0dc8009fd0 tx=0x7f0dc8004750 comp rx=0 tx=0).stop 2023-09-30T22:31:06.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.793+0000 7f0ddeccc700 1 -- 172.21.15.16:0/2203079333 shutdown_connections 2023-09-30T22:31:06.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.793+0000 7f0ddeccc700 1 --2- 172.21.15.16:0/2203079333 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0dd809d630 0x7f0dd8133760 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.793+0000 7f0ddeccc700 1 --2- 172.21.15.16:0/2203079333 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0dd8138d80 0x7f0dd81341e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.793+0000 7f0ddeccc700 1 --2- 172.21.15.16:0/2203079333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dd80a0a30 0x7f0dd8133ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:06.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.793+0000 7f0ddeccc700 1 -- 172.21.15.16:0/2203079333 >> 172.21.15.16:0/2203079333 conn(0x7f0dd80a3050 msgr2=0x7f0dd813ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:06.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.793+0000 7f0ddeccc700 1 -- 172.21.15.16:0/2203079333 shutdown_connections 2023-09-30T22:31:06.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:06.793+0000 7f0ddeccc700 1 -- 172.21.15.16:0/2203079333 wait complete. 2023-09-30T22:31:06.807 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:31:07.500 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:07.538 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:07.560 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:07.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.651+0000 7fa4313a7700 1 Processor -- start 2023-09-30T22:31:07.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.651+0000 7fa4313a7700 1 -- start start 2023-09-30T22:31:07.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.652+0000 7fa4313a7700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa42c131810 0x7fa42c131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:07.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.652+0000 7fa4313a7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa42c099a40 0x7fa42c099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:07.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.652+0000 7fa4313a7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa42c09a3b0 0x7fa42c138650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:07.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.652+0000 7fa4313a7700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa42c1313c0 con 0x7fa42c131810 2023-09-30T22:31:07.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.652+0000 7fa4313a7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa42c06e8c0 con 0x7fa42c099a40 2023-09-30T22:31:07.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.652+0000 7fa4313a7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa42c071d20 con 0x7fa42c09a3b0 2023-09-30T22:31:07.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.652+0000 7fa42affd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa42c131810 0x7fa42c131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:07.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.652+0000 7fa42a7fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa42c099a40 0x7fa42c099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:07.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.652+0000 7fa42b7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa42c09a3b0 0x7fa42c138650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:07.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.652+0000 7fa42affd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa42c131810 0x7fa42c131be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45742/0 (socket says 172.21.15.16:45742) 2023-09-30T22:31:07.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.652+0000 7fa42a7fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa42c099a40 0x7fa42c099e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:34990/0 (socket says 172.21.15.16:34990) 2023-09-30T22:31:07.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.652+0000 7fa42affd700 1 -- 172.21.15.16:0/745408509 learned_addr learned my addr 172.21.15.16:0/745408509 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:07.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.652+0000 7fa42b7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa42c09a3b0 0x7fa42c138650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:59614/0 (socket says 172.21.15.16:59614) 2023-09-30T22:31:07.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.652+0000 7fa42b7fe700 1 -- 172.21.15.16:0/745408509 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa42c099a40 msgr2=0x7fa42c099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:07.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.652+0000 7fa42b7fe700 1 --2- 172.21.15.16:0/745408509 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa42c099a40 0x7fa42c099e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:07.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.653+0000 7fa42b7fe700 1 -- 172.21.15.16:0/745408509 >> v2:172.21.15.16:3300/0 conn(0x7fa42c131810 msgr2=0x7fa42c131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:07.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.653+0000 7fa42b7fe700 1 --2- 172.21.15.16:0/745408509 >> v2:172.21.15.16:3300/0 conn(0x7fa42c131810 0x7fa42c131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:07.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.653+0000 7fa42b7fe700 1 -- 172.21.15.16:0/745408509 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa42c0721a0 con 0x7fa42c09a3b0 2023-09-30T22:31:07.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.653+0000 7fa42affd700 1 --2- 172.21.15.16:0/745408509 >> v2:172.21.15.16:3300/0 conn(0x7fa42c131810 0x7fa42c131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:07.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.653+0000 7fa42b7fe700 1 --2- 172.21.15.16:0/745408509 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa42c09a3b0 0x7fa42c138650 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa424009d90 tx=0x7fa42400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=baf47776a3cf3a5c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:07.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.653+0000 7fa429ffb700 1 -- 172.21.15.16:0/745408509 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa42400e070 con 0x7fa42c09a3b0 2023-09-30T22:31:07.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.653+0000 7fa429ffb700 1 -- 172.21.15.16:0/745408509 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa42400bb20 con 0x7fa42c09a3b0 2023-09-30T22:31:07.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.653+0000 7fa429ffb700 1 -- 172.21.15.16:0/745408509 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa424012430 con 0x7fa42c09a3b0 2023-09-30T22:31:07.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.654+0000 7fa4313a7700 1 -- 172.21.15.16:0/745408509 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa42c09a3b0 msgr2=0x7fa42c138650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:07.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.654+0000 7fa4313a7700 1 --2- 172.21.15.16:0/745408509 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa42c09a3b0 0x7fa42c138650 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa424009d90 tx=0x7fa42400b330 comp rx=0 tx=0).stop 2023-09-30T22:31:07.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.654+0000 7fa4313a7700 1 -- 172.21.15.16:0/745408509 shutdown_connections 2023-09-30T22:31:07.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.654+0000 7fa4313a7700 1 --2- 172.21.15.16:0/745408509 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa42c09a3b0 0x7fa42c138650 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:07.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.654+0000 7fa4313a7700 1 --2- 172.21.15.16:0/745408509 >> v2:172.21.15.16:3300/0 conn(0x7fa42c131810 0x7fa42c131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:07.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.654+0000 7fa4313a7700 1 --2- 172.21.15.16:0/745408509 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa42c099a40 0x7fa42c099e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:07.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.654+0000 7fa4313a7700 1 -- 172.21.15.16:0/745408509 >> 172.21.15.16:0/745408509 conn(0x7fa42c0a2aa0 msgr2=0x7fa42c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:07.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.654+0000 7fa4313a7700 1 -- 172.21.15.16:0/745408509 shutdown_connections 2023-09-30T22:31:07.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.654+0000 7fa4313a7700 1 -- 172.21.15.16:0/745408509 wait complete. 2023-09-30T22:31:07.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.655+0000 7fa4313a7700 1 Processor -- start 2023-09-30T22:31:07.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.655+0000 7fa4313a7700 1 -- start start 2023-09-30T22:31:07.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.655+0000 7fa4313a7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa42c099a40 0x7fa42c1196e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:07.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.655+0000 7fa4313a7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa42c09a3b0 0x7fa42c119c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:07.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.655+0000 7fa4313a7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa42c131810 0x7fa42c11a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:07.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.655+0000 7fa4313a7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa42c11b190 con 0x7fa42c099a40 2023-09-30T22:31:07.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.655+0000 7fa4313a7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa42c06eb00 con 0x7fa42c09a3b0 2023-09-30T22:31:07.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.655+0000 7fa4313a7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa42c11cdc0 con 0x7fa42c131810 2023-09-30T22:31:07.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.656+0000 7fa42b7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa42c131810 0x7fa42c11a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:07.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.656+0000 7fa42b7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa42c131810 0x7fa42c11a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:59630/0 (socket says 172.21.15.16:59630) 2023-09-30T22:31:07.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.656+0000 7fa42b7fe700 1 -- 172.21.15.16:0/1786357890 learned_addr learned my addr 172.21.15.16:0/1786357890 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:07.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.656+0000 7fa42affd700 1 --2- 172.21.15.16:0/1786357890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa42c099a40 0x7fa42c1196e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:07.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.656+0000 7fa42a7fc700 1 --2- 172.21.15.16:0/1786357890 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa42c09a3b0 0x7fa42c119c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:07.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.656+0000 7fa42b7fe700 1 -- 172.21.15.16:0/1786357890 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa42c09a3b0 msgr2=0x7fa42c119c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:07.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.656+0000 7fa42b7fe700 1 --2- 172.21.15.16:0/1786357890 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa42c09a3b0 0x7fa42c119c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:07.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.656+0000 7fa42b7fe700 1 -- 172.21.15.16:0/1786357890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa42c099a40 msgr2=0x7fa42c1196e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:07.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.656+0000 7fa42b7fe700 1 --2- 172.21.15.16:0/1786357890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa42c099a40 0x7fa42c1196e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:07.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.656+0000 7fa42b7fe700 1 -- 172.21.15.16:0/1786357890 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa41c009550 con 0x7fa42c131810 2023-09-30T22:31:07.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.656+0000 7fa42a7fc700 1 --2- 172.21.15.16:0/1786357890 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa42c09a3b0 0x7fa42c119c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:31:07.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.656+0000 7fa42b7fe700 1 --2- 172.21.15.16:0/1786357890 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa42c131810 0x7fa42c11a160 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa424014040 tx=0x7fa424004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:07.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.656+0000 7fa4137fe700 1 -- 172.21.15.16:0/1786357890 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa424012a40 con 0x7fa42c131810 2023-09-30T22:31:07.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.656+0000 7fa4137fe700 1 -- 172.21.15.16:0/1786357890 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa424020070 con 0x7fa42c131810 2023-09-30T22:31:07.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.656+0000 7fa4137fe700 1 -- 172.21.15.16:0/1786357890 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa424012e00 con 0x7fa42c131810 2023-09-30T22:31:07.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.657+0000 7fa4313a7700 1 -- 172.21.15.16:0/1786357890 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa424009a70 con 0x7fa42c131810 2023-09-30T22:31:07.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.657+0000 7fa4313a7700 1 -- 172.21.15.16:0/1786357890 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa42c11c3e0 con 0x7fa42c131810 2023-09-30T22:31:07.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.658+0000 7fa4137fe700 1 -- 172.21.15.16:0/1786357890 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa424004040 con 0x7fa42c131810 2023-09-30T22:31:07.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.658+0000 7fa4137fe700 1 -- 172.21.15.16:0/1786357890 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fa424022070 con 0x7fa42c131810 2023-09-30T22:31:07.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.658+0000 7fa4313a7700 1 -- 172.21.15.16:0/1786357890 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa42c074ea0 con 0x7fa42c131810 2023-09-30T22:31:07.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.663+0000 7fa4137fe700 1 -- 172.21.15.16:0/1786357890 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa42401a970 con 0x7fa42c131810 2023-09-30T22:31:07.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.689+0000 7f6fd89f2700 1 Processor -- start 2023-09-30T22:31:07.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.689+0000 7f6fd89f2700 1 -- start start 2023-09-30T22:31:07.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.689+0000 7f6fd89f2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6fd409d6f0 0x7f6fd409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:07.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.689+0000 7f6fd89f2700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6fd409e000 0x7f6fd41339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:07.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.689+0000 7f6fd89f2700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fd409b120 con 0x7f6fd409e410 2023-09-30T22:31:07.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.689+0000 7f6fd89f2700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fd406b460 con 0x7f6fd409e000 2023-09-30T22:31:07.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.689+0000 7f6fd89f2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd409a7e0 con 0x7f6fd409d6f0 2023-09-30T22:31:07.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.690+0000 7f6fd259c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6fd409d6f0 0x7f6fd409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:07.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.690+0000 7f6fd259c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6fd409d6f0 0x7f6fd409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:35000/0 (socket says 172.21.15.16:35000) 2023-09-30T22:31:07.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.690+0000 7f6fd259c700 1 -- 172.21.15.16:0/2181587043 learned_addr learned my addr 172.21.15.16:0/2181587043 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:07.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.690+0000 7f6fd1d9b700 1 --2- 172.21.15.16:0/2181587043 >> v2:172.21.15.16:3300/0 conn(0x7f6fd409e000 0x7f6fd41339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:07.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.690+0000 7f6fd259c700 1 -- 172.21.15.16:0/2181587043 >> v2:172.21.15.16:3300/0 conn(0x7f6fd409e000 msgr2=0x7f6fd41339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:07.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.690+0000 7f6fd259c700 1 --2- 172.21.15.16:0/2181587043 >> v2:172.21.15.16:3300/0 conn(0x7f6fd409e000 0x7f6fd41339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:07.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.690+0000 7f6fd259c700 1 -- 172.21.15.16:0/2181587043 >> v1:172.21.15.16:6789/0 conn(0x7f6fd409e410 legacy=0x7f6fd4133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:07.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.690+0000 7f6fd259c700 1 -- 172.21.15.16:0/2181587043 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd409a500 con 0x7f6fd409d6f0 2023-09-30T22:31:07.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.690+0000 7f6fd159a700 1 -- 172.21.15.16:0/2181587043 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 542045027 0 0) 0x7f6fd409b120 con 0x7f6fd409e410 2023-09-30T22:31:07.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.690+0000 7f6fd1d9b700 1 --2- 172.21.15.16:0/2181587043 >> v2:172.21.15.16:3300/0 conn(0x7f6fd409e000 0x7f6fd41339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:07.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.690+0000 7f6fd259c700 1 --2- 172.21.15.16:0/2181587043 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6fd409d6f0 0x7f6fd409dac0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4009920 tx=0x7f6fc4009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b97f3befba7f2a3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:07.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.690+0000 7f6fd159a700 1 -- 172.21.15.16:0/2181587043 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc4011830 con 0x7f6fd409d6f0 2023-09-30T22:31:07.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.691+0000 7f6fd159a700 1 -- 172.21.15.16:0/2181587043 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fc4011990 con 0x7f6fd409d6f0 2023-09-30T22:31:07.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.691+0000 7f6fd159a700 1 -- 172.21.15.16:0/2181587043 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc4011d20 con 0x7f6fd409d6f0 2023-09-30T22:31:07.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.691+0000 7f6fd89f2700 1 -- 172.21.15.16:0/2181587043 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6fd409d6f0 msgr2=0x7f6fd409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:07.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.691+0000 7f6fd89f2700 1 --2- 172.21.15.16:0/2181587043 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6fd409d6f0 0x7f6fd409dac0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4009920 tx=0x7f6fc4009c30 comp rx=0 tx=0).stop 2023-09-30T22:31:07.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.691+0000 7f6fd89f2700 1 -- 172.21.15.16:0/2181587043 shutdown_connections 2023-09-30T22:31:07.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.691+0000 7f6fd89f2700 1 --2- 172.21.15.16:0/2181587043 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6fd409d6f0 0x7f6fd409dac0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:07.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.691+0000 7f6fd89f2700 1 --2- 172.21.15.16:0/2181587043 >> v2:172.21.15.16:3300/0 conn(0x7f6fd409e000 0x7f6fd41339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:07.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.691+0000 7f6fd89f2700 1 -- 172.21.15.16:0/2181587043 >> 172.21.15.16:0/2181587043 conn(0x7f6fd4066f50 msgr2=0x7f6fd409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:07.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.691+0000 7f6fd89f2700 1 -- 172.21.15.16:0/2181587043 shutdown_connections 2023-09-30T22:31:07.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.691+0000 7f6fd89f2700 1 -- 172.21.15.16:0/2181587043 wait complete. 2023-09-30T22:31:07.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.693+0000 7f6fd89f2700 1 Processor -- start 2023-09-30T22:31:07.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.693+0000 7f6fd89f2700 1 -- start start 2023-09-30T22:31:07.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.694+0000 7f6fd89f2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fd409d6f0 0x7f6fd41c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:07.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.694+0000 7f6fd89f2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6fd409e000 0x7f6fd41c8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:07.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.694+0000 7f6fd89f2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6fd409e410 0x7f6fd41c89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:07.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.694+0000 7f6fd89f2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd41a7f00 con 0x7f6fd409d6f0 2023-09-30T22:31:07.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.694+0000 7f6fd89f2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd41a62d0 con 0x7f6fd409e410 2023-09-30T22:31:07.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.694+0000 7f6fd89f2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd41a7c40 con 0x7f6fd409e000 2023-09-30T22:31:07.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.694+0000 7f6fd1d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6fd409e000 0x7f6fd41c8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:07.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.694+0000 7f6fd1d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6fd409e000 0x7f6fd41c8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:59642/0 (socket says 172.21.15.16:59642) 2023-09-30T22:31:07.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.694+0000 7f6fd1d9b700 1 -- 172.21.15.16:0/24296981 learned_addr learned my addr 172.21.15.16:0/24296981 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:07.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.694+0000 7f6fd2d9d700 1 --2- 172.21.15.16:0/24296981 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6fd409e410 0x7f6fd41c89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:07.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.694+0000 7f6fd259c700 1 --2- 172.21.15.16:0/24296981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fd409d6f0 0x7f6fd41c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:07.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.694+0000 7f6fd1d9b700 1 -- 172.21.15.16:0/24296981 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6fd409e410 msgr2=0x7f6fd41c89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:07.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.694+0000 7f6fd1d9b700 1 --2- 172.21.15.16:0/24296981 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6fd409e410 0x7f6fd41c89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:07.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.694+0000 7f6fd1d9b700 1 -- 172.21.15.16:0/24296981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fd409d6f0 msgr2=0x7f6fd41c7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:07.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.694+0000 7f6fd1d9b700 1 --2- 172.21.15.16:0/24296981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fd409d6f0 0x7f6fd41c7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:07.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.694+0000 7f6fd1d9b700 1 -- 172.21.15.16:0/24296981 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc80094a0 con 0x7f6fd409e000 2023-09-30T22:31:07.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.695+0000 7f6fd1d9b700 1 --2- 172.21.15.16:0/24296981 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6fd409e000 0x7f6fd41c8480 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6fc800ebe0 tx=0x7f6fc800eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:07.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.695+0000 7f6fc2ffd700 1 -- 172.21.15.16:0/24296981 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc800c850 con 0x7f6fd409e000 2023-09-30T22:31:07.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.695+0000 7f6fc2ffd700 1 -- 172.21.15.16:0/24296981 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fc800c9b0 con 0x7f6fd409e000 2023-09-30T22:31:07.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.695+0000 7f6fd89f2700 1 -- 172.21.15.16:0/24296981 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fc4009550 con 0x7f6fd409e000 2023-09-30T22:31:07.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.695+0000 7f6fc2ffd700 1 -- 172.21.15.16:0/24296981 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc800cd40 con 0x7f6fd409e000 2023-09-30T22:31:07.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.695+0000 7f6fd89f2700 1 -- 172.21.15.16:0/24296981 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fd41b9170 con 0x7f6fd409e000 2023-09-30T22:31:07.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.696+0000 7f6fc2ffd700 1 -- 172.21.15.16:0/24296981 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6fc8013470 con 0x7f6fd409e000 2023-09-30T22:31:07.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.696+0000 7f6fc2ffd700 1 -- 172.21.15.16:0/24296981 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f6fc803fc40 con 0x7f6fd409e000 2023-09-30T22:31:07.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.696+0000 7f6fd89f2700 1 -- 172.21.15.16:0/24296981 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fd41a8050 con 0x7f6fd409e000 2023-09-30T22:31:07.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.701+0000 7f6fc2ffd700 1 -- 172.21.15.16:0/24296981 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6fc8021080 con 0x7f6fd409e000 2023-09-30T22:31:07.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.731+0000 7fe5a0359700 1 Processor -- start 2023-09-30T22:31:07.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.731+0000 7fe5a0359700 1 -- start start 2023-09-30T22:31:07.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.732+0000 7fe5a0359700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe5980a0a30 0x7fe5980a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:07.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.732+0000 7fe5a0359700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe598138d80 0x7fe59813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:07.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.732+0000 7fe5a0359700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5980a2a70 con 0x7fe59809d630 2023-09-30T22:31:07.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.732+0000 7fe5a0359700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5980a01d0 con 0x7fe598138d80 2023-09-30T22:31:07.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.732+0000 7fe5a0359700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe59809e2b0 con 0x7fe5980a0a30 2023-09-30T22:31:07.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.732+0000 7fe59e8f6700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe598138d80 0x7fe59813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:07.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.732+0000 7fe59e0f5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe5980a0a30 0x7fe5980a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:07.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.732+0000 7fe59e8f6700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe598138d80 0x7fe59813b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45786/0 (socket says 172.21.15.16:45786) 2023-09-30T22:31:07.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.732+0000 7fe59e8f6700 1 -- 172.21.15.16:0/3002939150 learned_addr learned my addr 172.21.15.16:0/3002939150 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:07.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.732+0000 7fe59e8f6700 1 -- 172.21.15.16:0/3002939150 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe5980a0a30 msgr2=0x7fe5980a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:07.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.732+0000 7fe59e8f6700 1 --2- 172.21.15.16:0/3002939150 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe5980a0a30 0x7fe5980a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:07.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.732+0000 7fe59e8f6700 1 -- 172.21.15.16:0/3002939150 >> v1:172.21.15.16:6789/0 conn(0x7fe59809d630 legacy=0x7fe5980a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:07.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.732+0000 7fe59d0f3700 1 -- 172.21.15.16:0/3002939150 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3538581907 0 0) 0x7fe5980a2a70 con 0x7fe59809d630 2023-09-30T22:31:07.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.732+0000 7fe59e8f6700 1 -- 172.21.15.16:0/3002939150 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe59809dc50 con 0x7fe598138d80 2023-09-30T22:31:07.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.732+0000 7fe59e0f5700 1 --2- 172.21.15.16:0/3002939150 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe5980a0a30 0x7fe5980a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:07.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.733+0000 7fe59e8f6700 1 --2- 172.21.15.16:0/3002939150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe598138d80 0x7fe59813b160 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fe5900098f0 tx=0x7fe590009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0cb59ee29cf8b55 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:07.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.733+0000 7fe59d0f3700 1 -- 172.21.15.16:0/3002939150 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe590011830 con 0x7fe598138d80 2023-09-30T22:31:07.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.733+0000 7fe59d0f3700 1 -- 172.21.15.16:0/3002939150 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe590011990 con 0x7fe598138d80 2023-09-30T22:31:07.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.733+0000 7fe59d0f3700 1 -- 172.21.15.16:0/3002939150 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe590011d20 con 0x7fe598138d80 2023-09-30T22:31:07.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.733+0000 7fe5a0359700 1 -- 172.21.15.16:0/3002939150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe598138d80 msgr2=0x7fe59813b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:07.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.733+0000 7fe5a0359700 1 --2- 172.21.15.16:0/3002939150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe598138d80 0x7fe59813b160 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fe5900098f0 tx=0x7fe590009c00 comp rx=0 tx=0).stop 2023-09-30T22:31:07.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.733+0000 7fe5a0359700 1 -- 172.21.15.16:0/3002939150 shutdown_connections 2023-09-30T22:31:07.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.734+0000 7fe5a0359700 1 --2- 172.21.15.16:0/3002939150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe598138d80 0x7fe59813b160 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:07.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.734+0000 7fe5a0359700 1 --2- 172.21.15.16:0/3002939150 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe5980a0a30 0x7fe5980a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:07.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.734+0000 7fe5a0359700 1 -- 172.21.15.16:0/3002939150 >> 172.21.15.16:0/3002939150 conn(0x7fe5980a3050 msgr2=0x7fe5980a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:07.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.734+0000 7fe5a0359700 1 -- 172.21.15.16:0/3002939150 shutdown_connections 2023-09-30T22:31:07.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.734+0000 7fe5a0359700 1 -- 172.21.15.16:0/3002939150 wait complete. 2023-09-30T22:31:07.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.735+0000 7fe5a0359700 1 Processor -- start 2023-09-30T22:31:07.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.735+0000 7fe5a0359700 1 -- start start 2023-09-30T22:31:07.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.735+0000 7fe5a0359700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe59809d630 0x7fe598133690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:07.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.735+0000 7fe5a0359700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe5980a0a30 0x7fe598133bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:07.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.735+0000 7fe5a0359700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe598138d80 0x7fe598134110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:07.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.735+0000 7fe5a0359700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5981361b0 con 0x7fe59809d630 2023-09-30T22:31:07.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.735+0000 7fe5a0359700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe598136ed0 con 0x7fe598138d80 2023-09-30T22:31:07.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.735+0000 7fe5a0359700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe5980a4b60 con 0x7fe5980a0a30 2023-09-30T22:31:07.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.735+0000 7fe59e0f5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe59809d630 0x7fe598133690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:07.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.735+0000 7fe59e0f5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe59809d630 0x7fe598133690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45802/0 (socket says 172.21.15.16:45802) 2023-09-30T22:31:07.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.735+0000 7fe59e0f5700 1 -- 172.21.15.16:0/3031765866 learned_addr learned my addr 172.21.15.16:0/3031765866 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:07.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.735+0000 7fe59d8f4700 1 --2- 172.21.15.16:0/3031765866 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe5980a0a30 0x7fe598133bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:07.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.736+0000 7fe59e8f6700 1 --2- 172.21.15.16:0/3031765866 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe598138d80 0x7fe598134110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:07.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.736+0000 7fe59e0f5700 1 -- 172.21.15.16:0/3031765866 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe5980a0a30 msgr2=0x7fe598133bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:07.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.736+0000 7fe59e0f5700 1 --2- 172.21.15.16:0/3031765866 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe5980a0a30 0x7fe598133bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:07.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.736+0000 7fe59e0f5700 1 -- 172.21.15.16:0/3031765866 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe598138d80 msgr2=0x7fe598134110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:07.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.736+0000 7fe59e0f5700 1 --2- 172.21.15.16:0/3031765866 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe598138d80 0x7fe598134110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:07.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.736+0000 7fe59e0f5700 1 -- 172.21.15.16:0/3031765866 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe588009a70 con 0x7fe59809d630 2023-09-30T22:31:07.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.736+0000 7fe59e0f5700 1 --2- 172.21.15.16:0/3031765866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe59809d630 0x7fe598133690 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fe588009f60 tx=0x7fe588009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:07.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.736+0000 7fe596ffd700 1 -- 172.21.15.16:0/3031765866 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58800ce70 con 0x7fe59809d630 2023-09-30T22:31:07.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.736+0000 7fe596ffd700 1 -- 172.21.15.16:0/3031765866 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe58800f040 con 0x7fe59809d630 2023-09-30T22:31:07.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.736+0000 7fe5a0359700 1 -- 172.21.15.16:0/3031765866 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe590009550 con 0x7fe59809d630 2023-09-30T22:31:07.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.736+0000 7fe596ffd700 1 -- 172.21.15.16:0/3031765866 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5880146a0 con 0x7fe59809d630 2023-09-30T22:31:07.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.736+0000 7fe5a0359700 1 -- 172.21.15.16:0/3031765866 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe598136560 con 0x7fe59809d630 2023-09-30T22:31:07.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.737+0000 7fe596ffd700 1 -- 172.21.15.16:0/3031765866 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe588014800 con 0x7fe59809d630 2023-09-30T22:31:07.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.737+0000 7fe5a0359700 1 -- 172.21.15.16:0/3031765866 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe598137a00 con 0x7fe59809d630 2023-09-30T22:31:07.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.738+0000 7fe596ffd700 1 -- 172.21.15.16:0/3031765866 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fe588040510 con 0x7fe59809d630 2023-09-30T22:31:07.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.742+0000 7fe596ffd700 1 -- 172.21.15.16:0/3031765866 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe588007d90 con 0x7fe59809d630 2023-09-30T22:31:07.808 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:07.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.874+0000 7fa4313a7700 1 -- 172.21.15.16:0/1786357890 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa42c06b9e0 con 0x7fa42c131810 2023-09-30T22:31:07.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.874+0000 7fa4137fe700 1 -- 172.21.15.16:0/1786357890 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa42402c030 con 0x7fa42c131810 2023-09-30T22:31:07.875 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:31:07.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.878+0000 7fa4313a7700 1 -- 172.21.15.16:0/1786357890 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa42c131810 msgr2=0x7fa42c11a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:07.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.878+0000 7fa4313a7700 1 --2- 172.21.15.16:0/1786357890 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa42c131810 0x7fa42c11a160 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa424014040 tx=0x7fa424004ad0 comp rx=0 tx=0).stop 2023-09-30T22:31:07.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.878+0000 7fa4313a7700 1 -- 172.21.15.16:0/1786357890 shutdown_connections 2023-09-30T22:31:07.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.878+0000 7fa4313a7700 1 --2- 172.21.15.16:0/1786357890 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa42c131810 0x7fa42c11a160 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:07.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.878+0000 7fa4313a7700 1 --2- 172.21.15.16:0/1786357890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa42c099a40 0x7fa42c1196e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:07.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.878+0000 7fa4313a7700 1 --2- 172.21.15.16:0/1786357890 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa42c09a3b0 0x7fa42c119c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:07.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.878+0000 7fa4313a7700 1 -- 172.21.15.16:0/1786357890 >> 172.21.15.16:0/1786357890 conn(0x7fa42c0a2aa0 msgr2=0x7fa42c12e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:07.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.878+0000 7fa4313a7700 1 -- 172.21.15.16:0/1786357890 shutdown_connections 2023-09-30T22:31:07.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.878+0000 7fa4313a7700 1 -- 172.21.15.16:0/1786357890 wait complete. 2023-09-30T22:31:07.898 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:31:07.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.917+0000 7f6fd89f2700 1 -- 172.21.15.16:0/24296981 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6fd4074ea0 con 0x7f6fd409e000 2023-09-30T22:31:07.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.917+0000 7f6fc2ffd700 1 -- 172.21.15.16:0/24296981 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6fc800cea0 con 0x7f6fd409e000 2023-09-30T22:31:07.918 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:31:07.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.920+0000 7f6fd89f2700 1 -- 172.21.15.16:0/24296981 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6fd409e000 msgr2=0x7f6fd41c8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:07.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.920+0000 7f6fd89f2700 1 --2- 172.21.15.16:0/24296981 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6fd409e000 0x7f6fd41c8480 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6fc800ebe0 tx=0x7f6fc800eef0 comp rx=0 tx=0).stop 2023-09-30T22:31:07.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.920+0000 7f6fd89f2700 1 -- 172.21.15.16:0/24296981 shutdown_connections 2023-09-30T22:31:07.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.920+0000 7f6fd89f2700 1 --2- 172.21.15.16:0/24296981 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6fd409e410 0x7f6fd41c89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:07.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.920+0000 7f6fd89f2700 1 --2- 172.21.15.16:0/24296981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fd409d6f0 0x7f6fd41c7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:07.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.920+0000 7f6fd89f2700 1 --2- 172.21.15.16:0/24296981 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6fd409e000 0x7f6fd41c8480 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:07.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.920+0000 7f6fd89f2700 1 -- 172.21.15.16:0/24296981 >> 172.21.15.16:0/24296981 conn(0x7f6fd4066f50 msgr2=0x7f6fd40a35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:07.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.920+0000 7f6fd89f2700 1 -- 172.21.15.16:0/24296981 shutdown_connections 2023-09-30T22:31:07.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.920+0000 7f6fd89f2700 1 -- 172.21.15.16:0/24296981 wait complete. 2023-09-30T22:31:07.935 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:31:07.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.951+0000 7fe5a0359700 1 -- 172.21.15.16:0/3031765866 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe5980747f0 con 0x7fe59809d630 2023-09-30T22:31:07.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.952+0000 7fe596ffd700 1 -- 172.21.15.16:0/3031765866 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe588007330 con 0x7fe59809d630 2023-09-30T22:31:07.952 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:31:07.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.954+0000 7fe5a0359700 1 -- 172.21.15.16:0/3031765866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe59809d630 msgr2=0x7fe598133690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:07.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.954+0000 7fe5a0359700 1 --2- 172.21.15.16:0/3031765866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe59809d630 0x7fe598133690 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fe588009f60 tx=0x7fe588009f90 comp rx=0 tx=0).stop 2023-09-30T22:31:07.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.954+0000 7fe5a0359700 1 -- 172.21.15.16:0/3031765866 shutdown_connections 2023-09-30T22:31:07.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.954+0000 7fe5a0359700 1 --2- 172.21.15.16:0/3031765866 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe598138d80 0x7fe598134110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:07.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.954+0000 7fe5a0359700 1 --2- 172.21.15.16:0/3031765866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe59809d630 0x7fe598133690 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:07.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.954+0000 7fe5a0359700 1 --2- 172.21.15.16:0/3031765866 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe5980a0a30 0x7fe598133bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:07.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.954+0000 7fe5a0359700 1 -- 172.21.15.16:0/3031765866 >> 172.21.15.16:0/3031765866 conn(0x7fe5980a3050 msgr2=0x7fe59813af90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:07.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.954+0000 7fe5a0359700 1 -- 172.21.15.16:0/3031765866 shutdown_connections 2023-09-30T22:31:07.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.954+0000 7fe5a0359700 1 -- 172.21.15.16:0/3031765866 wait complete. 2023-09-30T22:31:07.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.958+0000 7ff5701d0700 1 Processor -- start 2023-09-30T22:31:07.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.958+0000 7ff5701d0700 1 -- start start 2023-09-30T22:31:07.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.959+0000 7ff5701d0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff56809d6f0 0x7ff56809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:07.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.959+0000 7ff5701d0700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff56809e000 0x7ff5681339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:07.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.959+0000 7ff5701d0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff56809e410 0x7ff568133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:07.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.959+0000 7ff5701d0700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff56806b460 con 0x7ff56809e000 2023-09-30T22:31:07.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.959+0000 7ff5701d0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff56809b120 con 0x7ff56809d6f0 2023-09-30T22:31:07.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.959+0000 7ff5701d0700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff56809a7e0 con 0x7ff56809e410 2023-09-30T22:31:07.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.959+0000 7ff56e76d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff56809e410 0x7ff568133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:07.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.959+0000 7ff56d76b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff56809e000 0x7ff5681339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:07.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.959+0000 7ff56e76d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff56809e410 0x7ff568133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:59654/0 (socket says 172.21.15.16:59654) 2023-09-30T22:31:07.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.959+0000 7ff56e76d700 1 -- 172.21.15.16:0/2541961175 learned_addr learned my addr 172.21.15.16:0/2541961175 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:07.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.959+0000 7ff56df6c700 1 --2- 172.21.15.16:0/2541961175 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff56809d6f0 0x7ff56809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:07.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.959+0000 7ff56d76b700 1 -- 172.21.15.16:0/2541961175 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff56809e410 msgr2=0x7ff568133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:07.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.959+0000 7ff56d76b700 1 --2- 172.21.15.16:0/2541961175 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff56809e410 0x7ff568133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:07.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.959+0000 7ff56d76b700 1 -- 172.21.15.16:0/2541961175 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff56809d6f0 msgr2=0x7ff56809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:07.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.959+0000 7ff56d76b700 1 --2- 172.21.15.16:0/2541961175 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff56809d6f0 0x7ff56809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:07.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.959+0000 7ff56d76b700 1 -- 172.21.15.16:0/2541961175 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff56809a500 con 0x7ff56809e000 2023-09-30T22:31:07.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.959+0000 7ff56df6c700 1 --2- 172.21.15.16:0/2541961175 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff56809d6f0 0x7ff56809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:07.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.959+0000 7ff56e76d700 1 --2- 172.21.15.16:0/2541961175 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff56809e410 0x7ff568133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:07.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.960+0000 7ff56d76b700 1 --2- 172.21.15.16:0/2541961175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff56809e000 0x7ff5681339f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff558009840 tx=0x7ff558009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23925b612d11b4d3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:07.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.960+0000 7ff56cf6a700 1 -- 172.21.15.16:0/2541961175 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff558011830 con 0x7ff56809e000 2023-09-30T22:31:07.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.960+0000 7ff56cf6a700 1 -- 172.21.15.16:0/2541961175 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff558011990 con 0x7ff56809e000 2023-09-30T22:31:07.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.960+0000 7ff56cf6a700 1 -- 172.21.15.16:0/2541961175 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff558011d20 con 0x7ff56809e000 2023-09-30T22:31:07.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.961+0000 7ff5701d0700 1 -- 172.21.15.16:0/2541961175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff56809e000 msgr2=0x7ff5681339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:07.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.961+0000 7ff5701d0700 1 --2- 172.21.15.16:0/2541961175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff56809e000 0x7ff5681339f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff558009840 tx=0x7ff558009b50 comp rx=0 tx=0).stop 2023-09-30T22:31:07.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.961+0000 7ff5701d0700 1 -- 172.21.15.16:0/2541961175 shutdown_connections 2023-09-30T22:31:07.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.961+0000 7ff5701d0700 1 --2- 172.21.15.16:0/2541961175 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff56809d6f0 0x7ff56809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:07.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.961+0000 7ff5701d0700 1 --2- 172.21.15.16:0/2541961175 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff56809e410 0x7ff568133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:07.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.961+0000 7ff5701d0700 1 --2- 172.21.15.16:0/2541961175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff56809e000 0x7ff5681339f0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:07.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.961+0000 7ff5701d0700 1 -- 172.21.15.16:0/2541961175 >> 172.21.15.16:0/2541961175 conn(0x7ff568066f50 msgr2=0x7ff56809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:07.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.961+0000 7ff5701d0700 1 -- 172.21.15.16:0/2541961175 shutdown_connections 2023-09-30T22:31:07.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.961+0000 7ff5701d0700 1 -- 172.21.15.16:0/2541961175 wait complete. 2023-09-30T22:31:07.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.962+0000 7ff5701d0700 1 Processor -- start 2023-09-30T22:31:07.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.962+0000 7ff5701d0700 1 -- start start 2023-09-30T22:31:07.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.963+0000 7ff5701d0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff56809d6f0 0x7ff5681a5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:07.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.963+0000 7ff5701d0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff56809e000 0x7ff5681a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:07.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.963+0000 7ff5701d0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff56809e410 0x7ff5681a67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:07.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.963+0000 7ff5701d0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5681aa240 con 0x7ff56809d6f0 2023-09-30T22:31:07.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.963+0000 7ff5701d0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5681a7970 con 0x7ff56809e410 2023-09-30T22:31:07.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.963+0000 7ff5701d0700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff5681a8bc0 con 0x7ff56809e000 2023-09-30T22:31:07.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.963+0000 7ff56d76b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff56809e000 0x7ff5681a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:07.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.963+0000 7ff56df6c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff56809d6f0 0x7ff5681a5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:07.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.963+0000 7ff56df6c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff56809d6f0 0x7ff5681a5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45816/0 (socket says 172.21.15.16:45816) 2023-09-30T22:31:07.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.963+0000 7ff56d76b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff56809e000 0x7ff5681a62a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:59658/0 (socket says 172.21.15.16:59658) 2023-09-30T22:31:07.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.963+0000 7ff56df6c700 1 -- 172.21.15.16:0/808150228 learned_addr learned my addr 172.21.15.16:0/808150228 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:07.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.963+0000 7ff56e76d700 1 --2- 172.21.15.16:0/808150228 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff56809e410 0x7ff5681a67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:07.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.964+0000 7ff56df6c700 1 -- 172.21.15.16:0/808150228 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff56809e000 msgr2=0x7ff5681a62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:07.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.964+0000 7ff56df6c700 1 --2- 172.21.15.16:0/808150228 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff56809e000 0x7ff5681a62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:07.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.964+0000 7ff56df6c700 1 -- 172.21.15.16:0/808150228 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff56809e410 msgr2=0x7ff5681a67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:07.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.964+0000 7ff56df6c700 1 --2- 172.21.15.16:0/808150228 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff56809e410 0x7ff5681a67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:07.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.964+0000 7ff56df6c700 1 -- 172.21.15.16:0/808150228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5640094a0 con 0x7ff56809d6f0 2023-09-30T22:31:07.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.964+0000 7ff56d76b700 1 --2- 172.21.15.16:0/808150228 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff56809e000 0x7ff5681a62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:07.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.964+0000 7ff56e76d700 1 --2- 172.21.15.16:0/808150228 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff56809e410 0x7ff5681a67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:07.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.964+0000 7ff56df6c700 1 --2- 172.21.15.16:0/808150228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff56809d6f0 0x7ff5681a5d60 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff56400ec10 tx=0x7ff56400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:07.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.964+0000 7ff55e7fc700 1 -- 172.21.15.16:0/808150228 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56400c850 con 0x7ff56809d6f0 2023-09-30T22:31:07.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.964+0000 7ff55e7fc700 1 -- 172.21.15.16:0/808150228 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff56400c9b0 con 0x7ff56809d6f0 2023-09-30T22:31:07.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.964+0000 7ff5701d0700 1 -- 172.21.15.16:0/808150228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff560009b30 con 0x7ff56809d6f0 2023-09-30T22:31:07.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.964+0000 7ff55e7fc700 1 -- 172.21.15.16:0/808150228 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff56400cd20 con 0x7ff56809d6f0 2023-09-30T22:31:07.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.964+0000 7ff5701d0700 1 -- 172.21.15.16:0/808150228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5580094a0 con 0x7ff56809d6f0 2023-09-30T22:31:07.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.965+0000 7ff5701d0700 1 -- 172.21.15.16:0/808150228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5680a1f00 con 0x7ff56809d6f0 2023-09-30T22:31:07.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.965+0000 7ff55e7fc700 1 -- 172.21.15.16:0/808150228 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff564013470 con 0x7ff56809d6f0 2023-09-30T22:31:07.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.965+0000 7ff55e7fc700 1 -- 172.21.15.16:0/808150228 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7ff56403fb30 con 0x7ff56809d6f0 2023-09-30T22:31:07.974 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:31:07.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:07.970+0000 7ff55e7fc700 1 -- 172.21.15.16:0/808150228 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff564013720 con 0x7ff56809d6f0 2023-09-30T22:31:08.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:08.174+0000 7ff5701d0700 1 -- 172.21.15.16:0/808150228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff56809c710 con 0x7ff56809d6f0 2023-09-30T22:31:08.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:08.174+0000 7ff55e7fc700 1 -- 172.21.15.16:0/808150228 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff564011af0 con 0x7ff56809d6f0 2023-09-30T22:31:08.175 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:31:08.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:08.176+0000 7ff5701d0700 1 -- 172.21.15.16:0/808150228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff56809d6f0 msgr2=0x7ff5681a5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:08.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:08.176+0000 7ff5701d0700 1 --2- 172.21.15.16:0/808150228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff56809d6f0 0x7ff5681a5d60 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff56400ec10 tx=0x7ff56400ef20 comp rx=0 tx=0).stop 2023-09-30T22:31:08.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:08.176+0000 7ff5701d0700 1 -- 172.21.15.16:0/808150228 shutdown_connections 2023-09-30T22:31:08.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:08.176+0000 7ff5701d0700 1 --2- 172.21.15.16:0/808150228 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff56809e410 0x7ff5681a67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:08.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:08.176+0000 7ff5701d0700 1 --2- 172.21.15.16:0/808150228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff56809d6f0 0x7ff5681a5d60 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:08.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:08.176+0000 7ff5701d0700 1 --2- 172.21.15.16:0/808150228 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff56809e000 0x7ff5681a62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:08.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:08.176+0000 7ff5701d0700 1 -- 172.21.15.16:0/808150228 >> 172.21.15.16:0/808150228 conn(0x7ff568066f50 msgr2=0x7ff5681bae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:08.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:08.177+0000 7ff5701d0700 1 -- 172.21.15.16:0/808150228 shutdown_connections 2023-09-30T22:31:08.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:08.177+0000 7ff5701d0700 1 -- 172.21.15.16:0/808150228 wait complete. 2023-09-30T22:31:08.191 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:31:08.899 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:08.936 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:08.976 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:09.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.049+0000 7f362b6fd700 1 Processor -- start 2023-09-30T22:31:09.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.050+0000 7f362b6fd700 1 -- start start 2023-09-30T22:31:09.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.050+0000 7f362b6fd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f36240a0a30 0x7f36240a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:09.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.050+0000 7f362b6fd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3624134170 0x7f362413bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:09.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.050+0000 7f362b6fd700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f362409edd0 con 0x7f36240a1340 2023-09-30T22:31:09.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.050+0000 7f362b6fd700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f362409e1f0 con 0x7f36240a0a30 2023-09-30T22:31:09.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.050+0000 7f362b6fd700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f362409dc50 con 0x7f3624134170 2023-09-30T22:31:09.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.050+0000 7f3629499700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f36240a0a30 0x7f36240a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:09.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.050+0000 7f3629499700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f36240a0a30 0x7f36240a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:35070/0 (socket says 172.21.15.16:35070) 2023-09-30T22:31:09.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.050+0000 7f3629499700 1 -- 172.21.15.16:0/4080799495 learned_addr learned my addr 172.21.15.16:0/4080799495 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:09.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.050+0000 7f3629c9a700 1 --2- 172.21.15.16:0/4080799495 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3624134170 0x7f362413bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:09.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.051+0000 7f3629499700 1 -- 172.21.15.16:0/4080799495 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3624134170 msgr2=0x7f362413bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:09.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.051+0000 7f3629499700 1 --2- 172.21.15.16:0/4080799495 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3624134170 0x7f362413bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.051+0000 7f3629499700 1 -- 172.21.15.16:0/4080799495 >> v1:172.21.15.16:6789/0 conn(0x7f36240a1340 legacy=0x7f3624133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:09.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.051+0000 7f3629499700 1 -- 172.21.15.16:0/4080799495 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36240a1fe0 con 0x7f36240a0a30 2023-09-30T22:31:09.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.051+0000 7f3629c9a700 1 --2- 172.21.15.16:0/4080799495 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3624134170 0x7f362413bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:09.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.051+0000 7f3629499700 1 --2- 172.21.15.16:0/4080799495 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f36240a0a30 0x7f36240a0e00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3620009920 tx=0x7f3620009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7580126d80751cec server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:09.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.051+0000 7f361bfff700 1 -- 172.21.15.16:0/4080799495 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3620011830 con 0x7f36240a0a30 2023-09-30T22:31:09.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.051+0000 7f361bfff700 1 -- 172.21.15.16:0/4080799495 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3620011990 con 0x7f36240a0a30 2023-09-30T22:31:09.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.051+0000 7f361bfff700 1 -- 172.21.15.16:0/4080799495 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3620011d20 con 0x7f36240a0a30 2023-09-30T22:31:09.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.052+0000 7f362b6fd700 1 -- 172.21.15.16:0/4080799495 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f36240a0a30 msgr2=0x7f36240a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:09.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.052+0000 7f362b6fd700 1 --2- 172.21.15.16:0/4080799495 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f36240a0a30 0x7f36240a0e00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f3620009920 tx=0x7f3620009c30 comp rx=0 tx=0).stop 2023-09-30T22:31:09.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.052+0000 7f362b6fd700 1 -- 172.21.15.16:0/4080799495 shutdown_connections 2023-09-30T22:31:09.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.052+0000 7f362b6fd700 1 --2- 172.21.15.16:0/4080799495 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3624134170 0x7f362413bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.052+0000 7f362b6fd700 1 --2- 172.21.15.16:0/4080799495 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f36240a0a30 0x7f36240a0e00 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.052+0000 7f362b6fd700 1 -- 172.21.15.16:0/4080799495 >> 172.21.15.16:0/4080799495 conn(0x7f36240a6550 msgr2=0x7f36240a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:09.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.052+0000 7f362b6fd700 1 -- 172.21.15.16:0/4080799495 shutdown_connections 2023-09-30T22:31:09.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.052+0000 7f362b6fd700 1 -- 172.21.15.16:0/4080799495 wait complete. 2023-09-30T22:31:09.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.053+0000 7f362b6fd700 1 Processor -- start 2023-09-30T22:31:09.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.053+0000 7f362b6fd700 1 -- start start 2023-09-30T22:31:09.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.054+0000 7f362b6fd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f36240a0a30 0x7f36241c8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:09.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.054+0000 7f362b6fd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f36240a1340 0x7f36241c8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:09.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.054+0000 7f362b6fd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3624134170 0x7f36241c8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:09.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.054+0000 7f362b6fd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3624124790 con 0x7f3624134170 2023-09-30T22:31:09.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.054+0000 7f362b6fd700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36240a0220 con 0x7f36240a0a30 2023-09-30T22:31:09.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.054+0000 7f362b6fd700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3624124560 con 0x7f36240a1340 2023-09-30T22:31:09.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.054+0000 7f3628c98700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f36240a1340 0x7f36241c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:09.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.054+0000 7f3628c98700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f36240a1340 0x7f36241c8990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:59686/0 (socket says 172.21.15.16:59686) 2023-09-30T22:31:09.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.054+0000 7f3628c98700 1 -- 172.21.15.16:0/2690917950 learned_addr learned my addr 172.21.15.16:0/2690917950 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:09.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.054+0000 7f3629c9a700 1 --2- 172.21.15.16:0/2690917950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3624134170 0x7f36241c8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:09.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.054+0000 7f3629499700 1 --2- 172.21.15.16:0/2690917950 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f36240a0a30 0x7f36241c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:09.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.054+0000 7f3629499700 1 -- 172.21.15.16:0/2690917950 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f36240a1340 msgr2=0x7f36241c8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:09.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.054+0000 7f3629499700 1 --2- 172.21.15.16:0/2690917950 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f36240a1340 0x7f36241c8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.054+0000 7f3629499700 1 -- 172.21.15.16:0/2690917950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3624134170 msgr2=0x7f36241c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:09.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.054+0000 7f3629499700 1 --2- 172.21.15.16:0/2690917950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3624134170 0x7f36241c8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.054+0000 7f3629499700 1 -- 172.21.15.16:0/2690917950 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f361c009a70 con 0x7f36240a0a30 2023-09-30T22:31:09.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.054+0000 7f3628c98700 1 --2- 172.21.15.16:0/2690917950 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f36240a1340 0x7f36241c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:09.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.055+0000 7f3629499700 1 --2- 172.21.15.16:0/2690917950 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f36240a0a30 0x7f36241c8450 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f362406c3d0 tx=0x7f3620004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:09.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.055+0000 7f3619ffb700 1 -- 172.21.15.16:0/2690917950 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3620004450 con 0x7f36240a0a30 2023-09-30T22:31:09.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.055+0000 7f3619ffb700 1 -- 172.21.15.16:0/2690917950 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f362001f070 con 0x7f36240a0a30 2023-09-30T22:31:09.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.055+0000 7f362b6fd700 1 -- 172.21.15.16:0/2690917950 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3620009550 con 0x7f36240a0a30 2023-09-30T22:31:09.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.055+0000 7f3619ffb700 1 -- 172.21.15.16:0/2690917950 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36200038c0 con 0x7f36240a0a30 2023-09-30T22:31:09.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.055+0000 7f362b6fd700 1 -- 172.21.15.16:0/2690917950 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36240a25e0 con 0x7f36240a0a30 2023-09-30T22:31:09.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.056+0000 7f3619ffb700 1 -- 172.21.15.16:0/2690917950 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3620003a20 con 0x7f36240a0a30 2023-09-30T22:31:09.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.056+0000 7f3619ffb700 1 -- 172.21.15.16:0/2690917950 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f3620021070 con 0x7f36240a0a30 2023-09-30T22:31:09.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.056+0000 7f362b6fd700 1 -- 172.21.15.16:0/2690917950 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3624074ea0 con 0x7f36240a0a30 2023-09-30T22:31:09.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.061+0000 7f3619ffb700 1 -- 172.21.15.16:0/2690917950 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3620015080 con 0x7f36240a0a30 2023-09-30T22:31:09.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.090+0000 7f2509008700 1 Processor -- start 2023-09-30T22:31:09.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.090+0000 7f2509008700 1 -- start start 2023-09-30T22:31:09.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.091+0000 7f2509008700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f250409e1d0 0x7f2504135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:09.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.091+0000 7f2509008700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2504135e50 0x7f2504138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:09.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.091+0000 7f2509008700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f250409f130 con 0x7f250409d6f0 2023-09-30T22:31:09.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.091+0000 7f2509008700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f250409f520 con 0x7f250409e1d0 2023-09-30T22:31:09.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.091+0000 7f2509008700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f250409e930 con 0x7f2504135e50 2023-09-30T22:31:09.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.091+0000 7f2502d9d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f250409d6f0 0x7f250409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47368/0 (socket says 172.21.15.16:47368) 2023-09-30T22:31:09.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.091+0000 7f2502d9d700 1 -- 172.21.15.16:0/4185590139 learned_addr learned my addr 172.21.15.16:0/4185590139 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:31:09.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.091+0000 7f250259c700 1 --2- 172.21.15.16:0/4185590139 >> v2:172.21.15.16:3300/0 conn(0x7f250409e1d0 0x7f2504135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:09.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.092+0000 7f250359e700 1 --2- 172.21.15.16:0/4185590139 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2504135e50 0x7f2504138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:09.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.092+0000 7f2501d9b700 1 -- 172.21.15.16:0/4185590139 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2080674673 0 0) 0x7f250409f130 con 0x7f250409d6f0 2023-09-30T22:31:09.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.092+0000 7f250259c700 1 -- 172.21.15.16:0/4185590139 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2504135e50 msgr2=0x7f2504138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:09.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.092+0000 7f250259c700 1 --2- 172.21.15.16:0/4185590139 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2504135e50 0x7f2504138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.092+0000 7f250259c700 1 -- 172.21.15.16:0/4185590139 >> v1:172.21.15.16:6789/0 conn(0x7f250409d6f0 legacy=0x7f250409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:09.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.092+0000 7f250259c700 1 -- 172.21.15.16:0/4185590139 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f250409b2d0 con 0x7f250409e1d0 2023-09-30T22:31:09.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.092+0000 7f250259c700 1 --2- 172.21.15.16:0/4185590139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f250409e1d0 0x7f2504135910 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f24f8009840 tx=0x7f24f8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2ede5b7d5550873 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:09.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.092+0000 7f2501d9b700 1 -- 172.21.15.16:0/4185590139 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f8011830 con 0x7f250409e1d0 2023-09-30T22:31:09.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.092+0000 7f2501d9b700 1 -- 172.21.15.16:0/4185590139 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24f8011990 con 0x7f250409e1d0 2023-09-30T22:31:09.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.092+0000 7f2501d9b700 1 -- 172.21.15.16:0/4185590139 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f8011d20 con 0x7f250409e1d0 2023-09-30T22:31:09.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.093+0000 7f2509008700 1 -- 172.21.15.16:0/4185590139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f250409e1d0 msgr2=0x7f2504135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:09.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.093+0000 7f2509008700 1 --2- 172.21.15.16:0/4185590139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f250409e1d0 0x7f2504135910 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f24f8009840 tx=0x7f24f8009b50 comp rx=0 tx=0).stop 2023-09-30T22:31:09.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.093+0000 7f2509008700 1 -- 172.21.15.16:0/4185590139 shutdown_connections 2023-09-30T22:31:09.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.093+0000 7f2509008700 1 --2- 172.21.15.16:0/4185590139 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2504135e50 0x7f2504138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.093+0000 7f2509008700 1 --2- 172.21.15.16:0/4185590139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f250409e1d0 0x7f2504135910 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.093+0000 7f2509008700 1 -- 172.21.15.16:0/4185590139 >> 172.21.15.16:0/4185590139 conn(0x7f2504066f50 msgr2=0x7f250409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:09.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.093+0000 7f2509008700 1 -- 172.21.15.16:0/4185590139 shutdown_connections 2023-09-30T22:31:09.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.093+0000 7f2509008700 1 -- 172.21.15.16:0/4185590139 wait complete. 2023-09-30T22:31:09.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.095+0000 7f2509008700 1 Processor -- start 2023-09-30T22:31:09.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.095+0000 7f2509008700 1 -- start start 2023-09-30T22:31:09.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.096+0000 7f2509008700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f250409d6f0 0x7f25041c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:09.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.096+0000 7f2509008700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f250409e1d0 0x7f25041c8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:09.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.096+0000 7f2509008700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2504135e50 0x7f25041c8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:09.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.096+0000 7f2509008700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2504131270 con 0x7f250409d6f0 2023-09-30T22:31:09.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.096+0000 7f2509008700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f250411b540 con 0x7f2504135e50 2023-09-30T22:31:09.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.096+0000 7f2509008700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2504121da0 con 0x7f250409e1d0 2023-09-30T22:31:09.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.096+0000 7f250259c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f250409e1d0 0x7f25041c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:09.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.096+0000 7f250259c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f250409e1d0 0x7f25041c8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:59712/0 (socket says 172.21.15.16:59712) 2023-09-30T22:31:09.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.096+0000 7f250259c700 1 -- 172.21.15.16:0/1654360350 learned_addr learned my addr 172.21.15.16:0/1654360350 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:09.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.096+0000 7f2502d9d700 1 --2- 172.21.15.16:0/1654360350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f250409d6f0 0x7f25041c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:09.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.096+0000 7f250359e700 1 --2- 172.21.15.16:0/1654360350 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2504135e50 0x7f25041c8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:09.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.096+0000 7f250259c700 1 -- 172.21.15.16:0/1654360350 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2504135e50 msgr2=0x7f25041c8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:09.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.096+0000 7f250259c700 1 --2- 172.21.15.16:0/1654360350 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2504135e50 0x7f25041c8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.096+0000 7f250259c700 1 -- 172.21.15.16:0/1654360350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f250409d6f0 msgr2=0x7f25041c82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:09.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.096+0000 7f250259c700 1 --2- 172.21.15.16:0/1654360350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f250409d6f0 0x7f25041c82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.096+0000 7f250259c700 1 -- 172.21.15.16:0/1654360350 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f80094a0 con 0x7f250409e1d0 2023-09-30T22:31:09.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.096+0000 7f2502d9d700 1 --2- 172.21.15.16:0/1654360350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f250409d6f0 0x7f25041c82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:09.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.097+0000 7f250259c700 1 --2- 172.21.15.16:0/1654360350 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f250409e1d0 0x7f25041c8830 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f24f80090b0 tx=0x7f24f8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:09.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.097+0000 7f24eb7fe700 1 -- 172.21.15.16:0/1654360350 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f8004550 con 0x7f250409e1d0 2023-09-30T22:31:09.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.097+0000 7f24eb7fe700 1 -- 172.21.15.16:0/1654360350 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24f801f070 con 0x7f250409e1d0 2023-09-30T22:31:09.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.097+0000 7f24eb7fe700 1 -- 172.21.15.16:0/1654360350 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f80038e0 con 0x7f250409e1d0 2023-09-30T22:31:09.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.097+0000 7f2509008700 1 -- 172.21.15.16:0/1654360350 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f250411c7a0 con 0x7f250409e1d0 2023-09-30T22:31:09.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.097+0000 7f2509008700 1 -- 172.21.15.16:0/1654360350 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f250406a9e0 con 0x7f250409e1d0 2023-09-30T22:31:09.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.098+0000 7f2509008700 1 -- 172.21.15.16:0/1654360350 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2504122120 con 0x7f250409e1d0 2023-09-30T22:31:09.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.098+0000 7f24eb7fe700 1 -- 172.21.15.16:0/1654360350 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f24f8003a40 con 0x7f250409e1d0 2023-09-30T22:31:09.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.098+0000 7f24eb7fe700 1 -- 172.21.15.16:0/1654360350 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f24f8021070 con 0x7f250409e1d0 2023-09-30T22:31:09.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.103+0000 7f24eb7fe700 1 -- 172.21.15.16:0/1654360350 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f24f801a430 con 0x7f250409e1d0 2023-09-30T22:31:09.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.130+0000 7fd5aeae8700 1 Processor -- start 2023-09-30T22:31:09.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.130+0000 7fd5aeae8700 1 -- start start 2023-09-30T22:31:09.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.131+0000 7fd5aeae8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd5a809d6f0 0x7fd5a809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:09.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.131+0000 7fd5aeae8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd5a809e000 0x7fd5a8130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:09.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.131+0000 7fd5aeae8700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd5a809e410 0x7fd5a8130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:09.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.131+0000 7fd5aeae8700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5a809f520 con 0x7fd5a809e410 2023-09-30T22:31:09.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.131+0000 7fd5aeae8700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a809f130 con 0x7fd5a809e000 2023-09-30T22:31:09.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.131+0000 7fd5aeae8700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a809e930 con 0x7fd5a809d6f0 2023-09-30T22:31:09.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.131+0000 7fd5ac884700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd5a809d6f0 0x7fd5a809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:09.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.131+0000 7fd5a7fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd5a809e000 0x7fd5a8130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:09.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.131+0000 7fd5ac884700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd5a809d6f0 0x7fd5a809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:59724/0 (socket says 172.21.15.16:59724) 2023-09-30T22:31:09.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.131+0000 7fd5ac884700 1 -- 172.21.15.16:0/2269545088 learned_addr learned my addr 172.21.15.16:0/2269545088 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:09.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.131+0000 7fd5ad085700 1 --2- 172.21.15.16:0/2269545088 >> v2:172.21.15.16:3300/0 conn(0x7fd5a809e410 0x7fd5a8130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:09.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.131+0000 7fd5ac884700 1 -- 172.21.15.16:0/2269545088 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd5a809e000 msgr2=0x7fd5a8130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:09.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.131+0000 7fd5ac884700 1 --2- 172.21.15.16:0/2269545088 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd5a809e000 0x7fd5a8130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.131+0000 7fd5ac884700 1 -- 172.21.15.16:0/2269545088 >> v2:172.21.15.16:3300/0 conn(0x7fd5a809e410 msgr2=0x7fd5a8130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:09.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.131+0000 7fd5ac884700 1 --2- 172.21.15.16:0/2269545088 >> v2:172.21.15.16:3300/0 conn(0x7fd5a809e410 0x7fd5a8130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.131+0000 7fd5ac884700 1 -- 172.21.15.16:0/2269545088 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a809b2d0 con 0x7fd5a809d6f0 2023-09-30T22:31:09.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.131+0000 7fd5ad085700 1 --2- 172.21.15.16:0/2269545088 >> v2:172.21.15.16:3300/0 conn(0x7fd5a809e410 0x7fd5a8130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:09.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.131+0000 7fd5ac884700 1 --2- 172.21.15.16:0/2269545088 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd5a809d6f0 0x7fd5a809dac0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd59c00bbc0 tx=0x7fd59c00bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c81a200eb5f80d6f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:09.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.132+0000 7fd5a77fe700 1 -- 172.21.15.16:0/2269545088 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd59c00dcf0 con 0x7fd5a809d6f0 2023-09-30T22:31:09.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.132+0000 7fd5a77fe700 1 -- 172.21.15.16:0/2269545088 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd59c00de50 con 0x7fd5a809d6f0 2023-09-30T22:31:09.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.132+0000 7fd5a77fe700 1 -- 172.21.15.16:0/2269545088 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd59c010690 con 0x7fd5a809d6f0 2023-09-30T22:31:09.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.132+0000 7fd5aeae8700 1 -- 172.21.15.16:0/2269545088 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd5a809d6f0 msgr2=0x7fd5a809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:09.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.132+0000 7fd5aeae8700 1 --2- 172.21.15.16:0/2269545088 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd5a809d6f0 0x7fd5a809dac0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd59c00bbc0 tx=0x7fd59c00bed0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.132+0000 7fd5aeae8700 1 -- 172.21.15.16:0/2269545088 shutdown_connections 2023-09-30T22:31:09.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.132+0000 7fd5aeae8700 1 --2- 172.21.15.16:0/2269545088 >> v2:172.21.15.16:3300/0 conn(0x7fd5a809e410 0x7fd5a8130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.132+0000 7fd5aeae8700 1 --2- 172.21.15.16:0/2269545088 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd5a809d6f0 0x7fd5a809dac0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.132+0000 7fd5aeae8700 1 --2- 172.21.15.16:0/2269545088 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd5a809e000 0x7fd5a8130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.132+0000 7fd5aeae8700 1 -- 172.21.15.16:0/2269545088 >> 172.21.15.16:0/2269545088 conn(0x7fd5a8066f50 msgr2=0x7fd5a809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:09.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.133+0000 7fd5aeae8700 1 -- 172.21.15.16:0/2269545088 shutdown_connections 2023-09-30T22:31:09.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.133+0000 7fd5aeae8700 1 -- 172.21.15.16:0/2269545088 wait complete. 2023-09-30T22:31:09.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.134+0000 7fd5aeae8700 1 Processor -- start 2023-09-30T22:31:09.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.134+0000 7fd5aeae8700 1 -- start start 2023-09-30T22:31:09.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.135+0000 7fd5aeae8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5a809d6f0 0x7fd5a811e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:09.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.135+0000 7fd5aeae8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd5a809e000 0x7fd5a811e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:09.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.135+0000 7fd5aeae8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd5a809e410 0x7fd5a811eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:09.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.135+0000 7fd5aeae8700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a809c700 con 0x7fd5a809d6f0 2023-09-30T22:31:09.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.135+0000 7fd5aeae8700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a81202d0 con 0x7fd5a809e000 2023-09-30T22:31:09.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.135+0000 7fd5aeae8700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd5a80a1a80 con 0x7fd5a809e410 2023-09-30T22:31:09.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.135+0000 7fd5ac884700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5a809d6f0 0x7fd5a811e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:09.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.135+0000 7fd5ac884700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5a809d6f0 0x7fd5a811e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45868/0 (socket says 172.21.15.16:45868) 2023-09-30T22:31:09.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.135+0000 7fd5ac884700 1 -- 172.21.15.16:0/4028508660 learned_addr learned my addr 172.21.15.16:0/4028508660 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:09.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.135+0000 7fd5ad085700 1 --2- 172.21.15.16:0/4028508660 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd5a809e410 0x7fd5a811eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:09.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.135+0000 7fd5a7fff700 1 --2- 172.21.15.16:0/4028508660 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd5a809e000 0x7fd5a811e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:09.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.135+0000 7fd5ad085700 1 -- 172.21.15.16:0/4028508660 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd5a809e000 msgr2=0x7fd5a811e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:09.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.135+0000 7fd5ad085700 1 --2- 172.21.15.16:0/4028508660 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd5a809e000 0x7fd5a811e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.135+0000 7fd5ad085700 1 -- 172.21.15.16:0/4028508660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5a809d6f0 msgr2=0x7fd5a811e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:09.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.135+0000 7fd5ad085700 1 --2- 172.21.15.16:0/4028508660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5a809d6f0 0x7fd5a811e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.135+0000 7fd5ad085700 1 -- 172.21.15.16:0/4028508660 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a0009a70 con 0x7fd5a809e410 2023-09-30T22:31:09.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.135+0000 7fd5a7fff700 1 --2- 172.21.15.16:0/4028508660 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd5a809e000 0x7fd5a811e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:09.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.136+0000 7fd5ad085700 1 --2- 172.21.15.16:0/4028508660 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd5a809e410 0x7fd5a811eb90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd5a000efc0 tx=0x7fd5a000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:09.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.136+0000 7fd5a57fa700 1 -- 172.21.15.16:0/4028508660 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a000ca90 con 0x7fd5a809e410 2023-09-30T22:31:09.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.136+0000 7fd5a57fa700 1 -- 172.21.15.16:0/4028508660 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5a000f040 con 0x7fd5a809e410 2023-09-30T22:31:09.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.136+0000 7fd5a57fa700 1 -- 172.21.15.16:0/4028508660 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a000ce30 con 0x7fd5a809e410 2023-09-30T22:31:09.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.136+0000 7fd5aeae8700 1 -- 172.21.15.16:0/4028508660 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd59c00b7f0 con 0x7fd5a809e410 2023-09-30T22:31:09.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.136+0000 7fd5aeae8700 1 -- 172.21.15.16:0/4028508660 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5a8121950 con 0x7fd5a809e410 2023-09-30T22:31:09.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.137+0000 7fd5a57fa700 1 -- 172.21.15.16:0/4028508660 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd5a0014480 con 0x7fd5a809e410 2023-09-30T22:31:09.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.137+0000 7fd5a57fa700 1 -- 172.21.15.16:0/4028508660 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd5a003fc80 con 0x7fd5a809e410 2023-09-30T22:31:09.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.137+0000 7fd5aeae8700 1 -- 172.21.15.16:0/4028508660 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5a81215a0 con 0x7fd5a809e410 2023-09-30T22:31:09.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.147+0000 7fd5a57fa700 1 -- 172.21.15.16:0/4028508660 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd5a0012930 con 0x7fd5a809e410 2023-09-30T22:31:09.191 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:09.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.284+0000 7f362b6fd700 1 -- 172.21.15.16:0/2690917950 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f36240a28e0 con 0x7f36240a0a30 2023-09-30T22:31:09.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.284+0000 7f3619ffb700 1 -- 172.21.15.16:0/2690917950 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f362001a430 con 0x7f36240a0a30 2023-09-30T22:31:09.285 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:31:09.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.287+0000 7f362b6fd700 1 -- 172.21.15.16:0/2690917950 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f36240a0a30 msgr2=0x7f36241c8450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:09.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.287+0000 7f362b6fd700 1 --2- 172.21.15.16:0/2690917950 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f36240a0a30 0x7f36241c8450 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f362406c3d0 tx=0x7f3620004750 comp rx=0 tx=0).stop 2023-09-30T22:31:09.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.287+0000 7f362b6fd700 1 -- 172.21.15.16:0/2690917950 shutdown_connections 2023-09-30T22:31:09.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.287+0000 7f362b6fd700 1 --2- 172.21.15.16:0/2690917950 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f36240a0a30 0x7f36241c8450 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.287+0000 7f362b6fd700 1 --2- 172.21.15.16:0/2690917950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3624134170 0x7f36241c8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.287+0000 7f362b6fd700 1 --2- 172.21.15.16:0/2690917950 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f36240a1340 0x7f36241c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.287+0000 7f362b6fd700 1 -- 172.21.15.16:0/2690917950 >> 172.21.15.16:0/2690917950 conn(0x7f36240a6550 msgr2=0x7f362409ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:09.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.287+0000 7f362b6fd700 1 -- 172.21.15.16:0/2690917950 shutdown_connections 2023-09-30T22:31:09.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.287+0000 7f362b6fd700 1 -- 172.21.15.16:0/2690917950 wait complete. 2023-09-30T22:31:09.302 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:31:09.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.325+0000 7f2509008700 1 -- 172.21.15.16:0/1654360350 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2504074ea0 con 0x7f250409e1d0 2023-09-30T22:31:09.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.325+0000 7f24eb7fe700 1 -- 172.21.15.16:0/1654360350 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f24f8015030 con 0x7f250409e1d0 2023-09-30T22:31:09.325 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:31:09.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.328+0000 7f2509008700 1 -- 172.21.15.16:0/1654360350 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f250409e1d0 msgr2=0x7f25041c8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:09.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.328+0000 7f2509008700 1 --2- 172.21.15.16:0/1654360350 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f250409e1d0 0x7f25041c8830 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f24f80090b0 tx=0x7f24f8004750 comp rx=0 tx=0).stop 2023-09-30T22:31:09.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.328+0000 7f2509008700 1 -- 172.21.15.16:0/1654360350 shutdown_connections 2023-09-30T22:31:09.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.328+0000 7f2509008700 1 --2- 172.21.15.16:0/1654360350 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2504135e50 0x7f25041c8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.328+0000 7f2509008700 1 --2- 172.21.15.16:0/1654360350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f250409d6f0 0x7f25041c82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.328+0000 7f2509008700 1 --2- 172.21.15.16:0/1654360350 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f250409e1d0 0x7f25041c8830 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.328+0000 7f2509008700 1 -- 172.21.15.16:0/1654360350 >> 172.21.15.16:0/1654360350 conn(0x7f2504066f50 msgr2=0x7f2504133790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:09.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.328+0000 7f2509008700 1 -- 172.21.15.16:0/1654360350 shutdown_connections 2023-09-30T22:31:09.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.328+0000 7f2509008700 1 -- 172.21.15.16:0/1654360350 wait complete. 2023-09-30T22:31:09.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.336+0000 7fea95fa9700 1 Processor -- start 2023-09-30T22:31:09.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.336+0000 7fea95fa9700 1 -- start start 2023-09-30T22:31:09.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.337+0000 7fea95fa9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea9009e1d0 0x7fea90138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:09.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.337+0000 7fea95fa9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea90139170 0x7fea9013b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:09.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.337+0000 7fea95fa9700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea9009b120 con 0x7fea9009d6f0 2023-09-30T22:31:09.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.337+0000 7fea95fa9700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea9006b460 con 0x7fea90139170 2023-09-30T22:31:09.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.337+0000 7fea95fa9700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fea9009a7e0 con 0x7fea9009e1d0 2023-09-30T22:31:09.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.337+0000 7fea8effd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea9009e1d0 0x7fea90138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:09.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.337+0000 7fea8effd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea9009e1d0 0x7fea90138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:59744/0 (socket says 172.21.15.16:59744) 2023-09-30T22:31:09.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.337+0000 7fea8effd700 1 -- 172.21.15.16:0/1155286032 learned_addr learned my addr 172.21.15.16:0/1155286032 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:09.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.338+0000 7fea8ffff700 1 --2- 172.21.15.16:0/1155286032 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea90139170 0x7fea9013b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:09.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.338+0000 7fea8e7fc700 1 -- 172.21.15.16:0/1155286032 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2780551889 0 0) 0x7fea9009b120 con 0x7fea9009d6f0 2023-09-30T22:31:09.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.338+0000 7fea8effd700 1 -- 172.21.15.16:0/1155286032 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea90139170 msgr2=0x7fea9013b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:09.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.338+0000 7fea8effd700 1 --2- 172.21.15.16:0/1155286032 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea90139170 0x7fea9013b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.338+0000 7fea8effd700 1 -- 172.21.15.16:0/1155286032 >> v1:172.21.15.16:6789/0 conn(0x7fea9009d6f0 legacy=0x7fea9009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:09.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.338+0000 7fea8effd700 1 -- 172.21.15.16:0/1155286032 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea9009a500 con 0x7fea9009e1d0 2023-09-30T22:31:09.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.338+0000 7fea8effd700 1 --2- 172.21.15.16:0/1155286032 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea9009e1d0 0x7fea90138c30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fea84009950 tx=0x7fea84009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15d99cec2c34b9ee server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:09.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.338+0000 7fea8e7fc700 1 -- 172.21.15.16:0/1155286032 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea84011830 con 0x7fea9009e1d0 2023-09-30T22:31:09.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.338+0000 7fea8e7fc700 1 -- 172.21.15.16:0/1155286032 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea84011990 con 0x7fea9009e1d0 2023-09-30T22:31:09.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.338+0000 7fea8e7fc700 1 -- 172.21.15.16:0/1155286032 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea84011d40 con 0x7fea9009e1d0 2023-09-30T22:31:09.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.339+0000 7fea95fa9700 1 -- 172.21.15.16:0/1155286032 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea9009e1d0 msgr2=0x7fea90138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:09.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.339+0000 7fea95fa9700 1 --2- 172.21.15.16:0/1155286032 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea9009e1d0 0x7fea90138c30 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fea84009950 tx=0x7fea84009c60 comp rx=0 tx=0).stop 2023-09-30T22:31:09.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.339+0000 7fea95fa9700 1 -- 172.21.15.16:0/1155286032 shutdown_connections 2023-09-30T22:31:09.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.339+0000 7fea95fa9700 1 --2- 172.21.15.16:0/1155286032 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea90139170 0x7fea9013b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.339+0000 7fea95fa9700 1 --2- 172.21.15.16:0/1155286032 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea9009e1d0 0x7fea90138c30 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.339+0000 7fea95fa9700 1 -- 172.21.15.16:0/1155286032 >> 172.21.15.16:0/1155286032 conn(0x7fea90066f50 msgr2=0x7fea9009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:09.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.339+0000 7fea95fa9700 1 -- 172.21.15.16:0/1155286032 shutdown_connections 2023-09-30T22:31:09.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.339+0000 7fea95fa9700 1 -- 172.21.15.16:0/1155286032 wait complete. 2023-09-30T22:31:09.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.340+0000 7fea95fa9700 1 Processor -- start 2023-09-30T22:31:09.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.341+0000 7fea95fa9700 1 -- start start 2023-09-30T22:31:09.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.341+0000 7fea95fa9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea9009d6f0 0x7fea9011e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:09.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.341+0000 7fea95fa9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea9009e1d0 0x7fea9011ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:09.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.341+0000 7fea95fa9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea90139170 0x7fea9011f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:09.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.341+0000 7fea95fa9700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea90134590 con 0x7fea90139170 2023-09-30T22:31:09.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.341+0000 7fea95fa9700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea901221c0 con 0x7fea9009e1d0 2023-09-30T22:31:09.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.341+0000 7fea95fa9700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fea90120ad0 con 0x7fea9009d6f0 2023-09-30T22:31:09.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.341+0000 7fea8effd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea9009e1d0 0x7fea9011ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:09.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.341+0000 7fea8f7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea9009d6f0 0x7fea9011e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:09.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.341+0000 7fea8effd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea9009e1d0 0x7fea9011ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:35118/0 (socket says 172.21.15.16:35118) 2023-09-30T22:31:09.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.341+0000 7fea8effd700 1 -- 172.21.15.16:0/220550086 learned_addr learned my addr 172.21.15.16:0/220550086 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:09.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.341+0000 7fea8ffff700 1 --2- 172.21.15.16:0/220550086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea90139170 0x7fea9011f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:09.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.342+0000 7fea8f7fe700 1 -- 172.21.15.16:0/220550086 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea9009e1d0 msgr2=0x7fea9011ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:09.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.342+0000 7fea8f7fe700 1 --2- 172.21.15.16:0/220550086 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea9009e1d0 0x7fea9011ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.342+0000 7fea8f7fe700 1 -- 172.21.15.16:0/220550086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea90139170 msgr2=0x7fea9011f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:09.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.342+0000 7fea8f7fe700 1 --2- 172.21.15.16:0/220550086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea90139170 0x7fea9011f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.342+0000 7fea8f7fe700 1 -- 172.21.15.16:0/220550086 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea84009550 con 0x7fea9009d6f0 2023-09-30T22:31:09.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.342+0000 7fea8ffff700 1 --2- 172.21.15.16:0/220550086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea90139170 0x7fea9011f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:09.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.342+0000 7fea8f7fe700 1 --2- 172.21.15.16:0/220550086 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea9009d6f0 0x7fea9011e6a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fea8000c430 tx=0x7fea8000c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:09.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.342+0000 7fea77fff700 1 -- 172.21.15.16:0/220550086 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea800054e0 con 0x7fea9009d6f0 2023-09-30T22:31:09.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.342+0000 7fea77fff700 1 -- 172.21.15.16:0/220550086 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea8000ce40 con 0x7fea9009d6f0 2023-09-30T22:31:09.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.342+0000 7fea77fff700 1 -- 172.21.15.16:0/220550086 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea800038c0 con 0x7fea9009d6f0 2023-09-30T22:31:09.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.342+0000 7fea95fa9700 1 -- 172.21.15.16:0/220550086 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea900a2120 con 0x7fea9009d6f0 2023-09-30T22:31:09.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.342+0000 7fea95fa9700 1 -- 172.21.15.16:0/220550086 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea90121ee0 con 0x7fea9009d6f0 2023-09-30T22:31:09.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.343+0000 7fea77fff700 1 -- 172.21.15.16:0/220550086 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fea80003a20 con 0x7fea9009d6f0 2023-09-30T22:31:09.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.343+0000 7fea77fff700 1 -- 172.21.15.16:0/220550086 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fea8003e9c0 con 0x7fea9009d6f0 2023-09-30T22:31:09.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.343+0000 7fea95fa9700 1 -- 172.21.15.16:0/220550086 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea90074ea0 con 0x7fea9009d6f0 2023-09-30T22:31:09.346 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:31:09.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.348+0000 7fea77fff700 1 -- 172.21.15.16:0/220550086 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fea80017070 con 0x7fea9009d6f0 2023-09-30T22:31:09.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.386+0000 7fd5aeae8700 1 -- 172.21.15.16:0/4028508660 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd5a8074ea0 con 0x7fd5a809e410 2023-09-30T22:31:09.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.386+0000 7fd5a57fa700 1 -- 172.21.15.16:0/4028508660 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd5a0012340 con 0x7fd5a809e410 2023-09-30T22:31:09.387 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:31:09.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.389+0000 7fd5aeae8700 1 -- 172.21.15.16:0/4028508660 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd5a809e410 msgr2=0x7fd5a811eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:09.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.389+0000 7fd5aeae8700 1 --2- 172.21.15.16:0/4028508660 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd5a809e410 0x7fd5a811eb90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd5a000efc0 tx=0x7fd5a000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.389+0000 7fd5aeae8700 1 -- 172.21.15.16:0/4028508660 shutdown_connections 2023-09-30T22:31:09.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.389+0000 7fd5aeae8700 1 --2- 172.21.15.16:0/4028508660 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd5a809e410 0x7fd5a811eb90 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.389+0000 7fd5aeae8700 1 --2- 172.21.15.16:0/4028508660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5a809d6f0 0x7fd5a811e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.389+0000 7fd5aeae8700 1 --2- 172.21.15.16:0/4028508660 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd5a809e000 0x7fd5a811e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.389+0000 7fd5aeae8700 1 -- 172.21.15.16:0/4028508660 >> 172.21.15.16:0/4028508660 conn(0x7fd5a8066f50 msgr2=0x7fd5a806a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:09.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.389+0000 7fd5aeae8700 1 -- 172.21.15.16:0/4028508660 shutdown_connections 2023-09-30T22:31:09.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.389+0000 7fd5aeae8700 1 -- 172.21.15.16:0/4028508660 wait complete. 2023-09-30T22:31:09.403 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:31:09.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.573+0000 7fea95fa9700 1 -- 172.21.15.16:0/220550086 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fea9006c360 con 0x7fea9009d6f0 2023-09-30T22:31:09.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.573+0000 7fea77fff700 1 -- 172.21.15.16:0/220550086 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fea80003cd0 con 0x7fea9009d6f0 2023-09-30T22:31:09.574 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:31:09.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.576+0000 7fea95fa9700 1 -- 172.21.15.16:0/220550086 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea9009d6f0 msgr2=0x7fea9011e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:09.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.576+0000 7fea95fa9700 1 --2- 172.21.15.16:0/220550086 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea9009d6f0 0x7fea9011e6a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fea8000c430 tx=0x7fea8000c740 comp rx=0 tx=0).stop 2023-09-30T22:31:09.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.576+0000 7fea95fa9700 1 -- 172.21.15.16:0/220550086 shutdown_connections 2023-09-30T22:31:09.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.576+0000 7fea95fa9700 1 --2- 172.21.15.16:0/220550086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea90139170 0x7fea9011f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.576+0000 7fea95fa9700 1 --2- 172.21.15.16:0/220550086 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea9009d6f0 0x7fea9011e6a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.576+0000 7fea95fa9700 1 --2- 172.21.15.16:0/220550086 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea9009e1d0 0x7fea9011ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:09.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.576+0000 7fea95fa9700 1 -- 172.21.15.16:0/220550086 >> 172.21.15.16:0/220550086 conn(0x7fea90066f50 msgr2=0x7fea90136b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:09.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.576+0000 7fea95fa9700 1 -- 172.21.15.16:0/220550086 shutdown_connections 2023-09-30T22:31:09.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:09.576+0000 7fea95fa9700 1 -- 172.21.15.16:0/220550086 wait complete. 2023-09-30T22:31:09.590 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:31:10.303 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:10.348 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:10.403 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:10.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.454+0000 7f6460d65700 1 Processor -- start 2023-09-30T22:31:10.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.454+0000 7f6460d65700 1 -- start start 2023-09-30T22:31:10.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.455+0000 7f6460d65700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f645c1338a0 0x7f645c133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:10.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.455+0000 7f6460d65700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f645c09de50 0x7f645c09e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:10.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.455+0000 7f6460d65700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f645c09e760 0x7f645c13a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:10.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.455+0000 7f6460d65700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f645c06fef0 con 0x7f645c09de50 2023-09-30T22:31:10.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.455+0000 7f6460d65700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f645c06ef30 con 0x7f645c09e760 2023-09-30T22:31:10.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.455+0000 7f6460d65700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f645c1334b0 con 0x7f645c1338a0 2023-09-30T22:31:10.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.455+0000 7f6459d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f645c09de50 0x7f645c09e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:10.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.455+0000 7f645ad9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f645c09e760 0x7f645c13a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:10.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.455+0000 7f6459d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f645c09de50 0x7f645c09e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45886/0 (socket says 172.21.15.16:45886) 2023-09-30T22:31:10.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.455+0000 7f6459d9b700 1 -- 172.21.15.16:0/4187315562 learned_addr learned my addr 172.21.15.16:0/4187315562 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:10.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.455+0000 7f645a59c700 1 --2- 172.21.15.16:0/4187315562 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f645c1338a0 0x7f645c133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:10.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.456+0000 7f6459d9b700 1 -- 172.21.15.16:0/4187315562 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f645c1338a0 msgr2=0x7f645c133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:10.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.456+0000 7f6459d9b700 1 --2- 172.21.15.16:0/4187315562 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f645c1338a0 0x7f645c133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.456+0000 7f6459d9b700 1 -- 172.21.15.16:0/4187315562 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f645c09e760 msgr2=0x7f645c13a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:10.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.456+0000 7f6459d9b700 1 --2- 172.21.15.16:0/4187315562 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f645c09e760 0x7f645c13a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.456+0000 7f6459d9b700 1 -- 172.21.15.16:0/4187315562 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f645c06fc30 con 0x7f645c09de50 2023-09-30T22:31:10.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.456+0000 7f645a59c700 1 --2- 172.21.15.16:0/4187315562 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f645c1338a0 0x7f645c133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:10.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.456+0000 7f645ad9d700 1 --2- 172.21.15.16:0/4187315562 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f645c09e760 0x7f645c13a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:10.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.456+0000 7f6459d9b700 1 --2- 172.21.15.16:0/4187315562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f645c09de50 0x7f645c09e220 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f64500098f0 tx=0x7f6450009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ef51388bbd7d68b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:10.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.456+0000 7f645959a700 1 -- 172.21.15.16:0/4187315562 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6450011830 con 0x7f645c09de50 2023-09-30T22:31:10.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.456+0000 7f645959a700 1 -- 172.21.15.16:0/4187315562 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6450011990 con 0x7f645c09de50 2023-09-30T22:31:10.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.456+0000 7f645959a700 1 -- 172.21.15.16:0/4187315562 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6450011d20 con 0x7f645c09de50 2023-09-30T22:31:10.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.457+0000 7f6460d65700 1 -- 172.21.15.16:0/4187315562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f645c09de50 msgr2=0x7f645c09e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:10.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.457+0000 7f6460d65700 1 --2- 172.21.15.16:0/4187315562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f645c09de50 0x7f645c09e220 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f64500098f0 tx=0x7f6450009c00 comp rx=0 tx=0).stop 2023-09-30T22:31:10.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.457+0000 7f6460d65700 1 -- 172.21.15.16:0/4187315562 shutdown_connections 2023-09-30T22:31:10.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.457+0000 7f6460d65700 1 --2- 172.21.15.16:0/4187315562 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f645c09e760 0x7f645c13a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.457+0000 7f6460d65700 1 --2- 172.21.15.16:0/4187315562 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f645c1338a0 0x7f645c133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.457+0000 7f6460d65700 1 --2- 172.21.15.16:0/4187315562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f645c09de50 0x7f645c09e220 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.457+0000 7f6460d65700 1 -- 172.21.15.16:0/4187315562 >> 172.21.15.16:0/4187315562 conn(0x7f645c0a2aa0 msgr2=0x7f645c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:10.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.457+0000 7f6460d65700 1 -- 172.21.15.16:0/4187315562 shutdown_connections 2023-09-30T22:31:10.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.457+0000 7f6460d65700 1 -- 172.21.15.16:0/4187315562 wait complete. 2023-09-30T22:31:10.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.458+0000 7f6460d65700 1 Processor -- start 2023-09-30T22:31:10.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.458+0000 7f6460d65700 1 -- start start 2023-09-30T22:31:10.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.459+0000 7f6460d65700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f645c09de50 0x7f645c11ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:10.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.459+0000 7f6460d65700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f645c09e760 0x7f645c11ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:10.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.459+0000 7f6460d65700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f645c1338a0 0x7f645c11f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:10.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.459+0000 7f6460d65700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645c120ea0 con 0x7f645c09e760 2023-09-30T22:31:10.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.459+0000 7f6460d65700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f645c11feb0 con 0x7f645c1338a0 2023-09-30T22:31:10.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.459+0000 7f6460d65700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f645c120720 con 0x7f645c09de50 2023-09-30T22:31:10.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.459+0000 7f645a59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f645c09de50 0x7f645c11ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:10.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.459+0000 7f645a59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f645c09de50 0x7f645c11ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:59762/0 (socket says 172.21.15.16:59762) 2023-09-30T22:31:10.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.459+0000 7f645a59c700 1 -- 172.21.15.16:0/49836229 learned_addr learned my addr 172.21.15.16:0/49836229 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:10.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.459+0000 7f6459d9b700 1 --2- 172.21.15.16:0/49836229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f645c09e760 0x7f645c11ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:10.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.459+0000 7f645ad9d700 1 --2- 172.21.15.16:0/49836229 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f645c1338a0 0x7f645c11f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:10.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.460+0000 7f6459d9b700 1 -- 172.21.15.16:0/49836229 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f645c09de50 msgr2=0x7f645c11ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:10.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.460+0000 7f6459d9b700 1 --2- 172.21.15.16:0/49836229 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f645c09de50 0x7f645c11ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.460+0000 7f6459d9b700 1 -- 172.21.15.16:0/49836229 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f645c1338a0 msgr2=0x7f645c11f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:10.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.460+0000 7f6459d9b700 1 --2- 172.21.15.16:0/49836229 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f645c1338a0 0x7f645c11f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.460+0000 7f6459d9b700 1 -- 172.21.15.16:0/49836229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6454009a70 con 0x7f645c09e760 2023-09-30T22:31:10.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.460+0000 7f645a59c700 1 --2- 172.21.15.16:0/49836229 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f645c09de50 0x7f645c11ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:10.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.460+0000 7f6459d9b700 1 --2- 172.21.15.16:0/49836229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f645c09e760 0x7f645c11ef90 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6450016000 tx=0x7f6450004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:10.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.460+0000 7f644affd700 1 -- 172.21.15.16:0/49836229 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6450004450 con 0x7f645c09e760 2023-09-30T22:31:10.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.460+0000 7f644affd700 1 -- 172.21.15.16:0/49836229 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f645001e070 con 0x7f645c09e760 2023-09-30T22:31:10.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.460+0000 7f6460d65700 1 -- 172.21.15.16:0/49836229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f644c0094a0 con 0x7f645c09e760 2023-09-30T22:31:10.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.460+0000 7f644affd700 1 -- 172.21.15.16:0/49836229 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64500038e0 con 0x7f645c09e760 2023-09-30T22:31:10.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.460+0000 7f6460d65700 1 -- 172.21.15.16:0/49836229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6450009550 con 0x7f645c09e760 2023-09-30T22:31:10.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.461+0000 7f644affd700 1 -- 172.21.15.16:0/49836229 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6450003a40 con 0x7f645c09e760 2023-09-30T22:31:10.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.461+0000 7f644affd700 1 -- 172.21.15.16:0/49836229 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f6450040340 con 0x7f645c09e760 2023-09-30T22:31:10.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.462+0000 7f6460d65700 1 -- 172.21.15.16:0/49836229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f645c074ea0 con 0x7f645c09e760 2023-09-30T22:31:10.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.471+0000 7f644affd700 1 -- 172.21.15.16:0/49836229 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6450015080 con 0x7f645c09e760 2023-09-30T22:31:10.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.493+0000 7f865aae7700 1 Processor -- start 2023-09-30T22:31:10.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.493+0000 7f865aae7700 1 -- start start 2023-09-30T22:31:10.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.494+0000 7f865aae7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f865409f410 0x7f8654136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:10.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.494+0000 7f865aae7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8654136fe0 0x7f86541393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:10.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.494+0000 7f865aae7700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8654070940 con 0x7f865409e930 2023-09-30T22:31:10.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.494+0000 7f865aae7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f865406fd60 con 0x7f865409f410 2023-09-30T22:31:10.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.494+0000 7f865aae7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f865406fa80 con 0x7f8654136fe0 2023-09-30T22:31:10.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.494+0000 7f8658883700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f865409e930 0x7f865409ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47380/0 (socket says 172.21.15.16:47380) 2023-09-30T22:31:10.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.494+0000 7f8658883700 1 -- 172.21.15.16:0/2565132116 learned_addr learned my addr 172.21.15.16:0/2565132116 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:31:10.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.494+0000 7f8659084700 1 --2- 172.21.15.16:0/2565132116 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8654136fe0 0x7f86541393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:10.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.494+0000 7f8653fff700 1 --2- 172.21.15.16:0/2565132116 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f865409f410 0x7f8654136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:10.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.494+0000 7f86537fe700 1 -- 172.21.15.16:0/2565132116 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1658125809 0 0) 0x7f8654070940 con 0x7f865409e930 2023-09-30T22:31:10.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.494+0000 7f86537fe700 1 -- 172.21.15.16:0/2565132116 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f863c0035b0 con 0x7f865409e930 2023-09-30T22:31:10.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.494+0000 7f8653fff700 1 -- 172.21.15.16:0/2565132116 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8654136fe0 msgr2=0x7f86541393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:10.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.494+0000 7f8653fff700 1 --2- 172.21.15.16:0/2565132116 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8654136fe0 0x7f86541393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.494+0000 7f8653fff700 1 -- 172.21.15.16:0/2565132116 >> v1:172.21.15.16:6789/0 conn(0x7f865409e930 legacy=0x7f865409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:10.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.494+0000 7f8653fff700 1 -- 172.21.15.16:0/2565132116 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f865409fb60 con 0x7f865409f410 2023-09-30T22:31:10.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.495+0000 7f8653fff700 1 --2- 172.21.15.16:0/2565132116 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f865409f410 0x7f8654136aa0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8648009870 tx=0x7f8648009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c404d7eca3e7c96b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:10.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.495+0000 7f86537fe700 1 -- 172.21.15.16:0/2565132116 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8648011830 con 0x7f865409f410 2023-09-30T22:31:10.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.495+0000 7f86537fe700 1 -- 172.21.15.16:0/2565132116 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8648011990 con 0x7f865409f410 2023-09-30T22:31:10.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.495+0000 7f86537fe700 1 -- 172.21.15.16:0/2565132116 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8648011d00 con 0x7f865409f410 2023-09-30T22:31:10.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.496+0000 7f865aae7700 1 -- 172.21.15.16:0/2565132116 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f865409f410 msgr2=0x7f8654136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:10.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.496+0000 7f865aae7700 1 --2- 172.21.15.16:0/2565132116 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f865409f410 0x7f8654136aa0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8648009870 tx=0x7f8648009b80 comp rx=0 tx=0).stop 2023-09-30T22:31:10.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.496+0000 7f865aae7700 1 -- 172.21.15.16:0/2565132116 shutdown_connections 2023-09-30T22:31:10.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.496+0000 7f865aae7700 1 --2- 172.21.15.16:0/2565132116 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8654136fe0 0x7f86541393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.496+0000 7f865aae7700 1 --2- 172.21.15.16:0/2565132116 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f865409f410 0x7f8654136aa0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.496+0000 7f865aae7700 1 -- 172.21.15.16:0/2565132116 >> 172.21.15.16:0/2565132116 conn(0x7f86540a4210 msgr2=0x7f86540a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:10.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.496+0000 7f865aae7700 1 -- 172.21.15.16:0/2565132116 shutdown_connections 2023-09-30T22:31:10.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.496+0000 7f865aae7700 1 -- 172.21.15.16:0/2565132116 wait complete. 2023-09-30T22:31:10.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.497+0000 7f865aae7700 1 Processor -- start 2023-09-30T22:31:10.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.497+0000 7f865aae7700 1 -- start start 2023-09-30T22:31:10.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.498+0000 7f865aae7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f865409e930 0x7f86541c6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:10.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.498+0000 7f865aae7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f865409f410 0x7f86541c6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:10.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.498+0000 7f865aae7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8654136fe0 0x7f86541c6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:10.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.498+0000 7f865aae7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8654120150 con 0x7f865409e930 2023-09-30T22:31:10.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.498+0000 7f865aae7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8654071d20 con 0x7f865409f410 2023-09-30T22:31:10.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.498+0000 7f865aae7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f865411e440 con 0x7f8654136fe0 2023-09-30T22:31:10.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.498+0000 7f8653fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f865409f410 0x7f86541c6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:10.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.498+0000 7f8658883700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f865409e930 0x7f86541c6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:10.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.498+0000 7f8653fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f865409f410 0x7f86541c6780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:35154/0 (socket says 172.21.15.16:35154) 2023-09-30T22:31:10.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.498+0000 7f8653fff700 1 -- 172.21.15.16:0/1013175718 learned_addr learned my addr 172.21.15.16:0/1013175718 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:10.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.498+0000 7f8658883700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f865409e930 0x7f86541c6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45916/0 (socket says 172.21.15.16:45916) 2023-09-30T22:31:10.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.498+0000 7f8659084700 1 --2- 172.21.15.16:0/1013175718 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8654136fe0 0x7f86541c6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:10.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.498+0000 7f8653fff700 1 -- 172.21.15.16:0/1013175718 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8654136fe0 msgr2=0x7f86541c6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:10.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.498+0000 7f8653fff700 1 --2- 172.21.15.16:0/1013175718 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8654136fe0 0x7f86541c6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.498+0000 7f8653fff700 1 -- 172.21.15.16:0/1013175718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f865409e930 msgr2=0x7f86541c6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:10.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.498+0000 7f8653fff700 1 --2- 172.21.15.16:0/1013175718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f865409e930 0x7f86541c6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.498+0000 7f8653fff700 1 -- 172.21.15.16:0/1013175718 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86480094a0 con 0x7f865409f410 2023-09-30T22:31:10.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.498+0000 7f8658883700 1 --2- 172.21.15.16:0/1013175718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f865409e930 0x7f86541c6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:10.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.498+0000 7f8653fff700 1 --2- 172.21.15.16:0/1013175718 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f865409f410 0x7f86541c6780 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f86480090b0 tx=0x7f8648004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:10.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.499+0000 7f86517fa700 1 -- 172.21.15.16:0/1013175718 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8648004550 con 0x7f865409f410 2023-09-30T22:31:10.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.499+0000 7f86517fa700 1 -- 172.21.15.16:0/1013175718 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f864801f070 con 0x7f865409f410 2023-09-30T22:31:10.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.499+0000 7f865aae7700 1 -- 172.21.15.16:0/1013175718 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f865409df10 con 0x7f865409f410 2023-09-30T22:31:10.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.499+0000 7f86517fa700 1 -- 172.21.15.16:0/1013175718 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86480038e0 con 0x7f865409f410 2023-09-30T22:31:10.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.499+0000 7f865aae7700 1 -- 172.21.15.16:0/1013175718 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f865411f0b0 con 0x7f865409f410 2023-09-30T22:31:10.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.500+0000 7f86517fa700 1 -- 172.21.15.16:0/1013175718 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8648003a40 con 0x7f865409f410 2023-09-30T22:31:10.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.500+0000 7f865aae7700 1 -- 172.21.15.16:0/1013175718 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f865411fe10 con 0x7f865409f410 2023-09-30T22:31:10.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.500+0000 7f86517fa700 1 -- 172.21.15.16:0/1013175718 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f8648021070 con 0x7f865409f410 2023-09-30T22:31:10.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.505+0000 7f86517fa700 1 -- 172.21.15.16:0/1013175718 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f865411fe10 con 0x7f865409f410 2023-09-30T22:31:10.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.552+0000 7f700431b700 1 Processor -- start 2023-09-30T22:31:10.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.552+0000 7f700431b700 1 -- start start 2023-09-30T22:31:10.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.553+0000 7f700431b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6ffc09e930 0x7f6ffc09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:10.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.553+0000 7f700431b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6ffc09f650 0x7f6ffc131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:10.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.553+0000 7f700431b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ffc070940 con 0x7f6ffc09f240 2023-09-30T22:31:10.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.553+0000 7f700431b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ffc06fd60 con 0x7f6ffc09f650 2023-09-30T22:31:10.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.553+0000 7f700431b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6ffc06fa80 con 0x7f6ffc09e930 2023-09-30T22:31:10.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.553+0000 7f70018b6700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6ffc09f240 0x7f6ffc1317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47386/0 (socket says 172.21.15.16:47386) 2023-09-30T22:31:10.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.553+0000 7f70018b6700 1 -- 172.21.15.16:0/17452674 learned_addr learned my addr 172.21.15.16:0/17452674 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:31:10.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.553+0000 7f70020b7700 1 --2- 172.21.15.16:0/17452674 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6ffc09e930 0x7f6ffc09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:10.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.553+0000 7f70028b8700 1 --2- 172.21.15.16:0/17452674 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6ffc09f650 0x7f6ffc131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:10.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.554+0000 7f70010b5700 1 -- 172.21.15.16:0/17452674 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 311055090 0 0) 0x7f6ffc070940 con 0x7f6ffc09f240 2023-09-30T22:31:10.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.554+0000 7f70010b5700 1 -- 172.21.15.16:0/17452674 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fe80035b0 con 0x7f6ffc09f240 2023-09-30T22:31:10.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.554+0000 7f70020b7700 1 -- 172.21.15.16:0/17452674 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6ffc09f650 msgr2=0x7f6ffc131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:10.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.554+0000 7f70020b7700 1 --2- 172.21.15.16:0/17452674 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6ffc09f650 0x7f6ffc131f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.554+0000 7f70020b7700 1 -- 172.21.15.16:0/17452674 >> v1:172.21.15.16:6789/0 conn(0x7f6ffc09f240 legacy=0x7f6ffc1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:10.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.554+0000 7f70020b7700 1 -- 172.21.15.16:0/17452674 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ffc09fb60 con 0x7f6ffc09e930 2023-09-30T22:31:10.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.554+0000 7f70028b8700 1 --2- 172.21.15.16:0/17452674 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6ffc09f650 0x7f6ffc131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:10.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.554+0000 7f70020b7700 1 --2- 172.21.15.16:0/17452674 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6ffc09e930 0x7f6ffc09ed00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8009870 tx=0x7f6ff8009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e53de1f3ee86e9d0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:10.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.554+0000 7f70010b5700 1 -- 172.21.15.16:0/17452674 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff8011830 con 0x7f6ffc09e930 2023-09-30T22:31:10.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.554+0000 7f70010b5700 1 -- 172.21.15.16:0/17452674 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ff8011990 con 0x7f6ffc09e930 2023-09-30T22:31:10.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.554+0000 7f70010b5700 1 -- 172.21.15.16:0/17452674 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff8011d20 con 0x7f6ffc09e930 2023-09-30T22:31:10.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.555+0000 7f700431b700 1 -- 172.21.15.16:0/17452674 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6ffc09e930 msgr2=0x7f6ffc09ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:10.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.555+0000 7f700431b700 1 --2- 172.21.15.16:0/17452674 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6ffc09e930 0x7f6ffc09ed00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8009870 tx=0x7f6ff8009b80 comp rx=0 tx=0).stop 2023-09-30T22:31:10.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.555+0000 7f700431b700 1 -- 172.21.15.16:0/17452674 shutdown_connections 2023-09-30T22:31:10.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.555+0000 7f700431b700 1 --2- 172.21.15.16:0/17452674 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6ffc09f650 0x7f6ffc131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.555+0000 7f700431b700 1 --2- 172.21.15.16:0/17452674 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6ffc09e930 0x7f6ffc09ed00 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.555+0000 7f700431b700 1 -- 172.21.15.16:0/17452674 >> 172.21.15.16:0/17452674 conn(0x7f6ffc0a4210 msgr2=0x7f6ffc0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:10.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.555+0000 7f700431b700 1 -- 172.21.15.16:0/17452674 shutdown_connections 2023-09-30T22:31:10.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.555+0000 7f700431b700 1 -- 172.21.15.16:0/17452674 wait complete. 2023-09-30T22:31:10.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.556+0000 7f700431b700 1 Processor -- start 2023-09-30T22:31:10.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.556+0000 7f700431b700 1 -- start start 2023-09-30T22:31:10.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.557+0000 7f700431b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6ffc09e930 0x7f6ffc1a3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:10.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.557+0000 7f700431b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6ffc09f240 0x7f6ffc1a4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:10.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.557+0000 7f700431b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ffc09f650 0x7f6ffc1a45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:10.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.557+0000 7f700431b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ffc1a7670 con 0x7f6ffc09f650 2023-09-30T22:31:10.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.557+0000 7f700431b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ffc1a8010 con 0x7f6ffc09f240 2023-09-30T22:31:10.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.557+0000 7f700431b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6ffc1a6e00 con 0x7f6ffc09e930 2023-09-30T22:31:10.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.557+0000 7f70018b6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6ffc09f240 0x7f6ffc1a4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:10.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.557+0000 7f70018b6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6ffc09f240 0x7f6ffc1a4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:35184/0 (socket says 172.21.15.16:35184) 2023-09-30T22:31:10.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.557+0000 7f70018b6700 1 -- 172.21.15.16:0/3447418982 learned_addr learned my addr 172.21.15.16:0/3447418982 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:10.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.557+0000 7f70028b8700 1 --2- 172.21.15.16:0/3447418982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ffc09f650 0x7f6ffc1a45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:10.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.557+0000 7f70018b6700 1 -- 172.21.15.16:0/3447418982 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6ffc09e930 msgr2=0x7f6ffc1a3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:10.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.557+0000 7f70018b6700 1 --2- 172.21.15.16:0/3447418982 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6ffc09e930 0x7f6ffc1a3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.557+0000 7f70018b6700 1 -- 172.21.15.16:0/3447418982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ffc09f650 msgr2=0x7f6ffc1a45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:10.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.557+0000 7f70018b6700 1 --2- 172.21.15.16:0/3447418982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ffc09f650 0x7f6ffc1a45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.557+0000 7f70018b6700 1 -- 172.21.15.16:0/3447418982 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff4009a70 con 0x7f6ffc09f240 2023-09-30T22:31:10.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.557+0000 7f70018b6700 1 --2- 172.21.15.16:0/3447418982 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6ffc09f240 0x7f6ffc1a4090 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6fec005910 tx=0x7f6fec00c9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:10.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.557+0000 7f6ff2ffd700 1 -- 172.21.15.16:0/3447418982 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fec005d70 con 0x7f6ffc09f240 2023-09-30T22:31:10.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.558+0000 7f6ff2ffd700 1 -- 172.21.15.16:0/3447418982 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fec0070d0 con 0x7f6ffc09f240 2023-09-30T22:31:10.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.558+0000 7f700431b700 1 -- 172.21.15.16:0/3447418982 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ff80094a0 con 0x7f6ffc09f240 2023-09-30T22:31:10.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.558+0000 7f6ff2ffd700 1 -- 172.21.15.16:0/3447418982 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fec007440 con 0x7f6ffc09f240 2023-09-30T22:31:10.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.558+0000 7f700431b700 1 -- 172.21.15.16:0/3447418982 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ffc0a0190 con 0x7f6ffc09f240 2023-09-30T22:31:10.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.559+0000 7f700431b700 1 -- 172.21.15.16:0/3447418982 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ffc1a71f0 con 0x7f6ffc09f240 2023-09-30T22:31:10.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.559+0000 7f6ff2ffd700 1 -- 172.21.15.16:0/3447418982 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6fec017020 con 0x7f6ffc09f240 2023-09-30T22:31:10.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.559+0000 7f6ff2ffd700 1 -- 172.21.15.16:0/3447418982 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f6fec042c50 con 0x7f6ffc09f240 2023-09-30T22:31:10.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.564+0000 7f6ff2ffd700 1 -- 172.21.15.16:0/3447418982 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6ffc1a71f0 con 0x7f6ffc09f240 2023-09-30T22:31:10.591 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:10.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.692+0000 7f6460d65700 1 -- 172.21.15.16:0/49836229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f645c12f500 con 0x7f645c09e760 2023-09-30T22:31:10.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.693+0000 7f644affd700 1 -- 172.21.15.16:0/49836229 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f645002d450 con 0x7f645c09e760 2023-09-30T22:31:10.693 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:31:10.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.696+0000 7f6460d65700 1 -- 172.21.15.16:0/49836229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f645c09e760 msgr2=0x7f645c11ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:10.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.696+0000 7f6460d65700 1 --2- 172.21.15.16:0/49836229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f645c09e760 0x7f645c11ef90 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6450016000 tx=0x7f6450004750 comp rx=0 tx=0).stop 2023-09-30T22:31:10.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.696+0000 7f6460d65700 1 -- 172.21.15.16:0/49836229 shutdown_connections 2023-09-30T22:31:10.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.696+0000 7f6460d65700 1 --2- 172.21.15.16:0/49836229 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f645c1338a0 0x7f645c11f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.696+0000 7f6460d65700 1 --2- 172.21.15.16:0/49836229 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f645c09de50 0x7f645c11ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.696+0000 7f6460d65700 1 --2- 172.21.15.16:0/49836229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f645c09e760 0x7f645c11ef90 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.696+0000 7f6460d65700 1 -- 172.21.15.16:0/49836229 >> 172.21.15.16:0/49836229 conn(0x7f645c0a2aa0 msgr2=0x7f645c09ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:10.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.696+0000 7f6460d65700 1 -- 172.21.15.16:0/49836229 shutdown_connections 2023-09-30T22:31:10.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.696+0000 7f6460d65700 1 -- 172.21.15.16:0/49836229 wait complete. 2023-09-30T22:31:10.717 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:31:10.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.732+0000 7f865aae7700 1 -- 172.21.15.16:0/1013175718 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8654074ea0 con 0x7f865409f410 2023-09-30T22:31:10.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.732+0000 7f692d9df700 1 Processor -- start 2023-09-30T22:31:10.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.732+0000 7f692d9df700 1 -- start start 2023-09-30T22:31:10.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.732+0000 7f86517fa700 1 -- 172.21.15.16:0/1013175718 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8648015070 con 0x7f865409f410 2023-09-30T22:31:10.733 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:31:10.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.733+0000 7f692d9df700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f692812cf60 0x7f692806f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:10.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.733+0000 7f692d9df700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f692812f8b0 0x7f692812fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:10.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.733+0000 7f692d9df700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6928130260 0x7f6928136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:10.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.733+0000 7f692d9df700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f692812d330 con 0x7f692812cf60 2023-09-30T22:31:10.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.733+0000 7f692d9df700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f692812cd90 con 0x7f692812f8b0 2023-09-30T22:31:10.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.733+0000 7f692d9df700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f69281306d0 con 0x7f6928130260 2023-09-30T22:31:10.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.733+0000 7f69267fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f692812f8b0 0x7f692812fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:10.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.733+0000 7f69267fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f692812f8b0 0x7f692812fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:35188/0 (socket says 172.21.15.16:35188) 2023-09-30T22:31:10.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.733+0000 7f69267fc700 1 -- 172.21.15.16:0/1955361552 learned_addr learned my addr 172.21.15.16:0/1955361552 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:10.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.733+0000 7f69277fe700 1 --2- 172.21.15.16:0/1955361552 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6928130260 0x7f6928136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:10.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.733+0000 7f6926ffd700 1 --2- 172.21.15.16:0/1955361552 >> v2:172.21.15.16:3300/0 conn(0x7f692812cf60 0x7f692806f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:10.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.733+0000 7f69267fc700 1 -- 172.21.15.16:0/1955361552 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6928130260 msgr2=0x7f6928136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:10.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.734+0000 7f69267fc700 1 --2- 172.21.15.16:0/1955361552 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6928130260 0x7f6928136c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.734+0000 7f69267fc700 1 -- 172.21.15.16:0/1955361552 >> v2:172.21.15.16:3300/0 conn(0x7f692812cf60 msgr2=0x7f692806f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:10.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.734+0000 7f69267fc700 1 --2- 172.21.15.16:0/1955361552 >> v2:172.21.15.16:3300/0 conn(0x7f692812cf60 0x7f692806f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.734+0000 7f69267fc700 1 -- 172.21.15.16:0/1955361552 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6928071780 con 0x7f692812f8b0 2023-09-30T22:31:10.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.734+0000 7f69267fc700 1 --2- 172.21.15.16:0/1955361552 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f692812f8b0 0x7f692812fd20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f691c00bb90 tx=0x7f691c00bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6d4db0f25a5483 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:10.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.734+0000 7f6925ffb700 1 -- 172.21.15.16:0/1955361552 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691c00dc20 con 0x7f692812f8b0 2023-09-30T22:31:10.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.734+0000 7f6925ffb700 1 -- 172.21.15.16:0/1955361552 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f691c00dd80 con 0x7f692812f8b0 2023-09-30T22:31:10.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.734+0000 7f6925ffb700 1 -- 172.21.15.16:0/1955361552 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691c017650 con 0x7f692812f8b0 2023-09-30T22:31:10.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.735+0000 7f692d9df700 1 -- 172.21.15.16:0/1955361552 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f692812f8b0 msgr2=0x7f692812fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:10.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.735+0000 7f692d9df700 1 --2- 172.21.15.16:0/1955361552 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f692812f8b0 0x7f692812fd20 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f691c00bb90 tx=0x7f691c00bea0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.735+0000 7f692d9df700 1 -- 172.21.15.16:0/1955361552 shutdown_connections 2023-09-30T22:31:10.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.735+0000 7f692d9df700 1 --2- 172.21.15.16:0/1955361552 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6928130260 0x7f6928136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.735+0000 7f692d9df700 1 --2- 172.21.15.16:0/1955361552 >> v2:172.21.15.16:3300/0 conn(0x7f692812cf60 0x7f692806f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.735+0000 7f692d9df700 1 --2- 172.21.15.16:0/1955361552 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f692812f8b0 0x7f692812fd20 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.735+0000 7f692d9df700 1 -- 172.21.15.16:0/1955361552 >> 172.21.15.16:0/1955361552 conn(0x7f69280a2aa0 msgr2=0x7f69280a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:10.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.735+0000 7f692d9df700 1 -- 172.21.15.16:0/1955361552 shutdown_connections 2023-09-30T22:31:10.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.735+0000 7f692d9df700 1 -- 172.21.15.16:0/1955361552 wait complete. 2023-09-30T22:31:10.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.735+0000 7f865aae7700 1 -- 172.21.15.16:0/1013175718 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f865409f410 msgr2=0x7f86541c6780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:10.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.735+0000 7f865aae7700 1 --2- 172.21.15.16:0/1013175718 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f865409f410 0x7f86541c6780 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f86480090b0 tx=0x7f8648004750 comp rx=0 tx=0).stop 2023-09-30T22:31:10.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.735+0000 7f865aae7700 1 -- 172.21.15.16:0/1013175718 shutdown_connections 2023-09-30T22:31:10.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.735+0000 7f865aae7700 1 --2- 172.21.15.16:0/1013175718 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8654136fe0 0x7f86541c6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.735+0000 7f865aae7700 1 --2- 172.21.15.16:0/1013175718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f865409e930 0x7f86541c6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.735+0000 7f865aae7700 1 --2- 172.21.15.16:0/1013175718 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f865409f410 0x7f86541c6780 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.735+0000 7f865aae7700 1 -- 172.21.15.16:0/1013175718 >> 172.21.15.16:0/1013175718 conn(0x7f86540a4210 msgr2=0x7f8654134b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:10.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.735+0000 7f865aae7700 1 -- 172.21.15.16:0/1013175718 shutdown_connections 2023-09-30T22:31:10.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.735+0000 7f865aae7700 1 -- 172.21.15.16:0/1013175718 wait complete. 2023-09-30T22:31:10.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.736+0000 7f692d9df700 1 Processor -- start 2023-09-30T22:31:10.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.737+0000 7f692d9df700 1 -- start start 2023-09-30T22:31:10.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.737+0000 7f692d9df700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f692812cf60 0x7f692809bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:10.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.737+0000 7f692d9df700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f692812f8b0 0x7f692809c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:10.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.737+0000 7f692d9df700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6928130260 0x7f69281c3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:10.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.737+0000 7f692d9df700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6928129900 con 0x7f6928130260 2023-09-30T22:31:10.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.737+0000 7f692d9df700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f692809ed50 con 0x7f692812cf60 2023-09-30T22:31:10.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.737+0000 7f692d9df700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f692809b710 con 0x7f692812f8b0 2023-09-30T22:31:10.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.737+0000 7f69267fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f692812f8b0 0x7f692809c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:10.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.738+0000 7f6926ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f692812cf60 0x7f692809bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:10.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.738+0000 7f69267fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f692812f8b0 0x7f692809c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:59820/0 (socket says 172.21.15.16:59820) 2023-09-30T22:31:10.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.738+0000 7f69267fc700 1 -- 172.21.15.16:0/4285367129 learned_addr learned my addr 172.21.15.16:0/4285367129 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:10.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.738+0000 7f69277fe700 1 --2- 172.21.15.16:0/4285367129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6928130260 0x7f69281c3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:10.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.738+0000 7f6926ffd700 1 -- 172.21.15.16:0/4285367129 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f692812f8b0 msgr2=0x7f692809c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:10.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.738+0000 7f6926ffd700 1 --2- 172.21.15.16:0/4285367129 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f692812f8b0 0x7f692809c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.738+0000 7f6926ffd700 1 -- 172.21.15.16:0/4285367129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6928130260 msgr2=0x7f69281c3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:10.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.738+0000 7f6926ffd700 1 --2- 172.21.15.16:0/4285367129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6928130260 0x7f69281c3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.738+0000 7f6926ffd700 1 -- 172.21.15.16:0/4285367129 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f691c00b7f0 con 0x7f692812cf60 2023-09-30T22:31:10.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.738+0000 7f69267fc700 1 --2- 172.21.15.16:0/4285367129 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f692812f8b0 0x7f692809c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:10.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.738+0000 7f69277fe700 1 --2- 172.21.15.16:0/4285367129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6928130260 0x7f69281c3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:10.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.738+0000 7f6926ffd700 1 --2- 172.21.15.16:0/4285367129 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f692812cf60 0x7f692809bc00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6918009ca0 tx=0x7f6918010f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:10.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.738+0000 7f690ffff700 1 -- 172.21.15.16:0/4285367129 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6918015070 con 0x7f692812cf60 2023-09-30T22:31:10.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.738+0000 7f690ffff700 1 -- 172.21.15.16:0/4285367129 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f691800b930 con 0x7f692812cf60 2023-09-30T22:31:10.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.738+0000 7f692d9df700 1 -- 172.21.15.16:0/4285367129 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f692809cdb0 con 0x7f692812cf60 2023-09-30T22:31:10.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.738+0000 7f690ffff700 1 -- 172.21.15.16:0/4285367129 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691800bca0 con 0x7f692812cf60 2023-09-30T22:31:10.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.739+0000 7f692d9df700 1 -- 172.21.15.16:0/4285367129 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f692812d920 con 0x7f692812cf60 2023-09-30T22:31:10.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.739+0000 7f690ffff700 1 -- 172.21.15.16:0/4285367129 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6918019410 con 0x7f692812cf60 2023-09-30T22:31:10.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.740+0000 7f692d9df700 1 -- 172.21.15.16:0/4285367129 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f692809e820 con 0x7f692812cf60 2023-09-30T22:31:10.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.740+0000 7f690ffff700 1 -- 172.21.15.16:0/4285367129 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f6918042f70 con 0x7f692812cf60 2023-09-30T22:31:10.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.745+0000 7f690ffff700 1 -- 172.21.15.16:0/4285367129 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f69180196c0 con 0x7f692812cf60 2023-09-30T22:31:10.752 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:31:10.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.776+0000 7f700431b700 1 -- 172.21.15.16:0/3447418982 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f6ffc1a0b30 con 0x7f6ffc09f240 2023-09-30T22:31:10.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.777+0000 7f6ff2ffd700 1 -- 172.21.15.16:0/3447418982 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6fec017e60 con 0x7f6ffc09f240 2023-09-30T22:31:10.777 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:31:10.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.779+0000 7f700431b700 1 -- 172.21.15.16:0/3447418982 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6ffc09f240 msgr2=0x7f6ffc1a4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:10.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.779+0000 7f700431b700 1 --2- 172.21.15.16:0/3447418982 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6ffc09f240 0x7f6ffc1a4090 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6fec005910 tx=0x7f6fec00c9e0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.779+0000 7f700431b700 1 -- 172.21.15.16:0/3447418982 shutdown_connections 2023-09-30T22:31:10.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.779+0000 7f700431b700 1 --2- 172.21.15.16:0/3447418982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ffc09f650 0x7f6ffc1a45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.779+0000 7f700431b700 1 --2- 172.21.15.16:0/3447418982 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6ffc09e930 0x7f6ffc1a3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.779+0000 7f700431b700 1 --2- 172.21.15.16:0/3447418982 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6ffc09f240 0x7f6ffc1a4090 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.779+0000 7f700431b700 1 -- 172.21.15.16:0/3447418982 >> 172.21.15.16:0/3447418982 conn(0x7f6ffc0a4210 msgr2=0x7f6ffc1b8a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:10.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.779+0000 7f700431b700 1 -- 172.21.15.16:0/3447418982 shutdown_connections 2023-09-30T22:31:10.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.779+0000 7f700431b700 1 -- 172.21.15.16:0/3447418982 wait complete. 2023-09-30T22:31:10.793 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:31:10.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.951+0000 7f692d9df700 1 -- 172.21.15.16:0/4285367129 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6928074ea0 con 0x7f692812cf60 2023-09-30T22:31:10.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.951+0000 7f690ffff700 1 -- 172.21.15.16:0/4285367129 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f691801e070 con 0x7f692812cf60 2023-09-30T22:31:10.952 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:31:10.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.953+0000 7f692d9df700 1 -- 172.21.15.16:0/4285367129 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f692812cf60 msgr2=0x7f692809bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:10.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.953+0000 7f692d9df700 1 --2- 172.21.15.16:0/4285367129 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f692812cf60 0x7f692809bc00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6918009ca0 tx=0x7f6918010f60 comp rx=0 tx=0).stop 2023-09-30T22:31:10.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.953+0000 7f692d9df700 1 -- 172.21.15.16:0/4285367129 shutdown_connections 2023-09-30T22:31:10.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.953+0000 7f692d9df700 1 --2- 172.21.15.16:0/4285367129 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f692812cf60 0x7f692809bc00 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.953+0000 7f692d9df700 1 --2- 172.21.15.16:0/4285367129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6928130260 0x7f69281c3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.953+0000 7f692d9df700 1 --2- 172.21.15.16:0/4285367129 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f692812f8b0 0x7f692809c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:10.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.953+0000 7f692d9df700 1 -- 172.21.15.16:0/4285367129 >> 172.21.15.16:0/4285367129 conn(0x7f69280a2aa0 msgr2=0x7f692812a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:10.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.953+0000 7f692d9df700 1 -- 172.21.15.16:0/4285367129 shutdown_connections 2023-09-30T22:31:10.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:10.953+0000 7f692d9df700 1 -- 172.21.15.16:0/4285367129 wait complete. 2023-09-30T22:31:10.967 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:31:11.717 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:11.752 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:11.794 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:11.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.870+0000 7f70357ba700 1 Processor -- start 2023-09-30T22:31:11.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.870+0000 7f70357ba700 1 -- start start 2023-09-30T22:31:11.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.871+0000 7f70357ba700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f703009d6f0 0x7f703009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:11.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.871+0000 7f70357ba700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f703009e440 0x7f7030134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:11.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.871+0000 7f70357ba700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f703009b120 con 0x7f703009e000 2023-09-30T22:31:11.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.871+0000 7f70357ba700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f703006b460 con 0x7f703009d6f0 2023-09-30T22:31:11.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.871+0000 7f70357ba700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f703009a7e0 con 0x7f703009e440 2023-09-30T22:31:11.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.871+0000 7f702effd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f703009d6f0 0x7f703009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:11.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.871+0000 7f702effd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f703009d6f0 0x7f703009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45944/0 (socket says 172.21.15.16:45944) 2023-09-30T22:31:11.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.871+0000 7f702effd700 1 -- 172.21.15.16:0/235400405 learned_addr learned my addr 172.21.15.16:0/235400405 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:11.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.871+0000 7f7027fff700 1 --1- 172.21.15.16:0/235400405 >> v1:172.21.15.16:6789/0 conn(0x7f703009e000 0x7f70301339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47388/0 (socket says 172.21.15.16:47388) 2023-09-30T22:31:11.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.871+0000 7f702f7fe700 1 --2- 172.21.15.16:0/235400405 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f703009e440 0x7f7030134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:11.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.871+0000 7f702effd700 1 -- 172.21.15.16:0/235400405 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f703009e440 msgr2=0x7f7030134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:11.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.871+0000 7f702effd700 1 --2- 172.21.15.16:0/235400405 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f703009e440 0x7f7030134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:11.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.872+0000 7f702effd700 1 -- 172.21.15.16:0/235400405 >> v1:172.21.15.16:6789/0 conn(0x7f703009e000 legacy=0x7f70301339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:11.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.872+0000 7f702effd700 1 -- 172.21.15.16:0/235400405 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f703009a500 con 0x7f703009d6f0 2023-09-30T22:31:11.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.872+0000 7f702f7fe700 1 --2- 172.21.15.16:0/235400405 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f703009e440 0x7f7030134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:11.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.872+0000 7f702effd700 1 --2- 172.21.15.16:0/235400405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f703009d6f0 0x7f703009dac0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f7020009920 tx=0x7f7020009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb126a5adfc31291 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:11.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.872+0000 7f702e7fc700 1 -- 172.21.15.16:0/235400405 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7020011830 con 0x7f703009d6f0 2023-09-30T22:31:11.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.872+0000 7f702e7fc700 1 -- 172.21.15.16:0/235400405 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7020011990 con 0x7f703009d6f0 2023-09-30T22:31:11.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.872+0000 7f702e7fc700 1 -- 172.21.15.16:0/235400405 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7020011d20 con 0x7f703009d6f0 2023-09-30T22:31:11.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.873+0000 7f70357ba700 1 -- 172.21.15.16:0/235400405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f703009d6f0 msgr2=0x7f703009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:11.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.873+0000 7f70357ba700 1 --2- 172.21.15.16:0/235400405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f703009d6f0 0x7f703009dac0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f7020009920 tx=0x7f7020009c30 comp rx=0 tx=0).stop 2023-09-30T22:31:11.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.873+0000 7f70357ba700 1 -- 172.21.15.16:0/235400405 shutdown_connections 2023-09-30T22:31:11.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.873+0000 7f70357ba700 1 --2- 172.21.15.16:0/235400405 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f703009e440 0x7f7030134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:11.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.873+0000 7f70357ba700 1 --2- 172.21.15.16:0/235400405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f703009d6f0 0x7f703009dac0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:11.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.873+0000 7f70357ba700 1 -- 172.21.15.16:0/235400405 >> 172.21.15.16:0/235400405 conn(0x7f7030066f50 msgr2=0x7f703009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:11.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.873+0000 7f70357ba700 1 -- 172.21.15.16:0/235400405 shutdown_connections 2023-09-30T22:31:11.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.873+0000 7f70357ba700 1 -- 172.21.15.16:0/235400405 wait complete. 2023-09-30T22:31:11.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.874+0000 7f70357ba700 1 Processor -- start 2023-09-30T22:31:11.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.874+0000 7f70357ba700 1 -- start start 2023-09-30T22:31:11.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.875+0000 7f70357ba700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f703009d6f0 0x7f70301c85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:11.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.875+0000 7f70357ba700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f703009e000 0x7f70301c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:11.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.875+0000 7f70357ba700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f703009e440 0x7f70301c4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:11.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.875+0000 7f70357ba700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70301a7170 con 0x7f703009d6f0 2023-09-30T22:31:11.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.875+0000 7f70357ba700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70301a8bd0 con 0x7f703009e000 2023-09-30T22:31:11.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.875+0000 7f70357ba700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f70301a9280 con 0x7f703009e440 2023-09-30T22:31:11.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.875+0000 7f7027fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f703009e000 0x7f70301c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:11.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.875+0000 7f702effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f703009d6f0 0x7f70301c85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:11.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.875+0000 7f7027fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f703009e000 0x7f70301c8b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:35194/0 (socket says 172.21.15.16:35194) 2023-09-30T22:31:11.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.875+0000 7f7027fff700 1 -- 172.21.15.16:0/2762505356 learned_addr learned my addr 172.21.15.16:0/2762505356 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:11.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.875+0000 7f702effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f703009d6f0 0x7f70301c85f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45960/0 (socket says 172.21.15.16:45960) 2023-09-30T22:31:11.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.875+0000 7f702f7fe700 1 --2- 172.21.15.16:0/2762505356 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f703009e440 0x7f70301c4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:11.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.876+0000 7f702effd700 1 -- 172.21.15.16:0/2762505356 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f703009e440 msgr2=0x7f70301c4ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:11.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.876+0000 7f702effd700 1 --2- 172.21.15.16:0/2762505356 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f703009e440 0x7f70301c4ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:11.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.876+0000 7f702effd700 1 -- 172.21.15.16:0/2762505356 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f703009e000 msgr2=0x7f70301c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:11.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.876+0000 7f702effd700 1 --2- 172.21.15.16:0/2762505356 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f703009e000 0x7f70301c8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:11.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.876+0000 7f702effd700 1 -- 172.21.15.16:0/2762505356 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7028009a70 con 0x7f703009d6f0 2023-09-30T22:31:11.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.876+0000 7f7027fff700 1 --2- 172.21.15.16:0/2762505356 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f703009e000 0x7f70301c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:11.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.876+0000 7f702effd700 1 --2- 172.21.15.16:0/2762505356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f703009d6f0 0x7f70301c85f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7020016000 tx=0x7f7020004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:11.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.876+0000 7f70277fe700 1 -- 172.21.15.16:0/2762505356 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70200042c0 con 0x7f703009d6f0 2023-09-30T22:31:11.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.876+0000 7f70277fe700 1 -- 172.21.15.16:0/2762505356 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f702001f070 con 0x7f703009d6f0 2023-09-30T22:31:11.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.876+0000 7f70357ba700 1 -- 172.21.15.16:0/2762505356 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7020009550 con 0x7f703009d6f0 2023-09-30T22:31:11.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.876+0000 7f70277fe700 1 -- 172.21.15.16:0/2762505356 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70200036a0 con 0x7f703009d6f0 2023-09-30T22:31:11.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.876+0000 7f70357ba700 1 -- 172.21.15.16:0/2762505356 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70301a8710 con 0x7f703009d6f0 2023-09-30T22:31:11.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.877+0000 7f70357ba700 1 -- 172.21.15.16:0/2762505356 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7030074ea0 con 0x7f703009d6f0 2023-09-30T22:31:11.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.877+0000 7f70277fe700 1 -- 172.21.15.16:0/2762505356 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7020021030 con 0x7f703009d6f0 2023-09-30T22:31:11.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.878+0000 7f70277fe700 1 -- 172.21.15.16:0/2762505356 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f70200493a0 con 0x7f703009d6f0 2023-09-30T22:31:11.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.887+0000 7f70277fe700 1 -- 172.21.15.16:0/2762505356 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7020045020 con 0x7f703009d6f0 2023-09-30T22:31:11.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.906+0000 7f13944b3700 1 Processor -- start 2023-09-30T22:31:11.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.906+0000 7f13944b3700 1 -- start start 2023-09-30T22:31:11.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.906+0000 7f13944b3700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f138c133a80 0x7f138c133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:11.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.906+0000 7f13944b3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f138c130a70 0x7f138c13b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:11.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.906+0000 7f13944b3700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f138c06bc00 con 0x7f138c12ff20 2023-09-30T22:31:11.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.906+0000 7f13944b3700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f138c06c570 con 0x7f138c133a80 2023-09-30T22:31:11.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.907+0000 7f13944b3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f138c0a0040 con 0x7f138c130a70 2023-09-30T22:31:11.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.907+0000 7f1392a50700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f138c130a70 0x7f138c13b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:11.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.907+0000 7f1392a50700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f138c130a70 0x7f138c13b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:59836/0 (socket says 172.21.15.16:59836) 2023-09-30T22:31:11.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.907+0000 7f1391a4e700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f138c12ff20 0x7f138c130360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47396/0 (socket says 172.21.15.16:47396) 2023-09-30T22:31:11.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.907+0000 7f1392a50700 1 -- 172.21.15.16:0/2890533456 learned_addr learned my addr 172.21.15.16:0/2890533456 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:11.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.907+0000 7f139224f700 1 --2- 172.21.15.16:0/2890533456 >> v2:172.21.15.16:3300/0 conn(0x7f138c133a80 0x7f138c133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:11.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.907+0000 7f1392a50700 1 -- 172.21.15.16:0/2890533456 >> v2:172.21.15.16:3300/0 conn(0x7f138c133a80 msgr2=0x7f138c133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:11.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.907+0000 7f1392a50700 1 --2- 172.21.15.16:0/2890533456 >> v2:172.21.15.16:3300/0 conn(0x7f138c133a80 0x7f138c133e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:11.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.907+0000 7f1392a50700 1 -- 172.21.15.16:0/2890533456 >> v1:172.21.15.16:6789/0 conn(0x7f138c12ff20 legacy=0x7f138c130360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:11.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.907+0000 7f139124d700 1 -- 172.21.15.16:0/2890533456 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 335578050 0 0) 0x7f138c06bc00 con 0x7f138c12ff20 2023-09-30T22:31:11.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.907+0000 7f1392a50700 1 -- 172.21.15.16:0/2890533456 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f138c133860 con 0x7f138c130a70 2023-09-30T22:31:11.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.907+0000 7f139224f700 1 --2- 172.21.15.16:0/2890533456 >> v2:172.21.15.16:3300/0 conn(0x7f138c133a80 0x7f138c133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:11.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.907+0000 7f1392a50700 1 --2- 172.21.15.16:0/2890533456 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f138c130a70 0x7f138c13b850 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1384009e50 tx=0x7f138400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=848bd90ee0817d31 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:11.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.908+0000 7f139124d700 1 -- 172.21.15.16:0/2890533456 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138400e070 con 0x7f138c130a70 2023-09-30T22:31:11.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.908+0000 7f139124d700 1 -- 172.21.15.16:0/2890533456 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f138400b910 con 0x7f138c130a70 2023-09-30T22:31:11.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.908+0000 7f139124d700 1 -- 172.21.15.16:0/2890533456 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138400bcc0 con 0x7f138c130a70 2023-09-30T22:31:11.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.909+0000 7f13944b3700 1 -- 172.21.15.16:0/2890533456 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f138c130a70 msgr2=0x7f138c13b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:11.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.909+0000 7f13944b3700 1 --2- 172.21.15.16:0/2890533456 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f138c130a70 0x7f138c13b850 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1384009e50 tx=0x7f138400b330 comp rx=0 tx=0).stop 2023-09-30T22:31:11.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.909+0000 7f13944b3700 1 -- 172.21.15.16:0/2890533456 shutdown_connections 2023-09-30T22:31:11.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.909+0000 7f13944b3700 1 --2- 172.21.15.16:0/2890533456 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f138c130a70 0x7f138c13b850 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:11.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.909+0000 7f13944b3700 1 --2- 172.21.15.16:0/2890533456 >> v2:172.21.15.16:3300/0 conn(0x7f138c133a80 0x7f138c133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:11.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.909+0000 7f13944b3700 1 -- 172.21.15.16:0/2890533456 >> 172.21.15.16:0/2890533456 conn(0x7f138c0a4210 msgr2=0x7f138c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:11.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.909+0000 7f13944b3700 1 -- 172.21.15.16:0/2890533456 shutdown_connections 2023-09-30T22:31:11.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.909+0000 7f13944b3700 1 -- 172.21.15.16:0/2890533456 wait complete. 2023-09-30T22:31:11.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.910+0000 7f13944b3700 1 Processor -- start 2023-09-30T22:31:11.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.910+0000 7f13944b3700 1 -- start start 2023-09-30T22:31:11.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.910+0000 7f13944b3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f138c12ff20 0x7f138c09c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:11.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.910+0000 7f13944b3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f138c130a70 0x7f138c09c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:11.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.910+0000 7f13944b3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f138c133a80 0x7f138c09cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:11.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.910+0000 7f13944b3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138c11fd40 con 0x7f138c130a70 2023-09-30T22:31:11.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.910+0000 7f13944b3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f138c11ef80 con 0x7f138c133a80 2023-09-30T22:31:11.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.910+0000 7f13944b3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f138c11fa80 con 0x7f138c12ff20 2023-09-30T22:31:11.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.911+0000 7f1391a4e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f138c130a70 0x7f138c09c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:11.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.911+0000 7f1391a4e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f138c130a70 0x7f138c09c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45990/0 (socket says 172.21.15.16:45990) 2023-09-30T22:31:11.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.911+0000 7f1391a4e700 1 -- 172.21.15.16:0/1627638828 learned_addr learned my addr 172.21.15.16:0/1627638828 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:11.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.911+0000 7f1392a50700 1 --2- 172.21.15.16:0/1627638828 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f138c133a80 0x7f138c09cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:11.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.911+0000 7f139224f700 1 --2- 172.21.15.16:0/1627638828 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f138c12ff20 0x7f138c09c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:11.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.911+0000 7f1391a4e700 1 -- 172.21.15.16:0/1627638828 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f138c12ff20 msgr2=0x7f138c09c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:11.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.911+0000 7f1391a4e700 1 --2- 172.21.15.16:0/1627638828 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f138c12ff20 0x7f138c09c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:11.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.911+0000 7f1391a4e700 1 -- 172.21.15.16:0/1627638828 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f138c133a80 msgr2=0x7f138c09cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:11.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.911+0000 7f1391a4e700 1 --2- 172.21.15.16:0/1627638828 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f138c133a80 0x7f138c09cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:11.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.911+0000 7f1391a4e700 1 -- 172.21.15.16:0/1627638828 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13880094a0 con 0x7f138c130a70 2023-09-30T22:31:11.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.911+0000 7f1391a4e700 1 --2- 172.21.15.16:0/1627638828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f138c130a70 0x7f138c09c9c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f137c0011e0 tx=0x7f137c00ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:11.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.911+0000 7f1382ffd700 1 -- 172.21.15.16:0/1627638828 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137c004c80 con 0x7f138c130a70 2023-09-30T22:31:11.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.911+0000 7f1382ffd700 1 -- 172.21.15.16:0/1627638828 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f137c004de0 con 0x7f138c130a70 2023-09-30T22:31:11.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.911+0000 7f13944b3700 1 -- 172.21.15.16:0/1627638828 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1384009b30 con 0x7f138c130a70 2023-09-30T22:31:11.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.911+0000 7f1382ffd700 1 -- 172.21.15.16:0/1627638828 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137c007310 con 0x7f138c130a70 2023-09-30T22:31:11.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.912+0000 7f13944b3700 1 -- 172.21.15.16:0/1627638828 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f138c12dad0 con 0x7f138c130a70 2023-09-30T22:31:11.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.912+0000 7f1382ffd700 1 -- 172.21.15.16:0/1627638828 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f137c007470 con 0x7f138c130a70 2023-09-30T22:31:11.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.913+0000 7f1382ffd700 1 -- 172.21.15.16:0/1627638828 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f137c03f3c0 con 0x7f138c130a70 2023-09-30T22:31:11.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.913+0000 7f13944b3700 1 -- 172.21.15.16:0/1627638828 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f138c074ea0 con 0x7f138c130a70 2023-09-30T22:31:11.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.918+0000 7f1382ffd700 1 -- 172.21.15.16:0/1627638828 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f137c0123b0 con 0x7f138c130a70 2023-09-30T22:31:11.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.951+0000 7f34175a1700 1 Processor -- start 2023-09-30T22:31:11.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.951+0000 7f34175a1700 1 -- start start 2023-09-30T22:31:11.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.951+0000 7f34175a1700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f341006a770 0x7f341006ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:11.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.951+0000 7f34175a1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f341006b080 0x7f3410131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:11.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.951+0000 7f34175a1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3410131b40 0x7f3410131fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:11.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.951+0000 7f34175a1700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f341009ece0 con 0x7f341006a770 2023-09-30T22:31:11.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.951+0000 7f34175a1700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f341009e760 con 0x7f3410131b40 2023-09-30T22:31:11.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.951+0000 7f34175a1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f341009e040 con 0x7f341006b080 2023-09-30T22:31:11.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.952+0000 7f3414b3c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f341006b080 0x7f3410131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:11.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.952+0000 7f3414b3c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f341006b080 0x7f3410131600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:59864/0 (socket says 172.21.15.16:59864) 2023-09-30T22:31:11.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.952+0000 7f341533d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f341006a770 0x7f341006ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:11.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.952+0000 7f3414b3c700 1 -- 172.21.15.16:0/1212161920 learned_addr learned my addr 172.21.15.16:0/1212161920 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:11.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.952+0000 7f3415b3e700 1 --2- 172.21.15.16:0/1212161920 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3410131b40 0x7f3410131fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:11.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.952+0000 7f3414b3c700 1 -- 172.21.15.16:0/1212161920 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3410131b40 msgr2=0x7f3410131fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:11.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.952+0000 7f3414b3c700 1 --2- 172.21.15.16:0/1212161920 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3410131b40 0x7f3410131fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:11.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.952+0000 7f3414b3c700 1 -- 172.21.15.16:0/1212161920 >> v2:172.21.15.16:3300/0 conn(0x7f341006a770 msgr2=0x7f341006ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:11.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.952+0000 7f3414b3c700 1 --2- 172.21.15.16:0/1212161920 >> v2:172.21.15.16:3300/0 conn(0x7f341006a770 0x7f341006ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:11.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.952+0000 7f3414b3c700 1 -- 172.21.15.16:0/1212161920 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f341009f630 con 0x7f341006b080 2023-09-30T22:31:11.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.952+0000 7f3415b3e700 1 --2- 172.21.15.16:0/1212161920 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3410131b40 0x7f3410131fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:31:11.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.952+0000 7f341533d700 1 --2- 172.21.15.16:0/1212161920 >> v2:172.21.15.16:3300/0 conn(0x7f341006a770 0x7f341006ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:11.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.952+0000 7f3414b3c700 1 --2- 172.21.15.16:0/1212161920 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f341006b080 0x7f3410131600 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f34000098f0 tx=0x7f3400009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3f3bfd91fa43517 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:11.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.952+0000 7f3407fff700 1 -- 172.21.15.16:0/1212161920 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3400011830 con 0x7f341006b080 2023-09-30T22:31:11.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.952+0000 7f3407fff700 1 -- 172.21.15.16:0/1212161920 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3400011990 con 0x7f341006b080 2023-09-30T22:31:11.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.952+0000 7f3407fff700 1 -- 172.21.15.16:0/1212161920 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3400011d00 con 0x7f341006b080 2023-09-30T22:31:11.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.953+0000 7f34175a1700 1 -- 172.21.15.16:0/1212161920 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f341006b080 msgr2=0x7f3410131600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:11.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.953+0000 7f34175a1700 1 --2- 172.21.15.16:0/1212161920 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f341006b080 0x7f3410131600 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f34000098f0 tx=0x7f3400009c00 comp rx=0 tx=0).stop 2023-09-30T22:31:11.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.953+0000 7f34175a1700 1 -- 172.21.15.16:0/1212161920 shutdown_connections 2023-09-30T22:31:11.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.953+0000 7f34175a1700 1 --2- 172.21.15.16:0/1212161920 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3410131b40 0x7f3410131fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:11.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.953+0000 7f34175a1700 1 --2- 172.21.15.16:0/1212161920 >> v2:172.21.15.16:3300/0 conn(0x7f341006a770 0x7f341006ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:11.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.953+0000 7f34175a1700 1 --2- 172.21.15.16:0/1212161920 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f341006b080 0x7f3410131600 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:11.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.953+0000 7f34175a1700 1 -- 172.21.15.16:0/1212161920 >> 172.21.15.16:0/1212161920 conn(0x7f3410066f50 msgr2=0x7f341009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:11.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.953+0000 7f34175a1700 1 -- 172.21.15.16:0/1212161920 shutdown_connections 2023-09-30T22:31:11.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.953+0000 7f34175a1700 1 -- 172.21.15.16:0/1212161920 wait complete. 2023-09-30T22:31:11.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.954+0000 7f34175a1700 1 Processor -- start 2023-09-30T22:31:11.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.954+0000 7f34175a1700 1 -- start start 2023-09-30T22:31:11.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.955+0000 7f34175a1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f341006a770 0x7f341011e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:11.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.955+0000 7f34175a1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f341006b080 0x7f341011e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:11.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.955+0000 7f34175a1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3410131b40 0x7f341011ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:11.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.955+0000 7f34175a1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3410121c60 con 0x7f3410131b40 2023-09-30T22:31:11.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.955+0000 7f34175a1700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3410071e60 con 0x7f341006a770 2023-09-30T22:31:11.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.955+0000 7f34175a1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f34101201b0 con 0x7f341006b080 2023-09-30T22:31:11.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.955+0000 7f3415b3e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3410131b40 0x7f341011ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:11.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.955+0000 7f3415b3e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3410131b40 0x7f341011ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46010/0 (socket says 172.21.15.16:46010) 2023-09-30T22:31:11.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.955+0000 7f3415b3e700 1 -- 172.21.15.16:0/2615694035 learned_addr learned my addr 172.21.15.16:0/2615694035 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:11.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.955+0000 7f3414b3c700 1 --2- 172.21.15.16:0/2615694035 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f341006b080 0x7f341011e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:11.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.955+0000 7f341533d700 1 --2- 172.21.15.16:0/2615694035 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f341006a770 0x7f341011e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:11.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.955+0000 7f3414b3c700 1 -- 172.21.15.16:0/2615694035 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f341006a770 msgr2=0x7f341011e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:11.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.955+0000 7f3414b3c700 1 --2- 172.21.15.16:0/2615694035 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f341006a770 0x7f341011e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:11.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.955+0000 7f3414b3c700 1 -- 172.21.15.16:0/2615694035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3410131b40 msgr2=0x7f341011ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:11.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.955+0000 7f3414b3c700 1 --2- 172.21.15.16:0/2615694035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3410131b40 0x7f341011ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:11.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.955+0000 7f3414b3c700 1 -- 172.21.15.16:0/2615694035 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f340c00d940 con 0x7f341006b080 2023-09-30T22:31:11.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.955+0000 7f341533d700 1 --2- 172.21.15.16:0/2615694035 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f341006a770 0x7f341011e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:31:11.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.956+0000 7f3414b3c700 1 --2- 172.21.15.16:0/2615694035 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f341006b080 0x7f341011e9d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f341009d2a0 tx=0x7f3400004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:11.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.956+0000 7f3405ffb700 1 -- 172.21.15.16:0/2615694035 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3400004450 con 0x7f341006b080 2023-09-30T22:31:11.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.956+0000 7f3405ffb700 1 -- 172.21.15.16:0/2615694035 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f340001f070 con 0x7f341006b080 2023-09-30T22:31:11.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.956+0000 7f34175a1700 1 -- 172.21.15.16:0/2615694035 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3400009550 con 0x7f341006b080 2023-09-30T22:31:11.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.956+0000 7f3405ffb700 1 -- 172.21.15.16:0/2615694035 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34000038c0 con 0x7f341006b080 2023-09-30T22:31:11.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.956+0000 7f34175a1700 1 -- 172.21.15.16:0/2615694035 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3410121f40 con 0x7f341006b080 2023-09-30T22:31:11.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.957+0000 7f3405ffb700 1 -- 172.21.15.16:0/2615694035 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3400003a20 con 0x7f341006b080 2023-09-30T22:31:11.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.957+0000 7f34175a1700 1 -- 172.21.15.16:0/2615694035 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3410074ea0 con 0x7f341006b080 2023-09-30T22:31:11.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.957+0000 7f3405ffb700 1 -- 172.21.15.16:0/2615694035 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f3400021070 con 0x7f341006b080 2023-09-30T22:31:11.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:11.967+0000 7f3405ffb700 1 -- 172.21.15.16:0/2615694035 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f340002b080 con 0x7f341006b080 2023-09-30T22:31:11.968 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:12.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.107+0000 7f70357ba700 1 -- 172.21.15.16:0/2762505356 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f703006c360 con 0x7f703009d6f0 2023-09-30T22:31:12.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.107+0000 7f70277fe700 1 -- 172.21.15.16:0/2762505356 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7020015070 con 0x7f703009d6f0 2023-09-30T22:31:12.108 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:31:12.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.109+0000 7f70357ba700 1 -- 172.21.15.16:0/2762505356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f703009d6f0 msgr2=0x7f70301c85f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:12.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.109+0000 7f70357ba700 1 --2- 172.21.15.16:0/2762505356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f703009d6f0 0x7f70301c85f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7020016000 tx=0x7f7020004750 comp rx=0 tx=0).stop 2023-09-30T22:31:12.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.109+0000 7f70357ba700 1 -- 172.21.15.16:0/2762505356 shutdown_connections 2023-09-30T22:31:12.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.109+0000 7f70357ba700 1 --2- 172.21.15.16:0/2762505356 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f703009e440 0x7f70301c4ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:12.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.109+0000 7f70357ba700 1 --2- 172.21.15.16:0/2762505356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f703009d6f0 0x7f70301c85f0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:12.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.109+0000 7f70357ba700 1 --2- 172.21.15.16:0/2762505356 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f703009e000 0x7f70301c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:12.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.110+0000 7f70357ba700 1 -- 172.21.15.16:0/2762505356 >> 172.21.15.16:0/2762505356 conn(0x7f7030066f50 msgr2=0x7f70301bac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:12.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.110+0000 7f70357ba700 1 -- 172.21.15.16:0/2762505356 shutdown_connections 2023-09-30T22:31:12.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.110+0000 7f70357ba700 1 -- 172.21.15.16:0/2762505356 wait complete. 2023-09-30T22:31:12.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.111+0000 7f835d72a700 1 Processor -- start 2023-09-30T22:31:12.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.111+0000 7f835d72a700 1 -- start start 2023-09-30T22:31:12.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.111+0000 7f835d72a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83581338c0 0x7f8358133c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:12.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.111+0000 7f835d72a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f83581341d0 0x7f835809de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:12.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.112+0000 7f835d72a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f835809e390 0x7f835809e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:12.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.112+0000 7f835d72a700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8358070c80 con 0x7f83581341d0 2023-09-30T22:31:12.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.112+0000 7f835d72a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f835809ed20 con 0x7f83581338c0 2023-09-30T22:31:12.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.112+0000 7f835d72a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f835806f720 con 0x7f835809e390 2023-09-30T22:31:12.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.112+0000 7f83577fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f835809e390 0x7f835809e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:12.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.112+0000 7f83577fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f835809e390 0x7f835809e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:59882/0 (socket says 172.21.15.16:59882) 2023-09-30T22:31:12.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.112+0000 7f83577fe700 1 -- 172.21.15.16:0/674516837 learned_addr learned my addr 172.21.15.16:0/674516837 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:12.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.112+0000 7f83567fc700 1 --2- 172.21.15.16:0/674516837 >> v2:172.21.15.16:3300/0 conn(0x7f83581341d0 0x7f835809de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:12.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.112+0000 7f8356ffd700 1 --2- 172.21.15.16:0/674516837 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83581338c0 0x7f8358133c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:12.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.112+0000 7f83577fe700 1 -- 172.21.15.16:0/674516837 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83581338c0 msgr2=0x7f8358133c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:12.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.112+0000 7f83577fe700 1 --2- 172.21.15.16:0/674516837 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83581338c0 0x7f8358133c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:12.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.112+0000 7f83577fe700 1 -- 172.21.15.16:0/674516837 >> v2:172.21.15.16:3300/0 conn(0x7f83581341d0 msgr2=0x7f835809de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:12.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.112+0000 7f83577fe700 1 --2- 172.21.15.16:0/674516837 >> v2:172.21.15.16:3300/0 conn(0x7f83581341d0 0x7f835809de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:12.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.112+0000 7f83577fe700 1 -- 172.21.15.16:0/674516837 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f835806fce0 con 0x7f835809e390 2023-09-30T22:31:12.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.113+0000 7f83577fe700 1 --2- 172.21.15.16:0/674516837 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f835809e390 0x7f835809e7e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8350009e80 tx=0x7f835000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b42058289e7ac6c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:12.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.113+0000 7f8355ffb700 1 -- 172.21.15.16:0/674516837 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f835000e070 con 0x7f835809e390 2023-09-30T22:31:12.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.113+0000 7f8355ffb700 1 -- 172.21.15.16:0/674516837 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f835000b8d0 con 0x7f835809e390 2023-09-30T22:31:12.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.113+0000 7f8355ffb700 1 -- 172.21.15.16:0/674516837 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f835000bc80 con 0x7f835809e390 2023-09-30T22:31:12.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.113+0000 7f835d72a700 1 -- 172.21.15.16:0/674516837 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f835809e390 msgr2=0x7f835809e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:12.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.113+0000 7f835d72a700 1 --2- 172.21.15.16:0/674516837 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f835809e390 0x7f835809e7e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8350009e80 tx=0x7f835000b330 comp rx=0 tx=0).stop 2023-09-30T22:31:12.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.114+0000 7f835d72a700 1 -- 172.21.15.16:0/674516837 shutdown_connections 2023-09-30T22:31:12.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.114+0000 7f835d72a700 1 --2- 172.21.15.16:0/674516837 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83581338c0 0x7f8358133c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:12.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.114+0000 7f835d72a700 1 --2- 172.21.15.16:0/674516837 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f835809e390 0x7f835809e7e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:12.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.114+0000 7f835d72a700 1 --2- 172.21.15.16:0/674516837 >> v2:172.21.15.16:3300/0 conn(0x7f83581341d0 0x7f835809de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:12.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.114+0000 7f835d72a700 1 -- 172.21.15.16:0/674516837 >> 172.21.15.16:0/674516837 conn(0x7f83580a2aa0 msgr2=0x7f83580a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:12.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.114+0000 7f835d72a700 1 -- 172.21.15.16:0/674516837 shutdown_connections 2023-09-30T22:31:12.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.114+0000 7f835d72a700 1 -- 172.21.15.16:0/674516837 wait complete. 2023-09-30T22:31:12.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.115+0000 7f835d72a700 1 Processor -- start 2023-09-30T22:31:12.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.116+0000 7f835d72a700 1 -- start start 2023-09-30T22:31:12.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.116+0000 7f835d72a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f835809e390 0x7f835811a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:12.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.116+0000 7f835d72a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83581338c0 0x7f835811add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:12.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.116+0000 7f835d72a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83581341d0 0x7f835811b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:12.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.116+0000 7f835d72a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f835811e0d0 con 0x7f83581338c0 2023-09-30T22:31:12.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.116+0000 7f835d72a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f835812dc80 con 0x7f83581341d0 2023-09-30T22:31:12.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.116+0000 7f835d72a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8358130ca0 con 0x7f835809e390 2023-09-30T22:31:12.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.116+0000 7f8356ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f835809e390 0x7f835811a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:12.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.116+0000 7f8356ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f835809e390 0x7f835811a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:59884/0 (socket says 172.21.15.16:59884) 2023-09-30T22:31:12.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.117+0000 7f8356ffd700 1 -- 172.21.15.16:0/2987807944 learned_addr learned my addr 172.21.15.16:0/2987807944 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:12.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.117+0000 7f83567fc700 1 --2- 172.21.15.16:0/2987807944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83581338c0 0x7f835811add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:12.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.117+0000 7f83577fe700 1 --2- 172.21.15.16:0/2987807944 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83581341d0 0x7f835811b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:12.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.117+0000 7f8356ffd700 1 -- 172.21.15.16:0/2987807944 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83581341d0 msgr2=0x7f835811b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:12.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.117+0000 7f8356ffd700 1 --2- 172.21.15.16:0/2987807944 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83581341d0 0x7f835811b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:12.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.117+0000 7f8356ffd700 1 -- 172.21.15.16:0/2987807944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83581338c0 msgr2=0x7f835811add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:12.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.117+0000 7f8356ffd700 1 --2- 172.21.15.16:0/2987807944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83581338c0 0x7f835811add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:12.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.117+0000 7f8356ffd700 1 -- 172.21.15.16:0/2987807944 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8350009b30 con 0x7f835809e390 2023-09-30T22:31:12.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.117+0000 7f8356ffd700 1 --2- 172.21.15.16:0/2987807944 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f835809e390 0x7f835811a890 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f834800e9b0 tx=0x7f834800ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:12.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.118+0000 7f833f7fe700 1 -- 172.21.15.16:0/2987807944 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834800ee50 con 0x7f835809e390 2023-09-30T22:31:12.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.118+0000 7f833f7fe700 1 -- 172.21.15.16:0/2987807944 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f834800c850 con 0x7f835809e390 2023-09-30T22:31:12.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.118+0000 7f835d72a700 1 -- 172.21.15.16:0/2987807944 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f835811e3b0 con 0x7f835809e390 2023-09-30T22:31:12.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.118+0000 7f833f7fe700 1 -- 172.21.15.16:0/2987807944 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834800cbc0 con 0x7f835809e390 2023-09-30T22:31:12.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.118+0000 7f835d72a700 1 -- 172.21.15.16:0/2987807944 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f835811de10 con 0x7f835809e390 2023-09-30T22:31:12.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.118+0000 7f833f7fe700 1 -- 172.21.15.16:0/2987807944 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f834800cd20 con 0x7f835809e390 2023-09-30T22:31:12.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.119+0000 7f833f7fe700 1 -- 172.21.15.16:0/2987807944 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f834803f770 con 0x7f835809e390 2023-09-30T22:31:12.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.120+0000 7f835d72a700 1 -- 172.21.15.16:0/2987807944 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f835806e670 con 0x7f835809e390 2023-09-30T22:31:12.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.125+0000 7f833f7fe700 1 -- 172.21.15.16:0/2987807944 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8348021080 con 0x7f835809e390 2023-09-30T22:31:12.126 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:31:12.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.141+0000 7f13944b3700 1 -- 172.21.15.16:0/1627638828 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f138c06b250 con 0x7f138c130a70 2023-09-30T22:31:12.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.141+0000 7f1382ffd700 1 -- 172.21.15.16:0/1627638828 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f137c011360 con 0x7f138c130a70 2023-09-30T22:31:12.142 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:31:12.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.143+0000 7f13944b3700 1 -- 172.21.15.16:0/1627638828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f138c130a70 msgr2=0x7f138c09c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:12.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.143+0000 7f13944b3700 1 --2- 172.21.15.16:0/1627638828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f138c130a70 0x7f138c09c9c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f137c0011e0 tx=0x7f137c00ca30 comp rx=0 tx=0).stop 2023-09-30T22:31:12.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.143+0000 7f13944b3700 1 -- 172.21.15.16:0/1627638828 shutdown_connections 2023-09-30T22:31:12.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.143+0000 7f13944b3700 1 --2- 172.21.15.16:0/1627638828 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f138c133a80 0x7f138c09cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:12.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.143+0000 7f13944b3700 1 --2- 172.21.15.16:0/1627638828 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f138c12ff20 0x7f138c09c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:12.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.143+0000 7f13944b3700 1 --2- 172.21.15.16:0/1627638828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f138c130a70 0x7f138c09c9c0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:12.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.143+0000 7f13944b3700 1 -- 172.21.15.16:0/1627638828 >> 172.21.15.16:0/1627638828 conn(0x7f138c0a4210 msgr2=0x7f138c09e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:12.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.143+0000 7f13944b3700 1 -- 172.21.15.16:0/1627638828 shutdown_connections 2023-09-30T22:31:12.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.143+0000 7f13944b3700 1 -- 172.21.15.16:0/1627638828 wait complete. 2023-09-30T22:31:12.157 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:31:12.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.202+0000 7f34175a1700 1 -- 172.21.15.16:0/2615694035 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3410121da0 con 0x7f341006b080 2023-09-30T22:31:12.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.202+0000 7f3405ffb700 1 -- 172.21.15.16:0/2615694035 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3410121da0 con 0x7f341006b080 2023-09-30T22:31:12.203 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:31:12.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.205+0000 7f34175a1700 1 -- 172.21.15.16:0/2615694035 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f341006b080 msgr2=0x7f341011e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:12.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.205+0000 7f34175a1700 1 --2- 172.21.15.16:0/2615694035 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f341006b080 0x7f341011e9d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f341009d2a0 tx=0x7f3400004750 comp rx=0 tx=0).stop 2023-09-30T22:31:12.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.205+0000 7f34175a1700 1 -- 172.21.15.16:0/2615694035 shutdown_connections 2023-09-30T22:31:12.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.205+0000 7f34175a1700 1 --2- 172.21.15.16:0/2615694035 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f341006a770 0x7f341011e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:12.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.205+0000 7f34175a1700 1 --2- 172.21.15.16:0/2615694035 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3410131b40 0x7f341011ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:12.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.205+0000 7f34175a1700 1 --2- 172.21.15.16:0/2615694035 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f341006b080 0x7f341011e9d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:12.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.205+0000 7f34175a1700 1 -- 172.21.15.16:0/2615694035 >> 172.21.15.16:0/2615694035 conn(0x7f3410066f50 msgr2=0x7f34100a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:12.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.205+0000 7f34175a1700 1 -- 172.21.15.16:0/2615694035 shutdown_connections 2023-09-30T22:31:12.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.205+0000 7f34175a1700 1 -- 172.21.15.16:0/2615694035 wait complete. 2023-09-30T22:31:12.219 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:31:12.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.354+0000 7f835d72a700 1 -- 172.21.15.16:0/2987807944 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f835811dc00 con 0x7f835809e390 2023-09-30T22:31:12.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.354+0000 7f833f7fe700 1 -- 172.21.15.16:0/2987807944 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f834801bdd0 con 0x7f835809e390 2023-09-30T22:31:12.355 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:31:12.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.357+0000 7f835d72a700 1 -- 172.21.15.16:0/2987807944 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f835809e390 msgr2=0x7f835811a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:12.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.357+0000 7f835d72a700 1 --2- 172.21.15.16:0/2987807944 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f835809e390 0x7f835811a890 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f834800e9b0 tx=0x7f834800ecc0 comp rx=0 tx=0).stop 2023-09-30T22:31:12.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.357+0000 7f835d72a700 1 -- 172.21.15.16:0/2987807944 shutdown_connections 2023-09-30T22:31:12.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.357+0000 7f835d72a700 1 --2- 172.21.15.16:0/2987807944 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83581341d0 0x7f835811b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:12.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.357+0000 7f835d72a700 1 --2- 172.21.15.16:0/2987807944 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f835809e390 0x7f835811a890 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:12.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.357+0000 7f835d72a700 1 --2- 172.21.15.16:0/2987807944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83581338c0 0x7f835811add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:12.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.357+0000 7f835d72a700 1 -- 172.21.15.16:0/2987807944 >> 172.21.15.16:0/2987807944 conn(0x7f83580a2aa0 msgr2=0x7f835812f620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:12.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.357+0000 7f835d72a700 1 -- 172.21.15.16:0/2987807944 shutdown_connections 2023-09-30T22:31:12.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:12.357+0000 7f835d72a700 1 -- 172.21.15.16:0/2987807944 wait complete. 2023-09-30T22:31:12.371 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:31:13.128 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:13.158 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:13.221 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:13.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.277+0000 7faebe0f6700 1 Processor -- start 2023-09-30T22:31:13.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.277+0000 7faebe0f6700 1 -- start start 2023-09-30T22:31:13.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.278+0000 7faebe0f6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faeb8130670 0x7faeb8130a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:13.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.278+0000 7faebe0f6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faeb80701a0 0x7faeb8138540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:13.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.278+0000 7faebe0f6700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faeb8130260 con 0x7faeb806f680 2023-09-30T22:31:13.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.278+0000 7faebe0f6700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeb809a350 con 0x7faeb8130670 2023-09-30T22:31:13.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.278+0000 7faebe0f6700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faeb8071ff0 con 0x7faeb80701a0 2023-09-30T22:31:13.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.278+0000 7faeb77fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faeb8130670 0x7faeb8130a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:13.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.278+0000 7faeb6ffd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7faeb806f680 0x7faeb806fa90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47402/0 (socket says 172.21.15.16:47402) 2023-09-30T22:31:13.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.278+0000 7faeb6ffd700 1 -- 172.21.15.16:0/1712583589 learned_addr learned my addr 172.21.15.16:0/1712583589 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:31:13.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.278+0000 7faeb7fff700 1 --2- 172.21.15.16:0/1712583589 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faeb80701a0 0x7faeb8138540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:13.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.278+0000 7faeb67fc700 1 -- 172.21.15.16:0/1712583589 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4080098050 0 0) 0x7faeb8130260 con 0x7faeb806f680 2023-09-30T22:31:13.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.278+0000 7faeb67fc700 1 -- 172.21.15.16:0/1712583589 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faea00035b0 con 0x7faeb806f680 2023-09-30T22:31:13.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.278+0000 7faeb77fe700 1 -- 172.21.15.16:0/1712583589 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faeb80701a0 msgr2=0x7faeb8138540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:13.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.278+0000 7faeb77fe700 1 --2- 172.21.15.16:0/1712583589 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faeb80701a0 0x7faeb8138540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.278+0000 7faeb77fe700 1 -- 172.21.15.16:0/1712583589 >> v1:172.21.15.16:6789/0 conn(0x7faeb806f680 legacy=0x7faeb806fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:13.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.278+0000 7faeb77fe700 1 -- 172.21.15.16:0/1712583589 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb809ecf0 con 0x7faeb8130670 2023-09-30T22:31:13.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.278+0000 7faeb7fff700 1 --2- 172.21.15.16:0/1712583589 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faeb80701a0 0x7faeb8138540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:13.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.279+0000 7faeb77fe700 1 --2- 172.21.15.16:0/1712583589 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faeb8130670 0x7faeb8130a40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7faea8009870 tx=0x7faea8009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=612e85b8e5e1bcc2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:13.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.279+0000 7faeb67fc700 1 -- 172.21.15.16:0/1712583589 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea8011830 con 0x7faeb8130670 2023-09-30T22:31:13.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.279+0000 7faeb67fc700 1 -- 172.21.15.16:0/1712583589 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faea8011990 con 0x7faeb8130670 2023-09-30T22:31:13.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.279+0000 7faeb67fc700 1 -- 172.21.15.16:0/1712583589 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea8011d20 con 0x7faeb8130670 2023-09-30T22:31:13.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.280+0000 7faebe0f6700 1 -- 172.21.15.16:0/1712583589 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faeb8130670 msgr2=0x7faeb8130a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:13.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.280+0000 7faebe0f6700 1 --2- 172.21.15.16:0/1712583589 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faeb8130670 0x7faeb8130a40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7faea8009870 tx=0x7faea8009b80 comp rx=0 tx=0).stop 2023-09-30T22:31:13.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.280+0000 7faebe0f6700 1 -- 172.21.15.16:0/1712583589 shutdown_connections 2023-09-30T22:31:13.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.280+0000 7faebe0f6700 1 --2- 172.21.15.16:0/1712583589 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faeb80701a0 0x7faeb8138540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.280+0000 7faebe0f6700 1 --2- 172.21.15.16:0/1712583589 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faeb8130670 0x7faeb8130a40 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.280+0000 7faebe0f6700 1 -- 172.21.15.16:0/1712583589 >> 172.21.15.16:0/1712583589 conn(0x7faeb80a2aa0 msgr2=0x7faeb80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:13.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.280+0000 7faebe0f6700 1 -- 172.21.15.16:0/1712583589 shutdown_connections 2023-09-30T22:31:13.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.280+0000 7faebe0f6700 1 -- 172.21.15.16:0/1712583589 wait complete. 2023-09-30T22:31:13.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.281+0000 7faebe0f6700 1 Processor -- start 2023-09-30T22:31:13.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.282+0000 7faebe0f6700 1 -- start start 2023-09-30T22:31:13.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.282+0000 7faebe0f6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faeb806f680 0x7faeb811c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:13.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.282+0000 7faebe0f6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faeb80701a0 0x7faeb811ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:13.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.282+0000 7faebe0f6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faeb8130670 0x7faeb811d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:13.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.282+0000 7faebe0f6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb8071820 con 0x7faeb80701a0 2023-09-30T22:31:13.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.282+0000 7faebe0f6700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faeb809fa60 con 0x7faeb806f680 2023-09-30T22:31:13.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.282+0000 7faebe0f6700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faeb81203e0 con 0x7faeb8130670 2023-09-30T22:31:13.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.283+0000 7faeb6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faeb80701a0 0x7faeb811ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:13.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.283+0000 7faeb77fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faeb806f680 0x7faeb811c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:13.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.283+0000 7faeb6ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faeb80701a0 0x7faeb811ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46044/0 (socket says 172.21.15.16:46044) 2023-09-30T22:31:13.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.283+0000 7faeb6ffd700 1 -- 172.21.15.16:0/3713862893 learned_addr learned my addr 172.21.15.16:0/3713862893 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:13.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.283+0000 7faeb77fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faeb806f680 0x7faeb811c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:35268/0 (socket says 172.21.15.16:35268) 2023-09-30T22:31:13.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.283+0000 7faeb7fff700 1 --2- 172.21.15.16:0/3713862893 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faeb8130670 0x7faeb811d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:13.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.283+0000 7faeb77fe700 1 -- 172.21.15.16:0/3713862893 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faeb8130670 msgr2=0x7faeb811d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:13.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.283+0000 7faeb77fe700 1 --2- 172.21.15.16:0/3713862893 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faeb8130670 0x7faeb811d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.283+0000 7faeb77fe700 1 -- 172.21.15.16:0/3713862893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faeb80701a0 msgr2=0x7faeb811ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:13.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.283+0000 7faeb77fe700 1 --2- 172.21.15.16:0/3713862893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faeb80701a0 0x7faeb811ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.283+0000 7faeb77fe700 1 -- 172.21.15.16:0/3713862893 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb0009a70 con 0x7faeb806f680 2023-09-30T22:31:13.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.283+0000 7faeb6ffd700 1 --2- 172.21.15.16:0/3713862893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faeb80701a0 0x7faeb811ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:13.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.283+0000 7faeb77fe700 1 --2- 172.21.15.16:0/3713862893 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faeb806f680 0x7faeb811c8c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7faea8009870 tx=0x7faea8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:13.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.283+0000 7fae9ffff700 1 -- 172.21.15.16:0/3713862893 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea8004450 con 0x7faeb806f680 2023-09-30T22:31:13.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.283+0000 7fae9ffff700 1 -- 172.21.15.16:0/3713862893 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faea80045d0 con 0x7faeb806f680 2023-09-30T22:31:13.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.284+0000 7faebe0f6700 1 -- 172.21.15.16:0/3713862893 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faea80094a0 con 0x7faeb806f680 2023-09-30T22:31:13.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.284+0000 7fae9ffff700 1 -- 172.21.15.16:0/3713862893 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea80038e0 con 0x7faeb806f680 2023-09-30T22:31:13.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.284+0000 7faebe0f6700 1 -- 172.21.15.16:0/3713862893 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faeb811fb70 con 0x7faeb806f680 2023-09-30T22:31:13.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.284+0000 7fae9ffff700 1 -- 172.21.15.16:0/3713862893 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7faea8003a40 con 0x7faeb806f680 2023-09-30T22:31:13.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.285+0000 7fae9ffff700 1 -- 172.21.15.16:0/3713862893 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7faea8040750 con 0x7faeb806f680 2023-09-30T22:31:13.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.285+0000 7faebe0f6700 1 -- 172.21.15.16:0/3713862893 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faeb8074ea0 con 0x7faeb806f680 2023-09-30T22:31:13.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.290+0000 7fae9ffff700 1 -- 172.21.15.16:0/3713862893 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7faea8015080 con 0x7faeb806f680 2023-09-30T22:31:13.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.319+0000 7f2f94536700 1 Processor -- start 2023-09-30T22:31:13.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.319+0000 7f2f94536700 1 -- start start 2023-09-30T22:31:13.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.320+0000 7f2f94536700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f8c133490 0x7f2f8c135870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:13.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.320+0000 7f2f94536700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f8c135db0 0x7f2f8c138190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:13.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.320+0000 7f2f94536700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f8c09c190 con 0x7f2f8c130750 2023-09-30T22:31:13.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.320+0000 7f2f94536700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8c09b090 con 0x7f2f8c133490 2023-09-30T22:31:13.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.320+0000 7f2f94536700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8c09a6c0 con 0x7f2f8c135db0 2023-09-30T22:31:13.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.320+0000 7f2f922d2700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2f8c130750 0x7f2f8c130b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47412/0 (socket says 172.21.15.16:47412) 2023-09-30T22:31:13.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.320+0000 7f2f922d2700 1 -- 172.21.15.16:0/2250066090 learned_addr learned my addr 172.21.15.16:0/2250066090 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:31:13.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.320+0000 7f2f91ad1700 1 --2- 172.21.15.16:0/2250066090 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f8c133490 0x7f2f8c135870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:13.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.320+0000 7f2f92ad3700 1 --2- 172.21.15.16:0/2250066090 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f8c135db0 0x7f2f8c138190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:13.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.321+0000 7f2f912d0700 1 -- 172.21.15.16:0/2250066090 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1845401272 0 0) 0x7f2f8c09c190 con 0x7f2f8c130750 2023-09-30T22:31:13.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.321+0000 7f2f912d0700 1 -- 172.21.15.16:0/2250066090 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f780035b0 con 0x7f2f8c130750 2023-09-30T22:31:13.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.321+0000 7f2f92ad3700 1 -- 172.21.15.16:0/2250066090 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f8c133490 msgr2=0x7f2f8c135870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:13.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.321+0000 7f2f92ad3700 1 --2- 172.21.15.16:0/2250066090 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f8c133490 0x7f2f8c135870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.321+0000 7f2f92ad3700 1 -- 172.21.15.16:0/2250066090 >> v1:172.21.15.16:6789/0 conn(0x7f2f8c130750 legacy=0x7f2f8c130b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:13.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.321+0000 7f2f92ad3700 1 -- 172.21.15.16:0/2250066090 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f8c130300 con 0x7f2f8c135db0 2023-09-30T22:31:13.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.321+0000 7f2f91ad1700 1 --2- 172.21.15.16:0/2250066090 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f8c133490 0x7f2f8c135870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:13.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.321+0000 7f2f92ad3700 1 --2- 172.21.15.16:0/2250066090 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f8c135db0 0x7f2f8c138190 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2f84009dc0 tx=0x7f2f8400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=408b6519fd3c1b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:13.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.321+0000 7f2f912d0700 1 -- 172.21.15.16:0/2250066090 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f8400e070 con 0x7f2f8c135db0 2023-09-30T22:31:13.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.321+0000 7f2f912d0700 1 -- 172.21.15.16:0/2250066090 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f8400bb20 con 0x7f2f8c135db0 2023-09-30T22:31:13.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.321+0000 7f2f912d0700 1 -- 172.21.15.16:0/2250066090 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f84012430 con 0x7f2f8c135db0 2023-09-30T22:31:13.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.322+0000 7f2f94536700 1 -- 172.21.15.16:0/2250066090 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f8c135db0 msgr2=0x7f2f8c138190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:13.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.322+0000 7f2f94536700 1 --2- 172.21.15.16:0/2250066090 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f8c135db0 0x7f2f8c138190 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2f84009dc0 tx=0x7f2f8400b330 comp rx=0 tx=0).stop 2023-09-30T22:31:13.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.322+0000 7f2f94536700 1 -- 172.21.15.16:0/2250066090 shutdown_connections 2023-09-30T22:31:13.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.322+0000 7f2f94536700 1 --2- 172.21.15.16:0/2250066090 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f8c135db0 0x7f2f8c138190 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.322+0000 7f2f94536700 1 --2- 172.21.15.16:0/2250066090 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f8c133490 0x7f2f8c135870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.322+0000 7f2f94536700 1 -- 172.21.15.16:0/2250066090 >> 172.21.15.16:0/2250066090 conn(0x7f2f8c09f470 msgr2=0x7f2f8c0a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:13.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.322+0000 7f2f94536700 1 -- 172.21.15.16:0/2250066090 shutdown_connections 2023-09-30T22:31:13.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.322+0000 7f2f94536700 1 -- 172.21.15.16:0/2250066090 wait complete. 2023-09-30T22:31:13.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.323+0000 7f2f94536700 1 Processor -- start 2023-09-30T22:31:13.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.323+0000 7f2f94536700 1 -- start start 2023-09-30T22:31:13.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.324+0000 7f2f94536700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f8c130750 0x7f2f8c1c81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:13.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.324+0000 7f2f94536700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f8c133490 0x7f2f8c1c8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:13.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.324+0000 7f2f94536700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f8c135db0 0x7f2f8c1c8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:13.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.324+0000 7f2f94536700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8c12b5a0 con 0x7f2f8c130750 2023-09-30T22:31:13.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.324+0000 7f2f94536700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8c12efc0 con 0x7f2f8c133490 2023-09-30T22:31:13.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.324+0000 7f2f94536700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8c0a1f20 con 0x7f2f8c135db0 2023-09-30T22:31:13.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.324+0000 7f2f91ad1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f8c133490 0x7f2f8c1c8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:13.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.324+0000 7f2f91ad1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f8c133490 0x7f2f8c1c8720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:35300/0 (socket says 172.21.15.16:35300) 2023-09-30T22:31:13.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.324+0000 7f2f91ad1700 1 -- 172.21.15.16:0/1117004066 learned_addr learned my addr 172.21.15.16:0/1117004066 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:13.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.324+0000 7f2f922d2700 1 --2- 172.21.15.16:0/1117004066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f8c130750 0x7f2f8c1c81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:13.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.324+0000 7f2f92ad3700 1 --2- 172.21.15.16:0/1117004066 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f8c135db0 0x7f2f8c1c8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:13.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.324+0000 7f2f91ad1700 1 -- 172.21.15.16:0/1117004066 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f8c135db0 msgr2=0x7f2f8c1c8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:13.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.324+0000 7f2f91ad1700 1 --2- 172.21.15.16:0/1117004066 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f8c135db0 0x7f2f8c1c8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.324+0000 7f2f91ad1700 1 -- 172.21.15.16:0/1117004066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f8c130750 msgr2=0x7f2f8c1c81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:13.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.324+0000 7f2f91ad1700 1 --2- 172.21.15.16:0/1117004066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f8c130750 0x7f2f8c1c81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.324+0000 7f2f91ad1700 1 -- 172.21.15.16:0/1117004066 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f7c0094a0 con 0x7f2f8c133490 2023-09-30T22:31:13.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.324+0000 7f2f922d2700 1 --2- 172.21.15.16:0/1117004066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f8c130750 0x7f2f8c1c81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:13.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.325+0000 7f2f91ad1700 1 --2- 172.21.15.16:0/1117004066 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f8c133490 0x7f2f8c1c8720 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c00ebe0 tx=0x7f2f7c00eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:13.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.325+0000 7f2f82ffd700 1 -- 172.21.15.16:0/1117004066 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c00c850 con 0x7f2f8c133490 2023-09-30T22:31:13.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.325+0000 7f2f82ffd700 1 -- 172.21.15.16:0/1117004066 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f7c00c9b0 con 0x7f2f8c133490 2023-09-30T22:31:13.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.325+0000 7f2f94536700 1 -- 172.21.15.16:0/1117004066 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f84009a70 con 0x7f2f8c133490 2023-09-30T22:31:13.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.325+0000 7f2f82ffd700 1 -- 172.21.15.16:0/1117004066 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f7c00cd40 con 0x7f2f8c133490 2023-09-30T22:31:13.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.325+0000 7f2f94536700 1 -- 172.21.15.16:0/1117004066 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f8c12f670 con 0x7f2f8c133490 2023-09-30T22:31:13.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.326+0000 7f2f94536700 1 -- 172.21.15.16:0/1117004066 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f8c0747f0 con 0x7f2f8c133490 2023-09-30T22:31:13.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.327+0000 7f2f82ffd700 1 -- 172.21.15.16:0/1117004066 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2f7c013470 con 0x7f2f8c133490 2023-09-30T22:31:13.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.327+0000 7f2f82ffd700 1 -- 172.21.15.16:0/1117004066 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f2f7c03ffe0 con 0x7f2f8c133490 2023-09-30T22:31:13.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.331+0000 7f2f82ffd700 1 -- 172.21.15.16:0/1117004066 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2f7c013720 con 0x7f2f8c133490 2023-09-30T22:31:13.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.366+0000 7fc5a4199700 1 Processor -- start 2023-09-30T22:31:13.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.366+0000 7fc5a4199700 1 -- start start 2023-09-30T22:31:13.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.367+0000 7fc5a4199700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc59c1338c0 0x7fc59c133c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:13.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.367+0000 7fc5a4199700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc59c1341d0 0x7fc59c09de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:13.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.367+0000 7fc5a4199700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc59c06f720 con 0x7fc59c09e390 2023-09-30T22:31:13.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.367+0000 7fc5a4199700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc59c070c80 con 0x7fc59c1338c0 2023-09-30T22:31:13.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.367+0000 7fc5a4199700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc59c06fce0 con 0x7fc59c1341d0 2023-09-30T22:31:13.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.367+0000 7fc5a1f35700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc59c1338c0 0x7fc59c133c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:13.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.367+0000 7fc5a2736700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc59c09e390 0x7fc59c13a850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47426/0 (socket says 172.21.15.16:47426) 2023-09-30T22:31:13.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.367+0000 7fc5a2736700 1 -- 172.21.15.16:0/3124608366 learned_addr learned my addr 172.21.15.16:0/3124608366 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:31:13.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.368+0000 7fc5a1734700 1 --2- 172.21.15.16:0/3124608366 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc59c1341d0 0x7fc59c09de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:13.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.368+0000 7fc5a1f35700 1 -- 172.21.15.16:0/3124608366 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc59c1341d0 msgr2=0x7fc59c09de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:13.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.368+0000 7fc5a1f35700 1 --2- 172.21.15.16:0/3124608366 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc59c1341d0 0x7fc59c09de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.368+0000 7fc5a1f35700 1 -- 172.21.15.16:0/3124608366 >> v1:172.21.15.16:6789/0 conn(0x7fc59c09e390 legacy=0x7fc59c13a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:13.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.368+0000 7fc5a1f35700 1 -- 172.21.15.16:0/3124608366 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc59c1336a0 con 0x7fc59c1338c0 2023-09-30T22:31:13.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.368+0000 7fc5a0f33700 1 -- 172.21.15.16:0/3124608366 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 561232874 0 0) 0x7fc59c06f720 con 0x7fc59c09e390 2023-09-30T22:31:13.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.368+0000 7fc5a1f35700 1 --2- 172.21.15.16:0/3124608366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc59c1338c0 0x7fc59c133c90 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc598009840 tx=0x7fc598009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d57ea77586642fa server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:13.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.368+0000 7fc5a0f33700 1 -- 172.21.15.16:0/3124608366 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc598011830 con 0x7fc59c1338c0 2023-09-30T22:31:13.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.368+0000 7fc5a0f33700 1 -- 172.21.15.16:0/3124608366 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc598011990 con 0x7fc59c1338c0 2023-09-30T22:31:13.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.368+0000 7fc5a0f33700 1 -- 172.21.15.16:0/3124608366 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc598011d20 con 0x7fc59c1338c0 2023-09-30T22:31:13.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.369+0000 7fc5a4199700 1 -- 172.21.15.16:0/3124608366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc59c1338c0 msgr2=0x7fc59c133c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:13.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.369+0000 7fc5a4199700 1 --2- 172.21.15.16:0/3124608366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc59c1338c0 0x7fc59c133c90 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc598009840 tx=0x7fc598009b50 comp rx=0 tx=0).stop 2023-09-30T22:31:13.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.369+0000 7fc5a4199700 1 -- 172.21.15.16:0/3124608366 shutdown_connections 2023-09-30T22:31:13.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.369+0000 7fc5a4199700 1 --2- 172.21.15.16:0/3124608366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc59c1338c0 0x7fc59c133c90 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.369+0000 7fc5a4199700 1 --2- 172.21.15.16:0/3124608366 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc59c1341d0 0x7fc59c09de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.369+0000 7fc5a4199700 1 -- 172.21.15.16:0/3124608366 >> 172.21.15.16:0/3124608366 conn(0x7fc59c0a2aa0 msgr2=0x7fc59c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:13.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.369+0000 7fc5a4199700 1 -- 172.21.15.16:0/3124608366 shutdown_connections 2023-09-30T22:31:13.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.369+0000 7fc5a4199700 1 -- 172.21.15.16:0/3124608366 wait complete. 2023-09-30T22:31:13.373 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:13.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.370+0000 7fc5a4199700 1 Processor -- start 2023-09-30T22:31:13.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.370+0000 7fc5a4199700 1 -- start start 2023-09-30T22:31:13.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.371+0000 7fc5a4199700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc59c09e390 0x7fc59c11a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:13.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.371+0000 7fc5a4199700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc59c1338c0 0x7fc59c11a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:13.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.371+0000 7fc5a4199700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc59c1341d0 0x7fc59c11af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:13.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.371+0000 7fc5a4199700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc59c11e970 con 0x7fc59c1341d0 2023-09-30T22:31:13.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.371+0000 7fc5a4199700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc59c11dfb0 con 0x7fc59c09e390 2023-09-30T22:31:13.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.371+0000 7fc5a4199700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc59c11d360 con 0x7fc59c1338c0 2023-09-30T22:31:13.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.371+0000 7fc5a1734700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc59c1338c0 0x7fc59c11a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:13.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.371+0000 7fc5a1734700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc59c1338c0 0x7fc59c11a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:59946/0 (socket says 172.21.15.16:59946) 2023-09-30T22:31:13.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.371+0000 7fc5a1734700 1 -- 172.21.15.16:0/3845384028 learned_addr learned my addr 172.21.15.16:0/3845384028 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:13.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.371+0000 7fc5a1f35700 1 --2- 172.21.15.16:0/3845384028 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc59c09e390 0x7fc59c11a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:13.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.371+0000 7fc5a2736700 1 --2- 172.21.15.16:0/3845384028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc59c1341d0 0x7fc59c11af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:13.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.371+0000 7fc5a1734700 1 -- 172.21.15.16:0/3845384028 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc59c09e390 msgr2=0x7fc59c11a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:13.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.371+0000 7fc5a1734700 1 --2- 172.21.15.16:0/3845384028 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc59c09e390 0x7fc59c11a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.371+0000 7fc5a1734700 1 -- 172.21.15.16:0/3845384028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc59c1341d0 msgr2=0x7fc59c11af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:13.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.371+0000 7fc5a1734700 1 --2- 172.21.15.16:0/3845384028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc59c1341d0 0x7fc59c11af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.371+0000 7fc5a1734700 1 -- 172.21.15.16:0/3845384028 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5980094a0 con 0x7fc59c1338c0 2023-09-30T22:31:13.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.371+0000 7fc5a1f35700 1 --2- 172.21.15.16:0/3845384028 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc59c09e390 0x7fc59c11a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:13.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.371+0000 7fc5a2736700 1 --2- 172.21.15.16:0/3845384028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc59c1341d0 0x7fc59c11af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:31:13.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.371+0000 7fc5a1734700 1 --2- 172.21.15.16:0/3845384028 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc59c1338c0 0x7fc59c11a9d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc58c00e9b0 tx=0x7fc58c00ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:13.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.372+0000 7fc5927fc700 1 -- 172.21.15.16:0/3845384028 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc58c009710 con 0x7fc59c1338c0 2023-09-30T22:31:13.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.372+0000 7fc5927fc700 1 -- 172.21.15.16:0/3845384028 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc58c00c850 con 0x7fc59c1338c0 2023-09-30T22:31:13.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.372+0000 7fc5a4199700 1 -- 172.21.15.16:0/3845384028 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc59c11c140 con 0x7fc59c1338c0 2023-09-30T22:31:13.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.372+0000 7fc5927fc700 1 -- 172.21.15.16:0/3845384028 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc58c00cbe0 con 0x7fc59c1338c0 2023-09-30T22:31:13.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.372+0000 7fc5a4199700 1 -- 172.21.15.16:0/3845384028 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc59c11cea0 con 0x7fc59c1338c0 2023-09-30T22:31:13.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.373+0000 7fc5927fc700 1 -- 172.21.15.16:0/3845384028 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc58c00cd40 con 0x7fc59c1338c0 2023-09-30T22:31:13.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.373+0000 7fc5927fc700 1 -- 172.21.15.16:0/3845384028 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fc58c03fb10 con 0x7fc59c1338c0 2023-09-30T22:31:13.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.373+0000 7fc5a4199700 1 -- 172.21.15.16:0/3845384028 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc59c074ea0 con 0x7fc59c1338c0 2023-09-30T22:31:13.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.378+0000 7fc5927fc700 1 -- 172.21.15.16:0/3845384028 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc58c011800 con 0x7fc59c1338c0 2023-09-30T22:31:13.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.510+0000 7faebe0f6700 1 -- 172.21.15.16:0/3713862893 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7faeb8072210 con 0x7faeb806f680 2023-09-30T22:31:13.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.511+0000 7fae9ffff700 1 -- 172.21.15.16:0/3713862893 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7faea802e970 con 0x7faeb806f680 2023-09-30T22:31:13.512 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:31:13.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.514+0000 7f2d012df700 1 Processor -- start 2023-09-30T22:31:13.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.514+0000 7faebe0f6700 1 -- 172.21.15.16:0/3713862893 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faeb806f680 msgr2=0x7faeb811c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:13.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.514+0000 7faebe0f6700 1 --2- 172.21.15.16:0/3713862893 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faeb806f680 0x7faeb811c8c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7faea8009870 tx=0x7faea8004750 comp rx=0 tx=0).stop 2023-09-30T22:31:13.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.514+0000 7f2d012df700 1 -- start start 2023-09-30T22:31:13.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.514+0000 7f2d012df700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2cfc06e570 0x7f2cfc06e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:13.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.514+0000 7f2d012df700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2cfc09eca0 0x7f2cfc06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:13.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.514+0000 7f2d012df700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2cfc134610 0x7f2cfc134a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:13.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.514+0000 7f2d012df700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cfc070ee0 con 0x7f2cfc09eca0 2023-09-30T22:31:13.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.514+0000 7f2d012df700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cfc071480 con 0x7f2cfc06e570 2023-09-30T22:31:13.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.514+0000 7f2d012df700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2cfc06f3c0 con 0x7f2cfc134610 2023-09-30T22:31:13.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.514+0000 7f2cfaffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2cfc06e570 0x7f2cfc06e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:13.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.514+0000 7f2cfaffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2cfc06e570 0x7f2cfc06e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:35324/0 (socket says 172.21.15.16:35324) 2023-09-30T22:31:13.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.514+0000 7f2cfaffd700 1 -- 172.21.15.16:0/75090975 learned_addr learned my addr 172.21.15.16:0/75090975 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:13.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.514+0000 7f2cfb7fe700 1 --2- 172.21.15.16:0/75090975 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2cfc134610 0x7f2cfc134a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:13.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.514+0000 7f2cfa7fc700 1 --2- 172.21.15.16:0/75090975 >> v2:172.21.15.16:3300/0 conn(0x7f2cfc09eca0 0x7f2cfc06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:13.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.515+0000 7faebe0f6700 1 -- 172.21.15.16:0/3713862893 shutdown_connections 2023-09-30T22:31:13.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.515+0000 7faebe0f6700 1 --2- 172.21.15.16:0/3713862893 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faeb806f680 0x7faeb811c8c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.515+0000 7faebe0f6700 1 --2- 172.21.15.16:0/3713862893 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faeb8130670 0x7faeb811d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.515+0000 7faebe0f6700 1 --2- 172.21.15.16:0/3713862893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faeb80701a0 0x7faeb811ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.515+0000 7faebe0f6700 1 -- 172.21.15.16:0/3713862893 >> 172.21.15.16:0/3713862893 conn(0x7faeb80a2aa0 msgr2=0x7faeb812efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:13.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.515+0000 7f2cfaffd700 1 -- 172.21.15.16:0/75090975 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2cfc134610 msgr2=0x7f2cfc134a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:13.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.515+0000 7f2cfaffd700 1 --2- 172.21.15.16:0/75090975 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2cfc134610 0x7f2cfc134a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.515+0000 7f2cfaffd700 1 -- 172.21.15.16:0/75090975 >> v2:172.21.15.16:3300/0 conn(0x7f2cfc09eca0 msgr2=0x7f2cfc06ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:13.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.515+0000 7f2cfaffd700 1 --2- 172.21.15.16:0/75090975 >> v2:172.21.15.16:3300/0 conn(0x7f2cfc09eca0 0x7f2cfc06ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.515+0000 7f2cfaffd700 1 -- 172.21.15.16:0/75090975 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cfc071b90 con 0x7f2cfc06e570 2023-09-30T22:31:13.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.515+0000 7f2cfa7fc700 1 --2- 172.21.15.16:0/75090975 >> v2:172.21.15.16:3300/0 conn(0x7f2cfc09eca0 0x7f2cfc06ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:13.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.515+0000 7f2cfaffd700 1 --2- 172.21.15.16:0/75090975 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2cfc06e570 0x7f2cfc06e940 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2cec0098f0 tx=0x7f2cec009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d48ea51a6fbbf56 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:13.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.515+0000 7faebe0f6700 1 -- 172.21.15.16:0/3713862893 shutdown_connections 2023-09-30T22:31:13.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.515+0000 7faebe0f6700 1 -- 172.21.15.16:0/3713862893 wait complete. 2023-09-30T22:31:13.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.515+0000 7f2cf9ffb700 1 -- 172.21.15.16:0/75090975 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cec011830 con 0x7f2cfc06e570 2023-09-30T22:31:13.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.515+0000 7f2cf9ffb700 1 -- 172.21.15.16:0/75090975 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cec011990 con 0x7f2cfc06e570 2023-09-30T22:31:13.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.515+0000 7f2cf9ffb700 1 -- 172.21.15.16:0/75090975 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cec011d20 con 0x7f2cfc06e570 2023-09-30T22:31:13.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.516+0000 7f2d012df700 1 -- 172.21.15.16:0/75090975 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2cfc06e570 msgr2=0x7f2cfc06e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:13.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.516+0000 7f2d012df700 1 --2- 172.21.15.16:0/75090975 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2cfc06e570 0x7f2cfc06e940 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2cec0098f0 tx=0x7f2cec009c00 comp rx=0 tx=0).stop 2023-09-30T22:31:13.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.516+0000 7f2d012df700 1 -- 172.21.15.16:0/75090975 shutdown_connections 2023-09-30T22:31:13.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.516+0000 7f2d012df700 1 --2- 172.21.15.16:0/75090975 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2cfc06e570 0x7f2cfc06e940 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.516+0000 7f2d012df700 1 --2- 172.21.15.16:0/75090975 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2cfc134610 0x7f2cfc134a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.516+0000 7f2d012df700 1 --2- 172.21.15.16:0/75090975 >> v2:172.21.15.16:3300/0 conn(0x7f2cfc09eca0 0x7f2cfc06ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.516+0000 7f2d012df700 1 -- 172.21.15.16:0/75090975 >> 172.21.15.16:0/75090975 conn(0x7f2cfc0a2aa0 msgr2=0x7f2cfc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:13.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.517+0000 7f2d012df700 1 -- 172.21.15.16:0/75090975 shutdown_connections 2023-09-30T22:31:13.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.517+0000 7f2d012df700 1 -- 172.21.15.16:0/75090975 wait complete. 2023-09-30T22:31:13.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.518+0000 7f2d012df700 1 Processor -- start 2023-09-30T22:31:13.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.518+0000 7f2d012df700 1 -- start start 2023-09-30T22:31:13.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.518+0000 7f2d012df700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2cfc06e570 0x7f2cfc1273f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:13.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.518+0000 7f2d012df700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2cfc134610 0x7f2cfc139970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:13.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.518+0000 7f2d012df700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cfc127930 0x7f2cfc139eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:13.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.518+0000 7f2d012df700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cfc13abb0 con 0x7f2cfc127930 2023-09-30T22:31:13.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.518+0000 7f2d012df700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cfc13ce50 con 0x7f2cfc134610 2023-09-30T22:31:13.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.518+0000 7f2d012df700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2cfc13d110 con 0x7f2cfc06e570 2023-09-30T22:31:13.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.519+0000 7f2cfa7fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2cfc134610 0x7f2cfc139970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:13.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.519+0000 7f2cfaffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2cfc06e570 0x7f2cfc1273f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:13.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.519+0000 7f2cfa7fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2cfc134610 0x7f2cfc139970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:35328/0 (socket says 172.21.15.16:35328) 2023-09-30T22:31:13.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.519+0000 7f2cfa7fc700 1 -- 172.21.15.16:0/3967698406 learned_addr learned my addr 172.21.15.16:0/3967698406 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:13.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.519+0000 7f2cfb7fe700 1 --2- 172.21.15.16:0/3967698406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cfc127930 0x7f2cfc139eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:13.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.519+0000 7f2cfaffd700 1 -- 172.21.15.16:0/3967698406 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2cfc134610 msgr2=0x7f2cfc139970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:13.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.519+0000 7f2cfaffd700 1 --2- 172.21.15.16:0/3967698406 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2cfc134610 0x7f2cfc139970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.519+0000 7f2cfaffd700 1 -- 172.21.15.16:0/3967698406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cfc127930 msgr2=0x7f2cfc139eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:13.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.519+0000 7f2cfaffd700 1 --2- 172.21.15.16:0/3967698406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cfc127930 0x7f2cfc139eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.519+0000 7f2cfaffd700 1 -- 172.21.15.16:0/3967698406 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf00094a0 con 0x7f2cfc06e570 2023-09-30T22:31:13.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.519+0000 7f2cfaffd700 1 --2- 172.21.15.16:0/3967698406 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2cfc06e570 0x7f2cfc1273f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2cec016000 tx=0x7f2cec004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:13.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.520+0000 7f2ce37fe700 1 -- 172.21.15.16:0/3967698406 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cec004240 con 0x7f2cfc06e570 2023-09-30T22:31:13.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.520+0000 7f2ce37fe700 1 -- 172.21.15.16:0/3967698406 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cec01f070 con 0x7f2cfc06e570 2023-09-30T22:31:13.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.520+0000 7f2d012df700 1 -- 172.21.15.16:0/3967698406 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cec009550 con 0x7f2cfc06e570 2023-09-30T22:31:13.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.520+0000 7f2ce37fe700 1 -- 172.21.15.16:0/3967698406 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cec0036a0 con 0x7f2cfc06e570 2023-09-30T22:31:13.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.520+0000 7f2d012df700 1 -- 172.21.15.16:0/3967698406 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cfc13b4e0 con 0x7f2cfc06e570 2023-09-30T22:31:13.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.521+0000 7f2ce37fe700 1 -- 172.21.15.16:0/3967698406 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2cec021030 con 0x7f2cfc06e570 2023-09-30T22:31:13.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.521+0000 7f2ce37fe700 1 -- 172.21.15.16:0/3967698406 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f2cec040760 con 0x7f2cfc06e570 2023-09-30T22:31:13.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.521+0000 7f2d012df700 1 -- 172.21.15.16:0/3967698406 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cfc13b6d0 con 0x7f2cfc06e570 2023-09-30T22:31:13.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.530+0000 7f2ce37fe700 1 -- 172.21.15.16:0/3967698406 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2cec015020 con 0x7f2cfc06e570 2023-09-30T22:31:13.534 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:31:13.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.545+0000 7f2f94536700 1 -- 172.21.15.16:0/1117004066 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2f8c071f30 con 0x7f2f8c133490 2023-09-30T22:31:13.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.545+0000 7f2f82ffd700 1 -- 172.21.15.16:0/1117004066 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2f7c017070 con 0x7f2f8c133490 2023-09-30T22:31:13.547 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:31:13.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.548+0000 7f2f94536700 1 -- 172.21.15.16:0/1117004066 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f8c133490 msgr2=0x7f2f8c1c8720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:13.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.548+0000 7f2f94536700 1 --2- 172.21.15.16:0/1117004066 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f8c133490 0x7f2f8c1c8720 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c00ebe0 tx=0x7f2f7c00eef0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.548+0000 7f2f94536700 1 -- 172.21.15.16:0/1117004066 shutdown_connections 2023-09-30T22:31:13.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.548+0000 7f2f94536700 1 --2- 172.21.15.16:0/1117004066 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f8c135db0 0x7f2f8c1c8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.548+0000 7f2f94536700 1 --2- 172.21.15.16:0/1117004066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f8c130750 0x7f2f8c1c81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.548+0000 7f2f94536700 1 --2- 172.21.15.16:0/1117004066 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f8c133490 0x7f2f8c1c8720 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.549+0000 7f2f94536700 1 -- 172.21.15.16:0/1117004066 >> 172.21.15.16:0/1117004066 conn(0x7f2f8c09f470 msgr2=0x7f2f8c09cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:13.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.549+0000 7f2f94536700 1 -- 172.21.15.16:0/1117004066 shutdown_connections 2023-09-30T22:31:13.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.549+0000 7f2f94536700 1 -- 172.21.15.16:0/1117004066 wait complete. 2023-09-30T22:31:13.569 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:31:13.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.603+0000 7fc5a4199700 1 -- 172.21.15.16:0/3845384028 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc59c06b9e0 con 0x7fc59c1338c0 2023-09-30T22:31:13.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.603+0000 7fc5927fc700 1 -- 172.21.15.16:0/3845384028 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc58c017020 con 0x7fc59c1338c0 2023-09-30T22:31:13.605 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:31:13.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.606+0000 7fc5a4199700 1 -- 172.21.15.16:0/3845384028 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc59c1338c0 msgr2=0x7fc59c11a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:13.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.606+0000 7fc5a4199700 1 --2- 172.21.15.16:0/3845384028 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc59c1338c0 0x7fc59c11a9d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc58c00e9b0 tx=0x7fc58c00ecc0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.606+0000 7fc5a4199700 1 -- 172.21.15.16:0/3845384028 shutdown_connections 2023-09-30T22:31:13.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.606+0000 7fc5a4199700 1 --2- 172.21.15.16:0/3845384028 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc59c09e390 0x7fc59c11a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.606+0000 7fc5a4199700 1 --2- 172.21.15.16:0/3845384028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc59c1341d0 0x7fc59c11af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.606+0000 7fc5a4199700 1 --2- 172.21.15.16:0/3845384028 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc59c1338c0 0x7fc59c11a9d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.606+0000 7fc5a4199700 1 -- 172.21.15.16:0/3845384028 >> 172.21.15.16:0/3845384028 conn(0x7fc59c0a2aa0 msgr2=0x7fc59c12f570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:13.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.606+0000 7fc5a4199700 1 -- 172.21.15.16:0/3845384028 shutdown_connections 2023-09-30T22:31:13.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.606+0000 7fc5a4199700 1 -- 172.21.15.16:0/3845384028 wait complete. 2023-09-30T22:31:13.621 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:31:13.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.737+0000 7f2d012df700 1 -- 172.21.15.16:0/3967698406 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2cfc13d2d0 con 0x7f2cfc06e570 2023-09-30T22:31:13.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.737+0000 7f2ce37fe700 1 -- 172.21.15.16:0/3967698406 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2cec0239c0 con 0x7f2cfc06e570 2023-09-30T22:31:13.739 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:31:13.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.740+0000 7f2d012df700 1 -- 172.21.15.16:0/3967698406 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2cfc06e570 msgr2=0x7f2cfc1273f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:13.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.740+0000 7f2d012df700 1 --2- 172.21.15.16:0/3967698406 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2cfc06e570 0x7f2cfc1273f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f2cec016000 tx=0x7f2cec004750 comp rx=0 tx=0).stop 2023-09-30T22:31:13.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.740+0000 7f2d012df700 1 -- 172.21.15.16:0/3967698406 shutdown_connections 2023-09-30T22:31:13.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.740+0000 7f2d012df700 1 --2- 172.21.15.16:0/3967698406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cfc127930 0x7f2cfc139eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.740+0000 7f2d012df700 1 --2- 172.21.15.16:0/3967698406 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2cfc06e570 0x7f2cfc1273f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.740+0000 7f2d012df700 1 --2- 172.21.15.16:0/3967698406 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2cfc134610 0x7f2cfc139970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:13.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.740+0000 7f2d012df700 1 -- 172.21.15.16:0/3967698406 >> 172.21.15.16:0/3967698406 conn(0x7f2cfc0a2aa0 msgr2=0x7f2cfc099a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:13.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.740+0000 7f2d012df700 1 -- 172.21.15.16:0/3967698406 shutdown_connections 2023-09-30T22:31:13.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:13.740+0000 7f2d012df700 1 -- 172.21.15.16:0/3967698406 wait complete. 2023-09-30T22:31:13.755 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:31:14.536 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:14.570 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:14.622 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:14.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.690+0000 7f8c7eedd700 1 Processor -- start 2023-09-30T22:31:14.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.690+0000 7f8c7eedd700 1 -- start start 2023-09-30T22:31:14.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.691+0000 7f8c7eedd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8c7809e930 0x7f8c7809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:14.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.691+0000 7f8c7eedd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8c7809f240 0x7f8c781317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:14.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.691+0000 7f8c7eedd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8c7809f650 0x7f8c78131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:14.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.691+0000 7f8c7eedd700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c78070940 con 0x7f8c7809f240 2023-09-30T22:31:14.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.691+0000 7f8c7eedd700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7806fd60 con 0x7f8c7809e930 2023-09-30T22:31:14.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.691+0000 7f8c7eedd700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7806fa80 con 0x7f8c7809f650 2023-09-30T22:31:14.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.691+0000 7f8c77fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8c7809f240 0x7f8c781317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:14.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.691+0000 7f8c77fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8c7809f240 0x7f8c781317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46078/0 (socket says 172.21.15.16:46078) 2023-09-30T22:31:14.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.691+0000 7f8c77fff700 1 -- 172.21.15.16:0/2683507581 learned_addr learned my addr 172.21.15.16:0/2683507581 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:14.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.691+0000 7f8c7d47a700 1 --2- 172.21.15.16:0/2683507581 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8c7809f650 0x7f8c78131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:14.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.691+0000 7f8c7cc79700 1 --2- 172.21.15.16:0/2683507581 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8c7809e930 0x7f8c7809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:14.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.691+0000 7f8c77fff700 1 -- 172.21.15.16:0/2683507581 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8c7809f650 msgr2=0x7f8c78131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:14.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.691+0000 7f8c77fff700 1 --2- 172.21.15.16:0/2683507581 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8c7809f650 0x7f8c78131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:14.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.691+0000 7f8c77fff700 1 -- 172.21.15.16:0/2683507581 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8c7809e930 msgr2=0x7f8c7809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:14.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.691+0000 7f8c77fff700 1 --2- 172.21.15.16:0/2683507581 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8c7809e930 0x7f8c7809ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:14.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.691+0000 7f8c77fff700 1 -- 172.21.15.16:0/2683507581 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c7809fb60 con 0x7f8c7809f240 2023-09-30T22:31:14.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.691+0000 7f8c7d47a700 1 --2- 172.21.15.16:0/2683507581 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8c7809f650 0x7f8c78131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:14.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.692+0000 7f8c77fff700 1 --2- 172.21.15.16:0/2683507581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c7809f240 0x7f8c781317f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f8c640098f0 tx=0x7f8c64009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f785b752f97a4706 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:14.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.692+0000 7f8c777fe700 1 -- 172.21.15.16:0/2683507581 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c64011830 con 0x7f8c7809f240 2023-09-30T22:31:14.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.692+0000 7f8c777fe700 1 -- 172.21.15.16:0/2683507581 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c64011990 con 0x7f8c7809f240 2023-09-30T22:31:14.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.692+0000 7f8c777fe700 1 -- 172.21.15.16:0/2683507581 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c64011d20 con 0x7f8c7809f240 2023-09-30T22:31:14.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.692+0000 7f8c7eedd700 1 -- 172.21.15.16:0/2683507581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c7809f240 msgr2=0x7f8c781317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:14.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.692+0000 7f8c7eedd700 1 --2- 172.21.15.16:0/2683507581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c7809f240 0x7f8c781317f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f8c640098f0 tx=0x7f8c64009c00 comp rx=0 tx=0).stop 2023-09-30T22:31:14.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.693+0000 7f8c7eedd700 1 -- 172.21.15.16:0/2683507581 shutdown_connections 2023-09-30T22:31:14.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.693+0000 7f8c7eedd700 1 --2- 172.21.15.16:0/2683507581 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8c7809e930 0x7f8c7809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:14.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.693+0000 7f8c7eedd700 1 --2- 172.21.15.16:0/2683507581 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8c7809f650 0x7f8c78131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:14.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.693+0000 7f8c7eedd700 1 --2- 172.21.15.16:0/2683507581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c7809f240 0x7f8c781317f0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:14.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.693+0000 7f8c7eedd700 1 -- 172.21.15.16:0/2683507581 >> 172.21.15.16:0/2683507581 conn(0x7f8c780a4210 msgr2=0x7f8c780a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:14.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.693+0000 7f8c7eedd700 1 -- 172.21.15.16:0/2683507581 shutdown_connections 2023-09-30T22:31:14.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.693+0000 7f8c7eedd700 1 -- 172.21.15.16:0/2683507581 wait complete. 2023-09-30T22:31:14.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.694+0000 7f8c7eedd700 1 Processor -- start 2023-09-30T22:31:14.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.694+0000 7f8c7eedd700 1 -- start start 2023-09-30T22:31:14.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.694+0000 7f8c7eedd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8c7809e930 0x7f8c7806b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:14.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.694+0000 7f8c7eedd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c7809f240 0x7f8c7806bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:14.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.694+0000 7f8c7eedd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8c7809f650 0x7f8c7806c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:14.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.694+0000 7f8c7eedd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c781b72b0 con 0x7f8c7809f240 2023-09-30T22:31:14.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.694+0000 7f8c7eedd700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c781a9460 con 0x7f8c7809e930 2023-09-30T22:31:14.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.694+0000 7f8c7eedd700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8c781a8980 con 0x7f8c7809f650 2023-09-30T22:31:14.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.695+0000 7f8c7d47a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8c7809f650 0x7f8c7806c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:14.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.695+0000 7f8c77fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c7809f240 0x7f8c7806bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:14.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.695+0000 7f8c7d47a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8c7809f650 0x7f8c7806c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:59982/0 (socket says 172.21.15.16:59982) 2023-09-30T22:31:14.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.695+0000 7f8c77fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c7809f240 0x7f8c7806bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46094/0 (socket says 172.21.15.16:46094) 2023-09-30T22:31:14.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.695+0000 7f8c7d47a700 1 -- 172.21.15.16:0/664596051 learned_addr learned my addr 172.21.15.16:0/664596051 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:14.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.695+0000 7f8c7cc79700 1 --2- 172.21.15.16:0/664596051 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8c7809e930 0x7f8c7806b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:14.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.695+0000 7f8c77fff700 1 -- 172.21.15.16:0/664596051 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8c7809f650 msgr2=0x7f8c7806c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:14.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.695+0000 7f8c77fff700 1 --2- 172.21.15.16:0/664596051 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8c7809f650 0x7f8c7806c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:14.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.695+0000 7f8c77fff700 1 -- 172.21.15.16:0/664596051 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8c7809e930 msgr2=0x7f8c7806b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:14.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.695+0000 7f8c77fff700 1 --2- 172.21.15.16:0/664596051 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8c7809e930 0x7f8c7806b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:14.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.695+0000 7f8c77fff700 1 -- 172.21.15.16:0/664596051 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c70009a70 con 0x7f8c7809f240 2023-09-30T22:31:14.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.695+0000 7f8c77fff700 1 --2- 172.21.15.16:0/664596051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c7809f240 0x7f8c7806bdf0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8c64016000 tx=0x7f8c64004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:14.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.695+0000 7f8c757fa700 1 -- 172.21.15.16:0/664596051 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c64004240 con 0x7f8c7809f240 2023-09-30T22:31:14.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.696+0000 7f8c757fa700 1 -- 172.21.15.16:0/664596051 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c6401f070 con 0x7f8c7809f240 2023-09-30T22:31:14.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.696+0000 7f8c7eedd700 1 -- 172.21.15.16:0/664596051 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c64009550 con 0x7f8c7809f240 2023-09-30T22:31:14.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.696+0000 7f8c757fa700 1 -- 172.21.15.16:0/664596051 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c640036a0 con 0x7f8c7809f240 2023-09-30T22:31:14.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.696+0000 7f8c7eedd700 1 -- 172.21.15.16:0/664596051 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c780a0190 con 0x7f8c7809f240 2023-09-30T22:31:14.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.696+0000 7f8c757fa700 1 -- 172.21.15.16:0/664596051 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8c64021030 con 0x7f8c7809f240 2023-09-30T22:31:14.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.697+0000 7f8c7eedd700 1 -- 172.21.15.16:0/664596051 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c781a0b30 con 0x7f8c7809f240 2023-09-30T22:31:14.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.697+0000 7f8c757fa700 1 -- 172.21.15.16:0/664596051 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f8c640409c0 con 0x7f8c7809f240 2023-09-30T22:31:14.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.702+0000 7f8c757fa700 1 -- 172.21.15.16:0/664596051 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8c64023790 con 0x7f8c7809f240 2023-09-30T22:31:14.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.726+0000 7f100ca88700 1 Processor -- start 2023-09-30T22:31:14.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.727+0000 7f100ca88700 1 -- start start 2023-09-30T22:31:14.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.727+0000 7f100ca88700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f100809e930 0x7f100809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:14.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.727+0000 7f100ca88700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f100809f240 0x7f10081317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:14.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.727+0000 7f100ca88700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1008070940 con 0x7f100809f650 2023-09-30T22:31:14.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.727+0000 7f100ca88700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f100806fd60 con 0x7f100809e930 2023-09-30T22:31:14.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.727+0000 7f100ca88700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f100806fa80 con 0x7f100809f240 2023-09-30T22:31:14.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.727+0000 7f1005d9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f100809f240 0x7f10081317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:14.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.727+0000 7f1005d9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f100809f240 0x7f10081317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:35366/0 (socket says 172.21.15.16:35366) 2023-09-30T22:31:14.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.727+0000 7f1005d9b700 1 -- 172.21.15.16:0/729161866 learned_addr learned my addr 172.21.15.16:0/729161866 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:14.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.728+0000 7f100659c700 1 --2- 172.21.15.16:0/729161866 >> v2:172.21.15.16:3300/0 conn(0x7f100809e930 0x7f100809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:14.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.728+0000 7f1005d9b700 1 -- 172.21.15.16:0/729161866 >> v2:172.21.15.16:3300/0 conn(0x7f100809e930 msgr2=0x7f100809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:14.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.728+0000 7f1005d9b700 1 --2- 172.21.15.16:0/729161866 >> v2:172.21.15.16:3300/0 conn(0x7f100809e930 0x7f100809ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:14.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.728+0000 7f1005d9b700 1 -- 172.21.15.16:0/729161866 >> v1:172.21.15.16:6789/0 conn(0x7f100809f650 legacy=0x7f1008131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:14.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.728+0000 7f1005d9b700 1 -- 172.21.15.16:0/729161866 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f100809fb60 con 0x7f100809f240 2023-09-30T22:31:14.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.728+0000 7f100659c700 1 --2- 172.21.15.16:0/729161866 >> v2:172.21.15.16:3300/0 conn(0x7f100809e930 0x7f100809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:14.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.728+0000 7f1005d9b700 1 --2- 172.21.15.16:0/729161866 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f100809f240 0x7f10081317f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc0098f0 tx=0x7f0ffc009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9daac1287435fc7a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:14.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.728+0000 7f100559a700 1 -- 172.21.15.16:0/729161866 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc011830 con 0x7f100809f240 2023-09-30T22:31:14.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.728+0000 7f100559a700 1 -- 172.21.15.16:0/729161866 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ffc011990 con 0x7f100809f240 2023-09-30T22:31:14.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.728+0000 7f100559a700 1 -- 172.21.15.16:0/729161866 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc011d20 con 0x7f100809f240 2023-09-30T22:31:14.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.729+0000 7f100ca88700 1 -- 172.21.15.16:0/729161866 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f100809f240 msgr2=0x7f10081317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:14.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.729+0000 7f100ca88700 1 --2- 172.21.15.16:0/729161866 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f100809f240 0x7f10081317f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc0098f0 tx=0x7f0ffc009c00 comp rx=0 tx=0).stop 2023-09-30T22:31:14.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.729+0000 7f100ca88700 1 -- 172.21.15.16:0/729161866 shutdown_connections 2023-09-30T22:31:14.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.729+0000 7f100ca88700 1 --2- 172.21.15.16:0/729161866 >> v2:172.21.15.16:3300/0 conn(0x7f100809e930 0x7f100809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:14.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.729+0000 7f100ca88700 1 --2- 172.21.15.16:0/729161866 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f100809f240 0x7f10081317f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:14.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.729+0000 7f100ca88700 1 -- 172.21.15.16:0/729161866 >> 172.21.15.16:0/729161866 conn(0x7f10080a4210 msgr2=0x7f10080a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:14.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.729+0000 7f100ca88700 1 -- 172.21.15.16:0/729161866 shutdown_connections 2023-09-30T22:31:14.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.729+0000 7f100ca88700 1 -- 172.21.15.16:0/729161866 wait complete. 2023-09-30T22:31:14.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.730+0000 7f100ca88700 1 Processor -- start 2023-09-30T22:31:14.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.730+0000 7f100ca88700 1 -- start start 2023-09-30T22:31:14.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.731+0000 7f100ca88700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f100809e930 0x7f100806b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:14.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.731+0000 7f100ca88700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f100809f240 0x7f100806bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:14.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.731+0000 7f100ca88700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f100809f650 0x7f100806c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:14.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.731+0000 7f100ca88700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1008071d20 con 0x7f100809f240 2023-09-30T22:31:14.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.731+0000 7f100ca88700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f100806d580 con 0x7f100809f650 2023-09-30T22:31:14.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.731+0000 7f100ca88700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f10081ab710 con 0x7f100809e930 2023-09-30T22:31:14.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.731+0000 7f1005d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f100809f240 0x7f100806bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:14.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.731+0000 7f1005d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f100809f240 0x7f100806bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46106/0 (socket says 172.21.15.16:46106) 2023-09-30T22:31:14.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.731+0000 7f1005d9b700 1 -- 172.21.15.16:0/338507235 learned_addr learned my addr 172.21.15.16:0/338507235 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:14.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.731+0000 7f100659c700 1 --2- 172.21.15.16:0/338507235 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f100809e930 0x7f100806b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:14.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.731+0000 7f1006d9d700 1 --2- 172.21.15.16:0/338507235 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f100809f650 0x7f100806c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:14.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.731+0000 7f1005d9b700 1 -- 172.21.15.16:0/338507235 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f100809e930 msgr2=0x7f100806b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:14.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.732+0000 7f1005d9b700 1 --2- 172.21.15.16:0/338507235 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f100809e930 0x7f100806b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:14.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.732+0000 7f1005d9b700 1 -- 172.21.15.16:0/338507235 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f100809f650 msgr2=0x7f100806c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:14.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.732+0000 7f1005d9b700 1 --2- 172.21.15.16:0/338507235 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f100809f650 0x7f100806c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:14.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.732+0000 7f1005d9b700 1 -- 172.21.15.16:0/338507235 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff80094a0 con 0x7f100809f240 2023-09-30T22:31:14.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.732+0000 7f1005d9b700 1 --2- 172.21.15.16:0/338507235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f100809f240 0x7f100806bdf0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc000c00 tx=0x7f0ffc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:14.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.732+0000 7f0ff6ffd700 1 -- 172.21.15.16:0/338507235 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc0036a0 con 0x7f100809f240 2023-09-30T22:31:14.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.732+0000 7f0ff6ffd700 1 -- 172.21.15.16:0/338507235 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ffc003800 con 0x7f100809f240 2023-09-30T22:31:14.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.732+0000 7f100ca88700 1 -- 172.21.15.16:0/338507235 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ffc009550 con 0x7f100809f240 2023-09-30T22:31:14.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.732+0000 7f0ff6ffd700 1 -- 172.21.15.16:0/338507235 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc003bb0 con 0x7f100809f240 2023-09-30T22:31:14.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.732+0000 7f100ca88700 1 -- 172.21.15.16:0/338507235 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10080a0190 con 0x7f100809f240 2023-09-30T22:31:14.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.733+0000 7f0ff6ffd700 1 -- 172.21.15.16:0/338507235 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0ffc021070 con 0x7f100809f240 2023-09-30T22:31:14.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.733+0000 7f0ff6ffd700 1 -- 172.21.15.16:0/338507235 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f0ffc040ad0 con 0x7f100809f240 2023-09-30T22:31:14.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.734+0000 7f100ca88700 1 -- 172.21.15.16:0/338507235 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10081a2b40 con 0x7f100809f240 2023-09-30T22:31:14.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.743+0000 7f0ff6ffd700 1 -- 172.21.15.16:0/338507235 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0ffc015020 con 0x7f100809f240 2023-09-30T22:31:14.755 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:14.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.776+0000 7f5de3e22700 1 Processor -- start 2023-09-30T22:31:14.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.776+0000 7f5de3e22700 1 -- start start 2023-09-30T22:31:14.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.777+0000 7f5de3e22700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5ddc133420 0x7f5ddc1337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:14.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.777+0000 7f5de3e22700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5ddc133e30 0x7f5ddc12ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:14.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.777+0000 7f5de3e22700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ddc06b990 con 0x7f5ddc134260 2023-09-30T22:31:14.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.777+0000 7f5de3e22700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ddc0a0590 con 0x7f5ddc133420 2023-09-30T22:31:14.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.777+0000 7f5de3e22700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5ddc09fb80 con 0x7f5ddc133e30 2023-09-30T22:31:14.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.777+0000 7f5de23bf700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5ddc134260 0x7f5ddc1304d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47454/0 (socket says 172.21.15.16:47454) 2023-09-30T22:31:14.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.777+0000 7f5de23bf700 1 -- 172.21.15.16:0/3577526190 learned_addr learned my addr 172.21.15.16:0/3577526190 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:31:14.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.777+0000 7f5de13bd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5ddc133e30 0x7f5ddc12ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:14.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.777+0000 7f5de1bbe700 1 --2- 172.21.15.16:0/3577526190 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5ddc133420 0x7f5ddc1337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:14.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.778+0000 7f5de13bd700 1 -- 172.21.15.16:0/3577526190 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5ddc133420 msgr2=0x7f5ddc1337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:14.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.778+0000 7f5de13bd700 1 --2- 172.21.15.16:0/3577526190 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5ddc133420 0x7f5ddc1337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:14.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.778+0000 7f5de0bbc700 1 -- 172.21.15.16:0/3577526190 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 836237271 0 0) 0x7f5ddc06b990 con 0x7f5ddc134260 2023-09-30T22:31:14.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.778+0000 7f5de13bd700 1 -- 172.21.15.16:0/3577526190 >> v1:172.21.15.16:6789/0 conn(0x7f5ddc134260 legacy=0x7f5ddc1304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:14.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.778+0000 7f5de13bd700 1 -- 172.21.15.16:0/3577526190 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ddc0a0030 con 0x7f5ddc133e30 2023-09-30T22:31:14.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.778+0000 7f5de1bbe700 1 --2- 172.21.15.16:0/3577526190 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5ddc133420 0x7f5ddc1337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:14.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.778+0000 7f5de13bd700 1 --2- 172.21.15.16:0/3577526190 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5ddc133e30 0x7f5ddc12ff90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc009870 tx=0x7f5dcc009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef4f28282e71db2b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:14.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.778+0000 7f5de0bbc700 1 -- 172.21.15.16:0/3577526190 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc011830 con 0x7f5ddc133e30 2023-09-30T22:31:14.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.778+0000 7f5de0bbc700 1 -- 172.21.15.16:0/3577526190 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dcc011990 con 0x7f5ddc133e30 2023-09-30T22:31:14.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.778+0000 7f5de0bbc700 1 -- 172.21.15.16:0/3577526190 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc011d20 con 0x7f5ddc133e30 2023-09-30T22:31:14.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.779+0000 7f5de3e22700 1 -- 172.21.15.16:0/3577526190 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5ddc133e30 msgr2=0x7f5ddc12ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:14.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.779+0000 7f5de3e22700 1 --2- 172.21.15.16:0/3577526190 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5ddc133e30 0x7f5ddc12ff90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc009870 tx=0x7f5dcc009b80 comp rx=0 tx=0).stop 2023-09-30T22:31:14.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.779+0000 7f5de3e22700 1 -- 172.21.15.16:0/3577526190 shutdown_connections 2023-09-30T22:31:14.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.779+0000 7f5de3e22700 1 --2- 172.21.15.16:0/3577526190 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5ddc133420 0x7f5ddc1337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:14.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.779+0000 7f5de3e22700 1 --2- 172.21.15.16:0/3577526190 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5ddc133e30 0x7f5ddc12ff90 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:14.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.779+0000 7f5de3e22700 1 -- 172.21.15.16:0/3577526190 >> 172.21.15.16:0/3577526190 conn(0x7f5ddc0a4210 msgr2=0x7f5ddc0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:14.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.779+0000 7f5de3e22700 1 -- 172.21.15.16:0/3577526190 shutdown_connections 2023-09-30T22:31:14.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.779+0000 7f5de3e22700 1 -- 172.21.15.16:0/3577526190 wait complete. 2023-09-30T22:31:14.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.781+0000 7f5de3e22700 1 Processor -- start 2023-09-30T22:31:14.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.781+0000 7f5de3e22700 1 -- start start 2023-09-30T22:31:14.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.781+0000 7f5de3e22700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5ddc133420 0x7f5ddc09c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:14.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.781+0000 7f5de3e22700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5ddc133e30 0x7f5ddc09c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:14.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.781+0000 7f5de3e22700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ddc134260 0x7f5ddc09cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:14.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.781+0000 7f5de3e22700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ddc11f050 con 0x7f5ddc134260 2023-09-30T22:31:14.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.781+0000 7f5de3e22700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ddc09de50 con 0x7f5ddc133e30 2023-09-30T22:31:14.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.781+0000 7f5de3e22700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5ddc11fc50 con 0x7f5ddc133420 2023-09-30T22:31:14.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.782+0000 7f5de13bd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5ddc133e30 0x7f5ddc09c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:14.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.782+0000 7f5de13bd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5ddc133e30 0x7f5ddc09c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:35394/0 (socket says 172.21.15.16:35394) 2023-09-30T22:31:14.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.782+0000 7f5de13bd700 1 -- 172.21.15.16:0/3766354133 learned_addr learned my addr 172.21.15.16:0/3766354133 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:14.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.782+0000 7f5de23bf700 1 --2- 172.21.15.16:0/3766354133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ddc134260 0x7f5ddc09cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:14.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.782+0000 7f5de13bd700 1 -- 172.21.15.16:0/3766354133 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5ddc133420 msgr2=0x7f5ddc09c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:14.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.782+0000 7f5de13bd700 1 --2- 172.21.15.16:0/3766354133 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5ddc133420 0x7f5ddc09c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:14.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.782+0000 7f5de13bd700 1 -- 172.21.15.16:0/3766354133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ddc134260 msgr2=0x7f5ddc09cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:14.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.782+0000 7f5de13bd700 1 --2- 172.21.15.16:0/3766354133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ddc134260 0x7f5ddc09cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:14.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.782+0000 7f5de13bd700 1 -- 172.21.15.16:0/3766354133 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd80094a0 con 0x7f5ddc133e30 2023-09-30T22:31:14.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.782+0000 7f5de13bd700 1 --2- 172.21.15.16:0/3766354133 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5ddc133e30 0x7f5ddc09c9c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc0090b0 tx=0x7f5dcc004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:14.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.782+0000 7f5dd27fc700 1 -- 172.21.15.16:0/3766354133 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc004590 con 0x7f5ddc133e30 2023-09-30T22:31:14.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.782+0000 7f5dd27fc700 1 -- 172.21.15.16:0/3766354133 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dcc01f070 con 0x7f5ddc133e30 2023-09-30T22:31:14.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.782+0000 7f5dd27fc700 1 -- 172.21.15.16:0/3766354133 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc0038c0 con 0x7f5ddc133e30 2023-09-30T22:31:14.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.783+0000 7f5de3e22700 1 -- 172.21.15.16:0/3766354133 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dcc0094a0 con 0x7f5ddc133e30 2023-09-30T22:31:14.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.783+0000 7f5de3e22700 1 -- 172.21.15.16:0/3766354133 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ddc09e3e0 con 0x7f5ddc133e30 2023-09-30T22:31:14.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.784+0000 7f5de3e22700 1 -- 172.21.15.16:0/3766354133 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ddc074ea0 con 0x7f5ddc133e30 2023-09-30T22:31:14.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.784+0000 7f5dd27fc700 1 -- 172.21.15.16:0/3766354133 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5dcc003a20 con 0x7f5ddc133e30 2023-09-30T22:31:14.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.784+0000 7f5dd27fc700 1 -- 172.21.15.16:0/3766354133 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f5dcc021070 con 0x7f5ddc133e30 2023-09-30T22:31:14.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.790+0000 7f5dd27fc700 1 -- 172.21.15.16:0/3766354133 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5dcc010be0 con 0x7f5ddc133e30 2023-09-30T22:31:14.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.898+0000 7f121d019700 1 Processor -- start 2023-09-30T22:31:14.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.898+0000 7f121d019700 1 -- start start 2023-09-30T22:31:14.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.898+0000 7f121d019700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f121806a770 0x7f121809d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:14.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.898+0000 7f121d019700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f121813d1f0 0x7f121813f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:14.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.899+0000 7f121d019700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f121809a810 con 0x7f121809dc30 2023-09-30T22:31:14.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.899+0000 7f121d019700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f121806ab40 con 0x7f121806a770 2023-09-30T22:31:14.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.899+0000 7f121d019700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f121809ebc0 con 0x7f121813d1f0 2023-09-30T22:31:14.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.899+0000 7f121659c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f121809dc30 0x7f121809e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47462/0 (socket says 172.21.15.16:47462) 2023-09-30T22:31:14.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.899+0000 7f121659c700 1 -- 172.21.15.16:0/2863619778 learned_addr learned my addr 172.21.15.16:0/2863619778 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:31:14.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.899+0000 7f121759e700 1 --2- 172.21.15.16:0/2863619778 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f121813d1f0 0x7f121813f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:14.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.899+0000 7f1216d9d700 1 --2- 172.21.15.16:0/2863619778 >> v2:172.21.15.16:3300/0 conn(0x7f121806a770 0x7f121809d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:14.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.899+0000 7f1215d9b700 1 -- 172.21.15.16:0/2863619778 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 159054413 0 0) 0x7f121809a810 con 0x7f121809dc30 2023-09-30T22:31:14.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.899+0000 7f121759e700 1 -- 172.21.15.16:0/2863619778 >> v2:172.21.15.16:3300/0 conn(0x7f121806a770 msgr2=0x7f121809d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:14.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.899+0000 7f121759e700 1 --2- 172.21.15.16:0/2863619778 >> v2:172.21.15.16:3300/0 conn(0x7f121806a770 0x7f121809d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:14.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.899+0000 7f121759e700 1 -- 172.21.15.16:0/2863619778 >> v1:172.21.15.16:6789/0 conn(0x7f121809dc30 legacy=0x7f121809e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:14.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.899+0000 7f121759e700 1 -- 172.21.15.16:0/2863619778 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f121809e920 con 0x7f121813d1f0 2023-09-30T22:31:14.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.900+0000 7f121759e700 1 --2- 172.21.15.16:0/2863619778 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f121813d1f0 0x7f121813f5d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1210009d90 tx=0x7f121000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a66edb80d660ec8d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:14.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.900+0000 7f1215d9b700 1 -- 172.21.15.16:0/2863619778 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121000e070 con 0x7f121813d1f0 2023-09-30T22:31:14.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.900+0000 7f1215d9b700 1 -- 172.21.15.16:0/2863619778 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f121000bb20 con 0x7f121813d1f0 2023-09-30T22:31:14.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.900+0000 7f1215d9b700 1 -- 172.21.15.16:0/2863619778 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1210012430 con 0x7f121813d1f0 2023-09-30T22:31:14.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.900+0000 7f121d019700 1 -- 172.21.15.16:0/2863619778 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f121813d1f0 msgr2=0x7f121813f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:14.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.900+0000 7f121d019700 1 --2- 172.21.15.16:0/2863619778 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f121813d1f0 0x7f121813f5d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1210009d90 tx=0x7f121000b330 comp rx=0 tx=0).stop 2023-09-30T22:31:14.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.900+0000 7f121d019700 1 -- 172.21.15.16:0/2863619778 shutdown_connections 2023-09-30T22:31:14.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.900+0000 7f121d019700 1 --2- 172.21.15.16:0/2863619778 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f121813d1f0 0x7f121813f5d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:14.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.900+0000 7f121d019700 1 --2- 172.21.15.16:0/2863619778 >> v2:172.21.15.16:3300/0 conn(0x7f121806a770 0x7f121809d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:14.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.900+0000 7f121d019700 1 -- 172.21.15.16:0/2863619778 >> 172.21.15.16:0/2863619778 conn(0x7f1218066f50 msgr2=0x7f121809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:14.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.900+0000 7f121d019700 1 -- 172.21.15.16:0/2863619778 shutdown_connections 2023-09-30T22:31:14.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.900+0000 7f121d019700 1 -- 172.21.15.16:0/2863619778 wait complete. 2023-09-30T22:31:14.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.902+0000 7f121d019700 1 Processor -- start 2023-09-30T22:31:14.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.902+0000 7f121d019700 1 -- start start 2023-09-30T22:31:14.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.902+0000 7f121d019700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f121806a770 0x7f1218071580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:14.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.902+0000 7f121d019700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f121809dc30 0x7f121813b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:14.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.902+0000 7f121d019700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f121813d1f0 0x7f121813ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:14.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.902+0000 7f121d019700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f121813cdd0 con 0x7f121813d1f0 2023-09-30T22:31:14.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.902+0000 7f121d019700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f121811ef50 con 0x7f121809dc30 2023-09-30T22:31:14.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.902+0000 7f121d019700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1218137d40 con 0x7f121806a770 2023-09-30T22:31:14.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.903+0000 7f121759e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f121813d1f0 0x7f121813ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:14.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.903+0000 7f121759e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f121813d1f0 0x7f121813ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46130/0 (socket says 172.21.15.16:46130) 2023-09-30T22:31:14.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.903+0000 7f121759e700 1 -- 172.21.15.16:0/1525834963 learned_addr learned my addr 172.21.15.16:0/1525834963 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:14.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.903+0000 7f1216d9d700 1 --2- 172.21.15.16:0/1525834963 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f121806a770 0x7f1218071580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:14.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.903+0000 7f121659c700 1 --2- 172.21.15.16:0/1525834963 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f121809dc30 0x7f121813b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:14.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.903+0000 7f121759e700 1 -- 172.21.15.16:0/1525834963 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f121806a770 msgr2=0x7f1218071580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:14.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.903+0000 7f121759e700 1 --2- 172.21.15.16:0/1525834963 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f121806a770 0x7f1218071580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:14.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.903+0000 7f121759e700 1 -- 172.21.15.16:0/1525834963 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f121809dc30 msgr2=0x7f121813b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:14.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.903+0000 7f121759e700 1 --2- 172.21.15.16:0/1525834963 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f121809dc30 0x7f121813b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:14.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.903+0000 7f121759e700 1 -- 172.21.15.16:0/1525834963 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1210009a70 con 0x7f121813d1f0 2023-09-30T22:31:14.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.903+0000 7f121759e700 1 --2- 172.21.15.16:0/1525834963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f121813d1f0 0x7f121813ba20 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f1210013000 tx=0x7f1210004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:14.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.903+0000 7f11ff7fe700 1 -- 172.21.15.16:0/1525834963 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1210012430 con 0x7f121813d1f0 2023-09-30T22:31:14.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.904+0000 7f11ff7fe700 1 -- 172.21.15.16:0/1525834963 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1210020070 con 0x7f121813d1f0 2023-09-30T22:31:14.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.904+0000 7f11ff7fe700 1 -- 172.21.15.16:0/1525834963 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1210004280 con 0x7f121813d1f0 2023-09-30T22:31:14.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.904+0000 7f121d019700 1 -- 172.21.15.16:0/1525834963 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f121811e9b0 con 0x7f121813d1f0 2023-09-30T22:31:14.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.904+0000 7f121d019700 1 -- 172.21.15.16:0/1525834963 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f121811f6e0 con 0x7f121813d1f0 2023-09-30T22:31:14.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.905+0000 7f11ff7fe700 1 -- 172.21.15.16:0/1525834963 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f12100043e0 con 0x7f121813d1f0 2023-09-30T22:31:14.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.905+0000 7f11ff7fe700 1 -- 172.21.15.16:0/1525834963 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f1210022070 con 0x7f121813d1f0 2023-09-30T22:31:14.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.905+0000 7f121d019700 1 -- 172.21.15.16:0/1525834963 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f121811ed80 con 0x7f121813d1f0 2023-09-30T22:31:14.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.915+0000 7f11ff7fe700 1 -- 172.21.15.16:0/1525834963 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f121001a970 con 0x7f121813d1f0 2023-09-30T22:31:14.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.928+0000 7f8c7eedd700 1 -- 172.21.15.16:0/664596051 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f8c780a35c0 con 0x7f8c7809f240 2023-09-30T22:31:14.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.929+0000 7f8c757fa700 1 -- 172.21.15.16:0/664596051 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8c64015020 con 0x7f8c7809f240 2023-09-30T22:31:14.930 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:31:14.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.931+0000 7f8c7eedd700 1 -- 172.21.15.16:0/664596051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c7809f240 msgr2=0x7f8c7806bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:14.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.931+0000 7f8c7eedd700 1 --2- 172.21.15.16:0/664596051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c7809f240 0x7f8c7806bdf0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8c64016000 tx=0x7f8c64004750 comp rx=0 tx=0).stop 2023-09-30T22:31:14.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.931+0000 7f8c7eedd700 1 -- 172.21.15.16:0/664596051 shutdown_connections 2023-09-30T22:31:14.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.931+0000 7f8c7eedd700 1 --2- 172.21.15.16:0/664596051 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8c7809e930 0x7f8c7806b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:14.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.931+0000 7f8c7eedd700 1 --2- 172.21.15.16:0/664596051 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8c7809f650 0x7f8c7806c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:14.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.931+0000 7f8c7eedd700 1 --2- 172.21.15.16:0/664596051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c7809f240 0x7f8c7806bdf0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:14.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.931+0000 7f8c7eedd700 1 -- 172.21.15.16:0/664596051 >> 172.21.15.16:0/664596051 conn(0x7f8c780a4210 msgr2=0x7f8c781b8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:14.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.932+0000 7f8c7eedd700 1 -- 172.21.15.16:0/664596051 shutdown_connections 2023-09-30T22:31:14.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.932+0000 7f8c7eedd700 1 -- 172.21.15.16:0/664596051 wait complete. 2023-09-30T22:31:14.947 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:31:14.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.958+0000 7f100ca88700 1 -- 172.21.15.16:0/338507235 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f10081aa330 con 0x7f100809f240 2023-09-30T22:31:14.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.958+0000 7f0ff6ffd700 1 -- 172.21.15.16:0/338507235 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0ffc00b3e0 con 0x7f100809f240 2023-09-30T22:31:14.960 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:31:14.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.960+0000 7f100ca88700 1 -- 172.21.15.16:0/338507235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f100809f240 msgr2=0x7f100806bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:14.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.960+0000 7f100ca88700 1 --2- 172.21.15.16:0/338507235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f100809f240 0x7f100806bdf0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc000c00 tx=0x7f0ffc004750 comp rx=0 tx=0).stop 2023-09-30T22:31:14.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.960+0000 7f100ca88700 1 -- 172.21.15.16:0/338507235 shutdown_connections 2023-09-30T22:31:14.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.960+0000 7f100ca88700 1 --2- 172.21.15.16:0/338507235 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f100809f650 0x7f100806c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:14.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.960+0000 7f100ca88700 1 --2- 172.21.15.16:0/338507235 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f100809e930 0x7f100806b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:14.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.960+0000 7f100ca88700 1 --2- 172.21.15.16:0/338507235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f100809f240 0x7f100806bdf0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:14.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.960+0000 7f100ca88700 1 -- 172.21.15.16:0/338507235 >> 172.21.15.16:0/338507235 conn(0x7f10080a4210 msgr2=0x7f100809dd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:14.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.961+0000 7f100ca88700 1 -- 172.21.15.16:0/338507235 shutdown_connections 2023-09-30T22:31:14.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:14.961+0000 7f100ca88700 1 -- 172.21.15.16:0/338507235 wait complete. 2023-09-30T22:31:14.976 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:31:15.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:15.004+0000 7f5de3e22700 1 -- 172.21.15.16:0/3766354133 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5ddc120100 con 0x7f5ddc133e30 2023-09-30T22:31:15.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:15.004+0000 7f5dd27fc700 1 -- 172.21.15.16:0/3766354133 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5dcc034020 con 0x7f5ddc133e30 2023-09-30T22:31:15.006 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:31:15.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:15.007+0000 7f5de3e22700 1 -- 172.21.15.16:0/3766354133 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5ddc133e30 msgr2=0x7f5ddc09c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:15.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:15.007+0000 7f5de3e22700 1 --2- 172.21.15.16:0/3766354133 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5ddc133e30 0x7f5ddc09c9c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc0090b0 tx=0x7f5dcc004750 comp rx=0 tx=0).stop 2023-09-30T22:31:15.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:15.007+0000 7f5de3e22700 1 -- 172.21.15.16:0/3766354133 shutdown_connections 2023-09-30T22:31:15.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:15.007+0000 7f5de3e22700 1 --2- 172.21.15.16:0/3766354133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ddc134260 0x7f5ddc09cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:15.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:15.007+0000 7f5de3e22700 1 --2- 172.21.15.16:0/3766354133 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5ddc133420 0x7f5ddc09c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:15.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:15.007+0000 7f5de3e22700 1 --2- 172.21.15.16:0/3766354133 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5ddc133e30 0x7f5ddc09c9c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:15.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:15.007+0000 7f5de3e22700 1 -- 172.21.15.16:0/3766354133 >> 172.21.15.16:0/3766354133 conn(0x7f5ddc0a4210 msgr2=0x7f5ddc0a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:15.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:15.007+0000 7f5de3e22700 1 -- 172.21.15.16:0/3766354133 shutdown_connections 2023-09-30T22:31:15.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:15.007+0000 7f5de3e22700 1 -- 172.21.15.16:0/3766354133 wait complete. 2023-09-30T22:31:15.022 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:31:15.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:15.122+0000 7f121d019700 1 -- 172.21.15.16:0/1525834963 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1218074ea0 con 0x7f121813d1f0 2023-09-30T22:31:15.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:15.122+0000 7f11ff7fe700 1 -- 172.21.15.16:0/1525834963 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f121001a380 con 0x7f121813d1f0 2023-09-30T22:31:15.124 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:31:15.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:15.124+0000 7f121d019700 1 -- 172.21.15.16:0/1525834963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f121813d1f0 msgr2=0x7f121813ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:15.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:15.124+0000 7f121d019700 1 --2- 172.21.15.16:0/1525834963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f121813d1f0 0x7f121813ba20 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f1210013000 tx=0x7f1210004ad0 comp rx=0 tx=0).stop 2023-09-30T22:31:15.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:15.124+0000 7f121d019700 1 -- 172.21.15.16:0/1525834963 shutdown_connections 2023-09-30T22:31:15.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:15.124+0000 7f121d019700 1 --2- 172.21.15.16:0/1525834963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f121813d1f0 0x7f121813ba20 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:15.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:15.124+0000 7f121d019700 1 --2- 172.21.15.16:0/1525834963 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f121806a770 0x7f1218071580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:15.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:15.124+0000 7f121d019700 1 --2- 172.21.15.16:0/1525834963 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f121809dc30 0x7f121813b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:15.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:15.124+0000 7f121d019700 1 -- 172.21.15.16:0/1525834963 >> 172.21.15.16:0/1525834963 conn(0x7f1218066f50 msgr2=0x7f121809b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:15.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:15.124+0000 7f121d019700 1 -- 172.21.15.16:0/1525834963 shutdown_connections 2023-09-30T22:31:15.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:15.124+0000 7f121d019700 1 -- 172.21.15.16:0/1525834963 wait complete. 2023-09-30T22:31:15.142 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:31:15.948 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:15.976 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:16.023 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:16.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.106+0000 7f040a5fe700 1 Processor -- start 2023-09-30T22:31:16.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.106+0000 7f040a5fe700 1 -- start start 2023-09-30T22:31:16.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.107+0000 7f040a5fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f040409d6f0 0x7f040409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:16.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.107+0000 7f040a5fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f040409e440 0x7f0404134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:16.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.107+0000 7f040a5fe700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f040409b120 con 0x7f040409e000 2023-09-30T22:31:16.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.107+0000 7f040a5fe700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f040406b460 con 0x7f040409d6f0 2023-09-30T22:31:16.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.107+0000 7f040a5fe700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f040409a7e0 con 0x7f040409e440 2023-09-30T22:31:16.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.107+0000 7f0403fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f040409d6f0 0x7f040409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:16.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.107+0000 7f0403fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f040409d6f0 0x7f040409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46142/0 (socket says 172.21.15.16:46142) 2023-09-30T22:31:16.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.107+0000 7f0403fff700 1 -- 172.21.15.16:0/2577368467 learned_addr learned my addr 172.21.15.16:0/2577368467 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:16.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.107+0000 7f0408b9b700 1 --2- 172.21.15.16:0/2577368467 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f040409e440 0x7f0404134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:16.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.107+0000 7f0408b9b700 1 -- 172.21.15.16:0/2577368467 >> v2:172.21.15.16:3300/0 conn(0x7f040409d6f0 msgr2=0x7f040409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:16.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.107+0000 7f0408b9b700 1 --2- 172.21.15.16:0/2577368467 >> v2:172.21.15.16:3300/0 conn(0x7f040409d6f0 0x7f040409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.107+0000 7f0408b9b700 1 -- 172.21.15.16:0/2577368467 >> v1:172.21.15.16:6789/0 conn(0x7f040409e000 legacy=0x7f04041339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:16.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.107+0000 7f0408b9b700 1 -- 172.21.15.16:0/2577368467 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f040409a500 con 0x7f040409e440 2023-09-30T22:31:16.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.107+0000 7f0403fff700 1 --2- 172.21.15.16:0/2577368467 >> v2:172.21.15.16:3300/0 conn(0x7f040409d6f0 0x7f040409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:16.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.108+0000 7f0408b9b700 1 --2- 172.21.15.16:0/2577368467 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f040409e440 0x7f0404134100 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f03fc009dc0 tx=0x7f03fc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a7b5dc528a74389 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:16.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.108+0000 7f0402ffd700 1 -- 172.21.15.16:0/2577368467 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03fc00e070 con 0x7f040409e440 2023-09-30T22:31:16.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.108+0000 7f0402ffd700 1 -- 172.21.15.16:0/2577368467 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03fc00bae0 con 0x7f040409e440 2023-09-30T22:31:16.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.108+0000 7f0402ffd700 1 -- 172.21.15.16:0/2577368467 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03fc00be90 con 0x7f040409e440 2023-09-30T22:31:16.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.108+0000 7f040a5fe700 1 -- 172.21.15.16:0/2577368467 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f040409e440 msgr2=0x7f0404134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:16.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.108+0000 7f040a5fe700 1 --2- 172.21.15.16:0/2577368467 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f040409e440 0x7f0404134100 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f03fc009dc0 tx=0x7f03fc00b330 comp rx=0 tx=0).stop 2023-09-30T22:31:16.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.109+0000 7f040a5fe700 1 -- 172.21.15.16:0/2577368467 shutdown_connections 2023-09-30T22:31:16.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.109+0000 7f040a5fe700 1 --2- 172.21.15.16:0/2577368467 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f040409e440 0x7f0404134100 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.109+0000 7f040a5fe700 1 --2- 172.21.15.16:0/2577368467 >> v2:172.21.15.16:3300/0 conn(0x7f040409d6f0 0x7f040409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.109+0000 7f040a5fe700 1 -- 172.21.15.16:0/2577368467 >> 172.21.15.16:0/2577368467 conn(0x7f0404066f50 msgr2=0x7f040409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:16.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.109+0000 7f040a5fe700 1 -- 172.21.15.16:0/2577368467 shutdown_connections 2023-09-30T22:31:16.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.109+0000 7f040a5fe700 1 -- 172.21.15.16:0/2577368467 wait complete. 2023-09-30T22:31:16.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.110+0000 7f040a5fe700 1 Processor -- start 2023-09-30T22:31:16.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.110+0000 7f040a5fe700 1 -- start start 2023-09-30T22:31:16.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.110+0000 7f040a5fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f040409d6f0 0x7f04041a5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:16.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.110+0000 7f040a5fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f040409e000 0x7f04041a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:16.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.110+0000 7f040a5fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f040409e440 0x7f04041a6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:16.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.110+0000 7f040a5fe700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04041aa360 con 0x7f040409e000 2023-09-30T22:31:16.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.110+0000 7f040a5fe700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04041a99a0 con 0x7f040409d6f0 2023-09-30T22:31:16.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.110+0000 7f040a5fe700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f04041a8ce0 con 0x7f040409e440 2023-09-30T22:31:16.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.111+0000 7f0403fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f040409d6f0 0x7f04041a5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:16.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.111+0000 7f0403fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f040409d6f0 0x7f04041a5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:35428/0 (socket says 172.21.15.16:35428) 2023-09-30T22:31:16.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.111+0000 7f0403fff700 1 -- 172.21.15.16:0/1360681320 learned_addr learned my addr 172.21.15.16:0/1360681320 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:16.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.111+0000 7f0408b9b700 1 --2- 172.21.15.16:0/1360681320 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f040409e440 0x7f04041a6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:16.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.111+0000 7f04037fe700 1 --2- 172.21.15.16:0/1360681320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f040409e000 0x7f04041a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:16.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.111+0000 7f0403fff700 1 -- 172.21.15.16:0/1360681320 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f040409e440 msgr2=0x7f04041a6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:16.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.111+0000 7f0403fff700 1 --2- 172.21.15.16:0/1360681320 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f040409e440 0x7f04041a6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.111+0000 7f0403fff700 1 -- 172.21.15.16:0/1360681320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f040409e000 msgr2=0x7f04041a6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:16.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.111+0000 7f0403fff700 1 --2- 172.21.15.16:0/1360681320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f040409e000 0x7f04041a6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.111+0000 7f0403fff700 1 -- 172.21.15.16:0/1360681320 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03f4009550 con 0x7f040409d6f0 2023-09-30T22:31:16.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.111+0000 7f0403fff700 1 --2- 172.21.15.16:0/1360681320 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f040409d6f0 0x7f04041a5ed0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f03f400ec10 tx=0x7f03f400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:16.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.111+0000 7f0400ff9700 1 -- 172.21.15.16:0/1360681320 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f400c850 con 0x7f040409d6f0 2023-09-30T22:31:16.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.112+0000 7f0400ff9700 1 -- 172.21.15.16:0/1360681320 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03f400c9b0 con 0x7f040409d6f0 2023-09-30T22:31:16.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.112+0000 7f040a5fe700 1 -- 172.21.15.16:0/1360681320 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03fc009a70 con 0x7f040409d6f0 2023-09-30T22:31:16.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.112+0000 7f0400ff9700 1 -- 172.21.15.16:0/1360681320 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f400cd40 con 0x7f040409d6f0 2023-09-30T22:31:16.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.112+0000 7f040a5fe700 1 -- 172.21.15.16:0/1360681320 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04041a7a90 con 0x7f040409d6f0 2023-09-30T22:31:16.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.113+0000 7f0400ff9700 1 -- 172.21.15.16:0/1360681320 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f03f4013470 con 0x7f040409d6f0 2023-09-30T22:31:16.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.113+0000 7f0400ff9700 1 -- 172.21.15.16:0/1360681320 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f03f403fc20 con 0x7f040409d6f0 2023-09-30T22:31:16.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.113+0000 7f040a5fe700 1 -- 172.21.15.16:0/1360681320 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0404074ea0 con 0x7f040409d6f0 2023-09-30T22:31:16.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.122+0000 7f0400ff9700 1 -- 172.21.15.16:0/1360681320 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f03f4021080 con 0x7f040409d6f0 2023-09-30T22:31:16.143 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:16.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.149+0000 7fca4fa04700 1 Processor -- start 2023-09-30T22:31:16.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.150+0000 7fca4fa04700 1 -- start start 2023-09-30T22:31:16.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.150+0000 7fca4fa04700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca4809e1d0 0x7fca48138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:16.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.150+0000 7fca4fa04700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fca48139170 0x7fca4813b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:16.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.150+0000 7fca4fa04700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca4809b120 con 0x7fca4809d6f0 2023-09-30T22:31:16.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.150+0000 7fca4fa04700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca4806b460 con 0x7fca48139170 2023-09-30T22:31:16.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.150+0000 7fca4fa04700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fca4809a7e0 con 0x7fca4809e1d0 2023-09-30T22:31:16.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.151+0000 7fca4d7a0700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fca4809d6f0 0x7fca4809dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47470/0 (socket says 172.21.15.16:47470) 2023-09-30T22:31:16.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.151+0000 7fca4d7a0700 1 -- 172.21.15.16:0/2947307922 learned_addr learned my addr 172.21.15.16:0/2947307922 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:31:16.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.151+0000 7fca4dfa1700 1 --2- 172.21.15.16:0/2947307922 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fca48139170 0x7fca4813b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:16.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.151+0000 7fca4cf9f700 1 --2- 172.21.15.16:0/2947307922 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca4809e1d0 0x7fca48138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:16.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.151+0000 7fca3ffff700 1 -- 172.21.15.16:0/2947307922 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4049636276 0 0) 0x7fca4809b120 con 0x7fca4809d6f0 2023-09-30T22:31:16.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.151+0000 7fca3ffff700 1 -- 172.21.15.16:0/2947307922 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca340035b0 con 0x7fca4809d6f0 2023-09-30T22:31:16.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.151+0000 7fca4dfa1700 1 -- 172.21.15.16:0/2947307922 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca4809e1d0 msgr2=0x7fca48138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:16.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.151+0000 7fca4dfa1700 1 --2- 172.21.15.16:0/2947307922 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca4809e1d0 0x7fca48138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.151+0000 7fca4dfa1700 1 -- 172.21.15.16:0/2947307922 >> v1:172.21.15.16:6789/0 conn(0x7fca4809d6f0 legacy=0x7fca4809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:16.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.151+0000 7fca4dfa1700 1 -- 172.21.15.16:0/2947307922 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca4809a500 con 0x7fca48139170 2023-09-30T22:31:16.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.152+0000 7fca4dfa1700 1 --2- 172.21.15.16:0/2947307922 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fca48139170 0x7fca4813b550 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fca40009dc0 tx=0x7fca4000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=785562607a079512 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:16.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.152+0000 7fca3ffff700 1 -- 172.21.15.16:0/2947307922 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4000e070 con 0x7fca48139170 2023-09-30T22:31:16.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.152+0000 7fca3ffff700 1 -- 172.21.15.16:0/2947307922 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca4000bb20 con 0x7fca48139170 2023-09-30T22:31:16.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.152+0000 7fca3ffff700 1 -- 172.21.15.16:0/2947307922 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca40012430 con 0x7fca48139170 2023-09-30T22:31:16.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.153+0000 7fca4fa04700 1 -- 172.21.15.16:0/2947307922 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fca48139170 msgr2=0x7fca4813b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:16.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.153+0000 7fca4fa04700 1 --2- 172.21.15.16:0/2947307922 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fca48139170 0x7fca4813b550 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fca40009dc0 tx=0x7fca4000b330 comp rx=0 tx=0).stop 2023-09-30T22:31:16.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.153+0000 7fca4fa04700 1 -- 172.21.15.16:0/2947307922 shutdown_connections 2023-09-30T22:31:16.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.153+0000 7fca4fa04700 1 --2- 172.21.15.16:0/2947307922 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fca48139170 0x7fca4813b550 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.153+0000 7fca4fa04700 1 --2- 172.21.15.16:0/2947307922 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca4809e1d0 0x7fca48138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.153+0000 7fca4fa04700 1 -- 172.21.15.16:0/2947307922 >> 172.21.15.16:0/2947307922 conn(0x7fca48066f50 msgr2=0x7fca4809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:16.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.153+0000 7fca4fa04700 1 -- 172.21.15.16:0/2947307922 shutdown_connections 2023-09-30T22:31:16.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.153+0000 7fca4fa04700 1 -- 172.21.15.16:0/2947307922 wait complete. 2023-09-30T22:31:16.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.155+0000 7fca4fa04700 1 Processor -- start 2023-09-30T22:31:16.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.155+0000 7fca4fa04700 1 -- start start 2023-09-30T22:31:16.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.155+0000 7fca4fa04700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca4809d6f0 0x7fca48121640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:16.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.155+0000 7fca4fa04700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca4809e1d0 0x7fca48121b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:16.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.156+0000 7fca4fa04700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fca48139170 0x7fca481220c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:16.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.156+0000 7fca4fa04700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca48125120 con 0x7fca4809d6f0 2023-09-30T22:31:16.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.156+0000 7fca4fa04700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca480a3a20 con 0x7fca48139170 2023-09-30T22:31:16.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.156+0000 7fca4fa04700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fca48122a30 con 0x7fca4809e1d0 2023-09-30T22:31:16.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.156+0000 7fca4cf9f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca4809e1d0 0x7fca48121b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:16.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.156+0000 7fca4d7a0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca4809d6f0 0x7fca48121640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:16.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.156+0000 7fca4cf9f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca4809e1d0 0x7fca48121b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:60052/0 (socket says 172.21.15.16:60052) 2023-09-30T22:31:16.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.156+0000 7fca4cf9f700 1 -- 172.21.15.16:0/3172159617 learned_addr learned my addr 172.21.15.16:0/3172159617 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:16.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.156+0000 7fca4dfa1700 1 --2- 172.21.15.16:0/3172159617 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fca48139170 0x7fca481220c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:16.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.156+0000 7fca4cf9f700 1 -- 172.21.15.16:0/3172159617 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fca48139170 msgr2=0x7fca481220c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:16.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.156+0000 7fca4cf9f700 1 --2- 172.21.15.16:0/3172159617 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fca48139170 0x7fca481220c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.156+0000 7fca4cf9f700 1 -- 172.21.15.16:0/3172159617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca4809d6f0 msgr2=0x7fca48121640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:16.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.156+0000 7fca4cf9f700 1 --2- 172.21.15.16:0/3172159617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca4809d6f0 0x7fca48121640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.156+0000 7fca4cf9f700 1 -- 172.21.15.16:0/3172159617 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca40009a70 con 0x7fca4809e1d0 2023-09-30T22:31:16.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.156+0000 7fca4d7a0700 1 --2- 172.21.15.16:0/3172159617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca4809d6f0 0x7fca48121640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:16.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.157+0000 7fca4cf9f700 1 --2- 172.21.15.16:0/3172159617 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca4809e1d0 0x7fca48121b80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fca3800e9b0 tx=0x7fca3800ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:16.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.157+0000 7fca3dffb700 1 -- 172.21.15.16:0/3172159617 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca38009710 con 0x7fca4809e1d0 2023-09-30T22:31:16.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.157+0000 7fca3dffb700 1 -- 172.21.15.16:0/3172159617 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca3800ee50 con 0x7fca4809e1d0 2023-09-30T22:31:16.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.157+0000 7fca4fa04700 1 -- 172.21.15.16:0/3172159617 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca4809c710 con 0x7fca4809e1d0 2023-09-30T22:31:16.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.157+0000 7fca3dffb700 1 -- 172.21.15.16:0/3172159617 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3800ca50 con 0x7fca4809e1d0 2023-09-30T22:31:16.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.157+0000 7fca4fa04700 1 -- 172.21.15.16:0/3172159617 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca48123a00 con 0x7fca4809e1d0 2023-09-30T22:31:16.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.159+0000 7fca3dffb700 1 -- 172.21.15.16:0/3172159617 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fca3800cbb0 con 0x7fca4809e1d0 2023-09-30T22:31:16.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.159+0000 7fca3dffb700 1 -- 172.21.15.16:0/3172159617 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fca3803f740 con 0x7fca4809e1d0 2023-09-30T22:31:16.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.159+0000 7fca4fa04700 1 -- 172.21.15.16:0/3172159617 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca48074ea0 con 0x7fca4809e1d0 2023-09-30T22:31:16.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.164+0000 7fca3dffb700 1 -- 172.21.15.16:0/3172159617 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fca380232f0 con 0x7fca4809e1d0 2023-09-30T22:31:16.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.183+0000 7f2996d45700 1 Processor -- start 2023-09-30T22:31:16.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.184+0000 7f2996d45700 1 -- start start 2023-09-30T22:31:16.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.184+0000 7f2996d45700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f299009e1d0 0x7f2990138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:16.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.184+0000 7f2996d45700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2990139170 0x7f299013b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:16.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.184+0000 7f2996d45700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f299009b120 con 0x7f299009d6f0 2023-09-30T22:31:16.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.184+0000 7f2996d45700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f299006b460 con 0x7f2990139170 2023-09-30T22:31:16.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.184+0000 7f2996d45700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f299009a7e0 con 0x7f299009e1d0 2023-09-30T22:31:16.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.184+0000 7f298ffff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f299009e1d0 0x7f2990138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:16.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.184+0000 7f2994ae1700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f299009d6f0 0x7f299009dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47480/0 (socket says 172.21.15.16:47480) 2023-09-30T22:31:16.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.184+0000 7f2994ae1700 1 -- 172.21.15.16:0/2783883068 learned_addr learned my addr 172.21.15.16:0/2783883068 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:31:16.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.184+0000 7f29952e2700 1 --2- 172.21.15.16:0/2783883068 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2990139170 0x7f299013b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:16.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.185+0000 7f298f7fe700 1 -- 172.21.15.16:0/2783883068 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2381543312 0 0) 0x7f299009b120 con 0x7f299009d6f0 2023-09-30T22:31:16.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.185+0000 7f298f7fe700 1 -- 172.21.15.16:0/2783883068 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29800035b0 con 0x7f299009d6f0 2023-09-30T22:31:16.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.185+0000 7f29952e2700 1 -- 172.21.15.16:0/2783883068 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f299009e1d0 msgr2=0x7f2990138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:16.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.185+0000 7f29952e2700 1 --2- 172.21.15.16:0/2783883068 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f299009e1d0 0x7f2990138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.185+0000 7f29952e2700 1 -- 172.21.15.16:0/2783883068 >> v1:172.21.15.16:6789/0 conn(0x7f299009d6f0 legacy=0x7f299009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:16.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.185+0000 7f29952e2700 1 -- 172.21.15.16:0/2783883068 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f299009a500 con 0x7f2990139170 2023-09-30T22:31:16.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.185+0000 7f29952e2700 1 --2- 172.21.15.16:0/2783883068 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2990139170 0x7f299013b550 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2988009dc0 tx=0x7f298800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfc6fa1ae183826c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:16.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.185+0000 7f298f7fe700 1 -- 172.21.15.16:0/2783883068 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f298800e070 con 0x7f2990139170 2023-09-30T22:31:16.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.185+0000 7f298f7fe700 1 -- 172.21.15.16:0/2783883068 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f298800bb20 con 0x7f2990139170 2023-09-30T22:31:16.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.185+0000 7f298f7fe700 1 -- 172.21.15.16:0/2783883068 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2988012430 con 0x7f2990139170 2023-09-30T22:31:16.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.186+0000 7f2996d45700 1 -- 172.21.15.16:0/2783883068 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2990139170 msgr2=0x7f299013b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:16.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.186+0000 7f2996d45700 1 --2- 172.21.15.16:0/2783883068 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2990139170 0x7f299013b550 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f2988009dc0 tx=0x7f298800b330 comp rx=0 tx=0).stop 2023-09-30T22:31:16.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.186+0000 7f2996d45700 1 -- 172.21.15.16:0/2783883068 shutdown_connections 2023-09-30T22:31:16.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.186+0000 7f2996d45700 1 --2- 172.21.15.16:0/2783883068 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2990139170 0x7f299013b550 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.186+0000 7f2996d45700 1 --2- 172.21.15.16:0/2783883068 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f299009e1d0 0x7f2990138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.186+0000 7f2996d45700 1 -- 172.21.15.16:0/2783883068 >> 172.21.15.16:0/2783883068 conn(0x7f2990066f50 msgr2=0x7f299009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:16.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.186+0000 7f2996d45700 1 -- 172.21.15.16:0/2783883068 shutdown_connections 2023-09-30T22:31:16.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.186+0000 7f2996d45700 1 -- 172.21.15.16:0/2783883068 wait complete. 2023-09-30T22:31:16.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.187+0000 7f2996d45700 1 Processor -- start 2023-09-30T22:31:16.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.187+0000 7f2996d45700 1 -- start start 2023-09-30T22:31:16.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.188+0000 7f2996d45700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f299009d6f0 0x7f299011e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:16.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.188+0000 7f2996d45700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f299009e1d0 0x7f299011ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:16.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.188+0000 7f2996d45700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2990139170 0x7f299011f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:16.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.188+0000 7f2996d45700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f299009c710 con 0x7f2990139170 2023-09-30T22:31:16.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.188+0000 7f2996d45700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2990122050 con 0x7f299009e1d0 2023-09-30T22:31:16.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.188+0000 7f2996d45700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2990121aa0 con 0x7f299009d6f0 2023-09-30T22:31:16.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.188+0000 7f298ffff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f299009e1d0 0x7f299011ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:16.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.188+0000 7f298ffff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f299009e1d0 0x7f299011ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:35454/0 (socket says 172.21.15.16:35454) 2023-09-30T22:31:16.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.188+0000 7f298ffff700 1 -- 172.21.15.16:0/4232610995 learned_addr learned my addr 172.21.15.16:0/4232610995 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:16.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.188+0000 7f2994ae1700 1 --2- 172.21.15.16:0/4232610995 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f299009d6f0 0x7f299011e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:16.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.188+0000 7f29952e2700 1 --2- 172.21.15.16:0/4232610995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2990139170 0x7f299011f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:16.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.189+0000 7f298ffff700 1 -- 172.21.15.16:0/4232610995 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f299009d6f0 msgr2=0x7f299011e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:16.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.189+0000 7f298ffff700 1 --2- 172.21.15.16:0/4232610995 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f299009d6f0 0x7f299011e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.189+0000 7f298ffff700 1 -- 172.21.15.16:0/4232610995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2990139170 msgr2=0x7f299011f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:16.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.189+0000 7f298ffff700 1 --2- 172.21.15.16:0/4232610995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2990139170 0x7f299011f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.189+0000 7f298ffff700 1 -- 172.21.15.16:0/4232610995 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2988009a70 con 0x7f299009e1d0 2023-09-30T22:31:16.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.189+0000 7f298ffff700 1 --2- 172.21.15.16:0/4232610995 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f299009e1d0 0x7f299011ee20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f297c00e810 tx=0x7f297c00eb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:16.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.189+0000 7f298d7fa700 1 -- 172.21.15.16:0/4232610995 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297c00c070 con 0x7f299009e1d0 2023-09-30T22:31:16.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.189+0000 7f298d7fa700 1 -- 172.21.15.16:0/4232610995 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f297c013610 con 0x7f299009e1d0 2023-09-30T22:31:16.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.189+0000 7f298d7fa700 1 -- 172.21.15.16:0/4232610995 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297c0139c0 con 0x7f299009e1d0 2023-09-30T22:31:16.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.190+0000 7f2996d45700 1 -- 172.21.15.16:0/4232610995 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2990071eb0 con 0x7f299009e1d0 2023-09-30T22:31:16.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.190+0000 7f2996d45700 1 -- 172.21.15.16:0/4232610995 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2990120c10 con 0x7f299009e1d0 2023-09-30T22:31:16.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.192+0000 7f298d7fa700 1 -- 172.21.15.16:0/4232610995 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f297c013b20 con 0x7f299009e1d0 2023-09-30T22:31:16.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.192+0000 7f2996d45700 1 -- 172.21.15.16:0/4232610995 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2990134760 con 0x7f299009e1d0 2023-09-30T22:31:16.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.192+0000 7f298d7fa700 1 -- 172.21.15.16:0/4232610995 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f297c03f460 con 0x7f299009e1d0 2023-09-30T22:31:16.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.197+0000 7f298d7fa700 1 -- 172.21.15.16:0/4232610995 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2990134760 con 0x7f299009e1d0 2023-09-30T22:31:16.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.287+0000 7fc6b9fa3700 1 Processor -- start 2023-09-30T22:31:16.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.287+0000 7fc6b9fa3700 1 -- start start 2023-09-30T22:31:16.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.288+0000 7fc6b9fa3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc6b4133420 0x7fc6b41337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:16.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.288+0000 7fc6b9fa3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc6b4133e30 0x7fc6b412ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:16.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.288+0000 7fc6b9fa3700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6b406b990 con 0x7fc6b4134260 2023-09-30T22:31:16.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.288+0000 7fc6b9fa3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b40a0590 con 0x7fc6b4133e30 2023-09-30T22:31:16.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.288+0000 7fc6b9fa3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b409fb80 con 0x7fc6b4133420 2023-09-30T22:31:16.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.288+0000 7fc6b37fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc6b4133420 0x7fc6b41337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:16.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.288+0000 7fc6b2ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc6b4133e30 0x7fc6b412ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:16.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.288+0000 7fc6b37fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc6b4133420 0x7fc6b41337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:60080/0 (socket says 172.21.15.16:60080) 2023-09-30T22:31:16.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.288+0000 7fc6b2ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc6b4133e30 0x7fc6b412ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:35466/0 (socket says 172.21.15.16:35466) 2023-09-30T22:31:16.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.288+0000 7fc6b37fe700 1 -- 172.21.15.16:0/3003274939 learned_addr learned my addr 172.21.15.16:0/3003274939 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:16.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.289+0000 7fc6b37fe700 1 -- 172.21.15.16:0/3003274939 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc6b4133e30 msgr2=0x7fc6b412ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:16.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.289+0000 7fc6b37fe700 1 --2- 172.21.15.16:0/3003274939 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc6b4133e30 0x7fc6b412ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.289+0000 7fc6b37fe700 1 -- 172.21.15.16:0/3003274939 >> v1:172.21.15.16:6789/0 conn(0x7fc6b4134260 legacy=0x7fc6b41304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:16.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.289+0000 7fc6b37fe700 1 -- 172.21.15.16:0/3003274939 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6b40a0030 con 0x7fc6b4133420 2023-09-30T22:31:16.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.289+0000 7fc6b37fe700 1 --2- 172.21.15.16:0/3003274939 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc6b4133420 0x7fc6b41337f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4009920 tx=0x7fc6a4009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f8feb684510e7bc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:16.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.289+0000 7fc6b27fc700 1 -- 172.21.15.16:0/3003274939 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a4011830 con 0x7fc6b4133420 2023-09-30T22:31:16.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.289+0000 7fc6b27fc700 1 -- 172.21.15.16:0/3003274939 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6a4011990 con 0x7fc6b4133420 2023-09-30T22:31:16.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.289+0000 7fc6b27fc700 1 -- 172.21.15.16:0/3003274939 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a4011d20 con 0x7fc6b4133420 2023-09-30T22:31:16.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.290+0000 7fc6b9fa3700 1 -- 172.21.15.16:0/3003274939 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc6b4133420 msgr2=0x7fc6b41337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:16.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.290+0000 7fc6b9fa3700 1 --2- 172.21.15.16:0/3003274939 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc6b4133420 0x7fc6b41337f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4009920 tx=0x7fc6a4009c30 comp rx=0 tx=0).stop 2023-09-30T22:31:16.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.290+0000 7fc6b9fa3700 1 -- 172.21.15.16:0/3003274939 shutdown_connections 2023-09-30T22:31:16.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.290+0000 7fc6b9fa3700 1 --2- 172.21.15.16:0/3003274939 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc6b4133420 0x7fc6b41337f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.290+0000 7fc6b9fa3700 1 --2- 172.21.15.16:0/3003274939 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc6b4133e30 0x7fc6b412ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.290+0000 7fc6b9fa3700 1 -- 172.21.15.16:0/3003274939 >> 172.21.15.16:0/3003274939 conn(0x7fc6b40a4210 msgr2=0x7fc6b40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:16.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.290+0000 7fc6b9fa3700 1 -- 172.21.15.16:0/3003274939 shutdown_connections 2023-09-30T22:31:16.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.290+0000 7fc6b9fa3700 1 -- 172.21.15.16:0/3003274939 wait complete. 2023-09-30T22:31:16.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.291+0000 7fc6b9fa3700 1 Processor -- start 2023-09-30T22:31:16.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.291+0000 7fc6b9fa3700 1 -- start start 2023-09-30T22:31:16.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.292+0000 7fc6b9fa3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc6b4133420 0x7fc6b409c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:16.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.292+0000 7fc6b9fa3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc6b4133e30 0x7fc6b409c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:16.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.292+0000 7fc6b9fa3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6b4134260 0x7fc6b409cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:16.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.292+0000 7fc6b9fa3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b411ff90 con 0x7fc6b4134260 2023-09-30T22:31:16.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.292+0000 7fc6b9fa3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b4120170 con 0x7fc6b4133420 2023-09-30T22:31:16.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.292+0000 7fc6b9fa3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b412d940 con 0x7fc6b4133e30 2023-09-30T22:31:16.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.292+0000 7fc6b2ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc6b4133e30 0x7fc6b409c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:16.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.292+0000 7fc6b37fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc6b4133420 0x7fc6b409c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:16.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.292+0000 7fc6b2ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc6b4133e30 0x7fc6b409c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:60094/0 (socket says 172.21.15.16:60094) 2023-09-30T22:31:16.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.292+0000 7fc6b2ffd700 1 -- 172.21.15.16:0/1361776305 learned_addr learned my addr 172.21.15.16:0/1361776305 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:16.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.292+0000 7fc6b3fff700 1 --2- 172.21.15.16:0/1361776305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6b4134260 0x7fc6b409cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:16.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.293+0000 7fc6b37fe700 1 -- 172.21.15.16:0/1361776305 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc6b4133e30 msgr2=0x7fc6b409c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:16.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.293+0000 7fc6b37fe700 1 --2- 172.21.15.16:0/1361776305 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc6b4133e30 0x7fc6b409c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.293+0000 7fc6b37fe700 1 -- 172.21.15.16:0/1361776305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6b4134260 msgr2=0x7fc6b409cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:16.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.293+0000 7fc6b37fe700 1 --2- 172.21.15.16:0/1361776305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6b4134260 0x7fc6b409cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.293+0000 7fc6b37fe700 1 -- 172.21.15.16:0/1361776305 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6a4009550 con 0x7fc6b4133420 2023-09-30T22:31:16.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.293+0000 7fc6b37fe700 1 --2- 172.21.15.16:0/1361776305 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc6b4133420 0x7fc6b409c480 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4016000 tx=0x7fc6a4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:16.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.293+0000 7fc69bfff700 1 -- 172.21.15.16:0/1361776305 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a4004450 con 0x7fc6b4133420 2023-09-30T22:31:16.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.293+0000 7fc69bfff700 1 -- 172.21.15.16:0/1361776305 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6a40045b0 con 0x7fc6b4133420 2023-09-30T22:31:16.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.293+0000 7fc6b9fa3700 1 -- 172.21.15.16:0/1361776305 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6b409d970 con 0x7fc6b4133420 2023-09-30T22:31:16.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.293+0000 7fc69bfff700 1 -- 172.21.15.16:0/1361776305 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a40038c0 con 0x7fc6b4133420 2023-09-30T22:31:16.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.293+0000 7fc6b9fa3700 1 -- 172.21.15.16:0/1361776305 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6b411f400 con 0x7fc6b4133420 2023-09-30T22:31:16.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.294+0000 7fc69bfff700 1 -- 172.21.15.16:0/1361776305 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc6a4003a20 con 0x7fc6b4133420 2023-09-30T22:31:16.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.295+0000 7fc6b9fa3700 1 -- 172.21.15.16:0/1361776305 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6b411fdb0 con 0x7fc6b4133420 2023-09-30T22:31:16.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.295+0000 7fc69bfff700 1 -- 172.21.15.16:0/1361776305 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fc6a4046060 con 0x7fc6b4133420 2023-09-30T22:31:16.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.304+0000 7fc69bfff700 1 -- 172.21.15.16:0/1361776305 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc6a401a430 con 0x7fc6b4133420 2023-09-30T22:31:16.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.361+0000 7f040a5fe700 1 -- 172.21.15.16:0/1360681320 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f040409f420 con 0x7f040409d6f0 2023-09-30T22:31:16.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.361+0000 7f0400ff9700 1 -- 172.21.15.16:0/1360681320 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f03f4013720 con 0x7f040409d6f0 2023-09-30T22:31:16.363 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:31:16.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.365+0000 7f040a5fe700 1 -- 172.21.15.16:0/1360681320 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f040409d6f0 msgr2=0x7f04041a5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:16.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.365+0000 7f040a5fe700 1 --2- 172.21.15.16:0/1360681320 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f040409d6f0 0x7f04041a5ed0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f03f400ec10 tx=0x7f03f400ef20 comp rx=0 tx=0).stop 2023-09-30T22:31:16.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.365+0000 7f040a5fe700 1 -- 172.21.15.16:0/1360681320 shutdown_connections 2023-09-30T22:31:16.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.365+0000 7f040a5fe700 1 --2- 172.21.15.16:0/1360681320 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f040409d6f0 0x7f04041a5ed0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.365+0000 7f040a5fe700 1 --2- 172.21.15.16:0/1360681320 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f040409e440 0x7f04041a6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.365+0000 7f040a5fe700 1 --2- 172.21.15.16:0/1360681320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f040409e000 0x7f04041a6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.365+0000 7f040a5fe700 1 -- 172.21.15.16:0/1360681320 >> 172.21.15.16:0/1360681320 conn(0x7f0404066f50 msgr2=0x7f04041baf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:16.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.365+0000 7f040a5fe700 1 -- 172.21.15.16:0/1360681320 shutdown_connections 2023-09-30T22:31:16.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.365+0000 7f040a5fe700 1 -- 172.21.15.16:0/1360681320 wait complete. 2023-09-30T22:31:16.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.379+0000 7fca4fa04700 1 -- 172.21.15.16:0/3172159617 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fca4806c360 con 0x7fca4809e1d0 2023-09-30T22:31:16.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.379+0000 7fca3dffb700 1 -- 172.21.15.16:0/3172159617 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fca38017020 con 0x7fca4809e1d0 2023-09-30T22:31:16.382 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:31:16.382 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:31:16.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.382+0000 7fca4fa04700 1 -- 172.21.15.16:0/3172159617 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca4809e1d0 msgr2=0x7fca48121b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:16.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.382+0000 7fca4fa04700 1 --2- 172.21.15.16:0/3172159617 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca4809e1d0 0x7fca48121b80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fca3800e9b0 tx=0x7fca3800ecc0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.382+0000 7fca4fa04700 1 -- 172.21.15.16:0/3172159617 shutdown_connections 2023-09-30T22:31:16.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.382+0000 7fca4fa04700 1 --2- 172.21.15.16:0/3172159617 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fca48139170 0x7fca481220c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.382+0000 7fca4fa04700 1 --2- 172.21.15.16:0/3172159617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca4809d6f0 0x7fca48121640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.382+0000 7fca4fa04700 1 --2- 172.21.15.16:0/3172159617 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca4809e1d0 0x7fca48121b80 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.382+0000 7fca4fa04700 1 -- 172.21.15.16:0/3172159617 >> 172.21.15.16:0/3172159617 conn(0x7fca48066f50 msgr2=0x7fca48070c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:16.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.382+0000 7fca4fa04700 1 -- 172.21.15.16:0/3172159617 shutdown_connections 2023-09-30T22:31:16.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.382+0000 7fca4fa04700 1 -- 172.21.15.16:0/3172159617 wait complete. 2023-09-30T22:31:16.397 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:31:16.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.424+0000 7f2996d45700 1 -- 172.21.15.16:0/4232610995 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f2990134760 con 0x7f299009e1d0 2023-09-30T22:31:16.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.425+0000 7f298d7fa700 1 -- 172.21.15.16:0/4232610995 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2990134760 con 0x7f299009e1d0 2023-09-30T22:31:16.426 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:31:16.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.427+0000 7f2996d45700 1 -- 172.21.15.16:0/4232610995 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f299009e1d0 msgr2=0x7f299011ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:16.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.427+0000 7f2996d45700 1 --2- 172.21.15.16:0/4232610995 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f299009e1d0 0x7f299011ee20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f297c00e810 tx=0x7f297c00eb20 comp rx=0 tx=0).stop 2023-09-30T22:31:16.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.427+0000 7f2996d45700 1 -- 172.21.15.16:0/4232610995 shutdown_connections 2023-09-30T22:31:16.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.427+0000 7f2996d45700 1 --2- 172.21.15.16:0/4232610995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2990139170 0x7f299011f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.427+0000 7f2996d45700 1 --2- 172.21.15.16:0/4232610995 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f299009d6f0 0x7f299011e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.427+0000 7f2996d45700 1 --2- 172.21.15.16:0/4232610995 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f299009e1d0 0x7f299011ee20 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.427+0000 7f2996d45700 1 -- 172.21.15.16:0/4232610995 >> 172.21.15.16:0/4232610995 conn(0x7f2990066f50 msgr2=0x7f2990136b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:16.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.427+0000 7f2996d45700 1 -- 172.21.15.16:0/4232610995 shutdown_connections 2023-09-30T22:31:16.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.427+0000 7f2996d45700 1 -- 172.21.15.16:0/4232610995 wait complete. 2023-09-30T22:31:16.442 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:31:16.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.512+0000 7fc6b9fa3700 1 -- 172.21.15.16:0/1361776305 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc6b4074ea0 con 0x7fc6b4133420 2023-09-30T22:31:16.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.513+0000 7fc69bfff700 1 -- 172.21.15.16:0/1361776305 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc6a4015030 con 0x7fc6b4133420 2023-09-30T22:31:16.514 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:31:16.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.515+0000 7fc6b9fa3700 1 -- 172.21.15.16:0/1361776305 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc6b4133420 msgr2=0x7fc6b409c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:16.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.515+0000 7fc6b9fa3700 1 --2- 172.21.15.16:0/1361776305 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc6b4133420 0x7fc6b409c480 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4016000 tx=0x7fc6a4004750 comp rx=0 tx=0).stop 2023-09-30T22:31:16.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.515+0000 7fc6b9fa3700 1 -- 172.21.15.16:0/1361776305 shutdown_connections 2023-09-30T22:31:16.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.515+0000 7fc6b9fa3700 1 --2- 172.21.15.16:0/1361776305 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc6b4133420 0x7fc6b409c480 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.515+0000 7fc6b9fa3700 1 --2- 172.21.15.16:0/1361776305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6b4134260 0x7fc6b409cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.515+0000 7fc6b9fa3700 1 --2- 172.21.15.16:0/1361776305 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc6b4133e30 0x7fc6b409c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:16.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.515+0000 7fc6b9fa3700 1 -- 172.21.15.16:0/1361776305 >> 172.21.15.16:0/1361776305 conn(0x7fc6b40a4210 msgr2=0x7fc6b40a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:16.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.515+0000 7fc6b9fa3700 1 -- 172.21.15.16:0/1361776305 shutdown_connections 2023-09-30T22:31:16.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:16.515+0000 7fc6b9fa3700 1 -- 172.21.15.16:0/1361776305 wait complete. 2023-09-30T22:31:16.530 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:31:17.385 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:17.399 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:17.443 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:17.530 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:17.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.543+0000 7fd45ba1a700 1 Processor -- start 2023-09-30T22:31:17.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.543+0000 7fd45ba1a700 1 -- start start 2023-09-30T22:31:17.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.543+0000 7fd45ba1a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd454131810 0x7fd454131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:17.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.543+0000 7fd45ba1a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd45409a580 0x7fd454139660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:17.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.543+0000 7fd45ba1a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4541313c0 con 0x7fd454099a40 2023-09-30T22:31:17.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.543+0000 7fd45ba1a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd45406e8c0 con 0x7fd454131810 2023-09-30T22:31:17.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.544+0000 7fd45ba1a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd454071d20 con 0x7fd45409a580 2023-09-30T22:31:17.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.544+0000 7fd4597b6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd454131810 0x7fd454131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:17.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.544+0000 7fd4597b6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd454131810 0x7fd454131be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:57178/0 (socket says 172.21.15.16:57178) 2023-09-30T22:31:17.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.544+0000 7fd4597b6700 1 -- 172.21.15.16:0/320721419 learned_addr learned my addr 172.21.15.16:0/320721419 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:17.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.544+0000 7fd459fb7700 1 --2- 172.21.15.16:0/320721419 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd45409a580 0x7fd454139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:17.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.544+0000 7fd459fb7700 1 -- 172.21.15.16:0/320721419 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd454131810 msgr2=0x7fd454131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:17.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.544+0000 7fd459fb7700 1 --2- 172.21.15.16:0/320721419 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd454131810 0x7fd454131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:17.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.544+0000 7fd459fb7700 1 -- 172.21.15.16:0/320721419 >> v1:172.21.15.16:6789/0 conn(0x7fd454099a40 legacy=0x7fd454099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:17.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.544+0000 7fd459fb7700 1 -- 172.21.15.16:0/320721419 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4540721a0 con 0x7fd45409a580 2023-09-30T22:31:17.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.544+0000 7fd459fb7700 1 --2- 172.21.15.16:0/320721419 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd45409a580 0x7fd454139660 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd44c009dc0 tx=0x7fd44c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a224683b7fdf4735 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:17.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.545+0000 7fd44bfff700 1 -- 172.21.15.16:0/320721419 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44c00e070 con 0x7fd45409a580 2023-09-30T22:31:17.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.545+0000 7fd44bfff700 1 -- 172.21.15.16:0/320721419 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd44c00bae0 con 0x7fd45409a580 2023-09-30T22:31:17.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.545+0000 7fd44bfff700 1 -- 172.21.15.16:0/320721419 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44c00be90 con 0x7fd45409a580 2023-09-30T22:31:17.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.545+0000 7fd45ba1a700 1 -- 172.21.15.16:0/320721419 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd45409a580 msgr2=0x7fd454139660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:17.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.545+0000 7fd45ba1a700 1 --2- 172.21.15.16:0/320721419 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd45409a580 0x7fd454139660 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd44c009dc0 tx=0x7fd44c00b330 comp rx=0 tx=0).stop 2023-09-30T22:31:17.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.546+0000 7fd45ba1a700 1 -- 172.21.15.16:0/320721419 shutdown_connections 2023-09-30T22:31:17.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.546+0000 7fd45ba1a700 1 --2- 172.21.15.16:0/320721419 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd45409a580 0x7fd454139660 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:17.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.546+0000 7fd45ba1a700 1 --2- 172.21.15.16:0/320721419 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd454131810 0x7fd454131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:17.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.546+0000 7fd45ba1a700 1 -- 172.21.15.16:0/320721419 >> 172.21.15.16:0/320721419 conn(0x7fd4540a2aa0 msgr2=0x7fd4540a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:17.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.546+0000 7fd45ba1a700 1 -- 172.21.15.16:0/320721419 shutdown_connections 2023-09-30T22:31:17.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.546+0000 7fd45ba1a700 1 -- 172.21.15.16:0/320721419 wait complete. 2023-09-30T22:31:17.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.547+0000 7fd45ba1a700 1 Processor -- start 2023-09-30T22:31:17.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.547+0000 7fd45ba1a700 1 -- start start 2023-09-30T22:31:17.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.548+0000 7fd45ba1a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd454099a40 0x7fd45411b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:17.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.548+0000 7fd45ba1a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd45409a580 0x7fd45411ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:17.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.548+0000 7fd45ba1a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd454131810 0x7fd45411bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:17.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.548+0000 7fd45ba1a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45411ca00 con 0x7fd45409a580 2023-09-30T22:31:17.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.548+0000 7fd45ba1a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd45411f9c0 con 0x7fd454131810 2023-09-30T22:31:17.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.548+0000 7fd45ba1a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd45411f000 con 0x7fd454099a40 2023-09-30T22:31:17.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.548+0000 7fd459fb7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd454131810 0x7fd45411bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:17.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.548+0000 7fd4597b6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd454099a40 0x7fd45411b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:17.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.548+0000 7fd459fb7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd454131810 0x7fd45411bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:57186/0 (socket says 172.21.15.16:57186) 2023-09-30T22:31:17.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.548+0000 7fd459fb7700 1 -- 172.21.15.16:0/1346757032 learned_addr learned my addr 172.21.15.16:0/1346757032 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:17.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.548+0000 7fd458fb5700 1 --2- 172.21.15.16:0/1346757032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd45409a580 0x7fd45411ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:17.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.549+0000 7fd4597b6700 1 -- 172.21.15.16:0/1346757032 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd454131810 msgr2=0x7fd45411bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:17.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.549+0000 7fd4597b6700 1 --2- 172.21.15.16:0/1346757032 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd454131810 0x7fd45411bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:17.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.549+0000 7fd4597b6700 1 -- 172.21.15.16:0/1346757032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd45409a580 msgr2=0x7fd45411ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:17.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.549+0000 7fd4597b6700 1 --2- 172.21.15.16:0/1346757032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd45409a580 0x7fd45411ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:17.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.549+0000 7fd4597b6700 1 -- 172.21.15.16:0/1346757032 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd44c009a70 con 0x7fd454099a40 2023-09-30T22:31:17.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.549+0000 7fd4597b6700 1 --2- 172.21.15.16:0/1346757032 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd454099a40 0x7fd45411b530 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd45000e910 tx=0x7fd45000ec20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:17.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.549+0000 7fd459fb7700 1 --2- 172.21.15.16:0/1346757032 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd454131810 0x7fd45411bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:17.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.549+0000 7fd449ffb700 1 -- 172.21.15.16:0/1346757032 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd450013680 con 0x7fd454099a40 2023-09-30T22:31:17.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.549+0000 7fd449ffb700 1 -- 172.21.15.16:0/1346757032 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4500137e0 con 0x7fd454099a40 2023-09-30T22:31:17.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.549+0000 7fd45ba1a700 1 -- 172.21.15.16:0/1346757032 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd45411d190 con 0x7fd454099a40 2023-09-30T22:31:17.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.549+0000 7fd449ffb700 1 -- 172.21.15.16:0/1346757032 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd450013b50 con 0x7fd454099a40 2023-09-30T22:31:17.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.550+0000 7fd45ba1a700 1 -- 172.21.15.16:0/1346757032 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4540a1ea0 con 0x7fd454099a40 2023-09-30T22:31:17.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.551+0000 7fd45ba1a700 1 -- 172.21.15.16:0/1346757032 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd45411e010 con 0x7fd454099a40 2023-09-30T22:31:17.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.551+0000 7fd449ffb700 1 -- 172.21.15.16:0/1346757032 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd450013cb0 con 0x7fd454099a40 2023-09-30T22:31:17.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.551+0000 7fd449ffb700 1 -- 172.21.15.16:0/1346757032 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd45003f650 con 0x7fd454099a40 2023-09-30T22:31:17.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.560+0000 7fd449ffb700 1 -- 172.21.15.16:0/1346757032 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd45001be10 con 0x7fd454099a40 2023-09-30T22:31:17.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.585+0000 7f495725f700 1 Processor -- start 2023-09-30T22:31:17.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.585+0000 7f495725f700 1 -- start start 2023-09-30T22:31:17.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.585+0000 7f495725f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f49500a0a30 0x7f49500a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:17.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.585+0000 7f495725f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4950134170 0x7f495013bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:17.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.585+0000 7f495725f700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f495009edd0 con 0x7f49500a1340 2023-09-30T22:31:17.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.585+0000 7f495725f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f495009e1f0 con 0x7f49500a0a30 2023-09-30T22:31:17.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.585+0000 7f495725f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f495009dc50 con 0x7f4950134170 2023-09-30T22:31:17.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.586+0000 7f49557fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4950134170 0x7f495013bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:17.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.586+0000 7f49557fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4950134170 0x7f495013bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:38388/0 (socket says 172.21.15.16:38388) 2023-09-30T22:31:17.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.586+0000 7f49557fc700 1 -- 172.21.15.16:0/679866716 learned_addr learned my addr 172.21.15.16:0/679866716 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:17.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.586+0000 7f4954ffb700 1 --2- 172.21.15.16:0/679866716 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f49500a0a30 0x7f49500a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:17.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.586+0000 7f49557fc700 1 -- 172.21.15.16:0/679866716 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f49500a0a30 msgr2=0x7f49500a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:17.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.586+0000 7f49557fc700 1 --2- 172.21.15.16:0/679866716 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f49500a0a30 0x7f49500a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:17.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.586+0000 7f49557fc700 1 -- 172.21.15.16:0/679866716 >> v1:172.21.15.16:6789/0 conn(0x7f49500a1340 legacy=0x7f4950133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:17.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.586+0000 7f49557fc700 1 -- 172.21.15.16:0/679866716 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49500a1fe0 con 0x7f4950134170 2023-09-30T22:31:17.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.586+0000 7f4954ffb700 1 --2- 172.21.15.16:0/679866716 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f49500a0a30 0x7f49500a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:31:17.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.586+0000 7f49557fc700 1 --2- 172.21.15.16:0/679866716 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4950134170 0x7f495013bd30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4948009e80 tx=0x7f494800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be66eb01b404c513 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:17.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.587+0000 7f494f7fe700 1 -- 172.21.15.16:0/679866716 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494800e070 con 0x7f4950134170 2023-09-30T22:31:17.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.587+0000 7f494f7fe700 1 -- 172.21.15.16:0/679866716 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f494800b9c0 con 0x7f4950134170 2023-09-30T22:31:17.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.587+0000 7f494f7fe700 1 -- 172.21.15.16:0/679866716 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494800bd70 con 0x7f4950134170 2023-09-30T22:31:17.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.587+0000 7f495725f700 1 -- 172.21.15.16:0/679866716 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4950134170 msgr2=0x7f495013bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:17.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.587+0000 7f495725f700 1 --2- 172.21.15.16:0/679866716 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4950134170 0x7f495013bd30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4948009e80 tx=0x7f494800b330 comp rx=0 tx=0).stop 2023-09-30T22:31:17.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.587+0000 7f495725f700 1 -- 172.21.15.16:0/679866716 shutdown_connections 2023-09-30T22:31:17.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.587+0000 7f495725f700 1 --2- 172.21.15.16:0/679866716 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4950134170 0x7f495013bd30 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:17.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.587+0000 7f495725f700 1 --2- 172.21.15.16:0/679866716 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f49500a0a30 0x7f49500a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:17.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.587+0000 7f495725f700 1 -- 172.21.15.16:0/679866716 >> 172.21.15.16:0/679866716 conn(0x7f49500a6550 msgr2=0x7f49500a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:17.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.587+0000 7f495725f700 1 -- 172.21.15.16:0/679866716 shutdown_connections 2023-09-30T22:31:17.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.588+0000 7f495725f700 1 -- 172.21.15.16:0/679866716 wait complete. 2023-09-30T22:31:17.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.588+0000 7f495725f700 1 Processor -- start 2023-09-30T22:31:17.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.589+0000 7f495725f700 1 -- start start 2023-09-30T22:31:17.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.589+0000 7f495725f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f49500a0a30 0x7f495011e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:17.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.589+0000 7f495725f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f49500a1340 0x7f495011eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:17.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.589+0000 7f495725f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4950134170 0x7f495011f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:17.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.589+0000 7f495725f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4950120b00 con 0x7f4950134170 2023-09-30T22:31:17.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.589+0000 7f495725f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4950120230 con 0x7f49500a1340 2023-09-30T22:31:17.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.589+0000 7f495725f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4950132fb0 con 0x7f49500a0a30 2023-09-30T22:31:17.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.589+0000 7f494ffff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f49500a1340 0x7f495011eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:17.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.589+0000 7f4954ffb700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f49500a0a30 0x7f495011e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:17.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.589+0000 7f494ffff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f49500a1340 0x7f495011eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:57208/0 (socket says 172.21.15.16:57208) 2023-09-30T22:31:17.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.589+0000 7f4954ffb700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f49500a0a30 0x7f495011e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:38398/0 (socket says 172.21.15.16:38398) 2023-09-30T22:31:17.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.589+0000 7f494ffff700 1 -- 172.21.15.16:0/1034332306 learned_addr learned my addr 172.21.15.16:0/1034332306 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:17.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.590+0000 7f49557fc700 1 --2- 172.21.15.16:0/1034332306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4950134170 0x7f495011f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:17.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.590+0000 7f494ffff700 1 -- 172.21.15.16:0/1034332306 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f49500a0a30 msgr2=0x7f495011e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:17.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.590+0000 7f494ffff700 1 --2- 172.21.15.16:0/1034332306 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f49500a0a30 0x7f495011e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:17.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.590+0000 7f494ffff700 1 -- 172.21.15.16:0/1034332306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4950134170 msgr2=0x7f495011f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:17.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.590+0000 7f494ffff700 1 --2- 172.21.15.16:0/1034332306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4950134170 0x7f495011f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:17.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.590+0000 7f494ffff700 1 -- 172.21.15.16:0/1034332306 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4948009b30 con 0x7f49500a1340 2023-09-30T22:31:17.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.590+0000 7f494ffff700 1 --2- 172.21.15.16:0/1034332306 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f49500a1340 0x7f495011eb60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f493c00c4e0 tx=0x7f493c00c7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:17.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.590+0000 7f494d7fa700 1 -- 172.21.15.16:0/1034332306 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493c0052d0 con 0x7f49500a1340 2023-09-30T22:31:17.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.590+0000 7f494d7fa700 1 -- 172.21.15.16:0/1034332306 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f493c005430 con 0x7f49500a1340 2023-09-30T22:31:17.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.590+0000 7f495725f700 1 -- 172.21.15.16:0/1034332306 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49500a25e0 con 0x7f49500a1340 2023-09-30T22:31:17.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.590+0000 7f494d7fa700 1 -- 172.21.15.16:0/1034332306 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493c0038c0 con 0x7f49500a1340 2023-09-30T22:31:17.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.590+0000 7f495725f700 1 -- 172.21.15.16:0/1034332306 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49501335b0 con 0x7f49500a1340 2023-09-30T22:31:17.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.591+0000 7f494d7fa700 1 -- 172.21.15.16:0/1034332306 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f493c003a20 con 0x7f49500a1340 2023-09-30T22:31:17.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.591+0000 7f494d7fa700 1 -- 172.21.15.16:0/1034332306 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f493c03ea30 con 0x7f49500a1340 2023-09-30T22:31:17.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.592+0000 7f495725f700 1 -- 172.21.15.16:0/1034332306 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4950121a80 con 0x7f49500a1340 2023-09-30T22:31:17.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.597+0000 7f494d7fa700 1 -- 172.21.15.16:0/1034332306 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f493c017080 con 0x7f49500a1340 2023-09-30T22:31:17.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.651+0000 7fa6e0623700 1 Processor -- start 2023-09-30T22:31:17.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.651+0000 7fa6e0623700 1 -- start start 2023-09-30T22:31:17.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.652+0000 7fa6e0623700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa6d80a0a30 0x7fa6d80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:17.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.652+0000 7fa6e0623700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa6d809d630 0x7fa6d80a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:17.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.652+0000 7fa6e0623700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa6d8133930 0x7fa6d8133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:17.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.652+0000 7fa6e0623700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6d80a2a70 con 0x7fa6d8133930 2023-09-30T22:31:17.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.652+0000 7fa6e0623700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d80a01d0 con 0x7fa6d809d630 2023-09-30T22:31:17.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.652+0000 7fa6e0623700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d809e2b0 con 0x7fa6d80a0a30 2023-09-30T22:31:17.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.653+0000 7fa6de3bf700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa6d80a0a30 0x7fa6d80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:17.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.653+0000 7fa6debc0700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa6d8133930 0x7fa6d8133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:17.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.653+0000 7fa6de3bf700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa6d80a0a30 0x7fa6d80a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:38408/0 (socket says 172.21.15.16:38408) 2023-09-30T22:31:17.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.653+0000 7fa6de3bf700 1 -- 172.21.15.16:0/718940329 learned_addr learned my addr 172.21.15.16:0/718940329 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:17.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.653+0000 7fa6ddbbe700 1 --2- 172.21.15.16:0/718940329 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa6d809d630 0x7fa6d80a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:17.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.653+0000 7fa6de3bf700 1 -- 172.21.15.16:0/718940329 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa6d809d630 msgr2=0x7fa6d80a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:17.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.653+0000 7fa6de3bf700 1 --2- 172.21.15.16:0/718940329 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa6d809d630 0x7fa6d80a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:17.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.653+0000 7fa6de3bf700 1 -- 172.21.15.16:0/718940329 >> v2:172.21.15.16:3300/0 conn(0x7fa6d8133930 msgr2=0x7fa6d8133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:17.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.653+0000 7fa6de3bf700 1 --2- 172.21.15.16:0/718940329 >> v2:172.21.15.16:3300/0 conn(0x7fa6d8133930 0x7fa6d8133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:17.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.653+0000 7fa6de3bf700 1 -- 172.21.15.16:0/718940329 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d809dc50 con 0x7fa6d80a0a30 2023-09-30T22:31:17.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.653+0000 7fa6debc0700 1 --2- 172.21.15.16:0/718940329 >> v2:172.21.15.16:3300/0 conn(0x7fa6d8133930 0x7fa6d8133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:17.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.653+0000 7fa6de3bf700 1 --2- 172.21.15.16:0/718940329 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa6d80a0a30 0x7fa6d80a0e00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa6d40098f0 tx=0x7fa6d4009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd85d2b9c386e75c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:17.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.654+0000 7fa6dd3bd700 1 -- 172.21.15.16:0/718940329 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d4011830 con 0x7fa6d80a0a30 2023-09-30T22:31:17.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.654+0000 7fa6dd3bd700 1 -- 172.21.15.16:0/718940329 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6d4011990 con 0x7fa6d80a0a30 2023-09-30T22:31:17.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.654+0000 7fa6dd3bd700 1 -- 172.21.15.16:0/718940329 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d4011d20 con 0x7fa6d80a0a30 2023-09-30T22:31:17.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.655+0000 7fa6e0623700 1 -- 172.21.15.16:0/718940329 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa6d80a0a30 msgr2=0x7fa6d80a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:17.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.655+0000 7fa6e0623700 1 --2- 172.21.15.16:0/718940329 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa6d80a0a30 0x7fa6d80a0e00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa6d40098f0 tx=0x7fa6d4009c00 comp rx=0 tx=0).stop 2023-09-30T22:31:17.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.655+0000 7fa6e0623700 1 -- 172.21.15.16:0/718940329 shutdown_connections 2023-09-30T22:31:17.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.655+0000 7fa6e0623700 1 --2- 172.21.15.16:0/718940329 >> v2:172.21.15.16:3300/0 conn(0x7fa6d8133930 0x7fa6d8133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:17.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.655+0000 7fa6e0623700 1 --2- 172.21.15.16:0/718940329 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa6d80a0a30 0x7fa6d80a0e00 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:17.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.655+0000 7fa6e0623700 1 --2- 172.21.15.16:0/718940329 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa6d809d630 0x7fa6d80a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:17.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.655+0000 7fa6e0623700 1 -- 172.21.15.16:0/718940329 >> 172.21.15.16:0/718940329 conn(0x7fa6d80a3050 msgr2=0x7fa6d80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:17.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.655+0000 7fa6e0623700 1 -- 172.21.15.16:0/718940329 shutdown_connections 2023-09-30T22:31:17.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.655+0000 7fa6e0623700 1 -- 172.21.15.16:0/718940329 wait complete. 2023-09-30T22:31:17.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.656+0000 7fa6e0623700 1 Processor -- start 2023-09-30T22:31:17.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.656+0000 7fa6e0623700 1 -- start start 2023-09-30T22:31:17.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.657+0000 7fa6e0623700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa6d809d630 0x7fa6d811d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:17.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.657+0000 7fa6e0623700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa6d80a0a30 0x7fa6d811db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:17.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.657+0000 7fa6e0623700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6d8133930 0x7fa6d81c3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:17.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.657+0000 7fa6e0623700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d8120560 con 0x7fa6d8133930 2023-09-30T22:31:17.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.657+0000 7fa6e0623700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d811d2f0 con 0x7fa6d809d630 2023-09-30T22:31:17.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.657+0000 7fa6e0623700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d8071c40 con 0x7fa6d80a0a30 2023-09-30T22:31:17.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.657+0000 7fa6ddbbe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa6d80a0a30 0x7fa6d811db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:17.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.657+0000 7fa6de3bf700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa6d809d630 0x7fa6d811d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:17.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.657+0000 7fa6ddbbe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa6d80a0a30 0x7fa6d811db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:38420/0 (socket says 172.21.15.16:38420) 2023-09-30T22:31:17.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.657+0000 7fa6ddbbe700 1 -- 172.21.15.16:0/332906316 learned_addr learned my addr 172.21.15.16:0/332906316 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:17.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.657+0000 7fa6debc0700 1 --2- 172.21.15.16:0/332906316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6d8133930 0x7fa6d81c3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:17.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.658+0000 7fa6de3bf700 1 -- 172.21.15.16:0/332906316 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa6d80a0a30 msgr2=0x7fa6d811db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:17.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.658+0000 7fa6de3bf700 1 --2- 172.21.15.16:0/332906316 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa6d80a0a30 0x7fa6d811db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:17.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.658+0000 7fa6de3bf700 1 -- 172.21.15.16:0/332906316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6d8133930 msgr2=0x7fa6d81c3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:17.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.658+0000 7fa6de3bf700 1 --2- 172.21.15.16:0/332906316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6d8133930 0x7fa6d81c3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:17.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.658+0000 7fa6de3bf700 1 -- 172.21.15.16:0/332906316 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d0009a70 con 0x7fa6d809d630 2023-09-30T22:31:17.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.658+0000 7fa6de3bf700 1 --2- 172.21.15.16:0/332906316 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa6d809d630 0x7fa6d811d5f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4016000 tx=0x7fa6d4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:17.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.658+0000 7fa6ceffd700 1 -- 172.21.15.16:0/332906316 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d4004450 con 0x7fa6d809d630 2023-09-30T22:31:17.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.658+0000 7fa6ceffd700 1 -- 172.21.15.16:0/332906316 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6d40045b0 con 0x7fa6d809d630 2023-09-30T22:31:17.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.658+0000 7fa6e0623700 1 -- 172.21.15.16:0/332906316 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6d4009550 con 0x7fa6d809d630 2023-09-30T22:31:17.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.658+0000 7fa6ceffd700 1 -- 172.21.15.16:0/332906316 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d40038a0 con 0x7fa6d809d630 2023-09-30T22:31:17.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.658+0000 7fa6e0623700 1 -- 172.21.15.16:0/332906316 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6d811e070 con 0x7fa6d809d630 2023-09-30T22:31:17.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.660+0000 7fa6e0623700 1 -- 172.21.15.16:0/332906316 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6d80747f0 con 0x7fa6d809d630 2023-09-30T22:31:17.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.661+0000 7fa6ceffd700 1 -- 172.21.15.16:0/332906316 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa6d4021070 con 0x7fa6d809d630 2023-09-30T22:31:17.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.661+0000 7fa6ceffd700 1 -- 172.21.15.16:0/332906316 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fa6d40409c0 con 0x7fa6d809d630 2023-09-30T22:31:17.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.669+0000 7fa6ceffd700 1 -- 172.21.15.16:0/332906316 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa6d400b9d0 con 0x7fa6d809d630 2023-09-30T22:31:17.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.713+0000 7f63960c2700 1 Processor -- start 2023-09-30T22:31:17.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.713+0000 7f63960c2700 1 -- start start 2023-09-30T22:31:17.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.714+0000 7f63960c2700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f63900a0a30 0x7f63900a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:17.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.714+0000 7f63960c2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f63900a1340 0x7f6390133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:17.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.714+0000 7f63960c2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6390133fa0 0x7f6390134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:17.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.714+0000 7f63960c2700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f639009edd0 con 0x7f63900a0a30 2023-09-30T22:31:17.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.714+0000 7f63960c2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f639009e1f0 con 0x7f6390133fa0 2023-09-30T22:31:17.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.714+0000 7f63960c2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f639009dc50 con 0x7f63900a1340 2023-09-30T22:31:17.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.714+0000 7f638effd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f63900a1340 0x7f6390133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:17.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.714+0000 7f638effd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f63900a1340 0x7f6390133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:38424/0 (socket says 172.21.15.16:38424) 2023-09-30T22:31:17.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.714+0000 7f638effd700 1 -- 172.21.15.16:0/655061171 learned_addr learned my addr 172.21.15.16:0/655061171 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:17.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.714+0000 7f638f7fe700 1 --2- 172.21.15.16:0/655061171 >> v2:172.21.15.16:3300/0 conn(0x7f63900a0a30 0x7f63900a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:17.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.714+0000 7f638ffff700 1 --2- 172.21.15.16:0/655061171 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6390133fa0 0x7f6390134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:17.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.715+0000 7f638ffff700 1 -- 172.21.15.16:0/655061171 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f63900a1340 msgr2=0x7f6390133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:17.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.715+0000 7f638ffff700 1 --2- 172.21.15.16:0/655061171 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f63900a1340 0x7f6390133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:17.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.715+0000 7f638ffff700 1 -- 172.21.15.16:0/655061171 >> v2:172.21.15.16:3300/0 conn(0x7f63900a0a30 msgr2=0x7f63900a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:17.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.715+0000 7f638ffff700 1 --2- 172.21.15.16:0/655061171 >> v2:172.21.15.16:3300/0 conn(0x7f63900a0a30 0x7f63900a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:17.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.715+0000 7f638ffff700 1 -- 172.21.15.16:0/655061171 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63900a1fe0 con 0x7f6390133fa0 2023-09-30T22:31:17.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.715+0000 7f638f7fe700 1 --2- 172.21.15.16:0/655061171 >> v2:172.21.15.16:3300/0 conn(0x7f63900a0a30 0x7f63900a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:17.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.715+0000 7f638ffff700 1 --2- 172.21.15.16:0/655061171 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6390133fa0 0x7f6390134410 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6388009dc0 tx=0x7f638800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6ad5deb653025ae server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:17.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.715+0000 7f638e7fc700 1 -- 172.21.15.16:0/655061171 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638800e070 con 0x7f6390133fa0 2023-09-30T22:31:17.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.715+0000 7f638e7fc700 1 -- 172.21.15.16:0/655061171 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f638800bae0 con 0x7f6390133fa0 2023-09-30T22:31:17.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.715+0000 7f638e7fc700 1 -- 172.21.15.16:0/655061171 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638800be90 con 0x7f6390133fa0 2023-09-30T22:31:17.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.716+0000 7f63960c2700 1 -- 172.21.15.16:0/655061171 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6390133fa0 msgr2=0x7f6390134410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:17.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.716+0000 7f63960c2700 1 --2- 172.21.15.16:0/655061171 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6390133fa0 0x7f6390134410 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6388009dc0 tx=0x7f638800b330 comp rx=0 tx=0).stop 2023-09-30T22:31:17.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.716+0000 7f63960c2700 1 -- 172.21.15.16:0/655061171 shutdown_connections 2023-09-30T22:31:17.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.716+0000 7f63960c2700 1 --2- 172.21.15.16:0/655061171 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6390133fa0 0x7f6390134410 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:17.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.716+0000 7f63960c2700 1 --2- 172.21.15.16:0/655061171 >> v2:172.21.15.16:3300/0 conn(0x7f63900a0a30 0x7f63900a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:17.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.716+0000 7f63960c2700 1 --2- 172.21.15.16:0/655061171 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f63900a1340 0x7f6390133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:17.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.716+0000 7f63960c2700 1 -- 172.21.15.16:0/655061171 >> 172.21.15.16:0/655061171 conn(0x7f63900a6550 msgr2=0x7f63900a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:17.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.716+0000 7f63960c2700 1 -- 172.21.15.16:0/655061171 shutdown_connections 2023-09-30T22:31:17.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.716+0000 7f63960c2700 1 -- 172.21.15.16:0/655061171 wait complete. 2023-09-30T22:31:17.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.717+0000 7f63960c2700 1 Processor -- start 2023-09-30T22:31:17.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.717+0000 7f63960c2700 1 -- start start 2023-09-30T22:31:17.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.718+0000 7f63960c2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63900a0a30 0x7f6390120170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:17.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.718+0000 7f63960c2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f63900a1340 0x7f63901206b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:17.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.718+0000 7f63960c2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6390133fa0 0x7f6390120bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:17.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.718+0000 7f63960c2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6390121d30 con 0x7f63900a0a30 2023-09-30T22:31:17.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.718+0000 7f63960c2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6390122f80 con 0x7f63900a1340 2023-09-30T22:31:17.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.718+0000 7f63960c2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6390122320 con 0x7f6390133fa0 2023-09-30T22:31:17.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.718+0000 7f638f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63900a0a30 0x7f6390120170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:17.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.718+0000 7f638f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63900a0a30 0x7f6390120170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57034/0 (socket says 172.21.15.16:57034) 2023-09-30T22:31:17.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.718+0000 7f638f7fe700 1 -- 172.21.15.16:0/325308970 learned_addr learned my addr 172.21.15.16:0/325308970 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:17.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.718+0000 7f638ffff700 1 --2- 172.21.15.16:0/325308970 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6390133fa0 0x7f6390120bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:17.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.718+0000 7f638effd700 1 --2- 172.21.15.16:0/325308970 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f63900a1340 0x7f63901206b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:17.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.718+0000 7f638ffff700 1 -- 172.21.15.16:0/325308970 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f63900a1340 msgr2=0x7f63901206b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:17.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.718+0000 7f638ffff700 1 --2- 172.21.15.16:0/325308970 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f63900a1340 0x7f63901206b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:17.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.718+0000 7f638ffff700 1 -- 172.21.15.16:0/325308970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63900a0a30 msgr2=0x7f6390120170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:17.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.718+0000 7f638ffff700 1 --2- 172.21.15.16:0/325308970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63900a0a30 0x7f6390120170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:17.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.718+0000 7f638ffff700 1 -- 172.21.15.16:0/325308970 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63800094a0 con 0x7f6390133fa0 2023-09-30T22:31:17.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.718+0000 7f638effd700 1 --2- 172.21.15.16:0/325308970 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f63900a1340 0x7f63901206b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:31:17.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.718+0000 7f638f7fe700 1 --2- 172.21.15.16:0/325308970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63900a0a30 0x7f6390120170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:17.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.719+0000 7f638ffff700 1 --2- 172.21.15.16:0/325308970 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6390133fa0 0x7f6390120bf0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f639006c3a0 tx=0x7f6388004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:17.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.719+0000 7f6377fff700 1 -- 172.21.15.16:0/325308970 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6388004550 con 0x7f6390133fa0 2023-09-30T22:31:17.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.719+0000 7f63960c2700 1 -- 172.21.15.16:0/325308970 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6388009a70 con 0x7f6390133fa0 2023-09-30T22:31:17.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.719+0000 7f6377fff700 1 -- 172.21.15.16:0/325308970 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6388020070 con 0x7f6390133fa0 2023-09-30T22:31:17.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.719+0000 7f6377fff700 1 -- 172.21.15.16:0/325308970 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6388004950 con 0x7f6390133fa0 2023-09-30T22:31:17.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.719+0000 7f63960c2700 1 -- 172.21.15.16:0/325308970 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63900a0220 con 0x7f6390133fa0 2023-09-30T22:31:17.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.720+0000 7f63960c2700 1 -- 172.21.15.16:0/325308970 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6390074ea0 con 0x7f6390133fa0 2023-09-30T22:31:17.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.721+0000 7f6377fff700 1 -- 172.21.15.16:0/325308970 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6388027480 con 0x7f6390133fa0 2023-09-30T22:31:17.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.721+0000 7f6377fff700 1 -- 172.21.15.16:0/325308970 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f6388022070 con 0x7f6390133fa0 2023-09-30T22:31:17.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.730+0000 7f6377fff700 1 -- 172.21.15.16:0/325308970 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6388011980 con 0x7f6390133fa0 2023-09-30T22:31:17.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.807+0000 7f495725f700 1 -- 172.21.15.16:0/1034332306 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4950074ea0 con 0x7f49500a1340 2023-09-30T22:31:17.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.807+0000 7f494d7fa700 1 -- 172.21.15.16:0/1034332306 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f493c018260 con 0x7f49500a1340 2023-09-30T22:31:17.809 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:31:17.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.809+0000 7f495725f700 1 -- 172.21.15.16:0/1034332306 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f49500a1340 msgr2=0x7f495011eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:17.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.809+0000 7f495725f700 1 --2- 172.21.15.16:0/1034332306 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f49500a1340 0x7f495011eb60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f493c00c4e0 tx=0x7f493c00c7f0 comp rx=0 tx=0).stop 2023-09-30T22:31:17.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.810+0000 7f495725f700 1 -- 172.21.15.16:0/1034332306 shutdown_connections 2023-09-30T22:31:17.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.810+0000 7f495725f700 1 --2- 172.21.15.16:0/1034332306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4950134170 0x7f495011f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:17.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.810+0000 7f495725f700 1 --2- 172.21.15.16:0/1034332306 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f49500a0a30 0x7f495011e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:17.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.810+0000 7f495725f700 1 --2- 172.21.15.16:0/1034332306 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f49500a1340 0x7f495011eb60 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:17.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.810+0000 7f495725f700 1 -- 172.21.15.16:0/1034332306 >> 172.21.15.16:0/1034332306 conn(0x7f49500a6550 msgr2=0x7f495009ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:17.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.810+0000 7f495725f700 1 -- 172.21.15.16:0/1034332306 shutdown_connections 2023-09-30T22:31:17.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.810+0000 7f495725f700 1 -- 172.21.15.16:0/1034332306 wait complete. 2023-09-30T22:31:17.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.818+0000 7fd45ba1a700 1 -- 172.21.15.16:0/1346757032 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd45411ec80 con 0x7fd454099a40 2023-09-30T22:31:17.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.819+0000 7fd449ffb700 1 -- 172.21.15.16:0/1346757032 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd45411ec80 con 0x7fd454099a40 2023-09-30T22:31:17.821 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:31:17.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.821+0000 7fd45ba1a700 1 -- 172.21.15.16:0/1346757032 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd454099a40 msgr2=0x7fd45411b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:17.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.821+0000 7fd45ba1a700 1 --2- 172.21.15.16:0/1346757032 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd454099a40 0x7fd45411b530 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd45000e910 tx=0x7fd45000ec20 comp rx=0 tx=0).stop 2023-09-30T22:31:17.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.822+0000 7fd45ba1a700 1 -- 172.21.15.16:0/1346757032 shutdown_connections 2023-09-30T22:31:17.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.822+0000 7fd45ba1a700 1 --2- 172.21.15.16:0/1346757032 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd454131810 0x7fd45411bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:17.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.822+0000 7fd45ba1a700 1 --2- 172.21.15.16:0/1346757032 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd454099a40 0x7fd45411b530 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:17.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.822+0000 7fd45ba1a700 1 --2- 172.21.15.16:0/1346757032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd45409a580 0x7fd45411ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:17.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.822+0000 7fd45ba1a700 1 -- 172.21.15.16:0/1346757032 >> 172.21.15.16:0/1346757032 conn(0x7fd4540a2aa0 msgr2=0x7fd4541b8d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:17.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.822+0000 7fd45ba1a700 1 -- 172.21.15.16:0/1346757032 shutdown_connections 2023-09-30T22:31:17.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.822+0000 7fd45ba1a700 1 -- 172.21.15.16:0/1346757032 wait complete. 2023-09-30T22:31:17.826 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:31:17.837 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:31:17.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.901+0000 7fa6e0623700 1 -- 172.21.15.16:0/332906316 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa6d8072980 con 0x7fa6d809d630 2023-09-30T22:31:17.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.902+0000 7fa6ceffd700 1 -- 172.21.15.16:0/332906316 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa6d4033090 con 0x7fa6d809d630 2023-09-30T22:31:17.903 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:31:17.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.904+0000 7fa6e0623700 1 -- 172.21.15.16:0/332906316 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa6d809d630 msgr2=0x7fa6d811d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:17.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.904+0000 7fa6e0623700 1 --2- 172.21.15.16:0/332906316 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa6d809d630 0x7fa6d811d5f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4016000 tx=0x7fa6d4004750 comp rx=0 tx=0).stop 2023-09-30T22:31:17.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.904+0000 7fa6e0623700 1 -- 172.21.15.16:0/332906316 shutdown_connections 2023-09-30T22:31:17.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.904+0000 7fa6e0623700 1 --2- 172.21.15.16:0/332906316 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa6d809d630 0x7fa6d811d5f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:17.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.904+0000 7fa6e0623700 1 --2- 172.21.15.16:0/332906316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6d8133930 0x7fa6d81c3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:17.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.904+0000 7fa6e0623700 1 --2- 172.21.15.16:0/332906316 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa6d80a0a30 0x7fa6d811db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:17.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.904+0000 7fa6e0623700 1 -- 172.21.15.16:0/332906316 >> 172.21.15.16:0/332906316 conn(0x7fa6d80a3050 msgr2=0x7fa6d8132c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:17.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.904+0000 7fa6e0623700 1 -- 172.21.15.16:0/332906316 shutdown_connections 2023-09-30T22:31:17.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.904+0000 7fa6e0623700 1 -- 172.21.15.16:0/332906316 wait complete. 2023-09-30T22:31:17.919 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:31:17.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.962+0000 7f63960c2700 1 -- 172.21.15.16:0/325308970 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f63900a28e0 con 0x7f6390133fa0 2023-09-30T22:31:17.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.963+0000 7f6377fff700 1 -- 172.21.15.16:0/325308970 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f638801b400 con 0x7f6390133fa0 2023-09-30T22:31:17.964 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:31:17.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.965+0000 7f63960c2700 1 -- 172.21.15.16:0/325308970 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6390133fa0 msgr2=0x7f6390120bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:17.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.965+0000 7f63960c2700 1 --2- 172.21.15.16:0/325308970 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6390133fa0 0x7f6390120bf0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f639006c3a0 tx=0x7f6388004ad0 comp rx=0 tx=0).stop 2023-09-30T22:31:17.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.965+0000 7f63960c2700 1 -- 172.21.15.16:0/325308970 shutdown_connections 2023-09-30T22:31:17.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.965+0000 7f63960c2700 1 --2- 172.21.15.16:0/325308970 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6390133fa0 0x7f6390120bf0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:17.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.965+0000 7f63960c2700 1 --2- 172.21.15.16:0/325308970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63900a0a30 0x7f6390120170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:17.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.965+0000 7f63960c2700 1 --2- 172.21.15.16:0/325308970 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f63900a1340 0x7f63901206b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:17.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.965+0000 7f63960c2700 1 -- 172.21.15.16:0/325308970 >> 172.21.15.16:0/325308970 conn(0x7f63900a6550 msgr2=0x7f63901bb0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:17.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.965+0000 7f63960c2700 1 -- 172.21.15.16:0/325308970 shutdown_connections 2023-09-30T22:31:17.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:17.965+0000 7f63960c2700 1 -- 172.21.15.16:0/325308970 wait complete. 2023-09-30T22:31:17.980 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:31:18.827 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:18.837 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:18.920 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:18.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:18.976+0000 7f6d76a9d700 1 Processor -- start 2023-09-30T22:31:18.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:18.976+0000 7f6d76a9d700 1 -- start start 2023-09-30T22:31:18.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:18.977+0000 7f6d76a9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d7009e930 0x7f6d7009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:18.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:18.977+0000 7f6d76a9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6d7009f240 0x7f6d701317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:18.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:18.977+0000 7f6d76a9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d7009f650 0x7f6d70131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:18.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:18.977+0000 7f6d76a9d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d70070940 con 0x7f6d7009f240 2023-09-30T22:31:18.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:18.977+0000 7f6d76a9d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7006fd60 con 0x7f6d7009f650 2023-09-30T22:31:18.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:18.977+0000 7f6d76a9d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7006fa80 con 0x7f6d7009e930 2023-09-30T22:31:18.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:18.977+0000 7f6d74839700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d7009e930 0x7f6d7009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:18.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:18.977+0000 7f6d74839700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d7009e930 0x7f6d7009ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:38438/0 (socket says 172.21.15.16:38438) 2023-09-30T22:31:18.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:18.977+0000 7f6d74839700 1 -- 172.21.15.16:0/4093815604 learned_addr learned my addr 172.21.15.16:0/4093815604 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:18.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:18.977+0000 7f6d6ffff700 1 --2- 172.21.15.16:0/4093815604 >> v2:172.21.15.16:3300/0 conn(0x7f6d7009f240 0x7f6d701317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:18.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:18.977+0000 7f6d7503a700 1 --2- 172.21.15.16:0/4093815604 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d7009f650 0x7f6d70131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:18.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:18.977+0000 7f6d74839700 1 -- 172.21.15.16:0/4093815604 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d7009f650 msgr2=0x7f6d70131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:18.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:18.977+0000 7f6d74839700 1 --2- 172.21.15.16:0/4093815604 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d7009f650 0x7f6d70131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:18.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:18.977+0000 7f6d74839700 1 -- 172.21.15.16:0/4093815604 >> v2:172.21.15.16:3300/0 conn(0x7f6d7009f240 msgr2=0x7f6d701317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:18.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:18.977+0000 7f6d74839700 1 --2- 172.21.15.16:0/4093815604 >> v2:172.21.15.16:3300/0 conn(0x7f6d7009f240 0x7f6d701317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:18.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:18.977+0000 7f6d74839700 1 -- 172.21.15.16:0/4093815604 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d7009fb60 con 0x7f6d7009e930 2023-09-30T22:31:18.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:18.978+0000 7f6d74839700 1 --2- 172.21.15.16:0/4093815604 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d7009e930 0x7f6d7009ed00 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6d600098f0 tx=0x7f6d60009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45afb1bb8cbda0c7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:18.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:18.978+0000 7f6d6f7fe700 1 -- 172.21.15.16:0/4093815604 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d60011830 con 0x7f6d7009e930 2023-09-30T22:31:18.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:18.978+0000 7f6d6f7fe700 1 -- 172.21.15.16:0/4093815604 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d60011990 con 0x7f6d7009e930 2023-09-30T22:31:18.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:18.978+0000 7f6d6f7fe700 1 -- 172.21.15.16:0/4093815604 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d60011d20 con 0x7f6d7009e930 2023-09-30T22:31:18.982 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:18.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:18.979+0000 7f6d76a9d700 1 -- 172.21.15.16:0/4093815604 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d7009e930 msgr2=0x7f6d7009ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:18.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:18.979+0000 7f6d76a9d700 1 --2- 172.21.15.16:0/4093815604 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d7009e930 0x7f6d7009ed00 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6d600098f0 tx=0x7f6d60009c00 comp rx=0 tx=0).stop 2023-09-30T22:31:18.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:18.979+0000 7f6d76a9d700 1 -- 172.21.15.16:0/4093815604 shutdown_connections 2023-09-30T22:31:18.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:18.979+0000 7f6d76a9d700 1 --2- 172.21.15.16:0/4093815604 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d7009f650 0x7f6d70131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:18.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:18.979+0000 7f6d76a9d700 1 --2- 172.21.15.16:0/4093815604 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d7009e930 0x7f6d7009ed00 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:18.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:18.979+0000 7f6d76a9d700 1 --2- 172.21.15.16:0/4093815604 >> v2:172.21.15.16:3300/0 conn(0x7f6d7009f240 0x7f6d701317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:18.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:18.979+0000 7f6d76a9d700 1 -- 172.21.15.16:0/4093815604 >> 172.21.15.16:0/4093815604 conn(0x7f6d700a4210 msgr2=0x7f6d700a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:18.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:18.979+0000 7f6d76a9d700 1 -- 172.21.15.16:0/4093815604 shutdown_connections 2023-09-30T22:31:18.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:18.979+0000 7f6d76a9d700 1 -- 172.21.15.16:0/4093815604 wait complete. 2023-09-30T22:31:18.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:18.980+0000 7f6d76a9d700 1 Processor -- start 2023-09-30T22:31:18.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:18.980+0000 7f6d76a9d700 1 -- start start 2023-09-30T22:31:18.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:18.980+0000 7f6d76a9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d7009e930 0x7f6d701a3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:18.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:18.981+0000 7f6d76a9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d7009f240 0x7f6d701a44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:18.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:18.981+0000 7f6d76a9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d7009f650 0x7f6d701a4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:18.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:18.981+0000 7f6d76a9d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d701a5420 con 0x7f6d7009f650 2023-09-30T22:31:18.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:18.981+0000 7f6d76a9d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d701a5a70 con 0x7f6d7009f240 2023-09-30T22:31:18.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:18.981+0000 7f6d76a9d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6d701a6090 con 0x7f6d7009e930 2023-09-30T22:31:18.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:18.981+0000 7f6d74839700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d7009e930 0x7f6d701a3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:18.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:18.981+0000 7f6d6ffff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d7009f240 0x7f6d701a44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:18.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:18.981+0000 7f6d74839700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d7009e930 0x7f6d701a3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:38452/0 (socket says 172.21.15.16:38452) 2023-09-30T22:31:18.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:18.981+0000 7f6d74839700 1 -- 172.21.15.16:0/2958780212 learned_addr learned my addr 172.21.15.16:0/2958780212 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:18.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:18.981+0000 7f6d7503a700 1 --2- 172.21.15.16:0/2958780212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d7009f650 0x7f6d701a4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:18.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:18.981+0000 7f6d6ffff700 1 -- 172.21.15.16:0/2958780212 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d7009e930 msgr2=0x7f6d701a3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:18.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:18.981+0000 7f6d6ffff700 1 --2- 172.21.15.16:0/2958780212 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d7009e930 0x7f6d701a3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:18.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:18.981+0000 7f6d6ffff700 1 -- 172.21.15.16:0/2958780212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d7009f650 msgr2=0x7f6d701a4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:18.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:18.981+0000 7f6d6ffff700 1 --2- 172.21.15.16:0/2958780212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d7009f650 0x7f6d701a4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:18.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:18.981+0000 7f6d6ffff700 1 -- 172.21.15.16:0/2958780212 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d60009550 con 0x7f6d7009f240 2023-09-30T22:31:18.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:18.981+0000 7f6d6ffff700 1 --2- 172.21.15.16:0/2958780212 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d7009f240 0x7f6d701a44d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6d6400e870 tx=0x7f6d6400eb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:18.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:18.982+0000 7f6d6d7fa700 1 -- 172.21.15.16:0/2958780212 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d64013680 con 0x7f6d7009f240 2023-09-30T22:31:18.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:18.982+0000 7f6d6d7fa700 1 -- 172.21.15.16:0/2958780212 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d640137e0 con 0x7f6d7009f240 2023-09-30T22:31:18.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:18.982+0000 7f6d76a9d700 1 -- 172.21.15.16:0/2958780212 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d700a0190 con 0x7f6d7009f240 2023-09-30T22:31:18.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:18.982+0000 7f6d6d7fa700 1 -- 172.21.15.16:0/2958780212 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d64013b70 con 0x7f6d7009f240 2023-09-30T22:31:18.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:18.982+0000 7f6d76a9d700 1 -- 172.21.15.16:0/2958780212 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d701a76f0 con 0x7f6d7009f240 2023-09-30T22:31:18.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:18.982+0000 7f6d6d7fa700 1 -- 172.21.15.16:0/2958780212 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6d64013cd0 con 0x7f6d7009f240 2023-09-30T22:31:18.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:18.983+0000 7f6d6d7fa700 1 -- 172.21.15.16:0/2958780212 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f6d6403f3a0 con 0x7f6d7009f240 2023-09-30T22:31:18.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:18.983+0000 7f6d76a9d700 1 -- 172.21.15.16:0/2958780212 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d701a76f0 con 0x7f6d7009f240 2023-09-30T22:31:18.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:18.992+0000 7f6d6d7fa700 1 -- 172.21.15.16:0/2958780212 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6d701a76f0 con 0x7f6d7009f240 2023-09-30T22:31:19.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.021+0000 7f87e0e62700 1 Processor -- start 2023-09-30T22:31:19.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.021+0000 7f87e0e62700 1 -- start start 2023-09-30T22:31:19.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.021+0000 7f87e0e62700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f87dc09e930 0x7f87dc09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:19.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.021+0000 7f87e0e62700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f87dc09f240 0x7f87dc1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:19.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.022+0000 7f87e0e62700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87dc070940 con 0x7f87dc09f650 2023-09-30T22:31:19.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.022+0000 7f87e0e62700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc06fd60 con 0x7f87dc09e930 2023-09-30T22:31:19.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.022+0000 7f87e0e62700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc06fa80 con 0x7f87dc09f240 2023-09-30T22:31:19.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.022+0000 7f87da59c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f87dc09e930 0x7f87dc09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:19.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.022+0000 7f87da59c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f87dc09e930 0x7f87dc09ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:57272/0 (socket says 172.21.15.16:57272) 2023-09-30T22:31:19.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.022+0000 7f87da59c700 1 -- 172.21.15.16:0/3984142922 learned_addr learned my addr 172.21.15.16:0/3984142922 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:19.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.022+0000 7f87d9d9b700 1 --2- 172.21.15.16:0/3984142922 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f87dc09f240 0x7f87dc1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:19.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.022+0000 7f87da59c700 1 -- 172.21.15.16:0/3984142922 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f87dc09f240 msgr2=0x7f87dc1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:19.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.022+0000 7f87da59c700 1 --2- 172.21.15.16:0/3984142922 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f87dc09f240 0x7f87dc1317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.022+0000 7f87da59c700 1 -- 172.21.15.16:0/3984142922 >> v1:172.21.15.16:6789/0 conn(0x7f87dc09f650 legacy=0x7f87dc131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:19.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.022+0000 7f87da59c700 1 -- 172.21.15.16:0/3984142922 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87dc09fb60 con 0x7f87dc09e930 2023-09-30T22:31:19.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.023+0000 7f87da59c700 1 --2- 172.21.15.16:0/3984142922 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f87dc09e930 0x7f87dc09ed00 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f87cc009920 tx=0x7f87cc009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd088e9d8577a7b6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:19.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.023+0000 7f87d959a700 1 -- 172.21.15.16:0/3984142922 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc011830 con 0x7f87dc09e930 2023-09-30T22:31:19.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.023+0000 7f87d959a700 1 -- 172.21.15.16:0/3984142922 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87cc011990 con 0x7f87dc09e930 2023-09-30T22:31:19.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.023+0000 7f87d959a700 1 -- 172.21.15.16:0/3984142922 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc011d20 con 0x7f87dc09e930 2023-09-30T22:31:19.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.023+0000 7f87e0e62700 1 -- 172.21.15.16:0/3984142922 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f87dc09e930 msgr2=0x7f87dc09ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:19.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.023+0000 7f87e0e62700 1 --2- 172.21.15.16:0/3984142922 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f87dc09e930 0x7f87dc09ed00 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f87cc009920 tx=0x7f87cc009c30 comp rx=0 tx=0).stop 2023-09-30T22:31:19.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.024+0000 7f87e0e62700 1 -- 172.21.15.16:0/3984142922 shutdown_connections 2023-09-30T22:31:19.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.024+0000 7f87e0e62700 1 --2- 172.21.15.16:0/3984142922 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f87dc09e930 0x7f87dc09ed00 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.024+0000 7f87e0e62700 1 --2- 172.21.15.16:0/3984142922 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f87dc09f240 0x7f87dc1317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.024+0000 7f87e0e62700 1 -- 172.21.15.16:0/3984142922 >> 172.21.15.16:0/3984142922 conn(0x7f87dc0a4210 msgr2=0x7f87dc0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:19.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.024+0000 7f87e0e62700 1 -- 172.21.15.16:0/3984142922 shutdown_connections 2023-09-30T22:31:19.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.024+0000 7f87e0e62700 1 -- 172.21.15.16:0/3984142922 wait complete. 2023-09-30T22:31:19.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.025+0000 7f87e0e62700 1 Processor -- start 2023-09-30T22:31:19.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.025+0000 7f87e0e62700 1 -- start start 2023-09-30T22:31:19.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.026+0000 7f87e0e62700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f87dc09e930 0x7f87dc1abfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:19.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.026+0000 7f87e0e62700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87dc09f240 0x7f87dc1ac4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:19.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.026+0000 7f87e0e62700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f87dc09f650 0x7f87dc1aca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:19.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.026+0000 7f87e0e62700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc1afb60 con 0x7f87dc09f240 2023-09-30T22:31:19.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.026+0000 7f87e0e62700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc1af7e0 con 0x7f87dc09f650 2023-09-30T22:31:19.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.026+0000 7f87e0e62700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f87dc0a37e0 con 0x7f87dc09e930 2023-09-30T22:31:19.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.026+0000 7f87d9d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87dc09f240 0x7f87dc1ac4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:19.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.026+0000 7f87da59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f87dc09e930 0x7f87dc1abfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:19.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.026+0000 7f87d9d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87dc09f240 0x7f87dc1ac4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57054/0 (socket says 172.21.15.16:57054) 2023-09-30T22:31:19.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.026+0000 7f87d9d9b700 1 -- 172.21.15.16:0/4208281727 learned_addr learned my addr 172.21.15.16:0/4208281727 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:19.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.026+0000 7f87da59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f87dc09e930 0x7f87dc1abfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:38472/0 (socket says 172.21.15.16:38472) 2023-09-30T22:31:19.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.026+0000 7f87da59c700 1 -- 172.21.15.16:0/4208281727 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f87dc09f650 msgr2=0x7f87dc1aca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:19.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.026+0000 7f87da59c700 1 --2- 172.21.15.16:0/4208281727 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f87dc09f650 0x7f87dc1aca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.026+0000 7f87da59c700 1 -- 172.21.15.16:0/4208281727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87dc09f240 msgr2=0x7f87dc1ac4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:19.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.026+0000 7f87da59c700 1 --2- 172.21.15.16:0/4208281727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87dc09f240 0x7f87dc1ac4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.026+0000 7f87da59c700 1 -- 172.21.15.16:0/4208281727 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87cc009550 con 0x7f87dc09e930 2023-09-30T22:31:19.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.027+0000 7f87da59c700 1 --2- 172.21.15.16:0/4208281727 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f87dc09e930 0x7f87dc1abfb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f87cc016000 tx=0x7f87cc004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:19.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.027+0000 7f87caffd700 1 -- 172.21.15.16:0/4208281727 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc004510 con 0x7f87dc09e930 2023-09-30T22:31:19.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.027+0000 7f87caffd700 1 -- 172.21.15.16:0/4208281727 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87cc0036a0 con 0x7f87dc09e930 2023-09-30T22:31:19.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.027+0000 7f87caffd700 1 -- 172.21.15.16:0/4208281727 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc003a10 con 0x7f87dc09e930 2023-09-30T22:31:19.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.027+0000 7f87e0e62700 1 -- 172.21.15.16:0/4208281727 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87dc0a0190 con 0x7f87dc09e930 2023-09-30T22:31:19.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.027+0000 7f87e0e62700 1 -- 172.21.15.16:0/4208281727 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87dc1adad0 con 0x7f87dc09e930 2023-09-30T22:31:19.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.028+0000 7f87e0e62700 1 -- 172.21.15.16:0/4208281727 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87dc1b8e00 con 0x7f87dc09e930 2023-09-30T22:31:19.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.028+0000 7f87caffd700 1 -- 172.21.15.16:0/4208281727 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f87cc003b70 con 0x7f87dc09e930 2023-09-30T22:31:19.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.029+0000 7f87caffd700 1 -- 172.21.15.16:0/4208281727 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f87cc021030 con 0x7f87dc09e930 2023-09-30T22:31:19.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.038+0000 7f87caffd700 1 -- 172.21.15.16:0/4208281727 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f87cc01a430 con 0x7f87dc09e930 2023-09-30T22:31:19.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.073+0000 7f55eec28700 1 Processor -- start 2023-09-30T22:31:19.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.073+0000 7f55eec28700 1 -- start start 2023-09-30T22:31:19.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.074+0000 7f55eec28700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f55e809d6f0 0x7f55e809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:19.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.074+0000 7f55eec28700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55e809e000 0x7f55e81339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:19.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.074+0000 7f55eec28700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55e809b120 con 0x7f55e809e410 2023-09-30T22:31:19.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.074+0000 7f55eec28700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55e806b460 con 0x7f55e809e000 2023-09-30T22:31:19.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.074+0000 7f55eec28700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f55e809a7e0 con 0x7f55e809d6f0 2023-09-30T22:31:19.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.074+0000 7f55e7fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55e809e000 0x7f55e81339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:19.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.074+0000 7f55ec9c4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f55e809d6f0 0x7f55e809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:19.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.074+0000 7f55e7fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55e809e000 0x7f55e81339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:57294/0 (socket says 172.21.15.16:57294) 2023-09-30T22:31:19.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.074+0000 7f55e7fff700 1 -- 172.21.15.16:0/2026975699 learned_addr learned my addr 172.21.15.16:0/2026975699 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:19.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.074+0000 7f55e7fff700 1 -- 172.21.15.16:0/2026975699 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f55e809d6f0 msgr2=0x7f55e809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:19.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.074+0000 7f55e7fff700 1 --2- 172.21.15.16:0/2026975699 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f55e809d6f0 0x7f55e809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.074+0000 7f55e7fff700 1 -- 172.21.15.16:0/2026975699 >> v1:172.21.15.16:6789/0 conn(0x7f55e809e410 legacy=0x7f55e8133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:19.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.074+0000 7f55e7fff700 1 -- 172.21.15.16:0/2026975699 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55e809a500 con 0x7f55e809e000 2023-09-30T22:31:19.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.075+0000 7f55e7fff700 1 --2- 172.21.15.16:0/2026975699 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55e809e000 0x7f55e81339f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f55d4009920 tx=0x7f55d4009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18b1b6119389f08a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:19.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.075+0000 7f55e77fe700 1 -- 172.21.15.16:0/2026975699 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d4011830 con 0x7f55e809e000 2023-09-30T22:31:19.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.075+0000 7f55e77fe700 1 -- 172.21.15.16:0/2026975699 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55d4011990 con 0x7f55e809e000 2023-09-30T22:31:19.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.075+0000 7f55e77fe700 1 -- 172.21.15.16:0/2026975699 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d4011d20 con 0x7f55e809e000 2023-09-30T22:31:19.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.075+0000 7f55eec28700 1 -- 172.21.15.16:0/2026975699 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55e809e000 msgr2=0x7f55e81339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:19.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.075+0000 7f55eec28700 1 --2- 172.21.15.16:0/2026975699 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55e809e000 0x7f55e81339f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f55d4009920 tx=0x7f55d4009c30 comp rx=0 tx=0).stop 2023-09-30T22:31:19.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.075+0000 7f55eec28700 1 -- 172.21.15.16:0/2026975699 shutdown_connections 2023-09-30T22:31:19.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.075+0000 7f55eec28700 1 --2- 172.21.15.16:0/2026975699 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f55e809d6f0 0x7f55e809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.076+0000 7f55eec28700 1 --2- 172.21.15.16:0/2026975699 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55e809e000 0x7f55e81339f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.076+0000 7f55eec28700 1 -- 172.21.15.16:0/2026975699 >> 172.21.15.16:0/2026975699 conn(0x7f55e8066f50 msgr2=0x7f55e809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:19.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.076+0000 7f55eec28700 1 -- 172.21.15.16:0/2026975699 shutdown_connections 2023-09-30T22:31:19.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.076+0000 7f55eec28700 1 -- 172.21.15.16:0/2026975699 wait complete. 2023-09-30T22:31:19.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.077+0000 7f55eec28700 1 Processor -- start 2023-09-30T22:31:19.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.077+0000 7f55eec28700 1 -- start start 2023-09-30T22:31:19.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.078+0000 7f55eec28700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55e809d6f0 0x7f55e8122600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:19.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.078+0000 7f55eec28700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55e809e000 0x7f55e8122b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:19.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.078+0000 7f55eec28700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f55e809e410 0x7f55e8123080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:19.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.078+0000 7f55eec28700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55e8124320 con 0x7f55e809d6f0 2023-09-30T22:31:19.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.078+0000 7f55eec28700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55e8125e40 con 0x7f55e809e000 2023-09-30T22:31:19.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.078+0000 7f55eec28700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f55e80a2db0 con 0x7f55e809e410 2023-09-30T22:31:19.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.078+0000 7f55e7fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55e809e000 0x7f55e8122b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:19.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.078+0000 7f55e7fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55e809e000 0x7f55e8122b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:57302/0 (socket says 172.21.15.16:57302) 2023-09-30T22:31:19.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.078+0000 7f55e7fff700 1 -- 172.21.15.16:0/1841642737 learned_addr learned my addr 172.21.15.16:0/1841642737 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:19.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.078+0000 7f55ed1c5700 1 --2- 172.21.15.16:0/1841642737 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f55e809e410 0x7f55e8123080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:19.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.079+0000 7f55ec9c4700 1 --2- 172.21.15.16:0/1841642737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55e809d6f0 0x7f55e8122600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:19.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.079+0000 7f55e7fff700 1 -- 172.21.15.16:0/1841642737 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f55e809e410 msgr2=0x7f55e8123080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:19.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.079+0000 7f55e7fff700 1 --2- 172.21.15.16:0/1841642737 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f55e809e410 0x7f55e8123080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.079+0000 7f55e7fff700 1 -- 172.21.15.16:0/1841642737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55e809d6f0 msgr2=0x7f55e8122600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:19.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.079+0000 7f55e7fff700 1 --2- 172.21.15.16:0/1841642737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55e809d6f0 0x7f55e8122600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.079+0000 7f55e7fff700 1 -- 172.21.15.16:0/1841642737 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55d4009550 con 0x7f55e809e000 2023-09-30T22:31:19.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.079+0000 7f55e7fff700 1 --2- 172.21.15.16:0/1841642737 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55e809e000 0x7f55e8122b40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f55d4000c00 tx=0x7f55d4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:19.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.079+0000 7f55e57fa700 1 -- 172.21.15.16:0/1841642737 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d4004450 con 0x7f55e809e000 2023-09-30T22:31:19.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.079+0000 7f55e57fa700 1 -- 172.21.15.16:0/1841642737 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55d40045b0 con 0x7f55e809e000 2023-09-30T22:31:19.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.079+0000 7f55eec28700 1 -- 172.21.15.16:0/1841642737 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55e8133120 con 0x7f55e809e000 2023-09-30T22:31:19.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.079+0000 7f55e57fa700 1 -- 172.21.15.16:0/1841642737 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d40038c0 con 0x7f55e809e000 2023-09-30T22:31:19.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.080+0000 7f55eec28700 1 -- 172.21.15.16:0/1841642737 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55e81268b0 con 0x7f55e809e000 2023-09-30T22:31:19.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.081+0000 7f55e57fa700 1 -- 172.21.15.16:0/1841642737 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f55d4003a20 con 0x7f55e809e000 2023-09-30T22:31:19.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.081+0000 7f55e57fa700 1 -- 172.21.15.16:0/1841642737 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f55d4021030 con 0x7f55e809e000 2023-09-30T22:31:19.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.081+0000 7f55eec28700 1 -- 172.21.15.16:0/1841642737 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55e8074ea0 con 0x7f55e809e000 2023-09-30T22:31:19.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.086+0000 7f55e57fa700 1 -- 172.21.15.16:0/1841642737 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f55d402b080 con 0x7f55e809e000 2023-09-30T22:31:19.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.167+0000 7f6e3c5f3700 1 Processor -- start 2023-09-30T22:31:19.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.167+0000 7f6e3c5f3700 1 -- start start 2023-09-30T22:31:19.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.168+0000 7f6e3c5f3700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6e340a0a30 0x7f6e340a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:19.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.168+0000 7f6e3c5f3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e3409d630 0x7f6e340a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:19.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.168+0000 7f6e3c5f3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6e34133930 0x7f6e34133db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:19.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.168+0000 7f6e3c5f3700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e340a2a70 con 0x7f6e340a0a30 2023-09-30T22:31:19.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.168+0000 7f6e3c5f3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e340a01d0 con 0x7f6e34133930 2023-09-30T22:31:19.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.168+0000 7f6e3c5f3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3409e2b0 con 0x7f6e3409d630 2023-09-30T22:31:19.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.169+0000 7f6e39b8e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e3409d630 0x7f6e340a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:19.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.169+0000 7f6e39b8e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e3409d630 0x7f6e340a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:38494/0 (socket says 172.21.15.16:38494) 2023-09-30T22:31:19.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.169+0000 7f6e39b8e700 1 -- 172.21.15.16:0/2253808594 learned_addr learned my addr 172.21.15.16:0/2253808594 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:19.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.169+0000 7f6e3ab90700 1 --2- 172.21.15.16:0/2253808594 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6e34133930 0x7f6e34133db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:19.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.169+0000 7f6e3a38f700 1 --2- 172.21.15.16:0/2253808594 >> v2:172.21.15.16:3300/0 conn(0x7f6e340a0a30 0x7f6e340a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:19.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.169+0000 7f6e39b8e700 1 -- 172.21.15.16:0/2253808594 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6e34133930 msgr2=0x7f6e34133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:19.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.169+0000 7f6e39b8e700 1 --2- 172.21.15.16:0/2253808594 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6e34133930 0x7f6e34133db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.169+0000 7f6e39b8e700 1 -- 172.21.15.16:0/2253808594 >> v2:172.21.15.16:3300/0 conn(0x7f6e340a0a30 msgr2=0x7f6e340a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:19.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.169+0000 7f6e39b8e700 1 --2- 172.21.15.16:0/2253808594 >> v2:172.21.15.16:3300/0 conn(0x7f6e340a0a30 0x7f6e340a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.169+0000 7f6e39b8e700 1 -- 172.21.15.16:0/2253808594 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e3409dc50 con 0x7f6e3409d630 2023-09-30T22:31:19.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.169+0000 7f6e39b8e700 1 --2- 172.21.15.16:0/2253808594 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e3409d630 0x7f6e340a1340 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6e240098f0 tx=0x7f6e24009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96310b5e83041e4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:19.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.170+0000 7f6e3938d700 1 -- 172.21.15.16:0/2253808594 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e24011830 con 0x7f6e3409d630 2023-09-30T22:31:19.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.170+0000 7f6e3938d700 1 -- 172.21.15.16:0/2253808594 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e24011990 con 0x7f6e3409d630 2023-09-30T22:31:19.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.170+0000 7f6e3938d700 1 -- 172.21.15.16:0/2253808594 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e24011d20 con 0x7f6e3409d630 2023-09-30T22:31:19.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.170+0000 7f6e3c5f3700 1 -- 172.21.15.16:0/2253808594 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e3409d630 msgr2=0x7f6e340a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:19.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.170+0000 7f6e3c5f3700 1 --2- 172.21.15.16:0/2253808594 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e3409d630 0x7f6e340a1340 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6e240098f0 tx=0x7f6e24009c00 comp rx=0 tx=0).stop 2023-09-30T22:31:19.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.170+0000 7f6e3c5f3700 1 -- 172.21.15.16:0/2253808594 shutdown_connections 2023-09-30T22:31:19.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.170+0000 7f6e3c5f3700 1 --2- 172.21.15.16:0/2253808594 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6e34133930 0x7f6e34133db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.170+0000 7f6e3c5f3700 1 --2- 172.21.15.16:0/2253808594 >> v2:172.21.15.16:3300/0 conn(0x7f6e340a0a30 0x7f6e340a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.170+0000 7f6e3c5f3700 1 --2- 172.21.15.16:0/2253808594 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e3409d630 0x7f6e340a1340 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.170+0000 7f6e3c5f3700 1 -- 172.21.15.16:0/2253808594 >> 172.21.15.16:0/2253808594 conn(0x7f6e340a3050 msgr2=0x7f6e340a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:19.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.170+0000 7f6e3c5f3700 1 -- 172.21.15.16:0/2253808594 shutdown_connections 2023-09-30T22:31:19.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.170+0000 7f6e3c5f3700 1 -- 172.21.15.16:0/2253808594 wait complete. 2023-09-30T22:31:19.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.172+0000 7f6e3c5f3700 1 Processor -- start 2023-09-30T22:31:19.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.172+0000 7f6e3c5f3700 1 -- start start 2023-09-30T22:31:19.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.173+0000 7f6e3c5f3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e3409d630 0x7f6e341c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:19.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.173+0000 7f6e3c5f3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6e340a0a30 0x7f6e341c84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:19.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.173+0000 7f6e3c5f3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e34133930 0x7f6e341c8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:19.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.173+0000 7f6e3c5f3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e34120660 con 0x7f6e34133930 2023-09-30T22:31:19.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.173+0000 7f6e3c5f3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e340a5e50 con 0x7f6e340a0a30 2023-09-30T22:31:19.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.173+0000 7f6e3c5f3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6e340a0410 con 0x7f6e3409d630 2023-09-30T22:31:19.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.173+0000 7f6e3a38f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e3409d630 0x7f6e341c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:19.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.173+0000 7f6e3a38f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e3409d630 0x7f6e341c7f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:38498/0 (socket says 172.21.15.16:38498) 2023-09-30T22:31:19.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.173+0000 7f6e3a38f700 1 -- 172.21.15.16:0/3320154843 learned_addr learned my addr 172.21.15.16:0/3320154843 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:19.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.173+0000 7f6e39b8e700 1 --2- 172.21.15.16:0/3320154843 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6e340a0a30 0x7f6e341c84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:19.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.173+0000 7f6e3ab90700 1 --2- 172.21.15.16:0/3320154843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e34133930 0x7f6e341c8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:19.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.173+0000 7f6e39b8e700 1 -- 172.21.15.16:0/3320154843 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e3409d630 msgr2=0x7f6e341c7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:19.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.173+0000 7f6e39b8e700 1 --2- 172.21.15.16:0/3320154843 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e3409d630 0x7f6e341c7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.173+0000 7f6e39b8e700 1 -- 172.21.15.16:0/3320154843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e34133930 msgr2=0x7f6e341c8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:19.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.173+0000 7f6e39b8e700 1 --2- 172.21.15.16:0/3320154843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e34133930 0x7f6e341c8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.173+0000 7f6e39b8e700 1 -- 172.21.15.16:0/3320154843 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e24009550 con 0x7f6e340a0a30 2023-09-30T22:31:19.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.173+0000 7f6e3a38f700 1 --2- 172.21.15.16:0/3320154843 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e3409d630 0x7f6e341c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:19.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.174+0000 7f6e39b8e700 1 --2- 172.21.15.16:0/3320154843 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6e340a0a30 0x7f6e341c84c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6e24016000 tx=0x7f6e24004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:19.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.174+0000 7f6e2affd700 1 -- 172.21.15.16:0/3320154843 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e24004240 con 0x7f6e340a0a30 2023-09-30T22:31:19.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.174+0000 7f6e2affd700 1 -- 172.21.15.16:0/3320154843 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e2401f070 con 0x7f6e340a0a30 2023-09-30T22:31:19.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.174+0000 7f6e2affd700 1 -- 172.21.15.16:0/3320154843 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e240045e0 con 0x7f6e340a0a30 2023-09-30T22:31:19.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.174+0000 7f6e3c5f3700 1 -- 172.21.15.16:0/3320154843 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e34121ec0 con 0x7f6e340a0a30 2023-09-30T22:31:19.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.174+0000 7f6e3c5f3700 1 -- 172.21.15.16:0/3320154843 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e34122bf0 con 0x7f6e340a0a30 2023-09-30T22:31:19.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.175+0000 7f6e2affd700 1 -- 172.21.15.16:0/3320154843 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6e24021030 con 0x7f6e340a0a30 2023-09-30T22:31:19.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.175+0000 7f6e2affd700 1 -- 172.21.15.16:0/3320154843 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f6e24040810 con 0x7f6e340a0a30 2023-09-30T22:31:19.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.176+0000 7f6e3c5f3700 1 -- 172.21.15.16:0/3320154843 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e340747f0 con 0x7f6e340a0a30 2023-09-30T22:31:19.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.185+0000 7f6e2affd700 1 -- 172.21.15.16:0/3320154843 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6e24015020 con 0x7f6e340a0a30 2023-09-30T22:31:19.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.221+0000 7f6d76a9d700 1 -- 172.21.15.16:0/2958780212 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6d701a76f0 con 0x7f6d7009f240 2023-09-30T22:31:19.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.221+0000 7f6d6d7fa700 1 -- 172.21.15.16:0/2958780212 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6d701a76f0 con 0x7f6d7009f240 2023-09-30T22:31:19.226 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:31:19.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.224+0000 7f6d76a9d700 1 -- 172.21.15.16:0/2958780212 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d7009f240 msgr2=0x7f6d701a44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:19.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.224+0000 7f6d76a9d700 1 --2- 172.21.15.16:0/2958780212 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d7009f240 0x7f6d701a44d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6d6400e870 tx=0x7f6d6400eb80 comp rx=0 tx=0).stop 2023-09-30T22:31:19.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.224+0000 7f6d76a9d700 1 -- 172.21.15.16:0/2958780212 shutdown_connections 2023-09-30T22:31:19.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.224+0000 7f6d76a9d700 1 --2- 172.21.15.16:0/2958780212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d7009f650 0x7f6d701a4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.224+0000 7f6d76a9d700 1 --2- 172.21.15.16:0/2958780212 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d7009e930 0x7f6d701a3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.224+0000 7f6d76a9d700 1 --2- 172.21.15.16:0/2958780212 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d7009f240 0x7f6d701a44d0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.224+0000 7f6d76a9d700 1 -- 172.21.15.16:0/2958780212 >> 172.21.15.16:0/2958780212 conn(0x7f6d700a4210 msgr2=0x7f6d701b8d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:19.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.224+0000 7f6d76a9d700 1 -- 172.21.15.16:0/2958780212 shutdown_connections 2023-09-30T22:31:19.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.224+0000 7f6d76a9d700 1 -- 172.21.15.16:0/2958780212 wait complete. 2023-09-30T22:31:19.240 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:31:19.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.290+0000 7f87e0e62700 1 -- 172.21.15.16:0/4208281727 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f87dc074ea0 con 0x7f87dc09e930 2023-09-30T22:31:19.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.291+0000 7f87caffd700 1 -- 172.21.15.16:0/4208281727 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f87cc02b070 con 0x7f87dc09e930 2023-09-30T22:31:19.293 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:31:19.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.293+0000 7f87e0e62700 1 -- 172.21.15.16:0/4208281727 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f87dc09e930 msgr2=0x7f87dc1abfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:19.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.293+0000 7f87e0e62700 1 --2- 172.21.15.16:0/4208281727 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f87dc09e930 0x7f87dc1abfb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f87cc016000 tx=0x7f87cc004750 comp rx=0 tx=0).stop 2023-09-30T22:31:19.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.293+0000 7f87e0e62700 1 -- 172.21.15.16:0/4208281727 shutdown_connections 2023-09-30T22:31:19.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.293+0000 7f87e0e62700 1 --2- 172.21.15.16:0/4208281727 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f87dc09f650 0x7f87dc1aca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.293+0000 7f87e0e62700 1 --2- 172.21.15.16:0/4208281727 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f87dc09e930 0x7f87dc1abfb0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.293+0000 7f87e0e62700 1 --2- 172.21.15.16:0/4208281727 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87dc09f240 0x7f87dc1ac4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.293+0000 7f87e0e62700 1 -- 172.21.15.16:0/4208281727 >> 172.21.15.16:0/4208281727 conn(0x7f87dc0a4210 msgr2=0x7f87dc06b880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:19.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.293+0000 7f87e0e62700 1 -- 172.21.15.16:0/4208281727 shutdown_connections 2023-09-30T22:31:19.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.293+0000 7f87e0e62700 1 -- 172.21.15.16:0/4208281727 wait complete. 2023-09-30T22:31:19.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.300+0000 7f55eec28700 1 -- 172.21.15.16:0/1841642737 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f55e806c360 con 0x7f55e809e000 2023-09-30T22:31:19.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.300+0000 7f55e57fa700 1 -- 172.21.15.16:0/1841642737 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f55d4015020 con 0x7f55e809e000 2023-09-30T22:31:19.302 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:31:19.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.302+0000 7f55eec28700 1 -- 172.21.15.16:0/1841642737 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55e809e000 msgr2=0x7f55e8122b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:19.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.302+0000 7f55eec28700 1 --2- 172.21.15.16:0/1841642737 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55e809e000 0x7f55e8122b40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f55d4000c00 tx=0x7f55d4004750 comp rx=0 tx=0).stop 2023-09-30T22:31:19.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.302+0000 7f55eec28700 1 -- 172.21.15.16:0/1841642737 shutdown_connections 2023-09-30T22:31:19.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.302+0000 7f55eec28700 1 --2- 172.21.15.16:0/1841642737 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f55e809e410 0x7f55e8123080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.302+0000 7f55eec28700 1 --2- 172.21.15.16:0/1841642737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55e809d6f0 0x7f55e8122600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.302+0000 7f55eec28700 1 --2- 172.21.15.16:0/1841642737 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55e809e000 0x7f55e8122b40 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.302+0000 7f55eec28700 1 -- 172.21.15.16:0/1841642737 >> 172.21.15.16:0/1841642737 conn(0x7f55e8066f50 msgr2=0x7f55e80710c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:19.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.303+0000 7f55eec28700 1 -- 172.21.15.16:0/1841642737 shutdown_connections 2023-09-30T22:31:19.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.303+0000 7f55eec28700 1 -- 172.21.15.16:0/1841642737 wait complete. 2023-09-30T22:31:19.310 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:31:19.318 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:31:19.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.395+0000 7f6e3c5f3700 1 -- 172.21.15.16:0/3320154843 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6e34072980 con 0x7f6e340a0a30 2023-09-30T22:31:19.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.395+0000 7f6e2affd700 1 -- 172.21.15.16:0/3320154843 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6e2401a450 con 0x7f6e340a0a30 2023-09-30T22:31:19.397 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:31:19.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.397+0000 7f6e3c5f3700 1 -- 172.21.15.16:0/3320154843 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6e340a0a30 msgr2=0x7f6e341c84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:19.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.397+0000 7f6e3c5f3700 1 --2- 172.21.15.16:0/3320154843 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6e340a0a30 0x7f6e341c84c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6e24016000 tx=0x7f6e24004750 comp rx=0 tx=0).stop 2023-09-30T22:31:19.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.397+0000 7f6e3c5f3700 1 -- 172.21.15.16:0/3320154843 shutdown_connections 2023-09-30T22:31:19.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.397+0000 7f6e3c5f3700 1 --2- 172.21.15.16:0/3320154843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e34133930 0x7f6e341c8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.397+0000 7f6e3c5f3700 1 --2- 172.21.15.16:0/3320154843 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e3409d630 0x7f6e341c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.397+0000 7f6e3c5f3700 1 --2- 172.21.15.16:0/3320154843 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6e340a0a30 0x7f6e341c84c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:19.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.397+0000 7f6e3c5f3700 1 -- 172.21.15.16:0/3320154843 >> 172.21.15.16:0/3320154843 conn(0x7f6e340a3050 msgr2=0x7f6e3409ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:19.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.397+0000 7f6e3c5f3700 1 -- 172.21.15.16:0/3320154843 shutdown_connections 2023-09-30T22:31:19.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:19.397+0000 7f6e3c5f3700 1 -- 172.21.15.16:0/3320154843 wait complete. 2023-09-30T22:31:19.412 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:31:20.241 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:20.311 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:20.319 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:20.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.389+0000 7fd1a440c700 1 Processor -- start 2023-09-30T22:31:20.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.389+0000 7fd1a440c700 1 -- start start 2023-09-30T22:31:20.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.390+0000 7fd1a440c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd19c131810 0x7fd19c131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:20.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.390+0000 7fd1a440c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd19c09a580 0x7fd19c139660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:20.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.390+0000 7fd1a440c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd19c1313c0 con 0x7fd19c099a40 2023-09-30T22:31:20.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.390+0000 7fd1a440c700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd19c09b580 con 0x7fd19c09a580 2023-09-30T22:31:20.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.390+0000 7fd1a440c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c06e8c0 con 0x7fd19c131810 2023-09-30T22:31:20.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.390+0000 7fd1a21a8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd19c131810 0x7fd19c131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:20.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.390+0000 7fd1a19a7700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd19c099a40 0x7fd19c099e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58022/0 (socket says 172.21.15.16:58022) 2023-09-30T22:31:20.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.390+0000 7fd1a19a7700 1 -- 172.21.15.16:0/2536613792 learned_addr learned my addr 172.21.15.16:0/2536613792 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:31:20.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.390+0000 7fd1a29a9700 1 --2- 172.21.15.16:0/2536613792 >> v2:172.21.15.16:3300/0 conn(0x7fd19c09a580 0x7fd19c139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:20.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.390+0000 7fd1a11a6700 1 -- 172.21.15.16:0/2536613792 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2221241131 0 0) 0x7fd19c1313c0 con 0x7fd19c099a40 2023-09-30T22:31:20.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.390+0000 7fd1a11a6700 1 -- 172.21.15.16:0/2536613792 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1880035b0 con 0x7fd19c099a40 2023-09-30T22:31:20.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.390+0000 7fd1a29a9700 1 -- 172.21.15.16:0/2536613792 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd19c131810 msgr2=0x7fd19c131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:20.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.390+0000 7fd1a29a9700 1 --2- 172.21.15.16:0/2536613792 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd19c131810 0x7fd19c131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.390+0000 7fd1a29a9700 1 -- 172.21.15.16:0/2536613792 >> v1:172.21.15.16:6789/0 conn(0x7fd19c099a40 legacy=0x7fd19c099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:20.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.390+0000 7fd1a29a9700 1 -- 172.21.15.16:0/2536613792 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd19c071d20 con 0x7fd19c09a580 2023-09-30T22:31:20.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.391+0000 7fd1a21a8700 1 --2- 172.21.15.16:0/2536613792 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd19c131810 0x7fd19c131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:20.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.391+0000 7fd1a29a9700 1 --2- 172.21.15.16:0/2536613792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd19c09a580 0x7fd19c139660 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fd194009d90 tx=0x7fd19400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=415c09e97daea3a3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:20.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.391+0000 7fd1a11a6700 1 -- 172.21.15.16:0/2536613792 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19400e070 con 0x7fd19c09a580 2023-09-30T22:31:20.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.391+0000 7fd1a11a6700 1 -- 172.21.15.16:0/2536613792 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd19400bb20 con 0x7fd19c09a580 2023-09-30T22:31:20.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.391+0000 7fd1a11a6700 1 -- 172.21.15.16:0/2536613792 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd194012430 con 0x7fd19c09a580 2023-09-30T22:31:20.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.392+0000 7fd1a440c700 1 -- 172.21.15.16:0/2536613792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd19c09a580 msgr2=0x7fd19c139660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:20.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.392+0000 7fd1a440c700 1 --2- 172.21.15.16:0/2536613792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd19c09a580 0x7fd19c139660 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fd194009d90 tx=0x7fd19400b330 comp rx=0 tx=0).stop 2023-09-30T22:31:20.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.392+0000 7fd1a440c700 1 -- 172.21.15.16:0/2536613792 shutdown_connections 2023-09-30T22:31:20.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.392+0000 7fd1a440c700 1 --2- 172.21.15.16:0/2536613792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd19c09a580 0x7fd19c139660 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.392+0000 7fd1a440c700 1 --2- 172.21.15.16:0/2536613792 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd19c131810 0x7fd19c131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.392+0000 7fd1a440c700 1 -- 172.21.15.16:0/2536613792 >> 172.21.15.16:0/2536613792 conn(0x7fd19c0a2aa0 msgr2=0x7fd19c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:20.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.392+0000 7fd1a440c700 1 -- 172.21.15.16:0/2536613792 shutdown_connections 2023-09-30T22:31:20.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.392+0000 7fd1a440c700 1 -- 172.21.15.16:0/2536613792 wait complete. 2023-09-30T22:31:20.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.394+0000 7fd1a440c700 1 Processor -- start 2023-09-30T22:31:20.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.394+0000 7fd1a440c700 1 -- start start 2023-09-30T22:31:20.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.394+0000 7fd1a440c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd19c099a40 0x7fd19c11b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:20.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.394+0000 7fd1a440c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd19c09a580 0x7fd19c11bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:20.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.394+0000 7fd1a440c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd19c131810 0x7fd19c11c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:20.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.394+0000 7fd1a440c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c11e9f0 con 0x7fd19c099a40 2023-09-30T22:31:20.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.394+0000 7fd1a440c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c06eb00 con 0x7fd19c131810 2023-09-30T22:31:20.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.394+0000 7fd1a440c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd19c11df50 con 0x7fd19c09a580 2023-09-30T22:31:20.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.395+0000 7fd1a19a7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd19c09a580 0x7fd19c11bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:20.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.395+0000 7fd1a19a7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd19c09a580 0x7fd19c11bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:38506/0 (socket says 172.21.15.16:38506) 2023-09-30T22:31:20.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.395+0000 7fd1a19a7700 1 -- 172.21.15.16:0/2907560305 learned_addr learned my addr 172.21.15.16:0/2907560305 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:20.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.395+0000 7fd1a21a8700 1 --2- 172.21.15.16:0/2907560305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd19c099a40 0x7fd19c11b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:20.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.395+0000 7fd1a29a9700 1 --2- 172.21.15.16:0/2907560305 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd19c131810 0x7fd19c11c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:20.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.395+0000 7fd1a21a8700 1 -- 172.21.15.16:0/2907560305 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd19c09a580 msgr2=0x7fd19c11bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:20.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.395+0000 7fd1a21a8700 1 --2- 172.21.15.16:0/2907560305 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd19c09a580 0x7fd19c11bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.395+0000 7fd1a21a8700 1 -- 172.21.15.16:0/2907560305 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd19c131810 msgr2=0x7fd19c11c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:20.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.395+0000 7fd1a21a8700 1 --2- 172.21.15.16:0/2907560305 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd19c131810 0x7fd19c11c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.395+0000 7fd1a21a8700 1 -- 172.21.15.16:0/2907560305 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1980094a0 con 0x7fd19c099a40 2023-09-30T22:31:20.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.395+0000 7fd1a19a7700 1 --2- 172.21.15.16:0/2907560305 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd19c09a580 0x7fd19c11bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:20.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.395+0000 7fd1a21a8700 1 --2- 172.21.15.16:0/2907560305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd19c099a40 0x7fd19c11b800 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd19800e9b0 tx=0x7fd19800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:20.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.395+0000 7fd192ffd700 1 -- 172.21.15.16:0/2907560305 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19800ee50 con 0x7fd19c099a40 2023-09-30T22:31:20.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.395+0000 7fd192ffd700 1 -- 172.21.15.16:0/2907560305 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd19800c850 con 0x7fd19c099a40 2023-09-30T22:31:20.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.396+0000 7fd1a440c700 1 -- 172.21.15.16:0/2907560305 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd194009a70 con 0x7fd19c099a40 2023-09-30T22:31:20.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.396+0000 7fd192ffd700 1 -- 172.21.15.16:0/2907560305 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19800cc00 con 0x7fd19c099a40 2023-09-30T22:31:20.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.396+0000 7fd1a440c700 1 -- 172.21.15.16:0/2907560305 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd19c072250 con 0x7fd19c099a40 2023-09-30T22:31:20.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.397+0000 7fd192ffd700 1 -- 172.21.15.16:0/2907560305 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd198013470 con 0x7fd19c099a40 2023-09-30T22:31:20.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.397+0000 7fd192ffd700 1 -- 172.21.15.16:0/2907560305 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd19803fa90 con 0x7fd19c099a40 2023-09-30T22:31:20.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.397+0000 7fd1a440c700 1 -- 172.21.15.16:0/2907560305 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd19c074ea0 con 0x7fd19c099a40 2023-09-30T22:31:20.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.402+0000 7fd192ffd700 1 -- 172.21.15.16:0/2907560305 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd198013720 con 0x7fd19c099a40 2023-09-30T22:31:20.413 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:20.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.455+0000 7fcda05d1700 1 Processor -- start 2023-09-30T22:31:20.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.455+0000 7fcda05d1700 1 -- start start 2023-09-30T22:31:20.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.456+0000 7fcda05d1700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcd98133420 0x7fcd981337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:20.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.456+0000 7fcda05d1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd98133e30 0x7fcd9812ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:20.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.456+0000 7fcda05d1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd98134260 0x7fcd981304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:20.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.456+0000 7fcda05d1700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd9806b990 con 0x7fcd98133420 2023-09-30T22:31:20.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.456+0000 7fcda05d1700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd980a0590 con 0x7fcd98133e30 2023-09-30T22:31:20.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.456+0000 7fcda05d1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9809fb80 con 0x7fcd98134260 2023-09-30T22:31:20.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.456+0000 7fcd9e36d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcd98133420 0x7fcd981337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:20.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.456+0000 7fcd9e36d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcd98133420 0x7fcd981337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57114/0 (socket says 172.21.15.16:57114) 2023-09-30T22:31:20.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.456+0000 7fcd9e36d700 1 -- 172.21.15.16:0/2570304310 learned_addr learned my addr 172.21.15.16:0/2570304310 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:20.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.456+0000 7fcd9db6c700 1 --2- 172.21.15.16:0/2570304310 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd98133e30 0x7fcd9812ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:20.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.457+0000 7fcd9eb6e700 1 --2- 172.21.15.16:0/2570304310 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd98134260 0x7fcd981304d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:20.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.457+0000 7fcd9db6c700 1 -- 172.21.15.16:0/2570304310 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd98134260 msgr2=0x7fcd981304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:20.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.457+0000 7fcd9db6c700 1 --2- 172.21.15.16:0/2570304310 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd98134260 0x7fcd981304d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.457+0000 7fcd9db6c700 1 -- 172.21.15.16:0/2570304310 >> v2:172.21.15.16:3300/0 conn(0x7fcd98133420 msgr2=0x7fcd981337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:20.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.457+0000 7fcd9db6c700 1 --2- 172.21.15.16:0/2570304310 >> v2:172.21.15.16:3300/0 conn(0x7fcd98133420 0x7fcd981337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.457+0000 7fcd9db6c700 1 -- 172.21.15.16:0/2570304310 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd980a0030 con 0x7fcd98133e30 2023-09-30T22:31:20.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.457+0000 7fcd9db6c700 1 --2- 172.21.15.16:0/2570304310 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd98133e30 0x7fcd9812ff90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fcd88009840 tx=0x7fcd88009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5bff9d7665788cb8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:20.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.457+0000 7fcd9d36b700 1 -- 172.21.15.16:0/2570304310 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd88011830 con 0x7fcd98133e30 2023-09-30T22:31:20.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.457+0000 7fcd9d36b700 1 -- 172.21.15.16:0/2570304310 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd88011990 con 0x7fcd98133e30 2023-09-30T22:31:20.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.457+0000 7fcd9d36b700 1 -- 172.21.15.16:0/2570304310 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd88011d20 con 0x7fcd98133e30 2023-09-30T22:31:20.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.458+0000 7fcda05d1700 1 -- 172.21.15.16:0/2570304310 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd98133e30 msgr2=0x7fcd9812ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:20.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.458+0000 7fcda05d1700 1 --2- 172.21.15.16:0/2570304310 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd98133e30 0x7fcd9812ff90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fcd88009840 tx=0x7fcd88009b50 comp rx=0 tx=0).stop 2023-09-30T22:31:20.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.458+0000 7fcda05d1700 1 -- 172.21.15.16:0/2570304310 shutdown_connections 2023-09-30T22:31:20.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.458+0000 7fcda05d1700 1 --2- 172.21.15.16:0/2570304310 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd98134260 0x7fcd981304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.458+0000 7fcda05d1700 1 --2- 172.21.15.16:0/2570304310 >> v2:172.21.15.16:3300/0 conn(0x7fcd98133420 0x7fcd981337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.458+0000 7fcda05d1700 1 --2- 172.21.15.16:0/2570304310 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd98133e30 0x7fcd9812ff90 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.458+0000 7fcda05d1700 1 -- 172.21.15.16:0/2570304310 >> 172.21.15.16:0/2570304310 conn(0x7fcd980a4210 msgr2=0x7fcd980a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:20.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.458+0000 7fcda05d1700 1 -- 172.21.15.16:0/2570304310 shutdown_connections 2023-09-30T22:31:20.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.458+0000 7fcda05d1700 1 -- 172.21.15.16:0/2570304310 wait complete. 2023-09-30T22:31:20.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.459+0000 7fcda05d1700 1 Processor -- start 2023-09-30T22:31:20.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.459+0000 7fcda05d1700 1 -- start start 2023-09-30T22:31:20.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.460+0000 7fcda05d1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd98133420 0x7fcd9809c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:20.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.460+0000 7fcda05d1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd98133e30 0x7fcd9809c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:20.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.460+0000 7fcda05d1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd98134260 0x7fcd9809cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:20.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.460+0000 7fcda05d1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9812dc80 con 0x7fcd98133420 2023-09-30T22:31:20.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.460+0000 7fcda05d1700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9812da20 con 0x7fcd98133e30 2023-09-30T22:31:20.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.460+0000 7fcda05d1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcd98120400 con 0x7fcd98134260 2023-09-30T22:31:20.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.460+0000 7fcd9e36d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd98133420 0x7fcd9809c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:20.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.460+0000 7fcd9e36d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd98133420 0x7fcd9809c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57120/0 (socket says 172.21.15.16:57120) 2023-09-30T22:31:20.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.460+0000 7fcd9e36d700 1 -- 172.21.15.16:0/2067495758 learned_addr learned my addr 172.21.15.16:0/2067495758 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:20.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.460+0000 7fcd9db6c700 1 --2- 172.21.15.16:0/2067495758 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd98133e30 0x7fcd9809c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:20.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.460+0000 7fcd9eb6e700 1 --2- 172.21.15.16:0/2067495758 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd98134260 0x7fcd9809cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:20.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.460+0000 7fcd9db6c700 1 -- 172.21.15.16:0/2067495758 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd98134260 msgr2=0x7fcd9809cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:20.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.460+0000 7fcd9db6c700 1 --2- 172.21.15.16:0/2067495758 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd98134260 0x7fcd9809cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.460+0000 7fcd9db6c700 1 -- 172.21.15.16:0/2067495758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd98133420 msgr2=0x7fcd9809c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:20.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.460+0000 7fcd9db6c700 1 --2- 172.21.15.16:0/2067495758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd98133420 0x7fcd9809c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.460+0000 7fcd9db6c700 1 -- 172.21.15.16:0/2067495758 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd880094a0 con 0x7fcd98133e30 2023-09-30T22:31:20.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.461+0000 7fcd9db6c700 1 --2- 172.21.15.16:0/2067495758 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd98133e30 0x7fcd9809c9c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fcd88016000 tx=0x7fcd88004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:20.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.461+0000 7fcd8effd700 1 -- 172.21.15.16:0/2067495758 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd88004240 con 0x7fcd98133e30 2023-09-30T22:31:20.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.461+0000 7fcd8effd700 1 -- 172.21.15.16:0/2067495758 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd8801f070 con 0x7fcd98133e30 2023-09-30T22:31:20.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.461+0000 7fcda05d1700 1 -- 172.21.15.16:0/2067495758 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd9811f970 con 0x7fcd98133e30 2023-09-30T22:31:20.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.461+0000 7fcd8effd700 1 -- 172.21.15.16:0/2067495758 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd880036a0 con 0x7fcd98133e30 2023-09-30T22:31:20.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.461+0000 7fcda05d1700 1 -- 172.21.15.16:0/2067495758 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd9811ff00 con 0x7fcd98133e30 2023-09-30T22:31:20.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.462+0000 7fcd8effd700 1 -- 172.21.15.16:0/2067495758 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcd88021030 con 0x7fcd98133e30 2023-09-30T22:31:20.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.462+0000 7fcda05d1700 1 -- 172.21.15.16:0/2067495758 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd9811fd50 con 0x7fcd98133e30 2023-09-30T22:31:20.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.462+0000 7fcd8effd700 1 -- 172.21.15.16:0/2067495758 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fcd88040980 con 0x7fcd98133e30 2023-09-30T22:31:20.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.472+0000 7fcd8effd700 1 -- 172.21.15.16:0/2067495758 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcd880239c0 con 0x7fcd98133e30 2023-09-30T22:31:20.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.501+0000 7f934782b700 1 Processor -- start 2023-09-30T22:31:20.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.501+0000 7f934782b700 1 -- start start 2023-09-30T22:31:20.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.501+0000 7f934782b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f934006b250 0x7f9340136840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:20.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.501+0000 7f934782b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9340136d80 0x7f9340139160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:20.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.501+0000 7f934782b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f934009ece0 con 0x7f934006a770 2023-09-30T22:31:20.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.501+0000 7f934782b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f934009e760 con 0x7f934006b250 2023-09-30T22:31:20.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.501+0000 7f934782b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f934009e040 con 0x7f9340136d80 2023-09-30T22:31:20.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.502+0000 7f9345dc8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9340136d80 0x7f9340139160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:20.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.502+0000 7f9345dc8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9340136d80 0x7f9340139160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:57356/0 (socket says 172.21.15.16:57356) 2023-09-30T22:31:20.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.502+0000 7f9345dc8700 1 -- 172.21.15.16:0/4054794286 learned_addr learned my addr 172.21.15.16:0/4054794286 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:20.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.502+0000 7f9344dc6700 1 --2- 172.21.15.16:0/4054794286 >> v2:172.21.15.16:3300/0 conn(0x7f934006b250 0x7f9340136840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:20.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.502+0000 7f9345dc8700 1 -- 172.21.15.16:0/4054794286 >> v2:172.21.15.16:3300/0 conn(0x7f934006b250 msgr2=0x7f9340136840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:20.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.502+0000 7f9345dc8700 1 --2- 172.21.15.16:0/4054794286 >> v2:172.21.15.16:3300/0 conn(0x7f934006b250 0x7f9340136840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.502+0000 7f9345dc8700 1 -- 172.21.15.16:0/4054794286 >> v1:172.21.15.16:6789/0 conn(0x7f934006a770 legacy=0x7f934006ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:20.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.502+0000 7f9345dc8700 1 -- 172.21.15.16:0/4054794286 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f934009f630 con 0x7f9340136d80 2023-09-30T22:31:20.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.502+0000 7f9345dc8700 1 --2- 172.21.15.16:0/4054794286 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9340136d80 0x7f9340139160 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9338009e50 tx=0x7f933800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6283f12f7e90b387 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:20.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.503+0000 7f9337fff700 1 -- 172.21.15.16:0/4054794286 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933800e070 con 0x7f9340136d80 2023-09-30T22:31:20.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.503+0000 7f9337fff700 1 -- 172.21.15.16:0/4054794286 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f933800b910 con 0x7f9340136d80 2023-09-30T22:31:20.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.503+0000 7f9337fff700 1 -- 172.21.15.16:0/4054794286 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933800bcc0 con 0x7f9340136d80 2023-09-30T22:31:20.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.503+0000 7f934782b700 1 -- 172.21.15.16:0/4054794286 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9340136d80 msgr2=0x7f9340139160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:20.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.503+0000 7f934782b700 1 --2- 172.21.15.16:0/4054794286 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9340136d80 0x7f9340139160 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9338009e50 tx=0x7f933800b330 comp rx=0 tx=0).stop 2023-09-30T22:31:20.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.503+0000 7f934782b700 1 -- 172.21.15.16:0/4054794286 shutdown_connections 2023-09-30T22:31:20.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.503+0000 7f934782b700 1 --2- 172.21.15.16:0/4054794286 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9340136d80 0x7f9340139160 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.503+0000 7f934782b700 1 --2- 172.21.15.16:0/4054794286 >> v2:172.21.15.16:3300/0 conn(0x7f934006b250 0x7f9340136840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.503+0000 7f934782b700 1 -- 172.21.15.16:0/4054794286 >> 172.21.15.16:0/4054794286 conn(0x7f9340066f50 msgr2=0x7f934009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:20.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.503+0000 7f934782b700 1 -- 172.21.15.16:0/4054794286 shutdown_connections 2023-09-30T22:31:20.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.503+0000 7f934782b700 1 -- 172.21.15.16:0/4054794286 wait complete. 2023-09-30T22:31:20.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.504+0000 7f934782b700 1 Processor -- start 2023-09-30T22:31:20.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.505+0000 7f934782b700 1 -- start start 2023-09-30T22:31:20.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.505+0000 7f934782b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f934006a770 0x7f934011ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:20.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.505+0000 7f934782b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f934006b250 0x7f934011d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:20.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.505+0000 7f934782b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9340136d80 0x7f93401c5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:20.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.505+0000 7f934782b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93400718e0 con 0x7f934006a770 2023-09-30T22:31:20.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.505+0000 7f934782b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f934011fad0 con 0x7f9340136d80 2023-09-30T22:31:20.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.505+0000 7f934782b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9340071e60 con 0x7f934006b250 2023-09-30T22:31:20.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.505+0000 7f9344dc6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f934006b250 0x7f934011d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:20.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.505+0000 7f93455c7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f934006a770 0x7f934011ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:20.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.506+0000 7f9344dc6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f934006b250 0x7f934011d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:38540/0 (socket says 172.21.15.16:38540) 2023-09-30T22:31:20.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.506+0000 7f9344dc6700 1 -- 172.21.15.16:0/3475428387 learned_addr learned my addr 172.21.15.16:0/3475428387 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:20.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.506+0000 7f9345dc8700 1 --2- 172.21.15.16:0/3475428387 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9340136d80 0x7f93401c5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:20.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.506+0000 7f93455c7700 1 -- 172.21.15.16:0/3475428387 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f934006b250 msgr2=0x7f934011d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:20.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.506+0000 7f93455c7700 1 --2- 172.21.15.16:0/3475428387 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f934006b250 0x7f934011d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.506+0000 7f93455c7700 1 -- 172.21.15.16:0/3475428387 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9340136d80 msgr2=0x7f93401c5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:20.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.506+0000 7f93455c7700 1 --2- 172.21.15.16:0/3475428387 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9340136d80 0x7f93401c5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.506+0000 7f93455c7700 1 -- 172.21.15.16:0/3475428387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9338009b30 con 0x7f934006a770 2023-09-30T22:31:20.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.506+0000 7f93455c7700 1 --2- 172.21.15.16:0/3475428387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f934006a770 0x7f934011ce40 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f933c000f60 tx=0x7f933c00c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:20.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.506+0000 7f9335ffb700 1 -- 172.21.15.16:0/3475428387 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933c0052d0 con 0x7f934006a770 2023-09-30T22:31:20.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.506+0000 7f9335ffb700 1 -- 172.21.15.16:0/3475428387 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f933c005430 con 0x7f934006a770 2023-09-30T22:31:20.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.506+0000 7f934782b700 1 -- 172.21.15.16:0/3475428387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9340120800 con 0x7f934006a770 2023-09-30T22:31:20.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.507+0000 7f9335ffb700 1 -- 172.21.15.16:0/3475428387 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f933c0038c0 con 0x7f934006a770 2023-09-30T22:31:20.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.507+0000 7f934782b700 1 -- 172.21.15.16:0/3475428387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9340120070 con 0x7f934006a770 2023-09-30T22:31:20.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.508+0000 7f934782b700 1 -- 172.21.15.16:0/3475428387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9340074ea0 con 0x7f934006a770 2023-09-30T22:31:20.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.508+0000 7f9335ffb700 1 -- 172.21.15.16:0/3475428387 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f933c003a20 con 0x7f934006a770 2023-09-30T22:31:20.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.509+0000 7f9335ffb700 1 -- 172.21.15.16:0/3475428387 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f933c00ce30 con 0x7f934006a770 2023-09-30T22:31:20.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.518+0000 7f9335ffb700 1 -- 172.21.15.16:0/3475428387 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f933c044060 con 0x7f934006a770 2023-09-30T22:31:20.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.561+0000 7f38b5e87700 1 Processor -- start 2023-09-30T22:31:20.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.561+0000 7f38b5e87700 1 -- start start 2023-09-30T22:31:20.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.561+0000 7f38b5e87700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38b006e570 0x7f38b006e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:20.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.561+0000 7f38b5e87700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f38b009eca0 0x7f38b006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:20.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.561+0000 7f38b5e87700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f38b0134610 0x7f38b0134a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:20.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.561+0000 7f38b5e87700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38b0070ee0 con 0x7f38b009eca0 2023-09-30T22:31:20.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.561+0000 7f38b5e87700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38b0071480 con 0x7f38b006e570 2023-09-30T22:31:20.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.561+0000 7f38b5e87700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f38b006f3c0 con 0x7f38b0134610 2023-09-30T22:31:20.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.562+0000 7f38af7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38b006e570 0x7f38b006e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:20.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.562+0000 7f38af7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38b006e570 0x7f38b006e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:57376/0 (socket says 172.21.15.16:57376) 2023-09-30T22:31:20.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.562+0000 7f38af7fe700 1 -- 172.21.15.16:0/2289230461 learned_addr learned my addr 172.21.15.16:0/2289230461 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:20.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.562+0000 7f38aeffd700 1 --2- 172.21.15.16:0/2289230461 >> v2:172.21.15.16:3300/0 conn(0x7f38b009eca0 0x7f38b006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:20.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.562+0000 7f38affff700 1 --2- 172.21.15.16:0/2289230461 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f38b0134610 0x7f38b0134a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:20.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.563+0000 7f38af7fe700 1 -- 172.21.15.16:0/2289230461 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f38b0134610 msgr2=0x7f38b0134a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:20.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.563+0000 7f38af7fe700 1 --2- 172.21.15.16:0/2289230461 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f38b0134610 0x7f38b0134a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.563+0000 7f38af7fe700 1 -- 172.21.15.16:0/2289230461 >> v2:172.21.15.16:3300/0 conn(0x7f38b009eca0 msgr2=0x7f38b006ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:20.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.563+0000 7f38af7fe700 1 --2- 172.21.15.16:0/2289230461 >> v2:172.21.15.16:3300/0 conn(0x7f38b009eca0 0x7f38b006ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.563+0000 7f38af7fe700 1 -- 172.21.15.16:0/2289230461 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b0071b90 con 0x7f38b006e570 2023-09-30T22:31:20.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.563+0000 7f38af7fe700 1 --2- 172.21.15.16:0/2289230461 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38b006e570 0x7f38b006e940 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f38a00098f0 tx=0x7f38a0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5b25ac94a07ac4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:20.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.563+0000 7f38ae7fc700 1 -- 172.21.15.16:0/2289230461 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a0011830 con 0x7f38b006e570 2023-09-30T22:31:20.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.563+0000 7f38ae7fc700 1 -- 172.21.15.16:0/2289230461 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38a0011990 con 0x7f38b006e570 2023-09-30T22:31:20.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.563+0000 7f38ae7fc700 1 -- 172.21.15.16:0/2289230461 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a0011d20 con 0x7f38b006e570 2023-09-30T22:31:20.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.564+0000 7f38b5e87700 1 -- 172.21.15.16:0/2289230461 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38b006e570 msgr2=0x7f38b006e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:20.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.564+0000 7f38b5e87700 1 --2- 172.21.15.16:0/2289230461 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38b006e570 0x7f38b006e940 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f38a00098f0 tx=0x7f38a0009c00 comp rx=0 tx=0).stop 2023-09-30T22:31:20.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.564+0000 7f38b5e87700 1 -- 172.21.15.16:0/2289230461 shutdown_connections 2023-09-30T22:31:20.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.564+0000 7f38b5e87700 1 --2- 172.21.15.16:0/2289230461 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38b006e570 0x7f38b006e940 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.564+0000 7f38b5e87700 1 --2- 172.21.15.16:0/2289230461 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f38b0134610 0x7f38b0134a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.564+0000 7f38b5e87700 1 --2- 172.21.15.16:0/2289230461 >> v2:172.21.15.16:3300/0 conn(0x7f38b009eca0 0x7f38b006ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.564+0000 7f38b5e87700 1 -- 172.21.15.16:0/2289230461 >> 172.21.15.16:0/2289230461 conn(0x7f38b00a2aa0 msgr2=0x7f38b00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:20.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.564+0000 7f38b5e87700 1 -- 172.21.15.16:0/2289230461 shutdown_connections 2023-09-30T22:31:20.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.564+0000 7f38b5e87700 1 -- 172.21.15.16:0/2289230461 wait complete. 2023-09-30T22:31:20.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.566+0000 7f38b5e87700 1 Processor -- start 2023-09-30T22:31:20.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.566+0000 7f38b5e87700 1 -- start start 2023-09-30T22:31:20.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.566+0000 7f38b5e87700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38b006e570 0x7f38b01214e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:20.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.566+0000 7f38b5e87700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38b0134610 0x7f38b0121a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:20.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.566+0000 7f38b5e87700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f38b0124ea0 0x7f38b0121f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:20.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.566+0000 7f38b5e87700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b0123d70 con 0x7f38b006e570 2023-09-30T22:31:20.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.566+0000 7f38b5e87700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38b0070610 con 0x7f38b0134610 2023-09-30T22:31:20.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.566+0000 7f38b5e87700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f38b0123070 con 0x7f38b0124ea0 2023-09-30T22:31:20.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.567+0000 7f38affff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f38b0124ea0 0x7f38b0121f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:20.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.567+0000 7f38affff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f38b0124ea0 0x7f38b0121f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:38554/0 (socket says 172.21.15.16:38554) 2023-09-30T22:31:20.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.567+0000 7f38affff700 1 -- 172.21.15.16:0/3856394185 learned_addr learned my addr 172.21.15.16:0/3856394185 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:20.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.567+0000 7f38aeffd700 1 --2- 172.21.15.16:0/3856394185 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38b0134610 0x7f38b0121a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:20.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.567+0000 7f38af7fe700 1 --2- 172.21.15.16:0/3856394185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38b006e570 0x7f38b01214e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:20.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.567+0000 7f38affff700 1 -- 172.21.15.16:0/3856394185 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38b0134610 msgr2=0x7f38b0121a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:20.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.567+0000 7f38affff700 1 --2- 172.21.15.16:0/3856394185 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38b0134610 0x7f38b0121a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.567+0000 7f38affff700 1 -- 172.21.15.16:0/3856394185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38b006e570 msgr2=0x7f38b01214e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:20.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.567+0000 7f38affff700 1 --2- 172.21.15.16:0/3856394185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38b006e570 0x7f38b01214e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.567+0000 7f38affff700 1 -- 172.21.15.16:0/3856394185 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a0009550 con 0x7f38b0124ea0 2023-09-30T22:31:20.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.567+0000 7f38af7fe700 1 --2- 172.21.15.16:0/3856394185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38b006e570 0x7f38b01214e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:31:20.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.567+0000 7f38aeffd700 1 --2- 172.21.15.16:0/3856394185 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38b0134610 0x7f38b0121a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:20.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.567+0000 7f38affff700 1 --2- 172.21.15.16:0/3856394185 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f38b0124ea0 0x7f38b0121f60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f38a800efc0 tx=0x7f38a800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:20.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.568+0000 7f3897fff700 1 -- 172.21.15.16:0/3856394185 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a8009ea0 con 0x7f38b0124ea0 2023-09-30T22:31:20.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.568+0000 7f38b5e87700 1 -- 172.21.15.16:0/3856394185 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38b0124130 con 0x7f38b0124ea0 2023-09-30T22:31:20.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.568+0000 7f38b5e87700 1 -- 172.21.15.16:0/3856394185 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38b0122a90 con 0x7f38b0124ea0 2023-09-30T22:31:20.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.568+0000 7f3897fff700 1 -- 172.21.15.16:0/3856394185 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38a800f040 con 0x7f38b0124ea0 2023-09-30T22:31:20.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.568+0000 7f3897fff700 1 -- 172.21.15.16:0/3856394185 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a8014480 con 0x7f38b0124ea0 2023-09-30T22:31:20.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.569+0000 7f38b5e87700 1 -- 172.21.15.16:0/3856394185 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38900052e0 con 0x7f38b0124ea0 2023-09-30T22:31:20.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.570+0000 7f3897fff700 1 -- 172.21.15.16:0/3856394185 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f38a80145e0 con 0x7f38b0124ea0 2023-09-30T22:31:20.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.570+0000 7f3897fff700 1 -- 172.21.15.16:0/3856394185 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f38a80408c0 con 0x7f38b0124ea0 2023-09-30T22:31:20.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.578+0000 7f3897fff700 1 -- 172.21.15.16:0/3856394185 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f38a8012930 con 0x7f38b0124ea0 2023-09-30T22:31:20.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.633+0000 7fd1a440c700 1 -- 172.21.15.16:0/2907560305 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd19c06b9e0 con 0x7fd19c099a40 2023-09-30T22:31:20.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.634+0000 7fd192ffd700 1 -- 172.21.15.16:0/2907560305 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd198021030 con 0x7fd19c099a40 2023-09-30T22:31:20.635 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:31:20.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.636+0000 7fd1a440c700 1 -- 172.21.15.16:0/2907560305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd19c099a40 msgr2=0x7fd19c11b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:20.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.636+0000 7fd1a440c700 1 --2- 172.21.15.16:0/2907560305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd19c099a40 0x7fd19c11b800 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd19800e9b0 tx=0x7fd19800ecc0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.636+0000 7fd1a440c700 1 -- 172.21.15.16:0/2907560305 shutdown_connections 2023-09-30T22:31:20.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.636+0000 7fd1a440c700 1 --2- 172.21.15.16:0/2907560305 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd19c131810 0x7fd19c11c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.636+0000 7fd1a440c700 1 --2- 172.21.15.16:0/2907560305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd19c099a40 0x7fd19c11b800 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.636+0000 7fd1a440c700 1 --2- 172.21.15.16:0/2907560305 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd19c09a580 0x7fd19c11bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.636+0000 7fd1a440c700 1 -- 172.21.15.16:0/2907560305 >> 172.21.15.16:0/2907560305 conn(0x7fd19c0a2aa0 msgr2=0x7fd19c09d770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:20.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.636+0000 7fd1a440c700 1 -- 172.21.15.16:0/2907560305 shutdown_connections 2023-09-30T22:31:20.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.636+0000 7fd1a440c700 1 -- 172.21.15.16:0/2907560305 wait complete. 2023-09-30T22:31:20.652 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:31:20.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.699+0000 7fcda05d1700 1 -- 172.21.15.16:0/2067495758 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fcd9811f520 con 0x7fcd98133e30 2023-09-30T22:31:20.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.699+0000 7fcd8effd700 1 -- 172.21.15.16:0/2067495758 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcd88015030 con 0x7fcd98133e30 2023-09-30T22:31:20.701 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:31:20.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.701+0000 7fcda05d1700 1 -- 172.21.15.16:0/2067495758 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd98133e30 msgr2=0x7fcd9809c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:20.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.701+0000 7fcda05d1700 1 --2- 172.21.15.16:0/2067495758 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd98133e30 0x7fcd9809c9c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fcd88016000 tx=0x7fcd88004750 comp rx=0 tx=0).stop 2023-09-30T22:31:20.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.701+0000 7fcda05d1700 1 -- 172.21.15.16:0/2067495758 shutdown_connections 2023-09-30T22:31:20.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.701+0000 7fcda05d1700 1 --2- 172.21.15.16:0/2067495758 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd98134260 0x7fcd9809cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.701+0000 7fcda05d1700 1 --2- 172.21.15.16:0/2067495758 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd98133420 0x7fcd9809c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.701+0000 7fcda05d1700 1 --2- 172.21.15.16:0/2067495758 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd98133e30 0x7fcd9809c9c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.701+0000 7fcda05d1700 1 -- 172.21.15.16:0/2067495758 >> 172.21.15.16:0/2067495758 conn(0x7fcd980a4210 msgr2=0x7fcd980a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:20.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.702+0000 7fcda05d1700 1 -- 172.21.15.16:0/2067495758 shutdown_connections 2023-09-30T22:31:20.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.702+0000 7fcda05d1700 1 -- 172.21.15.16:0/2067495758 wait complete. 2023-09-30T22:31:20.722 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:31:20.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.746+0000 7f934782b700 1 -- 172.21.15.16:0/3475428387 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f934006c360 con 0x7f934006a770 2023-09-30T22:31:20.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.746+0000 7f9335ffb700 1 -- 172.21.15.16:0/3475428387 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f933c011590 con 0x7f934006a770 2023-09-30T22:31:20.747 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:31:20.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.748+0000 7f934782b700 1 -- 172.21.15.16:0/3475428387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f934006a770 msgr2=0x7f934011ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:20.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.748+0000 7f934782b700 1 --2- 172.21.15.16:0/3475428387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f934006a770 0x7f934011ce40 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f933c000f60 tx=0x7f933c00c730 comp rx=0 tx=0).stop 2023-09-30T22:31:20.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.748+0000 7f934782b700 1 -- 172.21.15.16:0/3475428387 shutdown_connections 2023-09-30T22:31:20.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.748+0000 7f934782b700 1 --2- 172.21.15.16:0/3475428387 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9340136d80 0x7f93401c5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.748+0000 7f934782b700 1 --2- 172.21.15.16:0/3475428387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f934006a770 0x7f934011ce40 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.748+0000 7f934782b700 1 --2- 172.21.15.16:0/3475428387 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f934006b250 0x7f934011d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.748+0000 7f934782b700 1 -- 172.21.15.16:0/3475428387 >> 172.21.15.16:0/3475428387 conn(0x7f9340066f50 msgr2=0x7f9340070170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:20.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.749+0000 7f934782b700 1 -- 172.21.15.16:0/3475428387 shutdown_connections 2023-09-30T22:31:20.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.749+0000 7f934782b700 1 -- 172.21.15.16:0/3475428387 wait complete. 2023-09-30T22:31:20.764 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:31:20.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.789+0000 7f38b5e87700 1 -- 172.21.15.16:0/3856394185 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3890005860 con 0x7f38b0124ea0 2023-09-30T22:31:20.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.790+0000 7f3897fff700 1 -- 172.21.15.16:0/3856394185 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f38a802a070 con 0x7f38b0124ea0 2023-09-30T22:31:20.791 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:31:20.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.792+0000 7f38b5e87700 1 -- 172.21.15.16:0/3856394185 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f38b0124ea0 msgr2=0x7f38b0121f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:20.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.792+0000 7f38b5e87700 1 --2- 172.21.15.16:0/3856394185 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f38b0124ea0 0x7f38b0121f60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f38a800efc0 tx=0x7f38a800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.792+0000 7f38b5e87700 1 -- 172.21.15.16:0/3856394185 shutdown_connections 2023-09-30T22:31:20.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.792+0000 7f38b5e87700 1 --2- 172.21.15.16:0/3856394185 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f38b0124ea0 0x7f38b0121f60 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.792+0000 7f38b5e87700 1 --2- 172.21.15.16:0/3856394185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38b006e570 0x7f38b01214e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.792+0000 7f38b5e87700 1 --2- 172.21.15.16:0/3856394185 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38b0134610 0x7f38b0121a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:20.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.792+0000 7f38b5e87700 1 -- 172.21.15.16:0/3856394185 >> 172.21.15.16:0/3856394185 conn(0x7f38b00a2aa0 msgr2=0x7f38b00a1020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:20.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.792+0000 7f38b5e87700 1 -- 172.21.15.16:0/3856394185 shutdown_connections 2023-09-30T22:31:20.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:20.792+0000 7f38b5e87700 1 -- 172.21.15.16:0/3856394185 wait complete. 2023-09-30T22:31:20.807 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:31:21.652 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:21.723 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:21.765 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:21.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.802+0000 7fd20e8e6700 1 Processor -- start 2023-09-30T22:31:21.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.802+0000 7fd20e8e6700 1 -- start start 2023-09-30T22:31:21.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.803+0000 7fd20e8e6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd20806a770 0x7fd20806ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:21.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.803+0000 7fd20e8e6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd20806b080 0x7fd208131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:21.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.803+0000 7fd20e8e6700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd208131b40 0x7fd208131fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:21.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.803+0000 7fd20e8e6700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd20809ece0 con 0x7fd208131b40 2023-09-30T22:31:21.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.803+0000 7fd20e8e6700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd20809e760 con 0x7fd20806a770 2023-09-30T22:31:21.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.803+0000 7fd20e8e6700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd20809e040 con 0x7fd20806b080 2023-09-30T22:31:21.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.803+0000 7fd2077fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd20806b080 0x7fd208131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:21.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.803+0000 7fd2077fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd20806b080 0x7fd208131600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:38566/0 (socket says 172.21.15.16:38566) 2023-09-30T22:31:21.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.803+0000 7fd2077fe700 1 -- 172.21.15.16:0/874151380 learned_addr learned my addr 172.21.15.16:0/874151380 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:21.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.803+0000 7fd207fff700 1 --2- 172.21.15.16:0/874151380 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd20806a770 0x7fd20806ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:21.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.803+0000 7fd20ce83700 1 --2- 172.21.15.16:0/874151380 >> v2:172.21.15.16:3300/0 conn(0x7fd208131b40 0x7fd208131fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:21.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.804+0000 7fd20ce83700 1 -- 172.21.15.16:0/874151380 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd20806b080 msgr2=0x7fd208131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:21.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.804+0000 7fd20ce83700 1 --2- 172.21.15.16:0/874151380 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd20806b080 0x7fd208131600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:21.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.804+0000 7fd20ce83700 1 -- 172.21.15.16:0/874151380 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd20806a770 msgr2=0x7fd20806ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:21.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.804+0000 7fd20ce83700 1 --2- 172.21.15.16:0/874151380 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd20806a770 0x7fd20806ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:21.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.804+0000 7fd20ce83700 1 -- 172.21.15.16:0/874151380 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd20809f630 con 0x7fd208131b40 2023-09-30T22:31:21.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.804+0000 7fd207fff700 1 --2- 172.21.15.16:0/874151380 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd20806a770 0x7fd20806ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:21.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.804+0000 7fd20ce83700 1 --2- 172.21.15.16:0/874151380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd208131b40 0x7fd208131fe0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd200009d90 tx=0x7fd20000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52cb1e9bdf62089e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:21.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.804+0000 7fd206ffd700 1 -- 172.21.15.16:0/874151380 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd20000e070 con 0x7fd208131b40 2023-09-30T22:31:21.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.804+0000 7fd206ffd700 1 -- 172.21.15.16:0/874151380 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd20000bb20 con 0x7fd208131b40 2023-09-30T22:31:21.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.804+0000 7fd206ffd700 1 -- 172.21.15.16:0/874151380 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd200012430 con 0x7fd208131b40 2023-09-30T22:31:21.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.805+0000 7fd20e8e6700 1 -- 172.21.15.16:0/874151380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd208131b40 msgr2=0x7fd208131fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:21.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.805+0000 7fd20e8e6700 1 --2- 172.21.15.16:0/874151380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd208131b40 0x7fd208131fe0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fd200009d90 tx=0x7fd20000b330 comp rx=0 tx=0).stop 2023-09-30T22:31:21.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.805+0000 7fd20e8e6700 1 -- 172.21.15.16:0/874151380 shutdown_connections 2023-09-30T22:31:21.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.805+0000 7fd20e8e6700 1 --2- 172.21.15.16:0/874151380 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd20806a770 0x7fd20806ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:21.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.805+0000 7fd20e8e6700 1 --2- 172.21.15.16:0/874151380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd208131b40 0x7fd208131fe0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:21.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.805+0000 7fd20e8e6700 1 --2- 172.21.15.16:0/874151380 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd20806b080 0x7fd208131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:21.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.805+0000 7fd20e8e6700 1 -- 172.21.15.16:0/874151380 >> 172.21.15.16:0/874151380 conn(0x7fd208066f50 msgr2=0x7fd20809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:21.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.805+0000 7fd20e8e6700 1 -- 172.21.15.16:0/874151380 shutdown_connections 2023-09-30T22:31:21.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.805+0000 7fd20e8e6700 1 -- 172.21.15.16:0/874151380 wait complete. 2023-09-30T22:31:21.809 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:21.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.806+0000 7fd20e8e6700 1 Processor -- start 2023-09-30T22:31:21.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.806+0000 7fd20e8e6700 1 -- start start 2023-09-30T22:31:21.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.807+0000 7fd20e8e6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd20806a770 0x7fd20809a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:21.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.807+0000 7fd20e8e6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd20806b080 0x7fd20809a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:21.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.807+0000 7fd20e8e6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd208131b40 0x7fd20809ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:21.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.807+0000 7fd20e8e6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2081245c0 con 0x7fd20806a770 2023-09-30T22:31:21.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.807+0000 7fd20e8e6700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd208125690 con 0x7fd208131b40 2023-09-30T22:31:21.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.807+0000 7fd20e8e6700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd2080a3c60 con 0x7fd20806b080 2023-09-30T22:31:21.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.807+0000 7fd2077fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd20806b080 0x7fd20809a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:21.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.807+0000 7fd2077fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd20806b080 0x7fd20809a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:38572/0 (socket says 172.21.15.16:38572) 2023-09-30T22:31:21.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.807+0000 7fd2077fe700 1 -- 172.21.15.16:0/1333420454 learned_addr learned my addr 172.21.15.16:0/1333420454 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:21.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.807+0000 7fd207fff700 1 --2- 172.21.15.16:0/1333420454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd20806a770 0x7fd20809a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:21.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.807+0000 7fd20ce83700 1 --2- 172.21.15.16:0/1333420454 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd208131b40 0x7fd20809ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:21.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.807+0000 7fd2077fe700 1 -- 172.21.15.16:0/1333420454 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd208131b40 msgr2=0x7fd20809ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:21.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.807+0000 7fd2077fe700 1 --2- 172.21.15.16:0/1333420454 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd208131b40 0x7fd20809ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:21.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.807+0000 7fd2077fe700 1 -- 172.21.15.16:0/1333420454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd20806a770 msgr2=0x7fd20809a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:21.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.807+0000 7fd2077fe700 1 --2- 172.21.15.16:0/1333420454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd20806a770 0x7fd20809a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:21.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.807+0000 7fd2077fe700 1 -- 172.21.15.16:0/1333420454 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f80094a0 con 0x7fd20806b080 2023-09-30T22:31:21.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.807+0000 7fd20ce83700 1 --2- 172.21.15.16:0/1333420454 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd208131b40 0x7fd20809ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:31:21.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.807+0000 7fd207fff700 1 --2- 172.21.15.16:0/1333420454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd20806a770 0x7fd20809a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:21.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.807+0000 7fd2077fe700 1 --2- 172.21.15.16:0/1333420454 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd20806b080 0x7fd20809a940 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc00e8e0 tx=0x7fd1fc00ebf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:21.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.808+0000 7fd204ff9700 1 -- 172.21.15.16:0/1333420454 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc013680 con 0x7fd20806b080 2023-09-30T22:31:21.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.808+0000 7fd204ff9700 1 -- 172.21.15.16:0/1333420454 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1fc0137e0 con 0x7fd20806b080 2023-09-30T22:31:21.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.808+0000 7fd20e8e6700 1 -- 172.21.15.16:0/1333420454 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd200009a70 con 0x7fd20806b080 2023-09-30T22:31:21.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.808+0000 7fd204ff9700 1 -- 172.21.15.16:0/1333420454 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc013b50 con 0x7fd20806b080 2023-09-30T22:31:21.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.808+0000 7fd20e8e6700 1 -- 172.21.15.16:0/1333420454 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd208125950 con 0x7fd20806b080 2023-09-30T22:31:21.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.809+0000 7fd20e8e6700 1 -- 172.21.15.16:0/1333420454 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd208074ea0 con 0x7fd20806b080 2023-09-30T22:31:21.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.809+0000 7fd204ff9700 1 -- 172.21.15.16:0/1333420454 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd1fc013cb0 con 0x7fd20806b080 2023-09-30T22:31:21.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.809+0000 7fd204ff9700 1 -- 172.21.15.16:0/1333420454 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd1fc03fa60 con 0x7fd20806b080 2023-09-30T22:31:21.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.815+0000 7fd204ff9700 1 -- 172.21.15.16:0/1333420454 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd1fc01ada0 con 0x7fd20806b080 2023-09-30T22:31:21.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.881+0000 7fe428b13700 1 Processor -- start 2023-09-30T22:31:21.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.881+0000 7fe428b13700 1 -- start start 2023-09-30T22:31:21.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.882+0000 7fe428b13700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe42409d6f0 0x7fe42409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:21.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.883+0000 7fe428b13700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe42409e000 0x7fe4241339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:21.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.883+0000 7fe428b13700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe42409e410 0x7fe424133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:21.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.883+0000 7fe428b13700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe42406b460 con 0x7fe42409e000 2023-09-30T22:31:21.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.883+0000 7fe428b13700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe42409b120 con 0x7fe42409d6f0 2023-09-30T22:31:21.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.883+0000 7fe428b13700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe42409a7e0 con 0x7fe42409e410 2023-09-30T22:31:21.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.883+0000 7fe421d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe42409e000 0x7fe4241339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:21.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.883+0000 7fe422d9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe42409e410 0x7fe424133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:21.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.883+0000 7fe421d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe42409e000 0x7fe4241339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57176/0 (socket says 172.21.15.16:57176) 2023-09-30T22:31:21.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.883+0000 7fe421d9b700 1 -- 172.21.15.16:0/1556141994 learned_addr learned my addr 172.21.15.16:0/1556141994 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:21.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.883+0000 7fe42259c700 1 --2- 172.21.15.16:0/1556141994 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe42409d6f0 0x7fe42409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:21.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.883+0000 7fe421d9b700 1 -- 172.21.15.16:0/1556141994 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe42409e410 msgr2=0x7fe424133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:21.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.883+0000 7fe421d9b700 1 --2- 172.21.15.16:0/1556141994 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe42409e410 0x7fe424133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:21.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.883+0000 7fe421d9b700 1 -- 172.21.15.16:0/1556141994 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe42409d6f0 msgr2=0x7fe42409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:21.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.883+0000 7fe421d9b700 1 --2- 172.21.15.16:0/1556141994 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe42409d6f0 0x7fe42409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:21.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.883+0000 7fe421d9b700 1 -- 172.21.15.16:0/1556141994 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe42409a500 con 0x7fe42409e000 2023-09-30T22:31:21.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.883+0000 7fe422d9d700 1 --2- 172.21.15.16:0/1556141994 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe42409e410 0x7fe424133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:21.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.884+0000 7fe421d9b700 1 --2- 172.21.15.16:0/1556141994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe42409e000 0x7fe4241339f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe4180098f0 tx=0x7fe418009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb4cb6b465af0195 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:21.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.884+0000 7fe42159a700 1 -- 172.21.15.16:0/1556141994 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe418011830 con 0x7fe42409e000 2023-09-30T22:31:21.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.884+0000 7fe42159a700 1 -- 172.21.15.16:0/1556141994 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe418011990 con 0x7fe42409e000 2023-09-30T22:31:21.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.884+0000 7fe42159a700 1 -- 172.21.15.16:0/1556141994 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe418011d20 con 0x7fe42409e000 2023-09-30T22:31:21.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.885+0000 7fe428b13700 1 -- 172.21.15.16:0/1556141994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe42409e000 msgr2=0x7fe4241339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:21.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.885+0000 7fe428b13700 1 --2- 172.21.15.16:0/1556141994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe42409e000 0x7fe4241339f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe4180098f0 tx=0x7fe418009c00 comp rx=0 tx=0).stop 2023-09-30T22:31:21.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.885+0000 7fe428b13700 1 -- 172.21.15.16:0/1556141994 shutdown_connections 2023-09-30T22:31:21.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.885+0000 7fe428b13700 1 --2- 172.21.15.16:0/1556141994 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe42409d6f0 0x7fe42409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:21.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.885+0000 7fe428b13700 1 --2- 172.21.15.16:0/1556141994 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe42409e410 0x7fe424133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:21.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.885+0000 7fe428b13700 1 --2- 172.21.15.16:0/1556141994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe42409e000 0x7fe4241339f0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:21.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.885+0000 7fe428b13700 1 -- 172.21.15.16:0/1556141994 >> 172.21.15.16:0/1556141994 conn(0x7fe424066f50 msgr2=0x7fe42409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:21.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.885+0000 7fe428b13700 1 -- 172.21.15.16:0/1556141994 shutdown_connections 2023-09-30T22:31:21.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.885+0000 7fe428b13700 1 -- 172.21.15.16:0/1556141994 wait complete. 2023-09-30T22:31:21.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.887+0000 7fe428b13700 1 Processor -- start 2023-09-30T22:31:21.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.887+0000 7fe428b13700 1 -- start start 2023-09-30T22:31:21.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.888+0000 7fe428b13700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe42409d6f0 0x7fe424122640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:21.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.888+0000 7fe428b13700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe42409e000 0x7fe424122b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:21.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.888+0000 7fe428b13700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe42409e410 0x7fe4241230c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:21.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.888+0000 7fe428b13700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe42409c710 con 0x7fe42409e410 2023-09-30T22:31:21.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.888+0000 7fe428b13700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe424124a20 con 0x7fe42409e000 2023-09-30T22:31:21.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.888+0000 7fe428b13700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe4241242c0 con 0x7fe42409d6f0 2023-09-30T22:31:21.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.888+0000 7fe421d9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe42409e000 0x7fe424122b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:21.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.888+0000 7fe422d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe42409e410 0x7fe4241230c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:21.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.888+0000 7fe421d9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe42409e000 0x7fe424122b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:57434/0 (socket says 172.21.15.16:57434) 2023-09-30T22:31:21.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.888+0000 7fe421d9b700 1 -- 172.21.15.16:0/1005783008 learned_addr learned my addr 172.21.15.16:0/1005783008 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:21.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.888+0000 7fe422d9d700 1 --2- 172.21.15.16:0/1005783008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe42409e410 0x7fe4241230c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57190/0 (socket says 172.21.15.16:57190) 2023-09-30T22:31:21.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.888+0000 7fe42259c700 1 --2- 172.21.15.16:0/1005783008 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe42409d6f0 0x7fe424122640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:21.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.888+0000 7fe421d9b700 1 -- 172.21.15.16:0/1005783008 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe42409d6f0 msgr2=0x7fe424122640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:21.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.888+0000 7fe421d9b700 1 --2- 172.21.15.16:0/1005783008 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe42409d6f0 0x7fe424122640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:21.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.888+0000 7fe421d9b700 1 -- 172.21.15.16:0/1005783008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe42409e410 msgr2=0x7fe4241230c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:21.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.888+0000 7fe421d9b700 1 --2- 172.21.15.16:0/1005783008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe42409e410 0x7fe4241230c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:21.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.888+0000 7fe421d9b700 1 -- 172.21.15.16:0/1005783008 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe41c009a70 con 0x7fe42409e000 2023-09-30T22:31:21.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.888+0000 7fe422d9d700 1 --2- 172.21.15.16:0/1005783008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe42409e410 0x7fe4241230c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:21.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.889+0000 7fe421d9b700 1 --2- 172.21.15.16:0/1005783008 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe42409e000 0x7fe424122b80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe418009f50 tx=0x7fe418004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:21.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.889+0000 7fe412ffd700 1 -- 172.21.15.16:0/1005783008 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe418004240 con 0x7fe42409e000 2023-09-30T22:31:21.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.889+0000 7fe412ffd700 1 -- 172.21.15.16:0/1005783008 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe41801f070 con 0x7fe42409e000 2023-09-30T22:31:21.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.889+0000 7fe412ffd700 1 -- 172.21.15.16:0/1005783008 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4180045a0 con 0x7fe42409e000 2023-09-30T22:31:21.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.889+0000 7fe428b13700 1 -- 172.21.15.16:0/1005783008 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe418009550 con 0x7fe42409e000 2023-09-30T22:31:21.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.889+0000 7fe428b13700 1 -- 172.21.15.16:0/1005783008 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4241330c0 con 0x7fe42409e000 2023-09-30T22:31:21.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.890+0000 7fe428b13700 1 -- 172.21.15.16:0/1005783008 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe424074ea0 con 0x7fe42409e000 2023-09-30T22:31:21.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.890+0000 7fe412ffd700 1 -- 172.21.15.16:0/1005783008 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe418021030 con 0x7fe42409e000 2023-09-30T22:31:21.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.890+0000 7fe412ffd700 1 -- 172.21.15.16:0/1005783008 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fe4180406c0 con 0x7fe42409e000 2023-09-30T22:31:21.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.896+0000 7fe412ffd700 1 -- 172.21.15.16:0/1005783008 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe4180036a0 con 0x7fe42409e000 2023-09-30T22:31:21.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.915+0000 7f668d3ac700 1 Processor -- start 2023-09-30T22:31:21.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.915+0000 7f668d3ac700 1 -- start start 2023-09-30T22:31:21.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.916+0000 7f668d3ac700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f668809d6f0 0x7f668809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:21.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.916+0000 7f668d3ac700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f668809e000 0x7f66881339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:21.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.916+0000 7f668d3ac700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f668809e410 0x7f6688133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:21.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.916+0000 7f668d3ac700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f668806b460 con 0x7f668809e000 2023-09-30T22:31:21.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.916+0000 7f668d3ac700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f668809b120 con 0x7f668809e410 2023-09-30T22:31:21.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.916+0000 7f668d3ac700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f668809a7e0 con 0x7f668809d6f0 2023-09-30T22:31:21.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.916+0000 7f6686ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f668809d6f0 0x7f668809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:21.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.916+0000 7f6686ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f668809d6f0 0x7f668809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:38610/0 (socket says 172.21.15.16:38610) 2023-09-30T22:31:21.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.916+0000 7f6686ffd700 1 -- 172.21.15.16:0/361568315 learned_addr learned my addr 172.21.15.16:0/361568315 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:21.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.916+0000 7f66867fc700 1 --2- 172.21.15.16:0/361568315 >> v2:172.21.15.16:3300/0 conn(0x7f668809e000 0x7f66881339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:21.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.916+0000 7f66877fe700 1 --2- 172.21.15.16:0/361568315 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f668809e410 0x7f6688133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:21.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.917+0000 7f6686ffd700 1 -- 172.21.15.16:0/361568315 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f668809e410 msgr2=0x7f6688133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:21.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.917+0000 7f6686ffd700 1 --2- 172.21.15.16:0/361568315 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f668809e410 0x7f6688133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:21.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.917+0000 7f6686ffd700 1 -- 172.21.15.16:0/361568315 >> v2:172.21.15.16:3300/0 conn(0x7f668809e000 msgr2=0x7f66881339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:21.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.917+0000 7f6686ffd700 1 --2- 172.21.15.16:0/361568315 >> v2:172.21.15.16:3300/0 conn(0x7f668809e000 0x7f66881339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:21.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.917+0000 7f6686ffd700 1 -- 172.21.15.16:0/361568315 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f668809a500 con 0x7f668809d6f0 2023-09-30T22:31:21.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.917+0000 7f66877fe700 1 --2- 172.21.15.16:0/361568315 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f668809e410 0x7f6688133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:31:21.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.917+0000 7f6686ffd700 1 --2- 172.21.15.16:0/361568315 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f668809d6f0 0x7f668809dac0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6678009920 tx=0x7f6678009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c69d5479ebd46ad server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:21.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.917+0000 7f6685ffb700 1 -- 172.21.15.16:0/361568315 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6678011830 con 0x7f668809d6f0 2023-09-30T22:31:21.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.917+0000 7f6685ffb700 1 -- 172.21.15.16:0/361568315 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6678011990 con 0x7f668809d6f0 2023-09-30T22:31:21.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.917+0000 7f6685ffb700 1 -- 172.21.15.16:0/361568315 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6678011d20 con 0x7f668809d6f0 2023-09-30T22:31:21.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.918+0000 7f668d3ac700 1 -- 172.21.15.16:0/361568315 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f668809d6f0 msgr2=0x7f668809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:21.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.918+0000 7f668d3ac700 1 --2- 172.21.15.16:0/361568315 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f668809d6f0 0x7f668809dac0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6678009920 tx=0x7f6678009c30 comp rx=0 tx=0).stop 2023-09-30T22:31:21.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.918+0000 7f668d3ac700 1 -- 172.21.15.16:0/361568315 shutdown_connections 2023-09-30T22:31:21.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.918+0000 7f668d3ac700 1 --2- 172.21.15.16:0/361568315 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f668809e410 0x7f6688133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:21.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.918+0000 7f668d3ac700 1 --2- 172.21.15.16:0/361568315 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f668809d6f0 0x7f668809dac0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:21.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.918+0000 7f668d3ac700 1 --2- 172.21.15.16:0/361568315 >> v2:172.21.15.16:3300/0 conn(0x7f668809e000 0x7f66881339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:21.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.918+0000 7f668d3ac700 1 -- 172.21.15.16:0/361568315 >> 172.21.15.16:0/361568315 conn(0x7f6688066f50 msgr2=0x7f668809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:21.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.918+0000 7f668d3ac700 1 -- 172.21.15.16:0/361568315 shutdown_connections 2023-09-30T22:31:21.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.918+0000 7f668d3ac700 1 -- 172.21.15.16:0/361568315 wait complete. 2023-09-30T22:31:21.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.919+0000 7f668d3ac700 1 Processor -- start 2023-09-30T22:31:21.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.919+0000 7f668d3ac700 1 -- start start 2023-09-30T22:31:21.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.920+0000 7f668d3ac700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f668809d6f0 0x7f66881a5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:21.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.920+0000 7f668d3ac700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f668809e000 0x7f66881a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:21.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.920+0000 7f668d3ac700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f668809e410 0x7f66881a6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:21.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.920+0000 7f668d3ac700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66881a95b0 con 0x7f668809e000 2023-09-30T22:31:21.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.920+0000 7f668d3ac700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66880a2240 con 0x7f668809d6f0 2023-09-30T22:31:21.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.920+0000 7f668d3ac700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f668809c710 con 0x7f668809e410 2023-09-30T22:31:21.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.920+0000 7f66877fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f668809e410 0x7f66881a6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:21.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.920+0000 7f66877fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f668809e410 0x7f66881a6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:38626/0 (socket says 172.21.15.16:38626) 2023-09-30T22:31:21.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.920+0000 7f66877fe700 1 -- 172.21.15.16:0/2408874429 learned_addr learned my addr 172.21.15.16:0/2408874429 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:21.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.920+0000 7f66867fc700 1 --2- 172.21.15.16:0/2408874429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f668809e000 0x7f66881a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:21.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.920+0000 7f6686ffd700 1 --2- 172.21.15.16:0/2408874429 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f668809d6f0 0x7f66881a5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:21.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.920+0000 7f66877fe700 1 -- 172.21.15.16:0/2408874429 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f668809d6f0 msgr2=0x7f66881a5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:21.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.920+0000 7f66877fe700 1 --2- 172.21.15.16:0/2408874429 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f668809d6f0 0x7f66881a5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:21.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.920+0000 7f66877fe700 1 -- 172.21.15.16:0/2408874429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f668809e000 msgr2=0x7f66881a6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:21.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.920+0000 7f66877fe700 1 --2- 172.21.15.16:0/2408874429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f668809e000 0x7f66881a6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:21.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.921+0000 7f66877fe700 1 -- 172.21.15.16:0/2408874429 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6678009550 con 0x7f668809e410 2023-09-30T22:31:21.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.921+0000 7f66877fe700 1 --2- 172.21.15.16:0/2408874429 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f668809e410 0x7f66881a6960 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6680009f50 tx=0x7f668000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:21.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.921+0000 7f666f7fe700 1 -- 172.21.15.16:0/2408874429 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6680016e80 con 0x7f668809e410 2023-09-30T22:31:21.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.921+0000 7f666f7fe700 1 -- 172.21.15.16:0/2408874429 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f668001a070 con 0x7f668809e410 2023-09-30T22:31:21.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.921+0000 7f668d3ac700 1 -- 172.21.15.16:0/2408874429 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66881a7a90 con 0x7f668809e410 2023-09-30T22:31:21.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.921+0000 7f666f7fe700 1 -- 172.21.15.16:0/2408874429 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6680003c60 con 0x7f668809e410 2023-09-30T22:31:21.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.921+0000 7f668d3ac700 1 -- 172.21.15.16:0/2408874429 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66881b9160 con 0x7f668809e410 2023-09-30T22:31:21.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.922+0000 7f668d3ac700 1 -- 172.21.15.16:0/2408874429 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66881a9120 con 0x7f668809e410 2023-09-30T22:31:21.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.923+0000 7f666f7fe700 1 -- 172.21.15.16:0/2408874429 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f668000d940 con 0x7f668809e410 2023-09-30T22:31:21.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.924+0000 7f666f7fe700 1 -- 172.21.15.16:0/2408874429 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f668001c070 con 0x7f668809e410 2023-09-30T22:31:21.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.928+0000 7f666f7fe700 1 -- 172.21.15.16:0/2408874429 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f668001d8a0 con 0x7f668809e410 2023-09-30T22:31:21.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.962+0000 7fc31c998700 1 Processor -- start 2023-09-30T22:31:21.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.962+0000 7fc31c998700 1 -- start start 2023-09-30T22:31:21.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.963+0000 7fc31c998700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc31806e570 0x7fc31806e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:21.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.963+0000 7fc31c998700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc31809eca0 0x7fc31806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:21.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.963+0000 7fc31c998700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc318070ee0 con 0x7fc318134610 2023-09-30T22:31:21.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.963+0000 7fc31c998700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc318071480 con 0x7fc31809eca0 2023-09-30T22:31:21.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.963+0000 7fc31c998700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc31806f3c0 con 0x7fc31806e570 2023-09-30T22:31:21.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.963+0000 7fc315d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc31809eca0 0x7fc31806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:21.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.963+0000 7fc316d9d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc318134610 0x7fc318134a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58054/0 (socket says 172.21.15.16:58054) 2023-09-30T22:31:21.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.963+0000 7fc316d9d700 1 -- 172.21.15.16:0/1271866382 learned_addr learned my addr 172.21.15.16:0/1271866382 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:31:21.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.963+0000 7fc31659c700 1 --2- 172.21.15.16:0/1271866382 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc31806e570 0x7fc31806e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:21.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.963+0000 7fc315d9b700 1 -- 172.21.15.16:0/1271866382 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc31806e570 msgr2=0x7fc31806e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:21.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.963+0000 7fc315d9b700 1 --2- 172.21.15.16:0/1271866382 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc31806e570 0x7fc31806e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:21.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.963+0000 7fc315d9b700 1 -- 172.21.15.16:0/1271866382 >> v1:172.21.15.16:6789/0 conn(0x7fc318134610 legacy=0x7fc318134a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:21.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.963+0000 7fc315d9b700 1 -- 172.21.15.16:0/1271866382 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc318071b90 con 0x7fc31809eca0 2023-09-30T22:31:21.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.964+0000 7fc315d9b700 1 --2- 172.21.15.16:0/1271866382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc31809eca0 0x7fc31806ee80 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fc30c009840 tx=0x7fc30c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=852fac2d64e5202a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:21.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.964+0000 7fc31559a700 1 -- 172.21.15.16:0/1271866382 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30c011830 con 0x7fc31809eca0 2023-09-30T22:31:21.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.964+0000 7fc31559a700 1 -- 172.21.15.16:0/1271866382 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc30c011990 con 0x7fc31809eca0 2023-09-30T22:31:21.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.964+0000 7fc31559a700 1 -- 172.21.15.16:0/1271866382 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30c011d20 con 0x7fc31809eca0 2023-09-30T22:31:21.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.964+0000 7fc31c998700 1 -- 172.21.15.16:0/1271866382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc31809eca0 msgr2=0x7fc31806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:21.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.964+0000 7fc31c998700 1 --2- 172.21.15.16:0/1271866382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc31809eca0 0x7fc31806ee80 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fc30c009840 tx=0x7fc30c009b50 comp rx=0 tx=0).stop 2023-09-30T22:31:21.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.965+0000 7fc31c998700 1 -- 172.21.15.16:0/1271866382 shutdown_connections 2023-09-30T22:31:21.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.965+0000 7fc31c998700 1 --2- 172.21.15.16:0/1271866382 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc31806e570 0x7fc31806e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:21.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.965+0000 7fc31c998700 1 --2- 172.21.15.16:0/1271866382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc31809eca0 0x7fc31806ee80 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:21.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.965+0000 7fc31c998700 1 -- 172.21.15.16:0/1271866382 >> 172.21.15.16:0/1271866382 conn(0x7fc3180a2aa0 msgr2=0x7fc3180a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:21.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.965+0000 7fc31c998700 1 -- 172.21.15.16:0/1271866382 shutdown_connections 2023-09-30T22:31:21.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.965+0000 7fc31c998700 1 -- 172.21.15.16:0/1271866382 wait complete. 2023-09-30T22:31:21.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.966+0000 7fc31c998700 1 Processor -- start 2023-09-30T22:31:21.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.966+0000 7fc31c998700 1 -- start start 2023-09-30T22:31:21.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.966+0000 7fc31c998700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc31806e570 0x7fc31813ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:21.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.966+0000 7fc31c998700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc318134610 0x7fc31813f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:21.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.966+0000 7fc31c998700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc31813f6e0 0x7fc318132e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:21.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.966+0000 7fc31c998700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc31813ba30 con 0x7fc31806e570 2023-09-30T22:31:21.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.966+0000 7fc31c998700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc31813d430 con 0x7fc31813f6e0 2023-09-30T22:31:21.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.966+0000 7fc31c998700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc3180a2150 con 0x7fc318134610 2023-09-30T22:31:21.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.967+0000 7fc315d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc318134610 0x7fc31813f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:21.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.967+0000 7fc315d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc318134610 0x7fc31813f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:38638/0 (socket says 172.21.15.16:38638) 2023-09-30T22:31:21.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.967+0000 7fc315d9b700 1 -- 172.21.15.16:0/1774915160 learned_addr learned my addr 172.21.15.16:0/1774915160 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:21.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.967+0000 7fc316d9d700 1 --2- 172.21.15.16:0/1774915160 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc31813f6e0 0x7fc318132e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:21.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.967+0000 7fc31659c700 1 --2- 172.21.15.16:0/1774915160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc31806e570 0x7fc31813ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:21.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.967+0000 7fc316d9d700 1 -- 172.21.15.16:0/1774915160 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc318134610 msgr2=0x7fc31813f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:21.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.967+0000 7fc316d9d700 1 --2- 172.21.15.16:0/1774915160 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc318134610 0x7fc31813f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:21.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.967+0000 7fc316d9d700 1 -- 172.21.15.16:0/1774915160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc31806e570 msgr2=0x7fc31813ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:21.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.967+0000 7fc316d9d700 1 --2- 172.21.15.16:0/1774915160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc31806e570 0x7fc31813ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:21.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.967+0000 7fc316d9d700 1 -- 172.21.15.16:0/1774915160 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc30c0094a0 con 0x7fc31813f6e0 2023-09-30T22:31:21.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.967+0000 7fc316d9d700 1 --2- 172.21.15.16:0/1774915160 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc31813f6e0 0x7fc318132e70 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc31000caa0 tx=0x7fc31000cdb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:21.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.967+0000 7fc306ffd700 1 -- 172.21.15.16:0/1774915160 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc31000a940 con 0x7fc31813f6e0 2023-09-30T22:31:21.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.968+0000 7fc306ffd700 1 -- 172.21.15.16:0/1774915160 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc31000aaa0 con 0x7fc31813f6e0 2023-09-30T22:31:21.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.968+0000 7fc31c998700 1 -- 172.21.15.16:0/1774915160 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc31813dae0 con 0x7fc31813f6e0 2023-09-30T22:31:21.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.968+0000 7fc306ffd700 1 -- 172.21.15.16:0/1774915160 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc31000ae10 con 0x7fc31813f6e0 2023-09-30T22:31:21.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.968+0000 7fc31c998700 1 -- 172.21.15.16:0/1774915160 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc31813dda0 con 0x7fc31813f6e0 2023-09-30T22:31:21.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.969+0000 7fc31c998700 1 -- 172.21.15.16:0/1774915160 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc318074ea0 con 0x7fc31813f6e0 2023-09-30T22:31:21.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.969+0000 7fc306ffd700 1 -- 172.21.15.16:0/1774915160 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc31001b3b0 con 0x7fc31813f6e0 2023-09-30T22:31:21.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.969+0000 7fc306ffd700 1 -- 172.21.15.16:0/1774915160 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fc31003f780 con 0x7fc31813f6e0 2023-09-30T22:31:21.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:21.978+0000 7fc306ffd700 1 -- 172.21.15.16:0/1774915160 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc3100192c0 con 0x7fc31813f6e0 2023-09-30T22:31:22.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:22.037+0000 7fd20e8e6700 1 -- 172.21.15.16:0/1333420454 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd208126420 con 0x7fd20806b080 2023-09-30T22:31:22.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:22.038+0000 7fd204ff9700 1 -- 172.21.15.16:0/1333420454 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd1fc021070 con 0x7fd20806b080 2023-09-30T22:31:22.039 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:31:22.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:22.041+0000 7fd20e8e6700 1 -- 172.21.15.16:0/1333420454 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd20806b080 msgr2=0x7fd20809a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:22.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:22.041+0000 7fd20e8e6700 1 --2- 172.21.15.16:0/1333420454 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd20806b080 0x7fd20809a940 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc00e8e0 tx=0x7fd1fc00ebf0 comp rx=0 tx=0).stop 2023-09-30T22:31:22.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:22.041+0000 7fd20e8e6700 1 -- 172.21.15.16:0/1333420454 shutdown_connections 2023-09-30T22:31:22.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:22.041+0000 7fd20e8e6700 1 --2- 172.21.15.16:0/1333420454 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd208131b40 0x7fd20809ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:22.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:22.041+0000 7fd20e8e6700 1 --2- 172.21.15.16:0/1333420454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd20806a770 0x7fd20809a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:22.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:22.041+0000 7fd20e8e6700 1 --2- 172.21.15.16:0/1333420454 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd20806b080 0x7fd20809a940 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:22.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:22.042+0000 7fd20e8e6700 1 -- 172.21.15.16:0/1333420454 >> 172.21.15.16:0/1333420454 conn(0x7fd208066f50 msgr2=0x7fd20806fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:22.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:22.043+0000 7fd20e8e6700 1 -- 172.21.15.16:0/1333420454 shutdown_connections 2023-09-30T22:31:22.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:22.043+0000 7fd20e8e6700 1 -- 172.21.15.16:0/1333420454 wait complete. 2023-09-30T22:31:22.058 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:31:22.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:22.111+0000 7fe428b13700 1 -- 172.21.15.16:0/1005783008 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe42406c360 con 0x7fe42409e000 2023-09-30T22:31:22.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:22.111+0000 7fe412ffd700 1 -- 172.21.15.16:0/1005783008 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe418015020 con 0x7fe42409e000 2023-09-30T22:31:22.113 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:31:22.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:22.115+0000 7fe428b13700 1 -- 172.21.15.16:0/1005783008 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe42409e000 msgr2=0x7fe424122b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:22.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:22.115+0000 7fe428b13700 1 --2- 172.21.15.16:0/1005783008 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe42409e000 0x7fe424122b80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe418009f50 tx=0x7fe418004750 comp rx=0 tx=0).stop 2023-09-30T22:31:22.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:22.115+0000 7fe428b13700 1 -- 172.21.15.16:0/1005783008 shutdown_connections 2023-09-30T22:31:22.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:22.115+0000 7fe428b13700 1 --2- 172.21.15.16:0/1005783008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe42409e410 0x7fe4241230c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:22.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:22.115+0000 7fe428b13700 1 --2- 172.21.15.16:0/1005783008 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe42409d6f0 0x7fe424122640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:22.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:22.115+0000 7fe428b13700 1 --2- 172.21.15.16:0/1005783008 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe42409e000 0x7fe424122b80 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:22.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:22.115+0000 7fe428b13700 1 -- 172.21.15.16:0/1005783008 >> 172.21.15.16:0/1005783008 conn(0x7fe424066f50 msgr2=0x7fe424070fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:22.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:22.115+0000 7fe428b13700 1 -- 172.21.15.16:0/1005783008 shutdown_connections 2023-09-30T22:31:22.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:22.115+0000 7fe428b13700 1 -- 172.21.15.16:0/1005783008 wait complete. 2023-09-30T22:31:22.135 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:31:22.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:22.148+0000 7f668d3ac700 1 -- 172.21.15.16:0/2408874429 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f66881a96f0 con 0x7f668809e410 2023-09-30T22:31:22.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:22.149+0000 7f666f7fe700 1 -- 172.21.15.16:0/2408874429 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6680025020 con 0x7f668809e410 2023-09-30T22:31:22.150 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:31:22.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:22.151+0000 7f668d3ac700 1 -- 172.21.15.16:0/2408874429 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f668809e410 msgr2=0x7f66881a6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:22.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:22.151+0000 7f668d3ac700 1 --2- 172.21.15.16:0/2408874429 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f668809e410 0x7f66881a6960 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f6680009f50 tx=0x7f668000d3b0 comp rx=0 tx=0).stop 2023-09-30T22:31:22.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:22.151+0000 7f668d3ac700 1 -- 172.21.15.16:0/2408874429 shutdown_connections 2023-09-30T22:31:22.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:22.151+0000 7f668d3ac700 1 --2- 172.21.15.16:0/2408874429 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f668809d6f0 0x7f66881a5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:22.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:22.151+0000 7f668d3ac700 1 --2- 172.21.15.16:0/2408874429 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f668809e410 0x7f66881a6960 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:22.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:22.151+0000 7f668d3ac700 1 --2- 172.21.15.16:0/2408874429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f668809e000 0x7f66881a6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:22.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:22.151+0000 7f668d3ac700 1 -- 172.21.15.16:0/2408874429 >> 172.21.15.16:0/2408874429 conn(0x7f6688066f50 msgr2=0x7f66881bae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:22.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:22.151+0000 7f668d3ac700 1 -- 172.21.15.16:0/2408874429 shutdown_connections 2023-09-30T22:31:22.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:22.151+0000 7f668d3ac700 1 -- 172.21.15.16:0/2408874429 wait complete. 2023-09-30T22:31:22.166 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:31:22.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:22.184+0000 7fc31c998700 1 -- 172.21.15.16:0/1774915160 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc318071100 con 0x7fc31813f6e0 2023-09-30T22:31:22.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:22.184+0000 7fc306ffd700 1 -- 172.21.15.16:0/1774915160 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc310017070 con 0x7fc31813f6e0 2023-09-30T22:31:22.186 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:31:22.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:22.186+0000 7fc31c998700 1 -- 172.21.15.16:0/1774915160 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc31813f6e0 msgr2=0x7fc318132e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:22.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:22.186+0000 7fc31c998700 1 --2- 172.21.15.16:0/1774915160 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc31813f6e0 0x7fc318132e70 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc31000caa0 tx=0x7fc31000cdb0 comp rx=0 tx=0).stop 2023-09-30T22:31:22.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:22.186+0000 7fc31c998700 1 -- 172.21.15.16:0/1774915160 shutdown_connections 2023-09-30T22:31:22.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:22.186+0000 7fc31c998700 1 --2- 172.21.15.16:0/1774915160 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc31813f6e0 0x7fc318132e70 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:22.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:22.186+0000 7fc31c998700 1 --2- 172.21.15.16:0/1774915160 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc31806e570 0x7fc31813ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:22.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:22.186+0000 7fc31c998700 1 --2- 172.21.15.16:0/1774915160 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc318134610 0x7fc31813f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:22.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:22.187+0000 7fc31c998700 1 -- 172.21.15.16:0/1774915160 >> 172.21.15.16:0/1774915160 conn(0x7fc3180a2aa0 msgr2=0x7fc3180a0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:22.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:22.187+0000 7fc31c998700 1 -- 172.21.15.16:0/1774915160 shutdown_connections 2023-09-30T22:31:22.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:22.187+0000 7fc31c998700 1 -- 172.21.15.16:0/1774915160 wait complete. 2023-09-30T22:31:22.201 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:31:23.059 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:23.136 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:23.167 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:23.203 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:23.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.206+0000 7f248cad1700 1 Processor -- start 2023-09-30T22:31:23.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.206+0000 7f248cad1700 1 -- start start 2023-09-30T22:31:23.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.207+0000 7f248cad1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f248812cf60 0x7f248806f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:23.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.207+0000 7f248cad1700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f248812f8b0 0x7f248812fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:23.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.207+0000 7f248cad1700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f248812cd90 con 0x7f2488130260 2023-09-30T22:31:23.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.207+0000 7f248cad1700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f248812d330 con 0x7f248812f8b0 2023-09-30T22:31:23.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.207+0000 7f248cad1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f24881306e0 con 0x7f248812cf60 2023-09-30T22:31:23.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.207+0000 7f2486d9d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2488130260 0x7f2488136c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58066/0 (socket says 172.21.15.16:58066) 2023-09-30T22:31:23.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.207+0000 7f2486d9d700 1 -- 172.21.15.16:0/3428294282 learned_addr learned my addr 172.21.15.16:0/3428294282 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:31:23.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.207+0000 7f248659c700 1 --2- 172.21.15.16:0/3428294282 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f248812cf60 0x7f248806f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:23.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.207+0000 7f2485d9b700 1 --2- 172.21.15.16:0/3428294282 >> v2:172.21.15.16:3300/0 conn(0x7f248812f8b0 0x7f248812fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:23.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.208+0000 7f248659c700 1 -- 172.21.15.16:0/3428294282 >> v2:172.21.15.16:3300/0 conn(0x7f248812f8b0 msgr2=0x7f248812fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:23.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.208+0000 7f248659c700 1 --2- 172.21.15.16:0/3428294282 >> v2:172.21.15.16:3300/0 conn(0x7f248812f8b0 0x7f248812fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.208+0000 7f248659c700 1 -- 172.21.15.16:0/3428294282 >> v1:172.21.15.16:6789/0 conn(0x7f2488130260 legacy=0x7f2488136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:23.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.208+0000 7f248659c700 1 -- 172.21.15.16:0/3428294282 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2488071780 con 0x7f248812cf60 2023-09-30T22:31:23.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.208+0000 7f2485d9b700 1 --2- 172.21.15.16:0/3428294282 >> v2:172.21.15.16:3300/0 conn(0x7f248812f8b0 0x7f248812fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:23.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.208+0000 7f248659c700 1 --2- 172.21.15.16:0/3428294282 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f248812cf60 0x7f248806f5d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2478009840 tx=0x7f2478009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26b90937e134bb8f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:23.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.209+0000 7f248559a700 1 -- 172.21.15.16:0/3428294282 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2478004230 con 0x7f248812cf60 2023-09-30T22:31:23.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.209+0000 7f248559a700 1 -- 172.21.15.16:0/3428294282 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2478004390 con 0x7f248812cf60 2023-09-30T22:31:23.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.209+0000 7f248559a700 1 -- 172.21.15.16:0/3428294282 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24780036a0 con 0x7f248812cf60 2023-09-30T22:31:23.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.210+0000 7f248cad1700 1 -- 172.21.15.16:0/3428294282 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f248812cf60 msgr2=0x7f248806f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:23.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.210+0000 7f248cad1700 1 --2- 172.21.15.16:0/3428294282 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f248812cf60 0x7f248806f5d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2478009840 tx=0x7f2478009b50 comp rx=0 tx=0).stop 2023-09-30T22:31:23.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.210+0000 7f248cad1700 1 -- 172.21.15.16:0/3428294282 shutdown_connections 2023-09-30T22:31:23.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.210+0000 7f248cad1700 1 --2- 172.21.15.16:0/3428294282 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f248812cf60 0x7f248806f5d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.210+0000 7f248cad1700 1 --2- 172.21.15.16:0/3428294282 >> v2:172.21.15.16:3300/0 conn(0x7f248812f8b0 0x7f248812fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.210+0000 7f248cad1700 1 -- 172.21.15.16:0/3428294282 >> 172.21.15.16:0/3428294282 conn(0x7f24880a2aa0 msgr2=0x7f24880a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:23.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.210+0000 7f248cad1700 1 -- 172.21.15.16:0/3428294282 shutdown_connections 2023-09-30T22:31:23.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.210+0000 7f248cad1700 1 -- 172.21.15.16:0/3428294282 wait complete. 2023-09-30T22:31:23.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.211+0000 7f248cad1700 1 Processor -- start 2023-09-30T22:31:23.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.212+0000 7f248cad1700 1 -- start start 2023-09-30T22:31:23.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.212+0000 7f248cad1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f248812cf60 0x7f24881c8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:23.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.213+0000 7f248cad1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f248812f8b0 0x7f24881c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:23.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.213+0000 7f248cad1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2488130260 0x7f24881c8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:23.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.213+0000 7f248cad1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2488135430 con 0x7f248812cf60 2023-09-30T22:31:23.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.213+0000 7f248cad1700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f248813b5b0 con 0x7f248812f8b0 2023-09-30T22:31:23.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.213+0000 7f248cad1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f248809eb40 con 0x7f2488130260 2023-09-30T22:31:23.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.213+0000 7f248659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f248812cf60 0x7f24881c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:23.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.213+0000 7f248659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f248812cf60 0x7f24881c8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57222/0 (socket says 172.21.15.16:57222) 2023-09-30T22:31:23.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.213+0000 7f248659c700 1 -- 172.21.15.16:0/468884939 learned_addr learned my addr 172.21.15.16:0/468884939 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:23.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.213+0000 7f2485d9b700 1 --2- 172.21.15.16:0/468884939 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f248812f8b0 0x7f24881c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:23.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.213+0000 7f248659c700 1 -- 172.21.15.16:0/468884939 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2488130260 msgr2=0x7f24881c8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:23.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.213+0000 7f248659c700 1 --2- 172.21.15.16:0/468884939 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2488130260 0x7f24881c8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.213+0000 7f248659c700 1 -- 172.21.15.16:0/468884939 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f248812f8b0 msgr2=0x7f24881c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:23.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.213+0000 7f248659c700 1 --2- 172.21.15.16:0/468884939 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f248812f8b0 0x7f24881c8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.213+0000 7f248659c700 1 -- 172.21.15.16:0/468884939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f247c0094a0 con 0x7f248812cf60 2023-09-30T22:31:23.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.213+0000 7f248659c700 1 --2- 172.21.15.16:0/468884939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f248812cf60 0x7f24881c8560 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f248812d470 tx=0x7f2478004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:23.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.214+0000 7f2476ffd700 1 -- 172.21.15.16:0/468884939 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24780045f0 con 0x7f248812cf60 2023-09-30T22:31:23.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.214+0000 7f2476ffd700 1 -- 172.21.15.16:0/468884939 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2478003e90 con 0x7f248812cf60 2023-09-30T22:31:23.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.214+0000 7f248cad1700 1 -- 172.21.15.16:0/468884939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24780094a0 con 0x7f248812cf60 2023-09-30T22:31:23.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.214+0000 7f2476ffd700 1 -- 172.21.15.16:0/468884939 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2478011870 con 0x7f248812cf60 2023-09-30T22:31:23.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.214+0000 7f248cad1700 1 -- 172.21.15.16:0/468884939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f248813b840 con 0x7f248812cf60 2023-09-30T22:31:23.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.215+0000 7f2476ffd700 1 -- 172.21.15.16:0/468884939 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2478022070 con 0x7f248812cf60 2023-09-30T22:31:23.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.215+0000 7f2476ffd700 1 -- 172.21.15.16:0/468884939 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f247801cca0 con 0x7f248812cf60 2023-09-30T22:31:23.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.215+0000 7f248cad1700 1 -- 172.21.15.16:0/468884939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f248809e160 con 0x7f248812cf60 2023-09-30T22:31:23.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.220+0000 7f2476ffd700 1 -- 172.21.15.16:0/468884939 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2478046060 con 0x7f248812cf60 2023-09-30T22:31:23.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.304+0000 7f9275564700 1 Processor -- start 2023-09-30T22:31:23.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.305+0000 7f9275564700 1 -- start start 2023-09-30T22:31:23.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.305+0000 7f9275564700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f927012cf60 0x7f927006f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:23.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.305+0000 7f9275564700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9270130430 0x7f9270137c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:23.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.305+0000 7f9275564700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f927012cd90 con 0x7f927012f8b0 2023-09-30T22:31:23.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.305+0000 7f9275564700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f927012d330 con 0x7f927012cf60 2023-09-30T22:31:23.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.305+0000 7f9275564700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9270071780 con 0x7f9270130430 2023-09-30T22:31:23.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.305+0000 7f926f7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9270130430 0x7f9270137c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:23.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.305+0000 7f926f7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9270130430 0x7f9270137c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:38656/0 (socket says 172.21.15.16:38656) 2023-09-30T22:31:23.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.305+0000 7f926f7fe700 1 -- 172.21.15.16:0/717551136 learned_addr learned my addr 172.21.15.16:0/717551136 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:23.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.305+0000 7f926e7fc700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f927012f8b0 0x7f927012fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58068/0 (socket says 172.21.15.16:58068) 2023-09-30T22:31:23.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.305+0000 7f926effd700 1 --2- 172.21.15.16:0/717551136 >> v2:172.21.15.16:3300/0 conn(0x7f927012cf60 0x7f927006f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:23.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.306+0000 7f926dffb700 1 -- 172.21.15.16:0/717551136 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 786096835 0 0) 0x7f927012cd90 con 0x7f927012f8b0 2023-09-30T22:31:23.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.306+0000 7f926f7fe700 1 -- 172.21.15.16:0/717551136 >> v2:172.21.15.16:3300/0 conn(0x7f927012cf60 msgr2=0x7f927006f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:23.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.306+0000 7f926f7fe700 1 --2- 172.21.15.16:0/717551136 >> v2:172.21.15.16:3300/0 conn(0x7f927012cf60 0x7f927006f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.306+0000 7f926f7fe700 1 -- 172.21.15.16:0/717551136 >> v1:172.21.15.16:6789/0 conn(0x7f927012f8b0 legacy=0x7f927012fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:23.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.306+0000 7f926f7fe700 1 -- 172.21.15.16:0/717551136 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9270071a40 con 0x7f9270130430 2023-09-30T22:31:23.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.306+0000 7f926f7fe700 1 --2- 172.21.15.16:0/717551136 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9270130430 0x7f9270137c30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9268009e50 tx=0x7f926800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cac96952af2109d8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:23.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.306+0000 7f926dffb700 1 -- 172.21.15.16:0/717551136 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f926800e070 con 0x7f9270130430 2023-09-30T22:31:23.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.306+0000 7f926dffb700 1 -- 172.21.15.16:0/717551136 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f926800b910 con 0x7f9270130430 2023-09-30T22:31:23.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.307+0000 7f926dffb700 1 -- 172.21.15.16:0/717551136 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f926800bcc0 con 0x7f9270130430 2023-09-30T22:31:23.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.307+0000 7f9275564700 1 -- 172.21.15.16:0/717551136 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9270130430 msgr2=0x7f9270137c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:23.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.307+0000 7f9275564700 1 --2- 172.21.15.16:0/717551136 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9270130430 0x7f9270137c30 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9268009e50 tx=0x7f926800b330 comp rx=0 tx=0).stop 2023-09-30T22:31:23.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.307+0000 7f9275564700 1 -- 172.21.15.16:0/717551136 shutdown_connections 2023-09-30T22:31:23.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.307+0000 7f9275564700 1 --2- 172.21.15.16:0/717551136 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9270130430 0x7f9270137c30 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.307+0000 7f9275564700 1 --2- 172.21.15.16:0/717551136 >> v2:172.21.15.16:3300/0 conn(0x7f927012cf60 0x7f927006f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.307+0000 7f9275564700 1 -- 172.21.15.16:0/717551136 >> 172.21.15.16:0/717551136 conn(0x7f92700a2aa0 msgr2=0x7f92700a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:23.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.307+0000 7f9275564700 1 -- 172.21.15.16:0/717551136 shutdown_connections 2023-09-30T22:31:23.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.308+0000 7f9275564700 1 -- 172.21.15.16:0/717551136 wait complete. 2023-09-30T22:31:23.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.309+0000 7f9275564700 1 Processor -- start 2023-09-30T22:31:23.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.309+0000 7f9275564700 1 -- start start 2023-09-30T22:31:23.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.309+0000 7f9275564700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f927012cf60 0x7f9270118cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:23.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.309+0000 7f9275564700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f927012f8b0 0x7f9270119210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:23.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.309+0000 7f9275564700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9270130430 0x7f9270119750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:23.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.309+0000 7f9275564700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f927011b560 con 0x7f927012f8b0 2023-09-30T22:31:23.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.309+0000 7f9275564700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9270129fa0 con 0x7f927012cf60 2023-09-30T22:31:23.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.309+0000 7f9275564700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f927011bf90 con 0x7f9270130430 2023-09-30T22:31:23.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.310+0000 7f926effd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f927012cf60 0x7f9270118cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:23.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.310+0000 7f926e7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f927012f8b0 0x7f9270119210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:23.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.310+0000 7f926f7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9270130430 0x7f9270119750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:23.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.310+0000 7f926f7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9270130430 0x7f9270119750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:38666/0 (socket says 172.21.15.16:38666) 2023-09-30T22:31:23.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.310+0000 7f926effd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f927012cf60 0x7f9270118cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:57490/0 (socket says 172.21.15.16:57490) 2023-09-30T22:31:23.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.310+0000 7f926e7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f927012f8b0 0x7f9270119210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57250/0 (socket says 172.21.15.16:57250) 2023-09-30T22:31:23.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.310+0000 7f926f7fe700 1 -- 172.21.15.16:0/1568130354 learned_addr learned my addr 172.21.15.16:0/1568130354 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:23.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.310+0000 7f926e7fc700 1 -- 172.21.15.16:0/1568130354 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9270130430 msgr2=0x7f9270119750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:23.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.310+0000 7f926e7fc700 1 --2- 172.21.15.16:0/1568130354 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9270130430 0x7f9270119750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.310+0000 7f926e7fc700 1 -- 172.21.15.16:0/1568130354 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f927012cf60 msgr2=0x7f9270118cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:23.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.310+0000 7f926e7fc700 1 --2- 172.21.15.16:0/1568130354 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f927012cf60 0x7f9270118cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.310+0000 7f926e7fc700 1 -- 172.21.15.16:0/1568130354 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9268009b30 con 0x7f927012f8b0 2023-09-30T22:31:23.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.310+0000 7f926e7fc700 1 --2- 172.21.15.16:0/1568130354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f927012f8b0 0x7f9270119210 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f926400c470 tx=0x7f926400c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:23.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.311+0000 7f92577fe700 1 -- 172.21.15.16:0/1568130354 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9264004c80 con 0x7f927012f8b0 2023-09-30T22:31:23.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.311+0000 7f92577fe700 1 -- 172.21.15.16:0/1568130354 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9264004de0 con 0x7f927012f8b0 2023-09-30T22:31:23.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.311+0000 7f9275564700 1 -- 172.21.15.16:0/1568130354 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f927011b8e0 con 0x7f927012f8b0 2023-09-30T22:31:23.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.311+0000 7f92577fe700 1 -- 172.21.15.16:0/1568130354 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9264007310 con 0x7f927012f8b0 2023-09-30T22:31:23.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.311+0000 7f9275564700 1 -- 172.21.15.16:0/1568130354 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f927011a8d0 con 0x7f927012f8b0 2023-09-30T22:31:23.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.312+0000 7f9275564700 1 -- 172.21.15.16:0/1568130354 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9270074ea0 con 0x7f927012f8b0 2023-09-30T22:31:23.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.312+0000 7f92577fe700 1 -- 172.21.15.16:0/1568130354 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9264007470 con 0x7f927012f8b0 2023-09-30T22:31:23.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.313+0000 7f92577fe700 1 -- 172.21.15.16:0/1568130354 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f926403f340 con 0x7f927012f8b0 2023-09-30T22:31:23.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.322+0000 7f92577fe700 1 -- 172.21.15.16:0/1568130354 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9264007720 con 0x7f927012f8b0 2023-09-30T22:31:23.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.348+0000 7fd6d0b8b700 1 Processor -- start 2023-09-30T22:31:23.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.348+0000 7fd6d0b8b700 1 -- start start 2023-09-30T22:31:23.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.349+0000 7fd6d0b8b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd6cc130540 0x7fd6cc130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:23.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.349+0000 7fd6d0b8b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6cc06fcf0 0x7fd6cc138570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:23.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.349+0000 7fd6d0b8b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6cc0716a0 con 0x7fd6cc130e50 2023-09-30T22:31:23.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.349+0000 7fd6d0b8b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6cc09a390 con 0x7fd6cc130540 2023-09-30T22:31:23.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.349+0000 7fd6d0b8b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6cc071290 con 0x7fd6cc06fcf0 2023-09-30T22:31:23.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.349+0000 7fd6ca59c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd6cc130540 0x7fd6cc130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:23.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.349+0000 7fd6ca59c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd6cc130540 0x7fd6cc130910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57262/0 (socket says 172.21.15.16:57262) 2023-09-30T22:31:23.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.349+0000 7fd6ca59c700 1 -- 172.21.15.16:0/202035432 learned_addr learned my addr 172.21.15.16:0/202035432 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:23.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.349+0000 7fd6cad9d700 1 --2- 172.21.15.16:0/202035432 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6cc06fcf0 0x7fd6cc138570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:23.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.349+0000 7fd6ca59c700 1 -- 172.21.15.16:0/202035432 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6cc06fcf0 msgr2=0x7fd6cc138570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:23.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.349+0000 7fd6ca59c700 1 --2- 172.21.15.16:0/202035432 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6cc06fcf0 0x7fd6cc138570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.349+0000 7fd6ca59c700 1 -- 172.21.15.16:0/202035432 >> v1:172.21.15.16:6789/0 conn(0x7fd6cc130e50 legacy=0x7fd6cc06f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:23.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.349+0000 7fd6ca59c700 1 -- 172.21.15.16:0/202035432 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6cc130340 con 0x7fd6cc130540 2023-09-30T22:31:23.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.350+0000 7fd6ca59c700 1 --2- 172.21.15.16:0/202035432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6cc130540 0x7fd6cc130910 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc00bb90 tx=0x7fd6bc00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2be2ae7caed0d5cd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:23.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.350+0000 7fd6c959a700 1 -- 172.21.15.16:0/202035432 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc00dcf0 con 0x7fd6cc130540 2023-09-30T22:31:23.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.350+0000 7fd6c959a700 1 -- 172.21.15.16:0/202035432 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6bc00de50 con 0x7fd6cc130540 2023-09-30T22:31:23.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.350+0000 7fd6c959a700 1 -- 172.21.15.16:0/202035432 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc004250 con 0x7fd6cc130540 2023-09-30T22:31:23.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.351+0000 7fd6d0b8b700 1 -- 172.21.15.16:0/202035432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6cc130540 msgr2=0x7fd6cc130910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:23.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.351+0000 7fd6d0b8b700 1 --2- 172.21.15.16:0/202035432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6cc130540 0x7fd6cc130910 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc00bb90 tx=0x7fd6bc00bea0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.351+0000 7fd6d0b8b700 1 -- 172.21.15.16:0/202035432 shutdown_connections 2023-09-30T22:31:23.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.351+0000 7fd6d0b8b700 1 --2- 172.21.15.16:0/202035432 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6cc06fcf0 0x7fd6cc138570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.351+0000 7fd6d0b8b700 1 --2- 172.21.15.16:0/202035432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6cc130540 0x7fd6cc130910 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.351+0000 7fd6d0b8b700 1 -- 172.21.15.16:0/202035432 >> 172.21.15.16:0/202035432 conn(0x7fd6cc0a2aa0 msgr2=0x7fd6cc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:23.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.351+0000 7fd6d0b8b700 1 -- 172.21.15.16:0/202035432 shutdown_connections 2023-09-30T22:31:23.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.351+0000 7fd6d0b8b700 1 -- 172.21.15.16:0/202035432 wait complete. 2023-09-30T22:31:23.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.353+0000 7fd6d0b8b700 1 Processor -- start 2023-09-30T22:31:23.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.353+0000 7fd6d0b8b700 1 -- start start 2023-09-30T22:31:23.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.353+0000 7fd6d0b8b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd6cc06fcf0 0x7fd6cc1196d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:23.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.354+0000 7fd6d0b8b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6cc130540 0x7fd6cc119c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:23.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.354+0000 7fd6d0b8b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6cc130e50 0x7fd6cc11a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:23.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.354+0000 7fd6d0b8b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6cc11cfb0 con 0x7fd6cc130e50 2023-09-30T22:31:23.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.354+0000 7fd6d0b8b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6cc11aac0 con 0x7fd6cc130540 2023-09-30T22:31:23.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.354+0000 7fd6d0b8b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd6cc11d1b0 con 0x7fd6cc06fcf0 2023-09-30T22:31:23.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.354+0000 7fd6ca59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd6cc06fcf0 0x7fd6cc1196d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:23.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.354+0000 7fd6ca59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd6cc06fcf0 0x7fd6cc1196d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:38682/0 (socket says 172.21.15.16:38682) 2023-09-30T22:31:23.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.354+0000 7fd6ca59c700 1 -- 172.21.15.16:0/2485799931 learned_addr learned my addr 172.21.15.16:0/2485799931 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:23.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.354+0000 7fd6c9d9b700 1 --2- 172.21.15.16:0/2485799931 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6cc130540 0x7fd6cc119c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:23.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.354+0000 7fd6cad9d700 1 --2- 172.21.15.16:0/2485799931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6cc130e50 0x7fd6cc11a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:23.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.354+0000 7fd6cad9d700 1 -- 172.21.15.16:0/2485799931 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd6cc06fcf0 msgr2=0x7fd6cc1196d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:23.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.354+0000 7fd6cad9d700 1 --2- 172.21.15.16:0/2485799931 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd6cc06fcf0 0x7fd6cc1196d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.354+0000 7fd6cad9d700 1 -- 172.21.15.16:0/2485799931 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6cc130540 msgr2=0x7fd6cc119c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:23.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.354+0000 7fd6cad9d700 1 --2- 172.21.15.16:0/2485799931 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6cc130540 0x7fd6cc119c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.354+0000 7fd6cad9d700 1 -- 172.21.15.16:0/2485799931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6bc00b7f0 con 0x7fd6cc130e50 2023-09-30T22:31:23.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.354+0000 7fd6c9d9b700 1 --2- 172.21.15.16:0/2485799931 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6cc130540 0x7fd6cc119c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:23.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.354+0000 7fd6cad9d700 1 --2- 172.21.15.16:0/2485799931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6cc130e50 0x7fd6cc11a150 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fd6c400efc0 tx=0x7fd6c400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:23.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.355+0000 7fa2dec70700 1 Processor -- start 2023-09-30T22:31:23.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.355+0000 7fa2dec70700 1 -- start start 2023-09-30T22:31:23.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.355+0000 7fa2dec70700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa2d8133940 0x7fa2d8133d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:23.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.355+0000 7fa2dec70700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa2d8134250 0x7fa2d809ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:23.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.355+0000 7fa2dec70700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa2d809d3c0 0x7fa2d809d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:23.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.355+0000 7fa2dec70700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2d809abd0 con 0x7fa2d809d3c0 2023-09-30T22:31:23.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.355+0000 7fa2dec70700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d8133580 con 0x7fa2d8134250 2023-09-30T22:31:23.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.355+0000 7fa2dec70700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d809a2d0 con 0x7fa2d8133940 2023-09-30T22:31:23.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.355+0000 7fa2dca0c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa2d8133940 0x7fa2d8133d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:23.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.355+0000 7fa2dca0c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa2d8133940 0x7fa2d8133d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:38696/0 (socket says 172.21.15.16:38696) 2023-09-30T22:31:23.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.355+0000 7fa2dca0c700 1 -- 172.21.15.16:0/1193227597 learned_addr learned my addr 172.21.15.16:0/1193227597 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:23.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.355+0000 7fa2dd20d700 1 --2- 172.21.15.16:0/1193227597 >> v2:172.21.15.16:3300/0 conn(0x7fa2d809d3c0 0x7fa2d809d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:23.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.356+0000 7fa2dca0c700 1 -- 172.21.15.16:0/1193227597 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa2d8134250 msgr2=0x7fa2d809ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:23.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.356+0000 7fa2dca0c700 1 --2- 172.21.15.16:0/1193227597 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa2d8134250 0x7fa2d809ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.356+0000 7fa2dca0c700 1 -- 172.21.15.16:0/1193227597 >> v2:172.21.15.16:3300/0 conn(0x7fa2d809d3c0 msgr2=0x7fa2d809d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:23.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.356+0000 7fa2dca0c700 1 --2- 172.21.15.16:0/1193227597 >> v2:172.21.15.16:3300/0 conn(0x7fa2d809d3c0 0x7fa2d809d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.356+0000 7fa2dca0c700 1 -- 172.21.15.16:0/1193227597 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d809e950 con 0x7fa2d8133940 2023-09-30T22:31:23.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.356+0000 7fd6baffd700 1 -- 172.21.15.16:0/2485799931 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c400cc90 con 0x7fd6cc130e50 2023-09-30T22:31:23.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.356+0000 7fd6baffd700 1 -- 172.21.15.16:0/2485799931 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6c400f040 con 0x7fd6cc130e50 2023-09-30T22:31:23.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.356+0000 7fd6d0b8b700 1 -- 172.21.15.16:0/2485799931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6cc11c0a0 con 0x7fd6cc130e50 2023-09-30T22:31:23.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.356+0000 7fa2dd20d700 1 --2- 172.21.15.16:0/1193227597 >> v2:172.21.15.16:3300/0 conn(0x7fa2d809d3c0 0x7fa2d809d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:23.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.356+0000 7fd6d0b8b700 1 -- 172.21.15.16:0/2485799931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6cc0a0ae0 con 0x7fd6cc130e50 2023-09-30T22:31:23.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.356+0000 7fd6baffd700 1 -- 172.21.15.16:0/2485799931 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c4009e60 con 0x7fd6cc130e50 2023-09-30T22:31:23.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.356+0000 7fa2dca0c700 1 --2- 172.21.15.16:0/1193227597 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa2d8133940 0x7fa2d8133d10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0009920 tx=0x7fa2c0009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f3efe8264730cfc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:23.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.356+0000 7fa2d77fe700 1 -- 172.21.15.16:0/1193227597 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c0011830 con 0x7fa2d8133940 2023-09-30T22:31:23.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.356+0000 7fa2d77fe700 1 -- 172.21.15.16:0/1193227597 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2c0011990 con 0x7fa2d8133940 2023-09-30T22:31:23.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.356+0000 7fa2d77fe700 1 -- 172.21.15.16:0/1193227597 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c0011d20 con 0x7fa2d8133940 2023-09-30T22:31:23.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.357+0000 7fd6d0b8b700 1 -- 172.21.15.16:0/2485799931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6b00052e0 con 0x7fd6cc130e50 2023-09-30T22:31:23.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.357+0000 7fa2dec70700 1 -- 172.21.15.16:0/1193227597 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa2d8133940 msgr2=0x7fa2d8133d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:23.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.357+0000 7fa2dec70700 1 --2- 172.21.15.16:0/1193227597 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa2d8133940 0x7fa2d8133d10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0009920 tx=0x7fa2c0009c30 comp rx=0 tx=0).stop 2023-09-30T22:31:23.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.358+0000 7fa2dec70700 1 -- 172.21.15.16:0/1193227597 shutdown_connections 2023-09-30T22:31:23.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.358+0000 7fa2dec70700 1 --2- 172.21.15.16:0/1193227597 >> v2:172.21.15.16:3300/0 conn(0x7fa2d809d3c0 0x7fa2d809d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.358+0000 7fa2dec70700 1 --2- 172.21.15.16:0/1193227597 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa2d8133940 0x7fa2d8133d10 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.358+0000 7fa2dec70700 1 --2- 172.21.15.16:0/1193227597 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa2d8134250 0x7fa2d809ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.358+0000 7fa2dec70700 1 -- 172.21.15.16:0/1193227597 >> 172.21.15.16:0/1193227597 conn(0x7fa2d80a2360 msgr2=0x7fa2d80a2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:23.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.358+0000 7fd6baffd700 1 -- 172.21.15.16:0/2485799931 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd6c4014480 con 0x7fd6cc130e50 2023-09-30T22:31:23.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.358+0000 7fd6baffd700 1 -- 172.21.15.16:0/2485799931 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd6c40406c0 con 0x7fd6cc130e50 2023-09-30T22:31:23.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.360+0000 7fa2dec70700 1 -- 172.21.15.16:0/1193227597 shutdown_connections 2023-09-30T22:31:23.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.360+0000 7fa2dec70700 1 -- 172.21.15.16:0/1193227597 wait complete. 2023-09-30T22:31:23.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.361+0000 7fa2dec70700 1 Processor -- start 2023-09-30T22:31:23.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.361+0000 7fa2dec70700 1 -- start start 2023-09-30T22:31:23.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.362+0000 7fa2dec70700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2d809d3c0 0x7fa2d812e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:23.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.362+0000 7fa2dec70700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa2d8133940 0x7fa2d812ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:23.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.362+0000 7fa2dec70700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa2d8134250 0x7fa2d812f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:23.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.362+0000 7fa2dec70700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d8130b90 con 0x7fa2d809d3c0 2023-09-30T22:31:23.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.362+0000 7fa2dec70700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d812fc80 con 0x7fa2d8134250 2023-09-30T22:31:23.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.362+0000 7fa2dec70700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d81302c0 con 0x7fa2d8133940 2023-09-30T22:31:23.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.362+0000 7fa2d7fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa2d8133940 0x7fa2d812ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:23.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.362+0000 7fa2d7fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa2d8133940 0x7fa2d812ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:38704/0 (socket says 172.21.15.16:38704) 2023-09-30T22:31:23.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.362+0000 7fa2d7fff700 1 -- 172.21.15.16:0/2850005104 learned_addr learned my addr 172.21.15.16:0/2850005104 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:23.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.362+0000 7fa2dd20d700 1 --2- 172.21.15.16:0/2850005104 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa2d8134250 0x7fa2d812f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:23.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.363+0000 7fa2dca0c700 1 --2- 172.21.15.16:0/2850005104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2d809d3c0 0x7fa2d812e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:23.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.363+0000 7fa2dd20d700 1 -- 172.21.15.16:0/2850005104 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa2d8133940 msgr2=0x7fa2d812ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:23.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.363+0000 7fa2dd20d700 1 --2- 172.21.15.16:0/2850005104 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa2d8133940 0x7fa2d812ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.363+0000 7fa2dd20d700 1 -- 172.21.15.16:0/2850005104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2d809d3c0 msgr2=0x7fa2d812e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:23.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.363+0000 7fa2dd20d700 1 --2- 172.21.15.16:0/2850005104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2d809d3c0 0x7fa2d812e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.363+0000 7fa2dd20d700 1 -- 172.21.15.16:0/2850005104 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d00094a0 con 0x7fa2d8134250 2023-09-30T22:31:23.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.363+0000 7fd6baffd700 1 -- 172.21.15.16:0/2485799931 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd6c4012930 con 0x7fd6cc130e50 2023-09-30T22:31:23.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.363+0000 7fa2dd20d700 1 --2- 172.21.15.16:0/2850005104 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa2d8134250 0x7fa2d812f2a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa2d000e8b0 tx=0x7fa2d000ebc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:23.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.363+0000 7fa2d57fa700 1 -- 172.21.15.16:0/2850005104 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d0017d80 con 0x7fa2d8134250 2023-09-30T22:31:23.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.363+0000 7fa2d57fa700 1 -- 172.21.15.16:0/2850005104 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2d0004020 con 0x7fa2d8134250 2023-09-30T22:31:23.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.363+0000 7fa2d57fa700 1 -- 172.21.15.16:0/2850005104 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d00043d0 con 0x7fa2d8134250 2023-09-30T22:31:23.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.364+0000 7fa2dec70700 1 -- 172.21.15.16:0/2850005104 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2c0009550 con 0x7fa2d8134250 2023-09-30T22:31:23.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.364+0000 7fa2dec70700 1 -- 172.21.15.16:0/2850005104 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2d808b150 con 0x7fa2d8134250 2023-09-30T22:31:23.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.365+0000 7fa2d57fa700 1 -- 172.21.15.16:0/2850005104 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa2d00036a0 con 0x7fa2d8134250 2023-09-30T22:31:23.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.365+0000 7fa2d57fa700 1 -- 172.21.15.16:0/2850005104 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fa2d0047920 con 0x7fa2d8134250 2023-09-30T22:31:23.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.365+0000 7fa2dec70700 1 -- 172.21.15.16:0/2850005104 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2d809fa30 con 0x7fa2d8134250 2023-09-30T22:31:23.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.370+0000 7fa2d57fa700 1 -- 172.21.15.16:0/2850005104 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa2d0026900 con 0x7fa2d8134250 2023-09-30T22:31:23.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.477+0000 7f248cad1700 1 -- 172.21.15.16:0/468884939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2488074ea0 con 0x7f248812cf60 2023-09-30T22:31:23.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.477+0000 7f2476ffd700 1 -- 172.21.15.16:0/468884939 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2478025470 con 0x7f248812cf60 2023-09-30T22:31:23.479 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:31:23.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.479+0000 7f248cad1700 1 -- 172.21.15.16:0/468884939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f248812cf60 msgr2=0x7f24881c8560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:23.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.479+0000 7f248cad1700 1 --2- 172.21.15.16:0/468884939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f248812cf60 0x7f24881c8560 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f248812d470 tx=0x7f2478004750 comp rx=0 tx=0).stop 2023-09-30T22:31:23.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.480+0000 7f248cad1700 1 -- 172.21.15.16:0/468884939 shutdown_connections 2023-09-30T22:31:23.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.480+0000 7f248cad1700 1 --2- 172.21.15.16:0/468884939 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2488130260 0x7f24881c8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.480+0000 7f248cad1700 1 --2- 172.21.15.16:0/468884939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f248812cf60 0x7f24881c8560 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.480+0000 7f248cad1700 1 --2- 172.21.15.16:0/468884939 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f248812f8b0 0x7f24881c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.480+0000 7f248cad1700 1 -- 172.21.15.16:0/468884939 >> 172.21.15.16:0/468884939 conn(0x7f24880a2aa0 msgr2=0x7f248806e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:23.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.480+0000 7f248cad1700 1 -- 172.21.15.16:0/468884939 shutdown_connections 2023-09-30T22:31:23.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.480+0000 7f248cad1700 1 -- 172.21.15.16:0/468884939 wait complete. 2023-09-30T22:31:23.500 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:31:23.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.601+0000 7f9275564700 1 -- 172.21.15.16:0/1568130354 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f92700713a0 con 0x7f927012f8b0 2023-09-30T22:31:23.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.601+0000 7f92577fe700 1 -- 172.21.15.16:0/1568130354 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9264011e80 con 0x7f927012f8b0 2023-09-30T22:31:23.603 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:31:23.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.603+0000 7f9275564700 1 -- 172.21.15.16:0/1568130354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f927012f8b0 msgr2=0x7f9270119210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:23.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.603+0000 7f9275564700 1 --2- 172.21.15.16:0/1568130354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f927012f8b0 0x7f9270119210 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f926400c470 tx=0x7f926400c780 comp rx=0 tx=0).stop 2023-09-30T22:31:23.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.604+0000 7f9275564700 1 -- 172.21.15.16:0/1568130354 shutdown_connections 2023-09-30T22:31:23.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.604+0000 7f9275564700 1 --2- 172.21.15.16:0/1568130354 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f927012cf60 0x7f9270118cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.604+0000 7f9275564700 1 --2- 172.21.15.16:0/1568130354 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9270130430 0x7f9270119750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.604+0000 7f9275564700 1 --2- 172.21.15.16:0/1568130354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f927012f8b0 0x7f9270119210 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.604+0000 7f9275564700 1 -- 172.21.15.16:0/1568130354 >> 172.21.15.16:0/1568130354 conn(0x7f92700a2aa0 msgr2=0x7f92700a0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:23.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.604+0000 7f9275564700 1 -- 172.21.15.16:0/1568130354 shutdown_connections 2023-09-30T22:31:23.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.604+0000 7f9275564700 1 -- 172.21.15.16:0/1568130354 wait complete. 2023-09-30T22:31:23.619 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:31:23.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.622+0000 7fa2dec70700 1 -- 172.21.15.16:0/2850005104 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa2d8074ea0 con 0x7fa2d8134250 2023-09-30T22:31:23.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.622+0000 7fa2d57fa700 1 -- 172.21.15.16:0/2850005104 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa2d0026310 con 0x7fa2d8134250 2023-09-30T22:31:23.624 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:31:23.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.625+0000 7fa2dec70700 1 -- 172.21.15.16:0/2850005104 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa2d8134250 msgr2=0x7fa2d812f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:23.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.625+0000 7fa2dec70700 1 --2- 172.21.15.16:0/2850005104 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa2d8134250 0x7fa2d812f2a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa2d000e8b0 tx=0x7fa2d000ebc0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.625+0000 7fa2dec70700 1 -- 172.21.15.16:0/2850005104 shutdown_connections 2023-09-30T22:31:23.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.625+0000 7fa2dec70700 1 --2- 172.21.15.16:0/2850005104 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa2d8134250 0x7fa2d812f2a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.625+0000 7fa2dec70700 1 --2- 172.21.15.16:0/2850005104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa2d809d3c0 0x7fa2d812e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.625+0000 7fa2dec70700 1 --2- 172.21.15.16:0/2850005104 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa2d8133940 0x7fa2d812ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.625+0000 7fa2dec70700 1 -- 172.21.15.16:0/2850005104 >> 172.21.15.16:0/2850005104 conn(0x7fa2d80a2360 msgr2=0x7fa2d80a0080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:23.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.625+0000 7fa2dec70700 1 -- 172.21.15.16:0/2850005104 shutdown_connections 2023-09-30T22:31:23.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.625+0000 7fa2dec70700 1 -- 172.21.15.16:0/2850005104 wait complete. 2023-09-30T22:31:23.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.629+0000 7fd6d0b8b700 1 -- 172.21.15.16:0/2485799931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd6b0005860 con 0x7fd6cc130e50 2023-09-30T22:31:23.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.629+0000 7fd6baffd700 1 -- 172.21.15.16:0/2485799931 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd6c402a030 con 0x7fd6cc130e50 2023-09-30T22:31:23.630 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:31:23.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.631+0000 7fd6d0b8b700 1 -- 172.21.15.16:0/2485799931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6cc130e50 msgr2=0x7fd6cc11a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:23.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.631+0000 7fd6d0b8b700 1 --2- 172.21.15.16:0/2485799931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6cc130e50 0x7fd6cc11a150 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fd6c400efc0 tx=0x7fd6c400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.631+0000 7fd6d0b8b700 1 -- 172.21.15.16:0/2485799931 shutdown_connections 2023-09-30T22:31:23.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.631+0000 7fd6d0b8b700 1 --2- 172.21.15.16:0/2485799931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6cc130e50 0x7fd6cc11a150 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.631+0000 7fd6d0b8b700 1 --2- 172.21.15.16:0/2485799931 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd6cc06fcf0 0x7fd6cc1196d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.631+0000 7fd6d0b8b700 1 --2- 172.21.15.16:0/2485799931 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6cc130540 0x7fd6cc119c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:23.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.631+0000 7fd6d0b8b700 1 -- 172.21.15.16:0/2485799931 >> 172.21.15.16:0/2485799931 conn(0x7fd6cc0a2aa0 msgr2=0x7fd6cc12d700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:23.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.631+0000 7fd6d0b8b700 1 -- 172.21.15.16:0/2485799931 shutdown_connections 2023-09-30T22:31:23.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:23.631+0000 7fd6d0b8b700 1 -- 172.21.15.16:0/2485799931 wait complete. 2023-09-30T22:31:23.641 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:31:23.647 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:31:24.502 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:24.620 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:24.642 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:24.648 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:24.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.649+0000 7fbd0143d700 1 Processor -- start 2023-09-30T22:31:24.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.649+0000 7fbd0143d700 1 -- start start 2023-09-30T22:31:24.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.650+0000 7fbd0143d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbcfc09e930 0x7fbcfc09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:24.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.650+0000 7fbd0143d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbcfc09f240 0x7fbcfc1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:24.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.650+0000 7fbd0143d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcfc070940 con 0x7fbcfc09f650 2023-09-30T22:31:24.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.650+0000 7fbd0143d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcfc06fd60 con 0x7fbcfc09e930 2023-09-30T22:31:24.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.650+0000 7fbd0143d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbcfc06fa80 con 0x7fbcfc09f240 2023-09-30T22:31:24.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.650+0000 7fbcfaffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbcfc09e930 0x7fbcfc09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:24.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.650+0000 7fbcfa7fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbcfc09f240 0x7fbcfc1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:24.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.650+0000 7fbcfb7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fbcfc09f650 0x7fbcfc131d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58084/0 (socket says 172.21.15.16:58084) 2023-09-30T22:31:24.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.650+0000 7fbcfb7fe700 1 -- 172.21.15.16:0/1642533785 learned_addr learned my addr 172.21.15.16:0/1642533785 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:31:24.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.650+0000 7fbcfaffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbcfc09e930 0x7fbcfc09ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57294/0 (socket says 172.21.15.16:57294) 2023-09-30T22:31:24.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.650+0000 7fbcfaffd700 1 -- 172.21.15.16:0/1642533785 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbcfc09f240 msgr2=0x7fbcfc1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:24.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.650+0000 7fbcfaffd700 1 --2- 172.21.15.16:0/1642533785 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbcfc09f240 0x7fbcfc1317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:24.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.650+0000 7fbcfaffd700 1 -- 172.21.15.16:0/1642533785 >> v1:172.21.15.16:6789/0 conn(0x7fbcfc09f650 legacy=0x7fbcfc131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:24.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.650+0000 7fbcfaffd700 1 -- 172.21.15.16:0/1642533785 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcfc09fb60 con 0x7fbcfc09e930 2023-09-30T22:31:24.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.650+0000 7fbcfa7fc700 1 --2- 172.21.15.16:0/1642533785 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbcfc09f240 0x7fbcfc1317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:24.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.651+0000 7fbcfaffd700 1 --2- 172.21.15.16:0/1642533785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcfc09e930 0x7fbcfc09ed00 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fbcec009d90 tx=0x7fbcec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf95ed3d39c0300d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:24.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.651+0000 7fbcf9ffb700 1 -- 172.21.15.16:0/1642533785 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcec00e070 con 0x7fbcfc09e930 2023-09-30T22:31:24.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.651+0000 7fbcf9ffb700 1 -- 172.21.15.16:0/1642533785 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcec00bb20 con 0x7fbcfc09e930 2023-09-30T22:31:24.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.651+0000 7fbcf9ffb700 1 -- 172.21.15.16:0/1642533785 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcec012430 con 0x7fbcfc09e930 2023-09-30T22:31:24.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.652+0000 7fbd0143d700 1 -- 172.21.15.16:0/1642533785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcfc09e930 msgr2=0x7fbcfc09ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:24.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.652+0000 7fbd0143d700 1 --2- 172.21.15.16:0/1642533785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcfc09e930 0x7fbcfc09ed00 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fbcec009d90 tx=0x7fbcec00b330 comp rx=0 tx=0).stop 2023-09-30T22:31:24.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.652+0000 7fbd0143d700 1 -- 172.21.15.16:0/1642533785 shutdown_connections 2023-09-30T22:31:24.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.652+0000 7fbd0143d700 1 --2- 172.21.15.16:0/1642533785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcfc09e930 0x7fbcfc09ed00 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:24.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.652+0000 7fbd0143d700 1 --2- 172.21.15.16:0/1642533785 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbcfc09f240 0x7fbcfc1317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:24.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.652+0000 7fbd0143d700 1 -- 172.21.15.16:0/1642533785 >> 172.21.15.16:0/1642533785 conn(0x7fbcfc0a4210 msgr2=0x7fbcfc0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:24.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.652+0000 7fbd0143d700 1 -- 172.21.15.16:0/1642533785 shutdown_connections 2023-09-30T22:31:24.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.652+0000 7fbd0143d700 1 -- 172.21.15.16:0/1642533785 wait complete. 2023-09-30T22:31:24.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.654+0000 7fbd0143d700 1 Processor -- start 2023-09-30T22:31:24.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.654+0000 7fbd0143d700 1 -- start start 2023-09-30T22:31:24.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.655+0000 7fbd0143d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbcfc09e930 0x7fbcfc1a3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:24.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.655+0000 7fbd0143d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcfc09f240 0x7fbcfc1a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:24.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.655+0000 7fbd0143d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbcfc09f650 0x7fbcfc1a4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:24.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.655+0000 7fbd0143d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcfc1a8260 con 0x7fbcfc09f240 2023-09-30T22:31:24.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.655+0000 7fbd0143d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcfc1a78a0 con 0x7fbcfc09e930 2023-09-30T22:31:24.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.655+0000 7fbd0143d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbcfc1a6be0 con 0x7fbcfc09f650 2023-09-30T22:31:24.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.655+0000 7fbcfaffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbcfc09e930 0x7fbcfc1a3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:24.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.655+0000 7fbcfaffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbcfc09e930 0x7fbcfc1a3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:57530/0 (socket says 172.21.15.16:57530) 2023-09-30T22:31:24.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.655+0000 7fbcfb7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbcfc09f650 0x7fbcfc1a4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:24.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.655+0000 7fbcfaffd700 1 -- 172.21.15.16:0/2690105359 learned_addr learned my addr 172.21.15.16:0/2690105359 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:24.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.655+0000 7fbcfaffd700 1 -- 172.21.15.16:0/2690105359 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbcfc09f650 msgr2=0x7fbcfc1a4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:24.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.655+0000 7fbcfaffd700 1 --2- 172.21.15.16:0/2690105359 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbcfc09f650 0x7fbcfc1a4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:24.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.655+0000 7fbcfaffd700 1 -- 172.21.15.16:0/2690105359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcfc09f240 msgr2=0x7fbcfc1a42c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:31:24.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.655+0000 7fbcfaffd700 1 --2- 172.21.15.16:0/2690105359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcfc09f240 0x7fbcfc1a42c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:24.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.655+0000 7fbcfaffd700 1 -- 172.21.15.16:0/2690105359 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbce40094a0 con 0x7fbcfc09e930 2023-09-30T22:31:24.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.655+0000 7fbcfb7fe700 1 --2- 172.21.15.16:0/2690105359 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbcfc09f650 0x7fbcfc1a4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:24.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.656+0000 7fbcfaffd700 1 --2- 172.21.15.16:0/2690105359 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbcfc09e930 0x7fbcfc1a3d80 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc0a0b00 tx=0x7fbcec004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:24.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.656+0000 7fbcf37fe700 1 -- 172.21.15.16:0/2690105359 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcec012430 con 0x7fbcfc09e930 2023-09-30T22:31:24.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.656+0000 7fbcf37fe700 1 -- 172.21.15.16:0/2690105359 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcec020070 con 0x7fbcfc09e930 2023-09-30T22:31:24.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.656+0000 7fbcf37fe700 1 -- 172.21.15.16:0/2690105359 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcec004280 con 0x7fbcfc09e930 2023-09-30T22:31:24.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.657+0000 7fbd0143d700 1 -- 172.21.15.16:0/2690105359 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcec009a70 con 0x7fbcfc09e930 2023-09-30T22:31:24.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.657+0000 7fbd0143d700 1 -- 172.21.15.16:0/2690105359 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcfc0a0190 con 0x7fbcfc09e930 2023-09-30T22:31:24.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.658+0000 7fbcf37fe700 1 -- 172.21.15.16:0/2690105359 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbcec0043e0 con 0x7fbcfc09e930 2023-09-30T22:31:24.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.658+0000 7fbcf37fe700 1 -- 172.21.15.16:0/2690105359 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fbcec022070 con 0x7fbcfc09e930 2023-09-30T22:31:24.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.658+0000 7fbd0143d700 1 -- 172.21.15.16:0/2690105359 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcfc1a7440 con 0x7fbcfc09e930 2023-09-30T22:31:24.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.665+0000 7fbcf37fe700 1 -- 172.21.15.16:0/2690105359 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbcec01c400 con 0x7fbcfc09e930 2023-09-30T22:31:24.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.776+0000 7fafc8f46700 1 Processor -- start 2023-09-30T22:31:24.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.776+0000 7fafc8f46700 1 -- start start 2023-09-30T22:31:24.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.776+0000 7fafc8f46700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fafc409d6f0 0x7fafc409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:24.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.776+0000 7fafc8f46700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fafc409e000 0x7fafc4130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:24.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.776+0000 7fafc8f46700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafc409f130 con 0x7fafc409e410 2023-09-30T22:31:24.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.776+0000 7fafc8f46700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafc409f520 con 0x7fafc409d6f0 2023-09-30T22:31:24.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.776+0000 7fafc8f46700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fafc409e930 con 0x7fafc409e000 2023-09-30T22:31:24.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.777+0000 7fafc259c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fafc409d6f0 0x7fafc409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:24.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.777+0000 7fafc2d9d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fafc409e410 0x7fafc4130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58098/0 (socket says 172.21.15.16:58098) 2023-09-30T22:31:24.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.777+0000 7fafc2d9d700 1 -- 172.21.15.16:0/2575470260 learned_addr learned my addr 172.21.15.16:0/2575470260 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:31:24.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.777+0000 7fafc259c700 1 -- 172.21.15.16:0/2575470260 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fafc409e000 msgr2=0x7fafc4130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:24.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.777+0000 7fafc259c700 1 --2- 172.21.15.16:0/2575470260 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fafc409e000 0x7fafc4130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:24.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.777+0000 7fafc259c700 1 -- 172.21.15.16:0/2575470260 >> v1:172.21.15.16:6789/0 conn(0x7fafc409e410 legacy=0x7fafc4130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:24.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.777+0000 7fafc159a700 1 -- 172.21.15.16:0/2575470260 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 942169906 0 0) 0x7fafc409f130 con 0x7fafc409e410 2023-09-30T22:31:24.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.777+0000 7fafc259c700 1 -- 172.21.15.16:0/2575470260 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc409b2d0 con 0x7fafc409d6f0 2023-09-30T22:31:24.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.777+0000 7fafc259c700 1 --2- 172.21.15.16:0/2575470260 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fafc409d6f0 0x7fafc409dac0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fafb4009870 tx=0x7fafb4009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b45af525adbfd387 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:24.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.777+0000 7fafc159a700 1 -- 172.21.15.16:0/2575470260 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb4011830 con 0x7fafc409d6f0 2023-09-30T22:31:24.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.778+0000 7fafc159a700 1 -- 172.21.15.16:0/2575470260 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafb4011990 con 0x7fafc409d6f0 2023-09-30T22:31:24.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.778+0000 7fafc159a700 1 -- 172.21.15.16:0/2575470260 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb4011d20 con 0x7fafc409d6f0 2023-09-30T22:31:24.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.779+0000 7fafc8f46700 1 -- 172.21.15.16:0/2575470260 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fafc409d6f0 msgr2=0x7fafc409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:24.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.779+0000 7fafc8f46700 1 --2- 172.21.15.16:0/2575470260 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fafc409d6f0 0x7fafc409dac0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fafb4009870 tx=0x7fafb4009b80 comp rx=0 tx=0).stop 2023-09-30T22:31:24.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.779+0000 7fafc8f46700 1 -- 172.21.15.16:0/2575470260 shutdown_connections 2023-09-30T22:31:24.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.779+0000 7fafc8f46700 1 --2- 172.21.15.16:0/2575470260 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fafc409d6f0 0x7fafc409dac0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:24.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.779+0000 7fafc8f46700 1 --2- 172.21.15.16:0/2575470260 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fafc409e000 0x7fafc4130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:24.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.779+0000 7fafc8f46700 1 -- 172.21.15.16:0/2575470260 >> 172.21.15.16:0/2575470260 conn(0x7fafc4066f50 msgr2=0x7fafc409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:24.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.779+0000 7fafc8f46700 1 -- 172.21.15.16:0/2575470260 shutdown_connections 2023-09-30T22:31:24.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.779+0000 7fafc8f46700 1 -- 172.21.15.16:0/2575470260 wait complete. 2023-09-30T22:31:24.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.780+0000 7fafc8f46700 1 Processor -- start 2023-09-30T22:31:24.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.781+0000 7fafc8f46700 1 -- start start 2023-09-30T22:31:24.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.781+0000 7fafc8f46700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafc409d6f0 0x7fafc4139940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:24.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.781+0000 7fafc8f46700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fafc409e000 0x7fafc4139e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:24.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.781+0000 7fafc8f46700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fafc409e410 0x7fafc413a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:24.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.781+0000 7fafc8f46700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc4132220 con 0x7fafc409d6f0 2023-09-30T22:31:24.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.781+0000 7fafc8f46700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafc413aea0 con 0x7fafc409e410 2023-09-30T22:31:24.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.781+0000 7fafc8f46700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fafc40a3980 con 0x7fafc409e000 2023-09-30T22:31:24.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.782+0000 7fafc1d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fafc409e000 0x7fafc4139e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:24.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.782+0000 7fafc1d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fafc409e000 0x7fafc4139e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:38750/0 (socket says 172.21.15.16:38750) 2023-09-30T22:31:24.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.782+0000 7fafc1d9b700 1 -- 172.21.15.16:0/2217387433 learned_addr learned my addr 172.21.15.16:0/2217387433 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:24.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.782+0000 7fafc2d9d700 1 --2- 172.21.15.16:0/2217387433 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fafc409e410 0x7fafc413a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:24.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.782+0000 7fafc259c700 1 --2- 172.21.15.16:0/2217387433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafc409d6f0 0x7fafc4139940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:24.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.782+0000 7fafc2d9d700 1 -- 172.21.15.16:0/2217387433 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fafc409e000 msgr2=0x7fafc4139e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:24.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.782+0000 7fafc2d9d700 1 --2- 172.21.15.16:0/2217387433 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fafc409e000 0x7fafc4139e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:24.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.782+0000 7fafc2d9d700 1 -- 172.21.15.16:0/2217387433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafc409d6f0 msgr2=0x7fafc4139940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:24.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.782+0000 7fafc2d9d700 1 --2- 172.21.15.16:0/2217387433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafc409d6f0 0x7fafc4139940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:24.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.782+0000 7fafc2d9d700 1 -- 172.21.15.16:0/2217387433 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb40094a0 con 0x7fafc409e410 2023-09-30T22:31:24.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.782+0000 7fafc259c700 1 --2- 172.21.15.16:0/2217387433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafc409d6f0 0x7fafc4139940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:24.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.782+0000 7fafc2d9d700 1 --2- 172.21.15.16:0/2217387433 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fafc409e410 0x7fafc413a3c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fafbc00cbe0 tx=0x7fafbc00cef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:24.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.783+0000 7fafb2ffd700 1 -- 172.21.15.16:0/2217387433 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafbc00aa40 con 0x7fafc409e410 2023-09-30T22:31:24.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.783+0000 7fafb2ffd700 1 -- 172.21.15.16:0/2217387433 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafbc00abc0 con 0x7fafc409e410 2023-09-30T22:31:24.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.783+0000 7fafc8f46700 1 -- 172.21.15.16:0/2217387433 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafc413b5f0 con 0x7fafc409e410 2023-09-30T22:31:24.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.783+0000 7fafb2ffd700 1 -- 172.21.15.16:0/2217387433 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafbc0070d0 con 0x7fafc409e410 2023-09-30T22:31:24.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.783+0000 7fafc8f46700 1 -- 172.21.15.16:0/2217387433 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafc41324e0 con 0x7fafc409e410 2023-09-30T22:31:24.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.784+0000 7fafc8f46700 1 -- 172.21.15.16:0/2217387433 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafc4130e20 con 0x7fafc409e410 2023-09-30T22:31:24.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.785+0000 7fafb2ffd700 1 -- 172.21.15.16:0/2217387433 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fafbc007230 con 0x7fafc409e410 2023-09-30T22:31:24.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.786+0000 7fafb2ffd700 1 -- 172.21.15.16:0/2217387433 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fafbc0104c0 con 0x7fafc409e410 2023-09-30T22:31:24.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.794+0000 7fafb2ffd700 1 -- 172.21.15.16:0/2217387433 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fafbc01a8b0 con 0x7fafc409e410 2023-09-30T22:31:24.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.797+0000 7f868949c700 1 Processor -- start 2023-09-30T22:31:24.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.797+0000 7f868949c700 1 -- start start 2023-09-30T22:31:24.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.797+0000 7f868949c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f868409d6f0 0x7f868409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:24.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.797+0000 7f868949c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f868409e410 0x7f8684130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:24.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.797+0000 7f868949c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f868409f130 con 0x7f868409e000 2023-09-30T22:31:24.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.797+0000 7f868949c700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f868409f520 con 0x7f868409d6f0 2023-09-30T22:31:24.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.797+0000 7f868949c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f868409e930 con 0x7f868409e410 2023-09-30T22:31:24.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.797+0000 7f86837fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f868409e410 0x7f8684130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:24.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.797+0000 7f86837fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f868409e410 0x7f8684130d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:38758/0 (socket says 172.21.15.16:38758) 2023-09-30T22:31:24.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.797+0000 7f86837fe700 1 -- 172.21.15.16:0/4137971985 learned_addr learned my addr 172.21.15.16:0/4137971985 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:24.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.797+0000 7f86827fc700 1 --1- 172.21.15.16:0/4137971985 >> v1:172.21.15.16:6789/0 conn(0x7f868409e000 0x7f8684130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58102/0 (socket says 172.21.15.16:58102) 2023-09-30T22:31:24.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.798+0000 7f8682ffd700 1 --2- 172.21.15.16:0/4137971985 >> v2:172.21.15.16:3300/0 conn(0x7f868409d6f0 0x7f868409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:24.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.798+0000 7f86837fe700 1 -- 172.21.15.16:0/4137971985 >> v2:172.21.15.16:3300/0 conn(0x7f868409d6f0 msgr2=0x7f868409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:24.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.798+0000 7f86837fe700 1 --2- 172.21.15.16:0/4137971985 >> v2:172.21.15.16:3300/0 conn(0x7f868409d6f0 0x7f868409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:24.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.798+0000 7f86837fe700 1 -- 172.21.15.16:0/4137971985 >> v1:172.21.15.16:6789/0 conn(0x7f868409e000 legacy=0x7f8684130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:24.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.798+0000 7f86837fe700 1 -- 172.21.15.16:0/4137971985 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f868409b2d0 con 0x7f868409e410 2023-09-30T22:31:24.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.798+0000 7f86837fe700 1 --2- 172.21.15.16:0/4137971985 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f868409e410 0x7f8684130d70 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f867c00d150 tx=0x7f867c00d460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea302fd4da6b94ed server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:24.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.798+0000 7f8681ffb700 1 -- 172.21.15.16:0/4137971985 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c010070 con 0x7f868409e410 2023-09-30T22:31:24.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.798+0000 7f8681ffb700 1 -- 172.21.15.16:0/4137971985 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f867c00dd10 con 0x7f868409e410 2023-09-30T22:31:24.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.798+0000 7f8681ffb700 1 -- 172.21.15.16:0/4137971985 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c00b310 con 0x7f868409e410 2023-09-30T22:31:24.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.799+0000 7f868949c700 1 -- 172.21.15.16:0/4137971985 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f868409e410 msgr2=0x7f8684130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:24.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.799+0000 7f868949c700 1 --2- 172.21.15.16:0/4137971985 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f868409e410 0x7f8684130d70 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f867c00d150 tx=0x7f867c00d460 comp rx=0 tx=0).stop 2023-09-30T22:31:24.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.799+0000 7f868949c700 1 -- 172.21.15.16:0/4137971985 shutdown_connections 2023-09-30T22:31:24.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.799+0000 7f868949c700 1 --2- 172.21.15.16:0/4137971985 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f868409e410 0x7f8684130d70 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:24.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.799+0000 7f868949c700 1 --2- 172.21.15.16:0/4137971985 >> v2:172.21.15.16:3300/0 conn(0x7f868409d6f0 0x7f868409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:24.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.799+0000 7f868949c700 1 -- 172.21.15.16:0/4137971985 >> 172.21.15.16:0/4137971985 conn(0x7f8684066f50 msgr2=0x7f868409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:24.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.800+0000 7f868949c700 1 -- 172.21.15.16:0/4137971985 shutdown_connections 2023-09-30T22:31:24.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.800+0000 7f868949c700 1 -- 172.21.15.16:0/4137971985 wait complete. 2023-09-30T22:31:24.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.801+0000 7f868949c700 1 Processor -- start 2023-09-30T22:31:24.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.801+0000 7f868949c700 1 -- start start 2023-09-30T22:31:24.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.802+0000 7f868949c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f868409d6f0 0x7f868411e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:24.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.802+0000 7f868949c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f868409e000 0x7f868411e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:24.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.802+0000 7f868949c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f868409e410 0x7f868411eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:24.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.802+0000 7f868949c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8684120580 con 0x7f868409d6f0 2023-09-30T22:31:24.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.802+0000 7f868949c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8684121bc0 con 0x7f868409e000 2023-09-30T22:31:24.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.802+0000 7f868949c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8684120260 con 0x7f868409e410 2023-09-30T22:31:24.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.803+0000 7f86827fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f868409e000 0x7f868411e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:24.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.803+0000 7f86837fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f868409e410 0x7f868411eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:24.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.803+0000 7f86827fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f868409e000 0x7f868411e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:57574/0 (socket says 172.21.15.16:57574) 2023-09-30T22:31:24.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.803+0000 7f86827fc700 1 -- 172.21.15.16:0/1392187141 learned_addr learned my addr 172.21.15.16:0/1392187141 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:24.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.803+0000 7f8682ffd700 1 --2- 172.21.15.16:0/1392187141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f868409d6f0 0x7f868411e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:24.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.803+0000 7f86837fe700 1 -- 172.21.15.16:0/1392187141 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f868409e000 msgr2=0x7f868411e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:24.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.803+0000 7f86837fe700 1 --2- 172.21.15.16:0/1392187141 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f868409e000 0x7f868411e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:24.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.803+0000 7f86837fe700 1 -- 172.21.15.16:0/1392187141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f868409d6f0 msgr2=0x7f868411e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:24.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.803+0000 7f86837fe700 1 --2- 172.21.15.16:0/1392187141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f868409d6f0 0x7f868411e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:24.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.803+0000 7f86837fe700 1 -- 172.21.15.16:0/1392187141 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f867c00bd40 con 0x7f868409e410 2023-09-30T22:31:24.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.803+0000 7f8682ffd700 1 --2- 172.21.15.16:0/1392187141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f868409d6f0 0x7f868411e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:31:24.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.803+0000 7f86837fe700 1 --2- 172.21.15.16:0/1392187141 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f868409e410 0x7f868411eb20 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f867c017000 tx=0x7f867c004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:24.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.804+0000 7f866b7fe700 1 -- 172.21.15.16:0/1392187141 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c009810 con 0x7f868409e410 2023-09-30T22:31:24.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.804+0000 7f866b7fe700 1 -- 172.21.15.16:0/1392187141 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f867c0099b0 con 0x7f868409e410 2023-09-30T22:31:24.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.804+0000 7f866b7fe700 1 -- 172.21.15.16:0/1392187141 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c009d60 con 0x7f868409e410 2023-09-30T22:31:24.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.804+0000 7f868949c700 1 -- 172.21.15.16:0/1392187141 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8684072150 con 0x7f868409e410 2023-09-30T22:31:24.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.804+0000 7f868949c700 1 -- 172.21.15.16:0/1392187141 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8684121620 con 0x7f868409e410 2023-09-30T22:31:24.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.805+0000 7f868949c700 1 -- 172.21.15.16:0/1392187141 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f868411ac20 con 0x7f868409e410 2023-09-30T22:31:24.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.806+0000 7f866b7fe700 1 -- 172.21.15.16:0/1392187141 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f867c004040 con 0x7f868409e410 2023-09-30T22:31:24.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.809+0000 7f866b7fe700 1 -- 172.21.15.16:0/1392187141 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f867c043030 con 0x7f868409e410 2023-09-30T22:31:24.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.811+0000 7f80ae1cd700 1 Processor -- start 2023-09-30T22:31:24.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.811+0000 7f80ae1cd700 1 -- start start 2023-09-30T22:31:24.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.812+0000 7f80ae1cd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f80a8136710 0x7f80a8138af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:24.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.812+0000 7f80ae1cd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f80a8139030 0x7f80a813b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:24.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.812+0000 7f866b7fe700 1 -- 172.21.15.16:0/1392187141 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f867c013980 con 0x7f868409e410 2023-09-30T22:31:24.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.812+0000 7f80ae1cd700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80a806ef30 con 0x7f80a81338a0 2023-09-30T22:31:24.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.812+0000 7f80ae1cd700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80a806fef0 con 0x7f80a8139030 2023-09-30T22:31:24.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.812+0000 7f80ae1cd700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80a81334b0 con 0x7f80a8136710 2023-09-30T22:31:24.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.812+0000 7f80a7fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f80a8139030 0x7f80a813b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:24.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.812+0000 7f80a7fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f80a8139030 0x7f80a813b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57322/0 (socket says 172.21.15.16:57322) 2023-09-30T22:31:24.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.812+0000 7f80a7fff700 1 -- 172.21.15.16:0/1985424737 learned_addr learned my addr 172.21.15.16:0/1985424737 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:24.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.812+0000 7f80a6ffd700 1 --2- 172.21.15.16:0/1985424737 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f80a8136710 0x7f80a8138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:24.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.812+0000 7f80a6ffd700 1 -- 172.21.15.16:0/1985424737 >> v2:172.21.15.16:3300/0 conn(0x7f80a8139030 msgr2=0x7f80a813b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:24.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.812+0000 7f80a6ffd700 1 --2- 172.21.15.16:0/1985424737 >> v2:172.21.15.16:3300/0 conn(0x7f80a8139030 0x7f80a813b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:24.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.812+0000 7f80a6ffd700 1 -- 172.21.15.16:0/1985424737 >> v1:172.21.15.16:6789/0 conn(0x7f80a81338a0 legacy=0x7f80a8133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:24.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.812+0000 7f80a6ffd700 1 -- 172.21.15.16:0/1985424737 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a806fc30 con 0x7f80a8136710 2023-09-30T22:31:24.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.813+0000 7f80a6ffd700 1 --2- 172.21.15.16:0/1985424737 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f80a8136710 0x7f80a8138af0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f809c009840 tx=0x7f809c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e4cdb2d96614ca0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:24.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.814+0000 7f80a67fc700 1 -- 172.21.15.16:0/1985424737 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809c011830 con 0x7f80a8136710 2023-09-30T22:31:24.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.814+0000 7f80a67fc700 1 -- 172.21.15.16:0/1985424737 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f809c011990 con 0x7f80a8136710 2023-09-30T22:31:24.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.814+0000 7f80a67fc700 1 -- 172.21.15.16:0/1985424737 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f809c011d20 con 0x7f80a8136710 2023-09-30T22:31:24.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.815+0000 7f80ae1cd700 1 -- 172.21.15.16:0/1985424737 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f80a8136710 msgr2=0x7f80a8138af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:24.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.815+0000 7f80ae1cd700 1 --2- 172.21.15.16:0/1985424737 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f80a8136710 0x7f80a8138af0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f809c009840 tx=0x7f809c009b50 comp rx=0 tx=0).stop 2023-09-30T22:31:24.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.816+0000 7f80ae1cd700 1 -- 172.21.15.16:0/1985424737 shutdown_connections 2023-09-30T22:31:24.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.816+0000 7f80ae1cd700 1 --2- 172.21.15.16:0/1985424737 >> v2:172.21.15.16:3300/0 conn(0x7f80a8139030 0x7f80a813b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:24.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.816+0000 7f80ae1cd700 1 --2- 172.21.15.16:0/1985424737 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f80a8136710 0x7f80a8138af0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:24.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.816+0000 7f80ae1cd700 1 -- 172.21.15.16:0/1985424737 >> 172.21.15.16:0/1985424737 conn(0x7f80a80a2aa0 msgr2=0x7f80a80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:24.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.816+0000 7f80ae1cd700 1 -- 172.21.15.16:0/1985424737 shutdown_connections 2023-09-30T22:31:24.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.816+0000 7f80ae1cd700 1 -- 172.21.15.16:0/1985424737 wait complete. 2023-09-30T22:31:24.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.817+0000 7f80ae1cd700 1 Processor -- start 2023-09-30T22:31:24.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.817+0000 7f80ae1cd700 1 -- start start 2023-09-30T22:31:24.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.818+0000 7f80ae1cd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f80a81338a0 0x7f80a811adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:24.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.818+0000 7f80ae1cd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f80a8136710 0x7f80a811b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:24.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.818+0000 7f80ae1cd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80a8139030 0x7f80a811b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:24.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.818+0000 7f80ae1cd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a806f640 con 0x7f80a8139030 2023-09-30T22:31:24.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.818+0000 7f80ae1cd700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80a811eb20 con 0x7f80a81338a0 2023-09-30T22:31:24.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.818+0000 7f80ae1cd700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f80a8130cb0 con 0x7f80a8136710 2023-09-30T22:31:24.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.818+0000 7f80a77fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f80a81338a0 0x7f80a811adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:24.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.818+0000 7f80a6ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f80a8136710 0x7f80a811b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:24.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.819+0000 7f80a77fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f80a81338a0 0x7f80a811adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:57586/0 (socket says 172.21.15.16:57586) 2023-09-30T22:31:24.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.819+0000 7f80a77fe700 1 -- 172.21.15.16:0/1157109047 learned_addr learned my addr 172.21.15.16:0/1157109047 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:24.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.819+0000 7f80a7fff700 1 --2- 172.21.15.16:0/1157109047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80a8139030 0x7f80a811b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:24.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.819+0000 7f80a7fff700 1 -- 172.21.15.16:0/1157109047 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f80a8136710 msgr2=0x7f80a811b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:24.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.819+0000 7f80a7fff700 1 --2- 172.21.15.16:0/1157109047 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f80a8136710 0x7f80a811b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:24.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.819+0000 7f80a7fff700 1 -- 172.21.15.16:0/1157109047 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f80a81338a0 msgr2=0x7f80a811adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:24.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.819+0000 7f80a7fff700 1 --2- 172.21.15.16:0/1157109047 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f80a81338a0 0x7f80a811adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:24.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.819+0000 7f80a7fff700 1 -- 172.21.15.16:0/1157109047 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f809c0094a0 con 0x7f80a8139030 2023-09-30T22:31:24.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.819+0000 7f80a7fff700 1 --2- 172.21.15.16:0/1157109047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80a8139030 0x7f80a811b840 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f80a000ef90 tx=0x7f80a000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:24.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.819+0000 7f808ffff700 1 -- 172.21.15.16:0/1157109047 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a0003c20 con 0x7f80a8139030 2023-09-30T22:31:24.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.819+0000 7f808ffff700 1 -- 172.21.15.16:0/1157109047 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80a000f040 con 0x7f80a8139030 2023-09-30T22:31:24.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.819+0000 7f80ae1cd700 1 -- 172.21.15.16:0/1157109047 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80a811d230 con 0x7f80a8139030 2023-09-30T22:31:24.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.819+0000 7f808ffff700 1 -- 172.21.15.16:0/1157109047 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a0009e60 con 0x7f80a8139030 2023-09-30T22:31:24.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.819+0000 7f80ae1cd700 1 -- 172.21.15.16:0/1157109047 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80a811e070 con 0x7f80a8139030 2023-09-30T22:31:24.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.820+0000 7f808ffff700 1 -- 172.21.15.16:0/1157109047 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f80a000c8d0 con 0x7f80a8139030 2023-09-30T22:31:24.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.821+0000 7f808ffff700 1 -- 172.21.15.16:0/1157109047 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f80a003fa40 con 0x7f80a8139030 2023-09-30T22:31:24.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.822+0000 7f80ae1cd700 1 -- 172.21.15.16:0/1157109047 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80a811e2a0 con 0x7f80a8139030 2023-09-30T22:31:24.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.827+0000 7f808ffff700 1 -- 172.21.15.16:0/1157109047 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f80a811e2a0 con 0x7f80a8139030 2023-09-30T22:31:24.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.889+0000 7fbd0143d700 1 -- 172.21.15.16:0/2690105359 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fbcfc074ea0 con 0x7fbcfc09e930 2023-09-30T22:31:24.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.890+0000 7fbcf37fe700 1 -- 172.21.15.16:0/2690105359 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbcec02c030 con 0x7fbcfc09e930 2023-09-30T22:31:24.891 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:31:24.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.892+0000 7fbd0143d700 1 -- 172.21.15.16:0/2690105359 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbcfc09e930 msgr2=0x7fbcfc1a3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:24.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.892+0000 7fbd0143d700 1 --2- 172.21.15.16:0/2690105359 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbcfc09e930 0x7fbcfc1a3d80 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc0a0b00 tx=0x7fbcec004ad0 comp rx=0 tx=0).stop 2023-09-30T22:31:24.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.892+0000 7fbd0143d700 1 -- 172.21.15.16:0/2690105359 shutdown_connections 2023-09-30T22:31:24.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.892+0000 7fbd0143d700 1 --2- 172.21.15.16:0/2690105359 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbcfc09e930 0x7fbcfc1a3d80 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:24.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.892+0000 7fbd0143d700 1 --2- 172.21.15.16:0/2690105359 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbcfc09f650 0x7fbcfc1a4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:24.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.892+0000 7fbd0143d700 1 --2- 172.21.15.16:0/2690105359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbcfc09f240 0x7fbcfc1a42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:24.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.892+0000 7fbd0143d700 1 -- 172.21.15.16:0/2690105359 >> 172.21.15.16:0/2690105359 conn(0x7fbcfc0a4210 msgr2=0x7fbcfc1b8dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:24.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.892+0000 7fbd0143d700 1 -- 172.21.15.16:0/2690105359 shutdown_connections 2023-09-30T22:31:24.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:24.892+0000 7fbd0143d700 1 -- 172.21.15.16:0/2690105359 wait complete. 2023-09-30T22:31:24.909 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:31:25.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:25.022+0000 7f868949c700 1 -- 172.21.15.16:0/1392187141 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f8684074ea0 con 0x7f868409e410 2023-09-30T22:31:25.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:25.022+0000 7f866b7fe700 1 -- 172.21.15.16:0/1392187141 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f867c021030 con 0x7f868409e410 2023-09-30T22:31:25.024 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:31:25.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:25.023+0000 7fafc8f46700 1 -- 172.21.15.16:0/2217387433 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fafc4074ea0 con 0x7fafc409e410 2023-09-30T22:31:25.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:25.023+0000 7fafb2ffd700 1 -- 172.21.15.16:0/2217387433 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fafbc018020 con 0x7fafc409e410 2023-09-30T22:31:25.025 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:31:25.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:25.025+0000 7f868949c700 1 -- 172.21.15.16:0/1392187141 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f868409e410 msgr2=0x7f868411eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:25.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:25.025+0000 7f868949c700 1 --2- 172.21.15.16:0/1392187141 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f868409e410 0x7f868411eb20 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f867c017000 tx=0x7f867c004ad0 comp rx=0 tx=0).stop 2023-09-30T22:31:25.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:25.025+0000 7f868949c700 1 -- 172.21.15.16:0/1392187141 shutdown_connections 2023-09-30T22:31:25.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:25.025+0000 7f868949c700 1 --2- 172.21.15.16:0/1392187141 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f868409e410 0x7f868411eb20 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:25.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:25.025+0000 7f868949c700 1 --2- 172.21.15.16:0/1392187141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f868409d6f0 0x7f868411e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:25.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:25.025+0000 7f868949c700 1 --2- 172.21.15.16:0/1392187141 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f868409e000 0x7f868411e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:25.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:25.025+0000 7f868949c700 1 -- 172.21.15.16:0/1392187141 >> 172.21.15.16:0/1392187141 conn(0x7f8684066f50 msgr2=0x7f8684130a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:25.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:25.025+0000 7f868949c700 1 -- 172.21.15.16:0/1392187141 shutdown_connections 2023-09-30T22:31:25.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:25.025+0000 7f868949c700 1 -- 172.21.15.16:0/1392187141 wait complete. 2023-09-30T22:31:25.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:25.026+0000 7fafc8f46700 1 -- 172.21.15.16:0/2217387433 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fafc409e410 msgr2=0x7fafc413a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:25.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:25.026+0000 7fafc8f46700 1 --2- 172.21.15.16:0/2217387433 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fafc409e410 0x7fafc413a3c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fafbc00cbe0 tx=0x7fafbc00cef0 comp rx=0 tx=0).stop 2023-09-30T22:31:25.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:25.026+0000 7fafc8f46700 1 -- 172.21.15.16:0/2217387433 shutdown_connections 2023-09-30T22:31:25.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:25.026+0000 7fafc8f46700 1 --2- 172.21.15.16:0/2217387433 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fafc409e410 0x7fafc413a3c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:25.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:25.026+0000 7fafc8f46700 1 --2- 172.21.15.16:0/2217387433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafc409d6f0 0x7fafc4139940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:25.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:25.026+0000 7fafc8f46700 1 --2- 172.21.15.16:0/2217387433 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fafc409e000 0x7fafc4139e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:25.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:25.026+0000 7fafc8f46700 1 -- 172.21.15.16:0/2217387433 >> 172.21.15.16:0/2217387433 conn(0x7fafc4066f50 msgr2=0x7fafc4070e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:25.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:25.027+0000 7fafc8f46700 1 -- 172.21.15.16:0/2217387433 shutdown_connections 2023-09-30T22:31:25.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:25.027+0000 7fafc8f46700 1 -- 172.21.15.16:0/2217387433 wait complete. 2023-09-30T22:31:25.046 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:31:25.047 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:31:25.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:25.051+0000 7f80ae1cd700 1 -- 172.21.15.16:0/1157109047 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f80a811e460 con 0x7f80a8139030 2023-09-30T22:31:25.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:25.051+0000 7f808ffff700 1 -- 172.21.15.16:0/1157109047 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f80a811e460 con 0x7f80a8139030 2023-09-30T22:31:25.052 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:31:25.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:25.054+0000 7f80ae1cd700 1 -- 172.21.15.16:0/1157109047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80a8139030 msgr2=0x7f80a811b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:25.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:25.054+0000 7f80ae1cd700 1 --2- 172.21.15.16:0/1157109047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80a8139030 0x7f80a811b840 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f80a000ef90 tx=0x7f80a000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:31:25.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:25.054+0000 7f80ae1cd700 1 -- 172.21.15.16:0/1157109047 shutdown_connections 2023-09-30T22:31:25.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:25.054+0000 7f80ae1cd700 1 --2- 172.21.15.16:0/1157109047 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f80a81338a0 0x7f80a811adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:25.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:25.054+0000 7f80ae1cd700 1 --2- 172.21.15.16:0/1157109047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80a8139030 0x7f80a811b840 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:25.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:25.054+0000 7f80ae1cd700 1 --2- 172.21.15.16:0/1157109047 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f80a8136710 0x7f80a811b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:25.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:25.054+0000 7f80ae1cd700 1 -- 172.21.15.16:0/1157109047 >> 172.21.15.16:0/1157109047 conn(0x7f80a80a2aa0 msgr2=0x7f80a81344f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:25.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:25.054+0000 7f80ae1cd700 1 -- 172.21.15.16:0/1157109047 shutdown_connections 2023-09-30T22:31:25.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:25.054+0000 7f80ae1cd700 1 -- 172.21.15.16:0/1157109047 wait complete. 2023-09-30T22:31:25.068 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:31:25.909 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:26.047 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:26.048 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:26.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.056+0000 7f16bead7700 1 Processor -- start 2023-09-30T22:31:26.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.056+0000 7f16bead7700 1 -- start start 2023-09-30T22:31:26.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.057+0000 7f16bead7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f16b809d6f0 0x7f16b809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:26.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.057+0000 7f16bead7700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f16b809e410 0x7f16b8134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:26.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.057+0000 7f16bead7700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16b809b120 con 0x7f16b809e000 2023-09-30T22:31:26.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.057+0000 7f16bead7700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16b806b460 con 0x7f16b809e410 2023-09-30T22:31:26.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.057+0000 7f16bead7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16b809a7e0 con 0x7f16b809d6f0 2023-09-30T22:31:26.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.057+0000 7f16bc873700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f16b809d6f0 0x7f16b809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:26.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.057+0000 7f16bc873700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f16b809d6f0 0x7f16b809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:57602/0 (socket says 172.21.15.16:57602) 2023-09-30T22:31:26.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.057+0000 7f16bd074700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f16b809e410 0x7f16b8134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:26.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.057+0000 7f16bc873700 1 -- 172.21.15.16:0/1969529176 learned_addr learned my addr 172.21.15.16:0/1969529176 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:26.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.058+0000 7f16bc873700 1 -- 172.21.15.16:0/1969529176 >> v2:172.21.15.16:3300/0 conn(0x7f16b809e410 msgr2=0x7f16b8134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:26.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.058+0000 7f16bc873700 1 --2- 172.21.15.16:0/1969529176 >> v2:172.21.15.16:3300/0 conn(0x7f16b809e410 0x7f16b8134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.058+0000 7f16bc873700 1 -- 172.21.15.16:0/1969529176 >> v1:172.21.15.16:6789/0 conn(0x7f16b809e000 legacy=0x7f16b81339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:26.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.058+0000 7f16bc873700 1 -- 172.21.15.16:0/1969529176 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b809a500 con 0x7f16b809d6f0 2023-09-30T22:31:26.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.058+0000 7f16bc873700 1 --2- 172.21.15.16:0/1969529176 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f16b809d6f0 0x7f16b809dac0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f16a8009920 tx=0x7f16a8009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3eb80b244f1f4f7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:26.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.058+0000 7f16b77fe700 1 -- 172.21.15.16:0/1969529176 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a8011830 con 0x7f16b809d6f0 2023-09-30T22:31:26.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.058+0000 7f16b77fe700 1 -- 172.21.15.16:0/1969529176 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16a8011990 con 0x7f16b809d6f0 2023-09-30T22:31:26.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.058+0000 7f16b77fe700 1 -- 172.21.15.16:0/1969529176 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a8011d20 con 0x7f16b809d6f0 2023-09-30T22:31:26.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.059+0000 7f16bead7700 1 -- 172.21.15.16:0/1969529176 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f16b809d6f0 msgr2=0x7f16b809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:26.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.059+0000 7f16bead7700 1 --2- 172.21.15.16:0/1969529176 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f16b809d6f0 0x7f16b809dac0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f16a8009920 tx=0x7f16a8009c30 comp rx=0 tx=0).stop 2023-09-30T22:31:26.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.059+0000 7f16bead7700 1 -- 172.21.15.16:0/1969529176 shutdown_connections 2023-09-30T22:31:26.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.059+0000 7f16bead7700 1 --2- 172.21.15.16:0/1969529176 >> v2:172.21.15.16:3300/0 conn(0x7f16b809e410 0x7f16b8134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.059+0000 7f16bead7700 1 --2- 172.21.15.16:0/1969529176 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f16b809d6f0 0x7f16b809dac0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.059+0000 7f16bead7700 1 -- 172.21.15.16:0/1969529176 >> 172.21.15.16:0/1969529176 conn(0x7f16b8066f50 msgr2=0x7f16b809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:26.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.059+0000 7f16bead7700 1 -- 172.21.15.16:0/1969529176 shutdown_connections 2023-09-30T22:31:26.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.059+0000 7f16bead7700 1 -- 172.21.15.16:0/1969529176 wait complete. 2023-09-30T22:31:26.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.060+0000 7f16bead7700 1 Processor -- start 2023-09-30T22:31:26.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.060+0000 7f16bead7700 1 -- start start 2023-09-30T22:31:26.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.061+0000 7f16bead7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f16b809d6f0 0x7f16b81a8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:26.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.061+0000 7f16bead7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b809e000 0x7f16b81a85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:26.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.061+0000 7f16bead7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f16b809e410 0x7f16b81a8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:26.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.061+0000 7f16bead7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b81ab880 con 0x7f16b809e000 2023-09-30T22:31:26.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.061+0000 7f16bead7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16b809c710 con 0x7f16b809e410 2023-09-30T22:31:26.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.061+0000 7f16bead7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f16b81a9ba0 con 0x7f16b809d6f0 2023-09-30T22:31:26.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.061+0000 7f16bc873700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f16b809d6f0 0x7f16b81a8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:26.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.061+0000 7f16bc873700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f16b809d6f0 0x7f16b81a8070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:38796/0 (socket says 172.21.15.16:38796) 2023-09-30T22:31:26.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.061+0000 7f16bc873700 1 -- 172.21.15.16:0/233383070 learned_addr learned my addr 172.21.15.16:0/233383070 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:26.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.061+0000 7f16b7fff700 1 --2- 172.21.15.16:0/233383070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b809e000 0x7f16b81a85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:26.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.061+0000 7f16bc873700 1 -- 172.21.15.16:0/233383070 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f16b809e410 msgr2=0x7f16b81a8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:26.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.061+0000 7f16bc873700 1 --2- 172.21.15.16:0/233383070 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f16b809e410 0x7f16b81a8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.061+0000 7f16bc873700 1 -- 172.21.15.16:0/233383070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b809e000 msgr2=0x7f16b81a85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:26.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.061+0000 7f16bc873700 1 --2- 172.21.15.16:0/233383070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b809e000 0x7f16b81a85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.061+0000 7f16bc873700 1 -- 172.21.15.16:0/233383070 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a8009550 con 0x7f16b809d6f0 2023-09-30T22:31:26.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.062+0000 7f16b7fff700 1 --2- 172.21.15.16:0/233383070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b809e000 0x7f16b81a85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:31:26.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.062+0000 7f16bc873700 1 --2- 172.21.15.16:0/233383070 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f16b809d6f0 0x7f16b81a8070 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f16a8009c70 tx=0x7f16a8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:26.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.062+0000 7f16b57fa700 1 -- 172.21.15.16:0/233383070 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a8004380 con 0x7f16b809d6f0 2023-09-30T22:31:26.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.062+0000 7f16b57fa700 1 -- 172.21.15.16:0/233383070 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16a801f070 con 0x7f16b809d6f0 2023-09-30T22:31:26.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.062+0000 7f16b57fa700 1 -- 172.21.15.16:0/233383070 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a80036a0 con 0x7f16b809d6f0 2023-09-30T22:31:26.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.063+0000 7f16bead7700 1 -- 172.21.15.16:0/233383070 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16b81abae0 con 0x7f16b809d6f0 2023-09-30T22:31:26.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.063+0000 7f16bead7700 1 -- 172.21.15.16:0/233383070 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16b81aa290 con 0x7f16b809d6f0 2023-09-30T22:31:26.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.064+0000 7f16bead7700 1 -- 172.21.15.16:0/233383070 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16b80a3410 con 0x7f16b809d6f0 2023-09-30T22:31:26.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.064+0000 7f16b57fa700 1 -- 172.21.15.16:0/233383070 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f16a8021030 con 0x7f16b809d6f0 2023-09-30T22:31:26.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.064+0000 7f16b57fa700 1 -- 172.21.15.16:0/233383070 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f16a8040940 con 0x7f16b809d6f0 2023-09-30T22:31:26.069 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:26.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.069+0000 7f16b57fa700 1 -- 172.21.15.16:0/233383070 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f16a80199a0 con 0x7f16b809d6f0 2023-09-30T22:31:26.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.202+0000 7ff32d219700 1 Processor -- start 2023-09-30T22:31:26.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.202+0000 7ff32d219700 1 -- start start 2023-09-30T22:31:26.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.203+0000 7ff32d219700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff32809d6f0 0x7ff32809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:26.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.203+0000 7ff32d219700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff32809e000 0x7ff328130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:26.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.203+0000 7ff32d219700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff32809f130 con 0x7ff32809e410 2023-09-30T22:31:26.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.203+0000 7ff32d219700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff32809f520 con 0x7ff32809e000 2023-09-30T22:31:26.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.203+0000 7ff32d219700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff32809e930 con 0x7ff32809d6f0 2023-09-30T22:31:26.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.204+0000 7ff326d9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff32809d6f0 0x7ff32809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:26.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.204+0000 7ff326d9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff32809d6f0 0x7ff32809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:38812/0 (socket says 172.21.15.16:38812) 2023-09-30T22:31:26.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.204+0000 7ff326d9d700 1 -- 172.21.15.16:0/1863418049 learned_addr learned my addr 172.21.15.16:0/1863418049 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:26.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.204+0000 7ff326d9d700 1 -- 172.21.15.16:0/1863418049 >> v2:172.21.15.16:3300/0 conn(0x7ff32809e000 msgr2=0x7ff328130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:26.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.204+0000 7ff326d9d700 1 --2- 172.21.15.16:0/1863418049 >> v2:172.21.15.16:3300/0 conn(0x7ff32809e000 0x7ff328130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.204+0000 7ff326d9d700 1 -- 172.21.15.16:0/1863418049 >> v1:172.21.15.16:6789/0 conn(0x7ff32809e410 legacy=0x7ff328130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:26.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.204+0000 7ff326d9d700 1 -- 172.21.15.16:0/1863418049 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff32809b2d0 con 0x7ff32809d6f0 2023-09-30T22:31:26.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.204+0000 7ff326d9d700 1 --2- 172.21.15.16:0/1863418049 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff32809d6f0 0x7ff32809dac0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff31800bb90 tx=0x7ff31800bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d86a323166654549 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:26.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.204+0000 7ff325d9b700 1 -- 172.21.15.16:0/1863418049 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31800dd10 con 0x7ff32809d6f0 2023-09-30T22:31:26.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.204+0000 7ff325d9b700 1 -- 172.21.15.16:0/1863418049 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff31800de70 con 0x7ff32809d6f0 2023-09-30T22:31:26.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.204+0000 7ff325d9b700 1 -- 172.21.15.16:0/1863418049 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff318004230 con 0x7ff32809d6f0 2023-09-30T22:31:26.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.205+0000 7ff32d219700 1 -- 172.21.15.16:0/1863418049 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff32809d6f0 msgr2=0x7ff32809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:26.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.205+0000 7ff32d219700 1 --2- 172.21.15.16:0/1863418049 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff32809d6f0 0x7ff32809dac0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff31800bb90 tx=0x7ff31800bea0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.205+0000 7ff32d219700 1 -- 172.21.15.16:0/1863418049 shutdown_connections 2023-09-30T22:31:26.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.205+0000 7ff32d219700 1 --2- 172.21.15.16:0/1863418049 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff32809d6f0 0x7ff32809dac0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.205+0000 7ff32d219700 1 --2- 172.21.15.16:0/1863418049 >> v2:172.21.15.16:3300/0 conn(0x7ff32809e000 0x7ff328130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.205+0000 7ff32d219700 1 -- 172.21.15.16:0/1863418049 >> 172.21.15.16:0/1863418049 conn(0x7ff328066f50 msgr2=0x7ff32809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:26.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.205+0000 7ff32d219700 1 -- 172.21.15.16:0/1863418049 shutdown_connections 2023-09-30T22:31:26.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.205+0000 7ff32d219700 1 -- 172.21.15.16:0/1863418049 wait complete. 2023-09-30T22:31:26.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.207+0000 7ff32d219700 1 Processor -- start 2023-09-30T22:31:26.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.207+0000 7ff32d219700 1 -- start start 2023-09-30T22:31:26.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.208+0000 7ff32d219700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff32809d6f0 0x7ff3281224d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:26.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.208+0000 7ff32d219700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff32809e000 0x7ff328122a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:26.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.208+0000 7ff32d219700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff32809e410 0x7ff328122f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:26.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.208+0000 7ff32d219700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3281248d0 con 0x7ff32809e000 2023-09-30T22:31:26.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.208+0000 7ff32d219700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3281241f0 con 0x7ff32809e410 2023-09-30T22:31:26.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.208+0000 7ff32d219700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff32812ffc0 con 0x7ff32809d6f0 2023-09-30T22:31:26.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.208+0000 7ff32659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff32809e000 0x7ff328122a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:26.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.208+0000 7ff326d9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff32809d6f0 0x7ff3281224d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:26.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.208+0000 7ff32659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff32809e000 0x7ff328122a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57382/0 (socket says 172.21.15.16:57382) 2023-09-30T22:31:26.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.208+0000 7ff32759e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff32809e410 0x7ff328122f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:26.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.208+0000 7ff32659c700 1 -- 172.21.15.16:0/1546640710 learned_addr learned my addr 172.21.15.16:0/1546640710 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:26.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.209+0000 7ff32659c700 1 -- 172.21.15.16:0/1546640710 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff32809d6f0 msgr2=0x7ff3281224d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:26.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.209+0000 7ff32659c700 1 --2- 172.21.15.16:0/1546640710 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff32809d6f0 0x7ff3281224d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.209+0000 7ff32659c700 1 -- 172.21.15.16:0/1546640710 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff32809e410 msgr2=0x7ff328122f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:26.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.209+0000 7ff32659c700 1 --2- 172.21.15.16:0/1546640710 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff32809e410 0x7ff328122f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.209+0000 7ff32659c700 1 -- 172.21.15.16:0/1546640710 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff31800b7f0 con 0x7ff32809e000 2023-09-30T22:31:26.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.209+0000 7ff32759e700 1 --2- 172.21.15.16:0/1546640710 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff32809e410 0x7ff328122f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:26.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.209+0000 7ff32659c700 1 --2- 172.21.15.16:0/1546640710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff32809e000 0x7ff328122a10 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7ff31c00b500 tx=0x7ff31c00b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:26.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.209+0000 7ff30f7fe700 1 -- 172.21.15.16:0/1546640710 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31c011830 con 0x7ff32809e000 2023-09-30T22:31:26.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.209+0000 7ff32d219700 1 -- 172.21.15.16:0/1546640710 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff328125a50 con 0x7ff32809e000 2023-09-30T22:31:26.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.209+0000 7ff30f7fe700 1 -- 172.21.15.16:0/1546640710 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff31c0119b0 con 0x7ff32809e000 2023-09-30T22:31:26.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.209+0000 7ff30f7fe700 1 -- 172.21.15.16:0/1546640710 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31c011d60 con 0x7ff32809e000 2023-09-30T22:31:26.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.209+0000 7ff32d219700 1 -- 172.21.15.16:0/1546640710 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff328124630 con 0x7ff32809e000 2023-09-30T22:31:26.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.211+0000 7ff32d219700 1 -- 172.21.15.16:0/1546640710 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff328074ea0 con 0x7ff32809e000 2023-09-30T22:31:26.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.211+0000 7ff30f7fe700 1 -- 172.21.15.16:0/1546640710 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff31c01a400 con 0x7ff32809e000 2023-09-30T22:31:26.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.211+0000 7ff30f7fe700 1 -- 172.21.15.16:0/1546640710 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7ff31c03f600 con 0x7ff32809e000 2023-09-30T22:31:26.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.212+0000 7f8b02975700 1 Processor -- start 2023-09-30T22:31:26.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.212+0000 7f8b02975700 1 -- start start 2023-09-30T22:31:26.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.213+0000 7f8b02975700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8afc133540 0x7f8afc135920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:26.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.213+0000 7f8b02975700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8afc135e60 0x7f8afc138240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:26.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.213+0000 7f8b02975700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8afc0716a0 con 0x7f8afc130540 2023-09-30T22:31:26.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.213+0000 7f8b02975700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8afc071290 con 0x7f8afc135e60 2023-09-30T22:31:26.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.213+0000 7f8b02975700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8afc130340 con 0x7f8afc133540 2023-09-30T22:31:26.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.213+0000 7f8afb7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8afc133540 0x7f8afc135920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:26.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.213+0000 7f8afb7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8afc133540 0x7f8afc135920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:57626/0 (socket says 172.21.15.16:57626) 2023-09-30T22:31:26.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.213+0000 7f8afb7fe700 1 -- 172.21.15.16:0/1656959955 learned_addr learned my addr 172.21.15.16:0/1656959955 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:26.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.214+0000 7f8afb7fe700 1 -- 172.21.15.16:0/1656959955 >> v2:172.21.15.16:3300/0 conn(0x7f8afc135e60 msgr2=0x7f8afc138240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:26.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.214+0000 7f8afb7fe700 1 --2- 172.21.15.16:0/1656959955 >> v2:172.21.15.16:3300/0 conn(0x7f8afc135e60 0x7f8afc138240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.214+0000 7f8afb7fe700 1 -- 172.21.15.16:0/1656959955 >> v1:172.21.15.16:6789/0 conn(0x7f8afc130540 legacy=0x7f8afc130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:26.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.214+0000 7f8afb7fe700 1 -- 172.21.15.16:0/1656959955 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8afc070b80 con 0x7f8afc133540 2023-09-30T22:31:26.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.214+0000 7f8afb7fe700 1 --2- 172.21.15.16:0/1656959955 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8afc133540 0x7f8afc135920 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8af000bb90 tx=0x7f8af000bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92d4c04f4feaa7a3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:26.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.214+0000 7f8afaffd700 1 -- 172.21.15.16:0/1656959955 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af000dd10 con 0x7f8afc133540 2023-09-30T22:31:26.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.214+0000 7f8afaffd700 1 -- 172.21.15.16:0/1656959955 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8af000de70 con 0x7f8afc133540 2023-09-30T22:31:26.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.214+0000 7f8afaffd700 1 -- 172.21.15.16:0/1656959955 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8af0004230 con 0x7f8afc133540 2023-09-30T22:31:26.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.215+0000 7f8b02975700 1 -- 172.21.15.16:0/1656959955 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8afc133540 msgr2=0x7f8afc135920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:26.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.215+0000 7f8b02975700 1 --2- 172.21.15.16:0/1656959955 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8afc133540 0x7f8afc135920 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8af000bb90 tx=0x7f8af000bea0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.215+0000 7f8b02975700 1 -- 172.21.15.16:0/1656959955 shutdown_connections 2023-09-30T22:31:26.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.215+0000 7f8b02975700 1 --2- 172.21.15.16:0/1656959955 >> v2:172.21.15.16:3300/0 conn(0x7f8afc135e60 0x7f8afc138240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.215+0000 7f8b02975700 1 --2- 172.21.15.16:0/1656959955 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8afc133540 0x7f8afc135920 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.215+0000 7f8b02975700 1 -- 172.21.15.16:0/1656959955 >> 172.21.15.16:0/1656959955 conn(0x7f8afc0a2aa0 msgr2=0x7f8afc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:26.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.215+0000 7f8b02975700 1 -- 172.21.15.16:0/1656959955 shutdown_connections 2023-09-30T22:31:26.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.215+0000 7f8b02975700 1 -- 172.21.15.16:0/1656959955 wait complete. 2023-09-30T22:31:26.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.216+0000 7f8b02975700 1 Processor -- start 2023-09-30T22:31:26.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.217+0000 7f8b02975700 1 -- start start 2023-09-30T22:31:26.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.217+0000 7f8b02975700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8afc130540 0x7f8afc1c45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:26.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.217+0000 7f8b02975700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8afc133540 0x7f8afc1c4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:26.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.217+0000 7f8b02975700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8afc135e60 0x7f8afc09c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:26.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.217+0000 7f8b02975700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8afc1baf40 con 0x7f8afc135e60 2023-09-30T22:31:26.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.217+0000 7f8b02975700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8afc0718e0 con 0x7f8afc130540 2023-09-30T22:31:26.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.217+0000 7f8b02975700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8afc119710 con 0x7f8afc133540 2023-09-30T22:31:26.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.217+0000 7f8afbfff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8afc130540 0x7f8afc1c45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:26.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.218+0000 7f8afbfff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8afc130540 0x7f8afc1c45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:57638/0 (socket says 172.21.15.16:57638) 2023-09-30T22:31:26.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.218+0000 7f8afbfff700 1 -- 172.21.15.16:0/179461445 learned_addr learned my addr 172.21.15.16:0/179461445 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:26.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.218+0000 7f8afb7fe700 1 --2- 172.21.15.16:0/179461445 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8afc133540 0x7f8afc1c4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:26.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.218+0000 7f8b00f12700 1 --2- 172.21.15.16:0/179461445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8afc135e60 0x7f8afc09c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:26.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.218+0000 7f8afbfff700 1 -- 172.21.15.16:0/179461445 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8afc133540 msgr2=0x7f8afc1c4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:26.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.218+0000 7f8afbfff700 1 --2- 172.21.15.16:0/179461445 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8afc133540 0x7f8afc1c4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.218+0000 7f8afbfff700 1 -- 172.21.15.16:0/179461445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8afc135e60 msgr2=0x7f8afc09c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:26.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.218+0000 7f8afbfff700 1 --2- 172.21.15.16:0/179461445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8afc135e60 0x7f8afc09c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.218+0000 7f8afbfff700 1 -- 172.21.15.16:0/179461445 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8af000b7f0 con 0x7f8afc130540 2023-09-30T22:31:26.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.218+0000 7ff30f7fe700 1 -- 172.21.15.16:0/1546640710 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff31c00f8d0 con 0x7ff32809e000 2023-09-30T22:31:26.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.218+0000 7f8b00f12700 1 --2- 172.21.15.16:0/179461445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8afc135e60 0x7f8afc09c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:31:26.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.218+0000 7f8afbfff700 1 --2- 172.21.15.16:0/179461445 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8afc130540 0x7f8afc1c45c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8aec00c4d0 tx=0x7f8aec00c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:26.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.218+0000 7f8af8ff9700 1 -- 172.21.15.16:0/179461445 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aec0054e0 con 0x7f8afc130540 2023-09-30T22:31:26.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.219+0000 7f8af8ff9700 1 -- 172.21.15.16:0/179461445 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aec003680 con 0x7f8afc130540 2023-09-30T22:31:26.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.219+0000 7f8b02975700 1 -- 172.21.15.16:0/179461445 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8afc11a770 con 0x7f8afc130540 2023-09-30T22:31:26.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.219+0000 7f8af8ff9700 1 -- 172.21.15.16:0/179461445 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aec003a30 con 0x7f8afc130540 2023-09-30T22:31:26.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.219+0000 7f8b02975700 1 -- 172.21.15.16:0/179461445 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8afc0a23d0 con 0x7f8afc130540 2023-09-30T22:31:26.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.220+0000 7f8af8ff9700 1 -- 172.21.15.16:0/179461445 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8aec003b90 con 0x7f8afc130540 2023-09-30T22:31:26.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.220+0000 7f8af8ff9700 1 -- 172.21.15.16:0/179461445 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f8aec03e900 con 0x7f8afc130540 2023-09-30T22:31:26.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.220+0000 7f8b02975700 1 -- 172.21.15.16:0/179461445 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8afc0a23d0 con 0x7f8afc130540 2023-09-30T22:31:26.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.228+0000 7f3548ee6700 1 Processor -- start 2023-09-30T22:31:26.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.228+0000 7f8af8ff9700 1 -- 172.21.15.16:0/179461445 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8afc0a23d0 con 0x7f8afc130540 2023-09-30T22:31:26.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.228+0000 7f3548ee6700 1 -- start start 2023-09-30T22:31:26.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.229+0000 7f3548ee6700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f354409d6f0 0x7f354409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:26.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.229+0000 7f3548ee6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f354409e000 0x7f35441339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:26.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.229+0000 7f3548ee6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f354409e410 0x7f3544133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:26.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.229+0000 7f3548ee6700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f354406b460 con 0x7f354409d6f0 2023-09-30T22:31:26.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.229+0000 7f3548ee6700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f354409b120 con 0x7f354409e410 2023-09-30T22:31:26.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.229+0000 7f3548ee6700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f354409a7e0 con 0x7f354409e000 2023-09-30T22:31:26.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.229+0000 7f354259c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f354409d6f0 0x7f354409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:26.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.229+0000 7f354259c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f354409d6f0 0x7f354409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57416/0 (socket says 172.21.15.16:57416) 2023-09-30T22:31:26.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.229+0000 7f354259c700 1 -- 172.21.15.16:0/3966294232 learned_addr learned my addr 172.21.15.16:0/3966294232 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:26.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.229+0000 7f3542d9d700 1 --2- 172.21.15.16:0/3966294232 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f354409e410 0x7f3544133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:26.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.229+0000 7f3541d9b700 1 --2- 172.21.15.16:0/3966294232 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f354409e000 0x7f35441339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:26.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.229+0000 7f354259c700 1 -- 172.21.15.16:0/3966294232 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f354409e000 msgr2=0x7f35441339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:26.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.229+0000 7f354259c700 1 --2- 172.21.15.16:0/3966294232 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f354409e000 0x7f35441339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.229+0000 7f354259c700 1 -- 172.21.15.16:0/3966294232 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f354409e410 msgr2=0x7f3544133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:26.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.229+0000 7f354259c700 1 --2- 172.21.15.16:0/3966294232 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f354409e410 0x7f3544133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.229+0000 7f354259c700 1 -- 172.21.15.16:0/3966294232 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f354409a500 con 0x7f354409d6f0 2023-09-30T22:31:26.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.229+0000 7f3542d9d700 1 --2- 172.21.15.16:0/3966294232 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f354409e410 0x7f3544133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:26.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.230+0000 7f354259c700 1 --2- 172.21.15.16:0/3966294232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f354409d6f0 0x7f354409dac0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f35340098f0 tx=0x7f3534009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8a1b56e1e99ce9d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:26.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.230+0000 7f354159a700 1 -- 172.21.15.16:0/3966294232 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3534011830 con 0x7f354409d6f0 2023-09-30T22:31:26.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.230+0000 7f354159a700 1 -- 172.21.15.16:0/3966294232 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3534011990 con 0x7f354409d6f0 2023-09-30T22:31:26.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.230+0000 7f354159a700 1 -- 172.21.15.16:0/3966294232 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3534011d20 con 0x7f354409d6f0 2023-09-30T22:31:26.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.231+0000 7f3548ee6700 1 -- 172.21.15.16:0/3966294232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f354409d6f0 msgr2=0x7f354409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:26.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.231+0000 7f3548ee6700 1 --2- 172.21.15.16:0/3966294232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f354409d6f0 0x7f354409dac0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f35340098f0 tx=0x7f3534009c00 comp rx=0 tx=0).stop 2023-09-30T22:31:26.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.231+0000 7f3548ee6700 1 -- 172.21.15.16:0/3966294232 shutdown_connections 2023-09-30T22:31:26.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.231+0000 7f3548ee6700 1 --2- 172.21.15.16:0/3966294232 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f354409e410 0x7f3544133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.231+0000 7f3548ee6700 1 --2- 172.21.15.16:0/3966294232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f354409d6f0 0x7f354409dac0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.231+0000 7f3548ee6700 1 --2- 172.21.15.16:0/3966294232 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f354409e000 0x7f35441339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.231+0000 7f3548ee6700 1 -- 172.21.15.16:0/3966294232 >> 172.21.15.16:0/3966294232 conn(0x7f3544066f50 msgr2=0x7f354409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:26.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.231+0000 7f3548ee6700 1 -- 172.21.15.16:0/3966294232 shutdown_connections 2023-09-30T22:31:26.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.231+0000 7f3548ee6700 1 -- 172.21.15.16:0/3966294232 wait complete. 2023-09-30T22:31:26.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.233+0000 7f3548ee6700 1 Processor -- start 2023-09-30T22:31:26.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.233+0000 7f3548ee6700 1 -- start start 2023-09-30T22:31:26.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.234+0000 7f3548ee6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f354409d6f0 0x7f35441a5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:26.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.234+0000 7f3548ee6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f354409e000 0x7f35441a63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:26.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.234+0000 7f3548ee6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f354409e410 0x7f35441a6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:26.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.234+0000 7f3548ee6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35441a7f70 con 0x7f354409e000 2023-09-30T22:31:26.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.234+0000 7f3548ee6700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35441a9830 con 0x7f354409d6f0 2023-09-30T22:31:26.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.234+0000 7f3548ee6700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f35441a8fc0 con 0x7f354409e410 2023-09-30T22:31:26.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.234+0000 7f3541d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f354409e000 0x7f35441a63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:26.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.234+0000 7f3541d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f354409e000 0x7f35441a63f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57422/0 (socket says 172.21.15.16:57422) 2023-09-30T22:31:26.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.234+0000 7f3541d9b700 1 -- 172.21.15.16:0/1769195602 learned_addr learned my addr 172.21.15.16:0/1769195602 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:26.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.234+0000 7f3542d9d700 1 --2- 172.21.15.16:0/1769195602 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f354409e410 0x7f35441a6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:26.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.234+0000 7f354259c700 1 --2- 172.21.15.16:0/1769195602 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f354409d6f0 0x7f35441a5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:26.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.234+0000 7f3541d9b700 1 -- 172.21.15.16:0/1769195602 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f354409e410 msgr2=0x7f35441a6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:26.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.234+0000 7f3541d9b700 1 --2- 172.21.15.16:0/1769195602 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f354409e410 0x7f35441a6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.234+0000 7f3541d9b700 1 -- 172.21.15.16:0/1769195602 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f354409d6f0 msgr2=0x7f35441a5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:26.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.234+0000 7f3541d9b700 1 --2- 172.21.15.16:0/1769195602 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f354409d6f0 0x7f35441a5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.234+0000 7f3541d9b700 1 -- 172.21.15.16:0/1769195602 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f353c009a70 con 0x7f354409e000 2023-09-30T22:31:26.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.235+0000 7f3541d9b700 1 --2- 172.21.15.16:0/1769195602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f354409e000 0x7f35441a63f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f353800e9b0 tx=0x7f353800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:26.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.235+0000 7f3532ffd700 1 -- 172.21.15.16:0/1769195602 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3538009710 con 0x7f354409e000 2023-09-30T22:31:26.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.235+0000 7f3532ffd700 1 -- 172.21.15.16:0/1769195602 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f353800c850 con 0x7f354409e000 2023-09-30T22:31:26.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.235+0000 7f3548ee6700 1 -- 172.21.15.16:0/1769195602 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3534009550 con 0x7f354409e000 2023-09-30T22:31:26.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.235+0000 7f3532ffd700 1 -- 172.21.15.16:0/1769195602 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f353800cbc0 con 0x7f354409e000 2023-09-30T22:31:26.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.235+0000 7f3548ee6700 1 -- 172.21.15.16:0/1769195602 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f354409c710 con 0x7f354409e000 2023-09-30T22:31:26.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.236+0000 7f3532ffd700 1 -- 172.21.15.16:0/1769195602 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f353800cd20 con 0x7f354409e000 2023-09-30T22:31:26.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.236+0000 7f3532ffd700 1 -- 172.21.15.16:0/1769195602 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f353803fb10 con 0x7f354409e000 2023-09-30T22:31:26.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.236+0000 7f3548ee6700 1 -- 172.21.15.16:0/1769195602 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35441a93b0 con 0x7f354409e000 2023-09-30T22:31:26.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.241+0000 7f3532ffd700 1 -- 172.21.15.16:0/1769195602 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3538017080 con 0x7f354409e000 2023-09-30T22:31:26.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.317+0000 7f16bead7700 1 -- 172.21.15.16:0/233383070 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f16b8074ea0 con 0x7f16b809d6f0 2023-09-30T22:31:26.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.317+0000 7f16b57fa700 1 -- 172.21.15.16:0/233383070 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f16a8015030 con 0x7f16b809d6f0 2023-09-30T22:31:26.318 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:31:26.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.320+0000 7f16bead7700 1 -- 172.21.15.16:0/233383070 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f16b809d6f0 msgr2=0x7f16b81a8070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:26.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.320+0000 7f16bead7700 1 --2- 172.21.15.16:0/233383070 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f16b809d6f0 0x7f16b81a8070 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f16a8009c70 tx=0x7f16a8004750 comp rx=0 tx=0).stop 2023-09-30T22:31:26.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.320+0000 7f16bead7700 1 -- 172.21.15.16:0/233383070 shutdown_connections 2023-09-30T22:31:26.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.320+0000 7f16bead7700 1 --2- 172.21.15.16:0/233383070 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f16b809e410 0x7f16b81a8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.320+0000 7f16bead7700 1 --2- 172.21.15.16:0/233383070 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f16b809d6f0 0x7f16b81a8070 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.320+0000 7f16bead7700 1 --2- 172.21.15.16:0/233383070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b809e000 0x7f16b81a85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.320+0000 7f16bead7700 1 -- 172.21.15.16:0/233383070 >> 172.21.15.16:0/233383070 conn(0x7f16b8066f50 msgr2=0x7f16b806f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:26.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.320+0000 7f16bead7700 1 -- 172.21.15.16:0/233383070 shutdown_connections 2023-09-30T22:31:26.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.320+0000 7f16bead7700 1 -- 172.21.15.16:0/233383070 wait complete. 2023-09-30T22:31:26.335 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:31:26.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.449+0000 7f3548ee6700 1 -- 172.21.15.16:0/1769195602 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3544074ea0 con 0x7f354409e000 2023-09-30T22:31:26.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.449+0000 7f3532ffd700 1 -- 172.21.15.16:0/1769195602 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f353801b3b0 con 0x7f354409e000 2023-09-30T22:31:26.450 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:31:26.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.452+0000 7f3548ee6700 1 -- 172.21.15.16:0/1769195602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f354409e000 msgr2=0x7f35441a63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:26.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.452+0000 7f3548ee6700 1 --2- 172.21.15.16:0/1769195602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f354409e000 0x7f35441a63f0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f353800e9b0 tx=0x7f353800ed70 comp rx=0 tx=0).stop 2023-09-30T22:31:26.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.453+0000 7f3548ee6700 1 -- 172.21.15.16:0/1769195602 shutdown_connections 2023-09-30T22:31:26.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.453+0000 7f3548ee6700 1 --2- 172.21.15.16:0/1769195602 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f354409d6f0 0x7f35441a5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.453+0000 7f3548ee6700 1 --2- 172.21.15.16:0/1769195602 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f354409e410 0x7f35441a6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.453+0000 7f3548ee6700 1 --2- 172.21.15.16:0/1769195602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f354409e000 0x7f35441a63f0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.453+0000 7f3548ee6700 1 -- 172.21.15.16:0/1769195602 >> 172.21.15.16:0/1769195602 conn(0x7f3544066f50 msgr2=0x7f35440a2690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:26.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.453+0000 7f3548ee6700 1 -- 172.21.15.16:0/1769195602 shutdown_connections 2023-09-30T22:31:26.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.453+0000 7f3548ee6700 1 -- 172.21.15.16:0/1769195602 wait complete. 2023-09-30T22:31:26.472 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:31:26.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.495+0000 7ff32d219700 1 -- 172.21.15.16:0/1546640710 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff32806c360 con 0x7ff32809e000 2023-09-30T22:31:26.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.496+0000 7ff30f7fe700 1 -- 172.21.15.16:0/1546640710 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff31c015030 con 0x7ff32809e000 2023-09-30T22:31:26.496 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:31:26.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.498+0000 7ff32d219700 1 -- 172.21.15.16:0/1546640710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff32809e000 msgr2=0x7ff328122a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:26.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.498+0000 7ff32d219700 1 --2- 172.21.15.16:0/1546640710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff32809e000 0x7ff328122a10 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7ff31c00b500 tx=0x7ff31c00b810 comp rx=0 tx=0).stop 2023-09-30T22:31:26.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.498+0000 7ff32d219700 1 -- 172.21.15.16:0/1546640710 shutdown_connections 2023-09-30T22:31:26.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.498+0000 7ff32d219700 1 --2- 172.21.15.16:0/1546640710 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff32809e410 0x7ff328122f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.498+0000 7ff32d219700 1 --2- 172.21.15.16:0/1546640710 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff32809d6f0 0x7ff3281224d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.498+0000 7ff32d219700 1 --2- 172.21.15.16:0/1546640710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff32809e000 0x7ff328122a10 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.498+0000 7ff32d219700 1 -- 172.21.15.16:0/1546640710 >> 172.21.15.16:0/1546640710 conn(0x7ff328066f50 msgr2=0x7ff32806f140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:26.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.498+0000 7ff32d219700 1 -- 172.21.15.16:0/1546640710 shutdown_connections 2023-09-30T22:31:26.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.498+0000 7ff32d219700 1 -- 172.21.15.16:0/1546640710 wait complete. 2023-09-30T22:31:26.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.511+0000 7f8b02975700 1 -- 172.21.15.16:0/179461445 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f8afc0a23d0 con 0x7f8afc130540 2023-09-30T22:31:26.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.511+0000 7f8af8ff9700 1 -- 172.21.15.16:0/179461445 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8afc0a23d0 con 0x7f8afc130540 2023-09-30T22:31:26.512 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:31:26.512 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:31:26.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.514+0000 7f8b02975700 1 -- 172.21.15.16:0/179461445 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8afc130540 msgr2=0x7f8afc1c45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:26.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.514+0000 7f8b02975700 1 --2- 172.21.15.16:0/179461445 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8afc130540 0x7f8afc1c45c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8aec00c4d0 tx=0x7f8aec00c890 comp rx=0 tx=0).stop 2023-09-30T22:31:26.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.514+0000 7f8b02975700 1 -- 172.21.15.16:0/179461445 shutdown_connections 2023-09-30T22:31:26.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.514+0000 7f8b02975700 1 --2- 172.21.15.16:0/179461445 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8afc130540 0x7f8afc1c45c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.514+0000 7f8b02975700 1 --2- 172.21.15.16:0/179461445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8afc135e60 0x7f8afc09c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.514+0000 7f8b02975700 1 --2- 172.21.15.16:0/179461445 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8afc133540 0x7f8afc1c4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:26.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.514+0000 7f8b02975700 1 -- 172.21.15.16:0/179461445 >> 172.21.15.16:0/179461445 conn(0x7f8afc0a2aa0 msgr2=0x7f8afc131300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:26.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.514+0000 7f8b02975700 1 -- 172.21.15.16:0/179461445 shutdown_connections 2023-09-30T22:31:26.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:26.514+0000 7f8b02975700 1 -- 172.21.15.16:0/179461445 wait complete. 2023-09-30T22:31:26.528 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:31:27.335 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:27.472 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:27.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.481+0000 7f476fd32700 1 Processor -- start 2023-09-30T22:31:27.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.482+0000 7f476fd32700 1 -- start start 2023-09-30T22:31:27.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.482+0000 7f476fd32700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f47680a0a30 0x7f47680a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:27.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.482+0000 7f476fd32700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f476809d630 0x7f47680a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:27.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.482+0000 7f476fd32700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47680a2a70 con 0x7f4768133930 2023-09-30T22:31:27.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.482+0000 7f476fd32700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47680a01d0 con 0x7f476809d630 2023-09-30T22:31:27.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.482+0000 7f476fd32700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f476809e2b0 con 0x7f47680a0a30 2023-09-30T22:31:27.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.482+0000 7f476dace700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f47680a0a30 0x7f47680a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:27.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.482+0000 7f476d2cd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f476809d630 0x7f47680a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:27.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.482+0000 7f476e2cf700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4768133930 0x7f4768133dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:43980/0 (socket says 172.21.15.16:43980) 2023-09-30T22:31:27.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.482+0000 7f476dace700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f47680a0a30 0x7f47680a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:33596/0 (socket says 172.21.15.16:33596) 2023-09-30T22:31:27.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.482+0000 7f476e2cf700 1 -- 172.21.15.16:0/734450012 learned_addr learned my addr 172.21.15.16:0/734450012 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:31:27.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.482+0000 7f476d2cd700 1 --2- 172.21.15.16:0/734450012 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f476809d630 0x7f47680a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:53946/0 (socket says 172.21.15.16:53946) 2023-09-30T22:31:27.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.483+0000 7f476dace700 1 -- 172.21.15.16:0/734450012 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f476809d630 msgr2=0x7f47680a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:27.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.483+0000 7f476cacc700 1 -- 172.21.15.16:0/734450012 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 667087633 0 0) 0x7f47680a2a70 con 0x7f4768133930 2023-09-30T22:31:27.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.483+0000 7f476dace700 1 --2- 172.21.15.16:0/734450012 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f476809d630 0x7f47680a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:27.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.483+0000 7f476dace700 1 -- 172.21.15.16:0/734450012 >> v1:172.21.15.16:6789/0 conn(0x7f4768133930 legacy=0x7f4768133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:27.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.483+0000 7f476dace700 1 -- 172.21.15.16:0/734450012 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f476809dc50 con 0x7f47680a0a30 2023-09-30T22:31:27.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.483+0000 7f476d2cd700 1 --2- 172.21.15.16:0/734450012 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f476809d630 0x7f47680a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:27.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.483+0000 7f476dace700 1 --2- 172.21.15.16:0/734450012 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f47680a0a30 0x7f47680a0e00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4764009870 tx=0x7f4764009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7917f4fc4382abc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:27.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.483+0000 7f476cacc700 1 -- 172.21.15.16:0/734450012 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4764011830 con 0x7f47680a0a30 2023-09-30T22:31:27.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.483+0000 7f476cacc700 1 -- 172.21.15.16:0/734450012 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4764011990 con 0x7f47680a0a30 2023-09-30T22:31:27.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.483+0000 7f476cacc700 1 -- 172.21.15.16:0/734450012 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4764011d20 con 0x7f47680a0a30 2023-09-30T22:31:27.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.484+0000 7f476fd32700 1 -- 172.21.15.16:0/734450012 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f47680a0a30 msgr2=0x7f47680a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:27.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.484+0000 7f476fd32700 1 --2- 172.21.15.16:0/734450012 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f47680a0a30 0x7f47680a0e00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4764009870 tx=0x7f4764009b80 comp rx=0 tx=0).stop 2023-09-30T22:31:27.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.484+0000 7f476fd32700 1 -- 172.21.15.16:0/734450012 shutdown_connections 2023-09-30T22:31:27.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.484+0000 7f476fd32700 1 --2- 172.21.15.16:0/734450012 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f47680a0a30 0x7f47680a0e00 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:27.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.484+0000 7f476fd32700 1 --2- 172.21.15.16:0/734450012 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f476809d630 0x7f47680a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:27.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.484+0000 7f476fd32700 1 -- 172.21.15.16:0/734450012 >> 172.21.15.16:0/734450012 conn(0x7f47680a3050 msgr2=0x7f47680a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:27.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.484+0000 7f476fd32700 1 -- 172.21.15.16:0/734450012 shutdown_connections 2023-09-30T22:31:27.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.484+0000 7f476fd32700 1 -- 172.21.15.16:0/734450012 wait complete. 2023-09-30T22:31:27.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.486+0000 7f476fd32700 1 Processor -- start 2023-09-30T22:31:27.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.486+0000 7f476fd32700 1 -- start start 2023-09-30T22:31:27.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.486+0000 7f476fd32700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f476809d630 0x7f47681c7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:27.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.486+0000 7f476fd32700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47680a0a30 0x7f47681c8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:27.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.486+0000 7f476fd32700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4768133930 0x7f47681c8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:27.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.486+0000 7f476fd32700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47681221b0 con 0x7f47680a0a30 2023-09-30T22:31:27.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.486+0000 7f476fd32700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4768133520 con 0x7f4768133930 2023-09-30T22:31:27.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.486+0000 7f476fd32700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4768120700 con 0x7f476809d630 2023-09-30T22:31:27.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.487+0000 7f476dace700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f476809d630 0x7f47681c7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:27.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.487+0000 7f476dace700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f476809d630 0x7f47681c7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:33600/0 (socket says 172.21.15.16:33600) 2023-09-30T22:31:27.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.487+0000 7f476dace700 1 -- 172.21.15.16:0/424773765 learned_addr learned my addr 172.21.15.16:0/424773765 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:27.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.487+0000 7f476e2cf700 1 --2- 172.21.15.16:0/424773765 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4768133930 0x7f47681c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:27.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.487+0000 7f476dace700 1 -- 172.21.15.16:0/424773765 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4768133930 msgr2=0x7f47681c8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:27.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.487+0000 7f476dace700 1 --2- 172.21.15.16:0/424773765 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4768133930 0x7f47681c8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:27.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.487+0000 7f476dace700 1 -- 172.21.15.16:0/424773765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47680a0a30 msgr2=0x7f47681c8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:27.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.487+0000 7f476dace700 1 --2- 172.21.15.16:0/424773765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47680a0a30 0x7f47681c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:27.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.487+0000 7f476dace700 1 -- 172.21.15.16:0/424773765 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47580094a0 con 0x7f476809d630 2023-09-30T22:31:27.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.487+0000 7f476dace700 1 --2- 172.21.15.16:0/424773765 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f476809d630 0x7f47681c7f10 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4764009bc0 tx=0x7f4764004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:27.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.487+0000 7f475e7fc700 1 -- 172.21.15.16:0/424773765 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47640045d0 con 0x7f476809d630 2023-09-30T22:31:27.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.487+0000 7f475e7fc700 1 -- 172.21.15.16:0/424773765 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f476401f070 con 0x7f476809d630 2023-09-30T22:31:27.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.487+0000 7f476fd32700 1 -- 172.21.15.16:0/424773765 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47640094a0 con 0x7f476809d630 2023-09-30T22:31:27.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.487+0000 7f475e7fc700 1 -- 172.21.15.16:0/424773765 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47640038c0 con 0x7f476809d630 2023-09-30T22:31:27.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.487+0000 7f476fd32700 1 -- 172.21.15.16:0/424773765 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4768122c20 con 0x7f476809d630 2023-09-30T22:31:27.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.488+0000 7f475e7fc700 1 -- 172.21.15.16:0/424773765 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4764003a20 con 0x7f476809d630 2023-09-30T22:31:27.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.489+0000 7f475e7fc700 1 -- 172.21.15.16:0/424773765 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f4764021070 con 0x7f476809d630 2023-09-30T22:31:27.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.489+0000 7f476fd32700 1 -- 172.21.15.16:0/424773765 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47680747f0 con 0x7f476809d630 2023-09-30T22:31:27.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.494+0000 7f475e7fc700 1 -- 172.21.15.16:0/424773765 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4764015070 con 0x7f476809d630 2023-09-30T22:31:27.513 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:27.529 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:27.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.617+0000 7fe2616a1700 1 Processor -- start 2023-09-30T22:31:27.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.617+0000 7fe2616a1700 1 -- start start 2023-09-30T22:31:27.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.617+0000 7fe2616a1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe25c06a770 0x7fe25c06ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:27.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.617+0000 7fe2616a1700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe25c06b080 0x7fe25c131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:27.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.617+0000 7fe2616a1700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe25c09ece0 con 0x7fe25c131b40 2023-09-30T22:31:27.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.617+0000 7fe2616a1700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe25c09e760 con 0x7fe25c06b080 2023-09-30T22:31:27.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.617+0000 7fe2616a1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe25c09e040 con 0x7fe25c06a770 2023-09-30T22:31:27.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.617+0000 7fe25affd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe25c06a770 0x7fe25c06ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:27.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.618+0000 7fe25affd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe25c06a770 0x7fe25c06ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:33606/0 (socket says 172.21.15.16:33606) 2023-09-30T22:31:27.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.618+0000 7fe25affd700 1 -- 172.21.15.16:0/1452619747 learned_addr learned my addr 172.21.15.16:0/1452619747 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:27.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.618+0000 7fe25a7fc700 1 --2- 172.21.15.16:0/1452619747 >> v2:172.21.15.16:3300/0 conn(0x7fe25c06b080 0x7fe25c131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:27.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.618+0000 7fe259ffb700 1 -- 172.21.15.16:0/1452619747 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3975948006 0 0) 0x7fe25c09ece0 con 0x7fe25c131b40 2023-09-30T22:31:27.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.618+0000 7fe259ffb700 1 -- 172.21.15.16:0/1452619747 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2440035b0 con 0x7fe25c131b40 2023-09-30T22:31:27.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.618+0000 7fe25affd700 1 -- 172.21.15.16:0/1452619747 >> v2:172.21.15.16:3300/0 conn(0x7fe25c06b080 msgr2=0x7fe25c131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:27.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.618+0000 7fe25affd700 1 --2- 172.21.15.16:0/1452619747 >> v2:172.21.15.16:3300/0 conn(0x7fe25c06b080 0x7fe25c131600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:27.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.618+0000 7fe25affd700 1 -- 172.21.15.16:0/1452619747 >> v1:172.21.15.16:6789/0 conn(0x7fe25c131b40 legacy=0x7fe25c138850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:27.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.618+0000 7fe25affd700 1 -- 172.21.15.16:0/1452619747 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe25c09f630 con 0x7fe25c06a770 2023-09-30T22:31:27.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.618+0000 7fe25a7fc700 1 --2- 172.21.15.16:0/1452619747 >> v2:172.21.15.16:3300/0 conn(0x7fe25c06b080 0x7fe25c131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:27.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.618+0000 7fe25affd700 1 --2- 172.21.15.16:0/1452619747 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe25c06a770 0x7fe25c06ab40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe24c0098f0 tx=0x7fe24c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7492fb2ae93d010 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:27.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.619+0000 7fe259ffb700 1 -- 172.21.15.16:0/1452619747 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24c011830 con 0x7fe25c06a770 2023-09-30T22:31:27.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.619+0000 7fe259ffb700 1 -- 172.21.15.16:0/1452619747 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe24c011990 con 0x7fe25c06a770 2023-09-30T22:31:27.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.619+0000 7fe259ffb700 1 -- 172.21.15.16:0/1452619747 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24c011d00 con 0x7fe25c06a770 2023-09-30T22:31:27.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.620+0000 7fe2616a1700 1 -- 172.21.15.16:0/1452619747 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe25c06a770 msgr2=0x7fe25c06ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:27.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.620+0000 7fe2616a1700 1 --2- 172.21.15.16:0/1452619747 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe25c06a770 0x7fe25c06ab40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe24c0098f0 tx=0x7fe24c009c00 comp rx=0 tx=0).stop 2023-09-30T22:31:27.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.620+0000 7fe2616a1700 1 -- 172.21.15.16:0/1452619747 shutdown_connections 2023-09-30T22:31:27.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.620+0000 7fe2616a1700 1 --2- 172.21.15.16:0/1452619747 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe25c06a770 0x7fe25c06ab40 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:27.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.620+0000 7fe2616a1700 1 --2- 172.21.15.16:0/1452619747 >> v2:172.21.15.16:3300/0 conn(0x7fe25c06b080 0x7fe25c131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:27.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.620+0000 7fe2616a1700 1 -- 172.21.15.16:0/1452619747 >> 172.21.15.16:0/1452619747 conn(0x7fe25c066f50 msgr2=0x7fe25c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:27.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.620+0000 7fe2616a1700 1 -- 172.21.15.16:0/1452619747 shutdown_connections 2023-09-30T22:31:27.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.620+0000 7fe2616a1700 1 -- 172.21.15.16:0/1452619747 wait complete. 2023-09-30T22:31:27.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.622+0000 7fe2616a1700 1 Processor -- start 2023-09-30T22:31:27.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.622+0000 7fe2616a1700 1 -- start start 2023-09-30T22:31:27.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.622+0000 7fe2616a1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe25c06a770 0x7fe25c1c5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:27.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.622+0000 7fe2616a1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe25c06b080 0x7fe25c1c6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:27.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.622+0000 7fe2616a1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe25c131b40 0x7fe25c1c6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:27.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.622+0000 7fe2616a1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe25c1223b0 con 0x7fe25c06a770 2023-09-30T22:31:27.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.622+0000 7fe2616a1700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe25c1311b0 con 0x7fe25c131b40 2023-09-30T22:31:27.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.622+0000 7fe2616a1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe25c120990 con 0x7fe25c06b080 2023-09-30T22:31:27.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.623+0000 7fe25a7fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe25c06b080 0x7fe25c1c6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:27.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.623+0000 7fe25a7fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe25c06b080 0x7fe25c1c6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:33616/0 (socket says 172.21.15.16:33616) 2023-09-30T22:31:27.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.623+0000 7fe25a7fc700 1 -- 172.21.15.16:0/3537215044 learned_addr learned my addr 172.21.15.16:0/3537215044 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:27.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.623+0000 7fe25affd700 1 --2- 172.21.15.16:0/3537215044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe25c06a770 0x7fe25c1c5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:27.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.623+0000 7fe25a7fc700 1 -- 172.21.15.16:0/3537215044 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe25c131b40 msgr2=0x7fe25c1c6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:27.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.623+0000 7fe25a7fc700 1 --2- 172.21.15.16:0/3537215044 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe25c131b40 0x7fe25c1c6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:27.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.623+0000 7fe25a7fc700 1 -- 172.21.15.16:0/3537215044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe25c06a770 msgr2=0x7fe25c1c5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:27.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.623+0000 7fe25a7fc700 1 --2- 172.21.15.16:0/3537215044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe25c06a770 0x7fe25c1c5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:27.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.623+0000 7fe25a7fc700 1 -- 172.21.15.16:0/3537215044 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2500094a0 con 0x7fe25c06b080 2023-09-30T22:31:27.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.623+0000 7fe25affd700 1 --2- 172.21.15.16:0/3537215044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe25c06a770 0x7fe25c1c5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:27.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.623+0000 7fe25a7fc700 1 --2- 172.21.15.16:0/3537215044 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe25c06b080 0x7fe25c1c6300 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe25000ebb0 tx=0x7fe25000eec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:27.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.623+0000 7fe2437fe700 1 -- 172.21.15.16:0/3537215044 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe25000c850 con 0x7fe25c06b080 2023-09-30T22:31:27.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.623+0000 7fe2437fe700 1 -- 172.21.15.16:0/3537215044 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe25000c9b0 con 0x7fe25c06b080 2023-09-30T22:31:27.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.624+0000 7fe2616a1700 1 -- 172.21.15.16:0/3537215044 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe24c009550 con 0x7fe25c06b080 2023-09-30T22:31:27.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.624+0000 7fe2437fe700 1 -- 172.21.15.16:0/3537215044 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe25000cd40 con 0x7fe25c06b080 2023-09-30T22:31:27.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.624+0000 7fe2616a1700 1 -- 172.21.15.16:0/3537215044 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe25c071e60 con 0x7fe25c06b080 2023-09-30T22:31:27.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.625+0000 7fe2616a1700 1 -- 172.21.15.16:0/3537215044 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe25c074ea0 con 0x7fe25c06b080 2023-09-30T22:31:27.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.625+0000 7fe2437fe700 1 -- 172.21.15.16:0/3537215044 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe250013470 con 0x7fe25c06b080 2023-09-30T22:31:27.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.625+0000 7fe2437fe700 1 -- 172.21.15.16:0/3537215044 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fe250011350 con 0x7fe25c06b080 2023-09-30T22:31:27.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.634+0000 7fe2437fe700 1 -- 172.21.15.16:0/3537215044 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe2500102b0 con 0x7fe25c06b080 2023-09-30T22:31:27.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.659+0000 7f0551388700 1 Processor -- start 2023-09-30T22:31:27.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.660+0000 7f0551388700 1 -- start start 2023-09-30T22:31:27.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.660+0000 7f0551388700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f054c09e1d0 0x7f054c135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:27.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.660+0000 7f0551388700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f054c135e50 0x7f054c138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:27.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.660+0000 7f0551388700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f054c09f130 con 0x7f054c09d6f0 2023-09-30T22:31:27.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.660+0000 7f0551388700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f054c09f520 con 0x7f054c09e1d0 2023-09-30T22:31:27.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.660+0000 7f0551388700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f054c09e930 con 0x7f054c135e50 2023-09-30T22:31:27.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.661+0000 7f054a7fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f054c09e1d0 0x7f054c135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:27.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.661+0000 7f054affd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f054c09d6f0 0x7f054c09dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44000/0 (socket says 172.21.15.16:44000) 2023-09-30T22:31:27.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.661+0000 7f054affd700 1 -- 172.21.15.16:0/3686257377 learned_addr learned my addr 172.21.15.16:0/3686257377 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:31:27.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.661+0000 7f054b7fe700 1 --2- 172.21.15.16:0/3686257377 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f054c135e50 0x7f054c138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:27.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.661+0000 7f0549ffb700 1 -- 172.21.15.16:0/3686257377 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2233415337 0 0) 0x7f054c09f130 con 0x7f054c09d6f0 2023-09-30T22:31:27.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.661+0000 7f0549ffb700 1 -- 172.21.15.16:0/3686257377 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05340035b0 con 0x7f054c09d6f0 2023-09-30T22:31:27.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.661+0000 7f054b7fe700 1 -- 172.21.15.16:0/3686257377 >> v2:172.21.15.16:3300/0 conn(0x7f054c09e1d0 msgr2=0x7f054c135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:27.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.661+0000 7f054b7fe700 1 --2- 172.21.15.16:0/3686257377 >> v2:172.21.15.16:3300/0 conn(0x7f054c09e1d0 0x7f054c135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:27.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.661+0000 7f054b7fe700 1 -- 172.21.15.16:0/3686257377 >> v1:172.21.15.16:6789/0 conn(0x7f054c09d6f0 legacy=0x7f054c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:27.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.661+0000 7f054b7fe700 1 -- 172.21.15.16:0/3686257377 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f054c09b2d0 con 0x7f054c135e50 2023-09-30T22:31:27.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.661+0000 7f054b7fe700 1 --2- 172.21.15.16:0/3686257377 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f054c135e50 0x7f054c138230 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0544009d90 tx=0x7f054400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c01a0b20db439e4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:27.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.662+0000 7f0549ffb700 1 -- 172.21.15.16:0/3686257377 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054400e070 con 0x7f054c135e50 2023-09-30T22:31:27.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.662+0000 7f0549ffb700 1 -- 172.21.15.16:0/3686257377 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f054400bb20 con 0x7f054c135e50 2023-09-30T22:31:27.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.662+0000 7f0549ffb700 1 -- 172.21.15.16:0/3686257377 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054400be90 con 0x7f054c135e50 2023-09-30T22:31:27.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.663+0000 7f0551388700 1 -- 172.21.15.16:0/3686257377 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f054c135e50 msgr2=0x7f054c138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:27.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.663+0000 7f0551388700 1 --2- 172.21.15.16:0/3686257377 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f054c135e50 0x7f054c138230 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0544009d90 tx=0x7f054400b330 comp rx=0 tx=0).stop 2023-09-30T22:31:27.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.663+0000 7f0551388700 1 -- 172.21.15.16:0/3686257377 shutdown_connections 2023-09-30T22:31:27.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.663+0000 7f0551388700 1 --2- 172.21.15.16:0/3686257377 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f054c135e50 0x7f054c138230 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:27.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.663+0000 7f0551388700 1 --2- 172.21.15.16:0/3686257377 >> v2:172.21.15.16:3300/0 conn(0x7f054c09e1d0 0x7f054c135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:27.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.663+0000 7f0551388700 1 -- 172.21.15.16:0/3686257377 >> 172.21.15.16:0/3686257377 conn(0x7f054c066f50 msgr2=0x7f054c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:27.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.663+0000 7f0551388700 1 -- 172.21.15.16:0/3686257377 shutdown_connections 2023-09-30T22:31:27.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.663+0000 7f0551388700 1 -- 172.21.15.16:0/3686257377 wait complete. 2023-09-30T22:31:27.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.664+0000 7f0551388700 1 Processor -- start 2023-09-30T22:31:27.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.665+0000 7f0551388700 1 -- start start 2023-09-30T22:31:27.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.665+0000 7f0551388700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f054c09d6f0 0x7f054c11b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:27.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.665+0000 7f0551388700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f054c09e1d0 0x7f054c11bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:27.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.665+0000 7f0551388700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f054c135e50 0x7f054c11c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:27.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.665+0000 7f0551388700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054c11e8f0 con 0x7f054c09d6f0 2023-09-30T22:31:27.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.665+0000 7f0551388700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f054c11d9b0 con 0x7f054c135e50 2023-09-30T22:31:27.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.665+0000 7f0551388700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f054c09c700 con 0x7f054c09e1d0 2023-09-30T22:31:27.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.666+0000 7f054b7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f054c135e50 0x7f054c11c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:27.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.666+0000 7f054b7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f054c135e50 0x7f054c11c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:53986/0 (socket says 172.21.15.16:53986) 2023-09-30T22:31:27.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.666+0000 7f054b7fe700 1 -- 172.21.15.16:0/1664486875 learned_addr learned my addr 172.21.15.16:0/1664486875 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:27.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.666+0000 7f054affd700 1 --2- 172.21.15.16:0/1664486875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f054c09d6f0 0x7f054c11b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:27.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.666+0000 7f054a7fc700 1 --2- 172.21.15.16:0/1664486875 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f054c09e1d0 0x7f054c11bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:27.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.666+0000 7f054b7fe700 1 -- 172.21.15.16:0/1664486875 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f054c09e1d0 msgr2=0x7f054c11bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:27.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.666+0000 7f054b7fe700 1 --2- 172.21.15.16:0/1664486875 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f054c09e1d0 0x7f054c11bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:27.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.666+0000 7f054b7fe700 1 -- 172.21.15.16:0/1664486875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f054c09d6f0 msgr2=0x7f054c11b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:27.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.666+0000 7f054b7fe700 1 --2- 172.21.15.16:0/1664486875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f054c09d6f0 0x7f054c11b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:27.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.666+0000 7f054b7fe700 1 -- 172.21.15.16:0/1664486875 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0544009a70 con 0x7f054c135e50 2023-09-30T22:31:27.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.666+0000 7f054a7fc700 1 --2- 172.21.15.16:0/1664486875 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f054c09e1d0 0x7f054c11bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:31:27.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.666+0000 7f054affd700 1 --2- 172.21.15.16:0/1664486875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f054c09d6f0 0x7f054c11b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:31:27.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.666+0000 7f054b7fe700 1 --2- 172.21.15.16:0/1664486875 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f054c135e50 0x7f054c11c000 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f054400b370 tx=0x7f0544004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:27.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.666+0000 7f05337fe700 1 -- 172.21.15.16:0/1664486875 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0544004240 con 0x7f054c135e50 2023-09-30T22:31:27.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.666+0000 7f05337fe700 1 -- 172.21.15.16:0/1664486875 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0544009480 con 0x7f054c135e50 2023-09-30T22:31:27.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.667+0000 7f05337fe700 1 -- 172.21.15.16:0/1664486875 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05440045e0 con 0x7f054c135e50 2023-09-30T22:31:27.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.667+0000 7f0551388700 1 -- 172.21.15.16:0/1664486875 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f054c0a2020 con 0x7f054c135e50 2023-09-30T22:31:27.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.667+0000 7f0551388700 1 -- 172.21.15.16:0/1664486875 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f054c11e3e0 con 0x7f054c135e50 2023-09-30T22:31:27.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.668+0000 7f05337fe700 1 -- 172.21.15.16:0/1664486875 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0544026070 con 0x7f054c135e50 2023-09-30T22:31:27.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.668+0000 7f0551388700 1 -- 172.21.15.16:0/1664486875 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f054c11ea60 con 0x7f054c135e50 2023-09-30T22:31:27.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.668+0000 7f05337fe700 1 -- 172.21.15.16:0/1664486875 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f0544003e10 con 0x7f054c135e50 2023-09-30T22:31:27.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.677+0000 7f05337fe700 1 -- 172.21.15.16:0/1664486875 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f054c11ea60 con 0x7f054c135e50 2023-09-30T22:31:27.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.710+0000 7f12e656a700 1 Processor -- start 2023-09-30T22:31:27.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.711+0000 7f12e656a700 1 -- start start 2023-09-30T22:31:27.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.711+0000 7f12e656a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12e009de00 0x7f12e009e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:27.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.711+0000 7f12e656a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f12e013d1f0 0x7f12e013f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:27.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.711+0000 7f12e656a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12e009a810 con 0x7f12e006a770 2023-09-30T22:31:27.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.711+0000 7f12e656a700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12e006ab40 con 0x7f12e013d1f0 2023-09-30T22:31:27.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.711+0000 7f12e656a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f12e009ebc0 con 0x7f12e009de00 2023-09-30T22:31:27.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.711+0000 7f12df7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12e009de00 0x7f12e009e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:27.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.711+0000 7f12df7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12e009de00 0x7f12e009e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:33620/0 (socket says 172.21.15.16:33620) 2023-09-30T22:31:27.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.711+0000 7f12df7fe700 1 -- 172.21.15.16:0/4266707579 learned_addr learned my addr 172.21.15.16:0/4266707579 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:27.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.711+0000 7f12e4b07700 1 --2- 172.21.15.16:0/4266707579 >> v2:172.21.15.16:3300/0 conn(0x7f12e013d1f0 0x7f12e013f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:27.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.712+0000 7f12df7fe700 1 -- 172.21.15.16:0/4266707579 >> v2:172.21.15.16:3300/0 conn(0x7f12e013d1f0 msgr2=0x7f12e013f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:27.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.712+0000 7f12df7fe700 1 --2- 172.21.15.16:0/4266707579 >> v2:172.21.15.16:3300/0 conn(0x7f12e013d1f0 0x7f12e013f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:27.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.712+0000 7f12df7fe700 1 -- 172.21.15.16:0/4266707579 >> v1:172.21.15.16:6789/0 conn(0x7f12e006a770 legacy=0x7f12e009d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:27.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.712+0000 7f12df7fe700 1 -- 172.21.15.16:0/4266707579 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12e009e920 con 0x7f12e009de00 2023-09-30T22:31:27.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.712+0000 7f12e4b07700 1 --2- 172.21.15.16:0/4266707579 >> v2:172.21.15.16:3300/0 conn(0x7f12e013d1f0 0x7f12e013f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:27.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.712+0000 7f12df7fe700 1 --2- 172.21.15.16:0/4266707579 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12e009de00 0x7f12e009e250 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f12d40098f0 tx=0x7f12d4009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c758cb0e0faa39a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:27.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.712+0000 7f12deffd700 1 -- 172.21.15.16:0/4266707579 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d4011830 con 0x7f12e009de00 2023-09-30T22:31:27.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.712+0000 7f12deffd700 1 -- 172.21.15.16:0/4266707579 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12d4011990 con 0x7f12e009de00 2023-09-30T22:31:27.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.713+0000 7f12deffd700 1 -- 172.21.15.16:0/4266707579 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d4011d20 con 0x7f12e009de00 2023-09-30T22:31:27.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.713+0000 7f12e656a700 1 -- 172.21.15.16:0/4266707579 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12e009de00 msgr2=0x7f12e009e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:27.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.713+0000 7f12e656a700 1 --2- 172.21.15.16:0/4266707579 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12e009de00 0x7f12e009e250 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f12d40098f0 tx=0x7f12d4009c00 comp rx=0 tx=0).stop 2023-09-30T22:31:27.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.713+0000 7f12e656a700 1 -- 172.21.15.16:0/4266707579 shutdown_connections 2023-09-30T22:31:27.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.713+0000 7f12e656a700 1 --2- 172.21.15.16:0/4266707579 >> v2:172.21.15.16:3300/0 conn(0x7f12e013d1f0 0x7f12e013f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:27.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.713+0000 7f12e656a700 1 --2- 172.21.15.16:0/4266707579 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12e009de00 0x7f12e009e250 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:27.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.713+0000 7f12e656a700 1 -- 172.21.15.16:0/4266707579 >> 172.21.15.16:0/4266707579 conn(0x7f12e0066f50 msgr2=0x7f12e009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:27.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.713+0000 7f12e656a700 1 -- 172.21.15.16:0/4266707579 shutdown_connections 2023-09-30T22:31:27.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.713+0000 7f12e656a700 1 -- 172.21.15.16:0/4266707579 wait complete. 2023-09-30T22:31:27.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.715+0000 7f12e656a700 1 Processor -- start 2023-09-30T22:31:27.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.715+0000 7f12e656a700 1 -- start start 2023-09-30T22:31:27.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.716+0000 7f12e656a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12e006a770 0x7f12e011c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:27.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.716+0000 7f12e656a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12e009de00 0x7f12e013b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:27.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.716+0000 7f12e656a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12e013d1f0 0x7f12e013bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:27.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.716+0000 7f12e656a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e011efc0 con 0x7f12e009de00 2023-09-30T22:31:27.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.716+0000 7f12e656a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12e009f1a0 con 0x7f12e013d1f0 2023-09-30T22:31:27.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.716+0000 7f12e656a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f12e011cf60 con 0x7f12e006a770 2023-09-30T22:31:27.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.716+0000 7f12e4b07700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12e013d1f0 0x7f12e013bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:27.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.716+0000 7f12e4b07700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12e013d1f0 0x7f12e013bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54002/0 (socket says 172.21.15.16:54002) 2023-09-30T22:31:27.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.716+0000 7f12dffff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12e006a770 0x7f12e011c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:27.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.716+0000 7f12e4b07700 1 -- 172.21.15.16:0/1402409754 learned_addr learned my addr 172.21.15.16:0/1402409754 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:27.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.716+0000 7f12df7fe700 1 --2- 172.21.15.16:0/1402409754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12e009de00 0x7f12e013b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:27.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.716+0000 7f12e4b07700 1 -- 172.21.15.16:0/1402409754 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12e006a770 msgr2=0x7f12e011c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:27.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.717+0000 7f12e4b07700 1 --2- 172.21.15.16:0/1402409754 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12e006a770 0x7f12e011c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:27.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.717+0000 7f12e4b07700 1 -- 172.21.15.16:0/1402409754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12e009de00 msgr2=0x7f12e013b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:27.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.717+0000 7f12e4b07700 1 --2- 172.21.15.16:0/1402409754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12e009de00 0x7f12e013b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:27.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.717+0000 7f12e4b07700 1 -- 172.21.15.16:0/1402409754 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d8009a70 con 0x7f12e013d1f0 2023-09-30T22:31:27.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.717+0000 7f12df7fe700 1 --2- 172.21.15.16:0/1402409754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12e009de00 0x7f12e013b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:27.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.717+0000 7f12dffff700 1 --2- 172.21.15.16:0/1402409754 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12e006a770 0x7f12e011c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:27.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.717+0000 7f12e4b07700 1 --2- 172.21.15.16:0/1402409754 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12e013d1f0 0x7f12e013bb70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f12d8009f00 tx=0x7f12d800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:27.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.717+0000 7f12dcff9700 1 -- 172.21.15.16:0/1402409754 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d800cb50 con 0x7f12e013d1f0 2023-09-30T22:31:27.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.717+0000 7f12dcff9700 1 -- 172.21.15.16:0/1402409754 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12d800f040 con 0x7f12e013d1f0 2023-09-30T22:31:27.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.717+0000 7f12e656a700 1 -- 172.21.15.16:0/1402409754 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12d4009550 con 0x7f12e013d1f0 2023-09-30T22:31:27.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.717+0000 7f12dcff9700 1 -- 172.21.15.16:0/1402409754 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d8003a20 con 0x7f12e013d1f0 2023-09-30T22:31:27.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.717+0000 7f12e656a700 1 -- 172.21.15.16:0/1402409754 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12e011ea20 con 0x7f12e013d1f0 2023-09-30T22:31:27.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.718+0000 7f476fd32700 1 -- 172.21.15.16:0/424773765 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f47681223f0 con 0x7f476809d630 2023-09-30T22:31:27.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.718+0000 7f12e656a700 1 -- 172.21.15.16:0/1402409754 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12e011ec10 con 0x7f12e013d1f0 2023-09-30T22:31:27.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.719+0000 7f475e7fc700 1 -- 172.21.15.16:0/424773765 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f476402b030 con 0x7f476809d630 2023-09-30T22:31:27.722 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:31:27.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.719+0000 7f12dcff9700 1 -- 172.21.15.16:0/1402409754 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f12d8014480 con 0x7f12e013d1f0 2023-09-30T22:31:27.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.719+0000 7f12dcff9700 1 -- 172.21.15.16:0/1402409754 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f12d80407d0 con 0x7f12e013d1f0 2023-09-30T22:31:27.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.721+0000 7f476fd32700 1 -- 172.21.15.16:0/424773765 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f476809d630 msgr2=0x7f47681c7f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:27.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.721+0000 7f476fd32700 1 --2- 172.21.15.16:0/424773765 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f476809d630 0x7f47681c7f10 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4764009bc0 tx=0x7f4764004750 comp rx=0 tx=0).stop 2023-09-30T22:31:27.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.721+0000 7f476fd32700 1 -- 172.21.15.16:0/424773765 shutdown_connections 2023-09-30T22:31:27.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.721+0000 7f476fd32700 1 --2- 172.21.15.16:0/424773765 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4768133930 0x7f47681c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:27.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.721+0000 7f476fd32700 1 --2- 172.21.15.16:0/424773765 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f476809d630 0x7f47681c7f10 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:27.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.721+0000 7f476fd32700 1 --2- 172.21.15.16:0/424773765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f47680a0a30 0x7f47681c8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:27.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.721+0000 7f476fd32700 1 -- 172.21.15.16:0/424773765 >> 172.21.15.16:0/424773765 conn(0x7f47680a3050 msgr2=0x7f47681bb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:27.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.724+0000 7f476fd32700 1 -- 172.21.15.16:0/424773765 shutdown_connections 2023-09-30T22:31:27.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.724+0000 7f476fd32700 1 -- 172.21.15.16:0/424773765 wait complete. 2023-09-30T22:31:27.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.724+0000 7f12dcff9700 1 -- 172.21.15.16:0/1402409754 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f12d801b950 con 0x7f12e013d1f0 2023-09-30T22:31:27.744 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:31:27.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.849+0000 7fe2616a1700 1 -- 172.21.15.16:0/3537215044 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe25c123e80 con 0x7fe25c06b080 2023-09-30T22:31:27.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.850+0000 7fe2437fe700 1 -- 172.21.15.16:0/3537215044 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe250017070 con 0x7fe25c06b080 2023-09-30T22:31:27.851 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:31:27.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.852+0000 7fe2616a1700 1 -- 172.21.15.16:0/3537215044 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe25c06b080 msgr2=0x7fe25c1c6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:27.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.852+0000 7fe2616a1700 1 --2- 172.21.15.16:0/3537215044 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe25c06b080 0x7fe25c1c6300 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe25000ebb0 tx=0x7fe25000eec0 comp rx=0 tx=0).stop 2023-09-30T22:31:27.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.852+0000 7fe2616a1700 1 -- 172.21.15.16:0/3537215044 shutdown_connections 2023-09-30T22:31:27.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.852+0000 7fe2616a1700 1 --2- 172.21.15.16:0/3537215044 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe25c131b40 0x7fe25c1c6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:27.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.852+0000 7fe2616a1700 1 --2- 172.21.15.16:0/3537215044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe25c06a770 0x7fe25c1c5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:27.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.852+0000 7fe2616a1700 1 --2- 172.21.15.16:0/3537215044 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe25c06b080 0x7fe25c1c6300 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:27.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.852+0000 7fe2616a1700 1 -- 172.21.15.16:0/3537215044 >> 172.21.15.16:0/3537215044 conn(0x7fe25c066f50 msgr2=0x7fe25c070030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:27.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.852+0000 7fe2616a1700 1 -- 172.21.15.16:0/3537215044 shutdown_connections 2023-09-30T22:31:27.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.853+0000 7fe2616a1700 1 -- 172.21.15.16:0/3537215044 wait complete. 2023-09-30T22:31:27.867 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:31:27.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.897+0000 7f0551388700 1 -- 172.21.15.16:0/1664486875 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f054c11ea60 con 0x7f054c135e50 2023-09-30T22:31:27.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.897+0000 7f05337fe700 1 -- 172.21.15.16:0/1664486875 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f054c11ea60 con 0x7f054c135e50 2023-09-30T22:31:27.898 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:31:27.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.899+0000 7f0551388700 1 -- 172.21.15.16:0/1664486875 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f054c135e50 msgr2=0x7f054c11c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:27.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.899+0000 7f0551388700 1 --2- 172.21.15.16:0/1664486875 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f054c135e50 0x7f054c11c000 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f054400b370 tx=0x7f0544004ad0 comp rx=0 tx=0).stop 2023-09-30T22:31:27.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.900+0000 7f0551388700 1 -- 172.21.15.16:0/1664486875 shutdown_connections 2023-09-30T22:31:27.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.900+0000 7f0551388700 1 --2- 172.21.15.16:0/1664486875 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f054c135e50 0x7f054c11c000 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:27.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.900+0000 7f0551388700 1 --2- 172.21.15.16:0/1664486875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f054c09d6f0 0x7f054c11b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:27.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.900+0000 7f0551388700 1 --2- 172.21.15.16:0/1664486875 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f054c09e1d0 0x7f054c11bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:27.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.900+0000 7f0551388700 1 -- 172.21.15.16:0/1664486875 >> 172.21.15.16:0/1664486875 conn(0x7f054c066f50 msgr2=0x7f054c0a2360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:27.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.900+0000 7f0551388700 1 -- 172.21.15.16:0/1664486875 shutdown_connections 2023-09-30T22:31:27.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.900+0000 7f0551388700 1 -- 172.21.15.16:0/1664486875 wait complete. 2023-09-30T22:31:27.913 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:31:27.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.934+0000 7f12e656a700 1 -- 172.21.15.16:0/1402409754 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f12e0074ea0 con 0x7f12e013d1f0 2023-09-30T22:31:27.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.934+0000 7f12dcff9700 1 -- 172.21.15.16:0/1402409754 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f12d802b090 con 0x7f12e013d1f0 2023-09-30T22:31:27.935 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:31:27.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.936+0000 7f12e656a700 1 -- 172.21.15.16:0/1402409754 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12e013d1f0 msgr2=0x7f12e013bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:27.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.936+0000 7f12e656a700 1 --2- 172.21.15.16:0/1402409754 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12e013d1f0 0x7f12e013bb70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f12d8009f00 tx=0x7f12d800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:31:27.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.937+0000 7f12e656a700 1 -- 172.21.15.16:0/1402409754 shutdown_connections 2023-09-30T22:31:27.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.937+0000 7f12e656a700 1 --2- 172.21.15.16:0/1402409754 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12e013d1f0 0x7f12e013bb70 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:27.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.937+0000 7f12e656a700 1 --2- 172.21.15.16:0/1402409754 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12e006a770 0x7f12e011c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:27.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.937+0000 7f12e656a700 1 --2- 172.21.15.16:0/1402409754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12e009de00 0x7f12e013b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:27.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.937+0000 7f12e656a700 1 -- 172.21.15.16:0/1402409754 >> 172.21.15.16:0/1402409754 conn(0x7f12e0066f50 msgr2=0x7f12e009b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:27.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.937+0000 7f12e656a700 1 -- 172.21.15.16:0/1402409754 shutdown_connections 2023-09-30T22:31:27.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:27.937+0000 7f12e656a700 1 -- 172.21.15.16:0/1402409754 wait complete. 2023-09-30T22:31:27.951 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:31:28.746 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:28.868 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:28.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:28.892+0000 7f221a8b9700 1 Processor -- start 2023-09-30T22:31:28.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:28.892+0000 7f221a8b9700 1 -- start start 2023-09-30T22:31:28.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:28.893+0000 7f221a8b9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f221409e930 0x7f221409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:28.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:28.893+0000 7f221a8b9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f221409f240 0x7f22141317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:28.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:28.893+0000 7f221a8b9700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2214070940 con 0x7f221409f650 2023-09-30T22:31:28.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:28.893+0000 7f221a8b9700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f221406fd60 con 0x7f221409e930 2023-09-30T22:31:28.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:28.893+0000 7f221a8b9700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f221406fa80 con 0x7f221409f240 2023-09-30T22:31:28.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:28.893+0000 7f2218e56700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f221409f650 0x7f2214131d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44020/0 (socket says 172.21.15.16:44020) 2023-09-30T22:31:28.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:28.893+0000 7f2218e56700 1 -- 172.21.15.16:0/1721544406 learned_addr learned my addr 172.21.15.16:0/1721544406 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:31:28.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:28.893+0000 7f2213fff700 1 --2- 172.21.15.16:0/1721544406 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f221409e930 0x7f221409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:28.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:28.893+0000 7f22137fe700 1 --2- 172.21.15.16:0/1721544406 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f221409f240 0x7f22141317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:28.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:28.893+0000 7f2212ffd700 1 -- 172.21.15.16:0/1721544406 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3093404495 0 0) 0x7f2214070940 con 0x7f221409f650 2023-09-30T22:31:28.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:28.893+0000 7f22137fe700 1 -- 172.21.15.16:0/1721544406 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f221409e930 msgr2=0x7f221409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:28.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:28.893+0000 7f22137fe700 1 --2- 172.21.15.16:0/1721544406 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f221409e930 0x7f221409ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:28.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:28.893+0000 7f22137fe700 1 -- 172.21.15.16:0/1721544406 >> v1:172.21.15.16:6789/0 conn(0x7f221409f650 legacy=0x7f2214131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:28.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:28.893+0000 7f22137fe700 1 -- 172.21.15.16:0/1721544406 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f221409fb60 con 0x7f221409f240 2023-09-30T22:31:28.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:28.893+0000 7f2213fff700 1 --2- 172.21.15.16:0/1721544406 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f221409e930 0x7f221409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:28.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:28.894+0000 7f22137fe700 1 --2- 172.21.15.16:0/1721544406 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f221409f240 0x7f22141317f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2208009870 tx=0x7f2208009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3efea18a82455573 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:28.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:28.894+0000 7f2212ffd700 1 -- 172.21.15.16:0/1721544406 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2208011830 con 0x7f221409f240 2023-09-30T22:31:28.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:28.894+0000 7f2212ffd700 1 -- 172.21.15.16:0/1721544406 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2208011990 con 0x7f221409f240 2023-09-30T22:31:28.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:28.894+0000 7f2212ffd700 1 -- 172.21.15.16:0/1721544406 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2208011d20 con 0x7f221409f240 2023-09-30T22:31:28.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:28.895+0000 7f221a8b9700 1 -- 172.21.15.16:0/1721544406 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f221409f240 msgr2=0x7f22141317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:28.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:28.895+0000 7f221a8b9700 1 --2- 172.21.15.16:0/1721544406 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f221409f240 0x7f22141317f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2208009870 tx=0x7f2208009b80 comp rx=0 tx=0).stop 2023-09-30T22:31:28.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:28.895+0000 7f221a8b9700 1 -- 172.21.15.16:0/1721544406 shutdown_connections 2023-09-30T22:31:28.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:28.895+0000 7f221a8b9700 1 --2- 172.21.15.16:0/1721544406 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f221409e930 0x7f221409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:28.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:28.895+0000 7f221a8b9700 1 --2- 172.21.15.16:0/1721544406 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f221409f240 0x7f22141317f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:28.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:28.895+0000 7f221a8b9700 1 -- 172.21.15.16:0/1721544406 >> 172.21.15.16:0/1721544406 conn(0x7f22140a4210 msgr2=0x7f22140a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:28.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:28.895+0000 7f221a8b9700 1 -- 172.21.15.16:0/1721544406 shutdown_connections 2023-09-30T22:31:28.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:28.895+0000 7f221a8b9700 1 -- 172.21.15.16:0/1721544406 wait complete. 2023-09-30T22:31:28.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:28.896+0000 7f221a8b9700 1 Processor -- start 2023-09-30T22:31:28.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:28.896+0000 7f221a8b9700 1 -- start start 2023-09-30T22:31:28.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:28.896+0000 7f221a8b9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f221409e930 0x7f22141c8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:28.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:28.897+0000 7f221a8b9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f221409f240 0x7f22141c8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:28.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:28.897+0000 7f221a8b9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f221409f650 0x7f22141c8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:28.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:28.897+0000 7f221a8b9700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22141a6a60 con 0x7f221409f650 2023-09-30T22:31:28.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:28.897+0000 7f221a8b9700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22140a3260 con 0x7f221409f240 2023-09-30T22:31:28.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:28.897+0000 7f221a8b9700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f22141a65c0 con 0x7f221409e930 2023-09-30T22:31:28.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:28.897+0000 7f2218e56700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f221409f650 0x7f22141c8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:28.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:28.897+0000 7f2218e56700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f221409f650 0x7f22141c8bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53384/0 (socket says 172.21.15.16:53384) 2023-09-30T22:31:28.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:28.897+0000 7f2218e56700 1 -- 172.21.15.16:0/2174614681 learned_addr learned my addr 172.21.15.16:0/2174614681 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:28.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:28.897+0000 7f2213fff700 1 --2- 172.21.15.16:0/2174614681 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f221409e930 0x7f22141c8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:28.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:28.897+0000 7f22137fe700 1 --2- 172.21.15.16:0/2174614681 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f221409f240 0x7f22141c8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:28.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:28.897+0000 7f2213fff700 1 -- 172.21.15.16:0/2174614681 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f221409f240 msgr2=0x7f22141c8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:28.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:28.897+0000 7f2213fff700 1 --2- 172.21.15.16:0/2174614681 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f221409f240 0x7f22141c8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:28.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:28.897+0000 7f2213fff700 1 -- 172.21.15.16:0/2174614681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f221409f650 msgr2=0x7f22141c8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:28.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:28.897+0000 7f2213fff700 1 --2- 172.21.15.16:0/2174614681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f221409f650 0x7f22141c8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:28.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:28.897+0000 7f2213fff700 1 -- 172.21.15.16:0/2174614681 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22040094a0 con 0x7f221409e930 2023-09-30T22:31:28.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:28.897+0000 7f2213fff700 1 --2- 172.21.15.16:0/2174614681 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f221409e930 0x7f22141c8130 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f220400e9b0 tx=0x7f220400ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:28.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:28.898+0000 7f2210ff9700 1 -- 172.21.15.16:0/2174614681 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f220400ee50 con 0x7f221409e930 2023-09-30T22:31:28.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:28.898+0000 7f2210ff9700 1 -- 172.21.15.16:0/2174614681 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f220400c850 con 0x7f221409e930 2023-09-30T22:31:28.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:28.898+0000 7f221a8b9700 1 -- 172.21.15.16:0/2174614681 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22080094a0 con 0x7f221409e930 2023-09-30T22:31:28.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:28.898+0000 7f2210ff9700 1 -- 172.21.15.16:0/2174614681 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f220400cc00 con 0x7f221409e930 2023-09-30T22:31:28.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:28.898+0000 7f221a8b9700 1 -- 172.21.15.16:0/2174614681 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22140a0190 con 0x7f221409e930 2023-09-30T22:31:28.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:28.899+0000 7f2210ff9700 1 -- 172.21.15.16:0/2174614681 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2204013470 con 0x7f221409e930 2023-09-30T22:31:28.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:28.899+0000 7f2210ff9700 1 -- 172.21.15.16:0/2174614681 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f220403f760 con 0x7f221409e930 2023-09-30T22:31:28.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:28.899+0000 7f221a8b9700 1 -- 172.21.15.16:0/2174614681 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2214074ea0 con 0x7f221409e930 2023-09-30T22:31:28.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:28.904+0000 7f2210ff9700 1 -- 172.21.15.16:0/2174614681 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2204013720 con 0x7f221409e930 2023-09-30T22:31:28.914 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:28.952 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:29.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.019+0000 7f8ea8c65700 1 Processor -- start 2023-09-30T22:31:29.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.019+0000 7f8ea8c65700 1 -- start start 2023-09-30T22:31:29.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.020+0000 7f8ea8c65700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8ea409d6f0 0x7f8ea409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:29.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.020+0000 7f8ea8c65700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8ea409e000 0x7f8ea4130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:29.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.020+0000 7f8ea8c65700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8ea409e410 0x7f8ea4130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:29.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.020+0000 7f8ea8c65700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ea409f520 con 0x7f8ea409e410 2023-09-30T22:31:29.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.020+0000 7f8ea8c65700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea409f130 con 0x7f8ea409d6f0 2023-09-30T22:31:29.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.020+0000 7f8ea8c65700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea409e930 con 0x7f8ea409e000 2023-09-30T22:31:29.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.020+0000 7f8ea2d9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8ea409e410 0x7f8ea4130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:29.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.020+0000 7f8ea1d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8ea409e000 0x7f8ea4130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:29.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.020+0000 7f8ea2d9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8ea409e410 0x7f8ea4130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53400/0 (socket says 172.21.15.16:53400) 2023-09-30T22:31:29.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.020+0000 7f8ea1d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8ea409e000 0x7f8ea4130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:33640/0 (socket says 172.21.15.16:33640) 2023-09-30T22:31:29.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.020+0000 7f8ea2d9d700 1 -- 172.21.15.16:0/3290996555 learned_addr learned my addr 172.21.15.16:0/3290996555 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:29.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.021+0000 7f8ea1d9b700 1 -- 172.21.15.16:0/3290996555 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8ea409d6f0 msgr2=0x7f8ea409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:29.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.021+0000 7f8ea1d9b700 1 --2- 172.21.15.16:0/3290996555 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8ea409d6f0 0x7f8ea409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:29.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.021+0000 7f8ea1d9b700 1 -- 172.21.15.16:0/3290996555 >> v2:172.21.15.16:3300/0 conn(0x7f8ea409e410 msgr2=0x7f8ea4130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:29.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.021+0000 7f8ea1d9b700 1 --2- 172.21.15.16:0/3290996555 >> v2:172.21.15.16:3300/0 conn(0x7f8ea409e410 0x7f8ea4130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:29.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.021+0000 7f8ea1d9b700 1 -- 172.21.15.16:0/3290996555 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea409b2d0 con 0x7f8ea409e000 2023-09-30T22:31:29.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.021+0000 7f8ea2d9d700 1 --2- 172.21.15.16:0/3290996555 >> v2:172.21.15.16:3300/0 conn(0x7f8ea409e410 0x7f8ea4130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:29.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.021+0000 7f8ea1d9b700 1 --2- 172.21.15.16:0/3290996555 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8ea409e000 0x7f8ea4130660 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8e9800bb10 tx=0x7f8e9800be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2ccdb74e387c7f3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:29.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.021+0000 7f8ea159a700 1 -- 172.21.15.16:0/3290996555 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9800dc30 con 0x7f8ea409e000 2023-09-30T22:31:29.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.021+0000 7f8ea159a700 1 -- 172.21.15.16:0/3290996555 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e9800dd90 con 0x7f8ea409e000 2023-09-30T22:31:29.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.021+0000 7f8ea159a700 1 -- 172.21.15.16:0/3290996555 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e98010690 con 0x7f8ea409e000 2023-09-30T22:31:29.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.022+0000 7f8ea8c65700 1 -- 172.21.15.16:0/3290996555 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8ea409e000 msgr2=0x7f8ea4130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:29.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.022+0000 7f8ea8c65700 1 --2- 172.21.15.16:0/3290996555 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8ea409e000 0x7f8ea4130660 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8e9800bb10 tx=0x7f8e9800be20 comp rx=0 tx=0).stop 2023-09-30T22:31:29.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.022+0000 7f8ea8c65700 1 -- 172.21.15.16:0/3290996555 shutdown_connections 2023-09-30T22:31:29.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.022+0000 7f8ea8c65700 1 --2- 172.21.15.16:0/3290996555 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8ea409d6f0 0x7f8ea409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:29.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.022+0000 7f8ea8c65700 1 --2- 172.21.15.16:0/3290996555 >> v2:172.21.15.16:3300/0 conn(0x7f8ea409e410 0x7f8ea4130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:29.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.022+0000 7f8ea8c65700 1 --2- 172.21.15.16:0/3290996555 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8ea409e000 0x7f8ea4130660 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:29.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.022+0000 7f8ea8c65700 1 -- 172.21.15.16:0/3290996555 >> 172.21.15.16:0/3290996555 conn(0x7f8ea4066f50 msgr2=0x7f8ea409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:29.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.022+0000 7f8ea8c65700 1 -- 172.21.15.16:0/3290996555 shutdown_connections 2023-09-30T22:31:29.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.022+0000 7f8ea8c65700 1 -- 172.21.15.16:0/3290996555 wait complete. 2023-09-30T22:31:29.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.023+0000 7f8ea8c65700 1 Processor -- start 2023-09-30T22:31:29.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.024+0000 7f8ea8c65700 1 -- start start 2023-09-30T22:31:29.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.024+0000 7f8ea8c65700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8ea409d6f0 0x7f8ea411e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:29.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.024+0000 7f8ea8c65700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8ea409e000 0x7f8ea411e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:29.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.024+0000 7f8ea8c65700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ea409e410 0x7f8ea411eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:29.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.024+0000 7f8ea8c65700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea41202c0 con 0x7f8ea409e410 2023-09-30T22:31:29.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.024+0000 7f8ea8c65700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea40a1bc0 con 0x7f8ea409e000 2023-09-30T22:31:29.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.024+0000 7f8ea8c65700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea409c700 con 0x7f8ea409d6f0 2023-09-30T22:31:29.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.024+0000 7f8ea259c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8ea409d6f0 0x7f8ea411e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:29.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.024+0000 7f8ea2d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ea409e410 0x7f8ea411eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:29.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.024+0000 7f8ea259c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8ea409d6f0 0x7f8ea411e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:33642/0 (socket says 172.21.15.16:33642) 2023-09-30T22:31:29.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.024+0000 7f8ea2d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ea409e410 0x7f8ea411eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53416/0 (socket says 172.21.15.16:53416) 2023-09-30T22:31:29.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.024+0000 7f8ea259c700 1 -- 172.21.15.16:0/3757854707 learned_addr learned my addr 172.21.15.16:0/3757854707 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:29.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.025+0000 7f8ea2d9d700 1 -- 172.21.15.16:0/3757854707 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8ea409d6f0 msgr2=0x7f8ea411e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:29.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.025+0000 7f8ea2d9d700 1 --2- 172.21.15.16:0/3757854707 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8ea409d6f0 0x7f8ea411e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:29.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.025+0000 7f8ea2d9d700 1 -- 172.21.15.16:0/3757854707 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8ea409e000 msgr2=0x7f8ea411e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:29.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.025+0000 7f8ea2d9d700 1 --2- 172.21.15.16:0/3757854707 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8ea409e000 0x7f8ea411e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:29.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.025+0000 7f8ea2d9d700 1 -- 172.21.15.16:0/3757854707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e9c009b30 con 0x7f8ea409e410 2023-09-30T22:31:29.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.025+0000 7f8ea259c700 1 --2- 172.21.15.16:0/3757854707 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8ea409d6f0 0x7f8ea411e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:29.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.025+0000 7f8ea2d9d700 1 --2- 172.21.15.16:0/3757854707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ea409e410 0x7f8ea411eb80 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c00efc0 tx=0x7f8e9c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:29.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.025+0000 7f8e92ffd700 1 -- 172.21.15.16:0/3757854707 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c018070 con 0x7f8ea409e410 2023-09-30T22:31:29.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.025+0000 7f8e92ffd700 1 -- 172.21.15.16:0/3757854707 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e9c00f040 con 0x7f8ea409e410 2023-09-30T22:31:29.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.025+0000 7f8e92ffd700 1 -- 172.21.15.16:0/3757854707 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c003c60 con 0x7f8ea409e410 2023-09-30T22:31:29.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.025+0000 7f8ea8c65700 1 -- 172.21.15.16:0/3757854707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e9800b740 con 0x7f8ea409e410 2023-09-30T22:31:29.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.025+0000 7f8ea8c65700 1 -- 172.21.15.16:0/3757854707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ea4121940 con 0x7f8ea409e410 2023-09-30T22:31:29.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.026+0000 7f8ea8c65700 1 -- 172.21.15.16:0/3757854707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ea4121590 con 0x7f8ea409e410 2023-09-30T22:31:29.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.026+0000 7f8e92ffd700 1 -- 172.21.15.16:0/3757854707 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8e9c00c930 con 0x7f8ea409e410 2023-09-30T22:31:29.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.031+0000 7f8e92ffd700 1 -- 172.21.15.16:0/3757854707 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f8e9c007c30 con 0x7f8ea409e410 2023-09-30T22:31:29.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.036+0000 7f8e92ffd700 1 -- 172.21.15.16:0/3757854707 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8e9c003dc0 con 0x7f8ea409e410 2023-09-30T22:31:29.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.060+0000 7fe47eb4d700 1 Processor -- start 2023-09-30T22:31:29.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.060+0000 7fe47eb4d700 1 -- start start 2023-09-30T22:31:29.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.061+0000 7fe47eb4d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe47806a770 0x7fe47809d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:29.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.061+0000 7fe47eb4d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe47809dc30 0x7fe47809e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:29.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.061+0000 7fe47eb4d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe47809a810 con 0x7fe47813c1e0 2023-09-30T22:31:29.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.061+0000 7fe47eb4d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe47806ab40 con 0x7fe47806a770 2023-09-30T22:31:29.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.061+0000 7fe47eb4d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe47809ebc0 con 0x7fe47809dc30 2023-09-30T22:31:29.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.061+0000 7fe47c8e9700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe47806a770 0x7fe47809d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:29.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.061+0000 7fe47c8e9700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe47806a770 0x7fe47809d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53432/0 (socket says 172.21.15.16:53432) 2023-09-30T22:31:29.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.061+0000 7fe47c8e9700 1 -- 172.21.15.16:0/1983369856 learned_addr learned my addr 172.21.15.16:0/1983369856 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:29.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.061+0000 7fe477fff700 1 --2- 172.21.15.16:0/1983369856 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe47809dc30 0x7fe47809e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:29.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.061+0000 7fe4777fe700 1 -- 172.21.15.16:0/1983369856 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4108600395 0 0) 0x7fe47809a810 con 0x7fe47813c1e0 2023-09-30T22:31:29.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.061+0000 7fe4777fe700 1 -- 172.21.15.16:0/1983369856 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4680035b0 con 0x7fe47813c1e0 2023-09-30T22:31:29.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.062+0000 7fe477fff700 1 -- 172.21.15.16:0/1983369856 >> v2:172.21.15.16:3300/0 conn(0x7fe47806a770 msgr2=0x7fe47809d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:29.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.062+0000 7fe477fff700 1 --2- 172.21.15.16:0/1983369856 >> v2:172.21.15.16:3300/0 conn(0x7fe47806a770 0x7fe47809d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:29.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.062+0000 7fe477fff700 1 -- 172.21.15.16:0/1983369856 >> v1:172.21.15.16:6789/0 conn(0x7fe47813c1e0 legacy=0x7fe47813e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:29.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.062+0000 7fe477fff700 1 -- 172.21.15.16:0/1983369856 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe47809e920 con 0x7fe47809dc30 2023-09-30T22:31:29.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.062+0000 7fe477fff700 1 --2- 172.21.15.16:0/1983369856 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe47809dc30 0x7fe47809e080 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe464009840 tx=0x7fe464009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9eba484b34fd5b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:29.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.062+0000 7fe4777fe700 1 -- 172.21.15.16:0/1983369856 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe464011830 con 0x7fe47809dc30 2023-09-30T22:31:29.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.062+0000 7fe4777fe700 1 -- 172.21.15.16:0/1983369856 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe464011990 con 0x7fe47809dc30 2023-09-30T22:31:29.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.062+0000 7fe4777fe700 1 -- 172.21.15.16:0/1983369856 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe464011d20 con 0x7fe47809dc30 2023-09-30T22:31:29.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.063+0000 7fe47eb4d700 1 -- 172.21.15.16:0/1983369856 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe47809dc30 msgr2=0x7fe47809e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:29.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.063+0000 7fe47eb4d700 1 --2- 172.21.15.16:0/1983369856 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe47809dc30 0x7fe47809e080 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe464009840 tx=0x7fe464009b50 comp rx=0 tx=0).stop 2023-09-30T22:31:29.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.063+0000 7fe47eb4d700 1 -- 172.21.15.16:0/1983369856 shutdown_connections 2023-09-30T22:31:29.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.063+0000 7fe47eb4d700 1 --2- 172.21.15.16:0/1983369856 >> v2:172.21.15.16:3300/0 conn(0x7fe47806a770 0x7fe47809d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:29.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.063+0000 7fe47eb4d700 1 --2- 172.21.15.16:0/1983369856 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe47809dc30 0x7fe47809e080 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:29.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.063+0000 7fe47eb4d700 1 -- 172.21.15.16:0/1983369856 >> 172.21.15.16:0/1983369856 conn(0x7fe478066f50 msgr2=0x7fe47809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:29.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.063+0000 7fe47eb4d700 1 -- 172.21.15.16:0/1983369856 shutdown_connections 2023-09-30T22:31:29.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.063+0000 7fe47eb4d700 1 -- 172.21.15.16:0/1983369856 wait complete. 2023-09-30T22:31:29.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.065+0000 7fe47eb4d700 1 Processor -- start 2023-09-30T22:31:29.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.065+0000 7fe47eb4d700 1 -- start start 2023-09-30T22:31:29.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.065+0000 7fe47eb4d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe47806a770 0x7fe4781cc880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:29.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.065+0000 7fe47eb4d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe47809dc30 0x7fe4781ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:29.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.065+0000 7fe47eb4d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe47813c1e0 0x7fe4781270c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:29.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.065+0000 7fe47eb4d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe47813bc80 con 0x7fe47806a770 2023-09-30T22:31:29.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.065+0000 7fe47eb4d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4781c1360 con 0x7fe47809dc30 2023-09-30T22:31:29.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.065+0000 7fe47eb4d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe47812a640 con 0x7fe47813c1e0 2023-09-30T22:31:29.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.066+0000 7fe47d0ea700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe47813c1e0 0x7fe4781270c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:29.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.066+0000 7fe477fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe47809dc30 0x7fe4781ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:29.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.066+0000 7fe477fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe47809dc30 0x7fe4781ccdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54066/0 (socket says 172.21.15.16:54066) 2023-09-30T22:31:29.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.066+0000 7fe477fff700 1 -- 172.21.15.16:0/1210291746 learned_addr learned my addr 172.21.15.16:0/1210291746 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:29.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.066+0000 7fe47c8e9700 1 --2- 172.21.15.16:0/1210291746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe47806a770 0x7fe4781cc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:29.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.066+0000 7fe47d0ea700 1 -- 172.21.15.16:0/1210291746 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe47809dc30 msgr2=0x7fe4781ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:29.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.066+0000 7fe47d0ea700 1 --2- 172.21.15.16:0/1210291746 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe47809dc30 0x7fe4781ccdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:29.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.066+0000 7fe47d0ea700 1 -- 172.21.15.16:0/1210291746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe47806a770 msgr2=0x7fe4781cc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:29.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.066+0000 7fe47d0ea700 1 --2- 172.21.15.16:0/1210291746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe47806a770 0x7fe4781cc880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:29.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.066+0000 7fe47d0ea700 1 -- 172.21.15.16:0/1210291746 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4640094a0 con 0x7fe47813c1e0 2023-09-30T22:31:29.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.066+0000 7fe47c8e9700 1 --2- 172.21.15.16:0/1210291746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe47806a770 0x7fe4781cc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:29.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.066+0000 7fe47d0ea700 1 --2- 172.21.15.16:0/1210291746 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe47813c1e0 0x7fe4781270c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe47000cbc0 tx=0x7fe47000ced0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:29.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.066+0000 7fe4757fa700 1 -- 172.21.15.16:0/1210291746 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe470003c00 con 0x7fe47813c1e0 2023-09-30T22:31:29.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.067+0000 7fe4757fa700 1 -- 172.21.15.16:0/1210291746 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe470003d60 con 0x7fe47813c1e0 2023-09-30T22:31:29.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.067+0000 7fe47eb4d700 1 -- 172.21.15.16:0/1210291746 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4780a19f0 con 0x7fe47813c1e0 2023-09-30T22:31:29.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.067+0000 7fe4757fa700 1 -- 172.21.15.16:0/1210291746 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47000a9c0 con 0x7fe47813c1e0 2023-09-30T22:31:29.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.067+0000 7fe47eb4d700 1 -- 172.21.15.16:0/1210291746 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe47812b370 con 0x7fe47813c1e0 2023-09-30T22:31:29.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.068+0000 7fe4757fa700 1 -- 172.21.15.16:0/1210291746 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe47000ab20 con 0x7fe47813c1e0 2023-09-30T22:31:29.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.068+0000 7fe47eb4d700 1 -- 172.21.15.16:0/1210291746 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe478074ea0 con 0x7fe47813c1e0 2023-09-30T22:31:29.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.068+0000 7fe4757fa700 1 -- 172.21.15.16:0/1210291746 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fe47003f6c0 con 0x7fe47813c1e0 2023-09-30T22:31:29.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.073+0000 7fe4757fa700 1 -- 172.21.15.16:0/1210291746 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe47000add0 con 0x7fe47813c1e0 2023-09-30T22:31:29.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.111+0000 7fbaf57ed700 1 Processor -- start 2023-09-30T22:31:29.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.111+0000 7fbaf57ed700 1 -- start start 2023-09-30T22:31:29.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.112+0000 7fbaf57ed700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbaf00984f0 0x7fbaf00988c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:29.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.112+0000 7fbaf57ed700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbaf009d6f0 0x7fbaf009db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:29.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.112+0000 7fbaf57ed700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbaf009e040 0x7fbaf01385c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:29.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.112+0000 7fbaf57ed700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbaf0099620 con 0x7fbaf009e040 2023-09-30T22:31:29.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.112+0000 7fbaf57ed700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf009aca0 con 0x7fbaf00984f0 2023-09-30T22:31:29.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.112+0000 7fbaf57ed700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf009a9e0 con 0x7fbaf009d6f0 2023-09-30T22:31:29.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.112+0000 7fbaeeffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbaf00984f0 0x7fbaf00988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:29.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.112+0000 7fbaeeffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbaf00984f0 0x7fbaf00988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54082/0 (socket says 172.21.15.16:54082) 2023-09-30T22:31:29.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.112+0000 7fbaeeffd700 1 -- 172.21.15.16:0/2407665328 learned_addr learned my addr 172.21.15.16:0/2407665328 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:29.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.112+0000 7fbaef7fe700 1 --2- 172.21.15.16:0/2407665328 >> v2:172.21.15.16:3300/0 conn(0x7fbaf009e040 0x7fbaf01385c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:29.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.112+0000 7fbaef7fe700 1 -- 172.21.15.16:0/2407665328 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbaf009d6f0 msgr2=0x7fbaf009db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:29.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.112+0000 7fbaef7fe700 1 --2- 172.21.15.16:0/2407665328 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbaf009d6f0 0x7fbaf009db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:29.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.112+0000 7fbaef7fe700 1 -- 172.21.15.16:0/2407665328 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbaf00984f0 msgr2=0x7fbaf00988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:29.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.112+0000 7fbaef7fe700 1 --2- 172.21.15.16:0/2407665328 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbaf00984f0 0x7fbaf00988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:29.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.112+0000 7fbaef7fe700 1 -- 172.21.15.16:0/2407665328 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaf009bce0 con 0x7fbaf009e040 2023-09-30T22:31:29.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.113+0000 7fbaeeffd700 1 --2- 172.21.15.16:0/2407665328 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbaf00984f0 0x7fbaf00988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:29.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.113+0000 7fbaef7fe700 1 --2- 172.21.15.16:0/2407665328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaf009e040 0x7fbaf01385c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fbae8009d90 tx=0x7fbae800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b066e3b7a45a341 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:29.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.113+0000 7fbaedffb700 1 -- 172.21.15.16:0/2407665328 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae800e070 con 0x7fbaf009e040 2023-09-30T22:31:29.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.113+0000 7fbaedffb700 1 -- 172.21.15.16:0/2407665328 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbae800bb20 con 0x7fbaf009e040 2023-09-30T22:31:29.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.113+0000 7fbaedffb700 1 -- 172.21.15.16:0/2407665328 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae8012430 con 0x7fbaf009e040 2023-09-30T22:31:29.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.114+0000 7fbaf57ed700 1 -- 172.21.15.16:0/2407665328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaf009e040 msgr2=0x7fbaf01385c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:29.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.114+0000 7fbaf57ed700 1 --2- 172.21.15.16:0/2407665328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaf009e040 0x7fbaf01385c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fbae8009d90 tx=0x7fbae800b330 comp rx=0 tx=0).stop 2023-09-30T22:31:29.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.114+0000 7fbaf57ed700 1 -- 172.21.15.16:0/2407665328 shutdown_connections 2023-09-30T22:31:29.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.114+0000 7fbaf57ed700 1 --2- 172.21.15.16:0/2407665328 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbaf00984f0 0x7fbaf00988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:29.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.114+0000 7fbaf57ed700 1 --2- 172.21.15.16:0/2407665328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaf009e040 0x7fbaf01385c0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:29.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.114+0000 7fbaf57ed700 1 --2- 172.21.15.16:0/2407665328 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbaf009d6f0 0x7fbaf009db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:29.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.114+0000 7fbaf57ed700 1 -- 172.21.15.16:0/2407665328 >> 172.21.15.16:0/2407665328 conn(0x7fbaf009f470 msgr2=0x7fbaf00a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:29.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.114+0000 7fbaf57ed700 1 -- 172.21.15.16:0/2407665328 shutdown_connections 2023-09-30T22:31:29.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.114+0000 7fbaf57ed700 1 -- 172.21.15.16:0/2407665328 wait complete. 2023-09-30T22:31:29.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.116+0000 7fbaf57ed700 1 Processor -- start 2023-09-30T22:31:29.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.116+0000 7fbaf57ed700 1 -- start start 2023-09-30T22:31:29.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.117+0000 7fbaf57ed700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaf00984f0 0x7fbaf01c8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:29.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.117+0000 7fbaf57ed700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbaf009d6f0 0x7fbaf01c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:29.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.117+0000 7fbaf57ed700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbaf009e040 0x7fbaf01c90b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:29.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.117+0000 7fbaf57ed700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf006d960 con 0x7fbaf00984f0 2023-09-30T22:31:29.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.117+0000 7fbaf57ed700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf006eef0 con 0x7fbaf009d6f0 2023-09-30T22:31:29.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.117+0000 7fbaf57ed700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf006e1f0 con 0x7fbaf009e040 2023-09-30T22:31:29.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.117+0000 7fbaee7fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbaf009d6f0 0x7fbaf01c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:29.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.117+0000 7fbaee7fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbaf009d6f0 0x7fbaf01c8b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54098/0 (socket says 172.21.15.16:54098) 2023-09-30T22:31:29.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.117+0000 7fbaee7fc700 1 -- 172.21.15.16:0/4193784223 learned_addr learned my addr 172.21.15.16:0/4193784223 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:29.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.117+0000 7fbaeeffd700 1 --2- 172.21.15.16:0/4193784223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaf00984f0 0x7fbaf01c8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:29.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.117+0000 7fbaef7fe700 1 --2- 172.21.15.16:0/4193784223 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbaf009e040 0x7fbaf01c90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:29.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.117+0000 7fbaee7fc700 1 -- 172.21.15.16:0/4193784223 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbaf009e040 msgr2=0x7fbaf01c90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:29.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.117+0000 7fbaee7fc700 1 --2- 172.21.15.16:0/4193784223 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbaf009e040 0x7fbaf01c90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:29.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.117+0000 7fbaee7fc700 1 -- 172.21.15.16:0/4193784223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaf00984f0 msgr2=0x7fbaf01c8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:29.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.117+0000 7fbaee7fc700 1 --2- 172.21.15.16:0/4193784223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaf00984f0 0x7fbaf01c8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:29.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.117+0000 7fbaee7fc700 1 -- 172.21.15.16:0/4193784223 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbae0009550 con 0x7fbaf009d6f0 2023-09-30T22:31:29.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.118+0000 7fbaee7fc700 1 --2- 172.21.15.16:0/4193784223 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbaf009d6f0 0x7fbaf01c8b70 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbae400b500 tx=0x7fbae400b810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:29.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.118+0000 7fbad77fe700 1 -- 172.21.15.16:0/4193784223 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae4011830 con 0x7fbaf009d6f0 2023-09-30T22:31:29.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.118+0000 7fbad77fe700 1 -- 172.21.15.16:0/4193784223 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbae40119b0 con 0x7fbaf009d6f0 2023-09-30T22:31:29.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.118+0000 7fbaf57ed700 1 -- 172.21.15.16:0/4193784223 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbae8009a70 con 0x7fbaf009d6f0 2023-09-30T22:31:29.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.118+0000 7fbad77fe700 1 -- 172.21.15.16:0/4193784223 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae4011d60 con 0x7fbaf009d6f0 2023-09-30T22:31:29.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.118+0000 7fbaf57ed700 1 -- 172.21.15.16:0/4193784223 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaf006f1b0 con 0x7fbaf009d6f0 2023-09-30T22:31:29.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.119+0000 7fbaf57ed700 1 -- 172.21.15.16:0/4193784223 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbaf01b8690 con 0x7fbaf009d6f0 2023-09-30T22:31:29.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.120+0000 7fbad77fe700 1 -- 172.21.15.16:0/4193784223 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbae401a400 con 0x7fbaf009d6f0 2023-09-30T22:31:29.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.120+0000 7fbad77fe700 1 -- 172.21.15.16:0/4193784223 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fbae403f650 con 0x7fbaf009d6f0 2023-09-30T22:31:29.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.129+0000 7fbad77fe700 1 -- 172.21.15.16:0/4193784223 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbae400f8d0 con 0x7fbaf009d6f0 2023-09-30T22:31:29.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.140+0000 7f221a8b9700 1 -- 172.21.15.16:0/2174614681 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f22141a7d50 con 0x7f221409e930 2023-09-30T22:31:29.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.140+0000 7f2210ff9700 1 -- 172.21.15.16:0/2174614681 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2204021030 con 0x7f221409e930 2023-09-30T22:31:29.141 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:31:29.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.143+0000 7f221a8b9700 1 -- 172.21.15.16:0/2174614681 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f221409e930 msgr2=0x7f22141c8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:29.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.143+0000 7f221a8b9700 1 --2- 172.21.15.16:0/2174614681 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f221409e930 0x7f22141c8130 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f220400e9b0 tx=0x7f220400ecc0 comp rx=0 tx=0).stop 2023-09-30T22:31:29.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.143+0000 7f221a8b9700 1 -- 172.21.15.16:0/2174614681 shutdown_connections 2023-09-30T22:31:29.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.143+0000 7f221a8b9700 1 --2- 172.21.15.16:0/2174614681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f221409f650 0x7f22141c8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:29.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.143+0000 7f221a8b9700 1 --2- 172.21.15.16:0/2174614681 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f221409e930 0x7f22141c8130 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:29.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.143+0000 7f221a8b9700 1 --2- 172.21.15.16:0/2174614681 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f221409f240 0x7f22141c8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:29.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.143+0000 7f221a8b9700 1 -- 172.21.15.16:0/2174614681 >> 172.21.15.16:0/2174614681 conn(0x7f22140a4210 msgr2=0x7f22141bbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:29.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.143+0000 7f221a8b9700 1 -- 172.21.15.16:0/2174614681 shutdown_connections 2023-09-30T22:31:29.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.143+0000 7f221a8b9700 1 -- 172.21.15.16:0/2174614681 wait complete. 2023-09-30T22:31:29.158 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:31:29.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.245+0000 7f8ea8c65700 1 -- 172.21.15.16:0/3757854707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8ea4120f60 con 0x7f8ea409e410 2023-09-30T22:31:29.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.246+0000 7f8e92ffd700 1 -- 172.21.15.16:0/3757854707 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8e9c01cbb0 con 0x7f8ea409e410 2023-09-30T22:31:29.247 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:31:29.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.248+0000 7f8ea8c65700 1 -- 172.21.15.16:0/3757854707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ea409e410 msgr2=0x7f8ea411eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:29.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.248+0000 7f8ea8c65700 1 --2- 172.21.15.16:0/3757854707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ea409e410 0x7f8ea411eb80 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c00efc0 tx=0x7f8e9c00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:31:29.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.248+0000 7f8ea8c65700 1 -- 172.21.15.16:0/3757854707 shutdown_connections 2023-09-30T22:31:29.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.248+0000 7f8ea8c65700 1 --2- 172.21.15.16:0/3757854707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ea409e410 0x7f8ea411eb80 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:29.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.248+0000 7f8ea8c65700 1 --2- 172.21.15.16:0/3757854707 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8ea409d6f0 0x7f8ea411e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:29.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.248+0000 7f8ea8c65700 1 --2- 172.21.15.16:0/3757854707 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8ea409e000 0x7f8ea411e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:29.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.248+0000 7f8ea8c65700 1 -- 172.21.15.16:0/3757854707 >> 172.21.15.16:0/3757854707 conn(0x7f8ea4066f50 msgr2=0x7f8ea406a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:29.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.249+0000 7f8ea8c65700 1 -- 172.21.15.16:0/3757854707 shutdown_connections 2023-09-30T22:31:29.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.249+0000 7f8ea8c65700 1 -- 172.21.15.16:0/3757854707 wait complete. 2023-09-30T22:31:29.263 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:31:29.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.314+0000 7fe47eb4d700 1 -- 172.21.15.16:0/1210291746 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe47806c360 con 0x7fe47813c1e0 2023-09-30T22:31:29.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.315+0000 7fe4757fa700 1 -- 172.21.15.16:0/1210291746 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe470011380 con 0x7fe47813c1e0 2023-09-30T22:31:29.315 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:31:29.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.317+0000 7fe47eb4d700 1 -- 172.21.15.16:0/1210291746 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe47813c1e0 msgr2=0x7fe4781270c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:29.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.317+0000 7fe47eb4d700 1 --2- 172.21.15.16:0/1210291746 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe47813c1e0 0x7fe4781270c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe47000cbc0 tx=0x7fe47000ced0 comp rx=0 tx=0).stop 2023-09-30T22:31:29.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.317+0000 7fe47eb4d700 1 -- 172.21.15.16:0/1210291746 shutdown_connections 2023-09-30T22:31:29.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.317+0000 7fe47eb4d700 1 --2- 172.21.15.16:0/1210291746 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe47813c1e0 0x7fe4781270c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:29.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.317+0000 7fe47eb4d700 1 --2- 172.21.15.16:0/1210291746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe47806a770 0x7fe4781cc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:29.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.317+0000 7fe47eb4d700 1 --2- 172.21.15.16:0/1210291746 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe47809dc30 0x7fe4781ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:29.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.317+0000 7fe47eb4d700 1 -- 172.21.15.16:0/1210291746 >> 172.21.15.16:0/1210291746 conn(0x7fe478066f50 msgr2=0x7fe47809b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:29.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.317+0000 7fe47eb4d700 1 -- 172.21.15.16:0/1210291746 shutdown_connections 2023-09-30T22:31:29.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.317+0000 7fe47eb4d700 1 -- 172.21.15.16:0/1210291746 wait complete. 2023-09-30T22:31:29.331 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:31:29.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.364+0000 7fbaf57ed700 1 -- 172.21.15.16:0/4193784223 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fbaf009bff0 con 0x7fbaf009d6f0 2023-09-30T22:31:29.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.364+0000 7fbad77fe700 1 -- 172.21.15.16:0/4193784223 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbae4015030 con 0x7fbaf009d6f0 2023-09-30T22:31:29.364 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:31:29.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.366+0000 7fbaf57ed700 1 -- 172.21.15.16:0/4193784223 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbaf009d6f0 msgr2=0x7fbaf01c8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:29.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.366+0000 7fbaf57ed700 1 --2- 172.21.15.16:0/4193784223 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbaf009d6f0 0x7fbaf01c8b70 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fbae400b500 tx=0x7fbae400b810 comp rx=0 tx=0).stop 2023-09-30T22:31:29.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.366+0000 7fbaf57ed700 1 -- 172.21.15.16:0/4193784223 shutdown_connections 2023-09-30T22:31:29.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.366+0000 7fbaf57ed700 1 --2- 172.21.15.16:0/4193784223 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbaf009e040 0x7fbaf01c90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:29.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.366+0000 7fbaf57ed700 1 --2- 172.21.15.16:0/4193784223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbaf00984f0 0x7fbaf01c8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:29.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.366+0000 7fbaf57ed700 1 --2- 172.21.15.16:0/4193784223 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbaf009d6f0 0x7fbaf01c8b70 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:29.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.367+0000 7fbaf57ed700 1 -- 172.21.15.16:0/4193784223 >> 172.21.15.16:0/4193784223 conn(0x7fbaf009f470 msgr2=0x7fbaf01b8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:29.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.367+0000 7fbaf57ed700 1 -- 172.21.15.16:0/4193784223 shutdown_connections 2023-09-30T22:31:29.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:29.367+0000 7fbaf57ed700 1 -- 172.21.15.16:0/4193784223 wait complete. 2023-09-30T22:31:29.380 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:31:30.159 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:30.264 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:30.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.308+0000 7f4a7ced1700 1 Processor -- start 2023-09-30T22:31:30.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.308+0000 7f4a7ced1700 1 -- start start 2023-09-30T22:31:30.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.309+0000 7f4a7ced1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4a780a1510 0x7f4a78138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:30.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.309+0000 7f4a7ced1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4a78139250 0x7f4a7813b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:30.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.309+0000 7f4a7ced1700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a7809edd0 con 0x7f4a780a0a30 2023-09-30T22:31:30.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.309+0000 7f4a7ced1700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a780a18e0 con 0x7f4a78139250 2023-09-30T22:31:30.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.309+0000 7f4a7ced1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7809e1f0 con 0x7f4a780a1510 2023-09-30T22:31:30.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.309+0000 7f4a76d9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4a78139250 0x7f4a7813b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:30.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.309+0000 7f4a76d9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4a78139250 0x7f4a7813b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54108/0 (socket says 172.21.15.16:54108) 2023-09-30T22:31:30.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.309+0000 7f4a6dd9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4a780a1510 0x7f4a78138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:30.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.309+0000 7f4a76d9d700 1 -- 172.21.15.16:0/3064982496 learned_addr learned my addr 172.21.15.16:0/3064982496 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:30.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.310+0000 7f4a76d9d700 1 -- 172.21.15.16:0/3064982496 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4a780a1510 msgr2=0x7f4a78138d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:30.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.310+0000 7f4a76d9d700 1 --2- 172.21.15.16:0/3064982496 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4a780a1510 0x7f4a78138d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:30.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.310+0000 7f4a75d9b700 1 -- 172.21.15.16:0/3064982496 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3070563720 0 0) 0x7f4a7809edd0 con 0x7f4a780a0a30 2023-09-30T22:31:30.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.310+0000 7f4a76d9d700 1 -- 172.21.15.16:0/3064982496 >> v1:172.21.15.16:6789/0 conn(0x7f4a780a0a30 legacy=0x7f4a780a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:30.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.310+0000 7f4a76d9d700 1 -- 172.21.15.16:0/3064982496 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a7809dc50 con 0x7f4a78139250 2023-09-30T22:31:30.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.310+0000 7f4a6dd9b700 1 --2- 172.21.15.16:0/3064982496 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4a780a1510 0x7f4a78138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:30.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.310+0000 7f4a76d9d700 1 --2- 172.21.15.16:0/3064982496 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4a78139250 0x7f4a7813b630 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4a70009e80 tx=0x7f4a7000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e862f673d0bc1864 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:30.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.310+0000 7f4a75d9b700 1 -- 172.21.15.16:0/3064982496 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7000e070 con 0x7f4a78139250 2023-09-30T22:31:30.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.310+0000 7f4a75d9b700 1 -- 172.21.15.16:0/3064982496 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a7000b910 con 0x7f4a78139250 2023-09-30T22:31:30.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.310+0000 7f4a75d9b700 1 -- 172.21.15.16:0/3064982496 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7000bcc0 con 0x7f4a78139250 2023-09-30T22:31:30.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.311+0000 7f4a7ced1700 1 -- 172.21.15.16:0/3064982496 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4a78139250 msgr2=0x7f4a7813b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:30.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.311+0000 7f4a7ced1700 1 --2- 172.21.15.16:0/3064982496 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4a78139250 0x7f4a7813b630 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f4a70009e80 tx=0x7f4a7000b330 comp rx=0 tx=0).stop 2023-09-30T22:31:30.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.311+0000 7f4a7ced1700 1 -- 172.21.15.16:0/3064982496 shutdown_connections 2023-09-30T22:31:30.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.311+0000 7f4a7ced1700 1 --2- 172.21.15.16:0/3064982496 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4a78139250 0x7f4a7813b630 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:30.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.311+0000 7f4a7ced1700 1 --2- 172.21.15.16:0/3064982496 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4a780a1510 0x7f4a78138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:30.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.311+0000 7f4a7ced1700 1 -- 172.21.15.16:0/3064982496 >> 172.21.15.16:0/3064982496 conn(0x7f4a780a6550 msgr2=0x7f4a780a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:30.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.311+0000 7f4a7ced1700 1 -- 172.21.15.16:0/3064982496 shutdown_connections 2023-09-30T22:31:30.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.311+0000 7f4a7ced1700 1 -- 172.21.15.16:0/3064982496 wait complete. 2023-09-30T22:31:30.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.312+0000 7f4a7ced1700 1 Processor -- start 2023-09-30T22:31:30.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.312+0000 7f4a7ced1700 1 -- start start 2023-09-30T22:31:30.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.313+0000 7f4a7ced1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4a780a0a30 0x7f4a781226f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:30.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.313+0000 7f4a7ced1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a780a1510 0x7f4a78122c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:30.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.313+0000 7f4a7ced1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4a78139250 0x7f4a78123170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:30.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.313+0000 7f4a7ced1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a780a0220 con 0x7f4a780a1510 2023-09-30T22:31:30.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.313+0000 7f4a7ced1700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a78124b60 con 0x7f4a780a0a30 2023-09-30T22:31:30.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.313+0000 7f4a7ced1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4a781385e0 con 0x7f4a78139250 2023-09-30T22:31:30.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.313+0000 7f4a7659c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4a780a0a30 0x7f4a781226f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:30.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.313+0000 7f4a6dd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a780a1510 0x7f4a78122c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:30.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.313+0000 7f4a6dd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a780a1510 0x7f4a78122c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53458/0 (socket says 172.21.15.16:53458) 2023-09-30T22:31:30.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.313+0000 7f4a7659c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4a780a0a30 0x7f4a781226f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54120/0 (socket says 172.21.15.16:54120) 2023-09-30T22:31:30.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.313+0000 7f4a6dd9b700 1 -- 172.21.15.16:0/2411580952 learned_addr learned my addr 172.21.15.16:0/2411580952 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:30.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.313+0000 7f4a76d9d700 1 --2- 172.21.15.16:0/2411580952 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4a78139250 0x7f4a78123170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:30.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.313+0000 7f4a6dd9b700 1 -- 172.21.15.16:0/2411580952 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4a78139250 msgr2=0x7f4a78123170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:30.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.313+0000 7f4a6dd9b700 1 --2- 172.21.15.16:0/2411580952 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4a78139250 0x7f4a78123170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:30.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.313+0000 7f4a6dd9b700 1 -- 172.21.15.16:0/2411580952 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4a780a0a30 msgr2=0x7f4a781226f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:30.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.313+0000 7f4a6dd9b700 1 --2- 172.21.15.16:0/2411580952 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4a780a0a30 0x7f4a781226f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:30.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.313+0000 7f4a6dd9b700 1 -- 172.21.15.16:0/2411580952 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a600094a0 con 0x7f4a780a1510 2023-09-30T22:31:30.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.313+0000 7f4a7659c700 1 --2- 172.21.15.16:0/2411580952 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4a780a0a30 0x7f4a781226f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:30.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.314+0000 7f4a6dd9b700 1 --2- 172.21.15.16:0/2411580952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a780a1510 0x7f4a78122c30 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f4a6000ebe0 tx=0x7f4a6000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:30.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.314+0000 7f4a6f7fe700 1 -- 172.21.15.16:0/2411580952 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a6000c850 con 0x7f4a780a1510 2023-09-30T22:31:30.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.314+0000 7f4a6f7fe700 1 -- 172.21.15.16:0/2411580952 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a6000c9b0 con 0x7f4a780a1510 2023-09-30T22:31:30.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.314+0000 7f4a7ced1700 1 -- 172.21.15.16:0/2411580952 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a70009b30 con 0x7f4a780a1510 2023-09-30T22:31:30.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.314+0000 7f4a6f7fe700 1 -- 172.21.15.16:0/2411580952 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a6000cd40 con 0x7f4a780a1510 2023-09-30T22:31:30.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.314+0000 7f4a7ced1700 1 -- 172.21.15.16:0/2411580952 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a781269a0 con 0x7f4a780a1510 2023-09-30T22:31:30.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.315+0000 7f4a7ced1700 1 -- 172.21.15.16:0/2411580952 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a78074ea0 con 0x7f4a780a1510 2023-09-30T22:31:30.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.315+0000 7f4a6f7fe700 1 -- 172.21.15.16:0/2411580952 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4a60013470 con 0x7f4a780a1510 2023-09-30T22:31:30.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.315+0000 7f4a6f7fe700 1 -- 172.21.15.16:0/2411580952 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f4a6003ffe0 con 0x7f4a780a1510 2023-09-30T22:31:30.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.324+0000 7f4a6f7fe700 1 -- 172.21.15.16:0/2411580952 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4a60013720 con 0x7f4a780a1510 2023-09-30T22:31:30.332 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:30.381 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:30.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.414+0000 7fca3ac38700 1 Processor -- start 2023-09-30T22:31:30.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.415+0000 7fca3ac38700 1 -- start start 2023-09-30T22:31:30.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.415+0000 7fca3ac38700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fca3409eca0 0x7fca3413cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:30.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.415+0000 7fca3ac38700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca3406f050 0x7fca3413f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:30.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.415+0000 7fca3ac38700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca34070ee0 con 0x7fca3406e570 2023-09-30T22:31:30.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.415+0000 7fca3ac38700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca34071980 con 0x7fca3409eca0 2023-09-30T22:31:30.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.415+0000 7fca3ac38700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fca34071480 con 0x7fca3406f050 2023-09-30T22:31:30.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.415+0000 7fca389d4700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fca3406e570 0x7fca3406e940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44060/0 (socket says 172.21.15.16:44060) 2023-09-30T22:31:30.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.415+0000 7fca389d4700 1 -- 172.21.15.16:0/3720188904 learned_addr learned my addr 172.21.15.16:0/3720188904 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:31:30.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.415+0000 7fca33fff700 1 --2- 172.21.15.16:0/3720188904 >> v2:172.21.15.16:3300/0 conn(0x7fca3409eca0 0x7fca3413cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:30.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.416+0000 7fca391d5700 1 --2- 172.21.15.16:0/3720188904 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca3406f050 0x7fca3413f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:30.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.416+0000 7fca337fe700 1 -- 172.21.15.16:0/3720188904 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3696294671 0 0) 0x7fca34070ee0 con 0x7fca3406e570 2023-09-30T22:31:30.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.416+0000 7fca337fe700 1 -- 172.21.15.16:0/3720188904 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca240035b0 con 0x7fca3406e570 2023-09-30T22:31:30.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.416+0000 7fca391d5700 1 -- 172.21.15.16:0/3720188904 >> v2:172.21.15.16:3300/0 conn(0x7fca3409eca0 msgr2=0x7fca3413cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:30.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.416+0000 7fca391d5700 1 --2- 172.21.15.16:0/3720188904 >> v2:172.21.15.16:3300/0 conn(0x7fca3409eca0 0x7fca3413cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:30.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.416+0000 7fca391d5700 1 -- 172.21.15.16:0/3720188904 >> v1:172.21.15.16:6789/0 conn(0x7fca3406e570 legacy=0x7fca3406e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:30.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.416+0000 7fca391d5700 1 -- 172.21.15.16:0/3720188904 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca34071b90 con 0x7fca3406f050 2023-09-30T22:31:30.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.416+0000 7fca391d5700 1 --2- 172.21.15.16:0/3720188904 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca3406f050 0x7fca3413f3f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fca2c009d90 tx=0x7fca2c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de8c9854c69afc57 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:30.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.416+0000 7fca337fe700 1 -- 172.21.15.16:0/3720188904 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2c00e070 con 0x7fca3406f050 2023-09-30T22:31:30.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.417+0000 7fca337fe700 1 -- 172.21.15.16:0/3720188904 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca2c00bb20 con 0x7fca3406f050 2023-09-30T22:31:30.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.417+0000 7fca337fe700 1 -- 172.21.15.16:0/3720188904 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2c00be90 con 0x7fca3406f050 2023-09-30T22:31:30.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.417+0000 7fca3ac38700 1 -- 172.21.15.16:0/3720188904 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca3406f050 msgr2=0x7fca3413f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:30.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.417+0000 7fca3ac38700 1 --2- 172.21.15.16:0/3720188904 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca3406f050 0x7fca3413f3f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fca2c009d90 tx=0x7fca2c00b330 comp rx=0 tx=0).stop 2023-09-30T22:31:30.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.417+0000 7fca3ac38700 1 -- 172.21.15.16:0/3720188904 shutdown_connections 2023-09-30T22:31:30.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.417+0000 7fca3ac38700 1 --2- 172.21.15.16:0/3720188904 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca3406f050 0x7fca3413f3f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:30.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.417+0000 7fca3ac38700 1 --2- 172.21.15.16:0/3720188904 >> v2:172.21.15.16:3300/0 conn(0x7fca3409eca0 0x7fca3413cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:30.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.417+0000 7fca3ac38700 1 -- 172.21.15.16:0/3720188904 >> 172.21.15.16:0/3720188904 conn(0x7fca340a2aa0 msgr2=0x7fca340a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:30.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.417+0000 7fca3ac38700 1 -- 172.21.15.16:0/3720188904 shutdown_connections 2023-09-30T22:31:30.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.417+0000 7fca3ac38700 1 -- 172.21.15.16:0/3720188904 wait complete. 2023-09-30T22:31:30.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.419+0000 7fca3ac38700 1 Processor -- start 2023-09-30T22:31:30.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.419+0000 7fca3ac38700 1 -- start start 2023-09-30T22:31:30.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.420+0000 7fca3ac38700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca3406e570 0x7fca3413db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:30.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.420+0000 7fca3ac38700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fca3406f050 0x7fca3412a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:30.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.420+0000 7fca3ac38700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca3412a8e0 0x7fca3412ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:30.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.420+0000 7fca3ac38700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3412d480 con 0x7fca3406e570 2023-09-30T22:31:30.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.420+0000 7fca3ac38700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca3413e770 con 0x7fca3406f050 2023-09-30T22:31:30.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.420+0000 7fca3ac38700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fca3412cc90 con 0x7fca3412a8e0 2023-09-30T22:31:30.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.420+0000 7fca389d4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca3406e570 0x7fca3413db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:30.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.420+0000 7fca389d4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca3406e570 0x7fca3413db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53468/0 (socket says 172.21.15.16:53468) 2023-09-30T22:31:30.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.420+0000 7fca389d4700 1 -- 172.21.15.16:0/213089870 learned_addr learned my addr 172.21.15.16:0/213089870 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:30.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.420+0000 7fca391d5700 1 --2- 172.21.15.16:0/213089870 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca3412a8e0 0x7fca3412ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:30.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.420+0000 7fca389d4700 1 -- 172.21.15.16:0/213089870 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca3412a8e0 msgr2=0x7fca3412ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:30.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.420+0000 7fca389d4700 1 --2- 172.21.15.16:0/213089870 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca3412a8e0 0x7fca3412ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:30.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.421+0000 7fca389d4700 1 -- 172.21.15.16:0/213089870 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fca3406f050 msgr2=0x7fca3412a3a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:31:30.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.421+0000 7fca389d4700 1 --2- 172.21.15.16:0/213089870 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fca3406f050 0x7fca3412a3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:30.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.421+0000 7fca389d4700 1 -- 172.21.15.16:0/213089870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca2c009a70 con 0x7fca3406e570 2023-09-30T22:31:30.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.421+0000 7fca391d5700 1 --2- 172.21.15.16:0/213089870 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca3412a8e0 0x7fca3412ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:30.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.421+0000 7fca389d4700 1 --2- 172.21.15.16:0/213089870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca3406e570 0x7fca3413db10 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fca2800c700 tx=0x7fca2800ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:30.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.421+0000 7fca317fa700 1 -- 172.21.15.16:0/213089870 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca28005d70 con 0x7fca3406e570 2023-09-30T22:31:30.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.421+0000 7fca317fa700 1 -- 172.21.15.16:0/213089870 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca280070d0 con 0x7fca3406e570 2023-09-30T22:31:30.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.421+0000 7fca3ac38700 1 -- 172.21.15.16:0/213089870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca3412e1b0 con 0x7fca3406e570 2023-09-30T22:31:30.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.421+0000 7fca317fa700 1 -- 172.21.15.16:0/213089870 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca28007460 con 0x7fca3406e570 2023-09-30T22:31:30.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.421+0000 7fca3ac38700 1 -- 172.21.15.16:0/213089870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca3412da20 con 0x7fca3406e570 2023-09-30T22:31:30.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.422+0000 7fca317fa700 1 -- 172.21.15.16:0/213089870 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fca280075c0 con 0x7fca3406e570 2023-09-30T22:31:30.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.422+0000 7fca3ac38700 1 -- 172.21.15.16:0/213089870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca34074ea0 con 0x7fca3406e570 2023-09-30T22:31:30.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.422+0000 7fca317fa700 1 -- 172.21.15.16:0/213089870 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fca2803ec40 con 0x7fca3406e570 2023-09-30T22:31:30.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.432+0000 7fca317fa700 1 -- 172.21.15.16:0/213089870 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fca2800e280 con 0x7fca3406e570 2023-09-30T22:31:30.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.484+0000 7fadd4e10700 1 Processor -- start 2023-09-30T22:31:30.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.484+0000 7fadd4e10700 1 -- start start 2023-09-30T22:31:30.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.485+0000 7fadd4e10700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fadd006e570 0x7fadd006e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:30.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.485+0000 7fadd4e10700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fadd006ee80 0x7fadd0134d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:30.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.485+0000 7fadd4e10700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadd0070ee0 con 0x7fadd009eca0 2023-09-30T22:31:30.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.485+0000 7fadd4e10700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadd0071480 con 0x7fadd006e570 2023-09-30T22:31:30.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.485+0000 7fadd4e10700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadd0071b90 con 0x7fadd006ee80 2023-09-30T22:31:30.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.485+0000 7fadce59c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fadd006e570 0x7fadd006e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:30.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.485+0000 7fadcdd9b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fadd009eca0 0x7fadd0134610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44066/0 (socket says 172.21.15.16:44066) 2023-09-30T22:31:30.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.485+0000 7fadcdd9b700 1 -- 172.21.15.16:0/2093625699 learned_addr learned my addr 172.21.15.16:0/2093625699 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:31:30.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.485+0000 7fadced9d700 1 --2- 172.21.15.16:0/2093625699 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fadd006ee80 0x7fadd0134d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:30.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.486+0000 7fadced9d700 1 -- 172.21.15.16:0/2093625699 >> v2:172.21.15.16:3300/0 conn(0x7fadd006e570 msgr2=0x7fadd006e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:30.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.486+0000 7fadced9d700 1 --2- 172.21.15.16:0/2093625699 >> v2:172.21.15.16:3300/0 conn(0x7fadd006e570 0x7fadd006e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:30.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.486+0000 7fadced9d700 1 -- 172.21.15.16:0/2093625699 >> v1:172.21.15.16:6789/0 conn(0x7fadd009eca0 legacy=0x7fadd0134610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:30.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.486+0000 7fadced9d700 1 -- 172.21.15.16:0/2093625699 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd00711a0 con 0x7fadd006ee80 2023-09-30T22:31:30.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.486+0000 7fadced9d700 1 --2- 172.21.15.16:0/2093625699 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fadd006ee80 0x7fadd0134d20 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fadc8009d90 tx=0x7fadc800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c38e9537fd962345 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:30.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.486+0000 7fadcd59a700 1 -- 172.21.15.16:0/2093625699 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc800e070 con 0x7fadd006ee80 2023-09-30T22:31:30.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.486+0000 7fadcd59a700 1 -- 172.21.15.16:0/2093625699 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadc800bb20 con 0x7fadd006ee80 2023-09-30T22:31:30.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.486+0000 7fadcd59a700 1 -- 172.21.15.16:0/2093625699 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc8012430 con 0x7fadd006ee80 2023-09-30T22:31:30.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.487+0000 7fadd4e10700 1 -- 172.21.15.16:0/2093625699 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fadd006ee80 msgr2=0x7fadd0134d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:30.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.487+0000 7fadd4e10700 1 --2- 172.21.15.16:0/2093625699 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fadd006ee80 0x7fadd0134d20 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fadc8009d90 tx=0x7fadc800b330 comp rx=0 tx=0).stop 2023-09-30T22:31:30.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.487+0000 7fadd4e10700 1 -- 172.21.15.16:0/2093625699 shutdown_connections 2023-09-30T22:31:30.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.487+0000 7fadd4e10700 1 --2- 172.21.15.16:0/2093625699 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fadd006ee80 0x7fadd0134d20 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:30.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.487+0000 7fadd4e10700 1 --2- 172.21.15.16:0/2093625699 >> v2:172.21.15.16:3300/0 conn(0x7fadd006e570 0x7fadd006e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:30.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.487+0000 7fadd4e10700 1 -- 172.21.15.16:0/2093625699 >> 172.21.15.16:0/2093625699 conn(0x7fadd00a2aa0 msgr2=0x7fadd00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:30.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.487+0000 7fadd4e10700 1 -- 172.21.15.16:0/2093625699 shutdown_connections 2023-09-30T22:31:30.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.487+0000 7fadd4e10700 1 -- 172.21.15.16:0/2093625699 wait complete. 2023-09-30T22:31:30.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.489+0000 7fadd4e10700 1 Processor -- start 2023-09-30T22:31:30.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.489+0000 7fadd4e10700 1 -- start start 2023-09-30T22:31:30.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.490+0000 7fadd4e10700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd006e570 0x7fadd01232d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:30.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.490+0000 7fadd4e10700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fadd006ee80 0x7fadd0123810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:30.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.490+0000 7fadd4e10700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fadd0123d50 0x7fadd01cca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:30.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.490+0000 7fadd4e10700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd0126e80 con 0x7fadd006e570 2023-09-30T22:31:30.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.490+0000 7fadd4e10700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadd0124710 con 0x7fadd006ee80 2023-09-30T22:31:30.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.490+0000 7fadd4e10700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fadd0126560 con 0x7fadd0123d50 2023-09-30T22:31:30.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.490+0000 7fadce59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd006e570 0x7fadd01232d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:30.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.490+0000 7fadce59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd006e570 0x7fadd01232d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53484/0 (socket says 172.21.15.16:53484) 2023-09-30T22:31:30.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.490+0000 7fadce59c700 1 -- 172.21.15.16:0/1153280566 learned_addr learned my addr 172.21.15.16:0/1153280566 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:30.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.490+0000 7fadcdd9b700 1 --2- 172.21.15.16:0/1153280566 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fadd006ee80 0x7fadd0123810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:30.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.490+0000 7fadced9d700 1 --2- 172.21.15.16:0/1153280566 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fadd0123d50 0x7fadd01cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:30.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.491+0000 7fadce59c700 1 -- 172.21.15.16:0/1153280566 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fadd0123d50 msgr2=0x7fadd01cca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:30.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.491+0000 7fadce59c700 1 --2- 172.21.15.16:0/1153280566 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fadd0123d50 0x7fadd01cca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:30.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.491+0000 7fadce59c700 1 -- 172.21.15.16:0/1153280566 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fadd006ee80 msgr2=0x7fadd0123810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:30.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.491+0000 7fadce59c700 1 --2- 172.21.15.16:0/1153280566 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fadd006ee80 0x7fadd0123810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:30.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.491+0000 7fadce59c700 1 -- 172.21.15.16:0/1153280566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc8009a70 con 0x7fadd006e570 2023-09-30T22:31:30.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.491+0000 7fadced9d700 1 --2- 172.21.15.16:0/1153280566 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fadd0123d50 0x7fadd01cca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:31:30.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.491+0000 7fadce59c700 1 --2- 172.21.15.16:0/1153280566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd006e570 0x7fadd01232d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fadc000e9e0 tx=0x7fadc000ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:30.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.491+0000 7fadc6ffd700 1 -- 172.21.15.16:0/1153280566 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc0009710 con 0x7fadd006e570 2023-09-30T22:31:30.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.491+0000 7fadc6ffd700 1 -- 172.21.15.16:0/1153280566 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadc000c850 con 0x7fadd006e570 2023-09-30T22:31:30.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.491+0000 7fadd4e10700 1 -- 172.21.15.16:0/1153280566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadd0134290 con 0x7fadd006e570 2023-09-30T22:31:30.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.491+0000 7fadd4e10700 1 -- 172.21.15.16:0/1153280566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadd0070660 con 0x7fadd006e570 2023-09-30T22:31:30.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.491+0000 7fadc6ffd700 1 -- 172.21.15.16:0/1153280566 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc000cbc0 con 0x7fadd006e570 2023-09-30T22:31:30.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.492+0000 7fadc6ffd700 1 -- 172.21.15.16:0/1153280566 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fadc000cd20 con 0x7fadd006e570 2023-09-30T22:31:30.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.493+0000 7fadc6ffd700 1 -- 172.21.15.16:0/1153280566 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fadc003f7c0 con 0x7fadd006e570 2023-09-30T22:31:30.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.493+0000 7fadd4e10700 1 -- 172.21.15.16:0/1153280566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadd0138680 con 0x7fadd006e570 2023-09-30T22:31:30.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.502+0000 7fadc6ffd700 1 -- 172.21.15.16:0/1153280566 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fadc0021080 con 0x7fadd006e570 2023-09-30T22:31:30.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.533+0000 7f98c6301700 1 Processor -- start 2023-09-30T22:31:30.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.533+0000 7f98c6301700 1 -- start start 2023-09-30T22:31:30.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.534+0000 7f98c6301700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f98c009d6f0 0x7f98c009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:30.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.534+0000 7f98c6301700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98c009e440 0x7f98c0134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:30.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.534+0000 7f98c6301700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98c009b120 con 0x7f98c009e000 2023-09-30T22:31:30.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.534+0000 7f98c6301700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98c006b460 con 0x7f98c009d6f0 2023-09-30T22:31:30.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.534+0000 7f98c6301700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f98c009a7e0 con 0x7f98c009e440 2023-09-30T22:31:30.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.534+0000 7f98c489e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98c009e440 0x7f98c0134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:30.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.534+0000 7f98bf7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f98c009e000 0x7f98c01339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44076/0 (socket says 172.21.15.16:44076) 2023-09-30T22:31:30.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.534+0000 7f98bf7fe700 1 -- 172.21.15.16:0/626831285 learned_addr learned my addr 172.21.15.16:0/626831285 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:31:30.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.534+0000 7f98bffff700 1 --2- 172.21.15.16:0/626831285 >> v2:172.21.15.16:3300/0 conn(0x7f98c009d6f0 0x7f98c009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:30.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.534+0000 7f98beffd700 1 -- 172.21.15.16:0/626831285 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2995522772 0 0) 0x7f98c009b120 con 0x7f98c009e000 2023-09-30T22:31:30.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.534+0000 7f98beffd700 1 -- 172.21.15.16:0/626831285 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98a80035b0 con 0x7f98c009e000 2023-09-30T22:31:30.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.535+0000 7f98c489e700 1 -- 172.21.15.16:0/626831285 >> v2:172.21.15.16:3300/0 conn(0x7f98c009d6f0 msgr2=0x7f98c009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:30.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.535+0000 7f98c489e700 1 --2- 172.21.15.16:0/626831285 >> v2:172.21.15.16:3300/0 conn(0x7f98c009d6f0 0x7f98c009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:30.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.535+0000 7f98c489e700 1 -- 172.21.15.16:0/626831285 >> v1:172.21.15.16:6789/0 conn(0x7f98c009e000 legacy=0x7f98c01339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:30.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.535+0000 7f98c489e700 1 -- 172.21.15.16:0/626831285 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c009a500 con 0x7f98c009e440 2023-09-30T22:31:30.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.535+0000 7f98bffff700 1 --2- 172.21.15.16:0/626831285 >> v2:172.21.15.16:3300/0 conn(0x7f98c009d6f0 0x7f98c009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:31:30.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.535+0000 7f98c489e700 1 --2- 172.21.15.16:0/626831285 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98c009e440 0x7f98c0134100 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f98b8009dc0 tx=0x7f98b800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3145901856d8e66c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:30.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.535+0000 7f98beffd700 1 -- 172.21.15.16:0/626831285 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b800e070 con 0x7f98c009e440 2023-09-30T22:31:30.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.535+0000 7f98beffd700 1 -- 172.21.15.16:0/626831285 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98b800bae0 con 0x7f98c009e440 2023-09-30T22:31:30.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.535+0000 7f98beffd700 1 -- 172.21.15.16:0/626831285 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b800be50 con 0x7f98c009e440 2023-09-30T22:31:30.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.536+0000 7f98c6301700 1 -- 172.21.15.16:0/626831285 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98c009e440 msgr2=0x7f98c0134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:30.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.536+0000 7f98c6301700 1 --2- 172.21.15.16:0/626831285 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98c009e440 0x7f98c0134100 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f98b8009dc0 tx=0x7f98b800b330 comp rx=0 tx=0).stop 2023-09-30T22:31:30.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.536+0000 7f98c6301700 1 -- 172.21.15.16:0/626831285 shutdown_connections 2023-09-30T22:31:30.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.536+0000 7f98c6301700 1 --2- 172.21.15.16:0/626831285 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98c009e440 0x7f98c0134100 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:30.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.536+0000 7f98c6301700 1 --2- 172.21.15.16:0/626831285 >> v2:172.21.15.16:3300/0 conn(0x7f98c009d6f0 0x7f98c009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:30.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.536+0000 7f98c6301700 1 -- 172.21.15.16:0/626831285 >> 172.21.15.16:0/626831285 conn(0x7f98c0066f50 msgr2=0x7f98c009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:30.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.536+0000 7f98c6301700 1 -- 172.21.15.16:0/626831285 shutdown_connections 2023-09-30T22:31:30.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.536+0000 7f98c6301700 1 -- 172.21.15.16:0/626831285 wait complete. 2023-09-30T22:31:30.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.538+0000 7f98c6301700 1 Processor -- start 2023-09-30T22:31:30.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.538+0000 7f98c6301700 1 -- start start 2023-09-30T22:31:30.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.539+0000 7f98c6301700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98c009d6f0 0x7f98c01c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:30.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.539+0000 7f98c6301700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98c009e000 0x7f98c01c8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:30.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.539+0000 7f98c6301700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f98c009e440 0x7f98c01c89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:30.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.539+0000 7f98c6301700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c01a4c40 con 0x7f98c009e000 2023-09-30T22:31:30.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.539+0000 7f98c6301700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98c01a8020 con 0x7f98c009e440 2023-09-30T22:31:30.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.539+0000 7f98c6301700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f98c009c710 con 0x7f98c009d6f0 2023-09-30T22:31:30.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.539+0000 7f98bffff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98c009d6f0 0x7f98c01c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:30.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.539+0000 7f98bffff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98c009d6f0 0x7f98c01c7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:33738/0 (socket says 172.21.15.16:33738) 2023-09-30T22:31:30.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.539+0000 7f98bffff700 1 -- 172.21.15.16:0/2714212742 learned_addr learned my addr 172.21.15.16:0/2714212742 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:30.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.539+0000 7f98bf7fe700 1 --2- 172.21.15.16:0/2714212742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98c009e000 0x7f98c01c8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:30.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.539+0000 7f98c489e700 1 --2- 172.21.15.16:0/2714212742 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f98c009e440 0x7f98c01c89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:30.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.539+0000 7f98bffff700 1 -- 172.21.15.16:0/2714212742 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f98c009e440 msgr2=0x7f98c01c89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:30.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.539+0000 7f98bffff700 1 --2- 172.21.15.16:0/2714212742 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f98c009e440 0x7f98c01c89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:30.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.539+0000 7f98bffff700 1 -- 172.21.15.16:0/2714212742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98c009e000 msgr2=0x7f98c01c8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:30.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.539+0000 7f98bffff700 1 --2- 172.21.15.16:0/2714212742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98c009e000 0x7f98c01c8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:30.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.539+0000 7f98bffff700 1 -- 172.21.15.16:0/2714212742 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b8009a70 con 0x7f98c009d6f0 2023-09-30T22:31:30.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.540+0000 7f98bffff700 1 --2- 172.21.15.16:0/2714212742 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98c009d6f0 0x7f98c01c7f40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f98b000e810 tx=0x7f98b000eb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:30.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.540+0000 7f98bcff9700 1 -- 172.21.15.16:0/2714212742 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b001f440 con 0x7f98c009d6f0 2023-09-30T22:31:30.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.540+0000 7f98bcff9700 1 -- 172.21.15.16:0/2714212742 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98b001f5a0 con 0x7f98c009d6f0 2023-09-30T22:31:30.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.540+0000 7f98c6301700 1 -- 172.21.15.16:0/2714212742 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98c01a6310 con 0x7f98c009d6f0 2023-09-30T22:31:30.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.540+0000 7f98bcff9700 1 -- 172.21.15.16:0/2714212742 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b001f930 con 0x7f98c009d6f0 2023-09-30T22:31:30.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.540+0000 7f98c6301700 1 -- 172.21.15.16:0/2714212742 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98c01a6f80 con 0x7f98c009d6f0 2023-09-30T22:31:30.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.541+0000 7f98bcff9700 1 -- 172.21.15.16:0/2714212742 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f98b001fa90 con 0x7f98c009d6f0 2023-09-30T22:31:30.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.541+0000 7f98c6301700 1 -- 172.21.15.16:0/2714212742 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98c0074ea0 con 0x7f98c009d6f0 2023-09-30T22:31:30.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.542+0000 7f98bcff9700 1 -- 172.21.15.16:0/2714212742 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f98b0047560 con 0x7f98c009d6f0 2023-09-30T22:31:30.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.544+0000 7f4a7ced1700 1 -- 172.21.15.16:0/2411580952 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4a78138820 con 0x7f4a780a1510 2023-09-30T22:31:30.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.545+0000 7f4a6f7fe700 1 -- 172.21.15.16:0/2411580952 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4a6001ab10 con 0x7f4a780a1510 2023-09-30T22:31:30.547 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:31:30.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.547+0000 7f98bcff9700 1 -- 172.21.15.16:0/2714212742 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f98b001fd40 con 0x7f98c009d6f0 2023-09-30T22:31:30.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.547+0000 7f4a7ced1700 1 -- 172.21.15.16:0/2411580952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a780a1510 msgr2=0x7f4a78122c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:30.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.547+0000 7f4a7ced1700 1 --2- 172.21.15.16:0/2411580952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a780a1510 0x7f4a78122c30 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f4a6000ebe0 tx=0x7f4a6000eef0 comp rx=0 tx=0).stop 2023-09-30T22:31:30.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.547+0000 7f4a7ced1700 1 -- 172.21.15.16:0/2411580952 shutdown_connections 2023-09-30T22:31:30.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.547+0000 7f4a7ced1700 1 --2- 172.21.15.16:0/2411580952 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4a780a0a30 0x7f4a781226f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:30.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.547+0000 7f4a7ced1700 1 --2- 172.21.15.16:0/2411580952 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4a78139250 0x7f4a78123170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:30.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.547+0000 7f4a7ced1700 1 --2- 172.21.15.16:0/2411580952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a780a1510 0x7f4a78122c30 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:30.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.547+0000 7f4a7ced1700 1 -- 172.21.15.16:0/2411580952 >> 172.21.15.16:0/2411580952 conn(0x7f4a780a6550 msgr2=0x7f4a7811c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:30.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.548+0000 7f4a7ced1700 1 -- 172.21.15.16:0/2411580952 shutdown_connections 2023-09-30T22:31:30.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.548+0000 7f4a7ced1700 1 -- 172.21.15.16:0/2411580952 wait complete. 2023-09-30T22:31:30.562 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:31:30.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.670+0000 7fca3ac38700 1 -- 172.21.15.16:0/213089870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fca340710c0 con 0x7fca3406e570 2023-09-30T22:31:30.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.671+0000 7fca317fa700 1 -- 172.21.15.16:0/213089870 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fca28017070 con 0x7fca3406e570 2023-09-30T22:31:30.671 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:31:30.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.674+0000 7fca3ac38700 1 -- 172.21.15.16:0/213089870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca3406e570 msgr2=0x7fca3413db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:30.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.674+0000 7fca3ac38700 1 --2- 172.21.15.16:0/213089870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca3406e570 0x7fca3413db10 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fca2800c700 tx=0x7fca2800ca10 comp rx=0 tx=0).stop 2023-09-30T22:31:30.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.674+0000 7fca3ac38700 1 -- 172.21.15.16:0/213089870 shutdown_connections 2023-09-30T22:31:30.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.674+0000 7fca3ac38700 1 --2- 172.21.15.16:0/213089870 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca3412a8e0 0x7fca3412ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:30.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.674+0000 7fca3ac38700 1 --2- 172.21.15.16:0/213089870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca3406e570 0x7fca3413db10 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:30.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.674+0000 7fca3ac38700 1 --2- 172.21.15.16:0/213089870 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fca3406f050 0x7fca3412a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:30.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.674+0000 7fca3ac38700 1 -- 172.21.15.16:0/213089870 >> 172.21.15.16:0/213089870 conn(0x7fca340a2aa0 msgr2=0x7fca3409c970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:30.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.674+0000 7fca3ac38700 1 -- 172.21.15.16:0/213089870 shutdown_connections 2023-09-30T22:31:30.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.674+0000 7fca3ac38700 1 -- 172.21.15.16:0/213089870 wait complete. 2023-09-30T22:31:30.693 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:31:30.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.714+0000 7fadd4e10700 1 -- 172.21.15.16:0/1153280566 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fadd0074ea0 con 0x7fadd006e570 2023-09-30T22:31:30.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.715+0000 7fadc6ffd700 1 -- 172.21.15.16:0/1153280566 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fadc0017020 con 0x7fadd006e570 2023-09-30T22:31:30.715 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:31:30.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.717+0000 7fadd4e10700 1 -- 172.21.15.16:0/1153280566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd006e570 msgr2=0x7fadd01232d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:30.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.717+0000 7fadd4e10700 1 --2- 172.21.15.16:0/1153280566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd006e570 0x7fadd01232d0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fadc000e9e0 tx=0x7fadc000ecf0 comp rx=0 tx=0).stop 2023-09-30T22:31:30.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.717+0000 7fadd4e10700 1 -- 172.21.15.16:0/1153280566 shutdown_connections 2023-09-30T22:31:30.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.717+0000 7fadd4e10700 1 --2- 172.21.15.16:0/1153280566 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fadd0123d50 0x7fadd01cca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:30.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.717+0000 7fadd4e10700 1 --2- 172.21.15.16:0/1153280566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fadd006e570 0x7fadd01232d0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:30.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.717+0000 7fadd4e10700 1 --2- 172.21.15.16:0/1153280566 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fadd006ee80 0x7fadd0123810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:30.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.717+0000 7fadd4e10700 1 -- 172.21.15.16:0/1153280566 >> 172.21.15.16:0/1153280566 conn(0x7fadd00a2aa0 msgr2=0x7fadd0136150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:30.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.717+0000 7fadd4e10700 1 -- 172.21.15.16:0/1153280566 shutdown_connections 2023-09-30T22:31:30.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.717+0000 7fadd4e10700 1 -- 172.21.15.16:0/1153280566 wait complete. 2023-09-30T22:31:30.731 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:31:30.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.783+0000 7f98c6301700 1 -- 172.21.15.16:0/2714212742 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f98c009acb0 con 0x7f98c009d6f0 2023-09-30T22:31:30.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.783+0000 7f98bcff9700 1 -- 172.21.15.16:0/2714212742 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f98b001d340 con 0x7f98c009d6f0 2023-09-30T22:31:30.783 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:31:30.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.785+0000 7f98c6301700 1 -- 172.21.15.16:0/2714212742 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98c009d6f0 msgr2=0x7f98c01c7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:30.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.785+0000 7f98c6301700 1 --2- 172.21.15.16:0/2714212742 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98c009d6f0 0x7f98c01c7f40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f98b000e810 tx=0x7f98b000eb20 comp rx=0 tx=0).stop 2023-09-30T22:31:30.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.785+0000 7f98c6301700 1 -- 172.21.15.16:0/2714212742 shutdown_connections 2023-09-30T22:31:30.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.785+0000 7f98c6301700 1 --2- 172.21.15.16:0/2714212742 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f98c009e440 0x7f98c01c89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:30.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.785+0000 7f98c6301700 1 --2- 172.21.15.16:0/2714212742 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98c009d6f0 0x7f98c01c7f40 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:30.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.785+0000 7f98c6301700 1 --2- 172.21.15.16:0/2714212742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98c009e000 0x7f98c01c8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:30.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.785+0000 7f98c6301700 1 -- 172.21.15.16:0/2714212742 >> 172.21.15.16:0/2714212742 conn(0x7f98c0066f50 msgr2=0x7f98c00a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:30.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.785+0000 7f98c6301700 1 -- 172.21.15.16:0/2714212742 shutdown_connections 2023-09-30T22:31:30.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:30.785+0000 7f98c6301700 1 -- 172.21.15.16:0/2714212742 wait complete. 2023-09-30T22:31:30.799 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:31:31.563 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:31.694 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:31.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.708+0000 7fd983ba3700 1 Processor -- start 2023-09-30T22:31:31.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.708+0000 7fd983ba3700 1 -- start start 2023-09-30T22:31:31.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.709+0000 7fd983ba3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd97c0a0a30 0x7fd97c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:31.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.709+0000 7fd983ba3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd97c09d630 0x7fd97c0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:31.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.709+0000 7fd983ba3700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd97c133930 0x7fd97c133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:31.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.709+0000 7fd983ba3700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd97c0a2a70 con 0x7fd97c133930 2023-09-30T22:31:31.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.709+0000 7fd983ba3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd97c0a01d0 con 0x7fd97c0a0a30 2023-09-30T22:31:31.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.709+0000 7fd983ba3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd97c09e2b0 con 0x7fd97c09d630 2023-09-30T22:31:31.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.709+0000 7fd98113e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd97c09d630 0x7fd97c0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:31.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.709+0000 7fd98113e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd97c09d630 0x7fd97c0a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:33754/0 (socket says 172.21.15.16:33754) 2023-09-30T22:31:31.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.709+0000 7fd98113e700 1 -- 172.21.15.16:0/3758178410 learned_addr learned my addr 172.21.15.16:0/3758178410 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:31.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.709+0000 7fd98193f700 1 --2- 172.21.15.16:0/3758178410 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd97c0a0a30 0x7fd97c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:31.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.709+0000 7fd982140700 1 --2- 172.21.15.16:0/3758178410 >> v2:172.21.15.16:3300/0 conn(0x7fd97c133930 0x7fd97c133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:31.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.709+0000 7fd98113e700 1 -- 172.21.15.16:0/3758178410 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd97c0a0a30 msgr2=0x7fd97c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:31.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.709+0000 7fd98113e700 1 --2- 172.21.15.16:0/3758178410 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd97c0a0a30 0x7fd97c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:31.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.709+0000 7fd98113e700 1 -- 172.21.15.16:0/3758178410 >> v2:172.21.15.16:3300/0 conn(0x7fd97c133930 msgr2=0x7fd97c133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:31.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.709+0000 7fd98113e700 1 --2- 172.21.15.16:0/3758178410 >> v2:172.21.15.16:3300/0 conn(0x7fd97c133930 0x7fd97c133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:31.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.709+0000 7fd98113e700 1 -- 172.21.15.16:0/3758178410 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd97c09dc50 con 0x7fd97c09d630 2023-09-30T22:31:31.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.709+0000 7fd982140700 1 --2- 172.21.15.16:0/3758178410 >> v2:172.21.15.16:3300/0 conn(0x7fd97c133930 0x7fd97c133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:31.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.709+0000 7fd98193f700 1 --2- 172.21.15.16:0/3758178410 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd97c0a0a30 0x7fd97c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:31.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.710+0000 7fd98113e700 1 --2- 172.21.15.16:0/3758178410 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd97c09d630 0x7fd97c0a1340 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd96c0098f0 tx=0x7fd96c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=253811eef3632590 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:31.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.710+0000 7fd98093d700 1 -- 172.21.15.16:0/3758178410 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96c011830 con 0x7fd97c09d630 2023-09-30T22:31:31.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.710+0000 7fd98093d700 1 -- 172.21.15.16:0/3758178410 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd96c011990 con 0x7fd97c09d630 2023-09-30T22:31:31.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.710+0000 7fd98093d700 1 -- 172.21.15.16:0/3758178410 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96c011d20 con 0x7fd97c09d630 2023-09-30T22:31:31.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.710+0000 7fd983ba3700 1 -- 172.21.15.16:0/3758178410 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd97c09d630 msgr2=0x7fd97c0a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:31.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.710+0000 7fd983ba3700 1 --2- 172.21.15.16:0/3758178410 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd97c09d630 0x7fd97c0a1340 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd96c0098f0 tx=0x7fd96c009c00 comp rx=0 tx=0).stop 2023-09-30T22:31:31.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.710+0000 7fd983ba3700 1 -- 172.21.15.16:0/3758178410 shutdown_connections 2023-09-30T22:31:31.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.710+0000 7fd983ba3700 1 --2- 172.21.15.16:0/3758178410 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd97c0a0a30 0x7fd97c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:31.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.710+0000 7fd983ba3700 1 --2- 172.21.15.16:0/3758178410 >> v2:172.21.15.16:3300/0 conn(0x7fd97c133930 0x7fd97c133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:31.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.710+0000 7fd983ba3700 1 --2- 172.21.15.16:0/3758178410 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd97c09d630 0x7fd97c0a1340 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:31.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.710+0000 7fd983ba3700 1 -- 172.21.15.16:0/3758178410 >> 172.21.15.16:0/3758178410 conn(0x7fd97c0a3050 msgr2=0x7fd97c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:31.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.711+0000 7fd983ba3700 1 -- 172.21.15.16:0/3758178410 shutdown_connections 2023-09-30T22:31:31.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.711+0000 7fd983ba3700 1 -- 172.21.15.16:0/3758178410 wait complete. 2023-09-30T22:31:31.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.712+0000 7fd983ba3700 1 Processor -- start 2023-09-30T22:31:31.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.712+0000 7fd983ba3700 1 -- start start 2023-09-30T22:31:31.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.712+0000 7fd983ba3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd97c09d630 0x7fd97c11f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:31.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.712+0000 7fd983ba3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd97c0a0a30 0x7fd97c11f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:31.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.712+0000 7fd983ba3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd97c133930 0x7fd97c1c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:31.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.712+0000 7fd983ba3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97c122390 con 0x7fd97c09d630 2023-09-30T22:31:31.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.712+0000 7fd983ba3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd97c122570 con 0x7fd97c133930 2023-09-30T22:31:31.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.713+0000 7fd983ba3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd97c1333e0 con 0x7fd97c0a0a30 2023-09-30T22:31:31.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.713+0000 7fd982140700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd97c133930 0x7fd97c1c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:31.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.713+0000 7fd982140700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd97c133930 0x7fd97c1c8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54162/0 (socket says 172.21.15.16:54162) 2023-09-30T22:31:31.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.713+0000 7fd98113e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd97c0a0a30 0x7fd97c11f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:31.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.713+0000 7fd982140700 1 -- 172.21.15.16:0/143215024 learned_addr learned my addr 172.21.15.16:0/143215024 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:31.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.713+0000 7fd98193f700 1 --2- 172.21.15.16:0/143215024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd97c09d630 0x7fd97c11f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:31.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.713+0000 7fd982140700 1 -- 172.21.15.16:0/143215024 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd97c0a0a30 msgr2=0x7fd97c11f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:31.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.713+0000 7fd982140700 1 --2- 172.21.15.16:0/143215024 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd97c0a0a30 0x7fd97c11f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:31.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.713+0000 7fd982140700 1 -- 172.21.15.16:0/143215024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd97c09d630 msgr2=0x7fd97c11f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:31.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.713+0000 7fd982140700 1 --2- 172.21.15.16:0/143215024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd97c09d630 0x7fd97c11f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:31.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.713+0000 7fd982140700 1 -- 172.21.15.16:0/143215024 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9780094a0 con 0x7fd97c133930 2023-09-30T22:31:31.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.713+0000 7fd98113e700 1 --2- 172.21.15.16:0/143215024 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd97c0a0a30 0x7fd97c11f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:31.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.713+0000 7fd98193f700 1 --2- 172.21.15.16:0/143215024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd97c09d630 0x7fd97c11f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:31:31.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.713+0000 7fd982140700 1 --2- 172.21.15.16:0/143215024 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd97c133930 0x7fd97c1c8100 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd97400efc0 tx=0x7fd97400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:31.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.713+0000 7fd9727fc700 1 -- 172.21.15.16:0/143215024 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd97400cb40 con 0x7fd97c133930 2023-09-30T22:31:31.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.713+0000 7fd9727fc700 1 -- 172.21.15.16:0/143215024 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd97400f040 con 0x7fd97c133930 2023-09-30T22:31:31.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.713+0000 7fd983ba3700 1 -- 172.21.15.16:0/143215024 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd974009a70 con 0x7fd97c133930 2023-09-30T22:31:31.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.713+0000 7fd9727fc700 1 -- 172.21.15.16:0/143215024 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd974003a20 con 0x7fd97c133930 2023-09-30T22:31:31.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.714+0000 7fd983ba3700 1 -- 172.21.15.16:0/143215024 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd96c009550 con 0x7fd97c133930 2023-09-30T22:31:31.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.714+0000 7fd9727fc700 1 -- 172.21.15.16:0/143215024 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd974014480 con 0x7fd97c133930 2023-09-30T22:31:31.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.714+0000 7fd9727fc700 1 -- 172.21.15.16:0/143215024 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd97403ffa0 con 0x7fd97c133930 2023-09-30T22:31:31.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.715+0000 7fd983ba3700 1 -- 172.21.15.16:0/143215024 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd97c11efc0 con 0x7fd97c133930 2023-09-30T22:31:31.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.720+0000 7fd9727fc700 1 -- 172.21.15.16:0/143215024 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd974012910 con 0x7fd97c133930 2023-09-30T22:31:31.731 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:31.800 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:31.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.849+0000 7f3d7ae37700 1 Processor -- start 2023-09-30T22:31:31.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.849+0000 7f3d7ae37700 1 -- start start 2023-09-30T22:31:31.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.850+0000 7f3d7ae37700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3d74131810 0x7f3d74131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:31.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.850+0000 7f3d7ae37700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3d74099a40 0x7f3d74099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:31.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.850+0000 7f3d7ae37700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d741313c0 con 0x7f3d7409a3b0 2023-09-30T22:31:31.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.850+0000 7f3d7ae37700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7409b580 con 0x7f3d74131810 2023-09-30T22:31:31.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.850+0000 7f3d7ae37700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7406e8c0 con 0x7f3d74099a40 2023-09-30T22:31:31.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.850+0000 7f3d78bd3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3d74131810 0x7f3d74131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:31.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.850+0000 7f3d793d4700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3d7409a3b0 0x7f3d74138650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44092/0 (socket says 172.21.15.16:44092) 2023-09-30T22:31:31.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.850+0000 7f3d793d4700 1 -- 172.21.15.16:0/2750575831 learned_addr learned my addr 172.21.15.16:0/2750575831 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:31:31.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.850+0000 7f3d73fff700 1 --2- 172.21.15.16:0/2750575831 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3d74099a40 0x7f3d74099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:31.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.851+0000 7f3d737fe700 1 -- 172.21.15.16:0/2750575831 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1194722383 0 0) 0x7f3d741313c0 con 0x7f3d7409a3b0 2023-09-30T22:31:31.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.851+0000 7f3d737fe700 1 -- 172.21.15.16:0/2750575831 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d640035b0 con 0x7f3d7409a3b0 2023-09-30T22:31:31.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.851+0000 7f3d73fff700 1 -- 172.21.15.16:0/2750575831 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3d74131810 msgr2=0x7f3d74131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:31.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.851+0000 7f3d73fff700 1 --2- 172.21.15.16:0/2750575831 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3d74131810 0x7f3d74131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:31.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.851+0000 7f3d73fff700 1 -- 172.21.15.16:0/2750575831 >> v1:172.21.15.16:6789/0 conn(0x7f3d7409a3b0 legacy=0x7f3d74138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:31.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.851+0000 7f3d73fff700 1 -- 172.21.15.16:0/2750575831 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d7409a820 con 0x7f3d74099a40 2023-09-30T22:31:31.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.851+0000 7f3d73fff700 1 --2- 172.21.15.16:0/2750575831 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3d74099a40 0x7f3d74099e70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3d60009870 tx=0x7f3d60009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9984c516d13e26aa server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:31.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.851+0000 7f3d737fe700 1 -- 172.21.15.16:0/2750575831 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d60011830 con 0x7f3d74099a40 2023-09-30T22:31:31.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.851+0000 7f3d737fe700 1 -- 172.21.15.16:0/2750575831 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d60011990 con 0x7f3d74099a40 2023-09-30T22:31:31.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.851+0000 7f3d737fe700 1 -- 172.21.15.16:0/2750575831 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d60011d20 con 0x7f3d74099a40 2023-09-30T22:31:31.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.852+0000 7f3d7ae37700 1 -- 172.21.15.16:0/2750575831 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3d74099a40 msgr2=0x7f3d74099e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:31.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.852+0000 7f3d7ae37700 1 --2- 172.21.15.16:0/2750575831 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3d74099a40 0x7f3d74099e70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f3d60009870 tx=0x7f3d60009b80 comp rx=0 tx=0).stop 2023-09-30T22:31:31.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.852+0000 7f3d7ae37700 1 -- 172.21.15.16:0/2750575831 shutdown_connections 2023-09-30T22:31:31.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.852+0000 7f3d7ae37700 1 --2- 172.21.15.16:0/2750575831 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3d74131810 0x7f3d74131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:31.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.852+0000 7f3d7ae37700 1 --2- 172.21.15.16:0/2750575831 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3d74099a40 0x7f3d74099e70 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:31.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.852+0000 7f3d7ae37700 1 -- 172.21.15.16:0/2750575831 >> 172.21.15.16:0/2750575831 conn(0x7f3d740a2aa0 msgr2=0x7f3d740a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:31.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.852+0000 7f3d7ae37700 1 -- 172.21.15.16:0/2750575831 shutdown_connections 2023-09-30T22:31:31.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.852+0000 7f3d7ae37700 1 -- 172.21.15.16:0/2750575831 wait complete. 2023-09-30T22:31:31.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.854+0000 7f3d7ae37700 1 Processor -- start 2023-09-30T22:31:31.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.854+0000 7f3d7ae37700 1 -- start start 2023-09-30T22:31:31.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.855+0000 7f3d7ae37700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d74099a40 0x7f3d7406f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:31.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.855+0000 7f3d7ae37700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3d7409a3b0 0x7f3d7406fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:31.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.855+0000 7f3d7ae37700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3d74131810 0x7f3d74070090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:31.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.855+0000 7f3d7ae37700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d74121380 con 0x7f3d74099a40 2023-09-30T22:31:31.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.855+0000 7f3d7ae37700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d74120170 con 0x7f3d74131810 2023-09-30T22:31:31.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.855+0000 7f3d7ae37700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3d74130700 con 0x7f3d7409a3b0 2023-09-30T22:31:31.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.855+0000 7f3d78bd3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d74099a40 0x7f3d7406f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:31.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.855+0000 7f3d78bd3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d74099a40 0x7f3d7406f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53522/0 (socket says 172.21.15.16:53522) 2023-09-30T22:31:31.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.855+0000 7f3d78bd3700 1 -- 172.21.15.16:0/1584435561 learned_addr learned my addr 172.21.15.16:0/1584435561 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:31.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.855+0000 7f3d73fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3d7409a3b0 0x7f3d7406fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:31.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.855+0000 7f3d793d4700 1 --2- 172.21.15.16:0/1584435561 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3d74131810 0x7f3d74070090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:31.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.855+0000 7f3d78bd3700 1 -- 172.21.15.16:0/1584435561 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3d7409a3b0 msgr2=0x7f3d7406fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:31.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.855+0000 7f3d78bd3700 1 --2- 172.21.15.16:0/1584435561 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3d7409a3b0 0x7f3d7406fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:31.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.855+0000 7f3d78bd3700 1 -- 172.21.15.16:0/1584435561 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3d74131810 msgr2=0x7f3d74070090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:31.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.855+0000 7f3d78bd3700 1 --2- 172.21.15.16:0/1584435561 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3d74131810 0x7f3d74070090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:31.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.856+0000 7f3d78bd3700 1 -- 172.21.15.16:0/1584435561 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d600094a0 con 0x7f3d74099a40 2023-09-30T22:31:31.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.856+0000 7f3d73fff700 1 --2- 172.21.15.16:0/1584435561 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3d7409a3b0 0x7f3d7406fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:31.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.856+0000 7f3d78bd3700 1 --2- 172.21.15.16:0/1584435561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d74099a40 0x7f3d7406f610 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f3d6800e9b0 tx=0x7f3d6800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:31.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.856+0000 7f3d717fa700 1 -- 172.21.15.16:0/1584435561 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6800ee50 con 0x7f3d74099a40 2023-09-30T22:31:31.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.856+0000 7f3d717fa700 1 -- 172.21.15.16:0/1584435561 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d6800c850 con 0x7f3d74099a40 2023-09-30T22:31:31.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.856+0000 7f3d717fa700 1 -- 172.21.15.16:0/1584435561 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6800cbc0 con 0x7f3d74099a40 2023-09-30T22:31:31.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.857+0000 7f3d7ae37700 1 -- 172.21.15.16:0/1584435561 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d7406eb00 con 0x7f3d74099a40 2023-09-30T22:31:31.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.857+0000 7f3d7ae37700 1 -- 172.21.15.16:0/1584435561 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d7412ecc0 con 0x7f3d74099a40 2023-09-30T22:31:31.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.858+0000 7f3d717fa700 1 -- 172.21.15.16:0/1584435561 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3d6800cd20 con 0x7f3d74099a40 2023-09-30T22:31:31.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.858+0000 7f3d717fa700 1 -- 172.21.15.16:0/1584435561 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f3d6803fac0 con 0x7f3d74099a40 2023-09-30T22:31:31.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.858+0000 7f3d7ae37700 1 -- 172.21.15.16:0/1584435561 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d74074ea0 con 0x7f3d74099a40 2023-09-30T22:31:31.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.863+0000 7f3d717fa700 1 -- 172.21.15.16:0/1584435561 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3d68017070 con 0x7f3d74099a40 2023-09-30T22:31:31.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.880+0000 7f1d3fb72700 1 Processor -- start 2023-09-30T22:31:31.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.880+0000 7f1d3fb72700 1 -- start start 2023-09-30T22:31:31.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.881+0000 7f1d3fb72700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1d380a0a30 0x7f1d380a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:31.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.881+0000 7f1d3fb72700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d38134170 0x7f1d3813bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:31.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.881+0000 7f1d3fb72700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d3809edd0 con 0x7f1d380a1340 2023-09-30T22:31:31.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.881+0000 7f1d3fb72700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d3809e1f0 con 0x7f1d380a0a30 2023-09-30T22:31:31.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.881+0000 7f1d3fb72700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3809dc50 con 0x7f1d38134170 2023-09-30T22:31:31.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.881+0000 7f1d3e10f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d38134170 0x7f1d3813bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:31.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.881+0000 7f1d3e10f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d38134170 0x7f1d3813bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54198/0 (socket says 172.21.15.16:54198) 2023-09-30T22:31:31.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.881+0000 7f1d3e10f700 1 -- 172.21.15.16:0/260593608 learned_addr learned my addr 172.21.15.16:0/260593608 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:31.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.881+0000 7f1d3d90e700 1 --2- 172.21.15.16:0/260593608 >> v2:172.21.15.16:3300/0 conn(0x7f1d380a0a30 0x7f1d380a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:31.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.882+0000 7f1d3e10f700 1 -- 172.21.15.16:0/260593608 >> v2:172.21.15.16:3300/0 conn(0x7f1d380a0a30 msgr2=0x7f1d380a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:31.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.882+0000 7f1d3e10f700 1 --2- 172.21.15.16:0/260593608 >> v2:172.21.15.16:3300/0 conn(0x7f1d380a0a30 0x7f1d380a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:31.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.882+0000 7f1d3e10f700 1 -- 172.21.15.16:0/260593608 >> v1:172.21.15.16:6789/0 conn(0x7f1d380a1340 legacy=0x7f1d38133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:31.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.882+0000 7f1d3e10f700 1 -- 172.21.15.16:0/260593608 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d380a1fe0 con 0x7f1d38134170 2023-09-30T22:31:31.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.882+0000 7f1d3c90c700 1 -- 172.21.15.16:0/260593608 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1134552859 0 0) 0x7f1d3809edd0 con 0x7f1d380a1340 2023-09-30T22:31:31.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.882+0000 7f1d3e10f700 1 --2- 172.21.15.16:0/260593608 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d38134170 0x7f1d3813bd30 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1d300098f0 tx=0x7f1d30009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fbc3dbbc002d79d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:31.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.882+0000 7f1d3c90c700 1 -- 172.21.15.16:0/260593608 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3000b8a0 con 0x7f1d38134170 2023-09-30T22:31:31.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.882+0000 7f1d3c90c700 1 -- 172.21.15.16:0/260593608 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d3000ba00 con 0x7f1d38134170 2023-09-30T22:31:31.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.882+0000 7f1d3c90c700 1 -- 172.21.15.16:0/260593608 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3000bd90 con 0x7f1d38134170 2023-09-30T22:31:31.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.883+0000 7f1d3fb72700 1 -- 172.21.15.16:0/260593608 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d38134170 msgr2=0x7f1d3813bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:31.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.883+0000 7f1d3fb72700 1 --2- 172.21.15.16:0/260593608 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d38134170 0x7f1d3813bd30 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1d300098f0 tx=0x7f1d30009c00 comp rx=0 tx=0).stop 2023-09-30T22:31:31.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.883+0000 7f1d3fb72700 1 -- 172.21.15.16:0/260593608 shutdown_connections 2023-09-30T22:31:31.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.883+0000 7f1d3fb72700 1 --2- 172.21.15.16:0/260593608 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d38134170 0x7f1d3813bd30 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:31.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.883+0000 7f1d3fb72700 1 --2- 172.21.15.16:0/260593608 >> v2:172.21.15.16:3300/0 conn(0x7f1d380a0a30 0x7f1d380a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:31.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.883+0000 7f1d3fb72700 1 -- 172.21.15.16:0/260593608 >> 172.21.15.16:0/260593608 conn(0x7f1d380a6550 msgr2=0x7f1d380a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:31.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.883+0000 7f1d3fb72700 1 -- 172.21.15.16:0/260593608 shutdown_connections 2023-09-30T22:31:31.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.883+0000 7f1d3fb72700 1 -- 172.21.15.16:0/260593608 wait complete. 2023-09-30T22:31:31.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.884+0000 7f1d3fb72700 1 Processor -- start 2023-09-30T22:31:31.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.884+0000 7f1d3fb72700 1 -- start start 2023-09-30T22:31:31.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.885+0000 7f1d3fb72700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1d380a0a30 0x7f1d3811e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:31.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.885+0000 7f1d3fb72700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d380a1340 0x7f1d3811ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:31.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.885+0000 7f1d3fb72700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d38134170 0x7f1d3811f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:31.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.885+0000 7f1d3fb72700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d381217f0 con 0x7f1d38134170 2023-09-30T22:31:31.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.885+0000 7f1d3fb72700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d381201e0 con 0x7f1d380a1340 2023-09-30T22:31:31.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.885+0000 7f1d3fb72700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1d38120ab0 con 0x7f1d380a0a30 2023-09-30T22:31:31.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.885+0000 7f1d3d90e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1d380a0a30 0x7f1d3811e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:31.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.885+0000 7f1d3d90e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1d380a0a30 0x7f1d3811e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:33772/0 (socket says 172.21.15.16:33772) 2023-09-30T22:31:31.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.885+0000 7f1d3d90e700 1 -- 172.21.15.16:0/3867030416 learned_addr learned my addr 172.21.15.16:0/3867030416 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:31.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.885+0000 7f1d3e10f700 1 --2- 172.21.15.16:0/3867030416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d38134170 0x7f1d3811f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:31.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.885+0000 7f1d3d10d700 1 --2- 172.21.15.16:0/3867030416 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d380a1340 0x7f1d3811ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:31.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.885+0000 7f1d3d90e700 1 -- 172.21.15.16:0/3867030416 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d380a1340 msgr2=0x7f1d3811ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:31.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.885+0000 7f1d3d90e700 1 --2- 172.21.15.16:0/3867030416 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d380a1340 0x7f1d3811ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:31.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.885+0000 7f1d3d90e700 1 -- 172.21.15.16:0/3867030416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d38134170 msgr2=0x7f1d3811f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:31.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.885+0000 7f1d3d90e700 1 --2- 172.21.15.16:0/3867030416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d38134170 0x7f1d3811f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:31.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.885+0000 7f1d3d90e700 1 -- 172.21.15.16:0/3867030416 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d30009550 con 0x7f1d380a0a30 2023-09-30T22:31:31.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.885+0000 7f1d3e10f700 1 --2- 172.21.15.16:0/3867030416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d38134170 0x7f1d3811f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:31:31.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.886+0000 7f1d3d10d700 1 --2- 172.21.15.16:0/3867030416 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d380a1340 0x7f1d3811ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:31:31.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.886+0000 7f1d3d90e700 1 --2- 172.21.15.16:0/3867030416 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1d380a0a30 0x7f1d3811e6b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1d28007f20 tx=0x7f1d2800e3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:31.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.886+0000 7f1d367fc700 1 -- 172.21.15.16:0/3867030416 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d28014420 con 0x7f1d380a0a30 2023-09-30T22:31:31.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.886+0000 7f1d367fc700 1 -- 172.21.15.16:0/3867030416 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d28014580 con 0x7f1d380a0a30 2023-09-30T22:31:31.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.886+0000 7f1d3fb72700 1 -- 172.21.15.16:0/3867030416 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d380a25e0 con 0x7f1d380a0a30 2023-09-30T22:31:31.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.886+0000 7f1d367fc700 1 -- 172.21.15.16:0/3867030416 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d28014910 con 0x7f1d380a0a30 2023-09-30T22:31:31.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.886+0000 7f1d3fb72700 1 -- 172.21.15.16:0/3867030416 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d38133560 con 0x7f1d380a0a30 2023-09-30T22:31:31.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.887+0000 7f1d3fb72700 1 -- 172.21.15.16:0/3867030416 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d38074ea0 con 0x7f1d380a0a30 2023-09-30T22:31:31.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.887+0000 7f1d367fc700 1 -- 172.21.15.16:0/3867030416 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1d28014a70 con 0x7f1d380a0a30 2023-09-30T22:31:31.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.888+0000 7f1d367fc700 1 -- 172.21.15.16:0/3867030416 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f1d28041970 con 0x7f1d380a0a30 2023-09-30T22:31:31.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.892+0000 7f1d367fc700 1 -- 172.21.15.16:0/3867030416 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1d28045060 con 0x7f1d380a0a30 2023-09-30T22:31:31.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.960+0000 7fde0f7fe700 1 Processor -- start 2023-09-30T22:31:31.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.960+0000 7fde0f7fe700 1 -- start start 2023-09-30T22:31:31.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.961+0000 7fde0f7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fde0809c650 0x7fde0809ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:31.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.961+0000 7fde0f7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fde0809cf60 0x7fde08132080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:31.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.961+0000 7fde0f7fe700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde0809ba70 con 0x7fde080a0090 2023-09-30T22:31:31.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.961+0000 7fde0f7fe700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde0809e8f0 con 0x7fde0809c650 2023-09-30T22:31:31.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.961+0000 7fde0f7fe700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde080a0760 con 0x7fde0809cf60 2023-09-30T22:31:31.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.961+0000 7fde0d59a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fde0809c650 0x7fde0809ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:31.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.961+0000 7fde0d59a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fde0809c650 0x7fde0809ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53548/0 (socket says 172.21.15.16:53548) 2023-09-30T22:31:31.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.961+0000 7fde0d59a700 1 -- 172.21.15.16:0/2135796693 learned_addr learned my addr 172.21.15.16:0/2135796693 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:31.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.961+0000 7fde0dd9b700 1 --2- 172.21.15.16:0/2135796693 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fde0809cf60 0x7fde08132080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:31.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.962+0000 7fde0d59a700 1 -- 172.21.15.16:0/2135796693 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fde0809cf60 msgr2=0x7fde08132080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:31.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.962+0000 7fde0d59a700 1 --2- 172.21.15.16:0/2135796693 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fde0809cf60 0x7fde08132080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:31.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.962+0000 7fde0d59a700 1 -- 172.21.15.16:0/2135796693 >> v1:172.21.15.16:6789/0 conn(0x7fde080a0090 legacy=0x7fde08131970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:31.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.962+0000 7fde0d59a700 1 -- 172.21.15.16:0/2135796693 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde0809a080 con 0x7fde0809c650 2023-09-30T22:31:31.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.962+0000 7fde0d59a700 1 --2- 172.21.15.16:0/2135796693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde0809c650 0x7fde0809ca20 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fde040098f0 tx=0x7fde04009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f599a2851c96d781 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:31.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.962+0000 7fddfffff700 1 -- 172.21.15.16:0/2135796693 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde04011830 con 0x7fde0809c650 2023-09-30T22:31:31.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.962+0000 7fddfffff700 1 -- 172.21.15.16:0/2135796693 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde04011990 con 0x7fde0809c650 2023-09-30T22:31:31.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.962+0000 7fddfffff700 1 -- 172.21.15.16:0/2135796693 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde04011d20 con 0x7fde0809c650 2023-09-30T22:31:31.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.963+0000 7fde0f7fe700 1 -- 172.21.15.16:0/2135796693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde0809c650 msgr2=0x7fde0809ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:31.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.963+0000 7fde0f7fe700 1 --2- 172.21.15.16:0/2135796693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde0809c650 0x7fde0809ca20 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fde040098f0 tx=0x7fde04009c00 comp rx=0 tx=0).stop 2023-09-30T22:31:31.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.963+0000 7fde0f7fe700 1 -- 172.21.15.16:0/2135796693 shutdown_connections 2023-09-30T22:31:31.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.963+0000 7fde0f7fe700 1 --2- 172.21.15.16:0/2135796693 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fde0809cf60 0x7fde08132080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:31.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.963+0000 7fde0f7fe700 1 --2- 172.21.15.16:0/2135796693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde0809c650 0x7fde0809ca20 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:31.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.963+0000 7fde0f7fe700 1 -- 172.21.15.16:0/2135796693 >> 172.21.15.16:0/2135796693 conn(0x7fde080a3f70 msgr2=0x7fde080a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:31.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.963+0000 7fde0f7fe700 1 -- 172.21.15.16:0/2135796693 shutdown_connections 2023-09-30T22:31:31.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.963+0000 7fde0f7fe700 1 -- 172.21.15.16:0/2135796693 wait complete. 2023-09-30T22:31:31.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.964+0000 7fde0f7fe700 1 Processor -- start 2023-09-30T22:31:31.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.964+0000 7fde0f7fe700 1 -- start start 2023-09-30T22:31:31.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.964+0000 7fde0f7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fde0809c650 0x7fde08138ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:31.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.964+0000 7fde0f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde0809cf60 0x7fde08138ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:31.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.964+0000 7fde0f7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fde080a0090 0x7fde08139530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:31.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.964+0000 7fde0f7fe700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde08139b50 con 0x7fde0809cf60 2023-09-30T22:31:31.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.964+0000 7fde0f7fe700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde0812fb20 con 0x7fde080a0090 2023-09-30T22:31:31.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.964+0000 7fde0f7fe700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fde0809d980 con 0x7fde0809c650 2023-09-30T22:31:31.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.965+0000 7fde0dd9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fde080a0090 0x7fde08139530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:31.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.965+0000 7fde0dd9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fde080a0090 0x7fde08139530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54218/0 (socket says 172.21.15.16:54218) 2023-09-30T22:31:31.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.965+0000 7fde0dd9b700 1 -- 172.21.15.16:0/1768924237 learned_addr learned my addr 172.21.15.16:0/1768924237 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:31.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.965+0000 7fde0cd99700 1 --2- 172.21.15.16:0/1768924237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde0809cf60 0x7fde08138ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:31.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.965+0000 7fde0d59a700 1 --2- 172.21.15.16:0/1768924237 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fde0809c650 0x7fde08138ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:31.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.965+0000 7fde0dd9b700 1 -- 172.21.15.16:0/1768924237 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fde0809c650 msgr2=0x7fde08138ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:31.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.965+0000 7fde0dd9b700 1 --2- 172.21.15.16:0/1768924237 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fde0809c650 0x7fde08138ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:31.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.965+0000 7fde0dd9b700 1 -- 172.21.15.16:0/1768924237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde0809cf60 msgr2=0x7fde08138ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:31.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.965+0000 7fde0dd9b700 1 --2- 172.21.15.16:0/1768924237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde0809cf60 0x7fde08138ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:31.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.965+0000 7fde0dd9b700 1 -- 172.21.15.16:0/1768924237 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde04009550 con 0x7fde080a0090 2023-09-30T22:31:31.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.965+0000 7fde0dd9b700 1 --2- 172.21.15.16:0/1768924237 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fde080a0090 0x7fde08139530 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fde0000efc0 tx=0x7fde0000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:31.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.965+0000 7fddfdffb700 1 -- 172.21.15.16:0/1768924237 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde00009ea0 con 0x7fde080a0090 2023-09-30T22:31:31.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.965+0000 7fddfdffb700 1 -- 172.21.15.16:0/1768924237 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde0000f040 con 0x7fde080a0090 2023-09-30T22:31:31.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.965+0000 7fde0f7fe700 1 -- 172.21.15.16:0/1768924237 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde08137bf0 con 0x7fde080a0090 2023-09-30T22:31:31.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.965+0000 7fddfdffb700 1 -- 172.21.15.16:0/1768924237 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde00014480 con 0x7fde080a0090 2023-09-30T22:31:31.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.966+0000 7fde0f7fe700 1 -- 172.21.15.16:0/1768924237 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde08137ed0 con 0x7fde080a0090 2023-09-30T22:31:31.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.967+0000 7fddfdffb700 1 -- 172.21.15.16:0/1768924237 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fde000145e0 con 0x7fde080a0090 2023-09-30T22:31:31.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.967+0000 7fde0f7fe700 1 -- 172.21.15.16:0/1768924237 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde08070fb0 con 0x7fde080a0090 2023-09-30T22:31:31.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.967+0000 7fddfdffb700 1 -- 172.21.15.16:0/1768924237 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fde000403f0 con 0x7fde080a0090 2023-09-30T22:31:31.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.972+0000 7fddfdffb700 1 -- 172.21.15.16:0/1768924237 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fde000076f0 con 0x7fde080a0090 2023-09-30T22:31:31.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.992+0000 7fd983ba3700 1 -- 172.21.15.16:0/143215024 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd97c0a0410 con 0x7fd97c133930 2023-09-30T22:31:31.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.993+0000 7fd9727fc700 1 -- 172.21.15.16:0/143215024 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd974012340 con 0x7fd97c133930 2023-09-30T22:31:31.993 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:31:31.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.996+0000 7fd983ba3700 1 -- 172.21.15.16:0/143215024 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd97c133930 msgr2=0x7fd97c1c8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:31.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.996+0000 7fd983ba3700 1 --2- 172.21.15.16:0/143215024 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd97c133930 0x7fd97c1c8100 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd97400efc0 tx=0x7fd97400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:31:31.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.996+0000 7fd983ba3700 1 -- 172.21.15.16:0/143215024 shutdown_connections 2023-09-30T22:31:31.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.996+0000 7fd983ba3700 1 --2- 172.21.15.16:0/143215024 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd97c133930 0x7fd97c1c8100 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:31.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.996+0000 7fd983ba3700 1 --2- 172.21.15.16:0/143215024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd97c09d630 0x7fd97c11f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:31.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.996+0000 7fd983ba3700 1 --2- 172.21.15.16:0/143215024 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd97c0a0a30 0x7fd97c11f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:31.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.996+0000 7fd983ba3700 1 -- 172.21.15.16:0/143215024 >> 172.21.15.16:0/143215024 conn(0x7fd97c0a3050 msgr2=0x7fd97c09ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:31.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.997+0000 7fd983ba3700 1 -- 172.21.15.16:0/143215024 shutdown_connections 2023-09-30T22:31:31.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:31.997+0000 7fd983ba3700 1 -- 172.21.15.16:0/143215024 wait complete. 2023-09-30T22:31:32.016 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:31:32.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:32.095+0000 7f3d7ae37700 1 -- 172.21.15.16:0/1584435561 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3d7406b9e0 con 0x7f3d74099a40 2023-09-30T22:31:32.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:32.096+0000 7f3d717fa700 1 -- 172.21.15.16:0/1584435561 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3d6802a090 con 0x7f3d74099a40 2023-09-30T22:31:32.096 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:31:32.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:32.098+0000 7f3d7ae37700 1 -- 172.21.15.16:0/1584435561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d74099a40 msgr2=0x7f3d7406f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:32.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:32.098+0000 7f3d7ae37700 1 --2- 172.21.15.16:0/1584435561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d74099a40 0x7f3d7406f610 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f3d6800e9b0 tx=0x7f3d6800ecc0 comp rx=0 tx=0).stop 2023-09-30T22:31:32.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:32.098+0000 7f3d7ae37700 1 -- 172.21.15.16:0/1584435561 shutdown_connections 2023-09-30T22:31:32.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:32.098+0000 7f3d7ae37700 1 --2- 172.21.15.16:0/1584435561 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3d74131810 0x7f3d74070090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:32.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:32.098+0000 7f3d7ae37700 1 --2- 172.21.15.16:0/1584435561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3d74099a40 0x7f3d7406f610 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:32.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:32.098+0000 7f3d7ae37700 1 --2- 172.21.15.16:0/1584435561 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3d7409a3b0 0x7f3d7406fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:32.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:32.098+0000 7f3d7ae37700 1 -- 172.21.15.16:0/1584435561 >> 172.21.15.16:0/1584435561 conn(0x7f3d740a2aa0 msgr2=0x7f3d741b8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:32.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:32.098+0000 7f3d7ae37700 1 -- 172.21.15.16:0/1584435561 shutdown_connections 2023-09-30T22:31:32.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:32.098+0000 7f3d7ae37700 1 -- 172.21.15.16:0/1584435561 wait complete. 2023-09-30T22:31:32.112 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:31:32.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:32.161+0000 7f1d3fb72700 1 -- 172.21.15.16:0/3867030416 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1d380a28e0 con 0x7f1d380a0a30 2023-09-30T22:31:32.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:32.162+0000 7f1d367fc700 1 -- 172.21.15.16:0/3867030416 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1d28045070 con 0x7f1d380a0a30 2023-09-30T22:31:32.162 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:31:32.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:32.164+0000 7f1d3fb72700 1 -- 172.21.15.16:0/3867030416 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1d380a0a30 msgr2=0x7f1d3811e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:32.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:32.164+0000 7f1d3fb72700 1 --2- 172.21.15.16:0/3867030416 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1d380a0a30 0x7f1d3811e6b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1d28007f20 tx=0x7f1d2800e3c0 comp rx=0 tx=0).stop 2023-09-30T22:31:32.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:32.164+0000 7f1d3fb72700 1 -- 172.21.15.16:0/3867030416 shutdown_connections 2023-09-30T22:31:32.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:32.164+0000 7f1d3fb72700 1 --2- 172.21.15.16:0/3867030416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d38134170 0x7f1d3811f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:32.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:32.164+0000 7f1d3fb72700 1 --2- 172.21.15.16:0/3867030416 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1d380a0a30 0x7f1d3811e6b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:32.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:32.164+0000 7f1d3fb72700 1 --2- 172.21.15.16:0/3867030416 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d380a1340 0x7f1d3811ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:32.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:32.164+0000 7f1d3fb72700 1 -- 172.21.15.16:0/3867030416 >> 172.21.15.16:0/3867030416 conn(0x7f1d380a6550 msgr2=0x7f1d3809edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:32.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:32.164+0000 7f1d3fb72700 1 -- 172.21.15.16:0/3867030416 shutdown_connections 2023-09-30T22:31:32.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:32.164+0000 7f1d3fb72700 1 -- 172.21.15.16:0/3867030416 wait complete. 2023-09-30T22:31:32.178 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:31:32.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:32.179+0000 7fde0f7fe700 1 -- 172.21.15.16:0/1768924237 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fde080747f0 con 0x7fde080a0090 2023-09-30T22:31:32.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:32.179+0000 7fddfdffb700 1 -- 172.21.15.16:0/1768924237 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fde00040020 con 0x7fde080a0090 2023-09-30T22:31:32.180 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:31:32.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:32.182+0000 7fde0f7fe700 1 -- 172.21.15.16:0/1768924237 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fde080a0090 msgr2=0x7fde08139530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:32.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:32.182+0000 7fde0f7fe700 1 --2- 172.21.15.16:0/1768924237 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fde080a0090 0x7fde08139530 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fde0000efc0 tx=0x7fde0000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:31:32.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:32.182+0000 7fde0f7fe700 1 -- 172.21.15.16:0/1768924237 shutdown_connections 2023-09-30T22:31:32.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:32.182+0000 7fde0f7fe700 1 --2- 172.21.15.16:0/1768924237 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fde080a0090 0x7fde08139530 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:32.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:32.182+0000 7fde0f7fe700 1 --2- 172.21.15.16:0/1768924237 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fde0809c650 0x7fde08138ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:32.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:32.182+0000 7fde0f7fe700 1 --2- 172.21.15.16:0/1768924237 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde0809cf60 0x7fde08138ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:32.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:32.182+0000 7fde0f7fe700 1 -- 172.21.15.16:0/1768924237 >> 172.21.15.16:0/1768924237 conn(0x7fde080a3f70 msgr2=0x7fde080a36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:32.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:32.182+0000 7fde0f7fe700 1 -- 172.21.15.16:0/1768924237 shutdown_connections 2023-09-30T22:31:32.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:32.182+0000 7fde0f7fe700 1 -- 172.21.15.16:0/1768924237 wait complete. 2023-09-30T22:31:32.195 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:31:33.017 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:33.113 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:33.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.164+0000 7f7d9d776700 1 Processor -- start 2023-09-30T22:31:33.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.164+0000 7f7d9d776700 1 -- start start 2023-09-30T22:31:33.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.164+0000 7f7d9d776700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7d9809e850 0x7f7d9809ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:33.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.164+0000 7f7d9d776700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7d9809f160 0x7f7d98133c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:33.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.164+0000 7f7d9d776700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7d9809f5b0 0x7f7d981341a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:33.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.164+0000 7f7d9d776700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d9806bec0 con 0x7f7d9809f5b0 2023-09-30T22:31:33.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.164+0000 7f7d9d776700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d980a0030 con 0x7f7d9809e850 2023-09-30T22:31:33.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.164+0000 7f7d9d776700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7d980a0830 con 0x7f7d9809f160 2023-09-30T22:31:33.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.164+0000 7f7d96ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7d9809e850 0x7f7d9809ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:33.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.164+0000 7f7d967fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7d9809f160 0x7f7d98133c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:33.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.164+0000 7f7d96ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7d9809e850 0x7f7d9809ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54234/0 (socket says 172.21.15.16:54234) 2023-09-30T22:31:33.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.164+0000 7f7d96ffd700 1 -- 172.21.15.16:0/2199438375 learned_addr learned my addr 172.21.15.16:0/2199438375 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:33.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.164+0000 7f7d977fe700 1 --2- 172.21.15.16:0/2199438375 >> v2:172.21.15.16:3300/0 conn(0x7f7d9809f5b0 0x7f7d981341a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:33.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.165+0000 7f7d967fc700 1 -- 172.21.15.16:0/2199438375 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7d9809e850 msgr2=0x7f7d9809ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:33.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.165+0000 7f7d967fc700 1 --2- 172.21.15.16:0/2199438375 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7d9809e850 0x7f7d9809ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:33.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.165+0000 7f7d967fc700 1 -- 172.21.15.16:0/2199438375 >> v2:172.21.15.16:3300/0 conn(0x7f7d9809f5b0 msgr2=0x7f7d981341a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:33.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.165+0000 7f7d967fc700 1 --2- 172.21.15.16:0/2199438375 >> v2:172.21.15.16:3300/0 conn(0x7f7d9809f5b0 0x7f7d981341a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:33.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.165+0000 7f7d967fc700 1 -- 172.21.15.16:0/2199438375 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d9806caa0 con 0x7f7d9809f160 2023-09-30T22:31:33.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.165+0000 7f7d96ffd700 1 --2- 172.21.15.16:0/2199438375 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7d9809e850 0x7f7d9809ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:33.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.165+0000 7f7d977fe700 1 --2- 172.21.15.16:0/2199438375 >> v2:172.21.15.16:3300/0 conn(0x7f7d9809f5b0 0x7f7d981341a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:33.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.165+0000 7f7d967fc700 1 --2- 172.21.15.16:0/2199438375 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7d9809f160 0x7f7d98133c60 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c009840 tx=0x7f7d8c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40782a162046b516 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:33.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.165+0000 7f7d95ffb700 1 -- 172.21.15.16:0/2199438375 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c011830 con 0x7f7d9809f160 2023-09-30T22:31:33.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.165+0000 7f7d95ffb700 1 -- 172.21.15.16:0/2199438375 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d8c011990 con 0x7f7d9809f160 2023-09-30T22:31:33.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.165+0000 7f7d95ffb700 1 -- 172.21.15.16:0/2199438375 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c011d20 con 0x7f7d9809f160 2023-09-30T22:31:33.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.166+0000 7f7d9d776700 1 -- 172.21.15.16:0/2199438375 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7d9809f160 msgr2=0x7f7d98133c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:33.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.166+0000 7f7d9d776700 1 --2- 172.21.15.16:0/2199438375 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7d9809f160 0x7f7d98133c60 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c009840 tx=0x7f7d8c009b50 comp rx=0 tx=0).stop 2023-09-30T22:31:33.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.166+0000 7f7d9d776700 1 -- 172.21.15.16:0/2199438375 shutdown_connections 2023-09-30T22:31:33.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.166+0000 7f7d9d776700 1 --2- 172.21.15.16:0/2199438375 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7d9809e850 0x7f7d9809ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:33.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.166+0000 7f7d9d776700 1 --2- 172.21.15.16:0/2199438375 >> v2:172.21.15.16:3300/0 conn(0x7f7d9809f5b0 0x7f7d981341a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:33.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.166+0000 7f7d9d776700 1 --2- 172.21.15.16:0/2199438375 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7d9809f160 0x7f7d98133c60 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:33.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.166+0000 7f7d9d776700 1 -- 172.21.15.16:0/2199438375 >> 172.21.15.16:0/2199438375 conn(0x7f7d980a3f70 msgr2=0x7f7d980a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:33.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.166+0000 7f7d9d776700 1 -- 172.21.15.16:0/2199438375 shutdown_connections 2023-09-30T22:31:33.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.166+0000 7f7d9d776700 1 -- 172.21.15.16:0/2199438375 wait complete. 2023-09-30T22:31:33.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.167+0000 7f7d9d776700 1 Processor -- start 2023-09-30T22:31:33.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.167+0000 7f7d9d776700 1 -- start start 2023-09-30T22:31:33.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.168+0000 7f7d9d776700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7d9809e850 0x7f7d98120520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:33.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.168+0000 7f7d9d776700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7d9809f160 0x7f7d98120a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:33.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.168+0000 7f7d9d776700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d9809f5b0 0x7f7d98120fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:33.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.168+0000 7f7d9d776700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d98123fb0 con 0x7f7d9809f5b0 2023-09-30T22:31:33.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.168+0000 7f7d9d776700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d98122680 con 0x7f7d9809f160 2023-09-30T22:31:33.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.168+0000 7f7d9d776700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7d98123740 con 0x7f7d9809e850 2023-09-30T22:31:33.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.168+0000 7f7d96ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7d9809e850 0x7f7d98120520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:33.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.168+0000 7f7d977fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d9809f5b0 0x7f7d98120fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:33.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.168+0000 7f7d96ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7d9809e850 0x7f7d98120520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:33782/0 (socket says 172.21.15.16:33782) 2023-09-30T22:31:33.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.168+0000 7f7d96ffd700 1 -- 172.21.15.16:0/3079367198 learned_addr learned my addr 172.21.15.16:0/3079367198 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:33.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.168+0000 7f7d967fc700 1 --2- 172.21.15.16:0/3079367198 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7d9809f160 0x7f7d98120a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:33.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.168+0000 7f7d96ffd700 1 -- 172.21.15.16:0/3079367198 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7d9809f160 msgr2=0x7f7d98120a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:33.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.168+0000 7f7d96ffd700 1 --2- 172.21.15.16:0/3079367198 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7d9809f160 0x7f7d98120a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:33.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.168+0000 7f7d96ffd700 1 -- 172.21.15.16:0/3079367198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d9809f5b0 msgr2=0x7f7d98120fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:33.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.168+0000 7f7d96ffd700 1 --2- 172.21.15.16:0/3079367198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d9809f5b0 0x7f7d98120fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:33.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.168+0000 7f7d96ffd700 1 -- 172.21.15.16:0/3079367198 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d88009b30 con 0x7f7d9809e850 2023-09-30T22:31:33.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.168+0000 7f7d977fe700 1 --2- 172.21.15.16:0/3079367198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d9809f5b0 0x7f7d98120fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:33.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.168+0000 7f7d967fc700 1 --2- 172.21.15.16:0/3079367198 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7d9809f160 0x7f7d98120a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:33.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.169+0000 7f7d96ffd700 1 --2- 172.21.15.16:0/3079367198 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7d9809e850 0x7f7d98120520 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7d88009f20 tx=0x7f7d8800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:33.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.169+0000 7f7d7f7fe700 1 -- 172.21.15.16:0/3079367198 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8800cb60 con 0x7f7d9809e850 2023-09-30T22:31:33.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.169+0000 7f7d7f7fe700 1 -- 172.21.15.16:0/3079367198 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d8800f040 con 0x7f7d9809e850 2023-09-30T22:31:33.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.169+0000 7f7d9d776700 1 -- 172.21.15.16:0/3079367198 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d900094a0 con 0x7f7d9809e850 2023-09-30T22:31:33.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.169+0000 7f7d7f7fe700 1 -- 172.21.15.16:0/3079367198 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8800cec0 con 0x7f7d9809e850 2023-09-30T22:31:33.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.169+0000 7f7d9d776700 1 -- 172.21.15.16:0/3079367198 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d8c0094a0 con 0x7f7d9809e850 2023-09-30T22:31:33.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.170+0000 7f7d7f7fe700 1 -- 172.21.15.16:0/3079367198 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7d88014480 con 0x7f7d9809e850 2023-09-30T22:31:33.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.170+0000 7f7d7f7fe700 1 -- 172.21.15.16:0/3079367198 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f7d8803fd60 con 0x7f7d9809e850 2023-09-30T22:31:33.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.170+0000 7f7d9d776700 1 -- 172.21.15.16:0/3079367198 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d980a35b0 con 0x7f7d9809e850 2023-09-30T22:31:33.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.175+0000 7f7d7f7fe700 1 -- 172.21.15.16:0/3079367198 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7d88012930 con 0x7f7d9809e850 2023-09-30T22:31:33.179 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:33.196 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:33.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.269+0000 7ff78ec4b700 1 Processor -- start 2023-09-30T22:31:33.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.269+0000 7ff78ec4b700 1 -- start start 2023-09-30T22:31:33.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.270+0000 7ff78ec4b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff78809d6f0 0x7ff78809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:33.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.270+0000 7ff78ec4b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff78809e000 0x7ff7881339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:33.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.270+0000 7ff78ec4b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff78809b120 con 0x7ff78809e410 2023-09-30T22:31:33.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.270+0000 7ff78ec4b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff78806b460 con 0x7ff78809d6f0 2023-09-30T22:31:33.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.270+0000 7ff78ec4b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff78809a7e0 con 0x7ff78809e000 2023-09-30T22:31:33.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.271+0000 7ff78c9e7700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff78809d6f0 0x7ff78809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:33.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.271+0000 7ff787fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff78809e000 0x7ff7881339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:33.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.271+0000 7ff78c9e7700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff78809d6f0 0x7ff78809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53560/0 (socket says 172.21.15.16:53560) 2023-09-30T22:31:33.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.271+0000 7ff78d1e8700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff78809e410 0x7ff788133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44116/0 (socket says 172.21.15.16:44116) 2023-09-30T22:31:33.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.271+0000 7ff78c9e7700 1 -- 172.21.15.16:0/429308278 learned_addr learned my addr 172.21.15.16:0/429308278 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:33.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.271+0000 7ff78c9e7700 1 -- 172.21.15.16:0/429308278 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff78809e000 msgr2=0x7ff7881339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:33.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.271+0000 7ff78c9e7700 1 --2- 172.21.15.16:0/429308278 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff78809e000 0x7ff7881339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:33.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.271+0000 7ff78c9e7700 1 -- 172.21.15.16:0/429308278 >> v1:172.21.15.16:6789/0 conn(0x7ff78809e410 legacy=0x7ff788133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:33.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.271+0000 7ff78c9e7700 1 -- 172.21.15.16:0/429308278 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff78809a500 con 0x7ff78809d6f0 2023-09-30T22:31:33.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.271+0000 7ff78c9e7700 1 --2- 172.21.15.16:0/429308278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff78809d6f0 0x7ff78809dac0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff77c0098f0 tx=0x7ff77c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dc50c5850483174 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:33.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.271+0000 7ff7877fe700 1 -- 172.21.15.16:0/429308278 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77c011830 con 0x7ff78809d6f0 2023-09-30T22:31:33.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.271+0000 7ff7877fe700 1 -- 172.21.15.16:0/429308278 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff77c011990 con 0x7ff78809d6f0 2023-09-30T22:31:33.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.272+0000 7ff7877fe700 1 -- 172.21.15.16:0/429308278 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77c011d20 con 0x7ff78809d6f0 2023-09-30T22:31:33.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.272+0000 7ff78ec4b700 1 -- 172.21.15.16:0/429308278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff78809d6f0 msgr2=0x7ff78809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:33.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.273+0000 7ff78ec4b700 1 --2- 172.21.15.16:0/429308278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff78809d6f0 0x7ff78809dac0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff77c0098f0 tx=0x7ff77c009c00 comp rx=0 tx=0).stop 2023-09-30T22:31:33.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.273+0000 7ff78ec4b700 1 -- 172.21.15.16:0/429308278 shutdown_connections 2023-09-30T22:31:33.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.273+0000 7ff78ec4b700 1 --2- 172.21.15.16:0/429308278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff78809d6f0 0x7ff78809dac0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:33.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.273+0000 7ff78ec4b700 1 --2- 172.21.15.16:0/429308278 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff78809e000 0x7ff7881339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:33.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.273+0000 7ff78ec4b700 1 -- 172.21.15.16:0/429308278 >> 172.21.15.16:0/429308278 conn(0x7ff788066f50 msgr2=0x7ff78809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:33.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.273+0000 7ff78ec4b700 1 -- 172.21.15.16:0/429308278 shutdown_connections 2023-09-30T22:31:33.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.273+0000 7ff78ec4b700 1 -- 172.21.15.16:0/429308278 wait complete. 2023-09-30T22:31:33.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.274+0000 7ff78ec4b700 1 Processor -- start 2023-09-30T22:31:33.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.274+0000 7ff78ec4b700 1 -- start start 2023-09-30T22:31:33.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.275+0000 7ff78ec4b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff78809d6f0 0x7ff7881a5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:33.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.275+0000 7ff78ec4b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff78809e000 0x7ff7881a63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:33.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.275+0000 7ff78ec4b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff78809e410 0x7ff7881a6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:33.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.275+0000 7ff78ec4b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7881a9960 con 0x7ff78809d6f0 2023-09-30T22:31:33.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.275+0000 7ff78ec4b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7881a7af0 con 0x7ff78809e000 2023-09-30T22:31:33.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.275+0000 7ff78ec4b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff7880a32f0 con 0x7ff78809e410 2023-09-30T22:31:33.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.276+0000 7ff78c9e7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff78809d6f0 0x7ff7881a5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:33.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.276+0000 7ff78d1e8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff78809e410 0x7ff7881a6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:33.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.276+0000 7ff78c9e7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff78809d6f0 0x7ff7881a5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53576/0 (socket says 172.21.15.16:53576) 2023-09-30T22:31:33.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.276+0000 7ff78c9e7700 1 -- 172.21.15.16:0/1924823333 learned_addr learned my addr 172.21.15.16:0/1924823333 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:33.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.276+0000 7ff78d1e8700 1 --2- 172.21.15.16:0/1924823333 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff78809e410 0x7ff7881a6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:33808/0 (socket says 172.21.15.16:33808) 2023-09-30T22:31:33.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.276+0000 7ff787fff700 1 --2- 172.21.15.16:0/1924823333 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff78809e000 0x7ff7881a63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:33.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.276+0000 7ff78d1e8700 1 -- 172.21.15.16:0/1924823333 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff78809e000 msgr2=0x7ff7881a63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:33.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.276+0000 7ff78d1e8700 1 --2- 172.21.15.16:0/1924823333 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff78809e000 0x7ff7881a63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:33.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.276+0000 7ff78d1e8700 1 -- 172.21.15.16:0/1924823333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff78809d6f0 msgr2=0x7ff7881a5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:33.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.276+0000 7ff78d1e8700 1 --2- 172.21.15.16:0/1924823333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff78809d6f0 0x7ff7881a5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:33.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.276+0000 7ff78d1e8700 1 -- 172.21.15.16:0/1924823333 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff77c009550 con 0x7ff78809e410 2023-09-30T22:31:33.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.276+0000 7ff78c9e7700 1 --2- 172.21.15.16:0/1924823333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff78809d6f0 0x7ff7881a5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:33.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.276+0000 7ff787fff700 1 --2- 172.21.15.16:0/1924823333 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff78809e000 0x7ff7881a63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:33.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.276+0000 7ff78d1e8700 1 --2- 172.21.15.16:0/1924823333 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff78809e410 0x7ff7881a6910 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff78000cb10 tx=0x7ff78000ce20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:33.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.277+0000 7ff7857fa700 1 -- 172.21.15.16:0/1924823333 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78000a9e0 con 0x7ff78809e410 2023-09-30T22:31:33.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.277+0000 7ff7857fa700 1 -- 172.21.15.16:0/1924823333 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff78000ab40 con 0x7ff78809e410 2023-09-30T22:31:33.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.277+0000 7ff78ec4b700 1 -- 172.21.15.16:0/1924823333 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff78809c710 con 0x7ff78809e410 2023-09-30T22:31:33.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.277+0000 7ff7857fa700 1 -- 172.21.15.16:0/1924823333 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff780007070 con 0x7ff78809e410 2023-09-30T22:31:33.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.277+0000 7ff78ec4b700 1 -- 172.21.15.16:0/1924823333 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7881a8850 con 0x7ff78809e410 2023-09-30T22:31:33.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.278+0000 7ff7857fa700 1 -- 172.21.15.16:0/1924823333 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff7800071d0 con 0x7ff78809e410 2023-09-30T22:31:33.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.278+0000 7ff7857fa700 1 -- 172.21.15.16:0/1924823333 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7ff78003f320 con 0x7ff78809e410 2023-09-30T22:31:33.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.279+0000 7ff78ec4b700 1 -- 172.21.15.16:0/1924823333 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff788074ea0 con 0x7ff78809e410 2023-09-30T22:31:33.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.284+0000 7ff7857fa700 1 -- 172.21.15.16:0/1924823333 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff780017080 con 0x7ff78809e410 2023-09-30T22:31:33.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.336+0000 7f6fb80f6700 1 Processor -- start 2023-09-30T22:31:33.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.337+0000 7f6fb80f6700 1 -- start start 2023-09-30T22:31:33.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.337+0000 7f6fb80f6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6fb006a770 0x7f6fb006ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:33.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.337+0000 7f6fb80f6700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6fb006b080 0x7f6fb0131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:33.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.337+0000 7f6fb80f6700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fb009ece0 con 0x7f6fb0131b40 2023-09-30T22:31:33.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.337+0000 7f6fb80f6700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fb009e760 con 0x7f6fb006b080 2023-09-30T22:31:33.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.337+0000 7f6fb80f6700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb009e040 con 0x7f6fb006a770 2023-09-30T22:31:33.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.338+0000 7f6fb5691700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6fb006b080 0x7f6fb0131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:33.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.338+0000 7f6fb5691700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6fb006b080 0x7f6fb0131600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53588/0 (socket says 172.21.15.16:53588) 2023-09-30T22:31:33.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.338+0000 7f6fb5e92700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6fb006a770 0x7f6fb006ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:33.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.338+0000 7f6fb5691700 1 -- 172.21.15.16:0/993561 learned_addr learned my addr 172.21.15.16:0/993561 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:33.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.338+0000 7f6fb5e92700 1 -- 172.21.15.16:0/993561 >> v2:172.21.15.16:3300/0 conn(0x7f6fb006b080 msgr2=0x7f6fb0131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:33.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.338+0000 7f6fb5e92700 1 --2- 172.21.15.16:0/993561 >> v2:172.21.15.16:3300/0 conn(0x7f6fb006b080 0x7f6fb0131600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:33.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.338+0000 7f6fb5e92700 1 -- 172.21.15.16:0/993561 >> v1:172.21.15.16:6789/0 conn(0x7f6fb0131b40 legacy=0x7f6fb0138850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:33.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.338+0000 7f6fb5e92700 1 -- 172.21.15.16:0/993561 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb009f630 con 0x7f6fb006a770 2023-09-30T22:31:33.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.338+0000 7f6fb5691700 1 --2- 172.21.15.16:0/993561 >> v2:172.21.15.16:3300/0 conn(0x7f6fb006b080 0x7f6fb0131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:33.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.338+0000 7f6fb5e92700 1 --2- 172.21.15.16:0/993561 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6fb006a770 0x7f6fb006ab40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6fac009840 tx=0x7f6fac009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9461c9146e2c012c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:33.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.339+0000 7f6fb4e90700 1 -- 172.21.15.16:0/993561 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fac004230 con 0x7f6fb006a770 2023-09-30T22:31:33.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.339+0000 7f6fb4e90700 1 -- 172.21.15.16:0/993561 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fac004390 con 0x7f6fb006a770 2023-09-30T22:31:33.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.339+0000 7f6fb4e90700 1 -- 172.21.15.16:0/993561 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fac0036a0 con 0x7f6fb006a770 2023-09-30T22:31:33.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.339+0000 7f6fb80f6700 1 -- 172.21.15.16:0/993561 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6fb006a770 msgr2=0x7f6fb006ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:33.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.339+0000 7f6fb80f6700 1 --2- 172.21.15.16:0/993561 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6fb006a770 0x7f6fb006ab40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6fac009840 tx=0x7f6fac009b50 comp rx=0 tx=0).stop 2023-09-30T22:31:33.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.339+0000 7f6fb80f6700 1 -- 172.21.15.16:0/993561 shutdown_connections 2023-09-30T22:31:33.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.339+0000 7f6fb80f6700 1 --2- 172.21.15.16:0/993561 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6fb006a770 0x7f6fb006ab40 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:33.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.339+0000 7f6fb80f6700 1 --2- 172.21.15.16:0/993561 >> v2:172.21.15.16:3300/0 conn(0x7f6fb006b080 0x7f6fb0131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:33.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.339+0000 7f6fb80f6700 1 -- 172.21.15.16:0/993561 >> 172.21.15.16:0/993561 conn(0x7f6fb0066f50 msgr2=0x7f6fb009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:33.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.340+0000 7f6fb80f6700 1 -- 172.21.15.16:0/993561 shutdown_connections 2023-09-30T22:31:33.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.340+0000 7f6fb80f6700 1 -- 172.21.15.16:0/993561 wait complete. 2023-09-30T22:31:33.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.341+0000 7f6fb80f6700 1 Processor -- start 2023-09-30T22:31:33.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.341+0000 7f6fb80f6700 1 -- start start 2023-09-30T22:31:33.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.342+0000 7f6fb80f6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6fb006a770 0x7f6fb011cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:33.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.342+0000 7f6fb80f6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6fb006b080 0x7f6fb011d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:33.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.342+0000 7f6fb80f6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fb0131b40 0x7f6fb01c6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:33.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.342+0000 7f6fb80f6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb011e220 con 0x7f6fb0131b40 2023-09-30T22:31:33.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.342+0000 7f6fb80f6700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb0130fa0 con 0x7f6fb006b080 2023-09-30T22:31:33.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.342+0000 7f6fb80f6700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb0131600 con 0x7f6fb006a770 2023-09-30T22:31:33.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.342+0000 7f6fb5691700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6fb006b080 0x7f6fb011d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:33.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.342+0000 7f6fb5691700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6fb006b080 0x7f6fb011d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54272/0 (socket says 172.21.15.16:54272) 2023-09-30T22:31:33.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.342+0000 7f6fb5691700 1 -- 172.21.15.16:0/480864752 learned_addr learned my addr 172.21.15.16:0/480864752 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:33.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.342+0000 7f6fb6693700 1 --2- 172.21.15.16:0/480864752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fb0131b40 0x7f6fb01c6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:33.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.342+0000 7f6fb5691700 1 -- 172.21.15.16:0/480864752 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6fb006a770 msgr2=0x7f6fb011cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:33.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.343+0000 7f6fb5691700 1 --2- 172.21.15.16:0/480864752 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6fb006a770 0x7f6fb011cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:33.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.343+0000 7f6fb5691700 1 -- 172.21.15.16:0/480864752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fb0131b40 msgr2=0x7f6fb01c6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:33.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.343+0000 7f6fb5691700 1 --2- 172.21.15.16:0/480864752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fb0131b40 0x7f6fb01c6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:33.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.343+0000 7f6fb5691700 1 -- 172.21.15.16:0/480864752 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa0009550 con 0x7f6fb006b080 2023-09-30T22:31:33.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.343+0000 7f6fb5691700 1 --2- 172.21.15.16:0/480864752 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6fb006b080 0x7f6fb011d480 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6fa000ebe0 tx=0x7f6fa000eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:33.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.343+0000 7f6fa67fc700 1 -- 172.21.15.16:0/480864752 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa000c850 con 0x7f6fb006b080 2023-09-30T22:31:33.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.343+0000 7f6fa67fc700 1 -- 172.21.15.16:0/480864752 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fa000c9b0 con 0x7f6fb006b080 2023-09-30T22:31:33.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.343+0000 7f6fb80f6700 1 -- 172.21.15.16:0/480864752 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fac0094a0 con 0x7f6fb006b080 2023-09-30T22:31:33.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.343+0000 7f6fa67fc700 1 -- 172.21.15.16:0/480864752 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa000cd60 con 0x7f6fb006b080 2023-09-30T22:31:33.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.343+0000 7f6fb80f6700 1 -- 172.21.15.16:0/480864752 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fb01201c0 con 0x7f6fb006b080 2023-09-30T22:31:33.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.344+0000 7f6fb80f6700 1 -- 172.21.15.16:0/480864752 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fb0074ea0 con 0x7f6fb006b080 2023-09-30T22:31:33.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.345+0000 7f6fa67fc700 1 -- 172.21.15.16:0/480864752 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6fa0013470 con 0x7f6fb006b080 2023-09-30T22:31:33.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.345+0000 7f6fa67fc700 1 -- 172.21.15.16:0/480864752 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f6fa003f970 con 0x7f6fb006b080 2023-09-30T22:31:33.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.350+0000 7f6fa67fc700 1 -- 172.21.15.16:0/480864752 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6fa00232f0 con 0x7f6fb006b080 2023-09-30T22:31:33.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.370+0000 7f8387d2c700 1 Processor -- start 2023-09-30T22:31:33.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.370+0000 7f8387d2c700 1 -- start start 2023-09-30T22:31:33.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.370+0000 7f8387d2c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f838006a770 0x7f838009d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:33.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.370+0000 7f8387d2c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f838013d1f0 0x7f838013f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:33.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.370+0000 7f8387d2c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f838009a810 con 0x7f838009dc30 2023-09-30T22:31:33.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.370+0000 7f8387d2c700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f838006ab40 con 0x7f838013d1f0 2023-09-30T22:31:33.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.370+0000 7f8387d2c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f838009ebc0 con 0x7f838006a770 2023-09-30T22:31:33.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.371+0000 7f83852c7700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f838009dc30 0x7f838009e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44132/0 (socket says 172.21.15.16:44132) 2023-09-30T22:31:33.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.371+0000 7f83852c7700 1 -- 172.21.15.16:0/3053184157 learned_addr learned my addr 172.21.15.16:0/3053184157 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:31:33.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.371+0000 7f83862c9700 1 --2- 172.21.15.16:0/3053184157 >> v2:172.21.15.16:3300/0 conn(0x7f838013d1f0 0x7f838013f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:33.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.371+0000 7f8385ac8700 1 --2- 172.21.15.16:0/3053184157 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f838006a770 0x7f838009d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:33.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.371+0000 7f8384ac6700 1 -- 172.21.15.16:0/3053184157 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3953582408 0 0) 0x7f838009a810 con 0x7f838009dc30 2023-09-30T22:31:33.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.371+0000 7f8384ac6700 1 -- 172.21.15.16:0/3053184157 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f836c0035b0 con 0x7f838009dc30 2023-09-30T22:31:33.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.371+0000 7f83862c9700 1 -- 172.21.15.16:0/3053184157 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f838006a770 msgr2=0x7f838009d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:33.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.371+0000 7f83862c9700 1 --2- 172.21.15.16:0/3053184157 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f838006a770 0x7f838009d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:33.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.371+0000 7f83862c9700 1 -- 172.21.15.16:0/3053184157 >> v1:172.21.15.16:6789/0 conn(0x7f838009dc30 legacy=0x7f838009e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:33.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.371+0000 7f83862c9700 1 -- 172.21.15.16:0/3053184157 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f838009e920 con 0x7f838013d1f0 2023-09-30T22:31:33.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.371+0000 7f83862c9700 1 --2- 172.21.15.16:0/3053184157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f838013d1f0 0x7f838013f5d0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f8378009dc0 tx=0x7f837800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b612d43e4432bd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:33.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.372+0000 7f8384ac6700 1 -- 172.21.15.16:0/3053184157 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f837800e070 con 0x7f838013d1f0 2023-09-30T22:31:33.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.372+0000 7f8384ac6700 1 -- 172.21.15.16:0/3053184157 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f837800bae0 con 0x7f838013d1f0 2023-09-30T22:31:33.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.372+0000 7f8384ac6700 1 -- 172.21.15.16:0/3053184157 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f837800be90 con 0x7f838013d1f0 2023-09-30T22:31:33.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.372+0000 7f8387d2c700 1 -- 172.21.15.16:0/3053184157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f838013d1f0 msgr2=0x7f838013f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:33.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.372+0000 7f8387d2c700 1 --2- 172.21.15.16:0/3053184157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f838013d1f0 0x7f838013f5d0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f8378009dc0 tx=0x7f837800b330 comp rx=0 tx=0).stop 2023-09-30T22:31:33.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.372+0000 7f8387d2c700 1 -- 172.21.15.16:0/3053184157 shutdown_connections 2023-09-30T22:31:33.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.372+0000 7f8387d2c700 1 --2- 172.21.15.16:0/3053184157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f838013d1f0 0x7f838013f5d0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:33.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.372+0000 7f8387d2c700 1 --2- 172.21.15.16:0/3053184157 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f838006a770 0x7f838009d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:33.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.372+0000 7f8387d2c700 1 -- 172.21.15.16:0/3053184157 >> 172.21.15.16:0/3053184157 conn(0x7f8380066f50 msgr2=0x7f838009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:33.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.372+0000 7f8387d2c700 1 -- 172.21.15.16:0/3053184157 shutdown_connections 2023-09-30T22:31:33.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.373+0000 7f8387d2c700 1 -- 172.21.15.16:0/3053184157 wait complete. 2023-09-30T22:31:33.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.374+0000 7f8387d2c700 1 Processor -- start 2023-09-30T22:31:33.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.374+0000 7f8387d2c700 1 -- start start 2023-09-30T22:31:33.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.375+0000 7f8387d2c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f838006a770 0x7f838012cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:33.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.375+0000 7f8387d2c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f838009dc30 0x7f838012d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:33.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.375+0000 7f8387d2c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f838013d1f0 0x7f838012d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:33.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.375+0000 7f8387d2c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f838012f6f0 con 0x7f838009dc30 2023-09-30T22:31:33.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.375+0000 7f8387d2c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f838012e6a0 con 0x7f838006a770 2023-09-30T22:31:33.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.375+0000 7f8387d2c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f838012f9b0 con 0x7f838013d1f0 2023-09-30T22:31:33.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.375+0000 7f83852c7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f838009dc30 0x7f838012d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:33.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.375+0000 7f83862c9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f838013d1f0 0x7f838012d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:33.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.375+0000 7f83852c7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f838009dc30 0x7f838012d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53614/0 (socket says 172.21.15.16:53614) 2023-09-30T22:31:33.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.375+0000 7f83852c7700 1 -- 172.21.15.16:0/3395323610 learned_addr learned my addr 172.21.15.16:0/3395323610 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:33.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.375+0000 7f83852c7700 1 -- 172.21.15.16:0/3395323610 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f838013d1f0 msgr2=0x7f838012d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:33.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.375+0000 7f83852c7700 1 --2- 172.21.15.16:0/3395323610 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f838013d1f0 0x7f838012d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:33.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.375+0000 7f83852c7700 1 -- 172.21.15.16:0/3395323610 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f838006a770 msgr2=0x7f838012cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:33.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.375+0000 7f83852c7700 1 --2- 172.21.15.16:0/3395323610 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f838006a770 0x7f838012cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:33.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.375+0000 7f83852c7700 1 -- 172.21.15.16:0/3395323610 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8378009a70 con 0x7f838009dc30 2023-09-30T22:31:33.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.375+0000 7f83862c9700 1 --2- 172.21.15.16:0/3395323610 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f838013d1f0 0x7f838012d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:33.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.376+0000 7f83852c7700 1 --2- 172.21.15.16:0/3395323610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f838009dc30 0x7f838012d2f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f8370005910 tx=0x7f837000c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:33.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.376+0000 7f83767fc700 1 -- 172.21.15.16:0/3395323610 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8370005d70 con 0x7f838009dc30 2023-09-30T22:31:33.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.376+0000 7f83767fc700 1 -- 172.21.15.16:0/3395323610 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83700070d0 con 0x7f838009dc30 2023-09-30T22:31:33.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.376+0000 7f8387d2c700 1 -- 172.21.15.16:0/3395323610 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83801404c0 con 0x7f838009dc30 2023-09-30T22:31:33.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.376+0000 7f83767fc700 1 -- 172.21.15.16:0/3395323610 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8370007440 con 0x7f838009dc30 2023-09-30T22:31:33.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.376+0000 7f8387d2c700 1 -- 172.21.15.16:0/3395323610 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8380072730 con 0x7f838009dc30 2023-09-30T22:31:33.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.377+0000 7f8387d2c700 1 -- 172.21.15.16:0/3395323610 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8380074ea0 con 0x7f838009dc30 2023-09-30T22:31:33.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.377+0000 7f83767fc700 1 -- 172.21.15.16:0/3395323610 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f83700075a0 con 0x7f838009dc30 2023-09-30T22:31:33.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.377+0000 7f83767fc700 1 -- 172.21.15.16:0/3395323610 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f837003ebc0 con 0x7f838009dc30 2023-09-30T22:31:33.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.382+0000 7f83767fc700 1 -- 172.21.15.16:0/3395323610 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8370019d20 con 0x7f838009dc30 2023-09-30T22:31:33.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.450+0000 7f7d9d776700 1 -- 172.21.15.16:0/3079367198 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7d98123b30 con 0x7f7d9809e850 2023-09-30T22:31:33.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.450+0000 7f7d7f7fe700 1 -- 172.21.15.16:0/3079367198 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7d98123b30 con 0x7f7d9809e850 2023-09-30T22:31:33.450 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:31:33.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.453+0000 7f7d9d776700 1 -- 172.21.15.16:0/3079367198 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7d9809e850 msgr2=0x7f7d98120520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:33.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.453+0000 7f7d9d776700 1 --2- 172.21.15.16:0/3079367198 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7d9809e850 0x7f7d98120520 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7d88009f20 tx=0x7f7d8800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:31:33.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.454+0000 7f7d9d776700 1 -- 172.21.15.16:0/3079367198 shutdown_connections 2023-09-30T22:31:33.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.454+0000 7f7d9d776700 1 --2- 172.21.15.16:0/3079367198 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d9809f5b0 0x7f7d98120fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:33.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.454+0000 7f7d9d776700 1 --2- 172.21.15.16:0/3079367198 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7d9809e850 0x7f7d98120520 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:33.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.454+0000 7f7d9d776700 1 --2- 172.21.15.16:0/3079367198 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7d9809f160 0x7f7d98120a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:33.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.454+0000 7f7d9d776700 1 -- 172.21.15.16:0/3079367198 >> 172.21.15.16:0/3079367198 conn(0x7f7d980a3f70 msgr2=0x7f7d981b97b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:33.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.454+0000 7f7d9d776700 1 -- 172.21.15.16:0/3079367198 shutdown_connections 2023-09-30T22:31:33.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.454+0000 7f7d9d776700 1 -- 172.21.15.16:0/3079367198 wait complete. 2023-09-30T22:31:33.469 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:31:33.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.520+0000 7ff78ec4b700 1 -- 172.21.15.16:0/1924823333 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff78806c360 con 0x7ff78809e410 2023-09-30T22:31:33.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.521+0000 7ff7857fa700 1 -- 172.21.15.16:0/1924823333 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff78001b3b0 con 0x7ff78809e410 2023-09-30T22:31:33.521 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:31:33.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.524+0000 7ff78ec4b700 1 -- 172.21.15.16:0/1924823333 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff78809e410 msgr2=0x7ff7881a6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:33.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.524+0000 7ff78ec4b700 1 --2- 172.21.15.16:0/1924823333 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff78809e410 0x7ff7881a6910 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff78000cb10 tx=0x7ff78000ce20 comp rx=0 tx=0).stop 2023-09-30T22:31:33.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.524+0000 7ff78ec4b700 1 -- 172.21.15.16:0/1924823333 shutdown_connections 2023-09-30T22:31:33.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.524+0000 7ff78ec4b700 1 --2- 172.21.15.16:0/1924823333 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff78809e410 0x7ff7881a6910 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:33.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.524+0000 7ff78ec4b700 1 --2- 172.21.15.16:0/1924823333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff78809d6f0 0x7ff7881a5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:33.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.524+0000 7ff78ec4b700 1 --2- 172.21.15.16:0/1924823333 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff78809e000 0x7ff7881a63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:33.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.524+0000 7ff78ec4b700 1 -- 172.21.15.16:0/1924823333 >> 172.21.15.16:0/1924823333 conn(0x7ff788066f50 msgr2=0x7ff7881bae40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:33.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.524+0000 7ff78ec4b700 1 -- 172.21.15.16:0/1924823333 shutdown_connections 2023-09-30T22:31:33.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.524+0000 7ff78ec4b700 1 -- 172.21.15.16:0/1924823333 wait complete. 2023-09-30T22:31:33.543 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:31:33.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.592+0000 7f6fb80f6700 1 -- 172.21.15.16:0/480864752 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f6fb011ee90 con 0x7f6fb006b080 2023-09-30T22:31:33.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.593+0000 7f6fa67fc700 1 -- 172.21.15.16:0/480864752 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6fa0021090 con 0x7f6fb006b080 2023-09-30T22:31:33.593 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:31:33.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.595+0000 7f6fb80f6700 1 -- 172.21.15.16:0/480864752 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6fb006b080 msgr2=0x7f6fb011d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:33.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.595+0000 7f6fb80f6700 1 --2- 172.21.15.16:0/480864752 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6fb006b080 0x7f6fb011d480 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6fa000ebe0 tx=0x7f6fa000eef0 comp rx=0 tx=0).stop 2023-09-30T22:31:33.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.595+0000 7f6fb80f6700 1 -- 172.21.15.16:0/480864752 shutdown_connections 2023-09-30T22:31:33.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.595+0000 7f6fb80f6700 1 --2- 172.21.15.16:0/480864752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fb0131b40 0x7f6fb01c6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:33.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.595+0000 7f6fb80f6700 1 --2- 172.21.15.16:0/480864752 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6fb006a770 0x7f6fb011cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:33.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.595+0000 7f6fb80f6700 1 --2- 172.21.15.16:0/480864752 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6fb006b080 0x7f6fb011d480 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:33.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.595+0000 7f6fb80f6700 1 -- 172.21.15.16:0/480864752 >> 172.21.15.16:0/480864752 conn(0x7f6fb0066f50 msgr2=0x7f6fb00a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:33.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.595+0000 7f6fb80f6700 1 -- 172.21.15.16:0/480864752 shutdown_connections 2023-09-30T22:31:33.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.595+0000 7f6fb80f6700 1 -- 172.21.15.16:0/480864752 wait complete. 2023-09-30T22:31:33.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.601+0000 7f8387d2c700 1 -- 172.21.15.16:0/3395323610 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f838009a3d0 con 0x7f838009dc30 2023-09-30T22:31:33.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.602+0000 7f83767fc700 1 -- 172.21.15.16:0/3395323610 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8370017030 con 0x7f838009dc30 2023-09-30T22:31:33.602 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:31:33.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.604+0000 7f8387d2c700 1 -- 172.21.15.16:0/3395323610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f838009dc30 msgr2=0x7f838012d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:33.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.604+0000 7f8387d2c700 1 --2- 172.21.15.16:0/3395323610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f838009dc30 0x7f838012d2f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f8370005910 tx=0x7f837000c9e0 comp rx=0 tx=0).stop 2023-09-30T22:31:33.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.604+0000 7f8387d2c700 1 -- 172.21.15.16:0/3395323610 shutdown_connections 2023-09-30T22:31:33.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.604+0000 7f8387d2c700 1 --2- 172.21.15.16:0/3395323610 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f838006a770 0x7f838012cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:33.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.604+0000 7f8387d2c700 1 --2- 172.21.15.16:0/3395323610 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f838013d1f0 0x7f838012d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:33.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.604+0000 7f8387d2c700 1 --2- 172.21.15.16:0/3395323610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f838009dc30 0x7f838012d2f0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:33.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.604+0000 7f8387d2c700 1 -- 172.21.15.16:0/3395323610 >> 172.21.15.16:0/3395323610 conn(0x7f8380066f50 msgr2=0x7f83800a3540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:33.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.604+0000 7f8387d2c700 1 -- 172.21.15.16:0/3395323610 shutdown_connections 2023-09-30T22:31:33.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:33.604+0000 7f8387d2c700 1 -- 172.21.15.16:0/3395323610 wait complete. 2023-09-30T22:31:33.611 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:31:33.619 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:31:34.470 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:34.544 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:34.612 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:34.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.618+0000 7f194f8a4700 1 Processor -- start 2023-09-30T22:31:34.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.618+0000 7f194f8a4700 1 -- start start 2023-09-30T22:31:34.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.618+0000 7f194f8a4700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f194809d6f0 0x7f194809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:34.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.618+0000 7f194f8a4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f194809e000 0x7f19481339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:34.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.618+0000 7f194f8a4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f194809e410 0x7f1948133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:34.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.618+0000 7f194f8a4700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f194806b460 con 0x7f194809e000 2023-09-30T22:31:34.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.618+0000 7f194f8a4700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f194809b120 con 0x7f194809d6f0 2023-09-30T22:31:34.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.618+0000 7f194f8a4700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f194809a7e0 con 0x7f194809e410 2023-09-30T22:31:34.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.619+0000 7f194ce3f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f194809e000 0x7f19481339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:34.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.619+0000 7f194d640700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f194809d6f0 0x7f194809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:34.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.619+0000 7f194ce3f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f194809e000 0x7f19481339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53622/0 (socket says 172.21.15.16:53622) 2023-09-30T22:31:34.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.619+0000 7f194ce3f700 1 -- 172.21.15.16:0/971482207 learned_addr learned my addr 172.21.15.16:0/971482207 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:34.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.619+0000 7f194de41700 1 --2- 172.21.15.16:0/971482207 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f194809e410 0x7f1948133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:34.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.619+0000 7f194de41700 1 -- 172.21.15.16:0/971482207 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f194809d6f0 msgr2=0x7f194809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:34.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.619+0000 7f194de41700 1 --2- 172.21.15.16:0/971482207 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f194809d6f0 0x7f194809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:34.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.619+0000 7f194de41700 1 -- 172.21.15.16:0/971482207 >> v2:172.21.15.16:3300/0 conn(0x7f194809e000 msgr2=0x7f19481339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:34.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.619+0000 7f194de41700 1 --2- 172.21.15.16:0/971482207 >> v2:172.21.15.16:3300/0 conn(0x7f194809e000 0x7f19481339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:34.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.619+0000 7f194de41700 1 -- 172.21.15.16:0/971482207 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f194809a500 con 0x7f194809e410 2023-09-30T22:31:34.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.619+0000 7f194ce3f700 1 --2- 172.21.15.16:0/971482207 >> v2:172.21.15.16:3300/0 conn(0x7f194809e000 0x7f19481339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:34.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.619+0000 7f194d640700 1 --2- 172.21.15.16:0/971482207 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f194809d6f0 0x7f194809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:34.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.619+0000 7f194de41700 1 --2- 172.21.15.16:0/971482207 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f194809e410 0x7f1948133f30 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1940009dc0 tx=0x7f194000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a51fd21703b73637 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:34.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.619+0000 7f193ffff700 1 -- 172.21.15.16:0/971482207 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194000e070 con 0x7f194809e410 2023-09-30T22:31:34.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.620+0000 7f193ffff700 1 -- 172.21.15.16:0/971482207 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f194000bae0 con 0x7f194809e410 2023-09-30T22:31:34.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.620+0000 7f193ffff700 1 -- 172.21.15.16:0/971482207 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194000be90 con 0x7f194809e410 2023-09-30T22:31:34.622 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:34.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.620+0000 7f194f8a4700 1 -- 172.21.15.16:0/971482207 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f194809e410 msgr2=0x7f1948133f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:34.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.620+0000 7f194f8a4700 1 --2- 172.21.15.16:0/971482207 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f194809e410 0x7f1948133f30 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1940009dc0 tx=0x7f194000b330 comp rx=0 tx=0).stop 2023-09-30T22:31:34.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.621+0000 7f194f8a4700 1 -- 172.21.15.16:0/971482207 shutdown_connections 2023-09-30T22:31:34.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.621+0000 7f194f8a4700 1 --2- 172.21.15.16:0/971482207 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f194809d6f0 0x7f194809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:34.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.621+0000 7f194f8a4700 1 --2- 172.21.15.16:0/971482207 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f194809e410 0x7f1948133f30 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:34.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.621+0000 7f194f8a4700 1 --2- 172.21.15.16:0/971482207 >> v2:172.21.15.16:3300/0 conn(0x7f194809e000 0x7f19481339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:34.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.621+0000 7f194f8a4700 1 -- 172.21.15.16:0/971482207 >> 172.21.15.16:0/971482207 conn(0x7f1948066f50 msgr2=0x7f194809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:34.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.621+0000 7f194f8a4700 1 -- 172.21.15.16:0/971482207 shutdown_connections 2023-09-30T22:31:34.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.621+0000 7f194f8a4700 1 -- 172.21.15.16:0/971482207 wait complete. 2023-09-30T22:31:34.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.622+0000 7f194f8a4700 1 Processor -- start 2023-09-30T22:31:34.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.622+0000 7f194f8a4700 1 -- start start 2023-09-30T22:31:34.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.622+0000 7f194f8a4700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f194809d6f0 0x7f19481c5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:34.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.622+0000 7f194f8a4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f194809e000 0x7f19481c5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:34.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.622+0000 7f194f8a4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f194809e410 0x7f19481c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:34.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.622+0000 7f194f8a4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19481a7240 con 0x7f194809e000 2023-09-30T22:31:34.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.622+0000 7f194f8a4700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19481c5ad0 con 0x7f194809d6f0 2023-09-30T22:31:34.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.622+0000 7f194f8a4700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f19481a9820 con 0x7f194809e410 2023-09-30T22:31:34.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.622+0000 7f194d640700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f194809d6f0 0x7f19481c5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:34.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.622+0000 7f194d640700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f194809d6f0 0x7f19481c5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54288/0 (socket says 172.21.15.16:54288) 2023-09-30T22:31:34.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.622+0000 7f194d640700 1 -- 172.21.15.16:0/3255083387 learned_addr learned my addr 172.21.15.16:0/3255083387 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:34.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.622+0000 7f194de41700 1 --2- 172.21.15.16:0/3255083387 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f194809e410 0x7f19481c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:34.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.623+0000 7f194d640700 1 -- 172.21.15.16:0/3255083387 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f194809e410 msgr2=0x7f19481c8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:34.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.623+0000 7f194d640700 1 --2- 172.21.15.16:0/3255083387 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f194809e410 0x7f19481c8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:34.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.623+0000 7f194d640700 1 -- 172.21.15.16:0/3255083387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f194809e000 msgr2=0x7f19481c5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:34.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.623+0000 7f194d640700 1 --2- 172.21.15.16:0/3255083387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f194809e000 0x7f19481c5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:34.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.623+0000 7f194d640700 1 -- 172.21.15.16:0/3255083387 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19440094a0 con 0x7f194809d6f0 2023-09-30T22:31:34.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.623+0000 7f194d640700 1 --2- 172.21.15.16:0/3255083387 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f194809d6f0 0x7f19481c5050 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f194400ec10 tx=0x7f194400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:34.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.623+0000 7f193dffb700 1 -- 172.21.15.16:0/3255083387 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194400c910 con 0x7f194809d6f0 2023-09-30T22:31:34.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.623+0000 7f193dffb700 1 -- 172.21.15.16:0/3255083387 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f194400ca70 con 0x7f194809d6f0 2023-09-30T22:31:34.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.623+0000 7f193dffb700 1 -- 172.21.15.16:0/3255083387 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194400cde0 con 0x7f194809d6f0 2023-09-30T22:31:34.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.623+0000 7f194f8a4700 1 -- 172.21.15.16:0/3255083387 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1938009550 con 0x7f194809d6f0 2023-09-30T22:31:34.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.624+0000 7f194f8a4700 1 -- 172.21.15.16:0/3255083387 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1940009a70 con 0x7f194809d6f0 2023-09-30T22:31:34.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.625+0000 7f193dffb700 1 -- 172.21.15.16:0/3255083387 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1944013470 con 0x7f194809d6f0 2023-09-30T22:31:34.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.625+0000 7f193dffb700 1 -- 172.21.15.16:0/3255083387 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f194403f850 con 0x7f194809d6f0 2023-09-30T22:31:34.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.625+0000 7f194f8a4700 1 -- 172.21.15.16:0/3255083387 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1948074ea0 con 0x7f194809d6f0 2023-09-30T22:31:34.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.631+0000 7f193dffb700 1 -- 172.21.15.16:0/3255083387 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f19440232f0 con 0x7f194809d6f0 2023-09-30T22:31:34.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.692+0000 7f3b6d30b700 1 Processor -- start 2023-09-30T22:31:34.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.692+0000 7f3b6d30b700 1 -- start start 2023-09-30T22:31:34.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.693+0000 7f3b6d30b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3b681338a0 0x7f3b68133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:34.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.693+0000 7f3b6d30b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3b6809de50 0x7f3b6809e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:34.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.693+0000 7f3b6d30b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b6806ef30 con 0x7f3b6809e760 2023-09-30T22:31:34.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.693+0000 7f3b6d30b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b6806fef0 con 0x7f3b6809de50 2023-09-30T22:31:34.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.693+0000 7f3b6d30b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b681334b0 con 0x7f3b681338a0 2023-09-30T22:31:34.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.693+0000 7f3b66ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3b681338a0 0x7f3b68133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:34.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.693+0000 7f3b667fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3b6809de50 0x7f3b6809e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:34.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.693+0000 7f3b66ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3b681338a0 0x7f3b68133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54298/0 (socket says 172.21.15.16:54298) 2023-09-30T22:31:34.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.693+0000 7f3b66ffd700 1 -- 172.21.15.16:0/3301172139 learned_addr learned my addr 172.21.15.16:0/3301172139 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:34.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.694+0000 7f3b667fc700 1 -- 172.21.15.16:0/3301172139 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3b681338a0 msgr2=0x7f3b68133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:34.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.694+0000 7f3b667fc700 1 --2- 172.21.15.16:0/3301172139 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3b681338a0 0x7f3b68133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:34.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.694+0000 7f3b667fc700 1 -- 172.21.15.16:0/3301172139 >> v1:172.21.15.16:6789/0 conn(0x7f3b6809e760 legacy=0x7f3b6813a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:34.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.694+0000 7f3b667fc700 1 -- 172.21.15.16:0/3301172139 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b6806fc30 con 0x7f3b6809de50 2023-09-30T22:31:34.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.694+0000 7f3b667fc700 1 --2- 172.21.15.16:0/3301172139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b6809de50 0x7f3b6809e220 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c009840 tx=0x7f3b5c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdd6b1aa3cfc6ab4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:34.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.694+0000 7f3b65ffb700 1 -- 172.21.15.16:0/3301172139 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c011830 con 0x7f3b6809de50 2023-09-30T22:31:34.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.694+0000 7f3b65ffb700 1 -- 172.21.15.16:0/3301172139 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b5c011990 con 0x7f3b6809de50 2023-09-30T22:31:34.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.694+0000 7f3b65ffb700 1 -- 172.21.15.16:0/3301172139 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c011d20 con 0x7f3b6809de50 2023-09-30T22:31:34.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.695+0000 7f3b6d30b700 1 -- 172.21.15.16:0/3301172139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b6809de50 msgr2=0x7f3b6809e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:34.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.695+0000 7f3b6d30b700 1 --2- 172.21.15.16:0/3301172139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b6809de50 0x7f3b6809e220 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c009840 tx=0x7f3b5c009b50 comp rx=0 tx=0).stop 2023-09-30T22:31:34.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.695+0000 7f3b6d30b700 1 -- 172.21.15.16:0/3301172139 shutdown_connections 2023-09-30T22:31:34.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.695+0000 7f3b6d30b700 1 --2- 172.21.15.16:0/3301172139 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3b681338a0 0x7f3b68133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:34.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.695+0000 7f3b6d30b700 1 --2- 172.21.15.16:0/3301172139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b6809de50 0x7f3b6809e220 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:34.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.695+0000 7f3b6d30b700 1 -- 172.21.15.16:0/3301172139 >> 172.21.15.16:0/3301172139 conn(0x7f3b680a2aa0 msgr2=0x7f3b680a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:34.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.695+0000 7f3b6d30b700 1 -- 172.21.15.16:0/3301172139 shutdown_connections 2023-09-30T22:31:34.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.695+0000 7f3b6d30b700 1 -- 172.21.15.16:0/3301172139 wait complete. 2023-09-30T22:31:34.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.697+0000 7f3b6d30b700 1 Processor -- start 2023-09-30T22:31:34.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.697+0000 7f3b6d30b700 1 -- start start 2023-09-30T22:31:34.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.698+0000 7f3b6d30b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3b6809de50 0x7f3b6811b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:34.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.698+0000 7f3b6d30b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b6809e760 0x7f3b6811bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:34.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.698+0000 7f3b6d30b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3b681338a0 0x7f3b6811c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:34.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.698+0000 7f3b6d30b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b6811f4f0 con 0x7f3b6809e760 2023-09-30T22:31:34.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.698+0000 7f3b6d30b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b680a0fb0 con 0x7f3b681338a0 2023-09-30T22:31:34.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.698+0000 7f3b6d30b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3b6806f640 con 0x7f3b6809de50 2023-09-30T22:31:34.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.698+0000 7f3b667fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b6809e760 0x7f3b6811bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:34.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.698+0000 7f3b667fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b6809e760 0x7f3b6811bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53644/0 (socket says 172.21.15.16:53644) 2023-09-30T22:31:34.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.698+0000 7f3b677fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3b681338a0 0x7f3b6811c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:34.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.698+0000 7f3b667fc700 1 -- 172.21.15.16:0/2955543480 learned_addr learned my addr 172.21.15.16:0/2955543480 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:34.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.698+0000 7f3b66ffd700 1 --2- 172.21.15.16:0/2955543480 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3b6809de50 0x7f3b6811b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:34.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.698+0000 7f3b667fc700 1 -- 172.21.15.16:0/2955543480 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3b6809de50 msgr2=0x7f3b6811b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:34.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.698+0000 7f3b667fc700 1 --2- 172.21.15.16:0/2955543480 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3b6809de50 0x7f3b6811b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:34.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.698+0000 7f3b667fc700 1 -- 172.21.15.16:0/2955543480 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3b681338a0 msgr2=0x7f3b6811c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:34.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.698+0000 7f3b667fc700 1 --2- 172.21.15.16:0/2955543480 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3b681338a0 0x7f3b6811c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:34.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.698+0000 7f3b667fc700 1 -- 172.21.15.16:0/2955543480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b5c0094a0 con 0x7f3b6809e760 2023-09-30T22:31:34.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.698+0000 7f3b66ffd700 1 --2- 172.21.15.16:0/2955543480 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3b6809de50 0x7f3b6811b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:31:34.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.699+0000 7f3b667fc700 1 --2- 172.21.15.16:0/2955543480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b6809e760 0x7f3b6811bf10 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c009fd0 tx=0x7f3b5c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:34.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.699+0000 7f3b4f7fe700 1 -- 172.21.15.16:0/2955543480 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c0036a0 con 0x7f3b6809e760 2023-09-30T22:31:34.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.699+0000 7f3b4f7fe700 1 -- 172.21.15.16:0/2955543480 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b5c01e070 con 0x7f3b6809e760 2023-09-30T22:31:34.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.699+0000 7f3b4f7fe700 1 -- 172.21.15.16:0/2955543480 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c003a20 con 0x7f3b6809e760 2023-09-30T22:31:34.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.699+0000 7f3b6d30b700 1 -- 172.21.15.16:0/2955543480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b6811ef50 con 0x7f3b6809e760 2023-09-30T22:31:34.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.699+0000 7f3b6d30b700 1 -- 172.21.15.16:0/2955543480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b6811db30 con 0x7f3b6809e760 2023-09-30T22:31:34.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.700+0000 7f3b6d30b700 1 -- 172.21.15.16:0/2955543480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b68074ea0 con 0x7f3b6809e760 2023-09-30T22:31:34.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.701+0000 7f3b4f7fe700 1 -- 172.21.15.16:0/2955543480 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3b5c003b80 con 0x7f3b6809e760 2023-09-30T22:31:34.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.701+0000 7f3b4f7fe700 1 -- 172.21.15.16:0/2955543480 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f3b5c040a50 con 0x7f3b6809e760 2023-09-30T22:31:34.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.710+0000 7f3b4f7fe700 1 -- 172.21.15.16:0/2955543480 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3b5c02d450 con 0x7f3b6809e760 2023-09-30T22:31:34.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.769+0000 7f18b63d2700 1 Processor -- start 2023-09-30T22:31:34.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.769+0000 7f18b63d2700 1 -- start start 2023-09-30T22:31:34.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.770+0000 7f18b63d2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f18b009d6f0 0x7f18b009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:34.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.770+0000 7f18b63d2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f18b009e000 0x7f18b0130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:34.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.770+0000 7f18b63d2700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f18b009e410 0x7f18b0130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:34.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.770+0000 7f18b63d2700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18b009f520 con 0x7f18b009e410 2023-09-30T22:31:34.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.770+0000 7f18b63d2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18b009f130 con 0x7f18b009d6f0 2023-09-30T22:31:34.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.770+0000 7f18b63d2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f18b009e930 con 0x7f18b009e000 2023-09-30T22:31:34.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.770+0000 7f18affff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f18b009d6f0 0x7f18b009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:34.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.770+0000 7f18af7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f18b009e000 0x7f18b0130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:34.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.770+0000 7f18affff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f18b009d6f0 0x7f18b009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54302/0 (socket says 172.21.15.16:54302) 2023-09-30T22:31:34.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.770+0000 7f18af7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f18b009e000 0x7f18b0130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:33876/0 (socket says 172.21.15.16:33876) 2023-09-30T22:31:34.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.770+0000 7f18affff700 1 -- 172.21.15.16:0/735661292 learned_addr learned my addr 172.21.15.16:0/735661292 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:34.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.770+0000 7f18affff700 1 -- 172.21.15.16:0/735661292 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f18b009e000 msgr2=0x7f18b0130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:34.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.771+0000 7f18affff700 1 --2- 172.21.15.16:0/735661292 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f18b009e000 0x7f18b0130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:34.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.771+0000 7f18affff700 1 -- 172.21.15.16:0/735661292 >> v2:172.21.15.16:3300/0 conn(0x7f18b009e410 msgr2=0x7f18b0130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:34.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.771+0000 7f18affff700 1 --2- 172.21.15.16:0/735661292 >> v2:172.21.15.16:3300/0 conn(0x7f18b009e410 0x7f18b0130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:34.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.771+0000 7f18affff700 1 -- 172.21.15.16:0/735661292 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b009b2d0 con 0x7f18b009d6f0 2023-09-30T22:31:34.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.771+0000 7f18affff700 1 --2- 172.21.15.16:0/735661292 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f18b009d6f0 0x7f18b009dac0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f18a000b9d0 tx=0x7f18a000bce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89588e4c046c5e65 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:34.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.771+0000 7f18aeffd700 1 -- 172.21.15.16:0/735661292 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a000cc20 con 0x7f18b009d6f0 2023-09-30T22:31:34.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.771+0000 7f18aeffd700 1 -- 172.21.15.16:0/735661292 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18a000cd80 con 0x7f18b009d6f0 2023-09-30T22:31:34.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.771+0000 7f18aeffd700 1 -- 172.21.15.16:0/735661292 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a000f680 con 0x7f18b009d6f0 2023-09-30T22:31:34.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.772+0000 7f18b63d2700 1 -- 172.21.15.16:0/735661292 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f18b009d6f0 msgr2=0x7f18b009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:34.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.772+0000 7f18b63d2700 1 --2- 172.21.15.16:0/735661292 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f18b009d6f0 0x7f18b009dac0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f18a000b9d0 tx=0x7f18a000bce0 comp rx=0 tx=0).stop 2023-09-30T22:31:34.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.772+0000 7f18b63d2700 1 -- 172.21.15.16:0/735661292 shutdown_connections 2023-09-30T22:31:34.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.772+0000 7f18b63d2700 1 --2- 172.21.15.16:0/735661292 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f18b009d6f0 0x7f18b009dac0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:34.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.772+0000 7f18b63d2700 1 --2- 172.21.15.16:0/735661292 >> v2:172.21.15.16:3300/0 conn(0x7f18b009e410 0x7f18b0130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:34.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.772+0000 7f18b63d2700 1 --2- 172.21.15.16:0/735661292 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f18b009e000 0x7f18b0130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:34.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.772+0000 7f18b63d2700 1 -- 172.21.15.16:0/735661292 >> 172.21.15.16:0/735661292 conn(0x7f18b0066f50 msgr2=0x7f18b009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:34.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.772+0000 7f18b63d2700 1 -- 172.21.15.16:0/735661292 shutdown_connections 2023-09-30T22:31:34.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.772+0000 7f18b63d2700 1 -- 172.21.15.16:0/735661292 wait complete. 2023-09-30T22:31:34.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.773+0000 7f18b63d2700 1 Processor -- start 2023-09-30T22:31:34.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.773+0000 7f18b63d2700 1 -- start start 2023-09-30T22:31:34.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.774+0000 7f18b63d2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18b009d6f0 0x7f18b01bf0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:34.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.774+0000 7f18b63d2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f18b009e000 0x7f18b01bf600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:34.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.774+0000 7f18b63d2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f18b009e410 0x7f18b01cc8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:34.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.774+0000 7f18b63d2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b01bc2e0 con 0x7f18b009d6f0 2023-09-30T22:31:34.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.774+0000 7f18b63d2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18b01bc9c0 con 0x7f18b009e410 2023-09-30T22:31:34.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.774+0000 7f18b63d2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f18b00a1a70 con 0x7f18b009e000 2023-09-30T22:31:34.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.774+0000 7f18affff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18b009d6f0 0x7f18b01bf0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:34.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.774+0000 7f18affff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18b009d6f0 0x7f18b01bf0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53660/0 (socket says 172.21.15.16:53660) 2023-09-30T22:31:34.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.774+0000 7f18affff700 1 -- 172.21.15.16:0/25815332 learned_addr learned my addr 172.21.15.16:0/25815332 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:34.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.774+0000 7f18af7fe700 1 --2- 172.21.15.16:0/25815332 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f18b009e000 0x7f18b01bf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:34.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.774+0000 7f18b496f700 1 --2- 172.21.15.16:0/25815332 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f18b009e410 0x7f18b01cc8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:34.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.775+0000 7f18affff700 1 -- 172.21.15.16:0/25815332 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f18b009e000 msgr2=0x7f18b01bf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:34.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.775+0000 7f18affff700 1 --2- 172.21.15.16:0/25815332 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f18b009e000 0x7f18b01bf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:34.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.775+0000 7f18affff700 1 -- 172.21.15.16:0/25815332 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f18b009e410 msgr2=0x7f18b01cc8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:34.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.775+0000 7f18affff700 1 --2- 172.21.15.16:0/25815332 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f18b009e410 0x7f18b01cc8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:34.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.775+0000 7f18affff700 1 -- 172.21.15.16:0/25815332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a000b600 con 0x7f18b009d6f0 2023-09-30T22:31:34.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.775+0000 7fa968baf700 1 Processor -- start 2023-09-30T22:31:34.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.775+0000 7f18affff700 1 --2- 172.21.15.16:0/25815332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18b009d6f0 0x7f18b01bf0c0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f18a00090b0 tx=0x7f18a0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:34.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.775+0000 7fa968baf700 1 -- start start 2023-09-30T22:31:34.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.775+0000 7f18acff9700 1 -- 172.21.15.16:0/25815332 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a001c070 con 0x7f18b009d6f0 2023-09-30T22:31:34.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.775+0000 7f18acff9700 1 -- 172.21.15.16:0/25815332 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18a000be70 con 0x7f18b009d6f0 2023-09-30T22:31:34.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.775+0000 7f18b63d2700 1 -- 172.21.15.16:0/25815332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18b01bdd20 con 0x7f18b009d6f0 2023-09-30T22:31:34.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.775+0000 7f18acff9700 1 -- 172.21.15.16:0/25815332 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a000cc20 con 0x7f18b009d6f0 2023-09-30T22:31:34.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.775+0000 7f18b63d2700 1 -- 172.21.15.16:0/25815332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18b01be790 con 0x7f18b009d6f0 2023-09-30T22:31:34.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.776+0000 7fa968baf700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa96406a770 0x7fa96406ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:34.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.776+0000 7fa968baf700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa964131d10 0x7fa964139860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:34.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.776+0000 7fa968baf700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa96409ece0 con 0x7fa96406b080 2023-09-30T22:31:34.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.776+0000 7fa968baf700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa96409e760 con 0x7fa96406a770 2023-09-30T22:31:34.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.776+0000 7fa968baf700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa96409e040 con 0x7fa964131d10 2023-09-30T22:31:34.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.776+0000 7fa961d9b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa96406b080 0x7fa964131600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44154/0 (socket says 172.21.15.16:44154) 2023-09-30T22:31:34.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.776+0000 7fa961d9b700 1 -- 172.21.15.16:0/3339475873 learned_addr learned my addr 172.21.15.16:0/3339475873 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:31:34.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.776+0000 7fa962d9d700 1 --2- 172.21.15.16:0/3339475873 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa964131d10 0x7fa964139860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:34.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.777+0000 7fa96259c700 1 --2- 172.21.15.16:0/3339475873 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa96406a770 0x7fa96406ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:34.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.776+0000 7f18acff9700 1 -- 172.21.15.16:0/25815332 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f18a0020070 con 0x7f18b009d6f0 2023-09-30T22:31:34.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.776+0000 7f18acff9700 1 -- 172.21.15.16:0/25815332 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f18a0040c10 con 0x7f18b009d6f0 2023-09-30T22:31:34.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.777+0000 7fa96159a700 1 -- 172.21.15.16:0/3339475873 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2644832836 0 0) 0x7fa96409ece0 con 0x7fa96406b080 2023-09-30T22:31:34.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.777+0000 7fa96159a700 1 -- 172.21.15.16:0/3339475873 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa94c0035b0 con 0x7fa96406b080 2023-09-30T22:31:34.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.777+0000 7fa962d9d700 1 -- 172.21.15.16:0/3339475873 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa96406a770 msgr2=0x7fa96406ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:34.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.777+0000 7fa962d9d700 1 --2- 172.21.15.16:0/3339475873 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa96406a770 0x7fa96406ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:34.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.777+0000 7fa962d9d700 1 -- 172.21.15.16:0/3339475873 >> v1:172.21.15.16:6789/0 conn(0x7fa96406b080 legacy=0x7fa964131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:34.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.777+0000 7fa962d9d700 1 -- 172.21.15.16:0/3339475873 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa96409f630 con 0x7fa964131d10 2023-09-30T22:31:34.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.777+0000 7fa962d9d700 1 --2- 172.21.15.16:0/3339475873 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa964131d10 0x7fa964139860 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa95c009dc0 tx=0x7fa95c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e615d0d3ac33fa1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:34.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.777+0000 7f18b63d2700 1 -- 172.21.15.16:0/25815332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18b01bd9a0 con 0x7f18b009d6f0 2023-09-30T22:31:34.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.777+0000 7fa96159a700 1 -- 172.21.15.16:0/3339475873 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95c00e070 con 0x7fa964131d10 2023-09-30T22:31:34.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.777+0000 7fa96159a700 1 -- 172.21.15.16:0/3339475873 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa95c00bb20 con 0x7fa964131d10 2023-09-30T22:31:34.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.777+0000 7fa96159a700 1 -- 172.21.15.16:0/3339475873 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95c012430 con 0x7fa964131d10 2023-09-30T22:31:34.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.778+0000 7fa968baf700 1 -- 172.21.15.16:0/3339475873 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa964131d10 msgr2=0x7fa964139860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:34.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.778+0000 7fa968baf700 1 --2- 172.21.15.16:0/3339475873 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa964131d10 0x7fa964139860 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa95c009dc0 tx=0x7fa95c00b330 comp rx=0 tx=0).stop 2023-09-30T22:31:34.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.779+0000 7fa968baf700 1 -- 172.21.15.16:0/3339475873 shutdown_connections 2023-09-30T22:31:34.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.779+0000 7fa968baf700 1 --2- 172.21.15.16:0/3339475873 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa964131d10 0x7fa964139860 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:34.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.779+0000 7fa968baf700 1 --2- 172.21.15.16:0/3339475873 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa96406a770 0x7fa96406ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:34.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.779+0000 7fa968baf700 1 -- 172.21.15.16:0/3339475873 >> 172.21.15.16:0/3339475873 conn(0x7fa964066f50 msgr2=0x7fa96409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:34.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.779+0000 7fa968baf700 1 -- 172.21.15.16:0/3339475873 shutdown_connections 2023-09-30T22:31:34.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.779+0000 7fa968baf700 1 -- 172.21.15.16:0/3339475873 wait complete. 2023-09-30T22:31:34.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.780+0000 7fa968baf700 1 Processor -- start 2023-09-30T22:31:34.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.780+0000 7fa968baf700 1 -- start start 2023-09-30T22:31:34.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.781+0000 7fa968baf700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa96406a770 0x7fa964122ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:34.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.781+0000 7fa968baf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa96406b080 0x7fa964123010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:34.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.781+0000 7fa968baf700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa964131d10 0x7fa964123550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:34.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.781+0000 7fa968baf700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa964124870 con 0x7fa96406b080 2023-09-30T22:31:34.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.781+0000 7fa968baf700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa964121ef0 con 0x7fa96406a770 2023-09-30T22:31:34.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.781+0000 7fa968baf700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa96409a0c0 con 0x7fa964131d10 2023-09-30T22:31:34.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.781+0000 7fa96259c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa96406a770 0x7fa964122ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:34.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.781+0000 7fa96259c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa96406a770 0x7fa964122ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54316/0 (socket says 172.21.15.16:54316) 2023-09-30T22:31:34.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.781+0000 7fa96259c700 1 -- 172.21.15.16:0/3059416107 learned_addr learned my addr 172.21.15.16:0/3059416107 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:34.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.782+0000 7fa962d9d700 1 --2- 172.21.15.16:0/3059416107 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa964131d10 0x7fa964123550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:34.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.782+0000 7fa961d9b700 1 --2- 172.21.15.16:0/3059416107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa96406b080 0x7fa964123010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:34.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.782+0000 7fa96259c700 1 -- 172.21.15.16:0/3059416107 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa964131d10 msgr2=0x7fa964123550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:34.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.782+0000 7fa96259c700 1 --2- 172.21.15.16:0/3059416107 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa964131d10 0x7fa964123550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:34.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.782+0000 7fa96259c700 1 -- 172.21.15.16:0/3059416107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa96406b080 msgr2=0x7fa964123010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:34.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.782+0000 7fa96259c700 1 --2- 172.21.15.16:0/3059416107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa96406b080 0x7fa964123010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:34.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.782+0000 7fa96259c700 1 -- 172.21.15.16:0/3059416107 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa95c009a70 con 0x7fa96406a770 2023-09-30T22:31:34.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.782+0000 7fa96259c700 1 --2- 172.21.15.16:0/3059416107 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa96406a770 0x7fa964122ad0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa95800e9b0 tx=0x7fa95800ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:34.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.782+0000 7fa952ffd700 1 -- 172.21.15.16:0/3059416107 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95800ee50 con 0x7fa96406a770 2023-09-30T22:31:34.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.782+0000 7fa952ffd700 1 -- 172.21.15.16:0/3059416107 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa95800c850 con 0x7fa96406a770 2023-09-30T22:31:34.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.782+0000 7fa968baf700 1 -- 172.21.15.16:0/3059416107 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa964120ed0 con 0x7fa96406a770 2023-09-30T22:31:34.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.783+0000 7fa952ffd700 1 -- 172.21.15.16:0/3059416107 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95800cbc0 con 0x7fa96406a770 2023-09-30T22:31:34.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.783+0000 7fa968baf700 1 -- 172.21.15.16:0/3059416107 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa96409c120 con 0x7fa96406a770 2023-09-30T22:31:34.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.784+0000 7fa952ffd700 1 -- 172.21.15.16:0/3059416107 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa95800cd20 con 0x7fa96406a770 2023-09-30T22:31:34.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.784+0000 7fa968baf700 1 -- 172.21.15.16:0/3059416107 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa964074ea0 con 0x7fa96406a770 2023-09-30T22:31:34.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.784+0000 7fa952ffd700 1 -- 172.21.15.16:0/3059416107 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fa95803f850 con 0x7fa96406a770 2023-09-30T22:31:34.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.784+0000 7f18acff9700 1 -- 172.21.15.16:0/25815332 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f18a0027080 con 0x7f18b009d6f0 2023-09-30T22:31:34.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.789+0000 7fa952ffd700 1 -- 172.21.15.16:0/3059416107 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa9580232f0 con 0x7fa96406a770 2023-09-30T22:31:34.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.863+0000 7f194f8a4700 1 -- 172.21.15.16:0/3255083387 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f194806c360 con 0x7f194809d6f0 2023-09-30T22:31:34.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.863+0000 7f193dffb700 1 -- 172.21.15.16:0/3255083387 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1944021030 con 0x7f194809d6f0 2023-09-30T22:31:34.864 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:31:34.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.867+0000 7f194f8a4700 1 -- 172.21.15.16:0/3255083387 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f194809d6f0 msgr2=0x7f19481c5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:34.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.867+0000 7f194f8a4700 1 --2- 172.21.15.16:0/3255083387 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f194809d6f0 0x7f19481c5050 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f194400ec10 tx=0x7f194400ef20 comp rx=0 tx=0).stop 2023-09-30T22:31:34.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.868+0000 7f194f8a4700 1 -- 172.21.15.16:0/3255083387 shutdown_connections 2023-09-30T22:31:34.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.868+0000 7f194f8a4700 1 --2- 172.21.15.16:0/3255083387 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f194809d6f0 0x7f19481c5050 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:34.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.868+0000 7f194f8a4700 1 --2- 172.21.15.16:0/3255083387 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f194809e410 0x7f19481c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:34.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.868+0000 7f194f8a4700 1 --2- 172.21.15.16:0/3255083387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f194809e000 0x7f19481c5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:34.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.868+0000 7f194f8a4700 1 -- 172.21.15.16:0/3255083387 >> 172.21.15.16:0/3255083387 conn(0x7f1948066f50 msgr2=0x7f19481bae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:34.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.868+0000 7f194f8a4700 1 -- 172.21.15.16:0/3255083387 shutdown_connections 2023-09-30T22:31:34.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.868+0000 7f194f8a4700 1 -- 172.21.15.16:0/3255083387 wait complete. 2023-09-30T22:31:34.887 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:31:34.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.935+0000 7f3b6d30b700 1 -- 172.21.15.16:0/2955543480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3b680a2460 con 0x7f3b6809e760 2023-09-30T22:31:34.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.935+0000 7f3b4f7fe700 1 -- 172.21.15.16:0/2955543480 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3b5c015030 con 0x7f3b6809e760 2023-09-30T22:31:34.936 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:31:34.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.937+0000 7f3b6d30b700 1 -- 172.21.15.16:0/2955543480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b6809e760 msgr2=0x7f3b6811bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:34.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.937+0000 7f3b6d30b700 1 --2- 172.21.15.16:0/2955543480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b6809e760 0x7f3b6811bf10 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c009fd0 tx=0x7f3b5c004750 comp rx=0 tx=0).stop 2023-09-30T22:31:34.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.938+0000 7f3b6d30b700 1 -- 172.21.15.16:0/2955543480 shutdown_connections 2023-09-30T22:31:34.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.938+0000 7f3b6d30b700 1 --2- 172.21.15.16:0/2955543480 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3b681338a0 0x7f3b6811c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:34.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.938+0000 7f3b6d30b700 1 --2- 172.21.15.16:0/2955543480 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3b6809de50 0x7f3b6811b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:34.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.938+0000 7f3b6d30b700 1 --2- 172.21.15.16:0/2955543480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3b6809e760 0x7f3b6811bf10 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:34.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.938+0000 7f3b6d30b700 1 -- 172.21.15.16:0/2955543480 >> 172.21.15.16:0/2955543480 conn(0x7f3b680a2aa0 msgr2=0x7f3b681306c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:34.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.938+0000 7f3b6d30b700 1 -- 172.21.15.16:0/2955543480 shutdown_connections 2023-09-30T22:31:34.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:34.938+0000 7f3b6d30b700 1 -- 172.21.15.16:0/2955543480 wait complete. 2023-09-30T22:31:34.952 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:31:35.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:35.011+0000 7f18b63d2700 1 -- 172.21.15.16:0/25815332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f18b01bdb80 con 0x7f18b009d6f0 2023-09-30T22:31:35.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:35.011+0000 7f18acff9700 1 -- 172.21.15.16:0/25815332 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f18b01bdb80 con 0x7f18b009d6f0 2023-09-30T22:31:35.012 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:31:35.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:35.012+0000 7fa968baf700 1 -- 172.21.15.16:0/3059416107 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa964071e60 con 0x7fa96406a770 2023-09-30T22:31:35.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:35.012+0000 7fa952ffd700 1 -- 172.21.15.16:0/3059416107 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa95801ae80 con 0x7fa96406a770 2023-09-30T22:31:35.013 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:31:35.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:35.014+0000 7f18b63d2700 1 -- 172.21.15.16:0/25815332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18b009d6f0 msgr2=0x7f18b01bf0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:35.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:35.014+0000 7f18b63d2700 1 --2- 172.21.15.16:0/25815332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18b009d6f0 0x7f18b01bf0c0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f18a00090b0 tx=0x7f18a0004750 comp rx=0 tx=0).stop 2023-09-30T22:31:35.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:35.014+0000 7f18b63d2700 1 -- 172.21.15.16:0/25815332 shutdown_connections 2023-09-30T22:31:35.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:35.014+0000 7f18b63d2700 1 --2- 172.21.15.16:0/25815332 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f18b009e410 0x7f18b01cc8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:35.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:35.014+0000 7f18b63d2700 1 --2- 172.21.15.16:0/25815332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18b009d6f0 0x7f18b01bf0c0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:35.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:35.014+0000 7f18b63d2700 1 --2- 172.21.15.16:0/25815332 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f18b009e000 0x7f18b01bf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:35.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:35.014+0000 7f18b63d2700 1 -- 172.21.15.16:0/25815332 >> 172.21.15.16:0/25815332 conn(0x7f18b0066f50 msgr2=0x7f18b006a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:35.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:35.014+0000 7f18b63d2700 1 -- 172.21.15.16:0/25815332 shutdown_connections 2023-09-30T22:31:35.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:35.014+0000 7f18b63d2700 1 -- 172.21.15.16:0/25815332 wait complete. 2023-09-30T22:31:35.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:35.015+0000 7fa968baf700 1 -- 172.21.15.16:0/3059416107 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa96406a770 msgr2=0x7fa964122ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:35.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:35.015+0000 7fa968baf700 1 --2- 172.21.15.16:0/3059416107 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa96406a770 0x7fa964122ad0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa95800e9b0 tx=0x7fa95800ecc0 comp rx=0 tx=0).stop 2023-09-30T22:31:35.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:35.015+0000 7fa968baf700 1 -- 172.21.15.16:0/3059416107 shutdown_connections 2023-09-30T22:31:35.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:35.015+0000 7fa968baf700 1 --2- 172.21.15.16:0/3059416107 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa96406a770 0x7fa964122ad0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:35.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:35.015+0000 7fa968baf700 1 --2- 172.21.15.16:0/3059416107 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa964131d10 0x7fa964123550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:35.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:35.015+0000 7fa968baf700 1 --2- 172.21.15.16:0/3059416107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa96406b080 0x7fa964123010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:35.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:35.015+0000 7fa968baf700 1 -- 172.21.15.16:0/3059416107 >> 172.21.15.16:0/3059416107 conn(0x7fa964066f50 msgr2=0x7fa96406ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:35.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:35.015+0000 7fa968baf700 1 -- 172.21.15.16:0/3059416107 shutdown_connections 2023-09-30T22:31:35.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:35.015+0000 7fa968baf700 1 -- 172.21.15.16:0/3059416107 wait complete. 2023-09-30T22:31:35.029 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:31:35.030 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:31:35.889 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:35.952 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:36.030 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:36.031 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:36.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.036+0000 7f1f0f250700 1 Processor -- start 2023-09-30T22:31:36.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.036+0000 7f1f0f250700 1 -- start start 2023-09-30T22:31:36.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.036+0000 7f1f0f250700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f081338c0 0x7f1f08133c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:36.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.036+0000 7f1f0f250700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1f081341d0 0x7f1f0809de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:36.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.036+0000 7f1f0f250700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f0806f720 con 0x7f1f0809e390 2023-09-30T22:31:36.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.036+0000 7f1f0f250700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f08070c80 con 0x7f1f081341d0 2023-09-30T22:31:36.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.036+0000 7f1f0f250700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0806fce0 con 0x7f1f081338c0 2023-09-30T22:31:36.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.037+0000 7f1f07fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1f081341d0 0x7f1f0809de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:36.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.037+0000 7f1f0d7ed700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1f0809e390 0x7f1f0813a850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44156/0 (socket says 172.21.15.16:44156) 2023-09-30T22:31:36.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.037+0000 7f1f0d7ed700 1 -- 172.21.15.16:0/577917133 learned_addr learned my addr 172.21.15.16:0/577917133 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:31:36.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.037+0000 7f1f07fff700 1 -- 172.21.15.16:0/577917133 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f081338c0 msgr2=0x7f1f08133c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:36.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.037+0000 7f1f07fff700 1 --2- 172.21.15.16:0/577917133 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f081338c0 0x7f1f08133c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:36.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.037+0000 7f1f07fff700 1 -- 172.21.15.16:0/577917133 >> v1:172.21.15.16:6789/0 conn(0x7f1f0809e390 legacy=0x7f1f0813a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:36.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.037+0000 7f1f077fe700 1 -- 172.21.15.16:0/577917133 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 343232512 0 0) 0x7f1f0806f720 con 0x7f1f0809e390 2023-09-30T22:31:36.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.037+0000 7f1f07fff700 1 -- 172.21.15.16:0/577917133 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f081336a0 con 0x7f1f081341d0 2023-09-30T22:31:36.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.038+0000 7f1f07fff700 1 --2- 172.21.15.16:0/577917133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f081341d0 0x7f1f0809de50 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4009840 tx=0x7f1ef4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25eb14aec189f022 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:36.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.038+0000 7f1f077fe700 1 -- 172.21.15.16:0/577917133 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef4011830 con 0x7f1f081341d0 2023-09-30T22:31:36.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.038+0000 7f1f077fe700 1 -- 172.21.15.16:0/577917133 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ef4011990 con 0x7f1f081341d0 2023-09-30T22:31:36.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.038+0000 7f1f077fe700 1 -- 172.21.15.16:0/577917133 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef4011d20 con 0x7f1f081341d0 2023-09-30T22:31:36.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.039+0000 7f1f0f250700 1 -- 172.21.15.16:0/577917133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f081341d0 msgr2=0x7f1f0809de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:36.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.039+0000 7f1f0f250700 1 --2- 172.21.15.16:0/577917133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f081341d0 0x7f1f0809de50 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4009840 tx=0x7f1ef4009b50 comp rx=0 tx=0).stop 2023-09-30T22:31:36.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.039+0000 7f1f0f250700 1 -- 172.21.15.16:0/577917133 shutdown_connections 2023-09-30T22:31:36.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.039+0000 7f1f0f250700 1 --2- 172.21.15.16:0/577917133 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f081338c0 0x7f1f08133c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:36.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.039+0000 7f1f0f250700 1 --2- 172.21.15.16:0/577917133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f081341d0 0x7f1f0809de50 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:36.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.039+0000 7f1f0f250700 1 -- 172.21.15.16:0/577917133 >> 172.21.15.16:0/577917133 conn(0x7f1f080a2aa0 msgr2=0x7f1f080a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:36.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.039+0000 7f1f0f250700 1 -- 172.21.15.16:0/577917133 shutdown_connections 2023-09-30T22:31:36.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.039+0000 7f1f0f250700 1 -- 172.21.15.16:0/577917133 wait complete. 2023-09-30T22:31:36.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.040+0000 7f1f0f250700 1 Processor -- start 2023-09-30T22:31:36.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.041+0000 7f1f0f250700 1 -- start start 2023-09-30T22:31:36.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.041+0000 7f1f0f250700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f0809e390 0x7f1f0811c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:36.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.041+0000 7f1f0f250700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f081338c0 0x7f1f0811cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:36.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.041+0000 7f1f0f250700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f081341d0 0x7f1f0811d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:36.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.041+0000 7f1f0f250700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f080a1240 con 0x7f1f0809e390 2023-09-30T22:31:36.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.042+0000 7f1f0f250700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0811e490 con 0x7f1f081338c0 2023-09-30T22:31:36.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.042+0000 7f1f0f250700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0806f960 con 0x7f1f081341d0 2023-09-30T22:31:36.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.042+0000 7f1f07fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f081338c0 0x7f1f0811cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:36.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.042+0000 7f1f07fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f081338c0 0x7f1f0811cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54328/0 (socket says 172.21.15.16:54328) 2023-09-30T22:31:36.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.042+0000 7f1f07fff700 1 -- 172.21.15.16:0/415017575 learned_addr learned my addr 172.21.15.16:0/415017575 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:36.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.042+0000 7f1f0cfec700 1 --2- 172.21.15.16:0/415017575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f0809e390 0x7f1f0811c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:36.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.042+0000 7f1f07fff700 1 -- 172.21.15.16:0/415017575 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f081341d0 msgr2=0x7f1f0811d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:36.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.042+0000 7f1f07fff700 1 --2- 172.21.15.16:0/415017575 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f081341d0 0x7f1f0811d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:36.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.042+0000 7f1f07fff700 1 -- 172.21.15.16:0/415017575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f0809e390 msgr2=0x7f1f0811c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:36.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.042+0000 7f1f07fff700 1 --2- 172.21.15.16:0/415017575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f0809e390 0x7f1f0811c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:36.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.042+0000 7f1f07fff700 1 -- 172.21.15.16:0/415017575 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef40094a0 con 0x7f1f081338c0 2023-09-30T22:31:36.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.042+0000 7f1f07fff700 1 --2- 172.21.15.16:0/415017575 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f081338c0 0x7f1f0811cd40 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1ef40090b0 tx=0x7f1ef4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:36.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.043+0000 7f1f057fa700 1 -- 172.21.15.16:0/415017575 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef40036a0 con 0x7f1f081338c0 2023-09-30T22:31:36.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.043+0000 7f1f057fa700 1 -- 172.21.15.16:0/415017575 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ef401f070 con 0x7f1f081338c0 2023-09-30T22:31:36.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.043+0000 7f1f0f250700 1 -- 172.21.15.16:0/415017575 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f0811fcf0 con 0x7f1f081338c0 2023-09-30T22:31:36.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.043+0000 7f1f057fa700 1 -- 172.21.15.16:0/415017575 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef4003a60 con 0x7f1f081338c0 2023-09-30T22:31:36.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.043+0000 7f1f0f250700 1 -- 172.21.15.16:0/415017575 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f08120a20 con 0x7f1f081338c0 2023-09-30T22:31:36.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.044+0000 7f1f057fa700 1 -- 172.21.15.16:0/415017575 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1ef4003bc0 con 0x7f1f081338c0 2023-09-30T22:31:36.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.044+0000 7f1f0f250700 1 -- 172.21.15.16:0/415017575 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f08074ea0 con 0x7f1f081338c0 2023-09-30T22:31:36.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.044+0000 7f1f057fa700 1 -- 172.21.15.16:0/415017575 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f1ef4021070 con 0x7f1f081338c0 2023-09-30T22:31:36.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.051+0000 7f1f057fa700 1 -- 172.21.15.16:0/415017575 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1ef400b3e0 con 0x7f1f081338c0 2023-09-30T22:31:36.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.099+0000 7f12cfb92700 1 Processor -- start 2023-09-30T22:31:36.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.099+0000 7f12cfb92700 1 -- start start 2023-09-30T22:31:36.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.099+0000 7f12cfb92700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f12c809e1d0 0x7f12c8135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:36.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.099+0000 7f12cfb92700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12c8135e50 0x7f12c8138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:36.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.099+0000 7f12cfb92700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12c809f130 con 0x7f12c809d6f0 2023-09-30T22:31:36.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.099+0000 7f12cfb92700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12c809f520 con 0x7f12c809e1d0 2023-09-30T22:31:36.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.099+0000 7f12cfb92700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12c809e930 con 0x7f12c8135e50 2023-09-30T22:31:36.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.100+0000 7f12cd92e700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f12c809d6f0 0x7f12c809dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:44160/0 (socket says 172.21.15.16:44160) 2023-09-30T22:31:36.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.100+0000 7f12cd92e700 1 -- 172.21.15.16:0/225993348 learned_addr learned my addr 172.21.15.16:0/225993348 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:31:36.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.100+0000 7f12ce12f700 1 --2- 172.21.15.16:0/225993348 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12c8135e50 0x7f12c8138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:36.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.100+0000 7f12cd12d700 1 --2- 172.21.15.16:0/225993348 >> v2:172.21.15.16:3300/0 conn(0x7f12c809e1d0 0x7f12c8135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:36.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.100+0000 7f12ce12f700 1 -- 172.21.15.16:0/225993348 >> v2:172.21.15.16:3300/0 conn(0x7f12c809e1d0 msgr2=0x7f12c8135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:36.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.100+0000 7f12ce12f700 1 --2- 172.21.15.16:0/225993348 >> v2:172.21.15.16:3300/0 conn(0x7f12c809e1d0 0x7f12c8135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:36.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.100+0000 7f12ce12f700 1 -- 172.21.15.16:0/225993348 >> v1:172.21.15.16:6789/0 conn(0x7f12c809d6f0 legacy=0x7f12c809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:36.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.100+0000 7f12ce12f700 1 -- 172.21.15.16:0/225993348 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c809b2d0 con 0x7f12c8135e50 2023-09-30T22:31:36.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.100+0000 7f12cd12d700 1 --2- 172.21.15.16:0/225993348 >> v2:172.21.15.16:3300/0 conn(0x7f12c809e1d0 0x7f12c8135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:36.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.100+0000 7f12ce12f700 1 --2- 172.21.15.16:0/225993348 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12c8135e50 0x7f12c8138230 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f12c0009d90 tx=0x7f12c000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15c1cf5977211457 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:36.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.100+0000 7f12cc92c700 1 -- 172.21.15.16:0/225993348 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c000e070 con 0x7f12c8135e50 2023-09-30T22:31:36.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.101+0000 7f12cc92c700 1 -- 172.21.15.16:0/225993348 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c000bb20 con 0x7f12c8135e50 2023-09-30T22:31:36.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.101+0000 7f12cc92c700 1 -- 172.21.15.16:0/225993348 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c0012430 con 0x7f12c8135e50 2023-09-30T22:31:36.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.101+0000 7f12cfb92700 1 -- 172.21.15.16:0/225993348 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12c8135e50 msgr2=0x7f12c8138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:36.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.101+0000 7f12cfb92700 1 --2- 172.21.15.16:0/225993348 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12c8135e50 0x7f12c8138230 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f12c0009d90 tx=0x7f12c000b330 comp rx=0 tx=0).stop 2023-09-30T22:31:36.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.101+0000 7f12cfb92700 1 -- 172.21.15.16:0/225993348 shutdown_connections 2023-09-30T22:31:36.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.101+0000 7f12cfb92700 1 --2- 172.21.15.16:0/225993348 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12c8135e50 0x7f12c8138230 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:36.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.101+0000 7f12cfb92700 1 --2- 172.21.15.16:0/225993348 >> v2:172.21.15.16:3300/0 conn(0x7f12c809e1d0 0x7f12c8135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:36.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.101+0000 7f12cfb92700 1 -- 172.21.15.16:0/225993348 >> 172.21.15.16:0/225993348 conn(0x7f12c8066f50 msgr2=0x7f12c809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:36.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.102+0000 7f12cfb92700 1 -- 172.21.15.16:0/225993348 shutdown_connections 2023-09-30T22:31:36.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.102+0000 7f12cfb92700 1 -- 172.21.15.16:0/225993348 wait complete. 2023-09-30T22:31:36.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.103+0000 7f12cfb92700 1 Processor -- start 2023-09-30T22:31:36.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.103+0000 7f12cfb92700 1 -- start start 2023-09-30T22:31:36.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.103+0000 7f12cfb92700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12c809d6f0 0x7f12c811b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:36.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.103+0000 7f12cfb92700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12c809e1d0 0x7f12c811b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:36.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.103+0000 7f12cfb92700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12c8135e50 0x7f12c811bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:36.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.103+0000 7f12cfb92700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c811d100 con 0x7f12c8135e50 2023-09-30T22:31:36.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.103+0000 7f12cfb92700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12c811ec20 con 0x7f12c809e1d0 2023-09-30T22:31:36.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.103+0000 7f12cfb92700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f12c80a1e20 con 0x7f12c809d6f0 2023-09-30T22:31:36.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.103+0000 7f12cd92e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12c809d6f0 0x7f12c811b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:36.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.103+0000 7f12cd12d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12c809e1d0 0x7f12c811b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:36.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.103+0000 7f12ce12f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12c8135e50 0x7f12c811bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:36.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.103+0000 7f12cd92e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12c809d6f0 0x7f12c811b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:33912/0 (socket says 172.21.15.16:33912) 2023-09-30T22:31:36.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.103+0000 7f12cd92e700 1 -- 172.21.15.16:0/4286038784 learned_addr learned my addr 172.21.15.16:0/4286038784 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:36.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.104+0000 7f12cd12d700 1 -- 172.21.15.16:0/4286038784 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12c809d6f0 msgr2=0x7f12c811b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:36.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.104+0000 7f12cd12d700 1 --2- 172.21.15.16:0/4286038784 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12c809d6f0 0x7f12c811b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:36.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.104+0000 7f12cd12d700 1 -- 172.21.15.16:0/4286038784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12c8135e50 msgr2=0x7f12c811bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:36.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.104+0000 7f12cd12d700 1 --2- 172.21.15.16:0/4286038784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12c8135e50 0x7f12c811bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:36.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.104+0000 7f12cd12d700 1 -- 172.21.15.16:0/4286038784 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b80094a0 con 0x7f12c809e1d0 2023-09-30T22:31:36.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.104+0000 7f12cd12d700 1 --2- 172.21.15.16:0/4286038784 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12c809e1d0 0x7f12c811b9c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f12b800e900 tx=0x7f12b800ec10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:36.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.104+0000 7f12be7fc700 1 -- 172.21.15.16:0/4286038784 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b8013680 con 0x7f12c809e1d0 2023-09-30T22:31:36.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.104+0000 7f12be7fc700 1 -- 172.21.15.16:0/4286038784 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12b80137e0 con 0x7f12c809e1d0 2023-09-30T22:31:36.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.104+0000 7f12be7fc700 1 -- 172.21.15.16:0/4286038784 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b8013b50 con 0x7f12c809e1d0 2023-09-30T22:31:36.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.104+0000 7f12cfb92700 1 -- 172.21.15.16:0/4286038784 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12c0009a70 con 0x7f12c809e1d0 2023-09-30T22:31:36.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.104+0000 7f12cfb92700 1 -- 172.21.15.16:0/4286038784 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12c811d7a0 con 0x7f12c809e1d0 2023-09-30T22:31:36.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.106+0000 7f12be7fc700 1 -- 172.21.15.16:0/4286038784 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f12b8013cb0 con 0x7f12c809e1d0 2023-09-30T22:31:36.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.106+0000 7f12cfb92700 1 -- 172.21.15.16:0/4286038784 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12c8074ea0 con 0x7f12c809e1d0 2023-09-30T22:31:36.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.106+0000 7f12be7fc700 1 -- 172.21.15.16:0/4286038784 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f12b8024be0 con 0x7f12c809e1d0 2023-09-30T22:31:36.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.111+0000 7f12be7fc700 1 -- 172.21.15.16:0/4286038784 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f12b801adc0 con 0x7f12c809e1d0 2023-09-30T22:31:36.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.177+0000 7fe8f119d700 1 Processor -- start 2023-09-30T22:31:36.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.177+0000 7fe8f119d700 1 -- start start 2023-09-30T22:31:36.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.177+0000 7f2c1e4b7700 1 Processor -- start 2023-09-30T22:31:36.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.177+0000 7fe8f119d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe8ec09d6f0 0x7fe8ec09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:36.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.178+0000 7fe8f119d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe8ec09e410 0x7fe8ec130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:36.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.178+0000 7fe8f119d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8ec09f130 con 0x7fe8ec09e000 2023-09-30T22:31:36.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.178+0000 7fe8f119d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ec09f520 con 0x7fe8ec09e410 2023-09-30T22:31:36.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.178+0000 7fe8f119d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ec09e930 con 0x7fe8ec09d6f0 2023-09-30T22:31:36.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.177+0000 7f2c1e4b7700 1 -- start start 2023-09-30T22:31:36.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.178+0000 7f2c1e4b7700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2c1812cf60 0x7f2c1806f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:36.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.178+0000 7f2c1e4b7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c1812f8b0 0x7f2c1812fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:36.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.178+0000 7f2c1e4b7700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c1812cd90 con 0x7f2c18130260 2023-09-30T22:31:36.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.178+0000 7f2c1e4b7700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c1812d330 con 0x7f2c1812cf60 2023-09-30T22:31:36.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.178+0000 7f2c1e4b7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2c181306d0 con 0x7f2c1812f8b0 2023-09-30T22:31:36.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.178+0000 7fe8ead9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe8ec09d6f0 0x7fe8ec09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:36.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.178+0000 7fe8eb59e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe8ec09e410 0x7fe8ec130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:36.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.178+0000 7fe8ead9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe8ec09d6f0 0x7fe8ec09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:33916/0 (socket says 172.21.15.16:33916) 2023-09-30T22:31:36.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.178+0000 7fe8ead9d700 1 -- 172.21.15.16:0/819871245 learned_addr learned my addr 172.21.15.16:0/819871245 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:36.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.178+0000 7f2c177fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c1812f8b0 0x7f2c1812fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:36.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.178+0000 7f2c177fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c1812f8b0 0x7f2c1812fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:33920/0 (socket says 172.21.15.16:33920) 2023-09-30T22:31:36.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.178+0000 7f2c177fe700 1 -- 172.21.15.16:0/164714108 learned_addr learned my addr 172.21.15.16:0/164714108 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:36.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.178+0000 7f2c17fff700 1 --2- 172.21.15.16:0/164714108 >> v2:172.21.15.16:3300/0 conn(0x7f2c1812cf60 0x7f2c1806f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:36.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.179+0000 7f2c17fff700 1 -- 172.21.15.16:0/164714108 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c1812f8b0 msgr2=0x7f2c1812fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:36.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.179+0000 7f2c17fff700 1 --2- 172.21.15.16:0/164714108 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c1812f8b0 0x7f2c1812fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:36.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.179+0000 7f2c17fff700 1 -- 172.21.15.16:0/164714108 >> v1:172.21.15.16:6789/0 conn(0x7f2c18130260 legacy=0x7f2c18136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:36.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.179+0000 7f2c17fff700 1 -- 172.21.15.16:0/164714108 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c18071780 con 0x7f2c1812cf60 2023-09-30T22:31:36.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.178+0000 7fe8eb59e700 1 -- 172.21.15.16:0/819871245 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe8ec09d6f0 msgr2=0x7fe8ec09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:36.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.178+0000 7fe8eb59e700 1 --2- 172.21.15.16:0/819871245 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe8ec09d6f0 0x7fe8ec09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:36.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.178+0000 7fe8eb59e700 1 -- 172.21.15.16:0/819871245 >> v1:172.21.15.16:6789/0 conn(0x7fe8ec09e000 legacy=0x7fe8ec130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:36.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.179+0000 7fe8eb59e700 1 -- 172.21.15.16:0/819871245 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8ec09b2d0 con 0x7fe8ec09e410 2023-09-30T22:31:36.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.179+0000 7f2c17fff700 1 --2- 172.21.15.16:0/164714108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c1812cf60 0x7f2c1806f5d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f2c0800dc60 tx=0x7f2c0800df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce0a0f82b4dba70e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:36.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.179+0000 7fe8eb59e700 1 --2- 172.21.15.16:0/819871245 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe8ec09e410 0x7fe8ec130d70 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe8e400f3a0 tx=0x7fe8e400f6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d2f376d9904b5fb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:36.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.179+0000 7f2c16ffd700 1 -- 172.21.15.16:0/164714108 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c08012070 con 0x7f2c1812cf60 2023-09-30T22:31:36.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.179+0000 7fe8e9d9b700 1 -- 172.21.15.16:0/819871245 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e4012070 con 0x7fe8ec09e410 2023-09-30T22:31:36.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.179+0000 7fe8e9d9b700 1 -- 172.21.15.16:0/819871245 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8e400d0d0 con 0x7fe8ec09e410 2023-09-30T22:31:36.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.179+0000 7fe8e9d9b700 1 -- 172.21.15.16:0/819871245 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e400fea0 con 0x7fe8ec09e410 2023-09-30T22:31:36.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.179+0000 7f2c16ffd700 1 -- 172.21.15.16:0/164714108 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c0800d2e0 con 0x7f2c1812cf60 2023-09-30T22:31:36.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.179+0000 7f2c16ffd700 1 -- 172.21.15.16:0/164714108 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0800f900 con 0x7f2c1812cf60 2023-09-30T22:31:36.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.180+0000 7fe8f119d700 1 -- 172.21.15.16:0/819871245 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe8ec09e410 msgr2=0x7fe8ec130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:36.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.180+0000 7fe8f119d700 1 --2- 172.21.15.16:0/819871245 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe8ec09e410 0x7fe8ec130d70 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe8e400f3a0 tx=0x7fe8e400f6b0 comp rx=0 tx=0).stop 2023-09-30T22:31:36.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.180+0000 7f2c1e4b7700 1 -- 172.21.15.16:0/164714108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c1812cf60 msgr2=0x7f2c1806f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:36.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.180+0000 7f2c1e4b7700 1 --2- 172.21.15.16:0/164714108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c1812cf60 0x7f2c1806f5d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f2c0800dc60 tx=0x7f2c0800df70 comp rx=0 tx=0).stop 2023-09-30T22:31:36.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.180+0000 7fe8f119d700 1 -- 172.21.15.16:0/819871245 shutdown_connections 2023-09-30T22:31:36.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.180+0000 7fe8f119d700 1 --2- 172.21.15.16:0/819871245 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe8ec09e410 0x7fe8ec130d70 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:36.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.180+0000 7fe8f119d700 1 --2- 172.21.15.16:0/819871245 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe8ec09d6f0 0x7fe8ec09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:36.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.180+0000 7fe8f119d700 1 -- 172.21.15.16:0/819871245 >> 172.21.15.16:0/819871245 conn(0x7fe8ec066f50 msgr2=0x7fe8ec09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:36.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.180+0000 7fe8f119d700 1 -- 172.21.15.16:0/819871245 shutdown_connections 2023-09-30T22:31:36.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.180+0000 7f2c1e4b7700 1 -- 172.21.15.16:0/164714108 shutdown_connections 2023-09-30T22:31:36.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.180+0000 7f2c1e4b7700 1 --2- 172.21.15.16:0/164714108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c1812cf60 0x7f2c1806f5d0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:36.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.180+0000 7f2c1e4b7700 1 --2- 172.21.15.16:0/164714108 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c1812f8b0 0x7f2c1812fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:36.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.180+0000 7f2c1e4b7700 1 -- 172.21.15.16:0/164714108 >> 172.21.15.16:0/164714108 conn(0x7f2c180a2aa0 msgr2=0x7f2c180a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:36.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.180+0000 7fe8f119d700 1 -- 172.21.15.16:0/819871245 wait complete. 2023-09-30T22:31:36.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.180+0000 7f2c1e4b7700 1 -- 172.21.15.16:0/164714108 shutdown_connections 2023-09-30T22:31:36.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.180+0000 7f2c1e4b7700 1 -- 172.21.15.16:0/164714108 wait complete. 2023-09-30T22:31:36.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.182+0000 7f2c1e4b7700 1 Processor -- start 2023-09-30T22:31:36.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.182+0000 7f2c1e4b7700 1 -- start start 2023-09-30T22:31:36.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.182+0000 7fe8f119d700 1 Processor -- start 2023-09-30T22:31:36.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.182+0000 7fe8f119d700 1 -- start start 2023-09-30T22:31:36.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.183+0000 7f2c1e4b7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c1812cf60 0x7f2c181c1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:36.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.183+0000 7f2c1e4b7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c1812f8b0 0x7f2c181c1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:36.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.183+0000 7f2c1e4b7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c18130260 0x7f2c181c4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:36.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.183+0000 7f2c1e4b7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c181c1da0 con 0x7f2c1812cf60 2023-09-30T22:31:36.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.183+0000 7f2c1e4b7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1809d070 con 0x7f2c18130260 2023-09-30T22:31:36.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.183+0000 7f2c1e4b7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2c18129fa0 con 0x7f2c1812f8b0 2023-09-30T22:31:36.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.183+0000 7f2c17fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c1812cf60 0x7f2c181c1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:36.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.183+0000 7fe8f119d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe8ec09d6f0 0x7fe8ec11df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:36.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.183+0000 7fe8f119d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe8ec09e000 0x7fe8ec11e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:36.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.183+0000 7fe8f119d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8ec09e410 0x7fe8ec11e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:36.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.183+0000 7fe8f119d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ec11f430 con 0x7fe8ec09e410 2023-09-30T22:31:36.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.183+0000 7fe8f119d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ec1302f0 con 0x7fe8ec09e000 2023-09-30T22:31:36.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.183+0000 7fe8f119d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe8ec120d70 con 0x7fe8ec09d6f0 2023-09-30T22:31:36.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.183+0000 7fe8ea59c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe8ec09e000 0x7fe8ec11e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:36.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.183+0000 7fe8ea59c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe8ec09e000 0x7fe8ec11e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54348/0 (socket says 172.21.15.16:54348) 2023-09-30T22:31:36.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.183+0000 7fe8ea59c700 1 -- 172.21.15.16:0/984924946 learned_addr learned my addr 172.21.15.16:0/984924946 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:36.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.183+0000 7fe8eb59e700 1 --2- 172.21.15.16:0/984924946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8ec09e410 0x7fe8ec11e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:36.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.183+0000 7f2c17fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c1812cf60 0x7f2c181c1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53724/0 (socket says 172.21.15.16:53724) 2023-09-30T22:31:36.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.183+0000 7f2c17fff700 1 -- 172.21.15.16:0/143737510 learned_addr learned my addr 172.21.15.16:0/143737510 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:36.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.183+0000 7f2c177fe700 1 --2- 172.21.15.16:0/143737510 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c1812f8b0 0x7f2c181c1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:36.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.183+0000 7f2c17fff700 1 -- 172.21.15.16:0/143737510 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c1812f8b0 msgr2=0x7f2c181c1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:36.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.183+0000 7f2c17fff700 1 --2- 172.21.15.16:0/143737510 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c1812f8b0 0x7f2c181c1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:36.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.183+0000 7f2c17fff700 1 -- 172.21.15.16:0/143737510 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c18130260 msgr2=0x7f2c181c4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:36.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.183+0000 7f2c17fff700 1 --2- 172.21.15.16:0/143737510 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c18130260 0x7f2c181c4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:36.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.183+0000 7f2c17fff700 1 -- 172.21.15.16:0/143737510 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c0800d940 con 0x7f2c1812cf60 2023-09-30T22:31:36.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.184+0000 7fe8ea59c700 1 -- 172.21.15.16:0/984924946 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe8ec09d6f0 msgr2=0x7fe8ec11df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:36.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.184+0000 7fe8ea59c700 1 --2- 172.21.15.16:0/984924946 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe8ec09d6f0 0x7fe8ec11df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:36.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.184+0000 7fe8ea59c700 1 -- 172.21.15.16:0/984924946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8ec09e410 msgr2=0x7fe8ec11e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:36.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.184+0000 7fe8ea59c700 1 --2- 172.21.15.16:0/984924946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8ec09e410 0x7fe8ec11e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:36.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.184+0000 7fe8ea59c700 1 -- 172.21.15.16:0/984924946 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e400f050 con 0x7fe8ec09e000 2023-09-30T22:31:36.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.184+0000 7f2c17fff700 1 --2- 172.21.15.16:0/143737510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c1812cf60 0x7f2c181c1320 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f2c1806fba0 tx=0x7f2c08004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:36.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.184+0000 7f2c14ff9700 1 -- 172.21.15.16:0/143737510 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c080036a0 con 0x7f2c1812cf60 2023-09-30T22:31:36.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.184+0000 7f2c14ff9700 1 -- 172.21.15.16:0/143737510 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c08003820 con 0x7f2c1812cf60 2023-09-30T22:31:36.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.184+0000 7f2c1e4b7700 1 -- 172.21.15.16:0/143737510 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c1809e1e0 con 0x7f2c1812cf60 2023-09-30T22:31:36.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.184+0000 7fe8ea59c700 1 --2- 172.21.15.16:0/984924946 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe8ec09e000 0x7fe8ec11e4a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe8e000c4b0 tx=0x7fe8e000c7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:36.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.184+0000 7f2c1e4b7700 1 -- 172.21.15.16:0/143737510 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c1809dcd0 con 0x7f2c1812cf60 2023-09-30T22:31:36.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.184+0000 7f2c14ff9700 1 -- 172.21.15.16:0/143737510 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c08003bd0 con 0x7f2c1812cf60 2023-09-30T22:31:36.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.184+0000 7fe8d37fe700 1 -- 172.21.15.16:0/984924946 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e00052d0 con 0x7fe8ec09e000 2023-09-30T22:31:36.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.184+0000 7fe8d37fe700 1 -- 172.21.15.16:0/984924946 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8e0005430 con 0x7fe8ec09e000 2023-09-30T22:31:36.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.184+0000 7fe8f119d700 1 -- 172.21.15.16:0/984924946 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8ec121750 con 0x7fe8ec09e000 2023-09-30T22:31:36.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.184+0000 7fe8f119d700 1 -- 172.21.15.16:0/984924946 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8ec0a1d00 con 0x7fe8ec09e000 2023-09-30T22:31:36.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.185+0000 7fe8d37fe700 1 -- 172.21.15.16:0/984924946 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e0003880 con 0x7fe8ec09e000 2023-09-30T22:31:36.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.185+0000 7f2c14ff9700 1 -- 172.21.15.16:0/143737510 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2c08003d30 con 0x7f2c1812cf60 2023-09-30T22:31:36.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.186+0000 7f2c1e4b7700 1 -- 172.21.15.16:0/143737510 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bf80052e0 con 0x7f2c1812cf60 2023-09-30T22:31:36.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.186+0000 7f2c14ff9700 1 -- 172.21.15.16:0/143737510 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f2c08044a30 con 0x7f2c1812cf60 2023-09-30T22:31:36.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.186+0000 7fe8f119d700 1 -- 172.21.15.16:0/984924946 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8cc0052e0 con 0x7fe8ec09e000 2023-09-30T22:31:36.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.186+0000 7fe8d37fe700 1 -- 172.21.15.16:0/984924946 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe8e00039e0 con 0x7fe8ec09e000 2023-09-30T22:31:36.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.187+0000 7fe8d37fe700 1 -- 172.21.15.16:0/984924946 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fe8e003ea70 con 0x7fe8ec09e000 2023-09-30T22:31:36.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.191+0000 7f2c14ff9700 1 -- 172.21.15.16:0/143737510 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2c08015390 con 0x7f2c1812cf60 2023-09-30T22:31:36.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.193+0000 7fe8d37fe700 1 -- 172.21.15.16:0/984924946 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe8e00108d0 con 0x7fe8ec09e000 2023-09-30T22:31:36.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.263+0000 7f1f0f250700 1 -- 172.21.15.16:0/415017575 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1f0806b9e0 con 0x7f1f081338c0 2023-09-30T22:31:36.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.264+0000 7f1f057fa700 1 -- 172.21.15.16:0/415017575 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1ef402eb90 con 0x7f1f081338c0 2023-09-30T22:31:36.264 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:31:36.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.267+0000 7f1f0f250700 1 -- 172.21.15.16:0/415017575 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f081338c0 msgr2=0x7f1f0811cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:36.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.267+0000 7f1f0f250700 1 --2- 172.21.15.16:0/415017575 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f081338c0 0x7f1f0811cd40 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1ef40090b0 tx=0x7f1ef4004750 comp rx=0 tx=0).stop 2023-09-30T22:31:36.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.267+0000 7f1f0f250700 1 -- 172.21.15.16:0/415017575 shutdown_connections 2023-09-30T22:31:36.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.267+0000 7f1f0f250700 1 --2- 172.21.15.16:0/415017575 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f081341d0 0x7f1f0811d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:36.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.267+0000 7f1f0f250700 1 --2- 172.21.15.16:0/415017575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f0809e390 0x7f1f0811c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:36.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.267+0000 7f1f0f250700 1 --2- 172.21.15.16:0/415017575 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f081338c0 0x7f1f0811cd40 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:36.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.267+0000 7f1f0f250700 1 -- 172.21.15.16:0/415017575 >> 172.21.15.16:0/415017575 conn(0x7f1f080a2aa0 msgr2=0x7f1f0809f180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:36.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.267+0000 7f1f0f250700 1 -- 172.21.15.16:0/415017575 shutdown_connections 2023-09-30T22:31:36.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.267+0000 7f1f0f250700 1 -- 172.21.15.16:0/415017575 wait complete. 2023-09-30T22:31:36.287 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:31:36.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.357+0000 7f12cfb92700 1 -- 172.21.15.16:0/4286038784 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f12c806c360 con 0x7f12c809e1d0 2023-09-30T22:31:36.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.358+0000 7f12be7fc700 1 -- 172.21.15.16:0/4286038784 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f12b801a360 con 0x7f12c809e1d0 2023-09-30T22:31:36.358 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:31:36.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.360+0000 7f12cfb92700 1 -- 172.21.15.16:0/4286038784 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12c809e1d0 msgr2=0x7f12c811b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:36.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.360+0000 7f12cfb92700 1 --2- 172.21.15.16:0/4286038784 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12c809e1d0 0x7f12c811b9c0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f12b800e900 tx=0x7f12b800ec10 comp rx=0 tx=0).stop 2023-09-30T22:31:36.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.360+0000 7f12cfb92700 1 -- 172.21.15.16:0/4286038784 shutdown_connections 2023-09-30T22:31:36.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.360+0000 7f12cfb92700 1 --2- 172.21.15.16:0/4286038784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12c8135e50 0x7f12c811bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:36.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.360+0000 7f12cfb92700 1 --2- 172.21.15.16:0/4286038784 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12c809d6f0 0x7f12c811b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:36.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.360+0000 7f12cfb92700 1 --2- 172.21.15.16:0/4286038784 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12c809e1d0 0x7f12c811b9c0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:36.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.360+0000 7f12cfb92700 1 -- 172.21.15.16:0/4286038784 >> 172.21.15.16:0/4286038784 conn(0x7f12c8066f50 msgr2=0x7f12c80a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:36.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.360+0000 7f12cfb92700 1 -- 172.21.15.16:0/4286038784 shutdown_connections 2023-09-30T22:31:36.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.360+0000 7f12cfb92700 1 -- 172.21.15.16:0/4286038784 wait complete. 2023-09-30T22:31:36.375 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:31:36.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.403+0000 7fe8f119d700 1 -- 172.21.15.16:0/984924946 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe8cc005590 con 0x7fe8ec09e000 2023-09-30T22:31:36.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.403+0000 7fe8d37fe700 1 -- 172.21.15.16:0/984924946 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe8e003ee60 con 0x7fe8ec09e000 2023-09-30T22:31:36.404 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:31:36.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.406+0000 7fe8f119d700 1 -- 172.21.15.16:0/984924946 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe8ec09e000 msgr2=0x7fe8ec11e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:36.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.406+0000 7fe8f119d700 1 --2- 172.21.15.16:0/984924946 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe8ec09e000 0x7fe8ec11e4a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe8e000c4b0 tx=0x7fe8e000c7c0 comp rx=0 tx=0).stop 2023-09-30T22:31:36.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.406+0000 7fe8f119d700 1 -- 172.21.15.16:0/984924946 shutdown_connections 2023-09-30T22:31:36.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.406+0000 7fe8f119d700 1 --2- 172.21.15.16:0/984924946 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8ec09e410 0x7fe8ec11e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:36.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.406+0000 7fe8f119d700 1 --2- 172.21.15.16:0/984924946 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe8ec09d6f0 0x7fe8ec11df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:36.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.406+0000 7fe8f119d700 1 --2- 172.21.15.16:0/984924946 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe8ec09e000 0x7fe8ec11e4a0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:36.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.406+0000 7fe8f119d700 1 -- 172.21.15.16:0/984924946 >> 172.21.15.16:0/984924946 conn(0x7fe8ec066f50 msgr2=0x7fe8ec071010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:36.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.406+0000 7fe8f119d700 1 -- 172.21.15.16:0/984924946 shutdown_connections 2023-09-30T22:31:36.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.406+0000 7fe8f119d700 1 -- 172.21.15.16:0/984924946 wait complete. 2023-09-30T22:31:36.420 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:31:36.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.424+0000 7f2c1e4b7700 1 -- 172.21.15.16:0/143737510 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2bf8005590 con 0x7f2c1812cf60 2023-09-30T22:31:36.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.424+0000 7f2c14ff9700 1 -- 172.21.15.16:0/143737510 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2c08037090 con 0x7f2c1812cf60 2023-09-30T22:31:36.424 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:31:36.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.426+0000 7f2c1e4b7700 1 -- 172.21.15.16:0/143737510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c1812cf60 msgr2=0x7f2c181c1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:36.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.426+0000 7f2c1e4b7700 1 --2- 172.21.15.16:0/143737510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c1812cf60 0x7f2c181c1320 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f2c1806fba0 tx=0x7f2c08004750 comp rx=0 tx=0).stop 2023-09-30T22:31:36.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.426+0000 7f2c1e4b7700 1 -- 172.21.15.16:0/143737510 shutdown_connections 2023-09-30T22:31:36.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.426+0000 7f2c1e4b7700 1 --2- 172.21.15.16:0/143737510 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c18130260 0x7f2c181c4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:36.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.426+0000 7f2c1e4b7700 1 --2- 172.21.15.16:0/143737510 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c1812cf60 0x7f2c181c1320 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:36.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.426+0000 7f2c1e4b7700 1 --2- 172.21.15.16:0/143737510 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c1812f8b0 0x7f2c181c1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:36.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.426+0000 7f2c1e4b7700 1 -- 172.21.15.16:0/143737510 >> 172.21.15.16:0/143737510 conn(0x7f2c180a2aa0 msgr2=0x7f2c1806e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:36.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.426+0000 7f2c1e4b7700 1 -- 172.21.15.16:0/143737510 shutdown_connections 2023-09-30T22:31:36.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:36.426+0000 7f2c1e4b7700 1 -- 172.21.15.16:0/143737510 wait complete. 2023-09-30T22:31:36.440 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:31:37.288 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:37.376 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:37.421 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:37.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.436+0000 7f62f416c700 1 Processor -- start 2023-09-30T22:31:37.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.436+0000 7f62f416c700 1 -- start start 2023-09-30T22:31:37.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.437+0000 7f62f416c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f62ec09d6f0 0x7f62ec09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:37.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.437+0000 7f62f416c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f62ec09e000 0x7f62ec130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:37.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.437+0000 7f62f416c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f62ec09e410 0x7f62ec130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:37.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.437+0000 7f62f416c700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62ec09f520 con 0x7f62ec09e410 2023-09-30T22:31:37.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.437+0000 7f62f416c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec09f130 con 0x7f62ec09d6f0 2023-09-30T22:31:37.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.437+0000 7f62f416c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec09e930 con 0x7f62ec09e000 2023-09-30T22:31:37.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.437+0000 7f62f2709700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f62ec09e410 0x7f62ec130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:37.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.437+0000 7f62f1707700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f62ec09e000 0x7f62ec130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:37.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.437+0000 7f62f1f08700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f62ec09d6f0 0x7f62ec09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:37.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.437+0000 7f62f1707700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f62ec09e000 0x7f62ec130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:39620/0 (socket says 172.21.15.16:39620) 2023-09-30T22:31:37.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.437+0000 7f62f2709700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f62ec09e410 0x7f62ec130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47932/0 (socket says 172.21.15.16:47932) 2023-09-30T22:31:37.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.437+0000 7f62f1707700 1 -- 172.21.15.16:0/3768163590 learned_addr learned my addr 172.21.15.16:0/3768163590 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:37.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.437+0000 7f62f1f08700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f62ec09d6f0 0x7f62ec09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:40288/0 (socket says 172.21.15.16:40288) 2023-09-30T22:31:37.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.438+0000 7f62f1f08700 1 -- 172.21.15.16:0/3768163590 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f62ec09e000 msgr2=0x7f62ec130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:37.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.438+0000 7f62f1f08700 1 --2- 172.21.15.16:0/3768163590 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f62ec09e000 0x7f62ec130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:37.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.438+0000 7f62f1f08700 1 -- 172.21.15.16:0/3768163590 >> v2:172.21.15.16:3300/0 conn(0x7f62ec09e410 msgr2=0x7f62ec130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:37.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.438+0000 7f62f1f08700 1 --2- 172.21.15.16:0/3768163590 >> v2:172.21.15.16:3300/0 conn(0x7f62ec09e410 0x7f62ec130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:37.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.438+0000 7f62f1f08700 1 -- 172.21.15.16:0/3768163590 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62ec09b2d0 con 0x7f62ec09d6f0 2023-09-30T22:31:37.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.438+0000 7f62f1707700 1 --2- 172.21.15.16:0/3768163590 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f62ec09e000 0x7f62ec130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:37.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.438+0000 7f62f2709700 1 --2- 172.21.15.16:0/3768163590 >> v2:172.21.15.16:3300/0 conn(0x7f62ec09e410 0x7f62ec130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:37.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.438+0000 7f62f1f08700 1 --2- 172.21.15.16:0/3768163590 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f62ec09d6f0 0x7f62ec09dac0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f62e800dc90 tx=0x7f62e800f050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a31e3fad2156a384 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:37.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.438+0000 7f62f0f06700 1 -- 172.21.15.16:0/3768163590 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e8012070 con 0x7f62ec09d6f0 2023-09-30T22:31:37.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.438+0000 7f62f0f06700 1 -- 172.21.15.16:0/3768163590 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62e800d2e0 con 0x7f62ec09d6f0 2023-09-30T22:31:37.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.438+0000 7f62f0f06700 1 -- 172.21.15.16:0/3768163590 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e800f8b0 con 0x7f62ec09d6f0 2023-09-30T22:31:37.441 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:37.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.439+0000 7f62f416c700 1 -- 172.21.15.16:0/3768163590 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f62ec09d6f0 msgr2=0x7f62ec09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:37.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.439+0000 7f62f416c700 1 --2- 172.21.15.16:0/3768163590 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f62ec09d6f0 0x7f62ec09dac0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f62e800dc90 tx=0x7f62e800f050 comp rx=0 tx=0).stop 2023-09-30T22:31:37.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.439+0000 7f62f416c700 1 -- 172.21.15.16:0/3768163590 shutdown_connections 2023-09-30T22:31:37.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.439+0000 7f62f416c700 1 --2- 172.21.15.16:0/3768163590 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f62ec09d6f0 0x7f62ec09dac0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:37.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.439+0000 7f62f416c700 1 --2- 172.21.15.16:0/3768163590 >> v2:172.21.15.16:3300/0 conn(0x7f62ec09e410 0x7f62ec130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:37.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.439+0000 7f62f416c700 1 --2- 172.21.15.16:0/3768163590 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f62ec09e000 0x7f62ec130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:37.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.439+0000 7f62f416c700 1 -- 172.21.15.16:0/3768163590 >> 172.21.15.16:0/3768163590 conn(0x7f62ec066f50 msgr2=0x7f62ec09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:37.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.439+0000 7f62f416c700 1 -- 172.21.15.16:0/3768163590 shutdown_connections 2023-09-30T22:31:37.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.439+0000 7f62f416c700 1 -- 172.21.15.16:0/3768163590 wait complete. 2023-09-30T22:31:37.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.440+0000 7f62f416c700 1 Processor -- start 2023-09-30T22:31:37.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.440+0000 7f62f416c700 1 -- start start 2023-09-30T22:31:37.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.441+0000 7f62f416c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62ec09d6f0 0x7f62ec11dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:37.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.441+0000 7f62f416c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f62ec09e000 0x7f62ec11e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:37.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.441+0000 7f62f416c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f62ec09e410 0x7f62ec11e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:37.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.441+0000 7f62f416c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec121a00 con 0x7f62ec09d6f0 2023-09-30T22:31:37.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.441+0000 7f62f416c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec0a17f0 con 0x7f62ec09e000 2023-09-30T22:31:37.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.441+0000 7f62f416c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f62ec11f320 con 0x7f62ec09e410 2023-09-30T22:31:37.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.441+0000 7f62f1707700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f62ec09e000 0x7f62ec11e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:37.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.441+0000 7f62f1f08700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62ec09d6f0 0x7f62ec11dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:37.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.441+0000 7f62f1707700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f62ec09e000 0x7f62ec11e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:40292/0 (socket says 172.21.15.16:40292) 2023-09-30T22:31:37.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.441+0000 7f62f2709700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f62ec09e410 0x7f62ec11e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:37.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.441+0000 7f62f1f08700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62ec09d6f0 0x7f62ec11dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47934/0 (socket says 172.21.15.16:47934) 2023-09-30T22:31:37.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.441+0000 7f62f1707700 1 -- 172.21.15.16:0/2563847535 learned_addr learned my addr 172.21.15.16:0/2563847535 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:37.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.441+0000 7f62f1707700 1 -- 172.21.15.16:0/2563847535 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f62ec09e410 msgr2=0x7f62ec11e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:37.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.441+0000 7f62f1707700 1 --2- 172.21.15.16:0/2563847535 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f62ec09e410 0x7f62ec11e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:37.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.441+0000 7f62f1707700 1 -- 172.21.15.16:0/2563847535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62ec09d6f0 msgr2=0x7f62ec11dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:37.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.441+0000 7f62f1707700 1 --2- 172.21.15.16:0/2563847535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62ec09d6f0 0x7f62ec11dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:37.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.441+0000 7f62f1707700 1 -- 172.21.15.16:0/2563847535 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62dc009550 con 0x7f62ec09e000 2023-09-30T22:31:37.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.441+0000 7f62f2709700 1 --2- 172.21.15.16:0/2563847535 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f62ec09e410 0x7f62ec11e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:37.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.441+0000 7f62f1707700 1 --2- 172.21.15.16:0/2563847535 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f62ec09e000 0x7f62ec11e420 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f62dc00ebe0 tx=0x7f62dc00eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:37.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.442+0000 7f62e27fc700 1 -- 172.21.15.16:0/2563847535 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62dc004240 con 0x7f62ec09e000 2023-09-30T22:31:37.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.442+0000 7f62e27fc700 1 -- 172.21.15.16:0/2563847535 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62dc0043a0 con 0x7f62ec09e000 2023-09-30T22:31:37.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.442+0000 7f62f416c700 1 -- 172.21.15.16:0/2563847535 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62e4009a70 con 0x7f62ec09e000 2023-09-30T22:31:37.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.442+0000 7f62e27fc700 1 -- 172.21.15.16:0/2563847535 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62dc0036a0 con 0x7f62ec09e000 2023-09-30T22:31:37.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.442+0000 7f62f416c700 1 -- 172.21.15.16:0/2563847535 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62e800d940 con 0x7f62ec09e000 2023-09-30T22:31:37.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.443+0000 7f62e27fc700 1 -- 172.21.15.16:0/2563847535 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f62dc003800 con 0x7f62ec09e000 2023-09-30T22:31:37.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.443+0000 7f62e27fc700 1 -- 172.21.15.16:0/2563847535 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f62dc03fc20 con 0x7f62ec09e000 2023-09-30T22:31:37.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.443+0000 7f62f416c700 1 -- 172.21.15.16:0/2563847535 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62ec074ea0 con 0x7f62ec09e000 2023-09-30T22:31:37.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.448+0000 7f62e27fc700 1 -- 172.21.15.16:0/2563847535 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f62dc024350 con 0x7f62ec09e000 2023-09-30T22:31:37.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.542+0000 7f27eb0f7700 1 Processor -- start 2023-09-30T22:31:37.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.542+0000 7f27eb0f7700 1 -- start start 2023-09-30T22:31:37.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.543+0000 7f27eb0f7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f27e406a770 0x7f27e406ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:37.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.543+0000 7f27eb0f7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f27e406b080 0x7f27e4131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:37.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.543+0000 7f27eb0f7700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f27e4131b40 0x7f27e4131fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:37.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.543+0000 7f27eb0f7700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27e409ece0 con 0x7f27e4131b40 2023-09-30T22:31:37.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.543+0000 7f27eb0f7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27e409e760 con 0x7f27e406a770 2023-09-30T22:31:37.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.543+0000 7f27eb0f7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f27e409e040 con 0x7f27e406b080 2023-09-30T22:31:37.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.543+0000 7f27e3fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f27e406b080 0x7f27e4131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:37.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.543+0000 7f27e8e93700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f27e406a770 0x7f27e406ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:37.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.543+0000 7f27e3fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f27e406b080 0x7f27e4131600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:39648/0 (socket says 172.21.15.16:39648) 2023-09-30T22:31:37.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.543+0000 7f27e8e93700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f27e406a770 0x7f27e406ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:40296/0 (socket says 172.21.15.16:40296) 2023-09-30T22:31:37.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.543+0000 7f27e3fff700 1 -- 172.21.15.16:0/948472749 learned_addr learned my addr 172.21.15.16:0/948472749 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:37.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.543+0000 7f27e9694700 1 --2- 172.21.15.16:0/948472749 >> v2:172.21.15.16:3300/0 conn(0x7f27e4131b40 0x7f27e4131fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:37.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.544+0000 7f27e3fff700 1 -- 172.21.15.16:0/948472749 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f27e406a770 msgr2=0x7f27e406ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:37.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.544+0000 7f27e3fff700 1 --2- 172.21.15.16:0/948472749 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f27e406a770 0x7f27e406ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:37.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.544+0000 7f27e3fff700 1 -- 172.21.15.16:0/948472749 >> v2:172.21.15.16:3300/0 conn(0x7f27e4131b40 msgr2=0x7f27e4131fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:37.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.544+0000 7f27e3fff700 1 --2- 172.21.15.16:0/948472749 >> v2:172.21.15.16:3300/0 conn(0x7f27e4131b40 0x7f27e4131fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:37.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.544+0000 7f27e3fff700 1 -- 172.21.15.16:0/948472749 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e409f630 con 0x7f27e406b080 2023-09-30T22:31:37.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.544+0000 7f27e3fff700 1 --2- 172.21.15.16:0/948472749 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f27e406b080 0x7f27e4131600 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f27d00098f0 tx=0x7f27d0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c770628a28ec76f4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:37.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.544+0000 7f27e37fe700 1 -- 172.21.15.16:0/948472749 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d0011830 con 0x7f27e406b080 2023-09-30T22:31:37.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.544+0000 7f27e37fe700 1 -- 172.21.15.16:0/948472749 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27d0011990 con 0x7f27e406b080 2023-09-30T22:31:37.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.544+0000 7f27e37fe700 1 -- 172.21.15.16:0/948472749 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d0011d20 con 0x7f27e406b080 2023-09-30T22:31:37.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.545+0000 7f27eb0f7700 1 -- 172.21.15.16:0/948472749 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f27e406b080 msgr2=0x7f27e4131600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:37.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.545+0000 7f27eb0f7700 1 --2- 172.21.15.16:0/948472749 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f27e406b080 0x7f27e4131600 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f27d00098f0 tx=0x7f27d0009c00 comp rx=0 tx=0).stop 2023-09-30T22:31:37.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.545+0000 7f27eb0f7700 1 -- 172.21.15.16:0/948472749 shutdown_connections 2023-09-30T22:31:37.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.545+0000 7f27eb0f7700 1 --2- 172.21.15.16:0/948472749 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f27e406a770 0x7f27e406ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:37.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.545+0000 7f27eb0f7700 1 --2- 172.21.15.16:0/948472749 >> v2:172.21.15.16:3300/0 conn(0x7f27e4131b40 0x7f27e4131fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:37.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.545+0000 7f27eb0f7700 1 --2- 172.21.15.16:0/948472749 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f27e406b080 0x7f27e4131600 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:37.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.545+0000 7f27eb0f7700 1 -- 172.21.15.16:0/948472749 >> 172.21.15.16:0/948472749 conn(0x7f27e4066f50 msgr2=0x7f27e409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:37.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.545+0000 7f27eb0f7700 1 -- 172.21.15.16:0/948472749 shutdown_connections 2023-09-30T22:31:37.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.545+0000 7f27eb0f7700 1 -- 172.21.15.16:0/948472749 wait complete. 2023-09-30T22:31:37.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.546+0000 7f27eb0f7700 1 Processor -- start 2023-09-30T22:31:37.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.546+0000 7f27eb0f7700 1 -- start start 2023-09-30T22:31:37.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.547+0000 7f27eb0f7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f27e406a770 0x7f27e409a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:37.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.547+0000 7f27eb0f7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f27e406b080 0x7f27e409a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:37.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.547+0000 7f27eb0f7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27e4131b40 0x7f27e409ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:37.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.547+0000 7f27eb0f7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e4071e60 con 0x7f27e4131b40 2023-09-30T22:31:37.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.547+0000 7f27eb0f7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27e409c080 con 0x7f27e406b080 2023-09-30T22:31:37.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.547+0000 7f27eb0f7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f27e41258c0 con 0x7f27e406a770 2023-09-30T22:31:37.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.548+0000 7f27e8e93700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f27e406a770 0x7f27e409a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:37.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.548+0000 7f27e8e93700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f27e406a770 0x7f27e409a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:39664/0 (socket says 172.21.15.16:39664) 2023-09-30T22:31:37.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.548+0000 7f27e9694700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27e4131b40 0x7f27e409ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:37.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.548+0000 7f27e8e93700 1 -- 172.21.15.16:0/3843341477 learned_addr learned my addr 172.21.15.16:0/3843341477 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:37.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.548+0000 7f27e3fff700 1 --2- 172.21.15.16:0/3843341477 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f27e406b080 0x7f27e409a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:37.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.548+0000 7f27e8e93700 1 -- 172.21.15.16:0/3843341477 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f27e406b080 msgr2=0x7f27e409a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:37.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.548+0000 7f27e8e93700 1 --2- 172.21.15.16:0/3843341477 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f27e406b080 0x7f27e409a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:37.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.548+0000 7f27e8e93700 1 -- 172.21.15.16:0/3843341477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27e4131b40 msgr2=0x7f27e409ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:37.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.548+0000 7f27e8e93700 1 --2- 172.21.15.16:0/3843341477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27e4131b40 0x7f27e409ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:37.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.548+0000 7f27e8e93700 1 -- 172.21.15.16:0/3843341477 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27d0009550 con 0x7f27e406a770 2023-09-30T22:31:37.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.548+0000 7f27e9694700 1 --2- 172.21.15.16:0/3843341477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27e4131b40 0x7f27e409ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:37.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.548+0000 7f27e8e93700 1 --2- 172.21.15.16:0/3843341477 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f27e406a770 0x7f27e409a3d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f27d800e9b0 tx=0x7f27d800ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:37.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.549+0000 7f27e17fa700 1 -- 172.21.15.16:0/3843341477 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d8009710 con 0x7f27e406a770 2023-09-30T22:31:37.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.549+0000 7f27e17fa700 1 -- 172.21.15.16:0/3843341477 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27d800c850 con 0x7f27e406a770 2023-09-30T22:31:37.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.549+0000 7f27eb0f7700 1 -- 172.21.15.16:0/3843341477 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27e41312e0 con 0x7f27e406a770 2023-09-30T22:31:37.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.549+0000 7f27e17fa700 1 -- 172.21.15.16:0/3843341477 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d800cbc0 con 0x7f27e406a770 2023-09-30T22:31:37.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.549+0000 7f27eb0f7700 1 -- 172.21.15.16:0/3843341477 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27e4131520 con 0x7f27e406a770 2023-09-30T22:31:37.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.550+0000 7f27eb0f7700 1 -- 172.21.15.16:0/3843341477 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27e4074ea0 con 0x7f27e406a770 2023-09-30T22:31:37.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.551+0000 7f27e17fa700 1 -- 172.21.15.16:0/3843341477 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f27d800cd20 con 0x7f27e406a770 2023-09-30T22:31:37.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.551+0000 7f27e17fa700 1 -- 172.21.15.16:0/3843341477 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f27d803ff80 con 0x7f27e406a770 2023-09-30T22:31:37.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.560+0000 7f27e17fa700 1 -- 172.21.15.16:0/3843341477 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f27d801a360 con 0x7f27e406a770 2023-09-30T22:31:37.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.572+0000 7fbbb4464700 1 Processor -- start 2023-09-30T22:31:37.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.573+0000 7fbbb4464700 1 -- start start 2023-09-30T22:31:37.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.573+0000 7fbbb4464700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbbac09e930 0x7fbbac09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:37.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.573+0000 7fbbb4464700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbbac09f240 0x7fbbac1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:37.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.573+0000 7fbbb4464700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbbac09f650 0x7fbbac131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:37.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.573+0000 7fbbb4464700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbac070940 con 0x7fbbac09f650 2023-09-30T22:31:37.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.573+0000 7fbbb4464700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbac06fd60 con 0x7fbbac09e930 2023-09-30T22:31:37.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.573+0000 7fbbb4464700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbbac06fa80 con 0x7fbbac09f240 2023-09-30T22:31:37.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.574+0000 7fbbb2200700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbbac09e930 0x7fbbac09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:37.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.574+0000 7fbbb2200700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbbac09e930 0x7fbbac09ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:40316/0 (socket says 172.21.15.16:40316) 2023-09-30T22:31:37.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.574+0000 7fbbb2200700 1 -- 172.21.15.16:0/1352650194 learned_addr learned my addr 172.21.15.16:0/1352650194 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:37.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.574+0000 7fbbb2a01700 1 --2- 172.21.15.16:0/1352650194 >> v2:172.21.15.16:3300/0 conn(0x7fbbac09f650 0x7fbbac131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:37.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.574+0000 7fbbb19ff700 1 --2- 172.21.15.16:0/1352650194 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbbac09f240 0x7fbbac1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:37.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.574+0000 7fbbb2200700 1 -- 172.21.15.16:0/1352650194 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbbac09f240 msgr2=0x7fbbac1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:37.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.574+0000 7fbbb2200700 1 --2- 172.21.15.16:0/1352650194 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbbac09f240 0x7fbbac1317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:37.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.574+0000 7fbbb2200700 1 -- 172.21.15.16:0/1352650194 >> v2:172.21.15.16:3300/0 conn(0x7fbbac09f650 msgr2=0x7fbbac131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:37.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.574+0000 7fbbb2200700 1 --2- 172.21.15.16:0/1352650194 >> v2:172.21.15.16:3300/0 conn(0x7fbbac09f650 0x7fbbac131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:37.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.574+0000 7fbbb2200700 1 -- 172.21.15.16:0/1352650194 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbac09fb60 con 0x7fbbac09e930 2023-09-30T22:31:37.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.574+0000 7fbbb2200700 1 --2- 172.21.15.16:0/1352650194 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbbac09e930 0x7fbbac09ed00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fbba80098f0 tx=0x7fbba8009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5f58e70a0e27c09 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:37.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.574+0000 7fbbb11fe700 1 -- 172.21.15.16:0/1352650194 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba8010a90 con 0x7fbbac09e930 2023-09-30T22:31:37.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.575+0000 7fbbb11fe700 1 -- 172.21.15.16:0/1352650194 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba8010bf0 con 0x7fbbac09e930 2023-09-30T22:31:37.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.575+0000 7fbbb11fe700 1 -- 172.21.15.16:0/1352650194 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba8018440 con 0x7fbbac09e930 2023-09-30T22:31:37.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.576+0000 7fbbb4464700 1 -- 172.21.15.16:0/1352650194 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbbac09e930 msgr2=0x7fbbac09ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:37.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.576+0000 7fbbb4464700 1 --2- 172.21.15.16:0/1352650194 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbbac09e930 0x7fbbac09ed00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fbba80098f0 tx=0x7fbba8009c00 comp rx=0 tx=0).stop 2023-09-30T22:31:37.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.576+0000 7fbbb4464700 1 -- 172.21.15.16:0/1352650194 shutdown_connections 2023-09-30T22:31:37.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.576+0000 7fbbb4464700 1 --2- 172.21.15.16:0/1352650194 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbbac09e930 0x7fbbac09ed00 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:37.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.576+0000 7fbbb4464700 1 --2- 172.21.15.16:0/1352650194 >> v2:172.21.15.16:3300/0 conn(0x7fbbac09f650 0x7fbbac131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:37.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.576+0000 7fbbb4464700 1 --2- 172.21.15.16:0/1352650194 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbbac09f240 0x7fbbac1317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:37.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.576+0000 7fbbb4464700 1 -- 172.21.15.16:0/1352650194 >> 172.21.15.16:0/1352650194 conn(0x7fbbac0a4210 msgr2=0x7fbbac0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:37.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.576+0000 7fbbb4464700 1 -- 172.21.15.16:0/1352650194 shutdown_connections 2023-09-30T22:31:37.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.576+0000 7fbbb4464700 1 -- 172.21.15.16:0/1352650194 wait complete. 2023-09-30T22:31:37.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.577+0000 7fbbb4464700 1 Processor -- start 2023-09-30T22:31:37.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.577+0000 7fbbb4464700 1 -- start start 2023-09-30T22:31:37.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.578+0000 7fbbb4464700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbbac09e930 0x7fbbac1a2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:37.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.578+0000 7fbbb4464700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbac09f240 0x7fbbac1a2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:37.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.578+0000 7fbbb4464700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbbac09f650 0x7fbbac1c3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:37.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.578+0000 7fbbb4464700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbac1a5240 con 0x7fbbac09f240 2023-09-30T22:31:37.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.578+0000 7fbbb4464700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbac1a4720 con 0x7fbbac09f650 2023-09-30T22:31:37.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.578+0000 7fbbb4464700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbbac1a4f80 con 0x7fbbac09e930 2023-09-30T22:31:37.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.578+0000 7fbbb19ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbac09f240 0x7fbbac1a2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:37.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.578+0000 7fbbb2200700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbbac09e930 0x7fbbac1a2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:37.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.578+0000 7fbbb19ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbac09f240 0x7fbbac1a2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47986/0 (socket says 172.21.15.16:47986) 2023-09-30T22:31:37.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.578+0000 7fbbb19ff700 1 -- 172.21.15.16:0/1882307204 learned_addr learned my addr 172.21.15.16:0/1882307204 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:37.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.578+0000 7fbbb2a01700 1 --2- 172.21.15.16:0/1882307204 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbbac09f650 0x7fbbac1c3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:37.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.579+0000 7fbbb2a01700 1 -- 172.21.15.16:0/1882307204 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbbac09e930 msgr2=0x7fbbac1a2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:37.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.579+0000 7fbbb2a01700 1 --2- 172.21.15.16:0/1882307204 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbbac09e930 0x7fbbac1a2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:37.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.579+0000 7fbbb2a01700 1 -- 172.21.15.16:0/1882307204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbac09f240 msgr2=0x7fbbac1a2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:37.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.579+0000 7fbbb2a01700 1 --2- 172.21.15.16:0/1882307204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbac09f240 0x7fbbac1a2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:37.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.579+0000 7fbbb2a01700 1 -- 172.21.15.16:0/1882307204 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba8009550 con 0x7fbbac09f650 2023-09-30T22:31:37.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.579+0000 7fbbb2a01700 1 --2- 172.21.15.16:0/1882307204 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbbac09f650 0x7fbbac1c3cd0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fbba400ef90 tx=0x7fbba400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:37.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.579+0000 7fbba2ffd700 1 -- 172.21.15.16:0/1882307204 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba400cc90 con 0x7fbbac09f650 2023-09-30T22:31:37.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.579+0000 7fbba2ffd700 1 -- 172.21.15.16:0/1882307204 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba400f040 con 0x7fbbac09f650 2023-09-30T22:31:37.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.579+0000 7fbbb4464700 1 -- 172.21.15.16:0/1882307204 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbac0a0190 con 0x7fbbac09f650 2023-09-30T22:31:37.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.579+0000 7fbba2ffd700 1 -- 172.21.15.16:0/1882307204 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba40146c0 con 0x7fbbac09f650 2023-09-30T22:31:37.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.579+0000 7fbbb4464700 1 -- 172.21.15.16:0/1882307204 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbac1a4280 con 0x7fbbac09f650 2023-09-30T22:31:37.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.580+0000 7fbba2ffd700 1 -- 172.21.15.16:0/1882307204 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbba4014820 con 0x7fbbac09f650 2023-09-30T22:31:37.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.581+0000 7fbba2ffd700 1 -- 172.21.15.16:0/1882307204 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fbba4040260 con 0x7fbbac09f650 2023-09-30T22:31:37.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.581+0000 7fbbb4464700 1 -- 172.21.15.16:0/1882307204 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbac1a4c90 con 0x7fbbac09f650 2023-09-30T22:31:37.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.586+0000 7fbba2ffd700 1 -- 172.21.15.16:0/1882307204 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbba401d400 con 0x7fbbac09f650 2023-09-30T22:31:37.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.589+0000 7fbbd13ed700 1 Processor -- start 2023-09-30T22:31:37.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.590+0000 7fbbd13ed700 1 -- start start 2023-09-30T22:31:37.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.590+0000 7fbbd13ed700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbbcc06a770 0x7fbbcc09d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:37.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.590+0000 7fbbd13ed700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbbcc09dc30 0x7fbbcc09e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:37.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.590+0000 7fbbd13ed700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbcc09a810 con 0x7fbbcc13c1e0 2023-09-30T22:31:37.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.590+0000 7fbbd13ed700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbcc06ab40 con 0x7fbbcc09dc30 2023-09-30T22:31:37.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.590+0000 7fbbd13ed700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbcc09ebc0 con 0x7fbbcc06a770 2023-09-30T22:31:37.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.590+0000 7fbbca7fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbbcc09dc30 0x7fbbcc09e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:37.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.590+0000 7fbbca7fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbbcc09dc30 0x7fbbcc09e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47998/0 (socket says 172.21.15.16:47998) 2023-09-30T22:31:37.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.590+0000 7fbbca7fc700 1 -- 172.21.15.16:0/3836685395 learned_addr learned my addr 172.21.15.16:0/3836685395 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:37.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.591+0000 7fbbcaffd700 1 --2- 172.21.15.16:0/3836685395 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbbcc06a770 0x7fbbcc09d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:37.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.591+0000 7fbbca7fc700 1 -- 172.21.15.16:0/3836685395 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbbcc06a770 msgr2=0x7fbbcc09d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:37.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.591+0000 7fbbca7fc700 1 --2- 172.21.15.16:0/3836685395 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbbcc06a770 0x7fbbcc09d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:37.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.591+0000 7fbbca7fc700 1 -- 172.21.15.16:0/3836685395 >> v1:172.21.15.16:6789/0 conn(0x7fbbcc13c1e0 legacy=0x7fbbcc13e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:37.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.591+0000 7fbbca7fc700 1 -- 172.21.15.16:0/3836685395 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbcc09e920 con 0x7fbbcc09dc30 2023-09-30T22:31:37.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.591+0000 7fbbca7fc700 1 --2- 172.21.15.16:0/3836685395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbcc09dc30 0x7fbbcc09e080 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0009920 tx=0x7fbbc0009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d401a3259ef8c34b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:37.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.591+0000 7fbbc9ffb700 1 -- 172.21.15.16:0/3836685395 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0004230 con 0x7fbbcc09dc30 2023-09-30T22:31:37.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.591+0000 7fbbc9ffb700 1 -- 172.21.15.16:0/3836685395 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbc0004390 con 0x7fbbcc09dc30 2023-09-30T22:31:37.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.591+0000 7fbbc9ffb700 1 -- 172.21.15.16:0/3836685395 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0009e70 con 0x7fbbcc09dc30 2023-09-30T22:31:37.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.592+0000 7fbbd13ed700 1 -- 172.21.15.16:0/3836685395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbcc09dc30 msgr2=0x7fbbcc09e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:37.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.592+0000 7fbbd13ed700 1 --2- 172.21.15.16:0/3836685395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbcc09dc30 0x7fbbcc09e080 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0009920 tx=0x7fbbc0009c30 comp rx=0 tx=0).stop 2023-09-30T22:31:37.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.592+0000 7fbbd13ed700 1 -- 172.21.15.16:0/3836685395 shutdown_connections 2023-09-30T22:31:37.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.592+0000 7fbbd13ed700 1 --2- 172.21.15.16:0/3836685395 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbbcc06a770 0x7fbbcc09d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:37.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.592+0000 7fbbd13ed700 1 --2- 172.21.15.16:0/3836685395 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbcc09dc30 0x7fbbcc09e080 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:37.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.592+0000 7fbbd13ed700 1 -- 172.21.15.16:0/3836685395 >> 172.21.15.16:0/3836685395 conn(0x7fbbcc066f50 msgr2=0x7fbbcc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:37.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.592+0000 7fbbd13ed700 1 -- 172.21.15.16:0/3836685395 shutdown_connections 2023-09-30T22:31:37.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.592+0000 7fbbd13ed700 1 -- 172.21.15.16:0/3836685395 wait complete. 2023-09-30T22:31:37.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.593+0000 7fbbd13ed700 1 Processor -- start 2023-09-30T22:31:37.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.593+0000 7fbbd13ed700 1 -- start start 2023-09-30T22:31:37.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.594+0000 7fbbd13ed700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbcc06a770 0x7fbbcc13e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:37.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.594+0000 7fbbd13ed700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbbcc09dc30 0x7fbbcc13a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:37.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.594+0000 7fbbd13ed700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbbcc13c1e0 0x7fbbcc13a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:37.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.594+0000 7fbbd13ed700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbcc134b90 con 0x7fbbcc06a770 2023-09-30T22:31:37.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.594+0000 7fbbd13ed700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbcc13bd40 con 0x7fbbcc09dc30 2023-09-30T22:31:37.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.594+0000 7fbbd13ed700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbbcc09f1a0 con 0x7fbbcc13c1e0 2023-09-30T22:31:37.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.594+0000 7fbbcaffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbcc06a770 0x7fbbcc13e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:37.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.594+0000 7fbbcaffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbcc06a770 0x7fbbcc13e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48000/0 (socket says 172.21.15.16:48000) 2023-09-30T22:31:37.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.594+0000 7fbbcaffd700 1 -- 172.21.15.16:0/4257793196 learned_addr learned my addr 172.21.15.16:0/4257793196 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:37.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.594+0000 7fbbca7fc700 1 --2- 172.21.15.16:0/4257793196 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbbcc09dc30 0x7fbbcc13a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:37.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.594+0000 7fbbcb7fe700 1 --2- 172.21.15.16:0/4257793196 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbbcc13c1e0 0x7fbbcc13a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:37.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.594+0000 7fbbcaffd700 1 -- 172.21.15.16:0/4257793196 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbbcc13c1e0 msgr2=0x7fbbcc13a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:37.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.594+0000 7fbbcaffd700 1 --2- 172.21.15.16:0/4257793196 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbbcc13c1e0 0x7fbbcc13a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:37.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.594+0000 7fbbcaffd700 1 -- 172.21.15.16:0/4257793196 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbbcc09dc30 msgr2=0x7fbbcc13a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:37.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.594+0000 7fbbcaffd700 1 --2- 172.21.15.16:0/4257793196 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbbcc09dc30 0x7fbbcc13a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:37.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.594+0000 7fbbcaffd700 1 -- 172.21.15.16:0/4257793196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbc0009550 con 0x7fbbcc06a770 2023-09-30T22:31:37.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.595+0000 7fbbcaffd700 1 --2- 172.21.15.16:0/4257793196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbcc06a770 0x7fbbcc13e010 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc00e810 tx=0x7fbbbc00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:37.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.595+0000 7fbbb37fe700 1 -- 172.21.15.16:0/4257793196 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc013610 con 0x7fbbcc06a770 2023-09-30T22:31:37.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.595+0000 7fbbb37fe700 1 -- 172.21.15.16:0/4257793196 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbbc013770 con 0x7fbbcc06a770 2023-09-30T22:31:37.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.595+0000 7fbbd13ed700 1 -- 172.21.15.16:0/4257793196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbcc0a3860 con 0x7fbbcc06a770 2023-09-30T22:31:37.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.595+0000 7fbbb37fe700 1 -- 172.21.15.16:0/4257793196 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc013b20 con 0x7fbbcc06a770 2023-09-30T22:31:37.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.595+0000 7fbbd13ed700 1 -- 172.21.15.16:0/4257793196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbcc13fe80 con 0x7fbbcc06a770 2023-09-30T22:31:37.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.596+0000 7fbbb37fe700 1 -- 172.21.15.16:0/4257793196 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbbbc013c80 con 0x7fbbcc06a770 2023-09-30T22:31:37.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.596+0000 7fbbb37fe700 1 -- 172.21.15.16:0/4257793196 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fbbbc03f220 con 0x7fbbcc06a770 2023-09-30T22:31:37.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.596+0000 7fbbd13ed700 1 -- 172.21.15.16:0/4257793196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbcc0a3860 con 0x7fbbcc06a770 2023-09-30T22:31:37.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.605+0000 7fbbb37fe700 1 -- 172.21.15.16:0/4257793196 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbbcc0a3860 con 0x7fbbcc06a770 2023-09-30T22:31:37.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.729+0000 7f62f416c700 1 -- 172.21.15.16:0/2563847535 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f62ec11faf0 con 0x7f62ec09e000 2023-09-30T22:31:37.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.730+0000 7f62e27fc700 1 -- 172.21.15.16:0/2563847535 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f62dc02a070 con 0x7f62ec09e000 2023-09-30T22:31:37.730 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:31:37.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.733+0000 7f62f416c700 1 -- 172.21.15.16:0/2563847535 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f62ec09e000 msgr2=0x7f62ec11e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:37.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.733+0000 7f62f416c700 1 --2- 172.21.15.16:0/2563847535 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f62ec09e000 0x7f62ec11e420 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f62dc00ebe0 tx=0x7f62dc00eef0 comp rx=0 tx=0).stop 2023-09-30T22:31:37.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.733+0000 7f62f416c700 1 -- 172.21.15.16:0/2563847535 shutdown_connections 2023-09-30T22:31:37.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.733+0000 7f62f416c700 1 --2- 172.21.15.16:0/2563847535 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f62ec09e410 0x7f62ec11e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:37.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.733+0000 7f62f416c700 1 --2- 172.21.15.16:0/2563847535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f62ec09d6f0 0x7f62ec11dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:37.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.733+0000 7f62f416c700 1 --2- 172.21.15.16:0/2563847535 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f62ec09e000 0x7f62ec11e420 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:37.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.733+0000 7f62f416c700 1 -- 172.21.15.16:0/2563847535 >> 172.21.15.16:0/2563847535 conn(0x7f62ec066f50 msgr2=0x7f62ec06a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:37.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.733+0000 7f62f416c700 1 -- 172.21.15.16:0/2563847535 shutdown_connections 2023-09-30T22:31:37.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.733+0000 7f62f416c700 1 -- 172.21.15.16:0/2563847535 wait complete. 2023-09-30T22:31:37.747 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:31:37.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.799+0000 7fbbb4464700 1 -- 172.21.15.16:0/1882307204 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fbbac074ea0 con 0x7fbbac09f650 2023-09-30T22:31:37.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.799+0000 7fbba2ffd700 1 -- 172.21.15.16:0/1882307204 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbba4012340 con 0x7fbbac09f650 2023-09-30T22:31:37.800 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:31:37.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.802+0000 7fbbb4464700 1 -- 172.21.15.16:0/1882307204 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbbac09f650 msgr2=0x7fbbac1c3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:37.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.802+0000 7fbbb4464700 1 --2- 172.21.15.16:0/1882307204 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbbac09f650 0x7fbbac1c3cd0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fbba400ef90 tx=0x7fbba400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:31:37.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.802+0000 7fbbb4464700 1 -- 172.21.15.16:0/1882307204 shutdown_connections 2023-09-30T22:31:37.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.802+0000 7fbbb4464700 1 --2- 172.21.15.16:0/1882307204 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbbac09f650 0x7fbbac1c3cd0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:37.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.802+0000 7fbbb4464700 1 --2- 172.21.15.16:0/1882307204 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbbac09e930 0x7fbbac1a2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:37.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.802+0000 7fbbb4464700 1 --2- 172.21.15.16:0/1882307204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbac09f240 0x7fbbac1a2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:37.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.802+0000 7fbbb4464700 1 -- 172.21.15.16:0/1882307204 >> 172.21.15.16:0/1882307204 conn(0x7fbbac0a4210 msgr2=0x7fbbac1b6ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:37.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.802+0000 7fbbb4464700 1 -- 172.21.15.16:0/1882307204 shutdown_connections 2023-09-30T22:31:37.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.802+0000 7fbbb4464700 1 -- 172.21.15.16:0/1882307204 wait complete. 2023-09-30T22:31:37.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.814+0000 7f27eb0f7700 1 -- 172.21.15.16:0/3843341477 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f27e406c360 con 0x7f27e406a770 2023-09-30T22:31:37.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.814+0000 7f27e17fa700 1 -- 172.21.15.16:0/3843341477 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f27d8021020 con 0x7f27e406a770 2023-09-30T22:31:37.815 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:31:37.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.817+0000 7f27eb0f7700 1 -- 172.21.15.16:0/3843341477 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f27e406a770 msgr2=0x7f27e409a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:37.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.817+0000 7f27eb0f7700 1 --2- 172.21.15.16:0/3843341477 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f27e406a770 0x7f27e409a3d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f27d800e9b0 tx=0x7f27d800ecc0 comp rx=0 tx=0).stop 2023-09-30T22:31:37.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.817+0000 7f27eb0f7700 1 -- 172.21.15.16:0/3843341477 shutdown_connections 2023-09-30T22:31:37.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.817+0000 7f27eb0f7700 1 --2- 172.21.15.16:0/3843341477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27e4131b40 0x7f27e409ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:37.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.817+0000 7f27eb0f7700 1 --2- 172.21.15.16:0/3843341477 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f27e406a770 0x7f27e409a3d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:37.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.817+0000 7f27eb0f7700 1 --2- 172.21.15.16:0/3843341477 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f27e406b080 0x7f27e409a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:37.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.817+0000 7f27eb0f7700 1 -- 172.21.15.16:0/3843341477 >> 172.21.15.16:0/3843341477 conn(0x7f27e4066f50 msgr2=0x7f27e40a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:37.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.817+0000 7f27eb0f7700 1 -- 172.21.15.16:0/3843341477 shutdown_connections 2023-09-30T22:31:37.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.817+0000 7f27eb0f7700 1 -- 172.21.15.16:0/3843341477 wait complete. 2023-09-30T22:31:37.821 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:31:37.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.824+0000 7fbbd13ed700 1 -- 172.21.15.16:0/4257793196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fbbcc0a3860 con 0x7fbbcc06a770 2023-09-30T22:31:37.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.824+0000 7fbbb37fe700 1 -- 172.21.15.16:0/4257793196 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbbcc0a3860 con 0x7fbbcc06a770 2023-09-30T22:31:37.825 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:31:37.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.827+0000 7fbbd13ed700 1 -- 172.21.15.16:0/4257793196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbcc06a770 msgr2=0x7fbbcc13e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:37.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.827+0000 7fbbd13ed700 1 --2- 172.21.15.16:0/4257793196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbcc06a770 0x7fbbcc13e010 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc00e810 tx=0x7fbbbc00eb20 comp rx=0 tx=0).stop 2023-09-30T22:31:37.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.827+0000 7fbbd13ed700 1 -- 172.21.15.16:0/4257793196 shutdown_connections 2023-09-30T22:31:37.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.827+0000 7fbbd13ed700 1 --2- 172.21.15.16:0/4257793196 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbbcc13c1e0 0x7fbbcc13a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:37.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.827+0000 7fbbd13ed700 1 --2- 172.21.15.16:0/4257793196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbcc06a770 0x7fbbcc13e010 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:37.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.827+0000 7fbbd13ed700 1 --2- 172.21.15.16:0/4257793196 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbbcc09dc30 0x7fbbcc13a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:37.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.827+0000 7fbbd13ed700 1 -- 172.21.15.16:0/4257793196 >> 172.21.15.16:0/4257793196 conn(0x7fbbcc066f50 msgr2=0x7fbbcc070ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:37.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.827+0000 7fbbd13ed700 1 -- 172.21.15.16:0/4257793196 shutdown_connections 2023-09-30T22:31:37.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:37.827+0000 7fbbd13ed700 1 -- 172.21.15.16:0/4257793196 wait complete. 2023-09-30T22:31:37.832 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:31:37.841 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:31:38.749 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:38.822 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:38.832 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:38.842 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:38.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.900+0000 7f0d88cd5700 1 Processor -- start 2023-09-30T22:31:38.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.901+0000 7f0d88cd5700 1 -- start start 2023-09-30T22:31:38.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.902+0000 7f0d88cd5700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0d8409d6f0 0x7f0d8409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:38.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.902+0000 7f0d88cd5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0d8409e000 0x7f0d841339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:38.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.902+0000 7f0d88cd5700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d8409b120 con 0x7f0d8409e410 2023-09-30T22:31:38.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.902+0000 7f0d88cd5700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d8406b460 con 0x7f0d8409d6f0 2023-09-30T22:31:38.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.902+0000 7f0d88cd5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8409a7e0 con 0x7f0d8409e000 2023-09-30T22:31:38.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.902+0000 7f0d82d9d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0d8409e410 0x7f0d84133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41270/0 (socket says 172.21.15.16:41270) 2023-09-30T22:31:38.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.902+0000 7f0d82d9d700 1 -- 172.21.15.16:0/1651951383 learned_addr learned my addr 172.21.15.16:0/1651951383 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:31:38.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.902+0000 7f0d81d9b700 1 --2- 172.21.15.16:0/1651951383 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0d8409e000 0x7f0d841339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:38.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.902+0000 7f0d8259c700 1 --2- 172.21.15.16:0/1651951383 >> v2:172.21.15.16:3300/0 conn(0x7f0d8409d6f0 0x7f0d8409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:38.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.902+0000 7f0d8159a700 1 -- 172.21.15.16:0/1651951383 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4211910937 0 0) 0x7f0d8409b120 con 0x7f0d8409e410 2023-09-30T22:31:38.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.902+0000 7f0d81d9b700 1 -- 172.21.15.16:0/1651951383 >> v2:172.21.15.16:3300/0 conn(0x7f0d8409d6f0 msgr2=0x7f0d8409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:38.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.902+0000 7f0d81d9b700 1 --2- 172.21.15.16:0/1651951383 >> v2:172.21.15.16:3300/0 conn(0x7f0d8409d6f0 0x7f0d8409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:38.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.902+0000 7f0d81d9b700 1 -- 172.21.15.16:0/1651951383 >> v1:172.21.15.16:6789/0 conn(0x7f0d8409e410 legacy=0x7f0d84133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:38.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.902+0000 7f0d81d9b700 1 -- 172.21.15.16:0/1651951383 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d8409a500 con 0x7f0d8409e000 2023-09-30T22:31:38.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.903+0000 7f0d81d9b700 1 --2- 172.21.15.16:0/1651951383 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0d8409e000 0x7f0d841339f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0d78009870 tx=0x7f0d78009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee7ad10048777267 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:38.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.903+0000 7f0d8259c700 1 --2- 172.21.15.16:0/1651951383 >> v2:172.21.15.16:3300/0 conn(0x7f0d8409d6f0 0x7f0d8409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:31:38.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.903+0000 7f0d8159a700 1 -- 172.21.15.16:0/1651951383 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d78011830 con 0x7f0d8409e000 2023-09-30T22:31:38.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.903+0000 7f0d8159a700 1 -- 172.21.15.16:0/1651951383 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d78011990 con 0x7f0d8409e000 2023-09-30T22:31:38.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.903+0000 7f0d8159a700 1 -- 172.21.15.16:0/1651951383 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d78011d00 con 0x7f0d8409e000 2023-09-30T22:31:38.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.904+0000 7f0d88cd5700 1 -- 172.21.15.16:0/1651951383 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0d8409e000 msgr2=0x7f0d841339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:38.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.904+0000 7f0d88cd5700 1 --2- 172.21.15.16:0/1651951383 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0d8409e000 0x7f0d841339f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0d78009870 tx=0x7f0d78009b80 comp rx=0 tx=0).stop 2023-09-30T22:31:38.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.904+0000 7f0d88cd5700 1 -- 172.21.15.16:0/1651951383 shutdown_connections 2023-09-30T22:31:38.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.904+0000 7f0d88cd5700 1 --2- 172.21.15.16:0/1651951383 >> v2:172.21.15.16:3300/0 conn(0x7f0d8409d6f0 0x7f0d8409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:38.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.904+0000 7f0d88cd5700 1 --2- 172.21.15.16:0/1651951383 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0d8409e000 0x7f0d841339f0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:38.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.904+0000 7f0d88cd5700 1 -- 172.21.15.16:0/1651951383 >> 172.21.15.16:0/1651951383 conn(0x7f0d84066f50 msgr2=0x7f0d8409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:38.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.904+0000 7f0d88cd5700 1 -- 172.21.15.16:0/1651951383 shutdown_connections 2023-09-30T22:31:38.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.904+0000 7f0d88cd5700 1 -- 172.21.15.16:0/1651951383 wait complete. 2023-09-30T22:31:38.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.906+0000 7f0d88cd5700 1 Processor -- start 2023-09-30T22:31:38.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.906+0000 7f0d88cd5700 1 -- start start 2023-09-30T22:31:38.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.907+0000 7f0d88cd5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d8409d6f0 0x7f0d841a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:38.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.907+0000 7f0d88cd5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0d8409e000 0x7f0d841a66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:38.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.907+0000 7f0d88cd5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0d8409e410 0x7f0d841a6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:38.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.907+0000 7f0d88cd5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8409c7a0 con 0x7f0d8409d6f0 2023-09-30T22:31:38.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.907+0000 7f0d88cd5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d841a90d0 con 0x7f0d8409e000 2023-09-30T22:31:38.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.907+0000 7f0d88cd5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0d841a9670 con 0x7f0d8409e410 2023-09-30T22:31:38.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.907+0000 7f0d82d9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0d8409e410 0x7f0d841a6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:38.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.907+0000 7f0d81d9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0d8409e000 0x7f0d841a66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:38.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.907+0000 7f0d82d9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0d8409e410 0x7f0d841a6bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:39702/0 (socket says 172.21.15.16:39702) 2023-09-30T22:31:38.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.907+0000 7f0d81d9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0d8409e000 0x7f0d841a66b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:40368/0 (socket says 172.21.15.16:40368) 2023-09-30T22:31:38.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.907+0000 7f0d82d9d700 1 -- 172.21.15.16:0/1531174057 learned_addr learned my addr 172.21.15.16:0/1531174057 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:38.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.907+0000 7f0d8259c700 1 --2- 172.21.15.16:0/1531174057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d8409d6f0 0x7f0d841a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:38.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.907+0000 7f0d81d9b700 1 -- 172.21.15.16:0/1531174057 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0d8409e410 msgr2=0x7f0d841a6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:38.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.907+0000 7f0d81d9b700 1 --2- 172.21.15.16:0/1531174057 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0d8409e410 0x7f0d841a6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:38.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.907+0000 7f0d81d9b700 1 -- 172.21.15.16:0/1531174057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d8409d6f0 msgr2=0x7f0d841a6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:38.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.907+0000 7f0d81d9b700 1 --2- 172.21.15.16:0/1531174057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d8409d6f0 0x7f0d841a6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:38.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.907+0000 7f0d81d9b700 1 -- 172.21.15.16:0/1531174057 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d780094a0 con 0x7f0d8409e000 2023-09-30T22:31:38.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.907+0000 7f0d82d9d700 1 --2- 172.21.15.16:0/1531174057 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0d8409e410 0x7f0d841a6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:38.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.907+0000 7f0d8259c700 1 --2- 172.21.15.16:0/1531174057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d8409d6f0 0x7f0d841a6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:38.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.908+0000 7f0d81d9b700 1 --2- 172.21.15.16:0/1531174057 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0d8409e000 0x7f0d841a66b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0d78016000 tx=0x7f0d78004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:38.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.908+0000 7f0d72ffd700 1 -- 172.21.15.16:0/1531174057 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d780036a0 con 0x7f0d8409e000 2023-09-30T22:31:38.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.908+0000 7f0d72ffd700 1 -- 172.21.15.16:0/1531174057 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d7801f070 con 0x7f0d8409e000 2023-09-30T22:31:38.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.908+0000 7f0d88cd5700 1 -- 172.21.15.16:0/1531174057 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d841a9390 con 0x7f0d8409e000 2023-09-30T22:31:38.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.908+0000 7f0d72ffd700 1 -- 172.21.15.16:0/1531174057 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d78003a60 con 0x7f0d8409e000 2023-09-30T22:31:38.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.908+0000 7f0d88cd5700 1 -- 172.21.15.16:0/1531174057 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d841a7760 con 0x7f0d8409e000 2023-09-30T22:31:38.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.909+0000 7f0d72ffd700 1 -- 172.21.15.16:0/1531174057 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0d78003bc0 con 0x7f0d8409e000 2023-09-30T22:31:38.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.909+0000 7f0d72ffd700 1 -- 172.21.15.16:0/1531174057 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f0d78021070 con 0x7f0d8409e000 2023-09-30T22:31:38.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.910+0000 7f0d88cd5700 1 -- 172.21.15.16:0/1531174057 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d84074ea0 con 0x7f0d8409e000 2023-09-30T22:31:38.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.915+0000 7f0d72ffd700 1 -- 172.21.15.16:0/1531174057 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0d78015080 con 0x7f0d8409e000 2023-09-30T22:31:38.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.971+0000 7f9e441cd700 1 Processor -- start 2023-09-30T22:31:38.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.971+0000 7f9e441cd700 1 -- start start 2023-09-30T22:31:38.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.972+0000 7f9e441cd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e3c06f5d0 0x7f9e3c12ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:38.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.972+0000 7f9e441cd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e3c135c20 0x7f9e3c138000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:38.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.972+0000 7f9e441cd700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e3c06fa40 con 0x7f9e3c12cf60 2023-09-30T22:31:38.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.972+0000 7f9e441cd700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c12cd90 con 0x7f9e3c135c20 2023-09-30T22:31:38.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.972+0000 7f9e441cd700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c071780 con 0x7f9e3c06f5d0 2023-09-30T22:31:38.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.972+0000 7f9e41f69700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9e3c12cf60 0x7f9e3c12f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41274/0 (socket says 172.21.15.16:41274) 2023-09-30T22:31:38.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.972+0000 7f9e41f69700 1 -- 172.21.15.16:0/4224988389 learned_addr learned my addr 172.21.15.16:0/4224988389 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:31:38.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.972+0000 7f9e4276a700 1 --2- 172.21.15.16:0/4224988389 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e3c135c20 0x7f9e3c138000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:38.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.972+0000 7f9e41768700 1 --2- 172.21.15.16:0/4224988389 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e3c06f5d0 0x7f9e3c12ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:38.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.972+0000 7f9e40f67700 1 -- 172.21.15.16:0/4224988389 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3415194715 0 0) 0x7f9e3c06fa40 con 0x7f9e3c12cf60 2023-09-30T22:31:38.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.972+0000 7f9e40f67700 1 -- 172.21.15.16:0/4224988389 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e280035b0 con 0x7f9e3c12cf60 2023-09-30T22:31:38.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.973+0000 7f9e4276a700 1 -- 172.21.15.16:0/4224988389 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e3c06f5d0 msgr2=0x7f9e3c12ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:38.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.973+0000 7f9e4276a700 1 --2- 172.21.15.16:0/4224988389 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e3c06f5d0 0x7f9e3c12ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:38.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.973+0000 7f9e4276a700 1 -- 172.21.15.16:0/4224988389 >> v1:172.21.15.16:6789/0 conn(0x7f9e3c12cf60 legacy=0x7f9e3c12f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:38.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.973+0000 7f9e4276a700 1 -- 172.21.15.16:0/4224988389 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e3c071a40 con 0x7f9e3c135c20 2023-09-30T22:31:38.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.973+0000 7f9e4276a700 1 --2- 172.21.15.16:0/4224988389 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e3c135c20 0x7f9e3c138000 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9e34009dc0 tx=0x7f9e3400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e0196d59ca6934c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:38.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.973+0000 7f9e40f67700 1 -- 172.21.15.16:0/4224988389 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e3400bae0 con 0x7f9e3c135c20 2023-09-30T22:31:38.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.973+0000 7f9e40f67700 1 -- 172.21.15.16:0/4224988389 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e3400bc40 con 0x7f9e3c135c20 2023-09-30T22:31:38.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.973+0000 7f9e40f67700 1 -- 172.21.15.16:0/4224988389 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e34003a20 con 0x7f9e3c135c20 2023-09-30T22:31:38.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.974+0000 7f9e441cd700 1 -- 172.21.15.16:0/4224988389 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e3c135c20 msgr2=0x7f9e3c138000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:38.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.974+0000 7f9e441cd700 1 --2- 172.21.15.16:0/4224988389 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e3c135c20 0x7f9e3c138000 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9e34009dc0 tx=0x7f9e3400b330 comp rx=0 tx=0).stop 2023-09-30T22:31:38.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.974+0000 7f9e441cd700 1 -- 172.21.15.16:0/4224988389 shutdown_connections 2023-09-30T22:31:38.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.974+0000 7f9e441cd700 1 --2- 172.21.15.16:0/4224988389 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e3c135c20 0x7f9e3c138000 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:38.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.974+0000 7f9e441cd700 1 --2- 172.21.15.16:0/4224988389 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e3c06f5d0 0x7f9e3c12ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:38.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.974+0000 7f9e441cd700 1 -- 172.21.15.16:0/4224988389 >> 172.21.15.16:0/4224988389 conn(0x7f9e3c0a2aa0 msgr2=0x7f9e3c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:38.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.974+0000 7f9e441cd700 1 -- 172.21.15.16:0/4224988389 shutdown_connections 2023-09-30T22:31:38.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.974+0000 7f9e441cd700 1 -- 172.21.15.16:0/4224988389 wait complete. 2023-09-30T22:31:38.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.975+0000 7f9e441cd700 1 Processor -- start 2023-09-30T22:31:38.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.976+0000 7f9e441cd700 1 -- start start 2023-09-30T22:31:38.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.976+0000 7f9e441cd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e3c06f5d0 0x7f9e3c099d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:38.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.976+0000 7f9e441cd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e3c12cf60 0x7f9e3c09a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:38.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.976+0000 7f9e441cd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e3c135c20 0x7f9e3c09a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:38.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.976+0000 7f9e441cd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c09eb60 con 0x7f9e3c12cf60 2023-09-30T22:31:38.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.976+0000 7f9e441cd700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c09c750 con 0x7f9e3c06f5d0 2023-09-30T22:31:38.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.976+0000 7f9e441cd700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c09d0a0 con 0x7f9e3c135c20 2023-09-30T22:31:38.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.976+0000 7f9e41f69700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e3c06f5d0 0x7f9e3c099d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:38.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.977+0000 7f9e41f69700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e3c06f5d0 0x7f9e3c099d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:40390/0 (socket says 172.21.15.16:40390) 2023-09-30T22:31:38.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.977+0000 7f9e41f69700 1 -- 172.21.15.16:0/1249289090 learned_addr learned my addr 172.21.15.16:0/1249289090 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:38.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.977+0000 7f9e41768700 1 --2- 172.21.15.16:0/1249289090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e3c12cf60 0x7f9e3c09a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:38.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.977+0000 7f9e4276a700 1 --2- 172.21.15.16:0/1249289090 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e3c135c20 0x7f9e3c09a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:38.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.977+0000 7f9e41f69700 1 -- 172.21.15.16:0/1249289090 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e3c135c20 msgr2=0x7f9e3c09a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:38.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.977+0000 7f9e41f69700 1 --2- 172.21.15.16:0/1249289090 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e3c135c20 0x7f9e3c09a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:38.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.977+0000 7f9e41f69700 1 -- 172.21.15.16:0/1249289090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e3c12cf60 msgr2=0x7f9e3c09a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:38.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.977+0000 7f9e41f69700 1 --2- 172.21.15.16:0/1249289090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e3c12cf60 0x7f9e3c09a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:38.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.977+0000 7f9e41f69700 1 -- 172.21.15.16:0/1249289090 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e34009a70 con 0x7f9e3c06f5d0 2023-09-30T22:31:38.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.977+0000 7f9e41f69700 1 --2- 172.21.15.16:0/1249289090 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e3c06f5d0 0x7f9e3c099d50 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9e380011e0 tx=0x7f9e3800ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:38.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.977+0000 7f9e327fc700 1 -- 172.21.15.16:0/1249289090 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e38004c80 con 0x7f9e3c06f5d0 2023-09-30T22:31:38.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.978+0000 7f9e327fc700 1 -- 172.21.15.16:0/1249289090 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e38004e00 con 0x7f9e3c06f5d0 2023-09-30T22:31:38.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.978+0000 7f9e327fc700 1 -- 172.21.15.16:0/1249289090 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e38007310 con 0x7f9e3c06f5d0 2023-09-30T22:31:38.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.978+0000 7f9e441cd700 1 -- 172.21.15.16:0/1249289090 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e3c09e680 con 0x7f9e3c06f5d0 2023-09-30T22:31:38.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.978+0000 7f9e441cd700 1 -- 172.21.15.16:0/1249289090 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e3c139ba0 con 0x7f9e3c06f5d0 2023-09-30T22:31:38.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.979+0000 7f9e327fc700 1 -- 172.21.15.16:0/1249289090 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9e38007470 con 0x7f9e3c06f5d0 2023-09-30T22:31:38.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.979+0000 7f9e327fc700 1 -- 172.21.15.16:0/1249289090 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f9e3803f320 con 0x7f9e3c06f5d0 2023-09-30T22:31:38.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.979+0000 7f9e441cd700 1 -- 172.21.15.16:0/1249289090 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e3c074ea0 con 0x7f9e3c06f5d0 2023-09-30T22:31:38.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:38.984+0000 7f9e327fc700 1 -- 172.21.15.16:0/1249289090 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9e380123b0 con 0x7f9e3c06f5d0 2023-09-30T22:31:39.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.014+0000 7fe61ce7b700 1 Processor -- start 2023-09-30T22:31:39.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.014+0000 7fe61ce7b700 1 -- start start 2023-09-30T22:31:39.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.015+0000 7fe61ce7b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe61809d6f0 0x7fe61809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:39.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.015+0000 7fe61ce7b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe61809e410 0x7fe618130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:39.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.015+0000 7fe61ce7b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe61809f130 con 0x7fe61809e000 2023-09-30T22:31:39.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.015+0000 7fe61ce7b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe61809f520 con 0x7fe61809d6f0 2023-09-30T22:31:39.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.015+0000 7fe61ce7b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe61809e930 con 0x7fe61809e410 2023-09-30T22:31:39.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.015+0000 7fe61659c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe61809d6f0 0x7fe61809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:39.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.015+0000 7fe61659c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe61809d6f0 0x7fe61809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48042/0 (socket says 172.21.15.16:48042) 2023-09-30T22:31:39.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.015+0000 7fe61659c700 1 -- 172.21.15.16:0/1976224247 learned_addr learned my addr 172.21.15.16:0/1976224247 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:39.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.016+0000 7fe616d9d700 1 --2- 172.21.15.16:0/1976224247 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe61809e410 0x7fe618130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:39.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.016+0000 7fe61659c700 1 -- 172.21.15.16:0/1976224247 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe61809e410 msgr2=0x7fe618130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:39.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.016+0000 7fe61659c700 1 --2- 172.21.15.16:0/1976224247 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe61809e410 0x7fe618130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:39.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.016+0000 7fe61659c700 1 -- 172.21.15.16:0/1976224247 >> v1:172.21.15.16:6789/0 conn(0x7fe61809e000 legacy=0x7fe618130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:39.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.016+0000 7fe61659c700 1 -- 172.21.15.16:0/1976224247 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe61809b2d0 con 0x7fe61809d6f0 2023-09-30T22:31:39.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.016+0000 7fe61659c700 1 --2- 172.21.15.16:0/1976224247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe61809d6f0 0x7fe61809dac0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fe6080098f0 tx=0x7fe608009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38f15061f1e42009 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:39.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.016+0000 7fe61559a700 1 -- 172.21.15.16:0/1976224247 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe608004230 con 0x7fe61809d6f0 2023-09-30T22:31:39.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.016+0000 7fe61559a700 1 -- 172.21.15.16:0/1976224247 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe608004390 con 0x7fe61809d6f0 2023-09-30T22:31:39.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.016+0000 7fe61559a700 1 -- 172.21.15.16:0/1976224247 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6080036a0 con 0x7fe61809d6f0 2023-09-30T22:31:39.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.017+0000 7fe61ce7b700 1 -- 172.21.15.16:0/1976224247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe61809d6f0 msgr2=0x7fe61809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:39.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.017+0000 7fe61ce7b700 1 --2- 172.21.15.16:0/1976224247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe61809d6f0 0x7fe61809dac0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fe6080098f0 tx=0x7fe608009c00 comp rx=0 tx=0).stop 2023-09-30T22:31:39.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.017+0000 7fe61ce7b700 1 -- 172.21.15.16:0/1976224247 shutdown_connections 2023-09-30T22:31:39.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.017+0000 7fe61ce7b700 1 --2- 172.21.15.16:0/1976224247 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe61809e410 0x7fe618130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:39.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.017+0000 7fe61ce7b700 1 --2- 172.21.15.16:0/1976224247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe61809d6f0 0x7fe61809dac0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:39.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.017+0000 7fe61ce7b700 1 -- 172.21.15.16:0/1976224247 >> 172.21.15.16:0/1976224247 conn(0x7fe618066f50 msgr2=0x7fe61809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:39.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.017+0000 7fe61ce7b700 1 -- 172.21.15.16:0/1976224247 shutdown_connections 2023-09-30T22:31:39.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.017+0000 7fe61ce7b700 1 -- 172.21.15.16:0/1976224247 wait complete. 2023-09-30T22:31:39.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.018+0000 7fe61ce7b700 1 Processor -- start 2023-09-30T22:31:39.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.018+0000 7fe61ce7b700 1 -- start start 2023-09-30T22:31:39.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.019+0000 7fe61ce7b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe61809d6f0 0x7fe61806e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:39.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.019+0000 7fe61ce7b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe61809e000 0x7fe61806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:39.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.019+0000 7fe61ce7b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe61809e410 0x7fe61806f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:39.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.019+0000 7fe61ce7b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe618125860 con 0x7fe61809e410 2023-09-30T22:31:39.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.019+0000 7fe61ce7b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe61806fcd0 con 0x7fe61809e000 2023-09-30T22:31:39.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.019+0000 7fe61ce7b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe618124750 con 0x7fe61809d6f0 2023-09-30T22:31:39.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.019+0000 7fe615d9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe61809e000 0x7fe61806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:39.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.019+0000 7fe61659c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe61809d6f0 0x7fe61806e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:39.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.019+0000 7fe615d9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe61809e000 0x7fe61806edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:40392/0 (socket says 172.21.15.16:40392) 2023-09-30T22:31:39.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.019+0000 7fe615d9b700 1 -- 172.21.15.16:0/704414817 learned_addr learned my addr 172.21.15.16:0/704414817 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:39.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.019+0000 7fe616d9d700 1 --2- 172.21.15.16:0/704414817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe61809e410 0x7fe61806f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:39.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.020+0000 7fe61659c700 1 -- 172.21.15.16:0/704414817 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe61809e000 msgr2=0x7fe61806edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:39.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.020+0000 7fe61659c700 1 --2- 172.21.15.16:0/704414817 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe61809e000 0x7fe61806edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:39.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.020+0000 7fe61659c700 1 -- 172.21.15.16:0/704414817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe61809e410 msgr2=0x7fe61806f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:39.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.020+0000 7fe61659c700 1 --2- 172.21.15.16:0/704414817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe61809e410 0x7fe61806f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:39.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.020+0000 7fe61659c700 1 -- 172.21.15.16:0/704414817 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe608009550 con 0x7fe61809d6f0 2023-09-30T22:31:39.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.020+0000 7fe61659c700 1 --2- 172.21.15.16:0/704414817 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe61809d6f0 0x7fe61806e870 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe608015000 tx=0x7fe608004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:39.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.020+0000 7fe606ffd700 1 -- 172.21.15.16:0/704414817 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe608003ba0 con 0x7fe61809d6f0 2023-09-30T22:31:39.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.020+0000 7fe606ffd700 1 -- 172.21.15.16:0/704414817 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe60801f070 con 0x7fe61809d6f0 2023-09-30T22:31:39.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.020+0000 7fe606ffd700 1 -- 172.21.15.16:0/704414817 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe608011630 con 0x7fe61809d6f0 2023-09-30T22:31:39.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.020+0000 7fe61ce7b700 1 -- 172.21.15.16:0/704414817 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6180a1de0 con 0x7fe61809d6f0 2023-09-30T22:31:39.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.020+0000 7fe61ce7b700 1 -- 172.21.15.16:0/704414817 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe618126590 con 0x7fe61809d6f0 2023-09-30T22:31:39.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.021+0000 7fe606ffd700 1 -- 172.21.15.16:0/704414817 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe608021030 con 0x7fe61809d6f0 2023-09-30T22:31:39.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.021+0000 7fe61ce7b700 1 -- 172.21.15.16:0/704414817 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe61809c760 con 0x7fe61809d6f0 2023-09-30T22:31:39.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.022+0000 7fe606ffd700 1 -- 172.21.15.16:0/704414817 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fe608040c30 con 0x7fe61809d6f0 2023-09-30T22:31:39.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.027+0000 7f7d2ba79700 1 Processor -- start 2023-09-30T22:31:39.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.027+0000 7f7d2ba79700 1 -- start start 2023-09-30T22:31:39.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.028+0000 7f7d2ba79700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7d2406a770 0x7f7d2406ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:39.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.028+0000 7f7d2ba79700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7d2406b080 0x7f7d24131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:39.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.028+0000 7f7d2ba79700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d2409ece0 con 0x7f7d24131b40 2023-09-30T22:31:39.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.028+0000 7f7d2ba79700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2409e760 con 0x7f7d2406b080 2023-09-30T22:31:39.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.028+0000 7f7d2ba79700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2409e040 con 0x7f7d2406a770 2023-09-30T22:31:39.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.028+0000 7f7d29014700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7d2406b080 0x7f7d24131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:39.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.028+0000 7f7d29014700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7d2406b080 0x7f7d24131600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:40398/0 (socket says 172.21.15.16:40398) 2023-09-30T22:31:39.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.028+0000 7f7d29014700 1 -- 172.21.15.16:0/1031789950 learned_addr learned my addr 172.21.15.16:0/1031789950 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:39.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.028+0000 7f7d29815700 1 --2- 172.21.15.16:0/1031789950 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7d2406a770 0x7f7d2406ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:39.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.028+0000 7f7d29014700 1 -- 172.21.15.16:0/1031789950 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7d2406a770 msgr2=0x7f7d2406ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:39.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.028+0000 7f7d29014700 1 --2- 172.21.15.16:0/1031789950 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7d2406a770 0x7f7d2406ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:39.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.028+0000 7f7d29014700 1 -- 172.21.15.16:0/1031789950 >> v1:172.21.15.16:6789/0 conn(0x7f7d24131b40 legacy=0x7f7d24138850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:39.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.028+0000 7f7d29014700 1 -- 172.21.15.16:0/1031789950 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d2409f630 con 0x7f7d2406b080 2023-09-30T22:31:39.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.028+0000 7f7d29014700 1 --2- 172.21.15.16:0/1031789950 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7d2406b080 0x7f7d24131600 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f7d14009920 tx=0x7f7d14009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c64d0eaec0d55c4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:39.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.029+0000 7f7d28813700 1 -- 172.21.15.16:0/1031789950 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d14011830 con 0x7f7d2406b080 2023-09-30T22:31:39.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.029+0000 7f7d28813700 1 -- 172.21.15.16:0/1031789950 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d14011990 con 0x7f7d2406b080 2023-09-30T22:31:39.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.029+0000 7f7d28813700 1 -- 172.21.15.16:0/1031789950 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d14011d00 con 0x7f7d2406b080 2023-09-30T22:31:39.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.029+0000 7f7d2ba79700 1 -- 172.21.15.16:0/1031789950 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7d2406b080 msgr2=0x7f7d24131600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:39.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.029+0000 7f7d2ba79700 1 --2- 172.21.15.16:0/1031789950 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7d2406b080 0x7f7d24131600 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f7d14009920 tx=0x7f7d14009c30 comp rx=0 tx=0).stop 2023-09-30T22:31:39.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.029+0000 7f7d2ba79700 1 -- 172.21.15.16:0/1031789950 shutdown_connections 2023-09-30T22:31:39.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.029+0000 7f7d2ba79700 1 --2- 172.21.15.16:0/1031789950 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7d2406a770 0x7f7d2406ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:39.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.029+0000 7f7d2ba79700 1 --2- 172.21.15.16:0/1031789950 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7d2406b080 0x7f7d24131600 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:39.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.029+0000 7f7d2ba79700 1 -- 172.21.15.16:0/1031789950 >> 172.21.15.16:0/1031789950 conn(0x7f7d24066f50 msgr2=0x7f7d2409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:39.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.030+0000 7f7d2ba79700 1 -- 172.21.15.16:0/1031789950 shutdown_connections 2023-09-30T22:31:39.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.030+0000 7f7d2ba79700 1 -- 172.21.15.16:0/1031789950 wait complete. 2023-09-30T22:31:39.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.030+0000 7fe606ffd700 1 -- 172.21.15.16:0/704414817 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe60800b7a0 con 0x7fe61809d6f0 2023-09-30T22:31:39.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.031+0000 7f7d2ba79700 1 Processor -- start 2023-09-30T22:31:39.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.031+0000 7f7d2ba79700 1 -- start start 2023-09-30T22:31:39.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.032+0000 7f7d2ba79700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7d2406a770 0x7f7d24130560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:39.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.032+0000 7f7d2ba79700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7d2406b080 0x7f7d24130aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:39.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.032+0000 7f7d2ba79700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d24131b40 0x7f7d241c8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:39.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.032+0000 7f7d2ba79700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d24120dc0 con 0x7f7d24131b40 2023-09-30T22:31:39.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.032+0000 7f7d2ba79700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d24071e60 con 0x7f7d2406a770 2023-09-30T22:31:39.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.032+0000 7f7d2ba79700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7d24121f50 con 0x7f7d2406b080 2023-09-30T22:31:39.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.032+0000 7f7d29815700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7d2406a770 0x7f7d24130560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:39.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.032+0000 7f7d2a016700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d24131b40 0x7f7d241c8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:39.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.032+0000 7f7d29815700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7d2406a770 0x7f7d24130560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:40402/0 (socket says 172.21.15.16:40402) 2023-09-30T22:31:39.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.032+0000 7f7d29815700 1 -- 172.21.15.16:0/1732803106 learned_addr learned my addr 172.21.15.16:0/1732803106 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:39.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.032+0000 7f7d2a016700 1 -- 172.21.15.16:0/1732803106 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7d2406b080 msgr2=0x7f7d24130aa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:31:39.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.032+0000 7f7d2a016700 1 --2- 172.21.15.16:0/1732803106 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7d2406b080 0x7f7d24130aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:39.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.032+0000 7f7d2a016700 1 -- 172.21.15.16:0/1732803106 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7d2406a770 msgr2=0x7f7d24130560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:39.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.032+0000 7f7d2a016700 1 --2- 172.21.15.16:0/1732803106 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7d2406a770 0x7f7d24130560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:39.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.032+0000 7f7d2a016700 1 -- 172.21.15.16:0/1732803106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d14009550 con 0x7f7d24131b40 2023-09-30T22:31:39.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.033+0000 7f7d2a016700 1 --2- 172.21.15.16:0/1732803106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d24131b40 0x7f7d241c8160 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c00cb90 tx=0x7f7d1c00cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:39.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.033+0000 7f7d1a7fc700 1 -- 172.21.15.16:0/1732803106 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c003c00 con 0x7f7d24131b40 2023-09-30T22:31:39.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.033+0000 7f7d1a7fc700 1 -- 172.21.15.16:0/1732803106 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d1c003d60 con 0x7f7d24131b40 2023-09-30T22:31:39.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.033+0000 7f7d1a7fc700 1 -- 172.21.15.16:0/1732803106 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c00aaa0 con 0x7f7d24131b40 2023-09-30T22:31:39.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.033+0000 7f7d2ba79700 1 -- 172.21.15.16:0/1732803106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d24131450 con 0x7f7d24131b40 2023-09-30T22:31:39.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.033+0000 7f7d2ba79700 1 -- 172.21.15.16:0/1732803106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d24130fe0 con 0x7f7d24131b40 2023-09-30T22:31:39.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.034+0000 7f7d2ba79700 1 -- 172.21.15.16:0/1732803106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d24122190 con 0x7f7d24131b40 2023-09-30T22:31:39.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.035+0000 7f7d1a7fc700 1 -- 172.21.15.16:0/1732803106 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7d1c00ac00 con 0x7f7d24131b40 2023-09-30T22:31:39.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.035+0000 7f7d1a7fc700 1 -- 172.21.15.16:0/1732803106 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f7d1c03f750 con 0x7f7d24131b40 2023-09-30T22:31:39.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.040+0000 7f7d1a7fc700 1 -- 172.21.15.16:0/1732803106 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7d1c0196c0 con 0x7f7d24131b40 2023-09-30T22:31:39.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.128+0000 7f0d88cd5700 1 -- 172.21.15.16:0/1531174057 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f0d8406c360 con 0x7f0d8409e000 2023-09-30T22:31:39.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.128+0000 7f0d72ffd700 1 -- 172.21.15.16:0/1531174057 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0d78035090 con 0x7f0d8409e000 2023-09-30T22:31:39.129 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:31:39.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.132+0000 7f0d88cd5700 1 -- 172.21.15.16:0/1531174057 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0d8409e000 msgr2=0x7f0d841a66b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:39.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.132+0000 7f0d88cd5700 1 --2- 172.21.15.16:0/1531174057 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0d8409e000 0x7f0d841a66b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0d78016000 tx=0x7f0d78004750 comp rx=0 tx=0).stop 2023-09-30T22:31:39.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.132+0000 7f0d88cd5700 1 -- 172.21.15.16:0/1531174057 shutdown_connections 2023-09-30T22:31:39.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.132+0000 7f0d88cd5700 1 --2- 172.21.15.16:0/1531174057 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0d8409e410 0x7f0d841a6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:39.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.132+0000 7f0d88cd5700 1 --2- 172.21.15.16:0/1531174057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d8409d6f0 0x7f0d841a6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:39.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.132+0000 7f0d88cd5700 1 --2- 172.21.15.16:0/1531174057 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0d8409e000 0x7f0d841a66b0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:39.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.132+0000 7f0d88cd5700 1 -- 172.21.15.16:0/1531174057 >> 172.21.15.16:0/1531174057 conn(0x7f0d84066f50 msgr2=0x7f0d8406f290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:39.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.132+0000 7f0d88cd5700 1 -- 172.21.15.16:0/1531174057 shutdown_connections 2023-09-30T22:31:39.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.132+0000 7f0d88cd5700 1 -- 172.21.15.16:0/1531174057 wait complete. 2023-09-30T22:31:39.151 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:31:39.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.206+0000 7f9e441cd700 1 -- 172.21.15.16:0/1249289090 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9e3c09c530 con 0x7f9e3c06f5d0 2023-09-30T22:31:39.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.207+0000 7f9e327fc700 1 -- 172.21.15.16:0/1249289090 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9e3801b340 con 0x7f9e3c06f5d0 2023-09-30T22:31:39.207 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:31:39.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.209+0000 7f9e441cd700 1 -- 172.21.15.16:0/1249289090 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e3c06f5d0 msgr2=0x7f9e3c099d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:39.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.209+0000 7f9e441cd700 1 --2- 172.21.15.16:0/1249289090 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e3c06f5d0 0x7f9e3c099d50 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9e380011e0 tx=0x7f9e3800ca30 comp rx=0 tx=0).stop 2023-09-30T22:31:39.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.209+0000 7f9e441cd700 1 -- 172.21.15.16:0/1249289090 shutdown_connections 2023-09-30T22:31:39.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.209+0000 7f9e441cd700 1 --2- 172.21.15.16:0/1249289090 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e3c06f5d0 0x7f9e3c099d50 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:39.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.209+0000 7f9e441cd700 1 --2- 172.21.15.16:0/1249289090 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e3c135c20 0x7f9e3c09a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:39.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.209+0000 7f9e441cd700 1 --2- 172.21.15.16:0/1249289090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e3c12cf60 0x7f9e3c09a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:39.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.209+0000 7f9e441cd700 1 -- 172.21.15.16:0/1249289090 >> 172.21.15.16:0/1249289090 conn(0x7f9e3c0a2aa0 msgr2=0x7f9e3c12d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:39.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.209+0000 7f9e441cd700 1 -- 172.21.15.16:0/1249289090 shutdown_connections 2023-09-30T22:31:39.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.209+0000 7f9e441cd700 1 -- 172.21.15.16:0/1249289090 wait complete. 2023-09-30T22:31:39.223 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:31:39.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.260+0000 7fe61ce7b700 1 -- 172.21.15.16:0/704414817 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe61809c760 con 0x7fe61809d6f0 2023-09-30T22:31:39.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.261+0000 7fe606ffd700 1 -- 172.21.15.16:0/704414817 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe60800b3e0 con 0x7fe61809d6f0 2023-09-30T22:31:39.263 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:31:39.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.263+0000 7fe61ce7b700 1 -- 172.21.15.16:0/704414817 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe61809d6f0 msgr2=0x7fe61806e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:39.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.263+0000 7fe61ce7b700 1 --2- 172.21.15.16:0/704414817 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe61809d6f0 0x7fe61806e870 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe608015000 tx=0x7fe608004750 comp rx=0 tx=0).stop 2023-09-30T22:31:39.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.263+0000 7fe61ce7b700 1 -- 172.21.15.16:0/704414817 shutdown_connections 2023-09-30T22:31:39.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.263+0000 7fe61ce7b700 1 --2- 172.21.15.16:0/704414817 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe61809e410 0x7fe61806f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:39.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.263+0000 7fe61ce7b700 1 --2- 172.21.15.16:0/704414817 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe61809d6f0 0x7fe61806e870 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:39.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.263+0000 7fe61ce7b700 1 --2- 172.21.15.16:0/704414817 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe61809e000 0x7fe61806edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:39.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.263+0000 7fe61ce7b700 1 -- 172.21.15.16:0/704414817 >> 172.21.15.16:0/704414817 conn(0x7fe618066f50 msgr2=0x7fe61806a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:39.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.263+0000 7fe61ce7b700 1 -- 172.21.15.16:0/704414817 shutdown_connections 2023-09-30T22:31:39.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.263+0000 7fe61ce7b700 1 -- 172.21.15.16:0/704414817 wait complete. 2023-09-30T22:31:39.277 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:31:39.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.279+0000 7f7d2ba79700 1 -- 172.21.15.16:0/1732803106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f7d24074ea0 con 0x7f7d24131b40 2023-09-30T22:31:39.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.280+0000 7f7d1a7fc700 1 -- 172.21.15.16:0/1732803106 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7d1c011380 con 0x7f7d24131b40 2023-09-30T22:31:39.280 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:31:39.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.282+0000 7f7d2ba79700 1 -- 172.21.15.16:0/1732803106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d24131b40 msgr2=0x7f7d241c8160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:39.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.282+0000 7f7d2ba79700 1 --2- 172.21.15.16:0/1732803106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d24131b40 0x7f7d241c8160 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c00cb90 tx=0x7f7d1c00cea0 comp rx=0 tx=0).stop 2023-09-30T22:31:39.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.282+0000 7f7d2ba79700 1 -- 172.21.15.16:0/1732803106 shutdown_connections 2023-09-30T22:31:39.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.282+0000 7f7d2ba79700 1 --2- 172.21.15.16:0/1732803106 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7d2406a770 0x7f7d24130560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:39.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.282+0000 7f7d2ba79700 1 --2- 172.21.15.16:0/1732803106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d24131b40 0x7f7d241c8160 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:39.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.282+0000 7f7d2ba79700 1 --2- 172.21.15.16:0/1732803106 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7d2406b080 0x7f7d24130aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:39.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.282+0000 7f7d2ba79700 1 -- 172.21.15.16:0/1732803106 >> 172.21.15.16:0/1732803106 conn(0x7f7d24066f50 msgr2=0x7f7d240a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:39.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.282+0000 7f7d2ba79700 1 -- 172.21.15.16:0/1732803106 shutdown_connections 2023-09-30T22:31:39.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:39.282+0000 7f7d2ba79700 1 -- 172.21.15.16:0/1732803106 wait complete. 2023-09-30T22:31:39.296 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:31:40.152 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:40.224 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:40.278 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:40.297 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:40.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.302+0000 7f5acbe9e700 1 Processor -- start 2023-09-30T22:31:40.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.303+0000 7f5acbe9e700 1 -- start start 2023-09-30T22:31:40.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.303+0000 7f5acbe9e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5ac41338a0 0x7f5ac4133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:40.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.303+0000 7f5acbe9e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5ac409de50 0x7f5ac409e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:40.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.303+0000 7f5acbe9e700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ac406ef30 con 0x7f5ac409e760 2023-09-30T22:31:40.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.303+0000 7f5acbe9e700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ac406fef0 con 0x7f5ac41338a0 2023-09-30T22:31:40.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.303+0000 7f5acbe9e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac41334b0 con 0x7f5ac409de50 2023-09-30T22:31:40.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.303+0000 7f5ac9c3a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5ac41338a0 0x7f5ac4133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:40.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.303+0000 7f5ac9c3a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5ac41338a0 0x7f5ac4133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48064/0 (socket says 172.21.15.16:48064) 2023-09-30T22:31:40.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.303+0000 7f5ac9c3a700 1 -- 172.21.15.16:0/3472390516 learned_addr learned my addr 172.21.15.16:0/3472390516 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:40.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.303+0000 7f5ac9439700 1 --2- 172.21.15.16:0/3472390516 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5ac409de50 0x7f5ac409e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:40.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.304+0000 7f5ac9439700 1 -- 172.21.15.16:0/3472390516 >> v2:172.21.15.16:3300/0 conn(0x7f5ac41338a0 msgr2=0x7f5ac4133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:40.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.304+0000 7f5ac9439700 1 --2- 172.21.15.16:0/3472390516 >> v2:172.21.15.16:3300/0 conn(0x7f5ac41338a0 0x7f5ac4133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:40.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.304+0000 7f5ac9439700 1 -- 172.21.15.16:0/3472390516 >> v1:172.21.15.16:6789/0 conn(0x7f5ac409e760 legacy=0x7f5ac413a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:40.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.304+0000 7f5ac9439700 1 -- 172.21.15.16:0/3472390516 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ac406fc30 con 0x7f5ac409de50 2023-09-30T22:31:40.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.304+0000 7f5ac9c3a700 1 --2- 172.21.15.16:0/3472390516 >> v2:172.21.15.16:3300/0 conn(0x7f5ac41338a0 0x7f5ac4133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:40.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.304+0000 7f5ac9439700 1 --2- 172.21.15.16:0/3472390516 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5ac409de50 0x7f5ac409e220 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4009840 tx=0x7f5ab4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22cc9f7744c0a464 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:40.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.304+0000 7f5ac8c38700 1 -- 172.21.15.16:0/3472390516 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab4004230 con 0x7f5ac409de50 2023-09-30T22:31:40.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.304+0000 7f5ac8c38700 1 -- 172.21.15.16:0/3472390516 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ab4004390 con 0x7f5ac409de50 2023-09-30T22:31:40.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.304+0000 7f5ac8c38700 1 -- 172.21.15.16:0/3472390516 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab40036a0 con 0x7f5ac409de50 2023-09-30T22:31:40.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.305+0000 7f5acbe9e700 1 -- 172.21.15.16:0/3472390516 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5ac409de50 msgr2=0x7f5ac409e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:40.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.305+0000 7f5acbe9e700 1 --2- 172.21.15.16:0/3472390516 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5ac409de50 0x7f5ac409e220 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4009840 tx=0x7f5ab4009b50 comp rx=0 tx=0).stop 2023-09-30T22:31:40.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.305+0000 7f5acbe9e700 1 -- 172.21.15.16:0/3472390516 shutdown_connections 2023-09-30T22:31:40.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.305+0000 7f5acbe9e700 1 --2- 172.21.15.16:0/3472390516 >> v2:172.21.15.16:3300/0 conn(0x7f5ac41338a0 0x7f5ac4133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:40.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.305+0000 7f5acbe9e700 1 --2- 172.21.15.16:0/3472390516 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5ac409de50 0x7f5ac409e220 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:40.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.305+0000 7f5acbe9e700 1 -- 172.21.15.16:0/3472390516 >> 172.21.15.16:0/3472390516 conn(0x7f5ac40a2aa0 msgr2=0x7f5ac40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:40.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.305+0000 7f5acbe9e700 1 -- 172.21.15.16:0/3472390516 shutdown_connections 2023-09-30T22:31:40.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.305+0000 7f5acbe9e700 1 -- 172.21.15.16:0/3472390516 wait complete. 2023-09-30T22:31:40.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.307+0000 7f5acbe9e700 1 Processor -- start 2023-09-30T22:31:40.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.307+0000 7f5acbe9e700 1 -- start start 2023-09-30T22:31:40.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.307+0000 7f5acbe9e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ac409de50 0x7f5ac411ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:40.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.307+0000 7f5acbe9e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5ac409e760 0x7f5ac411ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:40.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.307+0000 7f5acbe9e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5ac41338a0 0x7f5ac411f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:40.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.308+0000 7f5acbe9e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac40a1000 con 0x7f5ac409de50 2023-09-30T22:31:40.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.308+0000 7f5acbe9e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac4120750 con 0x7f5ac409e760 2023-09-30T22:31:40.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.308+0000 7f5acbe9e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac406f640 con 0x7f5ac41338a0 2023-09-30T22:31:40.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.308+0000 7f5ac9c3a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ac409de50 0x7f5ac411ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:40.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.308+0000 7f5ac9c3a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ac409de50 0x7f5ac411ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48070/0 (socket says 172.21.15.16:48070) 2023-09-30T22:31:40.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.308+0000 7f5ac9c3a700 1 -- 172.21.15.16:0/253289334 learned_addr learned my addr 172.21.15.16:0/253289334 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:40.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.308+0000 7f5aca43b700 1 --2- 172.21.15.16:0/253289334 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5ac41338a0 0x7f5ac411f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:40.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.308+0000 7f5ac9439700 1 --2- 172.21.15.16:0/253289334 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5ac409e760 0x7f5ac411ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:40.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.308+0000 7f5ac9c3a700 1 -- 172.21.15.16:0/253289334 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5ac41338a0 msgr2=0x7f5ac411f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:40.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.308+0000 7f5ac9c3a700 1 --2- 172.21.15.16:0/253289334 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5ac41338a0 0x7f5ac411f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:40.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.308+0000 7f5ac9c3a700 1 -- 172.21.15.16:0/253289334 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5ac409e760 msgr2=0x7f5ac411ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:40.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.308+0000 7f5ac9c3a700 1 --2- 172.21.15.16:0/253289334 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5ac409e760 0x7f5ac411ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:40.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.308+0000 7f5ac9c3a700 1 -- 172.21.15.16:0/253289334 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ac0009550 con 0x7f5ac409de50 2023-09-30T22:31:40.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.309+0000 7f5ac9c3a700 1 --2- 172.21.15.16:0/253289334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ac409de50 0x7f5ac411ea30 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f5ac000ebe0 tx=0x7f5ac000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:40.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.309+0000 7f5aba7fc700 1 -- 172.21.15.16:0/253289334 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac000c850 con 0x7f5ac409de50 2023-09-30T22:31:40.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.309+0000 7f5aba7fc700 1 -- 172.21.15.16:0/253289334 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ac000c9b0 con 0x7f5ac409de50 2023-09-30T22:31:40.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.309+0000 7f5acbe9e700 1 -- 172.21.15.16:0/253289334 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ab40094a0 con 0x7f5ac409de50 2023-09-30T22:31:40.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.309+0000 7f5acbe9e700 1 -- 172.21.15.16:0/253289334 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ac4122ce0 con 0x7f5ac409de50 2023-09-30T22:31:40.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.309+0000 7f5aba7fc700 1 -- 172.21.15.16:0/253289334 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac000cd20 con 0x7f5ac409de50 2023-09-30T22:31:40.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.310+0000 7f5aba7fc700 1 -- 172.21.15.16:0/253289334 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5ac0013470 con 0x7f5ac409de50 2023-09-30T22:31:40.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.310+0000 7f5acbe9e700 1 -- 172.21.15.16:0/253289334 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5aac0052e0 con 0x7f5ac409de50 2023-09-30T22:31:40.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.310+0000 7f5aba7fc700 1 -- 172.21.15.16:0/253289334 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f5ac003fd70 con 0x7f5ac409de50 2023-09-30T22:31:40.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.315+0000 7f5aba7fc700 1 -- 172.21.15.16:0/253289334 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5ac001b3b0 con 0x7f5ac409de50 2023-09-30T22:31:40.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.376+0000 7f79855a0700 1 Processor -- start 2023-09-30T22:31:40.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.376+0000 7f79855a0700 1 -- start start 2023-09-30T22:31:40.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.377+0000 7f79855a0700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f798009d800 0x7f798009dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:40.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.377+0000 7f79855a0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f798009e110 0x7f7980133860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:40.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.377+0000 7f79855a0700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7980099e10 con 0x7f7980133da0 2023-09-30T22:31:40.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.377+0000 7f79855a0700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f798009f560 con 0x7f798009d800 2023-09-30T22:31:40.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.377+0000 7f79855a0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f798006b4d0 con 0x7f798009e110 2023-09-30T22:31:40.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.377+0000 7f797f7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f798009e110 0x7f7980133860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:40.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.377+0000 7f797f7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f798009e110 0x7f7980133860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:40436/0 (socket says 172.21.15.16:40436) 2023-09-30T22:31:40.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.377+0000 7f797ffff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f798009d800 0x7f798009dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:40.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.377+0000 7f797f7fe700 1 -- 172.21.15.16:0/4131713209 learned_addr learned my addr 172.21.15.16:0/4131713209 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:40.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.377+0000 7f797f7fe700 1 -- 172.21.15.16:0/4131713209 >> v2:172.21.15.16:3300/0 conn(0x7f798009d800 msgr2=0x7f798009dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:40.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.377+0000 7f797f7fe700 1 --2- 172.21.15.16:0/4131713209 >> v2:172.21.15.16:3300/0 conn(0x7f798009d800 0x7f798009dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:40.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.377+0000 7f797f7fe700 1 -- 172.21.15.16:0/4131713209 >> v1:172.21.15.16:6789/0 conn(0x7f7980133da0 legacy=0x7f798013aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:40.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.378+0000 7f797f7fe700 1 -- 172.21.15.16:0/4131713209 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f798009a060 con 0x7f798009e110 2023-09-30T22:31:40.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.378+0000 7f797ffff700 1 --2- 172.21.15.16:0/4131713209 >> v2:172.21.15.16:3300/0 conn(0x7f798009d800 0x7f798009dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:40.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.378+0000 7f797f7fe700 1 --2- 172.21.15.16:0/4131713209 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f798009e110 0x7f7980133860 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7974006fd0 tx=0x7f7974009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9acddcbc1778ae6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:40.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.378+0000 7f797effd700 1 -- 172.21.15.16:0/4131713209 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7974005e30 con 0x7f798009e110 2023-09-30T22:31:40.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.378+0000 7f797effd700 1 -- 172.21.15.16:0/4131713209 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7974006030 con 0x7f798009e110 2023-09-30T22:31:40.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.378+0000 7f797effd700 1 -- 172.21.15.16:0/4131713209 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79740063c0 con 0x7f798009e110 2023-09-30T22:31:40.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.379+0000 7f79855a0700 1 -- 172.21.15.16:0/4131713209 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f798009e110 msgr2=0x7f7980133860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:40.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.379+0000 7f79855a0700 1 --2- 172.21.15.16:0/4131713209 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f798009e110 0x7f7980133860 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7974006fd0 tx=0x7f7974009bd0 comp rx=0 tx=0).stop 2023-09-30T22:31:40.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.379+0000 7f79855a0700 1 -- 172.21.15.16:0/4131713209 shutdown_connections 2023-09-30T22:31:40.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.379+0000 7f79855a0700 1 --2- 172.21.15.16:0/4131713209 >> v2:172.21.15.16:3300/0 conn(0x7f798009d800 0x7f798009dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:40.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.379+0000 7f79855a0700 1 --2- 172.21.15.16:0/4131713209 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f798009e110 0x7f7980133860 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:40.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.379+0000 7f79855a0700 1 -- 172.21.15.16:0/4131713209 >> 172.21.15.16:0/4131713209 conn(0x7f79800a2e20 msgr2=0x7f79800a3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:40.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.379+0000 7f79855a0700 1 -- 172.21.15.16:0/4131713209 shutdown_connections 2023-09-30T22:31:40.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.379+0000 7f79855a0700 1 -- 172.21.15.16:0/4131713209 wait complete. 2023-09-30T22:31:40.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.381+0000 7f79855a0700 1 Processor -- start 2023-09-30T22:31:40.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.381+0000 7f79855a0700 1 -- start start 2023-09-30T22:31:40.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.382+0000 7f79855a0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f798009d800 0x7f798011e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:40.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.382+0000 7f79855a0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f798009e110 0x7f798011eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:40.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.382+0000 7f79855a0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7980133da0 0x7f798011f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:40.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.382+0000 7f79855a0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7980121ea0 con 0x7f798009e110 2023-09-30T22:31:40.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.382+0000 7f79855a0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7980133510 con 0x7f798009d800 2023-09-30T22:31:40.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.382+0000 7f79855a0700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f798009c510 con 0x7f7980133da0 2023-09-30T22:31:40.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.382+0000 7f797ffff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f798009d800 0x7f798011e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:40.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.382+0000 7f797ffff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f798009d800 0x7f798011e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:40450/0 (socket says 172.21.15.16:40450) 2023-09-30T22:31:40.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.382+0000 7f797ffff700 1 -- 172.21.15.16:0/2560361250 learned_addr learned my addr 172.21.15.16:0/2560361250 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:40.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.382+0000 7f797f7fe700 1 --2- 172.21.15.16:0/2560361250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f798009e110 0x7f798011eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:40.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.382+0000 7f7984d9f700 1 --2- 172.21.15.16:0/2560361250 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7980133da0 0x7f798011f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:40.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.382+0000 7f797ffff700 1 -- 172.21.15.16:0/2560361250 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7980133da0 msgr2=0x7f798011f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:40.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.382+0000 7f797ffff700 1 --2- 172.21.15.16:0/2560361250 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7980133da0 0x7f798011f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:40.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.382+0000 7f797ffff700 1 -- 172.21.15.16:0/2560361250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f798009e110 msgr2=0x7f798011eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:40.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.382+0000 7f797ffff700 1 --2- 172.21.15.16:0/2560361250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f798009e110 0x7f798011eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:40.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.382+0000 7f797ffff700 1 -- 172.21.15.16:0/2560361250 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79700094a0 con 0x7f798009d800 2023-09-30T22:31:40.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.382+0000 7f797ffff700 1 --2- 172.21.15.16:0/2560361250 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f798009d800 0x7f798011e580 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f797000ec10 tx=0x7f7970009ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:40.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.383+0000 7f797cff9700 1 -- 172.21.15.16:0/2560361250 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797000c940 con 0x7f798009d800 2023-09-30T22:31:40.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.383+0000 7f797cff9700 1 -- 172.21.15.16:0/2560361250 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f797000caa0 con 0x7f798009d800 2023-09-30T22:31:40.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.383+0000 7f797cff9700 1 -- 172.21.15.16:0/2560361250 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f797000ce10 con 0x7f798009d800 2023-09-30T22:31:40.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.383+0000 7f79855a0700 1 -- 172.21.15.16:0/2560361250 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7974009500 con 0x7f798009d800 2023-09-30T22:31:40.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.383+0000 7f79855a0700 1 -- 172.21.15.16:0/2560361250 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7980120230 con 0x7f798009d800 2023-09-30T22:31:40.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.384+0000 7f79855a0700 1 -- 172.21.15.16:0/2560361250 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7980121d00 con 0x7f798009d800 2023-09-30T22:31:40.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.385+0000 7f797cff9700 1 -- 172.21.15.16:0/2560361250 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7970013470 con 0x7f798009d800 2023-09-30T22:31:40.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.385+0000 7f797cff9700 1 -- 172.21.15.16:0/2560361250 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f797003ff40 con 0x7f798009d800 2023-09-30T22:31:40.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.389+0000 7f797cff9700 1 -- 172.21.15.16:0/2560361250 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7970013720 con 0x7f798009d800 2023-09-30T22:31:40.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.432+0000 7f0e685e3700 1 Processor -- start 2023-09-30T22:31:40.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.432+0000 7f0e685e3700 1 -- start start 2023-09-30T22:31:40.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.433+0000 7f0e685e3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0e6009e930 0x7f0e6009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:40.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.433+0000 7f0e685e3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0e6009f650 0x7f0e60131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:40.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.433+0000 7f0e685e3700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e60070940 con 0x7f0e6009f240 2023-09-30T22:31:40.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.433+0000 7f0e685e3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6006fd60 con 0x7f0e6009f650 2023-09-30T22:31:40.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.433+0000 7f0e685e3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6006fa80 con 0x7f0e6009e930 2023-09-30T22:31:40.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.433+0000 7f0e6637f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0e6009e930 0x7f0e6009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:40.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.433+0000 7f0e6637f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0e6009e930 0x7f0e6009ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:39766/0 (socket says 172.21.15.16:39766) 2023-09-30T22:31:40.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.433+0000 7f0e65b7e700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0e6009f240 0x7f0e601317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41306/0 (socket says 172.21.15.16:41306) 2023-09-30T22:31:40.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.433+0000 7f0e6637f700 1 -- 172.21.15.16:0/492716343 learned_addr learned my addr 172.21.15.16:0/492716343 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:40.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.433+0000 7f0e66b80700 1 --2- 172.21.15.16:0/492716343 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0e6009f650 0x7f0e60131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:40.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.434+0000 7f0e66b80700 1 -- 172.21.15.16:0/492716343 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0e6009e930 msgr2=0x7f0e6009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:40.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.434+0000 7f0e66b80700 1 --2- 172.21.15.16:0/492716343 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0e6009e930 0x7f0e6009ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:40.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.434+0000 7f0e66b80700 1 -- 172.21.15.16:0/492716343 >> v1:172.21.15.16:6789/0 conn(0x7f0e6009f240 legacy=0x7f0e601317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:40.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.434+0000 7f0e66b80700 1 -- 172.21.15.16:0/492716343 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e6009fb60 con 0x7f0e6009f650 2023-09-30T22:31:40.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.434+0000 7f0e66b80700 1 --2- 172.21.15.16:0/492716343 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0e6009f650 0x7f0e60131f00 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0e58009dc0 tx=0x7f0e5800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=720455dfad33ee4d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:40.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.435+0000 7f0e6537d700 1 -- 172.21.15.16:0/492716343 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5800e070 con 0x7f0e6009f650 2023-09-30T22:31:40.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.435+0000 7f0e6537d700 1 -- 172.21.15.16:0/492716343 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e5800bb20 con 0x7f0e6009f650 2023-09-30T22:31:40.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.435+0000 7f0e6537d700 1 -- 172.21.15.16:0/492716343 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e58012430 con 0x7f0e6009f650 2023-09-30T22:31:40.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.435+0000 7f0e685e3700 1 -- 172.21.15.16:0/492716343 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0e6009f650 msgr2=0x7f0e60131f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:40.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.435+0000 7f0e685e3700 1 --2- 172.21.15.16:0/492716343 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0e6009f650 0x7f0e60131f00 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0e58009dc0 tx=0x7f0e5800b330 comp rx=0 tx=0).stop 2023-09-30T22:31:40.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.436+0000 7f0e685e3700 1 -- 172.21.15.16:0/492716343 shutdown_connections 2023-09-30T22:31:40.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.436+0000 7f0e685e3700 1 --2- 172.21.15.16:0/492716343 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0e6009f650 0x7f0e60131f00 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:40.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.436+0000 7f0e685e3700 1 --2- 172.21.15.16:0/492716343 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0e6009e930 0x7f0e6009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:40.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.436+0000 7f0e685e3700 1 -- 172.21.15.16:0/492716343 >> 172.21.15.16:0/492716343 conn(0x7f0e600a4210 msgr2=0x7f0e600a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:40.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.437+0000 7f0e685e3700 1 -- 172.21.15.16:0/492716343 shutdown_connections 2023-09-30T22:31:40.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.437+0000 7f0e685e3700 1 -- 172.21.15.16:0/492716343 wait complete. 2023-09-30T22:31:40.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.438+0000 7f0e685e3700 1 Processor -- start 2023-09-30T22:31:40.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.438+0000 7f0e685e3700 1 -- start start 2023-09-30T22:31:40.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.438+0000 7f0e685e3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0e6009e930 0x7f0e6006b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:40.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.438+0000 7f0e685e3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0e6009f240 0x7f0e6006bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:40.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.438+0000 7f0e685e3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e6009f650 0x7f0e6006c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:40.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.438+0000 7f0e685e3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e601b72e0 con 0x7f0e6009f650 2023-09-30T22:31:40.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.438+0000 7f0e685e3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e601a9490 con 0x7f0e6009f240 2023-09-30T22:31:40.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.438+0000 7f0e685e3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0e601a89b0 con 0x7f0e6009e930 2023-09-30T22:31:40.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.438+0000 7f0e6637f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0e6009e930 0x7f0e6006b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:40.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.438+0000 7f0e6637f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0e6009e930 0x7f0e6006b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:39782/0 (socket says 172.21.15.16:39782) 2023-09-30T22:31:40.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.438+0000 7f0e6637f700 1 -- 172.21.15.16:0/3370779614 learned_addr learned my addr 172.21.15.16:0/3370779614 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:40.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.438+0000 7f0e66b80700 1 --2- 172.21.15.16:0/3370779614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e6009f650 0x7f0e6006c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:40.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.438+0000 7f0e65b7e700 1 --2- 172.21.15.16:0/3370779614 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0e6009f240 0x7f0e6006bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:40.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.439+0000 7f0e6637f700 1 -- 172.21.15.16:0/3370779614 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0e6009f240 msgr2=0x7f0e6006bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:40.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.439+0000 7f0e6637f700 1 --2- 172.21.15.16:0/3370779614 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0e6009f240 0x7f0e6006bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:40.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.439+0000 7f0e6637f700 1 -- 172.21.15.16:0/3370779614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e6009f650 msgr2=0x7f0e6006c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:40.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.439+0000 7f0e6637f700 1 --2- 172.21.15.16:0/3370779614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e6009f650 0x7f0e6006c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:40.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.439+0000 7f0e6637f700 1 -- 172.21.15.16:0/3370779614 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e58009a70 con 0x7f0e6009e930 2023-09-30T22:31:40.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.439+0000 7f0e6637f700 1 --2- 172.21.15.16:0/3370779614 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0e6009e930 0x7f0e6006b8b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c00e910 tx=0x7f0e5c00ec20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:40.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.439+0000 7f0e56ffd700 1 -- 172.21.15.16:0/3370779614 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c0096a0 con 0x7f0e6009e930 2023-09-30T22:31:40.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.439+0000 7f0e56ffd700 1 -- 172.21.15.16:0/3370779614 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e5c013680 con 0x7f0e6009e930 2023-09-30T22:31:40.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.439+0000 7f0e56ffd700 1 -- 172.21.15.16:0/3370779614 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5c013a30 con 0x7f0e6009e930 2023-09-30T22:31:40.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.439+0000 7f0e685e3700 1 -- 172.21.15.16:0/3370779614 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e600a0190 con 0x7f0e6009e930 2023-09-30T22:31:40.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.439+0000 7f0e685e3700 1 -- 172.21.15.16:0/3370779614 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e601a97a0 con 0x7f0e6009e930 2023-09-30T22:31:40.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.440+0000 7f0e56ffd700 1 -- 172.21.15.16:0/3370779614 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0e5c013b90 con 0x7f0e6009e930 2023-09-30T22:31:40.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.441+0000 7f0e56ffd700 1 -- 172.21.15.16:0/3370779614 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f0e5c03f7b0 con 0x7f0e6009e930 2023-09-30T22:31:40.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.441+0000 7f0e685e3700 1 -- 172.21.15.16:0/3370779614 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e601a97a0 con 0x7f0e6009e930 2023-09-30T22:31:40.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.450+0000 7f0e56ffd700 1 -- 172.21.15.16:0/3370779614 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0e601a97a0 con 0x7f0e6009e930 2023-09-30T22:31:40.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.457+0000 7f72f112e700 1 Processor -- start 2023-09-30T22:31:40.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.457+0000 7f72f112e700 1 -- start start 2023-09-30T22:31:40.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.457+0000 7f72f112e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72ec0a0a30 0x7f72ec0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:40.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.457+0000 7f72f112e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f72ec09d630 0x7f72ec0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:40.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.457+0000 7f72f112e700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72ec0a2a70 con 0x7f72ec133930 2023-09-30T22:31:40.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.457+0000 7f72f112e700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72ec0a01d0 con 0x7f72ec09d630 2023-09-30T22:31:40.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.457+0000 7f72f112e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f72ec09e2b0 con 0x7f72ec0a0a30 2023-09-30T22:31:40.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.458+0000 7f72eb59e700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f72ec133930 0x7f72ec133db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41312/0 (socket says 172.21.15.16:41312) 2023-09-30T22:31:40.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.458+0000 7f72eb59e700 1 -- 172.21.15.16:0/2120875330 learned_addr learned my addr 172.21.15.16:0/2120875330 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:31:40.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.458+0000 7f72ead9d700 1 --2- 172.21.15.16:0/2120875330 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72ec0a0a30 0x7f72ec0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:40.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.458+0000 7f72ea59c700 1 --2- 172.21.15.16:0/2120875330 >> v2:172.21.15.16:3300/0 conn(0x7f72ec09d630 0x7f72ec0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:40.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.458+0000 7f72ea59c700 1 -- 172.21.15.16:0/2120875330 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72ec0a0a30 msgr2=0x7f72ec0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:40.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.458+0000 7f72ea59c700 1 --2- 172.21.15.16:0/2120875330 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72ec0a0a30 0x7f72ec0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:40.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.458+0000 7f72ea59c700 1 -- 172.21.15.16:0/2120875330 >> v1:172.21.15.16:6789/0 conn(0x7f72ec133930 legacy=0x7f72ec133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:40.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.458+0000 7f72ea59c700 1 -- 172.21.15.16:0/2120875330 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72ec09dc50 con 0x7f72ec09d630 2023-09-30T22:31:40.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.458+0000 7f72ea59c700 1 --2- 172.21.15.16:0/2120875330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72ec09d630 0x7f72ec0a1340 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f72d4009840 tx=0x7f72d4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10ae472900cadf9d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:40.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.459+0000 7f72e9d9b700 1 -- 172.21.15.16:0/2120875330 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d4011830 con 0x7f72ec09d630 2023-09-30T22:31:40.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.459+0000 7f72e9d9b700 1 -- 172.21.15.16:0/2120875330 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72d4011990 con 0x7f72ec09d630 2023-09-30T22:31:40.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.459+0000 7f72e9d9b700 1 -- 172.21.15.16:0/2120875330 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d4011d20 con 0x7f72ec09d630 2023-09-30T22:31:40.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.459+0000 7f72f112e700 1 -- 172.21.15.16:0/2120875330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72ec09d630 msgr2=0x7f72ec0a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:40.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.459+0000 7f72f112e700 1 --2- 172.21.15.16:0/2120875330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72ec09d630 0x7f72ec0a1340 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f72d4009840 tx=0x7f72d4009b50 comp rx=0 tx=0).stop 2023-09-30T22:31:40.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.459+0000 7f72f112e700 1 -- 172.21.15.16:0/2120875330 shutdown_connections 2023-09-30T22:31:40.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.459+0000 7f72f112e700 1 --2- 172.21.15.16:0/2120875330 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72ec0a0a30 0x7f72ec0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:40.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.459+0000 7f72f112e700 1 --2- 172.21.15.16:0/2120875330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72ec09d630 0x7f72ec0a1340 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:40.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.459+0000 7f72f112e700 1 -- 172.21.15.16:0/2120875330 >> 172.21.15.16:0/2120875330 conn(0x7f72ec0a3050 msgr2=0x7f72ec0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:40.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.459+0000 7f72f112e700 1 -- 172.21.15.16:0/2120875330 shutdown_connections 2023-09-30T22:31:40.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.459+0000 7f72f112e700 1 -- 172.21.15.16:0/2120875330 wait complete. 2023-09-30T22:31:40.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.461+0000 7f72f112e700 1 Processor -- start 2023-09-30T22:31:40.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.461+0000 7f72f112e700 1 -- start start 2023-09-30T22:31:40.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.461+0000 7f72f112e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72ec09d630 0x7f72ec127e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:40.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.461+0000 7f72f112e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72ec0a0a30 0x7f72ec126280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:40.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.461+0000 7f72f112e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72ec133930 0x7f72ec1267c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:40.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.461+0000 7f72f112e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72ec12a610 con 0x7f72ec0a0a30 2023-09-30T22:31:40.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.461+0000 7f72f112e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72ec129460 con 0x7f72ec09d630 2023-09-30T22:31:40.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.461+0000 7f72f112e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f72ec1296b0 con 0x7f72ec133930 2023-09-30T22:31:40.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.462+0000 7f72ead9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72ec09d630 0x7f72ec127e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:40.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.462+0000 7f72ead9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72ec09d630 0x7f72ec127e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:40492/0 (socket says 172.21.15.16:40492) 2023-09-30T22:31:40.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.462+0000 7f72ead9d700 1 -- 172.21.15.16:0/3181868358 learned_addr learned my addr 172.21.15.16:0/3181868358 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:40.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.462+0000 7f72ead9d700 1 -- 172.21.15.16:0/3181868358 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72ec133930 msgr2=0x7f72ec1267c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:40.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.462+0000 7f72eb59e700 1 --2- 172.21.15.16:0/3181868358 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72ec133930 0x7f72ec1267c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:40.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.462+0000 7f72ead9d700 1 --2- 172.21.15.16:0/3181868358 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72ec133930 0x7f72ec1267c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:40.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.462+0000 7f72ea59c700 1 --2- 172.21.15.16:0/3181868358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72ec0a0a30 0x7f72ec126280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:40.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.462+0000 7f72ead9d700 1 -- 172.21.15.16:0/3181868358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72ec0a0a30 msgr2=0x7f72ec126280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:40.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.462+0000 7f72ead9d700 1 --2- 172.21.15.16:0/3181868358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72ec0a0a30 0x7f72ec126280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:40.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.462+0000 7f72ead9d700 1 -- 172.21.15.16:0/3181868358 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d40094a0 con 0x7f72ec09d630 2023-09-30T22:31:40.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.462+0000 7f72eb59e700 1 --2- 172.21.15.16:0/3181868358 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72ec133930 0x7f72ec1267c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:31:40.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.462+0000 7f72ea59c700 1 --2- 172.21.15.16:0/3181868358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72ec0a0a30 0x7f72ec126280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:31:40.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.462+0000 7f72ead9d700 1 --2- 172.21.15.16:0/3181868358 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72ec09d630 0x7f72ec127e10 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f72dc00efc0 tx=0x7f72dc00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:40.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.462+0000 7f72e37fe700 1 -- 172.21.15.16:0/3181868358 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72dc00cdc0 con 0x7f72ec09d630 2023-09-30T22:31:40.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.462+0000 7f72e37fe700 1 -- 172.21.15.16:0/3181868358 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72dc00f040 con 0x7f72ec09d630 2023-09-30T22:31:40.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.462+0000 7f72e37fe700 1 -- 172.21.15.16:0/3181868358 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72dc0146a0 con 0x7f72ec09d630 2023-09-30T22:31:40.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.463+0000 7f72f112e700 1 -- 172.21.15.16:0/3181868358 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72ec12a120 con 0x7f72ec09d630 2023-09-30T22:31:40.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.463+0000 7f72f112e700 1 -- 172.21.15.16:0/3181868358 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72ec126d00 con 0x7f72ec09d630 2023-09-30T22:31:40.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.464+0000 7f72f112e700 1 -- 172.21.15.16:0/3181868358 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72ec127c60 con 0x7f72ec09d630 2023-09-30T22:31:40.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.464+0000 7f72e37fe700 1 -- 172.21.15.16:0/3181868358 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f72dc014800 con 0x7f72ec09d630 2023-09-30T22:31:40.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.464+0000 7f72e37fe700 1 -- 172.21.15.16:0/3181868358 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f72dc040650 con 0x7f72ec09d630 2023-09-30T22:31:40.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.473+0000 7f72e37fe700 1 -- 172.21.15.16:0/3181868358 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f72dc014e30 con 0x7f72ec09d630 2023-09-30T22:31:40.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.539+0000 7f5acbe9e700 1 -- 172.21.15.16:0/253289334 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5aac005590 con 0x7f5ac409de50 2023-09-30T22:31:40.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.539+0000 7f5aba7fc700 1 -- 172.21.15.16:0/253289334 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5ac0024350 con 0x7f5ac409de50 2023-09-30T22:31:40.540 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:31:40.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.542+0000 7f5acbe9e700 1 -- 172.21.15.16:0/253289334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ac409de50 msgr2=0x7f5ac411ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:40.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.542+0000 7f5acbe9e700 1 --2- 172.21.15.16:0/253289334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ac409de50 0x7f5ac411ea30 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f5ac000ebe0 tx=0x7f5ac000eef0 comp rx=0 tx=0).stop 2023-09-30T22:31:40.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.542+0000 7f5acbe9e700 1 -- 172.21.15.16:0/253289334 shutdown_connections 2023-09-30T22:31:40.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.542+0000 7f5acbe9e700 1 --2- 172.21.15.16:0/253289334 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5ac41338a0 0x7f5ac411f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:40.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.542+0000 7f5acbe9e700 1 --2- 172.21.15.16:0/253289334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ac409de50 0x7f5ac411ea30 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:40.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.542+0000 7f5acbe9e700 1 --2- 172.21.15.16:0/253289334 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5ac409e760 0x7f5ac411ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:40.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.542+0000 7f5acbe9e700 1 -- 172.21.15.16:0/253289334 >> 172.21.15.16:0/253289334 conn(0x7f5ac40a2aa0 msgr2=0x7f5ac409edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:40.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.542+0000 7f5acbe9e700 1 -- 172.21.15.16:0/253289334 shutdown_connections 2023-09-30T22:31:40.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.542+0000 7f5acbe9e700 1 -- 172.21.15.16:0/253289334 wait complete. 2023-09-30T22:31:40.557 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:31:40.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.639+0000 7f79855a0700 1 -- 172.21.15.16:0/2560361250 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7980121d00 con 0x7f798009d800 2023-09-30T22:31:40.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.639+0000 7f797cff9700 1 -- 172.21.15.16:0/2560361250 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7980121d00 con 0x7f798009d800 2023-09-30T22:31:40.640 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:31:40.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.641+0000 7f79855a0700 1 -- 172.21.15.16:0/2560361250 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f798009d800 msgr2=0x7f798011e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:40.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.641+0000 7f79855a0700 1 --2- 172.21.15.16:0/2560361250 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f798009d800 0x7f798011e580 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f797000ec10 tx=0x7f7970009ec0 comp rx=0 tx=0).stop 2023-09-30T22:31:40.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.641+0000 7f79855a0700 1 -- 172.21.15.16:0/2560361250 shutdown_connections 2023-09-30T22:31:40.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.641+0000 7f79855a0700 1 --2- 172.21.15.16:0/2560361250 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f798009d800 0x7f798011e580 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:40.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.642+0000 7f79855a0700 1 --2- 172.21.15.16:0/2560361250 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7980133da0 0x7f798011f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:40.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.642+0000 7f79855a0700 1 --2- 172.21.15.16:0/2560361250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f798009e110 0x7f798011eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:40.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.642+0000 7f79855a0700 1 -- 172.21.15.16:0/2560361250 >> 172.21.15.16:0/2560361250 conn(0x7f79800a2e20 msgr2=0x7f7980070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:40.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.642+0000 7f79855a0700 1 -- 172.21.15.16:0/2560361250 shutdown_connections 2023-09-30T22:31:40.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.642+0000 7f79855a0700 1 -- 172.21.15.16:0/2560361250 wait complete. 2023-09-30T22:31:40.656 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:31:40.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.679+0000 7f72f112e700 1 -- 172.21.15.16:0/3181868358 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f72ec0747f0 con 0x7f72ec09d630 2023-09-30T22:31:40.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.680+0000 7f72e37fe700 1 -- 172.21.15.16:0/3181868358 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f72dc012340 con 0x7f72ec09d630 2023-09-30T22:31:40.680 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:31:40.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.682+0000 7f72f112e700 1 -- 172.21.15.16:0/3181868358 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72ec09d630 msgr2=0x7f72ec127e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:40.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.682+0000 7f72f112e700 1 --2- 172.21.15.16:0/3181868358 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72ec09d630 0x7f72ec127e10 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f72dc00efc0 tx=0x7f72dc00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:31:40.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.682+0000 7f72f112e700 1 -- 172.21.15.16:0/3181868358 shutdown_connections 2023-09-30T22:31:40.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.682+0000 7f72f112e700 1 --2- 172.21.15.16:0/3181868358 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72ec09d630 0x7f72ec127e10 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:40.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.682+0000 7f72f112e700 1 --2- 172.21.15.16:0/3181868358 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72ec133930 0x7f72ec1267c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:40.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.682+0000 7f72f112e700 1 --2- 172.21.15.16:0/3181868358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72ec0a0a30 0x7f72ec126280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:40.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.682+0000 7f72f112e700 1 -- 172.21.15.16:0/3181868358 >> 172.21.15.16:0/3181868358 conn(0x7f72ec0a3050 msgr2=0x7f72ec09ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:40.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.682+0000 7f72f112e700 1 -- 172.21.15.16:0/3181868358 shutdown_connections 2023-09-30T22:31:40.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.682+0000 7f72f112e700 1 -- 172.21.15.16:0/3181868358 wait complete. 2023-09-30T22:31:40.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.694+0000 7f0e685e3700 1 -- 172.21.15.16:0/3370779614 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0e601a97a0 con 0x7f0e6009e930 2023-09-30T22:31:40.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.695+0000 7f0e56ffd700 1 -- 172.21.15.16:0/3370779614 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0e601a97a0 con 0x7f0e6009e930 2023-09-30T22:31:40.695 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:31:40.696 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:31:40.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.697+0000 7f0e685e3700 1 -- 172.21.15.16:0/3370779614 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0e6009e930 msgr2=0x7f0e6006b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:40.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.697+0000 7f0e685e3700 1 --2- 172.21.15.16:0/3370779614 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0e6009e930 0x7f0e6006b8b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c00e910 tx=0x7f0e5c00ec20 comp rx=0 tx=0).stop 2023-09-30T22:31:40.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.697+0000 7f0e685e3700 1 -- 172.21.15.16:0/3370779614 shutdown_connections 2023-09-30T22:31:40.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.697+0000 7f0e685e3700 1 --2- 172.21.15.16:0/3370779614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e6009f650 0x7f0e6006c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:40.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.697+0000 7f0e685e3700 1 --2- 172.21.15.16:0/3370779614 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0e6009e930 0x7f0e6006b8b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:40.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.697+0000 7f0e685e3700 1 --2- 172.21.15.16:0/3370779614 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0e6009f240 0x7f0e6006bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:40.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.697+0000 7f0e685e3700 1 -- 172.21.15.16:0/3370779614 >> 172.21.15.16:0/3370779614 conn(0x7f0e600a4210 msgr2=0x7f0e601b8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:40.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.697+0000 7f0e685e3700 1 -- 172.21.15.16:0/3370779614 shutdown_connections 2023-09-30T22:31:40.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:40.697+0000 7f0e685e3700 1 -- 172.21.15.16:0/3370779614 wait complete. 2023-09-30T22:31:40.711 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:31:41.559 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:41.656 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:41.697 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:41.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.705+0000 7efbfbee5700 1 Processor -- start 2023-09-30T22:31:41.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.706+0000 7efbfbee5700 1 -- start start 2023-09-30T22:31:41.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.706+0000 7efbfbee5700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efbf41338a0 0x7efbf4133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:41.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.706+0000 7efbfbee5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efbf409de50 0x7efbf409e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:41.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.706+0000 7efbfbee5700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efbf406ef30 con 0x7efbf409e760 2023-09-30T22:31:41.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.706+0000 7efbfbee5700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efbf406fef0 con 0x7efbf41338a0 2023-09-30T22:31:41.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.706+0000 7efbfbee5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efbf41334b0 con 0x7efbf409de50 2023-09-30T22:31:41.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.706+0000 7efbfa482700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7efbf409e760 0x7efbf413a730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41320/0 (socket says 172.21.15.16:41320) 2023-09-30T22:31:41.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.706+0000 7efbf9c81700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efbf41338a0 0x7efbf4133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:41.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.706+0000 7efbfa482700 1 -- 172.21.15.16:0/819769673 learned_addr learned my addr 172.21.15.16:0/819769673 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:31:41.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.707+0000 7efbf9480700 1 --2- 172.21.15.16:0/819769673 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efbf409de50 0x7efbf409e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:41.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.707+0000 7efbf9c81700 1 -- 172.21.15.16:0/819769673 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efbf409de50 msgr2=0x7efbf409e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:41.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.707+0000 7efbf9c81700 1 --2- 172.21.15.16:0/819769673 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efbf409de50 0x7efbf409e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:41.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.707+0000 7efbf9c81700 1 -- 172.21.15.16:0/819769673 >> v1:172.21.15.16:6789/0 conn(0x7efbf409e760 legacy=0x7efbf413a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:41.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.707+0000 7efbf9c81700 1 -- 172.21.15.16:0/819769673 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbf406fc30 con 0x7efbf41338a0 2023-09-30T22:31:41.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.707+0000 7efbf9c81700 1 --2- 172.21.15.16:0/819769673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efbf41338a0 0x7efbf4133c70 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7efbf0009840 tx=0x7efbf0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23832e3364901b77 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:41.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.707+0000 7efbf8c7f700 1 -- 172.21.15.16:0/819769673 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf0011830 con 0x7efbf41338a0 2023-09-30T22:31:41.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.707+0000 7efbf8c7f700 1 -- 172.21.15.16:0/819769673 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbf0011990 con 0x7efbf41338a0 2023-09-30T22:31:41.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.707+0000 7efbf8c7f700 1 -- 172.21.15.16:0/819769673 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf0011d20 con 0x7efbf41338a0 2023-09-30T22:31:41.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.708+0000 7efbfbee5700 1 -- 172.21.15.16:0/819769673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efbf41338a0 msgr2=0x7efbf4133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:41.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.708+0000 7efbfbee5700 1 --2- 172.21.15.16:0/819769673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efbf41338a0 0x7efbf4133c70 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7efbf0009840 tx=0x7efbf0009b50 comp rx=0 tx=0).stop 2023-09-30T22:31:41.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.708+0000 7efbfbee5700 1 -- 172.21.15.16:0/819769673 shutdown_connections 2023-09-30T22:31:41.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.708+0000 7efbfbee5700 1 --2- 172.21.15.16:0/819769673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efbf41338a0 0x7efbf4133c70 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:41.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.708+0000 7efbfbee5700 1 --2- 172.21.15.16:0/819769673 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efbf409de50 0x7efbf409e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:41.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.708+0000 7efbfbee5700 1 -- 172.21.15.16:0/819769673 >> 172.21.15.16:0/819769673 conn(0x7efbf40a2aa0 msgr2=0x7efbf40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:41.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.708+0000 7efbfbee5700 1 -- 172.21.15.16:0/819769673 shutdown_connections 2023-09-30T22:31:41.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.708+0000 7efbfbee5700 1 -- 172.21.15.16:0/819769673 wait complete. 2023-09-30T22:31:41.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.710+0000 7efbfbee5700 1 Processor -- start 2023-09-30T22:31:41.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.710+0000 7efbfbee5700 1 -- start start 2023-09-30T22:31:41.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.710+0000 7efbfbee5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efbf409de50 0x7efbf411eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:41.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.710+0000 7efbfbee5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efbf409e760 0x7efbf411f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:41.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.710+0000 7efbfbee5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efbf41338a0 0x7efbf411f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:41.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.710+0000 7efbfbee5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf412f710 con 0x7efbf409de50 2023-09-30T22:31:41.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.710+0000 7efbfbee5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efbf4121d30 con 0x7efbf41338a0 2023-09-30T22:31:41.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.710+0000 7efbfbee5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efbf4120720 con 0x7efbf409e760 2023-09-30T22:31:41.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.710+0000 7efbf9c81700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efbf409de50 0x7efbf411eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:41.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.710+0000 7efbf9c81700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efbf409de50 0x7efbf411eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48142/0 (socket says 172.21.15.16:48142) 2023-09-30T22:31:41.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.710+0000 7efbf9c81700 1 -- 172.21.15.16:0/2866141508 learned_addr learned my addr 172.21.15.16:0/2866141508 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:41.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.710+0000 7efbfa482700 1 --2- 172.21.15.16:0/2866141508 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efbf41338a0 0x7efbf411f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:41.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.711+0000 7efbf9c81700 1 -- 172.21.15.16:0/2866141508 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efbf409e760 msgr2=0x7efbf411f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:41.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.711+0000 7efbf9c81700 1 --2- 172.21.15.16:0/2866141508 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efbf409e760 0x7efbf411f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:41.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.711+0000 7efbf9c81700 1 -- 172.21.15.16:0/2866141508 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efbf41338a0 msgr2=0x7efbf411f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:41.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.711+0000 7efbf9c81700 1 --2- 172.21.15.16:0/2866141508 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efbf41338a0 0x7efbf411f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:41.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.711+0000 7efbf9c81700 1 -- 172.21.15.16:0/2866141508 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbf00094a0 con 0x7efbf409de50 2023-09-30T22:31:41.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.711+0000 7efbf9c81700 1 --2- 172.21.15.16:0/2866141508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efbf409de50 0x7efbf411eba0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7efbf00090b0 tx=0x7efbf0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:41.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.711+0000 7efbea7fc700 1 -- 172.21.15.16:0/2866141508 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf00036a0 con 0x7efbf409de50 2023-09-30T22:31:41.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.711+0000 7efbea7fc700 1 -- 172.21.15.16:0/2866141508 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbf001f070 con 0x7efbf409de50 2023-09-30T22:31:41.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.711+0000 7efbfbee5700 1 -- 172.21.15.16:0/2866141508 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efbf4122630 con 0x7efbf409de50 2023-09-30T22:31:41.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.711+0000 7efbea7fc700 1 -- 172.21.15.16:0/2866141508 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf0003a60 con 0x7efbf409de50 2023-09-30T22:31:41.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.711+0000 7efbfbee5700 1 -- 172.21.15.16:0/2866141508 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efbf4122350 con 0x7efbf409de50 2023-09-30T22:31:41.714 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:41.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.713+0000 7efbfbee5700 1 -- 172.21.15.16:0/2866141508 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efbf4121fb0 con 0x7efbf409de50 2023-09-30T22:31:41.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.713+0000 7efbea7fc700 1 -- 172.21.15.16:0/2866141508 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7efbf0003bc0 con 0x7efbf409de50 2023-09-30T22:31:41.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.714+0000 7efbea7fc700 1 -- 172.21.15.16:0/2866141508 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7efbf0021070 con 0x7efbf409de50 2023-09-30T22:31:41.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.718+0000 7efbea7fc700 1 -- 172.21.15.16:0/2866141508 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7efbf001a430 con 0x7efbf409de50 2023-09-30T22:31:41.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.806+0000 7f42aa926700 1 Processor -- start 2023-09-30T22:31:41.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.806+0000 7f42aa926700 1 -- start start 2023-09-30T22:31:41.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.806+0000 7f42aa926700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f42a409d6f0 0x7f42a409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:41.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.806+0000 7f42aa926700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f42a409e410 0x7f42a4130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:41.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.806+0000 7f42aa926700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42a409f130 con 0x7f42a409e000 2023-09-30T22:31:41.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.806+0000 7f42aa926700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42a409f520 con 0x7f42a409e410 2023-09-30T22:31:41.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.806+0000 7f42aa926700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f42a409e930 con 0x7f42a409d6f0 2023-09-30T22:31:41.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.807+0000 7f42a3fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f42a409d6f0 0x7f42a409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:41.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.807+0000 7f42a37fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f42a409e000 0x7f42a4130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41324/0 (socket says 172.21.15.16:41324) 2023-09-30T22:31:41.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.807+0000 7f42a3fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f42a409d6f0 0x7f42a409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:39808/0 (socket says 172.21.15.16:39808) 2023-09-30T22:31:41.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.807+0000 7f42a37fe700 1 -- 172.21.15.16:0/3992590330 learned_addr learned my addr 172.21.15.16:0/3992590330 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:31:41.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.807+0000 7f42a8ec3700 1 --2- 172.21.15.16:0/3992590330 >> v2:172.21.15.16:3300/0 conn(0x7f42a409e410 0x7f42a4130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:41.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.807+0000 7f42a3fff700 1 -- 172.21.15.16:0/3992590330 >> v2:172.21.15.16:3300/0 conn(0x7f42a409e410 msgr2=0x7f42a4130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:41.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.807+0000 7f42a3fff700 1 --2- 172.21.15.16:0/3992590330 >> v2:172.21.15.16:3300/0 conn(0x7f42a409e410 0x7f42a4130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:41.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.807+0000 7f42a3fff700 1 -- 172.21.15.16:0/3992590330 >> v1:172.21.15.16:6789/0 conn(0x7f42a409e000 legacy=0x7f42a4130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:41.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.807+0000 7f42a3fff700 1 -- 172.21.15.16:0/3992590330 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a409b2d0 con 0x7f42a409d6f0 2023-09-30T22:31:41.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.807+0000 7f42a8ec3700 1 --2- 172.21.15.16:0/3992590330 >> v2:172.21.15.16:3300/0 conn(0x7f42a409e410 0x7f42a4130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:41.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.807+0000 7f42a3fff700 1 --2- 172.21.15.16:0/3992590330 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f42a409d6f0 0x7f42a409dac0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4294009840 tx=0x7f4294009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cad0efe98b2d3686 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:41.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.808+0000 7f42a2ffd700 1 -- 172.21.15.16:0/3992590330 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4294011830 con 0x7f42a409d6f0 2023-09-30T22:31:41.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.808+0000 7f42a2ffd700 1 -- 172.21.15.16:0/3992590330 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4294011990 con 0x7f42a409d6f0 2023-09-30T22:31:41.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.808+0000 7f42a2ffd700 1 -- 172.21.15.16:0/3992590330 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4294011d20 con 0x7f42a409d6f0 2023-09-30T22:31:41.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.809+0000 7f42aa926700 1 -- 172.21.15.16:0/3992590330 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f42a409d6f0 msgr2=0x7f42a409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:41.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.809+0000 7f42aa926700 1 --2- 172.21.15.16:0/3992590330 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f42a409d6f0 0x7f42a409dac0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4294009840 tx=0x7f4294009b50 comp rx=0 tx=0).stop 2023-09-30T22:31:41.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.809+0000 7f42aa926700 1 -- 172.21.15.16:0/3992590330 shutdown_connections 2023-09-30T22:31:41.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.809+0000 7f42aa926700 1 --2- 172.21.15.16:0/3992590330 >> v2:172.21.15.16:3300/0 conn(0x7f42a409e410 0x7f42a4130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:41.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.809+0000 7f42aa926700 1 --2- 172.21.15.16:0/3992590330 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f42a409d6f0 0x7f42a409dac0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:41.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.809+0000 7f42aa926700 1 -- 172.21.15.16:0/3992590330 >> 172.21.15.16:0/3992590330 conn(0x7f42a4066f50 msgr2=0x7f42a409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:41.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.809+0000 7f42aa926700 1 -- 172.21.15.16:0/3992590330 shutdown_connections 2023-09-30T22:31:41.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.809+0000 7f42aa926700 1 -- 172.21.15.16:0/3992590330 wait complete. 2023-09-30T22:31:41.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.810+0000 7f42aa926700 1 Processor -- start 2023-09-30T22:31:41.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.810+0000 7f42aa926700 1 -- start start 2023-09-30T22:31:41.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.810+0000 7f42aa926700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f42a409d6f0 0x7f42a411e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:41.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.810+0000 7f42aa926700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f42a409e000 0x7f42a411e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:41.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.810+0000 7f42aa926700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42a409e410 0x7f42a411eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:41.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.810+0000 7f42aa926700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a4121b40 con 0x7f42a409e410 2023-09-30T22:31:41.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.810+0000 7f42aa926700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42a409c770 con 0x7f42a409e000 2023-09-30T22:31:41.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.810+0000 7f42aa926700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f42a40a1e90 con 0x7f42a409d6f0 2023-09-30T22:31:41.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.811+0000 7f42a3fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f42a409d6f0 0x7f42a411e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:41.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.811+0000 7f42a3fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f42a409d6f0 0x7f42a411e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:39810/0 (socket says 172.21.15.16:39810) 2023-09-30T22:31:41.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.811+0000 7f42a3fff700 1 -- 172.21.15.16:0/3538154256 learned_addr learned my addr 172.21.15.16:0/3538154256 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:41.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.811+0000 7f42a8ec3700 1 --2- 172.21.15.16:0/3538154256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42a409e410 0x7f42a411eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:41.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.811+0000 7f42a3fff700 1 -- 172.21.15.16:0/3538154256 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f42a409e000 msgr2=0x7f42a411e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:41.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.811+0000 7f42a3fff700 1 --2- 172.21.15.16:0/3538154256 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f42a409e000 0x7f42a411e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:41.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.811+0000 7f42a3fff700 1 -- 172.21.15.16:0/3538154256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42a409e410 msgr2=0x7f42a411eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:41.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.811+0000 7f42a3fff700 1 --2- 172.21.15.16:0/3538154256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42a409e410 0x7f42a411eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:41.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.811+0000 7f42a3fff700 1 -- 172.21.15.16:0/3538154256 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f429c00bc80 con 0x7f42a409d6f0 2023-09-30T22:31:41.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.811+0000 7f42a8ec3700 1 --2- 172.21.15.16:0/3538154256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42a409e410 0x7f42a411eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:41.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.811+0000 7f42a3fff700 1 --2- 172.21.15.16:0/3538154256 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f42a409d6f0 0x7f42a411e0b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4294009fd0 tx=0x7f4294004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:41.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.812+0000 7f42a0ff9700 1 -- 172.21.15.16:0/3538154256 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42940036a0 con 0x7f42a409d6f0 2023-09-30T22:31:41.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.812+0000 7f42a0ff9700 1 -- 172.21.15.16:0/3538154256 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f429401e070 con 0x7f42a409d6f0 2023-09-30T22:31:41.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.812+0000 7f42aa926700 1 -- 172.21.15.16:0/3538154256 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42940094a0 con 0x7f42a409d6f0 2023-09-30T22:31:41.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.812+0000 7f42a0ff9700 1 -- 172.21.15.16:0/3538154256 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4294003a60 con 0x7f42a409d6f0 2023-09-30T22:31:41.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.812+0000 7f42aa926700 1 -- 172.21.15.16:0/3538154256 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42a41215a0 con 0x7f42a409d6f0 2023-09-30T22:31:41.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.813+0000 7f42aa926700 1 -- 172.21.15.16:0/3538154256 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42a4120e80 con 0x7f42a409d6f0 2023-09-30T22:31:41.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.814+0000 7f42a0ff9700 1 -- 172.21.15.16:0/3538154256 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4294003bc0 con 0x7f42a409d6f0 2023-09-30T22:31:41.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.814+0000 7f42a0ff9700 1 -- 172.21.15.16:0/3538154256 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f429401a8f0 con 0x7f42a409d6f0 2023-09-30T22:31:41.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.823+0000 7f42a0ff9700 1 -- 172.21.15.16:0/3538154256 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f429400f2e0 con 0x7f42a409d6f0 2023-09-30T22:31:41.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.861+0000 7ff296056700 1 Processor -- start 2023-09-30T22:31:41.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.861+0000 7ff296056700 1 -- start start 2023-09-30T22:31:41.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.862+0000 7ff296056700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff2900a0a30 0x7ff2900a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:41.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.862+0000 7ff296056700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff290134170 0x7ff29013bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:41.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.862+0000 7ff296056700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff29009edd0 con 0x7ff2900a1340 2023-09-30T22:31:41.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.862+0000 7ff296056700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff29009e1f0 con 0x7ff290134170 2023-09-30T22:31:41.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.862+0000 7ff296056700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff29009dc50 con 0x7ff2900a0a30 2023-09-30T22:31:41.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.862+0000 7ff28f7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff2900a0a30 0x7ff2900a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:41.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.862+0000 7ff28f7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff2900a0a30 0x7ff2900a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:39826/0 (socket says 172.21.15.16:39826) 2023-09-30T22:31:41.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.862+0000 7ff28f7fe700 1 -- 172.21.15.16:0/2231946393 learned_addr learned my addr 172.21.15.16:0/2231946393 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:41.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.862+0000 7ff28ffff700 1 --2- 172.21.15.16:0/2231946393 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff290134170 0x7ff29013bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:41.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.862+0000 7ff28f7fe700 1 -- 172.21.15.16:0/2231946393 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff290134170 msgr2=0x7ff29013bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:41.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.862+0000 7ff28f7fe700 1 --2- 172.21.15.16:0/2231946393 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff290134170 0x7ff29013bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:41.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.863+0000 7ff28f7fe700 1 -- 172.21.15.16:0/2231946393 >> v1:172.21.15.16:6789/0 conn(0x7ff2900a1340 legacy=0x7ff290133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:41.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.862+0000 7ff28e7fc700 1 -- 172.21.15.16:0/2231946393 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2882205813 0 0) 0x7ff29009edd0 con 0x7ff2900a1340 2023-09-30T22:31:41.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.863+0000 7ff28f7fe700 1 -- 172.21.15.16:0/2231946393 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2900a1fe0 con 0x7ff2900a0a30 2023-09-30T22:31:41.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.863+0000 7ff28ffff700 1 --2- 172.21.15.16:0/2231946393 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff290134170 0x7ff29013bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:31:41.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.863+0000 7ff28f7fe700 1 --2- 172.21.15.16:0/2231946393 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff2900a0a30 0x7ff2900a0e00 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff280009920 tx=0x7ff280009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a50342035dbb5451 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:41.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.863+0000 7ff28e7fc700 1 -- 172.21.15.16:0/2231946393 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28000ba60 con 0x7ff2900a0a30 2023-09-30T22:31:41.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.863+0000 7ff28e7fc700 1 -- 172.21.15.16:0/2231946393 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff28000bbc0 con 0x7ff2900a0a30 2023-09-30T22:31:41.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.863+0000 7ff28e7fc700 1 -- 172.21.15.16:0/2231946393 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff280009e70 con 0x7ff2900a0a30 2023-09-30T22:31:41.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.864+0000 7ff296056700 1 -- 172.21.15.16:0/2231946393 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff2900a0a30 msgr2=0x7ff2900a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:41.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.864+0000 7ff296056700 1 --2- 172.21.15.16:0/2231946393 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff2900a0a30 0x7ff2900a0e00 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff280009920 tx=0x7ff280009c30 comp rx=0 tx=0).stop 2023-09-30T22:31:41.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.864+0000 7ff296056700 1 -- 172.21.15.16:0/2231946393 shutdown_connections 2023-09-30T22:31:41.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.864+0000 7ff296056700 1 --2- 172.21.15.16:0/2231946393 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff290134170 0x7ff29013bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:41.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.864+0000 7ff296056700 1 --2- 172.21.15.16:0/2231946393 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff2900a0a30 0x7ff2900a0e00 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:41.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.864+0000 7ff296056700 1 -- 172.21.15.16:0/2231946393 >> 172.21.15.16:0/2231946393 conn(0x7ff2900a6550 msgr2=0x7ff2900a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:41.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.864+0000 7ff296056700 1 -- 172.21.15.16:0/2231946393 shutdown_connections 2023-09-30T22:31:41.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.864+0000 7ff296056700 1 -- 172.21.15.16:0/2231946393 wait complete. 2023-09-30T22:31:41.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.865+0000 7fdd0490d700 1 Processor -- start 2023-09-30T22:31:41.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.865+0000 7fdd0490d700 1 -- start start 2023-09-30T22:31:41.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.866+0000 7fdd0490d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdd0012cf60 0x7fdd0006f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:41.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.866+0000 7fdd0490d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdd00130430 0x7fdd00137c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:41.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.866+0000 7fdd0490d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd0012cd90 con 0x7fdd0012f8b0 2023-09-30T22:31:41.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.866+0000 7fdd0490d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0012d330 con 0x7fdd00130430 2023-09-30T22:31:41.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.866+0000 7fdd0490d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdd00071780 con 0x7fdd0012cf60 2023-09-30T22:31:41.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.866+0000 7fdcfe59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdd0012cf60 0x7fdd0006f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:41.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.866+0000 7fdcfe59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdd0012cf60 0x7fdd0006f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:39840/0 (socket says 172.21.15.16:39840) 2023-09-30T22:31:41.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.866+0000 7fdcfe59c700 1 -- 172.21.15.16:0/3474076212 learned_addr learned my addr 172.21.15.16:0/3474076212 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:41.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.866+0000 7fdcfed9d700 1 --2- 172.21.15.16:0/3474076212 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdd00130430 0x7fdd00137c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:41.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.866+0000 7ff296056700 1 Processor -- start 2023-09-30T22:31:41.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.866+0000 7ff296056700 1 -- start start 2023-09-30T22:31:41.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.867+0000 7ff296056700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff2900a0a30 0x7ff290122870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:41.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.867+0000 7ff296056700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2900a1340 0x7ff290122db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:41.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.867+0000 7ff296056700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff290134170 0x7ff2901232f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:41.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.867+0000 7ff296056700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff290124c80 con 0x7ff2900a1340 2023-09-30T22:31:41.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.867+0000 7ff296056700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2900a0220 con 0x7ff290134170 2023-09-30T22:31:41.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.867+0000 7ff296056700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff290124500 con 0x7ff2900a0a30 2023-09-30T22:31:41.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.867+0000 7ff28ffff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff290134170 0x7ff2901232f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:41.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.867+0000 7ff28ffff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff290134170 0x7ff2901232f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:40548/0 (socket says 172.21.15.16:40548) 2023-09-30T22:31:41.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.867+0000 7ff28ffff700 1 -- 172.21.15.16:0/200035340 learned_addr learned my addr 172.21.15.16:0/200035340 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:41.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.867+0000 7fdcfe59c700 1 -- 172.21.15.16:0/3474076212 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdd00130430 msgr2=0x7fdd00137c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:41.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.867+0000 7fdcfe59c700 1 --2- 172.21.15.16:0/3474076212 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdd00130430 0x7fdd00137c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:41.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.867+0000 7fdcfe59c700 1 -- 172.21.15.16:0/3474076212 >> v1:172.21.15.16:6789/0 conn(0x7fdd0012f8b0 legacy=0x7fdd0012fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:41.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.867+0000 7fdcfe59c700 1 -- 172.21.15.16:0/3474076212 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd00071a40 con 0x7fdd0012cf60 2023-09-30T22:31:41.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.867+0000 7fdcfe59c700 1 --2- 172.21.15.16:0/3474076212 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdd0012cf60 0x7fdd0006f5d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fdcf000aac0 tx=0x7fdcf000add0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6cfc9422c0ffd6f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:41.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.867+0000 7ff28ffff700 1 -- 172.21.15.16:0/200035340 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff2900a0a30 msgr2=0x7ff290122870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:41.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.867+0000 7ff28ffff700 1 --2- 172.21.15.16:0/200035340 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff2900a0a30 0x7ff290122870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:41.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.867+0000 7ff28ffff700 1 -- 172.21.15.16:0/200035340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2900a1340 msgr2=0x7ff290122db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:41.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.867+0000 7ff28ffff700 1 --2- 172.21.15.16:0/200035340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2900a1340 0x7ff290122db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:41.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.867+0000 7ff28ffff700 1 -- 172.21.15.16:0/200035340 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff280009550 con 0x7ff290134170 2023-09-30T22:31:41.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.867+0000 7fdcfd59a700 1 -- 172.21.15.16:0/3474076212 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf0013c90 con 0x7fdd0012cf60 2023-09-30T22:31:41.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.867+0000 7fdcfd59a700 1 -- 172.21.15.16:0/3474076212 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcf0013df0 con 0x7fdd0012cf60 2023-09-30T22:31:41.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.867+0000 7fdcfd59a700 1 -- 172.21.15.16:0/3474076212 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf0004250 con 0x7fdd0012cf60 2023-09-30T22:31:41.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.869+0000 7fdd0490d700 1 -- 172.21.15.16:0/3474076212 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdd0012cf60 msgr2=0x7fdd0006f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:41.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.869+0000 7fdd0490d700 1 --2- 172.21.15.16:0/3474076212 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdd0012cf60 0x7fdd0006f5d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fdcf000aac0 tx=0x7fdcf000add0 comp rx=0 tx=0).stop 2023-09-30T22:31:41.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.869+0000 7ff28ffff700 1 --2- 172.21.15.16:0/200035340 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff290134170 0x7ff2901232f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff288009fa0 tx=0x7ff28800d460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:41.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.869+0000 7fdd0490d700 1 -- 172.21.15.16:0/3474076212 shutdown_connections 2023-09-30T22:31:41.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.869+0000 7fdd0490d700 1 --2- 172.21.15.16:0/3474076212 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdd00130430 0x7fdd00137c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:41.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.869+0000 7fdd0490d700 1 --2- 172.21.15.16:0/3474076212 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdd0012cf60 0x7fdd0006f5d0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:41.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.869+0000 7fdd0490d700 1 -- 172.21.15.16:0/3474076212 >> 172.21.15.16:0/3474076212 conn(0x7fdd000a2aa0 msgr2=0x7fdd000a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:41.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.869+0000 7fdd0490d700 1 -- 172.21.15.16:0/3474076212 shutdown_connections 2023-09-30T22:31:41.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.869+0000 7fdd0490d700 1 -- 172.21.15.16:0/3474076212 wait complete. 2023-09-30T22:31:41.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.869+0000 7ff277fff700 1 -- 172.21.15.16:0/200035340 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff288015400 con 0x7ff290134170 2023-09-30T22:31:41.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.869+0000 7ff277fff700 1 -- 172.21.15.16:0/200035340 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff28800f040 con 0x7ff290134170 2023-09-30T22:31:41.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.869+0000 7ff296056700 1 -- 172.21.15.16:0/200035340 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2900a25e0 con 0x7ff290134170 2023-09-30T22:31:41.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.869+0000 7ff277fff700 1 -- 172.21.15.16:0/200035340 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff288015760 con 0x7ff290134170 2023-09-30T22:31:41.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.869+0000 7ff296056700 1 -- 172.21.15.16:0/200035340 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff290133300 con 0x7ff290134170 2023-09-30T22:31:41.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.870+0000 7ff277fff700 1 -- 172.21.15.16:0/200035340 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff28801e3b0 con 0x7ff290134170 2023-09-30T22:31:41.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.870+0000 7ff277fff700 1 -- 172.21.15.16:0/200035340 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7ff28803f910 con 0x7ff290134170 2023-09-30T22:31:41.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.871+0000 7fdd0490d700 1 Processor -- start 2023-09-30T22:31:41.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.871+0000 7ff296056700 1 -- 172.21.15.16:0/200035340 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff290074ea0 con 0x7ff290134170 2023-09-30T22:31:41.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.871+0000 7fdd0490d700 1 -- start start 2023-09-30T22:31:41.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.872+0000 7fdd0490d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdd0012cf60 0x7fdd001ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:41.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.872+0000 7fdd0490d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd0012f8b0 0x7fdd001cd020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:41.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.872+0000 7fdd0490d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdd00130430 0x7fdd001cd560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:41.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.872+0000 7fdd0490d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0009ebc0 con 0x7fdd0012f8b0 2023-09-30T22:31:41.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.872+0000 7fdd0490d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd00123e40 con 0x7fdd00130430 2023-09-30T22:31:41.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.872+0000 7fdd0490d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0012d7b0 con 0x7fdd0012cf60 2023-09-30T22:31:41.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.872+0000 7fdcfe59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdd0012cf60 0x7fdd001ccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:41.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.872+0000 7fdcfe59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdd0012cf60 0x7fdd001ccae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:39864/0 (socket says 172.21.15.16:39864) 2023-09-30T22:31:41.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.872+0000 7fdcfe59c700 1 -- 172.21.15.16:0/2500273822 learned_addr learned my addr 172.21.15.16:0/2500273822 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:41.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.872+0000 7fdcfed9d700 1 --2- 172.21.15.16:0/2500273822 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdd00130430 0x7fdd001cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:41.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.873+0000 7fdcfe59c700 1 -- 172.21.15.16:0/2500273822 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdd00130430 msgr2=0x7fdd001cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:41.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.873+0000 7fdcfe59c700 1 --2- 172.21.15.16:0/2500273822 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdd00130430 0x7fdd001cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:41.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.873+0000 7fdcfe59c700 1 -- 172.21.15.16:0/2500273822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd0012f8b0 msgr2=0x7fdd001cd020 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:31:41.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.873+0000 7fdcfe59c700 1 --2- 172.21.15.16:0/2500273822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd0012f8b0 0x7fdd001cd020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:41.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.873+0000 7fdcfe59c700 1 -- 172.21.15.16:0/2500273822 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf000a6f0 con 0x7fdd0012cf60 2023-09-30T22:31:41.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.873+0000 7fdcfe59c700 1 --2- 172.21.15.16:0/2500273822 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdd0012cf60 0x7fdd001ccae0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0000c00 tx=0x7fdcf0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:41.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.873+0000 7fdceeffd700 1 -- 172.21.15.16:0/2500273822 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf00132c0 con 0x7fdd0012cf60 2023-09-30T22:31:41.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.873+0000 7fdceeffd700 1 -- 172.21.15.16:0/2500273822 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcf001f070 con 0x7fdd0012cf60 2023-09-30T22:31:41.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.873+0000 7fdceeffd700 1 -- 172.21.15.16:0/2500273822 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf0013680 con 0x7fdd0012cf60 2023-09-30T22:31:41.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.873+0000 7fdd0490d700 1 -- 172.21.15.16:0/2500273822 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd00129fa0 con 0x7fdd0012cf60 2023-09-30T22:31:41.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.873+0000 7fdd0490d700 1 -- 172.21.15.16:0/2500273822 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd0009e3e0 con 0x7fdd0012cf60 2023-09-30T22:31:41.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.874+0000 7fdd0490d700 1 -- 172.21.15.16:0/2500273822 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd000a23b0 con 0x7fdd0012cf60 2023-09-30T22:31:41.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.874+0000 7fdceeffd700 1 -- 172.21.15.16:0/2500273822 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdcf0021030 con 0x7fdd0012cf60 2023-09-30T22:31:41.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.875+0000 7fdceeffd700 1 -- 172.21.15.16:0/2500273822 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fdcf00403a0 con 0x7fdd0012cf60 2023-09-30T22:31:41.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.881+0000 7ff277fff700 1 -- 172.21.15.16:0/200035340 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff28801c2c0 con 0x7ff290134170 2023-09-30T22:31:41.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.884+0000 7fdceeffd700 1 -- 172.21.15.16:0/2500273822 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdcf00239c0 con 0x7fdd0012cf60 2023-09-30T22:31:41.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.966+0000 7efbfbee5700 1 -- 172.21.15.16:0/2866141508 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7efbf4121fb0 con 0x7efbf409de50 2023-09-30T22:31:41.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.966+0000 7efbea7fc700 1 -- 172.21.15.16:0/2866141508 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7efbf4121fb0 con 0x7efbf409de50 2023-09-30T22:31:41.967 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:31:41.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.970+0000 7efbfbee5700 1 -- 172.21.15.16:0/2866141508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efbf409de50 msgr2=0x7efbf411eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:41.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.970+0000 7efbfbee5700 1 --2- 172.21.15.16:0/2866141508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efbf409de50 0x7efbf411eba0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7efbf00090b0 tx=0x7efbf0004750 comp rx=0 tx=0).stop 2023-09-30T22:31:41.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.970+0000 7efbfbee5700 1 -- 172.21.15.16:0/2866141508 shutdown_connections 2023-09-30T22:31:41.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.970+0000 7efbfbee5700 1 --2- 172.21.15.16:0/2866141508 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efbf41338a0 0x7efbf411f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:41.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.970+0000 7efbfbee5700 1 --2- 172.21.15.16:0/2866141508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efbf409de50 0x7efbf411eba0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:41.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.970+0000 7efbfbee5700 1 --2- 172.21.15.16:0/2866141508 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efbf409e760 0x7efbf411f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:41.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.970+0000 7efbfbee5700 1 -- 172.21.15.16:0/2866141508 >> 172.21.15.16:0/2866141508 conn(0x7efbf40a2aa0 msgr2=0x7efbf409ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:41.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.970+0000 7efbfbee5700 1 -- 172.21.15.16:0/2866141508 shutdown_connections 2023-09-30T22:31:41.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:41.970+0000 7efbfbee5700 1 -- 172.21.15.16:0/2866141508 wait complete. 2023-09-30T22:31:41.989 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:31:42.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:42.049+0000 7f42aa926700 1 -- 172.21.15.16:0/3538154256 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f42a4074ea0 con 0x7f42a409d6f0 2023-09-30T22:31:42.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:42.049+0000 7f42a0ff9700 1 -- 172.21.15.16:0/3538154256 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f429400bb50 con 0x7f42a409d6f0 2023-09-30T22:31:42.050 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:31:42.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:42.051+0000 7f42aa926700 1 -- 172.21.15.16:0/3538154256 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f42a409d6f0 msgr2=0x7f42a411e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:42.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:42.051+0000 7f42aa926700 1 --2- 172.21.15.16:0/3538154256 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f42a409d6f0 0x7f42a411e0b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f4294009fd0 tx=0x7f4294004750 comp rx=0 tx=0).stop 2023-09-30T22:31:42.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:42.051+0000 7f42aa926700 1 -- 172.21.15.16:0/3538154256 shutdown_connections 2023-09-30T22:31:42.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:42.051+0000 7f42aa926700 1 --2- 172.21.15.16:0/3538154256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42a409e410 0x7f42a411eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:42.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:42.051+0000 7f42aa926700 1 --2- 172.21.15.16:0/3538154256 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f42a409d6f0 0x7f42a411e0b0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:42.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:42.051+0000 7f42aa926700 1 --2- 172.21.15.16:0/3538154256 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f42a409e000 0x7f42a411e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:42.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:42.051+0000 7f42aa926700 1 -- 172.21.15.16:0/3538154256 >> 172.21.15.16:0/3538154256 conn(0x7f42a4066f50 msgr2=0x7f42a40a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:42.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:42.051+0000 7f42aa926700 1 -- 172.21.15.16:0/3538154256 shutdown_connections 2023-09-30T22:31:42.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:42.052+0000 7f42aa926700 1 -- 172.21.15.16:0/3538154256 wait complete. 2023-09-30T22:31:42.066 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:31:42.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:42.089+0000 7ff296056700 1 -- 172.21.15.16:0/200035340 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff2900a28e0 con 0x7ff290134170 2023-09-30T22:31:42.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:42.090+0000 7ff277fff700 1 -- 172.21.15.16:0/200035340 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff28801a030 con 0x7ff290134170 2023-09-30T22:31:42.090 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:31:42.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:42.092+0000 7ff296056700 1 -- 172.21.15.16:0/200035340 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff290134170 msgr2=0x7ff2901232f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:42.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:42.092+0000 7ff296056700 1 --2- 172.21.15.16:0/200035340 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff290134170 0x7ff2901232f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7ff288009fa0 tx=0x7ff28800d460 comp rx=0 tx=0).stop 2023-09-30T22:31:42.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:42.092+0000 7ff296056700 1 -- 172.21.15.16:0/200035340 shutdown_connections 2023-09-30T22:31:42.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:42.092+0000 7ff296056700 1 --2- 172.21.15.16:0/200035340 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff290134170 0x7ff2901232f0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:42.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:42.092+0000 7ff296056700 1 --2- 172.21.15.16:0/200035340 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff2900a0a30 0x7ff290122870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:42.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:42.092+0000 7ff296056700 1 --2- 172.21.15.16:0/200035340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2900a1340 0x7ff290122db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:42.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:42.092+0000 7ff296056700 1 -- 172.21.15.16:0/200035340 >> 172.21.15.16:0/200035340 conn(0x7ff2900a6550 msgr2=0x7ff29009edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:42.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:42.092+0000 7ff296056700 1 -- 172.21.15.16:0/200035340 shutdown_connections 2023-09-30T22:31:42.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:42.092+0000 7ff296056700 1 -- 172.21.15.16:0/200035340 wait complete. 2023-09-30T22:31:42.106 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:31:42.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:42.120+0000 7fdd0490d700 1 -- 172.21.15.16:0/2500273822 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fdd00123a70 con 0x7fdd0012cf60 2023-09-30T22:31:42.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:42.121+0000 7fdceeffd700 1 -- 172.21.15.16:0/2500273822 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdcf00233d0 con 0x7fdd0012cf60 2023-09-30T22:31:42.121 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:31:42.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:42.123+0000 7fdd0490d700 1 -- 172.21.15.16:0/2500273822 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdd0012cf60 msgr2=0x7fdd001ccae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:42.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:42.123+0000 7fdd0490d700 1 --2- 172.21.15.16:0/2500273822 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdd0012cf60 0x7fdd001ccae0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0000c00 tx=0x7fdcf0004750 comp rx=0 tx=0).stop 2023-09-30T22:31:42.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:42.123+0000 7fdd0490d700 1 -- 172.21.15.16:0/2500273822 shutdown_connections 2023-09-30T22:31:42.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:42.123+0000 7fdd0490d700 1 --2- 172.21.15.16:0/2500273822 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdd00130430 0x7fdd001cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:42.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:42.123+0000 7fdd0490d700 1 --2- 172.21.15.16:0/2500273822 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdd0012cf60 0x7fdd001ccae0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:42.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:42.123+0000 7fdd0490d700 1 --2- 172.21.15.16:0/2500273822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd0012f8b0 0x7fdd001cd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:42.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:42.123+0000 7fdd0490d700 1 -- 172.21.15.16:0/2500273822 >> 172.21.15.16:0/2500273822 conn(0x7fdd000a2aa0 msgr2=0x7fdd0006e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:42.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:42.123+0000 7fdd0490d700 1 -- 172.21.15.16:0/2500273822 shutdown_connections 2023-09-30T22:31:42.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:42.123+0000 7fdd0490d700 1 -- 172.21.15.16:0/2500273822 wait complete. 2023-09-30T22:31:42.137 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:31:42.990 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:43.066 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:43.106 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:43.138 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:43.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.139+0000 7f31173a5700 1 Processor -- start 2023-09-30T22:31:43.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.139+0000 7f31173a5700 1 -- start start 2023-09-30T22:31:43.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.140+0000 7f31173a5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f311006a770 0x7f311006ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:43.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.140+0000 7f31173a5700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f311006b080 0x7f3110131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:43.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.140+0000 7f31173a5700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f311009ece0 con 0x7f3110131b40 2023-09-30T22:31:43.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.140+0000 7f31173a5700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f311009e760 con 0x7f311006b080 2023-09-30T22:31:43.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.140+0000 7f31173a5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f311009e040 con 0x7f311006a770 2023-09-30T22:31:43.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.140+0000 7f3115141700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f311006a770 0x7f311006ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:43.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.140+0000 7f3115141700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f311006a770 0x7f311006ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:39872/0 (socket says 172.21.15.16:39872) 2023-09-30T22:31:43.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.140+0000 7f3115141700 1 -- 172.21.15.16:0/3263016242 learned_addr learned my addr 172.21.15.16:0/3263016242 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:43.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.140+0000 7f3114940700 1 --2- 172.21.15.16:0/3263016242 >> v2:172.21.15.16:3300/0 conn(0x7f311006b080 0x7f3110131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:43.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.141+0000 7f3107fff700 1 -- 172.21.15.16:0/3263016242 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4046993101 0 0) 0x7f311009ece0 con 0x7f3110131b40 2023-09-30T22:31:43.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.141+0000 7f3107fff700 1 -- 172.21.15.16:0/3263016242 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30fc0035b0 con 0x7f3110131b40 2023-09-30T22:31:43.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.141+0000 7f3114940700 1 -- 172.21.15.16:0/3263016242 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f311006a770 msgr2=0x7f311006ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:43.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.141+0000 7f3114940700 1 --2- 172.21.15.16:0/3263016242 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f311006a770 0x7f311006ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:43.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.141+0000 7f3114940700 1 -- 172.21.15.16:0/3263016242 >> v1:172.21.15.16:6789/0 conn(0x7f3110131b40 legacy=0x7f3110138850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:43.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.141+0000 7f3114940700 1 -- 172.21.15.16:0/3263016242 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f311009f630 con 0x7f311006b080 2023-09-30T22:31:43.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.141+0000 7f3115141700 1 --2- 172.21.15.16:0/3263016242 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f311006a770 0x7f311006ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:43.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.141+0000 7f3114940700 1 --2- 172.21.15.16:0/3263016242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f311006b080 0x7f3110131600 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f3100009840 tx=0x7f3100009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82ce70dc736f854f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:43.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.141+0000 7f3107fff700 1 -- 172.21.15.16:0/3263016242 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3100011830 con 0x7f311006b080 2023-09-30T22:31:43.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.141+0000 7f3107fff700 1 -- 172.21.15.16:0/3263016242 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3100011990 con 0x7f311006b080 2023-09-30T22:31:43.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.141+0000 7f3107fff700 1 -- 172.21.15.16:0/3263016242 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3100011d20 con 0x7f311006b080 2023-09-30T22:31:43.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.142+0000 7f31173a5700 1 -- 172.21.15.16:0/3263016242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f311006b080 msgr2=0x7f3110131600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:43.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.142+0000 7f31173a5700 1 --2- 172.21.15.16:0/3263016242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f311006b080 0x7f3110131600 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f3100009840 tx=0x7f3100009b50 comp rx=0 tx=0).stop 2023-09-30T22:31:43.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.143+0000 7f31173a5700 1 -- 172.21.15.16:0/3263016242 shutdown_connections 2023-09-30T22:31:43.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.143+0000 7f31173a5700 1 --2- 172.21.15.16:0/3263016242 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f311006a770 0x7f311006ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:43.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.143+0000 7f31173a5700 1 --2- 172.21.15.16:0/3263016242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f311006b080 0x7f3110131600 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:43.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.143+0000 7f31173a5700 1 -- 172.21.15.16:0/3263016242 >> 172.21.15.16:0/3263016242 conn(0x7f3110066f50 msgr2=0x7f311009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:43.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.144+0000 7f31173a5700 1 -- 172.21.15.16:0/3263016242 shutdown_connections 2023-09-30T22:31:43.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.144+0000 7f31173a5700 1 -- 172.21.15.16:0/3263016242 wait complete. 2023-09-30T22:31:43.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.145+0000 7f31173a5700 1 Processor -- start 2023-09-30T22:31:43.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.145+0000 7f31173a5700 1 -- start start 2023-09-30T22:31:43.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.146+0000 7f31173a5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f311006a770 0x7f311009ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:43.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.146+0000 7f31173a5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f311006b080 0x7f311009b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:43.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.146+0000 7f31173a5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3110131b40 0x7f31101c8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:43.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.146+0000 7f31173a5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31101253a0 con 0x7f311006a770 2023-09-30T22:31:43.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.146+0000 7f31173a5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f311009a750 con 0x7f311006b080 2023-09-30T22:31:43.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.146+0000 7f31173a5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3110131310 con 0x7f3110131b40 2023-09-30T22:31:43.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.146+0000 7f3115942700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3110131b40 0x7f31101c8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:43.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.146+0000 7f3115942700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3110131b40 0x7f31101c8060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:39888/0 (socket says 172.21.15.16:39888) 2023-09-30T22:31:43.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.146+0000 7f3115942700 1 -- 172.21.15.16:0/3477292107 learned_addr learned my addr 172.21.15.16:0/3477292107 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:43.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.146+0000 7f3115141700 1 --2- 172.21.15.16:0/3477292107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f311006a770 0x7f311009ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:43.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.147+0000 7f3114940700 1 --2- 172.21.15.16:0/3477292107 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f311006b080 0x7f311009b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:43.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.147+0000 7f3115141700 1 -- 172.21.15.16:0/3477292107 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3110131b40 msgr2=0x7f31101c8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:43.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.147+0000 7f3115141700 1 --2- 172.21.15.16:0/3477292107 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3110131b40 0x7f31101c8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:43.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.147+0000 7f3115141700 1 -- 172.21.15.16:0/3477292107 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f311006b080 msgr2=0x7f311009b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:43.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.147+0000 7f3115141700 1 --2- 172.21.15.16:0/3477292107 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f311006b080 0x7f311009b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:43.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.147+0000 7f3115141700 1 -- 172.21.15.16:0/3477292107 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f310c009550 con 0x7f311006a770 2023-09-30T22:31:43.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.147+0000 7f3115942700 1 --2- 172.21.15.16:0/3477292107 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3110131b40 0x7f31101c8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:43.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.147+0000 7f3114940700 1 --2- 172.21.15.16:0/3477292107 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f311006b080 0x7f311009b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:31:43.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.147+0000 7f3115141700 1 --2- 172.21.15.16:0/3477292107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f311006a770 0x7f311009ad90 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f310c00e9a0 tx=0x7f310c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:43.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.147+0000 7f3105ffb700 1 -- 172.21.15.16:0/3477292107 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310c004030 con 0x7f311006a770 2023-09-30T22:31:43.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.147+0000 7f3105ffb700 1 -- 172.21.15.16:0/3477292107 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f310c004190 con 0x7f311006a770 2023-09-30T22:31:43.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.147+0000 7f31173a5700 1 -- 172.21.15.16:0/3477292107 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31000094a0 con 0x7f311006a770 2023-09-30T22:31:43.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.147+0000 7f3105ffb700 1 -- 172.21.15.16:0/3477292107 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310c004520 con 0x7f311006a770 2023-09-30T22:31:43.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.148+0000 7f31173a5700 1 -- 172.21.15.16:0/3477292107 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3110125db0 con 0x7f311006a770 2023-09-30T22:31:43.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.149+0000 7f3105ffb700 1 -- 172.21.15.16:0/3477292107 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f310c00c850 con 0x7f311006a770 2023-09-30T22:31:43.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.149+0000 7f3105ffb700 1 -- 172.21.15.16:0/3477292107 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f310c03fa50 con 0x7f311006a770 2023-09-30T22:31:43.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.149+0000 7f31173a5700 1 -- 172.21.15.16:0/3477292107 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3110074ea0 con 0x7f311006a770 2023-09-30T22:31:43.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.154+0000 7f3105ffb700 1 -- 172.21.15.16:0/3477292107 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f310c020080 con 0x7f311006a770 2023-09-30T22:31:43.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.212+0000 7f45b5fa3700 1 Processor -- start 2023-09-30T22:31:43.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.212+0000 7f45b5fa3700 1 -- start start 2023-09-30T22:31:43.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.213+0000 7f45b5fa3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f45b0131980 0x7f45b0131d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:43.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.213+0000 7f45b5fa3700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f45b0099a40 0x7f45b0099e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:43.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.213+0000 7f45b5fa3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f45b009a370 0x7f45b0138650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:43.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.213+0000 7f45b5fa3700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45b0131320 con 0x7f45b0099a40 2023-09-30T22:31:43.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.213+0000 7f45b5fa3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45b009abf0 con 0x7f45b0131980 2023-09-30T22:31:43.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.213+0000 7f45b5fa3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f45b0131550 con 0x7f45b009a370 2023-09-30T22:31:43.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.213+0000 7f45af7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f45b0131980 0x7f45b0131d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:43.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.214+0000 7f45af7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f45b0131980 0x7f45b0131d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:40588/0 (socket says 172.21.15.16:40588) 2023-09-30T22:31:43.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.214+0000 7f45af7fe700 1 -- 172.21.15.16:0/344673984 learned_addr learned my addr 172.21.15.16:0/344673984 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:43.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.214+0000 7f45affff700 1 --2- 172.21.15.16:0/344673984 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f45b009a370 0x7f45b0138650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:43.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.214+0000 7f45aeffd700 1 --2- 172.21.15.16:0/344673984 >> v2:172.21.15.16:3300/0 conn(0x7f45b0099a40 0x7f45b0099e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:43.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.214+0000 7f45aeffd700 1 -- 172.21.15.16:0/344673984 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f45b009a370 msgr2=0x7f45b0138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:43.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.214+0000 7f45aeffd700 1 --2- 172.21.15.16:0/344673984 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f45b009a370 0x7f45b0138650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:43.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.214+0000 7f45aeffd700 1 -- 172.21.15.16:0/344673984 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f45b0131980 msgr2=0x7f45b0131d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:43.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.214+0000 7f45aeffd700 1 --2- 172.21.15.16:0/344673984 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f45b0131980 0x7f45b0131d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:43.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.214+0000 7f45aeffd700 1 -- 172.21.15.16:0/344673984 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b006f140 con 0x7f45b0099a40 2023-09-30T22:31:43.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.214+0000 7f45affff700 1 --2- 172.21.15.16:0/344673984 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f45b009a370 0x7f45b0138650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:43.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.214+0000 7f45aeffd700 1 --2- 172.21.15.16:0/344673984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45b0099a40 0x7f45b0099e30 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f45a4009840 tx=0x7f45a4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0640a66ce40caf6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:43.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.214+0000 7f45ae7fc700 1 -- 172.21.15.16:0/344673984 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a4011830 con 0x7f45b0099a40 2023-09-30T22:31:43.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.214+0000 7f45ae7fc700 1 -- 172.21.15.16:0/344673984 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45a4011990 con 0x7f45b0099a40 2023-09-30T22:31:43.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.214+0000 7f45ae7fc700 1 -- 172.21.15.16:0/344673984 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a4011d20 con 0x7f45b0099a40 2023-09-30T22:31:43.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.216+0000 7f45b5fa3700 1 -- 172.21.15.16:0/344673984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45b0099a40 msgr2=0x7f45b0099e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:43.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.216+0000 7f45b5fa3700 1 --2- 172.21.15.16:0/344673984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45b0099a40 0x7f45b0099e30 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f45a4009840 tx=0x7f45a4009b50 comp rx=0 tx=0).stop 2023-09-30T22:31:43.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.216+0000 7f45b5fa3700 1 -- 172.21.15.16:0/344673984 shutdown_connections 2023-09-30T22:31:43.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.216+0000 7f45b5fa3700 1 --2- 172.21.15.16:0/344673984 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f45b0131980 0x7f45b0131d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:43.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.216+0000 7f45b5fa3700 1 --2- 172.21.15.16:0/344673984 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f45b009a370 0x7f45b0138650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:43.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.216+0000 7f45b5fa3700 1 --2- 172.21.15.16:0/344673984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45b0099a40 0x7f45b0099e30 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:43.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.216+0000 7f45b5fa3700 1 -- 172.21.15.16:0/344673984 >> 172.21.15.16:0/344673984 conn(0x7f45b00a2aa0 msgr2=0x7f45b00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:43.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.216+0000 7f45b5fa3700 1 -- 172.21.15.16:0/344673984 shutdown_connections 2023-09-30T22:31:43.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.216+0000 7f45b5fa3700 1 -- 172.21.15.16:0/344673984 wait complete. 2023-09-30T22:31:43.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.217+0000 7f45b5fa3700 1 Processor -- start 2023-09-30T22:31:43.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.218+0000 7f45b5fa3700 1 -- start start 2023-09-30T22:31:43.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.218+0000 7f45b5fa3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f45b0099a40 0x7f45b011d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:43.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.218+0000 7f45b5fa3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45b009a370 0x7f45b011dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:43.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.218+0000 7f45b5fa3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f45b0131980 0x7f45b011e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:43.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.218+0000 7f45b5fa3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b012eb90 con 0x7f45b009a370 2023-09-30T22:31:43.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.218+0000 7f45b5fa3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45b0120260 con 0x7f45b0131980 2023-09-30T22:31:43.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.218+0000 7f45b5fa3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f45b0120d40 con 0x7f45b0099a40 2023-09-30T22:31:43.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.219+0000 7f45af7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f45b0099a40 0x7f45b011d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:43.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.219+0000 7f45affff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f45b0131980 0x7f45b011e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:43.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.219+0000 7f45af7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f45b0099a40 0x7f45b011d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:39898/0 (socket says 172.21.15.16:39898) 2023-09-30T22:31:43.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.219+0000 7f45affff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f45b0131980 0x7f45b011e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:40600/0 (socket says 172.21.15.16:40600) 2023-09-30T22:31:43.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.219+0000 7f45af7fe700 1 -- 172.21.15.16:0/3921201575 learned_addr learned my addr 172.21.15.16:0/3921201575 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:43.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.219+0000 7f45aeffd700 1 --2- 172.21.15.16:0/3921201575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45b009a370 0x7f45b011dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:43.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.219+0000 7f45aeffd700 1 -- 172.21.15.16:0/3921201575 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f45b0099a40 msgr2=0x7f45b011d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:43.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.219+0000 7f45aeffd700 1 --2- 172.21.15.16:0/3921201575 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f45b0099a40 0x7f45b011d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:43.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.219+0000 7f45aeffd700 1 -- 172.21.15.16:0/3921201575 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f45b0131980 msgr2=0x7f45b011e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:43.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.219+0000 7f45aeffd700 1 --2- 172.21.15.16:0/3921201575 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f45b0131980 0x7f45b011e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:43.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.219+0000 7f45aeffd700 1 -- 172.21.15.16:0/3921201575 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a8009a70 con 0x7f45b009a370 2023-09-30T22:31:43.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.219+0000 7f45affff700 1 --2- 172.21.15.16:0/3921201575 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f45b0131980 0x7f45b011e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:31:43.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.219+0000 7f45aeffd700 1 --2- 172.21.15.16:0/3921201575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45b009a370 0x7f45b011dd50 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f45a4016000 tx=0x7f45a4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:43.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.219+0000 7f4597fff700 1 -- 172.21.15.16:0/3921201575 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a4004450 con 0x7f45b009a370 2023-09-30T22:31:43.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.220+0000 7f4597fff700 1 -- 172.21.15.16:0/3921201575 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45a401f070 con 0x7f45b009a370 2023-09-30T22:31:43.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.220+0000 7f45b5fa3700 1 -- 172.21.15.16:0/3921201575 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45a40094a0 con 0x7f45b009a370 2023-09-30T22:31:43.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.220+0000 7f4597fff700 1 -- 172.21.15.16:0/3921201575 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a40038e0 con 0x7f45b009a370 2023-09-30T22:31:43.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.220+0000 7f45b5fa3700 1 -- 172.21.15.16:0/3921201575 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45b0121050 con 0x7f45b009a370 2023-09-30T22:31:43.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.221+0000 7f4597fff700 1 -- 172.21.15.16:0/3921201575 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f45a4003a40 con 0x7f45b009a370 2023-09-30T22:31:43.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.221+0000 7f45b5fa3700 1 -- 172.21.15.16:0/3921201575 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45b0120620 con 0x7f45b009a370 2023-09-30T22:31:43.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.221+0000 7f4597fff700 1 -- 172.21.15.16:0/3921201575 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f45a4021070 con 0x7f45b009a370 2023-09-30T22:31:43.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.230+0000 7f4597fff700 1 -- 172.21.15.16:0/3921201575 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f45a40045b0 con 0x7f45b009a370 2023-09-30T22:31:43.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.262+0000 7f72af9e5700 1 Processor -- start 2023-09-30T22:31:43.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.262+0000 7f72af9e5700 1 -- start start 2023-09-30T22:31:43.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.263+0000 7f72af9e5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72a809d6f0 0x7f72a809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:43.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.263+0000 7f72af9e5700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f72a809e000 0x7f72a81339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:43.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.263+0000 7f72af9e5700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72a809b120 con 0x7f72a809e410 2023-09-30T22:31:43.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.263+0000 7f72af9e5700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72a806b460 con 0x7f72a809e000 2023-09-30T22:31:43.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.263+0000 7f72af9e5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72a809a7e0 con 0x7f72a809d6f0 2023-09-30T22:31:43.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.263+0000 7f72ad781700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72a809d6f0 0x7f72a809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:43.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.263+0000 7f72ad781700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72a809d6f0 0x7f72a809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:40610/0 (socket says 172.21.15.16:40610) 2023-09-30T22:31:43.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.263+0000 7f72ad781700 1 -- 172.21.15.16:0/1965925957 learned_addr learned my addr 172.21.15.16:0/1965925957 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:43.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.263+0000 7f72acf80700 1 --2- 172.21.15.16:0/1965925957 >> v2:172.21.15.16:3300/0 conn(0x7f72a809e000 0x7f72a81339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:43.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.263+0000 7f72ad781700 1 -- 172.21.15.16:0/1965925957 >> v2:172.21.15.16:3300/0 conn(0x7f72a809e000 msgr2=0x7f72a81339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:43.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.263+0000 7f72ad781700 1 --2- 172.21.15.16:0/1965925957 >> v2:172.21.15.16:3300/0 conn(0x7f72a809e000 0x7f72a81339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:43.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.263+0000 7f72ad781700 1 -- 172.21.15.16:0/1965925957 >> v1:172.21.15.16:6789/0 conn(0x7f72a809e410 legacy=0x7f72a8133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:43.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.263+0000 7f72ad781700 1 -- 172.21.15.16:0/1965925957 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72a809a500 con 0x7f72a809d6f0 2023-09-30T22:31:43.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.263+0000 7f72ad781700 1 --2- 172.21.15.16:0/1965925957 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72a809d6f0 0x7f72a809dac0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f72a4009920 tx=0x7f72a4009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c50c9bf1e97502eb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:43.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.264+0000 7f729ffff700 1 -- 172.21.15.16:0/1965925957 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a4011830 con 0x7f72a809d6f0 2023-09-30T22:31:43.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.264+0000 7f729ffff700 1 -- 172.21.15.16:0/1965925957 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72a4011990 con 0x7f72a809d6f0 2023-09-30T22:31:43.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.264+0000 7f729ffff700 1 -- 172.21.15.16:0/1965925957 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a4011d20 con 0x7f72a809d6f0 2023-09-30T22:31:43.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.264+0000 7f72af9e5700 1 -- 172.21.15.16:0/1965925957 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72a809d6f0 msgr2=0x7f72a809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:43.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.264+0000 7f72af9e5700 1 --2- 172.21.15.16:0/1965925957 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72a809d6f0 0x7f72a809dac0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f72a4009920 tx=0x7f72a4009c30 comp rx=0 tx=0).stop 2023-09-30T22:31:43.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.264+0000 7f72af9e5700 1 -- 172.21.15.16:0/1965925957 shutdown_connections 2023-09-30T22:31:43.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.264+0000 7f72af9e5700 1 --2- 172.21.15.16:0/1965925957 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72a809d6f0 0x7f72a809dac0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:43.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.264+0000 7f72af9e5700 1 --2- 172.21.15.16:0/1965925957 >> v2:172.21.15.16:3300/0 conn(0x7f72a809e000 0x7f72a81339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:43.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.264+0000 7f72af9e5700 1 -- 172.21.15.16:0/1965925957 >> 172.21.15.16:0/1965925957 conn(0x7f72a8066f50 msgr2=0x7f72a809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:43.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.265+0000 7f72af9e5700 1 -- 172.21.15.16:0/1965925957 shutdown_connections 2023-09-30T22:31:43.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.265+0000 7f72af9e5700 1 -- 172.21.15.16:0/1965925957 wait complete. 2023-09-30T22:31:43.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.266+0000 7f72af9e5700 1 Processor -- start 2023-09-30T22:31:43.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.266+0000 7f72af9e5700 1 -- start start 2023-09-30T22:31:43.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.266+0000 7f72af9e5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72a809d6f0 0x7f72a81a5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:43.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.266+0000 7f72af9e5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72a809e000 0x7f72a81a6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:43.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.266+0000 7f72af9e5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72a809e410 0x7f72a81a6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:43.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.266+0000 7f72af9e5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a81aa270 con 0x7f72a809d6f0 2023-09-30T22:31:43.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.266+0000 7f72af9e5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72a81a87a0 con 0x7f72a809e410 2023-09-30T22:31:43.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.266+0000 7f72af9e5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f72a81a7a40 con 0x7f72a809e000 2023-09-30T22:31:43.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.266+0000 7f72acf80700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72a809e000 0x7f72a81a6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:43.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.266+0000 7f72acf80700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72a809e000 0x7f72a81a6400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:39900/0 (socket says 172.21.15.16:39900) 2023-09-30T22:31:43.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.266+0000 7f72acf80700 1 -- 172.21.15.16:0/1228552911 learned_addr learned my addr 172.21.15.16:0/1228552911 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:43.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.266+0000 7f72ad781700 1 --2- 172.21.15.16:0/1228552911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72a809d6f0 0x7f72a81a5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:43.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.267+0000 7f72adf82700 1 --2- 172.21.15.16:0/1228552911 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72a809e410 0x7f72a81a6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:43.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.267+0000 7f72acf80700 1 -- 172.21.15.16:0/1228552911 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72a809e410 msgr2=0x7f72a81a6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:43.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.267+0000 7f72acf80700 1 --2- 172.21.15.16:0/1228552911 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72a809e410 0x7f72a81a6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:43.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.267+0000 7f72acf80700 1 -- 172.21.15.16:0/1228552911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72a809d6f0 msgr2=0x7f72a81a5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:43.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.267+0000 7f72acf80700 1 --2- 172.21.15.16:0/1228552911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72a809d6f0 0x7f72a81a5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:43.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.267+0000 7f72acf80700 1 -- 172.21.15.16:0/1228552911 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72a4009550 con 0x7f72a809e000 2023-09-30T22:31:43.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.267+0000 7f72acf80700 1 --2- 172.21.15.16:0/1228552911 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72a809e000 0x7f72a81a6400 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f729800e9b0 tx=0x7f729800ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:43.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.267+0000 7f729dffb700 1 -- 172.21.15.16:0/1228552911 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7298009710 con 0x7f72a809e000 2023-09-30T22:31:43.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.267+0000 7f729dffb700 1 -- 172.21.15.16:0/1228552911 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f729800c850 con 0x7f72a809e000 2023-09-30T22:31:43.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.267+0000 7f729dffb700 1 -- 172.21.15.16:0/1228552911 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729800cbe0 con 0x7f72a809e000 2023-09-30T22:31:43.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.267+0000 7f72af9e5700 1 -- 172.21.15.16:0/1228552911 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72a80a3190 con 0x7f72a809e000 2023-09-30T22:31:43.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.268+0000 7f72af9e5700 1 -- 172.21.15.16:0/1228552911 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72a81a98b0 con 0x7f72a809e000 2023-09-30T22:31:43.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.269+0000 7f72af9e5700 1 -- 172.21.15.16:0/1228552911 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72a8074ea0 con 0x7f72a809e000 2023-09-30T22:31:43.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.270+0000 7f729dffb700 1 -- 172.21.15.16:0/1228552911 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f729800cd40 con 0x7f72a809e000 2023-09-30T22:31:43.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.270+0000 7f729dffb700 1 -- 172.21.15.16:0/1228552911 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f729803fff0 con 0x7f72a809e000 2023-09-30T22:31:43.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.277+0000 7f729dffb700 1 -- 172.21.15.16:0/1228552911 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7298011350 con 0x7f72a809e000 2023-09-30T22:31:43.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.297+0000 7f29593b0700 1 Processor -- start 2023-09-30T22:31:43.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.297+0000 7f29593b0700 1 -- start start 2023-09-30T22:31:43.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.297+0000 7f29593b0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f295409d6f0 0x7f295409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:43.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.297+0000 7f29593b0700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f295409e000 0x7f29541339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:43.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.297+0000 7f29593b0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2954133f30 0x7f29541343b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:43.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.297+0000 7f29593b0700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f295409b0f0 con 0x7f295409e000 2023-09-30T22:31:43.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.297+0000 7f29593b0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f295409d2f0 con 0x7f295409d6f0 2023-09-30T22:31:43.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.297+0000 7f29593b0700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f295409a810 con 0x7f2954133f30 2023-09-30T22:31:43.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.298+0000 7f29527fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f295409e000 0x7f29541339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:43.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.298+0000 7f29527fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f295409e000 0x7f29541339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48228/0 (socket says 172.21.15.16:48228) 2023-09-30T22:31:43.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.298+0000 7f29527fc700 1 -- 172.21.15.16:0/100111465 learned_addr learned my addr 172.21.15.16:0/100111465 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:43.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.298+0000 7f29537fe700 1 --2- 172.21.15.16:0/100111465 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2954133f30 0x7f29541343b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:43.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.298+0000 7f2952ffd700 1 --2- 172.21.15.16:0/100111465 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f295409d6f0 0x7f295409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:43.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.298+0000 7f29527fc700 1 -- 172.21.15.16:0/100111465 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2954133f30 msgr2=0x7f29541343b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:43.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.298+0000 7f29527fc700 1 --2- 172.21.15.16:0/100111465 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2954133f30 0x7f29541343b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:43.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.298+0000 7f29527fc700 1 -- 172.21.15.16:0/100111465 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f295409d6f0 msgr2=0x7f295409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:43.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.298+0000 7f29527fc700 1 --2- 172.21.15.16:0/100111465 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f295409d6f0 0x7f295409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:43.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.298+0000 7f29527fc700 1 -- 172.21.15.16:0/100111465 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f295409a470 con 0x7f295409e000 2023-09-30T22:31:43.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.298+0000 7f29527fc700 1 --2- 172.21.15.16:0/100111465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f295409e000 0x7f29541339f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f29480098f0 tx=0x7f2948009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e475180175257169 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:43.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.299+0000 7f2951ffb700 1 -- 172.21.15.16:0/100111465 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2948011830 con 0x7f295409e000 2023-09-30T22:31:43.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.299+0000 7f2951ffb700 1 -- 172.21.15.16:0/100111465 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2948011990 con 0x7f295409e000 2023-09-30T22:31:43.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.299+0000 7f2951ffb700 1 -- 172.21.15.16:0/100111465 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2948011d20 con 0x7f295409e000 2023-09-30T22:31:43.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.299+0000 7f29593b0700 1 -- 172.21.15.16:0/100111465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f295409e000 msgr2=0x7f29541339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:43.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.299+0000 7f29593b0700 1 --2- 172.21.15.16:0/100111465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f295409e000 0x7f29541339f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f29480098f0 tx=0x7f2948009c00 comp rx=0 tx=0).stop 2023-09-30T22:31:43.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.299+0000 7f29593b0700 1 -- 172.21.15.16:0/100111465 shutdown_connections 2023-09-30T22:31:43.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.299+0000 7f29593b0700 1 --2- 172.21.15.16:0/100111465 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f295409d6f0 0x7f295409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:43.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.299+0000 7f29593b0700 1 --2- 172.21.15.16:0/100111465 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2954133f30 0x7f29541343b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:43.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.299+0000 7f29593b0700 1 --2- 172.21.15.16:0/100111465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f295409e000 0x7f29541339f0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:43.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.299+0000 7f29593b0700 1 -- 172.21.15.16:0/100111465 >> 172.21.15.16:0/100111465 conn(0x7f2954066f50 msgr2=0x7f295409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:43.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.299+0000 7f29593b0700 1 -- 172.21.15.16:0/100111465 shutdown_connections 2023-09-30T22:31:43.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.299+0000 7f29593b0700 1 -- 172.21.15.16:0/100111465 wait complete. 2023-09-30T22:31:43.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.301+0000 7f29593b0700 1 Processor -- start 2023-09-30T22:31:43.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.301+0000 7f29593b0700 1 -- start start 2023-09-30T22:31:43.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.302+0000 7f29593b0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f295409d6f0 0x7f29541260f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:43.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.302+0000 7f29593b0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f295409e000 0x7f2954126630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:43.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.302+0000 7f29593b0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2954133f30 0x7f2954126b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:43.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.302+0000 7f29593b0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2954129910 con 0x7f295409d6f0 2023-09-30T22:31:43.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.302+0000 7f29593b0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2954129af0 con 0x7f295409e000 2023-09-30T22:31:43.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.302+0000 7f29593b0700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f29540a17d0 con 0x7f2954133f30 2023-09-30T22:31:43.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.302+0000 7f29527fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f295409e000 0x7f2954126630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:43.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.302+0000 7f29527fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f295409e000 0x7f2954126630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:40650/0 (socket says 172.21.15.16:40650) 2023-09-30T22:31:43.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.302+0000 7f29527fc700 1 -- 172.21.15.16:0/23915392 learned_addr learned my addr 172.21.15.16:0/23915392 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:43.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.302+0000 7f29537fe700 1 --2- 172.21.15.16:0/23915392 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2954133f30 0x7f2954126b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:43.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.302+0000 7f2952ffd700 1 --2- 172.21.15.16:0/23915392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f295409d6f0 0x7f29541260f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:43.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.303+0000 7f29527fc700 1 -- 172.21.15.16:0/23915392 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2954133f30 msgr2=0x7f2954126b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:43.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.303+0000 7f29527fc700 1 --2- 172.21.15.16:0/23915392 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2954133f30 0x7f2954126b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:43.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.303+0000 7f29527fc700 1 -- 172.21.15.16:0/23915392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f295409d6f0 msgr2=0x7f29541260f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:43.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.303+0000 7f29527fc700 1 --2- 172.21.15.16:0/23915392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f295409d6f0 0x7f29541260f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:43.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.303+0000 7f29527fc700 1 -- 172.21.15.16:0/23915392 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2948009550 con 0x7f295409e000 2023-09-30T22:31:43.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.303+0000 7f29527fc700 1 --2- 172.21.15.16:0/23915392 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f295409e000 0x7f2954126630 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2948000c00 tx=0x7f2948004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:43.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.303+0000 7f293b7fe700 1 -- 172.21.15.16:0/23915392 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2948004450 con 0x7f295409e000 2023-09-30T22:31:43.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.303+0000 7f293b7fe700 1 -- 172.21.15.16:0/23915392 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29480045b0 con 0x7f295409e000 2023-09-30T22:31:43.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.303+0000 7f29593b0700 1 -- 172.21.15.16:0/23915392 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2954127c80 con 0x7f295409e000 2023-09-30T22:31:43.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.303+0000 7f293b7fe700 1 -- 172.21.15.16:0/23915392 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29480038e0 con 0x7f295409e000 2023-09-30T22:31:43.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.303+0000 7f29593b0700 1 -- 172.21.15.16:0/23915392 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f295409c640 con 0x7f295409e000 2023-09-30T22:31:43.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.305+0000 7f29593b0700 1 -- 172.21.15.16:0/23915392 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f295409dc70 con 0x7f295409e000 2023-09-30T22:31:43.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.305+0000 7f293b7fe700 1 -- 172.21.15.16:0/23915392 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2948021070 con 0x7f295409e000 2023-09-30T22:31:43.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.305+0000 7f293b7fe700 1 -- 172.21.15.16:0/23915392 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f29480409e0 con 0x7f295409e000 2023-09-30T22:31:43.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.314+0000 7f293b7fe700 1 -- 172.21.15.16:0/23915392 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2948019770 con 0x7f295409e000 2023-09-30T22:31:43.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.373+0000 7f31173a5700 1 -- 172.21.15.16:0/3477292107 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f3110126130 con 0x7f311006a770 2023-09-30T22:31:43.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.373+0000 7f3105ffb700 1 -- 172.21.15.16:0/3477292107 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f310c0036a0 con 0x7f311006a770 2023-09-30T22:31:43.374 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:31:43.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.376+0000 7f31173a5700 1 -- 172.21.15.16:0/3477292107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f311006a770 msgr2=0x7f311009ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:43.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.376+0000 7f31173a5700 1 --2- 172.21.15.16:0/3477292107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f311006a770 0x7f311009ad90 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f310c00e9a0 tx=0x7f310c00ecb0 comp rx=0 tx=0).stop 2023-09-30T22:31:43.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.376+0000 7f31173a5700 1 -- 172.21.15.16:0/3477292107 shutdown_connections 2023-09-30T22:31:43.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.376+0000 7f31173a5700 1 --2- 172.21.15.16:0/3477292107 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3110131b40 0x7f31101c8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:43.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.376+0000 7f31173a5700 1 --2- 172.21.15.16:0/3477292107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f311006a770 0x7f311009ad90 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:43.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.376+0000 7f31173a5700 1 --2- 172.21.15.16:0/3477292107 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f311006b080 0x7f311009b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:43.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.376+0000 7f31173a5700 1 -- 172.21.15.16:0/3477292107 >> 172.21.15.16:0/3477292107 conn(0x7f3110066f50 msgr2=0x7f311006fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:43.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.376+0000 7f31173a5700 1 -- 172.21.15.16:0/3477292107 shutdown_connections 2023-09-30T22:31:43.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.376+0000 7f31173a5700 1 -- 172.21.15.16:0/3477292107 wait complete. 2023-09-30T22:31:43.390 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:31:43.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.456+0000 7f45b5fa3700 1 -- 172.21.15.16:0/3921201575 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f45b0121050 con 0x7f45b009a370 2023-09-30T22:31:43.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.456+0000 7f4597fff700 1 -- 172.21.15.16:0/3921201575 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f45b0121050 con 0x7f45b009a370 2023-09-30T22:31:43.457 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:31:43.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.459+0000 7f45b5fa3700 1 -- 172.21.15.16:0/3921201575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45b009a370 msgr2=0x7f45b011dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:43.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.459+0000 7f45b5fa3700 1 --2- 172.21.15.16:0/3921201575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45b009a370 0x7f45b011dd50 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f45a4016000 tx=0x7f45a4004750 comp rx=0 tx=0).stop 2023-09-30T22:31:43.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.459+0000 7f45b5fa3700 1 -- 172.21.15.16:0/3921201575 shutdown_connections 2023-09-30T22:31:43.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.459+0000 7f45b5fa3700 1 --2- 172.21.15.16:0/3921201575 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f45b0131980 0x7f45b011e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:43.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.459+0000 7f45b5fa3700 1 --2- 172.21.15.16:0/3921201575 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f45b0099a40 0x7f45b011d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:43.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.459+0000 7f45b5fa3700 1 --2- 172.21.15.16:0/3921201575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f45b009a370 0x7f45b011dd50 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:43.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.459+0000 7f45b5fa3700 1 -- 172.21.15.16:0/3921201575 >> 172.21.15.16:0/3921201575 conn(0x7f45b00a2aa0 msgr2=0x7f45b012f220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:43.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.459+0000 7f45b5fa3700 1 -- 172.21.15.16:0/3921201575 shutdown_connections 2023-09-30T22:31:43.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.459+0000 7f45b5fa3700 1 -- 172.21.15.16:0/3921201575 wait complete. 2023-09-30T22:31:43.473 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:31:43.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.488+0000 7f72af9e5700 1 -- 172.21.15.16:0/1228552911 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f72a806c360 con 0x7f72a809e000 2023-09-30T22:31:43.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.488+0000 7f729dffb700 1 -- 172.21.15.16:0/1228552911 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7298017030 con 0x7f72a809e000 2023-09-30T22:31:43.488 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:31:43.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.490+0000 7f72af9e5700 1 -- 172.21.15.16:0/1228552911 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72a809e000 msgr2=0x7f72a81a6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:43.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.490+0000 7f72af9e5700 1 --2- 172.21.15.16:0/1228552911 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72a809e000 0x7f72a81a6400 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f729800e9b0 tx=0x7f729800ecc0 comp rx=0 tx=0).stop 2023-09-30T22:31:43.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.490+0000 7f72af9e5700 1 -- 172.21.15.16:0/1228552911 shutdown_connections 2023-09-30T22:31:43.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.490+0000 7f72af9e5700 1 --2- 172.21.15.16:0/1228552911 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72a809e410 0x7f72a81a6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:43.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.490+0000 7f72af9e5700 1 --2- 172.21.15.16:0/1228552911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72a809d6f0 0x7f72a81a5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:43.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.490+0000 7f72af9e5700 1 --2- 172.21.15.16:0/1228552911 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72a809e000 0x7f72a81a6400 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:43.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.490+0000 7f72af9e5700 1 -- 172.21.15.16:0/1228552911 >> 172.21.15.16:0/1228552911 conn(0x7f72a8066f50 msgr2=0x7f72a81bad60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:43.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.490+0000 7f72af9e5700 1 -- 172.21.15.16:0/1228552911 shutdown_connections 2023-09-30T22:31:43.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.490+0000 7f72af9e5700 1 -- 172.21.15.16:0/1228552911 wait complete. 2023-09-30T22:31:43.504 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:31:43.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.521+0000 7f29593b0700 1 -- 172.21.15.16:0/23915392 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2954132ee0 con 0x7f295409e000 2023-09-30T22:31:43.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.521+0000 7f293b7fe700 1 -- 172.21.15.16:0/23915392 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2948029030 con 0x7f295409e000 2023-09-30T22:31:43.522 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:31:43.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.524+0000 7f29593b0700 1 -- 172.21.15.16:0/23915392 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f295409e000 msgr2=0x7f2954126630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:43.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.524+0000 7f29593b0700 1 --2- 172.21.15.16:0/23915392 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f295409e000 0x7f2954126630 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2948000c00 tx=0x7f2948004750 comp rx=0 tx=0).stop 2023-09-30T22:31:43.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.524+0000 7f29593b0700 1 -- 172.21.15.16:0/23915392 shutdown_connections 2023-09-30T22:31:43.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.524+0000 7f29593b0700 1 --2- 172.21.15.16:0/23915392 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2954133f30 0x7f2954126b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:43.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.524+0000 7f29593b0700 1 --2- 172.21.15.16:0/23915392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f295409d6f0 0x7f29541260f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:43.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.524+0000 7f29593b0700 1 --2- 172.21.15.16:0/23915392 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f295409e000 0x7f2954126630 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:43.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.524+0000 7f29593b0700 1 -- 172.21.15.16:0/23915392 >> 172.21.15.16:0/23915392 conn(0x7f2954066f50 msgr2=0x7f295411b490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:43.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.524+0000 7f29593b0700 1 -- 172.21.15.16:0/23915392 shutdown_connections 2023-09-30T22:31:43.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:43.524+0000 7f29593b0700 1 -- 172.21.15.16:0/23915392 wait complete. 2023-09-30T22:31:43.538 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:31:44.391 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:44.474 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:44.505 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:44.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.537+0000 7f1f0ce57700 1 Processor -- start 2023-09-30T22:31:44.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.537+0000 7f1f0ce57700 1 -- start start 2023-09-30T22:31:44.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.538+0000 7f1f0ce57700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f0809d6f0 0x7f1f0809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:44.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.538+0000 7f1f0ce57700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f0809e000 0x7f1f081339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:44.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.538+0000 7f1f0ce57700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f0809b120 con 0x7f1f0809e410 2023-09-30T22:31:44.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.538+0000 7f1f0ce57700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0806b460 con 0x7f1f0809e000 2023-09-30T22:31:44.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.538+0000 7f1f0ce57700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0809a7e0 con 0x7f1f0809d6f0 2023-09-30T22:31:44.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.538+0000 7f1f0659c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f0809d6f0 0x7f1f0809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:44.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.538+0000 7f1f05d9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f0809e000 0x7f1f081339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:44.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.538+0000 7f1f0659c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f0809d6f0 0x7f1f0809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:39920/0 (socket says 172.21.15.16:39920) 2023-09-30T22:31:44.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.538+0000 7f1f0659c700 1 -- 172.21.15.16:0/1768797259 learned_addr learned my addr 172.21.15.16:0/1768797259 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:44.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.538+0000 7f1f06d9d700 1 --1- 172.21.15.16:0/1768797259 >> v1:172.21.15.16:6789/0 conn(0x7f1f0809e410 0x7f1f08133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41378/0 (socket says 172.21.15.16:41378) 2023-09-30T22:31:44.539 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:44.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.538+0000 7f1f0659c700 1 -- 172.21.15.16:0/1768797259 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f0809e000 msgr2=0x7f1f081339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:44.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.538+0000 7f1f0659c700 1 --2- 172.21.15.16:0/1768797259 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f0809e000 0x7f1f081339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:44.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.538+0000 7f1f0659c700 1 -- 172.21.15.16:0/1768797259 >> v1:172.21.15.16:6789/0 conn(0x7f1f0809e410 legacy=0x7f1f08133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:44.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.538+0000 7f1f0559a700 1 -- 172.21.15.16:0/1768797259 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1563504338 0 0) 0x7f1f0809b120 con 0x7f1f0809e410 2023-09-30T22:31:44.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.538+0000 7f1f0659c700 1 -- 172.21.15.16:0/1768797259 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f0809a500 con 0x7f1f0809d6f0 2023-09-30T22:31:44.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.539+0000 7f1f0659c700 1 --2- 172.21.15.16:0/1768797259 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f0809d6f0 0x7f1f0809dac0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1ef8009950 tx=0x7f1ef8009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a0d03d1d0b90415 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:44.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.539+0000 7f1f0559a700 1 -- 172.21.15.16:0/1768797259 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef8011830 con 0x7f1f0809d6f0 2023-09-30T22:31:44.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.539+0000 7f1f0559a700 1 -- 172.21.15.16:0/1768797259 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ef8011990 con 0x7f1f0809d6f0 2023-09-30T22:31:44.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.539+0000 7f1f0559a700 1 -- 172.21.15.16:0/1768797259 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef8011d40 con 0x7f1f0809d6f0 2023-09-30T22:31:44.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.540+0000 7f1f0ce57700 1 -- 172.21.15.16:0/1768797259 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f0809d6f0 msgr2=0x7f1f0809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:44.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.540+0000 7f1f0ce57700 1 --2- 172.21.15.16:0/1768797259 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f0809d6f0 0x7f1f0809dac0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1ef8009950 tx=0x7f1ef8009c60 comp rx=0 tx=0).stop 2023-09-30T22:31:44.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.540+0000 7f1f0ce57700 1 -- 172.21.15.16:0/1768797259 shutdown_connections 2023-09-30T22:31:44.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.540+0000 7f1f0ce57700 1 --2- 172.21.15.16:0/1768797259 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f0809d6f0 0x7f1f0809dac0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:44.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.540+0000 7f1f0ce57700 1 --2- 172.21.15.16:0/1768797259 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f0809e000 0x7f1f081339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:44.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.540+0000 7f1f0ce57700 1 -- 172.21.15.16:0/1768797259 >> 172.21.15.16:0/1768797259 conn(0x7f1f08066f50 msgr2=0x7f1f0809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:44.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.540+0000 7f1f0ce57700 1 -- 172.21.15.16:0/1768797259 shutdown_connections 2023-09-30T22:31:44.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.540+0000 7f1f0ce57700 1 -- 172.21.15.16:0/1768797259 wait complete. 2023-09-30T22:31:44.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.542+0000 7f1f0ce57700 1 Processor -- start 2023-09-30T22:31:44.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.542+0000 7f1f0ce57700 1 -- start start 2023-09-30T22:31:44.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.542+0000 7f1f0ce57700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f0809d6f0 0x7f1f081c5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:44.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.542+0000 7f1f0ce57700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f0809e000 0x7f1f081c5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:44.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.542+0000 7f1f0ce57700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f0809e410 0x7f1f081c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:44.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.542+0000 7f1f0ce57700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f081a95a0 con 0x7f1f0809d6f0 2023-09-30T22:31:44.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.542+0000 7f1f0ce57700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f081c5a80 con 0x7f1f0809e410 2023-09-30T22:31:44.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.542+0000 7f1f0ce57700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1f081a7890 con 0x7f1f0809e000 2023-09-30T22:31:44.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.543+0000 7f1f05d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f0809e000 0x7f1f081c5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:44.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.543+0000 7f1f05d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f0809e000 0x7f1f081c5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:39922/0 (socket says 172.21.15.16:39922) 2023-09-30T22:31:44.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.543+0000 7f1f0659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f0809d6f0 0x7f1f081c5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:44.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.543+0000 7f1f05d9b700 1 -- 172.21.15.16:0/2685975280 learned_addr learned my addr 172.21.15.16:0/2685975280 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:44.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.543+0000 7f1f06d9d700 1 --2- 172.21.15.16:0/2685975280 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f0809e410 0x7f1f081c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:44.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.543+0000 7f1f05d9b700 1 -- 172.21.15.16:0/2685975280 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f0809e410 msgr2=0x7f1f081c8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:44.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.543+0000 7f1f05d9b700 1 --2- 172.21.15.16:0/2685975280 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f0809e410 0x7f1f081c8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:44.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.543+0000 7f1f05d9b700 1 -- 172.21.15.16:0/2685975280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f0809d6f0 msgr2=0x7f1f081c5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:44.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.543+0000 7f1f05d9b700 1 --2- 172.21.15.16:0/2685975280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f0809d6f0 0x7f1f081c5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:44.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.543+0000 7f1f05d9b700 1 -- 172.21.15.16:0/2685975280 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef8009550 con 0x7f1f0809e000 2023-09-30T22:31:44.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.543+0000 7f1f05d9b700 1 --2- 172.21.15.16:0/2685975280 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f0809e000 0x7f1f081c5540 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1efc00e9b0 tx=0x7f1efc00ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:44.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.543+0000 7f1ef6ffd700 1 -- 172.21.15.16:0/2685975280 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1efc009710 con 0x7f1f0809e000 2023-09-30T22:31:44.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.544+0000 7f1ef6ffd700 1 -- 172.21.15.16:0/2685975280 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1efc00ee50 con 0x7f1f0809e000 2023-09-30T22:31:44.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.544+0000 7f1f0ce57700 1 -- 172.21.15.16:0/2685975280 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f080a3010 con 0x7f1f0809e000 2023-09-30T22:31:44.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.544+0000 7f1ef6ffd700 1 -- 172.21.15.16:0/2685975280 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1efc00ca70 con 0x7f1f0809e000 2023-09-30T22:31:44.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.544+0000 7f1f0ce57700 1 -- 172.21.15.16:0/2685975280 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f081a8ae0 con 0x7f1f0809e000 2023-09-30T22:31:44.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.545+0000 7f1ef6ffd700 1 -- 172.21.15.16:0/2685975280 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1efc00cbd0 con 0x7f1f0809e000 2023-09-30T22:31:44.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.545+0000 7f1ef6ffd700 1 -- 172.21.15.16:0/2685975280 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f1efc03f720 con 0x7f1f0809e000 2023-09-30T22:31:44.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.546+0000 7f1f0ce57700 1 -- 172.21.15.16:0/2685975280 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f08074ea0 con 0x7f1f0809e000 2023-09-30T22:31:44.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.552+0000 7f1ef6ffd700 1 -- 172.21.15.16:0/2685975280 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1efc0232f0 con 0x7f1f0809e000 2023-09-30T22:31:44.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.623+0000 7f366f848700 1 Processor -- start 2023-09-30T22:31:44.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.624+0000 7f366f848700 1 -- start start 2023-09-30T22:31:44.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.624+0000 7f366f848700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f36681338a0 0x7f3668133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:44.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.624+0000 7f366f848700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f366809e930 0x7f366813b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:44.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.624+0000 7f366f848700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f366806ef30 con 0x7f366809de50 2023-09-30T22:31:44.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.624+0000 7f366f848700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f366806fef0 con 0x7f366809e930 2023-09-30T22:31:44.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.624+0000 7f366f848700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f36681334b0 con 0x7f36681338a0 2023-09-30T22:31:44.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.625+0000 7f366dde5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f366809e930 0x7f366813b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:44.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.625+0000 7f366dde5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f366809e930 0x7f366813b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:40682/0 (socket says 172.21.15.16:40682) 2023-09-30T22:31:44.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.625+0000 7f366dde5700 1 -- 172.21.15.16:0/1706314869 learned_addr learned my addr 172.21.15.16:0/1706314869 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:44.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.625+0000 7f366d5e4700 1 --2- 172.21.15.16:0/1706314869 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f36681338a0 0x7f3668133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:44.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.625+0000 7f366dde5700 1 -- 172.21.15.16:0/1706314869 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f36681338a0 msgr2=0x7f3668133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:44.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.625+0000 7f366dde5700 1 --2- 172.21.15.16:0/1706314869 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f36681338a0 0x7f3668133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:44.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.625+0000 7f366dde5700 1 -- 172.21.15.16:0/1706314869 >> v1:172.21.15.16:6789/0 conn(0x7f366809de50 legacy=0x7f366809e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:44.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.625+0000 7f366dde5700 1 -- 172.21.15.16:0/1706314869 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f366806fc30 con 0x7f366809e930 2023-09-30T22:31:44.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.625+0000 7f366dde5700 1 --2- 172.21.15.16:0/1706314869 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f366809e930 0x7f366813b740 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3660009e80 tx=0x7f366000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49102814e32fa69d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:44.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.625+0000 7f365ffff700 1 -- 172.21.15.16:0/1706314869 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366000e070 con 0x7f366809e930 2023-09-30T22:31:44.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.626+0000 7f365ffff700 1 -- 172.21.15.16:0/1706314869 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3660003a20 con 0x7f366809e930 2023-09-30T22:31:44.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.626+0000 7f365ffff700 1 -- 172.21.15.16:0/1706314869 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3660003dd0 con 0x7f366809e930 2023-09-30T22:31:44.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.626+0000 7f366f848700 1 -- 172.21.15.16:0/1706314869 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f366809e930 msgr2=0x7f366813b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:44.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.626+0000 7f366f848700 1 --2- 172.21.15.16:0/1706314869 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f366809e930 0x7f366813b740 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3660009e80 tx=0x7f366000b330 comp rx=0 tx=0).stop 2023-09-30T22:31:44.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.626+0000 7f366f848700 1 -- 172.21.15.16:0/1706314869 shutdown_connections 2023-09-30T22:31:44.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.626+0000 7f366f848700 1 --2- 172.21.15.16:0/1706314869 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f366809e930 0x7f366813b740 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:44.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.626+0000 7f366f848700 1 --2- 172.21.15.16:0/1706314869 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f36681338a0 0x7f3668133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:44.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.626+0000 7f366f848700 1 -- 172.21.15.16:0/1706314869 >> 172.21.15.16:0/1706314869 conn(0x7f36680a2aa0 msgr2=0x7f36680a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:44.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.627+0000 7f366f848700 1 -- 172.21.15.16:0/1706314869 shutdown_connections 2023-09-30T22:31:44.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.627+0000 7f366f848700 1 -- 172.21.15.16:0/1706314869 wait complete. 2023-09-30T22:31:44.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.628+0000 7f366f848700 1 Processor -- start 2023-09-30T22:31:44.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.628+0000 7f366f848700 1 -- start start 2023-09-30T22:31:44.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.629+0000 7f366f848700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f366809de50 0x7f366811ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:44.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.629+0000 7f366f848700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f366809e930 0x7f366811ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:44.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.629+0000 7f366f848700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36681338a0 0x7f366811f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:44.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.629+0000 7f366f848700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3668122290 con 0x7f36681338a0 2023-09-30T22:31:44.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.629+0000 7f366f848700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36680a1020 con 0x7f366809de50 2023-09-30T22:31:44.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.629+0000 7f366f848700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3668120770 con 0x7f366809e930 2023-09-30T22:31:44.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.629+0000 7f366d5e4700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f366809de50 0x7f366811ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:44.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.629+0000 7f366d5e4700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f366809de50 0x7f366811ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:40694/0 (socket says 172.21.15.16:40694) 2023-09-30T22:31:44.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.629+0000 7f366d5e4700 1 -- 172.21.15.16:0/1255773389 learned_addr learned my addr 172.21.15.16:0/1255773389 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:44.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.629+0000 7f366dde5700 1 --2- 172.21.15.16:0/1255773389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36681338a0 0x7f366811f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:44.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.629+0000 7f366cde3700 1 --2- 172.21.15.16:0/1255773389 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f366809e930 0x7f366811ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:44.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.630+0000 7f366d5e4700 1 -- 172.21.15.16:0/1255773389 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f366809e930 msgr2=0x7f366811ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:44.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.630+0000 7f366d5e4700 1 --2- 172.21.15.16:0/1255773389 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f366809e930 0x7f366811ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:44.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.630+0000 7f366d5e4700 1 -- 172.21.15.16:0/1255773389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36681338a0 msgr2=0x7f366811f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:44.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.630+0000 7f366d5e4700 1 --2- 172.21.15.16:0/1255773389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36681338a0 0x7f366811f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:44.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.630+0000 7f366d5e4700 1 -- 172.21.15.16:0/1255773389 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3660009b30 con 0x7f366809de50 2023-09-30T22:31:44.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.630+0000 7f366dde5700 1 --2- 172.21.15.16:0/1255773389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36681338a0 0x7f366811f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:44.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.630+0000 7f366d5e4700 1 --2- 172.21.15.16:0/1255773389 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f366809de50 0x7f366811ea50 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f366400e840 tx=0x7f366400eb50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:44.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.630+0000 7f365dffb700 1 -- 172.21.15.16:0/1255773389 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3664013610 con 0x7f366809de50 2023-09-30T22:31:44.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.630+0000 7f365dffb700 1 -- 172.21.15.16:0/1255773389 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3664013770 con 0x7f366809de50 2023-09-30T22:31:44.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.630+0000 7f366f848700 1 -- 172.21.15.16:0/1255773389 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3668120e80 con 0x7f366809de50 2023-09-30T22:31:44.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.630+0000 7f365dffb700 1 -- 172.21.15.16:0/1255773389 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3664013b20 con 0x7f366809de50 2023-09-30T22:31:44.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.630+0000 7f366f848700 1 -- 172.21.15.16:0/1255773389 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3668122d00 con 0x7f366809de50 2023-09-30T22:31:44.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.631+0000 7f365dffb700 1 -- 172.21.15.16:0/1255773389 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3664013c80 con 0x7f366809de50 2023-09-30T22:31:44.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.632+0000 7f365dffb700 1 -- 172.21.15.16:0/1255773389 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f366403f3e0 con 0x7f366809de50 2023-09-30T22:31:44.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.632+0000 7f366f848700 1 -- 172.21.15.16:0/1255773389 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3668074ea0 con 0x7f366809de50 2023-09-30T22:31:44.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.637+0000 7f365dffb700 1 -- 172.21.15.16:0/1255773389 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3664012370 con 0x7f366809de50 2023-09-30T22:31:44.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.669+0000 7ff13415c700 1 Processor -- start 2023-09-30T22:31:44.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.669+0000 7ff13415c700 1 -- start start 2023-09-30T22:31:44.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.670+0000 7ff13415c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff12c12cf60 0x7ff12c06f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:44.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.670+0000 7ff13415c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff12c12f8b0 0x7ff12c12fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:44.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.670+0000 7ff13415c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff12c130260 0x7ff12c136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:44.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.670+0000 7ff13415c700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff12c12d330 con 0x7ff12c12cf60 2023-09-30T22:31:44.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.670+0000 7ff13415c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff12c12cd90 con 0x7ff12c12f8b0 2023-09-30T22:31:44.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.670+0000 7ff13415c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff12c1306d0 con 0x7ff12c130260 2023-09-30T22:31:44.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.670+0000 7ff1316f7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff12c12f8b0 0x7ff12c12fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:44.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.670+0000 7ff1316f7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff12c12f8b0 0x7ff12c12fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:40706/0 (socket says 172.21.15.16:40706) 2023-09-30T22:31:44.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.670+0000 7ff1316f7700 1 -- 172.21.15.16:0/1182155450 learned_addr learned my addr 172.21.15.16:0/1182155450 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:44.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.670+0000 7ff131ef8700 1 --2- 172.21.15.16:0/1182155450 >> v2:172.21.15.16:3300/0 conn(0x7ff12c12cf60 0x7ff12c06f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:44.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.670+0000 7ff1326f9700 1 --2- 172.21.15.16:0/1182155450 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff12c130260 0x7ff12c136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:44.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.670+0000 7ff1326f9700 1 -- 172.21.15.16:0/1182155450 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff12c12f8b0 msgr2=0x7ff12c12fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:44.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.670+0000 7ff1326f9700 1 --2- 172.21.15.16:0/1182155450 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff12c12f8b0 0x7ff12c12fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:44.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.670+0000 7ff1326f9700 1 -- 172.21.15.16:0/1182155450 >> v2:172.21.15.16:3300/0 conn(0x7ff12c12cf60 msgr2=0x7ff12c06f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:44.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.670+0000 7ff1326f9700 1 --2- 172.21.15.16:0/1182155450 >> v2:172.21.15.16:3300/0 conn(0x7ff12c12cf60 0x7ff12c06f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:44.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.671+0000 7ff1326f9700 1 -- 172.21.15.16:0/1182155450 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff12c071780 con 0x7ff12c130260 2023-09-30T22:31:44.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.671+0000 7ff1316f7700 1 --2- 172.21.15.16:0/1182155450 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff12c12f8b0 0x7ff12c12fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:44.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.671+0000 7ff131ef8700 1 --2- 172.21.15.16:0/1182155450 >> v2:172.21.15.16:3300/0 conn(0x7ff12c12cf60 0x7ff12c06f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:44.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.671+0000 7ff1326f9700 1 --2- 172.21.15.16:0/1182155450 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff12c130260 0x7ff12c136c20 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff124009dc0 tx=0x7ff12400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f08938abbfad69b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:44.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.671+0000 7ff130ef6700 1 -- 172.21.15.16:0/1182155450 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12400e070 con 0x7ff12c130260 2023-09-30T22:31:44.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.671+0000 7ff130ef6700 1 -- 172.21.15.16:0/1182155450 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff12400bae0 con 0x7ff12c130260 2023-09-30T22:31:44.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.671+0000 7ff130ef6700 1 -- 172.21.15.16:0/1182155450 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12400be90 con 0x7ff12c130260 2023-09-30T22:31:44.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.672+0000 7ff13415c700 1 -- 172.21.15.16:0/1182155450 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff12c130260 msgr2=0x7ff12c136c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:44.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.672+0000 7ff13415c700 1 --2- 172.21.15.16:0/1182155450 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff12c130260 0x7ff12c136c20 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff124009dc0 tx=0x7ff12400b330 comp rx=0 tx=0).stop 2023-09-30T22:31:44.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.672+0000 7ff13415c700 1 -- 172.21.15.16:0/1182155450 shutdown_connections 2023-09-30T22:31:44.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.672+0000 7ff13415c700 1 --2- 172.21.15.16:0/1182155450 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff12c130260 0x7ff12c136c20 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:44.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.672+0000 7ff13415c700 1 --2- 172.21.15.16:0/1182155450 >> v2:172.21.15.16:3300/0 conn(0x7ff12c12cf60 0x7ff12c06f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:44.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.672+0000 7ff13415c700 1 --2- 172.21.15.16:0/1182155450 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff12c12f8b0 0x7ff12c12fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:44.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.672+0000 7ff13415c700 1 -- 172.21.15.16:0/1182155450 >> 172.21.15.16:0/1182155450 conn(0x7ff12c0a2aa0 msgr2=0x7ff12c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:44.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.672+0000 7ff13415c700 1 -- 172.21.15.16:0/1182155450 shutdown_connections 2023-09-30T22:31:44.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.672+0000 7ff13415c700 1 -- 172.21.15.16:0/1182155450 wait complete. 2023-09-30T22:31:44.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.673+0000 7ff13415c700 1 Processor -- start 2023-09-30T22:31:44.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.673+0000 7ff13415c700 1 -- start start 2023-09-30T22:31:44.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.673+0000 7ff13415c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff12c12cf60 0x7ff12c1b6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:44.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.674+0000 7ff13415c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff12c12f8b0 0x7ff12c1b6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:44.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.674+0000 7ff13415c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff12c130260 0x7ff12c1c8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:44.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.674+0000 7ff13415c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12c1b96f0 con 0x7ff12c130260 2023-09-30T22:31:44.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.674+0000 7ff13415c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff12c1b60a0 con 0x7ff12c12f8b0 2023-09-30T22:31:44.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.674+0000 7ff13415c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff12c09e160 con 0x7ff12c12cf60 2023-09-30T22:31:44.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.674+0000 7ff1316f7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff12c12f8b0 0x7ff12c1b6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:44.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.674+0000 7ff131ef8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff12c12cf60 0x7ff12c1b6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:44.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.674+0000 7ff1316f7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff12c12f8b0 0x7ff12c1b6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:40720/0 (socket says 172.21.15.16:40720) 2023-09-30T22:31:44.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.674+0000 7ff1316f7700 1 -- 172.21.15.16:0/2369511952 learned_addr learned my addr 172.21.15.16:0/2369511952 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:44.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.674+0000 7ff1316f7700 1 -- 172.21.15.16:0/2369511952 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff12c12cf60 msgr2=0x7ff12c1b6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:44.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.674+0000 7ff1316f7700 1 --2- 172.21.15.16:0/2369511952 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff12c12cf60 0x7ff12c1b6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:44.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.674+0000 7ff1316f7700 1 -- 172.21.15.16:0/2369511952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff12c130260 msgr2=0x7ff12c1c8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:44.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.674+0000 7ff1316f7700 1 --2- 172.21.15.16:0/2369511952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff12c130260 0x7ff12c1c8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:44.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.674+0000 7ff1316f7700 1 -- 172.21.15.16:0/2369511952 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1280094a0 con 0x7ff12c12f8b0 2023-09-30T22:31:44.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.674+0000 7ff1316f7700 1 --2- 172.21.15.16:0/2369511952 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff12c12f8b0 0x7ff12c1b6b80 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff11c00dd00 tx=0x7ff11c009400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:44.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.674+0000 7ff1227fc700 1 -- 172.21.15.16:0/2369511952 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff11c00de50 con 0x7ff12c12f8b0 2023-09-30T22:31:44.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.675+0000 7ff1227fc700 1 -- 172.21.15.16:0/2369511952 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff11c009c20 con 0x7ff12c12f8b0 2023-09-30T22:31:44.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.675+0000 7ff13415c700 1 -- 172.21.15.16:0/2369511952 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff11c00b7f0 con 0x7ff12c12f8b0 2023-09-30T22:31:44.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.675+0000 7ff1227fc700 1 -- 172.21.15.16:0/2369511952 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff11c017410 con 0x7ff12c12f8b0 2023-09-30T22:31:44.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.675+0000 7ff13415c700 1 -- 172.21.15.16:0/2369511952 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff124009a70 con 0x7ff12c12f8b0 2023-09-30T22:31:44.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.676+0000 7ff13415c700 1 -- 172.21.15.16:0/2369511952 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff12c074ea0 con 0x7ff12c12f8b0 2023-09-30T22:31:44.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.676+0000 7ff1227fc700 1 -- 172.21.15.16:0/2369511952 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff11c017570 con 0x7ff12c12f8b0 2023-09-30T22:31:44.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.676+0000 7ff1227fc700 1 -- 172.21.15.16:0/2369511952 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7ff11c00b050 con 0x7ff12c12f8b0 2023-09-30T22:31:44.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.685+0000 7ff1227fc700 1 -- 172.21.15.16:0/2369511952 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff11c01e8b0 con 0x7ff12c12f8b0 2023-09-30T22:31:44.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.692+0000 7f480e1c5700 1 Processor -- start 2023-09-30T22:31:44.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.692+0000 7f480e1c5700 1 -- start start 2023-09-30T22:31:44.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.693+0000 7f480e1c5700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f480812cf60 0x7f480806f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:44.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.693+0000 7f480e1c5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f480812f8b0 0x7f480812fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:44.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.693+0000 7f480e1c5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4808130260 0x7f4808136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:44.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.693+0000 7f480e1c5700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f480812d330 con 0x7f480812cf60 2023-09-30T22:31:44.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.693+0000 7f480e1c5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f480812cd90 con 0x7f4808130260 2023-09-30T22:31:44.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.693+0000 7f480e1c5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f48081306d0 con 0x7f480812f8b0 2023-09-30T22:31:44.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.693+0000 7f4807fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4808130260 0x7f4808136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:44.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.693+0000 7f48077fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f480812cf60 0x7f480806f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:44.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.693+0000 7f4807fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4808130260 0x7f4808136c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:40736/0 (socket says 172.21.15.16:40736) 2023-09-30T22:31:44.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.693+0000 7f4807fff700 1 -- 172.21.15.16:0/1167563590 learned_addr learned my addr 172.21.15.16:0/1167563590 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:44.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.693+0000 7f4806ffd700 1 --2- 172.21.15.16:0/1167563590 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f480812f8b0 0x7f480812fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.693+0000 7f4807fff700 1 -- 172.21.15.16:0/1167563590 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f480812f8b0 msgr2=0x7f480812fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.693+0000 7f4807fff700 1 --2- 172.21.15.16:0/1167563590 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f480812f8b0 0x7f480812fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:44.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.693+0000 7f4807fff700 1 -- 172.21.15.16:0/1167563590 >> v2:172.21.15.16:3300/0 conn(0x7f480812cf60 msgr2=0x7f480806f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:44.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.693+0000 7f4807fff700 1 --2- 172.21.15.16:0/1167563590 >> v2:172.21.15.16:3300/0 conn(0x7f480812cf60 0x7f480806f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:44.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.693+0000 7f4807fff700 1 -- 172.21.15.16:0/1167563590 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4808071780 con 0x7f4808130260 2023-09-30T22:31:44.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.693+0000 7f48077fe700 1 --2- 172.21.15.16:0/1167563590 >> v2:172.21.15.16:3300/0 conn(0x7f480812cf60 0x7f480806f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:44.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.694+0000 7f4807fff700 1 --2- 172.21.15.16:0/1167563590 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4808130260 0x7f4808136c20 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4800009e50 tx=0x7f480000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1df0dc2ae84a05cb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:44.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.694+0000 7f48067fc700 1 -- 172.21.15.16:0/1167563590 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480000e070 con 0x7f4808130260 2023-09-30T22:31:44.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.694+0000 7f48067fc700 1 -- 172.21.15.16:0/1167563590 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f480000b910 con 0x7f4808130260 2023-09-30T22:31:44.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.694+0000 7f48067fc700 1 -- 172.21.15.16:0/1167563590 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480000bcc0 con 0x7f4808130260 2023-09-30T22:31:44.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.695+0000 7f480e1c5700 1 -- 172.21.15.16:0/1167563590 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4808130260 msgr2=0x7f4808136c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:44.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.695+0000 7f480e1c5700 1 --2- 172.21.15.16:0/1167563590 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4808130260 0x7f4808136c20 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4800009e50 tx=0x7f480000b330 comp rx=0 tx=0).stop 2023-09-30T22:31:44.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.695+0000 7f480e1c5700 1 -- 172.21.15.16:0/1167563590 shutdown_connections 2023-09-30T22:31:44.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.695+0000 7f480e1c5700 1 --2- 172.21.15.16:0/1167563590 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4808130260 0x7f4808136c20 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:44.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.695+0000 7f480e1c5700 1 --2- 172.21.15.16:0/1167563590 >> v2:172.21.15.16:3300/0 conn(0x7f480812cf60 0x7f480806f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.695+0000 7f480e1c5700 1 --2- 172.21.15.16:0/1167563590 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f480812f8b0 0x7f480812fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.695+0000 7f480e1c5700 1 -- 172.21.15.16:0/1167563590 >> 172.21.15.16:0/1167563590 conn(0x7f48080a2aa0 msgr2=0x7f48080a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.695+0000 7f480e1c5700 1 -- 172.21.15.16:0/1167563590 shutdown_connections 2023-09-30T22:31:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.695+0000 7f480e1c5700 1 -- 172.21.15.16:0/1167563590 wait complete. 2023-09-30T22:31:44.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.696+0000 7f480e1c5700 1 Processor -- start 2023-09-30T22:31:44.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.697+0000 7f480e1c5700 1 -- start start 2023-09-30T22:31:44.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.697+0000 7f480e1c5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f480812cf60 0x7f48081309e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:44.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.697+0000 7f480e1c5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f480812f8b0 0x7f4808130f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:44.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.697+0000 7f480e1c5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4808130260 0x7f4808131460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:44.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.697+0000 7f480e1c5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4808133da0 con 0x7f480812f8b0 2023-09-30T22:31:44.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.697+0000 7f480e1c5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4808132c20 con 0x7f480812cf60 2023-09-30T22:31:44.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.697+0000 7f480e1c5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4808129fa0 con 0x7f4808130260 2023-09-30T22:31:44.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.698+0000 7f4807fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4808130260 0x7f4808131460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:44.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.698+0000 7f4807fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4808130260 0x7f4808131460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:39986/0 (socket says 172.21.15.16:39986) 2023-09-30T22:31:44.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.698+0000 7f4806ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f480812f8b0 0x7f4808130f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:44.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.698+0000 7f4807fff700 1 -- 172.21.15.16:0/2729580235 learned_addr learned my addr 172.21.15.16:0/2729580235 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:44.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.698+0000 7f48077fe700 1 --2- 172.21.15.16:0/2729580235 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f480812cf60 0x7f48081309e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:44.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.698+0000 7f4807fff700 1 -- 172.21.15.16:0/2729580235 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f480812cf60 msgr2=0x7f48081309e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:44.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.698+0000 7f4807fff700 1 --2- 172.21.15.16:0/2729580235 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f480812cf60 0x7f48081309e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:44.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.698+0000 7f4807fff700 1 -- 172.21.15.16:0/2729580235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f480812f8b0 msgr2=0x7f4808130f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:44.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.698+0000 7f4807fff700 1 --2- 172.21.15.16:0/2729580235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f480812f8b0 0x7f4808130f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:44.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.698+0000 7f4807fff700 1 -- 172.21.15.16:0/2729580235 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f80094a0 con 0x7f4808130260 2023-09-30T22:31:44.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.698+0000 7f48077fe700 1 --2- 172.21.15.16:0/2729580235 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f480812cf60 0x7f48081309e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:31:44.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.698+0000 7f4807fff700 1 --2- 172.21.15.16:0/2729580235 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4808130260 0x7f4808131460 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4800003f50 tx=0x7f4800004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:44.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.699+0000 7f47effff700 1 -- 172.21.15.16:0/2729580235 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4800004450 con 0x7f4808130260 2023-09-30T22:31:44.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.699+0000 7f47effff700 1 -- 172.21.15.16:0/2729580235 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48000045b0 con 0x7f4808130260 2023-09-30T22:31:44.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.699+0000 7f480e1c5700 1 -- 172.21.15.16:0/2729580235 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4800009b30 con 0x7f4808130260 2023-09-30T22:31:44.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.699+0000 7f47effff700 1 -- 172.21.15.16:0/2729580235 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4800004940 con 0x7f4808130260 2023-09-30T22:31:44.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.699+0000 7f480e1c5700 1 -- 172.21.15.16:0/2729580235 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f480809e650 con 0x7f4808130260 2023-09-30T22:31:44.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.700+0000 7f47effff700 1 -- 172.21.15.16:0/2729580235 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4800025460 con 0x7f4808130260 2023-09-30T22:31:44.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.700+0000 7f47effff700 1 -- 172.21.15.16:0/2729580235 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f4800042740 con 0x7f4808130260 2023-09-30T22:31:44.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.700+0000 7f480e1c5700 1 -- 172.21.15.16:0/2729580235 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4808074ea0 con 0x7f4808130260 2023-09-30T22:31:44.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.705+0000 7f47effff700 1 -- 172.21.15.16:0/2729580235 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4800046060 con 0x7f4808130260 2023-09-30T22:31:44.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.784+0000 7f1f0ce57700 1 -- 172.21.15.16:0/2685975280 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1f0806c360 con 0x7f1f0809e000 2023-09-30T22:31:44.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.785+0000 7f1ef6ffd700 1 -- 172.21.15.16:0/2685975280 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1efc021020 con 0x7f1f0809e000 2023-09-30T22:31:44.786 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:31:44.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.787+0000 7f1f0ce57700 1 -- 172.21.15.16:0/2685975280 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f0809e000 msgr2=0x7f1f081c5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:44.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.787+0000 7f1f0ce57700 1 --2- 172.21.15.16:0/2685975280 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f0809e000 0x7f1f081c5540 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f1efc00e9b0 tx=0x7f1efc00ecc0 comp rx=0 tx=0).stop 2023-09-30T22:31:44.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.787+0000 7f1f0ce57700 1 -- 172.21.15.16:0/2685975280 shutdown_connections 2023-09-30T22:31:44.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.787+0000 7f1f0ce57700 1 --2- 172.21.15.16:0/2685975280 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f0809e410 0x7f1f081c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:44.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.787+0000 7f1f0ce57700 1 --2- 172.21.15.16:0/2685975280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f0809d6f0 0x7f1f081c5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:44.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.787+0000 7f1f0ce57700 1 --2- 172.21.15.16:0/2685975280 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f0809e000 0x7f1f081c5540 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:44.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.787+0000 7f1f0ce57700 1 -- 172.21.15.16:0/2685975280 >> 172.21.15.16:0/2685975280 conn(0x7f1f08066f50 msgr2=0x7f1f081bad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:44.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.787+0000 7f1f0ce57700 1 -- 172.21.15.16:0/2685975280 shutdown_connections 2023-09-30T22:31:44.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.787+0000 7f1f0ce57700 1 -- 172.21.15.16:0/2685975280 wait complete. 2023-09-30T22:31:44.802 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:31:44.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.853+0000 7f366f848700 1 -- 172.21.15.16:0/1255773389 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f366806b9e0 con 0x7f366809de50 2023-09-30T22:31:44.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.853+0000 7f365dffb700 1 -- 172.21.15.16:0/1255773389 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3664021020 con 0x7f366809de50 2023-09-30T22:31:44.854 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:31:44.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.856+0000 7f366f848700 1 -- 172.21.15.16:0/1255773389 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f366809de50 msgr2=0x7f366811ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:44.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.856+0000 7f366f848700 1 --2- 172.21.15.16:0/1255773389 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f366809de50 0x7f366811ea50 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f366400e840 tx=0x7f366400eb50 comp rx=0 tx=0).stop 2023-09-30T22:31:44.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.856+0000 7f366f848700 1 -- 172.21.15.16:0/1255773389 shutdown_connections 2023-09-30T22:31:44.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.856+0000 7f366f848700 1 --2- 172.21.15.16:0/1255773389 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f366809de50 0x7f366811ea50 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:44.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.856+0000 7f366f848700 1 --2- 172.21.15.16:0/1255773389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f36681338a0 0x7f366811f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:44.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.856+0000 7f366f848700 1 --2- 172.21.15.16:0/1255773389 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f366809e930 0x7f366811ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:44.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.856+0000 7f366f848700 1 -- 172.21.15.16:0/1255773389 >> 172.21.15.16:0/1255773389 conn(0x7f36680a2aa0 msgr2=0x7f36681315b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:44.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.856+0000 7f366f848700 1 -- 172.21.15.16:0/1255773389 shutdown_connections 2023-09-30T22:31:44.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.857+0000 7f366f848700 1 -- 172.21.15.16:0/1255773389 wait complete. 2023-09-30T22:31:44.875 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:31:44.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.915+0000 7ff13415c700 1 -- 172.21.15.16:0/2369511952 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff12c0713a0 con 0x7ff12c12f8b0 2023-09-30T22:31:44.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.915+0000 7ff1227fc700 1 -- 172.21.15.16:0/2369511952 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff11c01c070 con 0x7ff12c12f8b0 2023-09-30T22:31:44.916 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:31:44.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.917+0000 7ff13415c700 1 -- 172.21.15.16:0/2369511952 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff12c12f8b0 msgr2=0x7ff12c1b6b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:44.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.917+0000 7ff13415c700 1 --2- 172.21.15.16:0/2369511952 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff12c12f8b0 0x7ff12c1b6b80 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff11c00dd00 tx=0x7ff11c009400 comp rx=0 tx=0).stop 2023-09-30T22:31:44.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.918+0000 7ff13415c700 1 -- 172.21.15.16:0/2369511952 shutdown_connections 2023-09-30T22:31:44.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.918+0000 7ff13415c700 1 --2- 172.21.15.16:0/2369511952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff12c130260 0x7ff12c1c8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:44.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.918+0000 7ff13415c700 1 --2- 172.21.15.16:0/2369511952 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff12c12cf60 0x7ff12c1b6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:44.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.918+0000 7ff13415c700 1 --2- 172.21.15.16:0/2369511952 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff12c12f8b0 0x7ff12c1b6b80 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:44.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.918+0000 7ff13415c700 1 -- 172.21.15.16:0/2369511952 >> 172.21.15.16:0/2369511952 conn(0x7ff12c0a2aa0 msgr2=0x7ff12c12e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:44.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.918+0000 7ff13415c700 1 -- 172.21.15.16:0/2369511952 shutdown_connections 2023-09-30T22:31:44.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.918+0000 7ff13415c700 1 -- 172.21.15.16:0/2369511952 wait complete. 2023-09-30T22:31:44.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.920+0000 7f480e1c5700 1 -- 172.21.15.16:0/2729580235 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f48080713a0 con 0x7f4808130260 2023-09-30T22:31:44.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.921+0000 7f47effff700 1 -- 172.21.15.16:0/2729580235 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f480000ce80 con 0x7f4808130260 2023-09-30T22:31:44.921 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:31:44.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.923+0000 7f480e1c5700 1 -- 172.21.15.16:0/2729580235 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4808130260 msgr2=0x7f4808131460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:44.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.923+0000 7f480e1c5700 1 --2- 172.21.15.16:0/2729580235 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4808130260 0x7f4808131460 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4800003f50 tx=0x7f4800004ad0 comp rx=0 tx=0).stop 2023-09-30T22:31:44.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.923+0000 7f480e1c5700 1 -- 172.21.15.16:0/2729580235 shutdown_connections 2023-09-30T22:31:44.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.923+0000 7f480e1c5700 1 --2- 172.21.15.16:0/2729580235 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f480812cf60 0x7f48081309e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:44.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.923+0000 7f480e1c5700 1 --2- 172.21.15.16:0/2729580235 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4808130260 0x7f4808131460 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:44.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.923+0000 7f480e1c5700 1 --2- 172.21.15.16:0/2729580235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f480812f8b0 0x7f4808130f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:44.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.923+0000 7f480e1c5700 1 -- 172.21.15.16:0/2729580235 >> 172.21.15.16:0/2729580235 conn(0x7f48080a2aa0 msgr2=0x7f480812a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:44.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.923+0000 7f480e1c5700 1 -- 172.21.15.16:0/2729580235 shutdown_connections 2023-09-30T22:31:44.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:44.923+0000 7f480e1c5700 1 -- 172.21.15.16:0/2729580235 wait complete. 2023-09-30T22:31:44.933 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:31:44.937 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:31:45.803 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:45.876 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:45.933 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:45.938 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:45.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:45.953+0000 7f5d5759f700 1 Processor -- start 2023-09-30T22:31:45.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:45.953+0000 7f5d5759f700 1 -- start start 2023-09-30T22:31:45.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:45.954+0000 7f5d5759f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5d501305f0 0x7f5d501309c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:45.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:45.954+0000 7f5d5759f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d5006f660 0x7f5d5006fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:45.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:45.954+0000 7f5d5759f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d5006ff90 0x7f5d50137530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:45.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:45.954+0000 7f5d5759f700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d501301c0 con 0x7f5d501305f0 2023-09-30T22:31:45.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:45.954+0000 7f5d5759f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d50071ff0 con 0x7f5d5006f660 2023-09-30T22:31:45.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:45.954+0000 7f5d5759f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5d50070e70 con 0x7f5d5006ff90 2023-09-30T22:31:45.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:45.954+0000 7f5d5533b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5d501305f0 0x7f5d501309c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:45.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:45.954+0000 7f5d5533b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5d501305f0 0x7f5d501309c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48270/0 (socket says 172.21.15.16:48270) 2023-09-30T22:31:45.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:45.954+0000 7f5d5533b700 1 -- 172.21.15.16:0/1506423324 learned_addr learned my addr 172.21.15.16:0/1506423324 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:45.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:45.954+0000 7f5d54b3a700 1 --2- 172.21.15.16:0/1506423324 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d5006f660 0x7f5d5006fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:45.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:45.954+0000 7f5d55b3c700 1 --2- 172.21.15.16:0/1506423324 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d5006ff90 0x7f5d50137530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:45.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:45.955+0000 7f5d5533b700 1 -- 172.21.15.16:0/1506423324 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d5006ff90 msgr2=0x7f5d50137530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:45.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:45.955+0000 7f5d5533b700 1 --2- 172.21.15.16:0/1506423324 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d5006ff90 0x7f5d50137530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:45.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:45.955+0000 7f5d5533b700 1 -- 172.21.15.16:0/1506423324 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d5006f660 msgr2=0x7f5d5006fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:45.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:45.955+0000 7f5d5533b700 1 --2- 172.21.15.16:0/1506423324 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d5006f660 0x7f5d5006fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:45.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:45.955+0000 7f5d5533b700 1 -- 172.21.15.16:0/1506423324 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d500722b0 con 0x7f5d501305f0 2023-09-30T22:31:45.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:45.955+0000 7f5d54b3a700 1 --2- 172.21.15.16:0/1506423324 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d5006f660 0x7f5d5006fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:45.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:45.955+0000 7f5d55b3c700 1 --2- 172.21.15.16:0/1506423324 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d5006ff90 0x7f5d50137530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:45.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:45.955+0000 7f5d5533b700 1 --2- 172.21.15.16:0/1506423324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d501305f0 0x7f5d501309c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c0098f0 tx=0x7f5d4c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbbcc7181e768878 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:45.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:45.955+0000 7f5d47fff700 1 -- 172.21.15.16:0/1506423324 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c011830 con 0x7f5d501305f0 2023-09-30T22:31:45.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:45.955+0000 7f5d47fff700 1 -- 172.21.15.16:0/1506423324 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d4c011990 con 0x7f5d501305f0 2023-09-30T22:31:45.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:45.955+0000 7f5d47fff700 1 -- 172.21.15.16:0/1506423324 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4c011d20 con 0x7f5d501305f0 2023-09-30T22:31:45.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:45.956+0000 7f5d5759f700 1 -- 172.21.15.16:0/1506423324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d501305f0 msgr2=0x7f5d501309c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:45.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:45.956+0000 7f5d5759f700 1 --2- 172.21.15.16:0/1506423324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d501305f0 0x7f5d501309c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c0098f0 tx=0x7f5d4c009c00 comp rx=0 tx=0).stop 2023-09-30T22:31:45.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:45.957+0000 7f5d5759f700 1 -- 172.21.15.16:0/1506423324 shutdown_connections 2023-09-30T22:31:45.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:45.957+0000 7f5d5759f700 1 --2- 172.21.15.16:0/1506423324 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d5006ff90 0x7f5d50137530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:45.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:45.957+0000 7f5d5759f700 1 --2- 172.21.15.16:0/1506423324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d501305f0 0x7f5d501309c0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:45.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:45.957+0000 7f5d5759f700 1 --2- 172.21.15.16:0/1506423324 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d5006f660 0x7f5d5006fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:45.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:45.957+0000 7f5d5759f700 1 -- 172.21.15.16:0/1506423324 >> 172.21.15.16:0/1506423324 conn(0x7f5d500a2aa0 msgr2=0x7f5d500a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:45.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:45.957+0000 7f5d5759f700 1 -- 172.21.15.16:0/1506423324 shutdown_connections 2023-09-30T22:31:45.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:45.957+0000 7f5d5759f700 1 -- 172.21.15.16:0/1506423324 wait complete. 2023-09-30T22:31:45.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:45.958+0000 7f5d5759f700 1 Processor -- start 2023-09-30T22:31:45.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:45.958+0000 7f5d5759f700 1 -- start start 2023-09-30T22:31:45.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:45.959+0000 7f5d5759f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d5006f660 0x7f5d5011b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:45.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:45.959+0000 7f5d5759f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d5006ff90 0x7f5d5011be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:45.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:45.959+0000 7f5d5759f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d501305f0 0x7f5d5011c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:45.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:45.959+0000 7f5d5759f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5011f420 con 0x7f5d5006ff90 2023-09-30T22:31:45.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:45.959+0000 7f5d5759f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5012ecf0 con 0x7f5d5006f660 2023-09-30T22:31:45.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:45.959+0000 7f5d5759f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5d50071820 con 0x7f5d501305f0 2023-09-30T22:31:45.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:45.959+0000 7f5d54b3a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d5006ff90 0x7f5d5011be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:45.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:45.959+0000 7f5d54b3a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d5006ff90 0x7f5d5011be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48278/0 (socket says 172.21.15.16:48278) 2023-09-30T22:31:45.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:45.959+0000 7f5d54b3a700 1 -- 172.21.15.16:0/1726352825 learned_addr learned my addr 172.21.15.16:0/1726352825 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:45.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:45.960+0000 7f5d54b3a700 1 -- 172.21.15.16:0/1726352825 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d501305f0 msgr2=0x7f5d5011c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:45.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:45.960+0000 7f5d54b3a700 1 --2- 172.21.15.16:0/1726352825 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d501305f0 0x7f5d5011c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:45.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:45.960+0000 7f5d54b3a700 1 -- 172.21.15.16:0/1726352825 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d5006f660 msgr2=0x7f5d5011b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:45.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:45.960+0000 7f5d54b3a700 1 --2- 172.21.15.16:0/1726352825 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d5006f660 0x7f5d5011b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:45.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:45.960+0000 7f5d54b3a700 1 -- 172.21.15.16:0/1726352825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d4000b550 con 0x7f5d5006ff90 2023-09-30T22:31:45.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:45.960+0000 7f5d54b3a700 1 --2- 172.21.15.16:0/1726352825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d5006ff90 0x7f5d5011be40 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f5d4000fde0 tx=0x7f5d4000d630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:45.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:45.961+0000 7f5d45ffb700 1 -- 172.21.15.16:0/1726352825 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4000be60 con 0x7f5d5006ff90 2023-09-30T22:31:45.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:45.961+0000 7f5d45ffb700 1 -- 172.21.15.16:0/1726352825 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d40018440 con 0x7f5d5006ff90 2023-09-30T22:31:45.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:45.961+0000 7f5d5759f700 1 -- 172.21.15.16:0/1726352825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d4800f050 con 0x7f5d5006ff90 2023-09-30T22:31:45.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:45.961+0000 7f5d45ffb700 1 -- 172.21.15.16:0/1726352825 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d400187d0 con 0x7f5d5006ff90 2023-09-30T22:31:45.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:45.961+0000 7f5d5759f700 1 -- 172.21.15.16:0/1726352825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d4c009550 con 0x7f5d5006ff90 2023-09-30T22:31:45.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:45.962+0000 7f5d5759f700 1 -- 172.21.15.16:0/1726352825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d5011e7f0 con 0x7f5d5006ff90 2023-09-30T22:31:45.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:45.962+0000 7f5d45ffb700 1 -- 172.21.15.16:0/1726352825 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5d40018930 con 0x7f5d5006ff90 2023-09-30T22:31:45.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:45.963+0000 7f5d45ffb700 1 -- 172.21.15.16:0/1726352825 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f5d4003fe70 con 0x7f5d5006ff90 2023-09-30T22:31:45.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:45.969+0000 7f5d45ffb700 1 -- 172.21.15.16:0/1726352825 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5d40017990 con 0x7f5d5006ff90 2023-09-30T22:31:46.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.025+0000 7fa996453700 1 Processor -- start 2023-09-30T22:31:46.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.025+0000 7fa996453700 1 -- start start 2023-09-30T22:31:46.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.025+0000 7fa996453700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa99009d6f0 0x7fa99009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:46.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.025+0000 7fa996453700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa99009e000 0x7fa990130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:46.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.025+0000 7fa996453700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa99009f130 con 0x7fa99009e410 2023-09-30T22:31:46.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.025+0000 7fa996453700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa99009f520 con 0x7fa99009e000 2023-09-30T22:31:46.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.025+0000 7fa996453700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa99009e930 con 0x7fa99009d6f0 2023-09-30T22:31:46.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.025+0000 7fa98f7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa99009e000 0x7fa990130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:46.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.025+0000 7fa98f7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa99009e000 0x7fa990130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48282/0 (socket says 172.21.15.16:48282) 2023-09-30T22:31:46.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.025+0000 7fa98f7fe700 1 -- 172.21.15.16:0/3222495361 learned_addr learned my addr 172.21.15.16:0/3222495361 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:46.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.026+0000 7fa98ffff700 1 --2- 172.21.15.16:0/3222495361 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa99009d6f0 0x7fa99009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:46.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.026+0000 7fa98effd700 1 -- 172.21.15.16:0/3222495361 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3404504351 0 0) 0x7fa99009f130 con 0x7fa99009e410 2023-09-30T22:31:46.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.026+0000 7fa98effd700 1 -- 172.21.15.16:0/3222495361 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9780035b0 con 0x7fa99009e410 2023-09-30T22:31:46.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.026+0000 7fa98f7fe700 1 -- 172.21.15.16:0/3222495361 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa99009d6f0 msgr2=0x7fa99009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:46.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.026+0000 7fa98f7fe700 1 --2- 172.21.15.16:0/3222495361 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa99009d6f0 0x7fa99009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:46.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.026+0000 7fa98f7fe700 1 -- 172.21.15.16:0/3222495361 >> v1:172.21.15.16:6789/0 conn(0x7fa99009e410 legacy=0x7fa990130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:46.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.026+0000 7fa98f7fe700 1 -- 172.21.15.16:0/3222495361 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa99009b2d0 con 0x7fa99009e000 2023-09-30T22:31:46.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.026+0000 7fa98ffff700 1 --2- 172.21.15.16:0/3222495361 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa99009d6f0 0x7fa99009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:46.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.026+0000 7fa98f7fe700 1 --2- 172.21.15.16:0/3222495361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa99009e000 0x7fa990130660 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fa9840098f0 tx=0x7fa984009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d85520ba985e53cd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:46.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.026+0000 7fa98effd700 1 -- 172.21.15.16:0/3222495361 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa984011830 con 0x7fa99009e000 2023-09-30T22:31:46.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.027+0000 7fa98effd700 1 -- 172.21.15.16:0/3222495361 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa984011990 con 0x7fa99009e000 2023-09-30T22:31:46.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.027+0000 7fa98effd700 1 -- 172.21.15.16:0/3222495361 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa984011d20 con 0x7fa99009e000 2023-09-30T22:31:46.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.027+0000 7fa996453700 1 -- 172.21.15.16:0/3222495361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa99009e000 msgr2=0x7fa990130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:46.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.027+0000 7fa996453700 1 --2- 172.21.15.16:0/3222495361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa99009e000 0x7fa990130660 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fa9840098f0 tx=0x7fa984009c00 comp rx=0 tx=0).stop 2023-09-30T22:31:46.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.027+0000 7fa996453700 1 -- 172.21.15.16:0/3222495361 shutdown_connections 2023-09-30T22:31:46.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.027+0000 7fa996453700 1 --2- 172.21.15.16:0/3222495361 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa99009d6f0 0x7fa99009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:46.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.027+0000 7fa996453700 1 --2- 172.21.15.16:0/3222495361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa99009e000 0x7fa990130660 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:46.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.027+0000 7fa996453700 1 -- 172.21.15.16:0/3222495361 >> 172.21.15.16:0/3222495361 conn(0x7fa990066f50 msgr2=0x7fa99009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:46.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.027+0000 7fa996453700 1 -- 172.21.15.16:0/3222495361 shutdown_connections 2023-09-30T22:31:46.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.027+0000 7fa996453700 1 -- 172.21.15.16:0/3222495361 wait complete. 2023-09-30T22:31:46.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.028+0000 7fa996453700 1 Processor -- start 2023-09-30T22:31:46.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.029+0000 7fa996453700 1 -- start start 2023-09-30T22:31:46.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.029+0000 7fa996453700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa99009d6f0 0x7fa990137570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:46.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.029+0000 7fa996453700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa99009e000 0x7fa990137ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:46.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.029+0000 7fa996453700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa99009e410 0x7fa990137ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:46.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.029+0000 7fa996453700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9900724e0 con 0x7fa99009e000 2023-09-30T22:31:46.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.029+0000 7fa996453700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa99009c700 con 0x7fa99009e410 2023-09-30T22:31:46.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.029+0000 7fa996453700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa99013a820 con 0x7fa99009d6f0 2023-09-30T22:31:46.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.029+0000 7fa98ffff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa99009d6f0 0x7fa990137570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:46.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.029+0000 7fa98ffff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa99009d6f0 0x7fa990137570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:40030/0 (socket says 172.21.15.16:40030) 2023-09-30T22:31:46.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.029+0000 7fa98ffff700 1 -- 172.21.15.16:0/2595763062 learned_addr learned my addr 172.21.15.16:0/2595763062 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:46.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.029+0000 7fa98f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa99009e000 0x7fa990137ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:46.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.030+0000 7fa9949f0700 1 --2- 172.21.15.16:0/2595763062 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa99009e410 0x7fa990137ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:46.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.030+0000 7fa98ffff700 1 -- 172.21.15.16:0/2595763062 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa99009e410 msgr2=0x7fa990137ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:46.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.030+0000 7fa98ffff700 1 --2- 172.21.15.16:0/2595763062 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa99009e410 0x7fa990137ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:46.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.030+0000 7fa98ffff700 1 -- 172.21.15.16:0/2595763062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa99009e000 msgr2=0x7fa990137ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:46.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.030+0000 7fa98ffff700 1 --2- 172.21.15.16:0/2595763062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa99009e000 0x7fa990137ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:46.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.030+0000 7fa98ffff700 1 -- 172.21.15.16:0/2595763062 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9800094a0 con 0x7fa99009d6f0 2023-09-30T22:31:46.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.030+0000 7fa98f7fe700 1 --2- 172.21.15.16:0/2595763062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa99009e000 0x7fa990137ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:46.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.030+0000 7fa98ffff700 1 --2- 172.21.15.16:0/2595763062 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa99009d6f0 0x7fa990137570 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa98000ebe0 tx=0x7fa98000eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:46.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.030+0000 7fa98cff9700 1 -- 172.21.15.16:0/2595763062 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98000c850 con 0x7fa99009d6f0 2023-09-30T22:31:46.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.030+0000 7fa98cff9700 1 -- 172.21.15.16:0/2595763062 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa98000c9b0 con 0x7fa99009d6f0 2023-09-30T22:31:46.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.030+0000 7fa996453700 1 -- 172.21.15.16:0/2595763062 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa984009550 con 0x7fa99009d6f0 2023-09-30T22:31:46.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.030+0000 7fa98cff9700 1 -- 172.21.15.16:0/2595763062 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98000cd40 con 0x7fa99009d6f0 2023-09-30T22:31:46.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.030+0000 7fa996453700 1 -- 172.21.15.16:0/2595763062 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa990139970 con 0x7fa99009d6f0 2023-09-30T22:31:46.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.031+0000 7fa996453700 1 -- 172.21.15.16:0/2595763062 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa990074ea0 con 0x7fa99009d6f0 2023-09-30T22:31:46.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.032+0000 7fa98cff9700 1 -- 172.21.15.16:0/2595763062 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa980013470 con 0x7fa99009d6f0 2023-09-30T22:31:46.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.032+0000 7fa98cff9700 1 -- 172.21.15.16:0/2595763062 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fa98003ff80 con 0x7fa99009d6f0 2023-09-30T22:31:46.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.041+0000 7fa98cff9700 1 -- 172.21.15.16:0/2595763062 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa980013720 con 0x7fa99009d6f0 2023-09-30T22:31:46.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.088+0000 7f7b695a4700 1 Processor -- start 2023-09-30T22:31:46.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.088+0000 7f7b695a4700 1 -- start start 2023-09-30T22:31:46.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.089+0000 7f7b695a4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7b6409e1d0 0x7f7b64138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:46.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.089+0000 7f7b695a4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7b64139170 0x7f7b6413b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:46.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.089+0000 7f7b695a4700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b6409b120 con 0x7f7b6409d6f0 2023-09-30T22:31:46.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.089+0000 7f7b695a4700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b6406b460 con 0x7f7b64139170 2023-09-30T22:31:46.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.089+0000 7f7b695a4700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6409a7e0 con 0x7f7b6409e1d0 2023-09-30T22:31:46.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.089+0000 7f7b637fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7b64139170 0x7f7b6413b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:46.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.089+0000 7f7b627fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7b6409e1d0 0x7f7b64138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:46.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.089+0000 7f7b62ffd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7b6409d6f0 0x7f7b6409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41404/0 (socket says 172.21.15.16:41404) 2023-09-30T22:31:46.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.089+0000 7f7b62ffd700 1 -- 172.21.15.16:0/1287830997 learned_addr learned my addr 172.21.15.16:0/1287830997 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:31:46.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.089+0000 7f7b627fc700 1 -- 172.21.15.16:0/1287830997 >> v2:172.21.15.16:3300/0 conn(0x7f7b64139170 msgr2=0x7f7b6413b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:46.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.089+0000 7f7b627fc700 1 --2- 172.21.15.16:0/1287830997 >> v2:172.21.15.16:3300/0 conn(0x7f7b64139170 0x7f7b6413b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:46.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.089+0000 7f7b627fc700 1 -- 172.21.15.16:0/1287830997 >> v1:172.21.15.16:6789/0 conn(0x7f7b6409d6f0 legacy=0x7f7b6409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:46.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.089+0000 7f7b627fc700 1 -- 172.21.15.16:0/1287830997 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b6409a500 con 0x7f7b6409e1d0 2023-09-30T22:31:46.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.090+0000 7f7b627fc700 1 --2- 172.21.15.16:0/1287830997 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7b6409e1d0 0x7f7b64138c30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7b58009870 tx=0x7f7b58009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ede9cb9368d4ff9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:46.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.090+0000 7f7b61ffb700 1 -- 172.21.15.16:0/1287830997 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b58011830 con 0x7f7b6409e1d0 2023-09-30T22:31:46.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.090+0000 7f7b61ffb700 1 -- 172.21.15.16:0/1287830997 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b58011990 con 0x7f7b6409e1d0 2023-09-30T22:31:46.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.090+0000 7f7b61ffb700 1 -- 172.21.15.16:0/1287830997 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b58011d20 con 0x7f7b6409e1d0 2023-09-30T22:31:46.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.091+0000 7f7b695a4700 1 -- 172.21.15.16:0/1287830997 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7b6409e1d0 msgr2=0x7f7b64138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:46.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.091+0000 7f7b695a4700 1 --2- 172.21.15.16:0/1287830997 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7b6409e1d0 0x7f7b64138c30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f7b58009870 tx=0x7f7b58009b80 comp rx=0 tx=0).stop 2023-09-30T22:31:46.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.091+0000 7f7b695a4700 1 -- 172.21.15.16:0/1287830997 shutdown_connections 2023-09-30T22:31:46.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.091+0000 7f7b695a4700 1 --2- 172.21.15.16:0/1287830997 >> v2:172.21.15.16:3300/0 conn(0x7f7b64139170 0x7f7b6413b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:46.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.091+0000 7f7b695a4700 1 --2- 172.21.15.16:0/1287830997 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7b6409e1d0 0x7f7b64138c30 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:46.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.091+0000 7f7b695a4700 1 -- 172.21.15.16:0/1287830997 >> 172.21.15.16:0/1287830997 conn(0x7f7b64066f50 msgr2=0x7f7b6409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:46.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.091+0000 7f7b695a4700 1 -- 172.21.15.16:0/1287830997 shutdown_connections 2023-09-30T22:31:46.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.091+0000 7f7b695a4700 1 -- 172.21.15.16:0/1287830997 wait complete. 2023-09-30T22:31:46.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.092+0000 7f7b695a4700 1 Processor -- start 2023-09-30T22:31:46.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.092+0000 7f7b695a4700 1 -- start start 2023-09-30T22:31:46.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.093+0000 7f7b695a4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7b6409d6f0 0x7f7b641c8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:46.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.093+0000 7f7b695a4700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7b6409e1d0 0x7f7b641c8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:46.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.093+0000 7f7b695a4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b64139170 0x7f7b641207c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:46.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.093+0000 7f7b695a4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b64124090 con 0x7f7b64139170 2023-09-30T22:31:46.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.093+0000 7f7b695a4700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b641be250 con 0x7f7b6409e1d0 2023-09-30T22:31:46.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.093+0000 7f7b695a4700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7b641242e0 con 0x7f7b6409d6f0 2023-09-30T22:31:46.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.093+0000 7f7b627fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7b6409e1d0 0x7f7b641c8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:46.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.093+0000 7f7b627fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7b6409e1d0 0x7f7b641c8a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:40780/0 (socket says 172.21.15.16:40780) 2023-09-30T22:31:46.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.093+0000 7f7b627fc700 1 -- 172.21.15.16:0/4004804996 learned_addr learned my addr 172.21.15.16:0/4004804996 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:46.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.093+0000 7f7b637fe700 1 --2- 172.21.15.16:0/4004804996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b64139170 0x7f7b641207c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:46.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.094+0000 7f7b627fc700 1 -- 172.21.15.16:0/4004804996 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7b6409d6f0 msgr2=0x7f7b641c8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:46.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.094+0000 7f7b627fc700 1 --2- 172.21.15.16:0/4004804996 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7b6409d6f0 0x7f7b641c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:46.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.094+0000 7f7b627fc700 1 -- 172.21.15.16:0/4004804996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b64139170 msgr2=0x7f7b641207c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:46.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.094+0000 7f7b627fc700 1 --2- 172.21.15.16:0/4004804996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b64139170 0x7f7b641207c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:46.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.094+0000 7f7b627fc700 1 -- 172.21.15.16:0/4004804996 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b580094a0 con 0x7f7b6409e1d0 2023-09-30T22:31:46.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.094+0000 7f7b637fe700 1 --2- 172.21.15.16:0/4004804996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b64139170 0x7f7b641207c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:46.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.094+0000 7f7b627fc700 1 --2- 172.21.15.16:0/4004804996 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7b6409e1d0 0x7f7b641c8a50 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7b58000c00 tx=0x7f7b58004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:46.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.094+0000 7f7b4b7fe700 1 -- 172.21.15.16:0/4004804996 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b580036a0 con 0x7f7b6409e1d0 2023-09-30T22:31:46.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.094+0000 7f7b695a4700 1 -- 172.21.15.16:0/4004804996 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b641223d0 con 0x7f7b6409e1d0 2023-09-30T22:31:46.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.094+0000 7f7b695a4700 1 -- 172.21.15.16:0/4004804996 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b64136690 con 0x7f7b6409e1d0 2023-09-30T22:31:46.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.095+0000 7f7b4b7fe700 1 -- 172.21.15.16:0/4004804996 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b5801f070 con 0x7f7b6409e1d0 2023-09-30T22:31:46.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.095+0000 7f7b4b7fe700 1 -- 172.21.15.16:0/4004804996 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b58003a00 con 0x7f7b6409e1d0 2023-09-30T22:31:46.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.095+0000 7f7b4b7fe700 1 -- 172.21.15.16:0/4004804996 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7b58003b60 con 0x7f7b6409e1d0 2023-09-30T22:31:46.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.096+0000 7f7b695a4700 1 -- 172.21.15.16:0/4004804996 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b64074ea0 con 0x7f7b6409e1d0 2023-09-30T22:31:46.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.096+0000 7f7b4b7fe700 1 -- 172.21.15.16:0/4004804996 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f7b58021070 con 0x7f7b6409e1d0 2023-09-30T22:31:46.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.103+0000 7fca8190b700 1 Processor -- start 2023-09-30T22:31:46.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.103+0000 7fca8190b700 1 -- start start 2023-09-30T22:31:46.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.104+0000 7fca8190b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca7c09e1d0 0x7fca7c13cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:46.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.104+0000 7fca8190b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fca7c13d480 0x7fca7c13f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:46.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.104+0000 7fca8190b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca7c0992c0 con 0x7fca7c09d6f0 2023-09-30T22:31:46.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.104+0000 7fca8190b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca7c099520 con 0x7fca7c13d480 2023-09-30T22:31:46.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.104+0000 7fca8190b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fca7c0997e0 con 0x7fca7c09e1d0 2023-09-30T22:31:46.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.104+0000 7fca7a7fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca7c09e1d0 0x7fca7c13cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:46.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.104+0000 7fca7a7fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca7c09e1d0 0x7fca7c13cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:40042/0 (socket says 172.21.15.16:40042) 2023-09-30T22:31:46.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.104+0000 7fca7a7fc700 1 -- 172.21.15.16:0/2655099669 learned_addr learned my addr 172.21.15.16:0/2655099669 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:46.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.104+0000 7fca7affd700 1 --1- 172.21.15.16:0/2655099669 >> v1:172.21.15.16:6789/0 conn(0x7fca7c09d6f0 0x7fca7c09dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41406/0 (socket says 172.21.15.16:41406) 2023-09-30T22:31:46.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.104+0000 7fca7b7fe700 1 --2- 172.21.15.16:0/2655099669 >> v2:172.21.15.16:3300/0 conn(0x7fca7c13d480 0x7fca7c13f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:46.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.104+0000 7fca7a7fc700 1 -- 172.21.15.16:0/2655099669 >> v2:172.21.15.16:3300/0 conn(0x7fca7c13d480 msgr2=0x7fca7c13f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:46.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.104+0000 7fca7a7fc700 1 --2- 172.21.15.16:0/2655099669 >> v2:172.21.15.16:3300/0 conn(0x7fca7c13d480 0x7fca7c13f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:46.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.104+0000 7fca7a7fc700 1 -- 172.21.15.16:0/2655099669 >> v1:172.21.15.16:6789/0 conn(0x7fca7c09d6f0 legacy=0x7fca7c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:46.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.104+0000 7fca7a7fc700 1 -- 172.21.15.16:0/2655099669 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca7c099b30 con 0x7fca7c09e1d0 2023-09-30T22:31:46.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.105+0000 7fca7a7fc700 1 --2- 172.21.15.16:0/2655099669 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca7c09e1d0 0x7fca7c13cf40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fca700098f0 tx=0x7fca70009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b57f411a95389f8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:46.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.105+0000 7fca79ffb700 1 -- 172.21.15.16:0/2655099669 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7000b8a0 con 0x7fca7c09e1d0 2023-09-30T22:31:46.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.105+0000 7fca79ffb700 1 -- 172.21.15.16:0/2655099669 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca7000ba00 con 0x7fca7c09e1d0 2023-09-30T22:31:46.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.105+0000 7fca79ffb700 1 -- 172.21.15.16:0/2655099669 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7000bd90 con 0x7fca7c09e1d0 2023-09-30T22:31:46.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.105+0000 7f7b4b7fe700 1 -- 172.21.15.16:0/4004804996 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7b58003e10 con 0x7f7b6409e1d0 2023-09-30T22:31:46.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.105+0000 7fca8190b700 1 -- 172.21.15.16:0/2655099669 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca7c09e1d0 msgr2=0x7fca7c13cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:46.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.105+0000 7fca8190b700 1 --2- 172.21.15.16:0/2655099669 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca7c09e1d0 0x7fca7c13cf40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fca700098f0 tx=0x7fca70009c00 comp rx=0 tx=0).stop 2023-09-30T22:31:46.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.106+0000 7fca8190b700 1 -- 172.21.15.16:0/2655099669 shutdown_connections 2023-09-30T22:31:46.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.106+0000 7fca8190b700 1 --2- 172.21.15.16:0/2655099669 >> v2:172.21.15.16:3300/0 conn(0x7fca7c13d480 0x7fca7c13f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:46.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.106+0000 7fca8190b700 1 --2- 172.21.15.16:0/2655099669 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca7c09e1d0 0x7fca7c13cf40 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:46.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.106+0000 7fca8190b700 1 -- 172.21.15.16:0/2655099669 >> 172.21.15.16:0/2655099669 conn(0x7fca7c09f470 msgr2=0x7fca7c0a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:46.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.106+0000 7fca8190b700 1 -- 172.21.15.16:0/2655099669 shutdown_connections 2023-09-30T22:31:46.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.106+0000 7fca8190b700 1 -- 172.21.15.16:0/2655099669 wait complete. 2023-09-30T22:31:46.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.107+0000 7fca8190b700 1 Processor -- start 2023-09-30T22:31:46.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.107+0000 7fca8190b700 1 -- start start 2023-09-30T22:31:46.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.108+0000 7fca8190b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca7c09d6f0 0x7fca7c070660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:46.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.108+0000 7fca8190b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fca7c09e1d0 0x7fca7c070ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:46.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.108+0000 7fca8190b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca7c13d480 0x7fca7c0710e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:46.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.108+0000 7fca8190b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca7c128860 con 0x7fca7c09d6f0 2023-09-30T22:31:46.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.108+0000 7fca8190b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca7c127c30 con 0x7fca7c09e1d0 2023-09-30T22:31:46.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.108+0000 7fca8190b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fca7c13ef50 con 0x7fca7c13d480 2023-09-30T22:31:46.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.108+0000 7fca7affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca7c09d6f0 0x7fca7c070660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:46.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.108+0000 7fca7b7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca7c13d480 0x7fca7c0710e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:46.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.108+0000 7fca7affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca7c09d6f0 0x7fca7c070660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48342/0 (socket says 172.21.15.16:48342) 2023-09-30T22:31:46.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.108+0000 7fca7affd700 1 -- 172.21.15.16:0/3965662075 learned_addr learned my addr 172.21.15.16:0/3965662075 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:46.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.109+0000 7fca7affd700 1 -- 172.21.15.16:0/3965662075 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca7c13d480 msgr2=0x7fca7c0710e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:46.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.109+0000 7fca7affd700 1 --2- 172.21.15.16:0/3965662075 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca7c13d480 0x7fca7c0710e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:46.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.109+0000 7fca7affd700 1 -- 172.21.15.16:0/3965662075 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fca7c09e1d0 msgr2=0x7fca7c070ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:46.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.109+0000 7fca7affd700 1 --2- 172.21.15.16:0/3965662075 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fca7c09e1d0 0x7fca7c070ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:46.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.109+0000 7fca7affd700 1 -- 172.21.15.16:0/3965662075 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca70009550 con 0x7fca7c09d6f0 2023-09-30T22:31:46.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.109+0000 7fca7b7fe700 1 --2- 172.21.15.16:0/3965662075 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca7c13d480 0x7fca7c0710e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:46.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.109+0000 7fca7affd700 1 --2- 172.21.15.16:0/3965662075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca7c09d6f0 0x7fca7c070660 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fca6c00c520 tx=0x7fca6c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:46.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.109+0000 7fca63fff700 1 -- 172.21.15.16:0/3965662075 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca6c005cb0 con 0x7fca7c09d6f0 2023-09-30T22:31:46.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.109+0000 7fca63fff700 1 -- 172.21.15.16:0/3965662075 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca6c005e10 con 0x7fca7c09d6f0 2023-09-30T22:31:46.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.109+0000 7fca63fff700 1 -- 172.21.15.16:0/3965662075 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca6c0072d0 con 0x7fca7c09d6f0 2023-09-30T22:31:46.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.109+0000 7fca8190b700 1 -- 172.21.15.16:0/3965662075 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca7c1271b0 con 0x7fca7c09d6f0 2023-09-30T22:31:46.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.109+0000 7fca8190b700 1 -- 172.21.15.16:0/3965662075 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca7c127ff0 con 0x7fca7c09d6f0 2023-09-30T22:31:46.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.110+0000 7fca63fff700 1 -- 172.21.15.16:0/3965662075 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fca6c007430 con 0x7fca7c09d6f0 2023-09-30T22:31:46.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.111+0000 7fca63fff700 1 -- 172.21.15.16:0/3965662075 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fca6c03ebd0 con 0x7fca7c09d6f0 2023-09-30T22:31:46.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.111+0000 7fca8190b700 1 -- 172.21.15.16:0/3965662075 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca7c0747f0 con 0x7fca7c09d6f0 2023-09-30T22:31:46.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.120+0000 7fca63fff700 1 -- 172.21.15.16:0/3965662075 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fca6c017080 con 0x7fca7c09d6f0 2023-09-30T22:31:46.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.194+0000 7f5d5759f700 1 -- 172.21.15.16:0/1726352825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5d50074ea0 con 0x7f5d5006ff90 2023-09-30T22:31:46.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.195+0000 7f5d45ffb700 1 -- 172.21.15.16:0/1726352825 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5d400173a0 con 0x7f5d5006ff90 2023-09-30T22:31:46.195 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:31:46.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.197+0000 7f5d5759f700 1 -- 172.21.15.16:0/1726352825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d5006ff90 msgr2=0x7f5d5011be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:46.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.197+0000 7f5d5759f700 1 --2- 172.21.15.16:0/1726352825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d5006ff90 0x7f5d5011be40 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f5d4000fde0 tx=0x7f5d4000d630 comp rx=0 tx=0).stop 2023-09-30T22:31:46.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.197+0000 7f5d5759f700 1 -- 172.21.15.16:0/1726352825 shutdown_connections 2023-09-30T22:31:46.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.197+0000 7f5d5759f700 1 --2- 172.21.15.16:0/1726352825 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d5006f660 0x7f5d5011b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:46.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.197+0000 7f5d5759f700 1 --2- 172.21.15.16:0/1726352825 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d501305f0 0x7f5d5011c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:46.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.197+0000 7f5d5759f700 1 --2- 172.21.15.16:0/1726352825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d5006ff90 0x7f5d5011be40 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:46.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.197+0000 7f5d5759f700 1 -- 172.21.15.16:0/1726352825 >> 172.21.15.16:0/1726352825 conn(0x7f5d500a2aa0 msgr2=0x7f5d5006e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:46.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.197+0000 7f5d5759f700 1 -- 172.21.15.16:0/1726352825 shutdown_connections 2023-09-30T22:31:46.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.197+0000 7f5d5759f700 1 -- 172.21.15.16:0/1726352825 wait complete. 2023-09-30T22:31:46.212 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:31:46.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.278+0000 7fa996453700 1 -- 172.21.15.16:0/2595763062 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa99006c360 con 0x7fa99009d6f0 2023-09-30T22:31:46.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.278+0000 7fa98cff9700 1 -- 172.21.15.16:0/2595763062 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa980021070 con 0x7fa99009d6f0 2023-09-30T22:31:46.279 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:31:46.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.281+0000 7fa996453700 1 -- 172.21.15.16:0/2595763062 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa99009d6f0 msgr2=0x7fa990137570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:46.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.281+0000 7fa996453700 1 --2- 172.21.15.16:0/2595763062 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa99009d6f0 0x7fa990137570 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa98000ebe0 tx=0x7fa98000eef0 comp rx=0 tx=0).stop 2023-09-30T22:31:46.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.281+0000 7fa996453700 1 -- 172.21.15.16:0/2595763062 shutdown_connections 2023-09-30T22:31:46.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.281+0000 7fa996453700 1 --2- 172.21.15.16:0/2595763062 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa99009e410 0x7fa990137ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:46.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.281+0000 7fa996453700 1 --2- 172.21.15.16:0/2595763062 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa99009d6f0 0x7fa990137570 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:46.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.281+0000 7fa996453700 1 --2- 172.21.15.16:0/2595763062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa99009e000 0x7fa990137ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:46.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.281+0000 7fa996453700 1 -- 172.21.15.16:0/2595763062 >> 172.21.15.16:0/2595763062 conn(0x7fa990066f50 msgr2=0x7fa9900a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:46.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.281+0000 7fa996453700 1 -- 172.21.15.16:0/2595763062 shutdown_connections 2023-09-30T22:31:46.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.281+0000 7fa996453700 1 -- 172.21.15.16:0/2595763062 wait complete. 2023-09-30T22:31:46.295 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:31:46.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.349+0000 7f7b695a4700 1 -- 172.21.15.16:0/4004804996 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7b6406c360 con 0x7f7b6409e1d0 2023-09-30T22:31:46.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.349+0000 7f7b4b7fe700 1 -- 172.21.15.16:0/4004804996 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7b58010b70 con 0x7f7b6409e1d0 2023-09-30T22:31:46.350 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:31:46.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.352+0000 7f7b695a4700 1 -- 172.21.15.16:0/4004804996 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7b6409e1d0 msgr2=0x7f7b641c8a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:46.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.352+0000 7f7b695a4700 1 --2- 172.21.15.16:0/4004804996 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7b6409e1d0 0x7f7b641c8a50 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7b58000c00 tx=0x7f7b58004750 comp rx=0 tx=0).stop 2023-09-30T22:31:46.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.352+0000 7f7b695a4700 1 -- 172.21.15.16:0/4004804996 shutdown_connections 2023-09-30T22:31:46.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.352+0000 7f7b695a4700 1 --2- 172.21.15.16:0/4004804996 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b64139170 0x7f7b641207c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:46.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.352+0000 7f7b695a4700 1 --2- 172.21.15.16:0/4004804996 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7b6409d6f0 0x7f7b641c8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:46.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.352+0000 7f7b695a4700 1 --2- 172.21.15.16:0/4004804996 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7b6409e1d0 0x7f7b641c8a50 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:46.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.352+0000 7f7b695a4700 1 -- 172.21.15.16:0/4004804996 >> 172.21.15.16:0/4004804996 conn(0x7f7b64066f50 msgr2=0x7f7b6409b410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:46.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.352+0000 7f7b695a4700 1 -- 172.21.15.16:0/4004804996 shutdown_connections 2023-09-30T22:31:46.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.352+0000 7f7b695a4700 1 -- 172.21.15.16:0/4004804996 wait complete. 2023-09-30T22:31:46.371 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:31:46.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.370+0000 7fca8190b700 1 -- 172.21.15.16:0/3965662075 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fca7c097de0 con 0x7fca7c09d6f0 2023-09-30T22:31:46.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.371+0000 7fca63fff700 1 -- 172.21.15.16:0/3965662075 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fca6c012400 con 0x7fca7c09d6f0 2023-09-30T22:31:46.371 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:31:46.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.373+0000 7fca8190b700 1 -- 172.21.15.16:0/3965662075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca7c09d6f0 msgr2=0x7fca7c070660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:46.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.373+0000 7fca8190b700 1 --2- 172.21.15.16:0/3965662075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca7c09d6f0 0x7fca7c070660 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fca6c00c520 tx=0x7fca6c00c830 comp rx=0 tx=0).stop 2023-09-30T22:31:46.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.373+0000 7fca8190b700 1 -- 172.21.15.16:0/3965662075 shutdown_connections 2023-09-30T22:31:46.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.373+0000 7fca8190b700 1 --2- 172.21.15.16:0/3965662075 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca7c13d480 0x7fca7c0710e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:46.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.373+0000 7fca8190b700 1 --2- 172.21.15.16:0/3965662075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca7c09d6f0 0x7fca7c070660 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:46.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.373+0000 7fca8190b700 1 --2- 172.21.15.16:0/3965662075 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fca7c09e1d0 0x7fca7c070ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:46.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.373+0000 7fca8190b700 1 -- 172.21.15.16:0/3965662075 >> 172.21.15.16:0/3965662075 conn(0x7fca7c09f470 msgr2=0x7fca7c13d850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:46.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.373+0000 7fca8190b700 1 -- 172.21.15.16:0/3965662075 shutdown_connections 2023-09-30T22:31:46.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:46.373+0000 7fca8190b700 1 -- 172.21.15.16:0/3965662075 wait complete. 2023-09-30T22:31:46.387 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:31:47.213 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:47.296 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:47.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.362+0000 7fb12423b700 1 Processor -- start 2023-09-30T22:31:47.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.363+0000 7fb12423b700 1 -- start start 2023-09-30T22:31:47.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.363+0000 7fb12423b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb11c0a0a30 0x7fb11c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:47.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.363+0000 7fb12423b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb11c134170 0x7fb11c13bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:47.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.363+0000 7fb12423b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb11c09edd0 con 0x7fb11c0a1340 2023-09-30T22:31:47.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.363+0000 7fb12423b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb11c09e1f0 con 0x7fb11c134170 2023-09-30T22:31:47.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.363+0000 7fb12423b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb11c09dc50 con 0x7fb11c0a0a30 2023-09-30T22:31:47.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.363+0000 7fb121fd7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb11c0a0a30 0x7fb11c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:47.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.363+0000 7fb121fd7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb11c0a0a30 0x7fb11c0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:40062/0 (socket says 172.21.15.16:40062) 2023-09-30T22:31:47.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.363+0000 7fb121fd7700 1 -- 172.21.15.16:0/3070817310 learned_addr learned my addr 172.21.15.16:0/3070817310 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:47.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.363+0000 7fb1217d6700 1 --1- 172.21.15.16:0/3070817310 >> v1:172.21.15.16:6789/0 conn(0x7fb11c0a1340 0x7fb11c133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:41412/0 (socket says 172.21.15.16:41412) 2023-09-30T22:31:47.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.363+0000 7fb1227d8700 1 --2- 172.21.15.16:0/3070817310 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb11c134170 0x7fb11c13bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:47.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.364+0000 7fb121fd7700 1 -- 172.21.15.16:0/3070817310 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb11c134170 msgr2=0x7fb11c13bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:47.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.364+0000 7fb121fd7700 1 --2- 172.21.15.16:0/3070817310 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb11c134170 0x7fb11c13bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:47.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.364+0000 7fb120fd5700 1 -- 172.21.15.16:0/3070817310 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1006102836 0 0) 0x7fb11c09edd0 con 0x7fb11c0a1340 2023-09-30T22:31:47.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.364+0000 7fb121fd7700 1 -- 172.21.15.16:0/3070817310 >> v1:172.21.15.16:6789/0 conn(0x7fb11c0a1340 legacy=0x7fb11c133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:47.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.364+0000 7fb121fd7700 1 -- 172.21.15.16:0/3070817310 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb11c0a1fe0 con 0x7fb11c0a0a30 2023-09-30T22:31:47.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.364+0000 7fb121fd7700 1 --2- 172.21.15.16:0/3070817310 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb11c0a0a30 0x7fb11c0a0e00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb10c009e80 tx=0x7fb10c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f03e47ca50c1289a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:47.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.364+0000 7fb120fd5700 1 -- 172.21.15.16:0/3070817310 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10c00e070 con 0x7fb11c0a0a30 2023-09-30T22:31:47.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.364+0000 7fb120fd5700 1 -- 172.21.15.16:0/3070817310 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb10c00b910 con 0x7fb11c0a0a30 2023-09-30T22:31:47.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.364+0000 7fb120fd5700 1 -- 172.21.15.16:0/3070817310 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10c00bcc0 con 0x7fb11c0a0a30 2023-09-30T22:31:47.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.365+0000 7fb12423b700 1 -- 172.21.15.16:0/3070817310 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb11c0a0a30 msgr2=0x7fb11c0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:47.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.365+0000 7fb12423b700 1 --2- 172.21.15.16:0/3070817310 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb11c0a0a30 0x7fb11c0a0e00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb10c009e80 tx=0x7fb10c00b330 comp rx=0 tx=0).stop 2023-09-30T22:31:47.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.365+0000 7fb12423b700 1 -- 172.21.15.16:0/3070817310 shutdown_connections 2023-09-30T22:31:47.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.365+0000 7fb12423b700 1 --2- 172.21.15.16:0/3070817310 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb11c134170 0x7fb11c13bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:47.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.365+0000 7fb12423b700 1 --2- 172.21.15.16:0/3070817310 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb11c0a0a30 0x7fb11c0a0e00 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:47.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.365+0000 7fb12423b700 1 -- 172.21.15.16:0/3070817310 >> 172.21.15.16:0/3070817310 conn(0x7fb11c0a6550 msgr2=0x7fb11c0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:47.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.365+0000 7fb12423b700 1 -- 172.21.15.16:0/3070817310 shutdown_connections 2023-09-30T22:31:47.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.365+0000 7fb12423b700 1 -- 172.21.15.16:0/3070817310 wait complete. 2023-09-30T22:31:47.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.366+0000 7fb12423b700 1 Processor -- start 2023-09-30T22:31:47.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.366+0000 7fb12423b700 1 -- start start 2023-09-30T22:31:47.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.367+0000 7fb12423b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb11c0a0a30 0x7fb11c122640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:47.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.367+0000 7fb12423b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb11c0a1340 0x7fb11c122b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:47.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.367+0000 7fb12423b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb11c134170 0x7fb11c1230c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:47.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.367+0000 7fb12423b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb11c1331e0 con 0x7fb11c134170 2023-09-30T22:31:47.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.367+0000 7fb12423b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb11c124360 con 0x7fb11c0a0a30 2023-09-30T22:31:47.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.367+0000 7fb12423b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb11c0a0220 con 0x7fb11c0a1340 2023-09-30T22:31:47.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.367+0000 7fb121fd7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb11c0a0a30 0x7fb11c122640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:47.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.367+0000 7fb1217d6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb11c0a1340 0x7fb11c122b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:47.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.367+0000 7fb121fd7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb11c0a0a30 0x7fb11c122640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:40812/0 (socket says 172.21.15.16:40812) 2023-09-30T22:31:47.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.367+0000 7fb121fd7700 1 -- 172.21.15.16:0/1793041647 learned_addr learned my addr 172.21.15.16:0/1793041647 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:47.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.367+0000 7fb1227d8700 1 --2- 172.21.15.16:0/1793041647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb11c134170 0x7fb11c1230c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:47.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.367+0000 7fb121fd7700 1 -- 172.21.15.16:0/1793041647 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb11c0a1340 msgr2=0x7fb11c122b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:47.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.367+0000 7fb121fd7700 1 --2- 172.21.15.16:0/1793041647 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb11c0a1340 0x7fb11c122b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:47.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.367+0000 7fb121fd7700 1 -- 172.21.15.16:0/1793041647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb11c134170 msgr2=0x7fb11c1230c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:47.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.367+0000 7fb121fd7700 1 --2- 172.21.15.16:0/1793041647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb11c134170 0x7fb11c1230c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:47.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.367+0000 7fb121fd7700 1 -- 172.21.15.16:0/1793041647 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb10c009b30 con 0x7fb11c0a0a30 2023-09-30T22:31:47.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.367+0000 7fb1217d6700 1 --2- 172.21.15.16:0/1793041647 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb11c0a1340 0x7fb11c122b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:47.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.367+0000 7fb121fd7700 1 --2- 172.21.15.16:0/1793041647 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb11c0a0a30 0x7fb11c122640 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb10c00b680 tx=0x7fb10c004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:47.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.368+0000 7fb11a7fc700 1 -- 172.21.15.16:0/1793041647 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10c0099b0 con 0x7fb11c0a0a30 2023-09-30T22:31:47.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.368+0000 7fb11a7fc700 1 -- 172.21.15.16:0/1793041647 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb10c0092e0 con 0x7fb11c0a0a30 2023-09-30T22:31:47.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.368+0000 7fb12423b700 1 -- 172.21.15.16:0/1793041647 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb11c0a25e0 con 0x7fb11c0a0a30 2023-09-30T22:31:47.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.368+0000 7fb11a7fc700 1 -- 172.21.15.16:0/1793041647 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10c004450 con 0x7fb11c0a0a30 2023-09-30T22:31:47.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.368+0000 7fb12423b700 1 -- 172.21.15.16:0/1793041647 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb11c126160 con 0x7fb11c0a0a30 2023-09-30T22:31:47.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.369+0000 7fb11a7fc700 1 -- 172.21.15.16:0/1793041647 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb10c0045b0 con 0x7fb11c0a0a30 2023-09-30T22:31:47.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.369+0000 7fb12423b700 1 -- 172.21.15.16:0/1793041647 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb11c074ea0 con 0x7fb11c0a0a30 2023-09-30T22:31:47.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.369+0000 7fb11a7fc700 1 -- 172.21.15.16:0/1793041647 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fb10c0409e0 con 0x7fb11c0a0a30 2023-09-30T22:31:47.372 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:47.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.378+0000 7fb11a7fc700 1 -- 172.21.15.16:0/1793041647 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb10c011390 con 0x7fb11c0a0a30 2023-09-30T22:31:47.387 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:47.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.455+0000 7ffa1969b700 1 Processor -- start 2023-09-30T22:31:47.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.455+0000 7ffa1969b700 1 -- start start 2023-09-30T22:31:47.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.456+0000 7ffa1969b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ffa1409d6f0 0x7ffa1409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:47.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.456+0000 7ffa1969b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa1409e000 0x7ffa141339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:47.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.456+0000 7ffa1969b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa1409e410 0x7ffa14133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:47.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.456+0000 7ffa1969b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa1406b460 con 0x7ffa1409d6f0 2023-09-30T22:31:47.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.456+0000 7ffa1969b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1409b120 con 0x7ffa1409e410 2023-09-30T22:31:47.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.456+0000 7ffa1969b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1409a7e0 con 0x7ffa1409e000 2023-09-30T22:31:47.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.456+0000 7ffa127fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa1409e000 0x7ffa141339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:47.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.456+0000 7ffa127fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa1409e000 0x7ffa141339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:60008/0 (socket says 172.21.15.16:60008) 2023-09-30T22:31:47.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.456+0000 7ffa12ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ffa1409d6f0 0x7ffa1409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:47.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.456+0000 7ffa127fc700 1 -- 172.21.15.16:0/4064271878 learned_addr learned my addr 172.21.15.16:0/4064271878 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:47.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.457+0000 7ffa137fe700 1 --2- 172.21.15.16:0/4064271878 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa1409e410 0x7ffa14133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:47.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.457+0000 7ffa137fe700 1 -- 172.21.15.16:0/4064271878 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa1409e000 msgr2=0x7ffa141339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:47.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.457+0000 7ffa137fe700 1 --2- 172.21.15.16:0/4064271878 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa1409e000 0x7ffa141339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:47.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.457+0000 7ffa137fe700 1 -- 172.21.15.16:0/4064271878 >> v2:172.21.15.16:3300/0 conn(0x7ffa1409d6f0 msgr2=0x7ffa1409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:47.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.457+0000 7ffa137fe700 1 --2- 172.21.15.16:0/4064271878 >> v2:172.21.15.16:3300/0 conn(0x7ffa1409d6f0 0x7ffa1409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:47.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.457+0000 7ffa137fe700 1 -- 172.21.15.16:0/4064271878 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa1409a500 con 0x7ffa1409e410 2023-09-30T22:31:47.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.457+0000 7ffa12ffd700 1 --2- 172.21.15.16:0/4064271878 >> v2:172.21.15.16:3300/0 conn(0x7ffa1409d6f0 0x7ffa1409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:47.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.457+0000 7ffa137fe700 1 --2- 172.21.15.16:0/4064271878 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa1409e410 0x7ffa14133f30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c009dc0 tx=0x7ffa0c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdd91db57afa4d7c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:47.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.457+0000 7ffa11ffb700 1 -- 172.21.15.16:0/4064271878 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c00e070 con 0x7ffa1409e410 2023-09-30T22:31:47.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.457+0000 7ffa11ffb700 1 -- 172.21.15.16:0/4064271878 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa0c00bae0 con 0x7ffa1409e410 2023-09-30T22:31:47.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.457+0000 7ffa11ffb700 1 -- 172.21.15.16:0/4064271878 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c00be90 con 0x7ffa1409e410 2023-09-30T22:31:47.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.458+0000 7ffa1969b700 1 -- 172.21.15.16:0/4064271878 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa1409e410 msgr2=0x7ffa14133f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:47.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.458+0000 7ffa1969b700 1 --2- 172.21.15.16:0/4064271878 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa1409e410 0x7ffa14133f30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c009dc0 tx=0x7ffa0c00b330 comp rx=0 tx=0).stop 2023-09-30T22:31:47.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.459+0000 7ffa1969b700 1 -- 172.21.15.16:0/4064271878 shutdown_connections 2023-09-30T22:31:47.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.459+0000 7ffa1969b700 1 --2- 172.21.15.16:0/4064271878 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa1409e410 0x7ffa14133f30 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:47.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.459+0000 7ffa1969b700 1 --2- 172.21.15.16:0/4064271878 >> v2:172.21.15.16:3300/0 conn(0x7ffa1409d6f0 0x7ffa1409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:47.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.459+0000 7ffa1969b700 1 --2- 172.21.15.16:0/4064271878 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa1409e000 0x7ffa141339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:47.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.459+0000 7ffa1969b700 1 -- 172.21.15.16:0/4064271878 >> 172.21.15.16:0/4064271878 conn(0x7ffa14066f50 msgr2=0x7ffa1409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:47.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.459+0000 7ffa1969b700 1 -- 172.21.15.16:0/4064271878 shutdown_connections 2023-09-30T22:31:47.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.459+0000 7ffa1969b700 1 -- 172.21.15.16:0/4064271878 wait complete. 2023-09-30T22:31:47.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.460+0000 7ffa1969b700 1 Processor -- start 2023-09-30T22:31:47.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.460+0000 7ffa1969b700 1 -- start start 2023-09-30T22:31:47.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.461+0000 7ffa1969b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa1409d6f0 0x7ffa141a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:47.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.461+0000 7ffa1969b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa1409e000 0x7ffa141a3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:47.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.461+0000 7ffa1969b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa1409e410 0x7ffa141a4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:47.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.461+0000 7ffa1969b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa141a5260 con 0x7ffa1409d6f0 2023-09-30T22:31:47.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.461+0000 7ffa1969b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa140a2000 con 0x7ffa1409e000 2023-09-30T22:31:47.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.461+0000 7ffa1969b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1409c710 con 0x7ffa1409e410 2023-09-30T22:31:47.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.461+0000 7ffa127fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa1409e000 0x7ffa141a3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:47.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.461+0000 7ffa137fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa1409e410 0x7ffa141a4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:47.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.461+0000 7ffa127fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa1409e000 0x7ffa141a3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:32978/0 (socket says 172.21.15.16:32978) 2023-09-30T22:31:47.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.461+0000 7ffa127fc700 1 -- 172.21.15.16:0/1283307908 learned_addr learned my addr 172.21.15.16:0/1283307908 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:47.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.461+0000 7ffa12ffd700 1 --2- 172.21.15.16:0/1283307908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa1409d6f0 0x7ffa141a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:47.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.461+0000 7ffa127fc700 1 -- 172.21.15.16:0/1283307908 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa1409e410 msgr2=0x7ffa141a4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:47.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.461+0000 7ffa127fc700 1 --2- 172.21.15.16:0/1283307908 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa1409e410 0x7ffa141a4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:47.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.461+0000 7ffa127fc700 1 -- 172.21.15.16:0/1283307908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa1409d6f0 msgr2=0x7ffa141a36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:47.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.461+0000 7ffa127fc700 1 --2- 172.21.15.16:0/1283307908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa1409d6f0 0x7ffa141a36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:47.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.461+0000 7ffa127fc700 1 -- 172.21.15.16:0/1283307908 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa040094a0 con 0x7ffa1409e000 2023-09-30T22:31:47.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.461+0000 7ffa137fe700 1 --2- 172.21.15.16:0/1283307908 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa1409e410 0x7ffa141a4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:47.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.461+0000 7ffa12ffd700 1 --2- 172.21.15.16:0/1283307908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa1409d6f0 0x7ffa141a36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:31:47.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.462+0000 7ffa127fc700 1 --2- 172.21.15.16:0/1283307908 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa1409e000 0x7ffa141a3c30 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ffa0800e940 tx=0x7ffa0800ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:47.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.462+0000 7ff9fb7fe700 1 -- 172.21.15.16:0/1283307908 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa080096a0 con 0x7ffa1409e000 2023-09-30T22:31:47.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.462+0000 7ff9fb7fe700 1 -- 172.21.15.16:0/1283307908 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa0800ee90 con 0x7ffa1409e000 2023-09-30T22:31:47.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.462+0000 7ffa1969b700 1 -- 172.21.15.16:0/1283307908 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa0c009a70 con 0x7ffa1409e000 2023-09-30T22:31:47.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.462+0000 7ff9fb7fe700 1 -- 172.21.15.16:0/1283307908 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0800ca90 con 0x7ffa1409e000 2023-09-30T22:31:47.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.462+0000 7ffa1969b700 1 -- 172.21.15.16:0/1283307908 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa141a5a50 con 0x7ffa1409e000 2023-09-30T22:31:47.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.463+0000 7ff9fb7fe700 1 -- 172.21.15.16:0/1283307908 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ffa0800cbf0 con 0x7ffa1409e000 2023-09-30T22:31:47.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.463+0000 7ff9fb7fe700 1 -- 172.21.15.16:0/1283307908 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7ffa0803fa40 con 0x7ffa1409e000 2023-09-30T22:31:47.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.464+0000 7ffa1969b700 1 -- 172.21.15.16:0/1283307908 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa141a6b30 con 0x7ffa1409e000 2023-09-30T22:31:47.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.473+0000 7ff9fb7fe700 1 -- 172.21.15.16:0/1283307908 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ffa141a6b30 con 0x7ffa1409e000 2023-09-30T22:31:47.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.519+0000 7f0ac2425700 1 Processor -- start 2023-09-30T22:31:47.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.520+0000 7f0ac2425700 1 -- start start 2023-09-30T22:31:47.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.520+0000 7f0ac2425700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0abc134550 0x7f0abc136930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:47.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.520+0000 7f0ac2425700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0abc136e70 0x7f0abc139250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:47.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.520+0000 7f0ac2425700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0abc06f5d0 con 0x7f0abc131730 2023-09-30T22:31:47.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.520+0000 7f0ac2425700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0abc06f9a0 con 0x7f0abc134550 2023-09-30T22:31:47.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.520+0000 7f0ac2425700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0abc1312e0 con 0x7f0abc136e70 2023-09-30T22:31:47.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.520+0000 7f0ac09c2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0abc136e70 0x7f0abc139250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:47.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.520+0000 7f0ac09c2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0abc136e70 0x7f0abc139250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:32980/0 (socket says 172.21.15.16:32980) 2023-09-30T22:31:47.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.520+0000 7f0ac09c2700 1 -- 172.21.15.16:0/3687662673 learned_addr learned my addr 172.21.15.16:0/3687662673 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:47.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.520+0000 7f0abb7fe700 1 --2- 172.21.15.16:0/3687662673 >> v2:172.21.15.16:3300/0 conn(0x7f0abc134550 0x7f0abc136930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:47.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.521+0000 7f0ac09c2700 1 -- 172.21.15.16:0/3687662673 >> v2:172.21.15.16:3300/0 conn(0x7f0abc134550 msgr2=0x7f0abc136930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:47.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.521+0000 7f0ac09c2700 1 --2- 172.21.15.16:0/3687662673 >> v2:172.21.15.16:3300/0 conn(0x7f0abc134550 0x7f0abc136930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:47.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.521+0000 7f0ac09c2700 1 -- 172.21.15.16:0/3687662673 >> v1:172.21.15.16:6789/0 conn(0x7f0abc131730 legacy=0x7f0abc131b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:47.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.521+0000 7f0abaffd700 1 -- 172.21.15.16:0/3687662673 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2454362890 0 0) 0x7f0abc06f5d0 con 0x7f0abc131730 2023-09-30T22:31:47.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.521+0000 7f0ac09c2700 1 -- 172.21.15.16:0/3687662673 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0abc09b630 con 0x7f0abc136e70 2023-09-30T22:31:47.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.521+0000 7f0ac09c2700 1 --2- 172.21.15.16:0/3687662673 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0abc136e70 0x7f0abc139250 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4009e50 tx=0x7f0ab400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7805288355784f8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:47.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.521+0000 7f0abaffd700 1 -- 172.21.15.16:0/3687662673 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab400e070 con 0x7f0abc136e70 2023-09-30T22:31:47.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.521+0000 7f0abaffd700 1 -- 172.21.15.16:0/3687662673 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab400b910 con 0x7f0abc136e70 2023-09-30T22:31:47.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.521+0000 7f0abaffd700 1 -- 172.21.15.16:0/3687662673 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab400bcc0 con 0x7f0abc136e70 2023-09-30T22:31:47.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.522+0000 7f0ac2425700 1 -- 172.21.15.16:0/3687662673 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0abc136e70 msgr2=0x7f0abc139250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:47.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.522+0000 7f0ac2425700 1 --2- 172.21.15.16:0/3687662673 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0abc136e70 0x7f0abc139250 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4009e50 tx=0x7f0ab400b330 comp rx=0 tx=0).stop 2023-09-30T22:31:47.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.522+0000 7f0ac2425700 1 -- 172.21.15.16:0/3687662673 shutdown_connections 2023-09-30T22:31:47.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.522+0000 7f0ac2425700 1 --2- 172.21.15.16:0/3687662673 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0abc136e70 0x7f0abc139250 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:47.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.522+0000 7f0ac2425700 1 --2- 172.21.15.16:0/3687662673 >> v2:172.21.15.16:3300/0 conn(0x7f0abc134550 0x7f0abc136930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:47.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.522+0000 7f0ac2425700 1 -- 172.21.15.16:0/3687662673 >> 172.21.15.16:0/3687662673 conn(0x7f0abc0a27f0 msgr2=0x7f0abc0a2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:47.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.522+0000 7f0ac2425700 1 -- 172.21.15.16:0/3687662673 shutdown_connections 2023-09-30T22:31:47.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.522+0000 7f0ac2425700 1 -- 172.21.15.16:0/3687662673 wait complete. 2023-09-30T22:31:47.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.524+0000 7f0ac2425700 1 Processor -- start 2023-09-30T22:31:47.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.524+0000 7f0ac2425700 1 -- start start 2023-09-30T22:31:47.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.524+0000 7f0ac2425700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0abc131730 0x7f0abc118c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:47.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.524+0000 7f0ac2425700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0abc134550 0x7f0abc1191a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:47.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.525+0000 7f0ac2425700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0abc136e70 0x7f0abc1196e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:47.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.525+0000 7f0ac2425700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0abc11b010 con 0x7f0abc136e70 2023-09-30T22:31:47.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.525+0000 7f0ac2425700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0abc11c650 con 0x7f0abc134550 2023-09-30T22:31:47.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.525+0000 7f0ac2425700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0abc136bd0 con 0x7f0abc131730 2023-09-30T22:31:47.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.525+0000 7f0abb7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0abc134550 0x7f0abc1191a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:47.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.525+0000 7f0abb7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0abc134550 0x7f0abc1191a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:32992/0 (socket says 172.21.15.16:32992) 2023-09-30T22:31:47.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.525+0000 7f0ac09c2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0abc136e70 0x7f0abc1196e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:47.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.525+0000 7f0abb7fe700 1 -- 172.21.15.16:0/287590767 learned_addr learned my addr 172.21.15.16:0/287590767 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:47.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.525+0000 7f0abb7fe700 1 -- 172.21.15.16:0/287590767 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0abc131730 msgr2=0x7f0abc118c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:47.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.525+0000 7f0abb7fe700 1 --2- 172.21.15.16:0/287590767 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0abc131730 0x7f0abc118c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:47.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.525+0000 7f0abb7fe700 1 -- 172.21.15.16:0/287590767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0abc136e70 msgr2=0x7f0abc1196e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:47.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.525+0000 7f0abb7fe700 1 --2- 172.21.15.16:0/287590767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0abc136e70 0x7f0abc1196e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:47.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.525+0000 7f0abb7fe700 1 -- 172.21.15.16:0/287590767 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab4009b30 con 0x7f0abc134550 2023-09-30T22:31:47.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.525+0000 7f0abb7fe700 1 --2- 172.21.15.16:0/287590767 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0abc134550 0x7f0abc1191a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0ab000e9b0 tx=0x7f0ab000ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:47.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.525+0000 7f0ab8ff9700 1 -- 172.21.15.16:0/287590767 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab0009710 con 0x7f0abc134550 2023-09-30T22:31:47.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.525+0000 7f0ab8ff9700 1 -- 172.21.15.16:0/287590767 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ab000c850 con 0x7f0abc134550 2023-09-30T22:31:47.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.526+0000 7f0ac2425700 1 -- 172.21.15.16:0/287590767 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0abc09af60 con 0x7f0abc134550 2023-09-30T22:31:47.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.526+0000 7f0ac2425700 1 -- 172.21.15.16:0/287590767 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0abc11be30 con 0x7f0abc134550 2023-09-30T22:31:47.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.526+0000 7f0ab8ff9700 1 -- 172.21.15.16:0/287590767 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ab000cbc0 con 0x7f0abc134550 2023-09-30T22:31:47.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.527+0000 7f0ac2425700 1 -- 172.21.15.16:0/287590767 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a9c0052e0 con 0x7f0abc134550 2023-09-30T22:31:47.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.527+0000 7f0ab8ff9700 1 -- 172.21.15.16:0/287590767 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0ab000cd20 con 0x7f0abc134550 2023-09-30T22:31:47.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.528+0000 7f0ab8ff9700 1 -- 172.21.15.16:0/287590767 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f0ab003ff80 con 0x7f0abc134550 2023-09-30T22:31:47.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.532+0000 7f0ab8ff9700 1 -- 172.21.15.16:0/287590767 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0ab001a360 con 0x7f0abc134550 2023-09-30T22:31:47.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.551+0000 7fef05caa700 1 Processor -- start 2023-09-30T22:31:47.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.551+0000 7fef05caa700 1 -- start start 2023-09-30T22:31:47.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.551+0000 7fef05caa700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fef0012cc20 0x7fef0012f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:47.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.551+0000 7fef05caa700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fef0012cff0 0x7fef00137cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:47.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.551+0000 7fef05caa700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef0009a510 con 0x7fef0012c2f0 2023-09-30T22:31:47.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.551+0000 7fef05caa700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef0012c7d0 con 0x7fef0012cff0 2023-09-30T22:31:47.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.551+0000 7fef05caa700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fef0009c190 con 0x7fef0012cc20 2023-09-30T22:31:47.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.552+0000 7feefeffd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fef0012c2f0 0x7fef0012fe70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50248/0 (socket says 172.21.15.16:50248) 2023-09-30T22:31:47.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.552+0000 7feefeffd700 1 -- 172.21.15.16:0/3680033065 learned_addr learned my addr 172.21.15.16:0/3680033065 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:31:47.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.552+0000 7feeff7fe700 1 --2- 172.21.15.16:0/3680033065 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fef0012cc20 0x7fef0012f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:47.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.552+0000 7feefffff700 1 --2- 172.21.15.16:0/3680033065 >> v2:172.21.15.16:3300/0 conn(0x7fef0012cff0 0x7fef00137cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:47.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.552+0000 7feeff7fe700 1 -- 172.21.15.16:0/3680033065 >> v2:172.21.15.16:3300/0 conn(0x7fef0012cff0 msgr2=0x7fef00137cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:47.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.552+0000 7feeff7fe700 1 --2- 172.21.15.16:0/3680033065 >> v2:172.21.15.16:3300/0 conn(0x7fef0012cff0 0x7fef00137cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:47.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.552+0000 7feeff7fe700 1 -- 172.21.15.16:0/3680033065 >> v1:172.21.15.16:6789/0 conn(0x7fef0012c2f0 legacy=0x7fef0012fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:47.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.552+0000 7feeff7fe700 1 -- 172.21.15.16:0/3680033065 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef0009b540 con 0x7fef0012cc20 2023-09-30T22:31:47.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.552+0000 7feeff7fe700 1 --2- 172.21.15.16:0/3680033065 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fef0012cc20 0x7fef0012f930 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7feef000a9e0 tx=0x7feef000acf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1926992d058aa975 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:47.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.552+0000 7feefe7fc700 1 -- 172.21.15.16:0/3680033065 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef000cb70 con 0x7fef0012cc20 2023-09-30T22:31:47.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.553+0000 7feefe7fc700 1 -- 172.21.15.16:0/3680033065 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feef000ccd0 con 0x7fef0012cc20 2023-09-30T22:31:47.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.553+0000 7feefe7fc700 1 -- 172.21.15.16:0/3680033065 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef0018620 con 0x7fef0012cc20 2023-09-30T22:31:47.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.553+0000 7fef05caa700 1 -- 172.21.15.16:0/3680033065 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fef0012cc20 msgr2=0x7fef0012f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:47.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.553+0000 7fef05caa700 1 --2- 172.21.15.16:0/3680033065 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fef0012cc20 0x7fef0012f930 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7feef000a9e0 tx=0x7feef000acf0 comp rx=0 tx=0).stop 2023-09-30T22:31:47.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.553+0000 7fef05caa700 1 -- 172.21.15.16:0/3680033065 shutdown_connections 2023-09-30T22:31:47.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.553+0000 7fef05caa700 1 --2- 172.21.15.16:0/3680033065 >> v2:172.21.15.16:3300/0 conn(0x7fef0012cff0 0x7fef00137cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:47.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.553+0000 7fef05caa700 1 --2- 172.21.15.16:0/3680033065 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fef0012cc20 0x7fef0012f930 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:47.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.553+0000 7fef05caa700 1 -- 172.21.15.16:0/3680033065 >> 172.21.15.16:0/3680033065 conn(0x7fef0009f470 msgr2=0x7fef000a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:47.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.554+0000 7fef05caa700 1 -- 172.21.15.16:0/3680033065 shutdown_connections 2023-09-30T22:31:47.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.554+0000 7fef05caa700 1 -- 172.21.15.16:0/3680033065 wait complete. 2023-09-30T22:31:47.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.555+0000 7fef05caa700 1 Processor -- start 2023-09-30T22:31:47.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.555+0000 7fef05caa700 1 -- start start 2023-09-30T22:31:47.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.555+0000 7fef05caa700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fef0012c2f0 0x7fef0006f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:47.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.555+0000 7fef05caa700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef0012cc20 0x7fef0006f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:47.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.555+0000 7fef05caa700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fef0012cff0 0x7fef0006fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:47.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.555+0000 7fef05caa700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef00071ec0 con 0x7fef0012cc20 2023-09-30T22:31:47.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.555+0000 7fef05caa700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef0012b860 con 0x7fef0012cff0 2023-09-30T22:31:47.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.555+0000 7fef05caa700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fef000719b0 con 0x7fef0012c2f0 2023-09-30T22:31:47.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.555+0000 7feeff7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fef0012c2f0 0x7fef0006f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:47.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.556+0000 7feeff7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fef0012c2f0 0x7fef0006f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:60054/0 (socket says 172.21.15.16:60054) 2023-09-30T22:31:47.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.556+0000 7feeff7fe700 1 -- 172.21.15.16:0/3603510856 learned_addr learned my addr 172.21.15.16:0/3603510856 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:47.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.556+0000 7feefeffd700 1 --2- 172.21.15.16:0/3603510856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef0012cc20 0x7fef0006f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:47.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.556+0000 7feefffff700 1 --2- 172.21.15.16:0/3603510856 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fef0012cff0 0x7fef0006fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:47.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.556+0000 7feeff7fe700 1 -- 172.21.15.16:0/3603510856 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fef0012cff0 msgr2=0x7fef0006fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:47.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.556+0000 7feeff7fe700 1 --2- 172.21.15.16:0/3603510856 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fef0012cff0 0x7fef0006fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:47.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.556+0000 7feeff7fe700 1 -- 172.21.15.16:0/3603510856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef0012cc20 msgr2=0x7fef0006f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:47.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.556+0000 7feeff7fe700 1 --2- 172.21.15.16:0/3603510856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef0012cc20 0x7fef0006f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:47.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.556+0000 7feeff7fe700 1 -- 172.21.15.16:0/3603510856 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feef000a640 con 0x7fef0012c2f0 2023-09-30T22:31:47.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.556+0000 7feeff7fe700 1 --2- 172.21.15.16:0/3603510856 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fef0012c2f0 0x7fef0006f010 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7feef0000c00 tx=0x7feef00047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:47.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.556+0000 7feee7fff700 1 -- 172.21.15.16:0/3603510856 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef0003ab0 con 0x7fef0012c2f0 2023-09-30T22:31:47.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.556+0000 7feee7fff700 1 -- 172.21.15.16:0/3603510856 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feef0003c10 con 0x7fef0012c2f0 2023-09-30T22:31:47.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.556+0000 7fef05caa700 1 -- 172.21.15.16:0/3603510856 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef00072230 con 0x7fef0012c2f0 2023-09-30T22:31:47.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.556+0000 7feee7fff700 1 -- 172.21.15.16:0/3603510856 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef001e4c0 con 0x7fef0012c2f0 2023-09-30T22:31:47.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.557+0000 7fef05caa700 1 -- 172.21.15.16:0/3603510856 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef00071340 con 0x7fef0012c2f0 2023-09-30T22:31:47.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.558+0000 7feee7fff700 1 -- 172.21.15.16:0/3603510856 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7feef001e620 con 0x7fef0012c2f0 2023-09-30T22:31:47.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.558+0000 7fef05caa700 1 -- 172.21.15.16:0/3603510856 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef001285a0 con 0x7fef0012c2f0 2023-09-30T22:31:47.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.558+0000 7feee7fff700 1 -- 172.21.15.16:0/3603510856 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7feef00107f0 con 0x7fef0012c2f0 2023-09-30T22:31:47.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.567+0000 7feee7fff700 1 -- 172.21.15.16:0/3603510856 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7feef004a060 con 0x7fef0012c2f0 2023-09-30T22:31:47.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.625+0000 7fb12423b700 1 -- 172.21.15.16:0/1793041647 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb11c0a28e0 con 0x7fb11c0a0a30 2023-09-30T22:31:47.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.625+0000 7fb11a7fc700 1 -- 172.21.15.16:0/1793041647 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb10c024e60 con 0x7fb11c0a0a30 2023-09-30T22:31:47.626 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:31:47.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.628+0000 7fb12423b700 1 -- 172.21.15.16:0/1793041647 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb11c0a0a30 msgr2=0x7fb11c122640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:47.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.628+0000 7fb12423b700 1 --2- 172.21.15.16:0/1793041647 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb11c0a0a30 0x7fb11c122640 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fb10c00b680 tx=0x7fb10c004ad0 comp rx=0 tx=0).stop 2023-09-30T22:31:47.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.628+0000 7fb12423b700 1 -- 172.21.15.16:0/1793041647 shutdown_connections 2023-09-30T22:31:47.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.628+0000 7fb12423b700 1 --2- 172.21.15.16:0/1793041647 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb11c0a0a30 0x7fb11c122640 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:47.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.628+0000 7fb12423b700 1 --2- 172.21.15.16:0/1793041647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb11c134170 0x7fb11c1230c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:47.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.628+0000 7fb12423b700 1 --2- 172.21.15.16:0/1793041647 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb11c0a1340 0x7fb11c122b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:47.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.628+0000 7fb12423b700 1 -- 172.21.15.16:0/1793041647 >> 172.21.15.16:0/1793041647 conn(0x7fb11c0a6550 msgr2=0x7fb11c09edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:47.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.628+0000 7fb12423b700 1 -- 172.21.15.16:0/1793041647 shutdown_connections 2023-09-30T22:31:47.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.628+0000 7fb12423b700 1 -- 172.21.15.16:0/1793041647 wait complete. 2023-09-30T22:31:47.643 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:31:47.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.684+0000 7ffa1969b700 1 -- 172.21.15.16:0/1283307908 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ffa141a6b30 con 0x7ffa1409e000 2023-09-30T22:31:47.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.685+0000 7ff9fb7fe700 1 -- 172.21.15.16:0/1283307908 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ffa141a6b30 con 0x7ffa1409e000 2023-09-30T22:31:47.685 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:31:47.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.687+0000 7ffa1969b700 1 -- 172.21.15.16:0/1283307908 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa1409e000 msgr2=0x7ffa141a3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:47.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.687+0000 7ffa1969b700 1 --2- 172.21.15.16:0/1283307908 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa1409e000 0x7ffa141a3c30 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ffa0800e940 tx=0x7ffa0800ec50 comp rx=0 tx=0).stop 2023-09-30T22:31:47.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.687+0000 7ffa1969b700 1 -- 172.21.15.16:0/1283307908 shutdown_connections 2023-09-30T22:31:47.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.687+0000 7ffa1969b700 1 --2- 172.21.15.16:0/1283307908 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa1409e410 0x7ffa141a4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:47.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.687+0000 7ffa1969b700 1 --2- 172.21.15.16:0/1283307908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa1409d6f0 0x7ffa141a36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:47.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.687+0000 7ffa1969b700 1 --2- 172.21.15.16:0/1283307908 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa1409e000 0x7ffa141a3c30 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:47.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.687+0000 7ffa1969b700 1 -- 172.21.15.16:0/1283307908 >> 172.21.15.16:0/1283307908 conn(0x7ffa14066f50 msgr2=0x7ffa1406f390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:47.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.687+0000 7ffa1969b700 1 -- 172.21.15.16:0/1283307908 shutdown_connections 2023-09-30T22:31:47.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.687+0000 7ffa1969b700 1 -- 172.21.15.16:0/1283307908 wait complete. 2023-09-30T22:31:47.701 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:31:47.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.751+0000 7f0ac2425700 1 -- 172.21.15.16:0/287590767 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0a9c005860 con 0x7f0abc134550 2023-09-30T22:31:47.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.752+0000 7f0ab8ff9700 1 -- 172.21.15.16:0/287590767 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0ab0017030 con 0x7f0abc134550 2023-09-30T22:31:47.752 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:31:47.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.754+0000 7f0ac2425700 1 -- 172.21.15.16:0/287590767 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0abc134550 msgr2=0x7f0abc1191a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:47.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.754+0000 7f0ac2425700 1 --2- 172.21.15.16:0/287590767 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0abc134550 0x7f0abc1191a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0ab000e9b0 tx=0x7f0ab000ecc0 comp rx=0 tx=0).stop 2023-09-30T22:31:47.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.754+0000 7f0ac2425700 1 -- 172.21.15.16:0/287590767 shutdown_connections 2023-09-30T22:31:47.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.754+0000 7f0ac2425700 1 --2- 172.21.15.16:0/287590767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0abc136e70 0x7f0abc1196e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:47.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.754+0000 7f0ac2425700 1 --2- 172.21.15.16:0/287590767 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0abc131730 0x7f0abc118c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:47.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.754+0000 7f0ac2425700 1 --2- 172.21.15.16:0/287590767 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0abc134550 0x7f0abc1191a0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:47.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.754+0000 7f0ac2425700 1 -- 172.21.15.16:0/287590767 >> 172.21.15.16:0/287590767 conn(0x7f0abc0a27f0 msgr2=0x7f0abc0a13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:47.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.754+0000 7f0ac2425700 1 -- 172.21.15.16:0/287590767 shutdown_connections 2023-09-30T22:31:47.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.754+0000 7f0ac2425700 1 -- 172.21.15.16:0/287590767 wait complete. 2023-09-30T22:31:47.768 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:31:47.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.791+0000 7fef05caa700 1 -- 172.21.15.16:0/3603510856 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fef00129f10 con 0x7fef0012c2f0 2023-09-30T22:31:47.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.791+0000 7feee7fff700 1 -- 172.21.15.16:0/3603510856 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7feef002e950 con 0x7fef0012c2f0 2023-09-30T22:31:47.792 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:31:47.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.794+0000 7fef05caa700 1 -- 172.21.15.16:0/3603510856 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fef0012c2f0 msgr2=0x7fef0006f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:47.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.794+0000 7fef05caa700 1 --2- 172.21.15.16:0/3603510856 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fef0012c2f0 0x7fef0006f010 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7feef0000c00 tx=0x7feef00047c0 comp rx=0 tx=0).stop 2023-09-30T22:31:47.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.794+0000 7fef05caa700 1 -- 172.21.15.16:0/3603510856 shutdown_connections 2023-09-30T22:31:47.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.794+0000 7fef05caa700 1 --2- 172.21.15.16:0/3603510856 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fef0012cff0 0x7fef0006fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:47.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.794+0000 7fef05caa700 1 --2- 172.21.15.16:0/3603510856 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fef0012c2f0 0x7fef0006f010 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:47.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.794+0000 7fef05caa700 1 --2- 172.21.15.16:0/3603510856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef0012cc20 0x7fef0006f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:47.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.794+0000 7fef05caa700 1 -- 172.21.15.16:0/3603510856 >> 172.21.15.16:0/3603510856 conn(0x7fef0009f470 msgr2=0x7fef0012fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:47.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.794+0000 7fef05caa700 1 -- 172.21.15.16:0/3603510856 shutdown_connections 2023-09-30T22:31:47.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:47.794+0000 7fef05caa700 1 -- 172.21.15.16:0/3603510856 wait complete. 2023-09-30T22:31:47.808 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:31:48.643 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:48.703 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:48.769 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:48.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.791+0000 7fa443ecf700 1 Processor -- start 2023-09-30T22:31:48.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.791+0000 7fa443ecf700 1 -- start start 2023-09-30T22:31:48.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.792+0000 7fa443ecf700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa43c130670 0x7fa43c130a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:48.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.792+0000 7fa443ecf700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa43c06f680 0x7fa43c06fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:48.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.792+0000 7fa443ecf700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa43c06ffd0 0x7fa43c137530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:48.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.792+0000 7fa443ecf700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa43c09a350 con 0x7fa43c06ffd0 2023-09-30T22:31:48.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.792+0000 7fa443ecf700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa43c130260 con 0x7fa43c06f680 2023-09-30T22:31:48.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.792+0000 7fa443ecf700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa43c071ff0 con 0x7fa43c130670 2023-09-30T22:31:48.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.792+0000 7fa441c6b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa43c130670 0x7fa43c130a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:48.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.792+0000 7fa441c6b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa43c130670 0x7fa43c130a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:60064/0 (socket says 172.21.15.16:60064) 2023-09-30T22:31:48.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.792+0000 7fa441c6b700 1 -- 172.21.15.16:0/2109127617 learned_addr learned my addr 172.21.15.16:0/2109127617 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:48.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.792+0000 7fa44246c700 1 --2- 172.21.15.16:0/2109127617 >> v2:172.21.15.16:3300/0 conn(0x7fa43c06ffd0 0x7fa43c137530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:48.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.792+0000 7fa44146a700 1 --2- 172.21.15.16:0/2109127617 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa43c06f680 0x7fa43c06fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:48.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.793+0000 7fa441c6b700 1 -- 172.21.15.16:0/2109127617 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa43c06f680 msgr2=0x7fa43c06fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:48.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.793+0000 7fa441c6b700 1 --2- 172.21.15.16:0/2109127617 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa43c06f680 0x7fa43c06fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:48.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.793+0000 7fa441c6b700 1 -- 172.21.15.16:0/2109127617 >> v2:172.21.15.16:3300/0 conn(0x7fa43c06ffd0 msgr2=0x7fa43c137530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:48.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.793+0000 7fa441c6b700 1 --2- 172.21.15.16:0/2109127617 >> v2:172.21.15.16:3300/0 conn(0x7fa43c06ffd0 0x7fa43c137530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:48.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.793+0000 7fa441c6b700 1 -- 172.21.15.16:0/2109127617 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa43c070470 con 0x7fa43c130670 2023-09-30T22:31:48.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.793+0000 7fa44246c700 1 --2- 172.21.15.16:0/2109127617 >> v2:172.21.15.16:3300/0 conn(0x7fa43c06ffd0 0x7fa43c137530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:31:48.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.793+0000 7fa441c6b700 1 --2- 172.21.15.16:0/2109127617 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa43c130670 0x7fa43c130a40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa43800bbc0 tx=0x7fa43800bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b31bc4657b897ac0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:48.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.793+0000 7fa440c69700 1 -- 172.21.15.16:0/2109127617 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa43800dcf0 con 0x7fa43c130670 2023-09-30T22:31:48.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.793+0000 7fa440c69700 1 -- 172.21.15.16:0/2109127617 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa43800de50 con 0x7fa43c130670 2023-09-30T22:31:48.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.793+0000 7fa440c69700 1 -- 172.21.15.16:0/2109127617 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa438010690 con 0x7fa43c130670 2023-09-30T22:31:48.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.794+0000 7fa443ecf700 1 -- 172.21.15.16:0/2109127617 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa43c130670 msgr2=0x7fa43c130a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:48.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.794+0000 7fa443ecf700 1 --2- 172.21.15.16:0/2109127617 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa43c130670 0x7fa43c130a40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa43800bbc0 tx=0x7fa43800bed0 comp rx=0 tx=0).stop 2023-09-30T22:31:48.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.794+0000 7fa443ecf700 1 -- 172.21.15.16:0/2109127617 shutdown_connections 2023-09-30T22:31:48.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.794+0000 7fa443ecf700 1 --2- 172.21.15.16:0/2109127617 >> v2:172.21.15.16:3300/0 conn(0x7fa43c06ffd0 0x7fa43c137530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:48.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.794+0000 7fa443ecf700 1 --2- 172.21.15.16:0/2109127617 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa43c130670 0x7fa43c130a40 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:48.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.794+0000 7fa443ecf700 1 --2- 172.21.15.16:0/2109127617 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa43c06f680 0x7fa43c06fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:48.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.794+0000 7fa443ecf700 1 -- 172.21.15.16:0/2109127617 >> 172.21.15.16:0/2109127617 conn(0x7fa43c0a2aa0 msgr2=0x7fa43c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:48.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.794+0000 7fa443ecf700 1 -- 172.21.15.16:0/2109127617 shutdown_connections 2023-09-30T22:31:48.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.794+0000 7fa443ecf700 1 -- 172.21.15.16:0/2109127617 wait complete. 2023-09-30T22:31:48.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.796+0000 7fa443ecf700 1 Processor -- start 2023-09-30T22:31:48.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.796+0000 7fa443ecf700 1 -- start start 2023-09-30T22:31:48.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.796+0000 7fa443ecf700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa43c06f680 0x7fa43c11fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:48.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.796+0000 7fa443ecf700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa43c06ffd0 0x7fa43c120160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:48.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.796+0000 7fa443ecf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa43c130670 0x7fa43c1206a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:48.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.796+0000 7fa443ecf700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa43c122e40 con 0x7fa43c130670 2023-09-30T22:31:48.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.796+0000 7fa443ecf700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa43c121830 con 0x7fa43c06f680 2023-09-30T22:31:48.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.796+0000 7fa443ecf700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa43c123230 con 0x7fa43c06ffd0 2023-09-30T22:31:48.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.796+0000 7fa441c6b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa43c06f680 0x7fa43c11fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:48.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.797+0000 7fa44246c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa43c130670 0x7fa43c1206a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:48.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.797+0000 7fa441c6b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa43c06f680 0x7fa43c11fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:33018/0 (socket says 172.21.15.16:33018) 2023-09-30T22:31:48.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.797+0000 7fa441c6b700 1 -- 172.21.15.16:0/2228388853 learned_addr learned my addr 172.21.15.16:0/2228388853 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:48.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.797+0000 7fa441c6b700 1 -- 172.21.15.16:0/2228388853 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa43c06ffd0 msgr2=0x7fa43c120160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:31:48.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.797+0000 7fa441c6b700 1 --2- 172.21.15.16:0/2228388853 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa43c06ffd0 0x7fa43c120160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:48.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.797+0000 7fa441c6b700 1 -- 172.21.15.16:0/2228388853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa43c130670 msgr2=0x7fa43c1206a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:48.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.797+0000 7fa441c6b700 1 --2- 172.21.15.16:0/2228388853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa43c130670 0x7fa43c1206a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:48.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.797+0000 7fa441c6b700 1 -- 172.21.15.16:0/2228388853 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa43800b7f0 con 0x7fa43c06f680 2023-09-30T22:31:48.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.797+0000 7fa44246c700 1 --2- 172.21.15.16:0/2228388853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa43c130670 0x7fa43c1206a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:48.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.797+0000 7fa441c6b700 1 --2- 172.21.15.16:0/2228388853 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa43c06f680 0x7fa43c11fc20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa43c072130 tx=0x7fa438004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:48.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.797+0000 7fa4327fc700 1 -- 172.21.15.16:0/2228388853 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa438003bf0 con 0x7fa43c06f680 2023-09-30T22:31:48.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.797+0000 7fa4327fc700 1 -- 172.21.15.16:0/2228388853 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa438003d50 con 0x7fa43c06f680 2023-09-30T22:31:48.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.797+0000 7fa443ecf700 1 -- 172.21.15.16:0/2228388853 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa43c123740 con 0x7fa43c06f680 2023-09-30T22:31:48.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.797+0000 7fa4327fc700 1 -- 172.21.15.16:0/2228388853 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa438010690 con 0x7fa43c06f680 2023-09-30T22:31:48.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.797+0000 7fa443ecf700 1 -- 172.21.15.16:0/2228388853 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa43c12ec10 con 0x7fa43c06f680 2023-09-30T22:31:48.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.798+0000 7fa4327fc700 1 -- 172.21.15.16:0/2228388853 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa438019400 con 0x7fa43c06f680 2023-09-30T22:31:48.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.798+0000 7fa4327fc700 1 -- 172.21.15.16:0/2228388853 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fa438040a00 con 0x7fa43c06f680 2023-09-30T22:31:48.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.798+0000 7fa443ecf700 1 -- 172.21.15.16:0/2228388853 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa43c074ea0 con 0x7fa43c06f680 2023-09-30T22:31:48.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.804+0000 7fa4327fc700 1 -- 172.21.15.16:0/2228388853 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa438018950 con 0x7fa43c06f680 2023-09-30T22:31:48.809 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:48.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.862+0000 7fa94f1f4700 1 Processor -- start 2023-09-30T22:31:48.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.862+0000 7fa94f1f4700 1 -- start start 2023-09-30T22:31:48.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.863+0000 7fa94f1f4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa94809d6f0 0x7fa94809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:48.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.863+0000 7fa94f1f4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa94809e410 0x7fa948130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:48.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.863+0000 7fa94f1f4700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa94809f130 con 0x7fa94809e000 2023-09-30T22:31:48.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.863+0000 7fa94f1f4700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa94809f520 con 0x7fa94809d6f0 2023-09-30T22:31:48.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.863+0000 7fa94f1f4700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa94809e930 con 0x7fa94809e410 2023-09-30T22:31:48.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.863+0000 7fa94cf90700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa94809d6f0 0x7fa94809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:48.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.863+0000 7fa94cf90700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa94809d6f0 0x7fa94809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54318/0 (socket says 172.21.15.16:54318) 2023-09-30T22:31:48.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.863+0000 7fa94cf90700 1 -- 172.21.15.16:0/1469075006 learned_addr learned my addr 172.21.15.16:0/1469075006 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:48.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.863+0000 7fa947fff700 1 --1- 172.21.15.16:0/1469075006 >> v1:172.21.15.16:6789/0 conn(0x7fa94809e000 0x7fa948130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50264/0 (socket says 172.21.15.16:50264) 2023-09-30T22:31:48.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.863+0000 7fa94d791700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa94809e410 0x7fa948130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:48.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.864+0000 7fa9477fe700 1 -- 172.21.15.16:0/1469075006 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 479505458 0 0) 0x7fa94809f130 con 0x7fa94809e000 2023-09-30T22:31:48.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.864+0000 7fa9477fe700 1 -- 172.21.15.16:0/1469075006 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9380057c0 con 0x7fa94809e000 2023-09-30T22:31:48.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.864+0000 7fa94d791700 1 -- 172.21.15.16:0/1469075006 >> v2:172.21.15.16:3300/0 conn(0x7fa94809d6f0 msgr2=0x7fa94809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:48.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.864+0000 7fa94d791700 1 --2- 172.21.15.16:0/1469075006 >> v2:172.21.15.16:3300/0 conn(0x7fa94809d6f0 0x7fa94809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:48.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.864+0000 7fa94d791700 1 -- 172.21.15.16:0/1469075006 >> v1:172.21.15.16:6789/0 conn(0x7fa94809e000 legacy=0x7fa948130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:48.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.864+0000 7fa94d791700 1 -- 172.21.15.16:0/1469075006 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa94809b2d0 con 0x7fa94809e410 2023-09-30T22:31:48.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.864+0000 7fa94cf90700 1 --2- 172.21.15.16:0/1469075006 >> v2:172.21.15.16:3300/0 conn(0x7fa94809d6f0 0x7fa94809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:48.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.864+0000 7fa94d791700 1 --2- 172.21.15.16:0/1469075006 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa94809e410 0x7fa948130d70 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa940009d90 tx=0x7fa94000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f688204a6be3440d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:48.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.864+0000 7fa9477fe700 1 -- 172.21.15.16:0/1469075006 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94000e070 con 0x7fa94809e410 2023-09-30T22:31:48.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.864+0000 7fa9477fe700 1 -- 172.21.15.16:0/1469075006 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa94000bb20 con 0x7fa94809e410 2023-09-30T22:31:48.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.864+0000 7fa9477fe700 1 -- 172.21.15.16:0/1469075006 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa940012430 con 0x7fa94809e410 2023-09-30T22:31:48.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.865+0000 7fa94f1f4700 1 -- 172.21.15.16:0/1469075006 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa94809e410 msgr2=0x7fa948130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:48.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.865+0000 7fa94f1f4700 1 --2- 172.21.15.16:0/1469075006 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa94809e410 0x7fa948130d70 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa940009d90 tx=0x7fa94000b330 comp rx=0 tx=0).stop 2023-09-30T22:31:48.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.865+0000 7fa94f1f4700 1 -- 172.21.15.16:0/1469075006 shutdown_connections 2023-09-30T22:31:48.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.865+0000 7fa94f1f4700 1 --2- 172.21.15.16:0/1469075006 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa94809e410 0x7fa948130d70 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:48.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.865+0000 7fa94f1f4700 1 --2- 172.21.15.16:0/1469075006 >> v2:172.21.15.16:3300/0 conn(0x7fa94809d6f0 0x7fa94809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:48.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.866+0000 7fa94f1f4700 1 -- 172.21.15.16:0/1469075006 >> 172.21.15.16:0/1469075006 conn(0x7fa948066f50 msgr2=0x7fa94809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:48.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.866+0000 7fa94f1f4700 1 -- 172.21.15.16:0/1469075006 shutdown_connections 2023-09-30T22:31:48.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.866+0000 7fa94f1f4700 1 -- 172.21.15.16:0/1469075006 wait complete. 2023-09-30T22:31:48.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.867+0000 7fa94f1f4700 1 Processor -- start 2023-09-30T22:31:48.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.867+0000 7fa94f1f4700 1 -- start start 2023-09-30T22:31:48.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.868+0000 7fa94f1f4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa94809d6f0 0x7fa9481220c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:48.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.868+0000 7fa94f1f4700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa94809e000 0x7fa948122600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:48.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.868+0000 7fa94f1f4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa94809e410 0x7fa948122b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:48.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.868+0000 7fa94f1f4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9480a3c70 con 0x7fa94809d6f0 2023-09-30T22:31:48.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.868+0000 7fa94f1f4700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa948125be0 con 0x7fa94809e000 2023-09-30T22:31:48.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.868+0000 7fa94f1f4700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa94809c700 con 0x7fa94809e410 2023-09-30T22:31:48.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.868+0000 7fa94cf90700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa94809d6f0 0x7fa9481220c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:48.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.868+0000 7fa94cf90700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa94809d6f0 0x7fa9481220c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54332/0 (socket says 172.21.15.16:54332) 2023-09-30T22:31:48.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.868+0000 7fa94cf90700 1 -- 172.21.15.16:0/986282897 learned_addr learned my addr 172.21.15.16:0/986282897 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:48.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.869+0000 7fa94d791700 1 --2- 172.21.15.16:0/986282897 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa94809e410 0x7fa948122b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:48.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.869+0000 7fa947fff700 1 --2- 172.21.15.16:0/986282897 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa94809e000 0x7fa948122600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:48.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.869+0000 7fa94cf90700 1 -- 172.21.15.16:0/986282897 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa94809e410 msgr2=0x7fa948122b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:48.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.869+0000 7fa94cf90700 1 --2- 172.21.15.16:0/986282897 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa94809e410 0x7fa948122b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:48.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.869+0000 7fa94cf90700 1 -- 172.21.15.16:0/986282897 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa94809e000 msgr2=0x7fa948122600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:48.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.869+0000 7fa94cf90700 1 --2- 172.21.15.16:0/986282897 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa94809e000 0x7fa948122600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:48.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.869+0000 7fa94cf90700 1 -- 172.21.15.16:0/986282897 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa93c009550 con 0x7fa94809d6f0 2023-09-30T22:31:48.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.869+0000 7fa94cf90700 1 --2- 172.21.15.16:0/986282897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa94809d6f0 0x7fa9481220c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fa93c00ebe0 tx=0x7fa93c00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:48.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.869+0000 7fa9457fa700 1 -- 172.21.15.16:0/986282897 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93c00c850 con 0x7fa94809d6f0 2023-09-30T22:31:48.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.869+0000 7fa9457fa700 1 -- 172.21.15.16:0/986282897 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa93c00c9b0 con 0x7fa94809d6f0 2023-09-30T22:31:48.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.869+0000 7fa9457fa700 1 -- 172.21.15.16:0/986282897 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93c00cd20 con 0x7fa94809d6f0 2023-09-30T22:31:48.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.869+0000 7fa94f1f4700 1 -- 172.21.15.16:0/986282897 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa940009a70 con 0x7fa94809d6f0 2023-09-30T22:31:48.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.870+0000 7fa94f1f4700 1 -- 172.21.15.16:0/986282897 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa948125370 con 0x7fa94809d6f0 2023-09-30T22:31:48.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.871+0000 7fa9457fa700 1 -- 172.21.15.16:0/986282897 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa93c013470 con 0x7fa94809d6f0 2023-09-30T22:31:48.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.871+0000 7fa9457fa700 1 -- 172.21.15.16:0/986282897 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fa93c03fc00 con 0x7fa94809d6f0 2023-09-30T22:31:48.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.872+0000 7fa94f1f4700 1 -- 172.21.15.16:0/986282897 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa94811ac20 con 0x7fa94809d6f0 2023-09-30T22:31:48.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.877+0000 7fa9457fa700 1 -- 172.21.15.16:0/986282897 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa93c01b3b0 con 0x7fa94809d6f0 2023-09-30T22:31:48.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.926+0000 7f81b01a5700 1 Processor -- start 2023-09-30T22:31:48.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.927+0000 7f81b01a5700 1 -- start start 2023-09-30T22:31:48.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.928+0000 7f81b01a5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f81a809d6f0 0x7f81a809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:48.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.928+0000 7f81b01a5700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f81a809e000 0x7f81a8130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:48.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.928+0000 7f81b01a5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f81a809e410 0x7f81a8130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:48.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.928+0000 7f81b01a5700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81a809f520 con 0x7f81a809e000 2023-09-30T22:31:48.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.928+0000 7f81b01a5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81a809f130 con 0x7f81a809d6f0 2023-09-30T22:31:48.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.928+0000 7f81b01a5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f81a809e930 con 0x7f81a809e410 2023-09-30T22:31:48.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.928+0000 7f81ae742700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f81a809e410 0x7f81a8130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:48.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.928+0000 7f81adf41700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f81a809d6f0 0x7f81a809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:48.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.928+0000 7f81adf41700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f81a809d6f0 0x7f81a809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:33034/0 (socket says 172.21.15.16:33034) 2023-09-30T22:31:48.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.928+0000 7f81ae742700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f81a809e410 0x7f81a8130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:60096/0 (socket says 172.21.15.16:60096) 2023-09-30T22:31:48.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.928+0000 7f81adf41700 1 -- 172.21.15.16:0/3837362334 learned_addr learned my addr 172.21.15.16:0/3837362334 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:48.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.928+0000 7f81ad740700 1 --2- 172.21.15.16:0/3837362334 >> v2:172.21.15.16:3300/0 conn(0x7f81a809e000 0x7f81a8130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:48.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.928+0000 7f81adf41700 1 -- 172.21.15.16:0/3837362334 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f81a809e410 msgr2=0x7f81a8130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:48.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.928+0000 7f81adf41700 1 --2- 172.21.15.16:0/3837362334 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f81a809e410 0x7f81a8130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:48.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.928+0000 7f81adf41700 1 -- 172.21.15.16:0/3837362334 >> v2:172.21.15.16:3300/0 conn(0x7f81a809e000 msgr2=0x7f81a8130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:48.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.928+0000 7f81adf41700 1 --2- 172.21.15.16:0/3837362334 >> v2:172.21.15.16:3300/0 conn(0x7f81a809e000 0x7f81a8130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:48.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.928+0000 7f81adf41700 1 -- 172.21.15.16:0/3837362334 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a809b2d0 con 0x7f81a809d6f0 2023-09-30T22:31:48.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.929+0000 7f81adf41700 1 --2- 172.21.15.16:0/3837362334 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f81a809d6f0 0x7f81a809dac0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f81a400bb90 tx=0x7f81a400bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=468267021c691b80 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:48.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.929+0000 7f81acf3f700 1 -- 172.21.15.16:0/3837362334 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a400dd30 con 0x7f81a809d6f0 2023-09-30T22:31:48.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.929+0000 7f81acf3f700 1 -- 172.21.15.16:0/3837362334 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a400de90 con 0x7f81a809d6f0 2023-09-30T22:31:48.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.929+0000 7f81acf3f700 1 -- 172.21.15.16:0/3837362334 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a4010690 con 0x7f81a809d6f0 2023-09-30T22:31:48.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.929+0000 7f81b01a5700 1 -- 172.21.15.16:0/3837362334 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f81a809d6f0 msgr2=0x7f81a809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:48.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.929+0000 7f81b01a5700 1 --2- 172.21.15.16:0/3837362334 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f81a809d6f0 0x7f81a809dac0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f81a400bb90 tx=0x7f81a400bea0 comp rx=0 tx=0).stop 2023-09-30T22:31:48.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.930+0000 7f81b01a5700 1 -- 172.21.15.16:0/3837362334 shutdown_connections 2023-09-30T22:31:48.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.930+0000 7f81b01a5700 1 --2- 172.21.15.16:0/3837362334 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f81a809d6f0 0x7f81a809dac0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:48.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.930+0000 7f81b01a5700 1 --2- 172.21.15.16:0/3837362334 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f81a809e410 0x7f81a8130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:48.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.930+0000 7f81b01a5700 1 --2- 172.21.15.16:0/3837362334 >> v2:172.21.15.16:3300/0 conn(0x7f81a809e000 0x7f81a8130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:48.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.930+0000 7f81b01a5700 1 -- 172.21.15.16:0/3837362334 >> 172.21.15.16:0/3837362334 conn(0x7f81a8066f50 msgr2=0x7f81a809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:48.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.930+0000 7f81b01a5700 1 -- 172.21.15.16:0/3837362334 shutdown_connections 2023-09-30T22:31:48.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.930+0000 7f81b01a5700 1 -- 172.21.15.16:0/3837362334 wait complete. 2023-09-30T22:31:48.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.931+0000 7f81b01a5700 1 Processor -- start 2023-09-30T22:31:48.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.931+0000 7f81b01a5700 1 -- start start 2023-09-30T22:31:48.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.932+0000 7f81b01a5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81a809d6f0 0x7f81a8122460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:48.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.932+0000 7f81b01a5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f81a809e000 0x7f81a81229a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:48.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.932+0000 7f81b01a5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f81a809e410 0x7f81a8122ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:48.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.932+0000 7f81b01a5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a8125ca0 con 0x7f81a809d6f0 2023-09-30T22:31:48.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.932+0000 7f81b01a5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81a8124180 con 0x7f81a809e000 2023-09-30T22:31:48.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.932+0000 7f81b01a5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f81a80a19a0 con 0x7f81a809e410 2023-09-30T22:31:48.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.932+0000 7f81adf41700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81a809d6f0 0x7f81a8122460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:48.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.932+0000 7f81adf41700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81a809d6f0 0x7f81a8122460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54338/0 (socket says 172.21.15.16:54338) 2023-09-30T22:31:48.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.932+0000 7f81adf41700 1 -- 172.21.15.16:0/917136321 learned_addr learned my addr 172.21.15.16:0/917136321 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:48.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.932+0000 7f81ad740700 1 --2- 172.21.15.16:0/917136321 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f81a809e000 0x7f81a81229a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:48.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.932+0000 7f81adf41700 1 -- 172.21.15.16:0/917136321 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f81a809e410 msgr2=0x7f81a8122ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:48.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.932+0000 7f81adf41700 1 --2- 172.21.15.16:0/917136321 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f81a809e410 0x7f81a8122ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:48.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.932+0000 7f81adf41700 1 -- 172.21.15.16:0/917136321 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f81a809e000 msgr2=0x7f81a81229a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:48.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.932+0000 7f81adf41700 1 --2- 172.21.15.16:0/917136321 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f81a809e000 0x7f81a81229a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:48.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.932+0000 7f81adf41700 1 -- 172.21.15.16:0/917136321 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a400b7f0 con 0x7f81a809d6f0 2023-09-30T22:31:48.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.932+0000 7f81adf41700 1 --2- 172.21.15.16:0/917136321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81a809d6f0 0x7f81a8122460 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f81a400d5d0 tx=0x7f81a4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:48.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.933+0000 7f819e7fc700 1 -- 172.21.15.16:0/917136321 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a4003ab0 con 0x7f81a809d6f0 2023-09-30T22:31:48.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.933+0000 7f819e7fc700 1 -- 172.21.15.16:0/917136321 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a4003c10 con 0x7f81a809d6f0 2023-09-30T22:31:48.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.933+0000 7f819e7fc700 1 -- 172.21.15.16:0/917136321 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a4010470 con 0x7f81a809d6f0 2023-09-30T22:31:48.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.933+0000 7f81b01a5700 1 -- 172.21.15.16:0/917136321 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81a809c700 con 0x7f81a809d6f0 2023-09-30T22:31:48.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.933+0000 7f81b01a5700 1 -- 172.21.15.16:0/917136321 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81a8126710 con 0x7f81a809d6f0 2023-09-30T22:31:48.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.934+0000 7f819e7fc700 1 -- 172.21.15.16:0/917136321 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f81a40105d0 con 0x7f81a809d6f0 2023-09-30T22:31:48.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.934+0000 7f81b01a5700 1 -- 172.21.15.16:0/917136321 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81a8074ea0 con 0x7f81a809d6f0 2023-09-30T22:31:48.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.934+0000 7f819e7fc700 1 -- 172.21.15.16:0/917136321 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f81a4040dc0 con 0x7f81a809d6f0 2023-09-30T22:31:48.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.939+0000 7f819e7fc700 1 -- 172.21.15.16:0/917136321 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f81a4021700 con 0x7f81a809d6f0 2023-09-30T22:31:48.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.962+0000 7f8effc1b700 1 Processor -- start 2023-09-30T22:31:48.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.962+0000 7f8effc1b700 1 -- start start 2023-09-30T22:31:48.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.963+0000 7f8effc1b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8ef80a0a30 0x7f8ef80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:48.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.963+0000 7f8effc1b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8ef809d630 0x7f8ef80a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:48.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.963+0000 7f8effc1b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8ef8133930 0x7f8ef8133db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:48.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.963+0000 7f8effc1b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ef80a2a70 con 0x7f8ef80a0a30 2023-09-30T22:31:48.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.963+0000 7f8effc1b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef80a01d0 con 0x7f8ef8133930 2023-09-30T22:31:48.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.963+0000 7f8effc1b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef809e2b0 con 0x7f8ef809d630 2023-09-30T22:31:48.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.963+0000 7f8efd1b6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8ef809d630 0x7f8ef80a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:48.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.963+0000 7f8efd1b6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8ef809d630 0x7f8ef80a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:60124/0 (socket says 172.21.15.16:60124) 2023-09-30T22:31:48.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.963+0000 7f8efd1b6700 1 -- 172.21.15.16:0/256405689 learned_addr learned my addr 172.21.15.16:0/256405689 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:48.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.963+0000 7f8efe1b8700 1 --2- 172.21.15.16:0/256405689 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8ef8133930 0x7f8ef8133db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:48.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.963+0000 7f8efd9b7700 1 --2- 172.21.15.16:0/256405689 >> v2:172.21.15.16:3300/0 conn(0x7f8ef80a0a30 0x7f8ef80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:48.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.963+0000 7f8efd1b6700 1 -- 172.21.15.16:0/256405689 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8ef8133930 msgr2=0x7f8ef8133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:48.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.963+0000 7f8efd1b6700 1 --2- 172.21.15.16:0/256405689 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8ef8133930 0x7f8ef8133db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:48.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.963+0000 7f8efd1b6700 1 -- 172.21.15.16:0/256405689 >> v2:172.21.15.16:3300/0 conn(0x7f8ef80a0a30 msgr2=0x7f8ef80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:48.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.963+0000 7f8efd1b6700 1 --2- 172.21.15.16:0/256405689 >> v2:172.21.15.16:3300/0 conn(0x7f8ef80a0a30 0x7f8ef80a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:48.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.963+0000 7f8efd1b6700 1 -- 172.21.15.16:0/256405689 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef809dc50 con 0x7f8ef809d630 2023-09-30T22:31:48.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.964+0000 7f8efd1b6700 1 --2- 172.21.15.16:0/256405689 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8ef809d630 0x7f8ef80a1340 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8ee80098f0 tx=0x7f8ee8009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=936b2d1f75148c5d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:48.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.964+0000 7f8efc9b5700 1 -- 172.21.15.16:0/256405689 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee8011830 con 0x7f8ef809d630 2023-09-30T22:31:48.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.964+0000 7f8efc9b5700 1 -- 172.21.15.16:0/256405689 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ee8011990 con 0x7f8ef809d630 2023-09-30T22:31:48.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.964+0000 7f8efc9b5700 1 -- 172.21.15.16:0/256405689 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee8011d20 con 0x7f8ef809d630 2023-09-30T22:31:48.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.965+0000 7f8effc1b700 1 -- 172.21.15.16:0/256405689 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8ef809d630 msgr2=0x7f8ef80a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:48.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.965+0000 7f8effc1b700 1 --2- 172.21.15.16:0/256405689 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8ef809d630 0x7f8ef80a1340 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8ee80098f0 tx=0x7f8ee8009c00 comp rx=0 tx=0).stop 2023-09-30T22:31:48.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.965+0000 7f8effc1b700 1 -- 172.21.15.16:0/256405689 shutdown_connections 2023-09-30T22:31:48.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.965+0000 7f8effc1b700 1 --2- 172.21.15.16:0/256405689 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8ef8133930 0x7f8ef8133db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:48.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.965+0000 7f8effc1b700 1 --2- 172.21.15.16:0/256405689 >> v2:172.21.15.16:3300/0 conn(0x7f8ef80a0a30 0x7f8ef80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:48.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.965+0000 7f8effc1b700 1 --2- 172.21.15.16:0/256405689 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8ef809d630 0x7f8ef80a1340 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:48.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.965+0000 7f8effc1b700 1 -- 172.21.15.16:0/256405689 >> 172.21.15.16:0/256405689 conn(0x7f8ef80a3050 msgr2=0x7f8ef80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:48.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.965+0000 7f8effc1b700 1 -- 172.21.15.16:0/256405689 shutdown_connections 2023-09-30T22:31:48.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.965+0000 7f8effc1b700 1 -- 172.21.15.16:0/256405689 wait complete. 2023-09-30T22:31:48.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.966+0000 7f8effc1b700 1 Processor -- start 2023-09-30T22:31:48.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.966+0000 7f8effc1b700 1 -- start start 2023-09-30T22:31:48.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.966+0000 7f8effc1b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ef809d630 0x7f8ef81c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:48.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.966+0000 7f8effc1b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8ef80a0a30 0x7f8ef81c8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:48.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.966+0000 7f8effc1b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8ef8133930 0x7f8ef81c8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:48.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.966+0000 7f8effc1b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef8121140 con 0x7f8ef809d630 2023-09-30T22:31:48.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.966+0000 7f8effc1b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef80a4030 con 0x7f8ef80a0a30 2023-09-30T22:31:48.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.966+0000 7f8effc1b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef8120570 con 0x7f8ef8133930 2023-09-30T22:31:48.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.967+0000 7f8efd1b6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8ef80a0a30 0x7f8ef81c8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:48.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.967+0000 7f8efd1b6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8ef80a0a30 0x7f8ef81c8640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:33056/0 (socket says 172.21.15.16:33056) 2023-09-30T22:31:48.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.967+0000 7f8efd1b6700 1 -- 172.21.15.16:0/3063633030 learned_addr learned my addr 172.21.15.16:0/3063633030 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:48.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.967+0000 7f8efd9b7700 1 --2- 172.21.15.16:0/3063633030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ef809d630 0x7f8ef81c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:48.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.967+0000 7f8efe1b8700 1 --2- 172.21.15.16:0/3063633030 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8ef8133930 0x7f8ef81c8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:48.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.967+0000 7f8efd1b6700 1 -- 172.21.15.16:0/3063633030 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8ef8133930 msgr2=0x7f8ef81c8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:48.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.967+0000 7f8efd1b6700 1 --2- 172.21.15.16:0/3063633030 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8ef8133930 0x7f8ef81c8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:48.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.967+0000 7f8efd1b6700 1 -- 172.21.15.16:0/3063633030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ef809d630 msgr2=0x7f8ef81c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:48.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.967+0000 7f8efd1b6700 1 --2- 172.21.15.16:0/3063633030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ef809d630 0x7f8ef81c8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:48.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.967+0000 7f8efd1b6700 1 -- 172.21.15.16:0/3063633030 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ee8009550 con 0x7f8ef80a0a30 2023-09-30T22:31:48.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.967+0000 7f8efd1b6700 1 --2- 172.21.15.16:0/3063633030 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8ef80a0a30 0x7f8ef81c8640 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8ee8000c00 tx=0x7f8ee8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:48.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.967+0000 7f8eee7fc700 1 -- 172.21.15.16:0/3063633030 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee8004510 con 0x7f8ef80a0a30 2023-09-30T22:31:48.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.968+0000 7f8eee7fc700 1 -- 172.21.15.16:0/3063633030 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ee80036a0 con 0x7f8ef80a0a30 2023-09-30T22:31:48.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.968+0000 7f8effc1b700 1 -- 172.21.15.16:0/3063633030 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ef8122090 con 0x7f8ef80a0a30 2023-09-30T22:31:48.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.968+0000 7f8eee7fc700 1 -- 172.21.15.16:0/3063633030 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee8003a50 con 0x7f8ef80a0a30 2023-09-30T22:31:48.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.968+0000 7f8effc1b700 1 -- 172.21.15.16:0/3063633030 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ef8133300 con 0x7f8ef80a0a30 2023-09-30T22:31:48.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.968+0000 7f8eee7fc700 1 -- 172.21.15.16:0/3063633030 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8ee8021070 con 0x7f8ef80a0a30 2023-09-30T22:31:48.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.969+0000 7f8eee7fc700 1 -- 172.21.15.16:0/3063633030 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f8ee8040940 con 0x7f8ef80a0a30 2023-09-30T22:31:48.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.969+0000 7f8effc1b700 1 -- 172.21.15.16:0/3063633030 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ef80747f0 con 0x7f8ef80a0a30 2023-09-30T22:31:48.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:48.978+0000 7f8eee7fc700 1 -- 172.21.15.16:0/3063633030 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8ee8029080 con 0x7f8ef80a0a30 2023-09-30T22:31:49.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:49.039+0000 7fa443ecf700 1 -- 172.21.15.16:0/2228388853 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa43c072210 con 0x7fa43c06f680 2023-09-30T22:31:49.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:49.040+0000 7fa4327fc700 1 -- 172.21.15.16:0/2228388853 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa438018360 con 0x7fa43c06f680 2023-09-30T22:31:49.040 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:31:49.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:49.043+0000 7fa443ecf700 1 -- 172.21.15.16:0/2228388853 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa43c06f680 msgr2=0x7fa43c11fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:49.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:49.043+0000 7fa443ecf700 1 --2- 172.21.15.16:0/2228388853 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa43c06f680 0x7fa43c11fc20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa43c072130 tx=0x7fa438004750 comp rx=0 tx=0).stop 2023-09-30T22:31:49.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:49.043+0000 7fa443ecf700 1 -- 172.21.15.16:0/2228388853 shutdown_connections 2023-09-30T22:31:49.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:49.043+0000 7fa443ecf700 1 --2- 172.21.15.16:0/2228388853 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa43c06f680 0x7fa43c11fc20 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:49.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:49.043+0000 7fa443ecf700 1 --2- 172.21.15.16:0/2228388853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa43c130670 0x7fa43c1206a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:49.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:49.043+0000 7fa443ecf700 1 --2- 172.21.15.16:0/2228388853 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa43c06ffd0 0x7fa43c120160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:49.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:49.043+0000 7fa443ecf700 1 -- 172.21.15.16:0/2228388853 >> 172.21.15.16:0/2228388853 conn(0x7fa43c0a2aa0 msgr2=0x7fa43c06e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:49.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:49.043+0000 7fa443ecf700 1 -- 172.21.15.16:0/2228388853 shutdown_connections 2023-09-30T22:31:49.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:49.043+0000 7fa443ecf700 1 -- 172.21.15.16:0/2228388853 wait complete. 2023-09-30T22:31:49.058 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:31:49.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:49.106+0000 7fa94f1f4700 1 -- 172.21.15.16:0/986282897 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa948074ea0 con 0x7fa94809d6f0 2023-09-30T22:31:49.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:49.106+0000 7fa9457fa700 1 -- 172.21.15.16:0/986282897 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa93c017070 con 0x7fa94809d6f0 2023-09-30T22:31:49.107 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:31:49.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:49.109+0000 7fa94f1f4700 1 -- 172.21.15.16:0/986282897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa94809d6f0 msgr2=0x7fa9481220c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:49.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:49.109+0000 7fa94f1f4700 1 --2- 172.21.15.16:0/986282897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa94809d6f0 0x7fa9481220c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fa93c00ebe0 tx=0x7fa93c00eef0 comp rx=0 tx=0).stop 2023-09-30T22:31:49.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:49.109+0000 7fa94f1f4700 1 -- 172.21.15.16:0/986282897 shutdown_connections 2023-09-30T22:31:49.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:49.109+0000 7fa94f1f4700 1 --2- 172.21.15.16:0/986282897 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa94809e410 0x7fa948122b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:49.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:49.109+0000 7fa94f1f4700 1 --2- 172.21.15.16:0/986282897 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa94809d6f0 0x7fa9481220c0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:49.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:49.109+0000 7fa94f1f4700 1 --2- 172.21.15.16:0/986282897 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa94809e000 0x7fa948122600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:49.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:49.109+0000 7fa94f1f4700 1 -- 172.21.15.16:0/986282897 >> 172.21.15.16:0/986282897 conn(0x7fa948066f50 msgr2=0x7fa948130b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:49.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:49.109+0000 7fa94f1f4700 1 -- 172.21.15.16:0/986282897 shutdown_connections 2023-09-30T22:31:49.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:49.109+0000 7fa94f1f4700 1 -- 172.21.15.16:0/986282897 wait complete. 2023-09-30T22:31:49.123 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:31:49.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:49.148+0000 7f81b01a5700 1 -- 172.21.15.16:0/917136321 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f81a806c360 con 0x7f81a809d6f0 2023-09-30T22:31:49.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:49.149+0000 7f819e7fc700 1 -- 172.21.15.16:0/917136321 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f81a4021340 con 0x7f81a809d6f0 2023-09-30T22:31:49.149 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:31:49.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:49.151+0000 7f81b01a5700 1 -- 172.21.15.16:0/917136321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81a809d6f0 msgr2=0x7f81a8122460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:49.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:49.151+0000 7f81b01a5700 1 --2- 172.21.15.16:0/917136321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81a809d6f0 0x7f81a8122460 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f81a400d5d0 tx=0x7f81a4004750 comp rx=0 tx=0).stop 2023-09-30T22:31:49.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:49.151+0000 7f81b01a5700 1 -- 172.21.15.16:0/917136321 shutdown_connections 2023-09-30T22:31:49.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:49.151+0000 7f81b01a5700 1 --2- 172.21.15.16:0/917136321 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f81a809e410 0x7f81a8122ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:49.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:49.151+0000 7f81b01a5700 1 --2- 172.21.15.16:0/917136321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81a809d6f0 0x7f81a8122460 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:49.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:49.151+0000 7f81b01a5700 1 --2- 172.21.15.16:0/917136321 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f81a809e000 0x7f81a81229a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:49.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:49.151+0000 7f81b01a5700 1 -- 172.21.15.16:0/917136321 >> 172.21.15.16:0/917136321 conn(0x7f81a8066f50 msgr2=0x7f81a806a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:49.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:49.151+0000 7f81b01a5700 1 -- 172.21.15.16:0/917136321 shutdown_connections 2023-09-30T22:31:49.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:49.151+0000 7f81b01a5700 1 -- 172.21.15.16:0/917136321 wait complete. 2023-09-30T22:31:49.165 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:31:49.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:49.208+0000 7f8effc1b700 1 -- 172.21.15.16:0/3063633030 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f8ef8072980 con 0x7f8ef80a0a30 2023-09-30T22:31:49.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:49.209+0000 7f8eee7fc700 1 -- 172.21.15.16:0/3063633030 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8ee8015020 con 0x7f8ef80a0a30 2023-09-30T22:31:49.210 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:31:49.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:49.211+0000 7f8effc1b700 1 -- 172.21.15.16:0/3063633030 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8ef80a0a30 msgr2=0x7f8ef81c8640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:49.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:49.211+0000 7f8effc1b700 1 --2- 172.21.15.16:0/3063633030 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8ef80a0a30 0x7f8ef81c8640 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8ee8000c00 tx=0x7f8ee8004750 comp rx=0 tx=0).stop 2023-09-30T22:31:49.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:49.211+0000 7f8effc1b700 1 -- 172.21.15.16:0/3063633030 shutdown_connections 2023-09-30T22:31:49.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:49.211+0000 7f8effc1b700 1 --2- 172.21.15.16:0/3063633030 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8ef8133930 0x7f8ef81c8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:49.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:49.211+0000 7f8effc1b700 1 --2- 172.21.15.16:0/3063633030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ef809d630 0x7f8ef81c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:49.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:49.211+0000 7f8effc1b700 1 --2- 172.21.15.16:0/3063633030 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8ef80a0a30 0x7f8ef81c8640 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:49.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:49.211+0000 7f8effc1b700 1 -- 172.21.15.16:0/3063633030 >> 172.21.15.16:0/3063633030 conn(0x7f8ef80a3050 msgr2=0x7f8ef809ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:49.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:49.211+0000 7f8effc1b700 1 -- 172.21.15.16:0/3063633030 shutdown_connections 2023-09-30T22:31:49.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:49.211+0000 7f8effc1b700 1 -- 172.21.15.16:0/3063633030 wait complete. 2023-09-30T22:31:49.225 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:31:50.059 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:50.125 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:50.166 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:50.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.211+0000 7f97658e9700 1 Processor -- start 2023-09-30T22:31:50.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.211+0000 7f97658e9700 1 -- start start 2023-09-30T22:31:50.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.211+0000 7f97658e9700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f976009d6f0 0x7f976009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:50.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.211+0000 7f97658e9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f976009e440 0x7f9760134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:50.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.211+0000 7f97658e9700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f976009b120 con 0x7f976009e000 2023-09-30T22:31:50.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.211+0000 7f97658e9700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f976006b460 con 0x7f976009d6f0 2023-09-30T22:31:50.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.211+0000 7f97658e9700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f976009a7e0 con 0x7f976009e440 2023-09-30T22:31:50.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.211+0000 7f975effd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f976009d6f0 0x7f976009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:50.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.211+0000 7f975e7fc700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f976009e000 0x7f97601339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50270/0 (socket says 172.21.15.16:50270) 2023-09-30T22:31:50.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.211+0000 7f975e7fc700 1 -- 172.21.15.16:0/2635123443 learned_addr learned my addr 172.21.15.16:0/2635123443 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:31:50.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.212+0000 7f975f7fe700 1 --2- 172.21.15.16:0/2635123443 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f976009e440 0x7f9760134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:50.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.212+0000 7f975effd700 1 -- 172.21.15.16:0/2635123443 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f976009e440 msgr2=0x7f9760134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:50.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.212+0000 7f975effd700 1 --2- 172.21.15.16:0/2635123443 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f976009e440 0x7f9760134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:50.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.212+0000 7f975effd700 1 -- 172.21.15.16:0/2635123443 >> v1:172.21.15.16:6789/0 conn(0x7f976009e000 legacy=0x7f97601339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:50.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.212+0000 7f975dffb700 1 -- 172.21.15.16:0/2635123443 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 139233963 0 0) 0x7f976009b120 con 0x7f976009e000 2023-09-30T22:31:50.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.212+0000 7f975effd700 1 -- 172.21.15.16:0/2635123443 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f976009a500 con 0x7f976009d6f0 2023-09-30T22:31:50.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.212+0000 7f975effd700 1 --2- 172.21.15.16:0/2635123443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f976009d6f0 0x7f976009dac0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f9750009870 tx=0x7f9750009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35542b9c7ca66abd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:50.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.212+0000 7f975dffb700 1 -- 172.21.15.16:0/2635123443 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9750011830 con 0x7f976009d6f0 2023-09-30T22:31:50.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.212+0000 7f975dffb700 1 -- 172.21.15.16:0/2635123443 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9750011990 con 0x7f976009d6f0 2023-09-30T22:31:50.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.212+0000 7f975dffb700 1 -- 172.21.15.16:0/2635123443 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9750011d20 con 0x7f976009d6f0 2023-09-30T22:31:50.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.213+0000 7f97658e9700 1 -- 172.21.15.16:0/2635123443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f976009d6f0 msgr2=0x7f976009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:50.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.213+0000 7f97658e9700 1 --2- 172.21.15.16:0/2635123443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f976009d6f0 0x7f976009dac0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f9750009870 tx=0x7f9750009b80 comp rx=0 tx=0).stop 2023-09-30T22:31:50.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.213+0000 7f97658e9700 1 -- 172.21.15.16:0/2635123443 shutdown_connections 2023-09-30T22:31:50.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.213+0000 7f97658e9700 1 --2- 172.21.15.16:0/2635123443 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f976009e440 0x7f9760134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:50.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.213+0000 7f97658e9700 1 --2- 172.21.15.16:0/2635123443 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f976009d6f0 0x7f976009dac0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:50.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.213+0000 7f97658e9700 1 -- 172.21.15.16:0/2635123443 >> 172.21.15.16:0/2635123443 conn(0x7f9760066f50 msgr2=0x7f976009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:50.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.213+0000 7f97658e9700 1 -- 172.21.15.16:0/2635123443 shutdown_connections 2023-09-30T22:31:50.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.213+0000 7f97658e9700 1 -- 172.21.15.16:0/2635123443 wait complete. 2023-09-30T22:31:50.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.215+0000 7f97658e9700 1 Processor -- start 2023-09-30T22:31:50.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.215+0000 7f97658e9700 1 -- start start 2023-09-30T22:31:50.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.216+0000 7f97658e9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f976009d6f0 0x7f97601c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:50.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.216+0000 7f97658e9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f976009e000 0x7f97601c8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:50.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.216+0000 7f97658e9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f976009e440 0x7f97601c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:50.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.216+0000 7f97658e9700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97601a6c10 con 0x7f976009e440 2023-09-30T22:31:50.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.216+0000 7f97658e9700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97601a7020 con 0x7f976009e000 2023-09-30T22:31:50.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.216+0000 7f97658e9700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f97601a64a0 con 0x7f976009d6f0 2023-09-30T22:31:50.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.216+0000 7f975e7fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f976009e000 0x7f97601c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:50.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.216+0000 7f975e7fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f976009e000 0x7f97601c8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:33068/0 (socket says 172.21.15.16:33068) 2023-09-30T22:31:50.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.216+0000 7f975e7fc700 1 -- 172.21.15.16:0/2323609791 learned_addr learned my addr 172.21.15.16:0/2323609791 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:50.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.216+0000 7f975effd700 1 --2- 172.21.15.16:0/2323609791 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f976009d6f0 0x7f97601c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:50.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.216+0000 7f975f7fe700 1 --2- 172.21.15.16:0/2323609791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f976009e440 0x7f97601c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:50.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.216+0000 7f975e7fc700 1 -- 172.21.15.16:0/2323609791 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f976009d6f0 msgr2=0x7f97601c8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:50.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.216+0000 7f975e7fc700 1 --2- 172.21.15.16:0/2323609791 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f976009d6f0 0x7f97601c8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:50.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.216+0000 7f975e7fc700 1 -- 172.21.15.16:0/2323609791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f976009e440 msgr2=0x7f97601c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:50.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.216+0000 7f975e7fc700 1 --2- 172.21.15.16:0/2323609791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f976009e440 0x7f97601c8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:50.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.216+0000 7f975e7fc700 1 -- 172.21.15.16:0/2323609791 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97500094a0 con 0x7f976009e000 2023-09-30T22:31:50.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.216+0000 7f975e7fc700 1 --2- 172.21.15.16:0/2323609791 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f976009e000 0x7f97601c8560 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f975400c520 tx=0x7f975400c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:50.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.217+0000 7f9747fff700 1 -- 172.21.15.16:0/2323609791 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9754004c80 con 0x7f976009e000 2023-09-30T22:31:50.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.217+0000 7f9747fff700 1 -- 172.21.15.16:0/2323609791 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9754004e00 con 0x7f976009e000 2023-09-30T22:31:50.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.217+0000 7f97658e9700 1 -- 172.21.15.16:0/2323609791 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97600a2df0 con 0x7f976009e000 2023-09-30T22:31:50.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.217+0000 7f9747fff700 1 -- 172.21.15.16:0/2323609791 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9754007310 con 0x7f976009e000 2023-09-30T22:31:50.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.217+0000 7f97658e9700 1 -- 172.21.15.16:0/2323609791 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97601b91c0 con 0x7f976009e000 2023-09-30T22:31:50.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.218+0000 7f9747fff700 1 -- 172.21.15.16:0/2323609791 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9754007470 con 0x7f976009e000 2023-09-30T22:31:50.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.218+0000 7f97658e9700 1 -- 172.21.15.16:0/2323609791 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97601a7eb0 con 0x7f976009e000 2023-09-30T22:31:50.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.218+0000 7f9747fff700 1 -- 172.21.15.16:0/2323609791 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f975403f290 con 0x7f976009e000 2023-09-30T22:31:50.225 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:50.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.227+0000 7f9747fff700 1 -- 172.21.15.16:0/2323609791 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f97540133c0 con 0x7f976009e000 2023-09-30T22:31:50.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.276+0000 7fe211d70700 1 Processor -- start 2023-09-30T22:31:50.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.276+0000 7fe211d70700 1 -- start start 2023-09-30T22:31:50.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.276+0000 7fe211d70700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe20c09f410 0x7fe20c136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:50.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.276+0000 7fe211d70700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe20c136fe0 0x7fe20c1393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:50.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.276+0000 7fe211d70700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe20c070940 con 0x7fe20c09e930 2023-09-30T22:31:50.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.276+0000 7fe211d70700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe20c06fd60 con 0x7fe20c136fe0 2023-09-30T22:31:50.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.276+0000 7fe211d70700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe20c06fa80 con 0x7fe20c09f410 2023-09-30T22:31:50.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.277+0000 7fe20bfff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe20c136fe0 0x7fe20c1393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:50.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.277+0000 7fe20bfff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe20c136fe0 0x7fe20c1393c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54380/0 (socket says 172.21.15.16:54380) 2023-09-30T22:31:50.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.277+0000 7fe20bfff700 1 -- 172.21.15.16:0/2499550950 learned_addr learned my addr 172.21.15.16:0/2499550950 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:50.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.277+0000 7fe20affd700 1 --2- 172.21.15.16:0/2499550950 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe20c09f410 0x7fe20c136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:50.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.277+0000 7fe20a7fc700 1 -- 172.21.15.16:0/2499550950 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2255847631 0 0) 0x7fe20c070940 con 0x7fe20c09e930 2023-09-30T22:31:50.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.277+0000 7fe20bfff700 1 -- 172.21.15.16:0/2499550950 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe20c09f410 msgr2=0x7fe20c136aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:50.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.277+0000 7fe20bfff700 1 --2- 172.21.15.16:0/2499550950 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe20c09f410 0x7fe20c136aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:50.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.277+0000 7fe20bfff700 1 -- 172.21.15.16:0/2499550950 >> v1:172.21.15.16:6789/0 conn(0x7fe20c09e930 legacy=0x7fe20c09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:50.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.277+0000 7fe20bfff700 1 -- 172.21.15.16:0/2499550950 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe20c09fb60 con 0x7fe20c136fe0 2023-09-30T22:31:50.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.277+0000 7fe20bfff700 1 --2- 172.21.15.16:0/2499550950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe20c136fe0 0x7fe20c1393c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fe204009e50 tx=0x7fe20400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d8cea1fa68b73cf server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:50.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.278+0000 7fe20a7fc700 1 -- 172.21.15.16:0/2499550950 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20400e070 con 0x7fe20c136fe0 2023-09-30T22:31:50.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.278+0000 7fe20a7fc700 1 -- 172.21.15.16:0/2499550950 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe20400b910 con 0x7fe20c136fe0 2023-09-30T22:31:50.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.278+0000 7fe20a7fc700 1 -- 172.21.15.16:0/2499550950 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20400bcc0 con 0x7fe20c136fe0 2023-09-30T22:31:50.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.278+0000 7fe211d70700 1 -- 172.21.15.16:0/2499550950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe20c136fe0 msgr2=0x7fe20c1393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:50.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.279+0000 7fe211d70700 1 --2- 172.21.15.16:0/2499550950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe20c136fe0 0x7fe20c1393c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fe204009e50 tx=0x7fe20400b330 comp rx=0 tx=0).stop 2023-09-30T22:31:50.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.279+0000 7fe211d70700 1 -- 172.21.15.16:0/2499550950 shutdown_connections 2023-09-30T22:31:50.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.279+0000 7fe211d70700 1 --2- 172.21.15.16:0/2499550950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe20c136fe0 0x7fe20c1393c0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:50.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.279+0000 7fe211d70700 1 --2- 172.21.15.16:0/2499550950 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe20c09f410 0x7fe20c136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:50.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.279+0000 7fe211d70700 1 -- 172.21.15.16:0/2499550950 >> 172.21.15.16:0/2499550950 conn(0x7fe20c0a4210 msgr2=0x7fe20c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:50.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.279+0000 7fe211d70700 1 -- 172.21.15.16:0/2499550950 shutdown_connections 2023-09-30T22:31:50.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.279+0000 7fe211d70700 1 -- 172.21.15.16:0/2499550950 wait complete. 2023-09-30T22:31:50.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.280+0000 7fe211d70700 1 Processor -- start 2023-09-30T22:31:50.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.280+0000 7fe211d70700 1 -- start start 2023-09-30T22:31:50.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.280+0000 7fe211d70700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe20c09e930 0x7fe20c11e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:50.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.280+0000 7fe211d70700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe20c09f410 0x7fe20c11ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:50.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.281+0000 7fe211d70700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe20c136fe0 0x7fe20c11f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:50.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.281+0000 7fe211d70700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20c121cb0 con 0x7fe20c136fe0 2023-09-30T22:31:50.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.281+0000 7fe211d70700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe20c09de30 con 0x7fe20c09f410 2023-09-30T22:31:50.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.281+0000 7fe211d70700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe20c120370 con 0x7fe20c09e930 2023-09-30T22:31:50.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.281+0000 7fe20bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe20c136fe0 0x7fe20c11f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:50.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.281+0000 7fe20bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe20c136fe0 0x7fe20c11f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54392/0 (socket says 172.21.15.16:54392) 2023-09-30T22:31:50.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.281+0000 7fe20bfff700 1 -- 172.21.15.16:0/1058021333 learned_addr learned my addr 172.21.15.16:0/1058021333 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:50.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.281+0000 7fe20affd700 1 --2- 172.21.15.16:0/1058021333 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe20c09f410 0x7fe20c11ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:50.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.281+0000 7fe20bfff700 1 -- 172.21.15.16:0/1058021333 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe20c09e930 msgr2=0x7fe20c11e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:50.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.281+0000 7fe20bfff700 1 --2- 172.21.15.16:0/1058021333 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe20c09e930 0x7fe20c11e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:50.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.281+0000 7fe20bfff700 1 -- 172.21.15.16:0/1058021333 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe20c09f410 msgr2=0x7fe20c11ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:50.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.281+0000 7fe20bfff700 1 --2- 172.21.15.16:0/1058021333 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe20c09f410 0x7fe20c11ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:50.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.281+0000 7fe20bfff700 1 -- 172.21.15.16:0/1058021333 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe204009b30 con 0x7fe20c136fe0 2023-09-30T22:31:50.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.281+0000 7fe20bfff700 1 --2- 172.21.15.16:0/1058021333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe20c136fe0 0x7fe20c11f120 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fe204012000 tx=0x7fe204004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:50.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.282+0000 7fe1f3fff700 1 -- 172.21.15.16:0/1058021333 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe204003ea0 con 0x7fe20c136fe0 2023-09-30T22:31:50.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.282+0000 7fe211d70700 1 -- 172.21.15.16:0/1058021333 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe20c071d20 con 0x7fe20c136fe0 2023-09-30T22:31:50.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.282+0000 7fe211d70700 1 -- 172.21.15.16:0/1058021333 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe20c120a50 con 0x7fe20c136fe0 2023-09-30T22:31:50.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.282+0000 7fe1f3fff700 1 -- 172.21.15.16:0/1058021333 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2040092e0 con 0x7fe20c136fe0 2023-09-30T22:31:50.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.282+0000 7fe1f3fff700 1 -- 172.21.15.16:0/1058021333 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe204004970 con 0x7fe20c136fe0 2023-09-30T22:31:50.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.283+0000 7fe1f3fff700 1 -- 172.21.15.16:0/1058021333 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe204025650 con 0x7fe20c136fe0 2023-09-30T22:31:50.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.283+0000 7fe211d70700 1 -- 172.21.15.16:0/1058021333 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1ec0052e0 con 0x7fe20c136fe0 2023-09-30T22:31:50.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.283+0000 7fe1f3fff700 1 -- 172.21.15.16:0/1058021333 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fe204041330 con 0x7fe20c136fe0 2023-09-30T22:31:50.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.288+0000 7fe1f3fff700 1 -- 172.21.15.16:0/1058021333 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe20402d940 con 0x7fe20c136fe0 2023-09-30T22:31:50.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.325+0000 7f2064b85700 1 Processor -- start 2023-09-30T22:31:50.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.325+0000 7f2064b85700 1 -- start start 2023-09-30T22:31:50.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.326+0000 7f2064b85700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f206009d6f0 0x7f206009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:50.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.326+0000 7f2064b85700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f206009e000 0x7f2060130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:50.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.326+0000 7f2064b85700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f206009e410 0x7f2060130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:50.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.326+0000 7f2064b85700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f206009f520 con 0x7f206009e410 2023-09-30T22:31:50.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.326+0000 7f2064b85700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f206009f130 con 0x7f206009d6f0 2023-09-30T22:31:50.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.326+0000 7f2064b85700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f206009e930 con 0x7f206009e000 2023-09-30T22:31:50.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.326+0000 7f205e59c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f206009d6f0 0x7f206009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:50.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.326+0000 7f205dd9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f206009e000 0x7f2060130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:50.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.326+0000 7f205e59c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f206009d6f0 0x7f206009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:33114/0 (socket says 172.21.15.16:33114) 2023-09-30T22:31:50.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.326+0000 7f205e59c700 1 -- 172.21.15.16:0/1010671421 learned_addr learned my addr 172.21.15.16:0/1010671421 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:50.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.326+0000 7f205ed9d700 1 --2- 172.21.15.16:0/1010671421 >> v2:172.21.15.16:3300/0 conn(0x7f206009e410 0x7f2060130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:50.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.327+0000 7f205e59c700 1 -- 172.21.15.16:0/1010671421 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f206009e000 msgr2=0x7f2060130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:50.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.327+0000 7f205e59c700 1 --2- 172.21.15.16:0/1010671421 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f206009e000 0x7f2060130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:50.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.327+0000 7f205e59c700 1 -- 172.21.15.16:0/1010671421 >> v2:172.21.15.16:3300/0 conn(0x7f206009e410 msgr2=0x7f2060130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:50.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.327+0000 7f205e59c700 1 --2- 172.21.15.16:0/1010671421 >> v2:172.21.15.16:3300/0 conn(0x7f206009e410 0x7f2060130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:50.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.327+0000 7f205e59c700 1 -- 172.21.15.16:0/1010671421 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f206009b2d0 con 0x7f206009d6f0 2023-09-30T22:31:50.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.327+0000 7f205e59c700 1 --2- 172.21.15.16:0/1010671421 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f206009d6f0 0x7f206009dac0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2050009920 tx=0x7f2050009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90ba4a392ccbd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:50.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.327+0000 7f205d59a700 1 -- 172.21.15.16:0/1010671421 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2050011830 con 0x7f206009d6f0 2023-09-30T22:31:50.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.327+0000 7f205d59a700 1 -- 172.21.15.16:0/1010671421 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2050011990 con 0x7f206009d6f0 2023-09-30T22:31:50.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.327+0000 7f205d59a700 1 -- 172.21.15.16:0/1010671421 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2050011d20 con 0x7f206009d6f0 2023-09-30T22:31:50.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.328+0000 7f2064b85700 1 -- 172.21.15.16:0/1010671421 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f206009d6f0 msgr2=0x7f206009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:50.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.328+0000 7f2064b85700 1 --2- 172.21.15.16:0/1010671421 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f206009d6f0 0x7f206009dac0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2050009920 tx=0x7f2050009c30 comp rx=0 tx=0).stop 2023-09-30T22:31:50.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.328+0000 7f2064b85700 1 -- 172.21.15.16:0/1010671421 shutdown_connections 2023-09-30T22:31:50.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.328+0000 7f2064b85700 1 --2- 172.21.15.16:0/1010671421 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f206009d6f0 0x7f206009dac0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:50.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.328+0000 7f2064b85700 1 --2- 172.21.15.16:0/1010671421 >> v2:172.21.15.16:3300/0 conn(0x7f206009e410 0x7f2060130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:50.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.328+0000 7f2064b85700 1 --2- 172.21.15.16:0/1010671421 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f206009e000 0x7f2060130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:50.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.328+0000 7f2064b85700 1 -- 172.21.15.16:0/1010671421 >> 172.21.15.16:0/1010671421 conn(0x7f2060066f50 msgr2=0x7f206009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:50.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.328+0000 7f2064b85700 1 -- 172.21.15.16:0/1010671421 shutdown_connections 2023-09-30T22:31:50.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.328+0000 7f2064b85700 1 -- 172.21.15.16:0/1010671421 wait complete. 2023-09-30T22:31:50.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.329+0000 7f2064b85700 1 Processor -- start 2023-09-30T22:31:50.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.330+0000 7f2064b85700 1 -- start start 2023-09-30T22:31:50.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.330+0000 7f2064b85700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f206009d6f0 0x7f206011df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:50.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.330+0000 7f2064b85700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f206009e000 0x7f206011e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:50.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.330+0000 7f2064b85700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f206009e410 0x7f206011e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:50.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.330+0000 7f2064b85700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2060121a30 con 0x7f206009e410 2023-09-30T22:31:50.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.330+0000 7f2064b85700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2060120e00 con 0x7f206009d6f0 2023-09-30T22:31:50.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.330+0000 7f2064b85700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f206011f370 con 0x7f206009e000 2023-09-30T22:31:50.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.330+0000 7f205dd9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f206009e000 0x7f206011e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:50.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.330+0000 7f205dd9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f206009e000 0x7f206011e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:60170/0 (socket says 172.21.15.16:60170) 2023-09-30T22:31:50.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.330+0000 7f205dd9b700 1 -- 172.21.15.16:0/877847974 learned_addr learned my addr 172.21.15.16:0/877847974 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:50.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.331+0000 7f205ed9d700 1 --2- 172.21.15.16:0/877847974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f206009e410 0x7f206011e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:50.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.331+0000 7f205dd9b700 1 -- 172.21.15.16:0/877847974 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f206009d6f0 msgr2=0x7f206011df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:50.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.331+0000 7f205dd9b700 1 --2- 172.21.15.16:0/877847974 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f206009d6f0 0x7f206011df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:50.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.331+0000 7f205dd9b700 1 -- 172.21.15.16:0/877847974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f206009e410 msgr2=0x7f206011e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:50.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.331+0000 7f205dd9b700 1 --2- 172.21.15.16:0/877847974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f206009e410 0x7f206011e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:50.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.331+0000 7f205dd9b700 1 -- 172.21.15.16:0/877847974 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2050009550 con 0x7f206009e000 2023-09-30T22:31:50.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.331+0000 7f205dd9b700 1 --2- 172.21.15.16:0/877847974 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f206009e000 0x7f206011e450 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f20540089a0 tx=0x7f2054008d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:50.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.331+0000 7f204effd700 1 -- 172.21.15.16:0/877847974 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205400b960 con 0x7f206009e000 2023-09-30T22:31:50.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.331+0000 7f204effd700 1 -- 172.21.15.16:0/877847974 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f205400bac0 con 0x7f206009e000 2023-09-30T22:31:50.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.331+0000 7f204effd700 1 -- 172.21.15.16:0/877847974 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205400be30 con 0x7f206009e000 2023-09-30T22:31:50.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.331+0000 7f2064b85700 1 -- 172.21.15.16:0/877847974 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20600a1680 con 0x7f206009e000 2023-09-30T22:31:50.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.331+0000 7f2064b85700 1 -- 172.21.15.16:0/877847974 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f206009c760 con 0x7f206009e000 2023-09-30T22:31:50.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.332+0000 7f204effd700 1 -- 172.21.15.16:0/877847974 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2054015400 con 0x7f206009e000 2023-09-30T22:31:50.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.333+0000 7f204effd700 1 -- 172.21.15.16:0/877847974 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f205403f010 con 0x7f206009e000 2023-09-30T22:31:50.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.333+0000 7f2064b85700 1 -- 172.21.15.16:0/877847974 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f206011ac20 con 0x7f206009e000 2023-09-30T22:31:50.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.342+0000 7f204effd700 1 -- 172.21.15.16:0/877847974 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f205401a020 con 0x7f206009e000 2023-09-30T22:31:50.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.374+0000 7f72708e9700 1 Processor -- start 2023-09-30T22:31:50.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.374+0000 7f72708e9700 1 -- start start 2023-09-30T22:31:50.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.375+0000 7f72708e9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f726c0a0a30 0x7f726c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:50.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.375+0000 7f72708e9700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f726c0a1340 0x7f726c133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:50.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.375+0000 7f72708e9700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f726c09edd0 con 0x7f726c133fa0 2023-09-30T22:31:50.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.375+0000 7f72708e9700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f726c09e1f0 con 0x7f726c0a1340 2023-09-30T22:31:50.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.375+0000 7f72708e9700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f726c09dc50 con 0x7f726c0a0a30 2023-09-30T22:31:50.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.375+0000 7f726ad9d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f726c133fa0 0x7f726c13ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50284/0 (socket says 172.21.15.16:50284) 2023-09-30T22:31:50.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.375+0000 7f7269d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f726c0a1340 0x7f726c133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:50.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.375+0000 7f726ad9d700 1 -- 172.21.15.16:0/4185252213 learned_addr learned my addr 172.21.15.16:0/4185252213 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:31:50.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.375+0000 7f726a59c700 1 --2- 172.21.15.16:0/4185252213 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f726c0a0a30 0x7f726c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:50.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.376+0000 7f726959a700 1 -- 172.21.15.16:0/4185252213 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2011270237 0 0) 0x7f726c09edd0 con 0x7f726c133fa0 2023-09-30T22:31:50.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.376+0000 7f7269d9b700 1 -- 172.21.15.16:0/4185252213 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f726c0a0a30 msgr2=0x7f726c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:50.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.376+0000 7f7269d9b700 1 --2- 172.21.15.16:0/4185252213 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f726c0a0a30 0x7f726c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:50.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.376+0000 7f7269d9b700 1 -- 172.21.15.16:0/4185252213 >> v1:172.21.15.16:6789/0 conn(0x7f726c133fa0 legacy=0x7f726c13ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:50.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.376+0000 7f7269d9b700 1 -- 172.21.15.16:0/4185252213 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f726c0a1fe0 con 0x7f726c0a1340 2023-09-30T22:31:50.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.376+0000 7f7269d9b700 1 --2- 172.21.15.16:0/4185252213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f726c0a1340 0x7f726c133a60 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f7260009840 tx=0x7f7260009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a11e19fefa46e4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:50.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.376+0000 7f726959a700 1 -- 172.21.15.16:0/4185252213 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7260011830 con 0x7f726c0a1340 2023-09-30T22:31:50.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.376+0000 7f726959a700 1 -- 172.21.15.16:0/4185252213 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7260011990 con 0x7f726c0a1340 2023-09-30T22:31:50.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.376+0000 7f726959a700 1 -- 172.21.15.16:0/4185252213 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7260011d20 con 0x7f726c0a1340 2023-09-30T22:31:50.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.377+0000 7f72708e9700 1 -- 172.21.15.16:0/4185252213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f726c0a1340 msgr2=0x7f726c133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:50.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.377+0000 7f72708e9700 1 --2- 172.21.15.16:0/4185252213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f726c0a1340 0x7f726c133a60 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f7260009840 tx=0x7f7260009b50 comp rx=0 tx=0).stop 2023-09-30T22:31:50.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.377+0000 7f72708e9700 1 -- 172.21.15.16:0/4185252213 shutdown_connections 2023-09-30T22:31:50.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.377+0000 7f72708e9700 1 --2- 172.21.15.16:0/4185252213 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f726c0a0a30 0x7f726c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:50.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.377+0000 7f72708e9700 1 --2- 172.21.15.16:0/4185252213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f726c0a1340 0x7f726c133a60 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:50.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.377+0000 7f72708e9700 1 -- 172.21.15.16:0/4185252213 >> 172.21.15.16:0/4185252213 conn(0x7f726c0a6550 msgr2=0x7f726c0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:50.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.377+0000 7f72708e9700 1 -- 172.21.15.16:0/4185252213 shutdown_connections 2023-09-30T22:31:50.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.377+0000 7f72708e9700 1 -- 172.21.15.16:0/4185252213 wait complete. 2023-09-30T22:31:50.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.379+0000 7f72708e9700 1 Processor -- start 2023-09-30T22:31:50.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.379+0000 7f72708e9700 1 -- start start 2023-09-30T22:31:50.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.379+0000 7f72708e9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f726c0a0a30 0x7f726c1233d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:50.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.379+0000 7f72708e9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f726c0a1340 0x7f726c123910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:50.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.379+0000 7f72708e9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f726c133fa0 0x7f726c1c8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:50.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.379+0000 7f72708e9700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f726c122ff0 con 0x7f726c133fa0 2023-09-30T22:31:50.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.379+0000 7f72708e9700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f726c125fc0 con 0x7f726c0a0a30 2023-09-30T22:31:50.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.379+0000 7f72708e9700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f726c123e50 con 0x7f726c0a1340 2023-09-30T22:31:50.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.380+0000 7f726a59c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f726c0a0a30 0x7f726c1233d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:50.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.380+0000 7f726a59c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f726c0a0a30 0x7f726c1233d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:33142/0 (socket says 172.21.15.16:33142) 2023-09-30T22:31:50.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.380+0000 7f726a59c700 1 -- 172.21.15.16:0/3349392213 learned_addr learned my addr 172.21.15.16:0/3349392213 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:50.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.380+0000 7f7269d9b700 1 --2- 172.21.15.16:0/3349392213 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f726c0a1340 0x7f726c123910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:50.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.380+0000 7f726ad9d700 1 --2- 172.21.15.16:0/3349392213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f726c133fa0 0x7f726c1c8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:50.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.380+0000 7f726a59c700 1 -- 172.21.15.16:0/3349392213 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f726c0a1340 msgr2=0x7f726c123910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:50.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.380+0000 7f726a59c700 1 --2- 172.21.15.16:0/3349392213 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f726c0a1340 0x7f726c123910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:50.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.380+0000 7f726a59c700 1 -- 172.21.15.16:0/3349392213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f726c133fa0 msgr2=0x7f726c1c8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:50.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.380+0000 7f726a59c700 1 --2- 172.21.15.16:0/3349392213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f726c133fa0 0x7f726c1c8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:50.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.380+0000 7f726a59c700 1 -- 172.21.15.16:0/3349392213 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72600094a0 con 0x7f726c0a0a30 2023-09-30T22:31:50.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.380+0000 7f726a59c700 1 --2- 172.21.15.16:0/3349392213 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f726c0a0a30 0x7f726c1233d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f725c00e9b0 tx=0x7f725c00ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:50.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.381+0000 7f725affd700 1 -- 172.21.15.16:0/3349392213 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725c009710 con 0x7f726c0a0a30 2023-09-30T22:31:50.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.381+0000 7f725affd700 1 -- 172.21.15.16:0/3349392213 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f725c00ee50 con 0x7f726c0a0a30 2023-09-30T22:31:50.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.381+0000 7f72708e9700 1 -- 172.21.15.16:0/3349392213 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f726c0a0220 con 0x7f726c0a0a30 2023-09-30T22:31:50.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.381+0000 7f725affd700 1 -- 172.21.15.16:0/3349392213 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725c00ca50 con 0x7f726c0a0a30 2023-09-30T22:31:50.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.381+0000 7f72708e9700 1 -- 172.21.15.16:0/3349392213 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f726c133760 con 0x7f726c0a0a30 2023-09-30T22:31:50.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.382+0000 7f725affd700 1 -- 172.21.15.16:0/3349392213 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f725c00cbb0 con 0x7f726c0a0a30 2023-09-30T22:31:50.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.382+0000 7f725affd700 1 -- 172.21.15.16:0/3349392213 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f725c03f740 con 0x7f726c0a0a30 2023-09-30T22:31:50.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.382+0000 7f72708e9700 1 -- 172.21.15.16:0/3349392213 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f726c099a40 con 0x7f726c0a0a30 2023-09-30T22:31:50.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.387+0000 7f725affd700 1 -- 172.21.15.16:0/3349392213 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f725c0232f0 con 0x7f726c0a0a30 2023-09-30T22:31:50.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.451+0000 7f97658e9700 1 -- 172.21.15.16:0/2323609791 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f976009f420 con 0x7f976009e000 2023-09-30T22:31:50.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.452+0000 7f9747fff700 1 -- 172.21.15.16:0/2323609791 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f97540102c0 con 0x7f976009e000 2023-09-30T22:31:50.453 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:31:50.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.454+0000 7f97658e9700 1 -- 172.21.15.16:0/2323609791 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f976009e000 msgr2=0x7f97601c8560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:50.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.454+0000 7f97658e9700 1 --2- 172.21.15.16:0/2323609791 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f976009e000 0x7f97601c8560 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f975400c520 tx=0x7f975400c830 comp rx=0 tx=0).stop 2023-09-30T22:31:50.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.454+0000 7f97658e9700 1 -- 172.21.15.16:0/2323609791 shutdown_connections 2023-09-30T22:31:50.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.454+0000 7f97658e9700 1 --2- 172.21.15.16:0/2323609791 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f976009e440 0x7f97601c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:50.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.454+0000 7f97658e9700 1 --2- 172.21.15.16:0/2323609791 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f976009d6f0 0x7f97601c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:50.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.454+0000 7f97658e9700 1 --2- 172.21.15.16:0/2323609791 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f976009e000 0x7f97601c8560 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:50.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.454+0000 7f97658e9700 1 -- 172.21.15.16:0/2323609791 >> 172.21.15.16:0/2323609791 conn(0x7f9760066f50 msgr2=0x7f97600a3550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:50.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.455+0000 7f97658e9700 1 -- 172.21.15.16:0/2323609791 shutdown_connections 2023-09-30T22:31:50.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.455+0000 7f97658e9700 1 -- 172.21.15.16:0/2323609791 wait complete. 2023-09-30T22:31:50.470 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:31:50.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.563+0000 7fe211d70700 1 -- 172.21.15.16:0/1058021333 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe1ec005590 con 0x7fe20c136fe0 2023-09-30T22:31:50.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.564+0000 7fe1f3fff700 1 -- 172.21.15.16:0/1058021333 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe20402d350 con 0x7fe20c136fe0 2023-09-30T22:31:50.564 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:31:50.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.566+0000 7fe211d70700 1 -- 172.21.15.16:0/1058021333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe20c136fe0 msgr2=0x7fe20c11f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:50.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.566+0000 7fe211d70700 1 --2- 172.21.15.16:0/1058021333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe20c136fe0 0x7fe20c11f120 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fe204012000 tx=0x7fe204004ad0 comp rx=0 tx=0).stop 2023-09-30T22:31:50.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.566+0000 7fe211d70700 1 -- 172.21.15.16:0/1058021333 shutdown_connections 2023-09-30T22:31:50.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.566+0000 7fe211d70700 1 --2- 172.21.15.16:0/1058021333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe20c136fe0 0x7fe20c11f120 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:50.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.566+0000 7fe211d70700 1 --2- 172.21.15.16:0/1058021333 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe20c09e930 0x7fe20c11e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:50.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.566+0000 7fe211d70700 1 --2- 172.21.15.16:0/1058021333 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe20c09f410 0x7fe20c11ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:50.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.566+0000 7fe211d70700 1 -- 172.21.15.16:0/1058021333 >> 172.21.15.16:0/1058021333 conn(0x7fe20c0a4210 msgr2=0x7fe20c134890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:50.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.566+0000 7fe211d70700 1 -- 172.21.15.16:0/1058021333 shutdown_connections 2023-09-30T22:31:50.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.566+0000 7fe211d70700 1 -- 172.21.15.16:0/1058021333 wait complete. 2023-09-30T22:31:50.580 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:31:50.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.594+0000 7f72708e9700 1 -- 172.21.15.16:0/3349392213 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f726c0a1d40 con 0x7f726c0a0a30 2023-09-30T22:31:50.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.595+0000 7f725affd700 1 -- 172.21.15.16:0/3349392213 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f725c017020 con 0x7f726c0a0a30 2023-09-30T22:31:50.595 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:31:50.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.597+0000 7f72708e9700 1 -- 172.21.15.16:0/3349392213 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f726c0a0a30 msgr2=0x7f726c1233d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:50.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.597+0000 7f72708e9700 1 --2- 172.21.15.16:0/3349392213 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f726c0a0a30 0x7f726c1233d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f725c00e9b0 tx=0x7f725c00ecc0 comp rx=0 tx=0).stop 2023-09-30T22:31:50.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.597+0000 7f72708e9700 1 -- 172.21.15.16:0/3349392213 shutdown_connections 2023-09-30T22:31:50.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.597+0000 7f72708e9700 1 --2- 172.21.15.16:0/3349392213 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f726c0a0a30 0x7f726c1233d0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:50.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.597+0000 7f72708e9700 1 --2- 172.21.15.16:0/3349392213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f726c133fa0 0x7f726c1c8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:50.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.597+0000 7f72708e9700 1 --2- 172.21.15.16:0/3349392213 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f726c0a1340 0x7f726c123910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:50.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.597+0000 7f72708e9700 1 -- 172.21.15.16:0/3349392213 >> 172.21.15.16:0/3349392213 conn(0x7f726c0a6550 msgr2=0x7f726c0a52f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:50.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.597+0000 7f72708e9700 1 -- 172.21.15.16:0/3349392213 shutdown_connections 2023-09-30T22:31:50.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.597+0000 7f72708e9700 1 -- 172.21.15.16:0/3349392213 wait complete. 2023-09-30T22:31:50.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.605+0000 7f2064b85700 1 -- 172.21.15.16:0/877847974 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f2060074ea0 con 0x7f206009e000 2023-09-30T22:31:50.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.606+0000 7f204effd700 1 -- 172.21.15.16:0/877847974 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f205401c2c0 con 0x7f206009e000 2023-09-30T22:31:50.606 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:31:50.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.608+0000 7f2064b85700 1 -- 172.21.15.16:0/877847974 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f206009e000 msgr2=0x7f206011e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:50.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.608+0000 7f2064b85700 1 --2- 172.21.15.16:0/877847974 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f206009e000 0x7f206011e450 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f20540089a0 tx=0x7f2054008d60 comp rx=0 tx=0).stop 2023-09-30T22:31:50.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.608+0000 7f2064b85700 1 -- 172.21.15.16:0/877847974 shutdown_connections 2023-09-30T22:31:50.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.608+0000 7f2064b85700 1 --2- 172.21.15.16:0/877847974 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f206009d6f0 0x7f206011df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:50.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.608+0000 7f2064b85700 1 --2- 172.21.15.16:0/877847974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f206009e410 0x7f206011e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:50.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.608+0000 7f2064b85700 1 --2- 172.21.15.16:0/877847974 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f206009e000 0x7f206011e450 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:50.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.608+0000 7f2064b85700 1 -- 172.21.15.16:0/877847974 >> 172.21.15.16:0/877847974 conn(0x7f2060066f50 msgr2=0x7f2060130990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:50.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.608+0000 7f2064b85700 1 -- 172.21.15.16:0/877847974 shutdown_connections 2023-09-30T22:31:50.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:50.608+0000 7f2064b85700 1 -- 172.21.15.16:0/877847974 wait complete. 2023-09-30T22:31:50.612 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:31:50.622 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:31:51.471 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:51.581 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:51.613 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:51.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.619+0000 7fb29def6700 1 Processor -- start 2023-09-30T22:31:51.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.619+0000 7fb29def6700 1 -- start start 2023-09-30T22:31:51.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.620+0000 7fb29def6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb2980a0a30 0x7fb2980a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:51.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.620+0000 7fb29def6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb29809d630 0x7fb2980a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:51.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.620+0000 7fb29def6700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2980a2a70 con 0x7fb298133930 2023-09-30T22:31:51.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.620+0000 7fb29def6700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2980a01d0 con 0x7fb29809d630 2023-09-30T22:31:51.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.620+0000 7fb29def6700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb29809e2b0 con 0x7fb2980a0a30 2023-09-30T22:31:51.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.620+0000 7fb2977fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb2980a0a30 0x7fb2980a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:51.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.620+0000 7fb296ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb29809d630 0x7fb2980a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:51.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.620+0000 7fb2977fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb2980a0a30 0x7fb2980a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:60194/0 (socket says 172.21.15.16:60194) 2023-09-30T22:31:51.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.620+0000 7fb2977fe700 1 -- 172.21.15.16:0/72541088 learned_addr learned my addr 172.21.15.16:0/72541088 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:51.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.620+0000 7fb296ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb29809d630 0x7fb2980a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:33156/0 (socket says 172.21.15.16:33156) 2023-09-30T22:31:51.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.621+0000 7fb2977fe700 1 -- 172.21.15.16:0/72541088 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb29809d630 msgr2=0x7fb2980a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:51.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.621+0000 7fb2977fe700 1 --2- 172.21.15.16:0/72541088 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb29809d630 0x7fb2980a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:51.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.621+0000 7fb2977fe700 1 -- 172.21.15.16:0/72541088 >> v1:172.21.15.16:6789/0 conn(0x7fb298133930 legacy=0x7fb298133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:51.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.621+0000 7fb2977fe700 1 -- 172.21.15.16:0/72541088 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb29809dc50 con 0x7fb2980a0a30 2023-09-30T22:31:51.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.621+0000 7fb296ffd700 1 --2- 172.21.15.16:0/72541088 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb29809d630 0x7fb2980a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:51.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.621+0000 7fb2977fe700 1 --2- 172.21.15.16:0/72541088 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb2980a0a30 0x7fb2980a0e00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb288009e80 tx=0x7fb28800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e112c06da80ac4b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:51.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.621+0000 7fb2967fc700 1 -- 172.21.15.16:0/72541088 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28800e070 con 0x7fb2980a0a30 2023-09-30T22:31:51.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.621+0000 7fb2967fc700 1 -- 172.21.15.16:0/72541088 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb28800b910 con 0x7fb2980a0a30 2023-09-30T22:31:51.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.621+0000 7fb2967fc700 1 -- 172.21.15.16:0/72541088 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28800bcc0 con 0x7fb2980a0a30 2023-09-30T22:31:51.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.622+0000 7fb29def6700 1 -- 172.21.15.16:0/72541088 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb2980a0a30 msgr2=0x7fb2980a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:51.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.622+0000 7fb29def6700 1 --2- 172.21.15.16:0/72541088 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb2980a0a30 0x7fb2980a0e00 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb288009e80 tx=0x7fb28800b330 comp rx=0 tx=0).stop 2023-09-30T22:31:51.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.622+0000 7fb29def6700 1 -- 172.21.15.16:0/72541088 shutdown_connections 2023-09-30T22:31:51.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.622+0000 7fb29def6700 1 --2- 172.21.15.16:0/72541088 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb2980a0a30 0x7fb2980a0e00 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:51.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.622+0000 7fb29def6700 1 --2- 172.21.15.16:0/72541088 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb29809d630 0x7fb2980a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:51.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.622+0000 7fb29def6700 1 -- 172.21.15.16:0/72541088 >> 172.21.15.16:0/72541088 conn(0x7fb2980a3050 msgr2=0x7fb2980a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:51.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.622+0000 7fb29def6700 1 -- 172.21.15.16:0/72541088 shutdown_connections 2023-09-30T22:31:51.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.622+0000 7fb29def6700 1 -- 172.21.15.16:0/72541088 wait complete. 2023-09-30T22:31:51.625 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:51.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.623+0000 7fb29def6700 1 Processor -- start 2023-09-30T22:31:51.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.624+0000 7fb29def6700 1 -- start start 2023-09-30T22:31:51.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.624+0000 7fb29def6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb29809d630 0x7fb298121160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:51.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.624+0000 7fb29def6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2980a0a30 0x7fb2981216a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:51.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.624+0000 7fb29def6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb298133930 0x7fb298121be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:51.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.624+0000 7fb29def6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb298123ba0 con 0x7fb2980a0a30 2023-09-30T22:31:51.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.624+0000 7fb29def6700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb298124be0 con 0x7fb29809d630 2023-09-30T22:31:51.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.624+0000 7fb29def6700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb298124370 con 0x7fb298133930 2023-09-30T22:31:51.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.624+0000 7fb296ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2980a0a30 0x7fb2981216a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:51.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.624+0000 7fb296ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2980a0a30 0x7fb2981216a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54426/0 (socket says 172.21.15.16:54426) 2023-09-30T22:31:51.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.624+0000 7fb296ffd700 1 -- 172.21.15.16:0/3419102673 learned_addr learned my addr 172.21.15.16:0/3419102673 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:51.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.624+0000 7fb297fff700 1 --2- 172.21.15.16:0/3419102673 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb298133930 0x7fb298121be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:51.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.624+0000 7fb2977fe700 1 --2- 172.21.15.16:0/3419102673 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb29809d630 0x7fb298121160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:51.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.625+0000 7fb296ffd700 1 -- 172.21.15.16:0/3419102673 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb298133930 msgr2=0x7fb298121be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:51.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.625+0000 7fb296ffd700 1 --2- 172.21.15.16:0/3419102673 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb298133930 0x7fb298121be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:51.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.625+0000 7fb296ffd700 1 -- 172.21.15.16:0/3419102673 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb29809d630 msgr2=0x7fb298121160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:51.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.625+0000 7fb296ffd700 1 --2- 172.21.15.16:0/3419102673 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb29809d630 0x7fb298121160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:51.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.625+0000 7fb296ffd700 1 -- 172.21.15.16:0/3419102673 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2800094a0 con 0x7fb2980a0a30 2023-09-30T22:31:51.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.625+0000 7fb2977fe700 1 --2- 172.21.15.16:0/3419102673 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb29809d630 0x7fb298121160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:31:51.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.625+0000 7fb297fff700 1 --2- 172.21.15.16:0/3419102673 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb298133930 0x7fb298121be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:51.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.625+0000 7fb296ffd700 1 --2- 172.21.15.16:0/3419102673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2980a0a30 0x7fb2981216a0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fb28000ebb0 tx=0x7fb28000eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:51.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.625+0000 7fb28ffff700 1 -- 172.21.15.16:0/3419102673 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28000c850 con 0x7fb2980a0a30 2023-09-30T22:31:51.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.625+0000 7fb29def6700 1 -- 172.21.15.16:0/3419102673 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb288009b30 con 0x7fb2980a0a30 2023-09-30T22:31:51.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.625+0000 7fb29def6700 1 -- 172.21.15.16:0/3419102673 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2980a0410 con 0x7fb2980a0a30 2023-09-30T22:31:51.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.625+0000 7fb28ffff700 1 -- 172.21.15.16:0/3419102673 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb28000c9b0 con 0x7fb2980a0a30 2023-09-30T22:31:51.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.625+0000 7fb28ffff700 1 -- 172.21.15.16:0/3419102673 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28000cd20 con 0x7fb2980a0a30 2023-09-30T22:31:51.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.626+0000 7fb29def6700 1 -- 172.21.15.16:0/3419102673 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb27c0052e0 con 0x7fb2980a0a30 2023-09-30T22:31:51.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.627+0000 7fb28ffff700 1 -- 172.21.15.16:0/3419102673 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb280013470 con 0x7fb2980a0a30 2023-09-30T22:31:51.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.627+0000 7fb28ffff700 1 -- 172.21.15.16:0/3419102673 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fb28003fec0 con 0x7fb2980a0a30 2023-09-30T22:31:51.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.632+0000 7fb28ffff700 1 -- 172.21.15.16:0/3419102673 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb280024350 con 0x7fb2980a0a30 2023-09-30T22:31:51.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.732+0000 7fbe0ec36700 1 Processor -- start 2023-09-30T22:31:51.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.732+0000 7fbe0ec36700 1 -- start start 2023-09-30T22:31:51.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.733+0000 7fbe0ec36700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbe080954c0 0x7fbe08095890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:51.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.733+0000 7fbe0ec36700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbe08099680 0x7fbe08099ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:51.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.733+0000 7fbe0ec36700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe0809c280 con 0x7fbe08099ff0 2023-09-30T22:31:51.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.733+0000 7fbe0ec36700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0809aaf0 con 0x7fbe080954c0 2023-09-30T22:31:51.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.733+0000 7fbe0ec36700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0809b960 con 0x7fbe08099680 2023-09-30T22:31:51.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.733+0000 7fbe07fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbe08099680 0x7fbe08099ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:51.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.733+0000 7fbe07fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbe08099680 0x7fbe08099ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:60216/0 (socket says 172.21.15.16:60216) 2023-09-30T22:31:51.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.733+0000 7fbe07fff700 1 -- 172.21.15.16:0/3820008692 learned_addr learned my addr 172.21.15.16:0/3820008692 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:51.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.734+0000 7fbe07fff700 1 -- 172.21.15.16:0/3820008692 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbe080954c0 msgr2=0x7fbe08095890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:51.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.734+0000 7fbe07fff700 1 --2- 172.21.15.16:0/3820008692 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbe080954c0 0x7fbe08095890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:51.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.734+0000 7fbe07fff700 1 -- 172.21.15.16:0/3820008692 >> v1:172.21.15.16:6789/0 conn(0x7fbe08099ff0 legacy=0x7fbe08137590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:51.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.734+0000 7fbe07fff700 1 -- 172.21.15.16:0/3820008692 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe0809a7b0 con 0x7fbe08099680 2023-09-30T22:31:51.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.734+0000 7fbe07fff700 1 --2- 172.21.15.16:0/3820008692 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbe08099680 0x7fbe08099ab0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fbdf400bbc0 tx=0x7fbdf400bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f685a978a85fb6fc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:51.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.735+0000 7fbe077fe700 1 -- 172.21.15.16:0/3820008692 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf400dd90 con 0x7fbe08099680 2023-09-30T22:31:51.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.735+0000 7fbe077fe700 1 -- 172.21.15.16:0/3820008692 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdf4010470 con 0x7fbe08099680 2023-09-30T22:31:51.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.735+0000 7fbe077fe700 1 -- 172.21.15.16:0/3820008692 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf40107e0 con 0x7fbe08099680 2023-09-30T22:31:51.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.736+0000 7fbe0ec36700 1 -- 172.21.15.16:0/3820008692 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbe08099680 msgr2=0x7fbe08099ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:51.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.736+0000 7fbe0ec36700 1 --2- 172.21.15.16:0/3820008692 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbe08099680 0x7fbe08099ab0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fbdf400bbc0 tx=0x7fbdf400bed0 comp rx=0 tx=0).stop 2023-09-30T22:31:51.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.736+0000 7fbe0ec36700 1 -- 172.21.15.16:0/3820008692 shutdown_connections 2023-09-30T22:31:51.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.736+0000 7fbe0ec36700 1 --2- 172.21.15.16:0/3820008692 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbe080954c0 0x7fbe08095890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:51.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.736+0000 7fbe0ec36700 1 --2- 172.21.15.16:0/3820008692 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbe08099680 0x7fbe08099ab0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:51.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.736+0000 7fbe0ec36700 1 -- 172.21.15.16:0/3820008692 >> 172.21.15.16:0/3820008692 conn(0x7fbe080a2360 msgr2=0x7fbe080a2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:51.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.736+0000 7fbe0ec36700 1 -- 172.21.15.16:0/3820008692 shutdown_connections 2023-09-30T22:31:51.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.736+0000 7fbe0ec36700 1 -- 172.21.15.16:0/3820008692 wait complete. 2023-09-30T22:31:51.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.737+0000 7fbe0ec36700 1 Processor -- start 2023-09-30T22:31:51.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.737+0000 7fbe0ec36700 1 -- start start 2023-09-30T22:31:51.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.738+0000 7fbe0ec36700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbe080954c0 0x7fbe0808afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:51.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.738+0000 7fbe0ec36700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbe08099680 0x7fbe0808b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:51.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.738+0000 7fbe0ec36700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe08099ff0 0x7fbe0808ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:51.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.738+0000 7fbe0ec36700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe08071c50 con 0x7fbe08099ff0 2023-09-30T22:31:51.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.738+0000 7fbe0ec36700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe080a0dc0 con 0x7fbe080954c0 2023-09-30T22:31:51.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.738+0000 7fbe0ec36700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbe0808cfd0 con 0x7fbe08099680 2023-09-30T22:31:51.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.739+0000 7fbe07fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbe08099680 0x7fbe0808b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:51.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.739+0000 7fbe0c9d2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbe080954c0 0x7fbe0808afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:51.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.739+0000 7fbe07fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbe08099680 0x7fbe0808b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:60232/0 (socket says 172.21.15.16:60232) 2023-09-30T22:31:51.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.739+0000 7fbe07fff700 1 -- 172.21.15.16:0/3146441750 learned_addr learned my addr 172.21.15.16:0/3146441750 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:51.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.739+0000 7fbe0d1d3700 1 --2- 172.21.15.16:0/3146441750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe08099ff0 0x7fbe0808ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:51.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.739+0000 7fbe07fff700 1 -- 172.21.15.16:0/3146441750 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbe080954c0 msgr2=0x7fbe0808afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:51.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.739+0000 7fbe07fff700 1 --2- 172.21.15.16:0/3146441750 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbe080954c0 0x7fbe0808afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:51.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.739+0000 7fbe07fff700 1 -- 172.21.15.16:0/3146441750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe08099ff0 msgr2=0x7fbe0808ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:51.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.739+0000 7fbe07fff700 1 --2- 172.21.15.16:0/3146441750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe08099ff0 0x7fbe0808ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:51.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.739+0000 7fbe07fff700 1 -- 172.21.15.16:0/3146441750 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdf400b7f0 con 0x7fbe08099680 2023-09-30T22:31:51.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.739+0000 7fbe0c9d2700 1 --2- 172.21.15.16:0/3146441750 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbe080954c0 0x7fbe0808afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:51.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.739+0000 7fbe07fff700 1 --2- 172.21.15.16:0/3146441750 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbe08099680 0x7fbe0808b510 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fbdf4000c00 tx=0x7fbdf4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:51.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.740+0000 7fbe057fa700 1 -- 172.21.15.16:0/3146441750 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf4003b30 con 0x7fbe08099680 2023-09-30T22:31:51.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.740+0000 7fbe057fa700 1 -- 172.21.15.16:0/3146441750 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdf4003c90 con 0x7fbe08099680 2023-09-30T22:31:51.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.740+0000 7fbe057fa700 1 -- 172.21.15.16:0/3146441750 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf40105e0 con 0x7fbe08099680 2023-09-30T22:31:51.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.740+0000 7fbe0ec36700 1 -- 172.21.15.16:0/3146441750 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe0808da70 con 0x7fbe08099680 2023-09-30T22:31:51.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.740+0000 7fbe0ec36700 1 -- 172.21.15.16:0/3146441750 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe080724a0 con 0x7fbe08099680 2023-09-30T22:31:51.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.742+0000 7fbe0ec36700 1 -- 172.21.15.16:0/3146441750 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe0809b000 con 0x7fbe08099680 2023-09-30T22:31:51.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.742+0000 7fbe057fa700 1 -- 172.21.15.16:0/3146441750 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbdf4019400 con 0x7fbe08099680 2023-09-30T22:31:51.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.742+0000 7fbe057fa700 1 -- 172.21.15.16:0/3146441750 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fbdf4040a90 con 0x7fbe08099680 2023-09-30T22:31:51.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.747+0000 7fbe057fa700 1 -- 172.21.15.16:0/3146441750 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbdf400f3d0 con 0x7fbe08099680 2023-09-30T22:31:51.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.767+0000 7f2f33b95700 1 Processor -- start 2023-09-30T22:31:51.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.767+0000 7f2f33b95700 1 -- start start 2023-09-30T22:31:51.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.768+0000 7f2f33b95700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f2c09eca0 0x7f2f2c13cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:51.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.768+0000 7f2f33b95700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f2c06f050 0x7f2f2c13f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:51.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.768+0000 7f2f33b95700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f2c070ee0 con 0x7f2f2c06e570 2023-09-30T22:31:51.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.768+0000 7f2f33b95700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2c071980 con 0x7f2f2c09eca0 2023-09-30T22:31:51.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.768+0000 7f2f33b95700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2c071480 con 0x7f2f2c06f050 2023-09-30T22:31:51.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.768+0000 7f2f31931700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2f2c06e570 0x7f2f2c06e940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50308/0 (socket says 172.21.15.16:50308) 2023-09-30T22:31:51.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.768+0000 7f2f31931700 1 -- 172.21.15.16:0/2612903307 learned_addr learned my addr 172.21.15.16:0/2612903307 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:31:51.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.768+0000 7f2f32132700 1 --2- 172.21.15.16:0/2612903307 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f2c06f050 0x7f2f2c13f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:51.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.768+0000 7f2f31130700 1 --2- 172.21.15.16:0/2612903307 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f2c09eca0 0x7f2f2c13cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:51.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.769+0000 7f2f3092f700 1 -- 172.21.15.16:0/2612903307 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1943390493 0 0) 0x7f2f2c070ee0 con 0x7f2f2c06e570 2023-09-30T22:31:51.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.769+0000 7f2f3092f700 1 -- 172.21.15.16:0/2612903307 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f180035b0 con 0x7f2f2c06e570 2023-09-30T22:31:51.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.769+0000 7f2f32132700 1 -- 172.21.15.16:0/2612903307 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f2c09eca0 msgr2=0x7f2f2c13cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:51.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.769+0000 7f2f32132700 1 --2- 172.21.15.16:0/2612903307 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f2c09eca0 0x7f2f2c13cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:51.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.769+0000 7f2f32132700 1 -- 172.21.15.16:0/2612903307 >> v1:172.21.15.16:6789/0 conn(0x7f2f2c06e570 legacy=0x7f2f2c06e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:51.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.769+0000 7f2f32132700 1 -- 172.21.15.16:0/2612903307 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f2c071b90 con 0x7f2f2c06f050 2023-09-30T22:31:51.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.769+0000 7f2f32132700 1 --2- 172.21.15.16:0/2612903307 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f2c06f050 0x7f2f2c13f3f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2f24009dc0 tx=0x7f2f2400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40c883acd5589e69 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:51.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.769+0000 7f2f3092f700 1 -- 172.21.15.16:0/2612903307 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2400e070 con 0x7f2f2c06f050 2023-09-30T22:31:51.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.769+0000 7f2f3092f700 1 -- 172.21.15.16:0/2612903307 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f2400bb20 con 0x7f2f2c06f050 2023-09-30T22:31:51.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.769+0000 7f2f3092f700 1 -- 172.21.15.16:0/2612903307 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2400be90 con 0x7f2f2c06f050 2023-09-30T22:31:51.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.770+0000 7f2f33b95700 1 -- 172.21.15.16:0/2612903307 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f2c06f050 msgr2=0x7f2f2c13f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:51.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.770+0000 7f2f33b95700 1 --2- 172.21.15.16:0/2612903307 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f2c06f050 0x7f2f2c13f3f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2f24009dc0 tx=0x7f2f2400b330 comp rx=0 tx=0).stop 2023-09-30T22:31:51.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.770+0000 7f2f33b95700 1 -- 172.21.15.16:0/2612903307 shutdown_connections 2023-09-30T22:31:51.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.770+0000 7f2f33b95700 1 --2- 172.21.15.16:0/2612903307 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f2c06f050 0x7f2f2c13f3f0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:51.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.770+0000 7f2f33b95700 1 --2- 172.21.15.16:0/2612903307 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f2c09eca0 0x7f2f2c13cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:51.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.770+0000 7f2f33b95700 1 -- 172.21.15.16:0/2612903307 >> 172.21.15.16:0/2612903307 conn(0x7f2f2c0a2aa0 msgr2=0x7f2f2c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:51.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.770+0000 7f2f33b95700 1 -- 172.21.15.16:0/2612903307 shutdown_connections 2023-09-30T22:31:51.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.770+0000 7f2f33b95700 1 -- 172.21.15.16:0/2612903307 wait complete. 2023-09-30T22:31:51.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.771+0000 7f2f33b95700 1 Processor -- start 2023-09-30T22:31:51.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.771+0000 7f2f33b95700 1 -- start start 2023-09-30T22:31:51.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.772+0000 7f2f33b95700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f2c06e570 0x7f2f2c130390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:51.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.772+0000 7f2f33b95700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f2c06f050 0x7f2f2c1308d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:51.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.772+0000 7f2f33b95700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f2c137410 0x7f2f2c130e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:51.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.772+0000 7f2f33b95700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2c0a2210 con 0x7f2f2c137410 2023-09-30T22:31:51.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.772+0000 7f2f33b95700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2c070610 con 0x7f2f2c06f050 2023-09-30T22:31:51.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.772+0000 7f2f33b95700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2c138ef0 con 0x7f2f2c06e570 2023-09-30T22:31:51.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.772+0000 7f2f31130700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f2c06f050 0x7f2f2c1308d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:51.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.772+0000 7f2f31130700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f2c06f050 0x7f2f2c1308d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:33198/0 (socket says 172.21.15.16:33198) 2023-09-30T22:31:51.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.772+0000 7f2f31130700 1 -- 172.21.15.16:0/3948243778 learned_addr learned my addr 172.21.15.16:0/3948243778 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:51.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.772+0000 7f2f32132700 1 --2- 172.21.15.16:0/3948243778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f2c137410 0x7f2f2c130e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:51.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.773+0000 7f2f31130700 1 -- 172.21.15.16:0/3948243778 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f2c06e570 msgr2=0x7f2f2c130390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:51.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.773+0000 7f2f31130700 1 --2- 172.21.15.16:0/3948243778 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f2c06e570 0x7f2f2c130390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:51.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.773+0000 7f2f31130700 1 -- 172.21.15.16:0/3948243778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f2c137410 msgr2=0x7f2f2c130e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:51.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.773+0000 7f2f31130700 1 --2- 172.21.15.16:0/3948243778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f2c137410 0x7f2f2c130e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:51.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.773+0000 7f2f31130700 1 -- 172.21.15.16:0/3948243778 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f24009a70 con 0x7f2f2c06f050 2023-09-30T22:31:51.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.773+0000 7f2f31130700 1 --2- 172.21.15.16:0/3948243778 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f2c06f050 0x7f2f2c1308d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c00e980 tx=0x7f2f1c00ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:51.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.773+0000 7f2f227fc700 1 -- 172.21.15.16:0/3948243778 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c00eed0 con 0x7f2f2c06f050 2023-09-30T22:31:51.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.773+0000 7f2f33b95700 1 -- 172.21.15.16:0/3948243778 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f2c139750 con 0x7f2f2c06f050 2023-09-30T22:31:51.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.773+0000 7f2f33b95700 1 -- 172.21.15.16:0/3948243778 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f2c131be0 con 0x7f2f2c06f050 2023-09-30T22:31:51.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.773+0000 7f2f227fc700 1 -- 172.21.15.16:0/3948243778 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f1c00c850 con 0x7f2f2c06f050 2023-09-30T22:31:51.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.773+0000 7f2f227fc700 1 -- 172.21.15.16:0/3948243778 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c00cbc0 con 0x7f2f2c06f050 2023-09-30T22:31:51.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.774+0000 7f2f33b95700 1 -- 172.21.15.16:0/3948243778 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f140052e0 con 0x7f2f2c06f050 2023-09-30T22:31:51.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.774+0000 7f2f227fc700 1 -- 172.21.15.16:0/3948243778 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2f1c00cd20 con 0x7f2f2c06f050 2023-09-30T22:31:51.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.775+0000 7f2f227fc700 1 -- 172.21.15.16:0/3948243778 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f2f1c03fae0 con 0x7f2f2c06f050 2023-09-30T22:31:51.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.781+0000 7f391837f700 1 Processor -- start 2023-09-30T22:31:51.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.781+0000 7f391837f700 1 -- start start 2023-09-30T22:31:51.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.782+0000 7f391837f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f391009d6f0 0x7f391009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:51.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.782+0000 7f391837f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f391009e000 0x7f39101339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:51.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.782+0000 7f391837f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f391009e410 0x7f3910133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:51.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.782+0000 7f391837f700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f391006b460 con 0x7f391009d6f0 2023-09-30T22:31:51.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.782+0000 7f391837f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f391009b120 con 0x7f391009e410 2023-09-30T22:31:51.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.782+0000 7f391837f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f391009a7e0 con 0x7f391009e000 2023-09-30T22:31:51.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.782+0000 7f391591a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f391009e000 0x7f39101339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:51.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.782+0000 7f391611b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f391009d6f0 0x7f391009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:51.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.782+0000 7f391591a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f391009e000 0x7f39101339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:60258/0 (socket says 172.21.15.16:60258) 2023-09-30T22:31:51.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.782+0000 7f391591a700 1 -- 172.21.15.16:0/401194266 learned_addr learned my addr 172.21.15.16:0/401194266 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:51.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.782+0000 7f391691c700 1 --2- 172.21.15.16:0/401194266 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f391009e410 0x7f3910133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:51.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.782+0000 7f391611b700 1 -- 172.21.15.16:0/401194266 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f391009e000 msgr2=0x7f39101339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:51.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.782+0000 7f391611b700 1 --2- 172.21.15.16:0/401194266 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f391009e000 0x7f39101339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:51.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.782+0000 7f391611b700 1 -- 172.21.15.16:0/401194266 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f391009e410 msgr2=0x7f3910133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:51.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.782+0000 7f391611b700 1 --2- 172.21.15.16:0/401194266 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f391009e410 0x7f3910133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:51.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.782+0000 7f391611b700 1 -- 172.21.15.16:0/401194266 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f391009a500 con 0x7f391009d6f0 2023-09-30T22:31:51.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.782+0000 7f391591a700 1 --2- 172.21.15.16:0/401194266 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f391009e000 0x7f39101339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:51.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.782+0000 7f391691c700 1 --2- 172.21.15.16:0/401194266 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f391009e410 0x7f3910133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:31:51.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.783+0000 7f391611b700 1 --2- 172.21.15.16:0/401194266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f391009d6f0 0x7f391009dac0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f390c009840 tx=0x7f390c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61cfae670c48cb59 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:51.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.783+0000 7f3915119700 1 -- 172.21.15.16:0/401194266 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390c011830 con 0x7f391009d6f0 2023-09-30T22:31:51.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.783+0000 7f3915119700 1 -- 172.21.15.16:0/401194266 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f390c011990 con 0x7f391009d6f0 2023-09-30T22:31:51.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.783+0000 7f3915119700 1 -- 172.21.15.16:0/401194266 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390c011d20 con 0x7f391009d6f0 2023-09-30T22:31:51.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.784+0000 7f391837f700 1 -- 172.21.15.16:0/401194266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f391009d6f0 msgr2=0x7f391009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:51.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.784+0000 7f391837f700 1 --2- 172.21.15.16:0/401194266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f391009d6f0 0x7f391009dac0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f390c009840 tx=0x7f390c009b50 comp rx=0 tx=0).stop 2023-09-30T22:31:51.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.784+0000 7f391837f700 1 -- 172.21.15.16:0/401194266 shutdown_connections 2023-09-30T22:31:51.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.784+0000 7f391837f700 1 --2- 172.21.15.16:0/401194266 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f391009e410 0x7f3910133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:51.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.784+0000 7f391837f700 1 --2- 172.21.15.16:0/401194266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f391009d6f0 0x7f391009dac0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:51.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.784+0000 7f391837f700 1 --2- 172.21.15.16:0/401194266 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f391009e000 0x7f39101339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:51.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.784+0000 7f391837f700 1 -- 172.21.15.16:0/401194266 >> 172.21.15.16:0/401194266 conn(0x7f3910066f50 msgr2=0x7f391009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:51.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.784+0000 7f391837f700 1 -- 172.21.15.16:0/401194266 shutdown_connections 2023-09-30T22:31:51.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.784+0000 7f391837f700 1 -- 172.21.15.16:0/401194266 wait complete. 2023-09-30T22:31:51.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.784+0000 7f2f227fc700 1 -- 172.21.15.16:0/3948243778 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2f1c01b3b0 con 0x7f2f2c06f050 2023-09-30T22:31:51.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.785+0000 7f391837f700 1 Processor -- start 2023-09-30T22:31:51.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.785+0000 7f391837f700 1 -- start start 2023-09-30T22:31:51.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.785+0000 7f391837f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f391009d6f0 0x7f39101a26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:51.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.785+0000 7f391837f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f391009e000 0x7f39101a2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:51.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.785+0000 7f391837f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f391009e410 0x7f39101a3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:51.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.785+0000 7f391837f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39101a6250 con 0x7f391009d6f0 2023-09-30T22:31:51.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.785+0000 7f391837f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39101aa4b0 con 0x7f391009e410 2023-09-30T22:31:51.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.785+0000 7f391837f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f391009c710 con 0x7f391009e000 2023-09-30T22:31:51.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.786+0000 7f391691c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f391009e410 0x7f39101a3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:51.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.786+0000 7f391691c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f391009e410 0x7f39101a3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:33216/0 (socket says 172.21.15.16:33216) 2023-09-30T22:31:51.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.786+0000 7f391691c700 1 -- 172.21.15.16:0/1496556656 learned_addr learned my addr 172.21.15.16:0/1496556656 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:51.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.786+0000 7f391611b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f391009d6f0 0x7f39101a26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:51.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.786+0000 7f391591a700 1 --2- 172.21.15.16:0/1496556656 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f391009e000 0x7f39101a2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:51.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.786+0000 7f391691c700 1 -- 172.21.15.16:0/1496556656 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f391009e000 msgr2=0x7f39101a2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:51.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.786+0000 7f391691c700 1 --2- 172.21.15.16:0/1496556656 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f391009e000 0x7f39101a2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:51.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.786+0000 7f391691c700 1 -- 172.21.15.16:0/1496556656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f391009d6f0 msgr2=0x7f39101a26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:51.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.786+0000 7f391691c700 1 --2- 172.21.15.16:0/1496556656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f391009d6f0 0x7f39101a26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:51.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.786+0000 7f391691c700 1 -- 172.21.15.16:0/1496556656 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3900009550 con 0x7f391009e410 2023-09-30T22:31:51.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.787+0000 7f391691c700 1 --2- 172.21.15.16:0/1496556656 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f391009e410 0x7f39101a3120 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3908009fa0 tx=0x7f390800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:51.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.787+0000 7f3906ffd700 1 -- 172.21.15.16:0/1496556656 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390800ddb0 con 0x7f391009e410 2023-09-30T22:31:51.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.787+0000 7f391837f700 1 -- 172.21.15.16:0/1496556656 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f390c0094a0 con 0x7f391009e410 2023-09-30T22:31:51.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.787+0000 7f3906ffd700 1 -- 172.21.15.16:0/1496556656 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f390800f040 con 0x7f391009e410 2023-09-30T22:31:51.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.787+0000 7f391837f700 1 -- 172.21.15.16:0/1496556656 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39101aab70 con 0x7f391009e410 2023-09-30T22:31:51.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.787+0000 7f3906ffd700 1 -- 172.21.15.16:0/1496556656 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3908003c60 con 0x7f391009e410 2023-09-30T22:31:51.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.788+0000 7f391837f700 1 -- 172.21.15.16:0/1496556656 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39101a60b0 con 0x7f391009e410 2023-09-30T22:31:51.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.788+0000 7f3906ffd700 1 -- 172.21.15.16:0/1496556656 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3908004030 con 0x7f391009e410 2023-09-30T22:31:51.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.788+0000 7f3906ffd700 1 -- 172.21.15.16:0/1496556656 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f390803f710 con 0x7f391009e410 2023-09-30T22:31:51.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.798+0000 7f3906ffd700 1 -- 172.21.15.16:0/1496556656 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f39101a60b0 con 0x7f391009e410 2023-09-30T22:31:51.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.857+0000 7fb29def6700 1 -- 172.21.15.16:0/3419102673 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb27c005590 con 0x7fb2980a0a30 2023-09-30T22:31:51.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.858+0000 7fb28ffff700 1 -- 172.21.15.16:0/3419102673 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb280017020 con 0x7fb2980a0a30 2023-09-30T22:31:51.858 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:31:51.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.860+0000 7fb29def6700 1 -- 172.21.15.16:0/3419102673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2980a0a30 msgr2=0x7fb2981216a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:51.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.860+0000 7fb29def6700 1 --2- 172.21.15.16:0/3419102673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2980a0a30 0x7fb2981216a0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fb28000ebb0 tx=0x7fb28000eec0 comp rx=0 tx=0).stop 2023-09-30T22:31:51.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.860+0000 7fb29def6700 1 -- 172.21.15.16:0/3419102673 shutdown_connections 2023-09-30T22:31:51.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.860+0000 7fb29def6700 1 --2- 172.21.15.16:0/3419102673 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb29809d630 0x7fb298121160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:51.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.860+0000 7fb29def6700 1 --2- 172.21.15.16:0/3419102673 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb298133930 0x7fb298121be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:51.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.860+0000 7fb29def6700 1 --2- 172.21.15.16:0/3419102673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2980a0a30 0x7fb2981216a0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:51.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.860+0000 7fb29def6700 1 -- 172.21.15.16:0/3419102673 >> 172.21.15.16:0/3419102673 conn(0x7fb2980a3050 msgr2=0x7fb298133210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:51.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.860+0000 7fb29def6700 1 -- 172.21.15.16:0/3419102673 shutdown_connections 2023-09-30T22:31:51.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.860+0000 7fb29def6700 1 -- 172.21.15.16:0/3419102673 wait complete. 2023-09-30T22:31:51.875 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:31:51.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.966+0000 7fbe0ec36700 1 -- 172.21.15.16:0/3146441750 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fbe08074ea0 con 0x7fbe08099680 2023-09-30T22:31:51.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.967+0000 7fbe057fa700 1 -- 172.21.15.16:0/3146441750 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbdf4028020 con 0x7fbe08099680 2023-09-30T22:31:51.968 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:31:51.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.969+0000 7fbe0ec36700 1 -- 172.21.15.16:0/3146441750 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbe08099680 msgr2=0x7fbe0808b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:51.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.969+0000 7fbe0ec36700 1 --2- 172.21.15.16:0/3146441750 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbe08099680 0x7fbe0808b510 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fbdf4000c00 tx=0x7fbdf4004750 comp rx=0 tx=0).stop 2023-09-30T22:31:51.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.969+0000 7fbe0ec36700 1 -- 172.21.15.16:0/3146441750 shutdown_connections 2023-09-30T22:31:51.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.969+0000 7fbe0ec36700 1 --2- 172.21.15.16:0/3146441750 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbe080954c0 0x7fbe0808afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:51.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.969+0000 7fbe0ec36700 1 --2- 172.21.15.16:0/3146441750 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe08099ff0 0x7fbe0808ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:51.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.969+0000 7fbe0ec36700 1 --2- 172.21.15.16:0/3146441750 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbe08099680 0x7fbe0808b510 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:51.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.969+0000 7fbe0ec36700 1 -- 172.21.15.16:0/3146441750 >> 172.21.15.16:0/3146441750 conn(0x7fbe080a2360 msgr2=0x7fbe08094850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:51.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.969+0000 7fbe0ec36700 1 -- 172.21.15.16:0/3146441750 shutdown_connections 2023-09-30T22:31:51.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.969+0000 7fbe0ec36700 1 -- 172.21.15.16:0/3146441750 wait complete. 2023-09-30T22:31:51.984 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:31:51.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.995+0000 7f2f33b95700 1 -- 172.21.15.16:0/3948243778 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2f14005590 con 0x7f2f2c06f050 2023-09-30T22:31:51.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.995+0000 7f2f227fc700 1 -- 172.21.15.16:0/3948243778 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2f1c011ca0 con 0x7f2f2c06f050 2023-09-30T22:31:51.996 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:31:51.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.997+0000 7f2f33b95700 1 -- 172.21.15.16:0/3948243778 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f2c06f050 msgr2=0x7f2f2c1308d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:51.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.997+0000 7f2f33b95700 1 --2- 172.21.15.16:0/3948243778 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f2c06f050 0x7f2f2c1308d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c00e980 tx=0x7f2f1c00ec90 comp rx=0 tx=0).stop 2023-09-30T22:31:51.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.998+0000 7f2f33b95700 1 -- 172.21.15.16:0/3948243778 shutdown_connections 2023-09-30T22:31:51.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.998+0000 7f2f33b95700 1 --2- 172.21.15.16:0/3948243778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f2c137410 0x7f2f2c130e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:51.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.998+0000 7f2f33b95700 1 --2- 172.21.15.16:0/3948243778 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f2c06e570 0x7f2f2c130390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:51.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.998+0000 7f2f33b95700 1 --2- 172.21.15.16:0/3948243778 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f2c06f050 0x7f2f2c1308d0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:51.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.998+0000 7f2f33b95700 1 -- 172.21.15.16:0/3948243778 >> 172.21.15.16:0/3948243778 conn(0x7f2f2c0a2aa0 msgr2=0x7f2f2c09c970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:51.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.998+0000 7f2f33b95700 1 -- 172.21.15.16:0/3948243778 shutdown_connections 2023-09-30T22:31:52.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:51.998+0000 7f2f33b95700 1 -- 172.21.15.16:0/3948243778 wait complete. 2023-09-30T22:31:52.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:52.006+0000 7f391837f700 1 -- 172.21.15.16:0/1496556656 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f39101a60b0 con 0x7f391009e410 2023-09-30T22:31:52.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:52.006+0000 7f3906ffd700 1 -- 172.21.15.16:0/1496556656 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f39101a60b0 con 0x7f391009e410 2023-09-30T22:31:52.007 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:31:52.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:52.008+0000 7f391837f700 1 -- 172.21.15.16:0/1496556656 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f391009e410 msgr2=0x7f39101a3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:52.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:52.008+0000 7f391837f700 1 --2- 172.21.15.16:0/1496556656 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f391009e410 0x7f39101a3120 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3908009fa0 tx=0x7f390800d3b0 comp rx=0 tx=0).stop 2023-09-30T22:31:52.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:52.008+0000 7f391837f700 1 -- 172.21.15.16:0/1496556656 shutdown_connections 2023-09-30T22:31:52.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:52.008+0000 7f391837f700 1 --2- 172.21.15.16:0/1496556656 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f391009e410 0x7f39101a3120 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:52.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:52.008+0000 7f391837f700 1 --2- 172.21.15.16:0/1496556656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f391009d6f0 0x7f39101a26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:52.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:52.008+0000 7f391837f700 1 --2- 172.21.15.16:0/1496556656 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f391009e000 0x7f39101a2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:52.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:52.008+0000 7f391837f700 1 -- 172.21.15.16:0/1496556656 >> 172.21.15.16:0/1496556656 conn(0x7f3910066f50 msgr2=0x7f3910071360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:52.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:52.008+0000 7f391837f700 1 -- 172.21.15.16:0/1496556656 shutdown_connections 2023-09-30T22:31:52.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:52.009+0000 7f391837f700 1 -- 172.21.15.16:0/1496556656 wait complete. 2023-09-30T22:31:52.013 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:31:52.023 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:31:52.876 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:52.985 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:53.013 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:53.023 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:53.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.026+0000 7f829b18d700 1 Processor -- start 2023-09-30T22:31:53.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.026+0000 7f829b18d700 1 -- start start 2023-09-30T22:31:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.027+0000 7f829b18d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8294133420 0x7f82941337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.027+0000 7f829b18d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8294134260 0x7f82941306a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:53.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.027+0000 7f829b18d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f829406b990 con 0x7f8294133e30 2023-09-30T22:31:53.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.027+0000 7f829b18d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82940a0590 con 0x7f8294134260 2023-09-30T22:31:53.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.027+0000 7f829b18d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f829409fb80 con 0x7f8294133420 2023-09-30T22:31:53.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.027+0000 7f8298f29700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8294133420 0x7f82941337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:53.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.027+0000 7f8293fff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8294133e30 0x7f829412ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50316/0 (socket says 172.21.15.16:50316) 2023-09-30T22:31:53.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.027+0000 7f8293fff700 1 -- 172.21.15.16:0/2444209133 learned_addr learned my addr 172.21.15.16:0/2444209133 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:31:53.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.027+0000 7f829972a700 1 --2- 172.21.15.16:0/2444209133 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8294134260 0x7f82941306a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:53.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.027+0000 7f82937fe700 1 -- 172.21.15.16:0/2444209133 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4154120862 0 0) 0x7f829406b990 con 0x7f8294133e30 2023-09-30T22:31:53.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.027+0000 7f829972a700 1 -- 172.21.15.16:0/2444209133 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8294133420 msgr2=0x7f82941337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:53.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.027+0000 7f829972a700 1 --2- 172.21.15.16:0/2444209133 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8294133420 0x7f82941337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:53.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.027+0000 7f829972a700 1 -- 172.21.15.16:0/2444209133 >> v1:172.21.15.16:6789/0 conn(0x7f8294133e30 legacy=0x7f829412ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:53.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.027+0000 7f829972a700 1 -- 172.21.15.16:0/2444209133 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82940a0030 con 0x7f8294134260 2023-09-30T22:31:53.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.027+0000 7f8298f29700 1 --2- 172.21.15.16:0/2444209133 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8294133420 0x7f82941337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:53.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.027+0000 7f829972a700 1 --2- 172.21.15.16:0/2444209133 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8294134260 0x7f82941306a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f828c009dc0 tx=0x7f828c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35b262a04e095e46 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:53.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.028+0000 7f82937fe700 1 -- 172.21.15.16:0/2444209133 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828c00e070 con 0x7f8294134260 2023-09-30T22:31:53.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.028+0000 7f82937fe700 1 -- 172.21.15.16:0/2444209133 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f828c00bb20 con 0x7f8294134260 2023-09-30T22:31:53.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.028+0000 7f82937fe700 1 -- 172.21.15.16:0/2444209133 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828c012430 con 0x7f8294134260 2023-09-30T22:31:53.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.028+0000 7f829b18d700 1 -- 172.21.15.16:0/2444209133 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8294134260 msgr2=0x7f82941306a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:53.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.028+0000 7f829b18d700 1 --2- 172.21.15.16:0/2444209133 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8294134260 0x7f82941306a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f828c009dc0 tx=0x7f828c00b330 comp rx=0 tx=0).stop 2023-09-30T22:31:53.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.028+0000 7f829b18d700 1 -- 172.21.15.16:0/2444209133 shutdown_connections 2023-09-30T22:31:53.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.028+0000 7f829b18d700 1 --2- 172.21.15.16:0/2444209133 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8294134260 0x7f82941306a0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:53.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.028+0000 7f829b18d700 1 --2- 172.21.15.16:0/2444209133 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8294133420 0x7f82941337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:53.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.028+0000 7f829b18d700 1 -- 172.21.15.16:0/2444209133 >> 172.21.15.16:0/2444209133 conn(0x7f82940a4210 msgr2=0x7f82940a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:53.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.028+0000 7f829b18d700 1 -- 172.21.15.16:0/2444209133 shutdown_connections 2023-09-30T22:31:53.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.028+0000 7f829b18d700 1 -- 172.21.15.16:0/2444209133 wait complete. 2023-09-30T22:31:53.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.029+0000 7f829b18d700 1 Processor -- start 2023-09-30T22:31:53.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.029+0000 7f829b18d700 1 -- start start 2023-09-30T22:31:53.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.030+0000 7f829b18d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8294133420 0x7f829409c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:53.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.030+0000 7f829b18d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8294133e30 0x7f829409c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:53.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.030+0000 7f829b18d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8294134260 0x7f829409cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:53.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.030+0000 7f829b18d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829411feb0 con 0x7f8294134260 2023-09-30T22:31:53.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.030+0000 7f829b18d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f829412d9b0 con 0x7f8294133e30 2023-09-30T22:31:53.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.030+0000 7f829b18d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8294120100 con 0x7f8294133420 2023-09-30T22:31:53.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.030+0000 7f8298f29700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8294133420 0x7f829409c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:53.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.030+0000 7f8298f29700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8294133420 0x7f829409c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:60300/0 (socket says 172.21.15.16:60300) 2023-09-30T22:31:53.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.030+0000 7f8298f29700 1 -- 172.21.15.16:0/2546253879 learned_addr learned my addr 172.21.15.16:0/2546253879 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:53.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.030+0000 7f829972a700 1 --2- 172.21.15.16:0/2546253879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8294134260 0x7f829409cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:53.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.031+0000 7f829972a700 1 -- 172.21.15.16:0/2546253879 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8294133420 msgr2=0x7f829409c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:53.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.031+0000 7f829972a700 1 --2- 172.21.15.16:0/2546253879 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8294133420 0x7f829409c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:53.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.031+0000 7f829972a700 1 -- 172.21.15.16:0/2546253879 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8294133e30 msgr2=0x7f829409c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:53.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.031+0000 7f829972a700 1 --2- 172.21.15.16:0/2546253879 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8294133e30 0x7f829409c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:53.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.031+0000 7f829972a700 1 -- 172.21.15.16:0/2546253879 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82880094a0 con 0x7f8294134260 2023-09-30T22:31:53.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.031+0000 7f8298f29700 1 --2- 172.21.15.16:0/2546253879 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8294133420 0x7f829409c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:53.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.031+0000 7f829972a700 1 --2- 172.21.15.16:0/2546253879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8294134260 0x7f829409cf00 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f829406e650 tx=0x7f828c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:53.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.031+0000 7f82917fa700 1 -- 172.21.15.16:0/2546253879 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828c012430 con 0x7f8294134260 2023-09-30T22:31:53.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.031+0000 7f82917fa700 1 -- 172.21.15.16:0/2546253879 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f828c020070 con 0x7f8294134260 2023-09-30T22:31:53.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.031+0000 7f829b18d700 1 -- 172.21.15.16:0/2546253879 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f828c009a70 con 0x7f8294134260 2023-09-30T22:31:53.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.031+0000 7f82917fa700 1 -- 172.21.15.16:0/2546253879 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828c004280 con 0x7f8294134260 2023-09-30T22:31:53.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.032+0000 7f829b18d700 1 -- 172.21.15.16:0/2546253879 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8294131860 con 0x7f8294134260 2023-09-30T22:31:53.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.033+0000 7f82917fa700 1 -- 172.21.15.16:0/2546253879 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f828c0043e0 con 0x7f8294134260 2023-09-30T22:31:53.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.033+0000 7f829b18d700 1 -- 172.21.15.16:0/2546253879 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8294074ea0 con 0x7f8294134260 2023-09-30T22:31:53.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.033+0000 7f82917fa700 1 -- 172.21.15.16:0/2546253879 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f828c022070 con 0x7f8294134260 2023-09-30T22:31:53.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.038+0000 7f82917fa700 1 -- 172.21.15.16:0/2546253879 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f828c01b400 con 0x7f8294134260 2023-09-30T22:31:53.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.144+0000 7fe051a1c700 1 Processor -- start 2023-09-30T22:31:53.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.144+0000 7fe051a1c700 1 -- start start 2023-09-30T22:31:53.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.145+0000 7fe051a1c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe04c09d6f0 0x7fe04c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:53.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.145+0000 7fe051a1c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe04c09e000 0x7fe04c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:53.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.145+0000 7fe051a1c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe04c09e410 0x7fe04c133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:53.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.145+0000 7fe051a1c700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe04c06b460 con 0x7fe04c09e000 2023-09-30T22:31:53.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.145+0000 7fe051a1c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe04c09b120 con 0x7fe04c09e410 2023-09-30T22:31:53.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.145+0000 7fe051a1c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe04c09a7e0 con 0x7fe04c09d6f0 2023-09-30T22:31:53.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.145+0000 7fe04affd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe04c09d6f0 0x7fe04c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:53.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.145+0000 7fe04affd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe04c09d6f0 0x7fe04c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:60304/0 (socket says 172.21.15.16:60304) 2023-09-30T22:31:53.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.145+0000 7fe04affd700 1 -- 172.21.15.16:0/1573035221 learned_addr learned my addr 172.21.15.16:0/1573035221 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:53.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.145+0000 7fe04a7fc700 1 --2- 172.21.15.16:0/1573035221 >> v2:172.21.15.16:3300/0 conn(0x7fe04c09e000 0x7fe04c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:53.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.145+0000 7fe04b7fe700 1 --2- 172.21.15.16:0/1573035221 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe04c09e410 0x7fe04c133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:53.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.146+0000 7fe04a7fc700 1 -- 172.21.15.16:0/1573035221 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe04c09d6f0 msgr2=0x7fe04c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:53.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.146+0000 7fe04a7fc700 1 --2- 172.21.15.16:0/1573035221 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe04c09d6f0 0x7fe04c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:53.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.146+0000 7fe04a7fc700 1 -- 172.21.15.16:0/1573035221 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe04c09e410 msgr2=0x7fe04c133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:53.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.146+0000 7fe04a7fc700 1 --2- 172.21.15.16:0/1573035221 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe04c09e410 0x7fe04c133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:53.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.146+0000 7fe04a7fc700 1 -- 172.21.15.16:0/1573035221 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe04c09a500 con 0x7fe04c09e000 2023-09-30T22:31:53.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.146+0000 7fe04a7fc700 1 --2- 172.21.15.16:0/1573035221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe04c09e000 0x7fe04c1339f0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fe040009840 tx=0x7fe040009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df973efa86f61b65 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:53.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.146+0000 7fe049ffb700 1 -- 172.21.15.16:0/1573035221 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe040011830 con 0x7fe04c09e000 2023-09-30T22:31:53.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.146+0000 7fe049ffb700 1 -- 172.21.15.16:0/1573035221 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe040011990 con 0x7fe04c09e000 2023-09-30T22:31:53.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.146+0000 7fe049ffb700 1 -- 172.21.15.16:0/1573035221 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe040011d20 con 0x7fe04c09e000 2023-09-30T22:31:53.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.147+0000 7fe051a1c700 1 -- 172.21.15.16:0/1573035221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe04c09e000 msgr2=0x7fe04c1339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:53.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.147+0000 7fe051a1c700 1 --2- 172.21.15.16:0/1573035221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe04c09e000 0x7fe04c1339f0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fe040009840 tx=0x7fe040009b50 comp rx=0 tx=0).stop 2023-09-30T22:31:53.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.147+0000 7fe051a1c700 1 -- 172.21.15.16:0/1573035221 shutdown_connections 2023-09-30T22:31:53.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.147+0000 7fe051a1c700 1 --2- 172.21.15.16:0/1573035221 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe04c09e410 0x7fe04c133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:53.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.147+0000 7fe051a1c700 1 --2- 172.21.15.16:0/1573035221 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe04c09d6f0 0x7fe04c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:53.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.147+0000 7fe051a1c700 1 --2- 172.21.15.16:0/1573035221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe04c09e000 0x7fe04c1339f0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:53.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.147+0000 7fe051a1c700 1 -- 172.21.15.16:0/1573035221 >> 172.21.15.16:0/1573035221 conn(0x7fe04c066f50 msgr2=0x7fe04c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:53.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.147+0000 7fe051a1c700 1 -- 172.21.15.16:0/1573035221 shutdown_connections 2023-09-30T22:31:53.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.147+0000 7fe051a1c700 1 -- 172.21.15.16:0/1573035221 wait complete. 2023-09-30T22:31:53.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.149+0000 7fe051a1c700 1 Processor -- start 2023-09-30T22:31:53.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.149+0000 7fe051a1c700 1 -- start start 2023-09-30T22:31:53.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.149+0000 7fe051a1c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe04c09d6f0 0x7fe04c1a5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:53.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.149+0000 7fe051a1c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe04c09e000 0x7fe04c1a6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:53.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.149+0000 7fe051a1c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe04c09e410 0x7fe04c1a6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:53.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.149+0000 7fe051a1c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe04c1a7b50 con 0x7fe04c09d6f0 2023-09-30T22:31:53.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.149+0000 7fe051a1c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe04c1a8170 con 0x7fe04c09e410 2023-09-30T22:31:53.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.149+0000 7fe051a1c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe04c1a8da0 con 0x7fe04c09e000 2023-09-30T22:31:53.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.150+0000 7fe04a7fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe04c09e000 0x7fe04c1a6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:53.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.150+0000 7fe04a7fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe04c09e000 0x7fe04c1a6510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:60306/0 (socket says 172.21.15.16:60306) 2023-09-30T22:31:53.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.150+0000 7fe04a7fc700 1 -- 172.21.15.16:0/927255694 learned_addr learned my addr 172.21.15.16:0/927255694 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:53.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.150+0000 7fe04b7fe700 1 --2- 172.21.15.16:0/927255694 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe04c09e410 0x7fe04c1a6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:53.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.150+0000 7fe04a7fc700 1 -- 172.21.15.16:0/927255694 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe04c09e410 msgr2=0x7fe04c1a6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:53.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.150+0000 7fe04a7fc700 1 --2- 172.21.15.16:0/927255694 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe04c09e410 0x7fe04c1a6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:53.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.150+0000 7fe04a7fc700 1 -- 172.21.15.16:0/927255694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe04c09d6f0 msgr2=0x7fe04c1a5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:53.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.150+0000 7fe04a7fc700 1 --2- 172.21.15.16:0/927255694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe04c09d6f0 0x7fe04c1a5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:53.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.150+0000 7fe04a7fc700 1 -- 172.21.15.16:0/927255694 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0400094a0 con 0x7fe04c09e000 2023-09-30T22:31:53.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.150+0000 7fe04a7fc700 1 --2- 172.21.15.16:0/927255694 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe04c09e000 0x7fe04c1a6510 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe040009fd0 tx=0x7fe040004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:53.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.150+0000 7fe033fff700 1 -- 172.21.15.16:0/927255694 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe040004510 con 0x7fe04c09e000 2023-09-30T22:31:53.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.150+0000 7fe033fff700 1 -- 172.21.15.16:0/927255694 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe04001f070 con 0x7fe04c09e000 2023-09-30T22:31:53.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.150+0000 7fe051a1c700 1 -- 172.21.15.16:0/927255694 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe04c1a9780 con 0x7fe04c09e000 2023-09-30T22:31:53.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.151+0000 7fe033fff700 1 -- 172.21.15.16:0/927255694 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0400038e0 con 0x7fe04c09e000 2023-09-30T22:31:53.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.151+0000 7fe051a1c700 1 -- 172.21.15.16:0/927255694 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe04c09c710 con 0x7fe04c09e000 2023-09-30T22:31:53.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.152+0000 7fe033fff700 1 -- 172.21.15.16:0/927255694 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe040003a40 con 0x7fe04c09e000 2023-09-30T22:31:53.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.152+0000 7fe051a1c700 1 -- 172.21.15.16:0/927255694 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe04c09dd30 con 0x7fe04c09e000 2023-09-30T22:31:53.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.152+0000 7fe033fff700 1 -- 172.21.15.16:0/927255694 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fe040021070 con 0x7fe04c09e000 2023-09-30T22:31:53.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.157+0000 7fe033fff700 1 -- 172.21.15.16:0/927255694 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe040026480 con 0x7fe04c09e000 2023-09-30T22:31:53.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.172+0000 7f8ce44d8700 1 Processor -- start 2023-09-30T22:31:53.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.172+0000 7f8ce44d8700 1 -- start start 2023-09-30T22:31:53.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.173+0000 7f8ce44d8700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8cdc09d6f0 0x7f8cdc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:53.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.173+0000 7f8ce44d8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8cdc09e000 0x7f8cdc1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:53.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.173+0000 7f8ce44d8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8cdc09e410 0x7f8cdc133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:53.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.173+0000 7f8ce44d8700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cdc06b460 con 0x7f8cdc09d6f0 2023-09-30T22:31:53.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.173+0000 7f8ce44d8700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cdc09b120 con 0x7f8cdc09e410 2023-09-30T22:31:53.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.173+0000 7f8ce44d8700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8cdc09a7e0 con 0x7f8cdc09e000 2023-09-30T22:31:53.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.173+0000 7f8ce1a73700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8cdc09e000 0x7f8cdc1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:53.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.173+0000 7f8ce1a73700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8cdc09e000 0x7f8cdc1339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:60310/0 (socket says 172.21.15.16:60310) 2023-09-30T22:31:53.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.173+0000 7f8ce1a73700 1 -- 172.21.15.16:0/1718194130 learned_addr learned my addr 172.21.15.16:0/1718194130 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:53.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.173+0000 7f8ce2274700 1 --2- 172.21.15.16:0/1718194130 >> v2:172.21.15.16:3300/0 conn(0x7f8cdc09d6f0 0x7f8cdc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:53.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.173+0000 7f8ce2a75700 1 --2- 172.21.15.16:0/1718194130 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8cdc09e410 0x7f8cdc133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:53.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.174+0000 7f8ce1a73700 1 -- 172.21.15.16:0/1718194130 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8cdc09e410 msgr2=0x7f8cdc133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:53.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.174+0000 7f8ce1a73700 1 --2- 172.21.15.16:0/1718194130 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8cdc09e410 0x7f8cdc133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:53.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.174+0000 7f8ce1a73700 1 -- 172.21.15.16:0/1718194130 >> v2:172.21.15.16:3300/0 conn(0x7f8cdc09d6f0 msgr2=0x7f8cdc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:53.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.174+0000 7f8ce1a73700 1 --2- 172.21.15.16:0/1718194130 >> v2:172.21.15.16:3300/0 conn(0x7f8cdc09d6f0 0x7f8cdc09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:53.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.174+0000 7f8ce1a73700 1 -- 172.21.15.16:0/1718194130 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cdc09a500 con 0x7f8cdc09e000 2023-09-30T22:31:53.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.174+0000 7f8ce2274700 1 --2- 172.21.15.16:0/1718194130 >> v2:172.21.15.16:3300/0 conn(0x7f8cdc09d6f0 0x7f8cdc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:53.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.174+0000 7f8ce2a75700 1 --2- 172.21.15.16:0/1718194130 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8cdc09e410 0x7f8cdc133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:31:53.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.174+0000 7f8ce1a73700 1 --2- 172.21.15.16:0/1718194130 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8cdc09e000 0x7f8cdc1339f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc009920 tx=0x7f8ccc009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b33c5d40332dbb30 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:53.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.174+0000 7f8ce1272700 1 -- 172.21.15.16:0/1718194130 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc011830 con 0x7f8cdc09e000 2023-09-30T22:31:53.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.174+0000 7f8ce1272700 1 -- 172.21.15.16:0/1718194130 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ccc011990 con 0x7f8cdc09e000 2023-09-30T22:31:53.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.174+0000 7f8ce1272700 1 -- 172.21.15.16:0/1718194130 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc011d20 con 0x7f8cdc09e000 2023-09-30T22:31:53.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.175+0000 7f8ce44d8700 1 -- 172.21.15.16:0/1718194130 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8cdc09e000 msgr2=0x7f8cdc1339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:53.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.175+0000 7f8ce44d8700 1 --2- 172.21.15.16:0/1718194130 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8cdc09e000 0x7f8cdc1339f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc009920 tx=0x7f8ccc009c30 comp rx=0 tx=0).stop 2023-09-30T22:31:53.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.175+0000 7f8ce44d8700 1 -- 172.21.15.16:0/1718194130 shutdown_connections 2023-09-30T22:31:53.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.175+0000 7f8ce44d8700 1 --2- 172.21.15.16:0/1718194130 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8cdc09e410 0x7f8cdc133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:53.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.175+0000 7f8ce44d8700 1 --2- 172.21.15.16:0/1718194130 >> v2:172.21.15.16:3300/0 conn(0x7f8cdc09d6f0 0x7f8cdc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:53.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.175+0000 7f8ce44d8700 1 --2- 172.21.15.16:0/1718194130 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8cdc09e000 0x7f8cdc1339f0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:53.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.175+0000 7f8ce44d8700 1 -- 172.21.15.16:0/1718194130 >> 172.21.15.16:0/1718194130 conn(0x7f8cdc066f50 msgr2=0x7f8cdc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:53.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.175+0000 7f8ce44d8700 1 -- 172.21.15.16:0/1718194130 shutdown_connections 2023-09-30T22:31:53.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.175+0000 7f8ce44d8700 1 -- 172.21.15.16:0/1718194130 wait complete. 2023-09-30T22:31:53.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.176+0000 7f8ce44d8700 1 Processor -- start 2023-09-30T22:31:53.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.176+0000 7f8ce44d8700 1 -- start start 2023-09-30T22:31:53.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.176+0000 7f4d39f16700 1 Processor -- start 2023-09-30T22:31:53.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.177+0000 7f8ce44d8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8cdc09d6f0 0x7f8cdc1a5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:53.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.177+0000 7f8ce44d8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cdc09e000 0x7f8cdc1a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:53.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.177+0000 7f8ce44d8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8cdc09e410 0x7f8cdc1a67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:53.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.177+0000 7f8ce44d8700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cdc1aa1f0 con 0x7f8cdc09e000 2023-09-30T22:31:53.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.177+0000 7f8ce44d8700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cdc1a7920 con 0x7f8cdc09d6f0 2023-09-30T22:31:53.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.177+0000 7f8ce44d8700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8cdc1a7f40 con 0x7f8cdc09e410 2023-09-30T22:31:53.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.177+0000 7f8ce1a73700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cdc09e000 0x7f8cdc1a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:53.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.177+0000 7f8ce1a73700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cdc09e000 0x7f8cdc1a62a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54504/0 (socket says 172.21.15.16:54504) 2023-09-30T22:31:53.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.177+0000 7f8ce1a73700 1 -- 172.21.15.16:0/3037053399 learned_addr learned my addr 172.21.15.16:0/3037053399 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:53.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.177+0000 7f8ce2a75700 1 --2- 172.21.15.16:0/3037053399 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8cdc09e410 0x7f8cdc1a67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:53.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.177+0000 7f8ce2274700 1 --2- 172.21.15.16:0/3037053399 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8cdc09d6f0 0x7f8cdc1a5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:53.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.176+0000 7f4d39f16700 1 -- start start 2023-09-30T22:31:53.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.177+0000 7f4d39f16700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4d3412cf60 0x7f4d3406f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:53.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.177+0000 7f4d39f16700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4d3412f8b0 0x7f4d3412fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:53.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.177+0000 7f4d39f16700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d3412cd90 con 0x7f4d34130260 2023-09-30T22:31:53.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.177+0000 7f4d39f16700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d3412d330 con 0x7f4d3412f8b0 2023-09-30T22:31:53.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.177+0000 7f4d39f16700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4d341306e0 con 0x7f4d3412cf60 2023-09-30T22:31:53.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.177+0000 7f8ce1a73700 1 -- 172.21.15.16:0/3037053399 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8cdc09e410 msgr2=0x7f8cdc1a67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:53.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.177+0000 7f8ce1a73700 1 --2- 172.21.15.16:0/3037053399 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8cdc09e410 0x7f8cdc1a67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:53.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.177+0000 7f8ce1a73700 1 -- 172.21.15.16:0/3037053399 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8cdc09d6f0 msgr2=0x7f8cdc1a5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:53.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.177+0000 7f8ce1a73700 1 --2- 172.21.15.16:0/3037053399 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8cdc09d6f0 0x7f8cdc1a5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:53.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.177+0000 7f8ce1a73700 1 -- 172.21.15.16:0/3037053399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cd80094a0 con 0x7f8cdc09e000 2023-09-30T22:31:53.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.178+0000 7f4d33fff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4d34130260 0x7f4d34136c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50320/0 (socket says 172.21.15.16:50320) 2023-09-30T22:31:53.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.178+0000 7f4d33fff700 1 -- 172.21.15.16:0/2826180726 learned_addr learned my addr 172.21.15.16:0/2826180726 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:31:53.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.178+0000 7f8ce1a73700 1 --2- 172.21.15.16:0/3037053399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cdc09e000 0x7f8cdc1a62a0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc009c70 tx=0x7f8ccc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:53.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.178+0000 7f4d337fe700 1 --2- 172.21.15.16:0/2826180726 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4d3412cf60 0x7f4d3406f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:53.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.178+0000 7f4d327fc700 1 -- 172.21.15.16:0/2826180726 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1445130162 0 0) 0x7f4d3412cd90 con 0x7f4d34130260 2023-09-30T22:31:53.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.178+0000 7f4d327fc700 1 -- 172.21.15.16:0/2826180726 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d1c0035b0 con 0x7f4d34130260 2023-09-30T22:31:53.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.178+0000 7f4d337fe700 1 -- 172.21.15.16:0/2826180726 >> v2:172.21.15.16:3300/0 conn(0x7f4d3412f8b0 msgr2=0x7f4d3412fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:53.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.178+0000 7f4d337fe700 1 --2- 172.21.15.16:0/2826180726 >> v2:172.21.15.16:3300/0 conn(0x7f4d3412f8b0 0x7f4d3412fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:53.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.178+0000 7f4d337fe700 1 -- 172.21.15.16:0/2826180726 >> v1:172.21.15.16:6789/0 conn(0x7f4d34130260 legacy=0x7f4d34136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:53.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.178+0000 7f4d337fe700 1 -- 172.21.15.16:0/2826180726 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d34071780 con 0x7f4d3412cf60 2023-09-30T22:31:53.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.178+0000 7f8cd2ffd700 1 -- 172.21.15.16:0/3037053399 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc004300 con 0x7f8cdc09e000 2023-09-30T22:31:53.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.178+0000 7f8cd2ffd700 1 -- 172.21.15.16:0/3037053399 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ccc01f070 con 0x7f8cdc09e000 2023-09-30T22:31:53.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.178+0000 7f8cd2ffd700 1 -- 172.21.15.16:0/3037053399 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc0036a0 con 0x7f8cdc09e000 2023-09-30T22:31:53.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.179+0000 7f4d337fe700 1 --2- 172.21.15.16:0/2826180726 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4d3412cf60 0x7f4d3406f5d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4d2400a9e0 tx=0x7f4d2400acf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6272a4559f3cee71 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:53.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.179+0000 7f8ce44d8700 1 -- 172.21.15.16:0/3037053399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ccc009550 con 0x7f8cdc09e000 2023-09-30T22:31:53.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.179+0000 7f8ce44d8700 1 -- 172.21.15.16:0/3037053399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cdc1a8fc0 con 0x7f8cdc09e000 2023-09-30T22:31:53.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.179+0000 7f4d327fc700 1 -- 172.21.15.16:0/2826180726 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2400cb70 con 0x7f4d3412cf60 2023-09-30T22:31:53.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.179+0000 7f4d327fc700 1 -- 172.21.15.16:0/2826180726 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d2400ccd0 con 0x7f4d3412cf60 2023-09-30T22:31:53.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.179+0000 7f4d327fc700 1 -- 172.21.15.16:0/2826180726 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d24018600 con 0x7f4d3412cf60 2023-09-30T22:31:53.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.180+0000 7f4d39f16700 1 -- 172.21.15.16:0/2826180726 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4d3412cf60 msgr2=0x7f4d3406f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:53.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.180+0000 7f4d39f16700 1 --2- 172.21.15.16:0/2826180726 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4d3412cf60 0x7f4d3406f5d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4d2400a9e0 tx=0x7f4d2400acf0 comp rx=0 tx=0).stop 2023-09-30T22:31:53.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.180+0000 7f4d39f16700 1 -- 172.21.15.16:0/2826180726 shutdown_connections 2023-09-30T22:31:53.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.180+0000 7f4d39f16700 1 --2- 172.21.15.16:0/2826180726 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4d3412cf60 0x7f4d3406f5d0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:53.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.180+0000 7f4d39f16700 1 --2- 172.21.15.16:0/2826180726 >> v2:172.21.15.16:3300/0 conn(0x7f4d3412f8b0 0x7f4d3412fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:53.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.180+0000 7f4d39f16700 1 -- 172.21.15.16:0/2826180726 >> 172.21.15.16:0/2826180726 conn(0x7f4d340a2aa0 msgr2=0x7f4d340a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:53.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.180+0000 7f4d39f16700 1 -- 172.21.15.16:0/2826180726 shutdown_connections 2023-09-30T22:31:53.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.180+0000 7f8cd2ffd700 1 -- 172.21.15.16:0/3037053399 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8ccc021030 con 0x7f8cdc09e000 2023-09-30T22:31:53.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.181+0000 7f8ce44d8700 1 -- 172.21.15.16:0/3037053399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cdc0a1f20 con 0x7f8cdc09e000 2023-09-30T22:31:53.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.181+0000 7f8cd2ffd700 1 -- 172.21.15.16:0/3037053399 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f8ccc040a60 con 0x7f8cdc09e000 2023-09-30T22:31:53.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.180+0000 7f4d39f16700 1 -- 172.21.15.16:0/2826180726 wait complete. 2023-09-30T22:31:53.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.182+0000 7f4d39f16700 1 Processor -- start 2023-09-30T22:31:53.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.182+0000 7f4d39f16700 1 -- start start 2023-09-30T22:31:53.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.183+0000 7f4d39f16700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4d3412cf60 0x7f4d34139d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:53.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.183+0000 7f4d39f16700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4d3412f8b0 0x7f4d34138340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:53.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.183+0000 7f4d39f16700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d34130260 0x7f4d34138880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:53.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.183+0000 7f4d39f16700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d34131330 con 0x7f4d34130260 2023-09-30T22:31:53.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.183+0000 7f4d39f16700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d341309e0 con 0x7f4d3412f8b0 2023-09-30T22:31:53.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.183+0000 7f4d39f16700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3412d960 con 0x7f4d3412cf60 2023-09-30T22:31:53.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.183+0000 7f4d32ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4d3412f8b0 0x7f4d34138340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:53.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.183+0000 7f4d32ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4d3412f8b0 0x7f4d34138340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:33294/0 (socket says 172.21.15.16:33294) 2023-09-30T22:31:53.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.183+0000 7f4d32ffd700 1 -- 172.21.15.16:0/3830696269 learned_addr learned my addr 172.21.15.16:0/3830696269 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:53.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.183+0000 7f4d337fe700 1 --2- 172.21.15.16:0/3830696269 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4d3412cf60 0x7f4d34139d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:53.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.183+0000 7f4d33fff700 1 --2- 172.21.15.16:0/3830696269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d34130260 0x7f4d34138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:53.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.184+0000 7f4d337fe700 1 -- 172.21.15.16:0/3830696269 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4d3412f8b0 msgr2=0x7f4d34138340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:53.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.184+0000 7f4d337fe700 1 --2- 172.21.15.16:0/3830696269 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4d3412f8b0 0x7f4d34138340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:53.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.184+0000 7f4d337fe700 1 -- 172.21.15.16:0/3830696269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d34130260 msgr2=0x7f4d34138880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:53.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.184+0000 7f4d337fe700 1 --2- 172.21.15.16:0/3830696269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d34130260 0x7f4d34138880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:53.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.184+0000 7f4d337fe700 1 -- 172.21.15.16:0/3830696269 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d2400a640 con 0x7f4d3412cf60 2023-09-30T22:31:53.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.184+0000 7f4d32ffd700 1 --2- 172.21.15.16:0/3830696269 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4d3412f8b0 0x7f4d34138340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:53.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.184+0000 7f4d337fe700 1 --2- 172.21.15.16:0/3830696269 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4d3412cf60 0x7f4d34139d20 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4d24017000 tx=0x7f4d240047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:53.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.184+0000 7f4d1bfff700 1 -- 172.21.15.16:0/3830696269 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d24003ab0 con 0x7f4d3412cf60 2023-09-30T22:31:53.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.184+0000 7f4d1bfff700 1 -- 172.21.15.16:0/3830696269 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d24003c10 con 0x7f4d3412cf60 2023-09-30T22:31:53.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.184+0000 7f4d39f16700 1 -- 172.21.15.16:0/3830696269 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d34130c80 con 0x7f4d3412cf60 2023-09-30T22:31:53.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.184+0000 7f4d1bfff700 1 -- 172.21.15.16:0/3830696269 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2400cec0 con 0x7f4d3412cf60 2023-09-30T22:31:53.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.184+0000 7f4d39f16700 1 -- 172.21.15.16:0/3830696269 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d341397e0 con 0x7f4d3412cf60 2023-09-30T22:31:53.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.185+0000 7f4d1bfff700 1 -- 172.21.15.16:0/3830696269 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4d24018400 con 0x7f4d3412cf60 2023-09-30T22:31:53.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.186+0000 7f4d1bfff700 1 -- 172.21.15.16:0/3830696269 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f4d24040690 con 0x7f4d3412cf60 2023-09-30T22:31:53.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.186+0000 7f4d39f16700 1 -- 172.21.15.16:0/3830696269 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d34129fa0 con 0x7f4d3412cf60 2023-09-30T22:31:53.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.189+0000 7f8cd2ffd700 1 -- 172.21.15.16:0/3037053399 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8ccc0199a0 con 0x7f8cdc09e000 2023-09-30T22:31:53.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.192+0000 7f4d1bfff700 1 -- 172.21.15.16:0/3830696269 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4d2401e300 con 0x7f4d3412cf60 2023-09-30T22:31:53.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.265+0000 7f829b18d700 1 -- 172.21.15.16:0/2546253879 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f829406b250 con 0x7f8294134260 2023-09-30T22:31:53.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.265+0000 7f82917fa700 1 -- 172.21.15.16:0/2546253879 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f828c02c070 con 0x7f8294134260 2023-09-30T22:31:53.266 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:31:53.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.268+0000 7f829b18d700 1 -- 172.21.15.16:0/2546253879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8294134260 msgr2=0x7f829409cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:53.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.268+0000 7f829b18d700 1 --2- 172.21.15.16:0/2546253879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8294134260 0x7f829409cf00 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f829406e650 tx=0x7f828c004ad0 comp rx=0 tx=0).stop 2023-09-30T22:31:53.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.268+0000 7f829b18d700 1 -- 172.21.15.16:0/2546253879 shutdown_connections 2023-09-30T22:31:53.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.268+0000 7f829b18d700 1 --2- 172.21.15.16:0/2546253879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8294134260 0x7f829409cf00 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:53.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.268+0000 7f829b18d700 1 --2- 172.21.15.16:0/2546253879 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8294133420 0x7f829409c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:53.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.268+0000 7f829b18d700 1 --2- 172.21.15.16:0/2546253879 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8294133e30 0x7f829409c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:53.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.268+0000 7f829b18d700 1 -- 172.21.15.16:0/2546253879 >> 172.21.15.16:0/2546253879 conn(0x7f82940a4210 msgr2=0x7f82940a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:53.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.268+0000 7f829b18d700 1 -- 172.21.15.16:0/2546253879 shutdown_connections 2023-09-30T22:31:53.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.268+0000 7f829b18d700 1 -- 172.21.15.16:0/2546253879 wait complete. 2023-09-30T22:31:53.288 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:31:53.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.405+0000 7fe051a1c700 1 -- 172.21.15.16:0/927255694 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe04c074ea0 con 0x7fe04c09e000 2023-09-30T22:31:53.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.406+0000 7fe033fff700 1 -- 172.21.15.16:0/927255694 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe04002b070 con 0x7fe04c09e000 2023-09-30T22:31:53.406 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:31:53.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.408+0000 7f4d39f16700 1 -- 172.21.15.16:0/3830696269 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4d34074ea0 con 0x7f4d3412cf60 2023-09-30T22:31:53.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.408+0000 7f4d1bfff700 1 -- 172.21.15.16:0/3830696269 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4d24006cc0 con 0x7f4d3412cf60 2023-09-30T22:31:53.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.408+0000 7fe051a1c700 1 -- 172.21.15.16:0/927255694 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe04c09e000 msgr2=0x7fe04c1a6510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:53.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.408+0000 7fe051a1c700 1 --2- 172.21.15.16:0/927255694 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe04c09e000 0x7fe04c1a6510 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe040009fd0 tx=0x7fe040004750 comp rx=0 tx=0).stop 2023-09-30T22:31:53.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.408+0000 7fe051a1c700 1 -- 172.21.15.16:0/927255694 shutdown_connections 2023-09-30T22:31:53.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.408+0000 7fe051a1c700 1 --2- 172.21.15.16:0/927255694 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe04c09e410 0x7fe04c1a6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:53.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.408+0000 7fe051a1c700 1 --2- 172.21.15.16:0/927255694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe04c09d6f0 0x7fe04c1a5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:53.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.408+0000 7fe051a1c700 1 --2- 172.21.15.16:0/927255694 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe04c09e000 0x7fe04c1a6510 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:53.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.408+0000 7fe051a1c700 1 -- 172.21.15.16:0/927255694 >> 172.21.15.16:0/927255694 conn(0x7fe04c066f50 msgr2=0x7fe04c1bad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:53.410 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:31:53.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.408+0000 7fe051a1c700 1 -- 172.21.15.16:0/927255694 shutdown_connections 2023-09-30T22:31:53.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.408+0000 7fe051a1c700 1 -- 172.21.15.16:0/927255694 wait complete. 2023-09-30T22:31:53.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.411+0000 7f4d39f16700 1 -- 172.21.15.16:0/3830696269 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4d3412cf60 msgr2=0x7f4d34139d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:53.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.411+0000 7f4d39f16700 1 --2- 172.21.15.16:0/3830696269 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4d3412cf60 0x7f4d34139d20 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4d24017000 tx=0x7f4d240047c0 comp rx=0 tx=0).stop 2023-09-30T22:31:53.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.411+0000 7f4d39f16700 1 -- 172.21.15.16:0/3830696269 shutdown_connections 2023-09-30T22:31:53.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.411+0000 7f4d39f16700 1 --2- 172.21.15.16:0/3830696269 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d34130260 0x7f4d34138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:53.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.411+0000 7f4d39f16700 1 --2- 172.21.15.16:0/3830696269 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4d3412cf60 0x7f4d34139d20 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:53.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.411+0000 7f4d39f16700 1 --2- 172.21.15.16:0/3830696269 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4d3412f8b0 0x7f4d34138340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:53.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.411+0000 7f4d39f16700 1 -- 172.21.15.16:0/3830696269 >> 172.21.15.16:0/3830696269 conn(0x7f4d340a2aa0 msgr2=0x7f4d3406e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:53.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.411+0000 7f4d39f16700 1 -- 172.21.15.16:0/3830696269 shutdown_connections 2023-09-30T22:31:53.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.411+0000 7f4d39f16700 1 -- 172.21.15.16:0/3830696269 wait complete. 2023-09-30T22:31:53.424 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:31:53.426 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:31:53.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.431+0000 7f8ce44d8700 1 -- 172.21.15.16:0/3037053399 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8cdc09c710 con 0x7f8cdc09e000 2023-09-30T22:31:53.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.431+0000 7f8cd2ffd700 1 -- 172.21.15.16:0/3037053399 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8ccc015070 con 0x7f8cdc09e000 2023-09-30T22:31:53.432 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:31:53.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.433+0000 7f8ce44d8700 1 -- 172.21.15.16:0/3037053399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cdc09e000 msgr2=0x7f8cdc1a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:53.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.433+0000 7f8ce44d8700 1 --2- 172.21.15.16:0/3037053399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cdc09e000 0x7f8cdc1a62a0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc009c70 tx=0x7f8ccc004750 comp rx=0 tx=0).stop 2023-09-30T22:31:53.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.433+0000 7f8ce44d8700 1 -- 172.21.15.16:0/3037053399 shutdown_connections 2023-09-30T22:31:53.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.434+0000 7f8ce44d8700 1 --2- 172.21.15.16:0/3037053399 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8cdc09d6f0 0x7f8cdc1a5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:53.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.434+0000 7f8ce44d8700 1 --2- 172.21.15.16:0/3037053399 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8cdc09e410 0x7f8cdc1a67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:53.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.434+0000 7f8ce44d8700 1 --2- 172.21.15.16:0/3037053399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cdc09e000 0x7f8cdc1a62a0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:53.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.434+0000 7f8ce44d8700 1 -- 172.21.15.16:0/3037053399 >> 172.21.15.16:0/3037053399 conn(0x7f8cdc066f50 msgr2=0x7f8cdc1bad40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:53.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.434+0000 7f8ce44d8700 1 -- 172.21.15.16:0/3037053399 shutdown_connections 2023-09-30T22:31:53.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:53.434+0000 7f8ce44d8700 1 -- 172.21.15.16:0/3037053399 wait complete. 2023-09-30T22:31:53.447 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:31:54.289 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:54.424 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:54.427 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:54.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.435+0000 7f6a57e36700 1 Processor -- start 2023-09-30T22:31:54.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.435+0000 7f6a57e36700 1 -- start start 2023-09-30T22:31:54.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.436+0000 7f6a57e36700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6a50133540 0x7f6a50135920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:54.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.436+0000 7f6a57e36700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6a50135e60 0x7f6a50138240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:54.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.436+0000 7f6a57e36700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a500716a0 con 0x7f6a50130540 2023-09-30T22:31:54.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.436+0000 7f6a57e36700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a50071290 con 0x7f6a50135e60 2023-09-30T22:31:54.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.436+0000 7f6a57e36700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6a50130340 con 0x7f6a50133540 2023-09-30T22:31:54.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.436+0000 7f6a553d1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6a50133540 0x7f6a50135920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:54.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.436+0000 7f6a563d3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6a50135e60 0x7f6a50138240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:54.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.436+0000 7f6a55bd2700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6a50130540 0x7f6a50130910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50334/0 (socket says 172.21.15.16:50334) 2023-09-30T22:31:54.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.436+0000 7f6a553d1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6a50133540 0x7f6a50135920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:60344/0 (socket says 172.21.15.16:60344) 2023-09-30T22:31:54.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.436+0000 7f6a563d3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6a50135e60 0x7f6a50138240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:33296/0 (socket says 172.21.15.16:33296) 2023-09-30T22:31:54.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.436+0000 7f6a55bd2700 1 -- 172.21.15.16:0/2604959385 learned_addr learned my addr 172.21.15.16:0/2604959385 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:31:54.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.437+0000 7f6a553d1700 1 -- 172.21.15.16:0/2604959385 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6a50135e60 msgr2=0x7f6a50138240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:54.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.437+0000 7f6a553d1700 1 --2- 172.21.15.16:0/2604959385 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6a50135e60 0x7f6a50138240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:54.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.437+0000 7f6a553d1700 1 -- 172.21.15.16:0/2604959385 >> v1:172.21.15.16:6789/0 conn(0x7f6a50130540 legacy=0x7f6a50130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:54.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.437+0000 7f6a553d1700 1 -- 172.21.15.16:0/2604959385 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a50070b80 con 0x7f6a50133540 2023-09-30T22:31:54.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.437+0000 7f6a54bd0700 1 -- 172.21.15.16:0/2604959385 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2641156091 0 0) 0x7f6a500716a0 con 0x7f6a50130540 2023-09-30T22:31:54.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.437+0000 7f6a553d1700 1 --2- 172.21.15.16:0/2604959385 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6a50133540 0x7f6a50135920 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6a40009870 tx=0x7f6a40009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2408ddb1f402d3b7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:54.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.437+0000 7f6a54bd0700 1 -- 172.21.15.16:0/2604959385 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4000f8a0 con 0x7f6a50133540 2023-09-30T22:31:54.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.437+0000 7f6a54bd0700 1 -- 172.21.15.16:0/2604959385 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a4000fa00 con 0x7f6a50133540 2023-09-30T22:31:54.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.437+0000 7f6a54bd0700 1 -- 172.21.15.16:0/2604959385 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4000fd90 con 0x7f6a50133540 2023-09-30T22:31:54.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.438+0000 7f6a57e36700 1 -- 172.21.15.16:0/2604959385 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6a50133540 msgr2=0x7f6a50135920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:54.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.438+0000 7f6a57e36700 1 --2- 172.21.15.16:0/2604959385 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6a50133540 0x7f6a50135920 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6a40009870 tx=0x7f6a40009b80 comp rx=0 tx=0).stop 2023-09-30T22:31:54.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.438+0000 7f6a57e36700 1 -- 172.21.15.16:0/2604959385 shutdown_connections 2023-09-30T22:31:54.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.438+0000 7f6a57e36700 1 --2- 172.21.15.16:0/2604959385 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6a50135e60 0x7f6a50138240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:54.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.438+0000 7f6a57e36700 1 --2- 172.21.15.16:0/2604959385 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6a50133540 0x7f6a50135920 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:54.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.438+0000 7f6a57e36700 1 -- 172.21.15.16:0/2604959385 >> 172.21.15.16:0/2604959385 conn(0x7f6a500a2aa0 msgr2=0x7f6a500a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:54.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.438+0000 7f6a57e36700 1 -- 172.21.15.16:0/2604959385 shutdown_connections 2023-09-30T22:31:54.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.438+0000 7f6a57e36700 1 -- 172.21.15.16:0/2604959385 wait complete. 2023-09-30T22:31:54.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.439+0000 7f6a57e36700 1 Processor -- start 2023-09-30T22:31:54.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.439+0000 7f6a57e36700 1 -- start start 2023-09-30T22:31:54.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.440+0000 7f6a57e36700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a50130540 0x7f6a501c83b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:54.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.440+0000 7f6a57e36700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6a50133540 0x7f6a501c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:54.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.440+0000 7f6a57e36700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6a50135e60 0x7f6a5009c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:54.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.440+0000 7f6a57e36700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a500718e0 con 0x7f6a50130540 2023-09-30T22:31:54.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.440+0000 7f6a57e36700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5012ddc0 con 0x7f6a50135e60 2023-09-30T22:31:54.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.440+0000 7f6a57e36700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5012da10 con 0x7f6a50133540 2023-09-30T22:31:54.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.440+0000 7f6a563d3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6a50135e60 0x7f6a5009c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:54.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.440+0000 7f6a55bd2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a50130540 0x7f6a501c83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:54.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.440+0000 7f6a563d3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6a50135e60 0x7f6a5009c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:33304/0 (socket says 172.21.15.16:33304) 2023-09-30T22:31:54.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.440+0000 7f6a563d3700 1 -- 172.21.15.16:0/2402311262 learned_addr learned my addr 172.21.15.16:0/2402311262 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:54.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.440+0000 7f6a553d1700 1 --2- 172.21.15.16:0/2402311262 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6a50133540 0x7f6a501c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:54.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.440+0000 7f6a55bd2700 1 -- 172.21.15.16:0/2402311262 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6a50133540 msgr2=0x7f6a501c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:54.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.440+0000 7f6a55bd2700 1 --2- 172.21.15.16:0/2402311262 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6a50133540 0x7f6a501c88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:54.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.440+0000 7f6a55bd2700 1 -- 172.21.15.16:0/2402311262 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6a50135e60 msgr2=0x7f6a5009c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:54.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.440+0000 7f6a55bd2700 1 --2- 172.21.15.16:0/2402311262 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6a50135e60 0x7f6a5009c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:54.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.440+0000 7f6a55bd2700 1 -- 172.21.15.16:0/2402311262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a400094a0 con 0x7f6a50130540 2023-09-30T22:31:54.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.440+0000 7f6a55bd2700 1 --2- 172.21.15.16:0/2402311262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a50130540 0x7f6a501c83b0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c00c4a0 tx=0x7f6a4c00c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:54.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.441+0000 7f6a467fc700 1 -- 172.21.15.16:0/2402311262 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c004c80 con 0x7f6a50130540 2023-09-30T22:31:54.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.441+0000 7f6a467fc700 1 -- 172.21.15.16:0/2402311262 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a4c004de0 con 0x7f6a50130540 2023-09-30T22:31:54.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.441+0000 7f6a57e36700 1 -- 172.21.15.16:0/2402311262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a5009e0d0 con 0x7f6a50130540 2023-09-30T22:31:54.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.441+0000 7f6a467fc700 1 -- 172.21.15.16:0/2402311262 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c007310 con 0x7f6a50130540 2023-09-30T22:31:54.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.441+0000 7f6a57e36700 1 -- 172.21.15.16:0/2402311262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a5011e690 con 0x7f6a50130540 2023-09-30T22:31:54.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.442+0000 7f6a57e36700 1 -- 172.21.15.16:0/2402311262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a50074ea0 con 0x7f6a50130540 2023-09-30T22:31:54.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.442+0000 7f6a467fc700 1 -- 172.21.15.16:0/2402311262 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6a4c007470 con 0x7f6a50130540 2023-09-30T22:31:54.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.442+0000 7f6a467fc700 1 -- 172.21.15.16:0/2402311262 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f6a4c03f240 con 0x7f6a50130540 2023-09-30T22:31:54.448 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:54.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.448+0000 7f6a467fc700 1 -- 172.21.15.16:0/2402311262 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6a4c007720 con 0x7f6a50130540 2023-09-30T22:31:54.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.570+0000 7f624840c700 1 Processor -- start 2023-09-30T22:31:54.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.570+0000 7f624840c700 1 -- start start 2023-09-30T22:31:54.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.570+0000 7f4cf9647700 1 Processor -- start 2023-09-30T22:31:54.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.571+0000 7f4cf9647700 1 -- start start 2023-09-30T22:31:54.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.571+0000 7f624840c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f624009d6f0 0x7f624009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:54.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.571+0000 7f624840c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f624009e410 0x7f6240134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:54.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.571+0000 7f624840c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f624009b120 con 0x7f624009e000 2023-09-30T22:31:54.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.571+0000 7f624840c700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f624006b460 con 0x7f624009e410 2023-09-30T22:31:54.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.571+0000 7f624840c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f624009a7e0 con 0x7f624009d6f0 2023-09-30T22:31:54.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.571+0000 7f4cf9647700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4cf41338a0 0x7f4cf4133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:54.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.571+0000 7f4cf9647700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4cf409de50 0x7f4cf409e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:54.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.571+0000 7f4cf9647700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cf406ef30 con 0x7f4cf409e760 2023-09-30T22:31:54.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.571+0000 7f4cf9647700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4cf406fef0 con 0x7f4cf41338a0 2023-09-30T22:31:54.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.571+0000 7f4cf9647700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf41334b0 con 0x7f4cf409de50 2023-09-30T22:31:54.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.571+0000 7f62461a8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f624009d6f0 0x7f624009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:54.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.571+0000 7f62461a8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f624009d6f0 0x7f624009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:60354/0 (socket says 172.21.15.16:60354) 2023-09-30T22:31:54.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.571+0000 7f62461a8700 1 -- 172.21.15.16:0/1970260213 learned_addr learned my addr 172.21.15.16:0/1970260213 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:54.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.572+0000 7f4cf37fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4cf409e760 0x7f4cf413a730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50352/0 (socket says 172.21.15.16:50352) 2023-09-30T22:31:54.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.572+0000 7f4cf37fe700 1 -- 172.21.15.16:0/466074790 learned_addr learned my addr 172.21.15.16:0/466074790 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:31:54.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.572+0000 7f62469a9700 1 --2- 172.21.15.16:0/1970260213 >> v2:172.21.15.16:3300/0 conn(0x7f624009e410 0x7f6240134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:54.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.572+0000 7f4cf27fc700 1 --2- 172.21.15.16:0/466074790 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4cf409de50 0x7f4cf409e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:54.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.572+0000 7f62461a8700 1 -- 172.21.15.16:0/1970260213 >> v2:172.21.15.16:3300/0 conn(0x7f624009e410 msgr2=0x7f6240134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:54.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.572+0000 7f62461a8700 1 --2- 172.21.15.16:0/1970260213 >> v2:172.21.15.16:3300/0 conn(0x7f624009e410 0x7f6240134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:54.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.572+0000 7f62461a8700 1 -- 172.21.15.16:0/1970260213 >> v1:172.21.15.16:6789/0 conn(0x7f624009e000 legacy=0x7f62401339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:54.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.572+0000 7f62461a8700 1 -- 172.21.15.16:0/1970260213 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f624009a500 con 0x7f624009d6f0 2023-09-30T22:31:54.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.572+0000 7f4cf27fc700 1 -- 172.21.15.16:0/466074790 >> v2:172.21.15.16:3300/0 conn(0x7f4cf41338a0 msgr2=0x7f4cf4133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:54.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.572+0000 7f4cf27fc700 1 --2- 172.21.15.16:0/466074790 >> v2:172.21.15.16:3300/0 conn(0x7f4cf41338a0 0x7f4cf4133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:54.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.572+0000 7f4cf27fc700 1 -- 172.21.15.16:0/466074790 >> v1:172.21.15.16:6789/0 conn(0x7f4cf409e760 legacy=0x7f4cf413a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:54.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.572+0000 7f4cf27fc700 1 -- 172.21.15.16:0/466074790 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cf406fc30 con 0x7f4cf409de50 2023-09-30T22:31:54.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.572+0000 7f62461a8700 1 --2- 172.21.15.16:0/1970260213 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f624009d6f0 0x7f624009dac0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f623c009920 tx=0x7f623c009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d98d0e821136aa9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:54.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.572+0000 7f4cf27fc700 1 --2- 172.21.15.16:0/466074790 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4cf409de50 0x7f4cf409e220 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8009840 tx=0x7f4ce8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f72e14be0806c79 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:54.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.572+0000 7f62451a6700 1 -- 172.21.15.16:0/1970260213 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623c00ba20 con 0x7f624009d6f0 2023-09-30T22:31:54.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.572+0000 7f62451a6700 1 -- 172.21.15.16:0/1970260213 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f623c00bb80 con 0x7f624009d6f0 2023-09-30T22:31:54.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.573+0000 7f62451a6700 1 -- 172.21.15.16:0/1970260213 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623c009e70 con 0x7f624009d6f0 2023-09-30T22:31:54.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.573+0000 7f4cf1ffb700 1 -- 172.21.15.16:0/466074790 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce8004230 con 0x7f4cf409de50 2023-09-30T22:31:54.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.573+0000 7f4cf1ffb700 1 -- 172.21.15.16:0/466074790 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ce8004390 con 0x7f4cf409de50 2023-09-30T22:31:54.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.573+0000 7f4cf1ffb700 1 -- 172.21.15.16:0/466074790 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce80036a0 con 0x7f4cf409de50 2023-09-30T22:31:54.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.573+0000 7f624840c700 1 -- 172.21.15.16:0/1970260213 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f624009d6f0 msgr2=0x7f624009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:54.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.573+0000 7f624840c700 1 --2- 172.21.15.16:0/1970260213 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f624009d6f0 0x7f624009dac0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f623c009920 tx=0x7f623c009c30 comp rx=0 tx=0).stop 2023-09-30T22:31:54.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.573+0000 7f624840c700 1 -- 172.21.15.16:0/1970260213 shutdown_connections 2023-09-30T22:31:54.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.573+0000 7f624840c700 1 --2- 172.21.15.16:0/1970260213 >> v2:172.21.15.16:3300/0 conn(0x7f624009e410 0x7f6240134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:54.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.573+0000 7f624840c700 1 --2- 172.21.15.16:0/1970260213 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f624009d6f0 0x7f624009dac0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:54.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.573+0000 7f624840c700 1 -- 172.21.15.16:0/1970260213 >> 172.21.15.16:0/1970260213 conn(0x7f6240066f50 msgr2=0x7f624009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:54.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.573+0000 7f624840c700 1 -- 172.21.15.16:0/1970260213 shutdown_connections 2023-09-30T22:31:54.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.574+0000 7f624840c700 1 -- 172.21.15.16:0/1970260213 wait complete. 2023-09-30T22:31:54.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.574+0000 7f4cf9647700 1 -- 172.21.15.16:0/466074790 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4cf409de50 msgr2=0x7f4cf409e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:54.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.574+0000 7f4cf9647700 1 --2- 172.21.15.16:0/466074790 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4cf409de50 0x7f4cf409e220 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8009840 tx=0x7f4ce8009b50 comp rx=0 tx=0).stop 2023-09-30T22:31:54.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.574+0000 7f4cf9647700 1 -- 172.21.15.16:0/466074790 shutdown_connections 2023-09-30T22:31:54.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.574+0000 7f4cf9647700 1 --2- 172.21.15.16:0/466074790 >> v2:172.21.15.16:3300/0 conn(0x7f4cf41338a0 0x7f4cf4133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:54.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.574+0000 7f4cf9647700 1 --2- 172.21.15.16:0/466074790 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4cf409de50 0x7f4cf409e220 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:54.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.574+0000 7f4cf9647700 1 -- 172.21.15.16:0/466074790 >> 172.21.15.16:0/466074790 conn(0x7f4cf40a2aa0 msgr2=0x7f4cf40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:54.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.574+0000 7f4cf9647700 1 -- 172.21.15.16:0/466074790 shutdown_connections 2023-09-30T22:31:54.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.574+0000 7f4cf9647700 1 -- 172.21.15.16:0/466074790 wait complete. 2023-09-30T22:31:54.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.575+0000 7f624840c700 1 Processor -- start 2023-09-30T22:31:54.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.575+0000 7f624840c700 1 -- start start 2023-09-30T22:31:54.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.576+0000 7f4cf9647700 1 Processor -- start 2023-09-30T22:31:54.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.576+0000 7f4cf9647700 1 -- start start 2023-09-30T22:31:54.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.576+0000 7f624840c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f624009d6f0 0x7f62401a5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:54.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.576+0000 7f624840c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f624009e000 0x7f62401a63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:54.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.576+0000 7f624840c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f624009e410 0x7f62401a6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:54.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.576+0000 7f624840c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62401a9920 con 0x7f624009e410 2023-09-30T22:31:54.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.576+0000 7f624840c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62401a7ab0 con 0x7f624009d6f0 2023-09-30T22:31:54.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.576+0000 7f624840c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f62400a32e0 con 0x7f624009e000 2023-09-30T22:31:54.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.576+0000 7f62469a9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f624009e410 0x7f62401a6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:54.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.576+0000 7f62469a9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f624009e410 0x7f62401a6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54542/0 (socket says 172.21.15.16:54542) 2023-09-30T22:31:54.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.576+0000 7f62469a9700 1 -- 172.21.15.16:0/1064865363 learned_addr learned my addr 172.21.15.16:0/1064865363 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:54.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.576+0000 7f62459a7700 1 --2- 172.21.15.16:0/1064865363 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f624009e000 0x7f62401a63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:54.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.576+0000 7f62461a8700 1 --2- 172.21.15.16:0/1064865363 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f624009d6f0 0x7f62401a5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:54.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.576+0000 7f62469a9700 1 -- 172.21.15.16:0/1064865363 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f624009e000 msgr2=0x7f62401a63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:54.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.577+0000 7f62469a9700 1 --2- 172.21.15.16:0/1064865363 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f624009e000 0x7f62401a63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:54.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.577+0000 7f62469a9700 1 -- 172.21.15.16:0/1064865363 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f624009d6f0 msgr2=0x7f62401a5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:54.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.577+0000 7f62469a9700 1 --2- 172.21.15.16:0/1064865363 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f624009d6f0 0x7f62401a5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:54.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.577+0000 7f62469a9700 1 -- 172.21.15.16:0/1064865363 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f623c009550 con 0x7f624009e410 2023-09-30T22:31:54.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.577+0000 7f62461a8700 1 --2- 172.21.15.16:0/1064865363 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f624009d6f0 0x7f62401a5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:31:54.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.576+0000 7f4cf9647700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4cf409de50 0x7f4cf411ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:54.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.576+0000 7f4cf9647700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4cf409e760 0x7f4cf411f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:54.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.576+0000 7f4cf9647700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4cf41338a0 0x7f4cf411f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:54.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.576+0000 7f4cf9647700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf412f820 con 0x7f4cf409e760 2023-09-30T22:31:54.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.576+0000 7f4cf9647700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf41201b0 con 0x7f4cf409de50 2023-09-30T22:31:54.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.576+0000 7f4cf9647700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf4122540 con 0x7f4cf41338a0 2023-09-30T22:31:54.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.577+0000 7f4cf27fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4cf409e760 0x7f4cf411f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:54.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.577+0000 7f4cf27fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4cf409e760 0x7f4cf411f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54556/0 (socket says 172.21.15.16:54556) 2023-09-30T22:31:54.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.577+0000 7f4cf27fc700 1 -- 172.21.15.16:0/711462452 learned_addr learned my addr 172.21.15.16:0/711462452 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:54.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.577+0000 7f4cf37fe700 1 --2- 172.21.15.16:0/711462452 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4cf41338a0 0x7f4cf411f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:54.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.577+0000 7f4cf2ffd700 1 --2- 172.21.15.16:0/711462452 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4cf409de50 0x7f4cf411ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:54.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.577+0000 7f4cf2ffd700 1 -- 172.21.15.16:0/711462452 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4cf41338a0 msgr2=0x7f4cf411f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:54.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.577+0000 7f4cf2ffd700 1 --2- 172.21.15.16:0/711462452 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4cf41338a0 0x7f4cf411f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:54.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.577+0000 7f4cf2ffd700 1 -- 172.21.15.16:0/711462452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4cf409e760 msgr2=0x7f4cf411f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:54.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.577+0000 7f4cf2ffd700 1 --2- 172.21.15.16:0/711462452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4cf409e760 0x7f4cf411f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:54.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.577+0000 7f4cf2ffd700 1 -- 172.21.15.16:0/711462452 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce80094a0 con 0x7f4cf409de50 2023-09-30T22:31:54.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.577+0000 7f4cf37fe700 1 --2- 172.21.15.16:0/711462452 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4cf41338a0 0x7f4cf411f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:54.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.577+0000 7f4cf2ffd700 1 --2- 172.21.15.16:0/711462452 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4cf409de50 0x7f4cf411ed00 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4ce400b480 tx=0x7f4ce400b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:54.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.578+0000 7f4cdb7fe700 1 -- 172.21.15.16:0/711462452 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce4011830 con 0x7f4cf409de50 2023-09-30T22:31:54.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.578+0000 7f62469a9700 1 --2- 172.21.15.16:0/1064865363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f624009e410 0x7f62401a6910 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f6238007e40 tx=0x7f623800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:54.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.578+0000 7f4cdb7fe700 1 -- 172.21.15.16:0/711462452 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ce4011990 con 0x7f4cf409de50 2023-09-30T22:31:54.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.578+0000 7f4cf9647700 1 -- 172.21.15.16:0/711462452 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cf4120a20 con 0x7f4cf409de50 2023-09-30T22:31:54.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.578+0000 7f4cf9647700 1 -- 172.21.15.16:0/711462452 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cf4121100 con 0x7f4cf409de50 2023-09-30T22:31:54.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.579+0000 7f4cdb7fe700 1 -- 172.21.15.16:0/711462452 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce4011d40 con 0x7f4cf409de50 2023-09-30T22:31:54.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.579+0000 7f4cdb7fe700 1 -- 172.21.15.16:0/711462452 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4ce401a400 con 0x7f4cf409de50 2023-09-30T22:31:54.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.579+0000 7f6236ffd700 1 -- 172.21.15.16:0/1064865363 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623800eea0 con 0x7f624009e410 2023-09-30T22:31:54.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.579+0000 7f6236ffd700 1 -- 172.21.15.16:0/1064865363 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6238014420 con 0x7f624009e410 2023-09-30T22:31:54.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.579+0000 7f6236ffd700 1 -- 172.21.15.16:0/1064865363 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62380147b0 con 0x7f624009e410 2023-09-30T22:31:54.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.579+0000 7f624840c700 1 -- 172.21.15.16:0/1064865363 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f624009c710 con 0x7f624009e410 2023-09-30T22:31:54.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.579+0000 7f4cdb7fe700 1 -- 172.21.15.16:0/711462452 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f4ce403f5e0 con 0x7f4cf409de50 2023-09-30T22:31:54.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.580+0000 7f4cf9647700 1 -- 172.21.15.16:0/711462452 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cd40052e0 con 0x7f4cf409de50 2023-09-30T22:31:54.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.579+0000 7f624840c700 1 -- 172.21.15.16:0/1064865363 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62401a8810 con 0x7f624009e410 2023-09-30T22:31:54.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.580+0000 7f624840c700 1 -- 172.21.15.16:0/1064865363 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6240074ea0 con 0x7f624009e410 2023-09-30T22:31:54.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.580+0000 7f6236ffd700 1 -- 172.21.15.16:0/1064865363 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6238014910 con 0x7f624009e410 2023-09-30T22:31:54.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.581+0000 7f6236ffd700 1 -- 172.21.15.16:0/1064865363 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f6238040d20 con 0x7f624009e410 2023-09-30T22:31:54.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.585+0000 7f6236ffd700 1 -- 172.21.15.16:0/1064865363 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6238013d50 con 0x7f624009e410 2023-09-30T22:31:54.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.589+0000 7f4cdb7fe700 1 -- 172.21.15.16:0/711462452 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4ce400f6f0 con 0x7f4cf409de50 2023-09-30T22:31:54.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.595+0000 7f593b831700 1 Processor -- start 2023-09-30T22:31:54.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.595+0000 7f593b831700 1 -- start start 2023-09-30T22:31:54.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.596+0000 7f593b831700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f593409d6f0 0x7f593409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:54.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.596+0000 7f593b831700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f593409e410 0x7f5934130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:54.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.596+0000 7f593b831700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f593409f130 con 0x7f593409e000 2023-09-30T22:31:54.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.596+0000 7f593b831700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f593409f520 con 0x7f593409e410 2023-09-30T22:31:54.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.596+0000 7f593b831700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f593409e930 con 0x7f593409d6f0 2023-09-30T22:31:54.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.596+0000 7f5938dcc700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f593409e000 0x7f5934130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50366/0 (socket says 172.21.15.16:50366) 2023-09-30T22:31:54.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.596+0000 7f59395cd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f593409d6f0 0x7f593409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:54.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.596+0000 7f5938dcc700 1 -- 172.21.15.16:0/2158102571 learned_addr learned my addr 172.21.15.16:0/2158102571 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:31:54.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.596+0000 7f5939dce700 1 --2- 172.21.15.16:0/2158102571 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f593409e410 0x7f5934130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:54.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.597+0000 7f5939dce700 1 -- 172.21.15.16:0/2158102571 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f593409d6f0 msgr2=0x7f593409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:54.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.597+0000 7f592bfff700 1 -- 172.21.15.16:0/2158102571 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2395089850 0 0) 0x7f593409f130 con 0x7f593409e000 2023-09-30T22:31:54.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.597+0000 7f5939dce700 1 --2- 172.21.15.16:0/2158102571 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f593409d6f0 0x7f593409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:54.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.597+0000 7f5939dce700 1 -- 172.21.15.16:0/2158102571 >> v1:172.21.15.16:6789/0 conn(0x7f593409e000 legacy=0x7f5934130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:54.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.597+0000 7f5939dce700 1 -- 172.21.15.16:0/2158102571 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f593409b2d0 con 0x7f593409e410 2023-09-30T22:31:54.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.597+0000 7f59395cd700 1 --2- 172.21.15.16:0/2158102571 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f593409d6f0 0x7f593409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:54.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.597+0000 7f5939dce700 1 --2- 172.21.15.16:0/2158102571 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f593409e410 0x7f5934130d70 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f592c00f3a0 tx=0x7f592c00f6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f52f05c5efb2ecc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:54.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.597+0000 7f592bfff700 1 -- 172.21.15.16:0/2158102571 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f592c012070 con 0x7f593409e410 2023-09-30T22:31:54.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.597+0000 7f592bfff700 1 -- 172.21.15.16:0/2158102571 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f592c00d0d0 con 0x7f593409e410 2023-09-30T22:31:54.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.597+0000 7f592bfff700 1 -- 172.21.15.16:0/2158102571 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f592c00fea0 con 0x7f593409e410 2023-09-30T22:31:54.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.598+0000 7f593b831700 1 -- 172.21.15.16:0/2158102571 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f593409e410 msgr2=0x7f5934130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:54.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.598+0000 7f593b831700 1 --2- 172.21.15.16:0/2158102571 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f593409e410 0x7f5934130d70 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f592c00f3a0 tx=0x7f592c00f6b0 comp rx=0 tx=0).stop 2023-09-30T22:31:54.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.598+0000 7f593b831700 1 -- 172.21.15.16:0/2158102571 shutdown_connections 2023-09-30T22:31:54.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.598+0000 7f593b831700 1 --2- 172.21.15.16:0/2158102571 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f593409e410 0x7f5934130d70 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:54.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.598+0000 7f593b831700 1 --2- 172.21.15.16:0/2158102571 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f593409d6f0 0x7f593409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:54.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.598+0000 7f593b831700 1 -- 172.21.15.16:0/2158102571 >> 172.21.15.16:0/2158102571 conn(0x7f5934066f50 msgr2=0x7f593409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:54.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.599+0000 7f593b831700 1 -- 172.21.15.16:0/2158102571 shutdown_connections 2023-09-30T22:31:54.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.599+0000 7f593b831700 1 -- 172.21.15.16:0/2158102571 wait complete. 2023-09-30T22:31:54.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.600+0000 7f593b831700 1 Processor -- start 2023-09-30T22:31:54.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.600+0000 7f593b831700 1 -- start start 2023-09-30T22:31:54.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.600+0000 7f593b831700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f593409d6f0 0x7f593411ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:54.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.600+0000 7f593b831700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f593409e000 0x7f593411e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:54.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.600+0000 7f593b831700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f593409e410 0x7f593411e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:54.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.600+0000 7f593b831700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5934121600 con 0x7f593409e000 2023-09-30T22:31:54.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.600+0000 7f593b831700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f593411fae0 con 0x7f593409d6f0 2023-09-30T22:31:54.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.600+0000 7f593b831700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5934072090 con 0x7f593409e410 2023-09-30T22:31:54.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.600+0000 7f5938dcc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f593409e000 0x7f593411e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:54.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.600+0000 7f5938dcc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f593409e000 0x7f593411e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54558/0 (socket says 172.21.15.16:54558) 2023-09-30T22:31:54.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.600+0000 7f5938dcc700 1 -- 172.21.15.16:0/3169895059 learned_addr learned my addr 172.21.15.16:0/3169895059 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:54.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.600+0000 7f5939dce700 1 --2- 172.21.15.16:0/3169895059 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f593409e410 0x7f593411e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:54.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.600+0000 7f59395cd700 1 --2- 172.21.15.16:0/3169895059 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f593409d6f0 0x7f593411ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:54.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.601+0000 7f5938dcc700 1 -- 172.21.15.16:0/3169895059 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f593409e410 msgr2=0x7f593411e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:54.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.601+0000 7f5938dcc700 1 --2- 172.21.15.16:0/3169895059 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f593409e410 0x7f593411e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:54.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.601+0000 7f5938dcc700 1 -- 172.21.15.16:0/3169895059 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f593409d6f0 msgr2=0x7f593411ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:54.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.601+0000 7f5938dcc700 1 --2- 172.21.15.16:0/3169895059 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f593409d6f0 0x7f593411ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:54.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.601+0000 7f5938dcc700 1 -- 172.21.15.16:0/3169895059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f593000b550 con 0x7f593409e000 2023-09-30T22:31:54.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.601+0000 7f5938dcc700 1 --2- 172.21.15.16:0/3169895059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f593409e000 0x7f593411e300 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f5924001eb0 tx=0x7f592400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:54.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.601+0000 7f5929ffb700 1 -- 172.21.15.16:0/3169895059 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5924004c80 con 0x7f593409e000 2023-09-30T22:31:54.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.601+0000 7f5929ffb700 1 -- 172.21.15.16:0/3169895059 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5924004de0 con 0x7f593409e000 2023-09-30T22:31:54.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.601+0000 7f593b831700 1 -- 172.21.15.16:0/3169895059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f592c00f050 con 0x7f593409e000 2023-09-30T22:31:54.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.601+0000 7f5929ffb700 1 -- 172.21.15.16:0/3169895059 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59240072d0 con 0x7f593409e000 2023-09-30T22:31:54.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.602+0000 7f593b831700 1 -- 172.21.15.16:0/3169895059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5934120c20 con 0x7f593409e000 2023-09-30T22:31:54.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.602+0000 7f5929ffb700 1 -- 172.21.15.16:0/3169895059 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5924007430 con 0x7f593409e000 2023-09-30T22:31:54.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.603+0000 7f5929ffb700 1 -- 172.21.15.16:0/3169895059 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f5924040d50 con 0x7f593409e000 2023-09-30T22:31:54.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.603+0000 7f593b831700 1 -- 172.21.15.16:0/3169895059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f593411ac20 con 0x7f593409e000 2023-09-30T22:31:54.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.608+0000 7f5929ffb700 1 -- 172.21.15.16:0/3169895059 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5924044060 con 0x7f593409e000 2023-09-30T22:31:54.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.671+0000 7f6a57e36700 1 -- 172.21.15.16:0/2402311262 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6a5006b9e0 con 0x7f6a50130540 2023-09-30T22:31:54.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.672+0000 7f6a467fc700 1 -- 172.21.15.16:0/2402311262 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6a4c01b340 con 0x7f6a50130540 2023-09-30T22:31:54.672 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:31:54.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.674+0000 7f6a57e36700 1 -- 172.21.15.16:0/2402311262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a50130540 msgr2=0x7f6a501c83b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:54.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.674+0000 7f6a57e36700 1 --2- 172.21.15.16:0/2402311262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a50130540 0x7f6a501c83b0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c00c4a0 tx=0x7f6a4c00c7b0 comp rx=0 tx=0).stop 2023-09-30T22:31:54.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.674+0000 7f6a57e36700 1 -- 172.21.15.16:0/2402311262 shutdown_connections 2023-09-30T22:31:54.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.674+0000 7f6a57e36700 1 --2- 172.21.15.16:0/2402311262 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6a50135e60 0x7f6a5009c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:54.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.674+0000 7f6a57e36700 1 --2- 172.21.15.16:0/2402311262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6a50130540 0x7f6a501c83b0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:54.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.674+0000 7f6a57e36700 1 --2- 172.21.15.16:0/2402311262 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6a50133540 0x7f6a501c88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:54.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.674+0000 7f6a57e36700 1 -- 172.21.15.16:0/2402311262 >> 172.21.15.16:0/2402311262 conn(0x7f6a500a2aa0 msgr2=0x7f6a50131300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:54.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.675+0000 7f6a57e36700 1 -- 172.21.15.16:0/2402311262 shutdown_connections 2023-09-30T22:31:54.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.675+0000 7f6a57e36700 1 -- 172.21.15.16:0/2402311262 wait complete. 2023-09-30T22:31:54.689 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:31:54.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.795+0000 7f624840c700 1 -- 172.21.15.16:0/1064865363 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f624006c360 con 0x7f624009e410 2023-09-30T22:31:54.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.795+0000 7f6236ffd700 1 -- 172.21.15.16:0/1064865363 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6238013b70 con 0x7f624009e410 2023-09-30T22:31:54.796 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:31:54.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.798+0000 7f624840c700 1 -- 172.21.15.16:0/1064865363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f624009e410 msgr2=0x7f62401a6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:54.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.798+0000 7f624840c700 1 --2- 172.21.15.16:0/1064865363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f624009e410 0x7f62401a6910 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f6238007e40 tx=0x7f623800e3c0 comp rx=0 tx=0).stop 2023-09-30T22:31:54.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.798+0000 7f624840c700 1 -- 172.21.15.16:0/1064865363 shutdown_connections 2023-09-30T22:31:54.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.798+0000 7f624840c700 1 --2- 172.21.15.16:0/1064865363 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f624009d6f0 0x7f62401a5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:54.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.798+0000 7f624840c700 1 --2- 172.21.15.16:0/1064865363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f624009e410 0x7f62401a6910 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:54.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.798+0000 7f624840c700 1 --2- 172.21.15.16:0/1064865363 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f624009e000 0x7f62401a63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:54.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.798+0000 7f624840c700 1 -- 172.21.15.16:0/1064865363 >> 172.21.15.16:0/1064865363 conn(0x7f6240066f50 msgr2=0x7f62401badf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:54.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.798+0000 7f624840c700 1 -- 172.21.15.16:0/1064865363 shutdown_connections 2023-09-30T22:31:54.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.798+0000 7f624840c700 1 -- 172.21.15.16:0/1064865363 wait complete. 2023-09-30T22:31:54.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.803+0000 7f4cf9647700 1 -- 172.21.15.16:0/711462452 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f4cd4005150 con 0x7f4cf409de50 2023-09-30T22:31:54.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.803+0000 7f4cdb7fe700 1 -- 172.21.15.16:0/711462452 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4ce4022350 con 0x7f4cf409de50 2023-09-30T22:31:54.804 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:31:54.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.805+0000 7f4cf9647700 1 -- 172.21.15.16:0/711462452 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4cf409de50 msgr2=0x7f4cf411ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:54.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.805+0000 7f4cf9647700 1 --2- 172.21.15.16:0/711462452 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4cf409de50 0x7f4cf411ed00 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4ce400b480 tx=0x7f4ce400b790 comp rx=0 tx=0).stop 2023-09-30T22:31:54.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.805+0000 7f4cf9647700 1 -- 172.21.15.16:0/711462452 shutdown_connections 2023-09-30T22:31:54.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.805+0000 7f4cf9647700 1 --2- 172.21.15.16:0/711462452 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4cf409de50 0x7f4cf411ed00 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:54.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.805+0000 7f4cf9647700 1 --2- 172.21.15.16:0/711462452 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4cf41338a0 0x7f4cf411f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:54.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.805+0000 7f4cf9647700 1 --2- 172.21.15.16:0/711462452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4cf409e760 0x7f4cf411f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:54.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.805+0000 7f4cf9647700 1 -- 172.21.15.16:0/711462452 >> 172.21.15.16:0/711462452 conn(0x7f4cf40a2aa0 msgr2=0x7f4cf409ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:54.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.806+0000 7f4cf9647700 1 -- 172.21.15.16:0/711462452 shutdown_connections 2023-09-30T22:31:54.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.806+0000 7f4cf9647700 1 -- 172.21.15.16:0/711462452 wait complete. 2023-09-30T22:31:54.815 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:31:54.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.816+0000 7f593b831700 1 -- 172.21.15.16:0/3169895059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5934074ea0 con 0x7f593409e000 2023-09-30T22:31:54.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.817+0000 7f5929ffb700 1 -- 172.21.15.16:0/3169895059 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f59240076e0 con 0x7f593409e000 2023-09-30T22:31:54.817 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:31:54.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.819+0000 7f593b831700 1 -- 172.21.15.16:0/3169895059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f593409e000 msgr2=0x7f593411e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:54.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.819+0000 7f593b831700 1 --2- 172.21.15.16:0/3169895059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f593409e000 0x7f593411e300 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f5924001eb0 tx=0x7f592400c800 comp rx=0 tx=0).stop 2023-09-30T22:31:54.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.819+0000 7f593b831700 1 -- 172.21.15.16:0/3169895059 shutdown_connections 2023-09-30T22:31:54.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.819+0000 7f593b831700 1 --2- 172.21.15.16:0/3169895059 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f593409d6f0 0x7f593411ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:54.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.819+0000 7f593b831700 1 --2- 172.21.15.16:0/3169895059 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f593409e410 0x7f593411e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:54.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.819+0000 7f593b831700 1 --2- 172.21.15.16:0/3169895059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f593409e000 0x7f593411e300 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:54.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.819+0000 7f593b831700 1 -- 172.21.15.16:0/3169895059 >> 172.21.15.16:0/3169895059 conn(0x7f5934066f50 msgr2=0x7f5934130a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:54.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.819+0000 7f593b831700 1 -- 172.21.15.16:0/3169895059 shutdown_connections 2023-09-30T22:31:54.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:54.819+0000 7f593b831700 1 -- 172.21.15.16:0/3169895059 wait complete. 2023-09-30T22:31:54.823 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:31:54.833 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:31:55.690 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:55.816 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:55.824 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:55.834 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:55.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.836+0000 7fce41efd700 1 Processor -- start 2023-09-30T22:31:55.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.836+0000 7fce41efd700 1 -- start start 2023-09-30T22:31:55.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.836+0000 7fce41efd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fce3c06a770 0x7fce3c06ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:55.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.836+0000 7fce41efd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fce3c131d10 0x7fce3c139860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:55.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.836+0000 7fce41efd700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce3c09ece0 con 0x7fce3c06b080 2023-09-30T22:31:55.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.836+0000 7fce41efd700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce3c09e760 con 0x7fce3c06a770 2023-09-30T22:31:55.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.836+0000 7fce41efd700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c09e040 con 0x7fce3c131d10 2023-09-30T22:31:55.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.836+0000 7fce3bfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fce3c131d10 0x7fce3c139860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:55.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.836+0000 7fce3b7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fce3c06a770 0x7fce3c06ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:55.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.837+0000 7fce3affd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fce3c06b080 0x7fce3c131600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50380/0 (socket says 172.21.15.16:50380) 2023-09-30T22:31:55.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.837+0000 7fce3affd700 1 -- 172.21.15.16:0/1762716973 learned_addr learned my addr 172.21.15.16:0/1762716973 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:31:55.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.837+0000 7fce3b7fe700 1 -- 172.21.15.16:0/1762716973 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fce3c131d10 msgr2=0x7fce3c139860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:55.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.837+0000 7fce3b7fe700 1 --2- 172.21.15.16:0/1762716973 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fce3c131d10 0x7fce3c139860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:55.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.837+0000 7fce3b7fe700 1 -- 172.21.15.16:0/1762716973 >> v1:172.21.15.16:6789/0 conn(0x7fce3c06b080 legacy=0x7fce3c131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:55.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.837+0000 7fce3b7fe700 1 -- 172.21.15.16:0/1762716973 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce3c09f630 con 0x7fce3c06a770 2023-09-30T22:31:55.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.837+0000 7fce3a7fc700 1 -- 172.21.15.16:0/1762716973 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1124630376 0 0) 0x7fce3c09ece0 con 0x7fce3c06b080 2023-09-30T22:31:55.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.837+0000 7fce3bfff700 1 --2- 172.21.15.16:0/1762716973 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fce3c131d10 0x7fce3c139860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:55.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.837+0000 7fce3b7fe700 1 --2- 172.21.15.16:0/1762716973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce3c06a770 0x7fce3c06ab40 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fce2c009840 tx=0x7fce2c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12bf829ceb326d44 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:55.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.837+0000 7fce3a7fc700 1 -- 172.21.15.16:0/1762716973 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2c011830 con 0x7fce3c06a770 2023-09-30T22:31:55.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.837+0000 7fce3a7fc700 1 -- 172.21.15.16:0/1762716973 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce2c011990 con 0x7fce3c06a770 2023-09-30T22:31:55.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.837+0000 7fce3a7fc700 1 -- 172.21.15.16:0/1762716973 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2c011d20 con 0x7fce3c06a770 2023-09-30T22:31:55.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.838+0000 7fce41efd700 1 -- 172.21.15.16:0/1762716973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce3c06a770 msgr2=0x7fce3c06ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:55.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.838+0000 7fce41efd700 1 --2- 172.21.15.16:0/1762716973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce3c06a770 0x7fce3c06ab40 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fce2c009840 tx=0x7fce2c009b50 comp rx=0 tx=0).stop 2023-09-30T22:31:55.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.838+0000 7fce41efd700 1 -- 172.21.15.16:0/1762716973 shutdown_connections 2023-09-30T22:31:55.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.838+0000 7fce41efd700 1 --2- 172.21.15.16:0/1762716973 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fce3c131d10 0x7fce3c139860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:55.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.838+0000 7fce41efd700 1 --2- 172.21.15.16:0/1762716973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce3c06a770 0x7fce3c06ab40 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:55.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.838+0000 7fce41efd700 1 -- 172.21.15.16:0/1762716973 >> 172.21.15.16:0/1762716973 conn(0x7fce3c066f50 msgr2=0x7fce3c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:55.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.838+0000 7fce41efd700 1 -- 172.21.15.16:0/1762716973 shutdown_connections 2023-09-30T22:31:55.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.838+0000 7fce41efd700 1 -- 172.21.15.16:0/1762716973 wait complete. 2023-09-30T22:31:55.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.840+0000 7fce41efd700 1 Processor -- start 2023-09-30T22:31:55.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.840+0000 7fce41efd700 1 -- start start 2023-09-30T22:31:55.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.841+0000 7fce41efd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce3c06a770 0x7fce3c120370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:55.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.841+0000 7fce41efd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fce3c06b080 0x7fce3c1208b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:55.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.841+0000 7fce41efd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fce3c131d10 0x7fce3c120df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:55.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.841+0000 7fce41efd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c123e40 con 0x7fce3c06a770 2023-09-30T22:31:55.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.841+0000 7fce41efd700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c130e20 con 0x7fce3c131d10 2023-09-30T22:31:55.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.841+0000 7fce41efd700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c121fd0 con 0x7fce3c06b080 2023-09-30T22:31:55.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.841+0000 7fce3b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce3c06a770 0x7fce3c120370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:55.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.841+0000 7fce3b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce3c06a770 0x7fce3c120370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54580/0 (socket says 172.21.15.16:54580) 2023-09-30T22:31:55.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.841+0000 7fce3b7fe700 1 -- 172.21.15.16:0/22892489 learned_addr learned my addr 172.21.15.16:0/22892489 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:55.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.841+0000 7fce3bfff700 1 --2- 172.21.15.16:0/22892489 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fce3c131d10 0x7fce3c120df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:55.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.841+0000 7fce3affd700 1 --2- 172.21.15.16:0/22892489 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fce3c06b080 0x7fce3c1208b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:55.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.841+0000 7fce3b7fe700 1 -- 172.21.15.16:0/22892489 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fce3c06b080 msgr2=0x7fce3c1208b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:55.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.841+0000 7fce3b7fe700 1 --2- 172.21.15.16:0/22892489 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fce3c06b080 0x7fce3c1208b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:55.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.841+0000 7fce3b7fe700 1 -- 172.21.15.16:0/22892489 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fce3c131d10 msgr2=0x7fce3c120df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:55.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.841+0000 7fce3b7fe700 1 --2- 172.21.15.16:0/22892489 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fce3c131d10 0x7fce3c120df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:55.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.841+0000 7fce3b7fe700 1 -- 172.21.15.16:0/22892489 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce34009a70 con 0x7fce3c06a770 2023-09-30T22:31:55.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.842+0000 7fce3bfff700 1 --2- 172.21.15.16:0/22892489 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fce3c131d10 0x7fce3c120df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:55.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.842+0000 7fce3b7fe700 1 --2- 172.21.15.16:0/22892489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce3c06a770 0x7fce3c120370 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fce2c009fd0 tx=0x7fce2c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:55.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.842+0000 7fce23fff700 1 -- 172.21.15.16:0/22892489 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2c0045d0 con 0x7fce3c06a770 2023-09-30T22:31:55.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.842+0000 7fce23fff700 1 -- 172.21.15.16:0/22892489 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce2c01f070 con 0x7fce3c06a770 2023-09-30T22:31:55.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.842+0000 7fce23fff700 1 -- 172.21.15.16:0/22892489 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2c0038a0 con 0x7fce3c06a770 2023-09-30T22:31:55.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.842+0000 7fce41efd700 1 -- 172.21.15.16:0/22892489 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce2c0094a0 con 0x7fce3c06a770 2023-09-30T22:31:55.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.842+0000 7fce41efd700 1 -- 172.21.15.16:0/22892489 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce3c071e60 con 0x7fce3c06a770 2023-09-30T22:31:55.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.843+0000 7fce23fff700 1 -- 172.21.15.16:0/22892489 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fce2c003a00 con 0x7fce3c06a770 2023-09-30T22:31:55.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.843+0000 7fce41efd700 1 -- 172.21.15.16:0/22892489 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce3c074ea0 con 0x7fce3c06a770 2023-09-30T22:31:55.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.843+0000 7fce23fff700 1 -- 172.21.15.16:0/22892489 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fce2c021070 con 0x7fce3c06a770 2023-09-30T22:31:55.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.850+0000 7fce23fff700 1 -- 172.21.15.16:0/22892489 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fce2c019e10 con 0x7fce3c06a770 2023-09-30T22:31:55.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.961+0000 7fb9a4bef700 1 Processor -- start 2023-09-30T22:31:55.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.961+0000 7fb9a4bef700 1 -- start start 2023-09-30T22:31:55.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.962+0000 7fb9a4bef700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb9a009e1d0 0x7fb9a0138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:55.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.962+0000 7fb9a4bef700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb9a0139170 0x7fb9a013b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:55.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.962+0000 7fb9a4bef700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9a009b120 con 0x7fb9a009d6f0 2023-09-30T22:31:55.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.962+0000 7fb9a4bef700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9a006b460 con 0x7fb9a009e1d0 2023-09-30T22:31:55.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.962+0000 7fb9a4bef700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a009a7e0 con 0x7fb9a0139170 2023-09-30T22:31:55.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.962+0000 7fb99dd9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb9a009e1d0 0x7fb9a0138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:55.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.962+0000 7fb99dd9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb9a009e1d0 0x7fb9a0138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54596/0 (socket says 172.21.15.16:54596) 2023-09-30T22:31:55.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.962+0000 7fb99dd9b700 1 -- 172.21.15.16:0/2166928118 learned_addr learned my addr 172.21.15.16:0/2166928118 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:55.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.962+0000 7fb99ed9d700 1 --2- 172.21.15.16:0/2166928118 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb9a0139170 0x7fb9a013b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:55.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.962+0000 7fb99dd9b700 1 -- 172.21.15.16:0/2166928118 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb9a0139170 msgr2=0x7fb9a013b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:55.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.963+0000 7fb99dd9b700 1 --2- 172.21.15.16:0/2166928118 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb9a0139170 0x7fb9a013b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:55.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.963+0000 7fb99dd9b700 1 -- 172.21.15.16:0/2166928118 >> v1:172.21.15.16:6789/0 conn(0x7fb9a009d6f0 legacy=0x7fb9a009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:55.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.963+0000 7fb99dd9b700 1 -- 172.21.15.16:0/2166928118 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9a009a500 con 0x7fb9a009e1d0 2023-09-30T22:31:55.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.963+0000 7fb99ed9d700 1 --2- 172.21.15.16:0/2166928118 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb9a0139170 0x7fb9a013b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:55.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.963+0000 7fb99dd9b700 1 --2- 172.21.15.16:0/2166928118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9a009e1d0 0x7fb9a0138c30 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fb994009920 tx=0x7fb994009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dfebadf6543b9b5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:55.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.963+0000 7fb99d59a700 1 -- 172.21.15.16:0/2166928118 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb994011830 con 0x7fb9a009e1d0 2023-09-30T22:31:55.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.963+0000 7fb99d59a700 1 -- 172.21.15.16:0/2166928118 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb994011990 con 0x7fb9a009e1d0 2023-09-30T22:31:55.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.963+0000 7fb99d59a700 1 -- 172.21.15.16:0/2166928118 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb994011d20 con 0x7fb9a009e1d0 2023-09-30T22:31:55.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.964+0000 7fb9a4bef700 1 -- 172.21.15.16:0/2166928118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9a009e1d0 msgr2=0x7fb9a0138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:55.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.964+0000 7fb9a4bef700 1 --2- 172.21.15.16:0/2166928118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9a009e1d0 0x7fb9a0138c30 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fb994009920 tx=0x7fb994009c30 comp rx=0 tx=0).stop 2023-09-30T22:31:55.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.964+0000 7fb9a4bef700 1 -- 172.21.15.16:0/2166928118 shutdown_connections 2023-09-30T22:31:55.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.964+0000 7fb9a4bef700 1 --2- 172.21.15.16:0/2166928118 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb9a0139170 0x7fb9a013b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:55.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.964+0000 7fb9a4bef700 1 --2- 172.21.15.16:0/2166928118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9a009e1d0 0x7fb9a0138c30 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:55.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.964+0000 7fb9a4bef700 1 -- 172.21.15.16:0/2166928118 >> 172.21.15.16:0/2166928118 conn(0x7fb9a0066f50 msgr2=0x7fb9a009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:55.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.964+0000 7fb9a4bef700 1 -- 172.21.15.16:0/2166928118 shutdown_connections 2023-09-30T22:31:55.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.964+0000 7fb9a4bef700 1 -- 172.21.15.16:0/2166928118 wait complete. 2023-09-30T22:31:55.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.965+0000 7fb9a4bef700 1 Processor -- start 2023-09-30T22:31:55.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.966+0000 7fb9a4bef700 1 -- start start 2023-09-30T22:31:55.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.966+0000 7fb9a4bef700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb9a009d6f0 0x7fb9a01c84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:55.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.966+0000 7fb9a4bef700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb9a009e1d0 0x7fb9a01c8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:55.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.966+0000 7fb9a4bef700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9a0139170 0x7fb9a011e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:55.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.966+0000 7fb9a4bef700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a00a1a50 con 0x7fb9a0139170 2023-09-30T22:31:55.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.966+0000 7fb9a4bef700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a0121d20 con 0x7fb9a009d6f0 2023-09-30T22:31:55.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.966+0000 7fb9a4bef700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a009c710 con 0x7fb9a009e1d0 2023-09-30T22:31:55.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.966+0000 7fb99e59c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb9a009d6f0 0x7fb9a01c84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:55.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.966+0000 7fb99e59c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb9a009d6f0 0x7fb9a01c84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:33398/0 (socket says 172.21.15.16:33398) 2023-09-30T22:31:55.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.966+0000 7fb99e59c700 1 -- 172.21.15.16:0/421749629 learned_addr learned my addr 172.21.15.16:0/421749629 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:55.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.966+0000 7fb99ed9d700 1 --2- 172.21.15.16:0/421749629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9a0139170 0x7fb9a011e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:55.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.967+0000 7fb99ed9d700 1 -- 172.21.15.16:0/421749629 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb9a009e1d0 msgr2=0x7fb9a01c8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:55.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.967+0000 7fb99ed9d700 1 --2- 172.21.15.16:0/421749629 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb9a009e1d0 0x7fb9a01c8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:55.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.967+0000 7fb99ed9d700 1 -- 172.21.15.16:0/421749629 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb9a009d6f0 msgr2=0x7fb9a01c84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:55.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.967+0000 7fb99ed9d700 1 --2- 172.21.15.16:0/421749629 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb9a009d6f0 0x7fb9a01c84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:55.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.967+0000 7fb99ed9d700 1 -- 172.21.15.16:0/421749629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb998009a70 con 0x7fb9a0139170 2023-09-30T22:31:55.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.967+0000 7fb99e59c700 1 --2- 172.21.15.16:0/421749629 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb9a009d6f0 0x7fb9a01c84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:55.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.967+0000 7fb99ed9d700 1 --2- 172.21.15.16:0/421749629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9a0139170 0x7fb9a011e7e0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb99800efc0 tx=0x7fb99800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:55.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.967+0000 7fb98effd700 1 -- 172.21.15.16:0/421749629 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99800cd40 con 0x7fb9a0139170 2023-09-30T22:31:55.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.967+0000 7fb98effd700 1 -- 172.21.15.16:0/421749629 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb99800f040 con 0x7fb9a0139170 2023-09-30T22:31:55.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.967+0000 7fb98effd700 1 -- 172.21.15.16:0/421749629 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9980146c0 con 0x7fb9a0139170 2023-09-30T22:31:55.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.967+0000 7fb9a4bef700 1 -- 172.21.15.16:0/421749629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb994009550 con 0x7fb9a0139170 2023-09-30T22:31:55.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.967+0000 7fb9a4bef700 1 -- 172.21.15.16:0/421749629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9a0120450 con 0x7fb9a0139170 2023-09-30T22:31:55.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.968+0000 7fb98effd700 1 -- 172.21.15.16:0/421749629 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb998014820 con 0x7fb9a0139170 2023-09-30T22:31:55.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.968+0000 7fb98effd700 1 -- 172.21.15.16:0/421749629 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fb998040360 con 0x7fb9a0139170 2023-09-30T22:31:55.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.969+0000 7fb9a4bef700 1 -- 172.21.15.16:0/421749629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9a0122b10 con 0x7fb9a0139170 2023-09-30T22:31:55.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.974+0000 7fb98effd700 1 -- 172.21.15.16:0/421749629 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb99801d400 con 0x7fb9a0139170 2023-09-30T22:31:55.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.982+0000 7f3fc4ca7700 1 Processor -- start 2023-09-30T22:31:55.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.983+0000 7f8f2c6bc700 1 Processor -- start 2023-09-30T22:31:55.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.983+0000 7f8f2c6bc700 1 -- start start 2023-09-30T22:31:55.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.983+0000 7f3fc4ca7700 1 -- start start 2023-09-30T22:31:55.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.983+0000 7f3fc4ca7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3fc012cf60 0x7f3fc006f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:55.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.983+0000 7f3fc4ca7700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3fc012f8b0 0x7f3fc012fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:55.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.983+0000 7f3fc4ca7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3fc0130260 0x7f3fc0136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:55.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.983+0000 7f3fc4ca7700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fc012d330 con 0x7f3fc012f8b0 2023-09-30T22:31:55.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.983+0000 7f3fc4ca7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc012cd90 con 0x7f3fc0130260 2023-09-30T22:31:55.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.983+0000 7f3fc4ca7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc01306d0 con 0x7f3fc012cf60 2023-09-30T22:31:55.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.984+0000 7f3fbdd9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3fc012f8b0 0x7f3fc012fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:55.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.984+0000 7f3fbdd9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3fc012f8b0 0x7f3fc012fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54606/0 (socket says 172.21.15.16:54606) 2023-09-30T22:31:55.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.984+0000 7f3fbdd9b700 1 -- 172.21.15.16:0/4081296548 learned_addr learned my addr 172.21.15.16:0/4081296548 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:55.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.984+0000 7f3fbed9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3fc0130260 0x7f3fc0136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:55.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.984+0000 7f3fbed9d700 1 -- 172.21.15.16:0/4081296548 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3fc012cf60 msgr2=0x7f3fc006f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:55.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.984+0000 7f3fbed9d700 1 --2- 172.21.15.16:0/4081296548 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3fc012cf60 0x7f3fc006f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:55.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.984+0000 7f3fbed9d700 1 -- 172.21.15.16:0/4081296548 >> v2:172.21.15.16:3300/0 conn(0x7f3fc012f8b0 msgr2=0x7f3fc012fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:55.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.984+0000 7f3fbed9d700 1 --2- 172.21.15.16:0/4081296548 >> v2:172.21.15.16:3300/0 conn(0x7f3fc012f8b0 0x7f3fc012fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:55.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.984+0000 7f3fbed9d700 1 -- 172.21.15.16:0/4081296548 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc0071780 con 0x7f3fc0130260 2023-09-30T22:31:55.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.984+0000 7f8f2c6bc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8f24131810 0x7f8f24131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:55.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.984+0000 7f8f2c6bc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8f2409a580 0x7f8f24139660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:55.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.984+0000 7f8f2c6bc700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f241313c0 con 0x7f8f24099a40 2023-09-30T22:31:55.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.984+0000 7f8f2c6bc700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f2406e8c0 con 0x7f8f24131810 2023-09-30T22:31:55.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.984+0000 7f8f2c6bc700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f24071d20 con 0x7f8f2409a580 2023-09-30T22:31:55.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.984+0000 7f8f2a458700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8f24131810 0x7f8f24131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:55.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.984+0000 7f8f29c57700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8f24099a40 0x7f8f24099e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50398/0 (socket says 172.21.15.16:50398) 2023-09-30T22:31:55.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.984+0000 7f8f29c57700 1 -- 172.21.15.16:0/2759170188 learned_addr learned my addr 172.21.15.16:0/2759170188 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:31:55.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.984+0000 7f8f2ac59700 1 --2- 172.21.15.16:0/2759170188 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8f2409a580 0x7f8f24139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:55.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.984+0000 7f8f29456700 1 -- 172.21.15.16:0/2759170188 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3079692664 0 0) 0x7f8f241313c0 con 0x7f8f24099a40 2023-09-30T22:31:55.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.985+0000 7f8f29456700 1 -- 172.21.15.16:0/2759170188 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f100079c0 con 0x7f8f24099a40 2023-09-30T22:31:55.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.985+0000 7f8f2ac59700 1 -- 172.21.15.16:0/2759170188 >> v2:172.21.15.16:3300/0 conn(0x7f8f24131810 msgr2=0x7f8f24131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:55.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.985+0000 7f8f2ac59700 1 --2- 172.21.15.16:0/2759170188 >> v2:172.21.15.16:3300/0 conn(0x7f8f24131810 0x7f8f24131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:55.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.985+0000 7f8f2ac59700 1 -- 172.21.15.16:0/2759170188 >> v1:172.21.15.16:6789/0 conn(0x7f8f24099a40 legacy=0x7f8f24099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:55.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.985+0000 7f8f2ac59700 1 -- 172.21.15.16:0/2759170188 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f240721a0 con 0x7f8f2409a580 2023-09-30T22:31:55.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.985+0000 7f3fbed9d700 1 --2- 172.21.15.16:0/4081296548 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3fc0130260 0x7f3fc0136c20 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3fb800f370 tx=0x7f3fb800f680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51890dd9b3b5a79a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:55.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.985+0000 7f8f2a458700 1 --2- 172.21.15.16:0/2759170188 >> v2:172.21.15.16:3300/0 conn(0x7f8f24131810 0x7f8f24131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:55.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.985+0000 7f8f2ac59700 1 --2- 172.21.15.16:0/2759170188 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8f2409a580 0x7f8f24139660 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c009d90 tx=0x7f8f1c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7dbcb307e8546e4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:55.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.985+0000 7f3fbd59a700 1 -- 172.21.15.16:0/4081296548 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb8012070 con 0x7f3fc0130260 2023-09-30T22:31:55.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.985+0000 7f3fbd59a700 1 -- 172.21.15.16:0/4081296548 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fb800fe70 con 0x7f3fc0130260 2023-09-30T22:31:55.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.985+0000 7f8f29456700 1 -- 172.21.15.16:0/2759170188 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c00e070 con 0x7f8f2409a580 2023-09-30T22:31:55.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.985+0000 7f8f29456700 1 -- 172.21.15.16:0/2759170188 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f1c00bae0 con 0x7f8f2409a580 2023-09-30T22:31:55.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.985+0000 7f8f29456700 1 -- 172.21.15.16:0/2759170188 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c00be50 con 0x7f8f2409a580 2023-09-30T22:31:55.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.985+0000 7f3fbd59a700 1 -- 172.21.15.16:0/4081296548 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb800d2f0 con 0x7f3fc0130260 2023-09-30T22:31:55.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.986+0000 7f8f2c6bc700 1 -- 172.21.15.16:0/2759170188 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8f2409a580 msgr2=0x7f8f24139660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:55.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.986+0000 7f8f2c6bc700 1 --2- 172.21.15.16:0/2759170188 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8f2409a580 0x7f8f24139660 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c009d90 tx=0x7f8f1c00b330 comp rx=0 tx=0).stop 2023-09-30T22:31:55.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.986+0000 7f3fc4ca7700 1 -- 172.21.15.16:0/4081296548 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3fc0130260 msgr2=0x7f3fc0136c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:55.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.986+0000 7f3fc4ca7700 1 --2- 172.21.15.16:0/4081296548 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3fc0130260 0x7f3fc0136c20 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3fb800f370 tx=0x7f3fb800f680 comp rx=0 tx=0).stop 2023-09-30T22:31:55.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.986+0000 7f8f2c6bc700 1 -- 172.21.15.16:0/2759170188 shutdown_connections 2023-09-30T22:31:55.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.986+0000 7f8f2c6bc700 1 --2- 172.21.15.16:0/2759170188 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8f2409a580 0x7f8f24139660 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:55.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.986+0000 7f8f2c6bc700 1 --2- 172.21.15.16:0/2759170188 >> v2:172.21.15.16:3300/0 conn(0x7f8f24131810 0x7f8f24131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:55.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.986+0000 7f8f2c6bc700 1 -- 172.21.15.16:0/2759170188 >> 172.21.15.16:0/2759170188 conn(0x7f8f240a2aa0 msgr2=0x7f8f240a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:55.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.986+0000 7f3fc4ca7700 1 -- 172.21.15.16:0/4081296548 shutdown_connections 2023-09-30T22:31:55.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.986+0000 7f3fc4ca7700 1 --2- 172.21.15.16:0/4081296548 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3fc0130260 0x7f3fc0136c20 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:55.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.986+0000 7f3fc4ca7700 1 --2- 172.21.15.16:0/4081296548 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3fc012cf60 0x7f3fc006f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:55.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.986+0000 7f3fc4ca7700 1 --2- 172.21.15.16:0/4081296548 >> v2:172.21.15.16:3300/0 conn(0x7f3fc012f8b0 0x7f3fc012fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:55.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.986+0000 7f3fc4ca7700 1 -- 172.21.15.16:0/4081296548 >> 172.21.15.16:0/4081296548 conn(0x7f3fc00a2aa0 msgr2=0x7f3fc00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:55.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.986+0000 7f8f2c6bc700 1 -- 172.21.15.16:0/2759170188 shutdown_connections 2023-09-30T22:31:55.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.986+0000 7f8f2c6bc700 1 -- 172.21.15.16:0/2759170188 wait complete. 2023-09-30T22:31:55.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.986+0000 7f3fc4ca7700 1 -- 172.21.15.16:0/4081296548 shutdown_connections 2023-09-30T22:31:55.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.986+0000 7f3fc4ca7700 1 -- 172.21.15.16:0/4081296548 wait complete. 2023-09-30T22:31:55.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.988+0000 7f8f2c6bc700 1 Processor -- start 2023-09-30T22:31:55.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.988+0000 7f8f2c6bc700 1 -- start start 2023-09-30T22:31:55.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.988+0000 7f3fc4ca7700 1 Processor -- start 2023-09-30T22:31:55.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.988+0000 7f3fc4ca7700 1 -- start start 2023-09-30T22:31:55.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.988+0000 7f8f2c6bc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8f24099a40 0x7f8f241c3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:55.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.988+0000 7f8f2c6bc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8f2409a580 0x7f8f241c4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:55.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.988+0000 7f8f2c6bc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f24131810 0x7f8f241c04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:55.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.988+0000 7f8f2c6bc700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2411cc30 con 0x7f8f24131810 2023-09-30T22:31:55.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.988+0000 7f8f2c6bc700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2406eb70 con 0x7f8f24099a40 2023-09-30T22:31:55.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.988+0000 7f8f2c6bc700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2411be00 con 0x7f8f2409a580 2023-09-30T22:31:55.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.988+0000 7f3fc4ca7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fc012cf60 0x7f3fc009be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:55.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.988+0000 7f3fc4ca7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3fc012f8b0 0x7f3fc009c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:55.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.988+0000 7f3fc4ca7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3fc0130260 0x7f3fc01c4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:55.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.988+0000 7f3fc4ca7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc009ec10 con 0x7f3fc012cf60 2023-09-30T22:31:55.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.988+0000 7f3fc4ca7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc0118db0 con 0x7f3fc012f8b0 2023-09-30T22:31:55.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.988+0000 7f3fc4ca7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc009b890 con 0x7f3fc0130260 2023-09-30T22:31:55.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.988+0000 7f3fbe59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fc012cf60 0x7f3fc009be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:55.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.989+0000 7f3fbe59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fc012cf60 0x7f3fc009be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54644/0 (socket says 172.21.15.16:54644) 2023-09-30T22:31:55.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.989+0000 7f3fbe59c700 1 -- 172.21.15.16:0/3846008061 learned_addr learned my addr 172.21.15.16:0/3846008061 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:55.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.989+0000 7f8f2a458700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8f24099a40 0x7f8f241c3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:55.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.989+0000 7f8f2a458700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8f24099a40 0x7f8f241c3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:33428/0 (socket says 172.21.15.16:33428) 2023-09-30T22:31:55.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.989+0000 7f8f2a458700 1 -- 172.21.15.16:0/679025519 learned_addr learned my addr 172.21.15.16:0/679025519 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:55.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.989+0000 7f3fbdd9b700 1 --2- 172.21.15.16:0/3846008061 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3fc012f8b0 0x7f3fc009c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:55.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.989+0000 7f3fbe59c700 1 -- 172.21.15.16:0/3846008061 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3fc0130260 msgr2=0x7f3fc01c4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:55.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.989+0000 7f3fbe59c700 1 --2- 172.21.15.16:0/3846008061 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3fc0130260 0x7f3fc01c4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:55.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.989+0000 7f3fbe59c700 1 -- 172.21.15.16:0/3846008061 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3fc012f8b0 msgr2=0x7f3fc009c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:55.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.989+0000 7f3fbe59c700 1 --2- 172.21.15.16:0/3846008061 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3fc012f8b0 0x7f3fc009c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:55.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.989+0000 7f3fbe59c700 1 -- 172.21.15.16:0/3846008061 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fb800f050 con 0x7f3fc012cf60 2023-09-30T22:31:55.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.989+0000 7f8f2ac59700 1 --2- 172.21.15.16:0/679025519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f24131810 0x7f8f241c04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:55.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.989+0000 7f3fbe59c700 1 --2- 172.21.15.16:0/3846008061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fc012cf60 0x7f3fc009be30 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f3fb000b530 tx=0x7f3fb000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:55.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.990+0000 7f8f2ac59700 1 -- 172.21.15.16:0/679025519 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8f2409a580 msgr2=0x7f8f241c4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:55.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.990+0000 7f8f2ac59700 1 --2- 172.21.15.16:0/679025519 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8f2409a580 0x7f8f241c4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:55.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.990+0000 7f8f2ac59700 1 -- 172.21.15.16:0/679025519 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8f24099a40 msgr2=0x7f8f241c3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:55.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.990+0000 7f8f2ac59700 1 --2- 172.21.15.16:0/679025519 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8f24099a40 0x7f8f241c3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:55.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.990+0000 7f8f2ac59700 1 -- 172.21.15.16:0/679025519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f200094a0 con 0x7f8f24131810 2023-09-30T22:31:55.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.990+0000 7f3faeffd700 1 -- 172.21.15.16:0/3846008061 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb0011830 con 0x7f3fc012cf60 2023-09-30T22:31:55.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.990+0000 7f3faeffd700 1 -- 172.21.15.16:0/3846008061 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fb00119b0 con 0x7f3fc012cf60 2023-09-30T22:31:55.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.990+0000 7f3faeffd700 1 -- 172.21.15.16:0/3846008061 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb0011d60 con 0x7f3fc012cf60 2023-09-30T22:31:56.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.990+0000 7f8f2ac59700 1 --2- 172.21.15.16:0/679025519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f24131810 0x7f8f241c04c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c015000 tx=0x7f8f1c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:56.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.990+0000 7f3fc4ca7700 1 -- 172.21.15.16:0/3846008061 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fc0129fa0 con 0x7f3fc012cf60 2023-09-30T22:31:56.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.990+0000 7f3fc4ca7700 1 -- 172.21.15.16:0/3846008061 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fc009d7d0 con 0x7f3fc012cf60 2023-09-30T22:31:56.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.991+0000 7f8f1affd700 1 -- 172.21.15.16:0/679025519 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c009430 con 0x7f8f24131810 2023-09-30T22:31:56.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.991+0000 7f8f1affd700 1 -- 172.21.15.16:0/679025519 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f1c003e30 con 0x7f8f24131810 2023-09-30T22:31:56.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.991+0000 7f8f1affd700 1 -- 172.21.15.16:0/679025519 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c0266a0 con 0x7f8f24131810 2023-09-30T22:31:56.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.991+0000 7f8f2c6bc700 1 -- 172.21.15.16:0/679025519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f1c009a70 con 0x7f8f24131810 2023-09-30T22:31:56.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.991+0000 7f8f2c6bc700 1 -- 172.21.15.16:0/679025519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f2411b000 con 0x7f8f24131810 2023-09-30T22:31:56.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.991+0000 7f3faeffd700 1 -- 172.21.15.16:0/3846008061 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3fb001a400 con 0x7f3fc012cf60 2023-09-30T22:31:56.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.991+0000 7f3faeffd700 1 -- 172.21.15.16:0/3846008061 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f3fb003f800 con 0x7f3fc012cf60 2023-09-30T22:31:56.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.991+0000 7f3fc4ca7700 1 -- 172.21.15.16:0/3846008061 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fc0074ea0 con 0x7f3fc012cf60 2023-09-30T22:31:56.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.993+0000 7f8f2c6bc700 1 -- 172.21.15.16:0/679025519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f24074ea0 con 0x7f8f24131810 2023-09-30T22:31:56.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.994+0000 7f8f1affd700 1 -- 172.21.15.16:0/679025519 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8f1c012430 con 0x7f8f24131810 2023-09-30T22:31:56.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.994+0000 7f8f1affd700 1 -- 172.21.15.16:0/679025519 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f8f1c0419a0 con 0x7f8f24131810 2023-09-30T22:31:56.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:55.997+0000 7f3faeffd700 1 -- 172.21.15.16:0/3846008061 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3fb00193b0 con 0x7f3fc012cf60 2023-09-30T22:31:56.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:56.005+0000 7f8f1affd700 1 -- 172.21.15.16:0/679025519 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8f1c012e40 con 0x7f8f24131810 2023-09-30T22:31:56.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:56.071+0000 7fce41efd700 1 -- 172.21.15.16:0/22892489 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fce3c06c360 con 0x7fce3c06a770 2023-09-30T22:31:56.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:56.072+0000 7fce23fff700 1 -- 172.21.15.16:0/22892489 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fce2c015070 con 0x7fce3c06a770 2023-09-30T22:31:56.072 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:31:56.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:56.074+0000 7fce41efd700 1 -- 172.21.15.16:0/22892489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce3c06a770 msgr2=0x7fce3c120370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:56.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:56.074+0000 7fce41efd700 1 --2- 172.21.15.16:0/22892489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce3c06a770 0x7fce3c120370 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fce2c009fd0 tx=0x7fce2c004750 comp rx=0 tx=0).stop 2023-09-30T22:31:56.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:56.075+0000 7fce41efd700 1 -- 172.21.15.16:0/22892489 shutdown_connections 2023-09-30T22:31:56.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:56.075+0000 7fce41efd700 1 --2- 172.21.15.16:0/22892489 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fce3c131d10 0x7fce3c120df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:56.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:56.075+0000 7fce41efd700 1 --2- 172.21.15.16:0/22892489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce3c06a770 0x7fce3c120370 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:56.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:56.075+0000 7fce41efd700 1 --2- 172.21.15.16:0/22892489 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fce3c06b080 0x7fce3c1208b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:56.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:56.075+0000 7fce41efd700 1 -- 172.21.15.16:0/22892489 >> 172.21.15.16:0/22892489 conn(0x7fce3c066f50 msgr2=0x7fce3c0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:56.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:56.075+0000 7fce41efd700 1 -- 172.21.15.16:0/22892489 shutdown_connections 2023-09-30T22:31:56.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:56.075+0000 7fce41efd700 1 -- 172.21.15.16:0/22892489 wait complete. 2023-09-30T22:31:56.090 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:31:56.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:56.184+0000 7fb9a4bef700 1 -- 172.21.15.16:0/421749629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb9a009f420 con 0x7fb9a0139170 2023-09-30T22:31:56.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:56.184+0000 7fb98effd700 1 -- 172.21.15.16:0/421749629 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb99802b090 con 0x7fb9a0139170 2023-09-30T22:31:56.184 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:31:56.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:56.186+0000 7fb9a4bef700 1 -- 172.21.15.16:0/421749629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9a0139170 msgr2=0x7fb9a011e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:56.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:56.186+0000 7fb9a4bef700 1 --2- 172.21.15.16:0/421749629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9a0139170 0x7fb9a011e7e0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb99800efc0 tx=0x7fb99800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:31:56.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:56.186+0000 7fb9a4bef700 1 -- 172.21.15.16:0/421749629 shutdown_connections 2023-09-30T22:31:56.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:56.186+0000 7fb9a4bef700 1 --2- 172.21.15.16:0/421749629 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb9a009d6f0 0x7fb9a01c84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:56.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:56.186+0000 7fb9a4bef700 1 --2- 172.21.15.16:0/421749629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9a0139170 0x7fb9a011e7e0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:56.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:56.186+0000 7fb9a4bef700 1 --2- 172.21.15.16:0/421749629 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb9a009e1d0 0x7fb9a01c8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:56.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:56.186+0000 7fb9a4bef700 1 -- 172.21.15.16:0/421749629 >> 172.21.15.16:0/421749629 conn(0x7fb9a0066f50 msgr2=0x7fb9a009b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:56.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:56.187+0000 7fb9a4bef700 1 -- 172.21.15.16:0/421749629 shutdown_connections 2023-09-30T22:31:56.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:56.187+0000 7fb9a4bef700 1 -- 172.21.15.16:0/421749629 wait complete. 2023-09-30T22:31:56.200 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:31:56.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:56.207+0000 7f3fc4ca7700 1 -- 172.21.15.16:0/3846008061 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3fc009ed50 con 0x7f3fc012cf60 2023-09-30T22:31:56.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:56.207+0000 7f3faeffd700 1 -- 172.21.15.16:0/3846008061 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3fb001ae60 con 0x7f3fc012cf60 2023-09-30T22:31:56.208 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:31:56.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:56.209+0000 7f3fc4ca7700 1 -- 172.21.15.16:0/3846008061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fc012cf60 msgr2=0x7f3fc009be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:56.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:56.209+0000 7f3fc4ca7700 1 --2- 172.21.15.16:0/3846008061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fc012cf60 0x7f3fc009be30 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f3fb000b530 tx=0x7f3fb000b840 comp rx=0 tx=0).stop 2023-09-30T22:31:56.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:56.209+0000 7f3fc4ca7700 1 -- 172.21.15.16:0/3846008061 shutdown_connections 2023-09-30T22:31:56.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:56.209+0000 7f3fc4ca7700 1 --2- 172.21.15.16:0/3846008061 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3fc0130260 0x7f3fc01c4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:56.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:56.209+0000 7f3fc4ca7700 1 --2- 172.21.15.16:0/3846008061 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fc012cf60 0x7f3fc009be30 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:56.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:56.209+0000 7f3fc4ca7700 1 --2- 172.21.15.16:0/3846008061 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3fc012f8b0 0x7f3fc009c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:56.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:56.209+0000 7f3fc4ca7700 1 -- 172.21.15.16:0/3846008061 >> 172.21.15.16:0/3846008061 conn(0x7f3fc00a2aa0 msgr2=0x7f3fc012dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:56.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:56.210+0000 7f3fc4ca7700 1 -- 172.21.15.16:0/3846008061 shutdown_connections 2023-09-30T22:31:56.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:56.210+0000 7f3fc4ca7700 1 -- 172.21.15.16:0/3846008061 wait complete. 2023-09-30T22:31:56.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:56.213+0000 7f8f2c6bc700 1 -- 172.21.15.16:0/679025519 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8f2411cde0 con 0x7f8f24131810 2023-09-30T22:31:56.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:56.213+0000 7f8f1affd700 1 -- 172.21.15.16:0/679025519 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8f2411cde0 con 0x7f8f24131810 2023-09-30T22:31:56.214 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:31:56.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:56.216+0000 7f8f2c6bc700 1 -- 172.21.15.16:0/679025519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f24131810 msgr2=0x7f8f241c04c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:56.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:56.216+0000 7f8f2c6bc700 1 --2- 172.21.15.16:0/679025519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f24131810 0x7f8f241c04c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c015000 tx=0x7f8f1c004ad0 comp rx=0 tx=0).stop 2023-09-30T22:31:56.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:56.216+0000 7f8f2c6bc700 1 -- 172.21.15.16:0/679025519 shutdown_connections 2023-09-30T22:31:56.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:56.216+0000 7f8f2c6bc700 1 --2- 172.21.15.16:0/679025519 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8f24099a40 0x7f8f241c3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:56.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:56.216+0000 7f8f2c6bc700 1 --2- 172.21.15.16:0/679025519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8f24131810 0x7f8f241c04c0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:56.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:56.216+0000 7f8f2c6bc700 1 --2- 172.21.15.16:0/679025519 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8f2409a580 0x7f8f241c4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:56.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:56.216+0000 7f8f2c6bc700 1 -- 172.21.15.16:0/679025519 >> 172.21.15.16:0/679025519 conn(0x7f8f240a2aa0 msgr2=0x7f8f2412e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:56.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:56.216+0000 7f8f2c6bc700 1 -- 172.21.15.16:0/679025519 shutdown_connections 2023-09-30T22:31:56.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:56.216+0000 7f8f2c6bc700 1 -- 172.21.15.16:0/679025519 wait complete. 2023-09-30T22:31:56.224 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:31:56.230 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:31:57.091 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:57.202 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:57.225 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:57.231 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:57.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.238+0000 7fb2b2c5a700 1 Processor -- start 2023-09-30T22:31:57.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.238+0000 7fb2b2c5a700 1 -- start start 2023-09-30T22:31:57.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.239+0000 7fb2b2c5a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb2ac09d630 0x7fb2ac0a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:57.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.239+0000 7fb2b2c5a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb2ac138d80 0x7fb2ac13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:57.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.239+0000 7fb2b2c5a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2ac0a2a70 con 0x7fb2ac0a0a30 2023-09-30T22:31:57.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.239+0000 7fb2b2c5a700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2ac0a01d0 con 0x7fb2ac138d80 2023-09-30T22:31:57.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.239+0000 7fb2b2c5a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ac09e2b0 con 0x7fb2ac09d630 2023-09-30T22:31:57.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.239+0000 7fb2b11f7700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb2ac138d80 0x7fb2ac13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:57.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.239+0000 7fb2b11f7700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb2ac138d80 0x7fb2ac13b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54654/0 (socket says 172.21.15.16:54654) 2023-09-30T22:31:57.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.239+0000 7fb2b11f7700 1 -- 172.21.15.16:0/1650517077 learned_addr learned my addr 172.21.15.16:0/1650517077 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:57.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.239+0000 7fb2b09f6700 1 --1- 172.21.15.16:0/1650517077 >> v1:172.21.15.16:6789/0 conn(0x7fb2ac0a0a30 0x7fb2ac0a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50414/0 (socket says 172.21.15.16:50414) 2023-09-30T22:31:57.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.239+0000 7fb2abfff700 1 --2- 172.21.15.16:0/1650517077 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb2ac09d630 0x7fb2ac0a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:57.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.239+0000 7fb2b11f7700 1 -- 172.21.15.16:0/1650517077 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb2ac09d630 msgr2=0x7fb2ac0a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:57.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.239+0000 7fb2b11f7700 1 --2- 172.21.15.16:0/1650517077 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb2ac09d630 0x7fb2ac0a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:57.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.239+0000 7fb2b11f7700 1 -- 172.21.15.16:0/1650517077 >> v1:172.21.15.16:6789/0 conn(0x7fb2ac0a0a30 legacy=0x7fb2ac0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:57.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.239+0000 7fb2b11f7700 1 -- 172.21.15.16:0/1650517077 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2ac09dc50 con 0x7fb2ac138d80 2023-09-30T22:31:57.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.240+0000 7fb2b11f7700 1 --2- 172.21.15.16:0/1650517077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2ac138d80 0x7fb2ac13b160 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fb29c0098f0 tx=0x7fb29c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db4f5e9f02602868 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:57.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.240+0000 7fb2ab7fe700 1 -- 172.21.15.16:0/1650517077 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29c00b8a0 con 0x7fb2ac138d80 2023-09-30T22:31:57.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.240+0000 7fb2ab7fe700 1 -- 172.21.15.16:0/1650517077 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb29c00ba00 con 0x7fb2ac138d80 2023-09-30T22:31:57.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.240+0000 7fb2ab7fe700 1 -- 172.21.15.16:0/1650517077 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29c00bd90 con 0x7fb2ac138d80 2023-09-30T22:31:57.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.241+0000 7fb2b2c5a700 1 -- 172.21.15.16:0/1650517077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2ac138d80 msgr2=0x7fb2ac13b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:57.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.241+0000 7fb2b2c5a700 1 --2- 172.21.15.16:0/1650517077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2ac138d80 0x7fb2ac13b160 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fb29c0098f0 tx=0x7fb29c009c00 comp rx=0 tx=0).stop 2023-09-30T22:31:57.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.241+0000 7fb2b2c5a700 1 -- 172.21.15.16:0/1650517077 shutdown_connections 2023-09-30T22:31:57.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.241+0000 7fb2b2c5a700 1 --2- 172.21.15.16:0/1650517077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2ac138d80 0x7fb2ac13b160 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:57.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.241+0000 7fb2b2c5a700 1 --2- 172.21.15.16:0/1650517077 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb2ac09d630 0x7fb2ac0a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:57.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.241+0000 7fb2b2c5a700 1 -- 172.21.15.16:0/1650517077 >> 172.21.15.16:0/1650517077 conn(0x7fb2ac0a3050 msgr2=0x7fb2ac0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:57.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.241+0000 7fb2b2c5a700 1 -- 172.21.15.16:0/1650517077 shutdown_connections 2023-09-30T22:31:57.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.241+0000 7fb2b2c5a700 1 -- 172.21.15.16:0/1650517077 wait complete. 2023-09-30T22:31:57.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.242+0000 7fb2b2c5a700 1 Processor -- start 2023-09-30T22:31:57.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.242+0000 7fb2b2c5a700 1 -- start start 2023-09-30T22:31:57.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.243+0000 7fb2b2c5a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb2ac09d630 0x7fb2ac1339e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:57.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.243+0000 7fb2b2c5a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2ac0a0a30 0x7fb2ac133f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:57.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.243+0000 7fb2b2c5a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb2ac138d80 0x7fb2ac134460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:57.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.243+0000 7fb2b2c5a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ac137270 con 0x7fb2ac0a0a30 2023-09-30T22:31:57.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.243+0000 7fb2b2c5a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ac1380f0 con 0x7fb2ac09d630 2023-09-30T22:31:57.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.243+0000 7fb2b2c5a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ac134e40 con 0x7fb2ac138d80 2023-09-30T22:31:57.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.243+0000 7fb2abfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2ac0a0a30 0x7fb2ac133f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:57.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.243+0000 7fb2abfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2ac0a0a30 0x7fb2ac133f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54662/0 (socket says 172.21.15.16:54662) 2023-09-30T22:31:57.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.243+0000 7fb2abfff700 1 -- 172.21.15.16:0/2146161112 learned_addr learned my addr 172.21.15.16:0/2146161112 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:57.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.243+0000 7fb2b11f7700 1 --2- 172.21.15.16:0/2146161112 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb2ac138d80 0x7fb2ac134460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:57.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.243+0000 7fb2b09f6700 1 --2- 172.21.15.16:0/2146161112 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb2ac09d630 0x7fb2ac1339e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:57.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.244+0000 7fb2abfff700 1 -- 172.21.15.16:0/2146161112 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb2ac138d80 msgr2=0x7fb2ac134460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:57.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.244+0000 7fb2abfff700 1 --2- 172.21.15.16:0/2146161112 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb2ac138d80 0x7fb2ac134460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:57.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.244+0000 7fb2abfff700 1 -- 172.21.15.16:0/2146161112 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb2ac09d630 msgr2=0x7fb2ac1339e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:57.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.244+0000 7fb2abfff700 1 --2- 172.21.15.16:0/2146161112 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb2ac09d630 0x7fb2ac1339e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:57.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.244+0000 7fb2abfff700 1 -- 172.21.15.16:0/2146161112 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb29c009550 con 0x7fb2ac0a0a30 2023-09-30T22:31:57.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.244+0000 7fb2abfff700 1 --2- 172.21.15.16:0/2146161112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2ac0a0a30 0x7fb2ac133f20 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fb2a000e840 tx=0x7fb2a000eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:57.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.244+0000 7fb2a97fa700 1 -- 172.21.15.16:0/2146161112 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a0013610 con 0x7fb2ac0a0a30 2023-09-30T22:31:57.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.244+0000 7fb2a97fa700 1 -- 172.21.15.16:0/2146161112 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2a0013770 con 0x7fb2ac0a0a30 2023-09-30T22:31:57.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.244+0000 7fb2a97fa700 1 -- 172.21.15.16:0/2146161112 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a0013b20 con 0x7fb2ac0a0a30 2023-09-30T22:31:57.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.244+0000 7fb2b2c5a700 1 -- 172.21.15.16:0/2146161112 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2ac0a4d40 con 0x7fb2ac0a0a30 2023-09-30T22:31:57.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.244+0000 7fb2b2c5a700 1 -- 172.21.15.16:0/2146161112 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2ac138750 con 0x7fb2ac0a0a30 2023-09-30T22:31:57.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.246+0000 7fb2a97fa700 1 -- 172.21.15.16:0/2146161112 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb2a0013c80 con 0x7fb2ac0a0a30 2023-09-30T22:31:57.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.246+0000 7fb2b2c5a700 1 -- 172.21.15.16:0/2146161112 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2ac0747f0 con 0x7fb2ac0a0a30 2023-09-30T22:31:57.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.246+0000 7fb2a97fa700 1 -- 172.21.15.16:0/2146161112 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fb2a003f340 con 0x7fb2ac0a0a30 2023-09-30T22:31:57.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.251+0000 7fb2a97fa700 1 -- 172.21.15.16:0/2146161112 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb2a0012370 con 0x7fb2ac0a0a30 2023-09-30T22:31:57.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.349+0000 7f7bb8caf700 1 Processor -- start 2023-09-30T22:31:57.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.349+0000 7f7bb8caf700 1 -- start start 2023-09-30T22:31:57.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.350+0000 7f7bb8caf700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7bb406e570 0x7f7bb406e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:57.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.350+0000 7f7bb8caf700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7bb409eca0 0x7f7bb406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:57.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.350+0000 7f7bb8caf700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7bb4134610 0x7f7bb4134a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:57.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.350+0000 7f7bb8caf700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bb4070ee0 con 0x7f7bb409eca0 2023-09-30T22:31:57.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.350+0000 7f7bb8caf700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb4071480 con 0x7f7bb406e570 2023-09-30T22:31:57.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.350+0000 7f7bb8caf700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb406f3c0 con 0x7f7bb4134610 2023-09-30T22:31:57.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.350+0000 7f7bb1d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7bb409eca0 0x7f7bb406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:57.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.350+0000 7f7bb1d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7bb409eca0 0x7f7bb406ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54666/0 (socket says 172.21.15.16:54666) 2023-09-30T22:31:57.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.350+0000 7f7bb1d9b700 1 -- 172.21.15.16:0/2792213085 learned_addr learned my addr 172.21.15.16:0/2792213085 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:57.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.350+0000 7f7bb2d9d700 1 --2- 172.21.15.16:0/2792213085 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7bb4134610 0x7f7bb4134a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:57.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.350+0000 7f7bb259c700 1 --2- 172.21.15.16:0/2792213085 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7bb406e570 0x7f7bb406e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:57.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.350+0000 7f7bb1d9b700 1 -- 172.21.15.16:0/2792213085 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7bb4134610 msgr2=0x7f7bb4134a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:57.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.350+0000 7f7bb1d9b700 1 --2- 172.21.15.16:0/2792213085 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7bb4134610 0x7f7bb4134a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:57.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.351+0000 7f7bb1d9b700 1 -- 172.21.15.16:0/2792213085 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7bb406e570 msgr2=0x7f7bb406e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:57.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.351+0000 7f7bb1d9b700 1 --2- 172.21.15.16:0/2792213085 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7bb406e570 0x7f7bb406e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:57.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.351+0000 7f7bb1d9b700 1 -- 172.21.15.16:0/2792213085 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb4071b90 con 0x7f7bb409eca0 2023-09-30T22:31:57.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.351+0000 7f7bb259c700 1 --2- 172.21.15.16:0/2792213085 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7bb406e570 0x7f7bb406e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:57.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.351+0000 7f7bb1d9b700 1 --2- 172.21.15.16:0/2792213085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bb409eca0 0x7f7bb406ee80 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f7ba80098f0 tx=0x7f7ba8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40cd14a7acdaf7fc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:57.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.351+0000 7f7bb159a700 1 -- 172.21.15.16:0/2792213085 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba8011830 con 0x7f7bb409eca0 2023-09-30T22:31:57.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.351+0000 7f7bb159a700 1 -- 172.21.15.16:0/2792213085 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba8011990 con 0x7f7bb409eca0 2023-09-30T22:31:57.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.351+0000 7f7bb159a700 1 -- 172.21.15.16:0/2792213085 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba8011d20 con 0x7f7bb409eca0 2023-09-30T22:31:57.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.352+0000 7f7bb8caf700 1 -- 172.21.15.16:0/2792213085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bb409eca0 msgr2=0x7f7bb406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:57.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.352+0000 7f7bb8caf700 1 --2- 172.21.15.16:0/2792213085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bb409eca0 0x7f7bb406ee80 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f7ba80098f0 tx=0x7f7ba8009c00 comp rx=0 tx=0).stop 2023-09-30T22:31:57.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.352+0000 7f7bb8caf700 1 -- 172.21.15.16:0/2792213085 shutdown_connections 2023-09-30T22:31:57.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.352+0000 7f7bb8caf700 1 --2- 172.21.15.16:0/2792213085 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7bb406e570 0x7f7bb406e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:57.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.352+0000 7f7bb8caf700 1 --2- 172.21.15.16:0/2792213085 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7bb4134610 0x7f7bb4134a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:57.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.352+0000 7f7bb8caf700 1 --2- 172.21.15.16:0/2792213085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bb409eca0 0x7f7bb406ee80 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:57.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.352+0000 7f7bb8caf700 1 -- 172.21.15.16:0/2792213085 >> 172.21.15.16:0/2792213085 conn(0x7f7bb40a2aa0 msgr2=0x7f7bb40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:57.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.352+0000 7f7bb8caf700 1 -- 172.21.15.16:0/2792213085 shutdown_connections 2023-09-30T22:31:57.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.352+0000 7f7bb8caf700 1 -- 172.21.15.16:0/2792213085 wait complete. 2023-09-30T22:31:57.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.353+0000 7f7bb8caf700 1 Processor -- start 2023-09-30T22:31:57.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.353+0000 7f7bb8caf700 1 -- start start 2023-09-30T22:31:57.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.354+0000 7f7bb8caf700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7bb406e570 0x7f7bb4127520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:57.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.354+0000 7f7bb8caf700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7bb4134610 0x7f7bb409bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:57.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.354+0000 7f7bb8caf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bb409c1a0 0x7f7bb409c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:57.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.354+0000 7f7bb8caf700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb4139080 con 0x7f7bb409c1a0 2023-09-30T22:31:57.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.354+0000 7f7bb8caf700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb4139740 con 0x7f7bb406e570 2023-09-30T22:31:57.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.354+0000 7f7bb8caf700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb40a1e90 con 0x7f7bb4134610 2023-09-30T22:31:57.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.354+0000 7f7bb1d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7bb4134610 0x7f7bb409bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:57.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.354+0000 7f7bb259c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7bb406e570 0x7f7bb4127520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:57.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.354+0000 7f7bb1d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7bb4134610 0x7f7bb409bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:60516/0 (socket says 172.21.15.16:60516) 2023-09-30T22:31:57.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.354+0000 7f7bb1d9b700 1 -- 172.21.15.16:0/3567719457 learned_addr learned my addr 172.21.15.16:0/3567719457 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:57.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.354+0000 7f7bb2d9d700 1 --2- 172.21.15.16:0/3567719457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bb409c1a0 0x7f7bb409c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:57.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.354+0000 7f7bb259c700 1 -- 172.21.15.16:0/3567719457 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7bb4134610 msgr2=0x7f7bb409bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:57.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.354+0000 7f7bb259c700 1 --2- 172.21.15.16:0/3567719457 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7bb4134610 0x7f7bb409bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:57.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.354+0000 7f7bb259c700 1 -- 172.21.15.16:0/3567719457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bb409c1a0 msgr2=0x7f7bb409c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:57.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.354+0000 7f7bb259c700 1 --2- 172.21.15.16:0/3567719457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bb409c1a0 0x7f7bb409c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:57.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.354+0000 7f7bb259c700 1 -- 172.21.15.16:0/3567719457 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba40094a0 con 0x7f7bb406e570 2023-09-30T22:31:57.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.354+0000 7f7bb2d9d700 1 --2- 172.21.15.16:0/3567719457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bb409c1a0 0x7f7bb409c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:57.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.355+0000 7f7bb259c700 1 --2- 172.21.15.16:0/3567719457 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7bb406e570 0x7f7bb4127520 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7ba400e9b0 tx=0x7f7ba400ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:57.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.355+0000 7f7ba2ffd700 1 -- 172.21.15.16:0/3567719457 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba400ee50 con 0x7f7bb406e570 2023-09-30T22:31:57.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.355+0000 7f7ba2ffd700 1 -- 172.21.15.16:0/3567719457 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba400c850 con 0x7f7bb406e570 2023-09-30T22:31:57.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.355+0000 7f7bb8caf700 1 -- 172.21.15.16:0/3567719457 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ba8009550 con 0x7f7bb406e570 2023-09-30T22:31:57.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.355+0000 7f7ba2ffd700 1 -- 172.21.15.16:0/3567719457 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba400cc00 con 0x7f7bb406e570 2023-09-30T22:31:57.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.355+0000 7f7bb8caf700 1 -- 172.21.15.16:0/3567719457 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bb4070660 con 0x7f7bb406e570 2023-09-30T22:31:57.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.356+0000 7f7ba2ffd700 1 -- 172.21.15.16:0/3567719457 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7ba4013470 con 0x7f7bb406e570 2023-09-30T22:31:57.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.356+0000 7f7ba2ffd700 1 -- 172.21.15.16:0/3567719457 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f7ba403f6e0 con 0x7f7bb406e570 2023-09-30T22:31:57.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.356+0000 7f7bb8caf700 1 -- 172.21.15.16:0/3567719457 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bb4074ea0 con 0x7f7bb406e570 2023-09-30T22:31:57.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.365+0000 7f7ba2ffd700 1 -- 172.21.15.16:0/3567719457 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7ba4021080 con 0x7f7bb406e570 2023-09-30T22:31:57.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.386+0000 7f8e231ea700 1 Processor -- start 2023-09-30T22:31:57.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.386+0000 7f8e231ea700 1 -- start start 2023-09-30T22:31:57.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.386+0000 7f8e231ea700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8e1c06a770 0x7f8e1c06ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:57.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.386+0000 7f8e231ea700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8e1c06b080 0x7f8e1c131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:57.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.386+0000 7f8e231ea700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8e1c131b40 0x7f8e1c131fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:57.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.386+0000 7f8e231ea700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e1c09ece0 con 0x7f8e1c131b40 2023-09-30T22:31:57.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.386+0000 7f8e231ea700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1c09e760 con 0x7f8e1c06b080 2023-09-30T22:31:57.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.386+0000 7f8e231ea700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1c09e040 con 0x7f8e1c06a770 2023-09-30T22:31:57.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.387+0000 7f8e20f86700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8e1c06a770 0x7f8e1c06ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:57.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.387+0000 7f8e20f86700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8e1c06a770 0x7f8e1c06ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:60520/0 (socket says 172.21.15.16:60520) 2023-09-30T22:31:57.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.387+0000 7f8e20f86700 1 -- 172.21.15.16:0/4267074715 learned_addr learned my addr 172.21.15.16:0/4267074715 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:57.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.387+0000 7f8e21787700 1 --2- 172.21.15.16:0/4267074715 >> v2:172.21.15.16:3300/0 conn(0x7f8e1c131b40 0x7f8e1c131fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:57.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.387+0000 7f8e1bfff700 1 --2- 172.21.15.16:0/4267074715 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8e1c06b080 0x7f8e1c131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:57.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.387+0000 7f8e20f86700 1 -- 172.21.15.16:0/4267074715 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8e1c06b080 msgr2=0x7f8e1c131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:57.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.387+0000 7f8e20f86700 1 --2- 172.21.15.16:0/4267074715 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8e1c06b080 0x7f8e1c131600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:57.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.387+0000 7f8e20f86700 1 -- 172.21.15.16:0/4267074715 >> v2:172.21.15.16:3300/0 conn(0x7f8e1c131b40 msgr2=0x7f8e1c131fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:57.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.387+0000 7f8e20f86700 1 --2- 172.21.15.16:0/4267074715 >> v2:172.21.15.16:3300/0 conn(0x7f8e1c131b40 0x7f8e1c131fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:57.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.387+0000 7f8e20f86700 1 -- 172.21.15.16:0/4267074715 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e1c09f630 con 0x7f8e1c06a770 2023-09-30T22:31:57.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.387+0000 7f8e1bfff700 1 --2- 172.21.15.16:0/4267074715 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8e1c06b080 0x7f8e1c131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:31:57.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.387+0000 7f8e21787700 1 --2- 172.21.15.16:0/4267074715 >> v2:172.21.15.16:3300/0 conn(0x7f8e1c131b40 0x7f8e1c131fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:57.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.387+0000 7f8e20f86700 1 --2- 172.21.15.16:0/4267074715 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8e1c06a770 0x7f8e1c06ab40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8e100098f0 tx=0x7f8e10009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7accf98e62f5d4e6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:57.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.388+0000 7f8e1b7fe700 1 -- 172.21.15.16:0/4267074715 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e10011830 con 0x7f8e1c06a770 2023-09-30T22:31:57.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.388+0000 7f8e1b7fe700 1 -- 172.21.15.16:0/4267074715 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e10011990 con 0x7f8e1c06a770 2023-09-30T22:31:57.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.388+0000 7f8e1b7fe700 1 -- 172.21.15.16:0/4267074715 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e10011d20 con 0x7f8e1c06a770 2023-09-30T22:31:57.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.388+0000 7f8e231ea700 1 -- 172.21.15.16:0/4267074715 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8e1c06a770 msgr2=0x7f8e1c06ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:57.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.388+0000 7f8e231ea700 1 --2- 172.21.15.16:0/4267074715 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8e1c06a770 0x7f8e1c06ab40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8e100098f0 tx=0x7f8e10009c00 comp rx=0 tx=0).stop 2023-09-30T22:31:57.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.388+0000 7f2d849f3700 1 Processor -- start 2023-09-30T22:31:57.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.388+0000 7f2d849f3700 1 -- start start 2023-09-30T22:31:57.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.388+0000 7f8e231ea700 1 -- 172.21.15.16:0/4267074715 shutdown_connections 2023-09-30T22:31:57.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.388+0000 7f8e231ea700 1 --2- 172.21.15.16:0/4267074715 >> v2:172.21.15.16:3300/0 conn(0x7f8e1c131b40 0x7f8e1c131fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:57.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.388+0000 7f8e231ea700 1 --2- 172.21.15.16:0/4267074715 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8e1c06a770 0x7f8e1c06ab40 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:57.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.388+0000 7f8e231ea700 1 --2- 172.21.15.16:0/4267074715 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8e1c06b080 0x7f8e1c131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:57.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.388+0000 7f8e231ea700 1 -- 172.21.15.16:0/4267074715 >> 172.21.15.16:0/4267074715 conn(0x7f8e1c066f50 msgr2=0x7f8e1c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:57.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.389+0000 7f8e231ea700 1 -- 172.21.15.16:0/4267074715 shutdown_connections 2023-09-30T22:31:57.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.389+0000 7f8e231ea700 1 -- 172.21.15.16:0/4267074715 wait complete. 2023-09-30T22:31:57.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.389+0000 7f2d849f3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2d8009cb40 0x7f2d8006e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:57.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.389+0000 7f2d849f3700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2d8009d340 0x7f2d80131a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:57.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.389+0000 7f2d849f3700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d8009c6f0 con 0x7f2d8009cf10 2023-09-30T22:31:57.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.389+0000 7f2d849f3700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d8009e420 con 0x7f2d8009d340 2023-09-30T22:31:57.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.389+0000 7f2d849f3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d800723a0 con 0x7f2d8009cb40 2023-09-30T22:31:57.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.390+0000 7f2d7ed9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2d8009d340 0x7f2d80131a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:57.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.390+0000 7f2d7ed9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2d8009d340 0x7f2d80131a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54696/0 (socket says 172.21.15.16:54696) 2023-09-30T22:31:57.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.390+0000 7f2d7ed9d700 1 -- 172.21.15.16:0/392884429 learned_addr learned my addr 172.21.15.16:0/392884429 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:57.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.390+0000 7f2d7e59c700 1 --2- 172.21.15.16:0/392884429 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2d8009cb40 0x7f2d8006e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:57.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.390+0000 7f2d7ed9d700 1 -- 172.21.15.16:0/392884429 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2d8009cb40 msgr2=0x7f2d8006e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:57.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.390+0000 7f2d7ed9d700 1 --2- 172.21.15.16:0/392884429 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2d8009cb40 0x7f2d8006e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:57.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.390+0000 7f2d7ed9d700 1 -- 172.21.15.16:0/392884429 >> v1:172.21.15.16:6789/0 conn(0x7f2d8009cf10 legacy=0x7f2d80131370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:57.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.390+0000 7f2d7ed9d700 1 -- 172.21.15.16:0/392884429 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d8006e080 con 0x7f2d8009d340 2023-09-30T22:31:57.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.390+0000 7f8e231ea700 1 Processor -- start 2023-09-30T22:31:57.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.390+0000 7f8e231ea700 1 -- start start 2023-09-30T22:31:57.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.390+0000 7f2d7ed9d700 1 --2- 172.21.15.16:0/392884429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d8009d340 0x7f2d80131a80 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f2d78009e50 tx=0x7f2d7800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1b2f60af1a68a3d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:57.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.390+0000 7f2d7d59a700 1 -- 172.21.15.16:0/392884429 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7800e070 con 0x7f2d8009d340 2023-09-30T22:31:57.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.391+0000 7f2d7d59a700 1 -- 172.21.15.16:0/392884429 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d7800b910 con 0x7f2d8009d340 2023-09-30T22:31:57.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.391+0000 7f2d7d59a700 1 -- 172.21.15.16:0/392884429 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7800bcc0 con 0x7f2d8009d340 2023-09-30T22:31:57.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.391+0000 7f8e231ea700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8e1c06a770 0x7f8e1c126660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:57.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.391+0000 7f8e231ea700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8e1c06b080 0x7f8e1c126ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:57.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.391+0000 7f8e231ea700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e1c131b40 0x7f8e1c1cc480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:57.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.391+0000 7f8e231ea700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1c1297c0 con 0x7f8e1c131b40 2023-09-30T22:31:57.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.391+0000 7f8e231ea700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1c09a560 con 0x7f8e1c06b080 2023-09-30T22:31:57.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.391+0000 7f8e231ea700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1c071e60 con 0x7f8e1c06a770 2023-09-30T22:31:57.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.391+0000 7f8e20f86700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8e1c06a770 0x7f8e1c126660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:57.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.391+0000 7f8e20f86700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8e1c06a770 0x7f8e1c126660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:34222/0 (socket says 172.21.15.16:34222) 2023-09-30T22:31:57.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.391+0000 7f8e20f86700 1 -- 172.21.15.16:0/3893413634 learned_addr learned my addr 172.21.15.16:0/3893413634 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:57.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.391+0000 7f8e21787700 1 --2- 172.21.15.16:0/3893413634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e1c131b40 0x7f8e1c1cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:57.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.392+0000 7f2d849f3700 1 -- 172.21.15.16:0/392884429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d8009d340 msgr2=0x7f2d80131a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:57.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.392+0000 7f2d849f3700 1 --2- 172.21.15.16:0/392884429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d8009d340 0x7f2d80131a80 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f2d78009e50 tx=0x7f2d7800b330 comp rx=0 tx=0).stop 2023-09-30T22:31:57.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.392+0000 7f8e20f86700 1 -- 172.21.15.16:0/3893413634 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8e1c06b080 msgr2=0x7f8e1c126ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:57.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.392+0000 7f8e20f86700 1 --2- 172.21.15.16:0/3893413634 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8e1c06b080 0x7f8e1c126ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:57.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.392+0000 7f8e20f86700 1 -- 172.21.15.16:0/3893413634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e1c131b40 msgr2=0x7f8e1c1cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:57.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.392+0000 7f8e20f86700 1 --2- 172.21.15.16:0/3893413634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e1c131b40 0x7f8e1c1cc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:57.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.392+0000 7f8e20f86700 1 -- 172.21.15.16:0/3893413634 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e1400f050 con 0x7f8e1c06a770 2023-09-30T22:31:57.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.392+0000 7f8e20f86700 1 --2- 172.21.15.16:0/3893413634 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8e1c06a770 0x7f8e1c126660 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8e10000c00 tx=0x7f8e10004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:57.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.392+0000 7f2d849f3700 1 -- 172.21.15.16:0/392884429 shutdown_connections 2023-09-30T22:31:57.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.392+0000 7f2d849f3700 1 --2- 172.21.15.16:0/392884429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d8009d340 0x7f2d80131a80 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:57.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.392+0000 7f2d849f3700 1 --2- 172.21.15.16:0/392884429 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2d8009cb40 0x7f2d8006e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:57.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.392+0000 7f2d849f3700 1 -- 172.21.15.16:0/392884429 >> 172.21.15.16:0/392884429 conn(0x7f2d8009e7e0 msgr2=0x7f2d8009ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:57.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.392+0000 7f8e197fa700 1 -- 172.21.15.16:0/3893413634 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e10004380 con 0x7f8e1c06a770 2023-09-30T22:31:57.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.392+0000 7f2d849f3700 1 -- 172.21.15.16:0/392884429 shutdown_connections 2023-09-30T22:31:57.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.392+0000 7f8e197fa700 1 -- 172.21.15.16:0/3893413634 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e1001f070 con 0x7f8e1c06a770 2023-09-30T22:31:57.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.392+0000 7f8e197fa700 1 -- 172.21.15.16:0/3893413634 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e100036a0 con 0x7f8e1c06a770 2023-09-30T22:31:57.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.392+0000 7f8e231ea700 1 -- 172.21.15.16:0/3893413634 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e10009550 con 0x7f8e1c06a770 2023-09-30T22:31:57.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.392+0000 7f2d849f3700 1 -- 172.21.15.16:0/392884429 wait complete. 2023-09-30T22:31:57.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.393+0000 7f8e231ea700 1 -- 172.21.15.16:0/3893413634 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e1c12a130 con 0x7f8e1c06a770 2023-09-30T22:31:57.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.394+0000 7f8e197fa700 1 -- 172.21.15.16:0/3893413634 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8e10021030 con 0x7f8e1c06a770 2023-09-30T22:31:57.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.394+0000 7f8e197fa700 1 -- 172.21.15.16:0/3893413634 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f8e10040830 con 0x7f8e1c06a770 2023-09-30T22:31:57.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.394+0000 7f2d849f3700 1 Processor -- start 2023-09-30T22:31:57.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.394+0000 7f2d849f3700 1 -- start start 2023-09-30T22:31:57.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.394+0000 7f2d849f3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d8009cb40 0x7f2d80134400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:57.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.394+0000 7f2d849f3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2d8009cf10 0x7f2d80134940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:57.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.394+0000 7f2d849f3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2d8009d340 0x7f2d80134e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:57.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.394+0000 7f2d849f3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d80137b00 con 0x7f2d8009cb40 2023-09-30T22:31:57.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.394+0000 7f2d849f3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d801388e0 con 0x7f2d8009cf10 2023-09-30T22:31:57.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.394+0000 7f2d849f3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2d800725e0 con 0x7f2d8009d340 2023-09-30T22:31:57.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.394+0000 7f2d7e59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d8009cb40 0x7f2d80134400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:57.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.394+0000 7f2d7ed9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2d8009d340 0x7f2d80134e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:57.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.394+0000 7f2d7e59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d8009cb40 0x7f2d80134400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53912/0 (socket says 172.21.15.16:53912) 2023-09-30T22:31:57.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.394+0000 7f2d7ed9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2d8009d340 0x7f2d80134e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:34232/0 (socket says 172.21.15.16:34232) 2023-09-30T22:31:57.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.394+0000 7f2d7e59c700 1 -- 172.21.15.16:0/3730366940 learned_addr learned my addr 172.21.15.16:0/3730366940 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:57.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.395+0000 7f2d7dd9b700 1 --2- 172.21.15.16:0/3730366940 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2d8009cf10 0x7f2d80134940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:57.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.395+0000 7f8e231ea700 1 -- 172.21.15.16:0/3893413634 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e1c129620 con 0x7f8e1c06a770 2023-09-30T22:31:57.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.395+0000 7f2d7ed9d700 1 -- 172.21.15.16:0/3730366940 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2d8009cf10 msgr2=0x7f2d80134940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:57.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.395+0000 7f2d7ed9d700 1 --2- 172.21.15.16:0/3730366940 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2d8009cf10 0x7f2d80134940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:57.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.395+0000 7f2d7ed9d700 1 -- 172.21.15.16:0/3730366940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d8009cb40 msgr2=0x7f2d80134400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:57.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.395+0000 7f2d7ed9d700 1 --2- 172.21.15.16:0/3730366940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d8009cb40 0x7f2d80134400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:57.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.395+0000 7f2d7ed9d700 1 -- 172.21.15.16:0/3730366940 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d78009b30 con 0x7f2d8009d340 2023-09-30T22:31:57.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.395+0000 7f2d7e59c700 1 --2- 172.21.15.16:0/3730366940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d8009cb40 0x7f2d80134400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:57.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.395+0000 7f2d7ed9d700 1 --2- 172.21.15.16:0/3730366940 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2d8009d340 0x7f2d80134e80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2d78015000 tx=0x7f2d78004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:57.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.400+0000 7f2d6effd700 1 -- 172.21.15.16:0/3730366940 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d78003ea0 con 0x7f2d8009d340 2023-09-30T22:31:57.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.400+0000 7f2d6effd700 1 -- 172.21.15.16:0/3730366940 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d780048e0 con 0x7f2d8009d340 2023-09-30T22:31:57.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.400+0000 7f2d849f3700 1 -- 172.21.15.16:0/3730366940 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d80137760 con 0x7f2d8009d340 2023-09-30T22:31:57.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.400+0000 7f2d6effd700 1 -- 172.21.15.16:0/3730366940 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7801c680 con 0x7f2d8009d340 2023-09-30T22:31:57.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.400+0000 7f8e197fa700 1 -- 172.21.15.16:0/3893413634 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8e1c129620 con 0x7f8e1c06a770 2023-09-30T22:31:57.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.400+0000 7f2d849f3700 1 -- 172.21.15.16:0/3730366940 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d80138490 con 0x7f2d8009d340 2023-09-30T22:31:57.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.401+0000 7f2d849f3700 1 -- 172.21.15.16:0/3730366940 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d80138f30 con 0x7f2d8009d340 2023-09-30T22:31:57.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.402+0000 7f2d6effd700 1 -- 172.21.15.16:0/3730366940 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2d7801c7e0 con 0x7f2d8009d340 2023-09-30T22:31:57.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.402+0000 7f2d6effd700 1 -- 172.21.15.16:0/3730366940 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f2d78040dc0 con 0x7f2d8009d340 2023-09-30T22:31:57.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.411+0000 7f2d6effd700 1 -- 172.21.15.16:0/3730366940 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2d780249a0 con 0x7f2d8009d340 2023-09-30T22:31:57.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.491+0000 7fb2b2c5a700 1 -- 172.21.15.16:0/2146161112 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb2ac072980 con 0x7fb2ac0a0a30 2023-09-30T22:31:57.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.491+0000 7fb2a97fa700 1 -- 172.21.15.16:0/2146161112 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb2a0021070 con 0x7fb2ac0a0a30 2023-09-30T22:31:57.492 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:31:57.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.493+0000 7fb2b2c5a700 1 -- 172.21.15.16:0/2146161112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2ac0a0a30 msgr2=0x7fb2ac133f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:57.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.493+0000 7fb2b2c5a700 1 --2- 172.21.15.16:0/2146161112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2ac0a0a30 0x7fb2ac133f20 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fb2a000e840 tx=0x7fb2a000eb50 comp rx=0 tx=0).stop 2023-09-30T22:31:57.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.494+0000 7fb2b2c5a700 1 -- 172.21.15.16:0/2146161112 shutdown_connections 2023-09-30T22:31:57.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.494+0000 7fb2b2c5a700 1 --2- 172.21.15.16:0/2146161112 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb2ac09d630 0x7fb2ac1339e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:57.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.494+0000 7fb2b2c5a700 1 --2- 172.21.15.16:0/2146161112 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb2ac138d80 0x7fb2ac134460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:57.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.494+0000 7fb2b2c5a700 1 --2- 172.21.15.16:0/2146161112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2ac0a0a30 0x7fb2ac133f20 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:57.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.494+0000 7fb2b2c5a700 1 -- 172.21.15.16:0/2146161112 >> 172.21.15.16:0/2146161112 conn(0x7fb2ac0a3050 msgr2=0x7fb2ac09ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:57.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.494+0000 7fb2b2c5a700 1 -- 172.21.15.16:0/2146161112 shutdown_connections 2023-09-30T22:31:57.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.494+0000 7fb2b2c5a700 1 -- 172.21.15.16:0/2146161112 wait complete. 2023-09-30T22:31:57.509 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:31:57.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.580+0000 7f7bb8caf700 1 -- 172.21.15.16:0/3567719457 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f7bb4071100 con 0x7f7bb406e570 2023-09-30T22:31:57.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.581+0000 7f7ba2ffd700 1 -- 172.21.15.16:0/3567719457 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7ba4017020 con 0x7f7bb406e570 2023-09-30T22:31:57.581 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:31:57.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.583+0000 7f7bb8caf700 1 -- 172.21.15.16:0/3567719457 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7bb406e570 msgr2=0x7f7bb4127520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:57.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.583+0000 7f7bb8caf700 1 --2- 172.21.15.16:0/3567719457 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7bb406e570 0x7f7bb4127520 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7ba400e9b0 tx=0x7f7ba400ecc0 comp rx=0 tx=0).stop 2023-09-30T22:31:57.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.583+0000 7f7bb8caf700 1 -- 172.21.15.16:0/3567719457 shutdown_connections 2023-09-30T22:31:57.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.583+0000 7f7bb8caf700 1 --2- 172.21.15.16:0/3567719457 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7bb406e570 0x7f7bb4127520 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:57.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.583+0000 7f7bb8caf700 1 --2- 172.21.15.16:0/3567719457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7bb409c1a0 0x7f7bb409c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:57.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.583+0000 7f7bb8caf700 1 --2- 172.21.15.16:0/3567719457 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7bb4134610 0x7f7bb409bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:57.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.583+0000 7f7bb8caf700 1 -- 172.21.15.16:0/3567719457 >> 172.21.15.16:0/3567719457 conn(0x7f7bb40a2aa0 msgr2=0x7f7bb4136670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:57.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.583+0000 7f7bb8caf700 1 -- 172.21.15.16:0/3567719457 shutdown_connections 2023-09-30T22:31:57.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.583+0000 7f7bb8caf700 1 -- 172.21.15.16:0/3567719457 wait complete. 2023-09-30T22:31:57.597 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:31:57.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.614+0000 7f8e231ea700 1 -- 172.21.15.16:0/3893413634 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8e1c129620 con 0x7f8e1c06a770 2023-09-30T22:31:57.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.614+0000 7f8e197fa700 1 -- 172.21.15.16:0/3893413634 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8e1c129620 con 0x7f8e1c06a770 2023-09-30T22:31:57.615 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:31:57.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.617+0000 7f8e231ea700 1 -- 172.21.15.16:0/3893413634 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8e1c06a770 msgr2=0x7f8e1c126660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:57.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.617+0000 7f8e231ea700 1 --2- 172.21.15.16:0/3893413634 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8e1c06a770 0x7f8e1c126660 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8e10000c00 tx=0x7f8e10004750 comp rx=0 tx=0).stop 2023-09-30T22:31:57.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.617+0000 7f8e231ea700 1 -- 172.21.15.16:0/3893413634 shutdown_connections 2023-09-30T22:31:57.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.617+0000 7f8e231ea700 1 --2- 172.21.15.16:0/3893413634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e1c131b40 0x7f8e1c1cc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:57.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.617+0000 7f8e231ea700 1 --2- 172.21.15.16:0/3893413634 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8e1c06a770 0x7f8e1c126660 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:57.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.617+0000 7f8e231ea700 1 --2- 172.21.15.16:0/3893413634 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8e1c06b080 0x7f8e1c126ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:57.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.617+0000 7f8e231ea700 1 -- 172.21.15.16:0/3893413634 >> 172.21.15.16:0/3893413634 conn(0x7f8e1c066f50 msgr2=0x7f8e1c0a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:57.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.617+0000 7f8e231ea700 1 -- 172.21.15.16:0/3893413634 shutdown_connections 2023-09-30T22:31:57.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.617+0000 7f8e231ea700 1 -- 172.21.15.16:0/3893413634 wait complete. 2023-09-30T22:31:57.631 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:31:57.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.639+0000 7f2d849f3700 1 -- 172.21.15.16:0/3730366940 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f2d800747f0 con 0x7f2d8009d340 2023-09-30T22:31:57.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.640+0000 7f2d6effd700 1 -- 172.21.15.16:0/3730366940 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2d78013e80 con 0x7f2d8009d340 2023-09-30T22:31:57.640 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:31:57.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.642+0000 7f2d849f3700 1 -- 172.21.15.16:0/3730366940 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2d8009d340 msgr2=0x7f2d80134e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:57.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.642+0000 7f2d849f3700 1 --2- 172.21.15.16:0/3730366940 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2d8009d340 0x7f2d80134e80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f2d78015000 tx=0x7f2d78004ad0 comp rx=0 tx=0).stop 2023-09-30T22:31:57.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.642+0000 7f2d849f3700 1 -- 172.21.15.16:0/3730366940 shutdown_connections 2023-09-30T22:31:57.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.642+0000 7f2d849f3700 1 --2- 172.21.15.16:0/3730366940 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2d8009d340 0x7f2d80134e80 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:57.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.642+0000 7f2d849f3700 1 --2- 172.21.15.16:0/3730366940 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d8009cb40 0x7f2d80134400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:57.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.642+0000 7f2d849f3700 1 --2- 172.21.15.16:0/3730366940 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2d8009cf10 0x7f2d80134940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:57.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.642+0000 7f2d849f3700 1 -- 172.21.15.16:0/3730366940 >> 172.21.15.16:0/3730366940 conn(0x7f2d8009e7e0 msgr2=0x7f2d8009a150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:57.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.642+0000 7f2d849f3700 1 -- 172.21.15.16:0/3730366940 shutdown_connections 2023-09-30T22:31:57.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:57.642+0000 7f2d849f3700 1 -- 172.21.15.16:0/3730366940 wait complete. 2023-09-30T22:31:57.656 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:31:58.509 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:31:58.598 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:31:58.632 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:31:58.657 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:31:58.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.657+0000 7fbb11ae0700 1 Processor -- start 2023-09-30T22:31:58.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.657+0000 7fbb11ae0700 1 -- start start 2023-09-30T22:31:58.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.658+0000 7fbb11ae0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbb0c130540 0x7fbb0c130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:58.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.658+0000 7fbb11ae0700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbb0c130e50 0x7fbb0c06f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:58.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.658+0000 7fbb11ae0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbb0c06fb20 0x7fbb0c06ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:58.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.658+0000 7fbb11ae0700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb0c0704b0 con 0x7fbb0c130e50 2023-09-30T22:31:58.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.658+0000 7fbb11ae0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0c09a390 con 0x7fbb0c06fb20 2023-09-30T22:31:58.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.658+0000 7fbb11ae0700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0c0716a0 con 0x7fbb0c130540 2023-09-30T22:31:58.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.658+0000 7fbb0b7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbb0c130540 0x7fbb0c130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:58.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.658+0000 7fbb0b7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbb0c130540 0x7fbb0c130910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:34244/0 (socket says 172.21.15.16:34244) 2023-09-30T22:31:58.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.658+0000 7fbb0affd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbb0c130e50 0x7fbb0c06f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:58.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.658+0000 7fbb0b7fe700 1 -- 172.21.15.16:0/338480302 learned_addr learned my addr 172.21.15.16:0/338480302 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:58.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.658+0000 7fbb0bfff700 1 --2- 172.21.15.16:0/338480302 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbb0c06fb20 0x7fbb0c06ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:58.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.658+0000 7fbb0b7fe700 1 -- 172.21.15.16:0/338480302 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbb0c06fb20 msgr2=0x7fbb0c06ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:58.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.658+0000 7fbb0b7fe700 1 --2- 172.21.15.16:0/338480302 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbb0c06fb20 0x7fbb0c06ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:58.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.658+0000 7fbb0b7fe700 1 -- 172.21.15.16:0/338480302 >> v2:172.21.15.16:3300/0 conn(0x7fbb0c130e50 msgr2=0x7fbb0c06f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:58.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.658+0000 7fbb0b7fe700 1 --2- 172.21.15.16:0/338480302 >> v2:172.21.15.16:3300/0 conn(0x7fbb0c130e50 0x7fbb0c06f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:58.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.658+0000 7fbb0b7fe700 1 -- 172.21.15.16:0/338480302 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb0c071290 con 0x7fbb0c130540 2023-09-30T22:31:58.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.658+0000 7fbb0affd700 1 --2- 172.21.15.16:0/338480302 >> v2:172.21.15.16:3300/0 conn(0x7fbb0c130e50 0x7fbb0c06f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:58.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.659+0000 7fbb0b7fe700 1 --2- 172.21.15.16:0/338480302 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbb0c130540 0x7fbb0c130910 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fbafc00b9d0 tx=0x7fbafc00bce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c17e7c86284be7b3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:58.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.659+0000 7fbb0a7fc700 1 -- 172.21.15.16:0/338480302 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbafc00cc20 con 0x7fbb0c130540 2023-09-30T22:31:58.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.659+0000 7fbb0a7fc700 1 -- 172.21.15.16:0/338480302 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbafc00cd80 con 0x7fbb0c130540 2023-09-30T22:31:58.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.659+0000 7fbb0a7fc700 1 -- 172.21.15.16:0/338480302 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbafc00f680 con 0x7fbb0c130540 2023-09-30T22:31:58.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.660+0000 7fbb11ae0700 1 -- 172.21.15.16:0/338480302 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbb0c130540 msgr2=0x7fbb0c130910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:58.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.660+0000 7fbb11ae0700 1 --2- 172.21.15.16:0/338480302 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbb0c130540 0x7fbb0c130910 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fbafc00b9d0 tx=0x7fbafc00bce0 comp rx=0 tx=0).stop 2023-09-30T22:31:58.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.660+0000 7fbb11ae0700 1 -- 172.21.15.16:0/338480302 shutdown_connections 2023-09-30T22:31:58.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.660+0000 7fbb11ae0700 1 --2- 172.21.15.16:0/338480302 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbb0c06fb20 0x7fbb0c06ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:58.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.660+0000 7fbb11ae0700 1 --2- 172.21.15.16:0/338480302 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbb0c130540 0x7fbb0c130910 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:58.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.660+0000 7fbb11ae0700 1 --2- 172.21.15.16:0/338480302 >> v2:172.21.15.16:3300/0 conn(0x7fbb0c130e50 0x7fbb0c06f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:58.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.660+0000 7fbb11ae0700 1 -- 172.21.15.16:0/338480302 >> 172.21.15.16:0/338480302 conn(0x7fbb0c0a2aa0 msgr2=0x7fbb0c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:58.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.660+0000 7fbb11ae0700 1 -- 172.21.15.16:0/338480302 shutdown_connections 2023-09-30T22:31:58.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.660+0000 7fbb11ae0700 1 -- 172.21.15.16:0/338480302 wait complete. 2023-09-30T22:31:58.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.661+0000 7fbb11ae0700 1 Processor -- start 2023-09-30T22:31:58.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.662+0000 7fbb11ae0700 1 -- start start 2023-09-30T22:31:58.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.662+0000 7fbb11ae0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb0c06fb20 0x7fbb0c118cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:58.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.662+0000 7fbb11ae0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbb0c130540 0x7fbb0c119210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:58.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.663+0000 7fbb11ae0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbb0c130e50 0x7fbb0c119750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:58.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.663+0000 7fbb11ae0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0c0a0d50 con 0x7fbb0c06fb20 2023-09-30T22:31:58.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.663+0000 7fbb0b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb0c06fb20 0x7fbb0c118cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:58.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.663+0000 7fbb11ae0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0c11b990 con 0x7fbb0c130e50 2023-09-30T22:31:58.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.663+0000 7fbb11ae0700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0c11a130 con 0x7fbb0c130540 2023-09-30T22:31:58.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.663+0000 7fbb0b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb0c06fb20 0x7fbb0c118cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53938/0 (socket says 172.21.15.16:53938) 2023-09-30T22:31:58.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.663+0000 7fbb0b7fe700 1 -- 172.21.15.16:0/2897905197 learned_addr learned my addr 172.21.15.16:0/2897905197 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:58.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.663+0000 7fbb0b7fe700 1 -- 172.21.15.16:0/2897905197 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbb0c130540 msgr2=0x7fbb0c119210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:58.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.663+0000 7fbb0b7fe700 1 --2- 172.21.15.16:0/2897905197 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbb0c130540 0x7fbb0c119210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:58.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.663+0000 7fbb0b7fe700 1 -- 172.21.15.16:0/2897905197 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbb0c130e50 msgr2=0x7fbb0c119750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:58.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.663+0000 7fbb0b7fe700 1 --2- 172.21.15.16:0/2897905197 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbb0c130e50 0x7fbb0c119750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:58.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.663+0000 7fbb0b7fe700 1 -- 172.21.15.16:0/2897905197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb0000d940 con 0x7fbb0c06fb20 2023-09-30T22:31:58.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.663+0000 7fbb0b7fe700 1 --2- 172.21.15.16:0/2897905197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb0c06fb20 0x7fbb0c118cd0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fbafc000c00 tx=0x7fbafc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:58.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.664+0000 7fbaf3fff700 1 -- 172.21.15.16:0/2897905197 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbafc01c070 con 0x7fbb0c06fb20 2023-09-30T22:31:58.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.664+0000 7fbaf3fff700 1 -- 172.21.15.16:0/2897905197 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbafc00be70 con 0x7fbb0c06fb20 2023-09-30T22:31:58.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.664+0000 7fbaf3fff700 1 -- 172.21.15.16:0/2897905197 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbafc022420 con 0x7fbb0c06fb20 2023-09-30T22:31:58.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.664+0000 7fbb11ae0700 1 -- 172.21.15.16:0/2897905197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbafc00b600 con 0x7fbb0c06fb20 2023-09-30T22:31:58.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.664+0000 7fbb11ae0700 1 -- 172.21.15.16:0/2897905197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb0c11b2e0 con 0x7fbb0c06fb20 2023-09-30T22:31:58.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.665+0000 7fbaf3fff700 1 -- 172.21.15.16:0/2897905197 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbafc020070 con 0x7fbb0c06fb20 2023-09-30T22:31:58.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.665+0000 7fbb11ae0700 1 -- 172.21.15.16:0/2897905197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb0c11aee0 con 0x7fbb0c06fb20 2023-09-30T22:31:58.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.665+0000 7fbaf3fff700 1 -- 172.21.15.16:0/2897905197 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fbafc040800 con 0x7fbb0c06fb20 2023-09-30T22:31:58.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.675+0000 7fbaf3fff700 1 -- 172.21.15.16:0/2897905197 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbb0c11aee0 con 0x7fbb0c06fb20 2023-09-30T22:31:58.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.746+0000 7fec41d2c700 1 Processor -- start 2023-09-30T22:31:58.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.746+0000 7fec41d2c700 1 -- start start 2023-09-30T22:31:58.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.747+0000 7fec41d2c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fec3c09eca0 0x7fec3c13cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:58.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.747+0000 7fec41d2c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fec3c06f050 0x7fec3c13f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:58.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.747+0000 7fec41d2c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec3c070ee0 con 0x7fec3c06e570 2023-09-30T22:31:58.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.747+0000 7fec41d2c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec3c071980 con 0x7fec3c06f050 2023-09-30T22:31:58.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.747+0000 7fec41d2c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fec3c071480 con 0x7fec3c09eca0 2023-09-30T22:31:58.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.747+0000 7fec3b7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fec3c06e570 0x7fec3c06e940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60746/0 (socket says 172.21.15.16:60746) 2023-09-30T22:31:58.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.747+0000 7fec3b7fe700 1 -- 172.21.15.16:0/2489639372 learned_addr learned my addr 172.21.15.16:0/2489639372 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:31:58.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.747+0000 7fec3bfff700 1 --2- 172.21.15.16:0/2489639372 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fec3c06f050 0x7fec3c13f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:58.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.747+0000 7fec3affd700 1 --2- 172.21.15.16:0/2489639372 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fec3c09eca0 0x7fec3c13cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:58.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.747+0000 7fec3affd700 1 -- 172.21.15.16:0/2489639372 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fec3c06f050 msgr2=0x7fec3c13f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:58.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.747+0000 7fec3affd700 1 --2- 172.21.15.16:0/2489639372 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fec3c06f050 0x7fec3c13f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:58.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.747+0000 7fec3affd700 1 -- 172.21.15.16:0/2489639372 >> v1:172.21.15.16:6789/0 conn(0x7fec3c06e570 legacy=0x7fec3c06e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:58.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.747+0000 7fec3affd700 1 -- 172.21.15.16:0/2489639372 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec3c071b90 con 0x7fec3c09eca0 2023-09-30T22:31:58.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.748+0000 7fec3affd700 1 --2- 172.21.15.16:0/2489639372 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fec3c09eca0 0x7fec3c13cea0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fec30009870 tx=0x7fec30009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=293528319cefa060 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:58.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.748+0000 7fec3a7fc700 1 -- 172.21.15.16:0/2489639372 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec30011830 con 0x7fec3c09eca0 2023-09-30T22:31:58.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.748+0000 7fec3a7fc700 1 -- 172.21.15.16:0/2489639372 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec30011990 con 0x7fec3c09eca0 2023-09-30T22:31:58.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.748+0000 7fec3a7fc700 1 -- 172.21.15.16:0/2489639372 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec30011d20 con 0x7fec3c09eca0 2023-09-30T22:31:58.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.749+0000 7fec41d2c700 1 -- 172.21.15.16:0/2489639372 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fec3c09eca0 msgr2=0x7fec3c13cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:58.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.749+0000 7fec41d2c700 1 --2- 172.21.15.16:0/2489639372 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fec3c09eca0 0x7fec3c13cea0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fec30009870 tx=0x7fec30009b80 comp rx=0 tx=0).stop 2023-09-30T22:31:58.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.749+0000 7fec41d2c700 1 -- 172.21.15.16:0/2489639372 shutdown_connections 2023-09-30T22:31:58.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.749+0000 7fec41d2c700 1 --2- 172.21.15.16:0/2489639372 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fec3c06f050 0x7fec3c13f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:58.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.749+0000 7fec41d2c700 1 --2- 172.21.15.16:0/2489639372 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fec3c09eca0 0x7fec3c13cea0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:58.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.749+0000 7fec41d2c700 1 -- 172.21.15.16:0/2489639372 >> 172.21.15.16:0/2489639372 conn(0x7fec3c0a2aa0 msgr2=0x7fec3c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:58.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.749+0000 7fec41d2c700 1 -- 172.21.15.16:0/2489639372 shutdown_connections 2023-09-30T22:31:58.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.749+0000 7fec41d2c700 1 -- 172.21.15.16:0/2489639372 wait complete. 2023-09-30T22:31:58.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.750+0000 7fec41d2c700 1 Processor -- start 2023-09-30T22:31:58.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.750+0000 7fec41d2c700 1 -- start start 2023-09-30T22:31:58.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.751+0000 7fec41d2c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec3c06e570 0x7fec3c1231a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:58.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.751+0000 7fec41d2c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fec3c06f050 0x7fec3c1236e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:58.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.751+0000 7fec41d2c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fec3c123c20 0x7fec3c1d2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:58.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.751+0000 7fec41d2c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec3c126d60 con 0x7fec3c06e570 2023-09-30T22:31:58.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.751+0000 7fec41d2c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec3c09a380 con 0x7fec3c06f050 2023-09-30T22:31:58.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.751+0000 7fec41d2c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fec3c070610 con 0x7fec3c123c20 2023-09-30T22:31:58.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.751+0000 7fec3b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec3c06e570 0x7fec3c1231a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:58.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.751+0000 7fec3b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec3c06e570 0x7fec3c1231a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53940/0 (socket says 172.21.15.16:53940) 2023-09-30T22:31:58.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.751+0000 7fec3b7fe700 1 -- 172.21.15.16:0/2734706539 learned_addr learned my addr 172.21.15.16:0/2734706539 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:58.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.751+0000 7fec3affd700 1 --2- 172.21.15.16:0/2734706539 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fec3c06f050 0x7fec3c1236e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:58.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.751+0000 7fec3bfff700 1 --2- 172.21.15.16:0/2734706539 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fec3c123c20 0x7fec3c1d2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:58.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.752+0000 7fec3affd700 1 -- 172.21.15.16:0/2734706539 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fec3c123c20 msgr2=0x7fec3c1d2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:58.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.752+0000 7fec3affd700 1 --2- 172.21.15.16:0/2734706539 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fec3c123c20 0x7fec3c1d2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:58.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.752+0000 7fec3affd700 1 -- 172.21.15.16:0/2734706539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec3c06e570 msgr2=0x7fec3c1231a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:58.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.752+0000 7fec3affd700 1 --2- 172.21.15.16:0/2734706539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec3c06e570 0x7fec3c1231a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:58.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.752+0000 7fec3affd700 1 -- 172.21.15.16:0/2734706539 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec300094a0 con 0x7fec3c06f050 2023-09-30T22:31:58.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.752+0000 7fec3b7fe700 1 --2- 172.21.15.16:0/2734706539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec3c06e570 0x7fec3c1231a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:58.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.752+0000 7fec3affd700 1 --2- 172.21.15.16:0/2734706539 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fec3c06f050 0x7fec3c1236e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fec30016000 tx=0x7fec30004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:58.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.752+0000 7fec23fff700 1 -- 172.21.15.16:0/2734706539 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec30004450 con 0x7fec3c06f050 2023-09-30T22:31:58.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.752+0000 7fec41d2c700 1 -- 172.21.15.16:0/2734706539 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec3c126450 con 0x7fec3c06f050 2023-09-30T22:31:58.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.752+0000 7fec41d2c700 1 -- 172.21.15.16:0/2734706539 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec3c125560 con 0x7fec3c06f050 2023-09-30T22:31:58.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.752+0000 7fec23fff700 1 -- 172.21.15.16:0/2734706539 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec300036a0 con 0x7fec3c06f050 2023-09-30T22:31:58.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.752+0000 7fec23fff700 1 -- 172.21.15.16:0/2734706539 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec300045f0 con 0x7fec3c06f050 2023-09-30T22:31:58.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.753+0000 7fec23fff700 1 -- 172.21.15.16:0/2734706539 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fec30003a50 con 0x7fec3c06f050 2023-09-30T22:31:58.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.754+0000 7fec41d2c700 1 -- 172.21.15.16:0/2734706539 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec1c0052e0 con 0x7fec3c06f050 2023-09-30T22:31:58.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.754+0000 7fec23fff700 1 -- 172.21.15.16:0/2734706539 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fec30040a10 con 0x7fec3c06f050 2023-09-30T22:31:58.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.759+0000 7fec23fff700 1 -- 172.21.15.16:0/2734706539 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fec3002c350 con 0x7fec3c06f050 2023-09-30T22:31:58.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.798+0000 7fee96239700 1 Processor -- start 2023-09-30T22:31:58.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.798+0000 7fee96239700 1 -- start start 2023-09-30T22:31:58.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.799+0000 7fee96239700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fee901337f0 0x7fee90138880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:58.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.799+0000 7fee96239700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fee90138dc0 0x7fee9013b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:58.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.799+0000 7fee96239700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee9006b990 con 0x7fee90133420 2023-09-30T22:31:58.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.799+0000 7fee96239700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee9009b0e0 con 0x7fee90138dc0 2023-09-30T22:31:58.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.799+0000 7fee96239700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fee90133c20 con 0x7fee901337f0 2023-09-30T22:31:58.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.799+0000 7fee8f7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fee90133420 0x7fee90133e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60752/0 (socket says 172.21.15.16:60752) 2023-09-30T22:31:58.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.799+0000 7fee8ffff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fee90138dc0 0x7fee9013b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:58.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.800+0000 7fee8f7fe700 1 -- 172.21.15.16:0/2948031454 learned_addr learned my addr 172.21.15.16:0/2948031454 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:31:58.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.800+0000 7fee8effd700 1 -- 172.21.15.16:0/2948031454 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1427541249 0 0) 0x7fee9006b990 con 0x7fee90133420 2023-09-30T22:31:58.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.800+0000 7fee8effd700 1 -- 172.21.15.16:0/2948031454 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee7c0035b0 con 0x7fee90133420 2023-09-30T22:31:58.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.800+0000 7fee8ffff700 1 -- 172.21.15.16:0/2948031454 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fee901337f0 msgr2=0x7fee90138880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:58.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.800+0000 7fee8ffff700 1 --2- 172.21.15.16:0/2948031454 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fee901337f0 0x7fee90138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:58.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.800+0000 7fee8ffff700 1 -- 172.21.15.16:0/2948031454 >> v1:172.21.15.16:6789/0 conn(0x7fee90133420 legacy=0x7fee90133e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:58.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.800+0000 7fee8ffff700 1 -- 172.21.15.16:0/2948031454 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee900a0590 con 0x7fee90138dc0 2023-09-30T22:31:58.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.800+0000 7fee8ffff700 1 --2- 172.21.15.16:0/2948031454 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fee90138dc0 0x7fee9013b1a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fee88009dc0 tx=0x7fee8800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fac906375c50967 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:58.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.801+0000 7fee8effd700 1 -- 172.21.15.16:0/2948031454 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8800e070 con 0x7fee90138dc0 2023-09-30T22:31:58.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.801+0000 7fee8effd700 1 -- 172.21.15.16:0/2948031454 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee8800bb20 con 0x7fee90138dc0 2023-09-30T22:31:58.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.801+0000 7fee8effd700 1 -- 172.21.15.16:0/2948031454 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8800be90 con 0x7fee90138dc0 2023-09-30T22:31:58.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.801+0000 7fee96239700 1 -- 172.21.15.16:0/2948031454 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fee90138dc0 msgr2=0x7fee9013b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:58.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.801+0000 7fee96239700 1 --2- 172.21.15.16:0/2948031454 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fee90138dc0 0x7fee9013b1a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fee88009dc0 tx=0x7fee8800b330 comp rx=0 tx=0).stop 2023-09-30T22:31:58.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.802+0000 7fee96239700 1 -- 172.21.15.16:0/2948031454 shutdown_connections 2023-09-30T22:31:58.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.802+0000 7fee96239700 1 --2- 172.21.15.16:0/2948031454 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fee90138dc0 0x7fee9013b1a0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:58.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.802+0000 7fee96239700 1 --2- 172.21.15.16:0/2948031454 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fee901337f0 0x7fee90138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:58.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.802+0000 7fee96239700 1 -- 172.21.15.16:0/2948031454 >> 172.21.15.16:0/2948031454 conn(0x7fee900a4210 msgr2=0x7fee900a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:58.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.802+0000 7fee96239700 1 -- 172.21.15.16:0/2948031454 shutdown_connections 2023-09-30T22:31:58.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.802+0000 7fee96239700 1 -- 172.21.15.16:0/2948031454 wait complete. 2023-09-30T22:31:58.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.803+0000 7fee96239700 1 Processor -- start 2023-09-30T22:31:58.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.803+0000 7fee96239700 1 -- start start 2023-09-30T22:31:58.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.804+0000 7fee96239700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fee90133420 0x7fee900709f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:58.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.804+0000 7fee96239700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fee901337f0 0x7fee90070f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:58.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.804+0000 7fee96239700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee90138dc0 0x7fee90071470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:58.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.804+0000 7fee96239700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee90072600 con 0x7fee90138dc0 2023-09-30T22:31:58.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.804+0000 7fee96239700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee9009b690 con 0x7fee901337f0 2023-09-30T22:31:58.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.804+0000 7fee96239700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fee9009db60 con 0x7fee90133420 2023-09-30T22:31:58.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.804+0000 7fee87fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fee901337f0 0x7fee90070f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:58.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.804+0000 7fee8f7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fee90133420 0x7fee900709f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:58.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.804+0000 7fee87fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fee901337f0 0x7fee90070f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:44140/0 (socket says 172.21.15.16:44140) 2023-09-30T22:31:58.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.804+0000 7fee87fff700 1 -- 172.21.15.16:0/803043599 learned_addr learned my addr 172.21.15.16:0/803043599 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:58.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.804+0000 7fee8ffff700 1 --2- 172.21.15.16:0/803043599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee90138dc0 0x7fee90071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:58.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.805+0000 7fee8f7fe700 1 -- 172.21.15.16:0/803043599 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fee901337f0 msgr2=0x7fee90070f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:58.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.805+0000 7fee8f7fe700 1 --2- 172.21.15.16:0/803043599 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fee901337f0 0x7fee90070f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:58.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.805+0000 7fee8f7fe700 1 -- 172.21.15.16:0/803043599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee90138dc0 msgr2=0x7fee90071470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:58.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.805+0000 7fee8f7fe700 1 --2- 172.21.15.16:0/803043599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee90138dc0 0x7fee90071470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:58.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.805+0000 7fee8f7fe700 1 -- 172.21.15.16:0/803043599 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee88009a70 con 0x7fee90133420 2023-09-30T22:31:58.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.805+0000 7fee8ffff700 1 --2- 172.21.15.16:0/803043599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee90138dc0 0x7fee90071470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:31:58.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.805+0000 7fee8f7fe700 1 --2- 172.21.15.16:0/803043599 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fee90133420 0x7fee900709f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fee8000c490 tx=0x7fee8000c7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:58.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.805+0000 7fee8cff9700 1 -- 172.21.15.16:0/803043599 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee80005d70 con 0x7fee90133420 2023-09-30T22:31:58.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.805+0000 7fee8cff9700 1 -- 172.21.15.16:0/803043599 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee800070d0 con 0x7fee90133420 2023-09-30T22:31:58.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.805+0000 7fee96239700 1 -- 172.21.15.16:0/803043599 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee90133050 con 0x7fee90133420 2023-09-30T22:31:58.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.805+0000 7fee8cff9700 1 -- 172.21.15.16:0/803043599 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee80007440 con 0x7fee90133420 2023-09-30T22:31:58.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.805+0000 7fee96239700 1 -- 172.21.15.16:0/803043599 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee9009d710 con 0x7fee90133420 2023-09-30T22:31:58.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.806+0000 7fee8cff9700 1 -- 172.21.15.16:0/803043599 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fee800075a0 con 0x7fee90133420 2023-09-30T22:31:58.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.806+0000 7fee8cff9700 1 -- 172.21.15.16:0/803043599 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fee8003eae0 con 0x7fee90133420 2023-09-30T22:31:58.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.806+0000 7fee96239700 1 -- 172.21.15.16:0/803043599 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee9009cc00 con 0x7fee90133420 2023-09-30T22:31:58.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.807+0000 7f9faf39a700 1 Processor -- start 2023-09-30T22:31:58.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.807+0000 7f9faf39a700 1 -- start start 2023-09-30T22:31:58.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.808+0000 7f9faf39a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9fa81338a0 0x7f9fa8133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:58.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.808+0000 7f9faf39a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9fa809e930 0x7f9fa813b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:58.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.808+0000 7f9faf39a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fa806ef30 con 0x7f9fa809de50 2023-09-30T22:31:58.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.808+0000 7f9faf39a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa806fef0 con 0x7f9fa81338a0 2023-09-30T22:31:58.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.808+0000 7f9faf39a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa81334b0 con 0x7f9fa809e930 2023-09-30T22:31:58.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.808+0000 7f9fad136700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9fa81338a0 0x7f9fa8133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:58.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.808+0000 7f9fad937700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9fa809e930 0x7f9fa813b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:58.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.808+0000 7f9fad136700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9fa81338a0 0x7f9fa8133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:44156/0 (socket says 172.21.15.16:44156) 2023-09-30T22:31:58.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.808+0000 7f9fac935700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9fa809de50 0x7f9fa809e220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60754/0 (socket says 172.21.15.16:60754) 2023-09-30T22:31:58.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.808+0000 7f9fad136700 1 -- 172.21.15.16:0/2077472223 learned_addr learned my addr 172.21.15.16:0/2077472223 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:58.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.808+0000 7f9fad937700 1 -- 172.21.15.16:0/2077472223 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9fa81338a0 msgr2=0x7f9fa8133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:31:58.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.808+0000 7f9fad937700 1 --2- 172.21.15.16:0/2077472223 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9fa81338a0 0x7f9fa8133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:58.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.808+0000 7f9fad937700 1 -- 172.21.15.16:0/2077472223 >> v1:172.21.15.16:6789/0 conn(0x7f9fa809de50 legacy=0x7f9fa809e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:58.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.808+0000 7f9f9ffff700 1 -- 172.21.15.16:0/2077472223 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3817344731 0 0) 0x7f9fa806ef30 con 0x7f9fa809de50 2023-09-30T22:31:58.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.808+0000 7f9fad937700 1 -- 172.21.15.16:0/2077472223 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fa806fc30 con 0x7f9fa809e930 2023-09-30T22:31:58.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.809+0000 7f9fad937700 1 --2- 172.21.15.16:0/2077472223 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9fa809e930 0x7f9fa813b740 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9fa0009dc0 tx=0x7f9fa000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a968fa5ab077689f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:58.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.812+0000 7f9f9ffff700 1 -- 172.21.15.16:0/2077472223 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa000e070 con 0x7f9fa809e930 2023-09-30T22:31:58.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.812+0000 7f9f9ffff700 1 -- 172.21.15.16:0/2077472223 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fa000bb20 con 0x7f9fa809e930 2023-09-30T22:31:58.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.812+0000 7f9f9ffff700 1 -- 172.21.15.16:0/2077472223 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa0012430 con 0x7f9fa809e930 2023-09-30T22:31:58.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.812+0000 7fee8cff9700 1 -- 172.21.15.16:0/803043599 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fee80017020 con 0x7fee90133420 2023-09-30T22:31:58.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.813+0000 7f9faf39a700 1 -- 172.21.15.16:0/2077472223 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9fa809e930 msgr2=0x7f9fa813b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:58.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.813+0000 7f9faf39a700 1 --2- 172.21.15.16:0/2077472223 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9fa809e930 0x7f9fa813b740 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9fa0009dc0 tx=0x7f9fa000b330 comp rx=0 tx=0).stop 2023-09-30T22:31:58.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.813+0000 7f9faf39a700 1 -- 172.21.15.16:0/2077472223 shutdown_connections 2023-09-30T22:31:58.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.813+0000 7f9faf39a700 1 --2- 172.21.15.16:0/2077472223 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9fa809e930 0x7f9fa813b740 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:58.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.813+0000 7f9faf39a700 1 --2- 172.21.15.16:0/2077472223 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9fa81338a0 0x7f9fa8133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:58.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.813+0000 7f9faf39a700 1 -- 172.21.15.16:0/2077472223 >> 172.21.15.16:0/2077472223 conn(0x7f9fa80a2aa0 msgr2=0x7f9fa80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:58.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.813+0000 7f9faf39a700 1 -- 172.21.15.16:0/2077472223 shutdown_connections 2023-09-30T22:31:58.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.813+0000 7f9faf39a700 1 -- 172.21.15.16:0/2077472223 wait complete. 2023-09-30T22:31:58.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.814+0000 7f9faf39a700 1 Processor -- start 2023-09-30T22:31:58.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.814+0000 7f9faf39a700 1 -- start start 2023-09-30T22:31:58.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.815+0000 7f9faf39a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9fa809de50 0x7f9fa811d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:58.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.815+0000 7f9faf39a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9fa809e930 0x7f9fa811dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:58.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.815+0000 7f9faf39a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fa81338a0 0x7f9fa811e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:31:58.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.815+0000 7f9faf39a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa806f640 con 0x7f9fa81338a0 2023-09-30T22:31:58.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.815+0000 7f9faf39a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa8121b00 con 0x7f9fa809de50 2023-09-30T22:31:58.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.815+0000 7f9faf39a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9fa8120e60 con 0x7f9fa809e930 2023-09-30T22:31:58.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.815+0000 7f9fad136700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9fa809de50 0x7f9fa811d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:58.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.815+0000 7f9fad136700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9fa809de50 0x7f9fa811d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:44162/0 (socket says 172.21.15.16:44162) 2023-09-30T22:31:58.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.815+0000 7f9fad136700 1 -- 172.21.15.16:0/1932598434 learned_addr learned my addr 172.21.15.16:0/1932598434 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:31:58.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.815+0000 7f9fad937700 1 --2- 172.21.15.16:0/1932598434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fa81338a0 0x7f9fa811e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:58.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.815+0000 7f9fac935700 1 --2- 172.21.15.16:0/1932598434 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9fa809e930 0x7f9fa811dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:31:58.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.816+0000 7f9fad136700 1 -- 172.21.15.16:0/1932598434 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9fa809e930 msgr2=0x7f9fa811dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:58.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.816+0000 7f9fad136700 1 --2- 172.21.15.16:0/1932598434 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9fa809e930 0x7f9fa811dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:58.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.816+0000 7f9fad136700 1 -- 172.21.15.16:0/1932598434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fa81338a0 msgr2=0x7f9fa811e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:58.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.816+0000 7f9fad136700 1 --2- 172.21.15.16:0/1932598434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fa81338a0 0x7f9fa811e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:58.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.816+0000 7f9fad136700 1 -- 172.21.15.16:0/1932598434 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fa0009a70 con 0x7f9fa809de50 2023-09-30T22:31:58.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.816+0000 7f9fad136700 1 --2- 172.21.15.16:0/1932598434 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9fa809de50 0x7f9fa811d730 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9fa400e940 tx=0x7f9fa400ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:31:58.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.816+0000 7f9f9dffb700 1 -- 172.21.15.16:0/1932598434 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa40096a0 con 0x7f9fa809de50 2023-09-30T22:31:58.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.816+0000 7f9f9dffb700 1 -- 172.21.15.16:0/1932598434 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fa4013680 con 0x7f9fa809de50 2023-09-30T22:31:58.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.816+0000 7f9faf39a700 1 -- 172.21.15.16:0/1932598434 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fa811f2d0 con 0x7f9fa809de50 2023-09-30T22:31:58.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.816+0000 7f9f9dffb700 1 -- 172.21.15.16:0/1932598434 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa4013a30 con 0x7f9fa809de50 2023-09-30T22:31:58.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.816+0000 7f9faf39a700 1 -- 172.21.15.16:0/1932598434 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fa8121140 con 0x7f9fa809de50 2023-09-30T22:31:58.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.818+0000 7f9f9dffb700 1 -- 172.21.15.16:0/1932598434 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9fa4013b90 con 0x7f9fa809de50 2023-09-30T22:31:58.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.818+0000 7f9faf39a700 1 -- 172.21.15.16:0/1932598434 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fa811a470 con 0x7f9fa809de50 2023-09-30T22:31:58.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.818+0000 7f9f9dffb700 1 -- 172.21.15.16:0/1932598434 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f9fa403f710 con 0x7f9fa809de50 2023-09-30T22:31:58.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.827+0000 7f9f9dffb700 1 -- 172.21.15.16:0/1932598434 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9fa400c450 con 0x7f9fa809de50 2023-09-30T22:31:58.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.895+0000 7fbb11ae0700 1 -- 172.21.15.16:0/2897905197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fbb0c11aee0 con 0x7fbb0c06fb20 2023-09-30T22:31:58.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.895+0000 7fbaf3fff700 1 -- 172.21.15.16:0/2897905197 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbb0c11aee0 con 0x7fbb0c06fb20 2023-09-30T22:31:58.896 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:31:58.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.897+0000 7fbb11ae0700 1 -- 172.21.15.16:0/2897905197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb0c06fb20 msgr2=0x7fbb0c118cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:58.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.897+0000 7fbb11ae0700 1 --2- 172.21.15.16:0/2897905197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb0c06fb20 0x7fbb0c118cd0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fbafc000c00 tx=0x7fbafc004750 comp rx=0 tx=0).stop 2023-09-30T22:31:58.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.898+0000 7fbb11ae0700 1 -- 172.21.15.16:0/2897905197 shutdown_connections 2023-09-30T22:31:58.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.898+0000 7fbb11ae0700 1 --2- 172.21.15.16:0/2897905197 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbb0c130e50 0x7fbb0c119750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:58.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.898+0000 7fbb11ae0700 1 --2- 172.21.15.16:0/2897905197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb0c06fb20 0x7fbb0c118cd0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:58.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.898+0000 7fbb11ae0700 1 --2- 172.21.15.16:0/2897905197 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbb0c130540 0x7fbb0c119210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:58.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.898+0000 7fbb11ae0700 1 -- 172.21.15.16:0/2897905197 >> 172.21.15.16:0/2897905197 conn(0x7fbb0c0a2aa0 msgr2=0x7fbb0c12d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:58.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.898+0000 7fbb11ae0700 1 -- 172.21.15.16:0/2897905197 shutdown_connections 2023-09-30T22:31:58.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.898+0000 7fbb11ae0700 1 -- 172.21.15.16:0/2897905197 wait complete. 2023-09-30T22:31:58.917 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:31:58.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.985+0000 7fec41d2c700 1 -- 172.21.15.16:0/2734706539 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fec1c005590 con 0x7fec3c06f050 2023-09-30T22:31:58.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.985+0000 7fec23fff700 1 -- 172.21.15.16:0/2734706539 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fec30015020 con 0x7fec3c06f050 2023-09-30T22:31:58.986 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:31:58.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.988+0000 7fec41d2c700 1 -- 172.21.15.16:0/2734706539 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fec3c06f050 msgr2=0x7fec3c1236e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:58.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.988+0000 7fec41d2c700 1 --2- 172.21.15.16:0/2734706539 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fec3c06f050 0x7fec3c1236e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fec30016000 tx=0x7fec30004750 comp rx=0 tx=0).stop 2023-09-30T22:31:58.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.988+0000 7fec41d2c700 1 -- 172.21.15.16:0/2734706539 shutdown_connections 2023-09-30T22:31:58.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.988+0000 7fec41d2c700 1 --2- 172.21.15.16:0/2734706539 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fec3c123c20 0x7fec3c1d2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:58.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.988+0000 7fec41d2c700 1 --2- 172.21.15.16:0/2734706539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fec3c06e570 0x7fec3c1231a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:58.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.988+0000 7fec41d2c700 1 --2- 172.21.15.16:0/2734706539 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fec3c06f050 0x7fec3c1236e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:58.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.988+0000 7fec41d2c700 1 -- 172.21.15.16:0/2734706539 >> 172.21.15.16:0/2734706539 conn(0x7fec3c0a2aa0 msgr2=0x7fec3c09a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:58.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.988+0000 7fec41d2c700 1 -- 172.21.15.16:0/2734706539 shutdown_connections 2023-09-30T22:31:58.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:58.988+0000 7fec41d2c700 1 -- 172.21.15.16:0/2734706539 wait complete. 2023-09-30T22:31:59.003 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:31:59.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:59.028+0000 7fee96239700 1 -- 172.21.15.16:0/803043599 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fee90074ea0 con 0x7fee90133420 2023-09-30T22:31:59.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:59.028+0000 7fee8cff9700 1 -- 172.21.15.16:0/803043599 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fee8001bb10 con 0x7fee90133420 2023-09-30T22:31:59.028 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:31:59.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:59.030+0000 7fee96239700 1 -- 172.21.15.16:0/803043599 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fee90133420 msgr2=0x7fee900709f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:59.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:59.030+0000 7fee96239700 1 --2- 172.21.15.16:0/803043599 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fee90133420 0x7fee900709f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fee8000c490 tx=0x7fee8000c7a0 comp rx=0 tx=0).stop 2023-09-30T22:31:59.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:59.030+0000 7fee96239700 1 -- 172.21.15.16:0/803043599 shutdown_connections 2023-09-30T22:31:59.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:59.030+0000 7fee96239700 1 --2- 172.21.15.16:0/803043599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee90138dc0 0x7fee90071470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:59.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:59.030+0000 7fee96239700 1 --2- 172.21.15.16:0/803043599 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fee90133420 0x7fee900709f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:59.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:59.030+0000 7fee96239700 1 --2- 172.21.15.16:0/803043599 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fee901337f0 0x7fee90070f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:59.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:59.030+0000 7fee96239700 1 -- 172.21.15.16:0/803043599 >> 172.21.15.16:0/803043599 conn(0x7fee900a4210 msgr2=0x7fee900a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:59.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:59.030+0000 7fee96239700 1 -- 172.21.15.16:0/803043599 shutdown_connections 2023-09-30T22:31:59.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:59.031+0000 7fee96239700 1 -- 172.21.15.16:0/803043599 wait complete. 2023-09-30T22:31:59.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:59.033+0000 7f9faf39a700 1 -- 172.21.15.16:0/1932598434 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9fa8120fa0 con 0x7f9fa809de50 2023-09-30T22:31:59.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:59.033+0000 7f9f9dffb700 1 -- 172.21.15.16:0/1932598434 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9fa8120fa0 con 0x7f9fa809de50 2023-09-30T22:31:59.033 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:31:59.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:59.035+0000 7f9faf39a700 1 -- 172.21.15.16:0/1932598434 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9fa809de50 msgr2=0x7f9fa811d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:31:59.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:59.035+0000 7f9faf39a700 1 --2- 172.21.15.16:0/1932598434 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9fa809de50 0x7f9fa811d730 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9fa400e940 tx=0x7f9fa400ec50 comp rx=0 tx=0).stop 2023-09-30T22:31:59.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:59.036+0000 7f9faf39a700 1 -- 172.21.15.16:0/1932598434 shutdown_connections 2023-09-30T22:31:59.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:59.036+0000 7f9faf39a700 1 --2- 172.21.15.16:0/1932598434 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9fa809de50 0x7f9fa811d730 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:59.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:59.036+0000 7f9faf39a700 1 --2- 172.21.15.16:0/1932598434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fa81338a0 0x7f9fa811e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:59.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:59.036+0000 7f9faf39a700 1 --2- 172.21.15.16:0/1932598434 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9fa809e930 0x7f9fa811dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:31:59.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:59.036+0000 7f9faf39a700 1 -- 172.21.15.16:0/1932598434 >> 172.21.15.16:0/1932598434 conn(0x7f9fa80a2aa0 msgr2=0x7f9fa8132b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:31:59.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:59.036+0000 7f9faf39a700 1 -- 172.21.15.16:0/1932598434 shutdown_connections 2023-09-30T22:31:59.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:31:59.036+0000 7f9faf39a700 1 -- 172.21.15.16:0/1932598434 wait complete. 2023-09-30T22:31:59.046 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:31:59.050 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:31:59.917 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:32:00.003 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:32:00.046 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:32:00.051 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:32:00.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.064+0000 7f7eabccd700 1 Processor -- start 2023-09-30T22:32:00.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.064+0000 7f7eabccd700 1 -- start start 2023-09-30T22:32:00.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.065+0000 7f7eabccd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7ea409e930 0x7f7ea409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:00.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.065+0000 7f7eabccd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7ea409f650 0x7f7ea4131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:00.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.065+0000 7f7eabccd700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ea4070940 con 0x7f7ea409f240 2023-09-30T22:32:00.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.065+0000 7f7eabccd700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ea406fd60 con 0x7f7ea409e930 2023-09-30T22:32:00.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.065+0000 7f7eabccd700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea406fa80 con 0x7f7ea409f650 2023-09-30T22:32:00.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.065+0000 7f7ea9a69700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7ea409e930 0x7f7ea409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:00.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.065+0000 7f7ea9a69700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7ea409e930 0x7f7ea409ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53952/0 (socket says 172.21.15.16:53952) 2023-09-30T22:32:00.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.065+0000 7f7ea9a69700 1 -- 172.21.15.16:0/1049095442 learned_addr learned my addr 172.21.15.16:0/1049095442 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:00.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.065+0000 7f7ea9268700 1 --1- 172.21.15.16:0/1049095442 >> v1:172.21.15.16:6789/0 conn(0x7f7ea409f240 0x7f7ea41317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60768/0 (socket says 172.21.15.16:60768) 2023-09-30T22:32:00.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.065+0000 7f7eaa26a700 1 --2- 172.21.15.16:0/1049095442 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7ea409f650 0x7f7ea4131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:00.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.065+0000 7f7ea9a69700 1 -- 172.21.15.16:0/1049095442 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7ea409f650 msgr2=0x7f7ea4131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:00.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.065+0000 7f7ea9a69700 1 --2- 172.21.15.16:0/1049095442 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7ea409f650 0x7f7ea4131f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:00.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.066+0000 7f7ea9a69700 1 -- 172.21.15.16:0/1049095442 >> v1:172.21.15.16:6789/0 conn(0x7f7ea409f240 legacy=0x7f7ea41317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:00.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.066+0000 7f7ea9a69700 1 -- 172.21.15.16:0/1049095442 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea409fb60 con 0x7f7ea409e930 2023-09-30T22:32:00.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.066+0000 7f7ea9a69700 1 --2- 172.21.15.16:0/1049095442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ea409e930 0x7f7ea409ed00 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f7ea00098f0 tx=0x7f7ea0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd139aa36fcfd105 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:00.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.066+0000 7f7ea8a67700 1 -- 172.21.15.16:0/1049095442 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea0011830 con 0x7f7ea409e930 2023-09-30T22:32:00.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.066+0000 7f7ea8a67700 1 -- 172.21.15.16:0/1049095442 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ea0011990 con 0x7f7ea409e930 2023-09-30T22:32:00.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.066+0000 7f7ea8a67700 1 -- 172.21.15.16:0/1049095442 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea0011d20 con 0x7f7ea409e930 2023-09-30T22:32:00.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.067+0000 7f7eabccd700 1 -- 172.21.15.16:0/1049095442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ea409e930 msgr2=0x7f7ea409ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:00.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.067+0000 7f7eabccd700 1 --2- 172.21.15.16:0/1049095442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ea409e930 0x7f7ea409ed00 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f7ea00098f0 tx=0x7f7ea0009c00 comp rx=0 tx=0).stop 2023-09-30T22:32:00.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.067+0000 7f7eabccd700 1 -- 172.21.15.16:0/1049095442 shutdown_connections 2023-09-30T22:32:00.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.067+0000 7f7eabccd700 1 --2- 172.21.15.16:0/1049095442 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7ea409f650 0x7f7ea4131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:00.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.067+0000 7f7eabccd700 1 --2- 172.21.15.16:0/1049095442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ea409e930 0x7f7ea409ed00 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:00.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.067+0000 7f7eabccd700 1 -- 172.21.15.16:0/1049095442 >> 172.21.15.16:0/1049095442 conn(0x7f7ea40a4210 msgr2=0x7f7ea40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:00.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.067+0000 7f7eabccd700 1 -- 172.21.15.16:0/1049095442 shutdown_connections 2023-09-30T22:32:00.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.067+0000 7f7eabccd700 1 -- 172.21.15.16:0/1049095442 wait complete. 2023-09-30T22:32:00.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.068+0000 7f7eabccd700 1 Processor -- start 2023-09-30T22:32:00.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.069+0000 7f7eabccd700 1 -- start start 2023-09-30T22:32:00.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.069+0000 7f7eabccd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ea409e930 0x7f7ea41a3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:00.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.069+0000 7f7eabccd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7ea409f240 0x7f7ea41a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:00.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.069+0000 7f7eabccd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7ea409f650 0x7f7ea41a4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:00.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.069+0000 7f7eabccd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea41a7040 con 0x7f7ea409e930 2023-09-30T22:32:00.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.069+0000 7f7ea9268700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7ea409f240 0x7f7ea41a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:00.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.069+0000 7f7ea9a69700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ea409e930 0x7f7ea41a3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:00.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.069+0000 7f7ea9268700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7ea409f240 0x7f7ea41a42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:44168/0 (socket says 172.21.15.16:44168) 2023-09-30T22:32:00.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.069+0000 7f7ea9268700 1 -- 172.21.15.16:0/4052894963 learned_addr learned my addr 172.21.15.16:0/4052894963 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:00.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.069+0000 7f7eaa26a700 1 --2- 172.21.15.16:0/4052894963 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7ea409f650 0x7f7ea41a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:00.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.069+0000 7f7eabccd700 1 -- 172.21.15.16:0/4052894963 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea4071d20 con 0x7f7ea409f240 2023-09-30T22:32:00.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.069+0000 7f7eabccd700 1 -- 172.21.15.16:0/4052894963 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea40a34c0 con 0x7f7ea409f650 2023-09-30T22:32:00.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.070+0000 7f7ea9268700 1 -- 172.21.15.16:0/4052894963 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7ea409f650 msgr2=0x7f7ea41a4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:00.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.070+0000 7f7ea9268700 1 --2- 172.21.15.16:0/4052894963 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7ea409f650 0x7f7ea41a4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:00.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.070+0000 7f7ea9268700 1 -- 172.21.15.16:0/4052894963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ea409e930 msgr2=0x7f7ea41a3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:00.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.070+0000 7f7ea9268700 1 --2- 172.21.15.16:0/4052894963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ea409e930 0x7f7ea41a3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:00.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.070+0000 7f7ea9268700 1 -- 172.21.15.16:0/4052894963 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea0009550 con 0x7f7ea409f240 2023-09-30T22:32:00.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.070+0000 7f7ea9268700 1 --2- 172.21.15.16:0/4052894963 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7ea409f240 0x7f7ea41a42d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7e9400c520 tx=0x7f7e9400c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:00.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.070+0000 7f7e9a7fc700 1 -- 172.21.15.16:0/4052894963 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e94005cb0 con 0x7f7ea409f240 2023-09-30T22:32:00.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.071+0000 7f7e9a7fc700 1 -- 172.21.15.16:0/4052894963 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e94005e10 con 0x7f7ea409f240 2023-09-30T22:32:00.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.071+0000 7f7e9a7fc700 1 -- 172.21.15.16:0/4052894963 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e940072d0 con 0x7f7ea409f240 2023-09-30T22:32:00.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.071+0000 7f7eabccd700 1 -- 172.21.15.16:0/4052894963 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ea40a0190 con 0x7f7ea409f240 2023-09-30T22:32:00.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.071+0000 7f7eabccd700 1 -- 172.21.15.16:0/4052894963 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ea41a6bf0 con 0x7f7ea409f240 2023-09-30T22:32:00.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.072+0000 7f7eabccd700 1 -- 172.21.15.16:0/4052894963 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ea41a7270 con 0x7f7ea409f240 2023-09-30T22:32:00.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.072+0000 7f7e9a7fc700 1 -- 172.21.15.16:0/4052894963 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7e94007430 con 0x7f7ea409f240 2023-09-30T22:32:00.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.073+0000 7f7e9a7fc700 1 -- 172.21.15.16:0/4052894963 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f7e9403f160 con 0x7f7ea409f240 2023-09-30T22:32:00.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.078+0000 7f7e9a7fc700 1 -- 172.21.15.16:0/4052894963 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7e940076e0 con 0x7f7ea409f240 2023-09-30T22:32:00.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.159+0000 7f08eef86700 1 Processor -- start 2023-09-30T22:32:00.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.159+0000 7f08eef86700 1 -- start start 2023-09-30T22:32:00.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.160+0000 7f08eef86700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f08e809f410 0x7f08e8136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:00.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.160+0000 7f08eef86700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f08e8136fe0 0x7f08e81393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:00.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.160+0000 7f08eef86700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08e8070940 con 0x7f08e809e930 2023-09-30T22:32:00.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.160+0000 7f08eef86700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08e806fd60 con 0x7f08e8136fe0 2023-09-30T22:32:00.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.160+0000 7f08eef86700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08e806fa80 con 0x7f08e809f410 2023-09-30T22:32:00.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.160+0000 7f08ed523700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f08e8136fe0 0x7f08e81393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:00.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.160+0000 7f08ed523700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f08e8136fe0 0x7f08e81393c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53976/0 (socket says 172.21.15.16:53976) 2023-09-30T22:32:00.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.160+0000 7f08ecd22700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f08e809e930 0x7f08e809ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60784/0 (socket says 172.21.15.16:60784) 2023-09-30T22:32:00.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.160+0000 7f08ed523700 1 -- 172.21.15.16:0/123920042 learned_addr learned my addr 172.21.15.16:0/123920042 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:00.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.161+0000 7f08e7fff700 1 --2- 172.21.15.16:0/123920042 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f08e809f410 0x7f08e8136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:00.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.161+0000 7f08ed523700 1 -- 172.21.15.16:0/123920042 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f08e809f410 msgr2=0x7f08e8136aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:00.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.161+0000 7f08ed523700 1 --2- 172.21.15.16:0/123920042 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f08e809f410 0x7f08e8136aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:00.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.161+0000 7f08ed523700 1 -- 172.21.15.16:0/123920042 >> v1:172.21.15.16:6789/0 conn(0x7f08e809e930 legacy=0x7f08e809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:00.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.161+0000 7f08e77fe700 1 -- 172.21.15.16:0/123920042 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3387185003 0 0) 0x7f08e8070940 con 0x7f08e809e930 2023-09-30T22:32:00.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.161+0000 7f08ed523700 1 -- 172.21.15.16:0/123920042 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e809fb60 con 0x7f08e8136fe0 2023-09-30T22:32:00.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.161+0000 7f08ed523700 1 --2- 172.21.15.16:0/123920042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08e8136fe0 0x7f08e81393c0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f08e0009e50 tx=0x7f08e000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bc254701762688f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:00.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.161+0000 7f08e77fe700 1 -- 172.21.15.16:0/123920042 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e000e070 con 0x7f08e8136fe0 2023-09-30T22:32:00.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.161+0000 7f08e77fe700 1 -- 172.21.15.16:0/123920042 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e000b9c0 con 0x7f08e8136fe0 2023-09-30T22:32:00.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.161+0000 7f08e77fe700 1 -- 172.21.15.16:0/123920042 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e000bd70 con 0x7f08e8136fe0 2023-09-30T22:32:00.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.162+0000 7f08eef86700 1 -- 172.21.15.16:0/123920042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08e8136fe0 msgr2=0x7f08e81393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:00.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.162+0000 7f08eef86700 1 --2- 172.21.15.16:0/123920042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08e8136fe0 0x7f08e81393c0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f08e0009e50 tx=0x7f08e000b330 comp rx=0 tx=0).stop 2023-09-30T22:32:00.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.162+0000 7f08eef86700 1 -- 172.21.15.16:0/123920042 shutdown_connections 2023-09-30T22:32:00.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.162+0000 7f08eef86700 1 --2- 172.21.15.16:0/123920042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08e8136fe0 0x7f08e81393c0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:00.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.162+0000 7f08eef86700 1 --2- 172.21.15.16:0/123920042 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f08e809f410 0x7f08e8136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:00.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.162+0000 7f08eef86700 1 -- 172.21.15.16:0/123920042 >> 172.21.15.16:0/123920042 conn(0x7f08e80a4210 msgr2=0x7f08e80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:00.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.163+0000 7f08eef86700 1 -- 172.21.15.16:0/123920042 shutdown_connections 2023-09-30T22:32:00.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.163+0000 7f08eef86700 1 -- 172.21.15.16:0/123920042 wait complete. 2023-09-30T22:32:00.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.164+0000 7f08eef86700 1 Processor -- start 2023-09-30T22:32:00.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.164+0000 7f08eef86700 1 -- start start 2023-09-30T22:32:00.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.165+0000 7f08eef86700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08e809e930 0x7f08e81c63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:00.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.165+0000 7f08eef86700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f08e809f410 0x7f08e81c6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:00.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.165+0000 7f08eef86700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f08e8136fe0 0x7f08e81c6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:00.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.165+0000 7f08eef86700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08e811e6f0 con 0x7f08e809e930 2023-09-30T22:32:00.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.165+0000 7f08eef86700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08e8132490 con 0x7f08e8136fe0 2023-09-30T22:32:00.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.165+0000 7f08eef86700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f08e811eb90 con 0x7f08e809f410 2023-09-30T22:32:00.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.165+0000 7f08ecd22700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08e809e930 0x7f08e81c63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:00.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.165+0000 7f08ecd22700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08e809e930 0x7f08e81c63e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53980/0 (socket says 172.21.15.16:53980) 2023-09-30T22:32:00.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.165+0000 7f08ecd22700 1 -- 172.21.15.16:0/3524954492 learned_addr learned my addr 172.21.15.16:0/3524954492 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:00.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.165+0000 7f08ed523700 1 --2- 172.21.15.16:0/3524954492 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f08e8136fe0 0x7f08e81c6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:00.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.165+0000 7f08e7fff700 1 --2- 172.21.15.16:0/3524954492 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f08e809f410 0x7f08e81c6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:00.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.165+0000 7f08ecd22700 1 -- 172.21.15.16:0/3524954492 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f08e809f410 msgr2=0x7f08e81c6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:00.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.165+0000 7f08ecd22700 1 --2- 172.21.15.16:0/3524954492 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f08e809f410 0x7f08e81c6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:00.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.165+0000 7f08ecd22700 1 -- 172.21.15.16:0/3524954492 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f08e8136fe0 msgr2=0x7f08e81c6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:00.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.165+0000 7f08ecd22700 1 --2- 172.21.15.16:0/3524954492 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f08e8136fe0 0x7f08e81c6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:00.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.165+0000 7f08ecd22700 1 -- 172.21.15.16:0/3524954492 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e0009b30 con 0x7f08e809e930 2023-09-30T22:32:00.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.165+0000 7f08ed523700 1 --2- 172.21.15.16:0/3524954492 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f08e8136fe0 0x7f08e81c6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:32:00.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.166+0000 7f08ecd22700 1 --2- 172.21.15.16:0/3524954492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08e809e930 0x7f08e81c63e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f08dc0011e0 tx=0x7f08dc00ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:00.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.166+0000 7f08e57fa700 1 -- 172.21.15.16:0/3524954492 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc004c80 con 0x7f08e809e930 2023-09-30T22:32:00.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.166+0000 7f08e57fa700 1 -- 172.21.15.16:0/3524954492 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08dc004de0 con 0x7f08e809e930 2023-09-30T22:32:00.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.166+0000 7f08eef86700 1 -- 172.21.15.16:0/3524954492 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08e811ff50 con 0x7f08e809e930 2023-09-30T22:32:00.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.166+0000 7f08e57fa700 1 -- 172.21.15.16:0/3524954492 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc007310 con 0x7f08e809e930 2023-09-30T22:32:00.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.166+0000 7f08eef86700 1 -- 172.21.15.16:0/3524954492 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08e811edd0 con 0x7f08e809e930 2023-09-30T22:32:00.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.167+0000 7f08e57fa700 1 -- 172.21.15.16:0/3524954492 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f08dc007470 con 0x7f08e809e930 2023-09-30T22:32:00.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.167+0000 7f08e57fa700 1 -- 172.21.15.16:0/3524954492 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f08dc03f450 con 0x7f08e809e930 2023-09-30T22:32:00.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.167+0000 7f08eef86700 1 -- 172.21.15.16:0/3524954492 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08e809fdb0 con 0x7f08e809e930 2023-09-30T22:32:00.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.172+0000 7f08e57fa700 1 -- 172.21.15.16:0/3524954492 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f08dc0123b0 con 0x7f08e809e930 2023-09-30T22:32:00.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.207+0000 7fb548cc8700 1 Processor -- start 2023-09-30T22:32:00.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.208+0000 7fb548cc8700 1 -- start start 2023-09-30T22:32:00.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.208+0000 7fb548cc8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb5441338a0 0x7fb544133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:00.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.208+0000 7fb548cc8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb54409de50 0x7fb54409e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:00.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.208+0000 7fb548cc8700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb54406ef30 con 0x7fb54409e760 2023-09-30T22:32:00.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.208+0000 7fb548cc8700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb54406fef0 con 0x7fb54409de50 2023-09-30T22:32:00.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.208+0000 7fb548cc8700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb5441334b0 con 0x7fb5441338a0 2023-09-30T22:32:00.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.209+0000 7fb541d9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb54409de50 0x7fb54409e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:00.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.209+0000 7fb541d9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb54409de50 0x7fb54409e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:44208/0 (socket says 172.21.15.16:44208) 2023-09-30T22:32:00.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.209+0000 7fb542d9d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb54409e760 0x7fb54413a730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60786/0 (socket says 172.21.15.16:60786) 2023-09-30T22:32:00.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.209+0000 7fb541d9b700 1 -- 172.21.15.16:0/2258033008 learned_addr learned my addr 172.21.15.16:0/2258033008 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:00.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.209+0000 7fb54259c700 1 --2- 172.21.15.16:0/2258033008 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb5441338a0 0x7fb544133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:00.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.209+0000 7fb54259c700 1 -- 172.21.15.16:0/2258033008 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb54409de50 msgr2=0x7fb54409e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:00.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.209+0000 7fb54259c700 1 --2- 172.21.15.16:0/2258033008 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb54409de50 0x7fb54409e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:00.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.209+0000 7fb54259c700 1 -- 172.21.15.16:0/2258033008 >> v1:172.21.15.16:6789/0 conn(0x7fb54409e760 legacy=0x7fb54413a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:00.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.209+0000 7fb54259c700 1 -- 172.21.15.16:0/2258033008 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb54406fc30 con 0x7fb5441338a0 2023-09-30T22:32:00.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.209+0000 7fb54159a700 1 -- 172.21.15.16:0/2258033008 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1670146125 0 0) 0x7fb54406ef30 con 0x7fb54409e760 2023-09-30T22:32:00.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.209+0000 7fb541d9b700 1 --2- 172.21.15.16:0/2258033008 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb54409de50 0x7fb54409e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:00.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.209+0000 7fb54259c700 1 --2- 172.21.15.16:0/2258033008 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb5441338a0 0x7fb544133c70 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb5340098a0 tx=0x7fb534009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d4936e0ee3f5fbb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:00.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.210+0000 7fb54159a700 1 -- 172.21.15.16:0/2258033008 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb534011830 con 0x7fb5441338a0 2023-09-30T22:32:00.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.210+0000 7fb54159a700 1 -- 172.21.15.16:0/2258033008 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb534011990 con 0x7fb5441338a0 2023-09-30T22:32:00.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.210+0000 7fb54159a700 1 -- 172.21.15.16:0/2258033008 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb534011d20 con 0x7fb5441338a0 2023-09-30T22:32:00.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.210+0000 7fb548cc8700 1 -- 172.21.15.16:0/2258033008 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb5441338a0 msgr2=0x7fb544133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:00.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.210+0000 7fb548cc8700 1 --2- 172.21.15.16:0/2258033008 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb5441338a0 0x7fb544133c70 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb5340098a0 tx=0x7fb534009bb0 comp rx=0 tx=0).stop 2023-09-30T22:32:00.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.210+0000 7fb548cc8700 1 -- 172.21.15.16:0/2258033008 shutdown_connections 2023-09-30T22:32:00.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.210+0000 7fb548cc8700 1 --2- 172.21.15.16:0/2258033008 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb5441338a0 0x7fb544133c70 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:00.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.210+0000 7fb548cc8700 1 --2- 172.21.15.16:0/2258033008 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb54409de50 0x7fb54409e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:00.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.210+0000 7fb548cc8700 1 -- 172.21.15.16:0/2258033008 >> 172.21.15.16:0/2258033008 conn(0x7fb5440a2aa0 msgr2=0x7fb5440a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:00.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.210+0000 7fb548cc8700 1 -- 172.21.15.16:0/2258033008 shutdown_connections 2023-09-30T22:32:00.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.210+0000 7fb548cc8700 1 -- 172.21.15.16:0/2258033008 wait complete. 2023-09-30T22:32:00.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.212+0000 7fb548cc8700 1 Processor -- start 2023-09-30T22:32:00.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.212+0000 7fb548cc8700 1 -- start start 2023-09-30T22:32:00.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.212+0000 7fb548cc8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb54409de50 0x7fb54411b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:00.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.212+0000 7fb548cc8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb54409e760 0x7fb54411bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:00.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.212+0000 7fb548cc8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb5441338a0 0x7fb54411bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:00.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.212+0000 7fb548cc8700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb54411f090 con 0x7fb54409de50 2023-09-30T22:32:00.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.212+0000 7fb548cc8700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb54411d220 con 0x7fb5441338a0 2023-09-30T22:32:00.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.212+0000 7fb548cc8700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb5440a0e40 con 0x7fb54409e760 2023-09-30T22:32:00.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.212+0000 7fb54259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb54409de50 0x7fb54411b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:00.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.212+0000 7fb54259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb54409de50 0x7fb54411b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53982/0 (socket says 172.21.15.16:53982) 2023-09-30T22:32:00.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.212+0000 7fb54259c700 1 -- 172.21.15.16:0/2727398882 learned_addr learned my addr 172.21.15.16:0/2727398882 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:00.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.212+0000 7fb542d9d700 1 --2- 172.21.15.16:0/2727398882 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb5441338a0 0x7fb54411bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:00.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.213+0000 7fb541d9b700 1 --2- 172.21.15.16:0/2727398882 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb54409e760 0x7fb54411bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:00.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.213+0000 7fb54259c700 1 -- 172.21.15.16:0/2727398882 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb54409e760 msgr2=0x7fb54411bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:00.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.213+0000 7fb54259c700 1 --2- 172.21.15.16:0/2727398882 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb54409e760 0x7fb54411bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:00.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.213+0000 7fb54259c700 1 -- 172.21.15.16:0/2727398882 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb5441338a0 msgr2=0x7fb54411bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:00.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.213+0000 7fb54259c700 1 --2- 172.21.15.16:0/2727398882 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb5441338a0 0x7fb54411bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:00.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.213+0000 7fb54259c700 1 -- 172.21.15.16:0/2727398882 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb538009550 con 0x7fb54409de50 2023-09-30T22:32:00.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.213+0000 7fb54259c700 1 --2- 172.21.15.16:0/2727398882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb54409de50 0x7fb54411b570 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fb5340097c0 tx=0x7fb534004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:00.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.213+0000 7fb532ffd700 1 -- 172.21.15.16:0/2727398882 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb534004550 con 0x7fb54409de50 2023-09-30T22:32:00.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.213+0000 7fb532ffd700 1 -- 172.21.15.16:0/2727398882 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5340036a0 con 0x7fb54409de50 2023-09-30T22:32:00.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.213+0000 7fb532ffd700 1 -- 172.21.15.16:0/2727398882 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb534003a10 con 0x7fb54409de50 2023-09-30T22:32:00.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.213+0000 7fb548cc8700 1 -- 172.21.15.16:0/2727398882 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5340094a0 con 0x7fb54409de50 2023-09-30T22:32:00.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.213+0000 7fb548cc8700 1 -- 172.21.15.16:0/2727398882 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb54411df80 con 0x7fb54409de50 2023-09-30T22:32:00.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.214+0000 7fb532ffd700 1 -- 172.21.15.16:0/2727398882 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb534003b70 con 0x7fb54409de50 2023-09-30T22:32:00.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.214+0000 7f9ed961b700 1 Processor -- start 2023-09-30T22:32:00.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.215+0000 7fb548cc8700 1 -- 172.21.15.16:0/2727398882 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb544074ea0 con 0x7fb54409de50 2023-09-30T22:32:00.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.215+0000 7fb532ffd700 1 -- 172.21.15.16:0/2727398882 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fb534040b50 con 0x7fb54409de50 2023-09-30T22:32:00.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.215+0000 7f9ed961b700 1 -- start start 2023-09-30T22:32:00.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.215+0000 7f9ed961b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9ed40a0a30 0x7f9ed40a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:00.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.215+0000 7f9ed961b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9ed4134170 0x7f9ed413bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:00.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.215+0000 7f9ed961b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ed409edd0 con 0x7f9ed40a1340 2023-09-30T22:32:00.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.215+0000 7f9ed961b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ed409e1f0 con 0x7f9ed4134170 2023-09-30T22:32:00.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.215+0000 7f9ed961b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed409dc50 con 0x7f9ed40a0a30 2023-09-30T22:32:00.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.215+0000 7f9ed37fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9ed4134170 0x7f9ed413bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:00.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.216+0000 7f9ed37fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9ed4134170 0x7f9ed413bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53998/0 (socket says 172.21.15.16:53998) 2023-09-30T22:32:00.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.216+0000 7f9ed37fe700 1 -- 172.21.15.16:0/2706934572 learned_addr learned my addr 172.21.15.16:0/2706934572 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:00.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.216+0000 7f9ed2ffd700 1 --2- 172.21.15.16:0/2706934572 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9ed40a0a30 0x7f9ed40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:00.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.216+0000 7f9ed37fe700 1 -- 172.21.15.16:0/2706934572 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9ed40a0a30 msgr2=0x7f9ed40a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:00.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.216+0000 7f9ed37fe700 1 --2- 172.21.15.16:0/2706934572 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9ed40a0a30 0x7f9ed40a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:00.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.216+0000 7f9ed37fe700 1 -- 172.21.15.16:0/2706934572 >> v1:172.21.15.16:6789/0 conn(0x7f9ed40a1340 legacy=0x7f9ed4133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:00.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.216+0000 7f9ed37fe700 1 -- 172.21.15.16:0/2706934572 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed40a1fe0 con 0x7f9ed4134170 2023-09-30T22:32:00.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.216+0000 7f9ed37fe700 1 --2- 172.21.15.16:0/2706934572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ed4134170 0x7f9ed413bd30 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc009e50 tx=0x7f9ecc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2125141b8e3bb76 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:00.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.224+0000 7f9ed1ffb700 1 -- 172.21.15.16:0/2706934572 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ecc00e070 con 0x7f9ed4134170 2023-09-30T22:32:00.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.224+0000 7f9ed1ffb700 1 -- 172.21.15.16:0/2706934572 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ecc00b910 con 0x7f9ed4134170 2023-09-30T22:32:00.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.224+0000 7f9ed1ffb700 1 -- 172.21.15.16:0/2706934572 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ecc00bcc0 con 0x7f9ed4134170 2023-09-30T22:32:00.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.224+0000 7fb532ffd700 1 -- 172.21.15.16:0/2727398882 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb53401a420 con 0x7fb54409de50 2023-09-30T22:32:00.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.225+0000 7f9ed961b700 1 -- 172.21.15.16:0/2706934572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ed4134170 msgr2=0x7f9ed413bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:00.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.225+0000 7f9ed961b700 1 --2- 172.21.15.16:0/2706934572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ed4134170 0x7f9ed413bd30 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc009e50 tx=0x7f9ecc00b330 comp rx=0 tx=0).stop 2023-09-30T22:32:00.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.225+0000 7f9ed961b700 1 -- 172.21.15.16:0/2706934572 shutdown_connections 2023-09-30T22:32:00.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.225+0000 7f9ed961b700 1 --2- 172.21.15.16:0/2706934572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ed4134170 0x7f9ed413bd30 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:00.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.225+0000 7f9ed961b700 1 --2- 172.21.15.16:0/2706934572 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9ed40a0a30 0x7f9ed40a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:00.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.225+0000 7f9ed961b700 1 -- 172.21.15.16:0/2706934572 >> 172.21.15.16:0/2706934572 conn(0x7f9ed40a6550 msgr2=0x7f9ed40a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:00.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.225+0000 7f9ed961b700 1 -- 172.21.15.16:0/2706934572 shutdown_connections 2023-09-30T22:32:00.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.225+0000 7f9ed961b700 1 -- 172.21.15.16:0/2706934572 wait complete. 2023-09-30T22:32:00.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.227+0000 7f9ed961b700 1 Processor -- start 2023-09-30T22:32:00.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.227+0000 7f9ed961b700 1 -- start start 2023-09-30T22:32:00.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.227+0000 7f9ed961b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ed40a0a30 0x7f9ed4122a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:00.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.227+0000 7f9ed961b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9ed40a1340 0x7f9ed4122f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:00.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.227+0000 7f9ed961b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9ed4134170 0x7f9ed4123480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:00.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.227+0000 7f9ed961b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed4124e10 con 0x7f9ed40a0a30 2023-09-30T22:32:00.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.227+0000 7f9ed961b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed40a4e80 con 0x7f9ed4134170 2023-09-30T22:32:00.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.227+0000 7f9ed961b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed40a0220 con 0x7f9ed40a1340 2023-09-30T22:32:00.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.228+0000 7f9ed2ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ed40a0a30 0x7f9ed4122a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:00.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.228+0000 7f9ed2ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ed40a0a30 0x7f9ed4122a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54000/0 (socket says 172.21.15.16:54000) 2023-09-30T22:32:00.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.228+0000 7f9ed2ffd700 1 -- 172.21.15.16:0/2368362092 learned_addr learned my addr 172.21.15.16:0/2368362092 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:00.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.228+0000 7f9ed37fe700 1 --2- 172.21.15.16:0/2368362092 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9ed4134170 0x7f9ed4123480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:00.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.228+0000 7f9ed27fc700 1 --2- 172.21.15.16:0/2368362092 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9ed40a1340 0x7f9ed4122f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:00.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.228+0000 7f9ed27fc700 1 -- 172.21.15.16:0/2368362092 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9ed4134170 msgr2=0x7f9ed4123480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:00.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.228+0000 7f9ed27fc700 1 --2- 172.21.15.16:0/2368362092 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9ed4134170 0x7f9ed4123480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:00.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.228+0000 7f9ed27fc700 1 -- 172.21.15.16:0/2368362092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ed40a0a30 msgr2=0x7f9ed4122a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:00.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.228+0000 7f9ed27fc700 1 --2- 172.21.15.16:0/2368362092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ed40a0a30 0x7f9ed4122a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:00.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.228+0000 7f9ed27fc700 1 -- 172.21.15.16:0/2368362092 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ecc009b30 con 0x7f9ed40a1340 2023-09-30T22:32:00.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.228+0000 7f9ed27fc700 1 --2- 172.21.15.16:0/2368362092 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9ed40a1340 0x7f9ed4122f40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8000f60 tx=0x7f9ec800c730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:00.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.229+0000 7f9ebb7fe700 1 -- 172.21.15.16:0/2368362092 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8015d70 con 0x7f9ed40a1340 2023-09-30T22:32:00.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.229+0000 7f9ebb7fe700 1 -- 172.21.15.16:0/2368362092 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ec800ce30 con 0x7f9ed40a1340 2023-09-30T22:32:00.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.229+0000 7f9ebb7fe700 1 -- 172.21.15.16:0/2368362092 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec80052f0 con 0x7f9ed40a1340 2023-09-30T22:32:00.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.229+0000 7f9ed961b700 1 -- 172.21.15.16:0/2368362092 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ed4126170 con 0x7f9ed40a1340 2023-09-30T22:32:00.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.229+0000 7f9ed961b700 1 -- 172.21.15.16:0/2368362092 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ed4126be0 con 0x7f9ed40a1340 2023-09-30T22:32:00.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.230+0000 7f9ebb7fe700 1 -- 172.21.15.16:0/2368362092 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9ec8003680 con 0x7f9ed40a1340 2023-09-30T22:32:00.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.230+0000 7f9ed961b700 1 -- 172.21.15.16:0/2368362092 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ed40a1d00 con 0x7f9ed40a1340 2023-09-30T22:32:00.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.230+0000 7f9ebb7fe700 1 -- 172.21.15.16:0/2368362092 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f9ec80470f0 con 0x7f9ed40a1340 2023-09-30T22:32:00.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.235+0000 7f9ebb7fe700 1 -- 172.21.15.16:0/2368362092 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9ec801c760 con 0x7f9ed40a1340 2023-09-30T22:32:00.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.315+0000 7f7eabccd700 1 -- 172.21.15.16:0/4052894963 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7ea41a7270 con 0x7f7ea409f240 2023-09-30T22:32:00.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.316+0000 7f7e9a7fc700 1 -- 172.21.15.16:0/4052894963 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7ea41a7270 con 0x7f7ea409f240 2023-09-30T22:32:00.316 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:32:00.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.318+0000 7f7eabccd700 1 -- 172.21.15.16:0/4052894963 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7ea409f240 msgr2=0x7f7ea41a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:00.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.318+0000 7f7eabccd700 1 --2- 172.21.15.16:0/4052894963 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7ea409f240 0x7f7ea41a42d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7e9400c520 tx=0x7f7e9400c830 comp rx=0 tx=0).stop 2023-09-30T22:32:00.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.318+0000 7f7eabccd700 1 -- 172.21.15.16:0/4052894963 shutdown_connections 2023-09-30T22:32:00.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.318+0000 7f7eabccd700 1 --2- 172.21.15.16:0/4052894963 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7ea409f650 0x7f7ea41a4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:00.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.318+0000 7f7eabccd700 1 --2- 172.21.15.16:0/4052894963 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ea409e930 0x7f7ea41a3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:00.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.318+0000 7f7eabccd700 1 --2- 172.21.15.16:0/4052894963 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7ea409f240 0x7f7ea41a42d0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:00.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.318+0000 7f7eabccd700 1 -- 172.21.15.16:0/4052894963 >> 172.21.15.16:0/4052894963 conn(0x7f7ea40a4210 msgr2=0x7f7ea41b8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:00.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.318+0000 7f7eabccd700 1 -- 172.21.15.16:0/4052894963 shutdown_connections 2023-09-30T22:32:00.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.318+0000 7f7eabccd700 1 -- 172.21.15.16:0/4052894963 wait complete. 2023-09-30T22:32:00.332 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:32:00.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.432+0000 7fb548cc8700 1 -- 172.21.15.16:0/2727398882 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb54406b9e0 con 0x7fb54409de50 2023-09-30T22:32:00.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.432+0000 7fb532ffd700 1 -- 172.21.15.16:0/2727398882 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb534010700 con 0x7fb54409de50 2023-09-30T22:32:00.433 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:32:00.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.435+0000 7fb548cc8700 1 -- 172.21.15.16:0/2727398882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb54409de50 msgr2=0x7fb54411b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:00.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.435+0000 7fb548cc8700 1 --2- 172.21.15.16:0/2727398882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb54409de50 0x7fb54411b570 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fb5340097c0 tx=0x7fb534004750 comp rx=0 tx=0).stop 2023-09-30T22:32:00.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.435+0000 7fb548cc8700 1 -- 172.21.15.16:0/2727398882 shutdown_connections 2023-09-30T22:32:00.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.435+0000 7fb548cc8700 1 --2- 172.21.15.16:0/2727398882 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb5441338a0 0x7fb54411bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:00.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.435+0000 7fb548cc8700 1 --2- 172.21.15.16:0/2727398882 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb54409de50 0x7fb54411b570 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:00.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.435+0000 7fb548cc8700 1 --2- 172.21.15.16:0/2727398882 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb54409e760 0x7fb54411bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:00.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.435+0000 7fb548cc8700 1 -- 172.21.15.16:0/2727398882 >> 172.21.15.16:0/2727398882 conn(0x7fb5440a2aa0 msgr2=0x7fb5441305a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:00.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.435+0000 7fb548cc8700 1 -- 172.21.15.16:0/2727398882 shutdown_connections 2023-09-30T22:32:00.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.435+0000 7fb548cc8700 1 -- 172.21.15.16:0/2727398882 wait complete. 2023-09-30T22:32:00.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.441+0000 7f08eef86700 1 -- 172.21.15.16:0/3524954492 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f08e8074ea0 con 0x7f08e809e930 2023-09-30T22:32:00.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.441+0000 7f08e57fa700 1 -- 172.21.15.16:0/3524954492 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f08dc01b340 con 0x7f08e809e930 2023-09-30T22:32:00.441 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:32:00.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.445+0000 7f08eef86700 1 -- 172.21.15.16:0/3524954492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08e809e930 msgr2=0x7f08e81c63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:00.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.445+0000 7f08eef86700 1 --2- 172.21.15.16:0/3524954492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08e809e930 0x7f08e81c63e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f08dc0011e0 tx=0x7f08dc00ca30 comp rx=0 tx=0).stop 2023-09-30T22:32:00.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.445+0000 7f08eef86700 1 -- 172.21.15.16:0/3524954492 shutdown_connections 2023-09-30T22:32:00.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.445+0000 7f08eef86700 1 --2- 172.21.15.16:0/3524954492 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f08e8136fe0 0x7f08e81c6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:00.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.445+0000 7f08eef86700 1 --2- 172.21.15.16:0/3524954492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08e809e930 0x7f08e81c63e0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:00.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.445+0000 7f08eef86700 1 --2- 172.21.15.16:0/3524954492 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f08e809f410 0x7f08e81c6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:00.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.445+0000 7f08eef86700 1 -- 172.21.15.16:0/3524954492 >> 172.21.15.16:0/3524954492 conn(0x7f08e80a4210 msgr2=0x7f08e8134ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:00.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.445+0000 7f08eef86700 1 -- 172.21.15.16:0/3524954492 shutdown_connections 2023-09-30T22:32:00.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.445+0000 7f08eef86700 1 -- 172.21.15.16:0/3524954492 wait complete. 2023-09-30T22:32:00.450 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:32:00.464 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:32:00.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.491+0000 7f9ed961b700 1 -- 172.21.15.16:0/2368362092 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9ed4074ea0 con 0x7f9ed40a1340 2023-09-30T22:32:00.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.491+0000 7f9ebb7fe700 1 -- 172.21.15.16:0/2368362092 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9ec8022070 con 0x7f9ed40a1340 2023-09-30T22:32:00.492 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:32:00.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.494+0000 7f9ed961b700 1 -- 172.21.15.16:0/2368362092 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9ed40a1340 msgr2=0x7f9ed4122f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:00.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.494+0000 7f9ed961b700 1 --2- 172.21.15.16:0/2368362092 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9ed40a1340 0x7f9ed4122f40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8000f60 tx=0x7f9ec800c730 comp rx=0 tx=0).stop 2023-09-30T22:32:00.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.494+0000 7f9ed961b700 1 -- 172.21.15.16:0/2368362092 shutdown_connections 2023-09-30T22:32:00.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.494+0000 7f9ed961b700 1 --2- 172.21.15.16:0/2368362092 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9ed4134170 0x7f9ed4123480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:00.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.494+0000 7f9ed961b700 1 --2- 172.21.15.16:0/2368362092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ed40a0a30 0x7f9ed4122a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:00.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.494+0000 7f9ed961b700 1 --2- 172.21.15.16:0/2368362092 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9ed40a1340 0x7f9ed4122f40 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:00.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.494+0000 7f9ed961b700 1 -- 172.21.15.16:0/2368362092 >> 172.21.15.16:0/2368362092 conn(0x7f9ed40a6550 msgr2=0x7f9ed409ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:00.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.494+0000 7f9ed961b700 1 -- 172.21.15.16:0/2368362092 shutdown_connections 2023-09-30T22:32:00.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:00.494+0000 7f9ed961b700 1 -- 172.21.15.16:0/2368362092 wait complete. 2023-09-30T22:32:00.508 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:32:01.333 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:32:01.451 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:32:01.465 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:32:01.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.482+0000 7fdacee04700 1 Processor -- start 2023-09-30T22:32:01.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.482+0000 7fdacee04700 1 -- start start 2023-09-30T22:32:01.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.483+0000 7fdacee04700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdac80a0a30 0x7fdac80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:01.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.483+0000 7fdacee04700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdac809d630 0x7fdac80a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:01.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.483+0000 7fdacee04700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdac80a2a70 con 0x7fdac8133930 2023-09-30T22:32:01.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.483+0000 7fdacee04700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdac80a01d0 con 0x7fdac809d630 2023-09-30T22:32:01.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.483+0000 7fdacee04700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdac809e2b0 con 0x7fdac80a0a30 2023-09-30T22:32:01.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.483+0000 7fdac7fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdac809d630 0x7fdac80a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:01.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.483+0000 7fdaccba0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdac80a0a30 0x7fdac80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:01.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.483+0000 7fdacd3a1700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fdac8133930 0x7fdac8133db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60800/0 (socket says 172.21.15.16:60800) 2023-09-30T22:32:01.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.483+0000 7fdac7fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdac809d630 0x7fdac80a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54002/0 (socket says 172.21.15.16:54002) 2023-09-30T22:32:01.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.483+0000 7fdacd3a1700 1 -- 172.21.15.16:0/1446661401 learned_addr learned my addr 172.21.15.16:0/1446661401 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:01.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.483+0000 7fdaccba0700 1 -- 172.21.15.16:0/1446661401 >> v2:172.21.15.16:3300/0 conn(0x7fdac809d630 msgr2=0x7fdac80a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:01.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.483+0000 7fdaccba0700 1 --2- 172.21.15.16:0/1446661401 >> v2:172.21.15.16:3300/0 conn(0x7fdac809d630 0x7fdac80a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:01.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.483+0000 7fdaccba0700 1 -- 172.21.15.16:0/1446661401 >> v1:172.21.15.16:6789/0 conn(0x7fdac8133930 legacy=0x7fdac8133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:01.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.483+0000 7fdaccba0700 1 -- 172.21.15.16:0/1446661401 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdac809dc50 con 0x7fdac80a0a30 2023-09-30T22:32:01.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.484+0000 7fdac7fff700 1 --2- 172.21.15.16:0/1446661401 >> v2:172.21.15.16:3300/0 conn(0x7fdac809d630 0x7fdac80a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:01.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.483+0000 7fdac77fe700 1 -- 172.21.15.16:0/1446661401 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3425454589 0 0) 0x7fdac80a2a70 con 0x7fdac8133930 2023-09-30T22:32:01.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.484+0000 7fdaccba0700 1 --2- 172.21.15.16:0/1446661401 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdac80a0a30 0x7fdac80a0e00 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fdab8009840 tx=0x7fdab8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5b77e7aecc64824 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:01.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.484+0000 7fdac77fe700 1 -- 172.21.15.16:0/1446661401 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab8011830 con 0x7fdac80a0a30 2023-09-30T22:32:01.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.484+0000 7fdac77fe700 1 -- 172.21.15.16:0/1446661401 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdab8011990 con 0x7fdac80a0a30 2023-09-30T22:32:01.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.484+0000 7fdac77fe700 1 -- 172.21.15.16:0/1446661401 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab8011d20 con 0x7fdac80a0a30 2023-09-30T22:32:01.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.485+0000 7fdacee04700 1 -- 172.21.15.16:0/1446661401 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdac80a0a30 msgr2=0x7fdac80a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:01.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.485+0000 7fdacee04700 1 --2- 172.21.15.16:0/1446661401 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdac80a0a30 0x7fdac80a0e00 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fdab8009840 tx=0x7fdab8009b50 comp rx=0 tx=0).stop 2023-09-30T22:32:01.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.485+0000 7fdacee04700 1 -- 172.21.15.16:0/1446661401 shutdown_connections 2023-09-30T22:32:01.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.485+0000 7fdacee04700 1 --2- 172.21.15.16:0/1446661401 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdac80a0a30 0x7fdac80a0e00 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:01.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.485+0000 7fdacee04700 1 --2- 172.21.15.16:0/1446661401 >> v2:172.21.15.16:3300/0 conn(0x7fdac809d630 0x7fdac80a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:01.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.485+0000 7fdacee04700 1 -- 172.21.15.16:0/1446661401 >> 172.21.15.16:0/1446661401 conn(0x7fdac80a3050 msgr2=0x7fdac80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:01.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.485+0000 7fdacee04700 1 -- 172.21.15.16:0/1446661401 shutdown_connections 2023-09-30T22:32:01.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.485+0000 7fdacee04700 1 -- 172.21.15.16:0/1446661401 wait complete. 2023-09-30T22:32:01.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.486+0000 7fdacee04700 1 Processor -- start 2023-09-30T22:32:01.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.486+0000 7fdacee04700 1 -- start start 2023-09-30T22:32:01.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.487+0000 7fdacee04700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdac809d630 0x7fdac81c7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:01.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.487+0000 7fdacee04700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdac80a0a30 0x7fdac81c8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:01.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.487+0000 7fdacee04700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdac8133930 0x7fdac81c8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:01.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.487+0000 7fdacee04700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac8121e90 con 0x7fdac809d630 2023-09-30T22:32:01.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.487+0000 7fdacee04700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdac8121290 con 0x7fdac8133930 2023-09-30T22:32:01.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.487+0000 7fdacee04700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdac8120520 con 0x7fdac80a0a30 2023-09-30T22:32:01.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.487+0000 7fdaccba0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdac809d630 0x7fdac81c7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:01.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.487+0000 7fdacd3a1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdac8133930 0x7fdac81c8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:01.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.487+0000 7fdaccba0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdac809d630 0x7fdac81c7ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54018/0 (socket says 172.21.15.16:54018) 2023-09-30T22:32:01.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.487+0000 7fdaccba0700 1 -- 172.21.15.16:0/2481662721 learned_addr learned my addr 172.21.15.16:0/2481662721 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:01.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.487+0000 7fdac7fff700 1 --2- 172.21.15.16:0/2481662721 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdac80a0a30 0x7fdac81c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:01.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.488+0000 7fdaccba0700 1 -- 172.21.15.16:0/2481662721 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdac80a0a30 msgr2=0x7fdac81c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:01.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.488+0000 7fdaccba0700 1 --2- 172.21.15.16:0/2481662721 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdac80a0a30 0x7fdac81c8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:01.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.488+0000 7fdaccba0700 1 -- 172.21.15.16:0/2481662721 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdac8133930 msgr2=0x7fdac81c8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:01.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.488+0000 7fdaccba0700 1 --2- 172.21.15.16:0/2481662721 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdac8133930 0x7fdac81c8970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:01.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.488+0000 7fdaccba0700 1 -- 172.21.15.16:0/2481662721 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdabc0094a0 con 0x7fdac809d630 2023-09-30T22:32:01.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.488+0000 7fdaccba0700 1 --2- 172.21.15.16:0/2481662721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdac809d630 0x7fdac81c7ef0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fdab8009fd0 tx=0x7fdab8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:01.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.488+0000 7fdac57fa700 1 -- 172.21.15.16:0/2481662721 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab8004480 con 0x7fdac809d630 2023-09-30T22:32:01.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.488+0000 7fdac57fa700 1 -- 172.21.15.16:0/2481662721 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdab801f070 con 0x7fdac809d630 2023-09-30T22:32:01.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.488+0000 7fdac57fa700 1 -- 172.21.15.16:0/2481662721 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab80038e0 con 0x7fdac809d630 2023-09-30T22:32:01.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.488+0000 7fdacee04700 1 -- 172.21.15.16:0/2481662721 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdab80094a0 con 0x7fdac809d630 2023-09-30T22:32:01.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.489+0000 7fdacee04700 1 -- 172.21.15.16:0/2481662721 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdac80a0410 con 0x7fdac809d630 2023-09-30T22:32:01.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.490+0000 7fdacee04700 1 -- 172.21.15.16:0/2481662721 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdac80a1800 con 0x7fdac809d630 2023-09-30T22:32:01.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.490+0000 7fdac57fa700 1 -- 172.21.15.16:0/2481662721 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdab8021030 con 0x7fdac809d630 2023-09-30T22:32:01.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.491+0000 7fdac57fa700 1 -- 172.21.15.16:0/2481662721 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fdab8040c80 con 0x7fdac809d630 2023-09-30T22:32:01.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.495+0000 7fdac57fa700 1 -- 172.21.15.16:0/2481662721 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdab8023790 con 0x7fdac809d630 2023-09-30T22:32:01.509 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:32:01.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.613+0000 7f571994a700 1 Processor -- start 2023-09-30T22:32:01.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.614+0000 7f571994a700 1 -- start start 2023-09-30T22:32:01.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.614+0000 7f571994a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f571409d6f0 0x7f571409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:01.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.614+0000 7f571994a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f571409e000 0x7f5714130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:01.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.614+0000 7f571994a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f571409f130 con 0x7f571409e410 2023-09-30T22:32:01.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.614+0000 7f571994a700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f571409f520 con 0x7f571409d6f0 2023-09-30T22:32:01.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.614+0000 7f571994a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f571409e930 con 0x7f571409e000 2023-09-30T22:32:01.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.615+0000 7f57137fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f571409e410 0x7f5714130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60810/0 (socket says 172.21.15.16:60810) 2023-09-30T22:32:01.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.615+0000 7f57137fe700 1 -- 172.21.15.16:0/2445784796 learned_addr learned my addr 172.21.15.16:0/2445784796 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:01.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.615+0000 7f5712ffd700 1 --2- 172.21.15.16:0/2445784796 >> v2:172.21.15.16:3300/0 conn(0x7f571409d6f0 0x7f571409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:01.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.615+0000 7f57127fc700 1 --2- 172.21.15.16:0/2445784796 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f571409e000 0x7f5714130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:01.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.615+0000 7f5711ffb700 1 -- 172.21.15.16:0/2445784796 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2788479224 0 0) 0x7f571409f130 con 0x7f571409e410 2023-09-30T22:32:01.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.615+0000 7f5711ffb700 1 -- 172.21.15.16:0/2445784796 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56fc0035b0 con 0x7f571409e410 2023-09-30T22:32:01.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.615+0000 7f5712ffd700 1 -- 172.21.15.16:0/2445784796 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f571409e000 msgr2=0x7f5714130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:01.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.615+0000 7f5712ffd700 1 --2- 172.21.15.16:0/2445784796 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f571409e000 0x7f5714130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:01.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.615+0000 7f5712ffd700 1 -- 172.21.15.16:0/2445784796 >> v1:172.21.15.16:6789/0 conn(0x7f571409e410 legacy=0x7f5714130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:01.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.615+0000 7f5712ffd700 1 -- 172.21.15.16:0/2445784796 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f571409b2d0 con 0x7f571409d6f0 2023-09-30T22:32:01.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.615+0000 7f57127fc700 1 --2- 172.21.15.16:0/2445784796 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f571409e000 0x7f5714130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:01.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.615+0000 7f5712ffd700 1 --2- 172.21.15.16:0/2445784796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f571409d6f0 0x7f571409dac0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f5704009840 tx=0x7f5704009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93805ec7539ddf03 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:01.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.615+0000 7f5711ffb700 1 -- 172.21.15.16:0/2445784796 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5704004230 con 0x7f571409d6f0 2023-09-30T22:32:01.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.616+0000 7f5711ffb700 1 -- 172.21.15.16:0/2445784796 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5704004390 con 0x7f571409d6f0 2023-09-30T22:32:01.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.616+0000 7f5711ffb700 1 -- 172.21.15.16:0/2445784796 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57040036a0 con 0x7f571409d6f0 2023-09-30T22:32:01.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.617+0000 7f571994a700 1 -- 172.21.15.16:0/2445784796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f571409d6f0 msgr2=0x7f571409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:01.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.617+0000 7f571994a700 1 --2- 172.21.15.16:0/2445784796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f571409d6f0 0x7f571409dac0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f5704009840 tx=0x7f5704009b50 comp rx=0 tx=0).stop 2023-09-30T22:32:01.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.617+0000 7f571994a700 1 -- 172.21.15.16:0/2445784796 shutdown_connections 2023-09-30T22:32:01.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.617+0000 7f571994a700 1 --2- 172.21.15.16:0/2445784796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f571409d6f0 0x7f571409dac0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:01.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.617+0000 7f571994a700 1 --2- 172.21.15.16:0/2445784796 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f571409e000 0x7f5714130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:01.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.617+0000 7f571994a700 1 -- 172.21.15.16:0/2445784796 >> 172.21.15.16:0/2445784796 conn(0x7f5714066f50 msgr2=0x7f571409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:01.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.617+0000 7f571994a700 1 -- 172.21.15.16:0/2445784796 shutdown_connections 2023-09-30T22:32:01.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.617+0000 7f571994a700 1 -- 172.21.15.16:0/2445784796 wait complete. 2023-09-30T22:32:01.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.618+0000 7f571994a700 1 Processor -- start 2023-09-30T22:32:01.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.618+0000 7f571994a700 1 -- start start 2023-09-30T22:32:01.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.619+0000 7f571994a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f571409d6f0 0x7f57141c6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:01.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.619+0000 7f571994a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f571409e000 0x7f57141c6740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:01.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.619+0000 7f571994a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f571409e410 0x7f5714131630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:01.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.619+0000 7f571994a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f571409c700 con 0x7f571409e410 2023-09-30T22:32:01.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.619+0000 7f571994a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57141350b0 con 0x7f571409d6f0 2023-09-30T22:32:01.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.619+0000 7f571994a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f57141c6c80 con 0x7f571409e000 2023-09-30T22:32:01.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.619+0000 7f57127fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f571409e000 0x7f57141c6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:01.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.619+0000 7f57127fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f571409e000 0x7f57141c6740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:34358/0 (socket says 172.21.15.16:34358) 2023-09-30T22:32:01.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.619+0000 7f57127fc700 1 -- 172.21.15.16:0/8743249 learned_addr learned my addr 172.21.15.16:0/8743249 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:01.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.619+0000 7f5712ffd700 1 --2- 172.21.15.16:0/8743249 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f571409d6f0 0x7f57141c6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:01.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.619+0000 7f57137fe700 1 --2- 172.21.15.16:0/8743249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f571409e410 0x7f5714131630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:01.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.620+0000 7f57137fe700 1 -- 172.21.15.16:0/8743249 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f571409e000 msgr2=0x7f57141c6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:01.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.620+0000 7f57137fe700 1 --2- 172.21.15.16:0/8743249 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f571409e000 0x7f57141c6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:01.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.620+0000 7f57137fe700 1 -- 172.21.15.16:0/8743249 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f571409d6f0 msgr2=0x7f57141c6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:01.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.620+0000 7f57137fe700 1 --2- 172.21.15.16:0/8743249 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f571409d6f0 0x7f57141c6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:01.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.620+0000 7f57137fe700 1 -- 172.21.15.16:0/8743249 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57080094a0 con 0x7f571409e410 2023-09-30T22:32:01.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.620+0000 7f5712ffd700 1 --2- 172.21.15.16:0/8743249 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f571409d6f0 0x7f57141c6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:01.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.620+0000 7f57137fe700 1 --2- 172.21.15.16:0/8743249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f571409e410 0x7f5714131630 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f570c00cbe0 tx=0x7f570c00cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:01.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.620+0000 7f56fbfff700 1 -- 172.21.15.16:0/8743249 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f570c00aaa0 con 0x7f571409e410 2023-09-30T22:32:01.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.620+0000 7f56fbfff700 1 -- 172.21.15.16:0/8743249 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f570c00ac00 con 0x7f571409e410 2023-09-30T22:32:01.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.620+0000 7f571994a700 1 -- 172.21.15.16:0/8743249 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57040094a0 con 0x7f571409e410 2023-09-30T22:32:01.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.620+0000 7f56fbfff700 1 -- 172.21.15.16:0/8743249 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f570c0070d0 con 0x7f571409e410 2023-09-30T22:32:01.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.620+0000 7f571994a700 1 -- 172.21.15.16:0/8743249 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5714133990 con 0x7f571409e410 2023-09-30T22:32:01.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.621+0000 7f56fbfff700 1 -- 172.21.15.16:0/8743249 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f570c007230 con 0x7f571409e410 2023-09-30T22:32:01.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.622+0000 7f56fbfff700 1 -- 172.21.15.16:0/8743249 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f570c03f3c0 con 0x7f571409e410 2023-09-30T22:32:01.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.622+0000 7f571994a700 1 -- 172.21.15.16:0/8743249 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5714074ea0 con 0x7f571409e410 2023-09-30T22:32:01.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.627+0000 7f56fbfff700 1 -- 172.21.15.16:0/8743249 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f570c018080 con 0x7f571409e410 2023-09-30T22:32:01.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.634+0000 7f2c1ddd1700 1 Processor -- start 2023-09-30T22:32:01.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.634+0000 7f2c1ddd1700 1 -- start start 2023-09-30T22:32:01.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.634+0000 7f2c1ddd1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c1806c0e0 0x7f2c1809ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:01.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.634+0000 7f2c1ddd1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c181380f0 0x7f2c1813fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:01.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.634+0000 7f2c1ddd1700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c18134980 con 0x7f2c18134ec0 2023-09-30T22:32:01.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.634+0000 7f2c1ddd1700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1806c4b0 con 0x7f2c1806c0e0 2023-09-30T22:32:01.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.634+0000 7f2c1ddd1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2c180a09c0 con 0x7f2c181380f0 2023-09-30T22:32:01.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.635+0000 7f2c17fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c181380f0 0x7f2c1813fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:01.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.635+0000 7f2c17fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c181380f0 0x7f2c1813fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:34368/0 (socket says 172.21.15.16:34368) 2023-09-30T22:32:01.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.635+0000 7f2c17fff700 1 -- 172.21.15.16:0/4226501769 learned_addr learned my addr 172.21.15.16:0/4226501769 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:01.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.635+0000 7f2c177fe700 1 --2- 172.21.15.16:0/4226501769 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c1806c0e0 0x7f2c1809ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:01.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.635+0000 7f2c17fff700 1 -- 172.21.15.16:0/4226501769 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c1806c0e0 msgr2=0x7f2c1809ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:01.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.635+0000 7f2c17fff700 1 --2- 172.21.15.16:0/4226501769 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c1806c0e0 0x7f2c1809ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:01.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.635+0000 7f2c17fff700 1 -- 172.21.15.16:0/4226501769 >> v1:172.21.15.16:6789/0 conn(0x7f2c18134ec0 legacy=0x7f2c181379e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:01.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.635+0000 7f2c17fff700 1 -- 172.21.15.16:0/4226501769 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c180a0600 con 0x7f2c181380f0 2023-09-30T22:32:01.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.635+0000 7f2c177fe700 1 --2- 172.21.15.16:0/4226501769 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c1806c0e0 0x7f2c1809ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:01.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.635+0000 7f2c17fff700 1 --2- 172.21.15.16:0/4226501769 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c181380f0 0x7f2c1813fd20 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2c10009e80 tx=0x7f2c1000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d16c86e3126eb58 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:01.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.636+0000 7f2c167fc700 1 -- 172.21.15.16:0/4226501769 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c1000e070 con 0x7f2c181380f0 2023-09-30T22:32:01.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.636+0000 7f2c167fc700 1 -- 172.21.15.16:0/4226501769 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c1000b910 con 0x7f2c181380f0 2023-09-30T22:32:01.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.636+0000 7f2c167fc700 1 -- 172.21.15.16:0/4226501769 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c1000bcc0 con 0x7f2c181380f0 2023-09-30T22:32:01.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.636+0000 7f2c1ddd1700 1 -- 172.21.15.16:0/4226501769 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c181380f0 msgr2=0x7f2c1813fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:01.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.636+0000 7f2c1ddd1700 1 --2- 172.21.15.16:0/4226501769 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c181380f0 0x7f2c1813fd20 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2c10009e80 tx=0x7f2c1000b330 comp rx=0 tx=0).stop 2023-09-30T22:32:01.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.636+0000 7f2c1ddd1700 1 -- 172.21.15.16:0/4226501769 shutdown_connections 2023-09-30T22:32:01.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.636+0000 7f2c1ddd1700 1 --2- 172.21.15.16:0/4226501769 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c181380f0 0x7f2c1813fd20 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:01.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.636+0000 7f2c1ddd1700 1 --2- 172.21.15.16:0/4226501769 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c1806c0e0 0x7f2c1809ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:01.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.636+0000 7f2c1ddd1700 1 -- 172.21.15.16:0/4226501769 >> 172.21.15.16:0/4226501769 conn(0x7f2c180a4210 msgr2=0x7f2c180a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:01.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.636+0000 7f2c1ddd1700 1 -- 172.21.15.16:0/4226501769 shutdown_connections 2023-09-30T22:32:01.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.637+0000 7f2c1ddd1700 1 -- 172.21.15.16:0/4226501769 wait complete. 2023-09-30T22:32:01.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.637+0000 7f2c1ddd1700 1 Processor -- start 2023-09-30T22:32:01.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.638+0000 7f2c1ddd1700 1 -- start start 2023-09-30T22:32:01.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.638+0000 7f2c1ddd1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c1806c0e0 0x7f2c1811a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:01.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.638+0000 7f2c1ddd1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c18134ec0 0x7f2c1811abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:01.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.638+0000 7f2c1ddd1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c181380f0 0x7f2c1811b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:01.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.638+0000 7f2c1ddd1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1811da50 con 0x7f2c181380f0 2023-09-30T22:32:01.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.638+0000 7f2c1ddd1700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c181343a0 con 0x7f2c1806c0e0 2023-09-30T22:32:01.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.638+0000 7f2c1ddd1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2c181272a0 con 0x7f2c18134ec0 2023-09-30T22:32:01.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.638+0000 7f2c177fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c1806c0e0 0x7f2c1811a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:01.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.638+0000 7f2c17fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c181380f0 0x7f2c1811b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:01.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.639+0000 7f2c17fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c181380f0 0x7f2c1811b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54054/0 (socket says 172.21.15.16:54054) 2023-09-30T22:32:01.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.639+0000 7f2c16ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c18134ec0 0x7f2c1811abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:01.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.639+0000 7f2c17fff700 1 -- 172.21.15.16:0/3460593361 learned_addr learned my addr 172.21.15.16:0/3460593361 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:01.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.639+0000 7f2c177fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c1806c0e0 0x7f2c1811a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:44272/0 (socket says 172.21.15.16:44272) 2023-09-30T22:32:01.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.639+0000 7f2c17fff700 1 -- 172.21.15.16:0/3460593361 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c18134ec0 msgr2=0x7f2c1811abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:01.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.639+0000 7f2c17fff700 1 --2- 172.21.15.16:0/3460593361 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c18134ec0 0x7f2c1811abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:01.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.639+0000 7f2c17fff700 1 -- 172.21.15.16:0/3460593361 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c1806c0e0 msgr2=0x7f2c1811a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:01.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.639+0000 7f2c17fff700 1 --2- 172.21.15.16:0/3460593361 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c1806c0e0 0x7f2c1811a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:01.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.639+0000 7f2c17fff700 1 -- 172.21.15.16:0/3460593361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c080094a0 con 0x7f2c181380f0 2023-09-30T22:32:01.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.639+0000 7f2c177fe700 1 --2- 172.21.15.16:0/3460593361 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c1806c0e0 0x7f2c1811a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:01.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.639+0000 7f2c17fff700 1 --2- 172.21.15.16:0/3460593361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c181380f0 0x7f2c1811b100 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f2c180a0b00 tx=0x7f2c10004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:01.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.639+0000 7f2bfffff700 1 -- 172.21.15.16:0/3460593361 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c10004450 con 0x7f2c181380f0 2023-09-30T22:32:01.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.639+0000 7f2bfffff700 1 -- 172.21.15.16:0/3460593361 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c100092e0 con 0x7f2c181380f0 2023-09-30T22:32:01.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.639+0000 7f2c1ddd1700 1 -- 172.21.15.16:0/3460593361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c10009b30 con 0x7f2c181380f0 2023-09-30T22:32:01.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.639+0000 7f2bfffff700 1 -- 172.21.15.16:0/3460593361 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c10004830 con 0x7f2c181380f0 2023-09-30T22:32:01.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.640+0000 7f2c1ddd1700 1 -- 172.21.15.16:0/3460593361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c1811e600 con 0x7f2c181380f0 2023-09-30T22:32:01.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.640+0000 7f2bfffff700 1 -- 172.21.15.16:0/3460593361 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2c1001c410 con 0x7f2c181380f0 2023-09-30T22:32:01.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.641+0000 7f2c1ddd1700 1 -- 172.21.15.16:0/3460593361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c18074ea0 con 0x7f2c181380f0 2023-09-30T22:32:01.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.641+0000 7f2bfffff700 1 -- 172.21.15.16:0/3460593361 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f2c100410b0 con 0x7f2c181380f0 2023-09-30T22:32:01.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.650+0000 7f2bfffff700 1 -- 172.21.15.16:0/3460593361 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2c1001b7b0 con 0x7f2c181380f0 2023-09-30T22:32:01.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.667+0000 7f5b7ee78700 1 Processor -- start 2023-09-30T22:32:01.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.667+0000 7f5b7ee78700 1 -- start start 2023-09-30T22:32:01.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.667+0000 7f5b7ee78700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5b7809d6f0 0x7f5b7809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:01.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.667+0000 7f5b7ee78700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5b7809e000 0x7f5b781339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:01.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.667+0000 7f5b7ee78700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5b7809e410 0x7f5b78133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:01.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.668+0000 7f5b7ee78700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b7806b460 con 0x7f5b7809d6f0 2023-09-30T22:32:01.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.668+0000 7f5b7ee78700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7809b120 con 0x7f5b7809e410 2023-09-30T22:32:01.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.668+0000 7f5b7ee78700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7809a7e0 con 0x7f5b7809e000 2023-09-30T22:32:01.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.668+0000 7f5b77fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5b7809e000 0x7f5b781339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:01.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.668+0000 7f5b77fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5b7809e000 0x7f5b781339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:34392/0 (socket says 172.21.15.16:34392) 2023-09-30T22:32:01.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.668+0000 7f5b77fff700 1 -- 172.21.15.16:0/2128022462 learned_addr learned my addr 172.21.15.16:0/2128022462 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:01.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.668+0000 7f5b7d415700 1 --2- 172.21.15.16:0/2128022462 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5b7809e410 0x7f5b78133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:01.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.668+0000 7f5b7d415700 1 -- 172.21.15.16:0/2128022462 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5b7809e000 msgr2=0x7f5b781339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:01.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.668+0000 7f5b7d415700 1 --2- 172.21.15.16:0/2128022462 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5b7809e000 0x7f5b781339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:01.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.668+0000 7f5b7d415700 1 -- 172.21.15.16:0/2128022462 >> v2:172.21.15.16:3300/0 conn(0x7f5b7809d6f0 msgr2=0x7f5b7809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:01.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.668+0000 7f5b7d415700 1 --2- 172.21.15.16:0/2128022462 >> v2:172.21.15.16:3300/0 conn(0x7f5b7809d6f0 0x7f5b7809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:01.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.668+0000 7f5b7d415700 1 -- 172.21.15.16:0/2128022462 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b7809a500 con 0x7f5b7809e410 2023-09-30T22:32:01.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.668+0000 7f5b77fff700 1 --2- 172.21.15.16:0/2128022462 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5b7809e000 0x7f5b781339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:01.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.669+0000 7f5b7d415700 1 --2- 172.21.15.16:0/2128022462 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5b7809e410 0x7f5b78133f30 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5b70009dc0 tx=0x7f5b7000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d7a03ccf5376025 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:01.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.669+0000 7f5b777fe700 1 -- 172.21.15.16:0/2128022462 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7000e070 con 0x7f5b7809e410 2023-09-30T22:32:01.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.669+0000 7f5b777fe700 1 -- 172.21.15.16:0/2128022462 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b7000bae0 con 0x7f5b7809e410 2023-09-30T22:32:01.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.669+0000 7f5b777fe700 1 -- 172.21.15.16:0/2128022462 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7000be90 con 0x7f5b7809e410 2023-09-30T22:32:01.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.669+0000 7f5b7ee78700 1 -- 172.21.15.16:0/2128022462 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5b7809e410 msgr2=0x7f5b78133f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:01.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.669+0000 7f5b7ee78700 1 --2- 172.21.15.16:0/2128022462 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5b7809e410 0x7f5b78133f30 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5b70009dc0 tx=0x7f5b7000b330 comp rx=0 tx=0).stop 2023-09-30T22:32:01.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.669+0000 7f5b7ee78700 1 -- 172.21.15.16:0/2128022462 shutdown_connections 2023-09-30T22:32:01.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.669+0000 7f5b7ee78700 1 --2- 172.21.15.16:0/2128022462 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5b7809e410 0x7f5b78133f30 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:01.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.669+0000 7f5b7ee78700 1 --2- 172.21.15.16:0/2128022462 >> v2:172.21.15.16:3300/0 conn(0x7f5b7809d6f0 0x7f5b7809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:01.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.669+0000 7f5b7ee78700 1 --2- 172.21.15.16:0/2128022462 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5b7809e000 0x7f5b781339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:01.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.670+0000 7f5b7ee78700 1 -- 172.21.15.16:0/2128022462 >> 172.21.15.16:0/2128022462 conn(0x7f5b78066f50 msgr2=0x7f5b7809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:01.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.670+0000 7f5b7ee78700 1 -- 172.21.15.16:0/2128022462 shutdown_connections 2023-09-30T22:32:01.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.670+0000 7f5b7ee78700 1 -- 172.21.15.16:0/2128022462 wait complete. 2023-09-30T22:32:01.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.671+0000 7f5b7ee78700 1 Processor -- start 2023-09-30T22:32:01.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.671+0000 7f5b7ee78700 1 -- start start 2023-09-30T22:32:01.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.671+0000 7f5b7ee78700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5b7809d6f0 0x7f5b781c8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:01.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.671+0000 7f5b7ee78700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5b7809e000 0x7f5b781c8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:01.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.671+0000 7f5b7ee78700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b7809e410 0x7f5b781c8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:01.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.671+0000 7f5b7ee78700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b781a7e30 con 0x7f5b7809e410 2023-09-30T22:32:01.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.671+0000 7f5b7ee78700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b781a64c0 con 0x7f5b7809e000 2023-09-30T22:32:01.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.671+0000 7f5b7ee78700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5b781a84b0 con 0x7f5b7809d6f0 2023-09-30T22:32:01.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.671+0000 7f5b77fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5b7809e000 0x7f5b781c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:01.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.671+0000 7f5b7d415700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b7809e410 0x7f5b781c8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:01.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.672+0000 7f5b77fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5b7809e000 0x7f5b781c8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:44292/0 (socket says 172.21.15.16:44292) 2023-09-30T22:32:01.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.672+0000 7f5b77fff700 1 -- 172.21.15.16:0/3706812993 learned_addr learned my addr 172.21.15.16:0/3706812993 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:01.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.672+0000 7f5b7d415700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b7809e410 0x7f5b781c8ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54070/0 (socket says 172.21.15.16:54070) 2023-09-30T22:32:01.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.672+0000 7f5b7cc14700 1 --2- 172.21.15.16:0/3706812993 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5b7809d6f0 0x7f5b781c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:01.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.672+0000 7f5b77fff700 1 -- 172.21.15.16:0/3706812993 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5b7809d6f0 msgr2=0x7f5b781c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:01.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.672+0000 7f5b77fff700 1 --2- 172.21.15.16:0/3706812993 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5b7809d6f0 0x7f5b781c8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:01.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.672+0000 7f5b77fff700 1 -- 172.21.15.16:0/3706812993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b7809e410 msgr2=0x7f5b781c8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:01.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.672+0000 7f5b77fff700 1 --2- 172.21.15.16:0/3706812993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b7809e410 0x7f5b781c8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:01.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.672+0000 7f5b77fff700 1 -- 172.21.15.16:0/3706812993 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b64009550 con 0x7f5b7809e000 2023-09-30T22:32:01.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.672+0000 7f5b7d415700 1 --2- 172.21.15.16:0/3706812993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b7809e410 0x7f5b781c8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:01.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.672+0000 7f5b77fff700 1 --2- 172.21.15.16:0/3706812993 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5b7809e000 0x7f5b781c8660 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5b6400ec10 tx=0x7f5b6400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:01.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.672+0000 7f5b757fa700 1 -- 172.21.15.16:0/3706812993 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b6400c850 con 0x7f5b7809e000 2023-09-30T22:32:01.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.672+0000 7f5b757fa700 1 -- 172.21.15.16:0/3706812993 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b6400c9b0 con 0x7f5b7809e000 2023-09-30T22:32:01.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.672+0000 7f5b7ee78700 1 -- 172.21.15.16:0/3706812993 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b70009a70 con 0x7f5b7809e000 2023-09-30T22:32:01.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.672+0000 7f5b757fa700 1 -- 172.21.15.16:0/3706812993 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b6400cd20 con 0x7f5b7809e000 2023-09-30T22:32:01.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.673+0000 7f5b7ee78700 1 -- 172.21.15.16:0/3706812993 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b7809c710 con 0x7f5b7809e000 2023-09-30T22:32:01.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.674+0000 7f5b757fa700 1 -- 172.21.15.16:0/3706812993 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5b64013470 con 0x7f5b7809e000 2023-09-30T22:32:01.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.674+0000 7f5b7ee78700 1 -- 172.21.15.16:0/3706812993 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b781b9420 con 0x7f5b7809e000 2023-09-30T22:32:01.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.674+0000 7f5b757fa700 1 -- 172.21.15.16:0/3706812993 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f5b6403fcc0 con 0x7f5b7809e000 2023-09-30T22:32:01.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.679+0000 7f5b757fa700 1 -- 172.21.15.16:0/3706812993 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5b781b9420 con 0x7f5b7809e000 2023-09-30T22:32:01.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.716+0000 7fdacee04700 1 -- 172.21.15.16:0/2481662721 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fdac80747f0 con 0x7fdac809d630 2023-09-30T22:32:01.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.716+0000 7fdac57fa700 1 -- 172.21.15.16:0/2481662721 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdab8015070 con 0x7fdac809d630 2023-09-30T22:32:01.717 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:32:01.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.719+0000 7fdacee04700 1 -- 172.21.15.16:0/2481662721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdac809d630 msgr2=0x7fdac81c7ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:01.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.719+0000 7fdacee04700 1 --2- 172.21.15.16:0/2481662721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdac809d630 0x7fdac81c7ef0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fdab8009fd0 tx=0x7fdab8004750 comp rx=0 tx=0).stop 2023-09-30T22:32:01.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.720+0000 7fdacee04700 1 -- 172.21.15.16:0/2481662721 shutdown_connections 2023-09-30T22:32:01.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.720+0000 7fdacee04700 1 --2- 172.21.15.16:0/2481662721 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdac8133930 0x7fdac81c8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:01.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.720+0000 7fdacee04700 1 --2- 172.21.15.16:0/2481662721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdac809d630 0x7fdac81c7ef0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:01.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.720+0000 7fdacee04700 1 --2- 172.21.15.16:0/2481662721 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdac80a0a30 0x7fdac81c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:01.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.720+0000 7fdacee04700 1 -- 172.21.15.16:0/2481662721 >> 172.21.15.16:0/2481662721 conn(0x7fdac80a3050 msgr2=0x7fdac81bba60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:01.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.720+0000 7fdacee04700 1 -- 172.21.15.16:0/2481662721 shutdown_connections 2023-09-30T22:32:01.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.720+0000 7fdacee04700 1 -- 172.21.15.16:0/2481662721 wait complete. 2023-09-30T22:32:01.735 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:32:01.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.848+0000 7f571994a700 1 -- 172.21.15.16:0/8743249 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f571406c360 con 0x7f571409e410 2023-09-30T22:32:01.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.849+0000 7f56fbfff700 1 -- 172.21.15.16:0/8743249 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f570c0074e0 con 0x7f571409e410 2023-09-30T22:32:01.849 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:32:01.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.851+0000 7f571994a700 1 -- 172.21.15.16:0/8743249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f571409e410 msgr2=0x7f5714131630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:01.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.851+0000 7f571994a700 1 --2- 172.21.15.16:0/8743249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f571409e410 0x7f5714131630 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f570c00cbe0 tx=0x7f570c00cef0 comp rx=0 tx=0).stop 2023-09-30T22:32:01.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.851+0000 7f571994a700 1 -- 172.21.15.16:0/8743249 shutdown_connections 2023-09-30T22:32:01.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.851+0000 7f571994a700 1 --2- 172.21.15.16:0/8743249 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f571409d6f0 0x7f57141c6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:01.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.851+0000 7f571994a700 1 --2- 172.21.15.16:0/8743249 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f571409e410 0x7f5714131630 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:01.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.851+0000 7f571994a700 1 --2- 172.21.15.16:0/8743249 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f571409e000 0x7f57141c6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:01.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.851+0000 7f571994a700 1 -- 172.21.15.16:0/8743249 >> 172.21.15.16:0/8743249 conn(0x7f5714066f50 msgr2=0x7f5714070f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:01.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.851+0000 7f571994a700 1 -- 172.21.15.16:0/8743249 shutdown_connections 2023-09-30T22:32:01.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.851+0000 7f571994a700 1 -- 172.21.15.16:0/8743249 wait complete. 2023-09-30T22:32:01.866 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:32:01.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.874+0000 7f2c1ddd1700 1 -- 172.21.15.16:0/3460593361 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2c1806b250 con 0x7f2c181380f0 2023-09-30T22:32:01.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.874+0000 7f2bfffff700 1 -- 172.21.15.16:0/3460593361 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2c1001b3a0 con 0x7f2c181380f0 2023-09-30T22:32:01.875 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:32:01.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.876+0000 7f2c1ddd1700 1 -- 172.21.15.16:0/3460593361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c181380f0 msgr2=0x7f2c1811b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:01.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.876+0000 7f2c1ddd1700 1 --2- 172.21.15.16:0/3460593361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c181380f0 0x7f2c1811b100 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f2c180a0b00 tx=0x7f2c10004ad0 comp rx=0 tx=0).stop 2023-09-30T22:32:01.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.877+0000 7f2c1ddd1700 1 -- 172.21.15.16:0/3460593361 shutdown_connections 2023-09-30T22:32:01.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.877+0000 7f2c1ddd1700 1 --2- 172.21.15.16:0/3460593361 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c1806c0e0 0x7f2c1811a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:01.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.877+0000 7f2c1ddd1700 1 --2- 172.21.15.16:0/3460593361 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c181380f0 0x7f2c1811b100 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:01.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.877+0000 7f2c1ddd1700 1 --2- 172.21.15.16:0/3460593361 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c18134ec0 0x7f2c1811abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:01.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.877+0000 7f2c1ddd1700 1 -- 172.21.15.16:0/3460593361 >> 172.21.15.16:0/3460593361 conn(0x7f2c180a4210 msgr2=0x7f2c180a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:01.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.877+0000 7f2c1ddd1700 1 -- 172.21.15.16:0/3460593361 shutdown_connections 2023-09-30T22:32:01.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.877+0000 7f2c1ddd1700 1 -- 172.21.15.16:0/3460593361 wait complete. 2023-09-30T22:32:01.891 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:32:01.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.901+0000 7f5b7ee78700 1 -- 172.21.15.16:0/3706812993 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5b781b9420 con 0x7f5b7809e000 2023-09-30T22:32:01.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.901+0000 7f5b757fa700 1 -- 172.21.15.16:0/3706812993 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5b781b9420 con 0x7f5b7809e000 2023-09-30T22:32:01.902 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:32:01.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.904+0000 7f5b7ee78700 1 -- 172.21.15.16:0/3706812993 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5b7809e000 msgr2=0x7f5b781c8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:01.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.904+0000 7f5b7ee78700 1 --2- 172.21.15.16:0/3706812993 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5b7809e000 0x7f5b781c8660 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5b6400ec10 tx=0x7f5b6400ef20 comp rx=0 tx=0).stop 2023-09-30T22:32:01.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.904+0000 7f5b7ee78700 1 -- 172.21.15.16:0/3706812993 shutdown_connections 2023-09-30T22:32:01.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.904+0000 7f5b7ee78700 1 --2- 172.21.15.16:0/3706812993 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b7809e410 0x7f5b781c8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:01.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.904+0000 7f5b7ee78700 1 --2- 172.21.15.16:0/3706812993 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5b7809d6f0 0x7f5b781c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:01.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.904+0000 7f5b7ee78700 1 --2- 172.21.15.16:0/3706812993 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5b7809e000 0x7f5b781c8660 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:01.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.904+0000 7f5b7ee78700 1 -- 172.21.15.16:0/3706812993 >> 172.21.15.16:0/3706812993 conn(0x7f5b78066f50 msgr2=0x7f5b781bbb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:01.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.904+0000 7f5b7ee78700 1 -- 172.21.15.16:0/3706812993 shutdown_connections 2023-09-30T22:32:01.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:01.904+0000 7f5b7ee78700 1 -- 172.21.15.16:0/3706812993 wait complete. 2023-09-30T22:32:01.918 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:32:02.736 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:32:02.867 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:32:02.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:02.882+0000 7f12fb83e700 1 Processor -- start 2023-09-30T22:32:02.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:02.882+0000 7f12fb83e700 1 -- start start 2023-09-30T22:32:02.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:02.883+0000 7f12fb83e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f12f409d6f0 0x7f12f409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:02.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:02.883+0000 7f12fb83e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12f409e440 0x7f12f4134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:02.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:02.883+0000 7f12fb83e700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12f409b120 con 0x7f12f409e000 2023-09-30T22:32:02.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:02.883+0000 7f12fb83e700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12f406b460 con 0x7f12f409d6f0 2023-09-30T22:32:02.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:02.883+0000 7f12fb83e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f12f409a7e0 con 0x7f12f409e440 2023-09-30T22:32:02.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:02.883+0000 7f12f9ddb700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12f409e440 0x7f12f4134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:02.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:02.883+0000 7f12f9ddb700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12f409e440 0x7f12f4134100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:34412/0 (socket says 172.21.15.16:34412) 2023-09-30T22:32:02.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:02.883+0000 7f12f9ddb700 1 -- 172.21.15.16:0/3544144504 learned_addr learned my addr 172.21.15.16:0/3544144504 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:02.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:02.883+0000 7f12f95da700 1 --2- 172.21.15.16:0/3544144504 >> v2:172.21.15.16:3300/0 conn(0x7f12f409d6f0 0x7f12f409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:02.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:02.883+0000 7f12f9ddb700 1 -- 172.21.15.16:0/3544144504 >> v2:172.21.15.16:3300/0 conn(0x7f12f409d6f0 msgr2=0x7f12f409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:02.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:02.883+0000 7f12ebfff700 1 -- 172.21.15.16:0/3544144504 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1549432539 0 0) 0x7f12f409b120 con 0x7f12f409e000 2023-09-30T22:32:02.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:02.883+0000 7f12f9ddb700 1 --2- 172.21.15.16:0/3544144504 >> v2:172.21.15.16:3300/0 conn(0x7f12f409d6f0 0x7f12f409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:02.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:02.883+0000 7f12f9ddb700 1 -- 172.21.15.16:0/3544144504 >> v1:172.21.15.16:6789/0 conn(0x7f12f409e000 legacy=0x7f12f41339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:02.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:02.883+0000 7f12f9ddb700 1 -- 172.21.15.16:0/3544144504 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12f409a500 con 0x7f12f409e440 2023-09-30T22:32:02.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:02.884+0000 7f12f9ddb700 1 --2- 172.21.15.16:0/3544144504 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12f409e440 0x7f12f4134100 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f12ec009e80 tx=0x7f12ec00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7828c8e79ace9d4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:02.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:02.884+0000 7f12ebfff700 1 -- 172.21.15.16:0/3544144504 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ec00e070 con 0x7f12f409e440 2023-09-30T22:32:02.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:02.884+0000 7f12ebfff700 1 -- 172.21.15.16:0/3544144504 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12ec00b8d0 con 0x7f12f409e440 2023-09-30T22:32:02.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:02.884+0000 7f12ebfff700 1 -- 172.21.15.16:0/3544144504 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12ec00bc80 con 0x7f12f409e440 2023-09-30T22:32:02.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:02.884+0000 7f12fb83e700 1 -- 172.21.15.16:0/3544144504 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12f409e440 msgr2=0x7f12f4134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:02.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:02.884+0000 7f12fb83e700 1 --2- 172.21.15.16:0/3544144504 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12f409e440 0x7f12f4134100 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f12ec009e80 tx=0x7f12ec00b330 comp rx=0 tx=0).stop 2023-09-30T22:32:02.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:02.885+0000 7f12fb83e700 1 -- 172.21.15.16:0/3544144504 shutdown_connections 2023-09-30T22:32:02.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:02.885+0000 7f12fb83e700 1 --2- 172.21.15.16:0/3544144504 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12f409e440 0x7f12f4134100 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:02.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:02.885+0000 7f12fb83e700 1 --2- 172.21.15.16:0/3544144504 >> v2:172.21.15.16:3300/0 conn(0x7f12f409d6f0 0x7f12f409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:02.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:02.885+0000 7f12fb83e700 1 -- 172.21.15.16:0/3544144504 >> 172.21.15.16:0/3544144504 conn(0x7f12f4066f50 msgr2=0x7f12f409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:02.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:02.885+0000 7f12fb83e700 1 -- 172.21.15.16:0/3544144504 shutdown_connections 2023-09-30T22:32:02.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:02.885+0000 7f12fb83e700 1 -- 172.21.15.16:0/3544144504 wait complete. 2023-09-30T22:32:02.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:02.886+0000 7f12fb83e700 1 Processor -- start 2023-09-30T22:32:02.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:02.886+0000 7f12fb83e700 1 -- start start 2023-09-30T22:32:02.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:02.886+0000 7f12fb83e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12f409d6f0 0x7f12f41a73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:02.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:02.886+0000 7f12fb83e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12f409e000 0x7f12f41a78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:02.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:02.886+0000 7f12fb83e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12f409e440 0x7f12f41c7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:02.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:02.886+0000 7f12fb83e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12f41a6fe0 con 0x7f12f409e440 2023-09-30T22:32:02.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:02.886+0000 7f12fb83e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12f409c710 con 0x7f12f409d6f0 2023-09-30T22:32:02.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:02.886+0000 7f12fb83e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f12f41a9f30 con 0x7f12f409e000 2023-09-30T22:32:02.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:02.886+0000 7f12f95da700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12f409d6f0 0x7f12f41a73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:02.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:02.886+0000 7f12f8dd9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12f409e000 0x7f12f41a78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:02.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:02.886+0000 7f12f95da700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12f409d6f0 0x7f12f41a73a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:44296/0 (socket says 172.21.15.16:44296) 2023-09-30T22:32:02.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:02.886+0000 7f12f95da700 1 -- 172.21.15.16:0/3401314027 learned_addr learned my addr 172.21.15.16:0/3401314027 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:02.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:02.887+0000 7f12f9ddb700 1 --2- 172.21.15.16:0/3401314027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12f409e440 0x7f12f41c7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:02.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:02.887+0000 7f12f95da700 1 -- 172.21.15.16:0/3401314027 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12f409e000 msgr2=0x7f12f41a78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:02.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:02.887+0000 7f12f95da700 1 --2- 172.21.15.16:0/3401314027 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12f409e000 0x7f12f41a78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:02.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:02.887+0000 7f12f95da700 1 -- 172.21.15.16:0/3401314027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12f409e440 msgr2=0x7f12f41c7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:02.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:02.887+0000 7f12f95da700 1 --2- 172.21.15.16:0/3401314027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12f409e440 0x7f12f41c7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:02.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:02.887+0000 7f12f95da700 1 -- 172.21.15.16:0/3401314027 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12ec009b30 con 0x7f12f409d6f0 2023-09-30T22:32:02.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:02.887+0000 7f12f8dd9700 1 --2- 172.21.15.16:0/3401314027 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12f409e000 0x7f12f41a78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:02.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:02.887+0000 7f12f95da700 1 --2- 172.21.15.16:0/3401314027 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12f409d6f0 0x7f12f41a73a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f12f000e9b0 tx=0x7f12f000ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:02.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:02.887+0000 7f12e9ffb700 1 -- 172.21.15.16:0/3401314027 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f0009710 con 0x7f12f409d6f0 2023-09-30T22:32:02.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:02.887+0000 7f12e9ffb700 1 -- 172.21.15.16:0/3401314027 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12f000ee50 con 0x7f12f409d6f0 2023-09-30T22:32:02.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:02.887+0000 7f12fb83e700 1 -- 172.21.15.16:0/3401314027 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12f41a7e20 con 0x7f12f409d6f0 2023-09-30T22:32:02.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:02.887+0000 7f12e9ffb700 1 -- 172.21.15.16:0/3401314027 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f000ca50 con 0x7f12f409d6f0 2023-09-30T22:32:02.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:02.888+0000 7f12fb83e700 1 -- 172.21.15.16:0/3401314027 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12f41aac60 con 0x7f12f409d6f0 2023-09-30T22:32:02.891 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:32:02.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:02.888+0000 7f12e9ffb700 1 -- 172.21.15.16:0/3401314027 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f12f000cbb0 con 0x7f12f409d6f0 2023-09-30T22:32:02.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:02.888+0000 7f12e9ffb700 1 -- 172.21.15.16:0/3401314027 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f12f003f770 con 0x7f12f409d6f0 2023-09-30T22:32:02.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:02.889+0000 7f12fb83e700 1 -- 172.21.15.16:0/3401314027 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12f41aa0f0 con 0x7f12f409d6f0 2023-09-30T22:32:02.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:02.897+0000 7f12e9ffb700 1 -- 172.21.15.16:0/3401314027 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f12f0021020 con 0x7f12f409d6f0 2023-09-30T22:32:02.919 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:32:03.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.024+0000 7f1a60b3b700 1 Processor -- start 2023-09-30T22:32:03.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.024+0000 7f1a60b3b700 1 -- start start 2023-09-30T22:32:03.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.025+0000 7f1a60b3b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1a5c09d6f0 0x7f1a5c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:03.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.025+0000 7f1a60b3b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1a5c09e000 0x7f1a5c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:03.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.025+0000 7f1a60b3b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1a5c09e410 0x7f1a5c133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:03.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.025+0000 7f1a60b3b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a5c06b460 con 0x7f1a5c09e000 2023-09-30T22:32:03.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.025+0000 7f1a60b3b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5c09b120 con 0x7f1a5c09e410 2023-09-30T22:32:03.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.025+0000 7f1a60b3b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5c09a7e0 con 0x7f1a5c09d6f0 2023-09-30T22:32:03.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.025+0000 7f1a59d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1a5c09e000 0x7f1a5c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:03.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.025+0000 7f1a5ad9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1a5c09e410 0x7f1a5c133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:03.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.025+0000 7f1a59d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1a5c09e000 0x7f1a5c1339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54108/0 (socket says 172.21.15.16:54108) 2023-09-30T22:32:03.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.025+0000 7f1a59d9b700 1 -- 172.21.15.16:0/2879656493 learned_addr learned my addr 172.21.15.16:0/2879656493 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:03.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.025+0000 7f1a5a59c700 1 --2- 172.21.15.16:0/2879656493 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1a5c09d6f0 0x7f1a5c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:03.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.025+0000 7f1a5ad9d700 1 -- 172.21.15.16:0/2879656493 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1a5c09d6f0 msgr2=0x7f1a5c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:03.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.025+0000 7f1a5ad9d700 1 --2- 172.21.15.16:0/2879656493 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1a5c09d6f0 0x7f1a5c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:03.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.025+0000 7f1a5ad9d700 1 -- 172.21.15.16:0/2879656493 >> v2:172.21.15.16:3300/0 conn(0x7f1a5c09e000 msgr2=0x7f1a5c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:03.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.025+0000 7f1a5ad9d700 1 --2- 172.21.15.16:0/2879656493 >> v2:172.21.15.16:3300/0 conn(0x7f1a5c09e000 0x7f1a5c1339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:03.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.025+0000 7f1a5ad9d700 1 -- 172.21.15.16:0/2879656493 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a5c09a500 con 0x7f1a5c09e410 2023-09-30T22:32:03.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.026+0000 7f1a59d9b700 1 --2- 172.21.15.16:0/2879656493 >> v2:172.21.15.16:3300/0 conn(0x7f1a5c09e000 0x7f1a5c1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:03.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.026+0000 7f1a5ad9d700 1 --2- 172.21.15.16:0/2879656493 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1a5c09e410 0x7f1a5c133f30 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f1a54009dc0 tx=0x7f1a5400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e47e9618d369a918 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:03.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.026+0000 7f1a5959a700 1 -- 172.21.15.16:0/2879656493 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5400e070 con 0x7f1a5c09e410 2023-09-30T22:32:03.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.026+0000 7f1a5959a700 1 -- 172.21.15.16:0/2879656493 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a5400bae0 con 0x7f1a5c09e410 2023-09-30T22:32:03.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.026+0000 7f1a5959a700 1 -- 172.21.15.16:0/2879656493 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5400be90 con 0x7f1a5c09e410 2023-09-30T22:32:03.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.027+0000 7f1a60b3b700 1 -- 172.21.15.16:0/2879656493 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1a5c09e410 msgr2=0x7f1a5c133f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:03.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.027+0000 7f1a60b3b700 1 --2- 172.21.15.16:0/2879656493 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1a5c09e410 0x7f1a5c133f30 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f1a54009dc0 tx=0x7f1a5400b330 comp rx=0 tx=0).stop 2023-09-30T22:32:03.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.027+0000 7f1a60b3b700 1 -- 172.21.15.16:0/2879656493 shutdown_connections 2023-09-30T22:32:03.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.027+0000 7f1a60b3b700 1 --2- 172.21.15.16:0/2879656493 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1a5c09e410 0x7f1a5c133f30 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:03.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.027+0000 7f1a60b3b700 1 --2- 172.21.15.16:0/2879656493 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1a5c09d6f0 0x7f1a5c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:03.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.027+0000 7f1a60b3b700 1 --2- 172.21.15.16:0/2879656493 >> v2:172.21.15.16:3300/0 conn(0x7f1a5c09e000 0x7f1a5c1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:03.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.027+0000 7f1a60b3b700 1 -- 172.21.15.16:0/2879656493 >> 172.21.15.16:0/2879656493 conn(0x7f1a5c066f50 msgr2=0x7f1a5c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:03.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.027+0000 7f1a60b3b700 1 -- 172.21.15.16:0/2879656493 shutdown_connections 2023-09-30T22:32:03.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.027+0000 7f1a60b3b700 1 -- 172.21.15.16:0/2879656493 wait complete. 2023-09-30T22:32:03.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.029+0000 7f1a60b3b700 1 Processor -- start 2023-09-30T22:32:03.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.029+0000 7f1a60b3b700 1 -- start start 2023-09-30T22:32:03.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.029+0000 7f1a60b3b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a5c09d6f0 0x7f1a5c1c5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:03.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.029+0000 7f1a60b3b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1a5c09e000 0x7f1a5c1c5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:03.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.029+0000 7f1a60b3b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1a5c09e410 0x7f1a5c1c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:03.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.029+0000 7f1a60b3b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5c1a7aa0 con 0x7f1a5c09d6f0 2023-09-30T22:32:03.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.029+0000 7f1a60b3b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5c1a78b0 con 0x7f1a5c09e000 2023-09-30T22:32:03.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.029+0000 7f1a60b3b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5c0a20d0 con 0x7f1a5c09e410 2023-09-30T22:32:03.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.030+0000 7f1a5ad9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1a5c09e410 0x7f1a5c1c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:03.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.030+0000 7f1a5ad9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1a5c09e410 0x7f1a5c1c8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:34444/0 (socket says 172.21.15.16:34444) 2023-09-30T22:32:03.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.030+0000 7f1a5ad9d700 1 -- 172.21.15.16:0/3587698961 learned_addr learned my addr 172.21.15.16:0/3587698961 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:03.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.030+0000 7f1a5a59c700 1 --2- 172.21.15.16:0/3587698961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a5c09d6f0 0x7f1a5c1c5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:03.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.030+0000 7f1a59d9b700 1 --2- 172.21.15.16:0/3587698961 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1a5c09e000 0x7f1a5c1c5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:03.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.030+0000 7f1a5ad9d700 1 -- 172.21.15.16:0/3587698961 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1a5c09e000 msgr2=0x7f1a5c1c5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:03.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.030+0000 7f1a5ad9d700 1 --2- 172.21.15.16:0/3587698961 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1a5c09e000 0x7f1a5c1c5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:03.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.030+0000 7f1a5ad9d700 1 -- 172.21.15.16:0/3587698961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a5c09d6f0 msgr2=0x7f1a5c1c5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:03.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.030+0000 7f1a5ad9d700 1 --2- 172.21.15.16:0/3587698961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a5c09d6f0 0x7f1a5c1c5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:03.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.030+0000 7f1a5ad9d700 1 -- 172.21.15.16:0/3587698961 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a50009550 con 0x7f1a5c09e410 2023-09-30T22:32:03.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.030+0000 7f1a5ad9d700 1 --2- 172.21.15.16:0/3587698961 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1a5c09e410 0x7f1a5c1c8040 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c09b3c0 tx=0x7f1a54004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:03.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.030+0000 7f1a4affd700 1 -- 172.21.15.16:0/3587698961 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a54004550 con 0x7f1a5c09e410 2023-09-30T22:32:03.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.031+0000 7f1a4affd700 1 -- 172.21.15.16:0/3587698961 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a54020070 con 0x7f1a5c09e410 2023-09-30T22:32:03.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.031+0000 7f1a4affd700 1 -- 172.21.15.16:0/3587698961 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a540048f0 con 0x7f1a5c09e410 2023-09-30T22:32:03.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.031+0000 7f1a60b3b700 1 -- 172.21.15.16:0/3587698961 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a54009a70 con 0x7f1a5c09e410 2023-09-30T22:32:03.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.031+0000 7f1a60b3b700 1 -- 172.21.15.16:0/3587698961 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a5c1a9810 con 0x7f1a5c09e410 2023-09-30T22:32:03.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.032+0000 7f1a4affd700 1 -- 172.21.15.16:0/3587698961 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1a54012430 con 0x7f1a5c09e410 2023-09-30T22:32:03.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.032+0000 7f1a48ff9700 1 -- 172.21.15.16:0/3587698961 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a340052e0 con 0x7f1a5c09e410 2023-09-30T22:32:03.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.032+0000 7f1a4affd700 1 -- 172.21.15.16:0/3587698961 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f1a54022070 con 0x7f1a5c09e410 2023-09-30T22:32:03.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.037+0000 7f1a4affd700 1 -- 172.21.15.16:0/3587698961 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1a54072750 con 0x7f1a5c09e410 2023-09-30T22:32:03.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.045+0000 7fefcca7c700 1 Processor -- start 2023-09-30T22:32:03.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.045+0000 7fefcca7c700 1 -- start start 2023-09-30T22:32:03.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.046+0000 7fefcca7c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fefc8133420 0x7fefc81337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:03.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.046+0000 7fefcca7c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fefc8133e30 0x7fefc812ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:03.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.046+0000 7fefcca7c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fefc8134260 0x7fefc81304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:03.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.046+0000 7fefcca7c700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefc806b990 con 0x7fefc8133420 2023-09-30T22:32:03.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.046+0000 7fefcca7c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefc80a0590 con 0x7fefc8133e30 2023-09-30T22:32:03.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.046+0000 7fefcca7c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fefc809fb80 con 0x7fefc8134260 2023-09-30T22:32:03.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.046+0000 7fefc659c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fefc8133420 0x7fefc81337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:03.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.047+0000 7fefc659c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fefc8133420 0x7fefc81337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54128/0 (socket says 172.21.15.16:54128) 2023-09-30T22:32:03.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.047+0000 7fefc659c700 1 -- 172.21.15.16:0/3810176358 learned_addr learned my addr 172.21.15.16:0/3810176358 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:03.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.047+0000 7fefc6d9d700 1 --2- 172.21.15.16:0/3810176358 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fefc8134260 0x7fefc81304d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:03.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.047+0000 7fefc5d9b700 1 --2- 172.21.15.16:0/3810176358 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fefc8133e30 0x7fefc812ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:03.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.047+0000 7fefc6d9d700 1 -- 172.21.15.16:0/3810176358 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fefc8133e30 msgr2=0x7fefc812ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:03.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.047+0000 7fefc6d9d700 1 --2- 172.21.15.16:0/3810176358 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fefc8133e30 0x7fefc812ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:03.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.047+0000 7fefc6d9d700 1 -- 172.21.15.16:0/3810176358 >> v2:172.21.15.16:3300/0 conn(0x7fefc8133420 msgr2=0x7fefc81337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:03.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.047+0000 7fefc6d9d700 1 --2- 172.21.15.16:0/3810176358 >> v2:172.21.15.16:3300/0 conn(0x7fefc8133420 0x7fefc81337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:03.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.047+0000 7fefc6d9d700 1 -- 172.21.15.16:0/3810176358 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefc80a0030 con 0x7fefc8134260 2023-09-30T22:32:03.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.047+0000 7fefc659c700 1 --2- 172.21.15.16:0/3810176358 >> v2:172.21.15.16:3300/0 conn(0x7fefc8133420 0x7fefc81337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:03.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.047+0000 7fefc6d9d700 1 --2- 172.21.15.16:0/3810176358 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fefc8134260 0x7fefc81304d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fefc0009d90 tx=0x7fefc000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da470dd574a1b09a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:03.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.047+0000 7fefc559a700 1 -- 172.21.15.16:0/3810176358 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc000e070 con 0x7fefc8134260 2023-09-30T22:32:03.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.048+0000 7fefc559a700 1 -- 172.21.15.16:0/3810176358 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefc000bb20 con 0x7fefc8134260 2023-09-30T22:32:03.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.048+0000 7fefc559a700 1 -- 172.21.15.16:0/3810176358 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc0012430 con 0x7fefc8134260 2023-09-30T22:32:03.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.048+0000 7fefcca7c700 1 -- 172.21.15.16:0/3810176358 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fefc8134260 msgr2=0x7fefc81304d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:03.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.048+0000 7fefcca7c700 1 --2- 172.21.15.16:0/3810176358 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fefc8134260 0x7fefc81304d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fefc0009d90 tx=0x7fefc000b330 comp rx=0 tx=0).stop 2023-09-30T22:32:03.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.049+0000 7fefcca7c700 1 -- 172.21.15.16:0/3810176358 shutdown_connections 2023-09-30T22:32:03.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.049+0000 7fefcca7c700 1 --2- 172.21.15.16:0/3810176358 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fefc8134260 0x7fefc81304d0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:03.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.049+0000 7fefcca7c700 1 --2- 172.21.15.16:0/3810176358 >> v2:172.21.15.16:3300/0 conn(0x7fefc8133420 0x7fefc81337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:03.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.049+0000 7fefcca7c700 1 --2- 172.21.15.16:0/3810176358 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fefc8133e30 0x7fefc812ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:03.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.049+0000 7fefcca7c700 1 -- 172.21.15.16:0/3810176358 >> 172.21.15.16:0/3810176358 conn(0x7fefc80a4210 msgr2=0x7fefc80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:03.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.049+0000 7fefcca7c700 1 -- 172.21.15.16:0/3810176358 shutdown_connections 2023-09-30T22:32:03.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.049+0000 7fefcca7c700 1 -- 172.21.15.16:0/3810176358 wait complete. 2023-09-30T22:32:03.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.050+0000 7fefcca7c700 1 Processor -- start 2023-09-30T22:32:03.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.050+0000 7fefcca7c700 1 -- start start 2023-09-30T22:32:03.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.050+0000 7fefcca7c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fefc8133420 0x7fefc809c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:03.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.050+0000 7fefcca7c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fefc8133e30 0x7fefc809c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:03.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.050+0000 7fefcca7c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefc8134260 0x7fefc809cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:03.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.050+0000 7fefcca7c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefc812d940 con 0x7fefc8134260 2023-09-30T22:32:03.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.050+0000 7fefcca7c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefc811fe40 con 0x7fefc8133e30 2023-09-30T22:32:03.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.050+0000 7fefcca7c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fefc8131400 con 0x7fefc8133420 2023-09-30T22:32:03.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.051+0000 7fefc659c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fefc8133420 0x7fefc809c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:03.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.051+0000 7fefc5d9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fefc8133e30 0x7fefc809c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:03.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.051+0000 7fefc659c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fefc8133420 0x7fefc809c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:34460/0 (socket says 172.21.15.16:34460) 2023-09-30T22:32:03.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.051+0000 7fefc659c700 1 -- 172.21.15.16:0/1816940706 learned_addr learned my addr 172.21.15.16:0/1816940706 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:03.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.051+0000 7fefc659c700 1 -- 172.21.15.16:0/1816940706 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fefc8133e30 msgr2=0x7fefc809c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:03.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.051+0000 7fefc659c700 1 --2- 172.21.15.16:0/1816940706 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fefc8133e30 0x7fefc809c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:03.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.051+0000 7fefc659c700 1 -- 172.21.15.16:0/1816940706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefc8134260 msgr2=0x7fefc809cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:03.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.051+0000 7fefc659c700 1 --2- 172.21.15.16:0/1816940706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefc8134260 0x7fefc809cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:03.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.051+0000 7fefc659c700 1 -- 172.21.15.16:0/1816940706 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb8009550 con 0x7fefc8133420 2023-09-30T22:32:03.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.051+0000 7fefc659c700 1 --2- 172.21.15.16:0/1816940706 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fefc8133420 0x7fefc809c480 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fefb800ebe0 tx=0x7fefb800eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:03.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.052+0000 7fefb6ffd700 1 -- 172.21.15.16:0/1816940706 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb800c850 con 0x7fefc8133420 2023-09-30T22:32:03.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.052+0000 7fefb6ffd700 1 -- 172.21.15.16:0/1816940706 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefb800c9b0 con 0x7fefc8133420 2023-09-30T22:32:03.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.052+0000 7fefb6ffd700 1 -- 172.21.15.16:0/1816940706 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb800cd20 con 0x7fefc8133420 2023-09-30T22:32:03.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.052+0000 7fefcca7c700 1 -- 172.21.15.16:0/1816940706 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefc0009a70 con 0x7fefc8133420 2023-09-30T22:32:03.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.052+0000 7fefcca7c700 1 -- 172.21.15.16:0/1816940706 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefc811faf0 con 0x7fefc8133420 2023-09-30T22:32:03.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.053+0000 7fefb6ffd700 1 -- 172.21.15.16:0/1816940706 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fefb8013470 con 0x7fefc8133420 2023-09-30T22:32:03.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.053+0000 7fefcca7c700 1 -- 172.21.15.16:0/1816940706 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefc8074ea0 con 0x7fefc8133420 2023-09-30T22:32:03.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.053+0000 7fefb6ffd700 1 -- 172.21.15.16:0/1816940706 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fefb803f6a0 con 0x7fefc8133420 2023-09-30T22:32:03.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.062+0000 7fefb6ffd700 1 -- 172.21.15.16:0/1816940706 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fefb80102b0 con 0x7fefc8133420 2023-09-30T22:32:03.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.088+0000 7f4d61697700 1 Processor -- start 2023-09-30T22:32:03.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.088+0000 7f4d61697700 1 -- start start 2023-09-30T22:32:03.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.088+0000 7f4d61697700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4d5c09f410 0x7f4d5c136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:03.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.089+0000 7f4d61697700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4d5c136fe0 0x7f4d5c1393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:03.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.089+0000 7f4d61697700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d5c070940 con 0x7f4d5c09e930 2023-09-30T22:32:03.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.089+0000 7f4d61697700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d5c06fd60 con 0x7f4d5c136fe0 2023-09-30T22:32:03.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.089+0000 7f4d61697700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5c06fa80 con 0x7f4d5c09f410 2023-09-30T22:32:03.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.089+0000 7f4d5a7fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4d5c09f410 0x7f4d5c136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:03.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.089+0000 7f4d5b7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4d5c136fe0 0x7f4d5c1393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:03.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.089+0000 7f4d5a7fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4d5c09f410 0x7f4d5c136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:34470/0 (socket says 172.21.15.16:34470) 2023-09-30T22:32:03.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.089+0000 7f4d5a7fc700 1 -- 172.21.15.16:0/402329167 learned_addr learned my addr 172.21.15.16:0/402329167 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:03.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.089+0000 7f4d5a7fc700 1 -- 172.21.15.16:0/402329167 >> v2:172.21.15.16:3300/0 conn(0x7f4d5c136fe0 msgr2=0x7f4d5c1393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:03.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.089+0000 7f4d5a7fc700 1 --2- 172.21.15.16:0/402329167 >> v2:172.21.15.16:3300/0 conn(0x7f4d5c136fe0 0x7f4d5c1393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:03.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.089+0000 7f4d5a7fc700 1 -- 172.21.15.16:0/402329167 >> v1:172.21.15.16:6789/0 conn(0x7f4d5c09e930 legacy=0x7f4d5c09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:03.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.089+0000 7f4d5a7fc700 1 -- 172.21.15.16:0/402329167 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d5c09fb60 con 0x7f4d5c09f410 2023-09-30T22:32:03.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.089+0000 7f4d5a7fc700 1 --2- 172.21.15.16:0/402329167 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4d5c09f410 0x7f4d5c136aa0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4d500098f0 tx=0x7f4d50009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c62e367cef4112e8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:03.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.090+0000 7f4d59ffb700 1 -- 172.21.15.16:0/402329167 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d50011830 con 0x7f4d5c09f410 2023-09-30T22:32:03.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.090+0000 7f4d59ffb700 1 -- 172.21.15.16:0/402329167 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d50011990 con 0x7f4d5c09f410 2023-09-30T22:32:03.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.090+0000 7f4d59ffb700 1 -- 172.21.15.16:0/402329167 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d50011d20 con 0x7f4d5c09f410 2023-09-30T22:32:03.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.091+0000 7f4d61697700 1 -- 172.21.15.16:0/402329167 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4d5c09f410 msgr2=0x7f4d5c136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:03.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.091+0000 7f4d61697700 1 --2- 172.21.15.16:0/402329167 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4d5c09f410 0x7f4d5c136aa0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4d500098f0 tx=0x7f4d50009c00 comp rx=0 tx=0).stop 2023-09-30T22:32:03.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.091+0000 7f4d61697700 1 -- 172.21.15.16:0/402329167 shutdown_connections 2023-09-30T22:32:03.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.091+0000 7f4d61697700 1 --2- 172.21.15.16:0/402329167 >> v2:172.21.15.16:3300/0 conn(0x7f4d5c136fe0 0x7f4d5c1393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:03.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.091+0000 7f4d61697700 1 --2- 172.21.15.16:0/402329167 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4d5c09f410 0x7f4d5c136aa0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:03.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.091+0000 7f4d61697700 1 -- 172.21.15.16:0/402329167 >> 172.21.15.16:0/402329167 conn(0x7f4d5c0a4210 msgr2=0x7f4d5c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:03.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.091+0000 7f4d61697700 1 -- 172.21.15.16:0/402329167 shutdown_connections 2023-09-30T22:32:03.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.091+0000 7f4d61697700 1 -- 172.21.15.16:0/402329167 wait complete. 2023-09-30T22:32:03.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.092+0000 7f4d61697700 1 Processor -- start 2023-09-30T22:32:03.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.093+0000 7f4d61697700 1 -- start start 2023-09-30T22:32:03.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.093+0000 7f4d61697700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d5c09e930 0x7f4d5c134dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:03.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.093+0000 7f4d61697700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4d5c09f410 0x7f4d5c135300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:03.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.093+0000 7f4d61697700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4d5c136fe0 0x7f4d5c135840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:03.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.093+0000 7f4d61697700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5c06d810 con 0x7f4d5c09e930 2023-09-30T22:32:03.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.093+0000 7f4d61697700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5c071d20 con 0x7f4d5c09f410 2023-09-30T22:32:03.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.093+0000 7f4d61697700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5c11b6c0 con 0x7f4d5c136fe0 2023-09-30T22:32:03.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.094+0000 7f4d5affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d5c09e930 0x7f4d5c134dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:03.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.094+0000 7f4d5b7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4d5c136fe0 0x7f4d5c135840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:03.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.094+0000 7f4d5affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d5c09e930 0x7f4d5c134dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54150/0 (socket says 172.21.15.16:54150) 2023-09-30T22:32:03.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.094+0000 7f4d5affd700 1 -- 172.21.15.16:0/2274284896 learned_addr learned my addr 172.21.15.16:0/2274284896 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:03.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.094+0000 7f4d5a7fc700 1 --2- 172.21.15.16:0/2274284896 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4d5c09f410 0x7f4d5c135300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:03.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.094+0000 7f4d5b7fe700 1 -- 172.21.15.16:0/2274284896 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4d5c09f410 msgr2=0x7f4d5c135300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:03.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.094+0000 7f4d5b7fe700 1 --2- 172.21.15.16:0/2274284896 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4d5c09f410 0x7f4d5c135300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:03.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.094+0000 7f4d5b7fe700 1 -- 172.21.15.16:0/2274284896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d5c09e930 msgr2=0x7f4d5c134dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:03.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.094+0000 7f4d5b7fe700 1 --2- 172.21.15.16:0/2274284896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d5c09e930 0x7f4d5c134dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:03.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.094+0000 7f4d5b7fe700 1 -- 172.21.15.16:0/2274284896 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d50009550 con 0x7f4d5c136fe0 2023-09-30T22:32:03.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.094+0000 7f4d5b7fe700 1 --2- 172.21.15.16:0/2274284896 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4d5c136fe0 0x7f4d5c135840 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4d5400ef90 tx=0x7f4d5400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:03.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.094+0000 7f4d437fe700 1 -- 172.21.15.16:0/2274284896 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5400cc90 con 0x7f4d5c136fe0 2023-09-30T22:32:03.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.094+0000 7f4d61697700 1 -- 172.21.15.16:0/2274284896 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d5c11bfd0 con 0x7f4d5c136fe0 2023-09-30T22:32:03.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.095+0000 7f4d437fe700 1 -- 172.21.15.16:0/2274284896 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d5400f040 con 0x7f4d5c136fe0 2023-09-30T22:32:03.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.095+0000 7f4d437fe700 1 -- 172.21.15.16:0/2274284896 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d540146c0 con 0x7f4d5c136fe0 2023-09-30T22:32:03.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.095+0000 7f4d61697700 1 -- 172.21.15.16:0/2274284896 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d5c11a9f0 con 0x7f4d5c136fe0 2023-09-30T22:32:03.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.096+0000 7f4d437fe700 1 -- 172.21.15.16:0/2274284896 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4d54014820 con 0x7f4d5c136fe0 2023-09-30T22:32:03.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.096+0000 7f4d61697700 1 -- 172.21.15.16:0/2274284896 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d5c074ea0 con 0x7f4d5c136fe0 2023-09-30T22:32:03.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.096+0000 7f4d437fe700 1 -- 172.21.15.16:0/2274284896 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f4d540417a0 con 0x7f4d5c136fe0 2023-09-30T22:32:03.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.106+0000 7f4d437fe700 1 -- 172.21.15.16:0/2274284896 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4d54045060 con 0x7f4d5c136fe0 2023-09-30T22:32:03.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.120+0000 7f12fb83e700 1 -- 172.21.15.16:0/3401314027 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f12f409f420 con 0x7f12f409d6f0 2023-09-30T22:32:03.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.120+0000 7f12e9ffb700 1 -- 172.21.15.16:0/3401314027 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f12f001be10 con 0x7f12f409d6f0 2023-09-30T22:32:03.121 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:32:03.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.124+0000 7f12fb83e700 1 -- 172.21.15.16:0/3401314027 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12f409d6f0 msgr2=0x7f12f41a73a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:03.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.124+0000 7f12fb83e700 1 --2- 172.21.15.16:0/3401314027 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12f409d6f0 0x7f12f41a73a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f12f000e9b0 tx=0x7f12f000ecc0 comp rx=0 tx=0).stop 2023-09-30T22:32:03.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.124+0000 7f12fb83e700 1 -- 172.21.15.16:0/3401314027 shutdown_connections 2023-09-30T22:32:03.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.124+0000 7f12fb83e700 1 --2- 172.21.15.16:0/3401314027 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12f409d6f0 0x7f12f41a73a0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:03.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.124+0000 7f12fb83e700 1 --2- 172.21.15.16:0/3401314027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12f409e440 0x7f12f41c7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:03.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.124+0000 7f12fb83e700 1 --2- 172.21.15.16:0/3401314027 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12f409e000 0x7f12f41a78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:03.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.124+0000 7f12fb83e700 1 -- 172.21.15.16:0/3401314027 >> 172.21.15.16:0/3401314027 conn(0x7f12f4066f50 msgr2=0x7f12f406f070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:03.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.124+0000 7f12fb83e700 1 -- 172.21.15.16:0/3401314027 shutdown_connections 2023-09-30T22:32:03.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.124+0000 7f12fb83e700 1 -- 172.21.15.16:0/3401314027 wait complete. 2023-09-30T22:32:03.144 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:32:03.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.253+0000 7f1a48ff9700 1 -- 172.21.15.16:0/3587698961 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1a34005590 con 0x7f1a5c09e410 2023-09-30T22:32:03.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.253+0000 7f1a4affd700 1 -- 172.21.15.16:0/3587698961 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1a5402c030 con 0x7f1a5c09e410 2023-09-30T22:32:03.255 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:32:03.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.256+0000 7f1a48ff9700 1 -- 172.21.15.16:0/3587698961 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1a5c09e410 msgr2=0x7f1a5c1c8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:03.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.256+0000 7f1a48ff9700 1 --2- 172.21.15.16:0/3587698961 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1a5c09e410 0x7f1a5c1c8040 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c09b3c0 tx=0x7f1a54004ad0 comp rx=0 tx=0).stop 2023-09-30T22:32:03.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.256+0000 7f1a48ff9700 1 -- 172.21.15.16:0/3587698961 shutdown_connections 2023-09-30T22:32:03.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.256+0000 7f1a48ff9700 1 --2- 172.21.15.16:0/3587698961 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1a5c09e410 0x7f1a5c1c8040 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:03.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.256+0000 7f1a48ff9700 1 --2- 172.21.15.16:0/3587698961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a5c09d6f0 0x7f1a5c1c5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:03.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.256+0000 7f1a48ff9700 1 --2- 172.21.15.16:0/3587698961 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1a5c09e000 0x7f1a5c1c5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:03.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.256+0000 7f1a48ff9700 1 -- 172.21.15.16:0/3587698961 >> 172.21.15.16:0/3587698961 conn(0x7f1a5c066f50 msgr2=0x7f1a5c1bacf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:03.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.256+0000 7f1a48ff9700 1 -- 172.21.15.16:0/3587698961 shutdown_connections 2023-09-30T22:32:03.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.256+0000 7f1a48ff9700 1 -- 172.21.15.16:0/3587698961 wait complete. 2023-09-30T22:32:03.270 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:32:03.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.282+0000 7fefcca7c700 1 -- 172.21.15.16:0/1816940706 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fefc806b250 con 0x7fefc8133420 2023-09-30T22:32:03.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.283+0000 7fefb6ffd700 1 -- 172.21.15.16:0/1816940706 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fefb8011350 con 0x7fefc8133420 2023-09-30T22:32:03.283 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:32:03.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.285+0000 7fefcca7c700 1 -- 172.21.15.16:0/1816940706 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fefc8133420 msgr2=0x7fefc809c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:03.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.285+0000 7fefcca7c700 1 --2- 172.21.15.16:0/1816940706 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fefc8133420 0x7fefc809c480 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fefb800ebe0 tx=0x7fefb800eef0 comp rx=0 tx=0).stop 2023-09-30T22:32:03.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.285+0000 7fefcca7c700 1 -- 172.21.15.16:0/1816940706 shutdown_connections 2023-09-30T22:32:03.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.285+0000 7fefcca7c700 1 --2- 172.21.15.16:0/1816940706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefc8134260 0x7fefc809cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:03.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.285+0000 7fefcca7c700 1 --2- 172.21.15.16:0/1816940706 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fefc8133420 0x7fefc809c480 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:03.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.285+0000 7fefcca7c700 1 --2- 172.21.15.16:0/1816940706 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fefc8133e30 0x7fefc809c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:03.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.285+0000 7fefcca7c700 1 -- 172.21.15.16:0/1816940706 >> 172.21.15.16:0/1816940706 conn(0x7fefc80a4210 msgr2=0x7fefc81302c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:03.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.285+0000 7fefcca7c700 1 -- 172.21.15.16:0/1816940706 shutdown_connections 2023-09-30T22:32:03.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.285+0000 7fefcca7c700 1 -- 172.21.15.16:0/1816940706 wait complete. 2023-09-30T22:32:03.299 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:32:03.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.322+0000 7f4d61697700 1 -- 172.21.15.16:0/2274284896 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4d5c06b250 con 0x7f4d5c136fe0 2023-09-30T22:32:03.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.323+0000 7f4d437fe700 1 -- 172.21.15.16:0/2274284896 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4d54018070 con 0x7f4d5c136fe0 2023-09-30T22:32:03.324 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:32:03.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.325+0000 7f4d61697700 1 -- 172.21.15.16:0/2274284896 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4d5c136fe0 msgr2=0x7f4d5c135840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:03.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.325+0000 7f4d61697700 1 --2- 172.21.15.16:0/2274284896 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4d5c136fe0 0x7f4d5c135840 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4d5400ef90 tx=0x7f4d5400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:32:03.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.325+0000 7f4d61697700 1 -- 172.21.15.16:0/2274284896 shutdown_connections 2023-09-30T22:32:03.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.325+0000 7f4d61697700 1 --2- 172.21.15.16:0/2274284896 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4d5c136fe0 0x7f4d5c135840 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:03.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.325+0000 7f4d61697700 1 --2- 172.21.15.16:0/2274284896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d5c09e930 0x7f4d5c134dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:03.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.325+0000 7f4d61697700 1 --2- 172.21.15.16:0/2274284896 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4d5c09f410 0x7f4d5c135300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:03.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.325+0000 7f4d61697700 1 -- 172.21.15.16:0/2274284896 >> 172.21.15.16:0/2274284896 conn(0x7f4d5c0a4210 msgr2=0x7f4d5c0a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:03.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.325+0000 7f4d61697700 1 -- 172.21.15.16:0/2274284896 shutdown_connections 2023-09-30T22:32:03.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:03.325+0000 7f4d61697700 1 -- 172.21.15.16:0/2274284896 wait complete. 2023-09-30T22:32:03.339 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:32:04.145 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:32:04.271 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:32:04.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.291+0000 7fbb537b0700 1 Processor -- start 2023-09-30T22:32:04.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.291+0000 7fbb537b0700 1 -- start start 2023-09-30T22:32:04.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.292+0000 7fbb537b0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbb4c09d630 0x7fbb4c0a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:04.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.292+0000 7fbb537b0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbb4c138d80 0x7fbb4c13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:04.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.292+0000 7fbb537b0700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb4c0a2a70 con 0x7fbb4c0a0a30 2023-09-30T22:32:04.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.292+0000 7fbb537b0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4c0a01d0 con 0x7fbb4c138d80 2023-09-30T22:32:04.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.292+0000 7fbb537b0700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4c09e2b0 con 0x7fbb4c09d630 2023-09-30T22:32:04.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.292+0000 7fbb51d4d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbb4c138d80 0x7fbb4c13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:04.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.292+0000 7fbb50d4b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbb4c09d630 0x7fbb4c0a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:04.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.292+0000 7fbb5154c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fbb4c0a0a30 0x7fbb4c0a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60858/0 (socket says 172.21.15.16:60858) 2023-09-30T22:32:04.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.292+0000 7fbb5154c700 1 -- 172.21.15.16:0/4092563583 learned_addr learned my addr 172.21.15.16:0/4092563583 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:04.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.292+0000 7fbb50d4b700 1 -- 172.21.15.16:0/4092563583 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbb4c138d80 msgr2=0x7fbb4c13b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:04.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.292+0000 7fbb50d4b700 1 --2- 172.21.15.16:0/4092563583 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbb4c138d80 0x7fbb4c13b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:04.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.292+0000 7fbb50d4b700 1 -- 172.21.15.16:0/4092563583 >> v1:172.21.15.16:6789/0 conn(0x7fbb4c0a0a30 legacy=0x7fbb4c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:04.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.292+0000 7fbb50d4b700 1 -- 172.21.15.16:0/4092563583 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb4c09dc50 con 0x7fbb4c09d630 2023-09-30T22:32:04.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.292+0000 7fbb51d4d700 1 --2- 172.21.15.16:0/4092563583 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbb4c138d80 0x7fbb4c13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:04.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.292+0000 7fbb43fff700 1 -- 172.21.15.16:0/4092563583 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3088360439 0 0) 0x7fbb4c0a2a70 con 0x7fbb4c0a0a30 2023-09-30T22:32:04.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.293+0000 7fbb50d4b700 1 --2- 172.21.15.16:0/4092563583 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbb4c09d630 0x7fbb4c0a1510 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c009870 tx=0x7fbb3c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39897245773556e8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:04.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.293+0000 7fbb43fff700 1 -- 172.21.15.16:0/4092563583 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c011830 con 0x7fbb4c09d630 2023-09-30T22:32:04.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.293+0000 7fbb43fff700 1 -- 172.21.15.16:0/4092563583 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb3c011990 con 0x7fbb4c09d630 2023-09-30T22:32:04.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.293+0000 7fbb43fff700 1 -- 172.21.15.16:0/4092563583 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c011d20 con 0x7fbb4c09d630 2023-09-30T22:32:04.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.294+0000 7fbb537b0700 1 -- 172.21.15.16:0/4092563583 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbb4c09d630 msgr2=0x7fbb4c0a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:04.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.294+0000 7fbb537b0700 1 --2- 172.21.15.16:0/4092563583 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbb4c09d630 0x7fbb4c0a1510 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c009870 tx=0x7fbb3c009b80 comp rx=0 tx=0).stop 2023-09-30T22:32:04.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.294+0000 7fbb537b0700 1 -- 172.21.15.16:0/4092563583 shutdown_connections 2023-09-30T22:32:04.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.294+0000 7fbb537b0700 1 --2- 172.21.15.16:0/4092563583 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbb4c138d80 0x7fbb4c13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:04.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.294+0000 7fbb537b0700 1 --2- 172.21.15.16:0/4092563583 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbb4c09d630 0x7fbb4c0a1510 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:04.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.294+0000 7fbb537b0700 1 -- 172.21.15.16:0/4092563583 >> 172.21.15.16:0/4092563583 conn(0x7fbb4c0a3050 msgr2=0x7fbb4c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:04.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.294+0000 7fbb537b0700 1 -- 172.21.15.16:0/4092563583 shutdown_connections 2023-09-30T22:32:04.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.294+0000 7fbb537b0700 1 -- 172.21.15.16:0/4092563583 wait complete. 2023-09-30T22:32:04.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.295+0000 7fbb537b0700 1 Processor -- start 2023-09-30T22:32:04.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.295+0000 7fbb537b0700 1 -- start start 2023-09-30T22:32:04.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.295+0000 7fbb537b0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb4c09d630 0x7fbb4c133670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:04.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.295+0000 7fbb537b0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbb4c0a0a30 0x7fbb4c133bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:04.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.295+0000 7fbb537b0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbb4c138d80 0x7fbb4c1340f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:04.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.295+0000 7fbb537b0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4c136540 con 0x7fbb4c09d630 2023-09-30T22:32:04.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.295+0000 7fbb537b0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4c136eb0 con 0x7fbb4c138d80 2023-09-30T22:32:04.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.295+0000 7fbb537b0700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4c136190 con 0x7fbb4c0a0a30 2023-09-30T22:32:04.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.296+0000 7fbb51d4d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbb4c138d80 0x7fbb4c1340f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:04.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.296+0000 7fbb50d4b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbb4c0a0a30 0x7fbb4c133bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:04.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.296+0000 7fbb51d4d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbb4c138d80 0x7fbb4c1340f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:44360/0 (socket says 172.21.15.16:44360) 2023-09-30T22:32:04.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.296+0000 7fbb51d4d700 1 -- 172.21.15.16:0/1008369520 learned_addr learned my addr 172.21.15.16:0/1008369520 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:04.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.296+0000 7fbb5154c700 1 --2- 172.21.15.16:0/1008369520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb4c09d630 0x7fbb4c133670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:04.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.296+0000 7fbb51d4d700 1 -- 172.21.15.16:0/1008369520 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbb4c0a0a30 msgr2=0x7fbb4c133bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:04.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.296+0000 7fbb51d4d700 1 --2- 172.21.15.16:0/1008369520 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbb4c0a0a30 0x7fbb4c133bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:04.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.296+0000 7fbb51d4d700 1 -- 172.21.15.16:0/1008369520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb4c09d630 msgr2=0x7fbb4c133670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:04.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.296+0000 7fbb51d4d700 1 --2- 172.21.15.16:0/1008369520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb4c09d630 0x7fbb4c133670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:04.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.296+0000 7fbb51d4d700 1 -- 172.21.15.16:0/1008369520 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb44009a70 con 0x7fbb4c138d80 2023-09-30T22:32:04.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.296+0000 7fbb51d4d700 1 --2- 172.21.15.16:0/1008369520 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbb4c138d80 0x7fbb4c1340f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fbb4400efc0 tx=0x7fbb4400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:04.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.296+0000 7fbb41ffb700 1 -- 172.21.15.16:0/1008369520 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4400ca30 con 0x7fbb4c138d80 2023-09-30T22:32:04.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.296+0000 7fbb41ffb700 1 -- 172.21.15.16:0/1008369520 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb4400f040 con 0x7fbb4c138d80 2023-09-30T22:32:04.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.296+0000 7fbb41ffb700 1 -- 172.21.15.16:0/1008369520 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4400cdf0 con 0x7fbb4c138d80 2023-09-30T22:32:04.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.296+0000 7fbb537b0700 1 -- 172.21.15.16:0/1008369520 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb3c0094a0 con 0x7fbb4c138d80 2023-09-30T22:32:04.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.297+0000 7fbb537b0700 1 -- 172.21.15.16:0/1008369520 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb4c137190 con 0x7fbb4c138d80 2023-09-30T22:32:04.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.297+0000 7fbb41ffb700 1 -- 172.21.15.16:0/1008369520 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbb44014480 con 0x7fbb4c138d80 2023-09-30T22:32:04.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.297+0000 7fbb41ffb700 1 -- 172.21.15.16:0/1008369520 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fbb4403fce0 con 0x7fbb4c138d80 2023-09-30T22:32:04.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.298+0000 7fbb537b0700 1 -- 172.21.15.16:0/1008369520 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb4c136ff0 con 0x7fbb4c138d80 2023-09-30T22:32:04.302 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:32:04.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.303+0000 7fbb41ffb700 1 -- 172.21.15.16:0/1008369520 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbb4c136ff0 con 0x7fbb4c138d80 2023-09-30T22:32:04.340 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:32:04.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.417+0000 7fe2a0f89700 1 Processor -- start 2023-09-30T22:32:04.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.417+0000 7fe2a0f89700 1 -- start start 2023-09-30T22:32:04.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.418+0000 7fe2a0f89700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe29c1337f0 0x7fe29c138880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:04.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.418+0000 7fe2a0f89700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe29c138dc0 0x7fe29c13b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:04.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.418+0000 7fe2a0f89700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe29c06b990 con 0x7fe29c133420 2023-09-30T22:32:04.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.418+0000 7fe2a0f89700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe29c09b0e0 con 0x7fe29c1337f0 2023-09-30T22:32:04.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.418+0000 7fe2a0f89700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c133c20 con 0x7fe29c138dc0 2023-09-30T22:32:04.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.418+0000 7fe29ad9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe29c138dc0 0x7fe29c13b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:04.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.418+0000 7fe29a59c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe29c133420 0x7fe29c133e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60864/0 (socket says 172.21.15.16:60864) 2023-09-30T22:32:04.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.418+0000 7fe29ad9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe29c138dc0 0x7fe29c13b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:34504/0 (socket says 172.21.15.16:34504) 2023-09-30T22:32:04.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.418+0000 7fe29a59c700 1 -- 172.21.15.16:0/557846784 learned_addr learned my addr 172.21.15.16:0/557846784 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:04.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.418+0000 7fe299d9b700 1 --2- 172.21.15.16:0/557846784 >> v2:172.21.15.16:3300/0 conn(0x7fe29c1337f0 0x7fe29c138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:04.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.419+0000 7fe29ad9d700 1 -- 172.21.15.16:0/557846784 >> v2:172.21.15.16:3300/0 conn(0x7fe29c1337f0 msgr2=0x7fe29c138880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:04.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.419+0000 7fe29ad9d700 1 --2- 172.21.15.16:0/557846784 >> v2:172.21.15.16:3300/0 conn(0x7fe29c1337f0 0x7fe29c138880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:04.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.419+0000 7fe29ad9d700 1 -- 172.21.15.16:0/557846784 >> v1:172.21.15.16:6789/0 conn(0x7fe29c133420 legacy=0x7fe29c133e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:04.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.419+0000 7fe29ad9d700 1 -- 172.21.15.16:0/557846784 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe29c0a0590 con 0x7fe29c138dc0 2023-09-30T22:32:04.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.419+0000 7fe299d9b700 1 --2- 172.21.15.16:0/557846784 >> v2:172.21.15.16:3300/0 conn(0x7fe29c1337f0 0x7fe29c138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:04.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.419+0000 7fe29ad9d700 1 --2- 172.21.15.16:0/557846784 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe29c138dc0 0x7fe29c13b1a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe294009d90 tx=0x7fe29400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6acb4f5ee8644440 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:04.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.419+0000 7fe29959a700 1 -- 172.21.15.16:0/557846784 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe29400e070 con 0x7fe29c138dc0 2023-09-30T22:32:04.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.419+0000 7fe29959a700 1 -- 172.21.15.16:0/557846784 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe294003a20 con 0x7fe29c138dc0 2023-09-30T22:32:04.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.419+0000 7fe29959a700 1 -- 172.21.15.16:0/557846784 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe294003dd0 con 0x7fe29c138dc0 2023-09-30T22:32:04.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.420+0000 7fe2a0f89700 1 -- 172.21.15.16:0/557846784 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe29c138dc0 msgr2=0x7fe29c13b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:04.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.420+0000 7fe2a0f89700 1 --2- 172.21.15.16:0/557846784 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe29c138dc0 0x7fe29c13b1a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fe294009d90 tx=0x7fe29400b330 comp rx=0 tx=0).stop 2023-09-30T22:32:04.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.421+0000 7fe2a0f89700 1 -- 172.21.15.16:0/557846784 shutdown_connections 2023-09-30T22:32:04.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.421+0000 7fe2a0f89700 1 --2- 172.21.15.16:0/557846784 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe29c138dc0 0x7fe29c13b1a0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:04.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.421+0000 7fe2a0f89700 1 --2- 172.21.15.16:0/557846784 >> v2:172.21.15.16:3300/0 conn(0x7fe29c1337f0 0x7fe29c138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:04.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.421+0000 7fe2a0f89700 1 -- 172.21.15.16:0/557846784 >> 172.21.15.16:0/557846784 conn(0x7fe29c0a4210 msgr2=0x7fe29c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:04.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.421+0000 7fe2a0f89700 1 -- 172.21.15.16:0/557846784 shutdown_connections 2023-09-30T22:32:04.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.421+0000 7fe2a0f89700 1 -- 172.21.15.16:0/557846784 wait complete. 2023-09-30T22:32:04.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.422+0000 7fe2a0f89700 1 Processor -- start 2023-09-30T22:32:04.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.423+0000 7fe2a0f89700 1 -- start start 2023-09-30T22:32:04.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.423+0000 7fe2a0f89700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe29c133420 0x7fe29c12ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:04.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.423+0000 7fe2a0f89700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe29c1337f0 0x7fe29c12f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:04.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.423+0000 7fe2a0f89700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe29c138dc0 0x7fe29c12f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:04.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.423+0000 7fe2a0f89700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c138910 con 0x7fe29c138dc0 2023-09-30T22:32:04.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.423+0000 7fe2a0f89700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c1309b0 con 0x7fe29c1337f0 2023-09-30T22:32:04.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.423+0000 7fe2a0f89700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c09bc40 con 0x7fe29c133420 2023-09-30T22:32:04.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.423+0000 7fe29a59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe29c133420 0x7fe29c12ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:04.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.423+0000 7fe29a59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe29c133420 0x7fe29c12ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:34510/0 (socket says 172.21.15.16:34510) 2023-09-30T22:32:04.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.423+0000 7fe29a59c700 1 -- 172.21.15.16:0/2966337636 learned_addr learned my addr 172.21.15.16:0/2966337636 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:04.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.424+0000 7fe29ad9d700 1 --2- 172.21.15.16:0/2966337636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe29c138dc0 0x7fe29c12f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:04.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.424+0000 7fe299d9b700 1 --2- 172.21.15.16:0/2966337636 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe29c1337f0 0x7fe29c12f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:04.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.424+0000 7fe29a59c700 1 -- 172.21.15.16:0/2966337636 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe29c1337f0 msgr2=0x7fe29c12f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:04.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.424+0000 7fe29a59c700 1 --2- 172.21.15.16:0/2966337636 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe29c1337f0 0x7fe29c12f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:04.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.424+0000 7fe29a59c700 1 -- 172.21.15.16:0/2966337636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe29c138dc0 msgr2=0x7fe29c12f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:04.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.424+0000 7fe29a59c700 1 --2- 172.21.15.16:0/2966337636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe29c138dc0 0x7fe29c12f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:04.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.424+0000 7fe29a59c700 1 -- 172.21.15.16:0/2966337636 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2900094a0 con 0x7fe29c133420 2023-09-30T22:32:04.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.424+0000 7fe29a59c700 1 --2- 172.21.15.16:0/2966337636 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe29c133420 0x7fe29c12ee50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe28c00c780 tx=0x7fe28c00ca90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:04.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.425+0000 7fe28affd700 1 -- 172.21.15.16:0/2966337636 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28c004c80 con 0x7fe29c133420 2023-09-30T22:32:04.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.425+0000 7fe28affd700 1 -- 172.21.15.16:0/2966337636 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe28c004de0 con 0x7fe29c133420 2023-09-30T22:32:04.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.425+0000 7fe2a0f89700 1 -- 172.21.15.16:0/2966337636 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe294009a70 con 0x7fe29c133420 2023-09-30T22:32:04.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.425+0000 7fe28affd700 1 -- 172.21.15.16:0/2966337636 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28c007310 con 0x7fe29c133420 2023-09-30T22:32:04.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.425+0000 7fe2a0f89700 1 -- 172.21.15.16:0/2966337636 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe29c09c0c0 con 0x7fe29c133420 2023-09-30T22:32:04.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.426+0000 7fe2a0f89700 1 -- 172.21.15.16:0/2966337636 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe29c09c9d0 con 0x7fe29c133420 2023-09-30T22:32:04.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.426+0000 7fe28affd700 1 -- 172.21.15.16:0/2966337636 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe28c007470 con 0x7fe29c133420 2023-09-30T22:32:04.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.427+0000 7fe28affd700 1 -- 172.21.15.16:0/2966337636 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fe28c040d80 con 0x7fe29c133420 2023-09-30T22:32:04.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.435+0000 7fe28affd700 1 -- 172.21.15.16:0/2966337636 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe28c044060 con 0x7fe29c133420 2023-09-30T22:32:04.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.455+0000 7f834a2b1700 1 Processor -- start 2023-09-30T22:32:04.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.455+0000 7f834a2b1700 1 -- start start 2023-09-30T22:32:04.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.456+0000 7f834a2b1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f834409d6f0 0x7f834409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:04.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.456+0000 7f834a2b1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f834409e410 0x7f8344130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:04.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.456+0000 7f834a2b1700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f834409f130 con 0x7f834409e000 2023-09-30T22:32:04.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.456+0000 7f834a2b1700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f834409f520 con 0x7f834409d6f0 2023-09-30T22:32:04.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.456+0000 7f834a2b1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f834409e930 con 0x7f834409e410 2023-09-30T22:32:04.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.456+0000 7f8343fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f834409d6f0 0x7f834409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:04.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.456+0000 7f8343fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f834409d6f0 0x7f834409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:44380/0 (socket says 172.21.15.16:44380) 2023-09-30T22:32:04.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.456+0000 7f8343fff700 1 -- 172.21.15.16:0/3466692771 learned_addr learned my addr 172.21.15.16:0/3466692771 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:04.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.456+0000 7f834884e700 1 --2- 172.21.15.16:0/3466692771 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f834409e410 0x7f8344130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:04.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.456+0000 7f8343fff700 1 -- 172.21.15.16:0/3466692771 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f834409e410 msgr2=0x7f8344130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:04.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.456+0000 7f8343fff700 1 --2- 172.21.15.16:0/3466692771 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f834409e410 0x7f8344130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:04.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.456+0000 7f8343fff700 1 -- 172.21.15.16:0/3466692771 >> v1:172.21.15.16:6789/0 conn(0x7f834409e000 legacy=0x7f8344130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:04.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.457+0000 7f8343fff700 1 -- 172.21.15.16:0/3466692771 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f834409b2d0 con 0x7f834409d6f0 2023-09-30T22:32:04.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.457+0000 7f8343fff700 1 --2- 172.21.15.16:0/3466692771 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f834409d6f0 0x7f834409dac0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8334009920 tx=0x7f8334009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5532074a3a329906 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:04.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.457+0000 7f8342ffd700 1 -- 172.21.15.16:0/3466692771 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833400f8a0 con 0x7f834409d6f0 2023-09-30T22:32:04.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.457+0000 7f8342ffd700 1 -- 172.21.15.16:0/3466692771 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f833400fa00 con 0x7f834409d6f0 2023-09-30T22:32:04.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.457+0000 7f8342ffd700 1 -- 172.21.15.16:0/3466692771 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833400fd90 con 0x7f834409d6f0 2023-09-30T22:32:04.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.458+0000 7f834a2b1700 1 -- 172.21.15.16:0/3466692771 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f834409d6f0 msgr2=0x7f834409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:04.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.458+0000 7f834a2b1700 1 --2- 172.21.15.16:0/3466692771 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f834409d6f0 0x7f834409dac0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8334009920 tx=0x7f8334009c30 comp rx=0 tx=0).stop 2023-09-30T22:32:04.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.458+0000 7f834a2b1700 1 -- 172.21.15.16:0/3466692771 shutdown_connections 2023-09-30T22:32:04.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.458+0000 7f834a2b1700 1 --2- 172.21.15.16:0/3466692771 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f834409e410 0x7f8344130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:04.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.458+0000 7f834a2b1700 1 --2- 172.21.15.16:0/3466692771 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f834409d6f0 0x7f834409dac0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:04.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.458+0000 7f834a2b1700 1 -- 172.21.15.16:0/3466692771 >> 172.21.15.16:0/3466692771 conn(0x7f8344066f50 msgr2=0x7f834409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:04.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.458+0000 7f834a2b1700 1 -- 172.21.15.16:0/3466692771 shutdown_connections 2023-09-30T22:32:04.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.458+0000 7f834a2b1700 1 -- 172.21.15.16:0/3466692771 wait complete. 2023-09-30T22:32:04.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.459+0000 7f834a2b1700 1 Processor -- start 2023-09-30T22:32:04.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.459+0000 7f834a2b1700 1 -- start start 2023-09-30T22:32:04.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.460+0000 7f834a2b1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f834409d6f0 0x7f834411e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:04.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.460+0000 7f834a2b1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f834409e000 0x7f834411e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:04.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.460+0000 7f834a2b1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f834409e410 0x7f83441cc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:04.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.460+0000 7f834a2b1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f834411fc70 con 0x7f834409e000 2023-09-30T22:32:04.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.460+0000 7f834a2b1700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83440a1e20 con 0x7f834409e410 2023-09-30T22:32:04.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.460+0000 7f834a2b1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8344121910 con 0x7f834409d6f0 2023-09-30T22:32:04.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.460+0000 7f8343fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f834409d6f0 0x7f834411e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:04.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.460+0000 7f8343fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f834409d6f0 0x7f834411e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:34530/0 (socket says 172.21.15.16:34530) 2023-09-30T22:32:04.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.460+0000 7f8343fff700 1 -- 172.21.15.16:0/1918583980 learned_addr learned my addr 172.21.15.16:0/1918583980 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:04.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.460+0000 7f83437fe700 1 --2- 172.21.15.16:0/1918583980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f834409e000 0x7f834411e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:04.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.460+0000 7f834884e700 1 --2- 172.21.15.16:0/1918583980 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f834409e410 0x7f83441cc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:04.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.460+0000 7f8343fff700 1 -- 172.21.15.16:0/1918583980 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f834409e410 msgr2=0x7f83441cc6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:04.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.460+0000 7f8343fff700 1 --2- 172.21.15.16:0/1918583980 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f834409e410 0x7f83441cc6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:04.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.460+0000 7f8343fff700 1 -- 172.21.15.16:0/1918583980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f834409e000 msgr2=0x7f834411e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:04.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.460+0000 7f8343fff700 1 --2- 172.21.15.16:0/1918583980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f834409e000 0x7f834411e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:04.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.460+0000 7f8343fff700 1 -- 172.21.15.16:0/1918583980 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8334009550 con 0x7f834409d6f0 2023-09-30T22:32:04.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.461+0000 7f834884e700 1 --2- 172.21.15.16:0/1918583980 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f834409e410 0x7f83441cc6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:32:04.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.461+0000 7f8343fff700 1 --2- 172.21.15.16:0/1918583980 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f834409d6f0 0x7f834411e2d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f83340098f0 tx=0x7f8334004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:04.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.461+0000 7f8340ff9700 1 -- 172.21.15.16:0/1918583980 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8334003ab0 con 0x7f834409d6f0 2023-09-30T22:32:04.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.461+0000 7f8340ff9700 1 -- 172.21.15.16:0/1918583980 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8334003c10 con 0x7f834409d6f0 2023-09-30T22:32:04.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.461+0000 7f8340ff9700 1 -- 172.21.15.16:0/1918583980 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8334004560 con 0x7f834409d6f0 2023-09-30T22:32:04.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.461+0000 7f834a2b1700 1 -- 172.21.15.16:0/1918583980 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8344121b80 con 0x7f834409d6f0 2023-09-30T22:32:04.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.461+0000 7f834a2b1700 1 -- 172.21.15.16:0/1918583980 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f834411f3c0 con 0x7f834409d6f0 2023-09-30T22:32:04.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.462+0000 7f8340ff9700 1 -- 172.21.15.16:0/1918583980 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8334024450 con 0x7f834409d6f0 2023-09-30T22:32:04.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.462+0000 7f834a2b1700 1 -- 172.21.15.16:0/1918583980 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8344074ea0 con 0x7f834409d6f0 2023-09-30T22:32:04.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.463+0000 7f8340ff9700 1 -- 172.21.15.16:0/1918583980 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f8334024700 con 0x7f834409d6f0 2023-09-30T22:32:04.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.472+0000 7f8340ff9700 1 -- 172.21.15.16:0/1918583980 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8334029020 con 0x7f834409d6f0 2023-09-30T22:32:04.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.494+0000 7f174c5ed700 1 Processor -- start 2023-09-30T22:32:04.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.494+0000 7f174c5ed700 1 -- start start 2023-09-30T22:32:04.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.495+0000 7f174c5ed700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f174409d6f0 0x7f174409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:04.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.495+0000 7f174c5ed700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f174409e000 0x7f17441339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:04.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.495+0000 7f174c5ed700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f174409b120 con 0x7f174409e410 2023-09-30T22:32:04.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.495+0000 7f174c5ed700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f174406b460 con 0x7f174409d6f0 2023-09-30T22:32:04.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.495+0000 7f174c5ed700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f174409a7e0 con 0x7f174409e000 2023-09-30T22:32:04.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.495+0000 7f1749b88700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f174409e000 0x7f17441339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:04.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.495+0000 7f174a389700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f174409d6f0 0x7f174409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:04.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.495+0000 7f1749b88700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f174409e000 0x7f17441339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:34536/0 (socket says 172.21.15.16:34536) 2023-09-30T22:32:04.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.495+0000 7f1749b88700 1 -- 172.21.15.16:0/1150250927 learned_addr learned my addr 172.21.15.16:0/1150250927 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:04.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.496+0000 7f174a389700 1 -- 172.21.15.16:0/1150250927 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f174409e000 msgr2=0x7f17441339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:04.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.496+0000 7f174a389700 1 --2- 172.21.15.16:0/1150250927 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f174409e000 0x7f17441339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:04.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.496+0000 7f174a389700 1 -- 172.21.15.16:0/1150250927 >> v1:172.21.15.16:6789/0 conn(0x7f174409e410 legacy=0x7f1744133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:04.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.496+0000 7f174a389700 1 -- 172.21.15.16:0/1150250927 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f174409a500 con 0x7f174409d6f0 2023-09-30T22:32:04.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.496+0000 7f174a389700 1 --2- 172.21.15.16:0/1150250927 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f174409d6f0 0x7f174409dac0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1740009870 tx=0x7f1740009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=163dd0128f57f550 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:04.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.496+0000 7f1749387700 1 -- 172.21.15.16:0/1150250927 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1740011830 con 0x7f174409d6f0 2023-09-30T22:32:04.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.496+0000 7f1749387700 1 -- 172.21.15.16:0/1150250927 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1740011990 con 0x7f174409d6f0 2023-09-30T22:32:04.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.496+0000 7f1749387700 1 -- 172.21.15.16:0/1150250927 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1740011d20 con 0x7f174409d6f0 2023-09-30T22:32:04.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.497+0000 7f174c5ed700 1 -- 172.21.15.16:0/1150250927 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f174409d6f0 msgr2=0x7f174409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:04.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.497+0000 7f174c5ed700 1 --2- 172.21.15.16:0/1150250927 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f174409d6f0 0x7f174409dac0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f1740009870 tx=0x7f1740009b80 comp rx=0 tx=0).stop 2023-09-30T22:32:04.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.497+0000 7f174c5ed700 1 -- 172.21.15.16:0/1150250927 shutdown_connections 2023-09-30T22:32:04.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.497+0000 7f174c5ed700 1 --2- 172.21.15.16:0/1150250927 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f174409d6f0 0x7f174409dac0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:04.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.497+0000 7f174c5ed700 1 --2- 172.21.15.16:0/1150250927 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f174409e000 0x7f17441339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:04.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.497+0000 7f174c5ed700 1 -- 172.21.15.16:0/1150250927 >> 172.21.15.16:0/1150250927 conn(0x7f1744066f50 msgr2=0x7f174409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:04.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.497+0000 7f174c5ed700 1 -- 172.21.15.16:0/1150250927 shutdown_connections 2023-09-30T22:32:04.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.497+0000 7f174c5ed700 1 -- 172.21.15.16:0/1150250927 wait complete. 2023-09-30T22:32:04.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.498+0000 7f174c5ed700 1 Processor -- start 2023-09-30T22:32:04.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.499+0000 7f174c5ed700 1 -- start start 2023-09-30T22:32:04.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.499+0000 7f174c5ed700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f174409d6f0 0x7f17441225e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:04.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.499+0000 7f174c5ed700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f174409e000 0x7f1744122b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:04.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.499+0000 7f174c5ed700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f174409e410 0x7f1744123060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:04.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.499+0000 7f174c5ed700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17441241f0 con 0x7f174409e000 2023-09-30T22:32:04.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.499+0000 7f174c5ed700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f174409c780 con 0x7f174409d6f0 2023-09-30T22:32:04.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.499+0000 7f174c5ed700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1744125e20 con 0x7f174409e410 2023-09-30T22:32:04.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.500+0000 7f174ab8a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f174409e410 0x7f1744123060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:04.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.500+0000 7f174ab8a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f174409e410 0x7f1744123060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:34538/0 (socket says 172.21.15.16:34538) 2023-09-30T22:32:04.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.500+0000 7f174ab8a700 1 -- 172.21.15.16:0/73346618 learned_addr learned my addr 172.21.15.16:0/73346618 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:04.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.500+0000 7f1749b88700 1 --2- 172.21.15.16:0/73346618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f174409e000 0x7f1744122b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:04.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.500+0000 7f174a389700 1 --2- 172.21.15.16:0/73346618 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f174409d6f0 0x7f17441225e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:04.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.500+0000 7f1749b88700 1 -- 172.21.15.16:0/73346618 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f174409e410 msgr2=0x7f1744123060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:04.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.500+0000 7f1749b88700 1 --2- 172.21.15.16:0/73346618 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f174409e410 0x7f1744123060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:04.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.500+0000 7f1749b88700 1 -- 172.21.15.16:0/73346618 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f174409d6f0 msgr2=0x7f17441225e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:04.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.500+0000 7f1749b88700 1 --2- 172.21.15.16:0/73346618 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f174409d6f0 0x7f17441225e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:04.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.500+0000 7f1749b88700 1 -- 172.21.15.16:0/73346618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17400094a0 con 0x7f174409e000 2023-09-30T22:32:04.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.500+0000 7f174ab8a700 1 --2- 172.21.15.16:0/73346618 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f174409e410 0x7f1744123060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:04.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.500+0000 7f1749b88700 1 --2- 172.21.15.16:0/73346618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f174409e000 0x7f1744122b20 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f173400e8b0 tx=0x7f173400ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:04.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.501+0000 7f173affd700 1 -- 172.21.15.16:0/73346618 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1734013680 con 0x7f174409e000 2023-09-30T22:32:04.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.501+0000 7f173affd700 1 -- 172.21.15.16:0/73346618 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17340137e0 con 0x7f174409e000 2023-09-30T22:32:04.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.501+0000 7f173affd700 1 -- 172.21.15.16:0/73346618 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1734013b50 con 0x7f174409e000 2023-09-30T22:32:04.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.501+0000 7f174c5ed700 1 -- 172.21.15.16:0/73346618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1744124a10 con 0x7f174409e000 2023-09-30T22:32:04.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.501+0000 7f174c5ed700 1 -- 172.21.15.16:0/73346618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1744133530 con 0x7f174409e000 2023-09-30T22:32:04.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.502+0000 7f174c5ed700 1 -- 172.21.15.16:0/73346618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1744133160 con 0x7f174409e000 2023-09-30T22:32:04.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.502+0000 7f173affd700 1 -- 172.21.15.16:0/73346618 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1734013cb0 con 0x7f174409e000 2023-09-30T22:32:04.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.502+0000 7f173affd700 1 -- 172.21.15.16:0/73346618 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f173403f9c0 con 0x7f174409e000 2023-09-30T22:32:04.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.512+0000 7f173affd700 1 -- 172.21.15.16:0/73346618 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1734011d90 con 0x7f174409e000 2023-09-30T22:32:04.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.519+0000 7fbb537b0700 1 -- 172.21.15.16:0/1008369520 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fbb4c136ff0 con 0x7fbb4c138d80 2023-09-30T22:32:04.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.520+0000 7fbb41ffb700 1 -- 172.21.15.16:0/1008369520 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbb4c136ff0 con 0x7fbb4c138d80 2023-09-30T22:32:04.520 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:32:04.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.522+0000 7fbb537b0700 1 -- 172.21.15.16:0/1008369520 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbb4c138d80 msgr2=0x7fbb4c1340f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:04.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.522+0000 7fbb537b0700 1 --2- 172.21.15.16:0/1008369520 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbb4c138d80 0x7fbb4c1340f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fbb4400efc0 tx=0x7fbb4400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:32:04.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.522+0000 7fbb537b0700 1 -- 172.21.15.16:0/1008369520 shutdown_connections 2023-09-30T22:32:04.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.522+0000 7fbb537b0700 1 --2- 172.21.15.16:0/1008369520 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbb4c138d80 0x7fbb4c1340f0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:04.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.522+0000 7fbb537b0700 1 --2- 172.21.15.16:0/1008369520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbb4c09d630 0x7fbb4c133670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:04.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.522+0000 7fbb537b0700 1 --2- 172.21.15.16:0/1008369520 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbb4c0a0a30 0x7fbb4c133bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:04.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.522+0000 7fbb537b0700 1 -- 172.21.15.16:0/1008369520 >> 172.21.15.16:0/1008369520 conn(0x7fbb4c0a3050 msgr2=0x7fbb4c09ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:04.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.523+0000 7fbb537b0700 1 -- 172.21.15.16:0/1008369520 shutdown_connections 2023-09-30T22:32:04.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.523+0000 7fbb537b0700 1 -- 172.21.15.16:0/1008369520 wait complete. 2023-09-30T22:32:04.543 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:32:04.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.654+0000 7fe2a0f89700 1 -- 172.21.15.16:0/2966337636 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe29c074ea0 con 0x7fe29c133420 2023-09-30T22:32:04.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.654+0000 7fe28affd700 1 -- 172.21.15.16:0/2966337636 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe28c044070 con 0x7fe29c133420 2023-09-30T22:32:04.655 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:32:04.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.656+0000 7fe2a0f89700 1 -- 172.21.15.16:0/2966337636 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe29c133420 msgr2=0x7fe29c12ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:04.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.657+0000 7fe2a0f89700 1 --2- 172.21.15.16:0/2966337636 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe29c133420 0x7fe29c12ee50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fe28c00c780 tx=0x7fe28c00ca90 comp rx=0 tx=0).stop 2023-09-30T22:32:04.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.657+0000 7fe2a0f89700 1 -- 172.21.15.16:0/2966337636 shutdown_connections 2023-09-30T22:32:04.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.657+0000 7fe2a0f89700 1 --2- 172.21.15.16:0/2966337636 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe29c138dc0 0x7fe29c12f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:04.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.657+0000 7fe2a0f89700 1 --2- 172.21.15.16:0/2966337636 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe29c133420 0x7fe29c12ee50 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:04.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.657+0000 7fe2a0f89700 1 --2- 172.21.15.16:0/2966337636 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe29c1337f0 0x7fe29c12f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:04.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.657+0000 7fe2a0f89700 1 -- 172.21.15.16:0/2966337636 >> 172.21.15.16:0/2966337636 conn(0x7fe29c0a4210 msgr2=0x7fe29c139880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:04.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.657+0000 7fe2a0f89700 1 -- 172.21.15.16:0/2966337636 shutdown_connections 2023-09-30T22:32:04.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.657+0000 7fe2a0f89700 1 -- 172.21.15.16:0/2966337636 wait complete. 2023-09-30T22:32:04.671 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:32:04.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.689+0000 7f834a2b1700 1 -- 172.21.15.16:0/1918583980 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f834409c700 con 0x7f834409d6f0 2023-09-30T22:32:04.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.690+0000 7f8340ff9700 1 -- 172.21.15.16:0/1918583980 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8334024e60 con 0x7f834409d6f0 2023-09-30T22:32:04.690 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:32:04.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.692+0000 7f834a2b1700 1 -- 172.21.15.16:0/1918583980 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f834409d6f0 msgr2=0x7f834411e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:04.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.692+0000 7f834a2b1700 1 --2- 172.21.15.16:0/1918583980 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f834409d6f0 0x7f834411e2d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f83340098f0 tx=0x7f8334004750 comp rx=0 tx=0).stop 2023-09-30T22:32:04.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.692+0000 7f834a2b1700 1 -- 172.21.15.16:0/1918583980 shutdown_connections 2023-09-30T22:32:04.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.692+0000 7f834a2b1700 1 --2- 172.21.15.16:0/1918583980 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f834409e410 0x7f83441cc6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:04.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.692+0000 7f834a2b1700 1 --2- 172.21.15.16:0/1918583980 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f834409d6f0 0x7f834411e2d0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:04.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.692+0000 7f834a2b1700 1 --2- 172.21.15.16:0/1918583980 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f834409e000 0x7f834411e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:04.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.692+0000 7f834a2b1700 1 -- 172.21.15.16:0/1918583980 >> 172.21.15.16:0/1918583980 conn(0x7f8344066f50 msgr2=0x7f8344071110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:04.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.692+0000 7f834a2b1700 1 -- 172.21.15.16:0/1918583980 shutdown_connections 2023-09-30T22:32:04.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.692+0000 7f834a2b1700 1 -- 172.21.15.16:0/1918583980 wait complete. 2023-09-30T22:32:04.706 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:32:04.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.725+0000 7f174c5ed700 1 -- 172.21.15.16:0/73346618 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1744074ea0 con 0x7f174409e000 2023-09-30T22:32:04.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.725+0000 7f173affd700 1 -- 172.21.15.16:0/73346618 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1734021070 con 0x7f174409e000 2023-09-30T22:32:04.727 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:32:04.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.728+0000 7f174c5ed700 1 -- 172.21.15.16:0/73346618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f174409e000 msgr2=0x7f1744122b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:04.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.728+0000 7f174c5ed700 1 --2- 172.21.15.16:0/73346618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f174409e000 0x7f1744122b20 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f173400e8b0 tx=0x7f173400ebc0 comp rx=0 tx=0).stop 2023-09-30T22:32:04.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.728+0000 7f174c5ed700 1 -- 172.21.15.16:0/73346618 shutdown_connections 2023-09-30T22:32:04.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.728+0000 7f174c5ed700 1 --2- 172.21.15.16:0/73346618 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f174409d6f0 0x7f17441225e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:04.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.728+0000 7f174c5ed700 1 --2- 172.21.15.16:0/73346618 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f174409e410 0x7f1744123060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:04.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.728+0000 7f174c5ed700 1 --2- 172.21.15.16:0/73346618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f174409e000 0x7f1744122b20 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:04.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.728+0000 7f174c5ed700 1 -- 172.21.15.16:0/73346618 >> 172.21.15.16:0/73346618 conn(0x7f1744066f50 msgr2=0x7f1744071150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:04.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.728+0000 7f174c5ed700 1 -- 172.21.15.16:0/73346618 shutdown_connections 2023-09-30T22:32:04.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:04.728+0000 7f174c5ed700 1 -- 172.21.15.16:0/73346618 wait complete. 2023-09-30T22:32:04.742 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:32:05.544 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:32:05.671 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:32:05.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.690+0000 7fc6d01e9700 1 Processor -- start 2023-09-30T22:32:05.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.690+0000 7fc6d01e9700 1 -- start start 2023-09-30T22:32:05.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.691+0000 7fc6d01e9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc6c80a0a30 0x7fc6c80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:05.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.691+0000 7fc6d01e9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc6c809d630 0x7fc6c80a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:05.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.691+0000 7fc6d01e9700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6c80a2a70 con 0x7fc6c8133930 2023-09-30T22:32:05.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.691+0000 7fc6d01e9700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c80a01d0 con 0x7fc6c80a0a30 2023-09-30T22:32:05.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.691+0000 7fc6d01e9700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c809e2b0 con 0x7fc6c809d630 2023-09-30T22:32:05.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.691+0000 7fc6cdf85700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc6c80a0a30 0x7fc6c80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:05.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.691+0000 7fc6cdf85700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc6c80a0a30 0x7fc6c80a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:44420/0 (socket says 172.21.15.16:44420) 2023-09-30T22:32:05.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.691+0000 7fc6cdf85700 1 -- 172.21.15.16:0/752327157 learned_addr learned my addr 172.21.15.16:0/752327157 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:05.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.691+0000 7fc6cd784700 1 --2- 172.21.15.16:0/752327157 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc6c809d630 0x7fc6c80a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:05.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.691+0000 7fc6cdf85700 1 -- 172.21.15.16:0/752327157 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc6c809d630 msgr2=0x7fc6c80a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:05.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.691+0000 7fc6cdf85700 1 --2- 172.21.15.16:0/752327157 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc6c809d630 0x7fc6c80a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:05.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.691+0000 7fc6cdf85700 1 -- 172.21.15.16:0/752327157 >> v1:172.21.15.16:6789/0 conn(0x7fc6c8133930 legacy=0x7fc6c8133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:05.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.691+0000 7fc6cdf85700 1 -- 172.21.15.16:0/752327157 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c809dc50 con 0x7fc6c80a0a30 2023-09-30T22:32:05.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.691+0000 7fc6cdf85700 1 --2- 172.21.15.16:0/752327157 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc6c80a0a30 0x7fc6c80a0e00 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0009e80 tx=0x7fc6c000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a4272c937f3be57 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:05.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.692+0000 7fc6ccf83700 1 -- 172.21.15.16:0/752327157 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c000e070 con 0x7fc6c80a0a30 2023-09-30T22:32:05.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.692+0000 7fc6ccf83700 1 -- 172.21.15.16:0/752327157 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6c000b910 con 0x7fc6c80a0a30 2023-09-30T22:32:05.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.692+0000 7fc6ccf83700 1 -- 172.21.15.16:0/752327157 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c000bcc0 con 0x7fc6c80a0a30 2023-09-30T22:32:05.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.692+0000 7fc6d01e9700 1 -- 172.21.15.16:0/752327157 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc6c80a0a30 msgr2=0x7fc6c80a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:05.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.692+0000 7fc6d01e9700 1 --2- 172.21.15.16:0/752327157 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc6c80a0a30 0x7fc6c80a0e00 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0009e80 tx=0x7fc6c000b330 comp rx=0 tx=0).stop 2023-09-30T22:32:05.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.692+0000 7fc6d01e9700 1 -- 172.21.15.16:0/752327157 shutdown_connections 2023-09-30T22:32:05.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.692+0000 7fc6d01e9700 1 --2- 172.21.15.16:0/752327157 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc6c80a0a30 0x7fc6c80a0e00 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:05.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.692+0000 7fc6d01e9700 1 --2- 172.21.15.16:0/752327157 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc6c809d630 0x7fc6c80a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:05.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.692+0000 7fc6d01e9700 1 -- 172.21.15.16:0/752327157 >> 172.21.15.16:0/752327157 conn(0x7fc6c80a3050 msgr2=0x7fc6c80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:05.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.692+0000 7fc6d01e9700 1 -- 172.21.15.16:0/752327157 shutdown_connections 2023-09-30T22:32:05.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.692+0000 7fc6d01e9700 1 -- 172.21.15.16:0/752327157 wait complete. 2023-09-30T22:32:05.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.693+0000 7fc6d01e9700 1 Processor -- start 2023-09-30T22:32:05.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.693+0000 7fc6d01e9700 1 -- start start 2023-09-30T22:32:05.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.694+0000 7fc6d01e9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc6c809d630 0x7fc6c8132340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:05.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.694+0000 7fc6d01e9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6c80a0a30 0x7fc6c8132880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:05.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.694+0000 7fc6d01e9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc6c8133930 0x7fc6c8132dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:05.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.694+0000 7fc6d01e9700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c8121240 con 0x7fc6c80a0a30 2023-09-30T22:32:05.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.694+0000 7fc6d01e9700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c81204d0 con 0x7fc6c8133930 2023-09-30T22:32:05.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.694+0000 7fc6d01e9700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c8121e40 con 0x7fc6c809d630 2023-09-30T22:32:05.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.694+0000 7fc6cd784700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6c80a0a30 0x7fc6c8132880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:05.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.694+0000 7fc6cdf85700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc6c809d630 0x7fc6c8132340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:05.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.694+0000 7fc6cd784700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6c80a0a30 0x7fc6c8132880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54212/0 (socket says 172.21.15.16:54212) 2023-09-30T22:32:05.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.694+0000 7fc6cd784700 1 -- 172.21.15.16:0/1302019309 learned_addr learned my addr 172.21.15.16:0/1302019309 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:05.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.694+0000 7fc6ce786700 1 --2- 172.21.15.16:0/1302019309 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc6c8133930 0x7fc6c8132dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:05.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.694+0000 7fc6cdf85700 1 -- 172.21.15.16:0/1302019309 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc6c8133930 msgr2=0x7fc6c8132dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:05.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.694+0000 7fc6cdf85700 1 --2- 172.21.15.16:0/1302019309 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc6c8133930 0x7fc6c8132dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:05.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.694+0000 7fc6cdf85700 1 -- 172.21.15.16:0/1302019309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6c80a0a30 msgr2=0x7fc6c8132880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:05.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.694+0000 7fc6cdf85700 1 --2- 172.21.15.16:0/1302019309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6c80a0a30 0x7fc6c8132880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:05.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.694+0000 7fc6cdf85700 1 -- 172.21.15.16:0/1302019309 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c0009b30 con 0x7fc6c809d630 2023-09-30T22:32:05.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.694+0000 7fc6cd784700 1 --2- 172.21.15.16:0/1302019309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6c80a0a30 0x7fc6c8132880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:05.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.695+0000 7fc6cdf85700 1 --2- 172.21.15.16:0/1302019309 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc6c809d630 0x7fc6c8132340 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0015000 tx=0x7fc6c0004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:05.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.695+0000 7fc6c67fc700 1 -- 172.21.15.16:0/1302019309 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c00092e0 con 0x7fc6c809d630 2023-09-30T22:32:05.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.695+0000 7fc6c67fc700 1 -- 172.21.15.16:0/1302019309 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6c0004250 con 0x7fc6c809d630 2023-09-30T22:32:05.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.695+0000 7fc6c67fc700 1 -- 172.21.15.16:0/1302019309 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c0004600 con 0x7fc6c809d630 2023-09-30T22:32:05.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.695+0000 7fc6d01e9700 1 -- 172.21.15.16:0/1302019309 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6c80a4030 con 0x7fc6c809d630 2023-09-30T22:32:05.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.695+0000 7fc6d01e9700 1 -- 172.21.15.16:0/1302019309 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6c8133370 con 0x7fc6c809d630 2023-09-30T22:32:05.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.696+0000 7fc6c67fc700 1 -- 172.21.15.16:0/1302019309 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc6c0004760 con 0x7fc6c809d630 2023-09-30T22:32:05.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.696+0000 7fc6c67fc700 1 -- 172.21.15.16:0/1302019309 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fc6c00409a0 con 0x7fc6c809d630 2023-09-30T22:32:05.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.696+0000 7fc6d01e9700 1 -- 172.21.15.16:0/1302019309 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6c80747f0 con 0x7fc6c809d630 2023-09-30T22:32:05.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.702+0000 7fc6c67fc700 1 -- 172.21.15.16:0/1302019309 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc6c001a740 con 0x7fc6c809d630 2023-09-30T22:32:05.707 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:32:05.742 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:32:05.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.824+0000 7fddcd7b7700 1 Processor -- start 2023-09-30T22:32:05.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.824+0000 7fddcd7b7700 1 -- start start 2023-09-30T22:32:05.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.825+0000 7fddcd7b7700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fddc809f410 0x7fddc8136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:05.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.825+0000 7fddcd7b7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fddc8136fe0 0x7fddc81393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:05.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.825+0000 7fddcd7b7700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddc8070940 con 0x7fddc809e930 2023-09-30T22:32:05.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.825+0000 7fddcd7b7700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddc806fd60 con 0x7fddc809f410 2023-09-30T22:32:05.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.825+0000 7fddcd7b7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fddc806fa80 con 0x7fddc8136fe0 2023-09-30T22:32:05.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.825+0000 7fddc77fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fddc8136fe0 0x7fddc81393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:05.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.825+0000 7fddc77fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fddc8136fe0 0x7fddc81393c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:34556/0 (socket says 172.21.15.16:34556) 2023-09-30T22:32:05.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.825+0000 7fddc77fe700 1 -- 172.21.15.16:0/2276070069 learned_addr learned my addr 172.21.15.16:0/2276070069 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:05.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.826+0000 7fddc67fc700 1 --2- 172.21.15.16:0/2276070069 >> v2:172.21.15.16:3300/0 conn(0x7fddc809f410 0x7fddc8136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:05.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.826+0000 7fddc77fe700 1 -- 172.21.15.16:0/2276070069 >> v2:172.21.15.16:3300/0 conn(0x7fddc809f410 msgr2=0x7fddc8136aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:05.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.826+0000 7fddc77fe700 1 --2- 172.21.15.16:0/2276070069 >> v2:172.21.15.16:3300/0 conn(0x7fddc809f410 0x7fddc8136aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:05.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.826+0000 7fddc77fe700 1 -- 172.21.15.16:0/2276070069 >> v1:172.21.15.16:6789/0 conn(0x7fddc809e930 legacy=0x7fddc809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:05.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.826+0000 7fddc77fe700 1 -- 172.21.15.16:0/2276070069 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc809fb60 con 0x7fddc8136fe0 2023-09-30T22:32:05.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.826+0000 7fddc67fc700 1 --2- 172.21.15.16:0/2276070069 >> v2:172.21.15.16:3300/0 conn(0x7fddc809f410 0x7fddc8136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:32:05.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.826+0000 7fddc77fe700 1 --2- 172.21.15.16:0/2276070069 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fddc8136fe0 0x7fddc81393c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fddc0009e50 tx=0x7fddc000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=550f2edddb608405 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:05.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.826+0000 7fddc5ffb700 1 -- 172.21.15.16:0/2276070069 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc000e070 con 0x7fddc8136fe0 2023-09-30T22:32:05.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.826+0000 7fddc5ffb700 1 -- 172.21.15.16:0/2276070069 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddc000b9c0 con 0x7fddc8136fe0 2023-09-30T22:32:05.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.826+0000 7fddc5ffb700 1 -- 172.21.15.16:0/2276070069 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc000bd70 con 0x7fddc8136fe0 2023-09-30T22:32:05.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.827+0000 7fddcd7b7700 1 -- 172.21.15.16:0/2276070069 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fddc8136fe0 msgr2=0x7fddc81393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:05.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.827+0000 7fddcd7b7700 1 --2- 172.21.15.16:0/2276070069 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fddc8136fe0 0x7fddc81393c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fddc0009e50 tx=0x7fddc000b330 comp rx=0 tx=0).stop 2023-09-30T22:32:05.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.827+0000 7fddcd7b7700 1 -- 172.21.15.16:0/2276070069 shutdown_connections 2023-09-30T22:32:05.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.827+0000 7fddcd7b7700 1 --2- 172.21.15.16:0/2276070069 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fddc8136fe0 0x7fddc81393c0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:05.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.827+0000 7fddcd7b7700 1 --2- 172.21.15.16:0/2276070069 >> v2:172.21.15.16:3300/0 conn(0x7fddc809f410 0x7fddc8136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:05.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.827+0000 7fddcd7b7700 1 -- 172.21.15.16:0/2276070069 >> 172.21.15.16:0/2276070069 conn(0x7fddc80a4210 msgr2=0x7fddc80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:05.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.828+0000 7fddcd7b7700 1 -- 172.21.15.16:0/2276070069 shutdown_connections 2023-09-30T22:32:05.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.828+0000 7fddcd7b7700 1 -- 172.21.15.16:0/2276070069 wait complete. 2023-09-30T22:32:05.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.829+0000 7fddcd7b7700 1 Processor -- start 2023-09-30T22:32:05.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.829+0000 7fddcd7b7700 1 -- start start 2023-09-30T22:32:05.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.830+0000 7fddcd7b7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddc809e930 0x7fddc811c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:05.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.830+0000 7fddcd7b7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fddc809f410 0x7fddc811ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:05.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.830+0000 7fddcd7b7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fddc8136fe0 0x7fddc811cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:05.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.830+0000 7fddcd7b7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc8071db0 con 0x7fddc809e930 2023-09-30T22:32:05.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.830+0000 7fddcd7b7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddc80a33f0 con 0x7fddc809f410 2023-09-30T22:32:05.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.830+0000 7fddcd7b7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fddc811f820 con 0x7fddc8136fe0 2023-09-30T22:32:05.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.830+0000 7fddc67fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fddc809f410 0x7fddc811ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:05.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.830+0000 7fddc67fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fddc809f410 0x7fddc811ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:44436/0 (socket says 172.21.15.16:44436) 2023-09-30T22:32:05.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.830+0000 7fddc67fc700 1 -- 172.21.15.16:0/3823323914 learned_addr learned my addr 172.21.15.16:0/3823323914 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:05.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.830+0000 7fddc6ffd700 1 --2- 172.21.15.16:0/3823323914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddc809e930 0x7fddc811c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:05.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.830+0000 7fddc77fe700 1 --2- 172.21.15.16:0/3823323914 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fddc8136fe0 0x7fddc811cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:05.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.831+0000 7fddc6ffd700 1 -- 172.21.15.16:0/3823323914 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fddc8136fe0 msgr2=0x7fddc811cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:05.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.831+0000 7fddc6ffd700 1 --2- 172.21.15.16:0/3823323914 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fddc8136fe0 0x7fddc811cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:05.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.831+0000 7fddc6ffd700 1 -- 172.21.15.16:0/3823323914 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fddc809f410 msgr2=0x7fddc811ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:05.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.831+0000 7fddc6ffd700 1 --2- 172.21.15.16:0/3823323914 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fddc809f410 0x7fddc811ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:05.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.831+0000 7fddc6ffd700 1 -- 172.21.15.16:0/3823323914 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc0009b30 con 0x7fddc809e930 2023-09-30T22:32:05.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.831+0000 7fddc77fe700 1 --2- 172.21.15.16:0/3823323914 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fddc8136fe0 0x7fddc811cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:32:05.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.831+0000 7fddc6ffd700 1 --2- 172.21.15.16:0/3823323914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddc809e930 0x7fddc811c550 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fddb8000f60 tx=0x7fddb800c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:05.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.831+0000 7fddaf7fe700 1 -- 172.21.15.16:0/3823323914 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb80052d0 con 0x7fddc809e930 2023-09-30T22:32:05.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.831+0000 7fddaf7fe700 1 -- 172.21.15.16:0/3823323914 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb8005430 con 0x7fddc809e930 2023-09-30T22:32:05.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.831+0000 7fddcd7b7700 1 -- 172.21.15.16:0/3823323914 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddc811e9c0 con 0x7fddc809e930 2023-09-30T22:32:05.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.831+0000 7fddaf7fe700 1 -- 172.21.15.16:0/3823323914 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb80038c0 con 0x7fddc809e930 2023-09-30T22:32:05.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.831+0000 7fddcd7b7700 1 -- 172.21.15.16:0/3823323914 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddc8120800 con 0x7fddc809e930 2023-09-30T22:32:05.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.832+0000 7fddaf7fe700 1 -- 172.21.15.16:0/3823323914 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fddb8003a20 con 0x7fddc809e930 2023-09-30T22:32:05.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.832+0000 7fddaf7fe700 1 -- 172.21.15.16:0/3823323914 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fddb803ea20 con 0x7fddc809e930 2023-09-30T22:32:05.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.833+0000 7fddcd7b7700 1 -- 172.21.15.16:0/3823323914 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddc8074ea0 con 0x7fddc809e930 2023-09-30T22:32:05.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.843+0000 7fddaf7fe700 1 -- 172.21.15.16:0/3823323914 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fddb8018260 con 0x7fddc809e930 2023-09-30T22:32:05.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.859+0000 7f26feda3700 1 Processor -- start 2023-09-30T22:32:05.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.859+0000 7f26feda3700 1 -- start start 2023-09-30T22:32:05.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.860+0000 7f26feda3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f26f81338a0 0x7f26f8133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:05.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.860+0000 7f26feda3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f26f809e930 0x7f26f813b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:05.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.860+0000 7f26feda3700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26f806ef30 con 0x7f26f809de50 2023-09-30T22:32:05.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.860+0000 7f26feda3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26f806fef0 con 0x7f26f81338a0 2023-09-30T22:32:05.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.860+0000 7f26feda3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f26f81334b0 con 0x7f26f809e930 2023-09-30T22:32:05.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.860+0000 7f26f7fff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f26f809de50 0x7f26f809e220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60926/0 (socket says 172.21.15.16:60926) 2023-09-30T22:32:05.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.860+0000 7f26f7fff700 1 -- 172.21.15.16:0/969072542 learned_addr learned my addr 172.21.15.16:0/969072542 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:05.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.860+0000 7f26fcb3f700 1 --2- 172.21.15.16:0/969072542 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f26f81338a0 0x7f26f8133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:05.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.860+0000 7f26fd340700 1 --2- 172.21.15.16:0/969072542 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f26f809e930 0x7f26f813b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:05.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.860+0000 7f26fcb3f700 1 -- 172.21.15.16:0/969072542 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f26f809e930 msgr2=0x7f26f813b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:05.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.860+0000 7f26fcb3f700 1 --2- 172.21.15.16:0/969072542 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f26f809e930 0x7f26f813b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:05.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.860+0000 7f26fcb3f700 1 -- 172.21.15.16:0/969072542 >> v1:172.21.15.16:6789/0 conn(0x7f26f809de50 legacy=0x7f26f809e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:05.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.860+0000 7f26fcb3f700 1 -- 172.21.15.16:0/969072542 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f806fc30 con 0x7f26f81338a0 2023-09-30T22:32:05.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.860+0000 7f26f77fe700 1 -- 172.21.15.16:0/969072542 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2340245879 0 0) 0x7f26f806ef30 con 0x7f26f809de50 2023-09-30T22:32:05.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.861+0000 7f26fcb3f700 1 --2- 172.21.15.16:0/969072542 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f26f81338a0 0x7f26f8133c70 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f26ec009870 tx=0x7f26ec009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3cd2180d1742c72 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:05.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.861+0000 7f26f77fe700 1 -- 172.21.15.16:0/969072542 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ec011830 con 0x7f26f81338a0 2023-09-30T22:32:05.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.861+0000 7f26f77fe700 1 -- 172.21.15.16:0/969072542 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26ec011990 con 0x7f26f81338a0 2023-09-30T22:32:05.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.861+0000 7f26f77fe700 1 -- 172.21.15.16:0/969072542 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ec011d20 con 0x7f26f81338a0 2023-09-30T22:32:05.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.862+0000 7f26feda3700 1 -- 172.21.15.16:0/969072542 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f26f81338a0 msgr2=0x7f26f8133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:05.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.862+0000 7f26feda3700 1 --2- 172.21.15.16:0/969072542 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f26f81338a0 0x7f26f8133c70 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f26ec009870 tx=0x7f26ec009b80 comp rx=0 tx=0).stop 2023-09-30T22:32:05.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.862+0000 7f26feda3700 1 -- 172.21.15.16:0/969072542 shutdown_connections 2023-09-30T22:32:05.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.862+0000 7f26feda3700 1 --2- 172.21.15.16:0/969072542 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f26f809e930 0x7f26f813b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:05.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.862+0000 7f26feda3700 1 --2- 172.21.15.16:0/969072542 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f26f81338a0 0x7f26f8133c70 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:05.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.862+0000 7f26feda3700 1 -- 172.21.15.16:0/969072542 >> 172.21.15.16:0/969072542 conn(0x7f26f80a2aa0 msgr2=0x7f26f80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:05.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.862+0000 7f26feda3700 1 -- 172.21.15.16:0/969072542 shutdown_connections 2023-09-30T22:32:05.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.862+0000 7f26feda3700 1 -- 172.21.15.16:0/969072542 wait complete. 2023-09-30T22:32:05.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.863+0000 7f26feda3700 1 Processor -- start 2023-09-30T22:32:05.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.863+0000 7f26feda3700 1 -- start start 2023-09-30T22:32:05.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.864+0000 7f26feda3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f26f809de50 0x7f26f811d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:05.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.864+0000 7f26feda3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26f809e930 0x7f26f811dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:05.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.864+0000 7f26feda3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f26f81338a0 0x7f26f811e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:05.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.864+0000 7f26feda3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f811eb70 con 0x7f26f809e930 2023-09-30T22:32:05.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.864+0000 7f26feda3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26f811f900 con 0x7f26f809de50 2023-09-30T22:32:05.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.864+0000 7f26feda3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f26f81209c0 con 0x7f26f81338a0 2023-09-30T22:32:05.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.865+0000 7f26fd340700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f26f81338a0 0x7f26f811e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:05.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.865+0000 7f26fd340700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f26f81338a0 0x7f26f811e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:34568/0 (socket says 172.21.15.16:34568) 2023-09-30T22:32:05.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.865+0000 7f26fd340700 1 -- 172.21.15.16:0/4282490507 learned_addr learned my addr 172.21.15.16:0/4282490507 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:05.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.865+0000 7f26f7fff700 1 --2- 172.21.15.16:0/4282490507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26f809e930 0x7f26f811dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:05.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.865+0000 7f26fcb3f700 1 --2- 172.21.15.16:0/4282490507 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f26f809de50 0x7f26f811d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:05.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.865+0000 7f26fcb3f700 1 -- 172.21.15.16:0/4282490507 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f26f81338a0 msgr2=0x7f26f811e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:05.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.865+0000 7f26fcb3f700 1 --2- 172.21.15.16:0/4282490507 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f26f81338a0 0x7f26f811e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:05.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.865+0000 7f26fcb3f700 1 -- 172.21.15.16:0/4282490507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26f809e930 msgr2=0x7f26f811dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:05.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.865+0000 7f26fcb3f700 1 --2- 172.21.15.16:0/4282490507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26f809e930 0x7f26f811dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:05.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.865+0000 7f26fcb3f700 1 -- 172.21.15.16:0/4282490507 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26ec0094a0 con 0x7f26f809de50 2023-09-30T22:32:05.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.865+0000 7f26fd340700 1 --2- 172.21.15.16:0/4282490507 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f26f81338a0 0x7f26f811e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:05.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.865+0000 7f26fcb3f700 1 --2- 172.21.15.16:0/4282490507 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f26f809de50 0x7f26f811d710 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f26ec016000 tx=0x7f26ec004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:05.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.865+0000 7f26f57fa700 1 -- 172.21.15.16:0/4282490507 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ec004510 con 0x7f26f809de50 2023-09-30T22:32:05.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.866+0000 7f26f57fa700 1 -- 172.21.15.16:0/4282490507 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26ec0036a0 con 0x7f26f809de50 2023-09-30T22:32:05.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.866+0000 7f26feda3700 1 -- 172.21.15.16:0/4282490507 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26f806f640 con 0x7f26f809de50 2023-09-30T22:32:05.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.866+0000 7f26f57fa700 1 -- 172.21.15.16:0/4282490507 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ec003a50 con 0x7f26f809de50 2023-09-30T22:32:05.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.866+0000 7f26feda3700 1 -- 172.21.15.16:0/4282490507 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26f811f320 con 0x7f26f809de50 2023-09-30T22:32:05.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.867+0000 7f26feda3700 1 -- 172.21.15.16:0/4282490507 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26f8120bf0 con 0x7f26f809de50 2023-09-30T22:32:05.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.867+0000 7f26f57fa700 1 -- 172.21.15.16:0/4282490507 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f26ec003bb0 con 0x7f26f809de50 2023-09-30T22:32:05.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.867+0000 7f26f57fa700 1 -- 172.21.15.16:0/4282490507 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f26ec0409e0 con 0x7f26f809de50 2023-09-30T22:32:05.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.876+0000 7f26f57fa700 1 -- 172.21.15.16:0/4282490507 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f26f8120bf0 con 0x7f26f809de50 2023-09-30T22:32:05.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.900+0000 7ffb29b57700 1 Processor -- start 2023-09-30T22:32:05.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.901+0000 7ffb29b57700 1 -- start start 2023-09-30T22:32:05.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.901+0000 7ffb29b57700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffb240a0a30 0x7ffb240a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:05.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.902+0000 7ffb29b57700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ffb240a1340 0x7ffb24133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:05.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.902+0000 7ffb29b57700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb2409edd0 con 0x7ffb24133fa0 2023-09-30T22:32:05.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.902+0000 7ffb29b57700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb2409e1f0 con 0x7ffb240a1340 2023-09-30T22:32:05.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.902+0000 7ffb29b57700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2409dc50 con 0x7ffb240a0a30 2023-09-30T22:32:05.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.902+0000 7ffb237fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffb240a0a30 0x7ffb240a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:05.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.902+0000 7ffb237fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffb240a0a30 0x7ffb240a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:34572/0 (socket says 172.21.15.16:34572) 2023-09-30T22:32:05.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.902+0000 7ffb237fe700 1 -- 172.21.15.16:0/3652414283 learned_addr learned my addr 172.21.15.16:0/3652414283 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:05.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.902+0000 7ffb23fff700 1 --1- 172.21.15.16:0/3652414283 >> v1:172.21.15.16:6789/0 conn(0x7ffb24133fa0 0x7ffb2413ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60930/0 (socket says 172.21.15.16:60930) 2023-09-30T22:32:05.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.902+0000 7ffb22ffd700 1 --2- 172.21.15.16:0/3652414283 >> v2:172.21.15.16:3300/0 conn(0x7ffb240a1340 0x7ffb24133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:05.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.902+0000 7ffb237fe700 1 -- 172.21.15.16:0/3652414283 >> v2:172.21.15.16:3300/0 conn(0x7ffb240a1340 msgr2=0x7ffb24133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:05.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.902+0000 7ffb237fe700 1 --2- 172.21.15.16:0/3652414283 >> v2:172.21.15.16:3300/0 conn(0x7ffb240a1340 0x7ffb24133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:05.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.902+0000 7ffb237fe700 1 -- 172.21.15.16:0/3652414283 >> v1:172.21.15.16:6789/0 conn(0x7ffb24133fa0 legacy=0x7ffb2413ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:05.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.902+0000 7ffb237fe700 1 -- 172.21.15.16:0/3652414283 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb240a1fe0 con 0x7ffb240a0a30 2023-09-30T22:32:05.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.902+0000 7ffb237fe700 1 --2- 172.21.15.16:0/3652414283 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffb240a0a30 0x7ffb240a0e00 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ffb140098f0 tx=0x7ffb14009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a19560c225b3f103 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:05.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.903+0000 7ffb227fc700 1 -- 172.21.15.16:0/3652414283 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb14011830 con 0x7ffb240a0a30 2023-09-30T22:32:05.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.903+0000 7ffb227fc700 1 -- 172.21.15.16:0/3652414283 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb14011990 con 0x7ffb240a0a30 2023-09-30T22:32:05.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.903+0000 7ffb227fc700 1 -- 172.21.15.16:0/3652414283 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb14011d20 con 0x7ffb240a0a30 2023-09-30T22:32:05.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.904+0000 7ffb29b57700 1 -- 172.21.15.16:0/3652414283 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffb240a0a30 msgr2=0x7ffb240a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:05.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.904+0000 7ffb29b57700 1 --2- 172.21.15.16:0/3652414283 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffb240a0a30 0x7ffb240a0e00 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ffb140098f0 tx=0x7ffb14009c00 comp rx=0 tx=0).stop 2023-09-30T22:32:05.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.904+0000 7ffb29b57700 1 -- 172.21.15.16:0/3652414283 shutdown_connections 2023-09-30T22:32:05.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.904+0000 7ffb29b57700 1 --2- 172.21.15.16:0/3652414283 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffb240a0a30 0x7ffb240a0e00 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:05.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.904+0000 7ffb29b57700 1 --2- 172.21.15.16:0/3652414283 >> v2:172.21.15.16:3300/0 conn(0x7ffb240a1340 0x7ffb24133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:05.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.904+0000 7ffb29b57700 1 -- 172.21.15.16:0/3652414283 >> 172.21.15.16:0/3652414283 conn(0x7ffb240a6550 msgr2=0x7ffb240a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:05.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.904+0000 7ffb29b57700 1 -- 172.21.15.16:0/3652414283 shutdown_connections 2023-09-30T22:32:05.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.904+0000 7ffb29b57700 1 -- 172.21.15.16:0/3652414283 wait complete. 2023-09-30T22:32:05.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.905+0000 7ffb29b57700 1 Processor -- start 2023-09-30T22:32:05.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.906+0000 7ffb29b57700 1 -- start start 2023-09-30T22:32:05.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.906+0000 7ffb29b57700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb240a0a30 0x7ffb241ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:05.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.906+0000 7ffb29b57700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffb240a1340 0x7ffb241cd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:05.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.906+0000 7ffb29b57700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffb24133fa0 0x7ffb241c9250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:05.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.906+0000 7ffb29b57700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb240a0220 con 0x7ffb240a0a30 2023-09-30T22:32:05.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.906+0000 7ffb29b57700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb24129ec0 con 0x7ffb240a1340 2023-09-30T22:32:05.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.906+0000 7ffb29b57700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffb240a37e0 con 0x7ffb24133fa0 2023-09-30T22:32:05.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.907+0000 7ffb22ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffb240a1340 0x7ffb241cd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:05.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.907+0000 7ffb23fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffb24133fa0 0x7ffb241c9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:05.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.907+0000 7ffb22ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffb240a1340 0x7ffb241cd1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:44468/0 (socket says 172.21.15.16:44468) 2023-09-30T22:32:05.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.907+0000 7ffb22ffd700 1 -- 172.21.15.16:0/1207582321 learned_addr learned my addr 172.21.15.16:0/1207582321 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:05.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.907+0000 7ffb237fe700 1 --2- 172.21.15.16:0/1207582321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb240a0a30 0x7ffb241ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:05.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.907+0000 7ffb23fff700 1 -- 172.21.15.16:0/1207582321 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffb240a1340 msgr2=0x7ffb241cd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:05.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.907+0000 7ffb23fff700 1 --2- 172.21.15.16:0/1207582321 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffb240a1340 0x7ffb241cd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:05.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.907+0000 7ffb23fff700 1 -- 172.21.15.16:0/1207582321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb240a0a30 msgr2=0x7ffb241ccca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:05.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.907+0000 7ffb23fff700 1 --2- 172.21.15.16:0/1207582321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb240a0a30 0x7ffb241ccca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:05.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.907+0000 7ffb23fff700 1 -- 172.21.15.16:0/1207582321 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb14009550 con 0x7ffb24133fa0 2023-09-30T22:32:05.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.907+0000 7ffb22ffd700 1 --2- 172.21.15.16:0/1207582321 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffb240a1340 0x7ffb241cd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:05.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.907+0000 7ffb23fff700 1 --2- 172.21.15.16:0/1207582321 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffb24133fa0 0x7ffb241c9250 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c00cb40 tx=0x7ffb1c00ce50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:05.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.908+0000 7ffb0bfff700 1 -- 172.21.15.16:0/1207582321 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1c00a9a0 con 0x7ffb24133fa0 2023-09-30T22:32:05.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.908+0000 7ffb0bfff700 1 -- 172.21.15.16:0/1207582321 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb1c00ab00 con 0x7ffb24133fa0 2023-09-30T22:32:05.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.908+0000 7ffb29b57700 1 -- 172.21.15.16:0/1207582321 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb241cd720 con 0x7ffb24133fa0 2023-09-30T22:32:05.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.908+0000 7ffb0bfff700 1 -- 172.21.15.16:0/1207582321 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1c00ae70 con 0x7ffb24133fa0 2023-09-30T22:32:05.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.908+0000 7ffb29b57700 1 -- 172.21.15.16:0/1207582321 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb24128290 con 0x7ffb24133fa0 2023-09-30T22:32:05.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.909+0000 7ffb29b57700 1 -- 172.21.15.16:0/1207582321 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb24129b40 con 0x7ffb24133fa0 2023-09-30T22:32:05.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.909+0000 7ffb0bfff700 1 -- 172.21.15.16:0/1207582321 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ffb1c007070 con 0x7ffb24133fa0 2023-09-30T22:32:05.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.909+0000 7ffb0bfff700 1 -- 172.21.15.16:0/1207582321 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7ffb1c03f310 con 0x7ffb24133fa0 2023-09-30T22:32:05.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.919+0000 7ffb0bfff700 1 -- 172.21.15.16:0/1207582321 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ffb24129b40 con 0x7ffb24133fa0 2023-09-30T22:32:05.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.942+0000 7fc6d01e9700 1 -- 172.21.15.16:0/1302019309 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc6c8072980 con 0x7fc6c809d630 2023-09-30T22:32:05.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.942+0000 7fc6c67fc700 1 -- 172.21.15.16:0/1302019309 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc6c001b3a0 con 0x7fc6c809d630 2023-09-30T22:32:05.943 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:32:05.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.946+0000 7fc6d01e9700 1 -- 172.21.15.16:0/1302019309 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc6c809d630 msgr2=0x7fc6c8132340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:05.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.946+0000 7fc6d01e9700 1 --2- 172.21.15.16:0/1302019309 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc6c809d630 0x7fc6c8132340 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0015000 tx=0x7fc6c0004ad0 comp rx=0 tx=0).stop 2023-09-30T22:32:05.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.946+0000 7fc6d01e9700 1 -- 172.21.15.16:0/1302019309 shutdown_connections 2023-09-30T22:32:05.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.946+0000 7fc6d01e9700 1 --2- 172.21.15.16:0/1302019309 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc6c8133930 0x7fc6c8132dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:05.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.946+0000 7fc6d01e9700 1 --2- 172.21.15.16:0/1302019309 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc6c809d630 0x7fc6c8132340 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:05.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.946+0000 7fc6d01e9700 1 --2- 172.21.15.16:0/1302019309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6c80a0a30 0x7fc6c8132880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:05.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.946+0000 7fc6d01e9700 1 -- 172.21.15.16:0/1302019309 >> 172.21.15.16:0/1302019309 conn(0x7fc6c80a3050 msgr2=0x7fc6c81bbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:05.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.946+0000 7fc6d01e9700 1 -- 172.21.15.16:0/1302019309 shutdown_connections 2023-09-30T22:32:05.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:05.946+0000 7fc6d01e9700 1 -- 172.21.15.16:0/1302019309 wait complete. 2023-09-30T22:32:05.961 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:32:06.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:06.068+0000 7fddcd7b7700 1 -- 172.21.15.16:0/3823323914 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fddc811fa30 con 0x7fddc809e930 2023-09-30T22:32:06.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:06.069+0000 7fddaf7fe700 1 -- 172.21.15.16:0/3823323914 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fddb8017020 con 0x7fddc809e930 2023-09-30T22:32:06.069 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:32:06.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:06.071+0000 7fddcd7b7700 1 -- 172.21.15.16:0/3823323914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddc809e930 msgr2=0x7fddc811c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:06.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:06.071+0000 7fddcd7b7700 1 --2- 172.21.15.16:0/3823323914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddc809e930 0x7fddc811c550 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fddb8000f60 tx=0x7fddb800c730 comp rx=0 tx=0).stop 2023-09-30T22:32:06.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:06.071+0000 7fddcd7b7700 1 -- 172.21.15.16:0/3823323914 shutdown_connections 2023-09-30T22:32:06.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:06.071+0000 7fddcd7b7700 1 --2- 172.21.15.16:0/3823323914 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fddc8136fe0 0x7fddc811cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:06.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:06.071+0000 7fddcd7b7700 1 --2- 172.21.15.16:0/3823323914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fddc809e930 0x7fddc811c550 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:06.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:06.071+0000 7fddcd7b7700 1 --2- 172.21.15.16:0/3823323914 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fddc809f410 0x7fddc811ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:06.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:06.071+0000 7fddcd7b7700 1 -- 172.21.15.16:0/3823323914 >> 172.21.15.16:0/3823323914 conn(0x7fddc80a4210 msgr2=0x7fddc8135c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:06.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:06.071+0000 7fddcd7b7700 1 -- 172.21.15.16:0/3823323914 shutdown_connections 2023-09-30T22:32:06.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:06.071+0000 7fddcd7b7700 1 -- 172.21.15.16:0/3823323914 wait complete. 2023-09-30T22:32:06.085 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:32:06.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:06.143+0000 7f26feda3700 1 -- 172.21.15.16:0/4282490507 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f26f811a470 con 0x7f26f809de50 2023-09-30T22:32:06.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:06.144+0000 7f26f57fa700 1 -- 172.21.15.16:0/4282490507 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f26ec015070 con 0x7f26f809de50 2023-09-30T22:32:06.144 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:32:06.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:06.146+0000 7f26feda3700 1 -- 172.21.15.16:0/4282490507 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f26f809de50 msgr2=0x7f26f811d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:06.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:06.146+0000 7f26feda3700 1 --2- 172.21.15.16:0/4282490507 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f26f809de50 0x7f26f811d710 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f26ec016000 tx=0x7f26ec004750 comp rx=0 tx=0).stop 2023-09-30T22:32:06.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:06.146+0000 7f26feda3700 1 -- 172.21.15.16:0/4282490507 shutdown_connections 2023-09-30T22:32:06.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:06.146+0000 7f26feda3700 1 --2- 172.21.15.16:0/4282490507 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f26f809de50 0x7f26f811d710 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:06.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:06.146+0000 7f26feda3700 1 --2- 172.21.15.16:0/4282490507 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f26f81338a0 0x7f26f811e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:06.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:06.146+0000 7f26feda3700 1 --2- 172.21.15.16:0/4282490507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f26f809e930 0x7f26f811dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:06.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:06.146+0000 7f26feda3700 1 -- 172.21.15.16:0/4282490507 >> 172.21.15.16:0/4282490507 conn(0x7f26f80a2aa0 msgr2=0x7f26f81bac80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:06.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:06.146+0000 7f26feda3700 1 -- 172.21.15.16:0/4282490507 shutdown_connections 2023-09-30T22:32:06.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:06.146+0000 7f26feda3700 1 -- 172.21.15.16:0/4282490507 wait complete. 2023-09-30T22:32:06.160 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:32:06.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:06.179+0000 7ffb29b57700 1 -- 172.21.15.16:0/1207582321 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ffb24129b40 con 0x7ffb24133fa0 2023-09-30T22:32:06.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:06.180+0000 7ffb0bfff700 1 -- 172.21.15.16:0/1207582321 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ffb24129b40 con 0x7ffb24133fa0 2023-09-30T22:32:06.180 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:32:06.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:06.182+0000 7ffb29b57700 1 -- 172.21.15.16:0/1207582321 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffb24133fa0 msgr2=0x7ffb241c9250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:06.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:06.182+0000 7ffb29b57700 1 --2- 172.21.15.16:0/1207582321 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffb24133fa0 0x7ffb241c9250 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c00cb40 tx=0x7ffb1c00ce50 comp rx=0 tx=0).stop 2023-09-30T22:32:06.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:06.182+0000 7ffb29b57700 1 -- 172.21.15.16:0/1207582321 shutdown_connections 2023-09-30T22:32:06.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:06.182+0000 7ffb29b57700 1 --2- 172.21.15.16:0/1207582321 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffb24133fa0 0x7ffb241c9250 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:06.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:06.182+0000 7ffb29b57700 1 --2- 172.21.15.16:0/1207582321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb240a0a30 0x7ffb241ccca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:06.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:06.182+0000 7ffb29b57700 1 --2- 172.21.15.16:0/1207582321 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffb240a1340 0x7ffb241cd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:06.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:06.182+0000 7ffb29b57700 1 -- 172.21.15.16:0/1207582321 >> 172.21.15.16:0/1207582321 conn(0x7ffb240a6550 msgr2=0x7ffb24133530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:06.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:06.182+0000 7ffb29b57700 1 -- 172.21.15.16:0/1207582321 shutdown_connections 2023-09-30T22:32:06.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:06.182+0000 7ffb29b57700 1 -- 172.21.15.16:0/1207582321 wait complete. 2023-09-30T22:32:06.196 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:32:06.961 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:32:07.086 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:32:07.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.108+0000 7fb8859d7700 1 Processor -- start 2023-09-30T22:32:07.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.108+0000 7fb8859d7700 1 -- start start 2023-09-30T22:32:07.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.109+0000 7fb8859d7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb8800a0a30 0x7fb8800a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:07.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.109+0000 7fb8859d7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb88009d630 0x7fb8800a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:07.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.109+0000 7fb8859d7700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb880133930 0x7fb880133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:07.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.109+0000 7fb8859d7700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8800a2a70 con 0x7fb880133930 2023-09-30T22:32:07.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.109+0000 7fb8859d7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8800a01d0 con 0x7fb8800a0a30 2023-09-30T22:32:07.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.109+0000 7fb8859d7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb88009e2b0 con 0x7fb88009d630 2023-09-30T22:32:07.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.109+0000 7fb87effd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb8800a0a30 0x7fb8800a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:07.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.109+0000 7fb87effd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb8800a0a30 0x7fb8800a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:44472/0 (socket says 172.21.15.16:44472) 2023-09-30T22:32:07.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.109+0000 7fb87effd700 1 -- 172.21.15.16:0/2922594788 learned_addr learned my addr 172.21.15.16:0/2922594788 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:07.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.109+0000 7fb87e7fc700 1 --2- 172.21.15.16:0/2922594788 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb88009d630 0x7fb8800a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:07.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.109+0000 7fb87f7fe700 1 --2- 172.21.15.16:0/2922594788 >> v2:172.21.15.16:3300/0 conn(0x7fb880133930 0x7fb880133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:07.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.110+0000 7fb87effd700 1 -- 172.21.15.16:0/2922594788 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb88009d630 msgr2=0x7fb8800a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:07.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.110+0000 7fb87effd700 1 --2- 172.21.15.16:0/2922594788 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb88009d630 0x7fb8800a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:07.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.110+0000 7fb87effd700 1 -- 172.21.15.16:0/2922594788 >> v2:172.21.15.16:3300/0 conn(0x7fb880133930 msgr2=0x7fb880133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:07.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.110+0000 7fb87effd700 1 --2- 172.21.15.16:0/2922594788 >> v2:172.21.15.16:3300/0 conn(0x7fb880133930 0x7fb880133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:07.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.110+0000 7fb87effd700 1 -- 172.21.15.16:0/2922594788 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb88009dc50 con 0x7fb8800a0a30 2023-09-30T22:32:07.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.110+0000 7fb87e7fc700 1 --2- 172.21.15.16:0/2922594788 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb88009d630 0x7fb8800a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:07.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.110+0000 7fb87effd700 1 --2- 172.21.15.16:0/2922594788 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb8800a0a30 0x7fb8800a0e00 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb8700098f0 tx=0x7fb870009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdfd6f079b160063 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:07.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.110+0000 7fb87dffb700 1 -- 172.21.15.16:0/2922594788 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb870011830 con 0x7fb8800a0a30 2023-09-30T22:32:07.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.110+0000 7fb87dffb700 1 -- 172.21.15.16:0/2922594788 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb870011990 con 0x7fb8800a0a30 2023-09-30T22:32:07.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.110+0000 7fb87dffb700 1 -- 172.21.15.16:0/2922594788 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb870011d20 con 0x7fb8800a0a30 2023-09-30T22:32:07.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.111+0000 7fb8859d7700 1 -- 172.21.15.16:0/2922594788 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb8800a0a30 msgr2=0x7fb8800a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:07.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.111+0000 7fb8859d7700 1 --2- 172.21.15.16:0/2922594788 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb8800a0a30 0x7fb8800a0e00 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb8700098f0 tx=0x7fb870009c00 comp rx=0 tx=0).stop 2023-09-30T22:32:07.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.111+0000 7fb8859d7700 1 -- 172.21.15.16:0/2922594788 shutdown_connections 2023-09-30T22:32:07.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.111+0000 7fb8859d7700 1 --2- 172.21.15.16:0/2922594788 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb8800a0a30 0x7fb8800a0e00 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:07.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.111+0000 7fb8859d7700 1 --2- 172.21.15.16:0/2922594788 >> v2:172.21.15.16:3300/0 conn(0x7fb880133930 0x7fb880133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:07.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.111+0000 7fb8859d7700 1 --2- 172.21.15.16:0/2922594788 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb88009d630 0x7fb8800a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:07.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.111+0000 7fb8859d7700 1 -- 172.21.15.16:0/2922594788 >> 172.21.15.16:0/2922594788 conn(0x7fb8800a3050 msgr2=0x7fb8800a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:07.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.111+0000 7fb8859d7700 1 -- 172.21.15.16:0/2922594788 shutdown_connections 2023-09-30T22:32:07.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.111+0000 7fb8859d7700 1 -- 172.21.15.16:0/2922594788 wait complete. 2023-09-30T22:32:07.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.112+0000 7fb8859d7700 1 Processor -- start 2023-09-30T22:32:07.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.112+0000 7fb8859d7700 1 -- start start 2023-09-30T22:32:07.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.113+0000 7fb8859d7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb88009d630 0x7fb88011f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:07.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.113+0000 7fb8859d7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb8800a0a30 0x7fb88011f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:07.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.113+0000 7fb8859d7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb880133930 0x7fb8801c7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:07.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.113+0000 7fb8859d7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8800a4150 con 0x7fb88009d630 2023-09-30T22:32:07.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.113+0000 7fb8859d7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8800a0410 con 0x7fb880133930 2023-09-30T22:32:07.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.113+0000 7fb8859d7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb88011edd0 con 0x7fb8800a0a30 2023-09-30T22:32:07.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.113+0000 7fb87effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb88009d630 0x7fb88011f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:07.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.113+0000 7fb87effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb88009d630 0x7fb88011f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54282/0 (socket says 172.21.15.16:54282) 2023-09-30T22:32:07.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.113+0000 7fb87effd700 1 -- 172.21.15.16:0/355298729 learned_addr learned my addr 172.21.15.16:0/355298729 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:07.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.113+0000 7fb87e7fc700 1 --2- 172.21.15.16:0/355298729 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb8800a0a30 0x7fb88011f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:07.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.113+0000 7fb87f7fe700 1 --2- 172.21.15.16:0/355298729 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb880133930 0x7fb8801c7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:07.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.113+0000 7fb87effd700 1 -- 172.21.15.16:0/355298729 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb8800a0a30 msgr2=0x7fb88011f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:07.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.113+0000 7fb87effd700 1 --2- 172.21.15.16:0/355298729 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb8800a0a30 0x7fb88011f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:07.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.113+0000 7fb87effd700 1 -- 172.21.15.16:0/355298729 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb880133930 msgr2=0x7fb8801c7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:07.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.113+0000 7fb87effd700 1 --2- 172.21.15.16:0/355298729 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb880133930 0x7fb8801c7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:07.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.113+0000 7fb87effd700 1 -- 172.21.15.16:0/355298729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8740094a0 con 0x7fb88009d630 2023-09-30T22:32:07.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.113+0000 7fb87e7fc700 1 --2- 172.21.15.16:0/355298729 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb8800a0a30 0x7fb88011f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:32:07.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.113+0000 7fb87effd700 1 --2- 172.21.15.16:0/355298729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb88009d630 0x7fb88011f2c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fb870000c00 tx=0x7fb870004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:07.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.113+0000 7fb867fff700 1 -- 172.21.15.16:0/355298729 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb870004450 con 0x7fb88009d630 2023-09-30T22:32:07.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.114+0000 7fb867fff700 1 -- 172.21.15.16:0/355298729 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8700045b0 con 0x7fb88009d630 2023-09-30T22:32:07.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.114+0000 7fb8859d7700 1 -- 172.21.15.16:0/355298729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb870009550 con 0x7fb88009d630 2023-09-30T22:32:07.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.114+0000 7fb867fff700 1 -- 172.21.15.16:0/355298729 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8700038c0 con 0x7fb88009d630 2023-09-30T22:32:07.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.114+0000 7fb8859d7700 1 -- 172.21.15.16:0/355298729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb880121de0 con 0x7fb88009d630 2023-09-30T22:32:07.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.115+0000 7fb8859d7700 1 -- 172.21.15.16:0/355298729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8801222c0 con 0x7fb88009d630 2023-09-30T22:32:07.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.115+0000 7fb867fff700 1 -- 172.21.15.16:0/355298729 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb870021070 con 0x7fb88009d630 2023-09-30T22:32:07.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.115+0000 7fb867fff700 1 -- 172.21.15.16:0/355298729 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fb8700408c0 con 0x7fb88009d630 2023-09-30T22:32:07.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.121+0000 7fb867fff700 1 -- 172.21.15.16:0/355298729 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb87000b7a0 con 0x7fb88009d630 2023-09-30T22:32:07.161 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:32:07.197 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:32:07.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.229+0000 7fde1722e700 1 Processor -- start 2023-09-30T22:32:07.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.229+0000 7fde1722e700 1 -- start start 2023-09-30T22:32:07.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.229+0000 7fde1722e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fde101338a0 0x7fde10133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:07.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.229+0000 7fde1722e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fde1009de50 0x7fde1009e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:07.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.229+0000 7fde1722e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fde1009e760 0x7fde1013a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:07.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.229+0000 7fde1722e700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde1006fef0 con 0x7fde1009de50 2023-09-30T22:32:07.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.229+0000 7fde1722e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde1006ef30 con 0x7fde1009e760 2023-09-30T22:32:07.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.229+0000 7fde1722e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fde101334b0 con 0x7fde101338a0 2023-09-30T22:32:07.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.230+0000 7fde157cb700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fde1009e760 0x7fde1013a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:07.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.230+0000 7fde0ffff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fde1009de50 0x7fde1009e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:07.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.230+0000 7fde157cb700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fde1009e760 0x7fde1013a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:44494/0 (socket says 172.21.15.16:44494) 2023-09-30T22:32:07.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.230+0000 7fde0ffff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fde1009de50 0x7fde1009e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54296/0 (socket says 172.21.15.16:54296) 2023-09-30T22:32:07.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.230+0000 7fde157cb700 1 -- 172.21.15.16:0/3986128226 learned_addr learned my addr 172.21.15.16:0/3986128226 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:07.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.230+0000 7fde14fca700 1 --2- 172.21.15.16:0/3986128226 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fde101338a0 0x7fde10133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:07.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.230+0000 7fde0ffff700 1 -- 172.21.15.16:0/3986128226 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fde101338a0 msgr2=0x7fde10133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:07.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.230+0000 7fde0ffff700 1 --2- 172.21.15.16:0/3986128226 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fde101338a0 0x7fde10133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:07.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.230+0000 7fde0ffff700 1 -- 172.21.15.16:0/3986128226 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fde1009e760 msgr2=0x7fde1013a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:07.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.230+0000 7fde0ffff700 1 --2- 172.21.15.16:0/3986128226 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fde1009e760 0x7fde1013a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:07.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.230+0000 7fde0ffff700 1 -- 172.21.15.16:0/3986128226 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde1006fc30 con 0x7fde1009de50 2023-09-30T22:32:07.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.230+0000 7fde0ffff700 1 --2- 172.21.15.16:0/3986128226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde1009de50 0x7fde1009e220 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fddfc009840 tx=0x7fddfc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47575ddf544b8f3c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:07.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.231+0000 7fde0f7fe700 1 -- 172.21.15.16:0/3986128226 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddfc011830 con 0x7fde1009de50 2023-09-30T22:32:07.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.231+0000 7fde0f7fe700 1 -- 172.21.15.16:0/3986128226 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddfc011990 con 0x7fde1009de50 2023-09-30T22:32:07.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.231+0000 7fde0f7fe700 1 -- 172.21.15.16:0/3986128226 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddfc011d20 con 0x7fde1009de50 2023-09-30T22:32:07.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.231+0000 7fde1722e700 1 -- 172.21.15.16:0/3986128226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde1009de50 msgr2=0x7fde1009e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:07.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.231+0000 7fde1722e700 1 --2- 172.21.15.16:0/3986128226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde1009de50 0x7fde1009e220 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fddfc009840 tx=0x7fddfc009b50 comp rx=0 tx=0).stop 2023-09-30T22:32:07.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.232+0000 7fde1722e700 1 -- 172.21.15.16:0/3986128226 shutdown_connections 2023-09-30T22:32:07.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.232+0000 7fde1722e700 1 --2- 172.21.15.16:0/3986128226 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fde1009e760 0x7fde1013a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:07.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.232+0000 7fde1722e700 1 --2- 172.21.15.16:0/3986128226 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fde101338a0 0x7fde10133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:07.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.232+0000 7fde1722e700 1 --2- 172.21.15.16:0/3986128226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde1009de50 0x7fde1009e220 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:07.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.232+0000 7fde1722e700 1 -- 172.21.15.16:0/3986128226 >> 172.21.15.16:0/3986128226 conn(0x7fde100a2aa0 msgr2=0x7fde100a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:07.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.232+0000 7fde1722e700 1 -- 172.21.15.16:0/3986128226 shutdown_connections 2023-09-30T22:32:07.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.232+0000 7fde1722e700 1 -- 172.21.15.16:0/3986128226 wait complete. 2023-09-30T22:32:07.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.233+0000 7fde1722e700 1 Processor -- start 2023-09-30T22:32:07.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.233+0000 7fde1722e700 1 -- start start 2023-09-30T22:32:07.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.234+0000 7fde1722e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde1009de50 0x7fde101198a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:07.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.234+0000 7fde1722e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fde1009e760 0x7fde10119de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:07.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.234+0000 7fde1722e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fde101338a0 0x7fde1011a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:07.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.234+0000 7fde1722e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde1011d3c0 con 0x7fde1009de50 2023-09-30T22:32:07.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.234+0000 7fde1722e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde100a0bc0 con 0x7fde101338a0 2023-09-30T22:32:07.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.234+0000 7fde1722e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fde1006f640 con 0x7fde1009e760 2023-09-30T22:32:07.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.234+0000 7fde14fca700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde1009de50 0x7fde101198a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:07.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.234+0000 7fde14fca700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde1009de50 0x7fde101198a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54300/0 (socket says 172.21.15.16:54300) 2023-09-30T22:32:07.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.234+0000 7fde14fca700 1 -- 172.21.15.16:0/2637558690 learned_addr learned my addr 172.21.15.16:0/2637558690 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:07.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.235+0000 7fde0ffff700 1 --2- 172.21.15.16:0/2637558690 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fde1009e760 0x7fde10119de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:07.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.235+0000 7fde157cb700 1 --2- 172.21.15.16:0/2637558690 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fde101338a0 0x7fde1011a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:07.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.235+0000 7fde14fca700 1 -- 172.21.15.16:0/2637558690 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fde1009e760 msgr2=0x7fde10119de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:07.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.235+0000 7fde14fca700 1 --2- 172.21.15.16:0/2637558690 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fde1009e760 0x7fde10119de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:07.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.235+0000 7fde14fca700 1 -- 172.21.15.16:0/2637558690 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fde101338a0 msgr2=0x7fde1011a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:07.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.235+0000 7fde14fca700 1 --2- 172.21.15.16:0/2637558690 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fde101338a0 0x7fde1011a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:07.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.235+0000 7fde14fca700 1 -- 172.21.15.16:0/2637558690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddfc0094a0 con 0x7fde1009de50 2023-09-30T22:32:07.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.235+0000 7fde14fca700 1 --2- 172.21.15.16:0/2637558690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde1009de50 0x7fde101198a0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fde04014360 tx=0x7fde04014670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:07.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.235+0000 7fde0d7fa700 1 -- 172.21.15.16:0/2637558690 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde04016050 con 0x7fde1009de50 2023-09-30T22:32:07.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.235+0000 7fde0d7fa700 1 -- 172.21.15.16:0/2637558690 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde04017040 con 0x7fde1009de50 2023-09-30T22:32:07.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.235+0000 7fde1722e700 1 -- 172.21.15.16:0/2637558690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde1011ce20 con 0x7fde1009de50 2023-09-30T22:32:07.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.235+0000 7fde0d7fa700 1 -- 172.21.15.16:0/2637558690 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde040114d0 con 0x7fde1009de50 2023-09-30T22:32:07.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.235+0000 7fde1722e700 1 -- 172.21.15.16:0/2637558690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde1011db50 con 0x7fde1009de50 2023-09-30T22:32:07.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.236+0000 7fde0d7fa700 1 -- 172.21.15.16:0/2637558690 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fde0400f850 con 0x7fde1009de50 2023-09-30T22:32:07.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.237+0000 7fde0d7fa700 1 -- 172.21.15.16:0/2637558690 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fde04047610 con 0x7fde1009de50 2023-09-30T22:32:07.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.237+0000 7fde1722e700 1 -- 172.21.15.16:0/2637558690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde10133de0 con 0x7fde1009de50 2023-09-30T22:32:07.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.243+0000 7fde0d7fa700 1 -- 172.21.15.16:0/2637558690 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fde04024410 con 0x7fde1009de50 2023-09-30T22:32:07.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.330+0000 7fb8859d7700 1 -- 172.21.15.16:0/355298729 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb8801222c0 con 0x7fb88009d630 2023-09-30T22:32:07.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.330+0000 7fb867fff700 1 -- 172.21.15.16:0/355298729 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb8801222c0 con 0x7fb88009d630 2023-09-30T22:32:07.331 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:32:07.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.333+0000 7fb8859d7700 1 -- 172.21.15.16:0/355298729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb88009d630 msgr2=0x7fb88011f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:07.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.333+0000 7fb8859d7700 1 --2- 172.21.15.16:0/355298729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb88009d630 0x7fb88011f2c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fb870000c00 tx=0x7fb870004750 comp rx=0 tx=0).stop 2023-09-30T22:32:07.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.333+0000 7fb8859d7700 1 -- 172.21.15.16:0/355298729 shutdown_connections 2023-09-30T22:32:07.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.333+0000 7fb8859d7700 1 --2- 172.21.15.16:0/355298729 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb880133930 0x7fb8801c7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:07.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.333+0000 7fb8859d7700 1 --2- 172.21.15.16:0/355298729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb88009d630 0x7fb88011f2c0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:07.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.333+0000 7fb8859d7700 1 --2- 172.21.15.16:0/355298729 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb8800a0a30 0x7fb88011f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:07.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.333+0000 7fb8859d7700 1 -- 172.21.15.16:0/355298729 >> 172.21.15.16:0/355298729 conn(0x7fb8800a3050 msgr2=0x7fb8801bbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:07.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.333+0000 7fb8859d7700 1 -- 172.21.15.16:0/355298729 shutdown_connections 2023-09-30T22:32:07.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.333+0000 7fb8859d7700 1 -- 172.21.15.16:0/355298729 wait complete. 2023-09-30T22:32:07.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.338+0000 7f6e11e4e700 1 Processor -- start 2023-09-30T22:32:07.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.338+0000 7f6e11e4e700 1 -- start start 2023-09-30T22:32:07.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.339+0000 7f6e11e4e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e0c131810 0x7f6e0c131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:07.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.339+0000 7f6e11e4e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6e0c099a40 0x7f6e0c099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:07.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.339+0000 7f6e11e4e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6e0c09a3b0 0x7f6e0c138650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:07.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.339+0000 7f6e11e4e700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e0c1313c0 con 0x7f6e0c099a40 2023-09-30T22:32:07.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.339+0000 7f6e11e4e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0c06e8c0 con 0x7f6e0c09a3b0 2023-09-30T22:32:07.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.339+0000 7f6e11e4e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0c09a820 con 0x7f6e0c131810 2023-09-30T22:32:07.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.339+0000 7f6e0b7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e0c131810 0x7f6e0c131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:07.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.339+0000 7f6e0b7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e0c131810 0x7f6e0c131be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:34648/0 (socket says 172.21.15.16:34648) 2023-09-30T22:32:07.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.339+0000 7f6e0b7fe700 1 -- 172.21.15.16:0/170932160 learned_addr learned my addr 172.21.15.16:0/170932160 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:07.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.339+0000 7f6e0affd700 1 --2- 172.21.15.16:0/170932160 >> v2:172.21.15.16:3300/0 conn(0x7f6e0c099a40 0x7f6e0c099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:07.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.339+0000 7f6e0b7fe700 1 -- 172.21.15.16:0/170932160 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6e0c09a3b0 msgr2=0x7f6e0c138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:07.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.339+0000 7f6e0b7fe700 1 --2- 172.21.15.16:0/170932160 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6e0c09a3b0 0x7f6e0c138650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:07.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.339+0000 7f6e0b7fe700 1 -- 172.21.15.16:0/170932160 >> v2:172.21.15.16:3300/0 conn(0x7f6e0c099a40 msgr2=0x7f6e0c099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:07.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.339+0000 7f6e0b7fe700 1 --2- 172.21.15.16:0/170932160 >> v2:172.21.15.16:3300/0 conn(0x7f6e0c099a40 0x7f6e0c099e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:07.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.339+0000 7f6e0b7fe700 1 -- 172.21.15.16:0/170932160 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e0c071d20 con 0x7f6e0c131810 2023-09-30T22:32:07.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.340+0000 7f6e0b7fe700 1 --2- 172.21.15.16:0/170932160 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e0c131810 0x7f6e0c131be0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc0098f0 tx=0x7f6dfc009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee9ffadd57a1a36f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:07.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.340+0000 7f6e0a7fc700 1 -- 172.21.15.16:0/170932160 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc011830 con 0x7f6e0c131810 2023-09-30T22:32:07.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.340+0000 7f6e0a7fc700 1 -- 172.21.15.16:0/170932160 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dfc011990 con 0x7f6e0c131810 2023-09-30T22:32:07.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.340+0000 7f6e0a7fc700 1 -- 172.21.15.16:0/170932160 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc011d20 con 0x7f6e0c131810 2023-09-30T22:32:07.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.341+0000 7f6e11e4e700 1 -- 172.21.15.16:0/170932160 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e0c131810 msgr2=0x7f6e0c131be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:07.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.341+0000 7f6e11e4e700 1 --2- 172.21.15.16:0/170932160 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e0c131810 0x7f6e0c131be0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc0098f0 tx=0x7f6dfc009c00 comp rx=0 tx=0).stop 2023-09-30T22:32:07.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.341+0000 7f6e11e4e700 1 -- 172.21.15.16:0/170932160 shutdown_connections 2023-09-30T22:32:07.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.341+0000 7f6e11e4e700 1 --2- 172.21.15.16:0/170932160 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6e0c09a3b0 0x7f6e0c138650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:07.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.341+0000 7f6e11e4e700 1 --2- 172.21.15.16:0/170932160 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e0c131810 0x7f6e0c131be0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:07.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.341+0000 7f6e11e4e700 1 --2- 172.21.15.16:0/170932160 >> v2:172.21.15.16:3300/0 conn(0x7f6e0c099a40 0x7f6e0c099e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:07.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.341+0000 7f6e11e4e700 1 -- 172.21.15.16:0/170932160 >> 172.21.15.16:0/170932160 conn(0x7f6e0c0a2aa0 msgr2=0x7f6e0c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:07.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.341+0000 7f6e11e4e700 1 -- 172.21.15.16:0/170932160 shutdown_connections 2023-09-30T22:32:07.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.341+0000 7f6e11e4e700 1 -- 172.21.15.16:0/170932160 wait complete. 2023-09-30T22:32:07.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.343+0000 7f6e11e4e700 1 Processor -- start 2023-09-30T22:32:07.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.343+0000 7f6e11e4e700 1 -- start start 2023-09-30T22:32:07.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.344+0000 7f6e11e4e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e0c099a40 0x7f6e0c11b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:07.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.344+0000 7f6e11e4e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6e0c09a3b0 0x7f6e0c11bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:07.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.344+0000 7f6e11e4e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e0c131810 0x7f6e0c11c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:07.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.344+0000 7f6e11e4e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0c11d2d0 con 0x7f6e0c099a40 2023-09-30T22:32:07.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.344+0000 7f6e11e4e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0c11e520 con 0x7f6e0c09a3b0 2023-09-30T22:32:07.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.344+0000 7f6e11e4e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0c11d8b0 con 0x7f6e0c131810 2023-09-30T22:32:07.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.344+0000 7f6e0b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e0c099a40 0x7f6e0c11b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:07.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.344+0000 7f6e0b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e0c099a40 0x7f6e0c11b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54324/0 (socket says 172.21.15.16:54324) 2023-09-30T22:32:07.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.344+0000 7f6e0b7fe700 1 -- 172.21.15.16:0/637074737 learned_addr learned my addr 172.21.15.16:0/637074737 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:07.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.344+0000 7f6e0bfff700 1 --2- 172.21.15.16:0/637074737 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e0c131810 0x7f6e0c11c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:07.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.344+0000 7f6e0affd700 1 --2- 172.21.15.16:0/637074737 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6e0c09a3b0 0x7f6e0c11bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:07.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.344+0000 7f6e0b7fe700 1 -- 172.21.15.16:0/637074737 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e0c131810 msgr2=0x7f6e0c11c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:07.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.344+0000 7f6e0b7fe700 1 --2- 172.21.15.16:0/637074737 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e0c131810 0x7f6e0c11c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:07.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.344+0000 7f6e0b7fe700 1 -- 172.21.15.16:0/637074737 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6e0c09a3b0 msgr2=0x7f6e0c11bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:07.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.344+0000 7f6e0b7fe700 1 --2- 172.21.15.16:0/637074737 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6e0c09a3b0 0x7f6e0c11bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:07.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.344+0000 7f6e0b7fe700 1 -- 172.21.15.16:0/637074737 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dfc009550 con 0x7f6e0c099a40 2023-09-30T22:32:07.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.345+0000 7f6e0b7fe700 1 --2- 172.21.15.16:0/637074737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e0c099a40 0x7f6e0c11b6c0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c09a9b0 tx=0x7f6dfc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:07.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.345+0000 7f6df3fff700 1 -- 172.21.15.16:0/637074737 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc004450 con 0x7f6e0c099a40 2023-09-30T22:32:07.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.345+0000 7f6df3fff700 1 -- 172.21.15.16:0/637074737 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dfc0045d0 con 0x7f6e0c099a40 2023-09-30T22:32:07.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.345+0000 7f6e11e4e700 1 -- 172.21.15.16:0/637074737 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e0c11ef00 con 0x7f6e0c099a40 2023-09-30T22:32:07.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.345+0000 7f6df3fff700 1 -- 172.21.15.16:0/637074737 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc0038e0 con 0x7f6e0c099a40 2023-09-30T22:32:07.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.345+0000 7f6e11e4e700 1 -- 172.21.15.16:0/637074737 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e0c06eb00 con 0x7f6e0c099a40 2023-09-30T22:32:07.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.346+0000 7f6df3fff700 1 -- 172.21.15.16:0/637074737 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6dfc003a40 con 0x7f6e0c099a40 2023-09-30T22:32:07.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.346+0000 7f6e11e4e700 1 -- 172.21.15.16:0/637074737 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e0c074ea0 con 0x7f6e0c099a40 2023-09-30T22:32:07.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.347+0000 7f6df3fff700 1 -- 172.21.15.16:0/637074737 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f6dfc0406b0 con 0x7f6e0c099a40 2023-09-30T22:32:07.352 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:32:07.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.353+0000 7f6df3fff700 1 -- 172.21.15.16:0/637074737 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6dfc01a420 con 0x7f6e0c099a40 2023-09-30T22:32:07.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.361+0000 7f139eb6a700 1 Processor -- start 2023-09-30T22:32:07.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.361+0000 7f139eb6a700 1 -- start start 2023-09-30T22:32:07.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.362+0000 7f139eb6a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f13980a1510 0x7f1398138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:07.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.362+0000 7f139eb6a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1398139250 0x7f139813b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:07.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.362+0000 7f139eb6a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f139809edd0 con 0x7f13980a0a30 2023-09-30T22:32:07.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.362+0000 7f139eb6a700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13980a18e0 con 0x7f13980a1510 2023-09-30T22:32:07.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.362+0000 7f139eb6a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f139809e1f0 con 0x7f1398139250 2023-09-30T22:32:07.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.362+0000 7f1397fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f13980a1510 0x7f1398138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:07.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.362+0000 7f139c906700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f13980a0a30 0x7f13980a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60938/0 (socket says 172.21.15.16:60938) 2023-09-30T22:32:07.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.362+0000 7f1397fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f13980a1510 0x7f1398138d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54326/0 (socket says 172.21.15.16:54326) 2023-09-30T22:32:07.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.362+0000 7f139c906700 1 -- 172.21.15.16:0/1689502157 learned_addr learned my addr 172.21.15.16:0/1689502157 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:07.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.362+0000 7f139d107700 1 --2- 172.21.15.16:0/1689502157 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1398139250 0x7f139813b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:07.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.362+0000 7f13977fe700 1 -- 172.21.15.16:0/1689502157 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 413301873 0 0) 0x7f139809edd0 con 0x7f13980a0a30 2023-09-30T22:32:07.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.363+0000 7f13977fe700 1 -- 172.21.15.16:0/1689502157 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13800035b0 con 0x7f13980a0a30 2023-09-30T22:32:07.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.363+0000 7f1397fff700 1 -- 172.21.15.16:0/1689502157 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1398139250 msgr2=0x7f139813b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:07.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.363+0000 7f1397fff700 1 --2- 172.21.15.16:0/1689502157 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1398139250 0x7f139813b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:07.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.363+0000 7f1397fff700 1 -- 172.21.15.16:0/1689502157 >> v1:172.21.15.16:6789/0 conn(0x7f13980a0a30 legacy=0x7f13980a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:07.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.363+0000 7f1397fff700 1 -- 172.21.15.16:0/1689502157 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f139809dc50 con 0x7f13980a1510 2023-09-30T22:32:07.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.363+0000 7f139d107700 1 --2- 172.21.15.16:0/1689502157 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1398139250 0x7f139813b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:07.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.363+0000 7f1397fff700 1 --2- 172.21.15.16:0/1689502157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13980a1510 0x7f1398138d10 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f138c009840 tx=0x7f138c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eaa50ee04a6967f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:07.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.363+0000 7f13977fe700 1 -- 172.21.15.16:0/1689502157 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138c011830 con 0x7f13980a1510 2023-09-30T22:32:07.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.363+0000 7f13977fe700 1 -- 172.21.15.16:0/1689502157 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f138c011990 con 0x7f13980a1510 2023-09-30T22:32:07.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.363+0000 7f13977fe700 1 -- 172.21.15.16:0/1689502157 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138c011d20 con 0x7f13980a1510 2023-09-30T22:32:07.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.364+0000 7f139eb6a700 1 -- 172.21.15.16:0/1689502157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13980a1510 msgr2=0x7f1398138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:07.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.364+0000 7f139eb6a700 1 --2- 172.21.15.16:0/1689502157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13980a1510 0x7f1398138d10 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f138c009840 tx=0x7f138c009b50 comp rx=0 tx=0).stop 2023-09-30T22:32:07.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.364+0000 7f139eb6a700 1 -- 172.21.15.16:0/1689502157 shutdown_connections 2023-09-30T22:32:07.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.364+0000 7f139eb6a700 1 --2- 172.21.15.16:0/1689502157 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1398139250 0x7f139813b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:07.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.364+0000 7f139eb6a700 1 --2- 172.21.15.16:0/1689502157 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13980a1510 0x7f1398138d10 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:07.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.364+0000 7f139eb6a700 1 -- 172.21.15.16:0/1689502157 >> 172.21.15.16:0/1689502157 conn(0x7f13980a6550 msgr2=0x7f13980a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:07.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.364+0000 7f139eb6a700 1 -- 172.21.15.16:0/1689502157 shutdown_connections 2023-09-30T22:32:07.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.364+0000 7f139eb6a700 1 -- 172.21.15.16:0/1689502157 wait complete. 2023-09-30T22:32:07.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.365+0000 7f139eb6a700 1 Processor -- start 2023-09-30T22:32:07.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.365+0000 7f139eb6a700 1 -- start start 2023-09-30T22:32:07.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.366+0000 7f139eb6a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f13980a0a30 0x7f1398122750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:07.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.366+0000 7f139eb6a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f13980a1510 0x7f1398122c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:07.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.366+0000 7f139eb6a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1398139250 0x7f13981231d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:07.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.366+0000 7f139eb6a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1398138980 con 0x7f1398139250 2023-09-30T22:32:07.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.366+0000 7f139eb6a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1398138080 con 0x7f13980a1510 2023-09-30T22:32:07.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.366+0000 7f139eb6a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1398124360 con 0x7f13980a0a30 2023-09-30T22:32:07.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.366+0000 7f139c906700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f13980a0a30 0x7f1398122750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:07.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.366+0000 7f139c906700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f13980a0a30 0x7f1398122750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:34682/0 (socket says 172.21.15.16:34682) 2023-09-30T22:32:07.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.366+0000 7f139c906700 1 -- 172.21.15.16:0/810739657 learned_addr learned my addr 172.21.15.16:0/810739657 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:07.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.366+0000 7f139d107700 1 --2- 172.21.15.16:0/810739657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1398139250 0x7f13981231d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:07.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.367+0000 7f139c906700 1 -- 172.21.15.16:0/810739657 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f13980a1510 msgr2=0x7f1398122c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:07.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.367+0000 7f139c906700 1 --2- 172.21.15.16:0/810739657 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f13980a1510 0x7f1398122c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:07.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.367+0000 7f139c906700 1 -- 172.21.15.16:0/810739657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1398139250 msgr2=0x7f13981231d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:07.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.367+0000 7f139c906700 1 --2- 172.21.15.16:0/810739657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1398139250 0x7f13981231d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:07.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.367+0000 7f139c906700 1 -- 172.21.15.16:0/810739657 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1390009a70 con 0x7f13980a0a30 2023-09-30T22:32:07.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.367+0000 7f139c906700 1 --2- 172.21.15.16:0/810739657 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f13980a0a30 0x7f1398122750 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f138800c700 tx=0x7f138800cac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:07.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.367+0000 7f13957fa700 1 -- 172.21.15.16:0/810739657 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1388005d70 con 0x7f13980a0a30 2023-09-30T22:32:07.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.367+0000 7f13957fa700 1 -- 172.21.15.16:0/810739657 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13880070d0 con 0x7f13980a0a30 2023-09-30T22:32:07.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.367+0000 7f13957fa700 1 -- 172.21.15.16:0/810739657 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1388007460 con 0x7f13980a0a30 2023-09-30T22:32:07.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.367+0000 7f139eb6a700 1 -- 172.21.15.16:0/810739657 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f138c0094a0 con 0x7f13980a0a30 2023-09-30T22:32:07.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.367+0000 7f139eb6a700 1 -- 172.21.15.16:0/810739657 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1398138640 con 0x7f13980a0a30 2023-09-30T22:32:07.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.368+0000 7f13957fa700 1 -- 172.21.15.16:0/810739657 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f13880075c0 con 0x7f13980a0a30 2023-09-30T22:32:07.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.368+0000 7f13957fa700 1 -- 172.21.15.16:0/810739657 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f138803ec00 con 0x7f13980a0a30 2023-09-30T22:32:07.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.369+0000 7f139eb6a700 1 -- 172.21.15.16:0/810739657 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1398125c10 con 0x7f13980a0a30 2023-09-30T22:32:07.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.374+0000 7f13957fa700 1 -- 172.21.15.16:0/810739657 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1398125c10 con 0x7f13980a0a30 2023-09-30T22:32:07.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.480+0000 7fde1722e700 1 -- 172.21.15.16:0/2637558690 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fde10074ea0 con 0x7fde1009de50 2023-09-30T22:32:07.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.481+0000 7fde0d7fa700 1 -- 172.21.15.16:0/2637558690 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fde04022b00 con 0x7fde1009de50 2023-09-30T22:32:07.481 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:32:07.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.483+0000 7fde1722e700 1 -- 172.21.15.16:0/2637558690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde1009de50 msgr2=0x7fde101198a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:07.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.483+0000 7fde1722e700 1 --2- 172.21.15.16:0/2637558690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde1009de50 0x7fde101198a0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fde04014360 tx=0x7fde04014670 comp rx=0 tx=0).stop 2023-09-30T22:32:07.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.483+0000 7fde1722e700 1 -- 172.21.15.16:0/2637558690 shutdown_connections 2023-09-30T22:32:07.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.483+0000 7fde1722e700 1 --2- 172.21.15.16:0/2637558690 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fde101338a0 0x7fde1011a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:07.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.483+0000 7fde1722e700 1 --2- 172.21.15.16:0/2637558690 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde1009de50 0x7fde101198a0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:07.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.483+0000 7fde1722e700 1 --2- 172.21.15.16:0/2637558690 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fde1009e760 0x7fde10119de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:07.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.483+0000 7fde1722e700 1 -- 172.21.15.16:0/2637558690 >> 172.21.15.16:0/2637558690 conn(0x7fde100a2aa0 msgr2=0x7fde1012f040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:07.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.483+0000 7fde1722e700 1 -- 172.21.15.16:0/2637558690 shutdown_connections 2023-09-30T22:32:07.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.483+0000 7fde1722e700 1 -- 172.21.15.16:0/2637558690 wait complete. 2023-09-30T22:32:07.497 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:32:07.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.563+0000 7f6e11e4e700 1 -- 172.21.15.16:0/637074737 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6e0c06b9e0 con 0x7f6e0c099a40 2023-09-30T22:32:07.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.564+0000 7f6df3fff700 1 -- 172.21.15.16:0/637074737 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6dfc028070 con 0x7f6e0c099a40 2023-09-30T22:32:07.564 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:32:07.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.566+0000 7f6e11e4e700 1 -- 172.21.15.16:0/637074737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e0c099a40 msgr2=0x7f6e0c11b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:07.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.566+0000 7f6e11e4e700 1 --2- 172.21.15.16:0/637074737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e0c099a40 0x7f6e0c11b6c0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f6e0c09a9b0 tx=0x7f6dfc004750 comp rx=0 tx=0).stop 2023-09-30T22:32:07.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.566+0000 7f6e11e4e700 1 -- 172.21.15.16:0/637074737 shutdown_connections 2023-09-30T22:32:07.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.566+0000 7f6e11e4e700 1 --2- 172.21.15.16:0/637074737 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e0c131810 0x7f6e0c11c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:07.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.566+0000 7f6e11e4e700 1 --2- 172.21.15.16:0/637074737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e0c099a40 0x7f6e0c11b6c0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:07.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.566+0000 7f6e11e4e700 1 --2- 172.21.15.16:0/637074737 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6e0c09a3b0 0x7f6e0c11bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:07.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.566+0000 7f6e11e4e700 1 -- 172.21.15.16:0/637074737 >> 172.21.15.16:0/637074737 conn(0x7f6e0c0a2aa0 msgr2=0x7f6e0c1b8b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:07.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.566+0000 7f6e11e4e700 1 -- 172.21.15.16:0/637074737 shutdown_connections 2023-09-30T22:32:07.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.566+0000 7f6e11e4e700 1 -- 172.21.15.16:0/637074737 wait complete. 2023-09-30T22:32:07.580 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:32:07.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.581+0000 7f139eb6a700 1 -- 172.21.15.16:0/810739657 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1398125c10 con 0x7f13980a0a30 2023-09-30T22:32:07.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.582+0000 7f13957fa700 1 -- 172.21.15.16:0/810739657 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1398125c10 con 0x7f13980a0a30 2023-09-30T22:32:07.583 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:32:07.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.584+0000 7f139eb6a700 1 -- 172.21.15.16:0/810739657 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f13980a0a30 msgr2=0x7f1398122750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:07.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.584+0000 7f139eb6a700 1 --2- 172.21.15.16:0/810739657 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f13980a0a30 0x7f1398122750 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f138800c700 tx=0x7f138800cac0 comp rx=0 tx=0).stop 2023-09-30T22:32:07.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.584+0000 7f139eb6a700 1 -- 172.21.15.16:0/810739657 shutdown_connections 2023-09-30T22:32:07.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.584+0000 7f139eb6a700 1 --2- 172.21.15.16:0/810739657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1398139250 0x7f13981231d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:07.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.584+0000 7f139eb6a700 1 --2- 172.21.15.16:0/810739657 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f13980a0a30 0x7f1398122750 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:07.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.584+0000 7f139eb6a700 1 --2- 172.21.15.16:0/810739657 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f13980a1510 0x7f1398122c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:07.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.584+0000 7f139eb6a700 1 -- 172.21.15.16:0/810739657 >> 172.21.15.16:0/810739657 conn(0x7f13980a6550 msgr2=0x7f13980a48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:07.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.584+0000 7f139eb6a700 1 -- 172.21.15.16:0/810739657 shutdown_connections 2023-09-30T22:32:07.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:07.584+0000 7f139eb6a700 1 -- 172.21.15.16:0/810739657 wait complete. 2023-09-30T22:32:07.598 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:32:08.353 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:32:08.498 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:32:08.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.499+0000 7f76b0a91700 1 Processor -- start 2023-09-30T22:32:08.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.499+0000 7f76b0a91700 1 -- start start 2023-09-30T22:32:08.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.500+0000 7f76b0a91700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f76ac06b250 0x7f76ac136840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:08.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.500+0000 7f76b0a91700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f76ac136d80 0x7f76ac139160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:08.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.500+0000 7f76b0a91700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76ac09ece0 con 0x7f76ac06a770 2023-09-30T22:32:08.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.500+0000 7f76b0a91700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76ac06b640 con 0x7f76ac136d80 2023-09-30T22:32:08.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.500+0000 7f76b0a91700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f76ac09e760 con 0x7f76ac06b250 2023-09-30T22:32:08.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.500+0000 7f76aad9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f76ac136d80 0x7f76ac139160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:08.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.500+0000 7f76aa59c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f76ac06a770 0x7f76ac06ab40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48184/0 (socket says 172.21.15.16:48184) 2023-09-30T22:32:08.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.500+0000 7f76aa59c700 1 -- 172.21.15.16:0/324079375 learned_addr learned my addr 172.21.15.16:0/324079375 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:08.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.500+0000 7f76a9d9b700 1 --2- 172.21.15.16:0/324079375 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f76ac06b250 0x7f76ac136840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:08.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.500+0000 7f76aad9d700 1 -- 172.21.15.16:0/324079375 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f76ac06b250 msgr2=0x7f76ac136840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:08.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.500+0000 7f76aad9d700 1 --2- 172.21.15.16:0/324079375 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f76ac06b250 0x7f76ac136840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:08.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.500+0000 7f76aad9d700 1 -- 172.21.15.16:0/324079375 >> v1:172.21.15.16:6789/0 conn(0x7f76ac06a770 legacy=0x7f76ac06ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:08.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.500+0000 7f76aad9d700 1 -- 172.21.15.16:0/324079375 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76ac09e040 con 0x7f76ac136d80 2023-09-30T22:32:08.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.500+0000 7f76a959a700 1 -- 172.21.15.16:0/324079375 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3101453115 0 0) 0x7f76ac09ece0 con 0x7f76ac06a770 2023-09-30T22:32:08.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.500+0000 7f76aad9d700 1 --2- 172.21.15.16:0/324079375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76ac136d80 0x7f76ac139160 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f76a4009d90 tx=0x7f76a400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbb982503151aa85 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:08.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.500+0000 7f76a959a700 1 -- 172.21.15.16:0/324079375 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a400e070 con 0x7f76ac136d80 2023-09-30T22:32:08.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.501+0000 7f76a959a700 1 -- 172.21.15.16:0/324079375 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76a400bb20 con 0x7f76ac136d80 2023-09-30T22:32:08.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.501+0000 7f76a959a700 1 -- 172.21.15.16:0/324079375 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a4012430 con 0x7f76ac136d80 2023-09-30T22:32:08.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.501+0000 7f76b0a91700 1 -- 172.21.15.16:0/324079375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76ac136d80 msgr2=0x7f76ac139160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:08.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.501+0000 7f76b0a91700 1 --2- 172.21.15.16:0/324079375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76ac136d80 0x7f76ac139160 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f76a4009d90 tx=0x7f76a400b330 comp rx=0 tx=0).stop 2023-09-30T22:32:08.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.501+0000 7f76b0a91700 1 -- 172.21.15.16:0/324079375 shutdown_connections 2023-09-30T22:32:08.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.501+0000 7f76b0a91700 1 --2- 172.21.15.16:0/324079375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76ac136d80 0x7f76ac139160 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:08.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.501+0000 7f76b0a91700 1 --2- 172.21.15.16:0/324079375 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f76ac06b250 0x7f76ac136840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:08.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.501+0000 7f76b0a91700 1 -- 172.21.15.16:0/324079375 >> 172.21.15.16:0/324079375 conn(0x7f76ac066f50 msgr2=0x7f76ac09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:08.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.501+0000 7f76b0a91700 1 -- 172.21.15.16:0/324079375 shutdown_connections 2023-09-30T22:32:08.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.501+0000 7f76b0a91700 1 -- 172.21.15.16:0/324079375 wait complete. 2023-09-30T22:32:08.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.502+0000 7f76b0a91700 1 Processor -- start 2023-09-30T22:32:08.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.503+0000 7f76b0a91700 1 -- start start 2023-09-30T22:32:08.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.503+0000 7f76b0a91700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f76ac06a770 0x7f76ac1c62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:08.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.503+0000 7f76b0a91700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f76ac06b250 0x7f76ac1c6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:08.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.503+0000 7f76b0a91700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76ac136d80 0x7f76ac11c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:08.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.503+0000 7f76b0a91700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76ac11fcc0 con 0x7f76ac136d80 2023-09-30T22:32:08.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.503+0000 7f76b0a91700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76ac11fea0 con 0x7f76ac06a770 2023-09-30T22:32:08.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.503+0000 7f76b0a91700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f76ac1bbe60 con 0x7f76ac06b250 2023-09-30T22:32:08.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.503+0000 7f76a9d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f76ac06b250 0x7f76ac1c6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:08.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.503+0000 7f76aad9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76ac136d80 0x7f76ac11c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:08.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.503+0000 7f76a9d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f76ac06b250 0x7f76ac1c6800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:55440/0 (socket says 172.21.15.16:55440) 2023-09-30T22:32:08.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.503+0000 7f76a9d9b700 1 -- 172.21.15.16:0/16292487 learned_addr learned my addr 172.21.15.16:0/16292487 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:08.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.503+0000 7f76aa59c700 1 --2- 172.21.15.16:0/16292487 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f76ac06a770 0x7f76ac1c62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:08.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.503+0000 7f76aad9d700 1 -- 172.21.15.16:0/16292487 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f76ac06b250 msgr2=0x7f76ac1c6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:08.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.503+0000 7f76aad9d700 1 --2- 172.21.15.16:0/16292487 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f76ac06b250 0x7f76ac1c6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:08.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.504+0000 7f76aad9d700 1 -- 172.21.15.16:0/16292487 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f76ac06a770 msgr2=0x7f76ac1c62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:08.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.504+0000 7f76aad9d700 1 --2- 172.21.15.16:0/16292487 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f76ac06a770 0x7f76ac1c62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:08.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.504+0000 7f76aad9d700 1 -- 172.21.15.16:0/16292487 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a4009a70 con 0x7f76ac136d80 2023-09-30T22:32:08.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.504+0000 7f76a9d9b700 1 --2- 172.21.15.16:0/16292487 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f76ac06b250 0x7f76ac1c6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:08.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.504+0000 7f76aad9d700 1 --2- 172.21.15.16:0/16292487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76ac136d80 0x7f76ac11c380 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f76a4016000 tx=0x7f76a4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:08.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.504+0000 7f76a2ffd700 1 -- 172.21.15.16:0/16292487 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a4012430 con 0x7f76ac136d80 2023-09-30T22:32:08.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.504+0000 7f76a2ffd700 1 -- 172.21.15.16:0/16292487 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76a4020070 con 0x7f76ac136d80 2023-09-30T22:32:08.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.504+0000 7f76a2ffd700 1 -- 172.21.15.16:0/16292487 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a40098f0 con 0x7f76ac136d80 2023-09-30T22:32:08.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.504+0000 7f76b0a91700 1 -- 172.21.15.16:0/16292487 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76ac071e60 con 0x7f76ac136d80 2023-09-30T22:32:08.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.504+0000 7f76b0a91700 1 -- 172.21.15.16:0/16292487 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76ac132250 con 0x7f76ac136d80 2023-09-30T22:32:08.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.505+0000 7f76b0a91700 1 -- 172.21.15.16:0/16292487 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76ac11f860 con 0x7f76ac136d80 2023-09-30T22:32:08.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.506+0000 7f76a2ffd700 1 -- 172.21.15.16:0/16292487 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f76a4004040 con 0x7f76ac136d80 2023-09-30T22:32:08.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.506+0000 7f76a2ffd700 1 -- 172.21.15.16:0/16292487 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f76a4022070 con 0x7f76ac136d80 2023-09-30T22:32:08.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.510+0000 7f76a2ffd700 1 -- 172.21.15.16:0/16292487 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f76a4011980 con 0x7f76ac136d80 2023-09-30T22:32:08.583 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:32:08.599 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:32:08.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.645+0000 7f7a59cc4700 1 Processor -- start 2023-09-30T22:32:08.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.646+0000 7f7a59cc4700 1 -- start start 2023-09-30T22:32:08.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.646+0000 7f7a59cc4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7a5409d6f0 0x7f7a5409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:08.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.646+0000 7f7a59cc4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7a5409e440 0x7f7a54134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:08.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.646+0000 7f7a59cc4700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a5409b120 con 0x7f7a5409e000 2023-09-30T22:32:08.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.646+0000 7f7a59cc4700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a5406b460 con 0x7f7a5409d6f0 2023-09-30T22:32:08.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.646+0000 7f7a59cc4700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5409a7e0 con 0x7f7a5409e440 2023-09-30T22:32:08.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.646+0000 7f7a52ffd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7a5409e000 0x7f7a541339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48198/0 (socket says 172.21.15.16:48198) 2023-09-30T22:32:08.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.646+0000 7f7a52ffd700 1 -- 172.21.15.16:0/3252765702 learned_addr learned my addr 172.21.15.16:0/3252765702 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:08.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.646+0000 7f7a53fff700 1 --2- 172.21.15.16:0/3252765702 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7a5409e440 0x7f7a54134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:08.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.646+0000 7f7a537fe700 1 --2- 172.21.15.16:0/3252765702 >> v2:172.21.15.16:3300/0 conn(0x7f7a5409d6f0 0x7f7a5409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:08.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.647+0000 7f7a53fff700 1 -- 172.21.15.16:0/3252765702 >> v2:172.21.15.16:3300/0 conn(0x7f7a5409d6f0 msgr2=0x7f7a5409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:08.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.647+0000 7f7a53fff700 1 --2- 172.21.15.16:0/3252765702 >> v2:172.21.15.16:3300/0 conn(0x7f7a5409d6f0 0x7f7a5409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:08.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.647+0000 7f7a53fff700 1 -- 172.21.15.16:0/3252765702 >> v1:172.21.15.16:6789/0 conn(0x7f7a5409e000 legacy=0x7f7a541339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:08.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.647+0000 7f7a53fff700 1 -- 172.21.15.16:0/3252765702 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a5409a500 con 0x7f7a5409e440 2023-09-30T22:32:08.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.647+0000 7f7a53fff700 1 --2- 172.21.15.16:0/3252765702 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7a5409e440 0x7f7a54134100 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c009dc0 tx=0x7f7a4c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71bab6766b591ea8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:08.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.647+0000 7f7a527fc700 1 -- 172.21.15.16:0/3252765702 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c00e070 con 0x7f7a5409e440 2023-09-30T22:32:08.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.648+0000 7f7a527fc700 1 -- 172.21.15.16:0/3252765702 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a4c00bae0 con 0x7f7a5409e440 2023-09-30T22:32:08.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.648+0000 7f7a527fc700 1 -- 172.21.15.16:0/3252765702 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4c00be90 con 0x7f7a5409e440 2023-09-30T22:32:08.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.649+0000 7f7a59cc4700 1 -- 172.21.15.16:0/3252765702 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7a5409e440 msgr2=0x7f7a54134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:08.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.649+0000 7f7a59cc4700 1 --2- 172.21.15.16:0/3252765702 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7a5409e440 0x7f7a54134100 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c009dc0 tx=0x7f7a4c00b330 comp rx=0 tx=0).stop 2023-09-30T22:32:08.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.649+0000 7f7a59cc4700 1 -- 172.21.15.16:0/3252765702 shutdown_connections 2023-09-30T22:32:08.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.649+0000 7f7a59cc4700 1 --2- 172.21.15.16:0/3252765702 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7a5409e440 0x7f7a54134100 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:08.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.649+0000 7f7a59cc4700 1 --2- 172.21.15.16:0/3252765702 >> v2:172.21.15.16:3300/0 conn(0x7f7a5409d6f0 0x7f7a5409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:08.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.649+0000 7f7a59cc4700 1 -- 172.21.15.16:0/3252765702 >> 172.21.15.16:0/3252765702 conn(0x7f7a54066f50 msgr2=0x7f7a5409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:08.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.649+0000 7f7a59cc4700 1 -- 172.21.15.16:0/3252765702 shutdown_connections 2023-09-30T22:32:08.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.649+0000 7f7a59cc4700 1 -- 172.21.15.16:0/3252765702 wait complete. 2023-09-30T22:32:08.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.650+0000 7f7a59cc4700 1 Processor -- start 2023-09-30T22:32:08.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.651+0000 7f7a59cc4700 1 -- start start 2023-09-30T22:32:08.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.651+0000 7f7a59cc4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a5409d6f0 0x7f7a541a5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:08.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.651+0000 7f7a59cc4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7a5409e000 0x7f7a541a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:08.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.651+0000 7f7a59cc4700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7a5409e440 0x7f7a541a68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:08.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.651+0000 7f7a59cc4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a541a9590 con 0x7f7a5409d6f0 2023-09-30T22:32:08.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.651+0000 7f7a59cc4700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a540a1f20 con 0x7f7a5409e440 2023-09-30T22:32:08.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.651+0000 7f7a59cc4700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7a541a7a70 con 0x7f7a5409e000 2023-09-30T22:32:08.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.651+0000 7f7a537fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a5409d6f0 0x7f7a541a5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:08.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.651+0000 7f7a537fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a5409d6f0 0x7f7a541a5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33164/0 (socket says 172.21.15.16:33164) 2023-09-30T22:32:08.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.651+0000 7f7a537fe700 1 -- 172.21.15.16:0/38327151 learned_addr learned my addr 172.21.15.16:0/38327151 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:08.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.652+0000 7f7a52ffd700 1 --2- 172.21.15.16:0/38327151 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7a5409e000 0x7f7a541a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:08.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.652+0000 7f7a53fff700 1 --2- 172.21.15.16:0/38327151 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7a5409e440 0x7f7a541a68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:08.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.652+0000 7f7a537fe700 1 -- 172.21.15.16:0/38327151 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7a5409e000 msgr2=0x7f7a541a6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:08.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.652+0000 7f7a537fe700 1 --2- 172.21.15.16:0/38327151 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7a5409e000 0x7f7a541a6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:08.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.652+0000 7f7a537fe700 1 -- 172.21.15.16:0/38327151 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7a5409e440 msgr2=0x7f7a541a68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:08.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.652+0000 7f7a537fe700 1 --2- 172.21.15.16:0/38327151 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7a5409e440 0x7f7a541a68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:08.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.652+0000 7f7a537fe700 1 -- 172.21.15.16:0/38327151 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a4c009a70 con 0x7f7a5409d6f0 2023-09-30T22:32:08.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.652+0000 7f7a52ffd700 1 --2- 172.21.15.16:0/38327151 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7a5409e000 0x7f7a541a6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:32:08.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.652+0000 7f7a537fe700 1 --2- 172.21.15.16:0/38327151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a5409d6f0 0x7f7a541a5e30 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f7a4400e9b0 tx=0x7f7a4400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:08.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.652+0000 7f7a3bfff700 1 -- 172.21.15.16:0/38327151 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a44009710 con 0x7f7a5409d6f0 2023-09-30T22:32:08.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.652+0000 7f7a3bfff700 1 -- 172.21.15.16:0/38327151 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a4400c850 con 0x7f7a5409d6f0 2023-09-30T22:32:08.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.652+0000 7f7a3bfff700 1 -- 172.21.15.16:0/38327151 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a4400cbc0 con 0x7f7a5409d6f0 2023-09-30T22:32:08.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.652+0000 7f7a59cc4700 1 -- 172.21.15.16:0/38327151 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a5409c710 con 0x7f7a5409d6f0 2023-09-30T22:32:08.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.653+0000 7f7a59cc4700 1 -- 172.21.15.16:0/38327151 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a541a9870 con 0x7f7a5409d6f0 2023-09-30T22:32:08.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.654+0000 7f7a3bfff700 1 -- 172.21.15.16:0/38327151 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7a4400cd20 con 0x7f7a5409d6f0 2023-09-30T22:32:08.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.654+0000 7f7a3bfff700 1 -- 172.21.15.16:0/38327151 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f7a4403fb60 con 0x7f7a5409d6f0 2023-09-30T22:32:08.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.654+0000 7f7a59cc4700 1 -- 172.21.15.16:0/38327151 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a541a9100 con 0x7f7a5409d6f0 2023-09-30T22:32:08.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.660+0000 7f7a3bfff700 1 -- 172.21.15.16:0/38327151 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7a44021020 con 0x7f7a5409d6f0 2023-09-30T22:32:08.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.733+0000 7efd9600e700 1 Processor -- start 2023-09-30T22:32:08.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.733+0000 7efd9600e700 1 -- start start 2023-09-30T22:32:08.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.734+0000 7efd9600e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efd90090450 0x7efd90090820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:08.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.734+0000 7efd9600e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efd90098a30 0x7efd90090d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:08.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.734+0000 7efd9600e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efd9008dc50 0x7efd90088090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:08.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.734+0000 7efd9600e700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd9009ab80 con 0x7efd90098a30 2023-09-30T22:32:08.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.734+0000 7efd9600e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd9009be70 con 0x7efd90090450 2023-09-30T22:32:08.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.734+0000 7efd9600e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efd9009ca10 con 0x7efd9008dc50 2023-09-30T22:32:08.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.734+0000 7efd8effd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efd90098a30 0x7efd90090d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:08.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.734+0000 7efd8f7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efd90090450 0x7efd90090820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:08.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.734+0000 7efd8effd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efd90098a30 0x7efd90090d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33170/0 (socket says 172.21.15.16:33170) 2023-09-30T22:32:08.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.734+0000 7efd8effd700 1 -- 172.21.15.16:0/3329528540 learned_addr learned my addr 172.21.15.16:0/3329528540 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:08.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.734+0000 7efd8ffff700 1 --2- 172.21.15.16:0/3329528540 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efd9008dc50 0x7efd90088090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:08.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.734+0000 7efd8effd700 1 -- 172.21.15.16:0/3329528540 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efd9008dc50 msgr2=0x7efd90088090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:08.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.734+0000 7efd8effd700 1 --2- 172.21.15.16:0/3329528540 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efd9008dc50 0x7efd90088090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:08.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.734+0000 7efd8effd700 1 -- 172.21.15.16:0/3329528540 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efd90090450 msgr2=0x7efd90090820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:08.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.734+0000 7efd8effd700 1 --2- 172.21.15.16:0/3329528540 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efd90090450 0x7efd90090820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:08.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.734+0000 7efd8effd700 1 -- 172.21.15.16:0/3329528540 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd900912a0 con 0x7efd90098a30 2023-09-30T22:32:08.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.734+0000 7efd8f7fe700 1 --2- 172.21.15.16:0/3329528540 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efd90090450 0x7efd90090820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:08.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.734+0000 7f76b0a91700 1 -- 172.21.15.16:0/16292487 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f76ac074ea0 con 0x7f76ac136d80 2023-09-30T22:32:08.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.735+0000 7f76a2ffd700 1 -- 172.21.15.16:0/16292487 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f76a402c020 con 0x7f76ac136d80 2023-09-30T22:32:08.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.735+0000 7efd8effd700 1 --2- 172.21.15.16:0/3329528540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd90098a30 0x7efd90090d60 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7efd840098f0 tx=0x7efd84009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f82aacbb92c876a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:08.738 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:32:08.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.735+0000 7efd8e7fc700 1 -- 172.21.15.16:0/3329528540 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd84011830 con 0x7efd90098a30 2023-09-30T22:32:08.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.735+0000 7efd8e7fc700 1 -- 172.21.15.16:0/3329528540 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd84011990 con 0x7efd90098a30 2023-09-30T22:32:08.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.735+0000 7efd8e7fc700 1 -- 172.21.15.16:0/3329528540 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd84011d20 con 0x7efd90098a30 2023-09-30T22:32:08.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.736+0000 7efd9600e700 1 -- 172.21.15.16:0/3329528540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd90098a30 msgr2=0x7efd90090d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:08.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.736+0000 7efd9600e700 1 --2- 172.21.15.16:0/3329528540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd90098a30 0x7efd90090d60 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7efd840098f0 tx=0x7efd84009c00 comp rx=0 tx=0).stop 2023-09-30T22:32:08.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.736+0000 7efd9600e700 1 -- 172.21.15.16:0/3329528540 shutdown_connections 2023-09-30T22:32:08.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.736+0000 7efd9600e700 1 --2- 172.21.15.16:0/3329528540 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efd90090450 0x7efd90090820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:08.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.736+0000 7efd9600e700 1 --2- 172.21.15.16:0/3329528540 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efd9008dc50 0x7efd90088090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:08.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.736+0000 7efd9600e700 1 --2- 172.21.15.16:0/3329528540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd90098a30 0x7efd90090d60 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:08.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.736+0000 7efd9600e700 1 -- 172.21.15.16:0/3329528540 >> 172.21.15.16:0/3329528540 conn(0x7efd9009e7e0 msgr2=0x7efd9009ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:08.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.736+0000 7efd9600e700 1 -- 172.21.15.16:0/3329528540 shutdown_connections 2023-09-30T22:32:08.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.737+0000 7efd9600e700 1 -- 172.21.15.16:0/3329528540 wait complete. 2023-09-30T22:32:08.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.738+0000 7efd9600e700 1 Processor -- start 2023-09-30T22:32:08.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.738+0000 7f76b0a91700 1 -- 172.21.15.16:0/16292487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76ac136d80 msgr2=0x7f76ac11c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:08.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.738+0000 7f76b0a91700 1 --2- 172.21.15.16:0/16292487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76ac136d80 0x7f76ac11c380 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f76a4016000 tx=0x7f76a4004ad0 comp rx=0 tx=0).stop 2023-09-30T22:32:08.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.738+0000 7f76b0a91700 1 -- 172.21.15.16:0/16292487 shutdown_connections 2023-09-30T22:32:08.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.738+0000 7f76b0a91700 1 --2- 172.21.15.16:0/16292487 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f76ac06a770 0x7f76ac1c62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:08.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.738+0000 7f76b0a91700 1 --2- 172.21.15.16:0/16292487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76ac136d80 0x7f76ac11c380 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:08.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.738+0000 7f76b0a91700 1 --2- 172.21.15.16:0/16292487 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f76ac06b250 0x7f76ac1c6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:08.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.738+0000 7f76b0a91700 1 -- 172.21.15.16:0/16292487 >> 172.21.15.16:0/16292487 conn(0x7f76ac066f50 msgr2=0x7f76ac070010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:08.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.738+0000 7efd9600e700 1 -- start start 2023-09-30T22:32:08.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.739+0000 7efd9600e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efd9008dc50 0x7efd9014ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:08.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.739+0000 7efd9600e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efd90090450 0x7efd9014b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:08.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.739+0000 7efd9600e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd90098a30 0x7efd9014b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:08.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.739+0000 7efd9600e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd9014a800 con 0x7efd90098a30 2023-09-30T22:32:08.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.739+0000 7efd9600e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd90099ff0 con 0x7efd9008dc50 2023-09-30T22:32:08.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.739+0000 7efd9600e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efd9014e090 con 0x7efd90090450 2023-09-30T22:32:08.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.739+0000 7efd8ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd90098a30 0x7efd9014b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:08.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.739+0000 7efd8ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd90098a30 0x7efd9014b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33180/0 (socket says 172.21.15.16:33180) 2023-09-30T22:32:08.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.739+0000 7efd8ffff700 1 -- 172.21.15.16:0/4059127948 learned_addr learned my addr 172.21.15.16:0/4059127948 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:08.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.740+0000 7efd8f7fe700 1 --2- 172.21.15.16:0/4059127948 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efd9008dc50 0x7efd9014ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:08.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.740+0000 7efd8effd700 1 --2- 172.21.15.16:0/4059127948 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efd90090450 0x7efd9014b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:08.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.740+0000 7efd8ffff700 1 -- 172.21.15.16:0/4059127948 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efd90090450 msgr2=0x7efd9014b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:08.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.740+0000 7efd8ffff700 1 --2- 172.21.15.16:0/4059127948 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efd90090450 0x7efd9014b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:08.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.740+0000 7efd8ffff700 1 -- 172.21.15.16:0/4059127948 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efd9008dc50 msgr2=0x7efd9014ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:08.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.740+0000 7efd8ffff700 1 --2- 172.21.15.16:0/4059127948 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efd9008dc50 0x7efd9014ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:08.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.740+0000 7efd8ffff700 1 -- 172.21.15.16:0/4059127948 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd800094a0 con 0x7efd90098a30 2023-09-30T22:32:08.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.740+0000 7efd8f7fe700 1 --2- 172.21.15.16:0/4059127948 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efd9008dc50 0x7efd9014ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:08.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.740+0000 7efd8ffff700 1 --2- 172.21.15.16:0/4059127948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd90098a30 0x7efd9014b8a0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7efd8800efc0 tx=0x7efd88009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:08.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.740+0000 7efd77fff700 1 -- 172.21.15.16:0/4059127948 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd88009b20 con 0x7efd90098a30 2023-09-30T22:32:08.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.741+0000 7efd77fff700 1 -- 172.21.15.16:0/4059127948 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd8800f040 con 0x7efd90098a30 2023-09-30T22:32:08.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.741+0000 7efd77fff700 1 -- 172.21.15.16:0/4059127948 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8800cd40 con 0x7efd90098a30 2023-09-30T22:32:08.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.741+0000 7efd9600e700 1 -- 172.21.15.16:0/4059127948 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd84009550 con 0x7efd90098a30 2023-09-30T22:32:08.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.741+0000 7efd9600e700 1 -- 172.21.15.16:0/4059127948 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd9009c210 con 0x7efd90098a30 2023-09-30T22:32:08.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.742+0000 7efd77fff700 1 -- 172.21.15.16:0/4059127948 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7efd88014480 con 0x7efd90098a30 2023-09-30T22:32:08.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.742+0000 7f76b0a91700 1 -- 172.21.15.16:0/16292487 shutdown_connections 2023-09-30T22:32:08.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.742+0000 7efd77fff700 1 -- 172.21.15.16:0/4059127948 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7efd880402c0 con 0x7efd90098a30 2023-09-30T22:32:08.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.742+0000 7f76b0a91700 1 -- 172.21.15.16:0/16292487 wait complete. 2023-09-30T22:32:08.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.742+0000 7efd9600e700 1 -- 172.21.15.16:0/4059127948 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd900747f0 con 0x7efd90098a30 2023-09-30T22:32:08.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.751+0000 7efd77fff700 1 -- 172.21.15.16:0/4059127948 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7efd88012930 con 0x7efd90098a30 2023-09-30T22:32:08.760 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:32:08.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.785+0000 7fc8b92cc700 1 Processor -- start 2023-09-30T22:32:08.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.785+0000 7fc8b92cc700 1 -- start start 2023-09-30T22:32:08.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.786+0000 7fc8b92cc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc8b409d6f0 0x7fc8b409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.786+0000 7fc8b92cc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc8b409e000 0x7fc8b41339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.786+0000 7fc8b92cc700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8b409b120 con 0x7fc8b409e410 2023-09-30T22:32:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.786+0000 7fc8b92cc700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8b406b460 con 0x7fc8b409d6f0 2023-09-30T22:32:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.786+0000 7fc8b92cc700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b409a7e0 con 0x7fc8b409e000 2023-09-30T22:32:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.786+0000 7fc8b27fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc8b409e000 0x7fc8b41339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.786+0000 7fc8b2ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc8b409d6f0 0x7fc8b409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.786+0000 7fc8b27fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc8b409e000 0x7fc8b41339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36796/0 (socket says 172.21.15.16:36796) 2023-09-30T22:32:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.786+0000 7fc8b27fc700 1 -- 172.21.15.16:0/2073705157 learned_addr learned my addr 172.21.15.16:0/2073705157 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:08.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.787+0000 7fc8b27fc700 1 -- 172.21.15.16:0/2073705157 >> v2:172.21.15.16:3300/0 conn(0x7fc8b409d6f0 msgr2=0x7fc8b409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:08.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.787+0000 7fc8b27fc700 1 --2- 172.21.15.16:0/2073705157 >> v2:172.21.15.16:3300/0 conn(0x7fc8b409d6f0 0x7fc8b409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:08.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.787+0000 7fc8b27fc700 1 -- 172.21.15.16:0/2073705157 >> v1:172.21.15.16:6789/0 conn(0x7fc8b409e410 legacy=0x7fc8b4133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:08.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.787+0000 7fc8b27fc700 1 -- 172.21.15.16:0/2073705157 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b409a500 con 0x7fc8b409e000 2023-09-30T22:32:08.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.787+0000 7fc8b27fc700 1 --2- 172.21.15.16:0/2073705157 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc8b409e000 0x7fc8b41339f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8009920 tx=0x7fc8a8009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c19aecb63dece5b0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:08.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.787+0000 7fc8b1ffb700 1 -- 172.21.15.16:0/2073705157 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a8011830 con 0x7fc8b409e000 2023-09-30T22:32:08.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.787+0000 7fc8b1ffb700 1 -- 172.21.15.16:0/2073705157 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8a8011990 con 0x7fc8b409e000 2023-09-30T22:32:08.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.787+0000 7fc8b1ffb700 1 -- 172.21.15.16:0/2073705157 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a8011d20 con 0x7fc8b409e000 2023-09-30T22:32:08.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.788+0000 7fc8b92cc700 1 -- 172.21.15.16:0/2073705157 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc8b409e000 msgr2=0x7fc8b41339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:08.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.788+0000 7fc8b92cc700 1 --2- 172.21.15.16:0/2073705157 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc8b409e000 0x7fc8b41339f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc8a8009920 tx=0x7fc8a8009c30 comp rx=0 tx=0).stop 2023-09-30T22:32:08.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.788+0000 7fc8b92cc700 1 -- 172.21.15.16:0/2073705157 shutdown_connections 2023-09-30T22:32:08.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.788+0000 7fc8b92cc700 1 --2- 172.21.15.16:0/2073705157 >> v2:172.21.15.16:3300/0 conn(0x7fc8b409d6f0 0x7fc8b409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:08.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.788+0000 7fc8b92cc700 1 --2- 172.21.15.16:0/2073705157 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc8b409e000 0x7fc8b41339f0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:08.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.788+0000 7fc8b92cc700 1 -- 172.21.15.16:0/2073705157 >> 172.21.15.16:0/2073705157 conn(0x7fc8b4066f50 msgr2=0x7fc8b409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:08.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.788+0000 7fc8b92cc700 1 -- 172.21.15.16:0/2073705157 shutdown_connections 2023-09-30T22:32:08.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.788+0000 7fc8b92cc700 1 -- 172.21.15.16:0/2073705157 wait complete. 2023-09-30T22:32:08.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.790+0000 7fc8b92cc700 1 Processor -- start 2023-09-30T22:32:08.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.790+0000 7fc8b92cc700 1 -- start start 2023-09-30T22:32:08.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.791+0000 7fc8b92cc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8b409d6f0 0x7fc8b41a5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:08.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.791+0000 7fc8b92cc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc8b409e000 0x7fc8b41a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:08.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.791+0000 7fc8b92cc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc8b409e410 0x7fc8b41a6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:08.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.791+0000 7fc8b92cc700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b41a7380 con 0x7fc8b409d6f0 2023-09-30T22:32:08.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.791+0000 7fc8b92cc700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b41a7ae0 con 0x7fc8b409e000 2023-09-30T22:32:08.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.791+0000 7fc8b92cc700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b41a8100 con 0x7fc8b409e410 2023-09-30T22:32:08.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.791+0000 7fc8b2ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8b409d6f0 0x7fc8b41a5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:08.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.791+0000 7fc8b37fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc8b409e410 0x7fc8b41a6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:08.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.791+0000 7fc8b2ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8b409d6f0 0x7fc8b41a5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33186/0 (socket says 172.21.15.16:33186) 2023-09-30T22:32:08.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.791+0000 7fc8b2ffd700 1 -- 172.21.15.16:0/55128059 learned_addr learned my addr 172.21.15.16:0/55128059 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:08.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.791+0000 7fc8b27fc700 1 --2- 172.21.15.16:0/55128059 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc8b409e000 0x7fc8b41a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:08.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.791+0000 7fc8b2ffd700 1 -- 172.21.15.16:0/55128059 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc8b409e410 msgr2=0x7fc8b41a6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:08.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.791+0000 7fc8b2ffd700 1 --2- 172.21.15.16:0/55128059 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc8b409e410 0x7fc8b41a6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:08.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.791+0000 7fc8b2ffd700 1 -- 172.21.15.16:0/55128059 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc8b409e000 msgr2=0x7fc8b41a6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:08.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.791+0000 7fc8b2ffd700 1 --2- 172.21.15.16:0/55128059 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc8b409e000 0x7fc8b41a6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:08.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.791+0000 7fc8b2ffd700 1 -- 172.21.15.16:0/55128059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a8009550 con 0x7fc8b409d6f0 2023-09-30T22:32:08.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.792+0000 7fc8b2ffd700 1 --2- 172.21.15.16:0/55128059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8b409d6f0 0x7fc8b41a5ed0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fc8a400e8a0 tx=0x7fc8a400ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:08.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.792+0000 7fc89b7fe700 1 -- 172.21.15.16:0/55128059 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a4009590 con 0x7fc8b409d6f0 2023-09-30T22:32:08.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.792+0000 7fc89b7fe700 1 -- 172.21.15.16:0/55128059 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8a4013680 con 0x7fc8b409d6f0 2023-09-30T22:32:08.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.792+0000 7fc8b92cc700 1 -- 172.21.15.16:0/55128059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8b41a9710 con 0x7fc8b409d6f0 2023-09-30T22:32:08.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.792+0000 7fc89b7fe700 1 -- 172.21.15.16:0/55128059 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a40139f0 con 0x7fc8b409d6f0 2023-09-30T22:32:08.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.792+0000 7fc8b92cc700 1 -- 172.21.15.16:0/55128059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8b409c710 con 0x7fc8b409d6f0 2023-09-30T22:32:08.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.793+0000 7fc89b7fe700 1 -- 172.21.15.16:0/55128059 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc8a4013b50 con 0x7fc8b409d6f0 2023-09-30T22:32:08.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.793+0000 7fc89b7fe700 1 -- 172.21.15.16:0/55128059 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fc8a403f5a0 con 0x7fc8b409d6f0 2023-09-30T22:32:08.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.793+0000 7fc8b92cc700 1 -- 172.21.15.16:0/55128059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8b4074ea0 con 0x7fc8b409d6f0 2023-09-30T22:32:08.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.798+0000 7fc89b7fe700 1 -- 172.21.15.16:0/55128059 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc8a4017080 con 0x7fc8b409d6f0 2023-09-30T22:32:08.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.878+0000 7f7a59cc4700 1 -- 172.21.15.16:0/38327151 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7a5409f420 con 0x7f7a5409d6f0 2023-09-30T22:32:08.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.879+0000 7f7a3bfff700 1 -- 172.21.15.16:0/38327151 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7a44017070 con 0x7f7a5409d6f0 2023-09-30T22:32:08.879 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:32:08.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.881+0000 7f7a59cc4700 1 -- 172.21.15.16:0/38327151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a5409d6f0 msgr2=0x7f7a541a5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:08.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.881+0000 7f7a59cc4700 1 --2- 172.21.15.16:0/38327151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a5409d6f0 0x7f7a541a5e30 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f7a4400e9b0 tx=0x7f7a4400ecc0 comp rx=0 tx=0).stop 2023-09-30T22:32:08.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.881+0000 7f7a59cc4700 1 -- 172.21.15.16:0/38327151 shutdown_connections 2023-09-30T22:32:08.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.881+0000 7f7a59cc4700 1 --2- 172.21.15.16:0/38327151 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7a5409e440 0x7f7a541a68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:08.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.881+0000 7f7a59cc4700 1 --2- 172.21.15.16:0/38327151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a5409d6f0 0x7f7a541a5e30 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:08.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.881+0000 7f7a59cc4700 1 --2- 172.21.15.16:0/38327151 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7a5409e000 0x7f7a541a6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:08.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.881+0000 7f7a59cc4700 1 -- 172.21.15.16:0/38327151 >> 172.21.15.16:0/38327151 conn(0x7f7a54066f50 msgr2=0x7f7a541bae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:08.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.881+0000 7f7a59cc4700 1 -- 172.21.15.16:0/38327151 shutdown_connections 2023-09-30T22:32:08.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.881+0000 7f7a59cc4700 1 -- 172.21.15.16:0/38327151 wait complete. 2023-09-30T22:32:08.896 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:32:08.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.973+0000 7efd9600e700 1 -- 172.21.15.16:0/4059127948 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7efd9014ecd0 con 0x7efd90098a30 2023-09-30T22:32:08.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.973+0000 7efd77fff700 1 -- 172.21.15.16:0/4059127948 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7efd8802c090 con 0x7efd90098a30 2023-09-30T22:32:08.974 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:32:08.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.977+0000 7efd9600e700 1 -- 172.21.15.16:0/4059127948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd90098a30 msgr2=0x7efd9014b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:08.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.977+0000 7efd9600e700 1 --2- 172.21.15.16:0/4059127948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd90098a30 0x7efd9014b8a0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7efd8800efc0 tx=0x7efd88009f60 comp rx=0 tx=0).stop 2023-09-30T22:32:08.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.977+0000 7efd9600e700 1 -- 172.21.15.16:0/4059127948 shutdown_connections 2023-09-30T22:32:08.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.977+0000 7efd9600e700 1 --2- 172.21.15.16:0/4059127948 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efd9008dc50 0x7efd9014ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:08.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.977+0000 7efd9600e700 1 --2- 172.21.15.16:0/4059127948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efd90098a30 0x7efd9014b8a0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:08.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.977+0000 7efd9600e700 1 --2- 172.21.15.16:0/4059127948 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efd90090450 0x7efd9014b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:08.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.977+0000 7efd9600e700 1 -- 172.21.15.16:0/4059127948 >> 172.21.15.16:0/4059127948 conn(0x7efd9009e7e0 msgr2=0x7efd90148e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:08.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.977+0000 7efd9600e700 1 -- 172.21.15.16:0/4059127948 shutdown_connections 2023-09-30T22:32:08.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:08.977+0000 7efd9600e700 1 -- 172.21.15.16:0/4059127948 wait complete. 2023-09-30T22:32:08.991 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:32:09.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.009+0000 7fc8b92cc700 1 -- 172.21.15.16:0/55128059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc8b41a8d30 con 0x7fc8b409d6f0 2023-09-30T22:32:09.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.010+0000 7fc89b7fe700 1 -- 172.21.15.16:0/55128059 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc8a4021020 con 0x7fc8b409d6f0 2023-09-30T22:32:09.010 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:32:09.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.012+0000 7fc8b92cc700 1 -- 172.21.15.16:0/55128059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8b409d6f0 msgr2=0x7fc8b41a5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:09.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.012+0000 7fc8b92cc700 1 --2- 172.21.15.16:0/55128059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8b409d6f0 0x7fc8b41a5ed0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fc8a400e8a0 tx=0x7fc8a400ebb0 comp rx=0 tx=0).stop 2023-09-30T22:32:09.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.012+0000 7fc8b92cc700 1 -- 172.21.15.16:0/55128059 shutdown_connections 2023-09-30T22:32:09.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.012+0000 7fc8b92cc700 1 --2- 172.21.15.16:0/55128059 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc8b409e410 0x7fc8b41a6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:09.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.012+0000 7fc8b92cc700 1 --2- 172.21.15.16:0/55128059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8b409d6f0 0x7fc8b41a5ed0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:09.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.012+0000 7fc8b92cc700 1 --2- 172.21.15.16:0/55128059 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc8b409e000 0x7fc8b41a6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:09.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.012+0000 7fc8b92cc700 1 -- 172.21.15.16:0/55128059 >> 172.21.15.16:0/55128059 conn(0x7fc8b4066f50 msgr2=0x7fc8b41bad60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:09.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.012+0000 7fc8b92cc700 1 -- 172.21.15.16:0/55128059 shutdown_connections 2023-09-30T22:32:09.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.012+0000 7fc8b92cc700 1 -- 172.21.15.16:0/55128059 wait complete. 2023-09-30T22:32:09.026 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:32:09.761 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:32:09.897 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:32:09.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.909+0000 7fa78bc73700 1 Processor -- start 2023-09-30T22:32:09.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.910+0000 7fa78bc73700 1 -- start start 2023-09-30T22:32:09.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.910+0000 7fa78bc73700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa784130540 0x7fa784130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:09.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.910+0000 7fa78bc73700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa784130e50 0x7fa78406f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:09.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.910+0000 7fa78bc73700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa78406fb20 0x7fa78406ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:09.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.910+0000 7fa78bc73700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa78409a390 con 0x7fa78406fb20 2023-09-30T22:32:09.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.910+0000 7fa78bc73700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7840716a0 con 0x7fa784130e50 2023-09-30T22:32:09.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.910+0000 7fa78bc73700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa784071290 con 0x7fa784130540 2023-09-30T22:32:09.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.910+0000 7fa78920e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa784130e50 0x7fa78406f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:09.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.910+0000 7fa78a210700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa78406fb20 0x7fa78406ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:09.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.911+0000 7fa78920e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa784130e50 0x7fa78406f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36816/0 (socket says 172.21.15.16:36816) 2023-09-30T22:32:09.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.911+0000 7fa78a210700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa78406fb20 0x7fa78406ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33188/0 (socket says 172.21.15.16:33188) 2023-09-30T22:32:09.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.911+0000 7fa78920e700 1 -- 172.21.15.16:0/2203000506 learned_addr learned my addr 172.21.15.16:0/2203000506 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:09.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.911+0000 7fa789a0f700 1 --2- 172.21.15.16:0/2203000506 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa784130540 0x7fa784130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:09.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.911+0000 7fa78920e700 1 -- 172.21.15.16:0/2203000506 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa784130540 msgr2=0x7fa784130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:09.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.911+0000 7fa78920e700 1 --2- 172.21.15.16:0/2203000506 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa784130540 0x7fa784130910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:09.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.911+0000 7fa78920e700 1 -- 172.21.15.16:0/2203000506 >> v2:172.21.15.16:3300/0 conn(0x7fa78406fb20 msgr2=0x7fa78406ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:09.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.911+0000 7fa78920e700 1 --2- 172.21.15.16:0/2203000506 >> v2:172.21.15.16:3300/0 conn(0x7fa78406fb20 0x7fa78406ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:09.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.911+0000 7fa78920e700 1 -- 172.21.15.16:0/2203000506 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa784130340 con 0x7fa784130e50 2023-09-30T22:32:09.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.911+0000 7fa78a210700 1 --2- 172.21.15.16:0/2203000506 >> v2:172.21.15.16:3300/0 conn(0x7fa78406fb20 0x7fa78406ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:09.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.911+0000 7fa78920e700 1 --2- 172.21.15.16:0/2203000506 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa784130e50 0x7fa78406f5e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa774009920 tx=0x7fa774009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f10c0489d3cfb748 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:09.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.911+0000 7fa788a0d700 1 -- 172.21.15.16:0/2203000506 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa774011830 con 0x7fa784130e50 2023-09-30T22:32:09.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.911+0000 7fa788a0d700 1 -- 172.21.15.16:0/2203000506 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa774011990 con 0x7fa784130e50 2023-09-30T22:32:09.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.911+0000 7fa788a0d700 1 -- 172.21.15.16:0/2203000506 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa774011d20 con 0x7fa784130e50 2023-09-30T22:32:09.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.912+0000 7fa78bc73700 1 -- 172.21.15.16:0/2203000506 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa784130e50 msgr2=0x7fa78406f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:09.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.912+0000 7fa78bc73700 1 --2- 172.21.15.16:0/2203000506 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa784130e50 0x7fa78406f5e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa774009920 tx=0x7fa774009c30 comp rx=0 tx=0).stop 2023-09-30T22:32:09.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.912+0000 7fa78bc73700 1 -- 172.21.15.16:0/2203000506 shutdown_connections 2023-09-30T22:32:09.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.912+0000 7fa78bc73700 1 --2- 172.21.15.16:0/2203000506 >> v2:172.21.15.16:3300/0 conn(0x7fa78406fb20 0x7fa78406ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:09.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.912+0000 7fa78bc73700 1 --2- 172.21.15.16:0/2203000506 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa784130540 0x7fa784130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:09.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.912+0000 7fa78bc73700 1 --2- 172.21.15.16:0/2203000506 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa784130e50 0x7fa78406f5e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:09.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.912+0000 7fa78bc73700 1 -- 172.21.15.16:0/2203000506 >> 172.21.15.16:0/2203000506 conn(0x7fa7840a2aa0 msgr2=0x7fa7840a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:09.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.912+0000 7fa78bc73700 1 -- 172.21.15.16:0/2203000506 shutdown_connections 2023-09-30T22:32:09.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.912+0000 7fa78bc73700 1 -- 172.21.15.16:0/2203000506 wait complete. 2023-09-30T22:32:09.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.913+0000 7fa78bc73700 1 Processor -- start 2023-09-30T22:32:09.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.913+0000 7fa78bc73700 1 -- start start 2023-09-30T22:32:09.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.914+0000 7fa78bc73700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa78406fb20 0x7fa78409a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:09.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.914+0000 7fa78bc73700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa784130540 0x7fa78409aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:09.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.914+0000 7fa78bc73700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa784130e50 0x7fa78409b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:09.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.914+0000 7fa78bc73700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa78411e310 con 0x7fa784130e50 2023-09-30T22:32:09.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.914+0000 7fa78bc73700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa78411d200 con 0x7fa78406fb20 2023-09-30T22:32:09.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.914+0000 7fa78bc73700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa78412bbe0 con 0x7fa784130540 2023-09-30T22:32:09.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.914+0000 7fa789a0f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa78406fb20 0x7fa78409a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:09.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.914+0000 7fa789a0f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa78406fb20 0x7fa78409a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36830/0 (socket says 172.21.15.16:36830) 2023-09-30T22:32:09.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.914+0000 7fa789a0f700 1 -- 172.21.15.16:0/1146104409 learned_addr learned my addr 172.21.15.16:0/1146104409 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:09.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.914+0000 7fa78a210700 1 --2- 172.21.15.16:0/1146104409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa784130e50 0x7fa78409b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:09.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.914+0000 7fa78920e700 1 --2- 172.21.15.16:0/1146104409 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa784130540 0x7fa78409aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:09.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.914+0000 7fa789a0f700 1 -- 172.21.15.16:0/1146104409 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa784130540 msgr2=0x7fa78409aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:09.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.914+0000 7fa789a0f700 1 --2- 172.21.15.16:0/1146104409 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa784130540 0x7fa78409aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:09.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.914+0000 7fa789a0f700 1 -- 172.21.15.16:0/1146104409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa784130e50 msgr2=0x7fa78409b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:09.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.914+0000 7fa789a0f700 1 --2- 172.21.15.16:0/1146104409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa784130e50 0x7fa78409b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:09.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.914+0000 7fa789a0f700 1 -- 172.21.15.16:0/1146104409 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa77c009a70 con 0x7fa78406fb20 2023-09-30T22:32:09.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.914+0000 7fa78a210700 1 --2- 172.21.15.16:0/1146104409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa784130e50 0x7fa78409b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:09.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.914+0000 7fa789a0f700 1 --2- 172.21.15.16:0/1146104409 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa78406fb20 0x7fa78409a590 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa78000e9b0 tx=0x7fa78000ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:09.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.915+0000 7fa77a7fc700 1 -- 172.21.15.16:0/1146104409 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa780009710 con 0x7fa78406fb20 2023-09-30T22:32:09.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.915+0000 7fa77a7fc700 1 -- 172.21.15.16:0/1146104409 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa78000ee40 con 0x7fa78406fb20 2023-09-30T22:32:09.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.915+0000 7fa77a7fc700 1 -- 172.21.15.16:0/1146104409 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78000ca50 con 0x7fa78406fb20 2023-09-30T22:32:09.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.915+0000 7fa78bc73700 1 -- 172.21.15.16:0/1146104409 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa774009550 con 0x7fa78406fb20 2023-09-30T22:32:09.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.915+0000 7fa78bc73700 1 -- 172.21.15.16:0/1146104409 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7840718e0 con 0x7fa78406fb20 2023-09-30T22:32:09.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.915+0000 7fa77a7fc700 1 -- 172.21.15.16:0/1146104409 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa78000cbb0 con 0x7fa78406fb20 2023-09-30T22:32:09.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.916+0000 7fa77a7fc700 1 -- 172.21.15.16:0/1146104409 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fa78003fb40 con 0x7fa78406fb20 2023-09-30T22:32:09.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.916+0000 7fa78bc73700 1 -- 172.21.15.16:0/1146104409 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa78412f8d0 con 0x7fa78406fb20 2023-09-30T22:32:09.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:09.921+0000 7fa77a7fc700 1 -- 172.21.15.16:0/1146104409 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa780017080 con 0x7fa78406fb20 2023-09-30T22:32:09.992 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:32:10.027 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:32:10.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.045+0000 7f376d21a700 1 Processor -- start 2023-09-30T22:32:10.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.045+0000 7f376d21a700 1 -- start start 2023-09-30T22:32:10.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.046+0000 7f376d21a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3768136830 0x7f3768138c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:10.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.046+0000 7f376d21a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3768139150 0x7f376813b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:10.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.046+0000 7f376d21a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f376806f720 con 0x7f37681338c0 2023-09-30T22:32:10.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.046+0000 7f376d21a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3768070c80 con 0x7f3768136830 2023-09-30T22:32:10.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.046+0000 7f376d21a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f376806fce0 con 0x7f3768139150 2023-09-30T22:32:10.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.046+0000 7f3766d9d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f37681338c0 0x7f3768133c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48214/0 (socket says 172.21.15.16:48214) 2023-09-30T22:32:10.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.046+0000 7f3766d9d700 1 -- 172.21.15.16:0/316408955 learned_addr learned my addr 172.21.15.16:0/316408955 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:10.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.046+0000 7f376659c700 1 --2- 172.21.15.16:0/316408955 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3768136830 0x7f3768138c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:10.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.046+0000 7f376759e700 1 --2- 172.21.15.16:0/316408955 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3768139150 0x7f376813b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:10.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.047+0000 7f3765d9b700 1 -- 172.21.15.16:0/316408955 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3044275075 0 0) 0x7f376806f720 con 0x7f37681338c0 2023-09-30T22:32:10.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.047+0000 7f3765d9b700 1 -- 172.21.15.16:0/316408955 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37500035b0 con 0x7f37681338c0 2023-09-30T22:32:10.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.047+0000 7f376659c700 1 -- 172.21.15.16:0/316408955 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3768139150 msgr2=0x7f376813b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:10.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.047+0000 7f376659c700 1 --2- 172.21.15.16:0/316408955 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3768139150 0x7f376813b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:10.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.047+0000 7f376659c700 1 -- 172.21.15.16:0/316408955 >> v1:172.21.15.16:6789/0 conn(0x7f37681338c0 legacy=0x7f3768133c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:10.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.047+0000 7f376659c700 1 -- 172.21.15.16:0/316408955 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37681336a0 con 0x7f3768136830 2023-09-30T22:32:10.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.047+0000 7f376659c700 1 --2- 172.21.15.16:0/316408955 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3768136830 0x7f3768138c10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f375c009870 tx=0x7f375c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=243e3d7dd375a740 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:10.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.047+0000 7f3765d9b700 1 -- 172.21.15.16:0/316408955 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f375c011830 con 0x7f3768136830 2023-09-30T22:32:10.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.047+0000 7f3765d9b700 1 -- 172.21.15.16:0/316408955 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f375c011990 con 0x7f3768136830 2023-09-30T22:32:10.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.047+0000 7f3765d9b700 1 -- 172.21.15.16:0/316408955 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f375c011d20 con 0x7f3768136830 2023-09-30T22:32:10.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.048+0000 7f376d21a700 1 -- 172.21.15.16:0/316408955 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3768136830 msgr2=0x7f3768138c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:10.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.048+0000 7f376d21a700 1 --2- 172.21.15.16:0/316408955 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3768136830 0x7f3768138c10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f375c009870 tx=0x7f375c009b80 comp rx=0 tx=0).stop 2023-09-30T22:32:10.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.048+0000 7f376d21a700 1 -- 172.21.15.16:0/316408955 shutdown_connections 2023-09-30T22:32:10.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.048+0000 7f376d21a700 1 --2- 172.21.15.16:0/316408955 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3768139150 0x7f376813b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:10.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.048+0000 7f376d21a700 1 --2- 172.21.15.16:0/316408955 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3768136830 0x7f3768138c10 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:10.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.048+0000 7f376d21a700 1 -- 172.21.15.16:0/316408955 >> 172.21.15.16:0/316408955 conn(0x7f37680a2aa0 msgr2=0x7f37680a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:10.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.048+0000 7f376d21a700 1 -- 172.21.15.16:0/316408955 shutdown_connections 2023-09-30T22:32:10.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.048+0000 7f376d21a700 1 -- 172.21.15.16:0/316408955 wait complete. 2023-09-30T22:32:10.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.050+0000 7f376d21a700 1 Processor -- start 2023-09-30T22:32:10.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.050+0000 7f376d21a700 1 -- start start 2023-09-30T22:32:10.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.050+0000 7f376d21a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f37681338c0 0x7f376812dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:10.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.050+0000 7f376d21a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3768136830 0x7f376812e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:10.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.050+0000 7f376d21a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3768139150 0x7f3768118fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:10.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.050+0000 7f376d21a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f376811ab20 con 0x7f3768139150 2023-09-30T22:32:10.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.050+0000 7f376d21a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3768130000 con 0x7f37681338c0 2023-09-30T22:32:10.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.050+0000 7f376d21a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f376811a490 con 0x7f3768136830 2023-09-30T22:32:10.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.051+0000 7f3766d9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f37681338c0 0x7f376812dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:10.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.051+0000 7f3766d9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f37681338c0 0x7f376812dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36846/0 (socket says 172.21.15.16:36846) 2023-09-30T22:32:10.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.051+0000 7f3766d9d700 1 -- 172.21.15.16:0/1213971608 learned_addr learned my addr 172.21.15.16:0/1213971608 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:10.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.051+0000 7f376659c700 1 --2- 172.21.15.16:0/1213971608 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3768136830 0x7f376812e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:10.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.051+0000 7f376759e700 1 --2- 172.21.15.16:0/1213971608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3768139150 0x7f3768118fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:10.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.051+0000 7f3766d9d700 1 -- 172.21.15.16:0/1213971608 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3768136830 msgr2=0x7f376812e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:10.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.051+0000 7f3766d9d700 1 --2- 172.21.15.16:0/1213971608 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3768136830 0x7f376812e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:10.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.051+0000 7f3766d9d700 1 -- 172.21.15.16:0/1213971608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3768139150 msgr2=0x7f3768118fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:10.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.051+0000 7f3766d9d700 1 --2- 172.21.15.16:0/1213971608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3768139150 0x7f3768118fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:10.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.051+0000 7f3766d9d700 1 -- 172.21.15.16:0/1213971608 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f375c0094a0 con 0x7f37681338c0 2023-09-30T22:32:10.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.051+0000 7f376759e700 1 --2- 172.21.15.16:0/1213971608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3768139150 0x7f3768118fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:32:10.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.052+0000 7f3766d9d700 1 --2- 172.21.15.16:0/1213971608 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f37681338c0 0x7f376812dcf0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f375800c730 tx=0x7f375800ca40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:10.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.052+0000 7f374f7fe700 1 -- 172.21.15.16:0/1213971608 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3758005d70 con 0x7f37681338c0 2023-09-30T22:32:10.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.052+0000 7f374f7fe700 1 -- 172.21.15.16:0/1213971608 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37580070d0 con 0x7f37681338c0 2023-09-30T22:32:10.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.052+0000 7f374f7fe700 1 -- 172.21.15.16:0/1213971608 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3758007440 con 0x7f37681338c0 2023-09-30T22:32:10.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.052+0000 7f376d21a700 1 -- 172.21.15.16:0/1213971608 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f376811c5b0 con 0x7f37681338c0 2023-09-30T22:32:10.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.052+0000 7f376d21a700 1 -- 172.21.15.16:0/1213971608 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f376811c890 con 0x7f37681338c0 2023-09-30T22:32:10.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.053+0000 7f376d21a700 1 -- 172.21.15.16:0/1213971608 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3768074ea0 con 0x7f37681338c0 2023-09-30T22:32:10.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.053+0000 7f374f7fe700 1 -- 172.21.15.16:0/1213971608 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f37580075a0 con 0x7f37681338c0 2023-09-30T22:32:10.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.054+0000 7f374f7fe700 1 -- 172.21.15.16:0/1213971608 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f375803ebe0 con 0x7f37681338c0 2023-09-30T22:32:10.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.059+0000 7f374f7fe700 1 -- 172.21.15.16:0/1213971608 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3758011d70 con 0x7f37681338c0 2023-09-30T22:32:10.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.136+0000 7fb7114f4700 1 Processor -- start 2023-09-30T22:32:10.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.136+0000 7fb7114f4700 1 -- start start 2023-09-30T22:32:10.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.137+0000 7fb7114f4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb70c09d6f0 0x7fb70c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:10.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.137+0000 7fb7114f4700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb70c09e000 0x7fb70c130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:10.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.137+0000 7fb7114f4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb70c09e410 0x7fb70c130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:10.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.137+0000 7fb7114f4700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb70c09f520 con 0x7fb70c09d6f0 2023-09-30T22:32:10.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.137+0000 7fb7114f4700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb70c09f130 con 0x7fb70c09e000 2023-09-30T22:32:10.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.137+0000 7fb7114f4700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb70c09e930 con 0x7fb70c09e410 2023-09-30T22:32:10.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.137+0000 7fb70a7fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb70c09e000 0x7fb70c130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:10.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.137+0000 7fb70b7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb70c09e410 0x7fb70c130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:10.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.137+0000 7fb70a7fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb70c09e000 0x7fb70c130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36850/0 (socket says 172.21.15.16:36850) 2023-09-30T22:32:10.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.137+0000 7fb70a7fc700 1 -- 172.21.15.16:0/3649055480 learned_addr learned my addr 172.21.15.16:0/3649055480 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:10.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.137+0000 7fb70b7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb70c09e410 0x7fb70c130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:55534/0 (socket says 172.21.15.16:55534) 2023-09-30T22:32:10.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.138+0000 7fb70affd700 1 --2- 172.21.15.16:0/3649055480 >> v2:172.21.15.16:3300/0 conn(0x7fb70c09d6f0 0x7fb70c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:10.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.138+0000 7fb70a7fc700 1 -- 172.21.15.16:0/3649055480 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb70c09e410 msgr2=0x7fb70c130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:10.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.138+0000 7fb70a7fc700 1 --2- 172.21.15.16:0/3649055480 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb70c09e410 0x7fb70c130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:10.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.138+0000 7fb70a7fc700 1 -- 172.21.15.16:0/3649055480 >> v2:172.21.15.16:3300/0 conn(0x7fb70c09d6f0 msgr2=0x7fb70c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:10.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.138+0000 7fb70a7fc700 1 --2- 172.21.15.16:0/3649055480 >> v2:172.21.15.16:3300/0 conn(0x7fb70c09d6f0 0x7fb70c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:10.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.138+0000 7fb70a7fc700 1 -- 172.21.15.16:0/3649055480 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb70c09b2d0 con 0x7fb70c09e000 2023-09-30T22:32:10.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.138+0000 7fb70a7fc700 1 --2- 172.21.15.16:0/3649055480 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb70c09e000 0x7fb70c130660 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb70000bb90 tx=0x7fb70000bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31b5cff067f6e34f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:10.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.138+0000 7fb709ffb700 1 -- 172.21.15.16:0/3649055480 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb70000dd30 con 0x7fb70c09e000 2023-09-30T22:32:10.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.138+0000 7fb709ffb700 1 -- 172.21.15.16:0/3649055480 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb70000de90 con 0x7fb70c09e000 2023-09-30T22:32:10.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.138+0000 7fb709ffb700 1 -- 172.21.15.16:0/3649055480 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb700010690 con 0x7fb70c09e000 2023-09-30T22:32:10.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.139+0000 7fb7114f4700 1 -- 172.21.15.16:0/3649055480 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb70c09e000 msgr2=0x7fb70c130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:10.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.139+0000 7fb7114f4700 1 --2- 172.21.15.16:0/3649055480 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb70c09e000 0x7fb70c130660 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb70000bb90 tx=0x7fb70000bea0 comp rx=0 tx=0).stop 2023-09-30T22:32:10.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.139+0000 7fb7114f4700 1 -- 172.21.15.16:0/3649055480 shutdown_connections 2023-09-30T22:32:10.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.139+0000 7fb7114f4700 1 --2- 172.21.15.16:0/3649055480 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb70c09e410 0x7fb70c130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:10.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.139+0000 7fb7114f4700 1 --2- 172.21.15.16:0/3649055480 >> v2:172.21.15.16:3300/0 conn(0x7fb70c09d6f0 0x7fb70c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:10.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.139+0000 7fb7114f4700 1 --2- 172.21.15.16:0/3649055480 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb70c09e000 0x7fb70c130660 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:10.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.139+0000 7fb7114f4700 1 -- 172.21.15.16:0/3649055480 >> 172.21.15.16:0/3649055480 conn(0x7fb70c066f50 msgr2=0x7fb70c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:10.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.139+0000 7fb7114f4700 1 -- 172.21.15.16:0/3649055480 shutdown_connections 2023-09-30T22:32:10.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.139+0000 7fb7114f4700 1 -- 172.21.15.16:0/3649055480 wait complete. 2023-09-30T22:32:10.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.141+0000 7fb7114f4700 1 Processor -- start 2023-09-30T22:32:10.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.141+0000 7fb7114f4700 1 -- start start 2023-09-30T22:32:10.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.142+0000 7fb7114f4700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb70c09d6f0 0x7fb70c11e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:10.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.142+0000 7fb7114f4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb70c09e000 0x7fb70c11e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:10.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.142+0000 7fb7114f4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb70c09e410 0x7fb70c11ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:10.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.142+0000 7fb7114f4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb70c121480 con 0x7fb70c09e000 2023-09-30T22:32:10.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.142+0000 7fb7114f4700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb70c1204c0 con 0x7fb70c09d6f0 2023-09-30T22:32:10.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.142+0000 7fb7114f4700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb70c121da0 con 0x7fb70c09e410 2023-09-30T22:32:10.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.142+0000 7fb70affd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb70c09d6f0 0x7fb70c11e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:10.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.142+0000 7fb70affd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb70c09d6f0 0x7fb70c11e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36862/0 (socket says 172.21.15.16:36862) 2023-09-30T22:32:10.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.142+0000 7fb70affd700 1 -- 172.21.15.16:0/3400038860 learned_addr learned my addr 172.21.15.16:0/3400038860 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:10.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.142+0000 7fb70a7fc700 1 --2- 172.21.15.16:0/3400038860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb70c09e000 0x7fb70c11e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:10.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.142+0000 7fb70b7fe700 1 --2- 172.21.15.16:0/3400038860 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb70c09e410 0x7fb70c11ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:10.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.142+0000 7fb70affd700 1 -- 172.21.15.16:0/3400038860 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb70c09e410 msgr2=0x7fb70c11ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:10.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.142+0000 7fb70affd700 1 --2- 172.21.15.16:0/3400038860 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb70c09e410 0x7fb70c11ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:10.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.142+0000 7fb70affd700 1 -- 172.21.15.16:0/3400038860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb70c09e000 msgr2=0x7fb70c11e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:10.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.142+0000 7fb70affd700 1 --2- 172.21.15.16:0/3400038860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb70c09e000 0x7fb70c11e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:10.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.142+0000 7fb70affd700 1 -- 172.21.15.16:0/3400038860 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb70000b7f0 con 0x7fb70c09d6f0 2023-09-30T22:32:10.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.143+0000 7fb70affd700 1 --2- 172.21.15.16:0/3400038860 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb70c09d6f0 0x7fb70c11e280 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc00e930 tx=0x7fb6fc00ec40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:10.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.143+0000 7fb6f37fe700 1 -- 172.21.15.16:0/3400038860 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc013680 con 0x7fb70c09d6f0 2023-09-30T22:32:10.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.143+0000 7fb6f37fe700 1 -- 172.21.15.16:0/3400038860 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6fc0137e0 con 0x7fb70c09d6f0 2023-09-30T22:32:10.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.143+0000 7fb6f37fe700 1 -- 172.21.15.16:0/3400038860 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc013b70 con 0x7fb70c09d6f0 2023-09-30T22:32:10.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.143+0000 7fb7114f4700 1 -- 172.21.15.16:0/3400038860 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb70c1301e0 con 0x7fb70c09d6f0 2023-09-30T22:32:10.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.143+0000 7fb7114f4700 1 -- 172.21.15.16:0/3400038860 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb70c0a1ad0 con 0x7fb70c09d6f0 2023-09-30T22:32:10.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.144+0000 7fb6f37fe700 1 -- 172.21.15.16:0/3400038860 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb6fc013cd0 con 0x7fb70c09d6f0 2023-09-30T22:32:10.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.144+0000 7fb7114f4700 1 -- 172.21.15.16:0/3400038860 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb70c121920 con 0x7fb70c09d6f0 2023-09-30T22:32:10.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.144+0000 7fb6f37fe700 1 -- 172.21.15.16:0/3400038860 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fb6fc03f9f0 con 0x7fb70c09d6f0 2023-09-30T22:32:10.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.149+0000 7fb6f37fe700 1 -- 172.21.15.16:0/3400038860 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb70c121920 con 0x7fb70c09d6f0 2023-09-30T22:32:10.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.180+0000 7f5a2bf00700 1 Processor -- start 2023-09-30T22:32:10.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.180+0000 7f5a2bf00700 1 -- start start 2023-09-30T22:32:10.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.181+0000 7f5a2bf00700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a2409d6f0 0x7f5a2409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:10.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.181+0000 7f5a2bf00700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5a2409e410 0x7f5a24130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:10.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.181+0000 7f5a2bf00700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a2409f130 con 0x7f5a2409e000 2023-09-30T22:32:10.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.181+0000 7f5a2bf00700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a2409f520 con 0x7f5a2409e410 2023-09-30T22:32:10.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.181+0000 7f5a2bf00700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2409e930 con 0x7f5a2409d6f0 2023-09-30T22:32:10.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.181+0000 7f5a2a49d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5a2409e410 0x7f5a24130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:10.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.181+0000 7f5a2949b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5a2409e000 0x7f5a24130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48216/0 (socket says 172.21.15.16:48216) 2023-09-30T22:32:10.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.181+0000 7f5a2949b700 1 -- 172.21.15.16:0/2467192994 learned_addr learned my addr 172.21.15.16:0/2467192994 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:10.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.181+0000 7f5a2a49d700 1 -- 172.21.15.16:0/2467192994 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a2409d6f0 msgr2=0x7f5a2409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:10.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.181+0000 7f5a2a49d700 1 --2- 172.21.15.16:0/2467192994 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a2409d6f0 0x7f5a2409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:10.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.181+0000 7f5a2a49d700 1 -- 172.21.15.16:0/2467192994 >> v1:172.21.15.16:6789/0 conn(0x7f5a2409e000 legacy=0x7f5a24130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:10.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.181+0000 7f5a2a49d700 1 -- 172.21.15.16:0/2467192994 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a2409b2d0 con 0x7f5a2409e410 2023-09-30T22:32:10.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.181+0000 7f5a28c9a700 1 -- 172.21.15.16:0/2467192994 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2023607307 0 0) 0x7f5a2409f130 con 0x7f5a2409e000 2023-09-30T22:32:10.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.182+0000 7f5a2a49d700 1 --2- 172.21.15.16:0/2467192994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a2409e410 0x7f5a24130d70 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c009d90 tx=0x7f5a1c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b0ffd258129bac0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:10.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.182+0000 7f5a28c9a700 1 -- 172.21.15.16:0/2467192994 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c00e070 con 0x7f5a2409e410 2023-09-30T22:32:10.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.182+0000 7f5a28c9a700 1 -- 172.21.15.16:0/2467192994 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a1c00bbd0 con 0x7f5a2409e410 2023-09-30T22:32:10.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.182+0000 7f5a28c9a700 1 -- 172.21.15.16:0/2467192994 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c012430 con 0x7f5a2409e410 2023-09-30T22:32:10.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.183+0000 7f5a2bf00700 1 -- 172.21.15.16:0/2467192994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a2409e410 msgr2=0x7f5a24130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:10.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.183+0000 7f5a2bf00700 1 --2- 172.21.15.16:0/2467192994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a2409e410 0x7f5a24130d70 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c009d90 tx=0x7f5a1c00b330 comp rx=0 tx=0).stop 2023-09-30T22:32:10.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.183+0000 7f5a2bf00700 1 -- 172.21.15.16:0/2467192994 shutdown_connections 2023-09-30T22:32:10.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.183+0000 7f5a2bf00700 1 --2- 172.21.15.16:0/2467192994 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a2409e410 0x7f5a24130d70 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:10.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.183+0000 7f5a2bf00700 1 --2- 172.21.15.16:0/2467192994 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a2409d6f0 0x7f5a2409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:10.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.183+0000 7f5a2bf00700 1 -- 172.21.15.16:0/2467192994 >> 172.21.15.16:0/2467192994 conn(0x7f5a24066f50 msgr2=0x7f5a2409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:10.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.183+0000 7f5a2bf00700 1 -- 172.21.15.16:0/2467192994 shutdown_connections 2023-09-30T22:32:10.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.183+0000 7f5a2bf00700 1 -- 172.21.15.16:0/2467192994 wait complete. 2023-09-30T22:32:10.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.184+0000 7f5a2bf00700 1 Processor -- start 2023-09-30T22:32:10.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.184+0000 7f5a2bf00700 1 -- start start 2023-09-30T22:32:10.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.185+0000 7f5a2bf00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a2409d6f0 0x7f5a241c84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:10.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.185+0000 7f5a2bf00700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a2409e000 0x7f5a241c89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:10.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.185+0000 7f5a2bf00700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5a2409e410 0x7f5a241c8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:10.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.185+0000 7f5a2bf00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a24125770 con 0x7f5a2409d6f0 2023-09-30T22:32:10.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.185+0000 7f5a2bf00700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a24120ec0 con 0x7f5a2409e410 2023-09-30T22:32:10.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.185+0000 7f5a2bf00700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5a24125db0 con 0x7f5a2409e000 2023-09-30T22:32:10.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.185+0000 7f5a29c9c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a2409d6f0 0x7f5a241c84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:10.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.185+0000 7f5a2949b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a2409e000 0x7f5a241c89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:10.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.185+0000 7f5a2949b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a2409e000 0x7f5a241c89e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:55550/0 (socket says 172.21.15.16:55550) 2023-09-30T22:32:10.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.185+0000 7f5a29c9c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a2409d6f0 0x7f5a241c84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33262/0 (socket says 172.21.15.16:33262) 2023-09-30T22:32:10.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.185+0000 7f5a2949b700 1 -- 172.21.15.16:0/2962314479 learned_addr learned my addr 172.21.15.16:0/2962314479 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:10.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.186+0000 7f5a29c9c700 1 -- 172.21.15.16:0/2962314479 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a2409e000 msgr2=0x7f5a241c89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:10.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.186+0000 7f5a29c9c700 1 --2- 172.21.15.16:0/2962314479 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a2409e000 0x7f5a241c89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:10.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.186+0000 7f5a29c9c700 1 -- 172.21.15.16:0/2962314479 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5a2409e410 msgr2=0x7f5a241c8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:10.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.186+0000 7f5a29c9c700 1 --2- 172.21.15.16:0/2962314479 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5a2409e410 0x7f5a241c8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:10.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.186+0000 7f5a29c9c700 1 -- 172.21.15.16:0/2962314479 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a1c009a70 con 0x7f5a2409d6f0 2023-09-30T22:32:10.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.186+0000 7f5a2949b700 1 --2- 172.21.15.16:0/2962314479 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a2409e000 0x7f5a241c89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:10.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.186+0000 7f5a29c9c700 1 --2- 172.21.15.16:0/2962314479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a2409d6f0 0x7f5a241c84a0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f5a2000b410 tx=0x7f5a2000b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:10.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.186+0000 7f5a1a7fc700 1 -- 172.21.15.16:0/2962314479 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a20011830 con 0x7f5a2409d6f0 2023-09-30T22:32:10.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.186+0000 7f5a1a7fc700 1 -- 172.21.15.16:0/2962314479 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a2000be90 con 0x7f5a2409d6f0 2023-09-30T22:32:10.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.186+0000 7f5a2bf00700 1 -- 172.21.15.16:0/2962314479 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a2406e840 con 0x7f5a2409d6f0 2023-09-30T22:32:10.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.186+0000 7f5a1a7fc700 1 -- 172.21.15.16:0/2962314479 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a20011bd0 con 0x7f5a2409d6f0 2023-09-30T22:32:10.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.186+0000 7f5a2bf00700 1 -- 172.21.15.16:0/2962314479 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a24125ad0 con 0x7f5a2409d6f0 2023-09-30T22:32:10.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.187+0000 7f5a1a7fc700 1 -- 172.21.15.16:0/2962314479 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5a20011d30 con 0x7f5a2409d6f0 2023-09-30T22:32:10.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.187+0000 7f5a2bf00700 1 -- 172.21.15.16:0/2962314479 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a24074ea0 con 0x7f5a2409d6f0 2023-09-30T22:32:10.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.188+0000 7f5a1a7fc700 1 -- 172.21.15.16:0/2962314479 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f5a2003f630 con 0x7f5a2409d6f0 2023-09-30T22:32:10.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.192+0000 7f5a1a7fc700 1 -- 172.21.15.16:0/2962314479 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5a2000f2e0 con 0x7f5a2409d6f0 2023-09-30T22:32:10.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.236+0000 7fa78bc73700 1 -- 172.21.15.16:0/1146104409 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa78411dcd0 con 0x7fa78406fb20 2023-09-30T22:32:10.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.237+0000 7fa77a7fc700 1 -- 172.21.15.16:0/1146104409 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa78411dcd0 con 0x7fa78406fb20 2023-09-30T22:32:10.237 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:32:10.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.239+0000 7fa78bc73700 1 -- 172.21.15.16:0/1146104409 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa78406fb20 msgr2=0x7fa78409a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:10.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.239+0000 7fa78bc73700 1 --2- 172.21.15.16:0/1146104409 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa78406fb20 0x7fa78409a590 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa78000e9b0 tx=0x7fa78000ecc0 comp rx=0 tx=0).stop 2023-09-30T22:32:10.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.239+0000 7fa78bc73700 1 -- 172.21.15.16:0/1146104409 shutdown_connections 2023-09-30T22:32:10.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.239+0000 7fa78bc73700 1 --2- 172.21.15.16:0/1146104409 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa78406fb20 0x7fa78409a590 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:10.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.239+0000 7fa78bc73700 1 --2- 172.21.15.16:0/1146104409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa784130e50 0x7fa78409b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:10.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.239+0000 7fa78bc73700 1 --2- 172.21.15.16:0/1146104409 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa784130540 0x7fa78409aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:10.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.239+0000 7fa78bc73700 1 -- 172.21.15.16:0/1146104409 >> 172.21.15.16:0/1146104409 conn(0x7fa7840a2aa0 msgr2=0x7fa78412d550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:10.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.239+0000 7fa78bc73700 1 -- 172.21.15.16:0/1146104409 shutdown_connections 2023-09-30T22:32:10.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.239+0000 7fa78bc73700 1 -- 172.21.15.16:0/1146104409 wait complete. 2023-09-30T22:32:10.254 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:32:10.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.358+0000 7f376d21a700 1 -- 172.21.15.16:0/1213971608 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f376806b9e0 con 0x7f37681338c0 2023-09-30T22:32:10.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.359+0000 7f374f7fe700 1 -- 172.21.15.16:0/1213971608 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3758011360 con 0x7f37681338c0 2023-09-30T22:32:10.359 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:32:10.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.361+0000 7f376d21a700 1 -- 172.21.15.16:0/1213971608 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f37681338c0 msgr2=0x7f376812dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:10.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.361+0000 7f376d21a700 1 --2- 172.21.15.16:0/1213971608 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f37681338c0 0x7f376812dcf0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f375800c730 tx=0x7f375800ca40 comp rx=0 tx=0).stop 2023-09-30T22:32:10.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.361+0000 7fb7114f4700 1 -- 172.21.15.16:0/3400038860 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb70c121920 con 0x7fb70c09d6f0 2023-09-30T22:32:10.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.361+0000 7f376d21a700 1 -- 172.21.15.16:0/1213971608 shutdown_connections 2023-09-30T22:32:10.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.361+0000 7f376d21a700 1 --2- 172.21.15.16:0/1213971608 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f37681338c0 0x7f376812dcf0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:10.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.361+0000 7f376d21a700 1 --2- 172.21.15.16:0/1213971608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3768139150 0x7f3768118fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:10.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.361+0000 7f376d21a700 1 --2- 172.21.15.16:0/1213971608 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3768136830 0x7f376812e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:10.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.361+0000 7f376d21a700 1 -- 172.21.15.16:0/1213971608 >> 172.21.15.16:0/1213971608 conn(0x7f37680a2aa0 msgr2=0x7f376812f040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:10.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.361+0000 7f376d21a700 1 -- 172.21.15.16:0/1213971608 shutdown_connections 2023-09-30T22:32:10.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.361+0000 7f376d21a700 1 -- 172.21.15.16:0/1213971608 wait complete. 2023-09-30T22:32:10.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.362+0000 7fb6f37fe700 1 -- 172.21.15.16:0/3400038860 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb70c121920 con 0x7fb70c09d6f0 2023-09-30T22:32:10.364 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:32:10.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.364+0000 7fb7114f4700 1 -- 172.21.15.16:0/3400038860 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb70c09d6f0 msgr2=0x7fb70c11e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:10.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.364+0000 7fb7114f4700 1 --2- 172.21.15.16:0/3400038860 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb70c09d6f0 0x7fb70c11e280 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc00e930 tx=0x7fb6fc00ec40 comp rx=0 tx=0).stop 2023-09-30T22:32:10.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.364+0000 7fb7114f4700 1 -- 172.21.15.16:0/3400038860 shutdown_connections 2023-09-30T22:32:10.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.364+0000 7fb7114f4700 1 --2- 172.21.15.16:0/3400038860 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb70c09d6f0 0x7fb70c11e280 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:10.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.364+0000 7fb7114f4700 1 --2- 172.21.15.16:0/3400038860 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb70c09e410 0x7fb70c11ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:10.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.364+0000 7fb7114f4700 1 --2- 172.21.15.16:0/3400038860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb70c09e000 0x7fb70c11e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:10.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.364+0000 7fb7114f4700 1 -- 172.21.15.16:0/3400038860 >> 172.21.15.16:0/3400038860 conn(0x7fb70c066f50 msgr2=0x7fb70c06a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:10.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.365+0000 7fb7114f4700 1 -- 172.21.15.16:0/3400038860 shutdown_connections 2023-09-30T22:32:10.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.365+0000 7fb7114f4700 1 -- 172.21.15.16:0/3400038860 wait complete. 2023-09-30T22:32:10.377 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:32:10.379 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:32:10.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.402+0000 7f5a2bf00700 1 -- 172.21.15.16:0/2962314479 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5a2406c360 con 0x7f5a2409d6f0 2023-09-30T22:32:10.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.402+0000 7f5a1a7fc700 1 -- 172.21.15.16:0/2962314479 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5a20015020 con 0x7f5a2409d6f0 2023-09-30T22:32:10.402 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:32:10.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.404+0000 7f5a2bf00700 1 -- 172.21.15.16:0/2962314479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a2409d6f0 msgr2=0x7f5a241c84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:10.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.404+0000 7f5a2bf00700 1 --2- 172.21.15.16:0/2962314479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a2409d6f0 0x7f5a241c84a0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f5a2000b410 tx=0x7f5a2000b720 comp rx=0 tx=0).stop 2023-09-30T22:32:10.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.404+0000 7f5a2bf00700 1 -- 172.21.15.16:0/2962314479 shutdown_connections 2023-09-30T22:32:10.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.404+0000 7f5a2bf00700 1 --2- 172.21.15.16:0/2962314479 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5a2409e410 0x7f5a241c8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:10.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.404+0000 7f5a2bf00700 1 --2- 172.21.15.16:0/2962314479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a2409d6f0 0x7f5a241c84a0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:10.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.404+0000 7f5a2bf00700 1 --2- 172.21.15.16:0/2962314479 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a2409e000 0x7f5a241c89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:10.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.404+0000 7f5a2bf00700 1 -- 172.21.15.16:0/2962314479 >> 172.21.15.16:0/2962314479 conn(0x7f5a24066f50 msgr2=0x7f5a24119480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:10.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.404+0000 7f5a2bf00700 1 -- 172.21.15.16:0/2962314479 shutdown_connections 2023-09-30T22:32:10.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:10.404+0000 7f5a2bf00700 1 -- 172.21.15.16:0/2962314479 wait complete. 2023-09-30T22:32:10.419 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:32:11.254 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:32:11.378 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:32:11.379 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:32:11.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.401+0000 7f98cddbe700 1 Processor -- start 2023-09-30T22:32:11.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.401+0000 7f98cddbe700 1 -- start start 2023-09-30T22:32:11.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.402+0000 7f98cddbe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f98c8130540 0x7f98c8130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:11.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.402+0000 7f98cddbe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98c806fcf0 0x7f98c8138570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:11.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.402+0000 7f98cddbe700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98c80716a0 con 0x7f98c8130e50 2023-09-30T22:32:11.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.402+0000 7f98cddbe700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98c809a390 con 0x7f98c8130540 2023-09-30T22:32:11.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.402+0000 7f98cddbe700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f98c8071290 con 0x7f98c806fcf0 2023-09-30T22:32:11.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.402+0000 7f98c7fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98c806fcf0 0x7f98c8138570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:11.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.402+0000 7f98c77fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f98c8130540 0x7f98c8130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:11.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.402+0000 7f98c7fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98c806fcf0 0x7f98c8138570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:55566/0 (socket says 172.21.15.16:55566) 2023-09-30T22:32:11.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.402+0000 7f98c7fff700 1 -- 172.21.15.16:0/965739422 learned_addr learned my addr 172.21.15.16:0/965739422 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:11.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.403+0000 7f98c7fff700 1 -- 172.21.15.16:0/965739422 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f98c8130540 msgr2=0x7f98c8130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:11.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.403+0000 7f98c67fc700 1 -- 172.21.15.16:0/965739422 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3880205998 0 0) 0x7f98c80716a0 con 0x7f98c8130e50 2023-09-30T22:32:11.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.403+0000 7f98c7fff700 1 --2- 172.21.15.16:0/965739422 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f98c8130540 0x7f98c8130910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:11.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.403+0000 7f98c7fff700 1 -- 172.21.15.16:0/965739422 >> v1:172.21.15.16:6789/0 conn(0x7f98c8130e50 legacy=0x7f98c806f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:11.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.403+0000 7f98c7fff700 1 -- 172.21.15.16:0/965739422 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c8130340 con 0x7f98c806fcf0 2023-09-30T22:32:11.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.403+0000 7f98c7fff700 1 --2- 172.21.15.16:0/965739422 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98c806fcf0 0x7f98c8138570 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f98c0009e80 tx=0x7f98c000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3054428c16699557 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:11.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.403+0000 7f98c67fc700 1 -- 172.21.15.16:0/965739422 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c000e070 con 0x7f98c806fcf0 2023-09-30T22:32:11.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.403+0000 7f98c67fc700 1 -- 172.21.15.16:0/965739422 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98c000b910 con 0x7f98c806fcf0 2023-09-30T22:32:11.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.403+0000 7f98c67fc700 1 -- 172.21.15.16:0/965739422 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c000bcc0 con 0x7f98c806fcf0 2023-09-30T22:32:11.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.404+0000 7f98cddbe700 1 -- 172.21.15.16:0/965739422 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98c806fcf0 msgr2=0x7f98c8138570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:11.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.404+0000 7f98cddbe700 1 --2- 172.21.15.16:0/965739422 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98c806fcf0 0x7f98c8138570 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f98c0009e80 tx=0x7f98c000b330 comp rx=0 tx=0).stop 2023-09-30T22:32:11.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.404+0000 7f98cddbe700 1 -- 172.21.15.16:0/965739422 shutdown_connections 2023-09-30T22:32:11.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.404+0000 7f98cddbe700 1 --2- 172.21.15.16:0/965739422 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98c806fcf0 0x7f98c8138570 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:11.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.404+0000 7f98cddbe700 1 --2- 172.21.15.16:0/965739422 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f98c8130540 0x7f98c8130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:11.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.404+0000 7f98cddbe700 1 -- 172.21.15.16:0/965739422 >> 172.21.15.16:0/965739422 conn(0x7f98c80a2aa0 msgr2=0x7f98c80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:11.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.404+0000 7f98cddbe700 1 -- 172.21.15.16:0/965739422 shutdown_connections 2023-09-30T22:32:11.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.404+0000 7f98cddbe700 1 -- 172.21.15.16:0/965739422 wait complete. 2023-09-30T22:32:11.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.405+0000 7f98cddbe700 1 Processor -- start 2023-09-30T22:32:11.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.406+0000 7f98cddbe700 1 -- start start 2023-09-30T22:32:11.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.406+0000 7f98cddbe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98c806fcf0 0x7f98c81c3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:11.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.406+0000 7f98cddbe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f98c8130540 0x7f98c81c4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:11.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.406+0000 7f98cddbe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98c8130e50 0x7f98c81c03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:11.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.406+0000 7f98cddbe700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c811c8f0 con 0x7f98c8130e50 2023-09-30T22:32:11.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.406+0000 7f98cddbe700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98c811d2a0 con 0x7f98c8130540 2023-09-30T22:32:11.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.406+0000 7f98cddbe700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f98c811c000 con 0x7f98c806fcf0 2023-09-30T22:32:11.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.406+0000 7f98c77fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98c806fcf0 0x7f98c81c3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:11.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.406+0000 7f98c77fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98c806fcf0 0x7f98c81c3e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:55582/0 (socket says 172.21.15.16:55582) 2023-09-30T22:32:11.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.406+0000 7f98c77fe700 1 -- 172.21.15.16:0/3351581901 learned_addr learned my addr 172.21.15.16:0/3351581901 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:11.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.406+0000 7f98c6ffd700 1 --2- 172.21.15.16:0/3351581901 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f98c8130540 0x7f98c81c4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:11.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.407+0000 7f98c7fff700 1 --2- 172.21.15.16:0/3351581901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98c8130e50 0x7f98c81c03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:11.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.407+0000 7f98c77fe700 1 -- 172.21.15.16:0/3351581901 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f98c8130540 msgr2=0x7f98c81c4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:11.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.407+0000 7f98c77fe700 1 --2- 172.21.15.16:0/3351581901 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f98c8130540 0x7f98c81c4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:11.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.407+0000 7f98c77fe700 1 -- 172.21.15.16:0/3351581901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98c8130e50 msgr2=0x7f98c81c03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:11.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.407+0000 7f98c77fe700 1 --2- 172.21.15.16:0/3351581901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98c8130e50 0x7f98c81c03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:11.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.407+0000 7f98c77fe700 1 -- 172.21.15.16:0/3351581901 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c0009b30 con 0x7f98c806fcf0 2023-09-30T22:32:11.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.407+0000 7f98c77fe700 1 --2- 172.21.15.16:0/3351581901 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98c806fcf0 0x7f98c81c3e30 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f98b800e9b0 tx=0x7f98b800ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:11.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.407+0000 7f98affff700 1 -- 172.21.15.16:0/3351581901 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b800ee50 con 0x7f98c806fcf0 2023-09-30T22:32:11.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.407+0000 7f98cddbe700 1 -- 172.21.15.16:0/3351581901 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98c80a08a0 con 0x7f98c806fcf0 2023-09-30T22:32:11.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.407+0000 7f98cddbe700 1 -- 172.21.15.16:0/3351581901 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98c81c48b0 con 0x7f98c806fcf0 2023-09-30T22:32:11.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.408+0000 7f98affff700 1 -- 172.21.15.16:0/3351581901 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98b8017c60 con 0x7f98c806fcf0 2023-09-30T22:32:11.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.408+0000 7f98affff700 1 -- 172.21.15.16:0/3351581901 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b8005a20 con 0x7f98c806fcf0 2023-09-30T22:32:11.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.409+0000 7f98affff700 1 -- 172.21.15.16:0/3351581901 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f98b8005b80 con 0x7f98c806fcf0 2023-09-30T22:32:11.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.409+0000 7f98cddbe700 1 -- 172.21.15.16:0/3351581901 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98a80052e0 con 0x7f98c806fcf0 2023-09-30T22:32:11.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.409+0000 7f98affff700 1 -- 172.21.15.16:0/3351581901 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f98b8047920 con 0x7f98c806fcf0 2023-09-30T22:32:11.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.414+0000 7f98affff700 1 -- 172.21.15.16:0/3351581901 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f98b800cb60 con 0x7f98c806fcf0 2023-09-30T22:32:11.420 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:32:11.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.522+0000 7f55302d1700 1 Processor -- start 2023-09-30T22:32:11.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.522+0000 7f55302d1700 1 -- start start 2023-09-30T22:32:11.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.523+0000 7f24d9d46700 1 Processor -- start 2023-09-30T22:32:11.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.523+0000 7f24d9d46700 1 -- start start 2023-09-30T22:32:11.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.523+0000 7f55302d1700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f552809d6f0 0x7f552809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:11.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.523+0000 7f55302d1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f552809e000 0x7f55281339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:11.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.523+0000 7f55302d1700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f552809b120 con 0x7f552809e410 2023-09-30T22:32:11.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.523+0000 7f55302d1700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f552806b460 con 0x7f552809d6f0 2023-09-30T22:32:11.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.523+0000 7f55302d1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f552809a7e0 con 0x7f552809e000 2023-09-30T22:32:11.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.523+0000 7f24d9d46700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24d409d630 0x7f24d40a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:11.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.523+0000 7f24d9d46700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f24d4138d80 0x7f24d413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:11.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.523+0000 7f552d86c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f552809e000 0x7f55281339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:11.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.523+0000 7f552d86c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f552809e000 0x7f55281339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:55586/0 (socket says 172.21.15.16:55586) 2023-09-30T22:32:11.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.523+0000 7f552d86c700 1 -- 172.21.15.16:0/467577932 learned_addr learned my addr 172.21.15.16:0/467577932 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:11.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.524+0000 7f24d9d46700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24d40a2a70 con 0x7f24d40a0a30 2023-09-30T22:32:11.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.524+0000 7f24d9d46700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24d40a01d0 con 0x7f24d4138d80 2023-09-30T22:32:11.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.524+0000 7f24d9d46700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f24d409e2b0 con 0x7f24d409d630 2023-09-30T22:32:11.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.524+0000 7f24d3fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f24d4138d80 0x7f24d413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:11.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.524+0000 7f24d3fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f24d4138d80 0x7f24d413b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36904/0 (socket says 172.21.15.16:36904) 2023-09-30T22:32:11.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.524+0000 7f24d3fff700 1 -- 172.21.15.16:0/1825874130 learned_addr learned my addr 172.21.15.16:0/1825874130 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:11.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.524+0000 7f24d2ffd700 1 --2- 172.21.15.16:0/1825874130 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24d409d630 0x7f24d40a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:11.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.524+0000 7f552d86c700 1 -- 172.21.15.16:0/467577932 >> v2:172.21.15.16:3300/0 conn(0x7f552809d6f0 msgr2=0x7f552809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:11.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.524+0000 7f552d86c700 1 --2- 172.21.15.16:0/467577932 >> v2:172.21.15.16:3300/0 conn(0x7f552809d6f0 0x7f552809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:11.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.524+0000 7f552d86c700 1 -- 172.21.15.16:0/467577932 >> v1:172.21.15.16:6789/0 conn(0x7f552809e410 legacy=0x7f5528133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:11.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.524+0000 7f552d86c700 1 -- 172.21.15.16:0/467577932 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f552809a500 con 0x7f552809e000 2023-09-30T22:32:11.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.524+0000 7f24d3fff700 1 -- 172.21.15.16:0/1825874130 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24d409d630 msgr2=0x7f24d40a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:11.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.524+0000 7f24d3fff700 1 --2- 172.21.15.16:0/1825874130 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24d409d630 0x7f24d40a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:11.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.524+0000 7f24d3fff700 1 -- 172.21.15.16:0/1825874130 >> v1:172.21.15.16:6789/0 conn(0x7f24d40a0a30 legacy=0x7f24d40a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:11.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.524+0000 7f24d3fff700 1 -- 172.21.15.16:0/1825874130 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d409dc50 con 0x7f24d4138d80 2023-09-30T22:32:11.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.524+0000 7f24d3fff700 1 --2- 172.21.15.16:0/1825874130 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f24d4138d80 0x7f24d413b160 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f24cc009e80 tx=0x7f24cc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59b0d8b4ec04a5eb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:11.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.524+0000 7f552d86c700 1 --2- 172.21.15.16:0/467577932 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f552809e000 0x7f55281339f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5518009920 tx=0x7f5518009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=450ec0af82deceaa server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:11.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.524+0000 7f552d06b700 1 -- 172.21.15.16:0/467577932 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5518004230 con 0x7f552809e000 2023-09-30T22:32:11.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.525+0000 7f24d27fc700 1 -- 172.21.15.16:0/1825874130 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24cc00e070 con 0x7f24d4138d80 2023-09-30T22:32:11.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.525+0000 7f552d06b700 1 -- 172.21.15.16:0/467577932 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5518004390 con 0x7f552809e000 2023-09-30T22:32:11.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.525+0000 7f24d27fc700 1 -- 172.21.15.16:0/1825874130 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24cc003a20 con 0x7f24d4138d80 2023-09-30T22:32:11.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.525+0000 7f24d27fc700 1 -- 172.21.15.16:0/1825874130 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24cc003dd0 con 0x7f24d4138d80 2023-09-30T22:32:11.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.525+0000 7f552d06b700 1 -- 172.21.15.16:0/467577932 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5518009e70 con 0x7f552809e000 2023-09-30T22:32:11.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.525+0000 7f55302d1700 1 -- 172.21.15.16:0/467577932 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f552809e000 msgr2=0x7f55281339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:11.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.525+0000 7f55302d1700 1 --2- 172.21.15.16:0/467577932 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f552809e000 0x7f55281339f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5518009920 tx=0x7f5518009c30 comp rx=0 tx=0).stop 2023-09-30T22:32:11.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.526+0000 7f55302d1700 1 -- 172.21.15.16:0/467577932 shutdown_connections 2023-09-30T22:32:11.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.526+0000 7f55302d1700 1 --2- 172.21.15.16:0/467577932 >> v2:172.21.15.16:3300/0 conn(0x7f552809d6f0 0x7f552809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:11.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.526+0000 7f55302d1700 1 --2- 172.21.15.16:0/467577932 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f552809e000 0x7f55281339f0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:11.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.526+0000 7f55302d1700 1 -- 172.21.15.16:0/467577932 >> 172.21.15.16:0/467577932 conn(0x7f5528066f50 msgr2=0x7f552809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:11.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.526+0000 7f55302d1700 1 -- 172.21.15.16:0/467577932 shutdown_connections 2023-09-30T22:32:11.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.526+0000 7f24d9d46700 1 -- 172.21.15.16:0/1825874130 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f24d4138d80 msgr2=0x7f24d413b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:11.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.526+0000 7f24d9d46700 1 --2- 172.21.15.16:0/1825874130 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f24d4138d80 0x7f24d413b160 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f24cc009e80 tx=0x7f24cc00b330 comp rx=0 tx=0).stop 2023-09-30T22:32:11.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.526+0000 7f55302d1700 1 -- 172.21.15.16:0/467577932 wait complete. 2023-09-30T22:32:11.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.526+0000 7f24d9d46700 1 -- 172.21.15.16:0/1825874130 shutdown_connections 2023-09-30T22:32:11.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.526+0000 7f24d9d46700 1 --2- 172.21.15.16:0/1825874130 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f24d4138d80 0x7f24d413b160 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:11.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.526+0000 7f24d9d46700 1 --2- 172.21.15.16:0/1825874130 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24d409d630 0x7f24d40a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:11.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.526+0000 7f24d9d46700 1 -- 172.21.15.16:0/1825874130 >> 172.21.15.16:0/1825874130 conn(0x7f24d40a3050 msgr2=0x7f24d40a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:11.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.526+0000 7f24d9d46700 1 -- 172.21.15.16:0/1825874130 shutdown_connections 2023-09-30T22:32:11.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.526+0000 7f24d9d46700 1 -- 172.21.15.16:0/1825874130 wait complete. 2023-09-30T22:32:11.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.527+0000 7f55302d1700 1 Processor -- start 2023-09-30T22:32:11.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.527+0000 7f55302d1700 1 -- start start 2023-09-30T22:32:11.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.527+0000 7f24d9d46700 1 Processor -- start 2023-09-30T22:32:11.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.527+0000 7f55302d1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f552809d6f0 0x7f55281a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:11.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.527+0000 7f55302d1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f552809e000 0x7f55281a66b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:11.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.527+0000 7f55302d1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f552809e410 0x7f55281a6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:11.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.527+0000 7f55302d1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55281a7640 con 0x7f552809d6f0 2023-09-30T22:32:11.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.527+0000 7f55302d1700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55281a9c00 con 0x7f552809e000 2023-09-30T22:32:11.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.527+0000 7f55302d1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f55281b9590 con 0x7f552809e410 2023-09-30T22:32:11.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.528+0000 7f552e86e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f552809e410 0x7f55281a6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:11.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.528+0000 7f552e86e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f552809e410 0x7f55281a6bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:55602/0 (socket says 172.21.15.16:55602) 2023-09-30T22:32:11.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.528+0000 7f552e86e700 1 -- 172.21.15.16:0/819636164 learned_addr learned my addr 172.21.15.16:0/819636164 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:11.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.528+0000 7f552d86c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f552809e000 0x7f55281a66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:11.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.528+0000 7f552e06d700 1 --2- 172.21.15.16:0/819636164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f552809d6f0 0x7f55281a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:11.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.528+0000 7f552d86c700 1 -- 172.21.15.16:0/819636164 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f552809e410 msgr2=0x7f55281a6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:11.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.528+0000 7f552d86c700 1 --2- 172.21.15.16:0/819636164 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f552809e410 0x7f55281a6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:11.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.528+0000 7f552d86c700 1 -- 172.21.15.16:0/819636164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f552809d6f0 msgr2=0x7f55281a6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:11.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.528+0000 7f552d86c700 1 --2- 172.21.15.16:0/819636164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f552809d6f0 0x7f55281a6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:11.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.528+0000 7f552d86c700 1 -- 172.21.15.16:0/819636164 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5518009550 con 0x7f552809e000 2023-09-30T22:32:11.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.528+0000 7f552e06d700 1 --2- 172.21.15.16:0/819636164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f552809d6f0 0x7f55281a6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:32:11.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.528+0000 7f552e86e700 1 --2- 172.21.15.16:0/819636164 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f552809e410 0x7f55281a6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:32:11.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.528+0000 7f24d9d46700 1 -- start start 2023-09-30T22:32:11.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.528+0000 7f24d9d46700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24d409d630 0x7f24d4133af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:11.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.528+0000 7f24d9d46700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24d40a0a30 0x7f24d4134030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:11.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.528+0000 7f552d86c700 1 --2- 172.21.15.16:0/819636164 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f552809e000 0x7f55281a66b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f552809ac00 tx=0x7f5518004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:11.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.528+0000 7f24d9d46700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f24d4138d80 0x7f24d4134570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:11.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.528+0000 7f24d9d46700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d4137330 con 0x7f24d409d630 2023-09-30T22:32:11.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.528+0000 7f24d9d46700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24d41381b0 con 0x7f24d4138d80 2023-09-30T22:32:11.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.528+0000 7f24d9d46700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f24d4134f50 con 0x7f24d40a0a30 2023-09-30T22:32:11.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.529+0000 7f24d2ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24d40a0a30 0x7f24d4134030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:11.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.529+0000 7f24d2ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24d40a0a30 0x7f24d4134030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:55604/0 (socket says 172.21.15.16:55604) 2023-09-30T22:32:11.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.529+0000 7f24d2ffd700 1 -- 172.21.15.16:0/41682696 learned_addr learned my addr 172.21.15.16:0/41682696 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:11.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.529+0000 7f24d37fe700 1 --2- 172.21.15.16:0/41682696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24d409d630 0x7f24d4133af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:11.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.529+0000 7f24d2ffd700 1 -- 172.21.15.16:0/41682696 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f24d4138d80 msgr2=0x7f24d4134570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:11.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.529+0000 7f24d2ffd700 1 --2- 172.21.15.16:0/41682696 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f24d4138d80 0x7f24d4134570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:11.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.529+0000 7f24d2ffd700 1 -- 172.21.15.16:0/41682696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24d409d630 msgr2=0x7f24d4133af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:11.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.529+0000 7f24d2ffd700 1 --2- 172.21.15.16:0/41682696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24d409d630 0x7f24d4133af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:11.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.529+0000 7f24d2ffd700 1 -- 172.21.15.16:0/41682696 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24cc009b30 con 0x7f24d40a0a30 2023-09-30T22:32:11.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.529+0000 7f551effd700 1 -- 172.21.15.16:0/819636164 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551800bd00 con 0x7f552809e000 2023-09-30T22:32:11.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.529+0000 7f24d2ffd700 1 --2- 172.21.15.16:0/41682696 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24d40a0a30 0x7f24d4134030 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f24c800e840 tx=0x7f24c800eb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:11.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.529+0000 7f551effd700 1 -- 172.21.15.16:0/819636164 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f551801f070 con 0x7f552809e000 2023-09-30T22:32:11.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.529+0000 7f551effd700 1 -- 172.21.15.16:0/819636164 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55180038c0 con 0x7f552809e000 2023-09-30T22:32:11.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.530+0000 7f55302d1700 1 -- 172.21.15.16:0/819636164 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55280a3040 con 0x7f552809e000 2023-09-30T22:32:11.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.530+0000 7f55302d1700 1 -- 172.21.15.16:0/819636164 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55281a8af0 con 0x7f552809e000 2023-09-30T22:32:11.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.530+0000 7f24bbfff700 1 -- 172.21.15.16:0/41682696 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c8013610 con 0x7f24d40a0a30 2023-09-30T22:32:11.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.530+0000 7f24bbfff700 1 -- 172.21.15.16:0/41682696 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24c8013770 con 0x7f24d40a0a30 2023-09-30T22:32:11.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.530+0000 7f24bbfff700 1 -- 172.21.15.16:0/41682696 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c8013b20 con 0x7f24d40a0a30 2023-09-30T22:32:11.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.531+0000 7f551effd700 1 -- 172.21.15.16:0/819636164 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5518021030 con 0x7f552809e000 2023-09-30T22:32:11.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.531+0000 7f55302d1700 1 -- 172.21.15.16:0/819636164 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5528074ea0 con 0x7f552809e000 2023-09-30T22:32:11.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.531+0000 7f24d9d46700 1 -- 172.21.15.16:0/41682696 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24d4137590 con 0x7f24d40a0a30 2023-09-30T22:32:11.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.531+0000 7f24d9d46700 1 -- 172.21.15.16:0/41682696 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24d40a0410 con 0x7f24d40a0a30 2023-09-30T22:32:11.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.532+0000 7f24bbfff700 1 -- 172.21.15.16:0/41682696 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f24c8013c80 con 0x7f24d40a0a30 2023-09-30T22:32:11.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.532+0000 7f24d9d46700 1 -- 172.21.15.16:0/41682696 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24d40747f0 con 0x7f24d40a0a30 2023-09-30T22:32:11.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.533+0000 7f24bbfff700 1 -- 172.21.15.16:0/41682696 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f24c803f470 con 0x7f24d40a0a30 2023-09-30T22:32:11.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.532+0000 7f551effd700 1 -- 172.21.15.16:0/819636164 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f5518040d50 con 0x7f552809e000 2023-09-30T22:32:11.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.537+0000 7f551effd700 1 -- 172.21.15.16:0/819636164 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5518003a20 con 0x7f552809e000 2023-09-30T22:32:11.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.538+0000 7f24bbfff700 1 -- 172.21.15.16:0/41682696 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f24c8021080 con 0x7f24d40a0a30 2023-09-30T22:32:11.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.572+0000 7f20a79bf700 1 Processor -- start 2023-09-30T22:32:11.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.572+0000 7f20a79bf700 1 -- start start 2023-09-30T22:32:11.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.573+0000 7f20a79bf700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f20a009d6f0 0x7f20a009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:11.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.573+0000 7f20a79bf700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f20a009e410 0x7f20a0130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:11.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.573+0000 7f20a79bf700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20a009f130 con 0x7f20a009e000 2023-09-30T22:32:11.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.573+0000 7f20a79bf700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20a009f520 con 0x7f20a009d6f0 2023-09-30T22:32:11.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.573+0000 7f20a79bf700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20a009e930 con 0x7f20a009e410 2023-09-30T22:32:11.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.573+0000 7f20a5f5c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f20a009e410 0x7f20a0130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:11.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.574+0000 7f20a5f5c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f20a009e410 0x7f20a0130d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36926/0 (socket says 172.21.15.16:36926) 2023-09-30T22:32:11.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.574+0000 7f20a575b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f20a009d6f0 0x7f20a009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:11.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.574+0000 7f20a5f5c700 1 -- 172.21.15.16:0/2759798049 learned_addr learned my addr 172.21.15.16:0/2759798049 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:11.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.574+0000 7f20a5f5c700 1 -- 172.21.15.16:0/2759798049 >> v2:172.21.15.16:3300/0 conn(0x7f20a009d6f0 msgr2=0x7f20a009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:11.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.574+0000 7f20a5f5c700 1 --2- 172.21.15.16:0/2759798049 >> v2:172.21.15.16:3300/0 conn(0x7f20a009d6f0 0x7f20a009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:11.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.574+0000 7f20a5f5c700 1 -- 172.21.15.16:0/2759798049 >> v1:172.21.15.16:6789/0 conn(0x7f20a009e000 legacy=0x7f20a0130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:11.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.574+0000 7f20a5f5c700 1 -- 172.21.15.16:0/2759798049 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20a009b2d0 con 0x7f20a009e410 2023-09-30T22:32:11.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.574+0000 7f20a5f5c700 1 --2- 172.21.15.16:0/2759798049 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f20a009e410 0x7f20a0130d70 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f209800d150 tx=0x7f209800d460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95a87a6ef95c9624 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:11.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.574+0000 7f2097fff700 1 -- 172.21.15.16:0/2759798049 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2098010070 con 0x7f20a009e410 2023-09-30T22:32:11.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.574+0000 7f2097fff700 1 -- 172.21.15.16:0/2759798049 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f209800dd10 con 0x7f20a009e410 2023-09-30T22:32:11.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.574+0000 7f2097fff700 1 -- 172.21.15.16:0/2759798049 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209800b310 con 0x7f20a009e410 2023-09-30T22:32:11.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.575+0000 7f20a79bf700 1 -- 172.21.15.16:0/2759798049 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f20a009e410 msgr2=0x7f20a0130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:11.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.575+0000 7f20a79bf700 1 --2- 172.21.15.16:0/2759798049 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f20a009e410 0x7f20a0130d70 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f209800d150 tx=0x7f209800d460 comp rx=0 tx=0).stop 2023-09-30T22:32:11.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.575+0000 7f20a79bf700 1 -- 172.21.15.16:0/2759798049 shutdown_connections 2023-09-30T22:32:11.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.575+0000 7f20a79bf700 1 --2- 172.21.15.16:0/2759798049 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f20a009e410 0x7f20a0130d70 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:11.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.575+0000 7f20a79bf700 1 --2- 172.21.15.16:0/2759798049 >> v2:172.21.15.16:3300/0 conn(0x7f20a009d6f0 0x7f20a009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:11.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.575+0000 7f20a79bf700 1 -- 172.21.15.16:0/2759798049 >> 172.21.15.16:0/2759798049 conn(0x7f20a0066f50 msgr2=0x7f20a009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:11.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.575+0000 7f20a79bf700 1 -- 172.21.15.16:0/2759798049 shutdown_connections 2023-09-30T22:32:11.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.575+0000 7f20a79bf700 1 -- 172.21.15.16:0/2759798049 wait complete. 2023-09-30T22:32:11.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.577+0000 7f20a79bf700 1 Processor -- start 2023-09-30T22:32:11.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.577+0000 7f20a79bf700 1 -- start start 2023-09-30T22:32:11.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.577+0000 7f20a79bf700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f20a009d6f0 0x7f20a011de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:11.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.577+0000 7f20a79bf700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f20a009e000 0x7f20a011e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:11.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.578+0000 7f20a79bf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20a009e410 0x7f20a011e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:11.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.578+0000 7f20a79bf700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a0121940 con 0x7f20a009e410 2023-09-30T22:32:11.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.578+0000 7f20a79bf700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20a0120050 con 0x7f20a009e000 2023-09-30T22:32:11.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.578+0000 7f20a79bf700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f20a0120d10 con 0x7f20a009d6f0 2023-09-30T22:32:11.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.578+0000 7f20a4f5a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f20a009e000 0x7f20a011e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:11.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.578+0000 7f20a4f5a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f20a009e000 0x7f20a011e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36934/0 (socket says 172.21.15.16:36934) 2023-09-30T22:32:11.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.578+0000 7f20a4f5a700 1 -- 172.21.15.16:0/2068083718 learned_addr learned my addr 172.21.15.16:0/2068083718 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:11.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.578+0000 7f20a5f5c700 1 --2- 172.21.15.16:0/2068083718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20a009e410 0x7f20a011e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:11.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.578+0000 7f20a575b700 1 --2- 172.21.15.16:0/2068083718 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f20a009d6f0 0x7f20a011de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:11.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.578+0000 7f20a4f5a700 1 -- 172.21.15.16:0/2068083718 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f20a009d6f0 msgr2=0x7f20a011de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:11.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.578+0000 7f20a4f5a700 1 --2- 172.21.15.16:0/2068083718 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f20a009d6f0 0x7f20a011de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:11.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.578+0000 7f20a4f5a700 1 -- 172.21.15.16:0/2068083718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20a009e410 msgr2=0x7f20a011e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:11.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.578+0000 7f20a4f5a700 1 --2- 172.21.15.16:0/2068083718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20a009e410 0x7f20a011e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:11.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.578+0000 7f20a4f5a700 1 -- 172.21.15.16:0/2068083718 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f209800bd40 con 0x7f20a009e000 2023-09-30T22:32:11.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.578+0000 7f20a4f5a700 1 --2- 172.21.15.16:0/2068083718 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f20a009e000 0x7f20a011e360 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f209000c510 tx=0x7f209000c820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:11.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.579+0000 7f2095ffb700 1 -- 172.21.15.16:0/2068083718 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20900052d0 con 0x7f20a009e000 2023-09-30T22:32:11.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.579+0000 7f2095ffb700 1 -- 172.21.15.16:0/2068083718 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2090005430 con 0x7f20a009e000 2023-09-30T22:32:11.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.579+0000 7f20a79bf700 1 -- 172.21.15.16:0/2068083718 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20a011f280 con 0x7f20a009e000 2023-09-30T22:32:11.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.579+0000 7f2095ffb700 1 -- 172.21.15.16:0/2068083718 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20900038c0 con 0x7f20a009e000 2023-09-30T22:32:11.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.579+0000 7f20a79bf700 1 -- 172.21.15.16:0/2068083718 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20a009c700 con 0x7f20a009e000 2023-09-30T22:32:11.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.580+0000 7f2095ffb700 1 -- 172.21.15.16:0/2068083718 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2090003a20 con 0x7f20a009e000 2023-09-30T22:32:11.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.580+0000 7f20a79bf700 1 -- 172.21.15.16:0/2068083718 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20a01214c0 con 0x7f20a009e000 2023-09-30T22:32:11.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.580+0000 7f2095ffb700 1 -- 172.21.15.16:0/2068083718 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f209003ea70 con 0x7f20a009e000 2023-09-30T22:32:11.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.589+0000 7f2095ffb700 1 -- 172.21.15.16:0/2068083718 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f20a01214c0 con 0x7f20a009e000 2023-09-30T22:32:11.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.631+0000 7f98cddbe700 1 -- 172.21.15.16:0/3351581901 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f98a8005590 con 0x7f98c806fcf0 2023-09-30T22:32:11.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.632+0000 7f98affff700 1 -- 172.21.15.16:0/3351581901 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f98b80283b0 con 0x7f98c806fcf0 2023-09-30T22:32:11.633 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:32:11.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.635+0000 7f98cddbe700 1 -- 172.21.15.16:0/3351581901 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98c806fcf0 msgr2=0x7f98c81c3e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:11.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.635+0000 7f98cddbe700 1 --2- 172.21.15.16:0/3351581901 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98c806fcf0 0x7f98c81c3e30 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f98b800e9b0 tx=0x7f98b800ecc0 comp rx=0 tx=0).stop 2023-09-30T22:32:11.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.635+0000 7f98cddbe700 1 -- 172.21.15.16:0/3351581901 shutdown_connections 2023-09-30T22:32:11.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.635+0000 7f98cddbe700 1 --2- 172.21.15.16:0/3351581901 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98c8130e50 0x7f98c81c03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:11.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.635+0000 7f98cddbe700 1 --2- 172.21.15.16:0/3351581901 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98c806fcf0 0x7f98c81c3e30 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:11.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.635+0000 7f98cddbe700 1 --2- 172.21.15.16:0/3351581901 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f98c8130540 0x7f98c81c4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:11.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.635+0000 7f98cddbe700 1 -- 172.21.15.16:0/3351581901 >> 172.21.15.16:0/3351581901 conn(0x7f98c80a2aa0 msgr2=0x7f98c812d570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:11.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.635+0000 7f98cddbe700 1 -- 172.21.15.16:0/3351581901 shutdown_connections 2023-09-30T22:32:11.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.635+0000 7f98cddbe700 1 -- 172.21.15.16:0/3351581901 wait complete. 2023-09-30T22:32:11.655 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:32:11.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.752+0000 7f55302d1700 1 -- 172.21.15.16:0/819636164 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f5528074ea0 con 0x7f552809e000 2023-09-30T22:32:11.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.753+0000 7f551effd700 1 -- 172.21.15.16:0/819636164 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f551801b440 con 0x7f552809e000 2023-09-30T22:32:11.753 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:32:11.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.755+0000 7f55302d1700 1 -- 172.21.15.16:0/819636164 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f552809e000 msgr2=0x7f55281a66b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:11.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.755+0000 7f55302d1700 1 --2- 172.21.15.16:0/819636164 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f552809e000 0x7f55281a66b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f552809ac00 tx=0x7f5518004750 comp rx=0 tx=0).stop 2023-09-30T22:32:11.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.755+0000 7f55302d1700 1 -- 172.21.15.16:0/819636164 shutdown_connections 2023-09-30T22:32:11.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.755+0000 7f55302d1700 1 --2- 172.21.15.16:0/819636164 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f552809e410 0x7f55281a6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:11.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.755+0000 7f55302d1700 1 --2- 172.21.15.16:0/819636164 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f552809d6f0 0x7f55281a6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:11.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.755+0000 7f55302d1700 1 --2- 172.21.15.16:0/819636164 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f552809e000 0x7f55281a66b0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:11.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.755+0000 7f55302d1700 1 -- 172.21.15.16:0/819636164 >> 172.21.15.16:0/819636164 conn(0x7f5528066f50 msgr2=0x7f55281baf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:11.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.755+0000 7f55302d1700 1 -- 172.21.15.16:0/819636164 shutdown_connections 2023-09-30T22:32:11.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.755+0000 7f55302d1700 1 -- 172.21.15.16:0/819636164 wait complete. 2023-09-30T22:32:11.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.760+0000 7f24d9d46700 1 -- 172.21.15.16:0/41682696 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f24d4072980 con 0x7f24d40a0a30 2023-09-30T22:32:11.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.760+0000 7f24bbfff700 1 -- 172.21.15.16:0/41682696 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f24c8012370 con 0x7f24d40a0a30 2023-09-30T22:32:11.761 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:32:11.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.763+0000 7f24d9d46700 1 -- 172.21.15.16:0/41682696 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24d40a0a30 msgr2=0x7f24d4134030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:11.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.763+0000 7f24d9d46700 1 --2- 172.21.15.16:0/41682696 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24d40a0a30 0x7f24d4134030 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f24c800e840 tx=0x7f24c800eb50 comp rx=0 tx=0).stop 2023-09-30T22:32:11.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.763+0000 7f24d9d46700 1 -- 172.21.15.16:0/41682696 shutdown_connections 2023-09-30T22:32:11.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.763+0000 7f24d9d46700 1 --2- 172.21.15.16:0/41682696 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f24d4138d80 0x7f24d4134570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:11.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.763+0000 7f24d9d46700 1 --2- 172.21.15.16:0/41682696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24d409d630 0x7f24d4133af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:11.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.763+0000 7f24d9d46700 1 --2- 172.21.15.16:0/41682696 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24d40a0a30 0x7f24d4134030 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:11.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.763+0000 7f24d9d46700 1 -- 172.21.15.16:0/41682696 >> 172.21.15.16:0/41682696 conn(0x7f24d40a3050 msgr2=0x7f24d409ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:11.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.763+0000 7f24d9d46700 1 -- 172.21.15.16:0/41682696 shutdown_connections 2023-09-30T22:32:11.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.763+0000 7f24d9d46700 1 -- 172.21.15.16:0/41682696 wait complete. 2023-09-30T22:32:11.770 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:32:11.778 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:32:11.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.810+0000 7f20a79bf700 1 -- 172.21.15.16:0/2068083718 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f20a01214c0 con 0x7f20a009e000 2023-09-30T22:32:11.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.811+0000 7f2095ffb700 1 -- 172.21.15.16:0/2068083718 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f20a01214c0 con 0x7f20a009e000 2023-09-30T22:32:11.812 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:32:11.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.813+0000 7f20a79bf700 1 -- 172.21.15.16:0/2068083718 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f20a009e000 msgr2=0x7f20a011e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:11.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.813+0000 7f20a79bf700 1 --2- 172.21.15.16:0/2068083718 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f20a009e000 0x7f20a011e360 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f209000c510 tx=0x7f209000c820 comp rx=0 tx=0).stop 2023-09-30T22:32:11.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.813+0000 7f20a79bf700 1 -- 172.21.15.16:0/2068083718 shutdown_connections 2023-09-30T22:32:11.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.813+0000 7f20a79bf700 1 --2- 172.21.15.16:0/2068083718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20a009e410 0x7f20a011e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:11.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.813+0000 7f20a79bf700 1 --2- 172.21.15.16:0/2068083718 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f20a009d6f0 0x7f20a011de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:11.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.813+0000 7f20a79bf700 1 --2- 172.21.15.16:0/2068083718 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f20a009e000 0x7f20a011e360 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:11.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.813+0000 7f20a79bf700 1 -- 172.21.15.16:0/2068083718 >> 172.21.15.16:0/2068083718 conn(0x7f20a0066f50 msgr2=0x7f20a0070fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:11.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.813+0000 7f20a79bf700 1 -- 172.21.15.16:0/2068083718 shutdown_connections 2023-09-30T22:32:11.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:11.813+0000 7f20a79bf700 1 -- 172.21.15.16:0/2068083718 wait complete. 2023-09-30T22:32:11.827 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:32:12.656 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:32:12.772 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:32:12.779 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:32:12.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.803+0000 7f1ed3610700 1 Processor -- start 2023-09-30T22:32:12.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.803+0000 7f1ed3610700 1 -- start start 2023-09-30T22:32:12.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.803+0000 7f1ed3610700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1ecc0a0a30 0x7f1ecc0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:12.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.804+0000 7f1ed3610700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ecc138d80 0x7f1ecc13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:12.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.804+0000 7f1ed3610700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ecc0a2a70 con 0x7f1ecc09d630 2023-09-30T22:32:12.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.804+0000 7f1ed3610700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ecc0a01d0 con 0x7f1ecc0a0a30 2023-09-30T22:32:12.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.804+0000 7f1ed3610700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ecc09e2b0 con 0x7f1ecc138d80 2023-09-30T22:32:12.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.804+0000 7f1ed1bad700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ecc138d80 0x7f1ecc13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:12.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.804+0000 7f1ed13ac700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1ecc0a0a30 0x7f1ecc0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:12.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.804+0000 7f1ed13ac700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1ecc0a0a30 0x7f1ecc0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33314/0 (socket says 172.21.15.16:33314) 2023-09-30T22:32:12.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.804+0000 7f1ed1bad700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ecc138d80 0x7f1ecc13b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36948/0 (socket says 172.21.15.16:36948) 2023-09-30T22:32:12.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.804+0000 7f1ed13ac700 1 -- 172.21.15.16:0/148858615 learned_addr learned my addr 172.21.15.16:0/148858615 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:12.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.804+0000 7f1ed0bab700 1 --1- 172.21.15.16:0/148858615 >> v1:172.21.15.16:6789/0 conn(0x7f1ecc09d630 0x7f1ecc0a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48254/0 (socket says 172.21.15.16:48254) 2023-09-30T22:32:12.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.804+0000 7f1ed1bad700 1 -- 172.21.15.16:0/148858615 >> v2:172.21.15.16:3300/0 conn(0x7f1ecc0a0a30 msgr2=0x7f1ecc0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:12.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.804+0000 7f1ed1bad700 1 --2- 172.21.15.16:0/148858615 >> v2:172.21.15.16:3300/0 conn(0x7f1ecc0a0a30 0x7f1ecc0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:12.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.804+0000 7f1ed1bad700 1 -- 172.21.15.16:0/148858615 >> v1:172.21.15.16:6789/0 conn(0x7f1ecc09d630 legacy=0x7f1ecc0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:12.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.804+0000 7f1ed1bad700 1 -- 172.21.15.16:0/148858615 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ecc09dc50 con 0x7f1ecc138d80 2023-09-30T22:32:12.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.804+0000 7f1ed13ac700 1 --2- 172.21.15.16:0/148858615 >> v2:172.21.15.16:3300/0 conn(0x7f1ecc0a0a30 0x7f1ecc0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:12.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.804+0000 7f1ed1bad700 1 --2- 172.21.15.16:0/148858615 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ecc138d80 0x7f1ecc13b160 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4009d90 tx=0x7f1ec400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b76ab828a13b5d2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:12.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.805+0000 7f1ec3fff700 1 -- 172.21.15.16:0/148858615 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec400e070 con 0x7f1ecc138d80 2023-09-30T22:32:12.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.805+0000 7f1ec3fff700 1 -- 172.21.15.16:0/148858615 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ec400bae0 con 0x7f1ecc138d80 2023-09-30T22:32:12.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.805+0000 7f1ec3fff700 1 -- 172.21.15.16:0/148858615 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec400be90 con 0x7f1ecc138d80 2023-09-30T22:32:12.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.806+0000 7f1ed3610700 1 -- 172.21.15.16:0/148858615 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ecc138d80 msgr2=0x7f1ecc13b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:12.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.806+0000 7f1ed3610700 1 --2- 172.21.15.16:0/148858615 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ecc138d80 0x7f1ecc13b160 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4009d90 tx=0x7f1ec400b330 comp rx=0 tx=0).stop 2023-09-30T22:32:12.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.806+0000 7f1ed3610700 1 -- 172.21.15.16:0/148858615 shutdown_connections 2023-09-30T22:32:12.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.806+0000 7f1ed3610700 1 --2- 172.21.15.16:0/148858615 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ecc138d80 0x7f1ecc13b160 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:12.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.806+0000 7f1ed3610700 1 --2- 172.21.15.16:0/148858615 >> v2:172.21.15.16:3300/0 conn(0x7f1ecc0a0a30 0x7f1ecc0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:12.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.806+0000 7f1ed3610700 1 -- 172.21.15.16:0/148858615 >> 172.21.15.16:0/148858615 conn(0x7f1ecc0a3050 msgr2=0x7f1ecc0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:12.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.807+0000 7f1ed3610700 1 -- 172.21.15.16:0/148858615 shutdown_connections 2023-09-30T22:32:12.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.807+0000 7f1ed3610700 1 -- 172.21.15.16:0/148858615 wait complete. 2023-09-30T22:32:12.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.808+0000 7f1ed3610700 1 Processor -- start 2023-09-30T22:32:12.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.808+0000 7f1ed3610700 1 -- start start 2023-09-30T22:32:12.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.808+0000 7f1ed3610700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ecc09d630 0x7f1ecc120640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:12.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.808+0000 7f1ed3610700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ecc0a0a30 0x7f1ecc120b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:12.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.808+0000 7f1ed3610700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ecc138d80 0x7f1ecc1210c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:12.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.808+0000 7f1ed3610700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ecc124160 con 0x7f1ecc138d80 2023-09-30T22:32:12.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.808+0000 7f1ed3610700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ecc123510 con 0x7f1ecc0a0a30 2023-09-30T22:32:12.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.808+0000 7f1ed3610700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1ecc138b40 con 0x7f1ecc09d630 2023-09-30T22:32:12.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.808+0000 7f1ed13ac700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ecc09d630 0x7f1ecc120640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:12.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.808+0000 7f1ed13ac700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ecc09d630 0x7f1ecc120640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:55634/0 (socket says 172.21.15.16:55634) 2023-09-30T22:32:12.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.808+0000 7f1ed13ac700 1 -- 172.21.15.16:0/461600489 learned_addr learned my addr 172.21.15.16:0/461600489 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:12.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.808+0000 7f1ed0bab700 1 --2- 172.21.15.16:0/461600489 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ecc0a0a30 0x7f1ecc120b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:12.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.809+0000 7f1ed1bad700 1 --2- 172.21.15.16:0/461600489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ecc138d80 0x7f1ecc1210c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:12.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.809+0000 7f1ed13ac700 1 -- 172.21.15.16:0/461600489 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ecc0a0a30 msgr2=0x7f1ecc120b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:12.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.809+0000 7f1ed13ac700 1 --2- 172.21.15.16:0/461600489 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ecc0a0a30 0x7f1ecc120b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:12.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.809+0000 7f1ed13ac700 1 -- 172.21.15.16:0/461600489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ecc138d80 msgr2=0x7f1ecc1210c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:12.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.809+0000 7f1ed13ac700 1 --2- 172.21.15.16:0/461600489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ecc138d80 0x7f1ecc1210c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:12.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.809+0000 7f1ed13ac700 1 -- 172.21.15.16:0/461600489 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec8009550 con 0x7f1ecc09d630 2023-09-30T22:32:12.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.809+0000 7f1ed13ac700 1 --2- 172.21.15.16:0/461600489 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ecc09d630 0x7f1ecc120640 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1ec800ebe0 tx=0x7f1ec800eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:12.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.809+0000 7f1ec1ffb700 1 -- 172.21.15.16:0/461600489 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec800c850 con 0x7f1ecc09d630 2023-09-30T22:32:12.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.809+0000 7f1ec1ffb700 1 -- 172.21.15.16:0/461600489 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ec800c9b0 con 0x7f1ecc09d630 2023-09-30T22:32:12.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.809+0000 7f1ed3610700 1 -- 172.21.15.16:0/461600489 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ec4009a70 con 0x7f1ecc09d630 2023-09-30T22:32:12.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.809+0000 7f1ec1ffb700 1 -- 172.21.15.16:0/461600489 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec800cd40 con 0x7f1ecc09d630 2023-09-30T22:32:12.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.810+0000 7f1ed3610700 1 -- 172.21.15.16:0/461600489 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ecc1248f0 con 0x7f1ecc09d630 2023-09-30T22:32:12.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.810+0000 7f1ec1ffb700 1 -- 172.21.15.16:0/461600489 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1ec8013470 con 0x7f1ecc09d630 2023-09-30T22:32:12.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.811+0000 7f1ed3610700 1 -- 172.21.15.16:0/461600489 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ecc0747f0 con 0x7f1ecc09d630 2023-09-30T22:32:12.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.811+0000 7f1ec1ffb700 1 -- 172.21.15.16:0/461600489 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f1ec803f8c0 con 0x7f1ecc09d630 2023-09-30T22:32:12.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.816+0000 7f1ec1ffb700 1 -- 172.21.15.16:0/461600489 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1ec80232f0 con 0x7f1ecc09d630 2023-09-30T22:32:12.828 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:32:12.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.919+0000 7f5d4ef30700 1 Processor -- start 2023-09-30T22:32:12.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.919+0000 7f5d4ef30700 1 -- start start 2023-09-30T22:32:12.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.920+0000 7f5d4ef30700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5d4809d6f0 0x7f5d4809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:12.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.920+0000 7f5d4ef30700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d4809e000 0x7f5d48130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:12.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.920+0000 7f5d4ef30700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d4809e410 0x7f5d48130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:12.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.920+0000 7f5d4ef30700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d4809f520 con 0x7f5d4809d6f0 2023-09-30T22:32:12.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.920+0000 7f5d4ef30700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4809f130 con 0x7f5d4809e000 2023-09-30T22:32:12.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.920+0000 7f5d4ef30700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4809e930 con 0x7f5d4809e410 2023-09-30T22:32:12.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.920+0000 7f5d47fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d4809e000 0x7f5d48130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:12.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.920+0000 7f5d47fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d4809e000 0x7f5d48130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36970/0 (socket says 172.21.15.16:36970) 2023-09-30T22:32:12.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.920+0000 7f5d47fff700 1 -- 172.21.15.16:0/2960774127 learned_addr learned my addr 172.21.15.16:0/2960774127 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:12.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.920+0000 7f5d4d4cd700 1 --2- 172.21.15.16:0/2960774127 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d4809e410 0x7f5d48130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:12.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.920+0000 7f5d4cccc700 1 --2- 172.21.15.16:0/2960774127 >> v2:172.21.15.16:3300/0 conn(0x7f5d4809d6f0 0x7f5d4809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:12.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.920+0000 7f5d4d4cd700 1 -- 172.21.15.16:0/2960774127 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d4809e000 msgr2=0x7f5d48130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:12.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.920+0000 7f5d4d4cd700 1 --2- 172.21.15.16:0/2960774127 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d4809e000 0x7f5d48130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:12.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.920+0000 7f5d4d4cd700 1 -- 172.21.15.16:0/2960774127 >> v2:172.21.15.16:3300/0 conn(0x7f5d4809d6f0 msgr2=0x7f5d4809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:12.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.920+0000 7f5d4d4cd700 1 --2- 172.21.15.16:0/2960774127 >> v2:172.21.15.16:3300/0 conn(0x7f5d4809d6f0 0x7f5d4809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:12.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.920+0000 7f5d4d4cd700 1 -- 172.21.15.16:0/2960774127 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d4809b2d0 con 0x7f5d4809e410 2023-09-30T22:32:12.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.921+0000 7f5d4d4cd700 1 --2- 172.21.15.16:0/2960774127 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d4809e410 0x7f5d48130ba0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5d40009e40 tx=0x7f5d4000f740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54f9bf2e5d076cb4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:12.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.921+0000 7f5d477fe700 1 -- 172.21.15.16:0/2960774127 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d40017410 con 0x7f5d4809e410 2023-09-30T22:32:12.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.921+0000 7f5d477fe700 1 -- 172.21.15.16:0/2960774127 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d40017570 con 0x7f5d4809e410 2023-09-30T22:32:12.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.921+0000 7f5d477fe700 1 -- 172.21.15.16:0/2960774127 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d40017920 con 0x7f5d4809e410 2023-09-30T22:32:12.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.922+0000 7f5d4ef30700 1 -- 172.21.15.16:0/2960774127 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d4809e410 msgr2=0x7f5d48130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:12.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.922+0000 7f5d4ef30700 1 --2- 172.21.15.16:0/2960774127 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d4809e410 0x7f5d48130ba0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f5d40009e40 tx=0x7f5d4000f740 comp rx=0 tx=0).stop 2023-09-30T22:32:12.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.922+0000 7f5d4ef30700 1 -- 172.21.15.16:0/2960774127 shutdown_connections 2023-09-30T22:32:12.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.922+0000 7f5d4ef30700 1 --2- 172.21.15.16:0/2960774127 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d4809e410 0x7f5d48130ba0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:12.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.922+0000 7f5d4ef30700 1 --2- 172.21.15.16:0/2960774127 >> v2:172.21.15.16:3300/0 conn(0x7f5d4809d6f0 0x7f5d4809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:12.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.922+0000 7f5d4ef30700 1 --2- 172.21.15.16:0/2960774127 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d4809e000 0x7f5d48130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:12.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.922+0000 7f5d4ef30700 1 -- 172.21.15.16:0/2960774127 >> 172.21.15.16:0/2960774127 conn(0x7f5d48066f50 msgr2=0x7f5d4809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:12.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.922+0000 7f5d4ef30700 1 -- 172.21.15.16:0/2960774127 shutdown_connections 2023-09-30T22:32:12.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.922+0000 7f5d4ef30700 1 -- 172.21.15.16:0/2960774127 wait complete. 2023-09-30T22:32:12.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.923+0000 7f5d4ef30700 1 Processor -- start 2023-09-30T22:32:12.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.923+0000 7f5d4ef30700 1 -- start start 2023-09-30T22:32:12.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.924+0000 7f5d4ef30700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d4809d6f0 0x7f5d4811de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:12.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.924+0000 7f5d4ef30700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d4809e000 0x7f5d4811e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:12.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.924+0000 7f5d4ef30700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d4809e410 0x7f5d4811e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:12.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.924+0000 7f5d4ef30700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d48121760 con 0x7f5d4809d6f0 2023-09-30T22:32:12.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.924+0000 7f5d4ef30700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4811fa90 con 0x7f5d4809e000 2023-09-30T22:32:12.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.924+0000 7f5d4ef30700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5d48120ce0 con 0x7f5d4809e410 2023-09-30T22:32:12.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.924+0000 7f5d47fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d4809e000 0x7f5d4811e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:12.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.924+0000 7f5d47fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d4809e000 0x7f5d4811e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36978/0 (socket says 172.21.15.16:36978) 2023-09-30T22:32:12.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.924+0000 7f5d47fff700 1 -- 172.21.15.16:0/2338326385 learned_addr learned my addr 172.21.15.16:0/2338326385 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:12.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.924+0000 7f5d4d4cd700 1 --2- 172.21.15.16:0/2338326385 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d4809e410 0x7f5d4811e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:12.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.924+0000 7f5d4cccc700 1 --2- 172.21.15.16:0/2338326385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d4809d6f0 0x7f5d4811de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:12.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.924+0000 7f5d47fff700 1 -- 172.21.15.16:0/2338326385 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d4809e410 msgr2=0x7f5d4811e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:12.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.924+0000 7f5d47fff700 1 --2- 172.21.15.16:0/2338326385 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d4809e410 0x7f5d4811e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:12.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.924+0000 7f5d47fff700 1 -- 172.21.15.16:0/2338326385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d4809d6f0 msgr2=0x7f5d4811de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:12.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.924+0000 7f5d47fff700 1 --2- 172.21.15.16:0/2338326385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d4809d6f0 0x7f5d4811de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:12.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.924+0000 7f5d47fff700 1 -- 172.21.15.16:0/2338326385 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d40009a70 con 0x7f5d4809e000 2023-09-30T22:32:12.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.924+0000 7f5d4d4cd700 1 --2- 172.21.15.16:0/2338326385 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d4809e410 0x7f5d4811e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:12.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.925+0000 7f5d47fff700 1 --2- 172.21.15.16:0/2338326385 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d4809e000 0x7f5d4811e3c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5d3400fbb0 tx=0x7f5d3400fec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:12.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.925+0000 7f5d457fa700 1 -- 172.21.15.16:0/2338326385 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3400b750 con 0x7f5d4809e000 2023-09-30T22:32:12.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.925+0000 7f5d457fa700 1 -- 172.21.15.16:0/2338326385 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d3400dd70 con 0x7f5d4809e000 2023-09-30T22:32:12.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.925+0000 7f5d4ef30700 1 -- 172.21.15.16:0/2338326385 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d48120050 con 0x7f5d4809e000 2023-09-30T22:32:12.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.925+0000 7f5d457fa700 1 -- 172.21.15.16:0/2338326385 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3400dee0 con 0x7f5d4809e000 2023-09-30T22:32:12.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.925+0000 7f5d4ef30700 1 -- 172.21.15.16:0/2338326385 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d48121180 con 0x7f5d4809e000 2023-09-30T22:32:12.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.926+0000 7f5d457fa700 1 -- 172.21.15.16:0/2338326385 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5d34010680 con 0x7f5d4809e000 2023-09-30T22:32:12.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.926+0000 7f5d4ef30700 1 -- 172.21.15.16:0/2338326385 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d4811ac20 con 0x7f5d4809e000 2023-09-30T22:32:12.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.927+0000 7f5d457fa700 1 -- 172.21.15.16:0/2338326385 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f5d3403fe40 con 0x7f5d4809e000 2023-09-30T22:32:12.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.936+0000 7f5d457fa700 1 -- 172.21.15.16:0/2338326385 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5d34010b10 con 0x7f5d4809e000 2023-09-30T22:32:12.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.948+0000 7f235006d700 1 Processor -- start 2023-09-30T22:32:12.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.948+0000 7f235006d700 1 -- start start 2023-09-30T22:32:12.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.948+0000 7f235006d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f234809e1d0 0x7f2348135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:12.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.948+0000 7f235006d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2348135e50 0x7f2348138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:12.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.948+0000 7f235006d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f234809f130 con 0x7f234809d6f0 2023-09-30T22:32:12.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.948+0000 7f235006d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f234809f520 con 0x7f234809e1d0 2023-09-30T22:32:12.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.948+0000 7f235006d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f234809e930 con 0x7f2348135e50 2023-09-30T22:32:12.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.949+0000 7f234de09700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f234809d6f0 0x7f234809dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48268/0 (socket says 172.21.15.16:48268) 2023-09-30T22:32:12.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.949+0000 7f234de09700 1 -- 172.21.15.16:0/685246467 learned_addr learned my addr 172.21.15.16:0/685246467 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:12.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.949+0000 7f234d608700 1 --2- 172.21.15.16:0/685246467 >> v2:172.21.15.16:3300/0 conn(0x7f234809e1d0 0x7f2348135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:12.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.949+0000 7f234e60a700 1 --2- 172.21.15.16:0/685246467 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2348135e50 0x7f2348138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:12.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.949+0000 7f234d608700 1 -- 172.21.15.16:0/685246467 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2348135e50 msgr2=0x7f2348138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:12.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.949+0000 7f234d608700 1 --2- 172.21.15.16:0/685246467 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2348135e50 0x7f2348138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:12.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.949+0000 7f234d608700 1 -- 172.21.15.16:0/685246467 >> v1:172.21.15.16:6789/0 conn(0x7f234809d6f0 legacy=0x7f234809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:12.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.949+0000 7f234d608700 1 -- 172.21.15.16:0/685246467 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f234809b2d0 con 0x7f234809e1d0 2023-09-30T22:32:12.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.949+0000 7f234d608700 1 --2- 172.21.15.16:0/685246467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f234809e1d0 0x7f2348135910 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f2338009840 tx=0x7f2338009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cb36b685bd44745 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:12.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.950+0000 7f234ce07700 1 -- 172.21.15.16:0/685246467 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2338011830 con 0x7f234809e1d0 2023-09-30T22:32:12.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.950+0000 7f234ce07700 1 -- 172.21.15.16:0/685246467 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2338011990 con 0x7f234809e1d0 2023-09-30T22:32:12.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.950+0000 7f234ce07700 1 -- 172.21.15.16:0/685246467 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2338011d20 con 0x7f234809e1d0 2023-09-30T22:32:12.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.950+0000 7f235006d700 1 -- 172.21.15.16:0/685246467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f234809e1d0 msgr2=0x7f2348135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:12.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.950+0000 7f235006d700 1 --2- 172.21.15.16:0/685246467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f234809e1d0 0x7f2348135910 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f2338009840 tx=0x7f2338009b50 comp rx=0 tx=0).stop 2023-09-30T22:32:12.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.950+0000 7f235006d700 1 -- 172.21.15.16:0/685246467 shutdown_connections 2023-09-30T22:32:12.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.950+0000 7f235006d700 1 --2- 172.21.15.16:0/685246467 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2348135e50 0x7f2348138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:12.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.950+0000 7f235006d700 1 --2- 172.21.15.16:0/685246467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f234809e1d0 0x7f2348135910 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:12.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.950+0000 7f235006d700 1 -- 172.21.15.16:0/685246467 >> 172.21.15.16:0/685246467 conn(0x7f2348066f50 msgr2=0x7f234809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:12.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.951+0000 7f235006d700 1 -- 172.21.15.16:0/685246467 shutdown_connections 2023-09-30T22:32:12.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.951+0000 7f235006d700 1 -- 172.21.15.16:0/685246467 wait complete. 2023-09-30T22:32:12.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.952+0000 7f235006d700 1 Processor -- start 2023-09-30T22:32:12.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.952+0000 7f235006d700 1 -- start start 2023-09-30T22:32:12.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.952+0000 7f235006d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f234809d6f0 0x7f2348134d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:12.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.952+0000 7f235006d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f234809e1d0 0x7f2348135290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:12.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.952+0000 7f235006d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2348135e50 0x7f23481357d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:12.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.952+0000 7f235006d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23481303a0 con 0x7f2348135e50 2023-09-30T22:32:12.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.952+0000 7f235006d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2348133bd0 con 0x7f234809e1d0 2023-09-30T22:32:12.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.952+0000 7f235006d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f23480a28a0 con 0x7f234809d6f0 2023-09-30T22:32:12.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.952+0000 7f234e60a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2348135e50 0x7f23481357d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:12.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.952+0000 7f234e60a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2348135e50 0x7f23481357d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33368/0 (socket says 172.21.15.16:33368) 2023-09-30T22:32:12.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.952+0000 7f234e60a700 1 -- 172.21.15.16:0/1412344207 learned_addr learned my addr 172.21.15.16:0/1412344207 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:12.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.952+0000 7f234d608700 1 --2- 172.21.15.16:0/1412344207 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f234809e1d0 0x7f2348135290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:12.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.952+0000 7f234de09700 1 --2- 172.21.15.16:0/1412344207 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f234809d6f0 0x7f2348134d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:12.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.953+0000 7f234e60a700 1 -- 172.21.15.16:0/1412344207 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f234809d6f0 msgr2=0x7f2348134d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:12.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.953+0000 7f234e60a700 1 --2- 172.21.15.16:0/1412344207 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f234809d6f0 0x7f2348134d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:12.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.953+0000 7f234e60a700 1 -- 172.21.15.16:0/1412344207 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f234809e1d0 msgr2=0x7f2348135290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:12.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.953+0000 7f234e60a700 1 --2- 172.21.15.16:0/1412344207 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f234809e1d0 0x7f2348135290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:12.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.953+0000 7f234e60a700 1 -- 172.21.15.16:0/1412344207 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23380094a0 con 0x7f2348135e50 2023-09-30T22:32:12.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.953+0000 7f234de09700 1 --2- 172.21.15.16:0/1412344207 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f234809d6f0 0x7f2348134d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:32:12.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.953+0000 7f234e60a700 1 --2- 172.21.15.16:0/1412344207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2348135e50 0x7f23481357d0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f2340009e30 tx=0x7f234000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:12.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.953+0000 7f233e7fc700 1 -- 172.21.15.16:0/1412344207 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234000cce0 con 0x7f2348135e50 2023-09-30T22:32:12.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.953+0000 7f233e7fc700 1 -- 172.21.15.16:0/1412344207 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f234000f040 con 0x7f2348135e50 2023-09-30T22:32:12.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.953+0000 7f233e7fc700 1 -- 172.21.15.16:0/1412344207 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23400146c0 con 0x7f2348135e50 2023-09-30T22:32:12.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.953+0000 7f235006d700 1 -- 172.21.15.16:0/1412344207 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f234813ac10 con 0x7f2348135e50 2023-09-30T22:32:12.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.953+0000 7f235006d700 1 -- 172.21.15.16:0/1412344207 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23481334b0 con 0x7f2348135e50 2023-09-30T22:32:12.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.954+0000 7f235006d700 1 -- 172.21.15.16:0/1412344207 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2348074ea0 con 0x7f2348135e50 2023-09-30T22:32:12.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.955+0000 7f233e7fc700 1 -- 172.21.15.16:0/1412344207 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2340014820 con 0x7f2348135e50 2023-09-30T22:32:12.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.955+0000 7f233e7fc700 1 -- 172.21.15.16:0/1412344207 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f2340040340 con 0x7f2348135e50 2023-09-30T22:32:12.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.964+0000 7f233e7fc700 1 -- 172.21.15.16:0/1412344207 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2340007900 con 0x7f2348135e50 2023-09-30T22:32:12.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.983+0000 7f1146a46700 1 Processor -- start 2023-09-30T22:32:12.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.983+0000 7f1146a46700 1 -- start start 2023-09-30T22:32:12.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.984+0000 7f1146a46700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f11401316c0 0x7f1140131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:12.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.984+0000 7f1146a46700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f114006ae80 0x7f1140139550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:12.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.984+0000 7f1146a46700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f114009e130 con 0x7f114009d9f0 2023-09-30T22:32:12.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.984+0000 7f1146a46700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11401312b0 con 0x7f114006ae80 2023-09-30T22:32:12.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.984+0000 7f1146a46700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f114009f6e0 con 0x7f11401316c0 2023-09-30T22:32:12.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.984+0000 7f113ffff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f11401316c0 0x7f1140131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:12.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.984+0000 7f113ffff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f11401316c0 0x7f1140131a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:55674/0 (socket says 172.21.15.16:55674) 2023-09-30T22:32:12.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.984+0000 7f113ffff700 1 -- 172.21.15.16:0/4190283235 learned_addr learned my addr 172.21.15.16:0/4190283235 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:12.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.984+0000 7f1144fe3700 1 --2- 172.21.15.16:0/4190283235 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f114006ae80 0x7f1140139550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:12.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.985+0000 7f113ffff700 1 -- 172.21.15.16:0/4190283235 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f114006ae80 msgr2=0x7f1140139550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:12.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.985+0000 7f113effd700 1 -- 172.21.15.16:0/4190283235 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3302242985 0 0) 0x7f1134001110 con 0x7f114009d9f0 2023-09-30T22:32:12.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.985+0000 7f113ffff700 1 --2- 172.21.15.16:0/4190283235 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f114006ae80 0x7f1140139550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:12.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.985+0000 7f113ffff700 1 -- 172.21.15.16:0/4190283235 >> v1:172.21.15.16:6789/0 conn(0x7f114009d9f0 legacy=0x7f114006a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:12.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.985+0000 7f113ffff700 1 -- 172.21.15.16:0/4190283235 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f114009e760 con 0x7f11401316c0 2023-09-30T22:32:12.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.985+0000 7f113ffff700 1 --2- 172.21.15.16:0/4190283235 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f11401316c0 0x7f1140131a90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1130009920 tx=0x7f1130009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9639b8a2f62df2c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:12.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.985+0000 7f113effd700 1 -- 172.21.15.16:0/4190283235 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1130011830 con 0x7f11401316c0 2023-09-30T22:32:12.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.985+0000 7f113effd700 1 -- 172.21.15.16:0/4190283235 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1130011990 con 0x7f11401316c0 2023-09-30T22:32:12.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.985+0000 7f113effd700 1 -- 172.21.15.16:0/4190283235 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1130011d20 con 0x7f11401316c0 2023-09-30T22:32:12.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.986+0000 7f1146a46700 1 -- 172.21.15.16:0/4190283235 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f11401316c0 msgr2=0x7f1140131a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:12.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.986+0000 7f1146a46700 1 --2- 172.21.15.16:0/4190283235 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f11401316c0 0x7f1140131a90 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1130009920 tx=0x7f1130009c30 comp rx=0 tx=0).stop 2023-09-30T22:32:12.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.986+0000 7f1146a46700 1 -- 172.21.15.16:0/4190283235 shutdown_connections 2023-09-30T22:32:12.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.986+0000 7f1146a46700 1 --2- 172.21.15.16:0/4190283235 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f114006ae80 0x7f1140139550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:12.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.986+0000 7f1146a46700 1 --2- 172.21.15.16:0/4190283235 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f11401316c0 0x7f1140131a90 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:12.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.986+0000 7f1146a46700 1 -- 172.21.15.16:0/4190283235 >> 172.21.15.16:0/4190283235 conn(0x7f1140066f50 msgr2=0x7f114009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:12.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.986+0000 7f1146a46700 1 -- 172.21.15.16:0/4190283235 shutdown_connections 2023-09-30T22:32:12.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.986+0000 7f1146a46700 1 -- 172.21.15.16:0/4190283235 wait complete. 2023-09-30T22:32:12.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.988+0000 7f1146a46700 1 Processor -- start 2023-09-30T22:32:12.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.988+0000 7f1146a46700 1 -- start start 2023-09-30T22:32:12.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.989+0000 7f1146a46700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f114006ae80 0x7f1140139e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:12.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.989+0000 7f1146a46700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f114009d9f0 0x7f11400713a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:12.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.989+0000 7f1146a46700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f11401316c0 0x7f11400718e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:12.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.989+0000 7f1146a46700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f114012b900 con 0x7f114006ae80 2023-09-30T22:32:12.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.989+0000 7f1146a46700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1140134130 con 0x7f11401316c0 2023-09-30T22:32:12.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.989+0000 7f1146a46700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f114013a590 con 0x7f114009d9f0 2023-09-30T22:32:12.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.989+0000 7f113ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f114006ae80 0x7f1140139e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:12.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.989+0000 7f113ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f114006ae80 0x7f1140139e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33376/0 (socket says 172.21.15.16:33376) 2023-09-30T22:32:12.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.989+0000 7f113ffff700 1 -- 172.21.15.16:0/1677047247 learned_addr learned my addr 172.21.15.16:0/1677047247 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:12.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.989+0000 7f113f7fe700 1 --2- 172.21.15.16:0/1677047247 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f114009d9f0 0x7f11400713a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:12.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.989+0000 7f1144fe3700 1 --2- 172.21.15.16:0/1677047247 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f11401316c0 0x7f11400718e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:12.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.989+0000 7f113ffff700 1 -- 172.21.15.16:0/1677047247 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f114009d9f0 msgr2=0x7f11400713a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:12.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.989+0000 7f113ffff700 1 --2- 172.21.15.16:0/1677047247 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f114009d9f0 0x7f11400713a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:12.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.989+0000 7f113ffff700 1 -- 172.21.15.16:0/1677047247 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f11401316c0 msgr2=0x7f11400718e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:12.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.989+0000 7f113ffff700 1 --2- 172.21.15.16:0/1677047247 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f11401316c0 0x7f11400718e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:12.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.989+0000 7f113ffff700 1 -- 172.21.15.16:0/1677047247 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1130009550 con 0x7f114006ae80 2023-09-30T22:32:12.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.990+0000 7f113ffff700 1 --2- 172.21.15.16:0/1677047247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f114006ae80 0x7f1140139e10 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f11300098f0 tx=0x7f1130004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:12.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.990+0000 7f113cff9700 1 -- 172.21.15.16:0/1677047247 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1130004220 con 0x7f114006ae80 2023-09-30T22:32:12.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.990+0000 7f113cff9700 1 -- 172.21.15.16:0/1677047247 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f113001f070 con 0x7f114006ae80 2023-09-30T22:32:12.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.990+0000 7f113cff9700 1 -- 172.21.15.16:0/1677047247 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11300036a0 con 0x7f114006ae80 2023-09-30T22:32:12.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.991+0000 7f1146a46700 1 -- 172.21.15.16:0/1677047247 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11401328d0 con 0x7f114006ae80 2023-09-30T22:32:12.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.991+0000 7f1146a46700 1 -- 172.21.15.16:0/1677047247 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1140134e60 con 0x7f114006ae80 2023-09-30T22:32:12.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.992+0000 7f1146a46700 1 -- 172.21.15.16:0/1677047247 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1140133020 con 0x7f114006ae80 2023-09-30T22:32:12.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.992+0000 7f113cff9700 1 -- 172.21.15.16:0/1677047247 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1130003800 con 0x7f114006ae80 2023-09-30T22:32:12.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.993+0000 7f113cff9700 1 -- 172.21.15.16:0/1677047247 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f1130021070 con 0x7f114006ae80 2023-09-30T22:32:12.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:12.997+0000 7f113cff9700 1 -- 172.21.15.16:0/1677047247 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f113000bc50 con 0x7f114006ae80 2023-09-30T22:32:13.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:13.047+0000 7f1ed3610700 1 -- 172.21.15.16:0/461600489 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1ecc123de0 con 0x7f1ecc09d630 2023-09-30T22:32:13.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:13.047+0000 7f1ec1ffb700 1 -- 172.21.15.16:0/461600489 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1ecc123de0 con 0x7f1ecc09d630 2023-09-30T22:32:13.049 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:32:13.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:13.051+0000 7f1ed3610700 1 -- 172.21.15.16:0/461600489 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ecc09d630 msgr2=0x7f1ecc120640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:13.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:13.051+0000 7f1ed3610700 1 --2- 172.21.15.16:0/461600489 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ecc09d630 0x7f1ecc120640 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1ec800ebe0 tx=0x7f1ec800eef0 comp rx=0 tx=0).stop 2023-09-30T22:32:13.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:13.051+0000 7f1ed3610700 1 -- 172.21.15.16:0/461600489 shutdown_connections 2023-09-30T22:32:13.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:13.051+0000 7f1ed3610700 1 --2- 172.21.15.16:0/461600489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ecc138d80 0x7f1ecc1210c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:13.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:13.051+0000 7f1ed3610700 1 --2- 172.21.15.16:0/461600489 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ecc09d630 0x7f1ecc120640 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:13.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:13.051+0000 7f1ed3610700 1 --2- 172.21.15.16:0/461600489 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ecc0a0a30 0x7f1ecc120b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:13.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:13.051+0000 7f1ed3610700 1 -- 172.21.15.16:0/461600489 >> 172.21.15.16:0/461600489 conn(0x7f1ecc0a3050 msgr2=0x7f1ecc09ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:13.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:13.051+0000 7f1ed3610700 1 -- 172.21.15.16:0/461600489 shutdown_connections 2023-09-30T22:32:13.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:13.051+0000 7f1ed3610700 1 -- 172.21.15.16:0/461600489 wait complete. 2023-09-30T22:32:13.072 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:32:13.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:13.155+0000 7f5d4ef30700 1 -- 172.21.15.16:0/2338326385 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f5d48074ea0 con 0x7f5d4809e000 2023-09-30T22:32:13.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:13.155+0000 7f5d457fa700 1 -- 172.21.15.16:0/2338326385 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5d34020e40 con 0x7f5d4809e000 2023-09-30T22:32:13.156 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:32:13.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:13.157+0000 7f5d4ef30700 1 -- 172.21.15.16:0/2338326385 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d4809e000 msgr2=0x7f5d4811e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:13.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:13.158+0000 7f5d4ef30700 1 --2- 172.21.15.16:0/2338326385 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d4809e000 0x7f5d4811e3c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5d3400fbb0 tx=0x7f5d3400fec0 comp rx=0 tx=0).stop 2023-09-30T22:32:13.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:13.158+0000 7f5d4ef30700 1 -- 172.21.15.16:0/2338326385 shutdown_connections 2023-09-30T22:32:13.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:13.158+0000 7f5d4ef30700 1 --2- 172.21.15.16:0/2338326385 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d4809e410 0x7f5d4811e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:13.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:13.158+0000 7f5d4ef30700 1 --2- 172.21.15.16:0/2338326385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d4809d6f0 0x7f5d4811de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:13.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:13.158+0000 7f5d4ef30700 1 --2- 172.21.15.16:0/2338326385 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d4809e000 0x7f5d4811e3c0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:13.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:13.158+0000 7f5d4ef30700 1 -- 172.21.15.16:0/2338326385 >> 172.21.15.16:0/2338326385 conn(0x7f5d48066f50 msgr2=0x7f5d48130920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:13.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:13.158+0000 7f5d4ef30700 1 -- 172.21.15.16:0/2338326385 shutdown_connections 2023-09-30T22:32:13.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:13.158+0000 7f5d4ef30700 1 -- 172.21.15.16:0/2338326385 wait complete. 2023-09-30T22:32:13.172 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:32:13.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:13.188+0000 7f235006d700 1 -- 172.21.15.16:0/1412344207 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f234806c360 con 0x7f2348135e50 2023-09-30T22:32:13.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:13.189+0000 7f233e7fc700 1 -- 172.21.15.16:0/1412344207 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2340018020 con 0x7f2348135e50 2023-09-30T22:32:13.190 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:32:13.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:13.191+0000 7f235006d700 1 -- 172.21.15.16:0/1412344207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2348135e50 msgr2=0x7f23481357d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:13.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:13.191+0000 7f235006d700 1 --2- 172.21.15.16:0/1412344207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2348135e50 0x7f23481357d0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f2340009e30 tx=0x7f234000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:32:13.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:13.191+0000 7f235006d700 1 -- 172.21.15.16:0/1412344207 shutdown_connections 2023-09-30T22:32:13.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:13.191+0000 7f235006d700 1 --2- 172.21.15.16:0/1412344207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2348135e50 0x7f23481357d0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:13.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:13.191+0000 7f235006d700 1 --2- 172.21.15.16:0/1412344207 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f234809d6f0 0x7f2348134d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:13.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:13.191+0000 7f235006d700 1 --2- 172.21.15.16:0/1412344207 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f234809e1d0 0x7f2348135290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:13.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:13.191+0000 7f235006d700 1 -- 172.21.15.16:0/1412344207 >> 172.21.15.16:0/1412344207 conn(0x7f2348066f50 msgr2=0x7f234806a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:13.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:13.191+0000 7f235006d700 1 -- 172.21.15.16:0/1412344207 shutdown_connections 2023-09-30T22:32:13.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:13.191+0000 7f235006d700 1 -- 172.21.15.16:0/1412344207 wait complete. 2023-09-30T22:32:13.206 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:32:13.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:13.205+0000 7f1146a46700 1 -- 172.21.15.16:0/1677047247 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1140074ea0 con 0x7f114006ae80 2023-09-30T22:32:13.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:13.206+0000 7f113cff9700 1 -- 172.21.15.16:0/1677047247 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f113002a030 con 0x7f114006ae80 2023-09-30T22:32:13.207 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:32:13.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:13.208+0000 7f1146a46700 1 -- 172.21.15.16:0/1677047247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f114006ae80 msgr2=0x7f1140139e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:13.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:13.208+0000 7f1146a46700 1 --2- 172.21.15.16:0/1677047247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f114006ae80 0x7f1140139e10 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f11300098f0 tx=0x7f1130004750 comp rx=0 tx=0).stop 2023-09-30T22:32:13.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:13.208+0000 7f1146a46700 1 -- 172.21.15.16:0/1677047247 shutdown_connections 2023-09-30T22:32:13.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:13.208+0000 7f1146a46700 1 --2- 172.21.15.16:0/1677047247 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f11401316c0 0x7f11400718e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:13.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:13.208+0000 7f1146a46700 1 --2- 172.21.15.16:0/1677047247 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f114006ae80 0x7f1140139e10 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:13.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:13.208+0000 7f1146a46700 1 --2- 172.21.15.16:0/1677047247 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f114009d9f0 0x7f11400713a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:13.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:13.208+0000 7f1146a46700 1 -- 172.21.15.16:0/1677047247 >> 172.21.15.16:0/1677047247 conn(0x7f1140066f50 msgr2=0x7f114009b980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:13.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:13.208+0000 7f1146a46700 1 -- 172.21.15.16:0/1677047247 shutdown_connections 2023-09-30T22:32:13.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:13.208+0000 7f1146a46700 1 -- 172.21.15.16:0/1677047247 wait complete. 2023-09-30T22:32:13.222 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:32:14.073 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:32:14.173 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:32:14.207 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:32:14.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.220+0000 7f561b432700 1 Processor -- start 2023-09-30T22:32:14.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.220+0000 7f561b432700 1 -- start start 2023-09-30T22:32:14.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.221+0000 7f561b432700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f56141305f0 0x7f56141309c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:14.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.221+0000 7f561b432700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f561406f660 0x7f561406fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:14.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.221+0000 7f561b432700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f561406ff90 0x7f5614137530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:14.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.221+0000 7f561b432700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56141301c0 con 0x7f56141305f0 2023-09-30T22:32:14.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.221+0000 7f561b432700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5614071ff0 con 0x7f561406ff90 2023-09-30T22:32:14.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.221+0000 7f561b432700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5614070e70 con 0x7f561406f660 2023-09-30T22:32:14.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.221+0000 7f56191ce700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f56141305f0 0x7f56141309c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:14.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.221+0000 7f56189cd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f561406f660 0x7f561406fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:14.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.222+0000 7f56191ce700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f56141305f0 0x7f56141309c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33382/0 (socket says 172.21.15.16:33382) 2023-09-30T22:32:14.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.222+0000 7f56191ce700 1 -- 172.21.15.16:0/3526431317 learned_addr learned my addr 172.21.15.16:0/3526431317 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:14.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.222+0000 7f56189cd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f561406f660 0x7f561406fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:55686/0 (socket says 172.21.15.16:55686) 2023-09-30T22:32:14.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.222+0000 7f56199cf700 1 --2- 172.21.15.16:0/3526431317 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f561406ff90 0x7f5614137530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:14.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.222+0000 7f56189cd700 1 -- 172.21.15.16:0/3526431317 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f561406ff90 msgr2=0x7f5614137530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:14.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.222+0000 7f56189cd700 1 --2- 172.21.15.16:0/3526431317 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f561406ff90 0x7f5614137530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:14.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.222+0000 7f56189cd700 1 -- 172.21.15.16:0/3526431317 >> v2:172.21.15.16:3300/0 conn(0x7f56141305f0 msgr2=0x7f56141309c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:14.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.222+0000 7f56189cd700 1 --2- 172.21.15.16:0/3526431317 >> v2:172.21.15.16:3300/0 conn(0x7f56141305f0 0x7f56141309c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:14.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.222+0000 7f56189cd700 1 -- 172.21.15.16:0/3526431317 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56140722b0 con 0x7f561406f660 2023-09-30T22:32:14.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.222+0000 7f56191ce700 1 --2- 172.21.15.16:0/3526431317 >> v2:172.21.15.16:3300/0 conn(0x7f56141305f0 0x7f56141309c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:14.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.222+0000 7f56189cd700 1 --2- 172.21.15.16:0/3526431317 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f561406f660 0x7f561406fa50 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f5604009840 tx=0x7f5604009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6575d54fa03907cc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:14.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.222+0000 7f560bfff700 1 -- 172.21.15.16:0/3526431317 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560400f8a0 con 0x7f561406f660 2023-09-30T22:32:14.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.222+0000 7f560bfff700 1 -- 172.21.15.16:0/3526431317 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f560400fa00 con 0x7f561406f660 2023-09-30T22:32:14.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.222+0000 7f560bfff700 1 -- 172.21.15.16:0/3526431317 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560400fd90 con 0x7f561406f660 2023-09-30T22:32:14.226 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:32:14.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.223+0000 7f561b432700 1 -- 172.21.15.16:0/3526431317 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f561406f660 msgr2=0x7f561406fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:14.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.223+0000 7f561b432700 1 --2- 172.21.15.16:0/3526431317 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f561406f660 0x7f561406fa50 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f5604009840 tx=0x7f5604009b50 comp rx=0 tx=0).stop 2023-09-30T22:32:14.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.223+0000 7f561b432700 1 -- 172.21.15.16:0/3526431317 shutdown_connections 2023-09-30T22:32:14.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.223+0000 7f561b432700 1 --2- 172.21.15.16:0/3526431317 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f561406ff90 0x7f5614137530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:14.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.223+0000 7f561b432700 1 --2- 172.21.15.16:0/3526431317 >> v2:172.21.15.16:3300/0 conn(0x7f56141305f0 0x7f56141309c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:14.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.223+0000 7f561b432700 1 --2- 172.21.15.16:0/3526431317 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f561406f660 0x7f561406fa50 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:14.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.223+0000 7f561b432700 1 -- 172.21.15.16:0/3526431317 >> 172.21.15.16:0/3526431317 conn(0x7f56140a2aa0 msgr2=0x7f56140a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:14.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.223+0000 7f561b432700 1 -- 172.21.15.16:0/3526431317 shutdown_connections 2023-09-30T22:32:14.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.223+0000 7f561b432700 1 -- 172.21.15.16:0/3526431317 wait complete. 2023-09-30T22:32:14.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.224+0000 7f561b432700 1 Processor -- start 2023-09-30T22:32:14.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.224+0000 7f561b432700 1 -- start start 2023-09-30T22:32:14.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.225+0000 7f561b432700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f561406f660 0x7f561411b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:14.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.225+0000 7f561b432700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f561406ff90 0x7f561411bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:14.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.225+0000 7f561b432700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f56141305f0 0x7f561411c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:14.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.225+0000 7f561b432700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f561411cd20 con 0x7f561406ff90 2023-09-30T22:32:14.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.225+0000 7f561b432700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f561412dc30 con 0x7f561406f660 2023-09-30T22:32:14.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.225+0000 7f561b432700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f561411da80 con 0x7f56141305f0 2023-09-30T22:32:14.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.225+0000 7f56189cd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f561406ff90 0x7f561411bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:14.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.225+0000 7f56189cd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f561406ff90 0x7f561411bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33384/0 (socket says 172.21.15.16:33384) 2023-09-30T22:32:14.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.225+0000 7f56189cd700 1 -- 172.21.15.16:0/3330271745 learned_addr learned my addr 172.21.15.16:0/3330271745 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:14.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.225+0000 7f56189cd700 1 -- 172.21.15.16:0/3330271745 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f56141305f0 msgr2=0x7f561411c2d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:32:14.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.225+0000 7f56189cd700 1 --2- 172.21.15.16:0/3330271745 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f56141305f0 0x7f561411c2d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:14.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.225+0000 7f56189cd700 1 -- 172.21.15.16:0/3330271745 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f561406f660 msgr2=0x7f561411b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:14.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.225+0000 7f56189cd700 1 --2- 172.21.15.16:0/3330271745 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f561406f660 0x7f561411b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:14.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.226+0000 7f56189cd700 1 -- 172.21.15.16:0/3330271745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5610009550 con 0x7f561406ff90 2023-09-30T22:32:14.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.226+0000 7f56189cd700 1 --2- 172.21.15.16:0/3330271745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f561406ff90 0x7f561411bd90 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f5604000c00 tx=0x7f5604004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:14.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.226+0000 7f5609ffb700 1 -- 172.21.15.16:0/3330271745 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560401e070 con 0x7f561406ff90 2023-09-30T22:32:14.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.226+0000 7f5609ffb700 1 -- 172.21.15.16:0/3330271745 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56040044d0 con 0x7f561406ff90 2023-09-30T22:32:14.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.226+0000 7f5609ffb700 1 -- 172.21.15.16:0/3330271745 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5604003900 con 0x7f561406ff90 2023-09-30T22:32:14.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.226+0000 7f561b432700 1 -- 172.21.15.16:0/3330271745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56040094a0 con 0x7f561406ff90 2023-09-30T22:32:14.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.226+0000 7f561b432700 1 -- 172.21.15.16:0/3330271745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f561412ec40 con 0x7f561406ff90 2023-09-30T22:32:14.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.227+0000 7f5609ffb700 1 -- 172.21.15.16:0/3330271745 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5604022070 con 0x7f561406ff90 2023-09-30T22:32:14.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.227+0000 7f561b432700 1 -- 172.21.15.16:0/3330271745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5614071820 con 0x7f561406ff90 2023-09-30T22:32:14.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.227+0000 7f5609ffb700 1 -- 172.21.15.16:0/3330271745 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f56040423f0 con 0x7f561406ff90 2023-09-30T22:32:14.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.237+0000 7f5609ffb700 1 -- 172.21.15.16:0/3330271745 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f560402cd70 con 0x7f561406ff90 2023-09-30T22:32:14.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.323+0000 7f0adfae1700 1 Processor -- start 2023-09-30T22:32:14.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.323+0000 7f0adfae1700 1 -- start start 2023-09-30T22:32:14.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.323+0000 7f0adfae1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0ad809d6f0 0x7f0ad809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:14.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.323+0000 7f0adfae1700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0ad809e000 0x7f0ad8130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:14.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.323+0000 7f0adfae1700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ad809f130 con 0x7f0ad809e410 2023-09-30T22:32:14.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.323+0000 7f0adfae1700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ad809f520 con 0x7f0ad809e000 2023-09-30T22:32:14.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.324+0000 7f0adfae1700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad809e930 con 0x7f0ad809d6f0 2023-09-30T22:32:14.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.324+0000 7f0add87d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0ad809d6f0 0x7f0ad809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:14.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.324+0000 7f0add07c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0ad809e000 0x7f0ad8130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:14.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.324+0000 7f0add87d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0ad809d6f0 0x7f0ad809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37036/0 (socket says 172.21.15.16:37036) 2023-09-30T22:32:14.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.324+0000 7f0ade07e700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0ad809e410 0x7f0ad8130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48276/0 (socket says 172.21.15.16:48276) 2023-09-30T22:32:14.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.324+0000 7f0add87d700 1 -- 172.21.15.16:0/1197214937 learned_addr learned my addr 172.21.15.16:0/1197214937 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:14.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.324+0000 7f0add07c700 1 -- 172.21.15.16:0/1197214937 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0ad809d6f0 msgr2=0x7f0ad809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:14.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.324+0000 7f0add07c700 1 --2- 172.21.15.16:0/1197214937 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0ad809d6f0 0x7f0ad809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:14.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.324+0000 7f0adc87b700 1 -- 172.21.15.16:0/1197214937 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3549562336 0 0) 0x7f0ad809f130 con 0x7f0ad809e410 2023-09-30T22:32:14.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.324+0000 7f0add07c700 1 -- 172.21.15.16:0/1197214937 >> v1:172.21.15.16:6789/0 conn(0x7f0ad809e410 legacy=0x7f0ad8130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:14.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.324+0000 7f0add07c700 1 -- 172.21.15.16:0/1197214937 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ad809b2d0 con 0x7f0ad809e000 2023-09-30T22:32:14.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.324+0000 7f0add87d700 1 --2- 172.21.15.16:0/1197214937 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0ad809d6f0 0x7f0ad809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:14.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.325+0000 7f0add07c700 1 --2- 172.21.15.16:0/1197214937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ad809e000 0x7f0ad8130660 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f0ac8009840 tx=0x7f0ac8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3ffde5ff6c42d93 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:14.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.325+0000 7f0adc87b700 1 -- 172.21.15.16:0/1197214937 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac8011830 con 0x7f0ad809e000 2023-09-30T22:32:14.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.325+0000 7f0adc87b700 1 -- 172.21.15.16:0/1197214937 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ac8011990 con 0x7f0ad809e000 2023-09-30T22:32:14.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.325+0000 7f0adc87b700 1 -- 172.21.15.16:0/1197214937 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac8011d20 con 0x7f0ad809e000 2023-09-30T22:32:14.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.325+0000 7f0adfae1700 1 -- 172.21.15.16:0/1197214937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ad809e000 msgr2=0x7f0ad8130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:14.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.325+0000 7f0adfae1700 1 --2- 172.21.15.16:0/1197214937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ad809e000 0x7f0ad8130660 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f0ac8009840 tx=0x7f0ac8009b50 comp rx=0 tx=0).stop 2023-09-30T22:32:14.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.326+0000 7f0adfae1700 1 -- 172.21.15.16:0/1197214937 shutdown_connections 2023-09-30T22:32:14.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.326+0000 7f0adfae1700 1 --2- 172.21.15.16:0/1197214937 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0ad809d6f0 0x7f0ad809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:14.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.326+0000 7f0adfae1700 1 --2- 172.21.15.16:0/1197214937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ad809e000 0x7f0ad8130660 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:14.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.326+0000 7f0adfae1700 1 -- 172.21.15.16:0/1197214937 >> 172.21.15.16:0/1197214937 conn(0x7f0ad8066f50 msgr2=0x7f0ad809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:14.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.326+0000 7f0adfae1700 1 -- 172.21.15.16:0/1197214937 shutdown_connections 2023-09-30T22:32:14.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.326+0000 7f0adfae1700 1 -- 172.21.15.16:0/1197214937 wait complete. 2023-09-30T22:32:14.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.327+0000 7f0adfae1700 1 Processor -- start 2023-09-30T22:32:14.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.327+0000 7f0adfae1700 1 -- start start 2023-09-30T22:32:14.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.327+0000 7f0adfae1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ad809d6f0 0x7f0ad81ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:14.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.327+0000 7f0adfae1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0ad809e000 0x7f0ad81cd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:14.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.327+0000 7f0adfae1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0ad809e410 0x7f0ad81c94b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:14.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.327+0000 7f0adfae1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad812b030 con 0x7f0ad809d6f0 2023-09-30T22:32:14.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.327+0000 7f0adfae1700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad812aea0 con 0x7f0ad809e000 2023-09-30T22:32:14.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.327+0000 7f0adfae1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad81cd9e0 con 0x7f0ad809e410 2023-09-30T22:32:14.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.328+0000 7f0add87d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ad809d6f0 0x7f0ad81ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:14.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.328+0000 7f0add87d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ad809d6f0 0x7f0ad81ccf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33414/0 (socket says 172.21.15.16:33414) 2023-09-30T22:32:14.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.328+0000 7f0add87d700 1 -- 172.21.15.16:0/828332170 learned_addr learned my addr 172.21.15.16:0/828332170 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:14.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.328+0000 7f0ade07e700 1 --2- 172.21.15.16:0/828332170 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0ad809e410 0x7f0ad81c94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:14.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.328+0000 7f0add07c700 1 --2- 172.21.15.16:0/828332170 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0ad809e000 0x7f0ad81cd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:14.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.328+0000 7f0add87d700 1 -- 172.21.15.16:0/828332170 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0ad809e410 msgr2=0x7f0ad81c94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:14.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.328+0000 7f0add87d700 1 --2- 172.21.15.16:0/828332170 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0ad809e410 0x7f0ad81c94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:14.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.328+0000 7f0add87d700 1 -- 172.21.15.16:0/828332170 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0ad809e000 msgr2=0x7f0ad81cd4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:14.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.328+0000 7f0add87d700 1 --2- 172.21.15.16:0/828332170 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0ad809e000 0x7f0ad81cd4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:14.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.328+0000 7f0add87d700 1 -- 172.21.15.16:0/828332170 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ad400b7f0 con 0x7f0ad809d6f0 2023-09-30T22:32:14.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.328+0000 7f0add87d700 1 --2- 172.21.15.16:0/828332170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ad809d6f0 0x7f0ad81ccf60 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f0ad400ed70 tx=0x7f0ad400d630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:14.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.328+0000 7f0ace7fc700 1 -- 172.21.15.16:0/828332170 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad400de40 con 0x7f0ad809d6f0 2023-09-30T22:32:14.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.328+0000 7f0ace7fc700 1 -- 172.21.15.16:0/828332170 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ad4010440 con 0x7f0ad809d6f0 2023-09-30T22:32:14.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.328+0000 7f0adfae1700 1 -- 172.21.15.16:0/828332170 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ac80094a0 con 0x7f0ad809d6f0 2023-09-30T22:32:14.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.329+0000 7f0ace7fc700 1 -- 172.21.15.16:0/828332170 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad40107b0 con 0x7f0ad809d6f0 2023-09-30T22:32:14.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.329+0000 7f0adfae1700 1 -- 172.21.15.16:0/828332170 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ad81cca20 con 0x7f0ad809d6f0 2023-09-30T22:32:14.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.330+0000 7f0adfae1700 1 -- 172.21.15.16:0/828332170 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ad809de20 con 0x7f0ad809d6f0 2023-09-30T22:32:14.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.330+0000 7f0ace7fc700 1 -- 172.21.15.16:0/828332170 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0ad4010910 con 0x7f0ad809d6f0 2023-09-30T22:32:14.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.330+0000 7f0ace7fc700 1 -- 172.21.15.16:0/828332170 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f0ad403fe10 con 0x7f0ad809d6f0 2023-09-30T22:32:14.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.339+0000 7f0ace7fc700 1 -- 172.21.15.16:0/828332170 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0ad4016d60 con 0x7f0ad809d6f0 2023-09-30T22:32:14.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.369+0000 7f4a71ecf700 1 Processor -- start 2023-09-30T22:32:14.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.370+0000 7f4a71ecf700 1 -- start start 2023-09-30T22:32:14.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.370+0000 7f4a71ecf700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4a6c09d6f0 0x7f4a6c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:14.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.370+0000 7f4a71ecf700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4a6c09e410 0x7f4a6c134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:14.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.370+0000 7f4a71ecf700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a6c09b120 con 0x7f4a6c09e000 2023-09-30T22:32:14.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.370+0000 7f4a71ecf700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6c06b460 con 0x7f4a6c09e410 2023-09-30T22:32:14.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.370+0000 7f4a71ecf700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6c09a7e0 con 0x7f4a6c09d6f0 2023-09-30T22:32:14.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.371+0000 7f4a6b7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4a6c09d6f0 0x7f4a6c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:14.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.371+0000 7f4a6b7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4a6c09d6f0 0x7f4a6c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:55696/0 (socket says 172.21.15.16:55696) 2023-09-30T22:32:14.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.371+0000 7f4a6b7fe700 1 -- 172.21.15.16:0/2189447990 learned_addr learned my addr 172.21.15.16:0/2189447990 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:14.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.371+0000 7f4a6bfff700 1 --2- 172.21.15.16:0/2189447990 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4a6c09e410 0x7f4a6c134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:14.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.371+0000 7f4a6bfff700 1 -- 172.21.15.16:0/2189447990 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4a6c09d6f0 msgr2=0x7f4a6c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:14.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.371+0000 7f4a6bfff700 1 --2- 172.21.15.16:0/2189447990 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4a6c09d6f0 0x7f4a6c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:14.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.371+0000 7f4a6a7fc700 1 -- 172.21.15.16:0/2189447990 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2961061114 0 0) 0x7f4a6c09b120 con 0x7f4a6c09e000 2023-09-30T22:32:14.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.371+0000 7f4a6bfff700 1 -- 172.21.15.16:0/2189447990 >> v1:172.21.15.16:6789/0 conn(0x7f4a6c09e000 legacy=0x7f4a6c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:14.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.371+0000 7f4a6bfff700 1 -- 172.21.15.16:0/2189447990 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a6c09a500 con 0x7f4a6c09e410 2023-09-30T22:32:14.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.371+0000 7f4a6b7fe700 1 --2- 172.21.15.16:0/2189447990 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4a6c09d6f0 0x7f4a6c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:14.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.371+0000 7f4a6bfff700 1 --2- 172.21.15.16:0/2189447990 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4a6c09e410 0x7f4a6c134100 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4a64009dc0 tx=0x7f4a6400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c2db43ee874b28b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:14.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.372+0000 7f4a6a7fc700 1 -- 172.21.15.16:0/2189447990 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a6400e070 con 0x7f4a6c09e410 2023-09-30T22:32:14.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.372+0000 7f4a6a7fc700 1 -- 172.21.15.16:0/2189447990 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a6400bb20 con 0x7f4a6c09e410 2023-09-30T22:32:14.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.372+0000 7f4a6a7fc700 1 -- 172.21.15.16:0/2189447990 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a64012430 con 0x7f4a6c09e410 2023-09-30T22:32:14.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.372+0000 7f4a71ecf700 1 -- 172.21.15.16:0/2189447990 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4a6c09e410 msgr2=0x7f4a6c134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:14.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.372+0000 7f4a71ecf700 1 --2- 172.21.15.16:0/2189447990 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4a6c09e410 0x7f4a6c134100 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4a64009dc0 tx=0x7f4a6400b330 comp rx=0 tx=0).stop 2023-09-30T22:32:14.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.373+0000 7f4a71ecf700 1 -- 172.21.15.16:0/2189447990 shutdown_connections 2023-09-30T22:32:14.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.373+0000 7f4a71ecf700 1 --2- 172.21.15.16:0/2189447990 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4a6c09e410 0x7f4a6c134100 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:14.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.373+0000 7f4a71ecf700 1 --2- 172.21.15.16:0/2189447990 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4a6c09d6f0 0x7f4a6c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:14.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.373+0000 7f4a71ecf700 1 -- 172.21.15.16:0/2189447990 >> 172.21.15.16:0/2189447990 conn(0x7f4a6c066f50 msgr2=0x7f4a6c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:14.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.373+0000 7f4a71ecf700 1 -- 172.21.15.16:0/2189447990 shutdown_connections 2023-09-30T22:32:14.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.373+0000 7f4a71ecf700 1 -- 172.21.15.16:0/2189447990 wait complete. 2023-09-30T22:32:14.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.373+0000 7f3159c7b700 1 Processor -- start 2023-09-30T22:32:14.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.373+0000 7f3159c7b700 1 -- start start 2023-09-30T22:32:14.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.374+0000 7f4a71ecf700 1 Processor -- start 2023-09-30T22:32:14.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.374+0000 7f3159c7b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3154136830 0x7f3154138c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:14.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.374+0000 7f3159c7b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3154139150 0x7f315413b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:14.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.374+0000 7f3159c7b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f315406f720 con 0x7f31541338c0 2023-09-30T22:32:14.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.374+0000 7f3159c7b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3154070c80 con 0x7f3154139150 2023-09-30T22:32:14.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.374+0000 7f3159c7b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f315406fce0 con 0x7f3154136830 2023-09-30T22:32:14.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.374+0000 7f3153fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3154139150 0x7f315413b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:14.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.374+0000 7f3153fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3154139150 0x7f315413b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33430/0 (socket says 172.21.15.16:33430) 2023-09-30T22:32:14.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.374+0000 7f3153fff700 1 -- 172.21.15.16:0/1406311755 learned_addr learned my addr 172.21.15.16:0/1406311755 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:14.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.374+0000 7f3152ffd700 1 --2- 172.21.15.16:0/1406311755 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3154136830 0x7f3154138c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:14.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.375+0000 7f3153fff700 1 -- 172.21.15.16:0/1406311755 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3154136830 msgr2=0x7f3154138c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:14.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.375+0000 7f3153fff700 1 --2- 172.21.15.16:0/1406311755 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3154136830 0x7f3154138c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:14.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.375+0000 7f3153fff700 1 -- 172.21.15.16:0/1406311755 >> v1:172.21.15.16:6789/0 conn(0x7f31541338c0 legacy=0x7f3154133c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:14.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.375+0000 7f3153fff700 1 -- 172.21.15.16:0/1406311755 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31541336a0 con 0x7f3154139150 2023-09-30T22:32:14.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.375+0000 7f3153fff700 1 --2- 172.21.15.16:0/1406311755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3154139150 0x7f315413b530 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f314c009e50 tx=0x7f314c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80c591ea52ca839 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:14.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.374+0000 7f4a71ecf700 1 -- start start 2023-09-30T22:32:14.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.375+0000 7f4a71ecf700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4a6c09d6f0 0x7f4a6c1cc720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:14.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.375+0000 7f4a71ecf700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4a6c09e000 0x7f4a6c1ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:14.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.375+0000 7f4a71ecf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a6c09e410 0x7f4a6c1cd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:14.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.375+0000 7f4a71ecf700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6c11c730 con 0x7f4a6c09e410 2023-09-30T22:32:14.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.375+0000 7f4a71ecf700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6c11cc60 con 0x7f4a6c09d6f0 2023-09-30T22:32:14.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.375+0000 7f4a71ecf700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4a6c072000 con 0x7f4a6c09e000 2023-09-30T22:32:14.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.375+0000 7f4a6bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a6c09e410 0x7f4a6c1cd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:14.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.375+0000 7f4a6bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a6c09e410 0x7f4a6c1cd1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33438/0 (socket says 172.21.15.16:33438) 2023-09-30T22:32:14.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.375+0000 7f4a6bfff700 1 -- 172.21.15.16:0/3535621208 learned_addr learned my addr 172.21.15.16:0/3535621208 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:14.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.375+0000 7f4a6b7fe700 1 --2- 172.21.15.16:0/3535621208 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4a6c09d6f0 0x7f4a6c1cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:14.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.376+0000 7f4a6affd700 1 --2- 172.21.15.16:0/3535621208 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4a6c09e000 0x7f4a6c1ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:14.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.376+0000 7f4a6bfff700 1 -- 172.21.15.16:0/3535621208 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4a6c09e000 msgr2=0x7f4a6c1ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:14.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.376+0000 7f4a6bfff700 1 --2- 172.21.15.16:0/3535621208 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4a6c09e000 0x7f4a6c1ccc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:14.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.376+0000 7f4a6bfff700 1 -- 172.21.15.16:0/3535621208 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4a6c09d6f0 msgr2=0x7f4a6c1cc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:14.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.376+0000 7f4a6bfff700 1 --2- 172.21.15.16:0/3535621208 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4a6c09d6f0 0x7f4a6c1cc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:14.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.376+0000 7f4a6bfff700 1 -- 172.21.15.16:0/3535621208 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a5c009550 con 0x7f4a6c09e410 2023-09-30T22:32:14.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.376+0000 7f4a6affd700 1 --2- 172.21.15.16:0/3535621208 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4a6c09e000 0x7f4a6c1ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:32:14.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.376+0000 7f31527fc700 1 -- 172.21.15.16:0/1406311755 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314c00e070 con 0x7f3154139150 2023-09-30T22:32:14.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.376+0000 7f4a6bfff700 1 --2- 172.21.15.16:0/3535621208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a6c09e410 0x7f4a6c1cd1a0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f4a6400b680 tx=0x7f4a64004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:14.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.376+0000 7f31527fc700 1 -- 172.21.15.16:0/1406311755 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f314c00b910 con 0x7f3154139150 2023-09-30T22:32:14.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.376+0000 7f31527fc700 1 -- 172.21.15.16:0/1406311755 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314c00bcc0 con 0x7f3154139150 2023-09-30T22:32:14.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.376+0000 7f4a53fff700 1 -- 172.21.15.16:0/3535621208 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a6401b810 con 0x7f4a6c09e410 2023-09-30T22:32:14.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.376+0000 7f4a53fff700 1 -- 172.21.15.16:0/3535621208 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a64020070 con 0x7f4a6c09e410 2023-09-30T22:32:14.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.376+0000 7f4a71ecf700 1 -- 172.21.15.16:0/3535621208 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a64009a70 con 0x7f4a6c09e410 2023-09-30T22:32:14.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.376+0000 7f4a53fff700 1 -- 172.21.15.16:0/3535621208 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a6401bbd0 con 0x7f4a6c09e410 2023-09-30T22:32:14.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.377+0000 7f3159c7b700 1 -- 172.21.15.16:0/1406311755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3154139150 msgr2=0x7f315413b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:14.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.377+0000 7f3159c7b700 1 --2- 172.21.15.16:0/1406311755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3154139150 0x7f315413b530 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f314c009e50 tx=0x7f314c00b330 comp rx=0 tx=0).stop 2023-09-30T22:32:14.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.377+0000 7f3159c7b700 1 -- 172.21.15.16:0/1406311755 shutdown_connections 2023-09-30T22:32:14.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.377+0000 7f3159c7b700 1 --2- 172.21.15.16:0/1406311755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3154139150 0x7f315413b530 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:14.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.377+0000 7f3159c7b700 1 --2- 172.21.15.16:0/1406311755 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3154136830 0x7f3154138c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:14.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.377+0000 7f3159c7b700 1 -- 172.21.15.16:0/1406311755 >> 172.21.15.16:0/1406311755 conn(0x7f31540a2aa0 msgr2=0x7f31540a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:14.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.377+0000 7f3159c7b700 1 -- 172.21.15.16:0/1406311755 shutdown_connections 2023-09-30T22:32:14.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.377+0000 7f3159c7b700 1 -- 172.21.15.16:0/1406311755 wait complete. 2023-09-30T22:32:14.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.376+0000 7f4a71ecf700 1 -- 172.21.15.16:0/3535621208 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a6c11b820 con 0x7f4a6c09e410 2023-09-30T22:32:14.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.378+0000 7f4a53fff700 1 -- 172.21.15.16:0/3535621208 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4a6401bd30 con 0x7f4a6c09e410 2023-09-30T22:32:14.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.378+0000 7f4a53fff700 1 -- 172.21.15.16:0/3535621208 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f4a64022070 con 0x7f4a6c09e410 2023-09-30T22:32:14.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.378+0000 7f4a71ecf700 1 -- 172.21.15.16:0/3535621208 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a6c11cda0 con 0x7f4a6c09e410 2023-09-30T22:32:14.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.378+0000 7f3159c7b700 1 Processor -- start 2023-09-30T22:32:14.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.379+0000 7f3159c7b700 1 -- start start 2023-09-30T22:32:14.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.379+0000 7f3159c7b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f31541338c0 0x7f3154099a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:14.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.379+0000 7f3159c7b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3154136830 0x7f3154099f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:14.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.379+0000 7f3159c7b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3154139150 0x7f315409a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:14.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.379+0000 7f3159c7b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f315409c600 con 0x7f3154139150 2023-09-30T22:32:14.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.379+0000 7f3159c7b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31541227b0 con 0x7f31541338c0 2023-09-30T22:32:14.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.379+0000 7f3159c7b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f315406f9d0 con 0x7f3154136830 2023-09-30T22:32:14.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.380+0000 7f3152ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3154136830 0x7f3154099f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:14.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.380+0000 7f31537fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f31541338c0 0x7f3154099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:14.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.380+0000 7f3152ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3154136830 0x7f3154099f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:55722/0 (socket says 172.21.15.16:55722) 2023-09-30T22:32:14.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.380+0000 7f3152ffd700 1 -- 172.21.15.16:0/975387888 learned_addr learned my addr 172.21.15.16:0/975387888 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:14.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.380+0000 7f3153fff700 1 --2- 172.21.15.16:0/975387888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3154139150 0x7f315409a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:14.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.380+0000 7f3152ffd700 1 -- 172.21.15.16:0/975387888 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f31541338c0 msgr2=0x7f3154099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:14.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.380+0000 7f3152ffd700 1 --2- 172.21.15.16:0/975387888 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f31541338c0 0x7f3154099a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:14.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.380+0000 7f3152ffd700 1 -- 172.21.15.16:0/975387888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3154139150 msgr2=0x7f315409a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:14.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.380+0000 7f3152ffd700 1 --2- 172.21.15.16:0/975387888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3154139150 0x7f315409a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:14.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.380+0000 7f3152ffd700 1 -- 172.21.15.16:0/975387888 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f314c009b30 con 0x7f3154136830 2023-09-30T22:32:14.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.380+0000 7f31537fe700 1 --2- 172.21.15.16:0/975387888 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f31541338c0 0x7f3154099a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:32:14.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.380+0000 7f3152ffd700 1 --2- 172.21.15.16:0/975387888 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3154136830 0x7f3154099f80 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f314800e9b0 tx=0x7f314800ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:14.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.380+0000 7f313bfff700 1 -- 172.21.15.16:0/975387888 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3148009710 con 0x7f3154136830 2023-09-30T22:32:14.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.381+0000 7f313bfff700 1 -- 172.21.15.16:0/975387888 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f314800c850 con 0x7f3154136830 2023-09-30T22:32:14.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.381+0000 7f3159c7b700 1 -- 172.21.15.16:0/975387888 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f315409b760 con 0x7f3154136830 2023-09-30T22:32:14.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.381+0000 7f313bfff700 1 -- 172.21.15.16:0/975387888 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f314800cc00 con 0x7f3154136830 2023-09-30T22:32:14.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.381+0000 7f3159c7b700 1 -- 172.21.15.16:0/975387888 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f315409d210 con 0x7f3154136830 2023-09-30T22:32:14.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.381+0000 7f313bfff700 1 -- 172.21.15.16:0/975387888 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f314800cd60 con 0x7f3154136830 2023-09-30T22:32:14.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.382+0000 7f313bfff700 1 -- 172.21.15.16:0/975387888 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f314803fcc0 con 0x7f3154136830 2023-09-30T22:32:14.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.382+0000 7f3159c7b700 1 -- 172.21.15.16:0/975387888 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3154074ea0 con 0x7f3154136830 2023-09-30T22:32:14.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.385+0000 7f4a53fff700 1 -- 172.21.15.16:0/3535621208 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4a6c11cda0 con 0x7f4a6c09e410 2023-09-30T22:32:14.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.392+0000 7f313bfff700 1 -- 172.21.15.16:0/975387888 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3148021080 con 0x7f3154136830 2023-09-30T22:32:14.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.462+0000 7f561b432700 1 -- 172.21.15.16:0/3330271745 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5614074ea0 con 0x7f561406ff90 2023-09-30T22:32:14.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.463+0000 7f5609ffb700 1 -- 172.21.15.16:0/3330271745 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f560402cb90 con 0x7f561406ff90 2023-09-30T22:32:14.463 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:32:14.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.466+0000 7f561b432700 1 -- 172.21.15.16:0/3330271745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f561406ff90 msgr2=0x7f561411bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:14.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.466+0000 7f561b432700 1 --2- 172.21.15.16:0/3330271745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f561406ff90 0x7f561411bd90 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f5604000c00 tx=0x7f5604004750 comp rx=0 tx=0).stop 2023-09-30T22:32:14.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.466+0000 7f561b432700 1 -- 172.21.15.16:0/3330271745 shutdown_connections 2023-09-30T22:32:14.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.466+0000 7f561b432700 1 --2- 172.21.15.16:0/3330271745 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f561406f660 0x7f561411b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:14.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.466+0000 7f561b432700 1 --2- 172.21.15.16:0/3330271745 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f56141305f0 0x7f561411c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:14.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.466+0000 7f561b432700 1 --2- 172.21.15.16:0/3330271745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f561406ff90 0x7f561411bd90 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:14.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.466+0000 7f561b432700 1 -- 172.21.15.16:0/3330271745 >> 172.21.15.16:0/3330271745 conn(0x7f56140a2aa0 msgr2=0x7f561406e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:14.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.466+0000 7f561b432700 1 -- 172.21.15.16:0/3330271745 shutdown_connections 2023-09-30T22:32:14.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.466+0000 7f561b432700 1 -- 172.21.15.16:0/3330271745 wait complete. 2023-09-30T22:32:14.486 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:32:14.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.551+0000 7f0adfae1700 1 -- 172.21.15.16:0/828332170 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f0ad804ea50 con 0x7f0ad809d6f0 2023-09-30T22:32:14.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.552+0000 7f0ace7fc700 1 -- 172.21.15.16:0/828332170 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0ad401d030 con 0x7f0ad809d6f0 2023-09-30T22:32:14.552 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:32:14.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.554+0000 7f0adfae1700 1 -- 172.21.15.16:0/828332170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ad809d6f0 msgr2=0x7f0ad81ccf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:14.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.554+0000 7f0adfae1700 1 --2- 172.21.15.16:0/828332170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ad809d6f0 0x7f0ad81ccf60 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f0ad400ed70 tx=0x7f0ad400d630 comp rx=0 tx=0).stop 2023-09-30T22:32:14.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.554+0000 7f0adfae1700 1 -- 172.21.15.16:0/828332170 shutdown_connections 2023-09-30T22:32:14.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.554+0000 7f0adfae1700 1 --2- 172.21.15.16:0/828332170 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0ad809e410 0x7f0ad81c94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:14.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.554+0000 7f0adfae1700 1 --2- 172.21.15.16:0/828332170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0ad809d6f0 0x7f0ad81ccf60 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:14.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.554+0000 7f0adfae1700 1 --2- 172.21.15.16:0/828332170 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0ad809e000 0x7f0ad81cd4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:14.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.554+0000 7f0adfae1700 1 -- 172.21.15.16:0/828332170 >> 172.21.15.16:0/828332170 conn(0x7f0ad8066f50 msgr2=0x7f0ad8070fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:14.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.554+0000 7f0adfae1700 1 -- 172.21.15.16:0/828332170 shutdown_connections 2023-09-30T22:32:14.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.554+0000 7f0adfae1700 1 -- 172.21.15.16:0/828332170 wait complete. 2023-09-30T22:32:14.568 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:32:14.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.591+0000 7f4a71ecf700 1 -- 172.21.15.16:0/3535621208 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4a6c11cda0 con 0x7f4a6c09e410 2023-09-30T22:32:14.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.592+0000 7f4a53fff700 1 -- 172.21.15.16:0/3535621208 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4a6c11cda0 con 0x7f4a6c09e410 2023-09-30T22:32:14.592 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:32:14.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.594+0000 7f4a71ecf700 1 -- 172.21.15.16:0/3535621208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a6c09e410 msgr2=0x7f4a6c1cd1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:14.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.594+0000 7f4a71ecf700 1 --2- 172.21.15.16:0/3535621208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a6c09e410 0x7f4a6c1cd1a0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f4a6400b680 tx=0x7f4a64004ad0 comp rx=0 tx=0).stop 2023-09-30T22:32:14.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.594+0000 7f4a71ecf700 1 -- 172.21.15.16:0/3535621208 shutdown_connections 2023-09-30T22:32:14.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.594+0000 7f4a71ecf700 1 --2- 172.21.15.16:0/3535621208 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4a6c09d6f0 0x7f4a6c1cc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:14.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.594+0000 7f4a71ecf700 1 --2- 172.21.15.16:0/3535621208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a6c09e410 0x7f4a6c1cd1a0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:14.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.594+0000 7f4a71ecf700 1 --2- 172.21.15.16:0/3535621208 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4a6c09e000 0x7f4a6c1ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:14.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.594+0000 7f4a71ecf700 1 -- 172.21.15.16:0/3535621208 >> 172.21.15.16:0/3535621208 conn(0x7f4a6c066f50 msgr2=0x7f4a6c070dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:14.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.594+0000 7f4a71ecf700 1 -- 172.21.15.16:0/3535621208 shutdown_connections 2023-09-30T22:32:14.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.594+0000 7f4a71ecf700 1 -- 172.21.15.16:0/3535621208 wait complete. 2023-09-30T22:32:14.608 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:32:14.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.614+0000 7f3159c7b700 1 -- 172.21.15.16:0/975387888 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f315406b9e0 con 0x7f3154136830 2023-09-30T22:32:14.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.615+0000 7f313bfff700 1 -- 172.21.15.16:0/975387888 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f314801b3b0 con 0x7f3154136830 2023-09-30T22:32:14.615 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:32:14.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.617+0000 7f3159c7b700 1 -- 172.21.15.16:0/975387888 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3154136830 msgr2=0x7f3154099f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:14.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.617+0000 7f3159c7b700 1 --2- 172.21.15.16:0/975387888 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3154136830 0x7f3154099f80 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f314800e9b0 tx=0x7f314800ecc0 comp rx=0 tx=0).stop 2023-09-30T22:32:14.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.617+0000 7f3159c7b700 1 -- 172.21.15.16:0/975387888 shutdown_connections 2023-09-30T22:32:14.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.617+0000 7f3159c7b700 1 --2- 172.21.15.16:0/975387888 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f31541338c0 0x7f3154099a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:14.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.617+0000 7f3159c7b700 1 --2- 172.21.15.16:0/975387888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3154139150 0x7f315409a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:14.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.617+0000 7f3159c7b700 1 --2- 172.21.15.16:0/975387888 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3154136830 0x7f3154099f80 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:14.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.617+0000 7f3159c7b700 1 -- 172.21.15.16:0/975387888 >> 172.21.15.16:0/975387888 conn(0x7f31540a2aa0 msgr2=0x7f315409de50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:14.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.617+0000 7f3159c7b700 1 -- 172.21.15.16:0/975387888 shutdown_connections 2023-09-30T22:32:14.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:14.617+0000 7f3159c7b700 1 -- 172.21.15.16:0/975387888 wait complete. 2023-09-30T22:32:14.631 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:32:15.487 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:32:15.569 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:32:15.609 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:32:15.632 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:32:15.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.638+0000 7f589080b700 1 Processor -- start 2023-09-30T22:32:15.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.638+0000 7f589080b700 1 -- start start 2023-09-30T22:32:15.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.639+0000 7f589080b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f588c09d6f0 0x7f588c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:15.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.639+0000 7f589080b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f588c09e000 0x7f588c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:15.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.639+0000 7f589080b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f588c09e410 0x7f588c133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:15.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.639+0000 7f589080b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f588c06b460 con 0x7f588c09e410 2023-09-30T22:32:15.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.639+0000 7f589080b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f588c09b120 con 0x7f588c09d6f0 2023-09-30T22:32:15.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.639+0000 7f589080b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f588c09a7e0 con 0x7f588c09e000 2023-09-30T22:32:15.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.639+0000 7f588ad9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f588c09e410 0x7f588c133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:15.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.639+0000 7f5889d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f588c09e000 0x7f588c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:15.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.639+0000 7f588ad9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f588c09e410 0x7f588c133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33458/0 (socket says 172.21.15.16:33458) 2023-09-30T22:32:15.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.639+0000 7f5889d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f588c09e000 0x7f588c1339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:55732/0 (socket says 172.21.15.16:55732) 2023-09-30T22:32:15.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.639+0000 7f588ad9d700 1 -- 172.21.15.16:0/953271099 learned_addr learned my addr 172.21.15.16:0/953271099 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:15.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.639+0000 7f588a59c700 1 --2- 172.21.15.16:0/953271099 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f588c09d6f0 0x7f588c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:15.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.640+0000 7f588ad9d700 1 -- 172.21.15.16:0/953271099 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f588c09e000 msgr2=0x7f588c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:15.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.640+0000 7f588ad9d700 1 --2- 172.21.15.16:0/953271099 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f588c09e000 0x7f588c1339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:15.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.640+0000 7f588ad9d700 1 -- 172.21.15.16:0/953271099 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f588c09d6f0 msgr2=0x7f588c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:15.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.640+0000 7f588ad9d700 1 --2- 172.21.15.16:0/953271099 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f588c09d6f0 0x7f588c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:15.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.640+0000 7f588ad9d700 1 -- 172.21.15.16:0/953271099 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f588c09a500 con 0x7f588c09e410 2023-09-30T22:32:15.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.640+0000 7f588ad9d700 1 --2- 172.21.15.16:0/953271099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f588c09e410 0x7f588c133f30 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f5884009e50 tx=0x7f588400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92cb0bd9d5c45d22 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:15.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.640+0000 7f588959a700 1 -- 172.21.15.16:0/953271099 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f588400e070 con 0x7f588c09e410 2023-09-30T22:32:15.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.640+0000 7f588959a700 1 -- 172.21.15.16:0/953271099 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f588400b910 con 0x7f588c09e410 2023-09-30T22:32:15.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.640+0000 7f588959a700 1 -- 172.21.15.16:0/953271099 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f588400bcc0 con 0x7f588c09e410 2023-09-30T22:32:15.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.641+0000 7f589080b700 1 -- 172.21.15.16:0/953271099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f588c09e410 msgr2=0x7f588c133f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:15.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.641+0000 7f589080b700 1 --2- 172.21.15.16:0/953271099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f588c09e410 0x7f588c133f30 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f5884009e50 tx=0x7f588400b330 comp rx=0 tx=0).stop 2023-09-30T22:32:15.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.641+0000 7f589080b700 1 -- 172.21.15.16:0/953271099 shutdown_connections 2023-09-30T22:32:15.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.641+0000 7f589080b700 1 --2- 172.21.15.16:0/953271099 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f588c09d6f0 0x7f588c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:15.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.641+0000 7f589080b700 1 --2- 172.21.15.16:0/953271099 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f588c09e410 0x7f588c133f30 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:15.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.641+0000 7f589080b700 1 --2- 172.21.15.16:0/953271099 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f588c09e000 0x7f588c1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:15.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.641+0000 7f589080b700 1 -- 172.21.15.16:0/953271099 >> 172.21.15.16:0/953271099 conn(0x7f588c066f50 msgr2=0x7f588c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:15.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.641+0000 7f589080b700 1 -- 172.21.15.16:0/953271099 shutdown_connections 2023-09-30T22:32:15.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.641+0000 7f589080b700 1 -- 172.21.15.16:0/953271099 wait complete. 2023-09-30T22:32:15.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.643+0000 7f589080b700 1 Processor -- start 2023-09-30T22:32:15.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.643+0000 7f589080b700 1 -- start start 2023-09-30T22:32:15.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.644+0000 7f589080b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f588c09d6f0 0x7f588c1a52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:15.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.644+0000 7f589080b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f588c09e000 0x7f588c1a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:15.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.644+0000 7f589080b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f588c09e410 0x7f588c1c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:15.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.644+0000 7f589080b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588c1a7100 con 0x7f588c09e410 2023-09-30T22:32:15.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.644+0000 7f589080b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f588c1a4da0 con 0x7f588c09e000 2023-09-30T22:32:15.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.644+0000 7f589080b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f588c1a7d00 con 0x7f588c09d6f0 2023-09-30T22:32:15.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.644+0000 7f5889d9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f588c09e000 0x7f588c1a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:15.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.644+0000 7f5889d9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f588c09e000 0x7f588c1a5830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37098/0 (socket says 172.21.15.16:37098) 2023-09-30T22:32:15.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.644+0000 7f588ad9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f588c09e410 0x7f588c1c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:15.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.644+0000 7f5889d9b700 1 -- 172.21.15.16:0/829627118 learned_addr learned my addr 172.21.15.16:0/829627118 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:15.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.644+0000 7f588a59c700 1 --2- 172.21.15.16:0/829627118 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f588c09d6f0 0x7f588c1a52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:15.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.645+0000 7f588a59c700 1 -- 172.21.15.16:0/829627118 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f588c09e000 msgr2=0x7f588c1a5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:15.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.645+0000 7f588a59c700 1 --2- 172.21.15.16:0/829627118 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f588c09e000 0x7f588c1a5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:15.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.645+0000 7f588a59c700 1 -- 172.21.15.16:0/829627118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f588c09e410 msgr2=0x7f588c1c8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:15.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.645+0000 7f588a59c700 1 --2- 172.21.15.16:0/829627118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f588c09e410 0x7f588c1c8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:15.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.645+0000 7f588a59c700 1 -- 172.21.15.16:0/829627118 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5884009b30 con 0x7f588c09d6f0 2023-09-30T22:32:15.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.645+0000 7f588a59c700 1 --2- 172.21.15.16:0/829627118 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f588c09d6f0 0x7f588c1a52f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f587c00e930 tx=0x7f587c00ec40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:15.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.645+0000 7f587affd700 1 -- 172.21.15.16:0/829627118 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587c00ee80 con 0x7f588c09d6f0 2023-09-30T22:32:15.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.645+0000 7f587affd700 1 -- 172.21.15.16:0/829627118 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f587c00c850 con 0x7f588c09d6f0 2023-09-30T22:32:15.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.645+0000 7f587affd700 1 -- 172.21.15.16:0/829627118 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587c00cbc0 con 0x7f588c09d6f0 2023-09-30T22:32:15.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.645+0000 7f589080b700 1 -- 172.21.15.16:0/829627118 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f588c1a7fc0 con 0x7f588c09d6f0 2023-09-30T22:32:15.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.646+0000 7f589080b700 1 -- 172.21.15.16:0/829627118 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f588c1a82a0 con 0x7f588c09d6f0 2023-09-30T22:32:15.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.646+0000 7f587affd700 1 -- 172.21.15.16:0/829627118 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f587c00cd20 con 0x7f588c09d6f0 2023-09-30T22:32:15.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.647+0000 7f587affd700 1 -- 172.21.15.16:0/829627118 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f587c03f670 con 0x7f588c09d6f0 2023-09-30T22:32:15.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.647+0000 7f589080b700 1 -- 172.21.15.16:0/829627118 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f588c074ea0 con 0x7f588c09d6f0 2023-09-30T22:32:15.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.653+0000 7f587affd700 1 -- 172.21.15.16:0/829627118 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f587c0232f0 con 0x7f588c09d6f0 2023-09-30T22:32:15.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.716+0000 7f8c06355700 1 Processor -- start 2023-09-30T22:32:15.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.717+0000 7f8c06355700 1 -- start start 2023-09-30T22:32:15.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.717+0000 7f8c06355700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8c001338a0 0x7f8c00133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:15.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.717+0000 7f8c06355700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8c0009de50 0x7f8c0009e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:15.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.717+0000 7f8c06355700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c0006ef30 con 0x7f8c0009e760 2023-09-30T22:32:15.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.717+0000 7f8c06355700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c0006fef0 con 0x7f8c001338a0 2023-09-30T22:32:15.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.717+0000 7f8c06355700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8c001334b0 con 0x7f8c0009de50 2023-09-30T22:32:15.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.717+0000 7f8c048f2700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8c0009e760 0x7f8c0013a730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48306/0 (socket says 172.21.15.16:48306) 2023-09-30T22:32:15.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.717+0000 7f8c048f2700 1 -- 172.21.15.16:0/605323615 learned_addr learned my addr 172.21.15.16:0/605323615 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:15.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.717+0000 7f8bfffff700 1 --2- 172.21.15.16:0/605323615 >> v2:172.21.15.16:3300/0 conn(0x7f8c001338a0 0x7f8c00133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:15.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.717+0000 7f8bff7fe700 1 --2- 172.21.15.16:0/605323615 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8c0009de50 0x7f8c0009e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:15.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.718+0000 7f8bff7fe700 1 -- 172.21.15.16:0/605323615 >> v2:172.21.15.16:3300/0 conn(0x7f8c001338a0 msgr2=0x7f8c00133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:15.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.718+0000 7f8bff7fe700 1 --2- 172.21.15.16:0/605323615 >> v2:172.21.15.16:3300/0 conn(0x7f8c001338a0 0x7f8c00133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:15.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.718+0000 7f8bff7fe700 1 -- 172.21.15.16:0/605323615 >> v1:172.21.15.16:6789/0 conn(0x7f8c0009e760 legacy=0x7f8c0013a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:15.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.718+0000 7f8bff7fe700 1 -- 172.21.15.16:0/605323615 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c0006fc30 con 0x7f8c0009de50 2023-09-30T22:32:15.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.718+0000 7f8bff7fe700 1 --2- 172.21.15.16:0/605323615 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8c0009de50 0x7f8c0009e220 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4009840 tx=0x7f8bf4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=216e8614388fa35f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:15.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.718+0000 7f8bfeffd700 1 -- 172.21.15.16:0/605323615 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4011830 con 0x7f8c0009de50 2023-09-30T22:32:15.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.718+0000 7f8bfeffd700 1 -- 172.21.15.16:0/605323615 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bf4011990 con 0x7f8c0009de50 2023-09-30T22:32:15.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.718+0000 7f8bfeffd700 1 -- 172.21.15.16:0/605323615 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4011d20 con 0x7f8c0009de50 2023-09-30T22:32:15.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.719+0000 7f8c06355700 1 -- 172.21.15.16:0/605323615 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8c0009de50 msgr2=0x7f8c0009e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:15.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.719+0000 7f8c06355700 1 --2- 172.21.15.16:0/605323615 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8c0009de50 0x7f8c0009e220 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4009840 tx=0x7f8bf4009b50 comp rx=0 tx=0).stop 2023-09-30T22:32:15.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.719+0000 7f8c06355700 1 -- 172.21.15.16:0/605323615 shutdown_connections 2023-09-30T22:32:15.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.719+0000 7f8c06355700 1 --2- 172.21.15.16:0/605323615 >> v2:172.21.15.16:3300/0 conn(0x7f8c001338a0 0x7f8c00133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:15.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.719+0000 7f8c06355700 1 --2- 172.21.15.16:0/605323615 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8c0009de50 0x7f8c0009e220 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:15.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.719+0000 7f8c06355700 1 -- 172.21.15.16:0/605323615 >> 172.21.15.16:0/605323615 conn(0x7f8c000a2aa0 msgr2=0x7f8c000a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:15.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.719+0000 7f8c06355700 1 -- 172.21.15.16:0/605323615 shutdown_connections 2023-09-30T22:32:15.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.719+0000 7f8c06355700 1 -- 172.21.15.16:0/605323615 wait complete. 2023-09-30T22:32:15.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.720+0000 7f8c06355700 1 Processor -- start 2023-09-30T22:32:15.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.721+0000 7f8c06355700 1 -- start start 2023-09-30T22:32:15.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.721+0000 7f8c06355700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8c0009de50 0x7f8c0011ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:15.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.721+0000 7f8c06355700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8c0009e760 0x7f8c0011ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:15.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.721+0000 7f8c06355700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c001338a0 0x7f8c0011f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:15.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.721+0000 7f8c06355700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c00120ea0 con 0x7f8c001338a0 2023-09-30T22:32:15.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.721+0000 7f8c06355700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c00120720 con 0x7f8c0009de50 2023-09-30T22:32:15.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.721+0000 7f8c06355700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8c00122240 con 0x7f8c0009e760 2023-09-30T22:32:15.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.721+0000 7f8bff7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8c0009e760 0x7f8c0011ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:15.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.721+0000 7f8bff7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8c0009e760 0x7f8c0011ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:55766/0 (socket says 172.21.15.16:55766) 2023-09-30T22:32:15.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.721+0000 7f8bff7fe700 1 -- 172.21.15.16:0/3233242382 learned_addr learned my addr 172.21.15.16:0/3233242382 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:15.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.721+0000 7f8c048f2700 1 --2- 172.21.15.16:0/3233242382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c001338a0 0x7f8c0011f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:15.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.722+0000 7f8bff7fe700 1 -- 172.21.15.16:0/3233242382 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8c0009de50 msgr2=0x7f8c0011ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:15.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.722+0000 7f8bff7fe700 1 --2- 172.21.15.16:0/3233242382 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8c0009de50 0x7f8c0011ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:15.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.722+0000 7f8bff7fe700 1 -- 172.21.15.16:0/3233242382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c001338a0 msgr2=0x7f8c0011f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:15.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.722+0000 7f8bff7fe700 1 --2- 172.21.15.16:0/3233242382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c001338a0 0x7f8c0011f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:15.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.722+0000 7f8bff7fe700 1 -- 172.21.15.16:0/3233242382 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf40094a0 con 0x7f8c0009e760 2023-09-30T22:32:15.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.722+0000 7f8bff7fe700 1 --2- 172.21.15.16:0/3233242382 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8c0009e760 0x7f8c0011ef40 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4009fd0 tx=0x7f8bf4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:15.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.722+0000 7f8bfcff9700 1 -- 172.21.15.16:0/3233242382 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf40036a0 con 0x7f8c0009e760 2023-09-30T22:32:15.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.722+0000 7f8bfcff9700 1 -- 172.21.15.16:0/3233242382 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bf401e070 con 0x7f8c0009e760 2023-09-30T22:32:15.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.722+0000 7f8bfcff9700 1 -- 172.21.15.16:0/3233242382 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4003a60 con 0x7f8c0009e760 2023-09-30T22:32:15.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.722+0000 7f8c06355700 1 -- 172.21.15.16:0/3233242382 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c000a1020 con 0x7f8c0009e760 2023-09-30T22:32:15.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.722+0000 7f8c06355700 1 -- 172.21.15.16:0/3233242382 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c00122cb0 con 0x7f8c0009e760 2023-09-30T22:32:15.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.723+0000 7f8bfcff9700 1 -- 172.21.15.16:0/3233242382 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8bf4003bc0 con 0x7f8c0009e760 2023-09-30T22:32:15.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.723+0000 7f8c06355700 1 -- 172.21.15.16:0/3233242382 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c00074ea0 con 0x7f8c0009e760 2023-09-30T22:32:15.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.724+0000 7f8bfcff9700 1 -- 172.21.15.16:0/3233242382 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f8bf4040650 con 0x7f8c0009e760 2023-09-30T22:32:15.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.733+0000 7f8bfcff9700 1 -- 172.21.15.16:0/3233242382 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8bf402d970 con 0x7f8c0009e760 2023-09-30T22:32:15.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.771+0000 7fd3a8361700 1 Processor -- start 2023-09-30T22:32:15.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.771+0000 7fd3a8361700 1 -- start start 2023-09-30T22:32:15.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.772+0000 7fd3a8361700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd3a009d6f0 0x7fd3a009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:15.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.772+0000 7fd3a8361700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd3a009e440 0x7fd3a0134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:15.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.772+0000 7fd3a8361700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3a009b120 con 0x7fd3a009e000 2023-09-30T22:32:15.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.772+0000 7fd3a8361700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3a006b460 con 0x7fd3a009d6f0 2023-09-30T22:32:15.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.772+0000 7fd3a8361700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a009a7e0 con 0x7fd3a009e440 2023-09-30T22:32:15.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.772+0000 7fd3a60fd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd3a009d6f0 0x7fd3a009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:15.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.772+0000 7fd3a60fd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd3a009d6f0 0x7fd3a009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33484/0 (socket says 172.21.15.16:33484) 2023-09-30T22:32:15.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.772+0000 7fd3a60fd700 1 -- 172.21.15.16:0/871732213 learned_addr learned my addr 172.21.15.16:0/871732213 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:15.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.772+0000 7fd3a68fe700 1 --2- 172.21.15.16:0/871732213 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd3a009e440 0x7fd3a0134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:15.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.773+0000 7fd3a68fe700 1 -- 172.21.15.16:0/871732213 >> v2:172.21.15.16:3300/0 conn(0x7fd3a009d6f0 msgr2=0x7fd3a009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:15.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.773+0000 7fd3a68fe700 1 --2- 172.21.15.16:0/871732213 >> v2:172.21.15.16:3300/0 conn(0x7fd3a009d6f0 0x7fd3a009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:15.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.773+0000 7fd3a68fe700 1 -- 172.21.15.16:0/871732213 >> v1:172.21.15.16:6789/0 conn(0x7fd3a009e000 legacy=0x7fd3a01339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:15.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.773+0000 7fd3a68fe700 1 -- 172.21.15.16:0/871732213 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3a009a500 con 0x7fd3a009e440 2023-09-30T22:32:15.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.773+0000 7fd3a68fe700 1 --2- 172.21.15.16:0/871732213 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd3a009e440 0x7fd3a0134100 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd398009dc0 tx=0x7fd39800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5964ce23eca164b3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:15.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.773+0000 7fd3a50fb700 1 -- 172.21.15.16:0/871732213 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39800e070 con 0x7fd3a009e440 2023-09-30T22:32:15.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.773+0000 7fd3a50fb700 1 -- 172.21.15.16:0/871732213 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd39800bae0 con 0x7fd3a009e440 2023-09-30T22:32:15.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.773+0000 7fd3a50fb700 1 -- 172.21.15.16:0/871732213 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39800be90 con 0x7fd3a009e440 2023-09-30T22:32:15.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.774+0000 7fd3a8361700 1 -- 172.21.15.16:0/871732213 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd3a009e440 msgr2=0x7fd3a0134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:15.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.774+0000 7fd3a8361700 1 --2- 172.21.15.16:0/871732213 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd3a009e440 0x7fd3a0134100 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd398009dc0 tx=0x7fd39800b330 comp rx=0 tx=0).stop 2023-09-30T22:32:15.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.774+0000 7fd3a8361700 1 -- 172.21.15.16:0/871732213 shutdown_connections 2023-09-30T22:32:15.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.774+0000 7fd3a8361700 1 --2- 172.21.15.16:0/871732213 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd3a009e440 0x7fd3a0134100 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:15.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.774+0000 7fd3a8361700 1 --2- 172.21.15.16:0/871732213 >> v2:172.21.15.16:3300/0 conn(0x7fd3a009d6f0 0x7fd3a009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:15.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.774+0000 7fd3a8361700 1 -- 172.21.15.16:0/871732213 >> 172.21.15.16:0/871732213 conn(0x7fd3a0066f50 msgr2=0x7fd3a009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:15.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.775+0000 7fd3a8361700 1 -- 172.21.15.16:0/871732213 shutdown_connections 2023-09-30T22:32:15.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.775+0000 7fd3a8361700 1 -- 172.21.15.16:0/871732213 wait complete. 2023-09-30T22:32:15.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.776+0000 7fd3a8361700 1 Processor -- start 2023-09-30T22:32:15.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.776+0000 7fd3a8361700 1 -- start start 2023-09-30T22:32:15.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.777+0000 7fd3a8361700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd3a009d6f0 0x7fd3a01a5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:15.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.777+0000 7fd3a8361700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd3a009e000 0x7fd3a01a63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:15.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.777+0000 7fd3a8361700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3a009e440 0x7fd3a01a6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:15.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.777+0000 7fd3a8361700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a01a99d0 con 0x7fd3a009e440 2023-09-30T22:32:15.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.777+0000 7fd3a8361700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a01a7b60 con 0x7fd3a009d6f0 2023-09-30T22:32:15.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.777+0000 7fd3a8361700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a00a1f90 con 0x7fd3a009e000 2023-09-30T22:32:15.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.777+0000 7fd3a68fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3a009e440 0x7fd3a01a6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:15.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.777+0000 7fd3a58fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd3a009e000 0x7fd3a01a63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:15.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.777+0000 7fd3a68fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3a009e440 0x7fd3a01a6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33492/0 (socket says 172.21.15.16:33492) 2023-09-30T22:32:15.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.777+0000 7fd3a68fe700 1 -- 172.21.15.16:0/4018385227 learned_addr learned my addr 172.21.15.16:0/4018385227 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:15.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.778+0000 7fd3a68fe700 1 -- 172.21.15.16:0/4018385227 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd3a009e000 msgr2=0x7fd3a01a63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:15.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.778+0000 7fd3a68fe700 1 --2- 172.21.15.16:0/4018385227 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd3a009e000 0x7fd3a01a63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:15.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.778+0000 7fd3a68fe700 1 -- 172.21.15.16:0/4018385227 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd3a009d6f0 msgr2=0x7fd3a01a5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:15.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.778+0000 7fd3a60fd700 1 --2- 172.21.15.16:0/4018385227 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd3a009d6f0 0x7fd3a01a5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:15.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.778+0000 7fd3a68fe700 1 --2- 172.21.15.16:0/4018385227 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd3a009d6f0 0x7fd3a01a5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:15.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.778+0000 7fd3a68fe700 1 -- 172.21.15.16:0/4018385227 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd398009a70 con 0x7fd3a009e440 2023-09-30T22:32:15.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.778+0000 7fd3a58fc700 1 --2- 172.21.15.16:0/4018385227 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd3a009e000 0x7fd3a01a63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:15.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.778+0000 7fd3a60fd700 1 --2- 172.21.15.16:0/4018385227 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd3a009d6f0 0x7fd3a01a5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:32:15.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.778+0000 7fd3a68fe700 1 --2- 172.21.15.16:0/4018385227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3a009e440 0x7fd3a01a6930 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fd398014040 tx=0x7fd398004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:15.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.778+0000 7fd396ffd700 1 -- 172.21.15.16:0/4018385227 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd398004680 con 0x7fd3a009e440 2023-09-30T22:32:15.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.778+0000 7fd396ffd700 1 -- 172.21.15.16:0/4018385227 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd398020070 con 0x7fd3a009e440 2023-09-30T22:32:15.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.778+0000 7fd3a8361700 1 -- 172.21.15.16:0/4018385227 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3a009c710 con 0x7fd3a009e440 2023-09-30T22:32:15.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.778+0000 7fd396ffd700 1 -- 172.21.15.16:0/4018385227 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd398027480 con 0x7fd3a009e440 2023-09-30T22:32:15.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.778+0000 7fd3a8361700 1 -- 172.21.15.16:0/4018385227 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3a01a8de0 con 0x7fd3a009e440 2023-09-30T22:32:15.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.780+0000 7fd396ffd700 1 -- 172.21.15.16:0/4018385227 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd398012430 con 0x7fd3a009e440 2023-09-30T22:32:15.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.780+0000 7fd3a8361700 1 -- 172.21.15.16:0/4018385227 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3a0074ea0 con 0x7fd3a009e440 2023-09-30T22:32:15.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.780+0000 7fd396ffd700 1 -- 172.21.15.16:0/4018385227 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd398022070 con 0x7fd3a009e440 2023-09-30T22:32:15.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.785+0000 7fd396ffd700 1 -- 172.21.15.16:0/4018385227 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd39801c400 con 0x7fd3a009e440 2023-09-30T22:32:15.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.791+0000 7f0920094700 1 Processor -- start 2023-09-30T22:32:15.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.791+0000 7f0920094700 1 -- start start 2023-09-30T22:32:15.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.792+0000 7f0920094700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0918133420 0x7f09181337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:15.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.792+0000 7f0920094700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0918133e30 0x7f091812ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:15.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.792+0000 7f0920094700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f091806b990 con 0x7f0918134260 2023-09-30T22:32:15.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.792+0000 7f0920094700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09180a0590 con 0x7f0918133e30 2023-09-30T22:32:15.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.792+0000 7f0920094700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f091809fb80 con 0x7f0918133420 2023-09-30T22:32:15.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.792+0000 7f091d62f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0918133e30 0x7f091812ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:15.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.792+0000 7f091d62f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0918133e30 0x7f091812ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33494/0 (socket says 172.21.15.16:33494) 2023-09-30T22:32:15.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.792+0000 7f091d62f700 1 -- 172.21.15.16:0/2163438935 learned_addr learned my addr 172.21.15.16:0/2163438935 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:15.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.793+0000 7f091d62f700 1 -- 172.21.15.16:0/2163438935 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0918133420 msgr2=0x7f09181337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:15.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.793+0000 7f091d62f700 1 --2- 172.21.15.16:0/2163438935 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0918133420 0x7f09181337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:15.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.793+0000 7f091d62f700 1 -- 172.21.15.16:0/2163438935 >> v1:172.21.15.16:6789/0 conn(0x7f0918134260 legacy=0x7f09181304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:15.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.793+0000 7f091d62f700 1 -- 172.21.15.16:0/2163438935 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09180a0030 con 0x7f0918133e30 2023-09-30T22:32:15.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.793+0000 7f091ce2e700 1 -- 172.21.15.16:0/2163438935 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 939182754 0 0) 0x7f091806b990 con 0x7f0918134260 2023-09-30T22:32:15.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.793+0000 7f091d62f700 1 --2- 172.21.15.16:0/2163438935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0918133e30 0x7f091812ff90 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f090c0098f0 tx=0x7f090c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e4f328249bec88f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:15.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.793+0000 7f091ce2e700 1 -- 172.21.15.16:0/2163438935 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090c00b8a0 con 0x7f0918133e30 2023-09-30T22:32:15.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.793+0000 7f091ce2e700 1 -- 172.21.15.16:0/2163438935 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f090c00ba00 con 0x7f0918133e30 2023-09-30T22:32:15.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.793+0000 7f091ce2e700 1 -- 172.21.15.16:0/2163438935 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090c00bd90 con 0x7f0918133e30 2023-09-30T22:32:15.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.794+0000 7f0920094700 1 -- 172.21.15.16:0/2163438935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0918133e30 msgr2=0x7f091812ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:15.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.794+0000 7f0920094700 1 --2- 172.21.15.16:0/2163438935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0918133e30 0x7f091812ff90 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f090c0098f0 tx=0x7f090c009c00 comp rx=0 tx=0).stop 2023-09-30T22:32:15.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.794+0000 7f0920094700 1 -- 172.21.15.16:0/2163438935 shutdown_connections 2023-09-30T22:32:15.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.794+0000 7f0920094700 1 --2- 172.21.15.16:0/2163438935 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0918133420 0x7f09181337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:15.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.794+0000 7f0920094700 1 --2- 172.21.15.16:0/2163438935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0918133e30 0x7f091812ff90 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:15.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.794+0000 7f0920094700 1 -- 172.21.15.16:0/2163438935 >> 172.21.15.16:0/2163438935 conn(0x7f09180a4210 msgr2=0x7f09180a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:15.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.794+0000 7f0920094700 1 -- 172.21.15.16:0/2163438935 shutdown_connections 2023-09-30T22:32:15.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.794+0000 7f0920094700 1 -- 172.21.15.16:0/2163438935 wait complete. 2023-09-30T22:32:15.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.795+0000 7f0920094700 1 Processor -- start 2023-09-30T22:32:15.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.795+0000 7f0920094700 1 -- start start 2023-09-30T22:32:15.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.795+0000 7f0920094700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0918133420 0x7f091809c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:15.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.795+0000 7f0920094700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0918133e30 0x7f091809c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:15.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.795+0000 7f0920094700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0918134260 0x7f091809cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:15.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.795+0000 7f0920094700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091811ec40 con 0x7f0918133420 2023-09-30T22:32:15.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.795+0000 7f0920094700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f091809d850 con 0x7f0918134260 2023-09-30T22:32:15.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.795+0000 7f0920094700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0918131620 con 0x7f0918133e30 2023-09-30T22:32:15.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.796+0000 7f091d62f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0918133e30 0x7f091809c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:15.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.796+0000 7f091de30700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0918133420 0x7f091809c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:15.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.796+0000 7f091d62f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0918133e30 0x7f091809c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:55786/0 (socket says 172.21.15.16:55786) 2023-09-30T22:32:15.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.796+0000 7f091d62f700 1 -- 172.21.15.16:0/2043287822 learned_addr learned my addr 172.21.15.16:0/2043287822 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:15.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.796+0000 7f091de30700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0918133420 0x7f091809c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33510/0 (socket says 172.21.15.16:33510) 2023-09-30T22:32:15.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.796+0000 7f091e631700 1 --2- 172.21.15.16:0/2043287822 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0918134260 0x7f091809cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:15.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.796+0000 7f091d62f700 1 -- 172.21.15.16:0/2043287822 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0918134260 msgr2=0x7f091809cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:15.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.796+0000 7f091d62f700 1 --2- 172.21.15.16:0/2043287822 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0918134260 0x7f091809cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:15.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.796+0000 7f091d62f700 1 -- 172.21.15.16:0/2043287822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0918133420 msgr2=0x7f091809c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:15.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.796+0000 7f091d62f700 1 --2- 172.21.15.16:0/2043287822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0918133420 0x7f091809c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:15.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.796+0000 7f091d62f700 1 -- 172.21.15.16:0/2043287822 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f090c009550 con 0x7f0918133e30 2023-09-30T22:32:15.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.796+0000 7f091de30700 1 --2- 172.21.15.16:0/2043287822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0918133420 0x7f091809c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:15.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.796+0000 7f091d62f700 1 --2- 172.21.15.16:0/2043287822 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0918133e30 0x7f091809c9c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f090c013000 tx=0x7f090c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:15.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.797+0000 7f09167fc700 1 -- 172.21.15.16:0/2043287822 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090c004510 con 0x7f0918133e30 2023-09-30T22:32:15.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.797+0000 7f0920094700 1 -- 172.21.15.16:0/2043287822 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f091812d960 con 0x7f0918133e30 2023-09-30T22:32:15.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.797+0000 7f0920094700 1 -- 172.21.15.16:0/2043287822 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f091809dce0 con 0x7f0918133e30 2023-09-30T22:32:15.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.797+0000 7f09167fc700 1 -- 172.21.15.16:0/2043287822 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f090c0036a0 con 0x7f0918133e30 2023-09-30T22:32:15.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.797+0000 7f09167fc700 1 -- 172.21.15.16:0/2043287822 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090c003a10 con 0x7f0918133e30 2023-09-30T22:32:15.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.798+0000 7f0920094700 1 -- 172.21.15.16:0/2043287822 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08f80052e0 con 0x7f0918133e30 2023-09-30T22:32:15.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.798+0000 7f09167fc700 1 -- 172.21.15.16:0/2043287822 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f090c021070 con 0x7f0918133e30 2023-09-30T22:32:15.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.798+0000 7f09167fc700 1 -- 172.21.15.16:0/2043287822 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f090c040b90 con 0x7f0918133e30 2023-09-30T22:32:15.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.803+0000 7f09167fc700 1 -- 172.21.15.16:0/2043287822 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f090c0193b0 con 0x7f0918133e30 2023-09-30T22:32:15.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.929+0000 7f589080b700 1 -- 172.21.15.16:0/829627118 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f588c1b9230 con 0x7f588c09d6f0 2023-09-30T22:32:15.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.930+0000 7f587affd700 1 -- 172.21.15.16:0/829627118 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f587c021030 con 0x7f588c09d6f0 2023-09-30T22:32:15.930 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:32:15.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.932+0000 7f589080b700 1 -- 172.21.15.16:0/829627118 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f588c09d6f0 msgr2=0x7f588c1a52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:15.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.932+0000 7f589080b700 1 --2- 172.21.15.16:0/829627118 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f588c09d6f0 0x7f588c1a52f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f587c00e930 tx=0x7f587c00ec40 comp rx=0 tx=0).stop 2023-09-30T22:32:15.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.932+0000 7f589080b700 1 -- 172.21.15.16:0/829627118 shutdown_connections 2023-09-30T22:32:15.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.932+0000 7f589080b700 1 --2- 172.21.15.16:0/829627118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f588c09e410 0x7f588c1c8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:15.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.932+0000 7f589080b700 1 --2- 172.21.15.16:0/829627118 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f588c09d6f0 0x7f588c1a52f0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:15.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.932+0000 7f589080b700 1 --2- 172.21.15.16:0/829627118 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f588c09e000 0x7f588c1a5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:15.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.932+0000 7f589080b700 1 -- 172.21.15.16:0/829627118 >> 172.21.15.16:0/829627118 conn(0x7f588c066f50 msgr2=0x7f588c1bbb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:15.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.932+0000 7f589080b700 1 -- 172.21.15.16:0/829627118 shutdown_connections 2023-09-30T22:32:15.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.932+0000 7f589080b700 1 -- 172.21.15.16:0/829627118 wait complete. 2023-09-30T22:32:15.947 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:32:15.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.960+0000 7f8c06355700 1 -- 172.21.15.16:0/3233242382 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f8c0006b9e0 con 0x7f8c0009e760 2023-09-30T22:32:15.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.960+0000 7f8bfcff9700 1 -- 172.21.15.16:0/3233242382 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8bf4010b70 con 0x7f8c0009e760 2023-09-30T22:32:15.960 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:32:15.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.962+0000 7f8c06355700 1 -- 172.21.15.16:0/3233242382 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8c0009e760 msgr2=0x7f8c0011ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:15.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.962+0000 7f8c06355700 1 --2- 172.21.15.16:0/3233242382 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8c0009e760 0x7f8c0011ef40 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4009fd0 tx=0x7f8bf4004750 comp rx=0 tx=0).stop 2023-09-30T22:32:15.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.962+0000 7f8c06355700 1 -- 172.21.15.16:0/3233242382 shutdown_connections 2023-09-30T22:32:15.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.962+0000 7f8c06355700 1 --2- 172.21.15.16:0/3233242382 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8c0009de50 0x7f8c0011ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:15.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.962+0000 7f8c06355700 1 --2- 172.21.15.16:0/3233242382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c001338a0 0x7f8c0011f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:15.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.962+0000 7f8c06355700 1 --2- 172.21.15.16:0/3233242382 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8c0009e760 0x7f8c0011ef40 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:15.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.962+0000 7f8c06355700 1 -- 172.21.15.16:0/3233242382 >> 172.21.15.16:0/3233242382 conn(0x7f8c000a2aa0 msgr2=0x7f8c0009ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:15.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.963+0000 7f8c06355700 1 -- 172.21.15.16:0/3233242382 shutdown_connections 2023-09-30T22:32:15.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:15.963+0000 7f8c06355700 1 -- 172.21.15.16:0/3233242382 wait complete. 2023-09-30T22:32:15.977 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:32:16.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:16.010+0000 7f0920094700 1 -- 172.21.15.16:0/2043287822 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f08f8005150 con 0x7f0918133e30 2023-09-30T22:32:16.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:16.010+0000 7f09167fc700 1 -- 172.21.15.16:0/2043287822 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f090c02dbe0 con 0x7f0918133e30 2023-09-30T22:32:16.011 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:32:16.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:16.012+0000 7f0920094700 1 -- 172.21.15.16:0/2043287822 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0918133e30 msgr2=0x7f091809c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:16.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:16.012+0000 7f0920094700 1 --2- 172.21.15.16:0/2043287822 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0918133e30 0x7f091809c9c0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f090c013000 tx=0x7f090c004750 comp rx=0 tx=0).stop 2023-09-30T22:32:16.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:16.012+0000 7f0920094700 1 -- 172.21.15.16:0/2043287822 shutdown_connections 2023-09-30T22:32:16.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:16.012+0000 7f0920094700 1 --2- 172.21.15.16:0/2043287822 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0918134260 0x7f091809cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:16.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:16.012+0000 7f0920094700 1 --2- 172.21.15.16:0/2043287822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0918133420 0x7f091809c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:16.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:16.012+0000 7f0920094700 1 --2- 172.21.15.16:0/2043287822 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0918133e30 0x7f091809c9c0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:16.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:16.012+0000 7f0920094700 1 -- 172.21.15.16:0/2043287822 >> 172.21.15.16:0/2043287822 conn(0x7f09180a4210 msgr2=0x7f09180a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:16.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:16.013+0000 7f0920094700 1 -- 172.21.15.16:0/2043287822 shutdown_connections 2023-09-30T22:32:16.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:16.013+0000 7f0920094700 1 -- 172.21.15.16:0/2043287822 wait complete. 2023-09-30T22:32:16.026 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:32:16.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:16.053+0000 7fd3a8361700 1 -- 172.21.15.16:0/4018385227 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd3a009f420 con 0x7fd3a009e440 2023-09-30T22:32:16.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:16.054+0000 7fd396ffd700 1 -- 172.21.15.16:0/4018385227 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd398011390 con 0x7fd3a009e440 2023-09-30T22:32:16.054 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:32:16.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:16.056+0000 7fd3a8361700 1 -- 172.21.15.16:0/4018385227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3a009e440 msgr2=0x7fd3a01a6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:16.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:16.056+0000 7fd3a8361700 1 --2- 172.21.15.16:0/4018385227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3a009e440 0x7fd3a01a6930 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fd398014040 tx=0x7fd398004ad0 comp rx=0 tx=0).stop 2023-09-30T22:32:16.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:16.056+0000 7fd3a8361700 1 -- 172.21.15.16:0/4018385227 shutdown_connections 2023-09-30T22:32:16.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:16.056+0000 7fd3a8361700 1 --2- 172.21.15.16:0/4018385227 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd3a009d6f0 0x7fd3a01a5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:16.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:16.056+0000 7fd3a8361700 1 --2- 172.21.15.16:0/4018385227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3a009e440 0x7fd3a01a6930 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:16.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:16.056+0000 7fd3a8361700 1 --2- 172.21.15.16:0/4018385227 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd3a009e000 0x7fd3a01a63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:16.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:16.056+0000 7fd3a8361700 1 -- 172.21.15.16:0/4018385227 >> 172.21.15.16:0/4018385227 conn(0x7fd3a0066f50 msgr2=0x7fd3a01badf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:16.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:16.056+0000 7fd3a8361700 1 -- 172.21.15.16:0/4018385227 shutdown_connections 2023-09-30T22:32:16.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:16.056+0000 7fd3a8361700 1 -- 172.21.15.16:0/4018385227 wait complete. 2023-09-30T22:32:16.069 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:32:16.947 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:32:16.978 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:32:17.028 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:32:17.070 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:32:17.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.098+0000 7f83e6922700 1 Processor -- start 2023-09-30T22:32:17.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.098+0000 7f83e6922700 1 -- start start 2023-09-30T22:32:17.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.099+0000 7f83e6922700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f83e0131810 0x7f83e0131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:17.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.099+0000 7f83e6922700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83e009a580 0x7f83e0139660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:17.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.099+0000 7f83e6922700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83e01313c0 con 0x7f83e0099a40 2023-09-30T22:32:17.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.099+0000 7f83e6922700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83e006e8c0 con 0x7f83e0131810 2023-09-30T22:32:17.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.099+0000 7f83e6922700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83e0071d20 con 0x7f83e009a580 2023-09-30T22:32:17.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.099+0000 7f83dffff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f83e0131810 0x7f83e0131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:17.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.099+0000 7f83e4ebf700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83e009a580 0x7f83e0139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:17.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.099+0000 7f83d77fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f83e0099a40 0x7f83e0099e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48326/0 (socket says 172.21.15.16:48326) 2023-09-30T22:32:17.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.099+0000 7f83dffff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f83e0131810 0x7f83e0131be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33520/0 (socket says 172.21.15.16:33520) 2023-09-30T22:32:17.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.099+0000 7f83dffff700 1 -- 172.21.15.16:0/1206293953 learned_addr learned my addr 172.21.15.16:0/1206293953 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:17.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.100+0000 7f83e4ebf700 1 -- 172.21.15.16:0/1206293953 >> v2:172.21.15.16:3300/0 conn(0x7f83e0131810 msgr2=0x7f83e0131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:17.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.100+0000 7f83e4ebf700 1 --2- 172.21.15.16:0/1206293953 >> v2:172.21.15.16:3300/0 conn(0x7f83e0131810 0x7f83e0131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:17.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.100+0000 7f83e4ebf700 1 -- 172.21.15.16:0/1206293953 >> v1:172.21.15.16:6789/0 conn(0x7f83e0099a40 legacy=0x7f83e0099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:17.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.100+0000 7f83e4ebf700 1 -- 172.21.15.16:0/1206293953 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e00721a0 con 0x7f83e009a580 2023-09-30T22:32:17.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.100+0000 7f83dffff700 1 --2- 172.21.15.16:0/1206293953 >> v2:172.21.15.16:3300/0 conn(0x7f83e0131810 0x7f83e0131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:17.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.100+0000 7f83e4ebf700 1 --2- 172.21.15.16:0/1206293953 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83e009a580 0x7f83e0139660 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f83d8009840 tx=0x7f83d8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b693c4d1e471232c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:17.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.100+0000 7f83df7fe700 1 -- 172.21.15.16:0/1206293953 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d8011830 con 0x7f83e009a580 2023-09-30T22:32:17.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.100+0000 7f83df7fe700 1 -- 172.21.15.16:0/1206293953 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83d8011990 con 0x7f83e009a580 2023-09-30T22:32:17.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.100+0000 7f83df7fe700 1 -- 172.21.15.16:0/1206293953 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d8011d20 con 0x7f83e009a580 2023-09-30T22:32:17.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.101+0000 7f83e6922700 1 -- 172.21.15.16:0/1206293953 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83e009a580 msgr2=0x7f83e0139660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:17.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.101+0000 7f83e6922700 1 --2- 172.21.15.16:0/1206293953 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83e009a580 0x7f83e0139660 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f83d8009840 tx=0x7f83d8009b50 comp rx=0 tx=0).stop 2023-09-30T22:32:17.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.101+0000 7f83e6922700 1 -- 172.21.15.16:0/1206293953 shutdown_connections 2023-09-30T22:32:17.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.101+0000 7f83e6922700 1 --2- 172.21.15.16:0/1206293953 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83e009a580 0x7f83e0139660 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:17.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.101+0000 7f83e6922700 1 --2- 172.21.15.16:0/1206293953 >> v2:172.21.15.16:3300/0 conn(0x7f83e0131810 0x7f83e0131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:17.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.101+0000 7f83e6922700 1 -- 172.21.15.16:0/1206293953 >> 172.21.15.16:0/1206293953 conn(0x7f83e00a2aa0 msgr2=0x7f83e00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:17.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.101+0000 7f83e6922700 1 -- 172.21.15.16:0/1206293953 shutdown_connections 2023-09-30T22:32:17.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.102+0000 7f83e6922700 1 -- 172.21.15.16:0/1206293953 wait complete. 2023-09-30T22:32:17.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.103+0000 7f83e6922700 1 Processor -- start 2023-09-30T22:32:17.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.103+0000 7f83e6922700 1 -- start start 2023-09-30T22:32:17.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.104+0000 7f83e6922700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83e0099a40 0x7f83e0118cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:17.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.104+0000 7f83e6922700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83e009a580 0x7f83e0119210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:17.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.104+0000 7f83e6922700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f83e0131810 0x7f83e0119750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:17.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.104+0000 7f83e6922700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e011cba0 con 0x7f83e0099a40 2023-09-30T22:32:17.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.104+0000 7f83e6922700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83e011a8e0 con 0x7f83e009a580 2023-09-30T22:32:17.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.104+0000 7f83e6922700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f83e011c1a0 con 0x7f83e0131810 2023-09-30T22:32:17.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.104+0000 7f83d77fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83e009a580 0x7f83e0119210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:17.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.104+0000 7f83d77fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83e009a580 0x7f83e0119210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37166/0 (socket says 172.21.15.16:37166) 2023-09-30T22:32:17.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.104+0000 7f83d77fe700 1 -- 172.21.15.16:0/4223351161 learned_addr learned my addr 172.21.15.16:0/4223351161 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.104+0000 7f83e4ebf700 1 --2- 172.21.15.16:0/4223351161 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f83e0131810 0x7f83e0119750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.105+0000 7f83dffff700 1 --2- 172.21.15.16:0/4223351161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83e0099a40 0x7f83e0118cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.105+0000 7f83d77fe700 1 -- 172.21.15.16:0/4223351161 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f83e0131810 msgr2=0x7f83e0119750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.105+0000 7f83d77fe700 1 --2- 172.21.15.16:0/4223351161 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f83e0131810 0x7f83e0119750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.105+0000 7f83d77fe700 1 -- 172.21.15.16:0/4223351161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83e0099a40 msgr2=0x7f83e0118cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.105+0000 7f83d77fe700 1 --2- 172.21.15.16:0/4223351161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83e0099a40 0x7f83e0118cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.105+0000 7f83d77fe700 1 -- 172.21.15.16:0/4223351161 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83d0009b30 con 0x7f83e009a580 2023-09-30T22:32:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.105+0000 7f83dffff700 1 --2- 172.21.15.16:0/4223351161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83e0099a40 0x7f83e0118cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:32:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.105+0000 7f83d77fe700 1 --2- 172.21.15.16:0/4223351161 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83e009a580 0x7f83e0119210 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f83c800c4f0 tx=0x7f83c800c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.105+0000 7f83dd7fa700 1 -- 172.21.15.16:0/4223351161 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c8005cb0 con 0x7f83e009a580 2023-09-30T22:32:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.105+0000 7f83dd7fa700 1 -- 172.21.15.16:0/4223351161 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83c8005e10 con 0x7f83e009a580 2023-09-30T22:32:17.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.105+0000 7f83dd7fa700 1 -- 172.21.15.16:0/4223351161 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83c80072d0 con 0x7f83e009a580 2023-09-30T22:32:17.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.105+0000 7f83e6922700 1 -- 172.21.15.16:0/4223351161 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83d80094a0 con 0x7f83e009a580 2023-09-30T22:32:17.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.105+0000 7f83e6922700 1 -- 172.21.15.16:0/4223351161 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83e00a1b00 con 0x7f83e009a580 2023-09-30T22:32:17.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.106+0000 7f83dd7fa700 1 -- 172.21.15.16:0/4223351161 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f83c8007430 con 0x7f83e009a580 2023-09-30T22:32:17.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.107+0000 7f83dd7fa700 1 -- 172.21.15.16:0/4223351161 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f83c803eac0 con 0x7f83e009a580 2023-09-30T22:32:17.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.108+0000 7f83e6922700 1 -- 172.21.15.16:0/4223351161 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83e0074ea0 con 0x7f83e009a580 2023-09-30T22:32:17.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.113+0000 7f83dd7fa700 1 -- 172.21.15.16:0/4223351161 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f83c801b3b0 con 0x7f83e009a580 2023-09-30T22:32:17.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.152+0000 7f98c2807700 1 Processor -- start 2023-09-30T22:32:17.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.152+0000 7f98c2807700 1 -- start start 2023-09-30T22:32:17.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.153+0000 7f98c2807700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98bc09e1d0 0x7f98bc138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:17.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.153+0000 7f98c2807700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f98bc139170 0x7f98bc13b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:17.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.153+0000 7f98c2807700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98bc09b120 con 0x7f98bc09d6f0 2023-09-30T22:32:17.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.153+0000 7f98c2807700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98bc06b460 con 0x7f98bc139170 2023-09-30T22:32:17.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.153+0000 7f98c2807700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f98bc09a7e0 con 0x7f98bc09e1d0 2023-09-30T22:32:17.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.153+0000 7f98bbfff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f98bc09d6f0 0x7f98bc09dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48336/0 (socket says 172.21.15.16:48336) 2023-09-30T22:32:17.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.153+0000 7f98bbfff700 1 -- 172.21.15.16:0/4197366410 learned_addr learned my addr 172.21.15.16:0/4197366410 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:17.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.153+0000 7f98c0da4700 1 --2- 172.21.15.16:0/4197366410 >> v2:172.21.15.16:3300/0 conn(0x7f98bc139170 0x7f98bc13b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:17.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.153+0000 7f98bb7fe700 1 --2- 172.21.15.16:0/4197366410 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98bc09e1d0 0x7f98bc138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:17.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.153+0000 7f98bb7fe700 1 -- 172.21.15.16:0/4197366410 >> v2:172.21.15.16:3300/0 conn(0x7f98bc139170 msgr2=0x7f98bc13b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:17.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.153+0000 7f98bb7fe700 1 --2- 172.21.15.16:0/4197366410 >> v2:172.21.15.16:3300/0 conn(0x7f98bc139170 0x7f98bc13b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:17.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.153+0000 7f98bb7fe700 1 -- 172.21.15.16:0/4197366410 >> v1:172.21.15.16:6789/0 conn(0x7f98bc09d6f0 legacy=0x7f98bc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:17.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.153+0000 7f98bb7fe700 1 -- 172.21.15.16:0/4197366410 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98bc09a500 con 0x7f98bc09e1d0 2023-09-30T22:32:17.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.154+0000 7f98bb7fe700 1 --2- 172.21.15.16:0/4197366410 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98bc09e1d0 0x7f98bc138c30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f98b0009870 tx=0x7f98b0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7224d6fd58ab9b1b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:17.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.154+0000 7f98baffd700 1 -- 172.21.15.16:0/4197366410 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b0011830 con 0x7f98bc09e1d0 2023-09-30T22:32:17.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.154+0000 7f98baffd700 1 -- 172.21.15.16:0/4197366410 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98b0011990 con 0x7f98bc09e1d0 2023-09-30T22:32:17.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.154+0000 7f98baffd700 1 -- 172.21.15.16:0/4197366410 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b0011d20 con 0x7f98bc09e1d0 2023-09-30T22:32:17.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.155+0000 7f98c2807700 1 -- 172.21.15.16:0/4197366410 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98bc09e1d0 msgr2=0x7f98bc138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:17.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.155+0000 7f98c2807700 1 --2- 172.21.15.16:0/4197366410 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98bc09e1d0 0x7f98bc138c30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f98b0009870 tx=0x7f98b0009b80 comp rx=0 tx=0).stop 2023-09-30T22:32:17.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.155+0000 7f98c2807700 1 -- 172.21.15.16:0/4197366410 shutdown_connections 2023-09-30T22:32:17.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.155+0000 7f98c2807700 1 --2- 172.21.15.16:0/4197366410 >> v2:172.21.15.16:3300/0 conn(0x7f98bc139170 0x7f98bc13b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:17.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.155+0000 7f98c2807700 1 --2- 172.21.15.16:0/4197366410 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98bc09e1d0 0x7f98bc138c30 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:17.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.155+0000 7f98c2807700 1 -- 172.21.15.16:0/4197366410 >> 172.21.15.16:0/4197366410 conn(0x7f98bc066f50 msgr2=0x7f98bc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:17.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.155+0000 7f98c2807700 1 -- 172.21.15.16:0/4197366410 shutdown_connections 2023-09-30T22:32:17.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.155+0000 7f98c2807700 1 -- 172.21.15.16:0/4197366410 wait complete. 2023-09-30T22:32:17.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.156+0000 7f98c2807700 1 Processor -- start 2023-09-30T22:32:17.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.157+0000 7f98c2807700 1 -- start start 2023-09-30T22:32:17.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.158+0000 7f98c2807700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98bc09d6f0 0x7f98bc121770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:17.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.158+0000 7f98c2807700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98bc09e1d0 0x7f98bc121cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:17.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.158+0000 7f98c2807700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f98bc139170 0x7f98bc1221f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:17.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.158+0000 7f98c2807700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98bc123550 con 0x7f98bc09e1d0 2023-09-30T22:32:17.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.158+0000 7f98c2807700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98bc137640 con 0x7f98bc139170 2023-09-30T22:32:17.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.158+0000 7f98c2807700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f98bc09c710 con 0x7f98bc09d6f0 2023-09-30T22:32:17.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.158+0000 7f98bbfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98bc09d6f0 0x7f98bc121770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:17.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.158+0000 7f98bbfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98bc09d6f0 0x7f98bc121770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:55818/0 (socket says 172.21.15.16:55818) 2023-09-30T22:32:17.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.158+0000 7f98bb7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98bc09e1d0 0x7f98bc121cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:17.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.158+0000 7f98bbfff700 1 -- 172.21.15.16:0/1093015030 learned_addr learned my addr 172.21.15.16:0/1093015030 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:17.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.158+0000 7f98c0da4700 1 --2- 172.21.15.16:0/1093015030 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f98bc139170 0x7f98bc1221f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:17.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.158+0000 7f98bbfff700 1 -- 172.21.15.16:0/1093015030 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f98bc139170 msgr2=0x7f98bc1221f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:17.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.158+0000 7f98bbfff700 1 --2- 172.21.15.16:0/1093015030 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f98bc139170 0x7f98bc1221f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:17.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.158+0000 7f98bbfff700 1 -- 172.21.15.16:0/1093015030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98bc09e1d0 msgr2=0x7f98bc121cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:17.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.158+0000 7f98bbfff700 1 --2- 172.21.15.16:0/1093015030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98bc09e1d0 0x7f98bc121cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:17.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.158+0000 7f98bbfff700 1 -- 172.21.15.16:0/1093015030 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b00094a0 con 0x7f98bc09d6f0 2023-09-30T22:32:17.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.158+0000 7f98c0da4700 1 --2- 172.21.15.16:0/1093015030 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f98bc139170 0x7f98bc1221f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:17.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.159+0000 7f98bbfff700 1 --2- 172.21.15.16:0/1093015030 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98bc09d6f0 0x7f98bc121770 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f98ac00c550 tx=0x7f98ac00c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:17.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.159+0000 7f98b8ff9700 1 -- 172.21.15.16:0/1093015030 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ac005d70 con 0x7f98bc09d6f0 2023-09-30T22:32:17.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.159+0000 7f98b8ff9700 1 -- 172.21.15.16:0/1093015030 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98ac0070d0 con 0x7f98bc09d6f0 2023-09-30T22:32:17.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.159+0000 7f98c2807700 1 -- 172.21.15.16:0/1093015030 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98bc124cf0 con 0x7f98bc09d6f0 2023-09-30T22:32:17.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.159+0000 7f98b8ff9700 1 -- 172.21.15.16:0/1093015030 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ac007440 con 0x7f98bc09d6f0 2023-09-30T22:32:17.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.159+0000 7f98c2807700 1 -- 172.21.15.16:0/1093015030 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98bc1249a0 con 0x7f98bc09d6f0 2023-09-30T22:32:17.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.160+0000 7f98b8ff9700 1 -- 172.21.15.16:0/1093015030 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f98ac0075a0 con 0x7f98bc09d6f0 2023-09-30T22:32:17.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.160+0000 7f98b8ff9700 1 -- 172.21.15.16:0/1093015030 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f98ac040b00 con 0x7f98bc09d6f0 2023-09-30T22:32:17.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.160+0000 7f98c2807700 1 -- 172.21.15.16:0/1093015030 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98bc124ee0 con 0x7f98bc09d6f0 2023-09-30T22:32:17.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.165+0000 7f98b8ff9700 1 -- 172.21.15.16:0/1093015030 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f98ac017030 con 0x7f98bc09d6f0 2023-09-30T22:32:17.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.195+0000 7f0c3997f700 1 Processor -- start 2023-09-30T22:32:17.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.196+0000 7f0c3997f700 1 -- start start 2023-09-30T22:32:17.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.196+0000 7f0c3997f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0c3412cf60 0x7f0c3406f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:17.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.196+0000 7f0c3997f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0c34130430 0x7f0c34137c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:17.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.196+0000 7f0c3997f700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c3412cd90 con 0x7f0c3412f8b0 2023-09-30T22:32:17.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.196+0000 7f0c3997f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3412d330 con 0x7f0c34130430 2023-09-30T22:32:17.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.196+0000 7f0c3997f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0c34071780 con 0x7f0c3412cf60 2023-09-30T22:32:17.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.196+0000 7f0c327fc700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0c3412f8b0 0x7f0c3412fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48350/0 (socket says 172.21.15.16:48350) 2023-09-30T22:32:17.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.196+0000 7f0c327fc700 1 -- 172.21.15.16:0/2400816457 learned_addr learned my addr 172.21.15.16:0/2400816457 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:17.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.196+0000 7f0c32ffd700 1 --2- 172.21.15.16:0/2400816457 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0c3412cf60 0x7f0c3406f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:17.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.196+0000 7f0c337fe700 1 --2- 172.21.15.16:0/2400816457 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0c34130430 0x7f0c34137c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:17.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.197+0000 7f0c31ffb700 1 -- 172.21.15.16:0/2400816457 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2058761394 0 0) 0x7f0c3412cd90 con 0x7f0c3412f8b0 2023-09-30T22:32:17.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.197+0000 7f0c31ffb700 1 -- 172.21.15.16:0/2400816457 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c1c0035b0 con 0x7f0c3412f8b0 2023-09-30T22:32:17.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.197+0000 7f0c337fe700 1 -- 172.21.15.16:0/2400816457 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0c3412cf60 msgr2=0x7f0c3406f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:17.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.197+0000 7f0c337fe700 1 --2- 172.21.15.16:0/2400816457 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0c3412cf60 0x7f0c3406f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:17.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.197+0000 7f0c337fe700 1 -- 172.21.15.16:0/2400816457 >> v1:172.21.15.16:6789/0 conn(0x7f0c3412f8b0 legacy=0x7f0c3412fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:17.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.197+0000 7f0c337fe700 1 -- 172.21.15.16:0/2400816457 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c34071a40 con 0x7f0c34130430 2023-09-30T22:32:17.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.197+0000 7f0c337fe700 1 --2- 172.21.15.16:0/2400816457 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0c34130430 0x7f0c34137c30 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c00f3a0 tx=0x7f0c2c00f6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72f01e791902781e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:17.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.197+0000 7f0c31ffb700 1 -- 172.21.15.16:0/2400816457 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c012070 con 0x7f0c34130430 2023-09-30T22:32:17.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.197+0000 7f0c31ffb700 1 -- 172.21.15.16:0/2400816457 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c2c00d0d0 con 0x7f0c34130430 2023-09-30T22:32:17.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.197+0000 7f0c31ffb700 1 -- 172.21.15.16:0/2400816457 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c00fea0 con 0x7f0c34130430 2023-09-30T22:32:17.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.198+0000 7f0c3997f700 1 -- 172.21.15.16:0/2400816457 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0c34130430 msgr2=0x7f0c34137c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:17.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.198+0000 7f0c3997f700 1 --2- 172.21.15.16:0/2400816457 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0c34130430 0x7f0c34137c30 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c00f3a0 tx=0x7f0c2c00f6b0 comp rx=0 tx=0).stop 2023-09-30T22:32:17.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.198+0000 7f0c3997f700 1 -- 172.21.15.16:0/2400816457 shutdown_connections 2023-09-30T22:32:17.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.198+0000 7f0c3997f700 1 --2- 172.21.15.16:0/2400816457 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0c34130430 0x7f0c34137c30 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:17.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.198+0000 7f0c3997f700 1 --2- 172.21.15.16:0/2400816457 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0c3412cf60 0x7f0c3406f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:17.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.198+0000 7f0c3997f700 1 -- 172.21.15.16:0/2400816457 >> 172.21.15.16:0/2400816457 conn(0x7f0c340a2aa0 msgr2=0x7f0c340a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:17.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.198+0000 7f0c3997f700 1 -- 172.21.15.16:0/2400816457 shutdown_connections 2023-09-30T22:32:17.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.198+0000 7f0c3997f700 1 -- 172.21.15.16:0/2400816457 wait complete. 2023-09-30T22:32:17.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.199+0000 7f0c3997f700 1 Processor -- start 2023-09-30T22:32:17.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.200+0000 7f0c3997f700 1 -- start start 2023-09-30T22:32:17.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.200+0000 7f0c3997f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c3412cf60 0x7f0c3409be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:17.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.200+0000 7f0c3997f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0c3412f8b0 0x7f0c3409c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:17.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.200+0000 7f0c3997f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0c34130430 0x7f0c341c40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:17.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.200+0000 7f0c3997f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3409d900 con 0x7f0c3412cf60 2023-09-30T22:32:17.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.200+0000 7f0c3997f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3409c910 con 0x7f0c34130430 2023-09-30T22:32:17.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.200+0000 7f0c3997f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0c34067c70 con 0x7f0c3412f8b0 2023-09-30T22:32:17.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.200+0000 7f0c32ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c3412cf60 0x7f0c3409be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:17.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.200+0000 7f0c32ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c3412cf60 0x7f0c3409be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33572/0 (socket says 172.21.15.16:33572) 2023-09-30T22:32:17.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.200+0000 7f0c32ffd700 1 -- 172.21.15.16:0/2552377320 learned_addr learned my addr 172.21.15.16:0/2552377320 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:17.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.200+0000 7f0c327fc700 1 --2- 172.21.15.16:0/2552377320 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0c3412f8b0 0x7f0c3409c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:17.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.201+0000 7f0c337fe700 1 --2- 172.21.15.16:0/2552377320 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0c34130430 0x7f0c341c40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:17.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.201+0000 7f0c32ffd700 1 -- 172.21.15.16:0/2552377320 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0c3412f8b0 msgr2=0x7f0c3409c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:17.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.201+0000 7f0c32ffd700 1 --2- 172.21.15.16:0/2552377320 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0c3412f8b0 0x7f0c3409c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:17.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.201+0000 7f0c32ffd700 1 -- 172.21.15.16:0/2552377320 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0c34130430 msgr2=0x7f0c341c40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:17.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.201+0000 7f0c32ffd700 1 --2- 172.21.15.16:0/2552377320 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0c34130430 0x7f0c341c40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:17.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.201+0000 7f0c32ffd700 1 -- 172.21.15.16:0/2552377320 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c2c00f050 con 0x7f0c3412cf60 2023-09-30T22:32:17.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.201+0000 7f0c337fe700 1 --2- 172.21.15.16:0/2552377320 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0c34130430 0x7f0c341c40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:32:17.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.201+0000 7f0c32ffd700 1 --2- 172.21.15.16:0/2552377320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c3412cf60 0x7f0c3409be90 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f0c2400dbc0 tx=0x7f0c2400ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:17.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.201+0000 7f0c1bfff700 1 -- 172.21.15.16:0/2552377320 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2400ce00 con 0x7f0c3412cf60 2023-09-30T22:32:17.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.201+0000 7f0c1bfff700 1 -- 172.21.15.16:0/2552377320 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c24015420 con 0x7f0c3412cf60 2023-09-30T22:32:17.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.201+0000 7f0c1bfff700 1 -- 172.21.15.16:0/2552377320 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c240157b0 con 0x7f0c3412cf60 2023-09-30T22:32:17.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.202+0000 7f0c3997f700 1 -- 172.21.15.16:0/2552377320 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c34129fa0 con 0x7f0c3412cf60 2023-09-30T22:32:17.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.202+0000 7f0c3997f700 1 -- 172.21.15.16:0/2552377320 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c3409dd10 con 0x7f0c3412cf60 2023-09-30T22:32:17.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.203+0000 7f0c3997f700 1 -- 172.21.15.16:0/2552377320 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c3406d530 con 0x7f0c3412cf60 2023-09-30T22:32:17.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.204+0000 7f0c1bfff700 1 -- 172.21.15.16:0/2552377320 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0c24005a20 con 0x7f0c3412cf60 2023-09-30T22:32:17.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.204+0000 7f0c1bfff700 1 -- 172.21.15.16:0/2552377320 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f0c24040240 con 0x7f0c3412cf60 2023-09-30T22:32:17.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.212+0000 7f0c1bfff700 1 -- 172.21.15.16:0/2552377320 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0c24005cd0 con 0x7f0c3412cf60 2023-09-30T22:32:17.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.234+0000 7f8a8758d700 1 Processor -- start 2023-09-30T22:32:17.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.235+0000 7f8a8758d700 1 -- start start 2023-09-30T22:32:17.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.235+0000 7f8a8758d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8a8006a770 0x7f8a8009d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:17.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.235+0000 7f8a8758d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8a8013d1f0 0x7f8a8013f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:17.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.235+0000 7f8a8758d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a8009a810 con 0x7f8a8009dc30 2023-09-30T22:32:17.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.235+0000 7f8a8758d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a8006ab40 con 0x7f8a8013d1f0 2023-09-30T22:32:17.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.235+0000 7f8a8758d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8009ebc0 con 0x7f8a8006a770 2023-09-30T22:32:17.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.235+0000 7f8a84b28700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8a8009dc30 0x7f8a8009e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48352/0 (socket says 172.21.15.16:48352) 2023-09-30T22:32:17.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.235+0000 7f8a84b28700 1 -- 172.21.15.16:0/4106677109 learned_addr learned my addr 172.21.15.16:0/4106677109 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:17.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.235+0000 7f8a85b2a700 1 --2- 172.21.15.16:0/4106677109 >> v2:172.21.15.16:3300/0 conn(0x7f8a8013d1f0 0x7f8a8013f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:17.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.236+0000 7f8a85329700 1 --2- 172.21.15.16:0/4106677109 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8a8006a770 0x7f8a8009d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:17.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.236+0000 7f8a85b2a700 1 -- 172.21.15.16:0/4106677109 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8a8006a770 msgr2=0x7f8a8009d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:17.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.236+0000 7f8a85b2a700 1 --2- 172.21.15.16:0/4106677109 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8a8006a770 0x7f8a8009d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:17.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.236+0000 7f8a85b2a700 1 -- 172.21.15.16:0/4106677109 >> v1:172.21.15.16:6789/0 conn(0x7f8a8009dc30 legacy=0x7f8a8009e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:17.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.236+0000 7f8a77fff700 1 -- 172.21.15.16:0/4106677109 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 694671926 0 0) 0x7f8a8009a810 con 0x7f8a8009dc30 2023-09-30T22:32:17.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.236+0000 7f8a85b2a700 1 -- 172.21.15.16:0/4106677109 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a8009e920 con 0x7f8a8013d1f0 2023-09-30T22:32:17.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.236+0000 7f8a85b2a700 1 --2- 172.21.15.16:0/4106677109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a8013d1f0 0x7f8a8013f5d0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f8a78009dc0 tx=0x7f8a7800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e344eca3f7080eaa server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:17.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.236+0000 7f8a77fff700 1 -- 172.21.15.16:0/4106677109 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7800e070 con 0x7f8a8013d1f0 2023-09-30T22:32:17.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.236+0000 7f8a77fff700 1 -- 172.21.15.16:0/4106677109 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a7800bae0 con 0x7f8a8013d1f0 2023-09-30T22:32:17.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.236+0000 7f8a77fff700 1 -- 172.21.15.16:0/4106677109 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7800be90 con 0x7f8a8013d1f0 2023-09-30T22:32:17.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.237+0000 7f8a8758d700 1 -- 172.21.15.16:0/4106677109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a8013d1f0 msgr2=0x7f8a8013f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:17.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.237+0000 7f8a8758d700 1 --2- 172.21.15.16:0/4106677109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a8013d1f0 0x7f8a8013f5d0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f8a78009dc0 tx=0x7f8a7800b330 comp rx=0 tx=0).stop 2023-09-30T22:32:17.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.237+0000 7f8a8758d700 1 -- 172.21.15.16:0/4106677109 shutdown_connections 2023-09-30T22:32:17.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.237+0000 7f8a8758d700 1 --2- 172.21.15.16:0/4106677109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a8013d1f0 0x7f8a8013f5d0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:17.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.237+0000 7f8a8758d700 1 --2- 172.21.15.16:0/4106677109 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8a8006a770 0x7f8a8009d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:17.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.238+0000 7f8a8758d700 1 -- 172.21.15.16:0/4106677109 >> 172.21.15.16:0/4106677109 conn(0x7f8a80066f50 msgr2=0x7f8a8009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:17.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.238+0000 7f8a8758d700 1 -- 172.21.15.16:0/4106677109 shutdown_connections 2023-09-30T22:32:17.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.238+0000 7f8a8758d700 1 -- 172.21.15.16:0/4106677109 wait complete. 2023-09-30T22:32:17.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.239+0000 7f8a8758d700 1 Processor -- start 2023-09-30T22:32:17.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.239+0000 7f8a8758d700 1 -- start start 2023-09-30T22:32:17.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.240+0000 7f8a8758d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a8006a770 0x7f8a80071590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:17.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.240+0000 7f8a8758d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8a8009dc30 0x7f8a8013b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:17.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.240+0000 7f8a8758d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8a8013d1f0 0x7f8a8013bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:17.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.240+0000 7f8a8758d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8013f6e0 con 0x7f8a8006a770 2023-09-30T22:32:17.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.240+0000 7f8a8758d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a800722c0 con 0x7f8a8013d1f0 2023-09-30T22:32:17.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.240+0000 7f8a8758d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8011ea10 con 0x7f8a8009dc30 2023-09-30T22:32:17.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.240+0000 7f8a84b28700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8a8009dc30 0x7f8a8013b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:17.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.240+0000 7f8a84b28700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8a8009dc30 0x7f8a8013b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:55848/0 (socket says 172.21.15.16:55848) 2023-09-30T22:32:17.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.240+0000 7f8a84b28700 1 -- 172.21.15.16:0/3930102569 learned_addr learned my addr 172.21.15.16:0/3930102569 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:17.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.240+0000 7f8a85329700 1 --2- 172.21.15.16:0/3930102569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a8006a770 0x7f8a80071590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:17.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.240+0000 7f8a85b2a700 1 --2- 172.21.15.16:0/3930102569 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8a8013d1f0 0x7f8a8013bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:17.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.241+0000 7f8a85329700 1 -- 172.21.15.16:0/3930102569 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8a8009dc30 msgr2=0x7f8a8013b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:17.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.241+0000 7f8a85329700 1 --2- 172.21.15.16:0/3930102569 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8a8009dc30 0x7f8a8013b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:17.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.241+0000 7f8a85329700 1 -- 172.21.15.16:0/3930102569 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8a8013d1f0 msgr2=0x7f8a8013bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:17.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.241+0000 7f8a85329700 1 --2- 172.21.15.16:0/3930102569 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8a8013d1f0 0x7f8a8013bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:17.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.241+0000 7f8a85329700 1 -- 172.21.15.16:0/3930102569 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a78009a70 con 0x7f8a8006a770 2023-09-30T22:32:17.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.241+0000 7f8a85329700 1 --2- 172.21.15.16:0/3930102569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a8006a770 0x7f8a80071590 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c00e930 tx=0x7f8a7c00ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:17.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.241+0000 7f8a75ffb700 1 -- 172.21.15.16:0/3930102569 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7c00ee80 con 0x7f8a8006a770 2023-09-30T22:32:17.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.241+0000 7f8a75ffb700 1 -- 172.21.15.16:0/3930102569 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a7c00c850 con 0x7f8a8006a770 2023-09-30T22:32:17.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.241+0000 7f8a8758d700 1 -- 172.21.15.16:0/3930102569 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a8011efb0 con 0x7f8a8006a770 2023-09-30T22:32:17.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.241+0000 7f8a75ffb700 1 -- 172.21.15.16:0/3930102569 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7c00cbc0 con 0x7f8a8006a770 2023-09-30T22:32:17.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.241+0000 7f8a8758d700 1 -- 172.21.15.16:0/3930102569 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a8011e2f0 con 0x7f8a8006a770 2023-09-30T22:32:17.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.243+0000 7f8a75ffb700 1 -- 172.21.15.16:0/3930102569 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8a7c00cd20 con 0x7f8a8006a770 2023-09-30T22:32:17.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.243+0000 7f8a8758d700 1 -- 172.21.15.16:0/3930102569 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a80074ea0 con 0x7f8a8006a770 2023-09-30T22:32:17.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.243+0000 7f8a75ffb700 1 -- 172.21.15.16:0/3930102569 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f8a7c03f830 con 0x7f8a8006a770 2023-09-30T22:32:17.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.248+0000 7f8a75ffb700 1 -- 172.21.15.16:0/3930102569 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8a7c021080 con 0x7f8a8006a770 2023-09-30T22:32:17.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.349+0000 7f83e6922700 1 -- 172.21.15.16:0/4223351161 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f83e011bb60 con 0x7f83e009a580 2023-09-30T22:32:17.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.349+0000 7f83dd7fa700 1 -- 172.21.15.16:0/4223351161 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f83e011bb60 con 0x7f83e009a580 2023-09-30T22:32:17.351 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:32:17.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.351+0000 7f83e6922700 1 -- 172.21.15.16:0/4223351161 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83e009a580 msgr2=0x7f83e0119210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:17.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.351+0000 7f83e6922700 1 --2- 172.21.15.16:0/4223351161 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83e009a580 0x7f83e0119210 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f83c800c4f0 tx=0x7f83c800c800 comp rx=0 tx=0).stop 2023-09-30T22:32:17.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.352+0000 7f83e6922700 1 -- 172.21.15.16:0/4223351161 shutdown_connections 2023-09-30T22:32:17.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.352+0000 7f83e6922700 1 --2- 172.21.15.16:0/4223351161 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f83e0131810 0x7f83e0119750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:17.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.352+0000 7f83e6922700 1 --2- 172.21.15.16:0/4223351161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83e0099a40 0x7f83e0118cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:17.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.352+0000 7f83e6922700 1 --2- 172.21.15.16:0/4223351161 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83e009a580 0x7f83e0119210 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:17.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.352+0000 7f83e6922700 1 -- 172.21.15.16:0/4223351161 >> 172.21.15.16:0/4223351161 conn(0x7f83e00a2aa0 msgr2=0x7f83e012e710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:17.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.352+0000 7f83e6922700 1 -- 172.21.15.16:0/4223351161 shutdown_connections 2023-09-30T22:32:17.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.352+0000 7f83e6922700 1 -- 172.21.15.16:0/4223351161 wait complete. 2023-09-30T22:32:17.367 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:32:17.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.387+0000 7f98c2807700 1 -- 172.21.15.16:0/1093015030 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f98bc09f420 con 0x7f98bc09d6f0 2023-09-30T22:32:17.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.387+0000 7f98b8ff9700 1 -- 172.21.15.16:0/1093015030 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f98ac030090 con 0x7f98bc09d6f0 2023-09-30T22:32:17.389 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:32:17.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.389+0000 7f98c2807700 1 -- 172.21.15.16:0/1093015030 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98bc09d6f0 msgr2=0x7f98bc121770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:17.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.389+0000 7f98c2807700 1 --2- 172.21.15.16:0/1093015030 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98bc09d6f0 0x7f98bc121770 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f98ac00c550 tx=0x7f98ac00c860 comp rx=0 tx=0).stop 2023-09-30T22:32:17.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.389+0000 7f98c2807700 1 -- 172.21.15.16:0/1093015030 shutdown_connections 2023-09-30T22:32:17.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.389+0000 7f98c2807700 1 --2- 172.21.15.16:0/1093015030 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f98bc139170 0x7f98bc1221f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:17.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.389+0000 7f98c2807700 1 --2- 172.21.15.16:0/1093015030 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f98bc09d6f0 0x7f98bc121770 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:17.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.389+0000 7f98c2807700 1 --2- 172.21.15.16:0/1093015030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98bc09e1d0 0x7f98bc121cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:17.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.389+0000 7f98c2807700 1 -- 172.21.15.16:0/1093015030 >> 172.21.15.16:0/1093015030 conn(0x7f98bc066f50 msgr2=0x7f98bc09b410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:17.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.390+0000 7f98c2807700 1 -- 172.21.15.16:0/1093015030 shutdown_connections 2023-09-30T22:32:17.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.392+0000 7f98c2807700 1 -- 172.21.15.16:0/1093015030 wait complete. 2023-09-30T22:32:17.412 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:32:17.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.437+0000 7f0c3997f700 1 -- 172.21.15.16:0/2552377320 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0c340a2470 con 0x7f0c3412cf60 2023-09-30T22:32:17.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.437+0000 7f0c1bfff700 1 -- 172.21.15.16:0/2552377320 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0c24025330 con 0x7f0c3412cf60 2023-09-30T22:32:17.439 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:32:17.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.439+0000 7f0c3997f700 1 -- 172.21.15.16:0/2552377320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c3412cf60 msgr2=0x7f0c3409be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:17.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.439+0000 7f0c3997f700 1 --2- 172.21.15.16:0/2552377320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c3412cf60 0x7f0c3409be90 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f0c2400dbc0 tx=0x7f0c2400ded0 comp rx=0 tx=0).stop 2023-09-30T22:32:17.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.439+0000 7f0c3997f700 1 -- 172.21.15.16:0/2552377320 shutdown_connections 2023-09-30T22:32:17.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.440+0000 7f0c3997f700 1 --2- 172.21.15.16:0/2552377320 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0c34130430 0x7f0c341c40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:17.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.440+0000 7f0c3997f700 1 --2- 172.21.15.16:0/2552377320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c3412cf60 0x7f0c3409be90 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:17.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.440+0000 7f0c3997f700 1 --2- 172.21.15.16:0/2552377320 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0c3412f8b0 0x7f0c3409c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:17.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.440+0000 7f0c3997f700 1 -- 172.21.15.16:0/2552377320 >> 172.21.15.16:0/2552377320 conn(0x7f0c340a2aa0 msgr2=0x7f0c340a0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:17.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.440+0000 7f0c3997f700 1 -- 172.21.15.16:0/2552377320 shutdown_connections 2023-09-30T22:32:17.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.440+0000 7f0c3997f700 1 -- 172.21.15.16:0/2552377320 wait complete. 2023-09-30T22:32:17.455 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:32:17.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.474+0000 7f8a8758d700 1 -- 172.21.15.16:0/3930102569 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f8a8009a3d0 con 0x7f8a8006a770 2023-09-30T22:32:17.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.475+0000 7f8a75ffb700 1 -- 172.21.15.16:0/3930102569 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8a7c0232f0 con 0x7f8a8006a770 2023-09-30T22:32:17.476 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:32:17.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.477+0000 7f8a8758d700 1 -- 172.21.15.16:0/3930102569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a8006a770 msgr2=0x7f8a80071590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:17.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.477+0000 7f8a8758d700 1 --2- 172.21.15.16:0/3930102569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a8006a770 0x7f8a80071590 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c00e930 tx=0x7f8a7c00ec40 comp rx=0 tx=0).stop 2023-09-30T22:32:17.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.477+0000 7f8a8758d700 1 -- 172.21.15.16:0/3930102569 shutdown_connections 2023-09-30T22:32:17.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.477+0000 7f8a8758d700 1 --2- 172.21.15.16:0/3930102569 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8a8013d1f0 0x7f8a8013bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:17.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.477+0000 7f8a8758d700 1 --2- 172.21.15.16:0/3930102569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a8006a770 0x7f8a80071590 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:17.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.477+0000 7f8a8758d700 1 --2- 172.21.15.16:0/3930102569 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8a8009dc30 0x7f8a8013b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:17.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.477+0000 7f8a8758d700 1 -- 172.21.15.16:0/3930102569 >> 172.21.15.16:0/3930102569 conn(0x7f8a80066f50 msgr2=0x7f8a8009b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:17.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.477+0000 7f8a8758d700 1 -- 172.21.15.16:0/3930102569 shutdown_connections 2023-09-30T22:32:17.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:17.477+0000 7f8a8758d700 1 -- 172.21.15.16:0/3930102569 wait complete. 2023-09-30T22:32:17.492 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:32:18.369 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:32:18.413 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:32:18.455 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:32:18.492 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:32:18.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.524+0000 7f7d00b40700 1 Processor -- start 2023-09-30T22:32:18.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.524+0000 7f7d00b40700 1 -- start start 2023-09-30T22:32:18.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.525+0000 7f7d00b40700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7cfc1338a0 0x7f7cfc133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:18.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.525+0000 7f7d00b40700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7cfc09e930 0x7f7cfc13b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:18.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.525+0000 7f7d00b40700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cfc06ef30 con 0x7f7cfc09de50 2023-09-30T22:32:18.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.525+0000 7f7d00b40700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc06fef0 con 0x7f7cfc09e930 2023-09-30T22:32:18.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.525+0000 7f7d00b40700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc1334b0 con 0x7f7cfc1338a0 2023-09-30T22:32:18.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.525+0000 7f7cfad9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7cfc09e930 0x7f7cfc13b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:18.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.525+0000 7f7cf9d9b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7cfc09de50 0x7f7cfc09e220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53354/0 (socket says 172.21.15.16:53354) 2023-09-30T22:32:18.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.525+0000 7f7cf9d9b700 1 -- 172.21.15.16:0/2826063628 learned_addr learned my addr 172.21.15.16:0/2826063628 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:18.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.525+0000 7f7cf959a700 1 -- 172.21.15.16:0/2826063628 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 807914493 0 0) 0x7f7cfc06ef30 con 0x7f7cfc09de50 2023-09-30T22:32:18.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.525+0000 7f7cfad9d700 1 -- 172.21.15.16:0/2826063628 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7cfc1338a0 msgr2=0x7f7cfc133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:18.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.525+0000 7f7cfad9d700 1 --2- 172.21.15.16:0/2826063628 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7cfc1338a0 0x7f7cfc133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:18.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.525+0000 7f7cfad9d700 1 -- 172.21.15.16:0/2826063628 >> v1:172.21.15.16:6789/0 conn(0x7f7cfc09de50 legacy=0x7f7cfc09e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:18.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.525+0000 7f7cfad9d700 1 -- 172.21.15.16:0/2826063628 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cfc06fc30 con 0x7f7cfc09e930 2023-09-30T22:32:18.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.526+0000 7f7cfad9d700 1 --2- 172.21.15.16:0/2826063628 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7cfc09e930 0x7f7cfc13b740 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4009dc0 tx=0x7f7cf400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d58dab4db99fda17 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:18.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.526+0000 7f7cf959a700 1 -- 172.21.15.16:0/2826063628 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf400e070 con 0x7f7cfc09e930 2023-09-30T22:32:18.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.526+0000 7f7cf959a700 1 -- 172.21.15.16:0/2826063628 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cf400bbd0 con 0x7f7cfc09e930 2023-09-30T22:32:18.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.526+0000 7f7cf959a700 1 -- 172.21.15.16:0/2826063628 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf4012430 con 0x7f7cfc09e930 2023-09-30T22:32:18.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.527+0000 7f7d00b40700 1 -- 172.21.15.16:0/2826063628 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7cfc09e930 msgr2=0x7f7cfc13b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:18.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.527+0000 7f7d00b40700 1 --2- 172.21.15.16:0/2826063628 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7cfc09e930 0x7f7cfc13b740 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7cf4009dc0 tx=0x7f7cf400b330 comp rx=0 tx=0).stop 2023-09-30T22:32:18.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.527+0000 7f7d00b40700 1 -- 172.21.15.16:0/2826063628 shutdown_connections 2023-09-30T22:32:18.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.527+0000 7f7d00b40700 1 --2- 172.21.15.16:0/2826063628 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7cfc09e930 0x7f7cfc13b740 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:18.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.527+0000 7f7d00b40700 1 --2- 172.21.15.16:0/2826063628 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7cfc1338a0 0x7f7cfc133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:18.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.527+0000 7f7d00b40700 1 -- 172.21.15.16:0/2826063628 >> 172.21.15.16:0/2826063628 conn(0x7f7cfc0a2aa0 msgr2=0x7f7cfc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:18.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.527+0000 7f7d00b40700 1 -- 172.21.15.16:0/2826063628 shutdown_connections 2023-09-30T22:32:18.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.527+0000 7f7d00b40700 1 -- 172.21.15.16:0/2826063628 wait complete. 2023-09-30T22:32:18.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.529+0000 7f7d00b40700 1 Processor -- start 2023-09-30T22:32:18.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.529+0000 7f7d00b40700 1 -- start start 2023-09-30T22:32:18.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.529+0000 7f7d00b40700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7cfc09de50 0x7f7cfc11a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:18.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.529+0000 7f7d00b40700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7cfc09e930 0x7f7cfc11abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:18.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.529+0000 7f7d00b40700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cfc1338a0 0x7f7cfc11b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:18.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.529+0000 7f7d00b40700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc11c1f0 con 0x7f7cfc1338a0 2023-09-30T22:32:18.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.529+0000 7f7d00b40700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc11d440 con 0x7f7cfc09e930 2023-09-30T22:32:18.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.529+0000 7f7d00b40700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7cfc11c7e0 con 0x7f7cfc09de50 2023-09-30T22:32:18.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.529+0000 7f7cfa59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7cfc09de50 0x7f7cfc11a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:18.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.529+0000 7f7cfa59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7cfc09de50 0x7f7cfc11a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:36032/0 (socket says 172.21.15.16:36032) 2023-09-30T22:32:18.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.529+0000 7f7cfa59c700 1 -- 172.21.15.16:0/1909901676 learned_addr learned my addr 172.21.15.16:0/1909901676 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:18.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.530+0000 7f7cfad9d700 1 --2- 172.21.15.16:0/1909901676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cfc1338a0 0x7f7cfc11b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:18.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.530+0000 7f7cfa59c700 1 -- 172.21.15.16:0/1909901676 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7cfc09e930 msgr2=0x7f7cfc11abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:18.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.530+0000 7f7cfa59c700 1 --2- 172.21.15.16:0/1909901676 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7cfc09e930 0x7f7cfc11abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:18.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.530+0000 7f7cfa59c700 1 -- 172.21.15.16:0/1909901676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cfc1338a0 msgr2=0x7f7cfc11b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:18.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.530+0000 7f7cfa59c700 1 --2- 172.21.15.16:0/1909901676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cfc1338a0 0x7f7cfc11b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:18.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.530+0000 7f7cfa59c700 1 -- 172.21.15.16:0/1909901676 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cf4009a70 con 0x7f7cfc09de50 2023-09-30T22:32:18.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.530+0000 7f7cfa59c700 1 --2- 172.21.15.16:0/1909901676 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7cfc09de50 0x7f7cfc11a670 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7cec00b560 tx=0x7f7cec00b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:18.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.530+0000 7f7ceaffd700 1 -- 172.21.15.16:0/1909901676 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cec011830 con 0x7f7cfc09de50 2023-09-30T22:32:18.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.530+0000 7f7ceaffd700 1 -- 172.21.15.16:0/1909901676 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cec0119b0 con 0x7f7cfc09de50 2023-09-30T22:32:18.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.530+0000 7f7ceaffd700 1 -- 172.21.15.16:0/1909901676 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cec011d60 con 0x7f7cfc09de50 2023-09-30T22:32:18.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.530+0000 7f7d00b40700 1 -- 172.21.15.16:0/1909901676 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cfc11d890 con 0x7f7cfc09de50 2023-09-30T22:32:18.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.531+0000 7f7d00b40700 1 -- 172.21.15.16:0/1909901676 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cfc06f640 con 0x7f7cfc09de50 2023-09-30T22:32:18.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.531+0000 7f7ceaffd700 1 -- 172.21.15.16:0/1909901676 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7cec01a400 con 0x7f7cfc09de50 2023-09-30T22:32:18.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.532+0000 7f7ceaffd700 1 -- 172.21.15.16:0/1909901676 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f7cec03f6b0 con 0x7f7cfc09de50 2023-09-30T22:32:18.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.532+0000 7f7d00b40700 1 -- 172.21.15.16:0/1909901676 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cfc074ea0 con 0x7f7cfc09de50 2023-09-30T22:32:18.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.538+0000 7f7ceaffd700 1 -- 172.21.15.16:0/1909901676 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7cec015080 con 0x7f7cfc09de50 2023-09-30T22:32:18.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.569+0000 7f13e01c8700 1 Processor -- start 2023-09-30T22:32:18.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.569+0000 7f13e01c8700 1 -- start start 2023-09-30T22:32:18.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.570+0000 7f13e01c8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f13d809d6f0 0x7f13d809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:18.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.570+0000 7f13e01c8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f13d809e410 0x7f13d8130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:18.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.570+0000 7f13e01c8700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13d809f130 con 0x7f13d809e000 2023-09-30T22:32:18.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.570+0000 7f13e01c8700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13d809f520 con 0x7f13d809e410 2023-09-30T22:32:18.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.570+0000 7f13e01c8700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f13d809e930 con 0x7f13d809d6f0 2023-09-30T22:32:18.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.570+0000 7f13ddf64700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f13d809d6f0 0x7f13d809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:18.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.570+0000 7f13ddf64700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f13d809d6f0 0x7f13d809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:36048/0 (socket says 172.21.15.16:36048) 2023-09-30T22:32:18.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.570+0000 7f13ddf64700 1 -- 172.21.15.16:0/2319931875 learned_addr learned my addr 172.21.15.16:0/2319931875 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:18.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.570+0000 7f13dd763700 1 --1- 172.21.15.16:0/2319931875 >> v1:172.21.15.16:6789/0 conn(0x7f13d809e000 0x7f13d8130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53370/0 (socket says 172.21.15.16:53370) 2023-09-30T22:32:18.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.570+0000 7f13de765700 1 --2- 172.21.15.16:0/2319931875 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f13d809e410 0x7f13d8130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:18.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.570+0000 7f13ddf64700 1 -- 172.21.15.16:0/2319931875 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f13d809e410 msgr2=0x7f13d8130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:18.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.570+0000 7f13ddf64700 1 --2- 172.21.15.16:0/2319931875 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f13d809e410 0x7f13d8130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:18.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.570+0000 7f13dcf62700 1 -- 172.21.15.16:0/2319931875 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2333269542 0 0) 0x7f13d809f130 con 0x7f13d809e000 2023-09-30T22:32:18.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.571+0000 7f13ddf64700 1 -- 172.21.15.16:0/2319931875 >> v1:172.21.15.16:6789/0 conn(0x7f13d809e000 legacy=0x7f13d8130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:18.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.571+0000 7f13ddf64700 1 -- 172.21.15.16:0/2319931875 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d809b2d0 con 0x7f13d809d6f0 2023-09-30T22:32:18.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.571+0000 7f13ddf64700 1 --2- 172.21.15.16:0/2319931875 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f13d809d6f0 0x7f13d809dac0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f13d400bbc0 tx=0x7f13d400bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f095bfe0ac7d214f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:18.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.571+0000 7f13dcf62700 1 -- 172.21.15.16:0/2319931875 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d400dd30 con 0x7f13d809d6f0 2023-09-30T22:32:18.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.571+0000 7f13dcf62700 1 -- 172.21.15.16:0/2319931875 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13d400de90 con 0x7f13d809d6f0 2023-09-30T22:32:18.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.571+0000 7f13dcf62700 1 -- 172.21.15.16:0/2319931875 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d4010690 con 0x7f13d809d6f0 2023-09-30T22:32:18.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.572+0000 7f13e01c8700 1 -- 172.21.15.16:0/2319931875 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f13d809d6f0 msgr2=0x7f13d809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:18.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.572+0000 7f13e01c8700 1 --2- 172.21.15.16:0/2319931875 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f13d809d6f0 0x7f13d809dac0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f13d400bbc0 tx=0x7f13d400bed0 comp rx=0 tx=0).stop 2023-09-30T22:32:18.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.572+0000 7f13e01c8700 1 -- 172.21.15.16:0/2319931875 shutdown_connections 2023-09-30T22:32:18.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.572+0000 7f13e01c8700 1 --2- 172.21.15.16:0/2319931875 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f13d809e410 0x7f13d8130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:18.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.572+0000 7f13e01c8700 1 --2- 172.21.15.16:0/2319931875 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f13d809d6f0 0x7f13d809dac0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:18.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.572+0000 7f13e01c8700 1 -- 172.21.15.16:0/2319931875 >> 172.21.15.16:0/2319931875 conn(0x7f13d8066f50 msgr2=0x7f13d809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:18.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.572+0000 7f13e01c8700 1 -- 172.21.15.16:0/2319931875 shutdown_connections 2023-09-30T22:32:18.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.572+0000 7f13e01c8700 1 -- 172.21.15.16:0/2319931875 wait complete. 2023-09-30T22:32:18.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.573+0000 7f13e01c8700 1 Processor -- start 2023-09-30T22:32:18.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.573+0000 7f13e01c8700 1 -- start start 2023-09-30T22:32:18.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.574+0000 7f13e01c8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13d809d6f0 0x7f13d8122480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:18.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.574+0000 7f13e01c8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f13d809e000 0x7f13d81229c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:18.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.574+0000 7f13e01c8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f13d809e410 0x7f13d8122f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:18.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.574+0000 7f13e01c8700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d80a1d00 con 0x7f13d809d6f0 2023-09-30T22:32:18.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.574+0000 7f13e01c8700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13d8125cc0 con 0x7f13d809e000 2023-09-30T22:32:18.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.574+0000 7f13e01c8700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f13d81303a0 con 0x7f13d809e410 2023-09-30T22:32:18.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.574+0000 7f13de765700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f13d809e410 0x7f13d8122f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:18.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.574+0000 7f13de765700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f13d809e410 0x7f13d8122f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:36056/0 (socket says 172.21.15.16:36056) 2023-09-30T22:32:18.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.574+0000 7f13de765700 1 -- 172.21.15.16:0/3415927716 learned_addr learned my addr 172.21.15.16:0/3415927716 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:18.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.574+0000 7f13dd763700 1 --2- 172.21.15.16:0/3415927716 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f13d809e000 0x7f13d81229c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:18.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.574+0000 7f13ddf64700 1 --2- 172.21.15.16:0/3415927716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13d809d6f0 0x7f13d8122480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:18.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.574+0000 7f13de765700 1 -- 172.21.15.16:0/3415927716 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f13d809e000 msgr2=0x7f13d81229c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:18.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.574+0000 7f13de765700 1 --2- 172.21.15.16:0/3415927716 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f13d809e000 0x7f13d81229c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:18.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.574+0000 7f13de765700 1 -- 172.21.15.16:0/3415927716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13d809d6f0 msgr2=0x7f13d8122480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:18.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.574+0000 7f13de765700 1 --2- 172.21.15.16:0/3415927716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13d809d6f0 0x7f13d8122480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:18.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.574+0000 7f13de765700 1 -- 172.21.15.16:0/3415927716 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d400b7f0 con 0x7f13d809e410 2023-09-30T22:32:18.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.574+0000 7f13dd763700 1 --2- 172.21.15.16:0/3415927716 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f13d809e000 0x7f13d81229c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:32:18.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.575+0000 7f13de765700 1 --2- 172.21.15.16:0/3415927716 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f13d809e410 0x7f13d8122f00 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f13d000efc0 tx=0x7f13d000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:18.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.575+0000 7f13ce7fc700 1 -- 172.21.15.16:0/3415927716 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d0009ea0 con 0x7f13d809e410 2023-09-30T22:32:18.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.575+0000 7f13e01c8700 1 -- 172.21.15.16:0/3415927716 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13d8124640 con 0x7f13d809e410 2023-09-30T22:32:18.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.575+0000 7f13e01c8700 1 -- 172.21.15.16:0/3415927716 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13d8124880 con 0x7f13d809e410 2023-09-30T22:32:18.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.575+0000 7f13ce7fc700 1 -- 172.21.15.16:0/3415927716 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13d000f040 con 0x7f13d809e410 2023-09-30T22:32:18.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.575+0000 7f13ce7fc700 1 -- 172.21.15.16:0/3415927716 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d0014480 con 0x7f13d809e410 2023-09-30T22:32:18.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.576+0000 7f13ce7fc700 1 -- 172.21.15.16:0/3415927716 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f13d00145e0 con 0x7f13d809e410 2023-09-30T22:32:18.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.576+0000 7f13ce7fc700 1 -- 172.21.15.16:0/3415927716 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f13d0040190 con 0x7f13d809e410 2023-09-30T22:32:18.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.577+0000 7f13e01c8700 1 -- 172.21.15.16:0/3415927716 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13c00052e0 con 0x7f13d809e410 2023-09-30T22:32:18.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.582+0000 7f13ce7fc700 1 -- 172.21.15.16:0/3415927716 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f13d0018020 con 0x7f13d809e410 2023-09-30T22:32:18.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.638+0000 7f67359cf700 1 Processor -- start 2023-09-30T22:32:18.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.638+0000 7f67359cf700 1 -- start start 2023-09-30T22:32:18.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.639+0000 7f67359cf700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f673009e1d0 0x7f6730135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:18.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.639+0000 7f67359cf700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6730135e50 0x7f6730138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:18.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.639+0000 7f67359cf700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f673009f130 con 0x7f673009d6f0 2023-09-30T22:32:18.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.639+0000 7f67359cf700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f673009f520 con 0x7f6730135e50 2023-09-30T22:32:18.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.639+0000 7f67359cf700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f673009e930 con 0x7f673009e1d0 2023-09-30T22:32:18.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.639+0000 7f672e7fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f673009e1d0 0x7f6730135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:18.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.639+0000 7f672e7fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f673009e1d0 0x7f6730135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:36072/0 (socket says 172.21.15.16:36072) 2023-09-30T22:32:18.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.639+0000 7f672e7fc700 1 -- 172.21.15.16:0/2641742649 learned_addr learned my addr 172.21.15.16:0/2641742649 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:18.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.639+0000 7f672f7fe700 1 --2- 172.21.15.16:0/2641742649 >> v2:172.21.15.16:3300/0 conn(0x7f6730135e50 0x7f6730138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:18.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.639+0000 7f672e7fc700 1 -- 172.21.15.16:0/2641742649 >> v2:172.21.15.16:3300/0 conn(0x7f6730135e50 msgr2=0x7f6730138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:18.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.639+0000 7f672e7fc700 1 --2- 172.21.15.16:0/2641742649 >> v2:172.21.15.16:3300/0 conn(0x7f6730135e50 0x7f6730138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:18.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.639+0000 7f672e7fc700 1 -- 172.21.15.16:0/2641742649 >> v1:172.21.15.16:6789/0 conn(0x7f673009d6f0 legacy=0x7f673009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:18.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.640+0000 7f672e7fc700 1 -- 172.21.15.16:0/2641742649 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f673009b2d0 con 0x7f673009e1d0 2023-09-30T22:32:18.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.640+0000 7f672e7fc700 1 --2- 172.21.15.16:0/2641742649 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f673009e1d0 0x7f6730135910 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f6724009920 tx=0x7f6724009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20aa097573fe3f9f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:18.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.640+0000 7f672dffb700 1 -- 172.21.15.16:0/2641742649 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672400f8a0 con 0x7f673009e1d0 2023-09-30T22:32:18.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.640+0000 7f672dffb700 1 -- 172.21.15.16:0/2641742649 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f672400fa00 con 0x7f673009e1d0 2023-09-30T22:32:18.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.640+0000 7f672dffb700 1 -- 172.21.15.16:0/2641742649 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672400fd90 con 0x7f673009e1d0 2023-09-30T22:32:18.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.641+0000 7f67359cf700 1 -- 172.21.15.16:0/2641742649 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f673009e1d0 msgr2=0x7f6730135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:18.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.641+0000 7f67359cf700 1 --2- 172.21.15.16:0/2641742649 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f673009e1d0 0x7f6730135910 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f6724009920 tx=0x7f6724009c30 comp rx=0 tx=0).stop 2023-09-30T22:32:18.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.641+0000 7f67359cf700 1 -- 172.21.15.16:0/2641742649 shutdown_connections 2023-09-30T22:32:18.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.641+0000 7f67359cf700 1 --2- 172.21.15.16:0/2641742649 >> v2:172.21.15.16:3300/0 conn(0x7f6730135e50 0x7f6730138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:18.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.641+0000 7f67359cf700 1 --2- 172.21.15.16:0/2641742649 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f673009e1d0 0x7f6730135910 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:18.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.641+0000 7f67359cf700 1 -- 172.21.15.16:0/2641742649 >> 172.21.15.16:0/2641742649 conn(0x7f6730066f50 msgr2=0x7f673009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:18.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.641+0000 7f67359cf700 1 -- 172.21.15.16:0/2641742649 shutdown_connections 2023-09-30T22:32:18.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.641+0000 7f67359cf700 1 -- 172.21.15.16:0/2641742649 wait complete. 2023-09-30T22:32:18.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.642+0000 7f67359cf700 1 Processor -- start 2023-09-30T22:32:18.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.642+0000 7f67359cf700 1 -- start start 2023-09-30T22:32:18.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.643+0000 7f67359cf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f673009d6f0 0x7f67301206a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:18.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.643+0000 7f67359cf700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f673009e1d0 0x7f6730120be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:18.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.643+0000 7f67359cf700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6730135e50 0x7f67301c8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:18.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.643+0000 7f67359cf700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6730121120 con 0x7f673009d6f0 2023-09-30T22:32:18.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.643+0000 7f67359cf700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6730120140 con 0x7f673009e1d0 2023-09-30T22:32:18.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.643+0000 7f67359cf700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6730134570 con 0x7f6730135e50 2023-09-30T22:32:18.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.643+0000 7f672e7fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f673009e1d0 0x7f6730120be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:18.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.643+0000 7f672e7fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f673009e1d0 0x7f6730120be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:59558/0 (socket says 172.21.15.16:59558) 2023-09-30T22:32:18.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.643+0000 7f672e7fc700 1 -- 172.21.15.16:0/1880188639 learned_addr learned my addr 172.21.15.16:0/1880188639 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:18.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.643+0000 7f672f7fe700 1 --2- 172.21.15.16:0/1880188639 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6730135e50 0x7f67301c8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:18.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.643+0000 7f672effd700 1 --2- 172.21.15.16:0/1880188639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f673009d6f0 0x7f67301206a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:18.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.643+0000 7f672e7fc700 1 -- 172.21.15.16:0/1880188639 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6730135e50 msgr2=0x7f67301c8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:18.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.643+0000 7f672e7fc700 1 --2- 172.21.15.16:0/1880188639 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6730135e50 0x7f67301c8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:18.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.643+0000 7f672e7fc700 1 -- 172.21.15.16:0/1880188639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f673009d6f0 msgr2=0x7f67301206a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:18.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.643+0000 7f672e7fc700 1 --2- 172.21.15.16:0/1880188639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f673009d6f0 0x7f67301206a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:18.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.643+0000 7f672e7fc700 1 -- 172.21.15.16:0/1880188639 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6724009550 con 0x7f673009e1d0 2023-09-30T22:32:18.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.644+0000 7f672e7fc700 1 --2- 172.21.15.16:0/1880188639 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f673009e1d0 0x7f6730120be0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f67240098f0 tx=0x7f6724004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:18.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.644+0000 7f6717fff700 1 -- 172.21.15.16:0/1880188639 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67240038a0 con 0x7f673009e1d0 2023-09-30T22:32:18.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.644+0000 7f6717fff700 1 -- 172.21.15.16:0/1880188639 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6724003a00 con 0x7f673009e1d0 2023-09-30T22:32:18.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.644+0000 7f67359cf700 1 -- 172.21.15.16:0/1880188639 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6730123420 con 0x7f673009e1d0 2023-09-30T22:32:18.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.644+0000 7f6717fff700 1 -- 172.21.15.16:0/1880188639 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6724003db0 con 0x7f673009e1d0 2023-09-30T22:32:18.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.644+0000 7f67359cf700 1 -- 172.21.15.16:0/1880188639 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6730121fe0 con 0x7f673009e1d0 2023-09-30T22:32:18.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.645+0000 7f6717fff700 1 -- 172.21.15.16:0/1880188639 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6724019400 con 0x7f673009e1d0 2023-09-30T22:32:18.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.645+0000 7f6717fff700 1 -- 172.21.15.16:0/1880188639 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f67240405f0 con 0x7f673009e1d0 2023-09-30T22:32:18.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.645+0000 7f67359cf700 1 -- 172.21.15.16:0/1880188639 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6730074ea0 con 0x7f673009e1d0 2023-09-30T22:32:18.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.649+0000 7fb1c2919700 1 Processor -- start 2023-09-30T22:32:18.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.649+0000 7fb1c2919700 1 -- start start 2023-09-30T22:32:18.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.650+0000 7fb1c2919700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb1bc12cf60 0x7fb1bc06f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:18.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.650+0000 7fb1c2919700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb1bc12f8b0 0x7fb1bc12fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:18.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.650+0000 7fb1c2919700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb1bc130260 0x7fb1bc136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:18.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.650+0000 7fb1c2919700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1bc12d330 con 0x7fb1bc12f8b0 2023-09-30T22:32:18.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.650+0000 7fb1c2919700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1bc12cd90 con 0x7fb1bc12cf60 2023-09-30T22:32:18.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.650+0000 7fb1c2919700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb1bc1306d0 con 0x7fb1bc130260 2023-09-30T22:32:18.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.650+0000 7fb1bbfff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb1bc12cf60 0x7fb1bc06f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:18.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.650+0000 7fb1bbfff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb1bc12cf60 0x7fb1bc06f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:59574/0 (socket says 172.21.15.16:59574) 2023-09-30T22:32:18.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.650+0000 7fb1bbfff700 1 -- 172.21.15.16:0/2819406255 learned_addr learned my addr 172.21.15.16:0/2819406255 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:18.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.650+0000 7fb1bb7fe700 1 --2- 172.21.15.16:0/2819406255 >> v2:172.21.15.16:3300/0 conn(0x7fb1bc12f8b0 0x7fb1bc12fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:18.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.650+0000 7fb1c0eb6700 1 --2- 172.21.15.16:0/2819406255 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb1bc130260 0x7fb1bc136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:18.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.651+0000 7fb1bb7fe700 1 -- 172.21.15.16:0/2819406255 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb1bc130260 msgr2=0x7fb1bc136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:18.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.651+0000 7fb1bb7fe700 1 --2- 172.21.15.16:0/2819406255 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb1bc130260 0x7fb1bc136c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:18.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.651+0000 7fb1bb7fe700 1 -- 172.21.15.16:0/2819406255 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb1bc12cf60 msgr2=0x7fb1bc06f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:18.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.651+0000 7fb1bb7fe700 1 --2- 172.21.15.16:0/2819406255 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb1bc12cf60 0x7fb1bc06f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:18.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.651+0000 7fb1bb7fe700 1 -- 172.21.15.16:0/2819406255 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1bc071780 con 0x7fb1bc12f8b0 2023-09-30T22:32:18.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.651+0000 7fb1bb7fe700 1 --2- 172.21.15.16:0/2819406255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1bc12f8b0 0x7fb1bc12fd20 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0009870 tx=0x7fb1b0009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83106fc3cbf3aafe server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:18.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.651+0000 7fb1baffd700 1 -- 172.21.15.16:0/2819406255 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b000d8c0 con 0x7fb1bc12f8b0 2023-09-30T22:32:18.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.651+0000 7fb1baffd700 1 -- 172.21.15.16:0/2819406255 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b000c070 con 0x7fb1bc12f8b0 2023-09-30T22:32:18.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.651+0000 7f6717fff700 1 -- 172.21.15.16:0/1880188639 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6724026080 con 0x7f673009e1d0 2023-09-30T22:32:18.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.651+0000 7fb1baffd700 1 -- 172.21.15.16:0/2819406255 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b000dc40 con 0x7fb1bc12f8b0 2023-09-30T22:32:18.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.652+0000 7fb1c2919700 1 -- 172.21.15.16:0/2819406255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1bc12f8b0 msgr2=0x7fb1bc12fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:18.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.652+0000 7fb1c2919700 1 --2- 172.21.15.16:0/2819406255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1bc12f8b0 0x7fb1bc12fd20 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0009870 tx=0x7fb1b0009b80 comp rx=0 tx=0).stop 2023-09-30T22:32:18.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.652+0000 7fb1c2919700 1 -- 172.21.15.16:0/2819406255 shutdown_connections 2023-09-30T22:32:18.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.652+0000 7fb1c2919700 1 --2- 172.21.15.16:0/2819406255 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb1bc12cf60 0x7fb1bc06f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:18.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.652+0000 7fb1c2919700 1 --2- 172.21.15.16:0/2819406255 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb1bc130260 0x7fb1bc136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:18.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.652+0000 7fb1c2919700 1 --2- 172.21.15.16:0/2819406255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1bc12f8b0 0x7fb1bc12fd20 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:18.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.652+0000 7fb1c2919700 1 -- 172.21.15.16:0/2819406255 >> 172.21.15.16:0/2819406255 conn(0x7fb1bc0a2aa0 msgr2=0x7fb1bc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:18.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.652+0000 7fb1c2919700 1 -- 172.21.15.16:0/2819406255 shutdown_connections 2023-09-30T22:32:18.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.652+0000 7fb1c2919700 1 -- 172.21.15.16:0/2819406255 wait complete. 2023-09-30T22:32:18.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.653+0000 7fb1c2919700 1 Processor -- start 2023-09-30T22:32:18.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.653+0000 7fb1c2919700 1 -- start start 2023-09-30T22:32:18.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.654+0000 7fb1c2919700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb1bc12cf60 0x7fb1bc09bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:18.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.654+0000 7fb1c2919700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb1bc12f8b0 0x7fb1bc09c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:18.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.654+0000 7fb1c2919700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1bc130260 0x7fb1bc1c3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:18.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.654+0000 7fb1c2919700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1bc129ca0 con 0x7fb1bc130260 2023-09-30T22:32:18.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.654+0000 7fb1c2919700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1bc09b7f0 con 0x7fb1bc12f8b0 2023-09-30T22:32:18.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.654+0000 7fb1c2919700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb1bc09de10 con 0x7fb1bc12cf60 2023-09-30T22:32:18.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.654+0000 7fb1bb7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb1bc12f8b0 0x7fb1bc09c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:18.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.654+0000 7fb1bb7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb1bc12f8b0 0x7fb1bc09c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:59588/0 (socket says 172.21.15.16:59588) 2023-09-30T22:32:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.654+0000 7fb1bb7fe700 1 -- 172.21.15.16:0/44320210 learned_addr learned my addr 172.21.15.16:0/44320210 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.654+0000 7fb1c0eb6700 1 --2- 172.21.15.16:0/44320210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1bc130260 0x7fb1bc1c3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.654+0000 7fb1bb7fe700 1 -- 172.21.15.16:0/44320210 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb1bc12cf60 msgr2=0x7fb1bc09bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.654+0000 7fb1bb7fe700 1 --2- 172.21.15.16:0/44320210 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb1bc12cf60 0x7fb1bc09bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.654+0000 7fb1bb7fe700 1 -- 172.21.15.16:0/44320210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1bc130260 msgr2=0x7fb1bc1c3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.654+0000 7fb1bb7fe700 1 --2- 172.21.15.16:0/44320210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1bc130260 0x7fb1bc1c3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.654+0000 7fb1bb7fe700 1 -- 172.21.15.16:0/44320210 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b00094a0 con 0x7fb1bc12f8b0 2023-09-30T22:32:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.655+0000 7fb1bb7fe700 1 --2- 172.21.15.16:0/44320210 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb1bc12f8b0 0x7fb1bc09c2e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb1b00090b0 tx=0x7fb1b0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.655+0000 7fb1b8ff9700 1 -- 172.21.15.16:0/44320210 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b001e070 con 0x7fb1bc12f8b0 2023-09-30T22:32:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.655+0000 7fb1b8ff9700 1 -- 172.21.15.16:0/44320210 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b0003d00 con 0x7fb1bc12f8b0 2023-09-30T22:32:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.655+0000 7fb1b8ff9700 1 -- 172.21.15.16:0/44320210 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b000d8c0 con 0x7fb1bc12f8b0 2023-09-30T22:32:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.655+0000 7fb1c2919700 1 -- 172.21.15.16:0/44320210 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1bc09e890 con 0x7fb1bc12f8b0 2023-09-30T22:32:18.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.655+0000 7fb1c2919700 1 -- 172.21.15.16:0/44320210 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1bc09ba80 con 0x7fb1bc12f8b0 2023-09-30T22:32:18.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.656+0000 7fb1b8ff9700 1 -- 172.21.15.16:0/44320210 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb1b000da20 con 0x7fb1bc12f8b0 2023-09-30T22:32:18.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.656+0000 7fb1b8ff9700 1 -- 172.21.15.16:0/44320210 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fb1b0040610 con 0x7fb1bc12f8b0 2023-09-30T22:32:18.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.656+0000 7fb1c2919700 1 -- 172.21.15.16:0/44320210 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1bc09eab0 con 0x7fb1bc12f8b0 2023-09-30T22:32:18.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.661+0000 7fb1b8ff9700 1 -- 172.21.15.16:0/44320210 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb1bc09eab0 con 0x7fb1bc12f8b0 2023-09-30T22:32:18.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.799+0000 7f13e01c8700 1 -- 172.21.15.16:0/3415927716 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f13c0005150 con 0x7f13d809e410 2023-09-30T22:32:18.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.799+0000 7f13ce7fc700 1 -- 172.21.15.16:0/3415927716 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f13d002b090 con 0x7f13d809e410 2023-09-30T22:32:18.801 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:32:18.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.802+0000 7f13e01c8700 1 -- 172.21.15.16:0/3415927716 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f13d809e410 msgr2=0x7f13d8122f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:18.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.802+0000 7f13e01c8700 1 --2- 172.21.15.16:0/3415927716 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f13d809e410 0x7f13d8122f00 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f13d000efc0 tx=0x7f13d000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:32:18.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.802+0000 7f13e01c8700 1 -- 172.21.15.16:0/3415927716 shutdown_connections 2023-09-30T22:32:18.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.802+0000 7f13e01c8700 1 --2- 172.21.15.16:0/3415927716 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f13d809e410 0x7f13d8122f00 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:18.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.802+0000 7f13e01c8700 1 --2- 172.21.15.16:0/3415927716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f13d809d6f0 0x7f13d8122480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:18.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.802+0000 7f13e01c8700 1 --2- 172.21.15.16:0/3415927716 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f13d809e000 0x7f13d81229c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:18.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.802+0000 7f13e01c8700 1 -- 172.21.15.16:0/3415927716 >> 172.21.15.16:0/3415927716 conn(0x7f13d8066f50 msgr2=0x7f13d8130b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:18.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.802+0000 7f13e01c8700 1 -- 172.21.15.16:0/3415927716 shutdown_connections 2023-09-30T22:32:18.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.802+0000 7f13e01c8700 1 -- 172.21.15.16:0/3415927716 wait complete. 2023-09-30T22:32:18.818 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:32:18.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.850+0000 7f7d00b40700 1 -- 172.21.15.16:0/1909901676 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7cfc06b9e0 con 0x7f7cfc09de50 2023-09-30T22:32:18.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.850+0000 7f7ceaffd700 1 -- 172.21.15.16:0/1909901676 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7cec01ae60 con 0x7f7cfc09de50 2023-09-30T22:32:18.852 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:32:18.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.853+0000 7f7d00b40700 1 -- 172.21.15.16:0/1909901676 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7cfc09de50 msgr2=0x7f7cfc11a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:18.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.853+0000 7f7d00b40700 1 --2- 172.21.15.16:0/1909901676 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7cfc09de50 0x7f7cfc11a670 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7cec00b560 tx=0x7f7cec00b870 comp rx=0 tx=0).stop 2023-09-30T22:32:18.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.853+0000 7f7d00b40700 1 -- 172.21.15.16:0/1909901676 shutdown_connections 2023-09-30T22:32:18.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.853+0000 7f7d00b40700 1 --2- 172.21.15.16:0/1909901676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7cfc1338a0 0x7f7cfc11b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:18.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.853+0000 7f7d00b40700 1 --2- 172.21.15.16:0/1909901676 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7cfc09de50 0x7f7cfc11a670 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:18.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.853+0000 7f7d00b40700 1 --2- 172.21.15.16:0/1909901676 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7cfc09e930 0x7f7cfc11abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:18.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.853+0000 7f7d00b40700 1 -- 172.21.15.16:0/1909901676 >> 172.21.15.16:0/1909901676 conn(0x7f7cfc0a2aa0 msgr2=0x7f7cfc130730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:18.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.853+0000 7f7d00b40700 1 -- 172.21.15.16:0/1909901676 shutdown_connections 2023-09-30T22:32:18.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.853+0000 7f7d00b40700 1 -- 172.21.15.16:0/1909901676 wait complete. 2023-09-30T22:32:18.868 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:32:18.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.870+0000 7fb1c2919700 1 -- 172.21.15.16:0/44320210 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb1bc09eab0 con 0x7fb1bc12f8b0 2023-09-30T22:32:18.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.871+0000 7fb1b8ff9700 1 -- 172.21.15.16:0/44320210 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb1b0028020 con 0x7fb1bc12f8b0 2023-09-30T22:32:18.872 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:32:18.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.873+0000 7fb1c2919700 1 -- 172.21.15.16:0/44320210 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb1bc12f8b0 msgr2=0x7fb1bc09c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:18.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.873+0000 7fb1c2919700 1 --2- 172.21.15.16:0/44320210 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb1bc12f8b0 0x7fb1bc09c2e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb1b00090b0 tx=0x7fb1b0004750 comp rx=0 tx=0).stop 2023-09-30T22:32:18.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.873+0000 7fb1c2919700 1 -- 172.21.15.16:0/44320210 shutdown_connections 2023-09-30T22:32:18.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.873+0000 7fb1c2919700 1 --2- 172.21.15.16:0/44320210 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb1bc130260 0x7fb1bc1c3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:18.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.873+0000 7fb1c2919700 1 --2- 172.21.15.16:0/44320210 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb1bc12cf60 0x7fb1bc09bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:18.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.873+0000 7fb1c2919700 1 --2- 172.21.15.16:0/44320210 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb1bc12f8b0 0x7fb1bc09c2e0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:18.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.873+0000 7fb1c2919700 1 -- 172.21.15.16:0/44320210 >> 172.21.15.16:0/44320210 conn(0x7fb1bc0a2aa0 msgr2=0x7fb1bc12a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:18.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.874+0000 7fb1c2919700 1 -- 172.21.15.16:0/44320210 shutdown_connections 2023-09-30T22:32:18.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.874+0000 7fb1c2919700 1 -- 172.21.15.16:0/44320210 wait complete. 2023-09-30T22:32:18.888 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:32:18.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.910+0000 7f67359cf700 1 -- 172.21.15.16:0/1880188639 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f673009df50 con 0x7f673009e1d0 2023-09-30T22:32:18.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.911+0000 7f6717fff700 1 -- 172.21.15.16:0/1880188639 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f672401f2f0 con 0x7f673009e1d0 2023-09-30T22:32:18.912 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:32:18.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.913+0000 7f67359cf700 1 -- 172.21.15.16:0/1880188639 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f673009e1d0 msgr2=0x7f6730120be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:18.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.913+0000 7f67359cf700 1 --2- 172.21.15.16:0/1880188639 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f673009e1d0 0x7f6730120be0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f67240098f0 tx=0x7f6724004750 comp rx=0 tx=0).stop 2023-09-30T22:32:18.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.913+0000 7f67359cf700 1 -- 172.21.15.16:0/1880188639 shutdown_connections 2023-09-30T22:32:18.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.913+0000 7f67359cf700 1 --2- 172.21.15.16:0/1880188639 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6730135e50 0x7f67301c8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:18.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.913+0000 7f67359cf700 1 --2- 172.21.15.16:0/1880188639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f673009d6f0 0x7f67301206a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:18.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.913+0000 7f67359cf700 1 --2- 172.21.15.16:0/1880188639 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f673009e1d0 0x7f6730120be0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:18.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.913+0000 7f67359cf700 1 -- 172.21.15.16:0/1880188639 >> 172.21.15.16:0/1880188639 conn(0x7f6730066f50 msgr2=0x7f6730134c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:18.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.913+0000 7f67359cf700 1 -- 172.21.15.16:0/1880188639 shutdown_connections 2023-09-30T22:32:18.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:18.913+0000 7f67359cf700 1 -- 172.21.15.16:0/1880188639 wait complete. 2023-09-30T22:32:18.928 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:32:19.819 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:32:19.869 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:32:19.889 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:32:19.929 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:32:19.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:19.968+0000 7fe4e68d0700 1 Processor -- start 2023-09-30T22:32:19.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:19.968+0000 7fe4e68d0700 1 -- start start 2023-09-30T22:32:19.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:19.969+0000 7fe4e68d0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe4e0131810 0x7fe4e0131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:19.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:19.969+0000 7fe4e68d0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe4e0099a40 0x7fe4e0099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:19.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:19.969+0000 7fe4e68d0700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4e01313c0 con 0x7fe4e009a3b0 2023-09-30T22:32:19.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:19.969+0000 7fe4e68d0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e009b580 con 0x7fe4e0099a40 2023-09-30T22:32:19.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:19.969+0000 7fe4e68d0700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e006e8c0 con 0x7fe4e0131810 2023-09-30T22:32:19.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:19.969+0000 7fe4df7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe4e0099a40 0x7fe4e0099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:19.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:19.969+0000 7fe4e4e6d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe4e009a3b0 0x7fe4e0138650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53394/0 (socket says 172.21.15.16:53394) 2023-09-30T22:32:19.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:19.969+0000 7fe4e4e6d700 1 -- 172.21.15.16:0/893564581 learned_addr learned my addr 172.21.15.16:0/893564581 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:19.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:19.969+0000 7fe4dffff700 1 --2- 172.21.15.16:0/893564581 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe4e0131810 0x7fe4e0131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:19.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:19.970+0000 7fe4df7fe700 1 -- 172.21.15.16:0/893564581 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe4e0131810 msgr2=0x7fe4e0131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:19.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:19.970+0000 7fe4df7fe700 1 --2- 172.21.15.16:0/893564581 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe4e0131810 0x7fe4e0131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:19.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:19.970+0000 7fe4df7fe700 1 -- 172.21.15.16:0/893564581 >> v1:172.21.15.16:6789/0 conn(0x7fe4e009a3b0 legacy=0x7fe4e0138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:19.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:19.970+0000 7fe4df7fe700 1 -- 172.21.15.16:0/893564581 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4e009a820 con 0x7fe4e0099a40 2023-09-30T22:32:19.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:19.970+0000 7fe4df7fe700 1 --2- 172.21.15.16:0/893564581 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe4e0099a40 0x7fe4e0099e70 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4009870 tx=0x7fe4d4009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5e0bfe57529fb2dc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:19.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:19.970+0000 7fe4deffd700 1 -- 172.21.15.16:0/893564581 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d4011830 con 0x7fe4e0099a40 2023-09-30T22:32:19.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:19.970+0000 7fe4deffd700 1 -- 172.21.15.16:0/893564581 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4d4011990 con 0x7fe4e0099a40 2023-09-30T22:32:19.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:19.970+0000 7fe4deffd700 1 -- 172.21.15.16:0/893564581 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d4011d20 con 0x7fe4e0099a40 2023-09-30T22:32:19.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:19.971+0000 7fe4e68d0700 1 -- 172.21.15.16:0/893564581 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe4e0099a40 msgr2=0x7fe4e0099e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:19.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:19.971+0000 7fe4e68d0700 1 --2- 172.21.15.16:0/893564581 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe4e0099a40 0x7fe4e0099e70 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4009870 tx=0x7fe4d4009b80 comp rx=0 tx=0).stop 2023-09-30T22:32:19.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:19.971+0000 7fe4e68d0700 1 -- 172.21.15.16:0/893564581 shutdown_connections 2023-09-30T22:32:19.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:19.971+0000 7fe4e68d0700 1 --2- 172.21.15.16:0/893564581 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe4e0131810 0x7fe4e0131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:19.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:19.971+0000 7fe4e68d0700 1 --2- 172.21.15.16:0/893564581 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe4e0099a40 0x7fe4e0099e70 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:19.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:19.971+0000 7fe4e68d0700 1 -- 172.21.15.16:0/893564581 >> 172.21.15.16:0/893564581 conn(0x7fe4e00a2aa0 msgr2=0x7fe4e00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:19.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:19.971+0000 7fe4e68d0700 1 -- 172.21.15.16:0/893564581 shutdown_connections 2023-09-30T22:32:19.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:19.971+0000 7fe4e68d0700 1 -- 172.21.15.16:0/893564581 wait complete. 2023-09-30T22:32:19.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:19.972+0000 7fe4e68d0700 1 Processor -- start 2023-09-30T22:32:19.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:19.973+0000 7fe4e68d0700 1 -- start start 2023-09-30T22:32:19.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:19.973+0000 7fe4e68d0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe4e0099a40 0x7fe4e0118cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:19.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:19.973+0000 7fe4e68d0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe4e009a3b0 0x7fe4e0119210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:19.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:19.973+0000 7fe4e68d0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4e0131810 0x7fe4e0119750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:19.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:19.973+0000 7fe4e68d0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e011c250 con 0x7fe4e0131810 2023-09-30T22:32:19.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:19.973+0000 7fe4e68d0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e0130b50 con 0x7fe4e009a3b0 2023-09-30T22:32:19.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:19.973+0000 7fe4e68d0700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe4e011b9e0 con 0x7fe4e0099a40 2023-09-30T22:32:19.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:19.973+0000 7fe4df7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe4e009a3b0 0x7fe4e0119210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:19.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:19.973+0000 7fe4df7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe4e009a3b0 0x7fe4e0119210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:59614/0 (socket says 172.21.15.16:59614) 2023-09-30T22:32:19.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:19.973+0000 7fe4df7fe700 1 -- 172.21.15.16:0/656497948 learned_addr learned my addr 172.21.15.16:0/656497948 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:19.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:19.973+0000 7fe4e4e6d700 1 --2- 172.21.15.16:0/656497948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4e0131810 0x7fe4e0119750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:19.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:19.974+0000 7fe4dffff700 1 --2- 172.21.15.16:0/656497948 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe4e0099a40 0x7fe4e0118cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:19.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:19.974+0000 7fe4df7fe700 1 -- 172.21.15.16:0/656497948 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe4e0099a40 msgr2=0x7fe4e0118cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:19.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:19.974+0000 7fe4df7fe700 1 --2- 172.21.15.16:0/656497948 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe4e0099a40 0x7fe4e0118cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:19.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:19.974+0000 7fe4df7fe700 1 -- 172.21.15.16:0/656497948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4e0131810 msgr2=0x7fe4e0119750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:19.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:19.974+0000 7fe4df7fe700 1 --2- 172.21.15.16:0/656497948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4e0131810 0x7fe4e0119750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:19.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:19.974+0000 7fe4df7fe700 1 -- 172.21.15.16:0/656497948 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d40094a0 con 0x7fe4e009a3b0 2023-09-30T22:32:19.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:19.974+0000 7fe4dffff700 1 --2- 172.21.15.16:0/656497948 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe4e0099a40 0x7fe4e0118cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:32:19.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:19.974+0000 7fe4df7fe700 1 --2- 172.21.15.16:0/656497948 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe4e009a3b0 0x7fe4e0119210 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4000c00 tx=0x7fe4d4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:19.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:19.974+0000 7fe4dcff9700 1 -- 172.21.15.16:0/656497948 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d40036a0 con 0x7fe4e009a3b0 2023-09-30T22:32:19.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:19.974+0000 7fe4dcff9700 1 -- 172.21.15.16:0/656497948 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4d401f070 con 0x7fe4e009a3b0 2023-09-30T22:32:19.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:19.974+0000 7fe4dcff9700 1 -- 172.21.15.16:0/656497948 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d4003a40 con 0x7fe4e009a3b0 2023-09-30T22:32:19.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:19.974+0000 7fe4e68d0700 1 -- 172.21.15.16:0/656497948 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4e006eb00 con 0x7fe4e009a3b0 2023-09-30T22:32:19.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:19.974+0000 7fe4e68d0700 1 -- 172.21.15.16:0/656497948 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4e011a3b0 con 0x7fe4e009a3b0 2023-09-30T22:32:19.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:19.976+0000 7fe4e68d0700 1 -- 172.21.15.16:0/656497948 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4e011bc10 con 0x7fe4e009a3b0 2023-09-30T22:32:19.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:19.976+0000 7fe4dcff9700 1 -- 172.21.15.16:0/656497948 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe4d4003ba0 con 0x7fe4e009a3b0 2023-09-30T22:32:19.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:19.977+0000 7fe4dcff9700 1 -- 172.21.15.16:0/656497948 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fe4d4021070 con 0x7fe4e009a3b0 2023-09-30T22:32:19.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:19.981+0000 7fe4dcff9700 1 -- 172.21.15.16:0/656497948 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe4d400b7e0 con 0x7fe4e009a3b0 2023-09-30T22:32:20.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.019+0000 7f164f6d6700 1 Processor -- start 2023-09-30T22:32:20.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.019+0000 7f164f6d6700 1 -- start start 2023-09-30T22:32:20.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.020+0000 7f164f6d6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f164806a770 0x7f164809d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:20.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.020+0000 7f164f6d6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f164813d1f0 0x7f164813f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:20.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.020+0000 7f164f6d6700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f164809a810 con 0x7f164809dc30 2023-09-30T22:32:20.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.020+0000 7f164f6d6700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f164806ab40 con 0x7f164813d1f0 2023-09-30T22:32:20.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.020+0000 7f164f6d6700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f164809ebc0 con 0x7f164806a770 2023-09-30T22:32:20.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.020+0000 7f164cc71700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f164809dc30 0x7f164809e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53402/0 (socket says 172.21.15.16:53402) 2023-09-30T22:32:20.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.020+0000 7f164cc71700 1 -- 172.21.15.16:0/4072217704 learned_addr learned my addr 172.21.15.16:0/4072217704 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:20.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.020+0000 7f164d472700 1 --2- 172.21.15.16:0/4072217704 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f164806a770 0x7f164809d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:20.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.020+0000 7f164dc73700 1 --2- 172.21.15.16:0/4072217704 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f164813d1f0 0x7f164813f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:20.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.020+0000 7f164dc73700 1 -- 172.21.15.16:0/4072217704 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f164806a770 msgr2=0x7f164809d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:20.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.020+0000 7f164dc73700 1 --2- 172.21.15.16:0/4072217704 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f164806a770 0x7f164809d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:20.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.020+0000 7f164dc73700 1 -- 172.21.15.16:0/4072217704 >> v1:172.21.15.16:6789/0 conn(0x7f164809dc30 legacy=0x7f164809e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:20.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.020+0000 7f164dc73700 1 -- 172.21.15.16:0/4072217704 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f164809e920 con 0x7f164813d1f0 2023-09-30T22:32:20.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.021+0000 7f164dc73700 1 --2- 172.21.15.16:0/4072217704 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f164813d1f0 0x7f164813f5d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1640009dc0 tx=0x7f164000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=789474f252c3a987 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:20.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.021+0000 7f163ffff700 1 -- 172.21.15.16:0/4072217704 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f164000e070 con 0x7f164813d1f0 2023-09-30T22:32:20.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.021+0000 7f163ffff700 1 -- 172.21.15.16:0/4072217704 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1640003a20 con 0x7f164813d1f0 2023-09-30T22:32:20.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.021+0000 7f163ffff700 1 -- 172.21.15.16:0/4072217704 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1640003dd0 con 0x7f164813d1f0 2023-09-30T22:32:20.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.022+0000 7f164f6d6700 1 -- 172.21.15.16:0/4072217704 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f164813d1f0 msgr2=0x7f164813f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:20.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.022+0000 7f164f6d6700 1 --2- 172.21.15.16:0/4072217704 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f164813d1f0 0x7f164813f5d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f1640009dc0 tx=0x7f164000b330 comp rx=0 tx=0).stop 2023-09-30T22:32:20.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.022+0000 7f164f6d6700 1 -- 172.21.15.16:0/4072217704 shutdown_connections 2023-09-30T22:32:20.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.022+0000 7f164f6d6700 1 --2- 172.21.15.16:0/4072217704 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f164813d1f0 0x7f164813f5d0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:20.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.022+0000 7f164f6d6700 1 --2- 172.21.15.16:0/4072217704 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f164806a770 0x7f164809d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:20.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.022+0000 7f164f6d6700 1 -- 172.21.15.16:0/4072217704 >> 172.21.15.16:0/4072217704 conn(0x7f1648066f50 msgr2=0x7f164809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:20.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.022+0000 7f164f6d6700 1 -- 172.21.15.16:0/4072217704 shutdown_connections 2023-09-30T22:32:20.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.022+0000 7f164f6d6700 1 -- 172.21.15.16:0/4072217704 wait complete. 2023-09-30T22:32:20.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.023+0000 7f164f6d6700 1 Processor -- start 2023-09-30T22:32:20.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.023+0000 7f164f6d6700 1 -- start start 2023-09-30T22:32:20.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.024+0000 7f164f6d6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f164806a770 0x7f164813b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:20.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.024+0000 7f164f6d6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f164809dc30 0x7f164813bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:20.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.024+0000 7f164f6d6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f164813d1f0 0x7f164813c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:20.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.024+0000 7f164f6d6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f164811dfe0 con 0x7f164806a770 2023-09-30T22:32:20.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.024+0000 7f164f6d6700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16480723b0 con 0x7f164809dc30 2023-09-30T22:32:20.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.024+0000 7f164f6d6700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f164809f1a0 con 0x7f164813d1f0 2023-09-30T22:32:20.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.024+0000 7f164cc71700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f164809dc30 0x7f164813bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:20.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.024+0000 7f164dc73700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f164813d1f0 0x7f164813c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:20.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.024+0000 7f164cc71700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f164809dc30 0x7f164813bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:59632/0 (socket says 172.21.15.16:59632) 2023-09-30T22:32:20.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.024+0000 7f164cc71700 1 -- 172.21.15.16:0/1227578939 learned_addr learned my addr 172.21.15.16:0/1227578939 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:20.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.024+0000 7f164d472700 1 --2- 172.21.15.16:0/1227578939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f164806a770 0x7f164813b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:20.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.025+0000 7f164cc71700 1 -- 172.21.15.16:0/1227578939 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f164813d1f0 msgr2=0x7f164813c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:20.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.025+0000 7f164cc71700 1 --2- 172.21.15.16:0/1227578939 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f164813d1f0 0x7f164813c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:20.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.025+0000 7f164cc71700 1 -- 172.21.15.16:0/1227578939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f164806a770 msgr2=0x7f164813b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:20.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.025+0000 7f164cc71700 1 --2- 172.21.15.16:0/1227578939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f164806a770 0x7f164813b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:20.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.025+0000 7f164cc71700 1 -- 172.21.15.16:0/1227578939 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1640009a70 con 0x7f164809dc30 2023-09-30T22:32:20.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.025+0000 7f164cc71700 1 --2- 172.21.15.16:0/1227578939 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f164809dc30 0x7f164813bba0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f163800c750 tx=0x7f163800ca60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:20.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.025+0000 7f163dffb700 1 -- 172.21.15.16:0/1227578939 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1638004c80 con 0x7f164809dc30 2023-09-30T22:32:20.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.025+0000 7f163dffb700 1 -- 172.21.15.16:0/1227578939 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1638004de0 con 0x7f164809dc30 2023-09-30T22:32:20.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.025+0000 7f163dffb700 1 -- 172.21.15.16:0/1227578939 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1638007310 con 0x7f164809dc30 2023-09-30T22:32:20.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.026+0000 7f164f6d6700 1 -- 172.21.15.16:0/1227578939 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f164811eac0 con 0x7f164809dc30 2023-09-30T22:32:20.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.026+0000 7f164f6d6700 1 -- 172.21.15.16:0/1227578939 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f164811f560 con 0x7f164809dc30 2023-09-30T22:32:20.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.027+0000 7f163dffb700 1 -- 172.21.15.16:0/1227578939 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1638007470 con 0x7f164809dc30 2023-09-30T22:32:20.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.027+0000 7f163dffb700 1 -- 172.21.15.16:0/1227578939 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f163803f3f0 con 0x7f164809dc30 2023-09-30T22:32:20.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.027+0000 7f164f6d6700 1 -- 172.21.15.16:0/1227578939 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1648074ea0 con 0x7f164809dc30 2023-09-30T22:32:20.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.032+0000 7f163dffb700 1 -- 172.21.15.16:0/1227578939 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1638007720 con 0x7f164809dc30 2023-09-30T22:32:20.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.059+0000 7f1078f90700 1 Processor -- start 2023-09-30T22:32:20.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.060+0000 7f1078f90700 1 -- start start 2023-09-30T22:32:20.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.060+0000 7f1078f90700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f107409d6f0 0x7f107409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:20.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.060+0000 7f1078f90700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f107409e000 0x7f1074130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:20.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.060+0000 7f1078f90700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f107409e410 0x7f1074130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:20.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.060+0000 7f1078f90700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f107409f520 con 0x7f107409d6f0 2023-09-30T22:32:20.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.060+0000 7f1078f90700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f107409f130 con 0x7f107409e410 2023-09-30T22:32:20.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.060+0000 7f1078f90700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f107409e930 con 0x7f107409e000 2023-09-30T22:32:20.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.061+0000 7f107259c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f107409d6f0 0x7f107409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:20.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.061+0000 7f107259c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f107409d6f0 0x7f107409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34642/0 (socket says 172.21.15.16:34642) 2023-09-30T22:32:20.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.061+0000 7f107259c700 1 -- 172.21.15.16:0/743998668 learned_addr learned my addr 172.21.15.16:0/743998668 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:20.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.061+0000 7f1072d9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f107409e410 0x7f1074130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:20.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.061+0000 7f1071d9b700 1 --2- 172.21.15.16:0/743998668 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f107409e000 0x7f1074130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:20.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.061+0000 7f1072d9d700 1 -- 172.21.15.16:0/743998668 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f107409e000 msgr2=0x7f1074130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:20.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.061+0000 7f1072d9d700 1 --2- 172.21.15.16:0/743998668 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f107409e000 0x7f1074130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:20.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.061+0000 7f1072d9d700 1 -- 172.21.15.16:0/743998668 >> v2:172.21.15.16:3300/0 conn(0x7f107409d6f0 msgr2=0x7f107409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:20.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.061+0000 7f1072d9d700 1 --2- 172.21.15.16:0/743998668 >> v2:172.21.15.16:3300/0 conn(0x7f107409d6f0 0x7f107409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:20.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.061+0000 7f1072d9d700 1 -- 172.21.15.16:0/743998668 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f107409b2d0 con 0x7f107409e410 2023-09-30T22:32:20.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.061+0000 7f107259c700 1 --2- 172.21.15.16:0/743998668 >> v2:172.21.15.16:3300/0 conn(0x7f107409d6f0 0x7f107409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:20.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.061+0000 7f1072d9d700 1 --2- 172.21.15.16:0/743998668 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f107409e410 0x7f1074130ba0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f106c009d80 tx=0x7f106c00e430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc22a914de8e115d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:20.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.062+0000 7f107159a700 1 -- 172.21.15.16:0/743998668 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106c00ee20 con 0x7f107409e410 2023-09-30T22:32:20.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.062+0000 7f107159a700 1 -- 172.21.15.16:0/743998668 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f106c0133f0 con 0x7f107409e410 2023-09-30T22:32:20.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.062+0000 7f107159a700 1 -- 172.21.15.16:0/743998668 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106c013780 con 0x7f107409e410 2023-09-30T22:32:20.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.062+0000 7f1078f90700 1 -- 172.21.15.16:0/743998668 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f107409e410 msgr2=0x7f1074130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:20.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.062+0000 7f1078f90700 1 --2- 172.21.15.16:0/743998668 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f107409e410 0x7f1074130ba0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f106c009d80 tx=0x7f106c00e430 comp rx=0 tx=0).stop 2023-09-30T22:32:20.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.063+0000 7f1078f90700 1 -- 172.21.15.16:0/743998668 shutdown_connections 2023-09-30T22:32:20.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.063+0000 7f1078f90700 1 --2- 172.21.15.16:0/743998668 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f107409e410 0x7f1074130ba0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:20.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.063+0000 7f1078f90700 1 --2- 172.21.15.16:0/743998668 >> v2:172.21.15.16:3300/0 conn(0x7f107409d6f0 0x7f107409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:20.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.063+0000 7f1078f90700 1 --2- 172.21.15.16:0/743998668 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f107409e000 0x7f1074130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:20.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.063+0000 7f1078f90700 1 -- 172.21.15.16:0/743998668 >> 172.21.15.16:0/743998668 conn(0x7f1074066f50 msgr2=0x7f107409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:20.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.063+0000 7f1078f90700 1 -- 172.21.15.16:0/743998668 shutdown_connections 2023-09-30T22:32:20.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.063+0000 7f1078f90700 1 -- 172.21.15.16:0/743998668 wait complete. 2023-09-30T22:32:20.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.064+0000 7f1078f90700 1 Processor -- start 2023-09-30T22:32:20.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.064+0000 7f1078f90700 1 -- start start 2023-09-30T22:32:20.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.065+0000 7f1078f90700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f107409d6f0 0x7f10741221b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:20.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.065+0000 7f1078f90700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f107409e000 0x7f10741226f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:20.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.065+0000 7f1078f90700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f107409e410 0x7f1074122c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:20.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.065+0000 7f1078f90700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1074125cd0 con 0x7f107409e000 2023-09-30T22:32:20.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.065+0000 7f1078f90700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1074123dc0 con 0x7f107409d6f0 2023-09-30T22:32:20.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.065+0000 7f1078f90700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f10740a16b0 con 0x7f107409e410 2023-09-30T22:32:20.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.065+0000 7f1071d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f107409e000 0x7f10741226f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:20.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.065+0000 7f1071d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f107409e000 0x7f10741226f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34656/0 (socket says 172.21.15.16:34656) 2023-09-30T22:32:20.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.065+0000 7f1071d9b700 1 -- 172.21.15.16:0/1579699535 learned_addr learned my addr 172.21.15.16:0/1579699535 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:20.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.065+0000 7f1072d9d700 1 --2- 172.21.15.16:0/1579699535 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f107409e410 0x7f1074122c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:20.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.066+0000 7f1071d9b700 1 -- 172.21.15.16:0/1579699535 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f107409e410 msgr2=0x7f1074122c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:20.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.066+0000 7f1071d9b700 1 --2- 172.21.15.16:0/1579699535 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f107409e410 0x7f1074122c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:20.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.066+0000 7f1071d9b700 1 -- 172.21.15.16:0/1579699535 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f107409d6f0 msgr2=0x7f10741221b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:20.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.066+0000 7f1071d9b700 1 --2- 172.21.15.16:0/1579699535 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f107409d6f0 0x7f10741221b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:20.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.066+0000 7f1071d9b700 1 -- 172.21.15.16:0/1579699535 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1064009550 con 0x7f107409e000 2023-09-30T22:32:20.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.066+0000 7f1071d9b700 1 --2- 172.21.15.16:0/1579699535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f107409e000 0x7f10741226f0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f106800e810 tx=0x7f106800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:20.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.066+0000 7f1062ffd700 1 -- 172.21.15.16:0/1579699535 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1068013610 con 0x7f107409e000 2023-09-30T22:32:20.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.066+0000 7f1062ffd700 1 -- 172.21.15.16:0/1579699535 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1068013770 con 0x7f107409e000 2023-09-30T22:32:20.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.066+0000 7f1078f90700 1 -- 172.21.15.16:0/1579699535 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f106c0099e0 con 0x7f107409e000 2023-09-30T22:32:20.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.066+0000 7f1062ffd700 1 -- 172.21.15.16:0/1579699535 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1068013b20 con 0x7f107409e000 2023-09-30T22:32:20.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.066+0000 7f1078f90700 1 -- 172.21.15.16:0/1579699535 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1074124580 con 0x7f107409e000 2023-09-30T22:32:20.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.068+0000 7f1062ffd700 1 -- 172.21.15.16:0/1579699535 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1068013c80 con 0x7f107409e000 2023-09-30T22:32:20.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.068+0000 7f1078f90700 1 -- 172.21.15.16:0/1579699535 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1074074ea0 con 0x7f107409e000 2023-09-30T22:32:20.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.068+0000 7f1062ffd700 1 -- 172.21.15.16:0/1579699535 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f106803f590 con 0x7f107409e000 2023-09-30T22:32:20.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.073+0000 7f1062ffd700 1 -- 172.21.15.16:0/1579699535 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1068012370 con 0x7f107409e000 2023-09-30T22:32:20.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.121+0000 7f5d2b7e9700 1 Processor -- start 2023-09-30T22:32:20.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.122+0000 7f5d2b7e9700 1 -- start start 2023-09-30T22:32:20.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.122+0000 7f5d2b7e9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d2409d6f0 0x7f5d2409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:20.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.123+0000 7f5d2b7e9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d2409e410 0x7f5d24134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:20.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.123+0000 7f5d2b7e9700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d2409b120 con 0x7f5d2409e000 2023-09-30T22:32:20.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.123+0000 7f5d2b7e9700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2406b460 con 0x7f5d2409e410 2023-09-30T22:32:20.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.123+0000 7f5d2b7e9700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2409a7e0 con 0x7f5d2409d6f0 2023-09-30T22:32:20.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.123+0000 7f5d28d84700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5d2409e000 0x7f5d241339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53414/0 (socket says 172.21.15.16:53414) 2023-09-30T22:32:20.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.123+0000 7f5d28d84700 1 -- 172.21.15.16:0/1509972148 learned_addr learned my addr 172.21.15.16:0/1509972148 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:20.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.123+0000 7f5d29d86700 1 --2- 172.21.15.16:0/1509972148 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d2409e410 0x7f5d24134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:20.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.123+0000 7f5d29585700 1 --2- 172.21.15.16:0/1509972148 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d2409d6f0 0x7f5d2409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:20.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.123+0000 7f5d1bfff700 1 -- 172.21.15.16:0/1509972148 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1818901574 0 0) 0x7f5d2409b120 con 0x7f5d2409e000 2023-09-30T22:32:20.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.123+0000 7f5d1bfff700 1 -- 172.21.15.16:0/1509972148 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d100035b0 con 0x7f5d2409e000 2023-09-30T22:32:20.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.123+0000 7f5d29d86700 1 -- 172.21.15.16:0/1509972148 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d2409d6f0 msgr2=0x7f5d2409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:20.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.123+0000 7f5d29d86700 1 --2- 172.21.15.16:0/1509972148 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d2409d6f0 0x7f5d2409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:20.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.123+0000 7f5d29d86700 1 -- 172.21.15.16:0/1509972148 >> v1:172.21.15.16:6789/0 conn(0x7f5d2409e000 legacy=0x7f5d241339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:20.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.123+0000 7f5d29d86700 1 -- 172.21.15.16:0/1509972148 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d2409a500 con 0x7f5d2409e410 2023-09-30T22:32:20.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.123+0000 7f5d29585700 1 --2- 172.21.15.16:0/1509972148 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d2409d6f0 0x7f5d2409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:20.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.124+0000 7f5d29d86700 1 --2- 172.21.15.16:0/1509972148 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d2409e410 0x7f5d24134100 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c009dc0 tx=0x7f5d1c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae4046554294104 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:20.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.124+0000 7f5d1bfff700 1 -- 172.21.15.16:0/1509972148 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c00e070 con 0x7f5d2409e410 2023-09-30T22:32:20.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.124+0000 7f5d1bfff700 1 -- 172.21.15.16:0/1509972148 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d1c00bb20 con 0x7f5d2409e410 2023-09-30T22:32:20.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.124+0000 7f5d1bfff700 1 -- 172.21.15.16:0/1509972148 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c012430 con 0x7f5d2409e410 2023-09-30T22:32:20.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.124+0000 7f5d2b7e9700 1 -- 172.21.15.16:0/1509972148 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d2409e410 msgr2=0x7f5d24134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:20.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.124+0000 7f5d2b7e9700 1 --2- 172.21.15.16:0/1509972148 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d2409e410 0x7f5d24134100 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c009dc0 tx=0x7f5d1c00b330 comp rx=0 tx=0).stop 2023-09-30T22:32:20.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.125+0000 7f5d2b7e9700 1 -- 172.21.15.16:0/1509972148 shutdown_connections 2023-09-30T22:32:20.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.125+0000 7f5d2b7e9700 1 --2- 172.21.15.16:0/1509972148 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d2409e410 0x7f5d24134100 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:20.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.125+0000 7f5d2b7e9700 1 --2- 172.21.15.16:0/1509972148 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d2409d6f0 0x7f5d2409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:20.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.125+0000 7f5d2b7e9700 1 -- 172.21.15.16:0/1509972148 >> 172.21.15.16:0/1509972148 conn(0x7f5d24066f50 msgr2=0x7f5d2409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:20.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.125+0000 7f5d2b7e9700 1 -- 172.21.15.16:0/1509972148 shutdown_connections 2023-09-30T22:32:20.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.125+0000 7f5d2b7e9700 1 -- 172.21.15.16:0/1509972148 wait complete. 2023-09-30T22:32:20.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.126+0000 7f5d2b7e9700 1 Processor -- start 2023-09-30T22:32:20.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.126+0000 7f5d2b7e9700 1 -- start start 2023-09-30T22:32:20.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.126+0000 7f5d2b7e9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d2409d6f0 0x7f5d241222d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:20.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.126+0000 7f5d2b7e9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d2409e000 0x7f5d24122810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:20.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.126+0000 7f5d2b7e9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d2409e410 0x7f5d24122d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:20.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.126+0000 7f5d2b7e9700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d24124d60 con 0x7f5d2409d6f0 2023-09-30T22:32:20.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.126+0000 7f5d2b7e9700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d24123e60 con 0x7f5d2409e410 2023-09-30T22:32:20.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.126+0000 7f5d2b7e9700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2409c710 con 0x7f5d2409e000 2023-09-30T22:32:20.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.126+0000 7f5d29d86700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d2409e410 0x7f5d24122d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:20.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.126+0000 7f5d29d86700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d2409e410 0x7f5d24122d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:59658/0 (socket says 172.21.15.16:59658) 2023-09-30T22:32:20.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.126+0000 7f5d29d86700 1 -- 172.21.15.16:0/1753617733 learned_addr learned my addr 172.21.15.16:0/1753617733 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:20.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.127+0000 7f5d29585700 1 --2- 172.21.15.16:0/1753617733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d2409d6f0 0x7f5d241222d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:20.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.127+0000 7f5d28d84700 1 --2- 172.21.15.16:0/1753617733 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d2409e000 0x7f5d24122810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:20.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.127+0000 7f5d29d86700 1 -- 172.21.15.16:0/1753617733 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d2409e000 msgr2=0x7f5d24122810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:20.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.127+0000 7f5d29d86700 1 --2- 172.21.15.16:0/1753617733 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d2409e000 0x7f5d24122810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:20.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.127+0000 7f5d29d86700 1 -- 172.21.15.16:0/1753617733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d2409d6f0 msgr2=0x7f5d241222d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:20.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.127+0000 7f5d29d86700 1 --2- 172.21.15.16:0/1753617733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d2409d6f0 0x7f5d241222d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:20.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.127+0000 7f5d29d86700 1 -- 172.21.15.16:0/1753617733 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d200094a0 con 0x7f5d2409e410 2023-09-30T22:32:20.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.127+0000 7f5d29d86700 1 --2- 172.21.15.16:0/1753617733 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d2409e410 0x7f5d24122d50 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c014040 tx=0x7f5d1c004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:20.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.127+0000 7f5d19ffb700 1 -- 172.21.15.16:0/1753617733 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c004930 con 0x7f5d2409e410 2023-09-30T22:32:20.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.127+0000 7f5d19ffb700 1 -- 172.21.15.16:0/1753617733 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d1c020070 con 0x7f5d2409e410 2023-09-30T22:32:20.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.127+0000 7f5d2b7e9700 1 -- 172.21.15.16:0/1753617733 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d1c009a70 con 0x7f5d2409e410 2023-09-30T22:32:20.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.127+0000 7f5d19ffb700 1 -- 172.21.15.16:0/1753617733 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c0276c0 con 0x7f5d2409e410 2023-09-30T22:32:20.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.128+0000 7f5d2b7e9700 1 -- 172.21.15.16:0/1753617733 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d241257d0 con 0x7f5d2409e410 2023-09-30T22:32:20.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.129+0000 7f5d19ffb700 1 -- 172.21.15.16:0/1753617733 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5d1c012430 con 0x7f5d2409e410 2023-09-30T22:32:20.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.129+0000 7f5d2b7e9700 1 -- 172.21.15.16:0/1753617733 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d24125140 con 0x7f5d2409e410 2023-09-30T22:32:20.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.129+0000 7f5d19ffb700 1 -- 172.21.15.16:0/1753617733 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f5d1c022070 con 0x7f5d2409e410 2023-09-30T22:32:20.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.138+0000 7f5d19ffb700 1 -- 172.21.15.16:0/1753617733 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5d1c012e40 con 0x7f5d2409e410 2023-09-30T22:32:20.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.202+0000 7fe4e68d0700 1 -- 172.21.15.16:0/656497948 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe4e011bc10 con 0x7fe4e009a3b0 2023-09-30T22:32:20.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.202+0000 7fe4dcff9700 1 -- 172.21.15.16:0/656497948 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe4e011bc10 con 0x7fe4e009a3b0 2023-09-30T22:32:20.204 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:32:20.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.205+0000 7fe4e68d0700 1 -- 172.21.15.16:0/656497948 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe4e009a3b0 msgr2=0x7fe4e0119210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:20.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.205+0000 7fe4e68d0700 1 --2- 172.21.15.16:0/656497948 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe4e009a3b0 0x7fe4e0119210 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4000c00 tx=0x7fe4d4004750 comp rx=0 tx=0).stop 2023-09-30T22:32:20.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.205+0000 7fe4e68d0700 1 -- 172.21.15.16:0/656497948 shutdown_connections 2023-09-30T22:32:20.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.205+0000 7fe4e68d0700 1 --2- 172.21.15.16:0/656497948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4e0131810 0x7fe4e0119750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:20.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.205+0000 7fe4e68d0700 1 --2- 172.21.15.16:0/656497948 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe4e0099a40 0x7fe4e0118cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:20.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.205+0000 7fe4e68d0700 1 --2- 172.21.15.16:0/656497948 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe4e009a3b0 0x7fe4e0119210 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:20.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.205+0000 7fe4e68d0700 1 -- 172.21.15.16:0/656497948 >> 172.21.15.16:0/656497948 conn(0x7fe4e00a2aa0 msgr2=0x7fe4e012e780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:20.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.205+0000 7fe4e68d0700 1 -- 172.21.15.16:0/656497948 shutdown_connections 2023-09-30T22:32:20.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.205+0000 7fe4e68d0700 1 -- 172.21.15.16:0/656497948 wait complete. 2023-09-30T22:32:20.226 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:32:20.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.249+0000 7f164f6d6700 1 -- 172.21.15.16:0/1227578939 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f164809a3d0 con 0x7f164809dc30 2023-09-30T22:32:20.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.249+0000 7f163dffb700 1 -- 172.21.15.16:0/1227578939 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1638011360 con 0x7f164809dc30 2023-09-30T22:32:20.251 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:32:20.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.252+0000 7f164f6d6700 1 -- 172.21.15.16:0/1227578939 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f164809dc30 msgr2=0x7f164813bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:20.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.252+0000 7f164f6d6700 1 --2- 172.21.15.16:0/1227578939 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f164809dc30 0x7f164813bba0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f163800c750 tx=0x7f163800ca60 comp rx=0 tx=0).stop 2023-09-30T22:32:20.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.252+0000 7f164f6d6700 1 -- 172.21.15.16:0/1227578939 shutdown_connections 2023-09-30T22:32:20.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.252+0000 7f164f6d6700 1 --2- 172.21.15.16:0/1227578939 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f164813d1f0 0x7f164813c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:20.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.252+0000 7f164f6d6700 1 --2- 172.21.15.16:0/1227578939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f164806a770 0x7f164813b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:20.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.252+0000 7f164f6d6700 1 --2- 172.21.15.16:0/1227578939 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f164809dc30 0x7f164813bba0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:20.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.252+0000 7f164f6d6700 1 -- 172.21.15.16:0/1227578939 >> 172.21.15.16:0/1227578939 conn(0x7f1648066f50 msgr2=0x7f16480a2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:20.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.252+0000 7f164f6d6700 1 -- 172.21.15.16:0/1227578939 shutdown_connections 2023-09-30T22:32:20.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.252+0000 7f164f6d6700 1 -- 172.21.15.16:0/1227578939 wait complete. 2023-09-30T22:32:20.267 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:32:20.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.302+0000 7f1078f90700 1 -- 172.21.15.16:0/1579699535 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f107406c360 con 0x7f107409e000 2023-09-30T22:32:20.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.302+0000 7f1062ffd700 1 -- 172.21.15.16:0/1579699535 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f106802b090 con 0x7f107409e000 2023-09-30T22:32:20.304 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:32:20.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.304+0000 7f1078f90700 1 -- 172.21.15.16:0/1579699535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f107409e000 msgr2=0x7f10741226f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:20.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.304+0000 7f1078f90700 1 --2- 172.21.15.16:0/1579699535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f107409e000 0x7f10741226f0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f106800e810 tx=0x7f106800eb20 comp rx=0 tx=0).stop 2023-09-30T22:32:20.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.304+0000 7f1078f90700 1 -- 172.21.15.16:0/1579699535 shutdown_connections 2023-09-30T22:32:20.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.304+0000 7f1078f90700 1 --2- 172.21.15.16:0/1579699535 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f107409d6f0 0x7f10741221b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:20.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.304+0000 7f1078f90700 1 --2- 172.21.15.16:0/1579699535 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f107409e410 0x7f1074122c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:20.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.304+0000 7f1078f90700 1 --2- 172.21.15.16:0/1579699535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f107409e000 0x7f10741226f0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:20.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.304+0000 7f1078f90700 1 -- 172.21.15.16:0/1579699535 >> 172.21.15.16:0/1579699535 conn(0x7f1074066f50 msgr2=0x7f107406a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:20.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.304+0000 7f1078f90700 1 -- 172.21.15.16:0/1579699535 shutdown_connections 2023-09-30T22:32:20.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.305+0000 7f1078f90700 1 -- 172.21.15.16:0/1579699535 wait complete. 2023-09-30T22:32:20.325 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:32:20.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.363+0000 7f5d2b7e9700 1 -- 172.21.15.16:0/1753617733 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5d2411f030 con 0x7f5d2409e410 2023-09-30T22:32:20.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.363+0000 7f5d19ffb700 1 -- 172.21.15.16:0/1753617733 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5d1c02f390 con 0x7f5d2409e410 2023-09-30T22:32:20.365 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:32:20.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.365+0000 7f5d2b7e9700 1 -- 172.21.15.16:0/1753617733 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d2409e410 msgr2=0x7f5d24122d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:20.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.365+0000 7f5d2b7e9700 1 --2- 172.21.15.16:0/1753617733 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d2409e410 0x7f5d24122d50 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c014040 tx=0x7f5d1c004ad0 comp rx=0 tx=0).stop 2023-09-30T22:32:20.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.365+0000 7f5d2b7e9700 1 -- 172.21.15.16:0/1753617733 shutdown_connections 2023-09-30T22:32:20.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.365+0000 7f5d2b7e9700 1 --2- 172.21.15.16:0/1753617733 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d2409e410 0x7f5d24122d50 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:20.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.365+0000 7f5d2b7e9700 1 --2- 172.21.15.16:0/1753617733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d2409d6f0 0x7f5d241222d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:20.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.365+0000 7f5d2b7e9700 1 --2- 172.21.15.16:0/1753617733 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d2409e000 0x7f5d24122810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:20.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.366+0000 7f5d2b7e9700 1 -- 172.21.15.16:0/1753617733 >> 172.21.15.16:0/1753617733 conn(0x7f5d24066f50 msgr2=0x7f5d24133f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:20.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.366+0000 7f5d2b7e9700 1 -- 172.21.15.16:0/1753617733 shutdown_connections 2023-09-30T22:32:20.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:20.366+0000 7f5d2b7e9700 1 -- 172.21.15.16:0/1753617733 wait complete. 2023-09-30T22:32:20.380 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:32:21.226 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:32:21.268 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:32:21.326 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:32:21.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.377+0000 7ff93e968700 1 Processor -- start 2023-09-30T22:32:21.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.377+0000 7ff93e968700 1 -- start start 2023-09-30T22:32:21.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.377+0000 7ff93e968700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff938133420 0x7ff9381337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:21.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.377+0000 7ff93e968700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff938134260 0x7ff9381306a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:21.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.377+0000 7ff93e968700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff93806b990 con 0x7ff938133e30 2023-09-30T22:32:21.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.377+0000 7ff93e968700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9380a0590 con 0x7ff938133420 2023-09-30T22:32:21.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.377+0000 7ff93e968700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff93809fb80 con 0x7ff938134260 2023-09-30T22:32:21.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.377+0000 7ff937fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff938133420 0x7ff9381337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:21.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.377+0000 7ff937fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff938133420 0x7ff9381337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34670/0 (socket says 172.21.15.16:34670) 2023-09-30T22:32:21.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.377+0000 7ff937fff700 1 -- 172.21.15.16:0/529595151 learned_addr learned my addr 172.21.15.16:0/529595151 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:21.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.377+0000 7ff9377fe700 1 --1- 172.21.15.16:0/529595151 >> v1:172.21.15.16:6789/0 conn(0x7ff938133e30 0x7ff93812ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53424/0 (socket says 172.21.15.16:53424) 2023-09-30T22:32:21.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.377+0000 7ff93cf05700 1 --2- 172.21.15.16:0/529595151 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff938134260 0x7ff9381306a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:21.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.378+0000 7ff937fff700 1 -- 172.21.15.16:0/529595151 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff938134260 msgr2=0x7ff9381306a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:21.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.378+0000 7ff937fff700 1 --2- 172.21.15.16:0/529595151 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff938134260 0x7ff9381306a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:21.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.378+0000 7ff937fff700 1 -- 172.21.15.16:0/529595151 >> v1:172.21.15.16:6789/0 conn(0x7ff938133e30 legacy=0x7ff93812ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:21.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.378+0000 7ff936ffd700 1 -- 172.21.15.16:0/529595151 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1435650831 0 0) 0x7ff93806b990 con 0x7ff938133e30 2023-09-30T22:32:21.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.378+0000 7ff937fff700 1 -- 172.21.15.16:0/529595151 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9380a0030 con 0x7ff938133420 2023-09-30T22:32:21.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.378+0000 7ff937fff700 1 --2- 172.21.15.16:0/529595151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff938133420 0x7ff9381337f0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7ff9280098f0 tx=0x7ff928009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f3689ad6e3a0382 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:21.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.378+0000 7ff936ffd700 1 -- 172.21.15.16:0/529595151 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff928011830 con 0x7ff938133420 2023-09-30T22:32:21.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.378+0000 7ff936ffd700 1 -- 172.21.15.16:0/529595151 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff928011990 con 0x7ff938133420 2023-09-30T22:32:21.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.378+0000 7ff936ffd700 1 -- 172.21.15.16:0/529595151 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff928011d20 con 0x7ff938133420 2023-09-30T22:32:21.382 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:32:21.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.379+0000 7ff93e968700 1 -- 172.21.15.16:0/529595151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff938133420 msgr2=0x7ff9381337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:21.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.379+0000 7ff93e968700 1 --2- 172.21.15.16:0/529595151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff938133420 0x7ff9381337f0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7ff9280098f0 tx=0x7ff928009c00 comp rx=0 tx=0).stop 2023-09-30T22:32:21.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.379+0000 7ff93e968700 1 -- 172.21.15.16:0/529595151 shutdown_connections 2023-09-30T22:32:21.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.379+0000 7ff93e968700 1 --2- 172.21.15.16:0/529595151 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff938134260 0x7ff9381306a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:21.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.379+0000 7ff93e968700 1 --2- 172.21.15.16:0/529595151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff938133420 0x7ff9381337f0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:21.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.379+0000 7ff93e968700 1 -- 172.21.15.16:0/529595151 >> 172.21.15.16:0/529595151 conn(0x7ff9380a4210 msgr2=0x7ff9380a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:21.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.379+0000 7ff93e968700 1 -- 172.21.15.16:0/529595151 shutdown_connections 2023-09-30T22:32:21.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.380+0000 7ff93e968700 1 -- 172.21.15.16:0/529595151 wait complete. 2023-09-30T22:32:21.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.381+0000 7ff93e968700 1 Processor -- start 2023-09-30T22:32:21.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.381+0000 7ff93e968700 1 -- start start 2023-09-30T22:32:21.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.382+0000 7ff93e968700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff938133420 0x7ff93809c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:21.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.382+0000 7ff93e968700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff938133e30 0x7ff93809c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:21.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.382+0000 7ff93e968700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff938134260 0x7ff93809cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:21.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.382+0000 7ff93e968700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff938131690 con 0x7ff938133e30 2023-09-30T22:32:21.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.382+0000 7ff93e968700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9381200b0 con 0x7ff938134260 2023-09-30T22:32:21.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.382+0000 7ff93e968700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff93811fe20 con 0x7ff938133420 2023-09-30T22:32:21.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.382+0000 7ff9377fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff938133e30 0x7ff93809c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:21.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.382+0000 7ff937fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff938133420 0x7ff93809c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:21.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.382+0000 7ff937fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff938133420 0x7ff93809c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:36180/0 (socket says 172.21.15.16:36180) 2023-09-30T22:32:21.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.382+0000 7ff937fff700 1 -- 172.21.15.16:0/939451742 learned_addr learned my addr 172.21.15.16:0/939451742 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:21.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.382+0000 7ff93cf05700 1 --2- 172.21.15.16:0/939451742 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff938134260 0x7ff93809cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:21.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.383+0000 7ff9377fe700 1 -- 172.21.15.16:0/939451742 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff938133420 msgr2=0x7ff93809c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:21.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.383+0000 7ff9377fe700 1 --2- 172.21.15.16:0/939451742 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff938133420 0x7ff93809c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:21.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.383+0000 7ff9377fe700 1 -- 172.21.15.16:0/939451742 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff938134260 msgr2=0x7ff93809cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:21.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.383+0000 7ff9377fe700 1 --2- 172.21.15.16:0/939451742 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff938134260 0x7ff93809cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:21.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.383+0000 7ff9377fe700 1 -- 172.21.15.16:0/939451742 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff928009550 con 0x7ff938133e30 2023-09-30T22:32:21.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.383+0000 7ff93cf05700 1 --2- 172.21.15.16:0/939451742 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff938134260 0x7ff93809cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:32:21.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.383+0000 7ff937fff700 1 --2- 172.21.15.16:0/939451742 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff938133420 0x7ff93809c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:21.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.383+0000 7ff9377fe700 1 --2- 172.21.15.16:0/939451742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff938133e30 0x7ff93809c9c0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7ff92c00c470 tx=0x7ff92c00c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:21.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.383+0000 7ff934ff9700 1 -- 172.21.15.16:0/939451742 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92c004c80 con 0x7ff938133e30 2023-09-30T22:32:21.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.383+0000 7ff934ff9700 1 -- 172.21.15.16:0/939451742 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff92c004de0 con 0x7ff938133e30 2023-09-30T22:32:21.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.383+0000 7ff93e968700 1 -- 172.21.15.16:0/939451742 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff93812d7c0 con 0x7ff938133e30 2023-09-30T22:32:21.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.383+0000 7ff934ff9700 1 -- 172.21.15.16:0/939451742 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92c007310 con 0x7ff938133e30 2023-09-30T22:32:21.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.383+0000 7ff93e968700 1 -- 172.21.15.16:0/939451742 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff938120b20 con 0x7ff938133e30 2023-09-30T22:32:21.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.384+0000 7ff934ff9700 1 -- 172.21.15.16:0/939451742 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff92c007470 con 0x7ff938133e30 2023-09-30T22:32:21.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.384+0000 7ff934ff9700 1 -- 172.21.15.16:0/939451742 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7ff92c03f2b0 con 0x7ff938133e30 2023-09-30T22:32:21.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.385+0000 7ff93e968700 1 -- 172.21.15.16:0/939451742 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff938132eb0 con 0x7ff938133e30 2023-09-30T22:32:21.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.390+0000 7ff934ff9700 1 -- 172.21.15.16:0/939451742 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff92c007720 con 0x7ff938133e30 2023-09-30T22:32:21.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.424+0000 7f6d721c5700 1 Processor -- start 2023-09-30T22:32:21.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.424+0000 7f6d721c5700 1 -- start start 2023-09-30T22:32:21.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.424+0000 7f6d721c5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d6c09f410 0x7f6d6c136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:21.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.425+0000 7f6d721c5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d6c136fe0 0x7f6d6c1393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:21.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.425+0000 7f6d721c5700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d6c070940 con 0x7f6d6c09e930 2023-09-30T22:32:21.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.425+0000 7f6d721c5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6c06fd60 con 0x7f6d6c09f410 2023-09-30T22:32:21.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.425+0000 7f6d721c5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6c06fa80 con 0x7f6d6c136fe0 2023-09-30T22:32:21.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.425+0000 7f6d6b7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6d6c09e930 0x7f6d6c09ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53436/0 (socket says 172.21.15.16:53436) 2023-09-30T22:32:21.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.425+0000 7f6d6b7fe700 1 -- 172.21.15.16:0/592064779 learned_addr learned my addr 172.21.15.16:0/592064779 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:21.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.425+0000 7f6d6affd700 1 --2- 172.21.15.16:0/592064779 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d6c09f410 0x7f6d6c136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:21.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.425+0000 7f6d6bfff700 1 --2- 172.21.15.16:0/592064779 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d6c136fe0 0x7f6d6c1393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:21.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.425+0000 7f6d6affd700 1 -- 172.21.15.16:0/592064779 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d6c136fe0 msgr2=0x7f6d6c1393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:21.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.425+0000 7f6d6affd700 1 --2- 172.21.15.16:0/592064779 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d6c136fe0 0x7f6d6c1393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:21.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.425+0000 7f6d6affd700 1 -- 172.21.15.16:0/592064779 >> v1:172.21.15.16:6789/0 conn(0x7f6d6c09e930 legacy=0x7f6d6c09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:21.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.425+0000 7f6d6affd700 1 -- 172.21.15.16:0/592064779 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d6c09fb60 con 0x7f6d6c09f410 2023-09-30T22:32:21.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.426+0000 7f6d6affd700 1 --2- 172.21.15.16:0/592064779 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d6c09f410 0x7f6d6c136aa0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f6d60009870 tx=0x7f6d60009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75560b7cf2c209bc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:21.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.426+0000 7f6d6a7fc700 1 -- 172.21.15.16:0/592064779 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d60011830 con 0x7f6d6c09f410 2023-09-30T22:32:21.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.426+0000 7f6d6a7fc700 1 -- 172.21.15.16:0/592064779 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d60011990 con 0x7f6d6c09f410 2023-09-30T22:32:21.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.426+0000 7f6d6a7fc700 1 -- 172.21.15.16:0/592064779 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d60011d20 con 0x7f6d6c09f410 2023-09-30T22:32:21.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.427+0000 7f6d721c5700 1 -- 172.21.15.16:0/592064779 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d6c09f410 msgr2=0x7f6d6c136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:21.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.427+0000 7f6d721c5700 1 --2- 172.21.15.16:0/592064779 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d6c09f410 0x7f6d6c136aa0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f6d60009870 tx=0x7f6d60009b80 comp rx=0 tx=0).stop 2023-09-30T22:32:21.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.427+0000 7f6d721c5700 1 -- 172.21.15.16:0/592064779 shutdown_connections 2023-09-30T22:32:21.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.427+0000 7f6d721c5700 1 --2- 172.21.15.16:0/592064779 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d6c136fe0 0x7f6d6c1393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:21.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.427+0000 7f6d721c5700 1 --2- 172.21.15.16:0/592064779 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d6c09f410 0x7f6d6c136aa0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:21.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.427+0000 7f6d721c5700 1 -- 172.21.15.16:0/592064779 >> 172.21.15.16:0/592064779 conn(0x7f6d6c0a4210 msgr2=0x7f6d6c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:21.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.427+0000 7f6d721c5700 1 -- 172.21.15.16:0/592064779 shutdown_connections 2023-09-30T22:32:21.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.427+0000 7f6d721c5700 1 -- 172.21.15.16:0/592064779 wait complete. 2023-09-30T22:32:21.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.428+0000 7f6d721c5700 1 Processor -- start 2023-09-30T22:32:21.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.429+0000 7f6d721c5700 1 -- start start 2023-09-30T22:32:21.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.429+0000 7f6d721c5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d6c09e930 0x7f6d6c11c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:21.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.429+0000 7f6d721c5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d6c09f410 0x7f6d6c11cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:21.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.429+0000 7f6d721c5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d6c136fe0 0x7f6d6c11d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:21.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.429+0000 7f6d721c5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6c071d20 con 0x7f6d6c09f410 2023-09-30T22:32:21.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.429+0000 7f6d721c5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6c11e3f0 con 0x7f6d6c09e930 2023-09-30T22:32:21.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.429+0000 7f6d721c5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6c11ff10 con 0x7f6d6c136fe0 2023-09-30T22:32:21.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.429+0000 7f6d6affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d6c09f410 0x7f6d6c11cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:21.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.430+0000 7f6d6b7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d6c09e930 0x7f6d6c11c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:21.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.430+0000 7f6d6affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d6c09f410 0x7f6d6c11cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34686/0 (socket says 172.21.15.16:34686) 2023-09-30T22:32:21.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.430+0000 7f6d6affd700 1 -- 172.21.15.16:0/1160871521 learned_addr learned my addr 172.21.15.16:0/1160871521 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:21.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.430+0000 7f6d6bfff700 1 --2- 172.21.15.16:0/1160871521 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d6c136fe0 0x7f6d6c11d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:21.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.430+0000 7f6d6b7fe700 1 -- 172.21.15.16:0/1160871521 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d6c136fe0 msgr2=0x7f6d6c11d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:21.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.430+0000 7f6d6b7fe700 1 --2- 172.21.15.16:0/1160871521 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d6c136fe0 0x7f6d6c11d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:21.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.430+0000 7f6d6b7fe700 1 -- 172.21.15.16:0/1160871521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d6c09f410 msgr2=0x7f6d6c11cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:21.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.430+0000 7f6d6b7fe700 1 --2- 172.21.15.16:0/1160871521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d6c09f410 0x7f6d6c11cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:21.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.430+0000 7f6d6b7fe700 1 -- 172.21.15.16:0/1160871521 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d600094a0 con 0x7f6d6c09e930 2023-09-30T22:32:21.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.430+0000 7f6d6bfff700 1 --2- 172.21.15.16:0/1160871521 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d6c136fe0 0x7f6d6c11d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:32:21.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.430+0000 7f6d6b7fe700 1 --2- 172.21.15.16:0/1160871521 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d6c09e930 0x7f6d6c11c720 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c00c500 tx=0x7f6d5c00c810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:21.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.430+0000 7f6d53fff700 1 -- 172.21.15.16:0/1160871521 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c005d70 con 0x7f6d6c09e930 2023-09-30T22:32:21.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.430+0000 7f6d53fff700 1 -- 172.21.15.16:0/1160871521 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d5c0070d0 con 0x7f6d6c09e930 2023-09-30T22:32:21.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.430+0000 7f6d53fff700 1 -- 172.21.15.16:0/1160871521 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c007440 con 0x7f6d6c09e930 2023-09-30T22:32:21.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.430+0000 7f6d721c5700 1 -- 172.21.15.16:0/1160871521 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d6c11ea90 con 0x7f6d6c09e930 2023-09-30T22:32:21.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.431+0000 7f6d721c5700 1 -- 172.21.15.16:0/1160871521 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d6c1201f0 con 0x7f6d6c09e930 2023-09-30T22:32:21.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.432+0000 7f6d721c5700 1 -- 172.21.15.16:0/1160871521 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d6c074ea0 con 0x7f6d6c09e930 2023-09-30T22:32:21.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.432+0000 7f6d53fff700 1 -- 172.21.15.16:0/1160871521 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6d5c0075a0 con 0x7f6d6c09e930 2023-09-30T22:32:21.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.432+0000 7f6d53fff700 1 -- 172.21.15.16:0/1160871521 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f6d5c03eb20 con 0x7f6d6c09e930 2023-09-30T22:32:21.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.441+0000 7f6d53fff700 1 -- 172.21.15.16:0/1160871521 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6d5c019d20 con 0x7f6d6c09e930 2023-09-30T22:32:21.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.485+0000 7f1badd00700 1 Processor -- start 2023-09-30T22:32:21.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.486+0000 7f1badd00700 1 -- start start 2023-09-30T22:32:21.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.486+0000 7f1badd00700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ba8130670 0x7f1ba8130a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:21.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.486+0000 7f1badd00700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1ba806f680 0x7f1ba806fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:21.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.486+0000 7f1badd00700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ba8130260 con 0x7f1ba806ffd0 2023-09-30T22:32:21.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.486+0000 7f1badd00700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ba8071ff0 con 0x7f1ba806f680 2023-09-30T22:32:21.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.486+0000 7f1badd00700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba809ecf0 con 0x7f1ba8130670 2023-09-30T22:32:21.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.487+0000 7f1ba6ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1ba806f680 0x7f1ba806fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:21.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.487+0000 7f1ba77fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ba8130670 0x7f1ba8130a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:21.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.487+0000 7f1ba6ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1ba806f680 0x7f1ba806fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34688/0 (socket says 172.21.15.16:34688) 2023-09-30T22:32:21.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.487+0000 7f1ba6ffd700 1 -- 172.21.15.16:0/1951118796 learned_addr learned my addr 172.21.15.16:0/1951118796 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:21.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.487+0000 7f1ba77fe700 1 -- 172.21.15.16:0/1951118796 >> v2:172.21.15.16:3300/0 conn(0x7f1ba806f680 msgr2=0x7f1ba806fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:21.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.487+0000 7f1ba77fe700 1 --2- 172.21.15.16:0/1951118796 >> v2:172.21.15.16:3300/0 conn(0x7f1ba806f680 0x7f1ba806fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:21.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.487+0000 7f1ba77fe700 1 -- 172.21.15.16:0/1951118796 >> v1:172.21.15.16:6789/0 conn(0x7f1ba806ffd0 legacy=0x7f1ba8137530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:21.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.487+0000 7f1ba77fe700 1 -- 172.21.15.16:0/1951118796 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ba8070e70 con 0x7f1ba8130670 2023-09-30T22:32:21.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.487+0000 7f1ba77fe700 1 --2- 172.21.15.16:0/1951118796 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ba8130670 0x7f1ba8130a40 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1b98009840 tx=0x7f1b98009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d1ede7b96c6d7e7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:21.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.488+0000 7f1ba67fc700 1 -- 172.21.15.16:0/1951118796 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b98011830 con 0x7f1ba8130670 2023-09-30T22:32:21.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.488+0000 7f1ba67fc700 1 -- 172.21.15.16:0/1951118796 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b98011990 con 0x7f1ba8130670 2023-09-30T22:32:21.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.488+0000 7f1ba67fc700 1 -- 172.21.15.16:0/1951118796 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b98011d20 con 0x7f1ba8130670 2023-09-30T22:32:21.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.489+0000 7f1badd00700 1 -- 172.21.15.16:0/1951118796 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ba8130670 msgr2=0x7f1ba8130a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:21.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.489+0000 7f1badd00700 1 --2- 172.21.15.16:0/1951118796 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ba8130670 0x7f1ba8130a40 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1b98009840 tx=0x7f1b98009b50 comp rx=0 tx=0).stop 2023-09-30T22:32:21.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.489+0000 7f1badd00700 1 -- 172.21.15.16:0/1951118796 shutdown_connections 2023-09-30T22:32:21.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.489+0000 7f1badd00700 1 --2- 172.21.15.16:0/1951118796 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ba8130670 0x7f1ba8130a40 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:21.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.489+0000 7f1badd00700 1 --2- 172.21.15.16:0/1951118796 >> v2:172.21.15.16:3300/0 conn(0x7f1ba806f680 0x7f1ba806fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:21.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.489+0000 7f1badd00700 1 -- 172.21.15.16:0/1951118796 >> 172.21.15.16:0/1951118796 conn(0x7f1ba80a2aa0 msgr2=0x7f1ba80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:21.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.489+0000 7f1badd00700 1 -- 172.21.15.16:0/1951118796 shutdown_connections 2023-09-30T22:32:21.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.489+0000 7f1badd00700 1 -- 172.21.15.16:0/1951118796 wait complete. 2023-09-30T22:32:21.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.490+0000 7f1badd00700 1 Processor -- start 2023-09-30T22:32:21.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.490+0000 7f1badd00700 1 -- start start 2023-09-30T22:32:21.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.491+0000 7f1badd00700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ba806f680 0x7f1ba811da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:21.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.491+0000 7f1badd00700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ba806ffd0 0x7f1ba811dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:21.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.491+0000 7f1badd00700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ba8130670 0x7f1ba811e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:21.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.491+0000 7f1badd00700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba81213a0 con 0x7f1ba806f680 2023-09-30T22:32:21.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.491+0000 7f1badd00700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba811f690 con 0x7f1ba8130670 2023-09-30T22:32:21.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.491+0000 7f1badd00700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1ba8071820 con 0x7f1ba806ffd0 2023-09-30T22:32:21.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.491+0000 7f1ba77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ba806f680 0x7f1ba811da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:21.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.491+0000 7f1ba7fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ba8130670 0x7f1ba811e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:21.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.491+0000 7f1ba77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ba806f680 0x7f1ba811da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34704/0 (socket says 172.21.15.16:34704) 2023-09-30T22:32:21.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.491+0000 7f1ba77fe700 1 -- 172.21.15.16:0/1962565552 learned_addr learned my addr 172.21.15.16:0/1962565552 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:21.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.491+0000 7f1ba6ffd700 1 --2- 172.21.15.16:0/1962565552 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ba806ffd0 0x7f1ba811dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:21.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.491+0000 7f1ba77fe700 1 -- 172.21.15.16:0/1962565552 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ba806ffd0 msgr2=0x7f1ba811dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:21.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.491+0000 7f1ba77fe700 1 --2- 172.21.15.16:0/1962565552 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ba806ffd0 0x7f1ba811dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:21.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.491+0000 7f1ba77fe700 1 -- 172.21.15.16:0/1962565552 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ba8130670 msgr2=0x7f1ba811e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:21.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.491+0000 7f1ba77fe700 1 --2- 172.21.15.16:0/1962565552 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ba8130670 0x7f1ba811e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:21.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.491+0000 7f1ba77fe700 1 -- 172.21.15.16:0/1962565552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b980094a0 con 0x7f1ba806f680 2023-09-30T22:32:21.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.491+0000 7f1ba7fff700 1 --2- 172.21.15.16:0/1962565552 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ba8130670 0x7f1ba811e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:21.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.492+0000 7f1ba77fe700 1 --2- 172.21.15.16:0/1962565552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ba806f680 0x7f1ba811da80 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f1b98009b90 tx=0x7f1b98004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:21.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.492+0000 7f1b8ffff700 1 -- 172.21.15.16:0/1962565552 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b98004590 con 0x7f1ba806f680 2023-09-30T22:32:21.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.492+0000 7f1b8ffff700 1 -- 172.21.15.16:0/1962565552 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b980036a0 con 0x7f1ba806f680 2023-09-30T22:32:21.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.492+0000 7f1badd00700 1 -- 172.21.15.16:0/1962565552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ba8121000 con 0x7f1ba806f680 2023-09-30T22:32:21.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.492+0000 7f1b8ffff700 1 -- 172.21.15.16:0/1962565552 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b98003a30 con 0x7f1ba806f680 2023-09-30T22:32:21.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.492+0000 7f1badd00700 1 -- 172.21.15.16:0/1962565552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ba811fe20 con 0x7f1ba806f680 2023-09-30T22:32:21.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.493+0000 7f1b8ffff700 1 -- 172.21.15.16:0/1962565552 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1b98003b90 con 0x7f1ba806f680 2023-09-30T22:32:21.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.493+0000 7f1b8ffff700 1 -- 172.21.15.16:0/1962565552 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f1b9802b970 con 0x7f1ba806f680 2023-09-30T22:32:21.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.493+0000 7f1badd00700 1 -- 172.21.15.16:0/1962565552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ba81211c0 con 0x7f1ba806f680 2023-09-30T22:32:21.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.503+0000 7f1b8ffff700 1 -- 172.21.15.16:0/1962565552 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1b98015070 con 0x7f1ba806f680 2023-09-30T22:32:21.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.532+0000 7f6961447700 1 Processor -- start 2023-09-30T22:32:21.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.532+0000 7f6961447700 1 -- start start 2023-09-30T22:32:21.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.532+0000 7f6961447700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f695c09d6f0 0x7f695c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:21.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.532+0000 7f6961447700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f695c09e000 0x7f695c130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:21.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.532+0000 7f6961447700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f695c09e410 0x7f695c130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:21.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.532+0000 7f6961447700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f695c09f520 con 0x7f695c09e410 2023-09-30T22:32:21.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.532+0000 7f6961447700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f695c09f130 con 0x7f695c09e000 2023-09-30T22:32:21.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.532+0000 7f6961447700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f695c09e930 con 0x7f695c09d6f0 2023-09-30T22:32:21.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.533+0000 7f695affd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f695c09d6f0 0x7f695c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:21.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.533+0000 7f695affd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f695c09d6f0 0x7f695c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:36216/0 (socket says 172.21.15.16:36216) 2023-09-30T22:32:21.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.533+0000 7f695affd700 1 -- 172.21.15.16:0/1121273454 learned_addr learned my addr 172.21.15.16:0/1121273454 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:21.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.533+0000 7f695b7fe700 1 --2- 172.21.15.16:0/1121273454 >> v2:172.21.15.16:3300/0 conn(0x7f695c09e410 0x7f695c130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:21.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.533+0000 7f695a7fc700 1 --2- 172.21.15.16:0/1121273454 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f695c09e000 0x7f695c130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:21.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.533+0000 7f695affd700 1 -- 172.21.15.16:0/1121273454 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f695c09e000 msgr2=0x7f695c130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:21.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.533+0000 7f695affd700 1 --2- 172.21.15.16:0/1121273454 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f695c09e000 0x7f695c130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:21.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.533+0000 7f695affd700 1 -- 172.21.15.16:0/1121273454 >> v2:172.21.15.16:3300/0 conn(0x7f695c09e410 msgr2=0x7f695c130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:21.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.533+0000 7f695affd700 1 --2- 172.21.15.16:0/1121273454 >> v2:172.21.15.16:3300/0 conn(0x7f695c09e410 0x7f695c130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:21.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.533+0000 7f695affd700 1 -- 172.21.15.16:0/1121273454 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f695c09b2d0 con 0x7f695c09d6f0 2023-09-30T22:32:21.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.533+0000 7f695a7fc700 1 --2- 172.21.15.16:0/1121273454 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f695c09e000 0x7f695c130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:32:21.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.533+0000 7f695b7fe700 1 --2- 172.21.15.16:0/1121273454 >> v2:172.21.15.16:3300/0 conn(0x7f695c09e410 0x7f695c130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:21.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.533+0000 7f695affd700 1 --2- 172.21.15.16:0/1121273454 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f695c09d6f0 0x7f695c09dac0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f694c009920 tx=0x7f694c009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=468ff91c9218e6b7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:21.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.533+0000 7f6959ffb700 1 -- 172.21.15.16:0/1121273454 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f694c011830 con 0x7f695c09d6f0 2023-09-30T22:32:21.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.533+0000 7f6959ffb700 1 -- 172.21.15.16:0/1121273454 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f694c011990 con 0x7f695c09d6f0 2023-09-30T22:32:21.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.534+0000 7f6959ffb700 1 -- 172.21.15.16:0/1121273454 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f694c011d20 con 0x7f695c09d6f0 2023-09-30T22:32:21.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.534+0000 7f6961447700 1 -- 172.21.15.16:0/1121273454 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f695c09d6f0 msgr2=0x7f695c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:21.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.534+0000 7f6961447700 1 --2- 172.21.15.16:0/1121273454 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f695c09d6f0 0x7f695c09dac0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f694c009920 tx=0x7f694c009c30 comp rx=0 tx=0).stop 2023-09-30T22:32:21.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.534+0000 7f6961447700 1 -- 172.21.15.16:0/1121273454 shutdown_connections 2023-09-30T22:32:21.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.534+0000 7f6961447700 1 --2- 172.21.15.16:0/1121273454 >> v2:172.21.15.16:3300/0 conn(0x7f695c09e410 0x7f695c130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:21.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.534+0000 7f6961447700 1 --2- 172.21.15.16:0/1121273454 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f695c09d6f0 0x7f695c09dac0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:21.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.534+0000 7f6961447700 1 --2- 172.21.15.16:0/1121273454 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f695c09e000 0x7f695c130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:21.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.534+0000 7f6961447700 1 -- 172.21.15.16:0/1121273454 >> 172.21.15.16:0/1121273454 conn(0x7f695c066f50 msgr2=0x7f695c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:21.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.534+0000 7f6961447700 1 -- 172.21.15.16:0/1121273454 shutdown_connections 2023-09-30T22:32:21.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.534+0000 7f6961447700 1 -- 172.21.15.16:0/1121273454 wait complete. 2023-09-30T22:32:21.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.536+0000 7f6961447700 1 Processor -- start 2023-09-30T22:32:21.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.536+0000 7f6961447700 1 -- start start 2023-09-30T22:32:21.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.536+0000 7f6961447700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f695c09d6f0 0x7f695c11dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:21.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.536+0000 7f6961447700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f695c09e000 0x7f695c11e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:21.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.536+0000 7f6961447700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f695c09e410 0x7f695c11e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:21.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.536+0000 7f6961447700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f695c121520 con 0x7f695c09e000 2023-09-30T22:32:21.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.536+0000 7f6961447700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f695c11fa00 con 0x7f695c09d6f0 2023-09-30T22:32:21.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.536+0000 7f6961447700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f695c0a1c80 con 0x7f695c09e410 2023-09-30T22:32:21.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.536+0000 7f695a7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f695c09e000 0x7f695c11e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:21.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.536+0000 7f695a7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f695c09e000 0x7f695c11e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34722/0 (socket says 172.21.15.16:34722) 2023-09-30T22:32:21.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.536+0000 7f695a7fc700 1 -- 172.21.15.16:0/1407758390 learned_addr learned my addr 172.21.15.16:0/1407758390 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:21.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.536+0000 7f695affd700 1 --2- 172.21.15.16:0/1407758390 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f695c09d6f0 0x7f695c11dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:21.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.536+0000 7f695b7fe700 1 --2- 172.21.15.16:0/1407758390 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f695c09e410 0x7f695c11e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:21.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.537+0000 7f695a7fc700 1 -- 172.21.15.16:0/1407758390 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f695c09e410 msgr2=0x7f695c11e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:21.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.537+0000 7f695a7fc700 1 --2- 172.21.15.16:0/1407758390 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f695c09e410 0x7f695c11e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:21.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.537+0000 7f695a7fc700 1 -- 172.21.15.16:0/1407758390 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f695c09d6f0 msgr2=0x7f695c11dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:21.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.537+0000 7f695a7fc700 1 --2- 172.21.15.16:0/1407758390 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f695c09d6f0 0x7f695c11dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:21.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.537+0000 7f695a7fc700 1 -- 172.21.15.16:0/1407758390 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6954009a70 con 0x7f695c09e000 2023-09-30T22:32:21.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.537+0000 7f695a7fc700 1 --2- 172.21.15.16:0/1407758390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f695c09e000 0x7f695c11e270 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f695000d800 tx=0x7f695000db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:21.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.537+0000 7f69437fe700 1 -- 172.21.15.16:0/1407758390 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6950014620 con 0x7f695c09e000 2023-09-30T22:32:21.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.537+0000 7f6961447700 1 -- 172.21.15.16:0/1407758390 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f694c009550 con 0x7f695c09e000 2023-09-30T22:32:21.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.537+0000 7f69437fe700 1 -- 172.21.15.16:0/1407758390 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6950014780 con 0x7f695c09e000 2023-09-30T22:32:21.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.537+0000 7f69437fe700 1 -- 172.21.15.16:0/1407758390 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6950014af0 con 0x7f695c09e000 2023-09-30T22:32:21.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.537+0000 7f6961447700 1 -- 172.21.15.16:0/1407758390 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f695c120b40 con 0x7f695c09e000 2023-09-30T22:32:21.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.539+0000 7f6961447700 1 -- 172.21.15.16:0/1407758390 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f695c11ac20 con 0x7f695c09e000 2023-09-30T22:32:21.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.540+0000 7f69437fe700 1 -- 172.21.15.16:0/1407758390 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6950014c50 con 0x7f695c09e000 2023-09-30T22:32:21.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.540+0000 7f69437fe700 1 -- 172.21.15.16:0/1407758390 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f6950040010 con 0x7f695c09e000 2023-09-30T22:32:21.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.544+0000 7f69437fe700 1 -- 172.21.15.16:0/1407758390 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6950013780 con 0x7f695c09e000 2023-09-30T22:32:21.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.657+0000 7f6d721c5700 1 -- 172.21.15.16:0/1160871521 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6d6c11fbb0 con 0x7f6d6c09e930 2023-09-30T22:32:21.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.658+0000 7f6d53fff700 1 -- 172.21.15.16:0/1160871521 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6d6c11fbb0 con 0x7f6d6c09e930 2023-09-30T22:32:21.659 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:32:21.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.660+0000 7f6d721c5700 1 -- 172.21.15.16:0/1160871521 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d6c09e930 msgr2=0x7f6d6c11c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:21.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.660+0000 7f6d721c5700 1 --2- 172.21.15.16:0/1160871521 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d6c09e930 0x7f6d6c11c720 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c00c500 tx=0x7f6d5c00c810 comp rx=0 tx=0).stop 2023-09-30T22:32:21.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.660+0000 7f6d721c5700 1 -- 172.21.15.16:0/1160871521 shutdown_connections 2023-09-30T22:32:21.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.660+0000 7f6d721c5700 1 --2- 172.21.15.16:0/1160871521 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d6c09e930 0x7f6d6c11c720 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:21.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.660+0000 7f6d721c5700 1 --2- 172.21.15.16:0/1160871521 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d6c136fe0 0x7f6d6c11d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:21.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.660+0000 7f6d721c5700 1 --2- 172.21.15.16:0/1160871521 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d6c09f410 0x7f6d6c11cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:21.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.660+0000 7f6d721c5700 1 -- 172.21.15.16:0/1160871521 >> 172.21.15.16:0/1160871521 conn(0x7f6d6c0a4210 msgr2=0x7f6d6c135f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:21.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.660+0000 7f6d721c5700 1 -- 172.21.15.16:0/1160871521 shutdown_connections 2023-09-30T22:32:21.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.660+0000 7f6d721c5700 1 -- 172.21.15.16:0/1160871521 wait complete. 2023-09-30T22:32:21.676 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:32:21.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.676+0000 7ff93e968700 1 -- 172.21.15.16:0/939451742 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff9381202f0 con 0x7ff938133e30 2023-09-30T22:32:21.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.676+0000 7ff934ff9700 1 -- 172.21.15.16:0/939451742 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff92c011360 con 0x7ff938133e30 2023-09-30T22:32:21.678 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:32:21.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.680+0000 7ff93e968700 1 -- 172.21.15.16:0/939451742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff938133e30 msgr2=0x7ff93809c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:21.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.680+0000 7ff93e968700 1 --2- 172.21.15.16:0/939451742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff938133e30 0x7ff93809c9c0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7ff92c00c470 tx=0x7ff92c00c780 comp rx=0 tx=0).stop 2023-09-30T22:32:21.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.680+0000 7ff93e968700 1 -- 172.21.15.16:0/939451742 shutdown_connections 2023-09-30T22:32:21.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.680+0000 7ff93e968700 1 --2- 172.21.15.16:0/939451742 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff938134260 0x7ff93809cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:21.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.680+0000 7ff93e968700 1 --2- 172.21.15.16:0/939451742 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff938133420 0x7ff93809c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:21.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.680+0000 7ff93e968700 1 --2- 172.21.15.16:0/939451742 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff938133e30 0x7ff93809c9c0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:21.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.680+0000 7ff93e968700 1 -- 172.21.15.16:0/939451742 >> 172.21.15.16:0/939451742 conn(0x7ff9380a4210 msgr2=0x7ff9380a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:21.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.680+0000 7ff93e968700 1 -- 172.21.15.16:0/939451742 shutdown_connections 2023-09-30T22:32:21.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.680+0000 7ff93e968700 1 -- 172.21.15.16:0/939451742 wait complete. 2023-09-30T22:32:21.695 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:32:21.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.757+0000 7f6961447700 1 -- 172.21.15.16:0/1407758390 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f695c121660 con 0x7f695c09e000 2023-09-30T22:32:21.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.757+0000 7f69437fe700 1 -- 172.21.15.16:0/1407758390 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f695c121660 con 0x7f695c09e000 2023-09-30T22:32:21.759 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:32:21.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.759+0000 7f6961447700 1 -- 172.21.15.16:0/1407758390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f695c09e000 msgr2=0x7f695c11e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:21.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.759+0000 7f6961447700 1 --2- 172.21.15.16:0/1407758390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f695c09e000 0x7f695c11e270 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f695000d800 tx=0x7f695000db10 comp rx=0 tx=0).stop 2023-09-30T22:32:21.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.760+0000 7f6961447700 1 -- 172.21.15.16:0/1407758390 shutdown_connections 2023-09-30T22:32:21.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.760+0000 7f6961447700 1 --2- 172.21.15.16:0/1407758390 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f695c09d6f0 0x7f695c11dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:21.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.760+0000 7f6961447700 1 --2- 172.21.15.16:0/1407758390 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f695c09e410 0x7f695c11e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:21.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.760+0000 7f6961447700 1 --2- 172.21.15.16:0/1407758390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f695c09e000 0x7f695c11e270 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:21.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.760+0000 7f6961447700 1 -- 172.21.15.16:0/1407758390 >> 172.21.15.16:0/1407758390 conn(0x7f695c066f50 msgr2=0x7f695c130800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:21.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.760+0000 7f6961447700 1 -- 172.21.15.16:0/1407758390 shutdown_connections 2023-09-30T22:32:21.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.760+0000 7f6961447700 1 -- 172.21.15.16:0/1407758390 wait complete. 2023-09-30T22:32:21.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.768+0000 7f1badd00700 1 -- 172.21.15.16:0/1962565552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1ba8074ea0 con 0x7f1ba806f680 2023-09-30T22:32:21.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.768+0000 7f1b8ffff700 1 -- 172.21.15.16:0/1962565552 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1b98015030 con 0x7f1ba806f680 2023-09-30T22:32:21.770 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:32:21.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.771+0000 7f1badd00700 1 -- 172.21.15.16:0/1962565552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ba806f680 msgr2=0x7f1ba811da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:21.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.771+0000 7f1badd00700 1 --2- 172.21.15.16:0/1962565552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ba806f680 0x7f1ba811da80 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f1b98009b90 tx=0x7f1b98004750 comp rx=0 tx=0).stop 2023-09-30T22:32:21.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.771+0000 7f1badd00700 1 -- 172.21.15.16:0/1962565552 shutdown_connections 2023-09-30T22:32:21.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.771+0000 7f1badd00700 1 --2- 172.21.15.16:0/1962565552 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ba8130670 0x7f1ba811e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:21.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.771+0000 7f1badd00700 1 --2- 172.21.15.16:0/1962565552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ba806f680 0x7f1ba811da80 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:21.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.771+0000 7f1badd00700 1 --2- 172.21.15.16:0/1962565552 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ba806ffd0 0x7f1ba811dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:21.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.771+0000 7f1badd00700 1 -- 172.21.15.16:0/1962565552 >> 172.21.15.16:0/1962565552 conn(0x7f1ba80a2aa0 msgr2=0x7f1ba806e570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:21.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.771+0000 7f1badd00700 1 -- 172.21.15.16:0/1962565552 shutdown_connections 2023-09-30T22:32:21.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:21.771+0000 7f1badd00700 1 -- 172.21.15.16:0/1962565552 wait complete. 2023-09-30T22:32:21.775 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:32:21.788 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:32:22.677 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:32:22.696 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:32:22.776 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:32:22.789 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:32:22.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.828+0000 7f1f7f47f700 1 Processor -- start 2023-09-30T22:32:22.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.828+0000 7f1f7f47f700 1 -- start start 2023-09-30T22:32:22.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.829+0000 7f1f7f47f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1f781338c0 0x7f1f78133c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:22.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.829+0000 7f1f7f47f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f781341d0 0x7f1f7809de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:22.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.829+0000 7f1f7f47f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f7809e390 0x7f1f7809e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:22.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.829+0000 7f1f7f47f700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f78070c80 con 0x7f1f781338c0 2023-09-30T22:32:22.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.829+0000 7f1f7f47f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7809ed20 con 0x7f1f7809e390 2023-09-30T22:32:22.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.829+0000 7f1f7f47f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7806f720 con 0x7f1f781341d0 2023-09-30T22:32:22.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.829+0000 7f1f7d21b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1f781338c0 0x7f1f78133c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:22.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.829+0000 7f1f7ca1a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f781341d0 0x7f1f7809de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:22.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.829+0000 7f1f7d21b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1f781338c0 0x7f1f78133c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34730/0 (socket says 172.21.15.16:34730) 2023-09-30T22:32:22.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.829+0000 7f1f7ca1a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f781341d0 0x7f1f7809de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:36232/0 (socket says 172.21.15.16:36232) 2023-09-30T22:32:22.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.829+0000 7f1f7d21b700 1 -- 172.21.15.16:0/1583923609 learned_addr learned my addr 172.21.15.16:0/1583923609 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:22.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.829+0000 7f1f7da1c700 1 --2- 172.21.15.16:0/1583923609 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f7809e390 0x7f1f7809e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:22.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.830+0000 7f1f7ca1a700 1 -- 172.21.15.16:0/1583923609 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f7809e390 msgr2=0x7f1f7809e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:22.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.830+0000 7f1f7ca1a700 1 --2- 172.21.15.16:0/1583923609 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f7809e390 0x7f1f7809e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:22.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.830+0000 7f1f7ca1a700 1 -- 172.21.15.16:0/1583923609 >> v2:172.21.15.16:3300/0 conn(0x7f1f781338c0 msgr2=0x7f1f78133c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:22.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.830+0000 7f1f7ca1a700 1 --2- 172.21.15.16:0/1583923609 >> v2:172.21.15.16:3300/0 conn(0x7f1f781338c0 0x7f1f78133c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:22.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.830+0000 7f1f7ca1a700 1 -- 172.21.15.16:0/1583923609 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f7806fce0 con 0x7f1f781341d0 2023-09-30T22:32:22.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.830+0000 7f1f7d21b700 1 --2- 172.21.15.16:0/1583923609 >> v2:172.21.15.16:3300/0 conn(0x7f1f781338c0 0x7f1f78133c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:22.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.830+0000 7f1f7da1c700 1 --2- 172.21.15.16:0/1583923609 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f7809e390 0x7f1f7809e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:22.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.830+0000 7f1f7ca1a700 1 --2- 172.21.15.16:0/1583923609 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f781341d0 0x7f1f7809de50 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1f68009870 tx=0x7f1f68009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ba374d322085da4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:22.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.830+0000 7f1f6ffff700 1 -- 172.21.15.16:0/1583923609 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f68011830 con 0x7f1f781341d0 2023-09-30T22:32:22.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.830+0000 7f1f6ffff700 1 -- 172.21.15.16:0/1583923609 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f68011990 con 0x7f1f781341d0 2023-09-30T22:32:22.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.830+0000 7f1f6ffff700 1 -- 172.21.15.16:0/1583923609 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f68011d20 con 0x7f1f781341d0 2023-09-30T22:32:22.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.831+0000 7f1f7f47f700 1 -- 172.21.15.16:0/1583923609 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f781341d0 msgr2=0x7f1f7809de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:22.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.831+0000 7f1f7f47f700 1 --2- 172.21.15.16:0/1583923609 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f781341d0 0x7f1f7809de50 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1f68009870 tx=0x7f1f68009b80 comp rx=0 tx=0).stop 2023-09-30T22:32:22.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.831+0000 7f1f7f47f700 1 -- 172.21.15.16:0/1583923609 shutdown_connections 2023-09-30T22:32:22.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.832+0000 7f1f7f47f700 1 --2- 172.21.15.16:0/1583923609 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f7809e390 0x7f1f7809e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:22.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.832+0000 7f1f7f47f700 1 --2- 172.21.15.16:0/1583923609 >> v2:172.21.15.16:3300/0 conn(0x7f1f781338c0 0x7f1f78133c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:22.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.832+0000 7f1f7f47f700 1 --2- 172.21.15.16:0/1583923609 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f781341d0 0x7f1f7809de50 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:22.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.832+0000 7f1f7f47f700 1 -- 172.21.15.16:0/1583923609 >> 172.21.15.16:0/1583923609 conn(0x7f1f780a2aa0 msgr2=0x7f1f780a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:22.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.832+0000 7f1f7f47f700 1 -- 172.21.15.16:0/1583923609 shutdown_connections 2023-09-30T22:32:22.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.832+0000 7f1f7f47f700 1 -- 172.21.15.16:0/1583923609 wait complete. 2023-09-30T22:32:22.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.833+0000 7f1f7f47f700 1 Processor -- start 2023-09-30T22:32:22.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.833+0000 7f1f7f47f700 1 -- start start 2023-09-30T22:32:22.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.834+0000 7f1f7f47f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f7809e390 0x7f1f7811eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:22.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.834+0000 7f1f7f47f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f781338c0 0x7f1f7811f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:22.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.834+0000 7f1f7f47f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f781341d0 0x7f1f7811f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:22.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.834+0000 7f1f7f47f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f78122270 con 0x7f1f7809e390 2023-09-30T22:32:22.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.834+0000 7f1f7f47f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f78130c20 con 0x7f1f781338c0 2023-09-30T22:32:22.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.834+0000 7f1f7f47f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1f78120750 con 0x7f1f781341d0 2023-09-30T22:32:22.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.834+0000 7f1f7d21b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f7809e390 0x7f1f7811eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:22.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.834+0000 7f1f7d21b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f7809e390 0x7f1f7811eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34742/0 (socket says 172.21.15.16:34742) 2023-09-30T22:32:22.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.834+0000 7f1f7d21b700 1 -- 172.21.15.16:0/3530353305 learned_addr learned my addr 172.21.15.16:0/3530353305 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:22.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.834+0000 7f1f7da1c700 1 --2- 172.21.15.16:0/3530353305 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f781341d0 0x7f1f7811f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:22.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.834+0000 7f1f7ca1a700 1 --2- 172.21.15.16:0/3530353305 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f781338c0 0x7f1f7811f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:22.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.834+0000 7f1f7d21b700 1 -- 172.21.15.16:0/3530353305 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f781341d0 msgr2=0x7f1f7811f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:22.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.834+0000 7f1f7d21b700 1 --2- 172.21.15.16:0/3530353305 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f781341d0 0x7f1f7811f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:22.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.834+0000 7f1f7d21b700 1 -- 172.21.15.16:0/3530353305 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f781338c0 msgr2=0x7f1f7811f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:22.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.834+0000 7f1f7d21b700 1 --2- 172.21.15.16:0/3530353305 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f781338c0 0x7f1f7811f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:22.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.834+0000 7f1f7d21b700 1 -- 172.21.15.16:0/3530353305 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f70009a70 con 0x7f1f7809e390 2023-09-30T22:32:22.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.835+0000 7f1f7da1c700 1 --2- 172.21.15.16:0/3530353305 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f781341d0 0x7f1f7811f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:32:22.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.835+0000 7f1f7d21b700 1 --2- 172.21.15.16:0/3530353305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f7809e390 0x7f1f7811eac0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f1f7400ebe0 tx=0x7f1f74009e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:22.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.835+0000 7f1f6dffb700 1 -- 172.21.15.16:0/3530353305 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7400c8b0 con 0x7f1f7809e390 2023-09-30T22:32:22.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.835+0000 7f1f6dffb700 1 -- 172.21.15.16:0/3530353305 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f7400ca10 con 0x7f1f7809e390 2023-09-30T22:32:22.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.835+0000 7f1f7f47f700 1 -- 172.21.15.16:0/3530353305 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f74009550 con 0x7f1f7809e390 2023-09-30T22:32:22.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.835+0000 7f1f6dffb700 1 -- 172.21.15.16:0/3530353305 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7400cd80 con 0x7f1f7809e390 2023-09-30T22:32:22.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.835+0000 7f1f7f47f700 1 -- 172.21.15.16:0/3530353305 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f680094a0 con 0x7f1f7809e390 2023-09-30T22:32:22.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.837+0000 7f1f7f47f700 1 -- 172.21.15.16:0/3530353305 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f78074ea0 con 0x7f1f7809e390 2023-09-30T22:32:22.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.837+0000 7f1f6dffb700 1 -- 172.21.15.16:0/3530353305 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1f74013470 con 0x7f1f7809e390 2023-09-30T22:32:22.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.838+0000 7f1f6dffb700 1 -- 172.21.15.16:0/3530353305 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f1f74040180 con 0x7f1f7809e390 2023-09-30T22:32:22.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.842+0000 7f1f6dffb700 1 -- 172.21.15.16:0/3530353305 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1f74011760 con 0x7f1f7809e390 2023-09-30T22:32:22.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.886+0000 7fa67ccf5700 1 Processor -- start 2023-09-30T22:32:22.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.886+0000 7fa67ccf5700 1 -- start start 2023-09-30T22:32:22.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.887+0000 7fa67ccf5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa67809d6f0 0x7fa67809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:22.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.887+0000 7fa67ccf5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa67809e000 0x7fa678130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:22.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.887+0000 7fa67ccf5700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa67809f130 con 0x7fa67809e410 2023-09-30T22:32:22.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.887+0000 7fa67ccf5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa67809f520 con 0x7fa67809e000 2023-09-30T22:32:22.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.887+0000 7fa67ccf5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa67809e930 con 0x7fa67809d6f0 2023-09-30T22:32:22.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.887+0000 7fa67659c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa67809d6f0 0x7fa67809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:22.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.887+0000 7fa67659c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa67809d6f0 0x7fa67809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:36256/0 (socket says 172.21.15.16:36256) 2023-09-30T22:32:22.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.887+0000 7fa67659c700 1 -- 172.21.15.16:0/764504230 learned_addr learned my addr 172.21.15.16:0/764504230 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:22.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.887+0000 7fa675d9b700 1 --2- 172.21.15.16:0/764504230 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa67809e000 0x7fa678130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:22.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.888+0000 7fa67559a700 1 -- 172.21.15.16:0/764504230 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 687641112 0 0) 0x7fa67809f130 con 0x7fa67809e410 2023-09-30T22:32:22.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.888+0000 7fa675d9b700 1 -- 172.21.15.16:0/764504230 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa67809d6f0 msgr2=0x7fa67809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:22.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.888+0000 7fa675d9b700 1 --2- 172.21.15.16:0/764504230 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa67809d6f0 0x7fa67809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:22.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.888+0000 7fa675d9b700 1 -- 172.21.15.16:0/764504230 >> v1:172.21.15.16:6789/0 conn(0x7fa67809e410 legacy=0x7fa678130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:22.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.888+0000 7fa675d9b700 1 -- 172.21.15.16:0/764504230 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa67809b2d0 con 0x7fa67809e000 2023-09-30T22:32:22.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.888+0000 7fa67659c700 1 --2- 172.21.15.16:0/764504230 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa67809d6f0 0x7fa67809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:22.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.888+0000 7fa675d9b700 1 --2- 172.21.15.16:0/764504230 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa67809e000 0x7fa678130660 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa66c009870 tx=0x7fa66c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb202c680e09e1ca server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:22.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.888+0000 7fa67559a700 1 -- 172.21.15.16:0/764504230 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66c011830 con 0x7fa67809e000 2023-09-30T22:32:22.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.888+0000 7fa67559a700 1 -- 172.21.15.16:0/764504230 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa66c011990 con 0x7fa67809e000 2023-09-30T22:32:22.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.888+0000 7fa67559a700 1 -- 172.21.15.16:0/764504230 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66c011d20 con 0x7fa67809e000 2023-09-30T22:32:22.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.889+0000 7fa67ccf5700 1 -- 172.21.15.16:0/764504230 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa67809e000 msgr2=0x7fa678130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:22.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.889+0000 7fa67ccf5700 1 --2- 172.21.15.16:0/764504230 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa67809e000 0x7fa678130660 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa66c009870 tx=0x7fa66c009b80 comp rx=0 tx=0).stop 2023-09-30T22:32:22.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.889+0000 7fa67ccf5700 1 -- 172.21.15.16:0/764504230 shutdown_connections 2023-09-30T22:32:22.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.889+0000 7fa67ccf5700 1 --2- 172.21.15.16:0/764504230 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa67809d6f0 0x7fa67809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:22.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.889+0000 7fa67ccf5700 1 --2- 172.21.15.16:0/764504230 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa67809e000 0x7fa678130660 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:22.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.889+0000 7fa67ccf5700 1 -- 172.21.15.16:0/764504230 >> 172.21.15.16:0/764504230 conn(0x7fa678066f50 msgr2=0x7fa67809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:22.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.889+0000 7fa67ccf5700 1 -- 172.21.15.16:0/764504230 shutdown_connections 2023-09-30T22:32:22.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.890+0000 7fa67ccf5700 1 -- 172.21.15.16:0/764504230 wait complete. 2023-09-30T22:32:22.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.891+0000 7fa67ccf5700 1 Processor -- start 2023-09-30T22:32:22.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.891+0000 7fa67ccf5700 1 -- start start 2023-09-30T22:32:22.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.892+0000 7fa67ccf5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa67809d6f0 0x7fa67811e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:22.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.892+0000 7fa67ccf5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa67809e000 0x7fa67811e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:22.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.892+0000 7fa67ccf5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa67809e410 0x7fa6781cc650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:22.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.892+0000 7fa67ccf5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa678121860 con 0x7fa67809e000 2023-09-30T22:32:22.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.892+0000 7fa67ccf5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6781214c0 con 0x7fa67809d6f0 2023-09-30T22:32:22.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.892+0000 7fa67ccf5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa67811f3f0 con 0x7fa67809e410 2023-09-30T22:32:22.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.892+0000 7fa675d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa67809e000 0x7fa67811e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:22.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.892+0000 7fa675d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa67809e000 0x7fa67811e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34752/0 (socket says 172.21.15.16:34752) 2023-09-30T22:32:22.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.892+0000 7fa675d9b700 1 -- 172.21.15.16:0/3162304040 learned_addr learned my addr 172.21.15.16:0/3162304040 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:22.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.892+0000 7fa676d9d700 1 --2- 172.21.15.16:0/3162304040 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa67809e410 0x7fa6781cc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:22.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.892+0000 7fa67659c700 1 --2- 172.21.15.16:0/3162304040 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa67809d6f0 0x7fa67811e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:22.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.893+0000 7fa67659c700 1 -- 172.21.15.16:0/3162304040 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa67809e410 msgr2=0x7fa6781cc650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:22.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.893+0000 7fa67659c700 1 --2- 172.21.15.16:0/3162304040 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa67809e410 0x7fa6781cc650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:22.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.893+0000 7fa67659c700 1 -- 172.21.15.16:0/3162304040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa67809e000 msgr2=0x7fa67811e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:22.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.893+0000 7fa67659c700 1 --2- 172.21.15.16:0/3162304040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa67809e000 0x7fa67811e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:22.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.893+0000 7fa67659c700 1 -- 172.21.15.16:0/3162304040 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa66800b7f0 con 0x7fa67809d6f0 2023-09-30T22:32:22.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.893+0000 7fa676d9d700 1 --2- 172.21.15.16:0/3162304040 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa67809e410 0x7fa6781cc650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:22.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.893+0000 7fa67659c700 1 --2- 172.21.15.16:0/3162304040 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa67809d6f0 0x7fa67811e200 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa66800ed70 tx=0x7fa66800d630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:22.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.893+0000 7fa666ffd700 1 -- 172.21.15.16:0/3162304040 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66800dd90 con 0x7fa67809d6f0 2023-09-30T22:32:22.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.893+0000 7fa666ffd700 1 -- 172.21.15.16:0/3162304040 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa668004030 con 0x7fa67809d6f0 2023-09-30T22:32:22.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.893+0000 7fa67ccf5700 1 -- 172.21.15.16:0/3162304040 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa66c0094a0 con 0x7fa67809d6f0 2023-09-30T22:32:22.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.893+0000 7fa666ffd700 1 -- 172.21.15.16:0/3162304040 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6680043a0 con 0x7fa67809d6f0 2023-09-30T22:32:22.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.893+0000 7fa67ccf5700 1 -- 172.21.15.16:0/3162304040 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa678120300 con 0x7fa67809d6f0 2023-09-30T22:32:22.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.894+0000 7fa666ffd700 1 -- 172.21.15.16:0/3162304040 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa6680036a0 con 0x7fa67809d6f0 2023-09-30T22:32:22.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.894+0000 7fa666ffd700 1 -- 172.21.15.16:0/3162304040 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fa66803fbc0 con 0x7fa67809d6f0 2023-09-30T22:32:22.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.895+0000 7fa67ccf5700 1 -- 172.21.15.16:0/3162304040 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6781216c0 con 0x7fa67809d6f0 2023-09-30T22:32:22.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.900+0000 7fa666ffd700 1 -- 172.21.15.16:0/3162304040 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa6781216c0 con 0x7fa67809d6f0 2023-09-30T22:32:22.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.950+0000 7f79ee5a0700 1 Processor -- start 2023-09-30T22:32:22.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.951+0000 7f79ee5a0700 1 -- start start 2023-09-30T22:32:22.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.951+0000 7f79ee5a0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f79e809d800 0x7f79e809dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:22.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.951+0000 7f79ee5a0700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f79e809e110 0x7f79e8133860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:22.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.951+0000 7f79ee5a0700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79e8099e10 con 0x7f79e8133da0 2023-09-30T22:32:22.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.951+0000 7f79ee5a0700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79e809f560 con 0x7f79e809e110 2023-09-30T22:32:22.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.951+0000 7f79ee5a0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79e806b4d0 con 0x7f79e809d800 2023-09-30T22:32:22.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.952+0000 7f79ecd9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f79e809e110 0x7f79e8133860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:22.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.952+0000 7f79edd9f700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f79e8133da0 0x7f79e813aab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53452/0 (socket says 172.21.15.16:53452) 2023-09-30T22:32:22.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.952+0000 7f79edd9f700 1 -- 172.21.15.16:0/3668372448 learned_addr learned my addr 172.21.15.16:0/3668372448 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:22.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.952+0000 7f79ed59e700 1 --2- 172.21.15.16:0/3668372448 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f79e809d800 0x7f79e809dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:22.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.952+0000 7f79ecd9d700 1 -- 172.21.15.16:0/3668372448 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f79e809d800 msgr2=0x7f79e809dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:22.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.952+0000 7f79ecd9d700 1 --2- 172.21.15.16:0/3668372448 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f79e809d800 0x7f79e809dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:22.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.952+0000 7f79ecd9d700 1 -- 172.21.15.16:0/3668372448 >> v1:172.21.15.16:6789/0 conn(0x7f79e8133da0 legacy=0x7f79e813aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:22.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.952+0000 7f79ecd9d700 1 -- 172.21.15.16:0/3668372448 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e809a060 con 0x7f79e809e110 2023-09-30T22:32:22.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.952+0000 7f79ecd9d700 1 --2- 172.21.15.16:0/3668372448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79e809e110 0x7f79e8133860 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f79d8009820 tx=0x7f79d8009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de031ba2440d98b1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:22.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.953+0000 7f79dffff700 1 -- 172.21.15.16:0/3668372448 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d8005e30 con 0x7f79e809e110 2023-09-30T22:32:22.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.953+0000 7f79dffff700 1 -- 172.21.15.16:0/3668372448 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d8006030 con 0x7f79e809e110 2023-09-30T22:32:22.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.953+0000 7f79dffff700 1 -- 172.21.15.16:0/3668372448 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d80063e0 con 0x7f79e809e110 2023-09-30T22:32:22.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.954+0000 7f79ee5a0700 1 -- 172.21.15.16:0/3668372448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79e809e110 msgr2=0x7f79e8133860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:22.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.954+0000 7f79ee5a0700 1 --2- 172.21.15.16:0/3668372448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79e809e110 0x7f79e8133860 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f79d8009820 tx=0x7f79d8009b30 comp rx=0 tx=0).stop 2023-09-30T22:32:22.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.954+0000 7f79ee5a0700 1 -- 172.21.15.16:0/3668372448 shutdown_connections 2023-09-30T22:32:22.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.954+0000 7f79ee5a0700 1 --2- 172.21.15.16:0/3668372448 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f79e809d800 0x7f79e809dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:22.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.954+0000 7f79ee5a0700 1 --2- 172.21.15.16:0/3668372448 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79e809e110 0x7f79e8133860 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:22.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.954+0000 7f79ee5a0700 1 -- 172.21.15.16:0/3668372448 >> 172.21.15.16:0/3668372448 conn(0x7f79e80a2e20 msgr2=0x7f79e80a3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:22.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.954+0000 7f79ee5a0700 1 -- 172.21.15.16:0/3668372448 shutdown_connections 2023-09-30T22:32:22.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.954+0000 7f79ee5a0700 1 -- 172.21.15.16:0/3668372448 wait complete. 2023-09-30T22:32:22.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.955+0000 7f79ee5a0700 1 Processor -- start 2023-09-30T22:32:22.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.955+0000 7f79ee5a0700 1 -- start start 2023-09-30T22:32:22.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.955+0000 7f79ee5a0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f79e809d800 0x7f79e811e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:22.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.955+0000 7f79ee5a0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f79e809e110 0x7f79e811ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:22.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.955+0000 7f79ee5a0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79e8133da0 0x7f79e811f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:22.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.955+0000 7f79ee5a0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e8120af0 con 0x7f79e8133da0 2023-09-30T22:32:22.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.955+0000 7f79ee5a0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79e8121f30 con 0x7f79e809e110 2023-09-30T22:32:22.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.955+0000 7f79ee5a0700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f79e8120410 con 0x7f79e809d800 2023-09-30T22:32:22.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.955+0000 7f79ecd9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f79e809e110 0x7f79e811ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:22.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.956+0000 7f79ecd9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f79e809e110 0x7f79e811ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:59798/0 (socket says 172.21.15.16:59798) 2023-09-30T22:32:22.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.956+0000 7f79ecd9d700 1 -- 172.21.15.16:0/2556591582 learned_addr learned my addr 172.21.15.16:0/2556591582 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:22.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.956+0000 7f79edd9f700 1 --2- 172.21.15.16:0/2556591582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79e8133da0 0x7f79e811f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:22.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.956+0000 7f79ecd9d700 1 -- 172.21.15.16:0/2556591582 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f79e809d800 msgr2=0x7f79e811e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:22.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.956+0000 7f79ecd9d700 1 --2- 172.21.15.16:0/2556591582 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f79e809d800 0x7f79e811e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:22.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.956+0000 7f79ecd9d700 1 -- 172.21.15.16:0/2556591582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79e8133da0 msgr2=0x7f79e811f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:22.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.956+0000 7f79ecd9d700 1 --2- 172.21.15.16:0/2556591582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79e8133da0 0x7f79e811f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:22.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.956+0000 7f79ecd9d700 1 -- 172.21.15.16:0/2556591582 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d8009430 con 0x7f79e809e110 2023-09-30T22:32:22.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.956+0000 7f79ecd9d700 1 --2- 172.21.15.16:0/2556591582 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f79e809e110 0x7f79e811ec30 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f79d8000c00 tx=0x7f79d8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:22.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.956+0000 7f79ddffb700 1 -- 172.21.15.16:0/2556591582 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d8004240 con 0x7f79e809e110 2023-09-30T22:32:22.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.956+0000 7f79ddffb700 1 -- 172.21.15.16:0/2556591582 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d80043a0 con 0x7f79e809e110 2023-09-30T22:32:22.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.956+0000 7f79ee5a0700 1 -- 172.21.15.16:0/2556591582 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79e8133250 con 0x7f79e809e110 2023-09-30T22:32:22.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.956+0000 7f79ddffb700 1 -- 172.21.15.16:0/2556591582 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d80036a0 con 0x7f79e809e110 2023-09-30T22:32:22.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.957+0000 7f79ee5a0700 1 -- 172.21.15.16:0/2556591582 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79e8121c70 con 0x7f79e809e110 2023-09-30T22:32:22.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.958+0000 7f79ddffb700 1 -- 172.21.15.16:0/2556591582 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f79d8022070 con 0x7f79e809e110 2023-09-30T22:32:22.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.958+0000 7f79ddffb700 1 -- 172.21.15.16:0/2556591582 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f79d8040fa0 con 0x7f79e809e110 2023-09-30T22:32:22.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.959+0000 7f79ee5a0700 1 -- 172.21.15.16:0/2556591582 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79e804f2a0 con 0x7f79e809e110 2023-09-30T22:32:22.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.966+0000 7f79ddffb700 1 -- 172.21.15.16:0/2556591582 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f79d802a070 con 0x7f79e809e110 2023-09-30T22:32:22.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.973+0000 7f1dd10e2700 1 Processor -- start 2023-09-30T22:32:22.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.973+0000 7f1dd10e2700 1 -- start start 2023-09-30T22:32:22.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.974+0000 7f1dd10e2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1dcc09e1d0 0x7f1dcc138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:22.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.974+0000 7f1dd10e2700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1dcc139170 0x7f1dcc13b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:22.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.974+0000 7f1dd10e2700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dcc09b120 con 0x7f1dcc09d6f0 2023-09-30T22:32:22.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.974+0000 7f1dd10e2700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dcc06b460 con 0x7f1dcc139170 2023-09-30T22:32:22.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.974+0000 7f1dd10e2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dcc09a7e0 con 0x7f1dcc09e1d0 2023-09-30T22:32:22.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.974+0000 7f1dca59c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1dcc09e1d0 0x7f1dcc138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:22.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.974+0000 7f1dca59c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1dcc09e1d0 0x7f1dcc138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:59806/0 (socket says 172.21.15.16:59806) 2023-09-30T22:32:22.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.974+0000 7f1dca59c700 1 -- 172.21.15.16:0/3213722298 learned_addr learned my addr 172.21.15.16:0/3213722298 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:22.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.974+0000 7f1dcb59e700 1 --2- 172.21.15.16:0/3213722298 >> v2:172.21.15.16:3300/0 conn(0x7f1dcc139170 0x7f1dcc13b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:22.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.974+0000 7f1dca59c700 1 -- 172.21.15.16:0/3213722298 >> v2:172.21.15.16:3300/0 conn(0x7f1dcc139170 msgr2=0x7f1dcc13b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:22.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.974+0000 7f1dca59c700 1 --2- 172.21.15.16:0/3213722298 >> v2:172.21.15.16:3300/0 conn(0x7f1dcc139170 0x7f1dcc13b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:22.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.974+0000 7f1dca59c700 1 -- 172.21.15.16:0/3213722298 >> v1:172.21.15.16:6789/0 conn(0x7f1dcc09d6f0 legacy=0x7f1dcc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:22.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.974+0000 7f1dca59c700 1 -- 172.21.15.16:0/3213722298 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dcc09a500 con 0x7f1dcc09e1d0 2023-09-30T22:32:22.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.975+0000 7f1dca59c700 1 --2- 172.21.15.16:0/3213722298 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1dcc09e1d0 0x7f1dcc138c30 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0009920 tx=0x7f1dc0009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0a40c7861cfb4a7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:22.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.975+0000 7f1dc9d9b700 1 -- 172.21.15.16:0/3213722298 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc0011830 con 0x7f1dcc09e1d0 2023-09-30T22:32:22.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.975+0000 7f1dc9d9b700 1 -- 172.21.15.16:0/3213722298 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dc0011990 con 0x7f1dcc09e1d0 2023-09-30T22:32:22.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.975+0000 7f1dc9d9b700 1 -- 172.21.15.16:0/3213722298 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc0011d00 con 0x7f1dcc09e1d0 2023-09-30T22:32:22.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.975+0000 7f1dd10e2700 1 -- 172.21.15.16:0/3213722298 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1dcc09e1d0 msgr2=0x7f1dcc138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:22.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.975+0000 7f1dd10e2700 1 --2- 172.21.15.16:0/3213722298 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1dcc09e1d0 0x7f1dcc138c30 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0009920 tx=0x7f1dc0009c30 comp rx=0 tx=0).stop 2023-09-30T22:32:22.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.975+0000 7f1dd10e2700 1 -- 172.21.15.16:0/3213722298 shutdown_connections 2023-09-30T22:32:22.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.975+0000 7f1dd10e2700 1 --2- 172.21.15.16:0/3213722298 >> v2:172.21.15.16:3300/0 conn(0x7f1dcc139170 0x7f1dcc13b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:22.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.975+0000 7f1dd10e2700 1 --2- 172.21.15.16:0/3213722298 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1dcc09e1d0 0x7f1dcc138c30 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:22.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.975+0000 7f1dd10e2700 1 -- 172.21.15.16:0/3213722298 >> 172.21.15.16:0/3213722298 conn(0x7f1dcc066f50 msgr2=0x7f1dcc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:22.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.976+0000 7f1dd10e2700 1 -- 172.21.15.16:0/3213722298 shutdown_connections 2023-09-30T22:32:22.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.976+0000 7f1dd10e2700 1 -- 172.21.15.16:0/3213722298 wait complete. 2023-09-30T22:32:22.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.977+0000 7f1dd10e2700 1 Processor -- start 2023-09-30T22:32:22.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.977+0000 7f1dd10e2700 1 -- start start 2023-09-30T22:32:22.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.977+0000 7f1dd10e2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1dcc09d6f0 0x7f1dcc1c8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:22.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.977+0000 7f1dd10e2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dcc09e1d0 0x7f1dcc1c89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:22.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.978+0000 7f1dd10e2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1dcc139170 0x7f1dcc1c8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:22.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.978+0000 7f1dd10e2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dcc122650 con 0x7f1dcc09e1d0 2023-09-30T22:32:22.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.978+0000 7f1dd10e2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dcc120830 con 0x7f1dcc139170 2023-09-30T22:32:22.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.978+0000 7f1dd10e2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1dcc072690 con 0x7f1dcc09d6f0 2023-09-30T22:32:22.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.978+0000 7f1dca59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dcc09e1d0 0x7f1dcc1c89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:22.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.978+0000 7f1dca59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dcc09e1d0 0x7f1dcc1c89a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34792/0 (socket says 172.21.15.16:34792) 2023-09-30T22:32:22.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.978+0000 7f1dcb59e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1dcc139170 0x7f1dcc1c8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:22.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.978+0000 7f1dca59c700 1 -- 172.21.15.16:0/2488582702 learned_addr learned my addr 172.21.15.16:0/2488582702 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:22.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.978+0000 7f1dca59c700 1 -- 172.21.15.16:0/2488582702 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1dcc09d6f0 msgr2=0x7f1dcc1c8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:22.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.978+0000 7f1dca59c700 1 --2- 172.21.15.16:0/2488582702 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1dcc09d6f0 0x7f1dcc1c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:22.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.978+0000 7f1dca59c700 1 -- 172.21.15.16:0/2488582702 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1dcc139170 msgr2=0x7f1dcc1c8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:22.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.978+0000 7f1dca59c700 1 --2- 172.21.15.16:0/2488582702 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1dcc139170 0x7f1dcc1c8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:22.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.978+0000 7f1dca59c700 1 -- 172.21.15.16:0/2488582702 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc0009550 con 0x7f1dcc09e1d0 2023-09-30T22:32:22.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.978+0000 7f1dcb59e700 1 --2- 172.21.15.16:0/2488582702 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1dcc139170 0x7f1dcc1c8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:22.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.978+0000 7f1dca59c700 1 --2- 172.21.15.16:0/2488582702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dcc09e1d0 0x7f1dcc1c89a0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0009130 tx=0x7f1dc0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:22.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.979+0000 7f1db37fe700 1 -- 172.21.15.16:0/2488582702 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc0004300 con 0x7f1dcc09e1d0 2023-09-30T22:32:22.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.979+0000 7f1db37fe700 1 -- 172.21.15.16:0/2488582702 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dc001f070 con 0x7f1dcc09e1d0 2023-09-30T22:32:22.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.979+0000 7f1dd10e2700 1 -- 172.21.15.16:0/2488582702 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dcc09c710 con 0x7f1dcc09e1d0 2023-09-30T22:32:22.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.979+0000 7f1dd10e2700 1 -- 172.21.15.16:0/2488582702 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dcc122cc0 con 0x7f1dcc09e1d0 2023-09-30T22:32:22.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.979+0000 7f1db37fe700 1 -- 172.21.15.16:0/2488582702 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dc00036a0 con 0x7f1dcc09e1d0 2023-09-30T22:32:22.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.980+0000 7f1db37fe700 1 -- 172.21.15.16:0/2488582702 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1dc0021030 con 0x7f1dcc09e1d0 2023-09-30T22:32:22.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.980+0000 7f1db37fe700 1 -- 172.21.15.16:0/2488582702 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f1dc0040870 con 0x7f1dcc09e1d0 2023-09-30T22:32:22.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.980+0000 7f1dd10e2700 1 -- 172.21.15.16:0/2488582702 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dac0052e0 con 0x7f1dcc09e1d0 2023-09-30T22:32:22.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:22.985+0000 7f1db37fe700 1 -- 172.21.15.16:0/2488582702 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1dc00239c0 con 0x7f1dcc09e1d0 2023-09-30T22:32:23.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:23.126+0000 7f1f7f47f700 1 -- 172.21.15.16:0/3530353305 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1f7806b9e0 con 0x7f1f7809e390 2023-09-30T22:32:23.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:23.127+0000 7f1f6dffb700 1 -- 172.21.15.16:0/3530353305 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1f74017070 con 0x7f1f7809e390 2023-09-30T22:32:23.128 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:32:23.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:23.130+0000 7fa67ccf5700 1 -- 172.21.15.16:0/3162304040 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa6781216c0 con 0x7fa67809d6f0 2023-09-30T22:32:23.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:23.130+0000 7f1f7f47f700 1 -- 172.21.15.16:0/3530353305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f7809e390 msgr2=0x7f1f7811eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:23.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:23.130+0000 7f1f7f47f700 1 --2- 172.21.15.16:0/3530353305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f7809e390 0x7f1f7811eac0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f1f7400ebe0 tx=0x7f1f74009e70 comp rx=0 tx=0).stop 2023-09-30T22:32:23.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:23.130+0000 7f1f7f47f700 1 -- 172.21.15.16:0/3530353305 shutdown_connections 2023-09-30T22:32:23.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:23.130+0000 7f1f7f47f700 1 --2- 172.21.15.16:0/3530353305 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f781341d0 0x7f1f7811f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:23.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:23.130+0000 7f1f7f47f700 1 --2- 172.21.15.16:0/3530353305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f7809e390 0x7f1f7811eac0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:23.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:23.130+0000 7fa666ffd700 1 -- 172.21.15.16:0/3162304040 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa6781216c0 con 0x7fa67809d6f0 2023-09-30T22:32:23.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:23.130+0000 7f1f7f47f700 1 --2- 172.21.15.16:0/3530353305 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f781338c0 0x7f1f7811f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:23.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:23.130+0000 7f1f7f47f700 1 -- 172.21.15.16:0/3530353305 >> 172.21.15.16:0/3530353305 conn(0x7f1f780a2aa0 msgr2=0x7f1f7809f180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:23.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:23.130+0000 7f1f7f47f700 1 -- 172.21.15.16:0/3530353305 shutdown_connections 2023-09-30T22:32:23.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:23.130+0000 7f1f7f47f700 1 -- 172.21.15.16:0/3530353305 wait complete. 2023-09-30T22:32:23.133 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:32:23.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:23.133+0000 7fa67ccf5700 1 -- 172.21.15.16:0/3162304040 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa67809d6f0 msgr2=0x7fa67811e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:23.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:23.133+0000 7fa67ccf5700 1 --2- 172.21.15.16:0/3162304040 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa67809d6f0 0x7fa67811e200 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa66800ed70 tx=0x7fa66800d630 comp rx=0 tx=0).stop 2023-09-30T22:32:23.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:23.134+0000 7fa67ccf5700 1 -- 172.21.15.16:0/3162304040 shutdown_connections 2023-09-30T22:32:23.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:23.134+0000 7fa67ccf5700 1 --2- 172.21.15.16:0/3162304040 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa67809d6f0 0x7fa67811e200 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:23.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:23.134+0000 7fa67ccf5700 1 --2- 172.21.15.16:0/3162304040 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa67809e410 0x7fa6781cc650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:23.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:23.134+0000 7fa67ccf5700 1 --2- 172.21.15.16:0/3162304040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa67809e000 0x7fa67811e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:23.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:23.134+0000 7fa67ccf5700 1 -- 172.21.15.16:0/3162304040 >> 172.21.15.16:0/3162304040 conn(0x7fa678066f50 msgr2=0x7fa678130a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:23.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:23.134+0000 7fa67ccf5700 1 -- 172.21.15.16:0/3162304040 shutdown_connections 2023-09-30T22:32:23.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:23.134+0000 7fa67ccf5700 1 -- 172.21.15.16:0/3162304040 wait complete. 2023-09-30T22:32:23.150 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:32:23.152 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:32:23.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:23.202+0000 7f1dd10e2700 1 -- 172.21.15.16:0/2488582702 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1dac005590 con 0x7f1dcc09e1d0 2023-09-30T22:32:23.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:23.202+0000 7f1db37fe700 1 -- 172.21.15.16:0/2488582702 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1dc0015020 con 0x7f1dcc09e1d0 2023-09-30T22:32:23.205 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:32:23.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:23.204+0000 7f1dd10e2700 1 -- 172.21.15.16:0/2488582702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dcc09e1d0 msgr2=0x7f1dcc1c89a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:23.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:23.204+0000 7f1dd10e2700 1 --2- 172.21.15.16:0/2488582702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dcc09e1d0 0x7f1dcc1c89a0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0009130 tx=0x7f1dc0004750 comp rx=0 tx=0).stop 2023-09-30T22:32:23.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:23.204+0000 7f1dd10e2700 1 -- 172.21.15.16:0/2488582702 shutdown_connections 2023-09-30T22:32:23.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:23.204+0000 7f1dd10e2700 1 --2- 172.21.15.16:0/2488582702 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1dcc139170 0x7f1dcc1c8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:23.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:23.204+0000 7f1dd10e2700 1 --2- 172.21.15.16:0/2488582702 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1dcc09d6f0 0x7f1dcc1c8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:23.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:23.204+0000 7f1dd10e2700 1 --2- 172.21.15.16:0/2488582702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dcc09e1d0 0x7f1dcc1c89a0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:23.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:23.204+0000 7f1dd10e2700 1 -- 172.21.15.16:0/2488582702 >> 172.21.15.16:0/2488582702 conn(0x7f1dcc066f50 msgr2=0x7f1dcc0a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:23.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:23.204+0000 7f1dd10e2700 1 -- 172.21.15.16:0/2488582702 shutdown_connections 2023-09-30T22:32:23.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:23.204+0000 7f1dd10e2700 1 -- 172.21.15.16:0/2488582702 wait complete. 2023-09-30T22:32:23.219 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:32:23.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:23.238+0000 7f79ee5a0700 1 -- 172.21.15.16:0/2556591582 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f79e809a260 con 0x7f79e809e110 2023-09-30T22:32:23.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:23.238+0000 7f79ddffb700 1 -- 172.21.15.16:0/2556591582 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f79d802f9e0 con 0x7f79e809e110 2023-09-30T22:32:23.239 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:32:23.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:23.240+0000 7f79ee5a0700 1 -- 172.21.15.16:0/2556591582 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f79e809e110 msgr2=0x7f79e811ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:23.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:23.240+0000 7f79ee5a0700 1 --2- 172.21.15.16:0/2556591582 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f79e809e110 0x7f79e811ec30 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f79d8000c00 tx=0x7f79d8004750 comp rx=0 tx=0).stop 2023-09-30T22:32:23.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:23.240+0000 7f79ee5a0700 1 -- 172.21.15.16:0/2556591582 shutdown_connections 2023-09-30T22:32:23.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:23.240+0000 7f79ee5a0700 1 --2- 172.21.15.16:0/2556591582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f79e8133da0 0x7f79e811f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:23.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:23.240+0000 7f79ee5a0700 1 --2- 172.21.15.16:0/2556591582 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f79e809d800 0x7f79e811e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:23.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:23.240+0000 7f79ee5a0700 1 --2- 172.21.15.16:0/2556591582 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f79e809e110 0x7f79e811ec30 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:23.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:23.240+0000 7f79ee5a0700 1 -- 172.21.15.16:0/2556591582 >> 172.21.15.16:0/2556591582 conn(0x7f79e80a2e20 msgr2=0x7f79e8070c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:23.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:23.240+0000 7f79ee5a0700 1 -- 172.21.15.16:0/2556591582 shutdown_connections 2023-09-30T22:32:23.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:23.240+0000 7f79ee5a0700 1 -- 172.21.15.16:0/2556591582 wait complete. 2023-09-30T22:32:23.255 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:32:24.151 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:32:24.153 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:32:24.220 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:32:24.256 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:32:24.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.301+0000 7fb6170ab700 1 Processor -- start 2023-09-30T22:32:24.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.301+0000 7fb6170ab700 1 -- start start 2023-09-30T22:32:24.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.302+0000 7fb6170ab700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb6101338c0 0x7fb610133c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:24.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.302+0000 7fb6170ab700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb61009e560 0x7fb61013b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:24.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.302+0000 7fb6170ab700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb61006f720 con 0x7fb6101341d0 2023-09-30T22:32:24.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.302+0000 7fb6170ab700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb610070c80 con 0x7fb6101338c0 2023-09-30T22:32:24.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.302+0000 7fb6170ab700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb61006fce0 con 0x7fb61009e560 2023-09-30T22:32:24.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.302+0000 7fb614e47700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb6101338c0 0x7fb610133c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:24.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.302+0000 7fb60ffff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb6101341d0 0x7fb61009de50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53472/0 (socket says 172.21.15.16:53472) 2023-09-30T22:32:24.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.302+0000 7fb615648700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb61009e560 0x7fb61013b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:24.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.302+0000 7fb60ffff700 1 -- 172.21.15.16:0/1524448688 learned_addr learned my addr 172.21.15.16:0/1524448688 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:24.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.302+0000 7f6000303700 1 Processor -- start 2023-09-30T22:32:24.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.302+0000 7fb614e47700 1 -- 172.21.15.16:0/1524448688 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb61009e560 msgr2=0x7fb61013b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:24.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.302+0000 7fb614e47700 1 --2- 172.21.15.16:0/1524448688 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb61009e560 0x7fb61013b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:24.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.302+0000 7fb60f7fe700 1 -- 172.21.15.16:0/1524448688 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1169153154 0 0) 0x7fb61006f720 con 0x7fb6101341d0 2023-09-30T22:32:24.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.302+0000 7fb614e47700 1 -- 172.21.15.16:0/1524448688 >> v1:172.21.15.16:6789/0 conn(0x7fb6101341d0 legacy=0x7fb61009de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:24.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.302+0000 7fb614e47700 1 -- 172.21.15.16:0/1524448688 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6101336a0 con 0x7fb6101338c0 2023-09-30T22:32:24.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.303+0000 7fb614e47700 1 --2- 172.21.15.16:0/1524448688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6101338c0 0x7fb610133c90 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fb604009840 tx=0x7fb604009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21bc26d50842de79 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:24.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.303+0000 7fb60f7fe700 1 -- 172.21.15.16:0/1524448688 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb604011830 con 0x7fb6101338c0 2023-09-30T22:32:24.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.303+0000 7fb60f7fe700 1 -- 172.21.15.16:0/1524448688 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb604011990 con 0x7fb6101338c0 2023-09-30T22:32:24.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.303+0000 7fb60f7fe700 1 -- 172.21.15.16:0/1524448688 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb604011d20 con 0x7fb6101338c0 2023-09-30T22:32:24.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.302+0000 7f6000303700 1 -- start start 2023-09-30T22:32:24.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.303+0000 7f6000303700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5ff809d6f0 0x7f5ff809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:24.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.303+0000 7f6000303700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5ff809e000 0x7f5ff8130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:24.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.304+0000 7f6000303700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5ff809e410 0x7f5ff8130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:24.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.304+0000 7f6000303700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ff809f520 con 0x7f5ff809e410 2023-09-30T22:32:24.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.304+0000 7f6000303700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff809f130 con 0x7f5ff809e000 2023-09-30T22:32:24.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.304+0000 7f5ffe09f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5ff809d6f0 0x7f5ff809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:24.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.304+0000 7f5ffe09f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5ff809d6f0 0x7f5ff809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:36274/0 (socket says 172.21.15.16:36274) 2023-09-30T22:32:24.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.304+0000 7f5ffe09f700 1 -- 172.21.15.16:0/2947730149 learned_addr learned my addr 172.21.15.16:0/2947730149 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:24.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.304+0000 7f5ffe8a0700 1 --2- 172.21.15.16:0/2947730149 >> v2:172.21.15.16:3300/0 conn(0x7f5ff809e410 0x7f5ff8130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:24.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.304+0000 7fb6170ab700 1 -- 172.21.15.16:0/1524448688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6101338c0 msgr2=0x7fb610133c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:24.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.304+0000 7fb6170ab700 1 --2- 172.21.15.16:0/1524448688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6101338c0 0x7fb610133c90 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fb604009840 tx=0x7fb604009b50 comp rx=0 tx=0).stop 2023-09-30T22:32:24.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.304+0000 7f6000303700 1 -- 172.21.15.16:0/2947730149 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff809e930 con 0x7f5ff809d6f0 2023-09-30T22:32:24.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.304+0000 7f5ffd89e700 1 --2- 172.21.15.16:0/2947730149 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5ff809e000 0x7f5ff8130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:24.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.304+0000 7fb6170ab700 1 -- 172.21.15.16:0/1524448688 shutdown_connections 2023-09-30T22:32:24.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.304+0000 7fb6170ab700 1 --2- 172.21.15.16:0/1524448688 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb61009e560 0x7fb61013b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:24.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.304+0000 7fb6170ab700 1 --2- 172.21.15.16:0/1524448688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6101338c0 0x7fb610133c90 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:24.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.304+0000 7fb6170ab700 1 -- 172.21.15.16:0/1524448688 >> 172.21.15.16:0/1524448688 conn(0x7fb6100a2aa0 msgr2=0x7fb6100a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:24.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.304+0000 7fb6170ab700 1 -- 172.21.15.16:0/1524448688 shutdown_connections 2023-09-30T22:32:24.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.304+0000 7fb6170ab700 1 -- 172.21.15.16:0/1524448688 wait complete. 2023-09-30T22:32:24.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.304+0000 7f5ffe8a0700 1 -- 172.21.15.16:0/2947730149 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5ff809d6f0 msgr2=0x7f5ff809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:24.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.304+0000 7f5ffe8a0700 1 --2- 172.21.15.16:0/2947730149 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5ff809d6f0 0x7f5ff809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:24.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.304+0000 7f5ffe8a0700 1 -- 172.21.15.16:0/2947730149 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5ff809e000 msgr2=0x7f5ff8130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:24.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.304+0000 7f5ffe8a0700 1 --2- 172.21.15.16:0/2947730149 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5ff809e000 0x7f5ff8130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:24.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.304+0000 7f5ffe8a0700 1 -- 172.21.15.16:0/2947730149 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff809b2d0 con 0x7f5ff809e410 2023-09-30T22:32:24.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.305+0000 7f5ffe8a0700 1 --2- 172.21.15.16:0/2947730149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff809e410 0x7f5ff8130ba0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0009840 tx=0x7f5ff0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e6f99b567ecbd06 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:24.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.305+0000 7f5ffd09d700 1 -- 172.21.15.16:0/2947730149 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff0011830 con 0x7f5ff809e410 2023-09-30T22:32:24.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.305+0000 7f5ffd09d700 1 -- 172.21.15.16:0/2947730149 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff0011990 con 0x7f5ff809e410 2023-09-30T22:32:24.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.305+0000 7f5ffd09d700 1 -- 172.21.15.16:0/2947730149 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff0011d20 con 0x7f5ff809e410 2023-09-30T22:32:24.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.306+0000 7f6000303700 1 -- 172.21.15.16:0/2947730149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff809e410 msgr2=0x7f5ff8130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:24.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.306+0000 7f6000303700 1 --2- 172.21.15.16:0/2947730149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff809e410 0x7f5ff8130ba0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0009840 tx=0x7f5ff0009b50 comp rx=0 tx=0).stop 2023-09-30T22:32:24.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.306+0000 7fb6170ab700 1 Processor -- start 2023-09-30T22:32:24.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.306+0000 7fb6170ab700 1 -- start start 2023-09-30T22:32:24.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.306+0000 7f6000303700 1 -- 172.21.15.16:0/2947730149 shutdown_connections 2023-09-30T22:32:24.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.306+0000 7f6000303700 1 --2- 172.21.15.16:0/2947730149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff809e410 0x7f5ff8130ba0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:24.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.306+0000 7f6000303700 1 --2- 172.21.15.16:0/2947730149 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5ff809d6f0 0x7f5ff809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:24.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.306+0000 7f6000303700 1 --2- 172.21.15.16:0/2947730149 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5ff809e000 0x7f5ff8130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:24.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.306+0000 7f6000303700 1 -- 172.21.15.16:0/2947730149 >> 172.21.15.16:0/2947730149 conn(0x7f5ff8066f50 msgr2=0x7f5ff809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:24.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.306+0000 7f6000303700 1 -- 172.21.15.16:0/2947730149 shutdown_connections 2023-09-30T22:32:24.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.306+0000 7f6000303700 1 -- 172.21.15.16:0/2947730149 wait complete. 2023-09-30T22:32:24.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.307+0000 7fb6170ab700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb61009e560 0x7fb61011b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:24.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.307+0000 7fb6170ab700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6101338c0 0x7fb61011bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:24.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.307+0000 7fb6170ab700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb6101341d0 0x7fb61011c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:24.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.307+0000 7fb6170ab700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61011ef90 con 0x7fb6101338c0 2023-09-30T22:32:24.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.307+0000 7fb6170ab700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb61011cc40 con 0x7fb61009e560 2023-09-30T22:32:24.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.307+0000 7fb6170ab700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb61006f960 con 0x7fb6101341d0 2023-09-30T22:32:24.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.307+0000 7fb614e47700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb61009e560 0x7fb61011b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:24.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.307+0000 7fb60ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6101338c0 0x7fb61011bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:24.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.307+0000 7fb614e47700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb61009e560 0x7fb61011b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:59844/0 (socket says 172.21.15.16:59844) 2023-09-30T22:32:24.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.307+0000 7fb614e47700 1 -- 172.21.15.16:0/1902160406 learned_addr learned my addr 172.21.15.16:0/1902160406 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:24.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.307+0000 7fb615648700 1 --2- 172.21.15.16:0/1902160406 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb6101341d0 0x7fb61011c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:24.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.308+0000 7f6000303700 1 Processor -- start 2023-09-30T22:32:24.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.308+0000 7fb614e47700 1 -- 172.21.15.16:0/1902160406 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb6101341d0 msgr2=0x7fb61011c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:24.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.308+0000 7fb614e47700 1 --2- 172.21.15.16:0/1902160406 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb6101341d0 0x7fb61011c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:24.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.308+0000 7fb614e47700 1 -- 172.21.15.16:0/1902160406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6101338c0 msgr2=0x7fb61011bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:24.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.308+0000 7fb614e47700 1 --2- 172.21.15.16:0/1902160406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6101338c0 0x7fb61011bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:24.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.308+0000 7fb614e47700 1 -- 172.21.15.16:0/1902160406 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6040094a0 con 0x7fb61009e560 2023-09-30T22:32:24.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.308+0000 7fb60ffff700 1 --2- 172.21.15.16:0/1902160406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6101338c0 0x7fb61011bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:24.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.308+0000 7f6000303700 1 -- start start 2023-09-30T22:32:24.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.308+0000 7fb614e47700 1 --2- 172.21.15.16:0/1902160406 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb61009e560 0x7fb61011b7e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb6040090b0 tx=0x7fb604004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:24.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.308+0000 7fb60d7fa700 1 -- 172.21.15.16:0/1902160406 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6040036a0 con 0x7fb61009e560 2023-09-30T22:32:24.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.308+0000 7fb60d7fa700 1 -- 172.21.15.16:0/1902160406 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb60401f070 con 0x7fb61009e560 2023-09-30T22:32:24.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.308+0000 7fb60d7fa700 1 -- 172.21.15.16:0/1902160406 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb604003a40 con 0x7fb61009e560 2023-09-30T22:32:24.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.309+0000 7f6000303700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff809d6f0 0x7f5ff81225c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:24.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.309+0000 7f6000303700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5ff809e000 0x7f5ff8122b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:24.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.309+0000 7f6000303700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5ff809e410 0x7f5ff8123040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:24.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.309+0000 7f6000303700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff81260e0 con 0x7f5ff809d6f0 2023-09-30T22:32:24.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.309+0000 7f6000303700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff81241d0 con 0x7f5ff809e000 2023-09-30T22:32:24.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.309+0000 7f6000303700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff80a16b0 con 0x7f5ff809e410 2023-09-30T22:32:24.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.309+0000 7f5ffe09f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff809d6f0 0x7f5ff81225c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:24.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.309+0000 7f5ffe09f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff809d6f0 0x7f5ff81225c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34836/0 (socket says 172.21.15.16:34836) 2023-09-30T22:32:24.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.309+0000 7f5ffe09f700 1 -- 172.21.15.16:0/908611250 learned_addr learned my addr 172.21.15.16:0/908611250 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:24.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.309+0000 7f5ffe8a0700 1 --2- 172.21.15.16:0/908611250 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5ff809e410 0x7f5ff8123040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:24.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.309+0000 7fb6170ab700 1 -- 172.21.15.16:0/1902160406 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb61011d470 con 0x7fb61009e560 2023-09-30T22:32:24.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.309+0000 7f5ffe8a0700 1 -- 172.21.15.16:0/908611250 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5ff809e000 msgr2=0x7f5ff8122b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:24.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.309+0000 7f5ffe8a0700 1 --2- 172.21.15.16:0/908611250 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5ff809e000 0x7f5ff8122b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:24.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.309+0000 7f5ffe8a0700 1 -- 172.21.15.16:0/908611250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff809d6f0 msgr2=0x7f5ff81225c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:24.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.309+0000 7f5ffe8a0700 1 --2- 172.21.15.16:0/908611250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff809d6f0 0x7f5ff81225c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:24.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.309+0000 7f5ffe8a0700 1 -- 172.21.15.16:0/908611250 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff00094a0 con 0x7f5ff809e410 2023-09-30T22:32:24.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.309+0000 7fb6170ab700 1 -- 172.21.15.16:0/1902160406 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6100a10d0 con 0x7fb61009e560 2023-09-30T22:32:24.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.310+0000 7f5ffe8a0700 1 --2- 172.21.15.16:0/908611250 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5ff809e410 0x7f5ff8123040 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0009840 tx=0x7f5ff0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:24.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.310+0000 7fb60d7fa700 1 -- 172.21.15.16:0/1902160406 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb604003ba0 con 0x7fb61009e560 2023-09-30T22:32:24.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.310+0000 7fb60d7fa700 1 -- 172.21.15.16:0/1902160406 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fb604021070 con 0x7fb61009e560 2023-09-30T22:32:24.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.310+0000 7f5feeffd700 1 -- 172.21.15.16:0/908611250 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff0004450 con 0x7f5ff809e410 2023-09-30T22:32:24.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.311+0000 7f5feeffd700 1 -- 172.21.15.16:0/908611250 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff00036a0 con 0x7f5ff809e410 2023-09-30T22:32:24.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.311+0000 7f5feeffd700 1 -- 172.21.15.16:0/908611250 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff00045f0 con 0x7f5ff809e410 2023-09-30T22:32:24.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.311+0000 7f6000303700 1 -- 172.21.15.16:0/908611250 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ff809c700 con 0x7f5ff809e410 2023-09-30T22:32:24.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.311+0000 7f6000303700 1 -- 172.21.15.16:0/908611250 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ff81265e0 con 0x7f5ff809e410 2023-09-30T22:32:24.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.311+0000 7fb6170ab700 1 -- 172.21.15.16:0/1902160406 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb610074ea0 con 0x7fb61009e560 2023-09-30T22:32:24.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.312+0000 7f5feeffd700 1 -- 172.21.15.16:0/908611250 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5ff0003a50 con 0x7f5ff809e410 2023-09-30T22:32:24.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.312+0000 7f6000303700 1 -- 172.21.15.16:0/908611250 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ff8074ea0 con 0x7f5ff809e410 2023-09-30T22:32:24.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.312+0000 7f5feeffd700 1 -- 172.21.15.16:0/908611250 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f5ff0040520 con 0x7f5ff809e410 2023-09-30T22:32:24.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.317+0000 7fb60d7fa700 1 -- 172.21.15.16:0/1902160406 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb60402b080 con 0x7fb61009e560 2023-09-30T22:32:24.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.317+0000 7f5feeffd700 1 -- 172.21.15.16:0/908611250 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5ff0028020 con 0x7f5ff809e410 2023-09-30T22:32:24.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.367+0000 7fad7f760700 1 Processor -- start 2023-09-30T22:32:24.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.368+0000 7fad7f760700 1 -- start start 2023-09-30T22:32:24.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.368+0000 7fad7f760700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fad7809e930 0x7fad7809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:24.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.368+0000 7fad7f760700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fad7809f240 0x7fad781317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:24.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.368+0000 7fad7f760700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fad7809f650 0x7fad78131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:24.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.368+0000 7fad7f760700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad78070940 con 0x7fad7809f650 2023-09-30T22:32:24.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.368+0000 7fad7f760700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad7806fd60 con 0x7fad7809f240 2023-09-30T22:32:24.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.368+0000 7fad7f760700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fad7806fa80 con 0x7fad7809e930 2023-09-30T22:32:24.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.369+0000 7fad7d4fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fad7809e930 0x7fad7809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:24.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.369+0000 7fad7d4fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fad7809e930 0x7fad7809ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:36300/0 (socket says 172.21.15.16:36300) 2023-09-30T22:32:24.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.369+0000 7fad7d4fc700 1 -- 172.21.15.16:0/2056885675 learned_addr learned my addr 172.21.15.16:0/2056885675 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:24.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.369+0000 7fad7ccfb700 1 --2- 172.21.15.16:0/2056885675 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fad7809f240 0x7fad781317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:24.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.369+0000 7fad7dcfd700 1 --2- 172.21.15.16:0/2056885675 >> v2:172.21.15.16:3300/0 conn(0x7fad7809f650 0x7fad78131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:24.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.369+0000 7fad7dcfd700 1 -- 172.21.15.16:0/2056885675 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fad7809e930 msgr2=0x7fad7809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:24.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.369+0000 7fad7dcfd700 1 --2- 172.21.15.16:0/2056885675 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fad7809e930 0x7fad7809ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:24.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.369+0000 7fad7dcfd700 1 -- 172.21.15.16:0/2056885675 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fad7809f240 msgr2=0x7fad781317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:24.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.369+0000 7fad7dcfd700 1 --2- 172.21.15.16:0/2056885675 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fad7809f240 0x7fad781317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:24.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.369+0000 7fad7dcfd700 1 -- 172.21.15.16:0/2056885675 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad7809fb60 con 0x7fad7809f650 2023-09-30T22:32:24.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.369+0000 7fad7ccfb700 1 --2- 172.21.15.16:0/2056885675 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fad7809f240 0x7fad781317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:24.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.369+0000 7fad7dcfd700 1 --2- 172.21.15.16:0/2056885675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad7809f650 0x7fad78131d30 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fad70009d90 tx=0x7fad7000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68366c39f91ea9e8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:24.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.370+0000 7fad6ffff700 1 -- 172.21.15.16:0/2056885675 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7000e070 con 0x7fad7809f650 2023-09-30T22:32:24.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.370+0000 7fad6ffff700 1 -- 172.21.15.16:0/2056885675 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad7000bb20 con 0x7fad7809f650 2023-09-30T22:32:24.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.370+0000 7fad6ffff700 1 -- 172.21.15.16:0/2056885675 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad70012430 con 0x7fad7809f650 2023-09-30T22:32:24.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.370+0000 7fad7f760700 1 -- 172.21.15.16:0/2056885675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad7809f650 msgr2=0x7fad78131d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:24.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.370+0000 7fad7f760700 1 --2- 172.21.15.16:0/2056885675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad7809f650 0x7fad78131d30 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fad70009d90 tx=0x7fad7000b330 comp rx=0 tx=0).stop 2023-09-30T22:32:24.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.370+0000 7fad7f760700 1 -- 172.21.15.16:0/2056885675 shutdown_connections 2023-09-30T22:32:24.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.371+0000 7fad7f760700 1 --2- 172.21.15.16:0/2056885675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad7809f650 0x7fad78131d30 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:24.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.371+0000 7fad7f760700 1 --2- 172.21.15.16:0/2056885675 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fad7809e930 0x7fad7809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:24.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.371+0000 7fad7f760700 1 --2- 172.21.15.16:0/2056885675 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fad7809f240 0x7fad781317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:24.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.371+0000 7fad7f760700 1 -- 172.21.15.16:0/2056885675 >> 172.21.15.16:0/2056885675 conn(0x7fad780a4210 msgr2=0x7fad780a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:24.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.371+0000 7fad7f760700 1 -- 172.21.15.16:0/2056885675 shutdown_connections 2023-09-30T22:32:24.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.371+0000 7fad7f760700 1 -- 172.21.15.16:0/2056885675 wait complete. 2023-09-30T22:32:24.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.372+0000 7fad7f760700 1 Processor -- start 2023-09-30T22:32:24.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.372+0000 7fad7f760700 1 -- start start 2023-09-30T22:32:24.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.373+0000 7fad7f760700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fad7809e930 0x7fad781317c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:24.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.373+0000 7fad7f760700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad7809f240 0x7fad78131d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:24.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.373+0000 7fad7f760700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fad7809f650 0x7fad781c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:24.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.373+0000 7fad7f760700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad78131400 con 0x7fad7809f240 2023-09-30T22:32:24.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.373+0000 7fad7f760700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad78133710 con 0x7fad7809e930 2023-09-30T22:32:24.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.373+0000 7fad7f760700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fad781345e0 con 0x7fad7809f650 2023-09-30T22:32:24.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.373+0000 7fad7ccfb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad7809f240 0x7fad78131d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:24.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.373+0000 7fad7ccfb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad7809f240 0x7fad78131d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34860/0 (socket says 172.21.15.16:34860) 2023-09-30T22:32:24.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.373+0000 7fad7ccfb700 1 -- 172.21.15.16:0/1429177776 learned_addr learned my addr 172.21.15.16:0/1429177776 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:24.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.373+0000 7fad7d4fc700 1 --2- 172.21.15.16:0/1429177776 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fad7809e930 0x7fad781317c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:24.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.373+0000 7fad7dcfd700 1 --2- 172.21.15.16:0/1429177776 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fad7809f650 0x7fad781c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:24.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.374+0000 7fad7ccfb700 1 -- 172.21.15.16:0/1429177776 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fad7809f650 msgr2=0x7fad781c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:24.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.374+0000 7fad7ccfb700 1 --2- 172.21.15.16:0/1429177776 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fad7809f650 0x7fad781c8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:24.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.374+0000 7fad7ccfb700 1 -- 172.21.15.16:0/1429177776 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fad7809e930 msgr2=0x7fad781317c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:24.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.374+0000 7fad7ccfb700 1 --2- 172.21.15.16:0/1429177776 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fad7809e930 0x7fad781317c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:24.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.374+0000 7fad7ccfb700 1 -- 172.21.15.16:0/1429177776 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad680094a0 con 0x7fad7809f240 2023-09-30T22:32:24.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.374+0000 7fad7d4fc700 1 --2- 172.21.15.16:0/1429177776 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fad7809e930 0x7fad781317c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:24.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.374+0000 7fad7ccfb700 1 --2- 172.21.15.16:0/1429177776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad7809f240 0x7fad78131d00 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fad6800ebb0 tx=0x7fad6800eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:24.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.374+0000 7fad6dffb700 1 -- 172.21.15.16:0/1429177776 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6800c850 con 0x7fad7809f240 2023-09-30T22:32:24.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.374+0000 7fad6dffb700 1 -- 172.21.15.16:0/1429177776 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad6800c9b0 con 0x7fad7809f240 2023-09-30T22:32:24.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.374+0000 7fad7f760700 1 -- 172.21.15.16:0/1429177776 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad70009a70 con 0x7fad7809f240 2023-09-30T22:32:24.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.374+0000 7fad6dffb700 1 -- 172.21.15.16:0/1429177776 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6800cd40 con 0x7fad7809f240 2023-09-30T22:32:24.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.374+0000 7fad7f760700 1 -- 172.21.15.16:0/1429177776 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad780a0190 con 0x7fad7809f240 2023-09-30T22:32:24.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.376+0000 7fad7f760700 1 -- 172.21.15.16:0/1429177776 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad78074ea0 con 0x7fad7809f240 2023-09-30T22:32:24.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.376+0000 7fad6dffb700 1 -- 172.21.15.16:0/1429177776 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fad68013470 con 0x7fad7809f240 2023-09-30T22:32:24.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.377+0000 7fad6dffb700 1 -- 172.21.15.16:0/1429177776 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fad680400f0 con 0x7fad7809f240 2023-09-30T22:32:24.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.381+0000 7fad6dffb700 1 -- 172.21.15.16:0/1429177776 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fad68011760 con 0x7fad7809f240 2023-09-30T22:32:24.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.423+0000 7fc9d89a8700 1 Processor -- start 2023-09-30T22:32:24.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.423+0000 7fc9d89a8700 1 -- start start 2023-09-30T22:32:24.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.424+0000 7fc9d89a8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc9d409e1d0 0x7fc9d4138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:24.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.424+0000 7fc9d89a8700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc9d4139170 0x7fc9d413b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:24.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.424+0000 7fc9d89a8700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9d409b120 con 0x7fc9d409d6f0 2023-09-30T22:32:24.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.424+0000 7fc9d89a8700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9d406b460 con 0x7fc9d4139170 2023-09-30T22:32:24.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.424+0000 7fc9d89a8700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d409a7e0 con 0x7fc9d409e1d0 2023-09-30T22:32:24.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.425+0000 7fc9d259c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc9d409d6f0 0x7fc9d409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53488/0 (socket says 172.21.15.16:53488) 2023-09-30T22:32:24.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.425+0000 7fc9d259c700 1 -- 172.21.15.16:0/845174987 learned_addr learned my addr 172.21.15.16:0/845174987 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:24.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.425+0000 7fc9d2d9d700 1 --2- 172.21.15.16:0/845174987 >> v2:172.21.15.16:3300/0 conn(0x7fc9d4139170 0x7fc9d413b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:24.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.425+0000 7fc9d1d9b700 1 --2- 172.21.15.16:0/845174987 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc9d409e1d0 0x7fc9d4138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:24.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.425+0000 7fc9d159a700 1 -- 172.21.15.16:0/845174987 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1156525596 0 0) 0x7fc9d409b120 con 0x7fc9d409d6f0 2023-09-30T22:32:24.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.425+0000 7fc9d159a700 1 -- 172.21.15.16:0/845174987 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9bc0035b0 con 0x7fc9d409d6f0 2023-09-30T22:32:24.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.425+0000 7fc9d1d9b700 1 -- 172.21.15.16:0/845174987 >> v2:172.21.15.16:3300/0 conn(0x7fc9d4139170 msgr2=0x7fc9d413b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:24.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.425+0000 7fc9d1d9b700 1 --2- 172.21.15.16:0/845174987 >> v2:172.21.15.16:3300/0 conn(0x7fc9d4139170 0x7fc9d413b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:24.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.425+0000 7fc9d1d9b700 1 -- 172.21.15.16:0/845174987 >> v1:172.21.15.16:6789/0 conn(0x7fc9d409d6f0 legacy=0x7fc9d409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:24.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.425+0000 7fc9d1d9b700 1 -- 172.21.15.16:0/845174987 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d409a500 con 0x7fc9d409e1d0 2023-09-30T22:32:24.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.425+0000 7fc9d1d9b700 1 --2- 172.21.15.16:0/845174987 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc9d409e1d0 0x7fc9d4138c30 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8009870 tx=0x7fc9c8009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81ef0d67e2f8273d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:24.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.426+0000 7fc9d159a700 1 -- 172.21.15.16:0/845174987 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c8011830 con 0x7fc9d409e1d0 2023-09-30T22:32:24.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.426+0000 7fc9d159a700 1 -- 172.21.15.16:0/845174987 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c8011990 con 0x7fc9d409e1d0 2023-09-30T22:32:24.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.426+0000 7fc9d159a700 1 -- 172.21.15.16:0/845174987 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c8011d00 con 0x7fc9d409e1d0 2023-09-30T22:32:24.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.427+0000 7fc9d89a8700 1 -- 172.21.15.16:0/845174987 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc9d409e1d0 msgr2=0x7fc9d4138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:24.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.427+0000 7fc9d89a8700 1 --2- 172.21.15.16:0/845174987 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc9d409e1d0 0x7fc9d4138c30 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc9c8009870 tx=0x7fc9c8009b80 comp rx=0 tx=0).stop 2023-09-30T22:32:24.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.427+0000 7fc9d89a8700 1 -- 172.21.15.16:0/845174987 shutdown_connections 2023-09-30T22:32:24.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.427+0000 7fc9d89a8700 1 --2- 172.21.15.16:0/845174987 >> v2:172.21.15.16:3300/0 conn(0x7fc9d4139170 0x7fc9d413b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:24.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.427+0000 7fc9d89a8700 1 --2- 172.21.15.16:0/845174987 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc9d409e1d0 0x7fc9d4138c30 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:24.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.427+0000 7fc9d89a8700 1 -- 172.21.15.16:0/845174987 >> 172.21.15.16:0/845174987 conn(0x7fc9d4066f50 msgr2=0x7fc9d409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:24.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.427+0000 7fc9d89a8700 1 -- 172.21.15.16:0/845174987 shutdown_connections 2023-09-30T22:32:24.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.427+0000 7fc9d89a8700 1 -- 172.21.15.16:0/845174987 wait complete. 2023-09-30T22:32:24.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.428+0000 7fc9d89a8700 1 Processor -- start 2023-09-30T22:32:24.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.428+0000 7fc9d89a8700 1 -- start start 2023-09-30T22:32:24.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.429+0000 7fc9d89a8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9d409d6f0 0x7fc9d4120690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:24.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.429+0000 7fc9d89a8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc9d409e1d0 0x7fc9d4120bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:24.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.429+0000 7fc9d89a8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc9d4139170 0x7fc9d4121110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:24.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.429+0000 7fc9d89a8700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d4135fa0 con 0x7fc9d409d6f0 2023-09-30T22:32:24.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.429+0000 7fc9d89a8700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d4123ed0 con 0x7fc9d409e1d0 2023-09-30T22:32:24.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.429+0000 7fc9d89a8700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d4071f20 con 0x7fc9d4139170 2023-09-30T22:32:24.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.429+0000 7fc9d259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9d409d6f0 0x7fc9d4120690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:24.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.429+0000 7fc9d259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9d409d6f0 0x7fc9d4120690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34888/0 (socket says 172.21.15.16:34888) 2023-09-30T22:32:24.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.429+0000 7fc9d259c700 1 -- 172.21.15.16:0/2207359681 learned_addr learned my addr 172.21.15.16:0/2207359681 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:24.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.429+0000 7fc9d2d9d700 1 --2- 172.21.15.16:0/2207359681 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc9d4139170 0x7fc9d4121110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:24.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.429+0000 7fc9d1d9b700 1 --2- 172.21.15.16:0/2207359681 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc9d409e1d0 0x7fc9d4120bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:24.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.429+0000 7fc9d259c700 1 -- 172.21.15.16:0/2207359681 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc9d4139170 msgr2=0x7fc9d4121110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:24.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.429+0000 7fc9d259c700 1 --2- 172.21.15.16:0/2207359681 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc9d4139170 0x7fc9d4121110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:24.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.429+0000 7fc9d259c700 1 -- 172.21.15.16:0/2207359681 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc9d409e1d0 msgr2=0x7fc9d4120bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:24.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.429+0000 7fc9d259c700 1 --2- 172.21.15.16:0/2207359681 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc9d409e1d0 0x7fc9d4120bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:24.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.429+0000 7fc9d259c700 1 -- 172.21.15.16:0/2207359681 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9c80094a0 con 0x7fc9d409d6f0 2023-09-30T22:32:24.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.429+0000 7fc9d259c700 1 --2- 172.21.15.16:0/2207359681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9d409d6f0 0x7fc9d4120690 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fc9c400c700 tx=0x7fc9c400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:24.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.430+0000 7fc9c2ffd700 1 -- 172.21.15.16:0/2207359681 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4005d70 con 0x7fc9d409d6f0 2023-09-30T22:32:24.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.430+0000 7fc9c2ffd700 1 -- 172.21.15.16:0/2207359681 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c40070d0 con 0x7fc9d409d6f0 2023-09-30T22:32:24.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.430+0000 7fc9d89a8700 1 -- 172.21.15.16:0/2207359681 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9d409c710 con 0x7fc9d409d6f0 2023-09-30T22:32:24.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.430+0000 7fc9c2ffd700 1 -- 172.21.15.16:0/2207359681 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4007460 con 0x7fc9d409d6f0 2023-09-30T22:32:24.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.430+0000 7fc9d89a8700 1 -- 172.21.15.16:0/2207359681 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9d4124940 con 0x7fc9d409d6f0 2023-09-30T22:32:24.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.431+0000 7fc9c2ffd700 1 -- 172.21.15.16:0/2207359681 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc9c40075c0 con 0x7fc9d409d6f0 2023-09-30T22:32:24.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.431+0000 7fc9c2ffd700 1 -- 172.21.15.16:0/2207359681 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fc9c403eb00 con 0x7fc9d409d6f0 2023-09-30T22:32:24.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.431+0000 7fc9d89a8700 1 -- 172.21.15.16:0/2207359681 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9d4123b40 con 0x7fc9d409d6f0 2023-09-30T22:32:24.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.436+0000 7fc9c2ffd700 1 -- 172.21.15.16:0/2207359681 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc9c4017020 con 0x7fc9d409d6f0 2023-09-30T22:32:24.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.528+0000 7f6000303700 1 -- 172.21.15.16:0/908611250 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5ff81304e0 con 0x7f5ff809e410 2023-09-30T22:32:24.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.529+0000 7f5feeffd700 1 -- 172.21.15.16:0/908611250 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5ff0015070 con 0x7f5ff809e410 2023-09-30T22:32:24.531 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:32:24.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.532+0000 7f6000303700 1 -- 172.21.15.16:0/908611250 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5ff809e410 msgr2=0x7f5ff8123040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:24.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.532+0000 7f6000303700 1 --2- 172.21.15.16:0/908611250 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5ff809e410 0x7f5ff8123040 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0009840 tx=0x7f5ff0004750 comp rx=0 tx=0).stop 2023-09-30T22:32:24.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.532+0000 7f6000303700 1 -- 172.21.15.16:0/908611250 shutdown_connections 2023-09-30T22:32:24.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.532+0000 7f6000303700 1 --2- 172.21.15.16:0/908611250 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5ff809e410 0x7f5ff8123040 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:24.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.532+0000 7f6000303700 1 --2- 172.21.15.16:0/908611250 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5ff809d6f0 0x7f5ff81225c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:24.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.532+0000 7f6000303700 1 --2- 172.21.15.16:0/908611250 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5ff809e000 0x7f5ff8122b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:24.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.532+0000 7f6000303700 1 -- 172.21.15.16:0/908611250 >> 172.21.15.16:0/908611250 conn(0x7f5ff8066f50 msgr2=0x7f5ff8070ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:24.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.533+0000 7f6000303700 1 -- 172.21.15.16:0/908611250 shutdown_connections 2023-09-30T22:32:24.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.533+0000 7f6000303700 1 -- 172.21.15.16:0/908611250 wait complete. 2023-09-30T22:32:24.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.549+0000 7fb6170ab700 1 -- 172.21.15.16:0/1902160406 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb61006b9e0 con 0x7fb61009e560 2023-09-30T22:32:24.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.549+0000 7fb60d7fa700 1 -- 172.21.15.16:0/1902160406 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb604015020 con 0x7fb61009e560 2023-09-30T22:32:24.551 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:32:24.552 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:32:24.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.552+0000 7fb6170ab700 1 -- 172.21.15.16:0/1902160406 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb61009e560 msgr2=0x7fb61011b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:24.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.552+0000 7fb6170ab700 1 --2- 172.21.15.16:0/1902160406 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb61009e560 0x7fb61011b7e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb6040090b0 tx=0x7fb604004750 comp rx=0 tx=0).stop 2023-09-30T22:32:24.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.553+0000 7fb6170ab700 1 -- 172.21.15.16:0/1902160406 shutdown_connections 2023-09-30T22:32:24.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.553+0000 7fb6170ab700 1 --2- 172.21.15.16:0/1902160406 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb61009e560 0x7fb61011b7e0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:24.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.553+0000 7fb6170ab700 1 --2- 172.21.15.16:0/1902160406 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb6101341d0 0x7fb61011c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:24.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.553+0000 7fb6170ab700 1 --2- 172.21.15.16:0/1902160406 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6101338c0 0x7fb61011bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:24.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.553+0000 7fb6170ab700 1 -- 172.21.15.16:0/1902160406 >> 172.21.15.16:0/1902160406 conn(0x7fb6100a2aa0 msgr2=0x7fb61009f180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:24.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.553+0000 7fb6170ab700 1 -- 172.21.15.16:0/1902160406 shutdown_connections 2023-09-30T22:32:24.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.553+0000 7fb6170ab700 1 -- 172.21.15.16:0/1902160406 wait complete. 2023-09-30T22:32:24.573 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:32:24.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.602+0000 7fad7f760700 1 -- 172.21.15.16:0/1429177776 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fad78132240 con 0x7fad7809f240 2023-09-30T22:32:24.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.602+0000 7fad6dffb700 1 -- 172.21.15.16:0/1429177776 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fad68017030 con 0x7fad7809f240 2023-09-30T22:32:24.604 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:32:24.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.605+0000 7fad7f760700 1 -- 172.21.15.16:0/1429177776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad7809f240 msgr2=0x7fad78131d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:24.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.605+0000 7fad7f760700 1 --2- 172.21.15.16:0/1429177776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad7809f240 0x7fad78131d00 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fad6800ebb0 tx=0x7fad6800eec0 comp rx=0 tx=0).stop 2023-09-30T22:32:24.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.605+0000 7fad7f760700 1 -- 172.21.15.16:0/1429177776 shutdown_connections 2023-09-30T22:32:24.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.605+0000 7fad7f760700 1 --2- 172.21.15.16:0/1429177776 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fad7809e930 0x7fad781317c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:24.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.605+0000 7fad7f760700 1 --2- 172.21.15.16:0/1429177776 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fad7809f650 0x7fad781c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:24.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.605+0000 7fad7f760700 1 --2- 172.21.15.16:0/1429177776 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad7809f240 0x7fad78131d00 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:24.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.605+0000 7fad7f760700 1 -- 172.21.15.16:0/1429177776 >> 172.21.15.16:0/1429177776 conn(0x7fad780a4210 msgr2=0x7fad7806c460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:24.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.605+0000 7fad7f760700 1 -- 172.21.15.16:0/1429177776 shutdown_connections 2023-09-30T22:32:24.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.605+0000 7fad7f760700 1 -- 172.21.15.16:0/1429177776 wait complete. 2023-09-30T22:32:24.620 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:32:24.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.655+0000 7fc9d89a8700 1 -- 172.21.15.16:0/2207359681 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc9d409e5a0 con 0x7fc9d409d6f0 2023-09-30T22:32:24.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.656+0000 7fc9c2ffd700 1 -- 172.21.15.16:0/2207359681 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc9c4018260 con 0x7fc9d409d6f0 2023-09-30T22:32:24.657 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:32:24.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.658+0000 7fc9d89a8700 1 -- 172.21.15.16:0/2207359681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9d409d6f0 msgr2=0x7fc9d4120690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:24.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.658+0000 7fc9d89a8700 1 --2- 172.21.15.16:0/2207359681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9d409d6f0 0x7fc9d4120690 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fc9c400c700 tx=0x7fc9c400ca10 comp rx=0 tx=0).stop 2023-09-30T22:32:24.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.658+0000 7fc9d89a8700 1 -- 172.21.15.16:0/2207359681 shutdown_connections 2023-09-30T22:32:24.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.658+0000 7fc9d89a8700 1 --2- 172.21.15.16:0/2207359681 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc9d4139170 0x7fc9d4121110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:24.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.658+0000 7fc9d89a8700 1 --2- 172.21.15.16:0/2207359681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9d409d6f0 0x7fc9d4120690 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:24.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.658+0000 7fc9d89a8700 1 --2- 172.21.15.16:0/2207359681 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc9d409e1d0 0x7fc9d4120bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:24.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.658+0000 7fc9d89a8700 1 -- 172.21.15.16:0/2207359681 >> 172.21.15.16:0/2207359681 conn(0x7fc9d4066f50 msgr2=0x7fc9d4136a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:24.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.658+0000 7fc9d89a8700 1 -- 172.21.15.16:0/2207359681 shutdown_connections 2023-09-30T22:32:24.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:24.658+0000 7fc9d89a8700 1 -- 172.21.15.16:0/2207359681 wait complete. 2023-09-30T22:32:24.673 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:32:25.554 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:32:25.574 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:32:25.621 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:32:25.673 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:32:25.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.710+0000 7f7244c7a700 1 Processor -- start 2023-09-30T22:32:25.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.710+0000 7f7244c7a700 1 -- start start 2023-09-30T22:32:25.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.711+0000 7f7244c7a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7240070180 0x7f7240070550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:25.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.711+0000 7f7244c7a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7240070a90 0x7f724012cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:25.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.711+0000 7f7244c7a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f724009a790 con 0x7f724012d2e0 2023-09-30T22:32:25.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.711+0000 7f7244c7a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f724009b450 con 0x7f7240070a90 2023-09-30T22:32:25.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.711+0000 7f7244c7a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7240099f90 con 0x7f7240070180 2023-09-30T22:32:25.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.711+0000 7f723dd9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7240070a90 0x7f724012cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:25.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.711+0000 7f723ed9d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f724012d2e0 0x7f7240137770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53502/0 (socket says 172.21.15.16:53502) 2023-09-30T22:32:25.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.711+0000 7f723ed9d700 1 -- 172.21.15.16:0/2218030209 learned_addr learned my addr 172.21.15.16:0/2218030209 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:25.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.711+0000 7f723dd9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7240070a90 0x7f724012cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:59878/0 (socket says 172.21.15.16:59878) 2023-09-30T22:32:25.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.711+0000 7f723e59c700 1 --2- 172.21.15.16:0/2218030209 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7240070180 0x7f7240070550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:25.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.711+0000 7f723dd9b700 1 -- 172.21.15.16:0/2218030209 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7240070180 msgr2=0x7f7240070550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:25.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.711+0000 7f723dd9b700 1 --2- 172.21.15.16:0/2218030209 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7240070180 0x7f7240070550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:25.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.711+0000 7f723dd9b700 1 -- 172.21.15.16:0/2218030209 >> v1:172.21.15.16:6789/0 conn(0x7f724012d2e0 legacy=0x7f7240137770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:25.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.711+0000 7f723dd9b700 1 -- 172.21.15.16:0/2218030209 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7240072120 con 0x7f7240070a90 2023-09-30T22:32:25.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.712+0000 7f723dd9b700 1 --2- 172.21.15.16:0/2218030209 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7240070a90 0x7f724012cda0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7234009870 tx=0x7f7234009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7eb08fb8f89700ac server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:25.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.712+0000 7f723d59a700 1 -- 172.21.15.16:0/2218030209 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7234011830 con 0x7f7240070a90 2023-09-30T22:32:25.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.712+0000 7f723d59a700 1 -- 172.21.15.16:0/2218030209 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7234011990 con 0x7f7240070a90 2023-09-30T22:32:25.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.712+0000 7f723d59a700 1 -- 172.21.15.16:0/2218030209 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7234011d20 con 0x7f7240070a90 2023-09-30T22:32:25.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.713+0000 7f7244c7a700 1 -- 172.21.15.16:0/2218030209 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7240070a90 msgr2=0x7f724012cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:25.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.713+0000 7f7244c7a700 1 --2- 172.21.15.16:0/2218030209 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7240070a90 0x7f724012cda0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f7234009870 tx=0x7f7234009b80 comp rx=0 tx=0).stop 2023-09-30T22:32:25.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.713+0000 7f7244c7a700 1 -- 172.21.15.16:0/2218030209 shutdown_connections 2023-09-30T22:32:25.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.713+0000 7f7244c7a700 1 --2- 172.21.15.16:0/2218030209 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7240070180 0x7f7240070550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:25.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.713+0000 7f7244c7a700 1 --2- 172.21.15.16:0/2218030209 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7240070a90 0x7f724012cda0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:25.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.713+0000 7f7244c7a700 1 -- 172.21.15.16:0/2218030209 >> 172.21.15.16:0/2218030209 conn(0x7f72400a27f0 msgr2=0x7f72400a2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:25.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.713+0000 7f7244c7a700 1 -- 172.21.15.16:0/2218030209 shutdown_connections 2023-09-30T22:32:25.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.713+0000 7f7244c7a700 1 -- 172.21.15.16:0/2218030209 wait complete. 2023-09-30T22:32:25.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.714+0000 7f7244c7a700 1 Processor -- start 2023-09-30T22:32:25.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.714+0000 7f7244c7a700 1 -- start start 2023-09-30T22:32:25.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.715+0000 7f7244c7a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7240070180 0x7f724011a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:25.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.715+0000 7f7244c7a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7240070a90 0x7f724011ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:25.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.715+0000 7f7244c7a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f724012d2e0 0x7f724011b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:25.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.715+0000 7f7244c7a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f724011df70 con 0x7f7240070180 2023-09-30T22:32:25.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.715+0000 7f7244c7a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f724011c770 con 0x7f724012d2e0 2023-09-30T22:32:25.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.715+0000 7f7244c7a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f724011e1c0 con 0x7f7240070a90 2023-09-30T22:32:25.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.715+0000 7f723ed9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f724012d2e0 0x7f724011b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:25.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.715+0000 7f723ed9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f724012d2e0 0x7f724011b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:59888/0 (socket says 172.21.15.16:59888) 2023-09-30T22:32:25.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.715+0000 7f723ed9d700 1 -- 172.21.15.16:0/1608600316 learned_addr learned my addr 172.21.15.16:0/1608600316 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:25.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.715+0000 7f723e59c700 1 --2- 172.21.15.16:0/1608600316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7240070180 0x7f724011a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:25.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.716+0000 7f723ed9d700 1 -- 172.21.15.16:0/1608600316 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7240070a90 msgr2=0x7f724011ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:25.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.716+0000 7f723ed9d700 1 --2- 172.21.15.16:0/1608600316 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7240070a90 0x7f724011ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:25.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.716+0000 7f723ed9d700 1 -- 172.21.15.16:0/1608600316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7240070180 msgr2=0x7f724011a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:25.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.716+0000 7f723ed9d700 1 --2- 172.21.15.16:0/1608600316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7240070180 0x7f724011a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:25.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.716+0000 7f723ed9d700 1 -- 172.21.15.16:0/1608600316 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72340094a0 con 0x7f724012d2e0 2023-09-30T22:32:25.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.716+0000 7f723ed9d700 1 --2- 172.21.15.16:0/1608600316 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f724012d2e0 0x7f724011b2c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f723800cbc0 tx=0x7f723800cf80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:25.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.716+0000 7f722effd700 1 -- 172.21.15.16:0/1608600316 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f723800ab50 con 0x7f724012d2e0 2023-09-30T22:32:25.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.716+0000 7f722effd700 1 -- 172.21.15.16:0/1608600316 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f723800acb0 con 0x7f724012d2e0 2023-09-30T22:32:25.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.716+0000 7f7244c7a700 1 -- 172.21.15.16:0/1608600316 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f724011d0b0 con 0x7f724012d2e0 2023-09-30T22:32:25.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.716+0000 7f7244c7a700 1 -- 172.21.15.16:0/1608600316 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f724012cb40 con 0x7f724012d2e0 2023-09-30T22:32:25.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.716+0000 7f722effd700 1 -- 172.21.15.16:0/1608600316 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7238012600 con 0x7f724012d2e0 2023-09-30T22:32:25.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.718+0000 7f7244c7a700 1 -- 172.21.15.16:0/1608600316 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72240052e0 con 0x7f724012d2e0 2023-09-30T22:32:25.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.718+0000 7f722effd700 1 -- 172.21.15.16:0/1608600316 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7238012760 con 0x7f724012d2e0 2023-09-30T22:32:25.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.718+0000 7f722effd700 1 -- 172.21.15.16:0/1608600316 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f723803f3f0 con 0x7f724012d2e0 2023-09-30T22:32:25.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.723+0000 7f722effd700 1 -- 172.21.15.16:0/1608600316 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f723801b3b0 con 0x7f724012d2e0 2023-09-30T22:32:25.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.752+0000 7f70973e9700 1 Processor -- start 2023-09-30T22:32:25.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.752+0000 7f70973e9700 1 -- start start 2023-09-30T22:32:25.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.753+0000 7f70973e9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f709009d6f0 0x7f709009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:25.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.753+0000 7f70973e9700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f709009e000 0x7f70901339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:25.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.753+0000 7f70973e9700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f709009b120 con 0x7f709009e410 2023-09-30T22:32:25.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.753+0000 7f70973e9700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f709006b460 con 0x7f709009e000 2023-09-30T22:32:25.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.753+0000 7f70973e9700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f709009a7e0 con 0x7f709009d6f0 2023-09-30T22:32:25.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.753+0000 7f7095185700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f709009d6f0 0x7f709009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:25.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.753+0000 7f7095986700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f709009e410 0x7f7090133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53504/0 (socket says 172.21.15.16:53504) 2023-09-30T22:32:25.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.753+0000 7f7095986700 1 -- 172.21.15.16:0/577287552 learned_addr learned my addr 172.21.15.16:0/577287552 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:25.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.753+0000 7f7094984700 1 --2- 172.21.15.16:0/577287552 >> v2:172.21.15.16:3300/0 conn(0x7f709009e000 0x7f70901339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:25.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.754+0000 7f7095185700 1 -- 172.21.15.16:0/577287552 >> v2:172.21.15.16:3300/0 conn(0x7f709009e000 msgr2=0x7f70901339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:25.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.754+0000 7f7095185700 1 --2- 172.21.15.16:0/577287552 >> v2:172.21.15.16:3300/0 conn(0x7f709009e000 0x7f70901339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:25.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.754+0000 7f7095185700 1 -- 172.21.15.16:0/577287552 >> v1:172.21.15.16:6789/0 conn(0x7f709009e410 legacy=0x7f7090133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:25.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.754+0000 7f7095185700 1 -- 172.21.15.16:0/577287552 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f709009a500 con 0x7f709009d6f0 2023-09-30T22:32:25.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.754+0000 7f7095185700 1 --2- 172.21.15.16:0/577287552 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f709009d6f0 0x7f709009dac0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f708c009870 tx=0x7f708c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4156d26050fcb7d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:25.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.754+0000 7f7087fff700 1 -- 172.21.15.16:0/577287552 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708c011830 con 0x7f709009d6f0 2023-09-30T22:32:25.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.754+0000 7f7087fff700 1 -- 172.21.15.16:0/577287552 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f708c011990 con 0x7f709009d6f0 2023-09-30T22:32:25.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.754+0000 7f7087fff700 1 -- 172.21.15.16:0/577287552 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708c011d20 con 0x7f709009d6f0 2023-09-30T22:32:25.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.755+0000 7f70973e9700 1 -- 172.21.15.16:0/577287552 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f709009d6f0 msgr2=0x7f709009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:25.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.755+0000 7f70973e9700 1 --2- 172.21.15.16:0/577287552 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f709009d6f0 0x7f709009dac0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f708c009870 tx=0x7f708c009b80 comp rx=0 tx=0).stop 2023-09-30T22:32:25.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.756+0000 7f70973e9700 1 -- 172.21.15.16:0/577287552 shutdown_connections 2023-09-30T22:32:25.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.756+0000 7f70973e9700 1 --2- 172.21.15.16:0/577287552 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f709009d6f0 0x7f709009dac0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:25.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.756+0000 7f70973e9700 1 --2- 172.21.15.16:0/577287552 >> v2:172.21.15.16:3300/0 conn(0x7f709009e000 0x7f70901339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:25.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.756+0000 7f70973e9700 1 -- 172.21.15.16:0/577287552 >> 172.21.15.16:0/577287552 conn(0x7f7090066f50 msgr2=0x7f709009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:25.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.760+0000 7f70973e9700 1 -- 172.21.15.16:0/577287552 shutdown_connections 2023-09-30T22:32:25.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.760+0000 7f70973e9700 1 -- 172.21.15.16:0/577287552 wait complete. 2023-09-30T22:32:25.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.761+0000 7f70973e9700 1 Processor -- start 2023-09-30T22:32:25.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.761+0000 7f70973e9700 1 -- start start 2023-09-30T22:32:25.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.762+0000 7f70973e9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f709009d6f0 0x7f70901c8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:25.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.762+0000 7f70973e9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f709009e000 0x7f70901c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:25.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.762+0000 7f70973e9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f709009e410 0x7f70901c4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:25.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.762+0000 7f70973e9700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70901aaa90 con 0x7f709009d6f0 2023-09-30T22:32:25.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.762+0000 7f70973e9700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70900a30c0 con 0x7f709009e410 2023-09-30T22:32:25.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.762+0000 7f70973e9700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f709009c710 con 0x7f709009e000 2023-09-30T22:32:25.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.762+0000 7f7095986700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f709009e410 0x7f70901c4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:25.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.762+0000 7f7095986700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f709009e410 0x7f70901c4a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:59908/0 (socket says 172.21.15.16:59908) 2023-09-30T22:32:25.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.762+0000 7f7095986700 1 -- 172.21.15.16:0/3733844322 learned_addr learned my addr 172.21.15.16:0/3733844322 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:25.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.762+0000 7f7094984700 1 --2- 172.21.15.16:0/3733844322 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f709009e000 0x7f70901c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:25.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.762+0000 7f7095185700 1 --2- 172.21.15.16:0/3733844322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f709009d6f0 0x7f70901c8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:25.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.763+0000 7f7095986700 1 -- 172.21.15.16:0/3733844322 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f709009e000 msgr2=0x7f70901c8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:25.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.763+0000 7f7095986700 1 --2- 172.21.15.16:0/3733844322 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f709009e000 0x7f70901c8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:25.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.763+0000 7f7095986700 1 -- 172.21.15.16:0/3733844322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f709009d6f0 msgr2=0x7f70901c8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:25.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.763+0000 7f7095986700 1 --2- 172.21.15.16:0/3733844322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f709009d6f0 0x7f70901c8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:25.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.763+0000 7f7095986700 1 -- 172.21.15.16:0/3733844322 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f708c0094a0 con 0x7f709009e410 2023-09-30T22:32:25.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.763+0000 7f7095986700 1 --2- 172.21.15.16:0/3733844322 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f709009e410 0x7f70901c4a50 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f708800cbc0 tx=0x7f708800ced0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:25.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.763+0000 7f7085ffb700 1 -- 172.21.15.16:0/3733844322 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708800a9e0 con 0x7f709009e410 2023-09-30T22:32:25.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.763+0000 7f7085ffb700 1 -- 172.21.15.16:0/3733844322 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f708800ab40 con 0x7f709009e410 2023-09-30T22:32:25.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.763+0000 7f70973e9700 1 -- 172.21.15.16:0/3733844322 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70901a9c70 con 0x7f709009e410 2023-09-30T22:32:25.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.763+0000 7f7085ffb700 1 -- 172.21.15.16:0/3733844322 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7088012400 con 0x7f709009e410 2023-09-30T22:32:25.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.764+0000 7f70973e9700 1 -- 172.21.15.16:0/3733844322 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70901ac240 con 0x7f709009e410 2023-09-30T22:32:25.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.765+0000 7f7085ffb700 1 -- 172.21.15.16:0/3733844322 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f708801b3b0 con 0x7f709009e410 2023-09-30T22:32:25.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.765+0000 7f7085ffb700 1 -- 172.21.15.16:0/3733844322 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f708803f3c0 con 0x7f709009e410 2023-09-30T22:32:25.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.765+0000 7f70973e9700 1 -- 172.21.15.16:0/3733844322 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70901aa330 con 0x7f709009e410 2023-09-30T22:32:25.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.774+0000 7f7085ffb700 1 -- 172.21.15.16:0/3733844322 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f70901aa330 con 0x7f709009e410 2023-09-30T22:32:25.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.791+0000 7f32bf7ce700 1 Processor -- start 2023-09-30T22:32:25.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.792+0000 7f32bf7ce700 1 -- start start 2023-09-30T22:32:25.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.792+0000 7f32bf7ce700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f32b809f410 0x7f32b8136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:25.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.792+0000 7f32bf7ce700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f32b8136fe0 0x7f32b81393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:25.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.792+0000 7f32bf7ce700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32b8070940 con 0x7f32b809e930 2023-09-30T22:32:25.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.792+0000 7f32bf7ce700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32b806fd60 con 0x7f32b809f410 2023-09-30T22:32:25.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.792+0000 7f32bf7ce700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f32b806fa80 con 0x7f32b8136fe0 2023-09-30T22:32:25.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.793+0000 7f32bcd69700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f32b809f410 0x7f32b8136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:25.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.793+0000 7f32bcd69700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f32b809f410 0x7f32b8136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34924/0 (socket says 172.21.15.16:34924) 2023-09-30T22:32:25.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.793+0000 7f32bd56a700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f32b809e930 0x7f32b809ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53510/0 (socket says 172.21.15.16:53510) 2023-09-30T22:32:25.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.793+0000 7f32bcd69700 1 -- 172.21.15.16:0/2030794048 learned_addr learned my addr 172.21.15.16:0/2030794048 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:25.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.793+0000 7f32bdd6b700 1 --2- 172.21.15.16:0/2030794048 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f32b8136fe0 0x7f32b81393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:25.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.793+0000 7f32bdd6b700 1 -- 172.21.15.16:0/2030794048 >> v2:172.21.15.16:3300/0 conn(0x7f32b809f410 msgr2=0x7f32b8136aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:25.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.793+0000 7f32bdd6b700 1 --2- 172.21.15.16:0/2030794048 >> v2:172.21.15.16:3300/0 conn(0x7f32b809f410 0x7f32b8136aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:25.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.793+0000 7f32bdd6b700 1 -- 172.21.15.16:0/2030794048 >> v1:172.21.15.16:6789/0 conn(0x7f32b809e930 legacy=0x7f32b809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:25.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.793+0000 7f32bdd6b700 1 -- 172.21.15.16:0/2030794048 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b809fb60 con 0x7f32b8136fe0 2023-09-30T22:32:25.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.793+0000 7f32bdd6b700 1 --2- 172.21.15.16:0/2030794048 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f32b8136fe0 0x7f32b81393c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f32b0009d90 tx=0x7f32b000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de1f36ebae24188d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:25.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.794+0000 7f32affff700 1 -- 172.21.15.16:0/2030794048 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b000e070 con 0x7f32b8136fe0 2023-09-30T22:32:25.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.794+0000 7f32affff700 1 -- 172.21.15.16:0/2030794048 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32b000bb20 con 0x7f32b8136fe0 2023-09-30T22:32:25.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.794+0000 7f32affff700 1 -- 172.21.15.16:0/2030794048 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b0012430 con 0x7f32b8136fe0 2023-09-30T22:32:25.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.795+0000 7f32bf7ce700 1 -- 172.21.15.16:0/2030794048 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f32b8136fe0 msgr2=0x7f32b81393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:25.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.795+0000 7f32bf7ce700 1 --2- 172.21.15.16:0/2030794048 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f32b8136fe0 0x7f32b81393c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f32b0009d90 tx=0x7f32b000b330 comp rx=0 tx=0).stop 2023-09-30T22:32:25.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.795+0000 7f32bf7ce700 1 -- 172.21.15.16:0/2030794048 shutdown_connections 2023-09-30T22:32:25.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.795+0000 7f32bf7ce700 1 --2- 172.21.15.16:0/2030794048 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f32b8136fe0 0x7f32b81393c0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:25.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.795+0000 7f32bf7ce700 1 --2- 172.21.15.16:0/2030794048 >> v2:172.21.15.16:3300/0 conn(0x7f32b809f410 0x7f32b8136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:25.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.795+0000 7f32bf7ce700 1 -- 172.21.15.16:0/2030794048 >> 172.21.15.16:0/2030794048 conn(0x7f32b80a4210 msgr2=0x7f32b80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:25.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.795+0000 7f32bf7ce700 1 -- 172.21.15.16:0/2030794048 shutdown_connections 2023-09-30T22:32:25.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.795+0000 7f32bf7ce700 1 -- 172.21.15.16:0/2030794048 wait complete. 2023-09-30T22:32:25.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.796+0000 7f32bf7ce700 1 Processor -- start 2023-09-30T22:32:25.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.797+0000 7f32bf7ce700 1 -- start start 2023-09-30T22:32:25.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.797+0000 7f32bf7ce700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f32b809e930 0x7f32b811c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:25.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.797+0000 7f32bf7ce700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f32b809f410 0x7f32b811cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:25.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.797+0000 7f32bf7ce700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32b8136fe0 0x7f32b811d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:25.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.797+0000 7f32bf7ce700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b811e9c0 con 0x7f32b8136fe0 2023-09-30T22:32:25.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.797+0000 7f32bf7ce700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32b811e370 con 0x7f32b809f410 2023-09-30T22:32:25.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.797+0000 7f32bf7ce700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f32b81204a0 con 0x7f32b809e930 2023-09-30T22:32:25.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.797+0000 7f32bd56a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f32b809e930 0x7f32b811c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:25.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.798+0000 7f32bd56a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f32b809e930 0x7f32b811c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:36358/0 (socket says 172.21.15.16:36358) 2023-09-30T22:32:25.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.798+0000 7f32bd56a700 1 -- 172.21.15.16:0/2771617100 learned_addr learned my addr 172.21.15.16:0/2771617100 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:25.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.798+0000 7f32bdd6b700 1 --2- 172.21.15.16:0/2771617100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32b8136fe0 0x7f32b811d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:25.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.798+0000 7f32bcd69700 1 --2- 172.21.15.16:0/2771617100 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f32b809f410 0x7f32b811cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:25.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.798+0000 7f32bd56a700 1 -- 172.21.15.16:0/2771617100 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f32b809f410 msgr2=0x7f32b811cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:25.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.798+0000 7f32bd56a700 1 --2- 172.21.15.16:0/2771617100 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f32b809f410 0x7f32b811cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:25.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.798+0000 7f32bd56a700 1 -- 172.21.15.16:0/2771617100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32b8136fe0 msgr2=0x7f32b811d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:25.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.798+0000 7f32bd56a700 1 --2- 172.21.15.16:0/2771617100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32b8136fe0 0x7f32b811d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:25.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.798+0000 7f32bd56a700 1 -- 172.21.15.16:0/2771617100 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b0009a70 con 0x7f32b809e930 2023-09-30T22:32:25.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.798+0000 7f32bd56a700 1 --2- 172.21.15.16:0/2771617100 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f32b809e930 0x7f32b811c760 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f32b400c550 tx=0x7f32b400c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:25.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.798+0000 7f32adffb700 1 -- 172.21.15.16:0/2771617100 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b4005cb0 con 0x7f32b809e930 2023-09-30T22:32:25.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.798+0000 7f32adffb700 1 -- 172.21.15.16:0/2771617100 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32b4005e10 con 0x7f32b809e930 2023-09-30T22:32:25.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.798+0000 7f32bf7ce700 1 -- 172.21.15.16:0/2771617100 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32b809df30 con 0x7f32b809e930 2023-09-30T22:32:25.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.798+0000 7f32adffb700 1 -- 172.21.15.16:0/2771617100 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b40072d0 con 0x7f32b809e930 2023-09-30T22:32:25.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.799+0000 7f32bf7ce700 1 -- 172.21.15.16:0/2771617100 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32b8071d70 con 0x7f32b809e930 2023-09-30T22:32:25.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.800+0000 7f32adffb700 1 -- 172.21.15.16:0/2771617100 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f32b4007430 con 0x7f32b809e930 2023-09-30T22:32:25.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.800+0000 7f32adffb700 1 -- 172.21.15.16:0/2771617100 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f32b4040b30 con 0x7f32b809e930 2023-09-30T22:32:25.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.800+0000 7f32bf7ce700 1 -- 172.21.15.16:0/2771617100 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32b8074ea0 con 0x7f32b809e930 2023-09-30T22:32:25.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.809+0000 7f32adffb700 1 -- 172.21.15.16:0/2771617100 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f32b4044060 con 0x7f32b809e930 2023-09-30T22:32:25.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.852+0000 7f5b82ec3700 1 Processor -- start 2023-09-30T22:32:25.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.852+0000 7f5b82ec3700 1 -- start start 2023-09-30T22:32:25.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.853+0000 7f5b82ec3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5b7c0a0a30 0x7f5b7c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:25.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.853+0000 7f5b82ec3700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5b7c09d630 0x7f5b7c0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:25.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.853+0000 7f5b82ec3700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b7c0a2a70 con 0x7f5b7c133930 2023-09-30T22:32:25.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.853+0000 7f5b82ec3700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b7c0a01d0 con 0x7f5b7c09d630 2023-09-30T22:32:25.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.853+0000 7f5b82ec3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7c09e2b0 con 0x7f5b7c0a0a30 2023-09-30T22:32:25.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.853+0000 7f5b80c5f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5b7c0a0a30 0x7f5b7c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:25.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.853+0000 7f5b80c5f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5b7c0a0a30 0x7f5b7c0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:36368/0 (socket says 172.21.15.16:36368) 2023-09-30T22:32:25.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.853+0000 7f5b80c5f700 1 -- 172.21.15.16:0/490843926 learned_addr learned my addr 172.21.15.16:0/490843926 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:25.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.853+0000 7f5b80c5f700 1 -- 172.21.15.16:0/490843926 >> v2:172.21.15.16:3300/0 conn(0x7f5b7c09d630 msgr2=0x7f5b7c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:25.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.853+0000 7f5b80c5f700 1 --2- 172.21.15.16:0/490843926 >> v2:172.21.15.16:3300/0 conn(0x7f5b7c09d630 0x7f5b7c0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:25.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.853+0000 7f5b80c5f700 1 -- 172.21.15.16:0/490843926 >> v1:172.21.15.16:6789/0 conn(0x7f5b7c133930 legacy=0x7f5b7c133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:25.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.853+0000 7f5b80c5f700 1 -- 172.21.15.16:0/490843926 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b7c09dc50 con 0x7f5b7c0a0a30 2023-09-30T22:32:25.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.853+0000 7f5b80c5f700 1 --2- 172.21.15.16:0/490843926 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5b7c0a0a30 0x7f5b7c0a0e00 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f5b700098f0 tx=0x7f5b70009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acc28d88feacc300 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:25.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.854+0000 7f5b7b7fe700 1 -- 172.21.15.16:0/490843926 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b70004230 con 0x7f5b7c0a0a30 2023-09-30T22:32:25.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.854+0000 7f5b7b7fe700 1 -- 172.21.15.16:0/490843926 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b70004390 con 0x7f5b7c0a0a30 2023-09-30T22:32:25.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.854+0000 7f5b7b7fe700 1 -- 172.21.15.16:0/490843926 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b700036a0 con 0x7f5b7c0a0a30 2023-09-30T22:32:25.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.854+0000 7f5b82ec3700 1 -- 172.21.15.16:0/490843926 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5b7c0a0a30 msgr2=0x7f5b7c0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:25.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.854+0000 7f5b82ec3700 1 --2- 172.21.15.16:0/490843926 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5b7c0a0a30 0x7f5b7c0a0e00 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f5b700098f0 tx=0x7f5b70009c00 comp rx=0 tx=0).stop 2023-09-30T22:32:25.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.854+0000 7f5b82ec3700 1 -- 172.21.15.16:0/490843926 shutdown_connections 2023-09-30T22:32:25.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.854+0000 7f5b82ec3700 1 --2- 172.21.15.16:0/490843926 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5b7c0a0a30 0x7f5b7c0a0e00 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:25.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.854+0000 7f5b82ec3700 1 --2- 172.21.15.16:0/490843926 >> v2:172.21.15.16:3300/0 conn(0x7f5b7c09d630 0x7f5b7c0a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:25.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.854+0000 7f5b82ec3700 1 -- 172.21.15.16:0/490843926 >> 172.21.15.16:0/490843926 conn(0x7f5b7c0a3050 msgr2=0x7f5b7c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:25.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.854+0000 7f5b82ec3700 1 -- 172.21.15.16:0/490843926 shutdown_connections 2023-09-30T22:32:25.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.854+0000 7f5b82ec3700 1 -- 172.21.15.16:0/490843926 wait complete. 2023-09-30T22:32:25.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.856+0000 7f5b82ec3700 1 Processor -- start 2023-09-30T22:32:25.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.856+0000 7f5b82ec3700 1 -- start start 2023-09-30T22:32:25.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.857+0000 7f5b82ec3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5b7c09d630 0x7f5b7c1c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:25.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.857+0000 7f5b82ec3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b7c0a0a30 0x7f5b7c1c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:25.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.857+0000 7f5b82ec3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5b7c133930 0x7f5b7c1c8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:25.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.857+0000 7f5b82ec3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7c1208e0 con 0x7f5b7c0a0a30 2023-09-30T22:32:25.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.857+0000 7f5b82ec3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7c1223c0 con 0x7f5b7c09d630 2023-09-30T22:32:25.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.857+0000 7f5b80c5f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5b7c09d630 0x7f5b7c1c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:25.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.857+0000 7f5b82ec3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7c1206b0 con 0x7f5b7c133930 2023-09-30T22:32:25.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.857+0000 7f5b80c5f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5b7c09d630 0x7f5b7c1c81a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:59928/0 (socket says 172.21.15.16:59928) 2023-09-30T22:32:25.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.857+0000 7f5b80c5f700 1 -- 172.21.15.16:0/733353654 learned_addr learned my addr 172.21.15.16:0/733353654 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:25.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.857+0000 7f5b7bfff700 1 --2- 172.21.15.16:0/733353654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b7c0a0a30 0x7f5b7c1c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:25.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.857+0000 7f5b81460700 1 --2- 172.21.15.16:0/733353654 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5b7c133930 0x7f5b7c1c8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:25.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.857+0000 7f5b7bfff700 1 -- 172.21.15.16:0/733353654 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5b7c133930 msgr2=0x7f5b7c1c8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:25.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.857+0000 7f5b7bfff700 1 --2- 172.21.15.16:0/733353654 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5b7c133930 0x7f5b7c1c8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:25.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.857+0000 7f5b7bfff700 1 -- 172.21.15.16:0/733353654 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5b7c09d630 msgr2=0x7f5b7c1c81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:25.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.857+0000 7f5b7bfff700 1 --2- 172.21.15.16:0/733353654 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5b7c09d630 0x7f5b7c1c81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:25.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.857+0000 7f5b7bfff700 1 -- 172.21.15.16:0/733353654 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b70009550 con 0x7f5b7c0a0a30 2023-09-30T22:32:25.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.857+0000 7f5b80c5f700 1 --2- 172.21.15.16:0/733353654 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5b7c09d630 0x7f5b7c1c81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:32:25.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.857+0000 7f5b7bfff700 1 --2- 172.21.15.16:0/733353654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b7c0a0a30 0x7f5b7c1c86e0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c06d980 tx=0x7f5b6800b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:25.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.858+0000 7f5b797fa700 1 -- 172.21.15.16:0/733353654 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b68011830 con 0x7f5b7c0a0a30 2023-09-30T22:32:25.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.858+0000 7f5b82ec3700 1 -- 172.21.15.16:0/733353654 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b7c0a4330 con 0x7f5b7c0a0a30 2023-09-30T22:32:25.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.858+0000 7f5b82ec3700 1 -- 172.21.15.16:0/733353654 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b7c133550 con 0x7f5b7c0a0a30 2023-09-30T22:32:25.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.858+0000 7f5b797fa700 1 -- 172.21.15.16:0/733353654 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b68011990 con 0x7f5b7c0a0a30 2023-09-30T22:32:25.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.858+0000 7f5b797fa700 1 -- 172.21.15.16:0/733353654 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b68011d40 con 0x7f5b7c0a0a30 2023-09-30T22:32:25.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.859+0000 7f5b797fa700 1 -- 172.21.15.16:0/733353654 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5b6801a400 con 0x7f5b7c0a0a30 2023-09-30T22:32:25.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.859+0000 7f5b797fa700 1 -- 172.21.15.16:0/733353654 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f5b6803f5e0 con 0x7f5b7c0a0a30 2023-09-30T22:32:25.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.859+0000 7f5b82ec3700 1 -- 172.21.15.16:0/733353654 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b5c0052e0 con 0x7f5b7c0a0a30 2023-09-30T22:32:25.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.869+0000 7f5b797fa700 1 -- 172.21.15.16:0/733353654 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5b68015020 con 0x7f5b7c0a0a30 2023-09-30T22:32:25.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.971+0000 7f7244c7a700 1 -- 172.21.15.16:0/1608600316 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7224005590 con 0x7f724012d2e0 2023-09-30T22:32:25.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.972+0000 7f722effd700 1 -- 172.21.15.16:0/1608600316 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7238007070 con 0x7f724012d2e0 2023-09-30T22:32:25.973 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:32:25.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.974+0000 7f7244c7a700 1 -- 172.21.15.16:0/1608600316 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f724012d2e0 msgr2=0x7f724011b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:25.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.974+0000 7f7244c7a700 1 --2- 172.21.15.16:0/1608600316 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f724012d2e0 0x7f724011b2c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f723800cbc0 tx=0x7f723800cf80 comp rx=0 tx=0).stop 2023-09-30T22:32:25.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.974+0000 7f7244c7a700 1 -- 172.21.15.16:0/1608600316 shutdown_connections 2023-09-30T22:32:25.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.974+0000 7f7244c7a700 1 --2- 172.21.15.16:0/1608600316 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f724012d2e0 0x7f724011b2c0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:25.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.974+0000 7f7244c7a700 1 --2- 172.21.15.16:0/1608600316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7240070180 0x7f724011a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:25.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.974+0000 7f7244c7a700 1 --2- 172.21.15.16:0/1608600316 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7240070a90 0x7f724011ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:25.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.974+0000 7f7244c7a700 1 -- 172.21.15.16:0/1608600316 >> 172.21.15.16:0/1608600316 conn(0x7f72400a27f0 msgr2=0x7f724006f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:25.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.975+0000 7f7244c7a700 1 -- 172.21.15.16:0/1608600316 shutdown_connections 2023-09-30T22:32:25.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.975+0000 7f7244c7a700 1 -- 172.21.15.16:0/1608600316 wait complete. 2023-09-30T22:32:25.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.987+0000 7f70973e9700 1 -- 172.21.15.16:0/3733844322 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f70901aa330 con 0x7f709009e410 2023-09-30T22:32:25.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.987+0000 7f7085ffb700 1 -- 172.21.15.16:0/3733844322 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f70901aa330 con 0x7f709009e410 2023-09-30T22:32:25.989 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:32:25.990 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:32:25.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.991+0000 7f70973e9700 1 -- 172.21.15.16:0/3733844322 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f709009e410 msgr2=0x7f70901c4a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:25.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.991+0000 7f70973e9700 1 --2- 172.21.15.16:0/3733844322 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f709009e410 0x7f70901c4a50 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f708800cbc0 tx=0x7f708800ced0 comp rx=0 tx=0).stop 2023-09-30T22:32:25.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.991+0000 7f70973e9700 1 -- 172.21.15.16:0/3733844322 shutdown_connections 2023-09-30T22:32:25.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.991+0000 7f70973e9700 1 --2- 172.21.15.16:0/3733844322 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f709009e410 0x7f70901c4a50 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:25.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.991+0000 7f70973e9700 1 --2- 172.21.15.16:0/3733844322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f709009d6f0 0x7f70901c8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:25.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.991+0000 7f70973e9700 1 --2- 172.21.15.16:0/3733844322 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f709009e000 0x7f70901c8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:25.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.991+0000 7f70973e9700 1 -- 172.21.15.16:0/3733844322 >> 172.21.15.16:0/3733844322 conn(0x7f7090066f50 msgr2=0x7f70900708e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:25.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.991+0000 7f70973e9700 1 -- 172.21.15.16:0/3733844322 shutdown_connections 2023-09-30T22:32:25.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:25.991+0000 7f70973e9700 1 -- 172.21.15.16:0/3733844322 wait complete. 2023-09-30T22:32:26.011 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:32:26.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:26.030+0000 7f32bf7ce700 1 -- 172.21.15.16:0/2771617100 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f32b809dd10 con 0x7f32b809e930 2023-09-30T22:32:26.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:26.030+0000 7f32adffb700 1 -- 172.21.15.16:0/2771617100 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f32b40076e0 con 0x7f32b809e930 2023-09-30T22:32:26.031 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:32:26.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:26.032+0000 7f32bf7ce700 1 -- 172.21.15.16:0/2771617100 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f32b809e930 msgr2=0x7f32b811c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:26.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:26.032+0000 7f32bf7ce700 1 --2- 172.21.15.16:0/2771617100 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f32b809e930 0x7f32b811c760 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f32b400c550 tx=0x7f32b400c860 comp rx=0 tx=0).stop 2023-09-30T22:32:26.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:26.032+0000 7f32bf7ce700 1 -- 172.21.15.16:0/2771617100 shutdown_connections 2023-09-30T22:32:26.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:26.032+0000 7f32bf7ce700 1 --2- 172.21.15.16:0/2771617100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32b8136fe0 0x7f32b811d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:26.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:26.032+0000 7f32bf7ce700 1 --2- 172.21.15.16:0/2771617100 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f32b809e930 0x7f32b811c760 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:26.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:26.032+0000 7f32bf7ce700 1 --2- 172.21.15.16:0/2771617100 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f32b809f410 0x7f32b811cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:26.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:26.032+0000 7f32bf7ce700 1 -- 172.21.15.16:0/2771617100 >> 172.21.15.16:0/2771617100 conn(0x7f32b80a4210 msgr2=0x7f32b8134ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:26.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:26.032+0000 7f32bf7ce700 1 -- 172.21.15.16:0/2771617100 shutdown_connections 2023-09-30T22:32:26.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:26.032+0000 7f32bf7ce700 1 -- 172.21.15.16:0/2771617100 wait complete. 2023-09-30T22:32:26.047 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:32:26.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:26.080+0000 7f5b82ec3700 1 -- 172.21.15.16:0/733353654 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5b5c005590 con 0x7f5b7c0a0a30 2023-09-30T22:32:26.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:26.080+0000 7f5b797fa700 1 -- 172.21.15.16:0/733353654 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5b68029090 con 0x7f5b7c0a0a30 2023-09-30T22:32:26.081 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:32:26.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:26.082+0000 7f5b82ec3700 1 -- 172.21.15.16:0/733353654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b7c0a0a30 msgr2=0x7f5b7c1c86e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:26.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:26.082+0000 7f5b82ec3700 1 --2- 172.21.15.16:0/733353654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b7c0a0a30 0x7f5b7c1c86e0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f5b7c06d980 tx=0x7f5b6800b7e0 comp rx=0 tx=0).stop 2023-09-30T22:32:26.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:26.082+0000 7f5b82ec3700 1 -- 172.21.15.16:0/733353654 shutdown_connections 2023-09-30T22:32:26.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:26.082+0000 7f5b82ec3700 1 --2- 172.21.15.16:0/733353654 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5b7c09d630 0x7f5b7c1c81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:26.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:26.082+0000 7f5b82ec3700 1 --2- 172.21.15.16:0/733353654 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5b7c133930 0x7f5b7c1c8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:26.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:26.082+0000 7f5b82ec3700 1 --2- 172.21.15.16:0/733353654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b7c0a0a30 0x7f5b7c1c86e0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:26.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:26.082+0000 7f5b82ec3700 1 -- 172.21.15.16:0/733353654 >> 172.21.15.16:0/733353654 conn(0x7f5b7c0a3050 msgr2=0x7f5b7c1bbb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:26.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:26.082+0000 7f5b82ec3700 1 -- 172.21.15.16:0/733353654 shutdown_connections 2023-09-30T22:32:26.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:26.082+0000 7f5b82ec3700 1 -- 172.21.15.16:0/733353654 wait complete. 2023-09-30T22:32:26.097 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:32:26.991 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:32:27.012 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:32:27.048 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:32:27.098 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:32:27.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.141+0000 7f5579110700 1 Processor -- start 2023-09-30T22:32:27.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.141+0000 7f5579110700 1 -- start start 2023-09-30T22:32:27.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.142+0000 7f5579110700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f557412cf60 0x7f557406f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:27.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.142+0000 7f5579110700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f557412f8b0 0x7f557412fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:27.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.142+0000 7f5579110700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f557412cd90 con 0x7f5574130260 2023-09-30T22:32:27.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.142+0000 7f5579110700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f557412d330 con 0x7f557412f8b0 2023-09-30T22:32:27.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.142+0000 7f5579110700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f55741306e0 con 0x7f557412cf60 2023-09-30T22:32:27.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.142+0000 7f557259c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f557412f8b0 0x7f557412fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:27.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.142+0000 7f557259c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f557412f8b0 0x7f557412fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34964/0 (socket says 172.21.15.16:34964) 2023-09-30T22:32:27.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.142+0000 7f557259c700 1 -- 172.21.15.16:0/701020327 learned_addr learned my addr 172.21.15.16:0/701020327 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:27.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.142+0000 7f557359e700 1 --1- 172.21.15.16:0/701020327 >> v1:172.21.15.16:6789/0 conn(0x7f5574130260 0x7f5574136c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53518/0 (socket says 172.21.15.16:53518) 2023-09-30T22:32:27.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.142+0000 7f5572d9d700 1 --2- 172.21.15.16:0/701020327 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f557412cf60 0x7f557406f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:27.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.142+0000 7f5571d9b700 1 -- 172.21.15.16:0/701020327 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1139525439 0 0) 0x7f557412cd90 con 0x7f5574130260 2023-09-30T22:32:27.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.142+0000 7f5571d9b700 1 -- 172.21.15.16:0/701020327 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f555c0035b0 con 0x7f5574130260 2023-09-30T22:32:27.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.142+0000 7f557259c700 1 -- 172.21.15.16:0/701020327 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f557412cf60 msgr2=0x7f557406f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:27.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.142+0000 7f557259c700 1 --2- 172.21.15.16:0/701020327 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f557412cf60 0x7f557406f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:27.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.142+0000 7f557259c700 1 -- 172.21.15.16:0/701020327 >> v1:172.21.15.16:6789/0 conn(0x7f5574130260 legacy=0x7f5574136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:27.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.142+0000 7f557259c700 1 -- 172.21.15.16:0/701020327 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5574071780 con 0x7f557412f8b0 2023-09-30T22:32:27.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.142+0000 7f5572d9d700 1 --2- 172.21.15.16:0/701020327 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f557412cf60 0x7f557406f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:32:27.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.143+0000 7f557259c700 1 --2- 172.21.15.16:0/701020327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f557412f8b0 0x7f557412fd20 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f55680098f0 tx=0x7f5568009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53f1ea050eb0148 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:27.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.143+0000 7f5571d9b700 1 -- 172.21.15.16:0/701020327 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5568011830 con 0x7f557412f8b0 2023-09-30T22:32:27.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.143+0000 7f5571d9b700 1 -- 172.21.15.16:0/701020327 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5568011990 con 0x7f557412f8b0 2023-09-30T22:32:27.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.143+0000 7f5571d9b700 1 -- 172.21.15.16:0/701020327 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5568011d20 con 0x7f557412f8b0 2023-09-30T22:32:27.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.144+0000 7f5579110700 1 -- 172.21.15.16:0/701020327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f557412f8b0 msgr2=0x7f557412fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:27.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.144+0000 7f5579110700 1 --2- 172.21.15.16:0/701020327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f557412f8b0 0x7f557412fd20 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f55680098f0 tx=0x7f5568009c00 comp rx=0 tx=0).stop 2023-09-30T22:32:27.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.144+0000 7f5579110700 1 -- 172.21.15.16:0/701020327 shutdown_connections 2023-09-30T22:32:27.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.144+0000 7f5579110700 1 --2- 172.21.15.16:0/701020327 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f557412cf60 0x7f557406f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:27.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.144+0000 7f5579110700 1 --2- 172.21.15.16:0/701020327 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f557412f8b0 0x7f557412fd20 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:27.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.144+0000 7f5579110700 1 -- 172.21.15.16:0/701020327 >> 172.21.15.16:0/701020327 conn(0x7f55740a2aa0 msgr2=0x7f55740a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:27.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.144+0000 7f5579110700 1 -- 172.21.15.16:0/701020327 shutdown_connections 2023-09-30T22:32:27.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.144+0000 7f5579110700 1 -- 172.21.15.16:0/701020327 wait complete. 2023-09-30T22:32:27.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.146+0000 7f5579110700 1 Processor -- start 2023-09-30T22:32:27.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.146+0000 7f5579110700 1 -- start start 2023-09-30T22:32:27.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.147+0000 7f5579110700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f557412cf60 0x7f557409b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:27.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.147+0000 7f5579110700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f557412f8b0 0x7f557409bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:27.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.147+0000 7f5579110700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5574130260 0x7f55741c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:27.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.147+0000 7f5579110700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f557409d2b0 con 0x7f557412cf60 2023-09-30T22:32:27.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.147+0000 7f5579110700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f557412d8d0 con 0x7f557412f8b0 2023-09-30T22:32:27.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.147+0000 7f5579110700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f557409b4c0 con 0x7f5574130260 2023-09-30T22:32:27.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.147+0000 7f5572d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f557412cf60 0x7f557409b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:27.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.147+0000 7f5572d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f557412cf60 0x7f557409b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34970/0 (socket says 172.21.15.16:34970) 2023-09-30T22:32:27.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.147+0000 7f5572d9d700 1 -- 172.21.15.16:0/550174229 learned_addr learned my addr 172.21.15.16:0/550174229 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:27.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.147+0000 7f557359e700 1 --2- 172.21.15.16:0/550174229 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5574130260 0x7f55741c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:27.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.148+0000 7f5572d9d700 1 -- 172.21.15.16:0/550174229 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5574130260 msgr2=0x7f55741c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:27.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.148+0000 7f5572d9d700 1 --2- 172.21.15.16:0/550174229 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5574130260 0x7f55741c8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:27.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.148+0000 7f5572d9d700 1 -- 172.21.15.16:0/550174229 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f557412f8b0 msgr2=0x7f557409bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:27.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.148+0000 7f5572d9d700 1 --2- 172.21.15.16:0/550174229 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f557412f8b0 0x7f557409bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:27.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.148+0000 7f5572d9d700 1 -- 172.21.15.16:0/550174229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5568009550 con 0x7f557412cf60 2023-09-30T22:32:27.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.148+0000 7f557359e700 1 --2- 172.21.15.16:0/550174229 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5574130260 0x7f55741c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:27.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.148+0000 7f5572d9d700 1 --2- 172.21.15.16:0/550174229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f557412cf60 0x7f557409b700 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f556400e840 tx=0x7f556400eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:27.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.148+0000 7f555b7fe700 1 -- 172.21.15.16:0/550174229 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5564013610 con 0x7f557412cf60 2023-09-30T22:32:27.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.148+0000 7f555b7fe700 1 -- 172.21.15.16:0/550174229 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5564013770 con 0x7f557412cf60 2023-09-30T22:32:27.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.148+0000 7f555b7fe700 1 -- 172.21.15.16:0/550174229 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5564013b00 con 0x7f557412cf60 2023-09-30T22:32:27.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.148+0000 7f5579110700 1 -- 172.21.15.16:0/550174229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f557409d470 con 0x7f557412cf60 2023-09-30T22:32:27.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.148+0000 7f5579110700 1 -- 172.21.15.16:0/550174229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f557409cc80 con 0x7f557412cf60 2023-09-30T22:32:27.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.149+0000 7f555b7fe700 1 -- 172.21.15.16:0/550174229 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5564013c60 con 0x7f557412cf60 2023-09-30T22:32:27.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.150+0000 7f5579110700 1 -- 172.21.15.16:0/550174229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5574074ea0 con 0x7f557412cf60 2023-09-30T22:32:27.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.150+0000 7f555b7fe700 1 -- 172.21.15.16:0/550174229 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f556403f320 con 0x7f557412cf60 2023-09-30T22:32:27.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.155+0000 7f555b7fe700 1 -- 172.21.15.16:0/550174229 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5564021020 con 0x7f557412cf60 2023-09-30T22:32:27.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.183+0000 7f071aaa7700 1 Processor -- start 2023-09-30T22:32:27.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.183+0000 7f071aaa7700 1 -- start start 2023-09-30T22:32:27.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.184+0000 7f071aaa7700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f071409e1d0 0x7f0714138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:27.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.184+0000 7f071aaa7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0714139170 0x7f071413b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:27.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.184+0000 7f071aaa7700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f071409b120 con 0x7f071409d6f0 2023-09-30T22:32:27.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.184+0000 7f071aaa7700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f071406b460 con 0x7f071409e1d0 2023-09-30T22:32:27.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.184+0000 7f071aaa7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f071409a7e0 con 0x7f0714139170 2023-09-30T22:32:27.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.184+0000 7f0713fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f071409e1d0 0x7f0714138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:27.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.184+0000 7f0713fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f071409e1d0 0x7f0714138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34972/0 (socket says 172.21.15.16:34972) 2023-09-30T22:32:27.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.184+0000 7f0713fff700 1 -- 172.21.15.16:0/3288933186 learned_addr learned my addr 172.21.15.16:0/3288933186 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:27.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.184+0000 7f0718843700 1 --1- 172.21.15.16:0/3288933186 >> v1:172.21.15.16:6789/0 conn(0x7f071409d6f0 0x7f071409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53520/0 (socket says 172.21.15.16:53520) 2023-09-30T22:32:27.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.184+0000 7f0719044700 1 --2- 172.21.15.16:0/3288933186 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0714139170 0x7f071413b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:27.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.184+0000 7f07137fe700 1 -- 172.21.15.16:0/3288933186 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1106005569 0 0) 0x7f071409b120 con 0x7f071409d6f0 2023-09-30T22:32:27.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.184+0000 7f07137fe700 1 -- 172.21.15.16:0/3288933186 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07040035b0 con 0x7f071409d6f0 2023-09-30T22:32:27.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.184+0000 7f0713fff700 1 -- 172.21.15.16:0/3288933186 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0714139170 msgr2=0x7f071413b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:27.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.184+0000 7f0713fff700 1 --2- 172.21.15.16:0/3288933186 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0714139170 0x7f071413b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:27.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.184+0000 7f0713fff700 1 -- 172.21.15.16:0/3288933186 >> v1:172.21.15.16:6789/0 conn(0x7f071409d6f0 legacy=0x7f071409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:27.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.184+0000 7f0713fff700 1 -- 172.21.15.16:0/3288933186 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f071409a500 con 0x7f071409e1d0 2023-09-30T22:32:27.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.185+0000 7f0719044700 1 --2- 172.21.15.16:0/3288933186 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0714139170 0x7f071413b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:32:27.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.185+0000 7f0713fff700 1 --2- 172.21.15.16:0/3288933186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f071409e1d0 0x7f0714138c30 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f0700009920 tx=0x7f0700009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3dd1f32a4e4a1c8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:27.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.185+0000 7f07137fe700 1 -- 172.21.15.16:0/3288933186 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0700011830 con 0x7f071409e1d0 2023-09-30T22:32:27.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.185+0000 7f07137fe700 1 -- 172.21.15.16:0/3288933186 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0700011990 con 0x7f071409e1d0 2023-09-30T22:32:27.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.185+0000 7f07137fe700 1 -- 172.21.15.16:0/3288933186 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0700011d20 con 0x7f071409e1d0 2023-09-30T22:32:27.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.186+0000 7f071aaa7700 1 -- 172.21.15.16:0/3288933186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f071409e1d0 msgr2=0x7f0714138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:27.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.186+0000 7f071aaa7700 1 --2- 172.21.15.16:0/3288933186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f071409e1d0 0x7f0714138c30 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f0700009920 tx=0x7f0700009c30 comp rx=0 tx=0).stop 2023-09-30T22:32:27.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.186+0000 7f071aaa7700 1 -- 172.21.15.16:0/3288933186 shutdown_connections 2023-09-30T22:32:27.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.186+0000 7f071aaa7700 1 --2- 172.21.15.16:0/3288933186 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0714139170 0x7f071413b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:27.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.186+0000 7f071aaa7700 1 --2- 172.21.15.16:0/3288933186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f071409e1d0 0x7f0714138c30 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:27.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.186+0000 7f071aaa7700 1 -- 172.21.15.16:0/3288933186 >> 172.21.15.16:0/3288933186 conn(0x7f0714066f50 msgr2=0x7f071409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:27.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.186+0000 7f071aaa7700 1 -- 172.21.15.16:0/3288933186 shutdown_connections 2023-09-30T22:32:27.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.186+0000 7f071aaa7700 1 -- 172.21.15.16:0/3288933186 wait complete. 2023-09-30T22:32:27.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.187+0000 7f071aaa7700 1 Processor -- start 2023-09-30T22:32:27.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.187+0000 7f071aaa7700 1 -- start start 2023-09-30T22:32:27.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.188+0000 7f071aaa7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f071409d6f0 0x7f07141215f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:27.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.188+0000 7f071aaa7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f071409e1d0 0x7f0714121b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:27.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.188+0000 7f071aaa7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0714139170 0x7f0714122070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:27.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.188+0000 7f071aaa7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0714124f10 con 0x7f071409d6f0 2023-09-30T22:32:27.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.188+0000 7f071aaa7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07141244c0 con 0x7f071409e1d0 2023-09-30T22:32:27.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.188+0000 7f071aaa7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0714125110 con 0x7f0714139170 2023-09-30T22:32:27.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.189+0000 7f0718843700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f071409d6f0 0x7f07141215f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:27.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.189+0000 7f0719044700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0714139170 0x7f0714122070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:27.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.189+0000 7f0718843700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f071409d6f0 0x7f07141215f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34988/0 (socket says 172.21.15.16:34988) 2023-09-30T22:32:27.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.189+0000 7f0718843700 1 -- 172.21.15.16:0/2425397937 learned_addr learned my addr 172.21.15.16:0/2425397937 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:27.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.189+0000 7f0713fff700 1 --2- 172.21.15.16:0/2425397937 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f071409e1d0 0x7f0714121b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:27.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.189+0000 7f0718843700 1 -- 172.21.15.16:0/2425397937 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0714139170 msgr2=0x7f0714122070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:27.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.189+0000 7f0718843700 1 --2- 172.21.15.16:0/2425397937 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0714139170 0x7f0714122070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:27.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.189+0000 7f0718843700 1 -- 172.21.15.16:0/2425397937 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f071409e1d0 msgr2=0x7f0714121b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:27.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.189+0000 7f0718843700 1 --2- 172.21.15.16:0/2425397937 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f071409e1d0 0x7f0714121b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:27.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.189+0000 7f0718843700 1 -- 172.21.15.16:0/2425397937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0700009550 con 0x7f071409d6f0 2023-09-30T22:32:27.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.189+0000 7f0713fff700 1 --2- 172.21.15.16:0/2425397937 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f071409e1d0 0x7f0714121b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:27.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.189+0000 7f0719044700 1 --2- 172.21.15.16:0/2425397937 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0714139170 0x7f0714122070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:32:27.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.189+0000 7f0718843700 1 --2- 172.21.15.16:0/2425397937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f071409d6f0 0x7f07141215f0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f0708001cc0 tx=0x7f070800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:27.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.190+0000 7f07117fa700 1 -- 172.21.15.16:0/2425397937 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070800ad80 con 0x7f071409d6f0 2023-09-30T22:32:27.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.190+0000 7f07117fa700 1 -- 172.21.15.16:0/2425397937 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07080138e0 con 0x7f071409d6f0 2023-09-30T22:32:27.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.190+0000 7f071aaa7700 1 -- 172.21.15.16:0/2425397937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0714124000 con 0x7f071409d6f0 2023-09-30T22:32:27.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.190+0000 7f07117fa700 1 -- 172.21.15.16:0/2425397937 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0708013c70 con 0x7f071409d6f0 2023-09-30T22:32:27.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.190+0000 7f071aaa7700 1 -- 172.21.15.16:0/2425397937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07140a28a0 con 0x7f071409d6f0 2023-09-30T22:32:27.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.191+0000 7f071aaa7700 1 -- 172.21.15.16:0/2425397937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0714074ea0 con 0x7f071409d6f0 2023-09-30T22:32:27.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.192+0000 7f07117fa700 1 -- 172.21.15.16:0/2425397937 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f07080070d0 con 0x7f071409d6f0 2023-09-30T22:32:27.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.192+0000 7f07117fa700 1 -- 172.21.15.16:0/2425397937 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f0708047bf0 con 0x7f071409d6f0 2023-09-30T22:32:27.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.196+0000 7f07117fa700 1 -- 172.21.15.16:0/2425397937 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0708013dd0 con 0x7f071409d6f0 2023-09-30T22:32:27.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.225+0000 7f7156466700 1 Processor -- start 2023-09-30T22:32:27.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.225+0000 7f7156466700 1 -- start start 2023-09-30T22:32:27.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.225+0000 7f7156466700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f715009d6f0 0x7f715009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:27.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.225+0000 7f7156466700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f715009e000 0x7f71501339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:27.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.225+0000 7f7156466700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f715009b120 con 0x7f715009e410 2023-09-30T22:32:27.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.225+0000 7f7156466700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f715006b460 con 0x7f715009d6f0 2023-09-30T22:32:27.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.225+0000 7f7156466700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f715009a7e0 con 0x7f715009e000 2023-09-30T22:32:27.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.226+0000 7f714ffff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f715009d6f0 0x7f715009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:27.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.226+0000 7f7154a03700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f715009e410 0x7f7150133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53536/0 (socket says 172.21.15.16:53536) 2023-09-30T22:32:27.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.226+0000 7f7154a03700 1 -- 172.21.15.16:0/1814063705 learned_addr learned my addr 172.21.15.16:0/1814063705 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:27.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.226+0000 7f714f7fe700 1 --2- 172.21.15.16:0/1814063705 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f715009e000 0x7f71501339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:27.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.226+0000 7f714ffff700 1 -- 172.21.15.16:0/1814063705 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f715009e000 msgr2=0x7f71501339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:27.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.226+0000 7f714ffff700 1 --2- 172.21.15.16:0/1814063705 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f715009e000 0x7f71501339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:27.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.226+0000 7f714ffff700 1 -- 172.21.15.16:0/1814063705 >> v1:172.21.15.16:6789/0 conn(0x7f715009e410 legacy=0x7f7150133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:27.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.226+0000 7f714ffff700 1 -- 172.21.15.16:0/1814063705 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f715009a500 con 0x7f715009d6f0 2023-09-30T22:32:27.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.226+0000 7f714ffff700 1 --2- 172.21.15.16:0/1814063705 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f715009d6f0 0x7f715009dac0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7140009870 tx=0x7f7140009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c775208c1786698d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:27.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.226+0000 7f714effd700 1 -- 172.21.15.16:0/1814063705 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7140011830 con 0x7f715009d6f0 2023-09-30T22:32:27.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.227+0000 7f714effd700 1 -- 172.21.15.16:0/1814063705 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7140011990 con 0x7f715009d6f0 2023-09-30T22:32:27.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.227+0000 7f714effd700 1 -- 172.21.15.16:0/1814063705 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7140011d20 con 0x7f715009d6f0 2023-09-30T22:32:27.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.227+0000 7f7156466700 1 -- 172.21.15.16:0/1814063705 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f715009d6f0 msgr2=0x7f715009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:27.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.227+0000 7f7156466700 1 --2- 172.21.15.16:0/1814063705 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f715009d6f0 0x7f715009dac0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f7140009870 tx=0x7f7140009b80 comp rx=0 tx=0).stop 2023-09-30T22:32:27.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.227+0000 7f7156466700 1 -- 172.21.15.16:0/1814063705 shutdown_connections 2023-09-30T22:32:27.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.227+0000 7f7156466700 1 --2- 172.21.15.16:0/1814063705 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f715009d6f0 0x7f715009dac0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:27.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.227+0000 7f7156466700 1 --2- 172.21.15.16:0/1814063705 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f715009e000 0x7f71501339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:27.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.227+0000 7f7156466700 1 -- 172.21.15.16:0/1814063705 >> 172.21.15.16:0/1814063705 conn(0x7f7150066f50 msgr2=0x7f715009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:27.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.227+0000 7f7156466700 1 -- 172.21.15.16:0/1814063705 shutdown_connections 2023-09-30T22:32:27.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.227+0000 7f7156466700 1 -- 172.21.15.16:0/1814063705 wait complete. 2023-09-30T22:32:27.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.228+0000 7f7156466700 1 Processor -- start 2023-09-30T22:32:27.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.229+0000 7f7156466700 1 -- start start 2023-09-30T22:32:27.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.229+0000 7f7156466700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f715009d6f0 0x7f7150122490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:27.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.229+0000 7f7156466700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f715009e000 0x7f71501229d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:27.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.229+0000 7f7156466700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f715009e410 0x7f7150122f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:27.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.229+0000 7f7156466700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7150125a10 con 0x7f715009e000 2023-09-30T22:32:27.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.229+0000 7f7156466700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71501238f0 con 0x7f715009d6f0 2023-09-30T22:32:27.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.229+0000 7f7156466700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7150124900 con 0x7f715009e410 2023-09-30T22:32:27.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.229+0000 7f714f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f715009e000 0x7f71501229d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:27.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.229+0000 7f714f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f715009e000 0x7f71501229d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34996/0 (socket says 172.21.15.16:34996) 2023-09-30T22:32:27.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.229+0000 7f714f7fe700 1 -- 172.21.15.16:0/1070290413 learned_addr learned my addr 172.21.15.16:0/1070290413 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:27.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.229+0000 7f714ffff700 1 --2- 172.21.15.16:0/1070290413 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f715009d6f0 0x7f7150122490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:27.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.229+0000 7f7154a03700 1 --2- 172.21.15.16:0/1070290413 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f715009e410 0x7f7150122f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:27.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.230+0000 7f714ffff700 1 -- 172.21.15.16:0/1070290413 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f715009e410 msgr2=0x7f7150122f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:27.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.230+0000 7f714ffff700 1 --2- 172.21.15.16:0/1070290413 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f715009e410 0x7f7150122f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:27.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.230+0000 7f714ffff700 1 -- 172.21.15.16:0/1070290413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f715009e000 msgr2=0x7f71501229d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:27.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.230+0000 7f714ffff700 1 --2- 172.21.15.16:0/1070290413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f715009e000 0x7f71501229d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:27.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.230+0000 7f714ffff700 1 -- 172.21.15.16:0/1070290413 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71400094a0 con 0x7f715009d6f0 2023-09-30T22:32:27.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.230+0000 7f714f7fe700 1 --2- 172.21.15.16:0/1070290413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f715009e000 0x7f71501229d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:27.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.230+0000 7f714ffff700 1 --2- 172.21.15.16:0/1070290413 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f715009d6f0 0x7f7150122490 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7140009870 tx=0x7f7140004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:27.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.230+0000 7f714cff9700 1 -- 172.21.15.16:0/1070290413 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7140004510 con 0x7f715009d6f0 2023-09-30T22:32:27.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.230+0000 7f714cff9700 1 -- 172.21.15.16:0/1070290413 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f714001f070 con 0x7f715009d6f0 2023-09-30T22:32:27.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.230+0000 7f714cff9700 1 -- 172.21.15.16:0/1070290413 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71400038e0 con 0x7f715009d6f0 2023-09-30T22:32:27.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.230+0000 7f7156466700 1 -- 172.21.15.16:0/1070290413 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71500a2be0 con 0x7f715009d6f0 2023-09-30T22:32:27.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.231+0000 7f7156466700 1 -- 172.21.15.16:0/1070290413 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f715009c770 con 0x7f715009d6f0 2023-09-30T22:32:27.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.232+0000 7f7156466700 1 -- 172.21.15.16:0/1070290413 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7150125360 con 0x7f715009d6f0 2023-09-30T22:32:27.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.233+0000 7f714cff9700 1 -- 172.21.15.16:0/1070290413 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7140003a40 con 0x7f715009d6f0 2023-09-30T22:32:27.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.233+0000 7f714cff9700 1 -- 172.21.15.16:0/1070290413 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f7140021070 con 0x7f715009d6f0 2023-09-30T22:32:27.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.242+0000 7f714cff9700 1 -- 172.21.15.16:0/1070290413 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f714002f3a0 con 0x7f715009d6f0 2023-09-30T22:32:27.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.283+0000 7f5e64923700 1 Processor -- start 2023-09-30T22:32:27.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.284+0000 7f5e64923700 1 -- start start 2023-09-30T22:32:27.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.284+0000 7f5e64923700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5e600a0a30 0x7f5e600a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:27.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.285+0000 7f5e64923700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5e600a1340 0x7f5e60133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:27.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.285+0000 7f5e64923700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e6009edd0 con 0x7f5e60133fa0 2023-09-30T22:32:27.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.285+0000 7f5e64923700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e6009e1f0 con 0x7f5e600a0a30 2023-09-30T22:32:27.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.285+0000 7f5e64923700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5e6009dc50 con 0x7f5e600a1340 2023-09-30T22:32:27.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.285+0000 7f5e5e59c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5e600a0a30 0x7f5e600a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:27.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.285+0000 7f5e5e59c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5e600a0a30 0x7f5e600a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:59964/0 (socket says 172.21.15.16:59964) 2023-09-30T22:32:27.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.285+0000 7f5e5e59c700 1 -- 172.21.15.16:0/2172062803 learned_addr learned my addr 172.21.15.16:0/2172062803 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:27.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.285+0000 7f5e5dd9b700 1 --2- 172.21.15.16:0/2172062803 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5e600a1340 0x7f5e60133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:27.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.285+0000 7f5e5d59a700 1 -- 172.21.15.16:0/2172062803 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2570952670 0 0) 0x7f5e6009edd0 con 0x7f5e60133fa0 2023-09-30T22:32:27.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.285+0000 7f5e5d59a700 1 -- 172.21.15.16:0/2172062803 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e480035b0 con 0x7f5e60133fa0 2023-09-30T22:32:27.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.285+0000 7f5e5e59c700 1 -- 172.21.15.16:0/2172062803 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5e600a1340 msgr2=0x7f5e60133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:27.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.285+0000 7f5e5e59c700 1 --2- 172.21.15.16:0/2172062803 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5e600a1340 0x7f5e60133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:27.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.285+0000 7f5e5e59c700 1 -- 172.21.15.16:0/2172062803 >> v1:172.21.15.16:6789/0 conn(0x7f5e60133fa0 legacy=0x7f5e6013ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:27.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.285+0000 7f5e5e59c700 1 -- 172.21.15.16:0/2172062803 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e600a1fe0 con 0x7f5e600a0a30 2023-09-30T22:32:27.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.286+0000 7f5e5e59c700 1 --2- 172.21.15.16:0/2172062803 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5e600a0a30 0x7f5e600a0e00 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5e50009920 tx=0x7f5e50009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4286aa3aa40a6f3f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:27.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.286+0000 7f5e5d59a700 1 -- 172.21.15.16:0/2172062803 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e50011830 con 0x7f5e600a0a30 2023-09-30T22:32:27.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.286+0000 7f5e5d59a700 1 -- 172.21.15.16:0/2172062803 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e50011990 con 0x7f5e600a0a30 2023-09-30T22:32:27.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.286+0000 7f5e5d59a700 1 -- 172.21.15.16:0/2172062803 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e50011d00 con 0x7f5e600a0a30 2023-09-30T22:32:27.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.287+0000 7f5e64923700 1 -- 172.21.15.16:0/2172062803 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5e600a0a30 msgr2=0x7f5e600a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:27.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.287+0000 7f5e64923700 1 --2- 172.21.15.16:0/2172062803 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5e600a0a30 0x7f5e600a0e00 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5e50009920 tx=0x7f5e50009c30 comp rx=0 tx=0).stop 2023-09-30T22:32:27.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.287+0000 7f5e64923700 1 -- 172.21.15.16:0/2172062803 shutdown_connections 2023-09-30T22:32:27.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.287+0000 7f5e64923700 1 --2- 172.21.15.16:0/2172062803 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5e600a0a30 0x7f5e600a0e00 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:27.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.287+0000 7f5e64923700 1 --2- 172.21.15.16:0/2172062803 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5e600a1340 0x7f5e60133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:27.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.287+0000 7f5e64923700 1 -- 172.21.15.16:0/2172062803 >> 172.21.15.16:0/2172062803 conn(0x7f5e600a6550 msgr2=0x7f5e600a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:27.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.287+0000 7f5e64923700 1 -- 172.21.15.16:0/2172062803 shutdown_connections 2023-09-30T22:32:27.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.287+0000 7f5e64923700 1 -- 172.21.15.16:0/2172062803 wait complete. 2023-09-30T22:32:27.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.288+0000 7f5e64923700 1 Processor -- start 2023-09-30T22:32:27.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.288+0000 7f5e64923700 1 -- start start 2023-09-30T22:32:27.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.289+0000 7f5e64923700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e600a0a30 0x7f5e601c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:27.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.289+0000 7f5e64923700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5e600a1340 0x7f5e601c8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:27.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.289+0000 7f5e64923700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5e60133fa0 0x7f5e601c8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:27.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.289+0000 7f5e64923700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e60125260 con 0x7f5e600a0a30 2023-09-30T22:32:27.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.289+0000 7f5e64923700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e601244f0 con 0x7f5e60133fa0 2023-09-30T22:32:27.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.289+0000 7f5e64923700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5e60122d80 con 0x7f5e600a1340 2023-09-30T22:32:27.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.289+0000 7f5e5e59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e600a0a30 0x7f5e601c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:27.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.289+0000 7f5e5e59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e600a0a30 0x7f5e601c82f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34998/0 (socket says 172.21.15.16:34998) 2023-09-30T22:32:27.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.289+0000 7f5e5e59c700 1 -- 172.21.15.16:0/1191917969 learned_addr learned my addr 172.21.15.16:0/1191917969 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:27.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.289+0000 7f5e5dd9b700 1 --2- 172.21.15.16:0/1191917969 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5e600a1340 0x7f5e601c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:27.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.289+0000 7f5e5ed9d700 1 --2- 172.21.15.16:0/1191917969 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5e60133fa0 0x7f5e601c8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:27.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.290+0000 7f5e5e59c700 1 -- 172.21.15.16:0/1191917969 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5e600a1340 msgr2=0x7f5e601c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:27.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.290+0000 7f5e5e59c700 1 --2- 172.21.15.16:0/1191917969 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5e600a1340 0x7f5e601c8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:27.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.290+0000 7f5e5e59c700 1 -- 172.21.15.16:0/1191917969 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5e60133fa0 msgr2=0x7f5e601c8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:27.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.290+0000 7f5e5e59c700 1 --2- 172.21.15.16:0/1191917969 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5e60133fa0 0x7f5e601c8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:27.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.290+0000 7f5e5e59c700 1 -- 172.21.15.16:0/1191917969 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e50009550 con 0x7f5e600a0a30 2023-09-30T22:32:27.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.290+0000 7f5e5dd9b700 1 --2- 172.21.15.16:0/1191917969 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5e600a1340 0x7f5e601c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:27.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.290+0000 7f5e5e59c700 1 --2- 172.21.15.16:0/1191917969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e600a0a30 0x7f5e601c82f0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f5e6009e380 tx=0x7f5e50004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:27.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.290+0000 7f5e4effd700 1 -- 172.21.15.16:0/1191917969 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e50004450 con 0x7f5e600a0a30 2023-09-30T22:32:27.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.290+0000 7f5e4effd700 1 -- 172.21.15.16:0/1191917969 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e500045b0 con 0x7f5e600a0a30 2023-09-30T22:32:27.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.290+0000 7f5e64923700 1 -- 172.21.15.16:0/1191917969 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e600a25e0 con 0x7f5e600a0a30 2023-09-30T22:32:27.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.290+0000 7f5e4effd700 1 -- 172.21.15.16:0/1191917969 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e500038c0 con 0x7f5e600a0a30 2023-09-30T22:32:27.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.290+0000 7f5e64923700 1 -- 172.21.15.16:0/1191917969 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e60125e60 con 0x7f5e600a0a30 2023-09-30T22:32:27.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.291+0000 7f5e4effd700 1 -- 172.21.15.16:0/1191917969 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5e50021020 con 0x7f5e600a0a30 2023-09-30T22:32:27.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.291+0000 7f5e64923700 1 -- 172.21.15.16:0/1191917969 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e60074ea0 con 0x7f5e600a0a30 2023-09-30T22:32:27.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.291+0000 7f5e4effd700 1 -- 172.21.15.16:0/1191917969 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f5e50040760 con 0x7f5e600a0a30 2023-09-30T22:32:27.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.301+0000 7f5e4effd700 1 -- 172.21.15.16:0/1191917969 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5e5001a450 con 0x7f5e600a0a30 2023-09-30T22:32:27.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.373+0000 7f5579110700 1 -- 172.21.15.16:0/550174229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f55740713a0 con 0x7f557412cf60 2023-09-30T22:32:27.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.374+0000 7f555b7fe700 1 -- 172.21.15.16:0/550174229 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5564012370 con 0x7f557412cf60 2023-09-30T22:32:27.375 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:32:27.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.377+0000 7f5579110700 1 -- 172.21.15.16:0/550174229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f557412cf60 msgr2=0x7f557409b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:27.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.377+0000 7f5579110700 1 --2- 172.21.15.16:0/550174229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f557412cf60 0x7f557409b700 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f556400e840 tx=0x7f556400eb50 comp rx=0 tx=0).stop 2023-09-30T22:32:27.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.377+0000 7f5579110700 1 -- 172.21.15.16:0/550174229 shutdown_connections 2023-09-30T22:32:27.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.377+0000 7f5579110700 1 --2- 172.21.15.16:0/550174229 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5574130260 0x7f55741c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:27.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.377+0000 7f5579110700 1 --2- 172.21.15.16:0/550174229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f557412cf60 0x7f557409b700 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:27.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.377+0000 7f5579110700 1 --2- 172.21.15.16:0/550174229 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f557412f8b0 0x7f557409bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:27.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.377+0000 7f5579110700 1 -- 172.21.15.16:0/550174229 >> 172.21.15.16:0/550174229 conn(0x7f55740a2aa0 msgr2=0x7f557412a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:27.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.377+0000 7f5579110700 1 -- 172.21.15.16:0/550174229 shutdown_connections 2023-09-30T22:32:27.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.378+0000 7f5579110700 1 -- 172.21.15.16:0/550174229 wait complete. 2023-09-30T22:32:27.394 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:32:27.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.420+0000 7f071aaa7700 1 -- 172.21.15.16:0/2425397937 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f071409f420 con 0x7f071409d6f0 2023-09-30T22:32:27.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.420+0000 7f07117fa700 1 -- 172.21.15.16:0/2425397937 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f070802d070 con 0x7f071409d6f0 2023-09-30T22:32:27.421 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:32:27.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.422+0000 7f071aaa7700 1 -- 172.21.15.16:0/2425397937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f071409d6f0 msgr2=0x7f07141215f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:27.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.422+0000 7f071aaa7700 1 --2- 172.21.15.16:0/2425397937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f071409d6f0 0x7f07141215f0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f0708001cc0 tx=0x7f070800a340 comp rx=0 tx=0).stop 2023-09-30T22:32:27.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.423+0000 7f071aaa7700 1 -- 172.21.15.16:0/2425397937 shutdown_connections 2023-09-30T22:32:27.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.423+0000 7f071aaa7700 1 --2- 172.21.15.16:0/2425397937 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0714139170 0x7f0714122070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:27.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.423+0000 7f071aaa7700 1 --2- 172.21.15.16:0/2425397937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f071409d6f0 0x7f07141215f0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:27.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.423+0000 7f071aaa7700 1 --2- 172.21.15.16:0/2425397937 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f071409e1d0 0x7f0714121b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:27.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.423+0000 7f071aaa7700 1 -- 172.21.15.16:0/2425397937 >> 172.21.15.16:0/2425397937 conn(0x7f0714066f50 msgr2=0x7f071406ee90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:27.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.423+0000 7f071aaa7700 1 -- 172.21.15.16:0/2425397937 shutdown_connections 2023-09-30T22:32:27.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.423+0000 7f071aaa7700 1 -- 172.21.15.16:0/2425397937 wait complete. 2023-09-30T22:32:27.438 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:32:27.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.467+0000 7f7156466700 1 -- 172.21.15.16:0/1070290413 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f7150074ea0 con 0x7f715009d6f0 2023-09-30T22:32:27.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.468+0000 7f714cff9700 1 -- 172.21.15.16:0/1070290413 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f714002b020 con 0x7f715009d6f0 2023-09-30T22:32:27.469 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:32:27.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.470+0000 7f7156466700 1 -- 172.21.15.16:0/1070290413 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f715009d6f0 msgr2=0x7f7150122490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:27.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.470+0000 7f7156466700 1 --2- 172.21.15.16:0/1070290413 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f715009d6f0 0x7f7150122490 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7140009870 tx=0x7f7140004750 comp rx=0 tx=0).stop 2023-09-30T22:32:27.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.470+0000 7f7156466700 1 -- 172.21.15.16:0/1070290413 shutdown_connections 2023-09-30T22:32:27.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.470+0000 7f7156466700 1 --2- 172.21.15.16:0/1070290413 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f715009d6f0 0x7f7150122490 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:27.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.470+0000 7f7156466700 1 --2- 172.21.15.16:0/1070290413 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f715009e410 0x7f7150122f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:27.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.470+0000 7f7156466700 1 --2- 172.21.15.16:0/1070290413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f715009e000 0x7f71501229d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:27.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.470+0000 7f7156466700 1 -- 172.21.15.16:0/1070290413 >> 172.21.15.16:0/1070290413 conn(0x7f7150066f50 msgr2=0x7f71500a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:27.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.470+0000 7f7156466700 1 -- 172.21.15.16:0/1070290413 shutdown_connections 2023-09-30T22:32:27.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.470+0000 7f7156466700 1 -- 172.21.15.16:0/1070290413 wait complete. 2023-09-30T22:32:27.485 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:32:27.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.511+0000 7f5e64923700 1 -- 172.21.15.16:0/1191917969 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5e600a28e0 con 0x7f5e600a0a30 2023-09-30T22:32:27.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.512+0000 7f5e4effd700 1 -- 172.21.15.16:0/1191917969 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5e50029070 con 0x7f5e600a0a30 2023-09-30T22:32:27.513 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:32:27.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.514+0000 7f5e64923700 1 -- 172.21.15.16:0/1191917969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e600a0a30 msgr2=0x7f5e601c82f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:27.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.514+0000 7f5e64923700 1 --2- 172.21.15.16:0/1191917969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e600a0a30 0x7f5e601c82f0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f5e6009e380 tx=0x7f5e50004750 comp rx=0 tx=0).stop 2023-09-30T22:32:27.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.514+0000 7f5e64923700 1 -- 172.21.15.16:0/1191917969 shutdown_connections 2023-09-30T22:32:27.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.514+0000 7f5e64923700 1 --2- 172.21.15.16:0/1191917969 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5e60133fa0 0x7f5e601c8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:27.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.514+0000 7f5e64923700 1 --2- 172.21.15.16:0/1191917969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e600a0a30 0x7f5e601c82f0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:27.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.514+0000 7f5e64923700 1 --2- 172.21.15.16:0/1191917969 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5e600a1340 0x7f5e601c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:27.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.514+0000 7f5e64923700 1 -- 172.21.15.16:0/1191917969 >> 172.21.15.16:0/1191917969 conn(0x7f5e600a6550 msgr2=0x7f5e600a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:27.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.514+0000 7f5e64923700 1 -- 172.21.15.16:0/1191917969 shutdown_connections 2023-09-30T22:32:27.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:27.514+0000 7f5e64923700 1 -- 172.21.15.16:0/1191917969 wait complete. 2023-09-30T22:32:27.529 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:32:28.395 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:32:28.439 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:32:28.485 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:32:28.530 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:32:28.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.548+0000 7f07b0dd3700 1 Processor -- start 2023-09-30T22:32:28.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.548+0000 7f07b0dd3700 1 -- start start 2023-09-30T22:32:28.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.549+0000 7f07b0dd3700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f07ac09d6f0 0x7f07ac09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:28.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.549+0000 7f07b0dd3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f07ac09e410 0x7f07ac130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:28.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.549+0000 7f07b0dd3700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07ac09f130 con 0x7f07ac09e000 2023-09-30T22:32:28.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.549+0000 7f07b0dd3700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07ac09f520 con 0x7f07ac09d6f0 2023-09-30T22:32:28.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.549+0000 7f07b0dd3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07ac09e930 con 0x7f07ac09e410 2023-09-30T22:32:28.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.549+0000 7f07aa59c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f07ac09d6f0 0x7f07ac09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:28.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.549+0000 7f07aa59c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f07ac09d6f0 0x7f07ac09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56066/0 (socket says 172.21.15.16:56066) 2023-09-30T22:32:28.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.549+0000 7f07aa59c700 1 -- 172.21.15.16:0/3326638062 learned_addr learned my addr 172.21.15.16:0/3326638062 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:28.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.549+0000 7f07a9d9b700 1 --1- 172.21.15.16:0/3326638062 >> v1:172.21.15.16:6789/0 conn(0x7f07ac09e000 0x7f07ac130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38976/0 (socket says 172.21.15.16:38976) 2023-09-30T22:32:28.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.550+0000 7f07aad9d700 1 --2- 172.21.15.16:0/3326638062 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f07ac09e410 0x7f07ac130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:28.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.550+0000 7f07aa59c700 1 -- 172.21.15.16:0/3326638062 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f07ac09e410 msgr2=0x7f07ac130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:28.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.550+0000 7f07aa59c700 1 --2- 172.21.15.16:0/3326638062 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f07ac09e410 0x7f07ac130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:28.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.550+0000 7f07aa59c700 1 -- 172.21.15.16:0/3326638062 >> v1:172.21.15.16:6789/0 conn(0x7f07ac09e000 legacy=0x7f07ac130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:28.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.550+0000 7f07aa59c700 1 -- 172.21.15.16:0/3326638062 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07ac09b2d0 con 0x7f07ac09d6f0 2023-09-30T22:32:28.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.550+0000 7f07aad9d700 1 --2- 172.21.15.16:0/3326638062 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f07ac09e410 0x7f07ac130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:32:28.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.550+0000 7f07aa59c700 1 --2- 172.21.15.16:0/3326638062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07ac09d6f0 0x7f07ac09dac0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f079c0098f0 tx=0x7f079c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9944cb3a81e4172b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:28.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.550+0000 7f07a959a700 1 -- 172.21.15.16:0/3326638062 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079c00b8a0 con 0x7f07ac09d6f0 2023-09-30T22:32:28.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.550+0000 7f07a959a700 1 -- 172.21.15.16:0/3326638062 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f079c00ba00 con 0x7f07ac09d6f0 2023-09-30T22:32:28.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.550+0000 7f07a959a700 1 -- 172.21.15.16:0/3326638062 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f079c00bd90 con 0x7f07ac09d6f0 2023-09-30T22:32:28.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.551+0000 7f07b0dd3700 1 -- 172.21.15.16:0/3326638062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07ac09d6f0 msgr2=0x7f07ac09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:28.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.551+0000 7f07b0dd3700 1 --2- 172.21.15.16:0/3326638062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07ac09d6f0 0x7f07ac09dac0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f079c0098f0 tx=0x7f079c009c00 comp rx=0 tx=0).stop 2023-09-30T22:32:28.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.551+0000 7f07b0dd3700 1 -- 172.21.15.16:0/3326638062 shutdown_connections 2023-09-30T22:32:28.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.551+0000 7f07b0dd3700 1 --2- 172.21.15.16:0/3326638062 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f07ac09e410 0x7f07ac130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:28.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.551+0000 7f07b0dd3700 1 --2- 172.21.15.16:0/3326638062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07ac09d6f0 0x7f07ac09dac0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:28.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.551+0000 7f07b0dd3700 1 -- 172.21.15.16:0/3326638062 >> 172.21.15.16:0/3326638062 conn(0x7f07ac066f50 msgr2=0x7f07ac09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:28.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.551+0000 7f07b0dd3700 1 -- 172.21.15.16:0/3326638062 shutdown_connections 2023-09-30T22:32:28.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.551+0000 7f07b0dd3700 1 -- 172.21.15.16:0/3326638062 wait complete. 2023-09-30T22:32:28.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.552+0000 7f07b0dd3700 1 Processor -- start 2023-09-30T22:32:28.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.552+0000 7f07b0dd3700 1 -- start start 2023-09-30T22:32:28.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.553+0000 7f07b0dd3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f07ac09d6f0 0x7f07ac1c5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:28.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.553+0000 7f07b0dd3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f07ac09e000 0x7f07ac1c5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:28.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.553+0000 7f07b0dd3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07ac09e410 0x7f07ac1c8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:28.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.553+0000 7f07b0dd3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07ac12fe60 con 0x7f07ac09e410 2023-09-30T22:32:28.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.553+0000 7f07b0dd3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07ac131b00 con 0x7f07ac09d6f0 2023-09-30T22:32:28.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.553+0000 7f07b0dd3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f07ac0722a0 con 0x7f07ac09e000 2023-09-30T22:32:28.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.553+0000 7f07aad9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07ac09e410 0x7f07ac1c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:28.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.553+0000 7f07aad9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07ac09e410 0x7f07ac1c8170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56074/0 (socket says 172.21.15.16:56074) 2023-09-30T22:32:28.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.553+0000 7f07aad9d700 1 -- 172.21.15.16:0/3380323445 learned_addr learned my addr 172.21.15.16:0/3380323445 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:28.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.553+0000 7f07a9d9b700 1 --2- 172.21.15.16:0/3380323445 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f07ac09e000 0x7f07ac1c5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:28.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.553+0000 7f07aa59c700 1 --2- 172.21.15.16:0/3380323445 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f07ac09d6f0 0x7f07ac1c5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:28.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.553+0000 7f07aad9d700 1 -- 172.21.15.16:0/3380323445 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f07ac09e000 msgr2=0x7f07ac1c5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:28.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.553+0000 7f07aad9d700 1 --2- 172.21.15.16:0/3380323445 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f07ac09e000 0x7f07ac1c5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:28.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.553+0000 7f07aad9d700 1 -- 172.21.15.16:0/3380323445 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f07ac09d6f0 msgr2=0x7f07ac1c5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:28.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.553+0000 7f07aad9d700 1 --2- 172.21.15.16:0/3380323445 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f07ac09d6f0 0x7f07ac1c5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:28.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.553+0000 7f07aad9d700 1 -- 172.21.15.16:0/3380323445 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f079c009550 con 0x7f07ac09e410 2023-09-30T22:32:28.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.554+0000 7f07aad9d700 1 --2- 172.21.15.16:0/3380323445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07ac09e410 0x7f07ac1c8170 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f07a400cf20 tx=0x7f07a400cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:28.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.554+0000 7f079affd700 1 -- 172.21.15.16:0/3380323445 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a400ad60 con 0x7f07ac09e410 2023-09-30T22:32:28.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.554+0000 7f079affd700 1 -- 172.21.15.16:0/3380323445 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07a4015440 con 0x7f07ac09e410 2023-09-30T22:32:28.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.554+0000 7f07b0dd3700 1 -- 172.21.15.16:0/3380323445 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07ac09c700 con 0x7f07ac09e410 2023-09-30T22:32:28.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.554+0000 7f079affd700 1 -- 172.21.15.16:0/3380323445 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a40157f0 con 0x7f07ac09e410 2023-09-30T22:32:28.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.554+0000 7f07b0dd3700 1 -- 172.21.15.16:0/3380323445 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07ac130570 con 0x7f07ac09e410 2023-09-30T22:32:28.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.555+0000 7f079affd700 1 -- 172.21.15.16:0/3380323445 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f07a400d430 con 0x7f07ac09e410 2023-09-30T22:32:28.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.555+0000 7f079affd700 1 -- 172.21.15.16:0/3380323445 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f07a403fca0 con 0x7f07ac09e410 2023-09-30T22:32:28.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.555+0000 7f07b0dd3700 1 -- 172.21.15.16:0/3380323445 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07ac074ea0 con 0x7f07ac09e410 2023-09-30T22:32:28.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.561+0000 7f079affd700 1 -- 172.21.15.16:0/3380323445 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f07a401a020 con 0x7f07ac09e410 2023-09-30T22:32:28.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.593+0000 7f72947e4700 1 Processor -- start 2023-09-30T22:32:28.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.593+0000 7f72947e4700 1 -- start start 2023-09-30T22:32:28.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.594+0000 7f72947e4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f728c134630 0x7f728c136a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:28.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.594+0000 7f72947e4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f728c136f50 0x7f728c139330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:28.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.594+0000 7f72947e4700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f728c09b4f0 con 0x7f728c131810 2023-09-30T22:32:28.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.594+0000 7f72947e4700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f728c1313c0 con 0x7f728c136f50 2023-09-30T22:32:28.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.594+0000 7f72947e4700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f728c06e8c0 con 0x7f728c134630 2023-09-30T22:32:28.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.594+0000 7f7292580700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f728c131810 0x7f728c131be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38982/0 (socket says 172.21.15.16:38982) 2023-09-30T22:32:28.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.594+0000 7f7292580700 1 -- 172.21.15.16:0/3669951580 learned_addr learned my addr 172.21.15.16:0/3669951580 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:28.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.595+0000 7f7292d81700 1 --2- 172.21.15.16:0/3669951580 >> v2:172.21.15.16:3300/0 conn(0x7f728c136f50 0x7f728c139330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:28.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.595+0000 7f7291d7f700 1 --2- 172.21.15.16:0/3669951580 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f728c134630 0x7f728c136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:28.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.595+0000 7f729157e700 1 -- 172.21.15.16:0/3669951580 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2361082291 0 0) 0x7f728c09b4f0 con 0x7f728c131810 2023-09-30T22:32:28.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.595+0000 7f729157e700 1 -- 172.21.15.16:0/3669951580 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72780035b0 con 0x7f728c131810 2023-09-30T22:32:28.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.595+0000 7f7292d81700 1 -- 172.21.15.16:0/3669951580 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f728c134630 msgr2=0x7f728c136a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:28.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.595+0000 7f7292d81700 1 --2- 172.21.15.16:0/3669951580 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f728c134630 0x7f728c136a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:28.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.595+0000 7f7292d81700 1 -- 172.21.15.16:0/3669951580 >> v1:172.21.15.16:6789/0 conn(0x7f728c131810 legacy=0x7f728c131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:28.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.595+0000 7f7292d81700 1 -- 172.21.15.16:0/3669951580 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f728c071d20 con 0x7f728c136f50 2023-09-30T22:32:28.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.595+0000 7f7291d7f700 1 --2- 172.21.15.16:0/3669951580 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f728c134630 0x7f728c136a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:28.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.595+0000 7f7292d81700 1 --2- 172.21.15.16:0/3669951580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f728c136f50 0x7f728c139330 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f7284009d90 tx=0x7f728400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa81440cef330ed8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:28.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.595+0000 7f729157e700 1 -- 172.21.15.16:0/3669951580 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728400bce0 con 0x7f728c136f50 2023-09-30T22:32:28.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.596+0000 7f729157e700 1 -- 172.21.15.16:0/3669951580 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f728400be40 con 0x7f728c136f50 2023-09-30T22:32:28.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.596+0000 7f729157e700 1 -- 172.21.15.16:0/3669951580 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7284003c60 con 0x7f728c136f50 2023-09-30T22:32:28.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.596+0000 7f72947e4700 1 -- 172.21.15.16:0/3669951580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f728c136f50 msgr2=0x7f728c139330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:28.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.596+0000 7f72947e4700 1 --2- 172.21.15.16:0/3669951580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f728c136f50 0x7f728c139330 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f7284009d90 tx=0x7f728400b330 comp rx=0 tx=0).stop 2023-09-30T22:32:28.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.596+0000 7f72947e4700 1 -- 172.21.15.16:0/3669951580 shutdown_connections 2023-09-30T22:32:28.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.596+0000 7f72947e4700 1 --2- 172.21.15.16:0/3669951580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f728c136f50 0x7f728c139330 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:28.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.596+0000 7f72947e4700 1 --2- 172.21.15.16:0/3669951580 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f728c134630 0x7f728c136a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:28.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.596+0000 7f72947e4700 1 -- 172.21.15.16:0/3669951580 >> 172.21.15.16:0/3669951580 conn(0x7f728c0a2aa0 msgr2=0x7f728c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:28.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.597+0000 7f72947e4700 1 -- 172.21.15.16:0/3669951580 shutdown_connections 2023-09-30T22:32:28.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.597+0000 7f72947e4700 1 -- 172.21.15.16:0/3669951580 wait complete. 2023-09-30T22:32:28.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.598+0000 7f72947e4700 1 Processor -- start 2023-09-30T22:32:28.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.598+0000 7f72947e4700 1 -- start start 2023-09-30T22:32:28.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.598+0000 7f72947e4700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f728c131810 0x7f728c09c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:28.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.598+0000 7f72947e4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f728c134630 0x7f728c09c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:28.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.598+0000 7f72947e4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f728c136f50 0x7f728c09ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:28.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.598+0000 7f72947e4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f728c119750 con 0x7f728c136f50 2023-09-30T22:32:28.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.598+0000 7f72947e4700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f728c071840 con 0x7f728c131810 2023-09-30T22:32:28.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.598+0000 7f72947e4700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f728c06eb00 con 0x7f728c134630 2023-09-30T22:32:28.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.599+0000 7f7292580700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f728c131810 0x7f728c09c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:28.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.599+0000 7f7292580700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f728c131810 0x7f728c09c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:44704/0 (socket says 172.21.15.16:44704) 2023-09-30T22:32:28.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.599+0000 7f7292580700 1 -- 172.21.15.16:0/3724622948 learned_addr learned my addr 172.21.15.16:0/3724622948 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:28.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.599+0000 7f7292d81700 1 --2- 172.21.15.16:0/3724622948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f728c136f50 0x7f728c09ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:28.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.599+0000 7f7292580700 1 -- 172.21.15.16:0/3724622948 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f728c134630 msgr2=0x7f728c09c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:28.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.599+0000 7f7292580700 1 --2- 172.21.15.16:0/3724622948 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f728c134630 0x7f728c09c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:28.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.599+0000 7f7292580700 1 -- 172.21.15.16:0/3724622948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f728c136f50 msgr2=0x7f728c09ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:28.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.599+0000 7f7292580700 1 --2- 172.21.15.16:0/3724622948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f728c136f50 0x7f728c09ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:28.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.599+0000 7f7292580700 1 -- 172.21.15.16:0/3724622948 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f727c0094a0 con 0x7f728c131810 2023-09-30T22:32:28.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.599+0000 7f7292580700 1 --2- 172.21.15.16:0/3724622948 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f728c131810 0x7f728c09c3b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f728800c550 tx=0x7f728800c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:28.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.599+0000 7f7282ffd700 1 -- 172.21.15.16:0/3724622948 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7288004c80 con 0x7f728c131810 2023-09-30T22:32:28.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.599+0000 7f7282ffd700 1 -- 172.21.15.16:0/3724622948 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7288004e00 con 0x7f728c131810 2023-09-30T22:32:28.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.599+0000 7f72947e4700 1 -- 172.21.15.16:0/3724622948 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7284009a70 con 0x7f728c131810 2023-09-30T22:32:28.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.599+0000 7f7282ffd700 1 -- 172.21.15.16:0/3724622948 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7288007310 con 0x7f728c131810 2023-09-30T22:32:28.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.600+0000 7f72947e4700 1 -- 172.21.15.16:0/3724622948 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f728c11a130 con 0x7f728c131810 2023-09-30T22:32:28.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.601+0000 7f72947e4700 1 -- 172.21.15.16:0/3724622948 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f728c09e570 con 0x7f728c131810 2023-09-30T22:32:28.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.601+0000 7f7282ffd700 1 -- 172.21.15.16:0/3724622948 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7288007470 con 0x7f728c131810 2023-09-30T22:32:28.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.602+0000 7f7282ffd700 1 -- 172.21.15.16:0/3724622948 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f728803f320 con 0x7f728c131810 2023-09-30T22:32:28.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.607+0000 7f7282ffd700 1 -- 172.21.15.16:0/3724622948 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f72880133c0 con 0x7f728c131810 2023-09-30T22:32:28.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.634+0000 7f58f7961700 1 Processor -- start 2023-09-30T22:32:28.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.635+0000 7f58f7961700 1 -- start start 2023-09-30T22:32:28.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.635+0000 7f58f7961700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f58f009f410 0x7f58f0136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:28.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.635+0000 7f58f7961700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f58f0136fe0 0x7f58f01393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:28.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.635+0000 7f58f7961700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58f0070940 con 0x7f58f009e930 2023-09-30T22:32:28.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.635+0000 7f58f7961700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58f006fd60 con 0x7f58f009f410 2023-09-30T22:32:28.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.635+0000 7f58f7961700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f58f006fa80 con 0x7f58f0136fe0 2023-09-30T22:32:28.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.635+0000 7f58f4efc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f58f009f410 0x7f58f0136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:28.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.635+0000 7f58f56fd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f58f009e930 0x7f58f009ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38990/0 (socket says 172.21.15.16:38990) 2023-09-30T22:32:28.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.635+0000 7f58f4efc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f58f009f410 0x7f58f0136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56104/0 (socket says 172.21.15.16:56104) 2023-09-30T22:32:28.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.635+0000 7f58f56fd700 1 -- 172.21.15.16:0/3702538801 learned_addr learned my addr 172.21.15.16:0/3702538801 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:28.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.635+0000 7f58f5efe700 1 --2- 172.21.15.16:0/3702538801 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f58f0136fe0 0x7f58f01393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:28.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.636+0000 7f58f4efc700 1 -- 172.21.15.16:0/3702538801 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f58f0136fe0 msgr2=0x7f58f01393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:28.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.636+0000 7f58f4efc700 1 --2- 172.21.15.16:0/3702538801 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f58f0136fe0 0x7f58f01393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:28.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.636+0000 7f58f4efc700 1 -- 172.21.15.16:0/3702538801 >> v1:172.21.15.16:6789/0 conn(0x7f58f009e930 legacy=0x7f58f009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:28.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.636+0000 7f58f4efc700 1 -- 172.21.15.16:0/3702538801 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f009fb60 con 0x7f58f009f410 2023-09-30T22:32:28.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.636+0000 7f58f4efc700 1 --2- 172.21.15.16:0/3702538801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58f009f410 0x7f58f0136aa0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f58e0009840 tx=0x7f58e0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2158d8171a42e127 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:28.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.636+0000 7f58e7fff700 1 -- 172.21.15.16:0/3702538801 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e0011830 con 0x7f58f009f410 2023-09-30T22:32:28.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.636+0000 7f58e7fff700 1 -- 172.21.15.16:0/3702538801 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58e0011990 con 0x7f58f009f410 2023-09-30T22:32:28.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.636+0000 7f58e7fff700 1 -- 172.21.15.16:0/3702538801 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e0011d20 con 0x7f58f009f410 2023-09-30T22:32:28.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.637+0000 7f58f7961700 1 -- 172.21.15.16:0/3702538801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58f009f410 msgr2=0x7f58f0136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:28.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.637+0000 7f58f7961700 1 --2- 172.21.15.16:0/3702538801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58f009f410 0x7f58f0136aa0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f58e0009840 tx=0x7f58e0009b50 comp rx=0 tx=0).stop 2023-09-30T22:32:28.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.637+0000 7f58f7961700 1 -- 172.21.15.16:0/3702538801 shutdown_connections 2023-09-30T22:32:28.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.637+0000 7f58f7961700 1 --2- 172.21.15.16:0/3702538801 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f58f0136fe0 0x7f58f01393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:28.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.637+0000 7f58f7961700 1 --2- 172.21.15.16:0/3702538801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58f009f410 0x7f58f0136aa0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:28.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.637+0000 7f58f7961700 1 -- 172.21.15.16:0/3702538801 >> 172.21.15.16:0/3702538801 conn(0x7f58f00a4210 msgr2=0x7f58f00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:28.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.637+0000 7f58f7961700 1 -- 172.21.15.16:0/3702538801 shutdown_connections 2023-09-30T22:32:28.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.637+0000 7f58f7961700 1 -- 172.21.15.16:0/3702538801 wait complete. 2023-09-30T22:32:28.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.638+0000 7f58f7961700 1 Processor -- start 2023-09-30T22:32:28.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.638+0000 7f58f7961700 1 -- start start 2023-09-30T22:32:28.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.639+0000 7f58f7961700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f58f009e930 0x7f58f011f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:28.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.639+0000 7f58f7961700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f58f009f410 0x7f58f011fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:28.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.639+0000 7f58f7961700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58f0136fe0 0x7f58f0120400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:28.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.639+0000 7f58f7961700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f0071d20 con 0x7f58f0136fe0 2023-09-30T22:32:28.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.639+0000 7f58f7961700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58f01216a0 con 0x7f58f009e930 2023-09-30T22:32:28.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.639+0000 7f58f7961700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f58f01231c0 con 0x7f58f009f410 2023-09-30T22:32:28.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.639+0000 7f58f56fd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f58f009e930 0x7f58f011f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:28.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.639+0000 7f58f56fd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f58f009e930 0x7f58f011f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:44714/0 (socket says 172.21.15.16:44714) 2023-09-30T22:32:28.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.639+0000 7f58f56fd700 1 -- 172.21.15.16:0/2915334623 learned_addr learned my addr 172.21.15.16:0/2915334623 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:28.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.639+0000 7f58f5efe700 1 --2- 172.21.15.16:0/2915334623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58f0136fe0 0x7f58f0120400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:28.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.639+0000 7f58f4efc700 1 --2- 172.21.15.16:0/2915334623 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f58f009f410 0x7f58f011fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:28.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.639+0000 7f58f56fd700 1 -- 172.21.15.16:0/2915334623 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f58f009f410 msgr2=0x7f58f011fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:28.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.639+0000 7f58f56fd700 1 --2- 172.21.15.16:0/2915334623 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f58f009f410 0x7f58f011fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:28.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.639+0000 7f58f56fd700 1 -- 172.21.15.16:0/2915334623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58f0136fe0 msgr2=0x7f58f0120400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:28.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.639+0000 7f58f56fd700 1 --2- 172.21.15.16:0/2915334623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58f0136fe0 0x7f58f0120400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:28.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.639+0000 7f58f56fd700 1 -- 172.21.15.16:0/2915334623 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58e00094a0 con 0x7f58f009e930 2023-09-30T22:32:28.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.640+0000 7f58f56fd700 1 --2- 172.21.15.16:0/2915334623 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f58f009e930 0x7f58f011f980 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f58ec00c520 tx=0x7f58ec00c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:28.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.640+0000 7f58e5ffb700 1 -- 172.21.15.16:0/2915334623 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ec005d70 con 0x7f58f009e930 2023-09-30T22:32:28.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.640+0000 7f58e5ffb700 1 -- 172.21.15.16:0/2915334623 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58ec0070d0 con 0x7f58f009e930 2023-09-30T22:32:28.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.640+0000 7f58f7961700 1 -- 172.21.15.16:0/2915334623 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58f00a2f40 con 0x7f58f009e930 2023-09-30T22:32:28.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.640+0000 7f58e5ffb700 1 -- 172.21.15.16:0/2915334623 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ec007440 con 0x7f58f009e930 2023-09-30T22:32:28.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.640+0000 7f58f7961700 1 -- 172.21.15.16:0/2915334623 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58f01234a0 con 0x7f58f009e930 2023-09-30T22:32:28.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.641+0000 7f58f7961700 1 -- 172.21.15.16:0/2915334623 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58f0074ea0 con 0x7f58f009e930 2023-09-30T22:32:28.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.642+0000 7f58e5ffb700 1 -- 172.21.15.16:0/2915334623 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f58ec0075a0 con 0x7f58f009e930 2023-09-30T22:32:28.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.642+0000 7f58e5ffb700 1 -- 172.21.15.16:0/2915334623 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f58ec03eff0 con 0x7f58f009e930 2023-09-30T22:32:28.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.651+0000 7f58e5ffb700 1 -- 172.21.15.16:0/2915334623 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f58ec019b40 con 0x7f58f009e930 2023-09-30T22:32:28.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.692+0000 7fd84c5a0700 1 Processor -- start 2023-09-30T22:32:28.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.693+0000 7fd84c5a0700 1 -- start start 2023-09-30T22:32:28.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.694+0000 7fd84c5a0700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd84409d630 0x7fd8440a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:28.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.694+0000 7fd84c5a0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd844138d80 0x7fd84413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:28.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.694+0000 7fd84c5a0700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8440a2a70 con 0x7fd8440a0a30 2023-09-30T22:32:28.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.694+0000 7fd84c5a0700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8440a01d0 con 0x7fd84409d630 2023-09-30T22:32:28.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.694+0000 7fd84c5a0700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd84409e2b0 con 0x7fd844138d80 2023-09-30T22:32:28.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.694+0000 7fd84a33c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd8440a0a30 0x7fd8440a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38994/0 (socket says 172.21.15.16:38994) 2023-09-30T22:32:28.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.694+0000 7fd84a33c700 1 -- 172.21.15.16:0/3671547574 learned_addr learned my addr 172.21.15.16:0/3671547574 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:28.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.694+0000 7fd849b3b700 1 --2- 172.21.15.16:0/3671547574 >> v2:172.21.15.16:3300/0 conn(0x7fd84409d630 0x7fd8440a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:28.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.694+0000 7fd84ab3d700 1 --2- 172.21.15.16:0/3671547574 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd844138d80 0x7fd84413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:28.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.694+0000 7fd849b3b700 1 -- 172.21.15.16:0/3671547574 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd844138d80 msgr2=0x7fd84413b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:28.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.694+0000 7fd849b3b700 1 --2- 172.21.15.16:0/3671547574 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd844138d80 0x7fd84413b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:28.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.694+0000 7fd849b3b700 1 -- 172.21.15.16:0/3671547574 >> v1:172.21.15.16:6789/0 conn(0x7fd8440a0a30 legacy=0x7fd8440a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:28.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.694+0000 7fd849b3b700 1 -- 172.21.15.16:0/3671547574 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd84409dc50 con 0x7fd84409d630 2023-09-30T22:32:28.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.694+0000 7fd84ab3d700 1 --2- 172.21.15.16:0/3671547574 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd844138d80 0x7fd84413b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:28.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.695+0000 7fd849b3b700 1 --2- 172.21.15.16:0/3671547574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd84409d630 0x7fd8440a1510 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fd834009840 tx=0x7fd834009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5374f6effa2d8d94 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:28.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.695+0000 7fd84933a700 1 -- 172.21.15.16:0/3671547574 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd834011830 con 0x7fd84409d630 2023-09-30T22:32:28.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.695+0000 7fd84933a700 1 -- 172.21.15.16:0/3671547574 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd834011990 con 0x7fd84409d630 2023-09-30T22:32:28.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.695+0000 7fd84933a700 1 -- 172.21.15.16:0/3671547574 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd834011d20 con 0x7fd84409d630 2023-09-30T22:32:28.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.695+0000 7fd84c5a0700 1 -- 172.21.15.16:0/3671547574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd84409d630 msgr2=0x7fd8440a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:28.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.695+0000 7fd84c5a0700 1 --2- 172.21.15.16:0/3671547574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd84409d630 0x7fd8440a1510 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fd834009840 tx=0x7fd834009b50 comp rx=0 tx=0).stop 2023-09-30T22:32:28.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.696+0000 7fd84c5a0700 1 -- 172.21.15.16:0/3671547574 shutdown_connections 2023-09-30T22:32:28.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.696+0000 7fd84c5a0700 1 --2- 172.21.15.16:0/3671547574 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd844138d80 0x7fd84413b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:28.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.696+0000 7fd84c5a0700 1 --2- 172.21.15.16:0/3671547574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd84409d630 0x7fd8440a1510 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:28.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.696+0000 7fd84c5a0700 1 -- 172.21.15.16:0/3671547574 >> 172.21.15.16:0/3671547574 conn(0x7fd8440a3050 msgr2=0x7fd8440a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:28.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.696+0000 7fd84c5a0700 1 -- 172.21.15.16:0/3671547574 shutdown_connections 2023-09-30T22:32:28.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.696+0000 7fd84c5a0700 1 -- 172.21.15.16:0/3671547574 wait complete. 2023-09-30T22:32:28.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.697+0000 7fd84c5a0700 1 Processor -- start 2023-09-30T22:32:28.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.697+0000 7fd84c5a0700 1 -- start start 2023-09-30T22:32:28.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.697+0000 7fd84c5a0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd84409d630 0x7fd844120800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:28.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.697+0000 7fd84c5a0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd8440a0a30 0x7fd844120d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:28.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.697+0000 7fd84c5a0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd844138d80 0x7fd844121280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:28.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.697+0000 7fd84c5a0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd844123fb0 con 0x7fd84409d630 2023-09-30T22:32:28.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.697+0000 7fd84c5a0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8440a4e00 con 0x7fd844138d80 2023-09-30T22:32:28.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.697+0000 7fd84c5a0700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd8440a0410 con 0x7fd8440a0a30 2023-09-30T22:32:28.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.697+0000 7fd849b3b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd8440a0a30 0x7fd844120d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:28.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.697+0000 7fd84ab3d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd844138d80 0x7fd844121280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:28.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.697+0000 7fd849b3b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd8440a0a30 0x7fd844120d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:46288/0 (socket says 172.21.15.16:46288) 2023-09-30T22:32:28.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.697+0000 7fd849b3b700 1 -- 172.21.15.16:0/1254648664 learned_addr learned my addr 172.21.15.16:0/1254648664 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:28.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.698+0000 7fd84a33c700 1 --2- 172.21.15.16:0/1254648664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd84409d630 0x7fd844120800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:28.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.698+0000 7fd84ab3d700 1 -- 172.21.15.16:0/1254648664 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd8440a0a30 msgr2=0x7fd844120d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:28.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.698+0000 7fd84ab3d700 1 --2- 172.21.15.16:0/1254648664 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd8440a0a30 0x7fd844120d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:28.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.698+0000 7fd84ab3d700 1 -- 172.21.15.16:0/1254648664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd84409d630 msgr2=0x7fd844120800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:28.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.698+0000 7fd84ab3d700 1 --2- 172.21.15.16:0/1254648664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd84409d630 0x7fd844120800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:28.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.698+0000 7fd84ab3d700 1 -- 172.21.15.16:0/1254648664 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd83c009a70 con 0x7fd844138d80 2023-09-30T22:32:28.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.698+0000 7fd84a33c700 1 --2- 172.21.15.16:0/1254648664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd84409d630 0x7fd844120800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:32:28.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.698+0000 7fd84ab3d700 1 --2- 172.21.15.16:0/1254648664 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd844138d80 0x7fd844121280 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd83c00ef90 tx=0x7fd83c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:28.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.698+0000 7fd83affd700 1 -- 172.21.15.16:0/1254648664 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83c00cd40 con 0x7fd844138d80 2023-09-30T22:32:28.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.698+0000 7fd83affd700 1 -- 172.21.15.16:0/1254648664 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd83c00f040 con 0x7fd844138d80 2023-09-30T22:32:28.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.698+0000 7fd83affd700 1 -- 172.21.15.16:0/1254648664 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83c0146c0 con 0x7fd844138d80 2023-09-30T22:32:28.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.699+0000 7fd84c5a0700 1 -- 172.21.15.16:0/1254648664 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8340094a0 con 0x7fd844138d80 2023-09-30T22:32:28.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.699+0000 7fd84c5a0700 1 -- 172.21.15.16:0/1254648664 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd844124a20 con 0x7fd844138d80 2023-09-30T22:32:28.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.700+0000 7fd83affd700 1 -- 172.21.15.16:0/1254648664 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd83c014820 con 0x7fd844138d80 2023-09-30T22:32:28.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.700+0000 7fd83affd700 1 -- 172.21.15.16:0/1254648664 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd83c040350 con 0x7fd844138d80 2023-09-30T22:32:28.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.700+0000 7fd84c5a0700 1 -- 172.21.15.16:0/1254648664 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8440747f0 con 0x7fd844138d80 2023-09-30T22:32:28.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.705+0000 7fd83affd700 1 -- 172.21.15.16:0/1254648664 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd83c018080 con 0x7fd844138d80 2023-09-30T22:32:28.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.783+0000 7f07b0dd3700 1 -- 172.21.15.16:0/3380323445 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f07ac06c360 con 0x7f07ac09e410 2023-09-30T22:32:28.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.784+0000 7f079affd700 1 -- 172.21.15.16:0/3380323445 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f07a4008910 con 0x7f07ac09e410 2023-09-30T22:32:28.785 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:32:28.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.786+0000 7f07b0dd3700 1 -- 172.21.15.16:0/3380323445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07ac09e410 msgr2=0x7f07ac1c8170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:28.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.786+0000 7f07b0dd3700 1 --2- 172.21.15.16:0/3380323445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07ac09e410 0x7f07ac1c8170 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f07a400cf20 tx=0x7f07a400cf50 comp rx=0 tx=0).stop 2023-09-30T22:32:28.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.786+0000 7f07b0dd3700 1 -- 172.21.15.16:0/3380323445 shutdown_connections 2023-09-30T22:32:28.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.786+0000 7f07b0dd3700 1 --2- 172.21.15.16:0/3380323445 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f07ac09d6f0 0x7f07ac1c5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:28.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.786+0000 7f07b0dd3700 1 --2- 172.21.15.16:0/3380323445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07ac09e410 0x7f07ac1c8170 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:28.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.786+0000 7f07b0dd3700 1 --2- 172.21.15.16:0/3380323445 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f07ac09e000 0x7f07ac1c5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:28.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.786+0000 7f07b0dd3700 1 -- 172.21.15.16:0/3380323445 >> 172.21.15.16:0/3380323445 conn(0x7f07ac066f50 msgr2=0x7f07ac0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:28.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.786+0000 7f07b0dd3700 1 -- 172.21.15.16:0/3380323445 shutdown_connections 2023-09-30T22:32:28.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.786+0000 7f07b0dd3700 1 -- 172.21.15.16:0/3380323445 wait complete. 2023-09-30T22:32:28.806 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:32:28.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.825+0000 7f72947e4700 1 -- 172.21.15.16:0/3724622948 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f728c074ea0 con 0x7f728c131810 2023-09-30T22:32:28.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.825+0000 7f7282ffd700 1 -- 172.21.15.16:0/3724622948 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7288011360 con 0x7f728c131810 2023-09-30T22:32:28.827 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:32:28.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.828+0000 7f72947e4700 1 -- 172.21.15.16:0/3724622948 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f728c131810 msgr2=0x7f728c09c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:28.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.828+0000 7f72947e4700 1 --2- 172.21.15.16:0/3724622948 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f728c131810 0x7f728c09c3b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f728800c550 tx=0x7f728800c860 comp rx=0 tx=0).stop 2023-09-30T22:32:28.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.828+0000 7f72947e4700 1 -- 172.21.15.16:0/3724622948 shutdown_connections 2023-09-30T22:32:28.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.828+0000 7f72947e4700 1 --2- 172.21.15.16:0/3724622948 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f728c131810 0x7f728c09c3b0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:28.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.828+0000 7f72947e4700 1 --2- 172.21.15.16:0/3724622948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f728c136f50 0x7f728c09ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:28.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.828+0000 7f72947e4700 1 --2- 172.21.15.16:0/3724622948 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f728c134630 0x7f728c09c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:28.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.828+0000 7f72947e4700 1 -- 172.21.15.16:0/3724622948 >> 172.21.15.16:0/3724622948 conn(0x7f728c0a2aa0 msgr2=0x7f728c0a0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:28.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.828+0000 7f72947e4700 1 -- 172.21.15.16:0/3724622948 shutdown_connections 2023-09-30T22:32:28.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.828+0000 7f72947e4700 1 -- 172.21.15.16:0/3724622948 wait complete. 2023-09-30T22:32:28.843 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:32:28.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.897+0000 7f58f7961700 1 -- 172.21.15.16:0/2915334623 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f58f006b250 con 0x7f58f009e930 2023-09-30T22:32:28.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.897+0000 7f58e5ffb700 1 -- 172.21.15.16:0/2915334623 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f58ec019730 con 0x7f58f009e930 2023-09-30T22:32:28.899 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:32:28.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.901+0000 7f58f7961700 1 -- 172.21.15.16:0/2915334623 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f58f009e930 msgr2=0x7f58f011f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:28.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.901+0000 7f58f7961700 1 --2- 172.21.15.16:0/2915334623 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f58f009e930 0x7f58f011f980 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f58ec00c520 tx=0x7f58ec00c830 comp rx=0 tx=0).stop 2023-09-30T22:32:28.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.901+0000 7f58f7961700 1 -- 172.21.15.16:0/2915334623 shutdown_connections 2023-09-30T22:32:28.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.901+0000 7f58f7961700 1 --2- 172.21.15.16:0/2915334623 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f58f009e930 0x7f58f011f980 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:28.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.901+0000 7f58f7961700 1 --2- 172.21.15.16:0/2915334623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58f0136fe0 0x7f58f0120400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:28.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.901+0000 7f58f7961700 1 --2- 172.21.15.16:0/2915334623 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f58f009f410 0x7f58f011fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:28.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.901+0000 7f58f7961700 1 -- 172.21.15.16:0/2915334623 >> 172.21.15.16:0/2915334623 conn(0x7f58f00a4210 msgr2=0x7f58f006b880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:28.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.901+0000 7f58f7961700 1 -- 172.21.15.16:0/2915334623 shutdown_connections 2023-09-30T22:32:28.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.901+0000 7f58f7961700 1 -- 172.21.15.16:0/2915334623 wait complete. 2023-09-30T22:32:28.921 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:32:28.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.947+0000 7fd84c5a0700 1 -- 172.21.15.16:0/1254648664 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd844072980 con 0x7fd844138d80 2023-09-30T22:32:28.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.948+0000 7fd83affd700 1 -- 172.21.15.16:0/1254648664 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd83c007d90 con 0x7fd844138d80 2023-09-30T22:32:28.949 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:32:28.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.950+0000 7fd84c5a0700 1 -- 172.21.15.16:0/1254648664 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd844138d80 msgr2=0x7fd844121280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:28.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.950+0000 7fd84c5a0700 1 --2- 172.21.15.16:0/1254648664 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd844138d80 0x7fd844121280 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd83c00ef90 tx=0x7fd83c00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:32:28.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.950+0000 7fd84c5a0700 1 -- 172.21.15.16:0/1254648664 shutdown_connections 2023-09-30T22:32:28.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.950+0000 7fd84c5a0700 1 --2- 172.21.15.16:0/1254648664 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd844138d80 0x7fd844121280 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:28.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.950+0000 7fd84c5a0700 1 --2- 172.21.15.16:0/1254648664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd84409d630 0x7fd844120800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:28.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.950+0000 7fd84c5a0700 1 --2- 172.21.15.16:0/1254648664 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd8440a0a30 0x7fd844120d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:28.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.950+0000 7fd84c5a0700 1 -- 172.21.15.16:0/1254648664 >> 172.21.15.16:0/1254648664 conn(0x7fd8440a3050 msgr2=0x7fd84409ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:28.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.950+0000 7fd84c5a0700 1 -- 172.21.15.16:0/1254648664 shutdown_connections 2023-09-30T22:32:28.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:28.950+0000 7fd84c5a0700 1 -- 172.21.15.16:0/1254648664 wait complete. 2023-09-30T22:32:28.965 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:32:29.807 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:32:29.844 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:32:29.922 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:32:29.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:29.961+0000 7f769a1d2700 1 Processor -- start 2023-09-30T22:32:29.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:29.961+0000 7f769a1d2700 1 -- start start 2023-09-30T22:32:29.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:29.962+0000 7f769a1d2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f769409d6f0 0x7f769409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:29.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:29.962+0000 7f769a1d2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f769409e000 0x7f76941339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:29.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:29.962+0000 7f769a1d2700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f769409e410 0x7f7694133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:29.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:29.962+0000 7f769a1d2700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f769406b460 con 0x7f769409e410 2023-09-30T22:32:29.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:29.962+0000 7f769a1d2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f769409b120 con 0x7f769409d6f0 2023-09-30T22:32:29.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:29.962+0000 7f769a1d2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f769409a7e0 con 0x7f769409e000 2023-09-30T22:32:29.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:29.962+0000 7f76937fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f769409d6f0 0x7f769409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:29.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:29.962+0000 7f76937fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f769409d6f0 0x7f769409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:44736/0 (socket says 172.21.15.16:44736) 2023-09-30T22:32:29.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:29.962+0000 7f76937fe700 1 -- 172.21.15.16:0/3617162844 learned_addr learned my addr 172.21.15.16:0/3617162844 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:29.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:29.962+0000 7f768affd700 1 --2- 172.21.15.16:0/3617162844 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f769409e000 0x7f76941339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:29.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:29.963+0000 7f76937fe700 1 -- 172.21.15.16:0/3617162844 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f769409e000 msgr2=0x7f76941339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:29.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:29.963+0000 7f76937fe700 1 --2- 172.21.15.16:0/3617162844 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f769409e000 0x7f76941339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:29.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:29.963+0000 7f76937fe700 1 -- 172.21.15.16:0/3617162844 >> v2:172.21.15.16:3300/0 conn(0x7f769409e410 msgr2=0x7f7694133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:29.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:29.963+0000 7f76937fe700 1 --2- 172.21.15.16:0/3617162844 >> v2:172.21.15.16:3300/0 conn(0x7f769409e410 0x7f7694133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:29.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:29.963+0000 7f76937fe700 1 -- 172.21.15.16:0/3617162844 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f769409a500 con 0x7f769409d6f0 2023-09-30T22:32:29.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:29.963+0000 7f76937fe700 1 --2- 172.21.15.16:0/3617162844 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f769409d6f0 0x7f769409dac0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7684009920 tx=0x7f7684009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=324a1d66f678eba8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:29.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:29.963+0000 7f7692ffd700 1 -- 172.21.15.16:0/3617162844 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7684011830 con 0x7f769409d6f0 2023-09-30T22:32:29.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:29.963+0000 7f7692ffd700 1 -- 172.21.15.16:0/3617162844 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7684011990 con 0x7f769409d6f0 2023-09-30T22:32:29.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:29.963+0000 7f7692ffd700 1 -- 172.21.15.16:0/3617162844 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7684011d20 con 0x7f769409d6f0 2023-09-30T22:32:29.965 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:32:29.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:29.964+0000 7f769a1d2700 1 -- 172.21.15.16:0/3617162844 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f769409d6f0 msgr2=0x7f769409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:29.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:29.964+0000 7f769a1d2700 1 --2- 172.21.15.16:0/3617162844 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f769409d6f0 0x7f769409dac0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7684009920 tx=0x7f7684009c30 comp rx=0 tx=0).stop 2023-09-30T22:32:29.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:29.964+0000 7f769a1d2700 1 -- 172.21.15.16:0/3617162844 shutdown_connections 2023-09-30T22:32:29.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:29.964+0000 7f769a1d2700 1 --2- 172.21.15.16:0/3617162844 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f769409d6f0 0x7f769409dac0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:29.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:29.964+0000 7f769a1d2700 1 --2- 172.21.15.16:0/3617162844 >> v2:172.21.15.16:3300/0 conn(0x7f769409e410 0x7f7694133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:29.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:29.964+0000 7f769a1d2700 1 --2- 172.21.15.16:0/3617162844 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f769409e000 0x7f76941339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:29.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:29.964+0000 7f769a1d2700 1 -- 172.21.15.16:0/3617162844 >> 172.21.15.16:0/3617162844 conn(0x7f7694066f50 msgr2=0x7f769409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:29.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:29.964+0000 7f769a1d2700 1 -- 172.21.15.16:0/3617162844 shutdown_connections 2023-09-30T22:32:29.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:29.964+0000 7f769a1d2700 1 -- 172.21.15.16:0/3617162844 wait complete. 2023-09-30T22:32:29.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:29.965+0000 7f769a1d2700 1 Processor -- start 2023-09-30T22:32:29.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:29.965+0000 7f769a1d2700 1 -- start start 2023-09-30T22:32:29.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:29.966+0000 7f769a1d2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f769409d6f0 0x7f76941a5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:29.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:29.966+0000 7f769a1d2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f769409e000 0x7f76941a6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:29.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:29.966+0000 7f769a1d2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f769409e410 0x7f76941a6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:29.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:29.966+0000 7f769a1d2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76941a8cd0 con 0x7f769409e410 2023-09-30T22:32:29.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:29.966+0000 7f769a1d2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76941a72b0 con 0x7f769409e000 2023-09-30T22:32:29.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:29.966+0000 7f769a1d2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f76941a9720 con 0x7f769409d6f0 2023-09-30T22:32:29.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:29.966+0000 7f76937fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f769409d6f0 0x7f76941a5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:29.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:29.966+0000 7f76937fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f769409d6f0 0x7f76941a5e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:46302/0 (socket says 172.21.15.16:46302) 2023-09-30T22:32:29.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:29.966+0000 7f76937fe700 1 -- 172.21.15.16:0/1896841073 learned_addr learned my addr 172.21.15.16:0/1896841073 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:29.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:29.966+0000 7f7693fff700 1 --2- 172.21.15.16:0/1896841073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f769409e410 0x7f76941a6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:29.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:29.966+0000 7f768affd700 1 --2- 172.21.15.16:0/1896841073 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f769409e000 0x7f76941a6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:29.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:29.966+0000 7f7693fff700 1 -- 172.21.15.16:0/1896841073 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f769409d6f0 msgr2=0x7f76941a5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:29.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:29.966+0000 7f7693fff700 1 --2- 172.21.15.16:0/1896841073 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f769409d6f0 0x7f76941a5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:29.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:29.966+0000 7f7693fff700 1 -- 172.21.15.16:0/1896841073 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f769409e000 msgr2=0x7f76941a6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:29.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:29.966+0000 7f7693fff700 1 --2- 172.21.15.16:0/1896841073 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f769409e000 0x7f76941a6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:29.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:29.966+0000 7f7693fff700 1 -- 172.21.15.16:0/1896841073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7684009550 con 0x7f769409e410 2023-09-30T22:32:29.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:29.966+0000 7f76937fe700 1 --2- 172.21.15.16:0/1896841073 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f769409d6f0 0x7f76941a5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:29.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:29.966+0000 7f7693fff700 1 --2- 172.21.15.16:0/1896841073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f769409e410 0x7f76941a6880 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f768c00bdf0 tx=0x7f768c009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:29.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:29.967+0000 7f7690ff9700 1 -- 172.21.15.16:0/1896841073 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768c00cb30 con 0x7f769409e410 2023-09-30T22:32:29.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:29.967+0000 7f7690ff9700 1 -- 172.21.15.16:0/1896841073 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f768c00cc90 con 0x7f769409e410 2023-09-30T22:32:29.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:29.967+0000 7f7690ff9700 1 -- 172.21.15.16:0/1896841073 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768c014440 con 0x7f769409e410 2023-09-30T22:32:29.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:29.967+0000 7f769a1d2700 1 -- 172.21.15.16:0/1896841073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76941a8810 con 0x7f769409e410 2023-09-30T22:32:29.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:29.967+0000 7f769a1d2700 1 -- 172.21.15.16:0/1896841073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76941a7b00 con 0x7f769409e410 2023-09-30T22:32:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:29.968+0000 7f7690ff9700 1 -- 172.21.15.16:0/1896841073 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f768c01d3b0 con 0x7f769409e410 2023-09-30T22:32:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:29.968+0000 7f7690ff9700 1 -- 172.21.15.16:0/1896841073 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f768c006e40 con 0x7f769409e410 2023-09-30T22:32:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:29.968+0000 7f769a1d2700 1 -- 172.21.15.16:0/1896841073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7694074ea0 con 0x7f769409e410 2023-09-30T22:32:29.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:29.974+0000 7f7690ff9700 1 -- 172.21.15.16:0/1896841073 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f768c01c360 con 0x7f769409e410 2023-09-30T22:32:30.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.000+0000 7ff705b3f700 1 Processor -- start 2023-09-30T22:32:30.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.000+0000 7ff705b3f700 1 -- start start 2023-09-30T22:32:30.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.000+0000 7ff705b3f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff70006e570 0x7ff70006e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:30.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.000+0000 7ff705b3f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff70009eca0 0x7ff70006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:30.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.001+0000 7ff705b3f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff700134610 0x7ff700134ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:30.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.001+0000 7ff705b3f700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff700070ee0 con 0x7ff700134610 2023-09-30T22:32:30.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.001+0000 7ff705b3f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff700071480 con 0x7ff70009eca0 2023-09-30T22:32:30.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.001+0000 7ff705b3f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff70006f3c0 con 0x7ff70006e570 2023-09-30T22:32:30.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.001+0000 7ff6fffff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff700134610 0x7ff700134ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:30.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.001+0000 7ff6feffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff70009eca0 0x7ff70006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:30.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.001+0000 7ff6fffff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff700134610 0x7ff700134ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56144/0 (socket says 172.21.15.16:56144) 2023-09-30T22:32:30.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.001+0000 7ff6fffff700 1 -- 172.21.15.16:0/3401656287 learned_addr learned my addr 172.21.15.16:0/3401656287 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:30.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.001+0000 7ff6ff7fe700 1 --2- 172.21.15.16:0/3401656287 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff70006e570 0x7ff70006e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:30.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.001+0000 7ff6feffd700 1 -- 172.21.15.16:0/3401656287 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff70006e570 msgr2=0x7ff70006e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:30.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.001+0000 7ff6feffd700 1 --2- 172.21.15.16:0/3401656287 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff70006e570 0x7ff70006e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:30.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.001+0000 7ff6feffd700 1 -- 172.21.15.16:0/3401656287 >> v2:172.21.15.16:3300/0 conn(0x7ff700134610 msgr2=0x7ff700134ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:30.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.001+0000 7ff6feffd700 1 --2- 172.21.15.16:0/3401656287 >> v2:172.21.15.16:3300/0 conn(0x7ff700134610 0x7ff700134ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:30.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.001+0000 7ff6feffd700 1 -- 172.21.15.16:0/3401656287 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff700071b90 con 0x7ff70009eca0 2023-09-30T22:32:30.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.002+0000 7ff6feffd700 1 --2- 172.21.15.16:0/3401656287 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff70009eca0 0x7ff70006ee80 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff6f4009840 tx=0x7ff6f4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=441646a41497a3d8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:30.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.002+0000 7ff6fe7fc700 1 -- 172.21.15.16:0/3401656287 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f4011830 con 0x7ff70009eca0 2023-09-30T22:32:30.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.002+0000 7ff6fe7fc700 1 -- 172.21.15.16:0/3401656287 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6f4011990 con 0x7ff70009eca0 2023-09-30T22:32:30.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.002+0000 7ff6fe7fc700 1 -- 172.21.15.16:0/3401656287 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f4011d20 con 0x7ff70009eca0 2023-09-30T22:32:30.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.003+0000 7ff705b3f700 1 -- 172.21.15.16:0/3401656287 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff70009eca0 msgr2=0x7ff70006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:30.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.003+0000 7ff705b3f700 1 --2- 172.21.15.16:0/3401656287 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff70009eca0 0x7ff70006ee80 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff6f4009840 tx=0x7ff6f4009b50 comp rx=0 tx=0).stop 2023-09-30T22:32:30.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.003+0000 7ff705b3f700 1 -- 172.21.15.16:0/3401656287 shutdown_connections 2023-09-30T22:32:30.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.003+0000 7ff705b3f700 1 --2- 172.21.15.16:0/3401656287 >> v2:172.21.15.16:3300/0 conn(0x7ff700134610 0x7ff700134ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:30.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.003+0000 7ff705b3f700 1 --2- 172.21.15.16:0/3401656287 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff70006e570 0x7ff70006e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:30.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.003+0000 7ff705b3f700 1 --2- 172.21.15.16:0/3401656287 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff70009eca0 0x7ff70006ee80 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:30.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.003+0000 7ff705b3f700 1 -- 172.21.15.16:0/3401656287 >> 172.21.15.16:0/3401656287 conn(0x7ff7000a2aa0 msgr2=0x7ff7000a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:30.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.003+0000 7ff705b3f700 1 -- 172.21.15.16:0/3401656287 shutdown_connections 2023-09-30T22:32:30.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.003+0000 7ff705b3f700 1 -- 172.21.15.16:0/3401656287 wait complete. 2023-09-30T22:32:30.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.004+0000 7ff705b3f700 1 Processor -- start 2023-09-30T22:32:30.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.004+0000 7ff705b3f700 1 -- start start 2023-09-30T22:32:30.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.005+0000 7ff705b3f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff70006e570 0x7ff700127470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:30.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.005+0000 7ff705b3f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff700134610 0x7ff70009be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:30.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.005+0000 7ff705b3f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7001279b0 0x7ff70009c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:30.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.005+0000 7ff705b3f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff70009d1e0 con 0x7ff7001279b0 2023-09-30T22:32:30.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.005+0000 7ff705b3f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff70009dad0 con 0x7ff70006e570 2023-09-30T22:32:30.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.005+0000 7ff705b3f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff700138fa0 con 0x7ff700134610 2023-09-30T22:32:30.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.005+0000 7ff6ff7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff70006e570 0x7ff700127470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:30.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.005+0000 7ff6ff7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff70006e570 0x7ff700127470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:44766/0 (socket says 172.21.15.16:44766) 2023-09-30T22:32:30.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.005+0000 7ff6ff7fe700 1 -- 172.21.15.16:0/1482950933 learned_addr learned my addr 172.21.15.16:0/1482950933 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:30.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.005+0000 7ff6fffff700 1 --2- 172.21.15.16:0/1482950933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7001279b0 0x7ff70009c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:30.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.005+0000 7ff6feffd700 1 --2- 172.21.15.16:0/1482950933 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff700134610 0x7ff70009be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:30.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.006+0000 7ff6feffd700 1 -- 172.21.15.16:0/1482950933 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff70006e570 msgr2=0x7ff700127470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:30.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.006+0000 7ff6feffd700 1 --2- 172.21.15.16:0/1482950933 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff70006e570 0x7ff700127470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:30.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.006+0000 7ff6feffd700 1 -- 172.21.15.16:0/1482950933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7001279b0 msgr2=0x7ff70009c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:30.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.006+0000 7ff6feffd700 1 --2- 172.21.15.16:0/1482950933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7001279b0 0x7ff70009c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:30.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.006+0000 7ff6feffd700 1 -- 172.21.15.16:0/1482950933 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f40094a0 con 0x7ff700134610 2023-09-30T22:32:30.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.006+0000 7ff6ff7fe700 1 --2- 172.21.15.16:0/1482950933 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff70006e570 0x7ff700127470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:30.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.006+0000 7ff6feffd700 1 --2- 172.21.15.16:0/1482950933 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff700134610 0x7ff70009be10 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff6f4016000 tx=0x7ff6f4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:30.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.006+0000 7ff6e7fff700 1 -- 172.21.15.16:0/1482950933 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f4004450 con 0x7ff700134610 2023-09-30T22:32:30.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.006+0000 7ff6e7fff700 1 -- 172.21.15.16:0/1482950933 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6f401f070 con 0x7ff700134610 2023-09-30T22:32:30.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.006+0000 7ff705b3f700 1 -- 172.21.15.16:0/1482950933 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff700139260 con 0x7ff700134610 2023-09-30T22:32:30.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.006+0000 7ff6e7fff700 1 -- 172.21.15.16:0/1482950933 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f40038e0 con 0x7ff700134610 2023-09-30T22:32:30.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.006+0000 7ff705b3f700 1 -- 172.21.15.16:0/1482950933 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff70009d790 con 0x7ff700134610 2023-09-30T22:32:30.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.007+0000 7ff705b3f700 1 -- 172.21.15.16:0/1482950933 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7001394a0 con 0x7ff700134610 2023-09-30T22:32:30.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.008+0000 7ff6e7fff700 1 -- 172.21.15.16:0/1482950933 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff6f4003a40 con 0x7ff700134610 2023-09-30T22:32:30.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.009+0000 7ff6e7fff700 1 -- 172.21.15.16:0/1482950933 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7ff6f4021070 con 0x7ff700134610 2023-09-30T22:32:30.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.013+0000 7ff6e7fff700 1 -- 172.21.15.16:0/1482950933 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff6f4010d50 con 0x7ff700134610 2023-09-30T22:32:30.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.072+0000 7f32960ea700 1 Processor -- start 2023-09-30T22:32:30.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.072+0000 7f32960ea700 1 -- start start 2023-09-30T22:32:30.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.073+0000 7f32960ea700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f329009e1d0 0x7f3290138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:30.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.073+0000 7f32960ea700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3290139170 0x7f329013b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:30.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.073+0000 7f32960ea700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f329009b120 con 0x7f329009d6f0 2023-09-30T22:32:30.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.073+0000 7f32960ea700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f329006b460 con 0x7f329009e1d0 2023-09-30T22:32:30.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.073+0000 7f32960ea700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f329009a7e0 con 0x7f3290139170 2023-09-30T22:32:30.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.073+0000 7f328f7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f329009d6f0 0x7f329009dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38996/0 (socket says 172.21.15.16:38996) 2023-09-30T22:32:30.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.073+0000 7f328effd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f329009e1d0 0x7f3290138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:30.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.073+0000 7f328f7fe700 1 -- 172.21.15.16:0/3175491345 learned_addr learned my addr 172.21.15.16:0/3175491345 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:30.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.073+0000 7f328ffff700 1 --2- 172.21.15.16:0/3175491345 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3290139170 0x7f329013b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:30.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.074+0000 7f328effd700 1 -- 172.21.15.16:0/3175491345 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3290139170 msgr2=0x7f329013b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:30.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.074+0000 7f328effd700 1 --2- 172.21.15.16:0/3175491345 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3290139170 0x7f329013b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:30.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.074+0000 7f328effd700 1 -- 172.21.15.16:0/3175491345 >> v1:172.21.15.16:6789/0 conn(0x7f329009d6f0 legacy=0x7f329009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:30.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.074+0000 7f328effd700 1 -- 172.21.15.16:0/3175491345 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f329009a500 con 0x7f329009e1d0 2023-09-30T22:32:30.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.074+0000 7f328e7fc700 1 -- 172.21.15.16:0/3175491345 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 317332801 0 0) 0x7f329009b120 con 0x7f329009d6f0 2023-09-30T22:32:30.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.074+0000 7f328effd700 1 --2- 172.21.15.16:0/3175491345 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f329009e1d0 0x7f3290138c30 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3284009870 tx=0x7f3284009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccf3bbbd77899fff server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:30.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.074+0000 7f328e7fc700 1 -- 172.21.15.16:0/3175491345 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3284011830 con 0x7f329009e1d0 2023-09-30T22:32:30.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.074+0000 7f328e7fc700 1 -- 172.21.15.16:0/3175491345 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3284011990 con 0x7f329009e1d0 2023-09-30T22:32:30.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.074+0000 7f328e7fc700 1 -- 172.21.15.16:0/3175491345 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3284011d20 con 0x7f329009e1d0 2023-09-30T22:32:30.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.075+0000 7f32960ea700 1 -- 172.21.15.16:0/3175491345 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f329009e1d0 msgr2=0x7f3290138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:30.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.075+0000 7f32960ea700 1 --2- 172.21.15.16:0/3175491345 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f329009e1d0 0x7f3290138c30 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f3284009870 tx=0x7f3284009b80 comp rx=0 tx=0).stop 2023-09-30T22:32:30.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.075+0000 7f32960ea700 1 -- 172.21.15.16:0/3175491345 shutdown_connections 2023-09-30T22:32:30.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.075+0000 7f32960ea700 1 --2- 172.21.15.16:0/3175491345 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3290139170 0x7f329013b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:30.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.075+0000 7f32960ea700 1 --2- 172.21.15.16:0/3175491345 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f329009e1d0 0x7f3290138c30 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:30.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.075+0000 7f32960ea700 1 -- 172.21.15.16:0/3175491345 >> 172.21.15.16:0/3175491345 conn(0x7f3290066f50 msgr2=0x7f329009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:30.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.075+0000 7f32960ea700 1 -- 172.21.15.16:0/3175491345 shutdown_connections 2023-09-30T22:32:30.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.075+0000 7f32960ea700 1 -- 172.21.15.16:0/3175491345 wait complete. 2023-09-30T22:32:30.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.076+0000 7f32960ea700 1 Processor -- start 2023-09-30T22:32:30.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.076+0000 7f32960ea700 1 -- start start 2023-09-30T22:32:30.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.077+0000 7f32960ea700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f329009d6f0 0x7f329011e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:30.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.077+0000 7f32960ea700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f329009e1d0 0x7f329011ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:30.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.077+0000 7f32960ea700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3290139170 0x7f329011f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:30.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.077+0000 7f32960ea700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32901220b0 con 0x7f3290139170 2023-09-30T22:32:30.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.077+0000 7f32960ea700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f329011f9f0 con 0x7f329009e1d0 2023-09-30T22:32:30.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.077+0000 7f32960ea700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3290121460 con 0x7f329009d6f0 2023-09-30T22:32:30.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.077+0000 7f328f7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f329009d6f0 0x7f329011e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:30.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.077+0000 7f328ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3290139170 0x7f329011f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:30.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.077+0000 7f328ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3290139170 0x7f329011f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56164/0 (socket says 172.21.15.16:56164) 2023-09-30T22:32:30.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.077+0000 7f328f7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f329009d6f0 0x7f329011e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:46332/0 (socket says 172.21.15.16:46332) 2023-09-30T22:32:30.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.077+0000 7f328ffff700 1 -- 172.21.15.16:0/3070405322 learned_addr learned my addr 172.21.15.16:0/3070405322 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:30.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.077+0000 7f328effd700 1 --2- 172.21.15.16:0/3070405322 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f329009e1d0 0x7f329011ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:30.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.077+0000 7f328ffff700 1 -- 172.21.15.16:0/3070405322 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f329009d6f0 msgr2=0x7f329011e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:30.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.077+0000 7f328ffff700 1 --2- 172.21.15.16:0/3070405322 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f329009d6f0 0x7f329011e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:30.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.077+0000 7f328ffff700 1 -- 172.21.15.16:0/3070405322 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f329009e1d0 msgr2=0x7f329011ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:30.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.077+0000 7f328ffff700 1 --2- 172.21.15.16:0/3070405322 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f329009e1d0 0x7f329011ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:30.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.078+0000 7f328ffff700 1 -- 172.21.15.16:0/3070405322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32840094a0 con 0x7f3290139170 2023-09-30T22:32:30.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.078+0000 7f328f7fe700 1 --2- 172.21.15.16:0/3070405322 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f329009d6f0 0x7f329011e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:32:30.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.078+0000 7f328ffff700 1 --2- 172.21.15.16:0/3070405322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3290139170 0x7f329011f010 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f328800efc0 tx=0x7f328800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:30.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.078+0000 7f3277fff700 1 -- 172.21.15.16:0/3070405322 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3288009ea0 con 0x7f3290139170 2023-09-30T22:32:30.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.078+0000 7f3277fff700 1 -- 172.21.15.16:0/3070405322 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f328800f040 con 0x7f3290139170 2023-09-30T22:32:30.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.078+0000 7f3277fff700 1 -- 172.21.15.16:0/3070405322 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3288014480 con 0x7f3290139170 2023-09-30T22:32:30.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.078+0000 7f32960ea700 1 -- 172.21.15.16:0/3070405322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32900a3a20 con 0x7f3290139170 2023-09-30T22:32:30.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.078+0000 7f32960ea700 1 -- 172.21.15.16:0/3070405322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3290122840 con 0x7f3290139170 2023-09-30T22:32:30.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.079+0000 7f3277fff700 1 -- 172.21.15.16:0/3070405322 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f32880145e0 con 0x7f3290139170 2023-09-30T22:32:30.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.079+0000 7f32960ea700 1 -- 172.21.15.16:0/3070405322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3290074ea0 con 0x7f3290139170 2023-09-30T22:32:30.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.080+0000 7f3277fff700 1 -- 172.21.15.16:0/3070405322 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f328803ff90 con 0x7f3290139170 2023-09-30T22:32:30.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.085+0000 7f3277fff700 1 -- 172.21.15.16:0/3070405322 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f328801a8b0 con 0x7f3290139170 2023-09-30T22:32:30.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.113+0000 7f4f956d3700 1 Processor -- start 2023-09-30T22:32:30.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.113+0000 7f4f956d3700 1 -- start start 2023-09-30T22:32:30.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.114+0000 7f4f956d3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4f9009e930 0x7f4f9009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:30.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.114+0000 7f4f956d3700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4f9009f240 0x7f4f901317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:30.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.114+0000 7f4f956d3700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f90070940 con 0x7f4f9009f650 2023-09-30T22:32:30.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.114+0000 7f4f956d3700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f9006fd60 con 0x7f4f9009f240 2023-09-30T22:32:30.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.114+0000 7f4f956d3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f9006fa80 con 0x7f4f9009e930 2023-09-30T22:32:30.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.114+0000 7f4f8effd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4f9009e930 0x7f4f9009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:30.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.114+0000 7f4f8effd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4f9009e930 0x7f4f9009ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:44808/0 (socket says 172.21.15.16:44808) 2023-09-30T22:32:30.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.114+0000 7f4f8effd700 1 -- 172.21.15.16:0/2109159204 learned_addr learned my addr 172.21.15.16:0/2109159204 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:30.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.114+0000 7f4f8e7fc700 1 --2- 172.21.15.16:0/2109159204 >> v2:172.21.15.16:3300/0 conn(0x7f4f9009f240 0x7f4f901317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:30.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.114+0000 7f4f8effd700 1 -- 172.21.15.16:0/2109159204 >> v2:172.21.15.16:3300/0 conn(0x7f4f9009f240 msgr2=0x7f4f901317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:30.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.114+0000 7f4f8effd700 1 --2- 172.21.15.16:0/2109159204 >> v2:172.21.15.16:3300/0 conn(0x7f4f9009f240 0x7f4f901317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:30.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.114+0000 7f4f8effd700 1 -- 172.21.15.16:0/2109159204 >> v1:172.21.15.16:6789/0 conn(0x7f4f9009f650 legacy=0x7f4f90131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:30.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.114+0000 7f4f8effd700 1 -- 172.21.15.16:0/2109159204 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f9009fb60 con 0x7f4f9009e930 2023-09-30T22:32:30.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.115+0000 7f4f8effd700 1 --2- 172.21.15.16:0/2109159204 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4f9009e930 0x7f4f9009ed00 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f4f800098f0 tx=0x7f4f80009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53787b91c810ca6c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:30.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.115+0000 7f4f8dffb700 1 -- 172.21.15.16:0/2109159204 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f80004230 con 0x7f4f9009e930 2023-09-30T22:32:30.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.115+0000 7f4f8dffb700 1 -- 172.21.15.16:0/2109159204 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f80004390 con 0x7f4f9009e930 2023-09-30T22:32:30.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.115+0000 7f4f8dffb700 1 -- 172.21.15.16:0/2109159204 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f800036a0 con 0x7f4f9009e930 2023-09-30T22:32:30.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.116+0000 7f4f956d3700 1 -- 172.21.15.16:0/2109159204 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4f9009e930 msgr2=0x7f4f9009ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:30.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.116+0000 7f4f956d3700 1 --2- 172.21.15.16:0/2109159204 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4f9009e930 0x7f4f9009ed00 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f4f800098f0 tx=0x7f4f80009c00 comp rx=0 tx=0).stop 2023-09-30T22:32:30.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.116+0000 7f4f956d3700 1 -- 172.21.15.16:0/2109159204 shutdown_connections 2023-09-30T22:32:30.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.116+0000 7f4f956d3700 1 --2- 172.21.15.16:0/2109159204 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4f9009e930 0x7f4f9009ed00 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:30.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.116+0000 7f4f956d3700 1 --2- 172.21.15.16:0/2109159204 >> v2:172.21.15.16:3300/0 conn(0x7f4f9009f240 0x7f4f901317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:30.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.116+0000 7f4f956d3700 1 -- 172.21.15.16:0/2109159204 >> 172.21.15.16:0/2109159204 conn(0x7f4f900a4210 msgr2=0x7f4f900a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:30.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.116+0000 7f4f956d3700 1 -- 172.21.15.16:0/2109159204 shutdown_connections 2023-09-30T22:32:30.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.116+0000 7f4f956d3700 1 -- 172.21.15.16:0/2109159204 wait complete. 2023-09-30T22:32:30.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.117+0000 7f4f956d3700 1 Processor -- start 2023-09-30T22:32:30.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.117+0000 7f4f956d3700 1 -- start start 2023-09-30T22:32:30.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.118+0000 7f4f956d3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4f9009e930 0x7f4f9006b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:30.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.118+0000 7f4f956d3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f9009f240 0x7f4f9006bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:30.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.118+0000 7f4f956d3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4f9009f650 0x7f4f9006c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:30.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.118+0000 7f4f956d3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f901b7200 con 0x7f4f9009f240 2023-09-30T22:32:30.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.118+0000 7f4f956d3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f901a93b0 con 0x7f4f9009f650 2023-09-30T22:32:30.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.118+0000 7f4f956d3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4f901a88d0 con 0x7f4f9009e930 2023-09-30T22:32:30.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.118+0000 7f4f8effd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4f9009e930 0x7f4f9006b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:30.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.118+0000 7f4f8effd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4f9009e930 0x7f4f9006b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:46338/0 (socket says 172.21.15.16:46338) 2023-09-30T22:32:30.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.118+0000 7f4f8effd700 1 -- 172.21.15.16:0/3505174024 learned_addr learned my addr 172.21.15.16:0/3505174024 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:30.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.118+0000 7f4f8f7fe700 1 --2- 172.21.15.16:0/3505174024 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4f9009f650 0x7f4f9006c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:30.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.118+0000 7f4f8e7fc700 1 --2- 172.21.15.16:0/3505174024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f9009f240 0x7f4f9006bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:30.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.119+0000 7f4f8effd700 1 -- 172.21.15.16:0/3505174024 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4f9009f650 msgr2=0x7f4f9006c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:30.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.119+0000 7f4f8effd700 1 --2- 172.21.15.16:0/3505174024 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4f9009f650 0x7f4f9006c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:30.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.119+0000 7f4f8effd700 1 -- 172.21.15.16:0/3505174024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f9009f240 msgr2=0x7f4f9006bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:30.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.119+0000 7f4f8effd700 1 --2- 172.21.15.16:0/3505174024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f9009f240 0x7f4f9006bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:30.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.119+0000 7f4f8effd700 1 -- 172.21.15.16:0/3505174024 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f80009550 con 0x7f4f9009e930 2023-09-30T22:32:30.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.119+0000 7f4f8effd700 1 --2- 172.21.15.16:0/3505174024 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4f9009e930 0x7f4f9006b8b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4f80009130 tx=0x7f4f80004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:30.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.119+0000 7f4f777fe700 1 -- 172.21.15.16:0/3505174024 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f80003ce0 con 0x7f4f9009e930 2023-09-30T22:32:30.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.119+0000 7f4f777fe700 1 -- 172.21.15.16:0/3505174024 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f80003e40 con 0x7f4f9009e930 2023-09-30T22:32:30.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.119+0000 7f4f956d3700 1 -- 172.21.15.16:0/3505174024 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f900a0190 con 0x7f4f9009e930 2023-09-30T22:32:30.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.119+0000 7f4f956d3700 1 -- 172.21.15.16:0/3505174024 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f901a90e0 con 0x7f4f9009e930 2023-09-30T22:32:30.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.119+0000 7f4f777fe700 1 -- 172.21.15.16:0/3505174024 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f8000b8a0 con 0x7f4f9009e930 2023-09-30T22:32:30.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.120+0000 7f4f777fe700 1 -- 172.21.15.16:0/3505174024 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4f80021070 con 0x7f4f9009e930 2023-09-30T22:32:30.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.121+0000 7f4f777fe700 1 -- 172.21.15.16:0/3505174024 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f4f80040a90 con 0x7f4f9009e930 2023-09-30T22:32:30.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.121+0000 7f4f956d3700 1 -- 172.21.15.16:0/3505174024 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f901a0b30 con 0x7f4f9009e930 2023-09-30T22:32:30.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.126+0000 7f4f777fe700 1 -- 172.21.15.16:0/3505174024 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4f8001a460 con 0x7f4f9009e930 2023-09-30T22:32:30.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.194+0000 7f769a1d2700 1 -- 172.21.15.16:0/1896841073 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f769406c360 con 0x7f769409e410 2023-09-30T22:32:30.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.195+0000 7f7690ff9700 1 -- 172.21.15.16:0/1896841073 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f768c01b2c0 con 0x7f769409e410 2023-09-30T22:32:30.195 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:32:30.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.198+0000 7f769a1d2700 1 -- 172.21.15.16:0/1896841073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f769409e410 msgr2=0x7f76941a6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:30.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.198+0000 7f769a1d2700 1 --2- 172.21.15.16:0/1896841073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f769409e410 0x7f76941a6880 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f768c00bdf0 tx=0x7f768c009ed0 comp rx=0 tx=0).stop 2023-09-30T22:32:30.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.198+0000 7f769a1d2700 1 -- 172.21.15.16:0/1896841073 shutdown_connections 2023-09-30T22:32:30.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.198+0000 7f769a1d2700 1 --2- 172.21.15.16:0/1896841073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f769409e410 0x7f76941a6880 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:30.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.198+0000 7f769a1d2700 1 --2- 172.21.15.16:0/1896841073 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f769409d6f0 0x7f76941a5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:30.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.198+0000 7f769a1d2700 1 --2- 172.21.15.16:0/1896841073 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f769409e000 0x7f76941a6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:30.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.198+0000 7f769a1d2700 1 -- 172.21.15.16:0/1896841073 >> 172.21.15.16:0/1896841073 conn(0x7f7694066f50 msgr2=0x7f76941bae60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:30.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.198+0000 7f769a1d2700 1 -- 172.21.15.16:0/1896841073 shutdown_connections 2023-09-30T22:32:30.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.198+0000 7f769a1d2700 1 -- 172.21.15.16:0/1896841073 wait complete. 2023-09-30T22:32:30.218 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:32:30.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.227+0000 7ff705b3f700 1 -- 172.21.15.16:0/1482950933 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff700074ea0 con 0x7ff700134610 2023-09-30T22:32:30.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.227+0000 7ff6e7fff700 1 -- 172.21.15.16:0/1482950933 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff6f4015020 con 0x7ff700134610 2023-09-30T22:32:30.228 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:32:30.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.229+0000 7ff705b3f700 1 -- 172.21.15.16:0/1482950933 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff700134610 msgr2=0x7ff70009be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:30.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.229+0000 7ff705b3f700 1 --2- 172.21.15.16:0/1482950933 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff700134610 0x7ff70009be10 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff6f4016000 tx=0x7ff6f4004750 comp rx=0 tx=0).stop 2023-09-30T22:32:30.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.229+0000 7ff705b3f700 1 -- 172.21.15.16:0/1482950933 shutdown_connections 2023-09-30T22:32:30.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.229+0000 7ff705b3f700 1 --2- 172.21.15.16:0/1482950933 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff70006e570 0x7ff700127470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:30.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.229+0000 7ff705b3f700 1 --2- 172.21.15.16:0/1482950933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7001279b0 0x7ff70009c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:30.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.229+0000 7ff705b3f700 1 --2- 172.21.15.16:0/1482950933 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff700134610 0x7ff70009be10 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:30.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.229+0000 7ff705b3f700 1 -- 172.21.15.16:0/1482950933 >> 172.21.15.16:0/1482950933 conn(0x7ff7000a2aa0 msgr2=0x7ff7000a0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:30.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.230+0000 7ff705b3f700 1 -- 172.21.15.16:0/1482950933 shutdown_connections 2023-09-30T22:32:30.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.230+0000 7ff705b3f700 1 -- 172.21.15.16:0/1482950933 wait complete. 2023-09-30T22:32:30.244 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:32:30.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.292+0000 7f32960ea700 1 -- 172.21.15.16:0/3070405322 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3290120240 con 0x7f3290139170 2023-09-30T22:32:30.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.293+0000 7f3277fff700 1 -- 172.21.15.16:0/3070405322 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3288018020 con 0x7f3290139170 2023-09-30T22:32:30.293 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:32:30.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.295+0000 7f32960ea700 1 -- 172.21.15.16:0/3070405322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3290139170 msgr2=0x7f329011f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:30.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.295+0000 7f32960ea700 1 --2- 172.21.15.16:0/3070405322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3290139170 0x7f329011f010 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f328800efc0 tx=0x7f328800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:32:30.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.295+0000 7f32960ea700 1 -- 172.21.15.16:0/3070405322 shutdown_connections 2023-09-30T22:32:30.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.295+0000 7f32960ea700 1 --2- 172.21.15.16:0/3070405322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3290139170 0x7f329011f010 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:30.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.295+0000 7f32960ea700 1 --2- 172.21.15.16:0/3070405322 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f329009d6f0 0x7f329011e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:30.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.295+0000 7f32960ea700 1 --2- 172.21.15.16:0/3070405322 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f329009e1d0 0x7f329011ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:30.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.295+0000 7f32960ea700 1 -- 172.21.15.16:0/3070405322 >> 172.21.15.16:0/3070405322 conn(0x7f3290066f50 msgr2=0x7f3290136b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:30.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.295+0000 7f32960ea700 1 -- 172.21.15.16:0/3070405322 shutdown_connections 2023-09-30T22:32:30.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.295+0000 7f32960ea700 1 -- 172.21.15.16:0/3070405322 wait complete. 2023-09-30T22:32:30.309 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:32:30.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.342+0000 7f4f956d3700 1 -- 172.21.15.16:0/3505174024 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4f9009ffe0 con 0x7f4f9009e930 2023-09-30T22:32:30.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.343+0000 7f4f777fe700 1 -- 172.21.15.16:0/3505174024 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4f8002cbe0 con 0x7f4f9009e930 2023-09-30T22:32:30.343 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:32:30.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.345+0000 7f4f956d3700 1 -- 172.21.15.16:0/3505174024 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4f9009e930 msgr2=0x7f4f9006b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:30.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.345+0000 7f4f956d3700 1 --2- 172.21.15.16:0/3505174024 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4f9009e930 0x7f4f9006b8b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4f80009130 tx=0x7f4f80004750 comp rx=0 tx=0).stop 2023-09-30T22:32:30.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.345+0000 7f4f956d3700 1 -- 172.21.15.16:0/3505174024 shutdown_connections 2023-09-30T22:32:30.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.345+0000 7f4f956d3700 1 --2- 172.21.15.16:0/3505174024 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4f9009f650 0x7f4f9006c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:30.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.345+0000 7f4f956d3700 1 --2- 172.21.15.16:0/3505174024 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4f9009e930 0x7f4f9006b8b0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:30.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.345+0000 7f4f956d3700 1 --2- 172.21.15.16:0/3505174024 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f9009f240 0x7f4f9006bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:30.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.345+0000 7f4f956d3700 1 -- 172.21.15.16:0/3505174024 >> 172.21.15.16:0/3505174024 conn(0x7f4f900a4210 msgr2=0x7f4f901b8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:30.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.345+0000 7f4f956d3700 1 -- 172.21.15.16:0/3505174024 shutdown_connections 2023-09-30T22:32:30.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:30.345+0000 7f4f956d3700 1 -- 172.21.15.16:0/3505174024 wait complete. 2023-09-30T22:32:30.359 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:32:31.218 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:32:31.245 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:32:31.310 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:32:31.362 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:32:31.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.378+0000 7f10b794f700 1 Processor -- start 2023-09-30T22:32:31.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.378+0000 7f10b794f700 1 -- start start 2023-09-30T22:32:31.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.378+0000 7f10b794f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f10b0130540 0x7f10b0130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:31.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.378+0000 7f10b794f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f10b0130e50 0x7f10b006f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:31.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.378+0000 7f10b794f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f10b006fb20 0x7f10b006ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:31.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.378+0000 7f10b794f700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10b009a390 con 0x7f10b0130540 2023-09-30T22:32:31.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.378+0000 7f10b794f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10b00716a0 con 0x7f10b006fb20 2023-09-30T22:32:31.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.378+0000 7f10b794f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f10b0071290 con 0x7f10b0130e50 2023-09-30T22:32:31.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.379+0000 7f10b4eea700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f10b0130e50 0x7f10b006f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:31.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.379+0000 7f10b5eec700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f10b006fb20 0x7f10b006ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:31.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.379+0000 7f10b5eec700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f10b006fb20 0x7f10b006ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:44840/0 (socket says 172.21.15.16:44840) 2023-09-30T22:32:31.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.379+0000 7f10b4eea700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f10b0130e50 0x7f10b006f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:46346/0 (socket says 172.21.15.16:46346) 2023-09-30T22:32:31.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.379+0000 7f10b5eec700 1 -- 172.21.15.16:0/4203985917 learned_addr learned my addr 172.21.15.16:0/4203985917 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:31.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.379+0000 7f10b56eb700 1 --2- 172.21.15.16:0/4203985917 >> v2:172.21.15.16:3300/0 conn(0x7f10b0130540 0x7f10b0130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:31.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.379+0000 7f10b4eea700 1 -- 172.21.15.16:0/4203985917 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f10b006fb20 msgr2=0x7f10b006ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:31.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.379+0000 7f10b4eea700 1 --2- 172.21.15.16:0/4203985917 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f10b006fb20 0x7f10b006ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:31.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.379+0000 7f10b4eea700 1 -- 172.21.15.16:0/4203985917 >> v2:172.21.15.16:3300/0 conn(0x7f10b0130540 msgr2=0x7f10b0130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:31.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.379+0000 7f10b4eea700 1 --2- 172.21.15.16:0/4203985917 >> v2:172.21.15.16:3300/0 conn(0x7f10b0130540 0x7f10b0130910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:31.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.379+0000 7f10b4eea700 1 -- 172.21.15.16:0/4203985917 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10b0130340 con 0x7f10b0130e50 2023-09-30T22:32:31.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.379+0000 7f10b4eea700 1 --2- 172.21.15.16:0/4203985917 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f10b0130e50 0x7f10b006f5e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f10a00098a0 tx=0x7f10a0009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d0f09c6f85afe2a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:31.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.380+0000 7f10a7fff700 1 -- 172.21.15.16:0/4203985917 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a0004220 con 0x7f10b0130e50 2023-09-30T22:32:31.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.380+0000 7f10a7fff700 1 -- 172.21.15.16:0/4203985917 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10a0004380 con 0x7f10b0130e50 2023-09-30T22:32:31.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.380+0000 7f10a7fff700 1 -- 172.21.15.16:0/4203985917 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a0010890 con 0x7f10b0130e50 2023-09-30T22:32:31.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.380+0000 7f10b794f700 1 -- 172.21.15.16:0/4203985917 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f10b0130e50 msgr2=0x7f10b006f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:31.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.381+0000 7f10b794f700 1 --2- 172.21.15.16:0/4203985917 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f10b0130e50 0x7f10b006f5e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f10a00098a0 tx=0x7f10a0009bb0 comp rx=0 tx=0).stop 2023-09-30T22:32:31.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.381+0000 7f10b794f700 1 -- 172.21.15.16:0/4203985917 shutdown_connections 2023-09-30T22:32:31.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.381+0000 7f10b794f700 1 --2- 172.21.15.16:0/4203985917 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f10b006fb20 0x7f10b006ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:31.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.381+0000 7f10b794f700 1 --2- 172.21.15.16:0/4203985917 >> v2:172.21.15.16:3300/0 conn(0x7f10b0130540 0x7f10b0130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:31.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.381+0000 7f10b794f700 1 --2- 172.21.15.16:0/4203985917 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f10b0130e50 0x7f10b006f5e0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:31.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.381+0000 7f10b794f700 1 -- 172.21.15.16:0/4203985917 >> 172.21.15.16:0/4203985917 conn(0x7f10b00a2aa0 msgr2=0x7f10b00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:31.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.381+0000 7f10b794f700 1 -- 172.21.15.16:0/4203985917 shutdown_connections 2023-09-30T22:32:31.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.381+0000 7f10b794f700 1 -- 172.21.15.16:0/4203985917 wait complete. 2023-09-30T22:32:31.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.382+0000 7f10b794f700 1 Processor -- start 2023-09-30T22:32:31.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.382+0000 7f10b794f700 1 -- start start 2023-09-30T22:32:31.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.382+0000 7f10b794f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f10b006fb20 0x7f10b01c3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:31.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.383+0000 7f10b794f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f10b0130540 0x7f10b01c4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:31.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.383+0000 7f10b794f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10b0130e50 0x7f10b0118cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:31.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.383+0000 7f10b794f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b012bd00 con 0x7f10b0130e50 2023-09-30T22:32:31.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.383+0000 7f10b794f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10b011bfa0 con 0x7f10b0130540 2023-09-30T22:32:31.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.383+0000 7f10b794f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f10b01c3b60 con 0x7f10b006fb20 2023-09-30T22:32:31.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.383+0000 7f10b56eb700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f10b006fb20 0x7f10b01c3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:31.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.383+0000 7f10b56eb700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f10b006fb20 0x7f10b01c3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:46358/0 (socket says 172.21.15.16:46358) 2023-09-30T22:32:31.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.383+0000 7f10b56eb700 1 -- 172.21.15.16:0/635109553 learned_addr learned my addr 172.21.15.16:0/635109553 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:31.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.383+0000 7f10b4eea700 1 --2- 172.21.15.16:0/635109553 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f10b0130540 0x7f10b01c4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:31.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.383+0000 7f10b5eec700 1 --2- 172.21.15.16:0/635109553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10b0130e50 0x7f10b0118cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:31.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.383+0000 7f10b56eb700 1 -- 172.21.15.16:0/635109553 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f10b0130540 msgr2=0x7f10b01c4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:31.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.383+0000 7f10b56eb700 1 --2- 172.21.15.16:0/635109553 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f10b0130540 0x7f10b01c4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:31.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.383+0000 7f10b56eb700 1 -- 172.21.15.16:0/635109553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10b0130e50 msgr2=0x7f10b0118cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:31.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.383+0000 7f10b56eb700 1 --2- 172.21.15.16:0/635109553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10b0130e50 0x7f10b0118cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:31.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.383+0000 7f10b56eb700 1 -- 172.21.15.16:0/635109553 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10a00094a0 con 0x7f10b006fb20 2023-09-30T22:32:31.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.383+0000 7f10b5eec700 1 --2- 172.21.15.16:0/635109553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10b0130e50 0x7f10b0118cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:32:31.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.383+0000 7f10b56eb700 1 --2- 172.21.15.16:0/635109553 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f10b006fb20 0x7f10b01c3f20 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f10ac00e810 tx=0x7f10ac00eb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:31.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.384+0000 7f10a5ffb700 1 -- 172.21.15.16:0/635109553 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ac013610 con 0x7f10b006fb20 2023-09-30T22:32:31.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.384+0000 7f10a5ffb700 1 -- 172.21.15.16:0/635109553 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10ac013770 con 0x7f10b006fb20 2023-09-30T22:32:31.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.384+0000 7f10b794f700 1 -- 172.21.15.16:0/635109553 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10b00a0bf0 con 0x7f10b006fb20 2023-09-30T22:32:31.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.384+0000 7f10a5ffb700 1 -- 172.21.15.16:0/635109553 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10ac013b20 con 0x7f10b006fb20 2023-09-30T22:32:31.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.384+0000 7f10b794f700 1 -- 172.21.15.16:0/635109553 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10b011a0b0 con 0x7f10b006fb20 2023-09-30T22:32:31.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.385+0000 7f10a5ffb700 1 -- 172.21.15.16:0/635109553 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f10ac013c80 con 0x7f10b006fb20 2023-09-30T22:32:31.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.385+0000 7f10b794f700 1 -- 172.21.15.16:0/635109553 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10b0074ea0 con 0x7f10b006fb20 2023-09-30T22:32:31.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.386+0000 7f10a5ffb700 1 -- 172.21.15.16:0/635109553 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f10ac03f350 con 0x7f10b006fb20 2023-09-30T22:32:31.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.393+0000 7f10a5ffb700 1 -- 172.21.15.16:0/635109553 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f10ac012370 con 0x7f10b006fb20 2023-09-30T22:32:31.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.423+0000 7f513efa5700 1 Processor -- start 2023-09-30T22:32:31.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.423+0000 7f513efa5700 1 -- start start 2023-09-30T22:32:31.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.423+0000 7f513efa5700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f513809d6f0 0x7f513809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:31.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.424+0000 7f513efa5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f513809e440 0x7f5138134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:31.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.424+0000 7f513efa5700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f513809b120 con 0x7f513809e000 2023-09-30T22:32:31.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.424+0000 7f513efa5700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f513806b460 con 0x7f513809d6f0 2023-09-30T22:32:31.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.424+0000 7f513efa5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f513809a7e0 con 0x7f513809e440 2023-09-30T22:32:31.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.424+0000 7f5137fff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f513809e000 0x7f51381339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:39008/0 (socket says 172.21.15.16:39008) 2023-09-30T22:32:31.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.424+0000 7f5137fff700 1 -- 172.21.15.16:0/11810884 learned_addr learned my addr 172.21.15.16:0/11810884 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:31.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.424+0000 7f513cd41700 1 --2- 172.21.15.16:0/11810884 >> v2:172.21.15.16:3300/0 conn(0x7f513809d6f0 0x7f513809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:31.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.424+0000 7f513d542700 1 --2- 172.21.15.16:0/11810884 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f513809e440 0x7f5138134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:31.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.424+0000 7f513d542700 1 -- 172.21.15.16:0/11810884 >> v2:172.21.15.16:3300/0 conn(0x7f513809d6f0 msgr2=0x7f513809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:31.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.424+0000 7f513d542700 1 --2- 172.21.15.16:0/11810884 >> v2:172.21.15.16:3300/0 conn(0x7f513809d6f0 0x7f513809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:31.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.424+0000 7f513d542700 1 -- 172.21.15.16:0/11810884 >> v1:172.21.15.16:6789/0 conn(0x7f513809e000 legacy=0x7f51381339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:31.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.424+0000 7f513d542700 1 -- 172.21.15.16:0/11810884 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f513809a500 con 0x7f513809e440 2023-09-30T22:32:31.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.424+0000 7f51377fe700 1 -- 172.21.15.16:0/11810884 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2920843671 0 0) 0x7f513809b120 con 0x7f513809e000 2023-09-30T22:32:31.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.424+0000 7f513cd41700 1 --2- 172.21.15.16:0/11810884 >> v2:172.21.15.16:3300/0 conn(0x7f513809d6f0 0x7f513809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:31.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.425+0000 7f513d542700 1 --2- 172.21.15.16:0/11810884 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f513809e440 0x7f5138134100 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5130009dc0 tx=0x7f513000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=669c1bf10b6e1829 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:31.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.425+0000 7f51377fe700 1 -- 172.21.15.16:0/11810884 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513000e070 con 0x7f513809e440 2023-09-30T22:32:31.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.425+0000 7f51377fe700 1 -- 172.21.15.16:0/11810884 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f513000bae0 con 0x7f513809e440 2023-09-30T22:32:31.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.425+0000 7f51377fe700 1 -- 172.21.15.16:0/11810884 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513000be90 con 0x7f513809e440 2023-09-30T22:32:31.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.426+0000 7f513efa5700 1 -- 172.21.15.16:0/11810884 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f513809e440 msgr2=0x7f5138134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:31.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.426+0000 7f513efa5700 1 --2- 172.21.15.16:0/11810884 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f513809e440 0x7f5138134100 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5130009dc0 tx=0x7f513000b330 comp rx=0 tx=0).stop 2023-09-30T22:32:31.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.426+0000 7f513efa5700 1 -- 172.21.15.16:0/11810884 shutdown_connections 2023-09-30T22:32:31.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.426+0000 7f513efa5700 1 --2- 172.21.15.16:0/11810884 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f513809e440 0x7f5138134100 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:31.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.426+0000 7f513efa5700 1 --2- 172.21.15.16:0/11810884 >> v2:172.21.15.16:3300/0 conn(0x7f513809d6f0 0x7f513809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:31.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.426+0000 7f513efa5700 1 -- 172.21.15.16:0/11810884 >> 172.21.15.16:0/11810884 conn(0x7f5138066f50 msgr2=0x7f513809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:31.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.426+0000 7f513efa5700 1 -- 172.21.15.16:0/11810884 shutdown_connections 2023-09-30T22:32:31.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.426+0000 7f513efa5700 1 -- 172.21.15.16:0/11810884 wait complete. 2023-09-30T22:32:31.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.428+0000 7f513efa5700 1 Processor -- start 2023-09-30T22:32:31.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.428+0000 7f513efa5700 1 -- start start 2023-09-30T22:32:31.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.429+0000 7f513efa5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f513809d6f0 0x7f51381a4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:31.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.429+0000 7f513efa5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f513809e000 0x7f51381a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:31.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.429+0000 7f513efa5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f513809e440 0x7f51381c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:31.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.429+0000 7f513efa5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51381a52c0 con 0x7f513809d6f0 2023-09-30T22:32:31.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.429+0000 7f513efa5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51381b8040 con 0x7f513809e000 2023-09-30T22:32:31.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.429+0000 7f513efa5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f513809c710 con 0x7f513809e440 2023-09-30T22:32:31.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.429+0000 7f513d542700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f513809e440 0x7f51381c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:31.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.429+0000 7f513d542700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f513809e440 0x7f51381c8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:46368/0 (socket says 172.21.15.16:46368) 2023-09-30T22:32:31.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.429+0000 7f513d542700 1 -- 172.21.15.16:0/3395889486 learned_addr learned my addr 172.21.15.16:0/3395889486 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:31.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.429+0000 7f5137fff700 1 --2- 172.21.15.16:0/3395889486 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f513809e000 0x7f51381a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:31.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.429+0000 7f513cd41700 1 --2- 172.21.15.16:0/3395889486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f513809d6f0 0x7f51381a4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:31.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.429+0000 7f513d542700 1 -- 172.21.15.16:0/3395889486 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f513809e000 msgr2=0x7f51381a46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:31.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.429+0000 7f513d542700 1 --2- 172.21.15.16:0/3395889486 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f513809e000 0x7f51381a46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:31.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.429+0000 7f513d542700 1 -- 172.21.15.16:0/3395889486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f513809d6f0 msgr2=0x7f51381a4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:31.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.429+0000 7f513d542700 1 --2- 172.21.15.16:0/3395889486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f513809d6f0 0x7f51381a4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:31.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.429+0000 7f513d542700 1 -- 172.21.15.16:0/3395889486 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f512c0094a0 con 0x7f513809e440 2023-09-30T22:32:31.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.429+0000 7f5137fff700 1 --2- 172.21.15.16:0/3395889486 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f513809e000 0x7f51381a46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:31.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.430+0000 7f513d542700 1 --2- 172.21.15.16:0/3395889486 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f513809e440 0x7f51381c8090 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f513809a920 tx=0x7f5130004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:31.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.430+0000 7f51357fa700 1 -- 172.21.15.16:0/3395889486 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51300045d0 con 0x7f513809e440 2023-09-30T22:32:31.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.430+0000 7f51357fa700 1 -- 172.21.15.16:0/3395889486 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5130020070 con 0x7f513809e440 2023-09-30T22:32:31.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.430+0000 7f513efa5700 1 -- 172.21.15.16:0/3395889486 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5130009a70 con 0x7f513809e440 2023-09-30T22:32:31.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.430+0000 7f51357fa700 1 -- 172.21.15.16:0/3395889486 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5130027480 con 0x7f513809e440 2023-09-30T22:32:31.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.430+0000 7f513efa5700 1 -- 172.21.15.16:0/3395889486 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51381a5ab0 con 0x7f513809e440 2023-09-30T22:32:31.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.431+0000 7f51357fa700 1 -- 172.21.15.16:0/3395889486 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5130012430 con 0x7f513809e440 2023-09-30T22:32:31.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.431+0000 7f51357fa700 1 -- 172.21.15.16:0/3395889486 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f5130022070 con 0x7f513809e440 2023-09-30T22:32:31.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.432+0000 7f513efa5700 1 -- 172.21.15.16:0/3395889486 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5138074ea0 con 0x7f513809e440 2023-09-30T22:32:31.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.437+0000 7f51357fa700 1 -- 172.21.15.16:0/3395889486 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f513001b400 con 0x7f513809e440 2023-09-30T22:32:31.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.470+0000 7f8dab229700 1 Processor -- start 2023-09-30T22:32:31.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.470+0000 7f8dab229700 1 -- start start 2023-09-30T22:32:31.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.471+0000 7f8dab229700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8da4131810 0x7f8da4131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:31.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.471+0000 7f8dab229700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8da4099a40 0x7f8da4099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:31.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.471+0000 7f8dab229700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8da409a3b0 0x7f8da4138650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:31.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.471+0000 7f8dab229700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8da41313c0 con 0x7f8da4099a40 2023-09-30T22:32:31.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.471+0000 7f8dab229700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8da406e8c0 con 0x7f8da409a3b0 2023-09-30T22:32:31.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.471+0000 7f8dab229700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8da409a820 con 0x7f8da4131810 2023-09-30T22:32:31.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.471+0000 7f8da3fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8da4099a40 0x7f8da4099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:31.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.471+0000 7f8da3fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8da4099a40 0x7f8da4099e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56230/0 (socket says 172.21.15.16:56230) 2023-09-30T22:32:31.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.471+0000 7f8da8fc5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8da4131810 0x7f8da4131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:31.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.471+0000 7f8da3fff700 1 -- 172.21.15.16:0/285116575 learned_addr learned my addr 172.21.15.16:0/285116575 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:31.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.471+0000 7f8da97c6700 1 --2- 172.21.15.16:0/285116575 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8da409a3b0 0x7f8da4138650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:31.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.472+0000 7f8da3fff700 1 -- 172.21.15.16:0/285116575 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8da4131810 msgr2=0x7f8da4131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:31.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.472+0000 7f8da3fff700 1 --2- 172.21.15.16:0/285116575 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8da4131810 0x7f8da4131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:31.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.472+0000 7f8da3fff700 1 -- 172.21.15.16:0/285116575 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8da409a3b0 msgr2=0x7f8da4138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:31.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.472+0000 7f8da3fff700 1 --2- 172.21.15.16:0/285116575 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8da409a3b0 0x7f8da4138650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:31.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.472+0000 7f8da3fff700 1 -- 172.21.15.16:0/285116575 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da4071d20 con 0x7f8da4099a40 2023-09-30T22:32:31.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.472+0000 7f8da3fff700 1 --2- 172.21.15.16:0/285116575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8da4099a40 0x7f8da4099e70 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f8d900098f0 tx=0x7f8d90009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93542a1427dec872 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:31.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.472+0000 7f8da37fe700 1 -- 172.21.15.16:0/285116575 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d90011830 con 0x7f8da4099a40 2023-09-30T22:32:31.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.472+0000 7f8da37fe700 1 -- 172.21.15.16:0/285116575 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d90011990 con 0x7f8da4099a40 2023-09-30T22:32:31.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.472+0000 7f8da37fe700 1 -- 172.21.15.16:0/285116575 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d90011d20 con 0x7f8da4099a40 2023-09-30T22:32:31.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.473+0000 7f8dab229700 1 -- 172.21.15.16:0/285116575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8da4099a40 msgr2=0x7f8da4099e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:31.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.473+0000 7f8dab229700 1 --2- 172.21.15.16:0/285116575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8da4099a40 0x7f8da4099e70 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f8d900098f0 tx=0x7f8d90009c00 comp rx=0 tx=0).stop 2023-09-30T22:32:31.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.473+0000 7f8dab229700 1 -- 172.21.15.16:0/285116575 shutdown_connections 2023-09-30T22:32:31.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.473+0000 7f8dab229700 1 --2- 172.21.15.16:0/285116575 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8da409a3b0 0x7f8da4138650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:31.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.473+0000 7f8dab229700 1 --2- 172.21.15.16:0/285116575 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8da4131810 0x7f8da4131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:31.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.473+0000 7f8dab229700 1 --2- 172.21.15.16:0/285116575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8da4099a40 0x7f8da4099e70 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:31.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.473+0000 7f8dab229700 1 -- 172.21.15.16:0/285116575 >> 172.21.15.16:0/285116575 conn(0x7f8da40a2aa0 msgr2=0x7f8da40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:31.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.473+0000 7f8dab229700 1 -- 172.21.15.16:0/285116575 shutdown_connections 2023-09-30T22:32:31.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.473+0000 7f8dab229700 1 -- 172.21.15.16:0/285116575 wait complete. 2023-09-30T22:32:31.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.474+0000 7f8dab229700 1 Processor -- start 2023-09-30T22:32:31.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.474+0000 7f8dab229700 1 -- start start 2023-09-30T22:32:31.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.475+0000 7f8dab229700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8da4099a40 0x7f8da411b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:31.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.475+0000 7f8dab229700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8da409a3b0 0x7f8da411bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:31.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.475+0000 7f8dab229700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8da4131810 0x7f8da411c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:31.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.475+0000 7f8dab229700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da411f050 con 0x7f8da4099a40 2023-09-30T22:32:31.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.475+0000 7f8dab229700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8da411d370 con 0x7f8da409a3b0 2023-09-30T22:32:31.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.475+0000 7f8dab229700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8da40a1c20 con 0x7f8da4131810 2023-09-30T22:32:31.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.475+0000 7f8da8fc5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8da4099a40 0x7f8da411b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:31.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.475+0000 7f8da8fc5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8da4099a40 0x7f8da411b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56236/0 (socket says 172.21.15.16:56236) 2023-09-30T22:32:31.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.475+0000 7f8da8fc5700 1 -- 172.21.15.16:0/10326634 learned_addr learned my addr 172.21.15.16:0/10326634 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:31.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.475+0000 7f8da97c6700 1 --2- 172.21.15.16:0/10326634 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8da4131810 0x7f8da411c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:31.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.475+0000 7f8da3fff700 1 --2- 172.21.15.16:0/10326634 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8da409a3b0 0x7f8da411bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:31.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.475+0000 7f8da8fc5700 1 -- 172.21.15.16:0/10326634 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8da4131810 msgr2=0x7f8da411c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:31.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.475+0000 7f8da8fc5700 1 --2- 172.21.15.16:0/10326634 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8da4131810 0x7f8da411c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:31.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.475+0000 7f8da8fc5700 1 -- 172.21.15.16:0/10326634 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8da409a3b0 msgr2=0x7f8da411bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:31.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.475+0000 7f8da8fc5700 1 --2- 172.21.15.16:0/10326634 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8da409a3b0 0x7f8da411bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:31.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.475+0000 7f8da8fc5700 1 -- 172.21.15.16:0/10326634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d90009550 con 0x7f8da4099a40 2023-09-30T22:32:31.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.476+0000 7f8da8fc5700 1 --2- 172.21.15.16:0/10326634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8da4099a40 0x7f8da411b850 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f8d98012fc0 tx=0x7f8d980100c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:31.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.476+0000 7f8da17fa700 1 -- 172.21.15.16:0/10326634 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d98010940 con 0x7f8da4099a40 2023-09-30T22:32:31.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.476+0000 7f8da17fa700 1 -- 172.21.15.16:0/10326634 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d98013040 con 0x7f8da4099a40 2023-09-30T22:32:31.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.476+0000 7f8dab229700 1 -- 172.21.15.16:0/10326634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8da411e080 con 0x7f8da4099a40 2023-09-30T22:32:31.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.476+0000 7f8da17fa700 1 -- 172.21.15.16:0/10326634 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d98010ca0 con 0x7f8da4099a40 2023-09-30T22:32:31.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.476+0000 7f8dab229700 1 -- 172.21.15.16:0/10326634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8da411f750 con 0x7f8da4099a40 2023-09-30T22:32:31.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.477+0000 7f8da17fa700 1 -- 172.21.15.16:0/10326634 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8d98018480 con 0x7f8da4099a40 2023-09-30T22:32:31.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.477+0000 7f8dab229700 1 -- 172.21.15.16:0/10326634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8da411f190 con 0x7f8da4099a40 2023-09-30T22:32:31.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.477+0000 7f8da17fa700 1 -- 172.21.15.16:0/10326634 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f8d980461c0 con 0x7f8da4099a40 2023-09-30T22:32:31.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.482+0000 7f8da17fa700 1 -- 172.21.15.16:0/10326634 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8da411f190 con 0x7f8da4099a40 2023-09-30T22:32:31.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.541+0000 7fe79e9cc700 1 Processor -- start 2023-09-30T22:32:31.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.541+0000 7fe79e9cc700 1 -- start start 2023-09-30T22:32:31.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.541+0000 7fe79e9cc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe79812cf60 0x7fe79806f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:31.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.541+0000 7fe79e9cc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe79812f8b0 0x7fe79812fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:31.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.542+0000 7fe79e9cc700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe79812cd90 con 0x7fe798130260 2023-09-30T22:32:31.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.542+0000 7fe79e9cc700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe79812d330 con 0x7fe79812cf60 2023-09-30T22:32:31.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.542+0000 7fe79e9cc700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7981306d0 con 0x7fe79812f8b0 2023-09-30T22:32:31.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.542+0000 7fe7977fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe79812f8b0 0x7fe79812fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:31.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.542+0000 7fe7977fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe79812f8b0 0x7fe79812fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:44888/0 (socket says 172.21.15.16:44888) 2023-09-30T22:32:31.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.542+0000 7fe7977fe700 1 -- 172.21.15.16:0/4194818046 learned_addr learned my addr 172.21.15.16:0/4194818046 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:31.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.542+0000 7fe797fff700 1 --2- 172.21.15.16:0/4194818046 >> v2:172.21.15.16:3300/0 conn(0x7fe79812cf60 0x7fe79806f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:31.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.542+0000 7fe797fff700 1 -- 172.21.15.16:0/4194818046 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe79812f8b0 msgr2=0x7fe79812fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:31.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.542+0000 7fe797fff700 1 --2- 172.21.15.16:0/4194818046 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe79812f8b0 0x7fe79812fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:31.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.542+0000 7fe797fff700 1 -- 172.21.15.16:0/4194818046 >> v1:172.21.15.16:6789/0 conn(0x7fe798130260 legacy=0x7fe798136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:31.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.542+0000 7fe797fff700 1 -- 172.21.15.16:0/4194818046 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe798071780 con 0x7fe79812cf60 2023-09-30T22:32:31.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.542+0000 7fe797fff700 1 --2- 172.21.15.16:0/4194818046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe79812cf60 0x7fe79806f5d0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fe78800dc60 tx=0x7fe78800df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40106a646b313537 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:31.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.543+0000 7fe796ffd700 1 -- 172.21.15.16:0/4194818046 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe788012070 con 0x7fe79812cf60 2023-09-30T22:32:31.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.543+0000 7fe796ffd700 1 -- 172.21.15.16:0/4194818046 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe78800d2e0 con 0x7fe79812cf60 2023-09-30T22:32:31.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.543+0000 7fe796ffd700 1 -- 172.21.15.16:0/4194818046 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78800f900 con 0x7fe79812cf60 2023-09-30T22:32:31.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.544+0000 7fe79e9cc700 1 -- 172.21.15.16:0/4194818046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe79812cf60 msgr2=0x7fe79806f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:31.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.544+0000 7fe79e9cc700 1 --2- 172.21.15.16:0/4194818046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe79812cf60 0x7fe79806f5d0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fe78800dc60 tx=0x7fe78800df70 comp rx=0 tx=0).stop 2023-09-30T22:32:31.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.544+0000 7fe79e9cc700 1 -- 172.21.15.16:0/4194818046 shutdown_connections 2023-09-30T22:32:31.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.544+0000 7fe79e9cc700 1 --2- 172.21.15.16:0/4194818046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe79812cf60 0x7fe79806f5d0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:31.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.544+0000 7fe79e9cc700 1 --2- 172.21.15.16:0/4194818046 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe79812f8b0 0x7fe79812fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:31.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.544+0000 7fe79e9cc700 1 -- 172.21.15.16:0/4194818046 >> 172.21.15.16:0/4194818046 conn(0x7fe7980a2aa0 msgr2=0x7fe7980a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:31.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.544+0000 7fe79e9cc700 1 -- 172.21.15.16:0/4194818046 shutdown_connections 2023-09-30T22:32:31.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.544+0000 7fe79e9cc700 1 -- 172.21.15.16:0/4194818046 wait complete. 2023-09-30T22:32:31.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.545+0000 7fe79e9cc700 1 Processor -- start 2023-09-30T22:32:31.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.546+0000 7fe79e9cc700 1 -- start start 2023-09-30T22:32:31.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.546+0000 7fe79e9cc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe79812cf60 0x7fe7981c4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:31.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.546+0000 7fe79e9cc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe79812f8b0 0x7fe7981c4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:31.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.546+0000 7fe79e9cc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe798130260 0x7fe7981c4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:31.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.547+0000 7fe79e9cc700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79809e780 con 0x7fe79812f8b0 2023-09-30T22:32:31.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.547+0000 7fe79e9cc700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe79809d020 con 0x7fe79812cf60 2023-09-30T22:32:31.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.547+0000 7fe79e9cc700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe79809e030 con 0x7fe798130260 2023-09-30T22:32:31.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.547+0000 7fe7977fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe79812f8b0 0x7fe7981c4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:31.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.547+0000 7fe7977fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe79812f8b0 0x7fe7981c4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56254/0 (socket says 172.21.15.16:56254) 2023-09-30T22:32:31.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.547+0000 7fe7977fe700 1 -- 172.21.15.16:0/2475033191 learned_addr learned my addr 172.21.15.16:0/2475033191 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:31.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.547+0000 7fe797fff700 1 --2- 172.21.15.16:0/2475033191 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe79812cf60 0x7fe7981c4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:31.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.547+0000 7fe7977fe700 1 -- 172.21.15.16:0/2475033191 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe798130260 msgr2=0x7fe7981c4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:31.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.547+0000 7fe7977fe700 1 --2- 172.21.15.16:0/2475033191 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe798130260 0x7fe7981c4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:31.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.547+0000 7fe7977fe700 1 -- 172.21.15.16:0/2475033191 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe79812cf60 msgr2=0x7fe7981c4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:31.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.547+0000 7fe7977fe700 1 --2- 172.21.15.16:0/2475033191 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe79812cf60 0x7fe7981c4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:31.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.547+0000 7fe7977fe700 1 -- 172.21.15.16:0/2475033191 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe78800d940 con 0x7fe79812f8b0 2023-09-30T22:32:31.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.547+0000 7fe797fff700 1 --2- 172.21.15.16:0/2475033191 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe79812cf60 0x7fe7981c4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:31.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.547+0000 7fe7977fe700 1 --2- 172.21.15.16:0/2475033191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe79812f8b0 0x7fe7981c4670 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fe78c00db70 tx=0x7fe78c00de80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:31.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.548+0000 7fe794ff9700 1 -- 172.21.15.16:0/2475033191 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78c00cda0 con 0x7fe79812f8b0 2023-09-30T22:32:31.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.548+0000 7fe794ff9700 1 -- 172.21.15.16:0/2475033191 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe78c015420 con 0x7fe79812f8b0 2023-09-30T22:32:31.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.548+0000 7fe794ff9700 1 -- 172.21.15.16:0/2475033191 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78c015790 con 0x7fe79812f8b0 2023-09-30T22:32:31.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.548+0000 7fe79e9cc700 1 -- 172.21.15.16:0/2475033191 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe79812f560 con 0x7fe79812f8b0 2023-09-30T22:32:31.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.548+0000 7fe79e9cc700 1 -- 172.21.15.16:0/2475033191 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe798119500 con 0x7fe79812f8b0 2023-09-30T22:32:31.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.549+0000 7fe794ff9700 1 -- 172.21.15.16:0/2475033191 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe78c0158f0 con 0x7fe79812f8b0 2023-09-30T22:32:31.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.549+0000 7fe794ff9700 1 -- 172.21.15.16:0/2475033191 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fe78c03fe40 con 0x7fe79812f8b0 2023-09-30T22:32:31.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.549+0000 7fe79e9cc700 1 -- 172.21.15.16:0/2475033191 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe798074ea0 con 0x7fe79812f8b0 2023-09-30T22:32:31.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.558+0000 7fe794ff9700 1 -- 172.21.15.16:0/2475033191 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe78c0132e0 con 0x7fe79812f8b0 2023-09-30T22:32:31.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.611+0000 7f10b794f700 1 -- 172.21.15.16:0/635109553 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f10b011bb70 con 0x7f10b006fb20 2023-09-30T22:32:31.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.612+0000 7f10a5ffb700 1 -- 172.21.15.16:0/635109553 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f10ac010310 con 0x7f10b006fb20 2023-09-30T22:32:31.613 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:32:31.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.615+0000 7f10b794f700 1 -- 172.21.15.16:0/635109553 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f10b006fb20 msgr2=0x7f10b01c3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:31.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.615+0000 7f10b794f700 1 --2- 172.21.15.16:0/635109553 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f10b006fb20 0x7f10b01c3f20 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f10ac00e810 tx=0x7f10ac00eb20 comp rx=0 tx=0).stop 2023-09-30T22:32:31.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.615+0000 7f10b794f700 1 -- 172.21.15.16:0/635109553 shutdown_connections 2023-09-30T22:32:31.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.615+0000 7f10b794f700 1 --2- 172.21.15.16:0/635109553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10b0130e50 0x7f10b0118cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:31.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.615+0000 7f10b794f700 1 --2- 172.21.15.16:0/635109553 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f10b006fb20 0x7f10b01c3f20 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:31.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.615+0000 7f10b794f700 1 --2- 172.21.15.16:0/635109553 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f10b0130540 0x7f10b01c4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:31.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.615+0000 7f10b794f700 1 -- 172.21.15.16:0/635109553 >> 172.21.15.16:0/635109553 conn(0x7f10b00a2aa0 msgr2=0x7f10b012d600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:31.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.615+0000 7f10b794f700 1 -- 172.21.15.16:0/635109553 shutdown_connections 2023-09-30T22:32:31.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.615+0000 7f10b794f700 1 -- 172.21.15.16:0/635109553 wait complete. 2023-09-30T22:32:31.630 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:32:31.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.692+0000 7f513efa5700 1 -- 172.21.15.16:0/3395889486 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f513809f420 con 0x7f513809e440 2023-09-30T22:32:31.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.693+0000 7f51357fa700 1 -- 172.21.15.16:0/3395889486 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f513002c030 con 0x7f513809e440 2023-09-30T22:32:31.693 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:32:31.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.695+0000 7f513efa5700 1 -- 172.21.15.16:0/3395889486 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f513809e440 msgr2=0x7f51381c8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:31.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.695+0000 7f513efa5700 1 --2- 172.21.15.16:0/3395889486 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f513809e440 0x7f51381c8090 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f513809a920 tx=0x7f5130004ad0 comp rx=0 tx=0).stop 2023-09-30T22:32:31.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.695+0000 7f513efa5700 1 -- 172.21.15.16:0/3395889486 shutdown_connections 2023-09-30T22:32:31.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.695+0000 7f513efa5700 1 --2- 172.21.15.16:0/3395889486 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f513809e440 0x7f51381c8090 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:31.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.695+0000 7f513efa5700 1 --2- 172.21.15.16:0/3395889486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f513809d6f0 0x7f51381a4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:31.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.695+0000 7f513efa5700 1 --2- 172.21.15.16:0/3395889486 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f513809e000 0x7f51381a46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:31.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.695+0000 7f513efa5700 1 -- 172.21.15.16:0/3395889486 >> 172.21.15.16:0/3395889486 conn(0x7f5138066f50 msgr2=0x7f51380a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:31.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.695+0000 7f513efa5700 1 -- 172.21.15.16:0/3395889486 shutdown_connections 2023-09-30T22:32:31.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.695+0000 7f513efa5700 1 -- 172.21.15.16:0/3395889486 wait complete. 2023-09-30T22:32:31.709 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:32:31.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.710+0000 7f8dab229700 1 -- 172.21.15.16:0/10326634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f8da411f190 con 0x7f8da4099a40 2023-09-30T22:32:31.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.710+0000 7f8da17fa700 1 -- 172.21.15.16:0/10326634 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8da411f190 con 0x7f8da4099a40 2023-09-30T22:32:31.711 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:32:31.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.713+0000 7f8dab229700 1 -- 172.21.15.16:0/10326634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8da4099a40 msgr2=0x7f8da411b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:31.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.713+0000 7f8dab229700 1 --2- 172.21.15.16:0/10326634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8da4099a40 0x7f8da411b850 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f8d98012fc0 tx=0x7f8d980100c0 comp rx=0 tx=0).stop 2023-09-30T22:32:31.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.713+0000 7f8dab229700 1 -- 172.21.15.16:0/10326634 shutdown_connections 2023-09-30T22:32:31.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.713+0000 7f8dab229700 1 --2- 172.21.15.16:0/10326634 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8da4131810 0x7f8da411c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:31.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.713+0000 7f8dab229700 1 --2- 172.21.15.16:0/10326634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8da4099a40 0x7f8da411b850 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:31.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.713+0000 7f8dab229700 1 --2- 172.21.15.16:0/10326634 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8da409a3b0 0x7f8da411bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:31.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.713+0000 7f8dab229700 1 -- 172.21.15.16:0/10326634 >> 172.21.15.16:0/10326634 conn(0x7f8da40a2aa0 msgr2=0x7f8da4130220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:31.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.713+0000 7f8dab229700 1 -- 172.21.15.16:0/10326634 shutdown_connections 2023-09-30T22:32:31.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.713+0000 7f8dab229700 1 -- 172.21.15.16:0/10326634 wait complete. 2023-09-30T22:32:31.727 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:32:31.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.785+0000 7fe79e9cc700 1 -- 172.21.15.16:0/2475033191 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe798067e80 con 0x7fe79812f8b0 2023-09-30T22:32:31.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.785+0000 7fe794ff9700 1 -- 172.21.15.16:0/2475033191 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe78c01cb00 con 0x7fe79812f8b0 2023-09-30T22:32:31.785 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:32:31.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.787+0000 7fe79e9cc700 1 -- 172.21.15.16:0/2475033191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe79812f8b0 msgr2=0x7fe7981c4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:31.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.787+0000 7fe79e9cc700 1 --2- 172.21.15.16:0/2475033191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe79812f8b0 0x7fe7981c4670 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fe78c00db70 tx=0x7fe78c00de80 comp rx=0 tx=0).stop 2023-09-30T22:32:31.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.787+0000 7fe79e9cc700 1 -- 172.21.15.16:0/2475033191 shutdown_connections 2023-09-30T22:32:31.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.787+0000 7fe79e9cc700 1 --2- 172.21.15.16:0/2475033191 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe79812cf60 0x7fe7981c4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:31.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.787+0000 7fe79e9cc700 1 --2- 172.21.15.16:0/2475033191 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe798130260 0x7fe7981c4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:31.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.787+0000 7fe79e9cc700 1 --2- 172.21.15.16:0/2475033191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe79812f8b0 0x7fe7981c4670 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:31.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.787+0000 7fe79e9cc700 1 -- 172.21.15.16:0/2475033191 >> 172.21.15.16:0/2475033191 conn(0x7fe7980a2aa0 msgr2=0x7fe79806e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:31.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.787+0000 7fe79e9cc700 1 -- 172.21.15.16:0/2475033191 shutdown_connections 2023-09-30T22:32:31.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:31.787+0000 7fe79e9cc700 1 -- 172.21.15.16:0/2475033191 wait complete. 2023-09-30T22:32:31.801 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:32:32.630 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:32:32.710 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:32:32.728 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:32:32.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.791+0000 7f6d42fbb700 1 Processor -- start 2023-09-30T22:32:32.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.791+0000 7f6d42fbb700 1 -- start start 2023-09-30T22:32:32.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.792+0000 7f6d42fbb700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6d3c130670 0x7f6d3c130a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:32.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.792+0000 7f6d42fbb700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d3c06f680 0x7f6d3c06fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:32.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.792+0000 7f6d42fbb700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d3c06ffd0 0x7f6d3c137530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:32.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.792+0000 7f6d42fbb700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d3c09a350 con 0x7f6d3c130670 2023-09-30T22:32:32.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.792+0000 7f6d42fbb700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3c130260 con 0x7f6d3c06f680 2023-09-30T22:32:32.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.792+0000 7f6d42fbb700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3c071ff0 con 0x7f6d3c06ffd0 2023-09-30T22:32:32.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.792+0000 7f6d3bfff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d3c06f680 0x7f6d3c06fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:32.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.792+0000 7f6d3bfff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d3c06f680 0x7f6d3c06fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:44910/0 (socket says 172.21.15.16:44910) 2023-09-30T22:32:32.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.792+0000 7f6d3bfff700 1 -- 172.21.15.16:0/3832260484 learned_addr learned my addr 172.21.15.16:0/3832260484 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:32.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.792+0000 7f6d41558700 1 --2- 172.21.15.16:0/3832260484 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d3c06ffd0 0x7f6d3c137530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:32.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.792+0000 7f6d40d57700 1 --2- 172.21.15.16:0/3832260484 >> v2:172.21.15.16:3300/0 conn(0x7f6d3c130670 0x7f6d3c130a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:32.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.792+0000 7f6d3bfff700 1 -- 172.21.15.16:0/3832260484 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d3c06ffd0 msgr2=0x7f6d3c137530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:32.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.792+0000 7f6d3bfff700 1 --2- 172.21.15.16:0/3832260484 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d3c06ffd0 0x7f6d3c137530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:32.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.792+0000 7f6d3bfff700 1 -- 172.21.15.16:0/3832260484 >> v2:172.21.15.16:3300/0 conn(0x7f6d3c130670 msgr2=0x7f6d3c130a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:32.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.792+0000 7f6d3bfff700 1 --2- 172.21.15.16:0/3832260484 >> v2:172.21.15.16:3300/0 conn(0x7f6d3c130670 0x7f6d3c130a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:32.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.792+0000 7f6d3bfff700 1 -- 172.21.15.16:0/3832260484 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d3c09ecf0 con 0x7f6d3c06f680 2023-09-30T22:32:32.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.792+0000 7f6d41558700 1 --2- 172.21.15.16:0/3832260484 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d3c06ffd0 0x7f6d3c137530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:32.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.792+0000 7f6d40d57700 1 --2- 172.21.15.16:0/3832260484 >> v2:172.21.15.16:3300/0 conn(0x7f6d3c130670 0x7f6d3c130a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:32.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.792+0000 7f6d3bfff700 1 --2- 172.21.15.16:0/3832260484 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d3c06f680 0x7f6d3c06fa90 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f6d28009920 tx=0x7f6d28009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21e46d8a3015edf4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:32.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.793+0000 7f6d3b7fe700 1 -- 172.21.15.16:0/3832260484 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d28011830 con 0x7f6d3c06f680 2023-09-30T22:32:32.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.793+0000 7f6d3b7fe700 1 -- 172.21.15.16:0/3832260484 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d28011990 con 0x7f6d3c06f680 2023-09-30T22:32:32.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.793+0000 7f6d3b7fe700 1 -- 172.21.15.16:0/3832260484 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d28011d20 con 0x7f6d3c06f680 2023-09-30T22:32:32.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.794+0000 7f6d42fbb700 1 -- 172.21.15.16:0/3832260484 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d3c06f680 msgr2=0x7f6d3c06fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:32.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.794+0000 7f6d42fbb700 1 --2- 172.21.15.16:0/3832260484 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d3c06f680 0x7f6d3c06fa90 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f6d28009920 tx=0x7f6d28009c30 comp rx=0 tx=0).stop 2023-09-30T22:32:32.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.794+0000 7f6d42fbb700 1 -- 172.21.15.16:0/3832260484 shutdown_connections 2023-09-30T22:32:32.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.794+0000 7f6d42fbb700 1 --2- 172.21.15.16:0/3832260484 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d3c06ffd0 0x7f6d3c137530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:32.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.794+0000 7f6d42fbb700 1 --2- 172.21.15.16:0/3832260484 >> v2:172.21.15.16:3300/0 conn(0x7f6d3c130670 0x7f6d3c130a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:32.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.794+0000 7f6d42fbb700 1 --2- 172.21.15.16:0/3832260484 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d3c06f680 0x7f6d3c06fa90 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:32.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.794+0000 7f6d42fbb700 1 -- 172.21.15.16:0/3832260484 >> 172.21.15.16:0/3832260484 conn(0x7f6d3c0a2aa0 msgr2=0x7f6d3c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:32.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.794+0000 7f6d42fbb700 1 -- 172.21.15.16:0/3832260484 shutdown_connections 2023-09-30T22:32:32.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.794+0000 7f6d42fbb700 1 -- 172.21.15.16:0/3832260484 wait complete. 2023-09-30T22:32:32.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.795+0000 7f6d42fbb700 1 Processor -- start 2023-09-30T22:32:32.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.795+0000 7f6d42fbb700 1 -- start start 2023-09-30T22:32:32.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.796+0000 7f6d42fbb700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d3c06f680 0x7f6d3c11dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:32.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.796+0000 7f6d42fbb700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d3c06ffd0 0x7f6d3c11e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:32.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.796+0000 7f6d42fbb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d3c130670 0x7f6d3c11e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:32.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.796+0000 7f6d42fbb700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3c1216f0 con 0x7f6d3c130670 2023-09-30T22:32:32.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.796+0000 7f6d42fbb700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3c11f7e0 con 0x7f6d3c06f680 2023-09-30T22:32:32.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.796+0000 7f6d42fbb700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6d3c09d1b0 con 0x7f6d3c06ffd0 2023-09-30T22:32:32.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.796+0000 7f6d3bfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d3c06ffd0 0x7f6d3c11e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:32.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.796+0000 7f6d41558700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d3c130670 0x7f6d3c11e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:32.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.796+0000 7f6d3bfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d3c06ffd0 0x7f6d3c11e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:46412/0 (socket says 172.21.15.16:46412) 2023-09-30T22:32:32.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.796+0000 7f6d41558700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d3c130670 0x7f6d3c11e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56282/0 (socket says 172.21.15.16:56282) 2023-09-30T22:32:32.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.796+0000 7f6d3bfff700 1 -- 172.21.15.16:0/524121125 learned_addr learned my addr 172.21.15.16:0/524121125 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:32.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.796+0000 7f6d40d57700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d3c06f680 0x7f6d3c11dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:32.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.796+0000 7f6d41558700 1 -- 172.21.15.16:0/524121125 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d3c06ffd0 msgr2=0x7f6d3c11e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:32.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.796+0000 7f6d41558700 1 --2- 172.21.15.16:0/524121125 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d3c06ffd0 0x7f6d3c11e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:32.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.796+0000 7f6d41558700 1 -- 172.21.15.16:0/524121125 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d3c06f680 msgr2=0x7f6d3c11dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:32.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.796+0000 7f6d41558700 1 --2- 172.21.15.16:0/524121125 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d3c06f680 0x7f6d3c11dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:32.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.796+0000 7f6d41558700 1 -- 172.21.15.16:0/524121125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d34009a70 con 0x7f6d3c130670 2023-09-30T22:32:32.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.796+0000 7f6d41558700 1 --2- 172.21.15.16:0/524121125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d3c130670 0x7f6d3c11e650 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f6d3400efc0 tx=0x7f6d3400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:32.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.797+0000 7f6d397fa700 1 -- 172.21.15.16:0/524121125 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3400cb40 con 0x7f6d3c130670 2023-09-30T22:32:32.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.797+0000 7f6d397fa700 1 -- 172.21.15.16:0/524121125 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d3400f040 con 0x7f6d3c130670 2023-09-30T22:32:32.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.797+0000 7f6d397fa700 1 -- 172.21.15.16:0/524121125 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d34003a20 con 0x7f6d3c130670 2023-09-30T22:32:32.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.797+0000 7f6d42fbb700 1 -- 172.21.15.16:0/524121125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d300094a0 con 0x7f6d3c130670 2023-09-30T22:32:32.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.797+0000 7f6d42fbb700 1 -- 172.21.15.16:0/524121125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d28009550 con 0x7f6d3c130670 2023-09-30T22:32:32.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.797+0000 7f6d397fa700 1 -- 172.21.15.16:0/524121125 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6d34014480 con 0x7f6d3c130670 2023-09-30T22:32:32.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.798+0000 7f6d397fa700 1 -- 172.21.15.16:0/524121125 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f6d3403ff50 con 0x7f6d3c130670 2023-09-30T22:32:32.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.798+0000 7f6d42fbb700 1 -- 172.21.15.16:0/524121125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d3c0a0010 con 0x7f6d3c130670 2023-09-30T22:32:32.803 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:32:32.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.803+0000 7f6d397fa700 1 -- 172.21.15.16:0/524121125 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6d34012910 con 0x7f6d3c130670 2023-09-30T22:32:32.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.864+0000 7fd3e7008700 1 Processor -- start 2023-09-30T22:32:32.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.864+0000 7fd3e7008700 1 -- start start 2023-09-30T22:32:32.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.865+0000 7fd3e7008700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd3e009d6f0 0x7fd3e009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:32.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.865+0000 7fd3e7008700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd3e009e000 0x7fd3e01339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:32.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.865+0000 7fd3e7008700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3e009b120 con 0x7fd3e009e410 2023-09-30T22:32:32.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.865+0000 7fd3e7008700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e006b460 con 0x7fd3e009e000 2023-09-30T22:32:32.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.865+0000 7fd3e7008700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e009a7e0 con 0x7fd3e009d6f0 2023-09-30T22:32:32.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.865+0000 7fd3e55a5700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd3e009e410 0x7fd3e0133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:39022/0 (socket says 172.21.15.16:39022) 2023-09-30T22:32:32.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.865+0000 7fd3e55a5700 1 -- 172.21.15.16:0/3866041233 learned_addr learned my addr 172.21.15.16:0/3866041233 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:32.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.865+0000 7fd3dffff700 1 --2- 172.21.15.16:0/3866041233 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd3e009e000 0x7fd3e01339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:32.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.865+0000 7fd3e4da4700 1 --2- 172.21.15.16:0/3866041233 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd3e009d6f0 0x7fd3e009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:32.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.866+0000 7fd3df7fe700 1 -- 172.21.15.16:0/3866041233 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 119586101 0 0) 0x7fd3e009b120 con 0x7fd3e009e410 2023-09-30T22:32:32.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.866+0000 7fd3df7fe700 1 -- 172.21.15.16:0/3866041233 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3d00035b0 con 0x7fd3e009e410 2023-09-30T22:32:32.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.866+0000 7fd3dffff700 1 -- 172.21.15.16:0/3866041233 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd3e009d6f0 msgr2=0x7fd3e009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:32.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.866+0000 7fd3dffff700 1 --2- 172.21.15.16:0/3866041233 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd3e009d6f0 0x7fd3e009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:32.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.866+0000 7fd3dffff700 1 -- 172.21.15.16:0/3866041233 >> v1:172.21.15.16:6789/0 conn(0x7fd3e009e410 legacy=0x7fd3e0133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:32.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.866+0000 7fd3dffff700 1 -- 172.21.15.16:0/3866041233 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3e009a500 con 0x7fd3e009e000 2023-09-30T22:32:32.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.866+0000 7fd3dffff700 1 --2- 172.21.15.16:0/3866041233 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd3e009e000 0x7fd3e01339f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc009870 tx=0x7fd3cc009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a75af4006b298280 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:32.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.866+0000 7fd3df7fe700 1 -- 172.21.15.16:0/3866041233 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc011830 con 0x7fd3e009e000 2023-09-30T22:32:32.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.866+0000 7fd3df7fe700 1 -- 172.21.15.16:0/3866041233 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3cc011990 con 0x7fd3e009e000 2023-09-30T22:32:32.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.866+0000 7fd3df7fe700 1 -- 172.21.15.16:0/3866041233 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc011d20 con 0x7fd3e009e000 2023-09-30T22:32:32.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.867+0000 7fd3e7008700 1 -- 172.21.15.16:0/3866041233 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd3e009e000 msgr2=0x7fd3e01339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:32.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.867+0000 7fd3e7008700 1 --2- 172.21.15.16:0/3866041233 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd3e009e000 0x7fd3e01339f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc009870 tx=0x7fd3cc009b80 comp rx=0 tx=0).stop 2023-09-30T22:32:32.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.867+0000 7fd3e7008700 1 -- 172.21.15.16:0/3866041233 shutdown_connections 2023-09-30T22:32:32.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.867+0000 7fd3e7008700 1 --2- 172.21.15.16:0/3866041233 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd3e009d6f0 0x7fd3e009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:32.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.867+0000 7fd3e7008700 1 --2- 172.21.15.16:0/3866041233 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd3e009e000 0x7fd3e01339f0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:32.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.867+0000 7fd3e7008700 1 -- 172.21.15.16:0/3866041233 >> 172.21.15.16:0/3866041233 conn(0x7fd3e0066f50 msgr2=0x7fd3e009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:32.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.867+0000 7fd3e7008700 1 -- 172.21.15.16:0/3866041233 shutdown_connections 2023-09-30T22:32:32.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.867+0000 7fd3e7008700 1 -- 172.21.15.16:0/3866041233 wait complete. 2023-09-30T22:32:32.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.868+0000 7fd3e7008700 1 Processor -- start 2023-09-30T22:32:32.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.868+0000 7fd3e7008700 1 -- start start 2023-09-30T22:32:32.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.869+0000 7fd3e7008700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd3e009d6f0 0x7fd3e0122320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:32.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.869+0000 7fd3e7008700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3e009e000 0x7fd3e0122860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:32.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.869+0000 7fd3e7008700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd3e009e410 0x7fd3e0122da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:32.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.869+0000 7fd3e7008700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e0125e40 con 0x7fd3e009e000 2023-09-30T22:32:32.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.869+0000 7fd3e7008700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e0125930 con 0x7fd3e009d6f0 2023-09-30T22:32:32.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.869+0000 7fd3e7008700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd3e0124dc0 con 0x7fd3e009e410 2023-09-30T22:32:32.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.869+0000 7fd3dffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3e009e000 0x7fd3e0122860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:32.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.869+0000 7fd3e55a5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd3e009e410 0x7fd3e0122da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:32.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.869+0000 7fd3dffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3e009e000 0x7fd3e0122860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56296/0 (socket says 172.21.15.16:56296) 2023-09-30T22:32:32.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.869+0000 7fd3dffff700 1 -- 172.21.15.16:0/1984694313 learned_addr learned my addr 172.21.15.16:0/1984694313 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:32.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.869+0000 7fd3e4da4700 1 --2- 172.21.15.16:0/1984694313 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd3e009d6f0 0x7fd3e0122320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:32.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.870+0000 7fd3e55a5700 1 -- 172.21.15.16:0/1984694313 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd3e009d6f0 msgr2=0x7fd3e0122320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:32.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.870+0000 7fd3e55a5700 1 --2- 172.21.15.16:0/1984694313 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd3e009d6f0 0x7fd3e0122320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:32.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.870+0000 7fd3e55a5700 1 -- 172.21.15.16:0/1984694313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3e009e000 msgr2=0x7fd3e0122860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:32.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.870+0000 7fd3e55a5700 1 --2- 172.21.15.16:0/1984694313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3e009e000 0x7fd3e0122860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:32.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.870+0000 7fd3e55a5700 1 -- 172.21.15.16:0/1984694313 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3cc0094a0 con 0x7fd3e009e410 2023-09-30T22:32:32.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.870+0000 7fd3dffff700 1 --2- 172.21.15.16:0/1984694313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3e009e000 0x7fd3e0122860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:32.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.870+0000 7fd3e4da4700 1 --2- 172.21.15.16:0/1984694313 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd3e009d6f0 0x7fd3e0122320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:32.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.870+0000 7fd3e55a5700 1 --2- 172.21.15.16:0/1984694313 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd3e009e410 0x7fd3e0122da0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd3d800ca50 tx=0x7fd3d800cd60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:32.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.870+0000 7fd3dd7fa700 1 -- 172.21.15.16:0/1984694313 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d800a9e0 con 0x7fd3e009e410 2023-09-30T22:32:32.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.870+0000 7fd3dd7fa700 1 -- 172.21.15.16:0/1984694313 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3d800ab40 con 0x7fd3e009e410 2023-09-30T22:32:32.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.870+0000 7fd3e7008700 1 -- 172.21.15.16:0/1984694313 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3e009c710 con 0x7fd3e009e410 2023-09-30T22:32:32.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.870+0000 7fd3dd7fa700 1 -- 172.21.15.16:0/1984694313 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3d8012400 con 0x7fd3e009e410 2023-09-30T22:32:32.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.870+0000 7fd3e7008700 1 -- 172.21.15.16:0/1984694313 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3e00720b0 con 0x7fd3e009e410 2023-09-30T22:32:32.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.871+0000 7fd3dd7fa700 1 -- 172.21.15.16:0/1984694313 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd3d801b3b0 con 0x7fd3e009e410 2023-09-30T22:32:32.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.871+0000 7fd3e7008700 1 -- 172.21.15.16:0/1984694313 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3e011f030 con 0x7fd3e009e410 2023-09-30T22:32:32.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.872+0000 7fd3dd7fa700 1 -- 172.21.15.16:0/1984694313 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd3d803f340 con 0x7fd3e009e410 2023-09-30T22:32:32.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.881+0000 7fd3dd7fa700 1 -- 172.21.15.16:0/1984694313 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd3d8017080 con 0x7fd3e009e410 2023-09-30T22:32:32.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.905+0000 7fe55d979700 1 Processor -- start 2023-09-30T22:32:32.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.905+0000 7fe55d979700 1 -- start start 2023-09-30T22:32:32.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.906+0000 7fe55d979700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe55809d6f0 0x7fe55809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:32.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.906+0000 7fe55d979700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe55809e000 0x7fe558130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:32.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.906+0000 7fe55d979700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe55809e410 0x7fe558130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:32.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.906+0000 7fe55d979700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe55809f520 con 0x7fe55809e410 2023-09-30T22:32:32.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.906+0000 7fe55d979700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe55809f130 con 0x7fe55809e000 2023-09-30T22:32:32.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.906+0000 7fe55d979700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe55809e930 con 0x7fe55809d6f0 2023-09-30T22:32:32.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.906+0000 7fe5567fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe55809e000 0x7fe558130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:32.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.906+0000 7fe5577fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe55809e410 0x7fe558130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:32.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.906+0000 7fe5577fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe55809e410 0x7fe558130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56308/0 (socket says 172.21.15.16:56308) 2023-09-30T22:32:32.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.907+0000 7fe5577fe700 1 -- 172.21.15.16:0/2097481176 learned_addr learned my addr 172.21.15.16:0/2097481176 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:32.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.907+0000 7fe556ffd700 1 --2- 172.21.15.16:0/2097481176 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe55809d6f0 0x7fe55809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:32.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.906+0000 7fe5567fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe55809e000 0x7fe558130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:44956/0 (socket says 172.21.15.16:44956) 2023-09-30T22:32:32.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.907+0000 7fe5577fe700 1 -- 172.21.15.16:0/2097481176 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe55809d6f0 msgr2=0x7fe55809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:32.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.907+0000 7fe5577fe700 1 --2- 172.21.15.16:0/2097481176 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe55809d6f0 0x7fe55809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:32.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.907+0000 7fe5577fe700 1 -- 172.21.15.16:0/2097481176 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe55809e000 msgr2=0x7fe558130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:32.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.907+0000 7fe5577fe700 1 --2- 172.21.15.16:0/2097481176 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe55809e000 0x7fe558130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:32.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.907+0000 7fe5577fe700 1 -- 172.21.15.16:0/2097481176 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe55809b2d0 con 0x7fe55809e410 2023-09-30T22:32:32.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.907+0000 7fe5567fc700 1 --2- 172.21.15.16:0/2097481176 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe55809e000 0x7fe558130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:32.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.907+0000 7fe5577fe700 1 --2- 172.21.15.16:0/2097481176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe55809e410 0x7fe558130ba0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fe55000bef0 tx=0x7fe5500093b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=689d6458c7d9fea2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:32.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.907+0000 7fe555ffb700 1 -- 172.21.15.16:0/2097481176 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55000b4e0 con 0x7fe55809e410 2023-09-30T22:32:32.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.907+0000 7fe555ffb700 1 -- 172.21.15.16:0/2097481176 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe55000c950 con 0x7fe55809e410 2023-09-30T22:32:32.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.907+0000 7fe555ffb700 1 -- 172.21.15.16:0/2097481176 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe55000cd00 con 0x7fe55809e410 2023-09-30T22:32:32.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.908+0000 7fe55d979700 1 -- 172.21.15.16:0/2097481176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe55809e410 msgr2=0x7fe558130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:32.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.908+0000 7fe55d979700 1 --2- 172.21.15.16:0/2097481176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe55809e410 0x7fe558130ba0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fe55000bef0 tx=0x7fe5500093b0 comp rx=0 tx=0).stop 2023-09-30T22:32:32.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.909+0000 7fe55d979700 1 -- 172.21.15.16:0/2097481176 shutdown_connections 2023-09-30T22:32:32.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.909+0000 7fe55d979700 1 --2- 172.21.15.16:0/2097481176 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe55809e410 0x7fe558130ba0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:32.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.909+0000 7fe55d979700 1 --2- 172.21.15.16:0/2097481176 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe55809d6f0 0x7fe55809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:32.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.909+0000 7fe55d979700 1 --2- 172.21.15.16:0/2097481176 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe55809e000 0x7fe558130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:32.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.909+0000 7fe55d979700 1 -- 172.21.15.16:0/2097481176 >> 172.21.15.16:0/2097481176 conn(0x7fe558066f50 msgr2=0x7fe55809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:32.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.909+0000 7fe55d979700 1 -- 172.21.15.16:0/2097481176 shutdown_connections 2023-09-30T22:32:32.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.909+0000 7fe55d979700 1 -- 172.21.15.16:0/2097481176 wait complete. 2023-09-30T22:32:32.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.910+0000 7fe55d979700 1 Processor -- start 2023-09-30T22:32:32.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.910+0000 7fe55d979700 1 -- start start 2023-09-30T22:32:32.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.911+0000 7fe55d979700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe55809d6f0 0x7fe55811e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:32.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.911+0000 7fe55d979700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe55809e000 0x7fe55811e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:32.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.911+0000 7fe55d979700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe55809e410 0x7fe55811eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:32.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.911+0000 7fe55d979700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe55809c770 con 0x7fe55809e000 2023-09-30T22:32:32.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.911+0000 7fe55d979700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5580a1920 con 0x7fe55809d6f0 2023-09-30T22:32:32.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.911+0000 7fe55d979700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe5581201f0 con 0x7fe55809e410 2023-09-30T22:32:32.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.911+0000 7fe5567fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe55809e000 0x7fe55811e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:32.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.911+0000 7fe5567fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe55809e000 0x7fe55811e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56320/0 (socket says 172.21.15.16:56320) 2023-09-30T22:32:32.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.911+0000 7fe5567fc700 1 -- 172.21.15.16:0/1074300050 learned_addr learned my addr 172.21.15.16:0/1074300050 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:32.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.911+0000 7fe5577fe700 1 --2- 172.21.15.16:0/1074300050 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe55809e410 0x7fe55811eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:32.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.912+0000 7fe5567fc700 1 -- 172.21.15.16:0/1074300050 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe55809e410 msgr2=0x7fe55811eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:32.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.912+0000 7fe5567fc700 1 --2- 172.21.15.16:0/1074300050 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe55809e410 0x7fe55811eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:32.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.912+0000 7fe556ffd700 1 --2- 172.21.15.16:0/1074300050 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe55809d6f0 0x7fe55811e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:32.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.912+0000 7fe5567fc700 1 -- 172.21.15.16:0/1074300050 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe55809d6f0 msgr2=0x7fe55811e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:32.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.912+0000 7fe5567fc700 1 --2- 172.21.15.16:0/1074300050 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe55809d6f0 0x7fe55811e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:32.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.912+0000 7fe5567fc700 1 -- 172.21.15.16:0/1074300050 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe54c00d940 con 0x7fe55809e000 2023-09-30T22:32:32.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.912+0000 7fe5577fe700 1 --2- 172.21.15.16:0/1074300050 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe55809e410 0x7fe55811eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:32:32.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.912+0000 7fe556ffd700 1 --2- 172.21.15.16:0/1074300050 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe55809d6f0 0x7fe55811e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:32:32.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.912+0000 7fe5567fc700 1 --2- 172.21.15.16:0/1074300050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe55809e000 0x7fe55811e570 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fe54c012f90 tx=0x7fe54c0103a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:32.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.912+0000 7fe53ffff700 1 -- 172.21.15.16:0/1074300050 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54c010bb0 con 0x7fe55809e000 2023-09-30T22:32:32.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.912+0000 7fe53ffff700 1 -- 172.21.15.16:0/1074300050 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe54c013040 con 0x7fe55809e000 2023-09-30T22:32:32.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.912+0000 7fe55d979700 1 -- 172.21.15.16:0/1074300050 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe55000bb50 con 0x7fe55809e000 2023-09-30T22:32:32.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.912+0000 7fe53ffff700 1 -- 172.21.15.16:0/1074300050 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54c00d2e0 con 0x7fe55809e000 2023-09-30T22:32:32.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.912+0000 7fe55d979700 1 -- 172.21.15.16:0/1074300050 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5581215b0 con 0x7fe55809e000 2023-09-30T22:32:32.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.913+0000 7fe53ffff700 1 -- 172.21.15.16:0/1074300050 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe54c018480 con 0x7fe55809e000 2023-09-30T22:32:32.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.914+0000 7fe53ffff700 1 -- 172.21.15.16:0/1074300050 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fe54c043c20 con 0x7fe55809e000 2023-09-30T22:32:32.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.914+0000 7fe55d979700 1 -- 172.21.15.16:0/1074300050 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe558074ea0 con 0x7fe55809e000 2023-09-30T22:32:32.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.923+0000 7fe53ffff700 1 -- 172.21.15.16:0/1074300050 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe54c020950 con 0x7fe55809e000 2023-09-30T22:32:32.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.954+0000 7f787df39700 1 Processor -- start 2023-09-30T22:32:32.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.954+0000 7f787df39700 1 -- start start 2023-09-30T22:32:32.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.954+0000 7f787df39700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7878133420 0x7f78781337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:32.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.954+0000 7f787df39700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7878133e30 0x7f787812ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:32.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.954+0000 7f787df39700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7878134260 0x7f78781304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:32.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.954+0000 7f787df39700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f787806b990 con 0x7f7878133e30 2023-09-30T22:32:32.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.954+0000 7f787df39700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78780a0590 con 0x7f7878134260 2023-09-30T22:32:32.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.954+0000 7f787df39700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f787809fb80 con 0x7f7878133420 2023-09-30T22:32:32.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.955+0000 7f7876ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7878133e30 0x7f787812ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:32.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.955+0000 7f7876ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7878133e30 0x7f787812ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56328/0 (socket says 172.21.15.16:56328) 2023-09-30T22:32:32.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.955+0000 7f7876ffd700 1 -- 172.21.15.16:0/3123181511 learned_addr learned my addr 172.21.15.16:0/3123181511 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:32.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.955+0000 7f7876ffd700 1 -- 172.21.15.16:0/3123181511 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7878133420 msgr2=0x7f78781337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:32.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.955+0000 7f78777fe700 1 --2- 172.21.15.16:0/3123181511 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7878133420 0x7f78781337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:32.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.955+0000 7f7876ffd700 1 --2- 172.21.15.16:0/3123181511 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7878133420 0x7f78781337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:32.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.955+0000 7f7876ffd700 1 -- 172.21.15.16:0/3123181511 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7878134260 msgr2=0x7f78781304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:32.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.955+0000 7f7876ffd700 1 --2- 172.21.15.16:0/3123181511 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7878134260 0x7f78781304d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:32.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.955+0000 7f7876ffd700 1 -- 172.21.15.16:0/3123181511 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78780a0030 con 0x7f7878133e30 2023-09-30T22:32:32.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.955+0000 7f78777fe700 1 --2- 172.21.15.16:0/3123181511 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7878133420 0x7f78781337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:32:32.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.955+0000 7f7876ffd700 1 --2- 172.21.15.16:0/3123181511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7878133e30 0x7f787812ff90 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f786c0098f0 tx=0x7f786c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2878a7057058c4e7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:32.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.955+0000 7f78767fc700 1 -- 172.21.15.16:0/3123181511 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786c00b8a0 con 0x7f7878133e30 2023-09-30T22:32:32.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.956+0000 7f78767fc700 1 -- 172.21.15.16:0/3123181511 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f786c00ba00 con 0x7f7878133e30 2023-09-30T22:32:32.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.956+0000 7f78767fc700 1 -- 172.21.15.16:0/3123181511 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786c00bd90 con 0x7f7878133e30 2023-09-30T22:32:32.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.956+0000 7f787df39700 1 -- 172.21.15.16:0/3123181511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7878133e30 msgr2=0x7f787812ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:32.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.956+0000 7f787df39700 1 --2- 172.21.15.16:0/3123181511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7878133e30 0x7f787812ff90 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f786c0098f0 tx=0x7f786c009c00 comp rx=0 tx=0).stop 2023-09-30T22:32:32.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.956+0000 7f787df39700 1 -- 172.21.15.16:0/3123181511 shutdown_connections 2023-09-30T22:32:32.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.956+0000 7f787df39700 1 --2- 172.21.15.16:0/3123181511 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7878134260 0x7f78781304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:32.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.956+0000 7f787df39700 1 --2- 172.21.15.16:0/3123181511 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7878133420 0x7f78781337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:32.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.956+0000 7f787df39700 1 --2- 172.21.15.16:0/3123181511 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7878133e30 0x7f787812ff90 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:32.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.956+0000 7f787df39700 1 -- 172.21.15.16:0/3123181511 >> 172.21.15.16:0/3123181511 conn(0x7f78780a4210 msgr2=0x7f78780a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:32.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.956+0000 7f787df39700 1 -- 172.21.15.16:0/3123181511 shutdown_connections 2023-09-30T22:32:32.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.956+0000 7f787df39700 1 -- 172.21.15.16:0/3123181511 wait complete. 2023-09-30T22:32:32.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.957+0000 7f787df39700 1 Processor -- start 2023-09-30T22:32:32.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.958+0000 7f787df39700 1 -- start start 2023-09-30T22:32:32.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.958+0000 7f787df39700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7878133420 0x7f787809c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:32.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.958+0000 7f787df39700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7878133e30 0x7f787809c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:32.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.958+0000 7f787df39700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7878134260 0x7f787809cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:32.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.958+0000 7f787df39700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787811ffb0 con 0x7f7878134260 2023-09-30T22:32:32.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.958+0000 7f787df39700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7878120190 con 0x7f7878133e30 2023-09-30T22:32:32.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.958+0000 7f787df39700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f787812da40 con 0x7f7878133420 2023-09-30T22:32:32.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.958+0000 7f78777fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7878133420 0x7f787809c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:32.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.958+0000 7f78777fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7878133420 0x7f787809c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:46464/0 (socket says 172.21.15.16:46464) 2023-09-30T22:32:32.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.958+0000 7f78777fe700 1 -- 172.21.15.16:0/379711512 learned_addr learned my addr 172.21.15.16:0/379711512 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:32.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.958+0000 7f7876ffd700 1 --2- 172.21.15.16:0/379711512 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7878133e30 0x7f787809c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:32.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.958+0000 7f7877fff700 1 --2- 172.21.15.16:0/379711512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7878134260 0x7f787809cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:32.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.959+0000 7f7876ffd700 1 -- 172.21.15.16:0/379711512 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7878133420 msgr2=0x7f787809c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:32.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.959+0000 7f7876ffd700 1 --2- 172.21.15.16:0/379711512 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7878133420 0x7f787809c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:32.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.959+0000 7f7876ffd700 1 -- 172.21.15.16:0/379711512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7878134260 msgr2=0x7f787809cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:32.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.959+0000 7f7876ffd700 1 --2- 172.21.15.16:0/379711512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7878134260 0x7f787809cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:32.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.959+0000 7f7876ffd700 1 -- 172.21.15.16:0/379711512 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f786c009550 con 0x7f7878133e30 2023-09-30T22:32:32.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.959+0000 7f7876ffd700 1 --2- 172.21.15.16:0/379711512 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7878133e30 0x7f787809c9c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f786c016000 tx=0x7f786c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:32.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.959+0000 7f785ffff700 1 -- 172.21.15.16:0/379711512 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786c004300 con 0x7f7878133e30 2023-09-30T22:32:32.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.959+0000 7f785ffff700 1 -- 172.21.15.16:0/379711512 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f786c01f070 con 0x7f7878133e30 2023-09-30T22:32:32.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.959+0000 7f787df39700 1 -- 172.21.15.16:0/379711512 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f787811fbf0 con 0x7f7878133e30 2023-09-30T22:32:32.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.959+0000 7f785ffff700 1 -- 172.21.15.16:0/379711512 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786c0036a0 con 0x7f7878133e30 2023-09-30T22:32:32.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.959+0000 7f787df39700 1 -- 172.21.15.16:0/379711512 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f787811f520 con 0x7f7878133e30 2023-09-30T22:32:32.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.960+0000 7f787df39700 1 -- 172.21.15.16:0/379711512 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f787811fa40 con 0x7f7878133e30 2023-09-30T22:32:32.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.961+0000 7f785ffff700 1 -- 172.21.15.16:0/379711512 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f786c021030 con 0x7f7878133e30 2023-09-30T22:32:32.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.961+0000 7f785ffff700 1 -- 172.21.15.16:0/379711512 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f786c040780 con 0x7f7878133e30 2023-09-30T22:32:32.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:32.970+0000 7f785ffff700 1 -- 172.21.15.16:0/379711512 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f786c02e830 con 0x7f7878133e30 2023-09-30T22:32:33.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:33.020+0000 7f6d42fbb700 1 -- 172.21.15.16:0/524121125 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6d3c074ea0 con 0x7f6d3c130670 2023-09-30T22:32:33.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:33.021+0000 7f6d397fa700 1 -- 172.21.15.16:0/524121125 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6d3402b090 con 0x7f6d3c130670 2023-09-30T22:32:33.021 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:32:33.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:33.024+0000 7f6d42fbb700 1 -- 172.21.15.16:0/524121125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d3c130670 msgr2=0x7f6d3c11e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:33.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:33.024+0000 7f6d42fbb700 1 --2- 172.21.15.16:0/524121125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d3c130670 0x7f6d3c11e650 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f6d3400efc0 tx=0x7f6d3400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:32:33.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:33.024+0000 7f6d42fbb700 1 -- 172.21.15.16:0/524121125 shutdown_connections 2023-09-30T22:32:33.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:33.024+0000 7f6d42fbb700 1 --2- 172.21.15.16:0/524121125 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d3c06f680 0x7f6d3c11dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:33.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:33.024+0000 7f6d42fbb700 1 --2- 172.21.15.16:0/524121125 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d3c130670 0x7f6d3c11e650 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:33.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:33.024+0000 7f6d42fbb700 1 --2- 172.21.15.16:0/524121125 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d3c06ffd0 0x7f6d3c11e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:33.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:33.024+0000 7f6d42fbb700 1 -- 172.21.15.16:0/524121125 >> 172.21.15.16:0/524121125 conn(0x7f6d3c0a2aa0 msgr2=0x7f6d3c06e570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:33.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:33.024+0000 7f6d42fbb700 1 -- 172.21.15.16:0/524121125 shutdown_connections 2023-09-30T22:32:33.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:33.024+0000 7f6d42fbb700 1 -- 172.21.15.16:0/524121125 wait complete. 2023-09-30T22:32:33.040 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:32:33.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:33.097+0000 7fd3e7008700 1 -- 172.21.15.16:0/1984694313 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd3e0133750 con 0x7fd3e009e410 2023-09-30T22:32:33.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:33.097+0000 7fd3dd7fa700 1 -- 172.21.15.16:0/1984694313 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd3d80192c0 con 0x7fd3e009e410 2023-09-30T22:32:33.098 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:32:33.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:33.100+0000 7fd3e7008700 1 -- 172.21.15.16:0/1984694313 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd3e009e410 msgr2=0x7fd3e0122da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:33.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:33.100+0000 7fd3e7008700 1 --2- 172.21.15.16:0/1984694313 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd3e009e410 0x7fd3e0122da0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd3d800ca50 tx=0x7fd3d800cd60 comp rx=0 tx=0).stop 2023-09-30T22:32:33.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:33.100+0000 7fd3e7008700 1 -- 172.21.15.16:0/1984694313 shutdown_connections 2023-09-30T22:32:33.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:33.100+0000 7fd3e7008700 1 --2- 172.21.15.16:0/1984694313 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd3e009d6f0 0x7fd3e0122320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:33.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:33.100+0000 7fd3e7008700 1 --2- 172.21.15.16:0/1984694313 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd3e009e410 0x7fd3e0122da0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:33.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:33.100+0000 7fd3e7008700 1 --2- 172.21.15.16:0/1984694313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3e009e000 0x7fd3e0122860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:33.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:33.100+0000 7fd3e7008700 1 -- 172.21.15.16:0/1984694313 >> 172.21.15.16:0/1984694313 conn(0x7fd3e0066f50 msgr2=0x7fd3e0133f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:33.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:33.100+0000 7fd3e7008700 1 -- 172.21.15.16:0/1984694313 shutdown_connections 2023-09-30T22:32:33.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:33.100+0000 7fd3e7008700 1 -- 172.21.15.16:0/1984694313 wait complete. 2023-09-30T22:32:33.114 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:32:33.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:33.141+0000 7fe55d979700 1 -- 172.21.15.16:0/1074300050 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe558121340 con 0x7fe55809e000 2023-09-30T22:32:33.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:33.142+0000 7fe53ffff700 1 -- 172.21.15.16:0/1074300050 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe54c020360 con 0x7fe55809e000 2023-09-30T22:32:33.142 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:32:33.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:33.144+0000 7fe55d979700 1 -- 172.21.15.16:0/1074300050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe55809e000 msgr2=0x7fe55811e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:33.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:33.144+0000 7fe55d979700 1 --2- 172.21.15.16:0/1074300050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe55809e000 0x7fe55811e570 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fe54c012f90 tx=0x7fe54c0103a0 comp rx=0 tx=0).stop 2023-09-30T22:32:33.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:33.144+0000 7fe55d979700 1 -- 172.21.15.16:0/1074300050 shutdown_connections 2023-09-30T22:32:33.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:33.144+0000 7fe55d979700 1 --2- 172.21.15.16:0/1074300050 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe55809d6f0 0x7fe55811e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:33.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:33.144+0000 7fe55d979700 1 --2- 172.21.15.16:0/1074300050 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe55809e410 0x7fe55811eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:33.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:33.144+0000 7fe55d979700 1 --2- 172.21.15.16:0/1074300050 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe55809e000 0x7fe55811e570 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:33.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:33.144+0000 7fe55d979700 1 -- 172.21.15.16:0/1074300050 >> 172.21.15.16:0/1074300050 conn(0x7fe558066f50 msgr2=0x7fe55806a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:33.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:33.144+0000 7fe55d979700 1 -- 172.21.15.16:0/1074300050 shutdown_connections 2023-09-30T22:32:33.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:33.144+0000 7fe55d979700 1 -- 172.21.15.16:0/1074300050 wait complete. 2023-09-30T22:32:33.158 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:32:33.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:33.181+0000 7f787df39700 1 -- 172.21.15.16:0/379711512 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f78781209e0 con 0x7f7878133e30 2023-09-30T22:32:33.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:33.181+0000 7f785ffff700 1 -- 172.21.15.16:0/379711512 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f786c029030 con 0x7f7878133e30 2023-09-30T22:32:33.182 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:32:33.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:33.184+0000 7f787df39700 1 -- 172.21.15.16:0/379711512 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7878133e30 msgr2=0x7f787809c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:33.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:33.184+0000 7f787df39700 1 --2- 172.21.15.16:0/379711512 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7878133e30 0x7f787809c9c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f786c016000 tx=0x7f786c004750 comp rx=0 tx=0).stop 2023-09-30T22:32:33.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:33.184+0000 7f787df39700 1 -- 172.21.15.16:0/379711512 shutdown_connections 2023-09-30T22:32:33.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:33.184+0000 7f787df39700 1 --2- 172.21.15.16:0/379711512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7878134260 0x7f787809cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:33.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:33.184+0000 7f787df39700 1 --2- 172.21.15.16:0/379711512 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7878133420 0x7f787809c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:33.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:33.184+0000 7f787df39700 1 --2- 172.21.15.16:0/379711512 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7878133e30 0x7f787809c9c0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:33.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:33.184+0000 7f787df39700 1 -- 172.21.15.16:0/379711512 >> 172.21.15.16:0/379711512 conn(0x7f78780a4210 msgr2=0x7f78780a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:33.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:33.184+0000 7f787df39700 1 -- 172.21.15.16:0/379711512 shutdown_connections 2023-09-30T22:32:33.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:33.184+0000 7f787df39700 1 -- 172.21.15.16:0/379711512 wait complete. 2023-09-30T22:32:33.198 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:32:34.041 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:32:34.114 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:32:34.159 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:32:34.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.191+0000 7ffb9de45700 1 Processor -- start 2023-09-30T22:32:34.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.191+0000 7ffb9de45700 1 -- start start 2023-09-30T22:32:34.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.191+0000 7ffb9de45700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffb98136710 0x7ffb98138af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:34.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.191+0000 7ffb9de45700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffb98139030 0x7ffb9813b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:34.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.191+0000 7ffb9de45700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb9806ef30 con 0x7ffb981338a0 2023-09-30T22:32:34.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.191+0000 7ffb9de45700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9806fef0 con 0x7ffb98139030 2023-09-30T22:32:34.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.191+0000 7ffb9de45700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffb981334b0 con 0x7ffb98136710 2023-09-30T22:32:34.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.192+0000 7ffb97fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffb98139030 0x7ffb9813b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:34.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.192+0000 7ffb977fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ffb981338a0 0x7ffb98133c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:39038/0 (socket says 172.21.15.16:39038) 2023-09-30T22:32:34.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.192+0000 7ffb977fe700 1 -- 172.21.15.16:0/1247955240 learned_addr learned my addr 172.21.15.16:0/1247955240 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:34.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.192+0000 7ffb96ffd700 1 --2- 172.21.15.16:0/1247955240 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffb98136710 0x7ffb98138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:34.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.192+0000 7ffb967fc700 1 -- 172.21.15.16:0/1247955240 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3340241949 0 0) 0x7ffb9806ef30 con 0x7ffb981338a0 2023-09-30T22:32:34.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.192+0000 7ffb967fc700 1 -- 172.21.15.16:0/1247955240 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb800035b0 con 0x7ffb981338a0 2023-09-30T22:32:34.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.192+0000 7ffb97fff700 1 -- 172.21.15.16:0/1247955240 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffb98136710 msgr2=0x7ffb98138af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:34.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.192+0000 7ffb97fff700 1 --2- 172.21.15.16:0/1247955240 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffb98136710 0x7ffb98138af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:34.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.192+0000 7ffb967fc700 1 -- 172.21.15.16:0/1247955240 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 649537194 0 0) 0x7ffb800035b0 con 0x7ffb981338a0 2023-09-30T22:32:34.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.192+0000 7ffb97fff700 1 -- 172.21.15.16:0/1247955240 >> v1:172.21.15.16:6789/0 conn(0x7ffb981338a0 legacy=0x7ffb98133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:34.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.192+0000 7ffb97fff700 1 -- 172.21.15.16:0/1247955240 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb9806fc30 con 0x7ffb98139030 2023-09-30T22:32:34.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.192+0000 7ffb96ffd700 1 --2- 172.21.15.16:0/1247955240 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffb98136710 0x7ffb98138af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:34.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.192+0000 7ffb97fff700 1 --2- 172.21.15.16:0/1247955240 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffb98139030 0x7ffb9813b410 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ffb90009dc0 tx=0x7ffb9000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f5834353236d9c5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:34.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.193+0000 7ffb967fc700 1 -- 172.21.15.16:0/1247955240 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb9000e070 con 0x7ffb98139030 2023-09-30T22:32:34.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.193+0000 7ffb967fc700 1 -- 172.21.15.16:0/1247955240 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb90003a20 con 0x7ffb98139030 2023-09-30T22:32:34.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.193+0000 7ffb967fc700 1 -- 172.21.15.16:0/1247955240 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb88001670 con 0x7ffb98139030 2023-09-30T22:32:34.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.193+0000 7ffb9de45700 1 -- 172.21.15.16:0/1247955240 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffb98139030 msgr2=0x7ffb9813b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:34.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.193+0000 7ffb9de45700 1 --2- 172.21.15.16:0/1247955240 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffb98139030 0x7ffb9813b410 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ffb90009dc0 tx=0x7ffb9000b330 comp rx=0 tx=0).stop 2023-09-30T22:32:34.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.194+0000 7ffb9de45700 1 -- 172.21.15.16:0/1247955240 shutdown_connections 2023-09-30T22:32:34.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.194+0000 7ffb9de45700 1 --2- 172.21.15.16:0/1247955240 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffb98139030 0x7ffb9813b410 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:34.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.194+0000 7ffb9de45700 1 --2- 172.21.15.16:0/1247955240 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffb98136710 0x7ffb98138af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:34.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.194+0000 7ffb9de45700 1 -- 172.21.15.16:0/1247955240 >> 172.21.15.16:0/1247955240 conn(0x7ffb980a2aa0 msgr2=0x7ffb980a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:34.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.194+0000 7ffb9de45700 1 -- 172.21.15.16:0/1247955240 shutdown_connections 2023-09-30T22:32:34.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.194+0000 7ffb9de45700 1 -- 172.21.15.16:0/1247955240 wait complete. 2023-09-30T22:32:34.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.195+0000 7ffb9de45700 1 Processor -- start 2023-09-30T22:32:34.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.195+0000 7ffb9de45700 1 -- start start 2023-09-30T22:32:34.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.195+0000 7ffb9de45700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb981338a0 0x7ffb9811adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:34.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.195+0000 7ffb9de45700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffb98136710 0x7ffb9811b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:34.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.195+0000 7ffb9de45700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffb98139030 0x7ffb9811b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:34.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.195+0000 7ffb9de45700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb9811cae0 con 0x7ffb981338a0 2023-09-30T22:32:34.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.195+0000 7ffb9de45700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb98130cb0 con 0x7ffb98136710 2023-09-30T22:32:34.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.195+0000 7ffb9de45700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffb98138cb0 con 0x7ffb98139030 2023-09-30T22:32:34.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.196+0000 7ffb977fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb981338a0 0x7ffb9811adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:34.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.196+0000 7ffb96ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffb98136710 0x7ffb9811b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:34.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.196+0000 7ffb977fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb981338a0 0x7ffb9811adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56354/0 (socket says 172.21.15.16:56354) 2023-09-30T22:32:34.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.196+0000 7ffb977fe700 1 -- 172.21.15.16:0/1442138544 learned_addr learned my addr 172.21.15.16:0/1442138544 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:34.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.196+0000 7ffb97fff700 1 --2- 172.21.15.16:0/1442138544 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffb98139030 0x7ffb9811b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:34.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.196+0000 7ffb977fe700 1 -- 172.21.15.16:0/1442138544 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffb98139030 msgr2=0x7ffb9811b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:34.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.196+0000 7ffb977fe700 1 --2- 172.21.15.16:0/1442138544 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffb98139030 0x7ffb9811b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:34.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.196+0000 7ffb977fe700 1 -- 172.21.15.16:0/1442138544 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffb98136710 msgr2=0x7ffb9811b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:34.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.196+0000 7ffb977fe700 1 --2- 172.21.15.16:0/1442138544 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffb98136710 0x7ffb9811b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:34.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.196+0000 7ffb977fe700 1 -- 172.21.15.16:0/1442138544 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb8c0094a0 con 0x7ffb981338a0 2023-09-30T22:32:34.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.196+0000 7ffb96ffd700 1 --2- 172.21.15.16:0/1442138544 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffb98136710 0x7ffb9811b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:34.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.196+0000 7ffb977fe700 1 --2- 172.21.15.16:0/1442138544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb981338a0 0x7ffb9811adc0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7ffb8800c750 tx=0x7ffb8800ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:34.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.196+0000 7ffb7ffff700 1 -- 172.21.15.16:0/1442138544 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb88004d40 con 0x7ffb981338a0 2023-09-30T22:32:34.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.197+0000 7ffb7ffff700 1 -- 172.21.15.16:0/1442138544 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb880070d0 con 0x7ffb981338a0 2023-09-30T22:32:34.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.197+0000 7ffb9de45700 1 -- 172.21.15.16:0/1442138544 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb90009a70 con 0x7ffb981338a0 2023-09-30T22:32:34.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.197+0000 7ffb7ffff700 1 -- 172.21.15.16:0/1442138544 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb88004ea0 con 0x7ffb981338a0 2023-09-30T22:32:34.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.197+0000 7ffb9de45700 1 -- 172.21.15.16:0/1442138544 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb9811e600 con 0x7ffb981338a0 2023-09-30T22:32:34.200 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:32:34.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.197+0000 7ffb7ffff700 1 -- 172.21.15.16:0/1442138544 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ffb88007480 con 0x7ffb981338a0 2023-09-30T22:32:34.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.197+0000 7ffb7ffff700 1 -- 172.21.15.16:0/1442138544 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7ffb8803f500 con 0x7ffb981338a0 2023-09-30T22:32:34.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.198+0000 7ffb9de45700 1 -- 172.21.15.16:0/1442138544 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb9811e250 con 0x7ffb981338a0 2023-09-30T22:32:34.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.203+0000 7ffb7ffff700 1 -- 172.21.15.16:0/1442138544 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ffb880123b0 con 0x7ffb981338a0 2023-09-30T22:32:34.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.263+0000 7f4be8033700 1 Processor -- start 2023-09-30T22:32:34.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.263+0000 7f4be8033700 1 -- start start 2023-09-30T22:32:34.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.264+0000 7f4be8033700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4be01338c0 0x7f4be0133c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:34.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.264+0000 7f4be8033700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4be01341d0 0x7f4be009de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:34.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.264+0000 7f4be8033700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4be006f720 con 0x7f4be009e390 2023-09-30T22:32:34.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.264+0000 7f4be8033700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4be0070c80 con 0x7f4be01341d0 2023-09-30T22:32:34.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.264+0000 7f4be8033700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4be006fce0 con 0x7f4be01338c0 2023-09-30T22:32:34.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.264+0000 7f4be55ce700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4be01341d0 0x7f4be009de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:34.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.264+0000 7f4be5dcf700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4be01338c0 0x7f4be0133c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:34.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.264+0000 7f4be55ce700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4be01341d0 0x7f4be009de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56364/0 (socket says 172.21.15.16:56364) 2023-09-30T22:32:34.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.264+0000 7f4be55ce700 1 -- 172.21.15.16:0/2030847170 learned_addr learned my addr 172.21.15.16:0/2030847170 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:34.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.265+0000 7f4be5dcf700 1 -- 172.21.15.16:0/2030847170 >> v2:172.21.15.16:3300/0 conn(0x7f4be01341d0 msgr2=0x7f4be009de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:34.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.265+0000 7f4be5dcf700 1 --2- 172.21.15.16:0/2030847170 >> v2:172.21.15.16:3300/0 conn(0x7f4be01341d0 0x7f4be009de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:34.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.265+0000 7f4be5dcf700 1 -- 172.21.15.16:0/2030847170 >> v1:172.21.15.16:6789/0 conn(0x7f4be009e390 legacy=0x7f4be013a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:34.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.265+0000 7f4be5dcf700 1 -- 172.21.15.16:0/2030847170 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be01336a0 con 0x7f4be01338c0 2023-09-30T22:32:34.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.265+0000 7f4be5dcf700 1 --2- 172.21.15.16:0/2030847170 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4be01338c0 0x7f4be0133c90 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc009840 tx=0x7f4bdc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32b3836904236ea8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:34.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.265+0000 7f4be4dcd700 1 -- 172.21.15.16:0/2030847170 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc011830 con 0x7f4be01338c0 2023-09-30T22:32:34.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.265+0000 7f4be4dcd700 1 -- 172.21.15.16:0/2030847170 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bdc011990 con 0x7f4be01338c0 2023-09-30T22:32:34.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.265+0000 7f4be4dcd700 1 -- 172.21.15.16:0/2030847170 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc011d20 con 0x7f4be01338c0 2023-09-30T22:32:34.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.266+0000 7f4be8033700 1 -- 172.21.15.16:0/2030847170 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4be01338c0 msgr2=0x7f4be0133c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:34.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.266+0000 7f4be8033700 1 --2- 172.21.15.16:0/2030847170 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4be01338c0 0x7f4be0133c90 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc009840 tx=0x7f4bdc009b50 comp rx=0 tx=0).stop 2023-09-30T22:32:34.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.267+0000 7f4be8033700 1 -- 172.21.15.16:0/2030847170 shutdown_connections 2023-09-30T22:32:34.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.267+0000 7f4be8033700 1 --2- 172.21.15.16:0/2030847170 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4be01338c0 0x7f4be0133c90 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:34.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.267+0000 7f4be8033700 1 --2- 172.21.15.16:0/2030847170 >> v2:172.21.15.16:3300/0 conn(0x7f4be01341d0 0x7f4be009de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:34.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.267+0000 7f4be8033700 1 -- 172.21.15.16:0/2030847170 >> 172.21.15.16:0/2030847170 conn(0x7f4be00a2aa0 msgr2=0x7f4be00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:34.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.267+0000 7f4be8033700 1 -- 172.21.15.16:0/2030847170 shutdown_connections 2023-09-30T22:32:34.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.267+0000 7f4be8033700 1 -- 172.21.15.16:0/2030847170 wait complete. 2023-09-30T22:32:34.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.268+0000 7f4be8033700 1 Processor -- start 2023-09-30T22:32:34.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.268+0000 7f4be8033700 1 -- start start 2023-09-30T22:32:34.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.269+0000 7f4be8033700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4be009e390 0x7f4be011f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:34.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.269+0000 7f4be8033700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4be01338c0 0x7f4be011fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:34.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.269+0000 7f4be8033700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4be01341d0 0x7f4be01202f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:34.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.269+0000 7f4be8033700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be01230b0 con 0x7f4be01341d0 2023-09-30T22:32:34.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.269+0000 7f4be8033700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4be0122390 con 0x7f4be009e390 2023-09-30T22:32:34.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.269+0000 7f4be8033700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4be00a1260 con 0x7f4be01338c0 2023-09-30T22:32:34.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.269+0000 7f4be5dcf700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4be009e390 0x7f4be011f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:34.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.269+0000 7f4be5dcf700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4be009e390 0x7f4be011f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:45032/0 (socket says 172.21.15.16:45032) 2023-09-30T22:32:34.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.269+0000 7f4be5dcf700 1 -- 172.21.15.16:0/4139240721 learned_addr learned my addr 172.21.15.16:0/4139240721 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:34.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.269+0000 7f4be65d0700 1 --2- 172.21.15.16:0/4139240721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4be01341d0 0x7f4be01202f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:34.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.269+0000 7f4be5dcf700 1 -- 172.21.15.16:0/4139240721 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4be01338c0 msgr2=0x7f4be011fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:34.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.269+0000 7f4be5dcf700 1 --2- 172.21.15.16:0/4139240721 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4be01338c0 0x7f4be011fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:34.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.269+0000 7f4be5dcf700 1 -- 172.21.15.16:0/4139240721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4be01341d0 msgr2=0x7f4be01202f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:34.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.269+0000 7f4be5dcf700 1 --2- 172.21.15.16:0/4139240721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4be01341d0 0x7f4be01202f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:34.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.269+0000 7f4be5dcf700 1 -- 172.21.15.16:0/4139240721 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bdc0094a0 con 0x7f4be009e390 2023-09-30T22:32:34.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.270+0000 7f4be5dcf700 1 --2- 172.21.15.16:0/4139240721 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4be009e390 0x7f4be011f870 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc009fd0 tx=0x7f4bdc004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:34.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.270+0000 7f4bd67fc700 1 -- 172.21.15.16:0/4139240721 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc0036a0 con 0x7f4be009e390 2023-09-30T22:32:34.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.270+0000 7f4bd67fc700 1 -- 172.21.15.16:0/4139240721 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bdc01f070 con 0x7f4be009e390 2023-09-30T22:32:34.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.270+0000 7f4be8033700 1 -- 172.21.15.16:0/4139240721 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4be006f960 con 0x7f4be009e390 2023-09-30T22:32:34.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.270+0000 7f4bd67fc700 1 -- 172.21.15.16:0/4139240721 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bdc003a60 con 0x7f4be009e390 2023-09-30T22:32:34.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.270+0000 7f4be8033700 1 -- 172.21.15.16:0/4139240721 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4be0123390 con 0x7f4be009e390 2023-09-30T22:32:34.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.271+0000 7f4bd67fc700 1 -- 172.21.15.16:0/4139240721 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4bdc003bc0 con 0x7f4be009e390 2023-09-30T22:32:34.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.271+0000 7f4bd67fc700 1 -- 172.21.15.16:0/4139240721 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f4bdc021070 con 0x7f4be009e390 2023-09-30T22:32:34.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.271+0000 7f4be8033700 1 -- 172.21.15.16:0/4139240721 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4be011c480 con 0x7f4be009e390 2023-09-30T22:32:34.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.281+0000 7f4bd67fc700 1 -- 172.21.15.16:0/4139240721 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4bdc02b020 con 0x7f4be009e390 2023-09-30T22:32:34.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.308+0000 7f5b24332700 1 Processor -- start 2023-09-30T22:32:34.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.308+0000 7f5b24332700 1 -- start start 2023-09-30T22:32:34.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.308+0000 7f5b24332700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5b1c136710 0x7f5b1c138af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:34.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.308+0000 7f5b24332700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5b1c139030 0x7f5b1c13b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:34.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.308+0000 7f5b24332700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b1c06ef30 con 0x7f5b1c1338a0 2023-09-30T22:32:34.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.308+0000 7f5b24332700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b1c06fef0 con 0x7f5b1c136710 2023-09-30T22:32:34.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.308+0000 7f5b24332700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1c1334b0 con 0x7f5b1c139030 2023-09-30T22:32:34.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.309+0000 7f5b228cf700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5b1c139030 0x7f5b1c13b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:34.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.309+0000 7f5b228cf700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5b1c139030 0x7f5b1c13b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:46486/0 (socket says 172.21.15.16:46486) 2023-09-30T22:32:34.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.309+0000 7f5b228cf700 1 -- 172.21.15.16:0/163014051 learned_addr learned my addr 172.21.15.16:0/163014051 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:34.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.309+0000 7f5b218cd700 1 --2- 172.21.15.16:0/163014051 >> v2:172.21.15.16:3300/0 conn(0x7f5b1c136710 0x7f5b1c138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:34.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.309+0000 7f5b228cf700 1 -- 172.21.15.16:0/163014051 >> v2:172.21.15.16:3300/0 conn(0x7f5b1c136710 msgr2=0x7f5b1c138af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:34.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.309+0000 7f5b228cf700 1 --2- 172.21.15.16:0/163014051 >> v2:172.21.15.16:3300/0 conn(0x7f5b1c136710 0x7f5b1c138af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:34.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.309+0000 7f5b228cf700 1 -- 172.21.15.16:0/163014051 >> v1:172.21.15.16:6789/0 conn(0x7f5b1c1338a0 legacy=0x7f5b1c133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:34.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.309+0000 7f5b228cf700 1 -- 172.21.15.16:0/163014051 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b1c06fc30 con 0x7f5b1c139030 2023-09-30T22:32:34.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.309+0000 7f5b228cf700 1 --2- 172.21.15.16:0/163014051 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5b1c139030 0x7f5b1c13b410 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5b14009e50 tx=0x7f5b1400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e98ba029f5558bd6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:34.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.310+0000 7f5b210cc700 1 -- 172.21.15.16:0/163014051 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1400e070 con 0x7f5b1c139030 2023-09-30T22:32:34.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.310+0000 7f5b210cc700 1 -- 172.21.15.16:0/163014051 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b14003a20 con 0x7f5b1c139030 2023-09-30T22:32:34.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.310+0000 7f5b210cc700 1 -- 172.21.15.16:0/163014051 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b14003dd0 con 0x7f5b1c139030 2023-09-30T22:32:34.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.311+0000 7f5b24332700 1 -- 172.21.15.16:0/163014051 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5b1c139030 msgr2=0x7f5b1c13b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:34.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.311+0000 7f5b24332700 1 --2- 172.21.15.16:0/163014051 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5b1c139030 0x7f5b1c13b410 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5b14009e50 tx=0x7f5b1400b330 comp rx=0 tx=0).stop 2023-09-30T22:32:34.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.311+0000 7f5b24332700 1 -- 172.21.15.16:0/163014051 shutdown_connections 2023-09-30T22:32:34.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.311+0000 7f5b24332700 1 --2- 172.21.15.16:0/163014051 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5b1c139030 0x7f5b1c13b410 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:34.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.311+0000 7f5b24332700 1 --2- 172.21.15.16:0/163014051 >> v2:172.21.15.16:3300/0 conn(0x7f5b1c136710 0x7f5b1c138af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:34.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.311+0000 7f5b24332700 1 -- 172.21.15.16:0/163014051 >> 172.21.15.16:0/163014051 conn(0x7f5b1c0a2aa0 msgr2=0x7f5b1c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:34.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.311+0000 7f5b24332700 1 -- 172.21.15.16:0/163014051 shutdown_connections 2023-09-30T22:32:34.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.311+0000 7f5b24332700 1 -- 172.21.15.16:0/163014051 wait complete. 2023-09-30T22:32:34.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.312+0000 7f5b24332700 1 Processor -- start 2023-09-30T22:32:34.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.312+0000 7f5b24332700 1 -- start start 2023-09-30T22:32:34.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.312+0000 7f5b24332700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5b1c1338a0 0x7f5b1c118cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:34.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.312+0000 7f5b24332700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5b1c136710 0x7f5b1c119210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:34.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.312+0000 7f5b24332700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b1c139030 0x7f5b1c119750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:34.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.312+0000 7f5b24332700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1c11c6f0 con 0x7f5b1c139030 2023-09-30T22:32:34.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.312+0000 7f5b24332700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1c11c140 con 0x7f5b1c136710 2023-09-30T22:32:34.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.312+0000 7f5b24332700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1c0a2410 con 0x7f5b1c1338a0 2023-09-30T22:32:34.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.313+0000 7f5b218cd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5b1c136710 0x7f5b1c119210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:34.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.313+0000 7f5b218cd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5b1c136710 0x7f5b1c119210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:45044/0 (socket says 172.21.15.16:45044) 2023-09-30T22:32:34.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.313+0000 7f5b218cd700 1 -- 172.21.15.16:0/1659075555 learned_addr learned my addr 172.21.15.16:0/1659075555 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:34.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.313+0000 7f5b228cf700 1 --2- 172.21.15.16:0/1659075555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b1c139030 0x7f5b1c119750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:34.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.313+0000 7f5b220ce700 1 --2- 172.21.15.16:0/1659075555 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5b1c1338a0 0x7f5b1c118cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:34.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.313+0000 7f5b218cd700 1 -- 172.21.15.16:0/1659075555 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5b1c1338a0 msgr2=0x7f5b1c118cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:34.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.313+0000 7f5b218cd700 1 --2- 172.21.15.16:0/1659075555 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5b1c1338a0 0x7f5b1c118cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:34.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.313+0000 7f5b218cd700 1 -- 172.21.15.16:0/1659075555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b1c139030 msgr2=0x7f5b1c119750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:34.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.313+0000 7f5b218cd700 1 --2- 172.21.15.16:0/1659075555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b1c139030 0x7f5b1c119750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:34.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.313+0000 7f5b218cd700 1 -- 172.21.15.16:0/1659075555 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b14009b30 con 0x7f5b1c136710 2023-09-30T22:32:34.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.313+0000 7f5b218cd700 1 --2- 172.21.15.16:0/1659075555 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5b1c136710 0x7f5b1c119210 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c00e9b0 tx=0x7f5b0c00ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:34.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.314+0000 7f5b12ffd700 1 -- 172.21.15.16:0/1659075555 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c009710 con 0x7f5b1c136710 2023-09-30T22:32:34.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.314+0000 7f5b12ffd700 1 -- 172.21.15.16:0/1659075555 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b0c00ee50 con 0x7f5b1c136710 2023-09-30T22:32:34.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.314+0000 7f5b12ffd700 1 -- 172.21.15.16:0/1659075555 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c00ca50 con 0x7f5b1c136710 2023-09-30T22:32:34.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.314+0000 7f5b24332700 1 -- 172.21.15.16:0/1659075555 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b1c06f640 con 0x7f5b1c136710 2023-09-30T22:32:34.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.314+0000 7f5b24332700 1 -- 172.21.15.16:0/1659075555 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b1c11c370 con 0x7f5b1c136710 2023-09-30T22:32:34.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.315+0000 7f5b12ffd700 1 -- 172.21.15.16:0/1659075555 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5b0c00cbb0 con 0x7f5b1c136710 2023-09-30T22:32:34.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.315+0000 7f5b12ffd700 1 -- 172.21.15.16:0/1659075555 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f5b0c03fa80 con 0x7f5b1c136710 2023-09-30T22:32:34.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.315+0000 7f5b24332700 1 -- 172.21.15.16:0/1659075555 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b1c11bf50 con 0x7f5b1c136710 2023-09-30T22:32:34.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.324+0000 7f5b12ffd700 1 -- 172.21.15.16:0/1659075555 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5b0c021080 con 0x7f5b1c136710 2023-09-30T22:32:34.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.366+0000 7fa35b288700 1 Processor -- start 2023-09-30T22:32:34.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.366+0000 7fa35b288700 1 -- start start 2023-09-30T22:32:34.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.367+0000 7fa35b288700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa35409e1d0 0x7fa354138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:34.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.367+0000 7fa35b288700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa354139170 0x7fa35413b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:34.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.367+0000 7fa35b288700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa35409b120 con 0x7fa35409d6f0 2023-09-30T22:32:34.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.367+0000 7fa35b288700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa35406b460 con 0x7fa354139170 2023-09-30T22:32:34.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.367+0000 7fa35b288700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa35409a7e0 con 0x7fa35409e1d0 2023-09-30T22:32:34.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.367+0000 7fa359024700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa35409d6f0 0x7fa35409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:39076/0 (socket says 172.21.15.16:39076) 2023-09-30T22:32:34.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.367+0000 7fa359024700 1 -- 172.21.15.16:0/3919349027 learned_addr learned my addr 172.21.15.16:0/3919349027 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:34.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.367+0000 7fa358823700 1 --2- 172.21.15.16:0/3919349027 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa35409e1d0 0x7fa354138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:34.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.367+0000 7fa359825700 1 --2- 172.21.15.16:0/3919349027 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa354139170 0x7fa35413b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:34.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.367+0000 7fa34bfff700 1 -- 172.21.15.16:0/3919349027 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2031374963 0 0) 0x7fa35409b120 con 0x7fa35409d6f0 2023-09-30T22:32:34.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.367+0000 7fa34bfff700 1 -- 172.21.15.16:0/3919349027 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3400035b0 con 0x7fa35409d6f0 2023-09-30T22:32:34.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.367+0000 7fa358823700 1 -- 172.21.15.16:0/3919349027 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa354139170 msgr2=0x7fa35413b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:34.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.367+0000 7fa358823700 1 --2- 172.21.15.16:0/3919349027 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa354139170 0x7fa35413b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:34.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.367+0000 7fa358823700 1 -- 172.21.15.16:0/3919349027 >> v1:172.21.15.16:6789/0 conn(0x7fa35409d6f0 legacy=0x7fa35409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:34.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.367+0000 7fa358823700 1 -- 172.21.15.16:0/3919349027 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa35409a500 con 0x7fa35409e1d0 2023-09-30T22:32:34.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.368+0000 7fa358823700 1 --2- 172.21.15.16:0/3919349027 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa35409e1d0 0x7fa354138c30 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa3440098a0 tx=0x7fa344009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a19f0ba51b16f06 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:34.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.368+0000 7fa34bfff700 1 -- 172.21.15.16:0/3919349027 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa344011830 con 0x7fa35409e1d0 2023-09-30T22:32:34.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.368+0000 7fa34bfff700 1 -- 172.21.15.16:0/3919349027 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa344011990 con 0x7fa35409e1d0 2023-09-30T22:32:34.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.368+0000 7fa34bfff700 1 -- 172.21.15.16:0/3919349027 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa344011d20 con 0x7fa35409e1d0 2023-09-30T22:32:34.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.368+0000 7fa35b288700 1 -- 172.21.15.16:0/3919349027 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa35409e1d0 msgr2=0x7fa354138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:34.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.368+0000 7fa35b288700 1 --2- 172.21.15.16:0/3919349027 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa35409e1d0 0x7fa354138c30 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa3440098a0 tx=0x7fa344009bb0 comp rx=0 tx=0).stop 2023-09-30T22:32:34.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.369+0000 7fa35b288700 1 -- 172.21.15.16:0/3919349027 shutdown_connections 2023-09-30T22:32:34.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.369+0000 7fa35b288700 1 --2- 172.21.15.16:0/3919349027 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa354139170 0x7fa35413b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:34.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.369+0000 7fa35b288700 1 --2- 172.21.15.16:0/3919349027 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa35409e1d0 0x7fa354138c30 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:34.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.369+0000 7fa35b288700 1 -- 172.21.15.16:0/3919349027 >> 172.21.15.16:0/3919349027 conn(0x7fa354066f50 msgr2=0x7fa35409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:34.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.369+0000 7fa35b288700 1 -- 172.21.15.16:0/3919349027 shutdown_connections 2023-09-30T22:32:34.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.369+0000 7fa35b288700 1 -- 172.21.15.16:0/3919349027 wait complete. 2023-09-30T22:32:34.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.370+0000 7fa35b288700 1 Processor -- start 2023-09-30T22:32:34.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.370+0000 7fa35b288700 1 -- start start 2023-09-30T22:32:34.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.370+0000 7fa35b288700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa35409d6f0 0x7fa3541c84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:34.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.370+0000 7fa35b288700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa35409e1d0 0x7fa3541c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:34.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.370+0000 7fa35b288700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa354139170 0x7fa3541218b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:34.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.370+0000 7fa35b288700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa354124bf0 con 0x7fa35409d6f0 2023-09-30T22:32:34.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.370+0000 7fa35b288700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa354137730 con 0x7fa354139170 2023-09-30T22:32:34.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.370+0000 7fa35b288700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa354123470 con 0x7fa35409e1d0 2023-09-30T22:32:34.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.371+0000 7fa358823700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa35409e1d0 0x7fa3541c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:34.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.371+0000 7fa358823700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa35409e1d0 0x7fa3541c8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:46512/0 (socket says 172.21.15.16:46512) 2023-09-30T22:32:34.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.371+0000 7fa358823700 1 -- 172.21.15.16:0/2831894634 learned_addr learned my addr 172.21.15.16:0/2831894634 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:34.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.371+0000 7fa359825700 1 --2- 172.21.15.16:0/2831894634 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa354139170 0x7fa3541218b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:34.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.371+0000 7fa359024700 1 --2- 172.21.15.16:0/2831894634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa35409d6f0 0x7fa3541c84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:34.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.371+0000 7fa359825700 1 -- 172.21.15.16:0/2831894634 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa35409e1d0 msgr2=0x7fa3541c8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:34.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.371+0000 7fa359825700 1 --2- 172.21.15.16:0/2831894634 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa35409e1d0 0x7fa3541c8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:34.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.371+0000 7fa359825700 1 -- 172.21.15.16:0/2831894634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa35409d6f0 msgr2=0x7fa3541c84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:34.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.371+0000 7fa359825700 1 --2- 172.21.15.16:0/2831894634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa35409d6f0 0x7fa3541c84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:34.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.371+0000 7fa359825700 1 -- 172.21.15.16:0/2831894634 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3440094a0 con 0x7fa354139170 2023-09-30T22:32:34.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.371+0000 7fa359024700 1 --2- 172.21.15.16:0/2831894634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa35409d6f0 0x7fa3541c84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:32:34.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.371+0000 7fa359825700 1 --2- 172.21.15.16:0/2831894634 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa354139170 0x7fa3541218b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa34c007d90 tx=0x7fa34c00e3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:34.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.371+0000 7fa349ffb700 1 -- 172.21.15.16:0/2831894634 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34c018070 con 0x7fa354139170 2023-09-30T22:32:34.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.372+0000 7fa349ffb700 1 -- 172.21.15.16:0/2831894634 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa34c00ebe0 con 0x7fa354139170 2023-09-30T22:32:34.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.372+0000 7fa35b288700 1 -- 172.21.15.16:0/2831894634 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3540a3a90 con 0x7fa354139170 2023-09-30T22:32:34.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.372+0000 7fa349ffb700 1 -- 172.21.15.16:0/2831894634 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34c003a20 con 0x7fa354139170 2023-09-30T22:32:34.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.372+0000 7fa35b288700 1 -- 172.21.15.16:0/2831894634 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3541250a0 con 0x7fa354139170 2023-09-30T22:32:34.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.373+0000 7fa349ffb700 1 -- 172.21.15.16:0/2831894634 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa34c014420 con 0x7fa354139170 2023-09-30T22:32:34.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.373+0000 7fa349ffb700 1 -- 172.21.15.16:0/2831894634 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fa34c03f6d0 con 0x7fa354139170 2023-09-30T22:32:34.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.373+0000 7fa35b288700 1 -- 172.21.15.16:0/2831894634 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa354074ea0 con 0x7fa354139170 2023-09-30T22:32:34.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.382+0000 7fa349ffb700 1 -- 172.21.15.16:0/2831894634 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa34c0128d0 con 0x7fa354139170 2023-09-30T22:32:34.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.436+0000 7ffb9de45700 1 -- 172.21.15.16:0/1442138544 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ffb9804ea50 con 0x7ffb981338a0 2023-09-30T22:32:34.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.436+0000 7ffb7ffff700 1 -- 172.21.15.16:0/1442138544 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ffb88011360 con 0x7ffb981338a0 2023-09-30T22:32:34.437 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:32:34.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.439+0000 7ffb9de45700 1 -- 172.21.15.16:0/1442138544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb981338a0 msgr2=0x7ffb9811adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:34.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.439+0000 7ffb9de45700 1 --2- 172.21.15.16:0/1442138544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb981338a0 0x7ffb9811adc0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7ffb8800c750 tx=0x7ffb8800ca60 comp rx=0 tx=0).stop 2023-09-30T22:32:34.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.439+0000 7ffb9de45700 1 -- 172.21.15.16:0/1442138544 shutdown_connections 2023-09-30T22:32:34.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.439+0000 7ffb9de45700 1 --2- 172.21.15.16:0/1442138544 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffb98139030 0x7ffb9811b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:34.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.439+0000 7ffb9de45700 1 --2- 172.21.15.16:0/1442138544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb981338a0 0x7ffb9811adc0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:34.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.439+0000 7ffb9de45700 1 --2- 172.21.15.16:0/1442138544 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffb98136710 0x7ffb9811b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:34.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.439+0000 7ffb9de45700 1 -- 172.21.15.16:0/1442138544 >> 172.21.15.16:0/1442138544 conn(0x7ffb980a2aa0 msgr2=0x7ffb98137170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:34.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.439+0000 7ffb9de45700 1 -- 172.21.15.16:0/1442138544 shutdown_connections 2023-09-30T22:32:34.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.439+0000 7ffb9de45700 1 -- 172.21.15.16:0/1442138544 wait complete. 2023-09-30T22:32:34.453 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:32:34.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.534+0000 7f4be8033700 1 -- 172.21.15.16:0/4139240721 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f4be0074ea0 con 0x7f4be009e390 2023-09-30T22:32:34.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.535+0000 7f4bd67fc700 1 -- 172.21.15.16:0/4139240721 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4bdc034070 con 0x7f4be009e390 2023-09-30T22:32:34.535 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:32:34.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.537+0000 7f4be8033700 1 -- 172.21.15.16:0/4139240721 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4be009e390 msgr2=0x7f4be011f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:34.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.537+0000 7f4be8033700 1 --2- 172.21.15.16:0/4139240721 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4be009e390 0x7f4be011f870 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc009fd0 tx=0x7f4bdc004750 comp rx=0 tx=0).stop 2023-09-30T22:32:34.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.537+0000 7f4be8033700 1 -- 172.21.15.16:0/4139240721 shutdown_connections 2023-09-30T22:32:34.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.537+0000 7f4be8033700 1 --2- 172.21.15.16:0/4139240721 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4be009e390 0x7f4be011f870 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:34.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.537+0000 7f4be8033700 1 --2- 172.21.15.16:0/4139240721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4be01341d0 0x7f4be01202f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:34.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.537+0000 7f4be8033700 1 --2- 172.21.15.16:0/4139240721 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4be01338c0 0x7f4be011fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:34.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.537+0000 7f4be8033700 1 -- 172.21.15.16:0/4139240721 >> 172.21.15.16:0/4139240721 conn(0x7f4be00a2aa0 msgr2=0x7f4be0131ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:34.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.537+0000 7f4be8033700 1 -- 172.21.15.16:0/4139240721 shutdown_connections 2023-09-30T22:32:34.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.537+0000 7f4be8033700 1 -- 172.21.15.16:0/4139240721 wait complete. 2023-09-30T22:32:34.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.549+0000 7f5b24332700 1 -- 172.21.15.16:0/1659075555 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5b1c06ea40 con 0x7f5b1c136710 2023-09-30T22:32:34.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.549+0000 7f5b12ffd700 1 -- 172.21.15.16:0/1659075555 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5b0c017020 con 0x7f5b1c136710 2023-09-30T22:32:34.550 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:32:34.552 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:32:34.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.551+0000 7f5b24332700 1 -- 172.21.15.16:0/1659075555 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5b1c136710 msgr2=0x7f5b1c119210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:34.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.551+0000 7f5b24332700 1 --2- 172.21.15.16:0/1659075555 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5b1c136710 0x7f5b1c119210 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c00e9b0 tx=0x7f5b0c00ecc0 comp rx=0 tx=0).stop 2023-09-30T22:32:34.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.552+0000 7f5b24332700 1 -- 172.21.15.16:0/1659075555 shutdown_connections 2023-09-30T22:32:34.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.552+0000 7f5b24332700 1 --2- 172.21.15.16:0/1659075555 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b1c139030 0x7f5b1c119750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:34.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.552+0000 7f5b24332700 1 --2- 172.21.15.16:0/1659075555 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5b1c1338a0 0x7f5b1c118cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:34.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.552+0000 7f5b24332700 1 --2- 172.21.15.16:0/1659075555 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5b1c136710 0x7f5b1c119210 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:34.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.552+0000 7f5b24332700 1 -- 172.21.15.16:0/1659075555 >> 172.21.15.16:0/1659075555 conn(0x7f5b1c0a2aa0 msgr2=0x7f5b1c0a0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:34.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.552+0000 7f5b24332700 1 -- 172.21.15.16:0/1659075555 shutdown_connections 2023-09-30T22:32:34.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.552+0000 7f5b24332700 1 -- 172.21.15.16:0/1659075555 wait complete. 2023-09-30T22:32:34.565 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:32:34.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.635+0000 7fa35b288700 1 -- 172.21.15.16:0/2831894634 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa354071c90 con 0x7fa354139170 2023-09-30T22:32:34.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.635+0000 7fa349ffb700 1 -- 172.21.15.16:0/2831894634 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa34c0122e0 con 0x7fa354139170 2023-09-30T22:32:34.636 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:32:34.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.637+0000 7fa35b288700 1 -- 172.21.15.16:0/2831894634 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa354139170 msgr2=0x7fa3541218b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:34.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.637+0000 7fa35b288700 1 --2- 172.21.15.16:0/2831894634 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa354139170 0x7fa3541218b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa34c007d90 tx=0x7fa34c00e3c0 comp rx=0 tx=0).stop 2023-09-30T22:32:34.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.637+0000 7fa35b288700 1 -- 172.21.15.16:0/2831894634 shutdown_connections 2023-09-30T22:32:34.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.637+0000 7fa35b288700 1 --2- 172.21.15.16:0/2831894634 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa354139170 0x7fa3541218b0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:34.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.637+0000 7fa35b288700 1 --2- 172.21.15.16:0/2831894634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa35409d6f0 0x7fa3541c84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:34.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.637+0000 7fa35b288700 1 --2- 172.21.15.16:0/2831894634 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa35409e1d0 0x7fa3541c8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:34.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.637+0000 7fa35b288700 1 -- 172.21.15.16:0/2831894634 >> 172.21.15.16:0/2831894634 conn(0x7fa354066f50 msgr2=0x7fa354070c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:34.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.637+0000 7fa35b288700 1 -- 172.21.15.16:0/2831894634 shutdown_connections 2023-09-30T22:32:34.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:34.637+0000 7fa35b288700 1 -- 172.21.15.16:0/2831894634 wait complete. 2023-09-30T22:32:34.651 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:32:35.453 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:32:35.554 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:32:35.567 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:32:35.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.601+0000 7f1158746700 1 Processor -- start 2023-09-30T22:32:35.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.601+0000 7f1158746700 1 -- start start 2023-09-30T22:32:35.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.602+0000 7f1158746700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f115009e1d0 0x7f1150138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:35.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.602+0000 7f1158746700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1150139170 0x7f115013b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:35.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.602+0000 7f1158746700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f115009b120 con 0x7f115009d6f0 2023-09-30T22:32:35.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.602+0000 7f1158746700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f115006b460 con 0x7f1150139170 2023-09-30T22:32:35.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.602+0000 7f1158746700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f115009a7e0 con 0x7f115009e1d0 2023-09-30T22:32:35.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.602+0000 7f1155ce1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f115009e1d0 0x7f1150138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:35.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.602+0000 7f1156ce3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1150139170 0x7f115013b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:35.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.602+0000 7f11564e2700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f115009d6f0 0x7f115009dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:39092/0 (socket says 172.21.15.16:39092) 2023-09-30T22:32:35.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.602+0000 7f11564e2700 1 -- 172.21.15.16:0/3463363781 learned_addr learned my addr 172.21.15.16:0/3463363781 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:35.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.602+0000 7f1155ce1700 1 -- 172.21.15.16:0/3463363781 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1150139170 msgr2=0x7f115013b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:35.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.602+0000 7f1155ce1700 1 --2- 172.21.15.16:0/3463363781 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1150139170 0x7f115013b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:35.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.602+0000 7f1155ce1700 1 -- 172.21.15.16:0/3463363781 >> v1:172.21.15.16:6789/0 conn(0x7f115009d6f0 legacy=0x7f115009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:35.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.602+0000 7f1155ce1700 1 -- 172.21.15.16:0/3463363781 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f115009a500 con 0x7f115009e1d0 2023-09-30T22:32:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.602+0000 7f11554e0700 1 -- 172.21.15.16:0/3463363781 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1331531840 0 0) 0x7f115009b120 con 0x7f115009d6f0 2023-09-30T22:32:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.603+0000 7f1155ce1700 1 --2- 172.21.15.16:0/3463363781 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f115009e1d0 0x7f1150138c30 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f11400098a0 tx=0x7f1140009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adc054aca4ef7ec7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.603+0000 7f11554e0700 1 -- 172.21.15.16:0/3463363781 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1140011830 con 0x7f115009e1d0 2023-09-30T22:32:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.603+0000 7f11554e0700 1 -- 172.21.15.16:0/3463363781 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1140011990 con 0x7f115009e1d0 2023-09-30T22:32:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.603+0000 7f11554e0700 1 -- 172.21.15.16:0/3463363781 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1140011d20 con 0x7f115009e1d0 2023-09-30T22:32:35.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.603+0000 7f1158746700 1 -- 172.21.15.16:0/3463363781 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f115009e1d0 msgr2=0x7f1150138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:35.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.603+0000 7f1158746700 1 --2- 172.21.15.16:0/3463363781 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f115009e1d0 0x7f1150138c30 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f11400098a0 tx=0x7f1140009bb0 comp rx=0 tx=0).stop 2023-09-30T22:32:35.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.604+0000 7f1158746700 1 -- 172.21.15.16:0/3463363781 shutdown_connections 2023-09-30T22:32:35.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.604+0000 7f1158746700 1 --2- 172.21.15.16:0/3463363781 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1150139170 0x7f115013b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:35.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.604+0000 7f1158746700 1 --2- 172.21.15.16:0/3463363781 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f115009e1d0 0x7f1150138c30 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:35.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.604+0000 7f1158746700 1 -- 172.21.15.16:0/3463363781 >> 172.21.15.16:0/3463363781 conn(0x7f1150066f50 msgr2=0x7f115009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:35.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.604+0000 7f1158746700 1 -- 172.21.15.16:0/3463363781 shutdown_connections 2023-09-30T22:32:35.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.604+0000 7f1158746700 1 -- 172.21.15.16:0/3463363781 wait complete. 2023-09-30T22:32:35.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.605+0000 7f1158746700 1 Processor -- start 2023-09-30T22:32:35.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.605+0000 7f1158746700 1 -- start start 2023-09-30T22:32:35.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.605+0000 7f1158746700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f115009d6f0 0x7f11501217f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:35.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.605+0000 7f1158746700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f115009e1d0 0x7f1150121d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:35.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.605+0000 7f1158746700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1150139170 0x7f1150122270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:35.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.606+0000 7f1158746700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1150124ac0 con 0x7f115009e1d0 2023-09-30T22:32:35.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.606+0000 7f1158746700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1150124fe0 con 0x7f115009d6f0 2023-09-30T22:32:35.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.606+0000 7f1158746700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f11500a3a90 con 0x7f1150139170 2023-09-30T22:32:35.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.606+0000 7f11564e2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f115009d6f0 0x7f11501217f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:35.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.606+0000 7f1155ce1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f115009e1d0 0x7f1150121d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:35.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.606+0000 7f11564e2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f115009d6f0 0x7f11501217f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:45080/0 (socket says 172.21.15.16:45080) 2023-09-30T22:32:35.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.606+0000 7f1155ce1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f115009e1d0 0x7f1150121d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56410/0 (socket says 172.21.15.16:56410) 2023-09-30T22:32:35.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.606+0000 7f11564e2700 1 -- 172.21.15.16:0/3288717332 learned_addr learned my addr 172.21.15.16:0/3288717332 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:35.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.606+0000 7f1156ce3700 1 --2- 172.21.15.16:0/3288717332 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1150139170 0x7f1150122270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:35.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.606+0000 7f11564e2700 1 -- 172.21.15.16:0/3288717332 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1150139170 msgr2=0x7f1150122270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:35.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.606+0000 7f11564e2700 1 --2- 172.21.15.16:0/3288717332 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1150139170 0x7f1150122270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:35.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.606+0000 7f11564e2700 1 -- 172.21.15.16:0/3288717332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f115009e1d0 msgr2=0x7f1150121d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:35.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.606+0000 7f11564e2700 1 --2- 172.21.15.16:0/3288717332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f115009e1d0 0x7f1150121d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:35.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.606+0000 7f11564e2700 1 -- 172.21.15.16:0/3288717332 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11400094a0 con 0x7f115009d6f0 2023-09-30T22:32:35.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.606+0000 7f1155ce1700 1 --2- 172.21.15.16:0/3288717332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f115009e1d0 0x7f1150121d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:35.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.606+0000 7f11564e2700 1 --2- 172.21.15.16:0/3288717332 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f115009d6f0 0x7f11501217f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f114c00c520 tx=0x7f114c00c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:35.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.606+0000 7f1146ffd700 1 -- 172.21.15.16:0/3288717332 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f114c004c80 con 0x7f115009d6f0 2023-09-30T22:32:35.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.607+0000 7f1146ffd700 1 -- 172.21.15.16:0/3288717332 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f114c004e00 con 0x7f115009d6f0 2023-09-30T22:32:35.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.607+0000 7f1146ffd700 1 -- 172.21.15.16:0/3288717332 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f114c007310 con 0x7f115009d6f0 2023-09-30T22:32:35.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.607+0000 7f1158746700 1 -- 172.21.15.16:0/3288717332 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f115009c710 con 0x7f115009d6f0 2023-09-30T22:32:35.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.607+0000 7f1158746700 1 -- 172.21.15.16:0/3288717332 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1150125a50 con 0x7f115009d6f0 2023-09-30T22:32:35.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.608+0000 7f1146ffd700 1 -- 172.21.15.16:0/3288717332 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f114c007470 con 0x7f115009d6f0 2023-09-30T22:32:35.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.608+0000 7f1146ffd700 1 -- 172.21.15.16:0/3288717332 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f114c03f430 con 0x7f115009d6f0 2023-09-30T22:32:35.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.608+0000 7f1158746700 1 -- 172.21.15.16:0/3288717332 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1150074ea0 con 0x7f115009d6f0 2023-09-30T22:32:35.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.613+0000 7f1146ffd700 1 -- 172.21.15.16:0/3288717332 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f114c0123b0 con 0x7f115009d6f0 2023-09-30T22:32:35.652 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:32:35.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.715+0000 7fa5c1833700 1 Processor -- start 2023-09-30T22:32:35.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.715+0000 7fa5c1833700 1 -- start start 2023-09-30T22:32:35.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.716+0000 7fa5c1833700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa5bc06e570 0x7fa5bc06e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:35.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.716+0000 7fa5c1833700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa5bc09eca0 0x7fa5bc06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:35.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.716+0000 7fa5c1833700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5bc070ee0 con 0x7fa5bc134610 2023-09-30T22:32:35.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.716+0000 7fa5c1833700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5bc071480 con 0x7fa5bc09eca0 2023-09-30T22:32:35.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.716+0000 7fa5c1833700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa5bc06f3c0 con 0x7fa5bc06e570 2023-09-30T22:32:35.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.716+0000 7fa5bb7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa5bc134610 0x7fa5bc134ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:39096/0 (socket says 172.21.15.16:39096) 2023-09-30T22:32:35.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.716+0000 7fa5bb7fe700 1 -- 172.21.15.16:0/3605982930 learned_addr learned my addr 172.21.15.16:0/3605982930 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:35.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.716+0000 7fa5baffd700 1 --2- 172.21.15.16:0/3605982930 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa5bc06e570 0x7fa5bc06e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:35.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.716+0000 7fa5ba7fc700 1 --2- 172.21.15.16:0/3605982930 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa5bc09eca0 0x7fa5bc06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:35.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.716+0000 7fa5b9ffb700 1 -- 172.21.15.16:0/3605982930 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3581241441 0 0) 0x7fa5bc070ee0 con 0x7fa5bc134610 2023-09-30T22:32:35.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.717+0000 7fa5b9ffb700 1 -- 172.21.15.16:0/3605982930 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5a40035b0 con 0x7fa5bc134610 2023-09-30T22:32:35.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.717+0000 7fa5baffd700 1 -- 172.21.15.16:0/3605982930 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa5bc09eca0 msgr2=0x7fa5bc06ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:35.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.717+0000 7fa5baffd700 1 --2- 172.21.15.16:0/3605982930 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa5bc09eca0 0x7fa5bc06ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:35.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.717+0000 7fa5baffd700 1 -- 172.21.15.16:0/3605982930 >> v1:172.21.15.16:6789/0 conn(0x7fa5bc134610 legacy=0x7fa5bc134ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:35.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.717+0000 7fa5baffd700 1 -- 172.21.15.16:0/3605982930 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5bc071b90 con 0x7fa5bc06e570 2023-09-30T22:32:35.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.717+0000 7fa5baffd700 1 --2- 172.21.15.16:0/3605982930 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa5bc06e570 0x7fa5bc06e940 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac009870 tx=0x7fa5ac009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2712590baf17744 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:35.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.717+0000 7fa5b9ffb700 1 -- 172.21.15.16:0/3605982930 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac011830 con 0x7fa5bc06e570 2023-09-30T22:32:35.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.717+0000 7fa5b9ffb700 1 -- 172.21.15.16:0/3605982930 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5ac011990 con 0x7fa5bc06e570 2023-09-30T22:32:35.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.717+0000 7fa5b9ffb700 1 -- 172.21.15.16:0/3605982930 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac011d20 con 0x7fa5bc06e570 2023-09-30T22:32:35.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.718+0000 7fa5c1833700 1 -- 172.21.15.16:0/3605982930 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa5bc06e570 msgr2=0x7fa5bc06e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:35.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.718+0000 7fa5c1833700 1 --2- 172.21.15.16:0/3605982930 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa5bc06e570 0x7fa5bc06e940 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac009870 tx=0x7fa5ac009b80 comp rx=0 tx=0).stop 2023-09-30T22:32:35.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.718+0000 7fa5c1833700 1 -- 172.21.15.16:0/3605982930 shutdown_connections 2023-09-30T22:32:35.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.718+0000 7fa5c1833700 1 --2- 172.21.15.16:0/3605982930 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa5bc06e570 0x7fa5bc06e940 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:35.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.718+0000 7fa5c1833700 1 --2- 172.21.15.16:0/3605982930 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa5bc09eca0 0x7fa5bc06ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:35.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.718+0000 7fa5c1833700 1 -- 172.21.15.16:0/3605982930 >> 172.21.15.16:0/3605982930 conn(0x7fa5bc0a2aa0 msgr2=0x7fa5bc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:35.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.718+0000 7fa5c1833700 1 -- 172.21.15.16:0/3605982930 shutdown_connections 2023-09-30T22:32:35.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.718+0000 7fa5c1833700 1 -- 172.21.15.16:0/3605982930 wait complete. 2023-09-30T22:32:35.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.720+0000 7fa5c1833700 1 Processor -- start 2023-09-30T22:32:35.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.720+0000 7fa5c1833700 1 -- start start 2023-09-30T22:32:35.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.720+0000 7fa5c1833700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa5bc06e570 0x7fa5bc1214e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:35.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.720+0000 7fa5c1833700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5bc134610 0x7fa5bc121a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:35.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.720+0000 7fa5c1833700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa5bc09e480 0x7fa5bc121f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:35.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.720+0000 7fa5c1833700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5bc124740 con 0x7fa5bc134610 2023-09-30T22:32:35.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.720+0000 7fa5c1833700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5bc122a90 con 0x7fa5bc06e570 2023-09-30T22:32:35.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.720+0000 7fa5c1833700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa5bc123090 con 0x7fa5bc09e480 2023-09-30T22:32:35.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.721+0000 7fa5ba7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5bc134610 0x7fa5bc121a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:35.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.721+0000 7fa5bb7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa5bc09e480 0x7fa5bc121f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:35.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.721+0000 7fa5ba7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5bc134610 0x7fa5bc121a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56420/0 (socket says 172.21.15.16:56420) 2023-09-30T22:32:35.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.721+0000 7fa5bb7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa5bc09e480 0x7fa5bc121f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:46556/0 (socket says 172.21.15.16:46556) 2023-09-30T22:32:35.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.721+0000 7fa5ba7fc700 1 -- 172.21.15.16:0/683673154 learned_addr learned my addr 172.21.15.16:0/683673154 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:35.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.721+0000 7fa5baffd700 1 --2- 172.21.15.16:0/683673154 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa5bc06e570 0x7fa5bc1214e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:35.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.721+0000 7fa5ba7fc700 1 -- 172.21.15.16:0/683673154 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa5bc09e480 msgr2=0x7fa5bc121f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:35.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.721+0000 7fa5ba7fc700 1 --2- 172.21.15.16:0/683673154 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa5bc09e480 0x7fa5bc121f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:35.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.721+0000 7fa5ba7fc700 1 -- 172.21.15.16:0/683673154 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa5bc06e570 msgr2=0x7fa5bc1214e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:35.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.721+0000 7fa5ba7fc700 1 --2- 172.21.15.16:0/683673154 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa5bc06e570 0x7fa5bc1214e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:35.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.721+0000 7fa5ba7fc700 1 -- 172.21.15.16:0/683673154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5ac0094a0 con 0x7fa5bc134610 2023-09-30T22:32:35.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.721+0000 7fa5bb7fe700 1 --2- 172.21.15.16:0/683673154 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa5bc09e480 0x7fa5bc121f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:35.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.721+0000 7fa5baffd700 1 --2- 172.21.15.16:0/683673154 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa5bc06e570 0x7fa5bc1214e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:35.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.721+0000 7fa5ba7fc700 1 --2- 172.21.15.16:0/683673154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5bc134610 0x7fa5bc121a20 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fa5b000e9e0 tx=0x7fa5b000ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:35.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.722+0000 7fa5a3fff700 1 -- 172.21.15.16:0/683673154 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b00096b0 con 0x7fa5bc134610 2023-09-30T22:32:35.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.722+0000 7fa5c1833700 1 -- 172.21.15.16:0/683673154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5bc123cc0 con 0x7fa5bc134610 2023-09-30T22:32:35.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.722+0000 7fa5a3fff700 1 -- 172.21.15.16:0/683673154 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5b000ee80 con 0x7fa5bc134610 2023-09-30T22:32:35.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.722+0000 7fa5a3fff700 1 -- 172.21.15.16:0/683673154 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b000c9f0 con 0x7fa5bc134610 2023-09-30T22:32:35.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.722+0000 7fa5c1833700 1 -- 172.21.15.16:0/683673154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5bc0a2120 con 0x7fa5bc134610 2023-09-30T22:32:35.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.723+0000 7fa5a3fff700 1 -- 172.21.15.16:0/683673154 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa5b000cb50 con 0x7fa5bc134610 2023-09-30T22:32:35.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.724+0000 7fa5a3fff700 1 -- 172.21.15.16:0/683673154 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fa5b003fa90 con 0x7fa5bc134610 2023-09-30T22:32:35.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.724+0000 7fa5c1833700 1 -- 172.21.15.16:0/683673154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5bc09b1e0 con 0x7fa5bc134610 2023-09-30T22:32:35.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.733+0000 7fa5a3fff700 1 -- 172.21.15.16:0/683673154 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa5b0021080 con 0x7fa5bc134610 2023-09-30T22:32:35.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.744+0000 7fe802c17700 1 Processor -- start 2023-09-30T22:32:35.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.744+0000 7fe802c17700 1 -- start start 2023-09-30T22:32:35.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.745+0000 7fe802c17700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe7fc09d6f0 0x7fe7fc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:35.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.745+0000 7fe802c17700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe7fc09e440 0x7fe7fc134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:35.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.745+0000 7fe802c17700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7fc09b120 con 0x7fe7fc09e000 2023-09-30T22:32:35.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.745+0000 7fe802c17700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7fc06b460 con 0x7fe7fc09d6f0 2023-09-30T22:32:35.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.745+0000 7fe802c17700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe7fc09a7e0 con 0x7fe7fc09e440 2023-09-30T22:32:35.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.745+0000 7fe7fbfff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe7fc09e000 0x7fe7fc1339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:39102/0 (socket says 172.21.15.16:39102) 2023-09-30T22:32:35.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.745+0000 7fe7fbfff700 1 -- 172.21.15.16:0/1210392890 learned_addr learned my addr 172.21.15.16:0/1210392890 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:35.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.745+0000 7fe8011b4700 1 --2- 172.21.15.16:0/1210392890 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe7fc09e440 0x7fe7fc134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:35.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.746+0000 7fe8009b3700 1 --2- 172.21.15.16:0/1210392890 >> v2:172.21.15.16:3300/0 conn(0x7fe7fc09d6f0 0x7fe7fc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:35.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.746+0000 7fe7fb7fe700 1 -- 172.21.15.16:0/1210392890 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1214458166 0 0) 0x7fe7fc09b120 con 0x7fe7fc09e000 2023-09-30T22:32:35.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.746+0000 7fe7fb7fe700 1 -- 172.21.15.16:0/1210392890 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7ec0035b0 con 0x7fe7fc09e000 2023-09-30T22:32:35.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.746+0000 7fe8011b4700 1 -- 172.21.15.16:0/1210392890 >> v2:172.21.15.16:3300/0 conn(0x7fe7fc09d6f0 msgr2=0x7fe7fc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:35.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.746+0000 7fe8011b4700 1 --2- 172.21.15.16:0/1210392890 >> v2:172.21.15.16:3300/0 conn(0x7fe7fc09d6f0 0x7fe7fc09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:35.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.746+0000 7fe8011b4700 1 -- 172.21.15.16:0/1210392890 >> v1:172.21.15.16:6789/0 conn(0x7fe7fc09e000 legacy=0x7fe7fc1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:35.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.746+0000 7fe8011b4700 1 -- 172.21.15.16:0/1210392890 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7fc09a500 con 0x7fe7fc09e440 2023-09-30T22:32:35.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.746+0000 7fe8009b3700 1 --2- 172.21.15.16:0/1210392890 >> v2:172.21.15.16:3300/0 conn(0x7fe7fc09d6f0 0x7fe7fc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:32:35.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.746+0000 7fe8011b4700 1 --2- 172.21.15.16:0/1210392890 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe7fc09e440 0x7fe7fc134100 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4009dc0 tx=0x7fe7f400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87fd10a5ceadd75f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:35.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.746+0000 7fe7fb7fe700 1 -- 172.21.15.16:0/1210392890 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f400e070 con 0x7fe7fc09e440 2023-09-30T22:32:35.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.746+0000 7fe7fb7fe700 1 -- 172.21.15.16:0/1210392890 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7f400bae0 con 0x7fe7fc09e440 2023-09-30T22:32:35.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.746+0000 7fe7fb7fe700 1 -- 172.21.15.16:0/1210392890 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f400be90 con 0x7fe7fc09e440 2023-09-30T22:32:35.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.747+0000 7fe802c17700 1 -- 172.21.15.16:0/1210392890 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe7fc09e440 msgr2=0x7fe7fc134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:35.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.747+0000 7fe802c17700 1 --2- 172.21.15.16:0/1210392890 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe7fc09e440 0x7fe7fc134100 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4009dc0 tx=0x7fe7f400b330 comp rx=0 tx=0).stop 2023-09-30T22:32:35.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.747+0000 7fe802c17700 1 -- 172.21.15.16:0/1210392890 shutdown_connections 2023-09-30T22:32:35.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.747+0000 7fe802c17700 1 --2- 172.21.15.16:0/1210392890 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe7fc09e440 0x7fe7fc134100 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:35.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.747+0000 7fe802c17700 1 --2- 172.21.15.16:0/1210392890 >> v2:172.21.15.16:3300/0 conn(0x7fe7fc09d6f0 0x7fe7fc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:35.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.747+0000 7fe802c17700 1 -- 172.21.15.16:0/1210392890 >> 172.21.15.16:0/1210392890 conn(0x7fe7fc066f50 msgr2=0x7fe7fc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:35.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.747+0000 7fe802c17700 1 -- 172.21.15.16:0/1210392890 shutdown_connections 2023-09-30T22:32:35.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.747+0000 7fe802c17700 1 -- 172.21.15.16:0/1210392890 wait complete. 2023-09-30T22:32:35.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.749+0000 7fe802c17700 1 Processor -- start 2023-09-30T22:32:35.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.749+0000 7fe802c17700 1 -- start start 2023-09-30T22:32:35.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.750+0000 7fe802c17700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe7fc09d6f0 0x7fe7fc1a5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:35.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.750+0000 7fe802c17700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7fc09e000 0x7fe7fc1a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:35.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.750+0000 7fe802c17700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe7fc09e440 0x7fe7fc1c7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:35.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.750+0000 7fe802c17700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7fc1a7d90 con 0x7fe7fc09e000 2023-09-30T22:32:35.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.750+0000 7fe802c17700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7fc1a4cd0 con 0x7fe7fc09e440 2023-09-30T22:32:35.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.750+0000 7fe802c17700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe7fc1a6420 con 0x7fe7fc09d6f0 2023-09-30T22:32:35.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.750+0000 7fe8011b4700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe7fc09e440 0x7fe7fc1c7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:35.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.750+0000 7fe8011b4700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe7fc09e440 0x7fe7fc1c7ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:45106/0 (socket says 172.21.15.16:45106) 2023-09-30T22:32:35.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.750+0000 7fe8009b3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe7fc09d6f0 0x7fe7fc1a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:35.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.750+0000 7fe8011b4700 1 -- 172.21.15.16:0/2356638855 learned_addr learned my addr 172.21.15.16:0/2356638855 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:35.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.750+0000 7fe7fbfff700 1 --2- 172.21.15.16:0/2356638855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7fc09e000 0x7fe7fc1a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:35.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.750+0000 7fe8011b4700 1 -- 172.21.15.16:0/2356638855 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe7fc09d6f0 msgr2=0x7fe7fc1a5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:35.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.750+0000 7fe8011b4700 1 --2- 172.21.15.16:0/2356638855 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe7fc09d6f0 0x7fe7fc1a5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:35.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.750+0000 7fe8011b4700 1 -- 172.21.15.16:0/2356638855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7fc09e000 msgr2=0x7fe7fc1a57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:35.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.750+0000 7fe8011b4700 1 --2- 172.21.15.16:0/2356638855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7fc09e000 0x7fe7fc1a57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:35.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.750+0000 7fe8011b4700 1 -- 172.21.15.16:0/2356638855 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7f4009a70 con 0x7fe7fc09e440 2023-09-30T22:32:35.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.750+0000 7fe7fbfff700 1 --2- 172.21.15.16:0/2356638855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7fc09e000 0x7fe7fc1a57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:32:35.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.751+0000 7fe8011b4700 1 --2- 172.21.15.16:0/2356638855 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe7fc09e440 0x7fe7fc1c7ea0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4014040 tx=0x7fe7f4004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:35.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.751+0000 7fe7f97fa700 1 -- 172.21.15.16:0/2356638855 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f4004680 con 0x7fe7fc09e440 2023-09-30T22:32:35.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.751+0000 7fe802c17700 1 -- 172.21.15.16:0/2356638855 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7fc0a2c00 con 0x7fe7fc09e440 2023-09-30T22:32:35.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.751+0000 7fe7f97fa700 1 -- 172.21.15.16:0/2356638855 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7f4020070 con 0x7fe7fc09e440 2023-09-30T22:32:35.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.751+0000 7fe7f97fa700 1 -- 172.21.15.16:0/2356638855 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f4027480 con 0x7fe7fc09e440 2023-09-30T22:32:35.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.751+0000 7fe802c17700 1 -- 172.21.15.16:0/2356638855 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7fc1b92c0 con 0x7fe7fc09e440 2023-09-30T22:32:35.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.752+0000 7fe7f97fa700 1 -- 172.21.15.16:0/2356638855 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe7f4012430 con 0x7fe7fc09e440 2023-09-30T22:32:35.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.752+0000 7fe802c17700 1 -- 172.21.15.16:0/2356638855 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7fc1a5cf0 con 0x7fe7fc09e440 2023-09-30T22:32:35.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.752+0000 7fe7f97fa700 1 -- 172.21.15.16:0/2356638855 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fe7f4022070 con 0x7fe7fc09e440 2023-09-30T22:32:35.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.758+0000 7fe7f97fa700 1 -- 172.21.15.16:0/2356638855 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe7f401a970 con 0x7fe7fc09e440 2023-09-30T22:32:35.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.810+0000 7f24ea759700 1 Processor -- start 2023-09-30T22:32:35.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.810+0000 7f24ea759700 1 -- start start 2023-09-30T22:32:35.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.811+0000 7f24ea759700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f24e40a1510 0x7f24e4138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:35.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.811+0000 7f24ea759700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24e4139250 0x7f24e413b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:35.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.811+0000 7f24ea759700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24e409edd0 con 0x7f24e40a0a30 2023-09-30T22:32:35.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.811+0000 7f24ea759700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24e40a18e0 con 0x7f24e40a1510 2023-09-30T22:32:35.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.811+0000 7f24ea759700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f24e409e1f0 con 0x7f24e4139250 2023-09-30T22:32:35.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.812+0000 7f24e3fff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f24e40a0a30 0x7f24e40a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:39118/0 (socket says 172.21.15.16:39118) 2023-09-30T22:32:35.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.812+0000 7f24e3fff700 1 -- 172.21.15.16:0/4218201033 learned_addr learned my addr 172.21.15.16:0/4218201033 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:35.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.812+0000 7f24e37fe700 1 --2- 172.21.15.16:0/4218201033 >> v2:172.21.15.16:3300/0 conn(0x7f24e40a1510 0x7f24e4138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:35.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.812+0000 7f24e8cf6700 1 --2- 172.21.15.16:0/4218201033 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24e4139250 0x7f24e413b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:35.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.812+0000 7f24e8cf6700 1 -- 172.21.15.16:0/4218201033 >> v2:172.21.15.16:3300/0 conn(0x7f24e40a1510 msgr2=0x7f24e4138d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:35.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.812+0000 7f24e8cf6700 1 --2- 172.21.15.16:0/4218201033 >> v2:172.21.15.16:3300/0 conn(0x7f24e40a1510 0x7f24e4138d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:35.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.812+0000 7f24e8cf6700 1 -- 172.21.15.16:0/4218201033 >> v1:172.21.15.16:6789/0 conn(0x7f24e40a0a30 legacy=0x7f24e40a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:35.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.812+0000 7f24e2ffd700 1 -- 172.21.15.16:0/4218201033 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1060462990 0 0) 0x7f24e409edd0 con 0x7f24e40a0a30 2023-09-30T22:32:35.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.812+0000 7f24e8cf6700 1 -- 172.21.15.16:0/4218201033 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24e409dc50 con 0x7f24e4139250 2023-09-30T22:32:35.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.812+0000 7f24e8cf6700 1 --2- 172.21.15.16:0/4218201033 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24e4139250 0x7f24e413b630 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f24dc009d90 tx=0x7f24dc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35efbb3d441d52e7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:35.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.812+0000 7f24e2ffd700 1 -- 172.21.15.16:0/4218201033 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24dc00e070 con 0x7f24e4139250 2023-09-30T22:32:35.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.813+0000 7f24e2ffd700 1 -- 172.21.15.16:0/4218201033 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24dc00bb20 con 0x7f24e4139250 2023-09-30T22:32:35.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.813+0000 7f24e2ffd700 1 -- 172.21.15.16:0/4218201033 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24dc012430 con 0x7f24e4139250 2023-09-30T22:32:35.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.813+0000 7f24ea759700 1 -- 172.21.15.16:0/4218201033 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24e4139250 msgr2=0x7f24e413b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:35.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.813+0000 7f24ea759700 1 --2- 172.21.15.16:0/4218201033 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24e4139250 0x7f24e413b630 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f24dc009d90 tx=0x7f24dc00b330 comp rx=0 tx=0).stop 2023-09-30T22:32:35.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.814+0000 7f24ea759700 1 -- 172.21.15.16:0/4218201033 shutdown_connections 2023-09-30T22:32:35.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.814+0000 7f24ea759700 1 --2- 172.21.15.16:0/4218201033 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24e4139250 0x7f24e413b630 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:35.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.814+0000 7f24ea759700 1 --2- 172.21.15.16:0/4218201033 >> v2:172.21.15.16:3300/0 conn(0x7f24e40a1510 0x7f24e4138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:35.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.814+0000 7f24ea759700 1 -- 172.21.15.16:0/4218201033 >> 172.21.15.16:0/4218201033 conn(0x7f24e40a6550 msgr2=0x7f24e40a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:35.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.814+0000 7f24ea759700 1 -- 172.21.15.16:0/4218201033 shutdown_connections 2023-09-30T22:32:35.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.814+0000 7f24ea759700 1 -- 172.21.15.16:0/4218201033 wait complete. 2023-09-30T22:32:35.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.815+0000 7f24ea759700 1 Processor -- start 2023-09-30T22:32:35.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.815+0000 7f24ea759700 1 -- start start 2023-09-30T22:32:35.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.816+0000 7f24ea759700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f24e40a0a30 0x7f24e411e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:35.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.816+0000 7f24ea759700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24e40a1510 0x7f24e411eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:35.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.816+0000 7f24ea759700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24e4139250 0x7f24e411f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:35.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.816+0000 7f24ea759700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e4133ed0 con 0x7f24e4139250 2023-09-30T22:32:35.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.816+0000 7f24ea759700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24e41344b0 con 0x7f24e40a0a30 2023-09-30T22:32:35.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.816+0000 7f24ea759700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f24e409c4f0 con 0x7f24e40a1510 2023-09-30T22:32:35.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.816+0000 7f24e37fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24e40a1510 0x7f24e411eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:35.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.816+0000 7f24e37fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24e40a1510 0x7f24e411eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:46588/0 (socket says 172.21.15.16:46588) 2023-09-30T22:32:35.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.816+0000 7f24e37fe700 1 -- 172.21.15.16:0/2865064741 learned_addr learned my addr 172.21.15.16:0/2865064741 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:35.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.816+0000 7f24e8cf6700 1 --2- 172.21.15.16:0/2865064741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24e4139250 0x7f24e411f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:35.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.817+0000 7f24e3fff700 1 --2- 172.21.15.16:0/2865064741 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f24e40a0a30 0x7f24e411e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:35.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.817+0000 7f24e37fe700 1 -- 172.21.15.16:0/2865064741 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f24e40a0a30 msgr2=0x7f24e411e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:35.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.817+0000 7f24e37fe700 1 --2- 172.21.15.16:0/2865064741 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f24e40a0a30 0x7f24e411e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:35.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.817+0000 7f24e37fe700 1 -- 172.21.15.16:0/2865064741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24e4139250 msgr2=0x7f24e411f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:35.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.817+0000 7f24e37fe700 1 --2- 172.21.15.16:0/2865064741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24e4139250 0x7f24e411f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:35.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.817+0000 7f24e37fe700 1 -- 172.21.15.16:0/2865064741 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24dc009a70 con 0x7f24e40a1510 2023-09-30T22:32:35.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.817+0000 7f24e37fe700 1 --2- 172.21.15.16:0/2865064741 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24e40a1510 0x7f24e411eb50 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f24d800e9b0 tx=0x7f24d800ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:35.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.817+0000 7f24e0ff9700 1 -- 172.21.15.16:0/2865064741 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d8009710 con 0x7f24e40a1510 2023-09-30T22:32:35.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.817+0000 7f24e0ff9700 1 -- 172.21.15.16:0/2865064741 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24d800c850 con 0x7f24e40a1510 2023-09-30T22:32:35.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.817+0000 7f24e0ff9700 1 -- 172.21.15.16:0/2865064741 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d800cbc0 con 0x7f24e40a1510 2023-09-30T22:32:35.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.817+0000 7f24ea759700 1 -- 172.21.15.16:0/2865064741 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24e4120a30 con 0x7f24e40a1510 2023-09-30T22:32:35.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.817+0000 7f24ea759700 1 -- 172.21.15.16:0/2865064741 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24e40a0220 con 0x7f24e40a1510 2023-09-30T22:32:35.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.819+0000 7f24e0ff9700 1 -- 172.21.15.16:0/2865064741 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f24d800cd20 con 0x7f24e40a1510 2023-09-30T22:32:35.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.819+0000 7f24ea759700 1 -- 172.21.15.16:0/2865064741 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24e40a1d00 con 0x7f24e40a1510 2023-09-30T22:32:35.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.819+0000 7f24e0ff9700 1 -- 172.21.15.16:0/2865064741 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f24d803fc70 con 0x7f24e40a1510 2023-09-30T22:32:35.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.824+0000 7f24e0ff9700 1 -- 172.21.15.16:0/2865064741 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f24d8021020 con 0x7f24e40a1510 2023-09-30T22:32:35.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.863+0000 7f1158746700 1 -- 172.21.15.16:0/3288717332 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f115006c360 con 0x7f115009d6f0 2023-09-30T22:32:35.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.864+0000 7f1146ffd700 1 -- 172.21.15.16:0/3288717332 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f114c011360 con 0x7f115009d6f0 2023-09-30T22:32:35.881 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:32:35.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.866+0000 7f1158746700 1 -- 172.21.15.16:0/3288717332 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f115009d6f0 msgr2=0x7f11501217f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:35.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.866+0000 7f1158746700 1 --2- 172.21.15.16:0/3288717332 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f115009d6f0 0x7f11501217f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f114c00c520 tx=0x7f114c00c830 comp rx=0 tx=0).stop 2023-09-30T22:32:35.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.866+0000 7f1158746700 1 -- 172.21.15.16:0/3288717332 shutdown_connections 2023-09-30T22:32:35.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.866+0000 7f1158746700 1 --2- 172.21.15.16:0/3288717332 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f115009d6f0 0x7f11501217f0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:35.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.866+0000 7f1158746700 1 --2- 172.21.15.16:0/3288717332 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1150139170 0x7f1150122270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:35.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.866+0000 7f1158746700 1 --2- 172.21.15.16:0/3288717332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f115009e1d0 0x7f1150121d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:35.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.866+0000 7f1158746700 1 -- 172.21.15.16:0/3288717332 >> 172.21.15.16:0/3288717332 conn(0x7f1150066f50 msgr2=0x7f1150070c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:35.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.866+0000 7f1158746700 1 -- 172.21.15.16:0/3288717332 shutdown_connections 2023-09-30T22:32:35.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.866+0000 7f1158746700 1 -- 172.21.15.16:0/3288717332 wait complete. 2023-09-30T22:32:35.884 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:32:35.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.962+0000 7fa5c1833700 1 -- 172.21.15.16:0/683673154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa5bc074ea0 con 0x7fa5bc134610 2023-09-30T22:32:35.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.962+0000 7fa5a3fff700 1 -- 172.21.15.16:0/683673154 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa5b0017020 con 0x7fa5bc134610 2023-09-30T22:32:35.962 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:32:35.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.964+0000 7fa5c1833700 1 -- 172.21.15.16:0/683673154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5bc134610 msgr2=0x7fa5bc121a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:35.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.964+0000 7fa5c1833700 1 --2- 172.21.15.16:0/683673154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5bc134610 0x7fa5bc121a20 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fa5b000e9e0 tx=0x7fa5b000ecf0 comp rx=0 tx=0).stop 2023-09-30T22:32:35.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.964+0000 7fa5c1833700 1 -- 172.21.15.16:0/683673154 shutdown_connections 2023-09-30T22:32:35.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.964+0000 7fa5c1833700 1 --2- 172.21.15.16:0/683673154 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa5bc06e570 0x7fa5bc1214e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:35.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.964+0000 7fa5c1833700 1 --2- 172.21.15.16:0/683673154 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa5bc09e480 0x7fa5bc121f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:35.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.964+0000 7fa5c1833700 1 --2- 172.21.15.16:0/683673154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5bc134610 0x7fa5bc121a20 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:35.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.964+0000 7fa5c1833700 1 -- 172.21.15.16:0/683673154 >> 172.21.15.16:0/683673154 conn(0x7fa5bc0a2aa0 msgr2=0x7fa5bc1bf1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:35.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.964+0000 7fa5c1833700 1 -- 172.21.15.16:0/683673154 shutdown_connections 2023-09-30T22:32:35.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.964+0000 7fa5c1833700 1 -- 172.21.15.16:0/683673154 wait complete. 2023-09-30T22:32:35.979 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:32:35.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.979+0000 7fe802c17700 1 -- 172.21.15.16:0/2356638855 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe7fc09f420 con 0x7fe7fc09e440 2023-09-30T22:32:35.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.980+0000 7fe7f97fa700 1 -- 172.21.15.16:0/2356638855 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe7f4036090 con 0x7fe7fc09e440 2023-09-30T22:32:35.980 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:32:35.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.982+0000 7fe802c17700 1 -- 172.21.15.16:0/2356638855 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe7fc09e440 msgr2=0x7fe7fc1c7ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:35.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.982+0000 7fe802c17700 1 --2- 172.21.15.16:0/2356638855 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe7fc09e440 0x7fe7fc1c7ea0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4014040 tx=0x7fe7f4004ad0 comp rx=0 tx=0).stop 2023-09-30T22:32:35.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.982+0000 7fe802c17700 1 -- 172.21.15.16:0/2356638855 shutdown_connections 2023-09-30T22:32:35.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.982+0000 7fe802c17700 1 --2- 172.21.15.16:0/2356638855 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe7fc09e440 0x7fe7fc1c7ea0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:35.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.982+0000 7fe802c17700 1 --2- 172.21.15.16:0/2356638855 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe7fc09d6f0 0x7fe7fc1a5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:35.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.982+0000 7fe802c17700 1 --2- 172.21.15.16:0/2356638855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7fc09e000 0x7fe7fc1a57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:35.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.982+0000 7fe802c17700 1 -- 172.21.15.16:0/2356638855 >> 172.21.15.16:0/2356638855 conn(0x7fe7fc066f50 msgr2=0x7fe7fc0a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:35.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.982+0000 7fe802c17700 1 -- 172.21.15.16:0/2356638855 shutdown_connections 2023-09-30T22:32:35.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:35.982+0000 7fe802c17700 1 -- 172.21.15.16:0/2356638855 wait complete. 2023-09-30T22:32:35.996 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:32:36.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:36.048+0000 7f24ea759700 1 -- 172.21.15.16:0/2865064741 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f24e4074ea0 con 0x7f24e40a1510 2023-09-30T22:32:36.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:36.048+0000 7f24e0ff9700 1 -- 172.21.15.16:0/2865064741 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f24d8017070 con 0x7f24e40a1510 2023-09-30T22:32:36.048 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:32:36.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:36.050+0000 7f24ea759700 1 -- 172.21.15.16:0/2865064741 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24e40a1510 msgr2=0x7f24e411eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:36.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:36.050+0000 7f24ea759700 1 --2- 172.21.15.16:0/2865064741 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24e40a1510 0x7f24e411eb50 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f24d800e9b0 tx=0x7f24d800ecc0 comp rx=0 tx=0).stop 2023-09-30T22:32:36.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:36.050+0000 7f24ea759700 1 -- 172.21.15.16:0/2865064741 shutdown_connections 2023-09-30T22:32:36.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:36.050+0000 7f24ea759700 1 --2- 172.21.15.16:0/2865064741 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f24e40a0a30 0x7f24e411e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:36.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:36.050+0000 7f24ea759700 1 --2- 172.21.15.16:0/2865064741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24e4139250 0x7f24e411f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:36.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:36.050+0000 7f24ea759700 1 --2- 172.21.15.16:0/2865064741 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24e40a1510 0x7f24e411eb50 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:36.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:36.050+0000 7f24ea759700 1 -- 172.21.15.16:0/2865064741 >> 172.21.15.16:0/2865064741 conn(0x7f24e40a6550 msgr2=0x7f24e4136c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:36.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:36.050+0000 7f24ea759700 1 -- 172.21.15.16:0/2865064741 shutdown_connections 2023-09-30T22:32:36.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:36.050+0000 7f24ea759700 1 -- 172.21.15.16:0/2865064741 wait complete. 2023-09-30T22:32:36.064 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:32:36.885 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:32:36.979 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:32:36.997 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:32:37.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.034+0000 7fbe498ec700 1 Processor -- start 2023-09-30T22:32:37.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.034+0000 7fbe498ec700 1 -- start start 2023-09-30T22:32:37.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.035+0000 7fbe498ec700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbe44130540 0x7fbe44130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:37.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.035+0000 7fbe498ec700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbe44130e50 0x7fbe4406f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:37.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.035+0000 7fbe498ec700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe440716a0 con 0x7fbe4406fb20 2023-09-30T22:32:37.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.035+0000 7fbe498ec700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe4409a390 con 0x7fbe44130540 2023-09-30T22:32:37.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.035+0000 7fbe498ec700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe44071290 con 0x7fbe44130e50 2023-09-30T22:32:37.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.035+0000 7fbe427fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbe44130e50 0x7fbe4406f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:37.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.035+0000 7fbe427fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbe44130e50 0x7fbe4406f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:45128/0 (socket says 172.21.15.16:45128) 2023-09-30T22:32:37.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.035+0000 7fbe427fc700 1 -- 172.21.15.16:0/768526689 learned_addr learned my addr 172.21.15.16:0/768526689 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:37.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.035+0000 7fbe42ffd700 1 --2- 172.21.15.16:0/768526689 >> v2:172.21.15.16:3300/0 conn(0x7fbe44130540 0x7fbe44130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:37.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.035+0000 7fbe427fc700 1 -- 172.21.15.16:0/768526689 >> v2:172.21.15.16:3300/0 conn(0x7fbe44130540 msgr2=0x7fbe44130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:37.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.035+0000 7fbe427fc700 1 --2- 172.21.15.16:0/768526689 >> v2:172.21.15.16:3300/0 conn(0x7fbe44130540 0x7fbe44130910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:37.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.035+0000 7fbe427fc700 1 -- 172.21.15.16:0/768526689 >> v1:172.21.15.16:6789/0 conn(0x7fbe4406fb20 legacy=0x7fbe44137560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:37.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.035+0000 7fbe427fc700 1 -- 172.21.15.16:0/768526689 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe44130340 con 0x7fbe44130e50 2023-09-30T22:32:37.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.035+0000 7fbe42ffd700 1 --2- 172.21.15.16:0/768526689 >> v2:172.21.15.16:3300/0 conn(0x7fbe44130540 0x7fbe44130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:37.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.036+0000 7fbe427fc700 1 --2- 172.21.15.16:0/768526689 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbe44130e50 0x7fbe4406f5e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fbe38009920 tx=0x7fbe38009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c47c5bac6edf1f66 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:37.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.036+0000 7fbe41ffb700 1 -- 172.21.15.16:0/768526689 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe38011830 con 0x7fbe44130e50 2023-09-30T22:32:37.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.036+0000 7fbe41ffb700 1 -- 172.21.15.16:0/768526689 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe38011990 con 0x7fbe44130e50 2023-09-30T22:32:37.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.036+0000 7fbe41ffb700 1 -- 172.21.15.16:0/768526689 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe38011d20 con 0x7fbe44130e50 2023-09-30T22:32:37.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.037+0000 7fbe498ec700 1 -- 172.21.15.16:0/768526689 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbe44130e50 msgr2=0x7fbe4406f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:37.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.037+0000 7fbe498ec700 1 --2- 172.21.15.16:0/768526689 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbe44130e50 0x7fbe4406f5e0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fbe38009920 tx=0x7fbe38009c30 comp rx=0 tx=0).stop 2023-09-30T22:32:37.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.037+0000 7fbe498ec700 1 -- 172.21.15.16:0/768526689 shutdown_connections 2023-09-30T22:32:37.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.037+0000 7fbe498ec700 1 --2- 172.21.15.16:0/768526689 >> v2:172.21.15.16:3300/0 conn(0x7fbe44130540 0x7fbe44130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:37.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.037+0000 7fbe498ec700 1 --2- 172.21.15.16:0/768526689 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbe44130e50 0x7fbe4406f5e0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:37.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.037+0000 7fbe498ec700 1 -- 172.21.15.16:0/768526689 >> 172.21.15.16:0/768526689 conn(0x7fbe440a2aa0 msgr2=0x7fbe440a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:37.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.037+0000 7fbe498ec700 1 -- 172.21.15.16:0/768526689 shutdown_connections 2023-09-30T22:32:37.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.037+0000 7fbe498ec700 1 -- 172.21.15.16:0/768526689 wait complete. 2023-09-30T22:32:37.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.038+0000 7fbe498ec700 1 Processor -- start 2023-09-30T22:32:37.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.038+0000 7fbe498ec700 1 -- start start 2023-09-30T22:32:37.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.038+0000 7fbe498ec700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbe4406fb20 0x7fbe44137e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:37.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.038+0000 7fbe498ec700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe44130540 0x7fbe44138cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:37.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.039+0000 7fbe498ec700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbe44130e50 0x7fbe441391f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:37.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.039+0000 7fbe498ec700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe4412daf0 con 0x7fbe44130540 2023-09-30T22:32:37.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.039+0000 7fbe498ec700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe44071950 con 0x7fbe4406fb20 2023-09-30T22:32:37.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.039+0000 7fbe498ec700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbe440a06a0 con 0x7fbe44130e50 2023-09-30T22:32:37.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.039+0000 7fbe427fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe44130540 0x7fbe44138cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:37.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.039+0000 7fbe427fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe44130540 0x7fbe44138cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56476/0 (socket says 172.21.15.16:56476) 2023-09-30T22:32:37.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.039+0000 7fbe427fc700 1 -- 172.21.15.16:0/4034805222 learned_addr learned my addr 172.21.15.16:0/4034805222 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:37.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.039+0000 7fbe437fe700 1 --2- 172.21.15.16:0/4034805222 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbe44130e50 0x7fbe441391f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:37.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.039+0000 7fbe42ffd700 1 --2- 172.21.15.16:0/4034805222 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbe4406fb20 0x7fbe44137e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:37.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.039+0000 7fbe427fc700 1 -- 172.21.15.16:0/4034805222 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbe44130e50 msgr2=0x7fbe441391f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:37.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.039+0000 7fbe427fc700 1 --2- 172.21.15.16:0/4034805222 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbe44130e50 0x7fbe441391f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:37.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.039+0000 7fbe427fc700 1 -- 172.21.15.16:0/4034805222 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbe4406fb20 msgr2=0x7fbe44137e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:37.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.039+0000 7fbe427fc700 1 --2- 172.21.15.16:0/4034805222 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbe4406fb20 0x7fbe44137e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:37.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.039+0000 7fbe427fc700 1 -- 172.21.15.16:0/4034805222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe340094a0 con 0x7fbe44130540 2023-09-30T22:32:37.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.039+0000 7fbe42ffd700 1 --2- 172.21.15.16:0/4034805222 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbe4406fb20 0x7fbe44137e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:37.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.039+0000 7fbe427fc700 1 --2- 172.21.15.16:0/4034805222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe44130540 0x7fbe44138cb0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fbe38009870 tx=0x7fbe38004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:37.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.040+0000 7fbe2bfff700 1 -- 172.21.15.16:0/4034805222 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe38004510 con 0x7fbe44130540 2023-09-30T22:32:37.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.040+0000 7fbe2bfff700 1 -- 172.21.15.16:0/4034805222 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe380036a0 con 0x7fbe44130540 2023-09-30T22:32:37.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.040+0000 7fbe2bfff700 1 -- 172.21.15.16:0/4034805222 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe38003a50 con 0x7fbe44130540 2023-09-30T22:32:37.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.040+0000 7fbe498ec700 1 -- 172.21.15.16:0/4034805222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe38009550 con 0x7fbe44130540 2023-09-30T22:32:37.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.040+0000 7fbe498ec700 1 -- 172.21.15.16:0/4034805222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe4412b670 con 0x7fbe44130540 2023-09-30T22:32:37.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.041+0000 7fbe498ec700 1 -- 172.21.15.16:0/4034805222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe44074ea0 con 0x7fbe44130540 2023-09-30T22:32:37.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.041+0000 7fbe2bfff700 1 -- 172.21.15.16:0/4034805222 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbe38003bb0 con 0x7fbe44130540 2023-09-30T22:32:37.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.041+0000 7fbe2bfff700 1 -- 172.21.15.16:0/4034805222 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fbe38040550 con 0x7fbe44130540 2023-09-30T22:32:37.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.046+0000 7fbe2bfff700 1 -- 172.21.15.16:0/4034805222 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbe38010380 con 0x7fbe44130540 2023-09-30T22:32:37.065 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:32:37.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.125+0000 7ff8e3be1700 1 Processor -- start 2023-09-30T22:32:37.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.125+0000 7ff8e3be1700 1 -- start start 2023-09-30T22:32:37.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.126+0000 7ff8e3be1700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff8dc133420 0x7ff8dc1337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:37.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.126+0000 7ff8e3be1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff8dc133e30 0x7ff8dc12ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:37.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.126+0000 7ff8e3be1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff8dc134260 0x7ff8dc1304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:37.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.126+0000 7ff8e3be1700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8dc06b990 con 0x7ff8dc133420 2023-09-30T22:32:37.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.126+0000 7ff8e3be1700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8dc0a0590 con 0x7ff8dc133e30 2023-09-30T22:32:37.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.126+0000 7ff8e3be1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff8dc09fb80 con 0x7ff8dc134260 2023-09-30T22:32:37.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.126+0000 7ff8e197d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff8dc133420 0x7ff8dc1337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:37.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.126+0000 7ff8e197d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff8dc133420 0x7ff8dc1337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56482/0 (socket says 172.21.15.16:56482) 2023-09-30T22:32:37.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.126+0000 7ff8e197d700 1 -- 172.21.15.16:0/3559710969 learned_addr learned my addr 172.21.15.16:0/3559710969 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:37.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.126+0000 7ff8e117c700 1 --2- 172.21.15.16:0/3559710969 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff8dc133e30 0x7ff8dc12ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:37.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.126+0000 7ff8e217e700 1 --2- 172.21.15.16:0/3559710969 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff8dc134260 0x7ff8dc1304d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:37.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.127+0000 7ff8e197d700 1 -- 172.21.15.16:0/3559710969 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff8dc134260 msgr2=0x7ff8dc1304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:37.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.127+0000 7ff8e197d700 1 --2- 172.21.15.16:0/3559710969 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff8dc134260 0x7ff8dc1304d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:37.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.127+0000 7ff8e197d700 1 -- 172.21.15.16:0/3559710969 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff8dc133e30 msgr2=0x7ff8dc12ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:37.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.127+0000 7ff8e197d700 1 --2- 172.21.15.16:0/3559710969 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff8dc133e30 0x7ff8dc12ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:37.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.127+0000 7ff8e197d700 1 -- 172.21.15.16:0/3559710969 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8dc0a0030 con 0x7ff8dc133420 2023-09-30T22:32:37.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.127+0000 7ff8e117c700 1 --2- 172.21.15.16:0/3559710969 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff8dc133e30 0x7ff8dc12ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:37.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.127+0000 7ff8e217e700 1 --2- 172.21.15.16:0/3559710969 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff8dc134260 0x7ff8dc1304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:32:37.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.127+0000 7ff8e197d700 1 --2- 172.21.15.16:0/3559710969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8dc133420 0x7ff8dc1337f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7ff8d80098f0 tx=0x7ff8d8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f7382885333e01 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:37.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.127+0000 7ff8e097b700 1 -- 172.21.15.16:0/3559710969 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d8011830 con 0x7ff8dc133420 2023-09-30T22:32:37.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.127+0000 7ff8e097b700 1 -- 172.21.15.16:0/3559710969 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8d8011990 con 0x7ff8dc133420 2023-09-30T22:32:37.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.127+0000 7ff8e097b700 1 -- 172.21.15.16:0/3559710969 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d8011d20 con 0x7ff8dc133420 2023-09-30T22:32:37.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.128+0000 7ff8e3be1700 1 -- 172.21.15.16:0/3559710969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8dc133420 msgr2=0x7ff8dc1337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:37.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.128+0000 7ff8e3be1700 1 --2- 172.21.15.16:0/3559710969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8dc133420 0x7ff8dc1337f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7ff8d80098f0 tx=0x7ff8d8009c00 comp rx=0 tx=0).stop 2023-09-30T22:32:37.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.128+0000 7ff8e3be1700 1 -- 172.21.15.16:0/3559710969 shutdown_connections 2023-09-30T22:32:37.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.128+0000 7ff8e3be1700 1 --2- 172.21.15.16:0/3559710969 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff8dc134260 0x7ff8dc1304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:37.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.128+0000 7ff8e3be1700 1 --2- 172.21.15.16:0/3559710969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8dc133420 0x7ff8dc1337f0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:37.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.128+0000 7ff8e3be1700 1 --2- 172.21.15.16:0/3559710969 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff8dc133e30 0x7ff8dc12ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:37.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.128+0000 7ff8e3be1700 1 -- 172.21.15.16:0/3559710969 >> 172.21.15.16:0/3559710969 conn(0x7ff8dc0a4210 msgr2=0x7ff8dc0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:37.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.128+0000 7ff8e3be1700 1 -- 172.21.15.16:0/3559710969 shutdown_connections 2023-09-30T22:32:37.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.128+0000 7ff8e3be1700 1 -- 172.21.15.16:0/3559710969 wait complete. 2023-09-30T22:32:37.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.129+0000 7ff8e3be1700 1 Processor -- start 2023-09-30T22:32:37.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.129+0000 7ff8e3be1700 1 -- start start 2023-09-30T22:32:37.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.130+0000 7ff8e3be1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8dc133420 0x7ff8dc09c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:37.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.130+0000 7ff8e3be1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff8dc133e30 0x7ff8dc09c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:37.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.130+0000 7ff8e3be1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff8dc134260 0x7ff8dc09cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:37.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.130+0000 7ff8e3be1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8dc131470 con 0x7ff8dc133420 2023-09-30T22:32:37.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.130+0000 7ff8e3be1700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8dc120170 con 0x7ff8dc133e30 2023-09-30T22:32:37.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.130+0000 7ff8e3be1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff8dc12dc30 con 0x7ff8dc134260 2023-09-30T22:32:37.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.130+0000 7ff8e117c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff8dc133e30 0x7ff8dc09c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:37.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.130+0000 7ff8e117c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff8dc133e30 0x7ff8dc09c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:45158/0 (socket says 172.21.15.16:45158) 2023-09-30T22:32:37.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.130+0000 7ff8e117c700 1 -- 172.21.15.16:0/1208504544 learned_addr learned my addr 172.21.15.16:0/1208504544 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:37.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.130+0000 7ff8e217e700 1 --2- 172.21.15.16:0/1208504544 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff8dc134260 0x7ff8dc09cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:37.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.130+0000 7ff8e197d700 1 --2- 172.21.15.16:0/1208504544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8dc133420 0x7ff8dc09c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:37.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.130+0000 7ff8e117c700 1 -- 172.21.15.16:0/1208504544 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff8dc134260 msgr2=0x7ff8dc09cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:37.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.130+0000 7ff8e117c700 1 --2- 172.21.15.16:0/1208504544 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff8dc134260 0x7ff8dc09cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:37.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.130+0000 7ff8e117c700 1 -- 172.21.15.16:0/1208504544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8dc133420 msgr2=0x7ff8dc09c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:37.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.130+0000 7ff8e117c700 1 --2- 172.21.15.16:0/1208504544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8dc133420 0x7ff8dc09c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:37.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.130+0000 7ff8e117c700 1 -- 172.21.15.16:0/1208504544 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8cc0094a0 con 0x7ff8dc133e30 2023-09-30T22:32:37.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.130+0000 7ff8e197d700 1 --2- 172.21.15.16:0/1208504544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8dc133420 0x7ff8dc09c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:32:37.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.131+0000 7ff8e117c700 1 --2- 172.21.15.16:0/1208504544 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff8dc133e30 0x7ff8dc09c9c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc00ec40 tx=0x7ff8cc00ef50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:37.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.131+0000 7ff8d27fc700 1 -- 172.21.15.16:0/1208504544 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8cc00c910 con 0x7ff8dc133e30 2023-09-30T22:32:37.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.131+0000 7ff8d27fc700 1 -- 172.21.15.16:0/1208504544 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8cc00ca70 con 0x7ff8dc133e30 2023-09-30T22:32:37.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.131+0000 7ff8d27fc700 1 -- 172.21.15.16:0/1208504544 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8cc00ce00 con 0x7ff8dc133e30 2023-09-30T22:32:37.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.131+0000 7ff8e3be1700 1 -- 172.21.15.16:0/1208504544 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8d8009550 con 0x7ff8dc133e30 2023-09-30T22:32:37.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.131+0000 7ff8e3be1700 1 -- 172.21.15.16:0/1208504544 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8dc11fbd0 con 0x7ff8dc133e30 2023-09-30T22:32:37.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.132+0000 7ff8d27fc700 1 -- 172.21.15.16:0/1208504544 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff8cc013470 con 0x7ff8dc133e30 2023-09-30T22:32:37.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.132+0000 7ff8d27fc700 1 -- 172.21.15.16:0/1208504544 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7ff8cc03f870 con 0x7ff8dc133e30 2023-09-30T22:32:37.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.132+0000 7ff8e3be1700 1 -- 172.21.15.16:0/1208504544 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8dc074ea0 con 0x7ff8dc133e30 2023-09-30T22:32:37.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.142+0000 7ff8d27fc700 1 -- 172.21.15.16:0/1208504544 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff8cc0232f0 con 0x7ff8dc133e30 2023-09-30T22:32:37.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.168+0000 7f23847e9700 1 Processor -- start 2023-09-30T22:32:37.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.168+0000 7f23847e9700 1 -- start start 2023-09-30T22:32:37.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.169+0000 7f23847e9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f237c09d630 0x7f237c0a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:37.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.169+0000 7f23847e9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f237c138d80 0x7f237c13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:37.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.169+0000 7f23847e9700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f237c0a2a70 con 0x7f237c0a0a30 2023-09-30T22:32:37.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.169+0000 7f23847e9700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f237c0a01d0 con 0x7f237c09d630 2023-09-30T22:32:37.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.169+0000 7f23847e9700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f237c09e2b0 con 0x7f237c138d80 2023-09-30T22:32:37.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.169+0000 7f2381d84700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f237c09d630 0x7f237c0a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:37.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.169+0000 7f2382585700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f237c0a0a30 0x7f237c0a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:39148/0 (socket says 172.21.15.16:39148) 2023-09-30T22:32:37.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.169+0000 7f2382585700 1 -- 172.21.15.16:0/3822506603 learned_addr learned my addr 172.21.15.16:0/3822506603 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:37.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.169+0000 7f2382d86700 1 --2- 172.21.15.16:0/3822506603 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f237c138d80 0x7f237c13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:37.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.169+0000 7f2381d84700 1 -- 172.21.15.16:0/3822506603 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f237c138d80 msgr2=0x7f237c13b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:37.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.169+0000 7f2381d84700 1 --2- 172.21.15.16:0/3822506603 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f237c138d80 0x7f237c13b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:37.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.169+0000 7f2381d84700 1 -- 172.21.15.16:0/3822506603 >> v1:172.21.15.16:6789/0 conn(0x7f237c0a0a30 legacy=0x7f237c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:37.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.169+0000 7f2381d84700 1 -- 172.21.15.16:0/3822506603 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f237c09dc50 con 0x7f237c09d630 2023-09-30T22:32:37.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.169+0000 7f2382d86700 1 --2- 172.21.15.16:0/3822506603 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f237c138d80 0x7f237c13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:32:37.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.170+0000 7f2381d84700 1 --2- 172.21.15.16:0/3822506603 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f237c09d630 0x7f237c0a1510 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f236c009870 tx=0x7f236c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e3cb3d58d944798 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:37.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.170+0000 7f2381583700 1 -- 172.21.15.16:0/3822506603 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236c011830 con 0x7f237c09d630 2023-09-30T22:32:37.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.170+0000 7f2381583700 1 -- 172.21.15.16:0/3822506603 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f236c011990 con 0x7f237c09d630 2023-09-30T22:32:37.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.170+0000 7f2381583700 1 -- 172.21.15.16:0/3822506603 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f236c011d20 con 0x7f237c09d630 2023-09-30T22:32:37.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.170+0000 7f23847e9700 1 -- 172.21.15.16:0/3822506603 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f237c09d630 msgr2=0x7f237c0a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:37.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.170+0000 7f23847e9700 1 --2- 172.21.15.16:0/3822506603 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f237c09d630 0x7f237c0a1510 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f236c009870 tx=0x7f236c009b80 comp rx=0 tx=0).stop 2023-09-30T22:32:37.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.171+0000 7f23847e9700 1 -- 172.21.15.16:0/3822506603 shutdown_connections 2023-09-30T22:32:37.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.171+0000 7f23847e9700 1 --2- 172.21.15.16:0/3822506603 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f237c138d80 0x7f237c13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:37.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.171+0000 7f23847e9700 1 --2- 172.21.15.16:0/3822506603 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f237c09d630 0x7f237c0a1510 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:37.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.171+0000 7f23847e9700 1 -- 172.21.15.16:0/3822506603 >> 172.21.15.16:0/3822506603 conn(0x7f237c0a3050 msgr2=0x7f237c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:37.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.171+0000 7f23847e9700 1 -- 172.21.15.16:0/3822506603 shutdown_connections 2023-09-30T22:32:37.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.171+0000 7f23847e9700 1 -- 172.21.15.16:0/3822506603 wait complete. 2023-09-30T22:32:37.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.172+0000 7f23847e9700 1 Processor -- start 2023-09-30T22:32:37.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.172+0000 7f23847e9700 1 -- start start 2023-09-30T22:32:37.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.172+0000 7f23847e9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f237c09d630 0x7f237c1207d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:37.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.172+0000 7f23847e9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f237c0a0a30 0x7f237c120d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:37.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.172+0000 7f23847e9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f237c138d80 0x7f237c121250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:37.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.172+0000 7f23847e9700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237c122940 con 0x7f237c138d80 2023-09-30T22:32:37.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.172+0000 7f23847e9700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f237c123660 con 0x7f237c09d630 2023-09-30T22:32:37.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.172+0000 7f23847e9700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f237c122c00 con 0x7f237c0a0a30 2023-09-30T22:32:37.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.172+0000 7f2382585700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f237c09d630 0x7f237c1207d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:37.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.172+0000 7f2382585700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f237c09d630 0x7f237c1207d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:45172/0 (socket says 172.21.15.16:45172) 2023-09-30T22:32:37.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.172+0000 7f2382585700 1 -- 172.21.15.16:0/1024089067 learned_addr learned my addr 172.21.15.16:0/1024089067 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:37.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.172+0000 7f2382d86700 1 --2- 172.21.15.16:0/1024089067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f237c138d80 0x7f237c121250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:37.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.173+0000 7f2381d84700 1 --2- 172.21.15.16:0/1024089067 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f237c0a0a30 0x7f237c120d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:37.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.173+0000 7f2382585700 1 -- 172.21.15.16:0/1024089067 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f237c0a0a30 msgr2=0x7f237c120d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:37.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.173+0000 7f2382585700 1 --2- 172.21.15.16:0/1024089067 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f237c0a0a30 0x7f237c120d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:37.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.173+0000 7f2382585700 1 -- 172.21.15.16:0/1024089067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f237c138d80 msgr2=0x7f237c121250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:37.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.173+0000 7f2382585700 1 --2- 172.21.15.16:0/1024089067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f237c138d80 0x7f237c121250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:37.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.173+0000 7f2382585700 1 -- 172.21.15.16:0/1024089067 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f236c0094a0 con 0x7f237c09d630 2023-09-30T22:32:37.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.173+0000 7f2381d84700 1 --2- 172.21.15.16:0/1024089067 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f237c0a0a30 0x7f237c120d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:32:37.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.173+0000 7f2382585700 1 --2- 172.21.15.16:0/1024089067 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f237c09d630 0x7f237c1207d0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f237800c550 tx=0x7f237800c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:37.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.173+0000 7f2372ffd700 1 -- 172.21.15.16:0/1024089067 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2378005d70 con 0x7f237c09d630 2023-09-30T22:32:37.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.173+0000 7f2372ffd700 1 -- 172.21.15.16:0/1024089067 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23780070d0 con 0x7f237c09d630 2023-09-30T22:32:37.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.173+0000 7f2372ffd700 1 -- 172.21.15.16:0/1024089067 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2378007440 con 0x7f237c09d630 2023-09-30T22:32:37.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.173+0000 7f23847e9700 1 -- 172.21.15.16:0/1024089067 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f237c1242b0 con 0x7f237c09d630 2023-09-30T22:32:37.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.174+0000 7f23847e9700 1 -- 172.21.15.16:0/1024089067 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f237c0a0410 con 0x7f237c09d630 2023-09-30T22:32:37.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.175+0000 7f2372ffd700 1 -- 172.21.15.16:0/1024089067 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f23780075a0 con 0x7f237c09d630 2023-09-30T22:32:37.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.175+0000 7f23847e9700 1 -- 172.21.15.16:0/1024089067 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f237c0747f0 con 0x7f237c09d630 2023-09-30T22:32:37.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.175+0000 7f2372ffd700 1 -- 172.21.15.16:0/1024089067 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f237803eaf0 con 0x7f237c09d630 2023-09-30T22:32:37.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.184+0000 7f2372ffd700 1 -- 172.21.15.16:0/1024089067 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2378011d70 con 0x7f237c09d630 2023-09-30T22:32:37.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.244+0000 7f12d3ace700 1 Processor -- start 2023-09-30T22:32:37.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.245+0000 7f12d3ace700 1 -- start start 2023-09-30T22:32:37.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.245+0000 7f12d3ace700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f12cc1337f0 0x7f12cc138880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:37.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.245+0000 7f12d3ace700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12cc138dc0 0x7f12cc13b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:37.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.245+0000 7f12d3ace700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12cc06b990 con 0x7f12cc133420 2023-09-30T22:32:37.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.245+0000 7f12d3ace700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12cc09b0e0 con 0x7f12cc1337f0 2023-09-30T22:32:37.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.245+0000 7f12d3ace700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc133c20 con 0x7f12cc138dc0 2023-09-30T22:32:37.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.245+0000 7f12d1069700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f12cc1337f0 0x7f12cc138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:37.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.245+0000 7f12d1069700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f12cc1337f0 0x7f12cc138880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56512/0 (socket says 172.21.15.16:56512) 2023-09-30T22:32:37.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.245+0000 7f12d1069700 1 -- 172.21.15.16:0/1952787059 learned_addr learned my addr 172.21.15.16:0/1952787059 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:37.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.245+0000 7f12d206b700 1 --2- 172.21.15.16:0/1952787059 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12cc138dc0 0x7f12cc13b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:37.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.246+0000 7f12d1069700 1 -- 172.21.15.16:0/1952787059 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12cc138dc0 msgr2=0x7f12cc13b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:37.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.246+0000 7f12d1069700 1 --2- 172.21.15.16:0/1952787059 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12cc138dc0 0x7f12cc13b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:37.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.246+0000 7f12d0868700 1 -- 172.21.15.16:0/1952787059 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3080367844 0 0) 0x7f12cc06b990 con 0x7f12cc133420 2023-09-30T22:32:37.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.246+0000 7f12d1069700 1 -- 172.21.15.16:0/1952787059 >> v1:172.21.15.16:6789/0 conn(0x7f12cc133420 legacy=0x7f12cc133e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:37.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.246+0000 7f12d1069700 1 -- 172.21.15.16:0/1952787059 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12cc0a0590 con 0x7f12cc1337f0 2023-09-30T22:32:37.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.246+0000 7f12d206b700 1 --2- 172.21.15.16:0/1952787059 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12cc138dc0 0x7f12cc13b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:37.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.246+0000 7f12d1069700 1 --2- 172.21.15.16:0/1952787059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12cc1337f0 0x7f12cc138880 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f12bc0098f0 tx=0x7f12bc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c49386e250b3a1c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:37.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.246+0000 7f12d0868700 1 -- 172.21.15.16:0/1952787059 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12bc011830 con 0x7f12cc1337f0 2023-09-30T22:32:37.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.246+0000 7f12d0868700 1 -- 172.21.15.16:0/1952787059 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12bc011990 con 0x7f12cc1337f0 2023-09-30T22:32:37.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.246+0000 7f12d0868700 1 -- 172.21.15.16:0/1952787059 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12bc011d20 con 0x7f12cc1337f0 2023-09-30T22:32:37.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.247+0000 7f12d3ace700 1 -- 172.21.15.16:0/1952787059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12cc1337f0 msgr2=0x7f12cc138880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:37.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.247+0000 7f12d3ace700 1 --2- 172.21.15.16:0/1952787059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12cc1337f0 0x7f12cc138880 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f12bc0098f0 tx=0x7f12bc009c00 comp rx=0 tx=0).stop 2023-09-30T22:32:37.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.247+0000 7f12d3ace700 1 -- 172.21.15.16:0/1952787059 shutdown_connections 2023-09-30T22:32:37.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.247+0000 7f12d3ace700 1 --2- 172.21.15.16:0/1952787059 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12cc138dc0 0x7f12cc13b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:37.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.247+0000 7f12d3ace700 1 --2- 172.21.15.16:0/1952787059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12cc1337f0 0x7f12cc138880 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:37.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.247+0000 7f12d3ace700 1 -- 172.21.15.16:0/1952787059 >> 172.21.15.16:0/1952787059 conn(0x7f12cc0a4210 msgr2=0x7f12cc0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:37.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.247+0000 7f12d3ace700 1 -- 172.21.15.16:0/1952787059 shutdown_connections 2023-09-30T22:32:37.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.247+0000 7f12d3ace700 1 -- 172.21.15.16:0/1952787059 wait complete. 2023-09-30T22:32:37.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.248+0000 7f12d3ace700 1 Processor -- start 2023-09-30T22:32:37.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.248+0000 7f12d3ace700 1 -- start start 2023-09-30T22:32:37.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.249+0000 7f12d3ace700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12cc133420 0x7f12cc137810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:37.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.249+0000 7f12d3ace700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12cc1337f0 0x7f12cc137d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:37.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.249+0000 7f12d3ace700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12cc138dc0 0x7f12cc09c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:37.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.249+0000 7f12d3ace700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc134650 con 0x7f12cc138dc0 2023-09-30T22:32:37.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.249+0000 7f12d3ace700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc09e020 con 0x7f12cc1337f0 2023-09-30T22:32:37.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.249+0000 7f12d3ace700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f12cc130ab0 con 0x7f12cc133420 2023-09-30T22:32:37.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.249+0000 7f12d186a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12cc133420 0x7f12cc137810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:37.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.249+0000 7f12d186a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12cc133420 0x7f12cc137810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:46652/0 (socket says 172.21.15.16:46652) 2023-09-30T22:32:37.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.249+0000 7f12d186a700 1 -- 172.21.15.16:0/2921338893 learned_addr learned my addr 172.21.15.16:0/2921338893 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:37.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.249+0000 7f12d206b700 1 --2- 172.21.15.16:0/2921338893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12cc138dc0 0x7f12cc09c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:37.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.249+0000 7f12d1069700 1 --2- 172.21.15.16:0/2921338893 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12cc1337f0 0x7f12cc137d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:37.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.250+0000 7f12d186a700 1 -- 172.21.15.16:0/2921338893 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12cc1337f0 msgr2=0x7f12cc137d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:37.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.250+0000 7f12d186a700 1 --2- 172.21.15.16:0/2921338893 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12cc1337f0 0x7f12cc137d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:37.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.250+0000 7f12d186a700 1 -- 172.21.15.16:0/2921338893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12cc138dc0 msgr2=0x7f12cc09c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:37.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.250+0000 7f12d186a700 1 --2- 172.21.15.16:0/2921338893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12cc138dc0 0x7f12cc09c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:37.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.250+0000 7f12d186a700 1 -- 172.21.15.16:0/2921338893 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c4009a70 con 0x7f12cc133420 2023-09-30T22:32:37.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.250+0000 7f12d1069700 1 --2- 172.21.15.16:0/2921338893 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12cc1337f0 0x7f12cc137d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:32:37.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.250+0000 7f12d186a700 1 --2- 172.21.15.16:0/2921338893 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12cc133420 0x7f12cc137810 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f12c8001eb0 tx=0x7f12c800c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:37.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.250+0000 7f12c27fc700 1 -- 172.21.15.16:0/2921338893 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c80054e0 con 0x7f12cc133420 2023-09-30T22:32:37.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.250+0000 7f12c27fc700 1 -- 172.21.15.16:0/2921338893 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c8003680 con 0x7f12cc133420 2023-09-30T22:32:37.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.250+0000 7f12d3ace700 1 -- 172.21.15.16:0/2921338893 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12bc009550 con 0x7f12cc133420 2023-09-30T22:32:37.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.250+0000 7f12c27fc700 1 -- 172.21.15.16:0/2921338893 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c80039f0 con 0x7f12cc133420 2023-09-30T22:32:37.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.250+0000 7f12d3ace700 1 -- 172.21.15.16:0/2921338893 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12cc12f280 con 0x7f12cc133420 2023-09-30T22:32:37.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.251+0000 7f12d3ace700 1 -- 172.21.15.16:0/2921338893 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12cc0a47a0 con 0x7f12cc133420 2023-09-30T22:32:37.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.251+0000 7f12c27fc700 1 -- 172.21.15.16:0/2921338893 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f12c8003b50 con 0x7f12cc133420 2023-09-30T22:32:37.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.252+0000 7f12c27fc700 1 -- 172.21.15.16:0/2921338893 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f12c803ea70 con 0x7f12cc133420 2023-09-30T22:32:37.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.256+0000 7f12c27fc700 1 -- 172.21.15.16:0/2921338893 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f12c8040e60 con 0x7f12cc133420 2023-09-30T22:32:37.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.280+0000 7fbe498ec700 1 -- 172.21.15.16:0/4034805222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fbe4412d580 con 0x7fbe44130540 2023-09-30T22:32:37.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.280+0000 7fbe2bfff700 1 -- 172.21.15.16:0/4034805222 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbe38015070 con 0x7fbe44130540 2023-09-30T22:32:37.281 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:32:37.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.282+0000 7fbe498ec700 1 -- 172.21.15.16:0/4034805222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe44130540 msgr2=0x7fbe44138cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:37.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.282+0000 7fbe498ec700 1 --2- 172.21.15.16:0/4034805222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe44130540 0x7fbe44138cb0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fbe38009870 tx=0x7fbe38004750 comp rx=0 tx=0).stop 2023-09-30T22:32:37.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.282+0000 7fbe498ec700 1 -- 172.21.15.16:0/4034805222 shutdown_connections 2023-09-30T22:32:37.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.282+0000 7fbe498ec700 1 --2- 172.21.15.16:0/4034805222 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbe4406fb20 0x7fbe44137e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:37.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.282+0000 7fbe498ec700 1 --2- 172.21.15.16:0/4034805222 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbe44130e50 0x7fbe441391f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:37.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.282+0000 7fbe498ec700 1 --2- 172.21.15.16:0/4034805222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe44130540 0x7fbe44138cb0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:37.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.282+0000 7fbe498ec700 1 -- 172.21.15.16:0/4034805222 >> 172.21.15.16:0/4034805222 conn(0x7fbe440a2aa0 msgr2=0x7fbe4406e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:37.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.283+0000 7fbe498ec700 1 -- 172.21.15.16:0/4034805222 shutdown_connections 2023-09-30T22:32:37.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.283+0000 7fbe498ec700 1 -- 172.21.15.16:0/4034805222 wait complete. 2023-09-30T22:32:37.298 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:32:37.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.392+0000 7ff8e3be1700 1 -- 172.21.15.16:0/1208504544 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff8dc06b250 con 0x7ff8dc133e30 2023-09-30T22:32:37.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.392+0000 7f23847e9700 1 -- 172.21.15.16:0/1024089067 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f237c124090 con 0x7f237c09d630 2023-09-30T22:32:37.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.392+0000 7ff8d27fc700 1 -- 172.21.15.16:0/1208504544 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff8cc021030 con 0x7ff8dc133e30 2023-09-30T22:32:37.393 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:32:37.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.392+0000 7f2372ffd700 1 -- 172.21.15.16:0/1024089067 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f237803ee60 con 0x7f237c09d630 2023-09-30T22:32:37.394 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:32:37.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.394+0000 7ff8e3be1700 1 -- 172.21.15.16:0/1208504544 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff8dc133e30 msgr2=0x7ff8dc09c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:37.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.394+0000 7ff8e3be1700 1 --2- 172.21.15.16:0/1208504544 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff8dc133e30 0x7ff8dc09c9c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc00ec40 tx=0x7ff8cc00ef50 comp rx=0 tx=0).stop 2023-09-30T22:32:37.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.395+0000 7ff8e3be1700 1 -- 172.21.15.16:0/1208504544 shutdown_connections 2023-09-30T22:32:37.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.395+0000 7ff8e3be1700 1 --2- 172.21.15.16:0/1208504544 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff8dc134260 0x7ff8dc09cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:37.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.395+0000 7ff8e3be1700 1 --2- 172.21.15.16:0/1208504544 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8dc133420 0x7ff8dc09c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:37.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.395+0000 7ff8e3be1700 1 --2- 172.21.15.16:0/1208504544 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff8dc133e30 0x7ff8dc09c9c0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:37.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.395+0000 7ff8e3be1700 1 -- 172.21.15.16:0/1208504544 >> 172.21.15.16:0/1208504544 conn(0x7ff8dc0a4210 msgr2=0x7ff8dc130250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:37.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.395+0000 7f23847e9700 1 -- 172.21.15.16:0/1024089067 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f237c09d630 msgr2=0x7f237c1207d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:37.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.395+0000 7f23847e9700 1 --2- 172.21.15.16:0/1024089067 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f237c09d630 0x7f237c1207d0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f237800c550 tx=0x7f237800c860 comp rx=0 tx=0).stop 2023-09-30T22:32:37.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.395+0000 7f23847e9700 1 -- 172.21.15.16:0/1024089067 shutdown_connections 2023-09-30T22:32:37.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.395+0000 7f23847e9700 1 --2- 172.21.15.16:0/1024089067 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f237c09d630 0x7f237c1207d0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:37.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.395+0000 7f23847e9700 1 --2- 172.21.15.16:0/1024089067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f237c138d80 0x7f237c121250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:37.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.395+0000 7f23847e9700 1 --2- 172.21.15.16:0/1024089067 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f237c0a0a30 0x7f237c120d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:37.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.395+0000 7f23847e9700 1 -- 172.21.15.16:0/1024089067 >> 172.21.15.16:0/1024089067 conn(0x7f237c0a3050 msgr2=0x7f237c09ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:37.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.395+0000 7ff8e3be1700 1 -- 172.21.15.16:0/1208504544 shutdown_connections 2023-09-30T22:32:37.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.395+0000 7ff8e3be1700 1 -- 172.21.15.16:0/1208504544 wait complete. 2023-09-30T22:32:37.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.395+0000 7f23847e9700 1 -- 172.21.15.16:0/1024089067 shutdown_connections 2023-09-30T22:32:37.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.395+0000 7f23847e9700 1 -- 172.21.15.16:0/1024089067 wait complete. 2023-09-30T22:32:37.410 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:32:37.411 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:32:37.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.463+0000 7f12d3ace700 1 -- 172.21.15.16:0/2921338893 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f12cc074ea0 con 0x7f12cc133420 2023-09-30T22:32:37.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.464+0000 7f12c27fc700 1 -- 172.21.15.16:0/2921338893 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f12c8017030 con 0x7f12cc133420 2023-09-30T22:32:37.464 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:32:37.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.466+0000 7f12d3ace700 1 -- 172.21.15.16:0/2921338893 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12cc133420 msgr2=0x7f12cc137810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:37.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.466+0000 7f12d3ace700 1 --2- 172.21.15.16:0/2921338893 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12cc133420 0x7f12cc137810 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f12c8001eb0 tx=0x7f12c800c8b0 comp rx=0 tx=0).stop 2023-09-30T22:32:37.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.466+0000 7f12d3ace700 1 -- 172.21.15.16:0/2921338893 shutdown_connections 2023-09-30T22:32:37.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.466+0000 7f12d3ace700 1 --2- 172.21.15.16:0/2921338893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12cc138dc0 0x7f12cc09c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:37.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.466+0000 7f12d3ace700 1 --2- 172.21.15.16:0/2921338893 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12cc133420 0x7f12cc137810 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:37.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.466+0000 7f12d3ace700 1 --2- 172.21.15.16:0/2921338893 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12cc1337f0 0x7f12cc137d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:37.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.466+0000 7f12d3ace700 1 -- 172.21.15.16:0/2921338893 >> 172.21.15.16:0/2921338893 conn(0x7f12cc0a4210 msgr2=0x7f12cc0a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:37.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.466+0000 7f12d3ace700 1 -- 172.21.15.16:0/2921338893 shutdown_connections 2023-09-30T22:32:37.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:37.466+0000 7f12d3ace700 1 -- 172.21.15.16:0/2921338893 wait complete. 2023-09-30T22:32:37.480 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:32:38.299 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:32:38.411 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:32:38.412 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:32:38.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.445+0000 7f4437c3b700 1 Processor -- start 2023-09-30T22:32:38.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.445+0000 7f4437c3b700 1 -- start start 2023-09-30T22:32:38.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.446+0000 7f4437c3b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f443009d6f0 0x7f443009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:38.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.446+0000 7f4437c3b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f443009e000 0x7f44301339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:38.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.446+0000 7f4437c3b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f443009b120 con 0x7f443009e410 2023-09-30T22:32:38.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.446+0000 7f4437c3b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f443006b460 con 0x7f443009e000 2023-09-30T22:32:38.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.446+0000 7f4437c3b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f443009a7e0 con 0x7f443009d6f0 2023-09-30T22:32:38.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.446+0000 7f44361d8700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f443009e410 0x7f4430133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53602/0 (socket says 172.21.15.16:53602) 2023-09-30T22:32:38.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.446+0000 7f44361d8700 1 -- 172.21.15.16:0/1086783525 learned_addr learned my addr 172.21.15.16:0/1086783525 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:38.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.446+0000 7f44351d6700 1 --2- 172.21.15.16:0/1086783525 >> v2:172.21.15.16:3300/0 conn(0x7f443009e000 0x7f44301339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:38.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.446+0000 7f44359d7700 1 --2- 172.21.15.16:0/1086783525 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f443009d6f0 0x7f443009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:38.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.447+0000 7f44351d6700 1 -- 172.21.15.16:0/1086783525 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f443009d6f0 msgr2=0x7f443009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:38.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.447+0000 7f44351d6700 1 --2- 172.21.15.16:0/1086783525 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f443009d6f0 0x7f443009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:38.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.447+0000 7f44351d6700 1 -- 172.21.15.16:0/1086783525 >> v1:172.21.15.16:6789/0 conn(0x7f443009e410 legacy=0x7f4430133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:38.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.447+0000 7f44349d5700 1 -- 172.21.15.16:0/1086783525 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1814710948 0 0) 0x7f443009b120 con 0x7f443009e410 2023-09-30T22:32:38.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.447+0000 7f44351d6700 1 -- 172.21.15.16:0/1086783525 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f443009a500 con 0x7f443009e000 2023-09-30T22:32:38.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.447+0000 7f44351d6700 1 --2- 172.21.15.16:0/1086783525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f443009e000 0x7f44301339f0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f4420009840 tx=0x7f4420009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fdfdc61247f81c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:38.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.447+0000 7f44349d5700 1 -- 172.21.15.16:0/1086783525 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4420011830 con 0x7f443009e000 2023-09-30T22:32:38.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.447+0000 7f44349d5700 1 -- 172.21.15.16:0/1086783525 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4420011990 con 0x7f443009e000 2023-09-30T22:32:38.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.447+0000 7f44349d5700 1 -- 172.21.15.16:0/1086783525 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4420011d20 con 0x7f443009e000 2023-09-30T22:32:38.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.448+0000 7f4437c3b700 1 -- 172.21.15.16:0/1086783525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f443009e000 msgr2=0x7f44301339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:38.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.448+0000 7f4437c3b700 1 --2- 172.21.15.16:0/1086783525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f443009e000 0x7f44301339f0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f4420009840 tx=0x7f4420009b50 comp rx=0 tx=0).stop 2023-09-30T22:32:38.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.449+0000 7f4437c3b700 1 -- 172.21.15.16:0/1086783525 shutdown_connections 2023-09-30T22:32:38.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.449+0000 7f4437c3b700 1 --2- 172.21.15.16:0/1086783525 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f443009d6f0 0x7f443009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:38.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.449+0000 7f4437c3b700 1 --2- 172.21.15.16:0/1086783525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f443009e000 0x7f44301339f0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:38.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.449+0000 7f4437c3b700 1 -- 172.21.15.16:0/1086783525 >> 172.21.15.16:0/1086783525 conn(0x7f4430066f50 msgr2=0x7f443009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:38.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.449+0000 7f4437c3b700 1 -- 172.21.15.16:0/1086783525 shutdown_connections 2023-09-30T22:32:38.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.449+0000 7f4437c3b700 1 -- 172.21.15.16:0/1086783525 wait complete. 2023-09-30T22:32:38.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.450+0000 7f4437c3b700 1 Processor -- start 2023-09-30T22:32:38.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.450+0000 7f4437c3b700 1 -- start start 2023-09-30T22:32:38.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.450+0000 7f4437c3b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f443009d6f0 0x7f44301a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:38.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.450+0000 7f4437c3b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f443009e000 0x7f44301a56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:38.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.450+0000 7f4437c3b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f443009e410 0x7f44301c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:38.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.450+0000 7f4437c3b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44301a8100 con 0x7f443009e410 2023-09-30T22:32:38.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.450+0000 7f4437c3b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44301a6a00 con 0x7f443009e000 2023-09-30T22:32:38.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.450+0000 7f4437c3b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f44301a82e0 con 0x7f443009d6f0 2023-09-30T22:32:38.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.450+0000 7f44351d6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f443009e000 0x7f44301a56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:38.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.450+0000 7f44351d6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f443009e000 0x7f44301a56f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:39248/0 (socket says 172.21.15.16:39248) 2023-09-30T22:32:38.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.450+0000 7f44351d6700 1 -- 172.21.15.16:0/899515479 learned_addr learned my addr 172.21.15.16:0/899515479 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:38.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.451+0000 7f44361d8700 1 --2- 172.21.15.16:0/899515479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f443009e410 0x7f44301c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:38.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.451+0000 7f44351d6700 1 -- 172.21.15.16:0/899515479 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f443009d6f0 msgr2=0x7f44301a51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:38.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.451+0000 7f44351d6700 1 --2- 172.21.15.16:0/899515479 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f443009d6f0 0x7f44301a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:38.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.451+0000 7f44351d6700 1 -- 172.21.15.16:0/899515479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f443009e410 msgr2=0x7f44301c8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:38.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.451+0000 7f44351d6700 1 --2- 172.21.15.16:0/899515479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f443009e410 0x7f44301c8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:38.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.451+0000 7f44351d6700 1 -- 172.21.15.16:0/899515479 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44200094a0 con 0x7f443009e000 2023-09-30T22:32:38.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.451+0000 7f44361d8700 1 --2- 172.21.15.16:0/899515479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f443009e410 0x7f44301c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:32:38.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.451+0000 7f44351d6700 1 --2- 172.21.15.16:0/899515479 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f443009e000 0x7f44301a56f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f4420009fd0 tx=0x7f4420004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:38.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.451+0000 7f44267fc700 1 -- 172.21.15.16:0/899515479 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44200036a0 con 0x7f443009e000 2023-09-30T22:32:38.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.451+0000 7f44267fc700 1 -- 172.21.15.16:0/899515479 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f442001f070 con 0x7f443009e000 2023-09-30T22:32:38.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.451+0000 7f44267fc700 1 -- 172.21.15.16:0/899515479 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4420003a40 con 0x7f443009e000 2023-09-30T22:32:38.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.451+0000 7f4437c3b700 1 -- 172.21.15.16:0/899515479 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44301a4d10 con 0x7f443009e000 2023-09-30T22:32:38.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.451+0000 7f4437c3b700 1 -- 172.21.15.16:0/899515479 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44301a6bc0 con 0x7f443009e000 2023-09-30T22:32:38.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.452+0000 7f44267fc700 1 -- 172.21.15.16:0/899515479 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4420003ba0 con 0x7f443009e000 2023-09-30T22:32:38.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.453+0000 7f4437c3b700 1 -- 172.21.15.16:0/899515479 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4430074ea0 con 0x7f443009e000 2023-09-30T22:32:38.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.453+0000 7f44267fc700 1 -- 172.21.15.16:0/899515479 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f4420021070 con 0x7f443009e000 2023-09-30T22:32:38.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.458+0000 7f44267fc700 1 -- 172.21.15.16:0/899515479 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f442002b020 con 0x7f443009e000 2023-09-30T22:32:38.481 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:32:38.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.566+0000 7ff205b32700 1 Processor -- start 2023-09-30T22:32:38.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.566+0000 7ff205b32700 1 -- start start 2023-09-30T22:32:38.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.566+0000 7ff205b32700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff20009d6f0 0x7ff20009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:38.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.566+0000 7ff205b32700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff20009e000 0x7ff200130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:38.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.566+0000 7ff205b32700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff20009e410 0x7ff200130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:38.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.566+0000 7ff205b32700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff20009f520 con 0x7ff20009e410 2023-09-30T22:32:38.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.566+0000 7ff205b32700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff20009f130 con 0x7ff20009d6f0 2023-09-30T22:32:38.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.566+0000 7ff205b32700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff20009e930 con 0x7ff20009e000 2023-09-30T22:32:38.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.567+0000 7ff1ff7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff20009d6f0 0x7ff20009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:38.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.567+0000 7ff1ff7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff20009d6f0 0x7ff20009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:39262/0 (socket says 172.21.15.16:39262) 2023-09-30T22:32:38.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.567+0000 7ff1ff7fe700 1 -- 172.21.15.16:0/3571241391 learned_addr learned my addr 172.21.15.16:0/3571241391 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:38.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.567+0000 7ff1feffd700 1 --2- 172.21.15.16:0/3571241391 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff20009e000 0x7ff200130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:38.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.567+0000 7ff1fffff700 1 --2- 172.21.15.16:0/3571241391 >> v2:172.21.15.16:3300/0 conn(0x7ff20009e410 0x7ff200130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:38.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.567+0000 7ff1ff7fe700 1 -- 172.21.15.16:0/3571241391 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff20009e000 msgr2=0x7ff200130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:38.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.567+0000 7ff1ff7fe700 1 --2- 172.21.15.16:0/3571241391 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff20009e000 0x7ff200130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:38.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.567+0000 7ff1ff7fe700 1 -- 172.21.15.16:0/3571241391 >> v2:172.21.15.16:3300/0 conn(0x7ff20009e410 msgr2=0x7ff200130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:38.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.567+0000 7ff1ff7fe700 1 --2- 172.21.15.16:0/3571241391 >> v2:172.21.15.16:3300/0 conn(0x7ff20009e410 0x7ff200130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:38.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.567+0000 7ff1ff7fe700 1 -- 172.21.15.16:0/3571241391 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff20009b2d0 con 0x7ff20009d6f0 2023-09-30T22:32:38.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.567+0000 7ff1ff7fe700 1 --2- 172.21.15.16:0/3571241391 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff20009d6f0 0x7ff20009dac0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff1f000bbc0 tx=0x7ff1f000bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2eb2e5aa8b3d55fc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:38.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.568+0000 7ff1fe7fc700 1 -- 172.21.15.16:0/3571241391 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f000dcf0 con 0x7ff20009d6f0 2023-09-30T22:32:38.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.568+0000 7ff1fe7fc700 1 -- 172.21.15.16:0/3571241391 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1f000de50 con 0x7ff20009d6f0 2023-09-30T22:32:38.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.568+0000 7ff1fe7fc700 1 -- 172.21.15.16:0/3571241391 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f0010690 con 0x7ff20009d6f0 2023-09-30T22:32:38.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.568+0000 7ff205b32700 1 -- 172.21.15.16:0/3571241391 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff20009d6f0 msgr2=0x7ff20009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:38.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.568+0000 7ff205b32700 1 --2- 172.21.15.16:0/3571241391 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff20009d6f0 0x7ff20009dac0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff1f000bbc0 tx=0x7ff1f000bed0 comp rx=0 tx=0).stop 2023-09-30T22:32:38.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.569+0000 7ff205b32700 1 -- 172.21.15.16:0/3571241391 shutdown_connections 2023-09-30T22:32:38.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.569+0000 7ff205b32700 1 --2- 172.21.15.16:0/3571241391 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff20009d6f0 0x7ff20009dac0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:38.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.569+0000 7ff205b32700 1 --2- 172.21.15.16:0/3571241391 >> v2:172.21.15.16:3300/0 conn(0x7ff20009e410 0x7ff200130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:38.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.569+0000 7ff205b32700 1 --2- 172.21.15.16:0/3571241391 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff20009e000 0x7ff200130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:38.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.569+0000 7ff205b32700 1 -- 172.21.15.16:0/3571241391 >> 172.21.15.16:0/3571241391 conn(0x7ff200066f50 msgr2=0x7ff20009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:38.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.569+0000 7ff205b32700 1 -- 172.21.15.16:0/3571241391 shutdown_connections 2023-09-30T22:32:38.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.569+0000 7ff205b32700 1 -- 172.21.15.16:0/3571241391 wait complete. 2023-09-30T22:32:38.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.570+0000 7ff205b32700 1 Processor -- start 2023-09-30T22:32:38.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.570+0000 7ff205b32700 1 -- start start 2023-09-30T22:32:38.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.571+0000 7ff205b32700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff20009d6f0 0x7ff20011e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:38.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.571+0000 7ff205b32700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff20009e000 0x7ff20011e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:38.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.571+0000 7ff205b32700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff20009e410 0x7ff20011eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:38.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.571+0000 7ff205b32700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2001202c0 con 0x7ff20009d6f0 2023-09-30T22:32:38.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.571+0000 7ff205b32700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff200072620 con 0x7ff20009e000 2023-09-30T22:32:38.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.571+0000 7ff205b32700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff20009c700 con 0x7ff20009e410 2023-09-30T22:32:38.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.571+0000 7ff1ff7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff20009d6f0 0x7ff20011e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:38.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.571+0000 7ff1feffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff20009e000 0x7ff20011e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:38.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.571+0000 7ff1feffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff20009e000 0x7ff20011e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:39276/0 (socket says 172.21.15.16:39276) 2023-09-30T22:32:38.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.571+0000 7ff1feffd700 1 -- 172.21.15.16:0/1616893162 learned_addr learned my addr 172.21.15.16:0/1616893162 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:38.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.571+0000 7ff1ff7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff20009d6f0 0x7ff20011e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35758/0 (socket says 172.21.15.16:35758) 2023-09-30T22:32:38.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.571+0000 7ff1fffff700 1 --2- 172.21.15.16:0/1616893162 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff20009e410 0x7ff20011eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:38.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.572+0000 7ff1feffd700 1 -- 172.21.15.16:0/1616893162 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff20009e410 msgr2=0x7ff20011eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:38.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.572+0000 7ff1feffd700 1 --2- 172.21.15.16:0/1616893162 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff20009e410 0x7ff20011eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:38.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.572+0000 7ff1feffd700 1 -- 172.21.15.16:0/1616893162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff20009d6f0 msgr2=0x7ff20011e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:38.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.572+0000 7ff1feffd700 1 --2- 172.21.15.16:0/1616893162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff20009d6f0 0x7ff20011e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:38.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.572+0000 7ff1feffd700 1 -- 172.21.15.16:0/1616893162 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1f000b7f0 con 0x7ff20009e000 2023-09-30T22:32:38.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.572+0000 7ff1feffd700 1 --2- 172.21.15.16:0/1616893162 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff20009e000 0x7ff20011e640 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff1f400e900 tx=0x7ff1f400ec10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:38.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.572+0000 7ff1e7fff700 1 -- 172.21.15.16:0/1616893162 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f400ee50 con 0x7ff20009e000 2023-09-30T22:32:38.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.572+0000 7ff1e7fff700 1 -- 172.21.15.16:0/1616893162 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1f400c850 con 0x7ff20009e000 2023-09-30T22:32:38.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.572+0000 7ff205b32700 1 -- 172.21.15.16:0/1616893162 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2001213b0 con 0x7ff20009e000 2023-09-30T22:32:38.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.572+0000 7ff1e7fff700 1 -- 172.21.15.16:0/1616893162 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f400cbe0 con 0x7ff20009e000 2023-09-30T22:32:38.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.572+0000 7ff205b32700 1 -- 172.21.15.16:0/1616893162 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2001223b0 con 0x7ff20009e000 2023-09-30T22:32:38.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.574+0000 7ff205b32700 1 -- 172.21.15.16:0/1616893162 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2001215a0 con 0x7ff20009e000 2023-09-30T22:32:38.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.574+0000 7fdaaa68c700 1 Processor -- start 2023-09-30T22:32:38.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.574+0000 7ff1e7fff700 1 -- 172.21.15.16:0/1616893162 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff1f400cd40 con 0x7ff20009e000 2023-09-30T22:32:38.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.574+0000 7ff1e7fff700 1 -- 172.21.15.16:0/1616893162 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7ff1f403f3b0 con 0x7ff20009e000 2023-09-30T22:32:38.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.574+0000 7fdaaa68c700 1 -- start start 2023-09-30T22:32:38.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.574+0000 7fdaaa68c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdaa406e570 0x7fdaa406e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:38.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.574+0000 7fdaaa68c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdaa409eca0 0x7fdaa406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:38.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.574+0000 7fdaaa68c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdaa4134610 0x7fdaa4134ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:38.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.574+0000 7fdaaa68c700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdaa4070ee0 con 0x7fdaa4134610 2023-09-30T22:32:38.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.574+0000 7fdaaa68c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa4071480 con 0x7fdaa409eca0 2023-09-30T22:32:38.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.574+0000 7fdaaa68c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa406f3c0 con 0x7fdaa406e570 2023-09-30T22:32:38.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.575+0000 7fdaa37fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdaa409eca0 0x7fdaa406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:38.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.575+0000 7fdaa37fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdaa409eca0 0x7fdaa406ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:39292/0 (socket says 172.21.15.16:39292) 2023-09-30T22:32:38.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.575+0000 7fdaa37fe700 1 -- 172.21.15.16:0/1006122159 learned_addr learned my addr 172.21.15.16:0/1006122159 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:38.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.575+0000 7fdaa8c29700 1 --2- 172.21.15.16:0/1006122159 >> v2:172.21.15.16:3300/0 conn(0x7fdaa4134610 0x7fdaa4134ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:38.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.575+0000 7fdaa37fe700 1 -- 172.21.15.16:0/1006122159 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdaa406e570 msgr2=0x7fdaa406e940 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T22:32:38.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.575+0000 7fdaa37fe700 1 --2- 172.21.15.16:0/1006122159 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdaa406e570 0x7fdaa406e940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:38.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.575+0000 7fdaa37fe700 1 -- 172.21.15.16:0/1006122159 >> v2:172.21.15.16:3300/0 conn(0x7fdaa4134610 msgr2=0x7fdaa4134ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:38.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.575+0000 7fdaa37fe700 1 --2- 172.21.15.16:0/1006122159 >> v2:172.21.15.16:3300/0 conn(0x7fdaa4134610 0x7fdaa4134ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:38.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.575+0000 7fdaa37fe700 1 -- 172.21.15.16:0/1006122159 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa4071b90 con 0x7fdaa409eca0 2023-09-30T22:32:38.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.575+0000 7fdaa8c29700 1 --2- 172.21.15.16:0/1006122159 >> v2:172.21.15.16:3300/0 conn(0x7fdaa4134610 0x7fdaa4134ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:38.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.575+0000 7fdaa37fe700 1 --2- 172.21.15.16:0/1006122159 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdaa409eca0 0x7fdaa406ee80 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fda980098f0 tx=0x7fda98009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96ed9217b270e94c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:38.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.580+0000 7fdaa2ffd700 1 -- 172.21.15.16:0/1006122159 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda98011830 con 0x7fdaa409eca0 2023-09-30T22:32:38.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.580+0000 7fdaa2ffd700 1 -- 172.21.15.16:0/1006122159 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda98011990 con 0x7fdaa409eca0 2023-09-30T22:32:38.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.580+0000 7fdaa2ffd700 1 -- 172.21.15.16:0/1006122159 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda98011d20 con 0x7fdaa409eca0 2023-09-30T22:32:38.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.580+0000 7ff1e7fff700 1 -- 172.21.15.16:0/1616893162 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff1f401be10 con 0x7ff20009e000 2023-09-30T22:32:38.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.580+0000 7fdaaa68c700 1 -- 172.21.15.16:0/1006122159 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdaa409eca0 msgr2=0x7fdaa406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:38.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.580+0000 7fdaaa68c700 1 --2- 172.21.15.16:0/1006122159 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdaa409eca0 0x7fdaa406ee80 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fda980098f0 tx=0x7fda98009c00 comp rx=0 tx=0).stop 2023-09-30T22:32:38.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.580+0000 7fdaaa68c700 1 -- 172.21.15.16:0/1006122159 shutdown_connections 2023-09-30T22:32:38.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.580+0000 7fdaaa68c700 1 --2- 172.21.15.16:0/1006122159 >> v2:172.21.15.16:3300/0 conn(0x7fdaa4134610 0x7fdaa4134ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:38.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.580+0000 7fdaaa68c700 1 --2- 172.21.15.16:0/1006122159 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdaa406e570 0x7fdaa406e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:38.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.580+0000 7fdaaa68c700 1 --2- 172.21.15.16:0/1006122159 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdaa409eca0 0x7fdaa406ee80 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:38.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.580+0000 7fdaaa68c700 1 -- 172.21.15.16:0/1006122159 >> 172.21.15.16:0/1006122159 conn(0x7fdaa40a2aa0 msgr2=0x7fdaa40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:38.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.581+0000 7fdaaa68c700 1 -- 172.21.15.16:0/1006122159 shutdown_connections 2023-09-30T22:32:38.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.581+0000 7fdaaa68c700 1 -- 172.21.15.16:0/1006122159 wait complete. 2023-09-30T22:32:38.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.582+0000 7fdaaa68c700 1 Processor -- start 2023-09-30T22:32:38.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.582+0000 7fdaaa68c700 1 -- start start 2023-09-30T22:32:38.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.582+0000 7fdaaa68c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdaa406e570 0x7fdaa413b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:38.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.582+0000 7fdaaa68c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdaa4134610 0x7fdaa4132870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:38.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.582+0000 7fdaaa68c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdaa4133cb0 0x7fdaa4132db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:38.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.582+0000 7fdaaa68c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa413cac0 con 0x7fdaa4133cb0 2023-09-30T22:32:38.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.582+0000 7fdaaa68c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa413c260 con 0x7fdaa406e570 2023-09-30T22:32:38.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.582+0000 7fdaaa68c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa40a1dc0 con 0x7fdaa4134610 2023-09-30T22:32:38.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.583+0000 7fdaa3fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdaa406e570 0x7fdaa413b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:38.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.583+0000 7fdaa3fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdaa406e570 0x7fdaa413b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:39296/0 (socket says 172.21.15.16:39296) 2023-09-30T22:32:38.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.583+0000 7fdaa3fff700 1 -- 172.21.15.16:0/1304696831 learned_addr learned my addr 172.21.15.16:0/1304696831 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:38.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.583+0000 7fdaa37fe700 1 --2- 172.21.15.16:0/1304696831 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdaa4134610 0x7fdaa4132870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:38.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.583+0000 7fdaa37fe700 1 -- 172.21.15.16:0/1304696831 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdaa406e570 msgr2=0x7fdaa413b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:38.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.583+0000 7fdaa37fe700 1 --2- 172.21.15.16:0/1304696831 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdaa406e570 0x7fdaa413b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:38.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.583+0000 7fdaa37fe700 1 -- 172.21.15.16:0/1304696831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdaa4133cb0 msgr2=0x7fdaa4132db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:38.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.583+0000 7fdaa37fe700 1 --2- 172.21.15.16:0/1304696831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdaa4133cb0 0x7fdaa4132db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:38.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.583+0000 7fdaa37fe700 1 -- 172.21.15.16:0/1304696831 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda9c009a70 con 0x7fdaa4134610 2023-09-30T22:32:38.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.583+0000 7fdaa37fe700 1 --2- 172.21.15.16:0/1304696831 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdaa4134610 0x7fdaa4132870 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fda98016000 tx=0x7fda98004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:38.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.583+0000 7fdaa0ff9700 1 -- 172.21.15.16:0/1304696831 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda98004380 con 0x7fdaa4134610 2023-09-30T22:32:38.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.584+0000 7fdaa0ff9700 1 -- 172.21.15.16:0/1304696831 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda9801f070 con 0x7fdaa4134610 2023-09-30T22:32:38.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.584+0000 7fdaa0ff9700 1 -- 172.21.15.16:0/1304696831 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda980036a0 con 0x7fdaa4134610 2023-09-30T22:32:38.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.584+0000 7fdaaa68c700 1 -- 172.21.15.16:0/1304696831 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda98009550 con 0x7fdaa4134610 2023-09-30T22:32:38.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.584+0000 7fdaaa68c700 1 -- 172.21.15.16:0/1304696831 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaa413d440 con 0x7fdaa4134610 2023-09-30T22:32:38.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.585+0000 7fdaa0ff9700 1 -- 172.21.15.16:0/1304696831 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fda98021030 con 0x7fdaa4134610 2023-09-30T22:32:38.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.585+0000 7fdaa0ff9700 1 -- 172.21.15.16:0/1304696831 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fda98040a00 con 0x7fdaa4134610 2023-09-30T22:32:38.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.585+0000 7fdaaa68c700 1 -- 172.21.15.16:0/1304696831 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaa4074ea0 con 0x7fdaa4134610 2023-09-30T22:32:38.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.590+0000 7fdaa0ff9700 1 -- 172.21.15.16:0/1304696831 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fda980239c0 con 0x7fdaa4134610 2023-09-30T22:32:38.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.631+0000 7f2f6c6c9700 1 Processor -- start 2023-09-30T22:32:38.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.632+0000 7f2f6c6c9700 1 -- start start 2023-09-30T22:32:38.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.632+0000 7f2f6c6c9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f6409d6f0 0x7f2f6409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:38.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.632+0000 7f2f6c6c9700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2f6409e000 0x7f2f64130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:38.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.632+0000 7f2f6c6c9700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f6409f130 con 0x7f2f6409e410 2023-09-30T22:32:38.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.632+0000 7f2f6c6c9700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f6409f520 con 0x7f2f6409e000 2023-09-30T22:32:38.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.632+0000 7f2f6c6c9700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6409e930 con 0x7f2f6409d6f0 2023-09-30T22:32:38.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.633+0000 7f2f69c64700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2f6409e000 0x7f2f64130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:38.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.633+0000 7f2f69c64700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2f6409e000 0x7f2f64130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35778/0 (socket says 172.21.15.16:35778) 2023-09-30T22:32:38.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.633+0000 7f2f69c64700 1 -- 172.21.15.16:0/2876004451 learned_addr learned my addr 172.21.15.16:0/2876004451 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:38.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.633+0000 7f2f6ac66700 1 --1- 172.21.15.16:0/2876004451 >> v1:172.21.15.16:6789/0 conn(0x7f2f6409e410 0x7f2f64130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53612/0 (socket says 172.21.15.16:53612) 2023-09-30T22:32:38.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.633+0000 7f2f6a465700 1 --2- 172.21.15.16:0/2876004451 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f6409d6f0 0x7f2f6409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:38.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.633+0000 7f2f69c64700 1 -- 172.21.15.16:0/2876004451 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f6409d6f0 msgr2=0x7f2f6409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:38.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.633+0000 7f2f69c64700 1 --2- 172.21.15.16:0/2876004451 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f6409d6f0 0x7f2f6409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:38.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.633+0000 7f2f69c64700 1 -- 172.21.15.16:0/2876004451 >> v1:172.21.15.16:6789/0 conn(0x7f2f6409e410 legacy=0x7f2f64130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:38.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.633+0000 7f2f69c64700 1 -- 172.21.15.16:0/2876004451 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f6409b2d0 con 0x7f2f6409e000 2023-09-30T22:32:38.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.633+0000 7f2f69c64700 1 --2- 172.21.15.16:0/2876004451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f6409e000 0x7f2f64130660 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f2f5400bb90 tx=0x7f2f5400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6e12c9216e96793 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:38.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.633+0000 7f2f69463700 1 -- 172.21.15.16:0/2876004451 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f5400dd30 con 0x7f2f6409e000 2023-09-30T22:32:38.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.634+0000 7f2f69463700 1 -- 172.21.15.16:0/2876004451 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f5400de90 con 0x7f2f6409e000 2023-09-30T22:32:38.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.634+0000 7f2f69463700 1 -- 172.21.15.16:0/2876004451 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f54010690 con 0x7f2f6409e000 2023-09-30T22:32:38.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.634+0000 7f2f6c6c9700 1 -- 172.21.15.16:0/2876004451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f6409e000 msgr2=0x7f2f64130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:38.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.634+0000 7f2f6c6c9700 1 --2- 172.21.15.16:0/2876004451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f6409e000 0x7f2f64130660 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f2f5400bb90 tx=0x7f2f5400bea0 comp rx=0 tx=0).stop 2023-09-30T22:32:38.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.635+0000 7f2f6c6c9700 1 -- 172.21.15.16:0/2876004451 shutdown_connections 2023-09-30T22:32:38.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.635+0000 7f2f6c6c9700 1 --2- 172.21.15.16:0/2876004451 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f6409d6f0 0x7f2f6409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:38.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.635+0000 7f2f6c6c9700 1 --2- 172.21.15.16:0/2876004451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f6409e000 0x7f2f64130660 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:38.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.635+0000 7f2f6c6c9700 1 -- 172.21.15.16:0/2876004451 >> 172.21.15.16:0/2876004451 conn(0x7f2f64066f50 msgr2=0x7f2f6409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:38.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.635+0000 7f2f6c6c9700 1 -- 172.21.15.16:0/2876004451 shutdown_connections 2023-09-30T22:32:38.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.635+0000 7f2f6c6c9700 1 -- 172.21.15.16:0/2876004451 wait complete. 2023-09-30T22:32:38.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.636+0000 7f2f6c6c9700 1 Processor -- start 2023-09-30T22:32:38.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.636+0000 7f2f6c6c9700 1 -- start start 2023-09-30T22:32:38.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.636+0000 7f2f6c6c9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f6409d6f0 0x7f2f641223f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:38.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.636+0000 7f2f6c6c9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f6409e000 0x7f2f64122930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:38.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.636+0000 7f2f6c6c9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f6409e410 0x7f2f64122e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:38.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.636+0000 7f2f6c6c9700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f64124420 con 0x7f2f6409d6f0 2023-09-30T22:32:38.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.636+0000 7f2f6c6c9700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f64125f10 con 0x7f2f6409e000 2023-09-30T22:32:38.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.636+0000 7f2f6c6c9700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2f640a1d00 con 0x7f2f6409e410 2023-09-30T22:32:38.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.636+0000 7f2f6ac66700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f6409e410 0x7f2f64122e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:38.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.637+0000 7f2f6ac66700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f6409e410 0x7f2f64122e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:38974/0 (socket says 172.21.15.16:38974) 2023-09-30T22:32:38.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.637+0000 7f2f6ac66700 1 -- 172.21.15.16:0/1166013611 learned_addr learned my addr 172.21.15.16:0/1166013611 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:38.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.637+0000 7f2f6a465700 1 --2- 172.21.15.16:0/1166013611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f6409d6f0 0x7f2f641223f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:38.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.637+0000 7f2f69c64700 1 --2- 172.21.15.16:0/1166013611 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f6409e000 0x7f2f64122930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:38.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.637+0000 7f2f6ac66700 1 -- 172.21.15.16:0/1166013611 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f6409e000 msgr2=0x7f2f64122930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:38.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.637+0000 7f2f6ac66700 1 --2- 172.21.15.16:0/1166013611 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f6409e000 0x7f2f64122930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:38.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.637+0000 7f2f6ac66700 1 -- 172.21.15.16:0/1166013611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f6409d6f0 msgr2=0x7f2f641223f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:38.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.637+0000 7f2f6ac66700 1 --2- 172.21.15.16:0/1166013611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f6409d6f0 0x7f2f641223f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:38.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.637+0000 7f2f6ac66700 1 -- 172.21.15.16:0/1166013611 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f5400b7f0 con 0x7f2f6409e410 2023-09-30T22:32:38.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.637+0000 7f2f6ac66700 1 --2- 172.21.15.16:0/1166013611 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f6409e410 0x7f2f64122e70 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c00cba0 tx=0x7f2f5c00ceb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:38.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.637+0000 7f2f5affd700 1 -- 172.21.15.16:0/1166013611 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c00aaa0 con 0x7f2f6409e410 2023-09-30T22:32:38.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.637+0000 7f2f5affd700 1 -- 172.21.15.16:0/1166013611 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f5c00ac00 con 0x7f2f6409e410 2023-09-30T22:32:38.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.637+0000 7f2f6c6c9700 1 -- 172.21.15.16:0/1166013611 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f64124000 con 0x7f2f6409e410 2023-09-30T22:32:38.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.637+0000 7f2f5affd700 1 -- 172.21.15.16:0/1166013611 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c007070 con 0x7f2f6409e410 2023-09-30T22:32:38.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.638+0000 7f2f6c6c9700 1 -- 172.21.15.16:0/1166013611 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f64126610 con 0x7f2f6409e410 2023-09-30T22:32:38.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.639+0000 7f2f5affd700 1 -- 172.21.15.16:0/1166013611 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2f5c0071d0 con 0x7f2f6409e410 2023-09-30T22:32:38.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.639+0000 7f2f6c6c9700 1 -- 172.21.15.16:0/1166013611 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f641258d0 con 0x7f2f6409e410 2023-09-30T22:32:38.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.639+0000 7f2f5affd700 1 -- 172.21.15.16:0/1166013611 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f2f5c03f400 con 0x7f2f6409e410 2023-09-30T22:32:38.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.648+0000 7f2f5affd700 1 -- 172.21.15.16:0/1166013611 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2f641258d0 con 0x7f2f6409e410 2023-09-30T22:32:38.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.710+0000 7f4437c3b700 1 -- 172.21.15.16:0/899515479 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f443006c360 con 0x7f443009e000 2023-09-30T22:32:38.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.710+0000 7f44267fc700 1 -- 172.21.15.16:0/899515479 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4420034070 con 0x7f443009e000 2023-09-30T22:32:38.711 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:32:38.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.713+0000 7f4437c3b700 1 -- 172.21.15.16:0/899515479 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f443009e000 msgr2=0x7f44301a56f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:38.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.713+0000 7f4437c3b700 1 --2- 172.21.15.16:0/899515479 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f443009e000 0x7f44301a56f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f4420009fd0 tx=0x7f4420004750 comp rx=0 tx=0).stop 2023-09-30T22:32:38.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.713+0000 7f4437c3b700 1 -- 172.21.15.16:0/899515479 shutdown_connections 2023-09-30T22:32:38.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.713+0000 7f4437c3b700 1 --2- 172.21.15.16:0/899515479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f443009e410 0x7f44301c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:38.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.713+0000 7f4437c3b700 1 --2- 172.21.15.16:0/899515479 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f443009d6f0 0x7f44301a51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:38.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.713+0000 7f4437c3b700 1 --2- 172.21.15.16:0/899515479 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f443009e000 0x7f44301a56f0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:38.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.713+0000 7f4437c3b700 1 -- 172.21.15.16:0/899515479 >> 172.21.15.16:0/899515479 conn(0x7f4430066f50 msgr2=0x7f443006f040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:38.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.713+0000 7f4437c3b700 1 -- 172.21.15.16:0/899515479 shutdown_connections 2023-09-30T22:32:38.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.713+0000 7f4437c3b700 1 -- 172.21.15.16:0/899515479 wait complete. 2023-09-30T22:32:38.728 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:32:38.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.794+0000 7ff205b32700 1 -- 172.21.15.16:0/1616893162 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff200074ea0 con 0x7ff20009e000 2023-09-30T22:32:38.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.795+0000 7ff1e7fff700 1 -- 172.21.15.16:0/1616893162 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff1f4017070 con 0x7ff20009e000 2023-09-30T22:32:38.795 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:32:38.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.797+0000 7ff205b32700 1 -- 172.21.15.16:0/1616893162 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff20009e000 msgr2=0x7ff20011e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:38.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.797+0000 7ff205b32700 1 --2- 172.21.15.16:0/1616893162 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff20009e000 0x7ff20011e640 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ff1f400e900 tx=0x7ff1f400ec10 comp rx=0 tx=0).stop 2023-09-30T22:32:38.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.797+0000 7ff205b32700 1 -- 172.21.15.16:0/1616893162 shutdown_connections 2023-09-30T22:32:38.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.797+0000 7ff205b32700 1 --2- 172.21.15.16:0/1616893162 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff20009e410 0x7ff20011eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:38.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.797+0000 7ff205b32700 1 --2- 172.21.15.16:0/1616893162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff20009d6f0 0x7ff20011e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:38.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.797+0000 7ff205b32700 1 --2- 172.21.15.16:0/1616893162 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff20009e000 0x7ff20011e640 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:38.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.797+0000 7ff205b32700 1 -- 172.21.15.16:0/1616893162 >> 172.21.15.16:0/1616893162 conn(0x7ff200066f50 msgr2=0x7ff2000a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:38.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.797+0000 7ff205b32700 1 -- 172.21.15.16:0/1616893162 shutdown_connections 2023-09-30T22:32:38.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.797+0000 7ff205b32700 1 -- 172.21.15.16:0/1616893162 wait complete. 2023-09-30T22:32:38.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.805+0000 7fdaaa68c700 1 -- 172.21.15.16:0/1304696831 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fdaa4071100 con 0x7fdaa4134610 2023-09-30T22:32:38.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.805+0000 7fdaa0ff9700 1 -- 172.21.15.16:0/1304696831 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fda9801b430 con 0x7fdaa4134610 2023-09-30T22:32:38.805 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:32:38.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.807+0000 7fdaaa68c700 1 -- 172.21.15.16:0/1304696831 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdaa4134610 msgr2=0x7fdaa4132870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:38.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.807+0000 7fdaaa68c700 1 --2- 172.21.15.16:0/1304696831 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdaa4134610 0x7fdaa4132870 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fda98016000 tx=0x7fda98004750 comp rx=0 tx=0).stop 2023-09-30T22:32:38.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.807+0000 7fdaaa68c700 1 -- 172.21.15.16:0/1304696831 shutdown_connections 2023-09-30T22:32:38.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.807+0000 7fdaaa68c700 1 --2- 172.21.15.16:0/1304696831 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdaa406e570 0x7fdaa413b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:38.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.807+0000 7fdaaa68c700 1 --2- 172.21.15.16:0/1304696831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdaa4133cb0 0x7fdaa4132db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:38.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.807+0000 7fdaaa68c700 1 --2- 172.21.15.16:0/1304696831 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdaa4134610 0x7fdaa4132870 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:38.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.807+0000 7fdaaa68c700 1 -- 172.21.15.16:0/1304696831 >> 172.21.15.16:0/1304696831 conn(0x7fdaa40a2aa0 msgr2=0x7fdaa40a0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:38.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.807+0000 7fdaaa68c700 1 -- 172.21.15.16:0/1304696831 shutdown_connections 2023-09-30T22:32:38.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.807+0000 7fdaaa68c700 1 -- 172.21.15.16:0/1304696831 wait complete. 2023-09-30T22:32:38.813 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:32:38.822 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:32:38.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.870+0000 7f2f6c6c9700 1 -- 172.21.15.16:0/1166013611 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f2f641258d0 con 0x7f2f6409e410 2023-09-30T22:32:38.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.871+0000 7f2f5affd700 1 -- 172.21.15.16:0/1166013611 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2f641258d0 con 0x7f2f6409e410 2023-09-30T22:32:38.871 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:32:38.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.873+0000 7f2f6c6c9700 1 -- 172.21.15.16:0/1166013611 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f6409e410 msgr2=0x7f2f64122e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:38.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.873+0000 7f2f6c6c9700 1 --2- 172.21.15.16:0/1166013611 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f6409e410 0x7f2f64122e70 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c00cba0 tx=0x7f2f5c00ceb0 comp rx=0 tx=0).stop 2023-09-30T22:32:38.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.873+0000 7f2f6c6c9700 1 -- 172.21.15.16:0/1166013611 shutdown_connections 2023-09-30T22:32:38.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.873+0000 7f2f6c6c9700 1 --2- 172.21.15.16:0/1166013611 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f6409e410 0x7f2f64122e70 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:38.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.873+0000 7f2f6c6c9700 1 --2- 172.21.15.16:0/1166013611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f6409d6f0 0x7f2f641223f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:38.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.873+0000 7f2f6c6c9700 1 --2- 172.21.15.16:0/1166013611 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f6409e000 0x7f2f64122930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:38.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.873+0000 7f2f6c6c9700 1 -- 172.21.15.16:0/1166013611 >> 172.21.15.16:0/1166013611 conn(0x7f2f64066f50 msgr2=0x7f2f640a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:38.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.873+0000 7f2f6c6c9700 1 -- 172.21.15.16:0/1166013611 shutdown_connections 2023-09-30T22:32:38.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:38.873+0000 7f2f6c6c9700 1 -- 172.21.15.16:0/1166013611 wait complete. 2023-09-30T22:32:38.888 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:32:39.729 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:32:39.814 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:32:39.823 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:32:39.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.882+0000 7fb65bb4d700 1 Processor -- start 2023-09-30T22:32:39.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.882+0000 7fb65bb4d700 1 -- start start 2023-09-30T22:32:39.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.883+0000 7fb65bb4d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb65409d6f0 0x7fb65409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:39.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.883+0000 7fb65bb4d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb65409e000 0x7fb654130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:39.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.883+0000 7fb65bb4d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb65409f130 con 0x7fb65409e410 2023-09-30T22:32:39.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.883+0000 7fb65bb4d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb65409f520 con 0x7fb65409d6f0 2023-09-30T22:32:39.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.883+0000 7fb65bb4d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb65409e930 con 0x7fb65409e000 2023-09-30T22:32:39.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.883+0000 7fb6598e9700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb65409d6f0 0x7fb65409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:39.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.883+0000 7fb65a0ea700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb65409e410 0x7fb654130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53614/0 (socket says 172.21.15.16:53614) 2023-09-30T22:32:39.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.883+0000 7fb6598e9700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb65409d6f0 0x7fb65409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35798/0 (socket says 172.21.15.16:35798) 2023-09-30T22:32:39.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.883+0000 7fb65a0ea700 1 -- 172.21.15.16:0/482811765 learned_addr learned my addr 172.21.15.16:0/482811765 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:39.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.883+0000 7fb6590e8700 1 --2- 172.21.15.16:0/482811765 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb65409e000 0x7fb654130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:39.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.884+0000 7fb6598e9700 1 -- 172.21.15.16:0/482811765 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb65409e000 msgr2=0x7fb654130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:39.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.884+0000 7fb6598e9700 1 --2- 172.21.15.16:0/482811765 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb65409e000 0x7fb654130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:39.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.884+0000 7fb6598e9700 1 -- 172.21.15.16:0/482811765 >> v1:172.21.15.16:6789/0 conn(0x7fb65409e410 legacy=0x7fb654130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:39.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.884+0000 7fb6598e9700 1 -- 172.21.15.16:0/482811765 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb65409b2d0 con 0x7fb65409d6f0 2023-09-30T22:32:39.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.884+0000 7fb6588e7700 1 -- 172.21.15.16:0/482811765 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2520402639 0 0) 0x7fb65409f130 con 0x7fb65409e410 2023-09-30T22:32:39.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.884+0000 7fb6598e9700 1 --2- 172.21.15.16:0/482811765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb65409d6f0 0x7fb65409dac0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fb65000bae0 tx=0x7fb65000bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e00fce58db3e173 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:39.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.884+0000 7fb6588e7700 1 -- 172.21.15.16:0/482811765 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb65000dc70 con 0x7fb65409d6f0 2023-09-30T22:32:39.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.884+0000 7fb6588e7700 1 -- 172.21.15.16:0/482811765 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb65000ddd0 con 0x7fb65409d6f0 2023-09-30T22:32:39.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.884+0000 7fb6588e7700 1 -- 172.21.15.16:0/482811765 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb650010690 con 0x7fb65409d6f0 2023-09-30T22:32:39.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.885+0000 7fb65bb4d700 1 -- 172.21.15.16:0/482811765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb65409d6f0 msgr2=0x7fb65409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:39.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.885+0000 7fb65bb4d700 1 --2- 172.21.15.16:0/482811765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb65409d6f0 0x7fb65409dac0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fb65000bae0 tx=0x7fb65000bdf0 comp rx=0 tx=0).stop 2023-09-30T22:32:39.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.885+0000 7fb65bb4d700 1 -- 172.21.15.16:0/482811765 shutdown_connections 2023-09-30T22:32:39.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.885+0000 7fb65bb4d700 1 --2- 172.21.15.16:0/482811765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb65409d6f0 0x7fb65409dac0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:39.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.885+0000 7fb65bb4d700 1 --2- 172.21.15.16:0/482811765 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb65409e000 0x7fb654130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:39.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.885+0000 7fb65bb4d700 1 -- 172.21.15.16:0/482811765 >> 172.21.15.16:0/482811765 conn(0x7fb654066f50 msgr2=0x7fb65409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:39.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.886+0000 7fb65bb4d700 1 -- 172.21.15.16:0/482811765 shutdown_connections 2023-09-30T22:32:39.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.886+0000 7fb65bb4d700 1 -- 172.21.15.16:0/482811765 wait complete. 2023-09-30T22:32:39.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.887+0000 7fb65bb4d700 1 Processor -- start 2023-09-30T22:32:39.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.887+0000 7fb65bb4d700 1 -- start start 2023-09-30T22:32:39.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.888+0000 7fb65bb4d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb65409d6f0 0x7fb65411de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.888+0000 7fb65bb4d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb65409e000 0x7fb65411e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.888+0000 7fb65bb4d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb65409e410 0x7fb65411e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.888+0000 7fb65bb4d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65411fa80 con 0x7fb65409d6f0 2023-09-30T22:32:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.888+0000 7fb65bb4d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb654120cd0 con 0x7fb65409e000 2023-09-30T22:32:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.888+0000 7fb65bb4d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb654120070 con 0x7fb65409e410 2023-09-30T22:32:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.888+0000 7fb6598e9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb65409d6f0 0x7fb65411de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.888+0000 7fb6598e9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb65409d6f0 0x7fb65411de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35804/0 (socket says 172.21.15.16:35804) 2023-09-30T22:32:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.888+0000 7fb6598e9700 1 -- 172.21.15.16:0/1010788217 learned_addr learned my addr 172.21.15.16:0/1010788217 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.888+0000 7fb6590e8700 1 --2- 172.21.15.16:0/1010788217 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb65409e000 0x7fb65411e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:39.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.888+0000 7fb65a0ea700 1 --2- 172.21.15.16:0/1010788217 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb65409e410 0x7fb65411e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:39.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.888+0000 7fb6590e8700 1 -- 172.21.15.16:0/1010788217 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb65409e410 msgr2=0x7fb65411e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:39.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.888+0000 7fb6590e8700 1 --2- 172.21.15.16:0/1010788217 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb65409e410 0x7fb65411e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:39.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.888+0000 7fb6590e8700 1 -- 172.21.15.16:0/1010788217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb65409d6f0 msgr2=0x7fb65411de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:39.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.888+0000 7fb6590e8700 1 --2- 172.21.15.16:0/1010788217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb65409d6f0 0x7fb65411de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:39.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.888+0000 7fb6590e8700 1 -- 172.21.15.16:0/1010788217 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb65000b740 con 0x7fb65409e000 2023-09-30T22:32:39.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.888+0000 7fb6598e9700 1 --2- 172.21.15.16:0/1010788217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb65409d6f0 0x7fb65411de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:39.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.889+0000 7fb6590e8700 1 --2- 172.21.15.16:0/1010788217 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb65409e000 0x7fb65411e3b0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb64400e900 tx=0x7fb64400ec10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:39.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.889+0000 7fb64a7fc700 1 -- 172.21.15.16:0/1010788217 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb644013680 con 0x7fb65409e000 2023-09-30T22:32:39.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.889+0000 7fb64a7fc700 1 -- 172.21.15.16:0/1010788217 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6440137e0 con 0x7fb65409e000 2023-09-30T22:32:39.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.889+0000 7fb64a7fc700 1 -- 172.21.15.16:0/1010788217 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb644013b50 con 0x7fb65409e000 2023-09-30T22:32:39.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.889+0000 7fb65bb4d700 1 -- 172.21.15.16:0/1010788217 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb654121120 con 0x7fb65409e000 2023-09-30T22:32:39.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.889+0000 7fb65bb4d700 1 -- 172.21.15.16:0/1010788217 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6540a2130 con 0x7fb65409e000 2023-09-30T22:32:39.892 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:32:39.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.890+0000 7fb65bb4d700 1 -- 172.21.15.16:0/1010788217 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb65411ac20 con 0x7fb65409e000 2023-09-30T22:32:39.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.891+0000 7fb64a7fc700 1 -- 172.21.15.16:0/1010788217 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb644013cb0 con 0x7fb65409e000 2023-09-30T22:32:39.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.891+0000 7fb64a7fc700 1 -- 172.21.15.16:0/1010788217 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fb64403ffd0 con 0x7fb65409e000 2023-09-30T22:32:39.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.896+0000 7fb64a7fc700 1 -- 172.21.15.16:0/1010788217 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb644011350 con 0x7fb65409e000 2023-09-30T22:32:39.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.962+0000 7f8e1c2d3700 1 Processor -- start 2023-09-30T22:32:39.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.962+0000 7f8e1c2d3700 1 -- start start 2023-09-30T22:32:39.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.963+0000 7f8e1c2d3700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8e14131810 0x7f8e14131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:39.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.963+0000 7f8e1c2d3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8e14099a40 0x7f8e14099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:39.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.963+0000 7f8e1c2d3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8e1409a3b0 0x7f8e14138650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:39.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.963+0000 7f8e1c2d3700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e141313c0 con 0x7f8e14131810 2023-09-30T22:32:39.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.963+0000 7f8e1c2d3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1406e8c0 con 0x7f8e14099a40 2023-09-30T22:32:39.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.963+0000 7f8e1c2d3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8e14071d20 con 0x7f8e1409a3b0 2023-09-30T22:32:39.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.963+0000 7f8e1986e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8e14099a40 0x7f8e14099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:39.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.963+0000 7f8e1986e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8e14099a40 0x7f8e14099e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:39336/0 (socket says 172.21.15.16:39336) 2023-09-30T22:32:39.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.963+0000 7f8e1986e700 1 -- 172.21.15.16:0/1673727031 learned_addr learned my addr 172.21.15.16:0/1673727031 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:39.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.963+0000 7f8e1a870700 1 --2- 172.21.15.16:0/1673727031 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8e1409a3b0 0x7f8e14138650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:39.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.963+0000 7f8e1a06f700 1 --2- 172.21.15.16:0/1673727031 >> v2:172.21.15.16:3300/0 conn(0x7f8e14131810 0x7f8e14131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:39.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.964+0000 7f8e1986e700 1 -- 172.21.15.16:0/1673727031 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8e1409a3b0 msgr2=0x7f8e14138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:39.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.964+0000 7f8e1986e700 1 --2- 172.21.15.16:0/1673727031 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8e1409a3b0 0x7f8e14138650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:39.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.964+0000 7f8e1986e700 1 -- 172.21.15.16:0/1673727031 >> v2:172.21.15.16:3300/0 conn(0x7f8e14131810 msgr2=0x7f8e14131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:39.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.964+0000 7f8e1986e700 1 --2- 172.21.15.16:0/1673727031 >> v2:172.21.15.16:3300/0 conn(0x7f8e14131810 0x7f8e14131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:39.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.964+0000 7f8e1986e700 1 -- 172.21.15.16:0/1673727031 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e140721a0 con 0x7f8e14099a40 2023-09-30T22:32:39.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.964+0000 7f8e1a06f700 1 --2- 172.21.15.16:0/1673727031 >> v2:172.21.15.16:3300/0 conn(0x7f8e14131810 0x7f8e14131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:39.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.964+0000 7f8e1986e700 1 --2- 172.21.15.16:0/1673727031 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8e14099a40 0x7f8e14099e70 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8e040098f0 tx=0x7f8e04009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d5b46aef18fc46a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:39.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.964+0000 7f8e1906d700 1 -- 172.21.15.16:0/1673727031 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e04011830 con 0x7f8e14099a40 2023-09-30T22:32:39.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.965+0000 7f8e1906d700 1 -- 172.21.15.16:0/1673727031 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e04011990 con 0x7f8e14099a40 2023-09-30T22:32:39.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.965+0000 7f8e1906d700 1 -- 172.21.15.16:0/1673727031 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e04011d20 con 0x7f8e14099a40 2023-09-30T22:32:39.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.965+0000 7f8e1c2d3700 1 -- 172.21.15.16:0/1673727031 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8e14099a40 msgr2=0x7f8e14099e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:39.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.965+0000 7f8e1c2d3700 1 --2- 172.21.15.16:0/1673727031 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8e14099a40 0x7f8e14099e70 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8e040098f0 tx=0x7f8e04009c00 comp rx=0 tx=0).stop 2023-09-30T22:32:39.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.965+0000 7f8e1c2d3700 1 -- 172.21.15.16:0/1673727031 shutdown_connections 2023-09-30T22:32:39.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.965+0000 7f8e1c2d3700 1 --2- 172.21.15.16:0/1673727031 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8e1409a3b0 0x7f8e14138650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:39.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.965+0000 7f8e1c2d3700 1 --2- 172.21.15.16:0/1673727031 >> v2:172.21.15.16:3300/0 conn(0x7f8e14131810 0x7f8e14131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:39.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.965+0000 7f8e1c2d3700 1 --2- 172.21.15.16:0/1673727031 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8e14099a40 0x7f8e14099e70 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:39.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.965+0000 7f8e1c2d3700 1 -- 172.21.15.16:0/1673727031 >> 172.21.15.16:0/1673727031 conn(0x7f8e140a2aa0 msgr2=0x7f8e140a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:39.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.965+0000 7f8e1c2d3700 1 -- 172.21.15.16:0/1673727031 shutdown_connections 2023-09-30T22:32:39.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.965+0000 7f8e1c2d3700 1 -- 172.21.15.16:0/1673727031 wait complete. 2023-09-30T22:32:39.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.967+0000 7f8e1c2d3700 1 Processor -- start 2023-09-30T22:32:39.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.967+0000 7f8e1c2d3700 1 -- start start 2023-09-30T22:32:39.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.967+0000 7f8e1c2d3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8e14099a40 0x7f8e1406f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:39.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.967+0000 7f8e1c2d3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e1409a3b0 0x7f8e1406fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:39.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.967+0000 7f8e1c2d3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8e14131810 0x7f8e14070060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:39.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.967+0000 7f8e1c2d3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1406eb00 con 0x7f8e1409a3b0 2023-09-30T22:32:39.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.967+0000 7f8e1c2d3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e14070ae0 con 0x7f8e14099a40 2023-09-30T22:32:39.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.967+0000 7f8e1c2d3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1411df50 con 0x7f8e14131810 2023-09-30T22:32:39.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.967+0000 7f8e1a870700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8e14131810 0x7f8e14070060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:39.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.967+0000 7f8e1a870700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8e14131810 0x7f8e14070060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:39002/0 (socket says 172.21.15.16:39002) 2023-09-30T22:32:39.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.967+0000 7f8e1a870700 1 -- 172.21.15.16:0/2558107291 learned_addr learned my addr 172.21.15.16:0/2558107291 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:39.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.967+0000 7f8e1a06f700 1 --2- 172.21.15.16:0/2558107291 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8e14099a40 0x7f8e1406f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:39.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.968+0000 7f8e1986e700 1 --2- 172.21.15.16:0/2558107291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e1409a3b0 0x7f8e1406fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:39.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.968+0000 7f8e1a870700 1 -- 172.21.15.16:0/2558107291 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8e14099a40 msgr2=0x7f8e1406f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:39.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.968+0000 7f8e1a870700 1 --2- 172.21.15.16:0/2558107291 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8e14099a40 0x7f8e1406f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:39.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.968+0000 7f8e1a870700 1 -- 172.21.15.16:0/2558107291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e1409a3b0 msgr2=0x7f8e1406fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:39.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.968+0000 7f8e1a870700 1 --2- 172.21.15.16:0/2558107291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e1409a3b0 0x7f8e1406fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:39.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.968+0000 7f8e1a870700 1 -- 172.21.15.16:0/2558107291 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e100094a0 con 0x7f8e14131810 2023-09-30T22:32:39.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.968+0000 7f8e1a870700 1 --2- 172.21.15.16:0/2558107291 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8e14131810 0x7f8e14070060 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c00efc0 tx=0x7f8e0c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:39.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.968+0000 7f8e0affd700 1 -- 172.21.15.16:0/2558107291 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c009ea0 con 0x7f8e14131810 2023-09-30T22:32:39.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.968+0000 7f8e0affd700 1 -- 172.21.15.16:0/2558107291 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e0c00f040 con 0x7f8e14131810 2023-09-30T22:32:39.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.968+0000 7f8e0affd700 1 -- 172.21.15.16:0/2558107291 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c014480 con 0x7f8e14131810 2023-09-30T22:32:39.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.968+0000 7f8e1c2d3700 1 -- 172.21.15.16:0/2558107291 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e04009550 con 0x7f8e14131810 2023-09-30T22:32:39.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.968+0000 7f8e1c2d3700 1 -- 172.21.15.16:0/2558107291 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e1411d510 con 0x7f8e14131810 2023-09-30T22:32:39.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.969+0000 7f8e0affd700 1 -- 172.21.15.16:0/2558107291 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8e0c0145e0 con 0x7f8e14131810 2023-09-30T22:32:39.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.970+0000 7f8e1c2d3700 1 -- 172.21.15.16:0/2558107291 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e14074ea0 con 0x7f8e14131810 2023-09-30T22:32:39.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.970+0000 7f8e0affd700 1 -- 172.21.15.16:0/2558107291 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f8e0c03ff90 con 0x7f8e14131810 2023-09-30T22:32:39.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:39.979+0000 7f8e0affd700 1 -- 172.21.15.16:0/2558107291 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8e0c01a8b0 con 0x7f8e14131810 2023-09-30T22:32:40.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.010+0000 7f9c1d853700 1 Processor -- start 2023-09-30T22:32:40.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.010+0000 7f9c1d853700 1 -- start start 2023-09-30T22:32:40.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.011+0000 7f9c1d853700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9c18133420 0x7f9c181337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:40.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.011+0000 7f9c1d853700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9c18133e30 0x7f9c1812ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:40.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.011+0000 7f9c1d853700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c1806b990 con 0x7f9c18134260 2023-09-30T22:32:40.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.011+0000 7f9c1d853700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c180a0590 con 0x7f9c18133e30 2023-09-30T22:32:40.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.011+0000 7f9c1d853700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1809fb80 con 0x7f9c18133420 2023-09-30T22:32:40.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.011+0000 7f9c177fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9c18134260 0x7f9c181304d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53620/0 (socket says 172.21.15.16:53620) 2023-09-30T22:32:40.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.011+0000 7f9c177fe700 1 -- 172.21.15.16:0/2963664030 learned_addr learned my addr 172.21.15.16:0/2963664030 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:40.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.011+0000 7f9c167fc700 1 --2- 172.21.15.16:0/2963664030 >> v2:172.21.15.16:3300/0 conn(0x7f9c18133e30 0x7f9c1812ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:40.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.011+0000 7f9c16ffd700 1 --2- 172.21.15.16:0/2963664030 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9c18133420 0x7f9c181337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:40.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.012+0000 7f9c167fc700 1 -- 172.21.15.16:0/2963664030 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9c18133420 msgr2=0x7f9c181337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:40.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.012+0000 7f9c167fc700 1 --2- 172.21.15.16:0/2963664030 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9c18133420 0x7f9c181337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:40.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.012+0000 7f9c167fc700 1 -- 172.21.15.16:0/2963664030 >> v1:172.21.15.16:6789/0 conn(0x7f9c18134260 legacy=0x7f9c181304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:40.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.012+0000 7f9c167fc700 1 -- 172.21.15.16:0/2963664030 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c180a0030 con 0x7f9c18133e30 2023-09-30T22:32:40.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.012+0000 7f9c167fc700 1 --2- 172.21.15.16:0/2963664030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c18133e30 0x7f9c1812ff90 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c009840 tx=0x7f9c0c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f99de7adf536d47a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:40.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.012+0000 7f9c15ffb700 1 -- 172.21.15.16:0/2963664030 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c004230 con 0x7f9c18133e30 2023-09-30T22:32:40.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.012+0000 7f9c15ffb700 1 -- 172.21.15.16:0/2963664030 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c0c004390 con 0x7f9c18133e30 2023-09-30T22:32:40.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.012+0000 7f9c15ffb700 1 -- 172.21.15.16:0/2963664030 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c0036a0 con 0x7f9c18133e30 2023-09-30T22:32:40.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.013+0000 7f9c1d853700 1 -- 172.21.15.16:0/2963664030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c18133e30 msgr2=0x7f9c1812ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:40.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.013+0000 7f9c1d853700 1 --2- 172.21.15.16:0/2963664030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c18133e30 0x7f9c1812ff90 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c009840 tx=0x7f9c0c009b50 comp rx=0 tx=0).stop 2023-09-30T22:32:40.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.013+0000 7f9c1d853700 1 -- 172.21.15.16:0/2963664030 shutdown_connections 2023-09-30T22:32:40.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.013+0000 7f9c1d853700 1 --2- 172.21.15.16:0/2963664030 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9c18133420 0x7f9c181337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:40.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.013+0000 7f9c1d853700 1 --2- 172.21.15.16:0/2963664030 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c18133e30 0x7f9c1812ff90 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:40.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.013+0000 7f9c1d853700 1 -- 172.21.15.16:0/2963664030 >> 172.21.15.16:0/2963664030 conn(0x7f9c180a4210 msgr2=0x7f9c180a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:40.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.013+0000 7f9c1d853700 1 -- 172.21.15.16:0/2963664030 shutdown_connections 2023-09-30T22:32:40.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.013+0000 7f9c1d853700 1 -- 172.21.15.16:0/2963664030 wait complete. 2023-09-30T22:32:40.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.014+0000 7f9c1d853700 1 Processor -- start 2023-09-30T22:32:40.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.014+0000 7f9c1d853700 1 -- start start 2023-09-30T22:32:40.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.015+0000 7f9c1d853700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c18133420 0x7f9c1809c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:40.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.015+0000 7f9c1d853700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9c18133e30 0x7f9c1809c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:40.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.015+0000 7f9c1d853700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9c18134260 0x7f9c1809cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:40.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.015+0000 7f9c1d853700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1812d280 con 0x7f9c18133420 2023-09-30T22:32:40.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.015+0000 7f9c1d853700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1809dea0 con 0x7f9c18133e30 2023-09-30T22:32:40.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.015+0000 7f9c1d853700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1811fc80 con 0x7f9c18134260 2023-09-30T22:32:40.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.015+0000 7f9c16ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c18133420 0x7f9c1809c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:40.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.015+0000 7f9c16ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c18133420 0x7f9c1809c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35854/0 (socket says 172.21.15.16:35854) 2023-09-30T22:32:40.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.015+0000 7f9c16ffd700 1 -- 172.21.15.16:0/4293173409 learned_addr learned my addr 172.21.15.16:0/4293173409 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:40.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.015+0000 7f9c177fe700 1 --2- 172.21.15.16:0/4293173409 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9c18134260 0x7f9c1809cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:40.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.015+0000 7f9c167fc700 1 --2- 172.21.15.16:0/4293173409 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9c18133e30 0x7f9c1809c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:40.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.015+0000 7f9c16ffd700 1 -- 172.21.15.16:0/4293173409 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9c18134260 msgr2=0x7f9c1809cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:40.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.015+0000 7f9c16ffd700 1 --2- 172.21.15.16:0/4293173409 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9c18134260 0x7f9c1809cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:40.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.015+0000 7f9c16ffd700 1 -- 172.21.15.16:0/4293173409 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9c18133e30 msgr2=0x7f9c1809c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:40.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.015+0000 7f9c16ffd700 1 --2- 172.21.15.16:0/4293173409 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9c18133e30 0x7f9c1809c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:40.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.015+0000 7f9c16ffd700 1 -- 172.21.15.16:0/4293173409 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c0c0094a0 con 0x7f9c18133420 2023-09-30T22:32:40.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.016+0000 7f9c177fe700 1 --2- 172.21.15.16:0/4293173409 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9c18134260 0x7f9c1809cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:32:40.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.016+0000 7f9c16ffd700 1 --2- 172.21.15.16:0/4293173409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c18133420 0x7f9c1809c480 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f9c0800e9b0 tx=0x7f9c0800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:40.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.016+0000 7f9bfffff700 1 -- 172.21.15.16:0/4293173409 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c08009710 con 0x7f9c18133420 2023-09-30T22:32:40.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.016+0000 7f9c1d853700 1 -- 172.21.15.16:0/4293173409 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c1812d650 con 0x7f9c18133420 2023-09-30T22:32:40.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.016+0000 7f9c1d853700 1 -- 172.21.15.16:0/4293173409 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c1811ea90 con 0x7f9c18133420 2023-09-30T22:32:40.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.016+0000 7f9bfffff700 1 -- 172.21.15.16:0/4293173409 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c0800c850 con 0x7f9c18133420 2023-09-30T22:32:40.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.016+0000 7f9bfffff700 1 -- 172.21.15.16:0/4293173409 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0800cbc0 con 0x7f9c18133420 2023-09-30T22:32:40.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.017+0000 7f9bfffff700 1 -- 172.21.15.16:0/4293173409 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9c0800cd20 con 0x7f9c18133420 2023-09-30T22:32:40.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.017+0000 7f9c1d853700 1 -- 172.21.15.16:0/4293173409 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bf80052e0 con 0x7f9c18133420 2023-09-30T22:32:40.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.017+0000 7f9bfffff700 1 -- 172.21.15.16:0/4293173409 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f9c0803fb10 con 0x7f9c18133420 2023-09-30T22:32:40.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.027+0000 7f9bfffff700 1 -- 172.21.15.16:0/4293173409 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9c0801b3b0 con 0x7f9c18133420 2023-09-30T22:32:40.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.045+0000 7f72cd0b0700 1 Processor -- start 2023-09-30T22:32:40.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.046+0000 7f72cd0b0700 1 -- start start 2023-09-30T22:32:40.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.046+0000 7f72cd0b0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72c809d6f0 0x7f72c809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:40.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.046+0000 7f72cd0b0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72c809e000 0x7f72c81339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:40.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.046+0000 7f72cd0b0700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f72c809e410 0x7f72c8133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:40.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.046+0000 7f72cd0b0700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72c806b460 con 0x7f72c809e410 2023-09-30T22:32:40.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.046+0000 7f72cd0b0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72c809b120 con 0x7f72c809d6f0 2023-09-30T22:32:40.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.046+0000 7f72cd0b0700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f72c809a7e0 con 0x7f72c809e000 2023-09-30T22:32:40.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.047+0000 7f72c6d9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72c809d6f0 0x7f72c809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:40.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.047+0000 7f72c659c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72c809e000 0x7f72c81339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:40.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.047+0000 7f72c759e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f72c809e410 0x7f72c8133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:40.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.047+0000 7f72c6d9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72c809d6f0 0x7f72c809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:39368/0 (socket says 172.21.15.16:39368) 2023-09-30T22:32:40.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.047+0000 7f72c6d9d700 1 -- 172.21.15.16:0/1381075245 learned_addr learned my addr 172.21.15.16:0/1381075245 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:40.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.047+0000 7f72c759e700 1 -- 172.21.15.16:0/1381075245 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72c809e000 msgr2=0x7f72c81339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:40.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.047+0000 7f72c759e700 1 --2- 172.21.15.16:0/1381075245 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72c809e000 0x7f72c81339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:40.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.047+0000 7f72c759e700 1 -- 172.21.15.16:0/1381075245 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72c809d6f0 msgr2=0x7f72c809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:40.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.047+0000 7f72c759e700 1 --2- 172.21.15.16:0/1381075245 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72c809d6f0 0x7f72c809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:40.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.047+0000 7f72c759e700 1 -- 172.21.15.16:0/1381075245 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72c809a500 con 0x7f72c809e410 2023-09-30T22:32:40.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.047+0000 7f72c6d9d700 1 --2- 172.21.15.16:0/1381075245 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72c809d6f0 0x7f72c809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:40.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.047+0000 7f72c659c700 1 --2- 172.21.15.16:0/1381075245 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72c809e000 0x7f72c81339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:40.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.047+0000 7f72c759e700 1 --2- 172.21.15.16:0/1381075245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72c809e410 0x7f72c8133f30 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f72c0009d90 tx=0x7f72c000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f85e359080a83e2b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:40.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.048+0000 7f72c5d9b700 1 -- 172.21.15.16:0/1381075245 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c000e070 con 0x7f72c809e410 2023-09-30T22:32:40.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.048+0000 7f72c5d9b700 1 -- 172.21.15.16:0/1381075245 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72c000bb20 con 0x7f72c809e410 2023-09-30T22:32:40.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.048+0000 7f72c5d9b700 1 -- 172.21.15.16:0/1381075245 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c0012430 con 0x7f72c809e410 2023-09-30T22:32:40.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.049+0000 7f72cd0b0700 1 -- 172.21.15.16:0/1381075245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72c809e410 msgr2=0x7f72c8133f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:40.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.049+0000 7f72cd0b0700 1 --2- 172.21.15.16:0/1381075245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72c809e410 0x7f72c8133f30 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f72c0009d90 tx=0x7f72c000b330 comp rx=0 tx=0).stop 2023-09-30T22:32:40.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.049+0000 7f72cd0b0700 1 -- 172.21.15.16:0/1381075245 shutdown_connections 2023-09-30T22:32:40.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.049+0000 7f72cd0b0700 1 --2- 172.21.15.16:0/1381075245 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72c809d6f0 0x7f72c809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:40.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.049+0000 7f72cd0b0700 1 --2- 172.21.15.16:0/1381075245 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72c809e410 0x7f72c8133f30 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:40.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.049+0000 7f72cd0b0700 1 --2- 172.21.15.16:0/1381075245 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72c809e000 0x7f72c81339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:40.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.049+0000 7f72cd0b0700 1 -- 172.21.15.16:0/1381075245 >> 172.21.15.16:0/1381075245 conn(0x7f72c8066f50 msgr2=0x7f72c809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:40.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.049+0000 7f72cd0b0700 1 -- 172.21.15.16:0/1381075245 shutdown_connections 2023-09-30T22:32:40.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.049+0000 7f72cd0b0700 1 -- 172.21.15.16:0/1381075245 wait complete. 2023-09-30T22:32:40.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.051+0000 7f72cd0b0700 1 Processor -- start 2023-09-30T22:32:40.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.051+0000 7f72cd0b0700 1 -- start start 2023-09-30T22:32:40.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.051+0000 7f72cd0b0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72c809d6f0 0x7f72c81a5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:40.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.051+0000 7f72cd0b0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72c809e000 0x7f72c81a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:40.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.052+0000 7f72cd0b0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72c809e410 0x7f72c81a6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:40.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.052+0000 7f72cd0b0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c81a8100 con 0x7f72c809e410 2023-09-30T22:32:40.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.052+0000 7f72cd0b0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72c81a8dc0 con 0x7f72c809e000 2023-09-30T22:32:40.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.052+0000 7f72cd0b0700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f72c80a2090 con 0x7f72c809d6f0 2023-09-30T22:32:40.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.052+0000 7f72c6d9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72c809d6f0 0x7f72c81a5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:40.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.052+0000 7f72c6d9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72c809d6f0 0x7f72c81a5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:39030/0 (socket says 172.21.15.16:39030) 2023-09-30T22:32:40.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.052+0000 7f72c6d9d700 1 -- 172.21.15.16:0/1446032928 learned_addr learned my addr 172.21.15.16:0/1446032928 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:40.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.052+0000 7f72c659c700 1 --2- 172.21.15.16:0/1446032928 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72c809e000 0x7f72c81a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:40.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.052+0000 7f72c759e700 1 --2- 172.21.15.16:0/1446032928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72c809e410 0x7f72c81a6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:40.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.052+0000 7f72c659c700 1 -- 172.21.15.16:0/1446032928 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72c809d6f0 msgr2=0x7f72c81a5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:40.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.052+0000 7f72c659c700 1 --2- 172.21.15.16:0/1446032928 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72c809d6f0 0x7f72c81a5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:40.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.052+0000 7f72c659c700 1 -- 172.21.15.16:0/1446032928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72c809e410 msgr2=0x7f72c81a6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:40.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.052+0000 7f72c659c700 1 --2- 172.21.15.16:0/1446032928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72c809e410 0x7f72c81a6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:40.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.052+0000 7f72c659c700 1 -- 172.21.15.16:0/1446032928 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b8009550 con 0x7f72c809e000 2023-09-30T22:32:40.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.052+0000 7f72c659c700 1 --2- 172.21.15.16:0/1446032928 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72c809e000 0x7f72c81a6410 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f72bc00e980 tx=0x7f72bc00ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:40.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.053+0000 7f72af7fe700 1 -- 172.21.15.16:0/1446032928 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72bc00c690 con 0x7f72c809e000 2023-09-30T22:32:40.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.053+0000 7f72af7fe700 1 -- 172.21.15.16:0/1446032928 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72bc00c7f0 con 0x7f72c809e000 2023-09-30T22:32:40.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.053+0000 7f72cd0b0700 1 -- 172.21.15.16:0/1446032928 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72bc0094a0 con 0x7f72c809e000 2023-09-30T22:32:40.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.053+0000 7f72af7fe700 1 -- 172.21.15.16:0/1446032928 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72bc00cb80 con 0x7f72c809e000 2023-09-30T22:32:40.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.053+0000 7f72cd0b0700 1 -- 172.21.15.16:0/1446032928 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72c0009a70 con 0x7f72c809e000 2023-09-30T22:32:40.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.054+0000 7f72af7fe700 1 -- 172.21.15.16:0/1446032928 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f72bc00cce0 con 0x7f72c809e000 2023-09-30T22:32:40.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.054+0000 7f72cd0b0700 1 -- 172.21.15.16:0/1446032928 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72c8074ea0 con 0x7f72c809e000 2023-09-30T22:32:40.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.054+0000 7f72af7fe700 1 -- 172.21.15.16:0/1446032928 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f72bc03f9f0 con 0x7f72c809e000 2023-09-30T22:32:40.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.059+0000 7f72af7fe700 1 -- 172.21.15.16:0/1446032928 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f72bc011db0 con 0x7f72c809e000 2023-09-30T22:32:40.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.118+0000 7fb65bb4d700 1 -- 172.21.15.16:0/1010788217 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb6541217d0 con 0x7fb65409e000 2023-09-30T22:32:40.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.119+0000 7fb64a7fc700 1 -- 172.21.15.16:0/1010788217 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb64402a090 con 0x7fb65409e000 2023-09-30T22:32:40.119 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:32:40.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.121+0000 7fb65bb4d700 1 -- 172.21.15.16:0/1010788217 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb65409e000 msgr2=0x7fb65411e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:40.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.121+0000 7fb65bb4d700 1 --2- 172.21.15.16:0/1010788217 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb65409e000 0x7fb65411e3b0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb64400e900 tx=0x7fb64400ec10 comp rx=0 tx=0).stop 2023-09-30T22:32:40.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.122+0000 7fb65bb4d700 1 -- 172.21.15.16:0/1010788217 shutdown_connections 2023-09-30T22:32:40.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.122+0000 7fb65bb4d700 1 --2- 172.21.15.16:0/1010788217 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb65409e410 0x7fb65411e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:40.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.122+0000 7fb65bb4d700 1 --2- 172.21.15.16:0/1010788217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb65409d6f0 0x7fb65411de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:40.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.122+0000 7fb65bb4d700 1 --2- 172.21.15.16:0/1010788217 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb65409e000 0x7fb65411e3b0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:40.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.122+0000 7fb65bb4d700 1 -- 172.21.15.16:0/1010788217 >> 172.21.15.16:0/1010788217 conn(0x7fb654066f50 msgr2=0x7fb654130a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:40.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.122+0000 7fb65bb4d700 1 -- 172.21.15.16:0/1010788217 shutdown_connections 2023-09-30T22:32:40.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.122+0000 7fb65bb4d700 1 -- 172.21.15.16:0/1010788217 wait complete. 2023-09-30T22:32:40.141 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:32:40.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.204+0000 7f8e1c2d3700 1 -- 172.21.15.16:0/2558107291 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f8e1406b9e0 con 0x7f8e14131810 2023-09-30T22:32:40.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.205+0000 7f8e0affd700 1 -- 172.21.15.16:0/2558107291 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8e0c018020 con 0x7f8e14131810 2023-09-30T22:32:40.205 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:32:40.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.207+0000 7f8e1c2d3700 1 -- 172.21.15.16:0/2558107291 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8e14131810 msgr2=0x7f8e14070060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:40.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.207+0000 7f8e1c2d3700 1 --2- 172.21.15.16:0/2558107291 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8e14131810 0x7f8e14070060 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c00efc0 tx=0x7f8e0c00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:32:40.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.207+0000 7f8e1c2d3700 1 -- 172.21.15.16:0/2558107291 shutdown_connections 2023-09-30T22:32:40.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.207+0000 7f8e1c2d3700 1 --2- 172.21.15.16:0/2558107291 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8e14099a40 0x7f8e1406f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:40.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.207+0000 7f8e1c2d3700 1 --2- 172.21.15.16:0/2558107291 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8e14131810 0x7f8e14070060 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:40.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.207+0000 7f8e1c2d3700 1 --2- 172.21.15.16:0/2558107291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e1409a3b0 0x7f8e1406fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:40.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.207+0000 7f8e1c2d3700 1 -- 172.21.15.16:0/2558107291 >> 172.21.15.16:0/2558107291 conn(0x7f8e140a2aa0 msgr2=0x7f8e1412e710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:40.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.207+0000 7f8e1c2d3700 1 -- 172.21.15.16:0/2558107291 shutdown_connections 2023-09-30T22:32:40.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.207+0000 7f8e1c2d3700 1 -- 172.21.15.16:0/2558107291 wait complete. 2023-09-30T22:32:40.221 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:32:40.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.245+0000 7f9c1d853700 1 -- 172.21.15.16:0/4293173409 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f9bf8005590 con 0x7f9c18133420 2023-09-30T22:32:40.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.245+0000 7f9bfffff700 1 -- 172.21.15.16:0/4293173409 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9c08017030 con 0x7f9c18133420 2023-09-30T22:32:40.245 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:32:40.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.247+0000 7f9c1d853700 1 -- 172.21.15.16:0/4293173409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c18133420 msgr2=0x7f9c1809c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:40.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.247+0000 7f9c1d853700 1 --2- 172.21.15.16:0/4293173409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c18133420 0x7f9c1809c480 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f9c0800e9b0 tx=0x7f9c0800ecc0 comp rx=0 tx=0).stop 2023-09-30T22:32:40.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.247+0000 7f9c1d853700 1 -- 172.21.15.16:0/4293173409 shutdown_connections 2023-09-30T22:32:40.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.247+0000 7f9c1d853700 1 --2- 172.21.15.16:0/4293173409 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9c18134260 0x7f9c1809cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:40.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.247+0000 7f9c1d853700 1 --2- 172.21.15.16:0/4293173409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c18133420 0x7f9c1809c480 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:40.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.247+0000 7f9c1d853700 1 --2- 172.21.15.16:0/4293173409 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9c18133e30 0x7f9c1809c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:40.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.247+0000 7f9c1d853700 1 -- 172.21.15.16:0/4293173409 >> 172.21.15.16:0/4293173409 conn(0x7f9c180a4210 msgr2=0x7f9c180a0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:40.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.247+0000 7f9c1d853700 1 -- 172.21.15.16:0/4293173409 shutdown_connections 2023-09-30T22:32:40.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.247+0000 7f9c1d853700 1 -- 172.21.15.16:0/4293173409 wait complete. 2023-09-30T22:32:40.261 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:32:40.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.266+0000 7f72cd0b0700 1 -- 172.21.15.16:0/1446032928 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f72c81a7ae0 con 0x7f72c809e000 2023-09-30T22:32:40.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.267+0000 7f72af7fe700 1 -- 172.21.15.16:0/1446032928 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f72bc021070 con 0x7f72c809e000 2023-09-30T22:32:40.267 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:32:40.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.269+0000 7f72cd0b0700 1 -- 172.21.15.16:0/1446032928 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72c809e000 msgr2=0x7f72c81a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:40.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.269+0000 7f72cd0b0700 1 --2- 172.21.15.16:0/1446032928 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72c809e000 0x7f72c81a6410 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f72bc00e980 tx=0x7f72bc00ec90 comp rx=0 tx=0).stop 2023-09-30T22:32:40.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.269+0000 7f72cd0b0700 1 -- 172.21.15.16:0/1446032928 shutdown_connections 2023-09-30T22:32:40.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.269+0000 7f72cd0b0700 1 --2- 172.21.15.16:0/1446032928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72c809e410 0x7f72c81a6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:40.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.269+0000 7f72cd0b0700 1 --2- 172.21.15.16:0/1446032928 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72c809d6f0 0x7f72c81a5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:40.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.269+0000 7f72cd0b0700 1 --2- 172.21.15.16:0/1446032928 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72c809e000 0x7f72c81a6410 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:40.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.269+0000 7f72cd0b0700 1 -- 172.21.15.16:0/1446032928 >> 172.21.15.16:0/1446032928 conn(0x7f72c8066f50 msgr2=0x7f72c81bae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:40.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.269+0000 7f72cd0b0700 1 -- 172.21.15.16:0/1446032928 shutdown_connections 2023-09-30T22:32:40.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:40.269+0000 7f72cd0b0700 1 -- 172.21.15.16:0/1446032928 wait complete. 2023-09-30T22:32:40.283 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:32:41.142 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:32:41.222 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:32:41.263 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:32:41.284 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:32:41.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.304+0000 7f8b21af7700 1 Processor -- start 2023-09-30T22:32:41.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.304+0000 7f8b21af7700 1 -- start start 2023-09-30T22:32:41.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.305+0000 7f8b21af7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8b1c1338a0 0x7f8b1c133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:41.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.305+0000 7f8b21af7700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8b1c09de50 0x7f8b1c09e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:41.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.305+0000 7f8b21af7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8b1c09e760 0x7f8b1c13a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:41.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.305+0000 7f8b21af7700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b1c06fef0 con 0x7f8b1c09de50 2023-09-30T22:32:41.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.305+0000 7f8b21af7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1c06ef30 con 0x7f8b1c1338a0 2023-09-30T22:32:41.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.305+0000 7f8b21af7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1c1334b0 con 0x7f8b1c09e760 2023-09-30T22:32:41.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.305+0000 7f8b1b7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8b1c1338a0 0x7f8b1c133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:41.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.305+0000 7f8b1affd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8b1c09de50 0x7f8b1c09e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:41.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.305+0000 7f8b1affd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8b1c09de50 0x7f8b1c09e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35890/0 (socket says 172.21.15.16:35890) 2023-09-30T22:32:41.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.305+0000 7f8b1b7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8b1c1338a0 0x7f8b1c133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:39390/0 (socket says 172.21.15.16:39390) 2023-09-30T22:32:41.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.305+0000 7f8b1affd700 1 -- 172.21.15.16:0/474685183 learned_addr learned my addr 172.21.15.16:0/474685183 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:41.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.305+0000 7f8b1bfff700 1 --2- 172.21.15.16:0/474685183 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8b1c09e760 0x7f8b1c13a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:41.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.305+0000 7f8b1affd700 1 -- 172.21.15.16:0/474685183 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8b1c09e760 msgr2=0x7f8b1c13a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:41.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.305+0000 7f8b1affd700 1 --2- 172.21.15.16:0/474685183 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8b1c09e760 0x7f8b1c13a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:41.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.305+0000 7f8b1affd700 1 -- 172.21.15.16:0/474685183 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8b1c1338a0 msgr2=0x7f8b1c133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:41.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.305+0000 7f8b1affd700 1 --2- 172.21.15.16:0/474685183 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8b1c1338a0 0x7f8b1c133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:41.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.305+0000 7f8b1affd700 1 -- 172.21.15.16:0/474685183 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b1c06fc30 con 0x7f8b1c09de50 2023-09-30T22:32:41.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.306+0000 7f8b1affd700 1 --2- 172.21.15.16:0/474685183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b1c09de50 0x7f8b1c09e220 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f8b100098f0 tx=0x7f8b10009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2fee9c8cf55ce4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:41.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.306+0000 7f8b1a7fc700 1 -- 172.21.15.16:0/474685183 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b10011830 con 0x7f8b1c09de50 2023-09-30T22:32:41.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.306+0000 7f8b1a7fc700 1 -- 172.21.15.16:0/474685183 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b10011990 con 0x7f8b1c09de50 2023-09-30T22:32:41.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.306+0000 7f8b1a7fc700 1 -- 172.21.15.16:0/474685183 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b10011d20 con 0x7f8b1c09de50 2023-09-30T22:32:41.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.307+0000 7f8b21af7700 1 -- 172.21.15.16:0/474685183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b1c09de50 msgr2=0x7f8b1c09e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:41.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.307+0000 7f8b21af7700 1 --2- 172.21.15.16:0/474685183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b1c09de50 0x7f8b1c09e220 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f8b100098f0 tx=0x7f8b10009c00 comp rx=0 tx=0).stop 2023-09-30T22:32:41.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.307+0000 7f8b21af7700 1 -- 172.21.15.16:0/474685183 shutdown_connections 2023-09-30T22:32:41.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.307+0000 7f8b21af7700 1 --2- 172.21.15.16:0/474685183 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8b1c1338a0 0x7f8b1c133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:41.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.307+0000 7f8b21af7700 1 --2- 172.21.15.16:0/474685183 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8b1c09e760 0x7f8b1c13a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:41.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.307+0000 7f8b21af7700 1 --2- 172.21.15.16:0/474685183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b1c09de50 0x7f8b1c09e220 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:41.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.307+0000 7f8b21af7700 1 -- 172.21.15.16:0/474685183 >> 172.21.15.16:0/474685183 conn(0x7f8b1c0a2aa0 msgr2=0x7f8b1c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:41.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.307+0000 7f8b21af7700 1 -- 172.21.15.16:0/474685183 shutdown_connections 2023-09-30T22:32:41.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.307+0000 7f8b21af7700 1 -- 172.21.15.16:0/474685183 wait complete. 2023-09-30T22:32:41.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.308+0000 7f8b21af7700 1 Processor -- start 2023-09-30T22:32:41.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.308+0000 7f8b21af7700 1 -- start start 2023-09-30T22:32:41.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.309+0000 7f8b21af7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8b1c09de50 0x7f8b1c11d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:41.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.309+0000 7f8b21af7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b1c09e760 0x7f8b1c11db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:41.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.309+0000 7f8b21af7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8b1c1338a0 0x7f8b1c11e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:41.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.309+0000 7f8b21af7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1c120900 con 0x7f8b1c09e760 2023-09-30T22:32:41.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.309+0000 7f8b21af7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1c06f640 con 0x7f8b1c1338a0 2023-09-30T22:32:41.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.309+0000 7f8b21af7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1c0a0ff0 con 0x7f8b1c09de50 2023-09-30T22:32:41.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.309+0000 7f8b1affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b1c09e760 0x7f8b1c11db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:41.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.309+0000 7f8b1affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b1c09e760 0x7f8b1c11db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35892/0 (socket says 172.21.15.16:35892) 2023-09-30T22:32:41.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.309+0000 7f8b1affd700 1 -- 172.21.15.16:0/276292402 learned_addr learned my addr 172.21.15.16:0/276292402 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:41.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.309+0000 7f8b1bfff700 1 --2- 172.21.15.16:0/276292402 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8b1c1338a0 0x7f8b1c11e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:41.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.309+0000 7f8b1b7fe700 1 --2- 172.21.15.16:0/276292402 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8b1c09de50 0x7f8b1c11d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:41.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.310+0000 7f8b1bfff700 1 -- 172.21.15.16:0/276292402 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8b1c09de50 msgr2=0x7f8b1c11d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:41.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.310+0000 7f8b1bfff700 1 --2- 172.21.15.16:0/276292402 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8b1c09de50 0x7f8b1c11d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:41.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.310+0000 7f8b1bfff700 1 -- 172.21.15.16:0/276292402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b1c09e760 msgr2=0x7f8b1c11db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:41.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.310+0000 7f8b1bfff700 1 --2- 172.21.15.16:0/276292402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b1c09e760 0x7f8b1c11db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:41.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.310+0000 7f8b1bfff700 1 -- 172.21.15.16:0/276292402 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b10009550 con 0x7f8b1c1338a0 2023-09-30T22:32:41.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.310+0000 7f8b1b7fe700 1 --2- 172.21.15.16:0/276292402 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8b1c09de50 0x7f8b1c11d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:41.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.310+0000 7f8b1bfff700 1 --2- 172.21.15.16:0/276292402 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8b1c1338a0 0x7f8b1c11e0d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f8b1400efc0 tx=0x7f8b1400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:41.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.310+0000 7f8b03fff700 1 -- 172.21.15.16:0/276292402 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b1400cc90 con 0x7f8b1c1338a0 2023-09-30T22:32:41.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.310+0000 7f8b03fff700 1 -- 172.21.15.16:0/276292402 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b1400f040 con 0x7f8b1c1338a0 2023-09-30T22:32:41.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.310+0000 7f8b03fff700 1 -- 172.21.15.16:0/276292402 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b140146c0 con 0x7f8b1c1338a0 2023-09-30T22:32:41.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.311+0000 7f8b21af7700 1 -- 172.21.15.16:0/276292402 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b1c11f260 con 0x7f8b1c1338a0 2023-09-30T22:32:41.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.311+0000 7f8b21af7700 1 -- 172.21.15.16:0/276292402 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b1c121670 con 0x7f8b1c1338a0 2023-09-30T22:32:41.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.312+0000 7f8b03fff700 1 -- 172.21.15.16:0/276292402 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8b14014820 con 0x7f8b1c1338a0 2023-09-30T22:32:41.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.312+0000 7f8b03fff700 1 -- 172.21.15.16:0/276292402 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f8b14040200 con 0x7f8b1c1338a0 2023-09-30T22:32:41.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.312+0000 7f8b21af7700 1 -- 172.21.15.16:0/276292402 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b1c11a470 con 0x7f8b1c1338a0 2023-09-30T22:32:41.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.322+0000 7f8b03fff700 1 -- 172.21.15.16:0/276292402 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8b14018020 con 0x7f8b1c1338a0 2023-09-30T22:32:41.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.382+0000 7f662908a700 1 Processor -- start 2023-09-30T22:32:41.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.382+0000 7f662908a700 1 -- start start 2023-09-30T22:32:41.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.382+0000 7f662908a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f662409eca0 0x7f662413cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:41.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.382+0000 7f662908a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f662406f050 0x7f662413f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:41.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.382+0000 7f662908a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6624070ee0 con 0x7f662406e570 2023-09-30T22:32:41.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.383+0000 7f662908a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6624071980 con 0x7f662409eca0 2023-09-30T22:32:41.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.383+0000 7f662908a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6624071480 con 0x7f662406f050 2023-09-30T22:32:41.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.383+0000 7f662359e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f662406f050 0x7f662413f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:41.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.383+0000 7f6622d9d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f662406e570 0x7f662406e940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53624/0 (socket says 172.21.15.16:53624) 2023-09-30T22:32:41.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.383+0000 7f6622d9d700 1 -- 172.21.15.16:0/164338846 learned_addr learned my addr 172.21.15.16:0/164338846 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:41.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.383+0000 7f662259c700 1 --2- 172.21.15.16:0/164338846 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f662409eca0 0x7f662413cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:41.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.383+0000 7f662359e700 1 --2- 172.21.15.16:0/164338846 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f662406f050 0x7f662413f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:39064/0 (socket says 172.21.15.16:39064) 2023-09-30T22:32:41.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.383+0000 7f662359e700 1 -- 172.21.15.16:0/164338846 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f662409eca0 msgr2=0x7f662413cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:41.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.383+0000 7f662359e700 1 --2- 172.21.15.16:0/164338846 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f662409eca0 0x7f662413cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:41.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.383+0000 7f662359e700 1 -- 172.21.15.16:0/164338846 >> v1:172.21.15.16:6789/0 conn(0x7f662406e570 legacy=0x7f662406e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:41.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.383+0000 7f662359e700 1 -- 172.21.15.16:0/164338846 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6624071b90 con 0x7f662406f050 2023-09-30T22:32:41.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.383+0000 7f662259c700 1 --2- 172.21.15.16:0/164338846 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f662409eca0 0x7f662413cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:41.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.384+0000 7f662359e700 1 --2- 172.21.15.16:0/164338846 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f662406f050 0x7f662413f3f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f661c009dc0 tx=0x7f661c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1a4515f8f26bb39 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:41.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.384+0000 7f6621d9b700 1 -- 172.21.15.16:0/164338846 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661c00e070 con 0x7f662406f050 2023-09-30T22:32:41.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.384+0000 7f6621d9b700 1 -- 172.21.15.16:0/164338846 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f661c00bb20 con 0x7f662406f050 2023-09-30T22:32:41.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.384+0000 7f6621d9b700 1 -- 172.21.15.16:0/164338846 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661c012430 con 0x7f662406f050 2023-09-30T22:32:41.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.384+0000 7f662908a700 1 -- 172.21.15.16:0/164338846 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f662406f050 msgr2=0x7f662413f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:41.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.384+0000 7f662908a700 1 --2- 172.21.15.16:0/164338846 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f662406f050 0x7f662413f3f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f661c009dc0 tx=0x7f661c00b330 comp rx=0 tx=0).stop 2023-09-30T22:32:41.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.385+0000 7f662908a700 1 -- 172.21.15.16:0/164338846 shutdown_connections 2023-09-30T22:32:41.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.385+0000 7f662908a700 1 --2- 172.21.15.16:0/164338846 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f662406f050 0x7f662413f3f0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:41.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.385+0000 7f662908a700 1 --2- 172.21.15.16:0/164338846 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f662409eca0 0x7f662413cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:41.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.385+0000 7f662908a700 1 -- 172.21.15.16:0/164338846 >> 172.21.15.16:0/164338846 conn(0x7f66240a2aa0 msgr2=0x7f66240a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:41.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.385+0000 7f662908a700 1 -- 172.21.15.16:0/164338846 shutdown_connections 2023-09-30T22:32:41.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.385+0000 7f662908a700 1 -- 172.21.15.16:0/164338846 wait complete. 2023-09-30T22:32:41.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.386+0000 7f662908a700 1 Processor -- start 2023-09-30T22:32:41.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.386+0000 7f662908a700 1 -- start start 2023-09-30T22:32:41.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.387+0000 7f662908a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f662406e570 0x7f66241214e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:41.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.387+0000 7f662908a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f662406f050 0x7f6624121a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:41.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.387+0000 7f662908a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6624121f60 0x7f66241cca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:41.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.387+0000 7f662908a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66241232b0 con 0x7f662406f050 2023-09-30T22:32:41.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.387+0000 7f662908a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6624124870 con 0x7f6624121f60 2023-09-30T22:32:41.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.387+0000 7f662908a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6624070610 con 0x7f662406e570 2023-09-30T22:32:41.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.387+0000 7f6622d9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f662406e570 0x7f66241214e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:41.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.387+0000 7f6622d9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f662406e570 0x7f66241214e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:39078/0 (socket says 172.21.15.16:39078) 2023-09-30T22:32:41.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.387+0000 7f6622d9d700 1 -- 172.21.15.16:0/268299589 learned_addr learned my addr 172.21.15.16:0/268299589 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:41.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.387+0000 7f662359e700 1 --2- 172.21.15.16:0/268299589 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6624121f60 0x7f66241cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:41.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.387+0000 7f662259c700 1 --2- 172.21.15.16:0/268299589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f662406f050 0x7f6624121a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:41.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.387+0000 7f6622d9d700 1 -- 172.21.15.16:0/268299589 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6624121f60 msgr2=0x7f66241cca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:41.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.387+0000 7f6622d9d700 1 --2- 172.21.15.16:0/268299589 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6624121f60 0x7f66241cca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:41.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.387+0000 7f6622d9d700 1 -- 172.21.15.16:0/268299589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f662406f050 msgr2=0x7f6624121a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:41.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.387+0000 7f6622d9d700 1 --2- 172.21.15.16:0/268299589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f662406f050 0x7f6624121a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:41.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.387+0000 7f6622d9d700 1 -- 172.21.15.16:0/268299589 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66180094a0 con 0x7f662406e570 2023-09-30T22:32:41.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.388+0000 7f6622d9d700 1 --2- 172.21.15.16:0/268299589 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f662406e570 0x7f66241214e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f661400c550 tx=0x7f661400c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:41.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.388+0000 7f660b7fe700 1 -- 172.21.15.16:0/268299589 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6614005d70 con 0x7f662406e570 2023-09-30T22:32:41.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.388+0000 7f660b7fe700 1 -- 172.21.15.16:0/268299589 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66140070d0 con 0x7f662406e570 2023-09-30T22:32:41.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.388+0000 7f662908a700 1 -- 172.21.15.16:0/268299589 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f661c009a70 con 0x7f662406e570 2023-09-30T22:32:41.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.388+0000 7f662908a700 1 -- 172.21.15.16:0/268299589 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f662409a130 con 0x7f662406e570 2023-09-30T22:32:41.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.388+0000 7f660b7fe700 1 -- 172.21.15.16:0/268299589 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6614007440 con 0x7f662406e570 2023-09-30T22:32:41.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.389+0000 7f662908a700 1 -- 172.21.15.16:0/268299589 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66040052e0 con 0x7f662406e570 2023-09-30T22:32:41.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.389+0000 7f660b7fe700 1 -- 172.21.15.16:0/268299589 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f66140075a0 con 0x7f662406e570 2023-09-30T22:32:41.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.390+0000 7f660b7fe700 1 -- 172.21.15.16:0/268299589 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f661403eb50 con 0x7f662406e570 2023-09-30T22:32:41.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.399+0000 7f660b7fe700 1 -- 172.21.15.16:0/268299589 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6614019d20 con 0x7f662406e570 2023-09-30T22:32:41.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.430+0000 7f9e29764700 1 Processor -- start 2023-09-30T22:32:41.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.431+0000 7f9e29764700 1 -- start start 2023-09-30T22:32:41.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.431+0000 7f9e29764700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e241316c0 0x7f9e24131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:41.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.431+0000 7f9e29764700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9e2409d9f0 0x7f9e2406a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:41.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.431+0000 7f9e29764700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e2409e130 con 0x7f9e2406acb0 2023-09-30T22:32:41.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.431+0000 7f9e29764700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e241312b0 con 0x7f9e2409d9f0 2023-09-30T22:32:41.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.431+0000 7f9e29764700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2409f6e0 con 0x7f9e241316c0 2023-09-30T22:32:41.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.432+0000 7f9e237fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9e2406acb0 0x7f9e24138540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53638/0 (socket says 172.21.15.16:53638) 2023-09-30T22:32:41.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.432+0000 7f9e237fe700 1 -- 172.21.15.16:0/2257771773 learned_addr learned my addr 172.21.15.16:0/2257771773 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:41.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.432+0000 7f9e22ffd700 1 --2- 172.21.15.16:0/2257771773 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e241316c0 0x7f9e24131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:41.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.432+0000 7f9e21ffb700 1 -- 172.21.15.16:0/2257771773 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1021579873 0 0) 0x7f9e1c001490 con 0x7f9e2406acb0 2023-09-30T22:32:41.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.432+0000 7f9e21ffb700 1 -- 172.21.15.16:0/2257771773 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e0c0035b0 con 0x7f9e2406acb0 2023-09-30T22:32:41.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.432+0000 7f9e21ffb700 1 -- 172.21.15.16:0/2257771773 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3777718874 0 0) 0x7f9e0c0035b0 con 0x7f9e2406acb0 2023-09-30T22:32:41.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.432+0000 7f9e21ffb700 1 -- 172.21.15.16:0/2257771773 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e241316c0 msgr2=0x7f9e24131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:41.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.432+0000 7f9e21ffb700 1 --2- 172.21.15.16:0/2257771773 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e241316c0 0x7f9e24131a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:41.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.432+0000 7f9e21ffb700 1 -- 172.21.15.16:0/2257771773 >> v2:172.21.15.16:3300/0 conn(0x7f9e2409d9f0 msgr2=0x7f9e2406a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:41.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.432+0000 7f9e21ffb700 1 --2- 172.21.15.16:0/2257771773 >> v2:172.21.15.16:3300/0 conn(0x7f9e2409d9f0 0x7f9e2406a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:41.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.432+0000 7f9e21ffb700 1 -- 172.21.15.16:0/2257771773 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e2409e760 con 0x7f9e2406acb0 2023-09-30T22:32:41.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.432+0000 7f9e21ffb700 1 -- 172.21.15.16:0/2257771773 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1074163509 0 0) 0x7f9e1c001b80 con 0x7f9e2406acb0 2023-09-30T22:32:41.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.433+0000 7f9e21ffb700 1 -- 172.21.15.16:0/2257771773 >> v1:172.21.15.16:6789/0 conn(0x7f9e2406acb0 legacy=0x7f9e24138540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:41.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.433+0000 7f9e21ffb700 1 --2- 172.21.15.16:0/2257771773 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e0c003a00 0x7f9e0c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:41.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.433+0000 7f9e21ffb700 1 --2- 172.21.15.16:0/2257771773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e0c004430 0x7f9e0c0048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:41.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.433+0000 7f9e21ffb700 1 --2- 172.21.15.16:0/2257771773 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e241316c0 0x7f9e0c005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:41.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.433+0000 7f9e21ffb700 1 -- 172.21.15.16:0/2257771773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e241312b0 con 0x7f9e0c004430 2023-09-30T22:32:41.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.433+0000 7f9e21ffb700 1 -- 172.21.15.16:0/2257771773 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2409f6e0 con 0x7f9e0c003a00 2023-09-30T22:32:41.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.433+0000 7f9e21ffb700 1 -- 172.21.15.16:0/2257771773 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0c00bb90 con 0x7f9e241316c0 2023-09-30T22:32:41.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.433+0000 7f9e227fc700 1 --2- 172.21.15.16:0/2257771773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e0c004430 0x7f9e0c0048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:41.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.433+0000 7f9e237fe700 1 --2- 172.21.15.16:0/2257771773 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e241316c0 0x7f9e0c005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:41.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.433+0000 7f9e227fc700 1 -- 172.21.15.16:0/2257771773 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e241316c0 msgr2=0x7f9e0c005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:41.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.433+0000 7f9e227fc700 1 --2- 172.21.15.16:0/2257771773 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e241316c0 0x7f9e0c005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:41.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.433+0000 7f9e227fc700 1 -- 172.21.15.16:0/2257771773 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e0c003a00 msgr2=0x7f9e0c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:41.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.433+0000 7f9e22ffd700 1 --2- 172.21.15.16:0/2257771773 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e0c003a00 0x7f9e0c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:41.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.433+0000 7f9e227fc700 1 --2- 172.21.15.16:0/2257771773 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e0c003a00 0x7f9e0c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:41.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.433+0000 7f9e227fc700 1 -- 172.21.15.16:0/2257771773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e0c0035b0 con 0x7f9e0c004430 2023-09-30T22:32:41.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.433+0000 7f9e22ffd700 1 --2- 172.21.15.16:0/2257771773 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e0c003a00 0x7f9e0c003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:32:41.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.433+0000 7f9e237fe700 1 --2- 172.21.15.16:0/2257771773 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e241316c0 0x7f9e0c005220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:32:41.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.434+0000 7f9e227fc700 1 --2- 172.21.15.16:0/2257771773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e0c004430 0x7f9e0c0048f0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f9e2406deb0 tx=0x7f9e0c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b90aa575efc2011b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:41.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.434+0000 7f9e21ffb700 1 -- 172.21.15.16:0/2257771773 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e180093a0 con 0x7f9e0c004430 2023-09-30T22:32:41.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.434+0000 7f9e21ffb700 1 -- 172.21.15.16:0/2257771773 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e18011620 con 0x7f9e0c004430 2023-09-30T22:32:41.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.434+0000 7f9e21ffb700 1 -- 172.21.15.16:0/2257771773 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e180119b0 con 0x7f9e0c004430 2023-09-30T22:32:41.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.435+0000 7f9e29764700 1 -- 172.21.15.16:0/2257771773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e0c004430 msgr2=0x7f9e0c0048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:41.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.435+0000 7f9e29764700 1 --2- 172.21.15.16:0/2257771773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e0c004430 0x7f9e0c0048f0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f9e2406deb0 tx=0x7f9e0c005880 comp rx=0 tx=0).stop 2023-09-30T22:32:41.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.435+0000 7f9e237fe700 1 -- 172.21.15.16:0/2257771773 reap_dead start 2023-09-30T22:32:41.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.435+0000 7f9e29764700 1 -- 172.21.15.16:0/2257771773 shutdown_connections 2023-09-30T22:32:41.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.435+0000 7f9e29764700 1 -- 172.21.15.16:0/2257771773 >> 172.21.15.16:0/2257771773 conn(0x7f9e24066f50 msgr2=0x7f9e2409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:41.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.435+0000 7f9e29764700 1 -- 172.21.15.16:0/2257771773 shutdown_connections 2023-09-30T22:32:41.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.435+0000 7f9e29764700 1 -- 172.21.15.16:0/2257771773 wait complete. 2023-09-30T22:32:41.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.436+0000 7f9e29764700 1 Processor -- start 2023-09-30T22:32:41.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.436+0000 7f9e29764700 1 -- start start 2023-09-30T22:32:41.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.437+0000 7f9e29764700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e0c004430 0x7f9e241c8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:41.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.437+0000 7f9e29764700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e241c8a50 0x7f9e241c8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:41.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.437+0000 7f9e29764700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e241c5300 0x7f9e241c5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:41.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.437+0000 7f9e29764700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2409bfc0 con 0x7f9e241c5300 2023-09-30T22:32:41.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.437+0000 7f9e29764700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e241c9380 con 0x7f9e0c004430 2023-09-30T22:32:41.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.437+0000 7f9e29764700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9e241205d0 con 0x7f9e241c8a50 2023-09-30T22:32:41.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.437+0000 7f9e22ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e0c004430 0x7f9e241c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:41.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.437+0000 7f9e22ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e0c004430 0x7f9e241c8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:39438/0 (socket says 172.21.15.16:39438) 2023-09-30T22:32:41.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.437+0000 7f9e22ffd700 1 -- 172.21.15.16:0/3010862086 learned_addr learned my addr 172.21.15.16:0/3010862086 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:41.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.437+0000 7f9e237fe700 1 --2- 172.21.15.16:0/3010862086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e241c5300 0x7f9e241c5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:41.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.437+0000 7f9e227fc700 1 --2- 172.21.15.16:0/3010862086 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e241c8a50 0x7f9e241c8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:41.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.437+0000 7f9e22ffd700 1 -- 172.21.15.16:0/3010862086 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e241c8a50 msgr2=0x7f9e241c8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:41.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.437+0000 7f9e22ffd700 1 --2- 172.21.15.16:0/3010862086 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e241c8a50 0x7f9e241c8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:41.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.437+0000 7f9e22ffd700 1 -- 172.21.15.16:0/3010862086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e241c5300 msgr2=0x7f9e241c5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:41.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.437+0000 7f9e22ffd700 1 --2- 172.21.15.16:0/3010862086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e241c5300 0x7f9e241c5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:41.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.437+0000 7f9e22ffd700 1 -- 172.21.15.16:0/3010862086 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e2409ead0 con 0x7f9e0c004430 2023-09-30T22:32:41.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.437+0000 7f9e237fe700 1 --2- 172.21.15.16:0/3010862086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e241c5300 0x7f9e241c5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:32:41.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.437+0000 7f9e227fc700 1 --2- 172.21.15.16:0/3010862086 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e241c8a50 0x7f9e241c8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:32:41.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.438+0000 7f9e22ffd700 1 --2- 172.21.15.16:0/3010862086 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e0c004430 0x7f9e241c8510 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f9e14010980 tx=0x7f9e14010d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:41.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.438+0000 7f9e0b7fe700 1 -- 172.21.15.16:0/3010862086 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e14004220 con 0x7f9e0c004430 2023-09-30T22:32:41.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.438+0000 7f9e0b7fe700 1 -- 172.21.15.16:0/3010862086 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e14004380 con 0x7f9e0c004430 2023-09-30T22:32:41.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.438+0000 7f9e29764700 1 -- 172.21.15.16:0/3010862086 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e241201f0 con 0x7f9e0c004430 2023-09-30T22:32:41.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.438+0000 7f9e0b7fe700 1 -- 172.21.15.16:0/3010862086 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1400e870 con 0x7f9e0c004430 2023-09-30T22:32:41.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.438+0000 7f9e29764700 1 -- 172.21.15.16:0/3010862086 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e2411ff30 con 0x7f9e0c004430 2023-09-30T22:32:41.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.439+0000 7f9e0b7fe700 1 -- 172.21.15.16:0/3010862086 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9e1400e9d0 con 0x7f9e0c004430 2023-09-30T22:32:41.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.439+0000 7f9e0b7fe700 1 -- 172.21.15.16:0/3010862086 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f9e1403faf0 con 0x7f9e0c004430 2023-09-30T22:32:41.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.439+0000 7f9e29764700 1 -- 172.21.15.16:0/3010862086 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e24074ea0 con 0x7f9e0c004430 2023-09-30T22:32:41.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.445+0000 7f9e0b7fe700 1 -- 172.21.15.16:0/3010862086 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9e14024ba0 con 0x7f9e0c004430 2023-09-30T22:32:41.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.456+0000 7f52cf146700 1 Processor -- start 2023-09-30T22:32:41.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.456+0000 7f52cf146700 1 -- start start 2023-09-30T22:32:41.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.457+0000 7f52cf146700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f52c809d6f0 0x7f52c809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:41.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.457+0000 7f52cf146700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f52c809e410 0x7f52c8134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:41.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.457+0000 7f52cf146700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52c809b120 con 0x7f52c809e000 2023-09-30T22:32:41.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.457+0000 7f52cf146700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c806b460 con 0x7f52c809e410 2023-09-30T22:32:41.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.457+0000 7f52cf146700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f52c809a7e0 con 0x7f52c809d6f0 2023-09-30T22:32:41.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.457+0000 7f52ccee2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f52c809d6f0 0x7f52c809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:41.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.457+0000 7f52ccee2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f52c809d6f0 0x7f52c809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:39120/0 (socket says 172.21.15.16:39120) 2023-09-30T22:32:41.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.457+0000 7f52ccee2700 1 -- 172.21.15.16:0/744095708 learned_addr learned my addr 172.21.15.16:0/744095708 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:41.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.457+0000 7f52cd6e3700 1 --2- 172.21.15.16:0/744095708 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f52c809e410 0x7f52c8134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:41.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.457+0000 7f52ccee2700 1 -- 172.21.15.16:0/744095708 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f52c809e410 msgr2=0x7f52c8134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:41.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.457+0000 7f52ccee2700 1 --2- 172.21.15.16:0/744095708 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f52c809e410 0x7f52c8134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:41.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.457+0000 7f52ccee2700 1 -- 172.21.15.16:0/744095708 >> v1:172.21.15.16:6789/0 conn(0x7f52c809e000 legacy=0x7f52c81339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:41.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.457+0000 7f52ccee2700 1 -- 172.21.15.16:0/744095708 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c809a500 con 0x7f52c809d6f0 2023-09-30T22:32:41.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.458+0000 7f52ccee2700 1 --2- 172.21.15.16:0/744095708 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f52c809d6f0 0x7f52c809dac0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f52bc009950 tx=0x7f52bc009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2afd53540ee04ba server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:41.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.458+0000 7f52c77fe700 1 -- 172.21.15.16:0/744095708 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52bc004230 con 0x7f52c809d6f0 2023-09-30T22:32:41.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.458+0000 7f52c77fe700 1 -- 172.21.15.16:0/744095708 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52bc004390 con 0x7f52c809d6f0 2023-09-30T22:32:41.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.458+0000 7f52c77fe700 1 -- 172.21.15.16:0/744095708 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52bc0036a0 con 0x7f52c809d6f0 2023-09-30T22:32:41.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.458+0000 7f52cf146700 1 -- 172.21.15.16:0/744095708 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f52c809d6f0 msgr2=0x7f52c809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:41.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.458+0000 7f52cf146700 1 --2- 172.21.15.16:0/744095708 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f52c809d6f0 0x7f52c809dac0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f52bc009950 tx=0x7f52bc009c60 comp rx=0 tx=0).stop 2023-09-30T22:32:41.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.459+0000 7f52cf146700 1 -- 172.21.15.16:0/744095708 shutdown_connections 2023-09-30T22:32:41.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.459+0000 7f52cf146700 1 --2- 172.21.15.16:0/744095708 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f52c809e410 0x7f52c8134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:41.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.459+0000 7f52cf146700 1 --2- 172.21.15.16:0/744095708 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f52c809d6f0 0x7f52c809dac0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:41.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.459+0000 7f52cf146700 1 -- 172.21.15.16:0/744095708 >> 172.21.15.16:0/744095708 conn(0x7f52c8066f50 msgr2=0x7f52c809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:41.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.459+0000 7f52cf146700 1 -- 172.21.15.16:0/744095708 shutdown_connections 2023-09-30T22:32:41.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.459+0000 7f52cf146700 1 -- 172.21.15.16:0/744095708 wait complete. 2023-09-30T22:32:41.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.460+0000 7f52cf146700 1 Processor -- start 2023-09-30T22:32:41.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.460+0000 7f52cf146700 1 -- start start 2023-09-30T22:32:41.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.460+0000 7f52cf146700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f52c809d6f0 0x7f52c8122670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:41.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.460+0000 7f52cf146700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f52c809e000 0x7f52c8122bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:41.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.460+0000 7f52cf146700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52c809e410 0x7f52c81230f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:41.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.460+0000 7f52cf146700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52c809c710 con 0x7f52c809e410 2023-09-30T22:32:41.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.460+0000 7f52cf146700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52c80a1d00 con 0x7f52c809e000 2023-09-30T22:32:41.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.460+0000 7f52cf146700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f52c8125010 con 0x7f52c809d6f0 2023-09-30T22:32:41.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.460+0000 7f52c7fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f52c809e000 0x7f52c8122bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:41.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.461+0000 7f52c7fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f52c809e000 0x7f52c8122bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:39452/0 (socket says 172.21.15.16:39452) 2023-09-30T22:32:41.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.461+0000 7f52c7fff700 1 -- 172.21.15.16:0/3026351547 learned_addr learned my addr 172.21.15.16:0/3026351547 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:41.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.461+0000 7f52ccee2700 1 --2- 172.21.15.16:0/3026351547 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f52c809d6f0 0x7f52c8122670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:41.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.461+0000 7f52cd6e3700 1 --2- 172.21.15.16:0/3026351547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52c809e410 0x7f52c81230f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:41.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.461+0000 7f52ccee2700 1 -- 172.21.15.16:0/3026351547 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f52c809e000 msgr2=0x7f52c8122bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:41.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.461+0000 7f52ccee2700 1 --2- 172.21.15.16:0/3026351547 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f52c809e000 0x7f52c8122bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:41.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.461+0000 7f52ccee2700 1 -- 172.21.15.16:0/3026351547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52c809e410 msgr2=0x7f52c81230f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:41.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.461+0000 7f52ccee2700 1 --2- 172.21.15.16:0/3026351547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52c809e410 0x7f52c81230f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:41.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.461+0000 7f52ccee2700 1 -- 172.21.15.16:0/3026351547 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52bc009550 con 0x7f52c809d6f0 2023-09-30T22:32:41.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.461+0000 7f52cd6e3700 1 --2- 172.21.15.16:0/3026351547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52c809e410 0x7f52c81230f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:32:41.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.461+0000 7f52ccee2700 1 --2- 172.21.15.16:0/3026351547 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f52c809d6f0 0x7f52c8122670 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f52bc015000 tx=0x7f52bc004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:41.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.461+0000 7f52c57fa700 1 -- 172.21.15.16:0/3026351547 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52bc003d00 con 0x7f52c809d6f0 2023-09-30T22:32:41.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.461+0000 7f52c57fa700 1 -- 172.21.15.16:0/3026351547 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52bc01f070 con 0x7f52c809d6f0 2023-09-30T22:32:41.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.461+0000 7f52c57fa700 1 -- 172.21.15.16:0/3026351547 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52bc011870 con 0x7f52c809d6f0 2023-09-30T22:32:41.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.461+0000 7f52cf146700 1 -- 172.21.15.16:0/3026351547 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52c8125890 con 0x7f52c809d6f0 2023-09-30T22:32:41.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.461+0000 7f52cf146700 1 -- 172.21.15.16:0/3026351547 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52c8126890 con 0x7f52c809d6f0 2023-09-30T22:32:41.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.462+0000 7f52c57fa700 1 -- 172.21.15.16:0/3026351547 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f52bc021070 con 0x7f52c809d6f0 2023-09-30T22:32:41.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.463+0000 7f52c57fa700 1 -- 172.21.15.16:0/3026351547 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f52bc040be0 con 0x7f52c809d6f0 2023-09-30T22:32:41.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.463+0000 7f52cf146700 1 -- 172.21.15.16:0/3026351547 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52c8074ea0 con 0x7f52c809d6f0 2023-09-30T22:32:41.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.468+0000 7f52c57fa700 1 -- 172.21.15.16:0/3026351547 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f52bc003e60 con 0x7f52c809d6f0 2023-09-30T22:32:41.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.567+0000 7f8b21af7700 1 -- 172.21.15.16:0/276292402 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8b1c120d10 con 0x7f8b1c1338a0 2023-09-30T22:32:41.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.568+0000 7f8b03fff700 1 -- 172.21.15.16:0/276292402 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8b1402b090 con 0x7f8b1c1338a0 2023-09-30T22:32:41.568 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:32:41.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.570+0000 7f8b21af7700 1 -- 172.21.15.16:0/276292402 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8b1c1338a0 msgr2=0x7f8b1c11e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:41.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.570+0000 7f8b21af7700 1 --2- 172.21.15.16:0/276292402 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8b1c1338a0 0x7f8b1c11e0d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f8b1400efc0 tx=0x7f8b1400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:32:41.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.570+0000 7f8b21af7700 1 -- 172.21.15.16:0/276292402 shutdown_connections 2023-09-30T22:32:41.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.570+0000 7f8b21af7700 1 --2- 172.21.15.16:0/276292402 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8b1c1338a0 0x7f8b1c11e0d0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:41.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.570+0000 7f8b21af7700 1 --2- 172.21.15.16:0/276292402 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8b1c09de50 0x7f8b1c11d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:41.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.570+0000 7f8b21af7700 1 --2- 172.21.15.16:0/276292402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b1c09e760 0x7f8b1c11db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:41.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.570+0000 7f8b21af7700 1 -- 172.21.15.16:0/276292402 >> 172.21.15.16:0/276292402 conn(0x7f8b1c0a2aa0 msgr2=0x7f8b1c1bae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:41.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.570+0000 7f8b21af7700 1 -- 172.21.15.16:0/276292402 shutdown_connections 2023-09-30T22:32:41.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.570+0000 7f8b21af7700 1 -- 172.21.15.16:0/276292402 wait complete. 2023-09-30T22:32:41.585 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:32:41.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.612+0000 7f662908a700 1 -- 172.21.15.16:0/268299589 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f6604005590 con 0x7f662406e570 2023-09-30T22:32:41.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.613+0000 7f660b7fe700 1 -- 172.21.15.16:0/268299589 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6614017030 con 0x7f662406e570 2023-09-30T22:32:41.613 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:32:41.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.615+0000 7f662908a700 1 -- 172.21.15.16:0/268299589 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f662406e570 msgr2=0x7f66241214e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:41.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.615+0000 7f662908a700 1 --2- 172.21.15.16:0/268299589 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f662406e570 0x7f66241214e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f661400c550 tx=0x7f661400c860 comp rx=0 tx=0).stop 2023-09-30T22:32:41.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.615+0000 7f662908a700 1 -- 172.21.15.16:0/268299589 shutdown_connections 2023-09-30T22:32:41.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.615+0000 7f662908a700 1 --2- 172.21.15.16:0/268299589 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6624121f60 0x7f66241cca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:41.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.615+0000 7f662908a700 1 --2- 172.21.15.16:0/268299589 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f662406e570 0x7f66241214e0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:41.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.615+0000 7f662908a700 1 --2- 172.21.15.16:0/268299589 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f662406f050 0x7f6624121a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:41.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.615+0000 7f662908a700 1 -- 172.21.15.16:0/268299589 >> 172.21.15.16:0/268299589 conn(0x7f66240a2aa0 msgr2=0x7f662409a560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:41.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.615+0000 7f662908a700 1 -- 172.21.15.16:0/268299589 shutdown_connections 2023-09-30T22:32:41.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.615+0000 7f662908a700 1 -- 172.21.15.16:0/268299589 wait complete. 2023-09-30T22:32:41.631 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:32:41.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.670+0000 7f9e29764700 1 -- 172.21.15.16:0/3010862086 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f9e2411edb0 con 0x7f9e0c004430 2023-09-30T22:32:41.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.670+0000 7f9e0b7fe700 1 -- 172.21.15.16:0/3010862086 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9e1402b090 con 0x7f9e0c004430 2023-09-30T22:32:41.671 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:32:41.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.672+0000 7f9e29764700 1 -- 172.21.15.16:0/3010862086 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e0c004430 msgr2=0x7f9e241c8510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:41.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.672+0000 7f9e29764700 1 --2- 172.21.15.16:0/3010862086 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e0c004430 0x7f9e241c8510 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f9e14010980 tx=0x7f9e14010d40 comp rx=0 tx=0).stop 2023-09-30T22:32:41.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.672+0000 7f9e29764700 1 -- 172.21.15.16:0/3010862086 shutdown_connections 2023-09-30T22:32:41.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.672+0000 7f9e29764700 1 --2- 172.21.15.16:0/3010862086 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e0c004430 0x7f9e241c8510 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:41.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.672+0000 7f9e29764700 1 --2- 172.21.15.16:0/3010862086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e241c5300 0x7f9e241c5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:41.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.672+0000 7f9e29764700 1 --2- 172.21.15.16:0/3010862086 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e241c8a50 0x7f9e241c8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:41.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.672+0000 7f9e29764700 1 -- 172.21.15.16:0/3010862086 >> 172.21.15.16:0/3010862086 conn(0x7f9e24066f50 msgr2=0x7f9e2412db20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:41.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.673+0000 7f9e29764700 1 -- 172.21.15.16:0/3010862086 shutdown_connections 2023-09-30T22:32:41.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.673+0000 7f9e29764700 1 -- 172.21.15.16:0/3010862086 wait complete. 2023-09-30T22:32:41.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.678+0000 7f52cf146700 1 -- 172.21.15.16:0/3026351547 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f52c806c360 con 0x7f52c809d6f0 2023-09-30T22:32:41.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.678+0000 7f52c57fa700 1 -- 172.21.15.16:0/3026351547 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f52bc0193c0 con 0x7f52c809d6f0 2023-09-30T22:32:41.679 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:32:41.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.681+0000 7f52cf146700 1 -- 172.21.15.16:0/3026351547 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f52c809d6f0 msgr2=0x7f52c8122670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:41.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.681+0000 7f52cf146700 1 --2- 172.21.15.16:0/3026351547 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f52c809d6f0 0x7f52c8122670 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f52bc015000 tx=0x7f52bc004750 comp rx=0 tx=0).stop 2023-09-30T22:32:41.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.681+0000 7f52cf146700 1 -- 172.21.15.16:0/3026351547 shutdown_connections 2023-09-30T22:32:41.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.681+0000 7f52cf146700 1 --2- 172.21.15.16:0/3026351547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52c809e410 0x7f52c81230f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:41.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.681+0000 7f52cf146700 1 --2- 172.21.15.16:0/3026351547 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f52c809d6f0 0x7f52c8122670 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:41.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.681+0000 7f52cf146700 1 --2- 172.21.15.16:0/3026351547 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f52c809e000 0x7f52c8122bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:41.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.681+0000 7f52cf146700 1 -- 172.21.15.16:0/3026351547 >> 172.21.15.16:0/3026351547 conn(0x7f52c8066f50 msgr2=0x7f52c80a2360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:41.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.681+0000 7f52cf146700 1 -- 172.21.15.16:0/3026351547 shutdown_connections 2023-09-30T22:32:41.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:41.681+0000 7f52cf146700 1 -- 172.21.15.16:0/3026351547 wait complete. 2023-09-30T22:32:41.688 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:32:41.695 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:32:42.586 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:32:42.632 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:32:42.689 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:32:42.696 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:32:42.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.735+0000 7fd9c580b700 1 Processor -- start 2023-09-30T22:32:42.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.735+0000 7fd9c580b700 1 -- start start 2023-09-30T22:32:42.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.736+0000 7fd9c580b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd9c0133510 0x7fd9c01358f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:42.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.736+0000 7fd9c580b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd9c0135e30 0x7fd9c0138210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:42.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.736+0000 7fd9c580b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9c0130260 con 0x7fd9c0130670 2023-09-30T22:32:42.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.736+0000 7fd9c580b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c009a350 con 0x7fd9c0135e30 2023-09-30T22:32:42.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.736+0000 7fd9c580b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c0071ff0 con 0x7fd9c0133510 2023-09-30T22:32:42.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.737+0000 7fd9be7fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd9c0133510 0x7fd9c01358f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:42.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.737+0000 7fd9be7fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd9c0133510 0x7fd9c01358f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:39126/0 (socket says 172.21.15.16:39126) 2023-09-30T22:32:42.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.737+0000 7fd9be7fc700 1 -- 172.21.15.16:0/157813974 learned_addr learned my addr 172.21.15.16:0/157813974 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:42.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.737+0000 7fd9bf7fe700 1 --2- 172.21.15.16:0/157813974 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd9c0135e30 0x7fd9c0138210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:42.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.737+0000 7fd9be7fc700 1 -- 172.21.15.16:0/157813974 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd9c0135e30 msgr2=0x7fd9c0138210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:42.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.737+0000 7fd9be7fc700 1 --2- 172.21.15.16:0/157813974 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd9c0135e30 0x7fd9c0138210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:42.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.737+0000 7fd9be7fc700 1 -- 172.21.15.16:0/157813974 >> v1:172.21.15.16:6789/0 conn(0x7fd9c0130670 legacy=0x7fd9c0130a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:42.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.737+0000 7fd9be7fc700 1 -- 172.21.15.16:0/157813974 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c009ecf0 con 0x7fd9c0133510 2023-09-30T22:32:42.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.737+0000 7fd9be7fc700 1 --2- 172.21.15.16:0/157813974 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd9c0133510 0x7fd9c01358f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4009920 tx=0x7fd9b4009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5dab5d05c45bc08 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:42.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.737+0000 7fd9bdffb700 1 -- 172.21.15.16:0/157813974 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b4011830 con 0x7fd9c0133510 2023-09-30T22:32:42.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.738+0000 7fd9bdffb700 1 -- 172.21.15.16:0/157813974 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b4011990 con 0x7fd9c0133510 2023-09-30T22:32:42.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.738+0000 7fd9bdffb700 1 -- 172.21.15.16:0/157813974 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b4011d00 con 0x7fd9c0133510 2023-09-30T22:32:42.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.738+0000 7fd9c580b700 1 -- 172.21.15.16:0/157813974 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd9c0133510 msgr2=0x7fd9c01358f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:42.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.738+0000 7fd9c580b700 1 --2- 172.21.15.16:0/157813974 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd9c0133510 0x7fd9c01358f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4009920 tx=0x7fd9b4009c30 comp rx=0 tx=0).stop 2023-09-30T22:32:42.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.738+0000 7fd9c580b700 1 -- 172.21.15.16:0/157813974 shutdown_connections 2023-09-30T22:32:42.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.738+0000 7fd9c580b700 1 --2- 172.21.15.16:0/157813974 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd9c0135e30 0x7fd9c0138210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:42.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.738+0000 7fd9c580b700 1 --2- 172.21.15.16:0/157813974 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd9c0133510 0x7fd9c01358f0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:42.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.738+0000 7fd9c580b700 1 -- 172.21.15.16:0/157813974 >> 172.21.15.16:0/157813974 conn(0x7fd9c00a2aa0 msgr2=0x7fd9c00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:42.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.739+0000 7fd9c580b700 1 -- 172.21.15.16:0/157813974 shutdown_connections 2023-09-30T22:32:42.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.739+0000 7fd9c580b700 1 -- 172.21.15.16:0/157813974 wait complete. 2023-09-30T22:32:42.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.740+0000 7fd9c580b700 1 Processor -- start 2023-09-30T22:32:42.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.740+0000 7fd9c580b700 1 -- start start 2023-09-30T22:32:42.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.741+0000 7fd9c580b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd9c0130670 0x7fd9c009dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:42.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.741+0000 7fd9c580b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9c0133510 0x7fd9c009e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:42.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.741+0000 7fd9c580b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd9c0135e30 0x7fd9c0139760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:42.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.741+0000 7fd9c580b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c009a810 con 0x7fd9c0133510 2023-09-30T22:32:42.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.741+0000 7fd9c580b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c0138d60 con 0x7fd9c0130670 2023-09-30T22:32:42.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.741+0000 7fd9c580b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c009aa40 con 0x7fd9c0135e30 2023-09-30T22:32:42.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.741+0000 7fd9beffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd9c0130670 0x7fd9c009dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:42.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.741+0000 7fd9beffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd9c0130670 0x7fd9c009dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:39474/0 (socket says 172.21.15.16:39474) 2023-09-30T22:32:42.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.741+0000 7fd9beffd700 1 -- 172.21.15.16:0/2967632014 learned_addr learned my addr 172.21.15.16:0/2967632014 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:42.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.741+0000 7fd9bf7fe700 1 --2- 172.21.15.16:0/2967632014 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd9c0135e30 0x7fd9c0139760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:42.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.742+0000 7fd9be7fc700 1 --2- 172.21.15.16:0/2967632014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9c0133510 0x7fd9c009e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:42.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.742+0000 7fd9beffd700 1 -- 172.21.15.16:0/2967632014 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd9c0135e30 msgr2=0x7fd9c0139760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:42.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.742+0000 7fd9beffd700 1 --2- 172.21.15.16:0/2967632014 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd9c0135e30 0x7fd9c0139760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:42.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.742+0000 7fd9beffd700 1 -- 172.21.15.16:0/2967632014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9c0133510 msgr2=0x7fd9c009e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:42.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.742+0000 7fd9beffd700 1 --2- 172.21.15.16:0/2967632014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9c0133510 0x7fd9c009e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:42.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.742+0000 7fd9beffd700 1 -- 172.21.15.16:0/2967632014 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b4009550 con 0x7fd9c0130670 2023-09-30T22:32:42.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.743+0000 7fd9beffd700 1 --2- 172.21.15.16:0/2967632014 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd9c0130670 0x7fd9c009dbe0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd9b000c4e0 tx=0x7fd9b000c7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:42.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.743+0000 7fd9a7fff700 1 -- 172.21.15.16:0/2967632014 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b00052d0 con 0x7fd9c0130670 2023-09-30T22:32:42.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.743+0000 7fd9a7fff700 1 -- 172.21.15.16:0/2967632014 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b0005430 con 0x7fd9c0130670 2023-09-30T22:32:42.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.743+0000 7fd9a7fff700 1 -- 172.21.15.16:0/2967632014 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b00038c0 con 0x7fd9c0130670 2023-09-30T22:32:42.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.743+0000 7fd9c580b700 1 -- 172.21.15.16:0/2967632014 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9c011cda0 con 0x7fd9c0130670 2023-09-30T22:32:42.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.743+0000 7fd9c580b700 1 -- 172.21.15.16:0/2967632014 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9c0138330 con 0x7fd9c0130670 2023-09-30T22:32:42.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.744+0000 7fd9a7fff700 1 -- 172.21.15.16:0/2967632014 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd9b0003a20 con 0x7fd9c0130670 2023-09-30T22:32:42.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.744+0000 7fd9a7fff700 1 -- 172.21.15.16:0/2967632014 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd9b003eb70 con 0x7fd9c0130670 2023-09-30T22:32:42.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.744+0000 7fd9c580b700 1 -- 172.21.15.16:0/2967632014 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9c0070e70 con 0x7fd9c0130670 2023-09-30T22:32:42.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.750+0000 7fd9a7fff700 1 -- 172.21.15.16:0/2967632014 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd9b0017070 con 0x7fd9c0130670 2023-09-30T22:32:42.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.789+0000 7f97e6914700 1 Processor -- start 2023-09-30T22:32:42.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.789+0000 7f97e6914700 1 -- start start 2023-09-30T22:32:42.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.789+0000 7f97e6914700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f97e009d6f0 0x7f97e009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:42.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.790+0000 7f97e6914700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f97e009e410 0x7f97e0130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:42.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.790+0000 7f97e6914700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97e009f130 con 0x7f97e009e000 2023-09-30T22:32:42.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.790+0000 7f97e6914700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97e009f520 con 0x7f97e009d6f0 2023-09-30T22:32:42.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.790+0000 7f97e6914700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f97e009e930 con 0x7f97e009e410 2023-09-30T22:32:42.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.790+0000 7f97dffff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f97e009d6f0 0x7f97e009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:42.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.790+0000 7f97e4eb1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f97e009e410 0x7f97e0130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:42.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.790+0000 7f97dffff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f97e009d6f0 0x7f97e009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35952/0 (socket says 172.21.15.16:35952) 2023-09-30T22:32:42.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.790+0000 7f97e4eb1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f97e009e410 0x7f97e0130d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:39154/0 (socket says 172.21.15.16:39154) 2023-09-30T22:32:42.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.790+0000 7f97df7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f97e009e000 0x7f97e0130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53664/0 (socket says 172.21.15.16:53664) 2023-09-30T22:32:42.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.790+0000 7f97dffff700 1 -- 172.21.15.16:0/792814571 learned_addr learned my addr 172.21.15.16:0/792814571 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:42.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.790+0000 7f97e4eb1700 1 -- 172.21.15.16:0/792814571 >> v2:172.21.15.16:3300/0 conn(0x7f97e009d6f0 msgr2=0x7f97e009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:42.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.790+0000 7f97e4eb1700 1 --2- 172.21.15.16:0/792814571 >> v2:172.21.15.16:3300/0 conn(0x7f97e009d6f0 0x7f97e009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:42.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.790+0000 7f97e4eb1700 1 -- 172.21.15.16:0/792814571 >> v1:172.21.15.16:6789/0 conn(0x7f97e009e000 legacy=0x7f97e0130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:42.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.790+0000 7f97e4eb1700 1 -- 172.21.15.16:0/792814571 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e009b2d0 con 0x7f97e009e410 2023-09-30T22:32:42.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.790+0000 7f97dffff700 1 --2- 172.21.15.16:0/792814571 >> v2:172.21.15.16:3300/0 conn(0x7f97e009d6f0 0x7f97e009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:42.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.790+0000 7f97e4eb1700 1 --2- 172.21.15.16:0/792814571 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f97e009e410 0x7f97e0130d70 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f97d800bfa0 tx=0x7f97d800d330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40fee39f3829723a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:42.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.791+0000 7f97deffd700 1 -- 172.21.15.16:0/792814571 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d8010070 con 0x7f97e009e410 2023-09-30T22:32:42.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.791+0000 7f97deffd700 1 -- 172.21.15.16:0/792814571 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97d800b2e0 con 0x7f97e009e410 2023-09-30T22:32:42.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.791+0000 7f97deffd700 1 -- 172.21.15.16:0/792814571 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d800dc40 con 0x7f97e009e410 2023-09-30T22:32:42.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.791+0000 7f97e6914700 1 -- 172.21.15.16:0/792814571 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f97e009e410 msgr2=0x7f97e0130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:42.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.791+0000 7f97e6914700 1 --2- 172.21.15.16:0/792814571 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f97e009e410 0x7f97e0130d70 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f97d800bfa0 tx=0x7f97d800d330 comp rx=0 tx=0).stop 2023-09-30T22:32:42.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.791+0000 7f97e6914700 1 -- 172.21.15.16:0/792814571 shutdown_connections 2023-09-30T22:32:42.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.792+0000 7f97e6914700 1 --2- 172.21.15.16:0/792814571 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f97e009e410 0x7f97e0130d70 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:42.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.792+0000 7f97e6914700 1 --2- 172.21.15.16:0/792814571 >> v2:172.21.15.16:3300/0 conn(0x7f97e009d6f0 0x7f97e009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:42.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.792+0000 7f97e6914700 1 -- 172.21.15.16:0/792814571 >> 172.21.15.16:0/792814571 conn(0x7f97e0066f50 msgr2=0x7f97e009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:42.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.792+0000 7f97e6914700 1 -- 172.21.15.16:0/792814571 shutdown_connections 2023-09-30T22:32:42.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.792+0000 7f97e6914700 1 -- 172.21.15.16:0/792814571 wait complete. 2023-09-30T22:32:42.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.793+0000 7f97e6914700 1 Processor -- start 2023-09-30T22:32:42.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.793+0000 7f97e6914700 1 -- start start 2023-09-30T22:32:42.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.794+0000 7f97e6914700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f97e009d6f0 0x7f97e011de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:42.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.794+0000 7f97e6914700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f97e009e000 0x7f97e011e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:42.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.794+0000 7f97e6914700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97e009e410 0x7f97e011e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:42.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.794+0000 7f97e6914700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e0121940 con 0x7f97e009e410 2023-09-30T22:32:42.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.794+0000 7f97e6914700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97e0120050 con 0x7f97e009e000 2023-09-30T22:32:42.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.794+0000 7f97e6914700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f97e0120d10 con 0x7f97e009d6f0 2023-09-30T22:32:42.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.794+0000 7f97df7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f97e009e000 0x7f97e011e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:42.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.794+0000 7f97df7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f97e009e000 0x7f97e011e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:39484/0 (socket says 172.21.15.16:39484) 2023-09-30T22:32:42.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.794+0000 7f97df7fe700 1 -- 172.21.15.16:0/1204575445 learned_addr learned my addr 172.21.15.16:0/1204575445 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:42.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.794+0000 7f97dffff700 1 --2- 172.21.15.16:0/1204575445 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f97e009d6f0 0x7f97e011de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:42.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.795+0000 7f97e4eb1700 1 --2- 172.21.15.16:0/1204575445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97e009e410 0x7f97e011e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:42.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.795+0000 7f97df7fe700 1 -- 172.21.15.16:0/1204575445 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f97e009d6f0 msgr2=0x7f97e011de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:42.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.795+0000 7f97df7fe700 1 --2- 172.21.15.16:0/1204575445 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f97e009d6f0 0x7f97e011de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:42.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.795+0000 7f97df7fe700 1 -- 172.21.15.16:0/1204575445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97e009e410 msgr2=0x7f97e011e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:42.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.795+0000 7f97df7fe700 1 --2- 172.21.15.16:0/1204575445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97e009e410 0x7f97e011e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:42.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.795+0000 7f97df7fe700 1 -- 172.21.15.16:0/1204575445 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97d0009550 con 0x7f97e009e000 2023-09-30T22:32:42.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.795+0000 7f97dffff700 1 --2- 172.21.15.16:0/1204575445 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f97e009d6f0 0x7f97e011de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:42.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.795+0000 7f97df7fe700 1 --2- 172.21.15.16:0/1204575445 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f97e009e000 0x7f97e011e360 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f97d400c550 tx=0x7f97d400c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:42.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.795+0000 7f97dcff9700 1 -- 172.21.15.16:0/1204575445 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d4005cb0 con 0x7f97e009e000 2023-09-30T22:32:42.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.795+0000 7f97dcff9700 1 -- 172.21.15.16:0/1204575445 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97d4005e10 con 0x7f97e009e000 2023-09-30T22:32:42.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.795+0000 7f97dcff9700 1 -- 172.21.15.16:0/1204575445 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d40072d0 con 0x7f97e009e000 2023-09-30T22:32:42.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.795+0000 7f97e6914700 1 -- 172.21.15.16:0/1204575445 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97d800bc80 con 0x7f97e009e000 2023-09-30T22:32:42.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.796+0000 7f97e6914700 1 -- 172.21.15.16:0/1204575445 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97e0072500 con 0x7f97e009e000 2023-09-30T22:32:42.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.796+0000 7f97dcff9700 1 -- 172.21.15.16:0/1204575445 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f97d4007430 con 0x7f97e009e000 2023-09-30T22:32:42.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.797+0000 7f97dcff9700 1 -- 172.21.15.16:0/1204575445 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f97d403ebe0 con 0x7f97e009e000 2023-09-30T22:32:42.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.797+0000 7f97e6914700 1 -- 172.21.15.16:0/1204575445 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97e01214c0 con 0x7f97e009e000 2023-09-30T22:32:42.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.802+0000 7f97dcff9700 1 -- 172.21.15.16:0/1204575445 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f97e01214c0 con 0x7f97e009e000 2023-09-30T22:32:42.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.839+0000 7f8cb8254700 1 Processor -- start 2023-09-30T22:32:42.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.839+0000 7f8cb8254700 1 -- start start 2023-09-30T22:32:42.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.840+0000 7f8cb8254700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8cb00a0a30 0x7f8cb00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:42.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.840+0000 7f8cb8254700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8cb0134170 0x7f8cb013bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:42.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.840+0000 7f8cb8254700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cb009edd0 con 0x7f8cb00a1340 2023-09-30T22:32:42.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.840+0000 7f8cb8254700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb009e1f0 con 0x7f8cb00a0a30 2023-09-30T22:32:42.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.840+0000 7f8cb8254700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb009dc50 con 0x7f8cb0134170 2023-09-30T22:32:42.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.840+0000 7f8cb5ff0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8cb00a0a30 0x7f8cb00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:42.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.840+0000 7f8cb57ef700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8cb00a1340 0x7f8cb0133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53670/0 (socket says 172.21.15.16:53670) 2023-09-30T22:32:42.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.840+0000 7f8cb57ef700 1 -- 172.21.15.16:0/3617980961 learned_addr learned my addr 172.21.15.16:0/3617980961 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:42.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.840+0000 7f8cb67f1700 1 --2- 172.21.15.16:0/3617980961 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8cb0134170 0x7f8cb013bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:42.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.840+0000 7f8cb5ff0700 1 -- 172.21.15.16:0/3617980961 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8cb0134170 msgr2=0x7f8cb013bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:42.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.840+0000 7f8cb5ff0700 1 --2- 172.21.15.16:0/3617980961 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8cb0134170 0x7f8cb013bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:42.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.840+0000 7f8cb5ff0700 1 -- 172.21.15.16:0/3617980961 >> v1:172.21.15.16:6789/0 conn(0x7f8cb00a1340 legacy=0x7f8cb0133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:42.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.840+0000 7f8cb5ff0700 1 -- 172.21.15.16:0/3617980961 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cb00a1fe0 con 0x7f8cb00a0a30 2023-09-30T22:32:42.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.841+0000 7f8cb5ff0700 1 --2- 172.21.15.16:0/3617980961 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8cb00a0a30 0x7f8cb00a0e00 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f8cac009870 tx=0x7f8cac009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ce8ab310adc4087 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:42.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.841+0000 7f8cb4fee700 1 -- 172.21.15.16:0/3617980961 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cac011830 con 0x7f8cb00a0a30 2023-09-30T22:32:42.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.841+0000 7f8cb4fee700 1 -- 172.21.15.16:0/3617980961 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cac011990 con 0x7f8cb00a0a30 2023-09-30T22:32:42.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.841+0000 7f8cb4fee700 1 -- 172.21.15.16:0/3617980961 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cac011d20 con 0x7f8cb00a0a30 2023-09-30T22:32:42.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.842+0000 7f8cb8254700 1 -- 172.21.15.16:0/3617980961 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8cb00a0a30 msgr2=0x7f8cb00a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:42.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.842+0000 7f8cb8254700 1 --2- 172.21.15.16:0/3617980961 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8cb00a0a30 0x7f8cb00a0e00 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f8cac009870 tx=0x7f8cac009b80 comp rx=0 tx=0).stop 2023-09-30T22:32:42.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.842+0000 7f8cb8254700 1 -- 172.21.15.16:0/3617980961 shutdown_connections 2023-09-30T22:32:42.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.842+0000 7f8cb8254700 1 --2- 172.21.15.16:0/3617980961 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8cb0134170 0x7f8cb013bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:42.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.842+0000 7f8cb8254700 1 --2- 172.21.15.16:0/3617980961 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8cb00a0a30 0x7f8cb00a0e00 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:42.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.842+0000 7f8cb8254700 1 -- 172.21.15.16:0/3617980961 >> 172.21.15.16:0/3617980961 conn(0x7f8cb00a6550 msgr2=0x7f8cb00a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:42.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.842+0000 7f8cb8254700 1 -- 172.21.15.16:0/3617980961 shutdown_connections 2023-09-30T22:32:42.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.842+0000 7f8cb8254700 1 -- 172.21.15.16:0/3617980961 wait complete. 2023-09-30T22:32:42.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.843+0000 7f8cb8254700 1 Processor -- start 2023-09-30T22:32:42.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.843+0000 7f8cb8254700 1 -- start start 2023-09-30T22:32:42.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.843+0000 7f8cb8254700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8cb00a0a30 0x7f8cb0123200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:42.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.843+0000 7f8cb8254700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cb00a1340 0x7f8cb0123740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:42.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.844+0000 7f8cb8254700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8cb0134170 0x7f8cb01c8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:42.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.844+0000 7f8cb8254700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb0125f40 con 0x7f8cb00a1340 2023-09-30T22:32:42.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.844+0000 7f8cb8254700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb0124dd0 con 0x7f8cb0134170 2023-09-30T22:32:42.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.844+0000 7f8cb8254700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb0122e10 con 0x7f8cb00a0a30 2023-09-30T22:32:42.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.844+0000 7f8cb67f1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8cb0134170 0x7f8cb01c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:42.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.844+0000 7f8cb67f1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8cb0134170 0x7f8cb01c8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:39504/0 (socket says 172.21.15.16:39504) 2023-09-30T22:32:42.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.844+0000 7f8cb67f1700 1 -- 172.21.15.16:0/902463689 learned_addr learned my addr 172.21.15.16:0/902463689 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:42.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.844+0000 7f8cb57ef700 1 --2- 172.21.15.16:0/902463689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cb00a1340 0x7f8cb0123740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:42.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.844+0000 7f8cb5ff0700 1 --2- 172.21.15.16:0/902463689 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8cb00a0a30 0x7f8cb0123200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:42.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.844+0000 7f8cb67f1700 1 -- 172.21.15.16:0/902463689 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8cb00a0a30 msgr2=0x7f8cb0123200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:42.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.844+0000 7f8cb67f1700 1 --2- 172.21.15.16:0/902463689 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8cb00a0a30 0x7f8cb0123200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:42.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.844+0000 7f8cb67f1700 1 -- 172.21.15.16:0/902463689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cb00a1340 msgr2=0x7f8cb0123740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:42.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.844+0000 7f8cb67f1700 1 --2- 172.21.15.16:0/902463689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cb00a1340 0x7f8cb0123740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:42.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.844+0000 7f8cb67f1700 1 -- 172.21.15.16:0/902463689 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cac0094a0 con 0x7f8cb0134170 2023-09-30T22:32:42.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.844+0000 7f8cb67f1700 1 --2- 172.21.15.16:0/902463689 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8cb0134170 0x7f8cb01c8450 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8ca800efc0 tx=0x7f8ca800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:42.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.845+0000 7f8ca67fc700 1 -- 172.21.15.16:0/902463689 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca8009ea0 con 0x7f8cb0134170 2023-09-30T22:32:42.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.845+0000 7f8cb8254700 1 -- 172.21.15.16:0/902463689 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cb00a25e0 con 0x7f8cb0134170 2023-09-30T22:32:42.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.845+0000 7f8ca67fc700 1 -- 172.21.15.16:0/902463689 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ca800f040 con 0x7f8cb0134170 2023-09-30T22:32:42.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.845+0000 7f8ca67fc700 1 -- 172.21.15.16:0/902463689 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca8014480 con 0x7f8cb0134170 2023-09-30T22:32:42.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.845+0000 7f8cb8254700 1 -- 172.21.15.16:0/902463689 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cb01261c0 con 0x7f8cb0134170 2023-09-30T22:32:42.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.846+0000 7f8cb8254700 1 -- 172.21.15.16:0/902463689 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cb0125d90 con 0x7f8cb0134170 2023-09-30T22:32:42.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.847+0000 7f8ca67fc700 1 -- 172.21.15.16:0/902463689 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8ca80145e0 con 0x7f8cb0134170 2023-09-30T22:32:42.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.847+0000 7f8ca67fc700 1 -- 172.21.15.16:0/902463689 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f8ca8040570 con 0x7f8cb0134170 2023-09-30T22:32:42.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.851+0000 7f8ca67fc700 1 -- 172.21.15.16:0/902463689 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8ca8012730 con 0x7f8cb0134170 2023-09-30T22:32:42.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.896+0000 7f7aabf48700 1 Processor -- start 2023-09-30T22:32:42.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.896+0000 7f7aabf48700 1 -- start start 2023-09-30T22:32:42.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.897+0000 7f7aabf48700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7aa409e930 0x7f7aa409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:42.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.897+0000 7f7aabf48700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7aa409f240 0x7f7aa41317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:42.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.897+0000 7f7aabf48700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7aa409f650 0x7f7aa4131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:42.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.897+0000 7f7aabf48700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7aa4070940 con 0x7f7aa409e930 2023-09-30T22:32:42.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.897+0000 7f7aabf48700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa406fd60 con 0x7f7aa409f650 2023-09-30T22:32:42.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.897+0000 7f7aabf48700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa406fa80 con 0x7f7aa409f240 2023-09-30T22:32:42.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.897+0000 7f7aa9ce4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7aa409e930 0x7f7aa409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:42.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.897+0000 7f7aa9ce4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7aa409e930 0x7f7aa409ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35970/0 (socket says 172.21.15.16:35970) 2023-09-30T22:32:42.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.897+0000 7f7aa9ce4700 1 -- 172.21.15.16:0/1448629994 learned_addr learned my addr 172.21.15.16:0/1448629994 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:42.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.897+0000 7f7aa94e3700 1 --2- 172.21.15.16:0/1448629994 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7aa409f240 0x7f7aa41317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:42.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.897+0000 7f7aaa4e5700 1 --2- 172.21.15.16:0/1448629994 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7aa409f650 0x7f7aa4131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:42.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.898+0000 7f7aa94e3700 1 -- 172.21.15.16:0/1448629994 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7aa409f650 msgr2=0x7f7aa4131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:42.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.898+0000 7f7aa94e3700 1 --2- 172.21.15.16:0/1448629994 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7aa409f650 0x7f7aa4131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:42.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.898+0000 7f7aa94e3700 1 -- 172.21.15.16:0/1448629994 >> v2:172.21.15.16:3300/0 conn(0x7f7aa409e930 msgr2=0x7f7aa409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:42.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.898+0000 7f7aa94e3700 1 --2- 172.21.15.16:0/1448629994 >> v2:172.21.15.16:3300/0 conn(0x7f7aa409e930 0x7f7aa409ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:42.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.898+0000 7f7aa94e3700 1 -- 172.21.15.16:0/1448629994 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aa409fb60 con 0x7f7aa409f240 2023-09-30T22:32:42.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.898+0000 7f7aa9ce4700 1 --2- 172.21.15.16:0/1448629994 >> v2:172.21.15.16:3300/0 conn(0x7f7aa409e930 0x7f7aa409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:42.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.898+0000 7f7aa94e3700 1 --2- 172.21.15.16:0/1448629994 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7aa409f240 0x7f7aa41317f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7a94009840 tx=0x7f7a94009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16d2cfb6f93b08f6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:42.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.898+0000 7f7aa8ce2700 1 -- 172.21.15.16:0/1448629994 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a94011830 con 0x7f7aa409f240 2023-09-30T22:32:42.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.898+0000 7f7aa8ce2700 1 -- 172.21.15.16:0/1448629994 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a94011990 con 0x7f7aa409f240 2023-09-30T22:32:42.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.898+0000 7f7aa8ce2700 1 -- 172.21.15.16:0/1448629994 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a94011d20 con 0x7f7aa409f240 2023-09-30T22:32:42.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.899+0000 7f7aabf48700 1 -- 172.21.15.16:0/1448629994 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7aa409f240 msgr2=0x7f7aa41317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:42.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.899+0000 7f7aabf48700 1 --2- 172.21.15.16:0/1448629994 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7aa409f240 0x7f7aa41317f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f7a94009840 tx=0x7f7a94009b50 comp rx=0 tx=0).stop 2023-09-30T22:32:42.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.899+0000 7f7aabf48700 1 -- 172.21.15.16:0/1448629994 shutdown_connections 2023-09-30T22:32:42.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.899+0000 7f7aabf48700 1 --2- 172.21.15.16:0/1448629994 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7aa409f650 0x7f7aa4131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:42.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.899+0000 7f7aabf48700 1 --2- 172.21.15.16:0/1448629994 >> v2:172.21.15.16:3300/0 conn(0x7f7aa409e930 0x7f7aa409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:42.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.899+0000 7f7aabf48700 1 --2- 172.21.15.16:0/1448629994 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7aa409f240 0x7f7aa41317f0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:42.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.899+0000 7f7aabf48700 1 -- 172.21.15.16:0/1448629994 >> 172.21.15.16:0/1448629994 conn(0x7f7aa40a4210 msgr2=0x7f7aa40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:42.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.899+0000 7f7aabf48700 1 -- 172.21.15.16:0/1448629994 shutdown_connections 2023-09-30T22:32:42.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.899+0000 7f7aabf48700 1 -- 172.21.15.16:0/1448629994 wait complete. 2023-09-30T22:32:42.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.900+0000 7f7aabf48700 1 Processor -- start 2023-09-30T22:32:42.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.900+0000 7f7aabf48700 1 -- start start 2023-09-30T22:32:42.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.901+0000 7f7aabf48700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7aa409e930 0x7f7aa41a1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:42.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.901+0000 7f7aabf48700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7aa409f240 0x7f7aa41a2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:42.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.901+0000 7f7aabf48700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7aa409f650 0x7f7aa41a2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:42.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.901+0000 7f7aabf48700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa41a4f70 con 0x7f7aa409f650 2023-09-30T22:32:42.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.901+0000 7f7aabf48700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa41a3890 con 0x7f7aa409e930 2023-09-30T22:32:42.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.901+0000 7f7aabf48700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa41a5420 con 0x7f7aa409f240 2023-09-30T22:32:42.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.901+0000 7f7aaa4e5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7aa409f650 0x7f7aa41a2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:42.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.901+0000 7f7aa9ce4700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7aa409e930 0x7f7aa41a1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:42.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.901+0000 7f7aaa4e5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7aa409f650 0x7f7aa41a2790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35974/0 (socket says 172.21.15.16:35974) 2023-09-30T22:32:42.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.901+0000 7f7aaa4e5700 1 -- 172.21.15.16:0/50850735 learned_addr learned my addr 172.21.15.16:0/50850735 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:42.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.901+0000 7f7aa94e3700 1 --2- 172.21.15.16:0/50850735 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7aa409f240 0x7f7aa41a2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:42.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.901+0000 7f7aa9ce4700 1 -- 172.21.15.16:0/50850735 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7aa409f240 msgr2=0x7f7aa41a2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:42.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.901+0000 7f7aa9ce4700 1 --2- 172.21.15.16:0/50850735 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7aa409f240 0x7f7aa41a2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:42.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.901+0000 7f7aa9ce4700 1 -- 172.21.15.16:0/50850735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7aa409f650 msgr2=0x7f7aa41a2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:42.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.901+0000 7f7aa9ce4700 1 --2- 172.21.15.16:0/50850735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7aa409f650 0x7f7aa41a2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:42.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.901+0000 7f7aa9ce4700 1 -- 172.21.15.16:0/50850735 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aa000d9f0 con 0x7f7aa409e930 2023-09-30T22:32:42.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.901+0000 7f7aaa4e5700 1 --2- 172.21.15.16:0/50850735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7aa409f650 0x7f7aa41a2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:42.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.902+0000 7f7aa9ce4700 1 --2- 172.21.15.16:0/50850735 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7aa409e930 0x7f7aa41a1d10 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f7aa000bef0 tx=0x7f7aa000bf20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:42.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.902+0000 7f7a9a7fc700 1 -- 172.21.15.16:0/50850735 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa0004230 con 0x7f7aa409e930 2023-09-30T22:32:42.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.902+0000 7f7a9a7fc700 1 -- 172.21.15.16:0/50850735 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aa0004390 con 0x7f7aa409e930 2023-09-30T22:32:42.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.902+0000 7f7aabf48700 1 -- 172.21.15.16:0/50850735 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a940094a0 con 0x7f7aa409e930 2023-09-30T22:32:42.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.902+0000 7f7a9a7fc700 1 -- 172.21.15.16:0/50850735 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa000d280 con 0x7f7aa409e930 2023-09-30T22:32:42.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.902+0000 7f7aabf48700 1 -- 172.21.15.16:0/50850735 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aa40a0190 con 0x7f7aa409e930 2023-09-30T22:32:42.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.903+0000 7f7a9a7fc700 1 -- 172.21.15.16:0/50850735 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7aa000f7b0 con 0x7f7aa409e930 2023-09-30T22:32:42.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.903+0000 7f7aabf48700 1 -- 172.21.15.16:0/50850735 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7aa4074ea0 con 0x7f7aa409e930 2023-09-30T22:32:42.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.903+0000 7f7a9a7fc700 1 -- 172.21.15.16:0/50850735 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f7aa003ef10 con 0x7f7aa409e930 2023-09-30T22:32:42.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.912+0000 7f7a9a7fc700 1 -- 172.21.15.16:0/50850735 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7aa000fe40 con 0x7f7aa409e930 2023-09-30T22:32:42.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.980+0000 7fd9c580b700 1 -- 172.21.15.16:0/2967632014 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd9c0074ea0 con 0x7fd9c0130670 2023-09-30T22:32:42.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.980+0000 7fd9a7fff700 1 -- 172.21.15.16:0/2967632014 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd9b0018260 con 0x7fd9c0130670 2023-09-30T22:32:42.981 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:32:42.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.982+0000 7fd9c580b700 1 -- 172.21.15.16:0/2967632014 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd9c0130670 msgr2=0x7fd9c009dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:42.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.982+0000 7fd9c580b700 1 --2- 172.21.15.16:0/2967632014 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd9c0130670 0x7fd9c009dbe0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd9b000c4e0 tx=0x7fd9b000c7f0 comp rx=0 tx=0).stop 2023-09-30T22:32:42.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.983+0000 7fd9c580b700 1 -- 172.21.15.16:0/2967632014 shutdown_connections 2023-09-30T22:32:42.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.983+0000 7fd9c580b700 1 --2- 172.21.15.16:0/2967632014 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd9c0130670 0x7fd9c009dbe0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:42.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.983+0000 7fd9c580b700 1 --2- 172.21.15.16:0/2967632014 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd9c0135e30 0x7fd9c0139760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:42.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.983+0000 7fd9c580b700 1 --2- 172.21.15.16:0/2967632014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9c0133510 0x7fd9c009e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:42.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.983+0000 7fd9c580b700 1 -- 172.21.15.16:0/2967632014 >> 172.21.15.16:0/2967632014 conn(0x7fd9c00a2aa0 msgr2=0x7fd9c012e330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:42.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.983+0000 7fd9c580b700 1 -- 172.21.15.16:0/2967632014 shutdown_connections 2023-09-30T22:32:42.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:42.983+0000 7fd9c580b700 1 -- 172.21.15.16:0/2967632014 wait complete. 2023-09-30T22:32:42.998 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:32:43.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:43.018+0000 7f97e6914700 1 -- 172.21.15.16:0/1204575445 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f97e01214c0 con 0x7f97e009e000 2023-09-30T22:32:43.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:43.019+0000 7f97dcff9700 1 -- 172.21.15.16:0/1204575445 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f97e01214c0 con 0x7f97e009e000 2023-09-30T22:32:43.019 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:32:43.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:43.021+0000 7f97e6914700 1 -- 172.21.15.16:0/1204575445 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f97e009e000 msgr2=0x7f97e011e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:43.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:43.021+0000 7f97e6914700 1 --2- 172.21.15.16:0/1204575445 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f97e009e000 0x7f97e011e360 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f97d400c550 tx=0x7f97d400c860 comp rx=0 tx=0).stop 2023-09-30T22:32:43.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:43.021+0000 7f97e6914700 1 -- 172.21.15.16:0/1204575445 shutdown_connections 2023-09-30T22:32:43.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:43.021+0000 7f97e6914700 1 --2- 172.21.15.16:0/1204575445 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97e009e410 0x7f97e011e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:43.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:43.021+0000 7f97e6914700 1 --2- 172.21.15.16:0/1204575445 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f97e009d6f0 0x7f97e011de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:43.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:43.021+0000 7f97e6914700 1 --2- 172.21.15.16:0/1204575445 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f97e009e000 0x7f97e011e360 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:43.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:43.021+0000 7f97e6914700 1 -- 172.21.15.16:0/1204575445 >> 172.21.15.16:0/1204575445 conn(0x7f97e0066f50 msgr2=0x7f97e0070fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:43.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:43.021+0000 7f97e6914700 1 -- 172.21.15.16:0/1204575445 shutdown_connections 2023-09-30T22:32:43.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:43.021+0000 7f97e6914700 1 -- 172.21.15.16:0/1204575445 wait complete. 2023-09-30T22:32:43.037 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:32:43.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:43.072+0000 7f8cb8254700 1 -- 172.21.15.16:0/902463689 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f8cb01261c0 con 0x7f8cb0134170 2023-09-30T22:32:43.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:43.073+0000 7f8ca67fc700 1 -- 172.21.15.16:0/902463689 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8cb01261c0 con 0x7f8cb0134170 2023-09-30T22:32:43.073 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:32:43.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:43.075+0000 7f8cb8254700 1 -- 172.21.15.16:0/902463689 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8cb0134170 msgr2=0x7f8cb01c8450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:43.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:43.075+0000 7f8cb8254700 1 --2- 172.21.15.16:0/902463689 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8cb0134170 0x7f8cb01c8450 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8ca800efc0 tx=0x7f8ca800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:32:43.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:43.075+0000 7f8cb8254700 1 -- 172.21.15.16:0/902463689 shutdown_connections 2023-09-30T22:32:43.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:43.075+0000 7f8cb8254700 1 --2- 172.21.15.16:0/902463689 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8cb0134170 0x7f8cb01c8450 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:43.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:43.075+0000 7f8cb8254700 1 --2- 172.21.15.16:0/902463689 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8cb00a0a30 0x7f8cb0123200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:43.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:43.075+0000 7f8cb8254700 1 --2- 172.21.15.16:0/902463689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8cb00a1340 0x7f8cb0123740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:43.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:43.075+0000 7f8cb8254700 1 -- 172.21.15.16:0/902463689 >> 172.21.15.16:0/902463689 conn(0x7f8cb00a6550 msgr2=0x7f8cb009ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:43.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:43.075+0000 7f8cb8254700 1 -- 172.21.15.16:0/902463689 shutdown_connections 2023-09-30T22:32:43.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:43.075+0000 7f8cb8254700 1 -- 172.21.15.16:0/902463689 wait complete. 2023-09-30T22:32:43.089 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:32:43.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:43.123+0000 7f7aabf48700 1 -- 172.21.15.16:0/50850735 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7aa41a4a40 con 0x7f7aa409e930 2023-09-30T22:32:43.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:43.123+0000 7f7a9a7fc700 1 -- 172.21.15.16:0/50850735 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7aa0015e60 con 0x7f7aa409e930 2023-09-30T22:32:43.124 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:32:43.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:43.126+0000 7f7aabf48700 1 -- 172.21.15.16:0/50850735 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7aa409e930 msgr2=0x7f7aa41a1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:43.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:43.126+0000 7f7aabf48700 1 --2- 172.21.15.16:0/50850735 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7aa409e930 0x7f7aa41a1d10 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f7aa000bef0 tx=0x7f7aa000bf20 comp rx=0 tx=0).stop 2023-09-30T22:32:43.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:43.126+0000 7f7aabf48700 1 -- 172.21.15.16:0/50850735 shutdown_connections 2023-09-30T22:32:43.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:43.126+0000 7f7aabf48700 1 --2- 172.21.15.16:0/50850735 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7aa409e930 0x7f7aa41a1d10 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:43.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:43.126+0000 7f7aabf48700 1 --2- 172.21.15.16:0/50850735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7aa409f650 0x7f7aa41a2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:43.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:43.126+0000 7f7aabf48700 1 --2- 172.21.15.16:0/50850735 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7aa409f240 0x7f7aa41a2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:43.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:43.126+0000 7f7aabf48700 1 -- 172.21.15.16:0/50850735 >> 172.21.15.16:0/50850735 conn(0x7f7aa40a4210 msgr2=0x7f7aa41b6b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:43.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:43.126+0000 7f7aabf48700 1 -- 172.21.15.16:0/50850735 shutdown_connections 2023-09-30T22:32:43.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:43.126+0000 7f7aabf48700 1 -- 172.21.15.16:0/50850735 wait complete. 2023-09-30T22:32:43.140 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:32:43.999 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:32:44.039 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:32:44.090 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:32:44.140 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:32:44.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.152+0000 7f93c7653700 1 Processor -- start 2023-09-30T22:32:44.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.153+0000 7f93c7653700 1 -- start start 2023-09-30T22:32:44.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.154+0000 7f93c7653700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f93c006f5d0 0x7f93c012ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:44.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.154+0000 7f93c7653700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f93c0066410 0x7f93c0138d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:44.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.154+0000 7f93c7653700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93c006fa40 con 0x7f93c012cf60 2023-09-30T22:32:44.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.154+0000 7f93c7653700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93c012cd90 con 0x7f93c0066410 2023-09-30T22:32:44.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.154+0000 7f93c7653700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93c0071780 con 0x7f93c006f5d0 2023-09-30T22:32:44.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.154+0000 7f93c5bf0700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f93c0066410 0x7f93c0138d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:44.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.154+0000 7f93c5bf0700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f93c0066410 0x7f93c0138d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:35982/0 (socket says 172.21.15.16:35982) 2023-09-30T22:32:44.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.154+0000 7f93c5bf0700 1 -- 172.21.15.16:0/2216063966 learned_addr learned my addr 172.21.15.16:0/2216063966 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:44.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.154+0000 7f93c4bee700 1 --2- 172.21.15.16:0/2216063966 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f93c006f5d0 0x7f93c012ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:44.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.154+0000 7f93c5bf0700 1 -- 172.21.15.16:0/2216063966 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f93c006f5d0 msgr2=0x7f93c012ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:44.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.154+0000 7f93c5bf0700 1 --2- 172.21.15.16:0/2216063966 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f93c006f5d0 0x7f93c012ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:44.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.154+0000 7f93c5bf0700 1 -- 172.21.15.16:0/2216063966 >> v1:172.21.15.16:6789/0 conn(0x7f93c012cf60 legacy=0x7f93c012f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:44.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.154+0000 7f93c5bf0700 1 -- 172.21.15.16:0/2216063966 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93c0071a40 con 0x7f93c0066410 2023-09-30T22:32:44.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.155+0000 7f93c5bf0700 1 --2- 172.21.15.16:0/2216063966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93c0066410 0x7f93c0138d40 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f93b8009e50 tx=0x7f93b800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=819fd694c30454ee server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:44.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.155+0000 7f93b7fff700 1 -- 172.21.15.16:0/2216063966 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b800e070 con 0x7f93c0066410 2023-09-30T22:32:44.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.155+0000 7f93b7fff700 1 -- 172.21.15.16:0/2216063966 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93b8003a20 con 0x7f93c0066410 2023-09-30T22:32:44.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.155+0000 7f93b7fff700 1 -- 172.21.15.16:0/2216063966 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b8003dd0 con 0x7f93c0066410 2023-09-30T22:32:44.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.156+0000 7f93c7653700 1 -- 172.21.15.16:0/2216063966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93c0066410 msgr2=0x7f93c0138d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:44.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.156+0000 7f93c7653700 1 --2- 172.21.15.16:0/2216063966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93c0066410 0x7f93c0138d40 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f93b8009e50 tx=0x7f93b800b330 comp rx=0 tx=0).stop 2023-09-30T22:32:44.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.156+0000 7f93c7653700 1 -- 172.21.15.16:0/2216063966 shutdown_connections 2023-09-30T22:32:44.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.156+0000 7f93c7653700 1 --2- 172.21.15.16:0/2216063966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93c0066410 0x7f93c0138d40 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:44.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.156+0000 7f93c7653700 1 --2- 172.21.15.16:0/2216063966 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f93c006f5d0 0x7f93c012ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:44.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.156+0000 7f93c7653700 1 -- 172.21.15.16:0/2216063966 >> 172.21.15.16:0/2216063966 conn(0x7f93c00a2aa0 msgr2=0x7f93c00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:44.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.156+0000 7f93c7653700 1 -- 172.21.15.16:0/2216063966 shutdown_connections 2023-09-30T22:32:44.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.156+0000 7f93c7653700 1 -- 172.21.15.16:0/2216063966 wait complete. 2023-09-30T22:32:44.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.157+0000 7f93c7653700 1 Processor -- start 2023-09-30T22:32:44.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.158+0000 7f93c7653700 1 -- start start 2023-09-30T22:32:44.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.158+0000 7f93c7653700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f93c0066410 0x7f93c01c89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:44.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.158+0000 7f93c7653700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93c006f5d0 0x7f93c01c8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:44.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.158+0000 7f93c7653700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f93c012cf60 0x7f93c0130920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:44.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.158+0000 7f93c7653700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c0133db0 con 0x7f93c006f5d0 2023-09-30T22:32:44.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.158+0000 7f93c7653700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93c013a490 con 0x7f93c012cf60 2023-09-30T22:32:44.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.158+0000 7f93c7653700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f93c01c45b0 con 0x7f93c0066410 2023-09-30T22:32:44.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.159+0000 7f93c53ef700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f93c0066410 0x7f93c01c89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:44.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.159+0000 7f93c53ef700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f93c0066410 0x7f93c01c89b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:39212/0 (socket says 172.21.15.16:39212) 2023-09-30T22:32:44.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.159+0000 7f93c53ef700 1 -- 172.21.15.16:0/1503567821 learned_addr learned my addr 172.21.15.16:0/1503567821 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:44.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.159+0000 7f93c4bee700 1 --2- 172.21.15.16:0/1503567821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93c006f5d0 0x7f93c01c8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:44.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.159+0000 7f93c5bf0700 1 --2- 172.21.15.16:0/1503567821 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f93c012cf60 0x7f93c0130920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:44.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.159+0000 7f93c53ef700 1 -- 172.21.15.16:0/1503567821 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f93c012cf60 msgr2=0x7f93c0130920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:44.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.159+0000 7f93c53ef700 1 --2- 172.21.15.16:0/1503567821 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f93c012cf60 0x7f93c0130920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:44.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.159+0000 7f93c53ef700 1 -- 172.21.15.16:0/1503567821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93c006f5d0 msgr2=0x7f93c01c8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:44.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.159+0000 7f93c53ef700 1 --2- 172.21.15.16:0/1503567821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93c006f5d0 0x7f93c01c8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:44.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.159+0000 7f93c53ef700 1 -- 172.21.15.16:0/1503567821 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b8009b30 con 0x7f93c0066410 2023-09-30T22:32:44.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.159+0000 7f93c4bee700 1 --2- 172.21.15.16:0/1503567821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93c006f5d0 0x7f93c01c8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:44.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.159+0000 7f93c53ef700 1 --2- 172.21.15.16:0/1503567821 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f93c0066410 0x7f93c01c89b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f93bc00c500 tx=0x7f93bc00c810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:44.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.159+0000 7f93b5ffb700 1 -- 172.21.15.16:0/1503567821 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93bc011790 con 0x7f93c0066410 2023-09-30T22:32:44.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.160+0000 7f93b5ffb700 1 -- 172.21.15.16:0/1503567821 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93bc0118f0 con 0x7f93c0066410 2023-09-30T22:32:44.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.160+0000 7f93b5ffb700 1 -- 172.21.15.16:0/1503567821 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93bc011ca0 con 0x7f93c0066410 2023-09-30T22:32:44.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.160+0000 7f93c7653700 1 -- 172.21.15.16:0/1503567821 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93c013adb0 con 0x7f93c0066410 2023-09-30T22:32:44.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.160+0000 7f93c7653700 1 -- 172.21.15.16:0/1503567821 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93c01330d0 con 0x7f93c0066410 2023-09-30T22:32:44.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.161+0000 7f93b5ffb700 1 -- 172.21.15.16:0/1503567821 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f93bc01c460 con 0x7f93c0066410 2023-09-30T22:32:44.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.161+0000 7f93b5ffb700 1 -- 172.21.15.16:0/1503567821 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f93bc03f380 con 0x7f93c0066410 2023-09-30T22:32:44.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.161+0000 7f93c7653700 1 -- 172.21.15.16:0/1503567821 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93c01c9430 con 0x7f93c0066410 2023-09-30T22:32:44.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.167+0000 7f93b5ffb700 1 -- 172.21.15.16:0/1503567821 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f93bc0108d0 con 0x7f93c0066410 2023-09-30T22:32:44.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.194+0000 7f7eb28d1700 1 Processor -- start 2023-09-30T22:32:44.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.194+0000 7f7eb28d1700 1 -- start start 2023-09-30T22:32:44.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.194+0000 7f7eb28d1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7eac06c0e0 0x7f7eac09ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:44.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.194+0000 7f7eb28d1700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7eac134ec0 0x7f7eac1379e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:44.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.194+0000 7f7eb28d1700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7eac134980 con 0x7f7eac137f20 2023-09-30T22:32:44.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.194+0000 7f7eb28d1700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7eac06c4b0 con 0x7f7eac134ec0 2023-09-30T22:32:44.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.194+0000 7f7eb28d1700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7eac0a09c0 con 0x7f7eac06c0e0 2023-09-30T22:32:44.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.195+0000 7f7eb0e6e700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7eac137f20 0x7f7eac13ed10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53690/0 (socket says 172.21.15.16:53690) 2023-09-30T22:32:44.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.195+0000 7f7eb0e6e700 1 -- 172.21.15.16:0/356250694 learned_addr learned my addr 172.21.15.16:0/356250694 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:44.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.195+0000 7f7eab7fe700 1 --2- 172.21.15.16:0/356250694 >> v2:172.21.15.16:3300/0 conn(0x7f7eac134ec0 0x7f7eac1379e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:44.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.195+0000 7f7eabfff700 1 --2- 172.21.15.16:0/356250694 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7eac06c0e0 0x7f7eac09ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:44.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.195+0000 7f7eaaffd700 1 -- 172.21.15.16:0/356250694 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3408529523 0 0) 0x7f7eac134980 con 0x7f7eac137f20 2023-09-30T22:32:44.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.195+0000 7f7eaaffd700 1 -- 172.21.15.16:0/356250694 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e940035b0 con 0x7f7eac137f20 2023-09-30T22:32:44.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.195+0000 7f7eab7fe700 1 -- 172.21.15.16:0/356250694 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7eac06c0e0 msgr2=0x7f7eac09ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:44.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.195+0000 7f7eab7fe700 1 --2- 172.21.15.16:0/356250694 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7eac06c0e0 0x7f7eac09ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:44.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.195+0000 7f7eab7fe700 1 -- 172.21.15.16:0/356250694 >> v1:172.21.15.16:6789/0 conn(0x7f7eac137f20 legacy=0x7f7eac13ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:44.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.195+0000 7f7eab7fe700 1 -- 172.21.15.16:0/356250694 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eac0a0600 con 0x7f7eac134ec0 2023-09-30T22:32:44.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.195+0000 7f7eab7fe700 1 --2- 172.21.15.16:0/356250694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7eac134ec0 0x7f7eac1379e0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0009840 tx=0x7f7ea0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=975a5aeefca8fce1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:44.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.196+0000 7f7eaaffd700 1 -- 172.21.15.16:0/356250694 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea0011830 con 0x7f7eac134ec0 2023-09-30T22:32:44.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.196+0000 7f7eaaffd700 1 -- 172.21.15.16:0/356250694 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ea0011990 con 0x7f7eac134ec0 2023-09-30T22:32:44.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.196+0000 7f7eaaffd700 1 -- 172.21.15.16:0/356250694 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea0011d20 con 0x7f7eac134ec0 2023-09-30T22:32:44.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.196+0000 7f7eb28d1700 1 -- 172.21.15.16:0/356250694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7eac134ec0 msgr2=0x7f7eac1379e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:44.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.196+0000 7f7eb28d1700 1 --2- 172.21.15.16:0/356250694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7eac134ec0 0x7f7eac1379e0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0009840 tx=0x7f7ea0009b50 comp rx=0 tx=0).stop 2023-09-30T22:32:44.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.197+0000 7f7eb28d1700 1 -- 172.21.15.16:0/356250694 shutdown_connections 2023-09-30T22:32:44.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.197+0000 7f7eb28d1700 1 --2- 172.21.15.16:0/356250694 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7eac06c0e0 0x7f7eac09ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:44.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.197+0000 7f7eb28d1700 1 --2- 172.21.15.16:0/356250694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7eac134ec0 0x7f7eac1379e0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:44.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.197+0000 7f7eb28d1700 1 -- 172.21.15.16:0/356250694 >> 172.21.15.16:0/356250694 conn(0x7f7eac0a4210 msgr2=0x7f7eac0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:44.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.197+0000 7f7eb28d1700 1 -- 172.21.15.16:0/356250694 shutdown_connections 2023-09-30T22:32:44.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.197+0000 7f7eb28d1700 1 -- 172.21.15.16:0/356250694 wait complete. 2023-09-30T22:32:44.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.198+0000 7f7eb28d1700 1 Processor -- start 2023-09-30T22:32:44.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.198+0000 7f7eb28d1700 1 -- start start 2023-09-30T22:32:44.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.198+0000 7f7eb28d1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7eac06c0e0 0x7f7eac1333c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:44.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.198+0000 7f7eb28d1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7eac134ec0 0x7f7eac133900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:44.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.198+0000 7f7eb28d1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7eac137f20 0x7f7eac133e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:44.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.198+0000 7f7eb28d1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eac0a03f0 con 0x7f7eac134ec0 2023-09-30T22:32:44.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.198+0000 7f7eb28d1700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7eac11c220 con 0x7f7eac137f20 2023-09-30T22:32:44.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.198+0000 7f7eb28d1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7eac11dd20 con 0x7f7eac06c0e0 2023-09-30T22:32:44.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.199+0000 7f7eabfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7eac06c0e0 0x7f7eac1333c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:44.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.199+0000 7f7eabfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7eac06c0e0 0x7f7eac1333c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:39218/0 (socket says 172.21.15.16:39218) 2023-09-30T22:32:44.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.199+0000 7f7eabfff700 1 -- 172.21.15.16:0/4055398878 learned_addr learned my addr 172.21.15.16:0/4055398878 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:44.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.199+0000 7f7eb0e6e700 1 --2- 172.21.15.16:0/4055398878 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7eac137f20 0x7f7eac133e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:44.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.199+0000 7f7eab7fe700 1 --2- 172.21.15.16:0/4055398878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7eac134ec0 0x7f7eac133900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:44.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.199+0000 7f7eab7fe700 1 -- 172.21.15.16:0/4055398878 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7eac06c0e0 msgr2=0x7f7eac1333c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:44.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.199+0000 7f7eab7fe700 1 --2- 172.21.15.16:0/4055398878 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7eac06c0e0 0x7f7eac1333c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:44.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.199+0000 7f7eab7fe700 1 -- 172.21.15.16:0/4055398878 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7eac137f20 msgr2=0x7f7eac133e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:44.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.199+0000 7f7eab7fe700 1 --2- 172.21.15.16:0/4055398878 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7eac137f20 0x7f7eac133e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:44.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.199+0000 7f7eab7fe700 1 -- 172.21.15.16:0/4055398878 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea00094a0 con 0x7f7eac134ec0 2023-09-30T22:32:44.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.199+0000 7f7eb0e6e700 1 --2- 172.21.15.16:0/4055398878 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7eac137f20 0x7f7eac133e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:44.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.200+0000 7f7eab7fe700 1 --2- 172.21.15.16:0/4055398878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7eac134ec0 0x7f7eac133900 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f7eac06da40 tx=0x7f7ea0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:44.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.200+0000 7f7ea8ff9700 1 -- 172.21.15.16:0/4055398878 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea0004380 con 0x7f7eac134ec0 2023-09-30T22:32:44.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.200+0000 7f7ea8ff9700 1 -- 172.21.15.16:0/4055398878 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ea001f070 con 0x7f7eac134ec0 2023-09-30T22:32:44.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.200+0000 7f7eb28d1700 1 -- 172.21.15.16:0/4055398878 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7eac09a660 con 0x7f7eac134ec0 2023-09-30T22:32:44.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.200+0000 7f7ea8ff9700 1 -- 172.21.15.16:0/4055398878 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea00036a0 con 0x7f7eac134ec0 2023-09-30T22:32:44.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.200+0000 7f7eb28d1700 1 -- 172.21.15.16:0/4055398878 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7eac11e000 con 0x7f7eac134ec0 2023-09-30T22:32:44.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.201+0000 7f7ea8ff9700 1 -- 172.21.15.16:0/4055398878 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7ea0021030 con 0x7f7eac134ec0 2023-09-30T22:32:44.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.201+0000 7f7ea8ff9700 1 -- 172.21.15.16:0/4055398878 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f7ea00408a0 con 0x7f7eac134ec0 2023-09-30T22:32:44.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.201+0000 7f7eb28d1700 1 -- 172.21.15.16:0/4055398878 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7eac074ea0 con 0x7f7eac134ec0 2023-09-30T22:32:44.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.206+0000 7f7ea8ff9700 1 -- 172.21.15.16:0/4055398878 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7ea0010970 con 0x7f7eac134ec0 2023-09-30T22:32:44.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.238+0000 7f4df5083700 1 Processor -- start 2023-09-30T22:32:44.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.238+0000 7f4df5083700 1 -- start start 2023-09-30T22:32:44.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.238+0000 7f4df5083700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4df009e930 0x7f4df009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:44.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.238+0000 7f4df5083700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4df009f240 0x7f4df01317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:44.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.238+0000 7f4df5083700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4df009f650 0x7f4df0131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:44.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.238+0000 7f4df5083700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4df0070940 con 0x7f4df009f240 2023-09-30T22:32:44.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.238+0000 7f4df5083700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4df006fd60 con 0x7f4df009e930 2023-09-30T22:32:44.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.238+0000 7f4df5083700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4df006fa80 con 0x7f4df009f650 2023-09-30T22:32:44.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.239+0000 7f4deed9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4df009e930 0x7f4df009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:44.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.239+0000 7f4deed9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4df009e930 0x7f4df009ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:39562/0 (socket says 172.21.15.16:39562) 2023-09-30T22:32:44.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.239+0000 7f4deed9d700 1 -- 172.21.15.16:0/2491006373 learned_addr learned my addr 172.21.15.16:0/2491006373 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:44.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.239+0000 7f4dee59c700 1 --2- 172.21.15.16:0/2491006373 >> v2:172.21.15.16:3300/0 conn(0x7f4df009f240 0x7f4df01317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:44.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.239+0000 7f4def59e700 1 --2- 172.21.15.16:0/2491006373 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4df009f650 0x7f4df0131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:44.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.239+0000 7f4def59e700 1 -- 172.21.15.16:0/2491006373 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4df009e930 msgr2=0x7f4df009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:44.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.239+0000 7f4def59e700 1 --2- 172.21.15.16:0/2491006373 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4df009e930 0x7f4df009ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:44.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.239+0000 7f4def59e700 1 -- 172.21.15.16:0/2491006373 >> v2:172.21.15.16:3300/0 conn(0x7f4df009f240 msgr2=0x7f4df01317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:44.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.239+0000 7f4def59e700 1 --2- 172.21.15.16:0/2491006373 >> v2:172.21.15.16:3300/0 conn(0x7f4df009f240 0x7f4df01317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:44.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.239+0000 7f4def59e700 1 -- 172.21.15.16:0/2491006373 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df009fb60 con 0x7f4df009f650 2023-09-30T22:32:44.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.239+0000 7f4def59e700 1 --2- 172.21.15.16:0/2491006373 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4df009f650 0x7f4df0131d30 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4de8009d90 tx=0x7f4de800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eed4e50efac4aa74 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:44.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.240+0000 7f4dedd9b700 1 -- 172.21.15.16:0/2491006373 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de800e070 con 0x7f4df009f650 2023-09-30T22:32:44.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.240+0000 7f4dedd9b700 1 -- 172.21.15.16:0/2491006373 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4de800bb20 con 0x7f4df009f650 2023-09-30T22:32:44.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.240+0000 7f4dedd9b700 1 -- 172.21.15.16:0/2491006373 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de8012430 con 0x7f4df009f650 2023-09-30T22:32:44.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.240+0000 7f4df5083700 1 -- 172.21.15.16:0/2491006373 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4df009f650 msgr2=0x7f4df0131d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:44.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.240+0000 7f4df5083700 1 --2- 172.21.15.16:0/2491006373 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4df009f650 0x7f4df0131d30 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f4de8009d90 tx=0x7f4de800b330 comp rx=0 tx=0).stop 2023-09-30T22:32:44.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.240+0000 7f4df5083700 1 -- 172.21.15.16:0/2491006373 shutdown_connections 2023-09-30T22:32:44.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.240+0000 7f4df5083700 1 --2- 172.21.15.16:0/2491006373 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4df009e930 0x7f4df009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:44.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.240+0000 7f4df5083700 1 --2- 172.21.15.16:0/2491006373 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4df009f650 0x7f4df0131d30 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:44.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.240+0000 7f4df5083700 1 --2- 172.21.15.16:0/2491006373 >> v2:172.21.15.16:3300/0 conn(0x7f4df009f240 0x7f4df01317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:44.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.240+0000 7f4df5083700 1 -- 172.21.15.16:0/2491006373 >> 172.21.15.16:0/2491006373 conn(0x7f4df00a4210 msgr2=0x7f4df00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:44.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.240+0000 7f4df5083700 1 -- 172.21.15.16:0/2491006373 shutdown_connections 2023-09-30T22:32:44.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.240+0000 7f4df5083700 1 -- 172.21.15.16:0/2491006373 wait complete. 2023-09-30T22:32:44.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.242+0000 7f4df5083700 1 Processor -- start 2023-09-30T22:32:44.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.242+0000 7f4df5083700 1 -- start start 2023-09-30T22:32:44.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.243+0000 7f4df5083700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4df009e930 0x7f4df006b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:44.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.243+0000 7f4df5083700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4df009f240 0x7f4df006bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:44.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.243+0000 7f4df5083700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4df009f650 0x7f4df006c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:44.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.243+0000 7f4df5083700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df006d430 con 0x7f4df009e930 2023-09-30T22:32:44.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.243+0000 7f4df5083700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4df01a8880 con 0x7f4df009f240 2023-09-30T22:32:44.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.243+0000 7f4df5083700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4df01a9360 con 0x7f4df009f650 2023-09-30T22:32:44.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.243+0000 7f4def59e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4df009f650 0x7f4df006c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:44.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.243+0000 7f4deed9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4df009e930 0x7f4df006b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:44.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.243+0000 7f4def59e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4df009f650 0x7f4df006c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:39240/0 (socket says 172.21.15.16:39240) 2023-09-30T22:32:44.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.243+0000 7f4def59e700 1 -- 172.21.15.16:0/1035240906 learned_addr learned my addr 172.21.15.16:0/1035240906 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:44.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.243+0000 7f4dee59c700 1 --2- 172.21.15.16:0/1035240906 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4df009f240 0x7f4df006bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:44.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.244+0000 7f4def59e700 1 -- 172.21.15.16:0/1035240906 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4df009f240 msgr2=0x7f4df006bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:44.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.244+0000 7f4def59e700 1 --2- 172.21.15.16:0/1035240906 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4df009f240 0x7f4df006bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:44.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.244+0000 7f4def59e700 1 -- 172.21.15.16:0/1035240906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4df009e930 msgr2=0x7f4df006b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:44.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.244+0000 7f4def59e700 1 --2- 172.21.15.16:0/1035240906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4df009e930 0x7f4df006b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:44.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.244+0000 7f4def59e700 1 -- 172.21.15.16:0/1035240906 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4de8009a70 con 0x7f4df009f650 2023-09-30T22:32:44.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.244+0000 7f4dee59c700 1 --2- 172.21.15.16:0/1035240906 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4df009f240 0x7f4df006bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:44.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.244+0000 7f4def59e700 1 --2- 172.21.15.16:0/1035240906 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4df009f650 0x7f4df006c330 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4de800b370 tx=0x7f4de8004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:44.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.244+0000 7f4dd77fe700 1 -- 172.21.15.16:0/1035240906 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de8004800 con 0x7f4df009f650 2023-09-30T22:32:44.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.244+0000 7f4dd77fe700 1 -- 172.21.15.16:0/1035240906 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4de8020070 con 0x7f4df009f650 2023-09-30T22:32:44.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.244+0000 7f4df5083700 1 -- 172.21.15.16:0/1035240906 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4df00a0190 con 0x7f4df009f650 2023-09-30T22:32:44.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.244+0000 7f4dd77fe700 1 -- 172.21.15.16:0/1035240906 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de80276c0 con 0x7f4df009f650 2023-09-30T22:32:44.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.244+0000 7f4df5083700 1 -- 172.21.15.16:0/1035240906 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4df00a3540 con 0x7f4df009f650 2023-09-30T22:32:44.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.245+0000 7f4dd77fe700 1 -- 172.21.15.16:0/1035240906 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4de8012430 con 0x7f4df009f650 2023-09-30T22:32:44.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.245+0000 7f4dd77fe700 1 -- 172.21.15.16:0/1035240906 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f4de8022070 con 0x7f4df009f650 2023-09-30T22:32:44.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.246+0000 7f4df5083700 1 -- 172.21.15.16:0/1035240906 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4df0071d80 con 0x7f4df009f650 2023-09-30T22:32:44.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.250+0000 7f4dd77fe700 1 -- 172.21.15.16:0/1035240906 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4de80126e0 con 0x7f4df009f650 2023-09-30T22:32:44.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.308+0000 7fc2225c7700 1 Processor -- start 2023-09-30T22:32:44.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.308+0000 7fc2225c7700 1 -- start start 2023-09-30T22:32:44.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.309+0000 7fc2225c7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc21c1338c0 0x7fc21c133c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:44.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.309+0000 7fc2225c7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc21c1341d0 0x7fc21c09de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:44.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.309+0000 7fc2225c7700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc21c06f720 con 0x7fc21c09e390 2023-09-30T22:32:44.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.309+0000 7fc2225c7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc21c070c80 con 0x7fc21c1338c0 2023-09-30T22:32:44.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.309+0000 7fc2225c7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc21c06fce0 con 0x7fc21c1341d0 2023-09-30T22:32:44.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.309+0000 7fc21b7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc21c1341d0 0x7fc21c09de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:44.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.309+0000 7fc21b7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc21c1341d0 0x7fc21c09de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:39246/0 (socket says 172.21.15.16:39246) 2023-09-30T22:32:44.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.309+0000 7fc21b7fe700 1 -- 172.21.15.16:0/475864535 learned_addr learned my addr 172.21.15.16:0/475864535 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:44.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.309+0000 7fc21b7fe700 1 -- 172.21.15.16:0/475864535 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc21c1338c0 msgr2=0x7fc21c133c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:44.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.309+0000 7fc21b7fe700 1 --2- 172.21.15.16:0/475864535 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc21c1338c0 0x7fc21c133c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:44.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.309+0000 7fc21b7fe700 1 -- 172.21.15.16:0/475864535 >> v1:172.21.15.16:6789/0 conn(0x7fc21c09e390 legacy=0x7fc21c13a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:44.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.309+0000 7fc21b7fe700 1 -- 172.21.15.16:0/475864535 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc21c1336a0 con 0x7fc21c1341d0 2023-09-30T22:32:44.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.310+0000 7fc21b7fe700 1 --2- 172.21.15.16:0/475864535 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc21c1341d0 0x7fc21c09de50 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc210009920 tx=0x7fc210009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e4d5082128dd706 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:44.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.310+0000 7fc21affd700 1 -- 172.21.15.16:0/475864535 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21000ba60 con 0x7fc21c1341d0 2023-09-30T22:32:44.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.310+0000 7fc21affd700 1 -- 172.21.15.16:0/475864535 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc21000bbc0 con 0x7fc21c1341d0 2023-09-30T22:32:44.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.310+0000 7fc21affd700 1 -- 172.21.15.16:0/475864535 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc210009e70 con 0x7fc21c1341d0 2023-09-30T22:32:44.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.311+0000 7fc2225c7700 1 -- 172.21.15.16:0/475864535 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc21c1341d0 msgr2=0x7fc21c09de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:44.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.311+0000 7fc2225c7700 1 --2- 172.21.15.16:0/475864535 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc21c1341d0 0x7fc21c09de50 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fc210009920 tx=0x7fc210009c30 comp rx=0 tx=0).stop 2023-09-30T22:32:44.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.311+0000 7fc2225c7700 1 -- 172.21.15.16:0/475864535 shutdown_connections 2023-09-30T22:32:44.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.311+0000 7fc2225c7700 1 --2- 172.21.15.16:0/475864535 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc21c1338c0 0x7fc21c133c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:44.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.311+0000 7fc2225c7700 1 --2- 172.21.15.16:0/475864535 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc21c1341d0 0x7fc21c09de50 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:44.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.311+0000 7fc2225c7700 1 -- 172.21.15.16:0/475864535 >> 172.21.15.16:0/475864535 conn(0x7fc21c0a2aa0 msgr2=0x7fc21c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:44.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.311+0000 7fc2225c7700 1 -- 172.21.15.16:0/475864535 shutdown_connections 2023-09-30T22:32:44.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.311+0000 7fc2225c7700 1 -- 172.21.15.16:0/475864535 wait complete. 2023-09-30T22:32:44.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.312+0000 7fc2225c7700 1 Processor -- start 2023-09-30T22:32:44.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.312+0000 7fc2225c7700 1 -- start start 2023-09-30T22:32:44.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.313+0000 7fc2225c7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc21c09e390 0x7fc21c11faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:44.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.313+0000 7fc2225c7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc21c1338c0 0x7fc21c11ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:44.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.313+0000 7fc2225c7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc21c1341d0 0x7fc21c120520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:44.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.313+0000 7fc2225c7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc21c1309d0 con 0x7fc21c1338c0 2023-09-30T22:32:44.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.313+0000 7fc2225c7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc21c121730 con 0x7fc21c09e390 2023-09-30T22:32:44.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.313+0000 7fc2225c7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc21c122f90 con 0x7fc21c1341d0 2023-09-30T22:32:44.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.313+0000 7fc21b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc21c1338c0 0x7fc21c11ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:44.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.313+0000 7fc21b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc21c1338c0 0x7fc21c11ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36040/0 (socket says 172.21.15.16:36040) 2023-09-30T22:32:44.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.313+0000 7fc21b7fe700 1 -- 172.21.15.16:0/1527698728 learned_addr learned my addr 172.21.15.16:0/1527698728 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:44.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.313+0000 7fc220b64700 1 --2- 172.21.15.16:0/1527698728 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc21c1341d0 0x7fc21c120520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:44.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.313+0000 7fc21b7fe700 1 -- 172.21.15.16:0/1527698728 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc21c1341d0 msgr2=0x7fc21c120520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:44.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.313+0000 7fc21b7fe700 1 --2- 172.21.15.16:0/1527698728 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc21c1341d0 0x7fc21c120520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:44.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.313+0000 7fc21b7fe700 1 -- 172.21.15.16:0/1527698728 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc21c09e390 msgr2=0x7fc21c11faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:44.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.313+0000 7fc21b7fe700 1 --2- 172.21.15.16:0/1527698728 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc21c09e390 0x7fc21c11faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:44.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.313+0000 7fc21b7fe700 1 -- 172.21.15.16:0/1527698728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc210009550 con 0x7fc21c1338c0 2023-09-30T22:32:44.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.313+0000 7fc220b64700 1 --2- 172.21.15.16:0/1527698728 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc21c1341d0 0x7fc21c120520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:44.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.314+0000 7fc21b7fe700 1 --2- 172.21.15.16:0/1527698728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc21c1338c0 0x7fc21c11ffe0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fc210000c00 tx=0x7fc210004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:44.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.314+0000 7fc218ff9700 1 -- 172.21.15.16:0/1527698728 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2100036a0 con 0x7fc21c1338c0 2023-09-30T22:32:44.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.314+0000 7fc218ff9700 1 -- 172.21.15.16:0/1527698728 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc210003820 con 0x7fc21c1338c0 2023-09-30T22:32:44.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.314+0000 7fc218ff9700 1 -- 172.21.15.16:0/1527698728 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc210003bd0 con 0x7fc21c1338c0 2023-09-30T22:32:44.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.314+0000 7fc2225c7700 1 -- 172.21.15.16:0/1527698728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc21c130e20 con 0x7fc21c1338c0 2023-09-30T22:32:44.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.314+0000 7fc2225c7700 1 -- 172.21.15.16:0/1527698728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc21c123cc0 con 0x7fc21c1338c0 2023-09-30T22:32:44.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.315+0000 7fc218ff9700 1 -- 172.21.15.16:0/1527698728 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc210003d30 con 0x7fc21c1338c0 2023-09-30T22:32:44.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.315+0000 7fc218ff9700 1 -- 172.21.15.16:0/1527698728 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fc210040f80 con 0x7fc21c1338c0 2023-09-30T22:32:44.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.316+0000 7fc2225c7700 1 -- 172.21.15.16:0/1527698728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc21c074ea0 con 0x7fc21c1338c0 2023-09-30T22:32:44.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.326+0000 7fc218ff9700 1 -- 172.21.15.16:0/1527698728 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc210010790 con 0x7fc21c1338c0 2023-09-30T22:32:44.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.415+0000 7f93c7653700 1 -- 172.21.15.16:0/1503567821 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f93c0074ea0 con 0x7f93c0066410 2023-09-30T22:32:44.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.415+0000 7f93b5ffb700 1 -- 172.21.15.16:0/1503567821 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f93bc0102e0 con 0x7f93c0066410 2023-09-30T22:32:44.416 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:32:44.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.417+0000 7f7eb28d1700 1 -- 172.21.15.16:0/4055398878 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7eac06b250 con 0x7f7eac134ec0 2023-09-30T22:32:44.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.417+0000 7f7ea8ff9700 1 -- 172.21.15.16:0/4055398878 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7ea0015020 con 0x7f7eac134ec0 2023-09-30T22:32:44.418 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:32:44.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.419+0000 7f93c7653700 1 -- 172.21.15.16:0/1503567821 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f93c0066410 msgr2=0x7f93c01c89b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:44.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.419+0000 7f93c7653700 1 --2- 172.21.15.16:0/1503567821 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f93c0066410 0x7f93c01c89b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f93bc00c500 tx=0x7f93bc00c810 comp rx=0 tx=0).stop 2023-09-30T22:32:44.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.419+0000 7f93c7653700 1 -- 172.21.15.16:0/1503567821 shutdown_connections 2023-09-30T22:32:44.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.419+0000 7f93c7653700 1 --2- 172.21.15.16:0/1503567821 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f93c012cf60 0x7f93c0130920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:44.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.419+0000 7f93c7653700 1 --2- 172.21.15.16:0/1503567821 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f93c0066410 0x7f93c01c89b0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:44.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.419+0000 7f93c7653700 1 --2- 172.21.15.16:0/1503567821 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93c006f5d0 0x7f93c01c8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:44.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.419+0000 7f93c7653700 1 -- 172.21.15.16:0/1503567821 >> 172.21.15.16:0/1503567821 conn(0x7f93c00a2aa0 msgr2=0x7f93c00a1500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:44.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.420+0000 7f93c7653700 1 -- 172.21.15.16:0/1503567821 shutdown_connections 2023-09-30T22:32:44.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.420+0000 7f93c7653700 1 -- 172.21.15.16:0/1503567821 wait complete. 2023-09-30T22:32:44.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.420+0000 7f7eb28d1700 1 -- 172.21.15.16:0/4055398878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7eac134ec0 msgr2=0x7f7eac133900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:44.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.420+0000 7f7eb28d1700 1 --2- 172.21.15.16:0/4055398878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7eac134ec0 0x7f7eac133900 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f7eac06da40 tx=0x7f7ea0004750 comp rx=0 tx=0).stop 2023-09-30T22:32:44.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.420+0000 7f7eb28d1700 1 -- 172.21.15.16:0/4055398878 shutdown_connections 2023-09-30T22:32:44.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.420+0000 7f7eb28d1700 1 --2- 172.21.15.16:0/4055398878 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7eac137f20 0x7f7eac133e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:44.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.420+0000 7f7eb28d1700 1 --2- 172.21.15.16:0/4055398878 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7eac06c0e0 0x7f7eac1333c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:44.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.420+0000 7f7eb28d1700 1 --2- 172.21.15.16:0/4055398878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7eac134ec0 0x7f7eac133900 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:44.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.420+0000 7f7eb28d1700 1 -- 172.21.15.16:0/4055398878 >> 172.21.15.16:0/4055398878 conn(0x7f7eac0a4210 msgr2=0x7f7eac06c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:44.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.420+0000 7f7eb28d1700 1 -- 172.21.15.16:0/4055398878 shutdown_connections 2023-09-30T22:32:44.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.420+0000 7f7eb28d1700 1 -- 172.21.15.16:0/4055398878 wait complete. 2023-09-30T22:32:44.436 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:32:44.442 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:32:44.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.513+0000 7f4df5083700 1 -- 172.21.15.16:0/1035240906 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4df01a0b30 con 0x7f4df009f650 2023-09-30T22:32:44.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.514+0000 7f4dd77fe700 1 -- 172.21.15.16:0/1035240906 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4de8025b70 con 0x7f4df009f650 2023-09-30T22:32:44.515 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:32:44.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.516+0000 7f4df5083700 1 -- 172.21.15.16:0/1035240906 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4df009f650 msgr2=0x7f4df006c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:44.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.516+0000 7f4df5083700 1 --2- 172.21.15.16:0/1035240906 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4df009f650 0x7f4df006c330 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f4de800b370 tx=0x7f4de8004ad0 comp rx=0 tx=0).stop 2023-09-30T22:32:44.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.516+0000 7f4df5083700 1 -- 172.21.15.16:0/1035240906 shutdown_connections 2023-09-30T22:32:44.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.516+0000 7f4df5083700 1 --2- 172.21.15.16:0/1035240906 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4df009f650 0x7f4df006c330 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:44.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.516+0000 7f4df5083700 1 --2- 172.21.15.16:0/1035240906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4df009e930 0x7f4df006b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:44.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.516+0000 7f4df5083700 1 --2- 172.21.15.16:0/1035240906 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4df009f240 0x7f4df006bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:44.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.516+0000 7f4df5083700 1 -- 172.21.15.16:0/1035240906 >> 172.21.15.16:0/1035240906 conn(0x7f4df00a4210 msgr2=0x7f4df01b8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:44.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.516+0000 7f4df5083700 1 -- 172.21.15.16:0/1035240906 shutdown_connections 2023-09-30T22:32:44.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.516+0000 7f4df5083700 1 -- 172.21.15.16:0/1035240906 wait complete. 2023-09-30T22:32:44.531 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:32:44.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.579+0000 7fc2225c7700 1 -- 172.21.15.16:0/1527698728 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc21c123470 con 0x7fc21c1338c0 2023-09-30T22:32:44.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.579+0000 7fc218ff9700 1 -- 172.21.15.16:0/1527698728 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc2100233e0 con 0x7fc21c1338c0 2023-09-30T22:32:44.580 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:32:44.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.581+0000 7fc2225c7700 1 -- 172.21.15.16:0/1527698728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc21c1338c0 msgr2=0x7fc21c11ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:44.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.581+0000 7fc2225c7700 1 --2- 172.21.15.16:0/1527698728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc21c1338c0 0x7fc21c11ffe0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fc210000c00 tx=0x7fc210004750 comp rx=0 tx=0).stop 2023-09-30T22:32:44.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.581+0000 7fc2225c7700 1 -- 172.21.15.16:0/1527698728 shutdown_connections 2023-09-30T22:32:44.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.581+0000 7fc2225c7700 1 --2- 172.21.15.16:0/1527698728 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc21c09e390 0x7fc21c11faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:44.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.581+0000 7fc2225c7700 1 --2- 172.21.15.16:0/1527698728 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc21c1341d0 0x7fc21c120520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:44.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.581+0000 7fc2225c7700 1 --2- 172.21.15.16:0/1527698728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc21c1338c0 0x7fc21c11ffe0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:44.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.581+0000 7fc2225c7700 1 -- 172.21.15.16:0/1527698728 >> 172.21.15.16:0/1527698728 conn(0x7fc21c0a2aa0 msgr2=0x7fc21c0677c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:44.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.582+0000 7fc2225c7700 1 -- 172.21.15.16:0/1527698728 shutdown_connections 2023-09-30T22:32:44.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:44.582+0000 7fc2225c7700 1 -- 172.21.15.16:0/1527698728 wait complete. 2023-09-30T22:32:44.595 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:32:45.436 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:32:45.443 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:32:45.531 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:32:45.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.587+0000 7f05877e7700 1 Processor -- start 2023-09-30T22:32:45.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.587+0000 7f05877e7700 1 -- start start 2023-09-30T22:32:45.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.588+0000 7f05877e7700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f058009d6f0 0x7f058009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:45.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.588+0000 7f05877e7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f058009e440 0x7f0580134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:45.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.588+0000 7f05877e7700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f058009b120 con 0x7f058009e000 2023-09-30T22:32:45.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.588+0000 7f05877e7700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f058006b460 con 0x7f058009d6f0 2023-09-30T22:32:45.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.588+0000 7f05877e7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f058009a7e0 con 0x7f058009e440 2023-09-30T22:32:45.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.588+0000 7f0585d84700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f058009e440 0x7f0580134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:45.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.588+0000 7f0584d82700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f058009e000 0x7f05801339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53718/0 (socket says 172.21.15.16:53718) 2023-09-30T22:32:45.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.588+0000 7f0584d82700 1 -- 172.21.15.16:0/750329206 learned_addr learned my addr 172.21.15.16:0/750329206 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:45.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.588+0000 7f0585583700 1 --2- 172.21.15.16:0/750329206 >> v2:172.21.15.16:3300/0 conn(0x7f058009d6f0 0x7f058009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:45.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.588+0000 7f0585583700 1 -- 172.21.15.16:0/750329206 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f058009e440 msgr2=0x7f0580134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:45.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.588+0000 7f0585583700 1 --2- 172.21.15.16:0/750329206 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f058009e440 0x7f0580134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:45.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.588+0000 7f0585583700 1 -- 172.21.15.16:0/750329206 >> v1:172.21.15.16:6789/0 conn(0x7f058009e000 legacy=0x7f05801339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:45.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.588+0000 7f0577fff700 1 -- 172.21.15.16:0/750329206 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2251470902 0 0) 0x7f058009b120 con 0x7f058009e000 2023-09-30T22:32:45.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.588+0000 7f0585583700 1 -- 172.21.15.16:0/750329206 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f058009a500 con 0x7f058009d6f0 2023-09-30T22:32:45.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.588+0000 7f0585d84700 1 --2- 172.21.15.16:0/750329206 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f058009e440 0x7f0580134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:45.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.589+0000 7f0585583700 1 --2- 172.21.15.16:0/750329206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f058009d6f0 0x7f058009dac0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f057c009870 tx=0x7f057c009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0548e3f5d3eb0fb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:45.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.589+0000 7f0577fff700 1 -- 172.21.15.16:0/750329206 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057c011830 con 0x7f058009d6f0 2023-09-30T22:32:45.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.589+0000 7f0577fff700 1 -- 172.21.15.16:0/750329206 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f057c011990 con 0x7f058009d6f0 2023-09-30T22:32:45.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.589+0000 7f0577fff700 1 -- 172.21.15.16:0/750329206 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057c011d20 con 0x7f058009d6f0 2023-09-30T22:32:45.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.590+0000 7f05877e7700 1 -- 172.21.15.16:0/750329206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f058009d6f0 msgr2=0x7f058009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:45.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.590+0000 7f05877e7700 1 --2- 172.21.15.16:0/750329206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f058009d6f0 0x7f058009dac0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f057c009870 tx=0x7f057c009b80 comp rx=0 tx=0).stop 2023-09-30T22:32:45.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.590+0000 7f05877e7700 1 -- 172.21.15.16:0/750329206 shutdown_connections 2023-09-30T22:32:45.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.590+0000 7f05877e7700 1 --2- 172.21.15.16:0/750329206 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f058009e440 0x7f0580134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:45.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.590+0000 7f05877e7700 1 --2- 172.21.15.16:0/750329206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f058009d6f0 0x7f058009dac0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:45.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.590+0000 7f05877e7700 1 -- 172.21.15.16:0/750329206 >> 172.21.15.16:0/750329206 conn(0x7f0580066f50 msgr2=0x7f058009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:45.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.590+0000 7f05877e7700 1 -- 172.21.15.16:0/750329206 shutdown_connections 2023-09-30T22:32:45.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.590+0000 7f05877e7700 1 -- 172.21.15.16:0/750329206 wait complete. 2023-09-30T22:32:45.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.591+0000 7f05877e7700 1 Processor -- start 2023-09-30T22:32:45.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.591+0000 7f05877e7700 1 -- start start 2023-09-30T22:32:45.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.592+0000 7f05877e7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f058009d6f0 0x7f05801a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:45.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.592+0000 7f05877e7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f058009e000 0x7f05801a58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:45.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.592+0000 7f05877e7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f058009e440 0x7f05801c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:45.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.592+0000 7f05877e7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05801a8110 con 0x7f058009e000 2023-09-30T22:32:45.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.592+0000 7f05877e7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05801b9180 con 0x7f058009d6f0 2023-09-30T22:32:45.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.592+0000 7f05877e7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f05801a8310 con 0x7f058009e440 2023-09-30T22:32:45.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.592+0000 7f0584d82700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f058009e000 0x7f05801a58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:45.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.592+0000 7f0584d82700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f058009e000 0x7f05801a58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36056/0 (socket says 172.21.15.16:36056) 2023-09-30T22:32:45.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.592+0000 7f0584d82700 1 -- 172.21.15.16:0/353802123 learned_addr learned my addr 172.21.15.16:0/353802123 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:45.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.592+0000 7f0585583700 1 --2- 172.21.15.16:0/353802123 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f058009d6f0 0x7f05801a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:45.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.592+0000 7f0585d84700 1 --2- 172.21.15.16:0/353802123 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f058009e440 0x7f05801c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:45.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.593+0000 7f0585583700 1 -- 172.21.15.16:0/353802123 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f058009e440 msgr2=0x7f05801c7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:45.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.593+0000 7f0585583700 1 --2- 172.21.15.16:0/353802123 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f058009e440 0x7f05801c7fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:45.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.593+0000 7f0585583700 1 -- 172.21.15.16:0/353802123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f058009e000 msgr2=0x7f05801a58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:45.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.593+0000 7f0585583700 1 --2- 172.21.15.16:0/353802123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f058009e000 0x7f05801a58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:45.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.593+0000 7f0585583700 1 -- 172.21.15.16:0/353802123 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0578009a70 con 0x7f058009d6f0 2023-09-30T22:32:45.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.593+0000 7f0584d82700 1 --2- 172.21.15.16:0/353802123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f058009e000 0x7f05801a58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:45.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.593+0000 7f0585583700 1 --2- 172.21.15.16:0/353802123 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f058009d6f0 0x7f05801a5380 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f057c016000 tx=0x7f057c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:45.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.593+0000 7f0575ffb700 1 -- 172.21.15.16:0/353802123 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057c0045d0 con 0x7f058009d6f0 2023-09-30T22:32:45.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.593+0000 7f0575ffb700 1 -- 172.21.15.16:0/353802123 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f057c01f070 con 0x7f058009d6f0 2023-09-30T22:32:45.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.593+0000 7f0575ffb700 1 -- 172.21.15.16:0/353802123 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057c0038a0 con 0x7f058009d6f0 2023-09-30T22:32:45.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.593+0000 7f05877e7700 1 -- 172.21.15.16:0/353802123 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f057c0094a0 con 0x7f058009d6f0 2023-09-30T22:32:45.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.594+0000 7f05877e7700 1 -- 172.21.15.16:0/353802123 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05801a4d90 con 0x7f058009d6f0 2023-09-30T22:32:45.596 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:32:45.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.594+0000 7f0575ffb700 1 -- 172.21.15.16:0/353802123 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f057c003a00 con 0x7f058009d6f0 2023-09-30T22:32:45.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.595+0000 7f05877e7700 1 -- 172.21.15.16:0/353802123 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f058009c710 con 0x7f058009d6f0 2023-09-30T22:32:45.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.595+0000 7f0575ffb700 1 -- 172.21.15.16:0/353802123 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f057c021070 con 0x7f058009d6f0 2023-09-30T22:32:45.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.600+0000 7f0575ffb700 1 -- 172.21.15.16:0/353802123 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f057c010970 con 0x7f058009d6f0 2023-09-30T22:32:45.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.629+0000 7fce00f35700 1 Processor -- start 2023-09-30T22:32:45.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.629+0000 7fce00f35700 1 -- start start 2023-09-30T22:32:45.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.630+0000 7fce00f35700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcdfc136710 0x7fcdfc138af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:45.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.630+0000 7fce00f35700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcdfc139030 0x7fcdfc13b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:45.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.630+0000 7fce00f35700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdfc06ef30 con 0x7fcdfc1338a0 2023-09-30T22:32:45.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.630+0000 7fce00f35700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdfc06fef0 con 0x7fcdfc136710 2023-09-30T22:32:45.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.630+0000 7fce00f35700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdfc1334b0 con 0x7fcdfc139030 2023-09-30T22:32:45.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.630+0000 7fcdf9d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcdfc136710 0x7fcdfc138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:45.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.630+0000 7fcdf9d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcdfc136710 0x7fcdfc138af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36068/0 (socket says 172.21.15.16:36068) 2023-09-30T22:32:45.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.630+0000 7fcdf9d9b700 1 -- 172.21.15.16:0/3185325471 learned_addr learned my addr 172.21.15.16:0/3185325471 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:45.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.630+0000 7fcdfad9d700 1 --2- 172.21.15.16:0/3185325471 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcdfc139030 0x7fcdfc13b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:45.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.631+0000 7fcdf9d9b700 1 -- 172.21.15.16:0/3185325471 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcdfc139030 msgr2=0x7fcdfc13b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:45.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.631+0000 7fcdf9d9b700 1 --2- 172.21.15.16:0/3185325471 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcdfc139030 0x7fcdfc13b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:45.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.631+0000 7fcdf9d9b700 1 -- 172.21.15.16:0/3185325471 >> v1:172.21.15.16:6789/0 conn(0x7fcdfc1338a0 legacy=0x7fcdfc133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:45.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.631+0000 7fcdf9d9b700 1 -- 172.21.15.16:0/3185325471 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdfc06fc30 con 0x7fcdfc136710 2023-09-30T22:32:45.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.631+0000 7fcdf9d9b700 1 --2- 172.21.15.16:0/3185325471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdfc136710 0x7fcdfc138af0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fcdf00098f0 tx=0x7fcdf0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3db423fb04a6a88 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:45.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.631+0000 7fcdf959a700 1 -- 172.21.15.16:0/3185325471 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf0004230 con 0x7fcdfc136710 2023-09-30T22:32:45.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.631+0000 7fcdf959a700 1 -- 172.21.15.16:0/3185325471 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdf0004390 con 0x7fcdfc136710 2023-09-30T22:32:45.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.631+0000 7fcdf959a700 1 -- 172.21.15.16:0/3185325471 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf00036a0 con 0x7fcdfc136710 2023-09-30T22:32:45.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.632+0000 7fce00f35700 1 -- 172.21.15.16:0/3185325471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdfc136710 msgr2=0x7fcdfc138af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:45.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.632+0000 7fce00f35700 1 --2- 172.21.15.16:0/3185325471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdfc136710 0x7fcdfc138af0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fcdf00098f0 tx=0x7fcdf0009c00 comp rx=0 tx=0).stop 2023-09-30T22:32:45.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.632+0000 7fce00f35700 1 -- 172.21.15.16:0/3185325471 shutdown_connections 2023-09-30T22:32:45.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.632+0000 7fce00f35700 1 --2- 172.21.15.16:0/3185325471 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcdfc139030 0x7fcdfc13b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:45.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.632+0000 7fce00f35700 1 --2- 172.21.15.16:0/3185325471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdfc136710 0x7fcdfc138af0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:45.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.632+0000 7fce00f35700 1 -- 172.21.15.16:0/3185325471 >> 172.21.15.16:0/3185325471 conn(0x7fcdfc0a2aa0 msgr2=0x7fcdfc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:45.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.633+0000 7fce00f35700 1 -- 172.21.15.16:0/3185325471 shutdown_connections 2023-09-30T22:32:45.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.633+0000 7fce00f35700 1 -- 172.21.15.16:0/3185325471 wait complete. 2023-09-30T22:32:45.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.634+0000 7fce00f35700 1 Processor -- start 2023-09-30T22:32:45.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.634+0000 7fce00f35700 1 -- start start 2023-09-30T22:32:45.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.635+0000 7fce00f35700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcdfc1338a0 0x7fcdfc1c84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:45.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.635+0000 7fce00f35700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcdfc136710 0x7fcdfc1c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:45.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.635+0000 7fce00f35700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdfc139030 0x7fcdfc11ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:45.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.635+0000 7fce00f35700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdfc11e620 con 0x7fcdfc139030 2023-09-30T22:32:45.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.635+0000 7fce00f35700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdfc1be110 con 0x7fcdfc136710 2023-09-30T22:32:45.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.635+0000 7fce00f35700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcdfc11c910 con 0x7fcdfc1338a0 2023-09-30T22:32:45.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.635+0000 7fcdf9d9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcdfc136710 0x7fcdfc1c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:45.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.635+0000 7fcdfa59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcdfc1338a0 0x7fcdfc1c84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:45.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.635+0000 7fcdf9d9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcdfc136710 0x7fcdfc1c8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:39618/0 (socket says 172.21.15.16:39618) 2023-09-30T22:32:45.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.635+0000 7fcdf9d9b700 1 -- 172.21.15.16:0/2395763622 learned_addr learned my addr 172.21.15.16:0/2395763622 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:45.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.636+0000 7fcdfad9d700 1 --2- 172.21.15.16:0/2395763622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdfc139030 0x7fcdfc11ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:45.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.636+0000 7fcdf9d9b700 1 -- 172.21.15.16:0/2395763622 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcdfc1338a0 msgr2=0x7fcdfc1c84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:45.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.636+0000 7fcdf9d9b700 1 --2- 172.21.15.16:0/2395763622 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcdfc1338a0 0x7fcdfc1c84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:45.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.636+0000 7fcdf9d9b700 1 -- 172.21.15.16:0/2395763622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdfc139030 msgr2=0x7fcdfc11ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:45.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.636+0000 7fcdf9d9b700 1 --2- 172.21.15.16:0/2395763622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdfc139030 0x7fcdfc11ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:45.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.636+0000 7fcdf9d9b700 1 -- 172.21.15.16:0/2395763622 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf0009550 con 0x7fcdfc136710 2023-09-30T22:32:45.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.636+0000 7fcdf9d9b700 1 --2- 172.21.15.16:0/2395763622 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcdfc136710 0x7fcdfc1c8a10 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0000c00 tx=0x7fcdf0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:45.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.636+0000 7fcdeaffd700 1 -- 172.21.15.16:0/2395763622 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf0003c50 con 0x7fcdfc136710 2023-09-30T22:32:45.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.636+0000 7fcdeaffd700 1 -- 172.21.15.16:0/2395763622 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdf001f070 con 0x7fcdfc136710 2023-09-30T22:32:45.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.636+0000 7fcdeaffd700 1 -- 172.21.15.16:0/2395763622 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf0011630 con 0x7fcdfc136710 2023-09-30T22:32:45.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.637+0000 7fce00f35700 1 -- 172.21.15.16:0/2395763622 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdfc130c30 con 0x7fcdfc136710 2023-09-30T22:32:45.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.637+0000 7fce00f35700 1 -- 172.21.15.16:0/2395763622 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdfc11ef90 con 0x7fcdfc136710 2023-09-30T22:32:45.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.638+0000 7fce00f35700 1 -- 172.21.15.16:0/2395763622 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdfc074ea0 con 0x7fcdfc136710 2023-09-30T22:32:45.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.638+0000 7fcdeaffd700 1 -- 172.21.15.16:0/2395763622 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcdf0021030 con 0x7fcdfc136710 2023-09-30T22:32:45.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.638+0000 7fcdeaffd700 1 -- 172.21.15.16:0/2395763622 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fcdf0040c30 con 0x7fcdfc136710 2023-09-30T22:32:45.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.648+0000 7fcdeaffd700 1 -- 172.21.15.16:0/2395763622 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcdf00199b0 con 0x7fcdfc136710 2023-09-30T22:32:45.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.679+0000 7fdcfad5f700 1 Processor -- start 2023-09-30T22:32:45.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.679+0000 7fdcfad5f700 1 -- start start 2023-09-30T22:32:45.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.680+0000 7fdcfad5f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdcf412cf60 0x7fdcf406f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:45.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.680+0000 7fdcfad5f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdcf412f8b0 0x7fdcf412fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:45.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.680+0000 7fdcfad5f700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcf412cd90 con 0x7fdcf4130260 2023-09-30T22:32:45.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.680+0000 7fdcfad5f700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcf412d330 con 0x7fdcf412cf60 2023-09-30T22:32:45.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.680+0000 7fdcfad5f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf41306d0 con 0x7fdcf412f8b0 2023-09-30T22:32:45.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.680+0000 7fdcf3fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdcf412f8b0 0x7fdcf412fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:45.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.680+0000 7fdcf3fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdcf412f8b0 0x7fdcf412fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:39626/0 (socket says 172.21.15.16:39626) 2023-09-30T22:32:45.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.680+0000 7fdcf3fff700 1 -- 172.21.15.16:0/337209219 learned_addr learned my addr 172.21.15.16:0/337209219 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:45.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.680+0000 7fdcf92fc700 1 --1- 172.21.15.16:0/337209219 >> v1:172.21.15.16:6789/0 conn(0x7fdcf4130260 0x7fdcf4136c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53748/0 (socket says 172.21.15.16:53748) 2023-09-30T22:32:45.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.680+0000 7fdcf8afb700 1 --2- 172.21.15.16:0/337209219 >> v2:172.21.15.16:3300/0 conn(0x7fdcf412cf60 0x7fdcf406f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:45.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.681+0000 7fdcf37fe700 1 -- 172.21.15.16:0/337209219 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 651172223 0 0) 0x7fdcf412cd90 con 0x7fdcf4130260 2023-09-30T22:32:45.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.681+0000 7fdcf37fe700 1 -- 172.21.15.16:0/337209219 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdce40035b0 con 0x7fdcf4130260 2023-09-30T22:32:45.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.681+0000 7fdcf3fff700 1 -- 172.21.15.16:0/337209219 >> v2:172.21.15.16:3300/0 conn(0x7fdcf412cf60 msgr2=0x7fdcf406f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:45.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.681+0000 7fdcf3fff700 1 --2- 172.21.15.16:0/337209219 >> v2:172.21.15.16:3300/0 conn(0x7fdcf412cf60 0x7fdcf406f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:45.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.681+0000 7fdcf3fff700 1 -- 172.21.15.16:0/337209219 >> v1:172.21.15.16:6789/0 conn(0x7fdcf4130260 legacy=0x7fdcf4136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:45.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.681+0000 7fdcf3fff700 1 -- 172.21.15.16:0/337209219 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf4071780 con 0x7fdcf412f8b0 2023-09-30T22:32:45.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.681+0000 7fdcf3fff700 1 --2- 172.21.15.16:0/337209219 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdcf412f8b0 0x7fdcf412fd20 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fdce00098f0 tx=0x7fdce0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fceeb3e943145394 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:45.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.681+0000 7fdcf37fe700 1 -- 172.21.15.16:0/337209219 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce0011830 con 0x7fdcf412f8b0 2023-09-30T22:32:45.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.681+0000 7fdcf37fe700 1 -- 172.21.15.16:0/337209219 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdce0011990 con 0x7fdcf412f8b0 2023-09-30T22:32:45.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.681+0000 7fdcf37fe700 1 -- 172.21.15.16:0/337209219 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce0011d20 con 0x7fdcf412f8b0 2023-09-30T22:32:45.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.682+0000 7fdcfad5f700 1 -- 172.21.15.16:0/337209219 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdcf412f8b0 msgr2=0x7fdcf412fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:45.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.682+0000 7fdcfad5f700 1 --2- 172.21.15.16:0/337209219 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdcf412f8b0 0x7fdcf412fd20 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fdce00098f0 tx=0x7fdce0009c00 comp rx=0 tx=0).stop 2023-09-30T22:32:45.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.682+0000 7fdcfad5f700 1 -- 172.21.15.16:0/337209219 shutdown_connections 2023-09-30T22:32:45.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.682+0000 7fdcfad5f700 1 --2- 172.21.15.16:0/337209219 >> v2:172.21.15.16:3300/0 conn(0x7fdcf412cf60 0x7fdcf406f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:45.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.682+0000 7fdcfad5f700 1 --2- 172.21.15.16:0/337209219 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdcf412f8b0 0x7fdcf412fd20 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:45.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.682+0000 7fdcfad5f700 1 -- 172.21.15.16:0/337209219 >> 172.21.15.16:0/337209219 conn(0x7fdcf40a2aa0 msgr2=0x7fdcf40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:45.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.682+0000 7fdcfad5f700 1 -- 172.21.15.16:0/337209219 shutdown_connections 2023-09-30T22:32:45.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.682+0000 7fdcfad5f700 1 -- 172.21.15.16:0/337209219 wait complete. 2023-09-30T22:32:45.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.683+0000 7fdcfad5f700 1 Processor -- start 2023-09-30T22:32:45.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.683+0000 7fdcfad5f700 1 -- start start 2023-09-30T22:32:45.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.684+0000 7fdcfad5f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdcf412cf60 0x7fdcf409bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:45.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.684+0000 7fdcfad5f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcf412f8b0 0x7fdcf409c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:45.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.684+0000 7fdcfad5f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdcf4130260 0x7fdcf41c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:45.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.684+0000 7fdcfad5f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf409e960 con 0x7fdcf412f8b0 2023-09-30T22:32:45.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.684+0000 7fdcfad5f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf409c5d0 con 0x7fdcf4130260 2023-09-30T22:32:45.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.684+0000 7fdcfad5f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf409ed40 con 0x7fdcf412cf60 2023-09-30T22:32:45.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.684+0000 7fdcf3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcf412f8b0 0x7fdcf409c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:45.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.684+0000 7fdcf8afb700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdcf412cf60 0x7fdcf409bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:45.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.684+0000 7fdcf3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcf412f8b0 0x7fdcf409c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36088/0 (socket says 172.21.15.16:36088) 2023-09-30T22:32:45.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.684+0000 7fdcf3fff700 1 -- 172.21.15.16:0/2947870094 learned_addr learned my addr 172.21.15.16:0/2947870094 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:45.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.684+0000 7fdcf92fc700 1 --2- 172.21.15.16:0/2947870094 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdcf4130260 0x7fdcf41c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:45.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.684+0000 7fdcf3fff700 1 -- 172.21.15.16:0/2947870094 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdcf412cf60 msgr2=0x7fdcf409bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:45.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.684+0000 7fdcf3fff700 1 --2- 172.21.15.16:0/2947870094 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdcf412cf60 0x7fdcf409bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:45.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.684+0000 7fdcf3fff700 1 -- 172.21.15.16:0/2947870094 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdcf4130260 msgr2=0x7fdcf41c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:45.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.684+0000 7fdcf3fff700 1 --2- 172.21.15.16:0/2947870094 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdcf4130260 0x7fdcf41c8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:45.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.684+0000 7fdcf3fff700 1 -- 172.21.15.16:0/2947870094 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce0009550 con 0x7fdcf412f8b0 2023-09-30T22:32:45.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.684+0000 7fdcf92fc700 1 --2- 172.21.15.16:0/2947870094 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdcf4130260 0x7fdcf41c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:45.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.685+0000 7fdcf3fff700 1 --2- 172.21.15.16:0/2947870094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcf412f8b0 0x7fdcf409c090 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fdce0009c40 tx=0x7fdce0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:45.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.685+0000 7fdcf17fa700 1 -- 172.21.15.16:0/2947870094 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce0004480 con 0x7fdcf412f8b0 2023-09-30T22:32:45.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.685+0000 7fdcf17fa700 1 -- 172.21.15.16:0/2947870094 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdce001f070 con 0x7fdcf412f8b0 2023-09-30T22:32:45.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.685+0000 7fdcfad5f700 1 -- 172.21.15.16:0/2947870094 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcf409b740 con 0x7fdcf412f8b0 2023-09-30T22:32:45.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.685+0000 7fdcf17fa700 1 -- 172.21.15.16:0/2947870094 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce00038c0 con 0x7fdcf412f8b0 2023-09-30T22:32:45.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.685+0000 7fdcfad5f700 1 -- 172.21.15.16:0/2947870094 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcf409e6a0 con 0x7fdcf412f8b0 2023-09-30T22:32:45.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.686+0000 7fdcf17fa700 1 -- 172.21.15.16:0/2947870094 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdce0021030 con 0x7fdcf412f8b0 2023-09-30T22:32:45.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.686+0000 7fdcfad5f700 1 -- 172.21.15.16:0/2947870094 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcf409eb60 con 0x7fdcf412f8b0 2023-09-30T22:32:45.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.686+0000 7fdcf17fa700 1 -- 172.21.15.16:0/2947870094 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fdce0040990 con 0x7fdcf412f8b0 2023-09-30T22:32:45.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.696+0000 7fdcf17fa700 1 -- 172.21.15.16:0/2947870094 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdce0015070 con 0x7fdcf412f8b0 2023-09-30T22:32:45.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.745+0000 7f3fe5490700 1 Processor -- start 2023-09-30T22:32:45.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.745+0000 7f3fe5490700 1 -- start start 2023-09-30T22:32:45.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.746+0000 7f3fe5490700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3fe009d6f0 0x7f3fe009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:45.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.746+0000 7f3fe5490700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3fe009e410 0x7f3fe0130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:45.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.746+0000 7f3fe5490700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fe009f130 con 0x7f3fe009e000 2023-09-30T22:32:45.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.746+0000 7f3fe5490700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe009f520 con 0x7f3fe009d6f0 2023-09-30T22:32:45.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.746+0000 7f3fe5490700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe009e930 con 0x7f3fe009e410 2023-09-30T22:32:45.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.746+0000 7f3fde7fc700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3fe009e000 0x7f3fe0130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53752/0 (socket says 172.21.15.16:53752) 2023-09-30T22:32:45.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.746+0000 7f3fde7fc700 1 -- 172.21.15.16:0/362503952 learned_addr learned my addr 172.21.15.16:0/362503952 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:45.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.746+0000 7f3fdf7fe700 1 --2- 172.21.15.16:0/362503952 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3fe009e410 0x7f3fe0130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:45.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.746+0000 7f3fdeffd700 1 --2- 172.21.15.16:0/362503952 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3fe009d6f0 0x7f3fe009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:45.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.747+0000 7f3fddffb700 1 -- 172.21.15.16:0/362503952 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2139737317 0 0) 0x7f3fe009f130 con 0x7f3fe009e000 2023-09-30T22:32:45.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.747+0000 7f3fdf7fe700 1 -- 172.21.15.16:0/362503952 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3fe009d6f0 msgr2=0x7f3fe009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:45.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.747+0000 7f3fdf7fe700 1 --2- 172.21.15.16:0/362503952 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3fe009d6f0 0x7f3fe009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:45.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.747+0000 7f3fdf7fe700 1 -- 172.21.15.16:0/362503952 >> v1:172.21.15.16:6789/0 conn(0x7f3fe009e000 legacy=0x7f3fe0130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:45.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.747+0000 7f3fdf7fe700 1 -- 172.21.15.16:0/362503952 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe009b2d0 con 0x7f3fe009e410 2023-09-30T22:32:45.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.747+0000 7f3fdf7fe700 1 --2- 172.21.15.16:0/362503952 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3fe009e410 0x7f3fe0130d70 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8009dc0 tx=0x7f3fd800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=648171d64aad1932 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:45.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.747+0000 7f3fddffb700 1 -- 172.21.15.16:0/362503952 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd800e070 con 0x7f3fe009e410 2023-09-30T22:32:45.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.747+0000 7f3fddffb700 1 -- 172.21.15.16:0/362503952 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd800bbd0 con 0x7f3fe009e410 2023-09-30T22:32:45.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.747+0000 7f3fddffb700 1 -- 172.21.15.16:0/362503952 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd8012430 con 0x7f3fe009e410 2023-09-30T22:32:45.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.748+0000 7f3fe5490700 1 -- 172.21.15.16:0/362503952 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3fe009e410 msgr2=0x7f3fe0130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:45.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.748+0000 7f3fe5490700 1 --2- 172.21.15.16:0/362503952 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3fe009e410 0x7f3fe0130d70 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8009dc0 tx=0x7f3fd800b330 comp rx=0 tx=0).stop 2023-09-30T22:32:45.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.748+0000 7f3fe5490700 1 -- 172.21.15.16:0/362503952 shutdown_connections 2023-09-30T22:32:45.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.748+0000 7f3fe5490700 1 --2- 172.21.15.16:0/362503952 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3fe009e410 0x7f3fe0130d70 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:45.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.748+0000 7f3fe5490700 1 --2- 172.21.15.16:0/362503952 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3fe009d6f0 0x7f3fe009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:45.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.748+0000 7f3fe5490700 1 -- 172.21.15.16:0/362503952 >> 172.21.15.16:0/362503952 conn(0x7f3fe0066f50 msgr2=0x7f3fe009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:45.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.748+0000 7f3fe5490700 1 -- 172.21.15.16:0/362503952 shutdown_connections 2023-09-30T22:32:45.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.748+0000 7f3fe5490700 1 -- 172.21.15.16:0/362503952 wait complete. 2023-09-30T22:32:45.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.750+0000 7f3fe5490700 1 Processor -- start 2023-09-30T22:32:45.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.750+0000 7f3fe5490700 1 -- start start 2023-09-30T22:32:45.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.751+0000 7f3fe5490700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3fe009d6f0 0x7f3fe01c8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:45.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.751+0000 7f3fe5490700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3fe009e000 0x7f3fe01c89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:45.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.751+0000 7f3fe5490700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fe009e410 0x7f3fe01c8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:45.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.751+0000 7f3fe5490700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe0125a80 con 0x7f3fe009e410 2023-09-30T22:32:45.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.751+0000 7f3fe5490700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe009c700 con 0x7f3fe009e000 2023-09-30T22:32:45.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.751+0000 7f3fe5490700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe006ebf0 con 0x7f3fe009d6f0 2023-09-30T22:32:45.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.751+0000 7f3fde7fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3fe009e000 0x7f3fe01c89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:45.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.751+0000 7f3fde7fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3fe009e000 0x7f3fe01c89b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:39638/0 (socket says 172.21.15.16:39638) 2023-09-30T22:32:45.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.751+0000 7f3fde7fc700 1 -- 172.21.15.16:0/1867147588 learned_addr learned my addr 172.21.15.16:0/1867147588 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:45.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.751+0000 7f3fdf7fe700 1 --2- 172.21.15.16:0/1867147588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fe009e410 0x7f3fe01c8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:45.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.751+0000 7f3fdeffd700 1 --2- 172.21.15.16:0/1867147588 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3fe009d6f0 0x7f3fe01c8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:45.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.752+0000 7f3fdf7fe700 1 -- 172.21.15.16:0/1867147588 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3fe009d6f0 msgr2=0x7f3fe01c8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:45.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.752+0000 7f3fdf7fe700 1 --2- 172.21.15.16:0/1867147588 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3fe009d6f0 0x7f3fe01c8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:45.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.752+0000 7f3fdf7fe700 1 -- 172.21.15.16:0/1867147588 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3fe009e000 msgr2=0x7f3fe01c89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:45.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.752+0000 7f3fdf7fe700 1 --2- 172.21.15.16:0/1867147588 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3fe009e000 0x7f3fe01c89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:45.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.752+0000 7f3fdf7fe700 1 -- 172.21.15.16:0/1867147588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd8009a70 con 0x7f3fe009e410 2023-09-30T22:32:45.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.752+0000 7f3fdeffd700 1 --2- 172.21.15.16:0/1867147588 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3fe009d6f0 0x7f3fe01c8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:45.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.752+0000 7f3fdf7fe700 1 --2- 172.21.15.16:0/1867147588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fe009e410 0x7f3fe01c8ef0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8016000 tx=0x7f3fd8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:45.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.752+0000 7f3fc77fe700 1 -- 172.21.15.16:0/1867147588 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd8012430 con 0x7f3fe009e410 2023-09-30T22:32:45.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.752+0000 7f3fc77fe700 1 -- 172.21.15.16:0/1867147588 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd8020070 con 0x7f3fe009e410 2023-09-30T22:32:45.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.752+0000 7f3fe5490700 1 -- 172.21.15.16:0/1867147588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fe006b280 con 0x7f3fe009e410 2023-09-30T22:32:45.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.752+0000 7f3fc77fe700 1 -- 172.21.15.16:0/1867147588 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd8004280 con 0x7f3fe009e410 2023-09-30T22:32:45.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.752+0000 7f3fe5490700 1 -- 172.21.15.16:0/1867147588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fe006a9e0 con 0x7f3fe009e410 2023-09-30T22:32:45.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.754+0000 7f3fe5490700 1 -- 172.21.15.16:0/1867147588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fe0074ea0 con 0x7f3fe009e410 2023-09-30T22:32:45.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.754+0000 7f3fc77fe700 1 -- 172.21.15.16:0/1867147588 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3fd80043e0 con 0x7f3fe009e410 2023-09-30T22:32:45.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.755+0000 7f3fc77fe700 1 -- 172.21.15.16:0/1867147588 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f3fd8022070 con 0x7f3fe009e410 2023-09-30T22:32:45.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.759+0000 7f3fc77fe700 1 -- 172.21.15.16:0/1867147588 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3fd8012e60 con 0x7f3fe009e410 2023-09-30T22:32:45.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.832+0000 7f05877e7700 1 -- 172.21.15.16:0/353802123 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f058009f420 con 0x7f058009d6f0 2023-09-30T22:32:45.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.833+0000 7f0575ffb700 1 -- 172.21.15.16:0/353802123 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f057c015030 con 0x7f058009d6f0 2023-09-30T22:32:45.833 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:32:45.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.835+0000 7f05877e7700 1 -- 172.21.15.16:0/353802123 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f058009d6f0 msgr2=0x7f05801a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:45.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.835+0000 7f05877e7700 1 --2- 172.21.15.16:0/353802123 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f058009d6f0 0x7f05801a5380 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f057c016000 tx=0x7f057c004750 comp rx=0 tx=0).stop 2023-09-30T22:32:45.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.835+0000 7f05877e7700 1 -- 172.21.15.16:0/353802123 shutdown_connections 2023-09-30T22:32:45.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.835+0000 7f05877e7700 1 --2- 172.21.15.16:0/353802123 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f058009d6f0 0x7f05801a5380 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:45.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.835+0000 7f05877e7700 1 --2- 172.21.15.16:0/353802123 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f058009e440 0x7f05801c7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:45.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.835+0000 7f05877e7700 1 --2- 172.21.15.16:0/353802123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f058009e000 0x7f05801a58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:45.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.835+0000 7f05877e7700 1 -- 172.21.15.16:0/353802123 >> 172.21.15.16:0/353802123 conn(0x7f0580066f50 msgr2=0x7f05801bbab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:45.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.836+0000 7f05877e7700 1 -- 172.21.15.16:0/353802123 shutdown_connections 2023-09-30T22:32:45.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.836+0000 7f05877e7700 1 -- 172.21.15.16:0/353802123 wait complete. 2023-09-30T22:32:45.851 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:32:45.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.864+0000 7fce00f35700 1 -- 172.21.15.16:0/2395763622 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fcdfc130bd0 con 0x7fcdfc136710 2023-09-30T22:32:45.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.864+0000 7fcdeaffd700 1 -- 172.21.15.16:0/2395763622 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcdf002d430 con 0x7fcdfc136710 2023-09-30T22:32:45.865 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:32:45.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.866+0000 7fce00f35700 1 -- 172.21.15.16:0/2395763622 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcdfc136710 msgr2=0x7fcdfc1c8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:45.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.866+0000 7fce00f35700 1 --2- 172.21.15.16:0/2395763622 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcdfc136710 0x7fcdfc1c8a10 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0000c00 tx=0x7fcdf0004750 comp rx=0 tx=0).stop 2023-09-30T22:32:45.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.866+0000 7fce00f35700 1 -- 172.21.15.16:0/2395763622 shutdown_connections 2023-09-30T22:32:45.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.866+0000 7fce00f35700 1 --2- 172.21.15.16:0/2395763622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdfc139030 0x7fcdfc11ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:45.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.866+0000 7fce00f35700 1 --2- 172.21.15.16:0/2395763622 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcdfc1338a0 0x7fcdfc1c84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:45.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.866+0000 7fce00f35700 1 --2- 172.21.15.16:0/2395763622 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcdfc136710 0x7fcdfc1c8a10 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:45.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.867+0000 7fce00f35700 1 -- 172.21.15.16:0/2395763622 >> 172.21.15.16:0/2395763622 conn(0x7fcdfc0a2aa0 msgr2=0x7fcdfc137170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:45.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.867+0000 7fce00f35700 1 -- 172.21.15.16:0/2395763622 shutdown_connections 2023-09-30T22:32:45.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.867+0000 7fce00f35700 1 -- 172.21.15.16:0/2395763622 wait complete. 2023-09-30T22:32:45.881 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:32:45.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.920+0000 7fdcfad5f700 1 -- 172.21.15.16:0/2947870094 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fdcf4074ea0 con 0x7fdcf412f8b0 2023-09-30T22:32:45.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.920+0000 7fdcf17fa700 1 -- 172.21.15.16:0/2947870094 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdce0003a20 con 0x7fdcf412f8b0 2023-09-30T22:32:45.921 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:32:45.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.923+0000 7fdcfad5f700 1 -- 172.21.15.16:0/2947870094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcf412f8b0 msgr2=0x7fdcf409c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:45.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.923+0000 7fdcfad5f700 1 --2- 172.21.15.16:0/2947870094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcf412f8b0 0x7fdcf409c090 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fdce0009c40 tx=0x7fdce0004750 comp rx=0 tx=0).stop 2023-09-30T22:32:45.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.923+0000 7fdcfad5f700 1 -- 172.21.15.16:0/2947870094 shutdown_connections 2023-09-30T22:32:45.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.923+0000 7fdcfad5f700 1 --2- 172.21.15.16:0/2947870094 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdcf4130260 0x7fdcf41c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:45.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.923+0000 7fdcfad5f700 1 --2- 172.21.15.16:0/2947870094 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdcf412cf60 0x7fdcf409bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:45.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.923+0000 7fdcfad5f700 1 --2- 172.21.15.16:0/2947870094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdcf412f8b0 0x7fdcf409c090 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:45.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.923+0000 7fdcfad5f700 1 -- 172.21.15.16:0/2947870094 >> 172.21.15.16:0/2947870094 conn(0x7fdcf40a2aa0 msgr2=0x7fdcf412a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:45.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.923+0000 7fdcfad5f700 1 -- 172.21.15.16:0/2947870094 shutdown_connections 2023-09-30T22:32:45.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.923+0000 7fdcfad5f700 1 -- 172.21.15.16:0/2947870094 wait complete. 2023-09-30T22:32:45.937 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:32:45.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.968+0000 7f3fe5490700 1 -- 172.21.15.16:0/1867147588 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3fe006c360 con 0x7f3fe009e410 2023-09-30T22:32:45.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.968+0000 7f3fc77fe700 1 -- 172.21.15.16:0/1867147588 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3fd8035090 con 0x7f3fe009e410 2023-09-30T22:32:45.969 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:32:45.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.970+0000 7f3fe5490700 1 -- 172.21.15.16:0/1867147588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fe009e410 msgr2=0x7f3fe01c8ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:45.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.970+0000 7f3fe5490700 1 --2- 172.21.15.16:0/1867147588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fe009e410 0x7f3fe01c8ef0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8016000 tx=0x7f3fd8004ad0 comp rx=0 tx=0).stop 2023-09-30T22:32:45.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.970+0000 7f3fe5490700 1 -- 172.21.15.16:0/1867147588 shutdown_connections 2023-09-30T22:32:45.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.970+0000 7f3fe5490700 1 --2- 172.21.15.16:0/1867147588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fe009e410 0x7f3fe01c8ef0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:45.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.970+0000 7f3fe5490700 1 --2- 172.21.15.16:0/1867147588 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3fe009d6f0 0x7f3fe01c8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:45.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.970+0000 7f3fe5490700 1 --2- 172.21.15.16:0/1867147588 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3fe009e000 0x7f3fe01c89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:45.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.970+0000 7f3fe5490700 1 -- 172.21.15.16:0/1867147588 >> 172.21.15.16:0/1867147588 conn(0x7f3fe0066f50 msgr2=0x7f3fe0130b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:45.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.971+0000 7f3fe5490700 1 -- 172.21.15.16:0/1867147588 shutdown_connections 2023-09-30T22:32:45.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:45.971+0000 7f3fe5490700 1 -- 172.21.15.16:0/1867147588 wait complete. 2023-09-30T22:32:45.984 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:32:46.851 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:32:46.882 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:32:46.937 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:32:46.985 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:32:47.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.014+0000 7f4a36dc7700 1 Processor -- start 2023-09-30T22:32:47.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.014+0000 7f4a36dc7700 1 -- start start 2023-09-30T22:32:47.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.015+0000 7f4a36dc7700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4a30133540 0x7f4a30135920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:47.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.015+0000 7f4a36dc7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4a30135e60 0x7f4a30138240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:47.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.015+0000 7f4a36dc7700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a300716a0 con 0x7f4a30130540 2023-09-30T22:32:47.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.015+0000 7f4a36dc7700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a30071290 con 0x7f4a30133540 2023-09-30T22:32:47.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.015+0000 7f4a36dc7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4a30130340 con 0x7f4a30135e60 2023-09-30T22:32:47.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.015+0000 7f4a34b63700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4a30130540 0x7f4a30130910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53766/0 (socket says 172.21.15.16:53766) 2023-09-30T22:32:47.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.015+0000 7f4a34b63700 1 -- 172.21.15.16:0/748806856 learned_addr learned my addr 172.21.15.16:0/748806856 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:47.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.015+0000 7f4a2ffff700 1 --2- 172.21.15.16:0/748806856 >> v2:172.21.15.16:3300/0 conn(0x7f4a30133540 0x7f4a30135920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:47.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.015+0000 7f4a2f7fe700 1 -- 172.21.15.16:0/748806856 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 997945599 0 0) 0x7f4a300716a0 con 0x7f4a30130540 2023-09-30T22:32:47.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.016+0000 7f4a2f7fe700 1 -- 172.21.15.16:0/748806856 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a200057c0 con 0x7f4a30130540 2023-09-30T22:32:47.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.016+0000 7f4a35364700 1 --2- 172.21.15.16:0/748806856 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4a30135e60 0x7f4a30138240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:47.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.016+0000 7f4a2ffff700 1 -- 172.21.15.16:0/748806856 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4a30135e60 msgr2=0x7f4a30138240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:47.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.016+0000 7f4a2ffff700 1 --2- 172.21.15.16:0/748806856 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4a30135e60 0x7f4a30138240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:47.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.016+0000 7f4a2ffff700 1 -- 172.21.15.16:0/748806856 >> v1:172.21.15.16:6789/0 conn(0x7f4a30130540 legacy=0x7f4a30130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:47.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.016+0000 7f4a2ffff700 1 -- 172.21.15.16:0/748806856 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a30070b80 con 0x7f4a30133540 2023-09-30T22:32:47.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.016+0000 7f4a35364700 1 --2- 172.21.15.16:0/748806856 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4a30135e60 0x7f4a30138240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:32:47.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.016+0000 7f4a2ffff700 1 --2- 172.21.15.16:0/748806856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a30133540 0x7f4a30135920 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c009840 tx=0x7f4a1c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5726b04a7d667507 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:47.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.016+0000 7f4a2f7fe700 1 -- 172.21.15.16:0/748806856 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c011830 con 0x7f4a30133540 2023-09-30T22:32:47.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.016+0000 7f4a2f7fe700 1 -- 172.21.15.16:0/748806856 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a1c011990 con 0x7f4a30133540 2023-09-30T22:32:47.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.016+0000 7f4a2f7fe700 1 -- 172.21.15.16:0/748806856 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c011d20 con 0x7f4a30133540 2023-09-30T22:32:47.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.017+0000 7f4a36dc7700 1 -- 172.21.15.16:0/748806856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a30133540 msgr2=0x7f4a30135920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:47.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.017+0000 7f4a36dc7700 1 --2- 172.21.15.16:0/748806856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a30133540 0x7f4a30135920 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c009840 tx=0x7f4a1c009b50 comp rx=0 tx=0).stop 2023-09-30T22:32:47.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.017+0000 7f4a36dc7700 1 -- 172.21.15.16:0/748806856 shutdown_connections 2023-09-30T22:32:47.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.017+0000 7f4a36dc7700 1 --2- 172.21.15.16:0/748806856 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4a30135e60 0x7f4a30138240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:47.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.017+0000 7f4a36dc7700 1 --2- 172.21.15.16:0/748806856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a30133540 0x7f4a30135920 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:47.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.017+0000 7f4a36dc7700 1 -- 172.21.15.16:0/748806856 >> 172.21.15.16:0/748806856 conn(0x7f4a300a2aa0 msgr2=0x7f4a300a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:47.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.017+0000 7f4a36dc7700 1 -- 172.21.15.16:0/748806856 shutdown_connections 2023-09-30T22:32:47.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.017+0000 7f4a36dc7700 1 -- 172.21.15.16:0/748806856 wait complete. 2023-09-30T22:32:47.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.019+0000 7f4a36dc7700 1 Processor -- start 2023-09-30T22:32:47.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.019+0000 7f4a36dc7700 1 -- start start 2023-09-30T22:32:47.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.020+0000 7f4a36dc7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a30130540 0x7f4a3009c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:47.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.020+0000 7f4a36dc7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4a30133540 0x7f4a3009c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:47.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.020+0000 7f4a36dc7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4a30135e60 0x7f4a3009cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:47.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.020+0000 7f4a36dc7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a300a21f0 con 0x7f4a30130540 2023-09-30T22:32:47.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.020+0000 7f4a36dc7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a300718e0 con 0x7f4a30133540 2023-09-30T22:32:47.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.020+0000 7f4a36dc7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4a30119a30 con 0x7f4a30135e60 2023-09-30T22:32:47.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.020+0000 7f4a34b63700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a30130540 0x7f4a3009c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:47.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.020+0000 7f4a34b63700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a30130540 0x7f4a3009c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36110/0 (socket says 172.21.15.16:36110) 2023-09-30T22:32:47.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.020+0000 7f4a34b63700 1 -- 172.21.15.16:0/2911236253 learned_addr learned my addr 172.21.15.16:0/2911236253 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:47.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.020+0000 7f4a35364700 1 --2- 172.21.15.16:0/2911236253 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4a30135e60 0x7f4a3009cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:47.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.020+0000 7f4a2ffff700 1 --2- 172.21.15.16:0/2911236253 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4a30133540 0x7f4a3009c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:47.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.021+0000 7f4a2ffff700 1 -- 172.21.15.16:0/2911236253 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4a30135e60 msgr2=0x7f4a3009cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:47.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.021+0000 7f4a2ffff700 1 --2- 172.21.15.16:0/2911236253 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4a30135e60 0x7f4a3009cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:47.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.021+0000 7f4a2ffff700 1 -- 172.21.15.16:0/2911236253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a30130540 msgr2=0x7f4a3009c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:47.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.021+0000 7f4a2ffff700 1 --2- 172.21.15.16:0/2911236253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a30130540 0x7f4a3009c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:47.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.021+0000 7f4a2ffff700 1 -- 172.21.15.16:0/2911236253 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a1c0094a0 con 0x7f4a30133540 2023-09-30T22:32:47.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.021+0000 7f4a34b63700 1 --2- 172.21.15.16:0/2911236253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a30130540 0x7f4a3009c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:47.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.021+0000 7f4a2ffff700 1 --2- 172.21.15.16:0/2911236253 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4a30133540 0x7f4a3009c9b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c016000 tx=0x7f4a1c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:47.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.021+0000 7f4a2d7fa700 1 -- 172.21.15.16:0/2911236253 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c0036a0 con 0x7f4a30133540 2023-09-30T22:32:47.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.021+0000 7f4a2d7fa700 1 -- 172.21.15.16:0/2911236253 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a1c01f070 con 0x7f4a30133540 2023-09-30T22:32:47.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.021+0000 7f4a2d7fa700 1 -- 172.21.15.16:0/2911236253 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c003a40 con 0x7f4a30133540 2023-09-30T22:32:47.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.022+0000 7f4a36dc7700 1 -- 172.21.15.16:0/2911236253 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a3011a5f0 con 0x7f4a30133540 2023-09-30T22:32:47.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.022+0000 7f4a36dc7700 1 -- 172.21.15.16:0/2911236253 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a3011ad80 con 0x7f4a30133540 2023-09-30T22:32:47.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.023+0000 7f4a2d7fa700 1 -- 172.21.15.16:0/2911236253 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4a1c003ba0 con 0x7f4a30133540 2023-09-30T22:32:47.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.023+0000 7f4a36dc7700 1 -- 172.21.15.16:0/2911236253 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a3011a170 con 0x7f4a30133540 2023-09-30T22:32:47.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.023+0000 7f4a2d7fa700 1 -- 172.21.15.16:0/2911236253 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f4a1c021070 con 0x7f4a30133540 2023-09-30T22:32:47.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.033+0000 7f4a2d7fa700 1 -- 172.21.15.16:0/2911236253 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4a3011a170 con 0x7f4a30133540 2023-09-30T22:32:47.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.045+0000 7f323bd70700 1 Processor -- start 2023-09-30T22:32:47.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.046+0000 7f323bd70700 1 -- start start 2023-09-30T22:32:47.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.046+0000 7f323bd70700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f323406a770 0x7f323409d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:47.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.046+0000 7f323bd70700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f323409dc30 0x7f323409e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:47.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.046+0000 7f323bd70700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f323409a810 con 0x7f323413c1e0 2023-09-30T22:32:47.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.046+0000 7f323bd70700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f323406ab40 con 0x7f323409dc30 2023-09-30T22:32:47.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.046+0000 7f323bd70700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f323409ebc0 con 0x7f323406a770 2023-09-30T22:32:47.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.046+0000 7f323a30d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f323413c1e0 0x7f323413e5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53772/0 (socket says 172.21.15.16:53772) 2023-09-30T22:32:47.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.046+0000 7f323a30d700 1 -- 172.21.15.16:0/4093436047 learned_addr learned my addr 172.21.15.16:0/4093436047 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:47.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.047+0000 7f3239b0c700 1 --2- 172.21.15.16:0/4093436047 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f323406a770 0x7f323409d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:47.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.047+0000 7f323930b700 1 --2- 172.21.15.16:0/4093436047 >> v2:172.21.15.16:3300/0 conn(0x7f323409dc30 0x7f323409e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:47.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.047+0000 7f3238b0a700 1 -- 172.21.15.16:0/4093436047 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2481119781 0 0) 0x7f323409a810 con 0x7f323413c1e0 2023-09-30T22:32:47.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.047+0000 7f3238b0a700 1 -- 172.21.15.16:0/4093436047 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32200035b0 con 0x7f323413c1e0 2023-09-30T22:32:47.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.047+0000 7f3238b0a700 1 -- 172.21.15.16:0/4093436047 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1276809455 0 0) 0x7f32200035b0 con 0x7f323413c1e0 2023-09-30T22:32:47.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.047+0000 7f3238b0a700 1 -- 172.21.15.16:0/4093436047 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f323406a770 msgr2=0x7f323409d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:47.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.047+0000 7f3238b0a700 1 --2- 172.21.15.16:0/4093436047 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f323406a770 0x7f323409d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:47.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.047+0000 7f3238b0a700 1 -- 172.21.15.16:0/4093436047 >> v2:172.21.15.16:3300/0 conn(0x7f323409dc30 msgr2=0x7f323409e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:47.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.047+0000 7f3238b0a700 1 --2- 172.21.15.16:0/4093436047 >> v2:172.21.15.16:3300/0 conn(0x7f323409dc30 0x7f323409e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:47.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.047+0000 7f3238b0a700 1 -- 172.21.15.16:0/4093436047 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f323409e920 con 0x7f323413c1e0 2023-09-30T22:32:47.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.047+0000 7f3238b0a700 1 -- 172.21.15.16:0/4093436047 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1074163509 0 0) 0x7f322c0019f0 con 0x7f323413c1e0 2023-09-30T22:32:47.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.047+0000 7f3238b0a700 1 -- 172.21.15.16:0/4093436047 >> v1:172.21.15.16:6789/0 conn(0x7f323413c1e0 legacy=0x7f323413e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:47.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.047+0000 7f3238b0a700 1 --2- 172.21.15.16:0/4093436047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3220003a00 0x7f3220003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:47.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.047+0000 7f3238b0a700 1 --2- 172.21.15.16:0/4093436047 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3220004430 0x7f32200048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:47.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.047+0000 7f3238b0a700 1 --2- 172.21.15.16:0/4093436047 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f323406a770 0x7f3220005020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:47.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.047+0000 7f3238b0a700 1 -- 172.21.15.16:0/4093436047 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f323406ab40 con 0x7f3220003a00 2023-09-30T22:32:47.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.047+0000 7f3238b0a700 1 -- 172.21.15.16:0/4093436047 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f323409ebc0 con 0x7f323406a770 2023-09-30T22:32:47.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.047+0000 7f3238b0a700 1 -- 172.21.15.16:0/4093436047 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f322000ba10 con 0x7f3220004430 2023-09-30T22:32:47.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.047+0000 7f3239b0c700 1 --2- 172.21.15.16:0/4093436047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3220003a00 0x7f3220003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:47.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.048+0000 7f323930b700 1 --2- 172.21.15.16:0/4093436047 >> v2:172.21.15.16:3300/0 conn(0x7f323409dc30 0x7f323409e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:47.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.048+0000 7f3239b0c700 1 -- 172.21.15.16:0/4093436047 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3220004430 msgr2=0x7f32200048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:47.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.048+0000 7f3239b0c700 1 --2- 172.21.15.16:0/4093436047 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3220004430 0x7f32200048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:47.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.048+0000 7f3239b0c700 1 -- 172.21.15.16:0/4093436047 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f323406a770 msgr2=0x7f3220005020 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T22:32:47.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.048+0000 7f3239b0c700 1 --2- 172.21.15.16:0/4093436047 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f323406a770 0x7f3220005020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:47.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.048+0000 7f3239b0c700 1 -- 172.21.15.16:0/4093436047 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32200035b0 con 0x7f3220003a00 2023-09-30T22:32:47.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.048+0000 7f3239b0c700 1 --2- 172.21.15.16:0/4093436047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3220003a00 0x7f3220003e10 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f3230009b60 tx=0x7f3220005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4426a84d7b843a9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:47.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.048+0000 7f3238b0a700 1 -- 172.21.15.16:0/4093436047 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323000e6c0 con 0x7f3220003a00 2023-09-30T22:32:47.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.048+0000 7f3238b0a700 1 -- 172.21.15.16:0/4093436047 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f323000e820 con 0x7f3220003a00 2023-09-30T22:32:47.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.048+0000 7f3238b0a700 1 -- 172.21.15.16:0/4093436047 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323000eb90 con 0x7f3220003a00 2023-09-30T22:32:47.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.049+0000 7f323bd70700 1 -- 172.21.15.16:0/4093436047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3220003a00 msgr2=0x7f3220003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:47.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.049+0000 7f323bd70700 1 --2- 172.21.15.16:0/4093436047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3220003a00 0x7f3220003e10 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f3230009b60 tx=0x7f3220005880 comp rx=0 tx=0).stop 2023-09-30T22:32:47.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.049+0000 7f323a30d700 1 -- 172.21.15.16:0/4093436047 reap_dead start 2023-09-30T22:32:47.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.049+0000 7f323bd70700 1 -- 172.21.15.16:0/4093436047 shutdown_connections 2023-09-30T22:32:47.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.049+0000 7f323bd70700 1 -- 172.21.15.16:0/4093436047 >> 172.21.15.16:0/4093436047 conn(0x7f3234066f50 msgr2=0x7f323409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:47.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.049+0000 7f323bd70700 1 -- 172.21.15.16:0/4093436047 shutdown_connections 2023-09-30T22:32:47.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.049+0000 7f323bd70700 1 -- 172.21.15.16:0/4093436047 wait complete. 2023-09-30T22:32:47.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.051+0000 7f323bd70700 1 Processor -- start 2023-09-30T22:32:47.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.051+0000 7f323bd70700 1 -- start start 2023-09-30T22:32:47.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.052+0000 7f323bd70700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3220003a00 0x7f323413e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:47.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.052+0000 7f323bd70700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f323413a640 0x7f323413aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:47.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.052+0000 7f323bd70700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f323413afd0 0x7f3234133c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:47.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.052+0000 7f323bd70700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3234134d10 con 0x7f323413afd0 2023-09-30T22:32:47.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.052+0000 7f323bd70700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f323411df40 con 0x7f323413a640 2023-09-30T22:32:47.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.052+0000 7f323bd70700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f323409f1a0 con 0x7f3220003a00 2023-09-30T22:32:47.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.052+0000 7f3239b0c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3220003a00 0x7f323413e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:47.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.052+0000 7f3239b0c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3220003a00 0x7f323413e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:39338/0 (socket says 172.21.15.16:39338) 2023-09-30T22:32:47.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.052+0000 7f3239b0c700 1 -- 172.21.15.16:0/1812730342 learned_addr learned my addr 172.21.15.16:0/1812730342 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:47.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.052+0000 7f323930b700 1 --2- 172.21.15.16:0/1812730342 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f323413a640 0x7f323413aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:47.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.052+0000 7f323a30d700 1 --2- 172.21.15.16:0/1812730342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f323413afd0 0x7f3234133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:47.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.052+0000 7f3239b0c700 1 -- 172.21.15.16:0/1812730342 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f323413a640 msgr2=0x7f323413aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:47.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.052+0000 7f3239b0c700 1 --2- 172.21.15.16:0/1812730342 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f323413a640 0x7f323413aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:47.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.052+0000 7f3239b0c700 1 -- 172.21.15.16:0/1812730342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f323413afd0 msgr2=0x7f3234133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:47.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.052+0000 7f3239b0c700 1 --2- 172.21.15.16:0/1812730342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f323413afd0 0x7f3234133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:47.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.052+0000 7f3239b0c700 1 -- 172.21.15.16:0/1812730342 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f323413edd0 con 0x7f3220003a00 2023-09-30T22:32:47.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.052+0000 7f323930b700 1 --2- 172.21.15.16:0/1812730342 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f323413a640 0x7f323413aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:47.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.053+0000 7f3239b0c700 1 --2- 172.21.15.16:0/1812730342 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3220003a00 0x7f323413e1d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3220005730 tx=0x7f32300047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:47.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.053+0000 7f322a7fc700 1 -- 172.21.15.16:0/1812730342 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3230021070 con 0x7f3220003a00 2023-09-30T22:32:47.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.053+0000 7f322a7fc700 1 -- 172.21.15.16:0/1812730342 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3230004470 con 0x7f3220003a00 2023-09-30T22:32:47.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.053+0000 7f322a7fc700 1 -- 172.21.15.16:0/1812730342 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f323000e6c0 con 0x7f3220003a00 2023-09-30T22:32:47.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.053+0000 7f323bd70700 1 -- 172.21.15.16:0/1812730342 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32341400b0 con 0x7f3220003a00 2023-09-30T22:32:47.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.053+0000 7f323bd70700 1 -- 172.21.15.16:0/1812730342 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f323413b420 con 0x7f3220003a00 2023-09-30T22:32:47.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.054+0000 7f323bd70700 1 -- 172.21.15.16:0/1812730342 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3234074ea0 con 0x7f3220003a00 2023-09-30T22:32:47.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.055+0000 7f322a7fc700 1 -- 172.21.15.16:0/1812730342 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3230025070 con 0x7f3220003a00 2023-09-30T22:32:47.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.055+0000 7f322a7fc700 1 -- 172.21.15.16:0/1812730342 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f3230040b00 con 0x7f3220003a00 2023-09-30T22:32:47.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.064+0000 7f322a7fc700 1 -- 172.21.15.16:0/1812730342 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f32300038e0 con 0x7f3220003a00 2023-09-30T22:32:47.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.088+0000 7fc162170700 1 Processor -- start 2023-09-30T22:32:47.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.088+0000 7fc162170700 1 -- start start 2023-09-30T22:32:47.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.089+0000 7fc162170700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc15c09d6f0 0x7fc15c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:47.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.089+0000 7fc162170700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc15c09e000 0x7fc15c130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:47.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.089+0000 7fc162170700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc15c09e410 0x7fc15c130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:47.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.089+0000 7fc162170700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc15c09f520 con 0x7fc15c09e410 2023-09-30T22:32:47.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.089+0000 7fc162170700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc15c09f130 con 0x7fc15c09d6f0 2023-09-30T22:32:47.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.089+0000 7fc162170700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc15c09e930 con 0x7fc15c09e000 2023-09-30T22:32:47.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.089+0000 7fc15b7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc15c09d6f0 0x7fc15c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:47.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.089+0000 7fc15bfff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc15c09e410 0x7fc15c130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:47.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.089+0000 7fc15b7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc15c09d6f0 0x7fc15c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:39668/0 (socket says 172.21.15.16:39668) 2023-09-30T22:32:47.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.089+0000 7fc15b7fe700 1 -- 172.21.15.16:0/1965726567 learned_addr learned my addr 172.21.15.16:0/1965726567 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:47.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.089+0000 7fc15affd700 1 --2- 172.21.15.16:0/1965726567 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc15c09e000 0x7fc15c130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:47.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.089+0000 7fc15bfff700 1 -- 172.21.15.16:0/1965726567 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc15c09e000 msgr2=0x7fc15c130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:47.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.089+0000 7fc15bfff700 1 --2- 172.21.15.16:0/1965726567 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc15c09e000 0x7fc15c130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:47.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.089+0000 7fc15bfff700 1 -- 172.21.15.16:0/1965726567 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc15c09d6f0 msgr2=0x7fc15c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:47.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.089+0000 7fc15bfff700 1 --2- 172.21.15.16:0/1965726567 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc15c09d6f0 0x7fc15c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:47.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.089+0000 7fc15bfff700 1 -- 172.21.15.16:0/1965726567 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc15c09b2d0 con 0x7fc15c09e410 2023-09-30T22:32:47.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.089+0000 7fc15b7fe700 1 --2- 172.21.15.16:0/1965726567 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc15c09d6f0 0x7fc15c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:47.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.090+0000 7fc15bfff700 1 --2- 172.21.15.16:0/1965726567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc15c09e410 0x7fc15c130ba0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fc154009d90 tx=0x7fc15400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87fc7ba882454f3d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:47.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.090+0000 7fc15a7fc700 1 -- 172.21.15.16:0/1965726567 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc15400e070 con 0x7fc15c09e410 2023-09-30T22:32:47.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.090+0000 7fc15a7fc700 1 -- 172.21.15.16:0/1965726567 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc15400bb20 con 0x7fc15c09e410 2023-09-30T22:32:47.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.090+0000 7fc15a7fc700 1 -- 172.21.15.16:0/1965726567 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc154012430 con 0x7fc15c09e410 2023-09-30T22:32:47.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.091+0000 7fc162170700 1 -- 172.21.15.16:0/1965726567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc15c09e410 msgr2=0x7fc15c130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:47.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.091+0000 7fc162170700 1 --2- 172.21.15.16:0/1965726567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc15c09e410 0x7fc15c130ba0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fc154009d90 tx=0x7fc15400b330 comp rx=0 tx=0).stop 2023-09-30T22:32:47.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.091+0000 7fc162170700 1 -- 172.21.15.16:0/1965726567 shutdown_connections 2023-09-30T22:32:47.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.091+0000 7fc162170700 1 --2- 172.21.15.16:0/1965726567 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc15c09d6f0 0x7fc15c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:47.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.091+0000 7fc162170700 1 --2- 172.21.15.16:0/1965726567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc15c09e410 0x7fc15c130ba0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:47.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.091+0000 7fc162170700 1 --2- 172.21.15.16:0/1965726567 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc15c09e000 0x7fc15c130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:47.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.091+0000 7fc162170700 1 -- 172.21.15.16:0/1965726567 >> 172.21.15.16:0/1965726567 conn(0x7fc15c066f50 msgr2=0x7fc15c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:47.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.091+0000 7fc162170700 1 -- 172.21.15.16:0/1965726567 shutdown_connections 2023-09-30T22:32:47.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.091+0000 7fc162170700 1 -- 172.21.15.16:0/1965726567 wait complete. 2023-09-30T22:32:47.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.092+0000 7fc162170700 1 Processor -- start 2023-09-30T22:32:47.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.092+0000 7fc162170700 1 -- start start 2023-09-30T22:32:47.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.092+0000 7fc162170700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc15c09d6f0 0x7fc15c1202c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:47.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.092+0000 7fc162170700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc15c09e000 0x7fc15c120800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:47.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.092+0000 7fc162170700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc15c09e410 0x7fc15c120d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:47.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.092+0000 7fc162170700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc15c121f30 con 0x7fc15c09e410 2023-09-30T22:32:47.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.092+0000 7fc162170700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc15c123ac0 con 0x7fc15c09d6f0 2023-09-30T22:32:47.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.092+0000 7fc162170700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc15c0a3af0 con 0x7fc15c09e000 2023-09-30T22:32:47.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.093+0000 7fc15affd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc15c09e000 0x7fc15c120800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:47.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.093+0000 7fc15affd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc15c09e000 0x7fc15c120800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:39364/0 (socket says 172.21.15.16:39364) 2023-09-30T22:32:47.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.093+0000 7fc15affd700 1 -- 172.21.15.16:0/1061202514 learned_addr learned my addr 172.21.15.16:0/1061202514 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:47.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.093+0000 7fc15bfff700 1 --2- 172.21.15.16:0/1061202514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc15c09e410 0x7fc15c120d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:47.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.093+0000 7fc15affd700 1 -- 172.21.15.16:0/1061202514 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc15c09d6f0 msgr2=0x7fc15c1202c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:32:47.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.093+0000 7fc15affd700 1 --2- 172.21.15.16:0/1061202514 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc15c09d6f0 0x7fc15c1202c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:47.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.093+0000 7fc15affd700 1 -- 172.21.15.16:0/1061202514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc15c09e410 msgr2=0x7fc15c120d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:47.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.093+0000 7fc15affd700 1 --2- 172.21.15.16:0/1061202514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc15c09e410 0x7fc15c120d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:47.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.093+0000 7fc15affd700 1 -- 172.21.15.16:0/1061202514 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc14c009550 con 0x7fc15c09e000 2023-09-30T22:32:47.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.093+0000 7fc15bfff700 1 --2- 172.21.15.16:0/1061202514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc15c09e410 0x7fc15c120d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:47.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.093+0000 7fc15affd700 1 --2- 172.21.15.16:0/1061202514 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc15c09e000 0x7fc15c120800 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc15000e980 tx=0x7fc15000ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:47.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.094+0000 7fc143fff700 1 -- 172.21.15.16:0/1061202514 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc150009710 con 0x7fc15c09e000 2023-09-30T22:32:47.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.094+0000 7fc143fff700 1 -- 172.21.15.16:0/1061202514 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc15000eed0 con 0x7fc15c09e000 2023-09-30T22:32:47.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.094+0000 7fc143fff700 1 -- 172.21.15.16:0/1061202514 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc15000ca70 con 0x7fc15c09e000 2023-09-30T22:32:47.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.094+0000 7fc162170700 1 -- 172.21.15.16:0/1061202514 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc154009a70 con 0x7fc15c09e000 2023-09-30T22:32:47.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.094+0000 7fc162170700 1 -- 172.21.15.16:0/1061202514 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc15c122680 con 0x7fc15c09e000 2023-09-30T22:32:47.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.095+0000 7fc143fff700 1 -- 172.21.15.16:0/1061202514 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc15000cbd0 con 0x7fc15c09e000 2023-09-30T22:32:47.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.095+0000 7fc143fff700 1 -- 172.21.15.16:0/1061202514 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fc15003fc20 con 0x7fc15c09e000 2023-09-30T22:32:47.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.095+0000 7fc162170700 1 -- 172.21.15.16:0/1061202514 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc15c123760 con 0x7fc15c09e000 2023-09-30T22:32:47.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.105+0000 7fc143fff700 1 -- 172.21.15.16:0/1061202514 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc150017020 con 0x7fc15c09e000 2023-09-30T22:32:47.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.153+0000 7faf2735d700 1 Processor -- start 2023-09-30T22:32:47.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.153+0000 7faf2735d700 1 -- start start 2023-09-30T22:32:47.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.154+0000 7faf2735d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faf2009d6f0 0x7faf2009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:47.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.154+0000 7faf2735d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf2009e440 0x7faf20134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:47.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.154+0000 7faf2735d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf2009b120 con 0x7faf2009e000 2023-09-30T22:32:47.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.154+0000 7faf2735d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf2006b460 con 0x7faf2009d6f0 2023-09-30T22:32:47.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.154+0000 7faf2735d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faf2009a7e0 con 0x7faf2009e440 2023-09-30T22:32:47.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.154+0000 7faf250f9700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faf2009d6f0 0x7faf2009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:47.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.154+0000 7faf248f8700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7faf2009e000 0x7faf201339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53784/0 (socket says 172.21.15.16:53784) 2023-09-30T22:32:47.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.154+0000 7faf248f8700 1 -- 172.21.15.16:0/4003201978 learned_addr learned my addr 172.21.15.16:0/4003201978 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:47.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.154+0000 7faf258fa700 1 --2- 172.21.15.16:0/4003201978 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf2009e440 0x7faf20134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:47.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.155+0000 7faf250f9700 1 -- 172.21.15.16:0/4003201978 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf2009e440 msgr2=0x7faf20134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:47.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.155+0000 7faf250f9700 1 --2- 172.21.15.16:0/4003201978 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf2009e440 0x7faf20134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:47.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.155+0000 7faf250f9700 1 -- 172.21.15.16:0/4003201978 >> v1:172.21.15.16:6789/0 conn(0x7faf2009e000 legacy=0x7faf201339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:47.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.155+0000 7faf17fff700 1 -- 172.21.15.16:0/4003201978 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 151295589 0 0) 0x7faf2009b120 con 0x7faf2009e000 2023-09-30T22:32:47.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.155+0000 7faf250f9700 1 -- 172.21.15.16:0/4003201978 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf2009a500 con 0x7faf2009d6f0 2023-09-30T22:32:47.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.155+0000 7faf258fa700 1 --2- 172.21.15.16:0/4003201978 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf2009e440 0x7faf20134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:47.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.155+0000 7faf250f9700 1 --2- 172.21.15.16:0/4003201978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf2009d6f0 0x7faf2009dac0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7faf1c009870 tx=0x7faf1c009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e43a43e7c9a2852 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:47.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.155+0000 7faf17fff700 1 -- 172.21.15.16:0/4003201978 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1c011830 con 0x7faf2009d6f0 2023-09-30T22:32:47.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.155+0000 7faf17fff700 1 -- 172.21.15.16:0/4003201978 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf1c011990 con 0x7faf2009d6f0 2023-09-30T22:32:47.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.155+0000 7faf17fff700 1 -- 172.21.15.16:0/4003201978 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1c011d20 con 0x7faf2009d6f0 2023-09-30T22:32:47.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.156+0000 7faf2735d700 1 -- 172.21.15.16:0/4003201978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf2009d6f0 msgr2=0x7faf2009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:47.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.156+0000 7faf2735d700 1 --2- 172.21.15.16:0/4003201978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf2009d6f0 0x7faf2009dac0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7faf1c009870 tx=0x7faf1c009b80 comp rx=0 tx=0).stop 2023-09-30T22:32:47.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.156+0000 7faf2735d700 1 -- 172.21.15.16:0/4003201978 shutdown_connections 2023-09-30T22:32:47.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.156+0000 7faf2735d700 1 --2- 172.21.15.16:0/4003201978 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf2009e440 0x7faf20134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:47.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.156+0000 7faf2735d700 1 --2- 172.21.15.16:0/4003201978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf2009d6f0 0x7faf2009dac0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:47.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.156+0000 7faf2735d700 1 -- 172.21.15.16:0/4003201978 >> 172.21.15.16:0/4003201978 conn(0x7faf20066f50 msgr2=0x7faf2009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:47.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.156+0000 7faf2735d700 1 -- 172.21.15.16:0/4003201978 shutdown_connections 2023-09-30T22:32:47.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.156+0000 7faf2735d700 1 -- 172.21.15.16:0/4003201978 wait complete. 2023-09-30T22:32:47.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.157+0000 7faf2735d700 1 Processor -- start 2023-09-30T22:32:47.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.158+0000 7faf2735d700 1 -- start start 2023-09-30T22:32:47.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.158+0000 7faf2735d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf2009d6f0 0x7faf201a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:47.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.158+0000 7faf2735d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf2009e000 0x7faf201a58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:47.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.158+0000 7faf2735d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf2009e440 0x7faf201c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:47.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.158+0000 7faf2735d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf201a8110 con 0x7faf2009e000 2023-09-30T22:32:47.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.158+0000 7faf2735d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf201b9180 con 0x7faf2009e440 2023-09-30T22:32:47.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.158+0000 7faf2735d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faf201a8310 con 0x7faf2009d6f0 2023-09-30T22:32:47.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.158+0000 7faf250f9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf2009d6f0 0x7faf201a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:47.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.158+0000 7faf248f8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf2009e000 0x7faf201a58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:47.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.158+0000 7faf250f9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf2009d6f0 0x7faf201a5380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:39386/0 (socket says 172.21.15.16:39386) 2023-09-30T22:32:47.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.158+0000 7faf250f9700 1 -- 172.21.15.16:0/138307578 learned_addr learned my addr 172.21.15.16:0/138307578 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:47.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.159+0000 7faf250f9700 1 -- 172.21.15.16:0/138307578 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf2009e440 msgr2=0x7faf201c7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:47.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.159+0000 7faf250f9700 1 --2- 172.21.15.16:0/138307578 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf2009e440 0x7faf201c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:47.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.159+0000 7faf250f9700 1 -- 172.21.15.16:0/138307578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf2009e000 msgr2=0x7faf201a58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:47.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.159+0000 7faf250f9700 1 --2- 172.21.15.16:0/138307578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf2009e000 0x7faf201a58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:47.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.159+0000 7faf250f9700 1 -- 172.21.15.16:0/138307578 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf18009a70 con 0x7faf2009d6f0 2023-09-30T22:32:47.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.159+0000 7faf250f9700 1 --2- 172.21.15.16:0/138307578 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf2009d6f0 0x7faf201a5380 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7faf1c0090b0 tx=0x7faf1c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:47.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.159+0000 7faf15ffb700 1 -- 172.21.15.16:0/138307578 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1c01a960 con 0x7faf2009d6f0 2023-09-30T22:32:47.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.159+0000 7faf15ffb700 1 -- 172.21.15.16:0/138307578 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf1c01aac0 con 0x7faf2009d6f0 2023-09-30T22:32:47.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.159+0000 7faf15ffb700 1 -- 172.21.15.16:0/138307578 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1c01ae50 con 0x7faf2009d6f0 2023-09-30T22:32:47.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.159+0000 7faf2735d700 1 -- 172.21.15.16:0/138307578 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf1c0094a0 con 0x7faf2009d6f0 2023-09-30T22:32:47.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.159+0000 7faf2735d700 1 -- 172.21.15.16:0/138307578 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf201a6400 con 0x7faf2009d6f0 2023-09-30T22:32:47.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.161+0000 7faf15ffb700 1 -- 172.21.15.16:0/138307578 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7faf1c0243f0 con 0x7faf2009d6f0 2023-09-30T22:32:47.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.161+0000 7faf2735d700 1 -- 172.21.15.16:0/138307578 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf2009c710 con 0x7faf2009d6f0 2023-09-30T22:32:47.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.161+0000 7faf15ffb700 1 -- 172.21.15.16:0/138307578 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7faf1c040d20 con 0x7faf2009d6f0 2023-09-30T22:32:47.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.166+0000 7faf15ffb700 1 -- 172.21.15.16:0/138307578 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7faf1c046020 con 0x7faf2009d6f0 2023-09-30T22:32:47.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.261+0000 7f4a36dc7700 1 -- 172.21.15.16:0/2911236253 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f4a3011a170 con 0x7f4a30133540 2023-09-30T22:32:47.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.261+0000 7f4a2d7fa700 1 -- 172.21.15.16:0/2911236253 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4a3011a170 con 0x7f4a30133540 2023-09-30T22:32:47.262 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:32:47.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.264+0000 7f4a36dc7700 1 -- 172.21.15.16:0/2911236253 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4a30133540 msgr2=0x7f4a3009c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:47.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.264+0000 7f4a36dc7700 1 --2- 172.21.15.16:0/2911236253 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4a30133540 0x7f4a3009c9b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c016000 tx=0x7f4a1c004750 comp rx=0 tx=0).stop 2023-09-30T22:32:47.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.265+0000 7f4a36dc7700 1 -- 172.21.15.16:0/2911236253 shutdown_connections 2023-09-30T22:32:47.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.265+0000 7f4a36dc7700 1 --2- 172.21.15.16:0/2911236253 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4a30135e60 0x7f4a3009cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:47.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.265+0000 7f4a36dc7700 1 --2- 172.21.15.16:0/2911236253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4a30130540 0x7f4a3009c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:47.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.265+0000 7f4a36dc7700 1 --2- 172.21.15.16:0/2911236253 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4a30133540 0x7f4a3009c9b0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:47.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.265+0000 7f4a36dc7700 1 -- 172.21.15.16:0/2911236253 >> 172.21.15.16:0/2911236253 conn(0x7f4a300a2aa0 msgr2=0x7f4a3012e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:47.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.265+0000 7f4a36dc7700 1 -- 172.21.15.16:0/2911236253 shutdown_connections 2023-09-30T22:32:47.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.265+0000 7f4a36dc7700 1 -- 172.21.15.16:0/2911236253 wait complete. 2023-09-30T22:32:47.285 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:32:47.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.285+0000 7f323bd70700 1 -- 172.21.15.16:0/1812730342 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f323409f1a0 con 0x7f3220003a00 2023-09-30T22:32:47.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.286+0000 7f322a7fc700 1 -- 172.21.15.16:0/1812730342 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3230017020 con 0x7f3220003a00 2023-09-30T22:32:47.286 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:32:47.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.288+0000 7f323bd70700 1 -- 172.21.15.16:0/1812730342 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3220003a00 msgr2=0x7f323413e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:47.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.288+0000 7f323bd70700 1 --2- 172.21.15.16:0/1812730342 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3220003a00 0x7f323413e1d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f3220005730 tx=0x7f32300047c0 comp rx=0 tx=0).stop 2023-09-30T22:32:47.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.288+0000 7f323bd70700 1 -- 172.21.15.16:0/1812730342 shutdown_connections 2023-09-30T22:32:47.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.288+0000 7f323bd70700 1 --2- 172.21.15.16:0/1812730342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f323413afd0 0x7f3234133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:47.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.288+0000 7f323bd70700 1 --2- 172.21.15.16:0/1812730342 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3220003a00 0x7f323413e1d0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:47.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.288+0000 7f323bd70700 1 --2- 172.21.15.16:0/1812730342 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f323413a640 0x7f323413aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:47.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.288+0000 7f323bd70700 1 -- 172.21.15.16:0/1812730342 >> 172.21.15.16:0/1812730342 conn(0x7f3234066f50 msgr2=0x7f32340a1650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:47.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.288+0000 7f323bd70700 1 -- 172.21.15.16:0/1812730342 shutdown_connections 2023-09-30T22:32:47.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.288+0000 7f323bd70700 1 -- 172.21.15.16:0/1812730342 wait complete. 2023-09-30T22:32:47.307 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:32:47.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.355+0000 7fc162170700 1 -- 172.21.15.16:0/1061202514 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc15c074ea0 con 0x7fc15c09e000 2023-09-30T22:32:47.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.355+0000 7fc143fff700 1 -- 172.21.15.16:0/1061202514 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc15001ae40 con 0x7fc15c09e000 2023-09-30T22:32:47.356 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:32:47.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.358+0000 7fc162170700 1 -- 172.21.15.16:0/1061202514 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc15c09e000 msgr2=0x7fc15c120800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:47.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.358+0000 7fc162170700 1 --2- 172.21.15.16:0/1061202514 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc15c09e000 0x7fc15c120800 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc15000e980 tx=0x7fc15000ec90 comp rx=0 tx=0).stop 2023-09-30T22:32:47.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.358+0000 7fc162170700 1 -- 172.21.15.16:0/1061202514 shutdown_connections 2023-09-30T22:32:47.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.358+0000 7fc162170700 1 --2- 172.21.15.16:0/1061202514 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc15c09d6f0 0x7fc15c1202c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:47.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.358+0000 7fc162170700 1 --2- 172.21.15.16:0/1061202514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc15c09e410 0x7fc15c120d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:47.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.358+0000 7fc162170700 1 --2- 172.21.15.16:0/1061202514 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc15c09e000 0x7fc15c120800 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:47.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.358+0000 7fc162170700 1 -- 172.21.15.16:0/1061202514 >> 172.21.15.16:0/1061202514 conn(0x7fc15c066f50 msgr2=0x7fc15c070ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:47.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.358+0000 7fc162170700 1 -- 172.21.15.16:0/1061202514 shutdown_connections 2023-09-30T22:32:47.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.358+0000 7fc162170700 1 -- 172.21.15.16:0/1061202514 wait complete. 2023-09-30T22:32:47.372 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:32:47.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.402+0000 7faf2735d700 1 -- 172.21.15.16:0/138307578 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7faf2009f420 con 0x7faf2009d6f0 2023-09-30T22:32:47.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.403+0000 7faf15ffb700 1 -- 172.21.15.16:0/138307578 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7faf1c015030 con 0x7faf2009d6f0 2023-09-30T22:32:47.403 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:32:47.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.405+0000 7faf2735d700 1 -- 172.21.15.16:0/138307578 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf2009d6f0 msgr2=0x7faf201a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:47.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.405+0000 7faf2735d700 1 --2- 172.21.15.16:0/138307578 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf2009d6f0 0x7faf201a5380 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7faf1c0090b0 tx=0x7faf1c004750 comp rx=0 tx=0).stop 2023-09-30T22:32:47.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.405+0000 7faf2735d700 1 -- 172.21.15.16:0/138307578 shutdown_connections 2023-09-30T22:32:47.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.405+0000 7faf2735d700 1 --2- 172.21.15.16:0/138307578 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf2009e440 0x7faf201c7fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:47.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.405+0000 7faf2735d700 1 --2- 172.21.15.16:0/138307578 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf2009d6f0 0x7faf201a5380 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:47.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.405+0000 7faf2735d700 1 --2- 172.21.15.16:0/138307578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf2009e000 0x7faf201a58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:47.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.405+0000 7faf2735d700 1 -- 172.21.15.16:0/138307578 >> 172.21.15.16:0/138307578 conn(0x7faf20066f50 msgr2=0x7faf201bbab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:47.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.405+0000 7faf2735d700 1 -- 172.21.15.16:0/138307578 shutdown_connections 2023-09-30T22:32:47.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:47.405+0000 7faf2735d700 1 -- 172.21.15.16:0/138307578 wait complete. 2023-09-30T22:32:47.418 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:32:48.285 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:32:48.308 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:32:48.372 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:32:48.420 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:32:48.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.442+0000 7fd34a0a2700 1 Processor -- start 2023-09-30T22:32:48.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.442+0000 7fd34a0a2700 1 -- start start 2023-09-30T22:32:48.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.443+0000 7fd34a0a2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd34412cf60 0x7fd34406f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:48.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.443+0000 7fd34a0a2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd34412f8b0 0x7fd34412fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:48.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.443+0000 7fd34a0a2700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd34412cd90 con 0x7fd344130260 2023-09-30T22:32:48.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.443+0000 7fd34a0a2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd34412d330 con 0x7fd34412f8b0 2023-09-30T22:32:48.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.443+0000 7fd34a0a2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd3441306d0 con 0x7fd34412cf60 2023-09-30T22:32:48.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.443+0000 7fd342ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd34412f8b0 0x7fd34412fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:48.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.443+0000 7fd342ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd34412f8b0 0x7fd34412fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37068/0 (socket says 172.21.15.16:37068) 2023-09-30T22:32:48.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.443+0000 7fd342ffd700 1 -- 172.21.15.16:0/2395251929 learned_addr learned my addr 172.21.15.16:0/2395251929 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:48.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.443+0000 7fd342ffd700 1 -- 172.21.15.16:0/2395251929 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd34412cf60 msgr2=0x7fd34406f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:48.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.443+0000 7fd342ffd700 1 --2- 172.21.15.16:0/2395251929 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd34412cf60 0x7fd34406f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:48.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.443+0000 7fd342ffd700 1 -- 172.21.15.16:0/2395251929 >> v1:172.21.15.16:6789/0 conn(0x7fd344130260 legacy=0x7fd344136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:48.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.443+0000 7fd342ffd700 1 -- 172.21.15.16:0/2395251929 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd344071780 con 0x7fd34412f8b0 2023-09-30T22:32:48.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.444+0000 7fd342ffd700 1 --2- 172.21.15.16:0/2395251929 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd34412f8b0 0x7fd34412fd20 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd33800bbc0 tx=0x7fd33800bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71d4766220001b55 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:48.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.444+0000 7fd3427fc700 1 -- 172.21.15.16:0/2395251929 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33800dc40 con 0x7fd34412f8b0 2023-09-30T22:32:48.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.444+0000 7fd3427fc700 1 -- 172.21.15.16:0/2395251929 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd33800dda0 con 0x7fd34412f8b0 2023-09-30T22:32:48.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.444+0000 7fd3427fc700 1 -- 172.21.15.16:0/2395251929 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd338004270 con 0x7fd34412f8b0 2023-09-30T22:32:48.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.445+0000 7fd34a0a2700 1 -- 172.21.15.16:0/2395251929 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd34412f8b0 msgr2=0x7fd34412fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:48.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.445+0000 7fd34a0a2700 1 --2- 172.21.15.16:0/2395251929 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd34412f8b0 0x7fd34412fd20 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd33800bbc0 tx=0x7fd33800bed0 comp rx=0 tx=0).stop 2023-09-30T22:32:48.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.445+0000 7fd34a0a2700 1 -- 172.21.15.16:0/2395251929 shutdown_connections 2023-09-30T22:32:48.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.445+0000 7fd34a0a2700 1 --2- 172.21.15.16:0/2395251929 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd34412cf60 0x7fd34406f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:48.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.445+0000 7fd34a0a2700 1 --2- 172.21.15.16:0/2395251929 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd34412f8b0 0x7fd34412fd20 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:48.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.445+0000 7fd34a0a2700 1 -- 172.21.15.16:0/2395251929 >> 172.21.15.16:0/2395251929 conn(0x7fd3440a2aa0 msgr2=0x7fd3440a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:48.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.445+0000 7fd34a0a2700 1 -- 172.21.15.16:0/2395251929 shutdown_connections 2023-09-30T22:32:48.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.445+0000 7fd34a0a2700 1 -- 172.21.15.16:0/2395251929 wait complete. 2023-09-30T22:32:48.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.446+0000 7fd34a0a2700 1 Processor -- start 2023-09-30T22:32:48.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.447+0000 7fd34a0a2700 1 -- start start 2023-09-30T22:32:48.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.447+0000 7fd34a0a2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd34412cf60 0x7fd344118cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:48.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.447+0000 7fd34a0a2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd34412f8b0 0x7fd344119210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:48.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.447+0000 7fd34a0a2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd344130260 0x7fd344119750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:48.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.447+0000 7fd34a0a2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd34411c250 con 0x7fd34412f8b0 2023-09-30T22:32:48.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.447+0000 7fd34a0a2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd34411b140 con 0x7fd344130260 2023-09-30T22:32:48.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.447+0000 7fd34a0a2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd344129bc0 con 0x7fd34412cf60 2023-09-30T22:32:48.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.447+0000 7fd3437fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd34412cf60 0x7fd344118cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:48.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.447+0000 7fd3437fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd34412cf60 0x7fd344118cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:47160/0 (socket says 172.21.15.16:47160) 2023-09-30T22:32:48.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.447+0000 7fd3437fe700 1 -- 172.21.15.16:0/2611855565 learned_addr learned my addr 172.21.15.16:0/2611855565 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:48.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.447+0000 7fd343fff700 1 --2- 172.21.15.16:0/2611855565 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd344130260 0x7fd344119750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:48.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.447+0000 7fd342ffd700 1 --2- 172.21.15.16:0/2611855565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd34412f8b0 0x7fd344119210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:48.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.448+0000 7fd343fff700 1 -- 172.21.15.16:0/2611855565 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd34412cf60 msgr2=0x7fd344118cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:48.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.448+0000 7fd343fff700 1 --2- 172.21.15.16:0/2611855565 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd34412cf60 0x7fd344118cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:48.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.448+0000 7fd343fff700 1 -- 172.21.15.16:0/2611855565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd34412f8b0 msgr2=0x7fd344119210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:48.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.448+0000 7fd343fff700 1 --2- 172.21.15.16:0/2611855565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd34412f8b0 0x7fd344119210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:48.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.448+0000 7fd343fff700 1 -- 172.21.15.16:0/2611855565 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd33800b7f0 con 0x7fd344130260 2023-09-30T22:32:48.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.448+0000 7fd3437fe700 1 --2- 172.21.15.16:0/2611855565 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd34412cf60 0x7fd344118cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:48.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.448+0000 7fd343fff700 1 --2- 172.21.15.16:0/2611855565 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd344130260 0x7fd344119750 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fd33c00c750 tx=0x7fd33c00ca60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:48.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.448+0000 7fd32bfff700 1 -- 172.21.15.16:0/2611855565 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33c003e10 con 0x7fd344130260 2023-09-30T22:32:48.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.448+0000 7fd32bfff700 1 -- 172.21.15.16:0/2611855565 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd33c0050d0 con 0x7fd344130260 2023-09-30T22:32:48.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.448+0000 7fd34a0a2700 1 -- 172.21.15.16:0/2611855565 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd34411bcb0 con 0x7fd344130260 2023-09-30T22:32:48.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.448+0000 7fd32bfff700 1 -- 172.21.15.16:0/2611855565 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33c005480 con 0x7fd344130260 2023-09-30T22:32:48.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.448+0000 7fd34a0a2700 1 -- 172.21.15.16:0/2611855565 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd344129fa0 con 0x7fd344130260 2023-09-30T22:32:48.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.449+0000 7fd32bfff700 1 -- 172.21.15.16:0/2611855565 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd33c0055e0 con 0x7fd344130260 2023-09-30T22:32:48.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.450+0000 7fd32bfff700 1 -- 172.21.15.16:0/2611855565 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd33c03ed00 con 0x7fd344130260 2023-09-30T22:32:48.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.451+0000 7fd34a0a2700 1 -- 172.21.15.16:0/2611855565 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd34409b4f0 con 0x7fd344130260 2023-09-30T22:32:48.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.460+0000 7fd32bfff700 1 -- 172.21.15.16:0/2611855565 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd33c017080 con 0x7fd344130260 2023-09-30T22:32:48.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.484+0000 7f8a145e1700 1 Processor -- start 2023-09-30T22:32:48.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.484+0000 7f8a145e1700 1 -- start start 2023-09-30T22:32:48.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.485+0000 7f8a145e1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8a0c06a770 0x7f8a0c09d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:48.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.485+0000 7f8a145e1700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8a0c09dc30 0x7f8a0c09e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:48.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.485+0000 7f8a145e1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8a0c13c1e0 0x7f8a0c13e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:48.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.485+0000 7f8a145e1700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a0c06ab40 con 0x7f8a0c09dc30 2023-09-30T22:32:48.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.485+0000 7f8a145e1700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c09ebc0 con 0x7f8a0c06a770 2023-09-30T22:32:48.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.485+0000 7f8a145e1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c09a810 con 0x7f8a0c13c1e0 2023-09-30T22:32:48.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.485+0000 7f8a12b7e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8a0c13c1e0 0x7f8a0c13e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:48.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.486+0000 7f8a12b7e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8a0c13c1e0 0x7f8a0c13e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:47166/0 (socket says 172.21.15.16:47166) 2023-09-30T22:32:48.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.486+0000 7f8a12b7e700 1 -- 172.21.15.16:0/88703765 learned_addr learned my addr 172.21.15.16:0/88703765 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:48.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.486+0000 7f8a1237d700 1 --2- 172.21.15.16:0/88703765 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8a0c06a770 0x7f8a0c09d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:48.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.486+0000 7f8a11b7c700 1 --2- 172.21.15.16:0/88703765 >> v2:172.21.15.16:3300/0 conn(0x7f8a0c09dc30 0x7f8a0c09e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:48.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.486+0000 7f8a12b7e700 1 -- 172.21.15.16:0/88703765 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8a0c06a770 msgr2=0x7f8a0c09d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:48.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.486+0000 7f8a12b7e700 1 --2- 172.21.15.16:0/88703765 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8a0c06a770 0x7f8a0c09d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:48.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.486+0000 7f8a12b7e700 1 -- 172.21.15.16:0/88703765 >> v2:172.21.15.16:3300/0 conn(0x7f8a0c09dc30 msgr2=0x7f8a0c09e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:48.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.486+0000 7f8a12b7e700 1 --2- 172.21.15.16:0/88703765 >> v2:172.21.15.16:3300/0 conn(0x7f8a0c09dc30 0x7f8a0c09e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:48.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.486+0000 7f8a12b7e700 1 -- 172.21.15.16:0/88703765 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a0c09e920 con 0x7f8a0c13c1e0 2023-09-30T22:32:48.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.486+0000 7f8a12b7e700 1 --2- 172.21.15.16:0/88703765 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8a0c13c1e0 0x7f8a0c13e5c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8a04009e80 tx=0x7f8a0400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d591bc404f78d7b4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:48.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.486+0000 7f8a1137b700 1 -- 172.21.15.16:0/88703765 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0400e070 con 0x7f8a0c13c1e0 2023-09-30T22:32:48.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.486+0000 7f8a1137b700 1 -- 172.21.15.16:0/88703765 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a0400b8d0 con 0x7f8a0c13c1e0 2023-09-30T22:32:48.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.487+0000 7f8a1137b700 1 -- 172.21.15.16:0/88703765 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0400bc60 con 0x7f8a0c13c1e0 2023-09-30T22:32:48.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.487+0000 7f8a145e1700 1 -- 172.21.15.16:0/88703765 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8a0c13c1e0 msgr2=0x7f8a0c13e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:48.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.487+0000 7f8a145e1700 1 --2- 172.21.15.16:0/88703765 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8a0c13c1e0 0x7f8a0c13e5c0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f8a04009e80 tx=0x7f8a0400b330 comp rx=0 tx=0).stop 2023-09-30T22:32:48.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.488+0000 7f8a145e1700 1 -- 172.21.15.16:0/88703765 shutdown_connections 2023-09-30T22:32:48.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.488+0000 7f8a145e1700 1 --2- 172.21.15.16:0/88703765 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8a0c06a770 0x7f8a0c09d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:48.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.488+0000 7f8a145e1700 1 --2- 172.21.15.16:0/88703765 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8a0c13c1e0 0x7f8a0c13e5c0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:48.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.488+0000 7f8a145e1700 1 --2- 172.21.15.16:0/88703765 >> v2:172.21.15.16:3300/0 conn(0x7f8a0c09dc30 0x7f8a0c09e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:48.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.488+0000 7f8a145e1700 1 -- 172.21.15.16:0/88703765 >> 172.21.15.16:0/88703765 conn(0x7f8a0c066f50 msgr2=0x7f8a0c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:48.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.488+0000 7f8a145e1700 1 -- 172.21.15.16:0/88703765 shutdown_connections 2023-09-30T22:32:48.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.488+0000 7f8a145e1700 1 -- 172.21.15.16:0/88703765 wait complete. 2023-09-30T22:32:48.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.489+0000 7f8a145e1700 1 Processor -- start 2023-09-30T22:32:48.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.489+0000 7f8a145e1700 1 -- start start 2023-09-30T22:32:48.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.490+0000 7f8a145e1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8a0c06a770 0x7f8a0c12f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:48.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.490+0000 7f8a145e1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a0c09dc30 0x7f8a0c12dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:48.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.490+0000 7f8a145e1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8a0c13c1e0 0x7f8a0c12e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:48.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.490+0000 7f8a145e1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c12f870 con 0x7f8a0c09dc30 2023-09-30T22:32:48.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.490+0000 7f8a145e1700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c131880 con 0x7f8a0c06a770 2023-09-30T22:32:48.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.490+0000 7f8a145e1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c0a1950 con 0x7f8a0c13c1e0 2023-09-30T22:32:48.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.490+0000 7f8a11b7c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a0c09dc30 0x7f8a0c12dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:48.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.490+0000 7f8a11b7c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a0c09dc30 0x7f8a0c12dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58596/0 (socket says 172.21.15.16:58596) 2023-09-30T22:32:48.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.490+0000 7f8a11b7c700 1 -- 172.21.15.16:0/2908257663 learned_addr learned my addr 172.21.15.16:0/2908257663 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:48.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.490+0000 7f8a1237d700 1 --2- 172.21.15.16:0/2908257663 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8a0c06a770 0x7f8a0c12f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:48.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.490+0000 7f8a12b7e700 1 --2- 172.21.15.16:0/2908257663 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8a0c13c1e0 0x7f8a0c12e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:48.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.491+0000 7f8a11b7c700 1 -- 172.21.15.16:0/2908257663 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8a0c13c1e0 msgr2=0x7f8a0c12e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:48.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.491+0000 7f8a11b7c700 1 --2- 172.21.15.16:0/2908257663 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8a0c13c1e0 0x7f8a0c12e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:48.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.491+0000 7f8a11b7c700 1 -- 172.21.15.16:0/2908257663 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8a0c06a770 msgr2=0x7f8a0c12f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:48.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.491+0000 7f8a11b7c700 1 --2- 172.21.15.16:0/2908257663 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8a0c06a770 0x7f8a0c12f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:48.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.491+0000 7f8a11b7c700 1 -- 172.21.15.16:0/2908257663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a04009b30 con 0x7f8a0c09dc30 2023-09-30T22:32:48.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.491+0000 7f8a11b7c700 1 --2- 172.21.15.16:0/2908257663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a0c09dc30 0x7f8a0c12dcc0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f89fc00e7e0 tx=0x7f89fc00eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:48.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.491+0000 7f8a02ffd700 1 -- 172.21.15.16:0/2908257663 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89fc00c070 con 0x7f8a0c09dc30 2023-09-30T22:32:48.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.492+0000 7f8a02ffd700 1 -- 172.21.15.16:0/2908257663 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89fc013610 con 0x7f8a0c09dc30 2023-09-30T22:32:48.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.492+0000 7f8a02ffd700 1 -- 172.21.15.16:0/2908257663 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89fc013980 con 0x7f8a0c09dc30 2023-09-30T22:32:48.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.492+0000 7f8a145e1700 1 -- 172.21.15.16:0/2908257663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a0c09f1a0 con 0x7f8a0c09dc30 2023-09-30T22:32:48.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.492+0000 7f8a145e1700 1 -- 172.21.15.16:0/2908257663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a0c131260 con 0x7f8a0c09dc30 2023-09-30T22:32:48.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.493+0000 7f8a145e1700 1 -- 172.21.15.16:0/2908257663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a0c074ea0 con 0x7f8a0c09dc30 2023-09-30T22:32:48.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.493+0000 7f8a02ffd700 1 -- 172.21.15.16:0/2908257663 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f89fc013ae0 con 0x7f8a0c09dc30 2023-09-30T22:32:48.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.494+0000 7f8a02ffd700 1 -- 172.21.15.16:0/2908257663 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f89fc03f8b0 con 0x7f8a0c09dc30 2023-09-30T22:32:48.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.502+0000 7f8a02ffd700 1 -- 172.21.15.16:0/2908257663 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f89fc024350 con 0x7f8a0c09dc30 2023-09-30T22:32:48.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.534+0000 7f6711dcc700 1 Processor -- start 2023-09-30T22:32:48.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.534+0000 7f6711dcc700 1 -- start start 2023-09-30T22:32:48.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.535+0000 7f6711dcc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f670c133420 0x7f670c1337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:48.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.535+0000 7f6711dcc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f670c133e30 0x7f670c12ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:48.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.535+0000 7f6711dcc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f670c134260 0x7f670c1304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:48.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.535+0000 7f6711dcc700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f670c06b990 con 0x7f670c134260 2023-09-30T22:32:48.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.535+0000 7f6711dcc700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f670c0a0590 con 0x7f670c133420 2023-09-30T22:32:48.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.535+0000 7f6711dcc700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f670c09fb80 con 0x7f670c133e30 2023-09-30T22:32:48.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.535+0000 7f670affd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f670c133e30 0x7f670c12ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:48.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.535+0000 7f670affd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f670c133e30 0x7f670c12ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:47192/0 (socket says 172.21.15.16:47192) 2023-09-30T22:32:48.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.535+0000 7f670affd700 1 -- 172.21.15.16:0/365747832 learned_addr learned my addr 172.21.15.16:0/365747832 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:48.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.535+0000 7f670bfff700 1 --2- 172.21.15.16:0/365747832 >> v2:172.21.15.16:3300/0 conn(0x7f670c134260 0x7f670c1304d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:48.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.535+0000 7f670b7fe700 1 --2- 172.21.15.16:0/365747832 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f670c133420 0x7f670c1337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:48.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.536+0000 7f670affd700 1 -- 172.21.15.16:0/365747832 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f670c133420 msgr2=0x7f670c1337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:48.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.536+0000 7f670affd700 1 --2- 172.21.15.16:0/365747832 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f670c133420 0x7f670c1337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:48.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.536+0000 7f670affd700 1 -- 172.21.15.16:0/365747832 >> v2:172.21.15.16:3300/0 conn(0x7f670c134260 msgr2=0x7f670c1304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:48.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.536+0000 7f670affd700 1 --2- 172.21.15.16:0/365747832 >> v2:172.21.15.16:3300/0 conn(0x7f670c134260 0x7f670c1304d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:48.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.536+0000 7f670affd700 1 -- 172.21.15.16:0/365747832 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f670c0a0030 con 0x7f670c133e30 2023-09-30T22:32:48.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.536+0000 7f670bfff700 1 --2- 172.21.15.16:0/365747832 >> v2:172.21.15.16:3300/0 conn(0x7f670c134260 0x7f670c1304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:48.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.536+0000 7f670affd700 1 --2- 172.21.15.16:0/365747832 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f670c133e30 0x7f670c12ff90 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f67000098f0 tx=0x7f6700009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebfb7ee9e8a831d6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:48.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.536+0000 7f670a7fc700 1 -- 172.21.15.16:0/365747832 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6700011830 con 0x7f670c133e30 2023-09-30T22:32:48.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.536+0000 7f670a7fc700 1 -- 172.21.15.16:0/365747832 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6700011990 con 0x7f670c133e30 2023-09-30T22:32:48.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.536+0000 7f670a7fc700 1 -- 172.21.15.16:0/365747832 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6700011d20 con 0x7f670c133e30 2023-09-30T22:32:48.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.537+0000 7f6711dcc700 1 -- 172.21.15.16:0/365747832 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f670c133e30 msgr2=0x7f670c12ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:48.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.537+0000 7f6711dcc700 1 --2- 172.21.15.16:0/365747832 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f670c133e30 0x7f670c12ff90 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f67000098f0 tx=0x7f6700009c00 comp rx=0 tx=0).stop 2023-09-30T22:32:48.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.537+0000 7f6711dcc700 1 -- 172.21.15.16:0/365747832 shutdown_connections 2023-09-30T22:32:48.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.537+0000 7f6711dcc700 1 --2- 172.21.15.16:0/365747832 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f670c133420 0x7f670c1337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:48.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.537+0000 7f6711dcc700 1 --2- 172.21.15.16:0/365747832 >> v2:172.21.15.16:3300/0 conn(0x7f670c134260 0x7f670c1304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:48.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.537+0000 7f6711dcc700 1 --2- 172.21.15.16:0/365747832 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f670c133e30 0x7f670c12ff90 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:48.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.537+0000 7f6711dcc700 1 -- 172.21.15.16:0/365747832 >> 172.21.15.16:0/365747832 conn(0x7f670c0a4210 msgr2=0x7f670c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:48.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.537+0000 7f6711dcc700 1 -- 172.21.15.16:0/365747832 shutdown_connections 2023-09-30T22:32:48.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.538+0000 7f6711dcc700 1 -- 172.21.15.16:0/365747832 wait complete. 2023-09-30T22:32:48.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.539+0000 7f6711dcc700 1 Processor -- start 2023-09-30T22:32:48.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.539+0000 7f6711dcc700 1 -- start start 2023-09-30T22:32:48.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.540+0000 7f6711dcc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f670c133420 0x7f670c09c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:48.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.540+0000 7f6711dcc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f670c133e30 0x7f670c09cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:48.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.540+0000 7f6711dcc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f670c134260 0x7f670c09d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:48.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.540+0000 7f6711dcc700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f670c12d750 con 0x7f670c134260 2023-09-30T22:32:48.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.540+0000 7f6711dcc700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f670c09e000 con 0x7f670c133e30 2023-09-30T22:32:48.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.540+0000 7f6711dcc700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f670c11fd80 con 0x7f670c133420 2023-09-30T22:32:48.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.540+0000 7f670b7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f670c133420 0x7f670c09c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:48.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.540+0000 7f670affd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f670c133e30 0x7f670c09cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:48.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.540+0000 7f670b7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f670c133420 0x7f670c09c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:47204/0 (socket says 172.21.15.16:47204) 2023-09-30T22:32:48.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.540+0000 7f670b7fe700 1 -- 172.21.15.16:0/1374490437 learned_addr learned my addr 172.21.15.16:0/1374490437 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:48.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.540+0000 7f670bfff700 1 --2- 172.21.15.16:0/1374490437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f670c134260 0x7f670c09d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:48.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.541+0000 7f670bfff700 1 -- 172.21.15.16:0/1374490437 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f670c133420 msgr2=0x7f670c09c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:48.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.541+0000 7f670bfff700 1 --2- 172.21.15.16:0/1374490437 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f670c133420 0x7f670c09c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:48.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.541+0000 7f670bfff700 1 -- 172.21.15.16:0/1374490437 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f670c133e30 msgr2=0x7f670c09cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:48.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.541+0000 7f670bfff700 1 --2- 172.21.15.16:0/1374490437 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f670c133e30 0x7f670c09cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:48.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.541+0000 7f670bfff700 1 -- 172.21.15.16:0/1374490437 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6704009a70 con 0x7f670c134260 2023-09-30T22:32:48.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.541+0000 7f670bfff700 1 --2- 172.21.15.16:0/1374490437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f670c134260 0x7f670c09d160 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f670400efc0 tx=0x7f670400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:48.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.541+0000 7f66f3fff700 1 -- 172.21.15.16:0/1374490437 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670400ca90 con 0x7f670c134260 2023-09-30T22:32:48.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.541+0000 7f66f3fff700 1 -- 172.21.15.16:0/1374490437 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f670400f040 con 0x7f670c134260 2023-09-30T22:32:48.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.541+0000 7f66f3fff700 1 -- 172.21.15.16:0/1374490437 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f670400ce30 con 0x7f670c134260 2023-09-30T22:32:48.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.541+0000 7f6711dcc700 1 -- 172.21.15.16:0/1374490437 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6700009550 con 0x7f670c134260 2023-09-30T22:32:48.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.541+0000 7f6711dcc700 1 -- 172.21.15.16:0/1374490437 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f670c11f9c0 con 0x7f670c134260 2023-09-30T22:32:48.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.542+0000 7f66f3fff700 1 -- 172.21.15.16:0/1374490437 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6704014480 con 0x7f670c134260 2023-09-30T22:32:48.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.543+0000 7f66f3fff700 1 -- 172.21.15.16:0/1374490437 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f670403fc70 con 0x7f670c134260 2023-09-30T22:32:48.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.543+0000 7f6711dcc700 1 -- 172.21.15.16:0/1374490437 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f670c11fbb0 con 0x7f670c134260 2023-09-30T22:32:48.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.548+0000 7f66f3fff700 1 -- 172.21.15.16:0/1374490437 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6704012930 con 0x7f670c134260 2023-09-30T22:32:48.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.605+0000 7f28595f3700 1 Processor -- start 2023-09-30T22:32:48.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.605+0000 7f28595f3700 1 -- start start 2023-09-30T22:32:48.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.606+0000 7f28595f3700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f285409e1d0 0x7f2854138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:48.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.606+0000 7f28595f3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2854139170 0x7f285413b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:48.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.606+0000 7f28595f3700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f285409b120 con 0x7f285409d6f0 2023-09-30T22:32:48.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.606+0000 7f28595f3700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f285406b460 con 0x7f285409e1d0 2023-09-30T22:32:48.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.606+0000 7f28595f3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f285409a7e0 con 0x7f2854139170 2023-09-30T22:32:48.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.606+0000 7f2852ffd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f285409d6f0 0x7f285409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53700/0 (socket says 172.21.15.16:53700) 2023-09-30T22:32:48.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.606+0000 7f28537fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2854139170 0x7f285413b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:48.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.606+0000 7f2852ffd700 1 -- 172.21.15.16:0/470652757 learned_addr learned my addr 172.21.15.16:0/470652757 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:48.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.606+0000 7f284bfff700 1 --2- 172.21.15.16:0/470652757 >> v2:172.21.15.16:3300/0 conn(0x7f285409e1d0 0x7f2854138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:48.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.607+0000 7f28537fe700 1 -- 172.21.15.16:0/470652757 >> v2:172.21.15.16:3300/0 conn(0x7f285409e1d0 msgr2=0x7f2854138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:48.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.607+0000 7f28537fe700 1 --2- 172.21.15.16:0/470652757 >> v2:172.21.15.16:3300/0 conn(0x7f285409e1d0 0x7f2854138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:48.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.607+0000 7f28537fe700 1 -- 172.21.15.16:0/470652757 >> v1:172.21.15.16:6789/0 conn(0x7f285409d6f0 legacy=0x7f285409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:48.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.607+0000 7f28537fe700 1 -- 172.21.15.16:0/470652757 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f285409a500 con 0x7f2854139170 2023-09-30T22:32:48.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.607+0000 7f28527fc700 1 -- 172.21.15.16:0/470652757 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2773599300 0 0) 0x7f285409b120 con 0x7f285409d6f0 2023-09-30T22:32:48.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.607+0000 7f284bfff700 1 --2- 172.21.15.16:0/470652757 >> v2:172.21.15.16:3300/0 conn(0x7f285409e1d0 0x7f2854138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:48.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.607+0000 7f28537fe700 1 --2- 172.21.15.16:0/470652757 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2854139170 0x7f285413b550 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f284c009dc0 tx=0x7f284c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34126824f89dc268 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:48.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.607+0000 7f28527fc700 1 -- 172.21.15.16:0/470652757 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284c00e070 con 0x7f2854139170 2023-09-30T22:32:48.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.607+0000 7f28527fc700 1 -- 172.21.15.16:0/470652757 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f284c00bae0 con 0x7f2854139170 2023-09-30T22:32:48.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.607+0000 7f28527fc700 1 -- 172.21.15.16:0/470652757 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284c00be90 con 0x7f2854139170 2023-09-30T22:32:48.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.608+0000 7f28595f3700 1 -- 172.21.15.16:0/470652757 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2854139170 msgr2=0x7f285413b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:48.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.608+0000 7f28595f3700 1 --2- 172.21.15.16:0/470652757 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2854139170 0x7f285413b550 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f284c009dc0 tx=0x7f284c00b330 comp rx=0 tx=0).stop 2023-09-30T22:32:48.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.608+0000 7f28595f3700 1 -- 172.21.15.16:0/470652757 shutdown_connections 2023-09-30T22:32:48.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.608+0000 7f28595f3700 1 --2- 172.21.15.16:0/470652757 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2854139170 0x7f285413b550 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:48.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.608+0000 7f28595f3700 1 --2- 172.21.15.16:0/470652757 >> v2:172.21.15.16:3300/0 conn(0x7f285409e1d0 0x7f2854138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:48.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.608+0000 7f28595f3700 1 -- 172.21.15.16:0/470652757 >> 172.21.15.16:0/470652757 conn(0x7f2854066f50 msgr2=0x7f285409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:48.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.608+0000 7f28595f3700 1 -- 172.21.15.16:0/470652757 shutdown_connections 2023-09-30T22:32:48.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.608+0000 7f28595f3700 1 -- 172.21.15.16:0/470652757 wait complete. 2023-09-30T22:32:48.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.609+0000 7f28595f3700 1 Processor -- start 2023-09-30T22:32:48.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.609+0000 7f28595f3700 1 -- start start 2023-09-30T22:32:48.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.610+0000 7f28595f3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f285409d6f0 0x7f2854127ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:48.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.610+0000 7f28595f3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f285409e1d0 0x7f2854128020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:48.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.610+0000 7f28595f3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2854139170 0x7f28541cc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:48.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.610+0000 7f28595f3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2854129660 con 0x7f285409d6f0 2023-09-30T22:32:48.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.610+0000 7f28595f3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f285412a790 con 0x7f285409e1d0 2023-09-30T22:32:48.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.610+0000 7f28595f3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f28541276f0 con 0x7f2854139170 2023-09-30T22:32:48.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.610+0000 7f28537fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2854139170 0x7f28541cc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:48.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.610+0000 7f28537fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2854139170 0x7f28541cc3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:47220/0 (socket says 172.21.15.16:47220) 2023-09-30T22:32:48.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.610+0000 7f2852ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f285409d6f0 0x7f2854127ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:48.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.610+0000 7f28537fe700 1 -- 172.21.15.16:0/2731806420 learned_addr learned my addr 172.21.15.16:0/2731806420 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:48.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.610+0000 7f284bfff700 1 --2- 172.21.15.16:0/2731806420 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f285409e1d0 0x7f2854128020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:48.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.610+0000 7f28537fe700 1 -- 172.21.15.16:0/2731806420 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f285409e1d0 msgr2=0x7f2854128020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:48.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.610+0000 7f28537fe700 1 --2- 172.21.15.16:0/2731806420 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f285409e1d0 0x7f2854128020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:48.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.610+0000 7f28537fe700 1 -- 172.21.15.16:0/2731806420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f285409d6f0 msgr2=0x7f2854127ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:48.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.610+0000 7f28537fe700 1 --2- 172.21.15.16:0/2731806420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f285409d6f0 0x7f2854127ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:48.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.610+0000 7f28537fe700 1 -- 172.21.15.16:0/2731806420 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f283c0094a0 con 0x7f2854139170 2023-09-30T22:32:48.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.611+0000 7f28537fe700 1 --2- 172.21.15.16:0/2731806420 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2854139170 0x7f28541cc3a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f284c014040 tx=0x7f284c004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:48.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.611+0000 7f284b7fe700 1 -- 172.21.15.16:0/2731806420 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284c027680 con 0x7f2854139170 2023-09-30T22:32:48.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.611+0000 7f284b7fe700 1 -- 172.21.15.16:0/2731806420 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f284c020070 con 0x7f2854139170 2023-09-30T22:32:48.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.611+0000 7f284b7fe700 1 -- 172.21.15.16:0/2731806420 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284c027a20 con 0x7f2854139170 2023-09-30T22:32:48.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.611+0000 7f28595f3700 1 -- 172.21.15.16:0/2731806420 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f284c009a70 con 0x7f2854139170 2023-09-30T22:32:48.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.611+0000 7f28595f3700 1 -- 172.21.15.16:0/2731806420 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f285412aa20 con 0x7f2854139170 2023-09-30T22:32:48.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.612+0000 7f284b7fe700 1 -- 172.21.15.16:0/2731806420 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f284c01b400 con 0x7f2854139170 2023-09-30T22:32:48.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.613+0000 7f28595f3700 1 -- 172.21.15.16:0/2731806420 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f285412a5e0 con 0x7f2854139170 2023-09-30T22:32:48.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.613+0000 7f284b7fe700 1 -- 172.21.15.16:0/2731806420 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f284c022070 con 0x7f2854139170 2023-09-30T22:32:48.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.622+0000 7f284b7fe700 1 -- 172.21.15.16:0/2731806420 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f284c02f350 con 0x7f2854139170 2023-09-30T22:32:48.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.698+0000 7fd34a0a2700 1 -- 172.21.15.16:0/2611855565 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd34411b600 con 0x7fd344130260 2023-09-30T22:32:48.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.699+0000 7fd32bfff700 1 -- 172.21.15.16:0/2611855565 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd33c00e280 con 0x7fd344130260 2023-09-30T22:32:48.699 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:32:48.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.701+0000 7fd34a0a2700 1 -- 172.21.15.16:0/2611855565 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd344130260 msgr2=0x7fd344119750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:48.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.701+0000 7fd34a0a2700 1 --2- 172.21.15.16:0/2611855565 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd344130260 0x7fd344119750 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fd33c00c750 tx=0x7fd33c00ca60 comp rx=0 tx=0).stop 2023-09-30T22:32:48.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.701+0000 7fd34a0a2700 1 -- 172.21.15.16:0/2611855565 shutdown_connections 2023-09-30T22:32:48.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.701+0000 7fd34a0a2700 1 --2- 172.21.15.16:0/2611855565 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd344130260 0x7fd344119750 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:48.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.701+0000 7fd34a0a2700 1 --2- 172.21.15.16:0/2611855565 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd34412cf60 0x7fd344118cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:48.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.701+0000 7fd34a0a2700 1 --2- 172.21.15.16:0/2611855565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd34412f8b0 0x7fd344119210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:48.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.701+0000 7fd34a0a2700 1 -- 172.21.15.16:0/2611855565 >> 172.21.15.16:0/2611855565 conn(0x7fd3440a2aa0 msgr2=0x7fd3440a1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:48.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.701+0000 7fd34a0a2700 1 -- 172.21.15.16:0/2611855565 shutdown_connections 2023-09-30T22:32:48.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.701+0000 7fd34a0a2700 1 -- 172.21.15.16:0/2611855565 wait complete. 2023-09-30T22:32:48.716 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:32:48.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.732+0000 7f8a145e1700 1 -- 172.21.15.16:0/2908257663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8a0c131520 con 0x7f8a0c09dc30 2023-09-30T22:32:48.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.732+0000 7f8a02ffd700 1 -- 172.21.15.16:0/2908257663 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8a0c131520 con 0x7f8a0c09dc30 2023-09-30T22:32:48.733 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:32:48.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.735+0000 7f8a145e1700 1 -- 172.21.15.16:0/2908257663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a0c09dc30 msgr2=0x7f8a0c12dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:48.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.735+0000 7f8a145e1700 1 --2- 172.21.15.16:0/2908257663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a0c09dc30 0x7f8a0c12dcc0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f89fc00e7e0 tx=0x7f89fc00eaf0 comp rx=0 tx=0).stop 2023-09-30T22:32:48.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.735+0000 7f8a145e1700 1 -- 172.21.15.16:0/2908257663 shutdown_connections 2023-09-30T22:32:48.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.735+0000 7f8a145e1700 1 --2- 172.21.15.16:0/2908257663 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8a0c06a770 0x7f8a0c12f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:48.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.735+0000 7f8a145e1700 1 --2- 172.21.15.16:0/2908257663 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8a0c13c1e0 0x7f8a0c12e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:48.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.735+0000 7f8a145e1700 1 --2- 172.21.15.16:0/2908257663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a0c09dc30 0x7f8a0c12dcc0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:48.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.735+0000 7f8a145e1700 1 -- 172.21.15.16:0/2908257663 >> 172.21.15.16:0/2908257663 conn(0x7f8a0c066f50 msgr2=0x7f8a0c13db70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:48.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.735+0000 7f8a145e1700 1 -- 172.21.15.16:0/2908257663 shutdown_connections 2023-09-30T22:32:48.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.735+0000 7f8a145e1700 1 -- 172.21.15.16:0/2908257663 wait complete. 2023-09-30T22:32:48.751 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:32:48.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.788+0000 7f6711dcc700 1 -- 172.21.15.16:0/1374490437 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f670c09ef90 con 0x7f670c134260 2023-09-30T22:32:48.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.788+0000 7f66f3fff700 1 -- 172.21.15.16:0/1374490437 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6704012340 con 0x7f670c134260 2023-09-30T22:32:48.789 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:32:48.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.790+0000 7f6711dcc700 1 -- 172.21.15.16:0/1374490437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f670c134260 msgr2=0x7f670c09d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:48.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.790+0000 7f6711dcc700 1 --2- 172.21.15.16:0/1374490437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f670c134260 0x7f670c09d160 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f670400efc0 tx=0x7f670400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:32:48.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.790+0000 7f6711dcc700 1 -- 172.21.15.16:0/1374490437 shutdown_connections 2023-09-30T22:32:48.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.790+0000 7f6711dcc700 1 --2- 172.21.15.16:0/1374490437 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f670c134260 0x7f670c09d160 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:48.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.790+0000 7f6711dcc700 1 --2- 172.21.15.16:0/1374490437 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f670c133420 0x7f670c09c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:48.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.790+0000 7f6711dcc700 1 --2- 172.21.15.16:0/1374490437 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f670c133e30 0x7f670c09cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:48.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.790+0000 7f6711dcc700 1 -- 172.21.15.16:0/1374490437 >> 172.21.15.16:0/1374490437 conn(0x7f670c0a4210 msgr2=0x7f670c0a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:48.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.790+0000 7f6711dcc700 1 -- 172.21.15.16:0/1374490437 shutdown_connections 2023-09-30T22:32:48.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.790+0000 7f6711dcc700 1 -- 172.21.15.16:0/1374490437 wait complete. 2023-09-30T22:32:48.805 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:32:48.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.836+0000 7f28595f3700 1 -- 172.21.15.16:0/2731806420 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f285409f420 con 0x7f2854139170 2023-09-30T22:32:48.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.836+0000 7f284b7fe700 1 -- 172.21.15.16:0/2731806420 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f284c02c030 con 0x7f2854139170 2023-09-30T22:32:48.837 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:32:48.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.838+0000 7f28595f3700 1 -- 172.21.15.16:0/2731806420 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2854139170 msgr2=0x7f28541cc3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:48.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.838+0000 7f28595f3700 1 --2- 172.21.15.16:0/2731806420 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2854139170 0x7f28541cc3a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f284c014040 tx=0x7f284c004ad0 comp rx=0 tx=0).stop 2023-09-30T22:32:48.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.838+0000 7f28595f3700 1 -- 172.21.15.16:0/2731806420 shutdown_connections 2023-09-30T22:32:48.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.838+0000 7f28595f3700 1 --2- 172.21.15.16:0/2731806420 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2854139170 0x7f28541cc3a0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:48.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.838+0000 7f28595f3700 1 --2- 172.21.15.16:0/2731806420 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f285409d6f0 0x7f2854127ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:48.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.838+0000 7f28595f3700 1 --2- 172.21.15.16:0/2731806420 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f285409e1d0 0x7f2854128020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:48.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.838+0000 7f28595f3700 1 -- 172.21.15.16:0/2731806420 >> 172.21.15.16:0/2731806420 conn(0x7f2854066f50 msgr2=0x7f2854070e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:48.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.838+0000 7f28595f3700 1 -- 172.21.15.16:0/2731806420 shutdown_connections 2023-09-30T22:32:48.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:48.839+0000 7f28595f3700 1 -- 172.21.15.16:0/2731806420 wait complete. 2023-09-30T22:32:48.852 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:32:49.717 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:32:49.751 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:32:49.806 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:32:49.853 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:32:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.867+0000 7fd5855e8700 1 Processor -- start 2023-09-30T22:32:49.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.867+0000 7fd5855e8700 1 -- start start 2023-09-30T22:32:49.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.868+0000 7fd5855e8700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd58006f5d0 0x7fd58012ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:49.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.868+0000 7fd5855e8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd580135c20 0x7fd580138000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:49.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.868+0000 7fd5855e8700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd58006fa40 con 0x7fd58012cf60 2023-09-30T22:32:49.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.868+0000 7fd5855e8700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd58012cd90 con 0x7fd58006f5d0 2023-09-30T22:32:49.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.868+0000 7fd5855e8700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd580071780 con 0x7fd580135c20 2023-09-30T22:32:49.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.868+0000 7fd57e7fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd58006f5d0 0x7fd58012ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:49.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.868+0000 7fd57effd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd58012cf60 0x7fd58012f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53712/0 (socket says 172.21.15.16:53712) 2023-09-30T22:32:49.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.868+0000 7fd57effd700 1 -- 172.21.15.16:0/3733397195 learned_addr learned my addr 172.21.15.16:0/3733397195 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:49.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.868+0000 7fd57f7fe700 1 --2- 172.21.15.16:0/3733397195 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd580135c20 0x7fd580138000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:49.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.869+0000 7fd57dffb700 1 -- 172.21.15.16:0/3733397195 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2412197112 0 0) 0x7fd58006fa40 con 0x7fd58012cf60 2023-09-30T22:32:49.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.869+0000 7fd57dffb700 1 -- 172.21.15.16:0/3733397195 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5680035b0 con 0x7fd58012cf60 2023-09-30T22:32:49.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.869+0000 7fd57e7fc700 1 -- 172.21.15.16:0/3733397195 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd580135c20 msgr2=0x7fd580138000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:49.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.869+0000 7fd57e7fc700 1 --2- 172.21.15.16:0/3733397195 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd580135c20 0x7fd580138000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:49.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.869+0000 7fd57e7fc700 1 -- 172.21.15.16:0/3733397195 >> v1:172.21.15.16:6789/0 conn(0x7fd58012cf60 legacy=0x7fd58012f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:49.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.869+0000 7fd57e7fc700 1 -- 172.21.15.16:0/3733397195 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd580071a40 con 0x7fd58006f5d0 2023-09-30T22:32:49.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.869+0000 7fd57e7fc700 1 --2- 172.21.15.16:0/3733397195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd58006f5d0 0x7fd58012ffc0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fd57400bae0 tx=0x7fd57400bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=754bceed32cf19b7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:49.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.869+0000 7fd57dffb700 1 -- 172.21.15.16:0/3733397195 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd57400dc70 con 0x7fd58006f5d0 2023-09-30T22:32:49.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.869+0000 7fd57dffb700 1 -- 172.21.15.16:0/3733397195 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd57400ddd0 con 0x7fd58006f5d0 2023-09-30T22:32:49.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.869+0000 7fd57dffb700 1 -- 172.21.15.16:0/3733397195 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd574010670 con 0x7fd58006f5d0 2023-09-30T22:32:49.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.870+0000 7fd5855e8700 1 -- 172.21.15.16:0/3733397195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd58006f5d0 msgr2=0x7fd58012ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:49.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.870+0000 7fd5855e8700 1 --2- 172.21.15.16:0/3733397195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd58006f5d0 0x7fd58012ffc0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fd57400bae0 tx=0x7fd57400bdf0 comp rx=0 tx=0).stop 2023-09-30T22:32:49.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.870+0000 7fd5855e8700 1 -- 172.21.15.16:0/3733397195 shutdown_connections 2023-09-30T22:32:49.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.870+0000 7fd5855e8700 1 --2- 172.21.15.16:0/3733397195 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd580135c20 0x7fd580138000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:49.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.870+0000 7fd5855e8700 1 --2- 172.21.15.16:0/3733397195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd58006f5d0 0x7fd58012ffc0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:49.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.870+0000 7fd5855e8700 1 -- 172.21.15.16:0/3733397195 >> 172.21.15.16:0/3733397195 conn(0x7fd5800a2aa0 msgr2=0x7fd5800a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:49.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.871+0000 7fd5855e8700 1 -- 172.21.15.16:0/3733397195 shutdown_connections 2023-09-30T22:32:49.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.871+0000 7fd5855e8700 1 -- 172.21.15.16:0/3733397195 wait complete. 2023-09-30T22:32:49.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.872+0000 7fd5855e8700 1 Processor -- start 2023-09-30T22:32:49.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.872+0000 7fd5855e8700 1 -- start start 2023-09-30T22:32:49.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.873+0000 7fd5855e8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd58006f5d0 0x7fd580134690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:49.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.873+0000 7fd5855e8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd58012cf60 0x7fd580134bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:49.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.873+0000 7fd5855e8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd580135c20 0x7fd5801c8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:49.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.873+0000 7fd5855e8700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd58011aac0 con 0x7fd58012cf60 2023-09-30T22:32:49.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.873+0000 7fd5855e8700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd580129b60 con 0x7fd580135c20 2023-09-30T22:32:49.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.873+0000 7fd5855e8700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd5800a1490 con 0x7fd58006f5d0 2023-09-30T22:32:49.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.873+0000 7fd57effd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd58006f5d0 0x7fd580134690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:49.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.873+0000 7fd57effd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd58006f5d0 0x7fd580134690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:47236/0 (socket says 172.21.15.16:47236) 2023-09-30T22:32:49.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.873+0000 7fd57effd700 1 -- 172.21.15.16:0/211204484 learned_addr learned my addr 172.21.15.16:0/211204484 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:49.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.873+0000 7fd57f7fe700 1 --2- 172.21.15.16:0/211204484 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd580135c20 0x7fd5801c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:49.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.873+0000 7fd57e7fc700 1 --2- 172.21.15.16:0/211204484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd58012cf60 0x7fd580134bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:49.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.874+0000 7fd57effd700 1 -- 172.21.15.16:0/211204484 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd580135c20 msgr2=0x7fd5801c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:49.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.874+0000 7fd57effd700 1 --2- 172.21.15.16:0/211204484 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd580135c20 0x7fd5801c8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:49.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.874+0000 7fd57effd700 1 -- 172.21.15.16:0/211204484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd58012cf60 msgr2=0x7fd580134bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:49.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.874+0000 7fd57effd700 1 --2- 172.21.15.16:0/211204484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd58012cf60 0x7fd580134bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:49.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.874+0000 7fd57effd700 1 -- 172.21.15.16:0/211204484 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd57400b740 con 0x7fd58006f5d0 2023-09-30T22:32:49.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.874+0000 7fd57effd700 1 --2- 172.21.15.16:0/211204484 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd58006f5d0 0x7fd580134690 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd57000c700 tx=0x7fd57000ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:49.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.874+0000 7fd5677fe700 1 -- 172.21.15.16:0/211204484 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd570016070 con 0x7fd58006f5d0 2023-09-30T22:32:49.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.874+0000 7fd5677fe700 1 -- 172.21.15.16:0/211204484 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd570004870 con 0x7fd58006f5d0 2023-09-30T22:32:49.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.874+0000 7fd5677fe700 1 -- 172.21.15.16:0/211204484 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd570004c00 con 0x7fd58006f5d0 2023-09-30T22:32:49.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.874+0000 7fd5855e8700 1 -- 172.21.15.16:0/211204484 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd580129fa0 con 0x7fd58006f5d0 2023-09-30T22:32:49.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.875+0000 7fd5855e8700 1 -- 172.21.15.16:0/211204484 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd58011b4a0 con 0x7fd58006f5d0 2023-09-30T22:32:49.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.876+0000 7fd5855e8700 1 -- 172.21.15.16:0/211204484 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd580074ea0 con 0x7fd58006f5d0 2023-09-30T22:32:49.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.876+0000 7fd5677fe700 1 -- 172.21.15.16:0/211204484 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd570011590 con 0x7fd58006f5d0 2023-09-30T22:32:49.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.877+0000 7fd5677fe700 1 -- 172.21.15.16:0/211204484 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd57003ed90 con 0x7fd58006f5d0 2023-09-30T22:32:49.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.886+0000 7fd5677fe700 1 -- 172.21.15.16:0/211204484 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd570019960 con 0x7fd58006f5d0 2023-09-30T22:32:49.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.919+0000 7f6de0848700 1 Processor -- start 2023-09-30T22:32:49.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.919+0000 7f6de0848700 1 -- start start 2023-09-30T22:32:49.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.920+0000 7f6de0848700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6ddc136820 0x7f6ddc138c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:49.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.920+0000 7f6de0848700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6ddc139140 0x7f6ddc13b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:49.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.920+0000 7f6de0848700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ddc06bc00 con 0x7f6ddc133a80 2023-09-30T22:32:49.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.920+0000 7f6de0848700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ddc06c570 con 0x7f6ddc136820 2023-09-30T22:32:49.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.920+0000 7f6de0848700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6ddc0a0040 con 0x7f6ddc139140 2023-09-30T22:32:49.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.920+0000 7f6dd9d9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6ddc136820 0x7f6ddc138c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:49.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.920+0000 7f6ddad9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6ddc139140 0x7f6ddc13b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:49.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.920+0000 7f6dda59c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6ddc133a80 0x7f6ddc133e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53722/0 (socket says 172.21.15.16:53722) 2023-09-30T22:32:49.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.920+0000 7f6dda59c700 1 -- 172.21.15.16:0/1358884198 learned_addr learned my addr 172.21.15.16:0/1358884198 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:49.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.920+0000 7f6dd9d9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6ddc136820 0x7f6ddc138c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37138/0 (socket says 172.21.15.16:37138) 2023-09-30T22:32:49.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.920+0000 7f6dd9d9b700 1 -- 172.21.15.16:0/1358884198 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6ddc139140 msgr2=0x7f6ddc13b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:49.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.920+0000 7f6dd9d9b700 1 --2- 172.21.15.16:0/1358884198 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6ddc139140 0x7f6ddc13b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:49.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.920+0000 7f6dd9d9b700 1 -- 172.21.15.16:0/1358884198 >> v1:172.21.15.16:6789/0 conn(0x7f6ddc133a80 legacy=0x7f6ddc133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:49.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.920+0000 7f6dd9d9b700 1 -- 172.21.15.16:0/1358884198 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ddc133860 con 0x7f6ddc136820 2023-09-30T22:32:49.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.921+0000 7f6dd9d9b700 1 --2- 172.21.15.16:0/1358884198 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6ddc136820 0x7f6ddc138c00 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0009870 tx=0x7f6dd0009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b135cbc8c0d78611 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:49.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.921+0000 7f6dd959a700 1 -- 172.21.15.16:0/1358884198 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd0011830 con 0x7f6ddc136820 2023-09-30T22:32:49.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.921+0000 7f6dd959a700 1 -- 172.21.15.16:0/1358884198 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dd0011990 con 0x7f6ddc136820 2023-09-30T22:32:49.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.921+0000 7f6dd959a700 1 -- 172.21.15.16:0/1358884198 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd0011d20 con 0x7f6ddc136820 2023-09-30T22:32:49.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.922+0000 7f6de0848700 1 -- 172.21.15.16:0/1358884198 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6ddc136820 msgr2=0x7f6ddc138c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:49.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.922+0000 7f6de0848700 1 --2- 172.21.15.16:0/1358884198 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6ddc136820 0x7f6ddc138c00 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0009870 tx=0x7f6dd0009b80 comp rx=0 tx=0).stop 2023-09-30T22:32:49.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.922+0000 7f6de0848700 1 -- 172.21.15.16:0/1358884198 shutdown_connections 2023-09-30T22:32:49.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.922+0000 7f6de0848700 1 --2- 172.21.15.16:0/1358884198 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6ddc139140 0x7f6ddc13b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:49.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.922+0000 7f6de0848700 1 --2- 172.21.15.16:0/1358884198 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6ddc136820 0x7f6ddc138c00 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:49.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.922+0000 7f6de0848700 1 -- 172.21.15.16:0/1358884198 >> 172.21.15.16:0/1358884198 conn(0x7f6ddc0a4210 msgr2=0x7f6ddc0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:49.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.922+0000 7f6de0848700 1 -- 172.21.15.16:0/1358884198 shutdown_connections 2023-09-30T22:32:49.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.922+0000 7f6de0848700 1 -- 172.21.15.16:0/1358884198 wait complete. 2023-09-30T22:32:49.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.924+0000 7f6de0848700 1 Processor -- start 2023-09-30T22:32:49.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.924+0000 7f6de0848700 1 -- start start 2023-09-30T22:32:49.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.925+0000 7f6de0848700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6ddc133a80 0x7f6ddc071850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:49.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.925+0000 7f6de0848700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6ddc136820 0x7f6ddc071d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:49.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.925+0000 7f6de0848700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ddc139140 0x7f6ddc1c84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:49.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.925+0000 7f6de0848700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ddc09e310 con 0x7f6ddc139140 2023-09-30T22:32:49.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.925+0000 7f6de0848700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ddc071480 con 0x7f6ddc136820 2023-09-30T22:32:49.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.925+0000 7f6de0848700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6ddc138c10 con 0x7f6ddc133a80 2023-09-30T22:32:49.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.925+0000 7f6ddad9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ddc139140 0x7f6ddc1c84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:49.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.925+0000 7f6dda59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6ddc133a80 0x7f6ddc071850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:49.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.925+0000 7f6ddad9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ddc139140 0x7f6ddc1c84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58648/0 (socket says 172.21.15.16:58648) 2023-09-30T22:32:49.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.925+0000 7f6ddad9d700 1 -- 172.21.15.16:0/1598412208 learned_addr learned my addr 172.21.15.16:0/1598412208 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:49.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.925+0000 7f6dd9d9b700 1 --2- 172.21.15.16:0/1598412208 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6ddc136820 0x7f6ddc071d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:49.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.925+0000 7f6dda59c700 1 -- 172.21.15.16:0/1598412208 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6ddc136820 msgr2=0x7f6ddc071d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:49.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.925+0000 7f6dda59c700 1 --2- 172.21.15.16:0/1598412208 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6ddc136820 0x7f6ddc071d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:49.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.925+0000 7f6dda59c700 1 -- 172.21.15.16:0/1598412208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ddc139140 msgr2=0x7f6ddc1c84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:49.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.925+0000 7f6dda59c700 1 --2- 172.21.15.16:0/1598412208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ddc139140 0x7f6ddc1c84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:49.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.925+0000 7f6dda59c700 1 -- 172.21.15.16:0/1598412208 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd00094a0 con 0x7f6ddc133a80 2023-09-30T22:32:49.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.926+0000 7f6dd9d9b700 1 --2- 172.21.15.16:0/1598412208 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6ddc136820 0x7f6ddc071d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:49.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.926+0000 7f6ddad9d700 1 --2- 172.21.15.16:0/1598412208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ddc139140 0x7f6ddc1c84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:49.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.926+0000 7f6dda59c700 1 --2- 172.21.15.16:0/1598412208 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6ddc133a80 0x7f6ddc071850 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc00c500 tx=0x7f6dcc00c810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:49.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.926+0000 7f6dcaffd700 1 -- 172.21.15.16:0/1598412208 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc005d70 con 0x7f6ddc133a80 2023-09-30T22:32:49.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.926+0000 7f6dcaffd700 1 -- 172.21.15.16:0/1598412208 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dcc00ce50 con 0x7f6ddc133a80 2023-09-30T22:32:49.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.926+0000 7f6dcaffd700 1 -- 172.21.15.16:0/1598412208 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc0072d0 con 0x7f6ddc133a80 2023-09-30T22:32:49.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.926+0000 7f6de0848700 1 -- 172.21.15.16:0/1598412208 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ddc09e070 con 0x7f6ddc133a80 2023-09-30T22:32:49.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.926+0000 7f6de0848700 1 -- 172.21.15.16:0/1598412208 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ddc09dc20 con 0x7f6ddc133a80 2023-09-30T22:32:49.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.927+0000 7f6dcaffd700 1 -- 172.21.15.16:0/1598412208 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6dcc007430 con 0x7f6ddc133a80 2023-09-30T22:32:49.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.927+0000 7f6dcaffd700 1 -- 172.21.15.16:0/1598412208 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f6dcc040a70 con 0x7f6ddc133a80 2023-09-30T22:32:49.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.928+0000 7f6de0848700 1 -- 172.21.15.16:0/1598412208 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ddc074ea0 con 0x7f6ddc133a80 2023-09-30T22:32:49.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.937+0000 7f6dcaffd700 1 -- 172.21.15.16:0/1598412208 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6dcc044060 con 0x7f6ddc133a80 2023-09-30T22:32:49.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.962+0000 7fee67a96700 1 Processor -- start 2023-09-30T22:32:49.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.962+0000 7fee67a96700 1 -- start start 2023-09-30T22:32:49.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.963+0000 7fee67a96700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fee6009e930 0x7fee6009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:49.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.963+0000 7fee67a96700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fee6009f240 0x7fee601317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:49.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.963+0000 7fee67a96700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fee6009f650 0x7fee60131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:49.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.963+0000 7fee67a96700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee60070940 con 0x7fee6009e930 2023-09-30T22:32:49.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.963+0000 7fee67a96700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee6006fd60 con 0x7fee6009f650 2023-09-30T22:32:49.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.963+0000 7fee67a96700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fee6006fa80 con 0x7fee6009f240 2023-09-30T22:32:49.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.963+0000 7fee66033700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fee6009f650 0x7fee60131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:49.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.964+0000 7fee66033700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fee6009f650 0x7fee60131d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37146/0 (socket says 172.21.15.16:37146) 2023-09-30T22:32:49.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.964+0000 7fee65031700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fee6009f240 0x7fee601317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:49.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.964+0000 7fee66033700 1 -- 172.21.15.16:0/3450605472 learned_addr learned my addr 172.21.15.16:0/3450605472 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:49.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.964+0000 7fee65832700 1 --2- 172.21.15.16:0/3450605472 >> v2:172.21.15.16:3300/0 conn(0x7fee6009e930 0x7fee6009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:49.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.964+0000 7fee65031700 1 -- 172.21.15.16:0/3450605472 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fee6009f650 msgr2=0x7fee60131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:49.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.964+0000 7fee65031700 1 --2- 172.21.15.16:0/3450605472 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fee6009f650 0x7fee60131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:49.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.964+0000 7fee65031700 1 -- 172.21.15.16:0/3450605472 >> v2:172.21.15.16:3300/0 conn(0x7fee6009e930 msgr2=0x7fee6009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:49.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.964+0000 7fee65031700 1 --2- 172.21.15.16:0/3450605472 >> v2:172.21.15.16:3300/0 conn(0x7fee6009e930 0x7fee6009ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:49.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.964+0000 7fee65031700 1 -- 172.21.15.16:0/3450605472 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee6009fb60 con 0x7fee6009f240 2023-09-30T22:32:49.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.964+0000 7fee65832700 1 --2- 172.21.15.16:0/3450605472 >> v2:172.21.15.16:3300/0 conn(0x7fee6009e930 0x7fee6009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:49.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.964+0000 7fee65031700 1 --2- 172.21.15.16:0/3450605472 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fee6009f240 0x7fee601317f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fee50009840 tx=0x7fee50009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef106f5283bb2cb4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:49.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.964+0000 7fee64830700 1 -- 172.21.15.16:0/3450605472 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee50010a90 con 0x7fee6009f240 2023-09-30T22:32:49.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.965+0000 7fee64830700 1 -- 172.21.15.16:0/3450605472 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee50010bf0 con 0x7fee6009f240 2023-09-30T22:32:49.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.965+0000 7fee64830700 1 -- 172.21.15.16:0/3450605472 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee50018440 con 0x7fee6009f240 2023-09-30T22:32:49.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.966+0000 7fee67a96700 1 -- 172.21.15.16:0/3450605472 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fee6009f240 msgr2=0x7fee601317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:49.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.966+0000 7fee67a96700 1 --2- 172.21.15.16:0/3450605472 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fee6009f240 0x7fee601317f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fee50009840 tx=0x7fee50009b50 comp rx=0 tx=0).stop 2023-09-30T22:32:49.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.966+0000 7fee67a96700 1 -- 172.21.15.16:0/3450605472 shutdown_connections 2023-09-30T22:32:49.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.966+0000 7fee67a96700 1 --2- 172.21.15.16:0/3450605472 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fee6009f650 0x7fee60131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:49.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.966+0000 7fee67a96700 1 --2- 172.21.15.16:0/3450605472 >> v2:172.21.15.16:3300/0 conn(0x7fee6009e930 0x7fee6009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:49.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.966+0000 7fee67a96700 1 --2- 172.21.15.16:0/3450605472 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fee6009f240 0x7fee601317f0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:49.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.966+0000 7fee67a96700 1 -- 172.21.15.16:0/3450605472 >> 172.21.15.16:0/3450605472 conn(0x7fee600a4210 msgr2=0x7fee600a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:49.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.966+0000 7fee67a96700 1 -- 172.21.15.16:0/3450605472 shutdown_connections 2023-09-30T22:32:49.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.966+0000 7fee67a96700 1 -- 172.21.15.16:0/3450605472 wait complete. 2023-09-30T22:32:49.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.967+0000 7fee67a96700 1 Processor -- start 2023-09-30T22:32:49.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.968+0000 7fee67a96700 1 -- start start 2023-09-30T22:32:49.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.968+0000 7fee67a96700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee6009e930 0x7fee6011d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:49.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.968+0000 7fee67a96700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fee6009f240 0x7fee6011d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:49.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.968+0000 7fee67a96700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fee6009f650 0x7fee6011de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:49.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.968+0000 7fee67a96700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee60120e90 con 0x7fee6009e930 2023-09-30T22:32:49.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.968+0000 7fee67a96700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee601206c0 con 0x7fee6009f240 2023-09-30T22:32:49.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.968+0000 7fee67a96700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fee60071d20 con 0x7fee6009f650 2023-09-30T22:32:49.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.969+0000 7fee66033700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fee6009f650 0x7fee6011de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:49.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.969+0000 7fee66033700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fee6009f650 0x7fee6011de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:47290/0 (socket says 172.21.15.16:47290) 2023-09-30T22:32:49.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.969+0000 7fee66033700 1 -- 172.21.15.16:0/749084289 learned_addr learned my addr 172.21.15.16:0/749084289 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:49.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.969+0000 7fee65031700 1 --2- 172.21.15.16:0/749084289 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fee6009f240 0x7fee6011d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:49.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.969+0000 7fee65832700 1 --2- 172.21.15.16:0/749084289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee6009e930 0x7fee6011d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:49.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.969+0000 7fee66033700 1 -- 172.21.15.16:0/749084289 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fee6009f240 msgr2=0x7fee6011d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:49.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.969+0000 7fee66033700 1 --2- 172.21.15.16:0/749084289 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fee6009f240 0x7fee6011d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:49.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.969+0000 7fee66033700 1 -- 172.21.15.16:0/749084289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee6009e930 msgr2=0x7fee6011d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:49.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.969+0000 7fee66033700 1 --2- 172.21.15.16:0/749084289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee6009e930 0x7fee6011d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:49.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.969+0000 7fee66033700 1 -- 172.21.15.16:0/749084289 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee5c0094a0 con 0x7fee6009f650 2023-09-30T22:32:49.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.969+0000 7fee66033700 1 --2- 172.21.15.16:0/749084289 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fee6009f650 0x7fee6011de80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fee5800efc0 tx=0x7fee5800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:49.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.969+0000 7fee567fc700 1 -- 172.21.15.16:0/749084289 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee58003c20 con 0x7fee6009f650 2023-09-30T22:32:49.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.970+0000 7fee567fc700 1 -- 172.21.15.16:0/749084289 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee5800f040 con 0x7fee6009f650 2023-09-30T22:32:49.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.970+0000 7fee567fc700 1 -- 172.21.15.16:0/749084289 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee58009e60 con 0x7fee6009f650 2023-09-30T22:32:49.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.971+0000 7fee67a96700 1 -- 172.21.15.16:0/749084289 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee500094a0 con 0x7fee6009f650 2023-09-30T22:32:49.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.971+0000 7fee67a96700 1 -- 172.21.15.16:0/749084289 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee600a0190 con 0x7fee6009f650 2023-09-30T22:32:49.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.972+0000 7fee567fc700 1 -- 172.21.15.16:0/749084289 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fee5800c880 con 0x7fee6009f650 2023-09-30T22:32:49.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.972+0000 7fee567fc700 1 -- 172.21.15.16:0/749084289 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fee5803f990 con 0x7fee6009f650 2023-09-30T22:32:49.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.973+0000 7fee67a96700 1 -- 172.21.15.16:0/749084289 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee60074ea0 con 0x7fee6009f650 2023-09-30T22:32:49.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:49.982+0000 7fee567fc700 1 -- 172.21.15.16:0/749084289 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fee5801c360 con 0x7fee6009f650 2023-09-30T22:32:50.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.006+0000 7f0dc12e0700 1 Processor -- start 2023-09-30T22:32:50.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.007+0000 7f0dc12e0700 1 -- start start 2023-09-30T22:32:50.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.007+0000 7f0dc12e0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0dbc134630 0x7f0dbc136a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:50.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.007+0000 7f0dc12e0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0dbc136f50 0x7f0dbc139330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:50.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.007+0000 7f0dc12e0700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dbc09b4f0 con 0x7f0dbc131810 2023-09-30T22:32:50.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.007+0000 7f0dc12e0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc1313c0 con 0x7f0dbc134630 2023-09-30T22:32:50.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.007+0000 7f0dc12e0700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc06e8c0 con 0x7f0dbc136f50 2023-09-30T22:32:50.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.007+0000 7f0dbaffd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0dbc131810 0x7f0dbc131be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53728/0 (socket says 172.21.15.16:53728) 2023-09-30T22:32:50.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.007+0000 7f0dba7fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0dbc134630 0x7f0dbc136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:50.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.007+0000 7f0dbaffd700 1 -- 172.21.15.16:0/4005835412 learned_addr learned my addr 172.21.15.16:0/4005835412 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:50.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.008+0000 7f0dbb7fe700 1 --2- 172.21.15.16:0/4005835412 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0dbc136f50 0x7f0dbc139330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:50.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.008+0000 7f0db9ffb700 1 -- 172.21.15.16:0/4005835412 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1355018274 0 0) 0x7f0dbc09b4f0 con 0x7f0dbc131810 2023-09-30T22:32:50.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.008+0000 7f0dba7fc700 1 -- 172.21.15.16:0/4005835412 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0dbc136f50 msgr2=0x7f0dbc139330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:50.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.008+0000 7f0dba7fc700 1 --2- 172.21.15.16:0/4005835412 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0dbc136f50 0x7f0dbc139330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:50.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.008+0000 7f0dba7fc700 1 -- 172.21.15.16:0/4005835412 >> v1:172.21.15.16:6789/0 conn(0x7f0dbc131810 legacy=0x7f0dbc131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:50.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.008+0000 7f0dba7fc700 1 -- 172.21.15.16:0/4005835412 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dbc071d20 con 0x7f0dbc134630 2023-09-30T22:32:50.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.008+0000 7f0dba7fc700 1 --2- 172.21.15.16:0/4005835412 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0dbc134630 0x7f0dbc136a10 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0db0009870 tx=0x7f0db0009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=570bb2a5a5cdf5df server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:50.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.008+0000 7f0db9ffb700 1 -- 172.21.15.16:0/4005835412 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db0010a90 con 0x7f0dbc134630 2023-09-30T22:32:50.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.008+0000 7f0db9ffb700 1 -- 172.21.15.16:0/4005835412 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0db0010bf0 con 0x7f0dbc134630 2023-09-30T22:32:50.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.008+0000 7f0db9ffb700 1 -- 172.21.15.16:0/4005835412 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db0018440 con 0x7f0dbc134630 2023-09-30T22:32:50.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.009+0000 7f0dc12e0700 1 -- 172.21.15.16:0/4005835412 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0dbc134630 msgr2=0x7f0dbc136a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:50.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.009+0000 7f0dc12e0700 1 --2- 172.21.15.16:0/4005835412 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0dbc134630 0x7f0dbc136a10 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0db0009870 tx=0x7f0db0009b80 comp rx=0 tx=0).stop 2023-09-30T22:32:50.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.009+0000 7f0dc12e0700 1 -- 172.21.15.16:0/4005835412 shutdown_connections 2023-09-30T22:32:50.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.009+0000 7f0dc12e0700 1 --2- 172.21.15.16:0/4005835412 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0dbc136f50 0x7f0dbc139330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:50.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.009+0000 7f0dc12e0700 1 --2- 172.21.15.16:0/4005835412 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0dbc134630 0x7f0dbc136a10 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:50.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.009+0000 7f0dc12e0700 1 -- 172.21.15.16:0/4005835412 >> 172.21.15.16:0/4005835412 conn(0x7f0dbc0a2aa0 msgr2=0x7f0dbc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:50.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.009+0000 7f0dc12e0700 1 -- 172.21.15.16:0/4005835412 shutdown_connections 2023-09-30T22:32:50.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.009+0000 7f0dc12e0700 1 -- 172.21.15.16:0/4005835412 wait complete. 2023-09-30T22:32:50.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.010+0000 7f0dc12e0700 1 Processor -- start 2023-09-30T22:32:50.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.010+0000 7f0dc12e0700 1 -- start start 2023-09-30T22:32:50.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.011+0000 7f0dc12e0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0dbc131810 0x7f0dbc09c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:50.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.011+0000 7f0dc12e0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dbc134630 0x7f0dbc09cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:50.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.011+0000 7f0dc12e0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0dbc136f50 0x7f0dbc09d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:50.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.011+0000 7f0dc12e0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc118f10 con 0x7f0dbc134630 2023-09-30T22:32:50.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.011+0000 7f0dc12e0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc06eb90 con 0x7f0dbc136f50 2023-09-30T22:32:50.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.011+0000 7f0dc12e0700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc11a7c0 con 0x7f0dbc131810 2023-09-30T22:32:50.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.011+0000 7f0dbaffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0dbc131810 0x7f0dbc09c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:50.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.011+0000 7f0dbb7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0dbc136f50 0x7f0dbc09d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:50.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.011+0000 7f0dbaffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0dbc131810 0x7f0dbc09c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:47308/0 (socket says 172.21.15.16:47308) 2023-09-30T22:32:50.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.011+0000 7f0dbaffd700 1 -- 172.21.15.16:0/214084997 learned_addr learned my addr 172.21.15.16:0/214084997 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:50.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.011+0000 7f0dba7fc700 1 --2- 172.21.15.16:0/214084997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dbc134630 0x7f0dbc09cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:50.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.011+0000 7f0dbaffd700 1 -- 172.21.15.16:0/214084997 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0dbc136f50 msgr2=0x7f0dbc09d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:50.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.011+0000 7f0dbaffd700 1 --2- 172.21.15.16:0/214084997 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0dbc136f50 0x7f0dbc09d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:50.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.011+0000 7f0dbaffd700 1 -- 172.21.15.16:0/214084997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dbc134630 msgr2=0x7f0dbc09cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:50.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.011+0000 7f0dbaffd700 1 --2- 172.21.15.16:0/214084997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dbc134630 0x7f0dbc09cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:50.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.011+0000 7f0dbaffd700 1 -- 172.21.15.16:0/214084997 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0db00094a0 con 0x7f0dbc131810 2023-09-30T22:32:50.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.011+0000 7f0dba7fc700 1 --2- 172.21.15.16:0/214084997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dbc134630 0x7f0dbc09cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:50.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.012+0000 7f0dbaffd700 1 --2- 172.21.15.16:0/214084997 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0dbc131810 0x7f0dbc09c800 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0dac00c550 tx=0x7f0dac00c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:50.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.012+0000 7f0da37fe700 1 -- 172.21.15.16:0/214084997 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dac004c80 con 0x7f0dbc131810 2023-09-30T22:32:50.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.012+0000 7f0da37fe700 1 -- 172.21.15.16:0/214084997 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dac004e00 con 0x7f0dbc131810 2023-09-30T22:32:50.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.012+0000 7f0dc12e0700 1 -- 172.21.15.16:0/214084997 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dbc071690 con 0x7f0dbc131810 2023-09-30T22:32:50.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.012+0000 7f0da37fe700 1 -- 172.21.15.16:0/214084997 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dac007310 con 0x7f0dbc131810 2023-09-30T22:32:50.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.012+0000 7f0dc12e0700 1 -- 172.21.15.16:0/214084997 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dbc1190a0 con 0x7f0dbc131810 2023-09-30T22:32:50.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.013+0000 7f0dc12e0700 1 -- 172.21.15.16:0/214084997 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dbc074ea0 con 0x7f0dbc131810 2023-09-30T22:32:50.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.013+0000 7f0da37fe700 1 -- 172.21.15.16:0/214084997 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0dac007470 con 0x7f0dbc131810 2023-09-30T22:32:50.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.013+0000 7f0da37fe700 1 -- 172.21.15.16:0/214084997 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f0dac03f260 con 0x7f0dbc131810 2023-09-30T22:32:50.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.018+0000 7f0da37fe700 1 -- 172.21.15.16:0/214084997 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0dac0133c0 con 0x7f0dbc131810 2023-09-30T22:32:50.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.145+0000 7f6de0848700 1 -- 172.21.15.16:0/1598412208 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6ddc1362b0 con 0x7f6ddc133a80 2023-09-30T22:32:50.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.146+0000 7f6dcaffd700 1 -- 172.21.15.16:0/1598412208 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6dcc030020 con 0x7f6ddc133a80 2023-09-30T22:32:50.146 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:32:50.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.148+0000 7f6de0848700 1 -- 172.21.15.16:0/1598412208 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6ddc133a80 msgr2=0x7f6ddc071850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:50.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.148+0000 7f6de0848700 1 --2- 172.21.15.16:0/1598412208 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6ddc133a80 0x7f6ddc071850 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc00c500 tx=0x7f6dcc00c810 comp rx=0 tx=0).stop 2023-09-30T22:32:50.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.148+0000 7f6de0848700 1 -- 172.21.15.16:0/1598412208 shutdown_connections 2023-09-30T22:32:50.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.148+0000 7f6de0848700 1 --2- 172.21.15.16:0/1598412208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6ddc139140 0x7f6ddc1c84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:50.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.148+0000 7f6de0848700 1 --2- 172.21.15.16:0/1598412208 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6ddc133a80 0x7f6ddc071850 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:50.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.148+0000 7f6de0848700 1 --2- 172.21.15.16:0/1598412208 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6ddc136820 0x7f6ddc071d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:50.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.148+0000 7f6de0848700 1 -- 172.21.15.16:0/1598412208 >> 172.21.15.16:0/1598412208 conn(0x7f6ddc0a4210 msgr2=0x7f6ddc12cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:50.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.149+0000 7f6de0848700 1 -- 172.21.15.16:0/1598412208 shutdown_connections 2023-09-30T22:32:50.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.149+0000 7f6de0848700 1 -- 172.21.15.16:0/1598412208 wait complete. 2023-09-30T22:32:50.164 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:32:50.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.163+0000 7fd5855e8700 1 -- 172.21.15.16:0/211204484 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd58011a900 con 0x7fd58006f5d0 2023-09-30T22:32:50.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.163+0000 7fd5677fe700 1 -- 172.21.15.16:0/211204484 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd58011a900 con 0x7fd58006f5d0 2023-09-30T22:32:50.164 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:32:50.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.166+0000 7fd5855e8700 1 -- 172.21.15.16:0/211204484 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd58006f5d0 msgr2=0x7fd580134690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:50.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.166+0000 7fd5855e8700 1 --2- 172.21.15.16:0/211204484 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd58006f5d0 0x7fd580134690 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd57000c700 tx=0x7fd57000ca10 comp rx=0 tx=0).stop 2023-09-30T22:32:50.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.166+0000 7fd5855e8700 1 -- 172.21.15.16:0/211204484 shutdown_connections 2023-09-30T22:32:50.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.166+0000 7fd5855e8700 1 --2- 172.21.15.16:0/211204484 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd580135c20 0x7fd5801c8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:50.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.166+0000 7fd5855e8700 1 --2- 172.21.15.16:0/211204484 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd58006f5d0 0x7fd580134690 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:50.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.166+0000 7fd5855e8700 1 --2- 172.21.15.16:0/211204484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd58012cf60 0x7fd580134bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:50.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.166+0000 7fd5855e8700 1 -- 172.21.15.16:0/211204484 >> 172.21.15.16:0/211204484 conn(0x7fd5800a2aa0 msgr2=0x7fd58012a560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:50.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.166+0000 7fd5855e8700 1 -- 172.21.15.16:0/211204484 shutdown_connections 2023-09-30T22:32:50.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.166+0000 7fd5855e8700 1 -- 172.21.15.16:0/211204484 wait complete. 2023-09-30T22:32:50.180 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:32:50.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.227+0000 7f0dc12e0700 1 -- 172.21.15.16:0/214084997 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0dbc0a0bc0 con 0x7f0dbc131810 2023-09-30T22:32:50.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.228+0000 7f0da37fe700 1 -- 172.21.15.16:0/214084997 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0dac011360 con 0x7f0dbc131810 2023-09-30T22:32:50.228 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:32:50.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.230+0000 7f0dc12e0700 1 -- 172.21.15.16:0/214084997 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0dbc131810 msgr2=0x7f0dbc09c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:50.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.230+0000 7f0dc12e0700 1 --2- 172.21.15.16:0/214084997 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0dbc131810 0x7f0dbc09c800 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0dac00c550 tx=0x7f0dac00c860 comp rx=0 tx=0).stop 2023-09-30T22:32:50.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.230+0000 7f0dc12e0700 1 -- 172.21.15.16:0/214084997 shutdown_connections 2023-09-30T22:32:50.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.230+0000 7f0dc12e0700 1 --2- 172.21.15.16:0/214084997 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0dbc136f50 0x7f0dbc09d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:50.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.230+0000 7f0dc12e0700 1 --2- 172.21.15.16:0/214084997 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0dbc131810 0x7f0dbc09c800 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:50.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.230+0000 7f0dc12e0700 1 --2- 172.21.15.16:0/214084997 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dbc134630 0x7f0dbc09cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:50.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.230+0000 7f0dc12e0700 1 -- 172.21.15.16:0/214084997 >> 172.21.15.16:0/214084997 conn(0x7f0dbc0a2aa0 msgr2=0x7f0dbc099a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:50.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.230+0000 7f0dc12e0700 1 -- 172.21.15.16:0/214084997 shutdown_connections 2023-09-30T22:32:50.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.230+0000 7f0dc12e0700 1 -- 172.21.15.16:0/214084997 wait complete. 2023-09-30T22:32:50.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.232+0000 7fee67a96700 1 -- 172.21.15.16:0/749084289 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fee6011ef40 con 0x7fee6009f650 2023-09-30T22:32:50.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.233+0000 7fee567fc700 1 -- 172.21.15.16:0/749084289 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fee58014e00 con 0x7fee6009f650 2023-09-30T22:32:50.233 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:32:50.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.235+0000 7fee67a96700 1 -- 172.21.15.16:0/749084289 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fee6009f650 msgr2=0x7fee6011de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:50.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.235+0000 7fee67a96700 1 --2- 172.21.15.16:0/749084289 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fee6009f650 0x7fee6011de80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fee5800efc0 tx=0x7fee5800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:32:50.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.235+0000 7fee67a96700 1 -- 172.21.15.16:0/749084289 shutdown_connections 2023-09-30T22:32:50.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.235+0000 7fee67a96700 1 --2- 172.21.15.16:0/749084289 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fee6009f650 0x7fee6011de80 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:50.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.235+0000 7fee67a96700 1 --2- 172.21.15.16:0/749084289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee6009e930 0x7fee6011d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:50.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.235+0000 7fee67a96700 1 --2- 172.21.15.16:0/749084289 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fee6009f240 0x7fee6011d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:50.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.235+0000 7fee67a96700 1 -- 172.21.15.16:0/749084289 >> 172.21.15.16:0/749084289 conn(0x7fee600a4210 msgr2=0x7fee600a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:50.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.235+0000 7fee67a96700 1 -- 172.21.15.16:0/749084289 shutdown_connections 2023-09-30T22:32:50.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:50.235+0000 7fee67a96700 1 -- 172.21.15.16:0/749084289 wait complete. 2023-09-30T22:32:50.246 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:32:50.250 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:32:51.164 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:32:51.181 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:32:51.247 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:32:51.250 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:32:51.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.320+0000 7f6d60c67700 1 Processor -- start 2023-09-30T22:32:51.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.320+0000 7f6d60c67700 1 -- start start 2023-09-30T22:32:51.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.321+0000 7f6d60c67700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d5c1338a0 0x7f6d5c133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:51.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.321+0000 7f6d60c67700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6d5c09de50 0x7f6d5c09e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:51.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.321+0000 7f6d60c67700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d5c09e760 0x7f6d5c13a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:51.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.321+0000 7f6d60c67700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d5c06fef0 con 0x7f6d5c09de50 2023-09-30T22:32:51.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.321+0000 7f6d60c67700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5c06ef30 con 0x7f6d5c09e760 2023-09-30T22:32:51.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.321+0000 7f6d60c67700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5c1334b0 con 0x7f6d5c1338a0 2023-09-30T22:32:51.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.321+0000 7f6d5ad9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d5c09e760 0x7f6d5c13a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:51.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.321+0000 7f6d5ad9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d5c09e760 0x7f6d5c13a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37198/0 (socket says 172.21.15.16:37198) 2023-09-30T22:32:51.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.321+0000 7f6d5ad9d700 1 -- 172.21.15.16:0/2001804836 learned_addr learned my addr 172.21.15.16:0/2001804836 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:51.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.321+0000 7f6d59d9b700 1 --2- 172.21.15.16:0/2001804836 >> v2:172.21.15.16:3300/0 conn(0x7f6d5c09de50 0x7f6d5c09e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:51.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.321+0000 7f6d5a59c700 1 --2- 172.21.15.16:0/2001804836 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d5c1338a0 0x7f6d5c133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:51.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.322+0000 7f6d59d9b700 1 -- 172.21.15.16:0/2001804836 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d5c1338a0 msgr2=0x7f6d5c133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:51.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.322+0000 7f6d59d9b700 1 --2- 172.21.15.16:0/2001804836 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d5c1338a0 0x7f6d5c133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:51.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.322+0000 7f6d59d9b700 1 -- 172.21.15.16:0/2001804836 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d5c09e760 msgr2=0x7f6d5c13a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:51.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.322+0000 7f6d59d9b700 1 --2- 172.21.15.16:0/2001804836 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d5c09e760 0x7f6d5c13a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:51.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.322+0000 7f6d59d9b700 1 -- 172.21.15.16:0/2001804836 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d5c06fc30 con 0x7f6d5c09de50 2023-09-30T22:32:51.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.322+0000 7f6d59d9b700 1 --2- 172.21.15.16:0/2001804836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d5c09de50 0x7f6d5c09e220 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f6d50009840 tx=0x7f6d50009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a5a6bb903437d38 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:51.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.322+0000 7f6d5959a700 1 -- 172.21.15.16:0/2001804836 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d50011830 con 0x7f6d5c09de50 2023-09-30T22:32:51.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.322+0000 7f6d5959a700 1 -- 172.21.15.16:0/2001804836 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d50011990 con 0x7f6d5c09de50 2023-09-30T22:32:51.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.322+0000 7f6d5959a700 1 -- 172.21.15.16:0/2001804836 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d50011d20 con 0x7f6d5c09de50 2023-09-30T22:32:51.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.323+0000 7f6d60c67700 1 -- 172.21.15.16:0/2001804836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d5c09de50 msgr2=0x7f6d5c09e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:51.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.323+0000 7f6d60c67700 1 --2- 172.21.15.16:0/2001804836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d5c09de50 0x7f6d5c09e220 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f6d50009840 tx=0x7f6d50009b50 comp rx=0 tx=0).stop 2023-09-30T22:32:51.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.324+0000 7f6d60c67700 1 -- 172.21.15.16:0/2001804836 shutdown_connections 2023-09-30T22:32:51.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.324+0000 7f6d60c67700 1 --2- 172.21.15.16:0/2001804836 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d5c09e760 0x7f6d5c13a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:51.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.324+0000 7f6d60c67700 1 --2- 172.21.15.16:0/2001804836 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d5c1338a0 0x7f6d5c133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:51.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.324+0000 7f6d60c67700 1 --2- 172.21.15.16:0/2001804836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d5c09de50 0x7f6d5c09e220 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:51.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.324+0000 7f6d60c67700 1 -- 172.21.15.16:0/2001804836 >> 172.21.15.16:0/2001804836 conn(0x7f6d5c0a2aa0 msgr2=0x7f6d5c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:51.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.324+0000 7f6d60c67700 1 -- 172.21.15.16:0/2001804836 shutdown_connections 2023-09-30T22:32:51.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.324+0000 7f6d60c67700 1 -- 172.21.15.16:0/2001804836 wait complete. 2023-09-30T22:32:51.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.325+0000 7f6d60c67700 1 Processor -- start 2023-09-30T22:32:51.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.325+0000 7f6d60c67700 1 -- start start 2023-09-30T22:32:51.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.326+0000 7f6d60c67700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d5c09de50 0x7f6d5c11d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:51.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.326+0000 7f6d60c67700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d5c09e760 0x7f6d5c11dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:51.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.326+0000 7f6d60c67700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d5c1338a0 0x7f6d5c11e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:51.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.326+0000 7f6d60c67700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5c120fe0 con 0x7f6d5c1338a0 2023-09-30T22:32:51.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.326+0000 7f6d60c67700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5c11f330 con 0x7f6d5c09e760 2023-09-30T22:32:51.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.326+0000 7f6d60c67700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5c11f930 con 0x7f6d5c09de50 2023-09-30T22:32:51.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.326+0000 7f6d59d9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d5c09e760 0x7f6d5c11dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:51.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.326+0000 7f6d59d9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d5c09e760 0x7f6d5c11dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37214/0 (socket says 172.21.15.16:37214) 2023-09-30T22:32:51.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.326+0000 7f6d59d9b700 1 -- 172.21.15.16:0/2399677787 learned_addr learned my addr 172.21.15.16:0/2399677787 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:51.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.326+0000 7f6d5a59c700 1 --2- 172.21.15.16:0/2399677787 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d5c09de50 0x7f6d5c11d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:51.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.326+0000 7f6d5ad9d700 1 --2- 172.21.15.16:0/2399677787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d5c1338a0 0x7f6d5c11e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:51.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.327+0000 7f6d59d9b700 1 -- 172.21.15.16:0/2399677787 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d5c09de50 msgr2=0x7f6d5c11d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:51.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.327+0000 7f6d59d9b700 1 --2- 172.21.15.16:0/2399677787 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d5c09de50 0x7f6d5c11d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:51.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.327+0000 7f6d59d9b700 1 -- 172.21.15.16:0/2399677787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d5c1338a0 msgr2=0x7f6d5c11e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:51.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.327+0000 7f6d59d9b700 1 --2- 172.21.15.16:0/2399677787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d5c1338a0 0x7f6d5c11e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:51.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.327+0000 7f6d59d9b700 1 -- 172.21.15.16:0/2399677787 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d500094a0 con 0x7f6d5c09e760 2023-09-30T22:32:51.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.327+0000 7f6d59d9b700 1 --2- 172.21.15.16:0/2399677787 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d5c09e760 0x7f6d5c11dd40 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6d50013000 tx=0x7f6d50004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:51.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.327+0000 7f6d4affd700 1 -- 172.21.15.16:0/2399677787 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d50004450 con 0x7f6d5c09e760 2023-09-30T22:32:51.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.327+0000 7f6d4affd700 1 -- 172.21.15.16:0/2399677787 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d5001e070 con 0x7f6d5c09e760 2023-09-30T22:32:51.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.327+0000 7f6d60c67700 1 -- 172.21.15.16:0/2399677787 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d5c120560 con 0x7f6d5c09e760 2023-09-30T22:32:51.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.327+0000 7f6d4affd700 1 -- 172.21.15.16:0/2399677787 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d500038c0 con 0x7f6d5c09e760 2023-09-30T22:32:51.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.328+0000 7f6d60c67700 1 -- 172.21.15.16:0/2399677787 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d5c06f640 con 0x7f6d5c09e760 2023-09-30T22:32:51.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.329+0000 7f6d4affd700 1 -- 172.21.15.16:0/2399677787 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6d50003a20 con 0x7f6d5c09e760 2023-09-30T22:32:51.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.329+0000 7f6d60c67700 1 -- 172.21.15.16:0/2399677787 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d5c11a470 con 0x7f6d5c09e760 2023-09-30T22:32:51.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.329+0000 7f6d4affd700 1 -- 172.21.15.16:0/2399677787 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f6d50040370 con 0x7f6d5c09e760 2023-09-30T22:32:51.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.334+0000 7f6d4affd700 1 -- 172.21.15.16:0/2399677787 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6d500045b0 con 0x7f6d5c09e760 2023-09-30T22:32:51.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.362+0000 7f11bb2cf700 1 Processor -- start 2023-09-30T22:32:51.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.363+0000 7f11bb2cf700 1 -- start start 2023-09-30T22:32:51.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.363+0000 7f11bb2cf700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f11b406b250 0x7f11b4136840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:51.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.363+0000 7f11bb2cf700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f11b4136d80 0x7f11b4139160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:51.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.363+0000 7f11bb2cf700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11b409ece0 con 0x7f11b406a770 2023-09-30T22:32:51.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.363+0000 7f11bb2cf700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11b409e760 con 0x7f11b406b250 2023-09-30T22:32:51.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.363+0000 7f11bb2cf700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f11b409e040 con 0x7f11b4136d80 2023-09-30T22:32:51.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.364+0000 7f11b906b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f11b406a770 0x7f11b406ab40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53744/0 (socket says 172.21.15.16:53744) 2023-09-30T22:32:51.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.364+0000 7f11b906b700 1 -- 172.21.15.16:0/2696677832 learned_addr learned my addr 172.21.15.16:0/2696677832 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:51.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.364+0000 7f11b886a700 1 --2- 172.21.15.16:0/2696677832 >> v2:172.21.15.16:3300/0 conn(0x7f11b406b250 0x7f11b4136840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:51.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.364+0000 7f11b986c700 1 --2- 172.21.15.16:0/2696677832 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f11b4136d80 0x7f11b4139160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:51.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.364+0000 7f11b886a700 1 -- 172.21.15.16:0/2696677832 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f11b4136d80 msgr2=0x7f11b4139160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:51.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.364+0000 7f11b886a700 1 --2- 172.21.15.16:0/2696677832 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f11b4136d80 0x7f11b4139160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:51.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.364+0000 7f11b886a700 1 -- 172.21.15.16:0/2696677832 >> v1:172.21.15.16:6789/0 conn(0x7f11b406a770 legacy=0x7f11b406ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:51.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.364+0000 7f11b886a700 1 -- 172.21.15.16:0/2696677832 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b409f630 con 0x7f11b406b250 2023-09-30T22:32:51.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.364+0000 7f11b886a700 1 --2- 172.21.15.16:0/2696677832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11b406b250 0x7f11b4136840 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f11a4009840 tx=0x7f11a4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed61d141ac96ae7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:51.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.365+0000 7f11abfff700 1 -- 172.21.15.16:0/2696677832 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a4011830 con 0x7f11b406b250 2023-09-30T22:32:51.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.365+0000 7f11abfff700 1 -- 172.21.15.16:0/2696677832 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11a4011990 con 0x7f11b406b250 2023-09-30T22:32:51.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.365+0000 7f11abfff700 1 -- 172.21.15.16:0/2696677832 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a4011d20 con 0x7f11b406b250 2023-09-30T22:32:51.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.366+0000 7f11bb2cf700 1 -- 172.21.15.16:0/2696677832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11b406b250 msgr2=0x7f11b4136840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:51.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.366+0000 7f11bb2cf700 1 --2- 172.21.15.16:0/2696677832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11b406b250 0x7f11b4136840 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f11a4009840 tx=0x7f11a4009b50 comp rx=0 tx=0).stop 2023-09-30T22:32:51.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.366+0000 7f11bb2cf700 1 -- 172.21.15.16:0/2696677832 shutdown_connections 2023-09-30T22:32:51.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.366+0000 7f11bb2cf700 1 --2- 172.21.15.16:0/2696677832 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f11b4136d80 0x7f11b4139160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:51.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.366+0000 7f11bb2cf700 1 --2- 172.21.15.16:0/2696677832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11b406b250 0x7f11b4136840 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:51.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.366+0000 7f11bb2cf700 1 -- 172.21.15.16:0/2696677832 >> 172.21.15.16:0/2696677832 conn(0x7f11b4066f50 msgr2=0x7f11b409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:51.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.366+0000 7f11bb2cf700 1 -- 172.21.15.16:0/2696677832 shutdown_connections 2023-09-30T22:32:51.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.366+0000 7f11bb2cf700 1 -- 172.21.15.16:0/2696677832 wait complete. 2023-09-30T22:32:51.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.367+0000 7f11bb2cf700 1 Processor -- start 2023-09-30T22:32:51.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.367+0000 7f11bb2cf700 1 -- start start 2023-09-30T22:32:51.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.368+0000 7f11bb2cf700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f11b406a770 0x7f11b41c6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:51.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.368+0000 7f11bb2cf700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f11b406b250 0x7f11b41c65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:51.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.368+0000 7f11bb2cf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11b4136d80 0x7f11b41c6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:51.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.368+0000 7f11bb2cf700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b411f9a0 con 0x7f11b4136d80 2023-09-30T22:32:51.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.368+0000 7f11bb2cf700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11b411e3f0 con 0x7f11b406a770 2023-09-30T22:32:51.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.368+0000 7f11bb2cf700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f11b4071e60 con 0x7f11b406b250 2023-09-30T22:32:51.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.368+0000 7f11b906b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f11b406a770 0x7f11b41c6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:51.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.368+0000 7f11b906b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f11b406a770 0x7f11b41c6070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37222/0 (socket says 172.21.15.16:37222) 2023-09-30T22:32:51.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.368+0000 7f11b906b700 1 -- 172.21.15.16:0/4140062 learned_addr learned my addr 172.21.15.16:0/4140062 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:51.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.369+0000 7f11b886a700 1 --2- 172.21.15.16:0/4140062 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f11b406b250 0x7f11b41c65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:51.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.369+0000 7f11b986c700 1 --2- 172.21.15.16:0/4140062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11b4136d80 0x7f11b41c6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:51.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.369+0000 7f11b906b700 1 -- 172.21.15.16:0/4140062 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f11b406b250 msgr2=0x7f11b41c65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:51.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.369+0000 7f11b906b700 1 --2- 172.21.15.16:0/4140062 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f11b406b250 0x7f11b41c65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:51.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.369+0000 7f11b906b700 1 -- 172.21.15.16:0/4140062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11b4136d80 msgr2=0x7f11b41c6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:51.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.369+0000 7f11b906b700 1 --2- 172.21.15.16:0/4140062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11b4136d80 0x7f11b41c6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:51.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.369+0000 7f11b906b700 1 -- 172.21.15.16:0/4140062 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a40094a0 con 0x7f11b406a770 2023-09-30T22:32:51.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.369+0000 7f11b906b700 1 --2- 172.21.15.16:0/4140062 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f11b406a770 0x7f11b41c6070 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f11b000c520 tx=0x7f11b000c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:51.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.369+0000 7f11a9ffb700 1 -- 172.21.15.16:0/4140062 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b0005d70 con 0x7f11b406a770 2023-09-30T22:32:51.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.369+0000 7f11a9ffb700 1 -- 172.21.15.16:0/4140062 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11b00070d0 con 0x7f11b406a770 2023-09-30T22:32:51.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.369+0000 7f11a9ffb700 1 -- 172.21.15.16:0/4140062 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11b0007440 con 0x7f11b406a770 2023-09-30T22:32:51.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.370+0000 7f11bb2cf700 1 -- 172.21.15.16:0/4140062 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11b411fbe0 con 0x7f11b406a770 2023-09-30T22:32:51.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.371+0000 7f11bb2cf700 1 -- 172.21.15.16:0/4140062 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11b4120910 con 0x7f11b406a770 2023-09-30T22:32:51.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.371+0000 7f11a9ffb700 1 -- 172.21.15.16:0/4140062 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f11b00075a0 con 0x7f11b406a770 2023-09-30T22:32:51.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.372+0000 7f11a9ffb700 1 -- 172.21.15.16:0/4140062 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f11b003ea20 con 0x7f11b406a770 2023-09-30T22:32:51.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.372+0000 7f11bb2cf700 1 -- 172.21.15.16:0/4140062 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11b4074ea0 con 0x7f11b406a770 2023-09-30T22:32:51.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.381+0000 7f11a9ffb700 1 -- 172.21.15.16:0/4140062 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f11b0019b40 con 0x7f11b406a770 2023-09-30T22:32:51.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.401+0000 7f077acc2700 1 Processor -- start 2023-09-30T22:32:51.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.401+0000 7f077acc2700 1 -- start start 2023-09-30T22:32:51.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.402+0000 7f077acc2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f07741338a0 0x7f0774133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:51.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.402+0000 7f077acc2700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f077409e930 0x7f077413b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:51.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.402+0000 7f077acc2700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077406ef30 con 0x7f077409de50 2023-09-30T22:32:51.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.402+0000 7f077acc2700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f077406fef0 con 0x7f077409e930 2023-09-30T22:32:51.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.402+0000 7f077acc2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f07741334b0 con 0x7f07741338a0 2023-09-30T22:32:51.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.402+0000 7f077925f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f077409e930 0x7f077413b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:51.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.402+0000 7f077925f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f077409e930 0x7f077413b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58726/0 (socket says 172.21.15.16:58726) 2023-09-30T22:32:51.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.402+0000 7f0773fff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f077409de50 0x7f077409e220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53746/0 (socket says 172.21.15.16:53746) 2023-09-30T22:32:51.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.402+0000 7f077925f700 1 -- 172.21.15.16:0/3044264233 learned_addr learned my addr 172.21.15.16:0/3044264233 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:51.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.402+0000 7f0778a5e700 1 --2- 172.21.15.16:0/3044264233 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f07741338a0 0x7f0774133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:51.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.402+0000 7f07737fe700 1 -- 172.21.15.16:0/3044264233 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3242319564 0 0) 0x7f077406ef30 con 0x7f077409de50 2023-09-30T22:32:51.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.402+0000 7f07737fe700 1 -- 172.21.15.16:0/3044264233 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07640035b0 con 0x7f077409de50 2023-09-30T22:32:51.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.402+0000 7f0778a5e700 1 -- 172.21.15.16:0/3044264233 >> v2:172.21.15.16:3300/0 conn(0x7f077409e930 msgr2=0x7f077413b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:51.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.402+0000 7f0778a5e700 1 --2- 172.21.15.16:0/3044264233 >> v2:172.21.15.16:3300/0 conn(0x7f077409e930 0x7f077413b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:51.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.402+0000 7f0778a5e700 1 -- 172.21.15.16:0/3044264233 >> v1:172.21.15.16:6789/0 conn(0x7f077409de50 legacy=0x7f077409e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:51.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.402+0000 7f0778a5e700 1 -- 172.21.15.16:0/3044264233 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f077406fc30 con 0x7f07741338a0 2023-09-30T22:32:51.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.403+0000 7f0778a5e700 1 --2- 172.21.15.16:0/3044264233 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f07741338a0 0x7f0774133c70 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0768009840 tx=0x7f0768009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5783e2c5c9533b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:51.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.403+0000 7f07737fe700 1 -- 172.21.15.16:0/3044264233 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0768011830 con 0x7f07741338a0 2023-09-30T22:32:51.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.403+0000 7f07737fe700 1 -- 172.21.15.16:0/3044264233 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0768011990 con 0x7f07741338a0 2023-09-30T22:32:51.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.403+0000 7f07737fe700 1 -- 172.21.15.16:0/3044264233 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0768011d20 con 0x7f07741338a0 2023-09-30T22:32:51.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.404+0000 7f077acc2700 1 -- 172.21.15.16:0/3044264233 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f07741338a0 msgr2=0x7f0774133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:51.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.404+0000 7f077acc2700 1 --2- 172.21.15.16:0/3044264233 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f07741338a0 0x7f0774133c70 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0768009840 tx=0x7f0768009b50 comp rx=0 tx=0).stop 2023-09-30T22:32:51.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.404+0000 7f077acc2700 1 -- 172.21.15.16:0/3044264233 shutdown_connections 2023-09-30T22:32:51.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.404+0000 7f077acc2700 1 --2- 172.21.15.16:0/3044264233 >> v2:172.21.15.16:3300/0 conn(0x7f077409e930 0x7f077413b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:51.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.404+0000 7f077acc2700 1 --2- 172.21.15.16:0/3044264233 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f07741338a0 0x7f0774133c70 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:51.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.404+0000 7f077acc2700 1 -- 172.21.15.16:0/3044264233 >> 172.21.15.16:0/3044264233 conn(0x7f07740a2aa0 msgr2=0x7f07740a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:51.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.404+0000 7f077acc2700 1 -- 172.21.15.16:0/3044264233 shutdown_connections 2023-09-30T22:32:51.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.404+0000 7f077acc2700 1 -- 172.21.15.16:0/3044264233 wait complete. 2023-09-30T22:32:51.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.405+0000 7f077acc2700 1 Processor -- start 2023-09-30T22:32:51.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.405+0000 7f077acc2700 1 -- start start 2023-09-30T22:32:51.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.406+0000 7f077acc2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f077409de50 0x7f077411d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:51.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.406+0000 7f077acc2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f077409e930 0x7f077411dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:51.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.406+0000 7f077acc2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07741338a0 0x7f077411e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:51.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.406+0000 7f077acc2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077411f2d0 con 0x7f07741338a0 2023-09-30T22:32:51.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.406+0000 7f077acc2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0774120520 con 0x7f077409de50 2023-09-30T22:32:51.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.406+0000 7f077acc2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f077411f8b0 con 0x7f077409e930 2023-09-30T22:32:51.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.406+0000 7f0778a5e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f077409de50 0x7f077411d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:51.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.406+0000 7f0778a5e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f077409de50 0x7f077411d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37238/0 (socket says 172.21.15.16:37238) 2023-09-30T22:32:51.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.406+0000 7f0778a5e700 1 -- 172.21.15.16:0/4250903491 learned_addr learned my addr 172.21.15.16:0/4250903491 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:51.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.406+0000 7f0773fff700 1 --2- 172.21.15.16:0/4250903491 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f077409e930 0x7f077411dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:51.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.406+0000 7f077925f700 1 --2- 172.21.15.16:0/4250903491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07741338a0 0x7f077411e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:51.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.407+0000 7f0778a5e700 1 -- 172.21.15.16:0/4250903491 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f077409e930 msgr2=0x7f077411dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:51.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.407+0000 7f0778a5e700 1 --2- 172.21.15.16:0/4250903491 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f077409e930 0x7f077411dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:51.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.407+0000 7f0778a5e700 1 -- 172.21.15.16:0/4250903491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07741338a0 msgr2=0x7f077411e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:51.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.407+0000 7f0778a5e700 1 --2- 172.21.15.16:0/4250903491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07741338a0 0x7f077411e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:51.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.407+0000 7f0778a5e700 1 -- 172.21.15.16:0/4250903491 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07680094a0 con 0x7f077409de50 2023-09-30T22:32:51.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.407+0000 7f077925f700 1 --2- 172.21.15.16:0/4250903491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07741338a0 0x7f077411e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:51.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.407+0000 7f0778a5e700 1 --2- 172.21.15.16:0/4250903491 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f077409de50 0x7f077411d6c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0768000c00 tx=0x7f0768004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:51.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.407+0000 7f07717fa700 1 -- 172.21.15.16:0/4250903491 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0768004590 con 0x7f077409de50 2023-09-30T22:32:51.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.407+0000 7f07717fa700 1 -- 172.21.15.16:0/4250903491 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f076801f070 con 0x7f077409de50 2023-09-30T22:32:51.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.407+0000 7f077acc2700 1 -- 172.21.15.16:0/4250903491 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0774120f00 con 0x7f077409de50 2023-09-30T22:32:51.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.407+0000 7f07717fa700 1 -- 172.21.15.16:0/4250903491 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07680038e0 con 0x7f077409de50 2023-09-30T22:32:51.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.407+0000 7f077acc2700 1 -- 172.21.15.16:0/4250903491 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f077406f640 con 0x7f077409de50 2023-09-30T22:32:51.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.408+0000 7f07717fa700 1 -- 172.21.15.16:0/4250903491 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0768003a40 con 0x7f077409de50 2023-09-30T22:32:51.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.408+0000 7f07717fa700 1 -- 172.21.15.16:0/4250903491 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f0768021070 con 0x7f077409de50 2023-09-30T22:32:51.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.409+0000 7fd6db272700 1 Processor -- start 2023-09-30T22:32:51.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.409+0000 7f077acc2700 1 -- 172.21.15.16:0/4250903491 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f077411a470 con 0x7f077409de50 2023-09-30T22:32:51.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.409+0000 7fd6db272700 1 -- start start 2023-09-30T22:32:51.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.410+0000 7fd6db272700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6d41338c0 0x7fd6d4133c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:51.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.410+0000 7fd6db272700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd6d409e560 0x7fd6d413b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:51.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.410+0000 7fd6db272700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6d406f720 con 0x7fd6d41341d0 2023-09-30T22:32:51.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.410+0000 7fd6db272700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6d4070c80 con 0x7fd6d409e560 2023-09-30T22:32:51.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.410+0000 7fd6db272700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d406fce0 con 0x7fd6d41338c0 2023-09-30T22:32:51.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.410+0000 7fd6d880d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd6d41341d0 0x7fd6d409de50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53750/0 (socket says 172.21.15.16:53750) 2023-09-30T22:32:51.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.410+0000 7fd6d900e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6d41338c0 0x7fd6d4133c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:51.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.410+0000 7fd6d880d700 1 -- 172.21.15.16:0/1796044310 learned_addr learned my addr 172.21.15.16:0/1796044310 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:51.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.410+0000 7fd6d980f700 1 --2- 172.21.15.16:0/1796044310 >> v2:172.21.15.16:3300/0 conn(0x7fd6d409e560 0x7fd6d413b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:51.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.411+0000 7fd6d980f700 1 -- 172.21.15.16:0/1796044310 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6d41338c0 msgr2=0x7fd6d4133c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:51.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.411+0000 7fd6d980f700 1 --2- 172.21.15.16:0/1796044310 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6d41338c0 0x7fd6d4133c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:51.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.411+0000 7fd6d980f700 1 -- 172.21.15.16:0/1796044310 >> v1:172.21.15.16:6789/0 conn(0x7fd6d41341d0 legacy=0x7fd6d409de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:51.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.411+0000 7fd6cbfff700 1 -- 172.21.15.16:0/1796044310 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3194742353 0 0) 0x7fd6d406f720 con 0x7fd6d41341d0 2023-09-30T22:32:51.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.411+0000 7fd6d980f700 1 -- 172.21.15.16:0/1796044310 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d41336a0 con 0x7fd6d409e560 2023-09-30T22:32:51.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.411+0000 7fd6d980f700 1 --2- 172.21.15.16:0/1796044310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6d409e560 0x7fd6d413b860 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc009d90 tx=0x7fd6cc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=407dd519886cee9f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:51.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.411+0000 7fd6cbfff700 1 -- 172.21.15.16:0/1796044310 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc00e070 con 0x7fd6d409e560 2023-09-30T22:32:51.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.412+0000 7fd6cbfff700 1 -- 172.21.15.16:0/1796044310 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6cc00bb20 con 0x7fd6d409e560 2023-09-30T22:32:51.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.412+0000 7fd6cbfff700 1 -- 172.21.15.16:0/1796044310 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc012430 con 0x7fd6d409e560 2023-09-30T22:32:51.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.412+0000 7fd6db272700 1 -- 172.21.15.16:0/1796044310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6d409e560 msgr2=0x7fd6d413b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:51.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.412+0000 7fd6db272700 1 --2- 172.21.15.16:0/1796044310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6d409e560 0x7fd6d413b860 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc009d90 tx=0x7fd6cc00b330 comp rx=0 tx=0).stop 2023-09-30T22:32:51.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.412+0000 7fd6db272700 1 -- 172.21.15.16:0/1796044310 shutdown_connections 2023-09-30T22:32:51.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.412+0000 7fd6db272700 1 --2- 172.21.15.16:0/1796044310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6d409e560 0x7fd6d413b860 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:51.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.412+0000 7fd6db272700 1 --2- 172.21.15.16:0/1796044310 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6d41338c0 0x7fd6d4133c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:51.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.412+0000 7fd6db272700 1 -- 172.21.15.16:0/1796044310 >> 172.21.15.16:0/1796044310 conn(0x7fd6d40a2aa0 msgr2=0x7fd6d40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:51.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.413+0000 7fd6db272700 1 -- 172.21.15.16:0/1796044310 shutdown_connections 2023-09-30T22:32:51.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.413+0000 7fd6db272700 1 -- 172.21.15.16:0/1796044310 wait complete. 2023-09-30T22:32:51.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.414+0000 7fd6db272700 1 Processor -- start 2023-09-30T22:32:51.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.415+0000 7f07717fa700 1 -- 172.21.15.16:0/4250903491 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0768015080 con 0x7f077409de50 2023-09-30T22:32:51.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.414+0000 7fd6db272700 1 -- start start 2023-09-30T22:32:51.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.415+0000 7fd6db272700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6d409e560 0x7fd6d41cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:51.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.415+0000 7fd6db272700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6d41338c0 0x7fd6d41cce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:51.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.415+0000 7fd6db272700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd6d41341d0 0x7fd6d4099a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:51.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.415+0000 7fd6db272700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d406f960 con 0x7fd6d41338c0 2023-09-30T22:32:51.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.415+0000 7fd6db272700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d409b910 con 0x7fd6d409e560 2023-09-30T22:32:51.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.415+0000 7fd6db272700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d4126ef0 con 0x7fd6d41341d0 2023-09-30T22:32:51.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.415+0000 7fd6d900e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6d409e560 0x7fd6d41cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:51.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.415+0000 7fd6d900e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6d409e560 0x7fd6d41cc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37256/0 (socket says 172.21.15.16:37256) 2023-09-30T22:32:51.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.415+0000 7fd6d900e700 1 -- 172.21.15.16:0/2577036873 learned_addr learned my addr 172.21.15.16:0/2577036873 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:51.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.415+0000 7fd6d980f700 1 --2- 172.21.15.16:0/2577036873 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd6d41341d0 0x7fd6d4099a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:51.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.415+0000 7fd6d880d700 1 --2- 172.21.15.16:0/2577036873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6d41338c0 0x7fd6d41cce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:51.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.416+0000 7fd6d980f700 1 -- 172.21.15.16:0/2577036873 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6d409e560 msgr2=0x7fd6d41cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:51.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.416+0000 7fd6d980f700 1 --2- 172.21.15.16:0/2577036873 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6d409e560 0x7fd6d41cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:51.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.416+0000 7fd6d980f700 1 -- 172.21.15.16:0/2577036873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6d41338c0 msgr2=0x7fd6d41cce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:51.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.416+0000 7fd6d980f700 1 --2- 172.21.15.16:0/2577036873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6d41338c0 0x7fd6d41cce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:51.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.416+0000 7fd6d980f700 1 -- 172.21.15.16:0/2577036873 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6cc009a70 con 0x7fd6d41341d0 2023-09-30T22:32:51.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.416+0000 7fd6d980f700 1 --2- 172.21.15.16:0/2577036873 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd6d41341d0 0x7fd6d4099a70 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc016000 tx=0x7fd6cc004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:51.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.416+0000 7fd6c9ffb700 1 -- 172.21.15.16:0/2577036873 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc012430 con 0x7fd6d41341d0 2023-09-30T22:32:51.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.416+0000 7fd6c9ffb700 1 -- 172.21.15.16:0/2577036873 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6cc020070 con 0x7fd6d41341d0 2023-09-30T22:32:51.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.416+0000 7fd6db272700 1 -- 172.21.15.16:0/2577036873 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6d409d310 con 0x7fd6d41341d0 2023-09-30T22:32:51.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.416+0000 7fd6c9ffb700 1 -- 172.21.15.16:0/2577036873 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc004280 con 0x7fd6d41341d0 2023-09-30T22:32:51.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.416+0000 7fd6db272700 1 -- 172.21.15.16:0/2577036873 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6d409d590 con 0x7fd6d41341d0 2023-09-30T22:32:51.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.417+0000 7fd6db272700 1 -- 172.21.15.16:0/2577036873 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6d4074ea0 con 0x7fd6d41341d0 2023-09-30T22:32:51.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.422+0000 7fd6c9ffb700 1 -- 172.21.15.16:0/2577036873 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd6cc0043e0 con 0x7fd6d41341d0 2023-09-30T22:32:51.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.422+0000 7fd6c9ffb700 1 -- 172.21.15.16:0/2577036873 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd6cc022070 con 0x7fd6d41341d0 2023-09-30T22:32:51.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.423+0000 7fd6c9ffb700 1 -- 172.21.15.16:0/2577036873 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd6cc012e40 con 0x7fd6d41341d0 2023-09-30T22:32:51.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.575+0000 7f6d60c67700 1 -- 172.21.15.16:0/2399677787 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6d5c074ea0 con 0x7f6d5c09e760 2023-09-30T22:32:51.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.575+0000 7f6d4affd700 1 -- 172.21.15.16:0/2399677787 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6d50015030 con 0x7f6d5c09e760 2023-09-30T22:32:51.576 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:32:51.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.578+0000 7f6d60c67700 1 -- 172.21.15.16:0/2399677787 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d5c09e760 msgr2=0x7f6d5c11dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:51.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.578+0000 7f6d60c67700 1 --2- 172.21.15.16:0/2399677787 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d5c09e760 0x7f6d5c11dd40 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6d50013000 tx=0x7f6d50004750 comp rx=0 tx=0).stop 2023-09-30T22:32:51.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.579+0000 7f6d60c67700 1 -- 172.21.15.16:0/2399677787 shutdown_connections 2023-09-30T22:32:51.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.579+0000 7f6d60c67700 1 --2- 172.21.15.16:0/2399677787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d5c1338a0 0x7f6d5c11e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:51.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.579+0000 7f6d60c67700 1 --2- 172.21.15.16:0/2399677787 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d5c09de50 0x7f6d5c11d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:51.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.579+0000 7f6d60c67700 1 --2- 172.21.15.16:0/2399677787 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d5c09e760 0x7f6d5c11dd40 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:51.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.579+0000 7f6d60c67700 1 -- 172.21.15.16:0/2399677787 >> 172.21.15.16:0/2399677787 conn(0x7f6d5c0a2aa0 msgr2=0x7f6d5c1bae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:51.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.579+0000 7f6d60c67700 1 -- 172.21.15.16:0/2399677787 shutdown_connections 2023-09-30T22:32:51.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.579+0000 7f6d60c67700 1 -- 172.21.15.16:0/2399677787 wait complete. 2023-09-30T22:32:51.593 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:32:51.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.598+0000 7f11bb2cf700 1 -- 172.21.15.16:0/4140062 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f11b406c360 con 0x7f11b406a770 2023-09-30T22:32:51.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.599+0000 7f11a9ffb700 1 -- 172.21.15.16:0/4140062 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f11b0017030 con 0x7f11b406a770 2023-09-30T22:32:51.599 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:32:51.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.601+0000 7f11bb2cf700 1 -- 172.21.15.16:0/4140062 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f11b406a770 msgr2=0x7f11b41c6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:51.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.601+0000 7f11bb2cf700 1 --2- 172.21.15.16:0/4140062 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f11b406a770 0x7f11b41c6070 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f11b000c520 tx=0x7f11b000c830 comp rx=0 tx=0).stop 2023-09-30T22:32:51.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.601+0000 7f11bb2cf700 1 -- 172.21.15.16:0/4140062 shutdown_connections 2023-09-30T22:32:51.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.601+0000 7f11bb2cf700 1 --2- 172.21.15.16:0/4140062 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f11b406a770 0x7f11b41c6070 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:51.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.601+0000 7f11bb2cf700 1 --2- 172.21.15.16:0/4140062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11b4136d80 0x7f11b41c6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:51.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.601+0000 7f11bb2cf700 1 --2- 172.21.15.16:0/4140062 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f11b406b250 0x7f11b41c65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:51.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.601+0000 7f11bb2cf700 1 -- 172.21.15.16:0/4140062 >> 172.21.15.16:0/4140062 conn(0x7f11b4066f50 msgr2=0x7f11b40a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:51.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.601+0000 7f11bb2cf700 1 -- 172.21.15.16:0/4140062 shutdown_connections 2023-09-30T22:32:51.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.601+0000 7f11bb2cf700 1 -- 172.21.15.16:0/4140062 wait complete. 2023-09-30T22:32:51.616 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:32:51.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.633+0000 7fd6db272700 1 -- 172.21.15.16:0/2577036873 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd6d40a1040 con 0x7fd6d41341d0 2023-09-30T22:32:51.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.633+0000 7fd6c9ffb700 1 -- 172.21.15.16:0/2577036873 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd6cc01ab70 con 0x7fd6d41341d0 2023-09-30T22:32:51.634 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:32:51.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.636+0000 7fd6db272700 1 -- 172.21.15.16:0/2577036873 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd6d41341d0 msgr2=0x7fd6d4099a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:51.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.636+0000 7fd6db272700 1 --2- 172.21.15.16:0/2577036873 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd6d41341d0 0x7fd6d4099a70 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc016000 tx=0x7fd6cc004ad0 comp rx=0 tx=0).stop 2023-09-30T22:32:51.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.636+0000 7fd6db272700 1 -- 172.21.15.16:0/2577036873 shutdown_connections 2023-09-30T22:32:51.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.636+0000 7fd6db272700 1 --2- 172.21.15.16:0/2577036873 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6d409e560 0x7fd6d41cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:51.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.636+0000 7fd6db272700 1 --2- 172.21.15.16:0/2577036873 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd6d41341d0 0x7fd6d4099a70 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:51.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.636+0000 7fd6db272700 1 --2- 172.21.15.16:0/2577036873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6d41338c0 0x7fd6d41cce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:51.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.636+0000 7fd6db272700 1 -- 172.21.15.16:0/2577036873 >> 172.21.15.16:0/2577036873 conn(0x7fd6d40a2aa0 msgr2=0x7fd6d4130730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:51.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.636+0000 7fd6db272700 1 -- 172.21.15.16:0/2577036873 shutdown_connections 2023-09-30T22:32:51.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.636+0000 7fd6db272700 1 -- 172.21.15.16:0/2577036873 wait complete. 2023-09-30T22:32:51.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.644+0000 7f077acc2700 1 -- 172.21.15.16:0/4250903491 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0774074ea0 con 0x7f077409de50 2023-09-30T22:32:51.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.644+0000 7f07717fa700 1 -- 172.21.15.16:0/4250903491 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0768035090 con 0x7f077409de50 2023-09-30T22:32:51.645 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:32:51.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.647+0000 7f077acc2700 1 -- 172.21.15.16:0/4250903491 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f077409de50 msgr2=0x7f077411d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:51.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.647+0000 7f077acc2700 1 --2- 172.21.15.16:0/4250903491 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f077409de50 0x7f077411d6c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0768000c00 tx=0x7f0768004750 comp rx=0 tx=0).stop 2023-09-30T22:32:51.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.647+0000 7f077acc2700 1 -- 172.21.15.16:0/4250903491 shutdown_connections 2023-09-30T22:32:51.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.647+0000 7f077acc2700 1 --2- 172.21.15.16:0/4250903491 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f077409de50 0x7f077411d6c0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:51.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.647+0000 7f077acc2700 1 --2- 172.21.15.16:0/4250903491 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f07741338a0 0x7f077411e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:51.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.647+0000 7f077acc2700 1 --2- 172.21.15.16:0/4250903491 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f077409e930 0x7f077411dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:51.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.647+0000 7f077acc2700 1 -- 172.21.15.16:0/4250903491 >> 172.21.15.16:0/4250903491 conn(0x7f07740a2aa0 msgr2=0x7f0774132bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:51.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.647+0000 7f077acc2700 1 -- 172.21.15.16:0/4250903491 shutdown_connections 2023-09-30T22:32:51.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:51.647+0000 7f077acc2700 1 -- 172.21.15.16:0/4250903491 wait complete. 2023-09-30T22:32:51.650 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:32:51.662 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:32:52.594 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:32:52.616 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:32:52.651 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:32:52.663 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:32:52.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.748+0000 7fb618b83700 1 Processor -- start 2023-09-30T22:32:52.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.748+0000 7fb618b83700 1 -- start start 2023-09-30T22:32:52.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.749+0000 7fb618b83700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb61409d6f0 0x7fb61409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:52.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.749+0000 7fb618b83700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb61409e410 0x7fb614134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:52.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.749+0000 7fb618b83700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb61409b120 con 0x7fb61409e000 2023-09-30T22:32:52.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.749+0000 7fb618b83700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb61406b460 con 0x7fb61409e410 2023-09-30T22:32:52.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.749+0000 7fb618b83700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb61409a7e0 con 0x7fb61409d6f0 2023-09-30T22:32:52.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.749+0000 7fb611d9b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb61409e000 0x7fb6141339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53760/0 (socket says 172.21.15.16:53760) 2023-09-30T22:32:52.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.749+0000 7fb61259c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb61409d6f0 0x7fb61409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:52.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.749+0000 7fb611d9b700 1 -- 172.21.15.16:0/3221234767 learned_addr learned my addr 172.21.15.16:0/3221234767 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:52.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.749+0000 7fb612d9d700 1 --2- 172.21.15.16:0/3221234767 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb61409e410 0x7fb614134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:52.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.749+0000 7fb61259c700 1 -- 172.21.15.16:0/3221234767 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb61409e410 msgr2=0x7fb614134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:52.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.749+0000 7fb61259c700 1 --2- 172.21.15.16:0/3221234767 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb61409e410 0x7fb614134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:52.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.749+0000 7fb61259c700 1 -- 172.21.15.16:0/3221234767 >> v1:172.21.15.16:6789/0 conn(0x7fb61409e000 legacy=0x7fb6141339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:52.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.749+0000 7fb61259c700 1 -- 172.21.15.16:0/3221234767 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb61409a500 con 0x7fb61409d6f0 2023-09-30T22:32:52.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.750+0000 7fb61259c700 1 --2- 172.21.15.16:0/3221234767 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb61409d6f0 0x7fb61409dac0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb6040098a0 tx=0x7fb604009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ebc7e3557144a55 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:52.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.750+0000 7fb61159a700 1 -- 172.21.15.16:0/3221234767 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb604004230 con 0x7fb61409d6f0 2023-09-30T22:32:52.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.750+0000 7fb61159a700 1 -- 172.21.15.16:0/3221234767 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb604004390 con 0x7fb61409d6f0 2023-09-30T22:32:52.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.750+0000 7fb61159a700 1 -- 172.21.15.16:0/3221234767 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6040036a0 con 0x7fb61409d6f0 2023-09-30T22:32:52.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.751+0000 7fb618b83700 1 -- 172.21.15.16:0/3221234767 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb61409d6f0 msgr2=0x7fb61409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:52.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.751+0000 7fb618b83700 1 --2- 172.21.15.16:0/3221234767 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb61409d6f0 0x7fb61409dac0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb6040098a0 tx=0x7fb604009bb0 comp rx=0 tx=0).stop 2023-09-30T22:32:52.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.751+0000 7fb618b83700 1 -- 172.21.15.16:0/3221234767 shutdown_connections 2023-09-30T22:32:52.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.751+0000 7fb618b83700 1 --2- 172.21.15.16:0/3221234767 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb61409e410 0x7fb614134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:52.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.751+0000 7fb618b83700 1 --2- 172.21.15.16:0/3221234767 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb61409d6f0 0x7fb61409dac0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:52.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.751+0000 7fb618b83700 1 -- 172.21.15.16:0/3221234767 >> 172.21.15.16:0/3221234767 conn(0x7fb614066f50 msgr2=0x7fb61409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:52.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.751+0000 7fb618b83700 1 -- 172.21.15.16:0/3221234767 shutdown_connections 2023-09-30T22:32:52.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.751+0000 7fb618b83700 1 -- 172.21.15.16:0/3221234767 wait complete. 2023-09-30T22:32:52.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.753+0000 7fb618b83700 1 Processor -- start 2023-09-30T22:32:52.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.753+0000 7fb618b83700 1 -- start start 2023-09-30T22:32:52.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.754+0000 7fb618b83700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb61409d6f0 0x7fb6141225b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:52.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.754+0000 7fb618b83700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb61409e000 0x7fb614122af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.754+0000 7fb618b83700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb61409e410 0x7fb614123030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.754+0000 7fb618b83700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb614125410 con 0x7fb61409e000 2023-09-30T22:32:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.754+0000 7fb618b83700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6140a1d90 con 0x7fb61409e410 2023-09-30T22:32:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.754+0000 7fb618b83700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb61409c710 con 0x7fb61409d6f0 2023-09-30T22:32:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.754+0000 7fb61259c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb61409d6f0 0x7fb6141225b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.754+0000 7fb61259c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb61409d6f0 0x7fb6141225b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:47382/0 (socket says 172.21.15.16:47382) 2023-09-30T22:32:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.754+0000 7fb61259c700 1 -- 172.21.15.16:0/1887883029 learned_addr learned my addr 172.21.15.16:0/1887883029 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.754+0000 7fb611d9b700 1 --2- 172.21.15.16:0/1887883029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb61409e000 0x7fb614122af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.754+0000 7fb612d9d700 1 --2- 172.21.15.16:0/1887883029 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb61409e410 0x7fb614123030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.754+0000 7fb61259c700 1 -- 172.21.15.16:0/1887883029 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb61409e410 msgr2=0x7fb614123030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:52.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.754+0000 7fb61259c700 1 --2- 172.21.15.16:0/1887883029 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb61409e410 0x7fb614123030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:52.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.754+0000 7fb61259c700 1 -- 172.21.15.16:0/1887883029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb61409e000 msgr2=0x7fb614122af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:52.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.754+0000 7fb61259c700 1 --2- 172.21.15.16:0/1887883029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb61409e000 0x7fb614122af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:52.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.754+0000 7fb61259c700 1 -- 172.21.15.16:0/1887883029 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6040094a0 con 0x7fb61409d6f0 2023-09-30T22:32:52.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.755+0000 7fb61259c700 1 --2- 172.21.15.16:0/1887883029 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb61409d6f0 0x7fb6141225b0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb604009870 tx=0x7fb604004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:52.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.755+0000 7fb602ffd700 1 -- 172.21.15.16:0/1887883029 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb604003cf0 con 0x7fb61409d6f0 2023-09-30T22:32:52.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.755+0000 7fb602ffd700 1 -- 172.21.15.16:0/1887883029 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb60401e070 con 0x7fb61409d6f0 2023-09-30T22:32:52.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.755+0000 7fb618b83700 1 -- 172.21.15.16:0/1887883029 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb614124f50 con 0x7fb61409d6f0 2023-09-30T22:32:52.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.755+0000 7fb602ffd700 1 -- 172.21.15.16:0/1887883029 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb604011870 con 0x7fb61409d6f0 2023-09-30T22:32:52.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.755+0000 7fb618b83700 1 -- 172.21.15.16:0/1887883029 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb614125d60 con 0x7fb61409d6f0 2023-09-30T22:32:52.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.756+0000 7fb602ffd700 1 -- 172.21.15.16:0/1887883029 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb6040119d0 con 0x7fb61409d6f0 2023-09-30T22:32:52.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.756+0000 7fb618b83700 1 -- 172.21.15.16:0/1887883029 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6141259b0 con 0x7fb61409d6f0 2023-09-30T22:32:52.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.756+0000 7fb602ffd700 1 -- 172.21.15.16:0/1887883029 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fb6040405f0 con 0x7fb61409d6f0 2023-09-30T22:32:52.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.761+0000 7fb602ffd700 1 -- 172.21.15.16:0/1887883029 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb604010740 con 0x7fb61409d6f0 2023-09-30T22:32:52.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.790+0000 7f977939b700 1 Processor -- start 2023-09-30T22:32:52.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.790+0000 7f977939b700 1 -- start start 2023-09-30T22:32:52.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.791+0000 7f977939b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f977409e1d0 0x7f9774138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:52.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.791+0000 7f977939b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9774139170 0x7f977413b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:52.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.791+0000 7f977939b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f977409b120 con 0x7f977409d6f0 2023-09-30T22:32:52.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.791+0000 7f977939b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f977406b460 con 0x7f9774139170 2023-09-30T22:32:52.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.791+0000 7f977939b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f977409a7e0 con 0x7f977409e1d0 2023-09-30T22:32:52.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.791+0000 7f97727fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f977409e1d0 0x7f9774138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:52.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.791+0000 7f97727fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f977409e1d0 0x7f9774138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37280/0 (socket says 172.21.15.16:37280) 2023-09-30T22:32:52.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.791+0000 7f97727fc700 1 -- 172.21.15.16:0/833344490 learned_addr learned my addr 172.21.15.16:0/833344490 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:52.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.791+0000 7f97737fe700 1 --2- 172.21.15.16:0/833344490 >> v2:172.21.15.16:3300/0 conn(0x7f9774139170 0x7f977413b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:52.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.791+0000 7f97727fc700 1 -- 172.21.15.16:0/833344490 >> v2:172.21.15.16:3300/0 conn(0x7f9774139170 msgr2=0x7f977413b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:52.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.791+0000 7f97727fc700 1 --2- 172.21.15.16:0/833344490 >> v2:172.21.15.16:3300/0 conn(0x7f9774139170 0x7f977413b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:52.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.791+0000 7f97727fc700 1 -- 172.21.15.16:0/833344490 >> v1:172.21.15.16:6789/0 conn(0x7f977409d6f0 legacy=0x7f977409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:52.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.791+0000 7f97727fc700 1 -- 172.21.15.16:0/833344490 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f977409a500 con 0x7f977409e1d0 2023-09-30T22:32:52.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.792+0000 7f97727fc700 1 --2- 172.21.15.16:0/833344490 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f977409e1d0 0x7f9774138c30 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f9768009920 tx=0x7f9768009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f9cd900466353e8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:52.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.792+0000 7f9771ffb700 1 -- 172.21.15.16:0/833344490 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9768004230 con 0x7f977409e1d0 2023-09-30T22:32:52.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.792+0000 7f9771ffb700 1 -- 172.21.15.16:0/833344490 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9768004390 con 0x7f977409e1d0 2023-09-30T22:32:52.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.792+0000 7f9771ffb700 1 -- 172.21.15.16:0/833344490 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97680036a0 con 0x7f977409e1d0 2023-09-30T22:32:52.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.793+0000 7f977939b700 1 -- 172.21.15.16:0/833344490 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f977409e1d0 msgr2=0x7f9774138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:52.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.793+0000 7f977939b700 1 --2- 172.21.15.16:0/833344490 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f977409e1d0 0x7f9774138c30 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f9768009920 tx=0x7f9768009c30 comp rx=0 tx=0).stop 2023-09-30T22:32:52.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.793+0000 7f977939b700 1 -- 172.21.15.16:0/833344490 shutdown_connections 2023-09-30T22:32:52.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.793+0000 7f977939b700 1 --2- 172.21.15.16:0/833344490 >> v2:172.21.15.16:3300/0 conn(0x7f9774139170 0x7f977413b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:52.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.793+0000 7f977939b700 1 --2- 172.21.15.16:0/833344490 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f977409e1d0 0x7f9774138c30 secure :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f9768009920 tx=0x7f9768009c30 comp rx=0 tx=0).stop 2023-09-30T22:32:52.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.793+0000 7f977939b700 1 -- 172.21.15.16:0/833344490 >> 172.21.15.16:0/833344490 conn(0x7f9774066f50 msgr2=0x7f977409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:52.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.793+0000 7f977939b700 1 -- 172.21.15.16:0/833344490 shutdown_connections 2023-09-30T22:32:52.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.793+0000 7f977939b700 1 -- 172.21.15.16:0/833344490 wait complete. 2023-09-30T22:32:52.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.795+0000 7f977939b700 1 Processor -- start 2023-09-30T22:32:52.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.795+0000 7f977939b700 1 -- start start 2023-09-30T22:32:52.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.796+0000 7f977939b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f977409d6f0 0x7f97741c8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:52.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.796+0000 7f977939b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9774139170 0x7f97741c89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:52.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.796+0000 7f977939b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f97741c8ee0 0x7f9774133500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:52.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.796+0000 7f977939b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9774120800 con 0x7f9774139170 2023-09-30T22:32:52.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.796+0000 7f977939b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9774121ff0 con 0x7f977409d6f0 2023-09-30T22:32:52.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.796+0000 7f977939b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9774121d20 con 0x7f97741c8ee0 2023-09-30T22:32:52.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.796+0000 7f9772ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f977409d6f0 0x7f97741c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:52.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.796+0000 7f9772ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f977409d6f0 0x7f97741c8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37290/0 (socket says 172.21.15.16:37290) 2023-09-30T22:32:52.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.796+0000 7f9772ffd700 1 -- 172.21.15.16:0/2909185368 learned_addr learned my addr 172.21.15.16:0/2909185368 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:52.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.796+0000 7f97727fc700 1 --2- 172.21.15.16:0/2909185368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9774139170 0x7f97741c89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:52.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.796+0000 7f97737fe700 1 --2- 172.21.15.16:0/2909185368 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f97741c8ee0 0x7f9774133500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:52.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.796+0000 7f97727fc700 1 -- 172.21.15.16:0/2909185368 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f97741c8ee0 msgr2=0x7f9774133500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:52.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.796+0000 7f97727fc700 1 --2- 172.21.15.16:0/2909185368 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f97741c8ee0 0x7f9774133500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:52.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.796+0000 7f97727fc700 1 -- 172.21.15.16:0/2909185368 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f977409d6f0 msgr2=0x7f97741c8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:52.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.796+0000 7f97727fc700 1 --2- 172.21.15.16:0/2909185368 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f977409d6f0 0x7f97741c8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:52.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.796+0000 7f97727fc700 1 -- 172.21.15.16:0/2909185368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9768009550 con 0x7f9774139170 2023-09-30T22:32:52.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.797+0000 7f97727fc700 1 --2- 172.21.15.16:0/2909185368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9774139170 0x7f97741c89a0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f9768015000 tx=0x7f9768004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:52.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.797+0000 7f975b7fe700 1 -- 172.21.15.16:0/2909185368 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9768003cb0 con 0x7f9774139170 2023-09-30T22:32:52.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.797+0000 7f975b7fe700 1 -- 172.21.15.16:0/2909185368 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9768003e30 con 0x7f9774139170 2023-09-30T22:32:52.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.797+0000 7f977939b700 1 -- 172.21.15.16:0/2909185368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f977409c710 con 0x7f9774139170 2023-09-30T22:32:52.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.797+0000 7f975b7fe700 1 -- 172.21.15.16:0/2909185368 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9768011870 con 0x7f9774139170 2023-09-30T22:32:52.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.797+0000 7f977939b700 1 -- 172.21.15.16:0/2909185368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9774122d20 con 0x7f9774139170 2023-09-30T22:32:52.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.798+0000 7f975b7fe700 1 -- 172.21.15.16:0/2909185368 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9768021070 con 0x7f9774139170 2023-09-30T22:32:52.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.798+0000 7f975b7fe700 1 -- 172.21.15.16:0/2909185368 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f9768040c40 con 0x7f9774139170 2023-09-30T22:32:52.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.799+0000 7f977939b700 1 -- 172.21.15.16:0/2909185368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9774122210 con 0x7f9774139170 2023-09-30T22:32:52.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.804+0000 7f975b7fe700 1 -- 172.21.15.16:0/2909185368 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9774122210 con 0x7f9774139170 2023-09-30T22:32:52.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.835+0000 7fbd8feaa700 1 Processor -- start 2023-09-30T22:32:52.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.835+0000 7fbd8feaa700 1 -- start start 2023-09-30T22:32:52.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.836+0000 7fbd8feaa700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbd8809d6f0 0x7fbd8809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:52.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.836+0000 7fbd8feaa700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbd8809e000 0x7fbd88130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:52.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.836+0000 7fbd8feaa700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd8809f130 con 0x7fbd8809e410 2023-09-30T22:32:52.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.836+0000 7fbd8feaa700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8809f520 con 0x7fbd8809e000 2023-09-30T22:32:52.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.836+0000 7fbd8feaa700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8809e930 con 0x7fbd8809d6f0 2023-09-30T22:32:52.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.836+0000 7fbd8e447700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fbd8809e410 0x7fbd88130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53776/0 (socket says 172.21.15.16:53776) 2023-09-30T22:32:52.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.836+0000 7fbd8e447700 1 -- 172.21.15.16:0/1668211396 learned_addr learned my addr 172.21.15.16:0/1668211396 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:52.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.836+0000 7fbd8dc46700 1 --2- 172.21.15.16:0/1668211396 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbd8809d6f0 0x7fbd8809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:52.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.836+0000 7fbd8d445700 1 --2- 172.21.15.16:0/1668211396 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbd8809e000 0x7fbd88130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:52.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.836+0000 7fbd8cc44700 1 -- 172.21.15.16:0/1668211396 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3421033229 0 0) 0x7fbd8809f130 con 0x7fbd8809e410 2023-09-30T22:32:52.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.836+0000 7fbd8cc44700 1 -- 172.21.15.16:0/1668211396 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd740035b0 con 0x7fbd8809e410 2023-09-30T22:32:52.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.837+0000 7fbd8d445700 1 -- 172.21.15.16:0/1668211396 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbd8809d6f0 msgr2=0x7fbd8809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:52.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.837+0000 7fbd8d445700 1 --2- 172.21.15.16:0/1668211396 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbd8809d6f0 0x7fbd8809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:52.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.837+0000 7fbd8d445700 1 -- 172.21.15.16:0/1668211396 >> v1:172.21.15.16:6789/0 conn(0x7fbd8809e410 legacy=0x7fbd88130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:52.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.837+0000 7fbd8cc44700 1 -- 172.21.15.16:0/1668211396 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2246701581 0 0) 0x7fbd740035b0 con 0x7fbd8809e410 2023-09-30T22:32:52.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.837+0000 7fbd8d445700 1 -- 172.21.15.16:0/1668211396 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd8809b2d0 con 0x7fbd8809e000 2023-09-30T22:32:52.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.837+0000 7fbd8d445700 1 --2- 172.21.15.16:0/1668211396 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbd8809e000 0x7fbd88130660 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fbd78009870 tx=0x7fbd78009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b58ca22cf1a6af6c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:52.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.837+0000 7fbd8cc44700 1 -- 172.21.15.16:0/1668211396 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd800019f0 con 0x7fbd8809e000 2023-09-30T22:32:52.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.837+0000 7fbd8cc44700 1 -- 172.21.15.16:0/1668211396 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd78004030 con 0x7fbd8809e000 2023-09-30T22:32:52.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.837+0000 7fbd8cc44700 1 -- 172.21.15.16:0/1668211396 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd780043c0 con 0x7fbd8809e000 2023-09-30T22:32:52.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.838+0000 7fbd8feaa700 1 -- 172.21.15.16:0/1668211396 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbd8809e000 msgr2=0x7fbd88130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:52.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.838+0000 7fbd8feaa700 1 --2- 172.21.15.16:0/1668211396 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbd8809e000 0x7fbd88130660 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fbd78009870 tx=0x7fbd78009b80 comp rx=0 tx=0).stop 2023-09-30T22:32:52.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.838+0000 7fbd8feaa700 1 -- 172.21.15.16:0/1668211396 shutdown_connections 2023-09-30T22:32:52.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.838+0000 7fbd8feaa700 1 --2- 172.21.15.16:0/1668211396 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbd8809d6f0 0x7fbd8809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:52.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.838+0000 7fbd8feaa700 1 --2- 172.21.15.16:0/1668211396 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbd8809e000 0x7fbd88130660 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:52.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.838+0000 7fbd8feaa700 1 -- 172.21.15.16:0/1668211396 >> 172.21.15.16:0/1668211396 conn(0x7fbd88066f50 msgr2=0x7fbd8809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:52.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.838+0000 7fbd8feaa700 1 -- 172.21.15.16:0/1668211396 shutdown_connections 2023-09-30T22:32:52.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.838+0000 7fbd8feaa700 1 -- 172.21.15.16:0/1668211396 wait complete. 2023-09-30T22:32:52.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.839+0000 7fbd8feaa700 1 Processor -- start 2023-09-30T22:32:52.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.839+0000 7fbd8feaa700 1 -- start start 2023-09-30T22:32:52.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.840+0000 7fbd8feaa700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbd8809d6f0 0x7fbd88139940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:52.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.840+0000 7fbd8feaa700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbd8809e000 0x7fbd88139e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:52.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.840+0000 7fbd8feaa700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd8809e410 0x7fbd8813a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:52.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.840+0000 7fbd8feaa700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd88135e20 con 0x7fbd8809e410 2023-09-30T22:32:52.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.840+0000 7fbd8feaa700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8813ae30 con 0x7fbd8809e000 2023-09-30T22:32:52.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.840+0000 7fbd8feaa700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbd881365c0 con 0x7fbd8809d6f0 2023-09-30T22:32:52.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.840+0000 7fbd8dc46700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbd8809d6f0 0x7fbd88139940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:52.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.840+0000 7fbd8e447700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd8809e410 0x7fbd8813a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:52.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.840+0000 7fbd8dc46700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbd8809d6f0 0x7fbd88139940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:47412/0 (socket says 172.21.15.16:47412) 2023-09-30T22:32:52.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.840+0000 7fbd8dc46700 1 -- 172.21.15.16:0/3930698736 learned_addr learned my addr 172.21.15.16:0/3930698736 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:52.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.840+0000 7fbd8d445700 1 --2- 172.21.15.16:0/3930698736 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbd8809e000 0x7fbd88139e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:52.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.840+0000 7fbd8dc46700 1 -- 172.21.15.16:0/3930698736 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbd8809e000 msgr2=0x7fbd88139e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:52.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.840+0000 7fbd8dc46700 1 --2- 172.21.15.16:0/3930698736 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbd8809e000 0x7fbd88139e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:52.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.840+0000 7fbd8dc46700 1 -- 172.21.15.16:0/3930698736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd8809e410 msgr2=0x7fbd8813a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:52.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.840+0000 7fbd8dc46700 1 --2- 172.21.15.16:0/3930698736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd8809e410 0x7fbd8813a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:52.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.840+0000 7fbd8dc46700 1 -- 172.21.15.16:0/3930698736 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd780094a0 con 0x7fbd8809d6f0 2023-09-30T22:32:52.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.840+0000 7fbd8e447700 1 --2- 172.21.15.16:0/3930698736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd8809e410 0x7fbd8813a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:52.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.841+0000 7fbd8dc46700 1 --2- 172.21.15.16:0/3930698736 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbd8809d6f0 0x7fbd88139940 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fbd8400e8a0 tx=0x7fbd8400ebb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:52.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.841+0000 7fbd7e7fc700 1 -- 172.21.15.16:0/3930698736 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd84009590 con 0x7fbd8809d6f0 2023-09-30T22:32:52.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.841+0000 7fbd7e7fc700 1 -- 172.21.15.16:0/3930698736 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd84013680 con 0x7fbd8809d6f0 2023-09-30T22:32:52.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.841+0000 7fbd8feaa700 1 -- 172.21.15.16:0/3930698736 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd88136860 con 0x7fbd8809d6f0 2023-09-30T22:32:52.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.841+0000 7fbd7e7fc700 1 -- 172.21.15.16:0/3930698736 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd840139f0 con 0x7fbd8809d6f0 2023-09-30T22:32:52.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.841+0000 7fbd8feaa700 1 -- 172.21.15.16:0/3930698736 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd8813b6d0 con 0x7fbd8809d6f0 2023-09-30T22:32:52.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.842+0000 7fbd7e7fc700 1 -- 172.21.15.16:0/3930698736 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbd84013b50 con 0x7fbd8809d6f0 2023-09-30T22:32:52.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.842+0000 7fbd8feaa700 1 -- 172.21.15.16:0/3930698736 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd88074ea0 con 0x7fbd8809d6f0 2023-09-30T22:32:52.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.842+0000 7fbd7e7fc700 1 -- 172.21.15.16:0/3930698736 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fbd8403f5a0 con 0x7fbd8809d6f0 2023-09-30T22:32:52.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.848+0000 7f74937d6700 1 Processor -- start 2023-09-30T22:32:52.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.848+0000 7f74937d6700 1 -- start start 2023-09-30T22:32:52.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.848+0000 7f74937d6700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f748c1338a0 0x7f748c133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:52.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.848+0000 7f74937d6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f748c09e930 0x7f748c13b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:52.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.848+0000 7f74937d6700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f748c06ef30 con 0x7f748c09de50 2023-09-30T22:32:52.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.848+0000 7f74937d6700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f748c06fef0 con 0x7f748c1338a0 2023-09-30T22:32:52.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.848+0000 7f74937d6700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f748c1334b0 con 0x7f748c09e930 2023-09-30T22:32:52.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.849+0000 7f7491d73700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f748c09e930 0x7f748c13b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:52.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.849+0000 7f7491572700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f748c1338a0 0x7f748c133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:52.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.849+0000 7f7491d73700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f748c09e930 0x7f748c13b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37306/0 (socket says 172.21.15.16:37306) 2023-09-30T22:32:52.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.849+0000 7f7491d73700 1 -- 172.21.15.16:0/76312976 learned_addr learned my addr 172.21.15.16:0/76312976 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:52.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.849+0000 7f7491d73700 1 -- 172.21.15.16:0/76312976 >> v2:172.21.15.16:3300/0 conn(0x7f748c1338a0 msgr2=0x7f748c133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:52.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.849+0000 7f7491d73700 1 --2- 172.21.15.16:0/76312976 >> v2:172.21.15.16:3300/0 conn(0x7f748c1338a0 0x7f748c133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:52.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.849+0000 7f7491d73700 1 -- 172.21.15.16:0/76312976 >> v1:172.21.15.16:6789/0 conn(0x7f748c09de50 legacy=0x7f748c09e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:52.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.849+0000 7f7491d73700 1 -- 172.21.15.16:0/76312976 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f748c06fc30 con 0x7f748c09e930 2023-09-30T22:32:52.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.849+0000 7f7491d73700 1 --2- 172.21.15.16:0/76312976 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f748c09e930 0x7f748c13b740 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f7484009e50 tx=0x7f748400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c19eaecc2b7491d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:52.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.850+0000 7f7483fff700 1 -- 172.21.15.16:0/76312976 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748400e070 con 0x7f748c09e930 2023-09-30T22:32:52.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.850+0000 7f7483fff700 1 -- 172.21.15.16:0/76312976 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f748400b910 con 0x7f748c09e930 2023-09-30T22:32:52.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.850+0000 7f7483fff700 1 -- 172.21.15.16:0/76312976 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748400bcc0 con 0x7f748c09e930 2023-09-30T22:32:52.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.850+0000 7fbd7e7fc700 1 -- 172.21.15.16:0/3930698736 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbd84021020 con 0x7fbd8809d6f0 2023-09-30T22:32:52.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.850+0000 7f74937d6700 1 -- 172.21.15.16:0/76312976 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f748c09e930 msgr2=0x7f748c13b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:52.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.850+0000 7f74937d6700 1 --2- 172.21.15.16:0/76312976 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f748c09e930 0x7f748c13b740 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f7484009e50 tx=0x7f748400b330 comp rx=0 tx=0).stop 2023-09-30T22:32:52.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.850+0000 7f74937d6700 1 -- 172.21.15.16:0/76312976 shutdown_connections 2023-09-30T22:32:52.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.850+0000 7f74937d6700 1 --2- 172.21.15.16:0/76312976 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f748c09e930 0x7f748c13b740 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:52.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.850+0000 7f74937d6700 1 --2- 172.21.15.16:0/76312976 >> v2:172.21.15.16:3300/0 conn(0x7f748c1338a0 0x7f748c133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:52.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.850+0000 7f74937d6700 1 -- 172.21.15.16:0/76312976 >> 172.21.15.16:0/76312976 conn(0x7f748c0a2aa0 msgr2=0x7f748c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:52.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.851+0000 7f74937d6700 1 -- 172.21.15.16:0/76312976 shutdown_connections 2023-09-30T22:32:52.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.851+0000 7f74937d6700 1 -- 172.21.15.16:0/76312976 wait complete. 2023-09-30T22:32:52.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.852+0000 7f74937d6700 1 Processor -- start 2023-09-30T22:32:52.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.852+0000 7f74937d6700 1 -- start start 2023-09-30T22:32:52.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.852+0000 7f74937d6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f748c09de50 0x7f748c11d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:52.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.852+0000 7f74937d6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f748c09e930 0x7f748c11dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:52.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.852+0000 7f74937d6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f748c1338a0 0x7f748c11e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:52.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.852+0000 7f74937d6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f748c121260 con 0x7f748c09e930 2023-09-30T22:32:52.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.852+0000 7f74937d6700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f748c11f930 con 0x7f748c09de50 2023-09-30T22:32:52.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.852+0000 7f74937d6700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f748c1209f0 con 0x7f748c1338a0 2023-09-30T22:32:52.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.852+0000 7f7490d71700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f748c09e930 0x7f748c11dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:52.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.852+0000 7f7490d71700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f748c09e930 0x7f748c11dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58822/0 (socket says 172.21.15.16:58822) 2023-09-30T22:32:52.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.852+0000 7f7490d71700 1 -- 172.21.15.16:0/562831028 learned_addr learned my addr 172.21.15.16:0/562831028 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:52.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.852+0000 7f7491d73700 1 --2- 172.21.15.16:0/562831028 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f748c1338a0 0x7f748c11e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:52.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.853+0000 7f7490d71700 1 -- 172.21.15.16:0/562831028 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f748c1338a0 msgr2=0x7f748c11e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:52.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.853+0000 7f7490d71700 1 --2- 172.21.15.16:0/562831028 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f748c1338a0 0x7f748c11e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:52.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.853+0000 7f7490d71700 1 -- 172.21.15.16:0/562831028 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f748c09de50 msgr2=0x7f748c11d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:52.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.853+0000 7f7490d71700 1 --2- 172.21.15.16:0/562831028 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f748c09de50 0x7f748c11d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:52.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.853+0000 7f7490d71700 1 -- 172.21.15.16:0/562831028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7484009b30 con 0x7f748c09e930 2023-09-30T22:32:52.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.853+0000 7f7491d73700 1 --2- 172.21.15.16:0/562831028 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f748c1338a0 0x7f748c11e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:52.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.853+0000 7f7490d71700 1 --2- 172.21.15.16:0/562831028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f748c09e930 0x7f748c11dcd0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f747c00c480 tx=0x7f747c00c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:52.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.853+0000 7f7481ffb700 1 -- 172.21.15.16:0/562831028 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747c0052d0 con 0x7f748c09e930 2023-09-30T22:32:52.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.853+0000 7f7481ffb700 1 -- 172.21.15.16:0/562831028 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f747c005430 con 0x7f748c09e930 2023-09-30T22:32:52.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.853+0000 7f7481ffb700 1 -- 172.21.15.16:0/562831028 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747c0038c0 con 0x7f748c09e930 2023-09-30T22:32:52.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.853+0000 7f74937d6700 1 -- 172.21.15.16:0/562831028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f748c06f640 con 0x7f748c09e930 2023-09-30T22:32:52.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.854+0000 7f74937d6700 1 -- 172.21.15.16:0/562831028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f748c11f350 con 0x7f748c09e930 2023-09-30T22:32:52.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.855+0000 7f7481ffb700 1 -- 172.21.15.16:0/562831028 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f747c003a20 con 0x7f748c09e930 2023-09-30T22:32:52.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.856+0000 7f7481ffb700 1 -- 172.21.15.16:0/562831028 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f747c0102e0 con 0x7f748c09e930 2023-09-30T22:32:52.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.856+0000 7f74937d6700 1 -- 172.21.15.16:0/562831028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f748c120de0 con 0x7f748c09e930 2023-09-30T22:32:52.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:52.865+0000 7f7481ffb700 1 -- 172.21.15.16:0/562831028 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f748c120de0 con 0x7f748c09e930 2023-09-30T22:32:53.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:53.003+0000 7fb618b83700 1 -- 172.21.15.16:0/1887883029 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb614074ea0 con 0x7fb61409d6f0 2023-09-30T22:32:53.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:53.004+0000 7fb602ffd700 1 -- 172.21.15.16:0/1887883029 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb604010380 con 0x7fb61409d6f0 2023-09-30T22:32:53.005 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:32:53.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:53.006+0000 7fb618b83700 1 -- 172.21.15.16:0/1887883029 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb61409d6f0 msgr2=0x7fb6141225b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:53.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:53.006+0000 7fb618b83700 1 --2- 172.21.15.16:0/1887883029 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb61409d6f0 0x7fb6141225b0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb604009870 tx=0x7fb604004750 comp rx=0 tx=0).stop 2023-09-30T22:32:53.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:53.006+0000 7fb618b83700 1 -- 172.21.15.16:0/1887883029 shutdown_connections 2023-09-30T22:32:53.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:53.006+0000 7fb618b83700 1 --2- 172.21.15.16:0/1887883029 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb61409e410 0x7fb614123030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:53.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:53.006+0000 7fb618b83700 1 --2- 172.21.15.16:0/1887883029 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb61409d6f0 0x7fb6141225b0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:53.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:53.006+0000 7fb618b83700 1 --2- 172.21.15.16:0/1887883029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb61409e000 0x7fb614122af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:53.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:53.006+0000 7fb618b83700 1 -- 172.21.15.16:0/1887883029 >> 172.21.15.16:0/1887883029 conn(0x7fb614066f50 msgr2=0x7fb6140a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:53.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:53.006+0000 7fb618b83700 1 -- 172.21.15.16:0/1887883029 shutdown_connections 2023-09-30T22:32:53.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:53.006+0000 7fb618b83700 1 -- 172.21.15.16:0/1887883029 wait complete. 2023-09-30T22:32:53.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:53.020+0000 7f977939b700 1 -- 172.21.15.16:0/2909185368 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9774122210 con 0x7f9774139170 2023-09-30T22:32:53.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:53.020+0000 7f975b7fe700 1 -- 172.21.15.16:0/2909185368 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9774122210 con 0x7f9774139170 2023-09-30T22:32:53.021 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:32:53.022 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:32:53.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:53.023+0000 7f977939b700 1 -- 172.21.15.16:0/2909185368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9774139170 msgr2=0x7f97741c89a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:53.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:53.023+0000 7f977939b700 1 --2- 172.21.15.16:0/2909185368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9774139170 0x7f97741c89a0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f9768015000 tx=0x7f9768004750 comp rx=0 tx=0).stop 2023-09-30T22:32:53.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:53.023+0000 7f977939b700 1 -- 172.21.15.16:0/2909185368 shutdown_connections 2023-09-30T22:32:53.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:53.023+0000 7f977939b700 1 --2- 172.21.15.16:0/2909185368 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f977409d6f0 0x7f97741c8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:53.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:53.023+0000 7f977939b700 1 --2- 172.21.15.16:0/2909185368 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f97741c8ee0 0x7f9774133500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:53.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:53.023+0000 7f977939b700 1 --2- 172.21.15.16:0/2909185368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9774139170 0x7f97741c89a0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:53.023+0000 7f977939b700 1 -- 172.21.15.16:0/2909185368 >> 172.21.15.16:0/2909185368 conn(0x7f9774066f50 msgr2=0x7f977406efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:53.023+0000 7f977939b700 1 -- 172.21.15.16:0/2909185368 shutdown_connections 2023-09-30T22:32:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:53.023+0000 7f977939b700 1 -- 172.21.15.16:0/2909185368 wait complete. 2023-09-30T22:32:53.037 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:32:53.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:53.060+0000 7fbd8feaa700 1 -- 172.21.15.16:0/3930698736 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fbd8806c360 con 0x7fbd8809d6f0 2023-09-30T22:32:53.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:53.060+0000 7fbd7e7fc700 1 -- 172.21.15.16:0/3930698736 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbd84017070 con 0x7fbd8809d6f0 2023-09-30T22:32:53.061 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:32:53.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:53.062+0000 7fbd8feaa700 1 -- 172.21.15.16:0/3930698736 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbd8809d6f0 msgr2=0x7fbd88139940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:53.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:53.062+0000 7fbd8feaa700 1 --2- 172.21.15.16:0/3930698736 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbd8809d6f0 0x7fbd88139940 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fbd8400e8a0 tx=0x7fbd8400ebb0 comp rx=0 tx=0).stop 2023-09-30T22:32:53.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:53.062+0000 7fbd8feaa700 1 -- 172.21.15.16:0/3930698736 shutdown_connections 2023-09-30T22:32:53.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:53.062+0000 7fbd8feaa700 1 --2- 172.21.15.16:0/3930698736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbd8809e410 0x7fbd8813a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:53.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:53.062+0000 7fbd8feaa700 1 --2- 172.21.15.16:0/3930698736 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbd8809d6f0 0x7fbd88139940 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:53.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:53.062+0000 7fbd8feaa700 1 --2- 172.21.15.16:0/3930698736 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbd8809e000 0x7fbd88139e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:53.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:53.062+0000 7fbd8feaa700 1 -- 172.21.15.16:0/3930698736 >> 172.21.15.16:0/3930698736 conn(0x7fbd88066f50 msgr2=0x7fbd8806a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:53.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:53.062+0000 7fbd8feaa700 1 -- 172.21.15.16:0/3930698736 shutdown_connections 2023-09-30T22:32:53.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:53.062+0000 7fbd8feaa700 1 -- 172.21.15.16:0/3930698736 wait complete. 2023-09-30T22:32:53.076 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:32:53.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:53.088+0000 7f74937d6700 1 -- 172.21.15.16:0/562831028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f748c120de0 con 0x7f748c09e930 2023-09-30T22:32:53.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:53.088+0000 7f7481ffb700 1 -- 172.21.15.16:0/562831028 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f748c120de0 con 0x7f748c09e930 2023-09-30T22:32:53.090 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:32:53.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:53.090+0000 7f74937d6700 1 -- 172.21.15.16:0/562831028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f748c09e930 msgr2=0x7f748c11dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:53.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:53.090+0000 7f74937d6700 1 --2- 172.21.15.16:0/562831028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f748c09e930 0x7f748c11dcd0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f747c00c480 tx=0x7f747c00c790 comp rx=0 tx=0).stop 2023-09-30T22:32:53.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:53.090+0000 7f74937d6700 1 -- 172.21.15.16:0/562831028 shutdown_connections 2023-09-30T22:32:53.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:53.090+0000 7f74937d6700 1 --2- 172.21.15.16:0/562831028 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f748c09de50 0x7f748c11d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:53.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:53.090+0000 7f74937d6700 1 --2- 172.21.15.16:0/562831028 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f748c1338a0 0x7f748c11e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:53.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:53.090+0000 7f74937d6700 1 --2- 172.21.15.16:0/562831028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f748c09e930 0x7f748c11dcd0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:53.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:53.090+0000 7f74937d6700 1 -- 172.21.15.16:0/562831028 >> 172.21.15.16:0/562831028 conn(0x7f748c0a2aa0 msgr2=0x7f748c1b9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:53.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:53.091+0000 7f74937d6700 1 -- 172.21.15.16:0/562831028 shutdown_connections 2023-09-30T22:32:53.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:53.091+0000 7f74937d6700 1 -- 172.21.15.16:0/562831028 wait complete. 2023-09-30T22:32:53.105 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:32:54.023 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:32:54.038 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:32:54.077 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:32:54.106 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:32:54.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.188+0000 7ffb44120700 1 Processor -- start 2023-09-30T22:32:54.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.189+0000 7ffb44120700 1 -- start start 2023-09-30T22:32:54.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.190+0000 7ffb44120700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffb3c1337f0 0x7ffb3c138880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:54.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.190+0000 7ffb44120700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ffb3c138dc0 0x7ffb3c13b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:54.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.190+0000 7ffb44120700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb3c06b990 con 0x7ffb3c133420 2023-09-30T22:32:54.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.190+0000 7ffb44120700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb3c09b0e0 con 0x7ffb3c138dc0 2023-09-30T22:32:54.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.190+0000 7ffb44120700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3c133c20 con 0x7ffb3c1337f0 2023-09-30T22:32:54.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.190+0000 7ffb426bd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ffb3c138dc0 0x7ffb3c13b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:54.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.190+0000 7ffb426bd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ffb3c138dc0 0x7ffb3c13b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58824/0 (socket says 172.21.15.16:58824) 2023-09-30T22:32:54.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.190+0000 7ffb41ebc700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ffb3c133420 0x7ffb3c133e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53804/0 (socket says 172.21.15.16:53804) 2023-09-30T22:32:54.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.190+0000 7ffb426bd700 1 -- 172.21.15.16:0/2074317746 learned_addr learned my addr 172.21.15.16:0/2074317746 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:54.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.190+0000 7ffb416bb700 1 --2- 172.21.15.16:0/2074317746 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffb3c1337f0 0x7ffb3c138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:54.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.190+0000 7ffb426bd700 1 -- 172.21.15.16:0/2074317746 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffb3c1337f0 msgr2=0x7ffb3c138880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:54.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.190+0000 7ffb426bd700 1 --2- 172.21.15.16:0/2074317746 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffb3c1337f0 0x7ffb3c138880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:54.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.190+0000 7ffb426bd700 1 -- 172.21.15.16:0/2074317746 >> v1:172.21.15.16:6789/0 conn(0x7ffb3c133420 legacy=0x7ffb3c133e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:54.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.190+0000 7ffb426bd700 1 -- 172.21.15.16:0/2074317746 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb3c0a0590 con 0x7ffb3c138dc0 2023-09-30T22:32:54.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.191+0000 7ffb426bd700 1 --2- 172.21.15.16:0/2074317746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb3c138dc0 0x7ffb3c13b1a0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7ffb34009e50 tx=0x7ffb3400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c5732098fd15cae server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:54.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.191+0000 7ffb40eba700 1 -- 172.21.15.16:0/2074317746 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3400e070 con 0x7ffb3c138dc0 2023-09-30T22:32:54.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.191+0000 7ffb40eba700 1 -- 172.21.15.16:0/2074317746 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb3400b910 con 0x7ffb3c138dc0 2023-09-30T22:32:54.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.191+0000 7ffb40eba700 1 -- 172.21.15.16:0/2074317746 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3400bcc0 con 0x7ffb3c138dc0 2023-09-30T22:32:54.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.192+0000 7ffb44120700 1 -- 172.21.15.16:0/2074317746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb3c138dc0 msgr2=0x7ffb3c13b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:54.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.192+0000 7ffb44120700 1 --2- 172.21.15.16:0/2074317746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb3c138dc0 0x7ffb3c13b1a0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7ffb34009e50 tx=0x7ffb3400b330 comp rx=0 tx=0).stop 2023-09-30T22:32:54.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.192+0000 7ffb44120700 1 -- 172.21.15.16:0/2074317746 shutdown_connections 2023-09-30T22:32:54.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.192+0000 7ffb44120700 1 --2- 172.21.15.16:0/2074317746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb3c138dc0 0x7ffb3c13b1a0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:54.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.192+0000 7ffb44120700 1 --2- 172.21.15.16:0/2074317746 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffb3c1337f0 0x7ffb3c138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:54.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.192+0000 7ffb44120700 1 -- 172.21.15.16:0/2074317746 >> 172.21.15.16:0/2074317746 conn(0x7ffb3c0a4210 msgr2=0x7ffb3c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:54.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.192+0000 7ffb44120700 1 -- 172.21.15.16:0/2074317746 shutdown_connections 2023-09-30T22:32:54.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.192+0000 7ffb44120700 1 -- 172.21.15.16:0/2074317746 wait complete. 2023-09-30T22:32:54.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.194+0000 7ffb44120700 1 Processor -- start 2023-09-30T22:32:54.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.194+0000 7ffb44120700 1 -- start start 2023-09-30T22:32:54.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.194+0000 7ffb44120700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffb3c133420 0x7ffb3c09c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:54.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.194+0000 7ffb44120700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffb3c1337f0 0x7ffb3c09c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:54.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.194+0000 7ffb44120700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb3c138dc0 0x7ffb3c09cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:54.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.194+0000 7ffb44120700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3c09d880 con 0x7ffb3c138dc0 2023-09-30T22:32:54.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.194+0000 7ffb44120700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3c12f1c0 con 0x7ffb3c1337f0 2023-09-30T22:32:54.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.194+0000 7ffb44120700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3c09e050 con 0x7ffb3c133420 2023-09-30T22:32:54.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.194+0000 7ffb426bd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb3c138dc0 0x7ffb3c09cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:54.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.194+0000 7ffb426bd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb3c138dc0 0x7ffb3c09cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58826/0 (socket says 172.21.15.16:58826) 2023-09-30T22:32:54.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.195+0000 7ffb426bd700 1 -- 172.21.15.16:0/691676715 learned_addr learned my addr 172.21.15.16:0/691676715 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:54.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.195+0000 7ffb416bb700 1 --2- 172.21.15.16:0/691676715 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffb3c1337f0 0x7ffb3c09c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:54.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.195+0000 7ffb426bd700 1 -- 172.21.15.16:0/691676715 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffb3c133420 msgr2=0x7ffb3c09c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:54.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.195+0000 7ffb426bd700 1 --2- 172.21.15.16:0/691676715 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffb3c133420 0x7ffb3c09c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:54.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.195+0000 7ffb426bd700 1 -- 172.21.15.16:0/691676715 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffb3c1337f0 msgr2=0x7ffb3c09c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:54.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.195+0000 7ffb426bd700 1 --2- 172.21.15.16:0/691676715 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffb3c1337f0 0x7ffb3c09c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:54.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.195+0000 7ffb426bd700 1 -- 172.21.15.16:0/691676715 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb34009b30 con 0x7ffb3c138dc0 2023-09-30T22:32:54.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.195+0000 7ffb426bd700 1 --2- 172.21.15.16:0/691676715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb3c138dc0 0x7ffb3c09cf30 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7ffb34012000 tx=0x7ffb34004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:54.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.195+0000 7ffb327fc700 1 -- 172.21.15.16:0/691676715 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb340048b0 con 0x7ffb3c138dc0 2023-09-30T22:32:54.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.195+0000 7ffb44120700 1 -- 172.21.15.16:0/691676715 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb3c130990 con 0x7ffb3c138dc0 2023-09-30T22:32:54.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.195+0000 7ffb327fc700 1 -- 172.21.15.16:0/691676715 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb340092e0 con 0x7ffb3c138dc0 2023-09-30T22:32:54.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.195+0000 7ffb327fc700 1 -- 172.21.15.16:0/691676715 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3401c680 con 0x7ffb3c138dc0 2023-09-30T22:32:54.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.196+0000 7ffb44120700 1 -- 172.21.15.16:0/691676715 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb3c12eeb0 con 0x7ffb3c138dc0 2023-09-30T22:32:54.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.197+0000 7ffb44120700 1 -- 172.21.15.16:0/691676715 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb3c074ea0 con 0x7ffb3c138dc0 2023-09-30T22:32:54.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.197+0000 7ffb327fc700 1 -- 172.21.15.16:0/691676715 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ffb3401c7e0 con 0x7ffb3c138dc0 2023-09-30T22:32:54.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.197+0000 7ffb327fc700 1 -- 172.21.15.16:0/691676715 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7ffb34040ec0 con 0x7ffb3c138dc0 2023-09-30T22:32:54.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.206+0000 7ffb327fc700 1 -- 172.21.15.16:0/691676715 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ffb340249a0 con 0x7ffb3c138dc0 2023-09-30T22:32:54.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.216+0000 7f20031ca700 1 Processor -- start 2023-09-30T22:32:54.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.216+0000 7f20031ca700 1 -- start start 2023-09-30T22:32:54.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.217+0000 7f20031ca700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1ffc06a770 0x7f1ffc09d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:54.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.217+0000 7f20031ca700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ffc09dc30 0x7f1ffc09e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:54.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.217+0000 7f20031ca700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ffc09a810 con 0x7f1ffc13c1e0 2023-09-30T22:32:54.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.217+0000 7f20031ca700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ffc06ab40 con 0x7f1ffc06a770 2023-09-30T22:32:54.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.217+0000 7f20031ca700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ffc09ebc0 con 0x7f1ffc09dc30 2023-09-30T22:32:54.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.217+0000 7f2001767700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1ffc13c1e0 0x7f1ffc13e5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53812/0 (socket says 172.21.15.16:53812) 2023-09-30T22:32:54.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.217+0000 7f2001767700 1 -- 172.21.15.16:0/3353308811 learned_addr learned my addr 172.21.15.16:0/3353308811 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:54.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.217+0000 7f2000f66700 1 --2- 172.21.15.16:0/3353308811 >> v2:172.21.15.16:3300/0 conn(0x7f1ffc06a770 0x7f1ffc09d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:54.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.217+0000 7f1ffbfff700 1 --2- 172.21.15.16:0/3353308811 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ffc09dc30 0x7f1ffc09e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:54.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.217+0000 7f1ffb7fe700 1 -- 172.21.15.16:0/3353308811 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2202461522 0 0) 0x7f1ffc09a810 con 0x7f1ffc13c1e0 2023-09-30T22:32:54.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.217+0000 7f1ffb7fe700 1 -- 172.21.15.16:0/3353308811 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fec0035b0 con 0x7f1ffc13c1e0 2023-09-30T22:32:54.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.217+0000 7f1ffbfff700 1 -- 172.21.15.16:0/3353308811 >> v2:172.21.15.16:3300/0 conn(0x7f1ffc06a770 msgr2=0x7f1ffc09d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:54.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.217+0000 7f1ffbfff700 1 --2- 172.21.15.16:0/3353308811 >> v2:172.21.15.16:3300/0 conn(0x7f1ffc06a770 0x7f1ffc09d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:54.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.218+0000 7f1ffbfff700 1 -- 172.21.15.16:0/3353308811 >> v1:172.21.15.16:6789/0 conn(0x7f1ffc13c1e0 legacy=0x7f1ffc13e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:54.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.218+0000 7f1ffbfff700 1 -- 172.21.15.16:0/3353308811 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ffc09e920 con 0x7f1ffc09dc30 2023-09-30T22:32:54.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.218+0000 7f2000f66700 1 --2- 172.21.15.16:0/3353308811 >> v2:172.21.15.16:3300/0 conn(0x7f1ffc06a770 0x7f1ffc09d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:54.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.218+0000 7f1ffbfff700 1 --2- 172.21.15.16:0/3353308811 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ffc09dc30 0x7f1ffc09e080 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8009840 tx=0x7f1fe8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aad3cb76c259302b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:54.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.218+0000 7f1ffb7fe700 1 -- 172.21.15.16:0/3353308811 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe8011830 con 0x7f1ffc09dc30 2023-09-30T22:32:54.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.218+0000 7f1ffb7fe700 1 -- 172.21.15.16:0/3353308811 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fe8011990 con 0x7f1ffc09dc30 2023-09-30T22:32:54.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.218+0000 7f1ffb7fe700 1 -- 172.21.15.16:0/3353308811 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe8011d20 con 0x7f1ffc09dc30 2023-09-30T22:32:54.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.219+0000 7f20031ca700 1 -- 172.21.15.16:0/3353308811 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ffc09dc30 msgr2=0x7f1ffc09e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:54.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.219+0000 7f20031ca700 1 --2- 172.21.15.16:0/3353308811 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ffc09dc30 0x7f1ffc09e080 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8009840 tx=0x7f1fe8009b50 comp rx=0 tx=0).stop 2023-09-30T22:32:54.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.219+0000 7f20031ca700 1 -- 172.21.15.16:0/3353308811 shutdown_connections 2023-09-30T22:32:54.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.219+0000 7f20031ca700 1 --2- 172.21.15.16:0/3353308811 >> v2:172.21.15.16:3300/0 conn(0x7f1ffc06a770 0x7f1ffc09d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:54.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.219+0000 7f20031ca700 1 --2- 172.21.15.16:0/3353308811 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ffc09dc30 0x7f1ffc09e080 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:54.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.219+0000 7f20031ca700 1 -- 172.21.15.16:0/3353308811 >> 172.21.15.16:0/3353308811 conn(0x7f1ffc066f50 msgr2=0x7f1ffc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:54.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.219+0000 7f20031ca700 1 -- 172.21.15.16:0/3353308811 shutdown_connections 2023-09-30T22:32:54.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.219+0000 7f20031ca700 1 -- 172.21.15.16:0/3353308811 wait complete. 2023-09-30T22:32:54.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.220+0000 7f20031ca700 1 Processor -- start 2023-09-30T22:32:54.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.220+0000 7f20031ca700 1 -- start start 2023-09-30T22:32:54.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.221+0000 7f20031ca700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ffc06a770 0x7f1ffc126ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:54.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.221+0000 7f20031ca700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ffc09dc30 0x7f1ffc127530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:54.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.221+0000 7f20031ca700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ffc13c1e0 0x7f1ffc127a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:54.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.221+0000 7f20031ca700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ffc13ba20 con 0x7f1ffc06a770 2023-09-30T22:32:54.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.221+0000 7f20031ca700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ffc128c20 con 0x7f1ffc09dc30 2023-09-30T22:32:54.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.221+0000 7f20031ca700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1ffc0a19b0 con 0x7f1ffc13c1e0 2023-09-30T22:32:54.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.221+0000 7f1ffbfff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ffc09dc30 0x7f1ffc127530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:54.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.221+0000 7f1ffbfff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ffc09dc30 0x7f1ffc127530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37364/0 (socket says 172.21.15.16:37364) 2023-09-30T22:32:54.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.221+0000 7f1ffbfff700 1 -- 172.21.15.16:0/2622118226 learned_addr learned my addr 172.21.15.16:0/2622118226 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:54.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.221+0000 7f2000f66700 1 --2- 172.21.15.16:0/2622118226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ffc06a770 0x7f1ffc126ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:54.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.221+0000 7f2001767700 1 --2- 172.21.15.16:0/2622118226 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ffc13c1e0 0x7f1ffc127a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:54.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.222+0000 7f1ffbfff700 1 -- 172.21.15.16:0/2622118226 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ffc13c1e0 msgr2=0x7f1ffc127a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:54.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.222+0000 7f1ffbfff700 1 --2- 172.21.15.16:0/2622118226 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ffc13c1e0 0x7f1ffc127a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:54.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.222+0000 7f1ffbfff700 1 -- 172.21.15.16:0/2622118226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ffc06a770 msgr2=0x7f1ffc126ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:54.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.222+0000 7f1ffbfff700 1 --2- 172.21.15.16:0/2622118226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ffc06a770 0x7f1ffc126ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:54.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.222+0000 7f1ffbfff700 1 -- 172.21.15.16:0/2622118226 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ff00094a0 con 0x7f1ffc09dc30 2023-09-30T22:32:54.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.222+0000 7f2001767700 1 --2- 172.21.15.16:0/2622118226 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ffc13c1e0 0x7f1ffc127a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:54.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.222+0000 7f1ffbfff700 1 --2- 172.21.15.16:0/2622118226 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ffc09dc30 0x7f1ffc127530 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1fe80090b0 tx=0x7f1fe8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:54.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.222+0000 7f1ff97fa700 1 -- 172.21.15.16:0/2622118226 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe8004590 con 0x7f1ffc09dc30 2023-09-30T22:32:54.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.222+0000 7f1ff97fa700 1 -- 172.21.15.16:0/2622118226 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fe801f070 con 0x7f1ffc09dc30 2023-09-30T22:32:54.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.222+0000 7f20031ca700 1 -- 172.21.15.16:0/2622118226 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fe80094a0 con 0x7f1ffc09dc30 2023-09-30T22:32:54.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.222+0000 7f1ff97fa700 1 -- 172.21.15.16:0/2622118226 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe80038e0 con 0x7f1ffc09dc30 2023-09-30T22:32:54.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.223+0000 7f20031ca700 1 -- 172.21.15.16:0/2622118226 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ffc13bee0 con 0x7f1ffc09dc30 2023-09-30T22:32:54.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.224+0000 7f20031ca700 1 -- 172.21.15.16:0/2622118226 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ffc12a6f0 con 0x7f1ffc09dc30 2023-09-30T22:32:54.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.224+0000 7f1ff97fa700 1 -- 172.21.15.16:0/2622118226 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1fe8003a40 con 0x7f1ffc09dc30 2023-09-30T22:32:54.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.225+0000 7f1ff97fa700 1 -- 172.21.15.16:0/2622118226 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f1fe8021070 con 0x7f1ffc09dc30 2023-09-30T22:32:54.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.229+0000 7f1ff97fa700 1 -- 172.21.15.16:0/2622118226 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1fe80259d0 con 0x7f1ffc09dc30 2023-09-30T22:32:54.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.271+0000 7f8a4ec74700 1 Processor -- start 2023-09-30T22:32:54.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.271+0000 7f8a4ec74700 1 -- start start 2023-09-30T22:32:54.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.271+0000 7f8a4ec74700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8a4809d6f0 0x7f8a4809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:54.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.271+0000 7f8a4ec74700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8a4809e410 0x7f8a48134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:54.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.271+0000 7f8a4ec74700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a4809b120 con 0x7f8a4809e000 2023-09-30T22:32:54.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.271+0000 7f8a4ec74700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a4806b460 con 0x7f8a4809e410 2023-09-30T22:32:54.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.271+0000 7f8a4ec74700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4809a7e0 con 0x7f8a4809d6f0 2023-09-30T22:32:54.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.271+0000 7f8a4d211700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8a4809e410 0x7f8a48134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:54.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.271+0000 7f8a4d211700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8a4809e410 0x7f8a48134100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58860/0 (socket says 172.21.15.16:58860) 2023-09-30T22:32:54.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.271+0000 7f8a4d211700 1 -- 172.21.15.16:0/1947763549 learned_addr learned my addr 172.21.15.16:0/1947763549 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:54.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.272+0000 7f8a4ca10700 1 --2- 172.21.15.16:0/1947763549 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8a4809d6f0 0x7f8a4809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:54.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.272+0000 7f8a4d211700 1 -- 172.21.15.16:0/1947763549 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8a4809d6f0 msgr2=0x7f8a4809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:54.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.272+0000 7f8a4d211700 1 --2- 172.21.15.16:0/1947763549 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8a4809d6f0 0x7f8a4809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:54.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.272+0000 7f8a4d211700 1 -- 172.21.15.16:0/1947763549 >> v1:172.21.15.16:6789/0 conn(0x7f8a4809e000 legacy=0x7f8a481339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:54.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.272+0000 7f8a4d211700 1 -- 172.21.15.16:0/1947763549 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a4809a500 con 0x7f8a4809e410 2023-09-30T22:32:54.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.272+0000 7f8a4ca10700 1 --2- 172.21.15.16:0/1947763549 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8a4809d6f0 0x7f8a4809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:54.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.272+0000 7f8a4d211700 1 --2- 172.21.15.16:0/1947763549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a4809e410 0x7f8a48134100 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f8a40009e50 tx=0x7f8a4000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41fd9277d5441cd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:54.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.272+0000 7f8a477fe700 1 -- 172.21.15.16:0/1947763549 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a4000e070 con 0x7f8a4809e410 2023-09-30T22:32:54.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.272+0000 7f8a477fe700 1 -- 172.21.15.16:0/1947763549 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a4000b910 con 0x7f8a4809e410 2023-09-30T22:32:54.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.272+0000 7f8a477fe700 1 -- 172.21.15.16:0/1947763549 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a4000bcc0 con 0x7f8a4809e410 2023-09-30T22:32:54.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.273+0000 7f8a4ec74700 1 -- 172.21.15.16:0/1947763549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a4809e410 msgr2=0x7f8a48134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:54.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.273+0000 7f8a4ec74700 1 --2- 172.21.15.16:0/1947763549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a4809e410 0x7f8a48134100 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f8a40009e50 tx=0x7f8a4000b330 comp rx=0 tx=0).stop 2023-09-30T22:32:54.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.273+0000 7f8a4ec74700 1 -- 172.21.15.16:0/1947763549 shutdown_connections 2023-09-30T22:32:54.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.273+0000 7f8a4ec74700 1 --2- 172.21.15.16:0/1947763549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a4809e410 0x7f8a48134100 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:54.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.273+0000 7f8a4ec74700 1 --2- 172.21.15.16:0/1947763549 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8a4809d6f0 0x7f8a4809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:54.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.273+0000 7f8a4ec74700 1 -- 172.21.15.16:0/1947763549 >> 172.21.15.16:0/1947763549 conn(0x7f8a48066f50 msgr2=0x7f8a4809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:54.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.273+0000 7f8a4ec74700 1 -- 172.21.15.16:0/1947763549 shutdown_connections 2023-09-30T22:32:54.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.273+0000 7f8a4ec74700 1 -- 172.21.15.16:0/1947763549 wait complete. 2023-09-30T22:32:54.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.275+0000 7f8a4ec74700 1 Processor -- start 2023-09-30T22:32:54.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.275+0000 7f8a4ec74700 1 -- start start 2023-09-30T22:32:54.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.276+0000 7f8a4ec74700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a4809d6f0 0x7f8a481a5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:54.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.276+0000 7f8a4ec74700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8a4809e000 0x7f8a481a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:54.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.276+0000 7f8a4ec74700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8a4809e410 0x7f8a481a6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:54.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.276+0000 7f8a4ec74700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a481a8770 con 0x7f8a4809d6f0 2023-09-30T22:32:54.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.276+0000 7f8a4ec74700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a481a7a10 con 0x7f8a4809e000 2023-09-30T22:32:54.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.276+0000 7f8a4ec74700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8a480a3040 con 0x7f8a4809e410 2023-09-30T22:32:54.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.276+0000 7f8a47fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8a4809e000 0x7f8a481a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:54.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.276+0000 7f8a47fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8a4809e000 0x7f8a481a6330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37384/0 (socket says 172.21.15.16:37384) 2023-09-30T22:32:54.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.276+0000 7f8a47fff700 1 -- 172.21.15.16:0/1170767746 learned_addr learned my addr 172.21.15.16:0/1170767746 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:54.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.276+0000 7f8a4ca10700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a4809d6f0 0x7f8a481a5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:54.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.276+0000 7f8a4d211700 1 --2- 172.21.15.16:0/1170767746 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8a4809e410 0x7f8a481a6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:54.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.276+0000 7f8a47fff700 1 -- 172.21.15.16:0/1170767746 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8a4809e410 msgr2=0x7f8a481a6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:54.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.276+0000 7f8a47fff700 1 --2- 172.21.15.16:0/1170767746 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8a4809e410 0x7f8a481a6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:54.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.276+0000 7f8a47fff700 1 -- 172.21.15.16:0/1170767746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a4809d6f0 msgr2=0x7f8a481a5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:54.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.276+0000 7f8a47fff700 1 --2- 172.21.15.16:0/1170767746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a4809d6f0 0x7f8a481a5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:54.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.276+0000 7f8a47fff700 1 -- 172.21.15.16:0/1170767746 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a3c0094a0 con 0x7f8a4809e000 2023-09-30T22:32:54.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.276+0000 7f8a4ca10700 1 --2- 172.21.15.16:0/1170767746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a4809d6f0 0x7f8a481a5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:54.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.277+0000 7f8a47fff700 1 --2- 172.21.15.16:0/1170767746 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8a4809e000 0x7f8a481a6330 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f8a34001ba0 tx=0x7f8a3400a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:54.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.277+0000 7f8a457fa700 1 -- 172.21.15.16:0/1170767746 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a34013cf0 con 0x7f8a4809e000 2023-09-30T22:32:54.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.277+0000 7f8a457fa700 1 -- 172.21.15.16:0/1170767746 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a34013e50 con 0x7f8a4809e000 2023-09-30T22:32:54.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.277+0000 7f8a4ec74700 1 -- 172.21.15.16:0/1170767746 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a40009b30 con 0x7f8a4809e000 2023-09-30T22:32:54.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.277+0000 7f8a457fa700 1 -- 172.21.15.16:0/1170767746 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a34005310 con 0x7f8a4809e000 2023-09-30T22:32:54.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.277+0000 7f8a4ec74700 1 -- 172.21.15.16:0/1170767746 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a481a8c30 con 0x7f8a4809e000 2023-09-30T22:32:54.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.278+0000 7f8a457fa700 1 -- 172.21.15.16:0/1170767746 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8a3400a730 con 0x7f8a4809e000 2023-09-30T22:32:54.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.278+0000 7f8a4ec74700 1 -- 172.21.15.16:0/1170767746 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a48074ea0 con 0x7f8a4809e000 2023-09-30T22:32:54.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.278+0000 7f8a457fa700 1 -- 172.21.15.16:0/1170767746 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f8a34047290 con 0x7f8a4809e000 2023-09-30T22:32:54.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.279+0000 7fe41f6a9700 1 Processor -- start 2023-09-30T22:32:54.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.280+0000 7fe41f6a9700 1 -- start start 2023-09-30T22:32:54.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.281+0000 7fe41f6a9700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe418136710 0x7fe418138af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:54.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.281+0000 7fe41f6a9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe418139030 0x7fe41813b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:54.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.281+0000 7fe41f6a9700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe41806ef30 con 0x7fe4181338a0 2023-09-30T22:32:54.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.281+0000 7fe41f6a9700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe41806fef0 con 0x7fe418136710 2023-09-30T22:32:54.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.281+0000 7fe41f6a9700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4181334b0 con 0x7fe418139030 2023-09-30T22:32:54.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.281+0000 7fe41dc46700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe418139030 0x7fe41813b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:54.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.281+0000 7fe41dc46700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe418139030 0x7fe41813b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37400/0 (socket says 172.21.15.16:37400) 2023-09-30T22:32:54.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.281+0000 7fe41dc46700 1 -- 172.21.15.16:0/3247182560 learned_addr learned my addr 172.21.15.16:0/3247182560 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:54.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.281+0000 7fe41dc46700 1 -- 172.21.15.16:0/3247182560 >> v2:172.21.15.16:3300/0 conn(0x7fe418136710 msgr2=0x7fe418138af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:54.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.281+0000 7fe41dc46700 1 --2- 172.21.15.16:0/3247182560 >> v2:172.21.15.16:3300/0 conn(0x7fe418136710 0x7fe418138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:54.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.281+0000 7fe41dc46700 1 -- 172.21.15.16:0/3247182560 >> v1:172.21.15.16:6789/0 conn(0x7fe4181338a0 legacy=0x7fe418133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:54.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.281+0000 7fe41dc46700 1 -- 172.21.15.16:0/3247182560 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe41806fc30 con 0x7fe418139030 2023-09-30T22:32:54.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.281+0000 7fe41dc46700 1 --2- 172.21.15.16:0/3247182560 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe418139030 0x7fe41813b410 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe410009e50 tx=0x7fe41000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=adb395a09b7d2142 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:54.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.283+0000 7fe40ffff700 1 -- 172.21.15.16:0/3247182560 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe41000e070 con 0x7fe418139030 2023-09-30T22:32:54.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.283+0000 7fe40ffff700 1 -- 172.21.15.16:0/3247182560 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe410003a20 con 0x7fe418139030 2023-09-30T22:32:54.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.283+0000 7fe40ffff700 1 -- 172.21.15.16:0/3247182560 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe410003dd0 con 0x7fe418139030 2023-09-30T22:32:54.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.283+0000 7f8a457fa700 1 -- 172.21.15.16:0/1170767746 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8a34005470 con 0x7f8a4809e000 2023-09-30T22:32:54.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.284+0000 7fe41f6a9700 1 -- 172.21.15.16:0/3247182560 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe418139030 msgr2=0x7fe41813b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:54.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.284+0000 7fe41f6a9700 1 --2- 172.21.15.16:0/3247182560 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe418139030 0x7fe41813b410 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe410009e50 tx=0x7fe41000b330 comp rx=0 tx=0).stop 2023-09-30T22:32:54.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.284+0000 7fe41f6a9700 1 -- 172.21.15.16:0/3247182560 shutdown_connections 2023-09-30T22:32:54.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.284+0000 7fe41f6a9700 1 --2- 172.21.15.16:0/3247182560 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe418139030 0x7fe41813b410 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:54.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.284+0000 7fe41f6a9700 1 --2- 172.21.15.16:0/3247182560 >> v2:172.21.15.16:3300/0 conn(0x7fe418136710 0x7fe418138af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:54.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.284+0000 7fe41f6a9700 1 -- 172.21.15.16:0/3247182560 >> 172.21.15.16:0/3247182560 conn(0x7fe4180a2aa0 msgr2=0x7fe4180a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:54.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.285+0000 7fe41f6a9700 1 -- 172.21.15.16:0/3247182560 shutdown_connections 2023-09-30T22:32:54.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.285+0000 7fe41f6a9700 1 -- 172.21.15.16:0/3247182560 wait complete. 2023-09-30T22:32:54.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.286+0000 7fe41f6a9700 1 Processor -- start 2023-09-30T22:32:54.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.286+0000 7fe41f6a9700 1 -- start start 2023-09-30T22:32:54.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.287+0000 7fe41f6a9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe4181338a0 0x7fe418118cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:54.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.287+0000 7fe41f6a9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe418136710 0x7fe418119210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:54.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.287+0000 7fe41f6a9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe418139030 0x7fe418119750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:54.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.287+0000 7fe41f6a9700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe41811b7c0 con 0x7fe418136710 2023-09-30T22:32:54.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.287+0000 7fe41f6a9700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe41811ad60 con 0x7fe4181338a0 2023-09-30T22:32:54.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.287+0000 7fe41f6a9700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe41813b420 con 0x7fe418139030 2023-09-30T22:32:54.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.287+0000 7fe41d445700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe4181338a0 0x7fe418118cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:54.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.287+0000 7fe41dc46700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe418139030 0x7fe418119750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:54.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.287+0000 7fe41d445700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe4181338a0 0x7fe418118cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37406/0 (socket says 172.21.15.16:37406) 2023-09-30T22:32:54.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.287+0000 7fe41d445700 1 -- 172.21.15.16:0/2974710811 learned_addr learned my addr 172.21.15.16:0/2974710811 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:54.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.287+0000 7fe41cc44700 1 --2- 172.21.15.16:0/2974710811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe418136710 0x7fe418119210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:54.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.287+0000 7fe41dc46700 1 -- 172.21.15.16:0/2974710811 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe4181338a0 msgr2=0x7fe418118cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:54.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.287+0000 7fe41dc46700 1 --2- 172.21.15.16:0/2974710811 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe4181338a0 0x7fe418118cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:54.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.287+0000 7fe41dc46700 1 -- 172.21.15.16:0/2974710811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe418136710 msgr2=0x7fe418119210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:54.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.287+0000 7fe41dc46700 1 --2- 172.21.15.16:0/2974710811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe418136710 0x7fe418119210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:54.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.287+0000 7fe41dc46700 1 -- 172.21.15.16:0/2974710811 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe410009b30 con 0x7fe418139030 2023-09-30T22:32:54.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.288+0000 7fe41dc46700 1 --2- 172.21.15.16:0/2974710811 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe418139030 0x7fe418119750 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fe410014000 tx=0x7fe410004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:54.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.288+0000 7fe40dffb700 1 -- 172.21.15.16:0/2974710811 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe410004440 con 0x7fe418139030 2023-09-30T22:32:54.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.288+0000 7fe40dffb700 1 -- 172.21.15.16:0/2974710811 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4100045e0 con 0x7fe418139030 2023-09-30T22:32:54.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.288+0000 7fe40dffb700 1 -- 172.21.15.16:0/2974710811 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4100092e0 con 0x7fe418139030 2023-09-30T22:32:54.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.288+0000 7fe41f6a9700 1 -- 172.21.15.16:0/2974710811 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe41806f640 con 0x7fe418139030 2023-09-30T22:32:54.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.288+0000 7fe41f6a9700 1 -- 172.21.15.16:0/2974710811 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe41811c4b0 con 0x7fe418139030 2023-09-30T22:32:54.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.289+0000 7fe41f6a9700 1 -- 172.21.15.16:0/2974710811 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe41806ea40 con 0x7fe418139030 2023-09-30T22:32:54.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.289+0000 7fe40dffb700 1 -- 172.21.15.16:0/2974710811 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe410012430 con 0x7fe418139030 2023-09-30T22:32:54.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.290+0000 7fe40dffb700 1 -- 172.21.15.16:0/2974710811 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fe410049420 con 0x7fe418139030 2023-09-30T22:32:54.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.295+0000 7fe40dffb700 1 -- 172.21.15.16:0/2974710811 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe410024d80 con 0x7fe418139030 2023-09-30T22:32:54.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.423+0000 7ffb44120700 1 -- 172.21.15.16:0/691676715 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ffb3c06b250 con 0x7ffb3c138dc0 2023-09-30T22:32:54.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.424+0000 7ffb327fc700 1 -- 172.21.15.16:0/691676715 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ffb340243b0 con 0x7ffb3c138dc0 2023-09-30T22:32:54.424 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:32:54.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.426+0000 7ffb44120700 1 -- 172.21.15.16:0/691676715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb3c138dc0 msgr2=0x7ffb3c09cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:54.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.426+0000 7ffb44120700 1 --2- 172.21.15.16:0/691676715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb3c138dc0 0x7ffb3c09cf30 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7ffb34012000 tx=0x7ffb34004ad0 comp rx=0 tx=0).stop 2023-09-30T22:32:54.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.426+0000 7ffb44120700 1 -- 172.21.15.16:0/691676715 shutdown_connections 2023-09-30T22:32:54.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.426+0000 7ffb44120700 1 --2- 172.21.15.16:0/691676715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb3c138dc0 0x7ffb3c09cf30 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:54.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.426+0000 7ffb44120700 1 --2- 172.21.15.16:0/691676715 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffb3c133420 0x7ffb3c09c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:54.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.426+0000 7ffb44120700 1 --2- 172.21.15.16:0/691676715 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffb3c1337f0 0x7ffb3c09c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:54.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.426+0000 7ffb44120700 1 -- 172.21.15.16:0/691676715 >> 172.21.15.16:0/691676715 conn(0x7ffb3c0a4210 msgr2=0x7ffb3c0a0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:54.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.426+0000 7ffb44120700 1 -- 172.21.15.16:0/691676715 shutdown_connections 2023-09-30T22:32:54.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.426+0000 7ffb44120700 1 -- 172.21.15.16:0/691676715 wait complete. 2023-09-30T22:32:54.440 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:32:54.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.461+0000 7f20031ca700 1 -- 172.21.15.16:0/2622118226 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1ffc12a6f0 con 0x7f1ffc09dc30 2023-09-30T22:32:54.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.462+0000 7f1ff97fa700 1 -- 172.21.15.16:0/2622118226 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1ffc12a6f0 con 0x7f1ffc09dc30 2023-09-30T22:32:54.462 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:32:54.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.464+0000 7f20031ca700 1 -- 172.21.15.16:0/2622118226 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ffc09dc30 msgr2=0x7f1ffc127530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:54.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.464+0000 7f20031ca700 1 --2- 172.21.15.16:0/2622118226 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ffc09dc30 0x7f1ffc127530 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f1fe80090b0 tx=0x7f1fe8004750 comp rx=0 tx=0).stop 2023-09-30T22:32:54.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.464+0000 7f20031ca700 1 -- 172.21.15.16:0/2622118226 shutdown_connections 2023-09-30T22:32:54.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.464+0000 7f20031ca700 1 --2- 172.21.15.16:0/2622118226 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ffc13c1e0 0x7f1ffc127a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:54.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.464+0000 7f20031ca700 1 --2- 172.21.15.16:0/2622118226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ffc06a770 0x7f1ffc126ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:54.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.464+0000 7f20031ca700 1 --2- 172.21.15.16:0/2622118226 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ffc09dc30 0x7f1ffc127530 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:54.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.464+0000 7f20031ca700 1 -- 172.21.15.16:0/2622118226 >> 172.21.15.16:0/2622118226 conn(0x7f1ffc066f50 msgr2=0x7f1ffc09b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:54.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.464+0000 7f20031ca700 1 -- 172.21.15.16:0/2622118226 shutdown_connections 2023-09-30T22:32:54.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.464+0000 7f20031ca700 1 -- 172.21.15.16:0/2622118226 wait complete. 2023-09-30T22:32:54.479 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:32:54.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.492+0000 7f8a4ec74700 1 -- 172.21.15.16:0/1170767746 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f8a4806c360 con 0x7f8a4809e000 2023-09-30T22:32:54.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.493+0000 7f8a457fa700 1 -- 172.21.15.16:0/1170767746 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8a3402b020 con 0x7f8a4809e000 2023-09-30T22:32:54.493 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:32:54.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.495+0000 7f8a4ec74700 1 -- 172.21.15.16:0/1170767746 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8a4809e000 msgr2=0x7f8a481a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:54.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.495+0000 7f8a4ec74700 1 --2- 172.21.15.16:0/1170767746 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8a4809e000 0x7f8a481a6330 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f8a34001ba0 tx=0x7f8a3400a060 comp rx=0 tx=0).stop 2023-09-30T22:32:54.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.495+0000 7f8a4ec74700 1 -- 172.21.15.16:0/1170767746 shutdown_connections 2023-09-30T22:32:54.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.495+0000 7f8a4ec74700 1 --2- 172.21.15.16:0/1170767746 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8a4809e410 0x7f8a481a6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:54.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.495+0000 7f8a4ec74700 1 --2- 172.21.15.16:0/1170767746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8a4809d6f0 0x7f8a481a5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:54.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.495+0000 7f8a4ec74700 1 --2- 172.21.15.16:0/1170767746 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8a4809e000 0x7f8a481a6330 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:54.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.495+0000 7f8a4ec74700 1 -- 172.21.15.16:0/1170767746 >> 172.21.15.16:0/1170767746 conn(0x7f8a48066f50 msgr2=0x7f8a481bae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:54.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.495+0000 7f8a4ec74700 1 -- 172.21.15.16:0/1170767746 shutdown_connections 2023-09-30T22:32:54.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.495+0000 7f8a4ec74700 1 -- 172.21.15.16:0/1170767746 wait complete. 2023-09-30T22:32:54.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.504+0000 7fe41f6a9700 1 -- 172.21.15.16:0/2974710811 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe418074ea0 con 0x7fe418139030 2023-09-30T22:32:54.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.504+0000 7fe40dffb700 1 -- 172.21.15.16:0/2974710811 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe410017030 con 0x7fe418139030 2023-09-30T22:32:54.505 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:32:54.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.507+0000 7fe41f6a9700 1 -- 172.21.15.16:0/2974710811 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe418139030 msgr2=0x7fe418119750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:54.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.507+0000 7fe41f6a9700 1 --2- 172.21.15.16:0/2974710811 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe418139030 0x7fe418119750 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fe410014000 tx=0x7fe410004ad0 comp rx=0 tx=0).stop 2023-09-30T22:32:54.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.507+0000 7fe41f6a9700 1 -- 172.21.15.16:0/2974710811 shutdown_connections 2023-09-30T22:32:54.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.507+0000 7fe41f6a9700 1 --2- 172.21.15.16:0/2974710811 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe4181338a0 0x7fe418118cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:54.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.507+0000 7fe41f6a9700 1 --2- 172.21.15.16:0/2974710811 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe418139030 0x7fe418119750 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:54.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.507+0000 7fe41f6a9700 1 --2- 172.21.15.16:0/2974710811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe418136710 0x7fe418119210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:54.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.507+0000 7fe41f6a9700 1 -- 172.21.15.16:0/2974710811 >> 172.21.15.16:0/2974710811 conn(0x7fe4180a2aa0 msgr2=0x7fe4181344f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:54.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.507+0000 7fe41f6a9700 1 -- 172.21.15.16:0/2974710811 shutdown_connections 2023-09-30T22:32:54.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:54.507+0000 7fe41f6a9700 1 -- 172.21.15.16:0/2974710811 wait complete. 2023-09-30T22:32:54.509 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:32:54.521 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:32:55.441 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:32:55.479 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:32:55.511 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:32:55.521 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:32:55.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.594+0000 7ff618442700 1 Processor -- start 2023-09-30T22:32:55.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.594+0000 7ff618442700 1 -- start start 2023-09-30T22:32:55.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.595+0000 7ff618442700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff61009d6f0 0x7ff61009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:55.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.595+0000 7ff618442700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff61009e410 0x7ff610130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:55.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.595+0000 7ff618442700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff61009f130 con 0x7ff61009e000 2023-09-30T22:32:55.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.595+0000 7ff618442700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff61009f520 con 0x7ff61009d6f0 2023-09-30T22:32:55.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.595+0000 7ff618442700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff61009e930 con 0x7ff61009e410 2023-09-30T22:32:55.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.595+0000 7ff6169df700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff61009e410 0x7ff610130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:55.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.595+0000 7ff6169df700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff61009e410 0x7ff610130d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37422/0 (socket says 172.21.15.16:37422) 2023-09-30T22:32:55.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.595+0000 7ff6169df700 1 -- 172.21.15.16:0/1256661785 learned_addr learned my addr 172.21.15.16:0/1256661785 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:55.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.596+0000 7ff6161de700 1 --2- 172.21.15.16:0/1256661785 >> v2:172.21.15.16:3300/0 conn(0x7ff61009d6f0 0x7ff61009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:55.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.596+0000 7ff6169df700 1 -- 172.21.15.16:0/1256661785 >> v2:172.21.15.16:3300/0 conn(0x7ff61009d6f0 msgr2=0x7ff61009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:55.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.596+0000 7ff6169df700 1 --2- 172.21.15.16:0/1256661785 >> v2:172.21.15.16:3300/0 conn(0x7ff61009d6f0 0x7ff61009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:55.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.596+0000 7ff6169df700 1 -- 172.21.15.16:0/1256661785 >> v1:172.21.15.16:6789/0 conn(0x7ff61009e000 legacy=0x7ff610130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:55.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.596+0000 7ff6169df700 1 -- 172.21.15.16:0/1256661785 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff61009b2d0 con 0x7ff61009e410 2023-09-30T22:32:55.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.596+0000 7ff6169df700 1 --2- 172.21.15.16:0/1256661785 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff61009e410 0x7ff610130d70 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ff60800d150 tx=0x7ff60800d460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8233ec51b4f25528 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:55.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.596+0000 7ff6151dc700 1 -- 172.21.15.16:0/1256661785 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff608010070 con 0x7ff61009e410 2023-09-30T22:32:55.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.597+0000 7ff6151dc700 1 -- 172.21.15.16:0/1256661785 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff60800dd10 con 0x7ff61009e410 2023-09-30T22:32:55.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.597+0000 7ff6151dc700 1 -- 172.21.15.16:0/1256661785 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60800b310 con 0x7ff61009e410 2023-09-30T22:32:55.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.597+0000 7ff618442700 1 -- 172.21.15.16:0/1256661785 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff61009e410 msgr2=0x7ff610130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:55.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.597+0000 7ff618442700 1 --2- 172.21.15.16:0/1256661785 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff61009e410 0x7ff610130d70 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ff60800d150 tx=0x7ff60800d460 comp rx=0 tx=0).stop 2023-09-30T22:32:55.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.597+0000 7ff618442700 1 -- 172.21.15.16:0/1256661785 shutdown_connections 2023-09-30T22:32:55.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.597+0000 7ff618442700 1 --2- 172.21.15.16:0/1256661785 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff61009e410 0x7ff610130d70 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:55.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.597+0000 7ff618442700 1 --2- 172.21.15.16:0/1256661785 >> v2:172.21.15.16:3300/0 conn(0x7ff61009d6f0 0x7ff61009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:55.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.597+0000 7ff618442700 1 -- 172.21.15.16:0/1256661785 >> 172.21.15.16:0/1256661785 conn(0x7ff610066f50 msgr2=0x7ff61009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:55.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.597+0000 7ff618442700 1 -- 172.21.15.16:0/1256661785 shutdown_connections 2023-09-30T22:32:55.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.598+0000 7ff618442700 1 -- 172.21.15.16:0/1256661785 wait complete. 2023-09-30T22:32:55.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.598+0000 7ff618442700 1 Processor -- start 2023-09-30T22:32:55.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.599+0000 7ff618442700 1 -- start start 2023-09-30T22:32:55.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.599+0000 7ff618442700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff61009d6f0 0x7ff6101222b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:55.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.599+0000 7ff618442700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff61009e000 0x7ff6101227f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:55.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.599+0000 7ff618442700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff61009e410 0x7ff610122d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:55.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.599+0000 7ff618442700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff610124090 con 0x7ff61009e000 2023-09-30T22:32:55.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.599+0000 7ff618442700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff610123ea0 con 0x7ff61009d6f0 2023-09-30T22:32:55.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.599+0000 7ff618442700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff610124770 con 0x7ff61009e410 2023-09-30T22:32:55.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.599+0000 7ff6159dd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff61009e000 0x7ff6101227f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:55.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.599+0000 7ff6169df700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff61009e410 0x7ff610122d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:55.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.599+0000 7ff6159dd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff61009e000 0x7ff6101227f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58910/0 (socket says 172.21.15.16:58910) 2023-09-30T22:32:55.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.599+0000 7ff6159dd700 1 -- 172.21.15.16:0/787344950 learned_addr learned my addr 172.21.15.16:0/787344950 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:55.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.600+0000 7ff6161de700 1 --2- 172.21.15.16:0/787344950 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff61009d6f0 0x7ff6101222b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:55.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.600+0000 7ff6159dd700 1 -- 172.21.15.16:0/787344950 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff61009e410 msgr2=0x7ff610122d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:55.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.600+0000 7ff6159dd700 1 --2- 172.21.15.16:0/787344950 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff61009e410 0x7ff610122d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:55.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.600+0000 7ff6159dd700 1 -- 172.21.15.16:0/787344950 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff61009d6f0 msgr2=0x7ff6101222b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:55.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.600+0000 7ff6159dd700 1 --2- 172.21.15.16:0/787344950 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff61009d6f0 0x7ff6101222b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:55.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.600+0000 7ff6159dd700 1 -- 172.21.15.16:0/787344950 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff60800bd40 con 0x7ff61009e000 2023-09-30T22:32:55.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.600+0000 7ff6159dd700 1 --2- 172.21.15.16:0/787344950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff61009e000 0x7ff6101227f0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7ff60000c510 tx=0x7ff60000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:55.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.600+0000 7ff606ffd700 1 -- 172.21.15.16:0/787344950 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff600015d70 con 0x7ff61009e000 2023-09-30T22:32:55.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.600+0000 7ff606ffd700 1 -- 172.21.15.16:0/787344950 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff60000ce80 con 0x7ff61009e000 2023-09-30T22:32:55.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.600+0000 7ff618442700 1 -- 172.21.15.16:0/787344950 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff610125db0 con 0x7ff61009e000 2023-09-30T22:32:55.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.600+0000 7ff606ffd700 1 -- 172.21.15.16:0/787344950 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff600005310 con 0x7ff61009e000 2023-09-30T22:32:55.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.600+0000 7ff618442700 1 -- 172.21.15.16:0/787344950 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff610125ad0 con 0x7ff61009e000 2023-09-30T22:32:55.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.601+0000 7ff606ffd700 1 -- 172.21.15.16:0/787344950 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff600003680 con 0x7ff61009e000 2023-09-30T22:32:55.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.602+0000 7ff606ffd700 1 -- 172.21.15.16:0/787344950 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7ff600047070 con 0x7ff61009e000 2023-09-30T22:32:55.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.602+0000 7ff618442700 1 -- 172.21.15.16:0/787344950 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff610074ea0 con 0x7ff61009e000 2023-09-30T22:32:55.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.611+0000 7ff606ffd700 1 -- 172.21.15.16:0/787344950 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff600022020 con 0x7ff61009e000 2023-09-30T22:32:55.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.644+0000 7fa71a1b7700 1 Processor -- start 2023-09-30T22:32:55.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.644+0000 7fa71a1b7700 1 -- start start 2023-09-30T22:32:55.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.645+0000 7fa71a1b7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa71409d6f0 0x7fa71409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:55.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.645+0000 7fa71a1b7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa71409e410 0x7fa714134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:55.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.645+0000 7fa71a1b7700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa71409b120 con 0x7fa71409e000 2023-09-30T22:32:55.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.645+0000 7fa71a1b7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa71406b460 con 0x7fa71409d6f0 2023-09-30T22:32:55.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.645+0000 7fa71a1b7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa71409a7e0 con 0x7fa71409e410 2023-09-30T22:32:55.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.645+0000 7fa70bfff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa71409e000 0x7fa7141339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53846/0 (socket says 172.21.15.16:53846) 2023-09-30T22:32:55.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.645+0000 7fa70bfff700 1 -- 172.21.15.16:0/2932837222 learned_addr learned my addr 172.21.15.16:0/2932837222 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:55.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.645+0000 7fa7137fe700 1 --2- 172.21.15.16:0/2932837222 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa71409d6f0 0x7fa71409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:55.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.645+0000 7fa713fff700 1 --2- 172.21.15.16:0/2932837222 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa71409e410 0x7fa714134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:55.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.646+0000 7fa7137fe700 1 -- 172.21.15.16:0/2932837222 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa71409e410 msgr2=0x7fa714134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:55.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.646+0000 7fa7137fe700 1 --2- 172.21.15.16:0/2932837222 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa71409e410 0x7fa714134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:55.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.646+0000 7fa7137fe700 1 -- 172.21.15.16:0/2932837222 >> v1:172.21.15.16:6789/0 conn(0x7fa71409e000 legacy=0x7fa7141339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:55.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.646+0000 7fa712ffd700 1 -- 172.21.15.16:0/2932837222 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 872460548 0 0) 0x7fa71409b120 con 0x7fa71409e000 2023-09-30T22:32:55.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.646+0000 7fa7137fe700 1 -- 172.21.15.16:0/2932837222 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa71409a500 con 0x7fa71409d6f0 2023-09-30T22:32:55.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.646+0000 7fa713fff700 1 --2- 172.21.15.16:0/2932837222 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa71409e410 0x7fa714134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:55.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.646+0000 7fa7137fe700 1 --2- 172.21.15.16:0/2932837222 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa71409d6f0 0x7fa71409dac0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa704009870 tx=0x7fa704009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2bd3e6199a004020 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:55.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.646+0000 7fa712ffd700 1 -- 172.21.15.16:0/2932837222 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa704011830 con 0x7fa71409d6f0 2023-09-30T22:32:55.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.646+0000 7fa712ffd700 1 -- 172.21.15.16:0/2932837222 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa704011990 con 0x7fa71409d6f0 2023-09-30T22:32:55.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.646+0000 7fa712ffd700 1 -- 172.21.15.16:0/2932837222 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa704011d20 con 0x7fa71409d6f0 2023-09-30T22:32:55.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.647+0000 7fa71a1b7700 1 -- 172.21.15.16:0/2932837222 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa71409d6f0 msgr2=0x7fa71409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:55.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.647+0000 7fa71a1b7700 1 --2- 172.21.15.16:0/2932837222 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa71409d6f0 0x7fa71409dac0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa704009870 tx=0x7fa704009b80 comp rx=0 tx=0).stop 2023-09-30T22:32:55.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.647+0000 7fa71a1b7700 1 -- 172.21.15.16:0/2932837222 shutdown_connections 2023-09-30T22:32:55.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.647+0000 7fa71a1b7700 1 --2- 172.21.15.16:0/2932837222 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa71409e410 0x7fa714134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:55.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.647+0000 7fa71a1b7700 1 --2- 172.21.15.16:0/2932837222 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa71409d6f0 0x7fa71409dac0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:55.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.647+0000 7fa71a1b7700 1 -- 172.21.15.16:0/2932837222 >> 172.21.15.16:0/2932837222 conn(0x7fa714066f50 msgr2=0x7fa71409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:55.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.647+0000 7fa71a1b7700 1 -- 172.21.15.16:0/2932837222 shutdown_connections 2023-09-30T22:32:55.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.647+0000 7fa71a1b7700 1 -- 172.21.15.16:0/2932837222 wait complete. 2023-09-30T22:32:55.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.648+0000 7fa71a1b7700 1 Processor -- start 2023-09-30T22:32:55.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.648+0000 7fa71a1b7700 1 -- start start 2023-09-30T22:32:55.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.649+0000 7fa71a1b7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa71409d6f0 0x7fa7141223b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:55.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.649+0000 7fa71a1b7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa71409e000 0x7fa7141228f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:55.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.649+0000 7fa71a1b7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa71409e410 0x7fa714122e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:55.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.649+0000 7fa71a1b7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7141336f0 con 0x7fa71409e410 2023-09-30T22:32:55.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.649+0000 7fa71a1b7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa714123810 con 0x7fa71409e000 2023-09-30T22:32:55.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.649+0000 7fa71a1b7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa714124dc0 con 0x7fa71409d6f0 2023-09-30T22:32:55.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.649+0000 7fa7137fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa71409d6f0 0x7fa7141223b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:55.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.649+0000 7fa7137fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa71409d6f0 0x7fa7141223b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:47506/0 (socket says 172.21.15.16:47506) 2023-09-30T22:32:55.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.649+0000 7fa7137fe700 1 -- 172.21.15.16:0/1590891019 learned_addr learned my addr 172.21.15.16:0/1590891019 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:55.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.649+0000 7fa713fff700 1 --2- 172.21.15.16:0/1590891019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa71409e410 0x7fa714122e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:55.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.649+0000 7fa70bfff700 1 --2- 172.21.15.16:0/1590891019 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa71409e000 0x7fa7141228f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:55.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.650+0000 7fa7137fe700 1 -- 172.21.15.16:0/1590891019 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa71409e000 msgr2=0x7fa7141228f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:55.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.650+0000 7fa7137fe700 1 --2- 172.21.15.16:0/1590891019 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa71409e000 0x7fa7141228f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:55.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.650+0000 7fa7137fe700 1 -- 172.21.15.16:0/1590891019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa71409e410 msgr2=0x7fa714122e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:55.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.650+0000 7fa7137fe700 1 --2- 172.21.15.16:0/1590891019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa71409e410 0x7fa714122e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:55.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.650+0000 7fa7137fe700 1 -- 172.21.15.16:0/1590891019 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa70c009a70 con 0x7fa71409d6f0 2023-09-30T22:32:55.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.650+0000 7fa7137fe700 1 --2- 172.21.15.16:0/1590891019 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa71409d6f0 0x7fa7141223b0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa7040090b0 tx=0x7fa704004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:55.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.650+0000 7fa710ff9700 1 -- 172.21.15.16:0/1590891019 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa704024620 con 0x7fa71409d6f0 2023-09-30T22:32:55.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.650+0000 7fa710ff9700 1 -- 172.21.15.16:0/1590891019 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa704024780 con 0x7fa71409d6f0 2023-09-30T22:32:55.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.650+0000 7fa71a1b7700 1 -- 172.21.15.16:0/1590891019 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7040094a0 con 0x7fa71409d6f0 2023-09-30T22:32:55.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.650+0000 7fa710ff9700 1 -- 172.21.15.16:0/1590891019 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa704024af0 con 0x7fa71409d6f0 2023-09-30T22:32:55.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.650+0000 7fa71a1b7700 1 -- 172.21.15.16:0/1590891019 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7140a1aa0 con 0x7fa71409d6f0 2023-09-30T22:32:55.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.652+0000 7fa710ff9700 1 -- 172.21.15.16:0/1590891019 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa704022070 con 0x7fa71409d6f0 2023-09-30T22:32:55.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.652+0000 7fa71a1b7700 1 -- 172.21.15.16:0/1590891019 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa714125180 con 0x7fa71409d6f0 2023-09-30T22:32:55.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.652+0000 7fa710ff9700 1 -- 172.21.15.16:0/1590891019 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fa704040690 con 0x7fa71409d6f0 2023-09-30T22:32:55.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.661+0000 7fa710ff9700 1 -- 172.21.15.16:0/1590891019 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa704024c50 con 0x7fa71409d6f0 2023-09-30T22:32:55.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.684+0000 7fa98cfe3700 1 Processor -- start 2023-09-30T22:32:55.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.684+0000 7fa98cfe3700 1 -- start start 2023-09-30T22:32:55.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.685+0000 7fa98cfe3700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa988133420 0x7fa9881337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:55.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.685+0000 7fa98cfe3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa988133e30 0x7fa98812ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:55.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.685+0000 7fa98cfe3700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa98806b990 con 0x7fa988134260 2023-09-30T22:32:55.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.685+0000 7fa98cfe3700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9880a0590 con 0x7fa988133420 2023-09-30T22:32:55.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.685+0000 7fa98cfe3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa98809fb80 con 0x7fa988133e30 2023-09-30T22:32:55.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.685+0000 7fa98659c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa988133420 0x7fa9881337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:55.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.685+0000 7fa986d9d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa988134260 0x7fa9881304d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53856/0 (socket says 172.21.15.16:53856) 2023-09-30T22:32:55.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.685+0000 7fa986d9d700 1 -- 172.21.15.16:0/2870942098 learned_addr learned my addr 172.21.15.16:0/2870942098 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:55.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.685+0000 7fa98659c700 1 -- 172.21.15.16:0/2870942098 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa988133e30 msgr2=0x7fa98812ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:55.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.685+0000 7fa98659c700 1 --2- 172.21.15.16:0/2870942098 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa988133e30 0x7fa98812ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:55.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.685+0000 7fa98659c700 1 -- 172.21.15.16:0/2870942098 >> v1:172.21.15.16:6789/0 conn(0x7fa988134260 legacy=0x7fa9881304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:55.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.685+0000 7fa98559a700 1 -- 172.21.15.16:0/2870942098 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3349261721 0 0) 0x7fa98806b990 con 0x7fa988134260 2023-09-30T22:32:55.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.685+0000 7fa98659c700 1 -- 172.21.15.16:0/2870942098 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9880a0030 con 0x7fa988133420 2023-09-30T22:32:55.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.685+0000 7fa98659c700 1 --2- 172.21.15.16:0/2870942098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa988133420 0x7fa9881337f0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fa978009840 tx=0x7fa978009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c77469f639eb984 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:55.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.686+0000 7fa98559a700 1 -- 172.21.15.16:0/2870942098 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa978011830 con 0x7fa988133420 2023-09-30T22:32:55.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.686+0000 7fa98559a700 1 -- 172.21.15.16:0/2870942098 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa978011990 con 0x7fa988133420 2023-09-30T22:32:55.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.686+0000 7fa98559a700 1 -- 172.21.15.16:0/2870942098 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa978011d20 con 0x7fa988133420 2023-09-30T22:32:55.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.686+0000 7fa98cfe3700 1 -- 172.21.15.16:0/2870942098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa988133420 msgr2=0x7fa9881337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:55.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.686+0000 7fa98cfe3700 1 --2- 172.21.15.16:0/2870942098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa988133420 0x7fa9881337f0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fa978009840 tx=0x7fa978009b50 comp rx=0 tx=0).stop 2023-09-30T22:32:55.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.687+0000 7fa98cfe3700 1 -- 172.21.15.16:0/2870942098 shutdown_connections 2023-09-30T22:32:55.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.687+0000 7fa98cfe3700 1 --2- 172.21.15.16:0/2870942098 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa988133420 0x7fa9881337f0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:55.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.687+0000 7fa98cfe3700 1 --2- 172.21.15.16:0/2870942098 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa988133e30 0x7fa98812ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:55.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.687+0000 7fa98cfe3700 1 -- 172.21.15.16:0/2870942098 >> 172.21.15.16:0/2870942098 conn(0x7fa9880a4210 msgr2=0x7fa9880a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:55.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.687+0000 7fa98cfe3700 1 -- 172.21.15.16:0/2870942098 shutdown_connections 2023-09-30T22:32:55.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.687+0000 7fa98cfe3700 1 -- 172.21.15.16:0/2870942098 wait complete. 2023-09-30T22:32:55.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.688+0000 7fa98cfe3700 1 Processor -- start 2023-09-30T22:32:55.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.688+0000 7fa98cfe3700 1 -- start start 2023-09-30T22:32:55.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.688+0000 7fa98cfe3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa988133420 0x7fa98809c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:55.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.688+0000 7fa98cfe3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa988133e30 0x7fa98809c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:55.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.688+0000 7fa98cfe3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa988134260 0x7fa98809cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:55.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.688+0000 7fa98cfe3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa98811feb0 con 0x7fa988134260 2023-09-30T22:32:55.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.688+0000 7fa98cfe3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa98812d9b0 con 0x7fa988133420 2023-09-30T22:32:55.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.688+0000 7fa98cfe3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa988120100 con 0x7fa988133e30 2023-09-30T22:32:55.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.689+0000 7fa985d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa988133e30 0x7fa98809c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:55.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.689+0000 7fa986d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa988134260 0x7fa98809cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:55.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.689+0000 7fa985d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa988133e30 0x7fa98809c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:47510/0 (socket says 172.21.15.16:47510) 2023-09-30T22:32:55.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.689+0000 7fa985d9b700 1 -- 172.21.15.16:0/2134780895 learned_addr learned my addr 172.21.15.16:0/2134780895 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:55.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.689+0000 7fa986d9d700 1 -- 172.21.15.16:0/2134780895 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa988133e30 msgr2=0x7fa98809c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:55.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.689+0000 7fa986d9d700 1 --2- 172.21.15.16:0/2134780895 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa988133e30 0x7fa98809c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:55.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.689+0000 7fa986d9d700 1 -- 172.21.15.16:0/2134780895 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa988133420 msgr2=0x7fa98809c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:55.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.689+0000 7fa986d9d700 1 --2- 172.21.15.16:0/2134780895 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa988133420 0x7fa98809c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:55.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.689+0000 7fa986d9d700 1 -- 172.21.15.16:0/2134780895 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9780094a0 con 0x7fa988134260 2023-09-30T22:32:55.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.689+0000 7fa986d9d700 1 --2- 172.21.15.16:0/2134780895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa988134260 0x7fa98809cf00 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fa98000cbe0 tx=0x7fa98000cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:55.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.689+0000 7fa976ffd700 1 -- 172.21.15.16:0/2134780895 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98000aab0 con 0x7fa988134260 2023-09-30T22:32:55.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.689+0000 7fa976ffd700 1 -- 172.21.15.16:0/2134780895 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa98000ac30 con 0x7fa988134260 2023-09-30T22:32:55.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.689+0000 7fa976ffd700 1 -- 172.21.15.16:0/2134780895 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9800070d0 con 0x7fa988134260 2023-09-30T22:32:55.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.689+0000 7fa98cfe3700 1 -- 172.21.15.16:0/2134780895 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa98809dc90 con 0x7fa988134260 2023-09-30T22:32:55.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.689+0000 7fa98cfe3700 1 -- 172.21.15.16:0/2134780895 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa98811fb60 con 0x7fa988134260 2023-09-30T22:32:55.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.690+0000 7fa98cfe3700 1 -- 172.21.15.16:0/2134780895 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa988074ea0 con 0x7fa988134260 2023-09-30T22:32:55.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.691+0000 7fa976ffd700 1 -- 172.21.15.16:0/2134780895 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa980007230 con 0x7fa988134260 2023-09-30T22:32:55.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.691+0000 7fe36737b700 1 Processor -- start 2023-09-30T22:32:55.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.691+0000 7fa976ffd700 1 -- 172.21.15.16:0/2134780895 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fa98003f4e0 con 0x7fa988134260 2023-09-30T22:32:55.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.691+0000 7fe36737b700 1 -- start start 2023-09-30T22:32:55.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.692+0000 7fe36737b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe36009eca0 0x7fe36013cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:55.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.692+0000 7fe36737b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe36006f050 0x7fe36013f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:55.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.692+0000 7fe36737b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe360070ee0 con 0x7fe36006e570 2023-09-30T22:32:55.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.692+0000 7fe36737b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe360071980 con 0x7fe36006f050 2023-09-30T22:32:55.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.692+0000 7fe36737b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe360071480 con 0x7fe36009eca0 2023-09-30T22:32:55.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.692+0000 7fe365117700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe36006e570 0x7fe36006e940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53870/0 (socket says 172.21.15.16:53870) 2023-09-30T22:32:55.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.692+0000 7fe364916700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe36009eca0 0x7fe36013cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:55.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.692+0000 7fe365117700 1 -- 172.21.15.16:0/3977076393 learned_addr learned my addr 172.21.15.16:0/3977076393 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:55.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.692+0000 7fe365918700 1 --2- 172.21.15.16:0/3977076393 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe36006f050 0x7fe36013f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:55.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.692+0000 7fe364916700 1 -- 172.21.15.16:0/3977076393 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe36006f050 msgr2=0x7fe36013f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:55.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.692+0000 7fe364916700 1 --2- 172.21.15.16:0/3977076393 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe36006f050 0x7fe36013f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:55.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.692+0000 7fe364916700 1 -- 172.21.15.16:0/3977076393 >> v1:172.21.15.16:6789/0 conn(0x7fe36006e570 legacy=0x7fe36006e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:55.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.693+0000 7fe364916700 1 -- 172.21.15.16:0/3977076393 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe360071b90 con 0x7fe36009eca0 2023-09-30T22:32:55.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.693+0000 7fe365918700 1 --2- 172.21.15.16:0/3977076393 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe36006f050 0x7fe36013f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:32:55.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.693+0000 7fe364916700 1 --2- 172.21.15.16:0/3977076393 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe36009eca0 0x7fe36013cea0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe35c009870 tx=0x7fe35c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6da64b6fcad7a3d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:55.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.693+0000 7fe353fff700 1 -- 172.21.15.16:0/3977076393 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35c011830 con 0x7fe36009eca0 2023-09-30T22:32:55.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.693+0000 7fe353fff700 1 -- 172.21.15.16:0/3977076393 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe35c011990 con 0x7fe36009eca0 2023-09-30T22:32:55.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.693+0000 7fe353fff700 1 -- 172.21.15.16:0/3977076393 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35c011d20 con 0x7fe36009eca0 2023-09-30T22:32:55.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.694+0000 7fe36737b700 1 -- 172.21.15.16:0/3977076393 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe36009eca0 msgr2=0x7fe36013cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:55.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.694+0000 7fe36737b700 1 --2- 172.21.15.16:0/3977076393 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe36009eca0 0x7fe36013cea0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe35c009870 tx=0x7fe35c009b80 comp rx=0 tx=0).stop 2023-09-30T22:32:55.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.694+0000 7fe36737b700 1 -- 172.21.15.16:0/3977076393 shutdown_connections 2023-09-30T22:32:55.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.694+0000 7fe36737b700 1 --2- 172.21.15.16:0/3977076393 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe36006f050 0x7fe36013f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:55.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.694+0000 7fe36737b700 1 --2- 172.21.15.16:0/3977076393 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe36009eca0 0x7fe36013cea0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:55.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.694+0000 7fe36737b700 1 -- 172.21.15.16:0/3977076393 >> 172.21.15.16:0/3977076393 conn(0x7fe3600a2aa0 msgr2=0x7fe3600a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:55.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.694+0000 7fe36737b700 1 -- 172.21.15.16:0/3977076393 shutdown_connections 2023-09-30T22:32:55.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.694+0000 7fe36737b700 1 -- 172.21.15.16:0/3977076393 wait complete. 2023-09-30T22:32:55.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.695+0000 7fe36737b700 1 Processor -- start 2023-09-30T22:32:55.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.695+0000 7fe36737b700 1 -- start start 2023-09-30T22:32:55.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.696+0000 7fe36737b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe36006e570 0x7fe3601364b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:55.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.696+0000 7fe36737b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe36006f050 0x7fe3601369f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:55.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.696+0000 7fe36737b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe36013e700 0x7fe360136f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:55.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.696+0000 7fe36737b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3601381c0 con 0x7fe36013e700 2023-09-30T22:32:55.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.696+0000 7fe36737b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe360070610 con 0x7fe36006f050 2023-09-30T22:32:55.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.696+0000 7fe36737b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe360139300 con 0x7fe36006e570 2023-09-30T22:32:55.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.696+0000 7fa976ffd700 1 -- 172.21.15.16:0/2134780895 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa9800074e0 con 0x7fa988134260 2023-09-30T22:32:55.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.696+0000 7fe365117700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe36006e570 0x7fe3601364b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:55.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.696+0000 7fe365117700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe36006e570 0x7fe3601364b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:47542/0 (socket says 172.21.15.16:47542) 2023-09-30T22:32:55.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.696+0000 7fe365117700 1 -- 172.21.15.16:0/2743330707 learned_addr learned my addr 172.21.15.16:0/2743330707 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:55.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.696+0000 7fe364916700 1 --2- 172.21.15.16:0/2743330707 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe36006f050 0x7fe3601369f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:55.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.696+0000 7fe365918700 1 --2- 172.21.15.16:0/2743330707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe36013e700 0x7fe360136f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:55.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.696+0000 7fe365117700 1 -- 172.21.15.16:0/2743330707 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe36006f050 msgr2=0x7fe3601369f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:55.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.696+0000 7fe365117700 1 --2- 172.21.15.16:0/2743330707 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe36006f050 0x7fe3601369f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:55.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.696+0000 7fe365117700 1 -- 172.21.15.16:0/2743330707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe36013e700 msgr2=0x7fe360136f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:55.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.696+0000 7fe365117700 1 --2- 172.21.15.16:0/2743330707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe36013e700 0x7fe360136f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:55.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.696+0000 7fe365117700 1 -- 172.21.15.16:0/2743330707 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe35c0094a0 con 0x7fe36006e570 2023-09-30T22:32:55.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.697+0000 7fe365117700 1 --2- 172.21.15.16:0/2743330707 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe36006e570 0x7fe3601364b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fe35400c520 tx=0x7fe35400c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:55.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.697+0000 7fe351ffb700 1 -- 172.21.15.16:0/2743330707 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe354005d70 con 0x7fe36006e570 2023-09-30T22:32:55.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.697+0000 7fe351ffb700 1 -- 172.21.15.16:0/2743330707 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3540070d0 con 0x7fe36006e570 2023-09-30T22:32:55.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.697+0000 7fe351ffb700 1 -- 172.21.15.16:0/2743330707 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe354007440 con 0x7fe36006e570 2023-09-30T22:32:55.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.697+0000 7fe36737b700 1 -- 172.21.15.16:0/2743330707 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3601399b0 con 0x7fe36006e570 2023-09-30T22:32:55.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.697+0000 7fe365918700 1 --2- 172.21.15.16:0/2743330707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe36013e700 0x7fe360136f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:32:55.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.697+0000 7fe36737b700 1 -- 172.21.15.16:0/2743330707 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe360139f50 con 0x7fe36006e570 2023-09-30T22:32:55.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.699+0000 7fe351ffb700 1 -- 172.21.15.16:0/2743330707 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe3540075a0 con 0x7fe36006e570 2023-09-30T22:32:55.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.699+0000 7fe351ffb700 1 -- 172.21.15.16:0/2743330707 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fe35403ec50 con 0x7fe36006e570 2023-09-30T22:32:55.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.701+0000 7fe36737b700 1 -- 172.21.15.16:0/2743330707 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe360139d00 con 0x7fe36006e570 2023-09-30T22:32:55.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.706+0000 7fe351ffb700 1 -- 172.21.15.16:0/2743330707 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe35403ee60 con 0x7fe36006e570 2023-09-30T22:32:55.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.839+0000 7ff618442700 1 -- 172.21.15.16:0/787344950 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff61006c360 con 0x7ff61009e000 2023-09-30T22:32:55.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.840+0000 7ff606ffd700 1 -- 172.21.15.16:0/787344950 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff6000273f0 con 0x7ff61009e000 2023-09-30T22:32:55.840 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:32:55.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.842+0000 7ff618442700 1 -- 172.21.15.16:0/787344950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff61009e000 msgr2=0x7ff6101227f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:55.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.842+0000 7ff618442700 1 --2- 172.21.15.16:0/787344950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff61009e000 0x7ff6101227f0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7ff60000c510 tx=0x7ff60000c820 comp rx=0 tx=0).stop 2023-09-30T22:32:55.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.842+0000 7ff618442700 1 -- 172.21.15.16:0/787344950 shutdown_connections 2023-09-30T22:32:55.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.842+0000 7ff618442700 1 --2- 172.21.15.16:0/787344950 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff61009d6f0 0x7ff6101222b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:55.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.842+0000 7ff618442700 1 --2- 172.21.15.16:0/787344950 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff61009e410 0x7ff610122d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:55.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.842+0000 7ff618442700 1 --2- 172.21.15.16:0/787344950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff61009e000 0x7ff6101227f0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:55.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.842+0000 7ff618442700 1 -- 172.21.15.16:0/787344950 >> 172.21.15.16:0/787344950 conn(0x7ff610066f50 msgr2=0x7ff610070f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:55.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.843+0000 7ff618442700 1 -- 172.21.15.16:0/787344950 shutdown_connections 2023-09-30T22:32:55.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.843+0000 7ff618442700 1 -- 172.21.15.16:0/787344950 wait complete. 2023-09-30T22:32:55.862 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:32:55.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.876+0000 7fa71a1b7700 1 -- 172.21.15.16:0/1590891019 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa71411f030 con 0x7fa71409d6f0 2023-09-30T22:32:55.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.876+0000 7fa710ff9700 1 -- 172.21.15.16:0/1590891019 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa704029030 con 0x7fa71409d6f0 2023-09-30T22:32:55.877 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:32:55.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.878+0000 7fa71a1b7700 1 -- 172.21.15.16:0/1590891019 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa71409d6f0 msgr2=0x7fa7141223b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:55.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.878+0000 7fa71a1b7700 1 --2- 172.21.15.16:0/1590891019 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa71409d6f0 0x7fa7141223b0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa7040090b0 tx=0x7fa704004750 comp rx=0 tx=0).stop 2023-09-30T22:32:55.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.878+0000 7fa71a1b7700 1 -- 172.21.15.16:0/1590891019 shutdown_connections 2023-09-30T22:32:55.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.878+0000 7fa71a1b7700 1 --2- 172.21.15.16:0/1590891019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa71409e410 0x7fa714122e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:55.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.878+0000 7fa71a1b7700 1 --2- 172.21.15.16:0/1590891019 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa71409d6f0 0x7fa7141223b0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:55.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.878+0000 7fa71a1b7700 1 --2- 172.21.15.16:0/1590891019 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa71409e000 0x7fa7141228f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:55.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.878+0000 7fa71a1b7700 1 -- 172.21.15.16:0/1590891019 >> 172.21.15.16:0/1590891019 conn(0x7fa714066f50 msgr2=0x7fa714133f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:55.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.879+0000 7fa71a1b7700 1 -- 172.21.15.16:0/1590891019 shutdown_connections 2023-09-30T22:32:55.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.879+0000 7fa71a1b7700 1 -- 172.21.15.16:0/1590891019 wait complete. 2023-09-30T22:32:55.893 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:32:55.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.943+0000 7fa98cfe3700 1 -- 172.21.15.16:0/2134780895 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa98806b250 con 0x7fa988134260 2023-09-30T22:32:55.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.944+0000 7fa976ffd700 1 -- 172.21.15.16:0/2134780895 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa980018030 con 0x7fa988134260 2023-09-30T22:32:55.944 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:32:55.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.944+0000 7fe36737b700 1 -- 172.21.15.16:0/2743330707 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe360074ea0 con 0x7fe36006e570 2023-09-30T22:32:55.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.944+0000 7fe351ffb700 1 -- 172.21.15.16:0/2743330707 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe354017080 con 0x7fe36006e570 2023-09-30T22:32:55.945 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:32:55.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.947+0000 7fa98cfe3700 1 -- 172.21.15.16:0/2134780895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa988134260 msgr2=0x7fa98809cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:55.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.947+0000 7fa98cfe3700 1 --2- 172.21.15.16:0/2134780895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa988134260 0x7fa98809cf00 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fa98000cbe0 tx=0x7fa98000cef0 comp rx=0 tx=0).stop 2023-09-30T22:32:55.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.947+0000 7fa98cfe3700 1 -- 172.21.15.16:0/2134780895 shutdown_connections 2023-09-30T22:32:55.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.947+0000 7fa98cfe3700 1 --2- 172.21.15.16:0/2134780895 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa988133420 0x7fa98809c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:55.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.947+0000 7fa98cfe3700 1 --2- 172.21.15.16:0/2134780895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa988134260 0x7fa98809cf00 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:55.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.947+0000 7fa98cfe3700 1 --2- 172.21.15.16:0/2134780895 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa988133e30 0x7fa98809c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:55.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.947+0000 7fa98cfe3700 1 -- 172.21.15.16:0/2134780895 >> 172.21.15.16:0/2134780895 conn(0x7fa9880a4210 msgr2=0x7fa9880a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:55.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.947+0000 7fa98cfe3700 1 -- 172.21.15.16:0/2134780895 shutdown_connections 2023-09-30T22:32:55.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.947+0000 7fa98cfe3700 1 -- 172.21.15.16:0/2134780895 wait complete. 2023-09-30T22:32:55.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.947+0000 7fe36737b700 1 -- 172.21.15.16:0/2743330707 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe36006e570 msgr2=0x7fe3601364b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:55.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.947+0000 7fe36737b700 1 --2- 172.21.15.16:0/2743330707 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe36006e570 0x7fe3601364b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fe35400c520 tx=0x7fe35400c830 comp rx=0 tx=0).stop 2023-09-30T22:32:55.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.948+0000 7fe36737b700 1 -- 172.21.15.16:0/2743330707 shutdown_connections 2023-09-30T22:32:55.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.948+0000 7fe36737b700 1 --2- 172.21.15.16:0/2743330707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe36013e700 0x7fe360136f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:55.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.948+0000 7fe36737b700 1 --2- 172.21.15.16:0/2743330707 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe36006e570 0x7fe3601364b0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:55.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.948+0000 7fe36737b700 1 --2- 172.21.15.16:0/2743330707 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe36006f050 0x7fe3601369f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:55.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.948+0000 7fe36737b700 1 -- 172.21.15.16:0/2743330707 >> 172.21.15.16:0/2743330707 conn(0x7fe3600a2aa0 msgr2=0x7fe3600a0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:55.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.948+0000 7fe36737b700 1 -- 172.21.15.16:0/2743330707 shutdown_connections 2023-09-30T22:32:55.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:55.948+0000 7fe36737b700 1 -- 172.21.15.16:0/2743330707 wait complete. 2023-09-30T22:32:55.962 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:32:55.963 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:32:56.863 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:32:56.894 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:32:56.964 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:32:56.964 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:32:57.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.018+0000 7fb76ccb3700 1 Processor -- start 2023-09-30T22:32:57.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.018+0000 7fb76ccb3700 1 -- start start 2023-09-30T22:32:57.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.019+0000 7fb76ccb3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb768133510 0x7fb7681358f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:57.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.019+0000 7fb76ccb3700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb768135e30 0x7fb768138210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:57.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.019+0000 7fb76ccb3700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb768130260 con 0x7fb768130670 2023-09-30T22:32:57.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.019+0000 7fb76ccb3700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb76809a350 con 0x7fb768135e30 2023-09-30T22:32:57.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.019+0000 7fb76ccb3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb768071ff0 con 0x7fb768133510 2023-09-30T22:32:57.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.019+0000 7fb76659c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb768130670 0x7fb768130a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:53874/0 (socket says 172.21.15.16:53874) 2023-09-30T22:32:57.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.019+0000 7fb76659c700 1 -- 172.21.15.16:0/1405576485 learned_addr learned my addr 172.21.15.16:0/1405576485 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:57.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.019+0000 7fb766d9d700 1 --2- 172.21.15.16:0/1405576485 >> v2:172.21.15.16:3300/0 conn(0x7fb768135e30 0x7fb768138210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:57.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.019+0000 7fb765d9b700 1 --2- 172.21.15.16:0/1405576485 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb768133510 0x7fb7681358f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:57.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.020+0000 7fb766d9d700 1 -- 172.21.15.16:0/1405576485 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb768133510 msgr2=0x7fb7681358f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:57.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.020+0000 7fb766d9d700 1 --2- 172.21.15.16:0/1405576485 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb768133510 0x7fb7681358f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:57.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.020+0000 7fb766d9d700 1 -- 172.21.15.16:0/1405576485 >> v1:172.21.15.16:6789/0 conn(0x7fb768130670 legacy=0x7fb768130a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:57.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.020+0000 7fb766d9d700 1 -- 172.21.15.16:0/1405576485 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb76809ecf0 con 0x7fb768135e30 2023-09-30T22:32:57.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.020+0000 7fb765d9b700 1 --2- 172.21.15.16:0/1405576485 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb768133510 0x7fb7681358f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:57.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.020+0000 7fb766d9d700 1 --2- 172.21.15.16:0/1405576485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb768135e30 0x7fb768138210 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fb760009d90 tx=0x7fb76000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa6993c76927a12a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:57.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.020+0000 7fb76559a700 1 -- 172.21.15.16:0/1405576485 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76000e070 con 0x7fb768135e30 2023-09-30T22:32:57.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.020+0000 7fb76559a700 1 -- 172.21.15.16:0/1405576485 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb76000bae0 con 0x7fb768135e30 2023-09-30T22:32:57.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.020+0000 7fb76559a700 1 -- 172.21.15.16:0/1405576485 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76000be90 con 0x7fb768135e30 2023-09-30T22:32:57.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.021+0000 7fb76ccb3700 1 -- 172.21.15.16:0/1405576485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb768135e30 msgr2=0x7fb768138210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:57.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.021+0000 7fb76ccb3700 1 --2- 172.21.15.16:0/1405576485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb768135e30 0x7fb768138210 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fb760009d90 tx=0x7fb76000b330 comp rx=0 tx=0).stop 2023-09-30T22:32:57.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.021+0000 7fb76ccb3700 1 -- 172.21.15.16:0/1405576485 shutdown_connections 2023-09-30T22:32:57.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.021+0000 7fb76ccb3700 1 --2- 172.21.15.16:0/1405576485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb768135e30 0x7fb768138210 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:57.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.021+0000 7fb76ccb3700 1 --2- 172.21.15.16:0/1405576485 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb768133510 0x7fb7681358f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:57.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.021+0000 7fb76ccb3700 1 -- 172.21.15.16:0/1405576485 >> 172.21.15.16:0/1405576485 conn(0x7fb7680a2aa0 msgr2=0x7fb7680a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:57.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.021+0000 7fb76ccb3700 1 -- 172.21.15.16:0/1405576485 shutdown_connections 2023-09-30T22:32:57.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.021+0000 7fb76ccb3700 1 -- 172.21.15.16:0/1405576485 wait complete. 2023-09-30T22:32:57.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.023+0000 7fb76ccb3700 1 Processor -- start 2023-09-30T22:32:57.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.023+0000 7fb76ccb3700 1 -- start start 2023-09-30T22:32:57.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.024+0000 7fb76ccb3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb768130670 0x7fb76812d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:57.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.024+0000 7fb76ccb3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb768133510 0x7fb76812d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:57.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.024+0000 7fb76ccb3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb768135e30 0x7fb768139760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:57.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.024+0000 7fb76ccb3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb768071820 con 0x7fb768130670 2023-09-30T22:32:57.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.024+0000 7fb76ccb3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb76809b570 con 0x7fb768135e30 2023-09-30T22:32:57.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.024+0000 7fb76ccb3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb76809e410 con 0x7fb768133510 2023-09-30T22:32:57.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.024+0000 7fb766d9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb768135e30 0x7fb768139760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:57.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.024+0000 7fb766d9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb768135e30 0x7fb768139760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37508/0 (socket says 172.21.15.16:37508) 2023-09-30T22:32:57.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.024+0000 7fb766d9d700 1 -- 172.21.15.16:0/659227324 learned_addr learned my addr 172.21.15.16:0/659227324 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:57.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.024+0000 7fb765d9b700 1 --2- 172.21.15.16:0/659227324 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb768133510 0x7fb76812d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:57.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.024+0000 7fb76659c700 1 --2- 172.21.15.16:0/659227324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb768130670 0x7fb76812d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:57.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.024+0000 7fb766d9d700 1 -- 172.21.15.16:0/659227324 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb768133510 msgr2=0x7fb76812d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:57.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.024+0000 7fb766d9d700 1 --2- 172.21.15.16:0/659227324 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb768133510 0x7fb76812d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:57.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.024+0000 7fb766d9d700 1 -- 172.21.15.16:0/659227324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb768130670 msgr2=0x7fb76812d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:57.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.024+0000 7fb766d9d700 1 --2- 172.21.15.16:0/659227324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb768130670 0x7fb76812d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:57.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.024+0000 7fb766d9d700 1 -- 172.21.15.16:0/659227324 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb75c0094a0 con 0x7fb768135e30 2023-09-30T22:32:57.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.024+0000 7fb765d9b700 1 --2- 172.21.15.16:0/659227324 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb768133510 0x7fb76812d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:57.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.025+0000 7fb766d9d700 1 --2- 172.21.15.16:0/659227324 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb768135e30 0x7fb768139760 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb768071280 tx=0x7fb760004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:57.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.025+0000 7fb756ffd700 1 -- 172.21.15.16:0/659227324 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb760004540 con 0x7fb768135e30 2023-09-30T22:32:57.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.025+0000 7fb756ffd700 1 -- 172.21.15.16:0/659227324 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7600046a0 con 0x7fb768135e30 2023-09-30T22:32:57.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.025+0000 7fb76ccb3700 1 -- 172.21.15.16:0/659227324 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb760009a70 con 0x7fb768135e30 2023-09-30T22:32:57.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.025+0000 7fb76ccb3700 1 -- 172.21.15.16:0/659227324 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb768139310 con 0x7fb768135e30 2023-09-30T22:32:57.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.025+0000 7fb756ffd700 1 -- 172.21.15.16:0/659227324 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb760012430 con 0x7fb768135e30 2023-09-30T22:32:57.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.026+0000 7fb756ffd700 1 -- 172.21.15.16:0/659227324 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb760012590 con 0x7fb768135e30 2023-09-30T22:32:57.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.026+0000 7fb756ffd700 1 -- 172.21.15.16:0/659227324 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fb760041740 con 0x7fb768135e30 2023-09-30T22:32:57.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.026+0000 7fb76ccb3700 1 -- 172.21.15.16:0/659227324 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb74c0052e0 con 0x7fb768135e30 2023-09-30T22:32:57.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.036+0000 7fb756ffd700 1 -- 172.21.15.16:0/659227324 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb760011980 con 0x7fb768135e30 2023-09-30T22:32:57.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.057+0000 7f592ff21700 1 Processor -- start 2023-09-30T22:32:57.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.058+0000 7f592ff21700 1 -- start start 2023-09-30T22:32:57.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.058+0000 7f592ff21700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f592809d6f0 0x7f592809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:57.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.058+0000 7f592ff21700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f592809e000 0x7f59281339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:57.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.058+0000 7f592ff21700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f592809b120 con 0x7f592809e410 2023-09-30T22:32:57.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.058+0000 7f592ff21700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f592806b460 con 0x7f592809d6f0 2023-09-30T22:32:57.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.058+0000 7f592ff21700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f592809a7e0 con 0x7f592809e000 2023-09-30T22:32:57.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.058+0000 7f592d4bc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f592809e000 0x7f59281339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:57.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.058+0000 7f592d4bc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f592809e000 0x7f59281339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37512/0 (socket says 172.21.15.16:37512) 2023-09-30T22:32:57.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.058+0000 7f592d4bc700 1 -- 172.21.15.16:0/877273955 learned_addr learned my addr 172.21.15.16:0/877273955 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:57.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.058+0000 7f592dcbd700 1 --2- 172.21.15.16:0/877273955 >> v2:172.21.15.16:3300/0 conn(0x7f592809d6f0 0x7f592809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:57.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.059+0000 7f592dcbd700 1 -- 172.21.15.16:0/877273955 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f592809e000 msgr2=0x7f59281339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:57.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.059+0000 7f592dcbd700 1 --2- 172.21.15.16:0/877273955 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f592809e000 0x7f59281339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:57.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.059+0000 7f592dcbd700 1 -- 172.21.15.16:0/877273955 >> v1:172.21.15.16:6789/0 conn(0x7f592809e410 legacy=0x7f5928133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:57.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.059+0000 7f592dcbd700 1 -- 172.21.15.16:0/877273955 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f592809a500 con 0x7f592809d6f0 2023-09-30T22:32:57.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.059+0000 7f592d4bc700 1 --2- 172.21.15.16:0/877273955 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f592809e000 0x7f59281339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:57.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.059+0000 7f592dcbd700 1 --2- 172.21.15.16:0/877273955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f592809d6f0 0x7f592809dac0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f5924013150 tx=0x7f5924013460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dec306bc30d5dcfa server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:57.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.059+0000 7f592ccbb700 1 -- 172.21.15.16:0/877273955 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5924016070 con 0x7f592809d6f0 2023-09-30T22:32:57.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.059+0000 7f592ccbb700 1 -- 172.21.15.16:0/877273955 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5924013d10 con 0x7f592809d6f0 2023-09-30T22:32:57.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.059+0000 7f592ccbb700 1 -- 172.21.15.16:0/877273955 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59240112f0 con 0x7f592809d6f0 2023-09-30T22:32:57.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.060+0000 7f592ff21700 1 -- 172.21.15.16:0/877273955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f592809d6f0 msgr2=0x7f592809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:57.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.060+0000 7f592ff21700 1 --2- 172.21.15.16:0/877273955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f592809d6f0 0x7f592809dac0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f5924013150 tx=0x7f5924013460 comp rx=0 tx=0).stop 2023-09-30T22:32:57.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.060+0000 7f592ff21700 1 -- 172.21.15.16:0/877273955 shutdown_connections 2023-09-30T22:32:57.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.060+0000 7f592ff21700 1 --2- 172.21.15.16:0/877273955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f592809d6f0 0x7f592809dac0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:57.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.060+0000 7f592ff21700 1 --2- 172.21.15.16:0/877273955 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f592809e000 0x7f59281339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:57.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.060+0000 7f592ff21700 1 -- 172.21.15.16:0/877273955 >> 172.21.15.16:0/877273955 conn(0x7f5928066f50 msgr2=0x7f592809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:57.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.061+0000 7f592ff21700 1 -- 172.21.15.16:0/877273955 shutdown_connections 2023-09-30T22:32:57.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.061+0000 7f592ff21700 1 -- 172.21.15.16:0/877273955 wait complete. 2023-09-30T22:32:57.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.062+0000 7f592ff21700 1 Processor -- start 2023-09-30T22:32:57.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.062+0000 7f592ff21700 1 -- start start 2023-09-30T22:32:57.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.063+0000 7f592ff21700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f592809d6f0 0x7f59281a1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:57.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.063+0000 7f592ff21700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f592809e000 0x7f59281a1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:57.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.063+0000 7f592ff21700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f592809e410 0x7f59281a24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:57.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.063+0000 7f592ff21700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59281a4d20 con 0x7f592809d6f0 2023-09-30T22:32:57.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.063+0000 7f592ff21700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f592809c710 con 0x7f592809e000 2023-09-30T22:32:57.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.063+0000 7f592ff21700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f59280a3250 con 0x7f592809e410 2023-09-30T22:32:57.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.063+0000 7f592d4bc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f592809e000 0x7f59281a1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:57.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.063+0000 7f592d4bc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f592809e000 0x7f59281a1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37528/0 (socket says 172.21.15.16:37528) 2023-09-30T22:32:57.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.063+0000 7f592d4bc700 1 -- 172.21.15.16:0/2430626850 learned_addr learned my addr 172.21.15.16:0/2430626850 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:57.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.063+0000 7f592dcbd700 1 --2- 172.21.15.16:0/2430626850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f592809d6f0 0x7f59281a1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:57.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.063+0000 7f592e4be700 1 --2- 172.21.15.16:0/2430626850 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f592809e410 0x7f59281a24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:57.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.063+0000 7f592d4bc700 1 -- 172.21.15.16:0/2430626850 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f592809e410 msgr2=0x7f59281a24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:57.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.063+0000 7f592d4bc700 1 --2- 172.21.15.16:0/2430626850 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f592809e410 0x7f59281a24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:57.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.063+0000 7f592d4bc700 1 -- 172.21.15.16:0/2430626850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f592809d6f0 msgr2=0x7f59281a1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:57.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.063+0000 7f592d4bc700 1 --2- 172.21.15.16:0/2430626850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f592809d6f0 0x7f59281a1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:57.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.063+0000 7f592d4bc700 1 -- 172.21.15.16:0/2430626850 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5918009550 con 0x7f592809e000 2023-09-30T22:32:57.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.064+0000 7f592d4bc700 1 --2- 172.21.15.16:0/2430626850 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f592809e000 0x7f59281a1fb0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f591800ebe0 tx=0x7f591800eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:57.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.064+0000 7f591e7fc700 1 -- 172.21.15.16:0/2430626850 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f591800c850 con 0x7f592809e000 2023-09-30T22:32:57.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.064+0000 7f591e7fc700 1 -- 172.21.15.16:0/2430626850 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f591800c9b0 con 0x7f592809e000 2023-09-30T22:32:57.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.064+0000 7f592ff21700 1 -- 172.21.15.16:0/2430626850 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5924011d40 con 0x7f592809e000 2023-09-30T22:32:57.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.064+0000 7f592ff21700 1 -- 172.21.15.16:0/2430626850 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59281a48d0 con 0x7f592809e000 2023-09-30T22:32:57.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.064+0000 7f591e7fc700 1 -- 172.21.15.16:0/2430626850 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f591800cd20 con 0x7f592809e000 2023-09-30T22:32:57.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.065+0000 7f591e7fc700 1 -- 172.21.15.16:0/2430626850 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5918013470 con 0x7f592809e000 2023-09-30T22:32:57.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.065+0000 7f591e7fc700 1 -- 172.21.15.16:0/2430626850 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f591803f7f0 con 0x7f592809e000 2023-09-30T22:32:57.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.068+0000 7f592ff21700 1 -- 172.21.15.16:0/2430626850 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59100052e0 con 0x7f592809e000 2023-09-30T22:32:57.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.077+0000 7f591e7fc700 1 -- 172.21.15.16:0/2430626850 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f59180232f0 con 0x7f592809e000 2023-09-30T22:32:57.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.115+0000 7fa6c5aae700 1 Processor -- start 2023-09-30T22:32:57.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.115+0000 7fa6c5aae700 1 -- start start 2023-09-30T22:32:57.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.116+0000 7fa6c5aae700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa6c009d6f0 0x7fa6c009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:57.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.116+0000 7fa6c5aae700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa6c009e000 0x7fa6c01339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:57.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.116+0000 7fa6c5aae700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa6c009e410 0x7fa6c0133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:57.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.116+0000 7fa6c5aae700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6c006b460 con 0x7fa6c009e000 2023-09-30T22:32:57.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.116+0000 7fa6c5aae700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c009b120 con 0x7fa6c009e410 2023-09-30T22:32:57.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.116+0000 7fa6c5aae700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c009a7e0 con 0x7fa6c009d6f0 2023-09-30T22:32:57.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.116+0000 7fa6bf7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa6c009d6f0 0x7fa6c009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:57.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.116+0000 7fa6beffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa6c009e000 0x7fa6c01339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:57.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.116+0000 7fa6bf7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa6c009d6f0 0x7fa6c009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:47570/0 (socket says 172.21.15.16:47570) 2023-09-30T22:32:57.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.116+0000 7fa6beffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa6c009e000 0x7fa6c01339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58976/0 (socket says 172.21.15.16:58976) 2023-09-30T22:32:57.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.116+0000 7fa6bf7fe700 1 -- 172.21.15.16:0/3814098013 learned_addr learned my addr 172.21.15.16:0/3814098013 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:57.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.116+0000 7fa6bffff700 1 --2- 172.21.15.16:0/3814098013 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa6c009e410 0x7fa6c0133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:57.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.117+0000 7fa6bf7fe700 1 -- 172.21.15.16:0/3814098013 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa6c009e410 msgr2=0x7fa6c0133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:57.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.117+0000 7fa6bf7fe700 1 --2- 172.21.15.16:0/3814098013 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa6c009e410 0x7fa6c0133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:57.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.117+0000 7fa6bf7fe700 1 -- 172.21.15.16:0/3814098013 >> v2:172.21.15.16:3300/0 conn(0x7fa6c009e000 msgr2=0x7fa6c01339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:57.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.117+0000 7fa6bf7fe700 1 --2- 172.21.15.16:0/3814098013 >> v2:172.21.15.16:3300/0 conn(0x7fa6c009e000 0x7fa6c01339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:57.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.117+0000 7fa6bf7fe700 1 -- 172.21.15.16:0/3814098013 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c009a500 con 0x7fa6c009d6f0 2023-09-30T22:32:57.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.117+0000 7fa6beffd700 1 --2- 172.21.15.16:0/3814098013 >> v2:172.21.15.16:3300/0 conn(0x7fa6c009e000 0x7fa6c01339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:57.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.117+0000 7fa6bffff700 1 --2- 172.21.15.16:0/3814098013 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa6c009e410 0x7fa6c0133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:32:57.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.117+0000 7fa6bf7fe700 1 --2- 172.21.15.16:0/3814098013 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa6c009d6f0 0x7fa6c009dac0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0009920 tx=0x7fa6b0009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=474af48bf6cfa432 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:57.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.117+0000 7fa6be7fc700 1 -- 172.21.15.16:0/3814098013 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b0011830 con 0x7fa6c009d6f0 2023-09-30T22:32:57.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.117+0000 7fa6be7fc700 1 -- 172.21.15.16:0/3814098013 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6b0011990 con 0x7fa6c009d6f0 2023-09-30T22:32:57.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.117+0000 7fa6be7fc700 1 -- 172.21.15.16:0/3814098013 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b0011d20 con 0x7fa6c009d6f0 2023-09-30T22:32:57.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.118+0000 7fa6c5aae700 1 -- 172.21.15.16:0/3814098013 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa6c009d6f0 msgr2=0x7fa6c009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:57.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.118+0000 7fa6c5aae700 1 --2- 172.21.15.16:0/3814098013 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa6c009d6f0 0x7fa6c009dac0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0009920 tx=0x7fa6b0009c30 comp rx=0 tx=0).stop 2023-09-30T22:32:57.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.118+0000 7fa6c5aae700 1 -- 172.21.15.16:0/3814098013 shutdown_connections 2023-09-30T22:32:57.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.118+0000 7fa6c5aae700 1 --2- 172.21.15.16:0/3814098013 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa6c009e410 0x7fa6c0133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:57.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.118+0000 7fa6c5aae700 1 --2- 172.21.15.16:0/3814098013 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa6c009d6f0 0x7fa6c009dac0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:57.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.118+0000 7fa6c5aae700 1 --2- 172.21.15.16:0/3814098013 >> v2:172.21.15.16:3300/0 conn(0x7fa6c009e000 0x7fa6c01339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:57.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.118+0000 7fa6c5aae700 1 -- 172.21.15.16:0/3814098013 >> 172.21.15.16:0/3814098013 conn(0x7fa6c0066f50 msgr2=0x7fa6c009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:57.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.118+0000 7fa6c5aae700 1 -- 172.21.15.16:0/3814098013 shutdown_connections 2023-09-30T22:32:57.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.118+0000 7fa6c5aae700 1 -- 172.21.15.16:0/3814098013 wait complete. 2023-09-30T22:32:57.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.119+0000 7fa6c5aae700 1 Processor -- start 2023-09-30T22:32:57.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.119+0000 7fa6c5aae700 1 -- start start 2023-09-30T22:32:57.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.120+0000 7fa6c5aae700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6c009d6f0 0x7fa6c01a5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:57.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.120+0000 7fa6c5aae700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa6c009e000 0x7fa6c01a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:57.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.120+0000 7fa6c5aae700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa6c009e410 0x7fa6c01a6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:57.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.120+0000 7fa6c5aae700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c01aa220 con 0x7fa6c009d6f0 2023-09-30T22:32:57.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.120+0000 7fa6c5aae700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c01a7950 con 0x7fa6c009e000 2023-09-30T22:32:57.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.120+0000 7fa6c5aae700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c01a7f70 con 0x7fa6c009e410 2023-09-30T22:32:57.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.120+0000 7fa6beffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa6c009e000 0x7fa6c01a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:57.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.120+0000 7fa6bf7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6c009d6f0 0x7fa6c01a5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:57.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.120+0000 7fa6beffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa6c009e000 0x7fa6c01a6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37538/0 (socket says 172.21.15.16:37538) 2023-09-30T22:32:57.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.120+0000 7fa6beffd700 1 -- 172.21.15.16:0/3665891547 learned_addr learned my addr 172.21.15.16:0/3665891547 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:57.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.120+0000 7fa6bffff700 1 --2- 172.21.15.16:0/3665891547 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa6c009e410 0x7fa6c01a6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:57.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.120+0000 7fa6bf7fe700 1 -- 172.21.15.16:0/3665891547 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa6c009e410 msgr2=0x7fa6c01a6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:57.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.120+0000 7fa6bf7fe700 1 --2- 172.21.15.16:0/3665891547 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa6c009e410 0x7fa6c01a6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:57.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.120+0000 7fa6bf7fe700 1 -- 172.21.15.16:0/3665891547 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa6c009e000 msgr2=0x7fa6c01a6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:57.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.120+0000 7fa6bf7fe700 1 --2- 172.21.15.16:0/3665891547 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa6c009e000 0x7fa6c01a6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:57.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.120+0000 7fa6bf7fe700 1 -- 172.21.15.16:0/3665891547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b40094a0 con 0x7fa6c009d6f0 2023-09-30T22:32:57.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.120+0000 7fa6beffd700 1 --2- 172.21.15.16:0/3665891547 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa6c009e000 0x7fa6c01a6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:57.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.121+0000 7fa6bf7fe700 1 --2- 172.21.15.16:0/3665891547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6c009d6f0 0x7fa6c01a5de0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fa6c006e470 tx=0x7fa6b0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:57.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.121+0000 7fa6a7fff700 1 -- 172.21.15.16:0/3665891547 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b0004510 con 0x7fa6c009d6f0 2023-09-30T22:32:57.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.121+0000 7fa6a7fff700 1 -- 172.21.15.16:0/3665891547 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6b00036a0 con 0x7fa6c009d6f0 2023-09-30T22:32:57.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.121+0000 7fea98592700 1 Processor -- start 2023-09-30T22:32:57.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.121+0000 7fa6a7fff700 1 -- 172.21.15.16:0/3665891547 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b0003a50 con 0x7fa6c009d6f0 2023-09-30T22:32:57.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.121+0000 7fa6c5aae700 1 -- 172.21.15.16:0/3665891547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6b0009550 con 0x7fa6c009d6f0 2023-09-30T22:32:57.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.121+0000 7fa6c5aae700 1 -- 172.21.15.16:0/3665891547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6c01a8ba0 con 0x7fa6c009d6f0 2023-09-30T22:32:57.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.121+0000 7fea98592700 1 -- start start 2023-09-30T22:32:57.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.122+0000 7fea98592700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fea9009d6f0 0x7fea9009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:57.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.122+0000 7fea98592700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea9009e000 0x7fea90130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:57.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.122+0000 7fea98592700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea9009e410 0x7fea90130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:57.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.122+0000 7fea98592700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea9009f520 con 0x7fea9009d6f0 2023-09-30T22:32:57.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.122+0000 7fea98592700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea9009f130 con 0x7fea9009e410 2023-09-30T22:32:57.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.122+0000 7fea98592700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fea9009e930 con 0x7fea9009e000 2023-09-30T22:32:57.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.122+0000 7fea9632e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fea9009d6f0 0x7fea9009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:57.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.122+0000 7fea96b2f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea9009e410 0x7fea90130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:57.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.122+0000 7fea96b2f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea9009e410 0x7fea90130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37554/0 (socket says 172.21.15.16:37554) 2023-09-30T22:32:57.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.122+0000 7fea96b2f700 1 -- 172.21.15.16:0/4203604856 learned_addr learned my addr 172.21.15.16:0/4203604856 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:57.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.123+0000 7fa6a7fff700 1 -- 172.21.15.16:0/3665891547 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa6b0003bb0 con 0x7fa6c009d6f0 2023-09-30T22:32:57.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.123+0000 7fa6c5aae700 1 -- 172.21.15.16:0/3665891547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6c00a20b0 con 0x7fa6c009d6f0 2023-09-30T22:32:57.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.123+0000 7fea95b2d700 1 --2- 172.21.15.16:0/4203604856 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea9009e000 0x7fea90130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:57.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.123+0000 7fa6a7fff700 1 -- 172.21.15.16:0/3665891547 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fa6b0021030 con 0x7fa6c009d6f0 2023-09-30T22:32:57.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.123+0000 7fea96b2f700 1 -- 172.21.15.16:0/4203604856 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea9009e000 msgr2=0x7fea90130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:57.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.123+0000 7fea96b2f700 1 --2- 172.21.15.16:0/4203604856 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea9009e000 0x7fea90130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:57.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.123+0000 7fea96b2f700 1 -- 172.21.15.16:0/4203604856 >> v2:172.21.15.16:3300/0 conn(0x7fea9009d6f0 msgr2=0x7fea9009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:57.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.123+0000 7fea96b2f700 1 --2- 172.21.15.16:0/4203604856 >> v2:172.21.15.16:3300/0 conn(0x7fea9009d6f0 0x7fea9009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:57.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.123+0000 7fea96b2f700 1 -- 172.21.15.16:0/4203604856 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea9009b2d0 con 0x7fea9009e410 2023-09-30T22:32:57.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.123+0000 7fea96b2f700 1 --2- 172.21.15.16:0/4203604856 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea9009e410 0x7fea90130ba0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fea88009e50 tx=0x7fea8800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea0bd57fb5b1418 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:57.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.125+0000 7fea9532c700 1 -- 172.21.15.16:0/4203604856 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8800e070 con 0x7fea9009e410 2023-09-30T22:32:57.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.125+0000 7fea9532c700 1 -- 172.21.15.16:0/4203604856 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea8800b910 con 0x7fea9009e410 2023-09-30T22:32:57.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.125+0000 7fea9532c700 1 -- 172.21.15.16:0/4203604856 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8800bcc0 con 0x7fea9009e410 2023-09-30T22:32:57.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.125+0000 7fea98592700 1 -- 172.21.15.16:0/4203604856 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea9009e410 msgr2=0x7fea90130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:57.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.125+0000 7fea98592700 1 --2- 172.21.15.16:0/4203604856 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea9009e410 0x7fea90130ba0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fea88009e50 tx=0x7fea8800b330 comp rx=0 tx=0).stop 2023-09-30T22:32:57.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.126+0000 7fea98592700 1 -- 172.21.15.16:0/4203604856 shutdown_connections 2023-09-30T22:32:57.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.126+0000 7fea98592700 1 --2- 172.21.15.16:0/4203604856 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea9009e410 0x7fea90130ba0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:57.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.126+0000 7fea98592700 1 --2- 172.21.15.16:0/4203604856 >> v2:172.21.15.16:3300/0 conn(0x7fea9009d6f0 0x7fea9009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:57.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.126+0000 7fea98592700 1 --2- 172.21.15.16:0/4203604856 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea9009e000 0x7fea90130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:57.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.126+0000 7fea98592700 1 -- 172.21.15.16:0/4203604856 >> 172.21.15.16:0/4203604856 conn(0x7fea90066f50 msgr2=0x7fea9009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:57.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.126+0000 7fea98592700 1 -- 172.21.15.16:0/4203604856 shutdown_connections 2023-09-30T22:32:57.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.126+0000 7fea98592700 1 -- 172.21.15.16:0/4203604856 wait complete. 2023-09-30T22:32:57.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.127+0000 7fea98592700 1 Processor -- start 2023-09-30T22:32:57.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.127+0000 7fea98592700 1 -- start start 2023-09-30T22:32:57.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.127+0000 7fea98592700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea9009d6f0 0x7fea901305f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:57.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.127+0000 7fea98592700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea9009e000 0x7fea90130b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:57.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.127+0000 7fea98592700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea9009e410 0x7fea90136250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:57.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.127+0000 7fea98592700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea901384b0 con 0x7fea9009d6f0 2023-09-30T22:32:57.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.127+0000 7fea98592700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea900a3ad0 con 0x7fea9009e410 2023-09-30T22:32:57.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.127+0000 7fea98592700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fea90139500 con 0x7fea9009e000 2023-09-30T22:32:57.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.127+0000 7fea96b2f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea9009e410 0x7fea90136250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:57.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.127+0000 7fea96b2f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea9009e410 0x7fea90136250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37556/0 (socket says 172.21.15.16:37556) 2023-09-30T22:32:57.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.127+0000 7fea96b2f700 1 -- 172.21.15.16:0/3288275051 learned_addr learned my addr 172.21.15.16:0/3288275051 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:57.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.128+0000 7fea9632e700 1 --2- 172.21.15.16:0/3288275051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea9009d6f0 0x7fea901305f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:57.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.128+0000 7fea95b2d700 1 --2- 172.21.15.16:0/3288275051 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea9009e000 0x7fea90130b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:57.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.128+0000 7fea96b2f700 1 -- 172.21.15.16:0/3288275051 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea9009e000 msgr2=0x7fea90130b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:57.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.128+0000 7fea96b2f700 1 --2- 172.21.15.16:0/3288275051 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea9009e000 0x7fea90130b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:57.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.128+0000 7fea96b2f700 1 -- 172.21.15.16:0/3288275051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea9009d6f0 msgr2=0x7fea901305f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:57.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.128+0000 7fea96b2f700 1 --2- 172.21.15.16:0/3288275051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea9009d6f0 0x7fea901305f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:57.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.128+0000 7fea96b2f700 1 -- 172.21.15.16:0/3288275051 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea88009b30 con 0x7fea9009e410 2023-09-30T22:32:57.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.128+0000 7fea95b2d700 1 --2- 172.21.15.16:0/3288275051 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea9009e000 0x7fea90130b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:32:57.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.128+0000 7fea9632e700 1 --2- 172.21.15.16:0/3288275051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea9009d6f0 0x7fea901305f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:32:57.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.128+0000 7fea96b2f700 1 --2- 172.21.15.16:0/3288275051 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea9009e410 0x7fea90136250 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fea88003f50 tx=0x7fea88004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:57.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.128+0000 7fea86ffd700 1 -- 172.21.15.16:0/3288275051 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea88004450 con 0x7fea9009e410 2023-09-30T22:32:57.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.128+0000 7fea86ffd700 1 -- 172.21.15.16:0/3288275051 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea880045b0 con 0x7fea9009e410 2023-09-30T22:32:57.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.128+0000 7fea86ffd700 1 -- 172.21.15.16:0/3288275051 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea88004960 con 0x7fea9009e410 2023-09-30T22:32:57.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.128+0000 7fea98592700 1 -- 172.21.15.16:0/3288275051 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea90139e10 con 0x7fea9009e410 2023-09-30T22:32:57.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.128+0000 7fea98592700 1 -- 172.21.15.16:0/3288275051 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea90139a90 con 0x7fea9009e410 2023-09-30T22:32:57.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.129+0000 7fea98592700 1 -- 172.21.15.16:0/3288275051 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea901310a0 con 0x7fea9009e410 2023-09-30T22:32:57.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.130+0000 7fea86ffd700 1 -- 172.21.15.16:0/3288275051 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fea8801c410 con 0x7fea9009e410 2023-09-30T22:32:57.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.132+0000 7fa6a7fff700 1 -- 172.21.15.16:0/3665891547 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa6b0003e60 con 0x7fa6c009d6f0 2023-09-30T22:32:57.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.136+0000 7fea86ffd700 1 -- 172.21.15.16:0/3288275051 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fea8801ce20 con 0x7fea9009e410 2023-09-30T22:32:57.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.139+0000 7fea86ffd700 1 -- 172.21.15.16:0/3288275051 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fea88024400 con 0x7fea9009e410 2023-09-30T22:32:57.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.277+0000 7fb76ccb3700 1 -- 172.21.15.16:0/659227324 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb74c005590 con 0x7fb768135e30 2023-09-30T22:32:57.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.277+0000 7fb756ffd700 1 -- 172.21.15.16:0/659227324 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb760011390 con 0x7fb768135e30 2023-09-30T22:32:57.278 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:32:57.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.280+0000 7fb76ccb3700 1 -- 172.21.15.16:0/659227324 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb768135e30 msgr2=0x7fb768139760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:57.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.280+0000 7fb76ccb3700 1 --2- 172.21.15.16:0/659227324 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb768135e30 0x7fb768139760 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb768071280 tx=0x7fb760004ad0 comp rx=0 tx=0).stop 2023-09-30T22:32:57.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.280+0000 7fb76ccb3700 1 -- 172.21.15.16:0/659227324 shutdown_connections 2023-09-30T22:32:57.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.280+0000 7fb76ccb3700 1 --2- 172.21.15.16:0/659227324 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb768135e30 0x7fb768139760 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:57.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.280+0000 7fb76ccb3700 1 --2- 172.21.15.16:0/659227324 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb768130670 0x7fb76812d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:57.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.280+0000 7fb76ccb3700 1 --2- 172.21.15.16:0/659227324 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb768133510 0x7fb76812d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:57.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.280+0000 7fb76ccb3700 1 -- 172.21.15.16:0/659227324 >> 172.21.15.16:0/659227324 conn(0x7fb7680a2aa0 msgr2=0x7fb76812e130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:57.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.280+0000 7fb76ccb3700 1 -- 172.21.15.16:0/659227324 shutdown_connections 2023-09-30T22:32:57.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.280+0000 7fb76ccb3700 1 -- 172.21.15.16:0/659227324 wait complete. 2023-09-30T22:32:57.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.293+0000 7f592ff21700 1 -- 172.21.15.16:0/2430626850 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f5910005590 con 0x7f592809e000 2023-09-30T22:32:57.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.293+0000 7f591e7fc700 1 -- 172.21.15.16:0/2430626850 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5918017070 con 0x7f592809e000 2023-09-30T22:32:57.294 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:32:57.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.295+0000 7f592ff21700 1 -- 172.21.15.16:0/2430626850 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f592809e000 msgr2=0x7f59281a1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:57.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.295+0000 7f592ff21700 1 --2- 172.21.15.16:0/2430626850 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f592809e000 0x7f59281a1fb0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f591800ebe0 tx=0x7f591800eef0 comp rx=0 tx=0).stop 2023-09-30T22:32:57.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.295+0000 7f592ff21700 1 -- 172.21.15.16:0/2430626850 shutdown_connections 2023-09-30T22:32:57.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.295+0000 7f592ff21700 1 --2- 172.21.15.16:0/2430626850 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f592809e410 0x7f59281a24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:57.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.295+0000 7f592ff21700 1 --2- 172.21.15.16:0/2430626850 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f592809d6f0 0x7f59281a1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:57.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.295+0000 7f592ff21700 1 --2- 172.21.15.16:0/2430626850 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f592809e000 0x7f59281a1fb0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:57.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.295+0000 7f592ff21700 1 -- 172.21.15.16:0/2430626850 >> 172.21.15.16:0/2430626850 conn(0x7f5928066f50 msgr2=0x7f59281b6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:57.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.296+0000 7f592ff21700 1 -- 172.21.15.16:0/2430626850 shutdown_connections 2023-09-30T22:32:57.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.296+0000 7f592ff21700 1 -- 172.21.15.16:0/2430626850 wait complete. 2023-09-30T22:32:57.299 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:32:57.310 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:32:57.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.339+0000 7fa6c5aae700 1 -- 172.21.15.16:0/3665891547 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa6c009c710 con 0x7fa6c009d6f0 2023-09-30T22:32:57.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.340+0000 7fa6a7fff700 1 -- 172.21.15.16:0/3665891547 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa6b0015020 con 0x7fa6c009d6f0 2023-09-30T22:32:57.340 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:32:57.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.342+0000 7fa6c5aae700 1 -- 172.21.15.16:0/3665891547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6c009d6f0 msgr2=0x7fa6c01a5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:57.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.342+0000 7fa6c5aae700 1 --2- 172.21.15.16:0/3665891547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6c009d6f0 0x7fa6c01a5de0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fa6c006e470 tx=0x7fa6b0004750 comp rx=0 tx=0).stop 2023-09-30T22:32:57.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.342+0000 7fa6c5aae700 1 -- 172.21.15.16:0/3665891547 shutdown_connections 2023-09-30T22:32:57.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.342+0000 7fa6c5aae700 1 --2- 172.21.15.16:0/3665891547 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa6c009e410 0x7fa6c01a6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:57.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.342+0000 7fa6c5aae700 1 --2- 172.21.15.16:0/3665891547 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6c009d6f0 0x7fa6c01a5de0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:57.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.342+0000 7fa6c5aae700 1 --2- 172.21.15.16:0/3665891547 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa6c009e000 0x7fa6c01a6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:57.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.342+0000 7fa6c5aae700 1 -- 172.21.15.16:0/3665891547 >> 172.21.15.16:0/3665891547 conn(0x7fa6c0066f50 msgr2=0x7fa6c01bad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:57.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.342+0000 7fa6c5aae700 1 -- 172.21.15.16:0/3665891547 shutdown_connections 2023-09-30T22:32:57.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.342+0000 7fa6c5aae700 1 -- 172.21.15.16:0/3665891547 wait complete. 2023-09-30T22:32:57.356 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:32:57.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.373+0000 7fea98592700 1 -- 172.21.15.16:0/3288275051 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fea90074ea0 con 0x7fea9009e410 2023-09-30T22:32:57.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.373+0000 7fea86ffd700 1 -- 172.21.15.16:0/3288275051 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fea8801b3a0 con 0x7fea9009e410 2023-09-30T22:32:57.374 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:32:57.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.375+0000 7fea98592700 1 -- 172.21.15.16:0/3288275051 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea9009e410 msgr2=0x7fea90136250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:57.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.375+0000 7fea98592700 1 --2- 172.21.15.16:0/3288275051 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea9009e410 0x7fea90136250 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fea88003f50 tx=0x7fea88004ad0 comp rx=0 tx=0).stop 2023-09-30T22:32:57.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.376+0000 7fea98592700 1 -- 172.21.15.16:0/3288275051 shutdown_connections 2023-09-30T22:32:57.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.376+0000 7fea98592700 1 --2- 172.21.15.16:0/3288275051 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea9009e410 0x7fea90136250 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:57.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.376+0000 7fea98592700 1 --2- 172.21.15.16:0/3288275051 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea9009d6f0 0x7fea901305f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:57.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.376+0000 7fea98592700 1 --2- 172.21.15.16:0/3288275051 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea9009e000 0x7fea90130b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:57.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.376+0000 7fea98592700 1 -- 172.21.15.16:0/3288275051 >> 172.21.15.16:0/3288275051 conn(0x7fea90066f50 msgr2=0x7fea900a21f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:57.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.376+0000 7fea98592700 1 -- 172.21.15.16:0/3288275051 shutdown_connections 2023-09-30T22:32:57.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:57.376+0000 7fea98592700 1 -- 172.21.15.16:0/3288275051 wait complete. 2023-09-30T22:32:57.389 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:32:58.300 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:32:58.310 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:32:58.357 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:32:58.390 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:32:58.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.454+0000 7ff835f3e700 1 Processor -- start 2023-09-30T22:32:58.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.454+0000 7ff835f3e700 1 -- start start 2023-09-30T22:32:58.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.455+0000 7ff835f3e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff83009e1d0 0x7ff830138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:58.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.455+0000 7ff835f3e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff830139170 0x7ff83013b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:58.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.455+0000 7ff835f3e700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff83009b120 con 0x7ff83009d6f0 2023-09-30T22:32:58.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.455+0000 7ff835f3e700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff83006b460 con 0x7ff83009e1d0 2023-09-30T22:32:58.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.455+0000 7ff835f3e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff83009a7e0 con 0x7ff830139170 2023-09-30T22:32:58.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.455+0000 7ff82f7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff83009d6f0 0x7ff83009dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:37908/0 (socket says 172.21.15.16:37908) 2023-09-30T22:32:58.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.455+0000 7ff82f7fe700 1 -- 172.21.15.16:0/1825279330 learned_addr learned my addr 172.21.15.16:0/1825279330 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:58.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.455+0000 7ff82ffff700 1 --2- 172.21.15.16:0/1825279330 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff830139170 0x7ff83013b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:58.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.455+0000 7ff82effd700 1 --2- 172.21.15.16:0/1825279330 >> v2:172.21.15.16:3300/0 conn(0x7ff83009e1d0 0x7ff830138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:58.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.455+0000 7ff82effd700 1 -- 172.21.15.16:0/1825279330 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff830139170 msgr2=0x7ff83013b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:58.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.455+0000 7ff82effd700 1 --2- 172.21.15.16:0/1825279330 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff830139170 0x7ff83013b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:58.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.455+0000 7ff82effd700 1 -- 172.21.15.16:0/1825279330 >> v1:172.21.15.16:6789/0 conn(0x7ff83009d6f0 legacy=0x7ff83009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:58.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.455+0000 7ff82effd700 1 -- 172.21.15.16:0/1825279330 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff83009a500 con 0x7ff83009e1d0 2023-09-30T22:32:58.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.455+0000 7ff82ffff700 1 --2- 172.21.15.16:0/1825279330 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff830139170 0x7ff83013b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:58.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.456+0000 7ff82effd700 1 --2- 172.21.15.16:0/1825279330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff83009e1d0 0x7ff830138c30 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7ff824009870 tx=0x7ff824009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5554fe712901b8f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:58.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.456+0000 7ff82e7fc700 1 -- 172.21.15.16:0/1825279330 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff824011830 con 0x7ff83009e1d0 2023-09-30T22:32:58.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.456+0000 7ff82e7fc700 1 -- 172.21.15.16:0/1825279330 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff824011990 con 0x7ff83009e1d0 2023-09-30T22:32:58.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.456+0000 7ff82e7fc700 1 -- 172.21.15.16:0/1825279330 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff824011d20 con 0x7ff83009e1d0 2023-09-30T22:32:58.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.457+0000 7ff835f3e700 1 -- 172.21.15.16:0/1825279330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff83009e1d0 msgr2=0x7ff830138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:58.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.457+0000 7ff835f3e700 1 --2- 172.21.15.16:0/1825279330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff83009e1d0 0x7ff830138c30 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7ff824009870 tx=0x7ff824009b80 comp rx=0 tx=0).stop 2023-09-30T22:32:58.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.457+0000 7ff835f3e700 1 -- 172.21.15.16:0/1825279330 shutdown_connections 2023-09-30T22:32:58.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.457+0000 7ff835f3e700 1 --2- 172.21.15.16:0/1825279330 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff830139170 0x7ff83013b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:58.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.457+0000 7ff835f3e700 1 --2- 172.21.15.16:0/1825279330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff83009e1d0 0x7ff830138c30 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:58.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.457+0000 7ff835f3e700 1 -- 172.21.15.16:0/1825279330 >> 172.21.15.16:0/1825279330 conn(0x7ff830066f50 msgr2=0x7ff83009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:58.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.457+0000 7ff835f3e700 1 -- 172.21.15.16:0/1825279330 shutdown_connections 2023-09-30T22:32:58.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.457+0000 7ff835f3e700 1 -- 172.21.15.16:0/1825279330 wait complete. 2023-09-30T22:32:58.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.458+0000 7ff835f3e700 1 Processor -- start 2023-09-30T22:32:58.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.459+0000 7ff835f3e700 1 -- start start 2023-09-30T22:32:58.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.459+0000 7ff835f3e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff83009d6f0 0x7ff83011f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:58.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.459+0000 7ff835f3e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff830139170 0x7ff83011f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:58.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.459+0000 7ff835f3e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff8301c83d0 0x7ff8301c8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:58.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.459+0000 7ff835f3e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff83011eee0 con 0x7ff830139170 2023-09-30T22:32:58.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.459+0000 7ff835f3e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff830120610 con 0x7ff83009d6f0 2023-09-30T22:32:58.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.459+0000 7ff835f3e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff830122600 con 0x7ff8301c83d0 2023-09-30T22:32:58.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.460+0000 7ff82effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff830139170 0x7ff83011f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:58.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.460+0000 7ff82effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff830139170 0x7ff83011f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37360/0 (socket says 172.21.15.16:37360) 2023-09-30T22:32:58.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.460+0000 7ff82effd700 1 -- 172.21.15.16:0/3617503115 learned_addr learned my addr 172.21.15.16:0/3617503115 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:58.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.460+0000 7ff82ffff700 1 --2- 172.21.15.16:0/3617503115 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff8301c83d0 0x7ff8301c8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:58.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.460+0000 7ff82effd700 1 -- 172.21.15.16:0/3617503115 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff8301c83d0 msgr2=0x7ff8301c8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:58.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.460+0000 7ff82effd700 1 --2- 172.21.15.16:0/3617503115 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff8301c83d0 0x7ff8301c8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:58.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.460+0000 7ff82effd700 1 -- 172.21.15.16:0/3617503115 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff83009d6f0 msgr2=0x7ff83011f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:58.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.460+0000 7ff82effd700 1 --2- 172.21.15.16:0/3617503115 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff83009d6f0 0x7ff83011f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:58.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.460+0000 7ff82effd700 1 -- 172.21.15.16:0/3617503115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff828009a70 con 0x7ff830139170 2023-09-30T22:32:58.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.460+0000 7ff82ffff700 1 --2- 172.21.15.16:0/3617503115 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff8301c83d0 0x7ff8301c8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:58.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.460+0000 7ff82effd700 1 --2- 172.21.15.16:0/3617503115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff830139170 0x7ff83011f810 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7ff824009840 tx=0x7ff824004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:58.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.461+0000 7ff817fff700 1 -- 172.21.15.16:0/3617503115 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8240036a0 con 0x7ff830139170 2023-09-30T22:32:58.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.461+0000 7ff817fff700 1 -- 172.21.15.16:0/3617503115 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff82401f070 con 0x7ff830139170 2023-09-30T22:32:58.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.461+0000 7ff835f3e700 1 -- 172.21.15.16:0/3617503115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8240094a0 con 0x7ff830139170 2023-09-30T22:32:58.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.461+0000 7ff817fff700 1 -- 172.21.15.16:0/3617503115 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff824003a20 con 0x7ff830139170 2023-09-30T22:32:58.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.461+0000 7ff835f3e700 1 -- 172.21.15.16:0/3617503115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff830121f80 con 0x7ff830139170 2023-09-30T22:32:58.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.462+0000 7ff817fff700 1 -- 172.21.15.16:0/3617503115 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff824003b80 con 0x7ff830139170 2023-09-30T22:32:58.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.462+0000 7ff835f3e700 1 -- 172.21.15.16:0/3617503115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8301212e0 con 0x7ff830139170 2023-09-30T22:32:58.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.462+0000 7ff817fff700 1 -- 172.21.15.16:0/3617503115 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7ff824021070 con 0x7ff830139170 2023-09-30T22:32:58.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.467+0000 7ff817fff700 1 -- 172.21.15.16:0/3617503115 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff824010d50 con 0x7ff830139170 2023-09-30T22:32:58.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.496+0000 7fdecfbef700 1 Processor -- start 2023-09-30T22:32:58.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.497+0000 7fdecfbef700 1 -- start start 2023-09-30T22:32:58.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.497+0000 7fdecfbef700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdec806f5d0 0x7fdec812ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:58.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.497+0000 7fdecfbef700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdec8135c20 0x7fdec8138000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:58.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.498+0000 7fdecfbef700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdec806fa40 con 0x7fdec812cf60 2023-09-30T22:32:58.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.498+0000 7fdecfbef700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdec812cd90 con 0x7fdec806f5d0 2023-09-30T22:32:58.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.498+0000 7fdecfbef700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdec8071780 con 0x7fdec8135c20 2023-09-30T22:32:58.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.498+0000 7fdecd18a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdec806f5d0 0x7fdec812ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:58.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.498+0000 7fdece18c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdec8135c20 0x7fdec8138000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:58.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.498+0000 7fdecd98b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fdec812cf60 0x7fdec812f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:37916/0 (socket says 172.21.15.16:37916) 2023-09-30T22:32:58.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.498+0000 7fdecd98b700 1 -- 172.21.15.16:0/2281031739 learned_addr learned my addr 172.21.15.16:0/2281031739 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:58.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.498+0000 7fdece18c700 1 -- 172.21.15.16:0/2281031739 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdec806f5d0 msgr2=0x7fdec812ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:58.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.498+0000 7fdece18c700 1 --2- 172.21.15.16:0/2281031739 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdec806f5d0 0x7fdec812ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:58.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.498+0000 7fdece18c700 1 -- 172.21.15.16:0/2281031739 >> v1:172.21.15.16:6789/0 conn(0x7fdec812cf60 legacy=0x7fdec812f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:58.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.498+0000 7fdecc989700 1 -- 172.21.15.16:0/2281031739 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 853014509 0 0) 0x7fdec806fa40 con 0x7fdec812cf60 2023-09-30T22:32:58.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.498+0000 7fdece18c700 1 -- 172.21.15.16:0/2281031739 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec8071a40 con 0x7fdec8135c20 2023-09-30T22:32:58.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.498+0000 7fdece18c700 1 --2- 172.21.15.16:0/2281031739 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdec8135c20 0x7fdec8138000 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fdec000f3a0 tx=0x7fdec000f6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cccbed3a0d5aa353 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:58.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.499+0000 7fdecc989700 1 -- 172.21.15.16:0/2281031739 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec0012070 con 0x7fdec8135c20 2023-09-30T22:32:58.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.499+0000 7fdecc989700 1 -- 172.21.15.16:0/2281031739 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdec000d0d0 con 0x7fdec8135c20 2023-09-30T22:32:58.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.499+0000 7fdecc989700 1 -- 172.21.15.16:0/2281031739 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec000fea0 con 0x7fdec8135c20 2023-09-30T22:32:58.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.499+0000 7fdecfbef700 1 -- 172.21.15.16:0/2281031739 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdec8135c20 msgr2=0x7fdec8138000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:58.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.499+0000 7fdecfbef700 1 --2- 172.21.15.16:0/2281031739 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdec8135c20 0x7fdec8138000 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fdec000f3a0 tx=0x7fdec000f6b0 comp rx=0 tx=0).stop 2023-09-30T22:32:58.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.499+0000 7fdecfbef700 1 -- 172.21.15.16:0/2281031739 shutdown_connections 2023-09-30T22:32:58.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.499+0000 7fdecfbef700 1 --2- 172.21.15.16:0/2281031739 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdec8135c20 0x7fdec8138000 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:58.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.499+0000 7fdecfbef700 1 --2- 172.21.15.16:0/2281031739 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdec806f5d0 0x7fdec812ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:58.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.499+0000 7fdecfbef700 1 -- 172.21.15.16:0/2281031739 >> 172.21.15.16:0/2281031739 conn(0x7fdec80a2aa0 msgr2=0x7fdec80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:58.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.500+0000 7fdecfbef700 1 -- 172.21.15.16:0/2281031739 shutdown_connections 2023-09-30T22:32:58.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.500+0000 7fdecfbef700 1 -- 172.21.15.16:0/2281031739 wait complete. 2023-09-30T22:32:58.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.501+0000 7fdecfbef700 1 Processor -- start 2023-09-30T22:32:58.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.501+0000 7fdecfbef700 1 -- start start 2023-09-30T22:32:58.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.501+0000 7fdecfbef700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdec806f5d0 0x7fdec81c3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:58.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.501+0000 7fdecfbef700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdec812cf60 0x7fdec81c43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:58.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.501+0000 7fdecfbef700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdec8135c20 0x7fdec81c48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:58.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.501+0000 7fdecfbef700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec80a1410 con 0x7fdec806f5d0 2023-09-30T22:32:58.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.501+0000 7fdecfbef700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdec8129fa0 con 0x7fdec8135c20 2023-09-30T22:32:58.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.501+0000 7fdecfbef700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdec8133280 con 0x7fdec812cf60 2023-09-30T22:32:58.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.501+0000 7fdece18c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdec8135c20 0x7fdec81c48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:58.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.501+0000 7fdece18c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdec8135c20 0x7fdec81c48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36450/0 (socket says 172.21.15.16:36450) 2023-09-30T22:32:58.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.501+0000 7fdece18c700 1 -- 172.21.15.16:0/3281652922 learned_addr learned my addr 172.21.15.16:0/3281652922 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:58.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.501+0000 7fdecd18a700 1 --2- 172.21.15.16:0/3281652922 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdec812cf60 0x7fdec81c43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:58.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.501+0000 7fdecd98b700 1 --2- 172.21.15.16:0/3281652922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdec806f5d0 0x7fdec81c3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:58.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.502+0000 7fdece18c700 1 -- 172.21.15.16:0/3281652922 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdec812cf60 msgr2=0x7fdec81c43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:58.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.502+0000 7fdece18c700 1 --2- 172.21.15.16:0/3281652922 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdec812cf60 0x7fdec81c43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:58.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.502+0000 7fdece18c700 1 -- 172.21.15.16:0/3281652922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdec806f5d0 msgr2=0x7fdec81c3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:58.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.502+0000 7fdece18c700 1 --2- 172.21.15.16:0/3281652922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdec806f5d0 0x7fdec81c3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:58.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.502+0000 7fdece18c700 1 -- 172.21.15.16:0/3281652922 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec000f050 con 0x7fdec8135c20 2023-09-30T22:32:58.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.502+0000 7fdecd18a700 1 --2- 172.21.15.16:0/3281652922 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdec812cf60 0x7fdec81c43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:32:58.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.502+0000 7fdece18c700 1 --2- 172.21.15.16:0/3281652922 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdec8135c20 0x7fdec81c48f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fdec8071910 tx=0x7fdec0004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:58.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.502+0000 7fdebe7fc700 1 -- 172.21.15.16:0/3281652922 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec000bb50 con 0x7fdec8135c20 2023-09-30T22:32:58.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.502+0000 7fdebe7fc700 1 -- 172.21.15.16:0/3281652922 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdec000bcb0 con 0x7fdec8135c20 2023-09-30T22:32:58.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.502+0000 7fdecfbef700 1 -- 172.21.15.16:0/3281652922 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdec8132150 con 0x7fdec8135c20 2023-09-30T22:32:58.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.502+0000 7fdebe7fc700 1 -- 172.21.15.16:0/3281652922 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec0004280 con 0x7fdec8135c20 2023-09-30T22:32:58.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.502+0000 7fdecfbef700 1 -- 172.21.15.16:0/3281652922 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdec8132630 con 0x7fdec8135c20 2023-09-30T22:32:58.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.503+0000 7fdebe7fc700 1 -- 172.21.15.16:0/3281652922 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdec00043e0 con 0x7fdec8135c20 2023-09-30T22:32:58.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.503+0000 7fdebe7fc700 1 -- 172.21.15.16:0/3281652922 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fdec0045600 con 0x7fdec8135c20 2023-09-30T22:32:58.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.504+0000 7fdecfbef700 1 -- 172.21.15.16:0/3281652922 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdec80a0cb0 con 0x7fdec8135c20 2023-09-30T22:32:58.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.509+0000 7fdebe7fc700 1 -- 172.21.15.16:0/3281652922 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdec0004690 con 0x7fdec8135c20 2023-09-30T22:32:58.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.550+0000 7face2115700 1 Processor -- start 2023-09-30T22:32:58.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.551+0000 7face2115700 1 -- start start 2023-09-30T22:32:58.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.551+0000 7face2115700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7facdc09d6f0 0x7facdc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:58.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.551+0000 7face2115700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7facdc09e000 0x7facdc130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:58.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.551+0000 7face2115700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7facdc09e410 0x7facdc130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:58.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.551+0000 7face2115700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facdc09f520 con 0x7facdc09e000 2023-09-30T22:32:58.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.551+0000 7face2115700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facdc09f130 con 0x7facdc09d6f0 2023-09-30T22:32:58.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.551+0000 7face2115700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7facdc09e930 con 0x7facdc09e410 2023-09-30T22:32:58.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.551+0000 7facdb7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7facdc09d6f0 0x7facdc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:58.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.551+0000 7facdaffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7facdc09e000 0x7facdc130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:58.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.551+0000 7facdbfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7facdc09e410 0x7facdc130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:58.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.551+0000 7facdb7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7facdc09d6f0 0x7facdc09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36458/0 (socket says 172.21.15.16:36458) 2023-09-30T22:32:58.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.551+0000 7facdaffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7facdc09e000 0x7facdc130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37378/0 (socket says 172.21.15.16:37378) 2023-09-30T22:32:58.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.551+0000 7facdb7fe700 1 -- 172.21.15.16:0/536447300 learned_addr learned my addr 172.21.15.16:0/536447300 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:58.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.552+0000 7facdaffd700 1 -- 172.21.15.16:0/536447300 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7facdc09e410 msgr2=0x7facdc130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:58.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.552+0000 7facdaffd700 1 --2- 172.21.15.16:0/536447300 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7facdc09e410 0x7facdc130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:58.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.552+0000 7facdaffd700 1 -- 172.21.15.16:0/536447300 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7facdc09d6f0 msgr2=0x7facdc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:58.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.552+0000 7facdaffd700 1 --2- 172.21.15.16:0/536447300 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7facdc09d6f0 0x7facdc09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:58.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.552+0000 7facdaffd700 1 -- 172.21.15.16:0/536447300 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facdc09b2d0 con 0x7facdc09e000 2023-09-30T22:32:58.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.552+0000 7facdbfff700 1 --2- 172.21.15.16:0/536447300 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7facdc09e410 0x7facdc130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:58.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.552+0000 7facdb7fe700 1 --2- 172.21.15.16:0/536447300 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7facdc09d6f0 0x7facdc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:58.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.552+0000 7facdaffd700 1 --2- 172.21.15.16:0/536447300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facdc09e000 0x7facdc130660 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7facd0009870 tx=0x7facd0009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adf70c76b4ccfd63 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:58.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.552+0000 7facda7fc700 1 -- 172.21.15.16:0/536447300 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd0010a90 con 0x7facdc09e000 2023-09-30T22:32:58.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.552+0000 7facda7fc700 1 -- 172.21.15.16:0/536447300 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facd0010bf0 con 0x7facdc09e000 2023-09-30T22:32:58.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.552+0000 7facda7fc700 1 -- 172.21.15.16:0/536447300 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd0018440 con 0x7facdc09e000 2023-09-30T22:32:58.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.553+0000 7face2115700 1 -- 172.21.15.16:0/536447300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facdc09e000 msgr2=0x7facdc130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:58.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.553+0000 7face2115700 1 --2- 172.21.15.16:0/536447300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facdc09e000 0x7facdc130660 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7facd0009870 tx=0x7facd0009b80 comp rx=0 tx=0).stop 2023-09-30T22:32:58.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.553+0000 7face2115700 1 -- 172.21.15.16:0/536447300 shutdown_connections 2023-09-30T22:32:58.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.553+0000 7face2115700 1 --2- 172.21.15.16:0/536447300 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7facdc09d6f0 0x7facdc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:58.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.553+0000 7face2115700 1 --2- 172.21.15.16:0/536447300 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7facdc09e410 0x7facdc130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:58.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.553+0000 7face2115700 1 --2- 172.21.15.16:0/536447300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facdc09e000 0x7facdc130660 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:58.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.553+0000 7face2115700 1 -- 172.21.15.16:0/536447300 >> 172.21.15.16:0/536447300 conn(0x7facdc066f50 msgr2=0x7facdc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:58.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.553+0000 7face2115700 1 -- 172.21.15.16:0/536447300 shutdown_connections 2023-09-30T22:32:58.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.553+0000 7face2115700 1 -- 172.21.15.16:0/536447300 wait complete. 2023-09-30T22:32:58.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.554+0000 7face2115700 1 Processor -- start 2023-09-30T22:32:58.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.554+0000 7face2115700 1 -- start start 2023-09-30T22:32:58.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.555+0000 7face2115700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7facdc09d6f0 0x7facdc11e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:58.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.555+0000 7face2115700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7facdc09e000 0x7facdc11e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:58.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.555+0000 7face2115700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facdc09e410 0x7facdc11eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:58.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.555+0000 7face2115700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facdc1202e0 con 0x7facdc09e410 2023-09-30T22:32:58.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.555+0000 7face2115700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facdc0a1b60 con 0x7facdc09d6f0 2023-09-30T22:32:58.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.555+0000 7face2115700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7facdc09c700 con 0x7facdc09e000 2023-09-30T22:32:58.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.555+0000 7facdbfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facdc09e410 0x7facdc11eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:58.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.555+0000 7facdbfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facdc09e410 0x7facdc11eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37392/0 (socket says 172.21.15.16:37392) 2023-09-30T22:32:58.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.555+0000 7facdbfff700 1 -- 172.21.15.16:0/2388523831 learned_addr learned my addr 172.21.15.16:0/2388523831 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:58.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.555+0000 7facdb7fe700 1 --2- 172.21.15.16:0/2388523831 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7facdc09d6f0 0x7facdc11e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:58.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.555+0000 7facdaffd700 1 --2- 172.21.15.16:0/2388523831 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7facdc09e000 0x7facdc11e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:58.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.555+0000 7facdbfff700 1 -- 172.21.15.16:0/2388523831 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7facdc09e000 msgr2=0x7facdc11e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:58.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.555+0000 7facdbfff700 1 --2- 172.21.15.16:0/2388523831 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7facdc09e000 0x7facdc11e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:58.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.555+0000 7facdbfff700 1 -- 172.21.15.16:0/2388523831 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7facdc09d6f0 msgr2=0x7facdc11e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:58.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.555+0000 7facdbfff700 1 --2- 172.21.15.16:0/2388523831 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7facdc09d6f0 0x7facdc11e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:58.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.555+0000 7facdbfff700 1 -- 172.21.15.16:0/2388523831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faccc009550 con 0x7facdc09e410 2023-09-30T22:32:58.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.556+0000 7facdbfff700 1 --2- 172.21.15.16:0/2388523831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facdc09e410 0x7facdc11eba0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7facd4007fc0 tx=0x7facd400e3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:58.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.556+0000 7facc3fff700 1 -- 172.21.15.16:0/2388523831 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd4018070 con 0x7facdc09e410 2023-09-30T22:32:58.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.556+0000 7facc3fff700 1 -- 172.21.15.16:0/2388523831 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facd400b2e0 con 0x7facdc09e410 2023-09-30T22:32:58.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.556+0000 7facc3fff700 1 -- 172.21.15.16:0/2388523831 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd4010040 con 0x7facdc09e410 2023-09-30T22:32:58.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.556+0000 7face2115700 1 -- 172.21.15.16:0/2388523831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facd400bc80 con 0x7facdc09e410 2023-09-30T22:32:58.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.556+0000 7face2115700 1 -- 172.21.15.16:0/2388523831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facd00094a0 con 0x7facdc09e410 2023-09-30T22:32:58.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.557+0000 7facc3fff700 1 -- 172.21.15.16:0/2388523831 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7facd4003c20 con 0x7facdc09e410 2023-09-30T22:32:58.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.557+0000 7facc3fff700 1 -- 172.21.15.16:0/2388523831 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7facd403f810 con 0x7facdc09e410 2023-09-30T22:32:58.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.557+0000 7face2115700 1 -- 172.21.15.16:0/2388523831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facdc1215b0 con 0x7facdc09e410 2023-09-30T22:32:58.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.562+0000 7facc3fff700 1 -- 172.21.15.16:0/2388523831 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7facd400b450 con 0x7facdc09e410 2023-09-30T22:32:58.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.575+0000 7eff2455f700 1 Processor -- start 2023-09-30T22:32:58.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.575+0000 7eff2455f700 1 -- start start 2023-09-30T22:32:58.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.576+0000 7eff2455f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7eff1c09d6f0 0x7eff1c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:58.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.576+0000 7eff2455f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7eff1c09e000 0x7eff1c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:58.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.576+0000 7eff2455f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7eff1c09e410 0x7eff1c133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:58.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.576+0000 7eff2455f700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff1c06b460 con 0x7eff1c09d6f0 2023-09-30T22:32:58.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.576+0000 7eff2455f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff1c09b120 con 0x7eff1c09e000 2023-09-30T22:32:58.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.576+0000 7eff2455f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7eff1c09a7e0 con 0x7eff1c09e410 2023-09-30T22:32:58.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.576+0000 7eff222fb700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7eff1c09d6f0 0x7eff1c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:58.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.576+0000 7eff21afa700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7eff1c09e000 0x7eff1c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:58.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.576+0000 7eff21afa700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7eff1c09e000 0x7eff1c1339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36470/0 (socket says 172.21.15.16:36470) 2023-09-30T22:32:58.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.576+0000 7eff21afa700 1 -- 172.21.15.16:0/556634153 learned_addr learned my addr 172.21.15.16:0/556634153 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:58.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.576+0000 7eff222fb700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7eff1c09d6f0 0x7eff1c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37396/0 (socket says 172.21.15.16:37396) 2023-09-30T22:32:58.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.576+0000 7eff22afc700 1 --2- 172.21.15.16:0/556634153 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7eff1c09e410 0x7eff1c133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:58.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.576+0000 7eff21afa700 1 -- 172.21.15.16:0/556634153 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7eff1c09e410 msgr2=0x7eff1c133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:58.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.576+0000 7eff21afa700 1 --2- 172.21.15.16:0/556634153 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7eff1c09e410 0x7eff1c133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:58.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.576+0000 7eff21afa700 1 -- 172.21.15.16:0/556634153 >> v2:172.21.15.16:3300/0 conn(0x7eff1c09d6f0 msgr2=0x7eff1c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:58.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.576+0000 7eff21afa700 1 --2- 172.21.15.16:0/556634153 >> v2:172.21.15.16:3300/0 conn(0x7eff1c09d6f0 0x7eff1c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:58.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.576+0000 7eff21afa700 1 -- 172.21.15.16:0/556634153 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff1c09a500 con 0x7eff1c09e000 2023-09-30T22:32:58.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.576+0000 7eff21afa700 1 --2- 172.21.15.16:0/556634153 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7eff1c09e000 0x7eff1c1339f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7eff0c009920 tx=0x7eff0c009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b9516fb43dcbbbd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:58.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.577+0000 7eff212f9700 1 -- 172.21.15.16:0/556634153 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0c014e90 con 0x7eff1c09e000 2023-09-30T22:32:58.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.577+0000 7eff212f9700 1 -- 172.21.15.16:0/556634153 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff0c01c440 con 0x7eff1c09e000 2023-09-30T22:32:58.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.577+0000 7eff212f9700 1 -- 172.21.15.16:0/556634153 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0c01c7d0 con 0x7eff1c09e000 2023-09-30T22:32:58.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.577+0000 7eff2455f700 1 -- 172.21.15.16:0/556634153 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7eff1c09e000 msgr2=0x7eff1c1339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:58.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.577+0000 7eff2455f700 1 --2- 172.21.15.16:0/556634153 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7eff1c09e000 0x7eff1c1339f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7eff0c009920 tx=0x7eff0c009c30 comp rx=0 tx=0).stop 2023-09-30T22:32:58.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.578+0000 7eff2455f700 1 -- 172.21.15.16:0/556634153 shutdown_connections 2023-09-30T22:32:58.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.578+0000 7eff2455f700 1 --2- 172.21.15.16:0/556634153 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7eff1c09e410 0x7eff1c133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:58.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.578+0000 7eff2455f700 1 --2- 172.21.15.16:0/556634153 >> v2:172.21.15.16:3300/0 conn(0x7eff1c09d6f0 0x7eff1c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:58.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.578+0000 7eff2455f700 1 --2- 172.21.15.16:0/556634153 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7eff1c09e000 0x7eff1c1339f0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:58.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.578+0000 7eff2455f700 1 -- 172.21.15.16:0/556634153 >> 172.21.15.16:0/556634153 conn(0x7eff1c066f50 msgr2=0x7eff1c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:58.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.578+0000 7eff2455f700 1 -- 172.21.15.16:0/556634153 shutdown_connections 2023-09-30T22:32:58.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.578+0000 7eff2455f700 1 -- 172.21.15.16:0/556634153 wait complete. 2023-09-30T22:32:58.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.579+0000 7eff2455f700 1 Processor -- start 2023-09-30T22:32:58.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.579+0000 7eff2455f700 1 -- start start 2023-09-30T22:32:58.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.579+0000 7eff2455f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7eff1c09d6f0 0x7eff1c1a0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:58.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.579+0000 7eff2455f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff1c09e000 0x7eff1c1a0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:58.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.579+0000 7eff2455f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7eff1c09e410 0x7eff1c1a1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:58.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.579+0000 7eff2455f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff1c1a1e40 con 0x7eff1c09e000 2023-09-30T22:32:58.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.579+0000 7eff2455f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff1c1a4e70 con 0x7eff1c09d6f0 2023-09-30T22:32:58.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.579+0000 7eff2455f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7eff1c1a2640 con 0x7eff1c09e410 2023-09-30T22:32:58.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.579+0000 7eff22afc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7eff1c09e410 0x7eff1c1a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:58.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.580+0000 7eff22afc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7eff1c09e410 0x7eff1c1a1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37840/0 (socket says 172.21.15.16:37840) 2023-09-30T22:32:58.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.580+0000 7eff21afa700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff1c09e000 0x7eff1c1a0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:58.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.580+0000 7eff22afc700 1 -- 172.21.15.16:0/1550218027 learned_addr learned my addr 172.21.15.16:0/1550218027 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:58.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.580+0000 7eff222fb700 1 --2- 172.21.15.16:0/1550218027 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7eff1c09d6f0 0x7eff1c1a0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:58.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.580+0000 7eff21afa700 1 -- 172.21.15.16:0/1550218027 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7eff1c09e410 msgr2=0x7eff1c1a1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:58.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.580+0000 7eff21afa700 1 --2- 172.21.15.16:0/1550218027 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7eff1c09e410 0x7eff1c1a1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:58.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.580+0000 7eff21afa700 1 -- 172.21.15.16:0/1550218027 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7eff1c09d6f0 msgr2=0x7eff1c1a0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:58.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.580+0000 7eff21afa700 1 --2- 172.21.15.16:0/1550218027 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7eff1c09d6f0 0x7eff1c1a0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:58.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.580+0000 7eff21afa700 1 -- 172.21.15.16:0/1550218027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff0c009550 con 0x7eff1c09e000 2023-09-30T22:32:58.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.580+0000 7eff21afa700 1 --2- 172.21.15.16:0/1550218027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff1c09e000 0x7eff1c1a0ed0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7eff1c06b640 tx=0x7eff0c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:58.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.580+0000 7eff12ffd700 1 -- 172.21.15.16:0/1550218027 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0c003970 con 0x7eff1c09e000 2023-09-30T22:32:58.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.580+0000 7eff12ffd700 1 -- 172.21.15.16:0/1550218027 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff0c003af0 con 0x7eff1c09e000 2023-09-30T22:32:58.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.580+0000 7eff12ffd700 1 -- 172.21.15.16:0/1550218027 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0c003ea0 con 0x7eff1c09e000 2023-09-30T22:32:58.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.581+0000 7eff2455f700 1 -- 172.21.15.16:0/1550218027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff1c0a2140 con 0x7eff1c09e000 2023-09-30T22:32:58.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.581+0000 7eff2455f700 1 -- 172.21.15.16:0/1550218027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff1c1a37f0 con 0x7eff1c09e000 2023-09-30T22:32:58.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.582+0000 7eff12ffd700 1 -- 172.21.15.16:0/1550218027 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7eff0c023070 con 0x7eff1c09e000 2023-09-30T22:32:58.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.582+0000 7eff12ffd700 1 -- 172.21.15.16:0/1550218027 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7eff0c048950 con 0x7eff1c09e000 2023-09-30T22:32:58.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.582+0000 7eff2455f700 1 -- 172.21.15.16:0/1550218027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff1c074ea0 con 0x7eff1c09e000 2023-09-30T22:32:58.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.587+0000 7eff12ffd700 1 -- 172.21.15.16:0/1550218027 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7eff0c02b080 con 0x7eff1c09e000 2023-09-30T22:32:58.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.754+0000 7fdecfbef700 1 -- 172.21.15.16:0/3281652922 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fdec81333c0 con 0x7fdec8135c20 2023-09-30T22:32:58.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.755+0000 7fdebe7fc700 1 -- 172.21.15.16:0/3281652922 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdec001f3a0 con 0x7fdec8135c20 2023-09-30T22:32:58.755 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:32:58.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.758+0000 7fdecfbef700 1 -- 172.21.15.16:0/3281652922 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdec8135c20 msgr2=0x7fdec81c48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:58.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.758+0000 7fdecfbef700 1 --2- 172.21.15.16:0/3281652922 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdec8135c20 0x7fdec81c48f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fdec8071910 tx=0x7fdec0004ad0 comp rx=0 tx=0).stop 2023-09-30T22:32:58.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.758+0000 7fdecfbef700 1 -- 172.21.15.16:0/3281652922 shutdown_connections 2023-09-30T22:32:58.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.758+0000 7fdecfbef700 1 --2- 172.21.15.16:0/3281652922 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdec8135c20 0x7fdec81c48f0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:58.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.759+0000 7fdecfbef700 1 --2- 172.21.15.16:0/3281652922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdec806f5d0 0x7fdec81c3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:58.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.759+0000 7fdecfbef700 1 --2- 172.21.15.16:0/3281652922 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdec812cf60 0x7fdec81c43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:58.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.759+0000 7fdecfbef700 1 -- 172.21.15.16:0/3281652922 >> 172.21.15.16:0/3281652922 conn(0x7fdec80a2aa0 msgr2=0x7fdec806e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:58.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.759+0000 7fdecfbef700 1 -- 172.21.15.16:0/3281652922 shutdown_connections 2023-09-30T22:32:58.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.759+0000 7fdecfbef700 1 -- 172.21.15.16:0/3281652922 wait complete. 2023-09-30T22:32:58.778 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:32:58.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.779+0000 7ff835f3e700 1 -- 172.21.15.16:0/3617503115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff83009f420 con 0x7ff830139170 2023-09-30T22:32:58.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.779+0000 7ff817fff700 1 -- 172.21.15.16:0/3617503115 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff824015020 con 0x7ff830139170 2023-09-30T22:32:58.779 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:32:58.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.781+0000 7ff835f3e700 1 -- 172.21.15.16:0/3617503115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff830139170 msgr2=0x7ff83011f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:58.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.781+0000 7ff835f3e700 1 --2- 172.21.15.16:0/3617503115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff830139170 0x7ff83011f810 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7ff824009840 tx=0x7ff824004750 comp rx=0 tx=0).stop 2023-09-30T22:32:58.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.782+0000 7ff835f3e700 1 -- 172.21.15.16:0/3617503115 shutdown_connections 2023-09-30T22:32:58.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.782+0000 7ff835f3e700 1 --2- 172.21.15.16:0/3617503115 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff83009d6f0 0x7ff83011f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:58.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.782+0000 7ff835f3e700 1 --2- 172.21.15.16:0/3617503115 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff8301c83d0 0x7ff8301c8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:58.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.782+0000 7ff835f3e700 1 --2- 172.21.15.16:0/3617503115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff830139170 0x7ff83011f810 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:58.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.782+0000 7ff835f3e700 1 -- 172.21.15.16:0/3617503115 >> 172.21.15.16:0/3617503115 conn(0x7ff830066f50 msgr2=0x7ff830138170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:58.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.782+0000 7ff835f3e700 1 -- 172.21.15.16:0/3617503115 shutdown_connections 2023-09-30T22:32:58.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.782+0000 7ff835f3e700 1 -- 172.21.15.16:0/3617503115 wait complete. 2023-09-30T22:32:58.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.788+0000 7face2115700 1 -- 172.21.15.16:0/2388523831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7facdc074ea0 con 0x7facdc09e410 2023-09-30T22:32:58.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.789+0000 7facc3fff700 1 -- 172.21.15.16:0/2388523831 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7facd402b090 con 0x7facdc09e410 2023-09-30T22:32:58.789 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:32:58.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.791+0000 7face2115700 1 -- 172.21.15.16:0/2388523831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facdc09e410 msgr2=0x7facdc11eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:58.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.791+0000 7face2115700 1 --2- 172.21.15.16:0/2388523831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facdc09e410 0x7facdc11eba0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7facd4007fc0 tx=0x7facd400e3a0 comp rx=0 tx=0).stop 2023-09-30T22:32:58.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.791+0000 7face2115700 1 -- 172.21.15.16:0/2388523831 shutdown_connections 2023-09-30T22:32:58.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.791+0000 7face2115700 1 --2- 172.21.15.16:0/2388523831 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7facdc09d6f0 0x7facdc11e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:58.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.791+0000 7face2115700 1 --2- 172.21.15.16:0/2388523831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7facdc09e410 0x7facdc11eba0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:58.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.791+0000 7face2115700 1 --2- 172.21.15.16:0/2388523831 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7facdc09e000 0x7facdc11e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:58.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.791+0000 7face2115700 1 -- 172.21.15.16:0/2388523831 >> 172.21.15.16:0/2388523831 conn(0x7facdc066f50 msgr2=0x7facdc06a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:58.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.791+0000 7face2115700 1 -- 172.21.15.16:0/2388523831 shutdown_connections 2023-09-30T22:32:58.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.791+0000 7face2115700 1 -- 172.21.15.16:0/2388523831 wait complete. 2023-09-30T22:32:58.797 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:32:58.806 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:32:58.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.856+0000 7eff2455f700 1 -- 172.21.15.16:0/1550218027 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7eff1c1a4130 con 0x7eff1c09e000 2023-09-30T22:32:58.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.856+0000 7eff12ffd700 1 -- 172.21.15.16:0/1550218027 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7eff0c0263b0 con 0x7eff1c09e000 2023-09-30T22:32:58.857 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:32:58.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.858+0000 7eff2455f700 1 -- 172.21.15.16:0/1550218027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff1c09e000 msgr2=0x7eff1c1a0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:58.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.858+0000 7eff2455f700 1 --2- 172.21.15.16:0/1550218027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff1c09e000 0x7eff1c1a0ed0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7eff1c06b640 tx=0x7eff0c004750 comp rx=0 tx=0).stop 2023-09-30T22:32:58.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.858+0000 7eff2455f700 1 -- 172.21.15.16:0/1550218027 shutdown_connections 2023-09-30T22:32:58.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.858+0000 7eff2455f700 1 --2- 172.21.15.16:0/1550218027 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7eff1c09d6f0 0x7eff1c1a0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:58.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.858+0000 7eff2455f700 1 --2- 172.21.15.16:0/1550218027 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7eff1c09e410 0x7eff1c1a1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:58.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.858+0000 7eff2455f700 1 --2- 172.21.15.16:0/1550218027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff1c09e000 0x7eff1c1a0ed0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:58.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.858+0000 7eff2455f700 1 -- 172.21.15.16:0/1550218027 >> 172.21.15.16:0/1550218027 conn(0x7eff1c066f50 msgr2=0x7eff1c1b6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:58.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.858+0000 7eff2455f700 1 -- 172.21.15.16:0/1550218027 shutdown_connections 2023-09-30T22:32:58.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:58.858+0000 7eff2455f700 1 -- 172.21.15.16:0/1550218027 wait complete. 2023-09-30T22:32:58.872 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:32:59.779 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:32:59.798 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:32:59.806 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:32:59.873 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:32:59.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.945+0000 7fcdb5620700 1 Processor -- start 2023-09-30T22:32:59.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.945+0000 7fcdb5620700 1 -- start start 2023-09-30T22:32:59.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.946+0000 7fcdb5620700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcdb0131810 0x7fcdb0131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:59.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.946+0000 7fcdb5620700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcdb009a580 0x7fcdb0139660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:59.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.946+0000 7fcdb5620700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdb01313c0 con 0x7fcdb0099a40 2023-09-30T22:32:59.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.946+0000 7fcdb5620700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdb009b580 con 0x7fcdb009a580 2023-09-30T22:32:59.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.946+0000 7fcdb5620700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb006e8c0 con 0x7fcdb0131810 2023-09-30T22:32:59.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.946+0000 7fcdae7fc700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fcdb0099a40 0x7fcdb0099e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:37930/0 (socket says 172.21.15.16:37930) 2023-09-30T22:32:59.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.946+0000 7fcdae7fc700 1 -- 172.21.15.16:0/3203779258 learned_addr learned my addr 172.21.15.16:0/3203779258 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:59.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.946+0000 7fcdaeffd700 1 --2- 172.21.15.16:0/3203779258 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcdb0131810 0x7fcdb0131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:59.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.946+0000 7fcdaf7fe700 1 --2- 172.21.15.16:0/3203779258 >> v2:172.21.15.16:3300/0 conn(0x7fcdb009a580 0x7fcdb0139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:59.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.946+0000 7fcdaeffd700 1 -- 172.21.15.16:0/3203779258 >> v2:172.21.15.16:3300/0 conn(0x7fcdb009a580 msgr2=0x7fcdb0139660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:59.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.946+0000 7fcdaeffd700 1 --2- 172.21.15.16:0/3203779258 >> v2:172.21.15.16:3300/0 conn(0x7fcdb009a580 0x7fcdb0139660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:59.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.946+0000 7fcdaeffd700 1 -- 172.21.15.16:0/3203779258 >> v1:172.21.15.16:6789/0 conn(0x7fcdb0099a40 legacy=0x7fcdb0099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:59.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.946+0000 7fcdaeffd700 1 -- 172.21.15.16:0/3203779258 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb0071d20 con 0x7fcdb0131810 2023-09-30T22:32:59.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.947+0000 7fcdaeffd700 1 --2- 172.21.15.16:0/3203779258 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcdb0131810 0x7fcdb0131be0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fcda0009870 tx=0x7fcda0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=648df8e5b769123a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:59.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.947+0000 7fcdadffb700 1 -- 172.21.15.16:0/3203779258 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda0011830 con 0x7fcdb0131810 2023-09-30T22:32:59.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.947+0000 7fcdadffb700 1 -- 172.21.15.16:0/3203779258 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcda0011990 con 0x7fcdb0131810 2023-09-30T22:32:59.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.947+0000 7fcdadffb700 1 -- 172.21.15.16:0/3203779258 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda0011d20 con 0x7fcdb0131810 2023-09-30T22:32:59.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.948+0000 7fcdb5620700 1 -- 172.21.15.16:0/3203779258 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcdb0131810 msgr2=0x7fcdb0131be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:59.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.948+0000 7fcdb5620700 1 --2- 172.21.15.16:0/3203779258 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcdb0131810 0x7fcdb0131be0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fcda0009870 tx=0x7fcda0009b80 comp rx=0 tx=0).stop 2023-09-30T22:32:59.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.948+0000 7fcdb5620700 1 -- 172.21.15.16:0/3203779258 shutdown_connections 2023-09-30T22:32:59.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.948+0000 7fcdb5620700 1 --2- 172.21.15.16:0/3203779258 >> v2:172.21.15.16:3300/0 conn(0x7fcdb009a580 0x7fcdb0139660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:59.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.948+0000 7fcdb5620700 1 --2- 172.21.15.16:0/3203779258 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcdb0131810 0x7fcdb0131be0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:59.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.948+0000 7fcdb5620700 1 -- 172.21.15.16:0/3203779258 >> 172.21.15.16:0/3203779258 conn(0x7fcdb00a2aa0 msgr2=0x7fcdb00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:59.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.948+0000 7fcdb5620700 1 -- 172.21.15.16:0/3203779258 shutdown_connections 2023-09-30T22:32:59.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.948+0000 7fcdb5620700 1 -- 172.21.15.16:0/3203779258 wait complete. 2023-09-30T22:32:59.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.950+0000 7fcdb5620700 1 Processor -- start 2023-09-30T22:32:59.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.950+0000 7fcdb5620700 1 -- start start 2023-09-30T22:32:59.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.951+0000 7fcdb5620700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcdb0099a40 0x7fcdb011b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:59.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.951+0000 7fcdb5620700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcdb009a580 0x7fcdb011bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:59.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.951+0000 7fcdb5620700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdb0131810 0x7fcdb011c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:59.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.951+0000 7fcdb5620700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb011f190 con 0x7fcdb0131810 2023-09-30T22:32:59.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.951+0000 7fcdb5620700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb011d320 con 0x7fcdb009a580 2023-09-30T22:32:59.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.951+0000 7fcdb5620700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb00a1c60 con 0x7fcdb0099a40 2023-09-30T22:32:59.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.951+0000 7fcdaeffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcdb0099a40 0x7fcdb011b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:59.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.951+0000 7fcdaeffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcdb0099a40 0x7fcdb011b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37864/0 (socket says 172.21.15.16:37864) 2023-09-30T22:32:59.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.951+0000 7fcdaeffd700 1 -- 172.21.15.16:0/4089624853 learned_addr learned my addr 172.21.15.16:0/4089624853 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:59.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.951+0000 7fcdaf7fe700 1 --2- 172.21.15.16:0/4089624853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdb0131810 0x7fcdb011c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:59.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.951+0000 7fcdaeffd700 1 -- 172.21.15.16:0/4089624853 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcdb009a580 msgr2=0x7fcdb011bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:59.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.951+0000 7fcdaeffd700 1 --2- 172.21.15.16:0/4089624853 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcdb009a580 0x7fcdb011bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:59.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.951+0000 7fcdaeffd700 1 -- 172.21.15.16:0/4089624853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdb0131810 msgr2=0x7fcdb011c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:59.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.951+0000 7fcdaeffd700 1 --2- 172.21.15.16:0/4089624853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdb0131810 0x7fcdb011c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:59.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.951+0000 7fcdaeffd700 1 -- 172.21.15.16:0/4089624853 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcda00094a0 con 0x7fcdb0099a40 2023-09-30T22:32:59.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.951+0000 7fcdaf7fe700 1 --2- 172.21.15.16:0/4089624853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdb0131810 0x7fcdb011c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:32:59.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.952+0000 7fcdaeffd700 1 --2- 172.21.15.16:0/4089624853 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcdb0099a40 0x7fcdb011b670 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fcda0000c00 tx=0x7fcda0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:59.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.952+0000 7fcd977fe700 1 -- 172.21.15.16:0/4089624853 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda00044c0 con 0x7fcdb0099a40 2023-09-30T22:32:59.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.952+0000 7fcd977fe700 1 -- 172.21.15.16:0/4089624853 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcda001f070 con 0x7fcdb0099a40 2023-09-30T22:32:59.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.952+0000 7fcdb5620700 1 -- 172.21.15.16:0/4089624853 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdb006eb00 con 0x7fcdb0099a40 2023-09-30T22:32:59.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.952+0000 7fcd977fe700 1 -- 172.21.15.16:0/4089624853 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda00038c0 con 0x7fcdb0099a40 2023-09-30T22:32:59.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.952+0000 7fcdb5620700 1 -- 172.21.15.16:0/4089624853 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdb011f690 con 0x7fcdb0099a40 2023-09-30T22:32:59.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.953+0000 7fcd977fe700 1 -- 172.21.15.16:0/4089624853 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcda0021030 con 0x7fcdb0099a40 2023-09-30T22:32:59.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.953+0000 7fcd977fe700 1 -- 172.21.15.16:0/4089624853 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fcda0040880 con 0x7fcdb0099a40 2023-09-30T22:32:59.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.954+0000 7fcdb5620700 1 -- 172.21.15.16:0/4089624853 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdb0074ea0 con 0x7fcdb0099a40 2023-09-30T22:32:59.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.963+0000 7fcd977fe700 1 -- 172.21.15.16:0/4089624853 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcda0015080 con 0x7fcdb0099a40 2023-09-30T22:32:59.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.977+0000 7fd71a990700 1 Processor -- start 2023-09-30T22:32:59.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.977+0000 7fd71a990700 1 -- start start 2023-09-30T22:32:59.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.978+0000 7fd71a990700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd71409e930 0x7fd71409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:59.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.978+0000 7fd71a990700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd71409f240 0x7fd7141317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:59.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.978+0000 7fd71a990700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd71409f650 0x7fd714131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:59.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.978+0000 7fd71a990700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd714070940 con 0x7fd71409f650 2023-09-30T22:32:59.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.978+0000 7fd71a990700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd71406fd60 con 0x7fd71409f240 2023-09-30T22:32:59.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.978+0000 7fd71a990700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd71406fa80 con 0x7fd71409e930 2023-09-30T22:32:59.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.978+0000 7fd713fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd71409e930 0x7fd71409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:59.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.978+0000 7fd7137fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd71409f240 0x7fd7141317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:59.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.978+0000 7fd713fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd71409e930 0x7fd71409ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37870/0 (socket says 172.21.15.16:37870) 2023-09-30T22:32:59.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.978+0000 7fd7137fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd71409f240 0x7fd7141317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36492/0 (socket says 172.21.15.16:36492) 2023-09-30T22:32:59.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.978+0000 7fd713fff700 1 -- 172.21.15.16:0/2121911459 learned_addr learned my addr 172.21.15.16:0/2121911459 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:59.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.978+0000 7fd718f2d700 1 --2- 172.21.15.16:0/2121911459 >> v2:172.21.15.16:3300/0 conn(0x7fd71409f650 0x7fd714131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:59.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.978+0000 7fd713fff700 1 -- 172.21.15.16:0/2121911459 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd71409f240 msgr2=0x7fd7141317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:59.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.978+0000 7fd713fff700 1 --2- 172.21.15.16:0/2121911459 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd71409f240 0x7fd7141317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:59.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.978+0000 7fd713fff700 1 -- 172.21.15.16:0/2121911459 >> v2:172.21.15.16:3300/0 conn(0x7fd71409f650 msgr2=0x7fd714131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:59.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.978+0000 7fd713fff700 1 --2- 172.21.15.16:0/2121911459 >> v2:172.21.15.16:3300/0 conn(0x7fd71409f650 0x7fd714131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:59.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.978+0000 7fd713fff700 1 -- 172.21.15.16:0/2121911459 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd71409fb60 con 0x7fd71409e930 2023-09-30T22:32:59.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.979+0000 7fd713fff700 1 --2- 172.21.15.16:0/2121911459 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd71409e930 0x7fd71409ed00 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd7040098f0 tx=0x7fd704009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62f6b1655817a3f3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:59.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.979+0000 7fd712ffd700 1 -- 172.21.15.16:0/2121911459 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd704011830 con 0x7fd71409e930 2023-09-30T22:32:59.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.979+0000 7fd712ffd700 1 -- 172.21.15.16:0/2121911459 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd704011990 con 0x7fd71409e930 2023-09-30T22:32:59.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.979+0000 7fd712ffd700 1 -- 172.21.15.16:0/2121911459 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd704011d20 con 0x7fd71409e930 2023-09-30T22:32:59.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.980+0000 7fd71a990700 1 -- 172.21.15.16:0/2121911459 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd71409e930 msgr2=0x7fd71409ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:59.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.980+0000 7fd71a990700 1 --2- 172.21.15.16:0/2121911459 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd71409e930 0x7fd71409ed00 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd7040098f0 tx=0x7fd704009c00 comp rx=0 tx=0).stop 2023-09-30T22:32:59.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.980+0000 7fd71a990700 1 -- 172.21.15.16:0/2121911459 shutdown_connections 2023-09-30T22:32:59.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.980+0000 7fd71a990700 1 --2- 172.21.15.16:0/2121911459 >> v2:172.21.15.16:3300/0 conn(0x7fd71409f650 0x7fd714131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:59.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.980+0000 7fd71a990700 1 --2- 172.21.15.16:0/2121911459 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd71409e930 0x7fd71409ed00 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:59.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.980+0000 7fd71a990700 1 --2- 172.21.15.16:0/2121911459 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd71409f240 0x7fd7141317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:59.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.980+0000 7fd71a990700 1 -- 172.21.15.16:0/2121911459 >> 172.21.15.16:0/2121911459 conn(0x7fd7140a4210 msgr2=0x7fd7140a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:59.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.980+0000 7fd71a990700 1 -- 172.21.15.16:0/2121911459 shutdown_connections 2023-09-30T22:32:59.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.980+0000 7fd71a990700 1 -- 172.21.15.16:0/2121911459 wait complete. 2023-09-30T22:32:59.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.981+0000 7fd71a990700 1 Processor -- start 2023-09-30T22:32:59.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.982+0000 7fd71a990700 1 -- start start 2023-09-30T22:32:59.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.982+0000 7fd7db1ee700 1 Processor -- start 2023-09-30T22:32:59.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.982+0000 7fd7db1ee700 1 -- start start 2023-09-30T22:32:59.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.982+0000 7fd71a990700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd71409e930 0x7fd71406b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:59.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.982+0000 7fd71a990700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd71409f240 0x7fd71406bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:59.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.982+0000 7fd71a990700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd71409f650 0x7fd71406c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:59.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.982+0000 7fd71a990700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd71406cd80 con 0x7fd71409e930 2023-09-30T22:32:59.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.982+0000 7fd71a990700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7141b71d0 con 0x7fd71409f240 2023-09-30T22:32:59.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.982+0000 7fd71a990700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd7141a9380 con 0x7fd71409f650 2023-09-30T22:32:59.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.982+0000 7fd7db1ee700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd7d4133540 0x7fd7d4135920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:59.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.983+0000 7fd7db1ee700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd7d4135e60 0x7fd7d4138240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:59.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.983+0000 7fd7db1ee700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7d40716a0 con 0x7fd7d4130540 2023-09-30T22:32:59.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.983+0000 7fd7db1ee700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7d4071290 con 0x7fd7d4133540 2023-09-30T22:32:59.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.983+0000 7fd7db1ee700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d4130340 con 0x7fd7d4135e60 2023-09-30T22:32:59.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.983+0000 7fd7137fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd71409f240 0x7fd71406bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:59.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.983+0000 7fd718f2d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd71409f650 0x7fd71406c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:59.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.983+0000 7fd7137fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd71409f240 0x7fd71406bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36494/0 (socket says 172.21.15.16:36494) 2023-09-30T22:32:59.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.983+0000 7fd718f2d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd71409f650 0x7fd71406c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37878/0 (socket says 172.21.15.16:37878) 2023-09-30T22:32:59.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.983+0000 7fd7137fe700 1 -- 172.21.15.16:0/4250129781 learned_addr learned my addr 172.21.15.16:0/4250129781 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:59.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.983+0000 7fd718f2d700 1 -- 172.21.15.16:0/4250129781 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd71409f240 msgr2=0x7fd71406bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:59.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.983+0000 7fd718f2d700 1 --2- 172.21.15.16:0/4250129781 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd71409f240 0x7fd71406bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:59.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.983+0000 7fd718f2d700 1 -- 172.21.15.16:0/4250129781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd71409e930 msgr2=0x7fd71406b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:59.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.983+0000 7fd718f2d700 1 --2- 172.21.15.16:0/4250129781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd71409e930 0x7fd71406b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:59.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.983+0000 7fd718f2d700 1 -- 172.21.15.16:0/4250129781 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd704009550 con 0x7fd71409f650 2023-09-30T22:32:59.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.983+0000 7fd718f2d700 1 --2- 172.21.15.16:0/4250129781 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd71409f650 0x7fd71406c330 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd70c00efc0 tx=0x7fd70c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:59.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.984+0000 7fd710ff9700 1 -- 172.21.15.16:0/4250129781 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70c00cd40 con 0x7fd71409f650 2023-09-30T22:32:59.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.984+0000 7fd710ff9700 1 -- 172.21.15.16:0/4250129781 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd70c00f040 con 0x7fd71409f650 2023-09-30T22:32:59.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.984+0000 7fd710ff9700 1 -- 172.21.15.16:0/4250129781 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70c0146c0 con 0x7fd71409f650 2023-09-30T22:32:59.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.984+0000 7fd71a990700 1 -- 172.21.15.16:0/4250129781 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7140a0190 con 0x7fd71409f650 2023-09-30T22:32:59.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.984+0000 7fd71a990700 1 -- 172.21.15.16:0/4250129781 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7141a90b0 con 0x7fd71409f650 2023-09-30T22:32:59.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.984+0000 7fd7d8f8a700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd7d4130540 0x7fd7d4130910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:37936/0 (socket says 172.21.15.16:37936) 2023-09-30T22:32:59.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.984+0000 7fd7d978b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd7d4135e60 0x7fd7d4138240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:59.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.984+0000 7fd7d8f8a700 1 -- 172.21.15.16:0/49117755 learned_addr learned my addr 172.21.15.16:0/49117755 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:32:59.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.984+0000 7fd7d3fff700 1 --2- 172.21.15.16:0/49117755 >> v2:172.21.15.16:3300/0 conn(0x7fd7d4133540 0x7fd7d4135920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:59.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.984+0000 7fd7d37fe700 1 -- 172.21.15.16:0/49117755 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3093311492 0 0) 0x7fd7d40716a0 con 0x7fd7d4130540 2023-09-30T22:32:59.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.984+0000 7fd7d37fe700 1 -- 172.21.15.16:0/49117755 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7c40057c0 con 0x7fd7d4130540 2023-09-30T22:32:59.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.985+0000 7fd7d978b700 1 -- 172.21.15.16:0/49117755 >> v2:172.21.15.16:3300/0 conn(0x7fd7d4133540 msgr2=0x7fd7d4135920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:32:59.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.985+0000 7fd7d978b700 1 --2- 172.21.15.16:0/49117755 >> v2:172.21.15.16:3300/0 conn(0x7fd7d4133540 0x7fd7d4135920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:59.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.985+0000 7fd7d978b700 1 -- 172.21.15.16:0/49117755 >> v1:172.21.15.16:6789/0 conn(0x7fd7d4130540 legacy=0x7fd7d4130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:59.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.985+0000 7fd7d978b700 1 -- 172.21.15.16:0/49117755 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7d4070b80 con 0x7fd7d4135e60 2023-09-30T22:32:59.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.985+0000 7fd7d978b700 1 --2- 172.21.15.16:0/49117755 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd7d4135e60 0x7fd7d4138240 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc009d90 tx=0x7fd7cc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d346225a75dda6dd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:59.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.985+0000 7fd7d37fe700 1 -- 172.21.15.16:0/49117755 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc00e070 con 0x7fd7d4135e60 2023-09-30T22:32:59.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.985+0000 7fd7d37fe700 1 -- 172.21.15.16:0/49117755 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7cc00bb20 con 0x7fd7d4135e60 2023-09-30T22:32:59.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.985+0000 7fd7d37fe700 1 -- 172.21.15.16:0/49117755 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc012430 con 0x7fd7d4135e60 2023-09-30T22:32:59.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.985+0000 7fd71a990700 1 -- 172.21.15.16:0/4250129781 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7140a3670 con 0x7fd71409f650 2023-09-30T22:32:59.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.986+0000 7fd7db1ee700 1 -- 172.21.15.16:0/49117755 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd7d4135e60 msgr2=0x7fd7d4138240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:59.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.986+0000 7fd7db1ee700 1 --2- 172.21.15.16:0/49117755 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd7d4135e60 0x7fd7d4138240 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc009d90 tx=0x7fd7cc00b330 comp rx=0 tx=0).stop 2023-09-30T22:32:59.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.986+0000 7fd7db1ee700 1 -- 172.21.15.16:0/49117755 shutdown_connections 2023-09-30T22:32:59.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.986+0000 7fd7db1ee700 1 --2- 172.21.15.16:0/49117755 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd7d4135e60 0x7fd7d4138240 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:59.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.986+0000 7fd7db1ee700 1 --2- 172.21.15.16:0/49117755 >> v2:172.21.15.16:3300/0 conn(0x7fd7d4133540 0x7fd7d4135920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:59.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.986+0000 7fd7db1ee700 1 -- 172.21.15.16:0/49117755 >> 172.21.15.16:0/49117755 conn(0x7fd7d40a2aa0 msgr2=0x7fd7d40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:32:59.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.986+0000 7fd710ff9700 1 -- 172.21.15.16:0/4250129781 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd70c014820 con 0x7fd71409f650 2023-09-30T22:32:59.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.986+0000 7fd7db1ee700 1 -- 172.21.15.16:0/49117755 shutdown_connections 2023-09-30T22:32:59.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.986+0000 7fd7db1ee700 1 -- 172.21.15.16:0/49117755 wait complete. 2023-09-30T22:32:59.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.986+0000 7fd710ff9700 1 -- 172.21.15.16:0/4250129781 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd70c040a30 con 0x7fd71409f650 2023-09-30T22:32:59.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.987+0000 7fd7db1ee700 1 Processor -- start 2023-09-30T22:32:59.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.987+0000 7fd7db1ee700 1 -- start start 2023-09-30T22:32:59.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.988+0000 7fd7db1ee700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd7d4130540 0x7fd7d409c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:59.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.988+0000 7fd7db1ee700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd7d4133540 0x7fd7d409c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:59.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.988+0000 7fd7db1ee700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7d4135e60 0x7fd7d409cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:32:59.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.988+0000 7fd7db1ee700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d409d860 con 0x7fd7d4135e60 2023-09-30T22:32:59.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.988+0000 7fd7db1ee700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d412ddb0 con 0x7fd7d4130540 2023-09-30T22:32:59.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.988+0000 7fd7db1ee700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d409df60 con 0x7fd7d4133540 2023-09-30T22:32:59.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.988+0000 7fd7d8f8a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd7d4130540 0x7fd7d409c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:59.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.988+0000 7fd7d8f8a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd7d4130540 0x7fd7d409c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36502/0 (socket says 172.21.15.16:36502) 2023-09-30T22:32:59.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.988+0000 7fd7d8f8a700 1 -- 172.21.15.16:0/2774170501 learned_addr learned my addr 172.21.15.16:0/2774170501 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:32:59.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.988+0000 7fd7d3fff700 1 --2- 172.21.15.16:0/2774170501 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd7d4133540 0x7fd7d409c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:32:59.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.989+0000 7fd7d8f8a700 1 -- 172.21.15.16:0/2774170501 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd7d4133540 msgr2=0x7fd7d409c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:59.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.989+0000 7fd7d8f8a700 1 --2- 172.21.15.16:0/2774170501 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd7d4133540 0x7fd7d409c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:59.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.989+0000 7fd7d8f8a700 1 -- 172.21.15.16:0/2774170501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7d4135e60 msgr2=0x7fd7d409cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:32:59.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.989+0000 7fd7d8f8a700 1 --2- 172.21.15.16:0/2774170501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7d4135e60 0x7fd7d409cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:32:59.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.989+0000 7fd7d8f8a700 1 -- 172.21.15.16:0/2774170501 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7cc009a70 con 0x7fd7d4130540 2023-09-30T22:32:59.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.989+0000 7fd7d8f8a700 1 --2- 172.21.15.16:0/2774170501 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd7d4130540 0x7fd7d409c420 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8001eb0 tx=0x7fd7c800c7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:32:59.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.989+0000 7fd7d17fa700 1 -- 172.21.15.16:0/2774170501 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c8005d70 con 0x7fd7d4130540 2023-09-30T22:32:59.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.989+0000 7fd7d17fa700 1 -- 172.21.15.16:0/2774170501 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7c80070d0 con 0x7fd7d4130540 2023-09-30T22:32:59.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.989+0000 7fd7db1ee700 1 -- 172.21.15.16:0/2774170501 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7d40718e0 con 0x7fd7d4130540 2023-09-30T22:32:59.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.989+0000 7fd7d17fa700 1 -- 172.21.15.16:0/2774170501 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c8007460 con 0x7fd7d4130540 2023-09-30T22:32:59.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.989+0000 7fd7db1ee700 1 -- 172.21.15.16:0/2774170501 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7d411aca0 con 0x7fd7d4130540 2023-09-30T22:32:59.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.990+0000 7fd7db1ee700 1 -- 172.21.15.16:0/2774170501 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7d4074ea0 con 0x7fd7d4130540 2023-09-30T22:32:59.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.991+0000 7fd7d17fa700 1 -- 172.21.15.16:0/2774170501 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd7c80075c0 con 0x7fd7d4130540 2023-09-30T22:32:59.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.991+0000 7fd7d17fa700 1 -- 172.21.15.16:0/2774170501 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd7c803ebe0 con 0x7fd7d4130540 2023-09-30T22:32:59.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:32:59.995+0000 7fd710ff9700 1 -- 172.21.15.16:0/4250129781 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd70c012930 con 0x7fd71409f650 2023-09-30T22:33:00.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.000+0000 7fd7d17fa700 1 -- 172.21.15.16:0/2774170501 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd7c8019c90 con 0x7fd7d4130540 2023-09-30T22:33:00.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.034+0000 7fd83d589700 1 Processor -- start 2023-09-30T22:33:00.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.034+0000 7fd83d589700 1 -- start start 2023-09-30T22:33:00.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.035+0000 7fd83d589700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd83809e1d0 0x7fd838135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:00.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.035+0000 7fd83d589700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd838135e50 0x7fd838138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:00.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.035+0000 7fd83d589700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd83809f130 con 0x7fd83809d6f0 2023-09-30T22:33:00.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.035+0000 7fd83d589700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd83809f520 con 0x7fd838135e50 2023-09-30T22:33:00.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.035+0000 7fd83d589700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd83809e930 con 0x7fd83809e1d0 2023-09-30T22:33:00.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.035+0000 7fd8367fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd83809e1d0 0x7fd838135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:00.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.035+0000 7fd8367fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd83809e1d0 0x7fd838135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37918/0 (socket says 172.21.15.16:37918) 2023-09-30T22:33:00.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.035+0000 7fd836ffd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd83809d6f0 0x7fd83809dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:37938/0 (socket says 172.21.15.16:37938) 2023-09-30T22:33:00.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.035+0000 7fd8377fe700 1 --2- 172.21.15.16:0/3979755525 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd838135e50 0x7fd838138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:00.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.035+0000 7fd8367fc700 1 -- 172.21.15.16:0/3979755525 learned_addr learned my addr 172.21.15.16:0/3979755525 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:00.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.036+0000 7fd8377fe700 1 -- 172.21.15.16:0/3979755525 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd83809e1d0 msgr2=0x7fd838135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:00.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.036+0000 7fd8377fe700 1 --2- 172.21.15.16:0/3979755525 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd83809e1d0 0x7fd838135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:00.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.036+0000 7fd8377fe700 1 -- 172.21.15.16:0/3979755525 >> v1:172.21.15.16:6789/0 conn(0x7fd83809d6f0 legacy=0x7fd83809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:00.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.036+0000 7fd8377fe700 1 -- 172.21.15.16:0/3979755525 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd83809b2d0 con 0x7fd838135e50 2023-09-30T22:33:00.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.036+0000 7fd835ffb700 1 -- 172.21.15.16:0/3979755525 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 537713317 0 0) 0x7fd83809f130 con 0x7fd83809d6f0 2023-09-30T22:33:00.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.036+0000 7fd8377fe700 1 --2- 172.21.15.16:0/3979755525 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd838135e50 0x7fd838138230 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd830009dc0 tx=0x7fd83000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db57e7a162f40aaa server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:00.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.036+0000 7fd835ffb700 1 -- 172.21.15.16:0/3979755525 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83000e070 con 0x7fd838135e50 2023-09-30T22:33:00.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.036+0000 7fd835ffb700 1 -- 172.21.15.16:0/3979755525 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd83000bb20 con 0x7fd838135e50 2023-09-30T22:33:00.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.036+0000 7fd835ffb700 1 -- 172.21.15.16:0/3979755525 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd830012430 con 0x7fd838135e50 2023-09-30T22:33:00.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.037+0000 7fd83d589700 1 -- 172.21.15.16:0/3979755525 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd838135e50 msgr2=0x7fd838138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:00.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.037+0000 7fd83d589700 1 --2- 172.21.15.16:0/3979755525 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd838135e50 0x7fd838138230 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd830009dc0 tx=0x7fd83000b330 comp rx=0 tx=0).stop 2023-09-30T22:33:00.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.037+0000 7fd83d589700 1 -- 172.21.15.16:0/3979755525 shutdown_connections 2023-09-30T22:33:00.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.037+0000 7fd83d589700 1 --2- 172.21.15.16:0/3979755525 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd838135e50 0x7fd838138230 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:00.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.037+0000 7fd83d589700 1 --2- 172.21.15.16:0/3979755525 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd83809e1d0 0x7fd838135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:00.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.037+0000 7fd83d589700 1 -- 172.21.15.16:0/3979755525 >> 172.21.15.16:0/3979755525 conn(0x7fd838066f50 msgr2=0x7fd83809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:00.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.037+0000 7fd83d589700 1 -- 172.21.15.16:0/3979755525 shutdown_connections 2023-09-30T22:33:00.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.037+0000 7fd83d589700 1 -- 172.21.15.16:0/3979755525 wait complete. 2023-09-30T22:33:00.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.039+0000 7fd83d589700 1 Processor -- start 2023-09-30T22:33:00.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.039+0000 7fd83d589700 1 -- start start 2023-09-30T22:33:00.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.040+0000 7fd83d589700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd83809d6f0 0x7fd83811b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:00.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.040+0000 7fd83d589700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd83809e1d0 0x7fd83811b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:00.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.040+0000 7fd83d589700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd838135e50 0x7fd83811bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:00.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.040+0000 7fd83d589700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd83811e750 con 0x7fd838135e50 2023-09-30T22:33:00.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.040+0000 7fd83d589700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd83811cf60 con 0x7fd83809d6f0 2023-09-30T22:33:00.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.040+0000 7fd83d589700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd83809c700 con 0x7fd83809e1d0 2023-09-30T22:33:00.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.040+0000 7fd836ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd83809d6f0 0x7fd83811b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:00.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.040+0000 7fd836ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd83809d6f0 0x7fd83811b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36528/0 (socket says 172.21.15.16:36528) 2023-09-30T22:33:00.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.040+0000 7fd836ffd700 1 -- 172.21.15.16:0/3560151538 learned_addr learned my addr 172.21.15.16:0/3560151538 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:00.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.040+0000 7fd8377fe700 1 --2- 172.21.15.16:0/3560151538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd838135e50 0x7fd83811bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:00.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.040+0000 7fd8367fc700 1 --2- 172.21.15.16:0/3560151538 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd83809e1d0 0x7fd83811b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:00.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.041+0000 7fd836ffd700 1 -- 172.21.15.16:0/3560151538 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd83809e1d0 msgr2=0x7fd83811b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:00.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.041+0000 7fd836ffd700 1 --2- 172.21.15.16:0/3560151538 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd83809e1d0 0x7fd83811b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:00.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.041+0000 7fd836ffd700 1 -- 172.21.15.16:0/3560151538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd838135e50 msgr2=0x7fd83811bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:00.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.041+0000 7fd836ffd700 1 --2- 172.21.15.16:0/3560151538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd838135e50 0x7fd83811bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:00.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.041+0000 7fd836ffd700 1 -- 172.21.15.16:0/3560151538 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd830009a70 con 0x7fd83809d6f0 2023-09-30T22:33:00.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.041+0000 7fd836ffd700 1 --2- 172.21.15.16:0/3560151538 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd83809d6f0 0x7fd83811b1d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fd82800c4a0 tx=0x7fd82800c7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:00.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.041+0000 7fd81f7fe700 1 -- 172.21.15.16:0/3560151538 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd828015d70 con 0x7fd83809d6f0 2023-09-30T22:33:00.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.041+0000 7fd81f7fe700 1 -- 172.21.15.16:0/3560151538 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd828019070 con 0x7fd83809d6f0 2023-09-30T22:33:00.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.041+0000 7fd83d589700 1 -- 172.21.15.16:0/3560151538 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd83811ecf0 con 0x7fd83809d6f0 2023-09-30T22:33:00.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.041+0000 7fd81f7fe700 1 -- 172.21.15.16:0/3560151538 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd828004a90 con 0x7fd83809d6f0 2023-09-30T22:33:00.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.041+0000 7fd83d589700 1 -- 172.21.15.16:0/3560151538 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd83811e080 con 0x7fd83809d6f0 2023-09-30T22:33:00.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.042+0000 7fd81f7fe700 1 -- 172.21.15.16:0/3560151538 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd828004bf0 con 0x7fd83809d6f0 2023-09-30T22:33:00.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.042+0000 7fd83d589700 1 -- 172.21.15.16:0/3560151538 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd83811e480 con 0x7fd83809d6f0 2023-09-30T22:33:00.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.043+0000 7fd81f7fe700 1 -- 172.21.15.16:0/3560151538 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd8280471a0 con 0x7fd83809d6f0 2023-09-30T22:33:00.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.052+0000 7fd81f7fe700 1 -- 172.21.15.16:0/3560151538 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd82801b830 con 0x7fd83809d6f0 2023-09-30T22:33:00.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.187+0000 7fcdb5620700 1 -- 172.21.15.16:0/4089624853 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fcdb011f940 con 0x7fcdb0099a40 2023-09-30T22:33:00.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.188+0000 7fcd977fe700 1 -- 172.21.15.16:0/4089624853 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcda0003a20 con 0x7fcdb0099a40 2023-09-30T22:33:00.188 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:33:00.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.190+0000 7fcdb5620700 1 -- 172.21.15.16:0/4089624853 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcdb0099a40 msgr2=0x7fcdb011b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:00.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.190+0000 7fcdb5620700 1 --2- 172.21.15.16:0/4089624853 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcdb0099a40 0x7fcdb011b670 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fcda0000c00 tx=0x7fcda0004750 comp rx=0 tx=0).stop 2023-09-30T22:33:00.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.190+0000 7fcdb5620700 1 -- 172.21.15.16:0/4089624853 shutdown_connections 2023-09-30T22:33:00.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.190+0000 7fcdb5620700 1 --2- 172.21.15.16:0/4089624853 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcdb0131810 0x7fcdb011c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:00.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.190+0000 7fcdb5620700 1 --2- 172.21.15.16:0/4089624853 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcdb0099a40 0x7fcdb011b670 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:00.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.190+0000 7fcdb5620700 1 --2- 172.21.15.16:0/4089624853 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcdb009a580 0x7fcdb011bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:00.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.190+0000 7fcdb5620700 1 -- 172.21.15.16:0/4089624853 >> 172.21.15.16:0/4089624853 conn(0x7fcdb00a2aa0 msgr2=0x7fcdb01b8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:00.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.191+0000 7fcdb5620700 1 -- 172.21.15.16:0/4089624853 shutdown_connections 2023-09-30T22:33:00.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.191+0000 7fcdb5620700 1 -- 172.21.15.16:0/4089624853 wait complete. 2023-09-30T22:33:00.210 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:33:00.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.213+0000 7fd71a990700 1 -- 172.21.15.16:0/4250129781 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd7141a0b30 con 0x7fd71409f650 2023-09-30T22:33:00.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.213+0000 7fd710ff9700 1 -- 172.21.15.16:0/4250129781 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd70c02a020 con 0x7fd71409f650 2023-09-30T22:33:00.213 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:33:00.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.215+0000 7fd71a990700 1 -- 172.21.15.16:0/4250129781 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd71409f650 msgr2=0x7fd71406c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:00.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.215+0000 7fd71a990700 1 --2- 172.21.15.16:0/4250129781 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd71409f650 0x7fd71406c330 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd70c00efc0 tx=0x7fd70c00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:33:00.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.215+0000 7fd71a990700 1 -- 172.21.15.16:0/4250129781 shutdown_connections 2023-09-30T22:33:00.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.215+0000 7fd71a990700 1 --2- 172.21.15.16:0/4250129781 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd71409f650 0x7fd71406c330 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:00.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.215+0000 7fd71a990700 1 --2- 172.21.15.16:0/4250129781 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd71409e930 0x7fd71406b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:00.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.215+0000 7fd71a990700 1 --2- 172.21.15.16:0/4250129781 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd71409f240 0x7fd71406bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:00.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.216+0000 7fd71a990700 1 -- 172.21.15.16:0/4250129781 >> 172.21.15.16:0/4250129781 conn(0x7fd7140a4210 msgr2=0x7fd7141b8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:00.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.216+0000 7fd71a990700 1 -- 172.21.15.16:0/4250129781 shutdown_connections 2023-09-30T22:33:00.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.216+0000 7fd71a990700 1 -- 172.21.15.16:0/4250129781 wait complete. 2023-09-30T22:33:00.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.217+0000 7fd7db1ee700 1 -- 172.21.15.16:0/2774170501 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd7d406b9e0 con 0x7fd7d4130540 2023-09-30T22:33:00.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.218+0000 7fd7d17fa700 1 -- 172.21.15.16:0/2774170501 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd7c8017030 con 0x7fd7d4130540 2023-09-30T22:33:00.218 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:33:00.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.220+0000 7fd7db1ee700 1 -- 172.21.15.16:0/2774170501 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd7d4130540 msgr2=0x7fd7d409c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:00.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.220+0000 7fd7db1ee700 1 --2- 172.21.15.16:0/2774170501 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd7d4130540 0x7fd7d409c420 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd7c8001eb0 tx=0x7fd7c800c7b0 comp rx=0 tx=0).stop 2023-09-30T22:33:00.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.220+0000 7fd7db1ee700 1 -- 172.21.15.16:0/2774170501 shutdown_connections 2023-09-30T22:33:00.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.220+0000 7fd7db1ee700 1 --2- 172.21.15.16:0/2774170501 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd7d4130540 0x7fd7d409c420 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:00.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.220+0000 7fd7db1ee700 1 --2- 172.21.15.16:0/2774170501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd7d4135e60 0x7fd7d409cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:00.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.220+0000 7fd7db1ee700 1 --2- 172.21.15.16:0/2774170501 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd7d4133540 0x7fd7d409c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:00.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.220+0000 7fd7db1ee700 1 -- 172.21.15.16:0/2774170501 >> 172.21.15.16:0/2774170501 conn(0x7fd7d40a2aa0 msgr2=0x7fd7d4131300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:00.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.220+0000 7fd7db1ee700 1 -- 172.21.15.16:0/2774170501 shutdown_connections 2023-09-30T22:33:00.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.220+0000 7fd7db1ee700 1 -- 172.21.15.16:0/2774170501 wait complete. 2023-09-30T22:33:00.232 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:33:00.236 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:33:00.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.261+0000 7fd83d589700 1 -- 172.21.15.16:0/3560151538 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd838074ea0 con 0x7fd83809d6f0 2023-09-30T22:33:00.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.261+0000 7fd81f7fe700 1 -- 172.21.15.16:0/3560151538 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd82801b240 con 0x7fd83809d6f0 2023-09-30T22:33:00.261 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:33:00.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.263+0000 7fd83d589700 1 -- 172.21.15.16:0/3560151538 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd83809d6f0 msgr2=0x7fd83811b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:00.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.263+0000 7fd83d589700 1 --2- 172.21.15.16:0/3560151538 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd83809d6f0 0x7fd83811b1d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fd82800c4a0 tx=0x7fd82800c7b0 comp rx=0 tx=0).stop 2023-09-30T22:33:00.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.264+0000 7fd83d589700 1 -- 172.21.15.16:0/3560151538 shutdown_connections 2023-09-30T22:33:00.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.264+0000 7fd83d589700 1 --2- 172.21.15.16:0/3560151538 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd83809d6f0 0x7fd83811b1d0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:00.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.264+0000 7fd83d589700 1 --2- 172.21.15.16:0/3560151538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd838135e50 0x7fd83811bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:00.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.264+0000 7fd83d589700 1 --2- 172.21.15.16:0/3560151538 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd83809e1d0 0x7fd83811b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:00.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.264+0000 7fd83d589700 1 -- 172.21.15.16:0/3560151538 >> 172.21.15.16:0/3560151538 conn(0x7fd838066f50 msgr2=0x7fd8380a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:00.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.264+0000 7fd83d589700 1 -- 172.21.15.16:0/3560151538 shutdown_connections 2023-09-30T22:33:00.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:00.264+0000 7fd83d589700 1 -- 172.21.15.16:0/3560151538 wait complete. 2023-09-30T22:33:00.277 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:33:01.211 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:33:01.233 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:33:01.236 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:33:01.279 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:33:01.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.374+0000 7f84465fe700 1 Processor -- start 2023-09-30T22:33:01.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.374+0000 7f84465fe700 1 -- start start 2023-09-30T22:33:01.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.375+0000 7f84465fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f844009d6f0 0x7f844009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:01.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.375+0000 7f84465fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f844009e410 0x7f8440130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:01.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.375+0000 7f84465fe700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f844009f130 con 0x7f844009e000 2023-09-30T22:33:01.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.375+0000 7f84465fe700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f844009f520 con 0x7f844009d6f0 2023-09-30T22:33:01.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.375+0000 7f84465fe700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f844009e930 con 0x7f844009e410 2023-09-30T22:33:01.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.375+0000 7f843ffff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f844009d6f0 0x7f844009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:01.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.375+0000 7f843ffff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f844009d6f0 0x7f844009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36544/0 (socket says 172.21.15.16:36544) 2023-09-30T22:33:01.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.375+0000 7f843ffff700 1 -- 172.21.15.16:0/2142853634 learned_addr learned my addr 172.21.15.16:0/2142853634 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:01.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.375+0000 7f843f7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f844009e000 0x7f8440130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:37952/0 (socket says 172.21.15.16:37952) 2023-09-30T22:33:01.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.375+0000 7f8444b9b700 1 --2- 172.21.15.16:0/2142853634 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f844009e410 0x7f8440130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:01.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.375+0000 7f843ffff700 1 -- 172.21.15.16:0/2142853634 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f844009e410 msgr2=0x7f8440130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:01.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.375+0000 7f843ffff700 1 --2- 172.21.15.16:0/2142853634 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f844009e410 0x7f8440130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:01.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.376+0000 7f843ffff700 1 -- 172.21.15.16:0/2142853634 >> v1:172.21.15.16:6789/0 conn(0x7f844009e000 legacy=0x7f8440130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:01.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.376+0000 7f843ffff700 1 -- 172.21.15.16:0/2142853634 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f844009b2d0 con 0x7f844009d6f0 2023-09-30T22:33:01.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.376+0000 7f843ffff700 1 --2- 172.21.15.16:0/2142853634 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f844009d6f0 0x7f844009dac0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8430009920 tx=0x7f8430009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=59d1d9bcd9160a5d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:01.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.376+0000 7f843effd700 1 -- 172.21.15.16:0/2142853634 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843000f8a0 con 0x7f844009d6f0 2023-09-30T22:33:01.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.376+0000 7f843effd700 1 -- 172.21.15.16:0/2142853634 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f843000fa00 con 0x7f844009d6f0 2023-09-30T22:33:01.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.376+0000 7f843effd700 1 -- 172.21.15.16:0/2142853634 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843000fd90 con 0x7f844009d6f0 2023-09-30T22:33:01.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.377+0000 7f84465fe700 1 -- 172.21.15.16:0/2142853634 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f844009d6f0 msgr2=0x7f844009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:01.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.377+0000 7f84465fe700 1 --2- 172.21.15.16:0/2142853634 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f844009d6f0 0x7f844009dac0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8430009920 tx=0x7f8430009c30 comp rx=0 tx=0).stop 2023-09-30T22:33:01.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.377+0000 7f84465fe700 1 -- 172.21.15.16:0/2142853634 shutdown_connections 2023-09-30T22:33:01.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.377+0000 7f84465fe700 1 --2- 172.21.15.16:0/2142853634 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f844009e410 0x7f8440130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:01.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.377+0000 7f84465fe700 1 --2- 172.21.15.16:0/2142853634 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f844009d6f0 0x7f844009dac0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:01.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.377+0000 7f84465fe700 1 -- 172.21.15.16:0/2142853634 >> 172.21.15.16:0/2142853634 conn(0x7f8440066f50 msgr2=0x7f844009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:01.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.377+0000 7f84465fe700 1 -- 172.21.15.16:0/2142853634 shutdown_connections 2023-09-30T22:33:01.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.377+0000 7f84465fe700 1 -- 172.21.15.16:0/2142853634 wait complete. 2023-09-30T22:33:01.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.378+0000 7f84465fe700 1 Processor -- start 2023-09-30T22:33:01.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.379+0000 7f84465fe700 1 -- start start 2023-09-30T22:33:01.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.379+0000 7f84465fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f844009d6f0 0x7f84401214c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:01.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.379+0000 7f84465fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f844009e000 0x7f8440121a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:01.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.379+0000 7f84465fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f844009e410 0x7f8440121f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:01.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.380+0000 7f84465fe700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84401236f0 con 0x7f844009e410 2023-09-30T22:33:01.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.380+0000 7f84465fe700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84400a1e20 con 0x7f844009d6f0 2023-09-30T22:33:01.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.380+0000 7f84465fe700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8440123230 con 0x7f844009e000 2023-09-30T22:33:01.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.380+0000 7f843ffff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f844009d6f0 0x7f84401214c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:01.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.380+0000 7f843ffff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f844009d6f0 0x7f84401214c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36548/0 (socket says 172.21.15.16:36548) 2023-09-30T22:33:01.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.380+0000 7f843ffff700 1 -- 172.21.15.16:0/576710696 learned_addr learned my addr 172.21.15.16:0/576710696 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:01.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.380+0000 7f843f7fe700 1 --2- 172.21.15.16:0/576710696 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f844009e000 0x7f8440121a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:01.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.380+0000 7f8444b9b700 1 --2- 172.21.15.16:0/576710696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f844009e410 0x7f8440121f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:01.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.380+0000 7f843ffff700 1 -- 172.21.15.16:0/576710696 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f844009e000 msgr2=0x7f8440121a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:01.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.380+0000 7f843ffff700 1 --2- 172.21.15.16:0/576710696 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f844009e000 0x7f8440121a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:01.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.380+0000 7f843ffff700 1 -- 172.21.15.16:0/576710696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f844009e410 msgr2=0x7f8440121f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:01.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.380+0000 7f843ffff700 1 --2- 172.21.15.16:0/576710696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f844009e410 0x7f8440121f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:01.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.380+0000 7f843ffff700 1 -- 172.21.15.16:0/576710696 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8430009550 con 0x7f844009d6f0 2023-09-30T22:33:01.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.380+0000 7f843ffff700 1 --2- 172.21.15.16:0/576710696 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f844009d6f0 0x7f84401214c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8430000c00 tx=0x7f8430004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:01.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.381+0000 7f843cff9700 1 -- 172.21.15.16:0/576710696 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84300038a0 con 0x7f844009d6f0 2023-09-30T22:33:01.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.381+0000 7f843cff9700 1 -- 172.21.15.16:0/576710696 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8430003a00 con 0x7f844009d6f0 2023-09-30T22:33:01.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.381+0000 7f84465fe700 1 -- 172.21.15.16:0/576710696 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f844009c700 con 0x7f844009d6f0 2023-09-30T22:33:01.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.381+0000 7f843cff9700 1 -- 172.21.15.16:0/576710696 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8430003d90 con 0x7f844009d6f0 2023-09-30T22:33:01.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.381+0000 7f84465fe700 1 -- 172.21.15.16:0/576710696 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f844012f3c0 con 0x7f844009d6f0 2023-09-30T22:33:01.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.382+0000 7f843cff9700 1 -- 172.21.15.16:0/576710696 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8430010470 con 0x7f844009d6f0 2023-09-30T22:33:01.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.382+0000 7f843cff9700 1 -- 172.21.15.16:0/576710696 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f8430040a80 con 0x7f844009d6f0 2023-09-30T22:33:01.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.382+0000 7f84465fe700 1 -- 172.21.15.16:0/576710696 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8440124df0 con 0x7f844009d6f0 2023-09-30T22:33:01.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.391+0000 7f843cff9700 1 -- 172.21.15.16:0/576710696 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8430022940 con 0x7f844009d6f0 2023-09-30T22:33:01.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.407+0000 7f67f10b5700 1 Processor -- start 2023-09-30T22:33:01.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.407+0000 7f67f10b5700 1 -- start start 2023-09-30T22:33:01.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.408+0000 7f67f10b5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f67ec09e930 0x7f67ec09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:01.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.408+0000 7f67f10b5700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f67ec09f240 0x7f67ec1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:01.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.408+0000 7f67f10b5700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67ec070940 con 0x7f67ec09f650 2023-09-30T22:33:01.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.408+0000 7f67f10b5700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67ec06fd60 con 0x7f67ec09f240 2023-09-30T22:33:01.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.408+0000 7f67f10b5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67ec06fa80 con 0x7f67ec09e930 2023-09-30T22:33:01.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.408+0000 7f67ead9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f67ec09e930 0x7f67ec09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:01.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.408+0000 7f67ead9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f67ec09e930 0x7f67ec09ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36554/0 (socket says 172.21.15.16:36554) 2023-09-30T22:33:01.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.408+0000 7f67ead9d700 1 -- 172.21.15.16:0/1014964228 learned_addr learned my addr 172.21.15.16:0/1014964228 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:01.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.408+0000 7f67ea59c700 1 --2- 172.21.15.16:0/1014964228 >> v2:172.21.15.16:3300/0 conn(0x7f67ec09f240 0x7f67ec1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:01.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.408+0000 7f67ea59c700 1 -- 172.21.15.16:0/1014964228 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f67ec09e930 msgr2=0x7f67ec09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:01.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.408+0000 7f67ea59c700 1 --2- 172.21.15.16:0/1014964228 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f67ec09e930 0x7f67ec09ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:01.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.408+0000 7f67ea59c700 1 -- 172.21.15.16:0/1014964228 >> v1:172.21.15.16:6789/0 conn(0x7f67ec09f650 legacy=0x7f67ec131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:01.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.408+0000 7f67ea59c700 1 -- 172.21.15.16:0/1014964228 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67ec09fb60 con 0x7f67ec09f240 2023-09-30T22:33:01.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.408+0000 7f67ea59c700 1 --2- 172.21.15.16:0/1014964228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67ec09f240 0x7f67ec1317f0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f67e0009840 tx=0x7f67e0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6531303c6be89708 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:01.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.409+0000 7f67e9d9b700 1 -- 172.21.15.16:0/1014964228 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e0011830 con 0x7f67ec09f240 2023-09-30T22:33:01.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.409+0000 7f67e9d9b700 1 -- 172.21.15.16:0/1014964228 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67e0011990 con 0x7f67ec09f240 2023-09-30T22:33:01.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.409+0000 7f67e9d9b700 1 -- 172.21.15.16:0/1014964228 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e0011d20 con 0x7f67ec09f240 2023-09-30T22:33:01.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.409+0000 7f67f10b5700 1 -- 172.21.15.16:0/1014964228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67ec09f240 msgr2=0x7f67ec1317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:01.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.409+0000 7f67f10b5700 1 --2- 172.21.15.16:0/1014964228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67ec09f240 0x7f67ec1317f0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f67e0009840 tx=0x7f67e0009b50 comp rx=0 tx=0).stop 2023-09-30T22:33:01.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.409+0000 7f67f10b5700 1 -- 172.21.15.16:0/1014964228 shutdown_connections 2023-09-30T22:33:01.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.409+0000 7f67f10b5700 1 --2- 172.21.15.16:0/1014964228 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f67ec09e930 0x7f67ec09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:01.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.409+0000 7f67f10b5700 1 --2- 172.21.15.16:0/1014964228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67ec09f240 0x7f67ec1317f0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:01.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.409+0000 7f67f10b5700 1 -- 172.21.15.16:0/1014964228 >> 172.21.15.16:0/1014964228 conn(0x7f67ec0a4210 msgr2=0x7f67ec0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:01.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.409+0000 7f67f10b5700 1 -- 172.21.15.16:0/1014964228 shutdown_connections 2023-09-30T22:33:01.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.410+0000 7f67f10b5700 1 -- 172.21.15.16:0/1014964228 wait complete. 2023-09-30T22:33:01.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.411+0000 7f67f10b5700 1 Processor -- start 2023-09-30T22:33:01.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.411+0000 7f67f10b5700 1 -- start start 2023-09-30T22:33:01.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.412+0000 7f67f10b5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67ec09e930 0x7f67ec06b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:01.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.412+0000 7f67f10b5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f67ec09f240 0x7f67ec06bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:01.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.412+0000 7f67f10b5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f67ec09f650 0x7f67ec06c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:01.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.412+0000 7f67f10b5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67ec1a99e0 con 0x7f67ec09e930 2023-09-30T22:33:01.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.412+0000 7f67f10b5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67ec0a3560 con 0x7f67ec09f240 2023-09-30T22:33:01.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.412+0000 7f67f10b5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f67ec071d20 con 0x7f67ec09f650 2023-09-30T22:33:01.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.412+0000 7f67ea59c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f67ec09f240 0x7f67ec06bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:01.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.413+0000 7f67ea59c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f67ec09f240 0x7f67ec06bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36564/0 (socket says 172.21.15.16:36564) 2023-09-30T22:33:01.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.413+0000 7f67ead9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67ec09e930 0x7f67ec06b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:01.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.413+0000 7f67ea59c700 1 -- 172.21.15.16:0/1581292185 learned_addr learned my addr 172.21.15.16:0/1581292185 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:01.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.413+0000 7f67eb59e700 1 --2- 172.21.15.16:0/1581292185 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f67ec09f650 0x7f67ec06c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:01.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.413+0000 7f67ea59c700 1 -- 172.21.15.16:0/1581292185 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f67ec09f650 msgr2=0x7f67ec06c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:01.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.413+0000 7f67ea59c700 1 --2- 172.21.15.16:0/1581292185 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f67ec09f650 0x7f67ec06c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:01.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.413+0000 7f67ea59c700 1 -- 172.21.15.16:0/1581292185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67ec09e930 msgr2=0x7f67ec06b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:01.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.413+0000 7f67ea59c700 1 --2- 172.21.15.16:0/1581292185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67ec09e930 0x7f67ec06b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:01.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.413+0000 7f67ea59c700 1 -- 172.21.15.16:0/1581292185 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e00094a0 con 0x7f67ec09f240 2023-09-30T22:33:01.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.413+0000 7f67eb59e700 1 --2- 172.21.15.16:0/1581292185 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f67ec09f650 0x7f67ec06c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:33:01.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.413+0000 7f67ea59c700 1 --2- 172.21.15.16:0/1581292185 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f67ec09f240 0x7f67ec06bdc0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f67e00090b0 tx=0x7f67e0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:01.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.413+0000 7f67d37fe700 1 -- 172.21.15.16:0/1581292185 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e00036a0 con 0x7f67ec09f240 2023-09-30T22:33:01.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.413+0000 7f67d37fe700 1 -- 172.21.15.16:0/1581292185 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67e001f070 con 0x7f67ec09f240 2023-09-30T22:33:01.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.413+0000 7f67f10b5700 1 -- 172.21.15.16:0/1581292185 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67ec0a0190 con 0x7f67ec09f240 2023-09-30T22:33:01.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.413+0000 7f67d37fe700 1 -- 172.21.15.16:0/1581292185 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e0003a20 con 0x7f67ec09f240 2023-09-30T22:33:01.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.414+0000 7f67f10b5700 1 -- 172.21.15.16:0/1581292185 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67ec1aa450 con 0x7f67ec09f240 2023-09-30T22:33:01.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.415+0000 7f67f10b5700 1 -- 172.21.15.16:0/1581292185 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67ec0a3180 con 0x7f67ec09f240 2023-09-30T22:33:01.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.416+0000 7f67d37fe700 1 -- 172.21.15.16:0/1581292185 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f67e0003b80 con 0x7f67ec09f240 2023-09-30T22:33:01.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.416+0000 7f67d37fe700 1 -- 172.21.15.16:0/1581292185 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f67e0021070 con 0x7f67ec09f240 2023-09-30T22:33:01.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.422+0000 7fcefed45700 1 Processor -- start 2023-09-30T22:33:01.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.422+0000 7fcefed45700 1 -- start start 2023-09-30T22:33:01.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.423+0000 7fcefed45700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcef806e570 0x7fcef806e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:01.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.423+0000 7fcefed45700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcef809eca0 0x7fcef806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:01.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.423+0000 7fcefed45700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcef8134610 0x7fcef8134ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:01.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.423+0000 7fcefed45700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcef8070ee0 con 0x7fcef8134610 2023-09-30T22:33:01.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.423+0000 7fcefed45700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcef8071480 con 0x7fcef806e570 2023-09-30T22:33:01.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.423+0000 7fcefed45700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcef806f3c0 con 0x7fcef809eca0 2023-09-30T22:33:01.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.423+0000 7fcefd2e2700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcef8134610 0x7fcef8134ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:01.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.423+0000 7fcefd2e2700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcef8134610 0x7fcef8134ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37506/0 (socket says 172.21.15.16:37506) 2023-09-30T22:33:01.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.423+0000 7fcefd2e2700 1 -- 172.21.15.16:0/3540161561 learned_addr learned my addr 172.21.15.16:0/3540161561 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:01.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.423+0000 7fcefcae1700 1 --2- 172.21.15.16:0/3540161561 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcef806e570 0x7fcef806e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:01.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.423+0000 7fcefcae1700 1 -- 172.21.15.16:0/3540161561 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcef809eca0 msgr2=0x7fcef806ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:01.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.423+0000 7fcefcae1700 1 --2- 172.21.15.16:0/3540161561 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcef809eca0 0x7fcef806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:01.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.423+0000 7fcefcae1700 1 -- 172.21.15.16:0/3540161561 >> v2:172.21.15.16:3300/0 conn(0x7fcef8134610 msgr2=0x7fcef8134ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:01.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.423+0000 7fcefcae1700 1 --2- 172.21.15.16:0/3540161561 >> v2:172.21.15.16:3300/0 conn(0x7fcef8134610 0x7fcef8134ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:01.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.423+0000 7fcefcae1700 1 -- 172.21.15.16:0/3540161561 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef8071b90 con 0x7fcef806e570 2023-09-30T22:33:01.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.424+0000 7fcefd2e2700 1 --2- 172.21.15.16:0/3540161561 >> v2:172.21.15.16:3300/0 conn(0x7fcef8134610 0x7fcef8134ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:01.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.424+0000 7fcefcae1700 1 --2- 172.21.15.16:0/3540161561 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcef806e570 0x7fcef806e940 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fceec009840 tx=0x7fceec009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2ecfe1cde8cbe0c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:01.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.425+0000 7fcef77fe700 1 -- 172.21.15.16:0/3540161561 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceec011830 con 0x7fcef806e570 2023-09-30T22:33:01.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.425+0000 7fcef77fe700 1 -- 172.21.15.16:0/3540161561 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceec011990 con 0x7fcef806e570 2023-09-30T22:33:01.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.425+0000 7fcef77fe700 1 -- 172.21.15.16:0/3540161561 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceec011d20 con 0x7fcef806e570 2023-09-30T22:33:01.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.425+0000 7f67d37fe700 1 -- 172.21.15.16:0/1581292185 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f67e0047080 con 0x7f67ec09f240 2023-09-30T22:33:01.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.426+0000 7fcefed45700 1 -- 172.21.15.16:0/3540161561 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcef806e570 msgr2=0x7fcef806e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:01.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.426+0000 7fcefed45700 1 --2- 172.21.15.16:0/3540161561 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcef806e570 0x7fcef806e940 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fceec009840 tx=0x7fceec009b50 comp rx=0 tx=0).stop 2023-09-30T22:33:01.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.426+0000 7fcefed45700 1 -- 172.21.15.16:0/3540161561 shutdown_connections 2023-09-30T22:33:01.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.426+0000 7fcefed45700 1 --2- 172.21.15.16:0/3540161561 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcef806e570 0x7fcef806e940 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:01.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.426+0000 7fcefed45700 1 --2- 172.21.15.16:0/3540161561 >> v2:172.21.15.16:3300/0 conn(0x7fcef8134610 0x7fcef8134ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:01.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.426+0000 7fcefed45700 1 --2- 172.21.15.16:0/3540161561 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcef809eca0 0x7fcef806ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:01.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.426+0000 7fcefed45700 1 -- 172.21.15.16:0/3540161561 >> 172.21.15.16:0/3540161561 conn(0x7fcef80a2aa0 msgr2=0x7fcef80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:01.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.426+0000 7fcefed45700 1 -- 172.21.15.16:0/3540161561 shutdown_connections 2023-09-30T22:33:01.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.426+0000 7fcefed45700 1 -- 172.21.15.16:0/3540161561 wait complete. 2023-09-30T22:33:01.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.427+0000 7fcefed45700 1 Processor -- start 2023-09-30T22:33:01.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.427+0000 7fcefed45700 1 -- start start 2023-09-30T22:33:01.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.428+0000 7fcefed45700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcef806e570 0x7fcef8127640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:01.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.428+0000 7fcefed45700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcef8134610 0x7fcef8125fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:01.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.428+0000 7fcefed45700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcef81264f0 0x7fcef8126900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:01.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.428+0000 7fcefed45700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef8126e40 con 0x7fcef8134610 2023-09-30T22:33:01.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.428+0000 7fcefed45700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcef813e360 con 0x7fcef81264f0 2023-09-30T22:33:01.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.428+0000 7fcefed45700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcef8070610 con 0x7fcef806e570 2023-09-30T22:33:01.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.428+0000 7fcefd2e2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcef81264f0 0x7fcef8126900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:01.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.428+0000 7fcefd2e2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcef81264f0 0x7fcef8126900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36576/0 (socket says 172.21.15.16:36576) 2023-09-30T22:33:01.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.428+0000 7fcefd2e2700 1 -- 172.21.15.16:0/4265510223 learned_addr learned my addr 172.21.15.16:0/4265510223 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:01.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.428+0000 7fcef7fff700 1 --2- 172.21.15.16:0/4265510223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcef8134610 0x7fcef8125fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:01.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.429+0000 7fcefd2e2700 1 -- 172.21.15.16:0/4265510223 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcef806e570 msgr2=0x7fcef8127640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:01.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.429+0000 7fcefd2e2700 1 --2- 172.21.15.16:0/4265510223 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcef806e570 0x7fcef8127640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:01.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.429+0000 7fcefd2e2700 1 -- 172.21.15.16:0/4265510223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcef8134610 msgr2=0x7fcef8125fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:01.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.429+0000 7fcefd2e2700 1 --2- 172.21.15.16:0/4265510223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcef8134610 0x7fcef8125fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:01.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.429+0000 7fcefd2e2700 1 -- 172.21.15.16:0/4265510223 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef0009b30 con 0x7fcef81264f0 2023-09-30T22:33:01.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.429+0000 7fcefd2e2700 1 --2- 172.21.15.16:0/4265510223 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcef81264f0 0x7fcef8126900 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fcef0009f10 tx=0x7fcef000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:01.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.429+0000 7fcef57fa700 1 -- 172.21.15.16:0/4265510223 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef0018070 con 0x7fcef81264f0 2023-09-30T22:33:01.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.429+0000 7fcef57fa700 1 -- 172.21.15.16:0/4265510223 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef000f040 con 0x7fcef81264f0 2023-09-30T22:33:01.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.429+0000 7fcef57fa700 1 -- 172.21.15.16:0/4265510223 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef0003c40 con 0x7fcef81264f0 2023-09-30T22:33:01.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.430+0000 7fcefed45700 1 -- 172.21.15.16:0/4265510223 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fceec0094a0 con 0x7fcef81264f0 2023-09-30T22:33:01.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.430+0000 7fcefed45700 1 -- 172.21.15.16:0/4265510223 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcef813d060 con 0x7fcef81264f0 2023-09-30T22:33:01.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.431+0000 7fcef57fa700 1 -- 172.21.15.16:0/4265510223 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcef000c9f0 con 0x7fcef81264f0 2023-09-30T22:33:01.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.431+0000 7fcef57fa700 1 -- 172.21.15.16:0/4265510223 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fcef003f890 con 0x7fcef81264f0 2023-09-30T22:33:01.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.432+0000 7fcefed45700 1 -- 172.21.15.16:0/4265510223 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcef813d280 con 0x7fcef81264f0 2023-09-30T22:33:01.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.441+0000 7fcef57fa700 1 -- 172.21.15.16:0/4265510223 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcef813d280 con 0x7fcef81264f0 2023-09-30T22:33:01.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.450+0000 7f83f90ae700 1 Processor -- start 2023-09-30T22:33:01.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.450+0000 7f83f90ae700 1 -- start start 2023-09-30T22:33:01.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.451+0000 7f83f90ae700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83f409d6f0 0x7f83f409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:01.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.451+0000 7f83f90ae700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f83f409e000 0x7f83f4130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:01.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.451+0000 7f83f90ae700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f83f409e410 0x7f83f4130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:01.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.451+0000 7f83f90ae700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83f409f520 con 0x7f83f409e410 2023-09-30T22:33:01.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.451+0000 7f83f90ae700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83f409f130 con 0x7f83f409d6f0 2023-09-30T22:33:01.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.451+0000 7f83f90ae700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f83f409e930 con 0x7f83f409e000 2023-09-30T22:33:01.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.451+0000 7f83f359e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f83f409e410 0x7f83f4130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:01.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.451+0000 7f83f2d9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83f409d6f0 0x7f83f409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:01.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.451+0000 7f83f359e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f83f409e410 0x7f83f4130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37528/0 (socket says 172.21.15.16:37528) 2023-09-30T22:33:01.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.451+0000 7f83f359e700 1 -- 172.21.15.16:0/2142028962 learned_addr learned my addr 172.21.15.16:0/2142028962 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:01.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.451+0000 7f83f259c700 1 --2- 172.21.15.16:0/2142028962 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f83f409e000 0x7f83f4130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:01.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.452+0000 7f83f359e700 1 -- 172.21.15.16:0/2142028962 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f83f409e000 msgr2=0x7f83f4130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:01.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.452+0000 7f83f359e700 1 --2- 172.21.15.16:0/2142028962 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f83f409e000 0x7f83f4130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:01.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.452+0000 7f83f359e700 1 -- 172.21.15.16:0/2142028962 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83f409d6f0 msgr2=0x7f83f409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:01.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.452+0000 7f83f359e700 1 --2- 172.21.15.16:0/2142028962 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83f409d6f0 0x7f83f409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:01.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.452+0000 7f83f359e700 1 -- 172.21.15.16:0/2142028962 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f409b2d0 con 0x7f83f409e410 2023-09-30T22:33:01.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.452+0000 7f83f2d9d700 1 --2- 172.21.15.16:0/2142028962 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83f409d6f0 0x7f83f409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:01.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.452+0000 7f83f359e700 1 --2- 172.21.15.16:0/2142028962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83f409e410 0x7f83f4130ba0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f83ec014a00 tx=0x7f83ec014d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b0139cd2ef0a452 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:01.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.452+0000 7f83f1d9b700 1 -- 172.21.15.16:0/2142028962 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ec014ea0 con 0x7f83f409e410 2023-09-30T22:33:01.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.452+0000 7f83f1d9b700 1 -- 172.21.15.16:0/2142028962 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83ec013af0 con 0x7f83f409e410 2023-09-30T22:33:01.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.452+0000 7f83f1d9b700 1 -- 172.21.15.16:0/2142028962 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ec013ea0 con 0x7f83f409e410 2023-09-30T22:33:01.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.453+0000 7f83f90ae700 1 -- 172.21.15.16:0/2142028962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83f409e410 msgr2=0x7f83f4130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:01.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.453+0000 7f83f90ae700 1 --2- 172.21.15.16:0/2142028962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83f409e410 0x7f83f4130ba0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f83ec014a00 tx=0x7f83ec014d10 comp rx=0 tx=0).stop 2023-09-30T22:33:01.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.453+0000 7f83f90ae700 1 -- 172.21.15.16:0/2142028962 shutdown_connections 2023-09-30T22:33:01.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.453+0000 7f83f90ae700 1 --2- 172.21.15.16:0/2142028962 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83f409d6f0 0x7f83f409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:01.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.453+0000 7f83f90ae700 1 --2- 172.21.15.16:0/2142028962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83f409e410 0x7f83f4130ba0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:01.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.453+0000 7f83f90ae700 1 --2- 172.21.15.16:0/2142028962 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f83f409e000 0x7f83f4130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:01.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.453+0000 7f83f90ae700 1 -- 172.21.15.16:0/2142028962 >> 172.21.15.16:0/2142028962 conn(0x7f83f4066f50 msgr2=0x7f83f409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:01.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.453+0000 7f83f90ae700 1 -- 172.21.15.16:0/2142028962 shutdown_connections 2023-09-30T22:33:01.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.453+0000 7f83f90ae700 1 -- 172.21.15.16:0/2142028962 wait complete. 2023-09-30T22:33:01.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.454+0000 7f83f90ae700 1 Processor -- start 2023-09-30T22:33:01.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.454+0000 7f83f90ae700 1 -- start start 2023-09-30T22:33:01.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.455+0000 7f83f90ae700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83f409d6f0 0x7f83f411dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:01.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.455+0000 7f83f90ae700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83f409e000 0x7f83f411e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:01.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.455+0000 7f83f90ae700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f83f409e410 0x7f83f411e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:01.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.455+0000 7f83f90ae700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f41215a0 con 0x7f83f409e000 2023-09-30T22:33:01.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.455+0000 7f83f90ae700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83f411fa80 con 0x7f83f409d6f0 2023-09-30T22:33:01.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.455+0000 7f83f90ae700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f83f40a3a60 con 0x7f83f409e410 2023-09-30T22:33:01.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.455+0000 7f83f2d9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83f409d6f0 0x7f83f411dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:01.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.455+0000 7f83f359e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f83f409e410 0x7f83f411e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:01.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.455+0000 7f83f2d9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83f409d6f0 0x7f83f411dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36592/0 (socket says 172.21.15.16:36592) 2023-09-30T22:33:01.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.455+0000 7f83f2d9d700 1 -- 172.21.15.16:0/3541249398 learned_addr learned my addr 172.21.15.16:0/3541249398 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:01.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.455+0000 7f83f259c700 1 --2- 172.21.15.16:0/3541249398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83f409e000 0x7f83f411e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:01.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.455+0000 7f83f359e700 1 -- 172.21.15.16:0/3541249398 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83f409d6f0 msgr2=0x7f83f411dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:01.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.455+0000 7f83f359e700 1 --2- 172.21.15.16:0/3541249398 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83f409d6f0 0x7f83f411dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:01.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.455+0000 7f83f359e700 1 -- 172.21.15.16:0/3541249398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83f409e000 msgr2=0x7f83f411e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:01.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.455+0000 7f83f359e700 1 --2- 172.21.15.16:0/3541249398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83f409e000 0x7f83f411e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:01.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.455+0000 7f83f359e700 1 -- 172.21.15.16:0/3541249398 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e40094a0 con 0x7f83f409e410 2023-09-30T22:33:01.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.455+0000 7f83f2d9d700 1 --2- 172.21.15.16:0/3541249398 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83f409d6f0 0x7f83f411dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:01.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.456+0000 7f83f359e700 1 --2- 172.21.15.16:0/3541249398 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f83f409e410 0x7f83f411e7e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f83ec01e000 tx=0x7f83ec004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:01.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.456+0000 7f83db7fe700 1 -- 172.21.15.16:0/3541249398 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ec014ea0 con 0x7f83f409e410 2023-09-30T22:33:01.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.456+0000 7f83db7fe700 1 -- 172.21.15.16:0/3541249398 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83ec004240 con 0x7f83f409e410 2023-09-30T22:33:01.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.456+0000 7f83db7fe700 1 -- 172.21.15.16:0/3541249398 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ec0045f0 con 0x7f83f409e410 2023-09-30T22:33:01.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.456+0000 7f83f90ae700 1 -- 172.21.15.16:0/3541249398 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83ec014660 con 0x7f83f409e410 2023-09-30T22:33:01.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.456+0000 7f83f90ae700 1 -- 172.21.15.16:0/3541249398 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83f4120bc0 con 0x7f83f409e410 2023-09-30T22:33:01.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.457+0000 7f83db7fe700 1 -- 172.21.15.16:0/3541249398 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f83ec023030 con 0x7f83f409e410 2023-09-30T22:33:01.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.457+0000 7f83db7fe700 1 -- 172.21.15.16:0/3541249398 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f83ec049720 con 0x7f83f409e410 2023-09-30T22:33:01.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.458+0000 7f83f90ae700 1 -- 172.21.15.16:0/3541249398 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83f411ac20 con 0x7f83f409e410 2023-09-30T22:33:01.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.462+0000 7f83db7fe700 1 -- 172.21.15.16:0/3541249398 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f83ec004750 con 0x7f83f409e410 2023-09-30T22:33:01.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.628+0000 7f84465fe700 1 -- 172.21.15.16:0/576710696 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f8440074ea0 con 0x7f844009d6f0 2023-09-30T22:33:01.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.628+0000 7f843cff9700 1 -- 172.21.15.16:0/576710696 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8430029030 con 0x7f844009d6f0 2023-09-30T22:33:01.629 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:33:01.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.631+0000 7f84465fe700 1 -- 172.21.15.16:0/576710696 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f844009d6f0 msgr2=0x7f84401214c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:01.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.631+0000 7f84465fe700 1 --2- 172.21.15.16:0/576710696 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f844009d6f0 0x7f84401214c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8430000c00 tx=0x7f8430004750 comp rx=0 tx=0).stop 2023-09-30T22:33:01.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.631+0000 7f84465fe700 1 -- 172.21.15.16:0/576710696 shutdown_connections 2023-09-30T22:33:01.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.631+0000 7f84465fe700 1 --2- 172.21.15.16:0/576710696 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f844009d6f0 0x7f84401214c0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:01.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.631+0000 7f84465fe700 1 --2- 172.21.15.16:0/576710696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f844009e410 0x7f8440121f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:01.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.631+0000 7f84465fe700 1 --2- 172.21.15.16:0/576710696 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f844009e000 0x7f8440121a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:01.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.631+0000 7f84465fe700 1 -- 172.21.15.16:0/576710696 >> 172.21.15.16:0/576710696 conn(0x7f8440066f50 msgr2=0x7f844012f650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:01.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.632+0000 7f84465fe700 1 -- 172.21.15.16:0/576710696 shutdown_connections 2023-09-30T22:33:01.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.632+0000 7f84465fe700 1 -- 172.21.15.16:0/576710696 wait complete. 2023-09-30T22:33:01.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.639+0000 7f67f10b5700 1 -- 172.21.15.16:0/1581292185 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f67ec074ea0 con 0x7f67ec09f240 2023-09-30T22:33:01.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.639+0000 7f67d37fe700 1 -- 172.21.15.16:0/1581292185 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f67e002b020 con 0x7f67ec09f240 2023-09-30T22:33:01.639 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:33:01.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.641+0000 7f67f10b5700 1 -- 172.21.15.16:0/1581292185 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f67ec09f240 msgr2=0x7f67ec06bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:01.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.641+0000 7f67f10b5700 1 --2- 172.21.15.16:0/1581292185 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f67ec09f240 0x7f67ec06bdc0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f67e00090b0 tx=0x7f67e0004750 comp rx=0 tx=0).stop 2023-09-30T22:33:01.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.641+0000 7f67f10b5700 1 -- 172.21.15.16:0/1581292185 shutdown_connections 2023-09-30T22:33:01.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.641+0000 7f67f10b5700 1 --2- 172.21.15.16:0/1581292185 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f67ec09f650 0x7f67ec06c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:01.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.641+0000 7f67f10b5700 1 --2- 172.21.15.16:0/1581292185 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67ec09e930 0x7f67ec06b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:01.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.641+0000 7f67f10b5700 1 --2- 172.21.15.16:0/1581292185 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f67ec09f240 0x7f67ec06bdc0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:01.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.641+0000 7f67f10b5700 1 -- 172.21.15.16:0/1581292185 >> 172.21.15.16:0/1581292185 conn(0x7f67ec0a4210 msgr2=0x7f67ec1b8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:01.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.642+0000 7f67f10b5700 1 -- 172.21.15.16:0/1581292185 shutdown_connections 2023-09-30T22:33:01.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.642+0000 7f67f10b5700 1 -- 172.21.15.16:0/1581292185 wait complete. 2023-09-30T22:33:01.653 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:33:01.657 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:33:01.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.672+0000 7fcefed45700 1 -- 172.21.15.16:0/4265510223 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fcef813d280 con 0x7fcef81264f0 2023-09-30T22:33:01.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.672+0000 7fcef57fa700 1 -- 172.21.15.16:0/4265510223 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcef813d280 con 0x7fcef81264f0 2023-09-30T22:33:01.673 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:33:01.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.674+0000 7f83f90ae700 1 -- 172.21.15.16:0/3541249398 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f83f4074ea0 con 0x7f83f409e410 2023-09-30T22:33:01.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.675+0000 7f83db7fe700 1 -- 172.21.15.16:0/3541249398 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f83ec034030 con 0x7f83f409e410 2023-09-30T22:33:01.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.675+0000 7fcefed45700 1 -- 172.21.15.16:0/4265510223 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcef81264f0 msgr2=0x7fcef8126900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:01.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.675+0000 7fcefed45700 1 --2- 172.21.15.16:0/4265510223 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcef81264f0 0x7fcef8126900 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fcef0009f10 tx=0x7fcef000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:33:01.675 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:33:01.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.675+0000 7fcefed45700 1 -- 172.21.15.16:0/4265510223 shutdown_connections 2023-09-30T22:33:01.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.675+0000 7fcefed45700 1 --2- 172.21.15.16:0/4265510223 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcef81264f0 0x7fcef8126900 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:01.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.675+0000 7fcefed45700 1 --2- 172.21.15.16:0/4265510223 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcef806e570 0x7fcef8127640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:01.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.675+0000 7fcefed45700 1 --2- 172.21.15.16:0/4265510223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcef8134610 0x7fcef8125fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:01.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.675+0000 7fcefed45700 1 -- 172.21.15.16:0/4265510223 >> 172.21.15.16:0/4265510223 conn(0x7fcef80a2aa0 msgr2=0x7fcef8099a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:01.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.675+0000 7fcefed45700 1 -- 172.21.15.16:0/4265510223 shutdown_connections 2023-09-30T22:33:01.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.675+0000 7fcefed45700 1 -- 172.21.15.16:0/4265510223 wait complete. 2023-09-30T22:33:01.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.677+0000 7f83f90ae700 1 -- 172.21.15.16:0/3541249398 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f83f409e410 msgr2=0x7f83f411e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:01.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.677+0000 7f83f90ae700 1 --2- 172.21.15.16:0/3541249398 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f83f409e410 0x7f83f411e7e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f83ec01e000 tx=0x7f83ec004ad0 comp rx=0 tx=0).stop 2023-09-30T22:33:01.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.677+0000 7f83f90ae700 1 -- 172.21.15.16:0/3541249398 shutdown_connections 2023-09-30T22:33:01.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.677+0000 7f83f90ae700 1 --2- 172.21.15.16:0/3541249398 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83f409d6f0 0x7f83f411dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:01.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.677+0000 7f83f90ae700 1 --2- 172.21.15.16:0/3541249398 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f83f409e410 0x7f83f411e7e0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:01.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.677+0000 7f83f90ae700 1 --2- 172.21.15.16:0/3541249398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83f409e000 0x7f83f411e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:01.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.677+0000 7f83f90ae700 1 -- 172.21.15.16:0/3541249398 >> 172.21.15.16:0/3541249398 conn(0x7f83f4066f50 msgr2=0x7f83f4130990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:01.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.677+0000 7f83f90ae700 1 -- 172.21.15.16:0/3541249398 shutdown_connections 2023-09-30T22:33:01.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:01.677+0000 7f83f90ae700 1 -- 172.21.15.16:0/3541249398 wait complete. 2023-09-30T22:33:01.691 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:33:01.693 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:33:02.653 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:33:02.657 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:33:02.692 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:33:02.694 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:33:02.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.812+0000 7f934ad78700 1 Processor -- start 2023-09-30T22:33:02.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.813+0000 7f934ad78700 1 -- start start 2023-09-30T22:33:02.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.813+0000 7f934ad78700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f934409d6f0 0x7f934409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:02.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.813+0000 7f934ad78700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f934409e000 0x7f9344130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:02.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.813+0000 7f934ad78700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f934409e410 0x7f9344130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:02.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.813+0000 7f934ad78700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f934409f520 con 0x7f934409e000 2023-09-30T22:33:02.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.813+0000 7f934ad78700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f934409f130 con 0x7f934409d6f0 2023-09-30T22:33:02.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.813+0000 7f934ad78700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f934409e930 con 0x7f934409e410 2023-09-30T22:33:02.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.814+0000 7f9348b14700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f934409d6f0 0x7f934409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:02.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.814+0000 7f9348b14700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f934409d6f0 0x7f934409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36594/0 (socket says 172.21.15.16:36594) 2023-09-30T22:33:02.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.814+0000 7f9348b14700 1 -- 172.21.15.16:0/3777806301 learned_addr learned my addr 172.21.15.16:0/3777806301 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:02.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.814+0000 7f9343fff700 1 --2- 172.21.15.16:0/3777806301 >> v2:172.21.15.16:3300/0 conn(0x7f934409e000 0x7f9344130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:02.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.814+0000 7f9349315700 1 --2- 172.21.15.16:0/3777806301 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f934409e410 0x7f9344130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:02.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.814+0000 7f9343fff700 1 -- 172.21.15.16:0/3777806301 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f934409e410 msgr2=0x7f9344130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:02.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.814+0000 7f9343fff700 1 --2- 172.21.15.16:0/3777806301 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f934409e410 0x7f9344130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:02.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.814+0000 7f9343fff700 1 -- 172.21.15.16:0/3777806301 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f934409d6f0 msgr2=0x7f934409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:02.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.814+0000 7f9343fff700 1 --2- 172.21.15.16:0/3777806301 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f934409d6f0 0x7f934409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:02.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.814+0000 7f9343fff700 1 -- 172.21.15.16:0/3777806301 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f934409b2d0 con 0x7f934409e000 2023-09-30T22:33:02.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.814+0000 7f9349315700 1 --2- 172.21.15.16:0/3777806301 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f934409e410 0x7f9344130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:02.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.814+0000 7f9343fff700 1 --2- 172.21.15.16:0/3777806301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f934409e000 0x7f9344130660 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f9330009870 tx=0x7f9330009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1297f242a0e46372 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:02.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.815+0000 7f93437fe700 1 -- 172.21.15.16:0/3777806301 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9330011830 con 0x7f934409e000 2023-09-30T22:33:02.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.815+0000 7f93437fe700 1 -- 172.21.15.16:0/3777806301 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9330011990 con 0x7f934409e000 2023-09-30T22:33:02.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.815+0000 7f93437fe700 1 -- 172.21.15.16:0/3777806301 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9330011d20 con 0x7f934409e000 2023-09-30T22:33:02.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.816+0000 7f934ad78700 1 -- 172.21.15.16:0/3777806301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f934409e000 msgr2=0x7f9344130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:02.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.816+0000 7f934ad78700 1 --2- 172.21.15.16:0/3777806301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f934409e000 0x7f9344130660 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f9330009870 tx=0x7f9330009b80 comp rx=0 tx=0).stop 2023-09-30T22:33:02.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.816+0000 7f934ad78700 1 -- 172.21.15.16:0/3777806301 shutdown_connections 2023-09-30T22:33:02.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.816+0000 7f934ad78700 1 --2- 172.21.15.16:0/3777806301 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f934409d6f0 0x7f934409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:02.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.816+0000 7f934ad78700 1 --2- 172.21.15.16:0/3777806301 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f934409e410 0x7f9344130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:02.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.816+0000 7f934ad78700 1 --2- 172.21.15.16:0/3777806301 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f934409e000 0x7f9344130660 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:02.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.816+0000 7f934ad78700 1 -- 172.21.15.16:0/3777806301 >> 172.21.15.16:0/3777806301 conn(0x7f9344066f50 msgr2=0x7f934409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:02.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.816+0000 7f934ad78700 1 -- 172.21.15.16:0/3777806301 shutdown_connections 2023-09-30T22:33:02.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.816+0000 7f934ad78700 1 -- 172.21.15.16:0/3777806301 wait complete. 2023-09-30T22:33:02.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.817+0000 7f934ad78700 1 Processor -- start 2023-09-30T22:33:02.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.817+0000 7f934ad78700 1 -- start start 2023-09-30T22:33:02.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.818+0000 7f934ad78700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f934409d6f0 0x7f93441224c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:02.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.818+0000 7f934ad78700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f934409e000 0x7f9344122a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:02.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.818+0000 7f934ad78700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f934409e410 0x7f9344122f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:02.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.818+0000 7f934ad78700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9344123990 con 0x7f934409d6f0 2023-09-30T22:33:02.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.818+0000 7f934ad78700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93441303e0 con 0x7f934409e000 2023-09-30T22:33:02.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.818+0000 7f934ad78700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f93441241e0 con 0x7f934409e410 2023-09-30T22:33:02.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.818+0000 7f9343fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f934409e000 0x7f9344122a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:02.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.818+0000 7f9343fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f934409e000 0x7f9344122a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36608/0 (socket says 172.21.15.16:36608) 2023-09-30T22:33:02.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.818+0000 7f9343fff700 1 -- 172.21.15.16:0/4226880097 learned_addr learned my addr 172.21.15.16:0/4226880097 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:02.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.818+0000 7f9349315700 1 --2- 172.21.15.16:0/4226880097 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f934409e410 0x7f9344122f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:02.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.818+0000 7f9343fff700 1 -- 172.21.15.16:0/4226880097 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f934409e410 msgr2=0x7f9344122f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:02.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.818+0000 7f9343fff700 1 --2- 172.21.15.16:0/4226880097 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f934409e410 0x7f9344122f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:02.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.818+0000 7f9343fff700 1 -- 172.21.15.16:0/4226880097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f934409d6f0 msgr2=0x7f93441224c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:33:02.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.818+0000 7f9343fff700 1 --2- 172.21.15.16:0/4226880097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f934409d6f0 0x7f93441224c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:02.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.818+0000 7f9343fff700 1 -- 172.21.15.16:0/4226880097 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f933c009a70 con 0x7f934409e000 2023-09-30T22:33:02.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.819+0000 7f9343fff700 1 --2- 172.21.15.16:0/4226880097 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f934409e000 0x7f9344122a00 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f93300090b0 tx=0x7f9330004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:02.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.819+0000 7f93417fa700 1 -- 172.21.15.16:0/4226880097 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9330004550 con 0x7f934409e000 2023-09-30T22:33:02.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.819+0000 7f93417fa700 1 -- 172.21.15.16:0/4226880097 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f933001f070 con 0x7f934409e000 2023-09-30T22:33:02.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.819+0000 7f934ad78700 1 -- 172.21.15.16:0/4226880097 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93300094a0 con 0x7f934409e000 2023-09-30T22:33:02.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.819+0000 7f93417fa700 1 -- 172.21.15.16:0/4226880097 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93300038c0 con 0x7f934409e000 2023-09-30T22:33:02.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.819+0000 7f934ad78700 1 -- 172.21.15.16:0/4226880097 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93441248c0 con 0x7f934409e000 2023-09-30T22:33:02.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.820+0000 7f93417fa700 1 -- 172.21.15.16:0/4226880097 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9330003a20 con 0x7f934409e000 2023-09-30T22:33:02.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.820+0000 7f93417fa700 1 -- 172.21.15.16:0/4226880097 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f9330021070 con 0x7f934409e000 2023-09-30T22:33:02.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.820+0000 7f932effd700 1 -- 172.21.15.16:0/4226880097 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93441259a0 con 0x7f934409e000 2023-09-30T22:33:02.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.829+0000 7f93417fa700 1 -- 172.21.15.16:0/4226880097 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f933002b020 con 0x7f934409e000 2023-09-30T22:33:02.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.852+0000 7f3f479cd700 1 Processor -- start 2023-09-30T22:33:02.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.852+0000 7f44c5745700 1 Processor -- start 2023-09-30T22:33:02.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.852+0000 7f44c5745700 1 -- start start 2023-09-30T22:33:02.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.853+0000 7f44c5745700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f44c009d6f0 0x7f44c009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:02.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.853+0000 7f44c5745700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f44c009e000 0x7f44c0130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:02.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.853+0000 7f44c5745700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44c009f130 con 0x7f44c009e410 2023-09-30T22:33:02.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.853+0000 7f44c5745700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44c009f520 con 0x7f44c009d6f0 2023-09-30T22:33:02.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.853+0000 7f44c5745700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f44c009e930 con 0x7f44c009e000 2023-09-30T22:33:02.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.852+0000 7f3f479cd700 1 -- start start 2023-09-30T22:33:02.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.853+0000 7f3f479cd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3f4009e1d0 0x7f3f40135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:02.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.853+0000 7f3f479cd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3f40135e50 0x7f3f40138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:02.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.853+0000 7f3f479cd700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f4009f130 con 0x7f3f4009d6f0 2023-09-30T22:33:02.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.853+0000 7f3f479cd700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4009f520 con 0x7f3f4009e1d0 2023-09-30T22:33:02.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.853+0000 7f3f479cd700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4009e930 con 0x7f3f40135e50 2023-09-30T22:33:02.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.853+0000 7f44beffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f44c009d6f0 0x7f44c009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:02.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.853+0000 7f44bf7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f44c009e410 0x7f44c0130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:37968/0 (socket says 172.21.15.16:37968) 2023-09-30T22:33:02.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.853+0000 7f44beffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f44c009d6f0 0x7f44c009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36622/0 (socket says 172.21.15.16:36622) 2023-09-30T22:33:02.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.853+0000 7f44bf7fe700 1 -- 172.21.15.16:0/2910887585 learned_addr learned my addr 172.21.15.16:0/2910887585 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:02.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.853+0000 7f3f45769700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3f4009d6f0 0x7f3f4009dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:37978/0 (socket says 172.21.15.16:37978) 2023-09-30T22:33:02.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.853+0000 7f3f45769700 1 -- 172.21.15.16:0/2270848390 learned_addr learned my addr 172.21.15.16:0/2270848390 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:02.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.853+0000 7f3f44f68700 1 --2- 172.21.15.16:0/2270848390 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3f4009e1d0 0x7f3f40135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:02.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.853+0000 7f3f37fff700 1 -- 172.21.15.16:0/2270848390 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1390610661 0 0) 0x7f3f4009f130 con 0x7f3f4009d6f0 2023-09-30T22:33:02.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.853+0000 7f3f37fff700 1 -- 172.21.15.16:0/2270848390 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f2c0035b0 con 0x7f3f4009d6f0 2023-09-30T22:33:02.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.853+0000 7f44beffd700 1 -- 172.21.15.16:0/2910887585 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f44c009e000 msgr2=0x7f44c0130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:02.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.853+0000 7f44beffd700 1 --2- 172.21.15.16:0/2910887585 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f44c009e000 0x7f44c0130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:02.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.853+0000 7f44beffd700 1 -- 172.21.15.16:0/2910887585 >> v1:172.21.15.16:6789/0 conn(0x7f44c009e410 legacy=0x7f44c0130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:02.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.853+0000 7f44beffd700 1 -- 172.21.15.16:0/2910887585 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44c009b2d0 con 0x7f44c009d6f0 2023-09-30T22:33:02.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.853+0000 7f44bdffb700 1 -- 172.21.15.16:0/2910887585 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 989596060 0 0) 0x7f44c009f130 con 0x7f44c009e410 2023-09-30T22:33:02.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.854+0000 7f3f44f68700 1 -- 172.21.15.16:0/2270848390 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3f40135e50 msgr2=0x7f3f40138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:02.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.854+0000 7f3f44f68700 1 --2- 172.21.15.16:0/2270848390 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3f40135e50 0x7f3f40138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:02.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.854+0000 7f3f44f68700 1 -- 172.21.15.16:0/2270848390 >> v1:172.21.15.16:6789/0 conn(0x7f3f4009d6f0 legacy=0x7f3f4009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:02.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.854+0000 7f3f44f68700 1 -- 172.21.15.16:0/2270848390 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f4009b2d0 con 0x7f3f4009e1d0 2023-09-30T22:33:02.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.854+0000 7f3f44f68700 1 --2- 172.21.15.16:0/2270848390 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3f4009e1d0 0x7f3f40135910 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f3f30009870 tx=0x7f3f30009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=574fd2f796090233 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:02.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.854+0000 7f3f37fff700 1 -- 172.21.15.16:0/2270848390 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f30011830 con 0x7f3f4009e1d0 2023-09-30T22:33:02.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.854+0000 7f3f37fff700 1 -- 172.21.15.16:0/2270848390 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f30011990 con 0x7f3f4009e1d0 2023-09-30T22:33:02.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.854+0000 7f3f37fff700 1 -- 172.21.15.16:0/2270848390 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f30011d20 con 0x7f3f4009e1d0 2023-09-30T22:33:02.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.854+0000 7f44beffd700 1 --2- 172.21.15.16:0/2910887585 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f44c009d6f0 0x7f44c009dac0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f44b0009870 tx=0x7f44b0009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e63c6315742c6304 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:02.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.855+0000 7f3f479cd700 1 -- 172.21.15.16:0/2270848390 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3f4009e1d0 msgr2=0x7f3f40135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:02.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.855+0000 7f3f479cd700 1 --2- 172.21.15.16:0/2270848390 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3f4009e1d0 0x7f3f40135910 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f3f30009870 tx=0x7f3f30009b80 comp rx=0 tx=0).stop 2023-09-30T22:33:02.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.855+0000 7f3f479cd700 1 -- 172.21.15.16:0/2270848390 shutdown_connections 2023-09-30T22:33:02.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.855+0000 7f3f479cd700 1 --2- 172.21.15.16:0/2270848390 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3f40135e50 0x7f3f40138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:02.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.855+0000 7f3f479cd700 1 --2- 172.21.15.16:0/2270848390 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3f4009e1d0 0x7f3f40135910 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:02.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.855+0000 7f3f479cd700 1 -- 172.21.15.16:0/2270848390 >> 172.21.15.16:0/2270848390 conn(0x7f3f40066f50 msgr2=0x7f3f4009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:02.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.855+0000 7f3f479cd700 1 -- 172.21.15.16:0/2270848390 shutdown_connections 2023-09-30T22:33:02.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.855+0000 7f3f479cd700 1 -- 172.21.15.16:0/2270848390 wait complete. 2023-09-30T22:33:02.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.856+0000 7f3f479cd700 1 Processor -- start 2023-09-30T22:33:02.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.856+0000 7f3f479cd700 1 -- start start 2023-09-30T22:33:02.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.857+0000 7f3f479cd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3f4009d6f0 0x7f3f4006f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:02.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.857+0000 7f3f479cd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3f4009e1d0 0x7f3f4006f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:02.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.857+0000 7f3f479cd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f40135e50 0x7f3f401c80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:02.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.857+0000 7f3f479cd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f40122fb0 con 0x7f3f40135e50 2023-09-30T22:33:02.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.857+0000 7f3f479cd700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4009c700 con 0x7f3f4009e1d0 2023-09-30T22:33:02.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.857+0000 7f3f479cd700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4006fd10 con 0x7f3f4009d6f0 2023-09-30T22:33:02.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.857+0000 7f3f44f68700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3f4009e1d0 0x7f3f4006f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:02.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.857+0000 7f3f45769700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3f4009d6f0 0x7f3f4006f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:02.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.857+0000 7f3f44f68700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3f4009e1d0 0x7f3f4006f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36634/0 (socket says 172.21.15.16:36634) 2023-09-30T22:33:02.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.857+0000 7f3f44f68700 1 -- 172.21.15.16:0/247404083 learned_addr learned my addr 172.21.15.16:0/247404083 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:02.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.857+0000 7f3f45f6a700 1 --2- 172.21.15.16:0/247404083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f40135e50 0x7f3f401c80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:02.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.857+0000 7f3f45769700 1 -- 172.21.15.16:0/247404083 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3f4009e1d0 msgr2=0x7f3f4006f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:02.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.857+0000 7f3f45769700 1 --2- 172.21.15.16:0/247404083 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3f4009e1d0 0x7f3f4006f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:02.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.857+0000 7f3f45769700 1 -- 172.21.15.16:0/247404083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f40135e50 msgr2=0x7f3f401c80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:02.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.857+0000 7f3f45769700 1 --2- 172.21.15.16:0/247404083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f40135e50 0x7f3f401c80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:02.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.857+0000 7f3f45769700 1 -- 172.21.15.16:0/247404083 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f300094a0 con 0x7f3f4009d6f0 2023-09-30T22:33:02.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.857+0000 7f3f44f68700 1 --2- 172.21.15.16:0/247404083 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3f4009e1d0 0x7f3f4006f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:33:02.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.857+0000 7f3f45769700 1 --2- 172.21.15.16:0/247404083 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3f4009d6f0 0x7f3f4006f290 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c00c4c0 tx=0x7f3f3c00c7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:02.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.858+0000 7f3f35ffb700 1 -- 172.21.15.16:0/247404083 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c005d70 con 0x7f3f4009d6f0 2023-09-30T22:33:02.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.858+0000 7f3f35ffb700 1 -- 172.21.15.16:0/247404083 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f3c0070d0 con 0x7f3f4009d6f0 2023-09-30T22:33:02.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.858+0000 7f3f479cd700 1 -- 172.21.15.16:0/247404083 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f40131fe0 con 0x7f3f4009d6f0 2023-09-30T22:33:02.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.858+0000 7f3f35ffb700 1 -- 172.21.15.16:0/247404083 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c007440 con 0x7f3f4009d6f0 2023-09-30T22:33:02.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.858+0000 7f3f479cd700 1 -- 172.21.15.16:0/247404083 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f400a1bd0 con 0x7f3f4009d6f0 2023-09-30T22:33:02.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.859+0000 7f3f35ffb700 1 -- 172.21.15.16:0/247404083 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3f3c0075a0 con 0x7f3f4009d6f0 2023-09-30T22:33:02.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.859+0000 7f3f35ffb700 1 -- 172.21.15.16:0/247404083 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f3f3c03e7a0 con 0x7f3f4009d6f0 2023-09-30T22:33:02.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.859+0000 7f3f479cd700 1 -- 172.21.15.16:0/247404083 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f40121ad0 con 0x7f3f4009d6f0 2023-09-30T22:33:02.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.863+0000 7f44bdffb700 1 -- 172.21.15.16:0/2910887585 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b0011830 con 0x7f44c009d6f0 2023-09-30T22:33:02.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.863+0000 7f44bdffb700 1 -- 172.21.15.16:0/2910887585 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44b0011990 con 0x7f44c009d6f0 2023-09-30T22:33:02.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.863+0000 7f44bdffb700 1 -- 172.21.15.16:0/2910887585 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b0011d20 con 0x7f44c009d6f0 2023-09-30T22:33:02.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.864+0000 7fe0418a0700 1 Processor -- start 2023-09-30T22:33:02.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.864+0000 7f44c5745700 1 -- 172.21.15.16:0/2910887585 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f44c009d6f0 msgr2=0x7f44c009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:02.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.864+0000 7f44c5745700 1 --2- 172.21.15.16:0/2910887585 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f44c009d6f0 0x7f44c009dac0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f44b0009870 tx=0x7f44b0009b80 comp rx=0 tx=0).stop 2023-09-30T22:33:02.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.864+0000 7f44c5745700 1 -- 172.21.15.16:0/2910887585 shutdown_connections 2023-09-30T22:33:02.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.864+0000 7f44c5745700 1 --2- 172.21.15.16:0/2910887585 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f44c009d6f0 0x7f44c009dac0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:02.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.864+0000 7f44c5745700 1 --2- 172.21.15.16:0/2910887585 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f44c009e000 0x7f44c0130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:02.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.864+0000 7f44c5745700 1 -- 172.21.15.16:0/2910887585 >> 172.21.15.16:0/2910887585 conn(0x7f44c0066f50 msgr2=0x7f44c009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:02.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.864+0000 7fe0418a0700 1 -- start start 2023-09-30T22:33:02.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.864+0000 7fe0418a0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe03c06e570 0x7fe03c06e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:02.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.864+0000 7fe0418a0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe03c06ee80 0x7fe03c134d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:02.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.864+0000 7fe0418a0700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe03c070ee0 con 0x7fe03c09eca0 2023-09-30T22:33:02.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.864+0000 7fe0418a0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe03c071480 con 0x7fe03c06e570 2023-09-30T22:33:02.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.864+0000 7fe0418a0700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe03c071b90 con 0x7fe03c06ee80 2023-09-30T22:33:02.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.864+0000 7fe03b7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe03c06ee80 0x7fe03c134d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:02.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.864+0000 7fe03a7fc700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe03c09eca0 0x7fe03c134610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:37992/0 (socket says 172.21.15.16:37992) 2023-09-30T22:33:02.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.864+0000 7fe03a7fc700 1 -- 172.21.15.16:0/91856127 learned_addr learned my addr 172.21.15.16:0/91856127 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:02.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.865+0000 7fe03affd700 1 --2- 172.21.15.16:0/91856127 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe03c06e570 0x7fe03c06e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:02.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.865+0000 7f3f35ffb700 1 -- 172.21.15.16:0/247404083 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3f3c018af0 con 0x7f3f4009d6f0 2023-09-30T22:33:02.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.865+0000 7f44c5745700 1 -- 172.21.15.16:0/2910887585 shutdown_connections 2023-09-30T22:33:02.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.865+0000 7fe039ffb700 1 -- 172.21.15.16:0/91856127 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 660100757 0 0) 0x7fe03c070ee0 con 0x7fe03c09eca0 2023-09-30T22:33:02.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.865+0000 7fe039ffb700 1 -- 172.21.15.16:0/91856127 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0240035b0 con 0x7fe03c09eca0 2023-09-30T22:33:02.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.865+0000 7fe03b7fe700 1 -- 172.21.15.16:0/91856127 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe03c06e570 msgr2=0x7fe03c06e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:02.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.865+0000 7fe03b7fe700 1 --2- 172.21.15.16:0/91856127 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe03c06e570 0x7fe03c06e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:02.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.865+0000 7fe03b7fe700 1 -- 172.21.15.16:0/91856127 >> v1:172.21.15.16:6789/0 conn(0x7fe03c09eca0 legacy=0x7fe03c134610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:02.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.865+0000 7fe03b7fe700 1 -- 172.21.15.16:0/91856127 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe03c0711a0 con 0x7fe03c06ee80 2023-09-30T22:33:02.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.865+0000 7fe03affd700 1 --2- 172.21.15.16:0/91856127 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe03c06e570 0x7fe03c06e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:02.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.865+0000 7fe03b7fe700 1 --2- 172.21.15.16:0/91856127 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe03c06ee80 0x7fe03c134d20 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fe034009dc0 tx=0x7fe03400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6d4b81993b602c8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:02.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.865+0000 7fe039ffb700 1 -- 172.21.15.16:0/91856127 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03400e070 con 0x7fe03c06ee80 2023-09-30T22:33:02.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.866+0000 7fe039ffb700 1 -- 172.21.15.16:0/91856127 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe03400bb20 con 0x7fe03c06ee80 2023-09-30T22:33:02.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.866+0000 7fe039ffb700 1 -- 172.21.15.16:0/91856127 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe034012430 con 0x7fe03c06ee80 2023-09-30T22:33:02.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.865+0000 7f44c5745700 1 -- 172.21.15.16:0/2910887585 wait complete. 2023-09-30T22:33:02.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.866+0000 7fe0418a0700 1 -- 172.21.15.16:0/91856127 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe03c06ee80 msgr2=0x7fe03c134d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:02.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.866+0000 7fe0418a0700 1 --2- 172.21.15.16:0/91856127 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe03c06ee80 0x7fe03c134d20 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fe034009dc0 tx=0x7fe03400b330 comp rx=0 tx=0).stop 2023-09-30T22:33:02.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.867+0000 7f44c5745700 1 Processor -- start 2023-09-30T22:33:02.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.867+0000 7f44c5745700 1 -- start start 2023-09-30T22:33:02.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.867+0000 7f44c5745700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f44c009d6f0 0x7f44c011df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:02.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.867+0000 7f44c5745700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f44c009e000 0x7f44c011e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:02.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.867+0000 7f44c5745700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44c009e410 0x7f44c011e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:02.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.867+0000 7f44c5745700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c0121820 con 0x7f44c009e410 2023-09-30T22:33:02.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.867+0000 7f44c5745700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44c0130320 con 0x7f44c009e000 2023-09-30T22:33:02.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.867+0000 7f44c5745700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f44c0120da0 con 0x7f44c009d6f0 2023-09-30T22:33:02.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.868+0000 7f44beffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f44c009d6f0 0x7f44c011df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:02.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.868+0000 7f44beffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f44c009d6f0 0x7f44c011df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:38070/0 (socket says 172.21.15.16:38070) 2023-09-30T22:33:02.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.868+0000 7f44beffd700 1 -- 172.21.15.16:0/185375847 learned_addr learned my addr 172.21.15.16:0/185375847 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:02.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.868+0000 7f44bf7fe700 1 --2- 172.21.15.16:0/185375847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44c009e410 0x7f44c011e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:02.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.868+0000 7f44be7fc700 1 --2- 172.21.15.16:0/185375847 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f44c009e000 0x7f44c011e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:02.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.868+0000 7f44beffd700 1 -- 172.21.15.16:0/185375847 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f44c009e000 msgr2=0x7f44c011e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:02.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.868+0000 7f44beffd700 1 --2- 172.21.15.16:0/185375847 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f44c009e000 0x7f44c011e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:02.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.868+0000 7f44beffd700 1 -- 172.21.15.16:0/185375847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44c009e410 msgr2=0x7f44c011e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:02.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.868+0000 7f44beffd700 1 --2- 172.21.15.16:0/185375847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44c009e410 0x7f44c011e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:02.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.868+0000 7f44beffd700 1 -- 172.21.15.16:0/185375847 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44b00094a0 con 0x7f44c009d6f0 2023-09-30T22:33:02.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.868+0000 7f44be7fc700 1 --2- 172.21.15.16:0/185375847 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f44c009e000 0x7f44c011e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:33:02.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.868+0000 7f44beffd700 1 --2- 172.21.15.16:0/185375847 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f44c009d6f0 0x7f44c011df40 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f44b0009bc0 tx=0x7f44b0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:02.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.868+0000 7f44a77fe700 1 -- 172.21.15.16:0/185375847 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b00036a0 con 0x7f44c009d6f0 2023-09-30T22:33:02.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.868+0000 7f44a77fe700 1 -- 172.21.15.16:0/185375847 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44b001f070 con 0x7f44c009d6f0 2023-09-30T22:33:02.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.869+0000 7f44a77fe700 1 -- 172.21.15.16:0/185375847 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b0003a20 con 0x7f44c009d6f0 2023-09-30T22:33:02.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.869+0000 7f44c5745700 1 -- 172.21.15.16:0/185375847 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44c01211f0 con 0x7f44c009d6f0 2023-09-30T22:33:02.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.869+0000 7f44c5745700 1 -- 172.21.15.16:0/185375847 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44c009c700 con 0x7f44c009d6f0 2023-09-30T22:33:02.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.870+0000 7f44c5745700 1 -- 172.21.15.16:0/185375847 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44c0074ea0 con 0x7f44c009d6f0 2023-09-30T22:33:02.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.870+0000 7f44a77fe700 1 -- 172.21.15.16:0/185375847 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f44b0003b80 con 0x7f44c009d6f0 2023-09-30T22:33:02.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.870+0000 7f44a77fe700 1 -- 172.21.15.16:0/185375847 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f44b0021070 con 0x7f44c009d6f0 2023-09-30T22:33:02.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.871+0000 7fe0418a0700 1 -- 172.21.15.16:0/91856127 shutdown_connections 2023-09-30T22:33:02.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.871+0000 7fe0418a0700 1 --2- 172.21.15.16:0/91856127 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe03c06ee80 0x7fe03c134d20 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:02.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.871+0000 7fe0418a0700 1 --2- 172.21.15.16:0/91856127 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe03c06e570 0x7fe03c06e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:02.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.871+0000 7fe0418a0700 1 -- 172.21.15.16:0/91856127 >> 172.21.15.16:0/91856127 conn(0x7fe03c0a2aa0 msgr2=0x7fe03c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:02.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.873+0000 7fe0418a0700 1 -- 172.21.15.16:0/91856127 shutdown_connections 2023-09-30T22:33:02.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.873+0000 7fe0418a0700 1 -- 172.21.15.16:0/91856127 wait complete. 2023-09-30T22:33:02.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.874+0000 7fe0418a0700 1 Processor -- start 2023-09-30T22:33:02.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.874+0000 7fe0418a0700 1 -- start start 2023-09-30T22:33:02.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.874+0000 7fe0418a0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe03c06e570 0x7fe03c134c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:02.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.874+0000 7fe0418a0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe03c06ee80 0x7fe03c133410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:02.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.874+0000 7fe0418a0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe03c09dea0 0x7fe03c133950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:02.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.874+0000 7fe0418a0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe03c13a500 con 0x7fe03c09dea0 2023-09-30T22:33:02.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.874+0000 7fe0418a0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe03c135600 con 0x7fe03c06e570 2023-09-30T22:33:02.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.874+0000 7fe0418a0700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe03c13b8a0 con 0x7fe03c06ee80 2023-09-30T22:33:02.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.875+0000 7fe03b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe03c09dea0 0x7fe03c133950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:02.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.875+0000 7fe03b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe03c09dea0 0x7fe03c133950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37576/0 (socket says 172.21.15.16:37576) 2023-09-30T22:33:02.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.875+0000 7fe03b7fe700 1 -- 172.21.15.16:0/3515377028 learned_addr learned my addr 172.21.15.16:0/3515377028 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:02.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.875+0000 7fe03a7fc700 1 --2- 172.21.15.16:0/3515377028 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe03c06ee80 0x7fe03c133410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:02.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.875+0000 7fe03b7fe700 1 -- 172.21.15.16:0/3515377028 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe03c06ee80 msgr2=0x7fe03c133410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:02.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.875+0000 7fe03b7fe700 1 --2- 172.21.15.16:0/3515377028 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe03c06ee80 0x7fe03c133410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:02.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.875+0000 7fe03b7fe700 1 -- 172.21.15.16:0/3515377028 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe03c06e570 msgr2=0x7fe03c134c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:02.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.875+0000 7fe03b7fe700 1 --2- 172.21.15.16:0/3515377028 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe03c06e570 0x7fe03c134c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:02.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.875+0000 7fe03b7fe700 1 -- 172.21.15.16:0/3515377028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe02c0094a0 con 0x7fe03c09dea0 2023-09-30T22:33:02.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.876+0000 7fe03b7fe700 1 --2- 172.21.15.16:0/3515377028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe03c09dea0 0x7fe03c133950 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fe034014040 tx=0x7fe034004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:02.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.876+0000 7f44a77fe700 1 -- 172.21.15.16:0/185375847 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f44b0003e30 con 0x7f44c009d6f0 2023-09-30T22:33:02.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.876+0000 7fe023fff700 1 -- 172.21.15.16:0/3515377028 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0340048b0 con 0x7fe03c09dea0 2023-09-30T22:33:02.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.876+0000 7fe023fff700 1 -- 172.21.15.16:0/3515377028 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe034020070 con 0x7fe03c09dea0 2023-09-30T22:33:02.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.876+0000 7fe0418a0700 1 -- 172.21.15.16:0/3515377028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe034009a70 con 0x7fe03c09dea0 2023-09-30T22:33:02.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.876+0000 7fe023fff700 1 -- 172.21.15.16:0/3515377028 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0340276c0 con 0x7fe03c09dea0 2023-09-30T22:33:02.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.876+0000 7fe0418a0700 1 -- 172.21.15.16:0/3515377028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe03c13b200 con 0x7fe03c09dea0 2023-09-30T22:33:02.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.878+0000 7fe0418a0700 1 -- 172.21.15.16:0/3515377028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe03c135240 con 0x7fe03c09dea0 2023-09-30T22:33:02.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.878+0000 7fe023fff700 1 -- 172.21.15.16:0/3515377028 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe034012430 con 0x7fe03c09dea0 2023-09-30T22:33:02.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.879+0000 7fe023fff700 1 -- 172.21.15.16:0/3515377028 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fe034022070 con 0x7fe03c09dea0 2023-09-30T22:33:02.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:02.883+0000 7fe023fff700 1 -- 172.21.15.16:0/3515377028 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe034003e10 con 0x7fe03c09dea0 2023-09-30T22:33:03.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:03.072+0000 7f3f479cd700 1 -- 172.21.15.16:0/247404083 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f3f40074ea0 con 0x7f3f4009d6f0 2023-09-30T22:33:03.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:03.072+0000 7f3f35ffb700 1 -- 172.21.15.16:0/247404083 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3f3c017020 con 0x7f3f4009d6f0 2023-09-30T22:33:03.073 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:33:03.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:03.075+0000 7f3f479cd700 1 -- 172.21.15.16:0/247404083 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3f4009d6f0 msgr2=0x7f3f4006f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:03.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:03.075+0000 7f3f479cd700 1 --2- 172.21.15.16:0/247404083 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3f4009d6f0 0x7f3f4006f290 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c00c4c0 tx=0x7f3f3c00c7d0 comp rx=0 tx=0).stop 2023-09-30T22:33:03.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:03.075+0000 7f3f479cd700 1 -- 172.21.15.16:0/247404083 shutdown_connections 2023-09-30T22:33:03.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:03.075+0000 7f3f479cd700 1 --2- 172.21.15.16:0/247404083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f40135e50 0x7f3f401c80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:03.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:03.075+0000 7f3f479cd700 1 --2- 172.21.15.16:0/247404083 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3f4009d6f0 0x7f3f4006f290 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:03.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:03.075+0000 7f3f479cd700 1 --2- 172.21.15.16:0/247404083 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3f4009e1d0 0x7f3f4006f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:03.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:03.075+0000 7f3f479cd700 1 -- 172.21.15.16:0/247404083 >> 172.21.15.16:0/247404083 conn(0x7f3f40066f50 msgr2=0x7f3f40134810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:03.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:03.076+0000 7f3f479cd700 1 -- 172.21.15.16:0/247404083 shutdown_connections 2023-09-30T22:33:03.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:03.076+0000 7f3f479cd700 1 -- 172.21.15.16:0/247404083 wait complete. 2023-09-30T22:33:03.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:03.081+0000 7f44c5745700 1 -- 172.21.15.16:0/185375847 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f44c0121420 con 0x7f44c009d6f0 2023-09-30T22:33:03.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:03.082+0000 7f44a77fe700 1 -- 172.21.15.16:0/185375847 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f44b0024b70 con 0x7f44c009d6f0 2023-09-30T22:33:03.082 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:33:03.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:03.084+0000 7f44c5745700 1 -- 172.21.15.16:0/185375847 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f44c009d6f0 msgr2=0x7f44c011df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:03.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:03.084+0000 7f44c5745700 1 --2- 172.21.15.16:0/185375847 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f44c009d6f0 0x7f44c011df40 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f44b0009bc0 tx=0x7f44b0004750 comp rx=0 tx=0).stop 2023-09-30T22:33:03.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:03.084+0000 7f44c5745700 1 -- 172.21.15.16:0/185375847 shutdown_connections 2023-09-30T22:33:03.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:03.084+0000 7f44c5745700 1 --2- 172.21.15.16:0/185375847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44c009e410 0x7f44c011e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:03.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:03.084+0000 7f44c5745700 1 --2- 172.21.15.16:0/185375847 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f44c009d6f0 0x7f44c011df40 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:03.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:03.084+0000 7f44c5745700 1 --2- 172.21.15.16:0/185375847 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f44c009e000 0x7f44c011e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:03.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:03.084+0000 7f44c5745700 1 -- 172.21.15.16:0/185375847 >> 172.21.15.16:0/185375847 conn(0x7f44c0066f50 msgr2=0x7f44c0070ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:03.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:03.085+0000 7f44c5745700 1 -- 172.21.15.16:0/185375847 shutdown_connections 2023-09-30T22:33:03.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:03.085+0000 7f44c5745700 1 -- 172.21.15.16:0/185375847 wait complete. 2023-09-30T22:33:03.091 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:33:03.100 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:33:03.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:03.108+0000 7f932effd700 1 -- 172.21.15.16:0/4226880097 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9344074ea0 con 0x7f934409e000 2023-09-30T22:33:03.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:03.108+0000 7f93417fa700 1 -- 172.21.15.16:0/4226880097 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9330015070 con 0x7f934409e000 2023-09-30T22:33:03.109 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:33:03.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:03.110+0000 7f932effd700 1 -- 172.21.15.16:0/4226880097 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f934409e000 msgr2=0x7f9344122a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:03.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:03.110+0000 7f932effd700 1 --2- 172.21.15.16:0/4226880097 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f934409e000 0x7f9344122a00 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f93300090b0 tx=0x7f9330004750 comp rx=0 tx=0).stop 2023-09-30T22:33:03.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:03.110+0000 7f932effd700 1 -- 172.21.15.16:0/4226880097 shutdown_connections 2023-09-30T22:33:03.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:03.110+0000 7f932effd700 1 --2- 172.21.15.16:0/4226880097 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f934409e410 0x7f9344122f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:03.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:03.110+0000 7f932effd700 1 --2- 172.21.15.16:0/4226880097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f934409d6f0 0x7f93441224c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:03.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:03.110+0000 7f932effd700 1 --2- 172.21.15.16:0/4226880097 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f934409e000 0x7f9344122a00 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:03.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:03.110+0000 7f932effd700 1 -- 172.21.15.16:0/4226880097 >> 172.21.15.16:0/4226880097 conn(0x7f9344066f50 msgr2=0x7f934406a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:03.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:03.110+0000 7f932effd700 1 -- 172.21.15.16:0/4226880097 shutdown_connections 2023-09-30T22:33:03.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:03.110+0000 7f932effd700 1 -- 172.21.15.16:0/4226880097 wait complete. 2023-09-30T22:33:03.125 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:33:03.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:03.150+0000 7fe0418a0700 1 -- 172.21.15.16:0/3515377028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe03c135240 con 0x7fe03c09dea0 2023-09-30T22:33:03.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:03.150+0000 7fe023fff700 1 -- 172.21.15.16:0/3515377028 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe03c135240 con 0x7fe03c09dea0 2023-09-30T22:33:03.151 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:33:03.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:03.152+0000 7fe0418a0700 1 -- 172.21.15.16:0/3515377028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe03c09dea0 msgr2=0x7fe03c133950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:03.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:03.152+0000 7fe0418a0700 1 --2- 172.21.15.16:0/3515377028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe03c09dea0 0x7fe03c133950 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fe034014040 tx=0x7fe034004ad0 comp rx=0 tx=0).stop 2023-09-30T22:33:03.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:03.152+0000 7fe0418a0700 1 -- 172.21.15.16:0/3515377028 shutdown_connections 2023-09-30T22:33:03.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:03.152+0000 7fe0418a0700 1 --2- 172.21.15.16:0/3515377028 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe03c06e570 0x7fe03c134c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:03.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:03.152+0000 7fe0418a0700 1 --2- 172.21.15.16:0/3515377028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe03c09dea0 0x7fe03c133950 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:03.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:03.152+0000 7fe0418a0700 1 --2- 172.21.15.16:0/3515377028 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe03c06ee80 0x7fe03c133410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:03.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:03.152+0000 7fe0418a0700 1 -- 172.21.15.16:0/3515377028 >> 172.21.15.16:0/3515377028 conn(0x7fe03c0a2aa0 msgr2=0x7fe03c0a1410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:03.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:03.152+0000 7fe0418a0700 1 -- 172.21.15.16:0/3515377028 shutdown_connections 2023-09-30T22:33:03.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:03.152+0000 7fe0418a0700 1 -- 172.21.15.16:0/3515377028 wait complete. 2023-09-30T22:33:03.166 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:33:04.092 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:33:04.101 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:33:04.126 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:33:04.167 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:33:04.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.247+0000 7f6c643f6700 1 Processor -- start 2023-09-30T22:33:04.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.247+0000 7f6c643f6700 1 -- start start 2023-09-30T22:33:04.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.248+0000 7f6c643f6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6c5c12cf60 0x7f6c5c06f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:04.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.248+0000 7f6c643f6700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6c5c130430 0x7f6c5c137c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:04.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.248+0000 7f6c643f6700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c5c12cd90 con 0x7f6c5c12f8b0 2023-09-30T22:33:04.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.248+0000 7f6c643f6700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c5c12d330 con 0x7f6c5c130430 2023-09-30T22:33:04.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.248+0000 7f6c643f6700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5c071780 con 0x7f6c5c12cf60 2023-09-30T22:33:04.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.248+0000 7f6c62993700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6c5c130430 0x7f6c5c137c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:04.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.248+0000 7f6c62993700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6c5c130430 0x7f6c5c137c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37590/0 (socket says 172.21.15.16:37590) 2023-09-30T22:33:04.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.248+0000 7f6c62993700 1 -- 172.21.15.16:0/3211120559 learned_addr learned my addr 172.21.15.16:0/3211120559 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:04.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.248+0000 7f6c62192700 1 --2- 172.21.15.16:0/3211120559 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6c5c12cf60 0x7f6c5c06f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:04.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.248+0000 7f6c62993700 1 -- 172.21.15.16:0/3211120559 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6c5c12cf60 msgr2=0x7f6c5c06f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:04.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.248+0000 7f6c62993700 1 --2- 172.21.15.16:0/3211120559 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6c5c12cf60 0x7f6c5c06f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:04.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.248+0000 7f6c62993700 1 -- 172.21.15.16:0/3211120559 >> v1:172.21.15.16:6789/0 conn(0x7f6c5c12f8b0 legacy=0x7f6c5c12fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:04.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.248+0000 7f6c62993700 1 -- 172.21.15.16:0/3211120559 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c5c071a40 con 0x7f6c5c130430 2023-09-30T22:33:04.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.249+0000 7f6c62993700 1 --2- 172.21.15.16:0/3211120559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c5c130430 0x7f6c5c137c30 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f6c54009d40 tx=0x7f6c5400c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d476077d4ff6cd33 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:04.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.249+0000 7f6c61190700 1 -- 172.21.15.16:0/3211120559 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c54009e90 con 0x7f6c5c130430 2023-09-30T22:33:04.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.249+0000 7f6c61190700 1 -- 172.21.15.16:0/3211120559 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c5400ca20 con 0x7f6c5c130430 2023-09-30T22:33:04.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.249+0000 7f6c61190700 1 -- 172.21.15.16:0/3211120559 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5400cdd0 con 0x7f6c5c130430 2023-09-30T22:33:04.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.250+0000 7f6c643f6700 1 -- 172.21.15.16:0/3211120559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c5c130430 msgr2=0x7f6c5c137c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:04.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.250+0000 7f6c643f6700 1 --2- 172.21.15.16:0/3211120559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c5c130430 0x7f6c5c137c30 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f6c54009d40 tx=0x7f6c5400c440 comp rx=0 tx=0).stop 2023-09-30T22:33:04.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.250+0000 7f6c643f6700 1 -- 172.21.15.16:0/3211120559 shutdown_connections 2023-09-30T22:33:04.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.250+0000 7f6c643f6700 1 --2- 172.21.15.16:0/3211120559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c5c130430 0x7f6c5c137c30 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:04.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.250+0000 7f6c643f6700 1 --2- 172.21.15.16:0/3211120559 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6c5c12cf60 0x7f6c5c06f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:04.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.250+0000 7f6c643f6700 1 -- 172.21.15.16:0/3211120559 >> 172.21.15.16:0/3211120559 conn(0x7f6c5c0a2aa0 msgr2=0x7f6c5c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:04.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.250+0000 7f6c643f6700 1 -- 172.21.15.16:0/3211120559 shutdown_connections 2023-09-30T22:33:04.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.250+0000 7f6c643f6700 1 -- 172.21.15.16:0/3211120559 wait complete. 2023-09-30T22:33:04.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.251+0000 7f6c643f6700 1 Processor -- start 2023-09-30T22:33:04.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.251+0000 7f6c643f6700 1 -- start start 2023-09-30T22:33:04.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.251+0000 7f6c643f6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6c5c12cf60 0x7f6c5c09bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:04.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.251+0000 7f6c643f6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c5c12f8b0 0x7f6c5c09c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:04.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.251+0000 7f6c643f6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6c5c130430 0x7f6c5c1c82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:04.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.252+0000 7f6c643f6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5c09b740 con 0x7f6c5c12f8b0 2023-09-30T22:33:04.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.252+0000 7f6c643f6700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5c09e960 con 0x7f6c5c130430 2023-09-30T22:33:04.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.252+0000 7f6c643f6700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5c09c5a0 con 0x7f6c5c12cf60 2023-09-30T22:33:04.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.252+0000 7f6c62993700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6c5c130430 0x7f6c5c1c82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:04.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.252+0000 7f6c62993700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6c5c130430 0x7f6c5c1c82e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36688/0 (socket says 172.21.15.16:36688) 2023-09-30T22:33:04.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.252+0000 7f6c62993700 1 -- 172.21.15.16:0/598356941 learned_addr learned my addr 172.21.15.16:0/598356941 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:04.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.252+0000 7f6c61991700 1 --2- 172.21.15.16:0/598356941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c5c12f8b0 0x7f6c5c09c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:04.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.252+0000 7f6c62993700 1 -- 172.21.15.16:0/598356941 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6c5c12cf60 msgr2=0x7f6c5c09bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:04.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.252+0000 7f6c62993700 1 --2- 172.21.15.16:0/598356941 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6c5c12cf60 0x7f6c5c09bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:04.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.252+0000 7f6c62993700 1 -- 172.21.15.16:0/598356941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c5c12f8b0 msgr2=0x7f6c5c09c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:04.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.252+0000 7f6c62993700 1 --2- 172.21.15.16:0/598356941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c5c12f8b0 0x7f6c5c09c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:04.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.252+0000 7f6c62993700 1 -- 172.21.15.16:0/598356941 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c5400d040 con 0x7f6c5c130430 2023-09-30T22:33:04.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.252+0000 7f6c62993700 1 --2- 172.21.15.16:0/598356941 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6c5c130430 0x7f6c5c1c82e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c06fba0 tx=0x7f6c54004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:04.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.253+0000 7f6c52ffd700 1 -- 172.21.15.16:0/598356941 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c54009e90 con 0x7f6c5c130430 2023-09-30T22:33:04.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.253+0000 7f6c52ffd700 1 -- 172.21.15.16:0/598356941 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c5400cda0 con 0x7f6c5c130430 2023-09-30T22:33:04.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.253+0000 7f6c52ffd700 1 -- 172.21.15.16:0/598356941 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c54024690 con 0x7f6c5c130430 2023-09-30T22:33:04.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.253+0000 7f6c643f6700 1 -- 172.21.15.16:0/598356941 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c5c09e700 con 0x7f6c5c130430 2023-09-30T22:33:04.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.253+0000 7f6c643f6700 1 -- 172.21.15.16:0/598356941 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c5c09cdd0 con 0x7f6c5c130430 2023-09-30T22:33:04.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.254+0000 7f6c52ffd700 1 -- 172.21.15.16:0/598356941 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6c54021070 con 0x7f6c5c130430 2023-09-30T22:33:04.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.254+0000 7f6c52ffd700 1 -- 172.21.15.16:0/598356941 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f6c54044350 con 0x7f6c5c130430 2023-09-30T22:33:04.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.254+0000 7f6c643f6700 1 -- 172.21.15.16:0/598356941 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c5c09dea0 con 0x7f6c5c130430 2023-09-30T22:33:04.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.259+0000 7f6c52ffd700 1 -- 172.21.15.16:0/598356941 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6c54015070 con 0x7f6c5c130430 2023-09-30T22:33:04.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.285+0000 7fdba64e6700 1 Processor -- start 2023-09-30T22:33:04.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.285+0000 7fdba64e6700 1 -- start start 2023-09-30T22:33:04.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.286+0000 7fdba64e6700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdba009e930 0x7fdba009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:04.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.286+0000 7fdba64e6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdba009f240 0x7fdba01317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:04.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.286+0000 7fdba64e6700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdba0070940 con 0x7fdba009f650 2023-09-30T22:33:04.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.286+0000 7fdba64e6700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdba006fd60 con 0x7fdba009e930 2023-09-30T22:33:04.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.286+0000 7fdba64e6700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdba006fa80 con 0x7fdba009f240 2023-09-30T22:33:04.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.286+0000 7fdb9ffff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdba009e930 0x7fdba009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:04.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.286+0000 7fdb9ffff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdba009e930 0x7fdba009ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37610/0 (socket says 172.21.15.16:37610) 2023-09-30T22:33:04.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.286+0000 7fdb9ffff700 1 -- 172.21.15.16:0/4263444867 learned_addr learned my addr 172.21.15.16:0/4263444867 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:04.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.286+0000 7fdb9f7fe700 1 --2- 172.21.15.16:0/4263444867 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdba009f240 0x7fdba01317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:04.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.286+0000 7fdb9f7fe700 1 -- 172.21.15.16:0/4263444867 >> v2:172.21.15.16:3300/0 conn(0x7fdba009e930 msgr2=0x7fdba009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:04.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.286+0000 7fdb9f7fe700 1 --2- 172.21.15.16:0/4263444867 >> v2:172.21.15.16:3300/0 conn(0x7fdba009e930 0x7fdba009ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:04.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.286+0000 7fdb9f7fe700 1 -- 172.21.15.16:0/4263444867 >> v1:172.21.15.16:6789/0 conn(0x7fdba009f650 legacy=0x7fdba0131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:04.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.286+0000 7fdb9f7fe700 1 -- 172.21.15.16:0/4263444867 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba009fb60 con 0x7fdba009f240 2023-09-30T22:33:04.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.287+0000 7fdb9ffff700 1 --2- 172.21.15.16:0/4263444867 >> v2:172.21.15.16:3300/0 conn(0x7fdba009e930 0x7fdba009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:04.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.287+0000 7fdb9f7fe700 1 --2- 172.21.15.16:0/4263444867 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdba009f240 0x7fdba01317f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fdb94009840 tx=0x7fdb94009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b7065ffb2b03c45 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:04.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.287+0000 7fdb9effd700 1 -- 172.21.15.16:0/4263444867 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb94011830 con 0x7fdba009f240 2023-09-30T22:33:04.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.287+0000 7fdb9effd700 1 -- 172.21.15.16:0/4263444867 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb94011990 con 0x7fdba009f240 2023-09-30T22:33:04.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.287+0000 7fdb9effd700 1 -- 172.21.15.16:0/4263444867 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb94011d20 con 0x7fdba009f240 2023-09-30T22:33:04.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.288+0000 7fdba64e6700 1 -- 172.21.15.16:0/4263444867 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdba009f240 msgr2=0x7fdba01317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:04.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.288+0000 7fdba64e6700 1 --2- 172.21.15.16:0/4263444867 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdba009f240 0x7fdba01317f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fdb94009840 tx=0x7fdb94009b50 comp rx=0 tx=0).stop 2023-09-30T22:33:04.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.288+0000 7fdba64e6700 1 -- 172.21.15.16:0/4263444867 shutdown_connections 2023-09-30T22:33:04.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.288+0000 7fdba64e6700 1 --2- 172.21.15.16:0/4263444867 >> v2:172.21.15.16:3300/0 conn(0x7fdba009e930 0x7fdba009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:04.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.288+0000 7fdba64e6700 1 --2- 172.21.15.16:0/4263444867 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdba009f240 0x7fdba01317f0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:04.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.288+0000 7fdba64e6700 1 -- 172.21.15.16:0/4263444867 >> 172.21.15.16:0/4263444867 conn(0x7fdba00a4210 msgr2=0x7fdba00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:04.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.288+0000 7fdba64e6700 1 -- 172.21.15.16:0/4263444867 shutdown_connections 2023-09-30T22:33:04.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.288+0000 7fdba64e6700 1 -- 172.21.15.16:0/4263444867 wait complete. 2023-09-30T22:33:04.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.288+0000 7fb16ab64700 1 Processor -- start 2023-09-30T22:33:04.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.288+0000 7fb16ab64700 1 -- start start 2023-09-30T22:33:04.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.288+0000 7fb16ab64700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb16409c5e0 0x7fb16409c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:04.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.288+0000 7fb16ab64700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb16409cef0 0x7fb16413bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:04.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.289+0000 7fb16ab64700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb16409a830 con 0x7fb16413c470 2023-09-30T22:33:04.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.289+0000 7fb16ab64700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb16409bff0 con 0x7fb16409c5e0 2023-09-30T22:33:04.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.289+0000 7fb16ab64700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb164098ab0 con 0x7fb16409cef0 2023-09-30T22:33:04.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.289+0000 7fb163fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb16409cef0 0x7fb16413bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:04.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.289+0000 7fb163fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb16409cef0 0x7fb16413bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36692/0 (socket says 172.21.15.16:36692) 2023-09-30T22:33:04.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.289+0000 7fb163fff700 1 -- 172.21.15.16:0/2479555941 learned_addr learned my addr 172.21.15.16:0/2479555941 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:04.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.289+0000 7fb163fff700 1 -- 172.21.15.16:0/2479555941 >> v2:172.21.15.16:3300/0 conn(0x7fb16409c5e0 msgr2=0x7fb16409c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:04.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.289+0000 7fb163fff700 1 --2- 172.21.15.16:0/2479555941 >> v2:172.21.15.16:3300/0 conn(0x7fb16409c5e0 0x7fb16409c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:04.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.289+0000 7fb163fff700 1 -- 172.21.15.16:0/2479555941 >> v1:172.21.15.16:6789/0 conn(0x7fb16413c470 legacy=0x7fb16413e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:04.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.289+0000 7fb163fff700 1 -- 172.21.15.16:0/2479555941 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb16409c1a0 con 0x7fb16409cef0 2023-09-30T22:33:04.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.289+0000 7fb163fff700 1 --2- 172.21.15.16:0/2479555941 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb16409cef0 0x7fb16413bf30 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fb1500098f0 tx=0x7fb150009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=454830a7f7c5563a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:04.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.289+0000 7fdba64e6700 1 Processor -- start 2023-09-30T22:33:04.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.290+0000 7fdba64e6700 1 -- start start 2023-09-30T22:33:04.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.290+0000 7fdba64e6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdba009e930 0x7fdba01a3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:04.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.290+0000 7fdba64e6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdba009f240 0x7fdba01a4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:04.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.290+0000 7fdba64e6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdba009f650 0x7fdba01a4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:04.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.290+0000 7fdba64e6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba01a5860 con 0x7fdba009e930 2023-09-30T22:33:04.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.290+0000 7fdba64e6700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdba0071d20 con 0x7fdba009f650 2023-09-30T22:33:04.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.290+0000 7fdba64e6700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdba01a7490 con 0x7fdba009f240 2023-09-30T22:33:04.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.290+0000 7fb1637fe700 1 -- 172.21.15.16:0/2479555941 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb150004230 con 0x7fb16409cef0 2023-09-30T22:33:04.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.290+0000 7fb1637fe700 1 -- 172.21.15.16:0/2479555941 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb150004390 con 0x7fb16409cef0 2023-09-30T22:33:04.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.290+0000 7fb1637fe700 1 -- 172.21.15.16:0/2479555941 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1500036a0 con 0x7fb16409cef0 2023-09-30T22:33:04.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.290+0000 7fdba4a83700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdba009f650 0x7fdba01a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:04.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.290+0000 7fdba4a83700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdba009f650 0x7fdba01a4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36704/0 (socket says 172.21.15.16:36704) 2023-09-30T22:33:04.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.290+0000 7fdba4a83700 1 -- 172.21.15.16:0/2364504772 learned_addr learned my addr 172.21.15.16:0/2364504772 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:04.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.291+0000 7fdba4a83700 1 -- 172.21.15.16:0/2364504772 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdba009f240 msgr2=0x7fdba01a4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:04.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.291+0000 7fdba4a83700 1 --2- 172.21.15.16:0/2364504772 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdba009f240 0x7fdba01a4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:04.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.291+0000 7fdba4a83700 1 -- 172.21.15.16:0/2364504772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdba009e930 msgr2=0x7fdba01a3cd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:33:04.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.291+0000 7fdba4a83700 1 --2- 172.21.15.16:0/2364504772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdba009e930 0x7fdba01a3cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:04.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.291+0000 7fdba4a83700 1 -- 172.21.15.16:0/2364504772 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb90009550 con 0x7fdba009f650 2023-09-30T22:33:04.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.291+0000 7fdba4a83700 1 --2- 172.21.15.16:0/2364504772 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdba009f650 0x7fdba01a4750 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fdb9800cc50 tx=0x7fdb9800a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:04.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.291+0000 7fdb9cff9700 1 -- 172.21.15.16:0/2364504772 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9800ad80 con 0x7fdba009f650 2023-09-30T22:33:04.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.291+0000 7fdb9cff9700 1 -- 172.21.15.16:0/2364504772 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb9800ce50 con 0x7fdba009f650 2023-09-30T22:33:04.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.291+0000 7fdba64e6700 1 -- 172.21.15.16:0/2364504772 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb940094a0 con 0x7fdba009f650 2023-09-30T22:33:04.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.291+0000 7fdb9cff9700 1 -- 172.21.15.16:0/2364504772 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb98003c40 con 0x7fdba009f650 2023-09-30T22:33:04.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.291+0000 7fdba64e6700 1 -- 172.21.15.16:0/2364504772 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdba00a0190 con 0x7fdba009f650 2023-09-30T22:33:04.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.291+0000 7fb16ab64700 1 -- 172.21.15.16:0/2479555941 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb16409cef0 msgr2=0x7fb16413bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:04.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.291+0000 7fb16ab64700 1 --2- 172.21.15.16:0/2479555941 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb16409cef0 0x7fb16413bf30 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fb1500098f0 tx=0x7fb150009c00 comp rx=0 tx=0).stop 2023-09-30T22:33:04.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.292+0000 7fb16ab64700 1 -- 172.21.15.16:0/2479555941 shutdown_connections 2023-09-30T22:33:04.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.292+0000 7fb16ab64700 1 --2- 172.21.15.16:0/2479555941 >> v2:172.21.15.16:3300/0 conn(0x7fb16409c5e0 0x7fb16409c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:04.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.292+0000 7fb16ab64700 1 --2- 172.21.15.16:0/2479555941 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb16409cef0 0x7fb16413bf30 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:04.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.292+0000 7fb16ab64700 1 -- 172.21.15.16:0/2479555941 >> 172.21.15.16:0/2479555941 conn(0x7fb16409e7e0 msgr2=0x7fb16409ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:04.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.292+0000 7fb16ab64700 1 -- 172.21.15.16:0/2479555941 shutdown_connections 2023-09-30T22:33:04.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.292+0000 7fb16ab64700 1 -- 172.21.15.16:0/2479555941 wait complete. 2023-09-30T22:33:04.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.293+0000 7fdb9cff9700 1 -- 172.21.15.16:0/2364504772 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdb980050d0 con 0x7fdba009f650 2023-09-30T22:33:04.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.293+0000 7fdba64e6700 1 -- 172.21.15.16:0/2364504772 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdba01a0b30 con 0x7fdba009f650 2023-09-30T22:33:04.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.293+0000 7fdb9cff9700 1 -- 172.21.15.16:0/2364504772 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fdb9803f890 con 0x7fdba009f650 2023-09-30T22:33:04.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.293+0000 7fb16ab64700 1 Processor -- start 2023-09-30T22:33:04.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.294+0000 7fb16ab64700 1 -- start start 2023-09-30T22:33:04.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.294+0000 7fb16ab64700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb16409c5e0 0x7fb1640959e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:04.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.294+0000 7fb16ab64700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb16409cef0 0x7fb164095f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:04.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.294+0000 7fb16ab64700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb16413c470 0x7fb164096460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:04.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.294+0000 7fb16ab64700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb164138750 con 0x7fb16409cef0 2023-09-30T22:33:04.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.294+0000 7fb16ab64700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb164098260 con 0x7fb16409c5e0 2023-09-30T22:33:04.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.294+0000 7fb16ab64700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb1640975f0 con 0x7fb16413c470 2023-09-30T22:33:04.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.294+0000 7fb168900700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb16409c5e0 0x7fb1640959e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:04.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.294+0000 7fb169101700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb16413c470 0x7fb164096460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:04.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.294+0000 7fb168900700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb16409c5e0 0x7fb1640959e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36712/0 (socket says 172.21.15.16:36712) 2023-09-30T22:33:04.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.294+0000 7fb168900700 1 -- 172.21.15.16:0/2408272543 learned_addr learned my addr 172.21.15.16:0/2408272543 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:04.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.295+0000 7fb169101700 1 -- 172.21.15.16:0/2408272543 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb16409c5e0 msgr2=0x7fb1640959e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:04.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.295+0000 7fb169101700 1 --2- 172.21.15.16:0/2408272543 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb16409c5e0 0x7fb1640959e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:04.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.295+0000 7fb169101700 1 -- 172.21.15.16:0/2408272543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb16409cef0 msgr2=0x7fb164095f20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:33:04.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.295+0000 7fb169101700 1 --2- 172.21.15.16:0/2408272543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb16409cef0 0x7fb164095f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:04.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.295+0000 7fb169101700 1 -- 172.21.15.16:0/2408272543 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb150009550 con 0x7fb16413c470 2023-09-30T22:33:04.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.295+0000 7fb168900700 1 --2- 172.21.15.16:0/2408272543 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb16409c5e0 0x7fb1640959e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:04.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.295+0000 7fb169101700 1 --2- 172.21.15.16:0/2408272543 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb16413c470 0x7fb164096460 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fb15c00c750 tx=0x7fb15c00ca60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:04.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.295+0000 7fb1617fa700 1 -- 172.21.15.16:0/2408272543 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb15c003e10 con 0x7fb16413c470 2023-09-30T22:33:04.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.295+0000 7fb1617fa700 1 -- 172.21.15.16:0/2408272543 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb15c0050f0 con 0x7fb16413c470 2023-09-30T22:33:04.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.295+0000 7fb1617fa700 1 -- 172.21.15.16:0/2408272543 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb15c0054a0 con 0x7fb16413c470 2023-09-30T22:33:04.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.296+0000 7fb16ab64700 1 -- 172.21.15.16:0/2408272543 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1641276d0 con 0x7fb16413c470 2023-09-30T22:33:04.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.296+0000 7fb16ab64700 1 -- 172.21.15.16:0/2408272543 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1641280e0 con 0x7fb16413c470 2023-09-30T22:33:04.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.297+0000 7fb1617fa700 1 -- 172.21.15.16:0/2408272543 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb15c005600 con 0x7fb16413c470 2023-09-30T22:33:04.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.297+0000 7fb1617fa700 1 -- 172.21.15.16:0/2408272543 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fb15c03eeb0 con 0x7fb16413c470 2023-09-30T22:33:04.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.298+0000 7fb16ab64700 1 -- 172.21.15.16:0/2408272543 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1640a13d0 con 0x7fb16413c470 2023-09-30T22:33:04.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.300+0000 7fdb9cff9700 1 -- 172.21.15.16:0/2364504772 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdb98012420 con 0x7fdba009f650 2023-09-30T22:33:04.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.304+0000 7fb1617fa700 1 -- 172.21.15.16:0/2408272543 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb15c017020 con 0x7fb16413c470 2023-09-30T22:33:04.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.339+0000 7f7a3ff25700 1 Processor -- start 2023-09-30T22:33:04.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.339+0000 7f7a3ff25700 1 -- start start 2023-09-30T22:33:04.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.340+0000 7f7a3ff25700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7a380a0a30 0x7f7a380a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:04.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.340+0000 7f7a3ff25700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7a38134170 0x7f7a3813bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:04.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.340+0000 7f7a3ff25700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a3809edd0 con 0x7f7a380a1340 2023-09-30T22:33:04.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.340+0000 7f7a3ff25700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a3809e1f0 con 0x7f7a38134170 2023-09-30T22:33:04.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.340+0000 7f7a3ff25700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3809dc50 con 0x7f7a380a0a30 2023-09-30T22:33:04.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.340+0000 7f7a3dcc1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7a380a0a30 0x7f7a380a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:04.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.340+0000 7f7a3dcc1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7a380a0a30 0x7f7a380a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36714/0 (socket says 172.21.15.16:36714) 2023-09-30T22:33:04.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.340+0000 7f7a3dcc1700 1 -- 172.21.15.16:0/382470060 learned_addr learned my addr 172.21.15.16:0/382470060 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:04.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.340+0000 7f7a3e4c2700 1 --2- 172.21.15.16:0/382470060 >> v2:172.21.15.16:3300/0 conn(0x7f7a38134170 0x7f7a3813bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:04.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.341+0000 7f7a3dcc1700 1 -- 172.21.15.16:0/382470060 >> v2:172.21.15.16:3300/0 conn(0x7f7a38134170 msgr2=0x7f7a3813bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:04.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.341+0000 7f7a3dcc1700 1 --2- 172.21.15.16:0/382470060 >> v2:172.21.15.16:3300/0 conn(0x7f7a38134170 0x7f7a3813bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:04.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.341+0000 7f7a3dcc1700 1 -- 172.21.15.16:0/382470060 >> v1:172.21.15.16:6789/0 conn(0x7f7a380a1340 legacy=0x7f7a38133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:04.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.341+0000 7f7a3dcc1700 1 -- 172.21.15.16:0/382470060 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a380a1fe0 con 0x7f7a380a0a30 2023-09-30T22:33:04.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.341+0000 7f7a3dcc1700 1 --2- 172.21.15.16:0/382470060 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7a380a0a30 0x7f7a380a0e00 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f7a340098f0 tx=0x7f7a34009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba02d37d37107c4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:04.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.341+0000 7f7a3ccbf700 1 -- 172.21.15.16:0/382470060 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a34011830 con 0x7f7a380a0a30 2023-09-30T22:33:04.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.341+0000 7f7a3ccbf700 1 -- 172.21.15.16:0/382470060 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a34011990 con 0x7f7a380a0a30 2023-09-30T22:33:04.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.341+0000 7f7a3ccbf700 1 -- 172.21.15.16:0/382470060 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a34011d20 con 0x7f7a380a0a30 2023-09-30T22:33:04.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.342+0000 7f7a3ff25700 1 -- 172.21.15.16:0/382470060 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7a380a0a30 msgr2=0x7f7a380a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:04.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.342+0000 7f7a3ff25700 1 --2- 172.21.15.16:0/382470060 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7a380a0a30 0x7f7a380a0e00 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f7a340098f0 tx=0x7f7a34009c00 comp rx=0 tx=0).stop 2023-09-30T22:33:04.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.342+0000 7f7a3ff25700 1 -- 172.21.15.16:0/382470060 shutdown_connections 2023-09-30T22:33:04.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.342+0000 7f7a3ff25700 1 --2- 172.21.15.16:0/382470060 >> v2:172.21.15.16:3300/0 conn(0x7f7a38134170 0x7f7a3813bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:04.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.342+0000 7f7a3ff25700 1 --2- 172.21.15.16:0/382470060 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7a380a0a30 0x7f7a380a0e00 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:04.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.342+0000 7f7a3ff25700 1 -- 172.21.15.16:0/382470060 >> 172.21.15.16:0/382470060 conn(0x7f7a380a6550 msgr2=0x7f7a380a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:04.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.342+0000 7f7a3ff25700 1 -- 172.21.15.16:0/382470060 shutdown_connections 2023-09-30T22:33:04.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.342+0000 7f7a3ff25700 1 -- 172.21.15.16:0/382470060 wait complete. 2023-09-30T22:33:04.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.343+0000 7f7a3ff25700 1 Processor -- start 2023-09-30T22:33:04.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.344+0000 7f7a3ff25700 1 -- start start 2023-09-30T22:33:04.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.344+0000 7f7a3ff25700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7a380a0a30 0x7f7a381c8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:04.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.344+0000 7f7a3ff25700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a380a1340 0x7f7a381c89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:04.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.344+0000 7f7a3ff25700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7a38134170 0x7f7a381c8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:04.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.344+0000 7f7a3ff25700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a381244d0 con 0x7f7a380a1340 2023-09-30T22:33:04.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.344+0000 7f7a3ff25700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a38124da0 con 0x7f7a380a0a30 2023-09-30T22:33:04.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.344+0000 7f7a3ff25700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7a381263e0 con 0x7f7a38134170 2023-09-30T22:33:04.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.345+0000 7f7a3dcc1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7a380a0a30 0x7f7a381c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:04.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.345+0000 7f7a3dcc1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7a380a0a30 0x7f7a381c8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36718/0 (socket says 172.21.15.16:36718) 2023-09-30T22:33:04.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.345+0000 7f7a3dcc1700 1 -- 172.21.15.16:0/248691055 learned_addr learned my addr 172.21.15.16:0/248691055 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:04.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.345+0000 7f7a3d4c0700 1 --2- 172.21.15.16:0/248691055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a380a1340 0x7f7a381c89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:04.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.345+0000 7f7a3e4c2700 1 --2- 172.21.15.16:0/248691055 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7a38134170 0x7f7a381c8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:04.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.345+0000 7f7a3dcc1700 1 -- 172.21.15.16:0/248691055 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7a38134170 msgr2=0x7f7a381c8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:04.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.345+0000 7f7a3dcc1700 1 --2- 172.21.15.16:0/248691055 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7a38134170 0x7f7a381c8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:04.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.345+0000 7f7a3dcc1700 1 -- 172.21.15.16:0/248691055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a380a1340 msgr2=0x7f7a381c89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:04.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.345+0000 7f7a3dcc1700 1 --2- 172.21.15.16:0/248691055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a380a1340 0x7f7a381c89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:04.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.345+0000 7f7a3dcc1700 1 -- 172.21.15.16:0/248691055 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a34009550 con 0x7f7a380a0a30 2023-09-30T22:33:04.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.345+0000 7f7a3dcc1700 1 --2- 172.21.15.16:0/248691055 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7a380a0a30 0x7f7a381c8460 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f7a34000c00 tx=0x7f7a34004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:04.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.345+0000 7f7a2e7fc700 1 -- 172.21.15.16:0/248691055 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a34004400 con 0x7f7a380a0a30 2023-09-30T22:33:04.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.345+0000 7f7a2e7fc700 1 -- 172.21.15.16:0/248691055 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a3401f070 con 0x7f7a380a0a30 2023-09-30T22:33:04.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.345+0000 7f7a3ff25700 1 -- 172.21.15.16:0/248691055 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a38124a80 con 0x7f7a380a0a30 2023-09-30T22:33:04.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.345+0000 7f7a2e7fc700 1 -- 172.21.15.16:0/248691055 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a340038e0 con 0x7f7a380a0a30 2023-09-30T22:33:04.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.345+0000 7f7a3ff25700 1 -- 172.21.15.16:0/248691055 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a38126100 con 0x7f7a380a0a30 2023-09-30T22:33:04.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.346+0000 7f7a3ff25700 1 -- 172.21.15.16:0/248691055 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a38074ea0 con 0x7f7a380a0a30 2023-09-30T22:33:04.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.347+0000 7f7a2e7fc700 1 -- 172.21.15.16:0/248691055 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7a34021030 con 0x7f7a380a0a30 2023-09-30T22:33:04.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.347+0000 7f7a2e7fc700 1 -- 172.21.15.16:0/248691055 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f7a34040960 con 0x7f7a380a0a30 2023-09-30T22:33:04.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.352+0000 7f7a2e7fc700 1 -- 172.21.15.16:0/248691055 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7a340199a0 con 0x7f7a380a0a30 2023-09-30T22:33:04.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.511+0000 7fb16ab64700 1 -- 172.21.15.16:0/2408272543 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb1640747f0 con 0x7fb16413c470 2023-09-30T22:33:04.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.511+0000 7fb1617fa700 1 -- 172.21.15.16:0/2408272543 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb15c01bb10 con 0x7fb16413c470 2023-09-30T22:33:04.512 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:33:04.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.515+0000 7fb16ab64700 1 -- 172.21.15.16:0/2408272543 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb16413c470 msgr2=0x7fb164096460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:04.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.515+0000 7fb16ab64700 1 --2- 172.21.15.16:0/2408272543 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb16413c470 0x7fb164096460 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fb15c00c750 tx=0x7fb15c00ca60 comp rx=0 tx=0).stop 2023-09-30T22:33:04.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.515+0000 7fb16ab64700 1 -- 172.21.15.16:0/2408272543 shutdown_connections 2023-09-30T22:33:04.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.515+0000 7fb16ab64700 1 --2- 172.21.15.16:0/2408272543 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb16409c5e0 0x7fb1640959e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:04.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.515+0000 7fb16ab64700 1 --2- 172.21.15.16:0/2408272543 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb16413c470 0x7fb164096460 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:04.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.515+0000 7fb16ab64700 1 --2- 172.21.15.16:0/2408272543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb16409cef0 0x7fb164095f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:04.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.515+0000 7fb16ab64700 1 -- 172.21.15.16:0/2408272543 >> 172.21.15.16:0/2408272543 conn(0x7fb16409e7e0 msgr2=0x7fb16409fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:04.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.516+0000 7fb16ab64700 1 -- 172.21.15.16:0/2408272543 shutdown_connections 2023-09-30T22:33:04.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.516+0000 7fb16ab64700 1 -- 172.21.15.16:0/2408272543 wait complete. 2023-09-30T22:33:04.530 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:33:04.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.556+0000 7f6c643f6700 1 -- 172.21.15.16:0/598356941 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6c5c074ea0 con 0x7f6c5c130430 2023-09-30T22:33:04.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.557+0000 7f6c52ffd700 1 -- 172.21.15.16:0/598356941 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6c54015020 con 0x7f6c5c130430 2023-09-30T22:33:04.557 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:33:04.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.559+0000 7f6c643f6700 1 -- 172.21.15.16:0/598356941 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6c5c130430 msgr2=0x7f6c5c1c82e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:04.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.559+0000 7f6c643f6700 1 --2- 172.21.15.16:0/598356941 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6c5c130430 0x7f6c5c1c82e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c06fba0 tx=0x7f6c54004ad0 comp rx=0 tx=0).stop 2023-09-30T22:33:04.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.559+0000 7f6c643f6700 1 -- 172.21.15.16:0/598356941 shutdown_connections 2023-09-30T22:33:04.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.559+0000 7f6c643f6700 1 --2- 172.21.15.16:0/598356941 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6c5c130430 0x7f6c5c1c82e0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:04.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.559+0000 7f6c643f6700 1 --2- 172.21.15.16:0/598356941 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6c5c12cf60 0x7f6c5c09bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:04.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.559+0000 7f6c643f6700 1 --2- 172.21.15.16:0/598356941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c5c12f8b0 0x7f6c5c09c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:04.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.559+0000 7f6c643f6700 1 -- 172.21.15.16:0/598356941 >> 172.21.15.16:0/598356941 conn(0x7f6c5c0a2aa0 msgr2=0x7f6c5c06e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:04.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.559+0000 7f6c643f6700 1 -- 172.21.15.16:0/598356941 shutdown_connections 2023-09-30T22:33:04.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.559+0000 7f6c643f6700 1 -- 172.21.15.16:0/598356941 wait complete. 2023-09-30T22:33:04.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.562+0000 7f7a3ff25700 1 -- 172.21.15.16:0/248691055 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f7a380a28e0 con 0x7f7a380a0a30 2023-09-30T22:33:04.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.562+0000 7f7a2e7fc700 1 -- 172.21.15.16:0/248691055 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7a340193b0 con 0x7f7a380a0a30 2023-09-30T22:33:04.563 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:33:04.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.564+0000 7f7a3ff25700 1 -- 172.21.15.16:0/248691055 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7a380a0a30 msgr2=0x7f7a381c8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:04.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.564+0000 7f7a3ff25700 1 --2- 172.21.15.16:0/248691055 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7a380a0a30 0x7f7a381c8460 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f7a34000c00 tx=0x7f7a34004750 comp rx=0 tx=0).stop 2023-09-30T22:33:04.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.566+0000 7f7a3ff25700 1 -- 172.21.15.16:0/248691055 shutdown_connections 2023-09-30T22:33:04.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.566+0000 7f7a3ff25700 1 --2- 172.21.15.16:0/248691055 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7a380a0a30 0x7f7a381c8460 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:04.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.566+0000 7f7a3ff25700 1 --2- 172.21.15.16:0/248691055 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7a38134170 0x7f7a381c8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:04.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.566+0000 7f7a3ff25700 1 --2- 172.21.15.16:0/248691055 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a380a1340 0x7f7a381c89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:04.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.566+0000 7f7a3ff25700 1 -- 172.21.15.16:0/248691055 >> 172.21.15.16:0/248691055 conn(0x7f7a380a6550 msgr2=0x7f7a3809ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:04.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.566+0000 7f7a3ff25700 1 -- 172.21.15.16:0/248691055 shutdown_connections 2023-09-30T22:33:04.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.566+0000 7f7a3ff25700 1 -- 172.21.15.16:0/248691055 wait complete. 2023-09-30T22:33:04.578 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:33:04.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.581+0000 7fdba64e6700 1 -- 172.21.15.16:0/2364504772 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fdba0074ea0 con 0x7fdba009f650 2023-09-30T22:33:04.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.581+0000 7fdb9cff9700 1 -- 172.21.15.16:0/2364504772 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdb98019300 con 0x7fdba009f650 2023-09-30T22:33:04.581 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:33:04.583 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:33:04.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.583+0000 7fdba64e6700 1 -- 172.21.15.16:0/2364504772 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdba009f650 msgr2=0x7fdba01a4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:04.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.583+0000 7fdba64e6700 1 --2- 172.21.15.16:0/2364504772 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdba009f650 0x7fdba01a4750 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fdb9800cc50 tx=0x7fdb9800a340 comp rx=0 tx=0).stop 2023-09-30T22:33:04.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.583+0000 7fdba64e6700 1 -- 172.21.15.16:0/2364504772 shutdown_connections 2023-09-30T22:33:04.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.583+0000 7fdba64e6700 1 --2- 172.21.15.16:0/2364504772 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdba009f650 0x7fdba01a4750 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:04.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.583+0000 7fdba64e6700 1 --2- 172.21.15.16:0/2364504772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdba009e930 0x7fdba01a3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:04.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.583+0000 7fdba64e6700 1 --2- 172.21.15.16:0/2364504772 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdba009f240 0x7fdba01a4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:04.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.583+0000 7fdba64e6700 1 -- 172.21.15.16:0/2364504772 >> 172.21.15.16:0/2364504772 conn(0x7fdba00a4210 msgr2=0x7fdba01b8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:04.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.584+0000 7fdba64e6700 1 -- 172.21.15.16:0/2364504772 shutdown_connections 2023-09-30T22:33:04.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:04.584+0000 7fdba64e6700 1 -- 172.21.15.16:0/2364504772 wait complete. 2023-09-30T22:33:04.598 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:33:05.531 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:33:05.578 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:33:05.583 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:33:05.599 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:33:05.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.693+0000 7f476b69c700 1 Processor -- start 2023-09-30T22:33:05.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.693+0000 7f476b69c700 1 -- start start 2023-09-30T22:33:05.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.694+0000 7f476b69c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f476409d6f0 0x7f476409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:05.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.694+0000 7f476b69c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f476409e000 0x7f4764130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:05.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.694+0000 7f476b69c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f476409e410 0x7f4764130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:05.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.694+0000 7f476b69c700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f476409f520 con 0x7f476409e000 2023-09-30T22:33:05.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.694+0000 7f476b69c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f476409f130 con 0x7f476409d6f0 2023-09-30T22:33:05.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.694+0000 7f476b69c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f476409e930 con 0x7f476409e410 2023-09-30T22:33:05.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.694+0000 7f4769438700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f476409d6f0 0x7f476409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:05.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.694+0000 7f4769438700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f476409d6f0 0x7f476409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36724/0 (socket says 172.21.15.16:36724) 2023-09-30T22:33:05.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.694+0000 7f4769438700 1 -- 172.21.15.16:0/2548028367 learned_addr learned my addr 172.21.15.16:0/2548028367 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:05.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.694+0000 7f4769c39700 1 --2- 172.21.15.16:0/2548028367 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f476409e410 0x7f4764130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:05.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.695+0000 7f4768c37700 1 --2- 172.21.15.16:0/2548028367 >> v2:172.21.15.16:3300/0 conn(0x7f476409e000 0x7f4764130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:05.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.695+0000 7f4769438700 1 -- 172.21.15.16:0/2548028367 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f476409e410 msgr2=0x7f4764130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:05.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.695+0000 7f4769438700 1 --2- 172.21.15.16:0/2548028367 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f476409e410 0x7f4764130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:05.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.695+0000 7f4769438700 1 -- 172.21.15.16:0/2548028367 >> v2:172.21.15.16:3300/0 conn(0x7f476409e000 msgr2=0x7f4764130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:05.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.695+0000 7f4769438700 1 --2- 172.21.15.16:0/2548028367 >> v2:172.21.15.16:3300/0 conn(0x7f476409e000 0x7f4764130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:05.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.695+0000 7f4769438700 1 -- 172.21.15.16:0/2548028367 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f476409b2d0 con 0x7f476409d6f0 2023-09-30T22:33:05.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.695+0000 7f4768c37700 1 --2- 172.21.15.16:0/2548028367 >> v2:172.21.15.16:3300/0 conn(0x7f476409e000 0x7f4764130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:33:05.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.695+0000 7f4769438700 1 --2- 172.21.15.16:0/2548028367 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f476409d6f0 0x7f476409dac0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f47600098f0 tx=0x7f4760009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=838974a15c6c6c01 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:05.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.695+0000 7f475bfff700 1 -- 172.21.15.16:0/2548028367 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4760011830 con 0x7f476409d6f0 2023-09-30T22:33:05.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.695+0000 7f475bfff700 1 -- 172.21.15.16:0/2548028367 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4760011990 con 0x7f476409d6f0 2023-09-30T22:33:05.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.695+0000 7f475bfff700 1 -- 172.21.15.16:0/2548028367 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4760011d20 con 0x7f476409d6f0 2023-09-30T22:33:05.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.696+0000 7f476b69c700 1 -- 172.21.15.16:0/2548028367 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f476409d6f0 msgr2=0x7f476409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:05.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.696+0000 7f476b69c700 1 --2- 172.21.15.16:0/2548028367 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f476409d6f0 0x7f476409dac0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f47600098f0 tx=0x7f4760009c00 comp rx=0 tx=0).stop 2023-09-30T22:33:05.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.696+0000 7f476b69c700 1 -- 172.21.15.16:0/2548028367 shutdown_connections 2023-09-30T22:33:05.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.696+0000 7f476b69c700 1 --2- 172.21.15.16:0/2548028367 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f476409d6f0 0x7f476409dac0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:05.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.696+0000 7f476b69c700 1 --2- 172.21.15.16:0/2548028367 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f476409e410 0x7f4764130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:05.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.696+0000 7f476b69c700 1 --2- 172.21.15.16:0/2548028367 >> v2:172.21.15.16:3300/0 conn(0x7f476409e000 0x7f4764130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:05.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.696+0000 7f476b69c700 1 -- 172.21.15.16:0/2548028367 >> 172.21.15.16:0/2548028367 conn(0x7f4764066f50 msgr2=0x7f476409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:05.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.696+0000 7f476b69c700 1 -- 172.21.15.16:0/2548028367 shutdown_connections 2023-09-30T22:33:05.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.696+0000 7f476b69c700 1 -- 172.21.15.16:0/2548028367 wait complete. 2023-09-30T22:33:05.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.697+0000 7f476b69c700 1 Processor -- start 2023-09-30T22:33:05.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.697+0000 7f476b69c700 1 -- start start 2023-09-30T22:33:05.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.698+0000 7f476b69c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f476409d6f0 0x7f47641200f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:05.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.698+0000 7f476b69c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f476409e000 0x7f4764120630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:05.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.698+0000 7f476b69c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f476409e410 0x7f4764120b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:05.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.698+0000 7f476b69c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4764122eb0 con 0x7f476409e410 2023-09-30T22:33:05.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.698+0000 7f476b69c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4764121de0 con 0x7f476409e000 2023-09-30T22:33:05.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.698+0000 7f476b69c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f476409c700 con 0x7f476409d6f0 2023-09-30T22:33:05.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.698+0000 7f4769c39700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f476409e410 0x7f4764120b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:05.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.698+0000 7f4769c39700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f476409e410 0x7f4764120b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37660/0 (socket says 172.21.15.16:37660) 2023-09-30T22:33:05.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.698+0000 7f4768c37700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f476409e000 0x7f4764120630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:05.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.698+0000 7f4769c39700 1 -- 172.21.15.16:0/4172393844 learned_addr learned my addr 172.21.15.16:0/4172393844 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:05.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.698+0000 7f4769438700 1 --2- 172.21.15.16:0/4172393844 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f476409d6f0 0x7f47641200f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:05.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.699+0000 7f4768c37700 1 -- 172.21.15.16:0/4172393844 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f476409d6f0 msgr2=0x7f47641200f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:05.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.699+0000 7f4768c37700 1 --2- 172.21.15.16:0/4172393844 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f476409d6f0 0x7f47641200f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:05.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.699+0000 7f4768c37700 1 -- 172.21.15.16:0/4172393844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f476409e410 msgr2=0x7f4764120b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:05.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.699+0000 7f4768c37700 1 --2- 172.21.15.16:0/4172393844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f476409e410 0x7f4764120b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:05.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.699+0000 7f4768c37700 1 -- 172.21.15.16:0/4172393844 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4760009550 con 0x7f476409e000 2023-09-30T22:33:05.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.699+0000 7f4769c39700 1 --2- 172.21.15.16:0/4172393844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f476409e410 0x7f4764120b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:05.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.699+0000 7f4768c37700 1 --2- 172.21.15.16:0/4172393844 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f476409e000 0x7f4764120630 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f475400c7c0 tx=0x7f475400cad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:05.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.699+0000 7f4759ffb700 1 -- 172.21.15.16:0/4172393844 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4754004230 con 0x7f476409e000 2023-09-30T22:33:05.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.699+0000 7f4759ffb700 1 -- 172.21.15.16:0/4172393844 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4754004390 con 0x7f476409e000 2023-09-30T22:33:05.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.699+0000 7f476b69c700 1 -- 172.21.15.16:0/4172393844 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4764123b70 con 0x7f476409e000 2023-09-30T22:33:05.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.699+0000 7f476b69c700 1 -- 172.21.15.16:0/4172393844 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4764123620 con 0x7f476409e000 2023-09-30T22:33:05.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.700+0000 7f4759ffb700 1 -- 172.21.15.16:0/4172393844 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47540036a0 con 0x7f476409e000 2023-09-30T22:33:05.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.701+0000 7f4759ffb700 1 -- 172.21.15.16:0/4172393844 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4754003800 con 0x7f476409e000 2023-09-30T22:33:05.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.701+0000 7f4759ffb700 1 -- 172.21.15.16:0/4172393844 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f475403f650 con 0x7f476409e000 2023-09-30T22:33:05.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.701+0000 7f476b69c700 1 -- 172.21.15.16:0/4172393844 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47440052e0 con 0x7f476409e000 2023-09-30T22:33:05.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.706+0000 7f4759ffb700 1 -- 172.21.15.16:0/4172393844 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4754011600 con 0x7f476409e000 2023-09-30T22:33:05.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.750+0000 7fd6a2a72700 1 Processor -- start 2023-09-30T22:33:05.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.750+0000 7fd6a2a72700 1 -- start start 2023-09-30T22:33:05.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.751+0000 7fd6a2a72700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd69c09d6f0 0x7fd69c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:05.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.751+0000 7fd6a2a72700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd69c09e000 0x7fd69c130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:05.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.751+0000 7fd6a2a72700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd69c09f130 con 0x7fd69c09e410 2023-09-30T22:33:05.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.751+0000 7fd6a2a72700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd69c09f520 con 0x7fd69c09d6f0 2023-09-30T22:33:05.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.751+0000 7fd6a2a72700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd69c09e930 con 0x7fd69c09e000 2023-09-30T22:33:05.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.751+0000 7fd6a080e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd69c09d6f0 0x7fd69c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:05.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.751+0000 7fd69bfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd69c09e000 0x7fd69c130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:05.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.751+0000 7fd6a100f700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd69c09e410 0x7fd69c130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38040/0 (socket says 172.21.15.16:38040) 2023-09-30T22:33:05.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.751+0000 7fd6a080e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd69c09d6f0 0x7fd69c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36744/0 (socket says 172.21.15.16:36744) 2023-09-30T22:33:05.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.751+0000 7fd6a100f700 1 -- 172.21.15.16:0/1693259748 learned_addr learned my addr 172.21.15.16:0/1693259748 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:05.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.751+0000 7fd69bfff700 1 -- 172.21.15.16:0/1693259748 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd69c09d6f0 msgr2=0x7fd69c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:05.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.751+0000 7fd69bfff700 1 --2- 172.21.15.16:0/1693259748 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd69c09d6f0 0x7fd69c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:05.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.751+0000 7fd69bfff700 1 -- 172.21.15.16:0/1693259748 >> v1:172.21.15.16:6789/0 conn(0x7fd69c09e410 legacy=0x7fd69c130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:05.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.751+0000 7fd69bfff700 1 -- 172.21.15.16:0/1693259748 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd69c09b2d0 con 0x7fd69c09e000 2023-09-30T22:33:05.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.752+0000 7fd69bfff700 1 --2- 172.21.15.16:0/1693259748 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd69c09e000 0x7fd69c130660 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd6880098a0 tx=0x7fd688009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93c3de83f17e4145 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:05.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.752+0000 7fd69b7fe700 1 -- 172.21.15.16:0/1693259748 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd688011830 con 0x7fd69c09e000 2023-09-30T22:33:05.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.752+0000 7fd69b7fe700 1 -- 172.21.15.16:0/1693259748 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd688011990 con 0x7fd69c09e000 2023-09-30T22:33:05.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.752+0000 7fd69b7fe700 1 -- 172.21.15.16:0/1693259748 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd688011d20 con 0x7fd69c09e000 2023-09-30T22:33:05.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.753+0000 7fd6a2a72700 1 -- 172.21.15.16:0/1693259748 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd69c09e000 msgr2=0x7fd69c130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:05.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.753+0000 7fd6a2a72700 1 --2- 172.21.15.16:0/1693259748 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd69c09e000 0x7fd69c130660 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd6880098a0 tx=0x7fd688009bb0 comp rx=0 tx=0).stop 2023-09-30T22:33:05.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.753+0000 7fd6a2a72700 1 -- 172.21.15.16:0/1693259748 shutdown_connections 2023-09-30T22:33:05.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.753+0000 7fd6a2a72700 1 --2- 172.21.15.16:0/1693259748 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd69c09d6f0 0x7fd69c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:05.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.753+0000 7fd6a2a72700 1 --2- 172.21.15.16:0/1693259748 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd69c09e000 0x7fd69c130660 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:05.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.753+0000 7fd6a2a72700 1 -- 172.21.15.16:0/1693259748 >> 172.21.15.16:0/1693259748 conn(0x7fd69c066f50 msgr2=0x7fd69c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:05.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.753+0000 7fd6a2a72700 1 -- 172.21.15.16:0/1693259748 shutdown_connections 2023-09-30T22:33:05.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.753+0000 7fd6a2a72700 1 -- 172.21.15.16:0/1693259748 wait complete. 2023-09-30T22:33:05.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.754+0000 7fd6a2a72700 1 Processor -- start 2023-09-30T22:33:05.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.754+0000 7fd6a2a72700 1 -- start start 2023-09-30T22:33:05.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.755+0000 7fd6a2a72700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd69c09d6f0 0x7fd69c1300e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:05.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.755+0000 7fd6a2a72700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd69c09e000 0x7fd69c130620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:05.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.755+0000 7fd6a2a72700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd69c09e410 0x7fd69c130b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:05.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.755+0000 7fd6a2a72700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd69c12e940 con 0x7fd69c09e000 2023-09-30T22:33:05.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.755+0000 7fd6a2a72700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd69c13ada0 con 0x7fd69c09e410 2023-09-30T22:33:05.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.755+0000 7fd6a2a72700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd69c12f500 con 0x7fd69c09d6f0 2023-09-30T22:33:05.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.756+0000 7fd69bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd69c09e000 0x7fd69c130620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:05.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.756+0000 7fd6a100f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd69c09e410 0x7fd69c130b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:05.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.756+0000 7fd69bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd69c09e000 0x7fd69c130620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37662/0 (socket says 172.21.15.16:37662) 2023-09-30T22:33:05.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.756+0000 7fd69bfff700 1 -- 172.21.15.16:0/2010495939 learned_addr learned my addr 172.21.15.16:0/2010495939 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:05.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.756+0000 7fd69bfff700 1 -- 172.21.15.16:0/2010495939 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd69c09d6f0 msgr2=0x7fd69c1300e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:05.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.756+0000 7fd69bfff700 1 --2- 172.21.15.16:0/2010495939 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd69c09d6f0 0x7fd69c1300e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:05.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.756+0000 7fd69bfff700 1 -- 172.21.15.16:0/2010495939 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd69c09e410 msgr2=0x7fd69c130b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:05.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.756+0000 7fd69bfff700 1 --2- 172.21.15.16:0/2010495939 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd69c09e410 0x7fd69c130b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:05.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.756+0000 7fd69bfff700 1 -- 172.21.15.16:0/2010495939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6880094a0 con 0x7fd69c09e000 2023-09-30T22:33:05.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.756+0000 7fd6a100f700 1 --2- 172.21.15.16:0/2010495939 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd69c09e410 0x7fd69c130b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:05.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.756+0000 7fd69bfff700 1 --2- 172.21.15.16:0/2010495939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd69c09e000 0x7fd69c130620 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fd688013000 tx=0x7fd688004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:05.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.757+0000 7fd6997fa700 1 -- 172.21.15.16:0/2010495939 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd688004550 con 0x7fd69c09e000 2023-09-30T22:33:05.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.757+0000 7fd6997fa700 1 -- 172.21.15.16:0/2010495939 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd68801f070 con 0x7fd69c09e000 2023-09-30T22:33:05.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.757+0000 7fd6997fa700 1 -- 172.21.15.16:0/2010495939 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6880038e0 con 0x7fd69c09e000 2023-09-30T22:33:05.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.757+0000 7fd6a2a72700 1 -- 172.21.15.16:0/2010495939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd69c12e510 con 0x7fd69c09e000 2023-09-30T22:33:05.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.757+0000 7fd6a2a72700 1 -- 172.21.15.16:0/2010495939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd69c0724b0 con 0x7fd69c09e000 2023-09-30T22:33:05.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.759+0000 7fd6a2a72700 1 -- 172.21.15.16:0/2010495939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd69c131290 con 0x7fd69c09e000 2023-09-30T22:33:05.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.759+0000 7fd6997fa700 1 -- 172.21.15.16:0/2010495939 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd688003a40 con 0x7fd69c09e000 2023-09-30T22:33:05.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.759+0000 7fd6997fa700 1 -- 172.21.15.16:0/2010495939 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd688021070 con 0x7fd69c09e000 2023-09-30T22:33:05.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.764+0000 7fd6997fa700 1 -- 172.21.15.16:0/2010495939 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd69c131290 con 0x7fd69c09e000 2023-09-30T22:33:05.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.772+0000 7f2276e80700 1 Processor -- start 2023-09-30T22:33:05.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.772+0000 7f2276e80700 1 -- start start 2023-09-30T22:33:05.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.772+0000 7f2276e80700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f227009e1d0 0x7f2270135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:05.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.772+0000 7f2276e80700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2270135e50 0x7f2270138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:05.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.772+0000 7f2276e80700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f227009f130 con 0x7f227009d6f0 2023-09-30T22:33:05.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.772+0000 7f2276e80700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f227009f520 con 0x7f227009e1d0 2023-09-30T22:33:05.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.772+0000 7f2276e80700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f227009e930 con 0x7f2270135e50 2023-09-30T22:33:05.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.772+0000 7f2274c1c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f227009d6f0 0x7f227009dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38054/0 (socket says 172.21.15.16:38054) 2023-09-30T22:33:05.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.772+0000 7f2274c1c700 1 -- 172.21.15.16:0/4053483600 learned_addr learned my addr 172.21.15.16:0/4053483600 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:05.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.772+0000 7f227541d700 1 --2- 172.21.15.16:0/4053483600 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2270135e50 0x7f2270138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:05.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.772+0000 7f226ffff700 1 --2- 172.21.15.16:0/4053483600 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f227009e1d0 0x7f2270135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:05.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.773+0000 7f511a7b5700 1 Processor -- start 2023-09-30T22:33:05.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.773+0000 7f226ffff700 1 -- 172.21.15.16:0/4053483600 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2270135e50 msgr2=0x7f2270138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:05.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.773+0000 7f226ffff700 1 --2- 172.21.15.16:0/4053483600 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2270135e50 0x7f2270138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:05.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.773+0000 7f226ffff700 1 -- 172.21.15.16:0/4053483600 >> v1:172.21.15.16:6789/0 conn(0x7f227009d6f0 legacy=0x7f227009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:05.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.773+0000 7f226ffff700 1 -- 172.21.15.16:0/4053483600 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f227009b2d0 con 0x7f227009e1d0 2023-09-30T22:33:05.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.773+0000 7f226f7fe700 1 -- 172.21.15.16:0/4053483600 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 146516987 0 0) 0x7f227009f130 con 0x7f227009d6f0 2023-09-30T22:33:05.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.773+0000 7f227541d700 1 --2- 172.21.15.16:0/4053483600 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2270135e50 0x7f2270138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:05.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.773+0000 7f511a7b5700 1 -- start start 2023-09-30T22:33:05.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.773+0000 7f226ffff700 1 --2- 172.21.15.16:0/4053483600 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f227009e1d0 0x7f2270135910 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f225c0098a0 tx=0x7f225c009bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7f04c98e2abdd37 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:05.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.773+0000 7f511a7b5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f511412cf60 0x7f511406f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:05.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.773+0000 7f511a7b5700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f511412f8b0 0x7f511412fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:05.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.773+0000 7f511a7b5700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f511412cd90 con 0x7f5114130260 2023-09-30T22:33:05.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.773+0000 7f511a7b5700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f511412d330 con 0x7f511412f8b0 2023-09-30T22:33:05.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.773+0000 7f511a7b5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f51141306e0 con 0x7f511412cf60 2023-09-30T22:33:05.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.774+0000 7f226f7fe700 1 -- 172.21.15.16:0/4053483600 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225c011830 con 0x7f227009e1d0 2023-09-30T22:33:05.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.774+0000 7f51137fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f511412f8b0 0x7f511412fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:05.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.774+0000 7f5113fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f511412cf60 0x7f511406f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:05.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.774+0000 7f5118d52700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5114130260 0x7f5114136c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38066/0 (socket says 172.21.15.16:38066) 2023-09-30T22:33:05.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.774+0000 7f5118d52700 1 -- 172.21.15.16:0/1362980112 learned_addr learned my addr 172.21.15.16:0/1362980112 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:05.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.774+0000 7f51137fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f511412f8b0 0x7f511412fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37678/0 (socket says 172.21.15.16:37678) 2023-09-30T22:33:05.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.774+0000 7f226f7fe700 1 -- 172.21.15.16:0/4053483600 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f225c011990 con 0x7f227009e1d0 2023-09-30T22:33:05.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.774+0000 7f226f7fe700 1 -- 172.21.15.16:0/4053483600 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225c011d20 con 0x7f227009e1d0 2023-09-30T22:33:05.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.774+0000 7f5112ffd700 1 -- 172.21.15.16:0/1362980112 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1461673615 0 0) 0x7f511412cd90 con 0x7f5114130260 2023-09-30T22:33:05.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.774+0000 7f5113fff700 1 -- 172.21.15.16:0/1362980112 >> v2:172.21.15.16:3300/0 conn(0x7f511412f8b0 msgr2=0x7f511412fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:05.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.774+0000 7f5113fff700 1 --2- 172.21.15.16:0/1362980112 >> v2:172.21.15.16:3300/0 conn(0x7f511412f8b0 0x7f511412fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:05.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.774+0000 7f5113fff700 1 -- 172.21.15.16:0/1362980112 >> v1:172.21.15.16:6789/0 conn(0x7f5114130260 legacy=0x7f5114136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:05.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.774+0000 7f5113fff700 1 -- 172.21.15.16:0/1362980112 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5114071780 con 0x7f511412cf60 2023-09-30T22:33:05.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.774+0000 7f5113fff700 1 --2- 172.21.15.16:0/1362980112 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f511412cf60 0x7f511406f5d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5108009840 tx=0x7f5108009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=354b1a5d5634317e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:05.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.774+0000 7f5112ffd700 1 -- 172.21.15.16:0/1362980112 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5108011830 con 0x7f511412cf60 2023-09-30T22:33:05.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.775+0000 7f5112ffd700 1 -- 172.21.15.16:0/1362980112 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5108011990 con 0x7f511412cf60 2023-09-30T22:33:05.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.775+0000 7f5112ffd700 1 -- 172.21.15.16:0/1362980112 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5108011d20 con 0x7f511412cf60 2023-09-30T22:33:05.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.775+0000 7f2276e80700 1 -- 172.21.15.16:0/4053483600 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f227009e1d0 msgr2=0x7f2270135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:05.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.775+0000 7f2276e80700 1 --2- 172.21.15.16:0/4053483600 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f227009e1d0 0x7f2270135910 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f225c0098a0 tx=0x7f225c009bb0 comp rx=0 tx=0).stop 2023-09-30T22:33:05.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.775+0000 7f2276e80700 1 -- 172.21.15.16:0/4053483600 shutdown_connections 2023-09-30T22:33:05.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.775+0000 7f2276e80700 1 --2- 172.21.15.16:0/4053483600 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2270135e50 0x7f2270138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:05.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.775+0000 7f2276e80700 1 --2- 172.21.15.16:0/4053483600 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f227009e1d0 0x7f2270135910 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:05.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.775+0000 7f2276e80700 1 -- 172.21.15.16:0/4053483600 >> 172.21.15.16:0/4053483600 conn(0x7f2270066f50 msgr2=0x7f227009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:05.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.775+0000 7f2276e80700 1 -- 172.21.15.16:0/4053483600 shutdown_connections 2023-09-30T22:33:05.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.775+0000 7f2276e80700 1 -- 172.21.15.16:0/4053483600 wait complete. 2023-09-30T22:33:05.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.775+0000 7f511a7b5700 1 -- 172.21.15.16:0/1362980112 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f511412cf60 msgr2=0x7f511406f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:05.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.775+0000 7f511a7b5700 1 --2- 172.21.15.16:0/1362980112 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f511412cf60 0x7f511406f5d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5108009840 tx=0x7f5108009b50 comp rx=0 tx=0).stop 2023-09-30T22:33:05.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.776+0000 7f2276e80700 1 Processor -- start 2023-09-30T22:33:05.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.776+0000 7f511a7b5700 1 -- 172.21.15.16:0/1362980112 shutdown_connections 2023-09-30T22:33:05.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.776+0000 7f511a7b5700 1 --2- 172.21.15.16:0/1362980112 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f511412cf60 0x7f511406f5d0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:05.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.776+0000 7f511a7b5700 1 --2- 172.21.15.16:0/1362980112 >> v2:172.21.15.16:3300/0 conn(0x7f511412f8b0 0x7f511412fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:05.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.776+0000 7f511a7b5700 1 -- 172.21.15.16:0/1362980112 >> 172.21.15.16:0/1362980112 conn(0x7f51140a2aa0 msgr2=0x7f51140a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:05.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.777+0000 7f511a7b5700 1 -- 172.21.15.16:0/1362980112 shutdown_connections 2023-09-30T22:33:05.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.777+0000 7f511a7b5700 1 -- 172.21.15.16:0/1362980112 wait complete. 2023-09-30T22:33:05.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.776+0000 7f2276e80700 1 -- start start 2023-09-30T22:33:05.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.777+0000 7f2276e80700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f227009d6f0 0x7f227011b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:05.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.777+0000 7f2276e80700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f227009e1d0 0x7f227011ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:05.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.777+0000 7f2276e80700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2270135e50 0x7f227011bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:05.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.777+0000 7f2276e80700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f227011f910 con 0x7f227009e1d0 2023-09-30T22:33:05.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.777+0000 7f2276e80700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f227011e750 con 0x7f2270135e50 2023-09-30T22:33:05.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.777+0000 7f2276e80700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2270138a60 con 0x7f227009d6f0 2023-09-30T22:33:05.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.777+0000 7f226ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f227009e1d0 0x7f227011ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:05.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.777+0000 7f227541d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2270135e50 0x7f227011bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:05.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.778+0000 7f227541d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2270135e50 0x7f227011bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36760/0 (socket says 172.21.15.16:36760) 2023-09-30T22:33:05.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.778+0000 7f226ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f227009e1d0 0x7f227011ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37692/0 (socket says 172.21.15.16:37692) 2023-09-30T22:33:05.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.778+0000 7f227541d700 1 -- 172.21.15.16:0/1120714372 learned_addr learned my addr 172.21.15.16:0/1120714372 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:05.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.778+0000 7f2274c1c700 1 --2- 172.21.15.16:0/1120714372 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f227009d6f0 0x7f227011b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:05.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.778+0000 7f227541d700 1 -- 172.21.15.16:0/1120714372 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f227009d6f0 msgr2=0x7f227011b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:05.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.778+0000 7f227541d700 1 --2- 172.21.15.16:0/1120714372 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f227009d6f0 0x7f227011b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:05.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.778+0000 7f227541d700 1 -- 172.21.15.16:0/1120714372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f227009e1d0 msgr2=0x7f227011ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:05.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.778+0000 7f227541d700 1 --2- 172.21.15.16:0/1120714372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f227009e1d0 0x7f227011ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:05.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.778+0000 7f227541d700 1 -- 172.21.15.16:0/1120714372 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2268009a70 con 0x7f2270135e50 2023-09-30T22:33:05.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.778+0000 7f2274c1c700 1 --2- 172.21.15.16:0/1120714372 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f227009d6f0 0x7f227011b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:33:05.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.778+0000 7f511a7b5700 1 Processor -- start 2023-09-30T22:33:05.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.778+0000 7f227541d700 1 --2- 172.21.15.16:0/1120714372 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2270135e50 0x7f227011bf70 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f226800b330 tx=0x7f226800b640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:05.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.778+0000 7f226d7fa700 1 -- 172.21.15.16:0/1120714372 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2268015070 con 0x7f2270135e50 2023-09-30T22:33:05.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.778+0000 7f226d7fa700 1 -- 172.21.15.16:0/1120714372 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f226800be00 con 0x7f2270135e50 2023-09-30T22:33:05.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.778+0000 7f226d7fa700 1 -- 172.21.15.16:0/1120714372 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2268009d50 con 0x7f2270135e50 2023-09-30T22:33:05.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.779+0000 7f2276e80700 1 -- 172.21.15.16:0/1120714372 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f225c0094a0 con 0x7f2270135e50 2023-09-30T22:33:05.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.779+0000 7f2276e80700 1 -- 172.21.15.16:0/1120714372 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f227011e290 con 0x7f2270135e50 2023-09-30T22:33:05.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.778+0000 7f511a7b5700 1 -- start start 2023-09-30T22:33:05.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.779+0000 7f511a7b5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f511412cf60 0x7f51141214e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:05.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.779+0000 7f511a7b5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f511412f8b0 0x7f5114121a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:05.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.779+0000 7f511a7b5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5114130260 0x7f5114121f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:05.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.779+0000 7f511a7b5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51141246a0 con 0x7f511412cf60 2023-09-30T22:33:05.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.779+0000 7f511a7b5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51141239e0 con 0x7f511412f8b0 2023-09-30T22:33:05.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.779+0000 7f511a7b5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f511409d760 con 0x7f5114130260 2023-09-30T22:33:05.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.779+0000 7f51137fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f511412f8b0 0x7f5114121a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:05.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.779+0000 7f51137fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f511412f8b0 0x7f5114121a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36774/0 (socket says 172.21.15.16:36774) 2023-09-30T22:33:05.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.779+0000 7f51137fe700 1 -- 172.21.15.16:0/251533166 learned_addr learned my addr 172.21.15.16:0/251533166 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:05.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.779+0000 7f5113fff700 1 --2- 172.21.15.16:0/251533166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f511412cf60 0x7f51141214e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:05.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.780+0000 7f51137fe700 1 -- 172.21.15.16:0/251533166 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5114130260 msgr2=0x7f5114121f60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:33:05.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.780+0000 7f51137fe700 1 --2- 172.21.15.16:0/251533166 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5114130260 0x7f5114121f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:05.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.780+0000 7f51137fe700 1 -- 172.21.15.16:0/251533166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f511412cf60 msgr2=0x7f51141214e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:05.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.780+0000 7f51137fe700 1 --2- 172.21.15.16:0/251533166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f511412cf60 0x7f51141214e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:05.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.780+0000 7f51137fe700 1 -- 172.21.15.16:0/251533166 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51080094a0 con 0x7f511412f8b0 2023-09-30T22:33:05.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.780+0000 7f226d7fa700 1 -- 172.21.15.16:0/1120714372 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f22680223b0 con 0x7f2270135e50 2023-09-30T22:33:05.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.780+0000 7f2276e80700 1 -- 172.21.15.16:0/1120714372 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2270071700 con 0x7f2270135e50 2023-09-30T22:33:05.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.781+0000 7f226d7fa700 1 -- 172.21.15.16:0/1120714372 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f2268022c20 con 0x7f2270135e50 2023-09-30T22:33:05.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.781+0000 7f51137fe700 1 --2- 172.21.15.16:0/251533166 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f511412f8b0 0x7f5114121a20 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f510400dad0 tx=0x7f510400de90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:05.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.785+0000 7f5110ff9700 1 -- 172.21.15.16:0/251533166 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510400aa90 con 0x7f511412f8b0 2023-09-30T22:33:05.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.785+0000 7f5110ff9700 1 -- 172.21.15.16:0/251533166 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f510400abf0 con 0x7f511412f8b0 2023-09-30T22:33:05.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.785+0000 7f5110ff9700 1 -- 172.21.15.16:0/251533166 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5104010470 con 0x7f511412f8b0 2023-09-30T22:33:05.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.785+0000 7f511a7b5700 1 -- 172.21.15.16:0/251533166 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5114129fa0 con 0x7f511412f8b0 2023-09-30T22:33:05.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.786+0000 7f511a7b5700 1 -- 172.21.15.16:0/251533166 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f511409cda0 con 0x7f511412f8b0 2023-09-30T22:33:05.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.786+0000 7f226d7fa700 1 -- 172.21.15.16:0/1120714372 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f226801e080 con 0x7f2270135e50 2023-09-30T22:33:05.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.787+0000 7f511a7b5700 1 -- 172.21.15.16:0/251533166 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5114070110 con 0x7f511412f8b0 2023-09-30T22:33:05.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.787+0000 7f5110ff9700 1 -- 172.21.15.16:0/251533166 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f51040105d0 con 0x7f511412f8b0 2023-09-30T22:33:05.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.787+0000 7f5110ff9700 1 -- 172.21.15.16:0/251533166 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f510403f990 con 0x7f511412f8b0 2023-09-30T22:33:05.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.798+0000 7f5110ff9700 1 -- 172.21.15.16:0/251533166 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f510400e950 con 0x7f511412f8b0 2023-09-30T22:33:05.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.946+0000 7f476b69c700 1 -- 172.21.15.16:0/4172393844 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f4744005150 con 0x7f476409e000 2023-09-30T22:33:05.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.946+0000 7f4759ffb700 1 -- 172.21.15.16:0/4172393844 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4754027070 con 0x7f476409e000 2023-09-30T22:33:05.947 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:33:05.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.950+0000 7f476b69c700 1 -- 172.21.15.16:0/4172393844 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f476409e000 msgr2=0x7f4764120630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:05.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.950+0000 7f476b69c700 1 --2- 172.21.15.16:0/4172393844 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f476409e000 0x7f4764120630 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f475400c7c0 tx=0x7f475400cad0 comp rx=0 tx=0).stop 2023-09-30T22:33:05.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.950+0000 7f476b69c700 1 -- 172.21.15.16:0/4172393844 shutdown_connections 2023-09-30T22:33:05.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.950+0000 7f476b69c700 1 --2- 172.21.15.16:0/4172393844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f476409e410 0x7f4764120b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:05.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.950+0000 7f476b69c700 1 --2- 172.21.15.16:0/4172393844 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f476409d6f0 0x7f47641200f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:05.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.950+0000 7f476b69c700 1 --2- 172.21.15.16:0/4172393844 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f476409e000 0x7f4764120630 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:05.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.950+0000 7f476b69c700 1 -- 172.21.15.16:0/4172393844 >> 172.21.15.16:0/4172393844 conn(0x7f4764066f50 msgr2=0x7f47640712e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:05.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.950+0000 7f476b69c700 1 -- 172.21.15.16:0/4172393844 shutdown_connections 2023-09-30T22:33:05.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.950+0000 7f476b69c700 1 -- 172.21.15.16:0/4172393844 wait complete. 2023-09-30T22:33:05.969 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:33:05.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.989+0000 7fd6a2a72700 1 -- 172.21.15.16:0/2010495939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd69c131290 con 0x7fd69c09e000 2023-09-30T22:33:05.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.990+0000 7fd6997fa700 1 -- 172.21.15.16:0/2010495939 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd69c131290 con 0x7fd69c09e000 2023-09-30T22:33:05.990 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:33:05.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.992+0000 7fd6a2a72700 1 -- 172.21.15.16:0/2010495939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd69c09e000 msgr2=0x7fd69c130620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:05.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.992+0000 7fd6a2a72700 1 --2- 172.21.15.16:0/2010495939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd69c09e000 0x7fd69c130620 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fd688013000 tx=0x7fd688004750 comp rx=0 tx=0).stop 2023-09-30T22:33:05.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.992+0000 7f2276e80700 1 -- 172.21.15.16:0/1120714372 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2270074ea0 con 0x7f2270135e50 2023-09-30T22:33:05.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.992+0000 7fd6a2a72700 1 -- 172.21.15.16:0/2010495939 shutdown_connections 2023-09-30T22:33:05.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.992+0000 7fd6a2a72700 1 --2- 172.21.15.16:0/2010495939 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd69c09e410 0x7fd69c130b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:05.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.992+0000 7fd6a2a72700 1 --2- 172.21.15.16:0/2010495939 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd69c09d6f0 0x7fd69c1300e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:05.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.992+0000 7fd6a2a72700 1 --2- 172.21.15.16:0/2010495939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd69c09e000 0x7fd69c130620 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:05.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.992+0000 7fd6a2a72700 1 -- 172.21.15.16:0/2010495939 >> 172.21.15.16:0/2010495939 conn(0x7fd69c066f50 msgr2=0x7fd69c0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:05.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.992+0000 7fd6a2a72700 1 -- 172.21.15.16:0/2010495939 shutdown_connections 2023-09-30T22:33:05.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.992+0000 7fd6a2a72700 1 -- 172.21.15.16:0/2010495939 wait complete. 2023-09-30T22:33:05.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.992+0000 7f226d7fa700 1 -- 172.21.15.16:0/1120714372 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f22680202c0 con 0x7f2270135e50 2023-09-30T22:33:05.994 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:33:05.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.994+0000 7f2276e80700 1 -- 172.21.15.16:0/1120714372 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2270135e50 msgr2=0x7f227011bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:05.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.995+0000 7f2276e80700 1 --2- 172.21.15.16:0/1120714372 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2270135e50 0x7f227011bf70 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f226800b330 tx=0x7f226800b640 comp rx=0 tx=0).stop 2023-09-30T22:33:05.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.995+0000 7f2276e80700 1 -- 172.21.15.16:0/1120714372 shutdown_connections 2023-09-30T22:33:05.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.995+0000 7f2276e80700 1 --2- 172.21.15.16:0/1120714372 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2270135e50 0x7f227011bf70 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:05.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.995+0000 7f2276e80700 1 --2- 172.21.15.16:0/1120714372 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f227009d6f0 0x7f227011b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:05.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.995+0000 7f2276e80700 1 --2- 172.21.15.16:0/1120714372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f227009e1d0 0x7f227011ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:05.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.995+0000 7f2276e80700 1 -- 172.21.15.16:0/1120714372 >> 172.21.15.16:0/1120714372 conn(0x7f2270066f50 msgr2=0x7f227006e840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:05.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.995+0000 7f2276e80700 1 -- 172.21.15.16:0/1120714372 shutdown_connections 2023-09-30T22:33:05.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:05.995+0000 7f2276e80700 1 -- 172.21.15.16:0/1120714372 wait complete. 2023-09-30T22:33:06.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:06.005+0000 7f511a7b5700 1 -- 172.21.15.16:0/251533166 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5114074ea0 con 0x7f511412f8b0 2023-09-30T22:33:06.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:06.006+0000 7f5110ff9700 1 -- 172.21.15.16:0/251533166 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f510400e360 con 0x7f511412f8b0 2023-09-30T22:33:06.006 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:33:06.008 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:33:06.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:06.008+0000 7f511a7b5700 1 -- 172.21.15.16:0/251533166 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f511412f8b0 msgr2=0x7f5114121a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:06.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:06.008+0000 7f511a7b5700 1 --2- 172.21.15.16:0/251533166 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f511412f8b0 0x7f5114121a20 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f510400dad0 tx=0x7f510400de90 comp rx=0 tx=0).stop 2023-09-30T22:33:06.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:06.008+0000 7f511a7b5700 1 -- 172.21.15.16:0/251533166 shutdown_connections 2023-09-30T22:33:06.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:06.008+0000 7f511a7b5700 1 --2- 172.21.15.16:0/251533166 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5114130260 0x7f5114121f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:06.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:06.008+0000 7f511a7b5700 1 --2- 172.21.15.16:0/251533166 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f511412cf60 0x7f51141214e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:06.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:06.008+0000 7f511a7b5700 1 --2- 172.21.15.16:0/251533166 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f511412f8b0 0x7f5114121a20 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:06.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:06.008+0000 7f511a7b5700 1 -- 172.21.15.16:0/251533166 >> 172.21.15.16:0/251533166 conn(0x7f51140a2aa0 msgr2=0x7f511406e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:06.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:06.008+0000 7f511a7b5700 1 -- 172.21.15.16:0/251533166 shutdown_connections 2023-09-30T22:33:06.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:06.008+0000 7f511a7b5700 1 -- 172.21.15.16:0/251533166 wait complete. 2023-09-30T22:33:06.010 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:33:06.022 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:33:06.971 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:33:07.008 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:33:07.011 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:33:07.022 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:33:07.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.126+0000 7f3f4c7ca700 1 Processor -- start 2023-09-30T22:33:07.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.126+0000 7f3f4c7ca700 1 -- start start 2023-09-30T22:33:07.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.127+0000 7f3f4c7ca700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3f4406c0e0 0x7f3f4409ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:07.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.127+0000 7f3f4c7ca700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3f441380f0 0x7f3f4413fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:07.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.127+0000 7f3f4c7ca700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f44134980 con 0x7f3f44134ec0 2023-09-30T22:33:07.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.127+0000 7f3f4c7ca700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4406c4b0 con 0x7f3f441380f0 2023-09-30T22:33:07.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.127+0000 7f3f4c7ca700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3f440a09c0 con 0x7f3f4406c0e0 2023-09-30T22:33:07.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.127+0000 7f3f4a566700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3f4406c0e0 0x7f3f4409ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:07.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.127+0000 7f3f49d65700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3f44134ec0 0x7f3f441379e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38068/0 (socket says 172.21.15.16:38068) 2023-09-30T22:33:07.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.127+0000 7f3f4a566700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3f4406c0e0 0x7f3f4409ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:38168/0 (socket says 172.21.15.16:38168) 2023-09-30T22:33:07.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.127+0000 7f3f49d65700 1 -- 172.21.15.16:0/4026653337 learned_addr learned my addr 172.21.15.16:0/4026653337 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:07.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.127+0000 7f3f4ad67700 1 --2- 172.21.15.16:0/4026653337 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3f441380f0 0x7f3f4413fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:07.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.128+0000 7f3f4a566700 1 -- 172.21.15.16:0/4026653337 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3f441380f0 msgr2=0x7f3f4413fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:07.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.128+0000 7f3f4a566700 1 --2- 172.21.15.16:0/4026653337 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3f441380f0 0x7f3f4413fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:07.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.128+0000 7f3f4a566700 1 -- 172.21.15.16:0/4026653337 >> v1:172.21.15.16:6789/0 conn(0x7f3f44134ec0 legacy=0x7f3f441379e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:07.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.128+0000 7f3f49564700 1 -- 172.21.15.16:0/4026653337 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 298501170 0 0) 0x7f3f44134980 con 0x7f3f44134ec0 2023-09-30T22:33:07.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.128+0000 7f3f4a566700 1 -- 172.21.15.16:0/4026653337 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f440a0600 con 0x7f3f4406c0e0 2023-09-30T22:33:07.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.128+0000 7f3f4ad67700 1 --2- 172.21.15.16:0/4026653337 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3f441380f0 0x7f3f4413fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:07.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.128+0000 7f3f4a566700 1 --2- 172.21.15.16:0/4026653337 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3f4406c0e0 0x7f3f4409ac90 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3f40009870 tx=0x7f3f40009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=426e29119314c5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:07.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.128+0000 7f3f49564700 1 -- 172.21.15.16:0/4026653337 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f40011830 con 0x7f3f4406c0e0 2023-09-30T22:33:07.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.128+0000 7f3f49564700 1 -- 172.21.15.16:0/4026653337 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f40011990 con 0x7f3f4406c0e0 2023-09-30T22:33:07.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.128+0000 7f3f49564700 1 -- 172.21.15.16:0/4026653337 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f40011d20 con 0x7f3f4406c0e0 2023-09-30T22:33:07.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.129+0000 7f3f4c7ca700 1 -- 172.21.15.16:0/4026653337 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3f4406c0e0 msgr2=0x7f3f4409ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:07.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.129+0000 7f3f4c7ca700 1 --2- 172.21.15.16:0/4026653337 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3f4406c0e0 0x7f3f4409ac90 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3f40009870 tx=0x7f3f40009b80 comp rx=0 tx=0).stop 2023-09-30T22:33:07.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.129+0000 7f3f4c7ca700 1 -- 172.21.15.16:0/4026653337 shutdown_connections 2023-09-30T22:33:07.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.129+0000 7f3f4c7ca700 1 --2- 172.21.15.16:0/4026653337 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3f441380f0 0x7f3f4413fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:07.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.129+0000 7f3f4c7ca700 1 --2- 172.21.15.16:0/4026653337 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3f4406c0e0 0x7f3f4409ac90 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:07.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.129+0000 7f3f4c7ca700 1 -- 172.21.15.16:0/4026653337 >> 172.21.15.16:0/4026653337 conn(0x7f3f440a4210 msgr2=0x7f3f440a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:07.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.129+0000 7f3f4c7ca700 1 -- 172.21.15.16:0/4026653337 shutdown_connections 2023-09-30T22:33:07.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.130+0000 7f3f4c7ca700 1 -- 172.21.15.16:0/4026653337 wait complete. 2023-09-30T22:33:07.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.131+0000 7f3f4c7ca700 1 Processor -- start 2023-09-30T22:33:07.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.131+0000 7f3f4c7ca700 1 -- start start 2023-09-30T22:33:07.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.132+0000 7f3f4c7ca700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3f4406c0e0 0x7f3f441ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:07.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.132+0000 7f3f4c7ca700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3f44134ec0 0x7f3f441cd290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:07.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.132+0000 7f3f4c7ca700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f441380f0 0x7f3f4411a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:07.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.132+0000 7f3f4c7ca700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4411e590 con 0x7f3f441380f0 2023-09-30T22:33:07.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.132+0000 7f3f4c7ca700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4411bdd0 con 0x7f3f44134ec0 2023-09-30T22:33:07.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.132+0000 7f3f4c7ca700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3f440a03f0 con 0x7f3f4406c0e0 2023-09-30T22:33:07.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.132+0000 7f3f4ad67700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f441380f0 0x7f3f4411a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:07.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.132+0000 7f3f4ad67700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f441380f0 0x7f3f4411a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37716/0 (socket says 172.21.15.16:37716) 2023-09-30T22:33:07.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.132+0000 7f3f4ad67700 1 -- 172.21.15.16:0/1583792549 learned_addr learned my addr 172.21.15.16:0/1583792549 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:07.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.132+0000 7f3f4a566700 1 --2- 172.21.15.16:0/1583792549 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3f4406c0e0 0x7f3f441ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:07.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.132+0000 7f3f49d65700 1 --2- 172.21.15.16:0/1583792549 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3f44134ec0 0x7f3f441cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:07.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.133+0000 7f3f4ad67700 1 -- 172.21.15.16:0/1583792549 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3f4406c0e0 msgr2=0x7f3f441ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:07.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.133+0000 7f3f4ad67700 1 --2- 172.21.15.16:0/1583792549 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3f4406c0e0 0x7f3f441ccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:07.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.133+0000 7f3f4ad67700 1 -- 172.21.15.16:0/1583792549 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3f44134ec0 msgr2=0x7f3f441cd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:07.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.133+0000 7f3f4ad67700 1 --2- 172.21.15.16:0/1583792549 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3f44134ec0 0x7f3f441cd290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:07.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.133+0000 7f3f4ad67700 1 -- 172.21.15.16:0/1583792549 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f3c009a70 con 0x7f3f441380f0 2023-09-30T22:33:07.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.133+0000 7f3f4a566700 1 --2- 172.21.15.16:0/1583792549 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3f4406c0e0 0x7f3f441ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:07.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.133+0000 7f3f4ad67700 1 --2- 172.21.15.16:0/1583792549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f441380f0 0x7f3f4411a680 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c00efc0 tx=0x7f3f3c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:07.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.133+0000 7f3f3affd700 1 -- 172.21.15.16:0/1583792549 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c00ca90 con 0x7f3f441380f0 2023-09-30T22:33:07.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.133+0000 7f3f3affd700 1 -- 172.21.15.16:0/1583792549 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f3c00f040 con 0x7f3f441380f0 2023-09-30T22:33:07.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.133+0000 7f3f4c7ca700 1 -- 172.21.15.16:0/1583792549 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f400094a0 con 0x7f3f441380f0 2023-09-30T22:33:07.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.133+0000 7f3f3affd700 1 -- 172.21.15.16:0/1583792549 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c00ce10 con 0x7f3f441380f0 2023-09-30T22:33:07.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.133+0000 7f3f4c7ca700 1 -- 172.21.15.16:0/1583792549 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f4411dae0 con 0x7f3f441380f0 2023-09-30T22:33:07.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.134+0000 7f3f3affd700 1 -- 172.21.15.16:0/1583792549 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3f3c014480 con 0x7f3f441380f0 2023-09-30T22:33:07.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.134+0000 7f3f3affd700 1 -- 172.21.15.16:0/1583792549 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f3f3c03fce0 con 0x7f3f441380f0 2023-09-30T22:33:07.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.135+0000 7f3f4c7ca700 1 -- 172.21.15.16:0/1583792549 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f44074ea0 con 0x7f3f441380f0 2023-09-30T22:33:07.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.140+0000 7f3f3affd700 1 -- 172.21.15.16:0/1583792549 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3f3c012930 con 0x7f3f441380f0 2023-09-30T22:33:07.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.169+0000 7f85ca80d700 1 Processor -- start 2023-09-30T22:33:07.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.169+0000 7f85ca80d700 1 -- start start 2023-09-30T22:33:07.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.169+0000 7f85ca80d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f85c4133420 0x7f85c41337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:07.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.169+0000 7f85ca80d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85c4134260 0x7f85c41306a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:07.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.169+0000 7f85ca80d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85c406b990 con 0x7f85c4133e30 2023-09-30T22:33:07.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.169+0000 7f85ca80d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85c40a0590 con 0x7f85c4133420 2023-09-30T22:33:07.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.169+0000 7f85ca80d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85c409fb80 con 0x7f85c4134260 2023-09-30T22:33:07.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.170+0000 7f85c3fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f85c4133420 0x7f85c41337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:07.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.170+0000 7f85c37fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f85c4133e30 0x7f85c412ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38080/0 (socket says 172.21.15.16:38080) 2023-09-30T22:33:07.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.170+0000 7f85c37fe700 1 -- 172.21.15.16:0/912864204 learned_addr learned my addr 172.21.15.16:0/912864204 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:07.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.170+0000 7f85c8daa700 1 --2- 172.21.15.16:0/912864204 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85c4134260 0x7f85c41306a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:07.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.170+0000 7f85c3fff700 1 -- 172.21.15.16:0/912864204 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85c4134260 msgr2=0x7f85c41306a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:07.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.170+0000 7f85c3fff700 1 --2- 172.21.15.16:0/912864204 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85c4134260 0x7f85c41306a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:07.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.170+0000 7f85c3fff700 1 -- 172.21.15.16:0/912864204 >> v1:172.21.15.16:6789/0 conn(0x7f85c4133e30 legacy=0x7f85c412ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:07.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.170+0000 7f85c3fff700 1 -- 172.21.15.16:0/912864204 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c40a0030 con 0x7f85c4133420 2023-09-30T22:33:07.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.170+0000 7f85c3fff700 1 --2- 172.21.15.16:0/912864204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85c4133420 0x7f85c41337f0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f85b4009840 tx=0x7f85b4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f28bcb39a576c1e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:07.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.170+0000 7f85c2ffd700 1 -- 172.21.15.16:0/912864204 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b4011830 con 0x7f85c4133420 2023-09-30T22:33:07.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.171+0000 7f85c2ffd700 1 -- 172.21.15.16:0/912864204 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b4011990 con 0x7f85c4133420 2023-09-30T22:33:07.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.171+0000 7f85c2ffd700 1 -- 172.21.15.16:0/912864204 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b4011d20 con 0x7f85c4133420 2023-09-30T22:33:07.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.171+0000 7f85ca80d700 1 -- 172.21.15.16:0/912864204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85c4133420 msgr2=0x7f85c41337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:07.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.171+0000 7f85ca80d700 1 --2- 172.21.15.16:0/912864204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85c4133420 0x7f85c41337f0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f85b4009840 tx=0x7f85b4009b50 comp rx=0 tx=0).stop 2023-09-30T22:33:07.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.171+0000 7f85ca80d700 1 -- 172.21.15.16:0/912864204 shutdown_connections 2023-09-30T22:33:07.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.171+0000 7f85ca80d700 1 --2- 172.21.15.16:0/912864204 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85c4134260 0x7f85c41306a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:07.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.171+0000 7f85ca80d700 1 --2- 172.21.15.16:0/912864204 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85c4133420 0x7f85c41337f0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:07.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.171+0000 7f85ca80d700 1 -- 172.21.15.16:0/912864204 >> 172.21.15.16:0/912864204 conn(0x7f85c40a4210 msgr2=0x7f85c40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:07.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.171+0000 7f85ca80d700 1 -- 172.21.15.16:0/912864204 shutdown_connections 2023-09-30T22:33:07.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.171+0000 7f85ca80d700 1 -- 172.21.15.16:0/912864204 wait complete. 2023-09-30T22:33:07.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.173+0000 7f85ca80d700 1 Processor -- start 2023-09-30T22:33:07.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.173+0000 7f85ca80d700 1 -- start start 2023-09-30T22:33:07.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.174+0000 7f85ca80d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85c4133420 0x7f85c409c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:07.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.174+0000 7f85ca80d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85c4133e30 0x7f85c409c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:07.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.174+0000 7f85ca80d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85c4134260 0x7f85c409cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:07.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.174+0000 7f85ca80d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c4120340 con 0x7f85c4133420 2023-09-30T22:33:07.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.174+0000 7f85ca80d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85c412d3a0 con 0x7f85c4133e30 2023-09-30T22:33:07.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.174+0000 7f85ca80d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f85c4120060 con 0x7f85c4134260 2023-09-30T22:33:07.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.174+0000 7f85c3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85c4133420 0x7f85c409c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:07.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.174+0000 7f85c3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85c4133420 0x7f85c409c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37736/0 (socket says 172.21.15.16:37736) 2023-09-30T22:33:07.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.174+0000 7f85c3fff700 1 -- 172.21.15.16:0/1653275732 learned_addr learned my addr 172.21.15.16:0/1653275732 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:07.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.174+0000 7f85c37fe700 1 --2- 172.21.15.16:0/1653275732 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85c4133e30 0x7f85c409c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:07.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.174+0000 7f85c8daa700 1 --2- 172.21.15.16:0/1653275732 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85c4134260 0x7f85c409cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:07.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.175+0000 7f85c3fff700 1 -- 172.21.15.16:0/1653275732 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85c4134260 msgr2=0x7f85c409cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:07.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.175+0000 7f85c3fff700 1 --2- 172.21.15.16:0/1653275732 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85c4134260 0x7f85c409cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:07.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.175+0000 7f85c3fff700 1 -- 172.21.15.16:0/1653275732 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85c4133e30 msgr2=0x7f85c409c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:07.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.175+0000 7f85c3fff700 1 --2- 172.21.15.16:0/1653275732 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85c4133e30 0x7f85c409c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:07.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.175+0000 7f85c3fff700 1 -- 172.21.15.16:0/1653275732 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b40094a0 con 0x7f85c4133420 2023-09-30T22:33:07.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.175+0000 7f85c37fe700 1 --2- 172.21.15.16:0/1653275732 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85c4133e30 0x7f85c409c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:07.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.175+0000 7f85c3fff700 1 --2- 172.21.15.16:0/1653275732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85c4133420 0x7f85c409c480 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f85b4009fd0 tx=0x7f85b4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:07.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.175+0000 7f85c0ff9700 1 -- 172.21.15.16:0/1653275732 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b4004480 con 0x7f85c4133420 2023-09-30T22:33:07.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.175+0000 7f85c0ff9700 1 -- 172.21.15.16:0/1653275732 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b401f070 con 0x7f85c4133420 2023-09-30T22:33:07.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.175+0000 7f85ca80d700 1 -- 172.21.15.16:0/1653275732 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85c411fb30 con 0x7f85c4133420 2023-09-30T22:33:07.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.175+0000 7f85c0ff9700 1 -- 172.21.15.16:0/1653275732 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b40038e0 con 0x7f85c4133420 2023-09-30T22:33:07.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.175+0000 7f85ca80d700 1 -- 172.21.15.16:0/1653275732 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85c411fda0 con 0x7f85c4133420 2023-09-30T22:33:07.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.176+0000 7f85c0ff9700 1 -- 172.21.15.16:0/1653275732 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f85b4021030 con 0x7f85c4133420 2023-09-30T22:33:07.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.176+0000 7f85c0ff9700 1 -- 172.21.15.16:0/1653275732 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f85b40409d0 con 0x7f85c4133420 2023-09-30T22:33:07.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.176+0000 7f85ca80d700 1 -- 172.21.15.16:0/1653275732 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85c409ee20 con 0x7f85c4133420 2023-09-30T22:33:07.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.182+0000 7f38e1d82700 1 Processor -- start 2023-09-30T22:33:07.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.182+0000 7f38e1d82700 1 -- start start 2023-09-30T22:33:07.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.183+0000 7f38e1d82700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f38dc09e1d0 0x7f38dc135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:07.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.183+0000 7f38e1d82700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38dc135e50 0x7f38dc138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:07.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.183+0000 7f38e1d82700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38dc09f130 con 0x7f38dc09d6f0 2023-09-30T22:33:07.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.183+0000 7f38e1d82700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38dc09f520 con 0x7f38dc135e50 2023-09-30T22:33:07.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.183+0000 7f38e1d82700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f38dc09e930 con 0x7f38dc09e1d0 2023-09-30T22:33:07.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.183+0000 7f38d2ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f38dc09e1d0 0x7f38dc135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:07.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.183+0000 7f38d2ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f38dc09e1d0 0x7f38dc135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:38186/0 (socket says 172.21.15.16:38186) 2023-09-30T22:33:07.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.183+0000 7f38d2ffd700 1 -- 172.21.15.16:0/2586967090 learned_addr learned my addr 172.21.15.16:0/2586967090 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:07.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.183+0000 7f38dbfff700 1 --2- 172.21.15.16:0/2586967090 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38dc135e50 0x7f38dc138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:07.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.183+0000 7f38d2ffd700 1 -- 172.21.15.16:0/2586967090 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38dc135e50 msgr2=0x7f38dc138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:07.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.183+0000 7f38d2ffd700 1 --2- 172.21.15.16:0/2586967090 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38dc135e50 0x7f38dc138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:07.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.183+0000 7f38d2ffd700 1 -- 172.21.15.16:0/2586967090 >> v1:172.21.15.16:6789/0 conn(0x7f38dc09d6f0 legacy=0x7f38dc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:07.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.183+0000 7f38d2ffd700 1 -- 172.21.15.16:0/2586967090 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38dc09b2d0 con 0x7f38dc09e1d0 2023-09-30T22:33:07.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.183+0000 7f38d2ffd700 1 --2- 172.21.15.16:0/2586967090 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f38dc09e1d0 0x7f38dc135910 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f38c4009920 tx=0x7f38c4009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=362995c06fa78600 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:07.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.184+0000 7f38daffd700 1 -- 172.21.15.16:0/2586967090 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c4004230 con 0x7f38dc09e1d0 2023-09-30T22:33:07.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.184+0000 7f38daffd700 1 -- 172.21.15.16:0/2586967090 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38c4004390 con 0x7f38dc09e1d0 2023-09-30T22:33:07.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.184+0000 7f38daffd700 1 -- 172.21.15.16:0/2586967090 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c4009e70 con 0x7f38dc09e1d0 2023-09-30T22:33:07.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.184+0000 7f38e1d82700 1 -- 172.21.15.16:0/2586967090 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f38dc09e1d0 msgr2=0x7f38dc135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:07.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.184+0000 7f38e1d82700 1 --2- 172.21.15.16:0/2586967090 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f38dc09e1d0 0x7f38dc135910 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f38c4009920 tx=0x7f38c4009c30 comp rx=0 tx=0).stop 2023-09-30T22:33:07.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.184+0000 7f38e1d82700 1 -- 172.21.15.16:0/2586967090 shutdown_connections 2023-09-30T22:33:07.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.184+0000 7f38e1d82700 1 --2- 172.21.15.16:0/2586967090 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38dc135e50 0x7f38dc138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:07.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.184+0000 7f38e1d82700 1 --2- 172.21.15.16:0/2586967090 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f38dc09e1d0 0x7f38dc135910 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:07.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.184+0000 7f38e1d82700 1 -- 172.21.15.16:0/2586967090 >> 172.21.15.16:0/2586967090 conn(0x7f38dc066f50 msgr2=0x7f38dc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:07.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.184+0000 7f38e1d82700 1 -- 172.21.15.16:0/2586967090 shutdown_connections 2023-09-30T22:33:07.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.184+0000 7f38e1d82700 1 -- 172.21.15.16:0/2586967090 wait complete. 2023-09-30T22:33:07.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.185+0000 7f38e1d82700 1 Processor -- start 2023-09-30T22:33:07.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.186+0000 7f38e1d82700 1 -- start start 2023-09-30T22:33:07.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.186+0000 7f85c0ff9700 1 -- 172.21.15.16:0/1653275732 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f85b4015020 con 0x7f85c4133420 2023-09-30T22:33:07.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.186+0000 7f38e1d82700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38dc09d6f0 0x7f38dc11ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:07.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.186+0000 7f38e1d82700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38dc09e1d0 0x7f38dc11f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:07.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.186+0000 7f38e1d82700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f38dc135e50 0x7f38dc11f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:07.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.186+0000 7f38e1d82700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38dc120e60 con 0x7f38dc09e1d0 2023-09-30T22:33:07.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.186+0000 7f38e1d82700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38dc09c770 con 0x7f38dc09d6f0 2023-09-30T22:33:07.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.186+0000 7f38e1d82700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f38dc0a1e20 con 0x7f38dc135e50 2023-09-30T22:33:07.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.186+0000 7f38d2ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38dc09e1d0 0x7f38dc11f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:07.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.186+0000 7f38d2ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38dc09e1d0 0x7f38dc11f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37748/0 (socket says 172.21.15.16:37748) 2023-09-30T22:33:07.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.186+0000 7f38d2ffd700 1 -- 172.21.15.16:0/506245611 learned_addr learned my addr 172.21.15.16:0/506245611 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:07.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.186+0000 7f38db7fe700 1 --2- 172.21.15.16:0/506245611 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38dc09d6f0 0x7f38dc11ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:07.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.187+0000 7f38dbfff700 1 --2- 172.21.15.16:0/506245611 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f38dc135e50 0x7f38dc11f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:07.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.187+0000 7f38d2ffd700 1 -- 172.21.15.16:0/506245611 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f38dc135e50 msgr2=0x7f38dc11f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:07.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.187+0000 7f38d2ffd700 1 --2- 172.21.15.16:0/506245611 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f38dc135e50 0x7f38dc11f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:07.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.187+0000 7f38d2ffd700 1 -- 172.21.15.16:0/506245611 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38dc09d6f0 msgr2=0x7f38dc11ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:07.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.187+0000 7f38d2ffd700 1 --2- 172.21.15.16:0/506245611 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38dc09d6f0 0x7f38dc11ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:07.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.187+0000 7f38d2ffd700 1 -- 172.21.15.16:0/506245611 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38c4009550 con 0x7f38dc09e1d0 2023-09-30T22:33:07.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.187+0000 7f38d2ffd700 1 --2- 172.21.15.16:0/506245611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38dc09e1d0 0x7f38dc11f2b0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f38c40098f0 tx=0x7f38c4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:07.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.187+0000 7f38d8ff9700 1 -- 172.21.15.16:0/506245611 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c400bb60 con 0x7f38dc09e1d0 2023-09-30T22:33:07.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.187+0000 7f38d8ff9700 1 -- 172.21.15.16:0/506245611 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38c401f070 con 0x7f38dc09e1d0 2023-09-30T22:33:07.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.187+0000 7f38e1d82700 1 -- 172.21.15.16:0/506245611 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38dc1209c0 con 0x7f38dc09e1d0 2023-09-30T22:33:07.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.187+0000 7f38d8ff9700 1 -- 172.21.15.16:0/506245611 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c40036a0 con 0x7f38dc09e1d0 2023-09-30T22:33:07.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.187+0000 7f38e1d82700 1 -- 172.21.15.16:0/506245611 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38dc122f50 con 0x7f38dc09e1d0 2023-09-30T22:33:07.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.188+0000 7f38d8ff9700 1 -- 172.21.15.16:0/506245611 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f38c4021030 con 0x7f38dc09e1d0 2023-09-30T22:33:07.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.188+0000 7f38e1d82700 1 -- 172.21.15.16:0/506245611 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38dc074ea0 con 0x7f38dc09e1d0 2023-09-30T22:33:07.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.189+0000 7f38d8ff9700 1 -- 172.21.15.16:0/506245611 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f38c4040ac0 con 0x7f38dc09e1d0 2023-09-30T22:33:07.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.198+0000 7f38d8ff9700 1 -- 172.21.15.16:0/506245611 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f38c4019780 con 0x7f38dc09e1d0 2023-09-30T22:33:07.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.225+0000 7f6162c7e700 1 Processor -- start 2023-09-30T22:33:07.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.225+0000 7f6162c7e700 1 -- start start 2023-09-30T22:33:07.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.226+0000 7f6162c7e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f615c09e850 0x7f615c09ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:07.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.226+0000 7f6162c7e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f615c134300 0x7f615c13bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:07.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.226+0000 7f6162c7e700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f615c09f960 con 0x7f615c09f160 2023-09-30T22:33:07.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.226+0000 7f6162c7e700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f615c09ae60 con 0x7f615c134300 2023-09-30T22:33:07.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.226+0000 7f6162c7e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f615c06cb10 con 0x7f615c09e850 2023-09-30T22:33:07.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.226+0000 7f615bfff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f615c09f160 0x7f615c133bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38086/0 (socket says 172.21.15.16:38086) 2023-09-30T22:33:07.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.226+0000 7f616121b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f615c134300 0x7f615c13bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:07.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.226+0000 7f615bfff700 1 -- 172.21.15.16:0/1934510175 learned_addr learned my addr 172.21.15.16:0/1934510175 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:07.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.226+0000 7f6160a1a700 1 --2- 172.21.15.16:0/1934510175 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f615c09e850 0x7f615c09ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:07.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.226+0000 7f616121b700 1 -- 172.21.15.16:0/1934510175 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f615c09e850 msgr2=0x7f615c09ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:07.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.226+0000 7f616121b700 1 --2- 172.21.15.16:0/1934510175 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f615c09e850 0x7f615c09ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:07.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.226+0000 7f616121b700 1 -- 172.21.15.16:0/1934510175 >> v1:172.21.15.16:6789/0 conn(0x7f615c09f160 legacy=0x7f615c133bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:07.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.227+0000 7f616121b700 1 -- 172.21.15.16:0/1934510175 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f615c06b990 con 0x7f615c134300 2023-09-30T22:33:07.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.227+0000 7f616121b700 1 --2- 172.21.15.16:0/1934510175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f615c134300 0x7f615c13bec0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f6154009d90 tx=0x7f615400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d2018a077e2f813 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:07.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.227+0000 7f615b7fe700 1 -- 172.21.15.16:0/1934510175 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615400e070 con 0x7f615c134300 2023-09-30T22:33:07.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.227+0000 7f615b7fe700 1 -- 172.21.15.16:0/1934510175 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f615400bb20 con 0x7f615c134300 2023-09-30T22:33:07.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.227+0000 7f615b7fe700 1 -- 172.21.15.16:0/1934510175 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6154012430 con 0x7f615c134300 2023-09-30T22:33:07.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.228+0000 7f6162c7e700 1 -- 172.21.15.16:0/1934510175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f615c134300 msgr2=0x7f615c13bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:07.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.228+0000 7f6162c7e700 1 --2- 172.21.15.16:0/1934510175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f615c134300 0x7f615c13bec0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f6154009d90 tx=0x7f615400b330 comp rx=0 tx=0).stop 2023-09-30T22:33:07.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.228+0000 7f6162c7e700 1 -- 172.21.15.16:0/1934510175 shutdown_connections 2023-09-30T22:33:07.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.228+0000 7f6162c7e700 1 --2- 172.21.15.16:0/1934510175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f615c134300 0x7f615c13bec0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:07.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.228+0000 7f6162c7e700 1 --2- 172.21.15.16:0/1934510175 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f615c09e850 0x7f615c09ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:07.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.228+0000 7f6162c7e700 1 -- 172.21.15.16:0/1934510175 >> 172.21.15.16:0/1934510175 conn(0x7f615c0a3f70 msgr2=0x7f615c0a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:07.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.228+0000 7f6162c7e700 1 -- 172.21.15.16:0/1934510175 shutdown_connections 2023-09-30T22:33:07.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.228+0000 7f6162c7e700 1 -- 172.21.15.16:0/1934510175 wait complete. 2023-09-30T22:33:07.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.229+0000 7f6162c7e700 1 Processor -- start 2023-09-30T22:33:07.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.229+0000 7f6162c7e700 1 -- start start 2023-09-30T22:33:07.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.230+0000 7f6162c7e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f615c09e850 0x7f615c122b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:07.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.230+0000 7f6162c7e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f615c09f160 0x7f615c123060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:07.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.230+0000 7f6162c7e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f615c134300 0x7f615c1235a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:07.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.230+0000 7f6162c7e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f615c1339e0 con 0x7f615c09f160 2023-09-30T22:33:07.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.230+0000 7f6162c7e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f615c1262d0 con 0x7f615c134300 2023-09-30T22:33:07.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.230+0000 7f6162c7e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f615c133600 con 0x7f615c09e850 2023-09-30T22:33:07.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.230+0000 7f6160a1a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f615c09e850 0x7f615c122b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:07.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.230+0000 7f6160a1a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f615c09e850 0x7f615c122b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:38226/0 (socket says 172.21.15.16:38226) 2023-09-30T22:33:07.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.230+0000 7f6160a1a700 1 -- 172.21.15.16:0/3825114752 learned_addr learned my addr 172.21.15.16:0/3825114752 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:07.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.230+0000 7f615bfff700 1 --2- 172.21.15.16:0/3825114752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f615c09f160 0x7f615c123060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:07.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.230+0000 7f616121b700 1 --2- 172.21.15.16:0/3825114752 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f615c134300 0x7f615c1235a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:07.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.230+0000 7f6160a1a700 1 -- 172.21.15.16:0/3825114752 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f615c134300 msgr2=0x7f615c1235a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:07.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.230+0000 7f6160a1a700 1 --2- 172.21.15.16:0/3825114752 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f615c134300 0x7f615c1235a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:07.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.230+0000 7f6160a1a700 1 -- 172.21.15.16:0/3825114752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f615c09f160 msgr2=0x7f615c123060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:07.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.230+0000 7f6160a1a700 1 --2- 172.21.15.16:0/3825114752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f615c09f160 0x7f615c123060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:07.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.230+0000 7f6160a1a700 1 -- 172.21.15.16:0/3825114752 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6154009a70 con 0x7f615c09e850 2023-09-30T22:33:07.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.230+0000 7f616121b700 1 --2- 172.21.15.16:0/3825114752 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f615c134300 0x7f615c1235a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:33:07.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.231+0000 7f6160a1a700 1 --2- 172.21.15.16:0/3825114752 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f615c09e850 0x7f615c122b20 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f615000e9b0 tx=0x7f615000ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:07.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.231+0000 7f61597fa700 1 -- 172.21.15.16:0/3825114752 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6150009710 con 0x7f615c09e850 2023-09-30T22:33:07.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.231+0000 7f61597fa700 1 -- 172.21.15.16:0/3825114752 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f615000c850 con 0x7f615c09e850 2023-09-30T22:33:07.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.231+0000 7f6162c7e700 1 -- 172.21.15.16:0/3825114752 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f615c126010 con 0x7f615c09e850 2023-09-30T22:33:07.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.231+0000 7f61597fa700 1 -- 172.21.15.16:0/3825114752 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615000cbc0 con 0x7f615c09e850 2023-09-30T22:33:07.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.231+0000 7f6162c7e700 1 -- 172.21.15.16:0/3825114752 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f615c126d40 con 0x7f615c09e850 2023-09-30T22:33:07.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.232+0000 7f61597fa700 1 -- 172.21.15.16:0/3825114752 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f615000cd20 con 0x7f615c09e850 2023-09-30T22:33:07.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.232+0000 7f61597fa700 1 -- 172.21.15.16:0/3825114752 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f615003fc00 con 0x7f615c09e850 2023-09-30T22:33:07.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.232+0000 7f6162c7e700 1 -- 172.21.15.16:0/3825114752 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f615c0747f0 con 0x7f615c09e850 2023-09-30T22:33:07.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.237+0000 7f61597fa700 1 -- 172.21.15.16:0/3825114752 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6150017020 con 0x7f615c09e850 2023-09-30T22:33:07.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.409+0000 7f85ca80d700 1 -- 172.21.15.16:0/1653275732 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f85c4074ea0 con 0x7f85c4133420 2023-09-30T22:33:07.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.409+0000 7f85c0ff9700 1 -- 172.21.15.16:0/1653275732 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f85b401a7d0 con 0x7f85c4133420 2023-09-30T22:33:07.410 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:33:07.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.412+0000 7f85ca80d700 1 -- 172.21.15.16:0/1653275732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85c4133420 msgr2=0x7f85c409c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:07.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.412+0000 7f85ca80d700 1 --2- 172.21.15.16:0/1653275732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85c4133420 0x7f85c409c480 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f85b4009fd0 tx=0x7f85b4004750 comp rx=0 tx=0).stop 2023-09-30T22:33:07.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.412+0000 7f85ca80d700 1 -- 172.21.15.16:0/1653275732 shutdown_connections 2023-09-30T22:33:07.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.412+0000 7f85ca80d700 1 --2- 172.21.15.16:0/1653275732 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85c4134260 0x7f85c409cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:07.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.412+0000 7f85ca80d700 1 --2- 172.21.15.16:0/1653275732 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85c4133420 0x7f85c409c480 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:07.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.412+0000 7f85ca80d700 1 --2- 172.21.15.16:0/1653275732 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85c4133e30 0x7f85c409c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:07.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.412+0000 7f85ca80d700 1 -- 172.21.15.16:0/1653275732 >> 172.21.15.16:0/1653275732 conn(0x7f85c40a4210 msgr2=0x7f85c412e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:07.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.413+0000 7f85ca80d700 1 -- 172.21.15.16:0/1653275732 shutdown_connections 2023-09-30T22:33:07.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.413+0000 7f85ca80d700 1 -- 172.21.15.16:0/1653275732 wait complete. 2023-09-30T22:33:07.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.426+0000 7f3f4c7ca700 1 -- 172.21.15.16:0/1583792549 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3f4406b250 con 0x7f3f441380f0 2023-09-30T22:33:07.427 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:33:07.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.427+0000 7f3f3affd700 1 -- 172.21.15.16:0/1583792549 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3f3c012340 con 0x7f3f441380f0 2023-09-30T22:33:07.428 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:33:07.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.429+0000 7f3f4c7ca700 1 -- 172.21.15.16:0/1583792549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f441380f0 msgr2=0x7f3f4411a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:07.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.429+0000 7f3f4c7ca700 1 --2- 172.21.15.16:0/1583792549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f441380f0 0x7f3f4411a680 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c00efc0 tx=0x7f3f3c00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:33:07.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.429+0000 7f3f4c7ca700 1 -- 172.21.15.16:0/1583792549 shutdown_connections 2023-09-30T22:33:07.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.429+0000 7f3f4c7ca700 1 --2- 172.21.15.16:0/1583792549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f441380f0 0x7f3f4411a680 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:07.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.429+0000 7f3f4c7ca700 1 --2- 172.21.15.16:0/1583792549 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3f4406c0e0 0x7f3f441ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:07.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.429+0000 7f3f4c7ca700 1 --2- 172.21.15.16:0/1583792549 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3f44134ec0 0x7f3f441cd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:07.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.429+0000 7f3f4c7ca700 1 -- 172.21.15.16:0/1583792549 >> 172.21.15.16:0/1583792549 conn(0x7f3f440a4210 msgr2=0x7f3f44099c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:07.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.429+0000 7f3f4c7ca700 1 -- 172.21.15.16:0/1583792549 shutdown_connections 2023-09-30T22:33:07.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.429+0000 7f3f4c7ca700 1 -- 172.21.15.16:0/1583792549 wait complete. 2023-09-30T22:33:07.444 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:33:07.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.444+0000 7f6162c7e700 1 -- 172.21.15.16:0/3825114752 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f615c06bbb0 con 0x7f615c09e850 2023-09-30T22:33:07.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.444+0000 7f61597fa700 1 -- 172.21.15.16:0/3825114752 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f615002b090 con 0x7f615c09e850 2023-09-30T22:33:07.445 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:33:07.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.447+0000 7f6162c7e700 1 -- 172.21.15.16:0/3825114752 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f615c09e850 msgr2=0x7f615c122b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:07.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.447+0000 7f6162c7e700 1 --2- 172.21.15.16:0/3825114752 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f615c09e850 0x7f615c122b20 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f615000e9b0 tx=0x7f615000ed70 comp rx=0 tx=0).stop 2023-09-30T22:33:07.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.447+0000 7f6162c7e700 1 -- 172.21.15.16:0/3825114752 shutdown_connections 2023-09-30T22:33:07.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.447+0000 7f6162c7e700 1 --2- 172.21.15.16:0/3825114752 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f615c134300 0x7f615c1235a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:07.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.447+0000 7f6162c7e700 1 --2- 172.21.15.16:0/3825114752 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f615c09e850 0x7f615c122b20 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:07.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.447+0000 7f6162c7e700 1 --2- 172.21.15.16:0/3825114752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f615c09f160 0x7f615c123060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:07.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.447+0000 7f6162c7e700 1 -- 172.21.15.16:0/3825114752 >> 172.21.15.16:0/3825114752 conn(0x7f615c0a3f70 msgr2=0x7f615c0a2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:07.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.447+0000 7f6162c7e700 1 -- 172.21.15.16:0/3825114752 shutdown_connections 2023-09-30T22:33:07.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.447+0000 7f6162c7e700 1 -- 172.21.15.16:0/3825114752 wait complete. 2023-09-30T22:33:07.461 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:33:07.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.461+0000 7f38e1d82700 1 -- 172.21.15.16:0/506245611 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f38dc074ea0 con 0x7f38dc09e1d0 2023-09-30T22:33:07.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.462+0000 7f38d8ff9700 1 -- 172.21.15.16:0/506245611 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f38c40193c0 con 0x7f38dc09e1d0 2023-09-30T22:33:07.463 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:33:07.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.465+0000 7f38e1d82700 1 -- 172.21.15.16:0/506245611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38dc09e1d0 msgr2=0x7f38dc11f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:07.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.465+0000 7f38e1d82700 1 --2- 172.21.15.16:0/506245611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38dc09e1d0 0x7f38dc11f2b0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f38c40098f0 tx=0x7f38c4004750 comp rx=0 tx=0).stop 2023-09-30T22:33:07.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.465+0000 7f38e1d82700 1 -- 172.21.15.16:0/506245611 shutdown_connections 2023-09-30T22:33:07.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.465+0000 7f38e1d82700 1 --2- 172.21.15.16:0/506245611 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38dc09d6f0 0x7f38dc11ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:07.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.465+0000 7f38e1d82700 1 --2- 172.21.15.16:0/506245611 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f38dc135e50 0x7f38dc11f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:07.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.465+0000 7f38e1d82700 1 --2- 172.21.15.16:0/506245611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38dc09e1d0 0x7f38dc11f2b0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:07.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.465+0000 7f38e1d82700 1 -- 172.21.15.16:0/506245611 >> 172.21.15.16:0/506245611 conn(0x7f38dc066f50 msgr2=0x7f38dc071c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:07.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.465+0000 7f38e1d82700 1 -- 172.21.15.16:0/506245611 shutdown_connections 2023-09-30T22:33:07.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:07.465+0000 7f38e1d82700 1 -- 172.21.15.16:0/506245611 wait complete. 2023-09-30T22:33:07.479 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:33:08.428 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:33:08.445 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:33:08.462 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:33:08.479 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:33:08.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.588+0000 7fc2ed2d0700 1 Processor -- start 2023-09-30T22:33:08.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.588+0000 7fc2ed2d0700 1 -- start start 2023-09-30T22:33:08.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.588+0000 7fc2ed2d0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2e81338a0 0x7fc2e8133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:08.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.589+0000 7fc2ed2d0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2e809e930 0x7fc2e813b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:08.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.589+0000 7fc2ed2d0700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2e806ef30 con 0x7fc2e809de50 2023-09-30T22:33:08.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.589+0000 7fc2ed2d0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e806fef0 con 0x7fc2e81338a0 2023-09-30T22:33:08.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.589+0000 7fc2ed2d0700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e81334b0 con 0x7fc2e809e930 2023-09-30T22:33:08.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.589+0000 7fc2e77fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2e809e930 0x7fc2e813b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:08.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.589+0000 7fc2e67fc700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc2e809de50 0x7fc2e809e220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36348/0 (socket says 172.21.15.16:36348) 2023-09-30T22:33:08.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.589+0000 7fc2e67fc700 1 -- 172.21.15.16:0/1312744884 learned_addr learned my addr 172.21.15.16:0/1312744884 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:08.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.589+0000 7fc2e6ffd700 1 --2- 172.21.15.16:0/1312744884 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2e81338a0 0x7fc2e8133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:08.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.589+0000 7fc2e77fe700 1 -- 172.21.15.16:0/1312744884 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2e81338a0 msgr2=0x7fc2e8133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:08.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.589+0000 7fc2e77fe700 1 --2- 172.21.15.16:0/1312744884 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2e81338a0 0x7fc2e8133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:08.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.589+0000 7fc2e77fe700 1 -- 172.21.15.16:0/1312744884 >> v1:172.21.15.16:6789/0 conn(0x7fc2e809de50 legacy=0x7fc2e809e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:08.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.589+0000 7fc2e5ffb700 1 -- 172.21.15.16:0/1312744884 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1811037980 0 0) 0x7fc2e806ef30 con 0x7fc2e809de50 2023-09-30T22:33:08.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.590+0000 7fc2e77fe700 1 -- 172.21.15.16:0/1312744884 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2e806fc30 con 0x7fc2e809e930 2023-09-30T22:33:08.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.590+0000 7fc2e6ffd700 1 --2- 172.21.15.16:0/1312744884 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2e81338a0 0x7fc2e8133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:08.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.590+0000 7fc2e77fe700 1 --2- 172.21.15.16:0/1312744884 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2e809e930 0x7fc2e813b740 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0009dc0 tx=0x7fc2e000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d757b369a02c465f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:08.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.590+0000 7fc2e5ffb700 1 -- 172.21.15.16:0/1312744884 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e000e070 con 0x7fc2e809e930 2023-09-30T22:33:08.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.590+0000 7fc2e5ffb700 1 -- 172.21.15.16:0/1312744884 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2e000bb20 con 0x7fc2e809e930 2023-09-30T22:33:08.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.590+0000 7fc2e5ffb700 1 -- 172.21.15.16:0/1312744884 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e0012430 con 0x7fc2e809e930 2023-09-30T22:33:08.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.591+0000 7fc2ed2d0700 1 -- 172.21.15.16:0/1312744884 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2e809e930 msgr2=0x7fc2e813b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:08.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.591+0000 7fc2ed2d0700 1 --2- 172.21.15.16:0/1312744884 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2e809e930 0x7fc2e813b740 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0009dc0 tx=0x7fc2e000b330 comp rx=0 tx=0).stop 2023-09-30T22:33:08.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.591+0000 7fc2ed2d0700 1 -- 172.21.15.16:0/1312744884 shutdown_connections 2023-09-30T22:33:08.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.591+0000 7fc2ed2d0700 1 --2- 172.21.15.16:0/1312744884 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2e809e930 0x7fc2e813b740 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:08.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.591+0000 7fc2ed2d0700 1 --2- 172.21.15.16:0/1312744884 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2e81338a0 0x7fc2e8133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:08.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.591+0000 7fc2ed2d0700 1 -- 172.21.15.16:0/1312744884 >> 172.21.15.16:0/1312744884 conn(0x7fc2e80a2aa0 msgr2=0x7fc2e80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:08.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.591+0000 7fc2ed2d0700 1 -- 172.21.15.16:0/1312744884 shutdown_connections 2023-09-30T22:33:08.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.591+0000 7fc2ed2d0700 1 -- 172.21.15.16:0/1312744884 wait complete. 2023-09-30T22:33:08.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.593+0000 7fc2ed2d0700 1 Processor -- start 2023-09-30T22:33:08.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.593+0000 7fc2ed2d0700 1 -- start start 2023-09-30T22:33:08.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.594+0000 7fc2ed2d0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2e809de50 0x7fc2e811d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:08.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.594+0000 7fc2ed2d0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2e809e930 0x7fc2e811dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:08.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.594+0000 7fc2ed2d0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2e81338a0 0x7fc2e811e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:08.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.594+0000 7fc2ed2d0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e8120de0 con 0x7fc2e81338a0 2023-09-30T22:33:08.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.594+0000 7fc2ed2d0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e811f2c0 con 0x7fc2e809e930 2023-09-30T22:33:08.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.594+0000 7fc2ed2d0700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e80a0d00 con 0x7fc2e809de50 2023-09-30T22:33:08.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.594+0000 7fc2e67fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2e809e930 0x7fc2e811dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:08.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.594+0000 7fc2e6ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2e809de50 0x7fc2e811d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:08.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.594+0000 7fc2e6ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2e809de50 0x7fc2e811d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:36960/0 (socket says 172.21.15.16:36960) 2023-09-30T22:33:08.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.594+0000 7fc2e6ffd700 1 -- 172.21.15.16:0/451195066 learned_addr learned my addr 172.21.15.16:0/451195066 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:08.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.594+0000 7fc2e67fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2e809e930 0x7fc2e811dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41528/0 (socket says 172.21.15.16:41528) 2023-09-30T22:33:08.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.594+0000 7fc2e77fe700 1 --2- 172.21.15.16:0/451195066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2e81338a0 0x7fc2e811e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:08.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.594+0000 7fc2e67fc700 1 -- 172.21.15.16:0/451195066 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2e809de50 msgr2=0x7fc2e811d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:08.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.595+0000 7fc2e67fc700 1 --2- 172.21.15.16:0/451195066 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2e809de50 0x7fc2e811d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:08.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.595+0000 7fc2e67fc700 1 -- 172.21.15.16:0/451195066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2e81338a0 msgr2=0x7fc2e811e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:08.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.595+0000 7fc2e67fc700 1 --2- 172.21.15.16:0/451195066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2e81338a0 0x7fc2e811e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:08.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.595+0000 7fc2e67fc700 1 -- 172.21.15.16:0/451195066 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d80094a0 con 0x7fc2e809e930 2023-09-30T22:33:08.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.595+0000 7fc2e67fc700 1 --2- 172.21.15.16:0/451195066 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2e809e930 0x7fc2e811dc10 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fc2e806d750 tx=0x7fc2dc00c790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:08.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.595+0000 7fc2cf7fe700 1 -- 172.21.15.16:0/451195066 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc004c80 con 0x7fc2e809e930 2023-09-30T22:33:08.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.595+0000 7fc2cf7fe700 1 -- 172.21.15.16:0/451195066 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2dc00ce90 con 0x7fc2e809e930 2023-09-30T22:33:08.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.595+0000 7fc2ed2d0700 1 -- 172.21.15.16:0/451195066 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2e0009a70 con 0x7fc2e809e930 2023-09-30T22:33:08.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.595+0000 7fc2cf7fe700 1 -- 172.21.15.16:0/451195066 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc007110 con 0x7fc2e809e930 2023-09-30T22:33:08.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.595+0000 7fc2ed2d0700 1 -- 172.21.15.16:0/451195066 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2e8120400 con 0x7fc2e809e930 2023-09-30T22:33:08.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.596+0000 7fc2cf7fe700 1 -- 172.21.15.16:0/451195066 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc2dc007270 con 0x7fc2e809e930 2023-09-30T22:33:08.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.596+0000 7fc2cf7fe700 1 -- 172.21.15.16:0/451195066 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fc2dc03f330 con 0x7fc2e809e930 2023-09-30T22:33:08.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.597+0000 7fc2ed2d0700 1 -- 172.21.15.16:0/451195066 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2e8120f20 con 0x7fc2e809e930 2023-09-30T22:33:08.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.606+0000 7fc2cf7fe700 1 -- 172.21.15.16:0/451195066 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc2e8120f20 con 0x7fc2e809e930 2023-09-30T22:33:08.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.627+0000 7f063e9c1700 1 Processor -- start 2023-09-30T22:33:08.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.627+0000 7f063e9c1700 1 -- start start 2023-09-30T22:33:08.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.627+0000 7f063e9c1700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0638133540 0x7f0638135920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:08.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.627+0000 7f063e9c1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0638135e60 0x7f0638138240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:08.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.627+0000 7f063e9c1700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06380716a0 con 0x7f0638130540 2023-09-30T22:33:08.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.627+0000 7f063e9c1700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0638071290 con 0x7f0638133540 2023-09-30T22:33:08.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.627+0000 7f063e9c1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0638130340 con 0x7f0638135e60 2023-09-30T22:33:08.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.628+0000 7f06377fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0638133540 0x7f0638135920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:08.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.628+0000 7f06377fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0638133540 0x7f0638135920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36544/0 (socket says 172.21.15.16:36544) 2023-09-30T22:33:08.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.628+0000 7f06377fe700 1 -- 172.21.15.16:0/573333617 learned_addr learned my addr 172.21.15.16:0/573333617 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:08.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.628+0000 7f063cf5e700 1 --2- 172.21.15.16:0/573333617 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0638135e60 0x7f0638138240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:08.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.628+0000 7f063cf5e700 1 -- 172.21.15.16:0/573333617 >> v2:172.21.15.16:3300/0 conn(0x7f0638133540 msgr2=0x7f0638135920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:08.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.628+0000 7f063cf5e700 1 --2- 172.21.15.16:0/573333617 >> v2:172.21.15.16:3300/0 conn(0x7f0638133540 0x7f0638135920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:08.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.628+0000 7f063cf5e700 1 -- 172.21.15.16:0/573333617 >> v1:172.21.15.16:6789/0 conn(0x7f0638130540 legacy=0x7f0638130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:08.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.628+0000 7f063cf5e700 1 -- 172.21.15.16:0/573333617 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0638070b80 con 0x7f0638135e60 2023-09-30T22:33:08.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.628+0000 7f063cf5e700 1 --2- 172.21.15.16:0/573333617 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0638135e60 0x7f0638138240 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0630009d90 tx=0x7f063000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8908697eeaa0e28 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:08.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.629+0000 7f0636ffd700 1 -- 172.21.15.16:0/573333617 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063000e070 con 0x7f0638135e60 2023-09-30T22:33:08.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.629+0000 7f0636ffd700 1 -- 172.21.15.16:0/573333617 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f063000bb20 con 0x7f0638135e60 2023-09-30T22:33:08.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.629+0000 7f0636ffd700 1 -- 172.21.15.16:0/573333617 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0630012430 con 0x7f0638135e60 2023-09-30T22:33:08.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.629+0000 7f063e9c1700 1 -- 172.21.15.16:0/573333617 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0638135e60 msgr2=0x7f0638138240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:08.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.629+0000 7f063e9c1700 1 --2- 172.21.15.16:0/573333617 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0638135e60 0x7f0638138240 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0630009d90 tx=0x7f063000b330 comp rx=0 tx=0).stop 2023-09-30T22:33:08.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.629+0000 7f063e9c1700 1 -- 172.21.15.16:0/573333617 shutdown_connections 2023-09-30T22:33:08.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.629+0000 7f063e9c1700 1 --2- 172.21.15.16:0/573333617 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0638135e60 0x7f0638138240 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:08.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.629+0000 7f063e9c1700 1 --2- 172.21.15.16:0/573333617 >> v2:172.21.15.16:3300/0 conn(0x7f0638133540 0x7f0638135920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:08.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.629+0000 7f063e9c1700 1 -- 172.21.15.16:0/573333617 >> 172.21.15.16:0/573333617 conn(0x7f06380a2aa0 msgr2=0x7f06380a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:08.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.630+0000 7f063e9c1700 1 -- 172.21.15.16:0/573333617 shutdown_connections 2023-09-30T22:33:08.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.630+0000 7f063e9c1700 1 -- 172.21.15.16:0/573333617 wait complete. 2023-09-30T22:33:08.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.631+0000 7f063e9c1700 1 Processor -- start 2023-09-30T22:33:08.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.631+0000 7f063e9c1700 1 -- start start 2023-09-30T22:33:08.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.632+0000 7f063e9c1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0638130540 0x7f06381c6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:08.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.632+0000 7f063e9c1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0638133540 0x7f06381c6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:08.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.632+0000 7f063e9c1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0638135e60 0x7f063806e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:08.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.632+0000 7f063e9c1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f063811bde0 con 0x7f0638133540 2023-09-30T22:33:08.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.632+0000 7f063e9c1700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06381c65d0 con 0x7f0638130540 2023-09-30T22:33:08.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.632+0000 7f063e9c1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f063811c260 con 0x7f0638135e60 2023-09-30T22:33:08.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.632+0000 7f0637fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0638130540 0x7f06381c6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:08.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.632+0000 7f0637fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0638130540 0x7f06381c6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41538/0 (socket says 172.21.15.16:41538) 2023-09-30T22:33:08.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.632+0000 7f0637fff700 1 -- 172.21.15.16:0/3230206294 learned_addr learned my addr 172.21.15.16:0/3230206294 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:08.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.632+0000 7f06377fe700 1 --2- 172.21.15.16:0/3230206294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0638133540 0x7f06381c6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:08.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.633+0000 7f83afe21700 1 Processor -- start 2023-09-30T22:33:08.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.633+0000 7f06377fe700 1 -- 172.21.15.16:0/3230206294 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0638135e60 msgr2=0x7f063806e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:08.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.633+0000 7f06377fe700 1 --2- 172.21.15.16:0/3230206294 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0638135e60 0x7f063806e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:08.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.633+0000 7f06377fe700 1 -- 172.21.15.16:0/3230206294 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0638130540 msgr2=0x7f06381c6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:08.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.633+0000 7f06377fe700 1 --2- 172.21.15.16:0/3230206294 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0638130540 0x7f06381c6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:08.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.633+0000 7f06377fe700 1 -- 172.21.15.16:0/3230206294 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0630009a70 con 0x7f0638133540 2023-09-30T22:33:08.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.633+0000 7f06377fe700 1 --2- 172.21.15.16:0/3230206294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0638133540 0x7f06381c6f60 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f062c00e8b0 tx=0x7f062c00ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:08.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.633+0000 7f0634ff9700 1 -- 172.21.15.16:0/3230206294 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c013680 con 0x7f0638133540 2023-09-30T22:33:08.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.633+0000 7f0634ff9700 1 -- 172.21.15.16:0/3230206294 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f062c0137e0 con 0x7f0638133540 2023-09-30T22:33:08.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.633+0000 7f0634ff9700 1 -- 172.21.15.16:0/3230206294 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c013b70 con 0x7f0638133540 2023-09-30T22:33:08.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.633+0000 7f83afe21700 1 -- start start 2023-09-30T22:33:08.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.634+0000 7f83afe21700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f83a809d6f0 0x7f83a809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:08.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.634+0000 7f83afe21700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83a809e410 0x7f83a8130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:08.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.634+0000 7f83afe21700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83a809f130 con 0x7f83a809e000 2023-09-30T22:33:08.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.634+0000 7f83afe21700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83a809f520 con 0x7f83a809e410 2023-09-30T22:33:08.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.634+0000 7f83afe21700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f83a809e930 con 0x7f83a809d6f0 2023-09-30T22:33:08.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.634+0000 7f83adbbd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f83a809d6f0 0x7f83a809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:08.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.634+0000 7f83adbbd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f83a809d6f0 0x7f83a809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37004/0 (socket says 172.21.15.16:37004) 2023-09-30T22:33:08.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.634+0000 7f83adbbd700 1 -- 172.21.15.16:0/4046438250 learned_addr learned my addr 172.21.15.16:0/4046438250 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:08.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.634+0000 7f83ad3bc700 1 --1- 172.21.15.16:0/4046438250 >> v1:172.21.15.16:6789/0 conn(0x7f83a809e000 0x7f83a8130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36368/0 (socket says 172.21.15.16:36368) 2023-09-30T22:33:08.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.634+0000 7f83adbbd700 1 -- 172.21.15.16:0/4046438250 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83a809e410 msgr2=0x7f83a8130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:08.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.634+0000 7f83adbbd700 1 --2- 172.21.15.16:0/4046438250 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83a809e410 0x7f83a8130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:08.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.634+0000 7f83adbbd700 1 -- 172.21.15.16:0/4046438250 >> v1:172.21.15.16:6789/0 conn(0x7f83a809e000 legacy=0x7f83a8130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:08.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.634+0000 7f83adbbd700 1 -- 172.21.15.16:0/4046438250 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a809b2d0 con 0x7f83a809d6f0 2023-09-30T22:33:08.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.634+0000 7f063e9c1700 1 -- 172.21.15.16:0/3230206294 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f063811ca50 con 0x7f0638133540 2023-09-30T22:33:08.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.634+0000 7f063e9c1700 1 -- 172.21.15.16:0/3230206294 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f063811b1f0 con 0x7f0638133540 2023-09-30T22:33:08.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.635+0000 7f83adbbd700 1 --2- 172.21.15.16:0/4046438250 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f83a809d6f0 0x7f83a809dac0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f839800bbc0 tx=0x7f839800bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=156606d864ff1baf server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:08.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.635+0000 7f0634ff9700 1 -- 172.21.15.16:0/3230206294 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f062c013cd0 con 0x7f0638133540 2023-09-30T22:33:08.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.635+0000 7f0634ff9700 1 -- 172.21.15.16:0/3230206294 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f062c03f690 con 0x7f0638133540 2023-09-30T22:33:08.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.636+0000 7f83acbbb700 1 -- 172.21.15.16:0/4046438250 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f839800dd90 con 0x7f83a809d6f0 2023-09-30T22:33:08.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.636+0000 7f063e9c1700 1 -- 172.21.15.16:0/3230206294 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f063809eb50 con 0x7f0638133540 2023-09-30T22:33:08.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.636+0000 7f83acbbb700 1 -- 172.21.15.16:0/4046438250 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8398010470 con 0x7f83a809d6f0 2023-09-30T22:33:08.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.636+0000 7f83acbbb700 1 -- 172.21.15.16:0/4046438250 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8398010800 con 0x7f83a809d6f0 2023-09-30T22:33:08.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.637+0000 7f83afe21700 1 -- 172.21.15.16:0/4046438250 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f83a809d6f0 msgr2=0x7f83a809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:08.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.637+0000 7f83afe21700 1 --2- 172.21.15.16:0/4046438250 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f83a809d6f0 0x7f83a809dac0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f839800bbc0 tx=0x7f839800bed0 comp rx=0 tx=0).stop 2023-09-30T22:33:08.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.637+0000 7f83afe21700 1 -- 172.21.15.16:0/4046438250 shutdown_connections 2023-09-30T22:33:08.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.637+0000 7f83afe21700 1 --2- 172.21.15.16:0/4046438250 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83a809e410 0x7f83a8130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:08.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.637+0000 7f83afe21700 1 --2- 172.21.15.16:0/4046438250 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f83a809d6f0 0x7f83a809dac0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:08.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.637+0000 7f83afe21700 1 -- 172.21.15.16:0/4046438250 >> 172.21.15.16:0/4046438250 conn(0x7f83a8066f50 msgr2=0x7f83a809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:08.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.637+0000 7f83afe21700 1 -- 172.21.15.16:0/4046438250 shutdown_connections 2023-09-30T22:33:08.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.637+0000 7f83afe21700 1 -- 172.21.15.16:0/4046438250 wait complete. 2023-09-30T22:33:08.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.638+0000 7f83afe21700 1 Processor -- start 2023-09-30T22:33:08.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.638+0000 7f83afe21700 1 -- start start 2023-09-30T22:33:08.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.639+0000 7f83afe21700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83a809d6f0 0x7f83a811e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:08.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.639+0000 7f83afe21700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83a809e000 0x7f83a811e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:08.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.639+0000 7f83afe21700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f83a809e410 0x7f83a811ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:08.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.639+0000 7f83afe21700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83a8121cd0 con 0x7f83a809d6f0 2023-09-30T22:33:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.639+0000 7f83afe21700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83a80a1e00 con 0x7f83a809e000 2023-09-30T22:33:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.639+0000 7f83afe21700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f83a809c700 con 0x7f83a809e410 2023-09-30T22:33:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.639+0000 7f83adbbd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83a809d6f0 0x7f83a811e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.639+0000 7f83adbbd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83a809d6f0 0x7f83a811e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36564/0 (socket says 172.21.15.16:36564) 2023-09-30T22:33:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.639+0000 7f83adbbd700 1 -- 172.21.15.16:0/3186350724 learned_addr learned my addr 172.21.15.16:0/3186350724 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.639+0000 7f83ae3be700 1 --2- 172.21.15.16:0/3186350724 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f83a809e410 0x7f83a811ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.639+0000 7f83ad3bc700 1 --2- 172.21.15.16:0/3186350724 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83a809e000 0x7f83a811e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.639+0000 7f83adbbd700 1 -- 172.21.15.16:0/3186350724 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f83a809e410 msgr2=0x7f83a811ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.639+0000 7f83adbbd700 1 --2- 172.21.15.16:0/3186350724 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f83a809e410 0x7f83a811ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.639+0000 7f83adbbd700 1 -- 172.21.15.16:0/3186350724 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83a809e000 msgr2=0x7f83a811e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.639+0000 7f83adbbd700 1 --2- 172.21.15.16:0/3186350724 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83a809e000 0x7f83a811e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:08.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.639+0000 7f83adbbd700 1 -- 172.21.15.16:0/3186350724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f839800b7f0 con 0x7f83a809d6f0 2023-09-30T22:33:08.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.639+0000 7f83ae3be700 1 --2- 172.21.15.16:0/3186350724 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f83a809e410 0x7f83a811ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:08.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.639+0000 7f83ad3bc700 1 --2- 172.21.15.16:0/3186350724 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83a809e000 0x7f83a811e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:33:08.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.640+0000 7f83adbbd700 1 --2- 172.21.15.16:0/3186350724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83a809d6f0 0x7f83a811e2e0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f8398000c00 tx=0x7f8398004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:08.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.641+0000 7f839e7fc700 1 -- 172.21.15.16:0/3186350724 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8398003c30 con 0x7f83a809d6f0 2023-09-30T22:33:08.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.641+0000 7f839e7fc700 1 -- 172.21.15.16:0/3186350724 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8398003db0 con 0x7f83a809d6f0 2023-09-30T22:33:08.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.641+0000 7f839e7fc700 1 -- 172.21.15.16:0/3186350724 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83980226b0 con 0x7f83a809d6f0 2023-09-30T22:33:08.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.641+0000 7f0634ff9700 1 -- 172.21.15.16:0/3230206294 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f062c017080 con 0x7f0638133540 2023-09-30T22:33:08.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.642+0000 7f83afe21700 1 -- 172.21.15.16:0/3186350724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83a8121730 con 0x7f83a809d6f0 2023-09-30T22:33:08.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.642+0000 7f83afe21700 1 -- 172.21.15.16:0/3186350724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83a8120310 con 0x7f83a809d6f0 2023-09-30T22:33:08.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.644+0000 7f83afe21700 1 -- 172.21.15.16:0/3186350724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83a8074ea0 con 0x7f83a809d6f0 2023-09-30T22:33:08.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.648+0000 7f839e7fc700 1 -- 172.21.15.16:0/3186350724 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f839801f070 con 0x7f83a809d6f0 2023-09-30T22:33:08.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.649+0000 7f839e7fc700 1 -- 172.21.15.16:0/3186350724 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f8398052d50 con 0x7f83a809d6f0 2023-09-30T22:33:08.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.653+0000 7f839e7fc700 1 -- 172.21.15.16:0/3186350724 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8398020ba0 con 0x7f83a809d6f0 2023-09-30T22:33:08.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.681+0000 7f832a350700 1 Processor -- start 2023-09-30T22:33:08.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.681+0000 7f832a350700 1 -- start start 2023-09-30T22:33:08.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.682+0000 7f832a350700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f832409d6f0 0x7f832409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:08.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.682+0000 7f832a350700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f832409e410 0x7f8324130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:08.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.682+0000 7f832a350700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f832409f130 con 0x7f832409e000 2023-09-30T22:33:08.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.682+0000 7f832a350700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f832409f520 con 0x7f832409e410 2023-09-30T22:33:08.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.682+0000 7f832a350700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f832409e930 con 0x7f832409d6f0 2023-09-30T22:33:08.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.682+0000 7f83288ed700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f832409e410 0x7f8324130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:08.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.682+0000 7f83237fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f832409e000 0x7f8324130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36380/0 (socket says 172.21.15.16:36380) 2023-09-30T22:33:08.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.682+0000 7f83237fe700 1 -- 172.21.15.16:0/96650902 learned_addr learned my addr 172.21.15.16:0/96650902 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:08.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.682+0000 7f8323fff700 1 --2- 172.21.15.16:0/96650902 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f832409d6f0 0x7f832409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:08.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.683+0000 7f83288ed700 1 -- 172.21.15.16:0/96650902 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f832409d6f0 msgr2=0x7f832409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:08.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.683+0000 7f83288ed700 1 --2- 172.21.15.16:0/96650902 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f832409d6f0 0x7f832409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:08.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.683+0000 7f83288ed700 1 -- 172.21.15.16:0/96650902 >> v1:172.21.15.16:6789/0 conn(0x7f832409e000 legacy=0x7f8324130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:08.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.683+0000 7f83288ed700 1 -- 172.21.15.16:0/96650902 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f832409b2d0 con 0x7f832409e410 2023-09-30T22:33:08.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.683+0000 7f83288ed700 1 --2- 172.21.15.16:0/96650902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f832409e410 0x7f8324130d70 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f831c009d90 tx=0x7f831c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bc9fc0ee460f4ae server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:08.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.683+0000 7f8322ffd700 1 -- 172.21.15.16:0/96650902 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831c00e070 con 0x7f832409e410 2023-09-30T22:33:08.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.683+0000 7f8322ffd700 1 -- 172.21.15.16:0/96650902 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f831c00bb20 con 0x7f832409e410 2023-09-30T22:33:08.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.683+0000 7f8322ffd700 1 -- 172.21.15.16:0/96650902 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831c012430 con 0x7f832409e410 2023-09-30T22:33:08.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.684+0000 7f832a350700 1 -- 172.21.15.16:0/96650902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f832409e410 msgr2=0x7f8324130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:08.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.684+0000 7f832a350700 1 --2- 172.21.15.16:0/96650902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f832409e410 0x7f8324130d70 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f831c009d90 tx=0x7f831c00b330 comp rx=0 tx=0).stop 2023-09-30T22:33:08.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.684+0000 7f832a350700 1 -- 172.21.15.16:0/96650902 shutdown_connections 2023-09-30T22:33:08.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.684+0000 7f832a350700 1 --2- 172.21.15.16:0/96650902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f832409e410 0x7f8324130d70 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:08.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.684+0000 7f832a350700 1 --2- 172.21.15.16:0/96650902 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f832409d6f0 0x7f832409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:08.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.684+0000 7f832a350700 1 -- 172.21.15.16:0/96650902 >> 172.21.15.16:0/96650902 conn(0x7f8324066f50 msgr2=0x7f832409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:08.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.684+0000 7f832a350700 1 -- 172.21.15.16:0/96650902 shutdown_connections 2023-09-30T22:33:08.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.684+0000 7f832a350700 1 -- 172.21.15.16:0/96650902 wait complete. 2023-09-30T22:33:08.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.685+0000 7f832a350700 1 Processor -- start 2023-09-30T22:33:08.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.685+0000 7f832a350700 1 -- start start 2023-09-30T22:33:08.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.686+0000 7f832a350700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f832409d6f0 0x7f83241300c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:08.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.686+0000 7f832a350700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f832409e000 0x7f8324130600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:08.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.686+0000 7f832a350700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f832409e410 0x7f8324130b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:08.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.686+0000 7f832a350700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8324139530 con 0x7f832409e000 2023-09-30T22:33:08.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.686+0000 7f832a350700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f832412dcc0 con 0x7f832409e410 2023-09-30T22:33:08.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.686+0000 7f832a350700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f832413ad30 con 0x7f832409d6f0 2023-09-30T22:33:08.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.686+0000 7f8323fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f832409d6f0 0x7f83241300c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:08.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.686+0000 7f8323fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f832409d6f0 0x7f83241300c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37026/0 (socket says 172.21.15.16:37026) 2023-09-30T22:33:08.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.686+0000 7f8323fff700 1 -- 172.21.15.16:0/2460377779 learned_addr learned my addr 172.21.15.16:0/2460377779 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:08.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.686+0000 7f83237fe700 1 --2- 172.21.15.16:0/2460377779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f832409e000 0x7f8324130600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:08.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.686+0000 7f8323fff700 1 -- 172.21.15.16:0/2460377779 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f832409e410 msgr2=0x7f8324130b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:08.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.686+0000 7f8323fff700 1 --2- 172.21.15.16:0/2460377779 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f832409e410 0x7f8324130b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:08.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.686+0000 7f8323fff700 1 -- 172.21.15.16:0/2460377779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f832409e000 msgr2=0x7f8324130600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:08.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.686+0000 7f8323fff700 1 --2- 172.21.15.16:0/2460377779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f832409e000 0x7f8324130600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:08.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.686+0000 7f8323fff700 1 -- 172.21.15.16:0/2460377779 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f831c009a70 con 0x7f832409d6f0 2023-09-30T22:33:08.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.687+0000 7f8323fff700 1 --2- 172.21.15.16:0/2460377779 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f832409d6f0 0x7f83241300c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f831400e8a0 tx=0x7f831400ebb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:08.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.687+0000 7f8320ff9700 1 -- 172.21.15.16:0/2460377779 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8314009590 con 0x7f832409d6f0 2023-09-30T22:33:08.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.687+0000 7f8320ff9700 1 -- 172.21.15.16:0/2460377779 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8314013680 con 0x7f832409d6f0 2023-09-30T22:33:08.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.687+0000 7f8320ff9700 1 -- 172.21.15.16:0/2460377779 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83140139f0 con 0x7f832409d6f0 2023-09-30T22:33:08.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.688+0000 7f832a350700 1 -- 172.21.15.16:0/2460377779 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f832412f4e0 con 0x7f832409d6f0 2023-09-30T22:33:08.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.688+0000 7f832a350700 1 -- 172.21.15.16:0/2460377779 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f832412e910 con 0x7f832409d6f0 2023-09-30T22:33:08.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.689+0000 7f8320ff9700 1 -- 172.21.15.16:0/2460377779 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8314013b50 con 0x7f832409d6f0 2023-09-30T22:33:08.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.689+0000 7f832a350700 1 -- 172.21.15.16:0/2460377779 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8324074ea0 con 0x7f832409d6f0 2023-09-30T22:33:08.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.689+0000 7f8320ff9700 1 -- 172.21.15.16:0/2460377779 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f831403f7d0 con 0x7f832409d6f0 2023-09-30T22:33:08.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.694+0000 7f8320ff9700 1 -- 172.21.15.16:0/2460377779 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f831401a950 con 0x7f832409d6f0 2023-09-30T22:33:08.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.846+0000 7fc2ed2d0700 1 -- 172.21.15.16:0/451195066 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc2e8120f20 con 0x7fc2e809e930 2023-09-30T22:33:08.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.846+0000 7fc2cf7fe700 1 -- 172.21.15.16:0/451195066 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc2e8120f20 con 0x7fc2e809e930 2023-09-30T22:33:08.847 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:33:08.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.849+0000 7fc2ed2d0700 1 -- 172.21.15.16:0/451195066 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2e809e930 msgr2=0x7fc2e811dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:08.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.849+0000 7fc2ed2d0700 1 --2- 172.21.15.16:0/451195066 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2e809e930 0x7fc2e811dc10 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fc2e806d750 tx=0x7fc2dc00c790 comp rx=0 tx=0).stop 2023-09-30T22:33:08.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.849+0000 7fc2ed2d0700 1 -- 172.21.15.16:0/451195066 shutdown_connections 2023-09-30T22:33:08.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.849+0000 7fc2ed2d0700 1 --2- 172.21.15.16:0/451195066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2e81338a0 0x7fc2e811e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:08.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.849+0000 7fc2ed2d0700 1 --2- 172.21.15.16:0/451195066 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2e809de50 0x7fc2e811d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:08.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.849+0000 7fc2ed2d0700 1 --2- 172.21.15.16:0/451195066 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2e809e930 0x7fc2e811dc10 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:08.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.849+0000 7fc2ed2d0700 1 -- 172.21.15.16:0/451195066 >> 172.21.15.16:0/451195066 conn(0x7fc2e80a2aa0 msgr2=0x7fc2e81badc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:08.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.851+0000 7fc2ed2d0700 1 -- 172.21.15.16:0/451195066 shutdown_connections 2023-09-30T22:33:08.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.851+0000 7fc2ed2d0700 1 -- 172.21.15.16:0/451195066 wait complete. 2023-09-30T22:33:08.870 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:33:08.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.874+0000 7f063e9c1700 1 -- 172.21.15.16:0/3230206294 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f063811cf50 con 0x7f0638133540 2023-09-30T22:33:08.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.874+0000 7f0634ff9700 1 -- 172.21.15.16:0/3230206294 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f063811cf50 con 0x7f0638133540 2023-09-30T22:33:08.875 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:33:08.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.876+0000 7f063e9c1700 1 -- 172.21.15.16:0/3230206294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0638133540 msgr2=0x7f06381c6f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:08.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.876+0000 7f063e9c1700 1 --2- 172.21.15.16:0/3230206294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0638133540 0x7f06381c6f60 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f062c00e8b0 tx=0x7f062c00ebc0 comp rx=0 tx=0).stop 2023-09-30T22:33:08.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.877+0000 7f063e9c1700 1 -- 172.21.15.16:0/3230206294 shutdown_connections 2023-09-30T22:33:08.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.877+0000 7f063e9c1700 1 --2- 172.21.15.16:0/3230206294 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0638130540 0x7f06381c6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:08.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.877+0000 7f063e9c1700 1 --2- 172.21.15.16:0/3230206294 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0638135e60 0x7f063806e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:08.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.877+0000 7f063e9c1700 1 --2- 172.21.15.16:0/3230206294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0638133540 0x7f06381c6f60 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:08.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.877+0000 7f063e9c1700 1 -- 172.21.15.16:0/3230206294 >> 172.21.15.16:0/3230206294 conn(0x7f06380a2aa0 msgr2=0x7f063806f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:08.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.877+0000 7f063e9c1700 1 -- 172.21.15.16:0/3230206294 shutdown_connections 2023-09-30T22:33:08.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.877+0000 7f063e9c1700 1 -- 172.21.15.16:0/3230206294 wait complete. 2023-09-30T22:33:08.891 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:33:08.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.914+0000 7f83afe21700 1 -- 172.21.15.16:0/3186350724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f83a8121010 con 0x7f83a809d6f0 2023-09-30T22:33:08.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.914+0000 7f839e7fc700 1 -- 172.21.15.16:0/3186350724 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8398020370 con 0x7f83a809d6f0 2023-09-30T22:33:08.915 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:33:08.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.916+0000 7f83afe21700 1 -- 172.21.15.16:0/3186350724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83a809d6f0 msgr2=0x7f83a811e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:08.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.916+0000 7f83afe21700 1 --2- 172.21.15.16:0/3186350724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83a809d6f0 0x7f83a811e2e0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f8398000c00 tx=0x7f8398004750 comp rx=0 tx=0).stop 2023-09-30T22:33:08.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.916+0000 7f83afe21700 1 -- 172.21.15.16:0/3186350724 shutdown_connections 2023-09-30T22:33:08.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.916+0000 7f83afe21700 1 --2- 172.21.15.16:0/3186350724 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f83a809e410 0x7f83a811ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:08.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.916+0000 7f83afe21700 1 --2- 172.21.15.16:0/3186350724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83a809d6f0 0x7f83a811e2e0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:08.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.916+0000 7f83afe21700 1 --2- 172.21.15.16:0/3186350724 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83a809e000 0x7f83a811e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:08.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.916+0000 7f83afe21700 1 -- 172.21.15.16:0/3186350724 >> 172.21.15.16:0/3186350724 conn(0x7f83a8066f50 msgr2=0x7f83a806f120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:08.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.916+0000 7f83afe21700 1 -- 172.21.15.16:0/3186350724 shutdown_connections 2023-09-30T22:33:08.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.917+0000 7f83afe21700 1 -- 172.21.15.16:0/3186350724 wait complete. 2023-09-30T22:33:08.931 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:33:08.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.936+0000 7f832a350700 1 -- 172.21.15.16:0/2460377779 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f832412e910 con 0x7f832409d6f0 2023-09-30T22:33:08.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.936+0000 7f8320ff9700 1 -- 172.21.15.16:0/2460377779 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f832412e910 con 0x7f832409d6f0 2023-09-30T22:33:08.937 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:33:08.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.939+0000 7f832a350700 1 -- 172.21.15.16:0/2460377779 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f832409d6f0 msgr2=0x7f83241300c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:08.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.939+0000 7f832a350700 1 --2- 172.21.15.16:0/2460377779 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f832409d6f0 0x7f83241300c0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f831400e8a0 tx=0x7f831400ebb0 comp rx=0 tx=0).stop 2023-09-30T22:33:08.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.939+0000 7f832a350700 1 -- 172.21.15.16:0/2460377779 shutdown_connections 2023-09-30T22:33:08.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.939+0000 7f832a350700 1 --2- 172.21.15.16:0/2460377779 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f832409e410 0x7f8324130b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:08.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.939+0000 7f832a350700 1 --2- 172.21.15.16:0/2460377779 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f832409d6f0 0x7f83241300c0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:08.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.939+0000 7f832a350700 1 --2- 172.21.15.16:0/2460377779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f832409e000 0x7f8324130600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:08.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.939+0000 7f832a350700 1 -- 172.21.15.16:0/2460377779 >> 172.21.15.16:0/2460377779 conn(0x7f8324066f50 msgr2=0x7f83240a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:08.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.939+0000 7f832a350700 1 -- 172.21.15.16:0/2460377779 shutdown_connections 2023-09-30T22:33:08.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:08.939+0000 7f832a350700 1 -- 172.21.15.16:0/2460377779 wait complete. 2023-09-30T22:33:08.953 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:33:09.871 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:33:09.892 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:33:09.932 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:33:09.953 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:33:10.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.027+0000 7fdc9b3d8700 1 Processor -- start 2023-09-30T22:33:10.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.027+0000 7fdc9b3d8700 1 -- start start 2023-09-30T22:33:10.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.028+0000 7fdc9b3d8700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdc9406e570 0x7fdc9406e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:10.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.028+0000 7fdc9b3d8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdc9409eca0 0x7fdc9406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:10.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.028+0000 7fdc9b3d8700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc94070ee0 con 0x7fdc94134610 2023-09-30T22:33:10.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.028+0000 7fdc9b3d8700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc94071480 con 0x7fdc9406e570 2023-09-30T22:33:10.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.028+0000 7fdc9b3d8700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9406f3c0 con 0x7fdc9409eca0 2023-09-30T22:33:10.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.028+0000 7fdc99975700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fdc94134610 0x7fdc94134a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36390/0 (socket says 172.21.15.16:36390) 2023-09-30T22:33:10.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.028+0000 7fdc99174700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdc9406e570 0x7fdc9406e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:10.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.028+0000 7fdc98973700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdc9409eca0 0x7fdc9406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:10.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.028+0000 7fdc99975700 1 -- 172.21.15.16:0/3939601291 learned_addr learned my addr 172.21.15.16:0/3939601291 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:10.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.028+0000 7fdc99174700 1 -- 172.21.15.16:0/3939601291 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdc9409eca0 msgr2=0x7fdc9406ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:10.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.028+0000 7fdc99174700 1 --2- 172.21.15.16:0/3939601291 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdc9409eca0 0x7fdc9406ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:10.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.028+0000 7fdc99174700 1 -- 172.21.15.16:0/3939601291 >> v1:172.21.15.16:6789/0 conn(0x7fdc94134610 legacy=0x7fdc94134a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:10.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.028+0000 7fdc8bfff700 1 -- 172.21.15.16:0/3939601291 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 183375793 0 0) 0x7fdc94070ee0 con 0x7fdc94134610 2023-09-30T22:33:10.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.028+0000 7fdc99174700 1 -- 172.21.15.16:0/3939601291 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc94071b90 con 0x7fdc9406e570 2023-09-30T22:33:10.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.029+0000 7fdc99174700 1 --2- 172.21.15.16:0/3939601291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc9406e570 0x7fdc9406e940 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fdc90009840 tx=0x7fdc90009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87686d1194035bf server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:10.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.029+0000 7fdc8bfff700 1 -- 172.21.15.16:0/3939601291 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc90011830 con 0x7fdc9406e570 2023-09-30T22:33:10.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.029+0000 7fdc8bfff700 1 -- 172.21.15.16:0/3939601291 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc90011990 con 0x7fdc9406e570 2023-09-30T22:33:10.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.029+0000 7fdc8bfff700 1 -- 172.21.15.16:0/3939601291 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc90011d20 con 0x7fdc9406e570 2023-09-30T22:33:10.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.030+0000 7fdc9b3d8700 1 -- 172.21.15.16:0/3939601291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc9406e570 msgr2=0x7fdc9406e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:10.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.030+0000 7fdc9b3d8700 1 --2- 172.21.15.16:0/3939601291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc9406e570 0x7fdc9406e940 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fdc90009840 tx=0x7fdc90009b50 comp rx=0 tx=0).stop 2023-09-30T22:33:10.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.030+0000 7fdc9b3d8700 1 -- 172.21.15.16:0/3939601291 shutdown_connections 2023-09-30T22:33:10.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.030+0000 7fdc9b3d8700 1 --2- 172.21.15.16:0/3939601291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc9406e570 0x7fdc9406e940 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:10.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.030+0000 7fdc9b3d8700 1 --2- 172.21.15.16:0/3939601291 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdc9409eca0 0x7fdc9406ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:10.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.030+0000 7fdc9b3d8700 1 -- 172.21.15.16:0/3939601291 >> 172.21.15.16:0/3939601291 conn(0x7fdc940a2aa0 msgr2=0x7fdc940a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:10.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.030+0000 7fdc9b3d8700 1 -- 172.21.15.16:0/3939601291 shutdown_connections 2023-09-30T22:33:10.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.030+0000 7fdc9b3d8700 1 -- 172.21.15.16:0/3939601291 wait complete. 2023-09-30T22:33:10.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.031+0000 7fdc9b3d8700 1 Processor -- start 2023-09-30T22:33:10.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.031+0000 7fdc9b3d8700 1 -- start start 2023-09-30T22:33:10.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.032+0000 7fdc9b3d8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdc94134610 0x7fdc9413a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:10.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.032+0000 7fdc9b3d8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc941389f0 0x7fdc94138de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:10.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.032+0000 7fdc9b3d8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdc94139320 0x7fdc94133080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:10.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.032+0000 7fdc9b3d8700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc94139d50 con 0x7fdc941389f0 2023-09-30T22:33:10.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.032+0000 7fdc9b3d8700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc941343c0 con 0x7fdc94134610 2023-09-30T22:33:10.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.032+0000 7fdc9b3d8700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdc94140630 con 0x7fdc94139320 2023-09-30T22:33:10.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.032+0000 7fdc99174700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdc94134610 0x7fdc9413a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:10.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.032+0000 7fdc99174700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdc94134610 0x7fdc9413a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41586/0 (socket says 172.21.15.16:41586) 2023-09-30T22:33:10.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.032+0000 7fdc99174700 1 -- 172.21.15.16:0/3685809299 learned_addr learned my addr 172.21.15.16:0/3685809299 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:10.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.032+0000 7fdc98973700 1 --2- 172.21.15.16:0/3685809299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc941389f0 0x7fdc94138de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:10.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.032+0000 7fdc99174700 1 -- 172.21.15.16:0/3685809299 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdc94139320 msgr2=0x7fdc94133080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:10.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.032+0000 7fdc99174700 1 --2- 172.21.15.16:0/3685809299 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdc94139320 0x7fdc94133080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:10.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.032+0000 7fdc99174700 1 -- 172.21.15.16:0/3685809299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc941389f0 msgr2=0x7fdc94138de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:10.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.032+0000 7fdc99174700 1 --2- 172.21.15.16:0/3685809299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc941389f0 0x7fdc94138de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:10.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.032+0000 7fdc99174700 1 -- 172.21.15.16:0/3685809299 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc900094a0 con 0x7fdc94134610 2023-09-30T22:33:10.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.032+0000 7fdc99174700 1 --2- 172.21.15.16:0/3685809299 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdc94134610 0x7fdc9413a1d0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fdc90009fd0 tx=0x7fdc90004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:10.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.033+0000 7fdc89ffb700 1 -- 172.21.15.16:0/3685809299 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc900045d0 con 0x7fdc94134610 2023-09-30T22:33:10.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.033+0000 7fdc89ffb700 1 -- 172.21.15.16:0/3685809299 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc9001f070 con 0x7fdc94134610 2023-09-30T22:33:10.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.033+0000 7fdc9b3d8700 1 -- 172.21.15.16:0/3685809299 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc9413b880 con 0x7fdc94134610 2023-09-30T22:33:10.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.033+0000 7fdc89ffb700 1 -- 172.21.15.16:0/3685809299 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc900038a0 con 0x7fdc94134610 2023-09-30T22:33:10.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.033+0000 7fdc9b3d8700 1 -- 172.21.15.16:0/3685809299 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc941397a0 con 0x7fdc94134610 2023-09-30T22:33:10.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.034+0000 7fdc89ffb700 1 -- 172.21.15.16:0/3685809299 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdc90003a00 con 0x7fdc94134610 2023-09-30T22:33:10.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.034+0000 7fdc89ffb700 1 -- 172.21.15.16:0/3685809299 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fdc90021070 con 0x7fdc94134610 2023-09-30T22:33:10.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.034+0000 7fdc9b3d8700 1 -- 172.21.15.16:0/3685809299 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc9413ad30 con 0x7fdc94134610 2023-09-30T22:33:10.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.039+0000 7fdc89ffb700 1 -- 172.21.15.16:0/3685809299 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdc9002b080 con 0x7fdc94134610 2023-09-30T22:33:10.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.075+0000 7f49858dc700 1 Processor -- start 2023-09-30T22:33:10.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.076+0000 7f49858dc700 1 -- start start 2023-09-30T22:33:10.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.076+0000 7f49858dc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4980130540 0x7f4980130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:10.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.076+0000 7f49858dc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4980130e50 0x7f498006f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:10.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.076+0000 7f49858dc700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49800716a0 con 0x7f498006fb20 2023-09-30T22:33:10.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.076+0000 7f49858dc700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f498009a390 con 0x7f4980130e50 2023-09-30T22:33:10.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.076+0000 7f49858dc700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4980071290 con 0x7f4980130540 2023-09-30T22:33:10.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.077+0000 7f497e7fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4980130e50 0x7f498006f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:10.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.077+0000 7f497f7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f498006fb20 0x7f4980137560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36396/0 (socket says 172.21.15.16:36396) 2023-09-30T22:33:10.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.077+0000 7f497f7fe700 1 -- 172.21.15.16:0/432051894 learned_addr learned my addr 172.21.15.16:0/432051894 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:10.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.077+0000 7f497effd700 1 --2- 172.21.15.16:0/432051894 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4980130540 0x7f4980130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:10.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.077+0000 7f497e7fc700 1 -- 172.21.15.16:0/432051894 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4980130540 msgr2=0x7f4980130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:10.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.077+0000 7f497e7fc700 1 --2- 172.21.15.16:0/432051894 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4980130540 0x7f4980130910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:10.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.077+0000 7f497e7fc700 1 -- 172.21.15.16:0/432051894 >> v1:172.21.15.16:6789/0 conn(0x7f498006fb20 legacy=0x7f4980137560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:10.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.077+0000 7f497e7fc700 1 -- 172.21.15.16:0/432051894 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4980130340 con 0x7f4980130e50 2023-09-30T22:33:10.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.077+0000 7f497e7fc700 1 --2- 172.21.15.16:0/432051894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4980130e50 0x7f498006f5e0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f4974009840 tx=0x7f4974009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41883f6e29d8e99d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:10.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.077+0000 7f497dffb700 1 -- 172.21.15.16:0/432051894 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4974004230 con 0x7f4980130e50 2023-09-30T22:33:10.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.078+0000 7f497dffb700 1 -- 172.21.15.16:0/432051894 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4974004390 con 0x7f4980130e50 2023-09-30T22:33:10.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.078+0000 7f497dffb700 1 -- 172.21.15.16:0/432051894 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49740036a0 con 0x7f4980130e50 2023-09-30T22:33:10.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.078+0000 7f49858dc700 1 -- 172.21.15.16:0/432051894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4980130e50 msgr2=0x7f498006f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:10.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.078+0000 7f49858dc700 1 --2- 172.21.15.16:0/432051894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4980130e50 0x7f498006f5e0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f4974009840 tx=0x7f4974009b50 comp rx=0 tx=0).stop 2023-09-30T22:33:10.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.078+0000 7f49858dc700 1 -- 172.21.15.16:0/432051894 shutdown_connections 2023-09-30T22:33:10.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.078+0000 7f49858dc700 1 --2- 172.21.15.16:0/432051894 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4980130540 0x7f4980130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:10.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.078+0000 7f49858dc700 1 --2- 172.21.15.16:0/432051894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4980130e50 0x7f498006f5e0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:10.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.078+0000 7f49858dc700 1 -- 172.21.15.16:0/432051894 >> 172.21.15.16:0/432051894 conn(0x7f49800a2aa0 msgr2=0x7f49800a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:10.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.078+0000 7f49858dc700 1 -- 172.21.15.16:0/432051894 shutdown_connections 2023-09-30T22:33:10.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.078+0000 7f49858dc700 1 -- 172.21.15.16:0/432051894 wait complete. 2023-09-30T22:33:10.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.080+0000 7f49858dc700 1 Processor -- start 2023-09-30T22:33:10.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.080+0000 7f49858dc700 1 -- start start 2023-09-30T22:33:10.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.081+0000 7f49858dc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f498006fb20 0x7f498009c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:10.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.081+0000 7f49858dc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4980130540 0x7f498009c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:10.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.081+0000 7f49858dc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4980130e50 0x7f49801cc480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:10.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.081+0000 7f49858dc700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49800718e0 con 0x7f4980130e50 2023-09-30T22:33:10.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.081+0000 7f49858dc700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f498009c050 con 0x7f4980130540 2023-09-30T22:33:10.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.081+0000 7f49858dc700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4980126110 con 0x7f498006fb20 2023-09-30T22:33:10.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.081+0000 7f497f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4980130e50 0x7f49801cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:10.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.081+0000 7f497effd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f498006fb20 0x7f498009c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:10.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.081+0000 7f497effd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f498006fb20 0x7f498009c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37056/0 (socket says 172.21.15.16:37056) 2023-09-30T22:33:10.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.081+0000 7f497effd700 1 -- 172.21.15.16:0/3559245837 learned_addr learned my addr 172.21.15.16:0/3559245837 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:10.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.081+0000 7f497e7fc700 1 --2- 172.21.15.16:0/3559245837 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4980130540 0x7f498009c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:10.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.082+0000 7f497effd700 1 -- 172.21.15.16:0/3559245837 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4980130540 msgr2=0x7f498009c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:10.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.082+0000 7f497effd700 1 --2- 172.21.15.16:0/3559245837 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4980130540 0x7f498009c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:10.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.082+0000 7f497effd700 1 -- 172.21.15.16:0/3559245837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4980130e50 msgr2=0x7f49801cc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:10.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.082+0000 7f497effd700 1 --2- 172.21.15.16:0/3559245837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4980130e50 0x7f49801cc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:10.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.082+0000 7f497effd700 1 -- 172.21.15.16:0/3559245837 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49740094a0 con 0x7f498006fb20 2023-09-30T22:33:10.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.082+0000 7f497effd700 1 --2- 172.21.15.16:0/3559245837 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f498006fb20 0x7f498009c2f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f497000e810 tx=0x7f497000eb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:10.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.082+0000 7f4967fff700 1 -- 172.21.15.16:0/3559245837 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497000c070 con 0x7f498006fb20 2023-09-30T22:33:10.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.082+0000 7f4967fff700 1 -- 172.21.15.16:0/3559245837 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4970013610 con 0x7f498006fb20 2023-09-30T22:33:10.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.082+0000 7f49858dc700 1 -- 172.21.15.16:0/3559245837 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49801263d0 con 0x7f498006fb20 2023-09-30T22:33:10.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.082+0000 7f4967fff700 1 -- 172.21.15.16:0/3559245837 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49700139c0 con 0x7f498006fb20 2023-09-30T22:33:10.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.082+0000 7f49858dc700 1 -- 172.21.15.16:0/3559245837 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f498009cd70 con 0x7f498006fb20 2023-09-30T22:33:10.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.083+0000 7f4967fff700 1 -- 172.21.15.16:0/3559245837 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4970013b20 con 0x7f498006fb20 2023-09-30T22:33:10.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.084+0000 7f4967fff700 1 -- 172.21.15.16:0/3559245837 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f497003f430 con 0x7f498006fb20 2023-09-30T22:33:10.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.084+0000 7f49858dc700 1 -- 172.21.15.16:0/3559245837 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4980074ea0 con 0x7f498006fb20 2023-09-30T22:33:10.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.089+0000 7f4967fff700 1 -- 172.21.15.16:0/3559245837 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4970010540 con 0x7f498006fb20 2023-09-30T22:33:10.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.109+0000 7f3954329700 1 Processor -- start 2023-09-30T22:33:10.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.109+0000 7f3954329700 1 -- start start 2023-09-30T22:33:10.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.109+0000 7f3954329700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f394c09d6f0 0x7f394c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:10.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.109+0000 7f3954329700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f394c09e000 0x7f394c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:10.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.110+0000 7f3954329700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394c09b120 con 0x7f394c09e410 2023-09-30T22:33:10.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.110+0000 7f3954329700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f394c06b460 con 0x7f394c09e000 2023-09-30T22:33:10.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.110+0000 7f3954329700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f394c09a7e0 con 0x7f394c09d6f0 2023-09-30T22:33:10.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.110+0000 7f39518c4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f394c09e000 0x7f394c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:10.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.110+0000 7f39520c5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f394c09d6f0 0x7f394c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:10.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.110+0000 7f39520c5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f394c09d6f0 0x7f394c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41612/0 (socket says 172.21.15.16:41612) 2023-09-30T22:33:10.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.110+0000 7f39520c5700 1 -- 172.21.15.16:0/3301364404 learned_addr learned my addr 172.21.15.16:0/3301364404 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:10.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.110+0000 7f39518c4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f394c09e000 0x7f394c1339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36646/0 (socket says 172.21.15.16:36646) 2023-09-30T22:33:10.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.110+0000 7f39518c4700 1 -- 172.21.15.16:0/3301364404 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f394c09d6f0 msgr2=0x7f394c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:10.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.110+0000 7f39510c3700 1 -- 172.21.15.16:0/3301364404 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 941353491 0 0) 0x7f394c09b120 con 0x7f394c09e410 2023-09-30T22:33:10.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.110+0000 7f39518c4700 1 --2- 172.21.15.16:0/3301364404 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f394c09d6f0 0x7f394c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:10.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.110+0000 7f39518c4700 1 -- 172.21.15.16:0/3301364404 >> v1:172.21.15.16:6789/0 conn(0x7f394c09e410 legacy=0x7f394c133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:10.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.110+0000 7f39518c4700 1 -- 172.21.15.16:0/3301364404 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f394c09a500 con 0x7f394c09e000 2023-09-30T22:33:10.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.110+0000 7f39520c5700 1 --2- 172.21.15.16:0/3301364404 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f394c09d6f0 0x7f394c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:10.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.110+0000 7f39518c4700 1 --2- 172.21.15.16:0/3301364404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f394c09e000 0x7f394c1339f0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f393c009840 tx=0x7f393c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5293cded011b28cf server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:10.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.111+0000 7f39510c3700 1 -- 172.21.15.16:0/3301364404 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393c011830 con 0x7f394c09e000 2023-09-30T22:33:10.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.111+0000 7f39510c3700 1 -- 172.21.15.16:0/3301364404 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f393c011990 con 0x7f394c09e000 2023-09-30T22:33:10.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.111+0000 7f39510c3700 1 -- 172.21.15.16:0/3301364404 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393c011d20 con 0x7f394c09e000 2023-09-30T22:33:10.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.111+0000 7f3954329700 1 -- 172.21.15.16:0/3301364404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f394c09e000 msgr2=0x7f394c1339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:10.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.111+0000 7f3954329700 1 --2- 172.21.15.16:0/3301364404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f394c09e000 0x7f394c1339f0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f393c009840 tx=0x7f393c009b50 comp rx=0 tx=0).stop 2023-09-30T22:33:10.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.112+0000 7f3954329700 1 -- 172.21.15.16:0/3301364404 shutdown_connections 2023-09-30T22:33:10.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.112+0000 7f3954329700 1 --2- 172.21.15.16:0/3301364404 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f394c09d6f0 0x7f394c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:10.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.112+0000 7f3954329700 1 --2- 172.21.15.16:0/3301364404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f394c09e000 0x7f394c1339f0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:10.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.112+0000 7f3954329700 1 -- 172.21.15.16:0/3301364404 >> 172.21.15.16:0/3301364404 conn(0x7f394c066f50 msgr2=0x7f394c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:10.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.112+0000 7f3954329700 1 -- 172.21.15.16:0/3301364404 shutdown_connections 2023-09-30T22:33:10.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.112+0000 7f3954329700 1 -- 172.21.15.16:0/3301364404 wait complete. 2023-09-30T22:33:10.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.113+0000 7f3954329700 1 Processor -- start 2023-09-30T22:33:10.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.113+0000 7f3954329700 1 -- start start 2023-09-30T22:33:10.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.113+0000 7f3954329700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f394c09d6f0 0x7f394c1c83b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:10.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.113+0000 7f3954329700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f394c09e000 0x7f394c1c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:10.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.113+0000 7f3954329700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f394c09e410 0x7f394c1a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:10.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.113+0000 7f3954329700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394c1b8170 con 0x7f394c09d6f0 2023-09-30T22:33:10.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.113+0000 7f3954329700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f394c1bcfe0 con 0x7f394c09e000 2023-09-30T22:33:10.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.113+0000 7f3954329700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f394c1a6b30 con 0x7f394c09e410 2023-09-30T22:33:10.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.114+0000 7f39528c6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f394c09e410 0x7f394c1a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:10.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.114+0000 7f39520c5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f394c09d6f0 0x7f394c1c83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:10.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.114+0000 7f39528c6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f394c09e410 0x7f394c1a3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37066/0 (socket says 172.21.15.16:37066) 2023-09-30T22:33:10.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.114+0000 7f39528c6700 1 -- 172.21.15.16:0/3985676611 learned_addr learned my addr 172.21.15.16:0/3985676611 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:10.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.114+0000 7f39518c4700 1 --2- 172.21.15.16:0/3985676611 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f394c09e000 0x7f394c1c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:10.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.114+0000 7f39518c4700 1 -- 172.21.15.16:0/3985676611 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f394c09e410 msgr2=0x7f394c1a3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:10.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.114+0000 7f39518c4700 1 --2- 172.21.15.16:0/3985676611 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f394c09e410 0x7f394c1a3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:10.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.114+0000 7f39518c4700 1 -- 172.21.15.16:0/3985676611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f394c09d6f0 msgr2=0x7f394c1c83b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:10.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.114+0000 7f39518c4700 1 --2- 172.21.15.16:0/3985676611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f394c09d6f0 0x7f394c1c83b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:10.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.114+0000 7f39518c4700 1 -- 172.21.15.16:0/3985676611 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3948009550 con 0x7f394c09e000 2023-09-30T22:33:10.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.114+0000 7f39528c6700 1 --2- 172.21.15.16:0/3985676611 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f394c09e410 0x7f394c1a3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:10.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.114+0000 7f39520c5700 1 --2- 172.21.15.16:0/3985676611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f394c09d6f0 0x7f394c1c83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:10.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.114+0000 7f39518c4700 1 --2- 172.21.15.16:0/3985676611 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f394c09e000 0x7f394c1c88f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f393c016000 tx=0x7f393c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:10.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.114+0000 7f3942ffd700 1 -- 172.21.15.16:0/3985676611 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393c0044d0 con 0x7f394c09e000 2023-09-30T22:33:10.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.115+0000 7f3942ffd700 1 -- 172.21.15.16:0/3985676611 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f393c01f070 con 0x7f394c09e000 2023-09-30T22:33:10.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.115+0000 7f3942ffd700 1 -- 172.21.15.16:0/3985676611 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393c0038a0 con 0x7f394c09e000 2023-09-30T22:33:10.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.115+0000 7f3954329700 1 -- 172.21.15.16:0/3985676611 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f393c0094a0 con 0x7f394c09e000 2023-09-30T22:33:10.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.115+0000 7f3954329700 1 -- 172.21.15.16:0/3985676611 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f394c1a51c0 con 0x7f394c09e000 2023-09-30T22:33:10.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.116+0000 7f3942ffd700 1 -- 172.21.15.16:0/3985676611 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f393c003a00 con 0x7f394c09e000 2023-09-30T22:33:10.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.116+0000 7f3954329700 1 -- 172.21.15.16:0/3985676611 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f394c1a5a20 con 0x7f394c09e000 2023-09-30T22:33:10.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.116+0000 7f3942ffd700 1 -- 172.21.15.16:0/3985676611 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f393c021070 con 0x7f394c09e000 2023-09-30T22:33:10.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.123+0000 7f3942ffd700 1 -- 172.21.15.16:0/3985676611 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f393c047080 con 0x7f394c09e000 2023-09-30T22:33:10.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.130+0000 7fcd7c39b700 1 Processor -- start 2023-09-30T22:33:10.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.130+0000 7fcd7c39b700 1 -- start start 2023-09-30T22:33:10.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.131+0000 7fcd7c39b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcd7409d6f0 0x7fcd7409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:10.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.131+0000 7fcd7c39b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd7409e000 0x7fcd74130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:10.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.131+0000 7fcd7c39b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd7409e410 0x7fcd74130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:10.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.131+0000 7fcd7c39b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd7409f520 con 0x7fcd7409d6f0 2023-09-30T22:33:10.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.131+0000 7fcd7c39b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7409f130 con 0x7fcd7409e000 2023-09-30T22:33:10.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.131+0000 7fcd7c39b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7409e930 con 0x7fcd7409e410 2023-09-30T22:33:10.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.131+0000 7fcd79936700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd7409e000 0x7fcd74130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:10.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.131+0000 7fcd79936700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd7409e000 0x7fcd74130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41624/0 (socket says 172.21.15.16:41624) 2023-09-30T22:33:10.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.131+0000 7fcd79936700 1 -- 172.21.15.16:0/2859127004 learned_addr learned my addr 172.21.15.16:0/2859127004 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:10.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.131+0000 7fcd7a137700 1 --2- 172.21.15.16:0/2859127004 >> v2:172.21.15.16:3300/0 conn(0x7fcd7409d6f0 0x7fcd7409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:10.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.131+0000 7fcd7a938700 1 --2- 172.21.15.16:0/2859127004 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd7409e410 0x7fcd74130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:10.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.131+0000 7fcd7a938700 1 -- 172.21.15.16:0/2859127004 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd7409e000 msgr2=0x7fcd74130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:10.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.131+0000 7fcd7a938700 1 --2- 172.21.15.16:0/2859127004 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd7409e000 0x7fcd74130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:10.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.131+0000 7fcd7a938700 1 -- 172.21.15.16:0/2859127004 >> v2:172.21.15.16:3300/0 conn(0x7fcd7409d6f0 msgr2=0x7fcd7409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:10.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.131+0000 7fcd7a938700 1 --2- 172.21.15.16:0/2859127004 >> v2:172.21.15.16:3300/0 conn(0x7fcd7409d6f0 0x7fcd7409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:10.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.132+0000 7fcd7a938700 1 -- 172.21.15.16:0/2859127004 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd7409b2d0 con 0x7fcd7409e410 2023-09-30T22:33:10.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.132+0000 7fcd7a938700 1 --2- 172.21.15.16:0/2859127004 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd7409e410 0x7fcd74130ba0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c009dc0 tx=0x7fcd6c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3eb851e3f3301b8a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:10.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.132+0000 7fcd79135700 1 -- 172.21.15.16:0/2859127004 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c00e070 con 0x7fcd7409e410 2023-09-30T22:33:10.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.132+0000 7fcd79135700 1 -- 172.21.15.16:0/2859127004 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd6c00bae0 con 0x7fcd7409e410 2023-09-30T22:33:10.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.132+0000 7fcd79135700 1 -- 172.21.15.16:0/2859127004 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c00be90 con 0x7fcd7409e410 2023-09-30T22:33:10.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.133+0000 7fcd7c39b700 1 -- 172.21.15.16:0/2859127004 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd7409e410 msgr2=0x7fcd74130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:10.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.133+0000 7fcd7c39b700 1 --2- 172.21.15.16:0/2859127004 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd7409e410 0x7fcd74130ba0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c009dc0 tx=0x7fcd6c00b330 comp rx=0 tx=0).stop 2023-09-30T22:33:10.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.133+0000 7fcd7c39b700 1 -- 172.21.15.16:0/2859127004 shutdown_connections 2023-09-30T22:33:10.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.133+0000 7fcd7c39b700 1 --2- 172.21.15.16:0/2859127004 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd7409e410 0x7fcd74130ba0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:10.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.133+0000 7fcd7c39b700 1 --2- 172.21.15.16:0/2859127004 >> v2:172.21.15.16:3300/0 conn(0x7fcd7409d6f0 0x7fcd7409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:10.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.133+0000 7fcd7c39b700 1 --2- 172.21.15.16:0/2859127004 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd7409e000 0x7fcd74130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:10.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.133+0000 7fcd7c39b700 1 -- 172.21.15.16:0/2859127004 >> 172.21.15.16:0/2859127004 conn(0x7fcd74066f50 msgr2=0x7fcd7409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:10.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.133+0000 7fcd7c39b700 1 -- 172.21.15.16:0/2859127004 shutdown_connections 2023-09-30T22:33:10.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.133+0000 7fcd7c39b700 1 -- 172.21.15.16:0/2859127004 wait complete. 2023-09-30T22:33:10.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.134+0000 7fcd7c39b700 1 Processor -- start 2023-09-30T22:33:10.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.134+0000 7fcd7c39b700 1 -- start start 2023-09-30T22:33:10.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.134+0000 7fcd7c39b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd7409d6f0 0x7fcd74137920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:10.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.134+0000 7fcd7c39b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd7409e000 0x7fcd74137e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:10.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.134+0000 7fcd7c39b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd7409e410 0x7fcd741383a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:10.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.134+0000 7fcd7c39b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd741375b0 con 0x7fcd7409e410 2023-09-30T22:33:10.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.134+0000 7fcd7c39b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd74138c20 con 0x7fcd7409d6f0 2023-09-30T22:33:10.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.134+0000 7fcd7c39b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcd740724d0 con 0x7fcd7409e000 2023-09-30T22:33:10.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.135+0000 7fcd7a137700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd7409d6f0 0x7fcd74137920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:10.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.135+0000 7fcd7a938700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd7409e410 0x7fcd741383a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:10.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.135+0000 7fcd7a137700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd7409d6f0 0x7fcd74137920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41628/0 (socket says 172.21.15.16:41628) 2023-09-30T22:33:10.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.135+0000 7fcd7a137700 1 -- 172.21.15.16:0/3789670418 learned_addr learned my addr 172.21.15.16:0/3789670418 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:10.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.135+0000 7fcd79936700 1 --2- 172.21.15.16:0/3789670418 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd7409e000 0x7fcd74137e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:10.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.135+0000 7fcd7a938700 1 -- 172.21.15.16:0/3789670418 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd7409e000 msgr2=0x7fcd74137e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:10.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.135+0000 7fcd7a938700 1 --2- 172.21.15.16:0/3789670418 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd7409e000 0x7fcd74137e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:10.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.135+0000 7fcd7a938700 1 -- 172.21.15.16:0/3789670418 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd7409d6f0 msgr2=0x7fcd74137920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:10.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.135+0000 7fcd7a938700 1 --2- 172.21.15.16:0/3789670418 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd7409d6f0 0x7fcd74137920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:10.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.135+0000 7fcd7a938700 1 -- 172.21.15.16:0/3789670418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd6c009a70 con 0x7fcd7409e410 2023-09-30T22:33:10.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.135+0000 7fcd79936700 1 --2- 172.21.15.16:0/3789670418 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd7409e000 0x7fcd74137e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:10.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.135+0000 7fcd7a137700 1 --2- 172.21.15.16:0/3789670418 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd7409d6f0 0x7fcd74137920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:10.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.136+0000 7fcd7a938700 1 --2- 172.21.15.16:0/3789670418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd7409e410 0x7fcd741383a0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c016000 tx=0x7fcd6c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:10.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.136+0000 7fcd6affd700 1 -- 172.21.15.16:0/3789670418 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c0044d0 con 0x7fcd7409e410 2023-09-30T22:33:10.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.136+0000 7fcd6affd700 1 -- 172.21.15.16:0/3789670418 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd6c020070 con 0x7fcd7409e410 2023-09-30T22:33:10.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.136+0000 7fcd6affd700 1 -- 172.21.15.16:0/3789670418 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c0048b0 con 0x7fcd7409e410 2023-09-30T22:33:10.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.136+0000 7fcd7c39b700 1 -- 172.21.15.16:0/3789670418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd7411e060 con 0x7fcd7409e410 2023-09-30T22:33:10.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.136+0000 7fcd7c39b700 1 -- 172.21.15.16:0/3789670418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd7411e460 con 0x7fcd7409e410 2023-09-30T22:33:10.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.137+0000 7fcd6affd700 1 -- 172.21.15.16:0/3789670418 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcd6c027480 con 0x7fcd7409e410 2023-09-30T22:33:10.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.137+0000 7fcd6affd700 1 -- 172.21.15.16:0/3789670418 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fcd6c022070 con 0x7fcd7409e410 2023-09-30T22:33:10.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.138+0000 7fcd7c39b700 1 -- 172.21.15.16:0/3789670418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd74074ea0 con 0x7fcd7409e410 2023-09-30T22:33:10.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.147+0000 7fcd6affd700 1 -- 172.21.15.16:0/3789670418 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcd6c02c020 con 0x7fcd7409e410 2023-09-30T22:33:10.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.255+0000 7fdc9b3d8700 1 -- 172.21.15.16:0/3685809299 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fdc9413ba70 con 0x7fdc94134610 2023-09-30T22:33:10.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.256+0000 7fdc89ffb700 1 -- 172.21.15.16:0/3685809299 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdc90015020 con 0x7fdc94134610 2023-09-30T22:33:10.256 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:33:10.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.258+0000 7fdc9b3d8700 1 -- 172.21.15.16:0/3685809299 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdc94134610 msgr2=0x7fdc9413a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:10.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.258+0000 7fdc9b3d8700 1 --2- 172.21.15.16:0/3685809299 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdc94134610 0x7fdc9413a1d0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fdc90009fd0 tx=0x7fdc90004750 comp rx=0 tx=0).stop 2023-09-30T22:33:10.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.258+0000 7fdc9b3d8700 1 -- 172.21.15.16:0/3685809299 shutdown_connections 2023-09-30T22:33:10.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.258+0000 7fdc9b3d8700 1 --2- 172.21.15.16:0/3685809299 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdc94134610 0x7fdc9413a1d0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:10.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.258+0000 7fdc9b3d8700 1 --2- 172.21.15.16:0/3685809299 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdc94139320 0x7fdc94133080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:10.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.258+0000 7fdc9b3d8700 1 --2- 172.21.15.16:0/3685809299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdc941389f0 0x7fdc94138de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:10.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.258+0000 7fdc9b3d8700 1 -- 172.21.15.16:0/3685809299 >> 172.21.15.16:0/3685809299 conn(0x7fdc940a2aa0 msgr2=0x7fdc94125480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:10.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.258+0000 7fdc9b3d8700 1 -- 172.21.15.16:0/3685809299 shutdown_connections 2023-09-30T22:33:10.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.258+0000 7fdc9b3d8700 1 -- 172.21.15.16:0/3685809299 wait complete. 2023-09-30T22:33:10.273 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:33:10.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.337+0000 7f3954329700 1 -- 172.21.15.16:0/3985676611 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f394c074ea0 con 0x7f394c09e000 2023-09-30T22:33:10.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.337+0000 7f3942ffd700 1 -- 172.21.15.16:0/3985676611 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f393c02b030 con 0x7f394c09e000 2023-09-30T22:33:10.338 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:33:10.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.339+0000 7f3954329700 1 -- 172.21.15.16:0/3985676611 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f394c09e000 msgr2=0x7f394c1c88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:10.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.339+0000 7f3954329700 1 --2- 172.21.15.16:0/3985676611 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f394c09e000 0x7f394c1c88f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f393c016000 tx=0x7f393c004750 comp rx=0 tx=0).stop 2023-09-30T22:33:10.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.339+0000 7f3954329700 1 -- 172.21.15.16:0/3985676611 shutdown_connections 2023-09-30T22:33:10.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.339+0000 7f3954329700 1 --2- 172.21.15.16:0/3985676611 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f394c09e410 0x7f394c1a3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:10.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.339+0000 7f3954329700 1 --2- 172.21.15.16:0/3985676611 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f394c09d6f0 0x7f394c1c83b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:10.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.339+0000 7f3954329700 1 --2- 172.21.15.16:0/3985676611 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f394c09e000 0x7f394c1c88f0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:10.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.339+0000 7f3954329700 1 -- 172.21.15.16:0/3985676611 >> 172.21.15.16:0/3985676611 conn(0x7f394c066f50 msgr2=0x7f394c1bbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:10.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.340+0000 7f3954329700 1 -- 172.21.15.16:0/3985676611 shutdown_connections 2023-09-30T22:33:10.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.340+0000 7f3954329700 1 -- 172.21.15.16:0/3985676611 wait complete. 2023-09-30T22:33:10.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.341+0000 7f49858dc700 1 -- 172.21.15.16:0/3559245837 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f498006b9e0 con 0x7f498006fb20 2023-09-30T22:33:10.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.341+0000 7f4967fff700 1 -- 172.21.15.16:0/3559245837 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f497002b090 con 0x7f498006fb20 2023-09-30T22:33:10.342 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:33:10.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.344+0000 7f49858dc700 1 -- 172.21.15.16:0/3559245837 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f498006fb20 msgr2=0x7f498009c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:10.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.344+0000 7f49858dc700 1 --2- 172.21.15.16:0/3559245837 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f498006fb20 0x7f498009c2f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f497000e810 tx=0x7f497000eb20 comp rx=0 tx=0).stop 2023-09-30T22:33:10.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.344+0000 7f49858dc700 1 -- 172.21.15.16:0/3559245837 shutdown_connections 2023-09-30T22:33:10.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.344+0000 7f49858dc700 1 --2- 172.21.15.16:0/3559245837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4980130e50 0x7f49801cc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:10.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.344+0000 7f49858dc700 1 --2- 172.21.15.16:0/3559245837 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f498006fb20 0x7f498009c2f0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:10.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.344+0000 7f49858dc700 1 --2- 172.21.15.16:0/3559245837 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4980130540 0x7f498009c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:10.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.344+0000 7f49858dc700 1 -- 172.21.15.16:0/3559245837 >> 172.21.15.16:0/3559245837 conn(0x7f49800a2aa0 msgr2=0x7f498006e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:10.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.344+0000 7f49858dc700 1 -- 172.21.15.16:0/3559245837 shutdown_connections 2023-09-30T22:33:10.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.344+0000 7f49858dc700 1 -- 172.21.15.16:0/3559245837 wait complete. 2023-09-30T22:33:10.355 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:33:10.360 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:33:10.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.376+0000 7fcd7c39b700 1 -- 172.21.15.16:0/3789670418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fcd7406c360 con 0x7fcd7409e410 2023-09-30T22:33:10.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.376+0000 7fcd6affd700 1 -- 172.21.15.16:0/3789670418 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcd6c036090 con 0x7fcd7409e410 2023-09-30T22:33:10.377 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:33:10.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.379+0000 7fcd7c39b700 1 -- 172.21.15.16:0/3789670418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd7409e410 msgr2=0x7fcd741383a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:10.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.379+0000 7fcd7c39b700 1 --2- 172.21.15.16:0/3789670418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd7409e410 0x7fcd741383a0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c016000 tx=0x7fcd6c004ad0 comp rx=0 tx=0).stop 2023-09-30T22:33:10.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.379+0000 7fcd7c39b700 1 -- 172.21.15.16:0/3789670418 shutdown_connections 2023-09-30T22:33:10.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.379+0000 7fcd7c39b700 1 --2- 172.21.15.16:0/3789670418 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd7409d6f0 0x7fcd74137920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:10.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.379+0000 7fcd7c39b700 1 --2- 172.21.15.16:0/3789670418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd7409e410 0x7fcd741383a0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:10.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.379+0000 7fcd7c39b700 1 --2- 172.21.15.16:0/3789670418 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd7409e000 0x7fcd74137e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:10.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.379+0000 7fcd7c39b700 1 -- 172.21.15.16:0/3789670418 >> 172.21.15.16:0/3789670418 conn(0x7fcd74066f50 msgr2=0x7fcd7406a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:10.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.379+0000 7fcd7c39b700 1 -- 172.21.15.16:0/3789670418 shutdown_connections 2023-09-30T22:33:10.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:10.379+0000 7fcd7c39b700 1 -- 172.21.15.16:0/3789670418 wait complete. 2023-09-30T22:33:10.393 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:33:11.273 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:33:11.356 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:33:11.361 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:33:11.393 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:33:11.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.423+0000 7f6796b47700 1 Processor -- start 2023-09-30T22:33:11.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.423+0000 7f6796b47700 1 -- start start 2023-09-30T22:33:11.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.423+0000 7f6796b47700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6790131810 0x7f6790131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:11.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.424+0000 7f6796b47700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f679009a580 0x7f6790139660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:11.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.424+0000 7f6796b47700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67901313c0 con 0x7f6790099a40 2023-09-30T22:33:11.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.424+0000 7f6796b47700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f679006e8c0 con 0x7f6790131810 2023-09-30T22:33:11.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.424+0000 7f6796b47700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6790071d20 con 0x7f679009a580 2023-09-30T22:33:11.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.424+0000 7f67948e3700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6790131810 0x7f6790131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:11.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.424+0000 7f678ffff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6790099a40 0x7f6790099e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36412/0 (socket says 172.21.15.16:36412) 2023-09-30T22:33:11.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.424+0000 7f67948e3700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6790131810 0x7f6790131be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36668/0 (socket says 172.21.15.16:36668) 2023-09-30T22:33:11.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.424+0000 7f678ffff700 1 -- 172.21.15.16:0/1960989208 learned_addr learned my addr 172.21.15.16:0/1960989208 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:11.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.424+0000 7f67950e4700 1 --2- 172.21.15.16:0/1960989208 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f679009a580 0x7f6790139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:11.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.424+0000 7f67948e3700 1 -- 172.21.15.16:0/1960989208 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f679009a580 msgr2=0x7f6790139660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:11.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.424+0000 7f67948e3700 1 --2- 172.21.15.16:0/1960989208 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f679009a580 0x7f6790139660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:11.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.424+0000 7f67948e3700 1 -- 172.21.15.16:0/1960989208 >> v1:172.21.15.16:6789/0 conn(0x7f6790099a40 legacy=0x7f6790099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:11.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.424+0000 7f67948e3700 1 -- 172.21.15.16:0/1960989208 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67900721a0 con 0x7f6790131810 2023-09-30T22:33:11.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.424+0000 7f67950e4700 1 --2- 172.21.15.16:0/1960989208 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f679009a580 0x7f6790139660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:33:11.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.424+0000 7f67948e3700 1 --2- 172.21.15.16:0/1960989208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6790131810 0x7f6790131be0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f6784009840 tx=0x7f6784009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35631553228e5d0d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:11.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.425+0000 7f678f7fe700 1 -- 172.21.15.16:0/1960989208 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6784011830 con 0x7f6790131810 2023-09-30T22:33:11.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.425+0000 7f678f7fe700 1 -- 172.21.15.16:0/1960989208 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6784011990 con 0x7f6790131810 2023-09-30T22:33:11.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.425+0000 7f678f7fe700 1 -- 172.21.15.16:0/1960989208 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6784011d20 con 0x7f6790131810 2023-09-30T22:33:11.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.425+0000 7f6796b47700 1 -- 172.21.15.16:0/1960989208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6790131810 msgr2=0x7f6790131be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:11.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.425+0000 7f6796b47700 1 --2- 172.21.15.16:0/1960989208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6790131810 0x7f6790131be0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f6784009840 tx=0x7f6784009b50 comp rx=0 tx=0).stop 2023-09-30T22:33:11.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.426+0000 7f6796b47700 1 -- 172.21.15.16:0/1960989208 shutdown_connections 2023-09-30T22:33:11.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.426+0000 7f6796b47700 1 --2- 172.21.15.16:0/1960989208 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f679009a580 0x7f6790139660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:11.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.426+0000 7f6796b47700 1 --2- 172.21.15.16:0/1960989208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6790131810 0x7f6790131be0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:11.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.426+0000 7f6796b47700 1 -- 172.21.15.16:0/1960989208 >> 172.21.15.16:0/1960989208 conn(0x7f67900a2aa0 msgr2=0x7f67900a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:11.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.427+0000 7f6796b47700 1 -- 172.21.15.16:0/1960989208 shutdown_connections 2023-09-30T22:33:11.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.427+0000 7f6796b47700 1 -- 172.21.15.16:0/1960989208 wait complete. 2023-09-30T22:33:11.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.428+0000 7f6796b47700 1 Processor -- start 2023-09-30T22:33:11.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.428+0000 7f6796b47700 1 -- start start 2023-09-30T22:33:11.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.429+0000 7f6796b47700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6790099a40 0x7f679011b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:11.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.429+0000 7f6796b47700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f679009a580 0x7f679011bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:11.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.429+0000 7f6796b47700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6790131810 0x7f679011c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:11.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.429+0000 7f6796b47700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f679011d240 con 0x7f679009a580 2023-09-30T22:33:11.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.429+0000 7f6796b47700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f679011edd0 con 0x7f6790131810 2023-09-30T22:33:11.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.429+0000 7f6796b47700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f67900a1d50 con 0x7f6790099a40 2023-09-30T22:33:11.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.430+0000 7f67948e3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6790099a40 0x7f679011b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:11.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.430+0000 7f67948e3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6790099a40 0x7f679011b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37098/0 (socket says 172.21.15.16:37098) 2023-09-30T22:33:11.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.430+0000 7f67948e3700 1 -- 172.21.15.16:0/3813436638 learned_addr learned my addr 172.21.15.16:0/3813436638 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:11.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.430+0000 7f67950e4700 1 --2- 172.21.15.16:0/3813436638 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6790131810 0x7f679011c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:11.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.430+0000 7f678ffff700 1 --2- 172.21.15.16:0/3813436638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f679009a580 0x7f679011bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:11.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.430+0000 7f67950e4700 1 -- 172.21.15.16:0/3813436638 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6790099a40 msgr2=0x7f679011b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:11.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.430+0000 7f67950e4700 1 --2- 172.21.15.16:0/3813436638 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6790099a40 0x7f679011b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:11.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.430+0000 7f67950e4700 1 -- 172.21.15.16:0/3813436638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f679009a580 msgr2=0x7f679011bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:11.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.430+0000 7f67950e4700 1 --2- 172.21.15.16:0/3813436638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f679009a580 0x7f679011bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:11.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.430+0000 7f67950e4700 1 -- 172.21.15.16:0/3813436638 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67840094a0 con 0x7f6790131810 2023-09-30T22:33:11.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.431+0000 7f67950e4700 1 --2- 172.21.15.16:0/3813436638 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6790131810 0x7f679011c010 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f6788007fd0 tx=0x7f678800ce10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:11.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.431+0000 7f678d7fa700 1 -- 172.21.15.16:0/3813436638 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678800ac80 con 0x7f6790131810 2023-09-30T22:33:11.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.431+0000 7f678d7fa700 1 -- 172.21.15.16:0/3813436638 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f678800ae00 con 0x7f6790131810 2023-09-30T22:33:11.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.431+0000 7f678d7fa700 1 -- 172.21.15.16:0/3813436638 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6788003c60 con 0x7f6790131810 2023-09-30T22:33:11.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.431+0000 7f6796b47700 1 -- 172.21.15.16:0/3813436638 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f679006eb00 con 0x7f6790131810 2023-09-30T22:33:11.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.431+0000 7f6796b47700 1 -- 172.21.15.16:0/3813436638 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f679011f0b0 con 0x7f6790131810 2023-09-30T22:33:11.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.432+0000 7f6796b47700 1 -- 172.21.15.16:0/3813436638 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6790074ea0 con 0x7f6790131810 2023-09-30T22:33:11.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.433+0000 7f678d7fa700 1 -- 172.21.15.16:0/3813436638 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6788015440 con 0x7f6790131810 2023-09-30T22:33:11.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.433+0000 7f678d7fa700 1 -- 172.21.15.16:0/3813436638 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f678803fa40 con 0x7f6790131810 2023-09-30T22:33:11.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.438+0000 7f678d7fa700 1 -- 172.21.15.16:0/3813436638 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6788014990 con 0x7f6790131810 2023-09-30T22:33:11.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.504+0000 7f74f213f700 1 Processor -- start 2023-09-30T22:33:11.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.504+0000 7f74f213f700 1 -- start start 2023-09-30T22:33:11.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.504+0000 7f74f213f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f74ec06e570 0x7f74ec06e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:11.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.504+0000 7f74f213f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f74ec06ee80 0x7f74ec134d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:11.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.504+0000 7f74f213f700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74ec070ee0 con 0x7f74ec09eca0 2023-09-30T22:33:11.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.504+0000 7f74f213f700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74ec071480 con 0x7f74ec06ee80 2023-09-30T22:33:11.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.504+0000 7f74f213f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74ec071b90 con 0x7f74ec06e570 2023-09-30T22:33:11.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.505+0000 7f74eaffd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f74ec09eca0 0x7f74ec134610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36426/0 (socket says 172.21.15.16:36426) 2023-09-30T22:33:11.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.505+0000 7f74eb7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f74ec06e570 0x7f74ec06e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:11.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.505+0000 7f74eaffd700 1 -- 172.21.15.16:0/2202871033 learned_addr learned my addr 172.21.15.16:0/2202871033 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:11.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.505+0000 7f74ebfff700 1 --2- 172.21.15.16:0/2202871033 >> v2:172.21.15.16:3300/0 conn(0x7f74ec06ee80 0x7f74ec134d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:11.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.505+0000 7f74eb7fe700 1 -- 172.21.15.16:0/2202871033 >> v2:172.21.15.16:3300/0 conn(0x7f74ec06ee80 msgr2=0x7f74ec134d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:11.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.505+0000 7f74eb7fe700 1 --2- 172.21.15.16:0/2202871033 >> v2:172.21.15.16:3300/0 conn(0x7f74ec06ee80 0x7f74ec134d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:11.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.505+0000 7f74eb7fe700 1 -- 172.21.15.16:0/2202871033 >> v1:172.21.15.16:6789/0 conn(0x7f74ec09eca0 legacy=0x7f74ec134610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:11.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.505+0000 7f74eb7fe700 1 -- 172.21.15.16:0/2202871033 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74ec0711a0 con 0x7f74ec06e570 2023-09-30T22:33:11.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.505+0000 7f74ea7fc700 1 -- 172.21.15.16:0/2202871033 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3546793854 0 0) 0x7f74ec070ee0 con 0x7f74ec09eca0 2023-09-30T22:33:11.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.505+0000 7f74eb7fe700 1 --2- 172.21.15.16:0/2202871033 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f74ec06e570 0x7f74ec06e940 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f74dc009840 tx=0x7f74dc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69f9d1fa78ab96dc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:11.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.506+0000 7f74ea7fc700 1 -- 172.21.15.16:0/2202871033 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74dc011830 con 0x7f74ec06e570 2023-09-30T22:33:11.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.506+0000 7f74ea7fc700 1 -- 172.21.15.16:0/2202871033 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74dc011990 con 0x7f74ec06e570 2023-09-30T22:33:11.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.506+0000 7f74ea7fc700 1 -- 172.21.15.16:0/2202871033 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74dc011d20 con 0x7f74ec06e570 2023-09-30T22:33:11.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.507+0000 7f74f213f700 1 -- 172.21.15.16:0/2202871033 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f74ec06e570 msgr2=0x7f74ec06e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:11.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.507+0000 7f74f213f700 1 --2- 172.21.15.16:0/2202871033 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f74ec06e570 0x7f74ec06e940 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f74dc009840 tx=0x7f74dc009b50 comp rx=0 tx=0).stop 2023-09-30T22:33:11.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.507+0000 7f74f213f700 1 -- 172.21.15.16:0/2202871033 shutdown_connections 2023-09-30T22:33:11.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.507+0000 7f74f213f700 1 --2- 172.21.15.16:0/2202871033 >> v2:172.21.15.16:3300/0 conn(0x7f74ec06ee80 0x7f74ec134d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:11.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.507+0000 7f74f213f700 1 --2- 172.21.15.16:0/2202871033 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f74ec06e570 0x7f74ec06e940 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:11.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.507+0000 7f74f213f700 1 -- 172.21.15.16:0/2202871033 >> 172.21.15.16:0/2202871033 conn(0x7f74ec0a2aa0 msgr2=0x7f74ec0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:11.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.507+0000 7f74f213f700 1 -- 172.21.15.16:0/2202871033 shutdown_connections 2023-09-30T22:33:11.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.507+0000 7f74f213f700 1 -- 172.21.15.16:0/2202871033 wait complete. 2023-09-30T22:33:11.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.508+0000 7f74f213f700 1 Processor -- start 2023-09-30T22:33:11.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.508+0000 7f74f213f700 1 -- start start 2023-09-30T22:33:11.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.509+0000 7f74f213f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f74ec06e570 0x7f74ec1266e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:11.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.509+0000 7f74f213f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74ec06ee80 0x7f74ec124e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:11.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.509+0000 7f74f213f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f74ec13db60 0x7f74ec1253d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:11.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.509+0000 7f74f213f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ec13c3c0 con 0x7f74ec06ee80 2023-09-30T22:33:11.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.509+0000 7f74f213f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74ec13d0c0 con 0x7f74ec06e570 2023-09-30T22:33:11.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.509+0000 7f74f213f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f74ec1355e0 con 0x7f74ec13db60 2023-09-30T22:33:11.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.509+0000 7f74eb7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f74ec06e570 0x7f74ec1266e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:11.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.509+0000 7f74eb7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f74ec06e570 0x7f74ec1266e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41666/0 (socket says 172.21.15.16:41666) 2023-09-30T22:33:11.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.509+0000 7f74eb7fe700 1 -- 172.21.15.16:0/2432798387 learned_addr learned my addr 172.21.15.16:0/2432798387 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:11.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.509+0000 7f74eaffd700 1 --2- 172.21.15.16:0/2432798387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74ec06ee80 0x7f74ec124e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:11.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.509+0000 7f74eb7fe700 1 -- 172.21.15.16:0/2432798387 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f74ec13db60 msgr2=0x7f74ec1253d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:11.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.509+0000 7f74eb7fe700 1 --2- 172.21.15.16:0/2432798387 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f74ec13db60 0x7f74ec1253d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:11.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.509+0000 7f74eb7fe700 1 -- 172.21.15.16:0/2432798387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74ec06ee80 msgr2=0x7f74ec124e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:11.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.509+0000 7f74eb7fe700 1 --2- 172.21.15.16:0/2432798387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74ec06ee80 0x7f74ec124e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:11.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.509+0000 7f74eb7fe700 1 -- 172.21.15.16:0/2432798387 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74dc0094a0 con 0x7f74ec06e570 2023-09-30T22:33:11.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.509+0000 7f74eb7fe700 1 --2- 172.21.15.16:0/2432798387 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f74ec06e570 0x7f74ec1266e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f74dc0090b0 tx=0x7f74dc004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:11.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.510+0000 7f74d3fff700 1 -- 172.21.15.16:0/2432798387 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74dc0036a0 con 0x7f74ec06e570 2023-09-30T22:33:11.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.510+0000 7f74d3fff700 1 -- 172.21.15.16:0/2432798387 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74dc01f070 con 0x7f74ec06e570 2023-09-30T22:33:11.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.510+0000 7f74d3fff700 1 -- 172.21.15.16:0/2432798387 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74dc003a20 con 0x7f74ec06e570 2023-09-30T22:33:11.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.510+0000 7f74f213f700 1 -- 172.21.15.16:0/2432798387 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74ec070610 con 0x7f74ec06e570 2023-09-30T22:33:11.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.511+0000 7f74f213f700 1 -- 172.21.15.16:0/2432798387 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74ec13b750 con 0x7f74ec06e570 2023-09-30T22:33:11.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.512+0000 7f74d1ffb700 1 -- 172.21.15.16:0/2432798387 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74c40052e0 con 0x7f74ec06e570 2023-09-30T22:33:11.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.512+0000 7f74d3fff700 1 -- 172.21.15.16:0/2432798387 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f74dc003b80 con 0x7f74ec06e570 2023-09-30T22:33:11.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.513+0000 7f74d3fff700 1 -- 172.21.15.16:0/2432798387 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f74dc021070 con 0x7f74ec06e570 2023-09-30T22:33:11.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.517+0000 7f74d3fff700 1 -- 172.21.15.16:0/2432798387 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f74dc0193b0 con 0x7f74ec06e570 2023-09-30T22:33:11.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.546+0000 7f1430047700 1 Processor -- start 2023-09-30T22:33:11.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.546+0000 7fd32cbdc700 1 Processor -- start 2023-09-30T22:33:11.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.546+0000 7fd32cbdc700 1 -- start start 2023-09-30T22:33:11.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.547+0000 7f1430047700 1 -- start start 2023-09-30T22:33:11.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.547+0000 7fd32cbdc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd3280a0a30 0x7fd3280a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:11.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.547+0000 7fd32cbdc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd32809d630 0x7fd3280a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:11.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.547+0000 7f1430047700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f142809d6f0 0x7f142809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:11.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.547+0000 7f1430047700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f142809e000 0x7f14281339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:11.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.547+0000 7f1430047700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f142809b120 con 0x7f142809e410 2023-09-30T22:33:11.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.547+0000 7f1430047700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f142806b460 con 0x7f142809d6f0 2023-09-30T22:33:11.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.547+0000 7f1430047700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f142809a7e0 con 0x7f142809e000 2023-09-30T22:33:11.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.547+0000 7fd32cbdc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd328133930 0x7fd328133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:11.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.547+0000 7fd32cbdc700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3280a2a70 con 0x7fd32809d630 2023-09-30T22:33:11.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.547+0000 7fd32cbdc700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3280a01d0 con 0x7fd3280a0a30 2023-09-30T22:33:11.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.547+0000 7fd32cbdc700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd32809e2b0 con 0x7fd328133930 2023-09-30T22:33:11.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.547+0000 7fd32659c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd3280a0a30 0x7fd3280a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:11.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.547+0000 7fd32659c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd3280a0a30 0x7fd3280a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41668/0 (socket says 172.21.15.16:41668) 2023-09-30T22:33:11.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.547+0000 7fd32659c700 1 -- 172.21.15.16:0/1859837661 learned_addr learned my addr 172.21.15.16:0/1859837661 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:11.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.547+0000 7f142d5e2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f142809e000 0x7f14281339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:11.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.547+0000 7f142d5e2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f142809e000 0x7f14281339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41674/0 (socket says 172.21.15.16:41674) 2023-09-30T22:33:11.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.547+0000 7f142d5e2700 1 -- 172.21.15.16:0/616204974 learned_addr learned my addr 172.21.15.16:0/616204974 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:11.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.548+0000 7f142dde3700 1 --2- 172.21.15.16:0/616204974 >> v2:172.21.15.16:3300/0 conn(0x7f142809d6f0 0x7f142809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:11.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.548+0000 7fd325d9b700 1 --2- 172.21.15.16:0/1859837661 >> v2:172.21.15.16:3300/0 conn(0x7fd32809d630 0x7fd3280a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:11.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.548+0000 7fd32659c700 1 -- 172.21.15.16:0/1859837661 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd328133930 msgr2=0x7fd328133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:11.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.548+0000 7fd32659c700 1 --2- 172.21.15.16:0/1859837661 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd328133930 0x7fd328133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:11.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.548+0000 7fd32659c700 1 -- 172.21.15.16:0/1859837661 >> v2:172.21.15.16:3300/0 conn(0x7fd32809d630 msgr2=0x7fd3280a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:11.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.548+0000 7fd32659c700 1 --2- 172.21.15.16:0/1859837661 >> v2:172.21.15.16:3300/0 conn(0x7fd32809d630 0x7fd3280a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:11.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.548+0000 7fd32659c700 1 -- 172.21.15.16:0/1859837661 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd32809dc50 con 0x7fd3280a0a30 2023-09-30T22:33:11.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.548+0000 7fd32659c700 1 --2- 172.21.15.16:0/1859837661 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd3280a0a30 0x7fd3280a0e00 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd3180098f0 tx=0x7fd318009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4e7ff15829a8378 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:11.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.548+0000 7f142d5e2700 1 -- 172.21.15.16:0/616204974 >> v2:172.21.15.16:3300/0 conn(0x7f142809d6f0 msgr2=0x7f142809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:11.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.548+0000 7f142d5e2700 1 --2- 172.21.15.16:0/616204974 >> v2:172.21.15.16:3300/0 conn(0x7f142809d6f0 0x7f142809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:11.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.548+0000 7f142d5e2700 1 -- 172.21.15.16:0/616204974 >> v1:172.21.15.16:6789/0 conn(0x7f142809e410 legacy=0x7f1428133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:11.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.548+0000 7f142d5e2700 1 -- 172.21.15.16:0/616204974 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f142809a500 con 0x7f142809e000 2023-09-30T22:33:11.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.548+0000 7f142d5e2700 1 --2- 172.21.15.16:0/616204974 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f142809e000 0x7f14281339f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f1418009920 tx=0x7f1418009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa858c6141744bcb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:11.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.548+0000 7fd32559a700 1 -- 172.21.15.16:0/1859837661 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd31800b8a0 con 0x7fd3280a0a30 2023-09-30T22:33:11.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.549+0000 7fd32559a700 1 -- 172.21.15.16:0/1859837661 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd31800ba00 con 0x7fd3280a0a30 2023-09-30T22:33:11.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.549+0000 7f142cde1700 1 -- 172.21.15.16:0/616204974 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1418004230 con 0x7f142809e000 2023-09-30T22:33:11.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.549+0000 7fd32559a700 1 -- 172.21.15.16:0/1859837661 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd31800bd90 con 0x7fd3280a0a30 2023-09-30T22:33:11.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.549+0000 7f142cde1700 1 -- 172.21.15.16:0/616204974 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1418004390 con 0x7f142809e000 2023-09-30T22:33:11.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.549+0000 7f142cde1700 1 -- 172.21.15.16:0/616204974 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1418009e70 con 0x7f142809e000 2023-09-30T22:33:11.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.549+0000 7f1430047700 1 -- 172.21.15.16:0/616204974 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f142809e000 msgr2=0x7f14281339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:11.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.549+0000 7f1430047700 1 --2- 172.21.15.16:0/616204974 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f142809e000 0x7f14281339f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f1418009920 tx=0x7f1418009c30 comp rx=0 tx=0).stop 2023-09-30T22:33:11.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.549+0000 7f1430047700 1 -- 172.21.15.16:0/616204974 shutdown_connections 2023-09-30T22:33:11.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.549+0000 7f1430047700 1 --2- 172.21.15.16:0/616204974 >> v2:172.21.15.16:3300/0 conn(0x7f142809d6f0 0x7f142809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:11.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.549+0000 7f1430047700 1 --2- 172.21.15.16:0/616204974 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f142809e000 0x7f14281339f0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:11.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.549+0000 7f1430047700 1 -- 172.21.15.16:0/616204974 >> 172.21.15.16:0/616204974 conn(0x7f1428066f50 msgr2=0x7f142809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:11.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.550+0000 7fd32cbdc700 1 -- 172.21.15.16:0/1859837661 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd3280a0a30 msgr2=0x7fd3280a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:11.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.550+0000 7fd32cbdc700 1 --2- 172.21.15.16:0/1859837661 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd3280a0a30 0x7fd3280a0e00 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fd3180098f0 tx=0x7fd318009c00 comp rx=0 tx=0).stop 2023-09-30T22:33:11.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.550+0000 7f1430047700 1 -- 172.21.15.16:0/616204974 shutdown_connections 2023-09-30T22:33:11.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.550+0000 7f1430047700 1 -- 172.21.15.16:0/616204974 wait complete. 2023-09-30T22:33:11.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.550+0000 7fd32cbdc700 1 -- 172.21.15.16:0/1859837661 shutdown_connections 2023-09-30T22:33:11.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.550+0000 7fd32cbdc700 1 --2- 172.21.15.16:0/1859837661 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd3280a0a30 0x7fd3280a0e00 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:11.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.550+0000 7fd32cbdc700 1 --2- 172.21.15.16:0/1859837661 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd328133930 0x7fd328133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:11.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.550+0000 7fd32cbdc700 1 --2- 172.21.15.16:0/1859837661 >> v2:172.21.15.16:3300/0 conn(0x7fd32809d630 0x7fd3280a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:11.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.550+0000 7fd32cbdc700 1 -- 172.21.15.16:0/1859837661 >> 172.21.15.16:0/1859837661 conn(0x7fd3280a3050 msgr2=0x7fd3280a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:11.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.551+0000 7fd32cbdc700 1 -- 172.21.15.16:0/1859837661 shutdown_connections 2023-09-30T22:33:11.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.551+0000 7fd32cbdc700 1 -- 172.21.15.16:0/1859837661 wait complete. 2023-09-30T22:33:11.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.551+0000 7f1430047700 1 Processor -- start 2023-09-30T22:33:11.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.551+0000 7f1430047700 1 -- start start 2023-09-30T22:33:11.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.552+0000 7f1430047700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f142809d6f0 0x7f14281a5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:11.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.552+0000 7f1430047700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f142809e000 0x7f14281a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:11.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.552+0000 7f1430047700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f142809e410 0x7f14281a67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:11.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.552+0000 7f1430047700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14281aa210 con 0x7f142809e000 2023-09-30T22:33:11.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.552+0000 7f1430047700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14281a9870 con 0x7f142809d6f0 2023-09-30T22:33:11.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.552+0000 7f1430047700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f14281a7a00 con 0x7f142809e410 2023-09-30T22:33:11.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.552+0000 7f142e5e4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f142809e410 0x7f14281a67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:11.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.552+0000 7f142e5e4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f142809e410 0x7f14281a67d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37118/0 (socket says 172.21.15.16:37118) 2023-09-30T22:33:11.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.552+0000 7f142e5e4700 1 -- 172.21.15.16:0/2875432421 learned_addr learned my addr 172.21.15.16:0/2875432421 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:11.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.552+0000 7fd32cbdc700 1 Processor -- start 2023-09-30T22:33:11.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.552+0000 7f142d5e2700 1 --2- 172.21.15.16:0/2875432421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f142809e000 0x7f14281a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:11.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.552+0000 7f142e5e4700 1 -- 172.21.15.16:0/2875432421 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f142809d6f0 msgr2=0x7f14281a5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:11.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.552+0000 7f142e5e4700 1 --2- 172.21.15.16:0/2875432421 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f142809d6f0 0x7f14281a5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:11.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.552+0000 7f142e5e4700 1 -- 172.21.15.16:0/2875432421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f142809e000 msgr2=0x7f14281a6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:11.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.552+0000 7f142e5e4700 1 --2- 172.21.15.16:0/2875432421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f142809e000 0x7f14281a6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:11.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.552+0000 7f142e5e4700 1 -- 172.21.15.16:0/2875432421 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1418009550 con 0x7f142809e410 2023-09-30T22:33:11.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.553+0000 7f142e5e4700 1 --2- 172.21.15.16:0/2875432421 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f142809e410 0x7f14281a67d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f142000c7a0 tx=0x7f142000cab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:11.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.552+0000 7fd32cbdc700 1 -- start start 2023-09-30T22:33:11.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.553+0000 7f141e7fc700 1 -- 172.21.15.16:0/2875432421 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1420003e10 con 0x7f142809e410 2023-09-30T22:33:11.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.553+0000 7f1430047700 1 -- 172.21.15.16:0/2875432421 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14281a8c20 con 0x7f142809e410 2023-09-30T22:33:11.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.553+0000 7f141e7fc700 1 -- 172.21.15.16:0/2875432421 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1420005070 con 0x7f142809e410 2023-09-30T22:33:11.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.553+0000 7f141e7fc700 1 -- 172.21.15.16:0/2875432421 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1420005420 con 0x7f142809e410 2023-09-30T22:33:11.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.553+0000 7f1430047700 1 -- 172.21.15.16:0/2875432421 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14280a3020 con 0x7f142809e410 2023-09-30T22:33:11.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.553+0000 7fd32cbdc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd32809d630 0x7fd32806dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:11.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.553+0000 7fd32cbdc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd3280a0a30 0x7fd32806e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:11.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.553+0000 7fd32cbdc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd328133930 0x7fd3281cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:11.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.553+0000 7fd32cbdc700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd328070fd0 con 0x7fd328133930 2023-09-30T22:33:11.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.553+0000 7fd32cbdc700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd328071a00 con 0x7fd32809d630 2023-09-30T22:33:11.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.553+0000 7fd32cbdc700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd3280a4250 con 0x7fd3280a0a30 2023-09-30T22:33:11.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.554+0000 7fd32659c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd32809d630 0x7fd32806dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:11.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.554+0000 7fd32659c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd32809d630 0x7fd32806dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41688/0 (socket says 172.21.15.16:41688) 2023-09-30T22:33:11.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.554+0000 7fd32659c700 1 -- 172.21.15.16:0/4056388991 learned_addr learned my addr 172.21.15.16:0/4056388991 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:11.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.554+0000 7f141e7fc700 1 -- 172.21.15.16:0/2875432421 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1420005580 con 0x7f142809e410 2023-09-30T22:33:11.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.554+0000 7fd326d9d700 1 --2- 172.21.15.16:0/4056388991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd328133930 0x7fd3281cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:11.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.554+0000 7fd325d9b700 1 --2- 172.21.15.16:0/4056388991 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd3280a0a30 0x7fd32806e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:11.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.554+0000 7fd32659c700 1 -- 172.21.15.16:0/4056388991 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd3280a0a30 msgr2=0x7fd32806e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:11.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.554+0000 7fd32659c700 1 --2- 172.21.15.16:0/4056388991 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd3280a0a30 0x7fd32806e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:11.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.554+0000 7fd32659c700 1 -- 172.21.15.16:0/4056388991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd328133930 msgr2=0x7fd3281cc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:11.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.554+0000 7fd32659c700 1 --2- 172.21.15.16:0/4056388991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd328133930 0x7fd3281cc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:11.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.554+0000 7fd32659c700 1 -- 172.21.15.16:0/4056388991 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd318009550 con 0x7fd32809d630 2023-09-30T22:33:11.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.554+0000 7fd325d9b700 1 --2- 172.21.15.16:0/4056388991 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd3280a0a30 0x7fd32806e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:33:11.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.555+0000 7f141e7fc700 1 -- 172.21.15.16:0/2875432421 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f142003e930 con 0x7f142809e410 2023-09-30T22:33:11.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.555+0000 7f1430047700 1 -- 172.21.15.16:0/2875432421 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14281a8880 con 0x7f142809e410 2023-09-30T22:33:11.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.555+0000 7fd32659c700 1 --2- 172.21.15.16:0/4056388991 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd32809d630 0x7fd32806dd00 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fd318009c40 tx=0x7fd318004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:11.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.555+0000 7fd316ffd700 1 -- 172.21.15.16:0/4056388991 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd318004510 con 0x7fd32809d630 2023-09-30T22:33:11.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.555+0000 7fd316ffd700 1 -- 172.21.15.16:0/4056388991 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3180036a0 con 0x7fd32809d630 2023-09-30T22:33:11.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.555+0000 7fd32cbdc700 1 -- 172.21.15.16:0/4056388991 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd32806f5f0 con 0x7fd32809d630 2023-09-30T22:33:11.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.555+0000 7fd316ffd700 1 -- 172.21.15.16:0/4056388991 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd318003a50 con 0x7fd32809d630 2023-09-30T22:33:11.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.555+0000 7fd32cbdc700 1 -- 172.21.15.16:0/4056388991 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3280a0410 con 0x7fd32809d630 2023-09-30T22:33:11.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.556+0000 7fd316ffd700 1 -- 172.21.15.16:0/4056388991 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd318003bb0 con 0x7fd32809d630 2023-09-30T22:33:11.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.556+0000 7fd316ffd700 1 -- 172.21.15.16:0/4056388991 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd318040920 con 0x7fd32809d630 2023-09-30T22:33:11.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.558+0000 7fd32cbdc700 1 -- 172.21.15.16:0/4056388991 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd328071340 con 0x7fd32809d630 2023-09-30T22:33:11.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.561+0000 7f141e7fc700 1 -- 172.21.15.16:0/2875432421 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1420017080 con 0x7f142809e410 2023-09-30T22:33:11.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.564+0000 7fd316ffd700 1 -- 172.21.15.16:0/4056388991 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd318015080 con 0x7fd32809d630 2023-09-30T22:33:11.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.650+0000 7f6796b47700 1 -- 172.21.15.16:0/3813436638 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f679011eaf0 con 0x7f6790131810 2023-09-30T22:33:11.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.651+0000 7f678d7fa700 1 -- 172.21.15.16:0/3813436638 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f67880143a0 con 0x7f6790131810 2023-09-30T22:33:11.651 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:33:11.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.653+0000 7f6796b47700 1 -- 172.21.15.16:0/3813436638 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6790131810 msgr2=0x7f679011c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:11.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.653+0000 7f6796b47700 1 --2- 172.21.15.16:0/3813436638 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6790131810 0x7f679011c010 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f6788007fd0 tx=0x7f678800ce10 comp rx=0 tx=0).stop 2023-09-30T22:33:11.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.653+0000 7f6796b47700 1 -- 172.21.15.16:0/3813436638 shutdown_connections 2023-09-30T22:33:11.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.653+0000 7f6796b47700 1 --2- 172.21.15.16:0/3813436638 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6790131810 0x7f679011c010 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:11.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.653+0000 7f6796b47700 1 --2- 172.21.15.16:0/3813436638 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6790099a40 0x7f679011b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:11.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.653+0000 7f6796b47700 1 --2- 172.21.15.16:0/3813436638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f679009a580 0x7f679011bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:11.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.653+0000 7f6796b47700 1 -- 172.21.15.16:0/3813436638 >> 172.21.15.16:0/3813436638 conn(0x7f67900a2aa0 msgr2=0x7f67901b8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:11.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.653+0000 7f6796b47700 1 -- 172.21.15.16:0/3813436638 shutdown_connections 2023-09-30T22:33:11.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.654+0000 7f6796b47700 1 -- 172.21.15.16:0/3813436638 wait complete. 2023-09-30T22:33:11.673 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:33:11.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.728+0000 7f74d1ffb700 1 -- 172.21.15.16:0/2432798387 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f74c4005860 con 0x7f74ec06e570 2023-09-30T22:33:11.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.729+0000 7f74d3fff700 1 -- 172.21.15.16:0/2432798387 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f74dc015020 con 0x7f74ec06e570 2023-09-30T22:33:11.729 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:33:11.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.732+0000 7f74d1ffb700 1 -- 172.21.15.16:0/2432798387 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f74ec06e570 msgr2=0x7f74ec1266e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:11.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.732+0000 7f74d1ffb700 1 --2- 172.21.15.16:0/2432798387 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f74ec06e570 0x7f74ec1266e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f74dc0090b0 tx=0x7f74dc004750 comp rx=0 tx=0).stop 2023-09-30T22:33:11.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.732+0000 7f74d1ffb700 1 -- 172.21.15.16:0/2432798387 shutdown_connections 2023-09-30T22:33:11.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.732+0000 7f74d1ffb700 1 --2- 172.21.15.16:0/2432798387 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f74ec06e570 0x7f74ec1266e0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:11.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.732+0000 7f74d1ffb700 1 --2- 172.21.15.16:0/2432798387 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f74ec13db60 0x7f74ec1253d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:11.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.732+0000 7f74d1ffb700 1 --2- 172.21.15.16:0/2432798387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74ec06ee80 0x7f74ec124e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:11.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.732+0000 7f74d1ffb700 1 -- 172.21.15.16:0/2432798387 >> 172.21.15.16:0/2432798387 conn(0x7f74ec0a2aa0 msgr2=0x7f74ec0a1410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:11.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.732+0000 7f74d1ffb700 1 -- 172.21.15.16:0/2432798387 shutdown_connections 2023-09-30T22:33:11.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.732+0000 7f74d1ffb700 1 -- 172.21.15.16:0/2432798387 wait complete. 2023-09-30T22:33:11.747 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:33:11.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.777+0000 7fd32cbdc700 1 -- 172.21.15.16:0/4056388991 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd3280747f0 con 0x7fd32809d630 2023-09-30T22:33:11.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.777+0000 7fd316ffd700 1 -- 172.21.15.16:0/4056388991 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd318003e60 con 0x7fd32809d630 2023-09-30T22:33:11.778 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:33:11.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.779+0000 7fd32cbdc700 1 -- 172.21.15.16:0/4056388991 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd32809d630 msgr2=0x7fd32806dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:11.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.779+0000 7fd32cbdc700 1 --2- 172.21.15.16:0/4056388991 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd32809d630 0x7fd32806dd00 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fd318009c40 tx=0x7fd318004750 comp rx=0 tx=0).stop 2023-09-30T22:33:11.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.779+0000 7fd32cbdc700 1 -- 172.21.15.16:0/4056388991 shutdown_connections 2023-09-30T22:33:11.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.779+0000 7fd32cbdc700 1 --2- 172.21.15.16:0/4056388991 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd32809d630 0x7fd32806dd00 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:11.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.779+0000 7fd32cbdc700 1 --2- 172.21.15.16:0/4056388991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd328133930 0x7fd3281cc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:11.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.779+0000 7fd32cbdc700 1 --2- 172.21.15.16:0/4056388991 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd3280a0a30 0x7fd32806e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:11.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.779+0000 7fd32cbdc700 1 -- 172.21.15.16:0/4056388991 >> 172.21.15.16:0/4056388991 conn(0x7fd3280a3050 msgr2=0x7fd3281bbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:11.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.779+0000 7fd32cbdc700 1 -- 172.21.15.16:0/4056388991 shutdown_connections 2023-09-30T22:33:11.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.779+0000 7fd32cbdc700 1 -- 172.21.15.16:0/4056388991 wait complete. 2023-09-30T22:33:11.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.787+0000 7f1430047700 1 -- 172.21.15.16:0/2875432421 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1428074ea0 con 0x7f142809e410 2023-09-30T22:33:11.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.788+0000 7f141e7fc700 1 -- 172.21.15.16:0/2875432421 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1420005830 con 0x7f142809e410 2023-09-30T22:33:11.788 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:33:11.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.790+0000 7f1430047700 1 -- 172.21.15.16:0/2875432421 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f142809e410 msgr2=0x7f14281a67d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:11.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.790+0000 7f1430047700 1 --2- 172.21.15.16:0/2875432421 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f142809e410 0x7f14281a67d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f142000c7a0 tx=0x7f142000cab0 comp rx=0 tx=0).stop 2023-09-30T22:33:11.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.790+0000 7f1430047700 1 -- 172.21.15.16:0/2875432421 shutdown_connections 2023-09-30T22:33:11.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.790+0000 7f1430047700 1 --2- 172.21.15.16:0/2875432421 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f142809d6f0 0x7f14281a5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:11.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.790+0000 7f1430047700 1 --2- 172.21.15.16:0/2875432421 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f142809e410 0x7f14281a67d0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:11.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.790+0000 7f1430047700 1 --2- 172.21.15.16:0/2875432421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f142809e000 0x7f14281a6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:11.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.790+0000 7f1430047700 1 -- 172.21.15.16:0/2875432421 >> 172.21.15.16:0/2875432421 conn(0x7f1428066f50 msgr2=0x7f14281bac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:11.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.790+0000 7f1430047700 1 -- 172.21.15.16:0/2875432421 shutdown_connections 2023-09-30T22:33:11.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:11.790+0000 7f1430047700 1 -- 172.21.15.16:0/2875432421 wait complete. 2023-09-30T22:33:11.794 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:33:11.804 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:33:12.675 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:33:12.748 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:33:12.794 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:33:12.805 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:33:12.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.829+0000 7f897a8da700 1 Processor -- start 2023-09-30T22:33:12.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.829+0000 7f897a8da700 1 -- start start 2023-09-30T22:33:12.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.829+0000 7f897a8da700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8974133420 0x7f89741337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:12.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.829+0000 7f897a8da700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8974133e30 0x7f897412ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:12.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.829+0000 7f897a8da700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8974134260 0x7f89741304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:12.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.829+0000 7f897a8da700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f897406b990 con 0x7f8974134260 2023-09-30T22:33:12.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.829+0000 7f897a8da700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89740a0590 con 0x7f8974133e30 2023-09-30T22:33:12.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.829+0000 7f897a8da700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f897409fb80 con 0x7f8974133420 2023-09-30T22:33:12.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.830+0000 7f89737fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8974133e30 0x7f897412ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:12.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.830+0000 7f8978e77700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8974134260 0x7f89741304d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:12.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.830+0000 7f8973fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8974133420 0x7f89741337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:12.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.830+0000 7f89737fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8974133e30 0x7f897412ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41694/0 (socket says 172.21.15.16:41694) 2023-09-30T22:33:12.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.830+0000 7f8978e77700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8974134260 0x7f89741304d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36750/0 (socket says 172.21.15.16:36750) 2023-09-30T22:33:12.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.830+0000 7f89737fe700 1 -- 172.21.15.16:0/3353451041 learned_addr learned my addr 172.21.15.16:0/3353451041 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:12.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.830+0000 7f8973fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8974133420 0x7f89741337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37136/0 (socket says 172.21.15.16:37136) 2023-09-30T22:33:12.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.830+0000 7f8973fff700 1 -- 172.21.15.16:0/3353451041 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8974133e30 msgr2=0x7f897412ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:12.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.830+0000 7f8973fff700 1 --2- 172.21.15.16:0/3353451041 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8974133e30 0x7f897412ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:12.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.830+0000 7f8973fff700 1 -- 172.21.15.16:0/3353451041 >> v2:172.21.15.16:3300/0 conn(0x7f8974134260 msgr2=0x7f89741304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:12.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.830+0000 7f8973fff700 1 --2- 172.21.15.16:0/3353451041 >> v2:172.21.15.16:3300/0 conn(0x7f8974134260 0x7f89741304d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:12.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.830+0000 7f8973fff700 1 -- 172.21.15.16:0/3353451041 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89740a0030 con 0x7f8974133420 2023-09-30T22:33:12.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.830+0000 7f8973fff700 1 --2- 172.21.15.16:0/3353451041 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8974133420 0x7f89741337f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8964009840 tx=0x7f8964009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=163771774b174b7c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:12.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.830+0000 7f8972ffd700 1 -- 172.21.15.16:0/3353451041 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8964011830 con 0x7f8974133420 2023-09-30T22:33:12.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.831+0000 7f8972ffd700 1 -- 172.21.15.16:0/3353451041 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8964011990 con 0x7f8974133420 2023-09-30T22:33:12.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.831+0000 7f8972ffd700 1 -- 172.21.15.16:0/3353451041 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8964011d20 con 0x7f8974133420 2023-09-30T22:33:12.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.832+0000 7f897a8da700 1 -- 172.21.15.16:0/3353451041 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8974133420 msgr2=0x7f89741337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:12.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.832+0000 7f897a8da700 1 --2- 172.21.15.16:0/3353451041 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8974133420 0x7f89741337f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8964009840 tx=0x7f8964009b50 comp rx=0 tx=0).stop 2023-09-30T22:33:12.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.832+0000 7f897a8da700 1 -- 172.21.15.16:0/3353451041 shutdown_connections 2023-09-30T22:33:12.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.832+0000 7f897a8da700 1 --2- 172.21.15.16:0/3353451041 >> v2:172.21.15.16:3300/0 conn(0x7f8974134260 0x7f89741304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:12.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.832+0000 7f897a8da700 1 --2- 172.21.15.16:0/3353451041 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8974133420 0x7f89741337f0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:12.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.832+0000 7f897a8da700 1 --2- 172.21.15.16:0/3353451041 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8974133e30 0x7f897412ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:12.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.832+0000 7f897a8da700 1 -- 172.21.15.16:0/3353451041 >> 172.21.15.16:0/3353451041 conn(0x7f89740a4210 msgr2=0x7f89740a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:12.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.832+0000 7f897a8da700 1 -- 172.21.15.16:0/3353451041 shutdown_connections 2023-09-30T22:33:12.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.832+0000 7f897a8da700 1 -- 172.21.15.16:0/3353451041 wait complete. 2023-09-30T22:33:12.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.833+0000 7f897a8da700 1 Processor -- start 2023-09-30T22:33:12.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.833+0000 7f897a8da700 1 -- start start 2023-09-30T22:33:12.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.834+0000 7f897a8da700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8974133420 0x7f897409c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:12.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.834+0000 7f897a8da700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8974133e30 0x7f897409c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:12.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.834+0000 7f897a8da700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8974134260 0x7f897409cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:12.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.834+0000 7f897a8da700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8974120770 con 0x7f8974133e30 2023-09-30T22:33:12.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.834+0000 7f897a8da700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f897411fc10 con 0x7f8974134260 2023-09-30T22:33:12.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.834+0000 7f897a8da700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f897411f4f0 con 0x7f8974133420 2023-09-30T22:33:12.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.834+0000 7f8973fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8974133420 0x7f897409c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:12.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.834+0000 7f8973fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8974133420 0x7f897409c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37144/0 (socket says 172.21.15.16:37144) 2023-09-30T22:33:12.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.834+0000 7f8973fff700 1 -- 172.21.15.16:0/451807865 learned_addr learned my addr 172.21.15.16:0/451807865 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:12.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.834+0000 7f89737fe700 1 --2- 172.21.15.16:0/451807865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8974133e30 0x7f897409c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:12.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.834+0000 7f8973fff700 1 -- 172.21.15.16:0/451807865 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8974134260 msgr2=0x7f897409cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:12.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.834+0000 7f8973fff700 1 --2- 172.21.15.16:0/451807865 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8974134260 0x7f897409cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:12.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.834+0000 7f8973fff700 1 -- 172.21.15.16:0/451807865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8974133e30 msgr2=0x7f897409c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:12.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.834+0000 7f8973fff700 1 --2- 172.21.15.16:0/451807865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8974133e30 0x7f897409c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:12.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.834+0000 7f8973fff700 1 -- 172.21.15.16:0/451807865 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89640094a0 con 0x7f8974133420 2023-09-30T22:33:12.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.834+0000 7f8973fff700 1 --2- 172.21.15.16:0/451807865 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8974133420 0x7f897409c480 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f89640090b0 tx=0x7f8964004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:12.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.835+0000 7f8970ff9700 1 -- 172.21.15.16:0/451807865 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8964004240 con 0x7f8974133420 2023-09-30T22:33:12.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.835+0000 7f8970ff9700 1 -- 172.21.15.16:0/451807865 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f896401f070 con 0x7f8974133420 2023-09-30T22:33:12.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.835+0000 7f8970ff9700 1 -- 172.21.15.16:0/451807865 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89640036a0 con 0x7f8974133420 2023-09-30T22:33:12.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.835+0000 7f897a8da700 1 -- 172.21.15.16:0/451807865 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89741201b0 con 0x7f8974133420 2023-09-30T22:33:12.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.836+0000 7f897a8da700 1 -- 172.21.15.16:0/451807865 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f897412da60 con 0x7f8974133420 2023-09-30T22:33:12.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.836+0000 7f897a8da700 1 -- 172.21.15.16:0/451807865 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f897411fd80 con 0x7f8974133420 2023-09-30T22:33:12.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.837+0000 7f8970ff9700 1 -- 172.21.15.16:0/451807865 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8964021030 con 0x7f8974133420 2023-09-30T22:33:12.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.837+0000 7f8970ff9700 1 -- 172.21.15.16:0/451807865 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f8964040630 con 0x7f8974133420 2023-09-30T22:33:12.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.846+0000 7f8970ff9700 1 -- 172.21.15.16:0/451807865 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f89640239c0 con 0x7f8974133420 2023-09-30T22:33:12.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.909+0000 7feeffda1700 1 Processor -- start 2023-09-30T22:33:12.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.909+0000 7feeffda1700 1 -- start start 2023-09-30T22:33:12.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.910+0000 7feeffda1700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7feef809d6f0 0x7feef809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:12.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.910+0000 7feeffda1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7feef809e000 0x7feef81339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:12.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.910+0000 7feeffda1700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feef809b120 con 0x7feef809e410 2023-09-30T22:33:12.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.910+0000 7feeffda1700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feef806b460 con 0x7feef809d6f0 2023-09-30T22:33:12.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.910+0000 7feeffda1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feef809a7e0 con 0x7feef809e000 2023-09-30T22:33:12.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.910+0000 7feefdb3d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7feef809d6f0 0x7feef809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:12.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.910+0000 7feefe33e700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7feef809e410 0x7feef8133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36446/0 (socket says 172.21.15.16:36446) 2023-09-30T22:33:12.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.910+0000 7feefe33e700 1 -- 172.21.15.16:0/3234980693 learned_addr learned my addr 172.21.15.16:0/3234980693 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:12.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.910+0000 7feefd33c700 1 --2- 172.21.15.16:0/3234980693 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7feef809e000 0x7feef81339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:12.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.910+0000 7feefcb3b700 1 -- 172.21.15.16:0/3234980693 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3190225109 0 0) 0x7feef809b120 con 0x7feef809e410 2023-09-30T22:33:12.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.910+0000 7feefcb3b700 1 -- 172.21.15.16:0/3234980693 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feee40035b0 con 0x7feef809e410 2023-09-30T22:33:12.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.910+0000 7feefdb3d700 1 -- 172.21.15.16:0/3234980693 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7feef809e000 msgr2=0x7feef81339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:12.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.910+0000 7feefdb3d700 1 --2- 172.21.15.16:0/3234980693 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7feef809e000 0x7feef81339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:12.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.910+0000 7feefdb3d700 1 -- 172.21.15.16:0/3234980693 >> v1:172.21.15.16:6789/0 conn(0x7feef809e410 legacy=0x7feef8133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:12.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.910+0000 7feefdb3d700 1 -- 172.21.15.16:0/3234980693 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feef809a500 con 0x7feef809d6f0 2023-09-30T22:33:12.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.911+0000 7feefdb3d700 1 --2- 172.21.15.16:0/3234980693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feef809d6f0 0x7feef809dac0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7feef4009840 tx=0x7feef4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da6a1473eb24b60f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:12.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.911+0000 7feefcb3b700 1 -- 172.21.15.16:0/3234980693 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef4011830 con 0x7feef809d6f0 2023-09-30T22:33:12.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.911+0000 7feefcb3b700 1 -- 172.21.15.16:0/3234980693 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feef4011990 con 0x7feef809d6f0 2023-09-30T22:33:12.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.911+0000 7feefcb3b700 1 -- 172.21.15.16:0/3234980693 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef4011d20 con 0x7feef809d6f0 2023-09-30T22:33:12.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.912+0000 7feeffda1700 1 -- 172.21.15.16:0/3234980693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feef809d6f0 msgr2=0x7feef809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:12.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.912+0000 7feeffda1700 1 --2- 172.21.15.16:0/3234980693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feef809d6f0 0x7feef809dac0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7feef4009840 tx=0x7feef4009b50 comp rx=0 tx=0).stop 2023-09-30T22:33:12.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.912+0000 7feeffda1700 1 -- 172.21.15.16:0/3234980693 shutdown_connections 2023-09-30T22:33:12.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.912+0000 7feeffda1700 1 --2- 172.21.15.16:0/3234980693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feef809d6f0 0x7feef809dac0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:12.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.912+0000 7feeffda1700 1 --2- 172.21.15.16:0/3234980693 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7feef809e000 0x7feef81339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:12.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.912+0000 7feeffda1700 1 -- 172.21.15.16:0/3234980693 >> 172.21.15.16:0/3234980693 conn(0x7feef8066f50 msgr2=0x7feef809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:12.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.912+0000 7feeffda1700 1 -- 172.21.15.16:0/3234980693 shutdown_connections 2023-09-30T22:33:12.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.912+0000 7feeffda1700 1 -- 172.21.15.16:0/3234980693 wait complete. 2023-09-30T22:33:12.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.913+0000 7feeffda1700 1 Processor -- start 2023-09-30T22:33:12.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.913+0000 7feeffda1700 1 -- start start 2023-09-30T22:33:12.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.914+0000 7feeffda1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feef809d6f0 0x7feef81a25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:12.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.914+0000 7feeffda1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7feef809e000 0x7feef81a2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:12.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.914+0000 7feeffda1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7feef809e410 0x7feef81a3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:12.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.914+0000 7feeffda1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef81b5ec0 con 0x7feef809d6f0 2023-09-30T22:33:12.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.914+0000 7feeffda1700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feef81a38f0 con 0x7feef809e000 2023-09-30T22:33:12.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.914+0000 7feeffda1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feef8071fd0 con 0x7feef809e410 2023-09-30T22:33:12.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.914+0000 7feefd33c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7feef809e000 0x7feef81a2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:12.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.914+0000 7feefd33c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7feef809e000 0x7feef81a2ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41722/0 (socket says 172.21.15.16:41722) 2023-09-30T22:33:12.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.914+0000 7feefd33c700 1 -- 172.21.15.16:0/586099077 learned_addr learned my addr 172.21.15.16:0/586099077 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:12.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.914+0000 7feefe33e700 1 --2- 172.21.15.16:0/586099077 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7feef809e410 0x7feef81a3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:12.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.914+0000 7feefdb3d700 1 --2- 172.21.15.16:0/586099077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feef809d6f0 0x7feef81a25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:12.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.914+0000 7feefd33c700 1 -- 172.21.15.16:0/586099077 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7feef809e410 msgr2=0x7feef81a3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:12.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.914+0000 7feefd33c700 1 --2- 172.21.15.16:0/586099077 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7feef809e410 0x7feef81a3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:12.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.914+0000 7feefd33c700 1 -- 172.21.15.16:0/586099077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feef809d6f0 msgr2=0x7feef81a25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:12.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.914+0000 7feefd33c700 1 --2- 172.21.15.16:0/586099077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feef809d6f0 0x7feef81a25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:12.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.914+0000 7feefd33c700 1 -- 172.21.15.16:0/586099077 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feef40094a0 con 0x7feef809e000 2023-09-30T22:33:12.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.915+0000 7feefd33c700 1 --2- 172.21.15.16:0/586099077 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7feef809e000 0x7feef81a2ae0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7feee800e9b0 tx=0x7feee800ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:12.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.915+0000 7feeee7fc700 1 -- 172.21.15.16:0/586099077 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee8009710 con 0x7feef809e000 2023-09-30T22:33:12.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.915+0000 7feeee7fc700 1 -- 172.21.15.16:0/586099077 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feee800c850 con 0x7feef809e000 2023-09-30T22:33:12.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.915+0000 7feeee7fc700 1 -- 172.21.15.16:0/586099077 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee800cbc0 con 0x7feef809e000 2023-09-30T22:33:12.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.915+0000 7feeffda1700 1 -- 172.21.15.16:0/586099077 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feef81b6380 con 0x7feef809e000 2023-09-30T22:33:12.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.915+0000 7feeffda1700 1 -- 172.21.15.16:0/586099077 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feef809c710 con 0x7feef809e000 2023-09-30T22:33:12.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.916+0000 7feeffda1700 1 -- 172.21.15.16:0/586099077 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feef81a4ab0 con 0x7feef809e000 2023-09-30T22:33:12.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.917+0000 7feeee7fc700 1 -- 172.21.15.16:0/586099077 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7feee800cd20 con 0x7feef809e000 2023-09-30T22:33:12.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.917+0000 7feeee7fc700 1 -- 172.21.15.16:0/586099077 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7feee803ff00 con 0x7feef809e000 2023-09-30T22:33:12.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.926+0000 7feeee7fc700 1 -- 172.21.15.16:0/586099077 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7feee8013470 con 0x7feef809e000 2023-09-30T22:33:12.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.954+0000 7f28d688d700 1 Processor -- start 2023-09-30T22:33:12.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.954+0000 7f28d688d700 1 -- start start 2023-09-30T22:33:12.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.955+0000 7f28d688d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f28d0136820 0x7f28d0138c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:12.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.955+0000 7f28d688d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f28d0139140 0x7f28d013b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:12.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.955+0000 7f28d688d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28d006bc00 con 0x7f28d0133a80 2023-09-30T22:33:12.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.955+0000 7f28d688d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28d006c570 con 0x7f28d0136820 2023-09-30T22:33:12.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.955+0000 7f28d688d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28d00a0040 con 0x7f28d0139140 2023-09-30T22:33:12.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.955+0000 7f28cf7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f28d0136820 0x7f28d0138c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:12.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.955+0000 7f28cf7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f28d0136820 0x7f28d0138c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36782/0 (socket says 172.21.15.16:36782) 2023-09-30T22:33:12.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.955+0000 7f28d4e2a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f28d0139140 0x7f28d013b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:12.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.955+0000 7f28cf7fe700 1 -- 172.21.15.16:0/3116507276 learned_addr learned my addr 172.21.15.16:0/3116507276 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:12.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.955+0000 7f28cffff700 1 --1- 172.21.15.16:0/3116507276 >> v1:172.21.15.16:6789/0 conn(0x7f28d0133a80 0x7f28d0133e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36452/0 (socket says 172.21.15.16:36452) 2023-09-30T22:33:12.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.956+0000 7f28cf7fe700 1 -- 172.21.15.16:0/3116507276 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f28d0139140 msgr2=0x7f28d013b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:12.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.956+0000 7f28cf7fe700 1 --2- 172.21.15.16:0/3116507276 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f28d0139140 0x7f28d013b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:12.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.956+0000 7f28cf7fe700 1 -- 172.21.15.16:0/3116507276 >> v1:172.21.15.16:6789/0 conn(0x7f28d0133a80 legacy=0x7f28d0133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:12.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.956+0000 7f28cf7fe700 1 -- 172.21.15.16:0/3116507276 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28d0133860 con 0x7f28d0136820 2023-09-30T22:33:12.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.956+0000 7f28d4e2a700 1 --2- 172.21.15.16:0/3116507276 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f28d0139140 0x7f28d013b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:12.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.956+0000 7f28cf7fe700 1 --2- 172.21.15.16:0/3116507276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28d0136820 0x7f28d0138c00 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f28c40098f0 tx=0x7f28c4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e3485d99bafacd7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:12.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.956+0000 7f28ceffd700 1 -- 172.21.15.16:0/3116507276 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c4011830 con 0x7f28d0136820 2023-09-30T22:33:12.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.956+0000 7f28ceffd700 1 -- 172.21.15.16:0/3116507276 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28c4011990 con 0x7f28d0136820 2023-09-30T22:33:12.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.956+0000 7f28ceffd700 1 -- 172.21.15.16:0/3116507276 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c4011d20 con 0x7f28d0136820 2023-09-30T22:33:12.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.957+0000 7f28d688d700 1 -- 172.21.15.16:0/3116507276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28d0136820 msgr2=0x7f28d0138c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:12.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.957+0000 7f28d688d700 1 --2- 172.21.15.16:0/3116507276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28d0136820 0x7f28d0138c00 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f28c40098f0 tx=0x7f28c4009c00 comp rx=0 tx=0).stop 2023-09-30T22:33:12.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.957+0000 7f28d688d700 1 -- 172.21.15.16:0/3116507276 shutdown_connections 2023-09-30T22:33:12.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.957+0000 7f28d688d700 1 --2- 172.21.15.16:0/3116507276 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f28d0139140 0x7f28d013b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:12.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.957+0000 7f28d688d700 1 --2- 172.21.15.16:0/3116507276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28d0136820 0x7f28d0138c00 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:12.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.957+0000 7f28d688d700 1 -- 172.21.15.16:0/3116507276 >> 172.21.15.16:0/3116507276 conn(0x7f28d00a4210 msgr2=0x7f28d00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:12.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.957+0000 7f28d688d700 1 -- 172.21.15.16:0/3116507276 shutdown_connections 2023-09-30T22:33:12.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.957+0000 7f28d688d700 1 -- 172.21.15.16:0/3116507276 wait complete. 2023-09-30T22:33:12.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.958+0000 7f28d688d700 1 Processor -- start 2023-09-30T22:33:12.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.959+0000 7f28d688d700 1 -- start start 2023-09-30T22:33:12.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.959+0000 7f28d688d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f28d0133a80 0x7f28d01c83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:12.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.959+0000 7f28d688d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28d0136820 0x7f28d01c88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:12.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.959+0000 7f28d688d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f28d0139140 0x7f28d01c8e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:12.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.959+0000 7f28d688d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d0072470 con 0x7f28d0136820 2023-09-30T22:33:12.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.959+0000 7f28d688d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28d012db80 con 0x7f28d0133a80 2023-09-30T22:33:12.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.959+0000 7f28d688d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f28d00711e0 con 0x7f28d0139140 2023-09-30T22:33:12.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.959+0000 7f28d4e2a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f28d0139140 0x7f28d01c8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:12.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.960+0000 7f28d4e2a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f28d0139140 0x7f28d01c8e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37172/0 (socket says 172.21.15.16:37172) 2023-09-30T22:33:12.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.960+0000 7f28d4e2a700 1 -- 172.21.15.16:0/2300095683 learned_addr learned my addr 172.21.15.16:0/2300095683 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:12.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.960+0000 7f28cffff700 1 --2- 172.21.15.16:0/2300095683 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f28d0133a80 0x7f28d01c83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:12.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.960+0000 7f28cf7fe700 1 --2- 172.21.15.16:0/2300095683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28d0136820 0x7f28d01c88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:12.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.960+0000 7f28cffff700 1 -- 172.21.15.16:0/2300095683 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f28d0139140 msgr2=0x7f28d01c8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:12.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.960+0000 7f28cffff700 1 --2- 172.21.15.16:0/2300095683 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f28d0139140 0x7f28d01c8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:12.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.960+0000 7f28cffff700 1 -- 172.21.15.16:0/2300095683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28d0136820 msgr2=0x7f28d01c88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:12.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.960+0000 7f28cffff700 1 --2- 172.21.15.16:0/2300095683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28d0136820 0x7f28d01c88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:12.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.960+0000 7f28cffff700 1 -- 172.21.15.16:0/2300095683 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c8009a70 con 0x7f28d0133a80 2023-09-30T22:33:12.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.960+0000 7f28d4e2a700 1 --2- 172.21.15.16:0/2300095683 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f28d0139140 0x7f28d01c8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:12.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.960+0000 7f28cffff700 1 --2- 172.21.15.16:0/2300095683 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f28d0133a80 0x7f28d01c83a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f28c000c450 tx=0x7f28c000c760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:12.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.960+0000 7f28ccff9700 1 -- 172.21.15.16:0/2300095683 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c0005cb0 con 0x7f28d0133a80 2023-09-30T22:33:12.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.960+0000 7f28ccff9700 1 -- 172.21.15.16:0/2300095683 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28c0005e10 con 0x7f28d0133a80 2023-09-30T22:33:12.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.960+0000 7f28ccff9700 1 -- 172.21.15.16:0/2300095683 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c00072d0 con 0x7f28d0133a80 2023-09-30T22:33:12.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.960+0000 7f28d688d700 1 -- 172.21.15.16:0/2300095683 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28c4009550 con 0x7f28d0133a80 2023-09-30T22:33:12.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.961+0000 7f28d688d700 1 -- 172.21.15.16:0/2300095683 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28d00704e0 con 0x7f28d0133a80 2023-09-30T22:33:12.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.962+0000 7f28ccff9700 1 -- 172.21.15.16:0/2300095683 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f28c0007430 con 0x7f28d0133a80 2023-09-30T22:33:12.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.962+0000 7f28ccff9700 1 -- 172.21.15.16:0/2300095683 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f28c003ea30 con 0x7f28d0133a80 2023-09-30T22:33:12.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.962+0000 7f28d688d700 1 -- 172.21.15.16:0/2300095683 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28d0074ea0 con 0x7f28d0133a80 2023-09-30T22:33:12.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.967+0000 7f28ccff9700 1 -- 172.21.15.16:0/2300095683 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f28c0017020 con 0x7f28d0133a80 2023-09-30T22:33:12.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.994+0000 7fa3627ec700 1 Processor -- start 2023-09-30T22:33:12.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.994+0000 7fa3627ec700 1 -- start start 2023-09-30T22:33:12.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.995+0000 7fa3627ec700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa35c0a0a30 0x7fa35c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:12.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.995+0000 7fa3627ec700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa35c134170 0x7fa35c13bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:12.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.995+0000 7fa3627ec700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa35c09edd0 con 0x7fa35c0a1340 2023-09-30T22:33:12.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.995+0000 7fa3627ec700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa35c09e1f0 con 0x7fa35c0a0a30 2023-09-30T22:33:12.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.995+0000 7fa3627ec700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa35c09dc50 con 0x7fa35c134170 2023-09-30T22:33:12.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.995+0000 7fa35b7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa35c0a1340 0x7fa35c133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36466/0 (socket says 172.21.15.16:36466) 2023-09-30T22:33:12.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.995+0000 7fa35b7fe700 1 -- 172.21.15.16:0/4278770621 learned_addr learned my addr 172.21.15.16:0/4278770621 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:12.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.995+0000 7fa360d89700 1 --2- 172.21.15.16:0/4278770621 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa35c134170 0x7fa35c13bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:12.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.995+0000 7fa35bfff700 1 --2- 172.21.15.16:0/4278770621 >> v2:172.21.15.16:3300/0 conn(0x7fa35c0a0a30 0x7fa35c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:12.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.996+0000 7fa360d89700 1 -- 172.21.15.16:0/4278770621 >> v2:172.21.15.16:3300/0 conn(0x7fa35c0a0a30 msgr2=0x7fa35c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:12.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.996+0000 7fa360d89700 1 --2- 172.21.15.16:0/4278770621 >> v2:172.21.15.16:3300/0 conn(0x7fa35c0a0a30 0x7fa35c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:12.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.996+0000 7fa360d89700 1 -- 172.21.15.16:0/4278770621 >> v1:172.21.15.16:6789/0 conn(0x7fa35c0a1340 legacy=0x7fa35c133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:12.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.996+0000 7fa360d89700 1 -- 172.21.15.16:0/4278770621 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa35c0a1fe0 con 0x7fa35c134170 2023-09-30T22:33:12.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.996+0000 7fa360d89700 1 --2- 172.21.15.16:0/4278770621 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa35c134170 0x7fa35c13bd30 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fa354009d90 tx=0x7fa35400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aeae21b49997dc0b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:12.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.996+0000 7fa35affd700 1 -- 172.21.15.16:0/4278770621 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35400e070 con 0x7fa35c134170 2023-09-30T22:33:12.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.996+0000 7fa35affd700 1 -- 172.21.15.16:0/4278770621 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa35400bb20 con 0x7fa35c134170 2023-09-30T22:33:12.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.996+0000 7fa35affd700 1 -- 172.21.15.16:0/4278770621 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa354012430 con 0x7fa35c134170 2023-09-30T22:33:12.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.997+0000 7fa3627ec700 1 -- 172.21.15.16:0/4278770621 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa35c134170 msgr2=0x7fa35c13bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:12.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.997+0000 7fa3627ec700 1 --2- 172.21.15.16:0/4278770621 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa35c134170 0x7fa35c13bd30 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fa354009d90 tx=0x7fa35400b330 comp rx=0 tx=0).stop 2023-09-30T22:33:12.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.997+0000 7fa3627ec700 1 -- 172.21.15.16:0/4278770621 shutdown_connections 2023-09-30T22:33:12.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.997+0000 7fa3627ec700 1 --2- 172.21.15.16:0/4278770621 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa35c134170 0x7fa35c13bd30 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:12.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.997+0000 7fa3627ec700 1 --2- 172.21.15.16:0/4278770621 >> v2:172.21.15.16:3300/0 conn(0x7fa35c0a0a30 0x7fa35c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:13.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.997+0000 7fa3627ec700 1 -- 172.21.15.16:0/4278770621 >> 172.21.15.16:0/4278770621 conn(0x7fa35c0a6550 msgr2=0x7fa35c0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:13.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.997+0000 7fa3627ec700 1 -- 172.21.15.16:0/4278770621 shutdown_connections 2023-09-30T22:33:13.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.997+0000 7fa3627ec700 1 -- 172.21.15.16:0/4278770621 wait complete. 2023-09-30T22:33:13.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.999+0000 7fa3627ec700 1 Processor -- start 2023-09-30T22:33:13.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:12.999+0000 7fa3627ec700 1 -- start start 2023-09-30T22:33:13.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.000+0000 7fa3627ec700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa35c0a0a30 0x7fa35c11e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:13.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.000+0000 7fa3627ec700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa35c0a1340 0x7fa35c11ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:13.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.000+0000 7fa3627ec700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa35c134170 0x7fa35c11f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:13.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.000+0000 7fa3627ec700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa35c11fbe0 con 0x7fa35c0a1340 2023-09-30T22:33:13.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.000+0000 7fa3627ec700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa35c1221a0 con 0x7fa35c0a0a30 2023-09-30T22:33:13.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.000+0000 7fa3627ec700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa35c120290 con 0x7fa35c134170 2023-09-30T22:33:13.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.000+0000 7fa35bfff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa35c0a0a30 0x7fa35c11e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:13.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.000+0000 7fa35bfff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa35c0a0a30 0x7fa35c11e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41744/0 (socket says 172.21.15.16:41744) 2023-09-30T22:33:13.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.000+0000 7fa35bfff700 1 -- 172.21.15.16:0/3413773696 learned_addr learned my addr 172.21.15.16:0/3413773696 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:13.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.000+0000 7fa360d89700 1 --2- 172.21.15.16:0/3413773696 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa35c134170 0x7fa35c11f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:13.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.000+0000 7fa35b7fe700 1 --2- 172.21.15.16:0/3413773696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa35c0a1340 0x7fa35c11ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:13.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.000+0000 7fa35bfff700 1 -- 172.21.15.16:0/3413773696 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa35c134170 msgr2=0x7fa35c11f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:13.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.000+0000 7fa35bfff700 1 --2- 172.21.15.16:0/3413773696 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa35c134170 0x7fa35c11f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:13.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.000+0000 7fa35bfff700 1 -- 172.21.15.16:0/3413773696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa35c0a1340 msgr2=0x7fa35c11ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:13.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.000+0000 7fa35bfff700 1 --2- 172.21.15.16:0/3413773696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa35c0a1340 0x7fa35c11ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:13.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.000+0000 7fa35bfff700 1 -- 172.21.15.16:0/3413773696 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa354009a70 con 0x7fa35c0a0a30 2023-09-30T22:33:13.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.001+0000 7fa35bfff700 1 --2- 172.21.15.16:0/3413773696 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa35c0a0a30 0x7fa35c11e710 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa34c00e9b0 tx=0x7fa34c00ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:13.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.001+0000 7fa358ff9700 1 -- 172.21.15.16:0/3413773696 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34c009710 con 0x7fa35c0a0a30 2023-09-30T22:33:13.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.001+0000 7fa358ff9700 1 -- 172.21.15.16:0/3413773696 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa34c00c850 con 0x7fa35c0a0a30 2023-09-30T22:33:13.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.001+0000 7fa358ff9700 1 -- 172.21.15.16:0/3413773696 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34c00cbc0 con 0x7fa35c0a0a30 2023-09-30T22:33:13.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.001+0000 7fa3627ec700 1 -- 172.21.15.16:0/3413773696 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa35c0a25e0 con 0x7fa35c0a0a30 2023-09-30T22:33:13.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.001+0000 7fa3627ec700 1 -- 172.21.15.16:0/3413773696 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa35c121c00 con 0x7fa35c0a0a30 2023-09-30T22:33:13.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.002+0000 7fa358ff9700 1 -- 172.21.15.16:0/3413773696 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa34c00cd20 con 0x7fa35c0a0a30 2023-09-30T22:33:13.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.002+0000 7fa358ff9700 1 -- 172.21.15.16:0/3413773696 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fa34c03fb10 con 0x7fa35c0a0a30 2023-09-30T22:33:13.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.002+0000 7fa3627ec700 1 -- 172.21.15.16:0/3413773696 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa35c074ea0 con 0x7fa35c0a0a30 2023-09-30T22:33:13.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.007+0000 7fa358ff9700 1 -- 172.21.15.16:0/3413773696 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa34c01a9b0 con 0x7fa35c0a0a30 2023-09-30T22:33:13.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.062+0000 7f897a8da700 1 -- 172.21.15.16:0/451807865 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f897412dbf0 con 0x7f8974133420 2023-09-30T22:33:13.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.062+0000 7f8970ff9700 1 -- 172.21.15.16:0/451807865 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8964015070 con 0x7f8974133420 2023-09-30T22:33:13.063 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:33:13.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.064+0000 7f897a8da700 1 -- 172.21.15.16:0/451807865 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8974133420 msgr2=0x7f897409c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:13.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.064+0000 7f897a8da700 1 --2- 172.21.15.16:0/451807865 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8974133420 0x7f897409c480 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f89640090b0 tx=0x7f8964004750 comp rx=0 tx=0).stop 2023-09-30T22:33:13.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.064+0000 7f897a8da700 1 -- 172.21.15.16:0/451807865 shutdown_connections 2023-09-30T22:33:13.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.064+0000 7f897a8da700 1 --2- 172.21.15.16:0/451807865 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8974134260 0x7f897409cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:13.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.064+0000 7f897a8da700 1 --2- 172.21.15.16:0/451807865 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8974133420 0x7f897409c480 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:13.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.064+0000 7f897a8da700 1 --2- 172.21.15.16:0/451807865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8974133e30 0x7f897409c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:13.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.064+0000 7f897a8da700 1 -- 172.21.15.16:0/451807865 >> 172.21.15.16:0/451807865 conn(0x7f89740a4210 msgr2=0x7f89740a0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:13.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.065+0000 7f897a8da700 1 -- 172.21.15.16:0/451807865 shutdown_connections 2023-09-30T22:33:13.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.065+0000 7f897a8da700 1 -- 172.21.15.16:0/451807865 wait complete. 2023-09-30T22:33:13.079 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:33:13.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.177+0000 7f28d688d700 1 -- 172.21.15.16:0/2300095683 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f28d006b250 con 0x7f28d0133a80 2023-09-30T22:33:13.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.177+0000 7f28ccff9700 1 -- 172.21.15.16:0/2300095683 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f28c001b3b0 con 0x7f28d0133a80 2023-09-30T22:33:13.178 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:33:13.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.179+0000 7f28d688d700 1 -- 172.21.15.16:0/2300095683 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f28d0133a80 msgr2=0x7f28d01c83a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:13.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.179+0000 7f28d688d700 1 --2- 172.21.15.16:0/2300095683 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f28d0133a80 0x7f28d01c83a0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f28c000c450 tx=0x7f28c000c760 comp rx=0 tx=0).stop 2023-09-30T22:33:13.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.179+0000 7f28d688d700 1 -- 172.21.15.16:0/2300095683 shutdown_connections 2023-09-30T22:33:13.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.179+0000 7f28d688d700 1 --2- 172.21.15.16:0/2300095683 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f28d0133a80 0x7f28d01c83a0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:13.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.179+0000 7f28d688d700 1 --2- 172.21.15.16:0/2300095683 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f28d0139140 0x7f28d01c8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:13.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.179+0000 7f28d688d700 1 --2- 172.21.15.16:0/2300095683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28d0136820 0x7f28d01c88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:13.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.179+0000 7f28d688d700 1 -- 172.21.15.16:0/2300095683 >> 172.21.15.16:0/2300095683 conn(0x7f28d00a4210 msgr2=0x7f28d012cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:13.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.180+0000 7f28d688d700 1 -- 172.21.15.16:0/2300095683 shutdown_connections 2023-09-30T22:33:13.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.180+0000 7f28d688d700 1 -- 172.21.15.16:0/2300095683 wait complete. 2023-09-30T22:33:13.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.190+0000 7feeffda1700 1 -- 172.21.15.16:0/586099077 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7feef81a4ab0 con 0x7feef809e000 2023-09-30T22:33:13.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.190+0000 7feeee7fc700 1 -- 172.21.15.16:0/586099077 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7feef81a4ab0 con 0x7feef809e000 2023-09-30T22:33:13.190 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:33:13.194 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:33:13.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.193+0000 7feeffda1700 1 -- 172.21.15.16:0/586099077 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7feef809e000 msgr2=0x7feef81a2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:13.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.193+0000 7feeffda1700 1 --2- 172.21.15.16:0/586099077 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7feef809e000 0x7feef81a2ae0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7feee800e9b0 tx=0x7feee800ecc0 comp rx=0 tx=0).stop 2023-09-30T22:33:13.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.193+0000 7feeffda1700 1 -- 172.21.15.16:0/586099077 shutdown_connections 2023-09-30T22:33:13.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.193+0000 7feeffda1700 1 --2- 172.21.15.16:0/586099077 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7feef809e410 0x7feef81a3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:13.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.193+0000 7feeffda1700 1 --2- 172.21.15.16:0/586099077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feef809d6f0 0x7feef81a25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:13.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.193+0000 7feeffda1700 1 --2- 172.21.15.16:0/586099077 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7feef809e000 0x7feef81a2ae0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:13.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.194+0000 7feeffda1700 1 -- 172.21.15.16:0/586099077 >> 172.21.15.16:0/586099077 conn(0x7feef8066f50 msgr2=0x7feef80a3680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:13.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.194+0000 7feeffda1700 1 -- 172.21.15.16:0/586099077 shutdown_connections 2023-09-30T22:33:13.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.194+0000 7feeffda1700 1 -- 172.21.15.16:0/586099077 wait complete. 2023-09-30T22:33:13.212 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:33:13.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.248+0000 7fa3627ec700 1 -- 172.21.15.16:0/3413773696 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa35c0a28e0 con 0x7fa35c0a0a30 2023-09-30T22:33:13.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.249+0000 7fa358ff9700 1 -- 172.21.15.16:0/3413773696 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa34c017020 con 0x7fa35c0a0a30 2023-09-30T22:33:13.249 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:33:13.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.251+0000 7fa3627ec700 1 -- 172.21.15.16:0/3413773696 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa35c0a0a30 msgr2=0x7fa35c11e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:13.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.251+0000 7fa3627ec700 1 --2- 172.21.15.16:0/3413773696 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa35c0a0a30 0x7fa35c11e710 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa34c00e9b0 tx=0x7fa34c00ecc0 comp rx=0 tx=0).stop 2023-09-30T22:33:13.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.251+0000 7fa3627ec700 1 -- 172.21.15.16:0/3413773696 shutdown_connections 2023-09-30T22:33:13.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.251+0000 7fa3627ec700 1 --2- 172.21.15.16:0/3413773696 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa35c0a0a30 0x7fa35c11e710 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:13.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.251+0000 7fa3627ec700 1 --2- 172.21.15.16:0/3413773696 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa35c134170 0x7fa35c11f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:13.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.251+0000 7fa3627ec700 1 --2- 172.21.15.16:0/3413773696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa35c0a1340 0x7fa35c11ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:13.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.251+0000 7fa3627ec700 1 -- 172.21.15.16:0/3413773696 >> 172.21.15.16:0/3413773696 conn(0x7fa35c0a6550 msgr2=0x7fa35c09ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:13.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.251+0000 7fa3627ec700 1 -- 172.21.15.16:0/3413773696 shutdown_connections 2023-09-30T22:33:13.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:13.251+0000 7fa3627ec700 1 -- 172.21.15.16:0/3413773696 wait complete. 2023-09-30T22:33:13.265 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:33:14.080 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:33:14.196 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:33:14.213 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:33:14.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.227+0000 7fedf3262700 1 Processor -- start 2023-09-30T22:33:14.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.227+0000 7fedf3262700 1 -- start start 2023-09-30T22:33:14.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.228+0000 7fedf3262700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fedec0a0a30 0x7fedec0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:14.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.228+0000 7fedf3262700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fedec09d630 0x7fedec0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:14.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.228+0000 7fedf3262700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedec0a2a70 con 0x7fedec133930 2023-09-30T22:33:14.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.228+0000 7fedf3262700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedec0a01d0 con 0x7fedec09d630 2023-09-30T22:33:14.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.228+0000 7fedf3262700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedec09e2b0 con 0x7fedec0a0a30 2023-09-30T22:33:14.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.228+0000 7fedebfff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fedec09d630 0x7fedec0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:14.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.228+0000 7fedf17ff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fedec133930 0x7fedec133db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36474/0 (socket says 172.21.15.16:36474) 2023-09-30T22:33:14.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.228+0000 7fedf17ff700 1 -- 172.21.15.16:0/2636031832 learned_addr learned my addr 172.21.15.16:0/2636031832 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:14.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.228+0000 7fedf0ffe700 1 --2- 172.21.15.16:0/2636031832 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fedec0a0a30 0x7fedec0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:14.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.228+0000 7fedebfff700 1 -- 172.21.15.16:0/2636031832 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fedec0a0a30 msgr2=0x7fedec0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:14.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.228+0000 7fedebfff700 1 --2- 172.21.15.16:0/2636031832 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fedec0a0a30 0x7fedec0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:14.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.228+0000 7fedebfff700 1 -- 172.21.15.16:0/2636031832 >> v1:172.21.15.16:6789/0 conn(0x7fedec133930 legacy=0x7fedec133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:14.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.228+0000 7fedebfff700 1 -- 172.21.15.16:0/2636031832 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedec09dc50 con 0x7fedec09d630 2023-09-30T22:33:14.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.229+0000 7fedebfff700 1 --2- 172.21.15.16:0/2636031832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedec09d630 0x7fedec0a1340 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fedd4009840 tx=0x7fedd4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddf07c5f372c4640 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:14.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.229+0000 7fedeb7fe700 1 -- 172.21.15.16:0/2636031832 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd4011830 con 0x7fedec09d630 2023-09-30T22:33:14.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.229+0000 7fedeb7fe700 1 -- 172.21.15.16:0/2636031832 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedd4011990 con 0x7fedec09d630 2023-09-30T22:33:14.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.229+0000 7fedeb7fe700 1 -- 172.21.15.16:0/2636031832 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd4011d20 con 0x7fedec09d630 2023-09-30T22:33:14.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.230+0000 7fedf3262700 1 -- 172.21.15.16:0/2636031832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedec09d630 msgr2=0x7fedec0a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:14.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.230+0000 7fedf3262700 1 --2- 172.21.15.16:0/2636031832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedec09d630 0x7fedec0a1340 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fedd4009840 tx=0x7fedd4009b50 comp rx=0 tx=0).stop 2023-09-30T22:33:14.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.230+0000 7fedf3262700 1 -- 172.21.15.16:0/2636031832 shutdown_connections 2023-09-30T22:33:14.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.230+0000 7fedf3262700 1 --2- 172.21.15.16:0/2636031832 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fedec0a0a30 0x7fedec0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:14.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.230+0000 7fedf3262700 1 --2- 172.21.15.16:0/2636031832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedec09d630 0x7fedec0a1340 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:14.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.230+0000 7fedf3262700 1 -- 172.21.15.16:0/2636031832 >> 172.21.15.16:0/2636031832 conn(0x7fedec0a3050 msgr2=0x7fedec0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:14.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.230+0000 7fedf3262700 1 -- 172.21.15.16:0/2636031832 shutdown_connections 2023-09-30T22:33:14.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.230+0000 7fedf3262700 1 -- 172.21.15.16:0/2636031832 wait complete. 2023-09-30T22:33:14.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.231+0000 7fedf3262700 1 Processor -- start 2023-09-30T22:33:14.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.231+0000 7fedf3262700 1 -- start start 2023-09-30T22:33:14.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.232+0000 7fedf3262700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fedec09d630 0x7fedec1c8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:14.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.232+0000 7fedf3262700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fedec0a0a30 0x7fedec1c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:14.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.232+0000 7fedf3262700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedec133930 0x7fedec1c8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:14.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.232+0000 7fedf3262700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedec122240 con 0x7fedec133930 2023-09-30T22:33:14.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.232+0000 7fedf3262700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedec120580 con 0x7fedec09d630 2023-09-30T22:33:14.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.232+0000 7fedf3262700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fedec0a4030 con 0x7fedec0a0a30 2023-09-30T22:33:14.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.232+0000 7fedebfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fedec0a0a30 0x7fedec1c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:14.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.232+0000 7fedebfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fedec0a0a30 0x7fedec1c86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37194/0 (socket says 172.21.15.16:37194) 2023-09-30T22:33:14.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.232+0000 7fedebfff700 1 -- 172.21.15.16:0/1295021709 learned_addr learned my addr 172.21.15.16:0/1295021709 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:14.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.232+0000 7fedf17ff700 1 --2- 172.21.15.16:0/1295021709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedec133930 0x7fedec1c8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:14.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.232+0000 7fedf0ffe700 1 --2- 172.21.15.16:0/1295021709 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fedec09d630 0x7fedec1c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:14.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.232+0000 7fedf0ffe700 1 -- 172.21.15.16:0/1295021709 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fedec0a0a30 msgr2=0x7fedec1c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:14.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.232+0000 7fedf0ffe700 1 --2- 172.21.15.16:0/1295021709 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fedec0a0a30 0x7fedec1c86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:14.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.232+0000 7fedf0ffe700 1 -- 172.21.15.16:0/1295021709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedec133930 msgr2=0x7fedec1c8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:14.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.232+0000 7fedf0ffe700 1 --2- 172.21.15.16:0/1295021709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedec133930 0x7fedec1c8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:14.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.232+0000 7fedf0ffe700 1 -- 172.21.15.16:0/1295021709 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedd40094a0 con 0x7fedec09d630 2023-09-30T22:33:14.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.232+0000 7fedf17ff700 1 --2- 172.21.15.16:0/1295021709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedec133930 0x7fedec1c8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:14.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.232+0000 7fedebfff700 1 --2- 172.21.15.16:0/1295021709 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fedec0a0a30 0x7fedec1c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:14.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.233+0000 7fedf0ffe700 1 --2- 172.21.15.16:0/1295021709 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fedec09d630 0x7fedec1c8170 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7feddc00e900 tx=0x7feddc00ec10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:14.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.233+0000 7fede97fa700 1 -- 172.21.15.16:0/1295021709 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feddc013680 con 0x7fedec09d630 2023-09-30T22:33:14.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.233+0000 7fede97fa700 1 -- 172.21.15.16:0/1295021709 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feddc0137e0 con 0x7fedec09d630 2023-09-30T22:33:14.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.233+0000 7fedf3262700 1 -- 172.21.15.16:0/1295021709 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedec0a0410 con 0x7fedec09d630 2023-09-30T22:33:14.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.233+0000 7fede97fa700 1 -- 172.21.15.16:0/1295021709 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feddc013b50 con 0x7fedec09d630 2023-09-30T22:33:14.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.233+0000 7fedf3262700 1 -- 172.21.15.16:0/1295021709 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedec1211f0 con 0x7fedec09d630 2023-09-30T22:33:14.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.234+0000 7fede97fa700 1 -- 172.21.15.16:0/1295021709 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7feddc013cb0 con 0x7fedec09d630 2023-09-30T22:33:14.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.234+0000 7fede97fa700 1 -- 172.21.15.16:0/1295021709 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7feddc03f910 con 0x7fedec09d630 2023-09-30T22:33:14.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.234+0000 7fedf3262700 1 -- 172.21.15.16:0/1295021709 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedec0747f0 con 0x7fedec09d630 2023-09-30T22:33:14.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.239+0000 7fede97fa700 1 -- 172.21.15.16:0/1295021709 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7feddc017080 con 0x7fedec09d630 2023-09-30T22:33:14.266 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:33:14.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.348+0000 7fc171298700 1 Processor -- start 2023-09-30T22:33:14.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.348+0000 7fc171298700 1 -- start start 2023-09-30T22:33:14.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.349+0000 7fc171298700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc16c09e1d0 0x7fc16c135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:14.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.349+0000 7fc171298700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc16c135e50 0x7fc16c138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:14.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.349+0000 7fc171298700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc16c09f130 con 0x7fc16c09d6f0 2023-09-30T22:33:14.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.349+0000 7fc171298700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc16c09f520 con 0x7fc16c09e1d0 2023-09-30T22:33:14.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.349+0000 7fc171298700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc16c09e930 con 0x7fc16c135e50 2023-09-30T22:33:14.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.350+0000 7fc16a7fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc16c09e1d0 0x7fc16c135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:14.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.350+0000 7fc16b7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc16c135e50 0x7fc16c138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:14.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.350+0000 7fc16a7fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc16c09e1d0 0x7fc16c135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36838/0 (socket says 172.21.15.16:36838) 2023-09-30T22:33:14.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.350+0000 7fc16affd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc16c09d6f0 0x7fc16c09dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36476/0 (socket says 172.21.15.16:36476) 2023-09-30T22:33:14.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.350+0000 7fc16a7fc700 1 -- 172.21.15.16:0/163488873 learned_addr learned my addr 172.21.15.16:0/163488873 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:14.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.350+0000 7fc16b7fe700 1 -- 172.21.15.16:0/163488873 >> v2:172.21.15.16:3300/0 conn(0x7fc16c09e1d0 msgr2=0x7fc16c135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:14.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.350+0000 7fc16b7fe700 1 --2- 172.21.15.16:0/163488873 >> v2:172.21.15.16:3300/0 conn(0x7fc16c09e1d0 0x7fc16c135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:14.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.350+0000 7fc16b7fe700 1 -- 172.21.15.16:0/163488873 >> v1:172.21.15.16:6789/0 conn(0x7fc16c09d6f0 legacy=0x7fc16c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:14.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.350+0000 7fc16b7fe700 1 -- 172.21.15.16:0/163488873 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc16c09b2d0 con 0x7fc16c135e50 2023-09-30T22:33:14.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.350+0000 7fc16b7fe700 1 --2- 172.21.15.16:0/163488873 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc16c135e50 0x7fc16c138230 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc164009d90 tx=0x7fc16400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9c1787a32964ea4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:14.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.351+0000 7fc169ffb700 1 -- 172.21.15.16:0/163488873 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16400e070 con 0x7fc16c135e50 2023-09-30T22:33:14.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.351+0000 7fc169ffb700 1 -- 172.21.15.16:0/163488873 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc16400bb20 con 0x7fc16c135e50 2023-09-30T22:33:14.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.351+0000 7fc169ffb700 1 -- 172.21.15.16:0/163488873 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc164012430 con 0x7fc16c135e50 2023-09-30T22:33:14.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.352+0000 7fc171298700 1 -- 172.21.15.16:0/163488873 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc16c135e50 msgr2=0x7fc16c138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:14.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.352+0000 7fc171298700 1 --2- 172.21.15.16:0/163488873 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc16c135e50 0x7fc16c138230 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fc164009d90 tx=0x7fc16400b330 comp rx=0 tx=0).stop 2023-09-30T22:33:14.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.352+0000 7fc171298700 1 -- 172.21.15.16:0/163488873 shutdown_connections 2023-09-30T22:33:14.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.352+0000 7fc171298700 1 --2- 172.21.15.16:0/163488873 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc16c135e50 0x7fc16c138230 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:14.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.352+0000 7fc171298700 1 --2- 172.21.15.16:0/163488873 >> v2:172.21.15.16:3300/0 conn(0x7fc16c09e1d0 0x7fc16c135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:14.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.352+0000 7fc171298700 1 -- 172.21.15.16:0/163488873 >> 172.21.15.16:0/163488873 conn(0x7fc16c066f50 msgr2=0x7fc16c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:14.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.352+0000 7fc171298700 1 -- 172.21.15.16:0/163488873 shutdown_connections 2023-09-30T22:33:14.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.352+0000 7fc171298700 1 -- 172.21.15.16:0/163488873 wait complete. 2023-09-30T22:33:14.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.353+0000 7fc171298700 1 Processor -- start 2023-09-30T22:33:14.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.353+0000 7fc171298700 1 -- start start 2023-09-30T22:33:14.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.354+0000 7fc171298700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc16c09d6f0 0x7fc16c11d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:14.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.354+0000 7fc171298700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc16c09e1d0 0x7fc16c11ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:14.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.354+0000 7fc171298700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc16c135e50 0x7fc16c11e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:14.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.354+0000 7fc171298700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc16c09c790 con 0x7fc16c09e1d0 2023-09-30T22:33:14.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.354+0000 7fc171298700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc16c11fa80 con 0x7fc16c135e50 2023-09-30T22:33:14.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.354+0000 7fc171298700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc16c0a22d0 con 0x7fc16c09d6f0 2023-09-30T22:33:14.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.354+0000 7fc16a7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc16c09e1d0 0x7fc16c11ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:14.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.354+0000 7fc16a7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc16c09e1d0 0x7fc16c11ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36850/0 (socket says 172.21.15.16:36850) 2023-09-30T22:33:14.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.354+0000 7fc16a7fc700 1 -- 172.21.15.16:0/2248967766 learned_addr learned my addr 172.21.15.16:0/2248967766 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:14.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.354+0000 7fc16affd700 1 --2- 172.21.15.16:0/2248967766 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc16c09d6f0 0x7fc16c11d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:14.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.354+0000 7fc16a7fc700 1 -- 172.21.15.16:0/2248967766 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc16c09d6f0 msgr2=0x7fc16c11d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:14.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.354+0000 7fc16a7fc700 1 --2- 172.21.15.16:0/2248967766 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc16c09d6f0 0x7fc16c11d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:14.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.354+0000 7fc16a7fc700 1 -- 172.21.15.16:0/2248967766 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc16c135e50 msgr2=0x7fc16c11e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:14.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.354+0000 7fc16a7fc700 1 --2- 172.21.15.16:0/2248967766 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc16c135e50 0x7fc16c11e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:14.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.354+0000 7fc16a7fc700 1 -- 172.21.15.16:0/2248967766 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc164009a70 con 0x7fc16c09e1d0 2023-09-30T22:33:14.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.355+0000 7fc16a7fc700 1 --2- 172.21.15.16:0/2248967766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc16c09e1d0 0x7fc16c11ddd0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fc16000e940 tx=0x7fc16000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:14.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.355+0000 7fc1537fe700 1 -- 172.21.15.16:0/2248967766 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1600096a0 con 0x7fc16c09e1d0 2023-09-30T22:33:14.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.355+0000 7fc1537fe700 1 -- 172.21.15.16:0/2248967766 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc16000ee90 con 0x7fc16c09e1d0 2023-09-30T22:33:14.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.355+0000 7fc171298700 1 -- 172.21.15.16:0/2248967766 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc16c121360 con 0x7fc16c09e1d0 2023-09-30T22:33:14.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.355+0000 7fc1537fe700 1 -- 172.21.15.16:0/2248967766 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc16000ca90 con 0x7fc16c09e1d0 2023-09-30T22:33:14.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.355+0000 7fc171298700 1 -- 172.21.15.16:0/2248967766 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc16c11fc40 con 0x7fc16c09e1d0 2023-09-30T22:33:14.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.356+0000 7fc1537fe700 1 -- 172.21.15.16:0/2248967766 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc16000cbf0 con 0x7fc16c09e1d0 2023-09-30T22:33:14.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.356+0000 7fc171298700 1 -- 172.21.15.16:0/2248967766 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc16c120cd0 con 0x7fc16c09e1d0 2023-09-30T22:33:14.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.356+0000 7fc1537fe700 1 -- 172.21.15.16:0/2248967766 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fc16003fa30 con 0x7fc16c09e1d0 2023-09-30T22:33:14.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.361+0000 7fc1537fe700 1 -- 172.21.15.16:0/2248967766 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc16001b3b0 con 0x7fc16c09e1d0 2023-09-30T22:33:14.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.374+0000 7fe9603c0700 1 Processor -- start 2023-09-30T22:33:14.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.374+0000 7fe9603c0700 1 -- start start 2023-09-30T22:33:14.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.374+0000 7fe9603c0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe95806a770 0x7fe95809d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:14.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.374+0000 7fe9603c0700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe95809dc30 0x7fe95809e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:14.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.375+0000 7fe9603c0700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe95809a810 con 0x7fe95813c1e0 2023-09-30T22:33:14.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.375+0000 7fe9603c0700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe95806ab40 con 0x7fe95809dc30 2023-09-30T22:33:14.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.375+0000 7fe9603c0700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe95809ebc0 con 0x7fe95806a770 2023-09-30T22:33:14.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.375+0000 7fe95d95b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe95809dc30 0x7fe95809e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:14.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.375+0000 7fe95d95b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe95809dc30 0x7fe95809e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36860/0 (socket says 172.21.15.16:36860) 2023-09-30T22:33:14.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.375+0000 7fe95d95b700 1 -- 172.21.15.16:0/3266389304 learned_addr learned my addr 172.21.15.16:0/3266389304 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:14.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.375+0000 7fe95e15c700 1 --2- 172.21.15.16:0/3266389304 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe95806a770 0x7fe95809d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:14.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.375+0000 7fe95d95b700 1 -- 172.21.15.16:0/3266389304 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe95806a770 msgr2=0x7fe95809d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:14.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.375+0000 7fe95d95b700 1 --2- 172.21.15.16:0/3266389304 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe95806a770 0x7fe95809d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:14.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.375+0000 7fe95d95b700 1 -- 172.21.15.16:0/3266389304 >> v1:172.21.15.16:6789/0 conn(0x7fe95813c1e0 legacy=0x7fe95813e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:14.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.375+0000 7fe95d95b700 1 -- 172.21.15.16:0/3266389304 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe95809e920 con 0x7fe95809dc30 2023-09-30T22:33:14.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.375+0000 7fe95d95b700 1 --2- 172.21.15.16:0/3266389304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe95809dc30 0x7fe95809e080 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fe948009920 tx=0x7fe948009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4fee0789e8f500 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:14.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.376+0000 7fe95d15a700 1 -- 172.21.15.16:0/3266389304 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe948011830 con 0x7fe95809dc30 2023-09-30T22:33:14.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.376+0000 7fe95d15a700 1 -- 172.21.15.16:0/3266389304 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe948011990 con 0x7fe95809dc30 2023-09-30T22:33:14.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.376+0000 7fe95d15a700 1 -- 172.21.15.16:0/3266389304 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe948011d20 con 0x7fe95809dc30 2023-09-30T22:33:14.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.376+0000 7fe9603c0700 1 -- 172.21.15.16:0/3266389304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe95809dc30 msgr2=0x7fe95809e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:14.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.376+0000 7fe9603c0700 1 --2- 172.21.15.16:0/3266389304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe95809dc30 0x7fe95809e080 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fe948009920 tx=0x7fe948009c30 comp rx=0 tx=0).stop 2023-09-30T22:33:14.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.376+0000 7fe9603c0700 1 -- 172.21.15.16:0/3266389304 shutdown_connections 2023-09-30T22:33:14.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.376+0000 7fe9603c0700 1 --2- 172.21.15.16:0/3266389304 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe95806a770 0x7fe95809d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:14.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.376+0000 7fe9603c0700 1 --2- 172.21.15.16:0/3266389304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe95809dc30 0x7fe95809e080 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:14.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.376+0000 7fe9603c0700 1 -- 172.21.15.16:0/3266389304 >> 172.21.15.16:0/3266389304 conn(0x7fe958066f50 msgr2=0x7fe95809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:14.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.377+0000 7fe9603c0700 1 -- 172.21.15.16:0/3266389304 shutdown_connections 2023-09-30T22:33:14.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.377+0000 7fe9603c0700 1 -- 172.21.15.16:0/3266389304 wait complete. 2023-09-30T22:33:14.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.378+0000 7fe9603c0700 1 Processor -- start 2023-09-30T22:33:14.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.378+0000 7fe9603c0700 1 -- start start 2023-09-30T22:33:14.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.378+0000 7fe9603c0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe95806a770 0x7fe958126c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:14.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.378+0000 7fe9603c0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe95809dc30 0x7fe958127190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:14.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.378+0000 7fe9603c0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe95813c1e0 0x7fe9581cc550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:14.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.378+0000 7fe9603c0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95813ad60 con 0x7fe95806a770 2023-09-30T22:33:14.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.378+0000 7fe9603c0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe95809f1a0 con 0x7fe95813c1e0 2023-09-30T22:33:14.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.378+0000 7fe9603c0700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe958129830 con 0x7fe95809dc30 2023-09-30T22:33:14.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.378+0000 7fe95e15c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe95806a770 0x7fe958126c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:14.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.378+0000 7fe95e15c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe95806a770 0x7fe958126c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36870/0 (socket says 172.21.15.16:36870) 2023-09-30T22:33:14.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.378+0000 7fe95e15c700 1 -- 172.21.15.16:0/850546280 learned_addr learned my addr 172.21.15.16:0/850546280 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:14.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.378+0000 7fe95e95d700 1 --2- 172.21.15.16:0/850546280 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe95813c1e0 0x7fe9581cc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:14.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.379+0000 7fe95d95b700 1 --2- 172.21.15.16:0/850546280 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe95809dc30 0x7fe958127190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:14.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.379+0000 7fe95e15c700 1 -- 172.21.15.16:0/850546280 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe95809dc30 msgr2=0x7fe958127190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:14.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.379+0000 7fe95e15c700 1 --2- 172.21.15.16:0/850546280 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe95809dc30 0x7fe958127190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:14.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.379+0000 7fe95e15c700 1 -- 172.21.15.16:0/850546280 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe95813c1e0 msgr2=0x7fe9581cc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:14.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.379+0000 7fe95e15c700 1 --2- 172.21.15.16:0/850546280 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe95813c1e0 0x7fe9581cc550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:14.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.379+0000 7fe95e15c700 1 -- 172.21.15.16:0/850546280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe948009550 con 0x7fe95806a770 2023-09-30T22:33:14.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.379+0000 7fe95e95d700 1 --2- 172.21.15.16:0/850546280 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe95813c1e0 0x7fe9581cc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:33:14.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.379+0000 7fe95e15c700 1 --2- 172.21.15.16:0/850546280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe95806a770 0x7fe958126c50 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fe95400e9b0 tx=0x7fe95400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:14.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.379+0000 7fe94effd700 1 -- 172.21.15.16:0/850546280 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe954009710 con 0x7fe95806a770 2023-09-30T22:33:14.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.379+0000 7fe94effd700 1 -- 172.21.15.16:0/850546280 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe95400c850 con 0x7fe95806a770 2023-09-30T22:33:14.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.379+0000 7fe94effd700 1 -- 172.21.15.16:0/850546280 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95400cbc0 con 0x7fe95806a770 2023-09-30T22:33:14.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.380+0000 7fe9603c0700 1 -- 172.21.15.16:0/850546280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe958126840 con 0x7fe95806a770 2023-09-30T22:33:14.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.380+0000 7fe9603c0700 1 -- 172.21.15.16:0/850546280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe958127f60 con 0x7fe95806a770 2023-09-30T22:33:14.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.381+0000 7fe94effd700 1 -- 172.21.15.16:0/850546280 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe95400cd20 con 0x7fe95806a770 2023-09-30T22:33:14.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.381+0000 7fe94effd700 1 -- 172.21.15.16:0/850546280 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fe95403f6a0 con 0x7fe95806a770 2023-09-30T22:33:14.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.381+0000 7fe9603c0700 1 -- 172.21.15.16:0/850546280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9581277f0 con 0x7fe95806a770 2023-09-30T22:33:14.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.386+0000 7fe94effd700 1 -- 172.21.15.16:0/850546280 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe9581277f0 con 0x7fe95806a770 2023-09-30T22:33:14.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.410+0000 7fc26c7c9700 1 Processor -- start 2023-09-30T22:33:14.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.410+0000 7fc26c7c9700 1 -- start start 2023-09-30T22:33:14.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.411+0000 7fc26c7c9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc26409d6f0 0x7fc26409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:14.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.411+0000 7fc26c7c9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc26409e000 0x7fc2641339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:14.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.411+0000 7fc26c7c9700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc26409e410 0x7fc264133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:14.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.411+0000 7fc26c7c9700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc26406b460 con 0x7fc26409e410 2023-09-30T22:33:14.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.411+0000 7fc26c7c9700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc26409b120 con 0x7fc26409d6f0 2023-09-30T22:33:14.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.411+0000 7fc26c7c9700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc26409a7e0 con 0x7fc26409e000 2023-09-30T22:33:14.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.411+0000 7fc269d64700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc26409e000 0x7fc2641339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:14.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.411+0000 7fc269d64700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc26409e000 0x7fc2641339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37232/0 (socket says 172.21.15.16:37232) 2023-09-30T22:33:14.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.411+0000 7fc269d64700 1 -- 172.21.15.16:0/2265499081 learned_addr learned my addr 172.21.15.16:0/2265499081 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:14.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.411+0000 7fc26a565700 1 --2- 172.21.15.16:0/2265499081 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc26409d6f0 0x7fc26409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:14.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.411+0000 7fc26ad66700 1 --2- 172.21.15.16:0/2265499081 >> v2:172.21.15.16:3300/0 conn(0x7fc26409e410 0x7fc264133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:14.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.411+0000 7fc269d64700 1 -- 172.21.15.16:0/2265499081 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc26409d6f0 msgr2=0x7fc26409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:14.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.411+0000 7fc269d64700 1 --2- 172.21.15.16:0/2265499081 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc26409d6f0 0x7fc26409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:14.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.411+0000 7fc269d64700 1 -- 172.21.15.16:0/2265499081 >> v2:172.21.15.16:3300/0 conn(0x7fc26409e410 msgr2=0x7fc264133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:14.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.411+0000 7fc269d64700 1 --2- 172.21.15.16:0/2265499081 >> v2:172.21.15.16:3300/0 conn(0x7fc26409e410 0x7fc264133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:14.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.412+0000 7fc269d64700 1 -- 172.21.15.16:0/2265499081 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc26409a500 con 0x7fc26409e000 2023-09-30T22:33:14.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.412+0000 7fc26ad66700 1 --2- 172.21.15.16:0/2265499081 >> v2:172.21.15.16:3300/0 conn(0x7fc26409e410 0x7fc264133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:33:14.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.412+0000 7fc269d64700 1 --2- 172.21.15.16:0/2265499081 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc26409e000 0x7fc2641339f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc254009920 tx=0x7fc254009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e06e7e846ab1d680 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:14.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.412+0000 7fc269563700 1 -- 172.21.15.16:0/2265499081 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc254011830 con 0x7fc26409e000 2023-09-30T22:33:14.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.412+0000 7fc269563700 1 -- 172.21.15.16:0/2265499081 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc254011990 con 0x7fc26409e000 2023-09-30T22:33:14.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.412+0000 7fc269563700 1 -- 172.21.15.16:0/2265499081 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc254011d20 con 0x7fc26409e000 2023-09-30T22:33:14.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.413+0000 7fc26c7c9700 1 -- 172.21.15.16:0/2265499081 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc26409e000 msgr2=0x7fc2641339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:14.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.413+0000 7fc26c7c9700 1 --2- 172.21.15.16:0/2265499081 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc26409e000 0x7fc2641339f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fc254009920 tx=0x7fc254009c30 comp rx=0 tx=0).stop 2023-09-30T22:33:14.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.413+0000 7fc26c7c9700 1 -- 172.21.15.16:0/2265499081 shutdown_connections 2023-09-30T22:33:14.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.413+0000 7fc26c7c9700 1 --2- 172.21.15.16:0/2265499081 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc26409d6f0 0x7fc26409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:14.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.413+0000 7fc26c7c9700 1 --2- 172.21.15.16:0/2265499081 >> v2:172.21.15.16:3300/0 conn(0x7fc26409e410 0x7fc264133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:14.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.413+0000 7fc26c7c9700 1 --2- 172.21.15.16:0/2265499081 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc26409e000 0x7fc2641339f0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:14.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.413+0000 7fc26c7c9700 1 -- 172.21.15.16:0/2265499081 >> 172.21.15.16:0/2265499081 conn(0x7fc264066f50 msgr2=0x7fc26409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:14.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.413+0000 7fc26c7c9700 1 -- 172.21.15.16:0/2265499081 shutdown_connections 2023-09-30T22:33:14.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.413+0000 7fc26c7c9700 1 -- 172.21.15.16:0/2265499081 wait complete. 2023-09-30T22:33:14.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.414+0000 7fc26c7c9700 1 Processor -- start 2023-09-30T22:33:14.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.414+0000 7fc26c7c9700 1 -- start start 2023-09-30T22:33:14.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.414+0000 7fc26c7c9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc26409d6f0 0x7fc2641a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:14.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.414+0000 7fc26c7c9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc26409e000 0x7fc2641a63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:14.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.414+0000 7fc26c7c9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc26409e410 0x7fc2641a68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:14.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.414+0000 7fc26c7c9700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2641a95e0 con 0x7fc26409e410 2023-09-30T22:33:14.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.414+0000 7fc26c7c9700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2640a2200 con 0x7fc26409d6f0 2023-09-30T22:33:14.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.414+0000 7fc26c7c9700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc26409c710 con 0x7fc26409e000 2023-09-30T22:33:14.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.414+0000 7fc269d64700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc26409e000 0x7fc2641a63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:14.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.415+0000 7fc269d64700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc26409e000 0x7fc2641a63b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37246/0 (socket says 172.21.15.16:37246) 2023-09-30T22:33:14.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.415+0000 7fc269d64700 1 -- 172.21.15.16:0/1069285254 learned_addr learned my addr 172.21.15.16:0/1069285254 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:14.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.415+0000 7fc26ad66700 1 --2- 172.21.15.16:0/1069285254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc26409e410 0x7fc2641a68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:14.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.415+0000 7fc26a565700 1 --2- 172.21.15.16:0/1069285254 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc26409d6f0 0x7fc2641a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:14.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.415+0000 7fc269d64700 1 -- 172.21.15.16:0/1069285254 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc26409d6f0 msgr2=0x7fc2641a5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:14.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.415+0000 7fc269d64700 1 --2- 172.21.15.16:0/1069285254 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc26409d6f0 0x7fc2641a5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:14.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.415+0000 7fc269d64700 1 -- 172.21.15.16:0/1069285254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc26409e410 msgr2=0x7fc2641a68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:14.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.415+0000 7fc269d64700 1 --2- 172.21.15.16:0/1069285254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc26409e410 0x7fc2641a68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:14.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.415+0000 7fc269d64700 1 -- 172.21.15.16:0/1069285254 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc254009550 con 0x7fc26409e000 2023-09-30T22:33:14.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.415+0000 7fc26a565700 1 --2- 172.21.15.16:0/1069285254 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc26409d6f0 0x7fc2641a5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:33:14.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.415+0000 7fc26ad66700 1 --2- 172.21.15.16:0/1069285254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc26409e410 0x7fc2641a68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:33:14.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.415+0000 7fc269d64700 1 --2- 172.21.15.16:0/1069285254 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc26409e000 0x7fc2641a63b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc254013000 tx=0x7fc254004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:14.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.415+0000 7fc25affd700 1 -- 172.21.15.16:0/1069285254 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc254004590 con 0x7fc26409e000 2023-09-30T22:33:14.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.415+0000 7fc25affd700 1 -- 172.21.15.16:0/1069285254 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2540036a0 con 0x7fc26409e000 2023-09-30T22:33:14.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.415+0000 7fc26c7c9700 1 -- 172.21.15.16:0/1069285254 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2641a7ac0 con 0x7fc26409e000 2023-09-30T22:33:14.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.415+0000 7fc25affd700 1 -- 172.21.15.16:0/1069285254 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc254003a50 con 0x7fc26409e000 2023-09-30T22:33:14.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.415+0000 7fc26c7c9700 1 -- 172.21.15.16:0/1069285254 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2641a98c0 con 0x7fc26409e000 2023-09-30T22:33:14.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.416+0000 7fc25affd700 1 -- 172.21.15.16:0/1069285254 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc254003bb0 con 0x7fc26409e000 2023-09-30T22:33:14.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.416+0000 7fc26c7c9700 1 -- 172.21.15.16:0/1069285254 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2641a9150 con 0x7fc26409e000 2023-09-30T22:33:14.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.417+0000 7fc25affd700 1 -- 172.21.15.16:0/1069285254 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fc254021030 con 0x7fc26409e000 2023-09-30T22:33:14.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.426+0000 7fc25affd700 1 -- 172.21.15.16:0/1069285254 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc254015070 con 0x7fc26409e000 2023-09-30T22:33:14.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.507+0000 7fedf3262700 1 -- 172.21.15.16:0/1295021709 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fedec072980 con 0x7fedec09d630 2023-09-30T22:33:14.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.508+0000 7fede97fa700 1 -- 172.21.15.16:0/1295021709 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7feddc01b590 con 0x7fedec09d630 2023-09-30T22:33:14.509 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:33:14.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.510+0000 7fedf3262700 1 -- 172.21.15.16:0/1295021709 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fedec09d630 msgr2=0x7fedec1c8170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:14.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.510+0000 7fedf3262700 1 --2- 172.21.15.16:0/1295021709 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fedec09d630 0x7fedec1c8170 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7feddc00e900 tx=0x7feddc00ec10 comp rx=0 tx=0).stop 2023-09-30T22:33:14.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.510+0000 7fedf3262700 1 -- 172.21.15.16:0/1295021709 shutdown_connections 2023-09-30T22:33:14.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.510+0000 7fedf3262700 1 --2- 172.21.15.16:0/1295021709 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fedec09d630 0x7fedec1c8170 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:14.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.510+0000 7fedf3262700 1 --2- 172.21.15.16:0/1295021709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedec133930 0x7fedec1c8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:14.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.510+0000 7fedf3262700 1 --2- 172.21.15.16:0/1295021709 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fedec0a0a30 0x7fedec1c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:14.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.510+0000 7fedf3262700 1 -- 172.21.15.16:0/1295021709 >> 172.21.15.16:0/1295021709 conn(0x7fedec0a3050 msgr2=0x7fedec1bbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:14.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.510+0000 7fedf3262700 1 -- 172.21.15.16:0/1295021709 shutdown_connections 2023-09-30T22:33:14.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.510+0000 7fedf3262700 1 -- 172.21.15.16:0/1295021709 wait complete. 2023-09-30T22:33:14.527 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:33:14.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.601+0000 7fe9603c0700 1 -- 172.21.15.16:0/850546280 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe9581277f0 con 0x7fe95806a770 2023-09-30T22:33:14.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.601+0000 7fe94effd700 1 -- 172.21.15.16:0/850546280 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe9581277f0 con 0x7fe95806a770 2023-09-30T22:33:14.602 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:33:14.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.604+0000 7fe9603c0700 1 -- 172.21.15.16:0/850546280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe95806a770 msgr2=0x7fe958126c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:14.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.604+0000 7fe9603c0700 1 --2- 172.21.15.16:0/850546280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe95806a770 0x7fe958126c50 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fe95400e9b0 tx=0x7fe95400ed70 comp rx=0 tx=0).stop 2023-09-30T22:33:14.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.604+0000 7fe9603c0700 1 -- 172.21.15.16:0/850546280 shutdown_connections 2023-09-30T22:33:14.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.604+0000 7fe9603c0700 1 --2- 172.21.15.16:0/850546280 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe95813c1e0 0x7fe9581cc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:14.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.604+0000 7fe9603c0700 1 --2- 172.21.15.16:0/850546280 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe95806a770 0x7fe958126c50 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:14.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.604+0000 7fe9603c0700 1 --2- 172.21.15.16:0/850546280 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe95809dc30 0x7fe958127190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:14.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.604+0000 7fe9603c0700 1 -- 172.21.15.16:0/850546280 >> 172.21.15.16:0/850546280 conn(0x7fe958066f50 msgr2=0x7fe95813b400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:14.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.604+0000 7fe9603c0700 1 -- 172.21.15.16:0/850546280 shutdown_connections 2023-09-30T22:33:14.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.604+0000 7fe9603c0700 1 -- 172.21.15.16:0/850546280 wait complete. 2023-09-30T22:33:14.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.616+0000 7fc171298700 1 -- 172.21.15.16:0/2248967766 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc16c074ea0 con 0x7fc16c09e1d0 2023-09-30T22:33:14.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.617+0000 7fc1537fe700 1 -- 172.21.15.16:0/2248967766 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc160011b40 con 0x7fc16c09e1d0 2023-09-30T22:33:14.617 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:33:14.618 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:33:14.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.619+0000 7fc171298700 1 -- 172.21.15.16:0/2248967766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc16c09e1d0 msgr2=0x7fc16c11ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:14.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.619+0000 7fc171298700 1 --2- 172.21.15.16:0/2248967766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc16c09e1d0 0x7fc16c11ddd0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fc16000e940 tx=0x7fc16000ec50 comp rx=0 tx=0).stop 2023-09-30T22:33:14.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.619+0000 7fc171298700 1 -- 172.21.15.16:0/2248967766 shutdown_connections 2023-09-30T22:33:14.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.619+0000 7fc171298700 1 --2- 172.21.15.16:0/2248967766 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc16c135e50 0x7fc16c11e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:14.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.619+0000 7fc171298700 1 --2- 172.21.15.16:0/2248967766 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc16c09d6f0 0x7fc16c11d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:14.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.619+0000 7fc171298700 1 --2- 172.21.15.16:0/2248967766 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc16c09e1d0 0x7fc16c11ddd0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:14.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.619+0000 7fc171298700 1 -- 172.21.15.16:0/2248967766 >> 172.21.15.16:0/2248967766 conn(0x7fc16c066f50 msgr2=0x7fc16c136a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:14.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.619+0000 7fc171298700 1 -- 172.21.15.16:0/2248967766 shutdown_connections 2023-09-30T22:33:14.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.619+0000 7fc171298700 1 -- 172.21.15.16:0/2248967766 wait complete. 2023-09-30T22:33:14.634 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:33:14.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.635+0000 7fc26c7c9700 1 -- 172.21.15.16:0/1069285254 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc2641a9720 con 0x7fc26409e000 2023-09-30T22:33:14.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.636+0000 7fc25affd700 1 -- 172.21.15.16:0/1069285254 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc2641a9720 con 0x7fc26409e000 2023-09-30T22:33:14.636 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:33:14.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.638+0000 7fc26c7c9700 1 -- 172.21.15.16:0/1069285254 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc26409e000 msgr2=0x7fc2641a63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:14.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.638+0000 7fc26c7c9700 1 --2- 172.21.15.16:0/1069285254 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc26409e000 0x7fc2641a63b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fc254013000 tx=0x7fc254004750 comp rx=0 tx=0).stop 2023-09-30T22:33:14.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.638+0000 7fc26c7c9700 1 -- 172.21.15.16:0/1069285254 shutdown_connections 2023-09-30T22:33:14.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.638+0000 7fc26c7c9700 1 --2- 172.21.15.16:0/1069285254 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc26409d6f0 0x7fc2641a5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:14.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.638+0000 7fc26c7c9700 1 --2- 172.21.15.16:0/1069285254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc26409e410 0x7fc2641a68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:14.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.638+0000 7fc26c7c9700 1 --2- 172.21.15.16:0/1069285254 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc26409e000 0x7fc2641a63b0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:14.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.638+0000 7fc26c7c9700 1 -- 172.21.15.16:0/1069285254 >> 172.21.15.16:0/1069285254 conn(0x7fc264066f50 msgr2=0x7fc2641bad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:14.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.638+0000 7fc26c7c9700 1 -- 172.21.15.16:0/1069285254 shutdown_connections 2023-09-30T22:33:14.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:14.638+0000 7fc26c7c9700 1 -- 172.21.15.16:0/1069285254 wait complete. 2023-09-30T22:33:14.652 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:33:15.528 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:33:15.619 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:33:15.635 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:33:15.653 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:33:15.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.674+0000 7f7c0614d700 1 Processor -- start 2023-09-30T22:33:15.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.674+0000 7f7c0614d700 1 -- start start 2023-09-30T22:33:15.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.675+0000 7f7c0614d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7c0006a770 0x7f7c0006ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:15.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.675+0000 7f7c0614d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7c0006b080 0x7f7c00131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:15.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.675+0000 7f7c0614d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7c00131b40 0x7f7c00131fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:15.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.675+0000 7f7c0614d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c0009ece0 con 0x7f7c00131b40 2023-09-30T22:33:15.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.675+0000 7f7c0614d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0009e760 con 0x7f7c0006b080 2023-09-30T22:33:15.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.675+0000 7f7c0614d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0009e040 con 0x7f7c0006a770 2023-09-30T22:33:15.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.675+0000 7f7bff7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7c0006a770 0x7f7c0006ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:15.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.675+0000 7f7bff7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7c0006a770 0x7f7c0006ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37262/0 (socket says 172.21.15.16:37262) 2023-09-30T22:33:15.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.675+0000 7f7bff7fe700 1 -- 172.21.15.16:0/478902509 learned_addr learned my addr 172.21.15.16:0/478902509 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:15.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.675+0000 7f7bfeffd700 1 --2- 172.21.15.16:0/478902509 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7c0006b080 0x7f7c00131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:15.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.676+0000 7f7bff7fe700 1 -- 172.21.15.16:0/478902509 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7c0006b080 msgr2=0x7f7c00131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:15.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.676+0000 7f7bff7fe700 1 --2- 172.21.15.16:0/478902509 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7c0006b080 0x7f7c00131600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:15.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.676+0000 7f7bff7fe700 1 -- 172.21.15.16:0/478902509 >> v2:172.21.15.16:3300/0 conn(0x7f7c00131b40 msgr2=0x7f7c00131fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:15.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.676+0000 7f7bff7fe700 1 --2- 172.21.15.16:0/478902509 >> v2:172.21.15.16:3300/0 conn(0x7f7c00131b40 0x7f7c00131fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:15.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.676+0000 7f7bff7fe700 1 -- 172.21.15.16:0/478902509 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c0009f630 con 0x7f7c0006a770 2023-09-30T22:33:15.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.676+0000 7f7bfeffd700 1 --2- 172.21.15.16:0/478902509 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7c0006b080 0x7f7c00131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:15.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.676+0000 7f7bff7fe700 1 --2- 172.21.15.16:0/478902509 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7c0006a770 0x7f7c0006ab40 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f7bf00098f0 tx=0x7f7bf0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee1b3e2044ae1b9f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:15.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.676+0000 7f7bfe7fc700 1 -- 172.21.15.16:0/478902509 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf0011830 con 0x7f7c0006a770 2023-09-30T22:33:15.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.676+0000 7f7bfe7fc700 1 -- 172.21.15.16:0/478902509 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bf0011990 con 0x7f7c0006a770 2023-09-30T22:33:15.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.676+0000 7f7bfe7fc700 1 -- 172.21.15.16:0/478902509 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf0011d20 con 0x7f7c0006a770 2023-09-30T22:33:15.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.677+0000 7f7c0614d700 1 -- 172.21.15.16:0/478902509 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7c0006a770 msgr2=0x7f7c0006ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:15.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.677+0000 7f7c0614d700 1 --2- 172.21.15.16:0/478902509 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7c0006a770 0x7f7c0006ab40 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f7bf00098f0 tx=0x7f7bf0009c00 comp rx=0 tx=0).stop 2023-09-30T22:33:15.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.677+0000 7f7c0614d700 1 -- 172.21.15.16:0/478902509 shutdown_connections 2023-09-30T22:33:15.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.677+0000 7f7c0614d700 1 --2- 172.21.15.16:0/478902509 >> v2:172.21.15.16:3300/0 conn(0x7f7c00131b40 0x7f7c00131fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:15.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.677+0000 7f7c0614d700 1 --2- 172.21.15.16:0/478902509 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7c0006a770 0x7f7c0006ab40 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:15.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.677+0000 7f7c0614d700 1 --2- 172.21.15.16:0/478902509 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7c0006b080 0x7f7c00131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:15.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.677+0000 7f7c0614d700 1 -- 172.21.15.16:0/478902509 >> 172.21.15.16:0/478902509 conn(0x7f7c00066f50 msgr2=0x7f7c0009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:15.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.677+0000 7f7c0614d700 1 -- 172.21.15.16:0/478902509 shutdown_connections 2023-09-30T22:33:15.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.677+0000 7f7c0614d700 1 -- 172.21.15.16:0/478902509 wait complete. 2023-09-30T22:33:15.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.678+0000 7f7c0614d700 1 Processor -- start 2023-09-30T22:33:15.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.679+0000 7f7c0614d700 1 -- start start 2023-09-30T22:33:15.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.679+0000 7f7c0614d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c0006a770 0x7f7c001203f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:15.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.679+0000 7f7c0614d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7c0006b080 0x7f7c00120930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:15.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.679+0000 7f7c0614d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7c00131b40 0x7f7c00120e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:15.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.679+0000 7f7c0614d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c00130e60 con 0x7f7c0006a770 2023-09-30T22:33:15.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.679+0000 7f7c0614d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c0009a0c0 con 0x7f7c00131b40 2023-09-30T22:33:15.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.679+0000 7f7c0614d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7c00071e60 con 0x7f7c0006b080 2023-09-30T22:33:15.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.679+0000 7f7bff7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c0006a770 0x7f7c001203f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:15.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.679+0000 7f7bff7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c0006a770 0x7f7c001203f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36906/0 (socket says 172.21.15.16:36906) 2023-09-30T22:33:15.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.679+0000 7f7bff7fe700 1 -- 172.21.15.16:0/2221529506 learned_addr learned my addr 172.21.15.16:0/2221529506 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:15.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.679+0000 7f7bfeffd700 1 --2- 172.21.15.16:0/2221529506 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7c0006b080 0x7f7c00120930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:15.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.679+0000 7f7bfffff700 1 --2- 172.21.15.16:0/2221529506 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7c00131b40 0x7f7c00120e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:15.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.680+0000 7f7bff7fe700 1 -- 172.21.15.16:0/2221529506 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7c0006b080 msgr2=0x7f7c00120930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:15.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.680+0000 7f7bff7fe700 1 --2- 172.21.15.16:0/2221529506 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7c0006b080 0x7f7c00120930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:15.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.680+0000 7f7bff7fe700 1 -- 172.21.15.16:0/2221529506 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7c00131b40 msgr2=0x7f7c00120e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:15.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.680+0000 7f7bff7fe700 1 --2- 172.21.15.16:0/2221529506 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7c00131b40 0x7f7c00120e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:15.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.680+0000 7f7bff7fe700 1 -- 172.21.15.16:0/2221529506 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf40094a0 con 0x7f7c0006a770 2023-09-30T22:33:15.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.680+0000 7f7bfffff700 1 --2- 172.21.15.16:0/2221529506 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7c00131b40 0x7f7c00120e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:33:15.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.680+0000 7f7bff7fe700 1 --2- 172.21.15.16:0/2221529506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c0006a770 0x7f7c001203f0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f7bf0000c00 tx=0x7f7bf0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:15.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.680+0000 7f7be7fff700 1 -- 172.21.15.16:0/2221529506 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf0004380 con 0x7f7c0006a770 2023-09-30T22:33:15.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.680+0000 7f7be7fff700 1 -- 172.21.15.16:0/2221529506 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bf001f070 con 0x7f7c0006a770 2023-09-30T22:33:15.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.680+0000 7f7be7fff700 1 -- 172.21.15.16:0/2221529506 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf00036a0 con 0x7f7c0006a770 2023-09-30T22:33:15.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.680+0000 7f7c0614d700 1 -- 172.21.15.16:0/2221529506 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bf0009550 con 0x7f7c0006a770 2023-09-30T22:33:15.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.680+0000 7f7c0614d700 1 -- 172.21.15.16:0/2221529506 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c001238e0 con 0x7f7c0006a770 2023-09-30T22:33:15.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.681+0000 7f7be7fff700 1 -- 172.21.15.16:0/2221529506 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7bf0021030 con 0x7f7c0006a770 2023-09-30T22:33:15.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.681+0000 7f7c0614d700 1 -- 172.21.15.16:0/2221529506 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c00074ea0 con 0x7f7c0006a770 2023-09-30T22:33:15.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.682+0000 7f7be7fff700 1 -- 172.21.15.16:0/2221529506 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f7bf0040a80 con 0x7f7c0006a770 2023-09-30T22:33:15.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.687+0000 7f7be7fff700 1 -- 172.21.15.16:0/2221529506 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7bf00199a0 con 0x7f7c0006a770 2023-09-30T22:33:15.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.768+0000 7f59ae81a700 1 Processor -- start 2023-09-30T22:33:15.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.768+0000 7f59ae81a700 1 -- start start 2023-09-30T22:33:15.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.768+0000 7f59ae81a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f59a809d6f0 0x7f59a809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:15.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.768+0000 7f59ae81a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f59a809e000 0x7f59a8130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:15.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.768+0000 7f59ae81a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f59a809e410 0x7f59a8130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:15.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.768+0000 7f59ae81a700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59a809f520 con 0x7f59a809e410 2023-09-30T22:33:15.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.768+0000 7f59ae81a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59a809f130 con 0x7f59a809d6f0 2023-09-30T22:33:15.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.768+0000 7f59ae81a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f59a809e930 con 0x7f59a809e000 2023-09-30T22:33:15.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.769+0000 7f59acdb7700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f59a809e410 0x7f59a8130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:15.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.769+0000 7f59acdb7700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f59a809e410 0x7f59a8130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36908/0 (socket says 172.21.15.16:36908) 2023-09-30T22:33:15.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.769+0000 7f59acdb7700 1 -- 172.21.15.16:0/3384705785 learned_addr learned my addr 172.21.15.16:0/3384705785 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:15.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.769+0000 7f599ffff700 1 --2- 172.21.15.16:0/3384705785 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f59a809e000 0x7f59a8130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:15.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.769+0000 7f59a7fff700 1 --2- 172.21.15.16:0/3384705785 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f59a809d6f0 0x7f59a809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:15.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.769+0000 7f59acdb7700 1 -- 172.21.15.16:0/3384705785 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f59a809e000 msgr2=0x7f59a8130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:15.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.769+0000 7f59acdb7700 1 --2- 172.21.15.16:0/3384705785 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f59a809e000 0x7f59a8130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:15.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.769+0000 7f59acdb7700 1 -- 172.21.15.16:0/3384705785 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f59a809d6f0 msgr2=0x7f59a809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:15.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.769+0000 7f59acdb7700 1 --2- 172.21.15.16:0/3384705785 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f59a809d6f0 0x7f59a809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:15.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.769+0000 7f59acdb7700 1 -- 172.21.15.16:0/3384705785 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59a809b2d0 con 0x7f59a809e410 2023-09-30T22:33:15.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.769+0000 7f59a7fff700 1 --2- 172.21.15.16:0/3384705785 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f59a809d6f0 0x7f59a809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:33:15.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.769+0000 7f59acdb7700 1 --2- 172.21.15.16:0/3384705785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59a809e410 0x7f59a8130ba0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f59a000d150 tx=0x7f59a000d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66a440e438b11f6f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:15.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.770+0000 7f59a77fe700 1 -- 172.21.15.16:0/3384705785 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a0010070 con 0x7f59a809e410 2023-09-30T22:33:15.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.770+0000 7f59a77fe700 1 -- 172.21.15.16:0/3384705785 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59a000dd10 con 0x7f59a809e410 2023-09-30T22:33:15.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.770+0000 7f59a77fe700 1 -- 172.21.15.16:0/3384705785 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a000b310 con 0x7f59a809e410 2023-09-30T22:33:15.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.770+0000 7f59ae81a700 1 -- 172.21.15.16:0/3384705785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59a809e410 msgr2=0x7f59a8130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:15.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.770+0000 7f59ae81a700 1 --2- 172.21.15.16:0/3384705785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59a809e410 0x7f59a8130ba0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f59a000d150 tx=0x7f59a000d460 comp rx=0 tx=0).stop 2023-09-30T22:33:15.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.771+0000 7f59ae81a700 1 -- 172.21.15.16:0/3384705785 shutdown_connections 2023-09-30T22:33:15.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.771+0000 7f59ae81a700 1 --2- 172.21.15.16:0/3384705785 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f59a809d6f0 0x7f59a809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:15.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.771+0000 7f59ae81a700 1 --2- 172.21.15.16:0/3384705785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59a809e410 0x7f59a8130ba0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:15.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.771+0000 7f59ae81a700 1 --2- 172.21.15.16:0/3384705785 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f59a809e000 0x7f59a8130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:15.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.771+0000 7f59ae81a700 1 -- 172.21.15.16:0/3384705785 >> 172.21.15.16:0/3384705785 conn(0x7f59a8066f50 msgr2=0x7f59a809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:15.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.771+0000 7f59ae81a700 1 -- 172.21.15.16:0/3384705785 shutdown_connections 2023-09-30T22:33:15.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.771+0000 7f59ae81a700 1 -- 172.21.15.16:0/3384705785 wait complete. 2023-09-30T22:33:15.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.772+0000 7f59ae81a700 1 Processor -- start 2023-09-30T22:33:15.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.772+0000 7f59ae81a700 1 -- start start 2023-09-30T22:33:15.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.773+0000 7f59ae81a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f59a809d6f0 0x7f59a811dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:15.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.773+0000 7f59ae81a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59a809e000 0x7f59a811e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:15.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.773+0000 7f59ae81a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f59a809e410 0x7f59a811e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:15.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.773+0000 7f59ae81a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59a811f390 con 0x7f59a809e000 2023-09-30T22:33:15.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.773+0000 7f59ae81a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59a8120030 con 0x7f59a809e410 2023-09-30T22:33:15.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.773+0000 7f59ae81a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f59a80a15f0 con 0x7f59a809d6f0 2023-09-30T22:33:15.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.773+0000 7f59a7fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f59a809d6f0 0x7f59a811dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:15.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.773+0000 7f59a7fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f59a809d6f0 0x7f59a811dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37286/0 (socket says 172.21.15.16:37286) 2023-09-30T22:33:15.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.773+0000 7f59a7fff700 1 -- 172.21.15.16:0/4029190494 learned_addr learned my addr 172.21.15.16:0/4029190494 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:15.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.773+0000 7f599ffff700 1 --2- 172.21.15.16:0/4029190494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59a809e000 0x7f59a811e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:15.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.773+0000 7f59a7fff700 1 -- 172.21.15.16:0/4029190494 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f59a809e410 msgr2=0x7f59a811e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:15.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.773+0000 7f59a7fff700 1 --2- 172.21.15.16:0/4029190494 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f59a809e410 0x7f59a811e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:15.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.773+0000 7f59a7fff700 1 -- 172.21.15.16:0/4029190494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59a809e000 msgr2=0x7f59a811e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:15.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.773+0000 7f59a7fff700 1 --2- 172.21.15.16:0/4029190494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59a809e000 0x7f59a811e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:15.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.773+0000 7f59a7fff700 1 -- 172.21.15.16:0/4029190494 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59a000bd40 con 0x7f59a809d6f0 2023-09-30T22:33:15.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.773+0000 7f599ffff700 1 --2- 172.21.15.16:0/4029190494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59a809e000 0x7f59a811e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:15.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.773+0000 7f59a7fff700 1 --2- 172.21.15.16:0/4029190494 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f59a809d6f0 0x7f59a811dee0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f599800d800 tx=0x7f599800db10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:15.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.774+0000 7f59a57fa700 1 -- 172.21.15.16:0/4029190494 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5998004030 con 0x7f59a809d6f0 2023-09-30T22:33:15.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.774+0000 7f59a57fa700 1 -- 172.21.15.16:0/4029190494 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5998004190 con 0x7f59a809d6f0 2023-09-30T22:33:15.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.774+0000 7f59ae81a700 1 -- 172.21.15.16:0/4029190494 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59a809c700 con 0x7f59a809d6f0 2023-09-30T22:33:15.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.774+0000 7f59a57fa700 1 -- 172.21.15.16:0/4029190494 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5998004520 con 0x7f59a809d6f0 2023-09-30T22:33:15.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.774+0000 7f59ae81a700 1 -- 172.21.15.16:0/4029190494 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59a81216b0 con 0x7f59a809d6f0 2023-09-30T22:33:15.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.775+0000 7f59a57fa700 1 -- 172.21.15.16:0/4029190494 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f59980036a0 con 0x7f59a809d6f0 2023-09-30T22:33:15.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.775+0000 7f59ae81a700 1 -- 172.21.15.16:0/4029190494 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59a811ac20 con 0x7f59a809d6f0 2023-09-30T22:33:15.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.775+0000 7f59a57fa700 1 -- 172.21.15.16:0/4029190494 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f599803fb70 con 0x7f59a809d6f0 2023-09-30T22:33:15.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.780+0000 7f59a57fa700 1 -- 172.21.15.16:0/4029190494 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5998003950 con 0x7f59a809d6f0 2023-09-30T22:33:15.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.808+0000 7f4e540c2700 1 Processor -- start 2023-09-30T22:33:15.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.809+0000 7f4e540c2700 1 -- start start 2023-09-30T22:33:15.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.809+0000 7f4e540c2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4e4c133a80 0x7f4e4c133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:15.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.809+0000 7f4e540c2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4e4c12ff20 0x7f4e4c130330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:15.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.809+0000 7f4e540c2700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4e4c130870 0x7f4e4c13a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:15.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.809+0000 7f4e540c2700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e4c06c570 con 0x7f4e4c130870 2023-09-30T22:33:15.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.809+0000 7f4e540c2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4c06bc00 con 0x7f4e4c133a80 2023-09-30T22:33:15.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.809+0000 7f4e540c2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4c0a0040 con 0x7f4e4c12ff20 2023-09-30T22:33:15.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.810+0000 7f4e5165d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4e4c12ff20 0x7f4e4c130330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:15.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.810+0000 7f4e5165d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4e4c12ff20 0x7f4e4c130330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37296/0 (socket says 172.21.15.16:37296) 2023-09-30T22:33:15.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.810+0000 7f4e5165d700 1 -- 172.21.15.16:0/4010851581 learned_addr learned my addr 172.21.15.16:0/4010851581 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:15.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.810+0000 7f4e51e5e700 1 --2- 172.21.15.16:0/4010851581 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4e4c133a80 0x7f4e4c133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:15.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.810+0000 7f4e5265f700 1 --2- 172.21.15.16:0/4010851581 >> v2:172.21.15.16:3300/0 conn(0x7f4e4c130870 0x7f4e4c13a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:15.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.810+0000 7f4e5165d700 1 -- 172.21.15.16:0/4010851581 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4e4c133a80 msgr2=0x7f4e4c133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:15.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.810+0000 7f4e5165d700 1 --2- 172.21.15.16:0/4010851581 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4e4c133a80 0x7f4e4c133e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:15.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.810+0000 7f4e5165d700 1 -- 172.21.15.16:0/4010851581 >> v2:172.21.15.16:3300/0 conn(0x7f4e4c130870 msgr2=0x7f4e4c13a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:15.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.810+0000 7f4e5165d700 1 --2- 172.21.15.16:0/4010851581 >> v2:172.21.15.16:3300/0 conn(0x7f4e4c130870 0x7f4e4c13a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:15.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.810+0000 7f4e5165d700 1 -- 172.21.15.16:0/4010851581 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e4c133860 con 0x7f4e4c12ff20 2023-09-30T22:33:15.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.810+0000 7f4e5165d700 1 --2- 172.21.15.16:0/4010851581 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4e4c12ff20 0x7f4e4c130330 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c0098f0 tx=0x7f4e3c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56680ae4b7a7b46d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:15.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.810+0000 7f4e50e5c700 1 -- 172.21.15.16:0/4010851581 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c011830 con 0x7f4e4c12ff20 2023-09-30T22:33:15.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.811+0000 7f4e50e5c700 1 -- 172.21.15.16:0/4010851581 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e3c011990 con 0x7f4e4c12ff20 2023-09-30T22:33:15.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.811+0000 7f4e50e5c700 1 -- 172.21.15.16:0/4010851581 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c011d20 con 0x7f4e4c12ff20 2023-09-30T22:33:15.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.811+0000 7f4e540c2700 1 -- 172.21.15.16:0/4010851581 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4e4c12ff20 msgr2=0x7f4e4c130330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:15.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.811+0000 7f4e540c2700 1 --2- 172.21.15.16:0/4010851581 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4e4c12ff20 0x7f4e4c130330 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c0098f0 tx=0x7f4e3c009c00 comp rx=0 tx=0).stop 2023-09-30T22:33:15.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.811+0000 7f4e540c2700 1 -- 172.21.15.16:0/4010851581 shutdown_connections 2023-09-30T22:33:15.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.811+0000 7f4e540c2700 1 --2- 172.21.15.16:0/4010851581 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4e4c133a80 0x7f4e4c133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:15.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.811+0000 7f4e540c2700 1 --2- 172.21.15.16:0/4010851581 >> v2:172.21.15.16:3300/0 conn(0x7f4e4c130870 0x7f4e4c13a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:15.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.811+0000 7f4e540c2700 1 --2- 172.21.15.16:0/4010851581 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4e4c12ff20 0x7f4e4c130330 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:15.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.811+0000 7f4e540c2700 1 -- 172.21.15.16:0/4010851581 >> 172.21.15.16:0/4010851581 conn(0x7f4e4c0a4210 msgr2=0x7f4e4c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:15.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.811+0000 7f4e540c2700 1 -- 172.21.15.16:0/4010851581 shutdown_connections 2023-09-30T22:33:15.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.811+0000 7f4e540c2700 1 -- 172.21.15.16:0/4010851581 wait complete. 2023-09-30T22:33:15.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.812+0000 7f4e540c2700 1 Processor -- start 2023-09-30T22:33:15.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.812+0000 7f4e540c2700 1 -- start start 2023-09-30T22:33:15.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.813+0000 7f4e540c2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e4c12ff20 0x7f4e4c09c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:15.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.813+0000 7f4e540c2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4e4c130870 0x7f4e4c09c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:15.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.813+0000 7f4e540c2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4e4c133a80 0x7f4e4c09cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:15.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.813+0000 7f4e540c2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4c11f540 con 0x7f4e4c12ff20 2023-09-30T22:33:15.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.813+0000 7f4e540c2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4c11feb0 con 0x7f4e4c133a80 2023-09-30T22:33:15.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.813+0000 7f4e540c2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4e4c09dde0 con 0x7f4e4c130870 2023-09-30T22:33:15.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.813+0000 7f4e5165d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4e4c130870 0x7f4e4c09c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:15.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.813+0000 7f4e5165d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4e4c130870 0x7f4e4c09c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37298/0 (socket says 172.21.15.16:37298) 2023-09-30T22:33:15.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.813+0000 7f4e5165d700 1 -- 172.21.15.16:0/3873810494 learned_addr learned my addr 172.21.15.16:0/3873810494 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:15.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.813+0000 7f4e51e5e700 1 --2- 172.21.15.16:0/3873810494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e4c12ff20 0x7f4e4c09c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:15.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.813+0000 7f4e5165d700 1 -- 172.21.15.16:0/3873810494 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4e4c133a80 msgr2=0x7f4e4c09cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:15.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.813+0000 7f4e5165d700 1 --2- 172.21.15.16:0/3873810494 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4e4c133a80 0x7f4e4c09cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:15.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.813+0000 7f4e5165d700 1 -- 172.21.15.16:0/3873810494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e4c12ff20 msgr2=0x7f4e4c09c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:15.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.813+0000 7f4e5165d700 1 --2- 172.21.15.16:0/3873810494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e4c12ff20 0x7f4e4c09c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:15.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.813+0000 7f4e5165d700 1 -- 172.21.15.16:0/3873810494 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e3c009550 con 0x7f4e4c130870 2023-09-30T22:33:15.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.813+0000 7f4e51e5e700 1 --2- 172.21.15.16:0/3873810494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e4c12ff20 0x7f4e4c09c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:33:15.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.814+0000 7f4e5165d700 1 --2- 172.21.15.16:0/3873810494 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4e4c130870 0x7f4e4c09c9c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c009c40 tx=0x7f4e3c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:15.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.814+0000 7f4e427fc700 1 -- 172.21.15.16:0/3873810494 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c004300 con 0x7f4e4c130870 2023-09-30T22:33:15.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.814+0000 7f4e427fc700 1 -- 172.21.15.16:0/3873810494 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e3c01f070 con 0x7f4e4c130870 2023-09-30T22:33:15.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.814+0000 7f4e540c2700 1 -- 172.21.15.16:0/3873810494 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e4c11f920 con 0x7f4e4c130870 2023-09-30T22:33:15.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.814+0000 7f4e427fc700 1 -- 172.21.15.16:0/3873810494 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e3c0036a0 con 0x7f4e4c130870 2023-09-30T22:33:15.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.814+0000 7f4e540c2700 1 -- 172.21.15.16:0/3873810494 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e4c11ec50 con 0x7f4e4c130870 2023-09-30T22:33:15.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.815+0000 7f4e427fc700 1 -- 172.21.15.16:0/3873810494 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4e3c021030 con 0x7f4e4c130870 2023-09-30T22:33:15.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.815+0000 7f4e427fc700 1 -- 172.21.15.16:0/3873810494 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f4e3c0407a0 con 0x7f4e4c130870 2023-09-30T22:33:15.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.815+0000 7f4e540c2700 1 -- 172.21.15.16:0/3873810494 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e4c074ea0 con 0x7f4e4c130870 2023-09-30T22:33:15.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.817+0000 7f1d13098700 1 Processor -- start 2023-09-30T22:33:15.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.817+0000 7f1d13098700 1 -- start start 2023-09-30T22:33:15.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.817+0000 7f1d13098700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d0c09d6f0 0x7f1d0c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:15.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.817+0000 7f1d13098700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1d0c09e000 0x7f1d0c130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:15.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.817+0000 7f1d13098700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1d0c09e410 0x7f1d0c130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:15.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.817+0000 7f1d13098700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d0c09f520 con 0x7f1d0c09e410 2023-09-30T22:33:15.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.817+0000 7f1d13098700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0c09f130 con 0x7f1d0c09d6f0 2023-09-30T22:33:15.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.817+0000 7f1d13098700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0c09e930 con 0x7f1d0c09e000 2023-09-30T22:33:15.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.817+0000 7f1d0bfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1d0c09e000 0x7f1d0c130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:15.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.818+0000 7f1d0bfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1d0c09e000 0x7f1d0c130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37304/0 (socket says 172.21.15.16:37304) 2023-09-30T22:33:15.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.818+0000 7f1d0bfff700 1 -- 172.21.15.16:0/973074392 learned_addr learned my addr 172.21.15.16:0/973074392 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:15.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.818+0000 7f1d0bfff700 1 -- 172.21.15.16:0/973074392 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d0c09d6f0 msgr2=0x7f1d0c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:15.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.818+0000 7f1d0bfff700 1 --2- 172.21.15.16:0/973074392 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d0c09d6f0 0x7f1d0c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:15.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.818+0000 7f1d0bfff700 1 -- 172.21.15.16:0/973074392 >> v2:172.21.15.16:3300/0 conn(0x7f1d0c09e410 msgr2=0x7f1d0c130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:15.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.818+0000 7f1d0bfff700 1 --2- 172.21.15.16:0/973074392 >> v2:172.21.15.16:3300/0 conn(0x7f1d0c09e410 0x7f1d0c130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:15.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.818+0000 7f1d0bfff700 1 -- 172.21.15.16:0/973074392 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d0c09b2d0 con 0x7f1d0c09e000 2023-09-30T22:33:15.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.818+0000 7f1d0bfff700 1 --2- 172.21.15.16:0/973074392 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1d0c09e000 0x7f1d0c130660 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f1cf8009920 tx=0x7f1cf8009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84ba6e65c4642d87 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:15.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.820+0000 7f1d0b7fe700 1 -- 172.21.15.16:0/973074392 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf800ba20 con 0x7f1d0c09e000 2023-09-30T22:33:15.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.820+0000 7f1d0b7fe700 1 -- 172.21.15.16:0/973074392 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf800bb80 con 0x7f1d0c09e000 2023-09-30T22:33:15.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.820+0000 7f1d0b7fe700 1 -- 172.21.15.16:0/973074392 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf8009e70 con 0x7f1d0c09e000 2023-09-30T22:33:15.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.821+0000 7f4e427fc700 1 -- 172.21.15.16:0/3873810494 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4e3c0199a0 con 0x7f4e4c130870 2023-09-30T22:33:15.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.821+0000 7f1d13098700 1 -- 172.21.15.16:0/973074392 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1d0c09e000 msgr2=0x7f1d0c130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:15.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.821+0000 7f1d13098700 1 --2- 172.21.15.16:0/973074392 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1d0c09e000 0x7f1d0c130660 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f1cf8009920 tx=0x7f1cf8009c30 comp rx=0 tx=0).stop 2023-09-30T22:33:15.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.821+0000 7f1d13098700 1 -- 172.21.15.16:0/973074392 shutdown_connections 2023-09-30T22:33:15.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.821+0000 7f1d13098700 1 --2- 172.21.15.16:0/973074392 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d0c09d6f0 0x7f1d0c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:15.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.821+0000 7f1d13098700 1 --2- 172.21.15.16:0/973074392 >> v2:172.21.15.16:3300/0 conn(0x7f1d0c09e410 0x7f1d0c130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:15.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.821+0000 7f1d13098700 1 --2- 172.21.15.16:0/973074392 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1d0c09e000 0x7f1d0c130660 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:15.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.821+0000 7f1d13098700 1 -- 172.21.15.16:0/973074392 >> 172.21.15.16:0/973074392 conn(0x7f1d0c066f50 msgr2=0x7f1d0c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:15.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.821+0000 7f1d13098700 1 -- 172.21.15.16:0/973074392 shutdown_connections 2023-09-30T22:33:15.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.821+0000 7f1d13098700 1 -- 172.21.15.16:0/973074392 wait complete. 2023-09-30T22:33:15.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.822+0000 7f1d13098700 1 Processor -- start 2023-09-30T22:33:15.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.822+0000 7f1d13098700 1 -- start start 2023-09-30T22:33:15.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.823+0000 7f1d13098700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d0c09d6f0 0x7f1d0c134760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:15.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.823+0000 7f1d13098700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d0c09e000 0x7f1d0c134ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:15.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.823+0000 7f1d13098700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1d0c09e410 0x7f1d0c1351e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:15.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.823+0000 7f1d13098700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0c09c700 con 0x7f1d0c09d6f0 2023-09-30T22:33:15.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.823+0000 7f1d13098700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0c122500 con 0x7f1d0c09e000 2023-09-30T22:33:15.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.823+0000 7f1d13098700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1d0c0a1d90 con 0x7f1d0c09e410 2023-09-30T22:33:15.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.823+0000 7f1d10e34700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d0c09d6f0 0x7f1d0c134760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:15.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.823+0000 7f1d10e34700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d0c09d6f0 0x7f1d0c134760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36946/0 (socket says 172.21.15.16:36946) 2023-09-30T22:33:15.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.823+0000 7f1d10e34700 1 -- 172.21.15.16:0/1474136219 learned_addr learned my addr 172.21.15.16:0/1474136219 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:15.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.823+0000 7f1d0bfff700 1 --2- 172.21.15.16:0/1474136219 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d0c09e000 0x7f1d0c134ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:15.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.824+0000 7f1d11635700 1 --2- 172.21.15.16:0/1474136219 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1d0c09e410 0x7f1d0c1351e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:15.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.824+0000 7f1d10e34700 1 -- 172.21.15.16:0/1474136219 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1d0c09e410 msgr2=0x7f1d0c1351e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:15.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.824+0000 7f1d10e34700 1 --2- 172.21.15.16:0/1474136219 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1d0c09e410 0x7f1d0c1351e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:15.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.824+0000 7f1d10e34700 1 -- 172.21.15.16:0/1474136219 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d0c09e000 msgr2=0x7f1d0c134ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:15.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.824+0000 7f1d10e34700 1 --2- 172.21.15.16:0/1474136219 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d0c09e000 0x7f1d0c134ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:15.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.824+0000 7f1d10e34700 1 -- 172.21.15.16:0/1474136219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf8009550 con 0x7f1d0c09d6f0 2023-09-30T22:33:15.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.824+0000 7f1d10e34700 1 --2- 172.21.15.16:0/1474136219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d0c09d6f0 0x7f1d0c134760 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f1d0000b560 tx=0x7f1d0000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:15.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.824+0000 7f1d097fa700 1 -- 172.21.15.16:0/1474136219 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d00011830 con 0x7f1d0c09d6f0 2023-09-30T22:33:15.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.824+0000 7f1d097fa700 1 -- 172.21.15.16:0/1474136219 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d000119b0 con 0x7f1d0c09d6f0 2023-09-30T22:33:15.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.824+0000 7f1d097fa700 1 -- 172.21.15.16:0/1474136219 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d00011d60 con 0x7f1d0c09d6f0 2023-09-30T22:33:15.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.824+0000 7f1d13098700 1 -- 172.21.15.16:0/1474136219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d0c136310 con 0x7f1d0c09d6f0 2023-09-30T22:33:15.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.825+0000 7f1d13098700 1 -- 172.21.15.16:0/1474136219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d0c136ee0 con 0x7f1d0c09d6f0 2023-09-30T22:33:15.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.826+0000 7f1d13098700 1 -- 172.21.15.16:0/1474136219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d0c139930 con 0x7f1d0c09d6f0 2023-09-30T22:33:15.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.826+0000 7f1d097fa700 1 -- 172.21.15.16:0/1474136219 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1d0001a400 con 0x7f1d0c09d6f0 2023-09-30T22:33:15.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.827+0000 7f1d097fa700 1 -- 172.21.15.16:0/1474136219 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f1d0003fd60 con 0x7f1d0c09d6f0 2023-09-30T22:33:15.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.835+0000 7f1d097fa700 1 -- 172.21.15.16:0/1474136219 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1d000216f0 con 0x7f1d0c09d6f0 2023-09-30T22:33:15.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.920+0000 7f7c0614d700 1 -- 172.21.15.16:0/2221529506 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7c00131920 con 0x7f7c0006a770 2023-09-30T22:33:15.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.920+0000 7f7be7fff700 1 -- 172.21.15.16:0/2221529506 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7bf0015070 con 0x7f7c0006a770 2023-09-30T22:33:15.921 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:33:15.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.923+0000 7f7c0614d700 1 -- 172.21.15.16:0/2221529506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c0006a770 msgr2=0x7f7c001203f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:15.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.923+0000 7f7c0614d700 1 --2- 172.21.15.16:0/2221529506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c0006a770 0x7f7c001203f0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f7bf0000c00 tx=0x7f7bf0004750 comp rx=0 tx=0).stop 2023-09-30T22:33:15.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.923+0000 7f7c0614d700 1 -- 172.21.15.16:0/2221529506 shutdown_connections 2023-09-30T22:33:15.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.923+0000 7f7c0614d700 1 --2- 172.21.15.16:0/2221529506 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7c00131b40 0x7f7c00120e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:15.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.923+0000 7f7c0614d700 1 --2- 172.21.15.16:0/2221529506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7c0006a770 0x7f7c001203f0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:15.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.923+0000 7f7c0614d700 1 --2- 172.21.15.16:0/2221529506 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7c0006b080 0x7f7c00120930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:15.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.923+0000 7f7c0614d700 1 -- 172.21.15.16:0/2221529506 >> 172.21.15.16:0/2221529506 conn(0x7f7c00066f50 msgr2=0x7f7c0006fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:15.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.923+0000 7f7c0614d700 1 -- 172.21.15.16:0/2221529506 shutdown_connections 2023-09-30T22:33:15.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.923+0000 7f7c0614d700 1 -- 172.21.15.16:0/2221529506 wait complete. 2023-09-30T22:33:15.938 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:33:15.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.993+0000 7f59ae81a700 1 -- 172.21.15.16:0/4029190494 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f59a8121300 con 0x7f59a809d6f0 2023-09-30T22:33:15.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.994+0000 7f59a57fa700 1 -- 172.21.15.16:0/4029190494 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5998013b20 con 0x7f59a809d6f0 2023-09-30T22:33:15.994 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:33:15.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.996+0000 7f59ae81a700 1 -- 172.21.15.16:0/4029190494 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f59a809d6f0 msgr2=0x7f59a811dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:15.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.996+0000 7f59ae81a700 1 --2- 172.21.15.16:0/4029190494 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f59a809d6f0 0x7f59a811dee0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f599800d800 tx=0x7f599800db10 comp rx=0 tx=0).stop 2023-09-30T22:33:15.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.996+0000 7f59ae81a700 1 -- 172.21.15.16:0/4029190494 shutdown_connections 2023-09-30T22:33:15.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.996+0000 7f59ae81a700 1 --2- 172.21.15.16:0/4029190494 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f59a809e410 0x7f59a811e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:15.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.996+0000 7f59ae81a700 1 --2- 172.21.15.16:0/4029190494 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f59a809d6f0 0x7f59a811dee0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:15.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.996+0000 7f59ae81a700 1 --2- 172.21.15.16:0/4029190494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59a809e000 0x7f59a811e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:15.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.996+0000 7f59ae81a700 1 -- 172.21.15.16:0/4029190494 >> 172.21.15.16:0/4029190494 conn(0x7f59a8066f50 msgr2=0x7f59a8130920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:15.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.996+0000 7f59ae81a700 1 -- 172.21.15.16:0/4029190494 shutdown_connections 2023-09-30T22:33:15.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:15.996+0000 7f59ae81a700 1 -- 172.21.15.16:0/4029190494 wait complete. 2023-09-30T22:33:16.010 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:33:16.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:16.033+0000 7f4e540c2700 1 -- 172.21.15.16:0/3873810494 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4e4c134360 con 0x7f4e4c130870 2023-09-30T22:33:16.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:16.033+0000 7f4e427fc700 1 -- 172.21.15.16:0/3873810494 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4e3c0193b0 con 0x7f4e4c130870 2023-09-30T22:33:16.034 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:33:16.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:16.035+0000 7f4e540c2700 1 -- 172.21.15.16:0/3873810494 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4e4c130870 msgr2=0x7f4e4c09c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:16.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:16.035+0000 7f4e540c2700 1 --2- 172.21.15.16:0/3873810494 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4e4c130870 0x7f4e4c09c9c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c009c40 tx=0x7f4e3c004750 comp rx=0 tx=0).stop 2023-09-30T22:33:16.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:16.035+0000 7f4e540c2700 1 -- 172.21.15.16:0/3873810494 shutdown_connections 2023-09-30T22:33:16.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:16.036+0000 7f4e540c2700 1 --2- 172.21.15.16:0/3873810494 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4e4c133a80 0x7f4e4c09cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:16.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:16.036+0000 7f4e540c2700 1 --2- 172.21.15.16:0/3873810494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4e4c12ff20 0x7f4e4c09c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:16.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:16.036+0000 7f4e540c2700 1 --2- 172.21.15.16:0/3873810494 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4e4c130870 0x7f4e4c09c9c0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:16.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:16.036+0000 7f4e540c2700 1 -- 172.21.15.16:0/3873810494 >> 172.21.15.16:0/3873810494 conn(0x7f4e4c0a4210 msgr2=0x7f4e4c0a0890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:16.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:16.036+0000 7f4e540c2700 1 -- 172.21.15.16:0/3873810494 shutdown_connections 2023-09-30T22:33:16.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:16.036+0000 7f4e540c2700 1 -- 172.21.15.16:0/3873810494 wait complete. 2023-09-30T22:33:16.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:16.047+0000 7f1d13098700 1 -- 172.21.15.16:0/1474136219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1d0c074ea0 con 0x7f1d0c09d6f0 2023-09-30T22:33:16.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:16.047+0000 7f1d097fa700 1 -- 172.21.15.16:0/1474136219 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1d00028070 con 0x7f1d0c09d6f0 2023-09-30T22:33:16.048 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:33:16.049 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:33:16.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:16.050+0000 7f1d13098700 1 -- 172.21.15.16:0/1474136219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d0c09d6f0 msgr2=0x7f1d0c134760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:16.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:16.050+0000 7f1d13098700 1 --2- 172.21.15.16:0/1474136219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d0c09d6f0 0x7f1d0c134760 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f1d0000b560 tx=0x7f1d0000b870 comp rx=0 tx=0).stop 2023-09-30T22:33:16.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:16.050+0000 7f1d13098700 1 -- 172.21.15.16:0/1474136219 shutdown_connections 2023-09-30T22:33:16.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:16.050+0000 7f1d13098700 1 --2- 172.21.15.16:0/1474136219 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1d0c09e410 0x7f1d0c1351e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:16.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:16.050+0000 7f1d13098700 1 --2- 172.21.15.16:0/1474136219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d0c09d6f0 0x7f1d0c134760 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:16.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:16.050+0000 7f1d13098700 1 --2- 172.21.15.16:0/1474136219 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d0c09e000 0x7f1d0c134ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:16.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:16.050+0000 7f1d13098700 1 -- 172.21.15.16:0/1474136219 >> 172.21.15.16:0/1474136219 conn(0x7f1d0c066f50 msgr2=0x7f1d0c06eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:16.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:16.050+0000 7f1d13098700 1 -- 172.21.15.16:0/1474136219 shutdown_connections 2023-09-30T22:33:16.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:16.050+0000 7f1d13098700 1 -- 172.21.15.16:0/1474136219 wait complete. 2023-09-30T22:33:16.063 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:33:16.939 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:33:17.012 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:33:17.050 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:33:17.065 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:33:17.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.103+0000 7efdc69b4700 1 Processor -- start 2023-09-30T22:33:17.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.103+0000 7efdc69b4700 1 -- start start 2023-09-30T22:33:17.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.104+0000 7efdc69b4700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efdc0131810 0x7efdc0131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:17.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.104+0000 7efdc69b4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efdc009a580 0x7efdc0139660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:17.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.104+0000 7efdc69b4700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdc01313c0 con 0x7efdc0099a40 2023-09-30T22:33:17.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.104+0000 7efdc69b4700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdc009b580 con 0x7efdc009a580 2023-09-30T22:33:17.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.104+0000 7efdc69b4700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdc006e8c0 con 0x7efdc0131810 2023-09-30T22:33:17.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.104+0000 7efdbf7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7efdc0099a40 0x7efdc0099e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36498/0 (socket says 172.21.15.16:36498) 2023-09-30T22:33:17.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.104+0000 7efdbf7fe700 1 -- 172.21.15.16:0/1196653888 learned_addr learned my addr 172.21.15.16:0/1196653888 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:17.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.104+0000 7efdc4f51700 1 --2- 172.21.15.16:0/1196653888 >> v2:172.21.15.16:3300/0 conn(0x7efdc009a580 0x7efdc0139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:17.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.104+0000 7efdbffff700 1 --2- 172.21.15.16:0/1196653888 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efdc0131810 0x7efdc0131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:17.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.105+0000 7efdc4f51700 1 -- 172.21.15.16:0/1196653888 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efdc0131810 msgr2=0x7efdc0131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.105+0000 7efdc4f51700 1 --2- 172.21.15.16:0/1196653888 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efdc0131810 0x7efdc0131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.105+0000 7efdc4f51700 1 -- 172.21.15.16:0/1196653888 >> v1:172.21.15.16:6789/0 conn(0x7efdc0099a40 legacy=0x7efdc0099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.105+0000 7efdc4f51700 1 -- 172.21.15.16:0/1196653888 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdc0071d20 con 0x7efdc009a580 2023-09-30T22:33:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.105+0000 7efdc4f51700 1 --2- 172.21.15.16:0/1196653888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdc009a580 0x7efdc0139660 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7efdb8009d90 tx=0x7efdb800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5f5f4a7d54e2cef server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.105+0000 7efdbeffd700 1 -- 172.21.15.16:0/1196653888 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb800e070 con 0x7efdc009a580 2023-09-30T22:33:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.105+0000 7efdbeffd700 1 -- 172.21.15.16:0/1196653888 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdb800bb20 con 0x7efdc009a580 2023-09-30T22:33:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.105+0000 7efdbeffd700 1 -- 172.21.15.16:0/1196653888 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb8012430 con 0x7efdc009a580 2023-09-30T22:33:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.106+0000 7efdc69b4700 1 -- 172.21.15.16:0/1196653888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdc009a580 msgr2=0x7efdc0139660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.106+0000 7efdc69b4700 1 --2- 172.21.15.16:0/1196653888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdc009a580 0x7efdc0139660 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7efdb8009d90 tx=0x7efdb800b330 comp rx=0 tx=0).stop 2023-09-30T22:33:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.106+0000 7efdc69b4700 1 -- 172.21.15.16:0/1196653888 shutdown_connections 2023-09-30T22:33:17.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.106+0000 7efdc69b4700 1 --2- 172.21.15.16:0/1196653888 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdc009a580 0x7efdc0139660 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:17.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.106+0000 7efdc69b4700 1 --2- 172.21.15.16:0/1196653888 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efdc0131810 0x7efdc0131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:17.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.106+0000 7efdc69b4700 1 -- 172.21.15.16:0/1196653888 >> 172.21.15.16:0/1196653888 conn(0x7efdc00a2aa0 msgr2=0x7efdc00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:17.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.106+0000 7efdc69b4700 1 -- 172.21.15.16:0/1196653888 shutdown_connections 2023-09-30T22:33:17.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.107+0000 7efdc69b4700 1 -- 172.21.15.16:0/1196653888 wait complete. 2023-09-30T22:33:17.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.109+0000 7efdc69b4700 1 Processor -- start 2023-09-30T22:33:17.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.109+0000 7efdc69b4700 1 -- start start 2023-09-30T22:33:17.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.109+0000 7efdc69b4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdc0099a40 0x7efdc011b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:17.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.109+0000 7efdc69b4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efdc009a580 0x7efdc011bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:17.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.109+0000 7efdc69b4700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efdc0131810 0x7efdc011c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:17.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.110+0000 7efdc69b4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdc006eb00 con 0x7efdc0099a40 2023-09-30T22:33:17.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.110+0000 7efdc69b4700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdc011da90 con 0x7efdc0131810 2023-09-30T22:33:17.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.110+0000 7efdc69b4700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efdc012ec90 con 0x7efdc009a580 2023-09-30T22:33:17.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.110+0000 7efdbffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdc0099a40 0x7efdc011b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:17.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.110+0000 7efdbf7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efdc009a580 0x7efdc011bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:17.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.110+0000 7efdbffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdc0099a40 0x7efdc011b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36960/0 (socket says 172.21.15.16:36960) 2023-09-30T22:33:17.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.110+0000 7efdbffff700 1 -- 172.21.15.16:0/2714632745 learned_addr learned my addr 172.21.15.16:0/2714632745 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:17.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.110+0000 7efdc4f51700 1 --2- 172.21.15.16:0/2714632745 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efdc0131810 0x7efdc011c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:17.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.110+0000 7efdbf7fe700 1 -- 172.21.15.16:0/2714632745 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efdc0131810 msgr2=0x7efdc011c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:17.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.110+0000 7efdbf7fe700 1 --2- 172.21.15.16:0/2714632745 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efdc0131810 0x7efdc011c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:17.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.110+0000 7efdbf7fe700 1 -- 172.21.15.16:0/2714632745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdc0099a40 msgr2=0x7efdc011b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:17.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.110+0000 7efdbf7fe700 1 --2- 172.21.15.16:0/2714632745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdc0099a40 0x7efdc011b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:17.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.110+0000 7efdbf7fe700 1 -- 172.21.15.16:0/2714632745 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdb8009a70 con 0x7efdc009a580 2023-09-30T22:33:17.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.110+0000 7efdbffff700 1 --2- 172.21.15.16:0/2714632745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdc0099a40 0x7efdc011b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:17.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.110+0000 7efdc4f51700 1 --2- 172.21.15.16:0/2714632745 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efdc0131810 0x7efdc011c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:17.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.110+0000 7efdbf7fe700 1 --2- 172.21.15.16:0/2714632745 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efdc009a580 0x7efdc011bde0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7efdb400c4d0 tx=0x7efdb400c7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:17.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.111+0000 7efdbcff9700 1 -- 172.21.15.16:0/2714632745 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb4005d70 con 0x7efdc009a580 2023-09-30T22:33:17.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.111+0000 7efdbcff9700 1 -- 172.21.15.16:0/2714632745 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdb400ce80 con 0x7efdc009a580 2023-09-30T22:33:17.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.111+0000 7efdc69b4700 1 -- 172.21.15.16:0/2714632745 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdc0072250 con 0x7efdc009a580 2023-09-30T22:33:17.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.111+0000 7efdbcff9700 1 -- 172.21.15.16:0/2714632745 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb40072d0 con 0x7efdc009a580 2023-09-30T22:33:17.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.111+0000 7efdc69b4700 1 -- 172.21.15.16:0/2714632745 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdc011f460 con 0x7efdc009a580 2023-09-30T22:33:17.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.112+0000 7efdbcff9700 1 -- 172.21.15.16:0/2714632745 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7efdb4007430 con 0x7efdc009a580 2023-09-30T22:33:17.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.112+0000 7efdbcff9700 1 -- 172.21.15.16:0/2714632745 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7efdb403eb30 con 0x7efdc009a580 2023-09-30T22:33:17.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.112+0000 7efdc69b4700 1 -- 172.21.15.16:0/2714632745 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdc011ed30 con 0x7efdc009a580 2023-09-30T22:33:17.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.118+0000 7efdbcff9700 1 -- 172.21.15.16:0/2714632745 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7efdb4017080 con 0x7efdc009a580 2023-09-30T22:33:17.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.170+0000 7f68043e3700 1 Processor -- start 2023-09-30T22:33:17.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.170+0000 7f68043e3700 1 -- start start 2023-09-30T22:33:17.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.170+0000 7f68043e3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f67fc09e1d0 0x7f67fc135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:17.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.170+0000 7f68043e3700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f67fc135e50 0x7f67fc138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:17.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.170+0000 7f68043e3700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67fc09f130 con 0x7f67fc09d6f0 2023-09-30T22:33:17.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.170+0000 7f68043e3700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67fc09f520 con 0x7f67fc135e50 2023-09-30T22:33:17.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.170+0000 7f68043e3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f67fc09e930 con 0x7f67fc09e1d0 2023-09-30T22:33:17.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.171+0000 7f680197e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f67fc09e1d0 0x7f67fc135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:17.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.171+0000 7f680197e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f67fc09e1d0 0x7f67fc135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37324/0 (socket says 172.21.15.16:37324) 2023-09-30T22:33:17.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.171+0000 7f680197e700 1 -- 172.21.15.16:0/1279506421 learned_addr learned my addr 172.21.15.16:0/1279506421 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:17.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.171+0000 7f6802980700 1 --2- 172.21.15.16:0/1279506421 >> v2:172.21.15.16:3300/0 conn(0x7f67fc135e50 0x7f67fc138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:17.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.171+0000 7f6802980700 1 -- 172.21.15.16:0/1279506421 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f67fc09e1d0 msgr2=0x7f67fc135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:17.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.171+0000 7f6802980700 1 --2- 172.21.15.16:0/1279506421 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f67fc09e1d0 0x7f67fc135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:17.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.171+0000 7f6802980700 1 -- 172.21.15.16:0/1279506421 >> v1:172.21.15.16:6789/0 conn(0x7f67fc09d6f0 legacy=0x7f67fc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:17.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.171+0000 7f6802980700 1 -- 172.21.15.16:0/1279506421 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67fc09b2d0 con 0x7f67fc135e50 2023-09-30T22:33:17.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.171+0000 7f6802980700 1 --2- 172.21.15.16:0/1279506421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67fc135e50 0x7f67fc138230 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f67f4009dc0 tx=0x7f67f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b52e92ee34a94b7b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:17.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.172+0000 7f680117d700 1 -- 172.21.15.16:0/1279506421 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f400e070 con 0x7f67fc135e50 2023-09-30T22:33:17.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.172+0000 7f680117d700 1 -- 172.21.15.16:0/1279506421 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67f400bae0 con 0x7f67fc135e50 2023-09-30T22:33:17.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.172+0000 7f680117d700 1 -- 172.21.15.16:0/1279506421 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f400be90 con 0x7f67fc135e50 2023-09-30T22:33:17.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.173+0000 7f68043e3700 1 -- 172.21.15.16:0/1279506421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67fc135e50 msgr2=0x7f67fc138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:17.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.173+0000 7f68043e3700 1 --2- 172.21.15.16:0/1279506421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67fc135e50 0x7f67fc138230 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f67f4009dc0 tx=0x7f67f400b330 comp rx=0 tx=0).stop 2023-09-30T22:33:17.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.173+0000 7f68043e3700 1 -- 172.21.15.16:0/1279506421 shutdown_connections 2023-09-30T22:33:17.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.173+0000 7f68043e3700 1 --2- 172.21.15.16:0/1279506421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67fc135e50 0x7f67fc138230 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:17.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.173+0000 7f68043e3700 1 --2- 172.21.15.16:0/1279506421 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f67fc09e1d0 0x7f67fc135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:17.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.173+0000 7f68043e3700 1 -- 172.21.15.16:0/1279506421 >> 172.21.15.16:0/1279506421 conn(0x7f67fc066f50 msgr2=0x7f67fc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:17.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.173+0000 7f68043e3700 1 -- 172.21.15.16:0/1279506421 shutdown_connections 2023-09-30T22:33:17.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.173+0000 7f68043e3700 1 -- 172.21.15.16:0/1279506421 wait complete. 2023-09-30T22:33:17.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.174+0000 7f68043e3700 1 Processor -- start 2023-09-30T22:33:17.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.175+0000 7f68043e3700 1 -- start start 2023-09-30T22:33:17.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.175+0000 7f68043e3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f67fc09d6f0 0x7f67fc119480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:17.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.175+0000 7f68043e3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67fc09e1d0 0x7f67fc1199c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:17.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.175+0000 7f68043e3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f67fc135e50 0x7f67fc119f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:17.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.175+0000 7f68043e3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67fc11cdf0 con 0x7f67fc09e1d0 2023-09-30T22:33:17.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.175+0000 7f68043e3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67fc11c5e0 con 0x7f67fc09d6f0 2023-09-30T22:33:17.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.175+0000 7f68043e3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f67fc1300b0 con 0x7f67fc135e50 2023-09-30T22:33:17.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.176+0000 7f6802980700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f67fc135e50 0x7f67fc119f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:17.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.176+0000 7f680217f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f67fc09d6f0 0x7f67fc119480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:17.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.176+0000 7f6802980700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f67fc135e50 0x7f67fc119f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37334/0 (socket says 172.21.15.16:37334) 2023-09-30T22:33:17.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.176+0000 7f6802980700 1 -- 172.21.15.16:0/1958170767 learned_addr learned my addr 172.21.15.16:0/1958170767 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:17.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.176+0000 7f680197e700 1 --2- 172.21.15.16:0/1958170767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67fc09e1d0 0x7f67fc1199c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:17.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.176+0000 7f6802980700 1 -- 172.21.15.16:0/1958170767 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f67fc09d6f0 msgr2=0x7f67fc119480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:17.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.176+0000 7f6802980700 1 --2- 172.21.15.16:0/1958170767 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f67fc09d6f0 0x7f67fc119480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:17.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.176+0000 7f6802980700 1 -- 172.21.15.16:0/1958170767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67fc09e1d0 msgr2=0x7f67fc1199c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:17.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.176+0000 7f6802980700 1 --2- 172.21.15.16:0/1958170767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67fc09e1d0 0x7f67fc1199c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:17.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.176+0000 7f6802980700 1 -- 172.21.15.16:0/1958170767 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67f4009a70 con 0x7f67fc135e50 2023-09-30T22:33:17.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.176+0000 7f680217f700 1 --2- 172.21.15.16:0/1958170767 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f67fc09d6f0 0x7f67fc119480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:17.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.176+0000 7f6802980700 1 --2- 172.21.15.16:0/1958170767 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f67fc135e50 0x7f67fc119f00 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f67f4013000 tx=0x7f67f4004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:17.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.176+0000 7f67f2ffd700 1 -- 172.21.15.16:0/1958170767 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f40045d0 con 0x7f67fc135e50 2023-09-30T22:33:17.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.176+0000 7f67f2ffd700 1 -- 172.21.15.16:0/1958170767 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67f4020070 con 0x7f67fc135e50 2023-09-30T22:33:17.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.177+0000 7f68043e3700 1 -- 172.21.15.16:0/1958170767 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67fc0a1d00 con 0x7f67fc135e50 2023-09-30T22:33:17.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.177+0000 7f67f2ffd700 1 -- 172.21.15.16:0/1958170767 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f4027480 con 0x7f67fc135e50 2023-09-30T22:33:17.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.177+0000 7f68043e3700 1 -- 172.21.15.16:0/1958170767 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67fc09c750 con 0x7f67fc135e50 2023-09-30T22:33:17.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.178+0000 7f67f2ffd700 1 -- 172.21.15.16:0/1958170767 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f67f4012430 con 0x7f67fc135e50 2023-09-30T22:33:17.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.178+0000 7f67f2ffd700 1 -- 172.21.15.16:0/1958170767 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f67f4022070 con 0x7f67fc135e50 2023-09-30T22:33:17.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.178+0000 7f68043e3700 1 -- 172.21.15.16:0/1958170767 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67fc074ea0 con 0x7f67fc135e50 2023-09-30T22:33:17.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.187+0000 7f67f2ffd700 1 -- 172.21.15.16:0/1958170767 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f67f401c400 con 0x7f67fc135e50 2023-09-30T22:33:17.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.205+0000 7f706f5b8700 1 Processor -- start 2023-09-30T22:33:17.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.206+0000 7f706f5b8700 1 -- start start 2023-09-30T22:33:17.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.206+0000 7f706f5b8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f706809e930 0x7f706809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:17.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.206+0000 7f706f5b8700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f706809f240 0x7f70681317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:17.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.206+0000 7f706f5b8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f706809f650 0x7f7068131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:17.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.206+0000 7f706f5b8700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7068070940 con 0x7f706809f240 2023-09-30T22:33:17.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.206+0000 7f706f5b8700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f706806fd60 con 0x7f706809f650 2023-09-30T22:33:17.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.206+0000 7f706f5b8700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f706806fa80 con 0x7f706809e930 2023-09-30T22:33:17.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.207+0000 7f706db55700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f706809f650 0x7f7068131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:17.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.207+0000 7f706db55700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f706809f650 0x7f7068131d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41890/0 (socket says 172.21.15.16:41890) 2023-09-30T22:33:17.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.207+0000 7f706db55700 1 -- 172.21.15.16:0/904395278 learned_addr learned my addr 172.21.15.16:0/904395278 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:17.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.207+0000 7f706d354700 1 --2- 172.21.15.16:0/904395278 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f706809e930 0x7f706809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:17.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.207+0000 7f706cb53700 1 --2- 172.21.15.16:0/904395278 >> v2:172.21.15.16:3300/0 conn(0x7f706809f240 0x7f70681317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:17.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.207+0000 7f706db55700 1 -- 172.21.15.16:0/904395278 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f706809e930 msgr2=0x7f706809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:17.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.207+0000 7f706db55700 1 --2- 172.21.15.16:0/904395278 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f706809e930 0x7f706809ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:17.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.207+0000 7f706db55700 1 -- 172.21.15.16:0/904395278 >> v2:172.21.15.16:3300/0 conn(0x7f706809f240 msgr2=0x7f70681317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:17.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.207+0000 7f706db55700 1 --2- 172.21.15.16:0/904395278 >> v2:172.21.15.16:3300/0 conn(0x7f706809f240 0x7f70681317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:17.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.207+0000 7f706db55700 1 -- 172.21.15.16:0/904395278 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f706809fb60 con 0x7f706809f650 2023-09-30T22:33:17.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.207+0000 7f706db55700 1 --2- 172.21.15.16:0/904395278 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f706809f650 0x7f7068131d30 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f7060009e50 tx=0x7f706000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e62801a8006d30c8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:17.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.207+0000 7f705ffff700 1 -- 172.21.15.16:0/904395278 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706000e070 con 0x7f706809f650 2023-09-30T22:33:17.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.208+0000 7f705ffff700 1 -- 172.21.15.16:0/904395278 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f706000b910 con 0x7f706809f650 2023-09-30T22:33:17.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.208+0000 7f705ffff700 1 -- 172.21.15.16:0/904395278 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706000bcc0 con 0x7f706809f650 2023-09-30T22:33:17.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.208+0000 7f706f5b8700 1 -- 172.21.15.16:0/904395278 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f706809f650 msgr2=0x7f7068131d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:17.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.208+0000 7f706f5b8700 1 --2- 172.21.15.16:0/904395278 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f706809f650 0x7f7068131d30 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f7060009e50 tx=0x7f706000b330 comp rx=0 tx=0).stop 2023-09-30T22:33:17.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.208+0000 7f706f5b8700 1 -- 172.21.15.16:0/904395278 shutdown_connections 2023-09-30T22:33:17.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.208+0000 7f706f5b8700 1 --2- 172.21.15.16:0/904395278 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f706809f650 0x7f7068131d30 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:17.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.208+0000 7f706f5b8700 1 --2- 172.21.15.16:0/904395278 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f706809e930 0x7f706809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:17.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.208+0000 7f706f5b8700 1 --2- 172.21.15.16:0/904395278 >> v2:172.21.15.16:3300/0 conn(0x7f706809f240 0x7f70681317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:17.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.208+0000 7f706f5b8700 1 -- 172.21.15.16:0/904395278 >> 172.21.15.16:0/904395278 conn(0x7f70680a4210 msgr2=0x7f70680a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:17.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.208+0000 7f706f5b8700 1 -- 172.21.15.16:0/904395278 shutdown_connections 2023-09-30T22:33:17.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.208+0000 7f706f5b8700 1 -- 172.21.15.16:0/904395278 wait complete. 2023-09-30T22:33:17.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.210+0000 7f706f5b8700 1 Processor -- start 2023-09-30T22:33:17.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.210+0000 7f706f5b8700 1 -- start start 2023-09-30T22:33:17.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.211+0000 7f706f5b8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f706809e930 0x7f70681a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:17.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.211+0000 7f706f5b8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f706809f240 0x7f70681a40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:17.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.211+0000 7f706f5b8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f706809f650 0x7f70681a4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:17.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.211+0000 7f706f5b8700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70681a73f0 con 0x7f706809f240 2023-09-30T22:33:17.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.211+0000 7f706f5b8700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70680a3510 con 0x7f706809e930 2023-09-30T22:33:17.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.211+0000 7f706f5b8700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f70681a58d0 con 0x7f706809f650 2023-09-30T22:33:17.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.211+0000 7f706d354700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f706809e930 0x7f70681a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:17.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.211+0000 7f706d354700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f706809e930 0x7f70681a3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41906/0 (socket says 172.21.15.16:41906) 2023-09-30T22:33:17.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.211+0000 7f706d354700 1 -- 172.21.15.16:0/3584689926 learned_addr learned my addr 172.21.15.16:0/3584689926 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:17.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.211+0000 7f706db55700 1 --2- 172.21.15.16:0/3584689926 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f706809f650 0x7f70681a4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:17.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.211+0000 7f706cb53700 1 --2- 172.21.15.16:0/3584689926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f706809f240 0x7f70681a40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:17.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.211+0000 7f706d354700 1 -- 172.21.15.16:0/3584689926 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f706809f650 msgr2=0x7f70681a4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:17.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.211+0000 7f706d354700 1 --2- 172.21.15.16:0/3584689926 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f706809f650 0x7f70681a4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:17.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.211+0000 7f706d354700 1 -- 172.21.15.16:0/3584689926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f706809f240 msgr2=0x7f70681a40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:17.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.211+0000 7f706d354700 1 --2- 172.21.15.16:0/3584689926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f706809f240 0x7f70681a40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:17.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.211+0000 7f706d354700 1 -- 172.21.15.16:0/3584689926 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7060009b30 con 0x7f706809e930 2023-09-30T22:33:17.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.212+0000 7f706d354700 1 --2- 172.21.15.16:0/3584689926 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f706809e930 0x7f70681a3bb0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f706400e930 tx=0x7f706400ec40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:17.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.212+0000 7f705dffb700 1 -- 172.21.15.16:0/3584689926 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706400ee80 con 0x7f706809e930 2023-09-30T22:33:17.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.212+0000 7f705dffb700 1 -- 172.21.15.16:0/3584689926 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f706400c850 con 0x7f706809e930 2023-09-30T22:33:17.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.212+0000 7f705dffb700 1 -- 172.21.15.16:0/3584689926 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706400cbe0 con 0x7f706809e930 2023-09-30T22:33:17.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.213+0000 7f706f5b8700 1 -- 172.21.15.16:0/3584689926 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70680a0190 con 0x7f706809e930 2023-09-30T22:33:17.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.213+0000 7f706f5b8700 1 -- 172.21.15.16:0/3584689926 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70681a6a10 con 0x7f706809e930 2023-09-30T22:33:17.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.214+0000 7f705dffb700 1 -- 172.21.15.16:0/3584689926 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f706400cd40 con 0x7f706809e930 2023-09-30T22:33:17.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.214+0000 7f705dffb700 1 -- 172.21.15.16:0/3584689926 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f706403fc20 con 0x7f706809e930 2023-09-30T22:33:17.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.218+0000 7f706f5b8700 1 -- 172.21.15.16:0/3584689926 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70681a0b30 con 0x7f706809e930 2023-09-30T22:33:17.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.227+0000 7f705dffb700 1 -- 172.21.15.16:0/3584689926 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f706401b3b0 con 0x7f706809e930 2023-09-30T22:33:17.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.255+0000 7f5739700700 1 Processor -- start 2023-09-30T22:33:17.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.256+0000 7f5739700700 1 -- start start 2023-09-30T22:33:17.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.256+0000 7f5739700700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f573409d6f0 0x7f573409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:17.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.256+0000 7f5739700700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f573409e000 0x7f57341339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:17.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.256+0000 7f5739700700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f573409d2f0 con 0x7f5734133f30 2023-09-30T22:33:17.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.256+0000 7f5739700700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f573409a810 con 0x7f573409e000 2023-09-30T22:33:17.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.256+0000 7f5739700700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f573409a470 con 0x7f573409d6f0 2023-09-30T22:33:17.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.256+0000 7f57337fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5734133f30 0x7f573413ac40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:36530/0 (socket says 172.21.15.16:36530) 2023-09-30T22:33:17.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.256+0000 7f57327fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f573409e000 0x7f57341339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:17.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.256+0000 7f57337fe700 1 -- 172.21.15.16:0/2871108532 learned_addr learned my addr 172.21.15.16:0/2871108532 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:17.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.256+0000 7f5732ffd700 1 --2- 172.21.15.16:0/2871108532 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f573409d6f0 0x7f573409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:17.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.257+0000 7f57327fc700 1 -- 172.21.15.16:0/2871108532 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f573409d6f0 msgr2=0x7f573409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:17.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.257+0000 7f57327fc700 1 --2- 172.21.15.16:0/2871108532 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f573409d6f0 0x7f573409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:17.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.257+0000 7f57327fc700 1 -- 172.21.15.16:0/2871108532 >> v1:172.21.15.16:6789/0 conn(0x7f5734133f30 legacy=0x7f573413ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:17.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.257+0000 7f5731ffb700 1 -- 172.21.15.16:0/2871108532 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3525694101 0 0) 0x7f573409d2f0 con 0x7f5734133f30 2023-09-30T22:33:17.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.257+0000 7f57327fc700 1 -- 172.21.15.16:0/2871108532 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f573409ec40 con 0x7f573409e000 2023-09-30T22:33:17.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.257+0000 7f57327fc700 1 --2- 172.21.15.16:0/2871108532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f573409e000 0x7f57341339f0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f5728009840 tx=0x7f5728009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=691e73ea7820355b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:17.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.257+0000 7f5731ffb700 1 -- 172.21.15.16:0/2871108532 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5728011830 con 0x7f573409e000 2023-09-30T22:33:17.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.257+0000 7f5731ffb700 1 -- 172.21.15.16:0/2871108532 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5728011990 con 0x7f573409e000 2023-09-30T22:33:17.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.257+0000 7f5731ffb700 1 -- 172.21.15.16:0/2871108532 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5728011d20 con 0x7f573409e000 2023-09-30T22:33:17.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.258+0000 7f5739700700 1 -- 172.21.15.16:0/2871108532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f573409e000 msgr2=0x7f57341339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:17.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.258+0000 7f5739700700 1 --2- 172.21.15.16:0/2871108532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f573409e000 0x7f57341339f0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f5728009840 tx=0x7f5728009b50 comp rx=0 tx=0).stop 2023-09-30T22:33:17.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.258+0000 7f5739700700 1 -- 172.21.15.16:0/2871108532 shutdown_connections 2023-09-30T22:33:17.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.258+0000 7f5739700700 1 --2- 172.21.15.16:0/2871108532 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f573409d6f0 0x7f573409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:17.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.258+0000 7f5739700700 1 --2- 172.21.15.16:0/2871108532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f573409e000 0x7f57341339f0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:17.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.258+0000 7f5739700700 1 -- 172.21.15.16:0/2871108532 >> 172.21.15.16:0/2871108532 conn(0x7f5734066f50 msgr2=0x7f573409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:17.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.258+0000 7f5739700700 1 -- 172.21.15.16:0/2871108532 shutdown_connections 2023-09-30T22:33:17.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.258+0000 7f5739700700 1 -- 172.21.15.16:0/2871108532 wait complete. 2023-09-30T22:33:17.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.259+0000 7f5739700700 1 Processor -- start 2023-09-30T22:33:17.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.259+0000 7f5739700700 1 -- start start 2023-09-30T22:33:17.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.260+0000 7f5739700700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f573409d6f0 0x7f5734122780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:17.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.260+0000 7f5739700700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f573409e000 0x7f5734122cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:17.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.260+0000 7f5739700700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5734133f30 0x7f5734123200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:17.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.260+0000 7f5739700700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5734125d10 con 0x7f573409d6f0 2023-09-30T22:33:17.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.260+0000 7f5739700700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5734124400 con 0x7f573409e000 2023-09-30T22:33:17.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.260+0000 7f5739700700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f57341336a0 con 0x7f5734133f30 2023-09-30T22:33:17.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.260+0000 7f57327fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f573409e000 0x7f5734122cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:17.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.260+0000 7f57327fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f573409e000 0x7f5734122cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41930/0 (socket says 172.21.15.16:41930) 2023-09-30T22:33:17.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.260+0000 7f57327fc700 1 -- 172.21.15.16:0/656801300 learned_addr learned my addr 172.21.15.16:0/656801300 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:17.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.260+0000 7f5732ffd700 1 --2- 172.21.15.16:0/656801300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f573409d6f0 0x7f5734122780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:17.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.260+0000 7f57337fe700 1 --2- 172.21.15.16:0/656801300 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5734133f30 0x7f5734123200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:17.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.260+0000 7f57327fc700 1 -- 172.21.15.16:0/656801300 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5734133f30 msgr2=0x7f5734123200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:17.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.260+0000 7f57327fc700 1 --2- 172.21.15.16:0/656801300 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5734133f30 0x7f5734123200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:17.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.260+0000 7f57327fc700 1 -- 172.21.15.16:0/656801300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f573409d6f0 msgr2=0x7f5734122780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:17.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.260+0000 7f57327fc700 1 --2- 172.21.15.16:0/656801300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f573409d6f0 0x7f5734122780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:17.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.260+0000 7f57327fc700 1 -- 172.21.15.16:0/656801300 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57280094a0 con 0x7f573409e000 2023-09-30T22:33:17.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.260+0000 7f57327fc700 1 --2- 172.21.15.16:0/656801300 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f573409e000 0x7f5734122cc0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f5728009fd0 tx=0x7f5728004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:17.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.261+0000 7f571b7fe700 1 -- 172.21.15.16:0/656801300 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57280038a0 con 0x7f573409e000 2023-09-30T22:33:17.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.261+0000 7f571b7fe700 1 -- 172.21.15.16:0/656801300 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f572801f070 con 0x7f573409e000 2023-09-30T22:33:17.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.261+0000 7f571b7fe700 1 -- 172.21.15.16:0/656801300 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5728003c00 con 0x7f573409e000 2023-09-30T22:33:17.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.261+0000 7f5739700700 1 -- 172.21.15.16:0/656801300 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57341331e0 con 0x7f573409e000 2023-09-30T22:33:17.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.261+0000 7f5739700700 1 -- 172.21.15.16:0/656801300 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f573409c640 con 0x7f573409e000 2023-09-30T22:33:17.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.262+0000 7f5739700700 1 -- 172.21.15.16:0/656801300 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5734074ea0 con 0x7f573409e000 2023-09-30T22:33:17.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.263+0000 7f571b7fe700 1 -- 172.21.15.16:0/656801300 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f572801a430 con 0x7f573409e000 2023-09-30T22:33:17.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.263+0000 7f571b7fe700 1 -- 172.21.15.16:0/656801300 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f5728021070 con 0x7f573409e000 2023-09-30T22:33:17.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.267+0000 7f571b7fe700 1 -- 172.21.15.16:0/656801300 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f57280253e0 con 0x7f573409e000 2023-09-30T22:33:17.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.335+0000 7efdc69b4700 1 -- 172.21.15.16:0/2714632745 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7efdc0074ea0 con 0x7efdc009a580 2023-09-30T22:33:17.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.335+0000 7efdbcff9700 1 -- 172.21.15.16:0/2714632745 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7efdb401b3b0 con 0x7efdc009a580 2023-09-30T22:33:17.336 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:33:17.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.339+0000 7efdc69b4700 1 -- 172.21.15.16:0/2714632745 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efdc009a580 msgr2=0x7efdc011bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:17.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.339+0000 7efdc69b4700 1 --2- 172.21.15.16:0/2714632745 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efdc009a580 0x7efdc011bde0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7efdb400c4d0 tx=0x7efdb400c7e0 comp rx=0 tx=0).stop 2023-09-30T22:33:17.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.340+0000 7efdc69b4700 1 -- 172.21.15.16:0/2714632745 shutdown_connections 2023-09-30T22:33:17.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.340+0000 7efdc69b4700 1 --2- 172.21.15.16:0/2714632745 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efdc0131810 0x7efdc011c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:17.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.340+0000 7efdc69b4700 1 --2- 172.21.15.16:0/2714632745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efdc0099a40 0x7efdc011b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:17.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.340+0000 7efdc69b4700 1 --2- 172.21.15.16:0/2714632745 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efdc009a580 0x7efdc011bde0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:17.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.340+0000 7efdc69b4700 1 -- 172.21.15.16:0/2714632745 >> 172.21.15.16:0/2714632745 conn(0x7efdc00a2aa0 msgr2=0x7efdc009d770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:17.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.340+0000 7efdc69b4700 1 -- 172.21.15.16:0/2714632745 shutdown_connections 2023-09-30T22:33:17.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.340+0000 7efdc69b4700 1 -- 172.21.15.16:0/2714632745 wait complete. 2023-09-30T22:33:17.359 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:33:17.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.442+0000 7f68043e3700 1 -- 172.21.15.16:0/1958170767 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f67fc06c360 con 0x7f67fc135e50 2023-09-30T22:33:17.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.442+0000 7f67f2ffd700 1 -- 172.21.15.16:0/1958170767 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f67f401a380 con 0x7f67fc135e50 2023-09-30T22:33:17.443 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:33:17.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.444+0000 7f68043e3700 1 -- 172.21.15.16:0/1958170767 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f67fc135e50 msgr2=0x7f67fc119f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:17.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.444+0000 7f68043e3700 1 --2- 172.21.15.16:0/1958170767 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f67fc135e50 0x7f67fc119f00 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f67f4013000 tx=0x7f67f4004ad0 comp rx=0 tx=0).stop 2023-09-30T22:33:17.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.444+0000 7f68043e3700 1 -- 172.21.15.16:0/1958170767 shutdown_connections 2023-09-30T22:33:17.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.444+0000 7f68043e3700 1 --2- 172.21.15.16:0/1958170767 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f67fc09d6f0 0x7f67fc119480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:17.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.444+0000 7f68043e3700 1 --2- 172.21.15.16:0/1958170767 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f67fc135e50 0x7f67fc119f00 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:17.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.444+0000 7f68043e3700 1 --2- 172.21.15.16:0/1958170767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67fc09e1d0 0x7f67fc1199c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:17.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.444+0000 7f68043e3700 1 -- 172.21.15.16:0/1958170767 >> 172.21.15.16:0/1958170767 conn(0x7f67fc066f50 msgr2=0x7f67fc1327f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:17.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.444+0000 7f68043e3700 1 -- 172.21.15.16:0/1958170767 shutdown_connections 2023-09-30T22:33:17.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.444+0000 7f68043e3700 1 -- 172.21.15.16:0/1958170767 wait complete. 2023-09-30T22:33:17.458 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:33:17.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.482+0000 7f706f5b8700 1 -- 172.21.15.16:0/3584689926 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7068074ea0 con 0x7f706809e930 2023-09-30T22:33:17.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.482+0000 7f705dffb700 1 -- 172.21.15.16:0/3584689926 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f706402b090 con 0x7f706809e930 2023-09-30T22:33:17.483 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:33:17.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.485+0000 7f706f5b8700 1 -- 172.21.15.16:0/3584689926 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f706809e930 msgr2=0x7f70681a3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:17.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.485+0000 7f706f5b8700 1 --2- 172.21.15.16:0/3584689926 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f706809e930 0x7f70681a3bb0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f706400e930 tx=0x7f706400ec40 comp rx=0 tx=0).stop 2023-09-30T22:33:17.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.485+0000 7f706f5b8700 1 -- 172.21.15.16:0/3584689926 shutdown_connections 2023-09-30T22:33:17.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.485+0000 7f706f5b8700 1 --2- 172.21.15.16:0/3584689926 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f706809e930 0x7f70681a3bb0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:17.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.485+0000 7f706f5b8700 1 --2- 172.21.15.16:0/3584689926 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f706809f650 0x7f70681a4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:17.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.485+0000 7f706f5b8700 1 --2- 172.21.15.16:0/3584689926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f706809f240 0x7f70681a40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:17.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.485+0000 7f706f5b8700 1 -- 172.21.15.16:0/3584689926 >> 172.21.15.16:0/3584689926 conn(0x7f70680a4210 msgr2=0x7f70681b8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:17.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.485+0000 7f706f5b8700 1 -- 172.21.15.16:0/3584689926 shutdown_connections 2023-09-30T22:33:17.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.485+0000 7f706f5b8700 1 -- 172.21.15.16:0/3584689926 wait complete. 2023-09-30T22:33:17.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.485+0000 7f5739700700 1 -- 172.21.15.16:0/656801300 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f573406c360 con 0x7f573409e000 2023-09-30T22:33:17.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.486+0000 7f571b7fe700 1 -- 172.21.15.16:0/656801300 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f572802b020 con 0x7f573409e000 2023-09-30T22:33:17.487 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:33:17.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.488+0000 7f5739700700 1 -- 172.21.15.16:0/656801300 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f573409e000 msgr2=0x7f5734122cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:17.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.488+0000 7f5739700700 1 --2- 172.21.15.16:0/656801300 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f573409e000 0x7f5734122cc0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f5728009fd0 tx=0x7f5728004750 comp rx=0 tx=0).stop 2023-09-30T22:33:17.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.488+0000 7f5739700700 1 -- 172.21.15.16:0/656801300 shutdown_connections 2023-09-30T22:33:17.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.488+0000 7f5739700700 1 --2- 172.21.15.16:0/656801300 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5734133f30 0x7f5734123200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:17.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.488+0000 7f5739700700 1 --2- 172.21.15.16:0/656801300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f573409d6f0 0x7f5734122780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:17.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.488+0000 7f5739700700 1 --2- 172.21.15.16:0/656801300 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f573409e000 0x7f5734122cc0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:17.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.488+0000 7f5739700700 1 -- 172.21.15.16:0/656801300 >> 172.21.15.16:0/656801300 conn(0x7f5734066f50 msgr2=0x7f5734070c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:17.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.488+0000 7f5739700700 1 -- 172.21.15.16:0/656801300 shutdown_connections 2023-09-30T22:33:17.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:17.488+0000 7f5739700700 1 -- 172.21.15.16:0/656801300 wait complete. 2023-09-30T22:33:17.500 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:33:17.503 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:33:18.360 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:33:18.459 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:33:18.500 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:33:18.503 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:33:18.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.510+0000 7fcbb42a2700 1 Processor -- start 2023-09-30T22:33:18.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.510+0000 7fcbb42a2700 1 -- start start 2023-09-30T22:33:18.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.511+0000 7fcbb42a2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcbac06a770 0x7fcbac06ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:18.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.511+0000 7fcbb42a2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcbac06b080 0x7fcbac131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:18.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.511+0000 7fcbb42a2700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbac09ece0 con 0x7fcbac131b40 2023-09-30T22:33:18.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.511+0000 7fcbb42a2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbac09e760 con 0x7fcbac06a770 2023-09-30T22:33:18.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.511+0000 7fcbb42a2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcbac09e040 con 0x7fcbac06b080 2023-09-30T22:33:18.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.511+0000 7fcbb203e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcbac06a770 0x7fcbac06ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:18.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.511+0000 7fcbb203e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcbac06a770 0x7fcbac06ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54396/0 (socket says 172.21.15.16:54396) 2023-09-30T22:33:18.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.511+0000 7fcbb203e700 1 -- 172.21.15.16:0/4213634804 learned_addr learned my addr 172.21.15.16:0/4213634804 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:18.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.512+0000 7fcbb183d700 1 --2- 172.21.15.16:0/4213634804 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcbac06b080 0x7fcbac131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:18.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.512+0000 7fcbb203e700 1 -- 172.21.15.16:0/4213634804 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcbac06b080 msgr2=0x7fcbac131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:18.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.512+0000 7fcbb203e700 1 --2- 172.21.15.16:0/4213634804 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcbac06b080 0x7fcbac131600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:18.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.512+0000 7fcbb203e700 1 -- 172.21.15.16:0/4213634804 >> v1:172.21.15.16:6789/0 conn(0x7fcbac131b40 legacy=0x7fcbac138850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:18.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.512+0000 7fcbb203e700 1 -- 172.21.15.16:0/4213634804 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbac09f630 con 0x7fcbac06a770 2023-09-30T22:33:18.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.512+0000 7fcbb103c700 1 -- 172.21.15.16:0/4213634804 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 504321225 0 0) 0x7fcbac09ece0 con 0x7fcbac131b40 2023-09-30T22:33:18.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.512+0000 7fcbb203e700 1 --2- 172.21.15.16:0/4213634804 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcbac06a770 0x7fcbac06ab40 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fcba8009920 tx=0x7fcba8009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10a1a1d9048b2670 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:18.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.512+0000 7fcbb103c700 1 -- 172.21.15.16:0/4213634804 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba800ba60 con 0x7fcbac06a770 2023-09-30T22:33:18.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.512+0000 7fcbb103c700 1 -- 172.21.15.16:0/4213634804 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba800bbc0 con 0x7fcbac06a770 2023-09-30T22:33:18.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.512+0000 7fcbb103c700 1 -- 172.21.15.16:0/4213634804 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba8009e70 con 0x7fcbac06a770 2023-09-30T22:33:18.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.513+0000 7fcbb42a2700 1 -- 172.21.15.16:0/4213634804 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcbac06a770 msgr2=0x7fcbac06ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:18.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.513+0000 7fcbb42a2700 1 --2- 172.21.15.16:0/4213634804 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcbac06a770 0x7fcbac06ab40 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fcba8009920 tx=0x7fcba8009c30 comp rx=0 tx=0).stop 2023-09-30T22:33:18.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.513+0000 7fcbb42a2700 1 -- 172.21.15.16:0/4213634804 shutdown_connections 2023-09-30T22:33:18.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.513+0000 7fcbb42a2700 1 --2- 172.21.15.16:0/4213634804 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcbac06a770 0x7fcbac06ab40 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:18.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.513+0000 7fcbb42a2700 1 --2- 172.21.15.16:0/4213634804 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcbac06b080 0x7fcbac131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:18.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.513+0000 7fcbb42a2700 1 -- 172.21.15.16:0/4213634804 >> 172.21.15.16:0/4213634804 conn(0x7fcbac066f50 msgr2=0x7fcbac09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:18.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.513+0000 7fcbb42a2700 1 -- 172.21.15.16:0/4213634804 shutdown_connections 2023-09-30T22:33:18.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.514+0000 7fcbb42a2700 1 -- 172.21.15.16:0/4213634804 wait complete. 2023-09-30T22:33:18.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.515+0000 7fcbb42a2700 1 Processor -- start 2023-09-30T22:33:18.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.515+0000 7fcbb42a2700 1 -- start start 2023-09-30T22:33:18.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.515+0000 7fcbb42a2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcbac06a770 0x7fcbac09ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:18.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.515+0000 7fcbb42a2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbac06b080 0x7fcbac09b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:18.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.515+0000 7fcbb42a2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcbac131b40 0x7fcbac1c83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:18.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.515+0000 7fcbb42a2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbac1310d0 con 0x7fcbac06b080 2023-09-30T22:33:18.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.515+0000 7fcbb42a2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbac09a8e0 con 0x7fcbac131b40 2023-09-30T22:33:18.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.515+0000 7fcbb42a2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcbac071e60 con 0x7fcbac06a770 2023-09-30T22:33:18.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.515+0000 7fcbb203e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcbac06a770 0x7fcbac09ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:18.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.515+0000 7fcbb203e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcbac06a770 0x7fcbac09ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:36026/0 (socket says 172.21.15.16:36026) 2023-09-30T22:33:18.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.515+0000 7fcbb203e700 1 -- 172.21.15.16:0/3053313120 learned_addr learned my addr 172.21.15.16:0/3053313120 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:18.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.516+0000 7fcbb183d700 1 --2- 172.21.15.16:0/3053313120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbac06b080 0x7fcbac09b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:18.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.516+0000 7fcbb203e700 1 -- 172.21.15.16:0/3053313120 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcbac131b40 msgr2=0x7fcbac1c83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:18.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.516+0000 7fcbb203e700 1 --2- 172.21.15.16:0/3053313120 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcbac131b40 0x7fcbac1c83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:18.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.516+0000 7fcbb203e700 1 -- 172.21.15.16:0/3053313120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbac06b080 msgr2=0x7fcbac09b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:18.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.516+0000 7fcbb203e700 1 --2- 172.21.15.16:0/3053313120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbac06b080 0x7fcbac09b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:18.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.516+0000 7fcbb203e700 1 -- 172.21.15.16:0/3053313120 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba8009550 con 0x7fcbac06a770 2023-09-30T22:33:18.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.516+0000 7fcbb203e700 1 --2- 172.21.15.16:0/3053313120 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcbac06a770 0x7fcbac09ae60 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fcba8009c70 tx=0x7fcba8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:18.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.517+0000 7fcba2ffd700 1 -- 172.21.15.16:0/3053313120 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba8004380 con 0x7fcbac06a770 2023-09-30T22:33:18.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.517+0000 7fcba2ffd700 1 -- 172.21.15.16:0/3053313120 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba801f070 con 0x7fcbac06a770 2023-09-30T22:33:18.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.517+0000 7fcba2ffd700 1 -- 172.21.15.16:0/3053313120 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba80036a0 con 0x7fcbac06a770 2023-09-30T22:33:18.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.517+0000 7fcbb42a2700 1 -- 172.21.15.16:0/3053313120 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbac06b470 con 0x7fcbac06a770 2023-09-30T22:33:18.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.517+0000 7fcbb42a2700 1 -- 172.21.15.16:0/3053313120 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbac09c010 con 0x7fcbac06a770 2023-09-30T22:33:18.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.518+0000 7fcba2ffd700 1 -- 172.21.15.16:0/3053313120 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcba8021030 con 0x7fcbac06a770 2023-09-30T22:33:18.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.518+0000 7fcba2ffd700 1 -- 172.21.15.16:0/3053313120 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fcba8040a50 con 0x7fcbac06a770 2023-09-30T22:33:18.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.518+0000 7fcbb42a2700 1 -- 172.21.15.16:0/3053313120 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbac09b8e0 con 0x7fcbac06a770 2023-09-30T22:33:18.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.525+0000 7fcba2ffd700 1 -- 172.21.15.16:0/3053313120 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcba8015080 con 0x7fcbac06a770 2023-09-30T22:33:18.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.610+0000 7fd9a1656700 1 Processor -- start 2023-09-30T22:33:18.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.610+0000 7fd9a1656700 1 -- start start 2023-09-30T22:33:18.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.610+0000 7fd9a1656700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd99c0a0a30 0x7fd99c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:18.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.610+0000 7fd9a1656700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd99c134170 0x7fd99c13bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:18.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.610+0000 7fd9a1656700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd99c09edd0 con 0x7fd99c0a1340 2023-09-30T22:33:18.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.610+0000 7fd9a1656700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd99c09e1f0 con 0x7fd99c0a0a30 2023-09-30T22:33:18.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.610+0000 7fd9a1656700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd99c09dc50 con 0x7fd99c134170 2023-09-30T22:33:18.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.610+0000 7fd99affd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd99c0a0a30 0x7fd99c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:18.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.611+0000 7fd99affd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd99c0a0a30 0x7fd99c0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54414/0 (socket says 172.21.15.16:54414) 2023-09-30T22:33:18.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.611+0000 7fd99affd700 1 -- 172.21.15.16:0/1654702880 learned_addr learned my addr 172.21.15.16:0/1654702880 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:18.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.611+0000 7fd99b7fe700 1 --2- 172.21.15.16:0/1654702880 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd99c134170 0x7fd99c13bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:18.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.611+0000 7fd999ffb700 1 -- 172.21.15.16:0/1654702880 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4055420391 0 0) 0x7fd99c09edd0 con 0x7fd99c0a1340 2023-09-30T22:33:18.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.611+0000 7fd999ffb700 1 -- 172.21.15.16:0/1654702880 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9840035b0 con 0x7fd99c0a1340 2023-09-30T22:33:18.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.611+0000 7fd99b7fe700 1 -- 172.21.15.16:0/1654702880 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd99c0a0a30 msgr2=0x7fd99c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:18.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.611+0000 7fd99b7fe700 1 --2- 172.21.15.16:0/1654702880 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd99c0a0a30 0x7fd99c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:18.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.611+0000 7fd99b7fe700 1 -- 172.21.15.16:0/1654702880 >> v1:172.21.15.16:6789/0 conn(0x7fd99c0a1340 legacy=0x7fd99c133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:18.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.611+0000 7fd99b7fe700 1 -- 172.21.15.16:0/1654702880 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd99c0a1fe0 con 0x7fd99c134170 2023-09-30T22:33:18.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.611+0000 7fd99affd700 1 --2- 172.21.15.16:0/1654702880 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd99c0a0a30 0x7fd99c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:18.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.611+0000 7fd99b7fe700 1 --2- 172.21.15.16:0/1654702880 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd99c134170 0x7fd99c13bd30 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd994009dc0 tx=0x7fd99400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28d9d8294896240d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:18.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.612+0000 7fd999ffb700 1 -- 172.21.15.16:0/1654702880 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99400e070 con 0x7fd99c134170 2023-09-30T22:33:18.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.612+0000 7fd999ffb700 1 -- 172.21.15.16:0/1654702880 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd99400bb20 con 0x7fd99c134170 2023-09-30T22:33:18.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.612+0000 7fd999ffb700 1 -- 172.21.15.16:0/1654702880 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd994012430 con 0x7fd99c134170 2023-09-30T22:33:18.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.612+0000 7fd9a1656700 1 -- 172.21.15.16:0/1654702880 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd99c134170 msgr2=0x7fd99c13bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:18.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.612+0000 7fd9a1656700 1 --2- 172.21.15.16:0/1654702880 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd99c134170 0x7fd99c13bd30 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd994009dc0 tx=0x7fd99400b330 comp rx=0 tx=0).stop 2023-09-30T22:33:18.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.613+0000 7fd9a1656700 1 -- 172.21.15.16:0/1654702880 shutdown_connections 2023-09-30T22:33:18.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.613+0000 7fd9a1656700 1 --2- 172.21.15.16:0/1654702880 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd99c134170 0x7fd99c13bd30 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:18.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.613+0000 7fd9a1656700 1 --2- 172.21.15.16:0/1654702880 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd99c0a0a30 0x7fd99c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:18.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.613+0000 7fd9a1656700 1 -- 172.21.15.16:0/1654702880 >> 172.21.15.16:0/1654702880 conn(0x7fd99c0a6550 msgr2=0x7fd99c0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:18.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.613+0000 7fd9a1656700 1 -- 172.21.15.16:0/1654702880 shutdown_connections 2023-09-30T22:33:18.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.613+0000 7fd9a1656700 1 -- 172.21.15.16:0/1654702880 wait complete. 2023-09-30T22:33:18.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.614+0000 7fd9a1656700 1 Processor -- start 2023-09-30T22:33:18.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.614+0000 7fd9a1656700 1 -- start start 2023-09-30T22:33:18.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.615+0000 7fd9a1656700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd99c0a0a30 0x7fd99c1226f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:18.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.615+0000 7fd9a1656700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd99c0a1340 0x7fd99c122c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:18.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.615+0000 7fd9a1656700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd99c134170 0x7fd99c123170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:18.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.615+0000 7fd9a1656700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd99c126210 con 0x7fd99c0a0a30 2023-09-30T22:33:18.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.615+0000 7fd9a1656700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd99c124300 con 0x7fd99c134170 2023-09-30T22:33:18.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.615+0000 7fd9a1656700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd99c0a4ce0 con 0x7fd99c0a1340 2023-09-30T22:33:18.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.615+0000 7fd99a7fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd99c0a1340 0x7fd99c122c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:18.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.615+0000 7fd99a7fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd99c0a1340 0x7fd99c122c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:36044/0 (socket says 172.21.15.16:36044) 2023-09-30T22:33:18.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.615+0000 7fd99a7fc700 1 -- 172.21.15.16:0/4201724022 learned_addr learned my addr 172.21.15.16:0/4201724022 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:18.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.615+0000 7fd99affd700 1 --2- 172.21.15.16:0/4201724022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd99c0a0a30 0x7fd99c1226f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:18.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.616+0000 7fd99b7fe700 1 --2- 172.21.15.16:0/4201724022 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd99c134170 0x7fd99c123170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:18.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.616+0000 7fd99a7fc700 1 -- 172.21.15.16:0/4201724022 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd99c134170 msgr2=0x7fd99c123170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:18.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.616+0000 7fd99a7fc700 1 --2- 172.21.15.16:0/4201724022 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd99c134170 0x7fd99c123170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:18.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.616+0000 7fd99a7fc700 1 -- 172.21.15.16:0/4201724022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd99c0a0a30 msgr2=0x7fd99c1226f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:18.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.616+0000 7fd99a7fc700 1 --2- 172.21.15.16:0/4201724022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd99c0a0a30 0x7fd99c1226f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:18.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.616+0000 7fd99a7fc700 1 -- 172.21.15.16:0/4201724022 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd98c009550 con 0x7fd99c0a1340 2023-09-30T22:33:18.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.616+0000 7fd99a7fc700 1 --2- 172.21.15.16:0/4201724022 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd99c0a1340 0x7fd99c122c30 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd99000c520 tx=0x7fd99000c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:18.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.616+0000 7fd9837fe700 1 -- 172.21.15.16:0/4201724022 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9900052d0 con 0x7fd99c0a1340 2023-09-30T22:33:18.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.616+0000 7fd9837fe700 1 -- 172.21.15.16:0/4201724022 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd990005430 con 0x7fd99c0a1340 2023-09-30T22:33:18.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.616+0000 7fd9837fe700 1 -- 172.21.15.16:0/4201724022 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9900038c0 con 0x7fd99c0a1340 2023-09-30T22:33:18.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.616+0000 7fd9a1656700 1 -- 172.21.15.16:0/4201724022 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd994009a70 con 0x7fd99c0a1340 2023-09-30T22:33:18.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.617+0000 7fd9a1656700 1 -- 172.21.15.16:0/4201724022 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd99c0a25e0 con 0x7fd99c0a1340 2023-09-30T22:33:18.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.618+0000 7fd9a1656700 1 -- 172.21.15.16:0/4201724022 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd99c133270 con 0x7fd99c0a1340 2023-09-30T22:33:18.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.618+0000 7fd9837fe700 1 -- 172.21.15.16:0/4201724022 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd990003a20 con 0x7fd99c0a1340 2023-09-30T22:33:18.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.619+0000 7fd9837fe700 1 -- 172.21.15.16:0/4201724022 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd99003eb60 con 0x7fd99c0a1340 2023-09-30T22:33:18.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.623+0000 7fd9837fe700 1 -- 172.21.15.16:0/4201724022 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd99000f260 con 0x7fd99c0a1340 2023-09-30T22:33:18.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.656+0000 7f6423d57700 1 Processor -- start 2023-09-30T22:33:18.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.656+0000 7f6423d57700 1 -- start start 2023-09-30T22:33:18.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.657+0000 7f6423d57700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f641c134630 0x7f641c136a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:18.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.657+0000 7f6423d57700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f641c136f50 0x7f641c139330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:18.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.657+0000 7f6423d57700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f641c09b4f0 con 0x7f641c131810 2023-09-30T22:33:18.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.657+0000 7f6423d57700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f641c1313c0 con 0x7f641c136f50 2023-09-30T22:33:18.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.657+0000 7f6423d57700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f641c06e8c0 con 0x7f641c134630 2023-09-30T22:33:18.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.657+0000 7f64222f4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f641c136f50 0x7f641c139330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:18.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.657+0000 7f6421af3700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f641c131810 0x7f641c131be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54664/0 (socket says 172.21.15.16:54664) 2023-09-30T22:33:18.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.657+0000 7f64222f4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f641c136f50 0x7f641c139330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41346/0 (socket says 172.21.15.16:41346) 2023-09-30T22:33:18.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.657+0000 7f6421af3700 1 -- 172.21.15.16:0/1308768712 learned_addr learned my addr 172.21.15.16:0/1308768712 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:18.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.657+0000 7f64212f2700 1 --2- 172.21.15.16:0/1308768712 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f641c134630 0x7f641c136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:18.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.657+0000 7f64222f4700 1 -- 172.21.15.16:0/1308768712 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f641c134630 msgr2=0x7f641c136a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:18.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.657+0000 7f64222f4700 1 --2- 172.21.15.16:0/1308768712 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f641c134630 0x7f641c136a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:18.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.657+0000 7f64222f4700 1 -- 172.21.15.16:0/1308768712 >> v1:172.21.15.16:6789/0 conn(0x7f641c131810 legacy=0x7f641c131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:18.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.657+0000 7f64222f4700 1 -- 172.21.15.16:0/1308768712 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f641c071d20 con 0x7f641c136f50 2023-09-30T22:33:18.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.658+0000 7f64222f4700 1 --2- 172.21.15.16:0/1308768712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f641c136f50 0x7f641c139330 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f6414009d90 tx=0x7f641400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43e1d4f9af4488d9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:18.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.658+0000 7f6420af1700 1 -- 172.21.15.16:0/1308768712 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f641400e070 con 0x7f641c136f50 2023-09-30T22:33:18.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.658+0000 7f6420af1700 1 -- 172.21.15.16:0/1308768712 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f641400bbd0 con 0x7f641c136f50 2023-09-30T22:33:18.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.658+0000 7f6420af1700 1 -- 172.21.15.16:0/1308768712 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6414012430 con 0x7f641c136f50 2023-09-30T22:33:18.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.659+0000 7f6423d57700 1 -- 172.21.15.16:0/1308768712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f641c136f50 msgr2=0x7f641c139330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:18.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.659+0000 7f6423d57700 1 --2- 172.21.15.16:0/1308768712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f641c136f50 0x7f641c139330 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f6414009d90 tx=0x7f641400b330 comp rx=0 tx=0).stop 2023-09-30T22:33:18.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.659+0000 7f6423d57700 1 -- 172.21.15.16:0/1308768712 shutdown_connections 2023-09-30T22:33:18.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.659+0000 7f6423d57700 1 --2- 172.21.15.16:0/1308768712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f641c136f50 0x7f641c139330 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:18.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.659+0000 7f6423d57700 1 --2- 172.21.15.16:0/1308768712 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f641c134630 0x7f641c136a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:18.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.659+0000 7f6423d57700 1 -- 172.21.15.16:0/1308768712 >> 172.21.15.16:0/1308768712 conn(0x7f641c0a2aa0 msgr2=0x7f641c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:18.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.659+0000 7f6423d57700 1 -- 172.21.15.16:0/1308768712 shutdown_connections 2023-09-30T22:33:18.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.659+0000 7f6423d57700 1 -- 172.21.15.16:0/1308768712 wait complete. 2023-09-30T22:33:18.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.660+0000 7f6423d57700 1 Processor -- start 2023-09-30T22:33:18.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.660+0000 7f6423d57700 1 -- start start 2023-09-30T22:33:18.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.660+0000 7f6423d57700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f641c131810 0x7f641c118cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:18.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.660+0000 7f6423d57700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f641c134630 0x7f641c119210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:18.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.660+0000 7f6423d57700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f641c136f50 0x7f641c119750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:18.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.660+0000 7f6423d57700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f641c0a2180 con 0x7f641c136f50 2023-09-30T22:33:18.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.660+0000 7f6423d57700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f641c11c670 con 0x7f641c131810 2023-09-30T22:33:18.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.660+0000 7f6423d57700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f641c11a8e0 con 0x7f641c134630 2023-09-30T22:33:18.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.660+0000 7f64212f2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f641c134630 0x7f641c119210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:18.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.661+0000 7f64212f2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f641c134630 0x7f641c119210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:36054/0 (socket says 172.21.15.16:36054) 2023-09-30T22:33:18.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.661+0000 7f64212f2700 1 -- 172.21.15.16:0/3240747723 learned_addr learned my addr 172.21.15.16:0/3240747723 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:18.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.661+0000 7f64222f4700 1 --2- 172.21.15.16:0/3240747723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f641c136f50 0x7f641c119750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:18.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.661+0000 7f64212f2700 1 -- 172.21.15.16:0/3240747723 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f641c131810 msgr2=0x7f641c118cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:18.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.661+0000 7f64212f2700 1 --2- 172.21.15.16:0/3240747723 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f641c131810 0x7f641c118cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:18.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.661+0000 7f64212f2700 1 -- 172.21.15.16:0/3240747723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f641c136f50 msgr2=0x7f641c119750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:18.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.661+0000 7f64212f2700 1 --2- 172.21.15.16:0/3240747723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f641c136f50 0x7f641c119750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:18.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.661+0000 7f64212f2700 1 -- 172.21.15.16:0/3240747723 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6414009a70 con 0x7f641c134630 2023-09-30T22:33:18.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.661+0000 7f64222f4700 1 --2- 172.21.15.16:0/3240747723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f641c136f50 0x7f641c119750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:33:18.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.661+0000 7f64212f2700 1 --2- 172.21.15.16:0/3240747723 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f641c134630 0x7f641c119210 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f640c00e810 tx=0x7f640c00ebd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:18.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.661+0000 7f64127fc700 1 -- 172.21.15.16:0/3240747723 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640c013610 con 0x7f641c134630 2023-09-30T22:33:18.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.661+0000 7f64127fc700 1 -- 172.21.15.16:0/3240747723 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f640c013770 con 0x7f641c134630 2023-09-30T22:33:18.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.661+0000 7f6423d57700 1 -- 172.21.15.16:0/3240747723 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f641c12ea20 con 0x7f641c134630 2023-09-30T22:33:18.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.661+0000 7f6423d57700 1 -- 172.21.15.16:0/3240747723 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f641c11c390 con 0x7f641c134630 2023-09-30T22:33:18.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.662+0000 7f64127fc700 1 -- 172.21.15.16:0/3240747723 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640c013b20 con 0x7f641c134630 2023-09-30T22:33:18.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.662+0000 7fe027356700 1 Processor -- start 2023-09-30T22:33:18.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.663+0000 7fe027356700 1 -- start start 2023-09-30T22:33:18.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.663+0000 7fe027356700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe0200a0a30 0x7fe0200a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:18.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.663+0000 7fe027356700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe020138d80 0x7fe02013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:18.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.663+0000 7fe027356700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0200a2a70 con 0x7fe02009d630 2023-09-30T22:33:18.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.663+0000 7fe027356700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0200a01d0 con 0x7fe0200a0a30 2023-09-30T22:33:18.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.663+0000 7fe027356700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe02009e2b0 con 0x7fe020138d80 2023-09-30T22:33:18.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.663+0000 7f64127fc700 1 -- 172.21.15.16:0/3240747723 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f640c013c80 con 0x7f641c134630 2023-09-30T22:33:18.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.663+0000 7f6423d57700 1 -- 172.21.15.16:0/3240747723 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64040052e0 con 0x7f641c134630 2023-09-30T22:33:18.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.663+0000 7f64127fc700 1 -- 172.21.15.16:0/3240747723 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f640c03f440 con 0x7f641c134630 2023-09-30T22:33:18.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.664+0000 7fe0248f1700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe02009d630 0x7fe0200a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54666/0 (socket says 172.21.15.16:54666) 2023-09-30T22:33:18.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.664+0000 7fe0248f1700 1 -- 172.21.15.16:0/1875597511 learned_addr learned my addr 172.21.15.16:0/1875597511 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:18.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.665+0000 7fe0258f3700 1 --2- 172.21.15.16:0/1875597511 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe020138d80 0x7fe02013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:18.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.665+0000 7fe0250f2700 1 --2- 172.21.15.16:0/1875597511 >> v2:172.21.15.16:3300/0 conn(0x7fe0200a0a30 0x7fe0200a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:18.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.665+0000 7fe0258f3700 1 -- 172.21.15.16:0/1875597511 >> v2:172.21.15.16:3300/0 conn(0x7fe0200a0a30 msgr2=0x7fe0200a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:18.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.665+0000 7fe0258f3700 1 --2- 172.21.15.16:0/1875597511 >> v2:172.21.15.16:3300/0 conn(0x7fe0200a0a30 0x7fe0200a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:18.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.665+0000 7fe0258f3700 1 -- 172.21.15.16:0/1875597511 >> v1:172.21.15.16:6789/0 conn(0x7fe02009d630 legacy=0x7fe0200a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:18.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.665+0000 7fe0258f3700 1 -- 172.21.15.16:0/1875597511 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe02009dc50 con 0x7fe020138d80 2023-09-30T22:33:18.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.665+0000 7fe0258f3700 1 --2- 172.21.15.16:0/1875597511 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe020138d80 0x7fe02013b160 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe018009d90 tx=0x7fe01800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=483122f603194dc2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:18.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.665+0000 7fe017fff700 1 -- 172.21.15.16:0/1875597511 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01800e070 con 0x7fe020138d80 2023-09-30T22:33:18.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.666+0000 7fe017fff700 1 -- 172.21.15.16:0/1875597511 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe01800bbd0 con 0x7fe020138d80 2023-09-30T22:33:18.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.666+0000 7fe017fff700 1 -- 172.21.15.16:0/1875597511 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe018012430 con 0x7fe020138d80 2023-09-30T22:33:18.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.666+0000 7fe027356700 1 -- 172.21.15.16:0/1875597511 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe020138d80 msgr2=0x7fe02013b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:18.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.666+0000 7fe027356700 1 --2- 172.21.15.16:0/1875597511 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe020138d80 0x7fe02013b160 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe018009d90 tx=0x7fe01800b330 comp rx=0 tx=0).stop 2023-09-30T22:33:18.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.666+0000 7fe027356700 1 -- 172.21.15.16:0/1875597511 shutdown_connections 2023-09-30T22:33:18.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.666+0000 7fe027356700 1 --2- 172.21.15.16:0/1875597511 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe020138d80 0x7fe02013b160 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:18.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.666+0000 7fe027356700 1 --2- 172.21.15.16:0/1875597511 >> v2:172.21.15.16:3300/0 conn(0x7fe0200a0a30 0x7fe0200a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:18.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.666+0000 7fe027356700 1 -- 172.21.15.16:0/1875597511 >> 172.21.15.16:0/1875597511 conn(0x7fe0200a3050 msgr2=0x7fe0200a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:18.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.667+0000 7fe027356700 1 -- 172.21.15.16:0/1875597511 shutdown_connections 2023-09-30T22:33:18.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.667+0000 7fe027356700 1 -- 172.21.15.16:0/1875597511 wait complete. 2023-09-30T22:33:18.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.668+0000 7fe027356700 1 Processor -- start 2023-09-30T22:33:18.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.668+0000 7fe027356700 1 -- start start 2023-09-30T22:33:18.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.668+0000 7fe027356700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe02009d630 0x7fe020137fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:18.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.668+0000 7fe027356700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe0200a0a30 0x7fe020138510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:18.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.668+0000 7fe027356700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe020138d80 0x7fe02006db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:18.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.668+0000 7fe027356700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02006ef30 con 0x7fe020138d80 2023-09-30T22:33:18.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.668+0000 7fe027356700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe020071d40 con 0x7fe02009d630 2023-09-30T22:33:18.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.669+0000 7fe0250f2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe02009d630 0x7fe020137fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:18.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.669+0000 7fe0248f1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe0200a0a30 0x7fe020138510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:18.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.669+0000 7fe0250f2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe02009d630 0x7fe020137fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54460/0 (socket says 172.21.15.16:54460) 2023-09-30T22:33:18.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.669+0000 7fe0250f2700 1 -- 172.21.15.16:0/2257658484 learned_addr learned my addr 172.21.15.16:0/2257658484 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:18.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.669+0000 7fe0258f3700 1 --2- 172.21.15.16:0/2257658484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe020138d80 0x7fe02006db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:18.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.669+0000 7fe027356700 1 -- 172.21.15.16:0/2257658484 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe020071500 con 0x7fe0200a0a30 2023-09-30T22:33:18.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.669+0000 7fe0248f1700 1 -- 172.21.15.16:0/2257658484 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe02009d630 msgr2=0x7fe020137fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:18.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.669+0000 7fe0248f1700 1 --2- 172.21.15.16:0/2257658484 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe02009d630 0x7fe020137fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:18.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.669+0000 7fe0248f1700 1 -- 172.21.15.16:0/2257658484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe020138d80 msgr2=0x7fe02006db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:18.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.669+0000 7fe0248f1700 1 --2- 172.21.15.16:0/2257658484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe020138d80 0x7fe02006db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:18.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.669+0000 7fe0248f1700 1 -- 172.21.15.16:0/2257658484 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe018009a70 con 0x7fe0200a0a30 2023-09-30T22:33:18.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.669+0000 7fe0250f2700 1 --2- 172.21.15.16:0/2257658484 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe02009d630 0x7fe020137fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:18.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.669+0000 7fe0258f3700 1 --2- 172.21.15.16:0/2257658484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe020138d80 0x7fe02006db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:33:18.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.670+0000 7fe0248f1700 1 --2- 172.21.15.16:0/2257658484 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe0200a0a30 0x7fe020138510 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe01000c480 tx=0x7fe01000c790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:18.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.672+0000 7fe015ffb700 1 -- 172.21.15.16:0/2257658484 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe010005b70 con 0x7fe0200a0a30 2023-09-30T22:33:18.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.672+0000 7fe015ffb700 1 -- 172.21.15.16:0/2257658484 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe010005cd0 con 0x7fe0200a0a30 2023-09-30T22:33:18.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.672+0000 7fe027356700 1 -- 172.21.15.16:0/2257658484 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe02006f690 con 0x7fe0200a0a30 2023-09-30T22:33:18.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.673+0000 7fe027356700 1 -- 172.21.15.16:0/2257658484 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0200a0410 con 0x7fe0200a0a30 2023-09-30T22:33:18.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.673+0000 7f64127fc700 1 -- 172.21.15.16:0/3240747723 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f640c012370 con 0x7f641c134630 2023-09-30T22:33:18.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.673+0000 7fe015ffb700 1 -- 172.21.15.16:0/2257658484 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0100072d0 con 0x7fe0200a0a30 2023-09-30T22:33:18.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.675+0000 7fe027356700 1 -- 172.21.15.16:0/2257658484 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe02013b620 con 0x7fe0200a0a30 2023-09-30T22:33:18.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.675+0000 7fe015ffb700 1 -- 172.21.15.16:0/2257658484 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe010007430 con 0x7fe0200a0a30 2023-09-30T22:33:18.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.676+0000 7fe015ffb700 1 -- 172.21.15.16:0/2257658484 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fe010060f20 con 0x7fe0200a0a30 2023-09-30T22:33:18.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.684+0000 7fe015ffb700 1 -- 172.21.15.16:0/2257658484 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe0100076e0 con 0x7fe0200a0a30 2023-09-30T22:33:18.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.745+0000 7fcbb42a2700 1 -- 172.21.15.16:0/3053313120 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fcbac1221c0 con 0x7fcbac06a770 2023-09-30T22:33:18.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.745+0000 7fcba2ffd700 1 -- 172.21.15.16:0/3053313120 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcbac1221c0 con 0x7fcbac06a770 2023-09-30T22:33:18.746 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:33:18.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.749+0000 7fcbb42a2700 1 -- 172.21.15.16:0/3053313120 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcbac06a770 msgr2=0x7fcbac09ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:18.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.749+0000 7fcbb42a2700 1 --2- 172.21.15.16:0/3053313120 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcbac06a770 0x7fcbac09ae60 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fcba8009c70 tx=0x7fcba8004750 comp rx=0 tx=0).stop 2023-09-30T22:33:18.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.750+0000 7fcbb42a2700 1 -- 172.21.15.16:0/3053313120 shutdown_connections 2023-09-30T22:33:18.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.750+0000 7fcbb42a2700 1 --2- 172.21.15.16:0/3053313120 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcbac131b40 0x7fcbac1c83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:18.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.750+0000 7fcbb42a2700 1 --2- 172.21.15.16:0/3053313120 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcbac06a770 0x7fcbac09ae60 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:18.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.750+0000 7fcbb42a2700 1 --2- 172.21.15.16:0/3053313120 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcbac06b080 0x7fcbac09b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:18.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.750+0000 7fcbb42a2700 1 -- 172.21.15.16:0/3053313120 >> 172.21.15.16:0/3053313120 conn(0x7fcbac066f50 msgr2=0x7fcbac1b8d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:18.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.750+0000 7fcbb42a2700 1 -- 172.21.15.16:0/3053313120 shutdown_connections 2023-09-30T22:33:18.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.750+0000 7fcbb42a2700 1 -- 172.21.15.16:0/3053313120 wait complete. 2023-09-30T22:33:18.770 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:33:18.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.860+0000 7fd9a1656700 1 -- 172.21.15.16:0/4201724022 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd99c074ea0 con 0x7fd99c0a1340 2023-09-30T22:33:18.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.860+0000 7fd9837fe700 1 -- 172.21.15.16:0/4201724022 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd990017020 con 0x7fd99c0a1340 2023-09-30T22:33:18.861 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:33:18.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.863+0000 7fd9a1656700 1 -- 172.21.15.16:0/4201724022 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd99c0a1340 msgr2=0x7fd99c122c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:18.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.863+0000 7fd9a1656700 1 --2- 172.21.15.16:0/4201724022 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd99c0a1340 0x7fd99c122c30 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd99000c520 tx=0x7fd99000c830 comp rx=0 tx=0).stop 2023-09-30T22:33:18.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.863+0000 7fd9a1656700 1 -- 172.21.15.16:0/4201724022 shutdown_connections 2023-09-30T22:33:18.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.863+0000 7fd9a1656700 1 --2- 172.21.15.16:0/4201724022 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd99c134170 0x7fd99c123170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:18.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.863+0000 7fd9a1656700 1 --2- 172.21.15.16:0/4201724022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd99c0a0a30 0x7fd99c1226f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:18.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.863+0000 7fd9a1656700 1 --2- 172.21.15.16:0/4201724022 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd99c0a1340 0x7fd99c122c30 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:18.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.863+0000 7fd9a1656700 1 -- 172.21.15.16:0/4201724022 >> 172.21.15.16:0/4201724022 conn(0x7fd99c0a6550 msgr2=0x7fd99c09edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:18.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.863+0000 7fd9a1656700 1 -- 172.21.15.16:0/4201724022 shutdown_connections 2023-09-30T22:33:18.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.863+0000 7fd9a1656700 1 -- 172.21.15.16:0/4201724022 wait complete. 2023-09-30T22:33:18.878 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:33:18.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.890+0000 7f6423d57700 1 -- 172.21.15.16:0/3240747723 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f6404005590 con 0x7f641c134630 2023-09-30T22:33:18.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.890+0000 7f64127fc700 1 -- 172.21.15.16:0/3240747723 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f640c01b3f0 con 0x7f641c134630 2023-09-30T22:33:18.891 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:33:18.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.892+0000 7fe027356700 1 -- 172.21.15.16:0/2257658484 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe020071180 con 0x7fe0200a0a30 2023-09-30T22:33:18.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.892+0000 7f6423d57700 1 -- 172.21.15.16:0/3240747723 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f641c134630 msgr2=0x7f641c119210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:18.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.892+0000 7f6423d57700 1 --2- 172.21.15.16:0/3240747723 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f641c134630 0x7f641c119210 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f640c00e810 tx=0x7f640c00ebd0 comp rx=0 tx=0).stop 2023-09-30T22:33:18.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.893+0000 7f6423d57700 1 -- 172.21.15.16:0/3240747723 shutdown_connections 2023-09-30T22:33:18.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.893+0000 7f6423d57700 1 --2- 172.21.15.16:0/3240747723 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f641c131810 0x7f641c118cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:18.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.893+0000 7f6423d57700 1 --2- 172.21.15.16:0/3240747723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f641c136f50 0x7f641c119750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:18.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.893+0000 7f6423d57700 1 --2- 172.21.15.16:0/3240747723 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f641c134630 0x7f641c119210 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:18.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.893+0000 7fe015ffb700 1 -- 172.21.15.16:0/2257658484 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe010039070 con 0x7fe0200a0a30 2023-09-30T22:33:18.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.893+0000 7f6423d57700 1 -- 172.21.15.16:0/3240747723 >> 172.21.15.16:0/3240747723 conn(0x7f641c0a2aa0 msgr2=0x7f641c09f000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:18.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.893+0000 7f6423d57700 1 -- 172.21.15.16:0/3240747723 shutdown_connections 2023-09-30T22:33:18.895 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:33:18.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.893+0000 7f6423d57700 1 -- 172.21.15.16:0/3240747723 wait complete. 2023-09-30T22:33:18.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.895+0000 7fe027356700 1 -- 172.21.15.16:0/2257658484 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe0200a0a30 msgr2=0x7fe020138510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:18.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.895+0000 7fe027356700 1 --2- 172.21.15.16:0/2257658484 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe0200a0a30 0x7fe020138510 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe01000c480 tx=0x7fe01000c790 comp rx=0 tx=0).stop 2023-09-30T22:33:18.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.895+0000 7fe027356700 1 -- 172.21.15.16:0/2257658484 shutdown_connections 2023-09-30T22:33:18.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.895+0000 7fe027356700 1 --2- 172.21.15.16:0/2257658484 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe02009d630 0x7fe020137fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:18.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.895+0000 7fe027356700 1 --2- 172.21.15.16:0/2257658484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe020138d80 0x7fe02006db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:18.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.895+0000 7fe027356700 1 --2- 172.21.15.16:0/2257658484 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe0200a0a30 0x7fe020138510 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:18.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.895+0000 7fe027356700 1 -- 172.21.15.16:0/2257658484 >> 172.21.15.16:0/2257658484 conn(0x7fe0200a3050 msgr2=0x7fe02009ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:18.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.895+0000 7fe027356700 1 -- 172.21.15.16:0/2257658484 shutdown_connections 2023-09-30T22:33:18.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:18.895+0000 7fe027356700 1 -- 172.21.15.16:0/2257658484 wait complete. 2023-09-30T22:33:18.908 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:33:18.910 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:33:19.770 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:33:19.879 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:33:19.909 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:33:19.910 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:33:19.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:19.917+0000 7fbe9162d700 1 Processor -- start 2023-09-30T22:33:19.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:19.917+0000 7fbe9162d700 1 -- start start 2023-09-30T22:33:19.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:19.918+0000 7fbe9162d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbe8c09d630 0x7fbe8c0a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:19.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:19.918+0000 7fbe9162d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbe8c138d80 0x7fbe8c13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:19.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:19.918+0000 7fbe9162d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe8c0a2a70 con 0x7fbe8c0a0a30 2023-09-30T22:33:19.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:19.918+0000 7fbe9162d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe8c0a01d0 con 0x7fbe8c138d80 2023-09-30T22:33:19.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:19.918+0000 7fbe9162d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8c09e2b0 con 0x7fbe8c09d630 2023-09-30T22:33:19.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:19.918+0000 7fbe8a7fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbe8c09d630 0x7fbe8c0a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:19.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:19.918+0000 7fbe8b7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbe8c138d80 0x7fbe8c13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:19.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:19.918+0000 7fbe8affd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fbe8c0a0a30 0x7fbe8c0a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54672/0 (socket says 172.21.15.16:54672) 2023-09-30T22:33:19.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:19.918+0000 7fbe8affd700 1 -- 172.21.15.16:0/3102902442 learned_addr learned my addr 172.21.15.16:0/3102902442 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:19.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:19.918+0000 7fbe8b7fe700 1 -- 172.21.15.16:0/3102902442 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbe8c09d630 msgr2=0x7fbe8c0a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:19.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:19.918+0000 7fbe8b7fe700 1 --2- 172.21.15.16:0/3102902442 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbe8c09d630 0x7fbe8c0a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:19.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:19.918+0000 7fbe8b7fe700 1 -- 172.21.15.16:0/3102902442 >> v1:172.21.15.16:6789/0 conn(0x7fbe8c0a0a30 legacy=0x7fbe8c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:19.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:19.918+0000 7fbe8b7fe700 1 -- 172.21.15.16:0/3102902442 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe8c09dc50 con 0x7fbe8c138d80 2023-09-30T22:33:19.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:19.918+0000 7fbe8a7fc700 1 --2- 172.21.15.16:0/3102902442 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbe8c09d630 0x7fbe8c0a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:19.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:19.919+0000 7fbe8b7fe700 1 --2- 172.21.15.16:0/3102902442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe8c138d80 0x7fbe8c13b160 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fbe84009d90 tx=0x7fbe8400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39f7fab9a6fc6687 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:19.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:19.919+0000 7fbe89ffb700 1 -- 172.21.15.16:0/3102902442 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8400e070 con 0x7fbe8c138d80 2023-09-30T22:33:19.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:19.919+0000 7fbe89ffb700 1 -- 172.21.15.16:0/3102902442 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe8400bb20 con 0x7fbe8c138d80 2023-09-30T22:33:19.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:19.919+0000 7fbe89ffb700 1 -- 172.21.15.16:0/3102902442 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe84012430 con 0x7fbe8c138d80 2023-09-30T22:33:19.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:19.920+0000 7fbe9162d700 1 -- 172.21.15.16:0/3102902442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe8c138d80 msgr2=0x7fbe8c13b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:19.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:19.920+0000 7fbe9162d700 1 --2- 172.21.15.16:0/3102902442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe8c138d80 0x7fbe8c13b160 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fbe84009d90 tx=0x7fbe8400b330 comp rx=0 tx=0).stop 2023-09-30T22:33:19.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:19.920+0000 7fbe9162d700 1 -- 172.21.15.16:0/3102902442 shutdown_connections 2023-09-30T22:33:19.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:19.920+0000 7fbe9162d700 1 --2- 172.21.15.16:0/3102902442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe8c138d80 0x7fbe8c13b160 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:19.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:19.920+0000 7fbe9162d700 1 --2- 172.21.15.16:0/3102902442 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbe8c09d630 0x7fbe8c0a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:19.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:19.920+0000 7fbe9162d700 1 -- 172.21.15.16:0/3102902442 >> 172.21.15.16:0/3102902442 conn(0x7fbe8c0a3050 msgr2=0x7fbe8c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:19.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:19.921+0000 7fbe9162d700 1 -- 172.21.15.16:0/3102902442 shutdown_connections 2023-09-30T22:33:19.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:19.921+0000 7fbe9162d700 1 -- 172.21.15.16:0/3102902442 wait complete. 2023-09-30T22:33:19.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:19.922+0000 7fbe9162d700 1 Processor -- start 2023-09-30T22:33:19.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:19.922+0000 7fbe9162d700 1 -- start start 2023-09-30T22:33:19.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:19.922+0000 7fbe9162d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe8c09d630 0x7fbe8c1339b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:19.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:19.922+0000 7fbe9162d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbe8c0a0a30 0x7fbe8c133ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:19.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:19.922+0000 7fbe9162d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbe8c138d80 0x7fbe8c134430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:19.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:19.922+0000 7fbe9162d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8c136550 con 0x7fbe8c09d630 2023-09-30T22:33:19.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:19.922+0000 7fbe9162d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8c1389b0 con 0x7fbe8c0a0a30 2023-09-30T22:33:19.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:19.922+0000 7fbe9162d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8c0a4d10 con 0x7fbe8c138d80 2023-09-30T22:33:19.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:19.923+0000 7fbe8a7fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbe8c0a0a30 0x7fbe8c133ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:19.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:19.923+0000 7fbe8a7fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbe8c0a0a30 0x7fbe8c133ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54468/0 (socket says 172.21.15.16:54468) 2023-09-30T22:33:19.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:19.923+0000 7fbe8a7fc700 1 -- 172.21.15.16:0/2130958929 learned_addr learned my addr 172.21.15.16:0/2130958929 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:19.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:19.923+0000 7fbe8affd700 1 --2- 172.21.15.16:0/2130958929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe8c09d630 0x7fbe8c1339b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:19.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:19.923+0000 7fbe8a7fc700 1 -- 172.21.15.16:0/2130958929 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbe8c138d80 msgr2=0x7fbe8c134430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:19.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:19.923+0000 7fbe8a7fc700 1 --2- 172.21.15.16:0/2130958929 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbe8c138d80 0x7fbe8c134430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:19.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:19.923+0000 7fbe8a7fc700 1 -- 172.21.15.16:0/2130958929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe8c09d630 msgr2=0x7fbe8c1339b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:19.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:19.923+0000 7fbe8a7fc700 1 --2- 172.21.15.16:0/2130958929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe8c09d630 0x7fbe8c1339b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:19.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:19.923+0000 7fbe8a7fc700 1 -- 172.21.15.16:0/2130958929 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe740094a0 con 0x7fbe8c0a0a30 2023-09-30T22:33:19.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:19.923+0000 7fbe8a7fc700 1 --2- 172.21.15.16:0/2130958929 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbe8c0a0a30 0x7fbe8c133ef0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fbe7400ebe0 tx=0x7fbe7400eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:19.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:19.924+0000 7fbe837fe700 1 -- 172.21.15.16:0/2130958929 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7400c910 con 0x7fbe8c0a0a30 2023-09-30T22:33:19.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:19.924+0000 7fbe837fe700 1 -- 172.21.15.16:0/2130958929 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe7400ca70 con 0x7fbe8c0a0a30 2023-09-30T22:33:19.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:19.924+0000 7fbe9162d700 1 -- 172.21.15.16:0/2130958929 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe84009a70 con 0x7fbe8c0a0a30 2023-09-30T22:33:19.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:19.924+0000 7fbe837fe700 1 -- 172.21.15.16:0/2130958929 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7400cde0 con 0x7fbe8c0a0a30 2023-09-30T22:33:19.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:19.924+0000 7fbe9162d700 1 -- 172.21.15.16:0/2130958929 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe8c0a0410 con 0x7fbe8c0a0a30 2023-09-30T22:33:19.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:19.925+0000 7fbe9162d700 1 -- 172.21.15.16:0/2130958929 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe8c135bf0 con 0x7fbe8c0a0a30 2023-09-30T22:33:19.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:19.926+0000 7fbe837fe700 1 -- 172.21.15.16:0/2130958929 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbe74013470 con 0x7fbe8c0a0a30 2023-09-30T22:33:19.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:19.926+0000 7fbe837fe700 1 -- 172.21.15.16:0/2130958929 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fbe740400a0 con 0x7fbe8c0a0a30 2023-09-30T22:33:19.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:19.932+0000 7fbe837fe700 1 -- 172.21.15.16:0/2130958929 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbe7401a360 con 0x7fbe8c0a0a30 2023-09-30T22:33:20.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.035+0000 7f9f01b6f700 1 Processor -- start 2023-09-30T22:33:20.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.035+0000 7f9f01b6f700 1 -- start start 2023-09-30T22:33:20.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.036+0000 7f9f01b6f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9efc09d6f0 0x7f9efc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:20.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.036+0000 7f9f01b6f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9efc09e410 0x7f9efc130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:20.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.036+0000 7f9f01b6f700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9efc09f130 con 0x7f9efc09e000 2023-09-30T22:33:20.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.036+0000 7f9f01b6f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9efc09f520 con 0x7f9efc09d6f0 2023-09-30T22:33:20.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.036+0000 7f9f01b6f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9efc09e930 con 0x7f9efc09e410 2023-09-30T22:33:20.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.036+0000 7f9efaffd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9efc09e000 0x7f9efc130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54678/0 (socket says 172.21.15.16:54678) 2023-09-30T22:33:20.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.036+0000 7f9efaffd700 1 -- 172.21.15.16:0/2422725897 learned_addr learned my addr 172.21.15.16:0/2422725897 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:20.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.036+0000 7f9efbfff700 1 --2- 172.21.15.16:0/2422725897 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9efc09e410 0x7f9efc130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:20.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.037+0000 7f9efb7fe700 1 --2- 172.21.15.16:0/2422725897 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9efc09d6f0 0x7f9efc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:20.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.037+0000 7f9efbfff700 1 -- 172.21.15.16:0/2422725897 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9efc09d6f0 msgr2=0x7f9efc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:20.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.037+0000 7f9efbfff700 1 --2- 172.21.15.16:0/2422725897 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9efc09d6f0 0x7f9efc09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:20.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.037+0000 7f9efa7fc700 1 -- 172.21.15.16:0/2422725897 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4143928875 0 0) 0x7f9efc09f130 con 0x7f9efc09e000 2023-09-30T22:33:20.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.037+0000 7f9efbfff700 1 -- 172.21.15.16:0/2422725897 >> v1:172.21.15.16:6789/0 conn(0x7f9efc09e000 legacy=0x7f9efc130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:20.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.037+0000 7f9efbfff700 1 -- 172.21.15.16:0/2422725897 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9efc09b2d0 con 0x7f9efc09e410 2023-09-30T22:33:20.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.037+0000 7f9efbfff700 1 --2- 172.21.15.16:0/2422725897 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9efc09e410 0x7f9efc130d70 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4009dc0 tx=0x7f9ef400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e12225878d16e8bc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:20.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.037+0000 7f9efa7fc700 1 -- 172.21.15.16:0/2422725897 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef400e070 con 0x7f9efc09e410 2023-09-30T22:33:20.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.037+0000 7f9efa7fc700 1 -- 172.21.15.16:0/2422725897 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ef400bbd0 con 0x7f9efc09e410 2023-09-30T22:33:20.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.037+0000 7f9efa7fc700 1 -- 172.21.15.16:0/2422725897 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef4012430 con 0x7f9efc09e410 2023-09-30T22:33:20.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.038+0000 7f9f01b6f700 1 -- 172.21.15.16:0/2422725897 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9efc09e410 msgr2=0x7f9efc130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:20.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.038+0000 7f9f01b6f700 1 --2- 172.21.15.16:0/2422725897 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9efc09e410 0x7f9efc130d70 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4009dc0 tx=0x7f9ef400b330 comp rx=0 tx=0).stop 2023-09-30T22:33:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.038+0000 7f9f01b6f700 1 -- 172.21.15.16:0/2422725897 shutdown_connections 2023-09-30T22:33:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.038+0000 7f9f01b6f700 1 --2- 172.21.15.16:0/2422725897 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9efc09e410 0x7f9efc130d70 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.038+0000 7f9f01b6f700 1 --2- 172.21.15.16:0/2422725897 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9efc09d6f0 0x7f9efc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.038+0000 7f9f01b6f700 1 -- 172.21.15.16:0/2422725897 >> 172.21.15.16:0/2422725897 conn(0x7f9efc066f50 msgr2=0x7f9efc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.038+0000 7f9f01b6f700 1 -- 172.21.15.16:0/2422725897 shutdown_connections 2023-09-30T22:33:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.038+0000 7f9f01b6f700 1 -- 172.21.15.16:0/2422725897 wait complete. 2023-09-30T22:33:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.040+0000 7f9f01b6f700 1 Processor -- start 2023-09-30T22:33:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.040+0000 7f9f01b6f700 1 -- start start 2023-09-30T22:33:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.041+0000 7f9f01b6f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9efc09d6f0 0x7f9efc122590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.041+0000 7f9f01b6f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9efc09e000 0x7f9efc122ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.041+0000 7f9f01b6f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9efc09e410 0x7f9efc123010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.041+0000 7f9f01b6f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9efc124800 con 0x7f9efc09e000 2023-09-30T22:33:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.041+0000 7f9f01b6f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9efc0a1fe0 con 0x7f9efc09d6f0 2023-09-30T22:33:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.041+0000 7f9f01b6f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9efc1260b0 con 0x7f9efc09e410 2023-09-30T22:33:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.041+0000 7f9efbfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9efc09e410 0x7f9efc123010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.041+0000 7f9efbfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9efc09e410 0x7f9efc123010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:36110/0 (socket says 172.21.15.16:36110) 2023-09-30T22:33:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.041+0000 7f9efbfff700 1 -- 172.21.15.16:0/4231647878 learned_addr learned my addr 172.21.15.16:0/4231647878 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.041+0000 7f9efaffd700 1 --2- 172.21.15.16:0/4231647878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9efc09e000 0x7f9efc122ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.041+0000 7f9efbfff700 1 -- 172.21.15.16:0/4231647878 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9efc09d6f0 msgr2=0x7f9efc122590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.041+0000 7f9efbfff700 1 --2- 172.21.15.16:0/4231647878 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9efc09d6f0 0x7f9efc122590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.041+0000 7f9efbfff700 1 -- 172.21.15.16:0/4231647878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9efc09e000 msgr2=0x7f9efc122ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:20.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.041+0000 7f9efbfff700 1 --2- 172.21.15.16:0/4231647878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9efc09e000 0x7f9efc122ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:20.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.041+0000 7f9efbfff700 1 -- 172.21.15.16:0/4231647878 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef4009a70 con 0x7f9efc09e410 2023-09-30T22:33:20.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.042+0000 7f9efbfff700 1 --2- 172.21.15.16:0/4231647878 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9efc09e410 0x7f9efc123010 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4014040 tx=0x7f9ef4004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:20.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.042+0000 7f9ee3fff700 1 -- 172.21.15.16:0/4231647878 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef4004930 con 0x7f9efc09e410 2023-09-30T22:33:20.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.042+0000 7f9ee3fff700 1 -- 172.21.15.16:0/4231647878 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ef4020070 con 0x7f9efc09e410 2023-09-30T22:33:20.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.042+0000 7f9f01b6f700 1 -- 172.21.15.16:0/4231647878 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9efc09c700 con 0x7f9efc09e410 2023-09-30T22:33:20.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.042+0000 7f9ee3fff700 1 -- 172.21.15.16:0/4231647878 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef40276c0 con 0x7f9efc09e410 2023-09-30T22:33:20.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.042+0000 7f9f01b6f700 1 -- 172.21.15.16:0/4231647878 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9efc124990 con 0x7f9efc09e410 2023-09-30T22:33:20.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.044+0000 7f9ee1ffb700 1 -- 172.21.15.16:0/4231647878 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ed40052e0 con 0x7f9efc09e410 2023-09-30T22:33:20.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.044+0000 7f9ee3fff700 1 -- 172.21.15.16:0/4231647878 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9ef4012430 con 0x7f9efc09e410 2023-09-30T22:33:20.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.044+0000 7f9ee3fff700 1 -- 172.21.15.16:0/4231647878 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f9ef4022070 con 0x7f9efc09e410 2023-09-30T22:33:20.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.053+0000 7f9ee3fff700 1 -- 172.21.15.16:0/4231647878 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9ef401b400 con 0x7f9efc09e410 2023-09-30T22:33:20.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.062+0000 7f841f1aa700 1 Processor -- start 2023-09-30T22:33:20.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.062+0000 7f841f1aa700 1 -- start start 2023-09-30T22:33:20.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.063+0000 7f841f1aa700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8418131810 0x7f8418131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:20.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.063+0000 7f841f1aa700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8418099a40 0x7f8418099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:20.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.063+0000 7f841f1aa700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f841809a3b0 0x7f8418138650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:20.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.063+0000 7f841f1aa700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84181313c0 con 0x7f8418131810 2023-09-30T22:33:20.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.063+0000 7f841f1aa700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f841806e8c0 con 0x7f841809a3b0 2023-09-30T22:33:20.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.063+0000 7f841f1aa700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8418071d20 con 0x7f8418099a40 2023-09-30T22:33:20.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.063+0000 7f8417fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8418099a40 0x7f8418099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:20.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.063+0000 7f8417fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8418099a40 0x7f8418099e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:36122/0 (socket says 172.21.15.16:36122) 2023-09-30T22:33:20.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.063+0000 7f8417fff700 1 -- 172.21.15.16:0/3443475920 learned_addr learned my addr 172.21.15.16:0/3443475920 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:20.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.063+0000 7f8417fff700 1 -- 172.21.15.16:0/3443475920 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f841809a3b0 msgr2=0x7f8418138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:20.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.063+0000 7f8417fff700 1 --2- 172.21.15.16:0/3443475920 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f841809a3b0 0x7f8418138650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:20.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.063+0000 7f8417fff700 1 -- 172.21.15.16:0/3443475920 >> v2:172.21.15.16:3300/0 conn(0x7f8418131810 msgr2=0x7f8418131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:20.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.063+0000 7f8417fff700 1 --2- 172.21.15.16:0/3443475920 >> v2:172.21.15.16:3300/0 conn(0x7f8418131810 0x7f8418131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:20.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.063+0000 7f8417fff700 1 -- 172.21.15.16:0/3443475920 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84180721a0 con 0x7f8418099a40 2023-09-30T22:33:20.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.063+0000 7f8417fff700 1 --2- 172.21.15.16:0/3443475920 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8418099a40 0x7f8418099e70 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f84040098f0 tx=0x7f8404009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18642c224f8d54b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:20.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.063+0000 7f84177fe700 1 -- 172.21.15.16:0/3443475920 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840400b8a0 con 0x7f8418099a40 2023-09-30T22:33:20.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.064+0000 7f84177fe700 1 -- 172.21.15.16:0/3443475920 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f840400ba00 con 0x7f8418099a40 2023-09-30T22:33:20.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.064+0000 7f84177fe700 1 -- 172.21.15.16:0/3443475920 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840400bd90 con 0x7f8418099a40 2023-09-30T22:33:20.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.064+0000 7f841f1aa700 1 -- 172.21.15.16:0/3443475920 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8418099a40 msgr2=0x7f8418099e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:20.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.064+0000 7f841f1aa700 1 --2- 172.21.15.16:0/3443475920 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8418099a40 0x7f8418099e70 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f84040098f0 tx=0x7f8404009c00 comp rx=0 tx=0).stop 2023-09-30T22:33:20.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.064+0000 7f841f1aa700 1 -- 172.21.15.16:0/3443475920 shutdown_connections 2023-09-30T22:33:20.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.064+0000 7f841f1aa700 1 --2- 172.21.15.16:0/3443475920 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f841809a3b0 0x7f8418138650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:20.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.064+0000 7f841f1aa700 1 --2- 172.21.15.16:0/3443475920 >> v2:172.21.15.16:3300/0 conn(0x7f8418131810 0x7f8418131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:20.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.064+0000 7f841f1aa700 1 --2- 172.21.15.16:0/3443475920 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8418099a40 0x7f8418099e70 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:20.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.064+0000 7f841f1aa700 1 -- 172.21.15.16:0/3443475920 >> 172.21.15.16:0/3443475920 conn(0x7f84180a2aa0 msgr2=0x7f84180a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:20.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.064+0000 7f841f1aa700 1 -- 172.21.15.16:0/3443475920 shutdown_connections 2023-09-30T22:33:20.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.064+0000 7f841f1aa700 1 -- 172.21.15.16:0/3443475920 wait complete. 2023-09-30T22:33:20.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.065+0000 7f841f1aa700 1 Processor -- start 2023-09-30T22:33:20.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.065+0000 7f841f1aa700 1 -- start start 2023-09-30T22:33:20.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.066+0000 7f841f1aa700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8418099a40 0x7f841812c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:20.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.066+0000 7f841f1aa700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f841809a3b0 0x7f841812c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:20.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.066+0000 7f841f1aa700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8418131810 0x7f841812ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:20.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.066+0000 7f841f1aa700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84181390e0 con 0x7f8418099a40 2023-09-30T22:33:20.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.066+0000 7f841f1aa700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84181395f0 con 0x7f8418131810 2023-09-30T22:33:20.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.066+0000 7f841f1aa700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f841806eb00 con 0x7f841809a3b0 2023-09-30T22:33:20.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.066+0000 7f8417fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f841809a3b0 0x7f841812c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:20.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.066+0000 7f8417fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f841809a3b0 0x7f841812c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:36126/0 (socket says 172.21.15.16:36126) 2023-09-30T22:33:20.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.066+0000 7f8417fff700 1 -- 172.21.15.16:0/910455657 learned_addr learned my addr 172.21.15.16:0/910455657 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:20.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.066+0000 7f841d747700 1 --2- 172.21.15.16:0/910455657 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8418131810 0x7f841812ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:20.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.067+0000 7f8417fff700 1 -- 172.21.15.16:0/910455657 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8418131810 msgr2=0x7f841812ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:20.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.067+0000 7f8417fff700 1 --2- 172.21.15.16:0/910455657 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8418131810 0x7f841812ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:20.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.067+0000 7f8417fff700 1 -- 172.21.15.16:0/910455657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8418099a40 msgr2=0x7f841812c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:20.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.067+0000 7f8417fff700 1 --2- 172.21.15.16:0/910455657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8418099a40 0x7f841812c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:20.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.067+0000 7f8417fff700 1 -- 172.21.15.16:0/910455657 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8404009550 con 0x7f841809a3b0 2023-09-30T22:33:20.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.067+0000 7f841d747700 1 --2- 172.21.15.16:0/910455657 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8418131810 0x7f841812ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:20.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.067+0000 7f8417fff700 1 --2- 172.21.15.16:0/910455657 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f841809a3b0 0x7f841812c970 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f8418071eb0 tx=0x7f8404004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:20.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.067+0000 7f84157fa700 1 -- 172.21.15.16:0/910455657 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8404004400 con 0x7f841809a3b0 2023-09-30T22:33:20.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.067+0000 7f84157fa700 1 -- 172.21.15.16:0/910455657 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f840401f070 con 0x7f841809a3b0 2023-09-30T22:33:20.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.067+0000 7f841f1aa700 1 -- 172.21.15.16:0/910455657 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f841812e030 con 0x7f841809a3b0 2023-09-30T22:33:20.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.067+0000 7f84157fa700 1 -- 172.21.15.16:0/910455657 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84040038e0 con 0x7f841809a3b0 2023-09-30T22:33:20.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.067+0000 7f841f1aa700 1 -- 172.21.15.16:0/910455657 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84181398b0 con 0x7f841809a3b0 2023-09-30T22:33:20.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.068+0000 7f84157fa700 1 -- 172.21.15.16:0/910455657 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8404021030 con 0x7f841809a3b0 2023-09-30T22:33:20.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.068+0000 7f841f1aa700 1 -- 172.21.15.16:0/910455657 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8418074ea0 con 0x7f841809a3b0 2023-09-30T22:33:20.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.069+0000 7f84157fa700 1 -- 172.21.15.16:0/910455657 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f8404040970 con 0x7f841809a3b0 2023-09-30T22:33:20.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.070+0000 7efed0ad6700 1 Processor -- start 2023-09-30T22:33:20.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.071+0000 7efed0ad6700 1 -- start start 2023-09-30T22:33:20.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.071+0000 7efed0ad6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efecc12cf60 0x7efecc06f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:20.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.071+0000 7efed0ad6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efecc12f8b0 0x7efecc12fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:20.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.071+0000 7efed0ad6700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efecc12cd90 con 0x7efecc130260 2023-09-30T22:33:20.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.071+0000 7efed0ad6700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efecc12d330 con 0x7efecc12f8b0 2023-09-30T22:33:20.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.071+0000 7efed0ad6700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efecc1306d0 con 0x7efecc12cf60 2023-09-30T22:33:20.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.071+0000 7efeca59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efecc12cf60 0x7efecc06f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:20.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.072+0000 7efeca59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efecc12cf60 0x7efecc06f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:36142/0 (socket says 172.21.15.16:36142) 2023-09-30T22:33:20.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.072+0000 7efeca59c700 1 -- 172.21.15.16:0/1304660615 learned_addr learned my addr 172.21.15.16:0/1304660615 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:20.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.072+0000 7efec9d9b700 1 --2- 172.21.15.16:0/1304660615 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efecc12f8b0 0x7efecc12fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:20.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.072+0000 7efeca59c700 1 -- 172.21.15.16:0/1304660615 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efecc12f8b0 msgr2=0x7efecc12fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:20.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.072+0000 7efeca59c700 1 --2- 172.21.15.16:0/1304660615 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efecc12f8b0 0x7efecc12fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:20.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.072+0000 7efeca59c700 1 -- 172.21.15.16:0/1304660615 >> v1:172.21.15.16:6789/0 conn(0x7efecc130260 legacy=0x7efecc136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:20.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.072+0000 7efeca59c700 1 -- 172.21.15.16:0/1304660615 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efecc071780 con 0x7efecc12cf60 2023-09-30T22:33:20.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.072+0000 7efeca59c700 1 --2- 172.21.15.16:0/1304660615 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efecc12cf60 0x7efecc06f5d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7efebc00bbc0 tx=0x7efebc00bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba6711859fc56f21 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:20.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.074+0000 7efec959a700 1 -- 172.21.15.16:0/1304660615 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efebc00dc20 con 0x7efecc12cf60 2023-09-30T22:33:20.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.074+0000 7efec959a700 1 -- 172.21.15.16:0/1304660615 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efebc00dd80 con 0x7efecc12cf60 2023-09-30T22:33:20.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.074+0000 7efec959a700 1 -- 172.21.15.16:0/1304660615 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efebc017630 con 0x7efecc12cf60 2023-09-30T22:33:20.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.074+0000 7f84157fa700 1 -- 172.21.15.16:0/910455657 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8404029020 con 0x7f841809a3b0 2023-09-30T22:33:20.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.075+0000 7efed0ad6700 1 -- 172.21.15.16:0/1304660615 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efecc12cf60 msgr2=0x7efecc06f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:20.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.075+0000 7efed0ad6700 1 --2- 172.21.15.16:0/1304660615 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efecc12cf60 0x7efecc06f5d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7efebc00bbc0 tx=0x7efebc00bed0 comp rx=0 tx=0).stop 2023-09-30T22:33:20.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.075+0000 7efed0ad6700 1 -- 172.21.15.16:0/1304660615 shutdown_connections 2023-09-30T22:33:20.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.075+0000 7efed0ad6700 1 --2- 172.21.15.16:0/1304660615 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efecc12cf60 0x7efecc06f5d0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:20.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.075+0000 7efed0ad6700 1 --2- 172.21.15.16:0/1304660615 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efecc12f8b0 0x7efecc12fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:20.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.075+0000 7efed0ad6700 1 -- 172.21.15.16:0/1304660615 >> 172.21.15.16:0/1304660615 conn(0x7efecc0a2aa0 msgr2=0x7efecc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:20.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.075+0000 7efed0ad6700 1 -- 172.21.15.16:0/1304660615 shutdown_connections 2023-09-30T22:33:20.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.075+0000 7efed0ad6700 1 -- 172.21.15.16:0/1304660615 wait complete. 2023-09-30T22:33:20.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.076+0000 7efed0ad6700 1 Processor -- start 2023-09-30T22:33:20.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.076+0000 7efed0ad6700 1 -- start start 2023-09-30T22:33:20.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.077+0000 7efed0ad6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efecc12cf60 0x7efecc1c53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:20.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.077+0000 7efed0ad6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efecc12f8b0 0x7efecc1c58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:20.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.077+0000 7efed0ad6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efecc130260 0x7efecc1c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:20.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.077+0000 7efed0ad6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efecc12f6d0 con 0x7efecc12f8b0 2023-09-30T22:33:20.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.077+0000 7efed0ad6700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efecc09e6a0 con 0x7efecc130260 2023-09-30T22:33:20.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.077+0000 7efed0ad6700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efecc1c5e20 con 0x7efecc12cf60 2023-09-30T22:33:20.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.077+0000 7efec9d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efecc12f8b0 0x7efecc1c58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:20.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.077+0000 7efeca59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efecc12cf60 0x7efecc1c53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:20.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.077+0000 7efec9d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efecc12f8b0 0x7efecc1c58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41428/0 (socket says 172.21.15.16:41428) 2023-09-30T22:33:20.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.077+0000 7efec9d9b700 1 -- 172.21.15.16:0/2181220603 learned_addr learned my addr 172.21.15.16:0/2181220603 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:20.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.077+0000 7efecad9d700 1 --2- 172.21.15.16:0/2181220603 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efecc130260 0x7efecc1c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:20.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.077+0000 7efeca59c700 1 -- 172.21.15.16:0/2181220603 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efecc130260 msgr2=0x7efecc1c83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:20.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.077+0000 7efeca59c700 1 --2- 172.21.15.16:0/2181220603 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efecc130260 0x7efecc1c83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:20.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.077+0000 7efeca59c700 1 -- 172.21.15.16:0/2181220603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efecc12f8b0 msgr2=0x7efecc1c58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:20.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.077+0000 7efeca59c700 1 --2- 172.21.15.16:0/2181220603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efecc12f8b0 0x7efecc1c58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:20.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.077+0000 7efeca59c700 1 -- 172.21.15.16:0/2181220603 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efebc00b7f0 con 0x7efecc12cf60 2023-09-30T22:33:20.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.078+0000 7efeca59c700 1 --2- 172.21.15.16:0/2181220603 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efecc12cf60 0x7efecc1c53a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7efebc018000 tx=0x7efebc004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:20.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.078+0000 7efebaffd700 1 -- 172.21.15.16:0/2181220603 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efebc004050 con 0x7efecc12cf60 2023-09-30T22:33:20.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.078+0000 7efebaffd700 1 -- 172.21.15.16:0/2181220603 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efebc0041d0 con 0x7efecc12cf60 2023-09-30T22:33:20.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.078+0000 7efed0ad6700 1 -- 172.21.15.16:0/2181220603 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efecc129fa0 con 0x7efecc12cf60 2023-09-30T22:33:20.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.078+0000 7efebaffd700 1 -- 172.21.15.16:0/2181220603 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efebc004580 con 0x7efecc12cf60 2023-09-30T22:33:20.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.078+0000 7efed0ad6700 1 -- 172.21.15.16:0/2181220603 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efecc09ec40 con 0x7efecc12cf60 2023-09-30T22:33:20.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.079+0000 7efebaffd700 1 -- 172.21.15.16:0/2181220603 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7efebc015070 con 0x7efecc12cf60 2023-09-30T22:33:20.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.079+0000 7efed0ad6700 1 -- 172.21.15.16:0/2181220603 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efecc074ea0 con 0x7efecc12cf60 2023-09-30T22:33:20.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.079+0000 7efebaffd700 1 -- 172.21.15.16:0/2181220603 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7efebc040b30 con 0x7efecc12cf60 2023-09-30T22:33:20.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.089+0000 7efebaffd700 1 -- 172.21.15.16:0/2181220603 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7efebc01e540 con 0x7efecc12cf60 2023-09-30T22:33:20.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.178+0000 7fbe9162d700 1 -- 172.21.15.16:0/2130958929 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fbe8c0747f0 con 0x7fbe8c0a0a30 2023-09-30T22:33:20.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.179+0000 7fbe837fe700 1 -- 172.21.15.16:0/2130958929 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbe74017070 con 0x7fbe8c0a0a30 2023-09-30T22:33:20.179 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:33:20.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.181+0000 7fbe9162d700 1 -- 172.21.15.16:0/2130958929 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbe8c0a0a30 msgr2=0x7fbe8c133ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:20.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.181+0000 7fbe9162d700 1 --2- 172.21.15.16:0/2130958929 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbe8c0a0a30 0x7fbe8c133ef0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fbe7400ebe0 tx=0x7fbe7400eef0 comp rx=0 tx=0).stop 2023-09-30T22:33:20.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.181+0000 7fbe9162d700 1 -- 172.21.15.16:0/2130958929 shutdown_connections 2023-09-30T22:33:20.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.181+0000 7fbe9162d700 1 --2- 172.21.15.16:0/2130958929 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbe8c138d80 0x7fbe8c134430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:20.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.181+0000 7fbe9162d700 1 --2- 172.21.15.16:0/2130958929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbe8c09d630 0x7fbe8c1339b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:20.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.181+0000 7fbe9162d700 1 --2- 172.21.15.16:0/2130958929 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbe8c0a0a30 0x7fbe8c133ef0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:20.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.181+0000 7fbe9162d700 1 -- 172.21.15.16:0/2130958929 >> 172.21.15.16:0/2130958929 conn(0x7fbe8c0a3050 msgr2=0x7fbe8c09ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:20.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.181+0000 7fbe9162d700 1 -- 172.21.15.16:0/2130958929 shutdown_connections 2023-09-30T22:33:20.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.181+0000 7fbe9162d700 1 -- 172.21.15.16:0/2130958929 wait complete. 2023-09-30T22:33:20.201 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:33:20.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.293+0000 7f841f1aa700 1 -- 172.21.15.16:0/910455657 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f8418139ad0 con 0x7f841809a3b0 2023-09-30T22:33:20.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.293+0000 7f84157fa700 1 -- 172.21.15.16:0/910455657 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8404003a40 con 0x7f841809a3b0 2023-09-30T22:33:20.294 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:33:20.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.296+0000 7f841f1aa700 1 -- 172.21.15.16:0/910455657 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f841809a3b0 msgr2=0x7f841812c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:20.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.296+0000 7f841f1aa700 1 --2- 172.21.15.16:0/910455657 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f841809a3b0 0x7f841812c970 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f8418071eb0 tx=0x7f8404004750 comp rx=0 tx=0).stop 2023-09-30T22:33:20.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.296+0000 7f841f1aa700 1 -- 172.21.15.16:0/910455657 shutdown_connections 2023-09-30T22:33:20.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.296+0000 7f841f1aa700 1 --2- 172.21.15.16:0/910455657 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8418131810 0x7f841812ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:20.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.296+0000 7f841f1aa700 1 --2- 172.21.15.16:0/910455657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8418099a40 0x7f841812c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:20.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.296+0000 7f841f1aa700 1 --2- 172.21.15.16:0/910455657 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f841809a3b0 0x7f841812c970 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:20.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.296+0000 7f841f1aa700 1 -- 172.21.15.16:0/910455657 >> 172.21.15.16:0/910455657 conn(0x7f84180a2aa0 msgr2=0x7f84180a0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:20.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.296+0000 7f841f1aa700 1 -- 172.21.15.16:0/910455657 shutdown_connections 2023-09-30T22:33:20.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.296+0000 7f841f1aa700 1 -- 172.21.15.16:0/910455657 wait complete. 2023-09-30T22:33:20.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.298+0000 7f9ee1ffb700 1 -- 172.21.15.16:0/4231647878 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f9ed4005590 con 0x7f9efc09e410 2023-09-30T22:33:20.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.299+0000 7f9ee3fff700 1 -- 172.21.15.16:0/4231647878 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9ef402c070 con 0x7f9efc09e410 2023-09-30T22:33:20.299 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:33:20.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.300+0000 7efed0ad6700 1 -- 172.21.15.16:0/2181220603 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7efecc074ea0 con 0x7efecc12cf60 2023-09-30T22:33:20.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.301+0000 7efebaffd700 1 -- 172.21.15.16:0/2181220603 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7efebc028a40 con 0x7efecc12cf60 2023-09-30T22:33:20.301 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:33:20.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.301+0000 7f9ee1ffb700 1 -- 172.21.15.16:0/4231647878 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9efc09e410 msgr2=0x7f9efc123010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:20.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.301+0000 7f9ee1ffb700 1 --2- 172.21.15.16:0/4231647878 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9efc09e410 0x7f9efc123010 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4014040 tx=0x7f9ef4004ad0 comp rx=0 tx=0).stop 2023-09-30T22:33:20.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.301+0000 7f9ee1ffb700 1 -- 172.21.15.16:0/4231647878 shutdown_connections 2023-09-30T22:33:20.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.301+0000 7f9ee1ffb700 1 --2- 172.21.15.16:0/4231647878 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9efc09d6f0 0x7f9efc122590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:20.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.301+0000 7f9ee1ffb700 1 --2- 172.21.15.16:0/4231647878 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9efc09e410 0x7f9efc123010 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:20.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.301+0000 7f9ee1ffb700 1 --2- 172.21.15.16:0/4231647878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9efc09e000 0x7f9efc122ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:20.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.301+0000 7f9ee1ffb700 1 -- 172.21.15.16:0/4231647878 >> 172.21.15.16:0/4231647878 conn(0x7f9efc066f50 msgr2=0x7f9efc09fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:20.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.302+0000 7f9ee1ffb700 1 -- 172.21.15.16:0/4231647878 shutdown_connections 2023-09-30T22:33:20.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.302+0000 7f9ee1ffb700 1 -- 172.21.15.16:0/4231647878 wait complete. 2023-09-30T22:33:20.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.304+0000 7efed0ad6700 1 -- 172.21.15.16:0/2181220603 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efecc12cf60 msgr2=0x7efecc1c53a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:20.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.304+0000 7efed0ad6700 1 --2- 172.21.15.16:0/2181220603 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efecc12cf60 0x7efecc1c53a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7efebc018000 tx=0x7efebc004750 comp rx=0 tx=0).stop 2023-09-30T22:33:20.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.304+0000 7efed0ad6700 1 -- 172.21.15.16:0/2181220603 shutdown_connections 2023-09-30T22:33:20.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.304+0000 7efed0ad6700 1 --2- 172.21.15.16:0/2181220603 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efecc130260 0x7efecc1c83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:20.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.304+0000 7efed0ad6700 1 --2- 172.21.15.16:0/2181220603 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efecc12cf60 0x7efecc1c53a0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:20.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.304+0000 7efed0ad6700 1 --2- 172.21.15.16:0/2181220603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efecc12f8b0 0x7efecc1c58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:20.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.304+0000 7efed0ad6700 1 -- 172.21.15.16:0/2181220603 >> 172.21.15.16:0/2181220603 conn(0x7efecc0a2aa0 msgr2=0x7efecc06e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:20.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.304+0000 7efed0ad6700 1 -- 172.21.15.16:0/2181220603 shutdown_connections 2023-09-30T22:33:20.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:20.304+0000 7efed0ad6700 1 -- 172.21.15.16:0/2181220603 wait complete. 2023-09-30T22:33:20.313 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:33:20.318 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:33:20.320 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:33:21.202 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:33:21.313 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:33:21.319 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:33:21.321 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:33:21.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.350+0000 7f76d2e9d700 1 Processor -- start 2023-09-30T22:33:21.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.350+0000 7f76d2e9d700 1 -- start start 2023-09-30T22:33:21.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.351+0000 7f76d2e9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f76cc09d6f0 0x7f76cc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:21.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.351+0000 7f76d2e9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f76cc09e000 0x7f76cc130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:21.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.351+0000 7f76d2e9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f76cc09e410 0x7f76cc130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:21.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.351+0000 7f76d2e9d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76cc09f520 con 0x7f76cc09e410 2023-09-30T22:33:21.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.351+0000 7f76d2e9d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76cc09f130 con 0x7f76cc09d6f0 2023-09-30T22:33:21.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.351+0000 7f76d2e9d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f76cc09e930 con 0x7f76cc09e000 2023-09-30T22:33:21.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.351+0000 7f76cbfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f76cc09e000 0x7f76cc130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:21.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.351+0000 7f76d143a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f76cc09e410 0x7f76cc130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:21.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.351+0000 7f76d0c39700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f76cc09d6f0 0x7f76cc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:21.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.351+0000 7f76cbfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f76cc09e000 0x7f76cc130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:36152/0 (socket says 172.21.15.16:36152) 2023-09-30T22:33:21.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.351+0000 7f76d0c39700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f76cc09d6f0 0x7f76cc09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54536/0 (socket says 172.21.15.16:54536) 2023-09-30T22:33:21.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.351+0000 7f76d143a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f76cc09e410 0x7f76cc130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41434/0 (socket says 172.21.15.16:41434) 2023-09-30T22:33:21.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.351+0000 7f76cbfff700 1 -- 172.21.15.16:0/3456249337 learned_addr learned my addr 172.21.15.16:0/3456249337 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:21.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.351+0000 7f76cbfff700 1 -- 172.21.15.16:0/3456249337 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f76cc09d6f0 msgr2=0x7f76cc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:21.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.351+0000 7f76cbfff700 1 --2- 172.21.15.16:0/3456249337 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f76cc09d6f0 0x7f76cc09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:21.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.351+0000 7f76cbfff700 1 -- 172.21.15.16:0/3456249337 >> v2:172.21.15.16:3300/0 conn(0x7f76cc09e410 msgr2=0x7f76cc130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:21.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.351+0000 7f76cbfff700 1 --2- 172.21.15.16:0/3456249337 >> v2:172.21.15.16:3300/0 conn(0x7f76cc09e410 0x7f76cc130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:21.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.351+0000 7f76cbfff700 1 -- 172.21.15.16:0/3456249337 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76cc09b2d0 con 0x7f76cc09e000 2023-09-30T22:33:21.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.351+0000 7f76cbfff700 1 --2- 172.21.15.16:0/3456249337 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f76cc09e000 0x7f76cc130660 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f76b8009920 tx=0x7f76b8009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c9a3ac230a8efe7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:21.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.352+0000 7f76cb7fe700 1 -- 172.21.15.16:0/3456249337 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b8011830 con 0x7f76cc09e000 2023-09-30T22:33:21.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.352+0000 7f76cb7fe700 1 -- 172.21.15.16:0/3456249337 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76b8011990 con 0x7f76cc09e000 2023-09-30T22:33:21.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.352+0000 7f76cb7fe700 1 -- 172.21.15.16:0/3456249337 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b8011d20 con 0x7f76cc09e000 2023-09-30T22:33:21.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.353+0000 7f76d2e9d700 1 -- 172.21.15.16:0/3456249337 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f76cc09e000 msgr2=0x7f76cc130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:21.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.353+0000 7f76d2e9d700 1 --2- 172.21.15.16:0/3456249337 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f76cc09e000 0x7f76cc130660 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f76b8009920 tx=0x7f76b8009c30 comp rx=0 tx=0).stop 2023-09-30T22:33:21.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.353+0000 7f76d2e9d700 1 -- 172.21.15.16:0/3456249337 shutdown_connections 2023-09-30T22:33:21.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.353+0000 7f76d2e9d700 1 --2- 172.21.15.16:0/3456249337 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f76cc09d6f0 0x7f76cc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:21.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.353+0000 7f76d2e9d700 1 --2- 172.21.15.16:0/3456249337 >> v2:172.21.15.16:3300/0 conn(0x7f76cc09e410 0x7f76cc130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:21.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.353+0000 7f76d2e9d700 1 --2- 172.21.15.16:0/3456249337 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f76cc09e000 0x7f76cc130660 secure :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f76b8009920 tx=0x7f76b8009c30 comp rx=0 tx=0).stop 2023-09-30T22:33:21.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.353+0000 7f76d2e9d700 1 -- 172.21.15.16:0/3456249337 >> 172.21.15.16:0/3456249337 conn(0x7f76cc066f50 msgr2=0x7f76cc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:21.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.354+0000 7f76d2e9d700 1 -- 172.21.15.16:0/3456249337 shutdown_connections 2023-09-30T22:33:21.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.354+0000 7f76d2e9d700 1 -- 172.21.15.16:0/3456249337 wait complete. 2023-09-30T22:33:21.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.355+0000 7f76d2e9d700 1 Processor -- start 2023-09-30T22:33:21.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.356+0000 7f76d2e9d700 1 -- start start 2023-09-30T22:33:21.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.356+0000 7f76d2e9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f76cc09d6f0 0x7f76cc1231e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:21.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.356+0000 7f76d2e9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76cc09e410 0x7f76cc123720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:21.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.356+0000 7f76d2e9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f76cc1c8560 0x7f76cc1c89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:21.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.356+0000 7f76d2e9d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76cc125040 con 0x7f76cc09e410 2023-09-30T22:33:21.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.356+0000 7f76d2e9d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76cc09c700 con 0x7f76cc09d6f0 2023-09-30T22:33:21.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.356+0000 7f76d2e9d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f76cc122db0 con 0x7f76cc1c8560 2023-09-30T22:33:21.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.356+0000 7f76cbfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76cc09e410 0x7f76cc123720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:21.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.356+0000 7f76cbfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76cc09e410 0x7f76cc123720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41444/0 (socket says 172.21.15.16:41444) 2023-09-30T22:33:21.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.356+0000 7f76cbfff700 1 -- 172.21.15.16:0/885962984 learned_addr learned my addr 172.21.15.16:0/885962984 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:21.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.357+0000 7f76d0c39700 1 --2- 172.21.15.16:0/885962984 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f76cc09d6f0 0x7f76cc1231e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:21.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.357+0000 7f76cbfff700 1 -- 172.21.15.16:0/885962984 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f76cc1c8560 msgr2=0x7f76cc1c89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:21.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.357+0000 7f76cbfff700 1 --2- 172.21.15.16:0/885962984 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f76cc1c8560 0x7f76cc1c89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:21.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.357+0000 7f76cbfff700 1 -- 172.21.15.16:0/885962984 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f76cc09d6f0 msgr2=0x7f76cc1231e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:21.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.357+0000 7f76cbfff700 1 --2- 172.21.15.16:0/885962984 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f76cc09d6f0 0x7f76cc1231e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:21.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.357+0000 7f76cbfff700 1 -- 172.21.15.16:0/885962984 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76b8009550 con 0x7f76cc09e410 2023-09-30T22:33:21.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.357+0000 7f76cbfff700 1 --2- 172.21.15.16:0/885962984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76cc09e410 0x7f76cc123720 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f76b8009c70 tx=0x7f76b8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:21.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.357+0000 7f76c97fa700 1 -- 172.21.15.16:0/885962984 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b8004450 con 0x7f76cc09e410 2023-09-30T22:33:21.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.358+0000 7f76c97fa700 1 -- 172.21.15.16:0/885962984 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76b801f070 con 0x7f76cc09e410 2023-09-30T22:33:21.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.358+0000 7f76c97fa700 1 -- 172.21.15.16:0/885962984 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b80038e0 con 0x7f76cc09e410 2023-09-30T22:33:21.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.358+0000 7f76d2e9d700 1 -- 172.21.15.16:0/885962984 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76cc125c40 con 0x7f76cc09e410 2023-09-30T22:33:21.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.358+0000 7f76d2e9d700 1 -- 172.21.15.16:0/885962984 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76cc126970 con 0x7f76cc09e410 2023-09-30T22:33:21.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.359+0000 7f76d2e9d700 1 -- 172.21.15.16:0/885962984 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76cc074ea0 con 0x7f76cc09e410 2023-09-30T22:33:21.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.360+0000 7f76c97fa700 1 -- 172.21.15.16:0/885962984 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f76b8003a40 con 0x7f76cc09e410 2023-09-30T22:33:21.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.360+0000 7f76c97fa700 1 -- 172.21.15.16:0/885962984 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f76b8021070 con 0x7f76cc09e410 2023-09-30T22:33:21.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.366+0000 7f76c97fa700 1 -- 172.21.15.16:0/885962984 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f76b8030480 con 0x7f76cc09e410 2023-09-30T22:33:21.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.492+0000 7f7b0bd50700 1 Processor -- start 2023-09-30T22:33:21.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.492+0000 7f9fdf242700 1 Processor -- start 2023-09-30T22:33:21.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.492+0000 7f9fdf242700 1 -- start start 2023-09-30T22:33:21.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.493+0000 7f9fdf242700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9fd8131810 0x7f9fd8131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:21.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.493+0000 7f9fdf242700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9fd8099a40 0x7f9fd8099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:21.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.493+0000 7f9fdf242700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fd81313c0 con 0x7f9fd809a3b0 2023-09-30T22:33:21.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.493+0000 7f9fdf242700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd809b580 con 0x7f9fd8131810 2023-09-30T22:33:21.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.493+0000 7f9fdf242700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd806e8c0 con 0x7f9fd8099a40 2023-09-30T22:33:21.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.492+0000 7f7b0bd50700 1 -- start start 2023-09-30T22:33:21.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.492+0000 7f7b0bd50700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7b0409eca0 0x7f7b0413cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:21.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.492+0000 7f7b0bd50700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7b0406f050 0x7f7b0413f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:21.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.492+0000 7f7b0bd50700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b04070ee0 con 0x7f7b0406e570 2023-09-30T22:33:21.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.492+0000 7f7b0bd50700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b04071980 con 0x7f7b0409eca0 2023-09-30T22:33:21.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.493+0000 7f7b0bd50700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b04071480 con 0x7f7b0406f050 2023-09-30T22:33:21.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.493+0000 7f9fdd7df700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9fd809a3b0 0x7f9fd8138650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54714/0 (socket says 172.21.15.16:54714) 2023-09-30T22:33:21.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.493+0000 7f9fdd7df700 1 -- 172.21.15.16:0/283388971 learned_addr learned my addr 172.21.15.16:0/283388971 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:21.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.493+0000 7f9fd7fff700 1 --2- 172.21.15.16:0/283388971 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9fd8099a40 0x7f9fd8099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:21.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.493+0000 7f7b09aec700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7b0406e570 0x7f7b0406e940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54702/0 (socket says 172.21.15.16:54702) 2023-09-30T22:33:21.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.493+0000 7f7b09aec700 1 -- 172.21.15.16:0/2201342843 learned_addr learned my addr 172.21.15.16:0/2201342843 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:21.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.493+0000 7f7b092eb700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7b0409eca0 0x7f7b0413cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:21.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.493+0000 7f9fd7fff700 1 -- 172.21.15.16:0/283388971 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9fd8131810 msgr2=0x7f9fd8131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:21.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.493+0000 7f9fd7fff700 1 --2- 172.21.15.16:0/283388971 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9fd8131810 0x7f9fd8131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:21.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.493+0000 7f9fd7fff700 1 -- 172.21.15.16:0/283388971 >> v1:172.21.15.16:6789/0 conn(0x7f9fd809a3b0 legacy=0x7f9fd8138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:21.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.493+0000 7f9fd7fff700 1 -- 172.21.15.16:0/283388971 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd809a820 con 0x7f9fd8099a40 2023-09-30T22:33:21.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.493+0000 7f7b08aea700 1 -- 172.21.15.16:0/2201342843 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3627718578 0 0) 0x7f7b04070ee0 con 0x7f7b0406e570 2023-09-30T22:33:21.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.493+0000 7f7b08aea700 1 -- 172.21.15.16:0/2201342843 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7af00035b0 con 0x7f7b0406e570 2023-09-30T22:33:21.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.494+0000 7f7b092eb700 1 -- 172.21.15.16:0/2201342843 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7b0406f050 msgr2=0x7f7b0413f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:21.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.494+0000 7f7b092eb700 1 --2- 172.21.15.16:0/2201342843 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7b0406f050 0x7f7b0413f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:21.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.494+0000 7f7b092eb700 1 -- 172.21.15.16:0/2201342843 >> v1:172.21.15.16:6789/0 conn(0x7f7b0406e570 legacy=0x7f7b0406e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:21.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.494+0000 7f7b092eb700 1 -- 172.21.15.16:0/2201342843 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b04071b90 con 0x7f7b0409eca0 2023-09-30T22:33:21.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.494+0000 7f9fd7fff700 1 --2- 172.21.15.16:0/283388971 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9fd8099a40 0x7f9fd8099e70 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f9fc4009870 tx=0x7f9fc4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8856cf96d5545c65 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:21.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.494+0000 7f7b092eb700 1 --2- 172.21.15.16:0/2201342843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b0409eca0 0x7f7b0413cea0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f7af4009840 tx=0x7f7af4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d2d23ce886171f5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:21.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.494+0000 7f9fd77fe700 1 -- 172.21.15.16:0/283388971 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc4011830 con 0x7f9fd8099a40 2023-09-30T22:33:21.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.494+0000 7f7b08aea700 1 -- 172.21.15.16:0/2201342843 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af4011830 con 0x7f7b0409eca0 2023-09-30T22:33:21.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.494+0000 7f7b08aea700 1 -- 172.21.15.16:0/2201342843 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7af4011990 con 0x7f7b0409eca0 2023-09-30T22:33:21.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.494+0000 7f7b08aea700 1 -- 172.21.15.16:0/2201342843 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af4011d00 con 0x7f7b0409eca0 2023-09-30T22:33:21.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.494+0000 7f9fd77fe700 1 -- 172.21.15.16:0/283388971 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fc4011990 con 0x7f9fd8099a40 2023-09-30T22:33:21.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.494+0000 7f9fd77fe700 1 -- 172.21.15.16:0/283388971 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc4011d20 con 0x7f9fd8099a40 2023-09-30T22:33:21.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.495+0000 7f9fdf242700 1 -- 172.21.15.16:0/283388971 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9fd8099a40 msgr2=0x7f9fd8099e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:21.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.495+0000 7f9fdf242700 1 --2- 172.21.15.16:0/283388971 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9fd8099a40 0x7f9fd8099e70 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f9fc4009870 tx=0x7f9fc4009b80 comp rx=0 tx=0).stop 2023-09-30T22:33:21.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.495+0000 7f9fdf242700 1 -- 172.21.15.16:0/283388971 shutdown_connections 2023-09-30T22:33:21.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.495+0000 7f9fdf242700 1 --2- 172.21.15.16:0/283388971 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9fd8131810 0x7f9fd8131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:21.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.495+0000 7f9fdf242700 1 --2- 172.21.15.16:0/283388971 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9fd8099a40 0x7f9fd8099e70 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:21.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.495+0000 7f9fdf242700 1 -- 172.21.15.16:0/283388971 >> 172.21.15.16:0/283388971 conn(0x7f9fd80a2aa0 msgr2=0x7f9fd80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:21.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.495+0000 7f7b0bd50700 1 -- 172.21.15.16:0/2201342843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b0409eca0 msgr2=0x7f7b0413cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:21.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.495+0000 7f7b0bd50700 1 --2- 172.21.15.16:0/2201342843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b0409eca0 0x7f7b0413cea0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f7af4009840 tx=0x7f7af4009b50 comp rx=0 tx=0).stop 2023-09-30T22:33:21.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.495+0000 7f9fdf242700 1 -- 172.21.15.16:0/283388971 shutdown_connections 2023-09-30T22:33:21.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.495+0000 7f7b0bd50700 1 -- 172.21.15.16:0/2201342843 shutdown_connections 2023-09-30T22:33:21.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.495+0000 7f7b0bd50700 1 --2- 172.21.15.16:0/2201342843 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7b0406f050 0x7f7b0413f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:21.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.495+0000 7f7b0bd50700 1 --2- 172.21.15.16:0/2201342843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b0409eca0 0x7f7b0413cea0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:21.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.495+0000 7f7b0bd50700 1 -- 172.21.15.16:0/2201342843 >> 172.21.15.16:0/2201342843 conn(0x7f7b040a2aa0 msgr2=0x7f7b040a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:21.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.495+0000 7f7b0bd50700 1 -- 172.21.15.16:0/2201342843 shutdown_connections 2023-09-30T22:33:21.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.495+0000 7f7b0bd50700 1 -- 172.21.15.16:0/2201342843 wait complete. 2023-09-30T22:33:21.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.495+0000 7f9fdf242700 1 -- 172.21.15.16:0/283388971 wait complete. 2023-09-30T22:33:21.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.496+0000 7f9fdf242700 1 Processor -- start 2023-09-30T22:33:21.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.496+0000 7f9fdf242700 1 -- start start 2023-09-30T22:33:21.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.497+0000 7f7b0bd50700 1 Processor -- start 2023-09-30T22:33:21.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.497+0000 7f7b0bd50700 1 -- start start 2023-09-30T22:33:21.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.497+0000 7f9fdf242700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fd8099a40 0x7f9fd811b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:21.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.497+0000 7f9fdf242700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9fd809a3b0 0x7f9fd811bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:21.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.497+0000 7f9fdf242700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9fd8131810 0x7f9fd811c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:21.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.497+0000 7f9fdf242700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd811cd60 con 0x7f9fd8099a40 2023-09-30T22:33:21.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.497+0000 7f9fdf242700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd812ec30 con 0x7f9fd8131810 2023-09-30T22:33:21.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.497+0000 7f9fdf242700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd811da70 con 0x7f9fd809a3b0 2023-09-30T22:33:21.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.497+0000 7f9fdcfde700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fd8099a40 0x7f9fd811b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:21.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.497+0000 7f9fdcfde700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fd8099a40 0x7f9fd811b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41468/0 (socket says 172.21.15.16:41468) 2023-09-30T22:33:21.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.497+0000 7f9fdcfde700 1 -- 172.21.15.16:0/2015469307 learned_addr learned my addr 172.21.15.16:0/2015469307 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:21.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.497+0000 7f7b0bd50700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7b0406e570 0x7f7b0413cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:21.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.497+0000 7f7b0bd50700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b0406f050 0x7f7b0413d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:21.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.497+0000 7f7b0bd50700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7b04138360 0x7f7b0413d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:21.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.497+0000 7f7b0bd50700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0412d7e0 con 0x7f7b0406f050 2023-09-30T22:33:21.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.497+0000 7f7b0bd50700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b04070680 con 0x7f7b04138360 2023-09-30T22:33:21.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.497+0000 7f7b0bd50700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0412d520 con 0x7f7b0406e570 2023-09-30T22:33:21.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.498+0000 7f7b092eb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b0406f050 0x7f7b0413d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:21.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.498+0000 7f7b092eb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b0406f050 0x7f7b0413d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41484/0 (socket says 172.21.15.16:41484) 2023-09-30T22:33:21.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.498+0000 7f7b092eb700 1 -- 172.21.15.16:0/3892669773 learned_addr learned my addr 172.21.15.16:0/3892669773 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:21.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.498+0000 7f9fdd7df700 1 --2- 172.21.15.16:0/2015469307 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9fd8131810 0x7f9fd811c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:21.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.498+0000 7f9fdcfde700 1 -- 172.21.15.16:0/2015469307 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9fd809a3b0 msgr2=0x7f9fd811bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:21.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.498+0000 7f9fd7fff700 1 --2- 172.21.15.16:0/2015469307 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9fd809a3b0 0x7f9fd811bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:21.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.498+0000 7f9fdcfde700 1 --2- 172.21.15.16:0/2015469307 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9fd809a3b0 0x7f9fd811bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:21.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.498+0000 7f9fdcfde700 1 -- 172.21.15.16:0/2015469307 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9fd8131810 msgr2=0x7f9fd811c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:21.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.498+0000 7f9fdcfde700 1 --2- 172.21.15.16:0/2015469307 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9fd8131810 0x7f9fd811c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:21.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.498+0000 7f9fdcfde700 1 -- 172.21.15.16:0/2015469307 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc40094a0 con 0x7f9fd8099a40 2023-09-30T22:33:21.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.498+0000 7f9fd7fff700 1 --2- 172.21.15.16:0/2015469307 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9fd809a3b0 0x7f9fd811bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:33:21.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.498+0000 7f7b09aec700 1 --2- 172.21.15.16:0/3892669773 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7b0406e570 0x7f7b0413cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:21.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.498+0000 7f7b0a2ed700 1 --2- 172.21.15.16:0/3892669773 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7b04138360 0x7f7b0413d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:21.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.498+0000 7f7b092eb700 1 -- 172.21.15.16:0/3892669773 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7b0406e570 msgr2=0x7f7b0413cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:21.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.498+0000 7f7b092eb700 1 --2- 172.21.15.16:0/3892669773 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7b0406e570 0x7f7b0413cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:21.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.498+0000 7f7b092eb700 1 -- 172.21.15.16:0/3892669773 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7b04138360 msgr2=0x7f7b0413d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:21.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.498+0000 7f7b092eb700 1 --2- 172.21.15.16:0/3892669773 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7b04138360 0x7f7b0413d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:21.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.498+0000 7f7b092eb700 1 -- 172.21.15.16:0/3892669773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7af40094a0 con 0x7f7b0406f050 2023-09-30T22:33:21.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.498+0000 7f7b0a2ed700 1 --2- 172.21.15.16:0/3892669773 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7b04138360 0x7f7b0413d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:33:21.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.498+0000 7f9fdcfde700 1 --2- 172.21.15.16:0/2015469307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fd8099a40 0x7f9fd811b890 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc00b560 tx=0x7f9fcc00b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:21.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.498+0000 7f7b092eb700 1 --2- 172.21.15.16:0/3892669773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b0406f050 0x7f7b0413d0d0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f7af4000c00 tx=0x7f7af4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:21.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.498+0000 7f9fd57fa700 1 -- 172.21.15.16:0/2015469307 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc011830 con 0x7f9fd8099a40 2023-09-30T22:33:21.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.499+0000 7f9fd57fa700 1 -- 172.21.15.16:0/2015469307 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fcc011990 con 0x7f9fd8099a40 2023-09-30T22:33:21.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.499+0000 7f9fd57fa700 1 -- 172.21.15.16:0/2015469307 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc011d40 con 0x7f9fd8099a40 2023-09-30T22:33:21.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.499+0000 7f9fdf242700 1 -- 172.21.15.16:0/2015469307 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fd811f080 con 0x7f9fd8099a40 2023-09-30T22:33:21.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.499+0000 7f9fdf242700 1 -- 172.21.15.16:0/2015469307 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fd806eb00 con 0x7f9fd8099a40 2023-09-30T22:33:21.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.499+0000 7f7afa7fc700 1 -- 172.21.15.16:0/3892669773 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af40036a0 con 0x7f7b0406f050 2023-09-30T22:33:21.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.499+0000 7f7afa7fc700 1 -- 172.21.15.16:0/3892669773 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7af401f070 con 0x7f7b0406f050 2023-09-30T22:33:21.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.499+0000 7f7afa7fc700 1 -- 172.21.15.16:0/3892669773 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af4003a00 con 0x7f7b0406f050 2023-09-30T22:33:21.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.500+0000 7f7b0bd50700 1 -- 172.21.15.16:0/3892669773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b0413e840 con 0x7f7b0406f050 2023-09-30T22:33:21.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.500+0000 7f7b0bd50700 1 -- 172.21.15.16:0/3892669773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b0412c9b0 con 0x7f7b0406f050 2023-09-30T22:33:21.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.500+0000 7f9fdf242700 1 -- 172.21.15.16:0/2015469307 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fd8074ea0 con 0x7f9fd8099a40 2023-09-30T22:33:21.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.501+0000 7f7afa7fc700 1 -- 172.21.15.16:0/3892669773 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7af4003b60 con 0x7f7b0406f050 2023-09-30T22:33:21.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.501+0000 7f7b0bd50700 1 -- 172.21.15.16:0/3892669773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b04074ea0 con 0x7f7b0406f050 2023-09-30T22:33:21.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.501+0000 7f7afa7fc700 1 -- 172.21.15.16:0/3892669773 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f7af4021070 con 0x7f7b0406f050 2023-09-30T22:33:21.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.501+0000 7f9fd57fa700 1 -- 172.21.15.16:0/2015469307 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9fcc01a400 con 0x7f9fd8099a40 2023-09-30T22:33:21.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.501+0000 7f9fd57fa700 1 -- 172.21.15.16:0/2015469307 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f9fcc03f870 con 0x7f9fd8099a40 2023-09-30T22:33:21.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.504+0000 7ff6d446c700 1 Processor -- start 2023-09-30T22:33:21.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.504+0000 7ff6d446c700 1 -- start start 2023-09-30T22:33:21.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.505+0000 7ff6d446c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff6cc133a80 0x7ff6cc133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:21.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.505+0000 7ff6d446c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff6cc12ff20 0x7ff6cc130330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:21.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.505+0000 7ff6d446c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6cc06bc00 con 0x7ff6cc130870 2023-09-30T22:33:21.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.505+0000 7ff6d446c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6cc06c570 con 0x7ff6cc12ff20 2023-09-30T22:33:21.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.505+0000 7ff6d446c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff6cc0a0040 con 0x7ff6cc133a80 2023-09-30T22:33:21.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.505+0000 7ff6d2208700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff6cc133a80 0x7ff6cc133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:21.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.505+0000 7ff6d2208700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff6cc133a80 0x7ff6cc133e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:36192/0 (socket says 172.21.15.16:36192) 2023-09-30T22:33:21.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.505+0000 7ff6d2208700 1 -- 172.21.15.16:0/3516754375 learned_addr learned my addr 172.21.15.16:0/3516754375 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:21.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.505+0000 7ff6d1a07700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff6cc12ff20 0x7ff6cc130330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:21.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.505+0000 7ff6d2208700 1 -- 172.21.15.16:0/3516754375 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff6cc12ff20 msgr2=0x7ff6cc130330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:21.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.505+0000 7ff6d2208700 1 --2- 172.21.15.16:0/3516754375 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff6cc12ff20 0x7ff6cc130330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:21.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.505+0000 7ff6d2208700 1 -- 172.21.15.16:0/3516754375 >> v1:172.21.15.16:6789/0 conn(0x7ff6cc130870 legacy=0x7ff6cc13a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:21.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.505+0000 7ff6d2208700 1 -- 172.21.15.16:0/3516754375 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6cc133860 con 0x7ff6cc133a80 2023-09-30T22:33:21.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.505+0000 7ff6d1a07700 1 --2- 172.21.15.16:0/3516754375 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff6cc12ff20 0x7ff6cc130330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:21.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.505+0000 7ff6d2208700 1 --2- 172.21.15.16:0/3516754375 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff6cc133a80 0x7ff6cc133e50 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8009920 tx=0x7ff6c8009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b072a6422c3a9742 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:21.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.506+0000 7ff6d1206700 1 -- 172.21.15.16:0/3516754375 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c8004230 con 0x7ff6cc133a80 2023-09-30T22:33:21.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.506+0000 7ff6d1206700 1 -- 172.21.15.16:0/3516754375 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6c8004390 con 0x7ff6cc133a80 2023-09-30T22:33:21.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.506+0000 7ff6d1206700 1 -- 172.21.15.16:0/3516754375 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c80036a0 con 0x7ff6cc133a80 2023-09-30T22:33:21.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.506+0000 7ff6d446c700 1 -- 172.21.15.16:0/3516754375 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff6cc133a80 msgr2=0x7ff6cc133e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:21.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.506+0000 7ff6d446c700 1 --2- 172.21.15.16:0/3516754375 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff6cc133a80 0x7ff6cc133e50 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8009920 tx=0x7ff6c8009c30 comp rx=0 tx=0).stop 2023-09-30T22:33:21.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.506+0000 7ff6d446c700 1 -- 172.21.15.16:0/3516754375 shutdown_connections 2023-09-30T22:33:21.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.506+0000 7ff6d446c700 1 --2- 172.21.15.16:0/3516754375 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff6cc133a80 0x7ff6cc133e50 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:21.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.506+0000 7ff6d446c700 1 --2- 172.21.15.16:0/3516754375 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff6cc12ff20 0x7ff6cc130330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:21.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.506+0000 7ff6d446c700 1 -- 172.21.15.16:0/3516754375 >> 172.21.15.16:0/3516754375 conn(0x7ff6cc0a4210 msgr2=0x7ff6cc0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:21.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.506+0000 7ff6d446c700 1 -- 172.21.15.16:0/3516754375 shutdown_connections 2023-09-30T22:33:21.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.506+0000 7ff6d446c700 1 -- 172.21.15.16:0/3516754375 wait complete. 2023-09-30T22:33:21.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.507+0000 7ff6d446c700 1 Processor -- start 2023-09-30T22:33:21.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.508+0000 7ff6d446c700 1 -- start start 2023-09-30T22:33:21.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.508+0000 7ff6d446c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff6cc12ff20 0x7ff6cc09c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:21.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.508+0000 7ff6d446c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6cc130870 0x7ff6cc09c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:21.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.508+0000 7ff6d446c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff6cc133a80 0x7ff6cc09cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:21.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.508+0000 7ff6d446c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6cc11fab0 con 0x7ff6cc130870 2023-09-30T22:33:21.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.508+0000 7ff6d446c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6cc11fd60 con 0x7ff6cc133a80 2023-09-30T22:33:21.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.508+0000 7ff6d446c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff6cc12d750 con 0x7ff6cc12ff20 2023-09-30T22:33:21.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.508+0000 7ff6d1a07700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6cc130870 0x7ff6cc09c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:21.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.508+0000 7ff6d2a09700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff6cc133a80 0x7ff6cc09cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:21.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.508+0000 7ff6d1a07700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6cc130870 0x7ff6cc09c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41494/0 (socket says 172.21.15.16:41494) 2023-09-30T22:33:21.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.508+0000 7ff6d2208700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff6cc12ff20 0x7ff6cc09c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:21.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.508+0000 7ff6d1a07700 1 -- 172.21.15.16:0/1921656075 learned_addr learned my addr 172.21.15.16:0/1921656075 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:21.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.509+0000 7ff6d2208700 1 -- 172.21.15.16:0/1921656075 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff6cc133a80 msgr2=0x7ff6cc09cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:21.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.509+0000 7ff6d2208700 1 --2- 172.21.15.16:0/1921656075 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff6cc133a80 0x7ff6cc09cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:21.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.509+0000 7ff6d2208700 1 -- 172.21.15.16:0/1921656075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6cc130870 msgr2=0x7ff6cc09c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:21.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.509+0000 7ff6d2208700 1 --2- 172.21.15.16:0/1921656075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6cc130870 0x7ff6cc09c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:21.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.509+0000 7ff6d2208700 1 -- 172.21.15.16:0/1921656075 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6bc0094a0 con 0x7ff6cc12ff20 2023-09-30T22:33:21.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.509+0000 7ff6d1a07700 1 --2- 172.21.15.16:0/1921656075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6cc130870 0x7ff6cc09c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:21.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.509+0000 7ff6d2a09700 1 --2- 172.21.15.16:0/1921656075 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff6cc133a80 0x7ff6cc09cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:21.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.509+0000 7ff6d2208700 1 --2- 172.21.15.16:0/1921656075 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff6cc12ff20 0x7ff6cc09c480 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8015000 tx=0x7ff6c8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:21.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.509+0000 7ff6c2ffd700 1 -- 172.21.15.16:0/1921656075 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c8003df0 con 0x7ff6cc12ff20 2023-09-30T22:33:21.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.509+0000 7ff6c2ffd700 1 -- 172.21.15.16:0/1921656075 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6c8011630 con 0x7ff6cc12ff20 2023-09-30T22:33:21.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.509+0000 7ff6d446c700 1 -- 172.21.15.16:0/1921656075 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6c8009550 con 0x7ff6cc12ff20 2023-09-30T22:33:21.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.509+0000 7ff6c2ffd700 1 -- 172.21.15.16:0/1921656075 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c80119e0 con 0x7ff6cc12ff20 2023-09-30T22:33:21.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.509+0000 7ff6d446c700 1 -- 172.21.15.16:0/1921656075 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6cc11ffb0 con 0x7ff6cc12ff20 2023-09-30T22:33:21.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.510+0000 7ff6c2ffd700 1 -- 172.21.15.16:0/1921656075 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff6c8021070 con 0x7ff6cc12ff20 2023-09-30T22:33:21.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.510+0000 7ff6c2ffd700 1 -- 172.21.15.16:0/1921656075 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7ff6c8040890 con 0x7ff6cc12ff20 2023-09-30T22:33:21.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.510+0000 7ff6d446c700 1 -- 172.21.15.16:0/1921656075 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6cc074ea0 con 0x7ff6cc12ff20 2023-09-30T22:33:21.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.510+0000 7f7afa7fc700 1 -- 172.21.15.16:0/3892669773 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7af4010d50 con 0x7f7b0406f050 2023-09-30T22:33:21.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.515+0000 7ff6c2ffd700 1 -- 172.21.15.16:0/1921656075 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff6c801a460 con 0x7ff6cc12ff20 2023-09-30T22:33:21.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.519+0000 7f9fd57fa700 1 -- 172.21.15.16:0/2015469307 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9fcc00f8d0 con 0x7f9fd8099a40 2023-09-30T22:33:21.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.641+0000 7f76d2e9d700 1 -- 172.21.15.16:0/885962984 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f76cc1303d0 con 0x7f76cc09e410 2023-09-30T22:33:21.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.641+0000 7f76c97fa700 1 -- 172.21.15.16:0/885962984 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f76b8000e60 con 0x7f76cc09e410 2023-09-30T22:33:21.644 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:33:21.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.645+0000 7f76d2e9d700 1 -- 172.21.15.16:0/885962984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76cc09e410 msgr2=0x7f76cc123720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:21.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.645+0000 7f76d2e9d700 1 --2- 172.21.15.16:0/885962984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76cc09e410 0x7f76cc123720 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f76b8009c70 tx=0x7f76b8004750 comp rx=0 tx=0).stop 2023-09-30T22:33:21.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.645+0000 7f76d2e9d700 1 -- 172.21.15.16:0/885962984 shutdown_connections 2023-09-30T22:33:21.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.645+0000 7f76d2e9d700 1 --2- 172.21.15.16:0/885962984 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f76cc09d6f0 0x7f76cc1231e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:21.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.645+0000 7f76d2e9d700 1 --2- 172.21.15.16:0/885962984 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f76cc1c8560 0x7f76cc1c89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:21.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.645+0000 7f76d2e9d700 1 --2- 172.21.15.16:0/885962984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76cc09e410 0x7f76cc123720 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:21.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.645+0000 7f76d2e9d700 1 -- 172.21.15.16:0/885962984 >> 172.21.15.16:0/885962984 conn(0x7f76cc066f50 msgr2=0x7f76cc06a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:21.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.645+0000 7f76d2e9d700 1 -- 172.21.15.16:0/885962984 shutdown_connections 2023-09-30T22:33:21.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.645+0000 7f76d2e9d700 1 -- 172.21.15.16:0/885962984 wait complete. 2023-09-30T22:33:21.665 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:33:21.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.721+0000 7f7b0bd50700 1 -- 172.21.15.16:0/3892669773 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f7b0412d920 con 0x7f7b0406f050 2023-09-30T22:33:21.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.722+0000 7f7afa7fc700 1 -- 172.21.15.16:0/3892669773 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7af4015090 con 0x7f7b0406f050 2023-09-30T22:33:21.723 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:33:21.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.724+0000 7f7b0bd50700 1 -- 172.21.15.16:0/3892669773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b0406f050 msgr2=0x7f7b0413d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:21.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.724+0000 7f7b0bd50700 1 --2- 172.21.15.16:0/3892669773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b0406f050 0x7f7b0413d0d0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f7af4000c00 tx=0x7f7af4004750 comp rx=0 tx=0).stop 2023-09-30T22:33:21.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.724+0000 7f7b0bd50700 1 -- 172.21.15.16:0/3892669773 shutdown_connections 2023-09-30T22:33:21.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.724+0000 7f7b0bd50700 1 --2- 172.21.15.16:0/3892669773 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7b04138360 0x7f7b0413d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:21.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.724+0000 7f7b0bd50700 1 --2- 172.21.15.16:0/3892669773 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7b0406e570 0x7f7b0413cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:21.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.724+0000 7f7b0bd50700 1 --2- 172.21.15.16:0/3892669773 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b0406f050 0x7f7b0413d0d0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:21.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.724+0000 7f7b0bd50700 1 -- 172.21.15.16:0/3892669773 >> 172.21.15.16:0/3892669773 conn(0x7f7b040a2aa0 msgr2=0x7f7b0409ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:21.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.724+0000 7f7b0bd50700 1 -- 172.21.15.16:0/3892669773 shutdown_connections 2023-09-30T22:33:21.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.724+0000 7f7b0bd50700 1 -- 172.21.15.16:0/3892669773 wait complete. 2023-09-30T22:33:21.740 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:33:21.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.768+0000 7ff6d446c700 1 -- 172.21.15.16:0/1921656075 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff6cc120a80 con 0x7ff6cc12ff20 2023-09-30T22:33:21.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.769+0000 7f9fdf242700 1 -- 172.21.15.16:0/2015469307 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9fd806b9e0 con 0x7f9fd8099a40 2023-09-30T22:33:21.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.769+0000 7ff6c2ffd700 1 -- 172.21.15.16:0/1921656075 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff6c802cb90 con 0x7ff6cc12ff20 2023-09-30T22:33:21.770 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:33:21.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.769+0000 7f9fd57fa700 1 -- 172.21.15.16:0/2015469307 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9fcc015070 con 0x7f9fd8099a40 2023-09-30T22:33:21.771 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:33:21.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.772+0000 7ff6d446c700 1 -- 172.21.15.16:0/1921656075 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff6cc12ff20 msgr2=0x7ff6cc09c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:21.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.772+0000 7ff6d446c700 1 --2- 172.21.15.16:0/1921656075 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff6cc12ff20 0x7ff6cc09c480 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8015000 tx=0x7ff6c8004750 comp rx=0 tx=0).stop 2023-09-30T22:33:21.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.772+0000 7ff6d446c700 1 -- 172.21.15.16:0/1921656075 shutdown_connections 2023-09-30T22:33:21.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.772+0000 7ff6d446c700 1 --2- 172.21.15.16:0/1921656075 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff6cc133a80 0x7ff6cc09cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:21.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.772+0000 7ff6d446c700 1 --2- 172.21.15.16:0/1921656075 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff6cc12ff20 0x7ff6cc09c480 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:21.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.772+0000 7ff6d446c700 1 --2- 172.21.15.16:0/1921656075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6cc130870 0x7ff6cc09c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:21.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.772+0000 7ff6d446c700 1 -- 172.21.15.16:0/1921656075 >> 172.21.15.16:0/1921656075 conn(0x7ff6cc0a4210 msgr2=0x7ff6cc0a0890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:21.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.772+0000 7ff6d446c700 1 -- 172.21.15.16:0/1921656075 shutdown_connections 2023-09-30T22:33:21.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.772+0000 7f9fdf242700 1 -- 172.21.15.16:0/2015469307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fd8099a40 msgr2=0x7f9fd811b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:21.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.772+0000 7f9fdf242700 1 --2- 172.21.15.16:0/2015469307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fd8099a40 0x7f9fd811b890 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc00b560 tx=0x7f9fcc00b920 comp rx=0 tx=0).stop 2023-09-30T22:33:21.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.772+0000 7ff6d446c700 1 -- 172.21.15.16:0/1921656075 wait complete. 2023-09-30T22:33:21.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.772+0000 7f9fdf242700 1 -- 172.21.15.16:0/2015469307 shutdown_connections 2023-09-30T22:33:21.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.772+0000 7f9fdf242700 1 --2- 172.21.15.16:0/2015469307 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9fd8131810 0x7f9fd811c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:21.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.772+0000 7f9fdf242700 1 --2- 172.21.15.16:0/2015469307 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fd8099a40 0x7f9fd811b890 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:21.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.772+0000 7f9fdf242700 1 --2- 172.21.15.16:0/2015469307 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9fd809a3b0 0x7f9fd811bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:21.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.772+0000 7f9fdf242700 1 -- 172.21.15.16:0/2015469307 >> 172.21.15.16:0/2015469307 conn(0x7f9fd80a2aa0 msgr2=0x7f9fd80a1310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:21.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.773+0000 7f9fdf242700 1 -- 172.21.15.16:0/2015469307 shutdown_connections 2023-09-30T22:33:21.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:21.773+0000 7f9fdf242700 1 -- 172.21.15.16:0/2015469307 wait complete. 2023-09-30T22:33:21.789 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:33:21.789 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:33:22.666 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:33:22.741 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:33:22.790 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:33:22.790 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:33:22.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.821+0000 7fc33dbd2700 1 Processor -- start 2023-09-30T22:33:22.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.821+0000 7fc33dbd2700 1 -- start start 2023-09-30T22:33:22.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.822+0000 7fc33dbd2700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc33812cf60 0x7fc33806f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:22.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.822+0000 7fc33dbd2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc33812f8b0 0x7fc33812fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:22.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.822+0000 7fc33dbd2700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc33812cd90 con 0x7fc338130260 2023-09-30T22:33:22.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.822+0000 7fc33dbd2700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc33812d330 con 0x7fc33812cf60 2023-09-30T22:33:22.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.822+0000 7fc33dbd2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc3381306d0 con 0x7fc33812f8b0 2023-09-30T22:33:22.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.823+0000 7fc3377fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc33812cf60 0x7fc33806f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:22.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.823+0000 7fc3377fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc33812cf60 0x7fc33806f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41502/0 (socket says 172.21.15.16:41502) 2023-09-30T22:33:22.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.823+0000 7fc3377fe700 1 -- 172.21.15.16:0/555204171 learned_addr learned my addr 172.21.15.16:0/555204171 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:22.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.823+0000 7fc337fff700 1 --1- 172.21.15.16:0/555204171 >> v1:172.21.15.16:6789/0 conn(0x7fc338130260 0x7fc338136c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54722/0 (socket says 172.21.15.16:54722) 2023-09-30T22:33:22.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.823+0000 7fc3377fe700 1 -- 172.21.15.16:0/555204171 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc33812f8b0 msgr2=0x7fc33812fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:22.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.823+0000 7fc3377fe700 1 --2- 172.21.15.16:0/555204171 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc33812f8b0 0x7fc33812fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:22.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.823+0000 7fc3377fe700 1 -- 172.21.15.16:0/555204171 >> v1:172.21.15.16:6789/0 conn(0x7fc338130260 legacy=0x7fc338136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:22.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.823+0000 7fc3377fe700 1 -- 172.21.15.16:0/555204171 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc338071780 con 0x7fc33812cf60 2023-09-30T22:33:22.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.823+0000 7fc3377fe700 1 --2- 172.21.15.16:0/555204171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc33812cf60 0x7fc33806f5d0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fc3200098f0 tx=0x7fc320009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb2f8a3acf7da7c9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:22.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.823+0000 7fc3367fc700 1 -- 172.21.15.16:0/555204171 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32000b8a0 con 0x7fc33812cf60 2023-09-30T22:33:22.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.823+0000 7fc3367fc700 1 -- 172.21.15.16:0/555204171 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc32000ba00 con 0x7fc33812cf60 2023-09-30T22:33:22.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.823+0000 7fc3367fc700 1 -- 172.21.15.16:0/555204171 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32000bd90 con 0x7fc33812cf60 2023-09-30T22:33:22.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.824+0000 7fc33dbd2700 1 -- 172.21.15.16:0/555204171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc33812cf60 msgr2=0x7fc33806f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:22.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.824+0000 7fc33dbd2700 1 --2- 172.21.15.16:0/555204171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc33812cf60 0x7fc33806f5d0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fc3200098f0 tx=0x7fc320009c00 comp rx=0 tx=0).stop 2023-09-30T22:33:22.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.826+0000 7fc33dbd2700 1 -- 172.21.15.16:0/555204171 shutdown_connections 2023-09-30T22:33:22.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.826+0000 7fc33dbd2700 1 --2- 172.21.15.16:0/555204171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc33812cf60 0x7fc33806f5d0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:22.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.826+0000 7fc33dbd2700 1 --2- 172.21.15.16:0/555204171 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc33812f8b0 0x7fc33812fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:22.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.826+0000 7fc33dbd2700 1 -- 172.21.15.16:0/555204171 >> 172.21.15.16:0/555204171 conn(0x7fc3380a2aa0 msgr2=0x7fc3380a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:22.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.826+0000 7fc33dbd2700 1 -- 172.21.15.16:0/555204171 shutdown_connections 2023-09-30T22:33:22.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.826+0000 7fc33dbd2700 1 -- 172.21.15.16:0/555204171 wait complete. 2023-09-30T22:33:22.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.827+0000 7fc33dbd2700 1 Processor -- start 2023-09-30T22:33:22.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.827+0000 7fc33dbd2700 1 -- start start 2023-09-30T22:33:22.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.828+0000 7fc33dbd2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc33812cf60 0x7fc33811a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:22.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.828+0000 7fc33dbd2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc33812f8b0 0x7fc33811ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:22.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.828+0000 7fc33dbd2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc338130260 0x7fc33811b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:22.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.828+0000 7fc33dbd2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc33811cb60 con 0x7fc33812cf60 2023-09-30T22:33:22.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.828+0000 7fc33dbd2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc33811c290 con 0x7fc338130260 2023-09-30T22:33:22.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.828+0000 7fc33dbd2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc33811bba0 con 0x7fc33812f8b0 2023-09-30T22:33:22.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.828+0000 7fc337fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc338130260 0x7fc33811b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:22.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.828+0000 7fc337fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc338130260 0x7fc33811b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54612/0 (socket says 172.21.15.16:54612) 2023-09-30T22:33:22.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.828+0000 7fc337fff700 1 -- 172.21.15.16:0/3772862486 learned_addr learned my addr 172.21.15.16:0/3772862486 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:22.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.829+0000 7fc336ffd700 1 --2- 172.21.15.16:0/3772862486 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc33812f8b0 0x7fc33811ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:22.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.829+0000 7fc337fff700 1 -- 172.21.15.16:0/3772862486 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc33812f8b0 msgr2=0x7fc33811ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:22.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.829+0000 7fc337fff700 1 --2- 172.21.15.16:0/3772862486 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc33812f8b0 0x7fc33811ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:22.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.829+0000 7fc337fff700 1 -- 172.21.15.16:0/3772862486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc33812cf60 msgr2=0x7fc33811a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:22.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.829+0000 7fc337fff700 1 --2- 172.21.15.16:0/3772862486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc33812cf60 0x7fc33811a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:22.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.829+0000 7fc337fff700 1 -- 172.21.15.16:0/3772862486 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc320009550 con 0x7fc338130260 2023-09-30T22:33:22.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.829+0000 7fc336ffd700 1 --2- 172.21.15.16:0/3772862486 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc33812f8b0 0x7fc33811ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:33:22.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.829+0000 7fc337fff700 1 --2- 172.21.15.16:0/3772862486 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc338130260 0x7fc33811b250 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fc33000cc00 tx=0x7fc33000a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:22.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.829+0000 7fc32ffff700 1 -- 172.21.15.16:0/3772862486 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33000aba0 con 0x7fc338130260 2023-09-30T22:33:22.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.829+0000 7fc32ffff700 1 -- 172.21.15.16:0/3772862486 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc33000ad00 con 0x7fc338130260 2023-09-30T22:33:22.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.829+0000 7fc33dbd2700 1 -- 172.21.15.16:0/3772862486 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc338129fa0 con 0x7fc338130260 2023-09-30T22:33:22.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.829+0000 7fc32ffff700 1 -- 172.21.15.16:0/3772862486 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc330007270 con 0x7fc338130260 2023-09-30T22:33:22.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.829+0000 7fc33dbd2700 1 -- 172.21.15.16:0/3772862486 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc33811df10 con 0x7fc338130260 2023-09-30T22:33:22.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.830+0000 7fc32ffff700 1 -- 172.21.15.16:0/3772862486 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc3300073d0 con 0x7fc338130260 2023-09-30T22:33:22.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.831+0000 7fc32ffff700 1 -- 172.21.15.16:0/3772862486 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fc33003f310 con 0x7fc338130260 2023-09-30T22:33:22.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.831+0000 7fc33dbd2700 1 -- 172.21.15.16:0/3772862486 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc33811dd20 con 0x7fc338130260 2023-09-30T22:33:22.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.840+0000 7fc32ffff700 1 -- 172.21.15.16:0/3772862486 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc33811dd20 con 0x7fc338130260 2023-09-30T22:33:22.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.890+0000 7fb853a8d700 1 Processor -- start 2023-09-30T22:33:22.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.890+0000 7fb853a8d700 1 -- start start 2023-09-30T22:33:22.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.891+0000 7fb853a8d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb84c09e1d0 0x7fb84c138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:22.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.891+0000 7fb853a8d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb84c139170 0x7fb84c13b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:22.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.891+0000 7fb853a8d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb84c09b120 con 0x7fb84c09d6f0 2023-09-30T22:33:22.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.891+0000 7fb853a8d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb84c06b460 con 0x7fb84c09e1d0 2023-09-30T22:33:22.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.891+0000 7fb853a8d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb84c09a7e0 con 0x7fb84c139170 2023-09-30T22:33:22.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.891+0000 7fb851028700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb84c09e1d0 0x7fb84c138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:22.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.891+0000 7fb851028700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb84c09e1d0 0x7fb84c138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41528/0 (socket says 172.21.15.16:41528) 2023-09-30T22:33:22.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.891+0000 7fb851829700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb84c09d6f0 0x7fb84c09dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54728/0 (socket says 172.21.15.16:54728) 2023-09-30T22:33:22.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.891+0000 7fb851028700 1 -- 172.21.15.16:0/1684253064 learned_addr learned my addr 172.21.15.16:0/1684253064 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:22.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.891+0000 7fb85202a700 1 --2- 172.21.15.16:0/1684253064 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb84c139170 0x7fb84c13b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:22.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.892+0000 7fb85202a700 1 -- 172.21.15.16:0/1684253064 >> v2:172.21.15.16:3300/0 conn(0x7fb84c09e1d0 msgr2=0x7fb84c138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:22.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.892+0000 7fb85202a700 1 --2- 172.21.15.16:0/1684253064 >> v2:172.21.15.16:3300/0 conn(0x7fb84c09e1d0 0x7fb84c138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:22.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.892+0000 7fb85202a700 1 -- 172.21.15.16:0/1684253064 >> v1:172.21.15.16:6789/0 conn(0x7fb84c09d6f0 legacy=0x7fb84c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:22.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.892+0000 7fb85202a700 1 -- 172.21.15.16:0/1684253064 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb84c09a500 con 0x7fb84c139170 2023-09-30T22:33:22.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.892+0000 7fb850827700 1 -- 172.21.15.16:0/1684253064 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2202701377 0 0) 0x7fb84c09b120 con 0x7fb84c09d6f0 2023-09-30T22:33:22.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.892+0000 7fb851028700 1 --2- 172.21.15.16:0/1684253064 >> v2:172.21.15.16:3300/0 conn(0x7fb84c09e1d0 0x7fb84c138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:22.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.892+0000 7fb85202a700 1 --2- 172.21.15.16:0/1684253064 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb84c139170 0x7fb84c13b550 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb844009dc0 tx=0x7fb84400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4d080ac7621449e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:22.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.892+0000 7fb850827700 1 -- 172.21.15.16:0/1684253064 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84400e070 con 0x7fb84c139170 2023-09-30T22:33:22.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.892+0000 7fb850827700 1 -- 172.21.15.16:0/1684253064 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb84400bae0 con 0x7fb84c139170 2023-09-30T22:33:22.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.892+0000 7fb850827700 1 -- 172.21.15.16:0/1684253064 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84400be90 con 0x7fb84c139170 2023-09-30T22:33:22.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.893+0000 7fb853a8d700 1 -- 172.21.15.16:0/1684253064 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb84c139170 msgr2=0x7fb84c13b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:22.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.893+0000 7fb853a8d700 1 --2- 172.21.15.16:0/1684253064 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb84c139170 0x7fb84c13b550 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb844009dc0 tx=0x7fb84400b330 comp rx=0 tx=0).stop 2023-09-30T22:33:22.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.893+0000 7fb853a8d700 1 -- 172.21.15.16:0/1684253064 shutdown_connections 2023-09-30T22:33:22.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.893+0000 7fb853a8d700 1 --2- 172.21.15.16:0/1684253064 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb84c139170 0x7fb84c13b550 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:22.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.893+0000 7fb853a8d700 1 --2- 172.21.15.16:0/1684253064 >> v2:172.21.15.16:3300/0 conn(0x7fb84c09e1d0 0x7fb84c138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:22.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.893+0000 7fb853a8d700 1 -- 172.21.15.16:0/1684253064 >> 172.21.15.16:0/1684253064 conn(0x7fb84c066f50 msgr2=0x7fb84c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:22.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.893+0000 7fb853a8d700 1 -- 172.21.15.16:0/1684253064 shutdown_connections 2023-09-30T22:33:22.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.893+0000 7fb853a8d700 1 -- 172.21.15.16:0/1684253064 wait complete. 2023-09-30T22:33:22.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.894+0000 7fb853a8d700 1 Processor -- start 2023-09-30T22:33:22.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.894+0000 7fb853a8d700 1 -- start start 2023-09-30T22:33:22.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.895+0000 7fb853a8d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb84c09d6f0 0x7fb84c1216c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:22.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.895+0000 7fb853a8d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb84c09e1d0 0x7fb84c121c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:22.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.895+0000 7fb853a8d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb84c139170 0x7fb84c122140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:22.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.895+0000 7fb853a8d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb84c1375b0 con 0x7fb84c09d6f0 2023-09-30T22:33:22.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.895+0000 7fb853a8d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb84c1251e0 con 0x7fb84c09e1d0 2023-09-30T22:33:22.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.895+0000 7fb853a8d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb84c123af0 con 0x7fb84c139170 2023-09-30T22:33:22.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.895+0000 7fb851829700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb84c09d6f0 0x7fb84c1216c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:22.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.895+0000 7fb851829700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb84c09d6f0 0x7fb84c1216c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41540/0 (socket says 172.21.15.16:41540) 2023-09-30T22:33:22.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.895+0000 7fb851829700 1 -- 172.21.15.16:0/1145695805 learned_addr learned my addr 172.21.15.16:0/1145695805 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:22.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.895+0000 7fb85202a700 1 --2- 172.21.15.16:0/1145695805 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb84c139170 0x7fb84c122140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:22.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.895+0000 7fb851028700 1 --2- 172.21.15.16:0/1145695805 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb84c09e1d0 0x7fb84c121c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:22.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.895+0000 7fb851829700 1 -- 172.21.15.16:0/1145695805 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb84c139170 msgr2=0x7fb84c122140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:22.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.895+0000 7fb851829700 1 --2- 172.21.15.16:0/1145695805 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb84c139170 0x7fb84c122140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:22.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.895+0000 7fb851829700 1 -- 172.21.15.16:0/1145695805 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb84c09e1d0 msgr2=0x7fb84c121c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:22.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.895+0000 7fb851829700 1 --2- 172.21.15.16:0/1145695805 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb84c09e1d0 0x7fb84c121c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:22.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.895+0000 7fb851829700 1 -- 172.21.15.16:0/1145695805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb83c009550 con 0x7fb84c09d6f0 2023-09-30T22:33:22.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.896+0000 7fb85202a700 1 --2- 172.21.15.16:0/1145695805 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb84c139170 0x7fb84c122140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:33:22.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.896+0000 7fb851829700 1 --2- 172.21.15.16:0/1145695805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb84c09d6f0 0x7fb84c1216c0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fb84800c520 tx=0x7fb84800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:22.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.896+0000 7fb8427fc700 1 -- 172.21.15.16:0/1145695805 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb848004c80 con 0x7fb84c09d6f0 2023-09-30T22:33:22.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.896+0000 7fb8427fc700 1 -- 172.21.15.16:0/1145695805 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb848004de0 con 0x7fb84c09d6f0 2023-09-30T22:33:22.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.896+0000 7fb853a8d700 1 -- 172.21.15.16:0/1145695805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb844009a70 con 0x7fb84c09d6f0 2023-09-30T22:33:22.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.896+0000 7fb8427fc700 1 -- 172.21.15.16:0/1145695805 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb848007310 con 0x7fb84c09d6f0 2023-09-30T22:33:22.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.896+0000 7fb853a8d700 1 -- 172.21.15.16:0/1145695805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb84c09c710 con 0x7fb84c09d6f0 2023-09-30T22:33:22.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.897+0000 7fb8427fc700 1 -- 172.21.15.16:0/1145695805 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb848007470 con 0x7fb84c09d6f0 2023-09-30T22:33:22.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.897+0000 7fb8427fc700 1 -- 172.21.15.16:0/1145695805 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fb84803f3c0 con 0x7fb84c09d6f0 2023-09-30T22:33:22.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.897+0000 7fb853a8d700 1 -- 172.21.15.16:0/1145695805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb84c074ea0 con 0x7fb84c09d6f0 2023-09-30T22:33:22.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.907+0000 7fb8427fc700 1 -- 172.21.15.16:0/1145695805 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb8480123b0 con 0x7fb84c09d6f0 2023-09-30T22:33:22.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.942+0000 7f1932919700 1 Processor -- start 2023-09-30T22:33:22.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.943+0000 7f1932919700 1 -- start start 2023-09-30T22:33:22.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.943+0000 7f1932919700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f192c06a770 0x7f192c09d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:22.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.943+0000 7f1932919700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f192c09dc30 0x7f192c09e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:22.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.943+0000 7f1932919700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f192c09a810 con 0x7f192c13c1e0 2023-09-30T22:33:22.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.943+0000 7f1932919700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f192c06ab40 con 0x7f192c06a770 2023-09-30T22:33:22.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.943+0000 7f1932919700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f192c09ebc0 con 0x7f192c09dc30 2023-09-30T22:33:22.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.943+0000 7f192b7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f192c09dc30 0x7f192c09e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:22.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.943+0000 7f1930eb6700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f192c13c1e0 0x7f192c13e5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54740/0 (socket says 172.21.15.16:54740) 2023-09-30T22:33:22.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.943+0000 7f1930eb6700 1 -- 172.21.15.16:0/2660322086 learned_addr learned my addr 172.21.15.16:0/2660322086 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:22.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.943+0000 7f192bfff700 1 --2- 172.21.15.16:0/2660322086 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f192c06a770 0x7f192c09d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:22.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.944+0000 7f192bfff700 1 -- 172.21.15.16:0/2660322086 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f192c09dc30 msgr2=0x7f192c09e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:22.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.944+0000 7f192bfff700 1 --2- 172.21.15.16:0/2660322086 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f192c09dc30 0x7f192c09e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:22.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.944+0000 7f192affd700 1 -- 172.21.15.16:0/2660322086 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2867063666 0 0) 0x7f192c09a810 con 0x7f192c13c1e0 2023-09-30T22:33:22.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.944+0000 7f192bfff700 1 -- 172.21.15.16:0/2660322086 >> v1:172.21.15.16:6789/0 conn(0x7f192c13c1e0 legacy=0x7f192c13e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:22.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.944+0000 7f192bfff700 1 -- 172.21.15.16:0/2660322086 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f192c09e920 con 0x7f192c06a770 2023-09-30T22:33:22.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.944+0000 7f192bfff700 1 --2- 172.21.15.16:0/2660322086 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f192c06a770 0x7f192c09d6f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f191c009870 tx=0x7f191c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dadb9ce724c94267 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:22.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.944+0000 7f192affd700 1 -- 172.21.15.16:0/2660322086 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191c011830 con 0x7f192c06a770 2023-09-30T22:33:22.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.944+0000 7f192affd700 1 -- 172.21.15.16:0/2660322086 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f191c011990 con 0x7f192c06a770 2023-09-30T22:33:22.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.944+0000 7f192affd700 1 -- 172.21.15.16:0/2660322086 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191c011d20 con 0x7f192c06a770 2023-09-30T22:33:22.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.945+0000 7f1932919700 1 -- 172.21.15.16:0/2660322086 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f192c06a770 msgr2=0x7f192c09d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:22.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.945+0000 7f1932919700 1 --2- 172.21.15.16:0/2660322086 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f192c06a770 0x7f192c09d6f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f191c009870 tx=0x7f191c009b80 comp rx=0 tx=0).stop 2023-09-30T22:33:22.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.945+0000 7f1932919700 1 -- 172.21.15.16:0/2660322086 shutdown_connections 2023-09-30T22:33:22.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.945+0000 7f1932919700 1 --2- 172.21.15.16:0/2660322086 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f192c06a770 0x7f192c09d6f0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:22.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.945+0000 7f1932919700 1 --2- 172.21.15.16:0/2660322086 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f192c09dc30 0x7f192c09e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:22.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.945+0000 7f1932919700 1 -- 172.21.15.16:0/2660322086 >> 172.21.15.16:0/2660322086 conn(0x7f192c066f50 msgr2=0x7f192c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:22.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.945+0000 7f1932919700 1 -- 172.21.15.16:0/2660322086 shutdown_connections 2023-09-30T22:33:22.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.945+0000 7f1932919700 1 -- 172.21.15.16:0/2660322086 wait complete. 2023-09-30T22:33:22.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.946+0000 7f1932919700 1 Processor -- start 2023-09-30T22:33:22.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.946+0000 7f1932919700 1 -- start start 2023-09-30T22:33:22.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.947+0000 7f1932919700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f192c06a770 0x7f192c11ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:22.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.947+0000 7f1932919700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f192c09dc30 0x7f192c13a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:22.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.947+0000 7f1932919700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f192c13c1e0 0x7f192c13a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:22.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.947+0000 7f1932919700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f192c1349e0 con 0x7f192c09dc30 2023-09-30T22:33:22.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.947+0000 7f1932919700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f192c11d310 con 0x7f192c06a770 2023-09-30T22:33:22.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.947+0000 7f1932919700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f192c13baa0 con 0x7f192c13c1e0 2023-09-30T22:33:22.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.947+0000 7f1930eb6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f192c13c1e0 0x7f192c13a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:22.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.947+0000 7f192b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f192c09dc30 0x7f192c13a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:22.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.947+0000 7f1930eb6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f192c13c1e0 0x7f192c13a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:36252/0 (socket says 172.21.15.16:36252) 2023-09-30T22:33:22.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.947+0000 7f1930eb6700 1 -- 172.21.15.16:0/1910724256 learned_addr learned my addr 172.21.15.16:0/1910724256 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:22.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.947+0000 7f1930eb6700 1 -- 172.21.15.16:0/1910724256 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f192c06a770 msgr2=0x7f192c11ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:22.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.947+0000 7f1930eb6700 1 --2- 172.21.15.16:0/1910724256 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f192c06a770 0x7f192c11ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:22.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.947+0000 7f1930eb6700 1 -- 172.21.15.16:0/1910724256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f192c09dc30 msgr2=0x7f192c13a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:22.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.947+0000 7f1930eb6700 1 --2- 172.21.15.16:0/1910724256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f192c09dc30 0x7f192c13a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:22.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.947+0000 7f1930eb6700 1 -- 172.21.15.16:0/1910724256 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f191c0094a0 con 0x7f192c13c1e0 2023-09-30T22:33:22.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.947+0000 7f1930eb6700 1 --2- 172.21.15.16:0/1910724256 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f192c13c1e0 0x7f192c13a7e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f1924002230 tx=0x7f192400cf50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:23.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.948+0000 7f1928ff9700 1 -- 172.21.15.16:0/1910724256 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192400ab70 con 0x7f192c13c1e0 2023-09-30T22:33:23.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.948+0000 7f1928ff9700 1 -- 172.21.15.16:0/1910724256 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f192400acf0 con 0x7f192c13c1e0 2023-09-30T22:33:23.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.948+0000 7f1932919700 1 -- 172.21.15.16:0/1910724256 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f192c11b9a0 con 0x7f192c13c1e0 2023-09-30T22:33:23.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.948+0000 7f1928ff9700 1 -- 172.21.15.16:0/1910724256 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1924007310 con 0x7f192c13c1e0 2023-09-30T22:33:23.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.948+0000 7f1932919700 1 -- 172.21.15.16:0/1910724256 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f192c09f1a0 con 0x7f192c13c1e0 2023-09-30T22:33:23.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.949+0000 7f1928ff9700 1 -- 172.21.15.16:0/1910724256 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1924007470 con 0x7f192c13c1e0 2023-09-30T22:33:23.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.949+0000 7f1928ff9700 1 -- 172.21.15.16:0/1910724256 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f192403f580 con 0x7f192c13c1e0 2023-09-30T22:33:23.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.949+0000 7f1932919700 1 -- 172.21.15.16:0/1910724256 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f192c074ea0 con 0x7f192c13c1e0 2023-09-30T22:33:23.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.953+0000 7faf92c60700 1 Processor -- start 2023-09-30T22:33:23.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.953+0000 7faf92c60700 1 -- start start 2023-09-30T22:33:23.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.954+0000 7faf92c60700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf8c0a0a30 0x7faf8c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:23.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.954+0000 7faf92c60700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf8c134170 0x7faf8c13bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:23.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.954+0000 7faf92c60700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf8c09edd0 con 0x7faf8c0a1340 2023-09-30T22:33:23.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.954+0000 7faf92c60700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf8c09e1f0 con 0x7faf8c134170 2023-09-30T22:33:23.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.954+0000 7faf92c60700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faf8c09dc50 con 0x7faf8c0a0a30 2023-09-30T22:33:23.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.954+0000 7faf909fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf8c0a0a30 0x7faf8c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:23.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.954+0000 7faf909fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf8c0a0a30 0x7faf8c0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:36266/0 (socket says 172.21.15.16:36266) 2023-09-30T22:33:23.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.954+0000 7faf909fc700 1 -- 172.21.15.16:0/1760448584 learned_addr learned my addr 172.21.15.16:0/1760448584 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:23.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.954+0000 7faf911fd700 1 --2- 172.21.15.16:0/1760448584 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf8c134170 0x7faf8c13bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:23.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.955+0000 7faf911fd700 1 -- 172.21.15.16:0/1760448584 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf8c0a0a30 msgr2=0x7faf8c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:23.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.955+0000 7faf911fd700 1 --2- 172.21.15.16:0/1760448584 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf8c0a0a30 0x7faf8c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:23.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.955+0000 7faf911fd700 1 -- 172.21.15.16:0/1760448584 >> v1:172.21.15.16:6789/0 conn(0x7faf8c0a1340 legacy=0x7faf8c133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:23.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.955+0000 7faf911fd700 1 -- 172.21.15.16:0/1760448584 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf8c0a1fe0 con 0x7faf8c134170 2023-09-30T22:33:23.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.955+0000 7faf8b7fe700 1 -- 172.21.15.16:0/1760448584 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3363940406 0 0) 0x7faf8c09edd0 con 0x7faf8c0a1340 2023-09-30T22:33:23.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.955+0000 7faf909fc700 1 --2- 172.21.15.16:0/1760448584 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf8c0a0a30 0x7faf8c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:23.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.955+0000 7f1928ff9700 1 -- 172.21.15.16:0/1910724256 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1924018080 con 0x7f192c13c1e0 2023-09-30T22:33:23.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.955+0000 7faf911fd700 1 --2- 172.21.15.16:0/1760448584 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf8c134170 0x7faf8c13bd30 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7faf84009dc0 tx=0x7faf8400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4411dbbfd09dd046 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:23.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.955+0000 7faf8b7fe700 1 -- 172.21.15.16:0/1760448584 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf8400e070 con 0x7faf8c134170 2023-09-30T22:33:23.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.955+0000 7faf8b7fe700 1 -- 172.21.15.16:0/1760448584 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf8400bb20 con 0x7faf8c134170 2023-09-30T22:33:23.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.955+0000 7faf8b7fe700 1 -- 172.21.15.16:0/1760448584 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf84012430 con 0x7faf8c134170 2023-09-30T22:33:23.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.956+0000 7faf92c60700 1 -- 172.21.15.16:0/1760448584 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf8c134170 msgr2=0x7faf8c13bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:23.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.956+0000 7faf92c60700 1 --2- 172.21.15.16:0/1760448584 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf8c134170 0x7faf8c13bd30 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7faf84009dc0 tx=0x7faf8400b330 comp rx=0 tx=0).stop 2023-09-30T22:33:23.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.956+0000 7faf92c60700 1 -- 172.21.15.16:0/1760448584 shutdown_connections 2023-09-30T22:33:23.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.956+0000 7faf92c60700 1 --2- 172.21.15.16:0/1760448584 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf8c134170 0x7faf8c13bd30 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:23.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.956+0000 7faf92c60700 1 --2- 172.21.15.16:0/1760448584 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf8c0a0a30 0x7faf8c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:23.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.956+0000 7faf92c60700 1 -- 172.21.15.16:0/1760448584 >> 172.21.15.16:0/1760448584 conn(0x7faf8c0a6550 msgr2=0x7faf8c0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:23.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.956+0000 7faf92c60700 1 -- 172.21.15.16:0/1760448584 shutdown_connections 2023-09-30T22:33:23.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.956+0000 7faf92c60700 1 -- 172.21.15.16:0/1760448584 wait complete. 2023-09-30T22:33:23.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.957+0000 7faf92c60700 1 Processor -- start 2023-09-30T22:33:23.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.957+0000 7faf92c60700 1 -- start start 2023-09-30T22:33:23.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.958+0000 7faf92c60700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf8c0a0a30 0x7faf8c122780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:23.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.958+0000 7faf92c60700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf8c0a1340 0x7faf8c122cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:23.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.958+0000 7faf92c60700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf8c134170 0x7faf8c123200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:23.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.958+0000 7faf92c60700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf8c0a4ec0 con 0x7faf8c134170 2023-09-30T22:33:23.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.958+0000 7faf92c60700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf8c1336b0 con 0x7faf8c0a0a30 2023-09-30T22:33:23.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.958+0000 7faf92c60700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faf8c124410 con 0x7faf8c0a1340 2023-09-30T22:33:23.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.958+0000 7faf911fd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf8c134170 0x7faf8c123200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:23.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.958+0000 7faf8bfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf8c0a1340 0x7faf8c122cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:23.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.958+0000 7faf911fd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf8c134170 0x7faf8c123200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41548/0 (socket says 172.21.15.16:41548) 2023-09-30T22:33:23.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.958+0000 7faf909fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf8c0a0a30 0x7faf8c122780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:23.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.958+0000 7faf911fd700 1 -- 172.21.15.16:0/2168742398 learned_addr learned my addr 172.21.15.16:0/2168742398 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:23.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.958+0000 7faf911fd700 1 -- 172.21.15.16:0/2168742398 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf8c0a1340 msgr2=0x7faf8c122cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:23.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.958+0000 7faf911fd700 1 --2- 172.21.15.16:0/2168742398 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf8c0a1340 0x7faf8c122cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:23.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.958+0000 7faf911fd700 1 -- 172.21.15.16:0/2168742398 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf8c0a0a30 msgr2=0x7faf8c122780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:23.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.958+0000 7faf911fd700 1 --2- 172.21.15.16:0/2168742398 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf8c0a0a30 0x7faf8c122780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:23.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.958+0000 7faf911fd700 1 -- 172.21.15.16:0/2168742398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf80009550 con 0x7faf8c134170 2023-09-30T22:33:23.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.959+0000 7faf911fd700 1 --2- 172.21.15.16:0/2168742398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf8c134170 0x7faf8c123200 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7faf8c09e380 tx=0x7faf84004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:23.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.959+0000 7faf897fa700 1 -- 172.21.15.16:0/2168742398 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf840048f0 con 0x7faf8c134170 2023-09-30T22:33:23.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.959+0000 7faf897fa700 1 -- 172.21.15.16:0/2168742398 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf84020070 con 0x7faf8c134170 2023-09-30T22:33:23.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.959+0000 7faf897fa700 1 -- 172.21.15.16:0/2168742398 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf840276c0 con 0x7faf8c134170 2023-09-30T22:33:23.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.959+0000 7faf92c60700 1 -- 172.21.15.16:0/2168742398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf84009a70 con 0x7faf8c134170 2023-09-30T22:33:23.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.959+0000 7faf92c60700 1 -- 172.21.15.16:0/2168742398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf8c0a25e0 con 0x7faf8c134170 2023-09-30T22:33:23.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.960+0000 7faf897fa700 1 -- 172.21.15.16:0/2168742398 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7faf84012430 con 0x7faf8c134170 2023-09-30T22:33:23.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.960+0000 7faf897fa700 1 -- 172.21.15.16:0/2168742398 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7faf84022070 con 0x7faf8c134170 2023-09-30T22:33:23.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.960+0000 7faf92c60700 1 -- 172.21.15.16:0/2168742398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf8c1332a0 con 0x7faf8c134170 2023-09-30T22:33:23.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:22.965+0000 7faf897fa700 1 -- 172.21.15.16:0/2168742398 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7faf8401b400 con 0x7faf8c134170 2023-09-30T22:33:23.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:23.056+0000 7fc33dbd2700 1 -- 172.21.15.16:0/3772862486 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc33811df10 con 0x7fc338130260 2023-09-30T22:33:23.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:23.056+0000 7fc32ffff700 1 -- 172.21.15.16:0/3772862486 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc33811df10 con 0x7fc338130260 2023-09-30T22:33:23.075 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:33:23.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:23.059+0000 7fc33dbd2700 1 -- 172.21.15.16:0/3772862486 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc338130260 msgr2=0x7fc33811b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:23.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:23.059+0000 7fc33dbd2700 1 --2- 172.21.15.16:0/3772862486 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc338130260 0x7fc33811b250 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fc33000cc00 tx=0x7fc33000a060 comp rx=0 tx=0).stop 2023-09-30T22:33:23.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:23.059+0000 7fc33dbd2700 1 -- 172.21.15.16:0/3772862486 shutdown_connections 2023-09-30T22:33:23.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:23.059+0000 7fc33dbd2700 1 --2- 172.21.15.16:0/3772862486 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc338130260 0x7fc33811b250 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:23.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:23.059+0000 7fc33dbd2700 1 --2- 172.21.15.16:0/3772862486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc33812cf60 0x7fc33811a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:23.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:23.059+0000 7fc33dbd2700 1 --2- 172.21.15.16:0/3772862486 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc33812f8b0 0x7fc33811ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:23.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:23.059+0000 7fc33dbd2700 1 -- 172.21.15.16:0/3772862486 >> 172.21.15.16:0/3772862486 conn(0x7fc3380a2aa0 msgr2=0x7fc33806e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:23.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:23.059+0000 7fc33dbd2700 1 -- 172.21.15.16:0/3772862486 shutdown_connections 2023-09-30T22:33:23.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:23.059+0000 7fc33dbd2700 1 -- 172.21.15.16:0/3772862486 wait complete. 2023-09-30T22:33:23.080 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:33:23.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:23.144+0000 7fb853a8d700 1 -- 172.21.15.16:0/1145695805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb84c09f420 con 0x7fb84c09d6f0 2023-09-30T22:33:23.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:23.145+0000 7fb8427fc700 1 -- 172.21.15.16:0/1145695805 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb848011360 con 0x7fb84c09d6f0 2023-09-30T22:33:23.146 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:33:23.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:23.147+0000 7fb853a8d700 1 -- 172.21.15.16:0/1145695805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb84c09d6f0 msgr2=0x7fb84c1216c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:23.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:23.147+0000 7fb853a8d700 1 --2- 172.21.15.16:0/1145695805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb84c09d6f0 0x7fb84c1216c0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fb84800c520 tx=0x7fb84800c830 comp rx=0 tx=0).stop 2023-09-30T22:33:23.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:23.147+0000 7fb853a8d700 1 -- 172.21.15.16:0/1145695805 shutdown_connections 2023-09-30T22:33:23.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:23.147+0000 7fb853a8d700 1 --2- 172.21.15.16:0/1145695805 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb84c139170 0x7fb84c122140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:23.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:23.147+0000 7fb853a8d700 1 --2- 172.21.15.16:0/1145695805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb84c09d6f0 0x7fb84c1216c0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:23.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:23.147+0000 7fb853a8d700 1 --2- 172.21.15.16:0/1145695805 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb84c09e1d0 0x7fb84c121c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:23.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:23.147+0000 7fb853a8d700 1 -- 172.21.15.16:0/1145695805 >> 172.21.15.16:0/1145695805 conn(0x7fb84c066f50 msgr2=0x7fb84c070e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:23.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:23.147+0000 7fb853a8d700 1 -- 172.21.15.16:0/1145695805 shutdown_connections 2023-09-30T22:33:23.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:23.147+0000 7fb853a8d700 1 -- 172.21.15.16:0/1145695805 wait complete. 2023-09-30T22:33:23.162 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:33:23.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:23.181+0000 7faf92c60700 1 -- 172.21.15.16:0/2168742398 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7faf8c074ea0 con 0x7faf8c134170 2023-09-30T22:33:23.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:23.182+0000 7faf897fa700 1 -- 172.21.15.16:0/2168742398 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7faf8401a380 con 0x7faf8c134170 2023-09-30T22:33:23.183 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:33:23.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:23.182+0000 7f1932919700 1 -- 172.21.15.16:0/1910724256 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f192c11c7a0 con 0x7f192c13c1e0 2023-09-30T22:33:23.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:23.182+0000 7f1928ff9700 1 -- 172.21.15.16:0/1910724256 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f192400ae60 con 0x7f192c13c1e0 2023-09-30T22:33:23.184 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:33:23.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:23.184+0000 7faf92c60700 1 -- 172.21.15.16:0/2168742398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf8c134170 msgr2=0x7faf8c123200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:23.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:23.184+0000 7faf92c60700 1 --2- 172.21.15.16:0/2168742398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf8c134170 0x7faf8c123200 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7faf8c09e380 tx=0x7faf84004ad0 comp rx=0 tx=0).stop 2023-09-30T22:33:23.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:23.184+0000 7faf92c60700 1 -- 172.21.15.16:0/2168742398 shutdown_connections 2023-09-30T22:33:23.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:23.184+0000 7faf92c60700 1 --2- 172.21.15.16:0/2168742398 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf8c0a0a30 0x7faf8c122780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:23.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:23.184+0000 7faf92c60700 1 --2- 172.21.15.16:0/2168742398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf8c134170 0x7faf8c123200 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:23.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:23.184+0000 7faf92c60700 1 --2- 172.21.15.16:0/2168742398 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf8c0a1340 0x7faf8c122cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:23.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:23.184+0000 7faf92c60700 1 -- 172.21.15.16:0/2168742398 >> 172.21.15.16:0/2168742398 conn(0x7faf8c0a6550 msgr2=0x7faf8c09edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:23.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:23.184+0000 7faf92c60700 1 -- 172.21.15.16:0/2168742398 shutdown_connections 2023-09-30T22:33:23.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:23.184+0000 7faf92c60700 1 -- 172.21.15.16:0/2168742398 wait complete. 2023-09-30T22:33:23.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:23.185+0000 7f1932919700 1 -- 172.21.15.16:0/1910724256 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f192c13c1e0 msgr2=0x7f192c13a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:23.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:23.185+0000 7f1932919700 1 --2- 172.21.15.16:0/1910724256 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f192c13c1e0 0x7f192c13a7e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f1924002230 tx=0x7f192400cf50 comp rx=0 tx=0).stop 2023-09-30T22:33:23.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:23.185+0000 7f1932919700 1 -- 172.21.15.16:0/1910724256 shutdown_connections 2023-09-30T22:33:23.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:23.185+0000 7f1932919700 1 --2- 172.21.15.16:0/1910724256 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f192c06a770 0x7f192c11ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:23.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:23.185+0000 7f1932919700 1 --2- 172.21.15.16:0/1910724256 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f192c13c1e0 0x7f192c13a7e0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:23.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:23.185+0000 7f1932919700 1 --2- 172.21.15.16:0/1910724256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f192c09dc30 0x7f192c13a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:23.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:23.185+0000 7f1932919700 1 -- 172.21.15.16:0/1910724256 >> 172.21.15.16:0/1910724256 conn(0x7f192c066f50 msgr2=0x7f192c072050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:23.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:23.185+0000 7f1932919700 1 -- 172.21.15.16:0/1910724256 shutdown_connections 2023-09-30T22:33:23.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:23.185+0000 7f1932919700 1 -- 172.21.15.16:0/1910724256 wait complete. 2023-09-30T22:33:23.201 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:33:23.201 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:33:24.081 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:33:24.163 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:33:24.204 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:33:24.204 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:33:24.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.228+0000 7f9e6e96a700 1 Processor -- start 2023-09-30T22:33:24.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.229+0000 7f9e6e96a700 1 -- start start 2023-09-30T22:33:24.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.229+0000 7f9e6e96a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9e6812cf60 0x7f9e6806f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:24.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.229+0000 7f9e6e96a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e6812f8b0 0x7f9e6812fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:24.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.229+0000 7f9e6e96a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e6812cd90 con 0x7f9e68130260 2023-09-30T22:33:24.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.229+0000 7f9e6e96a700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e6812d330 con 0x7f9e6812cf60 2023-09-30T22:33:24.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.229+0000 7f9e6e96a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e681306d0 con 0x7f9e6812f8b0 2023-09-30T22:33:24.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.230+0000 7f9e6cf07700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9e68130260 0x7f9e68136c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54760/0 (socket says 172.21.15.16:54760) 2023-09-30T22:33:24.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.230+0000 7f9e6cf07700 1 -- 172.21.15.16:0/3419754907 learned_addr learned my addr 172.21.15.16:0/3419754907 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:24.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.230+0000 7f9e67fff700 1 --2- 172.21.15.16:0/3419754907 >> v2:172.21.15.16:3300/0 conn(0x7f9e6812cf60 0x7f9e6806f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:24.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.230+0000 7f9e67fff700 1 -- 172.21.15.16:0/3419754907 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e6812f8b0 msgr2=0x7f9e6812fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:24.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.230+0000 7f9e67fff700 1 --2- 172.21.15.16:0/3419754907 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e6812f8b0 0x7f9e6812fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:24.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.230+0000 7f9e67fff700 1 -- 172.21.15.16:0/3419754907 >> v1:172.21.15.16:6789/0 conn(0x7f9e68130260 legacy=0x7f9e68136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:24.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.230+0000 7f9e66ffd700 1 -- 172.21.15.16:0/3419754907 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 211340952 0 0) 0x7f9e6812cd90 con 0x7f9e68130260 2023-09-30T22:33:24.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.230+0000 7f9e67fff700 1 -- 172.21.15.16:0/3419754907 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e68071780 con 0x7f9e6812cf60 2023-09-30T22:33:24.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.230+0000 7f9e67fff700 1 --2- 172.21.15.16:0/3419754907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e6812cf60 0x7f9e6806f5d0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f9e58009840 tx=0x7f9e58009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9da2e2ac6b16389a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:24.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.231+0000 7f9e66ffd700 1 -- 172.21.15.16:0/3419754907 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e58011830 con 0x7f9e6812cf60 2023-09-30T22:33:24.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.231+0000 7f9e66ffd700 1 -- 172.21.15.16:0/3419754907 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e58011990 con 0x7f9e6812cf60 2023-09-30T22:33:24.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.231+0000 7f9e66ffd700 1 -- 172.21.15.16:0/3419754907 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e58011d20 con 0x7f9e6812cf60 2023-09-30T22:33:24.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.231+0000 7f9e6e96a700 1 -- 172.21.15.16:0/3419754907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e6812cf60 msgr2=0x7f9e6806f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:24.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.231+0000 7f9e6e96a700 1 --2- 172.21.15.16:0/3419754907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e6812cf60 0x7f9e6806f5d0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f9e58009840 tx=0x7f9e58009b50 comp rx=0 tx=0).stop 2023-09-30T22:33:24.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.232+0000 7f9e6e96a700 1 -- 172.21.15.16:0/3419754907 shutdown_connections 2023-09-30T22:33:24.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.232+0000 7f9e6e96a700 1 --2- 172.21.15.16:0/3419754907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e6812cf60 0x7f9e6806f5d0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:24.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.232+0000 7f9e6e96a700 1 --2- 172.21.15.16:0/3419754907 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e6812f8b0 0x7f9e6812fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:24.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.232+0000 7f9e6e96a700 1 -- 172.21.15.16:0/3419754907 >> 172.21.15.16:0/3419754907 conn(0x7f9e680a2aa0 msgr2=0x7f9e680a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:24.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.232+0000 7f9e6e96a700 1 -- 172.21.15.16:0/3419754907 shutdown_connections 2023-09-30T22:33:24.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.232+0000 7f9e6e96a700 1 -- 172.21.15.16:0/3419754907 wait complete. 2023-09-30T22:33:24.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.233+0000 7f9e6e96a700 1 Processor -- start 2023-09-30T22:33:24.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.234+0000 7f9e6e96a700 1 -- start start 2023-09-30T22:33:24.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.234+0000 7f9e6e96a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e6812cf60 0x7f9e6809b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:24.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.234+0000 7f9e6e96a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e6812f8b0 0x7f9e6809be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:24.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.234+0000 7f9e6e96a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e68130260 0x7f9e681c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:24.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.234+0000 7f9e6e96a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e68129fa0 con 0x7f9e6812f8b0 2023-09-30T22:33:24.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.234+0000 7f9e6e96a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e68129c30 con 0x7f9e6812cf60 2023-09-30T22:33:24.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.234+0000 7f9e6e96a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6809cbb0 con 0x7f9e68130260 2023-09-30T22:33:24.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.234+0000 7f9e67fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e6812cf60 0x7f9e6809b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:24.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.234+0000 7f9e67fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e6812cf60 0x7f9e6809b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54678/0 (socket says 172.21.15.16:54678) 2023-09-30T22:33:24.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.234+0000 7f9e67fff700 1 -- 172.21.15.16:0/2238845409 learned_addr learned my addr 172.21.15.16:0/2238845409 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:24.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.234+0000 7f9e677fe700 1 --2- 172.21.15.16:0/2238845409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e6812f8b0 0x7f9e6809be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:24.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.234+0000 7f9e6cf07700 1 --2- 172.21.15.16:0/2238845409 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e68130260 0x7f9e681c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:24.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.235+0000 7f9e67fff700 1 -- 172.21.15.16:0/2238845409 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e68130260 msgr2=0x7f9e681c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:24.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.235+0000 7f9e67fff700 1 --2- 172.21.15.16:0/2238845409 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e68130260 0x7f9e681c8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:24.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.235+0000 7f9e67fff700 1 -- 172.21.15.16:0/2238845409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e6812f8b0 msgr2=0x7f9e6809be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:24.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.235+0000 7f9e67fff700 1 --2- 172.21.15.16:0/2238845409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e6812f8b0 0x7f9e6809be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:24.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.235+0000 7f9e67fff700 1 -- 172.21.15.16:0/2238845409 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e580094a0 con 0x7f9e6812cf60 2023-09-30T22:33:24.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.235+0000 7f9e677fe700 1 --2- 172.21.15.16:0/2238845409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e6812f8b0 0x7f9e6809be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:24.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.235+0000 7f9e67fff700 1 --2- 172.21.15.16:0/2238845409 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e6812cf60 0x7f9e6809b940 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f9e580090b0 tx=0x7f9e58004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:24.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.235+0000 7f9e64ff9700 1 -- 172.21.15.16:0/2238845409 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e58004480 con 0x7f9e6812cf60 2023-09-30T22:33:24.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.235+0000 7f9e64ff9700 1 -- 172.21.15.16:0/2238845409 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e5801f070 con 0x7f9e6812cf60 2023-09-30T22:33:24.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.235+0000 7f9e64ff9700 1 -- 172.21.15.16:0/2238845409 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e580038e0 con 0x7f9e6812cf60 2023-09-30T22:33:24.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.235+0000 7f9e6e96a700 1 -- 172.21.15.16:0/2238845409 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e6809c3c0 con 0x7f9e6812cf60 2023-09-30T22:33:24.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.235+0000 7f9e6e96a700 1 -- 172.21.15.16:0/2238845409 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e6809b430 con 0x7f9e6812cf60 2023-09-30T22:33:24.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.236+0000 7f9e64ff9700 1 -- 172.21.15.16:0/2238845409 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9e58021030 con 0x7f9e6812cf60 2023-09-30T22:33:24.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.236+0000 7f9e64ff9700 1 -- 172.21.15.16:0/2238845409 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f9e58040ba0 con 0x7f9e6812cf60 2023-09-30T22:33:24.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.237+0000 7f9e6e96a700 1 -- 172.21.15.16:0/2238845409 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e68074ea0 con 0x7f9e6812cf60 2023-09-30T22:33:24.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.244+0000 7f9e64ff9700 1 -- 172.21.15.16:0/2238845409 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9e5801a7d0 con 0x7f9e6812cf60 2023-09-30T22:33:24.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.318+0000 7f53c1289700 1 Processor -- start 2023-09-30T22:33:24.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.319+0000 7f53c1289700 1 -- start start 2023-09-30T22:33:24.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.319+0000 7f53c1289700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f53bc09d6f0 0x7f53bc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:24.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.319+0000 7f53c1289700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f53bc09e000 0x7f53bc1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:24.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.319+0000 7f53c1289700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53bc09b120 con 0x7f53bc09e410 2023-09-30T22:33:24.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.319+0000 7f53c1289700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53bc06b460 con 0x7f53bc09e000 2023-09-30T22:33:24.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.319+0000 7f53c1289700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f53bc09a7e0 con 0x7f53bc09d6f0 2023-09-30T22:33:24.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.320+0000 7f53ba7fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f53bc09e000 0x7f53bc1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:24.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.320+0000 7f53bb7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f53bc09e410 0x7f53bc133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54774/0 (socket says 172.21.15.16:54774) 2023-09-30T22:33:24.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.320+0000 7f53bb7fe700 1 -- 172.21.15.16:0/3257632108 learned_addr learned my addr 172.21.15.16:0/3257632108 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:24.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.320+0000 7f53baffd700 1 --2- 172.21.15.16:0/3257632108 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f53bc09d6f0 0x7f53bc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:24.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.320+0000 7f53ba7fc700 1 -- 172.21.15.16:0/3257632108 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f53bc09d6f0 msgr2=0x7f53bc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:24.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.320+0000 7f53ba7fc700 1 --2- 172.21.15.16:0/3257632108 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f53bc09d6f0 0x7f53bc09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:24.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.320+0000 7f53ba7fc700 1 -- 172.21.15.16:0/3257632108 >> v1:172.21.15.16:6789/0 conn(0x7f53bc09e410 legacy=0x7f53bc133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:24.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.320+0000 7f53ba7fc700 1 -- 172.21.15.16:0/3257632108 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53bc09a500 con 0x7f53bc09e000 2023-09-30T22:33:24.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.320+0000 7f53ba7fc700 1 --2- 172.21.15.16:0/3257632108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53bc09e000 0x7f53bc1339f0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f53b0009840 tx=0x7f53b0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14d7f283078579e6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:24.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.320+0000 7f53b9ffb700 1 -- 172.21.15.16:0/3257632108 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b0004230 con 0x7f53bc09e000 2023-09-30T22:33:24.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.321+0000 7f53b9ffb700 1 -- 172.21.15.16:0/3257632108 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53b0004390 con 0x7f53bc09e000 2023-09-30T22:33:24.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.321+0000 7f53b9ffb700 1 -- 172.21.15.16:0/3257632108 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b00036a0 con 0x7f53bc09e000 2023-09-30T22:33:24.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.321+0000 7f53c1289700 1 -- 172.21.15.16:0/3257632108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53bc09e000 msgr2=0x7f53bc1339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:24.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.321+0000 7f53c1289700 1 --2- 172.21.15.16:0/3257632108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53bc09e000 0x7f53bc1339f0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f53b0009840 tx=0x7f53b0009b50 comp rx=0 tx=0).stop 2023-09-30T22:33:24.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.321+0000 7f53c1289700 1 -- 172.21.15.16:0/3257632108 shutdown_connections 2023-09-30T22:33:24.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.321+0000 7f53c1289700 1 --2- 172.21.15.16:0/3257632108 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f53bc09d6f0 0x7f53bc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:24.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.321+0000 7f53c1289700 1 --2- 172.21.15.16:0/3257632108 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53bc09e000 0x7f53bc1339f0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:24.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.321+0000 7f53c1289700 1 -- 172.21.15.16:0/3257632108 >> 172.21.15.16:0/3257632108 conn(0x7f53bc066f50 msgr2=0x7f53bc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:24.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.321+0000 7f53c1289700 1 -- 172.21.15.16:0/3257632108 shutdown_connections 2023-09-30T22:33:24.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.322+0000 7f53c1289700 1 -- 172.21.15.16:0/3257632108 wait complete. 2023-09-30T22:33:24.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.323+0000 7f53c1289700 1 Processor -- start 2023-09-30T22:33:24.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.323+0000 7f53c1289700 1 -- start start 2023-09-30T22:33:24.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.323+0000 7f53c1289700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53bc09d6f0 0x7f53bc1a5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:24.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.323+0000 7f53c1289700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f53bc09e000 0x7f53bc1a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:24.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.323+0000 7f53c1289700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f53bc09e410 0x7f53bc1a5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:24.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.323+0000 7f53c1289700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53bc1a7a10 con 0x7f53bc09d6f0 2023-09-30T22:33:24.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.323+0000 7f53c1289700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53bc0a30d0 con 0x7f53bc09e000 2023-09-30T22:33:24.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.323+0000 7f53c1289700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f53bc09c710 con 0x7f53bc09e410 2023-09-30T22:33:24.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.324+0000 7f53ba7fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f53bc09e000 0x7f53bc1a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:24.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.324+0000 7f53ba7fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f53bc09e000 0x7f53bc1a55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54680/0 (socket says 172.21.15.16:54680) 2023-09-30T22:33:24.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.324+0000 7f53ba7fc700 1 -- 172.21.15.16:0/3500876793 learned_addr learned my addr 172.21.15.16:0/3500876793 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:24.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.324+0000 7f53baffd700 1 --2- 172.21.15.16:0/3500876793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53bc09d6f0 0x7f53bc1a5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:24.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.324+0000 7f53bb7fe700 1 --2- 172.21.15.16:0/3500876793 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f53bc09e410 0x7f53bc1a5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:24.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.324+0000 7f53ba7fc700 1 -- 172.21.15.16:0/3500876793 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f53bc09e410 msgr2=0x7f53bc1a5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:24.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.324+0000 7f53ba7fc700 1 --2- 172.21.15.16:0/3500876793 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f53bc09e410 0x7f53bc1a5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:24.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.324+0000 7f53ba7fc700 1 -- 172.21.15.16:0/3500876793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53bc09d6f0 msgr2=0x7f53bc1a5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:24.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.324+0000 7f53ba7fc700 1 --2- 172.21.15.16:0/3500876793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53bc09d6f0 0x7f53bc1a5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:24.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.324+0000 7f53ba7fc700 1 -- 172.21.15.16:0/3500876793 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b00094a0 con 0x7f53bc09e000 2023-09-30T22:33:24.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.324+0000 7f53bb7fe700 1 --2- 172.21.15.16:0/3500876793 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f53bc09e410 0x7f53bc1a5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:33:24.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.324+0000 7f53ba7fc700 1 --2- 172.21.15.16:0/3500876793 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f53bc09e000 0x7f53bc1a55d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f53b0000c00 tx=0x7f53b0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:24.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.325+0000 7f53a37fe700 1 -- 172.21.15.16:0/3500876793 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b0003e20 con 0x7f53bc09e000 2023-09-30T22:33:24.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.325+0000 7f53a37fe700 1 -- 172.21.15.16:0/3500876793 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53b001e070 con 0x7f53bc09e000 2023-09-30T22:33:24.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.325+0000 7f53c1289700 1 -- 172.21.15.16:0/3500876793 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53bc1b84b0 con 0x7f53bc09e000 2023-09-30T22:33:24.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.325+0000 7f53a37fe700 1 -- 172.21.15.16:0/3500876793 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b0011870 con 0x7f53bc09e000 2023-09-30T22:33:24.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.325+0000 7f53c1289700 1 -- 172.21.15.16:0/3500876793 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53bc1a92b0 con 0x7f53bc09e000 2023-09-30T22:33:24.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.326+0000 7f53a37fe700 1 -- 172.21.15.16:0/3500876793 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f53b00119d0 con 0x7f53bc09e000 2023-09-30T22:33:24.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.326+0000 7f53c1289700 1 -- 172.21.15.16:0/3500876793 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53bc09d1b0 con 0x7f53bc09e000 2023-09-30T22:33:24.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.326+0000 7f53a37fe700 1 -- 172.21.15.16:0/3500876793 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f53b0040910 con 0x7f53bc09e000 2023-09-30T22:33:24.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.335+0000 7f53a37fe700 1 -- 172.21.15.16:0/3500876793 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f53b00239a0 con 0x7f53bc09e000 2023-09-30T22:33:24.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.359+0000 7f3170833700 1 Processor -- start 2023-09-30T22:33:24.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.359+0000 7f3170833700 1 -- start start 2023-09-30T22:33:24.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.360+0000 7f3170833700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f316c09e930 0x7f316c09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:24.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.360+0000 7f3170833700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f316c09f240 0x7f316c1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:24.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.360+0000 7f3170833700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f316c070940 con 0x7f316c09f650 2023-09-30T22:33:24.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.360+0000 7f3170833700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f316c06fd60 con 0x7f316c09e930 2023-09-30T22:33:24.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.360+0000 7f3170833700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f316c06fa80 con 0x7f316c09f240 2023-09-30T22:33:24.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.361+0000 7f3169d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f316c09f240 0x7f316c1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:24.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.361+0000 7f3169d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f316c09f240 0x7f316c1317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:36304/0 (socket says 172.21.15.16:36304) 2023-09-30T22:33:24.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.361+0000 7f316a59c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f316c09e930 0x7f316c09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:24.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.361+0000 7f3169d9b700 1 -- 172.21.15.16:0/1390988961 learned_addr learned my addr 172.21.15.16:0/1390988961 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:24.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.361+0000 7f316ad9d700 1 --1- 172.21.15.16:0/1390988961 >> v1:172.21.15.16:6789/0 conn(0x7f316c09f650 0x7f316c131d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54784/0 (socket says 172.21.15.16:54784) 2023-09-30T22:33:24.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.361+0000 7f316a59c700 1 -- 172.21.15.16:0/1390988961 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f316c09f240 msgr2=0x7f316c1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:24.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.361+0000 7f316a59c700 1 --2- 172.21.15.16:0/1390988961 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f316c09f240 0x7f316c1317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:24.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.361+0000 7f316a59c700 1 -- 172.21.15.16:0/1390988961 >> v1:172.21.15.16:6789/0 conn(0x7f316c09f650 legacy=0x7f316c131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:24.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.361+0000 7f316a59c700 1 -- 172.21.15.16:0/1390988961 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f316c09fb60 con 0x7f316c09e930 2023-09-30T22:33:24.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.361+0000 7f316959a700 1 -- 172.21.15.16:0/1390988961 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1601952629 0 0) 0x7f316c070940 con 0x7f316c09f650 2023-09-30T22:33:24.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.361+0000 7f316a59c700 1 --2- 172.21.15.16:0/1390988961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f316c09e930 0x7f316c09ed00 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f315c009840 tx=0x7f315c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e73e0b8197d65744 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:24.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.361+0000 7f316959a700 1 -- 172.21.15.16:0/1390988961 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315c011830 con 0x7f316c09e930 2023-09-30T22:33:24.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.362+0000 7f316959a700 1 -- 172.21.15.16:0/1390988961 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f315c011990 con 0x7f316c09e930 2023-09-30T22:33:24.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.362+0000 7f316959a700 1 -- 172.21.15.16:0/1390988961 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315c011d20 con 0x7f316c09e930 2023-09-30T22:33:24.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.362+0000 7f3170833700 1 -- 172.21.15.16:0/1390988961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f316c09e930 msgr2=0x7f316c09ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:24.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.362+0000 7f3170833700 1 --2- 172.21.15.16:0/1390988961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f316c09e930 0x7f316c09ed00 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f315c009840 tx=0x7f315c009b50 comp rx=0 tx=0).stop 2023-09-30T22:33:24.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.363+0000 7f3170833700 1 -- 172.21.15.16:0/1390988961 shutdown_connections 2023-09-30T22:33:24.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.363+0000 7f3170833700 1 --2- 172.21.15.16:0/1390988961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f316c09e930 0x7f316c09ed00 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:24.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.363+0000 7f3170833700 1 --2- 172.21.15.16:0/1390988961 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f316c09f240 0x7f316c1317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:24.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.363+0000 7f3170833700 1 -- 172.21.15.16:0/1390988961 >> 172.21.15.16:0/1390988961 conn(0x7f316c0a4210 msgr2=0x7f316c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:24.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.363+0000 7f3170833700 1 -- 172.21.15.16:0/1390988961 shutdown_connections 2023-09-30T22:33:24.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.363+0000 7f3170833700 1 -- 172.21.15.16:0/1390988961 wait complete. 2023-09-30T22:33:24.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.364+0000 7f3170833700 1 Processor -- start 2023-09-30T22:33:24.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.364+0000 7f3170833700 1 -- start start 2023-09-30T22:33:24.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.365+0000 7efeb52f9700 1 Processor -- start 2023-09-30T22:33:24.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.365+0000 7efeb52f9700 1 -- start start 2023-09-30T22:33:24.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.365+0000 7f3170833700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f316c09e930 0x7f316c1a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:24.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.365+0000 7f3170833700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f316c09f240 0x7f316c1a2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:24.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.365+0000 7f3170833700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f316c09f650 0x7f316c1c5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:24.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.365+0000 7f3170833700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316c1a4ae0 con 0x7f316c09f650 2023-09-30T22:33:24.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.365+0000 7f3170833700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f316c1b5e60 con 0x7f316c09f240 2023-09-30T22:33:24.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.365+0000 7f3170833700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f316c1a1990 con 0x7f316c09e930 2023-09-30T22:33:24.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.365+0000 7efeb52f9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efeb009d6f0 0x7efeb009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:24.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.365+0000 7efeb52f9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efeb009e000 0x7efeb01339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:24.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.365+0000 7efeb52f9700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efeb009b120 con 0x7efeb009e410 2023-09-30T22:33:24.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.366+0000 7efeb52f9700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeb006b460 con 0x7efeb009d6f0 2023-09-30T22:33:24.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.366+0000 7efeb52f9700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efeb009a7e0 con 0x7efeb009e000 2023-09-30T22:33:24.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.366+0000 7efeae7fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efeb009e000 0x7efeb01339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:24.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.366+0000 7efeaeffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efeb009d6f0 0x7efeb009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:24.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.366+0000 7efeae7fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efeb009e000 0x7efeb01339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:36320/0 (socket says 172.21.15.16:36320) 2023-09-30T22:33:24.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.366+0000 7efeae7fc700 1 -- 172.21.15.16:0/2140714869 learned_addr learned my addr 172.21.15.16:0/2140714869 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:24.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.366+0000 7f316ad9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f316c09f650 0x7f316c1c5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:24.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.366+0000 7f316ad9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f316c09f650 0x7f316c1c5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41594/0 (socket says 172.21.15.16:41594) 2023-09-30T22:33:24.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.366+0000 7f316ad9d700 1 -- 172.21.15.16:0/770855161 learned_addr learned my addr 172.21.15.16:0/770855161 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:24.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.366+0000 7f316a59c700 1 --2- 172.21.15.16:0/770855161 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f316c09e930 0x7f316c1a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:24.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.366+0000 7efeaeffd700 1 -- 172.21.15.16:0/2140714869 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efeb009e000 msgr2=0x7efeb01339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:24.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.366+0000 7efeaeffd700 1 --2- 172.21.15.16:0/2140714869 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efeb009e000 0x7efeb01339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:24.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.366+0000 7efeaeffd700 1 -- 172.21.15.16:0/2140714869 >> v1:172.21.15.16:6789/0 conn(0x7efeb009e410 legacy=0x7efeb0133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:24.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.366+0000 7efeaeffd700 1 -- 172.21.15.16:0/2140714869 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb009a500 con 0x7efeb009d6f0 2023-09-30T22:33:24.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.366+0000 7efeaeffd700 1 --2- 172.21.15.16:0/2140714869 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efeb009d6f0 0x7efeb009dac0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7efea0009870 tx=0x7efea0009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91c7f7ce779eae45 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:24.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.367+0000 7f316a59c700 1 -- 172.21.15.16:0/770855161 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f316c09f240 msgr2=0x7f316c1a2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:24.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.367+0000 7f316a59c700 1 --2- 172.21.15.16:0/770855161 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f316c09f240 0x7f316c1a2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:24.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.367+0000 7f316a59c700 1 -- 172.21.15.16:0/770855161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f316c09f650 msgr2=0x7f316c1c5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:24.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.367+0000 7f316a59c700 1 --2- 172.21.15.16:0/770855161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f316c09f650 0x7f316c1c5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:24.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.367+0000 7f316a59c700 1 -- 172.21.15.16:0/770855161 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f315c0094a0 con 0x7f316c09e930 2023-09-30T22:33:24.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.367+0000 7efeadffb700 1 -- 172.21.15.16:0/2140714869 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea0004230 con 0x7efeb009d6f0 2023-09-30T22:33:24.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.367+0000 7f316a59c700 1 --2- 172.21.15.16:0/770855161 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f316c09e930 0x7f316c1a1f30 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f315c016000 tx=0x7f315c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:24.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.367+0000 7efeadffb700 1 -- 172.21.15.16:0/2140714869 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efea0004390 con 0x7efeb009d6f0 2023-09-30T22:33:24.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.367+0000 7efeadffb700 1 -- 172.21.15.16:0/2140714869 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea00036a0 con 0x7efeb009d6f0 2023-09-30T22:33:24.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.367+0000 7f315affd700 1 -- 172.21.15.16:0/770855161 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315c004480 con 0x7f316c09e930 2023-09-30T22:33:24.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.367+0000 7f315affd700 1 -- 172.21.15.16:0/770855161 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f315c023070 con 0x7f316c09e930 2023-09-30T22:33:24.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.367+0000 7f3170833700 1 -- 172.21.15.16:0/770855161 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f316c0a0190 con 0x7f316c09e930 2023-09-30T22:33:24.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.367+0000 7f315affd700 1 -- 172.21.15.16:0/770855161 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315c01fa60 con 0x7f316c09e930 2023-09-30T22:33:24.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.367+0000 7f3170833700 1 -- 172.21.15.16:0/770855161 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f316c1a4d60 con 0x7f316c09e930 2023-09-30T22:33:24.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.368+0000 7efeb52f9700 1 -- 172.21.15.16:0/2140714869 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efeb009d6f0 msgr2=0x7efeb009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:24.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.368+0000 7efeb52f9700 1 --2- 172.21.15.16:0/2140714869 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efeb009d6f0 0x7efeb009dac0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7efea0009870 tx=0x7efea0009b80 comp rx=0 tx=0).stop 2023-09-30T22:33:24.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.368+0000 7efeb52f9700 1 -- 172.21.15.16:0/2140714869 shutdown_connections 2023-09-30T22:33:24.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.368+0000 7efeb52f9700 1 --2- 172.21.15.16:0/2140714869 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efeb009d6f0 0x7efeb009dac0 secure :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7efea0009870 tx=0x7efea0009b80 comp rx=0 tx=0).stop 2023-09-30T22:33:24.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.368+0000 7efeb52f9700 1 --2- 172.21.15.16:0/2140714869 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efeb009e000 0x7efeb01339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:24.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.368+0000 7efeb52f9700 1 -- 172.21.15.16:0/2140714869 >> 172.21.15.16:0/2140714869 conn(0x7efeb0066f50 msgr2=0x7efeb009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:24.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.368+0000 7f315affd700 1 -- 172.21.15.16:0/770855161 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f315c025030 con 0x7f316c09e930 2023-09-30T22:33:24.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.368+0000 7f315affd700 1 -- 172.21.15.16:0/770855161 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f315c044c50 con 0x7f316c09e930 2023-09-30T22:33:24.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.368+0000 7efeb52f9700 1 -- 172.21.15.16:0/2140714869 shutdown_connections 2023-09-30T22:33:24.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.369+0000 7efeb52f9700 1 -- 172.21.15.16:0/2140714869 wait complete. 2023-09-30T22:33:24.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.369+0000 7f3170833700 1 -- 172.21.15.16:0/770855161 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f316c1a29b0 con 0x7f316c09e930 2023-09-30T22:33:24.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.370+0000 7efeb52f9700 1 Processor -- start 2023-09-30T22:33:24.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.370+0000 7efeb52f9700 1 -- start start 2023-09-30T22:33:24.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.370+0000 7efeb52f9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efeb009e000 0x7efeb0122980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:24.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.370+0000 7efeb52f9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efeb009e410 0x7efeb0122ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:24.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.370+0000 7efeb52f9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efeb0123400 0x7efeb01c8750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:24.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.370+0000 7efeb52f9700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb009c780 con 0x7efeb009e000 2023-09-30T22:33:24.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.370+0000 7efeb52f9700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeb0124710 con 0x7efeb0123400 2023-09-30T22:33:24.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.370+0000 7efeb52f9700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efeb00a2f60 con 0x7efeb009e410 2023-09-30T22:33:24.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.370+0000 7efeaeffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efeb009e000 0x7efeb0122980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:24.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.370+0000 7efeaeffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efeb009e000 0x7efeb0122980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41608/0 (socket says 172.21.15.16:41608) 2023-09-30T22:33:24.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.370+0000 7efeaeffd700 1 -- 172.21.15.16:0/110868958 learned_addr learned my addr 172.21.15.16:0/110868958 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:24.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.370+0000 7efeaf7fe700 1 --2- 172.21.15.16:0/110868958 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efeb0123400 0x7efeb01c8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:24.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.371+0000 7efeae7fc700 1 --2- 172.21.15.16:0/110868958 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efeb009e410 0x7efeb0122ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:24.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.371+0000 7efeaeffd700 1 -- 172.21.15.16:0/110868958 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efeb009e410 msgr2=0x7efeb0122ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:24.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.371+0000 7efeaeffd700 1 --2- 172.21.15.16:0/110868958 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efeb009e410 0x7efeb0122ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:24.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.371+0000 7efeaeffd700 1 -- 172.21.15.16:0/110868958 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efeb0123400 msgr2=0x7efeb01c8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:24.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.371+0000 7efeaeffd700 1 --2- 172.21.15.16:0/110868958 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efeb0123400 0x7efeb01c8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:24.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.371+0000 7efeaeffd700 1 -- 172.21.15.16:0/110868958 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea00094a0 con 0x7efeb009e000 2023-09-30T22:33:24.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.371+0000 7efeaeffd700 1 --2- 172.21.15.16:0/110868958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efeb009e000 0x7efeb0122980 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7efeb009e9b0 tx=0x7efea0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:24.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.371+0000 7efe977fe700 1 -- 172.21.15.16:0/110868958 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea0003cb0 con 0x7efeb009e000 2023-09-30T22:33:24.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.371+0000 7efe977fe700 1 -- 172.21.15.16:0/110868958 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efea001e070 con 0x7efeb009e000 2023-09-30T22:33:24.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.371+0000 7efe977fe700 1 -- 172.21.15.16:0/110868958 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea0011870 con 0x7efeb009e000 2023-09-30T22:33:24.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.371+0000 7efeb52f9700 1 -- 172.21.15.16:0/110868958 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeb01264a0 con 0x7efeb009e000 2023-09-30T22:33:24.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.372+0000 7efeb52f9700 1 -- 172.21.15.16:0/110868958 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeb01334a0 con 0x7efeb009e000 2023-09-30T22:33:24.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.372+0000 7efe977fe700 1 -- 172.21.15.16:0/110868958 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7efea00119d0 con 0x7efeb009e000 2023-09-30T22:33:24.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.373+0000 7efe977fe700 1 -- 172.21.15.16:0/110868958 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7efea0040690 con 0x7efeb009e000 2023-09-30T22:33:24.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.373+0000 7efeb52f9700 1 -- 172.21.15.16:0/110868958 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efeb0074ea0 con 0x7efeb009e000 2023-09-30T22:33:24.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.375+0000 7f315affd700 1 -- 172.21.15.16:0/770855161 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f315c015080 con 0x7f316c09e930 2023-09-30T22:33:24.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.382+0000 7efe977fe700 1 -- 172.21.15.16:0/110868958 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7efea0010970 con 0x7efeb009e000 2023-09-30T22:33:24.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.482+0000 7f9e6e96a700 1 -- 172.21.15.16:0/2238845409 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9e680713a0 con 0x7f9e6812cf60 2023-09-30T22:33:24.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.483+0000 7f9e64ff9700 1 -- 172.21.15.16:0/2238845409 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9e58019ba0 con 0x7f9e6812cf60 2023-09-30T22:33:24.484 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:33:24.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.485+0000 7f9e6e96a700 1 -- 172.21.15.16:0/2238845409 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e6812cf60 msgr2=0x7f9e6809b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:24.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.485+0000 7f9e6e96a700 1 --2- 172.21.15.16:0/2238845409 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e6812cf60 0x7f9e6809b940 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f9e580090b0 tx=0x7f9e58004750 comp rx=0 tx=0).stop 2023-09-30T22:33:24.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.485+0000 7f9e6e96a700 1 -- 172.21.15.16:0/2238845409 shutdown_connections 2023-09-30T22:33:24.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.485+0000 7f9e6e96a700 1 --2- 172.21.15.16:0/2238845409 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e6812cf60 0x7f9e6809b940 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:24.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.485+0000 7f9e6e96a700 1 --2- 172.21.15.16:0/2238845409 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e68130260 0x7f9e681c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:24.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.485+0000 7f9e6e96a700 1 --2- 172.21.15.16:0/2238845409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e6812f8b0 0x7f9e6809be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:24.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.485+0000 7f9e6e96a700 1 -- 172.21.15.16:0/2238845409 >> 172.21.15.16:0/2238845409 conn(0x7f9e680a2aa0 msgr2=0x7f9e681b7990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:24.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.485+0000 7f9e6e96a700 1 -- 172.21.15.16:0/2238845409 shutdown_connections 2023-09-30T22:33:24.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.485+0000 7f9e6e96a700 1 -- 172.21.15.16:0/2238845409 wait complete. 2023-09-30T22:33:24.506 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:33:24.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.559+0000 7f53c1289700 1 -- 172.21.15.16:0/3500876793 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f53bc074ea0 con 0x7f53bc09e000 2023-09-30T22:33:24.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.560+0000 7f53a37fe700 1 -- 172.21.15.16:0/3500876793 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f53b00193c0 con 0x7f53bc09e000 2023-09-30T22:33:24.561 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:33:24.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.562+0000 7f53c1289700 1 -- 172.21.15.16:0/3500876793 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f53bc09e000 msgr2=0x7f53bc1a55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:24.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.562+0000 7f53c1289700 1 --2- 172.21.15.16:0/3500876793 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f53bc09e000 0x7f53bc1a55d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f53b0000c00 tx=0x7f53b0004750 comp rx=0 tx=0).stop 2023-09-30T22:33:24.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.562+0000 7f53c1289700 1 -- 172.21.15.16:0/3500876793 shutdown_connections 2023-09-30T22:33:24.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.562+0000 7f53c1289700 1 --2- 172.21.15.16:0/3500876793 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f53bc09e410 0x7f53bc1a5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:24.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.562+0000 7f53c1289700 1 --2- 172.21.15.16:0/3500876793 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53bc09d6f0 0x7f53bc1a5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:24.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.562+0000 7f53c1289700 1 --2- 172.21.15.16:0/3500876793 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f53bc09e000 0x7f53bc1a55d0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:24.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.562+0000 7f53c1289700 1 -- 172.21.15.16:0/3500876793 >> 172.21.15.16:0/3500876793 conn(0x7f53bc066f50 msgr2=0x7f53bc1b9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:24.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.562+0000 7f53c1289700 1 -- 172.21.15.16:0/3500876793 shutdown_connections 2023-09-30T22:33:24.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.562+0000 7f53c1289700 1 -- 172.21.15.16:0/3500876793 wait complete. 2023-09-30T22:33:24.577 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:33:24.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.590+0000 7efeb52f9700 1 -- 172.21.15.16:0/110868958 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7efeb0072770 con 0x7efeb009e000 2023-09-30T22:33:24.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.590+0000 7efe977fe700 1 -- 172.21.15.16:0/110868958 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7efea0010380 con 0x7efeb009e000 2023-09-30T22:33:24.591 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:33:24.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.592+0000 7efeb52f9700 1 -- 172.21.15.16:0/110868958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efeb009e000 msgr2=0x7efeb0122980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:24.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.592+0000 7efeb52f9700 1 --2- 172.21.15.16:0/110868958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efeb009e000 0x7efeb0122980 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7efeb009e9b0 tx=0x7efea0004750 comp rx=0 tx=0).stop 2023-09-30T22:33:24.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.592+0000 7efeb52f9700 1 -- 172.21.15.16:0/110868958 shutdown_connections 2023-09-30T22:33:24.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.592+0000 7efeb52f9700 1 --2- 172.21.15.16:0/110868958 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efeb0123400 0x7efeb01c8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:24.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.592+0000 7efeb52f9700 1 --2- 172.21.15.16:0/110868958 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efeb009e000 0x7efeb0122980 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:24.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.592+0000 7efeb52f9700 1 --2- 172.21.15.16:0/110868958 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efeb009e410 0x7efeb0122ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:24.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.592+0000 7efeb52f9700 1 -- 172.21.15.16:0/110868958 >> 172.21.15.16:0/110868958 conn(0x7efeb0066f50 msgr2=0x7efeb00712f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:24.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.593+0000 7efeb52f9700 1 -- 172.21.15.16:0/110868958 shutdown_connections 2023-09-30T22:33:24.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.593+0000 7efeb52f9700 1 -- 172.21.15.16:0/110868958 wait complete. 2023-09-30T22:33:24.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.595+0000 7f3170833700 1 -- 172.21.15.16:0/770855161 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f316c1a4d60 con 0x7f316c09e930 2023-09-30T22:33:24.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.596+0000 7f315affd700 1 -- 172.21.15.16:0/770855161 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f316c1a4d60 con 0x7f316c09e930 2023-09-30T22:33:24.597 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:33:24.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.598+0000 7f3170833700 1 -- 172.21.15.16:0/770855161 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f316c09e930 msgr2=0x7f316c1a1f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:24.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.598+0000 7f3170833700 1 --2- 172.21.15.16:0/770855161 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f316c09e930 0x7f316c1a1f30 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f315c016000 tx=0x7f315c004750 comp rx=0 tx=0).stop 2023-09-30T22:33:24.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.598+0000 7f3170833700 1 -- 172.21.15.16:0/770855161 shutdown_connections 2023-09-30T22:33:24.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.598+0000 7f3170833700 1 --2- 172.21.15.16:0/770855161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f316c09f650 0x7f316c1c5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:24.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.598+0000 7f3170833700 1 --2- 172.21.15.16:0/770855161 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f316c09e930 0x7f316c1a1f30 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:24.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.598+0000 7f3170833700 1 --2- 172.21.15.16:0/770855161 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f316c09f240 0x7f316c1a2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:24.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.598+0000 7f3170833700 1 -- 172.21.15.16:0/770855161 >> 172.21.15.16:0/770855161 conn(0x7f316c0a4210 msgr2=0x7f316c09bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:24.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.598+0000 7f3170833700 1 -- 172.21.15.16:0/770855161 shutdown_connections 2023-09-30T22:33:24.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:24.598+0000 7f3170833700 1 -- 172.21.15.16:0/770855161 wait complete. 2023-09-30T22:33:24.608 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:33:24.614 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:33:25.507 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:33:25.578 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:33:25.609 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:33:25.614 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:33:25.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.656+0000 7f063d4c5700 1 Processor -- start 2023-09-30T22:33:25.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.656+0000 7f063d4c5700 1 -- start start 2023-09-30T22:33:25.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.657+0000 7f063d4c5700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f063806f5d0 0x7f063812ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:25.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.657+0000 7f063d4c5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0638135c20 0x7f0638138000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:25.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.657+0000 7f063d4c5700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f063806fa40 con 0x7f063812cf60 2023-09-30T22:33:25.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.657+0000 7f063d4c5700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f063812cd90 con 0x7f063806f5d0 2023-09-30T22:33:25.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.657+0000 7f063d4c5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0638071780 con 0x7f0638135c20 2023-09-30T22:33:25.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.657+0000 7f06367fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f063806f5d0 0x7f063812ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:25.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.657+0000 7f06367fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f063806f5d0 0x7f063812ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41622/0 (socket says 172.21.15.16:41622) 2023-09-30T22:33:25.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.657+0000 7f06367fc700 1 -- 172.21.15.16:0/763121803 learned_addr learned my addr 172.21.15.16:0/763121803 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:25.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.657+0000 7f0636ffd700 1 --1- 172.21.15.16:0/763121803 >> v1:172.21.15.16:6789/0 conn(0x7f063812cf60 0x7f063812f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54806/0 (socket says 172.21.15.16:54806) 2023-09-30T22:33:25.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.657+0000 7f06377fe700 1 --2- 172.21.15.16:0/763121803 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0638135c20 0x7f0638138000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:25.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.658+0000 7f06367fc700 1 -- 172.21.15.16:0/763121803 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0638135c20 msgr2=0x7f0638138000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:25.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.658+0000 7f06367fc700 1 --2- 172.21.15.16:0/763121803 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0638135c20 0x7f0638138000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:25.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.658+0000 7f06367fc700 1 -- 172.21.15.16:0/763121803 >> v1:172.21.15.16:6789/0 conn(0x7f063812cf60 legacy=0x7f063812f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:25.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.658+0000 7f06367fc700 1 -- 172.21.15.16:0/763121803 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0638071a40 con 0x7f063806f5d0 2023-09-30T22:33:25.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.658+0000 7f06367fc700 1 --2- 172.21.15.16:0/763121803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f063806f5d0 0x7f063812ffc0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f062c009920 tx=0x7f062c009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=313dcca8db9b6d8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:25.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.658+0000 7f0635ffb700 1 -- 172.21.15.16:0/763121803 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c011830 con 0x7f063806f5d0 2023-09-30T22:33:25.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.658+0000 7f0635ffb700 1 -- 172.21.15.16:0/763121803 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f062c011990 con 0x7f063806f5d0 2023-09-30T22:33:25.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.658+0000 7f0635ffb700 1 -- 172.21.15.16:0/763121803 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c011d20 con 0x7f063806f5d0 2023-09-30T22:33:25.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.659+0000 7f063d4c5700 1 -- 172.21.15.16:0/763121803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f063806f5d0 msgr2=0x7f063812ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:25.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.659+0000 7f063d4c5700 1 --2- 172.21.15.16:0/763121803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f063806f5d0 0x7f063812ffc0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f062c009920 tx=0x7f062c009c30 comp rx=0 tx=0).stop 2023-09-30T22:33:25.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.659+0000 7f063d4c5700 1 -- 172.21.15.16:0/763121803 shutdown_connections 2023-09-30T22:33:25.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.659+0000 7f063d4c5700 1 --2- 172.21.15.16:0/763121803 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0638135c20 0x7f0638138000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:25.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.659+0000 7f063d4c5700 1 --2- 172.21.15.16:0/763121803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f063806f5d0 0x7f063812ffc0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:25.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.659+0000 7f063d4c5700 1 -- 172.21.15.16:0/763121803 >> 172.21.15.16:0/763121803 conn(0x7f06380a2aa0 msgr2=0x7f06380a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:25.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.659+0000 7f063d4c5700 1 -- 172.21.15.16:0/763121803 shutdown_connections 2023-09-30T22:33:25.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.659+0000 7f063d4c5700 1 -- 172.21.15.16:0/763121803 wait complete. 2023-09-30T22:33:25.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.661+0000 7f063d4c5700 1 Processor -- start 2023-09-30T22:33:25.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.661+0000 7f063d4c5700 1 -- start start 2023-09-30T22:33:25.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.662+0000 7f063d4c5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f063806f5d0 0x7f0638099e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:25.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.662+0000 7f063d4c5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f063812cf60 0x7f063809a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:25.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.662+0000 7f063d4c5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0638135c20 0x7f063809a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:25.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.662+0000 7f063d4c5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f063809d790 con 0x7f0638135c20 2023-09-30T22:33:25.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.662+0000 7f063d4c5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f063809d970 con 0x7f063806f5d0 2023-09-30T22:33:25.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.662+0000 7f063d4c5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f063809e550 con 0x7f063812cf60 2023-09-30T22:33:25.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.662+0000 7f06367fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f063812cf60 0x7f063809a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:25.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.662+0000 7f0636ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f063806f5d0 0x7f0638099e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:25.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.662+0000 7f0636ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f063806f5d0 0x7f0638099e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54726/0 (socket says 172.21.15.16:54726) 2023-09-30T22:33:25.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.662+0000 7f06367fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f063812cf60 0x7f063809a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:36352/0 (socket says 172.21.15.16:36352) 2023-09-30T22:33:25.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.662+0000 7f0636ffd700 1 -- 172.21.15.16:0/2687350659 learned_addr learned my addr 172.21.15.16:0/2687350659 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:25.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.662+0000 7f06377fe700 1 --2- 172.21.15.16:0/2687350659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0638135c20 0x7f063809a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:25.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.663+0000 7f06377fe700 1 -- 172.21.15.16:0/2687350659 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f063812cf60 msgr2=0x7f063809a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:25.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.663+0000 7f06377fe700 1 --2- 172.21.15.16:0/2687350659 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f063812cf60 0x7f063809a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:25.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.663+0000 7f06377fe700 1 -- 172.21.15.16:0/2687350659 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f063806f5d0 msgr2=0x7f0638099e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:25.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.663+0000 7f06377fe700 1 --2- 172.21.15.16:0/2687350659 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f063806f5d0 0x7f0638099e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:25.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.663+0000 7f06377fe700 1 -- 172.21.15.16:0/2687350659 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f062c009550 con 0x7f0638135c20 2023-09-30T22:33:25.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.663+0000 7f0636ffd700 1 --2- 172.21.15.16:0/2687350659 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f063806f5d0 0x7f0638099e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:33:25.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.663+0000 7f06377fe700 1 --2- 172.21.15.16:0/2687350659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0638135c20 0x7f063809a8d0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f063000b6a0 tx=0x7f063000b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:25.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.663+0000 7f061f7fe700 1 -- 172.21.15.16:0/2687350659 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0630019070 con 0x7f0638135c20 2023-09-30T22:33:25.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.663+0000 7f061f7fe700 1 -- 172.21.15.16:0/2687350659 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0630007540 con 0x7f0638135c20 2023-09-30T22:33:25.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.663+0000 7f063d4c5700 1 -- 172.21.15.16:0/2687350659 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0638129fa0 con 0x7f0638135c20 2023-09-30T22:33:25.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.663+0000 7f061f7fe700 1 -- 172.21.15.16:0/2687350659 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0630009c00 con 0x7f0638135c20 2023-09-30T22:33:25.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.663+0000 7f063d4c5700 1 -- 172.21.15.16:0/2687350659 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f063809e100 con 0x7f0638135c20 2023-09-30T22:33:25.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.664+0000 7f061f7fe700 1 -- 172.21.15.16:0/2687350659 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f06300078d0 con 0x7f0638135c20 2023-09-30T22:33:25.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.665+0000 7f063d4c5700 1 -- 172.21.15.16:0/2687350659 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f063809d5d0 con 0x7f0638135c20 2023-09-30T22:33:25.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.665+0000 7f061f7fe700 1 -- 172.21.15.16:0/2687350659 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f06300479c0 con 0x7f0638135c20 2023-09-30T22:33:25.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.674+0000 7f061f7fe700 1 -- 172.21.15.16:0/2687350659 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f063001c3b0 con 0x7f0638135c20 2023-09-30T22:33:25.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.737+0000 7f1be57ff700 1 Processor -- start 2023-09-30T22:33:25.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.737+0000 7f1be57ff700 1 -- start start 2023-09-30T22:33:25.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.738+0000 7f1be57ff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1be009e930 0x7f1be009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:25.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.738+0000 7f1be57ff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1be009f650 0x7f1be0131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:25.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.738+0000 7f1be57ff700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1be0070940 con 0x7f1be009f240 2023-09-30T22:33:25.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.738+0000 7f1be57ff700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1be006fd60 con 0x7f1be009e930 2023-09-30T22:33:25.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.738+0000 7f1be57ff700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1be006fa80 con 0x7f1be009f650 2023-09-30T22:33:25.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.738+0000 7f1bdeffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1be009e930 0x7f1be009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:25.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.738+0000 7f1bde7fc700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1be009f240 0x7f1be01317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54808/0 (socket says 172.21.15.16:54808) 2023-09-30T22:33:25.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.738+0000 7f1bde7fc700 1 -- 172.21.15.16:0/2302766477 learned_addr learned my addr 172.21.15.16:0/2302766477 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:25.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.738+0000 7f1bdf7fe700 1 --2- 172.21.15.16:0/2302766477 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1be009f650 0x7f1be0131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:25.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.738+0000 7f1bdeffd700 1 -- 172.21.15.16:0/2302766477 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1be009f650 msgr2=0x7f1be0131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:25.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.738+0000 7f1bdeffd700 1 --2- 172.21.15.16:0/2302766477 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1be009f650 0x7f1be0131f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:25.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.738+0000 7f1bdeffd700 1 -- 172.21.15.16:0/2302766477 >> v1:172.21.15.16:6789/0 conn(0x7f1be009f240 legacy=0x7f1be01317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:25.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.738+0000 7f1bdeffd700 1 -- 172.21.15.16:0/2302766477 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1be009fb60 con 0x7f1be009e930 2023-09-30T22:33:25.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.739+0000 7f1bdeffd700 1 --2- 172.21.15.16:0/2302766477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1be009e930 0x7f1be009ed00 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0009840 tx=0x7f1bd0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=897c36994dda6bcb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:25.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.739+0000 7f1bddffb700 1 -- 172.21.15.16:0/2302766477 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd0011830 con 0x7f1be009e930 2023-09-30T22:33:25.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.739+0000 7f1bddffb700 1 -- 172.21.15.16:0/2302766477 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bd0011990 con 0x7f1be009e930 2023-09-30T22:33:25.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.739+0000 7f1bddffb700 1 -- 172.21.15.16:0/2302766477 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd0011d20 con 0x7f1be009e930 2023-09-30T22:33:25.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.740+0000 7f1be57ff700 1 -- 172.21.15.16:0/2302766477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1be009e930 msgr2=0x7f1be009ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:25.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.740+0000 7f1be57ff700 1 --2- 172.21.15.16:0/2302766477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1be009e930 0x7f1be009ed00 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0009840 tx=0x7f1bd0009b50 comp rx=0 tx=0).stop 2023-09-30T22:33:25.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.740+0000 7f1be57ff700 1 -- 172.21.15.16:0/2302766477 shutdown_connections 2023-09-30T22:33:25.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.740+0000 7f1be57ff700 1 --2- 172.21.15.16:0/2302766477 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1be009f650 0x7f1be0131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:25.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.740+0000 7f1be57ff700 1 --2- 172.21.15.16:0/2302766477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1be009e930 0x7f1be009ed00 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:25.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.740+0000 7f1be57ff700 1 -- 172.21.15.16:0/2302766477 >> 172.21.15.16:0/2302766477 conn(0x7f1be00a4210 msgr2=0x7f1be00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:25.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.740+0000 7f1be57ff700 1 -- 172.21.15.16:0/2302766477 shutdown_connections 2023-09-30T22:33:25.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.740+0000 7f1be57ff700 1 -- 172.21.15.16:0/2302766477 wait complete. 2023-09-30T22:33:25.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.742+0000 7f1be57ff700 1 Processor -- start 2023-09-30T22:33:25.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.742+0000 7f1be57ff700 1 -- start start 2023-09-30T22:33:25.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.742+0000 7f1be57ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1be009e930 0x7f1be01a3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:25.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.742+0000 7f1be57ff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1be009f240 0x7f1be01a42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:25.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.742+0000 7f1be57ff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1be009f650 0x7f1be01a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:25.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.742+0000 7f1be57ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be01a7020 con 0x7f1be009e930 2023-09-30T22:33:25.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.742+0000 7f1be57ff700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1be0071d20 con 0x7f1be009f650 2023-09-30T22:33:25.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.742+0000 7f1be57ff700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1be00a3520 con 0x7f1be009f240 2023-09-30T22:33:25.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.743+0000 7f1bdeffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1be009e930 0x7f1be01a3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:25.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.743+0000 7f1bdeffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1be009e930 0x7f1be01a3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41636/0 (socket says 172.21.15.16:41636) 2023-09-30T22:33:25.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.743+0000 7f1bdeffd700 1 -- 172.21.15.16:0/2753709537 learned_addr learned my addr 172.21.15.16:0/2753709537 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:25.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.743+0000 7f1bde7fc700 1 --2- 172.21.15.16:0/2753709537 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1be009f240 0x7f1be01a42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:25.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.743+0000 7f1bdf7fe700 1 --2- 172.21.15.16:0/2753709537 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1be009f650 0x7f1be01a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:25.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.743+0000 7f1bdeffd700 1 -- 172.21.15.16:0/2753709537 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1be009f240 msgr2=0x7f1be01a42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:25.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.743+0000 7f1bdeffd700 1 --2- 172.21.15.16:0/2753709537 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1be009f240 0x7f1be01a42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:25.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.743+0000 7f1bdeffd700 1 -- 172.21.15.16:0/2753709537 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1be009f650 msgr2=0x7f1be01a47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:25.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.743+0000 7f1bdeffd700 1 --2- 172.21.15.16:0/2753709537 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1be009f650 0x7f1be01a47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:25.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.743+0000 7f1bdeffd700 1 -- 172.21.15.16:0/2753709537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bd00094a0 con 0x7f1be009e930 2023-09-30T22:33:25.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.743+0000 7f1bde7fc700 1 --2- 172.21.15.16:0/2753709537 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1be009f240 0x7f1be01a42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:33:25.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.743+0000 7f1bdf7fe700 1 --2- 172.21.15.16:0/2753709537 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1be009f650 0x7f1be01a47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:33:25.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.743+0000 7f1bdeffd700 1 --2- 172.21.15.16:0/2753709537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1be009e930 0x7f1be01a3d70 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0009fd0 tx=0x7f1bd0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:25.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.744+0000 7f1bc77fe700 1 -- 172.21.15.16:0/2753709537 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd00036a0 con 0x7f1be009e930 2023-09-30T22:33:25.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.744+0000 7f1bc77fe700 1 -- 172.21.15.16:0/2753709537 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bd001f070 con 0x7f1be009e930 2023-09-30T22:33:25.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.744+0000 7f1be57ff700 1 -- 172.21.15.16:0/2753709537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1be00a0190 con 0x7f1be009e930 2023-09-30T22:33:25.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.744+0000 7f1be57ff700 1 -- 172.21.15.16:0/2753709537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1be01a6bd0 con 0x7f1be009e930 2023-09-30T22:33:25.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.744+0000 7f1bc77fe700 1 -- 172.21.15.16:0/2753709537 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd0003a60 con 0x7f1be009e930 2023-09-30T22:33:25.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.745+0000 7f1bc77fe700 1 -- 172.21.15.16:0/2753709537 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1bd0003bc0 con 0x7f1be009e930 2023-09-30T22:33:25.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.745+0000 7f1bc77fe700 1 -- 172.21.15.16:0/2753709537 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f1bd0021070 con 0x7f1be009e930 2023-09-30T22:33:25.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.745+0000 7f1be57ff700 1 -- 172.21.15.16:0/2753709537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bc00052e0 con 0x7f1be009e930 2023-09-30T22:33:25.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.750+0000 7f1bc77fe700 1 -- 172.21.15.16:0/2753709537 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1bd0015020 con 0x7f1be009e930 2023-09-30T22:33:25.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.767+0000 7fc4a70a5700 1 Processor -- start 2023-09-30T22:33:25.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.767+0000 7fc4a70a5700 1 -- start start 2023-09-30T22:33:25.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.768+0000 7fc4a70a5700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc4a009e930 0x7fc4a009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:25.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.768+0000 7fc4a70a5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc4a009f650 0x7fc4a0131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:25.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.768+0000 7fc4a70a5700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4a0070940 con 0x7fc4a009f240 2023-09-30T22:33:25.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.768+0000 7fc4a70a5700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4a006fd60 con 0x7fc4a009e930 2023-09-30T22:33:25.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.768+0000 7fc4a70a5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a006fa80 con 0x7fc4a009f650 2023-09-30T22:33:25.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.768+0000 7fc4a4e41700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc4a009e930 0x7fc4a009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:25.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.768+0000 7fc4a4e41700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc4a009e930 0x7fc4a009ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41644/0 (socket says 172.21.15.16:41644) 2023-09-30T22:33:25.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.768+0000 7fc4a4e41700 1 -- 172.21.15.16:0/2675700706 learned_addr learned my addr 172.21.15.16:0/2675700706 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:25.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.768+0000 7fc4a4e41700 1 -- 172.21.15.16:0/2675700706 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc4a009f650 msgr2=0x7fc4a0131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:25.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.768+0000 7fc4a4e41700 1 --2- 172.21.15.16:0/2675700706 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc4a009f650 0x7fc4a0131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:25.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.768+0000 7fc4a4e41700 1 -- 172.21.15.16:0/2675700706 >> v1:172.21.15.16:6789/0 conn(0x7fc4a009f240 legacy=0x7fc4a01317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:25.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.768+0000 7fc4a4e41700 1 -- 172.21.15.16:0/2675700706 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4a009fb60 con 0x7fc4a009e930 2023-09-30T22:33:25.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.768+0000 7fc4a4e41700 1 --2- 172.21.15.16:0/2675700706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4a009e930 0x7fc4a009ed00 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fc4940098f0 tx=0x7fc494009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b50b569fa29d6f80 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:25.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.768+0000 7fc49f7fe700 1 -- 172.21.15.16:0/2675700706 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49400b8a0 con 0x7fc4a009e930 2023-09-30T22:33:25.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.769+0000 7fc49f7fe700 1 -- 172.21.15.16:0/2675700706 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc49400ba00 con 0x7fc4a009e930 2023-09-30T22:33:25.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.769+0000 7fc49f7fe700 1 -- 172.21.15.16:0/2675700706 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49400bd90 con 0x7fc4a009e930 2023-09-30T22:33:25.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.769+0000 7fc4a70a5700 1 -- 172.21.15.16:0/2675700706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4a009e930 msgr2=0x7fc4a009ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:25.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.769+0000 7fc4a70a5700 1 --2- 172.21.15.16:0/2675700706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4a009e930 0x7fc4a009ed00 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fc4940098f0 tx=0x7fc494009c00 comp rx=0 tx=0).stop 2023-09-30T22:33:25.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.769+0000 7fc4a70a5700 1 -- 172.21.15.16:0/2675700706 shutdown_connections 2023-09-30T22:33:25.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.769+0000 7fc4a70a5700 1 --2- 172.21.15.16:0/2675700706 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc4a009f650 0x7fc4a0131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:25.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.769+0000 7fc4a70a5700 1 --2- 172.21.15.16:0/2675700706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4a009e930 0x7fc4a009ed00 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:25.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.769+0000 7fc4a70a5700 1 -- 172.21.15.16:0/2675700706 >> 172.21.15.16:0/2675700706 conn(0x7fc4a00a4210 msgr2=0x7fc4a00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:25.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.769+0000 7fc4a70a5700 1 -- 172.21.15.16:0/2675700706 shutdown_connections 2023-09-30T22:33:25.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.769+0000 7fc4a70a5700 1 -- 172.21.15.16:0/2675700706 wait complete. 2023-09-30T22:33:25.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.770+0000 7f1df27ae700 1 Processor -- start 2023-09-30T22:33:25.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.770+0000 7f1df27ae700 1 -- start start 2023-09-30T22:33:25.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.771+0000 7f1df27ae700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1dec09e930 0x7f1dec09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:25.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.771+0000 7f1df27ae700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1dec09f240 0x7f1dec1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:25.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.771+0000 7fc4a70a5700 1 Processor -- start 2023-09-30T22:33:25.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.771+0000 7f1df27ae700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dec070940 con 0x7f1dec09f650 2023-09-30T22:33:25.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.771+0000 7f1df27ae700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dec06fd60 con 0x7f1dec09e930 2023-09-30T22:33:25.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.771+0000 7f1df27ae700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1dec06fa80 con 0x7f1dec09f240 2023-09-30T22:33:25.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.771+0000 7f1deb7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1dec09f240 0x7f1dec1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:25.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.771+0000 7f1deb7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1dec09f240 0x7f1dec1317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:36380/0 (socket says 172.21.15.16:36380) 2023-09-30T22:33:25.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.771+0000 7f1deb7fe700 1 -- 172.21.15.16:0/3603414234 learned_addr learned my addr 172.21.15.16:0/3603414234 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:25.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.771+0000 7f1deaffd700 1 -- 172.21.15.16:0/3603414234 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 590799093 0 0) 0x7f1dec070940 con 0x7f1dec09f650 2023-09-30T22:33:25.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.771+0000 7f1deaffd700 1 -- 172.21.15.16:0/3603414234 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1dd40035b0 con 0x7f1dec09f650 2023-09-30T22:33:25.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.771+0000 7f1deb7fe700 1 -- 172.21.15.16:0/3603414234 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1dec09e930 msgr2=0x7f1dec09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:25.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.771+0000 7f1deb7fe700 1 --2- 172.21.15.16:0/3603414234 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1dec09e930 0x7f1dec09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:25.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.771+0000 7f1deb7fe700 1 -- 172.21.15.16:0/3603414234 >> v1:172.21.15.16:6789/0 conn(0x7f1dec09f650 legacy=0x7f1dec131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:25.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.771+0000 7f1deb7fe700 1 -- 172.21.15.16:0/3603414234 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dec09fb60 con 0x7f1dec09f240 2023-09-30T22:33:25.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.772+0000 7f1deb7fe700 1 --2- 172.21.15.16:0/3603414234 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1dec09f240 0x7f1dec1317f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f1de0009920 tx=0x7f1de0009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=804ae232e767893c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:25.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.772+0000 7f1deaffd700 1 -- 172.21.15.16:0/3603414234 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de0011830 con 0x7f1dec09f240 2023-09-30T22:33:25.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.771+0000 7fc4a70a5700 1 -- start start 2023-09-30T22:33:25.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.772+0000 7fc4a70a5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4a009e930 0x7fc4a01a2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:25.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.772+0000 7fc4a70a5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc4a009f240 0x7fc4a01a32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:25.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.772+0000 7fc4a70a5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc4a009f650 0x7fc4a01a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:25.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.772+0000 7fc4a70a5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a01a4fc0 con 0x7fc4a009e930 2023-09-30T22:33:25.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.772+0000 7fc4a70a5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a01a6880 con 0x7fc4a009f240 2023-09-30T22:33:25.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.772+0000 7fc4a70a5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a01a5c50 con 0x7fc4a009f650 2023-09-30T22:33:25.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.772+0000 7f1deaffd700 1 -- 172.21.15.16:0/3603414234 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1de0011990 con 0x7f1dec09f240 2023-09-30T22:33:25.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.772+0000 7f1deaffd700 1 -- 172.21.15.16:0/3603414234 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de0011d20 con 0x7f1dec09f240 2023-09-30T22:33:25.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.772+0000 7fc4a5642700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc4a009f650 0x7fc4a01a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:25.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.772+0000 7fc4a5642700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc4a009f650 0x7fc4a01a3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:36382/0 (socket says 172.21.15.16:36382) 2023-09-30T22:33:25.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.772+0000 7fc4a5642700 1 -- 172.21.15.16:0/1595471549 learned_addr learned my addr 172.21.15.16:0/1595471549 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:25.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.773+0000 7fc4a5642700 1 -- 172.21.15.16:0/1595471549 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc4a009f240 msgr2=0x7fc4a01a32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:25.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.773+0000 7fc4a5642700 1 --2- 172.21.15.16:0/1595471549 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc4a009f240 0x7fc4a01a32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:25.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.773+0000 7fc4a5642700 1 -- 172.21.15.16:0/1595471549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4a009e930 msgr2=0x7fc4a01a2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:25.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.773+0000 7fc4a5642700 1 --2- 172.21.15.16:0/1595471549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4a009e930 0x7fc4a01a2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:25.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.773+0000 7fc4a5642700 1 -- 172.21.15.16:0/1595471549 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc494009550 con 0x7fc4a009f650 2023-09-30T22:33:25.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.773+0000 7fc4a5642700 1 --2- 172.21.15.16:0/1595471549 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc4a009f650 0x7fc4a01a3830 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fc49800b9a0 tx=0x7fc49800bd60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:25.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.773+0000 7f1df27ae700 1 -- 172.21.15.16:0/3603414234 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1dec09f240 msgr2=0x7f1dec1317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:25.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.773+0000 7f1df27ae700 1 --2- 172.21.15.16:0/3603414234 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1dec09f240 0x7f1dec1317f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f1de0009920 tx=0x7f1de0009c30 comp rx=0 tx=0).stop 2023-09-30T22:33:25.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.773+0000 7f1df27ae700 1 -- 172.21.15.16:0/3603414234 shutdown_connections 2023-09-30T22:33:25.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.773+0000 7f1df27ae700 1 --2- 172.21.15.16:0/3603414234 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1dec09e930 0x7f1dec09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:25.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.773+0000 7f1df27ae700 1 --2- 172.21.15.16:0/3603414234 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1dec09f240 0x7f1dec1317f0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:25.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.773+0000 7f1df27ae700 1 -- 172.21.15.16:0/3603414234 >> 172.21.15.16:0/3603414234 conn(0x7f1dec0a4210 msgr2=0x7f1dec0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:25.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.773+0000 7fc49d7fa700 1 -- 172.21.15.16:0/1595471549 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49800c9b0 con 0x7fc4a009f650 2023-09-30T22:33:25.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.773+0000 7f1df27ae700 1 -- 172.21.15.16:0/3603414234 shutdown_connections 2023-09-30T22:33:25.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.773+0000 7fc49d7fa700 1 -- 172.21.15.16:0/1595471549 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc49800cb30 con 0x7fc4a009f650 2023-09-30T22:33:25.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.773+0000 7fc4a70a5700 1 -- 172.21.15.16:0/1595471549 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4a00a0190 con 0x7fc4a009f650 2023-09-30T22:33:25.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.773+0000 7fc49d7fa700 1 -- 172.21.15.16:0/1595471549 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc498014440 con 0x7fc4a009f650 2023-09-30T22:33:25.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.773+0000 7fc4a70a5700 1 -- 172.21.15.16:0/1595471549 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4a0071d20 con 0x7fc4a009f650 2023-09-30T22:33:25.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.774+0000 7f1df27ae700 1 -- 172.21.15.16:0/3603414234 wait complete. 2023-09-30T22:33:25.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.774+0000 7fc4a70a5700 1 -- 172.21.15.16:0/1595471549 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4a01a49d0 con 0x7fc4a009f650 2023-09-30T22:33:25.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.775+0000 7fc49d7fa700 1 -- 172.21.15.16:0/1595471549 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc49801d3b0 con 0x7fc4a009f650 2023-09-30T22:33:25.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.775+0000 7fc49d7fa700 1 -- 172.21.15.16:0/1595471549 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fc498006e40 con 0x7fc4a009f650 2023-09-30T22:33:25.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.776+0000 7f1df27ae700 1 Processor -- start 2023-09-30T22:33:25.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.776+0000 7f1df27ae700 1 -- start start 2023-09-30T22:33:25.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.776+0000 7f1df27ae700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1dec09e930 0x7f1dec1abfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:25.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.777+0000 7f1df27ae700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1dec09f240 0x7f1dec1ac4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:25.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.777+0000 7f1df27ae700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dec09f650 0x7f1dec1aca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:25.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.777+0000 7f1df27ae700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dec1ad3a0 con 0x7f1dec09f650 2023-09-30T22:33:25.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.777+0000 7f1df27ae700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dec1af7c0 con 0x7f1dec09e930 2023-09-30T22:33:25.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.777+0000 7f1df27ae700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1dec0a3550 con 0x7f1dec09f240 2023-09-30T22:33:25.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.777+0000 7f1debfff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1dec09e930 0x7f1dec1abfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:25.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.777+0000 7f1debfff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1dec09e930 0x7f1dec1abfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54764/0 (socket says 172.21.15.16:54764) 2023-09-30T22:33:25.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.777+0000 7f1debfff700 1 -- 172.21.15.16:0/2039377234 learned_addr learned my addr 172.21.15.16:0/2039377234 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:25.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.777+0000 7f1df0d4b700 1 --2- 172.21.15.16:0/2039377234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dec09f650 0x7f1dec1aca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:25.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.777+0000 7f1debfff700 1 -- 172.21.15.16:0/2039377234 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1dec09f240 msgr2=0x7f1dec1ac4f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:33:25.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.777+0000 7f1debfff700 1 --2- 172.21.15.16:0/2039377234 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1dec09f240 0x7f1dec1ac4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:25.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.777+0000 7f1debfff700 1 -- 172.21.15.16:0/2039377234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dec09f650 msgr2=0x7f1dec1aca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:25.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.777+0000 7f1debfff700 1 --2- 172.21.15.16:0/2039377234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dec09f650 0x7f1dec1aca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:25.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.777+0000 7f1debfff700 1 -- 172.21.15.16:0/2039377234 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1de0009550 con 0x7f1dec09e930 2023-09-30T22:33:25.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.778+0000 7f1debfff700 1 --2- 172.21.15.16:0/2039377234 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1dec09e930 0x7f1dec1abfb0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc00b530 tx=0x7f1ddc00b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:25.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.778+0000 7f1de8ff9700 1 -- 172.21.15.16:0/2039377234 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ddc0118f0 con 0x7f1dec09e930 2023-09-30T22:33:25.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.778+0000 7f1df27ae700 1 -- 172.21.15.16:0/2039377234 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dec0a0190 con 0x7f1dec09e930 2023-09-30T22:33:25.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.778+0000 7f1de8ff9700 1 -- 172.21.15.16:0/2039377234 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ddc011a50 con 0x7f1dec09e930 2023-09-30T22:33:25.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.778+0000 7f1de8ff9700 1 -- 172.21.15.16:0/2039377234 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ddc011e00 con 0x7f1dec09e930 2023-09-30T22:33:25.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.778+0000 7f1df27ae700 1 -- 172.21.15.16:0/2039377234 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dec071d20 con 0x7f1dec09e930 2023-09-30T22:33:25.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.779+0000 7f1de8ff9700 1 -- 172.21.15.16:0/2039377234 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1ddc01a400 con 0x7f1dec09e930 2023-09-30T22:33:25.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.779+0000 7f1de8ff9700 1 -- 172.21.15.16:0/2039377234 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f1ddc03f800 con 0x7f1dec09e930 2023-09-30T22:33:25.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.779+0000 7f1df27ae700 1 -- 172.21.15.16:0/2039377234 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dcc0052e0 con 0x7f1dec09e930 2023-09-30T22:33:25.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.781+0000 7fc49d7fa700 1 -- 172.21.15.16:0/1595471549 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc498007370 con 0x7fc4a009f650 2023-09-30T22:33:25.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.784+0000 7f1de8ff9700 1 -- 172.21.15.16:0/2039377234 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1ddc015080 con 0x7f1dec09e930 2023-09-30T22:33:25.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.962+0000 7f063d4c5700 1 -- 172.21.15.16:0/2687350659 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f0638074ea0 con 0x7f0638135c20 2023-09-30T22:33:25.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.962+0000 7f061f7fe700 1 -- 172.21.15.16:0/2687350659 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0630022070 con 0x7f0638135c20 2023-09-30T22:33:25.964 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:33:25.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.965+0000 7f063d4c5700 1 -- 172.21.15.16:0/2687350659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0638135c20 msgr2=0x7f063809a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:25.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.965+0000 7f063d4c5700 1 --2- 172.21.15.16:0/2687350659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0638135c20 0x7f063809a8d0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f063000b6a0 tx=0x7f063000b9b0 comp rx=0 tx=0).stop 2023-09-30T22:33:25.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.965+0000 7f063d4c5700 1 -- 172.21.15.16:0/2687350659 shutdown_connections 2023-09-30T22:33:25.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.965+0000 7f063d4c5700 1 --2- 172.21.15.16:0/2687350659 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f063806f5d0 0x7f0638099e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:25.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.965+0000 7f063d4c5700 1 --2- 172.21.15.16:0/2687350659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0638135c20 0x7f063809a8d0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:25.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.965+0000 7f063d4c5700 1 --2- 172.21.15.16:0/2687350659 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f063812cf60 0x7f063809a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:25.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.965+0000 7f063d4c5700 1 -- 172.21.15.16:0/2687350659 >> 172.21.15.16:0/2687350659 conn(0x7f06380a2aa0 msgr2=0x7f063812aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:25.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.966+0000 7f063d4c5700 1 -- 172.21.15.16:0/2687350659 shutdown_connections 2023-09-30T22:33:25.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.966+0000 7f063d4c5700 1 -- 172.21.15.16:0/2687350659 wait complete. 2023-09-30T22:33:25.986 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:33:25.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.990+0000 7fc4a70a5700 1 -- 172.21.15.16:0/1595471549 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc4a0074ea0 con 0x7fc4a009f650 2023-09-30T22:33:25.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.991+0000 7fc49d7fa700 1 -- 172.21.15.16:0/1595471549 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc498019030 con 0x7fc4a009f650 2023-09-30T22:33:25.992 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:33:25.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.993+0000 7f1df27ae700 1 -- 172.21.15.16:0/2039377234 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1dcc005150 con 0x7f1dec09e930 2023-09-30T22:33:25.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.993+0000 7fc4a70a5700 1 -- 172.21.15.16:0/1595471549 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc4a009f650 msgr2=0x7fc4a01a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:25.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.993+0000 7fc4a70a5700 1 --2- 172.21.15.16:0/1595471549 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc4a009f650 0x7fc4a01a3830 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fc49800b9a0 tx=0x7fc49800bd60 comp rx=0 tx=0).stop 2023-09-30T22:33:25.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.993+0000 7f1de8ff9700 1 -- 172.21.15.16:0/2039377234 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1ddc00f8d0 con 0x7f1dec09e930 2023-09-30T22:33:25.995 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:33:25.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.993+0000 7fc4a70a5700 1 -- 172.21.15.16:0/1595471549 shutdown_connections 2023-09-30T22:33:25.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.993+0000 7fc4a70a5700 1 --2- 172.21.15.16:0/1595471549 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc4a009f650 0x7fc4a01a3830 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:25.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.993+0000 7fc4a70a5700 1 --2- 172.21.15.16:0/1595471549 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc4a009e930 0x7fc4a01a2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:25.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.993+0000 7fc4a70a5700 1 --2- 172.21.15.16:0/1595471549 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc4a009f240 0x7fc4a01a32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:25.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.993+0000 7fc4a70a5700 1 -- 172.21.15.16:0/1595471549 >> 172.21.15.16:0/1595471549 conn(0x7fc4a00a4210 msgr2=0x7fc4a009c210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:25.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.994+0000 7fc4a70a5700 1 -- 172.21.15.16:0/1595471549 shutdown_connections 2023-09-30T22:33:25.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.994+0000 7fc4a70a5700 1 -- 172.21.15.16:0/1595471549 wait complete. 2023-09-30T22:33:25.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.996+0000 7f1df27ae700 1 -- 172.21.15.16:0/2039377234 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1dec09e930 msgr2=0x7f1dec1abfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:25.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.996+0000 7f1df27ae700 1 --2- 172.21.15.16:0/2039377234 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1dec09e930 0x7f1dec1abfb0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc00b530 tx=0x7f1ddc00b840 comp rx=0 tx=0).stop 2023-09-30T22:33:25.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.996+0000 7f1df27ae700 1 -- 172.21.15.16:0/2039377234 shutdown_connections 2023-09-30T22:33:25.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.996+0000 7f1df27ae700 1 --2- 172.21.15.16:0/2039377234 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1dec09e930 0x7f1dec1abfb0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:25.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.996+0000 7f1df27ae700 1 --2- 172.21.15.16:0/2039377234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1dec09f650 0x7f1dec1aca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:25.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.996+0000 7f1df27ae700 1 --2- 172.21.15.16:0/2039377234 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1dec09f240 0x7f1dec1ac4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:25.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:25.996+0000 7f1df27ae700 1 -- 172.21.15.16:0/2039377234 >> 172.21.15.16:0/2039377234 conn(0x7f1dec0a4210 msgr2=0x7f1dec06b880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:26.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:26.001+0000 7f1df27ae700 1 -- 172.21.15.16:0/2039377234 shutdown_connections 2023-09-30T22:33:26.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:26.001+0000 7f1df27ae700 1 -- 172.21.15.16:0/2039377234 wait complete. 2023-09-30T22:33:26.012 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:33:26.020 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:33:26.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:26.035+0000 7f1be57ff700 1 -- 172.21.15.16:0/2753709537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1bc0005590 con 0x7f1be009e930 2023-09-30T22:33:26.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:26.036+0000 7f1bc77fe700 1 -- 172.21.15.16:0/2753709537 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1bd00253e0 con 0x7f1be009e930 2023-09-30T22:33:26.038 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:33:26.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:26.038+0000 7f1be57ff700 1 -- 172.21.15.16:0/2753709537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1be009e930 msgr2=0x7f1be01a3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:26.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:26.038+0000 7f1be57ff700 1 --2- 172.21.15.16:0/2753709537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1be009e930 0x7f1be01a3d70 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0009fd0 tx=0x7f1bd0004750 comp rx=0 tx=0).stop 2023-09-30T22:33:26.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:26.038+0000 7f1be57ff700 1 -- 172.21.15.16:0/2753709537 shutdown_connections 2023-09-30T22:33:26.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:26.038+0000 7f1be57ff700 1 --2- 172.21.15.16:0/2753709537 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1be009f650 0x7f1be01a47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:26.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:26.038+0000 7f1be57ff700 1 --2- 172.21.15.16:0/2753709537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1be009e930 0x7f1be01a3d70 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:26.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:26.038+0000 7f1be57ff700 1 --2- 172.21.15.16:0/2753709537 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1be009f240 0x7f1be01a42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:26.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:26.038+0000 7f1be57ff700 1 -- 172.21.15.16:0/2753709537 >> 172.21.15.16:0/2753709537 conn(0x7f1be00a4210 msgr2=0x7f1be01b8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:26.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:26.038+0000 7f1be57ff700 1 -- 172.21.15.16:0/2753709537 shutdown_connections 2023-09-30T22:33:26.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:26.038+0000 7f1be57ff700 1 -- 172.21.15.16:0/2753709537 wait complete. 2023-09-30T22:33:26.053 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:33:26.987 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:33:27.013 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:33:27.021 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:33:27.055 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:33:27.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.144+0000 7fee53eb7700 1 Processor -- start 2023-09-30T22:33:27.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.144+0000 7fee53eb7700 1 -- start start 2023-09-30T22:33:27.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.145+0000 7fee53eb7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fee4c09d6f0 0x7fee4c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:27.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.145+0000 7fee53eb7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fee4c09e000 0x7fee4c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:27.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.145+0000 7fee53eb7700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee4c09b120 con 0x7fee4c09e410 2023-09-30T22:33:27.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.145+0000 7fee53eb7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee4c06b460 con 0x7fee4c09e000 2023-09-30T22:33:27.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.145+0000 7fee53eb7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fee4c09a7e0 con 0x7fee4c09d6f0 2023-09-30T22:33:27.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.145+0000 7fee51c53700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fee4c09d6f0 0x7fee4c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:27.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.145+0000 7fee52454700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fee4c09e410 0x7fee4c133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54826/0 (socket says 172.21.15.16:54826) 2023-09-30T22:33:27.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.145+0000 7fee52454700 1 -- 172.21.15.16:0/922602900 learned_addr learned my addr 172.21.15.16:0/922602900 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:27.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.145+0000 7fee51452700 1 --2- 172.21.15.16:0/922602900 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fee4c09e000 0x7fee4c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:27.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.145+0000 7fee51c53700 1 -- 172.21.15.16:0/922602900 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fee4c09e000 msgr2=0x7fee4c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:27.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.145+0000 7fee51c53700 1 --2- 172.21.15.16:0/922602900 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fee4c09e000 0x7fee4c1339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:27.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.145+0000 7fee51c53700 1 -- 172.21.15.16:0/922602900 >> v1:172.21.15.16:6789/0 conn(0x7fee4c09e410 legacy=0x7fee4c133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:27.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.145+0000 7fee51c53700 1 -- 172.21.15.16:0/922602900 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee4c09a500 con 0x7fee4c09d6f0 2023-09-30T22:33:27.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.146+0000 7fee51452700 1 --2- 172.21.15.16:0/922602900 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fee4c09e000 0x7fee4c1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:27.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.146+0000 7fee51c53700 1 --2- 172.21.15.16:0/922602900 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fee4c09d6f0 0x7fee4c09dac0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fee480098a0 tx=0x7fee48009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cf0b1c2c88bf3c5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:27.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.146+0000 7fee50c51700 1 -- 172.21.15.16:0/922602900 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee48011830 con 0x7fee4c09d6f0 2023-09-30T22:33:27.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.146+0000 7fee50c51700 1 -- 172.21.15.16:0/922602900 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee48011990 con 0x7fee4c09d6f0 2023-09-30T22:33:27.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.146+0000 7fee50c51700 1 -- 172.21.15.16:0/922602900 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee48011d20 con 0x7fee4c09d6f0 2023-09-30T22:33:27.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.147+0000 7fee53eb7700 1 -- 172.21.15.16:0/922602900 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fee4c09d6f0 msgr2=0x7fee4c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:27.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.147+0000 7fee53eb7700 1 --2- 172.21.15.16:0/922602900 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fee4c09d6f0 0x7fee4c09dac0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fee480098a0 tx=0x7fee48009bb0 comp rx=0 tx=0).stop 2023-09-30T22:33:27.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.147+0000 7fee53eb7700 1 -- 172.21.15.16:0/922602900 shutdown_connections 2023-09-30T22:33:27.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.147+0000 7fee53eb7700 1 --2- 172.21.15.16:0/922602900 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fee4c09d6f0 0x7fee4c09dac0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:27.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.147+0000 7fee53eb7700 1 --2- 172.21.15.16:0/922602900 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fee4c09e000 0x7fee4c1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:27.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.147+0000 7fee53eb7700 1 -- 172.21.15.16:0/922602900 >> 172.21.15.16:0/922602900 conn(0x7fee4c066f50 msgr2=0x7fee4c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:27.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.147+0000 7fee53eb7700 1 -- 172.21.15.16:0/922602900 shutdown_connections 2023-09-30T22:33:27.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.147+0000 7fee53eb7700 1 -- 172.21.15.16:0/922602900 wait complete. 2023-09-30T22:33:27.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.148+0000 7fee53eb7700 1 Processor -- start 2023-09-30T22:33:27.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.148+0000 7fee53eb7700 1 -- start start 2023-09-30T22:33:27.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.149+0000 7fee53eb7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fee4c09d6f0 0x7fee4c1a5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:27.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.149+0000 7fee53eb7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fee4c09e000 0x7fee4c1a63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:27.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.149+0000 7fee53eb7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee4c09e410 0x7fee4c1a6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:27.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.149+0000 7fee53eb7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee4c1a8d60 con 0x7fee4c09e410 2023-09-30T22:33:27.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.149+0000 7fee53eb7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee4c1a7340 con 0x7fee4c09d6f0 2023-09-30T22:33:27.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.149+0000 7fee53eb7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fee4c1a97b0 con 0x7fee4c09e000 2023-09-30T22:33:27.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.149+0000 7fee51452700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fee4c09e000 0x7fee4c1a63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:27.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.149+0000 7fee51c53700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fee4c09d6f0 0x7fee4c1a5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:27.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.149+0000 7fee51452700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fee4c09e000 0x7fee4c1a63d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:36388/0 (socket says 172.21.15.16:36388) 2023-09-30T22:33:27.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.149+0000 7fee51c53700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fee4c09d6f0 0x7fee4c1a5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54776/0 (socket says 172.21.15.16:54776) 2023-09-30T22:33:27.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.149+0000 7fee51452700 1 -- 172.21.15.16:0/4249741489 learned_addr learned my addr 172.21.15.16:0/4249741489 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:27.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.149+0000 7fee52454700 1 --2- 172.21.15.16:0/4249741489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee4c09e410 0x7fee4c1a6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:27.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.149+0000 7fee51452700 1 -- 172.21.15.16:0/4249741489 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fee4c09d6f0 msgr2=0x7fee4c1a5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:27.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.149+0000 7fee51452700 1 --2- 172.21.15.16:0/4249741489 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fee4c09d6f0 0x7fee4c1a5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:27.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.149+0000 7fee51452700 1 -- 172.21.15.16:0/4249741489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee4c09e410 msgr2=0x7fee4c1a6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:27.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.149+0000 7fee51452700 1 --2- 172.21.15.16:0/4249741489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee4c09e410 0x7fee4c1a6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:27.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.149+0000 7fee51452700 1 -- 172.21.15.16:0/4249741489 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee3c0094a0 con 0x7fee4c09e000 2023-09-30T22:33:27.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.149+0000 7fee51c53700 1 --2- 172.21.15.16:0/4249741489 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fee4c09d6f0 0x7fee4c1a5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:27.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.149+0000 7fee52454700 1 --2- 172.21.15.16:0/4249741489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee4c09e410 0x7fee4c1a6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:27.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.150+0000 7fee51452700 1 --2- 172.21.15.16:0/4249741489 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fee4c09e000 0x7fee4c1a63d0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fee3c00ebe0 tx=0x7fee3c00eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:27.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.150+0000 7fee427fc700 1 -- 172.21.15.16:0/4249741489 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3c00c850 con 0x7fee4c09e000 2023-09-30T22:33:27.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.150+0000 7fee427fc700 1 -- 172.21.15.16:0/4249741489 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee3c00c9b0 con 0x7fee4c09e000 2023-09-30T22:33:27.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.150+0000 7fee53eb7700 1 -- 172.21.15.16:0/4249741489 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee480094a0 con 0x7fee4c09e000 2023-09-30T22:33:27.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.150+0000 7fee427fc700 1 -- 172.21.15.16:0/4249741489 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3c00cd40 con 0x7fee4c09e000 2023-09-30T22:33:27.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.150+0000 7fee53eb7700 1 -- 172.21.15.16:0/4249741489 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee4c0a32a0 con 0x7fee4c09e000 2023-09-30T22:33:27.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.151+0000 7fee427fc700 1 -- 172.21.15.16:0/4249741489 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fee3c013470 con 0x7fee4c09e000 2023-09-30T22:33:27.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.151+0000 7fee53eb7700 1 -- 172.21.15.16:0/4249741489 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee4c09c710 con 0x7fee4c09e000 2023-09-30T22:33:27.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.151+0000 7fee427fc700 1 -- 172.21.15.16:0/4249741489 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fee3c03f7a0 con 0x7fee4c09e000 2023-09-30T22:33:27.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.156+0000 7fee427fc700 1 -- 172.21.15.16:0/4249741489 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fee3c013720 con 0x7fee4c09e000 2023-09-30T22:33:27.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.179+0000 7f22633fc700 1 Processor -- start 2023-09-30T22:33:27.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.180+0000 7f22633fc700 1 -- start start 2023-09-30T22:33:27.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.180+0000 7f22633fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f225c136830 0x7f225c138c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:27.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.180+0000 7f22633fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f225c139150 0x7f225c13b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:27.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.180+0000 7f22633fc700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f225c06f720 con 0x7f225c1338c0 2023-09-30T22:33:27.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.180+0000 7f22633fc700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f225c070c80 con 0x7f225c136830 2023-09-30T22:33:27.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.180+0000 7f22633fc700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f225c06fce0 con 0x7f225c139150 2023-09-30T22:33:27.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.181+0000 7f2261999700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f225c139150 0x7f225c13b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:27.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.181+0000 7f2261999700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f225c139150 0x7f225c13b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:36404/0 (socket says 172.21.15.16:36404) 2023-09-30T22:33:27.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.181+0000 7f2261999700 1 -- 172.21.15.16:0/3529621444 learned_addr learned my addr 172.21.15.16:0/3529621444 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:27.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.181+0000 7f2260997700 1 --2- 172.21.15.16:0/3529621444 >> v2:172.21.15.16:3300/0 conn(0x7f225c136830 0x7f225c138c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:27.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.181+0000 7f2253fff700 1 -- 172.21.15.16:0/3529621444 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3918834850 0 0) 0x7f225c06f720 con 0x7f225c1338c0 2023-09-30T22:33:27.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.181+0000 7f2261999700 1 -- 172.21.15.16:0/3529621444 >> v2:172.21.15.16:3300/0 conn(0x7f225c136830 msgr2=0x7f225c138c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:27.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.181+0000 7f2261999700 1 --2- 172.21.15.16:0/3529621444 >> v2:172.21.15.16:3300/0 conn(0x7f225c136830 0x7f225c138c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:27.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.181+0000 7f2261999700 1 -- 172.21.15.16:0/3529621444 >> v1:172.21.15.16:6789/0 conn(0x7f225c1338c0 legacy=0x7f225c133c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:27.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.181+0000 7f2261999700 1 -- 172.21.15.16:0/3529621444 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f225c1336a0 con 0x7f225c139150 2023-09-30T22:33:27.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.181+0000 7f2261999700 1 --2- 172.21.15.16:0/3529621444 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f225c139150 0x7f225c13b530 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f2254009e50 tx=0x7f225400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26cc45903ea576b8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:27.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.182+0000 7f2253fff700 1 -- 172.21.15.16:0/3529621444 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225400e070 con 0x7f225c139150 2023-09-30T22:33:27.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.182+0000 7f2253fff700 1 -- 172.21.15.16:0/3529621444 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f225400b910 con 0x7f225c139150 2023-09-30T22:33:27.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.182+0000 7f2253fff700 1 -- 172.21.15.16:0/3529621444 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225400bcc0 con 0x7f225c139150 2023-09-30T22:33:27.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.182+0000 7f22633fc700 1 -- 172.21.15.16:0/3529621444 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f225c139150 msgr2=0x7f225c13b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:27.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.182+0000 7f22633fc700 1 --2- 172.21.15.16:0/3529621444 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f225c139150 0x7f225c13b530 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f2254009e50 tx=0x7f225400b330 comp rx=0 tx=0).stop 2023-09-30T22:33:27.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.182+0000 7f22633fc700 1 -- 172.21.15.16:0/3529621444 shutdown_connections 2023-09-30T22:33:27.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.182+0000 7f22633fc700 1 --2- 172.21.15.16:0/3529621444 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f225c139150 0x7f225c13b530 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:27.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.182+0000 7f22633fc700 1 --2- 172.21.15.16:0/3529621444 >> v2:172.21.15.16:3300/0 conn(0x7f225c136830 0x7f225c138c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:27.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.183+0000 7f22633fc700 1 -- 172.21.15.16:0/3529621444 >> 172.21.15.16:0/3529621444 conn(0x7f225c0a2aa0 msgr2=0x7f225c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:27.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.183+0000 7f22633fc700 1 -- 172.21.15.16:0/3529621444 shutdown_connections 2023-09-30T22:33:27.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.183+0000 7f22633fc700 1 -- 172.21.15.16:0/3529621444 wait complete. 2023-09-30T22:33:27.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.184+0000 7f22633fc700 1 Processor -- start 2023-09-30T22:33:27.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.184+0000 7f22633fc700 1 -- start start 2023-09-30T22:33:27.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.184+0000 7f22633fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f225c1338c0 0x7f225c119ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:27.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.184+0000 7f22633fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f225c136830 0x7f225c11a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:27.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.184+0000 7f22633fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f225c139150 0x7f225c11a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:27.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.184+0000 7f22633fc700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f225c11b6d0 con 0x7f225c136830 2023-09-30T22:33:27.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.184+0000 7f22633fc700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f225c06f960 con 0x7f225c1338c0 2023-09-30T22:33:27.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.184+0000 7f22633fc700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f225c11d5e0 con 0x7f225c139150 2023-09-30T22:33:27.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.184+0000 7f2261198700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f225c1338c0 0x7f225c119ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:27.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.185+0000 7f2261198700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f225c1338c0 0x7f225c119ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54788/0 (socket says 172.21.15.16:54788) 2023-09-30T22:33:27.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.185+0000 7f2261198700 1 -- 172.21.15.16:0/232915843 learned_addr learned my addr 172.21.15.16:0/232915843 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:27.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.185+0000 7f2261999700 1 --2- 172.21.15.16:0/232915843 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f225c139150 0x7f225c11a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:27.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.185+0000 7f2261198700 1 -- 172.21.15.16:0/232915843 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f225c139150 msgr2=0x7f225c11a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:27.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.185+0000 7f2261198700 1 --2- 172.21.15.16:0/232915843 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f225c139150 0x7f225c11a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:27.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.185+0000 7f2261198700 1 -- 172.21.15.16:0/232915843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f225c136830 msgr2=0x7f225c11a000 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:33:27.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.185+0000 7f2261198700 1 --2- 172.21.15.16:0/232915843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f225c136830 0x7f225c11a000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:27.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.185+0000 7f2261198700 1 -- 172.21.15.16:0/232915843 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2254009b30 con 0x7f225c1338c0 2023-09-30T22:33:27.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.185+0000 7f2261198700 1 --2- 172.21.15.16:0/232915843 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f225c1338c0 0x7f225c119ac0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f2258001eb0 tx=0x7f225800c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:27.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.186+0000 7f2251ffb700 1 -- 172.21.15.16:0/232915843 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22580054e0 con 0x7f225c1338c0 2023-09-30T22:33:27.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.186+0000 7f2251ffb700 1 -- 172.21.15.16:0/232915843 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2258003680 con 0x7f225c1338c0 2023-09-30T22:33:27.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.186+0000 7f2251ffb700 1 -- 172.21.15.16:0/232915843 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22580039f0 con 0x7f225c1338c0 2023-09-30T22:33:27.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.186+0000 7f22633fc700 1 -- 172.21.15.16:0/232915843 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f225c11c990 con 0x7f225c1338c0 2023-09-30T22:33:27.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.186+0000 7f22633fc700 1 -- 172.21.15.16:0/232915843 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f225c11d300 con 0x7f225c1338c0 2023-09-30T22:33:27.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.187+0000 7f22633fc700 1 -- 172.21.15.16:0/232915843 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f225c11de30 con 0x7f225c1338c0 2023-09-30T22:33:27.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.189+0000 7fedceffc700 1 Processor -- start 2023-09-30T22:33:27.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.189+0000 7f2251ffb700 1 -- 172.21.15.16:0/232915843 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2258003b50 con 0x7f225c1338c0 2023-09-30T22:33:27.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.190+0000 7f2251ffb700 1 -- 172.21.15.16:0/232915843 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f225803f510 con 0x7f225c1338c0 2023-09-30T22:33:27.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.189+0000 7fedceffc700 1 -- start start 2023-09-30T22:33:27.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.190+0000 7fedceffc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fedc809eca0 0x7fedc813cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:27.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.190+0000 7fedceffc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fedc806f050 0x7fedc813f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:27.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.190+0000 7fedceffc700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedc8070ee0 con 0x7fedc806e570 2023-09-30T22:33:27.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.190+0000 7fedceffc700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc8071980 con 0x7fedc809eca0 2023-09-30T22:33:27.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.190+0000 7fedceffc700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fedc8071480 con 0x7fedc806f050 2023-09-30T22:33:27.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.190+0000 7fedccd98700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fedc806e570 0x7fedc806e940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54842/0 (socket says 172.21.15.16:54842) 2023-09-30T22:33:27.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.190+0000 7fedccd98700 1 -- 172.21.15.16:0/1820845770 learned_addr learned my addr 172.21.15.16:0/1820845770 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:27.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.190+0000 7fedcd599700 1 --2- 172.21.15.16:0/1820845770 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fedc806f050 0x7fedc813f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:27.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.191+0000 7fedc7fff700 1 --2- 172.21.15.16:0/1820845770 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fedc809eca0 0x7fedc813cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:27.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.191+0000 7fedc77fe700 1 -- 172.21.15.16:0/1820845770 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1388298700 0 0) 0x7fedc8070ee0 con 0x7fedc806e570 2023-09-30T22:33:27.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.191+0000 7fedc77fe700 1 -- 172.21.15.16:0/1820845770 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedb80035b0 con 0x7fedc806e570 2023-09-30T22:33:27.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.191+0000 7fedcd599700 1 -- 172.21.15.16:0/1820845770 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fedc809eca0 msgr2=0x7fedc813cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:27.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.191+0000 7fedcd599700 1 --2- 172.21.15.16:0/1820845770 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fedc809eca0 0x7fedc813cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:27.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.191+0000 7fedcd599700 1 -- 172.21.15.16:0/1820845770 >> v1:172.21.15.16:6789/0 conn(0x7fedc806e570 legacy=0x7fedc806e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:27.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.191+0000 7fedcd599700 1 -- 172.21.15.16:0/1820845770 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc8071b90 con 0x7fedc806f050 2023-09-30T22:33:27.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.191+0000 7fedcd599700 1 --2- 172.21.15.16:0/1820845770 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fedc806f050 0x7fedc813f3f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fedc0009dc0 tx=0x7fedc000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e56b53763cd9261 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:27.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.192+0000 7fedc77fe700 1 -- 172.21.15.16:0/1820845770 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc000e070 con 0x7fedc806f050 2023-09-30T22:33:27.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.192+0000 7fedc77fe700 1 -- 172.21.15.16:0/1820845770 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedc000bb20 con 0x7fedc806f050 2023-09-30T22:33:27.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.192+0000 7fedc77fe700 1 -- 172.21.15.16:0/1820845770 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc0012430 con 0x7fedc806f050 2023-09-30T22:33:27.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.193+0000 7fedceffc700 1 -- 172.21.15.16:0/1820845770 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fedc806f050 msgr2=0x7fedc813f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:27.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.193+0000 7fedceffc700 1 --2- 172.21.15.16:0/1820845770 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fedc806f050 0x7fedc813f3f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fedc0009dc0 tx=0x7fedc000b330 comp rx=0 tx=0).stop 2023-09-30T22:33:27.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.193+0000 7fedceffc700 1 -- 172.21.15.16:0/1820845770 shutdown_connections 2023-09-30T22:33:27.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.193+0000 7fedceffc700 1 --2- 172.21.15.16:0/1820845770 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fedc806f050 0x7fedc813f3f0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:27.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.193+0000 7fedceffc700 1 --2- 172.21.15.16:0/1820845770 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fedc809eca0 0x7fedc813cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:27.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.193+0000 7fedceffc700 1 -- 172.21.15.16:0/1820845770 >> 172.21.15.16:0/1820845770 conn(0x7fedc80a2aa0 msgr2=0x7fedc80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:27.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.193+0000 7fedceffc700 1 -- 172.21.15.16:0/1820845770 shutdown_connections 2023-09-30T22:33:27.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.193+0000 7fedceffc700 1 -- 172.21.15.16:0/1820845770 wait complete. 2023-09-30T22:33:27.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.194+0000 7f2251ffb700 1 -- 172.21.15.16:0/232915843 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2258012400 con 0x7f225c1338c0 2023-09-30T22:33:27.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.194+0000 7fedceffc700 1 Processor -- start 2023-09-30T22:33:27.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.195+0000 7fedceffc700 1 -- start start 2023-09-30T22:33:27.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.195+0000 7fedceffc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedc806e570 0x7fedc8136470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:27.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.195+0000 7fedceffc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fedc806f050 0x7fedc81369b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:27.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.196+0000 7fedceffc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fedc813a390 0x7fedc8136ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:27.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.196+0000 7fedceffc700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc813e520 con 0x7fedc806e570 2023-09-30T22:33:27.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.196+0000 7fedceffc700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc8139f90 con 0x7fedc806f050 2023-09-30T22:33:27.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.196+0000 7fedceffc700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fedc8138d10 con 0x7fedc813a390 2023-09-30T22:33:27.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.196+0000 7fedccd98700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedc806e570 0x7fedc8136470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:27.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.196+0000 7fedccd98700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedc806e570 0x7fedc8136470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41698/0 (socket says 172.21.15.16:41698) 2023-09-30T22:33:27.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.196+0000 7fedccd98700 1 -- 172.21.15.16:0/2426310474 learned_addr learned my addr 172.21.15.16:0/2426310474 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:27.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.196+0000 7fedcd599700 1 --2- 172.21.15.16:0/2426310474 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fedc813a390 0x7fedc8136ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:27.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.196+0000 7fedccd98700 1 -- 172.21.15.16:0/2426310474 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fedc813a390 msgr2=0x7fedc8136ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:27.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.196+0000 7fedccd98700 1 --2- 172.21.15.16:0/2426310474 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fedc813a390 0x7fedc8136ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:27.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.196+0000 7fedccd98700 1 -- 172.21.15.16:0/2426310474 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fedc806f050 msgr2=0x7fedc81369b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:27.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.196+0000 7fedccd98700 1 --2- 172.21.15.16:0/2426310474 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fedc806f050 0x7fedc81369b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:27.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.196+0000 7fedccd98700 1 -- 172.21.15.16:0/2426310474 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc0009a70 con 0x7fedc806e570 2023-09-30T22:33:27.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.197+0000 7fedccd98700 1 --2- 172.21.15.16:0/2426310474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedc806e570 0x7fedc8136470 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fedbc00c700 tx=0x7fedbc00ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:27.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.197+0000 7fedc57fa700 1 -- 172.21.15.16:0/2426310474 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedbc005d70 con 0x7fedc806e570 2023-09-30T22:33:27.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.197+0000 7fedc57fa700 1 -- 172.21.15.16:0/2426310474 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedbc0070d0 con 0x7fedc806e570 2023-09-30T22:33:27.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.197+0000 7fedc57fa700 1 -- 172.21.15.16:0/2426310474 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedbc007440 con 0x7fedc806e570 2023-09-30T22:33:27.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.197+0000 7fedceffc700 1 -- 172.21.15.16:0/2426310474 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedc813ec70 con 0x7fedc806e570 2023-09-30T22:33:27.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.197+0000 7fedceffc700 1 -- 172.21.15.16:0/2426310474 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedc81398f0 con 0x7fedc806e570 2023-09-30T22:33:27.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.198+0000 7fedc57fa700 1 -- 172.21.15.16:0/2426310474 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fedbc0075a0 con 0x7fedc806e570 2023-09-30T22:33:27.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.198+0000 7fedceffc700 1 -- 172.21.15.16:0/2426310474 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedc8074ea0 con 0x7fedc806e570 2023-09-30T22:33:27.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.199+0000 7fedc57fa700 1 -- 172.21.15.16:0/2426310474 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fedbc03ece0 con 0x7fedc806e570 2023-09-30T22:33:27.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.208+0000 7fedc57fa700 1 -- 172.21.15.16:0/2426310474 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fedbc011d70 con 0x7fedc806e570 2023-09-30T22:33:27.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.222+0000 7fe7631ca700 1 Processor -- start 2023-09-30T22:33:27.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.222+0000 7fe7631ca700 1 -- start start 2023-09-30T22:33:27.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.223+0000 7fe7631ca700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe75c09d6f0 0x7fe75c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:27.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.223+0000 7fe7631ca700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe75c09e000 0x7fe75c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:27.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.223+0000 7fe7631ca700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe75c09b120 con 0x7fe75c09e410 2023-09-30T22:33:27.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.223+0000 7fe7631ca700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe75c06b460 con 0x7fe75c09d6f0 2023-09-30T22:33:27.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.223+0000 7fe7631ca700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe75c09a7e0 con 0x7fe75c09e000 2023-09-30T22:33:27.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.223+0000 7fe761767700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe75c09e410 0x7fe75c133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54854/0 (socket says 172.21.15.16:54854) 2023-09-30T22:33:27.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.223+0000 7fe761767700 1 -- 172.21.15.16:0/629151715 learned_addr learned my addr 172.21.15.16:0/629151715 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:27.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.223+0000 7fe760f66700 1 --2- 172.21.15.16:0/629151715 >> v2:172.21.15.16:3300/0 conn(0x7fe75c09d6f0 0x7fe75c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:27.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.223+0000 7fe75bfff700 1 --2- 172.21.15.16:0/629151715 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe75c09e000 0x7fe75c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:27.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.224+0000 7fe760f66700 1 -- 172.21.15.16:0/629151715 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe75c09e000 msgr2=0x7fe75c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:27.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.224+0000 7fe760f66700 1 --2- 172.21.15.16:0/629151715 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe75c09e000 0x7fe75c1339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:27.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.224+0000 7fe760f66700 1 -- 172.21.15.16:0/629151715 >> v1:172.21.15.16:6789/0 conn(0x7fe75c09e410 legacy=0x7fe75c133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:27.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.224+0000 7fe760f66700 1 -- 172.21.15.16:0/629151715 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe75c09a500 con 0x7fe75c09d6f0 2023-09-30T22:33:27.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.224+0000 7fe75bfff700 1 --2- 172.21.15.16:0/629151715 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe75c09e000 0x7fe75c1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:27.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.224+0000 7fe760f66700 1 --2- 172.21.15.16:0/629151715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe75c09d6f0 0x7fe75c09dac0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fe750009840 tx=0x7fe750009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a8a70ee83d11a81 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:27.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.224+0000 7fe75b7fe700 1 -- 172.21.15.16:0/629151715 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe750011830 con 0x7fe75c09d6f0 2023-09-30T22:33:27.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.224+0000 7fe75b7fe700 1 -- 172.21.15.16:0/629151715 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe750011990 con 0x7fe75c09d6f0 2023-09-30T22:33:27.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.224+0000 7fe75b7fe700 1 -- 172.21.15.16:0/629151715 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe750011d20 con 0x7fe75c09d6f0 2023-09-30T22:33:27.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.225+0000 7fe7631ca700 1 -- 172.21.15.16:0/629151715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe75c09d6f0 msgr2=0x7fe75c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:27.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.225+0000 7fe7631ca700 1 --2- 172.21.15.16:0/629151715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe75c09d6f0 0x7fe75c09dac0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fe750009840 tx=0x7fe750009b50 comp rx=0 tx=0).stop 2023-09-30T22:33:27.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.225+0000 7fe7631ca700 1 -- 172.21.15.16:0/629151715 shutdown_connections 2023-09-30T22:33:27.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.225+0000 7fe7631ca700 1 --2- 172.21.15.16:0/629151715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe75c09d6f0 0x7fe75c09dac0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:27.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.225+0000 7fe7631ca700 1 --2- 172.21.15.16:0/629151715 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe75c09e000 0x7fe75c1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:27.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.225+0000 7fe7631ca700 1 -- 172.21.15.16:0/629151715 >> 172.21.15.16:0/629151715 conn(0x7fe75c066f50 msgr2=0x7fe75c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:27.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.225+0000 7fe7631ca700 1 -- 172.21.15.16:0/629151715 shutdown_connections 2023-09-30T22:33:27.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.225+0000 7fe7631ca700 1 -- 172.21.15.16:0/629151715 wait complete. 2023-09-30T22:33:27.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.226+0000 7fe7631ca700 1 Processor -- start 2023-09-30T22:33:27.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.226+0000 7fe7631ca700 1 -- start start 2023-09-30T22:33:27.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.227+0000 7fe7631ca700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe75c09d6f0 0x7fe75c1a5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:27.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.227+0000 7fe7631ca700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe75c09e000 0x7fe75c1a63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:27.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.227+0000 7fe7631ca700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe75c09e410 0x7fe75c1a6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:27.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.227+0000 7fe7631ca700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe75c1a7340 con 0x7fe75c09e000 2023-09-30T22:33:27.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.227+0000 7fe7631ca700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe75c1a7ac0 con 0x7fe75c09d6f0 2023-09-30T22:33:27.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.227+0000 7fe7631ca700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe75c1a80e0 con 0x7fe75c09e410 2023-09-30T22:33:27.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.228+0000 7fe761767700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe75c09e410 0x7fe75c1a6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:27.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.228+0000 7fe761767700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe75c09e410 0x7fe75c1a6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:36448/0 (socket says 172.21.15.16:36448) 2023-09-30T22:33:27.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.228+0000 7fe761767700 1 -- 172.21.15.16:0/3113512505 learned_addr learned my addr 172.21.15.16:0/3113512505 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:27.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.228+0000 7fe75bfff700 1 --2- 172.21.15.16:0/3113512505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe75c09e000 0x7fe75c1a63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:27.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.228+0000 7fe760f66700 1 --2- 172.21.15.16:0/3113512505 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe75c09d6f0 0x7fe75c1a5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:27.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.228+0000 7fe761767700 1 -- 172.21.15.16:0/3113512505 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe75c09d6f0 msgr2=0x7fe75c1a5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:27.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.228+0000 7fe761767700 1 --2- 172.21.15.16:0/3113512505 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe75c09d6f0 0x7fe75c1a5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:27.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.228+0000 7fe761767700 1 -- 172.21.15.16:0/3113512505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe75c09e000 msgr2=0x7fe75c1a63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:27.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.228+0000 7fe761767700 1 --2- 172.21.15.16:0/3113512505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe75c09e000 0x7fe75c1a63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:27.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.228+0000 7fe761767700 1 -- 172.21.15.16:0/3113512505 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7480094a0 con 0x7fe75c09e410 2023-09-30T22:33:27.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.228+0000 7fe761767700 1 --2- 172.21.15.16:0/3113512505 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe75c09e410 0x7fe75c1a6930 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe75400cbf0 tx=0x7fe75400cf00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:27.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.229+0000 7fe7597fa700 1 -- 172.21.15.16:0/3113512505 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75400aa60 con 0x7fe75c09e410 2023-09-30T22:33:27.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.229+0000 7fe7597fa700 1 -- 172.21.15.16:0/3113512505 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe75400abc0 con 0x7fe75c09e410 2023-09-30T22:33:27.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.229+0000 7fe7631ca700 1 -- 172.21.15.16:0/3113512505 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7500094a0 con 0x7fe75c09e410 2023-09-30T22:33:27.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.229+0000 7fe7597fa700 1 -- 172.21.15.16:0/3113512505 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe754012400 con 0x7fe75c09e410 2023-09-30T22:33:27.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.229+0000 7fe7631ca700 1 -- 172.21.15.16:0/3113512505 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe75c0a3220 con 0x7fe75c09e410 2023-09-30T22:33:27.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.230+0000 7fe7597fa700 1 -- 172.21.15.16:0/3113512505 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe754012560 con 0x7fe75c09e410 2023-09-30T22:33:27.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.230+0000 7fe7597fa700 1 -- 172.21.15.16:0/3113512505 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fe75403f3e0 con 0x7fe75c09e410 2023-09-30T22:33:27.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.230+0000 7fe7631ca700 1 -- 172.21.15.16:0/3113512505 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe75c09c710 con 0x7fe75c09e410 2023-09-30T22:33:27.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.235+0000 7fe7597fa700 1 -- 172.21.15.16:0/3113512505 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe754017020 con 0x7fe75c09e410 2023-09-30T22:33:27.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.383+0000 7fee53eb7700 1 -- 172.21.15.16:0/4249741489 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fee4c1a9610 con 0x7fee4c09e000 2023-09-30T22:33:27.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.384+0000 7fee427fc700 1 -- 172.21.15.16:0/4249741489 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fee4c1a9610 con 0x7fee4c09e000 2023-09-30T22:33:27.385 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:33:27.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.386+0000 7fee53eb7700 1 -- 172.21.15.16:0/4249741489 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fee4c09e000 msgr2=0x7fee4c1a63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:27.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.386+0000 7fee53eb7700 1 --2- 172.21.15.16:0/4249741489 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fee4c09e000 0x7fee4c1a63d0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fee3c00ebe0 tx=0x7fee3c00eef0 comp rx=0 tx=0).stop 2023-09-30T22:33:27.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.386+0000 7fee53eb7700 1 -- 172.21.15.16:0/4249741489 shutdown_connections 2023-09-30T22:33:27.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.386+0000 7fee53eb7700 1 --2- 172.21.15.16:0/4249741489 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fee4c09d6f0 0x7fee4c1a5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:27.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.386+0000 7fee53eb7700 1 --2- 172.21.15.16:0/4249741489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee4c09e410 0x7fee4c1a6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:27.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.386+0000 7fee53eb7700 1 --2- 172.21.15.16:0/4249741489 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fee4c09e000 0x7fee4c1a63d0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:27.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.386+0000 7fee53eb7700 1 -- 172.21.15.16:0/4249741489 >> 172.21.15.16:0/4249741489 conn(0x7fee4c066f50 msgr2=0x7fee4c1baeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:27.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.386+0000 7fee53eb7700 1 -- 172.21.15.16:0/4249741489 shutdown_connections 2023-09-30T22:33:27.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.386+0000 7fee53eb7700 1 -- 172.21.15.16:0/4249741489 wait complete. 2023-09-30T22:33:27.402 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:33:27.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.410+0000 7f22633fc700 1 -- 172.21.15.16:0/232915843 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f225c074ea0 con 0x7f225c1338c0 2023-09-30T22:33:27.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.410+0000 7f2251ffb700 1 -- 172.21.15.16:0/232915843 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2258017030 con 0x7f225c1338c0 2023-09-30T22:33:27.412 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:33:27.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.414+0000 7f22633fc700 1 -- 172.21.15.16:0/232915843 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f225c1338c0 msgr2=0x7f225c119ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:27.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.414+0000 7f22633fc700 1 --2- 172.21.15.16:0/232915843 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f225c1338c0 0x7f225c119ac0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f2258001eb0 tx=0x7f225800c8b0 comp rx=0 tx=0).stop 2023-09-30T22:33:27.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.414+0000 7f22633fc700 1 -- 172.21.15.16:0/232915843 shutdown_connections 2023-09-30T22:33:27.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.414+0000 7f22633fc700 1 --2- 172.21.15.16:0/232915843 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f225c1338c0 0x7f225c119ac0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:27.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.414+0000 7f22633fc700 1 --2- 172.21.15.16:0/232915843 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f225c139150 0x7f225c11a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:27.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.414+0000 7f22633fc700 1 --2- 172.21.15.16:0/232915843 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f225c136830 0x7f225c11a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:27.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.414+0000 7f22633fc700 1 -- 172.21.15.16:0/232915843 >> 172.21.15.16:0/232915843 conn(0x7f225c0a2aa0 msgr2=0x7f225c09de50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:27.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.414+0000 7f22633fc700 1 -- 172.21.15.16:0/232915843 shutdown_connections 2023-09-30T22:33:27.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.414+0000 7f22633fc700 1 -- 172.21.15.16:0/232915843 wait complete. 2023-09-30T22:33:27.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.418+0000 7fedceffc700 1 -- 172.21.15.16:0/2426310474 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fedc8137f10 con 0x7fedc806e570 2023-09-30T22:33:27.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.419+0000 7fedc57fa700 1 -- 172.21.15.16:0/2426310474 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fedbc011360 con 0x7fedc806e570 2023-09-30T22:33:27.420 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:33:27.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.421+0000 7fedceffc700 1 -- 172.21.15.16:0/2426310474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedc806e570 msgr2=0x7fedc8136470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:27.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.421+0000 7fedceffc700 1 --2- 172.21.15.16:0/2426310474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedc806e570 0x7fedc8136470 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fedbc00c700 tx=0x7fedbc00ca10 comp rx=0 tx=0).stop 2023-09-30T22:33:27.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.421+0000 7fedceffc700 1 -- 172.21.15.16:0/2426310474 shutdown_connections 2023-09-30T22:33:27.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.421+0000 7fedceffc700 1 --2- 172.21.15.16:0/2426310474 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fedc813a390 0x7fedc8136ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:27.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.421+0000 7fedceffc700 1 --2- 172.21.15.16:0/2426310474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fedc806e570 0x7fedc8136470 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:27.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.421+0000 7fedceffc700 1 --2- 172.21.15.16:0/2426310474 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fedc806f050 0x7fedc81369b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:27.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.421+0000 7fedceffc700 1 -- 172.21.15.16:0/2426310474 >> 172.21.15.16:0/2426310474 conn(0x7fedc80a2aa0 msgr2=0x7fedc809ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:27.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.421+0000 7fedceffc700 1 -- 172.21.15.16:0/2426310474 shutdown_connections 2023-09-30T22:33:27.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.422+0000 7fedceffc700 1 -- 172.21.15.16:0/2426310474 wait complete. 2023-09-30T22:33:27.430 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:33:27.438 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:33:27.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.442+0000 7fe7631ca700 1 -- 172.21.15.16:0/3113512505 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe75c074ea0 con 0x7fe75c09e410 2023-09-30T22:33:27.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.443+0000 7fe7597fa700 1 -- 172.21.15.16:0/3113512505 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe754024070 con 0x7fe75c09e410 2023-09-30T22:33:27.444 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:33:27.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.445+0000 7fe7631ca700 1 -- 172.21.15.16:0/3113512505 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe75c09e410 msgr2=0x7fe75c1a6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:27.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.445+0000 7fe7631ca700 1 --2- 172.21.15.16:0/3113512505 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe75c09e410 0x7fe75c1a6930 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe75400cbf0 tx=0x7fe75400cf00 comp rx=0 tx=0).stop 2023-09-30T22:33:27.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.445+0000 7fe7631ca700 1 -- 172.21.15.16:0/3113512505 shutdown_connections 2023-09-30T22:33:27.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.445+0000 7fe7631ca700 1 --2- 172.21.15.16:0/3113512505 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe75c09d6f0 0x7fe75c1a5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:27.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.445+0000 7fe7631ca700 1 --2- 172.21.15.16:0/3113512505 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe75c09e410 0x7fe75c1a6930 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:27.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.445+0000 7fe7631ca700 1 --2- 172.21.15.16:0/3113512505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe75c09e000 0x7fe75c1a63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:27.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.445+0000 7fe7631ca700 1 -- 172.21.15.16:0/3113512505 >> 172.21.15.16:0/3113512505 conn(0x7fe75c066f50 msgr2=0x7fe75c1badf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:27.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.445+0000 7fe7631ca700 1 -- 172.21.15.16:0/3113512505 shutdown_connections 2023-09-30T22:33:27.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:27.445+0000 7fe7631ca700 1 -- 172.21.15.16:0/3113512505 wait complete. 2023-09-30T22:33:27.460 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:33:28.402 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:33:28.432 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:33:28.438 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:33:28.461 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:33:28.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.563+0000 7fdcacded700 1 Processor -- start 2023-09-30T22:33:28.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.563+0000 7fdcacded700 1 -- start start 2023-09-30T22:33:28.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.564+0000 7fdcacded700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdca81316c0 0x7fdca8131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:28.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.564+0000 7fdcacded700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdca806ae80 0x7fdca8139550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:28.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.564+0000 7fdcacded700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdca809e130 con 0x7fdca809d9f0 2023-09-30T22:33:28.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.564+0000 7fdcacded700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdca81312b0 con 0x7fdca81316c0 2023-09-30T22:33:28.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.564+0000 7fdcacded700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdca809f6e0 con 0x7fdca806ae80 2023-09-30T22:33:28.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.564+0000 7fdca659c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdca81316c0 0x7fdca8131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:28.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.564+0000 7fdca659c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdca81316c0 0x7fdca8131a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54582/0 (socket says 172.21.15.16:54582) 2023-09-30T22:33:28.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.564+0000 7fdca5d9b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fdca809d9f0 0x7fdca806a770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:34958/0 (socket says 172.21.15.16:34958) 2023-09-30T22:33:28.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.564+0000 7fdca659c700 1 -- 172.21.15.16:0/2999447575 learned_addr learned my addr 172.21.15.16:0/2999447575 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:28.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.564+0000 7fdca6d9d700 1 --2- 172.21.15.16:0/2999447575 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdca806ae80 0x7fdca8139550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:28.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.565+0000 7fdca659c700 1 -- 172.21.15.16:0/2999447575 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdca806ae80 msgr2=0x7fdca8139550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:28.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.565+0000 7fdca659c700 1 --2- 172.21.15.16:0/2999447575 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdca806ae80 0x7fdca8139550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:28.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.565+0000 7fdca659c700 1 -- 172.21.15.16:0/2999447575 >> v1:172.21.15.16:6789/0 conn(0x7fdca809d9f0 legacy=0x7fdca806a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:28.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.565+0000 7fdca659c700 1 -- 172.21.15.16:0/2999447575 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdca809e760 con 0x7fdca81316c0 2023-09-30T22:33:28.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.565+0000 7fdca659c700 1 --2- 172.21.15.16:0/2999447575 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdca81316c0 0x7fdca8131a90 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fdc98009920 tx=0x7fdc98009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b2a1af9e5fa17c8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:28.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.565+0000 7fdca559a700 1 -- 172.21.15.16:0/2999447575 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc98011830 con 0x7fdca81316c0 2023-09-30T22:33:28.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.565+0000 7fdca559a700 1 -- 172.21.15.16:0/2999447575 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc98011990 con 0x7fdca81316c0 2023-09-30T22:33:28.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.565+0000 7fdca559a700 1 -- 172.21.15.16:0/2999447575 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc98011d00 con 0x7fdca81316c0 2023-09-30T22:33:28.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.566+0000 7fdcacded700 1 -- 172.21.15.16:0/2999447575 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdca81316c0 msgr2=0x7fdca8131a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:28.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.566+0000 7fdcacded700 1 --2- 172.21.15.16:0/2999447575 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdca81316c0 0x7fdca8131a90 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fdc98009920 tx=0x7fdc98009c30 comp rx=0 tx=0).stop 2023-09-30T22:33:28.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.566+0000 7fdcacded700 1 -- 172.21.15.16:0/2999447575 shutdown_connections 2023-09-30T22:33:28.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.566+0000 7fdcacded700 1 --2- 172.21.15.16:0/2999447575 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdca806ae80 0x7fdca8139550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:28.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.566+0000 7fdcacded700 1 --2- 172.21.15.16:0/2999447575 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdca81316c0 0x7fdca8131a90 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:28.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.566+0000 7fdcacded700 1 -- 172.21.15.16:0/2999447575 >> 172.21.15.16:0/2999447575 conn(0x7fdca8066f50 msgr2=0x7fdca809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:28.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.566+0000 7fdcacded700 1 -- 172.21.15.16:0/2999447575 shutdown_connections 2023-09-30T22:33:28.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.566+0000 7fdcacded700 1 -- 172.21.15.16:0/2999447575 wait complete. 2023-09-30T22:33:28.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.568+0000 7fdcacded700 1 Processor -- start 2023-09-30T22:33:28.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.568+0000 7fdcacded700 1 -- start start 2023-09-30T22:33:28.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.568+0000 7fdcacded700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdca806ae80 0x7fdca8070860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:28.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.568+0000 7fdcacded700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdca809d9f0 0x7fdca8070da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:28.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.568+0000 7fdcacded700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdca81316c0 0x7fdca80712e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:28.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.568+0000 7fdcacded700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca812b900 con 0x7fdca809d9f0 2023-09-30T22:33:28.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.568+0000 7fdcacded700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdca811a8f0 con 0x7fdca806ae80 2023-09-30T22:33:28.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.568+0000 7fdcacded700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdca8071c50 con 0x7fdca81316c0 2023-09-30T22:33:28.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.569+0000 7fdca6d9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdca81316c0 0x7fdca80712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:28.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.569+0000 7fdca6d9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdca81316c0 0x7fdca80712e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:53554/0 (socket says 172.21.15.16:53554) 2023-09-30T22:33:28.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.569+0000 7fdca6d9d700 1 -- 172.21.15.16:0/1581646587 learned_addr learned my addr 172.21.15.16:0/1581646587 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:28.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.569+0000 7fdca659c700 1 --2- 172.21.15.16:0/1581646587 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdca806ae80 0x7fdca8070860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:28.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.569+0000 7fdca5d9b700 1 --2- 172.21.15.16:0/1581646587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdca809d9f0 0x7fdca8070da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:28.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.569+0000 7fdca659c700 1 -- 172.21.15.16:0/1581646587 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdca81316c0 msgr2=0x7fdca80712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:28.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.569+0000 7fdca659c700 1 --2- 172.21.15.16:0/1581646587 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdca81316c0 0x7fdca80712e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:28.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.569+0000 7fdca659c700 1 -- 172.21.15.16:0/1581646587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdca809d9f0 msgr2=0x7fdca8070da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:28.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.569+0000 7fdca659c700 1 --2- 172.21.15.16:0/1581646587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdca809d9f0 0x7fdca8070da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:28.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.569+0000 7fdca659c700 1 -- 172.21.15.16:0/1581646587 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc98009550 con 0x7fdca806ae80 2023-09-30T22:33:28.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.569+0000 7fdca5d9b700 1 --2- 172.21.15.16:0/1581646587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdca809d9f0 0x7fdca8070da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:33:28.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.569+0000 7fdca659c700 1 --2- 172.21.15.16:0/1581646587 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdca806ae80 0x7fdca8070860 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fdc98016000 tx=0x7fdc98004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:28.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.570+0000 7fdc96ffd700 1 -- 172.21.15.16:0/1581646587 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc98004450 con 0x7fdca806ae80 2023-09-30T22:33:28.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.570+0000 7fdc96ffd700 1 -- 172.21.15.16:0/1581646587 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc98023070 con 0x7fdca806ae80 2023-09-30T22:33:28.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.570+0000 7fdcacded700 1 -- 172.21.15.16:0/1581646587 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdca81197e0 con 0x7fdca806ae80 2023-09-30T22:33:28.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.570+0000 7fdc96ffd700 1 -- 172.21.15.16:0/1581646587 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc980045f0 con 0x7fdca806ae80 2023-09-30T22:33:28.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.570+0000 7fdcacded700 1 -- 172.21.15.16:0/1581646587 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdca811abb0 con 0x7fdca806ae80 2023-09-30T22:33:28.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.571+0000 7fdc96ffd700 1 -- 172.21.15.16:0/1581646587 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdc9801fa80 con 0x7fdca806ae80 2023-09-30T22:33:28.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.571+0000 7fdcacded700 1 -- 172.21.15.16:0/1581646587 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdca811ada0 con 0x7fdca806ae80 2023-09-30T22:33:28.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.571+0000 7fdc96ffd700 1 -- 172.21.15.16:0/1581646587 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fdc98025070 con 0x7fdca806ae80 2023-09-30T22:33:28.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.577+0000 7fdc96ffd700 1 -- 172.21.15.16:0/1581646587 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdc98003bc0 con 0x7fdca806ae80 2023-09-30T22:33:28.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.601+0000 7f1ef4377700 1 Processor -- start 2023-09-30T22:33:28.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.601+0000 7f1ef4377700 1 -- start start 2023-09-30T22:33:28.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.601+0000 7f7cab306700 1 Processor -- start 2023-09-30T22:33:28.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.602+0000 7f1ef4377700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1eec09f410 0x7f1eec136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:28.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.602+0000 7f1ef4377700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1eec136fe0 0x7f1eec1393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:28.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.602+0000 7f1ef4377700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1eec070940 con 0x7f1eec09e930 2023-09-30T22:33:28.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.602+0000 7f1ef4377700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1eec06fd60 con 0x7f1eec136fe0 2023-09-30T22:33:28.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.602+0000 7f1ef4377700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eec06fa80 con 0x7f1eec09f410 2023-09-30T22:33:28.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.602+0000 7f1ef1912700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1eec09f410 0x7f1eec136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:28.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.602+0000 7f1ef1912700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1eec09f410 0x7f1eec136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54594/0 (socket says 172.21.15.16:54594) 2023-09-30T22:33:28.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.602+0000 7f1ef1912700 1 -- 172.21.15.16:0/1988728749 learned_addr learned my addr 172.21.15.16:0/1988728749 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:28.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.603+0000 7f1ef1912700 1 -- 172.21.15.16:0/1988728749 >> v2:172.21.15.16:3300/0 conn(0x7f1eec136fe0 msgr2=0x7f1eec1393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:28.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.603+0000 7f1ef1912700 1 --2- 172.21.15.16:0/1988728749 >> v2:172.21.15.16:3300/0 conn(0x7f1eec136fe0 0x7f1eec1393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:28.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.603+0000 7f1ef1912700 1 -- 172.21.15.16:0/1988728749 >> v1:172.21.15.16:6789/0 conn(0x7f1eec09e930 legacy=0x7f1eec09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:28.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.603+0000 7f1ef1912700 1 -- 172.21.15.16:0/1988728749 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eec09fb60 con 0x7f1eec09f410 2023-09-30T22:33:28.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.602+0000 7f7cab306700 1 -- start start 2023-09-30T22:33:28.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.603+0000 7f7cab306700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7ca406e570 0x7f7ca406e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:28.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.603+0000 7f7cab306700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7ca409eca0 0x7f7ca406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:28.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.603+0000 7f7cab306700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ca4070ee0 con 0x7f7ca4134610 2023-09-30T22:33:28.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.603+0000 7f7cab306700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ca4071480 con 0x7f7ca406e570 2023-09-30T22:33:28.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.603+0000 7f7cab306700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca406f3c0 con 0x7f7ca409eca0 2023-09-30T22:33:28.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.603+0000 7f1ef1912700 1 --2- 172.21.15.16:0/1988728749 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1eec09f410 0x7f1eec136aa0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f1edc0098f0 tx=0x7f1edc009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d2cde5244be960e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:28.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.603+0000 7f1ef1111700 1 -- 172.21.15.16:0/1988728749 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1edc004230 con 0x7f1eec09f410 2023-09-30T22:33:28.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.603+0000 7f1ef1111700 1 -- 172.21.15.16:0/1988728749 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1edc004390 con 0x7f1eec09f410 2023-09-30T22:33:28.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.603+0000 7f1ef1111700 1 -- 172.21.15.16:0/1988728749 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1edc0036a0 con 0x7f1eec09f410 2023-09-30T22:33:28.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.604+0000 7f1ef4377700 1 -- 172.21.15.16:0/1988728749 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1eec09f410 msgr2=0x7f1eec136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:28.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.604+0000 7f1ef4377700 1 --2- 172.21.15.16:0/1988728749 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1eec09f410 0x7f1eec136aa0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f1edc0098f0 tx=0x7f1edc009c00 comp rx=0 tx=0).stop 2023-09-30T22:33:28.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.604+0000 7f7ca88a1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7ca409eca0 0x7f7ca406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:28.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.604+0000 7f7ca88a1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7ca409eca0 0x7f7ca406ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54604/0 (socket says 172.21.15.16:54604) 2023-09-30T22:33:28.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.604+0000 7f7ca88a1700 1 -- 172.21.15.16:0/2970743459 learned_addr learned my addr 172.21.15.16:0/2970743459 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:28.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.604+0000 7f1ef4377700 1 -- 172.21.15.16:0/1988728749 shutdown_connections 2023-09-30T22:33:28.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.604+0000 7f1ef4377700 1 --2- 172.21.15.16:0/1988728749 >> v2:172.21.15.16:3300/0 conn(0x7f1eec136fe0 0x7f1eec1393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:28.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.604+0000 7f1ef4377700 1 --2- 172.21.15.16:0/1988728749 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1eec09f410 0x7f1eec136aa0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:28.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.604+0000 7f1ef4377700 1 -- 172.21.15.16:0/1988728749 >> 172.21.15.16:0/1988728749 conn(0x7f1eec0a4210 msgr2=0x7f1eec0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:28.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.604+0000 7f1ef4377700 1 -- 172.21.15.16:0/1988728749 shutdown_connections 2023-09-30T22:33:28.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.605+0000 7f1ef4377700 1 -- 172.21.15.16:0/1988728749 wait complete. 2023-09-30T22:33:28.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.605+0000 7f7ca88a1700 1 -- 172.21.15.16:0/2970743459 >> v2:172.21.15.16:3300/0 conn(0x7f7ca406e570 msgr2=0x7f7ca406e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:28.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.605+0000 7f7ca88a1700 1 --2- 172.21.15.16:0/2970743459 >> v2:172.21.15.16:3300/0 conn(0x7f7ca406e570 0x7f7ca406e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:28.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.605+0000 7f7ca88a1700 1 -- 172.21.15.16:0/2970743459 >> v1:172.21.15.16:6789/0 conn(0x7f7ca4134610 legacy=0x7f7ca4134a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:28.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.605+0000 7f7ca88a1700 1 -- 172.21.15.16:0/2970743459 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca4071b90 con 0x7f7ca409eca0 2023-09-30T22:33:28.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.605+0000 7f7ca88a1700 1 --2- 172.21.15.16:0/2970743459 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7ca409eca0 0x7f7ca406ee80 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f7c940098f0 tx=0x7f7c94009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8363dff2967a25ea server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:28.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.605+0000 7f7c9bfff700 1 -- 172.21.15.16:0/2970743459 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c94004230 con 0x7f7ca409eca0 2023-09-30T22:33:28.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.605+0000 7f7c9bfff700 1 -- 172.21.15.16:0/2970743459 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c94004390 con 0x7f7ca409eca0 2023-09-30T22:33:28.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.605+0000 7f7c9bfff700 1 -- 172.21.15.16:0/2970743459 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c940036a0 con 0x7f7ca409eca0 2023-09-30T22:33:28.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.606+0000 7f1ef4377700 1 Processor -- start 2023-09-30T22:33:28.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.606+0000 7f7cab306700 1 -- 172.21.15.16:0/2970743459 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7ca409eca0 msgr2=0x7f7ca406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:28.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.606+0000 7f7cab306700 1 --2- 172.21.15.16:0/2970743459 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7ca409eca0 0x7f7ca406ee80 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f7c940098f0 tx=0x7f7c94009c00 comp rx=0 tx=0).stop 2023-09-30T22:33:28.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.606+0000 7f1ef4377700 1 -- start start 2023-09-30T22:33:28.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.606+0000 7f1ef4377700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1eec09e930 0x7f1eec1c6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:28.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.606+0000 7f1ef4377700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1eec09f410 0x7f1eec1c6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:28.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.606+0000 7f1ef4377700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1eec136fe0 0x7f1eec11c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:28.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.606+0000 7f1ef4377700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eec11e540 con 0x7f1eec136fe0 2023-09-30T22:33:28.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.606+0000 7f1ef4377700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eec11e350 con 0x7f1eec09f410 2023-09-30T22:33:28.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.606+0000 7f1ef4377700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1eec1c7080 con 0x7f1eec09e930 2023-09-30T22:33:28.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.606+0000 7f1ef2914700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1eec136fe0 0x7f1eec11c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:28.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.606+0000 7f1ef2113700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1eec09e930 0x7f1eec1c6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:28.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.606+0000 7f1ef2914700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1eec136fe0 0x7f1eec11c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55204/0 (socket says 172.21.15.16:55204) 2023-09-30T22:33:28.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.606+0000 7f1ef2914700 1 -- 172.21.15.16:0/4227509649 learned_addr learned my addr 172.21.15.16:0/4227509649 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:28.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.607+0000 7f1ef2914700 1 -- 172.21.15.16:0/4227509649 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1eec09e930 msgr2=0x7f1eec1c6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:28.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.607+0000 7f1ef2914700 1 --2- 172.21.15.16:0/4227509649 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1eec09e930 0x7f1eec1c6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:28.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.607+0000 7f1ef2914700 1 -- 172.21.15.16:0/4227509649 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1eec09f410 msgr2=0x7f1eec1c6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:28.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.607+0000 7f1ef2914700 1 --2- 172.21.15.16:0/4227509649 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1eec09f410 0x7f1eec1c6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:28.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.607+0000 7f1ef2914700 1 -- 172.21.15.16:0/4227509649 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1edc009550 con 0x7f1eec136fe0 2023-09-30T22:33:28.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.607+0000 7f1ef2113700 1 --2- 172.21.15.16:0/4227509649 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1eec09e930 0x7f1eec1c6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:28.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.607+0000 7f7cab306700 1 -- 172.21.15.16:0/2970743459 shutdown_connections 2023-09-30T22:33:28.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.607+0000 7f7cab306700 1 --2- 172.21.15.16:0/2970743459 >> v2:172.21.15.16:3300/0 conn(0x7f7ca406e570 0x7f7ca406e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:28.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.607+0000 7f7cab306700 1 --2- 172.21.15.16:0/2970743459 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7ca409eca0 0x7f7ca406ee80 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:28.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.607+0000 7f7cab306700 1 -- 172.21.15.16:0/2970743459 >> 172.21.15.16:0/2970743459 conn(0x7f7ca40a2aa0 msgr2=0x7f7ca40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:28.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.607+0000 7f1ef2914700 1 --2- 172.21.15.16:0/4227509649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1eec136fe0 0x7f1eec11c740 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f1ee400ba10 tx=0x7f1ee400bd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:28.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.607+0000 7f7cab306700 1 -- 172.21.15.16:0/2970743459 shutdown_connections 2023-09-30T22:33:28.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.607+0000 7f7cab306700 1 -- 172.21.15.16:0/2970743459 wait complete. 2023-09-30T22:33:28.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.607+0000 7f1ee2ffd700 1 -- 172.21.15.16:0/4227509649 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee400c930 con 0x7f1eec136fe0 2023-09-30T22:33:28.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.607+0000 7f1ee2ffd700 1 -- 172.21.15.16:0/4227509649 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ee400cad0 con 0x7f1eec136fe0 2023-09-30T22:33:28.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.607+0000 7f1ef4377700 1 -- 172.21.15.16:0/4227509649 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1eec11e930 con 0x7f1eec136fe0 2023-09-30T22:33:28.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.607+0000 7f1ee2ffd700 1 -- 172.21.15.16:0/4227509649 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee400ce80 con 0x7f1eec136fe0 2023-09-30T22:33:28.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.607+0000 7f1ef4377700 1 -- 172.21.15.16:0/4227509649 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eec11eb40 con 0x7f1eec136fe0 2023-09-30T22:33:28.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.608+0000 7f7cab306700 1 Processor -- start 2023-09-30T22:33:28.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.609+0000 7f1ef4377700 1 -- 172.21.15.16:0/4227509649 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1eec074ea0 con 0x7f1eec136fe0 2023-09-30T22:33:28.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.608+0000 7f7cab306700 1 -- start start 2023-09-30T22:33:28.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.609+0000 7f7cab306700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7ca406e570 0x7f7ca41340b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:28.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.609+0000 7f7cab306700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7ca4134610 0x7f7ca4132e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:28.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.609+0000 7f7cab306700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ca413e630 0x7f7ca41333b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:28.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.609+0000 7f7cab306700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca413d960 con 0x7f7ca413e630 2023-09-30T22:33:28.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.609+0000 7f7cab306700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca413e130 con 0x7f7ca4134610 2023-09-30T22:33:28.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.609+0000 7f7cab306700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca413ca80 con 0x7f7ca406e570 2023-09-30T22:33:28.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.609+0000 7f7ca90a2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7ca406e570 0x7f7ca41340b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:28.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.609+0000 7f7ca90a2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7ca406e570 0x7f7ca41340b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:53568/0 (socket says 172.21.15.16:53568) 2023-09-30T22:33:28.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.609+0000 7f7ca90a2700 1 -- 172.21.15.16:0/2496240683 learned_addr learned my addr 172.21.15.16:0/2496240683 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:28.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.609+0000 7f7ca88a1700 1 --2- 172.21.15.16:0/2496240683 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7ca4134610 0x7f7ca4132e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:28.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.609+0000 7f7ca98a3700 1 --2- 172.21.15.16:0/2496240683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ca413e630 0x7f7ca41333b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:28.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.610+0000 7f7ca90a2700 1 -- 172.21.15.16:0/2496240683 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7ca4134610 msgr2=0x7f7ca4132e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:28.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.610+0000 7f7ca90a2700 1 --2- 172.21.15.16:0/2496240683 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7ca4134610 0x7f7ca4132e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:28.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.610+0000 7f7ca90a2700 1 -- 172.21.15.16:0/2496240683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ca413e630 msgr2=0x7f7ca41333b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:28.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.610+0000 7f7ca90a2700 1 --2- 172.21.15.16:0/2496240683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ca413e630 0x7f7ca41333b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:28.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.610+0000 7f7ca90a2700 1 -- 172.21.15.16:0/2496240683 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c94009550 con 0x7f7ca406e570 2023-09-30T22:33:28.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.610+0000 7f7ca90a2700 1 --2- 172.21.15.16:0/2496240683 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7ca406e570 0x7f7ca41340b0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f7ca000b560 tx=0x7f7ca000b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:28.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.612+0000 7f1ee2ffd700 1 -- 172.21.15.16:0/4227509649 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1ee4014440 con 0x7f1eec136fe0 2023-09-30T22:33:28.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.612+0000 7f1ee2ffd700 1 -- 172.21.15.16:0/4227509649 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f1ee4006e40 con 0x7f1eec136fe0 2023-09-30T22:33:28.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.612+0000 7f7c99ffb700 1 -- 172.21.15.16:0/2496240683 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca0011830 con 0x7f7ca406e570 2023-09-30T22:33:28.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.612+0000 7f7c99ffb700 1 -- 172.21.15.16:0/2496240683 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ca0011990 con 0x7f7ca406e570 2023-09-30T22:33:28.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.612+0000 7f7c99ffb700 1 -- 172.21.15.16:0/2496240683 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca0011d20 con 0x7f7ca406e570 2023-09-30T22:33:28.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.612+0000 7f7cab306700 1 -- 172.21.15.16:0/2496240683 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ca40a1fe0 con 0x7f7ca406e570 2023-09-30T22:33:28.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.612+0000 7f7cab306700 1 -- 172.21.15.16:0/2496240683 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ca41338f0 con 0x7f7ca406e570 2023-09-30T22:33:28.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.614+0000 7f7cab306700 1 -- 172.21.15.16:0/2496240683 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ca4074ea0 con 0x7f7ca406e570 2023-09-30T22:33:28.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.614+0000 7f7c99ffb700 1 -- 172.21.15.16:0/2496240683 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7ca001a400 con 0x7f7ca406e570 2023-09-30T22:33:28.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.615+0000 7f7c99ffb700 1 -- 172.21.15.16:0/2496240683 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f7ca000f6e0 con 0x7f7ca406e570 2023-09-30T22:33:28.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.619+0000 7f1ee2ffd700 1 -- 172.21.15.16:0/4227509649 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1ee40133a0 con 0x7f1eec136fe0 2023-09-30T22:33:28.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.619+0000 7f7c99ffb700 1 -- 172.21.15.16:0/2496240683 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7ca0044060 con 0x7f7ca406e570 2023-09-30T22:33:28.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.651+0000 7f6f5724c700 1 Processor -- start 2023-09-30T22:33:28.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.651+0000 7f6f5724c700 1 -- start start 2023-09-30T22:33:28.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.652+0000 7f6f5724c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6f50134630 0x7f6f50136a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:28.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.652+0000 7f6f5724c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6f50136f50 0x7f6f50139330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:28.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.652+0000 7f6f5724c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f5009b4f0 con 0x7f6f50131810 2023-09-30T22:33:28.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.652+0000 7f6f5724c700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f501313c0 con 0x7f6f50134630 2023-09-30T22:33:28.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.652+0000 7f6f5724c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5006e8c0 con 0x7f6f50136f50 2023-09-30T22:33:28.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.652+0000 7f6f4ffff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6f50134630 0x7f6f50136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:28.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.652+0000 7f6f4ffff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6f50134630 0x7f6f50136a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55220/0 (socket says 172.21.15.16:55220) 2023-09-30T22:33:28.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.652+0000 7f6f4ffff700 1 -- 172.21.15.16:0/43394622 learned_addr learned my addr 172.21.15.16:0/43394622 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:28.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.652+0000 7f6f557e9700 1 --2- 172.21.15.16:0/43394622 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6f50136f50 0x7f6f50139330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:28.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.652+0000 7f6f4ffff700 1 -- 172.21.15.16:0/43394622 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6f50136f50 msgr2=0x7f6f50139330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:28.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.652+0000 7f6f4ffff700 1 --2- 172.21.15.16:0/43394622 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6f50136f50 0x7f6f50139330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:28.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.652+0000 7f6f4ffff700 1 -- 172.21.15.16:0/43394622 >> v1:172.21.15.16:6789/0 conn(0x7f6f50131810 legacy=0x7f6f50131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:28.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.652+0000 7f6f4f7fe700 1 -- 172.21.15.16:0/43394622 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2745717071 0 0) 0x7f6f5009b4f0 con 0x7f6f50131810 2023-09-30T22:33:28.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.653+0000 7f6f4ffff700 1 -- 172.21.15.16:0/43394622 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f50071d20 con 0x7f6f50134630 2023-09-30T22:33:28.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.653+0000 7f6f4ffff700 1 --2- 172.21.15.16:0/43394622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f50134630 0x7f6f50136a10 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c0098f0 tx=0x7f6f3c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d09626a1a12f35a1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:28.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.653+0000 7f6f4f7fe700 1 -- 172.21.15.16:0/43394622 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c011830 con 0x7f6f50134630 2023-09-30T22:33:28.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.653+0000 7f6f4f7fe700 1 -- 172.21.15.16:0/43394622 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f3c011990 con 0x7f6f50134630 2023-09-30T22:33:28.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.653+0000 7f6f4f7fe700 1 -- 172.21.15.16:0/43394622 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c011d20 con 0x7f6f50134630 2023-09-30T22:33:28.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.654+0000 7f6f5724c700 1 -- 172.21.15.16:0/43394622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f50134630 msgr2=0x7f6f50136a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:28.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.654+0000 7f6f5724c700 1 --2- 172.21.15.16:0/43394622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f50134630 0x7f6f50136a10 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c0098f0 tx=0x7f6f3c009c00 comp rx=0 tx=0).stop 2023-09-30T22:33:28.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.654+0000 7f6f5724c700 1 -- 172.21.15.16:0/43394622 shutdown_connections 2023-09-30T22:33:28.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.654+0000 7f6f5724c700 1 --2- 172.21.15.16:0/43394622 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6f50136f50 0x7f6f50139330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:28.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.654+0000 7f6f5724c700 1 --2- 172.21.15.16:0/43394622 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f50134630 0x7f6f50136a10 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:28.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.654+0000 7f6f5724c700 1 -- 172.21.15.16:0/43394622 >> 172.21.15.16:0/43394622 conn(0x7f6f500a2aa0 msgr2=0x7f6f500a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:28.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.654+0000 7f6f5724c700 1 -- 172.21.15.16:0/43394622 shutdown_connections 2023-09-30T22:33:28.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.654+0000 7f6f5724c700 1 -- 172.21.15.16:0/43394622 wait complete. 2023-09-30T22:33:28.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.655+0000 7f6f5724c700 1 Processor -- start 2023-09-30T22:33:28.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.656+0000 7f6f5724c700 1 -- start start 2023-09-30T22:33:28.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.656+0000 7f6f5724c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6f50131810 0x7f6f5006fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:28.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.656+0000 7f6f5724c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6f50134630 0x7f6f500703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:28.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.656+0000 7f6f5724c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f50136f50 0x7f6f501c8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:28.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.656+0000 7f6f5724c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f500712b0 con 0x7f6f50136f50 2023-09-30T22:33:28.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.656+0000 7f6f5724c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5011e3f0 con 0x7f6f50131810 2023-09-30T22:33:28.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.656+0000 7f6f5724c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6f50136d20 con 0x7f6f50134630 2023-09-30T22:33:28.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.657+0000 7f6f54fe8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6f50131810 0x7f6f5006fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:28.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.657+0000 7f6f54fe8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6f50131810 0x7f6f5006fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54646/0 (socket says 172.21.15.16:54646) 2023-09-30T22:33:28.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.657+0000 7f6f54fe8700 1 -- 172.21.15.16:0/907835870 learned_addr learned my addr 172.21.15.16:0/907835870 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:28.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.657+0000 7f6f557e9700 1 --2- 172.21.15.16:0/907835870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f50136f50 0x7f6f501c8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:28.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.657+0000 7f6f4ffff700 1 --2- 172.21.15.16:0/907835870 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6f50134630 0x7f6f500703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:28.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.657+0000 7f6f54fe8700 1 -- 172.21.15.16:0/907835870 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6f50134630 msgr2=0x7f6f500703d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:28.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.657+0000 7f6f54fe8700 1 --2- 172.21.15.16:0/907835870 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6f50134630 0x7f6f500703d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:28.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.657+0000 7f6f54fe8700 1 -- 172.21.15.16:0/907835870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f50136f50 msgr2=0x7f6f501c8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:28.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.657+0000 7f6f54fe8700 1 --2- 172.21.15.16:0/907835870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f50136f50 0x7f6f501c8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:28.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.657+0000 7f6f54fe8700 1 -- 172.21.15.16:0/907835870 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f3c009550 con 0x7f6f50131810 2023-09-30T22:33:28.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.657+0000 7f6f557e9700 1 --2- 172.21.15.16:0/907835870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f50136f50 0x7f6f501c8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:28.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.657+0000 7f6f54fe8700 1 --2- 172.21.15.16:0/907835870 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6f50131810 0x7f6f5006fe90 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f6f4400c450 tx=0x7f6f4400c760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:28.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.658+0000 7f6f4d7fa700 1 -- 172.21.15.16:0/907835870 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f440054e0 con 0x7f6f50131810 2023-09-30T22:33:28.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.658+0000 7f6f4d7fa700 1 -- 172.21.15.16:0/907835870 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f4400ce60 con 0x7f6f50131810 2023-09-30T22:33:28.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.658+0000 7f6f5724c700 1 -- 172.21.15.16:0/907835870 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f50070910 con 0x7f6f50131810 2023-09-30T22:33:28.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.658+0000 7f6f4d7fa700 1 -- 172.21.15.16:0/907835870 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f440038c0 con 0x7f6f50131810 2023-09-30T22:33:28.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.658+0000 7f6f5724c700 1 -- 172.21.15.16:0/907835870 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f5011f120 con 0x7f6f50131810 2023-09-30T22:33:28.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.659+0000 7f6f4d7fa700 1 -- 172.21.15.16:0/907835870 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6f44003a20 con 0x7f6f50131810 2023-09-30T22:33:28.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.659+0000 7f6f4d7fa700 1 -- 172.21.15.16:0/907835870 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f6f4403eb90 con 0x7f6f50131810 2023-09-30T22:33:28.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.659+0000 7f6f5724c700 1 -- 172.21.15.16:0/907835870 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f50074ea0 con 0x7f6f50131810 2023-09-30T22:33:28.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.664+0000 7f6f4d7fa700 1 -- 172.21.15.16:0/907835870 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6f44017030 con 0x7f6f50131810 2023-09-30T22:33:28.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.801+0000 7fdcacded700 1 -- 172.21.15.16:0/1581646587 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fdca8074ea0 con 0x7fdca806ae80 2023-09-30T22:33:28.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.802+0000 7fdc96ffd700 1 -- 172.21.15.16:0/1581646587 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdc980323f0 con 0x7fdca806ae80 2023-09-30T22:33:28.803 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:33:28.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.804+0000 7fdcacded700 1 -- 172.21.15.16:0/1581646587 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdca806ae80 msgr2=0x7fdca8070860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:28.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.804+0000 7fdcacded700 1 --2- 172.21.15.16:0/1581646587 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdca806ae80 0x7fdca8070860 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fdc98016000 tx=0x7fdc98004750 comp rx=0 tx=0).stop 2023-09-30T22:33:28.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.804+0000 7fdcacded700 1 -- 172.21.15.16:0/1581646587 shutdown_connections 2023-09-30T22:33:28.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.804+0000 7fdcacded700 1 --2- 172.21.15.16:0/1581646587 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdca806ae80 0x7fdca8070860 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:28.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.804+0000 7fdcacded700 1 --2- 172.21.15.16:0/1581646587 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdca81316c0 0x7fdca80712e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:28.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.804+0000 7fdcacded700 1 --2- 172.21.15.16:0/1581646587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdca809d9f0 0x7fdca8070da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:28.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.804+0000 7fdcacded700 1 -- 172.21.15.16:0/1581646587 >> 172.21.15.16:0/1581646587 conn(0x7fdca8066f50 msgr2=0x7fdca80a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:28.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.804+0000 7fdcacded700 1 -- 172.21.15.16:0/1581646587 shutdown_connections 2023-09-30T22:33:28.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.804+0000 7fdcacded700 1 -- 172.21.15.16:0/1581646587 wait complete. 2023-09-30T22:33:28.824 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:33:28.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.836+0000 7f7cab306700 1 -- 172.21.15.16:0/2496240683 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7ca4071100 con 0x7f7ca406e570 2023-09-30T22:33:28.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.837+0000 7f7c99ffb700 1 -- 172.21.15.16:0/2496240683 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7ca0044030 con 0x7f7ca406e570 2023-09-30T22:33:28.838 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:33:28.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.839+0000 7f7cab306700 1 -- 172.21.15.16:0/2496240683 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7ca406e570 msgr2=0x7f7ca41340b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:28.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.839+0000 7f7cab306700 1 --2- 172.21.15.16:0/2496240683 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7ca406e570 0x7f7ca41340b0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f7ca000b560 tx=0x7f7ca000b870 comp rx=0 tx=0).stop 2023-09-30T22:33:28.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.839+0000 7f7cab306700 1 -- 172.21.15.16:0/2496240683 shutdown_connections 2023-09-30T22:33:28.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.839+0000 7f7cab306700 1 --2- 172.21.15.16:0/2496240683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ca413e630 0x7f7ca41333b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:28.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.839+0000 7f7cab306700 1 --2- 172.21.15.16:0/2496240683 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7ca406e570 0x7f7ca41340b0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:28.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.839+0000 7f7cab306700 1 --2- 172.21.15.16:0/2496240683 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7ca4134610 0x7f7ca4132e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:28.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.839+0000 7f7cab306700 1 -- 172.21.15.16:0/2496240683 >> 172.21.15.16:0/2496240683 conn(0x7f7ca40a2aa0 msgr2=0x7f7ca40a0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:28.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.839+0000 7f7cab306700 1 -- 172.21.15.16:0/2496240683 shutdown_connections 2023-09-30T22:33:28.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.839+0000 7f7cab306700 1 -- 172.21.15.16:0/2496240683 wait complete. 2023-09-30T22:33:28.854 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:33:28.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.865+0000 7f1ef4377700 1 -- 172.21.15.16:0/4227509649 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1eec06b250 con 0x7f1eec136fe0 2023-09-30T22:33:28.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.866+0000 7f1ee2ffd700 1 -- 172.21.15.16:0/4227509649 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1ee4007b30 con 0x7f1eec136fe0 2023-09-30T22:33:28.867 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:33:28.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.868+0000 7f1ef4377700 1 -- 172.21.15.16:0/4227509649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1eec136fe0 msgr2=0x7f1eec11c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:28.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.868+0000 7f1ef4377700 1 --2- 172.21.15.16:0/4227509649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1eec136fe0 0x7f1eec11c740 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f1ee400ba10 tx=0x7f1ee400bd20 comp rx=0 tx=0).stop 2023-09-30T22:33:28.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.868+0000 7f1ef4377700 1 -- 172.21.15.16:0/4227509649 shutdown_connections 2023-09-30T22:33:28.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.868+0000 7f1ef4377700 1 --2- 172.21.15.16:0/4227509649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1eec136fe0 0x7f1eec11c740 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:28.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.868+0000 7f1ef4377700 1 --2- 172.21.15.16:0/4227509649 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1eec09e930 0x7f1eec1c6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:28.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.868+0000 7f1ef4377700 1 --2- 172.21.15.16:0/4227509649 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1eec09f410 0x7f1eec1c6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:28.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.868+0000 7f1ef4377700 1 -- 172.21.15.16:0/4227509649 >> 172.21.15.16:0/4227509649 conn(0x7f1eec0a4210 msgr2=0x7f1eec135f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:28.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.868+0000 7f1ef4377700 1 -- 172.21.15.16:0/4227509649 shutdown_connections 2023-09-30T22:33:28.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.868+0000 7f1ef4377700 1 -- 172.21.15.16:0/4227509649 wait complete. 2023-09-30T22:33:28.883 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:33:28.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.909+0000 7f6f5724c700 1 -- 172.21.15.16:0/907835870 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6f5006b9e0 con 0x7f6f50131810 2023-09-30T22:33:28.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.909+0000 7f6f4d7fa700 1 -- 172.21.15.16:0/907835870 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6f44019cd0 con 0x7f6f50131810 2023-09-30T22:33:28.911 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:33:28.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.912+0000 7f6f5724c700 1 -- 172.21.15.16:0/907835870 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6f50131810 msgr2=0x7f6f5006fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:28.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.912+0000 7f6f5724c700 1 --2- 172.21.15.16:0/907835870 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6f50131810 0x7f6f5006fe90 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f6f4400c450 tx=0x7f6f4400c760 comp rx=0 tx=0).stop 2023-09-30T22:33:28.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.912+0000 7f6f5724c700 1 -- 172.21.15.16:0/907835870 shutdown_connections 2023-09-30T22:33:28.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.912+0000 7f6f5724c700 1 --2- 172.21.15.16:0/907835870 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6f50131810 0x7f6f5006fe90 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:28.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.912+0000 7f6f5724c700 1 --2- 172.21.15.16:0/907835870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f50136f50 0x7f6f501c8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:28.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.912+0000 7f6f5724c700 1 --2- 172.21.15.16:0/907835870 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6f50134630 0x7f6f500703d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:28.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.912+0000 7f6f5724c700 1 -- 172.21.15.16:0/907835870 >> 172.21.15.16:0/907835870 conn(0x7f6f500a2aa0 msgr2=0x7f6f5012f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:28.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.912+0000 7f6f5724c700 1 -- 172.21.15.16:0/907835870 shutdown_connections 2023-09-30T22:33:28.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:28.912+0000 7f6f5724c700 1 -- 172.21.15.16:0/907835870 wait complete. 2023-09-30T22:33:28.927 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:33:29.825 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:33:29.856 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:33:29.884 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:33:29.928 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:33:29.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:29.982+0000 7ff570d16700 1 Processor -- start 2023-09-30T22:33:29.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:29.982+0000 7ff570d16700 1 -- start start 2023-09-30T22:33:29.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:29.983+0000 7ff570d16700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff56c06e570 0x7ff56c06e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:29.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:29.983+0000 7ff570d16700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff56c06ee80 0x7ff56c134d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:29.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:29.983+0000 7ff570d16700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff56c070ee0 con 0x7ff56c09eca0 2023-09-30T22:33:29.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:29.983+0000 7ff570d16700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff56c071480 con 0x7ff56c06ee80 2023-09-30T22:33:29.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:29.983+0000 7ff570d16700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff56c071b90 con 0x7ff56c06e570 2023-09-30T22:33:29.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:29.983+0000 7ff56a59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff56c06e570 0x7ff56c06e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:29.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:29.983+0000 7ff56a59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff56c06e570 0x7ff56c06e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:53602/0 (socket says 172.21.15.16:53602) 2023-09-30T22:33:29.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:29.983+0000 7ff56a59c700 1 -- 172.21.15.16:0/849417297 learned_addr learned my addr 172.21.15.16:0/849417297 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:29.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:29.984+0000 7ff56a59c700 1 -- 172.21.15.16:0/849417297 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff56c06ee80 msgr2=0x7ff56c134d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:29.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:29.984+0000 7ff56a59c700 1 --2- 172.21.15.16:0/849417297 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff56c06ee80 0x7ff56c134d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:29.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:29.984+0000 7ff56a59c700 1 -- 172.21.15.16:0/849417297 >> v1:172.21.15.16:6789/0 conn(0x7ff56c09eca0 legacy=0x7ff56c134610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:29.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:29.984+0000 7ff56a59c700 1 -- 172.21.15.16:0/849417297 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff56c0711a0 con 0x7ff56c06e570 2023-09-30T22:33:29.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:29.984+0000 7ff56a59c700 1 --2- 172.21.15.16:0/849417297 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff56c06e570 0x7ff56c06e940 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff554009920 tx=0x7ff554009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e68b934210b715b2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:29.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:29.984+0000 7ff56959a700 1 -- 172.21.15.16:0/849417297 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff554004230 con 0x7ff56c06e570 2023-09-30T22:33:29.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:29.984+0000 7ff56959a700 1 -- 172.21.15.16:0/849417297 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff554004390 con 0x7ff56c06e570 2023-09-30T22:33:29.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:29.984+0000 7ff56959a700 1 -- 172.21.15.16:0/849417297 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff554009e70 con 0x7ff56c06e570 2023-09-30T22:33:29.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:29.985+0000 7ff570d16700 1 -- 172.21.15.16:0/849417297 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff56c06e570 msgr2=0x7ff56c06e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:29.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:29.985+0000 7ff570d16700 1 --2- 172.21.15.16:0/849417297 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff56c06e570 0x7ff56c06e940 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff554009920 tx=0x7ff554009c30 comp rx=0 tx=0).stop 2023-09-30T22:33:29.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:29.985+0000 7ff570d16700 1 -- 172.21.15.16:0/849417297 shutdown_connections 2023-09-30T22:33:29.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:29.985+0000 7ff570d16700 1 --2- 172.21.15.16:0/849417297 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff56c06ee80 0x7ff56c134d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:29.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:29.985+0000 7ff570d16700 1 --2- 172.21.15.16:0/849417297 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff56c06e570 0x7ff56c06e940 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:29.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:29.985+0000 7ff570d16700 1 -- 172.21.15.16:0/849417297 >> 172.21.15.16:0/849417297 conn(0x7ff56c0a2aa0 msgr2=0x7ff56c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:29.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:29.985+0000 7ff570d16700 1 -- 172.21.15.16:0/849417297 shutdown_connections 2023-09-30T22:33:29.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:29.986+0000 7ff570d16700 1 -- 172.21.15.16:0/849417297 wait complete. 2023-09-30T22:33:29.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:29.987+0000 7ff570d16700 1 Processor -- start 2023-09-30T22:33:29.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:29.987+0000 7ff570d16700 1 -- start start 2023-09-30T22:33:29.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:29.987+0000 7ff570d16700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff56c06e570 0x7ff56c1233c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:29.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:29.987+0000 7ff570d16700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff56c06ee80 0x7ff56c123900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:29.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:29.987+0000 7ff570d16700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff56c123e40 0x7ff56c1cca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:29.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:29.987+0000 7ff570d16700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff56c124890 con 0x7ff56c06ee80 2023-09-30T22:33:29.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:29.987+0000 7ff570d16700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff56c126ce0 con 0x7ff56c06e570 2023-09-30T22:33:29.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:29.987+0000 7ff570d16700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff56c126ee0 con 0x7ff56c123e40 2023-09-30T22:33:29.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:29.988+0000 7ff56a59c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff56c06e570 0x7ff56c1233c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:29.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:29.988+0000 7ff56a59c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff56c06e570 0x7ff56c1233c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54652/0 (socket says 172.21.15.16:54652) 2023-09-30T22:33:29.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:29.988+0000 7ff56a59c700 1 -- 172.21.15.16:0/1335979460 learned_addr learned my addr 172.21.15.16:0/1335979460 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:29.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:29.988+0000 7ff56ad9d700 1 --2- 172.21.15.16:0/1335979460 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff56c123e40 0x7ff56c1cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:29.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:29.988+0000 7ff569d9b700 1 --2- 172.21.15.16:0/1335979460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff56c06ee80 0x7ff56c123900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:29.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:29.988+0000 7ff56a59c700 1 -- 172.21.15.16:0/1335979460 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff56c123e40 msgr2=0x7ff56c1cca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:29.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:29.988+0000 7ff56a59c700 1 --2- 172.21.15.16:0/1335979460 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff56c123e40 0x7ff56c1cca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:29.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:29.988+0000 7ff56a59c700 1 -- 172.21.15.16:0/1335979460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff56c06ee80 msgr2=0x7ff56c123900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:29.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:29.988+0000 7ff56a59c700 1 --2- 172.21.15.16:0/1335979460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff56c06ee80 0x7ff56c123900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:29.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:29.988+0000 7ff56a59c700 1 -- 172.21.15.16:0/1335979460 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff554009550 con 0x7ff56c06e570 2023-09-30T22:33:29.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:29.988+0000 7ff569d9b700 1 --2- 172.21.15.16:0/1335979460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff56c06ee80 0x7ff56c123900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:33:29.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:29.988+0000 7ff56a59c700 1 --2- 172.21.15.16:0/1335979460 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff56c06e570 0x7ff56c1233c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff5540098f0 tx=0x7ff554004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:29.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:29.988+0000 7ff562ffd700 1 -- 172.21.15.16:0/1335979460 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55400be60 con 0x7ff56c06e570 2023-09-30T22:33:29.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:29.988+0000 7ff562ffd700 1 -- 172.21.15.16:0/1335979460 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5540036a0 con 0x7ff56c06e570 2023-09-30T22:33:29.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:29.988+0000 7ff570d16700 1 -- 172.21.15.16:0/1335979460 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff56c133ec0 con 0x7ff56c06e570 2023-09-30T22:33:29.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:29.989+0000 7ff562ffd700 1 -- 172.21.15.16:0/1335979460 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff554003a50 con 0x7ff56c06e570 2023-09-30T22:33:29.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:29.989+0000 7ff570d16700 1 -- 172.21.15.16:0/1335979460 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff56c1250c0 con 0x7ff56c06e570 2023-09-30T22:33:29.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:29.990+0000 7ff562ffd700 1 -- 172.21.15.16:0/1335979460 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff554022070 con 0x7ff56c06e570 2023-09-30T22:33:29.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:29.990+0000 7ff562ffd700 1 -- 172.21.15.16:0/1335979460 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7ff554041350 con 0x7ff56c06e570 2023-09-30T22:33:29.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:29.990+0000 7ff570d16700 1 -- 172.21.15.16:0/1335979460 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff56c070610 con 0x7ff56c06e570 2023-09-30T22:33:29.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:29.995+0000 7ff562ffd700 1 -- 172.21.15.16:0/1335979460 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff55401b970 con 0x7ff56c06e570 2023-09-30T22:33:30.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.018+0000 7febde2eb700 1 Processor -- start 2023-09-30T22:33:30.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.018+0000 7febde2eb700 1 -- start start 2023-09-30T22:33:30.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.019+0000 7febde2eb700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7febd809e1d0 0x7febd8138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:30.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.019+0000 7febde2eb700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7febd8139170 0x7febd813b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:30.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.019+0000 7febde2eb700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febd809b120 con 0x7febd809d6f0 2023-09-30T22:33:30.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.019+0000 7febde2eb700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febd806b460 con 0x7febd8139170 2023-09-30T22:33:30.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.019+0000 7febde2eb700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7febd809a7e0 con 0x7febd809e1d0 2023-09-30T22:33:30.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.019+0000 7febd77fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7febd809e1d0 0x7febd8138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:30.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.019+0000 7febdc888700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7febd8139170 0x7febd813b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:30.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.019+0000 7febd77fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7febd809e1d0 0x7febd8138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:53622/0 (socket says 172.21.15.16:53622) 2023-09-30T22:33:30.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.019+0000 7febd77fe700 1 -- 172.21.15.16:0/2791106676 learned_addr learned my addr 172.21.15.16:0/2791106676 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:30.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.020+0000 7febd77fe700 1 -- 172.21.15.16:0/2791106676 >> v2:172.21.15.16:3300/0 conn(0x7febd8139170 msgr2=0x7febd813b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:30.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.020+0000 7febd77fe700 1 --2- 172.21.15.16:0/2791106676 >> v2:172.21.15.16:3300/0 conn(0x7febd8139170 0x7febd813b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:30.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.020+0000 7febd77fe700 1 -- 172.21.15.16:0/2791106676 >> v1:172.21.15.16:6789/0 conn(0x7febd809d6f0 legacy=0x7febd809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:30.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.020+0000 7febd77fe700 1 -- 172.21.15.16:0/2791106676 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd809a500 con 0x7febd809e1d0 2023-09-30T22:33:30.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.020+0000 7febd77fe700 1 --2- 172.21.15.16:0/2791106676 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7febd809e1d0 0x7febd8138c30 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7febcc009920 tx=0x7febcc009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9941b1b3eaeee2c7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:30.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.020+0000 7febd6ffd700 1 -- 172.21.15.16:0/2791106676 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc011830 con 0x7febd809e1d0 2023-09-30T22:33:30.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.020+0000 7febd6ffd700 1 -- 172.21.15.16:0/2791106676 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febcc011990 con 0x7febd809e1d0 2023-09-30T22:33:30.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.020+0000 7febd6ffd700 1 -- 172.21.15.16:0/2791106676 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc011d00 con 0x7febd809e1d0 2023-09-30T22:33:30.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.021+0000 7febde2eb700 1 -- 172.21.15.16:0/2791106676 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7febd809e1d0 msgr2=0x7febd8138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:30.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.021+0000 7febde2eb700 1 --2- 172.21.15.16:0/2791106676 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7febd809e1d0 0x7febd8138c30 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7febcc009920 tx=0x7febcc009c30 comp rx=0 tx=0).stop 2023-09-30T22:33:30.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.021+0000 7febde2eb700 1 -- 172.21.15.16:0/2791106676 shutdown_connections 2023-09-30T22:33:30.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.021+0000 7febde2eb700 1 --2- 172.21.15.16:0/2791106676 >> v2:172.21.15.16:3300/0 conn(0x7febd8139170 0x7febd813b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:30.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.021+0000 7febde2eb700 1 --2- 172.21.15.16:0/2791106676 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7febd809e1d0 0x7febd8138c30 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:30.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.021+0000 7febde2eb700 1 -- 172.21.15.16:0/2791106676 >> 172.21.15.16:0/2791106676 conn(0x7febd8066f50 msgr2=0x7febd809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:30.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.021+0000 7febde2eb700 1 -- 172.21.15.16:0/2791106676 shutdown_connections 2023-09-30T22:33:30.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.021+0000 7febde2eb700 1 -- 172.21.15.16:0/2791106676 wait complete. 2023-09-30T22:33:30.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.022+0000 7febde2eb700 1 Processor -- start 2023-09-30T22:33:30.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.023+0000 7febde2eb700 1 -- start start 2023-09-30T22:33:30.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.023+0000 7febde2eb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febd809d6f0 0x7febd811e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:30.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.023+0000 7febde2eb700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7febd809e1d0 0x7febd811edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:30.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.023+0000 7febde2eb700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7febd8139170 0x7febd811f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:30.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.023+0000 7febde2eb700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd8121e80 con 0x7febd809d6f0 2023-09-30T22:33:30.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.023+0000 7febde2eb700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febd8120540 con 0x7febd8139170 2023-09-30T22:33:30.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.023+0000 7febde2eb700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7febd80a20b0 con 0x7febd809e1d0 2023-09-30T22:33:30.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.023+0000 7febdc888700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7febd8139170 0x7febd811f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:30.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.023+0000 7febd77fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7febd809e1d0 0x7febd811edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:30.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.023+0000 7febdc888700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7febd8139170 0x7febd811f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54656/0 (socket says 172.21.15.16:54656) 2023-09-30T22:33:30.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.024+0000 7febdc888700 1 -- 172.21.15.16:0/4006363503 learned_addr learned my addr 172.21.15.16:0/4006363503 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:30.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.024+0000 7febd77fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7febd809e1d0 0x7febd811edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:53626/0 (socket says 172.21.15.16:53626) 2023-09-30T22:33:30.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.024+0000 7febd7fff700 1 --2- 172.21.15.16:0/4006363503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febd809d6f0 0x7febd811e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:30.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.024+0000 7febdc888700 1 -- 172.21.15.16:0/4006363503 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7febd809e1d0 msgr2=0x7febd811edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:30.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.024+0000 7febdc888700 1 --2- 172.21.15.16:0/4006363503 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7febd809e1d0 0x7febd811edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:30.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.024+0000 7febdc888700 1 -- 172.21.15.16:0/4006363503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febd809d6f0 msgr2=0x7febd811e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:30.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.024+0000 7febdc888700 1 --2- 172.21.15.16:0/4006363503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febd809d6f0 0x7febd811e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:30.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.024+0000 7febdc888700 1 -- 172.21.15.16:0/4006363503 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febcc009550 con 0x7febd8139170 2023-09-30T22:33:30.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.024+0000 7febd77fe700 1 --2- 172.21.15.16:0/4006363503 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7febd809e1d0 0x7febd811edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:30.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.024+0000 7febdc888700 1 --2- 172.21.15.16:0/4006363503 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7febd8139170 0x7febd811f310 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7febd000efc0 tx=0x7febd000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:30.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.024+0000 7febd4ff9700 1 -- 172.21.15.16:0/4006363503 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd0009ea0 con 0x7febd8139170 2023-09-30T22:33:30.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.024+0000 7febd4ff9700 1 -- 172.21.15.16:0/4006363503 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febd000f040 con 0x7febd8139170 2023-09-30T22:33:30.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.024+0000 7febde2eb700 1 -- 172.21.15.16:0/4006363503 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febd8120c90 con 0x7febd8139170 2023-09-30T22:33:30.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.025+0000 7febde2eb700 1 -- 172.21.15.16:0/4006363503 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febd81228b0 con 0x7febd8139170 2023-09-30T22:33:30.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.025+0000 7febd4ff9700 1 -- 172.21.15.16:0/4006363503 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd000ced0 con 0x7febd8139170 2023-09-30T22:33:30.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.025+0000 7febd4ff9700 1 -- 172.21.15.16:0/4006363503 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7febd0014480 con 0x7febd8139170 2023-09-30T22:33:30.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.026+0000 7febd4ff9700 1 -- 172.21.15.16:0/4006363503 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7febd003fec0 con 0x7febd8139170 2023-09-30T22:33:30.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.026+0000 7febde2eb700 1 -- 172.21.15.16:0/4006363503 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febb80052e0 con 0x7febd8139170 2023-09-30T22:33:30.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.031+0000 7febd4ff9700 1 -- 172.21.15.16:0/4006363503 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7febd0018070 con 0x7febd8139170 2023-09-30T22:33:30.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.060+0000 7f5089885700 1 Processor -- start 2023-09-30T22:33:30.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.060+0000 7f5089885700 1 -- start start 2023-09-30T22:33:30.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.061+0000 7f5089885700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f508406e570 0x7f508406e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:30.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.061+0000 7f5089885700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f508409eca0 0x7f508406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:30.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.061+0000 7f5089885700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5084134610 0x7f5084134a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:30.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.061+0000 7f5089885700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5084070ee0 con 0x7f508406e570 2023-09-30T22:33:30.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.061+0000 7f5089885700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5084071480 con 0x7f5084134610 2023-09-30T22:33:30.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.061+0000 7f5089885700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f508406f3c0 con 0x7f508409eca0 2023-09-30T22:33:30.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.061+0000 7f5082ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f508406e570 0x7f508406e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:30.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.061+0000 7f5082ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f508406e570 0x7f508406e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55258/0 (socket says 172.21.15.16:55258) 2023-09-30T22:33:30.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.061+0000 7f5082ffd700 1 -- 172.21.15.16:0/177054841 learned_addr learned my addr 172.21.15.16:0/177054841 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:30.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.061+0000 7f50837fe700 1 --2- 172.21.15.16:0/177054841 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5084134610 0x7f5084134a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:30.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.061+0000 7f50827fc700 1 --2- 172.21.15.16:0/177054841 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f508409eca0 0x7f508406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:30.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.061+0000 7f5082ffd700 1 -- 172.21.15.16:0/177054841 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f508409eca0 msgr2=0x7f508406ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:30.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.061+0000 7f5082ffd700 1 --2- 172.21.15.16:0/177054841 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f508409eca0 0x7f508406ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:30.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.061+0000 7f5082ffd700 1 -- 172.21.15.16:0/177054841 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5084134610 msgr2=0x7f5084134a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:30.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.061+0000 7f5082ffd700 1 --2- 172.21.15.16:0/177054841 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5084134610 0x7f5084134a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:30.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.061+0000 7f5082ffd700 1 -- 172.21.15.16:0/177054841 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5084071b90 con 0x7f508406e570 2023-09-30T22:33:30.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.061+0000 7f50827fc700 1 --2- 172.21.15.16:0/177054841 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f508409eca0 0x7f508406ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:33:30.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.061+0000 7f5082ffd700 1 --2- 172.21.15.16:0/177054841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f508406e570 0x7f508406e940 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f50740098f0 tx=0x7f5074009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2638065bb7735373 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:30.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.062+0000 7f5081ffb700 1 -- 172.21.15.16:0/177054841 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5074011830 con 0x7f508406e570 2023-09-30T22:33:30.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.062+0000 7f5081ffb700 1 -- 172.21.15.16:0/177054841 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5074011990 con 0x7f508406e570 2023-09-30T22:33:30.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.062+0000 7f5081ffb700 1 -- 172.21.15.16:0/177054841 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5074011d20 con 0x7f508406e570 2023-09-30T22:33:30.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.063+0000 7f5089885700 1 -- 172.21.15.16:0/177054841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f508406e570 msgr2=0x7f508406e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:30.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.063+0000 7f5089885700 1 --2- 172.21.15.16:0/177054841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f508406e570 0x7f508406e940 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f50740098f0 tx=0x7f5074009c00 comp rx=0 tx=0).stop 2023-09-30T22:33:30.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.063+0000 7f5089885700 1 -- 172.21.15.16:0/177054841 shutdown_connections 2023-09-30T22:33:30.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.063+0000 7f5089885700 1 --2- 172.21.15.16:0/177054841 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5084134610 0x7f5084134a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:30.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.063+0000 7f5089885700 1 --2- 172.21.15.16:0/177054841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f508406e570 0x7f508406e940 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:30.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.063+0000 7f5089885700 1 --2- 172.21.15.16:0/177054841 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f508409eca0 0x7f508406ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:30.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.063+0000 7f5089885700 1 -- 172.21.15.16:0/177054841 >> 172.21.15.16:0/177054841 conn(0x7f50840a2aa0 msgr2=0x7f50840a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:30.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.063+0000 7f5089885700 1 -- 172.21.15.16:0/177054841 shutdown_connections 2023-09-30T22:33:30.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.063+0000 7f5089885700 1 -- 172.21.15.16:0/177054841 wait complete. 2023-09-30T22:33:30.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.064+0000 7f5089885700 1 Processor -- start 2023-09-30T22:33:30.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.064+0000 7f5089885700 1 -- start start 2023-09-30T22:33:30.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.064+0000 7f5089885700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f508406e570 0x7f50841214e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:30.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.064+0000 7f5089885700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5084134610 0x7f5084121a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:30.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.064+0000 7f5089885700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5084121f60 0x7f50841d2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:30.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.064+0000 7f5089885700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50841230c0 con 0x7f5084121f60 2023-09-30T22:33:30.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.064+0000 7f5089885700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5084122a90 con 0x7f5084134610 2023-09-30T22:33:30.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.064+0000 7f5089885700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5084070610 con 0x7f508406e570 2023-09-30T22:33:30.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.065+0000 7f50827fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5084134610 0x7f5084121a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:30.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.065+0000 7f50827fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5084134610 0x7f5084121a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54676/0 (socket says 172.21.15.16:54676) 2023-09-30T22:33:30.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.065+0000 7f5082ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f508406e570 0x7f50841214e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:30.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.065+0000 7f50827fc700 1 -- 172.21.15.16:0/3058854356 learned_addr learned my addr 172.21.15.16:0/3058854356 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:30.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.065+0000 7f50837fe700 1 --2- 172.21.15.16:0/3058854356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5084121f60 0x7f50841d2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:30.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.065+0000 7f5082ffd700 1 -- 172.21.15.16:0/3058854356 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5084134610 msgr2=0x7f5084121a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:30.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.065+0000 7f5082ffd700 1 --2- 172.21.15.16:0/3058854356 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5084134610 0x7f5084121a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:30.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.065+0000 7f5082ffd700 1 -- 172.21.15.16:0/3058854356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5084121f60 msgr2=0x7f50841d2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:30.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.065+0000 7f5082ffd700 1 --2- 172.21.15.16:0/3058854356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5084121f60 0x7f50841d2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:30.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.065+0000 7f5082ffd700 1 -- 172.21.15.16:0/3058854356 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5074009550 con 0x7f508406e570 2023-09-30T22:33:30.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.065+0000 7f50827fc700 1 --2- 172.21.15.16:0/3058854356 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5084134610 0x7f5084121a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:33:30.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.065+0000 7f50837fe700 1 --2- 172.21.15.16:0/3058854356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5084121f60 0x7f50841d2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:30.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.065+0000 7f5082ffd700 1 --2- 172.21.15.16:0/3058854356 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f508406e570 0x7f50841214e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5084070170 tx=0x7f5074004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:30.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.065+0000 7f506bfff700 1 -- 172.21.15.16:0/3058854356 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5074004300 con 0x7f508406e570 2023-09-30T22:33:30.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.066+0000 7f506bfff700 1 -- 172.21.15.16:0/3058854356 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f507401f070 con 0x7f508406e570 2023-09-30T22:33:30.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.066+0000 7f5089885700 1 -- 172.21.15.16:0/3058854356 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5084123ce0 con 0x7f508406e570 2023-09-30T22:33:30.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.066+0000 7f506bfff700 1 -- 172.21.15.16:0/3058854356 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50740036a0 con 0x7f508406e570 2023-09-30T22:33:30.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.066+0000 7f5089885700 1 -- 172.21.15.16:0/3058854356 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5084125310 con 0x7f508406e570 2023-09-30T22:33:30.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.066+0000 7f506bfff700 1 -- 172.21.15.16:0/3058854356 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5074021030 con 0x7f508406e570 2023-09-30T22:33:30.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.067+0000 7f506bfff700 1 -- 172.21.15.16:0/3058854356 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f50740407d0 con 0x7f508406e570 2023-09-30T22:33:30.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.067+0000 7f5089885700 1 -- 172.21.15.16:0/3058854356 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50840a22a0 con 0x7f508406e570 2023-09-30T22:33:30.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.072+0000 7f506bfff700 1 -- 172.21.15.16:0/3058854356 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f50740199a0 con 0x7f508406e570 2023-09-30T22:33:30.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.144+0000 7ff9f5409700 1 Processor -- start 2023-09-30T22:33:30.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.144+0000 7ff9f5409700 1 -- start start 2023-09-30T22:33:30.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.145+0000 7ff9f5409700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff9f009e5d0 0x7ff9f01306a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:30.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.145+0000 7ff9f5409700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff9f0136320 0x7ff9f0138700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:30.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.145+0000 7ff9f5409700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9f009e9e0 con 0x7ff9f009ba40 2023-09-30T22:33:30.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.145+0000 7ff9f5409700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f012d540 con 0x7ff9f0136320 2023-09-30T22:33:30.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.145+0000 7ff9f5409700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f009afa0 con 0x7ff9f009e5d0 2023-09-30T22:33:30.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.145+0000 7ff9eeffd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff9f009ba40 0x7ff9f012ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35012/0 (socket says 172.21.15.16:35012) 2023-09-30T22:33:30.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.145+0000 7ff9eeffd700 1 -- 172.21.15.16:0/1580506906 learned_addr learned my addr 172.21.15.16:0/1580506906 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:30.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.145+0000 7ff9ee7fc700 1 --2- 172.21.15.16:0/1580506906 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff9f009e5d0 0x7ff9f01306a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:30.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.145+0000 7ff9ef7fe700 1 --2- 172.21.15.16:0/1580506906 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff9f0136320 0x7ff9f0138700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:30.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.146+0000 7ff9edffb700 1 -- 172.21.15.16:0/1580506906 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1732613937 0 0) 0x7ff9f009e9e0 con 0x7ff9f009ba40 2023-09-30T22:33:30.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.146+0000 7ff9edffb700 1 -- 172.21.15.16:0/1580506906 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9d80035b0 con 0x7ff9f009ba40 2023-09-30T22:33:30.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.146+0000 7ff9ee7fc700 1 -- 172.21.15.16:0/1580506906 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff9f0136320 msgr2=0x7ff9f0138700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:30.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.146+0000 7ff9ee7fc700 1 --2- 172.21.15.16:0/1580506906 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff9f0136320 0x7ff9f0138700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:30.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.146+0000 7ff9ee7fc700 1 -- 172.21.15.16:0/1580506906 >> v1:172.21.15.16:6789/0 conn(0x7ff9f009ba40 legacy=0x7ff9f012ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:30.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.146+0000 7ff9ee7fc700 1 -- 172.21.15.16:0/1580506906 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f009b260 con 0x7ff9f009e5d0 2023-09-30T22:33:30.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.146+0000 7ff9ee7fc700 1 --2- 172.21.15.16:0/1580506906 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff9f009e5d0 0x7ff9f01306a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff9e4009870 tx=0x7ff9e4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb1831fcd696ad6b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:30.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.146+0000 7ff9edffb700 1 -- 172.21.15.16:0/1580506906 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e400d980 con 0x7ff9f009e5d0 2023-09-30T22:33:30.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.146+0000 7ff9edffb700 1 -- 172.21.15.16:0/1580506906 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9e400dae0 con 0x7ff9f009e5d0 2023-09-30T22:33:30.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.146+0000 7ff9edffb700 1 -- 172.21.15.16:0/1580506906 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e400de70 con 0x7ff9f009e5d0 2023-09-30T22:33:30.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.147+0000 7ff9f5409700 1 -- 172.21.15.16:0/1580506906 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff9f009e5d0 msgr2=0x7ff9f01306a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:30.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.147+0000 7ff9f5409700 1 --2- 172.21.15.16:0/1580506906 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff9f009e5d0 0x7ff9f01306a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff9e4009870 tx=0x7ff9e4009b80 comp rx=0 tx=0).stop 2023-09-30T22:33:30.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.147+0000 7ff9f5409700 1 -- 172.21.15.16:0/1580506906 shutdown_connections 2023-09-30T22:33:30.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.147+0000 7ff9f5409700 1 --2- 172.21.15.16:0/1580506906 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff9f0136320 0x7ff9f0138700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:30.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.147+0000 7ff9f5409700 1 --2- 172.21.15.16:0/1580506906 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff9f009e5d0 0x7ff9f01306a0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:30.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.147+0000 7ff9f5409700 1 -- 172.21.15.16:0/1580506906 >> 172.21.15.16:0/1580506906 conn(0x7ff9f00a2cc0 msgr2=0x7ff9f00a30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:30.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.147+0000 7ff9f5409700 1 -- 172.21.15.16:0/1580506906 shutdown_connections 2023-09-30T22:33:30.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.147+0000 7ff9f5409700 1 -- 172.21.15.16:0/1580506906 wait complete. 2023-09-30T22:33:30.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.149+0000 7ff9f5409700 1 Processor -- start 2023-09-30T22:33:30.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.149+0000 7ff9f5409700 1 -- start start 2023-09-30T22:33:30.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.150+0000 7ff9f5409700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff9f009ba40 0x7ff9f0071aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:30.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.150+0000 7ff9f5409700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff9f009e5d0 0x7ff9f0071fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:30.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.150+0000 7ff9f5409700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9f0136320 0x7ff9f0132c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:30.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.150+0000 7ff9f5409700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f006e850 con 0x7ff9f0136320 2023-09-30T22:33:30.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.150+0000 7ff9f5409700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f006fc80 con 0x7ff9f009ba40 2023-09-30T22:33:30.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.150+0000 7ff9f5409700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f006f050 con 0x7ff9f009e5d0 2023-09-30T22:33:30.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.150+0000 7ff9eeffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff9f009ba40 0x7ff9f0071aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:30.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.150+0000 7ff9ee7fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff9f009e5d0 0x7ff9f0071fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:30.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.150+0000 7ff9eeffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff9f009ba40 0x7ff9f0071aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54686/0 (socket says 172.21.15.16:54686) 2023-09-30T22:33:30.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.150+0000 7ff9eeffd700 1 -- 172.21.15.16:0/2209688662 learned_addr learned my addr 172.21.15.16:0/2209688662 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:30.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.150+0000 7ff9ef7fe700 1 --2- 172.21.15.16:0/2209688662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9f0136320 0x7ff9f0132c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:30.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.150+0000 7ff9ef7fe700 1 -- 172.21.15.16:0/2209688662 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff9f009e5d0 msgr2=0x7ff9f0071fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:30.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.151+0000 7ff9ef7fe700 1 --2- 172.21.15.16:0/2209688662 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff9f009e5d0 0x7ff9f0071fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:30.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.151+0000 7ff9ef7fe700 1 -- 172.21.15.16:0/2209688662 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff9f009ba40 msgr2=0x7ff9f0071aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:30.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.151+0000 7ff9ef7fe700 1 --2- 172.21.15.16:0/2209688662 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff9f009ba40 0x7ff9f0071aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:30.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.151+0000 7ff9ef7fe700 1 -- 172.21.15.16:0/2209688662 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e40094a0 con 0x7ff9f0136320 2023-09-30T22:33:30.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.151+0000 7ff9ef7fe700 1 --2- 172.21.15.16:0/2209688662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9f0136320 0x7ff9f0132c10 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8009d90 tx=0x7ff9e800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:30.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.151+0000 7ff9d77fe700 1 -- 172.21.15.16:0/2209688662 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e8009c00 con 0x7ff9f0136320 2023-09-30T22:33:30.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.151+0000 7ff9d77fe700 1 -- 172.21.15.16:0/2209688662 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9e800f040 con 0x7ff9f0136320 2023-09-30T22:33:30.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.151+0000 7ff9f5409700 1 -- 172.21.15.16:0/2209688662 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9f012a5e0 con 0x7ff9f0136320 2023-09-30T22:33:30.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.151+0000 7ff9d77fe700 1 -- 172.21.15.16:0/2209688662 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e8014480 con 0x7ff9f0136320 2023-09-30T22:33:30.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.151+0000 7ff9f5409700 1 -- 172.21.15.16:0/2209688662 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9f006f410 con 0x7ff9f0136320 2023-09-30T22:33:30.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.153+0000 7ff9d77fe700 1 -- 172.21.15.16:0/2209688662 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff9e80145e0 con 0x7ff9f0136320 2023-09-30T22:33:30.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.153+0000 7ff9f5409700 1 -- 172.21.15.16:0/2209688662 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9f006f600 con 0x7ff9f0136320 2023-09-30T22:33:30.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.153+0000 7ff9d77fe700 1 -- 172.21.15.16:0/2209688662 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7ff9e80401f0 con 0x7ff9f0136320 2023-09-30T22:33:30.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.158+0000 7ff9d77fe700 1 -- 172.21.15.16:0/2209688662 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff9e8007900 con 0x7ff9f0136320 2023-09-30T22:33:30.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.243+0000 7febde2eb700 1 -- 172.21.15.16:0/4006363503 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7febb8005590 con 0x7febd8139170 2023-09-30T22:33:30.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.243+0000 7febd4ff9700 1 -- 172.21.15.16:0/4006363503 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7febd002b090 con 0x7febd8139170 2023-09-30T22:33:30.245 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:33:30.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.245+0000 7febde2eb700 1 -- 172.21.15.16:0/4006363503 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7febd8139170 msgr2=0x7febd811f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:30.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.245+0000 7febde2eb700 1 --2- 172.21.15.16:0/4006363503 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7febd8139170 0x7febd811f310 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7febd000efc0 tx=0x7febd000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:33:30.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.245+0000 7febde2eb700 1 -- 172.21.15.16:0/4006363503 shutdown_connections 2023-09-30T22:33:30.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.245+0000 7febde2eb700 1 --2- 172.21.15.16:0/4006363503 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7febd8139170 0x7febd811f310 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:30.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.245+0000 7febde2eb700 1 --2- 172.21.15.16:0/4006363503 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febd809d6f0 0x7febd811e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:30.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.245+0000 7febde2eb700 1 --2- 172.21.15.16:0/4006363503 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7febd809e1d0 0x7febd811edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:30.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.245+0000 7febde2eb700 1 -- 172.21.15.16:0/4006363503 >> 172.21.15.16:0/4006363503 conn(0x7febd8066f50 msgr2=0x7febd8136c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:30.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.245+0000 7febde2eb700 1 -- 172.21.15.16:0/4006363503 shutdown_connections 2023-09-30T22:33:30.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.245+0000 7febde2eb700 1 -- 172.21.15.16:0/4006363503 wait complete. 2023-09-30T22:33:30.261 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:33:30.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.265+0000 7ff570d16700 1 -- 172.21.15.16:0/1335979460 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff56c074ea0 con 0x7ff56c06e570 2023-09-30T22:33:30.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.265+0000 7ff562ffd700 1 -- 172.21.15.16:0/1335979460 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff554019ba0 con 0x7ff56c06e570 2023-09-30T22:33:30.267 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:33:30.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.269+0000 7ff570d16700 1 -- 172.21.15.16:0/1335979460 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff56c06e570 msgr2=0x7ff56c1233c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:30.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.269+0000 7ff570d16700 1 --2- 172.21.15.16:0/1335979460 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff56c06e570 0x7ff56c1233c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff5540098f0 tx=0x7ff554004750 comp rx=0 tx=0).stop 2023-09-30T22:33:30.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.269+0000 7ff570d16700 1 -- 172.21.15.16:0/1335979460 shutdown_connections 2023-09-30T22:33:30.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.269+0000 7ff570d16700 1 --2- 172.21.15.16:0/1335979460 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff56c06e570 0x7ff56c1233c0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:30.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.269+0000 7ff570d16700 1 --2- 172.21.15.16:0/1335979460 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff56c123e40 0x7ff56c1cca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:30.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.269+0000 7ff570d16700 1 --2- 172.21.15.16:0/1335979460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff56c06ee80 0x7ff56c123900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:30.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.269+0000 7ff570d16700 1 -- 172.21.15.16:0/1335979460 >> 172.21.15.16:0/1335979460 conn(0x7ff56c0a2aa0 msgr2=0x7ff56c0a1410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:30.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.269+0000 7ff570d16700 1 -- 172.21.15.16:0/1335979460 shutdown_connections 2023-09-30T22:33:30.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.269+0000 7ff570d16700 1 -- 172.21.15.16:0/1335979460 wait complete. 2023-09-30T22:33:30.290 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:33:30.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.292+0000 7f5089885700 1 -- 172.21.15.16:0/3058854356 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f50840a1ac0 con 0x7f508406e570 2023-09-30T22:33:30.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.293+0000 7f506bfff700 1 -- 172.21.15.16:0/3058854356 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5074015020 con 0x7f508406e570 2023-09-30T22:33:30.294 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:33:30.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.295+0000 7f5089885700 1 -- 172.21.15.16:0/3058854356 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f508406e570 msgr2=0x7f50841214e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:30.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.295+0000 7f5089885700 1 --2- 172.21.15.16:0/3058854356 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f508406e570 0x7f50841214e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f5084070170 tx=0x7f5074004750 comp rx=0 tx=0).stop 2023-09-30T22:33:30.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.295+0000 7f5089885700 1 -- 172.21.15.16:0/3058854356 shutdown_connections 2023-09-30T22:33:30.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.295+0000 7f5089885700 1 --2- 172.21.15.16:0/3058854356 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5084121f60 0x7f50841d2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:30.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.295+0000 7f5089885700 1 --2- 172.21.15.16:0/3058854356 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f508406e570 0x7f50841214e0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:30.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.295+0000 7f5089885700 1 --2- 172.21.15.16:0/3058854356 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5084134610 0x7f5084121a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:30.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.295+0000 7f5089885700 1 -- 172.21.15.16:0/3058854356 >> 172.21.15.16:0/3058854356 conn(0x7f50840a2aa0 msgr2=0x7f50840a0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:30.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.295+0000 7f5089885700 1 -- 172.21.15.16:0/3058854356 shutdown_connections 2023-09-30T22:33:30.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.295+0000 7f5089885700 1 -- 172.21.15.16:0/3058854356 wait complete. 2023-09-30T22:33:30.310 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:33:30.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.366+0000 7ff9f5409700 1 -- 172.21.15.16:0/2209688662 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff9f004f2a0 con 0x7ff9f0136320 2023-09-30T22:33:30.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.366+0000 7ff9d77fe700 1 -- 172.21.15.16:0/2209688662 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff9e8007330 con 0x7ff9f0136320 2023-09-30T22:33:30.368 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:33:30.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.368+0000 7ff9f5409700 1 -- 172.21.15.16:0/2209688662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9f0136320 msgr2=0x7ff9f0132c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:30.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.368+0000 7ff9f5409700 1 --2- 172.21.15.16:0/2209688662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9f0136320 0x7ff9f0132c10 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8009d90 tx=0x7ff9e800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:33:30.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.369+0000 7ff9f5409700 1 -- 172.21.15.16:0/2209688662 shutdown_connections 2023-09-30T22:33:30.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.369+0000 7ff9f5409700 1 --2- 172.21.15.16:0/2209688662 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff9f009ba40 0x7ff9f0071aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:30.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.369+0000 7ff9f5409700 1 --2- 172.21.15.16:0/2209688662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9f0136320 0x7ff9f0132c10 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:30.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.369+0000 7ff9f5409700 1 --2- 172.21.15.16:0/2209688662 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff9f009e5d0 0x7ff9f0071fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:30.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.369+0000 7ff9f5409700 1 -- 172.21.15.16:0/2209688662 >> 172.21.15.16:0/2209688662 conn(0x7ff9f00a2cc0 msgr2=0x7ff9f012f650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:30.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.369+0000 7ff9f5409700 1 -- 172.21.15.16:0/2209688662 shutdown_connections 2023-09-30T22:33:30.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:30.369+0000 7ff9f5409700 1 -- 172.21.15.16:0/2209688662 wait complete. 2023-09-30T22:33:30.383 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:33:31.262 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:33:31.291 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:33:31.311 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:33:31.384 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:33:31.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.413+0000 7f02eba15700 1 Processor -- start 2023-09-30T22:33:31.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.413+0000 7f02eba15700 1 -- start start 2023-09-30T22:33:31.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.414+0000 7f02eba15700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f02e40a0a30 0x7f02e40a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:31.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.414+0000 7f02eba15700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f02e409d630 0x7f02e40a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:31.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.414+0000 7f02eba15700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02e40a2a70 con 0x7f02e4133930 2023-09-30T22:33:31.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.414+0000 7f02eba15700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02e40a01d0 con 0x7f02e409d630 2023-09-30T22:33:31.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.414+0000 7f02eba15700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02e409e2b0 con 0x7f02e40a0a30 2023-09-30T22:33:31.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.414+0000 7f02e97b1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f02e40a0a30 0x7f02e40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:31.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.414+0000 7f02e97b1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f02e40a0a30 0x7f02e40a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54690/0 (socket says 172.21.15.16:54690) 2023-09-30T22:33:31.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.414+0000 7f02e9fb2700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f02e4133930 0x7f02e4133db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35028/0 (socket says 172.21.15.16:35028) 2023-09-30T22:33:31.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.414+0000 7f02e97b1700 1 -- 172.21.15.16:0/3307587406 learned_addr learned my addr 172.21.15.16:0/3307587406 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:31.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.414+0000 7f02e8fb0700 1 --2- 172.21.15.16:0/3307587406 >> v2:172.21.15.16:3300/0 conn(0x7f02e409d630 0x7f02e40a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:31.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.415+0000 7f02e97b1700 1 -- 172.21.15.16:0/3307587406 >> v2:172.21.15.16:3300/0 conn(0x7f02e409d630 msgr2=0x7f02e40a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:31.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.415+0000 7f02dbfff700 1 -- 172.21.15.16:0/3307587406 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1493459632 0 0) 0x7f02e40a2a70 con 0x7f02e4133930 2023-09-30T22:33:31.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.415+0000 7f02e97b1700 1 --2- 172.21.15.16:0/3307587406 >> v2:172.21.15.16:3300/0 conn(0x7f02e409d630 0x7f02e40a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:31.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.415+0000 7f02e97b1700 1 -- 172.21.15.16:0/3307587406 >> v1:172.21.15.16:6789/0 conn(0x7f02e4133930 legacy=0x7f02e4133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:31.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.415+0000 7f02e97b1700 1 -- 172.21.15.16:0/3307587406 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02e409dc50 con 0x7f02e40a0a30 2023-09-30T22:33:31.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.415+0000 7f02e8fb0700 1 --2- 172.21.15.16:0/3307587406 >> v2:172.21.15.16:3300/0 conn(0x7f02e409d630 0x7f02e40a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:31.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.415+0000 7f02e97b1700 1 --2- 172.21.15.16:0/3307587406 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f02e40a0a30 0x7f02e40a0e00 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f02e00098f0 tx=0x7f02e0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ecd616f803ba092 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:31.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.415+0000 7f02dbfff700 1 -- 172.21.15.16:0/3307587406 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e0011830 con 0x7f02e40a0a30 2023-09-30T22:33:31.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.415+0000 7f02dbfff700 1 -- 172.21.15.16:0/3307587406 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02e0011990 con 0x7f02e40a0a30 2023-09-30T22:33:31.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.415+0000 7f02dbfff700 1 -- 172.21.15.16:0/3307587406 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e0011d20 con 0x7f02e40a0a30 2023-09-30T22:33:31.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.416+0000 7f02eba15700 1 -- 172.21.15.16:0/3307587406 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f02e40a0a30 msgr2=0x7f02e40a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:31.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.416+0000 7f02eba15700 1 --2- 172.21.15.16:0/3307587406 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f02e40a0a30 0x7f02e40a0e00 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f02e00098f0 tx=0x7f02e0009c00 comp rx=0 tx=0).stop 2023-09-30T22:33:31.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.416+0000 7f02eba15700 1 -- 172.21.15.16:0/3307587406 shutdown_connections 2023-09-30T22:33:31.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.416+0000 7f02eba15700 1 --2- 172.21.15.16:0/3307587406 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f02e40a0a30 0x7f02e40a0e00 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:31.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.416+0000 7f02eba15700 1 --2- 172.21.15.16:0/3307587406 >> v2:172.21.15.16:3300/0 conn(0x7f02e409d630 0x7f02e40a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:31.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.416+0000 7f02eba15700 1 -- 172.21.15.16:0/3307587406 >> 172.21.15.16:0/3307587406 conn(0x7f02e40a3050 msgr2=0x7f02e40a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:31.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.416+0000 7f02eba15700 1 -- 172.21.15.16:0/3307587406 shutdown_connections 2023-09-30T22:33:31.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.417+0000 7f02eba15700 1 -- 172.21.15.16:0/3307587406 wait complete. 2023-09-30T22:33:31.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.418+0000 7f02eba15700 1 Processor -- start 2023-09-30T22:33:31.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.418+0000 7f02eba15700 1 -- start start 2023-09-30T22:33:31.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.418+0000 7f02eba15700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f02e409d630 0x7f02e41c7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:31.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.419+0000 7f02eba15700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f02e40a0a30 0x7f02e41c8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:31.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.419+0000 7f02eba15700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02e4133930 0x7f02e41c89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:31.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.419+0000 7f02eba15700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02e4120750 con 0x7f02e4133930 2023-09-30T22:33:31.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.419+0000 7f02eba15700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02e4120df0 con 0x7f02e409d630 2023-09-30T22:33:31.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.419+0000 7f02eba15700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f02e40a53d0 con 0x7f02e40a0a30 2023-09-30T22:33:31.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.419+0000 7f02e97b1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f02e409d630 0x7f02e41c7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:31.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.419+0000 7f02e97b1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f02e409d630 0x7f02e41c7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54706/0 (socket says 172.21.15.16:54706) 2023-09-30T22:33:31.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.419+0000 7f02e97b1700 1 -- 172.21.15.16:0/2764253059 learned_addr learned my addr 172.21.15.16:0/2764253059 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:31.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.419+0000 7f02e9fb2700 1 --2- 172.21.15.16:0/2764253059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02e4133930 0x7f02e41c89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:31.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.419+0000 7f02e8fb0700 1 --2- 172.21.15.16:0/2764253059 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f02e40a0a30 0x7f02e41c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:31.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.419+0000 7f02e9fb2700 1 -- 172.21.15.16:0/2764253059 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f02e40a0a30 msgr2=0x7f02e41c8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:31.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.419+0000 7f02e9fb2700 1 --2- 172.21.15.16:0/2764253059 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f02e40a0a30 0x7f02e41c8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:31.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.419+0000 7f02e9fb2700 1 -- 172.21.15.16:0/2764253059 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f02e409d630 msgr2=0x7f02e41c7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:31.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.419+0000 7f02e9fb2700 1 --2- 172.21.15.16:0/2764253059 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f02e409d630 0x7f02e41c7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:31.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.419+0000 7f02e9fb2700 1 -- 172.21.15.16:0/2764253059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d40094a0 con 0x7f02e4133930 2023-09-30T22:33:31.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.419+0000 7f02e9fb2700 1 --2- 172.21.15.16:0/2764253059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02e4133930 0x7f02e41c89a0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f02dc00cbe0 tx=0x7f02dc00cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:31.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.420+0000 7f02d9ffb700 1 -- 172.21.15.16:0/2764253059 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02dc00aa40 con 0x7f02e4133930 2023-09-30T22:33:31.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.420+0000 7f02d9ffb700 1 -- 172.21.15.16:0/2764253059 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02dc00abc0 con 0x7f02e4133930 2023-09-30T22:33:31.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.420+0000 7f02eba15700 1 -- 172.21.15.16:0/2764253059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02e0009550 con 0x7f02e4133930 2023-09-30T22:33:31.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.420+0000 7f02d9ffb700 1 -- 172.21.15.16:0/2764253059 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02dc0070d0 con 0x7f02e4133930 2023-09-30T22:33:31.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.420+0000 7f02eba15700 1 -- 172.21.15.16:0/2764253059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02e4121f20 con 0x7f02e4133930 2023-09-30T22:33:31.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.421+0000 7f02d9ffb700 1 -- 172.21.15.16:0/2764253059 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f02dc007230 con 0x7f02e4133930 2023-09-30T22:33:31.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.421+0000 7f02d9ffb700 1 -- 172.21.15.16:0/2764253059 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f02dc03f3c0 con 0x7f02e4133930 2023-09-30T22:33:31.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.421+0000 7f02eba15700 1 -- 172.21.15.16:0/2764253059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02e4122cc0 con 0x7f02e4133930 2023-09-30T22:33:31.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.427+0000 7f02d9ffb700 1 -- 172.21.15.16:0/2764253059 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f02dc018080 con 0x7f02e4133930 2023-09-30T22:33:31.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.458+0000 7fcfe8a32700 1 Processor -- start 2023-09-30T22:33:31.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.459+0000 7fcfe8a32700 1 -- start start 2023-09-30T22:33:31.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.460+0000 7fcfe8a32700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcfe406f5d0 0x7fcfe412ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:31.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.460+0000 7fcfe8a32700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcfe4135c20 0x7fcfe4138000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:31.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.460+0000 7fcfe8a32700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfe406fa40 con 0x7fcfe412cf60 2023-09-30T22:33:31.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.460+0000 7fcfe8a32700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfe412cd90 con 0x7fcfe4135c20 2023-09-30T22:33:31.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.460+0000 7fcfe8a32700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe4071780 con 0x7fcfe406f5d0 2023-09-30T22:33:31.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.460+0000 7fcfe259c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fcfe412cf60 0x7fcfe412f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35036/0 (socket says 172.21.15.16:35036) 2023-09-30T22:33:31.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.460+0000 7fcfe259c700 1 -- 172.21.15.16:0/2218809967 learned_addr learned my addr 172.21.15.16:0/2218809967 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:31.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.460+0000 7fcfe2d9d700 1 --2- 172.21.15.16:0/2218809967 >> v2:172.21.15.16:3300/0 conn(0x7fcfe4135c20 0x7fcfe4138000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:31.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.460+0000 7fcfe1d9b700 1 --2- 172.21.15.16:0/2218809967 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcfe406f5d0 0x7fcfe412ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:31.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.460+0000 7fcfe2d9d700 1 -- 172.21.15.16:0/2218809967 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcfe406f5d0 msgr2=0x7fcfe412ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:31.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.460+0000 7fcfe2d9d700 1 --2- 172.21.15.16:0/2218809967 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcfe406f5d0 0x7fcfe412ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:31.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.460+0000 7fcfe2d9d700 1 -- 172.21.15.16:0/2218809967 >> v1:172.21.15.16:6789/0 conn(0x7fcfe412cf60 legacy=0x7fcfe412f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:31.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.460+0000 7fcfe2d9d700 1 -- 172.21.15.16:0/2218809967 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe4071a40 con 0x7fcfe4135c20 2023-09-30T22:33:31.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.461+0000 7fcfe2d9d700 1 --2- 172.21.15.16:0/2218809967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfe4135c20 0x7fcfe4138000 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc009dc0 tx=0x7fcfdc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95db13856c535a59 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:31.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.461+0000 7fcfe159a700 1 -- 172.21.15.16:0/2218809967 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfdc00e070 con 0x7fcfe4135c20 2023-09-30T22:33:31.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.461+0000 7fcfe159a700 1 -- 172.21.15.16:0/2218809967 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfdc00bae0 con 0x7fcfe4135c20 2023-09-30T22:33:31.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.461+0000 7fcfe159a700 1 -- 172.21.15.16:0/2218809967 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfdc00be90 con 0x7fcfe4135c20 2023-09-30T22:33:31.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.462+0000 7fcfe8a32700 1 -- 172.21.15.16:0/2218809967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfe4135c20 msgr2=0x7fcfe4138000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:31.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.462+0000 7fcfe8a32700 1 --2- 172.21.15.16:0/2218809967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfe4135c20 0x7fcfe4138000 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fcfdc009dc0 tx=0x7fcfdc00b330 comp rx=0 tx=0).stop 2023-09-30T22:33:31.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.462+0000 7fcfe8a32700 1 -- 172.21.15.16:0/2218809967 shutdown_connections 2023-09-30T22:33:31.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.462+0000 7fcfe8a32700 1 --2- 172.21.15.16:0/2218809967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfe4135c20 0x7fcfe4138000 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:31.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.462+0000 7fcfe8a32700 1 --2- 172.21.15.16:0/2218809967 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcfe406f5d0 0x7fcfe412ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:31.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.462+0000 7fcfe8a32700 1 -- 172.21.15.16:0/2218809967 >> 172.21.15.16:0/2218809967 conn(0x7fcfe40a2aa0 msgr2=0x7fcfe40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:31.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.462+0000 7fcfe8a32700 1 -- 172.21.15.16:0/2218809967 shutdown_connections 2023-09-30T22:33:31.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.462+0000 7fcfe8a32700 1 -- 172.21.15.16:0/2218809967 wait complete. 2023-09-30T22:33:31.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.464+0000 7fcfe8a32700 1 Processor -- start 2023-09-30T22:33:31.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.464+0000 7fcfe8a32700 1 -- start start 2023-09-30T22:33:31.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.465+0000 7fcfe8a32700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfe406f5d0 0x7fcfe409bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:31.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.465+0000 7fcfe8a32700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcfe412cf60 0x7fcfe409c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:31.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.465+0000 7fcfe8a32700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcfe4135c20 0x7fcfe409c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:31.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.465+0000 7fcfe8a32700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe409da80 con 0x7fcfe406f5d0 2023-09-30T22:33:31.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.465+0000 7fcfe8a32700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe41310b0 con 0x7fcfe412cf60 2023-09-30T22:33:31.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.465+0000 7fcfe8a32700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe4131b20 con 0x7fcfe4135c20 2023-09-30T22:33:31.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.465+0000 7fcfe1d9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcfe412cf60 0x7fcfe409c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:31.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.465+0000 7fcfe1d9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcfe412cf60 0x7fcfe409c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54734/0 (socket says 172.21.15.16:54734) 2023-09-30T22:33:31.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.465+0000 7fcfe1d9b700 1 -- 172.21.15.16:0/1536199827 learned_addr learned my addr 172.21.15.16:0/1536199827 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:31.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.465+0000 7fcfe259c700 1 --2- 172.21.15.16:0/1536199827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfe406f5d0 0x7fcfe409bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:31.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.465+0000 7fcfe2d9d700 1 --2- 172.21.15.16:0/1536199827 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcfe4135c20 0x7fcfe409c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:31.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.465+0000 7fcfe259c700 1 -- 172.21.15.16:0/1536199827 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcfe4135c20 msgr2=0x7fcfe409c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:31.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.465+0000 7fcfe259c700 1 --2- 172.21.15.16:0/1536199827 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcfe4135c20 0x7fcfe409c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:31.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.465+0000 7fcfe259c700 1 -- 172.21.15.16:0/1536199827 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcfe412cf60 msgr2=0x7fcfe409c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:31.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.465+0000 7fcfe259c700 1 --2- 172.21.15.16:0/1536199827 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcfe412cf60 0x7fcfe409c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:31.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.465+0000 7fcfe259c700 1 -- 172.21.15.16:0/1536199827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfdc009a70 con 0x7fcfe406f5d0 2023-09-30T22:33:31.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.465+0000 7fcfe2d9d700 1 --2- 172.21.15.16:0/1536199827 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcfe4135c20 0x7fcfe409c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:31.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.465+0000 7fcfe1d9b700 1 --2- 172.21.15.16:0/1536199827 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcfe412cf60 0x7fcfe409c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:31.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.466+0000 7fcfe259c700 1 --2- 172.21.15.16:0/1536199827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfe406f5d0 0x7fcfe409bdb0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fcfd400c500 tx=0x7fcfd400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:31.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.466+0000 7fcfd2ffd700 1 -- 172.21.15.16:0/1536199827 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd4005d70 con 0x7fcfe406f5d0 2023-09-30T22:33:31.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.466+0000 7fcfd2ffd700 1 -- 172.21.15.16:0/1536199827 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfd40070d0 con 0x7fcfe406f5d0 2023-09-30T22:33:31.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.466+0000 7fcfe8a32700 1 -- 172.21.15.16:0/1536199827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfe4131390 con 0x7fcfe406f5d0 2023-09-30T22:33:31.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.466+0000 7fcfd2ffd700 1 -- 172.21.15.16:0/1536199827 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd4007440 con 0x7fcfe406f5d0 2023-09-30T22:33:31.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.466+0000 7fcfe8a32700 1 -- 172.21.15.16:0/1536199827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfe409e340 con 0x7fcfe406f5d0 2023-09-30T22:33:31.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.467+0000 7fcfd2ffd700 1 -- 172.21.15.16:0/1536199827 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcfd40075a0 con 0x7fcfe406f5d0 2023-09-30T22:33:31.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.467+0000 7fcfd2ffd700 1 -- 172.21.15.16:0/1536199827 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fcfd403eaf0 con 0x7fcfe406f5d0 2023-09-30T22:33:31.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.467+0000 7fcfe8a32700 1 -- 172.21.15.16:0/1536199827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfe4074ea0 con 0x7fcfe406f5d0 2023-09-30T22:33:31.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.472+0000 7fcfd2ffd700 1 -- 172.21.15.16:0/1536199827 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcfd4017020 con 0x7fcfe406f5d0 2023-09-30T22:33:31.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.501+0000 7fc568767700 1 Processor -- start 2023-09-30T22:33:31.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.501+0000 7fc568767700 1 -- start start 2023-09-30T22:33:31.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.502+0000 7fc568767700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc56009d6f0 0x7fc56009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:31.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.502+0000 7fc568767700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc56009e410 0x7fc560134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:31.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.502+0000 7fc568767700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc56009b120 con 0x7fc56009e000 2023-09-30T22:33:31.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.502+0000 7fc568767700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc56006b460 con 0x7fc56009d6f0 2023-09-30T22:33:31.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.502+0000 7fc568767700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc56009a7e0 con 0x7fc56009e410 2023-09-30T22:33:31.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.502+0000 7fc566503700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc56009d6f0 0x7fc56009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:31.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.502+0000 7fc565d02700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc56009e000 0x7fc5601339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35048/0 (socket says 172.21.15.16:35048) 2023-09-30T22:33:31.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.502+0000 7fc565d02700 1 -- 172.21.15.16:0/1951571955 learned_addr learned my addr 172.21.15.16:0/1951571955 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:31.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.502+0000 7fc566d04700 1 --2- 172.21.15.16:0/1951571955 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc56009e410 0x7fc560134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:31.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.502+0000 7fc566d04700 1 -- 172.21.15.16:0/1951571955 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc56009d6f0 msgr2=0x7fc56009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:31.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.502+0000 7fc566d04700 1 --2- 172.21.15.16:0/1951571955 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc56009d6f0 0x7fc56009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:31.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.502+0000 7fc566d04700 1 -- 172.21.15.16:0/1951571955 >> v1:172.21.15.16:6789/0 conn(0x7fc56009e000 legacy=0x7fc5601339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:31.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.502+0000 7fc566d04700 1 -- 172.21.15.16:0/1951571955 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc56009a500 con 0x7fc56009e410 2023-09-30T22:33:31.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.503+0000 7fc566d04700 1 --2- 172.21.15.16:0/1951571955 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc56009e410 0x7fc560134100 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fc558009df0 tx=0x7fc55800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c33c30583c384da server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:31.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.503+0000 7fc565501700 1 -- 172.21.15.16:0/1951571955 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc55800e070 con 0x7fc56009e410 2023-09-30T22:33:31.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.503+0000 7fc565501700 1 -- 172.21.15.16:0/1951571955 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc558003a20 con 0x7fc56009e410 2023-09-30T22:33:31.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.503+0000 7fc565501700 1 -- 172.21.15.16:0/1951571955 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc558003db0 con 0x7fc56009e410 2023-09-30T22:33:31.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.504+0000 7fc568767700 1 -- 172.21.15.16:0/1951571955 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc56009e410 msgr2=0x7fc560134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:31.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.504+0000 7fc568767700 1 --2- 172.21.15.16:0/1951571955 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc56009e410 0x7fc560134100 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fc558009df0 tx=0x7fc55800b330 comp rx=0 tx=0).stop 2023-09-30T22:33:31.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.504+0000 7fc568767700 1 -- 172.21.15.16:0/1951571955 shutdown_connections 2023-09-30T22:33:31.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.504+0000 7fc568767700 1 --2- 172.21.15.16:0/1951571955 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc56009e410 0x7fc560134100 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:31.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.504+0000 7fc568767700 1 --2- 172.21.15.16:0/1951571955 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc56009d6f0 0x7fc56009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:31.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.504+0000 7fc568767700 1 -- 172.21.15.16:0/1951571955 >> 172.21.15.16:0/1951571955 conn(0x7fc560066f50 msgr2=0x7fc56009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:31.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.504+0000 7fc568767700 1 -- 172.21.15.16:0/1951571955 shutdown_connections 2023-09-30T22:33:31.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.504+0000 7fc568767700 1 -- 172.21.15.16:0/1951571955 wait complete. 2023-09-30T22:33:31.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.506+0000 7fc568767700 1 Processor -- start 2023-09-30T22:33:31.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.506+0000 7fc568767700 1 -- start start 2023-09-30T22:33:31.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.506+0000 7fc568767700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc56009d6f0 0x7fc560122540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:31.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.507+0000 7fc568767700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc56009e000 0x7fc560122a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:31.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.507+0000 7fc568767700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc56009e410 0x7fc560122fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:31.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.507+0000 7fc568767700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc560125cf0 con 0x7fc56009e000 2023-09-30T22:33:31.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.507+0000 7fc568767700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc560125fd0 con 0x7fc56009e410 2023-09-30T22:33:31.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.507+0000 7fc568767700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc5600a1d00 con 0x7fc56009d6f0 2023-09-30T22:33:31.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.507+0000 7fc566d04700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc56009e410 0x7fc560122fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:31.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.507+0000 7fc566d04700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc56009e410 0x7fc560122fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54744/0 (socket says 172.21.15.16:54744) 2023-09-30T22:33:31.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.507+0000 7fc566d04700 1 -- 172.21.15.16:0/2815209238 learned_addr learned my addr 172.21.15.16:0/2815209238 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:31.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.507+0000 7fc565d02700 1 --2- 172.21.15.16:0/2815209238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc56009e000 0x7fc560122a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:31.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.507+0000 7fc566503700 1 --2- 172.21.15.16:0/2815209238 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc56009d6f0 0x7fc560122540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:31.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.507+0000 7fc566d04700 1 -- 172.21.15.16:0/2815209238 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc56009d6f0 msgr2=0x7fc560122540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:31.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.507+0000 7fc566d04700 1 --2- 172.21.15.16:0/2815209238 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc56009d6f0 0x7fc560122540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:31.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.507+0000 7fc566d04700 1 -- 172.21.15.16:0/2815209238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc56009e000 msgr2=0x7fc560122a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:31.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.507+0000 7fc566d04700 1 --2- 172.21.15.16:0/2815209238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc56009e000 0x7fc560122a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:31.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.507+0000 7fc566d04700 1 -- 172.21.15.16:0/2815209238 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc558009a70 con 0x7fc56009e410 2023-09-30T22:33:31.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.507+0000 7fc566503700 1 --2- 172.21.15.16:0/2815209238 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc56009d6f0 0x7fc560122540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:33:31.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.508+0000 7fc566d04700 1 --2- 172.21.15.16:0/2815209238 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc56009e410 0x7fc560122fc0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fc55800b370 tx=0x7fc558004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:31.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.508+0000 7fc556ffd700 1 -- 172.21.15.16:0/2815209238 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc558025070 con 0x7fc56009e410 2023-09-30T22:33:31.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.508+0000 7fc556ffd700 1 -- 172.21.15.16:0/2815209238 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc55800bc30 con 0x7fc56009e410 2023-09-30T22:33:31.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.508+0000 7fc568767700 1 -- 172.21.15.16:0/2815209238 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc56009c710 con 0x7fc56009e410 2023-09-30T22:33:31.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.508+0000 7fc556ffd700 1 -- 172.21.15.16:0/2815209238 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc558004030 con 0x7fc56009e410 2023-09-30T22:33:31.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.508+0000 7fc568767700 1 -- 172.21.15.16:0/2815209238 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc560125760 con 0x7fc56009e410 2023-09-30T22:33:31.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.509+0000 7fc556ffd700 1 -- 172.21.15.16:0/2815209238 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc558004190 con 0x7fc56009e410 2023-09-30T22:33:31.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.509+0000 7fc556ffd700 1 -- 172.21.15.16:0/2815209238 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fc558048160 con 0x7fc56009e410 2023-09-30T22:33:31.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.509+0000 7fc568767700 1 -- 172.21.15.16:0/2815209238 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc560074ea0 con 0x7fc56009e410 2023-09-30T22:33:31.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.514+0000 7fc556ffd700 1 -- 172.21.15.16:0/2815209238 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc55802e080 con 0x7fc56009e410 2023-09-30T22:33:31.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.540+0000 7fd4e47aa700 1 Processor -- start 2023-09-30T22:33:31.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.541+0000 7fd4e47aa700 1 -- start start 2023-09-30T22:33:31.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.542+0000 7fd4e47aa700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd4dc130540 0x7fd4dc130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:31.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.542+0000 7fd4e47aa700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4dc130e50 0x7fd4dc06f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:31.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.542+0000 7fd4e47aa700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd4dc06fb20 0x7fd4dc06ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:31.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.542+0000 7fd4e47aa700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4dc09a390 con 0x7fd4dc130540 2023-09-30T22:33:31.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.542+0000 7fd4e47aa700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc0716a0 con 0x7fd4dc130e50 2023-09-30T22:33:31.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.542+0000 7fd4e47aa700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc071290 con 0x7fd4dc06fb20 2023-09-30T22:33:31.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.542+0000 7fd4e1d45700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4dc130e50 0x7fd4dc06f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:31.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.542+0000 7fd4e1d45700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4dc130e50 0x7fd4dc06f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54748/0 (socket says 172.21.15.16:54748) 2023-09-30T22:33:31.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.542+0000 7fd4e1d45700 1 -- 172.21.15.16:0/1431475741 learned_addr learned my addr 172.21.15.16:0/1431475741 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:31.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.542+0000 7fd4e2d47700 1 --2- 172.21.15.16:0/1431475741 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd4dc06fb20 0x7fd4dc06ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:31.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.542+0000 7fd4e2546700 1 --2- 172.21.15.16:0/1431475741 >> v2:172.21.15.16:3300/0 conn(0x7fd4dc130540 0x7fd4dc130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:31.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.542+0000 7fd4e1d45700 1 -- 172.21.15.16:0/1431475741 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd4dc06fb20 msgr2=0x7fd4dc06ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:31.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.542+0000 7fd4e1d45700 1 --2- 172.21.15.16:0/1431475741 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd4dc06fb20 0x7fd4dc06ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:31.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.542+0000 7fd4e1d45700 1 -- 172.21.15.16:0/1431475741 >> v2:172.21.15.16:3300/0 conn(0x7fd4dc130540 msgr2=0x7fd4dc130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:31.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.542+0000 7fd4e1d45700 1 --2- 172.21.15.16:0/1431475741 >> v2:172.21.15.16:3300/0 conn(0x7fd4dc130540 0x7fd4dc130910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:31.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.542+0000 7fd4e1d45700 1 -- 172.21.15.16:0/1431475741 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4dc130340 con 0x7fd4dc130e50 2023-09-30T22:33:31.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.542+0000 7fd4e2d47700 1 --2- 172.21.15.16:0/1431475741 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd4dc06fb20 0x7fd4dc06ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:31.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.543+0000 7fd4e1d45700 1 --2- 172.21.15.16:0/1431475741 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4dc130e50 0x7fd4dc06f5e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc00bb90 tx=0x7fd4cc00bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1f9e199b0af7635 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:31.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.543+0000 7fd4e1544700 1 -- 172.21.15.16:0/1431475741 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc00dc70 con 0x7fd4dc130e50 2023-09-30T22:33:31.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.543+0000 7fd4e1544700 1 -- 172.21.15.16:0/1431475741 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4cc00ddd0 con 0x7fd4dc130e50 2023-09-30T22:33:31.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.543+0000 7fd4e1544700 1 -- 172.21.15.16:0/1431475741 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc012990 con 0x7fd4dc130e50 2023-09-30T22:33:31.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.544+0000 7fd4e47aa700 1 -- 172.21.15.16:0/1431475741 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4dc130e50 msgr2=0x7fd4dc06f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:31.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.544+0000 7fd4e47aa700 1 --2- 172.21.15.16:0/1431475741 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4dc130e50 0x7fd4dc06f5e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc00bb90 tx=0x7fd4cc00bea0 comp rx=0 tx=0).stop 2023-09-30T22:33:31.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.544+0000 7fd4e47aa700 1 -- 172.21.15.16:0/1431475741 shutdown_connections 2023-09-30T22:33:31.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.544+0000 7fd4e47aa700 1 --2- 172.21.15.16:0/1431475741 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd4dc06fb20 0x7fd4dc06ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:31.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.544+0000 7fd4e47aa700 1 --2- 172.21.15.16:0/1431475741 >> v2:172.21.15.16:3300/0 conn(0x7fd4dc130540 0x7fd4dc130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:31.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.544+0000 7fd4e47aa700 1 --2- 172.21.15.16:0/1431475741 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4dc130e50 0x7fd4dc06f5e0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:31.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.544+0000 7fd4e47aa700 1 -- 172.21.15.16:0/1431475741 >> 172.21.15.16:0/1431475741 conn(0x7fd4dc0a2aa0 msgr2=0x7fd4dc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:31.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.544+0000 7fd4e47aa700 1 -- 172.21.15.16:0/1431475741 shutdown_connections 2023-09-30T22:33:31.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.544+0000 7fd4e47aa700 1 -- 172.21.15.16:0/1431475741 wait complete. 2023-09-30T22:33:31.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.545+0000 7fd4e47aa700 1 Processor -- start 2023-09-30T22:33:31.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.545+0000 7fd4e47aa700 1 -- start start 2023-09-30T22:33:31.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.545+0000 7fd4e47aa700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd4dc06fb20 0x7fd4dc1196f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:31.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.545+0000 7fd4e47aa700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4dc130540 0x7fd4dc119c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:31.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.545+0000 7fd4e47aa700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4dc130e50 0x7fd4dc11a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:31.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.545+0000 7fd4e47aa700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc11abc0 con 0x7fd4dc130540 2023-09-30T22:33:31.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.545+0000 7fd4e47aa700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc11b950 con 0x7fd4dc130e50 2023-09-30T22:33:31.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.545+0000 7fd4e47aa700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc11d210 con 0x7fd4dc06fb20 2023-09-30T22:33:31.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.546+0000 7fd4e2546700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd4dc06fb20 0x7fd4dc1196f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:31.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.546+0000 7fd4e2546700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd4dc06fb20 0x7fd4dc1196f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:53714/0 (socket says 172.21.15.16:53714) 2023-09-30T22:33:31.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.546+0000 7fd4e2546700 1 -- 172.21.15.16:0/2772984803 learned_addr learned my addr 172.21.15.16:0/2772984803 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:31.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.546+0000 7fd4e1d45700 1 --2- 172.21.15.16:0/2772984803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4dc130540 0x7fd4dc119c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:31.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.546+0000 7fd4e2d47700 1 --2- 172.21.15.16:0/2772984803 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4dc130e50 0x7fd4dc11a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:31.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.546+0000 7fd4e2546700 1 -- 172.21.15.16:0/2772984803 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4dc130e50 msgr2=0x7fd4dc11a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:31.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.546+0000 7fd4e2546700 1 --2- 172.21.15.16:0/2772984803 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4dc130e50 0x7fd4dc11a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:31.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.546+0000 7fd4e2546700 1 -- 172.21.15.16:0/2772984803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4dc130540 msgr2=0x7fd4dc119c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:31.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.546+0000 7fd4e2546700 1 --2- 172.21.15.16:0/2772984803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4dc130540 0x7fd4dc119c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:31.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.546+0000 7fd4e2546700 1 -- 172.21.15.16:0/2772984803 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d4009a70 con 0x7fd4dc06fb20 2023-09-30T22:33:31.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.546+0000 7fd4e2546700 1 --2- 172.21.15.16:0/2772984803 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd4dc06fb20 0x7fd4dc1196f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fd4d800e9e0 tx=0x7fd4d800ecf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:31.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.546+0000 7fd4d2ffd700 1 -- 172.21.15.16:0/2772984803 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d800ee70 con 0x7fd4dc06fb20 2023-09-30T22:33:31.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.546+0000 7fd4d2ffd700 1 -- 172.21.15.16:0/2772984803 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4d800c850 con 0x7fd4dc06fb20 2023-09-30T22:33:31.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.547+0000 7fd4e47aa700 1 -- 172.21.15.16:0/2772984803 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4cc00b7f0 con 0x7fd4dc06fb20 2023-09-30T22:33:31.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.547+0000 7fd4d2ffd700 1 -- 172.21.15.16:0/2772984803 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d800cbc0 con 0x7fd4dc06fb20 2023-09-30T22:33:31.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.547+0000 7fd4e47aa700 1 -- 172.21.15.16:0/2772984803 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4dc0a0aa0 con 0x7fd4dc06fb20 2023-09-30T22:33:31.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.548+0000 7fd4d2ffd700 1 -- 172.21.15.16:0/2772984803 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd4d800cd20 con 0x7fd4dc06fb20 2023-09-30T22:33:31.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.548+0000 7fd4e47aa700 1 -- 172.21.15.16:0/2772984803 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4dc074ea0 con 0x7fd4dc06fb20 2023-09-30T22:33:31.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.548+0000 7fd4d2ffd700 1 -- 172.21.15.16:0/2772984803 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd4d803f700 con 0x7fd4dc06fb20 2023-09-30T22:33:31.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.557+0000 7fd4d2ffd700 1 -- 172.21.15.16:0/2772984803 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd4d801bc30 con 0x7fd4dc06fb20 2023-09-30T22:33:31.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.650+0000 7f02eba15700 1 -- 172.21.15.16:0/2764253059 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f02e41220b0 con 0x7f02e4133930 2023-09-30T22:33:31.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.650+0000 7f02d9ffb700 1 -- 172.21.15.16:0/2764253059 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f02dc0074e0 con 0x7f02e4133930 2023-09-30T22:33:31.652 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:33:31.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.653+0000 7f02eba15700 1 -- 172.21.15.16:0/2764253059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02e4133930 msgr2=0x7f02e41c89a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:31.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.653+0000 7f02eba15700 1 --2- 172.21.15.16:0/2764253059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02e4133930 0x7f02e41c89a0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f02dc00cbe0 tx=0x7f02dc00cef0 comp rx=0 tx=0).stop 2023-09-30T22:33:31.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.653+0000 7f02eba15700 1 -- 172.21.15.16:0/2764253059 shutdown_connections 2023-09-30T22:33:31.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.653+0000 7f02eba15700 1 --2- 172.21.15.16:0/2764253059 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f02e409d630 0x7f02e41c7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:31.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.653+0000 7f02eba15700 1 --2- 172.21.15.16:0/2764253059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02e4133930 0x7f02e41c89a0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:31.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.653+0000 7f02eba15700 1 --2- 172.21.15.16:0/2764253059 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f02e40a0a30 0x7f02e41c8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:31.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.653+0000 7f02eba15700 1 -- 172.21.15.16:0/2764253059 >> 172.21.15.16:0/2764253059 conn(0x7f02e40a3050 msgr2=0x7f02e409ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:31.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.653+0000 7f02eba15700 1 -- 172.21.15.16:0/2764253059 shutdown_connections 2023-09-30T22:33:31.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.653+0000 7f02eba15700 1 -- 172.21.15.16:0/2764253059 wait complete. 2023-09-30T22:33:31.669 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:33:31.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.685+0000 7fcfe8a32700 1 -- 172.21.15.16:0/1536199827 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fcfe4135a00 con 0x7fcfe406f5d0 2023-09-30T22:33:31.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.686+0000 7fcfd2ffd700 1 -- 172.21.15.16:0/1536199827 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcfd4024070 con 0x7fcfe406f5d0 2023-09-30T22:33:31.687 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:33:31.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.688+0000 7fcfe8a32700 1 -- 172.21.15.16:0/1536199827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfe406f5d0 msgr2=0x7fcfe409bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:31.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.688+0000 7fcfe8a32700 1 --2- 172.21.15.16:0/1536199827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfe406f5d0 0x7fcfe409bdb0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fcfd400c500 tx=0x7fcfd400c810 comp rx=0 tx=0).stop 2023-09-30T22:33:31.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.688+0000 7fcfe8a32700 1 -- 172.21.15.16:0/1536199827 shutdown_connections 2023-09-30T22:33:31.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.688+0000 7fcfe8a32700 1 --2- 172.21.15.16:0/1536199827 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcfe4135c20 0x7fcfe409c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:31.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.688+0000 7fcfe8a32700 1 --2- 172.21.15.16:0/1536199827 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcfe406f5d0 0x7fcfe409bdb0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:31.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.688+0000 7fcfe8a32700 1 --2- 172.21.15.16:0/1536199827 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcfe412cf60 0x7fcfe409c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:31.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.688+0000 7fcfe8a32700 1 -- 172.21.15.16:0/1536199827 >> 172.21.15.16:0/1536199827 conn(0x7fcfe40a2aa0 msgr2=0x7fcfe412a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:31.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.688+0000 7fcfe8a32700 1 -- 172.21.15.16:0/1536199827 shutdown_connections 2023-09-30T22:33:31.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.688+0000 7fcfe8a32700 1 -- 172.21.15.16:0/1536199827 wait complete. 2023-09-30T22:33:31.704 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:33:31.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.727+0000 7fc568767700 1 -- 172.21.15.16:0/2815209238 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc560125b50 con 0x7fc56009e410 2023-09-30T22:33:31.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.727+0000 7fc556ffd700 1 -- 172.21.15.16:0/2815209238 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc558017020 con 0x7fc56009e410 2023-09-30T22:33:31.728 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:33:31.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.729+0000 7fc568767700 1 -- 172.21.15.16:0/2815209238 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc56009e410 msgr2=0x7fc560122fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:31.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.729+0000 7fc568767700 1 --2- 172.21.15.16:0/2815209238 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc56009e410 0x7fc560122fc0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fc55800b370 tx=0x7fc558004ad0 comp rx=0 tx=0).stop 2023-09-30T22:33:31.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.729+0000 7fc568767700 1 -- 172.21.15.16:0/2815209238 shutdown_connections 2023-09-30T22:33:31.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.729+0000 7fc568767700 1 --2- 172.21.15.16:0/2815209238 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc56009e410 0x7fc560122fc0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:31.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.729+0000 7fc568767700 1 --2- 172.21.15.16:0/2815209238 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc56009d6f0 0x7fc560122540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:31.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.729+0000 7fc568767700 1 --2- 172.21.15.16:0/2815209238 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc56009e000 0x7fc560122a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:31.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.729+0000 7fc568767700 1 -- 172.21.15.16:0/2815209238 >> 172.21.15.16:0/2815209238 conn(0x7fc560066f50 msgr2=0x7fc560070ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:31.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.729+0000 7fc568767700 1 -- 172.21.15.16:0/2815209238 shutdown_connections 2023-09-30T22:33:31.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.730+0000 7fc568767700 1 -- 172.21.15.16:0/2815209238 wait complete. 2023-09-30T22:33:31.744 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:33:31.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.767+0000 7fd4e47aa700 1 -- 172.21.15.16:0/2772984803 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd4dc06b9e0 con 0x7fd4dc06fb20 2023-09-30T22:33:31.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.767+0000 7fd4d2ffd700 1 -- 172.21.15.16:0/2772984803 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd4d8017070 con 0x7fd4dc06fb20 2023-09-30T22:33:31.769 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:33:31.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.769+0000 7fd4e47aa700 1 -- 172.21.15.16:0/2772984803 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd4dc06fb20 msgr2=0x7fd4dc1196f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:31.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.770+0000 7fd4e47aa700 1 --2- 172.21.15.16:0/2772984803 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd4dc06fb20 0x7fd4dc1196f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fd4d800e9e0 tx=0x7fd4d800ecf0 comp rx=0 tx=0).stop 2023-09-30T22:33:31.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.770+0000 7fd4e47aa700 1 -- 172.21.15.16:0/2772984803 shutdown_connections 2023-09-30T22:33:31.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.770+0000 7fd4e47aa700 1 --2- 172.21.15.16:0/2772984803 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4dc130e50 0x7fd4dc11a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:31.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.770+0000 7fd4e47aa700 1 --2- 172.21.15.16:0/2772984803 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd4dc06fb20 0x7fd4dc1196f0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:31.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.770+0000 7fd4e47aa700 1 --2- 172.21.15.16:0/2772984803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4dc130540 0x7fd4dc119c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:31.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.770+0000 7fd4e47aa700 1 -- 172.21.15.16:0/2772984803 >> 172.21.15.16:0/2772984803 conn(0x7fd4dc0a2aa0 msgr2=0x7fd4dc12bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:31.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.770+0000 7fd4e47aa700 1 -- 172.21.15.16:0/2772984803 shutdown_connections 2023-09-30T22:33:31.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:31.770+0000 7fd4e47aa700 1 -- 172.21.15.16:0/2772984803 wait complete. 2023-09-30T22:33:31.784 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:33:32.670 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:33:32.704 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:33:32.745 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:33:32.785 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:33:32.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.822+0000 7fb09b757700 1 Processor -- start 2023-09-30T22:33:32.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.822+0000 7fb09b757700 1 -- start start 2023-09-30T22:33:32.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.822+0000 7fb09b757700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb09406a770 0x7fb09406ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:32.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.822+0000 7fb09b757700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb09406b080 0x7fb094131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:32.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.822+0000 7fb09b757700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb09409ece0 con 0x7fb094131b40 2023-09-30T22:33:32.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.822+0000 7fb09b757700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb09409e760 con 0x7fb09406a770 2023-09-30T22:33:32.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.822+0000 7fb09b757700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb09409e040 con 0x7fb09406b080 2023-09-30T22:33:32.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.823+0000 7fb0994f3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb09406a770 0x7fb09406ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:32.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.823+0000 7fb098cf2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb09406b080 0x7fb094131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:32.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.823+0000 7fb0994f3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb09406a770 0x7fb09406ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54760/0 (socket says 172.21.15.16:54760) 2023-09-30T22:33:32.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.823+0000 7fb0994f3700 1 -- 172.21.15.16:0/4268281845 learned_addr learned my addr 172.21.15.16:0/4268281845 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:32.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.823+0000 7fb0994f3700 1 -- 172.21.15.16:0/4268281845 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb09406b080 msgr2=0x7fb094131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:32.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.823+0000 7fb0994f3700 1 --2- 172.21.15.16:0/4268281845 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb09406b080 0x7fb094131600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:32.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.823+0000 7fb0994f3700 1 -- 172.21.15.16:0/4268281845 >> v1:172.21.15.16:6789/0 conn(0x7fb094131b40 legacy=0x7fb094138850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:32.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.823+0000 7fb0994f3700 1 -- 172.21.15.16:0/4268281845 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb09409f630 con 0x7fb09406a770 2023-09-30T22:33:32.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.824+0000 7fb0994f3700 1 --2- 172.21.15.16:0/4268281845 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb09406a770 0x7fb09406ab40 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fb090009920 tx=0x7fb090009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ac8f6e47470a574 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:32.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.824+0000 7fb08bfff700 1 -- 172.21.15.16:0/4268281845 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb090011830 con 0x7fb09406a770 2023-09-30T22:33:32.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.824+0000 7fb08bfff700 1 -- 172.21.15.16:0/4268281845 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb090011990 con 0x7fb09406a770 2023-09-30T22:33:32.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.824+0000 7fb08bfff700 1 -- 172.21.15.16:0/4268281845 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb090011d20 con 0x7fb09406a770 2023-09-30T22:33:32.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.824+0000 7fb09b757700 1 -- 172.21.15.16:0/4268281845 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb09406a770 msgr2=0x7fb09406ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:32.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.824+0000 7fb09b757700 1 --2- 172.21.15.16:0/4268281845 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb09406a770 0x7fb09406ab40 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fb090009920 tx=0x7fb090009c30 comp rx=0 tx=0).stop 2023-09-30T22:33:32.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.825+0000 7fb09b757700 1 -- 172.21.15.16:0/4268281845 shutdown_connections 2023-09-30T22:33:32.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.825+0000 7fb09b757700 1 --2- 172.21.15.16:0/4268281845 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb09406a770 0x7fb09406ab40 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:32.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.825+0000 7fb09b757700 1 --2- 172.21.15.16:0/4268281845 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb09406b080 0x7fb094131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:32.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.825+0000 7fb09b757700 1 -- 172.21.15.16:0/4268281845 >> 172.21.15.16:0/4268281845 conn(0x7fb094066f50 msgr2=0x7fb09409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:32.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.825+0000 7fb09b757700 1 -- 172.21.15.16:0/4268281845 shutdown_connections 2023-09-30T22:33:32.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.825+0000 7fb09b757700 1 -- 172.21.15.16:0/4268281845 wait complete. 2023-09-30T22:33:32.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.826+0000 7fb09b757700 1 Processor -- start 2023-09-30T22:33:32.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.826+0000 7fb09b757700 1 -- start start 2023-09-30T22:33:32.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.826+0000 7fb09b757700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb09406a770 0x7fb0941c8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:32.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.826+0000 7fb09b757700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb09406b080 0x7fb0941c87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:32.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.826+0000 7fb09b757700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb094131b40 0x7fb0941c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:32.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.826+0000 7fb09b757700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb094131830 con 0x7fb094131b40 2023-09-30T22:33:32.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.826+0000 7fb09b757700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb094120460 con 0x7fb09406a770 2023-09-30T22:33:32.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.826+0000 7fb09b757700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb09409c140 con 0x7fb09406b080 2023-09-30T22:33:32.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.827+0000 7fb0994f3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb09406a770 0x7fb0941c8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:32.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.827+0000 7fb099cf4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb094131b40 0x7fb0941c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:32.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.827+0000 7fb0994f3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb09406a770 0x7fb0941c8270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54772/0 (socket says 172.21.15.16:54772) 2023-09-30T22:33:32.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.827+0000 7fb0994f3700 1 -- 172.21.15.16:0/808218455 learned_addr learned my addr 172.21.15.16:0/808218455 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:32.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.827+0000 7fb098cf2700 1 --2- 172.21.15.16:0/808218455 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb09406b080 0x7fb0941c87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:32.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.827+0000 7fb0994f3700 1 -- 172.21.15.16:0/808218455 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb09406b080 msgr2=0x7fb0941c87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:32.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.827+0000 7fb0994f3700 1 --2- 172.21.15.16:0/808218455 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb09406b080 0x7fb0941c87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:32.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.827+0000 7fb0994f3700 1 -- 172.21.15.16:0/808218455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb094131b40 msgr2=0x7fb0941c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:32.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.827+0000 7fb0994f3700 1 --2- 172.21.15.16:0/808218455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb094131b40 0x7fb0941c8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:32.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.827+0000 7fb0994f3700 1 -- 172.21.15.16:0/808218455 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb090009550 con 0x7fb09406a770 2023-09-30T22:33:32.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.827+0000 7fb099cf4700 1 --2- 172.21.15.16:0/808218455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb094131b40 0x7fb0941c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:32.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.827+0000 7fb0994f3700 1 --2- 172.21.15.16:0/808218455 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb09406a770 0x7fb0941c8270 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fb090000c00 tx=0x7fb090004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:32.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.828+0000 7fb089ffb700 1 -- 172.21.15.16:0/808218455 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb090004450 con 0x7fb09406a770 2023-09-30T22:33:32.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.828+0000 7fb09b757700 1 -- 172.21.15.16:0/808218455 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb09411d890 con 0x7fb09406a770 2023-09-30T22:33:32.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.828+0000 7fb09b757700 1 -- 172.21.15.16:0/808218455 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb09411f020 con 0x7fb09406a770 2023-09-30T22:33:32.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.828+0000 7fb089ffb700 1 -- 172.21.15.16:0/808218455 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0900045b0 con 0x7fb09406a770 2023-09-30T22:33:32.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.828+0000 7fb089ffb700 1 -- 172.21.15.16:0/808218455 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0900038c0 con 0x7fb09406a770 2023-09-30T22:33:32.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.829+0000 7fb089ffb700 1 -- 172.21.15.16:0/808218455 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb090003a20 con 0x7fb09406a770 2023-09-30T22:33:32.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.829+0000 7fb09b757700 1 -- 172.21.15.16:0/808218455 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0740052e0 con 0x7fb09406a770 2023-09-30T22:33:32.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.830+0000 7fb089ffb700 1 -- 172.21.15.16:0/808218455 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fb090021030 con 0x7fb09406a770 2023-09-30T22:33:32.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.835+0000 7fb089ffb700 1 -- 172.21.15.16:0/808218455 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb09001a430 con 0x7fb09406a770 2023-09-30T22:33:32.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.860+0000 7f853b3d8700 1 Processor -- start 2023-09-30T22:33:32.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.861+0000 7f853b3d8700 1 -- start start 2023-09-30T22:33:32.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.861+0000 7f853b3d8700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8534130560 0x7f8534130930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:32.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.861+0000 7f853b3d8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f853409eca0 0x7f853406f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:32.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.861+0000 7f853b3d8700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8534070a50 con 0x7f8534130e70 2023-09-30T22:33:32.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.861+0000 7f853b3d8700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85340717d0 con 0x7f8534130560 2023-09-30T22:33:32.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.861+0000 7f853b3d8700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8534130340 con 0x7f853409eca0 2023-09-30T22:33:32.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.862+0000 7f8538973700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f853409eca0 0x7f853406f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:32.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.862+0000 7f8538973700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f853409eca0 0x7f853406f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54780/0 (socket says 172.21.15.16:54780) 2023-09-30T22:33:32.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.862+0000 7f8538973700 1 -- 172.21.15.16:0/669664879 learned_addr learned my addr 172.21.15.16:0/669664879 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:32.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.862+0000 7f8539174700 1 --2- 172.21.15.16:0/669664879 >> v2:172.21.15.16:3300/0 conn(0x7f8534130560 0x7f8534130930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:32.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.862+0000 7f8538973700 1 -- 172.21.15.16:0/669664879 >> v2:172.21.15.16:3300/0 conn(0x7f8534130560 msgr2=0x7f8534130930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:32.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.862+0000 7f8538973700 1 --2- 172.21.15.16:0/669664879 >> v2:172.21.15.16:3300/0 conn(0x7f8534130560 0x7f8534130930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:32.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.862+0000 7f8538973700 1 -- 172.21.15.16:0/669664879 >> v1:172.21.15.16:6789/0 conn(0x7f8534130e70 legacy=0x7f853406fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:32.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.862+0000 7f8538973700 1 -- 172.21.15.16:0/669664879 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8534071060 con 0x7f853409eca0 2023-09-30T22:33:32.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.862+0000 7f8538973700 1 --2- 172.21.15.16:0/669664879 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f853409eca0 0x7f853406f620 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f85240098f0 tx=0x7f8524009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa76bc26a89e2b53 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:32.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.863+0000 7f852bfff700 1 -- 172.21.15.16:0/669664879 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8524011830 con 0x7f853409eca0 2023-09-30T22:33:32.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.863+0000 7f852bfff700 1 -- 172.21.15.16:0/669664879 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8524011990 con 0x7f853409eca0 2023-09-30T22:33:32.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.863+0000 7f852bfff700 1 -- 172.21.15.16:0/669664879 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8524011d20 con 0x7f853409eca0 2023-09-30T22:33:32.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.864+0000 7f853b3d8700 1 -- 172.21.15.16:0/669664879 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f853409eca0 msgr2=0x7f853406f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:32.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.864+0000 7f853b3d8700 1 --2- 172.21.15.16:0/669664879 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f853409eca0 0x7f853406f620 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f85240098f0 tx=0x7f8524009c00 comp rx=0 tx=0).stop 2023-09-30T22:33:32.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.864+0000 7f853b3d8700 1 -- 172.21.15.16:0/669664879 shutdown_connections 2023-09-30T22:33:32.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.864+0000 7f853b3d8700 1 --2- 172.21.15.16:0/669664879 >> v2:172.21.15.16:3300/0 conn(0x7f8534130560 0x7f8534130930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:32.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.864+0000 7f853b3d8700 1 --2- 172.21.15.16:0/669664879 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f853409eca0 0x7f853406f620 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:32.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.864+0000 7f853b3d8700 1 -- 172.21.15.16:0/669664879 >> 172.21.15.16:0/669664879 conn(0x7f85340a2aa0 msgr2=0x7f85340a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:32.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.864+0000 7f853b3d8700 1 -- 172.21.15.16:0/669664879 shutdown_connections 2023-09-30T22:33:32.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.864+0000 7f853b3d8700 1 -- 172.21.15.16:0/669664879 wait complete. 2023-09-30T22:33:32.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.865+0000 7f853b3d8700 1 Processor -- start 2023-09-30T22:33:32.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.865+0000 7f853b3d8700 1 -- start start 2023-09-30T22:33:32.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.866+0000 7f853b3d8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f853409eca0 0x7f8534132fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:32.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.866+0000 7f853b3d8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8534130560 0x7f85341334e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:32.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.866+0000 7f853b3d8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8534130e70 0x7f8534133a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:32.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.866+0000 7f853b3d8700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8534136740 con 0x7f8534130e70 2023-09-30T22:33:32.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.866+0000 7f853b3d8700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8534134c20 con 0x7f8534130560 2023-09-30T22:33:32.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.866+0000 7f853b3d8700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f85340719f0 con 0x7f853409eca0 2023-09-30T22:33:32.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.866+0000 7f8539174700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f853409eca0 0x7f8534132fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:32.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.866+0000 7f8539174700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f853409eca0 0x7f8534132fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:53734/0 (socket says 172.21.15.16:53734) 2023-09-30T22:33:32.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.866+0000 7f8539174700 1 -- 172.21.15.16:0/2321469415 learned_addr learned my addr 172.21.15.16:0/2321469415 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:32.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.866+0000 7f8538973700 1 --2- 172.21.15.16:0/2321469415 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8534130560 0x7f85341334e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:32.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.867+0000 7f8539975700 1 --2- 172.21.15.16:0/2321469415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8534130e70 0x7f8534133a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:32.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.867+0000 7f8538973700 1 -- 172.21.15.16:0/2321469415 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f853409eca0 msgr2=0x7f8534132fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:32.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.867+0000 7f8538973700 1 --2- 172.21.15.16:0/2321469415 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f853409eca0 0x7f8534132fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:32.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.867+0000 7f8538973700 1 -- 172.21.15.16:0/2321469415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8534130e70 msgr2=0x7f8534133a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:32.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.867+0000 7f8538973700 1 --2- 172.21.15.16:0/2321469415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8534130e70 0x7f8534133a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:32.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.867+0000 7f8538973700 1 -- 172.21.15.16:0/2321469415 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8524009550 con 0x7f8534130560 2023-09-30T22:33:32.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.867+0000 7f8539174700 1 --2- 172.21.15.16:0/2321469415 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f853409eca0 0x7f8534132fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:32.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.867+0000 7f8538973700 1 --2- 172.21.15.16:0/2321469415 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8534130560 0x7f85341334e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f8524016000 tx=0x7f8524004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:32.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.867+0000 7f8529ffb700 1 -- 172.21.15.16:0/2321469415 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85240036a0 con 0x7f8534130560 2023-09-30T22:33:32.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.867+0000 7f8529ffb700 1 -- 172.21.15.16:0/2321469415 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8524003800 con 0x7f8534130560 2023-09-30T22:33:32.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.867+0000 7f8529ffb700 1 -- 172.21.15.16:0/2321469415 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8524003bb0 con 0x7f8534130560 2023-09-30T22:33:32.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.867+0000 7f853b3d8700 1 -- 172.21.15.16:0/2321469415 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f853412e880 con 0x7f8534130560 2023-09-30T22:33:32.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.868+0000 7f853b3d8700 1 -- 172.21.15.16:0/2321469415 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8534135dc0 con 0x7f8534130560 2023-09-30T22:33:32.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.869+0000 7f8529ffb700 1 -- 172.21.15.16:0/2321469415 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8524003d10 con 0x7f8534130560 2023-09-30T22:33:32.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.869+0000 7f8529ffb700 1 -- 172.21.15.16:0/2321469415 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f8524021030 con 0x7f8534130560 2023-09-30T22:33:32.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.869+0000 7f853b3d8700 1 -- 172.21.15.16:0/2321469415 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8534074ea0 con 0x7f8534130560 2023-09-30T22:33:32.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.874+0000 7f8529ffb700 1 -- 172.21.15.16:0/2321469415 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8524015080 con 0x7f8534130560 2023-09-30T22:33:32.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.906+0000 7f7caba95700 1 Processor -- start 2023-09-30T22:33:32.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.906+0000 7f7caba95700 1 -- start start 2023-09-30T22:33:32.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.907+0000 7f7caba95700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7ca409e1d0 0x7f7ca4135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:32.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.907+0000 7f7caba95700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7ca4135e50 0x7f7ca4138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:32.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.907+0000 7f7caba95700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ca409f130 con 0x7f7ca409d6f0 2023-09-30T22:33:32.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.907+0000 7f7caba95700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ca409f520 con 0x7f7ca4135e50 2023-09-30T22:33:32.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.907+0000 7f7caba95700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca409e930 con 0x7f7ca409e1d0 2023-09-30T22:33:32.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.907+0000 7f7ca9030700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7ca409e1d0 0x7f7ca4135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:32.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.907+0000 7f7ca9030700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7ca409e1d0 0x7f7ca4135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:53746/0 (socket says 172.21.15.16:53746) 2023-09-30T22:33:32.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.907+0000 7f7ca9030700 1 -- 172.21.15.16:0/1402838438 learned_addr learned my addr 172.21.15.16:0/1402838438 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:32.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.908+0000 7f7caa032700 1 --2- 172.21.15.16:0/1402838438 >> v2:172.21.15.16:3300/0 conn(0x7f7ca4135e50 0x7f7ca4138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:32.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.908+0000 7f7ca9030700 1 -- 172.21.15.16:0/1402838438 >> v2:172.21.15.16:3300/0 conn(0x7f7ca4135e50 msgr2=0x7f7ca4138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:32.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.908+0000 7f7ca9030700 1 --2- 172.21.15.16:0/1402838438 >> v2:172.21.15.16:3300/0 conn(0x7f7ca4135e50 0x7f7ca4138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:32.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.908+0000 7f7ca9030700 1 -- 172.21.15.16:0/1402838438 >> v1:172.21.15.16:6789/0 conn(0x7f7ca409d6f0 legacy=0x7f7ca409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:32.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.908+0000 7f7ca9030700 1 -- 172.21.15.16:0/1402838438 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca409b2d0 con 0x7f7ca409e1d0 2023-09-30T22:33:32.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.908+0000 7f7caa032700 1 --2- 172.21.15.16:0/1402838438 >> v2:172.21.15.16:3300/0 conn(0x7f7ca4135e50 0x7f7ca4138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:33:32.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.908+0000 7f7ca9030700 1 --2- 172.21.15.16:0/1402838438 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7ca409e1d0 0x7f7ca4135910 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f7c94009920 tx=0x7f7c94009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9966af6fe3b1223 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:32.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.908+0000 7f7ca882f700 1 -- 172.21.15.16:0/1402838438 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9400ba20 con 0x7f7ca409e1d0 2023-09-30T22:33:32.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.908+0000 7f7ca882f700 1 -- 172.21.15.16:0/1402838438 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c9400bb80 con 0x7f7ca409e1d0 2023-09-30T22:33:32.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.908+0000 7f7ca882f700 1 -- 172.21.15.16:0/1402838438 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c94009e70 con 0x7f7ca409e1d0 2023-09-30T22:33:32.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.909+0000 7f7caba95700 1 -- 172.21.15.16:0/1402838438 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7ca409e1d0 msgr2=0x7f7ca4135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:32.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.909+0000 7f7caba95700 1 --2- 172.21.15.16:0/1402838438 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7ca409e1d0 0x7f7ca4135910 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f7c94009920 tx=0x7f7c94009c30 comp rx=0 tx=0).stop 2023-09-30T22:33:32.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.909+0000 7f7caba95700 1 -- 172.21.15.16:0/1402838438 shutdown_connections 2023-09-30T22:33:32.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.909+0000 7f7caba95700 1 --2- 172.21.15.16:0/1402838438 >> v2:172.21.15.16:3300/0 conn(0x7f7ca4135e50 0x7f7ca4138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:32.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.909+0000 7f7caba95700 1 --2- 172.21.15.16:0/1402838438 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7ca409e1d0 0x7f7ca4135910 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:32.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.909+0000 7f7caba95700 1 -- 172.21.15.16:0/1402838438 >> 172.21.15.16:0/1402838438 conn(0x7f7ca4066f50 msgr2=0x7f7ca409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:32.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.909+0000 7f7caba95700 1 -- 172.21.15.16:0/1402838438 shutdown_connections 2023-09-30T22:33:32.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.909+0000 7f7caba95700 1 -- 172.21.15.16:0/1402838438 wait complete. 2023-09-30T22:33:32.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.911+0000 7f7caba95700 1 Processor -- start 2023-09-30T22:33:32.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.911+0000 7f7caba95700 1 -- start start 2023-09-30T22:33:32.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.912+0000 7f7caba95700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7ca409d6f0 0x7f7ca412e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:32.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.912+0000 7f7caba95700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7ca409e1d0 0x7f7ca412ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:32.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.912+0000 7f7caba95700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ca4135e50 0x7f7ca412f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:32.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.912+0000 7f7caba95700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca4130b90 con 0x7f7ca4135e50 2023-09-30T22:33:32.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.912+0000 7f7caba95700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca41302c0 con 0x7f7ca409e1d0 2023-09-30T22:33:32.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.912+0000 7f7caba95700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca4131f90 con 0x7f7ca409d6f0 2023-09-30T22:33:32.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.912+0000 7f7ca9831700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7ca409d6f0 0x7f7ca412e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:32.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.912+0000 7f7ca9831700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7ca409d6f0 0x7f7ca412e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:53754/0 (socket says 172.21.15.16:53754) 2023-09-30T22:33:32.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.912+0000 7f7ca9831700 1 -- 172.21.15.16:0/3480874718 learned_addr learned my addr 172.21.15.16:0/3480874718 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:32.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.912+0000 7f7caa032700 1 --2- 172.21.15.16:0/3480874718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ca4135e50 0x7f7ca412f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:32.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.912+0000 7f7ca9030700 1 --2- 172.21.15.16:0/3480874718 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7ca409e1d0 0x7f7ca412ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:32.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.912+0000 7f7ca9831700 1 -- 172.21.15.16:0/3480874718 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7ca409e1d0 msgr2=0x7f7ca412ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:32.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.912+0000 7f7ca9831700 1 --2- 172.21.15.16:0/3480874718 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7ca409e1d0 0x7f7ca412ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:32.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.912+0000 7f7ca9831700 1 -- 172.21.15.16:0/3480874718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ca4135e50 msgr2=0x7f7ca412f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:32.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.912+0000 7f7ca9831700 1 --2- 172.21.15.16:0/3480874718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ca4135e50 0x7f7ca412f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:32.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.912+0000 7f7ca9831700 1 -- 172.21.15.16:0/3480874718 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c94009550 con 0x7f7ca409d6f0 2023-09-30T22:33:32.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.913+0000 7f7ca9831700 1 --2- 172.21.15.16:0/3480874718 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7ca409d6f0 0x7f7ca412e7e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f7ca000c510 tx=0x7f7ca000c820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:32.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.913+0000 7f7c9a7fc700 1 -- 172.21.15.16:0/3480874718 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca00052d0 con 0x7f7ca409d6f0 2023-09-30T22:33:32.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.913+0000 7f7c9a7fc700 1 -- 172.21.15.16:0/3480874718 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ca0005430 con 0x7f7ca409d6f0 2023-09-30T22:33:32.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.913+0000 7f7c9a7fc700 1 -- 172.21.15.16:0/3480874718 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca00038c0 con 0x7f7ca409d6f0 2023-09-30T22:33:32.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.914+0000 7f7caba95700 1 -- 172.21.15.16:0/3480874718 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ca409c700 con 0x7f7ca409d6f0 2023-09-30T22:33:32.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.914+0000 7f7caba95700 1 -- 172.21.15.16:0/3480874718 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ca4131510 con 0x7f7ca409d6f0 2023-09-30T22:33:32.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.915+0000 7f7c9a7fc700 1 -- 172.21.15.16:0/3480874718 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7ca0003a20 con 0x7f7ca409d6f0 2023-09-30T22:33:32.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.915+0000 7f7caba95700 1 -- 172.21.15.16:0/3480874718 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ca4074ea0 con 0x7f7ca409d6f0 2023-09-30T22:33:32.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.915+0000 7f7c9a7fc700 1 -- 172.21.15.16:0/3480874718 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f7ca003ea90 con 0x7f7ca409d6f0 2023-09-30T22:33:32.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.924+0000 7f7c9a7fc700 1 -- 172.21.15.16:0/3480874718 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7ca0011590 con 0x7f7ca409d6f0 2023-09-30T22:33:32.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.960+0000 7f5a20993700 1 Processor -- start 2023-09-30T22:33:32.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.961+0000 7f5a20993700 1 -- start start 2023-09-30T22:33:32.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.961+0000 7f5a20993700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a1c133420 0x7f5a1c1337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:32.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.961+0000 7f5a20993700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5a1c133e30 0x7f5a1c12ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:32.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.961+0000 7f5a20993700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a1c06b990 con 0x7f5a1c134260 2023-09-30T22:33:32.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.961+0000 7f5a20993700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a1c0a0590 con 0x7f5a1c133e30 2023-09-30T22:33:32.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.961+0000 7f5a20993700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1c09fb80 con 0x7f5a1c133420 2023-09-30T22:33:32.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.961+0000 7f5a1ad9d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5a1c134260 0x7f5a1c1304d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35090/0 (socket says 172.21.15.16:35090) 2023-09-30T22:33:32.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.961+0000 7f5a1ad9d700 1 -- 172.21.15.16:0/913378436 learned_addr learned my addr 172.21.15.16:0/913378436 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:32.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.961+0000 7f5a19d9b700 1 --2- 172.21.15.16:0/913378436 >> v2:172.21.15.16:3300/0 conn(0x7f5a1c133e30 0x7f5a1c12ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:32.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.962+0000 7f5a1a59c700 1 --2- 172.21.15.16:0/913378436 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a1c133420 0x7f5a1c1337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:32.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.962+0000 7f5a19d9b700 1 -- 172.21.15.16:0/913378436 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a1c133420 msgr2=0x7f5a1c1337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:32.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.962+0000 7f5a19d9b700 1 --2- 172.21.15.16:0/913378436 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a1c133420 0x7f5a1c1337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:32.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.962+0000 7f5a19d9b700 1 -- 172.21.15.16:0/913378436 >> v1:172.21.15.16:6789/0 conn(0x7f5a1c134260 legacy=0x7f5a1c1304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:32.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.962+0000 7f5a19d9b700 1 -- 172.21.15.16:0/913378436 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a1c0a0030 con 0x7f5a1c133e30 2023-09-30T22:33:32.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.962+0000 7f5a1a59c700 1 --2- 172.21.15.16:0/913378436 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a1c133420 0x7f5a1c1337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:33:32.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.962+0000 7f5a19d9b700 1 --2- 172.21.15.16:0/913378436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a1c133e30 0x7f5a1c12ff90 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f5a10009840 tx=0x7f5a10009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3412e84bba56c2af server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:32.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.962+0000 7f5a1959a700 1 -- 172.21.15.16:0/913378436 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a10011830 con 0x7f5a1c133e30 2023-09-30T22:33:32.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.962+0000 7f5a1959a700 1 -- 172.21.15.16:0/913378436 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a10011990 con 0x7f5a1c133e30 2023-09-30T22:33:32.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.962+0000 7f5a1959a700 1 -- 172.21.15.16:0/913378436 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a10011d20 con 0x7f5a1c133e30 2023-09-30T22:33:32.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.963+0000 7f5a20993700 1 -- 172.21.15.16:0/913378436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a1c133e30 msgr2=0x7f5a1c12ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:32.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.963+0000 7f5a20993700 1 --2- 172.21.15.16:0/913378436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a1c133e30 0x7f5a1c12ff90 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f5a10009840 tx=0x7f5a10009b50 comp rx=0 tx=0).stop 2023-09-30T22:33:32.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.963+0000 7f5a20993700 1 -- 172.21.15.16:0/913378436 shutdown_connections 2023-09-30T22:33:32.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.963+0000 7f5a20993700 1 --2- 172.21.15.16:0/913378436 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a1c133420 0x7f5a1c1337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:32.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.963+0000 7f5a20993700 1 --2- 172.21.15.16:0/913378436 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a1c133e30 0x7f5a1c12ff90 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:32.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.963+0000 7f5a20993700 1 -- 172.21.15.16:0/913378436 >> 172.21.15.16:0/913378436 conn(0x7f5a1c0a4210 msgr2=0x7f5a1c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:32.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.963+0000 7f5a20993700 1 -- 172.21.15.16:0/913378436 shutdown_connections 2023-09-30T22:33:32.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.963+0000 7f5a20993700 1 -- 172.21.15.16:0/913378436 wait complete. 2023-09-30T22:33:32.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.965+0000 7f5a20993700 1 Processor -- start 2023-09-30T22:33:32.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.965+0000 7f5a20993700 1 -- start start 2023-09-30T22:33:32.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.965+0000 7f5a20993700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a1c133420 0x7f5a1c09c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:32.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.965+0000 7f5a20993700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a1c133e30 0x7f5a1c09ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:32.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.965+0000 7f5a20993700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5a1c134260 0x7f5a1c09d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:32.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.965+0000 7f5a20993700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1c09e2b0 con 0x7f5a1c133420 2023-09-30T22:33:32.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.965+0000 7f5a20993700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1c120130 con 0x7f5a1c134260 2023-09-30T22:33:32.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.965+0000 7f5a20993700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1c12db60 con 0x7f5a1c133e30 2023-09-30T22:33:32.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.965+0000 7f5a1a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a1c133420 0x7f5a1c09c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:32.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.965+0000 7f5a1a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a1c133420 0x7f5a1c09c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55380/0 (socket says 172.21.15.16:55380) 2023-09-30T22:33:32.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.965+0000 7f5a1a59c700 1 -- 172.21.15.16:0/3525628985 learned_addr learned my addr 172.21.15.16:0/3525628985 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:32.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.965+0000 7f5a1ad9d700 1 --2- 172.21.15.16:0/3525628985 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5a1c134260 0x7f5a1c09d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:32.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.966+0000 7f5a19d9b700 1 --2- 172.21.15.16:0/3525628985 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a1c133e30 0x7f5a1c09ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:32.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.966+0000 7f5a1a59c700 1 -- 172.21.15.16:0/3525628985 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a1c133e30 msgr2=0x7f5a1c09ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:32.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.966+0000 7f5a1a59c700 1 --2- 172.21.15.16:0/3525628985 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a1c133e30 0x7f5a1c09ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:32.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.966+0000 7f5a1a59c700 1 -- 172.21.15.16:0/3525628985 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5a1c134260 msgr2=0x7f5a1c09d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:32.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.966+0000 7f5a1a59c700 1 --2- 172.21.15.16:0/3525628985 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5a1c134260 0x7f5a1c09d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:32.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.966+0000 7f5a1a59c700 1 -- 172.21.15.16:0/3525628985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a100094a0 con 0x7f5a1c133420 2023-09-30T22:33:32.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.966+0000 7f5a19d9b700 1 --2- 172.21.15.16:0/3525628985 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a1c133e30 0x7f5a1c09ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:33:32.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.966+0000 7f5a1a59c700 1 --2- 172.21.15.16:0/3525628985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a1c133420 0x7f5a1c09c8c0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c00e810 tx=0x7f5a0c00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:32.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.966+0000 7f5a0affd700 1 -- 172.21.15.16:0/3525628985 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c013610 con 0x7f5a1c133420 2023-09-30T22:33:32.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.966+0000 7f5a0affd700 1 -- 172.21.15.16:0/3525628985 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a0c013770 con 0x7f5a1c133420 2023-09-30T22:33:32.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.966+0000 7f5a20993700 1 -- 172.21.15.16:0/3525628985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a1c120ba0 con 0x7f5a1c133420 2023-09-30T22:33:32.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.966+0000 7f5a0affd700 1 -- 172.21.15.16:0/3525628985 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c013b00 con 0x7f5a1c133420 2023-09-30T22:33:32.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.966+0000 7f5a20993700 1 -- 172.21.15.16:0/3525628985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a1c11ea90 con 0x7f5a1c133420 2023-09-30T22:33:32.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.967+0000 7f5a0affd700 1 -- 172.21.15.16:0/3525628985 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5a0c013c60 con 0x7f5a1c133420 2023-09-30T22:33:32.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.967+0000 7f5a0affd700 1 -- 172.21.15.16:0/3525628985 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f5a0c03f480 con 0x7f5a1c133420 2023-09-30T22:33:32.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.968+0000 7f5a20993700 1 -- 172.21.15.16:0/3525628985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a1c11ed10 con 0x7f5a1c133420 2023-09-30T22:33:32.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:32.977+0000 7f5a0affd700 1 -- 172.21.15.16:0/3525628985 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5a0c021080 con 0x7f5a1c133420 2023-09-30T22:33:33.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:33.058+0000 7fb09b757700 1 -- 172.21.15.16:0/808218455 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb074005590 con 0x7fb09406a770 2023-09-30T22:33:33.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:33.058+0000 7fb089ffb700 1 -- 172.21.15.16:0/808218455 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb09002b070 con 0x7fb09406a770 2023-09-30T22:33:33.060 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:33:33.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:33.061+0000 7fb09b757700 1 -- 172.21.15.16:0/808218455 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb09406a770 msgr2=0x7fb0941c8270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:33.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:33.061+0000 7fb09b757700 1 --2- 172.21.15.16:0/808218455 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb09406a770 0x7fb0941c8270 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fb090000c00 tx=0x7fb090004750 comp rx=0 tx=0).stop 2023-09-30T22:33:33.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:33.061+0000 7fb09b757700 1 -- 172.21.15.16:0/808218455 shutdown_connections 2023-09-30T22:33:33.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:33.061+0000 7fb09b757700 1 --2- 172.21.15.16:0/808218455 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb09406a770 0x7fb0941c8270 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:33.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:33.061+0000 7fb09b757700 1 --2- 172.21.15.16:0/808218455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb094131b40 0x7fb0941c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:33.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:33.061+0000 7fb09b757700 1 --2- 172.21.15.16:0/808218455 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb09406b080 0x7fb0941c87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:33.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:33.061+0000 7fb09b757700 1 -- 172.21.15.16:0/808218455 >> 172.21.15.16:0/808218455 conn(0x7fb094066f50 msgr2=0x7fb0940a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:33.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:33.062+0000 7fb09b757700 1 -- 172.21.15.16:0/808218455 shutdown_connections 2023-09-30T22:33:33.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:33.062+0000 7fb09b757700 1 -- 172.21.15.16:0/808218455 wait complete. 2023-09-30T22:33:33.077 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:33:33.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:33.088+0000 7f853b3d8700 1 -- 172.21.15.16:0/2321469415 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f853406b9e0 con 0x7f8534130560 2023-09-30T22:33:33.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:33.088+0000 7f8529ffb700 1 -- 172.21.15.16:0/2321469415 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f852402b070 con 0x7f8534130560 2023-09-30T22:33:33.089 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:33:33.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:33.090+0000 7f853b3d8700 1 -- 172.21.15.16:0/2321469415 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8534130560 msgr2=0x7f85341334e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:33.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:33.090+0000 7f853b3d8700 1 --2- 172.21.15.16:0/2321469415 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8534130560 0x7f85341334e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f8524016000 tx=0x7f8524004750 comp rx=0 tx=0).stop 2023-09-30T22:33:33.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:33.090+0000 7f853b3d8700 1 -- 172.21.15.16:0/2321469415 shutdown_connections 2023-09-30T22:33:33.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:33.090+0000 7f853b3d8700 1 --2- 172.21.15.16:0/2321469415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8534130e70 0x7f8534133a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:33.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:33.090+0000 7f853b3d8700 1 --2- 172.21.15.16:0/2321469415 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f853409eca0 0x7f8534132fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:33.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:33.090+0000 7f853b3d8700 1 --2- 172.21.15.16:0/2321469415 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8534130560 0x7f85341334e0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:33.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:33.090+0000 7f853b3d8700 1 -- 172.21.15.16:0/2321469415 >> 172.21.15.16:0/2321469415 conn(0x7f85340a2aa0 msgr2=0x7f853406e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:33.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:33.090+0000 7f853b3d8700 1 -- 172.21.15.16:0/2321469415 shutdown_connections 2023-09-30T22:33:33.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:33.090+0000 7f853b3d8700 1 -- 172.21.15.16:0/2321469415 wait complete. 2023-09-30T22:33:33.105 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:33:33.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:33.167+0000 7f7caba95700 1 -- 172.21.15.16:0/3480874718 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7ca406c360 con 0x7f7ca409d6f0 2023-09-30T22:33:33.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:33.167+0000 7f7c9a7fc700 1 -- 172.21.15.16:0/3480874718 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7ca0017030 con 0x7f7ca409d6f0 2023-09-30T22:33:33.169 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:33:33.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:33.170+0000 7f7caba95700 1 -- 172.21.15.16:0/3480874718 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7ca409d6f0 msgr2=0x7f7ca412e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:33.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:33.170+0000 7f7caba95700 1 --2- 172.21.15.16:0/3480874718 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7ca409d6f0 0x7f7ca412e7e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f7ca000c510 tx=0x7f7ca000c820 comp rx=0 tx=0).stop 2023-09-30T22:33:33.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:33.170+0000 7f7caba95700 1 -- 172.21.15.16:0/3480874718 shutdown_connections 2023-09-30T22:33:33.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:33.170+0000 7f7caba95700 1 --2- 172.21.15.16:0/3480874718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ca4135e50 0x7f7ca412f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:33.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:33.170+0000 7f7caba95700 1 --2- 172.21.15.16:0/3480874718 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7ca409d6f0 0x7f7ca412e7e0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:33.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:33.170+0000 7f7caba95700 1 --2- 172.21.15.16:0/3480874718 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7ca409e1d0 0x7f7ca412ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:33.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:33.170+0000 7f7caba95700 1 -- 172.21.15.16:0/3480874718 >> 172.21.15.16:0/3480874718 conn(0x7f7ca4066f50 msgr2=0x7f7ca40a1580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:33.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:33.170+0000 7f7caba95700 1 -- 172.21.15.16:0/3480874718 shutdown_connections 2023-09-30T22:33:33.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:33.170+0000 7f7caba95700 1 -- 172.21.15.16:0/3480874718 wait complete. 2023-09-30T22:33:33.185 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:33:33.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:33.205+0000 7f5a20993700 1 -- 172.21.15.16:0/3525628985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5a1c133a00 con 0x7f5a1c133420 2023-09-30T22:33:33.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:33.205+0000 7f5a0affd700 1 -- 172.21.15.16:0/3525628985 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5a0c012370 con 0x7f5a1c133420 2023-09-30T22:33:33.207 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:33:33.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:33.207+0000 7f5a20993700 1 -- 172.21.15.16:0/3525628985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a1c133420 msgr2=0x7f5a1c09c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:33.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:33.207+0000 7f5a20993700 1 --2- 172.21.15.16:0/3525628985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a1c133420 0x7f5a1c09c8c0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c00e810 tx=0x7f5a0c00eb20 comp rx=0 tx=0).stop 2023-09-30T22:33:33.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:33.207+0000 7f5a20993700 1 -- 172.21.15.16:0/3525628985 shutdown_connections 2023-09-30T22:33:33.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:33.207+0000 7f5a20993700 1 --2- 172.21.15.16:0/3525628985 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5a1c134260 0x7f5a1c09d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:33.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:33.207+0000 7f5a20993700 1 --2- 172.21.15.16:0/3525628985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a1c133420 0x7f5a1c09c8c0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:33.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:33.207+0000 7f5a20993700 1 --2- 172.21.15.16:0/3525628985 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a1c133e30 0x7f5a1c09ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:33.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:33.207+0000 7f5a20993700 1 -- 172.21.15.16:0/3525628985 >> 172.21.15.16:0/3525628985 conn(0x7f5a1c0a4210 msgr2=0x7f5a1c0a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:33.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:33.208+0000 7f5a20993700 1 -- 172.21.15.16:0/3525628985 shutdown_connections 2023-09-30T22:33:33.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:33.208+0000 7f5a20993700 1 -- 172.21.15.16:0/3525628985 wait complete. 2023-09-30T22:33:33.222 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:33:34.078 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:33:34.107 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:33:34.186 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:33:34.224 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:33:34.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.230+0000 7f44a1819700 1 Processor -- start 2023-09-30T22:33:34.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.230+0000 7f44a1819700 1 -- start start 2023-09-30T22:33:34.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.231+0000 7f44a1819700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f449c134630 0x7f449c136a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:34.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.231+0000 7f44a1819700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f449c136f50 0x7f449c139330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:34.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.231+0000 7f44a1819700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f449c09b4f0 con 0x7f449c131810 2023-09-30T22:33:34.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.231+0000 7f44a1819700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f449c1313c0 con 0x7f449c134630 2023-09-30T22:33:34.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.231+0000 7f44a1819700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f449c06e8c0 con 0x7f449c136f50 2023-09-30T22:33:34.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.231+0000 7f449affd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f449c131810 0x7f449c131be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35100/0 (socket says 172.21.15.16:35100) 2023-09-30T22:33:34.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.231+0000 7f449affd700 1 -- 172.21.15.16:0/2867418047 learned_addr learned my addr 172.21.15.16:0/2867418047 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:34.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.231+0000 7f449a7fc700 1 --2- 172.21.15.16:0/2867418047 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f449c134630 0x7f449c136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:34.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.231+0000 7f449b7fe700 1 --2- 172.21.15.16:0/2867418047 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f449c136f50 0x7f449c139330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:34.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.231+0000 7f4499ffb700 1 -- 172.21.15.16:0/2867418047 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2227017595 0 0) 0x7f449c09b4f0 con 0x7f449c131810 2023-09-30T22:33:34.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.231+0000 7f4499ffb700 1 -- 172.21.15.16:0/2867418047 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44840035b0 con 0x7f449c131810 2023-09-30T22:33:34.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.232+0000 7f449a7fc700 1 -- 172.21.15.16:0/2867418047 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f449c136f50 msgr2=0x7f449c139330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:34.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.232+0000 7f449a7fc700 1 --2- 172.21.15.16:0/2867418047 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f449c136f50 0x7f449c139330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:34.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.232+0000 7f449a7fc700 1 -- 172.21.15.16:0/2867418047 >> v1:172.21.15.16:6789/0 conn(0x7f449c131810 legacy=0x7f449c131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:34.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.232+0000 7f449a7fc700 1 -- 172.21.15.16:0/2867418047 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f449c071d20 con 0x7f449c134630 2023-09-30T22:33:34.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.232+0000 7f449a7fc700 1 --2- 172.21.15.16:0/2867418047 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f449c134630 0x7f449c136a10 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f4490009870 tx=0x7f4490009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b90f2e58434018b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:34.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.232+0000 7f4499ffb700 1 -- 172.21.15.16:0/2867418047 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4490011830 con 0x7f449c134630 2023-09-30T22:33:34.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.232+0000 7f4499ffb700 1 -- 172.21.15.16:0/2867418047 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4490011990 con 0x7f449c134630 2023-09-30T22:33:34.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.232+0000 7f4499ffb700 1 -- 172.21.15.16:0/2867418047 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4490011d20 con 0x7f449c134630 2023-09-30T22:33:34.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.233+0000 7f44a1819700 1 -- 172.21.15.16:0/2867418047 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f449c134630 msgr2=0x7f449c136a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:34.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.233+0000 7f44a1819700 1 --2- 172.21.15.16:0/2867418047 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f449c134630 0x7f449c136a10 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f4490009870 tx=0x7f4490009b80 comp rx=0 tx=0).stop 2023-09-30T22:33:34.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.233+0000 7f44a1819700 1 -- 172.21.15.16:0/2867418047 shutdown_connections 2023-09-30T22:33:34.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.233+0000 7f44a1819700 1 --2- 172.21.15.16:0/2867418047 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f449c136f50 0x7f449c139330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:34.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.233+0000 7f44a1819700 1 --2- 172.21.15.16:0/2867418047 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f449c134630 0x7f449c136a10 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:34.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.233+0000 7f44a1819700 1 -- 172.21.15.16:0/2867418047 >> 172.21.15.16:0/2867418047 conn(0x7f449c0a2aa0 msgr2=0x7f449c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:34.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.233+0000 7f44a1819700 1 -- 172.21.15.16:0/2867418047 shutdown_connections 2023-09-30T22:33:34.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.233+0000 7f44a1819700 1 -- 172.21.15.16:0/2867418047 wait complete. 2023-09-30T22:33:34.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.234+0000 7f44a1819700 1 Processor -- start 2023-09-30T22:33:34.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.234+0000 7f44a1819700 1 -- start start 2023-09-30T22:33:34.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.235+0000 7f44a1819700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f449c131810 0x7f449c118cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:34.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.235+0000 7f44a1819700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f449c134630 0x7f449c119210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:34.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.235+0000 7f44a1819700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f449c136f50 0x7f449c119750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:34.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.235+0000 7f44a1819700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f449c12e560 con 0x7f449c131810 2023-09-30T22:33:34.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.235+0000 7f44a1819700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f449c06eb90 con 0x7f449c134630 2023-09-30T22:33:34.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.235+0000 7f44a1819700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f449c11a950 con 0x7f449c136f50 2023-09-30T22:33:34.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.235+0000 7f449affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f449c131810 0x7f449c118cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:34.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.235+0000 7f449affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f449c131810 0x7f449c118cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55388/0 (socket says 172.21.15.16:55388) 2023-09-30T22:33:34.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.235+0000 7f449affd700 1 -- 172.21.15.16:0/2012639064 learned_addr learned my addr 172.21.15.16:0/2012639064 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:34.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.236+0000 7f449b7fe700 1 --2- 172.21.15.16:0/2012639064 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f449c136f50 0x7f449c119750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:34.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.236+0000 7f449a7fc700 1 --2- 172.21.15.16:0/2012639064 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f449c134630 0x7f449c119210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:34.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.236+0000 7f449affd700 1 -- 172.21.15.16:0/2012639064 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f449c136f50 msgr2=0x7f449c119750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:34.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.236+0000 7f449affd700 1 --2- 172.21.15.16:0/2012639064 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f449c136f50 0x7f449c119750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:34.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.236+0000 7f449affd700 1 -- 172.21.15.16:0/2012639064 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f449c134630 msgr2=0x7f449c119210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:34.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.236+0000 7f449affd700 1 --2- 172.21.15.16:0/2012639064 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f449c134630 0x7f449c119210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:34.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.236+0000 7f449affd700 1 -- 172.21.15.16:0/2012639064 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44900094a0 con 0x7f449c131810 2023-09-30T22:33:34.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.236+0000 7f449affd700 1 --2- 172.21.15.16:0/2012639064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f449c131810 0x7f449c118cd0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f448c00c730 tx=0x7f448c00ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:34.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.236+0000 7f4483fff700 1 -- 172.21.15.16:0/2012639064 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448c005d70 con 0x7f449c131810 2023-09-30T22:33:34.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.236+0000 7f4483fff700 1 -- 172.21.15.16:0/2012639064 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f448c0070d0 con 0x7f449c131810 2023-09-30T22:33:34.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.236+0000 7f44a1819700 1 -- 172.21.15.16:0/2012639064 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f449c0a1e60 con 0x7f449c131810 2023-09-30T22:33:34.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.236+0000 7f4483fff700 1 -- 172.21.15.16:0/2012639064 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448c007460 con 0x7f449c131810 2023-09-30T22:33:34.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.236+0000 7f44a1819700 1 -- 172.21.15.16:0/2012639064 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f449c11c4d0 con 0x7f449c131810 2023-09-30T22:33:34.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.238+0000 7f44a1819700 1 -- 172.21.15.16:0/2012639064 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f449c11c110 con 0x7f449c131810 2023-09-30T22:33:34.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.238+0000 7f4483fff700 1 -- 172.21.15.16:0/2012639064 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f448c0075c0 con 0x7f449c131810 2023-09-30T22:33:34.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.238+0000 7f4483fff700 1 -- 172.21.15.16:0/2012639064 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f448c03ec10 con 0x7f449c131810 2023-09-30T22:33:34.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.244+0000 7f4483fff700 1 -- 172.21.15.16:0/2012639064 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f449c11c110 con 0x7f449c131810 2023-09-30T22:33:34.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.276+0000 7ff9cbf20700 1 Processor -- start 2023-09-30T22:33:34.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.276+0000 7ff9cbf20700 1 -- start start 2023-09-30T22:33:34.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.276+0000 7ff9cbf20700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff9c406e570 0x7ff9c406e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:34.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.276+0000 7ff9cbf20700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff9c409eca0 0x7ff9c406ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:34.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.276+0000 7ff9cbf20700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9c4070ee0 con 0x7ff9c4134610 2023-09-30T22:33:34.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.276+0000 7ff9cbf20700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9c4071480 con 0x7ff9c406e570 2023-09-30T22:33:34.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.276+0000 7ff9cbf20700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c406f3c0 con 0x7ff9c409eca0 2023-09-30T22:33:34.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.277+0000 7ff9c9cbc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff9c406e570 0x7ff9c406e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:34.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.277+0000 7ff9ca4bd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff9c4134610 0x7ff9c4134a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35110/0 (socket says 172.21.15.16:35110) 2023-09-30T22:33:34.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.277+0000 7ff9c9cbc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff9c406e570 0x7ff9c406e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55392/0 (socket says 172.21.15.16:55392) 2023-09-30T22:33:34.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.277+0000 7ff9ca4bd700 1 -- 172.21.15.16:0/3867933600 learned_addr learned my addr 172.21.15.16:0/3867933600 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:34.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.277+0000 7ff9c94bb700 1 --2- 172.21.15.16:0/3867933600 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff9c409eca0 0x7ff9c406ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:34.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.277+0000 7ff9c8cba700 1 -- 172.21.15.16:0/3867933600 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2688792267 0 0) 0x7ff9c4070ee0 con 0x7ff9c4134610 2023-09-30T22:33:34.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.277+0000 7ff9c8cba700 1 -- 172.21.15.16:0/3867933600 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9b00035b0 con 0x7ff9c4134610 2023-09-30T22:33:34.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.277+0000 7ff9c9cbc700 1 -- 172.21.15.16:0/3867933600 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff9c409eca0 msgr2=0x7ff9c406ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:34.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.277+0000 7ff9c9cbc700 1 --2- 172.21.15.16:0/3867933600 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff9c409eca0 0x7ff9c406ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:34.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.277+0000 7ff9c9cbc700 1 -- 172.21.15.16:0/3867933600 >> v1:172.21.15.16:6789/0 conn(0x7ff9c4134610 legacy=0x7ff9c4134a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:34.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.277+0000 7ff9c9cbc700 1 -- 172.21.15.16:0/3867933600 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9c4071b90 con 0x7ff9c406e570 2023-09-30T22:33:34.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.277+0000 7ff9c94bb700 1 --2- 172.21.15.16:0/3867933600 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff9c409eca0 0x7ff9c406ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:34.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.278+0000 7ff9c9cbc700 1 --2- 172.21.15.16:0/3867933600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9c406e570 0x7ff9c406e940 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0009840 tx=0x7ff9c0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e22e911835b8f314 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:34.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.278+0000 7ff9c8cba700 1 -- 172.21.15.16:0/3867933600 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c0011830 con 0x7ff9c406e570 2023-09-30T22:33:34.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.278+0000 7ff9c8cba700 1 -- 172.21.15.16:0/3867933600 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9c0011990 con 0x7ff9c406e570 2023-09-30T22:33:34.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.278+0000 7ff9c8cba700 1 -- 172.21.15.16:0/3867933600 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c0011d20 con 0x7ff9c406e570 2023-09-30T22:33:34.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.278+0000 7ff9cbf20700 1 -- 172.21.15.16:0/3867933600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9c406e570 msgr2=0x7ff9c406e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:34.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.278+0000 7ff9cbf20700 1 --2- 172.21.15.16:0/3867933600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9c406e570 0x7ff9c406e940 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0009840 tx=0x7ff9c0009b50 comp rx=0 tx=0).stop 2023-09-30T22:33:34.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.279+0000 7ff9cbf20700 1 -- 172.21.15.16:0/3867933600 shutdown_connections 2023-09-30T22:33:34.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.279+0000 7ff9cbf20700 1 --2- 172.21.15.16:0/3867933600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9c406e570 0x7ff9c406e940 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:34.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.279+0000 7ff9cbf20700 1 --2- 172.21.15.16:0/3867933600 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff9c409eca0 0x7ff9c406ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:34.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.279+0000 7ff9cbf20700 1 -- 172.21.15.16:0/3867933600 >> 172.21.15.16:0/3867933600 conn(0x7ff9c40a2aa0 msgr2=0x7ff9c40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:34.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.279+0000 7ff9cbf20700 1 -- 172.21.15.16:0/3867933600 shutdown_connections 2023-09-30T22:33:34.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.279+0000 7ff9cbf20700 1 -- 172.21.15.16:0/3867933600 wait complete. 2023-09-30T22:33:34.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.280+0000 7ff9cbf20700 1 Processor -- start 2023-09-30T22:33:34.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.280+0000 7ff9cbf20700 1 -- start start 2023-09-30T22:33:34.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.280+0000 7ff9cbf20700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff9c406e570 0x7ff9c413e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:34.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.280+0000 7ff9cbf20700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9c4134610 0x7ff9c413ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:34.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.280+0000 7ff9cbf20700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff9c413e020 0x7ff9c413f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:34.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.280+0000 7ff9cbf20700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c413c500 con 0x7ff9c4134610 2023-09-30T22:33:34.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.280+0000 7ff9cbf20700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c413d580 con 0x7ff9c413e020 2023-09-30T22:33:34.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.280+0000 7ff9cbf20700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c40a1d80 con 0x7ff9c406e570 2023-09-30T22:33:34.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.280+0000 7ff9c94bb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9c4134610 0x7ff9c413ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:34.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.280+0000 7ff9c94bb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9c4134610 0x7ff9c413ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55394/0 (socket says 172.21.15.16:55394) 2023-09-30T22:33:34.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.280+0000 7ff9c94bb700 1 -- 172.21.15.16:0/3271396798 learned_addr learned my addr 172.21.15.16:0/3271396798 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:34.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.280+0000 7ff9c9cbc700 1 --2- 172.21.15.16:0/3271396798 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff9c406e570 0x7ff9c413e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:34.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.281+0000 7ff9ca4bd700 1 --2- 172.21.15.16:0/3271396798 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff9c413e020 0x7ff9c413f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:34.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.281+0000 7ff9c94bb700 1 -- 172.21.15.16:0/3271396798 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff9c406e570 msgr2=0x7ff9c413e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:34.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.281+0000 7ff9c94bb700 1 --2- 172.21.15.16:0/3271396798 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff9c406e570 0x7ff9c413e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:34.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.281+0000 7ff9c94bb700 1 -- 172.21.15.16:0/3271396798 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff9c413e020 msgr2=0x7ff9c413f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:34.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.281+0000 7ff9c94bb700 1 --2- 172.21.15.16:0/3271396798 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff9c413e020 0x7ff9c413f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:34.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.281+0000 7ff9c94bb700 1 -- 172.21.15.16:0/3271396798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b40094a0 con 0x7ff9c4134610 2023-09-30T22:33:34.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.281+0000 7ff9c94bb700 1 --2- 172.21.15.16:0/3271396798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9c4134610 0x7ff9c413ec40 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7ff9b400ebe0 tx=0x7ff9b400eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:34.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.281+0000 7ff9ba7fc700 1 -- 172.21.15.16:0/3271396798 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b400c850 con 0x7ff9c4134610 2023-09-30T22:33:34.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.281+0000 7ff9ba7fc700 1 -- 172.21.15.16:0/3271396798 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9b400c9b0 con 0x7ff9c4134610 2023-09-30T22:33:34.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.281+0000 7ff9cbf20700 1 -- 172.21.15.16:0/3271396798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9c00094a0 con 0x7ff9c4134610 2023-09-30T22:33:34.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.281+0000 7ff9ba7fc700 1 -- 172.21.15.16:0/3271396798 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b400cd40 con 0x7ff9c4134610 2023-09-30T22:33:34.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.282+0000 7ff9cbf20700 1 -- 172.21.15.16:0/3271396798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9c413d2b0 con 0x7ff9c4134610 2023-09-30T22:33:34.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.283+0000 7ff9ba7fc700 1 -- 172.21.15.16:0/3271396798 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff9b4013470 con 0x7ff9c4134610 2023-09-30T22:33:34.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.283+0000 7ff9ba7fc700 1 -- 172.21.15.16:0/3271396798 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7ff9b403fb00 con 0x7ff9c4134610 2023-09-30T22:33:34.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.283+0000 7ff9cbf20700 1 -- 172.21.15.16:0/3271396798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9c4074ea0 con 0x7ff9c4134610 2023-09-30T22:33:34.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.293+0000 7ff9ba7fc700 1 -- 172.21.15.16:0/3271396798 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff9b4013720 con 0x7ff9c4134610 2023-09-30T22:33:34.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.343+0000 7f3e001d5700 1 Processor -- start 2023-09-30T22:33:34.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.343+0000 7f3e001d5700 1 -- start start 2023-09-30T22:33:34.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.344+0000 7f3e001d5700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3df809e930 0x7f3df809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:34.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.344+0000 7f3e001d5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3df809f240 0x7f3df81317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:34.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.344+0000 7f3e001d5700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3df8070940 con 0x7f3df809f650 2023-09-30T22:33:34.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.344+0000 7f3e001d5700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3df806fd60 con 0x7f3df809e930 2023-09-30T22:33:34.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.344+0000 7f3e001d5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3df806fa80 con 0x7f3df809f240 2023-09-30T22:33:34.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.344+0000 7f3dfd770700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3df809f240 0x7f3df81317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:34.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.344+0000 7f3dfd770700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3df809f240 0x7f3df81317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:53844/0 (socket says 172.21.15.16:53844) 2023-09-30T22:33:34.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.344+0000 7f3dfd770700 1 -- 172.21.15.16:0/4163598970 learned_addr learned my addr 172.21.15.16:0/4163598970 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:34.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.344+0000 7f3dfdf71700 1 --2- 172.21.15.16:0/4163598970 >> v2:172.21.15.16:3300/0 conn(0x7f3df809e930 0x7f3df809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:34.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.344+0000 7f3dfd770700 1 -- 172.21.15.16:0/4163598970 >> v2:172.21.15.16:3300/0 conn(0x7f3df809e930 msgr2=0x7f3df809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:34.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.344+0000 7f3dfd770700 1 --2- 172.21.15.16:0/4163598970 >> v2:172.21.15.16:3300/0 conn(0x7f3df809e930 0x7f3df809ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:34.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.344+0000 7f3dfd770700 1 -- 172.21.15.16:0/4163598970 >> v1:172.21.15.16:6789/0 conn(0x7f3df809f650 legacy=0x7f3df8131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:34.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.344+0000 7f3dfd770700 1 -- 172.21.15.16:0/4163598970 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3df809fb60 con 0x7f3df809f240 2023-09-30T22:33:34.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.344+0000 7f3dfdf71700 1 --2- 172.21.15.16:0/4163598970 >> v2:172.21.15.16:3300/0 conn(0x7f3df809e930 0x7f3df809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:34.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.345+0000 7f3dfd770700 1 --2- 172.21.15.16:0/4163598970 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3df809f240 0x7f3df81317f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f3de80098f0 tx=0x7f3de8009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9715d22d56de702c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:34.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.345+0000 7f3dfcf6f700 1 -- 172.21.15.16:0/4163598970 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de8011830 con 0x7f3df809f240 2023-09-30T22:33:34.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.345+0000 7f3dfcf6f700 1 -- 172.21.15.16:0/4163598970 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3de8011990 con 0x7f3df809f240 2023-09-30T22:33:34.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.345+0000 7f3dfcf6f700 1 -- 172.21.15.16:0/4163598970 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de8011d20 con 0x7f3df809f240 2023-09-30T22:33:34.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.346+0000 7f3e001d5700 1 -- 172.21.15.16:0/4163598970 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3df809f240 msgr2=0x7f3df81317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:34.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.346+0000 7f3e001d5700 1 --2- 172.21.15.16:0/4163598970 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3df809f240 0x7f3df81317f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f3de80098f0 tx=0x7f3de8009c00 comp rx=0 tx=0).stop 2023-09-30T22:33:34.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.346+0000 7f3e001d5700 1 -- 172.21.15.16:0/4163598970 shutdown_connections 2023-09-30T22:33:34.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.346+0000 7f3e001d5700 1 --2- 172.21.15.16:0/4163598970 >> v2:172.21.15.16:3300/0 conn(0x7f3df809e930 0x7f3df809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:34.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.346+0000 7f3e001d5700 1 --2- 172.21.15.16:0/4163598970 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3df809f240 0x7f3df81317f0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:34.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.346+0000 7f3e001d5700 1 -- 172.21.15.16:0/4163598970 >> 172.21.15.16:0/4163598970 conn(0x7f3df80a4210 msgr2=0x7f3df80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:34.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.346+0000 7f3e001d5700 1 -- 172.21.15.16:0/4163598970 shutdown_connections 2023-09-30T22:33:34.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.346+0000 7f3e001d5700 1 -- 172.21.15.16:0/4163598970 wait complete. 2023-09-30T22:33:34.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.347+0000 7f3e001d5700 1 Processor -- start 2023-09-30T22:33:34.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.347+0000 7f3e001d5700 1 -- start start 2023-09-30T22:33:34.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.348+0000 7f3e001d5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3df809e930 0x7f3df81a6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:34.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.348+0000 7f3e001d5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3df809f240 0x7f3df81a65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:34.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.348+0000 7f3e001d5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3df809f650 0x7f3df81a6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:34.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.348+0000 7f3e001d5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df81a7c30 con 0x7f3df809e930 2023-09-30T22:33:34.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.348+0000 7f3e001d5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3df8071d20 con 0x7f3df809f650 2023-09-30T22:33:34.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.348+0000 7f3e001d5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3df81a9860 con 0x7f3df809f240 2023-09-30T22:33:34.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.348+0000 7f3dfd770700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3df809f240 0x7f3df81a65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:34.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.348+0000 7f3dfd770700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3df809f240 0x7f3df81a65a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:53848/0 (socket says 172.21.15.16:53848) 2023-09-30T22:33:34.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.348+0000 7f3dfd770700 1 -- 172.21.15.16:0/2422519036 learned_addr learned my addr 172.21.15.16:0/2422519036 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:34.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.348+0000 7f3dfe772700 1 --2- 172.21.15.16:0/2422519036 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3df809f650 0x7f3df81a6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:34.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.348+0000 7f3dfd770700 1 -- 172.21.15.16:0/2422519036 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3df809f650 msgr2=0x7f3df81a6ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:34.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.348+0000 7f3dfd770700 1 --2- 172.21.15.16:0/2422519036 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3df809f650 0x7f3df81a6ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:34.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.348+0000 7f3dfd770700 1 -- 172.21.15.16:0/2422519036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3df809e930 msgr2=0x7f3df81a6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:34.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.348+0000 7f3dfd770700 1 --2- 172.21.15.16:0/2422519036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3df809e930 0x7f3df81a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:34.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.348+0000 7f3dfd770700 1 -- 172.21.15.16:0/2422519036 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3df40094a0 con 0x7f3df809f240 2023-09-30T22:33:34.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.349+0000 7f3dfd770700 1 --2- 172.21.15.16:0/2422519036 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3df809f240 0x7f3df81a65a0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3de8009c40 tx=0x7f3de8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:34.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.349+0000 7f3dee7fc700 1 -- 172.21.15.16:0/2422519036 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de80043c0 con 0x7f3df809f240 2023-09-30T22:33:34.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.349+0000 7f3dee7fc700 1 -- 172.21.15.16:0/2422519036 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3de801f070 con 0x7f3df809f240 2023-09-30T22:33:34.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.349+0000 7f3e001d5700 1 -- 172.21.15.16:0/2422519036 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3de8009550 con 0x7f3df809f240 2023-09-30T22:33:34.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.349+0000 7f3dee7fc700 1 -- 172.21.15.16:0/2422519036 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de80036a0 con 0x7f3df809f240 2023-09-30T22:33:34.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.349+0000 7f3e001d5700 1 -- 172.21.15.16:0/2422519036 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3df80a0190 con 0x7f3df809f240 2023-09-30T22:33:34.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.350+0000 7f3dee7fc700 1 -- 172.21.15.16:0/2422519036 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3de8021030 con 0x7f3df809f240 2023-09-30T22:33:34.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.350+0000 7f3e001d5700 1 -- 172.21.15.16:0/2422519036 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3df804ea50 con 0x7f3df809f240 2023-09-30T22:33:34.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.351+0000 7f3dee7fc700 1 -- 172.21.15.16:0/2422519036 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f3de8040b20 con 0x7f3df809f240 2023-09-30T22:33:34.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.360+0000 7f3dee7fc700 1 -- 172.21.15.16:0/2422519036 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3de80199a0 con 0x7f3df809f240 2023-09-30T22:33:34.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.389+0000 7f8d295d7700 1 Processor -- start 2023-09-30T22:33:34.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.389+0000 7f8d295d7700 1 -- start start 2023-09-30T22:33:34.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.390+0000 7f8d295d7700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8d2409d6f0 0x7f8d2409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:34.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.390+0000 7f8d295d7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8d2409e000 0x7f8d241339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:34.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.390+0000 7f8d295d7700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d2409b120 con 0x7f8d2409e410 2023-09-30T22:33:34.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.390+0000 7f8d295d7700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d2406b460 con 0x7f8d2409d6f0 2023-09-30T22:33:34.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.390+0000 7f8d295d7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2409a7e0 con 0x7f8d2409e000 2023-09-30T22:33:34.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.390+0000 7f8d22ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8d2409d6f0 0x7f8d2409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:34.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.390+0000 7f8d22ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8d2409d6f0 0x7f8d2409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55428/0 (socket says 172.21.15.16:55428) 2023-09-30T22:33:34.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.390+0000 7f8d22ffd700 1 -- 172.21.15.16:0/951530229 learned_addr learned my addr 172.21.15.16:0/951530229 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:34.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.390+0000 7f8d227fc700 1 --2- 172.21.15.16:0/951530229 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8d2409e000 0x7f8d241339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:34.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.391+0000 7f8d22ffd700 1 -- 172.21.15.16:0/951530229 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8d2409e000 msgr2=0x7f8d241339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:34.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.391+0000 7f8d22ffd700 1 --2- 172.21.15.16:0/951530229 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8d2409e000 0x7f8d241339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:34.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.391+0000 7f8d22ffd700 1 -- 172.21.15.16:0/951530229 >> v1:172.21.15.16:6789/0 conn(0x7f8d2409e410 legacy=0x7f8d24133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:34.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.391+0000 7f8d21ffb700 1 -- 172.21.15.16:0/951530229 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 147158929 0 0) 0x7f8d2409b120 con 0x7f8d2409e410 2023-09-30T22:33:34.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.391+0000 7f8d22ffd700 1 -- 172.21.15.16:0/951530229 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d2409a500 con 0x7f8d2409d6f0 2023-09-30T22:33:34.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.391+0000 7f8d22ffd700 1 --2- 172.21.15.16:0/951530229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d2409d6f0 0x7f8d2409dac0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f8d140098f0 tx=0x7f8d14009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d7c8eb7c30acfae server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:34.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.391+0000 7f8d21ffb700 1 -- 172.21.15.16:0/951530229 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1400b8a0 con 0x7f8d2409d6f0 2023-09-30T22:33:34.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.391+0000 7f8d21ffb700 1 -- 172.21.15.16:0/951530229 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d1400ba00 con 0x7f8d2409d6f0 2023-09-30T22:33:34.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.391+0000 7f8d21ffb700 1 -- 172.21.15.16:0/951530229 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1400bd90 con 0x7f8d2409d6f0 2023-09-30T22:33:34.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.392+0000 7f8d295d7700 1 -- 172.21.15.16:0/951530229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d2409d6f0 msgr2=0x7f8d2409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:34.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.392+0000 7f8d295d7700 1 --2- 172.21.15.16:0/951530229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d2409d6f0 0x7f8d2409dac0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f8d140098f0 tx=0x7f8d14009c00 comp rx=0 tx=0).stop 2023-09-30T22:33:34.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.392+0000 7f8d295d7700 1 -- 172.21.15.16:0/951530229 shutdown_connections 2023-09-30T22:33:34.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.392+0000 7f8d295d7700 1 --2- 172.21.15.16:0/951530229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d2409d6f0 0x7f8d2409dac0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:34.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.392+0000 7f8d295d7700 1 --2- 172.21.15.16:0/951530229 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8d2409e000 0x7f8d241339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:34.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.392+0000 7f8d295d7700 1 -- 172.21.15.16:0/951530229 >> 172.21.15.16:0/951530229 conn(0x7f8d24066f50 msgr2=0x7f8d2409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:34.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.392+0000 7f8d295d7700 1 -- 172.21.15.16:0/951530229 shutdown_connections 2023-09-30T22:33:34.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.392+0000 7f8d295d7700 1 -- 172.21.15.16:0/951530229 wait complete. 2023-09-30T22:33:34.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.394+0000 7f8d295d7700 1 Processor -- start 2023-09-30T22:33:34.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.394+0000 7f8d295d7700 1 -- start start 2023-09-30T22:33:34.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.394+0000 7f8d295d7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8d2409d6f0 0x7f8d241b85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:34.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.394+0000 7f8d295d7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8d2409e000 0x7f8d241b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:34.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.394+0000 7f8d295d7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d2409e410 0x7f8d241c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:34.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.394+0000 7f8d295d7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d241a4cc0 con 0x7f8d2409e410 2023-09-30T22:33:34.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.394+0000 7f8d295d7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d241a7d90 con 0x7f8d2409e000 2023-09-30T22:33:34.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.394+0000 7f8d295d7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8d241a6420 con 0x7f8d2409d6f0 2023-09-30T22:33:34.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.395+0000 7f8d22ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8d2409d6f0 0x7f8d241b85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:34.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.395+0000 7f8d227fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8d2409e000 0x7f8d241b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:34.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.395+0000 7f8d22ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8d2409d6f0 0x7f8d241b85f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:53858/0 (socket says 172.21.15.16:53858) 2023-09-30T22:33:34.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.395+0000 7f8d22ffd700 1 -- 172.21.15.16:0/1992390057 learned_addr learned my addr 172.21.15.16:0/1992390057 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:34.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.395+0000 7f8d237fe700 1 --2- 172.21.15.16:0/1992390057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d2409e410 0x7f8d241c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:34.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.395+0000 7f8d227fc700 1 -- 172.21.15.16:0/1992390057 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8d2409d6f0 msgr2=0x7f8d241b85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:34.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.395+0000 7f8d227fc700 1 --2- 172.21.15.16:0/1992390057 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8d2409d6f0 0x7f8d241b85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:34.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.395+0000 7f8d227fc700 1 -- 172.21.15.16:0/1992390057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d2409e410 msgr2=0x7f8d241c8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:34.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.395+0000 7f8d227fc700 1 --2- 172.21.15.16:0/1992390057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d2409e410 0x7f8d241c8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:34.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.395+0000 7f8d227fc700 1 -- 172.21.15.16:0/1992390057 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d14009550 con 0x7f8d2409e000 2023-09-30T22:33:34.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.395+0000 7f8d237fe700 1 --2- 172.21.15.16:0/1992390057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d2409e410 0x7f8d241c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:33:34.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.395+0000 7f8d227fc700 1 --2- 172.21.15.16:0/1992390057 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8d2409e000 0x7f8d241b8b30 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f8d1800e810 tx=0x7f8d1800eb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:34.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.395+0000 7f8d0b7fe700 1 -- 172.21.15.16:0/1992390057 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1800c070 con 0x7f8d2409e000 2023-09-30T22:33:34.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.396+0000 7f8d0b7fe700 1 -- 172.21.15.16:0/1992390057 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d18013610 con 0x7f8d2409e000 2023-09-30T22:33:34.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.396+0000 7f8d295d7700 1 -- 172.21.15.16:0/1992390057 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d241a8410 con 0x7f8d2409e000 2023-09-30T22:33:34.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.396+0000 7f8d0b7fe700 1 -- 172.21.15.16:0/1992390057 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d180139a0 con 0x7f8d2409e000 2023-09-30T22:33:34.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.396+0000 7f8d295d7700 1 -- 172.21.15.16:0/1992390057 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d241b9070 con 0x7f8d2409e000 2023-09-30T22:33:34.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.397+0000 7f8d0b7fe700 1 -- 172.21.15.16:0/1992390057 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8d18013b00 con 0x7f8d2409e000 2023-09-30T22:33:34.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.397+0000 7f8d295d7700 1 -- 172.21.15.16:0/1992390057 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d241a8270 con 0x7f8d2409e000 2023-09-30T22:33:34.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.397+0000 7f8d0b7fe700 1 -- 172.21.15.16:0/1992390057 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f8d1803f4f0 con 0x7f8d2409e000 2023-09-30T22:33:34.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.403+0000 7f8d0b7fe700 1 -- 172.21.15.16:0/1992390057 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8d241a8270 con 0x7f8d2409e000 2023-09-30T22:33:34.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.523+0000 7f44a1819700 1 -- 172.21.15.16:0/2012639064 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f449c11c110 con 0x7f449c131810 2023-09-30T22:33:34.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.524+0000 7ff9cbf20700 1 -- 172.21.15.16:0/3271396798 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff9c413ce60 con 0x7ff9c4134610 2023-09-30T22:33:34.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.524+0000 7f4483fff700 1 -- 172.21.15.16:0/2012639064 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f449c11c110 con 0x7f449c131810 2023-09-30T22:33:34.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.524+0000 7ff9ba7fc700 1 -- 172.21.15.16:0/3271396798 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff9b402b090 con 0x7ff9c4134610 2023-09-30T22:33:34.525 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:33:34.525 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:33:34.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.527+0000 7ff9cbf20700 1 -- 172.21.15.16:0/3271396798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9c4134610 msgr2=0x7ff9c413ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:34.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.527+0000 7ff9cbf20700 1 --2- 172.21.15.16:0/3271396798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9c4134610 0x7ff9c413ec40 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7ff9b400ebe0 tx=0x7ff9b400eef0 comp rx=0 tx=0).stop 2023-09-30T22:33:34.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.527+0000 7ff9cbf20700 1 -- 172.21.15.16:0/3271396798 shutdown_connections 2023-09-30T22:33:34.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.527+0000 7ff9cbf20700 1 --2- 172.21.15.16:0/3271396798 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff9c413e020 0x7ff9c413f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:34.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.527+0000 7ff9cbf20700 1 --2- 172.21.15.16:0/3271396798 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff9c406e570 0x7ff9c413e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:34.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.527+0000 7ff9cbf20700 1 --2- 172.21.15.16:0/3271396798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9c4134610 0x7ff9c413ec40 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:34.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.527+0000 7ff9cbf20700 1 -- 172.21.15.16:0/3271396798 >> 172.21.15.16:0/3271396798 conn(0x7ff9c40a2aa0 msgr2=0x7ff9c409e340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:34.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.527+0000 7f44a1819700 1 -- 172.21.15.16:0/2012639064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f449c131810 msgr2=0x7f449c118cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:34.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.527+0000 7f44a1819700 1 --2- 172.21.15.16:0/2012639064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f449c131810 0x7f449c118cd0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f448c00c730 tx=0x7f448c00ca40 comp rx=0 tx=0).stop 2023-09-30T22:33:34.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.527+0000 7f44a1819700 1 -- 172.21.15.16:0/2012639064 shutdown_connections 2023-09-30T22:33:34.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.527+0000 7f44a1819700 1 --2- 172.21.15.16:0/2012639064 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f449c136f50 0x7f449c119750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:34.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.528+0000 7f44a1819700 1 --2- 172.21.15.16:0/2012639064 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f449c131810 0x7f449c118cd0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:34.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.528+0000 7f44a1819700 1 --2- 172.21.15.16:0/2012639064 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f449c134630 0x7f449c119210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:34.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.528+0000 7f44a1819700 1 -- 172.21.15.16:0/2012639064 >> 172.21.15.16:0/2012639064 conn(0x7f449c0a2aa0 msgr2=0x7f449c0a0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:34.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.528+0000 7ff9cbf20700 1 -- 172.21.15.16:0/3271396798 shutdown_connections 2023-09-30T22:33:34.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.528+0000 7f44a1819700 1 -- 172.21.15.16:0/2012639064 shutdown_connections 2023-09-30T22:33:34.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.528+0000 7f44a1819700 1 -- 172.21.15.16:0/2012639064 wait complete. 2023-09-30T22:33:34.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.529+0000 7ff9cbf20700 1 -- 172.21.15.16:0/3271396798 wait complete. 2023-09-30T22:33:34.544 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:33:34.547 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:33:34.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.615+0000 7f3e001d5700 1 -- 172.21.15.16:0/2422519036 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3df8074ea0 con 0x7f3df809f240 2023-09-30T22:33:34.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.616+0000 7f3dee7fc700 1 -- 172.21.15.16:0/2422519036 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3de80193b0 con 0x7f3df809f240 2023-09-30T22:33:34.616 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:33:34.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.618+0000 7f8d295d7700 1 -- 172.21.15.16:0/1992390057 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f8d241a8210 con 0x7f8d2409e000 2023-09-30T22:33:34.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.618+0000 7f8d0b7fe700 1 -- 172.21.15.16:0/1992390057 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8d180108e0 con 0x7f8d2409e000 2023-09-30T22:33:34.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.618+0000 7f3e001d5700 1 -- 172.21.15.16:0/2422519036 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3df809f240 msgr2=0x7f3df81a65a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:34.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.618+0000 7f3e001d5700 1 --2- 172.21.15.16:0/2422519036 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3df809f240 0x7f3df81a65a0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3de8009c40 tx=0x7f3de8004750 comp rx=0 tx=0).stop 2023-09-30T22:33:34.619 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:33:34.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.618+0000 7f3e001d5700 1 -- 172.21.15.16:0/2422519036 shutdown_connections 2023-09-30T22:33:34.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.618+0000 7f3e001d5700 1 --2- 172.21.15.16:0/2422519036 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3df809f650 0x7f3df81a6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:34.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.618+0000 7f3e001d5700 1 --2- 172.21.15.16:0/2422519036 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3df809e930 0x7f3df81a6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:34.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.618+0000 7f3e001d5700 1 --2- 172.21.15.16:0/2422519036 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3df809f240 0x7f3df81a65a0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:34.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.618+0000 7f3e001d5700 1 -- 172.21.15.16:0/2422519036 >> 172.21.15.16:0/2422519036 conn(0x7f3df80a4210 msgr2=0x7f3df81bae40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:34.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.618+0000 7f3e001d5700 1 -- 172.21.15.16:0/2422519036 shutdown_connections 2023-09-30T22:33:34.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.618+0000 7f3e001d5700 1 -- 172.21.15.16:0/2422519036 wait complete. 2023-09-30T22:33:34.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.620+0000 7f8d295d7700 1 -- 172.21.15.16:0/1992390057 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8d2409e000 msgr2=0x7f8d241b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:34.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.620+0000 7f8d295d7700 1 --2- 172.21.15.16:0/1992390057 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8d2409e000 0x7f8d241b8b30 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f8d1800e810 tx=0x7f8d1800eb20 comp rx=0 tx=0).stop 2023-09-30T22:33:34.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.620+0000 7f8d295d7700 1 -- 172.21.15.16:0/1992390057 shutdown_connections 2023-09-30T22:33:34.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.620+0000 7f8d295d7700 1 --2- 172.21.15.16:0/1992390057 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d2409e410 0x7f8d241c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:34.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.620+0000 7f8d295d7700 1 --2- 172.21.15.16:0/1992390057 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8d2409d6f0 0x7f8d241b85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:34.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.620+0000 7f8d295d7700 1 --2- 172.21.15.16:0/1992390057 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8d2409e000 0x7f8d241b8b30 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:34.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.620+0000 7f8d295d7700 1 -- 172.21.15.16:0/1992390057 >> 172.21.15.16:0/1992390057 conn(0x7f8d24066f50 msgr2=0x7f8d240a3640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:34.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.621+0000 7f8d295d7700 1 -- 172.21.15.16:0/1992390057 shutdown_connections 2023-09-30T22:33:34.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:34.621+0000 7f8d295d7700 1 -- 172.21.15.16:0/1992390057 wait complete. 2023-09-30T22:33:34.633 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:33:34.635 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:33:35.545 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:33:35.548 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:33:35.634 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:33:35.635 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:33:35.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.702+0000 7f286e235700 1 Processor -- start 2023-09-30T22:33:35.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.702+0000 7f286e235700 1 -- start start 2023-09-30T22:33:35.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.702+0000 7f286e235700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f28681338a0 0x7f2868133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:35.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.702+0000 7f286e235700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f286809e930 0x7f286813b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:35.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.702+0000 7f286e235700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f286806ef30 con 0x7f286809de50 2023-09-30T22:33:35.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.702+0000 7f286e235700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f286806fef0 con 0x7f286809e930 2023-09-30T22:33:35.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.702+0000 7f286e235700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f28681334b0 con 0x7f28681338a0 2023-09-30T22:33:35.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.702+0000 7f28677fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f28681338a0 0x7f2868133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:35.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.702+0000 7f28677fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f28681338a0 0x7f2868133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:53860/0 (socket says 172.21.15.16:53860) 2023-09-30T22:33:35.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.702+0000 7f28677fe700 1 -- 172.21.15.16:0/3260196472 learned_addr learned my addr 172.21.15.16:0/3260196472 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:35.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.703+0000 7f28677fe700 1 -- 172.21.15.16:0/3260196472 >> v2:172.21.15.16:3300/0 conn(0x7f286809e930 msgr2=0x7f286813b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:35.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.703+0000 7f28677fe700 1 --2- 172.21.15.16:0/3260196472 >> v2:172.21.15.16:3300/0 conn(0x7f286809e930 0x7f286813b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:35.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.703+0000 7f28677fe700 1 -- 172.21.15.16:0/3260196472 >> v1:172.21.15.16:6789/0 conn(0x7f286809de50 legacy=0x7f286809e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:35.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.703+0000 7f28677fe700 1 -- 172.21.15.16:0/3260196472 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f286806fc30 con 0x7f28681338a0 2023-09-30T22:33:35.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.703+0000 7f28677fe700 1 --2- 172.21.15.16:0/3260196472 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f28681338a0 0x7f2868133c70 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f28580098f0 tx=0x7f2858009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9f2f5a8b3f39bd8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:35.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.703+0000 7f28667fc700 1 -- 172.21.15.16:0/3260196472 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f285800b8a0 con 0x7f28681338a0 2023-09-30T22:33:35.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.703+0000 7f28667fc700 1 -- 172.21.15.16:0/3260196472 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f285800ba00 con 0x7f28681338a0 2023-09-30T22:33:35.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.703+0000 7f28667fc700 1 -- 172.21.15.16:0/3260196472 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f285800bd90 con 0x7f28681338a0 2023-09-30T22:33:35.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.704+0000 7f286e235700 1 -- 172.21.15.16:0/3260196472 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f28681338a0 msgr2=0x7f2868133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:35.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.704+0000 7f286e235700 1 --2- 172.21.15.16:0/3260196472 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f28681338a0 0x7f2868133c70 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f28580098f0 tx=0x7f2858009c00 comp rx=0 tx=0).stop 2023-09-30T22:33:35.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.704+0000 7f286e235700 1 -- 172.21.15.16:0/3260196472 shutdown_connections 2023-09-30T22:33:35.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.704+0000 7f286e235700 1 --2- 172.21.15.16:0/3260196472 >> v2:172.21.15.16:3300/0 conn(0x7f286809e930 0x7f286813b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:35.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.704+0000 7f286e235700 1 --2- 172.21.15.16:0/3260196472 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f28681338a0 0x7f2868133c70 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:35.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.704+0000 7f286e235700 1 -- 172.21.15.16:0/3260196472 >> 172.21.15.16:0/3260196472 conn(0x7f28680a2aa0 msgr2=0x7f28680a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:35.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.704+0000 7f286e235700 1 -- 172.21.15.16:0/3260196472 shutdown_connections 2023-09-30T22:33:35.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.704+0000 7f286e235700 1 -- 172.21.15.16:0/3260196472 wait complete. 2023-09-30T22:33:35.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.705+0000 7f286e235700 1 Processor -- start 2023-09-30T22:33:35.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.706+0000 7f286e235700 1 -- start start 2023-09-30T22:33:35.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.706+0000 7f286e235700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f286809de50 0x7f286811eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:35.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.706+0000 7f286e235700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f286809e930 0x7f286811f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:35.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.706+0000 7f286e235700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28681338a0 0x7f286811f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:35.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.706+0000 7f286e235700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f286812faa0 con 0x7f28681338a0 2023-09-30T22:33:35.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.706+0000 7f286e235700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2868122060 con 0x7f286809e930 2023-09-30T22:33:35.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.706+0000 7f286e235700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f286812f5e0 con 0x7f286809de50 2023-09-30T22:33:35.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.707+0000 7f28677fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f286809de50 0x7f286811eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:35.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.707+0000 7f2867fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28681338a0 0x7f286811f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:35.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.707+0000 7f28677fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f286809de50 0x7f286811eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:53876/0 (socket says 172.21.15.16:53876) 2023-09-30T22:33:35.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.707+0000 7f28677fe700 1 -- 172.21.15.16:0/2268489644 learned_addr learned my addr 172.21.15.16:0/2268489644 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:35.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.707+0000 7f2866ffd700 1 --2- 172.21.15.16:0/2268489644 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f286809e930 0x7f286811f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:35.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.707+0000 7f2867fff700 1 -- 172.21.15.16:0/2268489644 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f286809de50 msgr2=0x7f286811eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:35.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.707+0000 7f2867fff700 1 --2- 172.21.15.16:0/2268489644 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f286809de50 0x7f286811eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:35.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.707+0000 7f2867fff700 1 -- 172.21.15.16:0/2268489644 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f286809e930 msgr2=0x7f286811f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:35.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.707+0000 7f2867fff700 1 --2- 172.21.15.16:0/2268489644 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f286809e930 0x7f286811f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:35.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.707+0000 7f2867fff700 1 -- 172.21.15.16:0/2268489644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2858009550 con 0x7f28681338a0 2023-09-30T22:33:35.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.707+0000 7f2867fff700 1 --2- 172.21.15.16:0/2268489644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28681338a0 0x7f286811f560 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f286000be20 tx=0x7f2860009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:35.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.707+0000 7f284ffff700 1 -- 172.21.15.16:0/2268489644 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f286000cb90 con 0x7f28681338a0 2023-09-30T22:33:35.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.707+0000 7f284ffff700 1 -- 172.21.15.16:0/2268489644 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f286000cd10 con 0x7f28681338a0 2023-09-30T22:33:35.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.708+0000 7f286e235700 1 -- 172.21.15.16:0/2268489644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2868120790 con 0x7f28681338a0 2023-09-30T22:33:35.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.708+0000 7f284ffff700 1 -- 172.21.15.16:0/2268489644 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2860014680 con 0x7f28681338a0 2023-09-30T22:33:35.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.708+0000 7f286e235700 1 -- 172.21.15.16:0/2268489644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2868121d70 con 0x7f28681338a0 2023-09-30T22:33:35.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.709+0000 7f284ffff700 1 -- 172.21.15.16:0/2268489644 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f286001d3b0 con 0x7f28681338a0 2023-09-30T22:33:35.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.709+0000 7f286e235700 1 -- 172.21.15.16:0/2268489644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2868074ea0 con 0x7f28681338a0 2023-09-30T22:33:35.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.709+0000 7f284ffff700 1 -- 172.21.15.16:0/2268489644 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f2860006e40 con 0x7f28681338a0 2023-09-30T22:33:35.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.718+0000 7f284ffff700 1 -- 172.21.15.16:0/2268489644 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2860019020 con 0x7f28681338a0 2023-09-30T22:33:35.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.748+0000 7fd90f828700 1 Processor -- start 2023-09-30T22:33:35.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.748+0000 7fd90f828700 1 -- start start 2023-09-30T22:33:35.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.749+0000 7fd90f828700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd90809eca0 0x7fd908135550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:35.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.749+0000 7fd90f828700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd908135a90 0x7fd908137e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:35.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.749+0000 7fd90f828700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd908070a50 con 0x7fd908130560 2023-09-30T22:33:35.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.749+0000 7fd90f828700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9080717d0 con 0x7fd90809eca0 2023-09-30T22:33:35.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.749+0000 7fd90f828700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd908130340 con 0x7fd908135a90 2023-09-30T22:33:35.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.749+0000 7fd90cdc3700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd90809eca0 0x7fd908135550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:35.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.749+0000 7fd90cdc3700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd90809eca0 0x7fd908135550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55478/0 (socket says 172.21.15.16:55478) 2023-09-30T22:33:35.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.749+0000 7fd90d5c4700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd908130560 0x7fd908130930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35160/0 (socket says 172.21.15.16:35160) 2023-09-30T22:33:35.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.749+0000 7fd90cdc3700 1 -- 172.21.15.16:0/459328797 learned_addr learned my addr 172.21.15.16:0/459328797 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:35.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.750+0000 7fd8fffff700 1 -- 172.21.15.16:0/459328797 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2134327907 0 0) 0x7fd908070a50 con 0x7fd908130560 2023-09-30T22:33:35.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.750+0000 7fd8fffff700 1 -- 172.21.15.16:0/459328797 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8f40035b0 con 0x7fd908130560 2023-09-30T22:33:35.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.750+0000 7fd90cdc3700 1 -- 172.21.15.16:0/459328797 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd908135a90 msgr2=0x7fd908137e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:35.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.750+0000 7fd90cdc3700 1 --2- 172.21.15.16:0/459328797 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd908135a90 0x7fd908137e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:35.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.750+0000 7fd90cdc3700 1 -- 172.21.15.16:0/459328797 >> v1:172.21.15.16:6789/0 conn(0x7fd908130560 legacy=0x7fd908130930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:35.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.750+0000 7fd90cdc3700 1 -- 172.21.15.16:0/459328797 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd908071060 con 0x7fd90809eca0 2023-09-30T22:33:35.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.750+0000 7fd90cdc3700 1 --2- 172.21.15.16:0/459328797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd90809eca0 0x7fd908135550 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fd8f80098f0 tx=0x7fd8f8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58a2ab255c9b0adb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:35.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.750+0000 7fd8fffff700 1 -- 172.21.15.16:0/459328797 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f8011830 con 0x7fd90809eca0 2023-09-30T22:33:35.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.750+0000 7fd8fffff700 1 -- 172.21.15.16:0/459328797 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8f8011990 con 0x7fd90809eca0 2023-09-30T22:33:35.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.750+0000 7fd8fffff700 1 -- 172.21.15.16:0/459328797 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f8011d20 con 0x7fd90809eca0 2023-09-30T22:33:35.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.751+0000 7fd90f828700 1 -- 172.21.15.16:0/459328797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd90809eca0 msgr2=0x7fd908135550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:35.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.751+0000 7fd90f828700 1 --2- 172.21.15.16:0/459328797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd90809eca0 0x7fd908135550 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fd8f80098f0 tx=0x7fd8f8009c00 comp rx=0 tx=0).stop 2023-09-30T22:33:35.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.752+0000 7fd90f828700 1 -- 172.21.15.16:0/459328797 shutdown_connections 2023-09-30T22:33:35.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.752+0000 7fd90f828700 1 --2- 172.21.15.16:0/459328797 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd908135a90 0x7fd908137e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:35.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.752+0000 7fd90f828700 1 --2- 172.21.15.16:0/459328797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd90809eca0 0x7fd908135550 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:35.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.752+0000 7fd90f828700 1 -- 172.21.15.16:0/459328797 >> 172.21.15.16:0/459328797 conn(0x7fd9080a2aa0 msgr2=0x7fd9080a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:35.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.752+0000 7fd90f828700 1 -- 172.21.15.16:0/459328797 shutdown_connections 2023-09-30T22:33:35.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.752+0000 7fd90f828700 1 -- 172.21.15.16:0/459328797 wait complete. 2023-09-30T22:33:35.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.753+0000 7fd90f828700 1 Processor -- start 2023-09-30T22:33:35.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.753+0000 7fd90f828700 1 -- start start 2023-09-30T22:33:35.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.753+0000 7fd90f828700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd90809eca0 0x7fd9081c3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:35.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.753+0000 7fd90f828700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd908130560 0x7fd9081c41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:35.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.753+0000 7fd90f828700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd908135a90 0x7fd9081c46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:35.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.753+0000 7fd90f828700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9080719f0 con 0x7fd908130560 2023-09-30T22:33:35.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.753+0000 7fd90f828700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd90809cee0 con 0x7fd908135a90 2023-09-30T22:33:35.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.753+0000 7fd90f828700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd90809c3f0 con 0x7fd90809eca0 2023-09-30T22:33:35.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.754+0000 7fd90d5c4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd90809eca0 0x7fd9081c3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:35.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.754+0000 7fd90cdc3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd908130560 0x7fd9081c41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:35.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.754+0000 7fd90d5c4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd90809eca0 0x7fd9081c3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:53888/0 (socket says 172.21.15.16:53888) 2023-09-30T22:33:35.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.754+0000 7fd90d5c4700 1 -- 172.21.15.16:0/1681058282 learned_addr learned my addr 172.21.15.16:0/1681058282 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:35.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.754+0000 7fd90ddc5700 1 --2- 172.21.15.16:0/1681058282 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd908135a90 0x7fd9081c46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:35.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.754+0000 7fd90d5c4700 1 -- 172.21.15.16:0/1681058282 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd908135a90 msgr2=0x7fd9081c46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:35.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.754+0000 7fd90d5c4700 1 --2- 172.21.15.16:0/1681058282 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd908135a90 0x7fd9081c46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:35.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.754+0000 7fd90d5c4700 1 -- 172.21.15.16:0/1681058282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd908130560 msgr2=0x7fd9081c41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:35.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.754+0000 7fd90d5c4700 1 --2- 172.21.15.16:0/1681058282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd908130560 0x7fd9081c41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:35.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.754+0000 7fd90d5c4700 1 -- 172.21.15.16:0/1681058282 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f8009550 con 0x7fd90809eca0 2023-09-30T22:33:35.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.754+0000 7fd90cdc3700 1 --2- 172.21.15.16:0/1681058282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd908130560 0x7fd9081c41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:35.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.754+0000 7fd90d5c4700 1 --2- 172.21.15.16:0/1681058282 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd90809eca0 0x7fd9081c3c70 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd90400c730 tx=0x7fd90400ca40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:35.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.755+0000 7fd8fdffb700 1 -- 172.21.15.16:0/1681058282 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd904005cb0 con 0x7fd90809eca0 2023-09-30T22:33:35.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.755+0000 7fd8fdffb700 1 -- 172.21.15.16:0/1681058282 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd904005e10 con 0x7fd90809eca0 2023-09-30T22:33:35.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.755+0000 7fd90f828700 1 -- 172.21.15.16:0/1681058282 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd90809e180 con 0x7fd90809eca0 2023-09-30T22:33:35.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.755+0000 7fd8fdffb700 1 -- 172.21.15.16:0/1681058282 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9040072d0 con 0x7fd90809eca0 2023-09-30T22:33:35.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.755+0000 7fd90f828700 1 -- 172.21.15.16:0/1681058282 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd90809dea0 con 0x7fd90809eca0 2023-09-30T22:33:35.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.756+0000 7fd90f828700 1 -- 172.21.15.16:0/1681058282 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd90809dac0 con 0x7fd90809eca0 2023-09-30T22:33:35.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.757+0000 7fd8fdffb700 1 -- 172.21.15.16:0/1681058282 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd904007430 con 0x7fd90809eca0 2023-09-30T22:33:35.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.757+0000 7fd8fdffb700 1 -- 172.21.15.16:0/1681058282 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd9040076e0 con 0x7fd90809eca0 2023-09-30T22:33:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.766+0000 7fd8fdffb700 1 -- 172.21.15.16:0/1681058282 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd904044060 con 0x7fd90809eca0 2023-09-30T22:33:35.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.786+0000 7f9f4b961700 1 Processor -- start 2023-09-30T22:33:35.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.787+0000 7f9f4b961700 1 -- start start 2023-09-30T22:33:35.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.787+0000 7f9f4b961700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9f4409d6f0 0x7f9f4409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:35.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.787+0000 7f9f4b961700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9f4409e410 0x7f9f44130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:35.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.787+0000 7f9f4b961700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f4409f130 con 0x7f9f4409e000 2023-09-30T22:33:35.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.787+0000 7f9f4b961700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f4409f520 con 0x7f9f4409d6f0 2023-09-30T22:33:35.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.787+0000 7f9f4b961700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4409e930 con 0x7f9f4409e410 2023-09-30T22:33:35.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.788+0000 7f9f48efc700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9f4409e000 0x7f9f44130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35168/0 (socket says 172.21.15.16:35168) 2023-09-30T22:33:35.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.788+0000 7f9f496fd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9f4409d6f0 0x7f9f4409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:35.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.788+0000 7f9f48efc700 1 -- 172.21.15.16:0/3075495874 learned_addr learned my addr 172.21.15.16:0/3075495874 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:35.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.788+0000 7f9f49efe700 1 --2- 172.21.15.16:0/3075495874 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9f4409e410 0x7f9f44130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:35.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.788+0000 7f9f3bfff700 1 -- 172.21.15.16:0/3075495874 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4076294504 0 0) 0x7f9f4409f130 con 0x7f9f4409e000 2023-09-30T22:33:35.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.788+0000 7f9f496fd700 1 -- 172.21.15.16:0/3075495874 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9f4409e410 msgr2=0x7f9f44130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:35.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.788+0000 7f9f496fd700 1 --2- 172.21.15.16:0/3075495874 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9f4409e410 0x7f9f44130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:35.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.788+0000 7f9f496fd700 1 -- 172.21.15.16:0/3075495874 >> v1:172.21.15.16:6789/0 conn(0x7f9f4409e000 legacy=0x7f9f44130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:35.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.788+0000 7f9f496fd700 1 -- 172.21.15.16:0/3075495874 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f4409b2d0 con 0x7f9f4409d6f0 2023-09-30T22:33:35.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.788+0000 7f9f496fd700 1 --2- 172.21.15.16:0/3075495874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f4409d6f0 0x7f9f4409dac0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f9f40009840 tx=0x7f9f40009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5454915ba6d5340c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:35.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.788+0000 7f9f3bfff700 1 -- 172.21.15.16:0/3075495874 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f40011830 con 0x7f9f4409d6f0 2023-09-30T22:33:35.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.789+0000 7f9f3bfff700 1 -- 172.21.15.16:0/3075495874 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f40011990 con 0x7f9f4409d6f0 2023-09-30T22:33:35.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.789+0000 7f9f3bfff700 1 -- 172.21.15.16:0/3075495874 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f40011d20 con 0x7f9f4409d6f0 2023-09-30T22:33:35.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.789+0000 7f9f4b961700 1 -- 172.21.15.16:0/3075495874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f4409d6f0 msgr2=0x7f9f4409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:35.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.789+0000 7f9f4b961700 1 --2- 172.21.15.16:0/3075495874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f4409d6f0 0x7f9f4409dac0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f9f40009840 tx=0x7f9f40009b50 comp rx=0 tx=0).stop 2023-09-30T22:33:35.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.789+0000 7f9f4b961700 1 -- 172.21.15.16:0/3075495874 shutdown_connections 2023-09-30T22:33:35.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.789+0000 7f9f4b961700 1 --2- 172.21.15.16:0/3075495874 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9f4409e410 0x7f9f44130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:35.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.789+0000 7f9f4b961700 1 --2- 172.21.15.16:0/3075495874 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f4409d6f0 0x7f9f4409dac0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:35.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.789+0000 7f9f4b961700 1 -- 172.21.15.16:0/3075495874 >> 172.21.15.16:0/3075495874 conn(0x7f9f44066f50 msgr2=0x7f9f4409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:35.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.789+0000 7f9f4b961700 1 -- 172.21.15.16:0/3075495874 shutdown_connections 2023-09-30T22:33:35.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.789+0000 7f9f4b961700 1 -- 172.21.15.16:0/3075495874 wait complete. 2023-09-30T22:33:35.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.790+0000 7f9f4b961700 1 Processor -- start 2023-09-30T22:33:35.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.791+0000 7f9f4b961700 1 -- start start 2023-09-30T22:33:35.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.791+0000 7f9f4b961700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9f4409d6f0 0x7f9f44130bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:35.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.791+0000 7f9f4b961700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f4409e000 0x7f9f44136810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:35.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.791+0000 7f9f4b961700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9f4409e410 0x7f9f44136d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:35.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.791+0000 7f9f4b961700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f44131100 con 0x7f9f4409e000 2023-09-30T22:33:35.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.791+0000 7f9f4b961700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f440a3890 con 0x7f9f4409d6f0 2023-09-30T22:33:35.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.791+0000 7f9f4b961700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9f441389d0 con 0x7f9f4409e410 2023-09-30T22:33:35.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.791+0000 7f9f48efc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f4409e000 0x7f9f44136810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:35.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.791+0000 7f9f48efc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f4409e000 0x7f9f44136810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55504/0 (socket says 172.21.15.16:55504) 2023-09-30T22:33:35.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.792+0000 7f9f48efc700 1 -- 172.21.15.16:0/964618480 learned_addr learned my addr 172.21.15.16:0/964618480 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:35.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.792+0000 7f9f496fd700 1 --2- 172.21.15.16:0/964618480 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9f4409d6f0 0x7f9f44130bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:35.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.792+0000 7f9f48efc700 1 -- 172.21.15.16:0/964618480 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9f4409e410 msgr2=0x7f9f44136d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:35.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.792+0000 7f9f48efc700 1 --2- 172.21.15.16:0/964618480 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9f4409e410 0x7f9f44136d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:35.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.792+0000 7f9f48efc700 1 -- 172.21.15.16:0/964618480 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9f4409d6f0 msgr2=0x7f9f44130bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:35.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.792+0000 7f9f48efc700 1 --2- 172.21.15.16:0/964618480 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9f4409d6f0 0x7f9f44130bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:35.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.792+0000 7f9f48efc700 1 -- 172.21.15.16:0/964618480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f400094a0 con 0x7f9f4409e000 2023-09-30T22:33:35.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.792+0000 7f9f496fd700 1 --2- 172.21.15.16:0/964618480 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9f4409d6f0 0x7f9f44130bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:33:35.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.792+0000 7f9f48efc700 1 --2- 172.21.15.16:0/964618480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f4409e000 0x7f9f44136810 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f9f3400c520 tx=0x7f9f3400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:35.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.792+0000 7f9f39ffb700 1 -- 172.21.15.16:0/964618480 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f34004c80 con 0x7f9f4409e000 2023-09-30T22:33:35.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.792+0000 7fae47037700 1 Processor -- start 2023-09-30T22:33:35.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.792+0000 7f9f39ffb700 1 -- 172.21.15.16:0/964618480 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f34004e00 con 0x7f9f4409e000 2023-09-30T22:33:35.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.792+0000 7fae47037700 1 -- start start 2023-09-30T22:33:35.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.792+0000 7f9f4b961700 1 -- 172.21.15.16:0/964618480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f4413b020 con 0x7f9f4409e000 2023-09-30T22:33:35.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.792+0000 7f9f39ffb700 1 -- 172.21.15.16:0/964618480 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f34007310 con 0x7f9f4409e000 2023-09-30T22:33:35.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.793+0000 7f9f4b961700 1 -- 172.21.15.16:0/964618480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f4413aac0 con 0x7f9f4409e000 2023-09-30T22:33:35.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.793+0000 7fae47037700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fae4009e1d0 0x7fae40135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:35.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.793+0000 7fae47037700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fae40135e50 0x7fae40138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:35.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.793+0000 7fae47037700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae4009f130 con 0x7fae4009d6f0 2023-09-30T22:33:35.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.793+0000 7fae47037700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae4009f520 con 0x7fae4009e1d0 2023-09-30T22:33:35.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.793+0000 7fae47037700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fae4009e930 con 0x7fae40135e50 2023-09-30T22:33:35.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.794+0000 7fae44dd3700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fae4009d6f0 0x7fae4009dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35184/0 (socket says 172.21.15.16:35184) 2023-09-30T22:33:35.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.794+0000 7fae44dd3700 1 -- 172.21.15.16:0/1906959494 learned_addr learned my addr 172.21.15.16:0/1906959494 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:35.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.794+0000 7f9f4b961700 1 -- 172.21.15.16:0/964618480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f44139f70 con 0x7f9f4409e000 2023-09-30T22:33:35.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.794+0000 7fae3ffff700 1 --2- 172.21.15.16:0/1906959494 >> v2:172.21.15.16:3300/0 conn(0x7fae4009e1d0 0x7fae40135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:35.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.794+0000 7fae3f7fe700 1 -- 172.21.15.16:0/1906959494 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2876601307 0 0) 0x7fae4009f130 con 0x7fae4009d6f0 2023-09-30T22:33:35.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.794+0000 7fae3f7fe700 1 -- 172.21.15.16:0/1906959494 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae280035b0 con 0x7fae4009d6f0 2023-09-30T22:33:35.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.794+0000 7f9f39ffb700 1 -- 172.21.15.16:0/964618480 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9f34007470 con 0x7f9f4409e000 2023-09-30T22:33:35.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.794+0000 7fae3ffff700 1 -- 172.21.15.16:0/1906959494 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fae40135e50 msgr2=0x7fae40138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:35.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.794+0000 7fae3ffff700 1 --2- 172.21.15.16:0/1906959494 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fae40135e50 0x7fae40138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:35.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.794+0000 7fae3ffff700 1 -- 172.21.15.16:0/1906959494 >> v1:172.21.15.16:6789/0 conn(0x7fae4009d6f0 legacy=0x7fae4009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:35.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.794+0000 7f9f39ffb700 1 -- 172.21.15.16:0/964618480 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f9f3403f940 con 0x7f9f4409e000 2023-09-30T22:33:35.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.794+0000 7fae3ffff700 1 -- 172.21.15.16:0/1906959494 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae4009b2d0 con 0x7fae4009e1d0 2023-09-30T22:33:35.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.795+0000 7fae3ffff700 1 --2- 172.21.15.16:0/1906959494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae4009e1d0 0x7fae40135910 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fae34009840 tx=0x7fae34009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48537d77a157bbd4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:35.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.799+0000 7fae3f7fe700 1 -- 172.21.15.16:0/1906959494 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae34011830 con 0x7fae4009e1d0 2023-09-30T22:33:35.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.799+0000 7fae3f7fe700 1 -- 172.21.15.16:0/1906959494 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae34011990 con 0x7fae4009e1d0 2023-09-30T22:33:35.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.799+0000 7fae3f7fe700 1 -- 172.21.15.16:0/1906959494 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae34011d20 con 0x7fae4009e1d0 2023-09-30T22:33:35.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.799+0000 7f9f39ffb700 1 -- 172.21.15.16:0/964618480 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9f340102c0 con 0x7f9f4409e000 2023-09-30T22:33:35.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.800+0000 7fae47037700 1 -- 172.21.15.16:0/1906959494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae4009e1d0 msgr2=0x7fae40135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:35.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.800+0000 7fae47037700 1 --2- 172.21.15.16:0/1906959494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae4009e1d0 0x7fae40135910 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fae34009840 tx=0x7fae34009b50 comp rx=0 tx=0).stop 2023-09-30T22:33:35.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.800+0000 7fae47037700 1 -- 172.21.15.16:0/1906959494 shutdown_connections 2023-09-30T22:33:35.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.800+0000 7fae47037700 1 --2- 172.21.15.16:0/1906959494 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fae40135e50 0x7fae40138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:35.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.800+0000 7fae47037700 1 --2- 172.21.15.16:0/1906959494 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae4009e1d0 0x7fae40135910 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:35.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.800+0000 7fae47037700 1 -- 172.21.15.16:0/1906959494 >> 172.21.15.16:0/1906959494 conn(0x7fae40066f50 msgr2=0x7fae4009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:35.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.800+0000 7fae47037700 1 -- 172.21.15.16:0/1906959494 shutdown_connections 2023-09-30T22:33:35.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.800+0000 7fae47037700 1 -- 172.21.15.16:0/1906959494 wait complete. 2023-09-30T22:33:35.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.802+0000 7fae47037700 1 Processor -- start 2023-09-30T22:33:35.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.802+0000 7fae47037700 1 -- start start 2023-09-30T22:33:35.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.803+0000 7fae47037700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fae4009d6f0 0x7fae40132bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:35.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.803+0000 7fae47037700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae4009e1d0 0x7fae401330f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:35.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.803+0000 7fae47037700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fae40135e50 0x7fae40133630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:35.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.803+0000 7fae47037700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae40119cc0 con 0x7fae4009e1d0 2023-09-30T22:33:35.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.803+0000 7fae47037700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae401194f0 con 0x7fae4009d6f0 2023-09-30T22:33:35.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.803+0000 7fae47037700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fae400a2160 con 0x7fae40135e50 2023-09-30T22:33:35.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.803+0000 7fae455d4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fae40135e50 0x7fae40133630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:35.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.803+0000 7fae455d4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fae40135e50 0x7fae40133630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:53920/0 (socket says 172.21.15.16:53920) 2023-09-30T22:33:35.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.803+0000 7fae455d4700 1 -- 172.21.15.16:0/784984492 learned_addr learned my addr 172.21.15.16:0/784984492 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:35.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.803+0000 7fae44dd3700 1 --2- 172.21.15.16:0/784984492 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fae4009d6f0 0x7fae40132bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:35.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.803+0000 7fae3ffff700 1 --2- 172.21.15.16:0/784984492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae4009e1d0 0x7fae401330f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:35.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.804+0000 7fae3ffff700 1 -- 172.21.15.16:0/784984492 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fae40135e50 msgr2=0x7fae40133630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:35.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.804+0000 7fae3ffff700 1 --2- 172.21.15.16:0/784984492 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fae40135e50 0x7fae40133630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:35.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.804+0000 7fae3ffff700 1 -- 172.21.15.16:0/784984492 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fae4009d6f0 msgr2=0x7fae40132bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:35.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.804+0000 7fae3ffff700 1 --2- 172.21.15.16:0/784984492 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fae4009d6f0 0x7fae40132bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:35.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.804+0000 7fae3ffff700 1 -- 172.21.15.16:0/784984492 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae340094a0 con 0x7fae4009e1d0 2023-09-30T22:33:35.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.804+0000 7fae455d4700 1 --2- 172.21.15.16:0/784984492 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fae40135e50 0x7fae40133630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:35.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.804+0000 7fae3ffff700 1 --2- 172.21.15.16:0/784984492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae4009e1d0 0x7fae401330f0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fae34016000 tx=0x7fae34004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:35.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.804+0000 7fae3d7fa700 1 -- 172.21.15.16:0/784984492 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae340036a0 con 0x7fae4009e1d0 2023-09-30T22:33:35.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.804+0000 7fae3d7fa700 1 -- 172.21.15.16:0/784984492 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae3401f070 con 0x7fae4009e1d0 2023-09-30T22:33:35.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.804+0000 7fae47037700 1 -- 172.21.15.16:0/784984492 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae40134720 con 0x7fae4009e1d0 2023-09-30T22:33:35.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.804+0000 7fae3d7fa700 1 -- 172.21.15.16:0/784984492 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae34003a60 con 0x7fae4009e1d0 2023-09-30T22:33:35.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.804+0000 7fae47037700 1 -- 172.21.15.16:0/784984492 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae4011a450 con 0x7fae4009e1d0 2023-09-30T22:33:35.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.805+0000 7fae3d7fa700 1 -- 172.21.15.16:0/784984492 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fae34003bc0 con 0x7fae4009e1d0 2023-09-30T22:33:35.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.805+0000 7fae3d7fa700 1 -- 172.21.15.16:0/784984492 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fae34021070 con 0x7fae4009e1d0 2023-09-30T22:33:35.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.807+0000 7fae47037700 1 -- 172.21.15.16:0/784984492 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae4009c700 con 0x7fae4009e1d0 2023-09-30T22:33:35.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.816+0000 7fae3d7fa700 1 -- 172.21.15.16:0/784984492 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fae3402b020 con 0x7fae4009e1d0 2023-09-30T22:33:35.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.941+0000 7f286e235700 1 -- 172.21.15.16:0/2268489644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2868122560 con 0x7f28681338a0 2023-09-30T22:33:35.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.941+0000 7f284ffff700 1 -- 172.21.15.16:0/2268489644 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f286001c360 con 0x7f28681338a0 2023-09-30T22:33:35.942 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:33:35.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.943+0000 7f286e235700 1 -- 172.21.15.16:0/2268489644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28681338a0 msgr2=0x7f286811f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:35.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.943+0000 7f286e235700 1 --2- 172.21.15.16:0/2268489644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28681338a0 0x7f286811f560 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f286000be20 tx=0x7f2860009ed0 comp rx=0 tx=0).stop 2023-09-30T22:33:35.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.943+0000 7f286e235700 1 -- 172.21.15.16:0/2268489644 shutdown_connections 2023-09-30T22:33:35.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.943+0000 7f286e235700 1 --2- 172.21.15.16:0/2268489644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28681338a0 0x7f286811f560 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:35.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.943+0000 7f286e235700 1 --2- 172.21.15.16:0/2268489644 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f286809de50 0x7f286811eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:35.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.943+0000 7f286e235700 1 --2- 172.21.15.16:0/2268489644 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f286809e930 0x7f286811f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:35.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.943+0000 7f286e235700 1 -- 172.21.15.16:0/2268489644 >> 172.21.15.16:0/2268489644 conn(0x7f28680a2aa0 msgr2=0x7f286809ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:35.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.944+0000 7f286e235700 1 -- 172.21.15.16:0/2268489644 shutdown_connections 2023-09-30T22:33:35.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.944+0000 7f286e235700 1 -- 172.21.15.16:0/2268489644 wait complete. 2023-09-30T22:33:35.958 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:33:35.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.980+0000 7fd90f828700 1 -- 172.21.15.16:0/1681058282 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd908074ea0 con 0x7fd90809eca0 2023-09-30T22:33:35.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.980+0000 7fd8fdffb700 1 -- 172.21.15.16:0/1681058282 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd904044070 con 0x7fd90809eca0 2023-09-30T22:33:35.981 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:33:35.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.983+0000 7fd90f828700 1 -- 172.21.15.16:0/1681058282 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd90809eca0 msgr2=0x7fd9081c3c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:35.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.983+0000 7fd90f828700 1 --2- 172.21.15.16:0/1681058282 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd90809eca0 0x7fd9081c3c70 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fd90400c730 tx=0x7fd90400ca40 comp rx=0 tx=0).stop 2023-09-30T22:33:35.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.983+0000 7fd90f828700 1 -- 172.21.15.16:0/1681058282 shutdown_connections 2023-09-30T22:33:35.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.983+0000 7fd90f828700 1 --2- 172.21.15.16:0/1681058282 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd908135a90 0x7fd9081c46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:35.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.983+0000 7fd90f828700 1 --2- 172.21.15.16:0/1681058282 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd90809eca0 0x7fd9081c3c70 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:35.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.983+0000 7fd90f828700 1 --2- 172.21.15.16:0/1681058282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd908130560 0x7fd9081c41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:35.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.983+0000 7fd90f828700 1 -- 172.21.15.16:0/1681058282 >> 172.21.15.16:0/1681058282 conn(0x7fd9080a2aa0 msgr2=0x7fd90812e130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:35.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.983+0000 7fd90f828700 1 -- 172.21.15.16:0/1681058282 shutdown_connections 2023-09-30T22:33:35.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:35.983+0000 7fd90f828700 1 -- 172.21.15.16:0/1681058282 wait complete. 2023-09-30T22:33:36.003 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:33:36.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:36.020+0000 7f9f4b961700 1 -- 172.21.15.16:0/964618480 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f9f44139f70 con 0x7f9f4409e000 2023-09-30T22:33:36.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:36.020+0000 7f9f39ffb700 1 -- 172.21.15.16:0/964618480 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9f44139f70 con 0x7f9f4409e000 2023-09-30T22:33:36.021 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:33:36.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:36.023+0000 7f9f4b961700 1 -- 172.21.15.16:0/964618480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f4409e000 msgr2=0x7f9f44136810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:36.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:36.023+0000 7f9f4b961700 1 --2- 172.21.15.16:0/964618480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f4409e000 0x7f9f44136810 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f9f3400c520 tx=0x7f9f3400c8e0 comp rx=0 tx=0).stop 2023-09-30T22:33:36.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:36.023+0000 7f9f4b961700 1 -- 172.21.15.16:0/964618480 shutdown_connections 2023-09-30T22:33:36.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:36.023+0000 7f9f4b961700 1 --2- 172.21.15.16:0/964618480 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9f4409d6f0 0x7f9f44130bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:36.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:36.023+0000 7f9f4b961700 1 --2- 172.21.15.16:0/964618480 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9f4409e410 0x7f9f44136d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:36.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:36.023+0000 7f9f4b961700 1 --2- 172.21.15.16:0/964618480 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f4409e000 0x7f9f44136810 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:36.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:36.023+0000 7f9f4b961700 1 -- 172.21.15.16:0/964618480 >> 172.21.15.16:0/964618480 conn(0x7f9f44066f50 msgr2=0x7f9f44070e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:36.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:36.023+0000 7f9f4b961700 1 -- 172.21.15.16:0/964618480 shutdown_connections 2023-09-30T22:33:36.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:36.023+0000 7f9f4b961700 1 -- 172.21.15.16:0/964618480 wait complete. 2023-09-30T22:33:36.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:36.030+0000 7fae47037700 1 -- 172.21.15.16:0/784984492 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fae40071700 con 0x7fae4009e1d0 2023-09-30T22:33:36.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:36.031+0000 7fae3d7fa700 1 -- 172.21.15.16:0/784984492 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fae34015070 con 0x7fae4009e1d0 2023-09-30T22:33:36.031 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:33:36.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:36.033+0000 7fae47037700 1 -- 172.21.15.16:0/784984492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae4009e1d0 msgr2=0x7fae401330f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:36.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:36.033+0000 7fae47037700 1 --2- 172.21.15.16:0/784984492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae4009e1d0 0x7fae401330f0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fae34016000 tx=0x7fae34004750 comp rx=0 tx=0).stop 2023-09-30T22:33:36.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:36.033+0000 7fae47037700 1 -- 172.21.15.16:0/784984492 shutdown_connections 2023-09-30T22:33:36.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:36.033+0000 7fae47037700 1 --2- 172.21.15.16:0/784984492 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fae4009d6f0 0x7fae40132bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:36.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:36.033+0000 7fae47037700 1 --2- 172.21.15.16:0/784984492 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fae40135e50 0x7fae40133630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:36.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:36.033+0000 7fae47037700 1 --2- 172.21.15.16:0/784984492 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae4009e1d0 0x7fae401330f0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:36.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:36.033+0000 7fae47037700 1 -- 172.21.15.16:0/784984492 >> 172.21.15.16:0/784984492 conn(0x7fae40066f50 msgr2=0x7fae4006e840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:36.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:36.033+0000 7fae47037700 1 -- 172.21.15.16:0/784984492 shutdown_connections 2023-09-30T22:33:36.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:36.033+0000 7fae47037700 1 -- 172.21.15.16:0/784984492 wait complete. 2023-09-30T22:33:36.037 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:33:36.048 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:33:36.959 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:33:37.003 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:33:37.038 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:33:37.048 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:33:37.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.113+0000 7fc24b65c700 1 Processor -- start 2023-09-30T22:33:37.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.114+0000 7fc24b65c700 1 -- start start 2023-09-30T22:33:37.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.114+0000 7fc24b65c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc2440a1510 0x7fc244138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:37.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.114+0000 7fc24b65c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc244139250 0x7fc24413b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:37.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.114+0000 7fc24b65c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc24409edd0 con 0x7fc2440a0a30 2023-09-30T22:33:37.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.114+0000 7fc24b65c700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2440a18e0 con 0x7fc2440a1510 2023-09-30T22:33:37.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.114+0000 7fc24b65c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc24409e1f0 con 0x7fc244139250 2023-09-30T22:33:37.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.114+0000 7fc248bf7700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc2440a1510 0x7fc244138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:37.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.115+0000 7fc248bf7700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc2440a1510 0x7fc244138d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55534/0 (socket says 172.21.15.16:55534) 2023-09-30T22:33:37.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.115+0000 7fc2493f8700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc2440a0a30 0x7fc2440a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:35194/0 (socket says 172.21.15.16:35194) 2023-09-30T22:33:37.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.115+0000 7fc248bf7700 1 -- 172.21.15.16:0/2017798985 learned_addr learned my addr 172.21.15.16:0/2017798985 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:37.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.115+0000 7fc249bf9700 1 --2- 172.21.15.16:0/2017798985 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc244139250 0x7fc24413b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:37.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.115+0000 7fc248bf7700 1 -- 172.21.15.16:0/2017798985 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc244139250 msgr2=0x7fc24413b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:37.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.115+0000 7fc248bf7700 1 --2- 172.21.15.16:0/2017798985 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc244139250 0x7fc24413b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:37.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.115+0000 7fc248bf7700 1 -- 172.21.15.16:0/2017798985 >> v1:172.21.15.16:6789/0 conn(0x7fc2440a0a30 legacy=0x7fc2440a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:37.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.115+0000 7fc248bf7700 1 -- 172.21.15.16:0/2017798985 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc24409dc50 con 0x7fc2440a1510 2023-09-30T22:33:37.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.115+0000 7fc248bf7700 1 --2- 172.21.15.16:0/2017798985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2440a1510 0x7fc244138d10 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fc2340098f0 tx=0x7fc234009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c99a52653ec8df7a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:37.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.115+0000 7fc23bfff700 1 -- 172.21.15.16:0/2017798985 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc234004230 con 0x7fc2440a1510 2023-09-30T22:33:37.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.115+0000 7fc23bfff700 1 -- 172.21.15.16:0/2017798985 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc234004390 con 0x7fc2440a1510 2023-09-30T22:33:37.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.115+0000 7fc23bfff700 1 -- 172.21.15.16:0/2017798985 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2340036a0 con 0x7fc2440a1510 2023-09-30T22:33:37.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.116+0000 7fc24b65c700 1 -- 172.21.15.16:0/2017798985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2440a1510 msgr2=0x7fc244138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:37.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.116+0000 7fc24b65c700 1 --2- 172.21.15.16:0/2017798985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2440a1510 0x7fc244138d10 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fc2340098f0 tx=0x7fc234009c00 comp rx=0 tx=0).stop 2023-09-30T22:33:37.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.116+0000 7fc24b65c700 1 -- 172.21.15.16:0/2017798985 shutdown_connections 2023-09-30T22:33:37.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.116+0000 7fc24b65c700 1 --2- 172.21.15.16:0/2017798985 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc244139250 0x7fc24413b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:37.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.116+0000 7fc24b65c700 1 --2- 172.21.15.16:0/2017798985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2440a1510 0x7fc244138d10 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:37.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.116+0000 7fc24b65c700 1 -- 172.21.15.16:0/2017798985 >> 172.21.15.16:0/2017798985 conn(0x7fc2440a6550 msgr2=0x7fc2440a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:37.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.116+0000 7fc24b65c700 1 -- 172.21.15.16:0/2017798985 shutdown_connections 2023-09-30T22:33:37.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.117+0000 7fc24b65c700 1 -- 172.21.15.16:0/2017798985 wait complete. 2023-09-30T22:33:37.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.117+0000 7fc24b65c700 1 Processor -- start 2023-09-30T22:33:37.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.118+0000 7fc24b65c700 1 -- start start 2023-09-30T22:33:37.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.118+0000 7fc24b65c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2440a0a30 0x7fc2441c8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:37.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.118+0000 7fc24b65c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2440a1510 0x7fc2441c8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:37.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.118+0000 7fc24b65c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc244139250 0x7fc2441c8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:37.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.118+0000 7fc24b65c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc244123850 con 0x7fc244139250 2023-09-30T22:33:37.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.118+0000 7fc24b65c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc244125600 con 0x7fc2440a1510 2023-09-30T22:33:37.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.118+0000 7fc24b65c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc244138d40 con 0x7fc2440a0a30 2023-09-30T22:33:37.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.118+0000 7fc2493f8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2440a0a30 0x7fc2441c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:37.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.118+0000 7fc248bf7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2440a1510 0x7fc2441c8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:37.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.118+0000 7fc248bf7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2440a1510 0x7fc2441c8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54932/0 (socket says 172.21.15.16:54932) 2023-09-30T22:33:37.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.118+0000 7fc248bf7700 1 -- 172.21.15.16:0/2777184634 learned_addr learned my addr 172.21.15.16:0/2777184634 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:37.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.119+0000 7fc248bf7700 1 -- 172.21.15.16:0/2777184634 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2440a0a30 msgr2=0x7fc2441c8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:37.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.119+0000 7fc248bf7700 1 --2- 172.21.15.16:0/2777184634 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2440a0a30 0x7fc2441c8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:37.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.119+0000 7fc248bf7700 1 -- 172.21.15.16:0/2777184634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc244139250 msgr2=0x7fc2441c8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:37.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.119+0000 7fc248bf7700 1 --2- 172.21.15.16:0/2777184634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc244139250 0x7fc2441c8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:37.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.119+0000 7fc248bf7700 1 -- 172.21.15.16:0/2777184634 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc234009550 con 0x7fc2440a1510 2023-09-30T22:33:37.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.119+0000 7fc2493f8700 1 --2- 172.21.15.16:0/2777184634 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2440a0a30 0x7fc2441c8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:37.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.119+0000 7fc248bf7700 1 --2- 172.21.15.16:0/2777184634 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2440a1510 0x7fc2441c8a80 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc234009130 tx=0x7fc234004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:37.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.119+0000 7fc239ffb700 1 -- 172.21.15.16:0/2777184634 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2340045d0 con 0x7fc2440a1510 2023-09-30T22:33:37.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.119+0000 7fc239ffb700 1 -- 172.21.15.16:0/2777184634 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc234011630 con 0x7fc2440a1510 2023-09-30T22:33:37.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.119+0000 7fc24b65c700 1 -- 172.21.15.16:0/2777184634 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2440a0220 con 0x7fc2440a1510 2023-09-30T22:33:37.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.119+0000 7fc239ffb700 1 -- 172.21.15.16:0/2777184634 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2340119a0 con 0x7fc2440a1510 2023-09-30T22:33:37.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.119+0000 7fc24b65c700 1 -- 172.21.15.16:0/2777184634 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc244125cb0 con 0x7fc2440a1510 2023-09-30T22:33:37.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.121+0000 7fc239ffb700 1 -- 172.21.15.16:0/2777184634 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc234022020 con 0x7fc2440a1510 2023-09-30T22:33:37.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.121+0000 7fc22f7fe700 1 -- 172.21.15.16:0/2777184634 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2440a3310 con 0x7fc2440a1510 2023-09-30T22:33:37.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.121+0000 7fc239ffb700 1 -- 172.21.15.16:0/2777184634 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fc234040d20 con 0x7fc2440a1510 2023-09-30T22:33:37.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.130+0000 7fc239ffb700 1 -- 172.21.15.16:0/2777184634 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc234046020 con 0x7fc2440a1510 2023-09-30T22:33:37.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.171+0000 7fde53336700 1 Processor -- start 2023-09-30T22:33:37.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.171+0000 7fde53336700 1 -- start start 2023-09-30T22:33:37.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.172+0000 7fde53336700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fde4c130540 0x7fde4c130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:37.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.172+0000 7fde53336700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fde4c130e50 0x7fde4c06f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:37.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.172+0000 7fde53336700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fde4c06fb20 0x7fde4c06ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:37.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.172+0000 7fde53336700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde4c09a390 con 0x7fde4c130540 2023-09-30T22:33:37.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.172+0000 7fde53336700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c0716a0 con 0x7fde4c130e50 2023-09-30T22:33:37.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.172+0000 7fde53336700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c071290 con 0x7fde4c06fb20 2023-09-30T22:33:37.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.172+0000 7fde510d2700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fde4c130540 0x7fde4c130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:37.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.172+0000 7fde510d2700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fde4c130540 0x7fde4c130910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55544/0 (socket says 172.21.15.16:55544) 2023-09-30T22:33:37.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.172+0000 7fde508d1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fde4c130e50 0x7fde4c06f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:37.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.172+0000 7fde510d2700 1 -- 172.21.15.16:0/1795586287 learned_addr learned my addr 172.21.15.16:0/1795586287 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:37.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.172+0000 7fde518d3700 1 --2- 172.21.15.16:0/1795586287 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fde4c06fb20 0x7fde4c06ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:37.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.173+0000 7fde510d2700 1 -- 172.21.15.16:0/1795586287 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fde4c06fb20 msgr2=0x7fde4c06ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:37.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.173+0000 7fde510d2700 1 --2- 172.21.15.16:0/1795586287 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fde4c06fb20 0x7fde4c06ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:37.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.173+0000 7fde510d2700 1 -- 172.21.15.16:0/1795586287 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fde4c130e50 msgr2=0x7fde4c06f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:37.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.173+0000 7fde510d2700 1 --2- 172.21.15.16:0/1795586287 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fde4c130e50 0x7fde4c06f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:37.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.173+0000 7fde510d2700 1 -- 172.21.15.16:0/1795586287 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde4c130340 con 0x7fde4c130540 2023-09-30T22:33:37.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.173+0000 7fde510d2700 1 --2- 172.21.15.16:0/1795586287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde4c130540 0x7fde4c130910 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fde4800b9a0 tx=0x7fde4800bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f0e110e8ccdb7f9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:37.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.173+0000 7fde43fff700 1 -- 172.21.15.16:0/1795586287 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4800caa0 con 0x7fde4c130540 2023-09-30T22:33:37.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.173+0000 7fde43fff700 1 -- 172.21.15.16:0/1795586287 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde4800cc00 con 0x7fde4c130540 2023-09-30T22:33:37.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.173+0000 7fde43fff700 1 -- 172.21.15.16:0/1795586287 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4800f460 con 0x7fde4c130540 2023-09-30T22:33:37.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.174+0000 7fde53336700 1 -- 172.21.15.16:0/1795586287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde4c130540 msgr2=0x7fde4c130910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:37.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.174+0000 7fde53336700 1 --2- 172.21.15.16:0/1795586287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde4c130540 0x7fde4c130910 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fde4800b9a0 tx=0x7fde4800bcb0 comp rx=0 tx=0).stop 2023-09-30T22:33:37.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.174+0000 7fde53336700 1 -- 172.21.15.16:0/1795586287 shutdown_connections 2023-09-30T22:33:37.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.174+0000 7fde53336700 1 --2- 172.21.15.16:0/1795586287 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fde4c06fb20 0x7fde4c06ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:37.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.174+0000 7fde53336700 1 --2- 172.21.15.16:0/1795586287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde4c130540 0x7fde4c130910 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:37.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.174+0000 7fde53336700 1 --2- 172.21.15.16:0/1795586287 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fde4c130e50 0x7fde4c06f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:37.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.174+0000 7fde53336700 1 -- 172.21.15.16:0/1795586287 >> 172.21.15.16:0/1795586287 conn(0x7fde4c0a2aa0 msgr2=0x7fde4c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:37.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.174+0000 7fde53336700 1 -- 172.21.15.16:0/1795586287 shutdown_connections 2023-09-30T22:33:37.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.174+0000 7fde53336700 1 -- 172.21.15.16:0/1795586287 wait complete. 2023-09-30T22:33:37.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.175+0000 7fde53336700 1 Processor -- start 2023-09-30T22:33:37.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.175+0000 7fde53336700 1 -- start start 2023-09-30T22:33:37.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.176+0000 7fde53336700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fde4c06fb20 0x7fde4c118cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:37.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.176+0000 7fde53336700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde4c130540 0x7fde4c119210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:37.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.176+0000 7fde53336700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fde4c130e50 0x7fde4c119750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:37.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.176+0000 7fde53336700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c11c940 con 0x7fde4c130540 2023-09-30T22:33:37.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.176+0000 7fde53336700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c11a070 con 0x7fde4c130e50 2023-09-30T22:33:37.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.176+0000 7fde53336700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c11b2c0 con 0x7fde4c06fb20 2023-09-30T22:33:37.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.176+0000 7fde510d2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fde4c06fb20 0x7fde4c118cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:37.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.176+0000 7fde510d2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fde4c06fb20 0x7fde4c118cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:53960/0 (socket says 172.21.15.16:53960) 2023-09-30T22:33:37.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.176+0000 7fde510d2700 1 -- 172.21.15.16:0/776455318 learned_addr learned my addr 172.21.15.16:0/776455318 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:37.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.176+0000 7fde518d3700 1 --2- 172.21.15.16:0/776455318 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fde4c130e50 0x7fde4c119750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:37.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.176+0000 7fde508d1700 1 --2- 172.21.15.16:0/776455318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde4c130540 0x7fde4c119210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:37.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.176+0000 7fde510d2700 1 -- 172.21.15.16:0/776455318 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fde4c130e50 msgr2=0x7fde4c119750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:37.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.176+0000 7fde510d2700 1 --2- 172.21.15.16:0/776455318 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fde4c130e50 0x7fde4c119750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:37.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.176+0000 7fde510d2700 1 -- 172.21.15.16:0/776455318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde4c130540 msgr2=0x7fde4c119210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:37.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.176+0000 7fde510d2700 1 --2- 172.21.15.16:0/776455318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde4c130540 0x7fde4c119210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:37.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.176+0000 7fde510d2700 1 -- 172.21.15.16:0/776455318 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde4800b600 con 0x7fde4c06fb20 2023-09-30T22:33:37.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.176+0000 7fde508d1700 1 --2- 172.21.15.16:0/776455318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde4c130540 0x7fde4c119210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:33:37.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.177+0000 7fde510d2700 1 --2- 172.21.15.16:0/776455318 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fde4c06fb20 0x7fde4c118cd0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fde4800b340 tx=0x7fde48004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:37.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.177+0000 7fde41ffb700 1 -- 172.21.15.16:0/776455318 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4801c070 con 0x7fde4c06fb20 2023-09-30T22:33:37.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.177+0000 7fde41ffb700 1 -- 172.21.15.16:0/776455318 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde4800c8a0 con 0x7fde4c06fb20 2023-09-30T22:33:37.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.177+0000 7fde53336700 1 -- 172.21.15.16:0/776455318 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde4c11a660 con 0x7fde4c06fb20 2023-09-30T22:33:37.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.177+0000 7fde41ffb700 1 -- 172.21.15.16:0/776455318 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde48004270 con 0x7fde4c06fb20 2023-09-30T22:33:37.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.177+0000 7fde53336700 1 -- 172.21.15.16:0/776455318 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde4c0718e0 con 0x7fde4c06fb20 2023-09-30T22:33:37.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.178+0000 7fde41ffb700 1 -- 172.21.15.16:0/776455318 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fde480043d0 con 0x7fde4c06fb20 2023-09-30T22:33:37.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.178+0000 7fde41ffb700 1 -- 172.21.15.16:0/776455318 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fde48040aa0 con 0x7fde4c06fb20 2023-09-30T22:33:37.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.182+0000 7fde53336700 1 -- 172.21.15.16:0/776455318 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde4c074ea0 con 0x7fde4c06fb20 2023-09-30T22:33:37.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.187+0000 7fde41ffb700 1 -- 172.21.15.16:0/776455318 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fde48025080 con 0x7fde4c06fb20 2023-09-30T22:33:37.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.195+0000 7f8e49033700 1 Processor -- start 2023-09-30T22:33:37.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.195+0000 7f8e49033700 1 -- start start 2023-09-30T22:33:37.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.195+0000 7f8e49033700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8e4409f410 0x7f8e44136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:37.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.196+0000 7f8e49033700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8e44136fe0 0x7f8e441393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:37.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.196+0000 7f8e49033700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e44070940 con 0x7f8e4409e930 2023-09-30T22:33:37.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.196+0000 7f8e49033700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e4406fd60 con 0x7f8e44136fe0 2023-09-30T22:33:37.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.196+0000 7f8e49033700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4406fa80 con 0x7f8e4409f410 2023-09-30T22:33:37.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.196+0000 7f8e4259c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8e4409f410 0x7f8e44136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:37.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.196+0000 7f8e4259c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8e4409f410 0x7f8e44136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54964/0 (socket says 172.21.15.16:54964) 2023-09-30T22:33:37.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.196+0000 7f8e4259c700 1 -- 172.21.15.16:0/3803125488 learned_addr learned my addr 172.21.15.16:0/3803125488 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:37.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.196+0000 7f8e4259c700 1 -- 172.21.15.16:0/3803125488 >> v2:172.21.15.16:3300/0 conn(0x7f8e44136fe0 msgr2=0x7f8e441393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:37.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.196+0000 7f8e4259c700 1 --2- 172.21.15.16:0/3803125488 >> v2:172.21.15.16:3300/0 conn(0x7f8e44136fe0 0x7f8e441393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:37.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.196+0000 7f8e4259c700 1 -- 172.21.15.16:0/3803125488 >> v1:172.21.15.16:6789/0 conn(0x7f8e4409e930 legacy=0x7f8e4409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:37.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.196+0000 7f8e4259c700 1 -- 172.21.15.16:0/3803125488 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e4409fb60 con 0x7f8e4409f410 2023-09-30T22:33:37.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.196+0000 7f8e4259c700 1 --2- 172.21.15.16:0/3803125488 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8e4409f410 0x7f8e44136aa0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8e380098f0 tx=0x7f8e38009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=594382e75985051 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:37.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.196+0000 7f8e41d9b700 1 -- 172.21.15.16:0/3803125488 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e38004230 con 0x7f8e4409f410 2023-09-30T22:33:37.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.197+0000 7f8e41d9b700 1 -- 172.21.15.16:0/3803125488 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e38004390 con 0x7f8e4409f410 2023-09-30T22:33:37.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.197+0000 7f8e41d9b700 1 -- 172.21.15.16:0/3803125488 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e380036a0 con 0x7f8e4409f410 2023-09-30T22:33:37.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.197+0000 7f8e49033700 1 -- 172.21.15.16:0/3803125488 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8e4409f410 msgr2=0x7f8e44136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:37.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.197+0000 7f8e49033700 1 --2- 172.21.15.16:0/3803125488 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8e4409f410 0x7f8e44136aa0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f8e380098f0 tx=0x7f8e38009c00 comp rx=0 tx=0).stop 2023-09-30T22:33:37.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.197+0000 7f8e49033700 1 -- 172.21.15.16:0/3803125488 shutdown_connections 2023-09-30T22:33:37.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.197+0000 7f8e49033700 1 --2- 172.21.15.16:0/3803125488 >> v2:172.21.15.16:3300/0 conn(0x7f8e44136fe0 0x7f8e441393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:37.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.197+0000 7f8e49033700 1 --2- 172.21.15.16:0/3803125488 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8e4409f410 0x7f8e44136aa0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:37.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.197+0000 7f8e49033700 1 -- 172.21.15.16:0/3803125488 >> 172.21.15.16:0/3803125488 conn(0x7f8e440a4210 msgr2=0x7f8e440a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:37.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.197+0000 7f8e49033700 1 -- 172.21.15.16:0/3803125488 shutdown_connections 2023-09-30T22:33:37.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.197+0000 7f8e49033700 1 -- 172.21.15.16:0/3803125488 wait complete. 2023-09-30T22:33:37.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.199+0000 7f8e49033700 1 Processor -- start 2023-09-30T22:33:37.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.199+0000 7f8e49033700 1 -- start start 2023-09-30T22:33:37.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.200+0000 7f8e49033700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e4409e930 0x7f8e4411b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:37.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.200+0000 7f8e49033700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8e4409f410 0x7f8e4411be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:37.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.200+0000 7f8e49033700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8e44136fe0 0x7f8e4411c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:37.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.200+0000 7f8e49033700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4411f1d0 con 0x7f8e4409e930 2023-09-30T22:33:37.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.200+0000 7f8e49033700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4411f3d0 con 0x7f8e44136fe0 2023-09-30T22:33:37.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.200+0000 7f8e49033700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4411e2c0 con 0x7f8e4409f410 2023-09-30T22:33:37.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.200+0000 7f8e4359e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8e44136fe0 0x7f8e4411c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:37.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.200+0000 7f8e4359e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8e44136fe0 0x7f8e4411c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:54968/0 (socket says 172.21.15.16:54968) 2023-09-30T22:33:37.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.200+0000 7f8e4359e700 1 -- 172.21.15.16:0/803809883 learned_addr learned my addr 172.21.15.16:0/803809883 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:37.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.200+0000 7f8e42d9d700 1 --2- 172.21.15.16:0/803809883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e4409e930 0x7f8e4411b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:37.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.200+0000 7f8e4259c700 1 --2- 172.21.15.16:0/803809883 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8e4409f410 0x7f8e4411be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:37.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.201+0000 7f8e42d9d700 1 -- 172.21.15.16:0/803809883 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8e4409f410 msgr2=0x7f8e4411be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:37.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.201+0000 7f8e42d9d700 1 --2- 172.21.15.16:0/803809883 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8e4409f410 0x7f8e4411be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:37.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.201+0000 7f8e42d9d700 1 -- 172.21.15.16:0/803809883 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8e44136fe0 msgr2=0x7f8e4411c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:37.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.201+0000 7f8e42d9d700 1 --2- 172.21.15.16:0/803809883 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8e44136fe0 0x7f8e4411c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:37.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.201+0000 7f8e42d9d700 1 -- 172.21.15.16:0/803809883 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e38009550 con 0x7f8e4409e930 2023-09-30T22:33:37.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.201+0000 7f8e4359e700 1 --2- 172.21.15.16:0/803809883 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8e44136fe0 0x7f8e4411c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:37.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.201+0000 7f8e42d9d700 1 --2- 172.21.15.16:0/803809883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e4409e930 0x7f8e4411b950 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f8e3400c3e0 tx=0x7f8e3400c6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:37.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.201+0000 7f8e2b7fe700 1 -- 172.21.15.16:0/803809883 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3400cea0 con 0x7f8e4409e930 2023-09-30T22:33:37.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.201+0000 7f8e2b7fe700 1 -- 172.21.15.16:0/803809883 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e34005500 con 0x7f8e4409e930 2023-09-30T22:33:37.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.201+0000 7f8e49033700 1 -- 172.21.15.16:0/803809883 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e44136de0 con 0x7f8e4409e930 2023-09-30T22:33:37.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.201+0000 7f8e2b7fe700 1 -- 172.21.15.16:0/803809883 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e340038c0 con 0x7f8e4409e930 2023-09-30T22:33:37.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.201+0000 7f8e49033700 1 -- 172.21.15.16:0/803809883 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e44071d20 con 0x7f8e4409e930 2023-09-30T22:33:37.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.202+0000 7f8e2b7fe700 1 -- 172.21.15.16:0/803809883 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8e34003a20 con 0x7f8e4409e930 2023-09-30T22:33:37.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.202+0000 7f8e2b7fe700 1 -- 172.21.15.16:0/803809883 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f8e3403eb10 con 0x7f8e4409e930 2023-09-30T22:33:37.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.204+0000 7f8e49033700 1 -- 172.21.15.16:0/803809883 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e240052e0 con 0x7f8e4409e930 2023-09-30T22:33:37.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.204+0000 7efdae100700 1 Processor -- start 2023-09-30T22:33:37.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.204+0000 7efdae100700 1 -- start start 2023-09-30T22:33:37.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.205+0000 7efdae100700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efda809d6f0 0x7efda809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:37.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.205+0000 7efdae100700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efda809e000 0x7efda81339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:37.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.205+0000 7efdae100700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efda809b120 con 0x7efda809e410 2023-09-30T22:33:37.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.205+0000 7efdae100700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efda806b460 con 0x7efda809d6f0 2023-09-30T22:33:37.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.205+0000 7efdae100700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efda809a7e0 con 0x7efda809e000 2023-09-30T22:33:37.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.205+0000 7efda6ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efda809e000 0x7efda81339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:37.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.205+0000 7efda6ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efda809e000 0x7efda81339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:53982/0 (socket says 172.21.15.16:53982) 2023-09-30T22:33:37.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.205+0000 7efda6ffd700 1 -- 172.21.15.16:0/1612196169 learned_addr learned my addr 172.21.15.16:0/1612196169 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:37.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.206+0000 7efda77fe700 1 --2- 172.21.15.16:0/1612196169 >> v2:172.21.15.16:3300/0 conn(0x7efda809d6f0 0x7efda809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:37.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.206+0000 7efda6ffd700 1 -- 172.21.15.16:0/1612196169 >> v2:172.21.15.16:3300/0 conn(0x7efda809d6f0 msgr2=0x7efda809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:37.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.206+0000 7efda6ffd700 1 --2- 172.21.15.16:0/1612196169 >> v2:172.21.15.16:3300/0 conn(0x7efda809d6f0 0x7efda809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:37.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.206+0000 7efda6ffd700 1 -- 172.21.15.16:0/1612196169 >> v1:172.21.15.16:6789/0 conn(0x7efda809e410 legacy=0x7efda8133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:37.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.206+0000 7efda6ffd700 1 -- 172.21.15.16:0/1612196169 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efda809a500 con 0x7efda809e000 2023-09-30T22:33:37.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.206+0000 7efda6ffd700 1 --2- 172.21.15.16:0/1612196169 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efda809e000 0x7efda81339f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7efd9c009920 tx=0x7efd9c009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c1990289429b9dd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:37.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.206+0000 7efda67fc700 1 -- 172.21.15.16:0/1612196169 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9c011830 con 0x7efda809e000 2023-09-30T22:33:37.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.206+0000 7efda67fc700 1 -- 172.21.15.16:0/1612196169 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd9c011990 con 0x7efda809e000 2023-09-30T22:33:37.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.206+0000 7efda67fc700 1 -- 172.21.15.16:0/1612196169 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd9c011d20 con 0x7efda809e000 2023-09-30T22:33:37.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.207+0000 7efdae100700 1 -- 172.21.15.16:0/1612196169 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efda809e000 msgr2=0x7efda81339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:37.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.207+0000 7efdae100700 1 --2- 172.21.15.16:0/1612196169 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efda809e000 0x7efda81339f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7efd9c009920 tx=0x7efd9c009c30 comp rx=0 tx=0).stop 2023-09-30T22:33:37.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.207+0000 7efdae100700 1 -- 172.21.15.16:0/1612196169 shutdown_connections 2023-09-30T22:33:37.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.207+0000 7efdae100700 1 --2- 172.21.15.16:0/1612196169 >> v2:172.21.15.16:3300/0 conn(0x7efda809d6f0 0x7efda809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:37.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.207+0000 7efdae100700 1 --2- 172.21.15.16:0/1612196169 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efda809e000 0x7efda81339f0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:37.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.207+0000 7efdae100700 1 -- 172.21.15.16:0/1612196169 >> 172.21.15.16:0/1612196169 conn(0x7efda8066f50 msgr2=0x7efda809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:37.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.207+0000 7efdae100700 1 -- 172.21.15.16:0/1612196169 shutdown_connections 2023-09-30T22:33:37.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.207+0000 7efdae100700 1 -- 172.21.15.16:0/1612196169 wait complete. 2023-09-30T22:33:37.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.209+0000 7efdae100700 1 Processor -- start 2023-09-30T22:33:37.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.209+0000 7efdae100700 1 -- start start 2023-09-30T22:33:37.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.210+0000 7efdae100700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efda809d6f0 0x7efda81a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:37.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.210+0000 7efdae100700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efda809e000 0x7efda81a6640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:37.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.210+0000 7efdae100700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efda809e410 0x7efda81a6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:37.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.210+0000 7efdae100700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efda81a7d10 con 0x7efda809d6f0 2023-09-30T22:33:37.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.210+0000 7efdae100700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efda81a8f60 con 0x7efda809e410 2023-09-30T22:33:37.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.210+0000 7efdae100700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efda81a82f0 con 0x7efda809e000 2023-09-30T22:33:37.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.210+0000 7efda77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efda809d6f0 0x7efda81a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:37.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.210+0000 7efda77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efda809d6f0 0x7efda81a6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55576/0 (socket says 172.21.15.16:55576) 2023-09-30T22:33:37.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.210+0000 7efda77fe700 1 -- 172.21.15.16:0/3021906136 learned_addr learned my addr 172.21.15.16:0/3021906136 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:37.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.210+0000 7efda7fff700 1 --2- 172.21.15.16:0/3021906136 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efda809e410 0x7efda81a6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:37.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.210+0000 7efda6ffd700 1 --2- 172.21.15.16:0/3021906136 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efda809e000 0x7efda81a6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:37.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.210+0000 7efda77fe700 1 -- 172.21.15.16:0/3021906136 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efda809e000 msgr2=0x7efda81a6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:37.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.210+0000 7efda77fe700 1 --2- 172.21.15.16:0/3021906136 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efda809e000 0x7efda81a6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:37.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.210+0000 7efda77fe700 1 -- 172.21.15.16:0/3021906136 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efda809e410 msgr2=0x7efda81a6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:37.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.210+0000 7efda77fe700 1 --2- 172.21.15.16:0/3021906136 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efda809e410 0x7efda81a6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:37.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.210+0000 7efda77fe700 1 -- 172.21.15.16:0/3021906136 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd9c009550 con 0x7efda809d6f0 2023-09-30T22:33:37.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.211+0000 7efda77fe700 1 --2- 172.21.15.16:0/3021906136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efda809d6f0 0x7efda81a6100 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7efd9800e8a0 tx=0x7efd9800ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:37.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.211+0000 7f8e2b7fe700 1 -- 172.21.15.16:0/803809883 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8e34017080 con 0x7f8e4409e930 2023-09-30T22:33:37.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.211+0000 7efd8ffff700 1 -- 172.21.15.16:0/3021906136 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd98009590 con 0x7efda809d6f0 2023-09-30T22:33:37.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.211+0000 7efd8ffff700 1 -- 172.21.15.16:0/3021906136 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd98013680 con 0x7efda809d6f0 2023-09-30T22:33:37.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.211+0000 7efd8ffff700 1 -- 172.21.15.16:0/3021906136 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd980139f0 con 0x7efda809d6f0 2023-09-30T22:33:37.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.211+0000 7efdae100700 1 -- 172.21.15.16:0/3021906136 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efda81a9940 con 0x7efda809d6f0 2023-09-30T22:33:37.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.211+0000 7efdae100700 1 -- 172.21.15.16:0/3021906136 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efda809c710 con 0x7efda809d6f0 2023-09-30T22:33:37.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.212+0000 7efd8ffff700 1 -- 172.21.15.16:0/3021906136 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7efd98013b50 con 0x7efda809d6f0 2023-09-30T22:33:37.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.212+0000 7efd8ffff700 1 -- 172.21.15.16:0/3021906136 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7efd9803f630 con 0x7efda809d6f0 2023-09-30T22:33:37.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.212+0000 7efdae100700 1 -- 172.21.15.16:0/3021906136 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efda8074ea0 con 0x7efda809d6f0 2023-09-30T22:33:37.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.217+0000 7efd8ffff700 1 -- 172.21.15.16:0/3021906136 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7efd98017020 con 0x7efda809d6f0 2023-09-30T22:33:37.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.367+0000 7fc22f7fe700 1 -- 172.21.15.16:0/2777184634 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc244074ea0 con 0x7fc2440a1510 2023-09-30T22:33:37.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.368+0000 7fc239ffb700 1 -- 172.21.15.16:0/2777184634 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc23401c450 con 0x7fc2440a1510 2023-09-30T22:33:37.368 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:33:37.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.370+0000 7fc22f7fe700 1 -- 172.21.15.16:0/2777184634 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2440a1510 msgr2=0x7fc2441c8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:37.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.370+0000 7fc22f7fe700 1 --2- 172.21.15.16:0/2777184634 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2440a1510 0x7fc2441c8a80 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc234009130 tx=0x7fc234004750 comp rx=0 tx=0).stop 2023-09-30T22:33:37.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.370+0000 7fc22f7fe700 1 -- 172.21.15.16:0/2777184634 shutdown_connections 2023-09-30T22:33:37.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.370+0000 7fc22f7fe700 1 --2- 172.21.15.16:0/2777184634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc244139250 0x7fc2441c8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:37.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.370+0000 7fc22f7fe700 1 --2- 172.21.15.16:0/2777184634 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2440a0a30 0x7fc2441c8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:37.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.370+0000 7fc22f7fe700 1 --2- 172.21.15.16:0/2777184634 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2440a1510 0x7fc2441c8a80 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:37.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.370+0000 7fc22f7fe700 1 -- 172.21.15.16:0/2777184634 >> 172.21.15.16:0/2777184634 conn(0x7fc2440a6550 msgr2=0x7fc244137b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:37.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.370+0000 7fc22f7fe700 1 -- 172.21.15.16:0/2777184634 shutdown_connections 2023-09-30T22:33:37.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.370+0000 7fc22f7fe700 1 -- 172.21.15.16:0/2777184634 wait complete. 2023-09-30T22:33:37.384 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:33:37.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.397+0000 7fde53336700 1 -- 172.21.15.16:0/776455318 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fde4c06b9e0 con 0x7fde4c06fb20 2023-09-30T22:33:37.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.398+0000 7fde41ffb700 1 -- 172.21.15.16:0/776455318 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fde4803ee60 con 0x7fde4c06fb20 2023-09-30T22:33:37.398 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:33:37.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.400+0000 7fde53336700 1 -- 172.21.15.16:0/776455318 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fde4c06fb20 msgr2=0x7fde4c118cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:37.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.400+0000 7fde53336700 1 --2- 172.21.15.16:0/776455318 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fde4c06fb20 0x7fde4c118cd0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fde4800b340 tx=0x7fde48004750 comp rx=0 tx=0).stop 2023-09-30T22:33:37.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.400+0000 7fde53336700 1 -- 172.21.15.16:0/776455318 shutdown_connections 2023-09-30T22:33:37.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.400+0000 7fde53336700 1 --2- 172.21.15.16:0/776455318 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fde4c130e50 0x7fde4c119750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:37.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.400+0000 7fde53336700 1 --2- 172.21.15.16:0/776455318 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fde4c06fb20 0x7fde4c118cd0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:37.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.400+0000 7fde53336700 1 --2- 172.21.15.16:0/776455318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde4c130540 0x7fde4c119210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:37.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.400+0000 7fde53336700 1 -- 172.21.15.16:0/776455318 >> 172.21.15.16:0/776455318 conn(0x7fde4c0a2aa0 msgr2=0x7fde4c12d450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:37.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.400+0000 7fde53336700 1 -- 172.21.15.16:0/776455318 shutdown_connections 2023-09-30T22:33:37.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.400+0000 7fde53336700 1 -- 172.21.15.16:0/776455318 wait complete. 2023-09-30T22:33:37.414 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:33:37.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.419+0000 7f8e49033700 1 -- 172.21.15.16:0/803809883 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8e24005590 con 0x7f8e4409e930 2023-09-30T22:33:37.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.420+0000 7f8e2b7fe700 1 -- 172.21.15.16:0/803809883 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8e34019d20 con 0x7f8e4409e930 2023-09-30T22:33:37.420 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:33:37.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.422+0000 7f8e49033700 1 -- 172.21.15.16:0/803809883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e4409e930 msgr2=0x7f8e4411b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:37.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.422+0000 7f8e49033700 1 --2- 172.21.15.16:0/803809883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e4409e930 0x7f8e4411b950 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f8e3400c3e0 tx=0x7f8e3400c6f0 comp rx=0 tx=0).stop 2023-09-30T22:33:37.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.422+0000 7f8e49033700 1 -- 172.21.15.16:0/803809883 shutdown_connections 2023-09-30T22:33:37.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.422+0000 7f8e49033700 1 --2- 172.21.15.16:0/803809883 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8e44136fe0 0x7f8e4411c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:37.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.422+0000 7f8e49033700 1 --2- 172.21.15.16:0/803809883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e4409e930 0x7f8e4411b950 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:37.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.422+0000 7f8e49033700 1 --2- 172.21.15.16:0/803809883 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8e4409f410 0x7f8e4411be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:37.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.422+0000 7f8e49033700 1 -- 172.21.15.16:0/803809883 >> 172.21.15.16:0/803809883 conn(0x7f8e440a4210 msgr2=0x7f8e440a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:37.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.422+0000 7f8e49033700 1 -- 172.21.15.16:0/803809883 shutdown_connections 2023-09-30T22:33:37.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.422+0000 7f8e49033700 1 -- 172.21.15.16:0/803809883 wait complete. 2023-09-30T22:33:37.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.428+0000 7efdae100700 1 -- 172.21.15.16:0/3021906136 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7efda806c360 con 0x7efda809d6f0 2023-09-30T22:33:37.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.428+0000 7efd8ffff700 1 -- 172.21.15.16:0/3021906136 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7efd9800c480 con 0x7efda809d6f0 2023-09-30T22:33:37.429 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:33:37.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.430+0000 7efdae100700 1 -- 172.21.15.16:0/3021906136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efda809d6f0 msgr2=0x7efda81a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:37.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.430+0000 7efdae100700 1 --2- 172.21.15.16:0/3021906136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efda809d6f0 0x7efda81a6100 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7efd9800e8a0 tx=0x7efd9800ebb0 comp rx=0 tx=0).stop 2023-09-30T22:33:37.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.430+0000 7efdae100700 1 -- 172.21.15.16:0/3021906136 shutdown_connections 2023-09-30T22:33:37.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.430+0000 7efdae100700 1 --2- 172.21.15.16:0/3021906136 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efda809e410 0x7efda81a6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:37.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.430+0000 7efdae100700 1 --2- 172.21.15.16:0/3021906136 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efda809d6f0 0x7efda81a6100 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:37.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.430+0000 7efdae100700 1 --2- 172.21.15.16:0/3021906136 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efda809e000 0x7efda81a6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:37.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.430+0000 7efdae100700 1 -- 172.21.15.16:0/3021906136 >> 172.21.15.16:0/3021906136 conn(0x7efda8066f50 msgr2=0x7efda81b9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:37.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.431+0000 7efdae100700 1 -- 172.21.15.16:0/3021906136 shutdown_connections 2023-09-30T22:33:37.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:37.431+0000 7efdae100700 1 -- 172.21.15.16:0/3021906136 wait complete. 2023-09-30T22:33:37.438 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:33:37.445 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:33:38.385 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:33:38.415 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:33:38.439 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:33:38.446 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:33:38.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.542+0000 7fa67dafb700 1 Processor -- start 2023-09-30T22:33:38.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.542+0000 7fa67dafb700 1 -- start start 2023-09-30T22:33:38.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.543+0000 7fa67dafb700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa67806f5d0 0x7fa67812ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:38.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.543+0000 7fa67dafb700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa678135c20 0x7fa678138000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:38.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.543+0000 7fa67dafb700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa67806fa40 con 0x7fa67812cf60 2023-09-30T22:33:38.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.543+0000 7fa67dafb700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa67812cd90 con 0x7fa678135c20 2023-09-30T22:33:38.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.543+0000 7fa67dafb700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa678071780 con 0x7fa67806f5d0 2023-09-30T22:33:38.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.543+0000 7fa6777fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa67812cf60 0x7fa67812f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38026/0 (socket says 172.21.15.16:38026) 2023-09-30T22:33:38.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.543+0000 7fa6777fe700 1 -- 172.21.15.16:0/2121801449 learned_addr learned my addr 172.21.15.16:0/2121801449 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:38.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.543+0000 7fa6767fc700 1 -- 172.21.15.16:0/2121801449 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1423551330 0 0) 0x7fa67806fa40 con 0x7fa67812cf60 2023-09-30T22:33:38.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.543+0000 7fa6767fc700 1 -- 172.21.15.16:0/2121801449 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6600046c0 con 0x7fa67812cf60 2023-09-30T22:33:38.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.544+0000 7fa6767fc700 1 -- 172.21.15.16:0/2121801449 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2737875850 0 0) 0x7fa6600046c0 con 0x7fa67812cf60 2023-09-30T22:33:38.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.544+0000 7fa6767fc700 1 -- 172.21.15.16:0/2121801449 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa67806f5d0 msgr2=0x7fa67812ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:38.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.544+0000 7fa6767fc700 1 --2- 172.21.15.16:0/2121801449 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa67806f5d0 0x7fa67812ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:38.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.544+0000 7fa6767fc700 1 -- 172.21.15.16:0/2121801449 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa678135c20 msgr2=0x7fa678138000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:38.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.544+0000 7fa677fff700 1 --2- 172.21.15.16:0/2121801449 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa678135c20 0x7fa678138000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:38.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.544+0000 7fa6767fc700 1 --2- 172.21.15.16:0/2121801449 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa678135c20 0x7fa678138000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:38.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.544+0000 7fa6767fc700 1 -- 172.21.15.16:0/2121801449 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa678071a40 con 0x7fa67812cf60 2023-09-30T22:33:38.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.544+0000 7fa6767fc700 1 -- 172.21.15.16:0/2121801449 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1074163509 0 0) 0x7fa668001670 con 0x7fa67812cf60 2023-09-30T22:33:38.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.544+0000 7fa6767fc700 1 -- 172.21.15.16:0/2121801449 >> v1:172.21.15.16:6789/0 conn(0x7fa67812cf60 legacy=0x7fa67812f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:38.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.544+0000 7fa6767fc700 1 --2- 172.21.15.16:0/2121801449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa660004b40 0x7fa660004f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:38.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.544+0000 7fa6767fc700 1 --2- 172.21.15.16:0/2121801449 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa67806f5d0 0x7fa660005960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:38.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.544+0000 7fa6767fc700 1 --2- 172.21.15.16:0/2121801449 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa660005f10 0x7fa660006380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:38.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.544+0000 7fa6767fc700 1 -- 172.21.15.16:0/2121801449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa67812cd90 con 0x7fa660004b40 2023-09-30T22:33:38.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.544+0000 7fa6767fc700 1 -- 172.21.15.16:0/2121801449 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa678071780 con 0x7fa660005f10 2023-09-30T22:33:38.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.544+0000 7fa6767fc700 1 -- 172.21.15.16:0/2121801449 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa66000ce20 con 0x7fa67806f5d0 2023-09-30T22:33:38.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.544+0000 7fa677fff700 1 --2- 172.21.15.16:0/2121801449 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa678135c20 0x7fa678138000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:33:38.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.544+0000 7fa677fff700 1 --2- 172.21.15.16:0/2121801449 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa660005f10 0x7fa660006380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:38.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.544+0000 7fa676ffd700 1 --2- 172.21.15.16:0/2121801449 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa67806f5d0 0x7fa660005960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:38.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.544+0000 7fa6777fe700 1 --2- 172.21.15.16:0/2121801449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa660004b40 0x7fa660004f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:38.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.545+0000 7fa6777fe700 1 -- 172.21.15.16:0/2121801449 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa67806f5d0 msgr2=0x7fa660005960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:38.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.545+0000 7fa6777fe700 1 --2- 172.21.15.16:0/2121801449 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa67806f5d0 0x7fa660005960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:38.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.545+0000 7fa6777fe700 1 -- 172.21.15.16:0/2121801449 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa660005f10 msgr2=0x7fa660006380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:38.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.545+0000 7fa6777fe700 1 --2- 172.21.15.16:0/2121801449 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa660005f10 0x7fa660006380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:38.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.545+0000 7fa6777fe700 1 -- 172.21.15.16:0/2121801449 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6600046c0 con 0x7fa660004b40 2023-09-30T22:33:38.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.545+0000 7fa677fff700 1 --2- 172.21.15.16:0/2121801449 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa660005f10 0x7fa660006380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:38.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.545+0000 7fa6777fe700 1 --2- 172.21.15.16:0/2121801449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa660004b40 0x7fa660004f50 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fa66800c240 tx=0x7fa660006990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f871b209fefca79 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:38.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.545+0000 7fa6767fc700 1 -- 172.21.15.16:0/2121801449 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa668007c10 con 0x7fa660004b40 2023-09-30T22:33:38.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.545+0000 7fa6767fc700 1 -- 172.21.15.16:0/2121801449 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa66800ce00 con 0x7fa660004b40 2023-09-30T22:33:38.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.545+0000 7fa6767fc700 1 -- 172.21.15.16:0/2121801449 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa668005ba0 con 0x7fa660004b40 2023-09-30T22:33:38.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.546+0000 7fa67dafb700 1 -- 172.21.15.16:0/2121801449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa660004b40 msgr2=0x7fa660004f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:38.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.546+0000 7fa67dafb700 1 --2- 172.21.15.16:0/2121801449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa660004b40 0x7fa660004f50 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fa66800c240 tx=0x7fa660006990 comp rx=0 tx=0).stop 2023-09-30T22:33:38.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.546+0000 7fa67dafb700 1 -- 172.21.15.16:0/2121801449 shutdown_connections 2023-09-30T22:33:38.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.546+0000 7fa67dafb700 1 --2- 172.21.15.16:0/2121801449 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa660005f10 0x7fa660006380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:38.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.546+0000 7fa67dafb700 1 --2- 172.21.15.16:0/2121801449 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa67806f5d0 0x7fa660005960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:38.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.546+0000 7fa67dafb700 1 --2- 172.21.15.16:0/2121801449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa660004b40 0x7fa660004f50 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:38.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.546+0000 7fa67dafb700 1 -- 172.21.15.16:0/2121801449 >> 172.21.15.16:0/2121801449 conn(0x7fa6780a2aa0 msgr2=0x7fa6780a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:38.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.546+0000 7fa67dafb700 1 -- 172.21.15.16:0/2121801449 shutdown_connections 2023-09-30T22:33:38.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.546+0000 7fa67dafb700 1 -- 172.21.15.16:0/2121801449 wait complete. 2023-09-30T22:33:38.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.547+0000 7fa67dafb700 1 Processor -- start 2023-09-30T22:33:38.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.548+0000 7fa67dafb700 1 -- start start 2023-09-30T22:33:38.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.548+0000 7fa67dafb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa660004b40 0x7fa678099ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:38.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.548+0000 7fa67dafb700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa660005f10 0x7fa67809a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:38.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.548+0000 7fa67dafb700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa67806f5d0 0x7fa67809a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:38.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.548+0000 7fa67dafb700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa67809ead0 con 0x7fa660004b40 2023-09-30T22:33:38.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.548+0000 7fa67dafb700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa678135710 con 0x7fa67806f5d0 2023-09-30T22:33:38.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.548+0000 7fa67dafb700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa6780a12d0 con 0x7fa660005f10 2023-09-30T22:33:38.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.549+0000 7fa676ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa660005f10 0x7fa67809a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:38.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.549+0000 7fa6777fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa660004b40 0x7fa678099ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:38.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.549+0000 7fa677fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa67806f5d0 0x7fa67809a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:38.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.549+0000 7fa6777fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa660004b40 0x7fa678099ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55898/0 (socket says 172.21.15.16:55898) 2023-09-30T22:33:38.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.549+0000 7fa677fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa67806f5d0 0x7fa67809a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:51688/0 (socket says 172.21.15.16:51688) 2023-09-30T22:33:38.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.549+0000 7fa6777fe700 1 -- 172.21.15.16:0/464492357 learned_addr learned my addr 172.21.15.16:0/464492357 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:38.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.549+0000 7fa676ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa660005f10 0x7fa67809a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:47336/0 (socket says 172.21.15.16:47336) 2023-09-30T22:33:38.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.549+0000 7fa677fff700 1 -- 172.21.15.16:0/464492357 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa660005f10 msgr2=0x7fa67809a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:38.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.549+0000 7fa677fff700 1 --2- 172.21.15.16:0/464492357 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa660005f10 0x7fa67809a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:38.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.549+0000 7fa677fff700 1 -- 172.21.15.16:0/464492357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa660004b40 msgr2=0x7fa678099ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:38.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.549+0000 7fa677fff700 1 --2- 172.21.15.16:0/464492357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa660004b40 0x7fa678099ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:38.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.549+0000 7fa677fff700 1 -- 172.21.15.16:0/464492357 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa678129fa0 con 0x7fa67806f5d0 2023-09-30T22:33:38.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.549+0000 7fa676ffd700 1 --2- 172.21.15.16:0/464492357 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa660005f10 0x7fa67809a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:38.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.549+0000 7fa6777fe700 1 --2- 172.21.15.16:0/464492357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa660004b40 0x7fa678099ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:38.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.549+0000 7fa677fff700 1 --2- 172.21.15.16:0/464492357 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa67806f5d0 0x7fa67809a550 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa67806dd30 tx=0x7fa67000ded0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:38.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.550+0000 7fa65ffff700 1 -- 172.21.15.16:0/464492357 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa670012070 con 0x7fa67806f5d0 2023-09-30T22:33:38.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.550+0000 7fa65ffff700 1 -- 172.21.15.16:0/464492357 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa67000cb30 con 0x7fa67806f5d0 2023-09-30T22:33:38.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.550+0000 7fa67dafb700 1 -- 172.21.15.16:0/464492357 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa67809c400 con 0x7fa67806f5d0 2023-09-30T22:33:38.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.550+0000 7fa65ffff700 1 -- 172.21.15.16:0/464492357 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa670003a20 con 0x7fa67806f5d0 2023-09-30T22:33:38.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.550+0000 7fa67dafb700 1 -- 172.21.15.16:0/464492357 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa678134d80 con 0x7fa67806f5d0 2023-09-30T22:33:38.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.550+0000 7fa65ffff700 1 -- 172.21.15.16:0/464492357 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa670017470 con 0x7fa67806f5d0 2023-09-30T22:33:38.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.551+0000 7fa65ffff700 1 -- 172.21.15.16:0/464492357 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fa67003fcd0 con 0x7fa67806f5d0 2023-09-30T22:33:38.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.551+0000 7fa67dafb700 1 -- 172.21.15.16:0/464492357 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa678074ea0 con 0x7fa67806f5d0 2023-09-30T22:33:38.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.556+0000 7fa65ffff700 1 -- 172.21.15.16:0/464492357 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa67000a440 con 0x7fa67806f5d0 2023-09-30T22:33:38.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.580+0000 7f0d1b6dc700 1 Processor -- start 2023-09-30T22:33:38.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.580+0000 7f0d1b6dc700 1 -- start start 2023-09-30T22:33:38.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.581+0000 7f0d1b6dc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0d1409e1d0 0x7f0d14138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:38.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.581+0000 7f0d1b6dc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0d14139170 0x7f0d1413b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:38.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.581+0000 7f0d1b6dc700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d1409b120 con 0x7f0d1409d6f0 2023-09-30T22:33:38.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.581+0000 7f0d1b6dc700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d1406b460 con 0x7f0d14139170 2023-09-30T22:33:38.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.581+0000 7f0d1b6dc700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1409a7e0 con 0x7f0d1409e1d0 2023-09-30T22:33:38.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.581+0000 7f0d19c79700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0d14139170 0x7f0d1413b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:38.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.581+0000 7f0d18c77700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0d1409e1d0 0x7f0d14138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:38.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.581+0000 7f0d19c79700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0d14139170 0x7f0d1413b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55900/0 (socket says 172.21.15.16:55900) 2023-09-30T22:33:38.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.581+0000 7f0d19c79700 1 -- 172.21.15.16:0/157593211 learned_addr learned my addr 172.21.15.16:0/157593211 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:38.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.581+0000 7f0d18c77700 1 -- 172.21.15.16:0/157593211 >> v2:172.21.15.16:3300/0 conn(0x7f0d14139170 msgr2=0x7f0d1413b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:38.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.581+0000 7f0d18c77700 1 --2- 172.21.15.16:0/157593211 >> v2:172.21.15.16:3300/0 conn(0x7f0d14139170 0x7f0d1413b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:38.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.581+0000 7f0d18c77700 1 -- 172.21.15.16:0/157593211 >> v1:172.21.15.16:6789/0 conn(0x7f0d1409d6f0 legacy=0x7f0d1409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:38.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.581+0000 7f0d18c77700 1 -- 172.21.15.16:0/157593211 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d1409a500 con 0x7f0d1409e1d0 2023-09-30T22:33:38.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.582+0000 7f0d19c79700 1 --2- 172.21.15.16:0/157593211 >> v2:172.21.15.16:3300/0 conn(0x7f0d14139170 0x7f0d1413b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:38.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.582+0000 7f0d18c77700 1 --2- 172.21.15.16:0/157593211 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0d1409e1d0 0x7f0d14138c30 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f0d04009870 tx=0x7f0d04009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e96009119a6a5514 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:38.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.582+0000 7f0d0bfff700 1 -- 172.21.15.16:0/157593211 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d04011830 con 0x7f0d1409e1d0 2023-09-30T22:33:38.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.582+0000 7f0d0bfff700 1 -- 172.21.15.16:0/157593211 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d04011990 con 0x7f0d1409e1d0 2023-09-30T22:33:38.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.582+0000 7f0d0bfff700 1 -- 172.21.15.16:0/157593211 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d04011d20 con 0x7f0d1409e1d0 2023-09-30T22:33:38.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.583+0000 7f0d1b6dc700 1 -- 172.21.15.16:0/157593211 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0d1409e1d0 msgr2=0x7f0d14138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:38.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.583+0000 7f0d1b6dc700 1 --2- 172.21.15.16:0/157593211 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0d1409e1d0 0x7f0d14138c30 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f0d04009870 tx=0x7f0d04009b80 comp rx=0 tx=0).stop 2023-09-30T22:33:38.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.583+0000 7f0d1b6dc700 1 -- 172.21.15.16:0/157593211 shutdown_connections 2023-09-30T22:33:38.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.583+0000 7f0d1b6dc700 1 --2- 172.21.15.16:0/157593211 >> v2:172.21.15.16:3300/0 conn(0x7f0d14139170 0x7f0d1413b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:38.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.583+0000 7f0d1b6dc700 1 --2- 172.21.15.16:0/157593211 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0d1409e1d0 0x7f0d14138c30 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:38.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.583+0000 7f0d1b6dc700 1 -- 172.21.15.16:0/157593211 >> 172.21.15.16:0/157593211 conn(0x7f0d14066f50 msgr2=0x7f0d1409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:38.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.583+0000 7f0d1b6dc700 1 -- 172.21.15.16:0/157593211 shutdown_connections 2023-09-30T22:33:38.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.583+0000 7f0d1b6dc700 1 -- 172.21.15.16:0/157593211 wait complete. 2023-09-30T22:33:38.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.584+0000 7f0d1b6dc700 1 Processor -- start 2023-09-30T22:33:38.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.584+0000 7f0d1b6dc700 1 -- start start 2023-09-30T22:33:38.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.585+0000 7f0d1b6dc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0d1409d6f0 0x7f0d141c8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:38.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.585+0000 7f0d1b6dc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d1409e1d0 0x7f0d141c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:38.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.585+0000 7f0d1b6dc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0d14139170 0x7f0d1411e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:38.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.585+0000 7f0d1b6dc700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d141220e0 con 0x7f0d1409e1d0 2023-09-30T22:33:38.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.585+0000 7f0d1b6dc700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d141222c0 con 0x7f0d14139170 2023-09-30T22:33:38.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.585+0000 7f0d1b6dc700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0d141be250 con 0x7f0d1409d6f0 2023-09-30T22:33:38.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.585+0000 7f0d18c77700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d1409e1d0 0x7f0d141c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:38.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.586+0000 7f0d18c77700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d1409e1d0 0x7f0d141c8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55908/0 (socket says 172.21.15.16:55908) 2023-09-30T22:33:38.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.586+0000 7f0d18c77700 1 -- 172.21.15.16:0/992554515 learned_addr learned my addr 172.21.15.16:0/992554515 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:38.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.586+0000 7f0d19c79700 1 --2- 172.21.15.16:0/992554515 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0d14139170 0x7f0d1411e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:38.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.586+0000 7f0d19478700 1 --2- 172.21.15.16:0/992554515 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0d1409d6f0 0x7f0d141c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:38.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.586+0000 7f0d19c79700 1 -- 172.21.15.16:0/992554515 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0d1409d6f0 msgr2=0x7f0d141c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:38.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.586+0000 7f0d19c79700 1 --2- 172.21.15.16:0/992554515 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0d1409d6f0 0x7f0d141c8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:38.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.586+0000 7f0d19c79700 1 -- 172.21.15.16:0/992554515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d1409e1d0 msgr2=0x7f0d141c8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:38.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.586+0000 7f0d19c79700 1 --2- 172.21.15.16:0/992554515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d1409e1d0 0x7f0d141c8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:38.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.586+0000 7f0d19c79700 1 -- 172.21.15.16:0/992554515 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d0c009b30 con 0x7f0d14139170 2023-09-30T22:33:38.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.586+0000 7f0d18c77700 1 --2- 172.21.15.16:0/992554515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d1409e1d0 0x7f0d141c8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:38.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.586+0000 7f0d19c79700 1 --2- 172.21.15.16:0/992554515 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0d14139170 0x7f0d1411e7a0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c009e50 tx=0x7f0d0c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:38.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.587+0000 7f0d09ffb700 1 -- 172.21.15.16:0/992554515 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c018070 con 0x7f0d14139170 2023-09-30T22:33:38.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.587+0000 7f0d09ffb700 1 -- 172.21.15.16:0/992554515 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d0c00f040 con 0x7f0d14139170 2023-09-30T22:33:38.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.587+0000 7f0d1b6dc700 1 -- 172.21.15.16:0/992554515 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d040094a0 con 0x7f0d14139170 2023-09-30T22:33:38.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.587+0000 7f0d09ffb700 1 -- 172.21.15.16:0/992554515 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c003c40 con 0x7f0d14139170 2023-09-30T22:33:38.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.587+0000 7f0d1b6dc700 1 -- 172.21.15.16:0/992554515 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d14121d20 con 0x7f0d14139170 2023-09-30T22:33:38.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.588+0000 7f0d09ffb700 1 -- 172.21.15.16:0/992554515 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0d0c00c930 con 0x7f0d14139170 2023-09-30T22:33:38.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.588+0000 7f0d09ffb700 1 -- 172.21.15.16:0/992554515 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f0d0c03f800 con 0x7f0d14139170 2023-09-30T22:33:38.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.588+0000 7f0d1b6dc700 1 -- 172.21.15.16:0/992554515 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d14074ea0 con 0x7f0d14139170 2023-09-30T22:33:38.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.598+0000 7f0d09ffb700 1 -- 172.21.15.16:0/992554515 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0d0c00cbe0 con 0x7f0d14139170 2023-09-30T22:33:38.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.627+0000 7f67ee883700 1 Processor -- start 2023-09-30T22:33:38.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.627+0000 7f67ee883700 1 -- start start 2023-09-30T22:33:38.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.628+0000 7f67ee883700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f67e809d6f0 0x7f67e809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:38.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.628+0000 7f67ee883700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f67e809e000 0x7f67e8130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:38.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.628+0000 7f67ee883700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67e809f130 con 0x7f67e809e410 2023-09-30T22:33:38.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.628+0000 7f67ee883700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67e809f520 con 0x7f67e809d6f0 2023-09-30T22:33:38.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.628+0000 7f67ee883700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67e809e930 con 0x7f67e809e000 2023-09-30T22:33:38.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.628+0000 7f67e7fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f67e809d6f0 0x7f67e809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:38.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.628+0000 7f67e7fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f67e809d6f0 0x7f67e809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55922/0 (socket says 172.21.15.16:55922) 2023-09-30T22:33:38.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.628+0000 7f67e7fff700 1 -- 172.21.15.16:0/3168730367 learned_addr learned my addr 172.21.15.16:0/3168730367 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:38.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.628+0000 7f67e77fe700 1 --2- 172.21.15.16:0/3168730367 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f67e809e000 0x7f67e8130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:38.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.628+0000 7f67e7fff700 1 -- 172.21.15.16:0/3168730367 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f67e809e000 msgr2=0x7f67e8130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:38.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.628+0000 7f67e7fff700 1 --2- 172.21.15.16:0/3168730367 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f67e809e000 0x7f67e8130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:38.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.628+0000 7f67e7fff700 1 -- 172.21.15.16:0/3168730367 >> v1:172.21.15.16:6789/0 conn(0x7f67e809e410 legacy=0x7f67e8130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:38.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.628+0000 7f67e7fff700 1 -- 172.21.15.16:0/3168730367 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e809b2d0 con 0x7f67e809d6f0 2023-09-30T22:33:38.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.630+0000 7f67e7fff700 1 --2- 172.21.15.16:0/3168730367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67e809d6f0 0x7f67e809dac0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f67d80098f0 tx=0x7f67d8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ea5905cb4bbc362 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:38.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.630+0000 7f67e6ffd700 1 -- 172.21.15.16:0/3168730367 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d8011830 con 0x7f67e809d6f0 2023-09-30T22:33:38.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.630+0000 7f67e6ffd700 1 -- 172.21.15.16:0/3168730367 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67d8011990 con 0x7f67e809d6f0 2023-09-30T22:33:38.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.630+0000 7f67e6ffd700 1 -- 172.21.15.16:0/3168730367 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d8011d20 con 0x7f67e809d6f0 2023-09-30T22:33:38.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.630+0000 7f67ee883700 1 -- 172.21.15.16:0/3168730367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67e809d6f0 msgr2=0x7f67e809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:38.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.630+0000 7f67ee883700 1 --2- 172.21.15.16:0/3168730367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67e809d6f0 0x7f67e809dac0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f67d80098f0 tx=0x7f67d8009c00 comp rx=0 tx=0).stop 2023-09-30T22:33:38.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.631+0000 7f67ee883700 1 -- 172.21.15.16:0/3168730367 shutdown_connections 2023-09-30T22:33:38.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.631+0000 7f67ee883700 1 --2- 172.21.15.16:0/3168730367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67e809d6f0 0x7f67e809dac0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:38.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.631+0000 7f67ee883700 1 --2- 172.21.15.16:0/3168730367 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f67e809e000 0x7f67e8130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:38.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.631+0000 7f67ee883700 1 -- 172.21.15.16:0/3168730367 >> 172.21.15.16:0/3168730367 conn(0x7f67e8066f50 msgr2=0x7f67e809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:38.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.631+0000 7f67ee883700 1 -- 172.21.15.16:0/3168730367 shutdown_connections 2023-09-30T22:33:38.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.631+0000 7f67ee883700 1 -- 172.21.15.16:0/3168730367 wait complete. 2023-09-30T22:33:38.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.632+0000 7f67ee883700 1 Processor -- start 2023-09-30T22:33:38.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.632+0000 7f67ee883700 1 -- start start 2023-09-30T22:33:38.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.632+0000 7f67ee883700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67e809d6f0 0x7f67e8136510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:38.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.632+0000 7f67ee883700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f67e809e000 0x7f67e8136a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:38.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.632+0000 7f67ee883700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f67e809e410 0x7f67e8136f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:38.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.632+0000 7f67ee883700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e81379e0 con 0x7f67e809d6f0 2023-09-30T22:33:38.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.632+0000 7f67ee883700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67e813ad10 con 0x7f67e809e000 2023-09-30T22:33:38.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.632+0000 7f67ee883700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f67e8139a90 con 0x7f67e809e410 2023-09-30T22:33:38.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.633+0000 7f67e7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67e809d6f0 0x7f67e8136510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:38.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.633+0000 7f67e7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67e809d6f0 0x7f67e8136510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55938/0 (socket says 172.21.15.16:55938) 2023-09-30T22:33:38.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.633+0000 7f67e7fff700 1 -- 172.21.15.16:0/2899462984 learned_addr learned my addr 172.21.15.16:0/2899462984 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:38.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.633+0000 7f67e77fe700 1 --2- 172.21.15.16:0/2899462984 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f67e809e000 0x7f67e8136a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:38.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.633+0000 7f67e7fff700 1 -- 172.21.15.16:0/2899462984 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f67e809e410 msgr2=0x7f67e8136f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:38.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.633+0000 7f67ece20700 1 --2- 172.21.15.16:0/2899462984 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f67e809e410 0x7f67e8136f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:38.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.633+0000 7f67e7fff700 1 --2- 172.21.15.16:0/2899462984 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f67e809e410 0x7f67e8136f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:38.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.633+0000 7f67e7fff700 1 -- 172.21.15.16:0/2899462984 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f67e809e000 msgr2=0x7f67e8136a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:38.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.633+0000 7f67e7fff700 1 --2- 172.21.15.16:0/2899462984 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f67e809e000 0x7f67e8136a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:38.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.633+0000 7f67e7fff700 1 -- 172.21.15.16:0/2899462984 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d8009550 con 0x7f67e809d6f0 2023-09-30T22:33:38.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.633+0000 7f67e77fe700 1 --2- 172.21.15.16:0/2899462984 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f67e809e000 0x7f67e8136a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:33:38.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.633+0000 7f67ece20700 1 --2- 172.21.15.16:0/2899462984 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f67e809e410 0x7f67e8136f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:33:38.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.633+0000 7f67e7fff700 1 --2- 172.21.15.16:0/2899462984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67e809d6f0 0x7f67e8136510 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f67d8009c40 tx=0x7f67d8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:38.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.633+0000 7f67e4ff9700 1 -- 172.21.15.16:0/2899462984 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d8004530 con 0x7f67e809d6f0 2023-09-30T22:33:38.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.633+0000 7f67e4ff9700 1 -- 172.21.15.16:0/2899462984 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67d801f070 con 0x7f67e809d6f0 2023-09-30T22:33:38.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.634+0000 7f67e4ff9700 1 -- 172.21.15.16:0/2899462984 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d80038c0 con 0x7f67e809d6f0 2023-09-30T22:33:38.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.634+0000 7f67ee883700 1 -- 172.21.15.16:0/2899462984 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67e80a1d00 con 0x7f67e809d6f0 2023-09-30T22:33:38.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.634+0000 7f67ee883700 1 -- 172.21.15.16:0/2899462984 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67e813a7c0 con 0x7f67e809d6f0 2023-09-30T22:33:38.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.634+0000 7f67e4ff9700 1 -- 172.21.15.16:0/2899462984 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f67d8021030 con 0x7f67e809d6f0 2023-09-30T22:33:38.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.635+0000 7f67e4ff9700 1 -- 172.21.15.16:0/2899462984 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f67d8040b30 con 0x7f67e809d6f0 2023-09-30T22:33:38.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.635+0000 7f67ee883700 1 -- 172.21.15.16:0/2899462984 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67e8074ea0 con 0x7f67e809d6f0 2023-09-30T22:33:38.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.638+0000 7f0c4deac700 1 Processor -- start 2023-09-30T22:33:38.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.638+0000 7f0c4deac700 1 -- start start 2023-09-30T22:33:38.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.639+0000 7f0c4deac700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0c4809e1d0 0x7f0c48138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:38.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.639+0000 7f0c4deac700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0c48139170 0x7f0c4813b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:38.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.639+0000 7f0c4deac700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c4809b120 con 0x7f0c4809d6f0 2023-09-30T22:33:38.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.639+0000 7f0c4deac700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4806b460 con 0x7f0c4809e1d0 2023-09-30T22:33:38.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.639+0000 7f0c4deac700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4809a7e0 con 0x7f0c48139170 2023-09-30T22:33:38.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.639+0000 7f0c477fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0c4809d6f0 0x7f0c4809dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38058/0 (socket says 172.21.15.16:38058) 2023-09-30T22:33:38.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.639+0000 7f0c477fe700 1 -- 172.21.15.16:0/2332336628 learned_addr learned my addr 172.21.15.16:0/2332336628 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:38.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.639+0000 7f0c46ffd700 1 --2- 172.21.15.16:0/2332336628 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0c4809e1d0 0x7f0c48138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:38.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.639+0000 7f0c46ffd700 1 -- 172.21.15.16:0/2332336628 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0c48139170 msgr2=0x7f0c4813b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:38.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.639+0000 7f0c46ffd700 1 --2- 172.21.15.16:0/2332336628 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0c48139170 0x7f0c4813b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:38.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.639+0000 7f0c46ffd700 1 -- 172.21.15.16:0/2332336628 >> v1:172.21.15.16:6789/0 conn(0x7f0c4809d6f0 legacy=0x7f0c4809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:38.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.639+0000 7f0c46ffd700 1 -- 172.21.15.16:0/2332336628 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c4809a500 con 0x7f0c4809e1d0 2023-09-30T22:33:38.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.639+0000 7f0c46ffd700 1 --2- 172.21.15.16:0/2332336628 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0c4809e1d0 0x7f0c48138c30 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c009870 tx=0x7f0c3c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bcabf9ca8a9cd14 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:38.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.639+0000 7f0c467fc700 1 -- 172.21.15.16:0/2332336628 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c011830 con 0x7f0c4809e1d0 2023-09-30T22:33:38.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.640+0000 7f0c467fc700 1 -- 172.21.15.16:0/2332336628 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c3c011990 con 0x7f0c4809e1d0 2023-09-30T22:33:38.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.640+0000 7f0c467fc700 1 -- 172.21.15.16:0/2332336628 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c011d20 con 0x7f0c4809e1d0 2023-09-30T22:33:38.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.640+0000 7f0c4deac700 1 -- 172.21.15.16:0/2332336628 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0c4809e1d0 msgr2=0x7f0c48138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:38.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.640+0000 7f0c4deac700 1 --2- 172.21.15.16:0/2332336628 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0c4809e1d0 0x7f0c48138c30 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c009870 tx=0x7f0c3c009b80 comp rx=0 tx=0).stop 2023-09-30T22:33:38.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.641+0000 7f0c4deac700 1 -- 172.21.15.16:0/2332336628 shutdown_connections 2023-09-30T22:33:38.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.641+0000 7f0c4deac700 1 --2- 172.21.15.16:0/2332336628 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0c48139170 0x7f0c4813b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:38.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.641+0000 7f0c4deac700 1 --2- 172.21.15.16:0/2332336628 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0c4809e1d0 0x7f0c48138c30 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:38.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.641+0000 7f0c4deac700 1 -- 172.21.15.16:0/2332336628 >> 172.21.15.16:0/2332336628 conn(0x7f0c48066f50 msgr2=0x7f0c4809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:38.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.641+0000 7f0c4deac700 1 -- 172.21.15.16:0/2332336628 shutdown_connections 2023-09-30T22:33:38.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.641+0000 7f0c4deac700 1 -- 172.21.15.16:0/2332336628 wait complete. 2023-09-30T22:33:38.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.641+0000 7f67e4ff9700 1 -- 172.21.15.16:0/2899462984 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f67d8015080 con 0x7f67e809d6f0 2023-09-30T22:33:38.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.642+0000 7f0c4deac700 1 Processor -- start 2023-09-30T22:33:38.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.642+0000 7f0c4deac700 1 -- start start 2023-09-30T22:33:38.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.643+0000 7f0c4deac700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0c4809d6f0 0x7f0c4811e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:38.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.643+0000 7f0c4deac700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0c4809e1d0 0x7f0c4811eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:38.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.643+0000 7f0c4deac700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c48139170 0x7f0c4811f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:38.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.643+0000 7f0c4deac700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c48120620 con 0x7f0c48139170 2023-09-30T22:33:38.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.643+0000 7f0c4deac700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c481347f0 con 0x7f0c4809d6f0 2023-09-30T22:33:38.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.643+0000 7f0c4deac700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0c480a20f0 con 0x7f0c4809e1d0 2023-09-30T22:33:38.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.643+0000 7f0c47fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c48139170 0x7f0c4811f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:38.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.643+0000 7f0c46ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0c4809e1d0 0x7f0c4811eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:38.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.643+0000 7f0c46ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0c4809e1d0 0x7f0c4811eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:47368/0 (socket says 172.21.15.16:47368) 2023-09-30T22:33:38.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.643+0000 7f0c47fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c48139170 0x7f0c4811f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55948/0 (socket says 172.21.15.16:55948) 2023-09-30T22:33:38.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.643+0000 7f0c46ffd700 1 -- 172.21.15.16:0/3722846349 learned_addr learned my addr 172.21.15.16:0/3722846349 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:38.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.643+0000 7f0c477fe700 1 --2- 172.21.15.16:0/3722846349 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0c4809d6f0 0x7f0c4811e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:38.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.643+0000 7f0c46ffd700 1 -- 172.21.15.16:0/3722846349 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0c4809d6f0 msgr2=0x7f0c4811e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:38.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.643+0000 7f0c46ffd700 1 --2- 172.21.15.16:0/3722846349 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0c4809d6f0 0x7f0c4811e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:38.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.643+0000 7f0c46ffd700 1 -- 172.21.15.16:0/3722846349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c48139170 msgr2=0x7f0c4811f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:38.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.643+0000 7f0c46ffd700 1 --2- 172.21.15.16:0/3722846349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c48139170 0x7f0c4811f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:38.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.644+0000 7f0c46ffd700 1 -- 172.21.15.16:0/3722846349 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c3c0094a0 con 0x7f0c4809e1d0 2023-09-30T22:33:38.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.644+0000 7f0c46ffd700 1 --2- 172.21.15.16:0/3722846349 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0c4809e1d0 0x7f0c4811eed0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c0090b0 tx=0x7f0c3c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:38.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.644+0000 7f0c2ffff700 1 -- 172.21.15.16:0/3722846349 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c004510 con 0x7f0c4809e1d0 2023-09-30T22:33:38.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.644+0000 7f0c2ffff700 1 -- 172.21.15.16:0/3722846349 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c3c01f070 con 0x7f0c4809e1d0 2023-09-30T22:33:38.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.644+0000 7f0c2ffff700 1 -- 172.21.15.16:0/3722846349 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c0038e0 con 0x7f0c4809e1d0 2023-09-30T22:33:38.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.644+0000 7f0c4deac700 1 -- 172.21.15.16:0/3722846349 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c48121bb0 con 0x7f0c4809e1d0 2023-09-30T22:33:38.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.645+0000 7f0c4deac700 1 -- 172.21.15.16:0/3722846349 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c48120cc0 con 0x7f0c4809e1d0 2023-09-30T22:33:38.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.646+0000 7f0c2ffff700 1 -- 172.21.15.16:0/3722846349 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0c3c003a40 con 0x7f0c4809e1d0 2023-09-30T22:33:38.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.646+0000 7f0c2ffff700 1 -- 172.21.15.16:0/3722846349 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f0c3c021070 con 0x7f0c4809e1d0 2023-09-30T22:33:38.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.646+0000 7f0c4deac700 1 -- 172.21.15.16:0/3722846349 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c48121da0 con 0x7f0c4809e1d0 2023-09-30T22:33:38.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.656+0000 7f0c2ffff700 1 -- 172.21.15.16:0/3722846349 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0c3c01a430 con 0x7f0c4809e1d0 2023-09-30T22:33:38.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.789+0000 7fa67dafb700 1 -- 172.21.15.16:0/464492357 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa6780713a0 con 0x7fa67806f5d0 2023-09-30T22:33:38.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.789+0000 7fa65ffff700 1 -- 172.21.15.16:0/464492357 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa6700093a0 con 0x7fa67806f5d0 2023-09-30T22:33:38.790 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:33:38.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.792+0000 7fa67dafb700 1 -- 172.21.15.16:0/464492357 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa67806f5d0 msgr2=0x7fa67809a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:38.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.792+0000 7fa67dafb700 1 --2- 172.21.15.16:0/464492357 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa67806f5d0 0x7fa67809a550 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa67806dd30 tx=0x7fa67000ded0 comp rx=0 tx=0).stop 2023-09-30T22:33:38.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.792+0000 7fa67dafb700 1 -- 172.21.15.16:0/464492357 shutdown_connections 2023-09-30T22:33:38.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.792+0000 7fa67dafb700 1 --2- 172.21.15.16:0/464492357 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa67806f5d0 0x7fa67809a550 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:38.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.792+0000 7fa67dafb700 1 --2- 172.21.15.16:0/464492357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa660004b40 0x7fa678099ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:38.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.792+0000 7fa67dafb700 1 --2- 172.21.15.16:0/464492357 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa660005f10 0x7fa67809a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:38.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.792+0000 7fa67dafb700 1 -- 172.21.15.16:0/464492357 >> 172.21.15.16:0/464492357 conn(0x7fa6780a2aa0 msgr2=0x7fa67806e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:38.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.792+0000 7fa67dafb700 1 -- 172.21.15.16:0/464492357 shutdown_connections 2023-09-30T22:33:38.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.792+0000 7fa67dafb700 1 -- 172.21.15.16:0/464492357 wait complete. 2023-09-30T22:33:38.807 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:33:38.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.830+0000 7f0d1b6dc700 1 -- 172.21.15.16:0/992554515 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f0d1409f420 con 0x7f0d14139170 2023-09-30T22:33:38.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.830+0000 7f0d09ffb700 1 -- 172.21.15.16:0/992554515 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0d0c01de00 con 0x7f0d14139170 2023-09-30T22:33:38.831 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:33:38.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.833+0000 7f0d1b6dc700 1 -- 172.21.15.16:0/992554515 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0d14139170 msgr2=0x7f0d1411e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:38.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.833+0000 7f0d1b6dc700 1 --2- 172.21.15.16:0/992554515 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0d14139170 0x7f0d1411e7a0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c009e50 tx=0x7f0d0c00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:33:38.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.833+0000 7f0d1b6dc700 1 -- 172.21.15.16:0/992554515 shutdown_connections 2023-09-30T22:33:38.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.833+0000 7f0d1b6dc700 1 --2- 172.21.15.16:0/992554515 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0d14139170 0x7f0d1411e7a0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:38.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.833+0000 7f0d1b6dc700 1 --2- 172.21.15.16:0/992554515 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0d1409d6f0 0x7f0d141c8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:38.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.833+0000 7f0d1b6dc700 1 --2- 172.21.15.16:0/992554515 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0d1409e1d0 0x7f0d141c8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:38.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.833+0000 7f0d1b6dc700 1 -- 172.21.15.16:0/992554515 >> 172.21.15.16:0/992554515 conn(0x7f0d14066f50 msgr2=0x7f0d140a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:38.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.833+0000 7f0d1b6dc700 1 -- 172.21.15.16:0/992554515 shutdown_connections 2023-09-30T22:33:38.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.833+0000 7f0d1b6dc700 1 -- 172.21.15.16:0/992554515 wait complete. 2023-09-30T22:33:38.847 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:33:38.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.851+0000 7f67ee883700 1 -- 172.21.15.16:0/2899462984 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f67e806c360 con 0x7f67e809d6f0 2023-09-30T22:33:38.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.851+0000 7f67e4ff9700 1 -- 172.21.15.16:0/2899462984 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f67d80199a0 con 0x7f67e809d6f0 2023-09-30T22:33:38.851 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:33:38.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.853+0000 7f67ee883700 1 -- 172.21.15.16:0/2899462984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67e809d6f0 msgr2=0x7f67e8136510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:38.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.854+0000 7f67ee883700 1 --2- 172.21.15.16:0/2899462984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67e809d6f0 0x7f67e8136510 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f67d8009c40 tx=0x7f67d8004750 comp rx=0 tx=0).stop 2023-09-30T22:33:38.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.854+0000 7f67ee883700 1 -- 172.21.15.16:0/2899462984 shutdown_connections 2023-09-30T22:33:38.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.854+0000 7f67ee883700 1 --2- 172.21.15.16:0/2899462984 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f67e809e410 0x7f67e8136f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:38.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.854+0000 7f67ee883700 1 --2- 172.21.15.16:0/2899462984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67e809d6f0 0x7f67e8136510 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:38.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.854+0000 7f67ee883700 1 --2- 172.21.15.16:0/2899462984 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f67e809e000 0x7f67e8136a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:38.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.854+0000 7f67ee883700 1 -- 172.21.15.16:0/2899462984 >> 172.21.15.16:0/2899462984 conn(0x7f67e8066f50 msgr2=0x7f67e80a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:38.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.854+0000 7f67ee883700 1 -- 172.21.15.16:0/2899462984 shutdown_connections 2023-09-30T22:33:38.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.854+0000 7f67ee883700 1 -- 172.21.15.16:0/2899462984 wait complete. 2023-09-30T22:33:38.868 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:33:38.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.881+0000 7f0c4deac700 1 -- 172.21.15.16:0/3722846349 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0c48074ea0 con 0x7f0c4809e1d0 2023-09-30T22:33:38.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.881+0000 7f0c2ffff700 1 -- 172.21.15.16:0/3722846349 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0c3c015020 con 0x7f0c4809e1d0 2023-09-30T22:33:38.881 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:33:38.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.883+0000 7f0c4deac700 1 -- 172.21.15.16:0/3722846349 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0c4809e1d0 msgr2=0x7f0c4811eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:38.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.883+0000 7f0c4deac700 1 --2- 172.21.15.16:0/3722846349 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0c4809e1d0 0x7f0c4811eed0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c0090b0 tx=0x7f0c3c004750 comp rx=0 tx=0).stop 2023-09-30T22:33:38.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.883+0000 7f0c4deac700 1 -- 172.21.15.16:0/3722846349 shutdown_connections 2023-09-30T22:33:38.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.883+0000 7f0c4deac700 1 --2- 172.21.15.16:0/3722846349 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0c4809d6f0 0x7f0c4811e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:38.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.883+0000 7f0c4deac700 1 --2- 172.21.15.16:0/3722846349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c48139170 0x7f0c4811f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:38.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.883+0000 7f0c4deac700 1 --2- 172.21.15.16:0/3722846349 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0c4809e1d0 0x7f0c4811eed0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:38.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.883+0000 7f0c4deac700 1 -- 172.21.15.16:0/3722846349 >> 172.21.15.16:0/3722846349 conn(0x7f0c48066f50 msgr2=0x7f0c48136cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:38.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.884+0000 7f0c4deac700 1 -- 172.21.15.16:0/3722846349 shutdown_connections 2023-09-30T22:33:38.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:38.884+0000 7f0c4deac700 1 -- 172.21.15.16:0/3722846349 wait complete. 2023-09-30T22:33:38.897 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:33:39.808 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:33:39.848 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:33:39.869 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:33:39.898 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:33:39.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:39.960+0000 7f31ead03700 1 Processor -- start 2023-09-30T22:33:39.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:39.960+0000 7f31ead03700 1 -- start start 2023-09-30T22:33:39.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:39.960+0000 7f31ead03700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f31e406a770 0x7f31e409d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:39.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:39.960+0000 7f31ead03700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f31e409dc30 0x7f31e409e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:39.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:39.960+0000 7f31ead03700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31e409a810 con 0x7f31e413c1e0 2023-09-30T22:33:39.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:39.960+0000 7f31ead03700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31e406ab40 con 0x7f31e406a770 2023-09-30T22:33:39.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:39.960+0000 7f31ead03700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f31e409ebc0 con 0x7f31e409dc30 2023-09-30T22:33:39.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:39.960+0000 7f31e8a9f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f31e406a770 0x7f31e409d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:39.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:39.960+0000 7f31e92a0700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f31e413c1e0 0x7f31e413e5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38070/0 (socket says 172.21.15.16:38070) 2023-09-30T22:33:39.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:39.961+0000 7f31e8a9f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f31e406a770 0x7f31e409d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55964/0 (socket says 172.21.15.16:55964) 2023-09-30T22:33:39.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:39.961+0000 7f31e92a0700 1 -- 172.21.15.16:0/3032696449 learned_addr learned my addr 172.21.15.16:0/3032696449 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:39.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:39.961+0000 7f31e3fff700 1 --2- 172.21.15.16:0/3032696449 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f31e409dc30 0x7f31e409e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:39.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:39.961+0000 7f31e37fe700 1 -- 172.21.15.16:0/3032696449 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1664802077 0 0) 0x7f31e409a810 con 0x7f31e413c1e0 2023-09-30T22:33:39.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:39.961+0000 7f31e8a9f700 1 -- 172.21.15.16:0/3032696449 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f31e409dc30 msgr2=0x7f31e409e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:39.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:39.961+0000 7f31e8a9f700 1 --2- 172.21.15.16:0/3032696449 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f31e409dc30 0x7f31e409e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:39.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:39.961+0000 7f31e8a9f700 1 -- 172.21.15.16:0/3032696449 >> v1:172.21.15.16:6789/0 conn(0x7f31e413c1e0 legacy=0x7f31e413e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:39.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:39.961+0000 7f31e8a9f700 1 -- 172.21.15.16:0/3032696449 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e409e920 con 0x7f31e406a770 2023-09-30T22:33:39.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:39.961+0000 7f31e8a9f700 1 --2- 172.21.15.16:0/3032696449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31e406a770 0x7f31e409d6f0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f31d8009840 tx=0x7f31d8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b49bdadff612b87 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:39.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:39.961+0000 7f31e37fe700 1 -- 172.21.15.16:0/3032696449 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d8011830 con 0x7f31e406a770 2023-09-30T22:33:39.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:39.962+0000 7f31e37fe700 1 -- 172.21.15.16:0/3032696449 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31d8011990 con 0x7f31e406a770 2023-09-30T22:33:39.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:39.962+0000 7f31e37fe700 1 -- 172.21.15.16:0/3032696449 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d8011d20 con 0x7f31e406a770 2023-09-30T22:33:39.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:39.962+0000 7f31ead03700 1 -- 172.21.15.16:0/3032696449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31e406a770 msgr2=0x7f31e409d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:39.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:39.962+0000 7f31ead03700 1 --2- 172.21.15.16:0/3032696449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31e406a770 0x7f31e409d6f0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f31d8009840 tx=0x7f31d8009b50 comp rx=0 tx=0).stop 2023-09-30T22:33:39.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:39.962+0000 7f31ead03700 1 -- 172.21.15.16:0/3032696449 shutdown_connections 2023-09-30T22:33:39.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:39.962+0000 7f31ead03700 1 --2- 172.21.15.16:0/3032696449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31e406a770 0x7f31e409d6f0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:39.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:39.962+0000 7f31ead03700 1 --2- 172.21.15.16:0/3032696449 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f31e409dc30 0x7f31e409e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:39.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:39.962+0000 7f31ead03700 1 -- 172.21.15.16:0/3032696449 >> 172.21.15.16:0/3032696449 conn(0x7f31e4066f50 msgr2=0x7f31e409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:39.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:39.962+0000 7f31ead03700 1 -- 172.21.15.16:0/3032696449 shutdown_connections 2023-09-30T22:33:39.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:39.963+0000 7f31ead03700 1 -- 172.21.15.16:0/3032696449 wait complete. 2023-09-30T22:33:39.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:39.964+0000 7f31ead03700 1 Processor -- start 2023-09-30T22:33:39.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:39.964+0000 7f31ead03700 1 -- start start 2023-09-30T22:33:39.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:39.965+0000 7f31ead03700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f31e406a770 0x7f31e413e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:39.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:39.965+0000 7f31ead03700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31e409dc30 0x7f31e413a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:39.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:39.965+0000 7f31ead03700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f31e413c1e0 0x7f31e413a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:39.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:39.965+0000 7f31ead03700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31e413bba0 con 0x7f31e409dc30 2023-09-30T22:33:39.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:39.965+0000 7f31ead03700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31e409f210 con 0x7f31e413c1e0 2023-09-30T22:33:39.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:39.965+0000 7f31ead03700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f31e411d940 con 0x7f31e406a770 2023-09-30T22:33:39.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:39.965+0000 7f31e8a9f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f31e406a770 0x7f31e413e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:39.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:39.965+0000 7f31e8a9f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f31e406a770 0x7f31e413e180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:47376/0 (socket says 172.21.15.16:47376) 2023-09-30T22:33:39.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:39.965+0000 7f31e8a9f700 1 -- 172.21.15.16:0/3100873770 learned_addr learned my addr 172.21.15.16:0/3100873770 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:39.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:39.965+0000 7f31e3fff700 1 --2- 172.21.15.16:0/3100873770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31e409dc30 0x7f31e413a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:39.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:39.965+0000 7f31e92a0700 1 --2- 172.21.15.16:0/3100873770 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f31e413c1e0 0x7f31e413a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:39.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:39.966+0000 7f31e8a9f700 1 -- 172.21.15.16:0/3100873770 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f31e413c1e0 msgr2=0x7f31e413a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:39.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:39.966+0000 7f31e8a9f700 1 --2- 172.21.15.16:0/3100873770 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f31e413c1e0 0x7f31e413a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:39.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:39.966+0000 7f31e8a9f700 1 -- 172.21.15.16:0/3100873770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31e409dc30 msgr2=0x7f31e413a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:39.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:39.966+0000 7f31e8a9f700 1 --2- 172.21.15.16:0/3100873770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31e409dc30 0x7f31e413a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:39.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:39.966+0000 7f31e8a9f700 1 -- 172.21.15.16:0/3100873770 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31d80094a0 con 0x7f31e406a770 2023-09-30T22:33:39.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:39.966+0000 7f31e3fff700 1 --2- 172.21.15.16:0/3100873770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31e409dc30 0x7f31e413a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:39.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:39.966+0000 7f31e8a9f700 1 --2- 172.21.15.16:0/3100873770 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f31e406a770 0x7f31e413e180 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f31d8009fd0 tx=0x7f31d8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:39.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:39.966+0000 7f31e17fa700 1 -- 172.21.15.16:0/3100873770 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d80036a0 con 0x7f31e406a770 2023-09-30T22:33:39.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:39.966+0000 7f31e17fa700 1 -- 172.21.15.16:0/3100873770 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31d801f070 con 0x7f31e406a770 2023-09-30T22:33:39.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:39.966+0000 7f31ead03700 1 -- 172.21.15.16:0/3100873770 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31e413fb80 con 0x7f31e406a770 2023-09-30T22:33:39.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:39.966+0000 7f31e17fa700 1 -- 172.21.15.16:0/3100873770 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d8003a20 con 0x7f31e406a770 2023-09-30T22:33:39.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:39.966+0000 7f31ead03700 1 -- 172.21.15.16:0/3100873770 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31e4140090 con 0x7f31e406a770 2023-09-30T22:33:39.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:39.967+0000 7f31e17fa700 1 -- 172.21.15.16:0/3100873770 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f31d8003b80 con 0x7f31e406a770 2023-09-30T22:33:39.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:39.967+0000 7f31e17fa700 1 -- 172.21.15.16:0/3100873770 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f31d8021070 con 0x7f31e406a770 2023-09-30T22:33:39.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:39.968+0000 7f31ead03700 1 -- 172.21.15.16:0/3100873770 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31c40052e0 con 0x7f31e406a770 2023-09-30T22:33:39.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:39.977+0000 7f31e17fa700 1 -- 172.21.15.16:0/3100873770 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f31d8015020 con 0x7f31e406a770 2023-09-30T22:33:40.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.004+0000 7f6e79a61700 1 Processor -- start 2023-09-30T22:33:40.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.004+0000 7f6e79a61700 1 -- start start 2023-09-30T22:33:40.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.004+0000 7f6e79a61700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6e741338a0 0x7f6e74133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:40.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.004+0000 7f6e79a61700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e7409de50 0x7f6e7409e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:40.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.004+0000 7f6e79a61700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6e7409e760 0x7f6e7413a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:40.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.004+0000 7f6e79a61700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e7406fef0 con 0x7f6e741338a0 2023-09-30T22:33:40.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.004+0000 7f6e79a61700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7406ef30 con 0x7f6e7409e760 2023-09-30T22:33:40.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.004+0000 7f6e79a61700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6e741334b0 con 0x7f6e7409de50 2023-09-30T22:33:40.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.005+0000 7f6e727fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e7409de50 0x7f6e7409e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:40.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.005+0000 7f6e727fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e7409de50 0x7f6e7409e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:47390/0 (socket says 172.21.15.16:47390) 2023-09-30T22:33:40.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.005+0000 7f6e727fc700 1 -- 172.21.15.16:0/2233094541 learned_addr learned my addr 172.21.15.16:0/2233094541 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:40.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.005+0000 7f6e737fe700 1 --2- 172.21.15.16:0/2233094541 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6e7409e760 0x7f6e7413a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:40.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.005+0000 7f6e727fc700 1 -- 172.21.15.16:0/2233094541 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6e7409e760 msgr2=0x7f6e7413a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:40.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.005+0000 7f6e727fc700 1 --2- 172.21.15.16:0/2233094541 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6e7409e760 0x7f6e7413a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:40.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.005+0000 7f6e727fc700 1 -- 172.21.15.16:0/2233094541 >> v2:172.21.15.16:3300/0 conn(0x7f6e741338a0 msgr2=0x7f6e74133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:40.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.005+0000 7f6e727fc700 1 --2- 172.21.15.16:0/2233094541 >> v2:172.21.15.16:3300/0 conn(0x7f6e741338a0 0x7f6e74133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:40.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.005+0000 7f6e727fc700 1 -- 172.21.15.16:0/2233094541 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e7406fc30 con 0x7f6e7409de50 2023-09-30T22:33:40.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.005+0000 7f6e727fc700 1 --2- 172.21.15.16:0/2233094541 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e7409de50 0x7f6e7409e220 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f6e680098f0 tx=0x7f6e68009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aed095ea450515cb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:40.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.006+0000 7f6e71ffb700 1 -- 172.21.15.16:0/2233094541 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6800b8a0 con 0x7f6e7409de50 2023-09-30T22:33:40.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.006+0000 7f6e71ffb700 1 -- 172.21.15.16:0/2233094541 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e6800ba00 con 0x7f6e7409de50 2023-09-30T22:33:40.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.006+0000 7f6e71ffb700 1 -- 172.21.15.16:0/2233094541 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6800bd90 con 0x7f6e7409de50 2023-09-30T22:33:40.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.006+0000 7f6e79a61700 1 -- 172.21.15.16:0/2233094541 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e7409de50 msgr2=0x7f6e7409e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:40.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.006+0000 7f6e79a61700 1 --2- 172.21.15.16:0/2233094541 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e7409de50 0x7f6e7409e220 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f6e680098f0 tx=0x7f6e68009c00 comp rx=0 tx=0).stop 2023-09-30T22:33:40.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.006+0000 7f6e79a61700 1 -- 172.21.15.16:0/2233094541 shutdown_connections 2023-09-30T22:33:40.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.006+0000 7f6e79a61700 1 --2- 172.21.15.16:0/2233094541 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6e7409e760 0x7f6e7413a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:40.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.006+0000 7f6e79a61700 1 --2- 172.21.15.16:0/2233094541 >> v2:172.21.15.16:3300/0 conn(0x7f6e741338a0 0x7f6e74133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:40.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.006+0000 7f6e79a61700 1 --2- 172.21.15.16:0/2233094541 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e7409de50 0x7f6e7409e220 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:40.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.006+0000 7f6e79a61700 1 -- 172.21.15.16:0/2233094541 >> 172.21.15.16:0/2233094541 conn(0x7f6e740a2aa0 msgr2=0x7f6e740a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:40.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.006+0000 7f6e79a61700 1 -- 172.21.15.16:0/2233094541 shutdown_connections 2023-09-30T22:33:40.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.007+0000 7f6e79a61700 1 -- 172.21.15.16:0/2233094541 wait complete. 2023-09-30T22:33:40.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.008+0000 7f6e79a61700 1 Processor -- start 2023-09-30T22:33:40.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.008+0000 7f6e79a61700 1 -- start start 2023-09-30T22:33:40.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.008+0000 7f6e79a61700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6e7409de50 0x7f6e7411eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:40.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.008+0000 7f6e79a61700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e7409e760 0x7f6e7411f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:40.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.008+0000 7f6e79a61700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e741338a0 0x7f6e7411f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:40.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.008+0000 7f6e79a61700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7412f6e0 con 0x7f6e7409e760 2023-09-30T22:33:40.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.008+0000 7f6e79a61700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e74121ce0 con 0x7f6e7409de50 2023-09-30T22:33:40.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.008+0000 7f6e79a61700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6e741206f0 con 0x7f6e741338a0 2023-09-30T22:33:40.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.008+0000 7f6e737fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e741338a0 0x7f6e7411f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:40.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.008+0000 7f6e737fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e741338a0 0x7f6e7411f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:47392/0 (socket says 172.21.15.16:47392) 2023-09-30T22:33:40.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.008+0000 7f6e737fe700 1 -- 172.21.15.16:0/3506978654 learned_addr learned my addr 172.21.15.16:0/3506978654 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:40.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.009+0000 7f6e72ffd700 1 --2- 172.21.15.16:0/3506978654 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6e7409de50 0x7f6e7411eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:40.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.009+0000 7f6e727fc700 1 --2- 172.21.15.16:0/3506978654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e7409e760 0x7f6e7411f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:40.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.009+0000 7f6e737fe700 1 -- 172.21.15.16:0/3506978654 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6e7409de50 msgr2=0x7f6e7411eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:40.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.009+0000 7f6e737fe700 1 --2- 172.21.15.16:0/3506978654 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6e7409de50 0x7f6e7411eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:40.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.009+0000 7f6e737fe700 1 -- 172.21.15.16:0/3506978654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e7409e760 msgr2=0x7f6e7411f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:40.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.009+0000 7f6e737fe700 1 --2- 172.21.15.16:0/3506978654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e7409e760 0x7f6e7411f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:40.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.009+0000 7f6e737fe700 1 -- 172.21.15.16:0/3506978654 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e68009550 con 0x7f6e741338a0 2023-09-30T22:33:40.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.009+0000 7f6e737fe700 1 --2- 172.21.15.16:0/3506978654 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e741338a0 0x7f6e7411f560 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c007e70 tx=0x7f6e6c00e3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:40.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.009+0000 7f6e5bfff700 1 -- 172.21.15.16:0/3506978654 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6c017d70 con 0x7f6e741338a0 2023-09-30T22:33:40.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.009+0000 7f6e79a61700 1 -- 172.21.15.16:0/3506978654 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e74122600 con 0x7f6e741338a0 2023-09-30T22:33:40.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.010+0000 7f6e79a61700 1 -- 172.21.15.16:0/3506978654 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e7406f690 con 0x7f6e741338a0 2023-09-30T22:33:40.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.010+0000 7f6e5bfff700 1 -- 172.21.15.16:0/3506978654 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e6c00e920 con 0x7f6e741338a0 2023-09-30T22:33:40.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.010+0000 7f6e5bfff700 1 -- 172.21.15.16:0/3506978654 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6c00ec90 con 0x7f6e741338a0 2023-09-30T22:33:40.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.011+0000 7f6e79a61700 1 -- 172.21.15.16:0/3506978654 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e540052e0 con 0x7f6e741338a0 2023-09-30T22:33:40.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.011+0000 7f6e5bfff700 1 -- 172.21.15.16:0/3506978654 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6e6c01f3f0 con 0x7f6e741338a0 2023-09-30T22:33:40.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.012+0000 7f6e5bfff700 1 -- 172.21.15.16:0/3506978654 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f6e6c048720 con 0x7f6e741338a0 2023-09-30T22:33:40.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.020+0000 7f6e5bfff700 1 -- 172.21.15.16:0/3506978654 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6e6c00edf0 con 0x7f6e741338a0 2023-09-30T22:33:40.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.046+0000 7f013023e700 1 Processor -- start 2023-09-30T22:33:40.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.046+0000 7f013023e700 1 -- start start 2023-09-30T22:33:40.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.047+0000 7f013023e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f012812cf60 0x7f012806f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:40.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.047+0000 7f013023e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0128130430 0x7f0128137c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:40.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.047+0000 7f013023e700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f012812cd90 con 0x7f012812f8b0 2023-09-30T22:33:40.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.047+0000 7f013023e700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f012812d330 con 0x7f0128130430 2023-09-30T22:33:40.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.047+0000 7f013023e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0128071780 con 0x7f012812cf60 2023-09-30T22:33:40.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.047+0000 7f012e7db700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0128130430 0x7f0128137c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:40.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.047+0000 7f012dfda700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f012812cf60 0x7f012806f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:40.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.047+0000 7f012d7d9700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f012812f8b0 0x7f012812fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38082/0 (socket says 172.21.15.16:38082) 2023-09-30T22:33:40.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.047+0000 7f012dfda700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f012812cf60 0x7f012806f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:47408/0 (socket says 172.21.15.16:47408) 2023-09-30T22:33:40.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.047+0000 7f012e7db700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0128130430 0x7f0128137c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56004/0 (socket says 172.21.15.16:56004) 2023-09-30T22:33:40.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.047+0000 7f012d7d9700 1 -- 172.21.15.16:0/1034549553 learned_addr learned my addr 172.21.15.16:0/1034549553 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:40.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.048+0000 7f012dfda700 1 -- 172.21.15.16:0/1034549553 >> v2:172.21.15.16:3300/0 conn(0x7f0128130430 msgr2=0x7f0128137c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:40.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.048+0000 7f012dfda700 1 --2- 172.21.15.16:0/1034549553 >> v2:172.21.15.16:3300/0 conn(0x7f0128130430 0x7f0128137c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:40.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.048+0000 7f012dfda700 1 -- 172.21.15.16:0/1034549553 >> v1:172.21.15.16:6789/0 conn(0x7f012812f8b0 legacy=0x7f012812fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:40.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.048+0000 7f012dfda700 1 -- 172.21.15.16:0/1034549553 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0128071a40 con 0x7f012812cf60 2023-09-30T22:33:40.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.048+0000 7f012e7db700 1 --2- 172.21.15.16:0/1034549553 >> v2:172.21.15.16:3300/0 conn(0x7f0128130430 0x7f0128137c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:40.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.048+0000 7f012dfda700 1 --2- 172.21.15.16:0/1034549553 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f012812cf60 0x7f012806f5d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f0124009840 tx=0x7f0124009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bdede0b9a0ebee6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:40.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.048+0000 7f012cfd8700 1 -- 172.21.15.16:0/1034549553 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0124011830 con 0x7f012812cf60 2023-09-30T22:33:40.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.048+0000 7f012cfd8700 1 -- 172.21.15.16:0/1034549553 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0124011990 con 0x7f012812cf60 2023-09-30T22:33:40.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.048+0000 7f012cfd8700 1 -- 172.21.15.16:0/1034549553 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0124011d20 con 0x7f012812cf60 2023-09-30T22:33:40.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.049+0000 7f013023e700 1 -- 172.21.15.16:0/1034549553 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f012812cf60 msgr2=0x7f012806f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:40.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.049+0000 7f013023e700 1 --2- 172.21.15.16:0/1034549553 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f012812cf60 0x7f012806f5d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f0124009840 tx=0x7f0124009b50 comp rx=0 tx=0).stop 2023-09-30T22:33:40.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.049+0000 7f013023e700 1 -- 172.21.15.16:0/1034549553 shutdown_connections 2023-09-30T22:33:40.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.049+0000 7f013023e700 1 --2- 172.21.15.16:0/1034549553 >> v2:172.21.15.16:3300/0 conn(0x7f0128130430 0x7f0128137c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:40.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.049+0000 7f013023e700 1 --2- 172.21.15.16:0/1034549553 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f012812cf60 0x7f012806f5d0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:40.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.049+0000 7f013023e700 1 -- 172.21.15.16:0/1034549553 >> 172.21.15.16:0/1034549553 conn(0x7f01280a2aa0 msgr2=0x7f01280a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:40.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.050+0000 7f013023e700 1 -- 172.21.15.16:0/1034549553 shutdown_connections 2023-09-30T22:33:40.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.050+0000 7f013023e700 1 -- 172.21.15.16:0/1034549553 wait complete. 2023-09-30T22:33:40.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.051+0000 7f013023e700 1 Processor -- start 2023-09-30T22:33:40.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.051+0000 7f013023e700 1 -- start start 2023-09-30T22:33:40.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.052+0000 7f013023e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f012812cf60 0x7f012809b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:40.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.052+0000 7f013023e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f012812f8b0 0x7f012809b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:40.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.052+0000 7f013023e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0128130430 0x7f012809bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:40.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.052+0000 7f013023e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f012809c6d0 con 0x7f0128130430 2023-09-30T22:33:40.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.052+0000 7f013023e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f012811d960 con 0x7f012812cf60 2023-09-30T22:33:40.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.052+0000 7f013023e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f012809dfe0 con 0x7f012812f8b0 2023-09-30T22:33:40.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.052+0000 7f012dfda700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f012812cf60 0x7f012809b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:40.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.052+0000 7f012dfda700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f012812cf60 0x7f012809b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:51782/0 (socket says 172.21.15.16:51782) 2023-09-30T22:33:40.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.052+0000 7f012dfda700 1 -- 172.21.15.16:0/3265989143 learned_addr learned my addr 172.21.15.16:0/3265989143 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:40.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.052+0000 7f012e7db700 1 --2- 172.21.15.16:0/3265989143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0128130430 0x7f012809bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:40.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.052+0000 7f012d7d9700 1 --2- 172.21.15.16:0/3265989143 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f012812f8b0 0x7f012809b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:40.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.053+0000 7f012dfda700 1 -- 172.21.15.16:0/3265989143 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f012812f8b0 msgr2=0x7f012809b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:40.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.053+0000 7f012dfda700 1 --2- 172.21.15.16:0/3265989143 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f012812f8b0 0x7f012809b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:40.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.053+0000 7f012dfda700 1 -- 172.21.15.16:0/3265989143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0128130430 msgr2=0x7f012809bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:40.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.053+0000 7f012dfda700 1 --2- 172.21.15.16:0/3265989143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0128130430 0x7f012809bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:40.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.053+0000 7f012dfda700 1 -- 172.21.15.16:0/3265989143 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f012000bc80 con 0x7f012812cf60 2023-09-30T22:33:40.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.053+0000 7f012dfda700 1 --2- 172.21.15.16:0/3265989143 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f012812cf60 0x7f012809b2e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f0124009fd0 tx=0x7f0124004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:40.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.053+0000 7f011e7fc700 1 -- 172.21.15.16:0/3265989143 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0124004400 con 0x7f012812cf60 2023-09-30T22:33:40.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.053+0000 7f011e7fc700 1 -- 172.21.15.16:0/3265989143 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f012401f070 con 0x7f012812cf60 2023-09-30T22:33:40.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.053+0000 7f011e7fc700 1 -- 172.21.15.16:0/3265989143 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01240038e0 con 0x7f012812cf60 2023-09-30T22:33:40.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.053+0000 7f013023e700 1 -- 172.21.15.16:0/3265989143 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01240094a0 con 0x7f012812cf60 2023-09-30T22:33:40.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.053+0000 7f013023e700 1 -- 172.21.15.16:0/3265989143 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f012809e430 con 0x7f012812cf60 2023-09-30T22:33:40.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.054+0000 7f013023e700 1 -- 172.21.15.16:0/3265989143 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0128129fa0 con 0x7f012812cf60 2023-09-30T22:33:40.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.054+0000 7f011e7fc700 1 -- 172.21.15.16:0/3265989143 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0124021030 con 0x7f012812cf60 2023-09-30T22:33:40.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.055+0000 7f011e7fc700 1 -- 172.21.15.16:0/3265989143 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f01240409a0 con 0x7f012812cf60 2023-09-30T22:33:40.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.064+0000 7f011e7fc700 1 -- 172.21.15.16:0/3265989143 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f01240239c0 con 0x7f012812cf60 2023-09-30T22:33:40.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.097+0000 7f89326a2700 1 Processor -- start 2023-09-30T22:33:40.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.097+0000 7f89326a2700 1 -- start start 2023-09-30T22:33:40.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.098+0000 7f89326a2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f892c0a0a30 0x7f892c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:40.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.098+0000 7f89326a2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f892c09d630 0x7f892c0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:40.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.098+0000 7f89326a2700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f892c133930 0x7f892c133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:40.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.098+0000 7f89326a2700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f892c0a2a70 con 0x7f892c133930 2023-09-30T22:33:40.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.098+0000 7f89326a2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f892c0a01d0 con 0x7f892c09d630 2023-09-30T22:33:40.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.098+0000 7f89326a2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f892c09e2b0 con 0x7f892c0a0a30 2023-09-30T22:33:40.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.098+0000 7f892b7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f892c09d630 0x7f892c0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:40.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.098+0000 7f892b7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f892c09d630 0x7f892c0a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:51792/0 (socket says 172.21.15.16:51792) 2023-09-30T22:33:40.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.098+0000 7f892b7fe700 1 -- 172.21.15.16:0/50385095 learned_addr learned my addr 172.21.15.16:0/50385095 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:40.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.098+0000 7f892bfff700 1 --2- 172.21.15.16:0/50385095 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f892c0a0a30 0x7f892c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:40.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.098+0000 7f8930c3f700 1 --2- 172.21.15.16:0/50385095 >> v2:172.21.15.16:3300/0 conn(0x7f892c133930 0x7f892c133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:40.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.099+0000 7f892bfff700 1 -- 172.21.15.16:0/50385095 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f892c09d630 msgr2=0x7f892c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:40.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.099+0000 7f892bfff700 1 --2- 172.21.15.16:0/50385095 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f892c09d630 0x7f892c0a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:40.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.099+0000 7f892bfff700 1 -- 172.21.15.16:0/50385095 >> v2:172.21.15.16:3300/0 conn(0x7f892c133930 msgr2=0x7f892c133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:40.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.099+0000 7f892bfff700 1 --2- 172.21.15.16:0/50385095 >> v2:172.21.15.16:3300/0 conn(0x7f892c133930 0x7f892c133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:40.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.099+0000 7f892bfff700 1 -- 172.21.15.16:0/50385095 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f892c09dc50 con 0x7f892c0a0a30 2023-09-30T22:33:40.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.099+0000 7f892bfff700 1 --2- 172.21.15.16:0/50385095 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f892c0a0a30 0x7f892c0a0e00 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f891c009840 tx=0x7f891c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a70023cb585df22b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:40.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.099+0000 7f892affd700 1 -- 172.21.15.16:0/50385095 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891c011830 con 0x7f892c0a0a30 2023-09-30T22:33:40.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.099+0000 7f892affd700 1 -- 172.21.15.16:0/50385095 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f891c011990 con 0x7f892c0a0a30 2023-09-30T22:33:40.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.099+0000 7f892affd700 1 -- 172.21.15.16:0/50385095 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891c011d20 con 0x7f892c0a0a30 2023-09-30T22:33:40.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.100+0000 7f89326a2700 1 -- 172.21.15.16:0/50385095 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f892c0a0a30 msgr2=0x7f892c0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:40.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.100+0000 7f89326a2700 1 --2- 172.21.15.16:0/50385095 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f892c0a0a30 0x7f892c0a0e00 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f891c009840 tx=0x7f891c009b50 comp rx=0 tx=0).stop 2023-09-30T22:33:40.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.100+0000 7f89326a2700 1 -- 172.21.15.16:0/50385095 shutdown_connections 2023-09-30T22:33:40.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.100+0000 7f89326a2700 1 --2- 172.21.15.16:0/50385095 >> v2:172.21.15.16:3300/0 conn(0x7f892c133930 0x7f892c133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:40.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.100+0000 7f89326a2700 1 --2- 172.21.15.16:0/50385095 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f892c0a0a30 0x7f892c0a0e00 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:40.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.100+0000 7f89326a2700 1 --2- 172.21.15.16:0/50385095 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f892c09d630 0x7f892c0a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:40.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.100+0000 7f89326a2700 1 -- 172.21.15.16:0/50385095 >> 172.21.15.16:0/50385095 conn(0x7f892c0a3050 msgr2=0x7f892c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:40.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.101+0000 7f89326a2700 1 -- 172.21.15.16:0/50385095 shutdown_connections 2023-09-30T22:33:40.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.101+0000 7f89326a2700 1 -- 172.21.15.16:0/50385095 wait complete. 2023-09-30T22:33:40.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.102+0000 7f89326a2700 1 Processor -- start 2023-09-30T22:33:40.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.102+0000 7f89326a2700 1 -- start start 2023-09-30T22:33:40.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.102+0000 7f89326a2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f892c09d630 0x7f892c11f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:40.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.102+0000 7f89326a2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f892c0a0a30 0x7f892c11fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:40.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.102+0000 7f89326a2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f892c133930 0x7f892c1c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:40.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.102+0000 7f89326a2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892c121fb0 con 0x7f892c0a0a30 2023-09-30T22:33:40.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.102+0000 7f89326a2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f892c11f020 con 0x7f892c09d630 2023-09-30T22:33:40.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.102+0000 7f89326a2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f892c120640 con 0x7f892c133930 2023-09-30T22:33:40.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.103+0000 7f892bfff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f892c09d630 0x7f892c11f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:40.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.103+0000 7f892bfff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f892c09d630 0x7f892c11f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:51808/0 (socket says 172.21.15.16:51808) 2023-09-30T22:33:40.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.103+0000 7f892bfff700 1 -- 172.21.15.16:0/827316229 learned_addr learned my addr 172.21.15.16:0/827316229 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:40.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.103+0000 7f8930c3f700 1 --2- 172.21.15.16:0/827316229 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f892c133930 0x7f892c1c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:40.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.103+0000 7f892bfff700 1 -- 172.21.15.16:0/827316229 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f892c133930 msgr2=0x7f892c1c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:40.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.103+0000 7f892bfff700 1 --2- 172.21.15.16:0/827316229 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f892c133930 0x7f892c1c8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:40.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.103+0000 7f892bfff700 1 -- 172.21.15.16:0/827316229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f892c0a0a30 msgr2=0x7f892c11fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:40.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.103+0000 7f892bfff700 1 --2- 172.21.15.16:0/827316229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f892c0a0a30 0x7f892c11fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:40.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.103+0000 7f892bfff700 1 -- 172.21.15.16:0/827316229 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f891c0094a0 con 0x7f892c09d630 2023-09-30T22:33:40.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.103+0000 7f892bfff700 1 --2- 172.21.15.16:0/827316229 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f892c09d630 0x7f892c11f5c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f891c0090b0 tx=0x7f891c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:40.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.103+0000 7f8928ff9700 1 -- 172.21.15.16:0/827316229 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891c004300 con 0x7f892c09d630 2023-09-30T22:33:40.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.103+0000 7f8928ff9700 1 -- 172.21.15.16:0/827316229 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f891c01f070 con 0x7f892c09d630 2023-09-30T22:33:40.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.103+0000 7f8928ff9700 1 -- 172.21.15.16:0/827316229 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891c0036a0 con 0x7f892c09d630 2023-09-30T22:33:40.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.104+0000 7f89326a2700 1 -- 172.21.15.16:0/827316229 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f892c0a40c0 con 0x7f892c09d630 2023-09-30T22:33:40.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.104+0000 7f89326a2700 1 -- 172.21.15.16:0/827316229 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f892c1334e0 con 0x7f892c09d630 2023-09-30T22:33:40.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.105+0000 7f89326a2700 1 -- 172.21.15.16:0/827316229 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f892c120040 con 0x7f892c09d630 2023-09-30T22:33:40.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.105+0000 7f8928ff9700 1 -- 172.21.15.16:0/827316229 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f891c021030 con 0x7f892c09d630 2023-09-30T22:33:40.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.105+0000 7f8928ff9700 1 -- 172.21.15.16:0/827316229 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f891c040740 con 0x7f892c09d630 2023-09-30T22:33:40.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.110+0000 7f8928ff9700 1 -- 172.21.15.16:0/827316229 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f891c0239c0 con 0x7f892c09d630 2023-09-30T22:33:40.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.193+0000 7f31ead03700 1 -- 172.21.15.16:0/3100873770 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f31c4005590 con 0x7f31e406a770 2023-09-30T22:33:40.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.194+0000 7f31e17fa700 1 -- 172.21.15.16:0/3100873770 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f31d8003e30 con 0x7f31e406a770 2023-09-30T22:33:40.194 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:33:40.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.196+0000 7f31ead03700 1 -- 172.21.15.16:0/3100873770 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f31e406a770 msgr2=0x7f31e413e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:40.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.196+0000 7f31ead03700 1 --2- 172.21.15.16:0/3100873770 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f31e406a770 0x7f31e413e180 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f31d8009fd0 tx=0x7f31d8004750 comp rx=0 tx=0).stop 2023-09-30T22:33:40.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.196+0000 7f31ead03700 1 -- 172.21.15.16:0/3100873770 shutdown_connections 2023-09-30T22:33:40.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.196+0000 7f31ead03700 1 --2- 172.21.15.16:0/3100873770 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f31e413c1e0 0x7f31e413a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:40.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.196+0000 7f31ead03700 1 --2- 172.21.15.16:0/3100873770 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f31e406a770 0x7f31e413e180 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:40.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.196+0000 7f31ead03700 1 --2- 172.21.15.16:0/3100873770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f31e409dc30 0x7f31e413a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:40.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.196+0000 7f31ead03700 1 -- 172.21.15.16:0/3100873770 >> 172.21.15.16:0/3100873770 conn(0x7f31e4066f50 msgr2=0x7f31e4072260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:40.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.197+0000 7f31ead03700 1 -- 172.21.15.16:0/3100873770 shutdown_connections 2023-09-30T22:33:40.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.197+0000 7f31ead03700 1 -- 172.21.15.16:0/3100873770 wait complete. 2023-09-30T22:33:40.211 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:33:40.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.251+0000 7f6e79a61700 1 -- 172.21.15.16:0/3506978654 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f6e54005860 con 0x7f6e741338a0 2023-09-30T22:33:40.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.252+0000 7f6e5bfff700 1 -- 172.21.15.16:0/3506978654 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6e6c027360 con 0x7f6e741338a0 2023-09-30T22:33:40.253 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:33:40.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.254+0000 7f6e79a61700 1 -- 172.21.15.16:0/3506978654 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e741338a0 msgr2=0x7f6e7411f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:40.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.254+0000 7f6e79a61700 1 --2- 172.21.15.16:0/3506978654 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e741338a0 0x7f6e7411f560 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c007e70 tx=0x7f6e6c00e3c0 comp rx=0 tx=0).stop 2023-09-30T22:33:40.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.254+0000 7f6e79a61700 1 -- 172.21.15.16:0/3506978654 shutdown_connections 2023-09-30T22:33:40.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.254+0000 7f6e79a61700 1 --2- 172.21.15.16:0/3506978654 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6e7409de50 0x7f6e7411eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:40.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.254+0000 7f6e79a61700 1 --2- 172.21.15.16:0/3506978654 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6e741338a0 0x7f6e7411f560 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:40.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.254+0000 7f6e79a61700 1 --2- 172.21.15.16:0/3506978654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6e7409e760 0x7f6e7411f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:40.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.254+0000 7f6e79a61700 1 -- 172.21.15.16:0/3506978654 >> 172.21.15.16:0/3506978654 conn(0x7f6e740a2aa0 msgr2=0x7f6e7409ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:40.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.254+0000 7f6e79a61700 1 -- 172.21.15.16:0/3506978654 shutdown_connections 2023-09-30T22:33:40.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.254+0000 7f6e79a61700 1 -- 172.21.15.16:0/3506978654 wait complete. 2023-09-30T22:33:40.272 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:33:40.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.284+0000 7f013023e700 1 -- 172.21.15.16:0/3265989143 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f012809e620 con 0x7f012812cf60 2023-09-30T22:33:40.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.285+0000 7f011e7fc700 1 -- 172.21.15.16:0/3265989143 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f012401b430 con 0x7f012812cf60 2023-09-30T22:33:40.285 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:33:40.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.287+0000 7f013023e700 1 -- 172.21.15.16:0/3265989143 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f012812cf60 msgr2=0x7f012809b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:40.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.287+0000 7f013023e700 1 --2- 172.21.15.16:0/3265989143 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f012812cf60 0x7f012809b2e0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f0124009fd0 tx=0x7f0124004750 comp rx=0 tx=0).stop 2023-09-30T22:33:40.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.287+0000 7f013023e700 1 -- 172.21.15.16:0/3265989143 shutdown_connections 2023-09-30T22:33:40.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.287+0000 7f013023e700 1 --2- 172.21.15.16:0/3265989143 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f012812cf60 0x7f012809b2e0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:40.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.287+0000 7f013023e700 1 --2- 172.21.15.16:0/3265989143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0128130430 0x7f012809bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:40.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.287+0000 7f013023e700 1 --2- 172.21.15.16:0/3265989143 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f012812f8b0 0x7f012809b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:40.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.287+0000 7f013023e700 1 -- 172.21.15.16:0/3265989143 >> 172.21.15.16:0/3265989143 conn(0x7f01280a2aa0 msgr2=0x7f012806e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:40.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.287+0000 7f013023e700 1 -- 172.21.15.16:0/3265989143 shutdown_connections 2023-09-30T22:33:40.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.287+0000 7f013023e700 1 -- 172.21.15.16:0/3265989143 wait complete. 2023-09-30T22:33:40.301 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:33:40.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.336+0000 7f89326a2700 1 -- 172.21.15.16:0/827316229 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f892c0747f0 con 0x7f892c09d630 2023-09-30T22:33:40.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.337+0000 7f8928ff9700 1 -- 172.21.15.16:0/827316229 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f891c015070 con 0x7f892c09d630 2023-09-30T22:33:40.337 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:33:40.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.339+0000 7f89326a2700 1 -- 172.21.15.16:0/827316229 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f892c09d630 msgr2=0x7f892c11f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:40.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.339+0000 7f89326a2700 1 --2- 172.21.15.16:0/827316229 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f892c09d630 0x7f892c11f5c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f891c0090b0 tx=0x7f891c004750 comp rx=0 tx=0).stop 2023-09-30T22:33:40.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.339+0000 7f89326a2700 1 -- 172.21.15.16:0/827316229 shutdown_connections 2023-09-30T22:33:40.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.339+0000 7f89326a2700 1 --2- 172.21.15.16:0/827316229 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f892c09d630 0x7f892c11f5c0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:40.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.339+0000 7f89326a2700 1 --2- 172.21.15.16:0/827316229 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f892c133930 0x7f892c1c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:40.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.339+0000 7f89326a2700 1 --2- 172.21.15.16:0/827316229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f892c0a0a30 0x7f892c11fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:40.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.339+0000 7f89326a2700 1 -- 172.21.15.16:0/827316229 >> 172.21.15.16:0/827316229 conn(0x7f892c0a3050 msgr2=0x7f892c09ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:40.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.339+0000 7f89326a2700 1 -- 172.21.15.16:0/827316229 shutdown_connections 2023-09-30T22:33:40.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:40.339+0000 7f89326a2700 1 -- 172.21.15.16:0/827316229 wait complete. 2023-09-30T22:33:40.353 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:33:41.212 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:33:41.273 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:33:41.302 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:33:41.354 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:33:41.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.363+0000 7f6d193db700 1 Processor -- start 2023-09-30T22:33:41.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.363+0000 7f6d193db700 1 -- start start 2023-09-30T22:33:41.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.364+0000 7f6d193db700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6d1409e1d0 0x7f6d14135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:41.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.364+0000 7f6d193db700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d14135e50 0x7f6d14138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:41.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.364+0000 7f6d193db700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d1409f130 con 0x7f6d1409d6f0 2023-09-30T22:33:41.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.364+0000 7f6d193db700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d1409f520 con 0x7f6d1409e1d0 2023-09-30T22:33:41.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.364+0000 7f6d193db700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1409e930 con 0x7f6d14135e50 2023-09-30T22:33:41.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.364+0000 7f6d137fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d14135e50 0x7f6d14138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:41.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.364+0000 7f6d127fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6d1409e1d0 0x7f6d14135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:41.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.364+0000 7f6d137fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d14135e50 0x7f6d14138230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:47434/0 (socket says 172.21.15.16:47434) 2023-09-30T22:33:41.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.364+0000 7f6d127fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6d1409e1d0 0x7f6d14135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56060/0 (socket says 172.21.15.16:56060) 2023-09-30T22:33:41.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.364+0000 7f6d137fe700 1 -- 172.21.15.16:0/155362270 learned_addr learned my addr 172.21.15.16:0/155362270 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:41.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.365+0000 7f6d137fe700 1 -- 172.21.15.16:0/155362270 >> v2:172.21.15.16:3300/0 conn(0x7f6d1409e1d0 msgr2=0x7f6d14135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:41.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.365+0000 7f6d137fe700 1 --2- 172.21.15.16:0/155362270 >> v2:172.21.15.16:3300/0 conn(0x7f6d1409e1d0 0x7f6d14135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:41.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.365+0000 7f6d137fe700 1 -- 172.21.15.16:0/155362270 >> v1:172.21.15.16:6789/0 conn(0x7f6d1409d6f0 legacy=0x7f6d1409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:41.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.365+0000 7f6d137fe700 1 -- 172.21.15.16:0/155362270 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d1409b2d0 con 0x7f6d14135e50 2023-09-30T22:33:41.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.365+0000 7f6d127fc700 1 --2- 172.21.15.16:0/155362270 >> v2:172.21.15.16:3300/0 conn(0x7f6d1409e1d0 0x7f6d14135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:41.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.365+0000 7f6d137fe700 1 --2- 172.21.15.16:0/155362270 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d14135e50 0x7f6d14138230 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c009e50 tx=0x7f6d0c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b884fbe26358816f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:41.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.366+0000 7f6d11ffb700 1 -- 172.21.15.16:0/155362270 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c00e070 con 0x7f6d14135e50 2023-09-30T22:33:41.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.366+0000 7f6d11ffb700 1 -- 172.21.15.16:0/155362270 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d0c003a20 con 0x7f6d14135e50 2023-09-30T22:33:41.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.366+0000 7f6d11ffb700 1 -- 172.21.15.16:0/155362270 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c003dd0 con 0x7f6d14135e50 2023-09-30T22:33:41.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.366+0000 7f6d193db700 1 -- 172.21.15.16:0/155362270 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d14135e50 msgr2=0x7f6d14138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:41.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.366+0000 7f6d193db700 1 --2- 172.21.15.16:0/155362270 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d14135e50 0x7f6d14138230 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c009e50 tx=0x7f6d0c00b330 comp rx=0 tx=0).stop 2023-09-30T22:33:41.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.366+0000 7f6d193db700 1 -- 172.21.15.16:0/155362270 shutdown_connections 2023-09-30T22:33:41.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.366+0000 7f6d193db700 1 --2- 172.21.15.16:0/155362270 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d14135e50 0x7f6d14138230 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:41.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.366+0000 7f6d193db700 1 --2- 172.21.15.16:0/155362270 >> v2:172.21.15.16:3300/0 conn(0x7f6d1409e1d0 0x7f6d14135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:41.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.366+0000 7f6d193db700 1 -- 172.21.15.16:0/155362270 >> 172.21.15.16:0/155362270 conn(0x7f6d14066f50 msgr2=0x7f6d1409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:41.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.367+0000 7f6d193db700 1 -- 172.21.15.16:0/155362270 shutdown_connections 2023-09-30T22:33:41.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.367+0000 7f6d193db700 1 -- 172.21.15.16:0/155362270 wait complete. 2023-09-30T22:33:41.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.368+0000 7f6d193db700 1 Processor -- start 2023-09-30T22:33:41.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.368+0000 7f6d193db700 1 -- start start 2023-09-30T22:33:41.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.369+0000 7f6d193db700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d1409d6f0 0x7f6d1411d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:41.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.369+0000 7f6d193db700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d1409e1d0 0x7f6d1411ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:41.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.369+0000 7f6d193db700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d14135e50 0x7f6d1411e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:41.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.369+0000 7f6d193db700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1413b300 con 0x7f6d1409e1d0 2023-09-30T22:33:41.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.369+0000 7f6d193db700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d141213a0 con 0x7f6d1409d6f0 2023-09-30T22:33:41.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.369+0000 7f6d193db700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1409c700 con 0x7f6d14135e50 2023-09-30T22:33:41.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.369+0000 7f6d137fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d14135e50 0x7f6d1411e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:41.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.369+0000 7f6d137fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d14135e50 0x7f6d1411e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:47448/0 (socket says 172.21.15.16:47448) 2023-09-30T22:33:41.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.369+0000 7f6d137fe700 1 -- 172.21.15.16:0/3321448139 learned_addr learned my addr 172.21.15.16:0/3321448139 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:41.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.369+0000 7f6d127fc700 1 --2- 172.21.15.16:0/3321448139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d1409e1d0 0x7f6d1411ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:41.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.369+0000 7f6d137fe700 1 -- 172.21.15.16:0/3321448139 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d1409d6f0 msgr2=0x7f6d1411d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:41.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.369+0000 7f6d137fe700 1 --2- 172.21.15.16:0/3321448139 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d1409d6f0 0x7f6d1411d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:41.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.369+0000 7f6d137fe700 1 -- 172.21.15.16:0/3321448139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d1409e1d0 msgr2=0x7f6d1411ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:41.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.369+0000 7f6d137fe700 1 --2- 172.21.15.16:0/3321448139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d1409e1d0 0x7f6d1411ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:41.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.369+0000 7f6d137fe700 1 -- 172.21.15.16:0/3321448139 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d080094a0 con 0x7f6d14135e50 2023-09-30T22:33:41.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.370+0000 7f6d137fe700 1 --2- 172.21.15.16:0/3321448139 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d14135e50 0x7f6d1411e310 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c004000 tx=0x7f6d0c004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:41.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.370+0000 7f6cfb7fe700 1 -- 172.21.15.16:0/3321448139 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c004440 con 0x7f6d14135e50 2023-09-30T22:33:41.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.370+0000 7f6cfb7fe700 1 -- 172.21.15.16:0/3321448139 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d0c0092e0 con 0x7f6d14135e50 2023-09-30T22:33:41.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.370+0000 7f6cfb7fe700 1 -- 172.21.15.16:0/3321448139 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c0047e0 con 0x7f6d14135e50 2023-09-30T22:33:41.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.370+0000 7f6d193db700 1 -- 172.21.15.16:0/3321448139 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d0c009b30 con 0x7f6d14135e50 2023-09-30T22:33:41.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.370+0000 7f6d193db700 1 -- 172.21.15.16:0/3321448139 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d14120b30 con 0x7f6d14135e50 2023-09-30T22:33:41.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.371+0000 7f6cfb7fe700 1 -- 172.21.15.16:0/3321448139 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6d0c01e430 con 0x7f6d14135e50 2023-09-30T22:33:41.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.371+0000 7f6d193db700 1 -- 172.21.15.16:0/3321448139 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d14074ea0 con 0x7f6d14135e50 2023-09-30T22:33:41.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.371+0000 7f6cfb7fe700 1 -- 172.21.15.16:0/3321448139 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f6d0c0409c0 con 0x7f6d14135e50 2023-09-30T22:33:41.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.377+0000 7f6cfb7fe700 1 -- 172.21.15.16:0/3321448139 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6d0c019530 con 0x7f6d14135e50 2023-09-30T22:33:41.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.421+0000 7f7798b6b700 1 Processor -- start 2023-09-30T22:33:41.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.421+0000 7f7798b6b700 1 -- start start 2023-09-30T22:33:41.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.421+0000 7f7798b6b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f779406c0e0 0x7f779409ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:41.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.421+0000 7f7798b6b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f77941380f0 0x7f779413fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:41.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.421+0000 7f7798b6b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7794134980 con 0x7f7794134ec0 2023-09-30T22:33:41.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.422+0000 7f7798b6b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f779406c4b0 con 0x7f77941380f0 2023-09-30T22:33:41.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.422+0000 7f7798b6b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f77940a09c0 con 0x7f779406c0e0 2023-09-30T22:33:41.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.422+0000 7f7792d9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f77941380f0 0x7f779413fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:41.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.422+0000 7f7792d9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f77941380f0 0x7f779413fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56078/0 (socket says 172.21.15.16:56078) 2023-09-30T22:33:41.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.422+0000 7f7792d9d700 1 -- 172.21.15.16:0/2634825951 learned_addr learned my addr 172.21.15.16:0/2634825951 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:41.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.422+0000 7f7791d9b700 1 --1- 172.21.15.16:0/2634825951 >> v1:172.21.15.16:6789/0 conn(0x7f7794134ec0 0x7f77941379e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38102/0 (socket says 172.21.15.16:38102) 2023-09-30T22:33:41.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.422+0000 7f779259c700 1 --2- 172.21.15.16:0/2634825951 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f779406c0e0 0x7f779409ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:41.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.422+0000 7f7792d9d700 1 -- 172.21.15.16:0/2634825951 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f779406c0e0 msgr2=0x7f779409ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:41.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.422+0000 7f7792d9d700 1 --2- 172.21.15.16:0/2634825951 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f779406c0e0 0x7f779409ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:41.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.422+0000 7f7792d9d700 1 -- 172.21.15.16:0/2634825951 >> v1:172.21.15.16:6789/0 conn(0x7f7794134ec0 legacy=0x7f77941379e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:41.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.422+0000 7f7792d9d700 1 -- 172.21.15.16:0/2634825951 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77940a0600 con 0x7f77941380f0 2023-09-30T22:33:41.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.422+0000 7f7792d9d700 1 --2- 172.21.15.16:0/2634825951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77941380f0 0x7f779413fd20 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f778c009e50 tx=0x7f778c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a288d8c5027e2bdb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:41.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.423+0000 7f779159a700 1 -- 172.21.15.16:0/2634825951 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778c00e070 con 0x7f77941380f0 2023-09-30T22:33:41.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.423+0000 7f779159a700 1 -- 172.21.15.16:0/2634825951 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f778c00b910 con 0x7f77941380f0 2023-09-30T22:33:41.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.423+0000 7f779159a700 1 -- 172.21.15.16:0/2634825951 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f778c00bcc0 con 0x7f77941380f0 2023-09-30T22:33:41.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.423+0000 7f7798b6b700 1 -- 172.21.15.16:0/2634825951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77941380f0 msgr2=0x7f779413fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:41.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.423+0000 7f7798b6b700 1 --2- 172.21.15.16:0/2634825951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77941380f0 0x7f779413fd20 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f778c009e50 tx=0x7f778c00b330 comp rx=0 tx=0).stop 2023-09-30T22:33:41.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.424+0000 7f7798b6b700 1 -- 172.21.15.16:0/2634825951 shutdown_connections 2023-09-30T22:33:41.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.424+0000 7f7798b6b700 1 --2- 172.21.15.16:0/2634825951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77941380f0 0x7f779413fd20 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:41.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.424+0000 7f7798b6b700 1 --2- 172.21.15.16:0/2634825951 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f779406c0e0 0x7f779409ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:41.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.424+0000 7f7798b6b700 1 -- 172.21.15.16:0/2634825951 >> 172.21.15.16:0/2634825951 conn(0x7f77940a4210 msgr2=0x7f77940a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:41.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.424+0000 7f7798b6b700 1 -- 172.21.15.16:0/2634825951 shutdown_connections 2023-09-30T22:33:41.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.424+0000 7f7798b6b700 1 -- 172.21.15.16:0/2634825951 wait complete. 2023-09-30T22:33:41.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.425+0000 7f7798b6b700 1 Processor -- start 2023-09-30T22:33:41.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.425+0000 7f7798b6b700 1 -- start start 2023-09-30T22:33:41.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.425+0000 7f7798b6b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f779406c0e0 0x7f779411a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:41.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.425+0000 7f7798b6b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7794134ec0 0x7f779411abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:41.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.425+0000 7f7798b6b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f77941380f0 0x7f779411b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:41.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.425+0000 7f7798b6b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779411bb50 con 0x7f779406c0e0 2023-09-30T22:33:41.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.425+0000 7f7798b6b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f779411c630 con 0x7f77941380f0 2023-09-30T22:33:41.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.425+0000 7f7798b6b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f779411bd40 con 0x7f7794134ec0 2023-09-30T22:33:41.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.426+0000 7f779259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f779406c0e0 0x7f779411a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:41.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.426+0000 7f779259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f779406c0e0 0x7f779411a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56090/0 (socket says 172.21.15.16:56090) 2023-09-30T22:33:41.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.426+0000 7f779259c700 1 -- 172.21.15.16:0/165649619 learned_addr learned my addr 172.21.15.16:0/165649619 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:41.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.426+0000 7f7792d9d700 1 --2- 172.21.15.16:0/165649619 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f77941380f0 0x7f779411b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:41.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.426+0000 7f7791d9b700 1 --2- 172.21.15.16:0/165649619 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7794134ec0 0x7f779411abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:41.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.426+0000 7f779259c700 1 -- 172.21.15.16:0/165649619 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7794134ec0 msgr2=0x7f779411abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:41.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.426+0000 7f779259c700 1 --2- 172.21.15.16:0/165649619 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7794134ec0 0x7f779411abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:41.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.426+0000 7f779259c700 1 -- 172.21.15.16:0/165649619 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f77941380f0 msgr2=0x7f779411b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:41.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.426+0000 7f779259c700 1 --2- 172.21.15.16:0/165649619 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f77941380f0 0x7f779411b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:41.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.426+0000 7f779259c700 1 -- 172.21.15.16:0/165649619 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f778c009b30 con 0x7f779406c0e0 2023-09-30T22:33:41.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.426+0000 7f7791d9b700 1 --2- 172.21.15.16:0/165649619 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7794134ec0 0x7f779411abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:41.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.426+0000 7f779259c700 1 --2- 172.21.15.16:0/165649619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f779406c0e0 0x7f779411a680 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f778400e870 tx=0x7f778400eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:41.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.427+0000 7f7782ffd700 1 -- 172.21.15.16:0/165649619 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7784013680 con 0x7f779406c0e0 2023-09-30T22:33:41.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.427+0000 7f7782ffd700 1 -- 172.21.15.16:0/165649619 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77840137e0 con 0x7f779406c0e0 2023-09-30T22:33:41.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.427+0000 7f7798b6b700 1 -- 172.21.15.16:0/165649619 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77940a3a80 con 0x7f779406c0e0 2023-09-30T22:33:41.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.427+0000 7f7782ffd700 1 -- 172.21.15.16:0/165649619 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7784013b50 con 0x7f779406c0e0 2023-09-30T22:33:41.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.427+0000 7f7798b6b700 1 -- 172.21.15.16:0/165649619 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f779411d780 con 0x7f779406c0e0 2023-09-30T22:33:41.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.428+0000 7f7782ffd700 1 -- 172.21.15.16:0/165649619 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7784013cb0 con 0x7f779406c0e0 2023-09-30T22:33:41.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.428+0000 7f7798b6b700 1 -- 172.21.15.16:0/165649619 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7794074ea0 con 0x7f779406c0e0 2023-09-30T22:33:41.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.428+0000 7f7782ffd700 1 -- 172.21.15.16:0/165649619 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f778403f570 con 0x7f779406c0e0 2023-09-30T22:33:41.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.437+0000 7f7782ffd700 1 -- 172.21.15.16:0/165649619 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f778401be10 con 0x7f779406c0e0 2023-09-30T22:33:41.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.474+0000 7f0f7d589700 1 Processor -- start 2023-09-30T22:33:41.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.474+0000 7f0f7d589700 1 -- start start 2023-09-30T22:33:41.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.475+0000 7f0f7d589700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0f7809d6f0 0x7f0f7809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:41.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.475+0000 7f0f7d589700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0f7809e410 0x7f0f78130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:41.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.475+0000 7f0f7d589700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f7809f130 con 0x7f0f7809e000 2023-09-30T22:33:41.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.475+0000 7f0f7d589700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7809f520 con 0x7f0f7809d6f0 2023-09-30T22:33:41.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.475+0000 7f0f7d589700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0f7809e930 con 0x7f0f7809e410 2023-09-30T22:33:41.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.476+0000 7f0f767fc700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0f7809e000 0x7f0f78130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38112/0 (socket says 172.21.15.16:38112) 2023-09-30T22:33:41.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.476+0000 7f0f767fc700 1 -- 172.21.15.16:0/3564092252 learned_addr learned my addr 172.21.15.16:0/3564092252 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:41.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.476+0000 7f0f76ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0f7809d6f0 0x7f0f7809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:41.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.476+0000 7f0f777fe700 1 --2- 172.21.15.16:0/3564092252 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0f7809e410 0x7f0f78130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:41.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.476+0000 7f0f76ffd700 1 -- 172.21.15.16:0/3564092252 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0f7809e410 msgr2=0x7f0f78130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:41.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.476+0000 7f0f76ffd700 1 --2- 172.21.15.16:0/3564092252 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0f7809e410 0x7f0f78130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:41.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.476+0000 7f0f76ffd700 1 -- 172.21.15.16:0/3564092252 >> v1:172.21.15.16:6789/0 conn(0x7f0f7809e000 legacy=0x7f0f78130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:41.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.476+0000 7f0f76ffd700 1 -- 172.21.15.16:0/3564092252 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f7809b2d0 con 0x7f0f7809d6f0 2023-09-30T22:33:41.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.476+0000 7f0f777fe700 1 --2- 172.21.15.16:0/3564092252 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0f7809e410 0x7f0f78130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:41.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.476+0000 7f0f76ffd700 1 --2- 172.21.15.16:0/3564092252 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0f7809d6f0 0x7f0f7809dac0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f0f68009870 tx=0x7f0f68009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=223b8c40ddfc328e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:41.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.476+0000 7f0f75ffb700 1 -- 172.21.15.16:0/3564092252 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f68004230 con 0x7f0f7809d6f0 2023-09-30T22:33:41.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.476+0000 7f0f75ffb700 1 -- 172.21.15.16:0/3564092252 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f68004390 con 0x7f0f7809d6f0 2023-09-30T22:33:41.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.477+0000 7f0f75ffb700 1 -- 172.21.15.16:0/3564092252 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f680036a0 con 0x7f0f7809d6f0 2023-09-30T22:33:41.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.477+0000 7f0f7d589700 1 -- 172.21.15.16:0/3564092252 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0f7809d6f0 msgr2=0x7f0f7809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:41.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.477+0000 7f0f7d589700 1 --2- 172.21.15.16:0/3564092252 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0f7809d6f0 0x7f0f7809dac0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f0f68009870 tx=0x7f0f68009b80 comp rx=0 tx=0).stop 2023-09-30T22:33:41.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.478+0000 7f0f7d589700 1 -- 172.21.15.16:0/3564092252 shutdown_connections 2023-09-30T22:33:41.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.478+0000 7f0f7d589700 1 --2- 172.21.15.16:0/3564092252 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0f7809e410 0x7f0f78130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:41.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.478+0000 7f0f7d589700 1 --2- 172.21.15.16:0/3564092252 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0f7809d6f0 0x7f0f7809dac0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:41.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.478+0000 7f0f7d589700 1 -- 172.21.15.16:0/3564092252 >> 172.21.15.16:0/3564092252 conn(0x7f0f78066f50 msgr2=0x7f0f7809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:41.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.478+0000 7f0f7d589700 1 -- 172.21.15.16:0/3564092252 shutdown_connections 2023-09-30T22:33:41.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.478+0000 7f0f7d589700 1 -- 172.21.15.16:0/3564092252 wait complete. 2023-09-30T22:33:41.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.479+0000 7f0f7d589700 1 Processor -- start 2023-09-30T22:33:41.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.479+0000 7f0f7d589700 1 -- start start 2023-09-30T22:33:41.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.480+0000 7f0f7d589700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f7809d6f0 0x7f0f781b8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:41.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.480+0000 7f0f7d589700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0f7809e000 0x7f0f781b8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:41.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.480+0000 7f0f7d589700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0f7809e410 0x7f0f781cc810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:41.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.480+0000 7f0f7d589700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f781baf20 con 0x7f0f7809d6f0 2023-09-30T22:33:41.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.480+0000 7f0f7d589700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f781bb990 con 0x7f0f7809e410 2023-09-30T22:33:41.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.480+0000 7f0f7d589700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0f781bb200 con 0x7f0f7809e000 2023-09-30T22:33:41.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.480+0000 7f0f76ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f7809d6f0 0x7f0f781b8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:41.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.480+0000 7f0f76ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f7809d6f0 0x7f0f781b8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56094/0 (socket says 172.21.15.16:56094) 2023-09-30T22:33:41.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.480+0000 7f0f76ffd700 1 -- 172.21.15.16:0/201053905 learned_addr learned my addr 172.21.15.16:0/201053905 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:41.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.480+0000 7f0f767fc700 1 --2- 172.21.15.16:0/201053905 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0f7809e000 0x7f0f781b8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:41.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.480+0000 7f0f777fe700 1 --2- 172.21.15.16:0/201053905 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0f7809e410 0x7f0f781cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:41.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.481+0000 7f0f767fc700 1 -- 172.21.15.16:0/201053905 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0f7809e410 msgr2=0x7f0f781cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:41.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.481+0000 7f0f767fc700 1 --2- 172.21.15.16:0/201053905 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0f7809e410 0x7f0f781cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:41.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.481+0000 7f0f767fc700 1 -- 172.21.15.16:0/201053905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f7809d6f0 msgr2=0x7f0f781b8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:41.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.481+0000 7f0f767fc700 1 --2- 172.21.15.16:0/201053905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f7809d6f0 0x7f0f781b8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:41.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.481+0000 7f0f767fc700 1 -- 172.21.15.16:0/201053905 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f70009a70 con 0x7f0f7809e000 2023-09-30T22:33:41.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.481+0000 7f0f767fc700 1 --2- 172.21.15.16:0/201053905 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0f7809e000 0x7f0f781b8580 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c00c750 tx=0x7f0f6c00ca60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:41.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.481+0000 7f0f5f7fe700 1 -- 172.21.15.16:0/201053905 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c004c80 con 0x7f0f7809e000 2023-09-30T22:33:41.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.481+0000 7f0f5f7fe700 1 -- 172.21.15.16:0/201053905 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f6c004de0 con 0x7f0f7809e000 2023-09-30T22:33:41.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.481+0000 7f0f7d589700 1 -- 172.21.15.16:0/201053905 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f680094a0 con 0x7f0f7809e000 2023-09-30T22:33:41.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.481+0000 7f0f5f7fe700 1 -- 172.21.15.16:0/201053905 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f6c007310 con 0x7f0f7809e000 2023-09-30T22:33:41.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.482+0000 7f0f7d589700 1 -- 172.21.15.16:0/201053905 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f781b93c0 con 0x7f0f7809e000 2023-09-30T22:33:41.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.483+0000 7f0f7d589700 1 -- 172.21.15.16:0/201053905 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f78074ea0 con 0x7f0f7809e000 2023-09-30T22:33:41.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.483+0000 7f0f5f7fe700 1 -- 172.21.15.16:0/201053905 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0f6c007470 con 0x7f0f7809e000 2023-09-30T22:33:41.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.484+0000 7f0f5f7fe700 1 -- 172.21.15.16:0/201053905 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f0f6c03f8c0 con 0x7f0f7809e000 2023-09-30T22:33:41.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.492+0000 7f0f5f7fe700 1 -- 172.21.15.16:0/201053905 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0f6c007720 con 0x7f0f7809e000 2023-09-30T22:33:41.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.521+0000 7f069b186700 1 Processor -- start 2023-09-30T22:33:41.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.521+0000 7f069b186700 1 -- start start 2023-09-30T22:33:41.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.522+0000 7f069b186700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f069409e930 0x7f069409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:41.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.522+0000 7f069b186700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f069409f650 0x7f0694131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:41.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.522+0000 7f069b186700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0694070940 con 0x7f069409f240 2023-09-30T22:33:41.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.522+0000 7f069b186700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f069406fd60 con 0x7f069409f650 2023-09-30T22:33:41.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.522+0000 7f069b186700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f069406fa80 con 0x7f069409e930 2023-09-30T22:33:41.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.522+0000 7f068bfff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f069409f240 0x7f06941317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38128/0 (socket says 172.21.15.16:38128) 2023-09-30T22:33:41.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.522+0000 7f068bfff700 1 -- 172.21.15.16:0/546507884 learned_addr learned my addr 172.21.15.16:0/546507884 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:41.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.522+0000 7f0698f22700 1 --2- 172.21.15.16:0/546507884 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f069409e930 0x7f069409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:41.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.522+0000 7f0699723700 1 --2- 172.21.15.16:0/546507884 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f069409f650 0x7f0694131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:41.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.523+0000 7f068b7fe700 1 -- 172.21.15.16:0/546507884 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2476018735 0 0) 0x7f0694070940 con 0x7f069409f240 2023-09-30T22:33:41.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.523+0000 7f068b7fe700 1 -- 172.21.15.16:0/546507884 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f067c0035b0 con 0x7f069409f240 2023-09-30T22:33:41.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.523+0000 7f0699723700 1 -- 172.21.15.16:0/546507884 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f069409e930 msgr2=0x7f069409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:41.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.523+0000 7f0699723700 1 --2- 172.21.15.16:0/546507884 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f069409e930 0x7f069409ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:41.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.523+0000 7f0699723700 1 -- 172.21.15.16:0/546507884 >> v1:172.21.15.16:6789/0 conn(0x7f069409f240 legacy=0x7f06941317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:41.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.523+0000 7f0699723700 1 -- 172.21.15.16:0/546507884 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f069409fb60 con 0x7f069409f650 2023-09-30T22:33:41.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.523+0000 7f0698f22700 1 --2- 172.21.15.16:0/546507884 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f069409e930 0x7f069409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:41.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.523+0000 7f0699723700 1 --2- 172.21.15.16:0/546507884 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f069409f650 0x7f0694131f00 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f068c009dc0 tx=0x7f068c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d4c865920e10f44 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:41.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.523+0000 7f068b7fe700 1 -- 172.21.15.16:0/546507884 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f068c00e070 con 0x7f069409f650 2023-09-30T22:33:41.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.523+0000 7f068b7fe700 1 -- 172.21.15.16:0/546507884 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f068c00bb20 con 0x7f069409f650 2023-09-30T22:33:41.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.523+0000 7f068b7fe700 1 -- 172.21.15.16:0/546507884 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f068c012430 con 0x7f069409f650 2023-09-30T22:33:41.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.524+0000 7f069b186700 1 -- 172.21.15.16:0/546507884 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f069409f650 msgr2=0x7f0694131f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:41.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.524+0000 7f069b186700 1 --2- 172.21.15.16:0/546507884 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f069409f650 0x7f0694131f00 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f068c009dc0 tx=0x7f068c00b330 comp rx=0 tx=0).stop 2023-09-30T22:33:41.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.524+0000 7f069b186700 1 -- 172.21.15.16:0/546507884 shutdown_connections 2023-09-30T22:33:41.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.524+0000 7f069b186700 1 --2- 172.21.15.16:0/546507884 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f069409f650 0x7f0694131f00 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:41.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.524+0000 7f069b186700 1 --2- 172.21.15.16:0/546507884 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f069409e930 0x7f069409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:41.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.524+0000 7f069b186700 1 -- 172.21.15.16:0/546507884 >> 172.21.15.16:0/546507884 conn(0x7f06940a4210 msgr2=0x7f06940a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:41.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.524+0000 7f069b186700 1 -- 172.21.15.16:0/546507884 shutdown_connections 2023-09-30T22:33:41.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.524+0000 7f069b186700 1 -- 172.21.15.16:0/546507884 wait complete. 2023-09-30T22:33:41.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.526+0000 7f069b186700 1 Processor -- start 2023-09-30T22:33:41.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.526+0000 7f069b186700 1 -- start start 2023-09-30T22:33:41.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.527+0000 7f069b186700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f069409e930 0x7f06941c8610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:41.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.527+0000 7f069b186700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f069409f240 0x7f06941c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:41.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.527+0000 7f068bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f069409f240 0x7f06941c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:41.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.527+0000 7f068bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f069409f240 0x7f06941c8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56106/0 (socket says 172.21.15.16:56106) 2023-09-30T22:33:41.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.527+0000 7f069b186700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f069409f650 0x7f06941a4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:41.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.527+0000 7f0698f22700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f069409e930 0x7f06941c8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:41.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.527+0000 7f069b186700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06941bd010 con 0x7f069409f240 2023-09-30T22:33:41.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.527+0000 7f069b186700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06941b9190 con 0x7f069409e930 2023-09-30T22:33:41.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.527+0000 7f0698f22700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f069409e930 0x7f06941c8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:51862/0 (socket says 172.21.15.16:51862) 2023-09-30T22:33:41.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.527+0000 7f0698f22700 1 -- 172.21.15.16:0/2707408227 learned_addr learned my addr 172.21.15.16:0/2707408227 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:41.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.527+0000 7f069b186700 1 -- 172.21.15.16:0/2707408227 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f06941a80d0 con 0x7f069409f650 2023-09-30T22:33:41.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.528+0000 7f0698f22700 1 -- 172.21.15.16:0/2707408227 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f069409f650 msgr2=0x7f06941a4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:41.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.528+0000 7f0698f22700 1 --2- 172.21.15.16:0/2707408227 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f069409f650 0x7f06941a4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:41.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.528+0000 7f0698f22700 1 -- 172.21.15.16:0/2707408227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f069409f240 msgr2=0x7f06941c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:41.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.528+0000 7f0698f22700 1 --2- 172.21.15.16:0/2707408227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f069409f240 0x7f06941c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:41.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.528+0000 7f0698f22700 1 -- 172.21.15.16:0/2707408227 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06900094a0 con 0x7f069409e930 2023-09-30T22:33:41.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.528+0000 7f0698f22700 1 --2- 172.21.15.16:0/2707408227 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f069409e930 0x7f06941c8610 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f069000ec10 tx=0x7f0690009ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:41.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.528+0000 7f06897fa700 1 -- 172.21.15.16:0/2707408227 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069000c940 con 0x7f069409e930 2023-09-30T22:33:41.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.528+0000 7f06897fa700 1 -- 172.21.15.16:0/2707408227 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f069000caa0 con 0x7f069409e930 2023-09-30T22:33:41.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.528+0000 7f069b186700 1 -- 172.21.15.16:0/2707408227 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f068c009a70 con 0x7f069409e930 2023-09-30T22:33:41.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.528+0000 7f069b186700 1 -- 172.21.15.16:0/2707408227 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06940a0190 con 0x7f069409e930 2023-09-30T22:33:41.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.528+0000 7f06897fa700 1 -- 172.21.15.16:0/2707408227 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f069000ce10 con 0x7f069409e930 2023-09-30T22:33:41.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.529+0000 7f069b186700 1 -- 172.21.15.16:0/2707408227 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06740052e0 con 0x7f069409e930 2023-09-30T22:33:41.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.530+0000 7f06897fa700 1 -- 172.21.15.16:0/2707408227 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0690013470 con 0x7f069409e930 2023-09-30T22:33:41.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.530+0000 7f06897fa700 1 -- 172.21.15.16:0/2707408227 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f069003f8e0 con 0x7f069409e930 2023-09-30T22:33:41.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.535+0000 7f06897fa700 1 -- 172.21.15.16:0/2707408227 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f06900232f0 con 0x7f069409e930 2023-09-30T22:33:41.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.599+0000 7f6d193db700 1 -- 172.21.15.16:0/3321448139 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6d1411fc80 con 0x7f6d14135e50 2023-09-30T22:33:41.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.599+0000 7f6cfb7fe700 1 -- 172.21.15.16:0/3321448139 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6d0c017030 con 0x7f6d14135e50 2023-09-30T22:33:41.600 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:33:41.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.603+0000 7f6d193db700 1 -- 172.21.15.16:0/3321448139 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d14135e50 msgr2=0x7f6d1411e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:41.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.603+0000 7f6d193db700 1 --2- 172.21.15.16:0/3321448139 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d14135e50 0x7f6d1411e310 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c004000 tx=0x7f6d0c004ad0 comp rx=0 tx=0).stop 2023-09-30T22:33:41.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.603+0000 7f6d193db700 1 -- 172.21.15.16:0/3321448139 shutdown_connections 2023-09-30T22:33:41.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.603+0000 7f6d193db700 1 --2- 172.21.15.16:0/3321448139 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d1409d6f0 0x7f6d1411d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:41.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.603+0000 7f6d193db700 1 --2- 172.21.15.16:0/3321448139 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d14135e50 0x7f6d1411e310 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:41.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.603+0000 7f6d193db700 1 --2- 172.21.15.16:0/3321448139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d1409e1d0 0x7f6d1411ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:41.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.603+0000 7f6d193db700 1 -- 172.21.15.16:0/3321448139 >> 172.21.15.16:0/3321448139 conn(0x7f6d14066f50 msgr2=0x7f6d140a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:41.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.604+0000 7f6d193db700 1 -- 172.21.15.16:0/3321448139 shutdown_connections 2023-09-30T22:33:41.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.606+0000 7f6d193db700 1 -- 172.21.15.16:0/3321448139 wait complete. 2023-09-30T22:33:41.625 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:33:41.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.684+0000 7f7798b6b700 1 -- 172.21.15.16:0/165649619 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f779406b250 con 0x7f779406c0e0 2023-09-30T22:33:41.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.685+0000 7f7782ffd700 1 -- 172.21.15.16:0/165649619 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7784021070 con 0x7f779406c0e0 2023-09-30T22:33:41.685 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:33:41.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.687+0000 7f7798b6b700 1 -- 172.21.15.16:0/165649619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f779406c0e0 msgr2=0x7f779411a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:41.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.687+0000 7f7798b6b700 1 --2- 172.21.15.16:0/165649619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f779406c0e0 0x7f779411a680 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f778400e870 tx=0x7f778400eb80 comp rx=0 tx=0).stop 2023-09-30T22:33:41.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.687+0000 7f7798b6b700 1 -- 172.21.15.16:0/165649619 shutdown_connections 2023-09-30T22:33:41.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.687+0000 7f7798b6b700 1 --2- 172.21.15.16:0/165649619 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f77941380f0 0x7f779411b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:41.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.687+0000 7f7798b6b700 1 --2- 172.21.15.16:0/165649619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f779406c0e0 0x7f779411a680 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:41.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.687+0000 7f7798b6b700 1 --2- 172.21.15.16:0/165649619 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7794134ec0 0x7f779411abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:41.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.687+0000 7f7798b6b700 1 -- 172.21.15.16:0/165649619 >> 172.21.15.16:0/165649619 conn(0x7f77940a4210 msgr2=0x7f77941378a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:41.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.687+0000 7f7798b6b700 1 -- 172.21.15.16:0/165649619 shutdown_connections 2023-09-30T22:33:41.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.687+0000 7f7798b6b700 1 -- 172.21.15.16:0/165649619 wait complete. 2023-09-30T22:33:41.701 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:33:41.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.719+0000 7f0f7d589700 1 -- 172.21.15.16:0/201053905 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0f78072350 con 0x7f0f7809e000 2023-09-30T22:33:41.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.719+0000 7f0f5f7fe700 1 -- 172.21.15.16:0/201053905 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0f6c011e80 con 0x7f0f7809e000 2023-09-30T22:33:41.719 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:33:41.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.721+0000 7f0f7d589700 1 -- 172.21.15.16:0/201053905 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0f7809e000 msgr2=0x7f0f781b8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:41.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.721+0000 7f0f7d589700 1 --2- 172.21.15.16:0/201053905 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0f7809e000 0x7f0f781b8580 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f0f6c00c750 tx=0x7f0f6c00ca60 comp rx=0 tx=0).stop 2023-09-30T22:33:41.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.721+0000 7f0f7d589700 1 -- 172.21.15.16:0/201053905 shutdown_connections 2023-09-30T22:33:41.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.721+0000 7f0f7d589700 1 --2- 172.21.15.16:0/201053905 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0f7809e410 0x7f0f781cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:41.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.721+0000 7f0f7d589700 1 --2- 172.21.15.16:0/201053905 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f7809d6f0 0x7f0f781b8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:41.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.721+0000 7f0f7d589700 1 --2- 172.21.15.16:0/201053905 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0f7809e000 0x7f0f781b8580 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:41.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.721+0000 7f0f7d589700 1 -- 172.21.15.16:0/201053905 >> 172.21.15.16:0/201053905 conn(0x7f0f78066f50 msgr2=0x7f0f780710a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:41.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.722+0000 7f0f7d589700 1 -- 172.21.15.16:0/201053905 shutdown_connections 2023-09-30T22:33:41.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.722+0000 7f0f7d589700 1 -- 172.21.15.16:0/201053905 wait complete. 2023-09-30T22:33:41.735 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:33:41.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.746+0000 7f069b186700 1 -- 172.21.15.16:0/2707408227 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0674005590 con 0x7f069409e930 2023-09-30T22:33:41.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.747+0000 7f06897fa700 1 -- 172.21.15.16:0/2707408227 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f069001bb60 con 0x7f069409e930 2023-09-30T22:33:41.747 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:33:41.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.749+0000 7f069b186700 1 -- 172.21.15.16:0/2707408227 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f069409e930 msgr2=0x7f06941c8610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:41.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.749+0000 7f069b186700 1 --2- 172.21.15.16:0/2707408227 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f069409e930 0x7f06941c8610 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f069000ec10 tx=0x7f0690009ec0 comp rx=0 tx=0).stop 2023-09-30T22:33:41.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.749+0000 7f069b186700 1 -- 172.21.15.16:0/2707408227 shutdown_connections 2023-09-30T22:33:41.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.749+0000 7f069b186700 1 --2- 172.21.15.16:0/2707408227 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f069409e930 0x7f06941c8610 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:41.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.749+0000 7f069b186700 1 --2- 172.21.15.16:0/2707408227 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f069409f650 0x7f06941a4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:41.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.749+0000 7f069b186700 1 --2- 172.21.15.16:0/2707408227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f069409f240 0x7f06941c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:41.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.749+0000 7f069b186700 1 -- 172.21.15.16:0/2707408227 >> 172.21.15.16:0/2707408227 conn(0x7f06940a4210 msgr2=0x7f069406c320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:41.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.749+0000 7f069b186700 1 -- 172.21.15.16:0/2707408227 shutdown_connections 2023-09-30T22:33:41.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:41.749+0000 7f069b186700 1 -- 172.21.15.16:0/2707408227 wait complete. 2023-09-30T22:33:41.763 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:33:42.625 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:33:42.702 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:33:42.738 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:33:42.764 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:33:42.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.777+0000 7f0c991da700 1 Processor -- start 2023-09-30T22:33:42.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.777+0000 7f0c991da700 1 -- start start 2023-09-30T22:33:42.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.778+0000 7f0c991da700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0c94130670 0x7f0c94130a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:42.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.778+0000 7f0c991da700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0c940701a0 0x7f0c94138540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:42.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.778+0000 7f0c991da700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c94130260 con 0x7f0c9406f680 2023-09-30T22:33:42.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.778+0000 7f0c991da700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c9409a350 con 0x7f0c94130670 2023-09-30T22:33:42.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.778+0000 7f0c991da700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c94071ff0 con 0x7f0c940701a0 2023-09-30T22:33:42.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.778+0000 7f0c9359e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0c940701a0 0x7f0c94138540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:42.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.779+0000 7f0c9359e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0c940701a0 0x7f0c94138540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:51876/0 (socket says 172.21.15.16:51876) 2023-09-30T22:33:42.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.779+0000 7f0c9359e700 1 -- 172.21.15.16:0/1592651649 learned_addr learned my addr 172.21.15.16:0/1592651649 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:42.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.779+0000 7f0c92d9d700 1 --2- 172.21.15.16:0/1592651649 >> v2:172.21.15.16:3300/0 conn(0x7f0c94130670 0x7f0c94130a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:42.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.779+0000 7f0c92d9d700 1 -- 172.21.15.16:0/1592651649 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0c940701a0 msgr2=0x7f0c94138540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:42.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.779+0000 7f0c92d9d700 1 --2- 172.21.15.16:0/1592651649 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0c940701a0 0x7f0c94138540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:42.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.779+0000 7f0c92d9d700 1 -- 172.21.15.16:0/1592651649 >> v1:172.21.15.16:6789/0 conn(0x7f0c9406f680 legacy=0x7f0c9406fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:42.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.779+0000 7f0c92d9d700 1 -- 172.21.15.16:0/1592651649 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c9409ecf0 con 0x7f0c94130670 2023-09-30T22:33:42.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.779+0000 7f0c92d9d700 1 --2- 172.21.15.16:0/1592651649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c94130670 0x7f0c94130a40 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f0c84009840 tx=0x7f0c84009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=548ecfd985a5beb4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:42.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.779+0000 7f0c91d9b700 1 -- 172.21.15.16:0/1592651649 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c84004230 con 0x7f0c94130670 2023-09-30T22:33:42.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.779+0000 7f0c91d9b700 1 -- 172.21.15.16:0/1592651649 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c84004390 con 0x7f0c94130670 2023-09-30T22:33:42.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.779+0000 7f0c91d9b700 1 -- 172.21.15.16:0/1592651649 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c840036a0 con 0x7f0c94130670 2023-09-30T22:33:42.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.780+0000 7f0c991da700 1 -- 172.21.15.16:0/1592651649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c94130670 msgr2=0x7f0c94130a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:42.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.780+0000 7f0c991da700 1 --2- 172.21.15.16:0/1592651649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c94130670 0x7f0c94130a40 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f0c84009840 tx=0x7f0c84009b50 comp rx=0 tx=0).stop 2023-09-30T22:33:42.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.780+0000 7f0c991da700 1 -- 172.21.15.16:0/1592651649 shutdown_connections 2023-09-30T22:33:42.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.780+0000 7f0c991da700 1 --2- 172.21.15.16:0/1592651649 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0c940701a0 0x7f0c94138540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:42.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.780+0000 7f0c991da700 1 --2- 172.21.15.16:0/1592651649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c94130670 0x7f0c94130a40 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:42.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.780+0000 7f0c991da700 1 -- 172.21.15.16:0/1592651649 >> 172.21.15.16:0/1592651649 conn(0x7f0c940a2aa0 msgr2=0x7f0c940a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:42.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.780+0000 7f0c991da700 1 -- 172.21.15.16:0/1592651649 shutdown_connections 2023-09-30T22:33:42.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.780+0000 7f0c991da700 1 -- 172.21.15.16:0/1592651649 wait complete. 2023-09-30T22:33:42.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.782+0000 7f0c991da700 1 Processor -- start 2023-09-30T22:33:42.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.782+0000 7f0c991da700 1 -- start start 2023-09-30T22:33:42.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.783+0000 7f0c991da700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0c9406f680 0x7f0c9411b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:42.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.783+0000 7f0c991da700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0c940701a0 0x7f0c9411be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:42.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.783+0000 7f0c991da700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c94130670 0x7f0c9411c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:42.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.783+0000 7f0c991da700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9411ddc0 con 0x7f0c94130670 2023-09-30T22:33:42.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.783+0000 7f0c991da700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c94071820 con 0x7f0c9406f680 2023-09-30T22:33:42.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.783+0000 7f0c991da700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0c9411daa0 con 0x7f0c940701a0 2023-09-30T22:33:42.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.783+0000 7f0c92d9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0c9406f680 0x7f0c9411b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:42.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.783+0000 7f0c92d9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0c9406f680 0x7f0c9411b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:51884/0 (socket says 172.21.15.16:51884) 2023-09-30T22:33:42.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.783+0000 7f0c92d9d700 1 -- 172.21.15.16:0/3980283723 learned_addr learned my addr 172.21.15.16:0/3980283723 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:42.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.784+0000 7f0c9259c700 1 --2- 172.21.15.16:0/3980283723 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0c940701a0 0x7f0c9411be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:42.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.784+0000 7f0c9359e700 1 --2- 172.21.15.16:0/3980283723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c94130670 0x7f0c9411c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:42.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.784+0000 7f0c92d9d700 1 -- 172.21.15.16:0/3980283723 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0c940701a0 msgr2=0x7f0c9411be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:42.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.784+0000 7f0c92d9d700 1 --2- 172.21.15.16:0/3980283723 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0c940701a0 0x7f0c9411be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:42.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.784+0000 7f0c92d9d700 1 -- 172.21.15.16:0/3980283723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c94130670 msgr2=0x7f0c9411c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:42.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.784+0000 7f0c92d9d700 1 --2- 172.21.15.16:0/3980283723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c94130670 0x7f0c9411c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:42.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.784+0000 7f0c92d9d700 1 -- 172.21.15.16:0/3980283723 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c840094a0 con 0x7f0c9406f680 2023-09-30T22:33:42.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.784+0000 7f0c9359e700 1 --2- 172.21.15.16:0/3980283723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c94130670 0x7f0c9411c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:33:42.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.784+0000 7f0c92d9d700 1 --2- 172.21.15.16:0/3980283723 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0c9406f680 0x7f0c9411b8e0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f0c84000c00 tx=0x7f0c84004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:42.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.784+0000 7f0c7b7fe700 1 -- 172.21.15.16:0/3980283723 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c84003e10 con 0x7f0c9406f680 2023-09-30T22:33:42.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.784+0000 7f0c7b7fe700 1 -- 172.21.15.16:0/3980283723 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c84011630 con 0x7f0c9406f680 2023-09-30T22:33:42.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.784+0000 7f0c7b7fe700 1 -- 172.21.15.16:0/3980283723 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c840119e0 con 0x7f0c9406f680 2023-09-30T22:33:42.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.785+0000 7f0c991da700 1 -- 172.21.15.16:0/3980283723 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c9409fd30 con 0x7f0c9406f680 2023-09-30T22:33:42.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.785+0000 7f0c991da700 1 -- 172.21.15.16:0/3980283723 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c9411ebf0 con 0x7f0c9406f680 2023-09-30T22:33:42.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.786+0000 7f0c7b7fe700 1 -- 172.21.15.16:0/3980283723 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0c84018040 con 0x7f0c9406f680 2023-09-30T22:33:42.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.787+0000 7f0c7b7fe700 1 -- 172.21.15.16:0/3980283723 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f0c84012e40 con 0x7f0c9406f680 2023-09-30T22:33:42.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.787+0000 7f0c991da700 1 -- 172.21.15.16:0/3980283723 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c94074ea0 con 0x7f0c9406f680 2023-09-30T22:33:42.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.793+0000 7f0c7b7fe700 1 -- 172.21.15.16:0/3980283723 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0c8402b030 con 0x7f0c9406f680 2023-09-30T22:33:42.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.848+0000 7f0123876700 1 Processor -- start 2023-09-30T22:33:42.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.848+0000 7f0123876700 1 -- start start 2023-09-30T22:33:42.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.848+0000 7f0123876700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f011c136710 0x7f011c138af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:42.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.848+0000 7f0123876700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f011c139030 0x7f011c13b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:42.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.849+0000 7f0123876700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f011c06ef30 con 0x7f011c1338a0 2023-09-30T22:33:42.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.849+0000 7f0123876700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f011c06fef0 con 0x7f011c136710 2023-09-30T22:33:42.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.849+0000 7f0123876700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f011c1334b0 con 0x7f011c139030 2023-09-30T22:33:42.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.849+0000 7f0121e13700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f011c139030 0x7f011c13b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:42.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.849+0000 7f0121e13700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f011c139030 0x7f011c13b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:47518/0 (socket says 172.21.15.16:47518) 2023-09-30T22:33:42.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.849+0000 7f0121e13700 1 -- 172.21.15.16:0/1493865131 learned_addr learned my addr 172.21.15.16:0/1493865131 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:42.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.849+0000 7f0121612700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f011c1338a0 0x7f011c133c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38132/0 (socket says 172.21.15.16:38132) 2023-09-30T22:33:42.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.849+0000 7f0120e11700 1 --2- 172.21.15.16:0/1493865131 >> v2:172.21.15.16:3300/0 conn(0x7f011c136710 0x7f011c138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:42.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.849+0000 7f0120e11700 1 -- 172.21.15.16:0/1493865131 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f011c139030 msgr2=0x7f011c13b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:42.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.849+0000 7f0120e11700 1 --2- 172.21.15.16:0/1493865131 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f011c139030 0x7f011c13b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:42.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.849+0000 7f0120e11700 1 -- 172.21.15.16:0/1493865131 >> v1:172.21.15.16:6789/0 conn(0x7f011c1338a0 legacy=0x7f011c133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:42.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.849+0000 7f0113fff700 1 -- 172.21.15.16:0/1493865131 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2822774037 0 0) 0x7f011c06ef30 con 0x7f011c1338a0 2023-09-30T22:33:42.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.849+0000 7f0120e11700 1 -- 172.21.15.16:0/1493865131 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f011c06fc30 con 0x7f011c136710 2023-09-30T22:33:42.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.850+0000 7f0120e11700 1 --2- 172.21.15.16:0/1493865131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f011c136710 0x7f011c138af0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f010c009840 tx=0x7f010c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2873cb9b72aa1a3e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:42.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.850+0000 7f0113fff700 1 -- 172.21.15.16:0/1493865131 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010c011830 con 0x7f011c136710 2023-09-30T22:33:42.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.850+0000 7f0113fff700 1 -- 172.21.15.16:0/1493865131 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f010c011990 con 0x7f011c136710 2023-09-30T22:33:42.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.850+0000 7f0113fff700 1 -- 172.21.15.16:0/1493865131 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f010c011d20 con 0x7f011c136710 2023-09-30T22:33:42.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.850+0000 7f0123876700 1 -- 172.21.15.16:0/1493865131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f011c136710 msgr2=0x7f011c138af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:42.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.850+0000 7f0123876700 1 --2- 172.21.15.16:0/1493865131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f011c136710 0x7f011c138af0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f010c009840 tx=0x7f010c009b50 comp rx=0 tx=0).stop 2023-09-30T22:33:42.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.851+0000 7f0123876700 1 -- 172.21.15.16:0/1493865131 shutdown_connections 2023-09-30T22:33:42.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.851+0000 7f0123876700 1 --2- 172.21.15.16:0/1493865131 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f011c139030 0x7f011c13b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:42.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.851+0000 7f0123876700 1 --2- 172.21.15.16:0/1493865131 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f011c136710 0x7f011c138af0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:42.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.851+0000 7f0123876700 1 -- 172.21.15.16:0/1493865131 >> 172.21.15.16:0/1493865131 conn(0x7f011c0a2aa0 msgr2=0x7f011c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:42.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.851+0000 7f0123876700 1 -- 172.21.15.16:0/1493865131 shutdown_connections 2023-09-30T22:33:42.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.851+0000 7f0123876700 1 -- 172.21.15.16:0/1493865131 wait complete. 2023-09-30T22:33:42.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.852+0000 7f0123876700 1 Processor -- start 2023-09-30T22:33:42.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.852+0000 7f0123876700 1 -- start start 2023-09-30T22:33:42.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.852+0000 7f0123876700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f011c1338a0 0x7f011c11ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:42.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.852+0000 7f0123876700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f011c136710 0x7f011c11b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:42.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.852+0000 7f0123876700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f011c139030 0x7f011c11b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.852+0000 7f0123876700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f011c11e770 con 0x7f011c136710 2023-09-30T22:33:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.852+0000 7f0123876700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f011c138d90 con 0x7f011c139030 2023-09-30T22:33:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.852+0000 7f0123876700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f011c06f640 con 0x7f011c1338a0 2023-09-30T22:33:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.853+0000 7f0121612700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f011c1338a0 0x7f011c11ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.853+0000 7f0121612700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f011c1338a0 0x7f011c11ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:47522/0 (socket says 172.21.15.16:47522) 2023-09-30T22:33:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.853+0000 7f0121612700 1 -- 172.21.15.16:0/2917065675 learned_addr learned my addr 172.21.15.16:0/2917065675 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.853+0000 7f0121e13700 1 --2- 172.21.15.16:0/2917065675 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f011c139030 0x7f011c11b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.853+0000 7f0121612700 1 -- 172.21.15.16:0/2917065675 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f011c139030 msgr2=0x7f011c11b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.853+0000 7f0121612700 1 --2- 172.21.15.16:0/2917065675 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f011c139030 0x7f011c11b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.853+0000 7f0121612700 1 -- 172.21.15.16:0/2917065675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f011c136710 msgr2=0x7f011c11b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.853+0000 7f0120e11700 1 --2- 172.21.15.16:0/2917065675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f011c136710 0x7f011c11b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:42.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.853+0000 7f0121612700 1 --2- 172.21.15.16:0/2917065675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f011c136710 0x7f011c11b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:42.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.853+0000 7f0121612700 1 -- 172.21.15.16:0/2917065675 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f010c0094a0 con 0x7f011c1338a0 2023-09-30T22:33:42.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.853+0000 7f0120e11700 1 --2- 172.21.15.16:0/2917065675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f011c136710 0x7f011c11b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:33:42.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.853+0000 7f0121612700 1 --2- 172.21.15.16:0/2917065675 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f011c1338a0 0x7f011c11ac50 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f011800c520 tx=0x7f011800c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:42.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.853+0000 7f0111ffb700 1 -- 172.21.15.16:0/2917065675 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0118004c80 con 0x7f011c1338a0 2023-09-30T22:33:42.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.853+0000 7f0111ffb700 1 -- 172.21.15.16:0/2917065675 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0118004e20 con 0x7f011c1338a0 2023-09-30T22:33:42.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.854+0000 7f0123876700 1 -- 172.21.15.16:0/2917065675 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f011c11c860 con 0x7f011c1338a0 2023-09-30T22:33:42.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.854+0000 7f0111ffb700 1 -- 172.21.15.16:0/2917065675 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0118007310 con 0x7f011c1338a0 2023-09-30T22:33:42.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.854+0000 7f0123876700 1 -- 172.21.15.16:0/2917065675 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f011c11d050 con 0x7f011c1338a0 2023-09-30T22:33:42.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.855+0000 7f0121e13700 1 --2- 172.21.15.16:0/2917065675 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f011c139030 0x7f011c11b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:33:42.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.855+0000 7f0111ffb700 1 -- 172.21.15.16:0/2917065675 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0118007470 con 0x7f011c1338a0 2023-09-30T22:33:42.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.855+0000 7f0123876700 1 -- 172.21.15.16:0/2917065675 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f011c074ea0 con 0x7f011c1338a0 2023-09-30T22:33:42.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.855+0000 7f0111ffb700 1 -- 172.21.15.16:0/2917065675 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f011803f560 con 0x7f011c1338a0 2023-09-30T22:33:42.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.864+0000 7f0111ffb700 1 -- 172.21.15.16:0/2917065675 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f01180123b0 con 0x7f011c1338a0 2023-09-30T22:33:42.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.900+0000 7f860136f700 1 Processor -- start 2023-09-30T22:33:42.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.901+0000 7f860136f700 1 -- start start 2023-09-30T22:33:42.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.901+0000 7f860136f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85fc1337f0 0x7f85fc138880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:42.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.901+0000 7f860136f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85fc138dc0 0x7f85fc13b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:42.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.901+0000 7f860136f700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85fc06b990 con 0x7f85fc133420 2023-09-30T22:33:42.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.901+0000 7f860136f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85fc09b0e0 con 0x7f85fc138dc0 2023-09-30T22:33:42.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.901+0000 7f860136f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f85fc133c20 con 0x7f85fc1337f0 2023-09-30T22:33:42.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.902+0000 7f85faffd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f85fc133420 0x7f85fc133e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38142/0 (socket says 172.21.15.16:38142) 2023-09-30T22:33:42.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.902+0000 7f85faffd700 1 -- 172.21.15.16:0/1588164819 learned_addr learned my addr 172.21.15.16:0/1588164819 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:42.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.902+0000 7f85fb7fe700 1 --2- 172.21.15.16:0/1588164819 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85fc138dc0 0x7f85fc13b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:42.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.902+0000 7f85fa7fc700 1 --2- 172.21.15.16:0/1588164819 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85fc1337f0 0x7f85fc138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:42.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.902+0000 7f85f9ffb700 1 -- 172.21.15.16:0/1588164819 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 546128385 0 0) 0x7f85fc06b990 con 0x7f85fc133420 2023-09-30T22:33:42.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.902+0000 7f85f9ffb700 1 -- 172.21.15.16:0/1588164819 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85e40035b0 con 0x7f85fc133420 2023-09-30T22:33:42.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.902+0000 7f85fb7fe700 1 -- 172.21.15.16:0/1588164819 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85fc1337f0 msgr2=0x7f85fc138880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:42.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.902+0000 7f85fb7fe700 1 --2- 172.21.15.16:0/1588164819 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85fc1337f0 0x7f85fc138880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:42.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.902+0000 7f85fb7fe700 1 -- 172.21.15.16:0/1588164819 >> v1:172.21.15.16:6789/0 conn(0x7f85fc133420 legacy=0x7f85fc133e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:42.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.902+0000 7f85fb7fe700 1 -- 172.21.15.16:0/1588164819 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85fc0a0590 con 0x7f85fc138dc0 2023-09-30T22:33:42.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.902+0000 7f85fb7fe700 1 --2- 172.21.15.16:0/1588164819 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85fc138dc0 0x7f85fc13b1a0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f85f4009dc0 tx=0x7f85f400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bc0c8af8c6e2c20 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:42.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.903+0000 7f85f9ffb700 1 -- 172.21.15.16:0/1588164819 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f400e070 con 0x7f85fc138dc0 2023-09-30T22:33:42.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.903+0000 7f85f9ffb700 1 -- 172.21.15.16:0/1588164819 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f400bb20 con 0x7f85fc138dc0 2023-09-30T22:33:42.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.903+0000 7f85f9ffb700 1 -- 172.21.15.16:0/1588164819 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f4012430 con 0x7f85fc138dc0 2023-09-30T22:33:42.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.903+0000 7f860136f700 1 -- 172.21.15.16:0/1588164819 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85fc138dc0 msgr2=0x7f85fc13b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:42.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.903+0000 7f860136f700 1 --2- 172.21.15.16:0/1588164819 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85fc138dc0 0x7f85fc13b1a0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f85f4009dc0 tx=0x7f85f400b330 comp rx=0 tx=0).stop 2023-09-30T22:33:42.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.903+0000 7f860136f700 1 -- 172.21.15.16:0/1588164819 shutdown_connections 2023-09-30T22:33:42.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.903+0000 7f860136f700 1 --2- 172.21.15.16:0/1588164819 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85fc138dc0 0x7f85fc13b1a0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:42.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.904+0000 7f860136f700 1 --2- 172.21.15.16:0/1588164819 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85fc1337f0 0x7f85fc138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:42.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.904+0000 7f860136f700 1 -- 172.21.15.16:0/1588164819 >> 172.21.15.16:0/1588164819 conn(0x7f85fc0a4210 msgr2=0x7f85fc0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:42.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.904+0000 7f860136f700 1 -- 172.21.15.16:0/1588164819 shutdown_connections 2023-09-30T22:33:42.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.904+0000 7f860136f700 1 -- 172.21.15.16:0/1588164819 wait complete. 2023-09-30T22:33:42.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.905+0000 7f860136f700 1 Processor -- start 2023-09-30T22:33:42.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.905+0000 7f860136f700 1 -- start start 2023-09-30T22:33:42.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.906+0000 7f860136f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85fc133420 0x7f85fc12f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:42.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.906+0000 7f860136f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85fc1337f0 0x7f85fc12f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:42.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.906+0000 7f860136f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85fc138dc0 0x7f85fc12fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:42.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.906+0000 7f860136f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85fc130cb0 con 0x7f85fc1337f0 2023-09-30T22:33:42.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.906+0000 7f860136f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85fc1388a0 con 0x7f85fc133420 2023-09-30T22:33:42.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.906+0000 7f860136f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f85fc09c980 con 0x7f85fc138dc0 2023-09-30T22:33:42.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.906+0000 7f85fa7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85fc1337f0 0x7f85fc12f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:42.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.906+0000 7f85faffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85fc133420 0x7f85fc12f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:42.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.906+0000 7f85fa7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85fc1337f0 0x7f85fc12f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56152/0 (socket says 172.21.15.16:56152) 2023-09-30T22:33:42.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.906+0000 7f85fa7fc700 1 -- 172.21.15.16:0/2155442884 learned_addr learned my addr 172.21.15.16:0/2155442884 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:42.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.906+0000 7f85fb7fe700 1 --2- 172.21.15.16:0/2155442884 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85fc138dc0 0x7f85fc12fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:42.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.907+0000 7f85fa7fc700 1 -- 172.21.15.16:0/2155442884 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85fc138dc0 msgr2=0x7f85fc12fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:42.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.907+0000 7f85fa7fc700 1 --2- 172.21.15.16:0/2155442884 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85fc138dc0 0x7f85fc12fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:42.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.907+0000 7f85fa7fc700 1 -- 172.21.15.16:0/2155442884 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85fc133420 msgr2=0x7f85fc12f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:42.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.907+0000 7f85fa7fc700 1 --2- 172.21.15.16:0/2155442884 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85fc133420 0x7f85fc12f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:42.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.907+0000 7f85fa7fc700 1 -- 172.21.15.16:0/2155442884 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f4009a70 con 0x7f85fc1337f0 2023-09-30T22:33:42.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.907+0000 7f85fa7fc700 1 --2- 172.21.15.16:0/2155442884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85fc1337f0 0x7f85fc12f5f0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f85f000e9b0 tx=0x7f85f000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:42.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.907+0000 7f85e37fe700 1 -- 172.21.15.16:0/2155442884 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f0009710 con 0x7f85fc1337f0 2023-09-30T22:33:42.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.907+0000 7f85e37fe700 1 -- 172.21.15.16:0/2155442884 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f000ee50 con 0x7f85fc1337f0 2023-09-30T22:33:42.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.907+0000 7f860136f700 1 -- 172.21.15.16:0/2155442884 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85fc09e150 con 0x7f85fc1337f0 2023-09-30T22:33:42.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.907+0000 7f85e37fe700 1 -- 172.21.15.16:0/2155442884 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f000ca50 con 0x7f85fc1337f0 2023-09-30T22:33:42.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.907+0000 7f860136f700 1 -- 172.21.15.16:0/2155442884 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85fc09d690 con 0x7f85fc1337f0 2023-09-30T22:33:42.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.908+0000 7f85e37fe700 1 -- 172.21.15.16:0/2155442884 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f85f000cbb0 con 0x7f85fc1337f0 2023-09-30T22:33:42.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.909+0000 7f85e37fe700 1 -- 172.21.15.16:0/2155442884 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f85f003f740 con 0x7f85fc1337f0 2023-09-30T22:33:42.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.909+0000 7f860136f700 1 -- 172.21.15.16:0/2155442884 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85fc09c7c0 con 0x7f85fc1337f0 2023-09-30T22:33:42.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.918+0000 7f85e37fe700 1 -- 172.21.15.16:0/2155442884 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f85f00232f0 con 0x7f85fc1337f0 2023-09-30T22:33:42.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.924+0000 7f644a967700 1 Processor -- start 2023-09-30T22:33:42.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.924+0000 7f644a967700 1 -- start start 2023-09-30T22:33:42.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.925+0000 7f644a967700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6444133540 0x7f6444135920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:42.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.925+0000 7f644a967700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6444135e60 0x7f6444138240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:42.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.925+0000 7f644a967700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64440716a0 con 0x7f6444130540 2023-09-30T22:33:42.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.925+0000 7f644a967700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6444071290 con 0x7f6444135e60 2023-09-30T22:33:42.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.925+0000 7f644a967700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6444130340 con 0x7f6444133540 2023-09-30T22:33:42.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.925+0000 7f64437fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6444133540 0x7f6444135920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:42.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.925+0000 7f64437fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6444133540 0x7f6444135920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:47546/0 (socket says 172.21.15.16:47546) 2023-09-30T22:33:42.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.925+0000 7f64437fe700 1 -- 172.21.15.16:0/336768014 learned_addr learned my addr 172.21.15.16:0/336768014 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:42.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.925+0000 7f6448f04700 1 --2- 172.21.15.16:0/336768014 >> v2:172.21.15.16:3300/0 conn(0x7f6444135e60 0x7f6444138240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:42.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.925+0000 7f64437fe700 1 -- 172.21.15.16:0/336768014 >> v2:172.21.15.16:3300/0 conn(0x7f6444135e60 msgr2=0x7f6444138240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:42.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.925+0000 7f64437fe700 1 --2- 172.21.15.16:0/336768014 >> v2:172.21.15.16:3300/0 conn(0x7f6444135e60 0x7f6444138240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:42.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.925+0000 7f64437fe700 1 -- 172.21.15.16:0/336768014 >> v1:172.21.15.16:6789/0 conn(0x7f6444130540 legacy=0x7f6444130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:42.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.925+0000 7f64437fe700 1 -- 172.21.15.16:0/336768014 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6444070b80 con 0x7f6444133540 2023-09-30T22:33:42.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.926+0000 7f64437fe700 1 --2- 172.21.15.16:0/336768014 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6444133540 0x7f6444135920 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f64380098f0 tx=0x7f6438009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14d11d5a867822aa server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:42.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.926+0000 7f6442ffd700 1 -- 172.21.15.16:0/336768014 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6438011830 con 0x7f6444133540 2023-09-30T22:33:42.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.926+0000 7f6442ffd700 1 -- 172.21.15.16:0/336768014 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6438011990 con 0x7f6444133540 2023-09-30T22:33:42.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.926+0000 7f6442ffd700 1 -- 172.21.15.16:0/336768014 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6438011d20 con 0x7f6444133540 2023-09-30T22:33:42.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.926+0000 7f644a967700 1 -- 172.21.15.16:0/336768014 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6444133540 msgr2=0x7f6444135920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:42.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.926+0000 7f644a967700 1 --2- 172.21.15.16:0/336768014 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6444133540 0x7f6444135920 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f64380098f0 tx=0x7f6438009c00 comp rx=0 tx=0).stop 2023-09-30T22:33:42.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.927+0000 7f644a967700 1 -- 172.21.15.16:0/336768014 shutdown_connections 2023-09-30T22:33:42.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.927+0000 7f644a967700 1 --2- 172.21.15.16:0/336768014 >> v2:172.21.15.16:3300/0 conn(0x7f6444135e60 0x7f6444138240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:42.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.927+0000 7f644a967700 1 --2- 172.21.15.16:0/336768014 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6444133540 0x7f6444135920 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:42.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.927+0000 7f644a967700 1 -- 172.21.15.16:0/336768014 >> 172.21.15.16:0/336768014 conn(0x7f64440a2aa0 msgr2=0x7f64440a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:42.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.927+0000 7f644a967700 1 -- 172.21.15.16:0/336768014 shutdown_connections 2023-09-30T22:33:42.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.927+0000 7f644a967700 1 -- 172.21.15.16:0/336768014 wait complete. 2023-09-30T22:33:42.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.928+0000 7f644a967700 1 Processor -- start 2023-09-30T22:33:42.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.928+0000 7f644a967700 1 -- start start 2023-09-30T22:33:42.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.928+0000 7f644a967700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6444130540 0x7f644409a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:42.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.928+0000 7f644a967700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6444133540 0x7f644409aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:42.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.928+0000 7f644a967700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6444135e60 0x7f644409b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:42.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.928+0000 7f644a967700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64441392c0 con 0x7f6444135e60 2023-09-30T22:33:42.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.928+0000 7f644a967700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f644409c6e0 con 0x7f6444130540 2023-09-30T22:33:42.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.928+0000 7f644a967700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f64440718e0 con 0x7f6444133540 2023-09-30T22:33:42.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.928+0000 7f64437fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6444133540 0x7f644409aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:42.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.929+0000 7f6443fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6444130540 0x7f644409a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:42.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.929+0000 7f64437fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6444133540 0x7f644409aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:47550/0 (socket says 172.21.15.16:47550) 2023-09-30T22:33:42.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.929+0000 7f64437fe700 1 -- 172.21.15.16:0/2555117992 learned_addr learned my addr 172.21.15.16:0/2555117992 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:42.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.929+0000 7f6448f04700 1 --2- 172.21.15.16:0/2555117992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6444135e60 0x7f644409b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:42.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.929+0000 7f64437fe700 1 -- 172.21.15.16:0/2555117992 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6444130540 msgr2=0x7f644409a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:42.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.929+0000 7f64437fe700 1 --2- 172.21.15.16:0/2555117992 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6444130540 0x7f644409a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:42.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.929+0000 7f64437fe700 1 -- 172.21.15.16:0/2555117992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6444135e60 msgr2=0x7f644409b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:42.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.929+0000 7f64437fe700 1 --2- 172.21.15.16:0/2555117992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6444135e60 0x7f644409b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:42.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.929+0000 7f64437fe700 1 -- 172.21.15.16:0/2555117992 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6438009550 con 0x7f6444133540 2023-09-30T22:33:42.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.929+0000 7f6443fff700 1 --2- 172.21.15.16:0/2555117992 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6444130540 0x7f644409a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:42.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.929+0000 7f64437fe700 1 --2- 172.21.15.16:0/2555117992 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6444133540 0x7f644409aad0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f6438009c40 tx=0x7f6438004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:42.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.929+0000 7f6440ff9700 1 -- 172.21.15.16:0/2555117992 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6438004510 con 0x7f6444133540 2023-09-30T22:33:42.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.929+0000 7f6440ff9700 1 -- 172.21.15.16:0/2555117992 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64380036a0 con 0x7f6444133540 2023-09-30T22:33:42.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.929+0000 7f644a967700 1 -- 172.21.15.16:0/2555117992 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f644409c3a0 con 0x7f6444133540 2023-09-30T22:33:42.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.929+0000 7f6440ff9700 1 -- 172.21.15.16:0/2555117992 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6438003a50 con 0x7f6444133540 2023-09-30T22:33:42.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.930+0000 7f644a967700 1 -- 172.21.15.16:0/2555117992 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6444137e80 con 0x7f6444133540 2023-09-30T22:33:42.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.930+0000 7f6440ff9700 1 -- 172.21.15.16:0/2555117992 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6438021070 con 0x7f6444133540 2023-09-30T22:33:42.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.931+0000 7f644a967700 1 -- 172.21.15.16:0/2555117992 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6444070e10 con 0x7f6444133540 2023-09-30T22:33:42.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.931+0000 7f6440ff9700 1 -- 172.21.15.16:0/2555117992 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f6438040a80 con 0x7f6444133540 2023-09-30T22:33:42.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:42.936+0000 7f6440ff9700 1 -- 172.21.15.16:0/2555117992 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6438019770 con 0x7f6444133540 2023-09-30T22:33:43.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:43.032+0000 7f0c991da700 1 -- 172.21.15.16:0/3980283723 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f0c94072210 con 0x7f0c9406f680 2023-09-30T22:33:43.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:43.032+0000 7f0c7b7fe700 1 -- 172.21.15.16:0/3980283723 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0c84016020 con 0x7f0c9406f680 2023-09-30T22:33:43.033 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:33:43.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:43.036+0000 7f0c991da700 1 -- 172.21.15.16:0/3980283723 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0c9406f680 msgr2=0x7f0c9411b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:43.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:43.036+0000 7f0c991da700 1 --2- 172.21.15.16:0/3980283723 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0c9406f680 0x7f0c9411b8e0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f0c84000c00 tx=0x7f0c84004750 comp rx=0 tx=0).stop 2023-09-30T22:33:43.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:43.036+0000 7f0c991da700 1 -- 172.21.15.16:0/3980283723 shutdown_connections 2023-09-30T22:33:43.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:43.036+0000 7f0c991da700 1 --2- 172.21.15.16:0/3980283723 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0c9406f680 0x7f0c9411b8e0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:43.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:43.036+0000 7f0c991da700 1 --2- 172.21.15.16:0/3980283723 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c94130670 0x7f0c9411c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:43.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:43.036+0000 7f0c991da700 1 --2- 172.21.15.16:0/3980283723 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0c940701a0 0x7f0c9411be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:43.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:43.036+0000 7f0c991da700 1 -- 172.21.15.16:0/3980283723 >> 172.21.15.16:0/3980283723 conn(0x7f0c940a2aa0 msgr2=0x7f0c940a0620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:43.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:43.036+0000 7f0c991da700 1 -- 172.21.15.16:0/3980283723 shutdown_connections 2023-09-30T22:33:43.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:43.036+0000 7f0c991da700 1 -- 172.21.15.16:0/3980283723 wait complete. 2023-09-30T22:33:43.055 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:33:43.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:43.105+0000 7f0123876700 1 -- 172.21.15.16:0/2917065675 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f011c06b9e0 con 0x7f011c1338a0 2023-09-30T22:33:43.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:43.105+0000 7f0111ffb700 1 -- 172.21.15.16:0/2917065675 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0118011360 con 0x7f011c1338a0 2023-09-30T22:33:43.106 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:33:43.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:43.107+0000 7f0123876700 1 -- 172.21.15.16:0/2917065675 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f011c1338a0 msgr2=0x7f011c11ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:43.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:43.107+0000 7f0123876700 1 --2- 172.21.15.16:0/2917065675 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f011c1338a0 0x7f011c11ac50 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f011800c520 tx=0x7f011800c8e0 comp rx=0 tx=0).stop 2023-09-30T22:33:43.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:43.107+0000 7f0123876700 1 -- 172.21.15.16:0/2917065675 shutdown_connections 2023-09-30T22:33:43.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:43.107+0000 7f0123876700 1 --2- 172.21.15.16:0/2917065675 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f011c139030 0x7f011c11b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:43.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:43.107+0000 7f0123876700 1 --2- 172.21.15.16:0/2917065675 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f011c1338a0 0x7f011c11ac50 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:43.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:43.107+0000 7f0123876700 1 --2- 172.21.15.16:0/2917065675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f011c136710 0x7f011c11b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:43.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:43.107+0000 7f0123876700 1 -- 172.21.15.16:0/2917065675 >> 172.21.15.16:0/2917065675 conn(0x7f011c0a2aa0 msgr2=0x7f011c1344f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:43.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:43.108+0000 7f0123876700 1 -- 172.21.15.16:0/2917065675 shutdown_connections 2023-09-30T22:33:43.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:43.108+0000 7f0123876700 1 -- 172.21.15.16:0/2917065675 wait complete. 2023-09-30T22:33:43.122 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:33:43.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:43.141+0000 7f644a967700 1 -- 172.21.15.16:0/2555117992 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f6444074ea0 con 0x7f6444133540 2023-09-30T22:33:43.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:43.141+0000 7f6440ff9700 1 -- 172.21.15.16:0/2555117992 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6438029020 con 0x7f6444133540 2023-09-30T22:33:43.141 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:33:43.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:43.143+0000 7f644a967700 1 -- 172.21.15.16:0/2555117992 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6444133540 msgr2=0x7f644409aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:43.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:43.143+0000 7f644a967700 1 --2- 172.21.15.16:0/2555117992 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6444133540 0x7f644409aad0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f6438009c40 tx=0x7f6438004750 comp rx=0 tx=0).stop 2023-09-30T22:33:43.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:43.143+0000 7f644a967700 1 -- 172.21.15.16:0/2555117992 shutdown_connections 2023-09-30T22:33:43.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:43.143+0000 7f644a967700 1 --2- 172.21.15.16:0/2555117992 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6444130540 0x7f644409a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:43.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:43.143+0000 7f644a967700 1 --2- 172.21.15.16:0/2555117992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6444135e60 0x7f644409b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:43.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:43.143+0000 7f644a967700 1 --2- 172.21.15.16:0/2555117992 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6444133540 0x7f644409aad0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:43.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:43.143+0000 7f644a967700 1 -- 172.21.15.16:0/2555117992 >> 172.21.15.16:0/2555117992 conn(0x7f64440a2aa0 msgr2=0x7f644406fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:43.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:43.143+0000 7f644a967700 1 -- 172.21.15.16:0/2555117992 shutdown_connections 2023-09-30T22:33:43.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:43.143+0000 7f644a967700 1 -- 172.21.15.16:0/2555117992 wait complete. 2023-09-30T22:33:43.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:43.152+0000 7f860136f700 1 -- 172.21.15.16:0/2155442884 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f85fc074ea0 con 0x7f85fc1337f0 2023-09-30T22:33:43.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:43.153+0000 7f85e37fe700 1 -- 172.21.15.16:0/2155442884 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f85f001ae80 con 0x7f85fc1337f0 2023-09-30T22:33:43.153 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:33:43.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:43.155+0000 7f860136f700 1 -- 172.21.15.16:0/2155442884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85fc1337f0 msgr2=0x7f85fc12f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:43.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:43.155+0000 7f860136f700 1 --2- 172.21.15.16:0/2155442884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85fc1337f0 0x7f85fc12f5f0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f85f000e9b0 tx=0x7f85f000ecc0 comp rx=0 tx=0).stop 2023-09-30T22:33:43.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:43.155+0000 7f860136f700 1 -- 172.21.15.16:0/2155442884 shutdown_connections 2023-09-30T22:33:43.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:43.155+0000 7f860136f700 1 --2- 172.21.15.16:0/2155442884 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85fc133420 0x7f85fc12f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:43.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:43.155+0000 7f860136f700 1 --2- 172.21.15.16:0/2155442884 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85fc138dc0 0x7f85fc12fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:43.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:43.155+0000 7f860136f700 1 --2- 172.21.15.16:0/2155442884 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85fc1337f0 0x7f85fc12f5f0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:43.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:43.155+0000 7f860136f700 1 -- 172.21.15.16:0/2155442884 >> 172.21.15.16:0/2155442884 conn(0x7f85fc0a4210 msgr2=0x7f85fc0a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:43.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:43.155+0000 7f860136f700 1 -- 172.21.15.16:0/2155442884 shutdown_connections 2023-09-30T22:33:43.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:43.155+0000 7f860136f700 1 -- 172.21.15.16:0/2155442884 wait complete. 2023-09-30T22:33:43.158 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:33:43.170 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:33:44.056 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:33:44.123 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:33:44.159 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:33:44.171 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:33:44.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.204+0000 7fdd0576e700 1 Processor -- start 2023-09-30T22:33:44.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.204+0000 7fdd0576e700 1 -- start start 2023-09-30T22:33:44.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.205+0000 7fdd0576e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdd0009d6f0 0x7fdd0009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:44.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.205+0000 7fdd0576e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdd0009e000 0x7fdd00130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:44.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.205+0000 7fdd0576e700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd0009f130 con 0x7fdd0009e410 2023-09-30T22:33:44.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.205+0000 7fdd0576e700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd0009f520 con 0x7fdd0009d6f0 2023-09-30T22:33:44.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.205+0000 7fdd0576e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0009e930 con 0x7fdd0009e000 2023-09-30T22:33:44.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.205+0000 7fdcff7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fdd0009e410 0x7fdd00130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38148/0 (socket says 172.21.15.16:38148) 2023-09-30T22:33:44.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.205+0000 7fdcfeffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdd0009d6f0 0x7fdd0009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:44.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.205+0000 7fdcff7fe700 1 -- 172.21.15.16:0/2499638834 learned_addr learned my addr 172.21.15.16:0/2499638834 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:44.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.205+0000 7fdcfe7fc700 1 --2- 172.21.15.16:0/2499638834 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdd0009e000 0x7fdd00130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:44.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.205+0000 7fdcfdffb700 1 -- 172.21.15.16:0/2499638834 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1030971326 0 0) 0x7fdd0009f130 con 0x7fdd0009e410 2023-09-30T22:33:44.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.206+0000 7fdcfdffb700 1 -- 172.21.15.16:0/2499638834 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdce80035b0 con 0x7fdd0009e410 2023-09-30T22:33:44.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.206+0000 7fdcfeffd700 1 -- 172.21.15.16:0/2499638834 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdd0009e000 msgr2=0x7fdd00130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:44.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.206+0000 7fdcfeffd700 1 --2- 172.21.15.16:0/2499638834 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdd0009e000 0x7fdd00130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:44.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.206+0000 7fdcfeffd700 1 -- 172.21.15.16:0/2499638834 >> v1:172.21.15.16:6789/0 conn(0x7fdd0009e410 legacy=0x7fdd00130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:44.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.206+0000 7fdcfeffd700 1 -- 172.21.15.16:0/2499638834 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd0009b2d0 con 0x7fdd0009d6f0 2023-09-30T22:33:44.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.206+0000 7fdcfeffd700 1 --2- 172.21.15.16:0/2499638834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd0009d6f0 0x7fdd0009dac0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0009840 tx=0x7fdcf0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db3171c1a9849968 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:44.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.206+0000 7fdcfdffb700 1 -- 172.21.15.16:0/2499638834 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf0011830 con 0x7fdd0009d6f0 2023-09-30T22:33:44.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.206+0000 7fdcfdffb700 1 -- 172.21.15.16:0/2499638834 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcf0011990 con 0x7fdd0009d6f0 2023-09-30T22:33:44.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.206+0000 7fdcfdffb700 1 -- 172.21.15.16:0/2499638834 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf0011d20 con 0x7fdd0009d6f0 2023-09-30T22:33:44.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.207+0000 7fdd0576e700 1 -- 172.21.15.16:0/2499638834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd0009d6f0 msgr2=0x7fdd0009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:44.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.207+0000 7fdd0576e700 1 --2- 172.21.15.16:0/2499638834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd0009d6f0 0x7fdd0009dac0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0009840 tx=0x7fdcf0009b50 comp rx=0 tx=0).stop 2023-09-30T22:33:44.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.207+0000 7fdd0576e700 1 -- 172.21.15.16:0/2499638834 shutdown_connections 2023-09-30T22:33:44.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.207+0000 7fdd0576e700 1 --2- 172.21.15.16:0/2499638834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd0009d6f0 0x7fdd0009dac0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:44.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.207+0000 7fdd0576e700 1 --2- 172.21.15.16:0/2499638834 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdd0009e000 0x7fdd00130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:44.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.207+0000 7fdd0576e700 1 -- 172.21.15.16:0/2499638834 >> 172.21.15.16:0/2499638834 conn(0x7fdd00066f50 msgr2=0x7fdd0009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:44.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.207+0000 7fdd0576e700 1 -- 172.21.15.16:0/2499638834 shutdown_connections 2023-09-30T22:33:44.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.207+0000 7fdd0576e700 1 -- 172.21.15.16:0/2499638834 wait complete. 2023-09-30T22:33:44.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.209+0000 7fdd0576e700 1 Processor -- start 2023-09-30T22:33:44.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.209+0000 7fdd0576e700 1 -- start start 2023-09-30T22:33:44.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.209+0000 7fdd0576e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdd0009d6f0 0x7fdd0011ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:44.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.209+0000 7fdd0576e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd0009e000 0x7fdd0011e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:44.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.209+0000 7fdd0576e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdd0009e410 0x7fdd0011e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:44.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.209+0000 7fdd0576e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd001200e0 con 0x7fdd0009e000 2023-09-30T22:33:44.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.209+0000 7fdd0576e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd001219a0 con 0x7fdd0009d6f0 2023-09-30T22:33:44.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.209+0000 7fdd0576e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdd00120d70 con 0x7fdd0009e410 2023-09-30T22:33:44.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.210+0000 7fdcfeffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdd0009d6f0 0x7fdd0011ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:44.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.210+0000 7fdcfeffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdd0009d6f0 0x7fdd0011ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:51928/0 (socket says 172.21.15.16:51928) 2023-09-30T22:33:44.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.210+0000 7fdcfeffd700 1 -- 172.21.15.16:0/433163227 learned_addr learned my addr 172.21.15.16:0/433163227 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:44.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.210+0000 7fdcff7fe700 1 --2- 172.21.15.16:0/433163227 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdd0009e410 0x7fdd0011e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:44.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.210+0000 7fdcfeffd700 1 -- 172.21.15.16:0/433163227 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdd0009e410 msgr2=0x7fdd0011e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:44.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.210+0000 7fdcfeffd700 1 --2- 172.21.15.16:0/433163227 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdd0009e410 0x7fdd0011e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:44.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.210+0000 7fdcfeffd700 1 -- 172.21.15.16:0/433163227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd0009e000 msgr2=0x7fdd0011e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:44.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.210+0000 7fdcfeffd700 1 --2- 172.21.15.16:0/433163227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd0009e000 0x7fdd0011e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:44.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.210+0000 7fdcfeffd700 1 -- 172.21.15.16:0/433163227 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf00094a0 con 0x7fdd0009d6f0 2023-09-30T22:33:44.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.210+0000 7fdcfeffd700 1 --2- 172.21.15.16:0/433163227 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdd0009d6f0 0x7fdd0011ded0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fdcf00090b0 tx=0x7fdcf0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:44.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.211+0000 7fdce77fe700 1 -- 172.21.15.16:0/433163227 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf00036a0 con 0x7fdd0009d6f0 2023-09-30T22:33:44.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.211+0000 7fdce77fe700 1 -- 172.21.15.16:0/433163227 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcf001f070 con 0x7fdd0009d6f0 2023-09-30T22:33:44.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.211+0000 7fdce77fe700 1 -- 172.21.15.16:0/433163227 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf0003a40 con 0x7fdd0009d6f0 2023-09-30T22:33:44.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.211+0000 7fdd0576e700 1 -- 172.21.15.16:0/433163227 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd0011f330 con 0x7fdd0009d6f0 2023-09-30T22:33:44.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.211+0000 7fdd0576e700 1 -- 172.21.15.16:0/433163227 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd0011fa90 con 0x7fdd0009d6f0 2023-09-30T22:33:44.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.212+0000 7fdce77fe700 1 -- 172.21.15.16:0/433163227 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdcf0003ba0 con 0x7fdd0009d6f0 2023-09-30T22:33:44.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.212+0000 7fdce77fe700 1 -- 172.21.15.16:0/433163227 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fdcf0021070 con 0x7fdd0009d6f0 2023-09-30T22:33:44.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.212+0000 7fdd0576e700 1 -- 172.21.15.16:0/433163227 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd0011ac20 con 0x7fdd0009d6f0 2023-09-30T22:33:44.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.222+0000 7fdce77fe700 1 -- 172.21.15.16:0/433163227 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdcf0015020 con 0x7fdd0009d6f0 2023-09-30T22:33:44.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.272+0000 7fea1c3b5700 1 Processor -- start 2023-09-30T22:33:44.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.272+0000 7fea1c3b5700 1 -- start start 2023-09-30T22:33:44.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.273+0000 7fea1c3b5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea1409d6f0 0x7fea1409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:44.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.273+0000 7fea1c3b5700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fea1409e000 0x7fea141339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:44.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.273+0000 7fea1c3b5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea1409e410 0x7fea14133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:44.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.273+0000 7fea1c3b5700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea1406b460 con 0x7fea1409e000 2023-09-30T22:33:44.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.273+0000 7fea1c3b5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea1409b120 con 0x7fea1409e410 2023-09-30T22:33:44.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.273+0000 7fea1c3b5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fea1409a7e0 con 0x7fea1409d6f0 2023-09-30T22:33:44.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.273+0000 7fea19950700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fea1409e000 0x7fea141339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:44.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.273+0000 7fea1a151700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea1409d6f0 0x7fea1409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:44.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.273+0000 7fea19950700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fea1409e000 0x7fea141339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56204/0 (socket says 172.21.15.16:56204) 2023-09-30T22:33:44.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.273+0000 7fea19950700 1 -- 172.21.15.16:0/693409119 learned_addr learned my addr 172.21.15.16:0/693409119 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:44.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.273+0000 7fea1a952700 1 --2- 172.21.15.16:0/693409119 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea1409e410 0x7fea14133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:44.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.274+0000 7fea19950700 1 -- 172.21.15.16:0/693409119 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea1409d6f0 msgr2=0x7fea1409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:44.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.274+0000 7fea19950700 1 --2- 172.21.15.16:0/693409119 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea1409d6f0 0x7fea1409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:44.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.274+0000 7fea19950700 1 -- 172.21.15.16:0/693409119 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea1409e410 msgr2=0x7fea14133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:44.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.274+0000 7fea19950700 1 --2- 172.21.15.16:0/693409119 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea1409e410 0x7fea14133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:44.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.274+0000 7fea19950700 1 -- 172.21.15.16:0/693409119 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea1409a500 con 0x7fea1409e000 2023-09-30T22:33:44.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.274+0000 7fea19950700 1 --2- 172.21.15.16:0/693409119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea1409e000 0x7fea141339f0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fea040098f0 tx=0x7fea04009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf75b235323a079c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:44.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.274+0000 7fea1914f700 1 -- 172.21.15.16:0/693409119 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea04011830 con 0x7fea1409e000 2023-09-30T22:33:44.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.274+0000 7fea1914f700 1 -- 172.21.15.16:0/693409119 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea04011990 con 0x7fea1409e000 2023-09-30T22:33:44.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.274+0000 7fea1914f700 1 -- 172.21.15.16:0/693409119 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea04011d20 con 0x7fea1409e000 2023-09-30T22:33:44.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.275+0000 7fea1c3b5700 1 -- 172.21.15.16:0/693409119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea1409e000 msgr2=0x7fea141339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:44.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.275+0000 7fea1c3b5700 1 --2- 172.21.15.16:0/693409119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea1409e000 0x7fea141339f0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fea040098f0 tx=0x7fea04009c00 comp rx=0 tx=0).stop 2023-09-30T22:33:44.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.275+0000 7fea1c3b5700 1 -- 172.21.15.16:0/693409119 shutdown_connections 2023-09-30T22:33:44.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.275+0000 7fea1c3b5700 1 --2- 172.21.15.16:0/693409119 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea1409e410 0x7fea14133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:44.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.275+0000 7fea1c3b5700 1 --2- 172.21.15.16:0/693409119 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea1409d6f0 0x7fea1409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:44.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.275+0000 7fea1c3b5700 1 --2- 172.21.15.16:0/693409119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea1409e000 0x7fea141339f0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:44.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.275+0000 7fea1c3b5700 1 -- 172.21.15.16:0/693409119 >> 172.21.15.16:0/693409119 conn(0x7fea14066f50 msgr2=0x7fea1409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:44.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.275+0000 7fea1c3b5700 1 -- 172.21.15.16:0/693409119 shutdown_connections 2023-09-30T22:33:44.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.275+0000 7fea1c3b5700 1 -- 172.21.15.16:0/693409119 wait complete. 2023-09-30T22:33:44.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.276+0000 7fea1c3b5700 1 Processor -- start 2023-09-30T22:33:44.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.276+0000 7fea1c3b5700 1 -- start start 2023-09-30T22:33:44.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.277+0000 7fea1c3b5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea1409d6f0 0x7fea141c8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:44.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.277+0000 7fea1c3b5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea1409e000 0x7fea141c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:44.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.277+0000 7fea1c3b5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea1409e410 0x7fea141c4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:44.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.277+0000 7fea1c3b5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea141a86d0 con 0x7fea1409e000 2023-09-30T22:33:44.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.277+0000 7fea1c3b5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea141a8b90 con 0x7fea1409d6f0 2023-09-30T22:33:44.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.277+0000 7fea1c3b5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fea141c8f80 con 0x7fea1409e410 2023-09-30T22:33:44.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.277+0000 7fea1a952700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea1409e410 0x7fea141c4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:44.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.277+0000 7fea1a151700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea1409d6f0 0x7fea141c8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:44.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.277+0000 7fea1a952700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea1409e410 0x7fea141c4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:47576/0 (socket says 172.21.15.16:47576) 2023-09-30T22:33:44.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.277+0000 7fea1a952700 1 -- 172.21.15.16:0/4240252070 learned_addr learned my addr 172.21.15.16:0/4240252070 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:44.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.277+0000 7fea19950700 1 --2- 172.21.15.16:0/4240252070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea1409e000 0x7fea141c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:44.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.277+0000 7fea1a952700 1 -- 172.21.15.16:0/4240252070 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea1409d6f0 msgr2=0x7fea141c8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:44.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.277+0000 7fea1a952700 1 --2- 172.21.15.16:0/4240252070 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea1409d6f0 0x7fea141c8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:44.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.277+0000 7fea1a952700 1 -- 172.21.15.16:0/4240252070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea1409e000 msgr2=0x7fea141c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:44.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.277+0000 7fea1a952700 1 --2- 172.21.15.16:0/4240252070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea1409e000 0x7fea141c8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:44.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.277+0000 7fea1a952700 1 -- 172.21.15.16:0/4240252070 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea04009550 con 0x7fea1409e410 2023-09-30T22:33:44.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.277+0000 7fea19950700 1 --2- 172.21.15.16:0/4240252070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea1409e000 0x7fea141c8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:33:44.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.278+0000 7fea1a952700 1 --2- 172.21.15.16:0/4240252070 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea1409e410 0x7fea141c4ad0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fea0c009dd0 tx=0x7fea0c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:44.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.278+0000 7fea0affd700 1 -- 172.21.15.16:0/4240252070 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0c009c40 con 0x7fea1409e410 2023-09-30T22:33:44.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.278+0000 7fea0affd700 1 -- 172.21.15.16:0/4240252070 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea0c00f040 con 0x7fea1409e410 2023-09-30T22:33:44.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.278+0000 7fea1c3b5700 1 -- 172.21.15.16:0/4240252070 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea141a78f0 con 0x7fea1409e410 2023-09-30T22:33:44.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.278+0000 7fea0affd700 1 -- 172.21.15.16:0/4240252070 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0c014480 con 0x7fea1409e410 2023-09-30T22:33:44.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.278+0000 7fea1c3b5700 1 -- 172.21.15.16:0/4240252070 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea1409c710 con 0x7fea1409e410 2023-09-30T22:33:44.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.279+0000 7fea1c3b5700 1 -- 172.21.15.16:0/4240252070 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea14074ea0 con 0x7fea1409e410 2023-09-30T22:33:44.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.279+0000 7fea0affd700 1 -- 172.21.15.16:0/4240252070 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fea0c0145e0 con 0x7fea1409e410 2023-09-30T22:33:44.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.279+0000 7fea0affd700 1 -- 172.21.15.16:0/4240252070 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fea0c040540 con 0x7fea1409e410 2023-09-30T22:33:44.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.289+0000 7fea0affd700 1 -- 172.21.15.16:0/4240252070 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fea0c012930 con 0x7fea1409e410 2023-09-30T22:33:44.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.314+0000 7f0340cf0700 1 Processor -- start 2023-09-30T22:33:44.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.314+0000 7f0340cf0700 1 -- start start 2023-09-30T22:33:44.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.314+0000 7f0340cf0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f033c133a80 0x7f033c133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:44.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.314+0000 7f0340cf0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f033c12ff20 0x7f033c130330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:44.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.314+0000 7f0340cf0700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f033c130870 0x7f033c13a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:44.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.314+0000 7f0340cf0700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f033c06c570 con 0x7f033c130870 2023-09-30T22:33:44.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.314+0000 7f0340cf0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f033c06bc00 con 0x7f033c133a80 2023-09-30T22:33:44.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.314+0000 7f0340cf0700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f033c0a0040 con 0x7f033c12ff20 2023-09-30T22:33:44.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.315+0000 7f033a59c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f033c133a80 0x7f033c133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:44.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.315+0000 7f0339d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f033c12ff20 0x7f033c130330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:44.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.315+0000 7f0339d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f033c12ff20 0x7f033c130330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:47592/0 (socket says 172.21.15.16:47592) 2023-09-30T22:33:44.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.315+0000 7f033a59c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f033c133a80 0x7f033c133e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:51954/0 (socket says 172.21.15.16:51954) 2023-09-30T22:33:44.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.315+0000 7f0339d9b700 1 -- 172.21.15.16:0/57513356 learned_addr learned my addr 172.21.15.16:0/57513356 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:44.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.315+0000 7f033ad9d700 1 --2- 172.21.15.16:0/57513356 >> v2:172.21.15.16:3300/0 conn(0x7f033c130870 0x7f033c13a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:44.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.315+0000 7f0339d9b700 1 -- 172.21.15.16:0/57513356 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f033c133a80 msgr2=0x7f033c133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:44.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.315+0000 7f0339d9b700 1 --2- 172.21.15.16:0/57513356 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f033c133a80 0x7f033c133e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:44.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.315+0000 7f0339d9b700 1 -- 172.21.15.16:0/57513356 >> v2:172.21.15.16:3300/0 conn(0x7f033c130870 msgr2=0x7f033c13a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:44.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.315+0000 7f0339d9b700 1 --2- 172.21.15.16:0/57513356 >> v2:172.21.15.16:3300/0 conn(0x7f033c130870 0x7f033c13a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:44.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.315+0000 7f0339d9b700 1 -- 172.21.15.16:0/57513356 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f033c133860 con 0x7f033c12ff20 2023-09-30T22:33:44.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.315+0000 7f033ad9d700 1 --2- 172.21.15.16:0/57513356 >> v2:172.21.15.16:3300/0 conn(0x7f033c130870 0x7f033c13a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:33:44.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.315+0000 7f033a59c700 1 --2- 172.21.15.16:0/57513356 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f033c133a80 0x7f033c133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:44.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.316+0000 7f0339d9b700 1 --2- 172.21.15.16:0/57513356 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f033c12ff20 0x7f033c130330 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f03300098f0 tx=0x7f0330009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79f83cd6c8e56081 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:44.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.316+0000 7f033959a700 1 -- 172.21.15.16:0/57513356 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0330011830 con 0x7f033c12ff20 2023-09-30T22:33:44.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.316+0000 7f033959a700 1 -- 172.21.15.16:0/57513356 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0330011990 con 0x7f033c12ff20 2023-09-30T22:33:44.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.316+0000 7f033959a700 1 -- 172.21.15.16:0/57513356 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0330011d20 con 0x7f033c12ff20 2023-09-30T22:33:44.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.317+0000 7f0340cf0700 1 -- 172.21.15.16:0/57513356 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f033c12ff20 msgr2=0x7f033c130330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:44.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.317+0000 7f0340cf0700 1 --2- 172.21.15.16:0/57513356 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f033c12ff20 0x7f033c130330 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f03300098f0 tx=0x7f0330009c00 comp rx=0 tx=0).stop 2023-09-30T22:33:44.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.317+0000 7f0340cf0700 1 -- 172.21.15.16:0/57513356 shutdown_connections 2023-09-30T22:33:44.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.317+0000 7f0340cf0700 1 --2- 172.21.15.16:0/57513356 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f033c133a80 0x7f033c133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:44.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.317+0000 7f0340cf0700 1 --2- 172.21.15.16:0/57513356 >> v2:172.21.15.16:3300/0 conn(0x7f033c130870 0x7f033c13a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:44.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.317+0000 7f0340cf0700 1 --2- 172.21.15.16:0/57513356 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f033c12ff20 0x7f033c130330 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:44.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.317+0000 7f0340cf0700 1 -- 172.21.15.16:0/57513356 >> 172.21.15.16:0/57513356 conn(0x7f033c0a4210 msgr2=0x7f033c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:44.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.317+0000 7f0340cf0700 1 -- 172.21.15.16:0/57513356 shutdown_connections 2023-09-30T22:33:44.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.317+0000 7f0340cf0700 1 -- 172.21.15.16:0/57513356 wait complete. 2023-09-30T22:33:44.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.319+0000 7f0340cf0700 1 Processor -- start 2023-09-30T22:33:44.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.319+0000 7f0340cf0700 1 -- start start 2023-09-30T22:33:44.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.319+0000 7f0340cf0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f033c12ff20 0x7f033c09c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:44.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.319+0000 7f0340cf0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f033c130870 0x7f033c09c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:44.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.319+0000 7f0340cf0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f033c133a80 0x7f033c09cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:44.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.319+0000 7f0340cf0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033c09d8e0 con 0x7f033c133a80 2023-09-30T22:33:44.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.319+0000 7f0340cf0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f033c12da40 con 0x7f033c12ff20 2023-09-30T22:33:44.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.319+0000 7f0340cf0700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f033c0a2110 con 0x7f033c130870 2023-09-30T22:33:44.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.320+0000 7f033a59c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f033c12ff20 0x7f033c09c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:44.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.320+0000 7f0339d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f033c130870 0x7f033c09c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:44.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.320+0000 7f0339d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f033c130870 0x7f033c09c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:47606/0 (socket says 172.21.15.16:47606) 2023-09-30T22:33:44.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.320+0000 7f033a59c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f033c12ff20 0x7f033c09c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:51968/0 (socket says 172.21.15.16:51968) 2023-09-30T22:33:44.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.320+0000 7f0339d9b700 1 -- 172.21.15.16:0/194239368 learned_addr learned my addr 172.21.15.16:0/194239368 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:44.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.320+0000 7f033ad9d700 1 --2- 172.21.15.16:0/194239368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f033c133a80 0x7f033c09cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:44.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.320+0000 7f0339d9b700 1 -- 172.21.15.16:0/194239368 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f033c12ff20 msgr2=0x7f033c09c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:44.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.320+0000 7f0339d9b700 1 --2- 172.21.15.16:0/194239368 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f033c12ff20 0x7f033c09c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:44.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.320+0000 7f0339d9b700 1 -- 172.21.15.16:0/194239368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f033c133a80 msgr2=0x7f033c09cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:44.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.320+0000 7f0339d9b700 1 --2- 172.21.15.16:0/194239368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f033c133a80 0x7f033c09cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:44.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.320+0000 7f0339d9b700 1 -- 172.21.15.16:0/194239368 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f032c0094a0 con 0x7f033c130870 2023-09-30T22:33:44.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.320+0000 7f0339d9b700 1 --2- 172.21.15.16:0/194239368 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f033c130870 0x7f033c09c9c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f0330000c00 tx=0x7f0330004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:44.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.320+0000 7f032affd700 1 -- 172.21.15.16:0/194239368 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0330004300 con 0x7f033c130870 2023-09-30T22:33:44.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.321+0000 7f032affd700 1 -- 172.21.15.16:0/194239368 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f033001f070 con 0x7f033c130870 2023-09-30T22:33:44.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.321+0000 7f032affd700 1 -- 172.21.15.16:0/194239368 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03300036a0 con 0x7f033c130870 2023-09-30T22:33:44.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.321+0000 7f0340cf0700 1 -- 172.21.15.16:0/194239368 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0330009550 con 0x7f033c130870 2023-09-30T22:33:44.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.321+0000 7f0340cf0700 1 -- 172.21.15.16:0/194239368 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f033c11ff20 con 0x7f033c130870 2023-09-30T22:33:44.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.322+0000 7f0340cf0700 1 -- 172.21.15.16:0/194239368 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f033c074ea0 con 0x7f033c130870 2023-09-30T22:33:44.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.322+0000 7f032affd700 1 -- 172.21.15.16:0/194239368 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0330021030 con 0x7f033c130870 2023-09-30T22:33:44.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.322+0000 7f032affd700 1 -- 172.21.15.16:0/194239368 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f0330040710 con 0x7f033c130870 2023-09-30T22:33:44.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.331+0000 7f032affd700 1 -- 172.21.15.16:0/194239368 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f03300239c0 con 0x7f033c130870 2023-09-30T22:33:44.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.356+0000 7fc14271a700 1 Processor -- start 2023-09-30T22:33:44.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.356+0000 7fc14271a700 1 -- start start 2023-09-30T22:33:44.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.357+0000 7fc14271a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc13c09e930 0x7fc13c09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:44.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.357+0000 7fc14271a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc13c09f240 0x7fc13c1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:44.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.357+0000 7fc14271a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc13c070940 con 0x7fc13c09f650 2023-09-30T22:33:44.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.357+0000 7fc14271a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc13c06fd60 con 0x7fc13c09f240 2023-09-30T22:33:44.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.357+0000 7fc14271a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc13c06fa80 con 0x7fc13c09e930 2023-09-30T22:33:44.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.357+0000 7fc13b7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc13c09f240 0x7fc13c1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:44.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.357+0000 7fc13bfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc13c09e930 0x7fc13c09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:44.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.357+0000 7fc13b7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc13c09f240 0x7fc13c1317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:51974/0 (socket says 172.21.15.16:51974) 2023-09-30T22:33:44.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.357+0000 7fc13b7fe700 1 -- 172.21.15.16:0/2084398824 learned_addr learned my addr 172.21.15.16:0/2084398824 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:44.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.357+0000 7fc13bfff700 1 -- 172.21.15.16:0/2084398824 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc13c09f240 msgr2=0x7fc13c1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:44.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.357+0000 7fc13bfff700 1 --2- 172.21.15.16:0/2084398824 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc13c09f240 0x7fc13c1317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:44.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.357+0000 7fc13bfff700 1 -- 172.21.15.16:0/2084398824 >> v1:172.21.15.16:6789/0 conn(0x7fc13c09f650 legacy=0x7fc13c131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:44.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.357+0000 7fc13bfff700 1 -- 172.21.15.16:0/2084398824 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc13c09fb60 con 0x7fc13c09e930 2023-09-30T22:33:44.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.357+0000 7fc13b7fe700 1 --2- 172.21.15.16:0/2084398824 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc13c09f240 0x7fc13c1317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:33:44.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.358+0000 7fc13bfff700 1 --2- 172.21.15.16:0/2084398824 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc13c09e930 0x7fc13c09ed00 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fc12c009870 tx=0x7fc12c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bbe088b234eed78 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:44.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.358+0000 7fc13affd700 1 -- 172.21.15.16:0/2084398824 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc12c011830 con 0x7fc13c09e930 2023-09-30T22:33:44.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.358+0000 7fc13affd700 1 -- 172.21.15.16:0/2084398824 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc12c011990 con 0x7fc13c09e930 2023-09-30T22:33:44.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.358+0000 7fc13affd700 1 -- 172.21.15.16:0/2084398824 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc12c011d20 con 0x7fc13c09e930 2023-09-30T22:33:44.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.358+0000 7fc14271a700 1 -- 172.21.15.16:0/2084398824 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc13c09e930 msgr2=0x7fc13c09ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:44.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.358+0000 7fc14271a700 1 --2- 172.21.15.16:0/2084398824 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc13c09e930 0x7fc13c09ed00 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fc12c009870 tx=0x7fc12c009b80 comp rx=0 tx=0).stop 2023-09-30T22:33:44.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.359+0000 7fc14271a700 1 -- 172.21.15.16:0/2084398824 shutdown_connections 2023-09-30T22:33:44.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.359+0000 7fc14271a700 1 --2- 172.21.15.16:0/2084398824 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc13c09e930 0x7fc13c09ed00 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:44.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.359+0000 7fc14271a700 1 --2- 172.21.15.16:0/2084398824 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc13c09f240 0x7fc13c1317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:44.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.359+0000 7fc14271a700 1 -- 172.21.15.16:0/2084398824 >> 172.21.15.16:0/2084398824 conn(0x7fc13c0a4210 msgr2=0x7fc13c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:44.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.359+0000 7fc14271a700 1 -- 172.21.15.16:0/2084398824 shutdown_connections 2023-09-30T22:33:44.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.359+0000 7fc14271a700 1 -- 172.21.15.16:0/2084398824 wait complete. 2023-09-30T22:33:44.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.360+0000 7fc14271a700 1 Processor -- start 2023-09-30T22:33:44.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.360+0000 7fc14271a700 1 -- start start 2023-09-30T22:33:44.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.360+0000 7fc14271a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc13c09e930 0x7fc13c06b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:44.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.360+0000 7fc14271a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc13c09f240 0x7fc13c06bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:44.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.360+0000 7fc14271a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc13c09f650 0x7fc13c06c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:44.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.360+0000 7fc14271a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc13c1a9a70 con 0x7fc13c09e930 2023-09-30T22:33:44.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.360+0000 7fc14271a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc13c1a8a10 con 0x7fc13c09f650 2023-09-30T22:33:44.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.360+0000 7fc14271a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc13c0a3700 con 0x7fc13c09f240 2023-09-30T22:33:44.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.361+0000 7fc140cb7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc13c09f650 0x7fc13c06c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:44.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.361+0000 7fc140cb7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc13c09f650 0x7fc13c06c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:51984/0 (socket says 172.21.15.16:51984) 2023-09-30T22:33:44.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.361+0000 7fc140cb7700 1 -- 172.21.15.16:0/983196243 learned_addr learned my addr 172.21.15.16:0/983196243 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:44.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.361+0000 7fc13b7fe700 1 --2- 172.21.15.16:0/983196243 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc13c09f240 0x7fc13c06bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:44.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.361+0000 7fc13bfff700 1 --2- 172.21.15.16:0/983196243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc13c09e930 0x7fc13c06b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:44.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.361+0000 7fc140cb7700 1 -- 172.21.15.16:0/983196243 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc13c09f240 msgr2=0x7fc13c06bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:44.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.361+0000 7fc140cb7700 1 --2- 172.21.15.16:0/983196243 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc13c09f240 0x7fc13c06bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:44.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.361+0000 7fc140cb7700 1 -- 172.21.15.16:0/983196243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc13c09e930 msgr2=0x7fc13c06b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:44.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.361+0000 7fc140cb7700 1 --2- 172.21.15.16:0/983196243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc13c09e930 0x7fc13c06b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:44.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.361+0000 7fc140cb7700 1 -- 172.21.15.16:0/983196243 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc12c0094a0 con 0x7fc13c09f650 2023-09-30T22:33:44.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.361+0000 7fc13b7fe700 1 --2- 172.21.15.16:0/983196243 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc13c09f240 0x7fc13c06bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:44.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.361+0000 7fc13bfff700 1 --2- 172.21.15.16:0/983196243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc13c09e930 0x7fc13c06b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:33:44.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.361+0000 7fc140cb7700 1 --2- 172.21.15.16:0/983196243 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc13c09f650 0x7fc13c06c330 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fc13400cbc0 tx=0x7fc13400ced0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:44.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.362+0000 7fc138ff9700 1 -- 172.21.15.16:0/983196243 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc134003c00 con 0x7fc13c09f650 2023-09-30T22:33:44.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.362+0000 7fc138ff9700 1 -- 172.21.15.16:0/983196243 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc134003d60 con 0x7fc13c09f650 2023-09-30T22:33:44.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.362+0000 7fc14271a700 1 -- 172.21.15.16:0/983196243 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc13c0a0190 con 0x7fc13c09f650 2023-09-30T22:33:44.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.362+0000 7fc138ff9700 1 -- 172.21.15.16:0/983196243 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13400ab40 con 0x7fc13c09f650 2023-09-30T22:33:44.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.362+0000 7fc14271a700 1 -- 172.21.15.16:0/983196243 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc13c071d20 con 0x7fc13c09f650 2023-09-30T22:33:44.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.363+0000 7fc138ff9700 1 -- 172.21.15.16:0/983196243 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc13400aca0 con 0x7fc13c09f650 2023-09-30T22:33:44.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.363+0000 7fc14271a700 1 -- 172.21.15.16:0/983196243 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc13c1a0b30 con 0x7fc13c09f650 2023-09-30T22:33:44.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.363+0000 7fc138ff9700 1 -- 172.21.15.16:0/983196243 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fc13403f720 con 0x7fc13c09f650 2023-09-30T22:33:44.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.372+0000 7fc138ff9700 1 -- 172.21.15.16:0/983196243 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc1340196c0 con 0x7fc13c09f650 2023-09-30T22:33:44.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.442+0000 7fdd0576e700 1 -- 172.21.15.16:0/433163227 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fdd00074ea0 con 0x7fdd0009d6f0 2023-09-30T22:33:44.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.442+0000 7fdce77fe700 1 -- 172.21.15.16:0/433163227 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdcf002b070 con 0x7fdd0009d6f0 2023-09-30T22:33:44.443 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:33:44.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.445+0000 7fdd0576e700 1 -- 172.21.15.16:0/433163227 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdd0009d6f0 msgr2=0x7fdd0011ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:44.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.445+0000 7fdd0576e700 1 --2- 172.21.15.16:0/433163227 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdd0009d6f0 0x7fdd0011ded0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fdcf00090b0 tx=0x7fdcf0004750 comp rx=0 tx=0).stop 2023-09-30T22:33:44.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.445+0000 7fdd0576e700 1 -- 172.21.15.16:0/433163227 shutdown_connections 2023-09-30T22:33:44.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.445+0000 7fdd0576e700 1 --2- 172.21.15.16:0/433163227 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdd0009d6f0 0x7fdd0011ded0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:44.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.445+0000 7fdd0576e700 1 --2- 172.21.15.16:0/433163227 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdd0009e410 0x7fdd0011e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:44.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.445+0000 7fdd0576e700 1 --2- 172.21.15.16:0/433163227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd0009e000 0x7fdd0011e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:44.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.445+0000 7fdd0576e700 1 -- 172.21.15.16:0/433163227 >> 172.21.15.16:0/433163227 conn(0x7fdd00066f50 msgr2=0x7fdd00130a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:44.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.445+0000 7fdd0576e700 1 -- 172.21.15.16:0/433163227 shutdown_connections 2023-09-30T22:33:44.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.445+0000 7fdd0576e700 1 -- 172.21.15.16:0/433163227 wait complete. 2023-09-30T22:33:44.459 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:33:44.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.506+0000 7fea1c3b5700 1 -- 172.21.15.16:0/4240252070 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fea1406c360 con 0x7fea1409e410 2023-09-30T22:33:44.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.506+0000 7fea0affd700 1 -- 172.21.15.16:0/4240252070 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fea0c012340 con 0x7fea1409e410 2023-09-30T22:33:44.507 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:33:44.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.508+0000 7fea1c3b5700 1 -- 172.21.15.16:0/4240252070 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea1409e410 msgr2=0x7fea141c4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:44.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.508+0000 7fea1c3b5700 1 --2- 172.21.15.16:0/4240252070 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea1409e410 0x7fea141c4ad0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fea0c009dd0 tx=0x7fea0c00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:33:44.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.508+0000 7fea1c3b5700 1 -- 172.21.15.16:0/4240252070 shutdown_connections 2023-09-30T22:33:44.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.508+0000 7fea1c3b5700 1 --2- 172.21.15.16:0/4240252070 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea1409d6f0 0x7fea141c8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:44.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.508+0000 7fea1c3b5700 1 --2- 172.21.15.16:0/4240252070 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea1409e410 0x7fea141c4ad0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:44.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.508+0000 7fea1c3b5700 1 --2- 172.21.15.16:0/4240252070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea1409e000 0x7fea141c8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:44.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.508+0000 7fea1c3b5700 1 -- 172.21.15.16:0/4240252070 >> 172.21.15.16:0/4240252070 conn(0x7fea14066f50 msgr2=0x7fea141badf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:44.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.508+0000 7fea1c3b5700 1 -- 172.21.15.16:0/4240252070 shutdown_connections 2023-09-30T22:33:44.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.508+0000 7fea1c3b5700 1 -- 172.21.15.16:0/4240252070 wait complete. 2023-09-30T22:33:44.523 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:33:44.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.541+0000 7f0340cf0700 1 -- 172.21.15.16:0/194239368 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f033c06b250 con 0x7f033c130870 2023-09-30T22:33:44.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.541+0000 7f032affd700 1 -- 172.21.15.16:0/194239368 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0330015030 con 0x7f033c130870 2023-09-30T22:33:44.542 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:33:44.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.544+0000 7f0340cf0700 1 -- 172.21.15.16:0/194239368 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f033c130870 msgr2=0x7f033c09c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:44.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.544+0000 7f0340cf0700 1 --2- 172.21.15.16:0/194239368 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f033c130870 0x7f033c09c9c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f0330000c00 tx=0x7f0330004750 comp rx=0 tx=0).stop 2023-09-30T22:33:44.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.544+0000 7f0340cf0700 1 -- 172.21.15.16:0/194239368 shutdown_connections 2023-09-30T22:33:44.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.544+0000 7f0340cf0700 1 --2- 172.21.15.16:0/194239368 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f033c12ff20 0x7f033c09c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:44.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.544+0000 7f0340cf0700 1 --2- 172.21.15.16:0/194239368 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f033c133a80 0x7f033c09cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:44.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.544+0000 7f0340cf0700 1 --2- 172.21.15.16:0/194239368 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f033c130870 0x7f033c09c9c0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:44.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.544+0000 7f0340cf0700 1 -- 172.21.15.16:0/194239368 >> 172.21.15.16:0/194239368 conn(0x7f033c0a4210 msgr2=0x7f033c0a0920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:44.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.544+0000 7f0340cf0700 1 -- 172.21.15.16:0/194239368 shutdown_connections 2023-09-30T22:33:44.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.544+0000 7f0340cf0700 1 -- 172.21.15.16:0/194239368 wait complete. 2023-09-30T22:33:44.558 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:33:44.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.582+0000 7fc14271a700 1 -- 172.21.15.16:0/983196243 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc13c1b7380 con 0x7fc13c09f650 2023-09-30T22:33:44.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.582+0000 7fc138ff9700 1 -- 172.21.15.16:0/983196243 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc134019300 con 0x7fc13c09f650 2023-09-30T22:33:44.583 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:33:44.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.585+0000 7fc14271a700 1 -- 172.21.15.16:0/983196243 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc13c09f650 msgr2=0x7fc13c06c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:44.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.585+0000 7fc14271a700 1 --2- 172.21.15.16:0/983196243 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc13c09f650 0x7fc13c06c330 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fc13400cbc0 tx=0x7fc13400ced0 comp rx=0 tx=0).stop 2023-09-30T22:33:44.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.585+0000 7fc14271a700 1 -- 172.21.15.16:0/983196243 shutdown_connections 2023-09-30T22:33:44.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.585+0000 7fc14271a700 1 --2- 172.21.15.16:0/983196243 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc13c09f650 0x7fc13c06c330 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:44.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.585+0000 7fc14271a700 1 --2- 172.21.15.16:0/983196243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc13c09e930 0x7fc13c06b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:44.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.585+0000 7fc14271a700 1 --2- 172.21.15.16:0/983196243 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc13c09f240 0x7fc13c06bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:44.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.585+0000 7fc14271a700 1 -- 172.21.15.16:0/983196243 >> 172.21.15.16:0/983196243 conn(0x7fc13c0a4210 msgr2=0x7fc13c1b8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:44.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.585+0000 7fc14271a700 1 -- 172.21.15.16:0/983196243 shutdown_connections 2023-09-30T22:33:44.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:44.585+0000 7fc14271a700 1 -- 172.21.15.16:0/983196243 wait complete. 2023-09-30T22:33:44.599 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:33:45.460 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:33:45.524 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:33:45.560 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:33:45.601 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:33:45.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.609+0000 7fdb04eaf700 1 Processor -- start 2023-09-30T22:33:45.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.609+0000 7fdb04eaf700 1 -- start start 2023-09-30T22:33:45.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.610+0000 7fdb04eaf700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdb0006a770 0x7fdb0006ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:45.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.610+0000 7fdb04eaf700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdb0006b080 0x7fdb00131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:45.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.610+0000 7fdb04eaf700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdb00131b40 0x7fdb00131fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:45.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.610+0000 7fdb04eaf700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb0009ece0 con 0x7fdb0006b080 2023-09-30T22:33:45.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.610+0000 7fdb04eaf700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0009e760 con 0x7fdb0006a770 2023-09-30T22:33:45.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.610+0000 7fdb04eaf700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdb0009e040 con 0x7fdb00131b40 2023-09-30T22:33:45.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.610+0000 7fdafe59c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdb0006a770 0x7fdb0006ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:45.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.610+0000 7fdafe59c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdb0006a770 0x7fdb0006ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:51992/0 (socket says 172.21.15.16:51992) 2023-09-30T22:33:45.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.610+0000 7fdafed9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdb00131b40 0x7fdb00131fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:45.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.610+0000 7fdafe59c700 1 -- 172.21.15.16:0/2065819432 learned_addr learned my addr 172.21.15.16:0/2065819432 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:45.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.610+0000 7fdafdd9b700 1 --2- 172.21.15.16:0/2065819432 >> v2:172.21.15.16:3300/0 conn(0x7fdb0006b080 0x7fdb00131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:45.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.610+0000 7fdafe59c700 1 -- 172.21.15.16:0/2065819432 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdb00131b40 msgr2=0x7fdb00131fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:45.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.610+0000 7fdafe59c700 1 --2- 172.21.15.16:0/2065819432 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdb00131b40 0x7fdb00131fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:45.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.610+0000 7fdafe59c700 1 -- 172.21.15.16:0/2065819432 >> v2:172.21.15.16:3300/0 conn(0x7fdb0006b080 msgr2=0x7fdb00131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:45.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.610+0000 7fdafe59c700 1 --2- 172.21.15.16:0/2065819432 >> v2:172.21.15.16:3300/0 conn(0x7fdb0006b080 0x7fdb00131600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:45.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.610+0000 7fdafe59c700 1 -- 172.21.15.16:0/2065819432 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb0009f630 con 0x7fdb0006a770 2023-09-30T22:33:45.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.610+0000 7fdafed9d700 1 --2- 172.21.15.16:0/2065819432 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdb00131b40 0x7fdb00131fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:45.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.611+0000 7fdafe59c700 1 --2- 172.21.15.16:0/2065819432 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdb0006a770 0x7fdb0006ab40 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fdaf00098f0 tx=0x7fdaf0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb0fc49324291a35 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:45.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.611+0000 7fdafd59a700 1 -- 172.21.15.16:0/2065819432 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf0011830 con 0x7fdb0006a770 2023-09-30T22:33:45.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.611+0000 7fdafd59a700 1 -- 172.21.15.16:0/2065819432 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaf0011990 con 0x7fdb0006a770 2023-09-30T22:33:45.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.611+0000 7fdafd59a700 1 -- 172.21.15.16:0/2065819432 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf0011d20 con 0x7fdb0006a770 2023-09-30T22:33:45.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.612+0000 7fdb04eaf700 1 -- 172.21.15.16:0/2065819432 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdb0006a770 msgr2=0x7fdb0006ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:45.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.612+0000 7fdb04eaf700 1 --2- 172.21.15.16:0/2065819432 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdb0006a770 0x7fdb0006ab40 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fdaf00098f0 tx=0x7fdaf0009c00 comp rx=0 tx=0).stop 2023-09-30T22:33:45.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.612+0000 7fdb04eaf700 1 -- 172.21.15.16:0/2065819432 shutdown_connections 2023-09-30T22:33:45.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.612+0000 7fdb04eaf700 1 --2- 172.21.15.16:0/2065819432 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdb0006a770 0x7fdb0006ab40 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:45.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.612+0000 7fdb04eaf700 1 --2- 172.21.15.16:0/2065819432 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdb00131b40 0x7fdb00131fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:45.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.612+0000 7fdb04eaf700 1 --2- 172.21.15.16:0/2065819432 >> v2:172.21.15.16:3300/0 conn(0x7fdb0006b080 0x7fdb00131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:45.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.612+0000 7fdb04eaf700 1 -- 172.21.15.16:0/2065819432 >> 172.21.15.16:0/2065819432 conn(0x7fdb00066f50 msgr2=0x7fdb0009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:45.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.612+0000 7fdb04eaf700 1 -- 172.21.15.16:0/2065819432 shutdown_connections 2023-09-30T22:33:45.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.612+0000 7fdb04eaf700 1 -- 172.21.15.16:0/2065819432 wait complete. 2023-09-30T22:33:45.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.613+0000 7fdb04eaf700 1 Processor -- start 2023-09-30T22:33:45.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.613+0000 7fdb04eaf700 1 -- start start 2023-09-30T22:33:45.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.614+0000 7fdb04eaf700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdb0006a770 0x7fdb001201a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:45.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.614+0000 7fdb04eaf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb0006b080 0x7fdb001206e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:45.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.614+0000 7fdb04eaf700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdb00131b40 0x7fdb00120c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:45.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.614+0000 7fdb04eaf700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb00071f10 con 0x7fdb0006b080 2023-09-30T22:33:45.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.614+0000 7fdb04eaf700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb00121f30 con 0x7fdb00131b40 2023-09-30T22:33:45.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.614+0000 7fdb04eaf700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdb00123720 con 0x7fdb0006a770 2023-09-30T22:33:45.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.614+0000 7fdafe59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdb0006a770 0x7fdb001201a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:45.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.614+0000 7fdafe59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdb0006a770 0x7fdb001201a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:47644/0 (socket says 172.21.15.16:47644) 2023-09-30T22:33:45.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.614+0000 7fdafe59c700 1 -- 172.21.15.16:0/3398206362 learned_addr learned my addr 172.21.15.16:0/3398206362 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:45.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.614+0000 7fdafed9d700 1 --2- 172.21.15.16:0/3398206362 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdb00131b40 0x7fdb00120c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:45.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.614+0000 7fdafe59c700 1 -- 172.21.15.16:0/3398206362 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdb00131b40 msgr2=0x7fdb00120c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:45.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.614+0000 7fdafe59c700 1 --2- 172.21.15.16:0/3398206362 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdb00131b40 0x7fdb00120c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:45.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.614+0000 7fdafe59c700 1 -- 172.21.15.16:0/3398206362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb0006b080 msgr2=0x7fdb001206e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:45.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.614+0000 7fdafe59c700 1 --2- 172.21.15.16:0/3398206362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb0006b080 0x7fdb001206e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:45.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.614+0000 7fdafe59c700 1 -- 172.21.15.16:0/3398206362 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf8009a70 con 0x7fdb0006a770 2023-09-30T22:33:45.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.615+0000 7fdafed9d700 1 --2- 172.21.15.16:0/3398206362 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdb00131b40 0x7fdb00120c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:33:45.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.615+0000 7fdafe59c700 1 --2- 172.21.15.16:0/3398206362 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdb0006a770 0x7fdb001201a0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0000c00 tx=0x7fdaf0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:45.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.615+0000 7fdaeeffd700 1 -- 172.21.15.16:0/3398206362 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf0004300 con 0x7fdb0006a770 2023-09-30T22:33:45.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.615+0000 7fdaeeffd700 1 -- 172.21.15.16:0/3398206362 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaf001f070 con 0x7fdb0006a770 2023-09-30T22:33:45.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.615+0000 7fdaeeffd700 1 -- 172.21.15.16:0/3398206362 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf00036a0 con 0x7fdb0006a770 2023-09-30T22:33:45.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.616+0000 7fdb04eaf700 1 -- 172.21.15.16:0/3398206362 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaf0009550 con 0x7fdb0006a770 2023-09-30T22:33:45.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.616+0000 7fdb04eaf700 1 -- 172.21.15.16:0/3398206362 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb001315b0 con 0x7fdb0006a770 2023-09-30T22:33:45.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.616+0000 7fdaeeffd700 1 -- 172.21.15.16:0/3398206362 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdaf0021030 con 0x7fdb0006a770 2023-09-30T22:33:45.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.617+0000 7fdb04eaf700 1 -- 172.21.15.16:0/3398206362 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb00074ea0 con 0x7fdb0006a770 2023-09-30T22:33:45.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.617+0000 7fdaeeffd700 1 -- 172.21.15.16:0/3398206362 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fdaf00408e0 con 0x7fdb0006a770 2023-09-30T22:33:45.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.622+0000 7fdaeeffd700 1 -- 172.21.15.16:0/3398206362 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdaf00199a0 con 0x7fdb0006a770 2023-09-30T22:33:45.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.673+0000 7f63c4879700 1 Processor -- start 2023-09-30T22:33:45.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.673+0000 7f63c4879700 1 -- start start 2023-09-30T22:33:45.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.673+0000 7f63c4879700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f63c0131810 0x7f63c0131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:45.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.673+0000 7f63c4879700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f63c0099a40 0x7f63c0099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:45.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.674+0000 7f63c4879700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63c009a850 con 0x7f63c009a3b0 2023-09-30T22:33:45.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.674+0000 7f63c4879700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63c009b580 con 0x7f63c0099a40 2023-09-30T22:33:45.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.674+0000 7f63c4879700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63c01313c0 con 0x7f63c0131810 2023-09-30T22:33:45.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.674+0000 7f63bdd9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f63c0099a40 0x7f63c0099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:45.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.674+0000 7f63bdd9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f63c0099a40 0x7f63c0099e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56234/0 (socket says 172.21.15.16:56234) 2023-09-30T22:33:45.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.674+0000 7f63bdd9b700 1 -- 172.21.15.16:0/16582262 learned_addr learned my addr 172.21.15.16:0/16582262 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:45.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.674+0000 7f63be59c700 1 --2- 172.21.15.16:0/16582262 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f63c0131810 0x7f63c0131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:45.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.674+0000 7f63bdd9b700 1 -- 172.21.15.16:0/16582262 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f63c0131810 msgr2=0x7f63c0131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:45.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.674+0000 7f63bdd9b700 1 --2- 172.21.15.16:0/16582262 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f63c0131810 0x7f63c0131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:45.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.674+0000 7f63bdd9b700 1 -- 172.21.15.16:0/16582262 >> v1:172.21.15.16:6789/0 conn(0x7f63c009a3b0 legacy=0x7f63c0138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:45.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.674+0000 7f63bdd9b700 1 -- 172.21.15.16:0/16582262 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63c006e8c0 con 0x7f63c0099a40 2023-09-30T22:33:45.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.674+0000 7f63bdd9b700 1 --2- 172.21.15.16:0/16582262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63c0099a40 0x7f63c0099e70 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f63b4009920 tx=0x7f63b4009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e86a0b39f269ad4d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:45.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.675+0000 7f63bd59a700 1 -- 172.21.15.16:0/16582262 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b4011830 con 0x7f63c0099a40 2023-09-30T22:33:45.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.675+0000 7f63bd59a700 1 -- 172.21.15.16:0/16582262 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63b4011990 con 0x7f63c0099a40 2023-09-30T22:33:45.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.675+0000 7f63bd59a700 1 -- 172.21.15.16:0/16582262 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b4011d20 con 0x7f63c0099a40 2023-09-30T22:33:45.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.675+0000 7f63c4879700 1 -- 172.21.15.16:0/16582262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63c0099a40 msgr2=0x7f63c0099e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:45.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.675+0000 7f63c4879700 1 --2- 172.21.15.16:0/16582262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63c0099a40 0x7f63c0099e70 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f63b4009920 tx=0x7f63b4009c30 comp rx=0 tx=0).stop 2023-09-30T22:33:45.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.676+0000 7f63c4879700 1 -- 172.21.15.16:0/16582262 shutdown_connections 2023-09-30T22:33:45.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.676+0000 7f63c4879700 1 --2- 172.21.15.16:0/16582262 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f63c0131810 0x7f63c0131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:45.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.676+0000 7f63c4879700 1 --2- 172.21.15.16:0/16582262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63c0099a40 0x7f63c0099e70 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:45.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.676+0000 7f63c4879700 1 -- 172.21.15.16:0/16582262 >> 172.21.15.16:0/16582262 conn(0x7f63c00a2aa0 msgr2=0x7f63c00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:45.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.676+0000 7f63c4879700 1 -- 172.21.15.16:0/16582262 shutdown_connections 2023-09-30T22:33:45.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.676+0000 7f63c4879700 1 -- 172.21.15.16:0/16582262 wait complete. 2023-09-30T22:33:45.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.677+0000 7f63c4879700 1 Processor -- start 2023-09-30T22:33:45.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.677+0000 7f63c4879700 1 -- start start 2023-09-30T22:33:45.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.678+0000 7f63c4879700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f63c0099a40 0x7f63c011b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:45.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.678+0000 7f63c4879700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f63c009a3b0 0x7f63c011bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:45.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.678+0000 7f63c4879700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63c0131810 0x7f63c011c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:45.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.678+0000 7f63c4879700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c011d2a0 con 0x7f63c0131810 2023-09-30T22:33:45.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.678+0000 7f63c4879700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63c011e4f0 con 0x7f63c009a3b0 2023-09-30T22:33:45.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.678+0000 7f63c4879700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f63c011ef70 con 0x7f63c0099a40 2023-09-30T22:33:45.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.678+0000 7f63be59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f63c0099a40 0x7f63c011b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:45.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.678+0000 7f63be59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f63c0099a40 0x7f63c011b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:47654/0 (socket says 172.21.15.16:47654) 2023-09-30T22:33:45.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.678+0000 7f63be59c700 1 -- 172.21.15.16:0/3576907042 learned_addr learned my addr 172.21.15.16:0/3576907042 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:45.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.678+0000 7f63bed9d700 1 --2- 172.21.15.16:0/3576907042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63c0131810 0x7f63c011c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:45.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.678+0000 7f63bdd9b700 1 --2- 172.21.15.16:0/3576907042 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f63c009a3b0 0x7f63c011bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:45.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.678+0000 7f63be59c700 1 -- 172.21.15.16:0/3576907042 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f63c009a3b0 msgr2=0x7f63c011bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:45.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.678+0000 7f63be59c700 1 --2- 172.21.15.16:0/3576907042 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f63c009a3b0 0x7f63c011bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:45.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.678+0000 7f63be59c700 1 -- 172.21.15.16:0/3576907042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63c0131810 msgr2=0x7f63c011c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:45.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.678+0000 7f63be59c700 1 --2- 172.21.15.16:0/3576907042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63c0131810 0x7f63c011c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:45.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.678+0000 7f63be59c700 1 -- 172.21.15.16:0/3576907042 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b4009550 con 0x7f63c0099a40 2023-09-30T22:33:45.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.678+0000 7f63bdd9b700 1 --2- 172.21.15.16:0/3576907042 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f63c009a3b0 0x7f63c011bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:33:45.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.679+0000 7f63be59c700 1 --2- 172.21.15.16:0/3576907042 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f63c0099a40 0x7f63c011b6e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f63b000e9b0 tx=0x7f63b000ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:45.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.679+0000 7f63aeffd700 1 -- 172.21.15.16:0/3576907042 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b0009710 con 0x7f63c0099a40 2023-09-30T22:33:45.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.679+0000 7f63aeffd700 1 -- 172.21.15.16:0/3576907042 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63b000c850 con 0x7f63c0099a40 2023-09-30T22:33:45.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.679+0000 7f63c4879700 1 -- 172.21.15.16:0/3576907042 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63c011d890 con 0x7f63c0099a40 2023-09-30T22:33:45.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.679+0000 7f63aeffd700 1 -- 172.21.15.16:0/3576907042 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b000cbc0 con 0x7f63c0099a40 2023-09-30T22:33:45.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.679+0000 7f63c4879700 1 -- 172.21.15.16:0/3576907042 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63c006eb00 con 0x7f63c0099a40 2023-09-30T22:33:45.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.680+0000 7f63aeffd700 1 -- 172.21.15.16:0/3576907042 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f63b000cd20 con 0x7f63c0099a40 2023-09-30T22:33:45.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.680+0000 7f63c4879700 1 -- 172.21.15.16:0/3576907042 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63c0074ea0 con 0x7f63c0099a40 2023-09-30T22:33:45.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.680+0000 7f63aeffd700 1 -- 172.21.15.16:0/3576907042 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f63b003fc70 con 0x7f63c0099a40 2023-09-30T22:33:45.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.689+0000 7f63aeffd700 1 -- 172.21.15.16:0/3576907042 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f63b001b3b0 con 0x7f63c0099a40 2023-09-30T22:33:45.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.717+0000 7f8257858700 1 Processor -- start 2023-09-30T22:33:45.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.718+0000 7f8257858700 1 -- start start 2023-09-30T22:33:45.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.718+0000 7f8257858700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f82500a1510 0x7f8250138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:45.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.718+0000 7f8257858700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8250139250 0x7f825013b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:45.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.718+0000 7f8257858700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f825009edd0 con 0x7f82500a0a30 2023-09-30T22:33:45.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.718+0000 7f8257858700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82500a18e0 con 0x7f8250139250 2023-09-30T22:33:45.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.718+0000 7f8257858700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f825009e1f0 con 0x7f82500a1510 2023-09-30T22:33:45.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.718+0000 7f8254df3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f82500a1510 0x7f8250138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:45.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.718+0000 7f8254df3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f82500a1510 0x7f8250138d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:47658/0 (socket says 172.21.15.16:47658) 2023-09-30T22:33:45.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.718+0000 7f8254df3700 1 -- 172.21.15.16:0/854095142 learned_addr learned my addr 172.21.15.16:0/854095142 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:45.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.719+0000 7f8255df5700 1 --2- 172.21.15.16:0/854095142 >> v2:172.21.15.16:3300/0 conn(0x7f8250139250 0x7f825013b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:45.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.719+0000 7f8254df3700 1 -- 172.21.15.16:0/854095142 >> v2:172.21.15.16:3300/0 conn(0x7f8250139250 msgr2=0x7f825013b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:45.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.719+0000 7f8254df3700 1 --2- 172.21.15.16:0/854095142 >> v2:172.21.15.16:3300/0 conn(0x7f8250139250 0x7f825013b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:45.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.719+0000 7f8254df3700 1 -- 172.21.15.16:0/854095142 >> v1:172.21.15.16:6789/0 conn(0x7f82500a0a30 legacy=0x7f82500a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:45.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.719+0000 7f8254df3700 1 -- 172.21.15.16:0/854095142 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f825009dc50 con 0x7f82500a1510 2023-09-30T22:33:45.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.719+0000 7f8254df3700 1 --2- 172.21.15.16:0/854095142 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f82500a1510 0x7f8250138d10 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f8240009920 tx=0x7f8240009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5751793beed08a3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:45.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.719+0000 7f8247fff700 1 -- 172.21.15.16:0/854095142 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8240011830 con 0x7f82500a1510 2023-09-30T22:33:45.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.719+0000 7f8247fff700 1 -- 172.21.15.16:0/854095142 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8240011990 con 0x7f82500a1510 2023-09-30T22:33:45.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.719+0000 7f8247fff700 1 -- 172.21.15.16:0/854095142 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8240011d20 con 0x7f82500a1510 2023-09-30T22:33:45.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.720+0000 7f8257858700 1 -- 172.21.15.16:0/854095142 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f82500a1510 msgr2=0x7f8250138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:45.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.720+0000 7f8257858700 1 --2- 172.21.15.16:0/854095142 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f82500a1510 0x7f8250138d10 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f8240009920 tx=0x7f8240009c30 comp rx=0 tx=0).stop 2023-09-30T22:33:45.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.721+0000 7f8257858700 1 -- 172.21.15.16:0/854095142 shutdown_connections 2023-09-30T22:33:45.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.721+0000 7f8257858700 1 --2- 172.21.15.16:0/854095142 >> v2:172.21.15.16:3300/0 conn(0x7f8250139250 0x7f825013b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:45.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.721+0000 7f8257858700 1 --2- 172.21.15.16:0/854095142 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f82500a1510 0x7f8250138d10 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:45.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.721+0000 7f8257858700 1 -- 172.21.15.16:0/854095142 >> 172.21.15.16:0/854095142 conn(0x7f82500a6550 msgr2=0x7f82500a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:45.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.721+0000 7f8257858700 1 -- 172.21.15.16:0/854095142 shutdown_connections 2023-09-30T22:33:45.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.721+0000 7f8257858700 1 -- 172.21.15.16:0/854095142 wait complete. 2023-09-30T22:33:45.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.722+0000 7f8257858700 1 Processor -- start 2023-09-30T22:33:45.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.722+0000 7f8257858700 1 -- start start 2023-09-30T22:33:45.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.722+0000 7f8257858700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82500a0a30 0x7f8250121940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:45.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.722+0000 7f8257858700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f82500a1510 0x7f8250121e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:45.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.722+0000 7f8257858700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8250139250 0x7f82501223c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:45.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.722+0000 7f8257858700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82501235d0 con 0x7f82500a0a30 2023-09-30T22:33:45.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.722+0000 7f8257858700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8250124e30 con 0x7f8250139250 2023-09-30T22:33:45.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.722+0000 7f8257858700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8250123a70 con 0x7f82500a1510 2023-09-30T22:33:45.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.722+0000 7f8254df3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f82500a1510 0x7f8250121e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:45.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.722+0000 7f8254df3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f82500a1510 0x7f8250121e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:47668/0 (socket says 172.21.15.16:47668) 2023-09-30T22:33:45.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.722+0000 7f8254df3700 1 -- 172.21.15.16:0/1064269809 learned_addr learned my addr 172.21.15.16:0/1064269809 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:45.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.723+0000 7f8255df5700 1 --2- 172.21.15.16:0/1064269809 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8250139250 0x7f82501223c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:45.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.723+0000 7f82555f4700 1 --2- 172.21.15.16:0/1064269809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82500a0a30 0x7f8250121940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:45.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.723+0000 7f8254df3700 1 -- 172.21.15.16:0/1064269809 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8250139250 msgr2=0x7f82501223c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:45.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.723+0000 7f8254df3700 1 --2- 172.21.15.16:0/1064269809 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8250139250 0x7f82501223c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:45.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.723+0000 7f8254df3700 1 -- 172.21.15.16:0/1064269809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82500a0a30 msgr2=0x7f8250121940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:45.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.723+0000 7f8254df3700 1 --2- 172.21.15.16:0/1064269809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82500a0a30 0x7f8250121940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:45.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.723+0000 7f8254df3700 1 -- 172.21.15.16:0/1064269809 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8240009550 con 0x7f82500a1510 2023-09-30T22:33:45.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.723+0000 7f82555f4700 1 --2- 172.21.15.16:0/1064269809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82500a0a30 0x7f8250121940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:33:45.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.723+0000 7f8254df3700 1 --2- 172.21.15.16:0/1064269809 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f82500a1510 0x7f8250121e80 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f82400098f0 tx=0x7f8240004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:45.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.723+0000 7f8245ffb700 1 -- 172.21.15.16:0/1064269809 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82400036a0 con 0x7f82500a1510 2023-09-30T22:33:45.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.723+0000 7f8245ffb700 1 -- 172.21.15.16:0/1064269809 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f824001f070 con 0x7f82500a1510 2023-09-30T22:33:45.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.723+0000 7f8257858700 1 -- 172.21.15.16:0/1064269809 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8250138b60 con 0x7f82500a1510 2023-09-30T22:33:45.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.723+0000 7f8245ffb700 1 -- 172.21.15.16:0/1064269809 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8240003a60 con 0x7f82500a1510 2023-09-30T22:33:45.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.724+0000 7f8257858700 1 -- 172.21.15.16:0/1064269809 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8250123cb0 con 0x7f82500a1510 2023-09-30T22:33:45.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.725+0000 7f8257858700 1 -- 172.21.15.16:0/1064269809 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8250074ea0 con 0x7f82500a1510 2023-09-30T22:33:45.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.729+0000 7f8245ffb700 1 -- 172.21.15.16:0/1064269809 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8240003bc0 con 0x7f82500a1510 2023-09-30T22:33:45.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.729+0000 7f8245ffb700 1 -- 172.21.15.16:0/1064269809 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f8240021070 con 0x7f82500a1510 2023-09-30T22:33:45.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.734+0000 7f8245ffb700 1 -- 172.21.15.16:0/1064269809 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f824002fce0 con 0x7f82500a1510 2023-09-30T22:33:45.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.761+0000 7f958744d700 1 Processor -- start 2023-09-30T22:33:45.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.761+0000 7f958744d700 1 -- start start 2023-09-30T22:33:45.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.762+0000 7f958744d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f958009d6f0 0x7f958009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:45.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.762+0000 7f958744d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f958009e000 0x7f95801339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:45.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.762+0000 7f958744d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f958009b120 con 0x7f958009e410 2023-09-30T22:33:45.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.762+0000 7f958744d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f958006b460 con 0x7f958009d6f0 2023-09-30T22:33:45.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.762+0000 7f958744d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f958009a7e0 con 0x7f958009e000 2023-09-30T22:33:45.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.762+0000 7f95851e9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f958009d6f0 0x7f958009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:45.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.762+0000 7f95859ea700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f958009e410 0x7f9580133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38184/0 (socket says 172.21.15.16:38184) 2023-09-30T22:33:45.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.762+0000 7f95859ea700 1 -- 172.21.15.16:0/344728735 learned_addr learned my addr 172.21.15.16:0/344728735 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:45.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.762+0000 7f95849e8700 1 --2- 172.21.15.16:0/344728735 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f958009e000 0x7f95801339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:45.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.762+0000 7f95849e8700 1 -- 172.21.15.16:0/344728735 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f958009d6f0 msgr2=0x7f958009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:45.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.762+0000 7f95849e8700 1 --2- 172.21.15.16:0/344728735 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f958009d6f0 0x7f958009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:45.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.762+0000 7f95849e8700 1 -- 172.21.15.16:0/344728735 >> v1:172.21.15.16:6789/0 conn(0x7f958009e410 legacy=0x7f9580133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:45.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.762+0000 7f95849e8700 1 -- 172.21.15.16:0/344728735 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f958009a500 con 0x7f958009e000 2023-09-30T22:33:45.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.763+0000 7f95849e8700 1 --2- 172.21.15.16:0/344728735 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f958009e000 0x7f95801339f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f95700098a0 tx=0x7f9570009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1a342b3445593bb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:45.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.763+0000 7f9577fff700 1 -- 172.21.15.16:0/344728735 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9570011830 con 0x7f958009e000 2023-09-30T22:33:45.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.763+0000 7f9577fff700 1 -- 172.21.15.16:0/344728735 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9570011990 con 0x7f958009e000 2023-09-30T22:33:45.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.763+0000 7f9577fff700 1 -- 172.21.15.16:0/344728735 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9570011d20 con 0x7f958009e000 2023-09-30T22:33:45.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.764+0000 7f958744d700 1 -- 172.21.15.16:0/344728735 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f958009e000 msgr2=0x7f95801339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:45.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.764+0000 7f958744d700 1 --2- 172.21.15.16:0/344728735 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f958009e000 0x7f95801339f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f95700098a0 tx=0x7f9570009bb0 comp rx=0 tx=0).stop 2023-09-30T22:33:45.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.764+0000 7f958744d700 1 -- 172.21.15.16:0/344728735 shutdown_connections 2023-09-30T22:33:45.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.764+0000 7f958744d700 1 --2- 172.21.15.16:0/344728735 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f958009d6f0 0x7f958009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:45.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.764+0000 7f958744d700 1 --2- 172.21.15.16:0/344728735 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f958009e000 0x7f95801339f0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:45.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.764+0000 7f958744d700 1 -- 172.21.15.16:0/344728735 >> 172.21.15.16:0/344728735 conn(0x7f9580066f50 msgr2=0x7f958009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:45.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.764+0000 7f958744d700 1 -- 172.21.15.16:0/344728735 shutdown_connections 2023-09-30T22:33:45.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.764+0000 7f958744d700 1 -- 172.21.15.16:0/344728735 wait complete. 2023-09-30T22:33:45.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.765+0000 7f958744d700 1 Processor -- start 2023-09-30T22:33:45.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.765+0000 7f958744d700 1 -- start start 2023-09-30T22:33:45.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.765+0000 7f958744d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f958009d6f0 0x7f95801a5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:45.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.765+0000 7f958744d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f958009e000 0x7f95801a64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:45.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.766+0000 7f958744d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f958009e410 0x7f95801a6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:45.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.766+0000 7f958744d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95801a7b10 con 0x7f958009d6f0 2023-09-30T22:33:45.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.766+0000 7f958744d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95801a8d60 con 0x7f958009e000 2023-09-30T22:33:45.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.766+0000 7f958744d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f95801a80f0 con 0x7f958009e410 2023-09-30T22:33:45.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.766+0000 7f95859ea700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f958009e410 0x7f95801a6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:45.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.766+0000 7f95859ea700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f958009e410 0x7f95801a6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:47694/0 (socket says 172.21.15.16:47694) 2023-09-30T22:33:45.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.766+0000 7f95859ea700 1 -- 172.21.15.16:0/579336409 learned_addr learned my addr 172.21.15.16:0/579336409 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:45.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.766+0000 7f95851e9700 1 --2- 172.21.15.16:0/579336409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f958009d6f0 0x7f95801a5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:45.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.766+0000 7f95849e8700 1 --2- 172.21.15.16:0/579336409 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f958009e000 0x7f95801a64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:45.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.766+0000 7f95859ea700 1 -- 172.21.15.16:0/579336409 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f958009e000 msgr2=0x7f95801a64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:45.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.766+0000 7f95859ea700 1 --2- 172.21.15.16:0/579336409 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f958009e000 0x7f95801a64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:45.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.766+0000 7f95859ea700 1 -- 172.21.15.16:0/579336409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f958009d6f0 msgr2=0x7f95801a5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:45.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.766+0000 7f95859ea700 1 --2- 172.21.15.16:0/579336409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f958009d6f0 0x7f95801a5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:45.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.766+0000 7f95859ea700 1 -- 172.21.15.16:0/579336409 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95700094a0 con 0x7f958009e410 2023-09-30T22:33:45.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.766+0000 7f95859ea700 1 --2- 172.21.15.16:0/579336409 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f958009e410 0x7f95801a6a10 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f957800caf0 tx=0x7f957800ceb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:45.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.766+0000 7f9575ffb700 1 -- 172.21.15.16:0/579336409 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957800aad0 con 0x7f958009e410 2023-09-30T22:33:45.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.767+0000 7f9575ffb700 1 -- 172.21.15.16:0/579336409 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f957800ac30 con 0x7f958009e410 2023-09-30T22:33:45.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.767+0000 7f9575ffb700 1 -- 172.21.15.16:0/579336409 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9578012400 con 0x7f958009e410 2023-09-30T22:33:45.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.767+0000 7f958744d700 1 -- 172.21.15.16:0/579336409 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95801a9740 con 0x7f958009e410 2023-09-30T22:33:45.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.767+0000 7f958744d700 1 -- 172.21.15.16:0/579336409 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f958009c710 con 0x7f958009e410 2023-09-30T22:33:45.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.768+0000 7f958744d700 1 -- 172.21.15.16:0/579336409 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9580074ea0 con 0x7f958009e410 2023-09-30T22:33:45.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.768+0000 7f9575ffb700 1 -- 172.21.15.16:0/579336409 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f957801b3b0 con 0x7f958009e410 2023-09-30T22:33:45.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.769+0000 7f9575ffb700 1 -- 172.21.15.16:0/579336409 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f957803f880 con 0x7f958009e410 2023-09-30T22:33:45.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.777+0000 7f9575ffb700 1 -- 172.21.15.16:0/579336409 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f95780192c0 con 0x7f958009e410 2023-09-30T22:33:45.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.846+0000 7fdb04eaf700 1 -- 172.21.15.16:0/3398206362 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fdb00123c20 con 0x7fdb0006a770 2023-09-30T22:33:45.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.847+0000 7fdaeeffd700 1 -- 172.21.15.16:0/3398206362 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdaf00193b0 con 0x7fdb0006a770 2023-09-30T22:33:45.847 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:33:45.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.849+0000 7fdb04eaf700 1 -- 172.21.15.16:0/3398206362 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdb0006a770 msgr2=0x7fdb001201a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:45.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.849+0000 7fdb04eaf700 1 --2- 172.21.15.16:0/3398206362 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdb0006a770 0x7fdb001201a0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0000c00 tx=0x7fdaf0004750 comp rx=0 tx=0).stop 2023-09-30T22:33:45.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.849+0000 7fdb04eaf700 1 -- 172.21.15.16:0/3398206362 shutdown_connections 2023-09-30T22:33:45.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.849+0000 7fdb04eaf700 1 --2- 172.21.15.16:0/3398206362 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdb00131b40 0x7fdb00120c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:45.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.849+0000 7fdb04eaf700 1 --2- 172.21.15.16:0/3398206362 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdb0006a770 0x7fdb001201a0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:45.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.849+0000 7fdb04eaf700 1 --2- 172.21.15.16:0/3398206362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb0006b080 0x7fdb001206e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:45.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.849+0000 7fdb04eaf700 1 -- 172.21.15.16:0/3398206362 >> 172.21.15.16:0/3398206362 conn(0x7fdb00066f50 msgr2=0x7fdb000a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:45.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.849+0000 7fdb04eaf700 1 -- 172.21.15.16:0/3398206362 shutdown_connections 2023-09-30T22:33:45.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.849+0000 7fdb04eaf700 1 -- 172.21.15.16:0/3398206362 wait complete. 2023-09-30T22:33:45.864 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:33:45.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.905+0000 7f63c4879700 1 -- 172.21.15.16:0/3576907042 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f63c006b9e0 con 0x7f63c0099a40 2023-09-30T22:33:45.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.905+0000 7f63aeffd700 1 -- 172.21.15.16:0/3576907042 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f63b0021020 con 0x7f63c0099a40 2023-09-30T22:33:45.906 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:33:45.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.908+0000 7f63c4879700 1 -- 172.21.15.16:0/3576907042 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f63c0099a40 msgr2=0x7f63c011b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:45.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.908+0000 7f63c4879700 1 --2- 172.21.15.16:0/3576907042 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f63c0099a40 0x7f63c011b6e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f63b000e9b0 tx=0x7f63b000ecc0 comp rx=0 tx=0).stop 2023-09-30T22:33:45.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.908+0000 7f63c4879700 1 -- 172.21.15.16:0/3576907042 shutdown_connections 2023-09-30T22:33:45.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.908+0000 7f63c4879700 1 --2- 172.21.15.16:0/3576907042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63c0131810 0x7f63c011c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:45.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.908+0000 7f63c4879700 1 --2- 172.21.15.16:0/3576907042 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f63c0099a40 0x7f63c011b6e0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:45.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.908+0000 7f63c4879700 1 --2- 172.21.15.16:0/3576907042 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f63c009a3b0 0x7f63c011bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:45.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.908+0000 7f63c4879700 1 -- 172.21.15.16:0/3576907042 >> 172.21.15.16:0/3576907042 conn(0x7f63c00a2aa0 msgr2=0x7f63c009d770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:45.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.908+0000 7f63c4879700 1 -- 172.21.15.16:0/3576907042 shutdown_connections 2023-09-30T22:33:45.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.908+0000 7f63c4879700 1 -- 172.21.15.16:0/3576907042 wait complete. 2023-09-30T22:33:45.922 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:33:45.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.948+0000 7f8257858700 1 -- 172.21.15.16:0/1064269809 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f82500a1d10 con 0x7f82500a1510 2023-09-30T22:33:45.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.948+0000 7f8245ffb700 1 -- 172.21.15.16:0/1064269809 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8240015070 con 0x7f82500a1510 2023-09-30T22:33:45.949 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:33:45.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.951+0000 7f8257858700 1 -- 172.21.15.16:0/1064269809 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f82500a1510 msgr2=0x7f8250121e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:45.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.951+0000 7f8257858700 1 --2- 172.21.15.16:0/1064269809 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f82500a1510 0x7f8250121e80 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f82400098f0 tx=0x7f8240004750 comp rx=0 tx=0).stop 2023-09-30T22:33:45.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.951+0000 7f8257858700 1 -- 172.21.15.16:0/1064269809 shutdown_connections 2023-09-30T22:33:45.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.951+0000 7f8257858700 1 --2- 172.21.15.16:0/1064269809 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8250139250 0x7f82501223c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:45.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.951+0000 7f8257858700 1 --2- 172.21.15.16:0/1064269809 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82500a0a30 0x7f8250121940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:45.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.951+0000 7f8257858700 1 --2- 172.21.15.16:0/1064269809 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f82500a1510 0x7f8250121e80 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:45.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.951+0000 7f8257858700 1 -- 172.21.15.16:0/1064269809 >> 172.21.15.16:0/1064269809 conn(0x7f82500a6550 msgr2=0x7f8250137bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:45.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.951+0000 7f8257858700 1 -- 172.21.15.16:0/1064269809 shutdown_connections 2023-09-30T22:33:45.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.951+0000 7f8257858700 1 -- 172.21.15.16:0/1064269809 wait complete. 2023-09-30T22:33:45.965 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:33:45.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.988+0000 7f958744d700 1 -- 172.21.15.16:0/579336409 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f958006c360 con 0x7f958009e410 2023-09-30T22:33:45.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.988+0000 7f9575ffb700 1 -- 172.21.15.16:0/579336409 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9578017030 con 0x7f958009e410 2023-09-30T22:33:45.989 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:33:45.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.991+0000 7f958744d700 1 -- 172.21.15.16:0/579336409 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f958009e410 msgr2=0x7f95801a6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:45.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.991+0000 7f958744d700 1 --2- 172.21.15.16:0/579336409 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f958009e410 0x7f95801a6a10 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f957800caf0 tx=0x7f957800ceb0 comp rx=0 tx=0).stop 2023-09-30T22:33:45.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.991+0000 7f958744d700 1 -- 172.21.15.16:0/579336409 shutdown_connections 2023-09-30T22:33:45.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.991+0000 7f958744d700 1 --2- 172.21.15.16:0/579336409 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f958009e410 0x7f95801a6a10 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:45.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.991+0000 7f958744d700 1 --2- 172.21.15.16:0/579336409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f958009d6f0 0x7f95801a5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:45.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.991+0000 7f958744d700 1 --2- 172.21.15.16:0/579336409 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f958009e000 0x7f95801a64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:45.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.991+0000 7f958744d700 1 -- 172.21.15.16:0/579336409 >> 172.21.15.16:0/579336409 conn(0x7f9580066f50 msgr2=0x7f95801badf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:45.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.991+0000 7f958744d700 1 -- 172.21.15.16:0/579336409 shutdown_connections 2023-09-30T22:33:45.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:45.991+0000 7f958744d700 1 -- 172.21.15.16:0/579336409 wait complete. 2023-09-30T22:33:46.004 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:33:46.865 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:33:46.923 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:33:46.966 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:33:47.005 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:33:47.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.019+0000 7f9ac5d02700 1 Processor -- start 2023-09-30T22:33:47.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.019+0000 7f9ac5d02700 1 -- start start 2023-09-30T22:33:47.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.020+0000 7f9ac5d02700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9ac01305f0 0x7f9ac01309c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:47.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.020+0000 7f9ac5d02700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9ac006f660 0x7f9ac006fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:47.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.020+0000 7f9ac5d02700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9ac006ff90 0x7f9ac0137530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:47.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.020+0000 7f9ac5d02700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ac01301c0 con 0x7f9ac01305f0 2023-09-30T22:33:47.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.020+0000 7f9ac5d02700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac0071ff0 con 0x7f9ac006ff90 2023-09-30T22:33:47.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.020+0000 7f9ac5d02700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac0070e70 con 0x7f9ac006f660 2023-09-30T22:33:47.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.020+0000 7f9abeffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9ac006f660 0x7f9ac006fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:47.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.020+0000 7f9abeffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9ac006f660 0x7f9ac006fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:47700/0 (socket says 172.21.15.16:47700) 2023-09-30T22:33:47.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.020+0000 7f9abeffd700 1 -- 172.21.15.16:0/2084683856 learned_addr learned my addr 172.21.15.16:0/2084683856 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:47.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.020+0000 7f9abffff700 1 --2- 172.21.15.16:0/2084683856 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9ac006ff90 0x7f9ac0137530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:47.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.020+0000 7f9abeffd700 1 -- 172.21.15.16:0/2084683856 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9ac006ff90 msgr2=0x7f9ac0137530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:47.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.020+0000 7f9abeffd700 1 --2- 172.21.15.16:0/2084683856 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9ac006ff90 0x7f9ac0137530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:47.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.020+0000 7f9abeffd700 1 -- 172.21.15.16:0/2084683856 >> v2:172.21.15.16:3300/0 conn(0x7f9ac01305f0 msgr2=0x7f9ac01309c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:47.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.020+0000 7f9abeffd700 1 --2- 172.21.15.16:0/2084683856 >> v2:172.21.15.16:3300/0 conn(0x7f9ac01305f0 0x7f9ac01309c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:47.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.020+0000 7f9abeffd700 1 -- 172.21.15.16:0/2084683856 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac00722b0 con 0x7f9ac006f660 2023-09-30T22:33:47.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.021+0000 7f9abeffd700 1 --2- 172.21.15.16:0/2084683856 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9ac006f660 0x7f9ac006fa50 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f9ab400b9a0 tx=0x7f9ab400bcb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d40d15d1fe1ca60 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:47.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.021+0000 7f9abe7fc700 1 -- 172.21.15.16:0/2084683856 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab400caa0 con 0x7f9ac006f660 2023-09-30T22:33:47.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.021+0000 7f9abe7fc700 1 -- 172.21.15.16:0/2084683856 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ab400cc00 con 0x7f9ac006f660 2023-09-30T22:33:47.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.021+0000 7f9abe7fc700 1 -- 172.21.15.16:0/2084683856 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab400f460 con 0x7f9ac006f660 2023-09-30T22:33:47.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.022+0000 7f9ac5d02700 1 -- 172.21.15.16:0/2084683856 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9ac006f660 msgr2=0x7f9ac006fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:47.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.022+0000 7f9ac5d02700 1 --2- 172.21.15.16:0/2084683856 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9ac006f660 0x7f9ac006fa50 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f9ab400b9a0 tx=0x7f9ab400bcb0 comp rx=0 tx=0).stop 2023-09-30T22:33:47.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.022+0000 7f9ac5d02700 1 -- 172.21.15.16:0/2084683856 shutdown_connections 2023-09-30T22:33:47.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.022+0000 7f9ac5d02700 1 --2- 172.21.15.16:0/2084683856 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9ac006ff90 0x7f9ac0137530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:47.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.022+0000 7f9ac5d02700 1 --2- 172.21.15.16:0/2084683856 >> v2:172.21.15.16:3300/0 conn(0x7f9ac01305f0 0x7f9ac01309c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:47.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.022+0000 7f9ac5d02700 1 --2- 172.21.15.16:0/2084683856 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9ac006f660 0x7f9ac006fa50 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:47.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.022+0000 7f9ac5d02700 1 -- 172.21.15.16:0/2084683856 >> 172.21.15.16:0/2084683856 conn(0x7f9ac00a2aa0 msgr2=0x7f9ac00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:47.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.022+0000 7f9ac5d02700 1 -- 172.21.15.16:0/2084683856 shutdown_connections 2023-09-30T22:33:47.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.023+0000 7f9ac5d02700 1 -- 172.21.15.16:0/2084683856 wait complete. 2023-09-30T22:33:47.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.024+0000 7f9ac5d02700 1 Processor -- start 2023-09-30T22:33:47.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.024+0000 7f9ac5d02700 1 -- start start 2023-09-30T22:33:47.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.024+0000 7f9ac5d02700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ac006f660 0x7f9ac011b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:47.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.024+0000 7f9ac5d02700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9ac006ff90 0x7f9ac011be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:47.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.024+0000 7f9ac5d02700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9ac01305f0 0x7f9ac011c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:47.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.024+0000 7f9ac5d02700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac012ecf0 con 0x7f9ac006f660 2023-09-30T22:33:47.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.024+0000 7f9ac5d02700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac0071820 con 0x7f9ac006ff90 2023-09-30T22:33:47.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.024+0000 7f9ac5d02700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac011da50 con 0x7f9ac01305f0 2023-09-30T22:33:47.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.025+0000 7f9abf7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ac006f660 0x7f9ac011b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:47.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.025+0000 7f9abf7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ac006f660 0x7f9ac011b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56288/0 (socket says 172.21.15.16:56288) 2023-09-30T22:33:47.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.025+0000 7f9abf7fe700 1 -- 172.21.15.16:0/2954949763 learned_addr learned my addr 172.21.15.16:0/2954949763 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:47.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.025+0000 7f9abffff700 1 --2- 172.21.15.16:0/2954949763 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9ac01305f0 0x7f9ac011c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:47.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.025+0000 7f9abeffd700 1 --2- 172.21.15.16:0/2954949763 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9ac006ff90 0x7f9ac011be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:47.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.025+0000 7f9abf7fe700 1 -- 172.21.15.16:0/2954949763 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9ac01305f0 msgr2=0x7f9ac011c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:47.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.025+0000 7f9abf7fe700 1 --2- 172.21.15.16:0/2954949763 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9ac01305f0 0x7f9ac011c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:47.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.025+0000 7f9abf7fe700 1 -- 172.21.15.16:0/2954949763 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9ac006ff90 msgr2=0x7f9ac011be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:47.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.025+0000 7f9abf7fe700 1 --2- 172.21.15.16:0/2954949763 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9ac006ff90 0x7f9ac011be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:47.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.025+0000 7f9abf7fe700 1 -- 172.21.15.16:0/2954949763 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab400b600 con 0x7f9ac006f660 2023-09-30T22:33:47.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.026+0000 7f9abf7fe700 1 --2- 172.21.15.16:0/2954949763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ac006f660 0x7f9ac011b8d0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f9ab000b530 tx=0x7f9ab000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:47.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.026+0000 7f9aa7fff700 1 -- 172.21.15.16:0/2954949763 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab0011830 con 0x7f9ac006f660 2023-09-30T22:33:47.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.026+0000 7f9aa7fff700 1 -- 172.21.15.16:0/2954949763 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ab0011990 con 0x7f9ac006f660 2023-09-30T22:33:47.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.026+0000 7f9aa7fff700 1 -- 172.21.15.16:0/2954949763 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ab0011d20 con 0x7f9ac006f660 2023-09-30T22:33:47.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.026+0000 7f9ac5d02700 1 -- 172.21.15.16:0/2954949763 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ac011eb40 con 0x7f9ac006f660 2023-09-30T22:33:47.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.026+0000 7f9ac5d02700 1 -- 172.21.15.16:0/2954949763 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ac011f0d0 con 0x7f9ac006f660 2023-09-30T22:33:47.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.027+0000 7f9aa7fff700 1 -- 172.21.15.16:0/2954949763 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9ab001a400 con 0x7f9ac006f660 2023-09-30T22:33:47.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.027+0000 7f9aa7fff700 1 -- 172.21.15.16:0/2954949763 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f9ab0040cc0 con 0x7f9ac006f660 2023-09-30T22:33:47.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.027+0000 7f9ac5d02700 1 -- 172.21.15.16:0/2954949763 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ac0074ea0 con 0x7f9ac006f660 2023-09-30T22:33:47.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.033+0000 7f9aa7fff700 1 -- 172.21.15.16:0/2954949763 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9ab0015070 con 0x7f9ac006f660 2023-09-30T22:33:47.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.072+0000 7fa054c10700 1 Processor -- start 2023-09-30T22:33:47.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.072+0000 7fa054c10700 1 -- start start 2023-09-30T22:33:47.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.073+0000 7fa054c10700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa05009d6f0 0x7fa05009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:47.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.073+0000 7fa054c10700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa05009e410 0x7fa050134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:47.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.073+0000 7fa054c10700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa05009b120 con 0x7fa05009e000 2023-09-30T22:33:47.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.073+0000 7fa054c10700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa05006b460 con 0x7fa05009e410 2023-09-30T22:33:47.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.073+0000 7fa054c10700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa05009a7e0 con 0x7fa05009d6f0 2023-09-30T22:33:47.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.073+0000 7fa04dd9b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa05009e000 0x7fa0501339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38190/0 (socket says 172.21.15.16:38190) 2023-09-30T22:33:47.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.073+0000 7fa04dd9b700 1 -- 172.21.15.16:0/1091543940 learned_addr learned my addr 172.21.15.16:0/1091543940 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:47.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.073+0000 7fa04ed9d700 1 --2- 172.21.15.16:0/1091543940 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa05009e410 0x7fa050134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:47.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.073+0000 7fa04e59c700 1 --2- 172.21.15.16:0/1091543940 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa05009d6f0 0x7fa05009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:47.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.073+0000 7fa04d59a700 1 -- 172.21.15.16:0/1091543940 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3858087152 0 0) 0x7fa05009b120 con 0x7fa05009e000 2023-09-30T22:33:47.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.073+0000 7fa04d59a700 1 -- 172.21.15.16:0/1091543940 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0380035b0 con 0x7fa05009e000 2023-09-30T22:33:47.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.074+0000 7fa04e59c700 1 -- 172.21.15.16:0/1091543940 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa05009e410 msgr2=0x7fa050134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:47.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.074+0000 7fa04e59c700 1 --2- 172.21.15.16:0/1091543940 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa05009e410 0x7fa050134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:47.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.074+0000 7fa04e59c700 1 -- 172.21.15.16:0/1091543940 >> v1:172.21.15.16:6789/0 conn(0x7fa05009e000 legacy=0x7fa0501339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:47.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.074+0000 7fa04d59a700 1 -- 172.21.15.16:0/1091543940 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3223276237 0 0) 0x7fa0380035b0 con 0x7fa05009e000 2023-09-30T22:33:47.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.074+0000 7fa04e59c700 1 -- 172.21.15.16:0/1091543940 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa05009a500 con 0x7fa05009d6f0 2023-09-30T22:33:47.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.074+0000 7fa04ed9d700 1 --2- 172.21.15.16:0/1091543940 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa05009e410 0x7fa050134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:47.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.074+0000 7fa04e59c700 1 --2- 172.21.15.16:0/1091543940 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa05009d6f0 0x7fa05009dac0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa0400098a0 tx=0x7fa040009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a09a44a1ddb92d1c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:47.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.074+0000 7fa04d59a700 1 -- 172.21.15.16:0/1091543940 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa044001670 con 0x7fa05009d6f0 2023-09-30T22:33:47.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.074+0000 7fa04d59a700 1 -- 172.21.15.16:0/1091543940 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa040004030 con 0x7fa05009d6f0 2023-09-30T22:33:47.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.074+0000 7fa04d59a700 1 -- 172.21.15.16:0/1091543940 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0400043c0 con 0x7fa05009d6f0 2023-09-30T22:33:47.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.075+0000 7fa054c10700 1 -- 172.21.15.16:0/1091543940 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa05009d6f0 msgr2=0x7fa05009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:47.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.075+0000 7fa054c10700 1 --2- 172.21.15.16:0/1091543940 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa05009d6f0 0x7fa05009dac0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa0400098a0 tx=0x7fa040009bb0 comp rx=0 tx=0).stop 2023-09-30T22:33:47.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.075+0000 7fa054c10700 1 -- 172.21.15.16:0/1091543940 shutdown_connections 2023-09-30T22:33:47.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.075+0000 7fa054c10700 1 --2- 172.21.15.16:0/1091543940 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa05009e410 0x7fa050134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:47.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.075+0000 7fa054c10700 1 --2- 172.21.15.16:0/1091543940 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa05009d6f0 0x7fa05009dac0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:47.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.075+0000 7fa054c10700 1 -- 172.21.15.16:0/1091543940 >> 172.21.15.16:0/1091543940 conn(0x7fa050066f50 msgr2=0x7fa05009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:47.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.075+0000 7fa054c10700 1 -- 172.21.15.16:0/1091543940 shutdown_connections 2023-09-30T22:33:47.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.075+0000 7fa054c10700 1 -- 172.21.15.16:0/1091543940 wait complete. 2023-09-30T22:33:47.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.077+0000 7fa054c10700 1 Processor -- start 2023-09-30T22:33:47.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.077+0000 7fa054c10700 1 -- start start 2023-09-30T22:33:47.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.078+0000 7fa054c10700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa05009d6f0 0x7fa050122330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:47.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.078+0000 7fa054c10700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa05009e000 0x7fa050122870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:47.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.078+0000 7fa054c10700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa05009e410 0x7fa050122db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:47.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.078+0000 7fa054c10700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa050123800 con 0x7fa05009d6f0 2023-09-30T22:33:47.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.078+0000 7fa054c10700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa050133700 con 0x7fa05009e000 2023-09-30T22:33:47.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.078+0000 7fa054c10700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa050124dd0 con 0x7fa05009e410 2023-09-30T22:33:47.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.078+0000 7fa04e59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa05009d6f0 0x7fa050122330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:47.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.078+0000 7fa04e59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa05009d6f0 0x7fa050122330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56292/0 (socket says 172.21.15.16:56292) 2023-09-30T22:33:47.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.078+0000 7fa04e59c700 1 -- 172.21.15.16:0/2067007501 learned_addr learned my addr 172.21.15.16:0/2067007501 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:47.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.078+0000 7fa04ed9d700 1 --2- 172.21.15.16:0/2067007501 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa05009e410 0x7fa050122db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:47.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.078+0000 7fa04dd9b700 1 --2- 172.21.15.16:0/2067007501 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa05009e000 0x7fa050122870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:47.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.078+0000 7fa04e59c700 1 -- 172.21.15.16:0/2067007501 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa05009e410 msgr2=0x7fa050122db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:47.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.078+0000 7fa04e59c700 1 --2- 172.21.15.16:0/2067007501 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa05009e410 0x7fa050122db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:47.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.079+0000 7fa04e59c700 1 -- 172.21.15.16:0/2067007501 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa05009e000 msgr2=0x7fa050122870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:47.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.079+0000 7fa04e59c700 1 --2- 172.21.15.16:0/2067007501 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa05009e000 0x7fa050122870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:47.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.079+0000 7fa04e59c700 1 -- 172.21.15.16:0/2067007501 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa048009a70 con 0x7fa05009d6f0 2023-09-30T22:33:47.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.079+0000 7fa04dd9b700 1 --2- 172.21.15.16:0/2067007501 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa05009e000 0x7fa050122870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:33:47.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.079+0000 7fa04e59c700 1 --2- 172.21.15.16:0/2067007501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa05009d6f0 0x7fa050122330 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fa05009a970 tx=0x7fa040004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:47.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.079+0000 7fa03effd700 1 -- 172.21.15.16:0/2067007501 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa040003d20 con 0x7fa05009d6f0 2023-09-30T22:33:47.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.079+0000 7fa03effd700 1 -- 172.21.15.16:0/2067007501 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa04001e070 con 0x7fa05009d6f0 2023-09-30T22:33:47.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.079+0000 7fa03effd700 1 -- 172.21.15.16:0/2067007501 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa040011850 con 0x7fa05009d6f0 2023-09-30T22:33:47.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.079+0000 7fa054c10700 1 -- 172.21.15.16:0/2067007501 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0400094a0 con 0x7fa05009d6f0 2023-09-30T22:33:47.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.079+0000 7fa054c10700 1 -- 172.21.15.16:0/2067007501 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0501255e0 con 0x7fa05009d6f0 2023-09-30T22:33:47.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.080+0000 7fa03effd700 1 -- 172.21.15.16:0/2067007501 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa0400119b0 con 0x7fa05009d6f0 2023-09-30T22:33:47.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.081+0000 7fa03effd700 1 -- 172.21.15.16:0/2067007501 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fa040042780 con 0x7fa05009d6f0 2023-09-30T22:33:47.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.081+0000 7fa054c10700 1 -- 172.21.15.16:0/2067007501 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa050125190 con 0x7fa05009d6f0 2023-09-30T22:33:47.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.090+0000 7fa03effd700 1 -- 172.21.15.16:0/2067007501 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa040046060 con 0x7fa05009d6f0 2023-09-30T22:33:47.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.115+0000 7f11ab9f5700 1 Processor -- start 2023-09-30T22:33:47.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.116+0000 7f11ab9f5700 1 -- start start 2023-09-30T22:33:47.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.116+0000 7f11ab9f5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f11a406a770 0x7f11a406ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:47.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.116+0000 7f11ab9f5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f11a4131d10 0x7f11a4139860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:47.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.116+0000 7f11ab9f5700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11a409ece0 con 0x7f11a406b080 2023-09-30T22:33:47.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.116+0000 7f11ab9f5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11a409e760 con 0x7f11a4131d10 2023-09-30T22:33:47.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.116+0000 7f11ab9f5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f11a409e040 con 0x7f11a406a770 2023-09-30T22:33:47.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.117+0000 7f11a9f92700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f11a4131d10 0x7f11a4139860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:47.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.117+0000 7f11a8f90700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f11a406b080 0x7f11a4131600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38198/0 (socket says 172.21.15.16:38198) 2023-09-30T22:33:47.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.117+0000 7f11a8f90700 1 -- 172.21.15.16:0/367530274 learned_addr learned my addr 172.21.15.16:0/367530274 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:47.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.117+0000 7f11a9791700 1 --2- 172.21.15.16:0/367530274 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f11a406a770 0x7f11a406ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:47.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.117+0000 7f11a9f92700 1 -- 172.21.15.16:0/367530274 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f11a406a770 msgr2=0x7f11a406ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:47.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.117+0000 7f119bfff700 1 -- 172.21.15.16:0/367530274 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1410969920 0 0) 0x7f11a409ece0 con 0x7f11a406b080 2023-09-30T22:33:47.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.117+0000 7f11a9f92700 1 --2- 172.21.15.16:0/367530274 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f11a406a770 0x7f11a406ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:47.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.117+0000 7f11a9f92700 1 -- 172.21.15.16:0/367530274 >> v1:172.21.15.16:6789/0 conn(0x7f11a406b080 legacy=0x7f11a4131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:47.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.117+0000 7f11a9f92700 1 -- 172.21.15.16:0/367530274 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a409f630 con 0x7f11a4131d10 2023-09-30T22:33:47.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.117+0000 7f11a9f92700 1 --2- 172.21.15.16:0/367530274 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f11a4131d10 0x7f11a4139860 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f119c009dc0 tx=0x7f119c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3203f3ab7ad5e8ad server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:47.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.117+0000 7f119bfff700 1 -- 172.21.15.16:0/367530274 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119c00e070 con 0x7f11a4131d10 2023-09-30T22:33:47.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.117+0000 7f119bfff700 1 -- 172.21.15.16:0/367530274 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f119c00bb20 con 0x7f11a4131d10 2023-09-30T22:33:47.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.118+0000 7f119bfff700 1 -- 172.21.15.16:0/367530274 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f119c012430 con 0x7f11a4131d10 2023-09-30T22:33:47.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.118+0000 7f11ab9f5700 1 -- 172.21.15.16:0/367530274 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f11a4131d10 msgr2=0x7f11a4139860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:47.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.118+0000 7f11ab9f5700 1 --2- 172.21.15.16:0/367530274 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f11a4131d10 0x7f11a4139860 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f119c009dc0 tx=0x7f119c00b330 comp rx=0 tx=0).stop 2023-09-30T22:33:47.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.118+0000 7f11ab9f5700 1 -- 172.21.15.16:0/367530274 shutdown_connections 2023-09-30T22:33:47.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.118+0000 7f11ab9f5700 1 --2- 172.21.15.16:0/367530274 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f11a4131d10 0x7f11a4139860 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:47.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.118+0000 7f11ab9f5700 1 --2- 172.21.15.16:0/367530274 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f11a406a770 0x7f11a406ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:47.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.118+0000 7f11ab9f5700 1 -- 172.21.15.16:0/367530274 >> 172.21.15.16:0/367530274 conn(0x7f11a4066f50 msgr2=0x7f11a409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:47.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.118+0000 7f11ab9f5700 1 -- 172.21.15.16:0/367530274 shutdown_connections 2023-09-30T22:33:47.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.118+0000 7f11ab9f5700 1 -- 172.21.15.16:0/367530274 wait complete. 2023-09-30T22:33:47.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.119+0000 7f11ab9f5700 1 Processor -- start 2023-09-30T22:33:47.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.120+0000 7f11ab9f5700 1 -- start start 2023-09-30T22:33:47.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.120+0000 7f11ab9f5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f11a406a770 0x7f11a409ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:47.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.120+0000 7f11ab9f5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11a406b080 0x7f11a409b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:47.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.120+0000 7f11ab9f5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f11a4131d10 0x7f11a41c8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:47.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.120+0000 7f11ab9f5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a4131670 con 0x7f11a406b080 2023-09-30T22:33:47.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.120+0000 7f11ab9f5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11a4125c50 con 0x7f11a4131d10 2023-09-30T22:33:47.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.120+0000 7f11ab9f5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f11a4124f50 con 0x7f11a406a770 2023-09-30T22:33:47.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.120+0000 7f11a9791700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f11a406a770 0x7f11a409ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:47.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.120+0000 7f11a9791700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f11a406a770 0x7f11a409ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:47752/0 (socket says 172.21.15.16:47752) 2023-09-30T22:33:47.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.120+0000 7f11a9791700 1 -- 172.21.15.16:0/2843638221 learned_addr learned my addr 172.21.15.16:0/2843638221 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:47.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.120+0000 7f11a9f92700 1 --2- 172.21.15.16:0/2843638221 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f11a4131d10 0x7f11a41c8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:47.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.121+0000 7f11a9791700 1 -- 172.21.15.16:0/2843638221 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f11a4131d10 msgr2=0x7f11a41c8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:47.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.121+0000 7f11a9791700 1 --2- 172.21.15.16:0/2843638221 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f11a4131d10 0x7f11a41c8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:47.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.121+0000 7f11a9791700 1 -- 172.21.15.16:0/2843638221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11a406b080 msgr2=0x7f11a409b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:47.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.121+0000 7f11a8f90700 1 --2- 172.21.15.16:0/2843638221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11a406b080 0x7f11a409b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:47.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.121+0000 7f11a9791700 1 --2- 172.21.15.16:0/2843638221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11a406b080 0x7f11a409b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:47.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.121+0000 7f11a9791700 1 -- 172.21.15.16:0/2843638221 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f119c009a70 con 0x7f11a406a770 2023-09-30T22:33:47.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.121+0000 7f11a9f92700 1 --2- 172.21.15.16:0/2843638221 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f11a4131d10 0x7f11a41c8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:47.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.121+0000 7f11a8f90700 1 --2- 172.21.15.16:0/2843638221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11a406b080 0x7f11a409b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:33:47.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.121+0000 7f11a9791700 1 --2- 172.21.15.16:0/2843638221 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f11a406a770 0x7f11a409ad20 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f11a000e9b0 tx=0x7f11a000ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:47.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.121+0000 7f1199ffb700 1 -- 172.21.15.16:0/2843638221 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a0009710 con 0x7f11a406a770 2023-09-30T22:33:47.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.121+0000 7f1199ffb700 1 -- 172.21.15.16:0/2843638221 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11a000c850 con 0x7f11a406a770 2023-09-30T22:33:47.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.121+0000 7f1199ffb700 1 -- 172.21.15.16:0/2843638221 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a000cbc0 con 0x7f11a406a770 2023-09-30T22:33:47.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.121+0000 7f11ab9f5700 1 -- 172.21.15.16:0/2843638221 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11a409b7a0 con 0x7f11a406a770 2023-09-30T22:33:47.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.122+0000 7f11ab9f5700 1 -- 172.21.15.16:0/2843638221 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11a409bf80 con 0x7f11a406a770 2023-09-30T22:33:47.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.123+0000 7f11ab9f5700 1 -- 172.21.15.16:0/2843638221 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11a4074ea0 con 0x7f11a406a770 2023-09-30T22:33:47.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.123+0000 7f1199ffb700 1 -- 172.21.15.16:0/2843638221 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f11a000cd20 con 0x7f11a406a770 2023-09-30T22:33:47.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.123+0000 7f1199ffb700 1 -- 172.21.15.16:0/2843638221 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f11a003f780 con 0x7f11a406a770 2023-09-30T22:33:47.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.128+0000 7f1199ffb700 1 -- 172.21.15.16:0/2843638221 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f11a001bc10 con 0x7f11a406a770 2023-09-30T22:33:47.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.165+0000 7fe25efbb700 1 Processor -- start 2023-09-30T22:33:47.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.165+0000 7fe25efbb700 1 -- start start 2023-09-30T22:33:47.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.165+0000 7fe25efbb700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe25806a770 0x7fe25809d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:47.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.165+0000 7fe25efbb700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe25809dc30 0x7fe25809e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:47.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.165+0000 7fe25efbb700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe25813c1e0 0x7fe25813e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:47.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.165+0000 7fe25efbb700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe25806ab40 con 0x7fe25813c1e0 2023-09-30T22:33:47.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.165+0000 7fe25efbb700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe25809ebc0 con 0x7fe25806a770 2023-09-30T22:33:47.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.165+0000 7fe25efbb700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe25809a810 con 0x7fe25809dc30 2023-09-30T22:33:47.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.166+0000 7fe257fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe25809dc30 0x7fe25809e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:47.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.166+0000 7fe25cd57700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe25806a770 0x7fe25809d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:47.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.166+0000 7fe257fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe25809dc30 0x7fe25809e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:47762/0 (socket says 172.21.15.16:47762) 2023-09-30T22:33:47.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.166+0000 7fe257fff700 1 -- 172.21.15.16:0/4253261918 learned_addr learned my addr 172.21.15.16:0/4253261918 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:47.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.166+0000 7fe25d558700 1 --2- 172.21.15.16:0/4253261918 >> v2:172.21.15.16:3300/0 conn(0x7fe25813c1e0 0x7fe25813e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:47.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.166+0000 7fe257fff700 1 -- 172.21.15.16:0/4253261918 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe25806a770 msgr2=0x7fe25809d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:47.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.166+0000 7fe257fff700 1 --2- 172.21.15.16:0/4253261918 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe25806a770 0x7fe25809d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:47.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.166+0000 7fe257fff700 1 -- 172.21.15.16:0/4253261918 >> v2:172.21.15.16:3300/0 conn(0x7fe25813c1e0 msgr2=0x7fe25813e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:47.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.166+0000 7fe257fff700 1 --2- 172.21.15.16:0/4253261918 >> v2:172.21.15.16:3300/0 conn(0x7fe25813c1e0 0x7fe25813e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:47.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.166+0000 7fe257fff700 1 -- 172.21.15.16:0/4253261918 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe25809e920 con 0x7fe25809dc30 2023-09-30T22:33:47.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.166+0000 7fe25d558700 1 --2- 172.21.15.16:0/4253261918 >> v2:172.21.15.16:3300/0 conn(0x7fe25813c1e0 0x7fe25813e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:33:47.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.166+0000 7fe257fff700 1 --2- 172.21.15.16:0/4253261918 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe25809dc30 0x7fe25809e080 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fe244009920 tx=0x7fe244009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2940030bae10716f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:47.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.166+0000 7fe2577fe700 1 -- 172.21.15.16:0/4253261918 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24400ba20 con 0x7fe25809dc30 2023-09-30T22:33:47.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.167+0000 7fe2577fe700 1 -- 172.21.15.16:0/4253261918 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe24400bb80 con 0x7fe25809dc30 2023-09-30T22:33:47.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.167+0000 7fe2577fe700 1 -- 172.21.15.16:0/4253261918 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe244009e70 con 0x7fe25809dc30 2023-09-30T22:33:47.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.167+0000 7fe25efbb700 1 -- 172.21.15.16:0/4253261918 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe25809dc30 msgr2=0x7fe25809e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:47.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.167+0000 7fe25efbb700 1 --2- 172.21.15.16:0/4253261918 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe25809dc30 0x7fe25809e080 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fe244009920 tx=0x7fe244009c30 comp rx=0 tx=0).stop 2023-09-30T22:33:47.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.167+0000 7fe25efbb700 1 -- 172.21.15.16:0/4253261918 shutdown_connections 2023-09-30T22:33:47.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.167+0000 7fe25efbb700 1 --2- 172.21.15.16:0/4253261918 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe25806a770 0x7fe25809d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:47.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.167+0000 7fe25efbb700 1 --2- 172.21.15.16:0/4253261918 >> v2:172.21.15.16:3300/0 conn(0x7fe25813c1e0 0x7fe25813e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:47.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.167+0000 7fe25efbb700 1 --2- 172.21.15.16:0/4253261918 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe25809dc30 0x7fe25809e080 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:47.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.167+0000 7fe25efbb700 1 -- 172.21.15.16:0/4253261918 >> 172.21.15.16:0/4253261918 conn(0x7fe258066f50 msgr2=0x7fe25809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:47.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.167+0000 7fe25efbb700 1 -- 172.21.15.16:0/4253261918 shutdown_connections 2023-09-30T22:33:47.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.167+0000 7fe25efbb700 1 -- 172.21.15.16:0/4253261918 wait complete. 2023-09-30T22:33:47.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.169+0000 7fe25efbb700 1 Processor -- start 2023-09-30T22:33:47.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.169+0000 7fe25efbb700 1 -- start start 2023-09-30T22:33:47.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.170+0000 7fe25efbb700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe25806a770 0x7fe258130070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:47.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.170+0000 7fe25efbb700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe25809dc30 0x7fe25812dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:47.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.170+0000 7fe25efbb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe25813c1e0 0x7fe25812e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:47.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.170+0000 7fe25efbb700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe258131c70 con 0x7fe25813c1e0 2023-09-30T22:33:47.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.170+0000 7fe25efbb700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2581305b0 con 0x7fe25809dc30 2023-09-30T22:33:47.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.170+0000 7fe25efbb700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe25812f4b0 con 0x7fe25806a770 2023-09-30T22:33:47.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.170+0000 7fe25cd57700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe25806a770 0x7fe258130070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:47.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.170+0000 7fe25cd57700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe25806a770 0x7fe258130070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:47772/0 (socket says 172.21.15.16:47772) 2023-09-30T22:33:47.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.170+0000 7fe25cd57700 1 -- 172.21.15.16:0/2258585129 learned_addr learned my addr 172.21.15.16:0/2258585129 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:47.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.170+0000 7fe257fff700 1 --2- 172.21.15.16:0/2258585129 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe25809dc30 0x7fe25812dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:47.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.170+0000 7fe25d558700 1 --2- 172.21.15.16:0/2258585129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe25813c1e0 0x7fe25812e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:47.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.170+0000 7fe25cd57700 1 -- 172.21.15.16:0/2258585129 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe25809dc30 msgr2=0x7fe25812dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:47.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.170+0000 7fe25cd57700 1 --2- 172.21.15.16:0/2258585129 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe25809dc30 0x7fe25812dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:47.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.170+0000 7fe25cd57700 1 -- 172.21.15.16:0/2258585129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe25813c1e0 msgr2=0x7fe25812e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:47.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.170+0000 7fe25cd57700 1 --2- 172.21.15.16:0/2258585129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe25813c1e0 0x7fe25812e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:47.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.170+0000 7fe25cd57700 1 -- 172.21.15.16:0/2258585129 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe244009550 con 0x7fe25806a770 2023-09-30T22:33:47.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.171+0000 7fe25cd57700 1 --2- 172.21.15.16:0/2258585129 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe25806a770 0x7fe258130070 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fe24c00e810 tx=0x7fe24c00eb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:47.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.171+0000 7fe2557fa700 1 -- 172.21.15.16:0/2258585129 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24c00c070 con 0x7fe25806a770 2023-09-30T22:33:47.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.171+0000 7fe2557fa700 1 -- 172.21.15.16:0/2258585129 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe24c013610 con 0x7fe25806a770 2023-09-30T22:33:47.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.171+0000 7fe25efbb700 1 -- 172.21.15.16:0/2258585129 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2580a37d0 con 0x7fe25806a770 2023-09-30T22:33:47.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.171+0000 7fe2557fa700 1 -- 172.21.15.16:0/2258585129 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24c0139a0 con 0x7fe25806a770 2023-09-30T22:33:47.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.171+0000 7fe25efbb700 1 -- 172.21.15.16:0/2258585129 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe258130cc0 con 0x7fe25806a770 2023-09-30T22:33:47.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.172+0000 7fe25efbb700 1 -- 172.21.15.16:0/2258585129 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe25812fe20 con 0x7fe25806a770 2023-09-30T22:33:47.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.173+0000 7fe2557fa700 1 -- 172.21.15.16:0/2258585129 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe24c013b00 con 0x7fe25806a770 2023-09-30T22:33:47.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.173+0000 7fe2557fa700 1 -- 172.21.15.16:0/2258585129 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fe24c03f9f0 con 0x7fe25806a770 2023-09-30T22:33:47.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.182+0000 7fe2557fa700 1 -- 172.21.15.16:0/2258585129 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe24c00f040 con 0x7fe25806a770 2023-09-30T22:33:47.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.273+0000 7f9ac5d02700 1 -- 172.21.15.16:0/2954949763 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9ac011ecd0 con 0x7f9ac006f660 2023-09-30T22:33:47.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.273+0000 7f9aa7fff700 1 -- 172.21.15.16:0/2954949763 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9ab0015020 con 0x7f9ac006f660 2023-09-30T22:33:47.274 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:33:47.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.275+0000 7f9ac5d02700 1 -- 172.21.15.16:0/2954949763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ac006f660 msgr2=0x7f9ac011b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:47.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.275+0000 7f9ac5d02700 1 --2- 172.21.15.16:0/2954949763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ac006f660 0x7f9ac011b8d0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f9ab000b530 tx=0x7f9ab000b840 comp rx=0 tx=0).stop 2023-09-30T22:33:47.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.275+0000 7f9ac5d02700 1 -- 172.21.15.16:0/2954949763 shutdown_connections 2023-09-30T22:33:47.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.275+0000 7f9ac5d02700 1 --2- 172.21.15.16:0/2954949763 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9ac01305f0 0x7f9ac011c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:47.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.275+0000 7f9ac5d02700 1 --2- 172.21.15.16:0/2954949763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ac006f660 0x7f9ac011b8d0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:47.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.275+0000 7f9ac5d02700 1 --2- 172.21.15.16:0/2954949763 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9ac006ff90 0x7f9ac011be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:47.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.275+0000 7f9ac5d02700 1 -- 172.21.15.16:0/2954949763 >> 172.21.15.16:0/2954949763 conn(0x7f9ac00a2aa0 msgr2=0x7f9ac006e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:47.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.276+0000 7f9ac5d02700 1 -- 172.21.15.16:0/2954949763 shutdown_connections 2023-09-30T22:33:47.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.276+0000 7f9ac5d02700 1 -- 172.21.15.16:0/2954949763 wait complete. 2023-09-30T22:33:47.295 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:33:47.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.332+0000 7fa054c10700 1 -- 172.21.15.16:0/2067007501 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa0501259c0 con 0x7fa05009d6f0 2023-09-30T22:33:47.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.332+0000 7fa03effd700 1 -- 172.21.15.16:0/2067007501 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa04001a950 con 0x7fa05009d6f0 2023-09-30T22:33:47.333 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:33:47.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.335+0000 7fa054c10700 1 -- 172.21.15.16:0/2067007501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa05009d6f0 msgr2=0x7fa050122330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:47.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.335+0000 7fa054c10700 1 --2- 172.21.15.16:0/2067007501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa05009d6f0 0x7fa050122330 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fa05009a970 tx=0x7fa040004750 comp rx=0 tx=0).stop 2023-09-30T22:33:47.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.335+0000 7fa054c10700 1 -- 172.21.15.16:0/2067007501 shutdown_connections 2023-09-30T22:33:47.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.335+0000 7fa054c10700 1 --2- 172.21.15.16:0/2067007501 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa05009e410 0x7fa050122db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:47.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.335+0000 7fa054c10700 1 --2- 172.21.15.16:0/2067007501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa05009d6f0 0x7fa050122330 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:47.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.335+0000 7fa054c10700 1 --2- 172.21.15.16:0/2067007501 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa05009e000 0x7fa050122870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:47.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.335+0000 7fa054c10700 1 -- 172.21.15.16:0/2067007501 >> 172.21.15.16:0/2067007501 conn(0x7fa050066f50 msgr2=0x7fa050133e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:47.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.335+0000 7fa054c10700 1 -- 172.21.15.16:0/2067007501 shutdown_connections 2023-09-30T22:33:47.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.335+0000 7fa054c10700 1 -- 172.21.15.16:0/2067007501 wait complete. 2023-09-30T22:33:47.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.346+0000 7f11ab9f5700 1 -- 172.21.15.16:0/2843638221 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f11a4126130 con 0x7f11a406a770 2023-09-30T22:33:47.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.347+0000 7f1199ffb700 1 -- 172.21.15.16:0/2843638221 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f11a0021020 con 0x7f11a406a770 2023-09-30T22:33:47.347 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:33:47.349 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:33:47.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.349+0000 7f11ab9f5700 1 -- 172.21.15.16:0/2843638221 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f11a406a770 msgr2=0x7f11a409ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:47.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.349+0000 7f11ab9f5700 1 --2- 172.21.15.16:0/2843638221 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f11a406a770 0x7f11a409ad20 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f11a000e9b0 tx=0x7f11a000ecc0 comp rx=0 tx=0).stop 2023-09-30T22:33:47.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.349+0000 7f11ab9f5700 1 -- 172.21.15.16:0/2843638221 shutdown_connections 2023-09-30T22:33:47.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.349+0000 7f11ab9f5700 1 --2- 172.21.15.16:0/2843638221 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f11a4131d10 0x7f11a41c8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:47.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.349+0000 7f11ab9f5700 1 --2- 172.21.15.16:0/2843638221 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f11a406a770 0x7f11a409ad20 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:47.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.349+0000 7f11ab9f5700 1 --2- 172.21.15.16:0/2843638221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f11a406b080 0x7f11a409b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:47.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.349+0000 7f11ab9f5700 1 -- 172.21.15.16:0/2843638221 >> 172.21.15.16:0/2843638221 conn(0x7f11a4066f50 msgr2=0x7f11a4070070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:47.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.349+0000 7f11ab9f5700 1 -- 172.21.15.16:0/2843638221 shutdown_connections 2023-09-30T22:33:47.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.349+0000 7f11ab9f5700 1 -- 172.21.15.16:0/2843638221 wait complete. 2023-09-30T22:33:47.363 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:33:47.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.391+0000 7fe25efbb700 1 -- 172.21.15.16:0/2258585129 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe25811cc30 con 0x7fe25806a770 2023-09-30T22:33:47.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.391+0000 7fe2557fa700 1 -- 172.21.15.16:0/2258585129 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe24c02a030 con 0x7fe25806a770 2023-09-30T22:33:47.392 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:33:47.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.393+0000 7fe25efbb700 1 -- 172.21.15.16:0/2258585129 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe25806a770 msgr2=0x7fe258130070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:47.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.393+0000 7fe25efbb700 1 --2- 172.21.15.16:0/2258585129 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe25806a770 0x7fe258130070 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fe24c00e810 tx=0x7fe24c00eb20 comp rx=0 tx=0).stop 2023-09-30T22:33:47.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.394+0000 7fe25efbb700 1 -- 172.21.15.16:0/2258585129 shutdown_connections 2023-09-30T22:33:47.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.394+0000 7fe25efbb700 1 --2- 172.21.15.16:0/2258585129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe25813c1e0 0x7fe25812e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:47.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.394+0000 7fe25efbb700 1 --2- 172.21.15.16:0/2258585129 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe25806a770 0x7fe258130070 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:47.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.394+0000 7fe25efbb700 1 --2- 172.21.15.16:0/2258585129 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe25809dc30 0x7fe25812dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:47.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.394+0000 7fe25efbb700 1 -- 172.21.15.16:0/2258585129 >> 172.21.15.16:0/2258585129 conn(0x7fe258066f50 msgr2=0x7fe25813ded0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:47.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.394+0000 7fe25efbb700 1 -- 172.21.15.16:0/2258585129 shutdown_connections 2023-09-30T22:33:47.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:47.394+0000 7fe25efbb700 1 -- 172.21.15.16:0/2258585129 wait complete. 2023-09-30T22:33:47.408 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:33:48.296 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:33:48.350 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:33:48.363 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:33:48.409 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:33:48.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.445+0000 7f8b60cd5700 1 Processor -- start 2023-09-30T22:33:48.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.445+0000 7f8b60cd5700 1 -- start start 2023-09-30T22:33:48.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.445+0000 7f8b60cd5700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8b5c12cf60 0x7f8b5c06f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:48.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.445+0000 7f8b60cd5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8b5c12f8b0 0x7f8b5c12fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:48.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.445+0000 7f8b60cd5700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b5c12cd90 con 0x7f8b5c130260 2023-09-30T22:33:48.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.445+0000 7f8b60cd5700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b5c12d330 con 0x7f8b5c12cf60 2023-09-30T22:33:48.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.445+0000 7f8b60cd5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5c1306d0 con 0x7f8b5c12f8b0 2023-09-30T22:33:48.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.446+0000 7f8b59d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8b5c12f8b0 0x7f8b5c12fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:48.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.446+0000 7f8b5ad9d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8b5c130260 0x7f8b5c136c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49998/0 (socket says 172.21.15.16:49998) 2023-09-30T22:33:48.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.446+0000 7f8b59d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8b5c12f8b0 0x7f8b5c12fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37448/0 (socket says 172.21.15.16:37448) 2023-09-30T22:33:48.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.446+0000 7f8b5ad9d700 1 -- 172.21.15.16:0/4186562818 learned_addr learned my addr 172.21.15.16:0/4186562818 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:48.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.446+0000 7f8b5a59c700 1 --2- 172.21.15.16:0/4186562818 >> v2:172.21.15.16:3300/0 conn(0x7f8b5c12cf60 0x7f8b5c06f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:48.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.446+0000 7f8b59d9b700 1 -- 172.21.15.16:0/4186562818 >> v2:172.21.15.16:3300/0 conn(0x7f8b5c12cf60 msgr2=0x7f8b5c06f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:48.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.446+0000 7f8b59d9b700 1 --2- 172.21.15.16:0/4186562818 >> v2:172.21.15.16:3300/0 conn(0x7f8b5c12cf60 0x7f8b5c06f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:48.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.446+0000 7f8b59d9b700 1 -- 172.21.15.16:0/4186562818 >> v1:172.21.15.16:6789/0 conn(0x7f8b5c130260 legacy=0x7f8b5c136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:48.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.446+0000 7f8b59d9b700 1 -- 172.21.15.16:0/4186562818 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b5c071780 con 0x7f8b5c12f8b0 2023-09-30T22:33:48.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.446+0000 7f8b59d9b700 1 --2- 172.21.15.16:0/4186562818 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8b5c12f8b0 0x7f8b5c12fd20 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f8b5000a9e0 tx=0x7f8b5000acf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2690aa74ad0b8e08 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:48.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.447+0000 7f8b5959a700 1 -- 172.21.15.16:0/4186562818 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b5000cb70 con 0x7f8b5c12f8b0 2023-09-30T22:33:48.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.447+0000 7f8b5959a700 1 -- 172.21.15.16:0/4186562818 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b5000ccd0 con 0x7f8b5c12f8b0 2023-09-30T22:33:48.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.447+0000 7f8b5959a700 1 -- 172.21.15.16:0/4186562818 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b5000f690 con 0x7f8b5c12f8b0 2023-09-30T22:33:48.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.447+0000 7f8b60cd5700 1 -- 172.21.15.16:0/4186562818 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8b5c12f8b0 msgr2=0x7f8b5c12fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:48.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.447+0000 7f8b60cd5700 1 --2- 172.21.15.16:0/4186562818 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8b5c12f8b0 0x7f8b5c12fd20 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f8b5000a9e0 tx=0x7f8b5000acf0 comp rx=0 tx=0).stop 2023-09-30T22:33:48.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.448+0000 7f8b60cd5700 1 -- 172.21.15.16:0/4186562818 shutdown_connections 2023-09-30T22:33:48.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.448+0000 7f8b60cd5700 1 --2- 172.21.15.16:0/4186562818 >> v2:172.21.15.16:3300/0 conn(0x7f8b5c12cf60 0x7f8b5c06f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:48.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.448+0000 7f8b60cd5700 1 --2- 172.21.15.16:0/4186562818 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8b5c12f8b0 0x7f8b5c12fd20 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:48.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.448+0000 7f8b60cd5700 1 -- 172.21.15.16:0/4186562818 >> 172.21.15.16:0/4186562818 conn(0x7f8b5c0a2aa0 msgr2=0x7f8b5c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:48.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.448+0000 7f8b60cd5700 1 -- 172.21.15.16:0/4186562818 shutdown_connections 2023-09-30T22:33:48.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.448+0000 7f8b60cd5700 1 -- 172.21.15.16:0/4186562818 wait complete. 2023-09-30T22:33:48.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.449+0000 7f8b60cd5700 1 Processor -- start 2023-09-30T22:33:48.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.449+0000 7f8b60cd5700 1 -- start start 2023-09-30T22:33:48.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.449+0000 7f8b60cd5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8b5c12cf60 0x7f8b5c1c5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:48.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.449+0000 7f8b60cd5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b5c12f8b0 0x7f8b5c1c57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:48.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.449+0000 7f8b60cd5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8b5c130260 0x7f8b5c1c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:48.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.449+0000 7f8b60cd5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5c09ea90 con 0x7f8b5c12f8b0 2023-09-30T22:33:48.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.449+0000 7f8b60cd5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5c129fa0 con 0x7f8b5c130260 2023-09-30T22:33:48.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.449+0000 7f8b60cd5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5c1c5d00 con 0x7f8b5c12cf60 2023-09-30T22:33:48.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.450+0000 7f8b5ad9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8b5c130260 0x7f8b5c1c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:48.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.450+0000 7f8b5a59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8b5c12cf60 0x7f8b5c1c5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:48.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.450+0000 7f8b59d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b5c12f8b0 0x7f8b5c1c57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:48.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.450+0000 7f8b5ad9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8b5c130260 0x7f8b5c1c82c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37452/0 (socket says 172.21.15.16:37452) 2023-09-30T22:33:48.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.450+0000 7f8b5ad9d700 1 -- 172.21.15.16:0/4069376502 learned_addr learned my addr 172.21.15.16:0/4069376502 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:48.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.450+0000 7f8b5ad9d700 1 -- 172.21.15.16:0/4069376502 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8b5c12cf60 msgr2=0x7f8b5c1c5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:48.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.450+0000 7f8b5ad9d700 1 --2- 172.21.15.16:0/4069376502 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8b5c12cf60 0x7f8b5c1c5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:48.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.450+0000 7f8b5ad9d700 1 -- 172.21.15.16:0/4069376502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b5c12f8b0 msgr2=0x7f8b5c1c57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:48.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.450+0000 7f8b5ad9d700 1 --2- 172.21.15.16:0/4069376502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b5c12f8b0 0x7f8b5c1c57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:48.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.450+0000 7f8b5ad9d700 1 -- 172.21.15.16:0/4069376502 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b5000a640 con 0x7f8b5c130260 2023-09-30T22:33:48.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.450+0000 7f8b59d9b700 1 --2- 172.21.15.16:0/4069376502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b5c12f8b0 0x7f8b5c1c57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:33:48.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.450+0000 7f8b5ad9d700 1 --2- 172.21.15.16:0/4069376502 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8b5c130260 0x7f8b5c1c82c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f8b5400cbc0 tx=0x7f8b5400ced0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:48.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.451+0000 7f8b4affd700 1 -- 172.21.15.16:0/4069376502 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b5400aaa0 con 0x7f8b5c130260 2023-09-30T22:33:48.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.451+0000 7f8b4affd700 1 -- 172.21.15.16:0/4069376502 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b5400ac00 con 0x7f8b5c130260 2023-09-30T22:33:48.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.451+0000 7f8b60cd5700 1 -- 172.21.15.16:0/4069376502 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b5c12da20 con 0x7f8b5c130260 2023-09-30T22:33:48.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.451+0000 7f8b60cd5700 1 -- 172.21.15.16:0/4069376502 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b5c09cd60 con 0x7f8b5c130260 2023-09-30T22:33:48.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.452+0000 7f8b4affd700 1 -- 172.21.15.16:0/4069376502 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b54012400 con 0x7f8b5c130260 2023-09-30T22:33:48.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.452+0000 7f8b4affd700 1 -- 172.21.15.16:0/4069376502 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8b5401b3b0 con 0x7f8b5c130260 2023-09-30T22:33:48.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.452+0000 7f8b4affd700 1 -- 172.21.15.16:0/4069376502 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f8b5403f4a0 con 0x7f8b5c130260 2023-09-30T22:33:48.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.453+0000 7f8b60cd5700 1 -- 172.21.15.16:0/4069376502 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b5c074ea0 con 0x7f8b5c130260 2023-09-30T22:33:48.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.459+0000 7f8b4affd700 1 -- 172.21.15.16:0/4069376502 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8b540192c0 con 0x7f8b5c130260 2023-09-30T22:33:48.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.496+0000 7f704dfd6700 1 Processor -- start 2023-09-30T22:33:48.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.497+0000 7f704dfd6700 1 -- start start 2023-09-30T22:33:48.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.497+0000 7f704dfd6700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f704809d630 0x7f70480a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:48.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.497+0000 7f704dfd6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7048138d80 0x7f704813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:48.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.497+0000 7f704dfd6700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70480a2a70 con 0x7f70480a0a30 2023-09-30T22:33:48.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.497+0000 7f704dfd6700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70480a01d0 con 0x7f704809d630 2023-09-30T22:33:48.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.497+0000 7f704dfd6700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f704809e2b0 con 0x7f7048138d80 2023-09-30T22:33:48.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.497+0000 7f7046ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f704809d630 0x7f70480a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:48.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.497+0000 7f7046ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f704809d630 0x7f70480a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43430/0 (socket says 172.21.15.16:43430) 2023-09-30T22:33:48.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.497+0000 7f7046ffd700 1 -- 172.21.15.16:0/2793869013 learned_addr learned my addr 172.21.15.16:0/2793869013 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:48.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.497+0000 7f7047fff700 1 --2- 172.21.15.16:0/2793869013 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7048138d80 0x7f704813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:48.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.498+0000 7f7047fff700 1 -- 172.21.15.16:0/2793869013 >> v2:172.21.15.16:3300/0 conn(0x7f704809d630 msgr2=0x7f70480a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:48.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.498+0000 7f7047fff700 1 --2- 172.21.15.16:0/2793869013 >> v2:172.21.15.16:3300/0 conn(0x7f704809d630 0x7f70480a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:48.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.498+0000 7f70467fc700 1 -- 172.21.15.16:0/2793869013 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 526594487 0 0) 0x7f70480a2a70 con 0x7f70480a0a30 2023-09-30T22:33:48.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.498+0000 7f7047fff700 1 -- 172.21.15.16:0/2793869013 >> v1:172.21.15.16:6789/0 conn(0x7f70480a0a30 legacy=0x7f70480a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:48.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.498+0000 7f7047fff700 1 -- 172.21.15.16:0/2793869013 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f704809dc50 con 0x7f7048138d80 2023-09-30T22:33:48.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.498+0000 7f7047fff700 1 --2- 172.21.15.16:0/2793869013 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7048138d80 0x7f704813b160 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f7040009d90 tx=0x7f704000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d33c9dff1465a5b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:48.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.498+0000 7f70467fc700 1 -- 172.21.15.16:0/2793869013 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704000e070 con 0x7f7048138d80 2023-09-30T22:33:48.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.498+0000 7f70467fc700 1 -- 172.21.15.16:0/2793869013 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f704000bb20 con 0x7f7048138d80 2023-09-30T22:33:48.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.498+0000 7f70467fc700 1 -- 172.21.15.16:0/2793869013 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7040012430 con 0x7f7048138d80 2023-09-30T22:33:48.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.499+0000 7f704dfd6700 1 -- 172.21.15.16:0/2793869013 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7048138d80 msgr2=0x7f704813b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:48.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.499+0000 7f704dfd6700 1 --2- 172.21.15.16:0/2793869013 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7048138d80 0x7f704813b160 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f7040009d90 tx=0x7f704000b330 comp rx=0 tx=0).stop 2023-09-30T22:33:48.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.499+0000 7f704dfd6700 1 -- 172.21.15.16:0/2793869013 shutdown_connections 2023-09-30T22:33:48.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.499+0000 7f704dfd6700 1 --2- 172.21.15.16:0/2793869013 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7048138d80 0x7f704813b160 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:48.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.499+0000 7f704dfd6700 1 --2- 172.21.15.16:0/2793869013 >> v2:172.21.15.16:3300/0 conn(0x7f704809d630 0x7f70480a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:48.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.499+0000 7f704dfd6700 1 -- 172.21.15.16:0/2793869013 >> 172.21.15.16:0/2793869013 conn(0x7f70480a3050 msgr2=0x7f70480a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:48.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.499+0000 7f704dfd6700 1 -- 172.21.15.16:0/2793869013 shutdown_connections 2023-09-30T22:33:48.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.499+0000 7f704dfd6700 1 -- 172.21.15.16:0/2793869013 wait complete. 2023-09-30T22:33:48.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.500+0000 7f704dfd6700 1 Processor -- start 2023-09-30T22:33:48.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.501+0000 7f704dfd6700 1 -- start start 2023-09-30T22:33:48.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.501+0000 7f704dfd6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f704809d630 0x7f704812c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:48.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.501+0000 7f704dfd6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f70480a0a30 0x7f704812ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:48.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.501+0000 7f704dfd6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7048138d80 0x7f704812cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:48.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.501+0000 7f704dfd6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f704812e070 con 0x7f7048138d80 2023-09-30T22:33:48.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.501+0000 7f704dfd6700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7048138980 con 0x7f70480a0a30 2023-09-30T22:33:48.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.501+0000 7f704dfd6700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f704813ac50 con 0x7f704809d630 2023-09-30T22:33:48.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.501+0000 7f70477fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f704809d630 0x7f704812c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:48.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.501+0000 7f7046ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f70480a0a30 0x7f704812ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:48.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.501+0000 7f70477fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f704809d630 0x7f704812c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37476/0 (socket says 172.21.15.16:37476) 2023-09-30T22:33:48.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.501+0000 7f7046ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f70480a0a30 0x7f704812ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37454/0 (socket says 172.21.15.16:37454) 2023-09-30T22:33:48.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.501+0000 7f70477fe700 1 -- 172.21.15.16:0/916816468 learned_addr learned my addr 172.21.15.16:0/916816468 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:48.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.502+0000 7f7047fff700 1 --2- 172.21.15.16:0/916816468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7048138d80 0x7f704812cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:48.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.502+0000 7f7046ffd700 1 -- 172.21.15.16:0/916816468 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f704809d630 msgr2=0x7f704812c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:48.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.502+0000 7f7046ffd700 1 --2- 172.21.15.16:0/916816468 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f704809d630 0x7f704812c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:48.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.502+0000 7f7046ffd700 1 -- 172.21.15.16:0/916816468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7048138d80 msgr2=0x7f704812cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:48.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.502+0000 7f7046ffd700 1 --2- 172.21.15.16:0/916816468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7048138d80 0x7f704812cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:48.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.502+0000 7f7046ffd700 1 -- 172.21.15.16:0/916816468 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7040009a70 con 0x7f70480a0a30 2023-09-30T22:33:48.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.502+0000 7f7046ffd700 1 --2- 172.21.15.16:0/916816468 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f70480a0a30 0x7f704812ca70 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f703c00e8e0 tx=0x7f703c00ebf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:48.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.502+0000 7f702ffff700 1 -- 172.21.15.16:0/916816468 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703c017d80 con 0x7f70480a0a30 2023-09-30T22:33:48.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.502+0000 7f702ffff700 1 -- 172.21.15.16:0/916816468 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f703c01f440 con 0x7f70480a0a30 2023-09-30T22:33:48.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.502+0000 7f704dfd6700 1 -- 172.21.15.16:0/916816468 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f704812ff20 con 0x7f70480a0a30 2023-09-30T22:33:48.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.502+0000 7f704dfd6700 1 -- 172.21.15.16:0/916816468 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70480a0410 con 0x7f70480a0a30 2023-09-30T22:33:48.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.503+0000 7f702ffff700 1 -- 172.21.15.16:0/916816468 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703c01f7d0 con 0x7f70480a0a30 2023-09-30T22:33:48.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.504+0000 7f702ffff700 1 -- 172.21.15.16:0/916816468 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f703c01f930 con 0x7f70480a0a30 2023-09-30T22:33:48.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.504+0000 7f702ffff700 1 -- 172.21.15.16:0/916816468 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f703c047890 con 0x7f70480a0a30 2023-09-30T22:33:48.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.504+0000 7f704dfd6700 1 -- 172.21.15.16:0/916816468 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70280052e0 con 0x7f70480a0a30 2023-09-30T22:33:48.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.514+0000 7f702ffff700 1 -- 172.21.15.16:0/916816468 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f703c0283b0 con 0x7f70480a0a30 2023-09-30T22:33:48.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.546+0000 7f72294d8700 1 Processor -- start 2023-09-30T22:33:48.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.546+0000 7f72294d8700 1 -- start start 2023-09-30T22:33:48.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.547+0000 7f72294d8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72241338a0 0x7f7224133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:48.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.547+0000 7f72294d8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f722409de50 0x7f722409e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:48.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.547+0000 7f72294d8700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f722409e760 0x7f722413a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:48.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.547+0000 7f72294d8700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f722406fef0 con 0x7f722409e760 2023-09-30T22:33:48.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.547+0000 7f72294d8700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f722406ef30 con 0x7f722409de50 2023-09-30T22:33:48.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.547+0000 7f72294d8700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f72241334b0 con 0x7f72241338a0 2023-09-30T22:33:48.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.547+0000 7f7222ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72241338a0 0x7f7224133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:48.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.547+0000 7f7222ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72241338a0 0x7f7224133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37490/0 (socket says 172.21.15.16:37490) 2023-09-30T22:33:48.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.547+0000 7f7222ffd700 1 -- 172.21.15.16:0/1554154320 learned_addr learned my addr 172.21.15.16:0/1554154320 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:48.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.547+0000 7f72237fe700 1 --2- 172.21.15.16:0/1554154320 >> v2:172.21.15.16:3300/0 conn(0x7f722409e760 0x7f722413a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:48.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.547+0000 7f72227fc700 1 --2- 172.21.15.16:0/1554154320 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f722409de50 0x7f722409e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:48.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.547+0000 7f7222ffd700 1 -- 172.21.15.16:0/1554154320 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f722409de50 msgr2=0x7f722409e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:48.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.547+0000 7f7222ffd700 1 --2- 172.21.15.16:0/1554154320 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f722409de50 0x7f722409e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:48.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.547+0000 7f7222ffd700 1 -- 172.21.15.16:0/1554154320 >> v2:172.21.15.16:3300/0 conn(0x7f722409e760 msgr2=0x7f722413a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:48.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.547+0000 7f7222ffd700 1 --2- 172.21.15.16:0/1554154320 >> v2:172.21.15.16:3300/0 conn(0x7f722409e760 0x7f722413a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:48.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.547+0000 7f7222ffd700 1 -- 172.21.15.16:0/1554154320 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f722406fc30 con 0x7f72241338a0 2023-09-30T22:33:48.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.548+0000 7f7222ffd700 1 --2- 172.21.15.16:0/1554154320 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72241338a0 0x7f7224133c70 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7214009920 tx=0x7f7214009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34b72d4af921320d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:48.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.548+0000 7f7221ffb700 1 -- 172.21.15.16:0/1554154320 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7214011830 con 0x7f72241338a0 2023-09-30T22:33:48.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.548+0000 7f7221ffb700 1 -- 172.21.15.16:0/1554154320 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7214011990 con 0x7f72241338a0 2023-09-30T22:33:48.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.548+0000 7f7221ffb700 1 -- 172.21.15.16:0/1554154320 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7214011d20 con 0x7f72241338a0 2023-09-30T22:33:48.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.549+0000 7f72294d8700 1 -- 172.21.15.16:0/1554154320 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72241338a0 msgr2=0x7f7224133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:48.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.549+0000 7f72294d8700 1 --2- 172.21.15.16:0/1554154320 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72241338a0 0x7f7224133c70 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7214009920 tx=0x7f7214009c30 comp rx=0 tx=0).stop 2023-09-30T22:33:48.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.549+0000 7f72294d8700 1 -- 172.21.15.16:0/1554154320 shutdown_connections 2023-09-30T22:33:48.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.549+0000 7f72294d8700 1 --2- 172.21.15.16:0/1554154320 >> v2:172.21.15.16:3300/0 conn(0x7f722409e760 0x7f722413a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:48.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.549+0000 7f72294d8700 1 --2- 172.21.15.16:0/1554154320 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72241338a0 0x7f7224133c70 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:48.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.549+0000 7f72294d8700 1 --2- 172.21.15.16:0/1554154320 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f722409de50 0x7f722409e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:48.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.549+0000 7f72294d8700 1 -- 172.21.15.16:0/1554154320 >> 172.21.15.16:0/1554154320 conn(0x7f72240a2aa0 msgr2=0x7f72240a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:48.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.549+0000 7f72294d8700 1 -- 172.21.15.16:0/1554154320 shutdown_connections 2023-09-30T22:33:48.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.549+0000 7f72294d8700 1 -- 172.21.15.16:0/1554154320 wait complete. 2023-09-30T22:33:48.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.551+0000 7f72294d8700 1 Processor -- start 2023-09-30T22:33:48.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.551+0000 7f72294d8700 1 -- start start 2023-09-30T22:33:48.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.552+0000 7f72294d8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f722409de50 0x7f7224099ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:48.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.552+0000 7f72294d8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f722409e760 0x7f722409a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:48.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.552+0000 7f72294d8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72241338a0 0x7f72241cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:48.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.552+0000 7f72294d8700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f722406f640 con 0x7f72241338a0 2023-09-30T22:33:48.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.552+0000 7f72294d8700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f722409b780 con 0x7f722409e760 2023-09-30T22:33:48.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.552+0000 7f72294d8700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f722409bbe0 con 0x7f722409de50 2023-09-30T22:33:48.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.552+0000 7f72227fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f722409e760 0x7f722409a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:48.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.552+0000 7f72227fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f722409e760 0x7f722409a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37464/0 (socket says 172.21.15.16:37464) 2023-09-30T22:33:48.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.552+0000 7f72227fc700 1 -- 172.21.15.16:0/2404853321 learned_addr learned my addr 172.21.15.16:0/2404853321 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:48.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.552+0000 7f7222ffd700 1 --2- 172.21.15.16:0/2404853321 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f722409de50 0x7f7224099ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:48.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.552+0000 7f72237fe700 1 --2- 172.21.15.16:0/2404853321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72241338a0 0x7f72241cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:48.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.553+0000 7f72227fc700 1 -- 172.21.15.16:0/2404853321 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f722409de50 msgr2=0x7f7224099ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:48.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.553+0000 7f72227fc700 1 --2- 172.21.15.16:0/2404853321 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f722409de50 0x7f7224099ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:48.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.553+0000 7f72227fc700 1 -- 172.21.15.16:0/2404853321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72241338a0 msgr2=0x7f72241cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:48.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.553+0000 7f72227fc700 1 --2- 172.21.15.16:0/2404853321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72241338a0 0x7f72241cc5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:48.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.553+0000 7f72227fc700 1 -- 172.21.15.16:0/2404853321 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7214009550 con 0x7f722409e760 2023-09-30T22:33:48.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.553+0000 7f72227fc700 1 --2- 172.21.15.16:0/2404853321 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f722409e760 0x7f722409a220 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f721800e870 tx=0x7f721800eb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:48.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.553+0000 7f720b7fe700 1 -- 172.21.15.16:0/2404853321 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7218009590 con 0x7f722409e760 2023-09-30T22:33:48.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.553+0000 7f720b7fe700 1 -- 172.21.15.16:0/2404853321 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7218013680 con 0x7f722409e760 2023-09-30T22:33:48.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.553+0000 7f72294d8700 1 -- 172.21.15.16:0/2404853321 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72240a1040 con 0x7f722409e760 2023-09-30T22:33:48.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.553+0000 7f72294d8700 1 -- 172.21.15.16:0/2404853321 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f722409bdf0 con 0x7f722409e760 2023-09-30T22:33:48.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.553+0000 7f720b7fe700 1 -- 172.21.15.16:0/2404853321 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7218013a10 con 0x7f722409e760 2023-09-30T22:33:48.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.555+0000 7f720b7fe700 1 -- 172.21.15.16:0/2404853321 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7218013b70 con 0x7f722409e760 2023-09-30T22:33:48.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.555+0000 7f72294d8700 1 -- 172.21.15.16:0/2404853321 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72040052e0 con 0x7f722409e760 2023-09-30T22:33:48.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.555+0000 7f720b7fe700 1 -- 172.21.15.16:0/2404853321 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f721803f910 con 0x7f722409e760 2023-09-30T22:33:48.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.560+0000 7f720b7fe700 1 -- 172.21.15.16:0/2404853321 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7218011bd0 con 0x7f722409e760 2023-09-30T22:33:48.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.586+0000 7fa73130c700 1 Processor -- start 2023-09-30T22:33:48.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.587+0000 7fa73130c700 1 -- start start 2023-09-30T22:33:48.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.587+0000 7fa73130c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa72c09d6f0 0x7fa72c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:48.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.587+0000 7fa73130c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa72c09e410 0x7fa72c134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:48.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.587+0000 7fa73130c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa72c09b120 con 0x7fa72c09e000 2023-09-30T22:33:48.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.587+0000 7fa73130c700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa72c06b460 con 0x7fa72c09e410 2023-09-30T22:33:48.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.587+0000 7fa73130c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa72c09a7e0 con 0x7fa72c09d6f0 2023-09-30T22:33:48.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.587+0000 7fa72affd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa72c09d6f0 0x7fa72c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:48.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.587+0000 7fa72affd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa72c09d6f0 0x7fa72c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37512/0 (socket says 172.21.15.16:37512) 2023-09-30T22:33:48.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.587+0000 7fa72affd700 1 -- 172.21.15.16:0/1779689602 learned_addr learned my addr 172.21.15.16:0/1779689602 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:48.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.588+0000 7fa72b7fe700 1 --2- 172.21.15.16:0/1779689602 >> v2:172.21.15.16:3300/0 conn(0x7fa72c09e410 0x7fa72c134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:48.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.588+0000 7fa72affd700 1 -- 172.21.15.16:0/1779689602 >> v2:172.21.15.16:3300/0 conn(0x7fa72c09e410 msgr2=0x7fa72c134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:48.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.588+0000 7fa72affd700 1 --2- 172.21.15.16:0/1779689602 >> v2:172.21.15.16:3300/0 conn(0x7fa72c09e410 0x7fa72c134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:48.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.588+0000 7fa72affd700 1 -- 172.21.15.16:0/1779689602 >> v1:172.21.15.16:6789/0 conn(0x7fa72c09e000 legacy=0x7fa72c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:48.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.588+0000 7fa72affd700 1 -- 172.21.15.16:0/1779689602 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa72c09a500 con 0x7fa72c09d6f0 2023-09-30T22:33:48.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.588+0000 7fa729ffb700 1 -- 172.21.15.16:0/1779689602 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2122405766 0 0) 0x7fa72c09b120 con 0x7fa72c09e000 2023-09-30T22:33:48.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.588+0000 7fa72affd700 1 --2- 172.21.15.16:0/1779689602 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa72c09d6f0 0x7fa72c09dac0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa71c009920 tx=0x7fa71c009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=621c9da100f2ef5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:48.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.588+0000 7fa729ffb700 1 -- 172.21.15.16:0/1779689602 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71c011830 con 0x7fa72c09d6f0 2023-09-30T22:33:48.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.588+0000 7fa729ffb700 1 -- 172.21.15.16:0/1779689602 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa71c011990 con 0x7fa72c09d6f0 2023-09-30T22:33:48.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.588+0000 7fa729ffb700 1 -- 172.21.15.16:0/1779689602 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71c011d20 con 0x7fa72c09d6f0 2023-09-30T22:33:48.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.589+0000 7fa73130c700 1 -- 172.21.15.16:0/1779689602 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa72c09d6f0 msgr2=0x7fa72c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:48.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.589+0000 7fa73130c700 1 --2- 172.21.15.16:0/1779689602 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa72c09d6f0 0x7fa72c09dac0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa71c009920 tx=0x7fa71c009c30 comp rx=0 tx=0).stop 2023-09-30T22:33:48.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.589+0000 7fa73130c700 1 -- 172.21.15.16:0/1779689602 shutdown_connections 2023-09-30T22:33:48.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.589+0000 7fa73130c700 1 --2- 172.21.15.16:0/1779689602 >> v2:172.21.15.16:3300/0 conn(0x7fa72c09e410 0x7fa72c134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:48.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.589+0000 7fa73130c700 1 --2- 172.21.15.16:0/1779689602 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa72c09d6f0 0x7fa72c09dac0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:48.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.589+0000 7fa73130c700 1 -- 172.21.15.16:0/1779689602 >> 172.21.15.16:0/1779689602 conn(0x7fa72c066f50 msgr2=0x7fa72c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:48.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.589+0000 7fa73130c700 1 -- 172.21.15.16:0/1779689602 shutdown_connections 2023-09-30T22:33:48.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.589+0000 7fa73130c700 1 -- 172.21.15.16:0/1779689602 wait complete. 2023-09-30T22:33:48.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.590+0000 7fa73130c700 1 Processor -- start 2023-09-30T22:33:48.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.591+0000 7fa73130c700 1 -- start start 2023-09-30T22:33:48.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.591+0000 7fa73130c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa72c09d6f0 0x7fa72c1c0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:48.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.591+0000 7fa73130c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa72c09e000 0x7fa72c1c1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:48.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.591+0000 7fa73130c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa72c09e410 0x7fa72c1c3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:48.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.591+0000 7fa73130c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72c1a4c30 con 0x7fa72c09e000 2023-09-30T22:33:48.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.591+0000 7fa73130c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa72c1c1690 con 0x7fa72c09d6f0 2023-09-30T22:33:48.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.591+0000 7fa73130c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa72c09c710 con 0x7fa72c09e410 2023-09-30T22:33:48.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.591+0000 7fa72b7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa72c09e410 0x7fa72c1c3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:48.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.591+0000 7fa72a7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa72c09e000 0x7fa72c1c1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:48.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.591+0000 7fa72a7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa72c09e000 0x7fa72c1c1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43474/0 (socket says 172.21.15.16:43474) 2023-09-30T22:33:48.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.591+0000 7fa72b7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa72c09e410 0x7fa72c1c3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37526/0 (socket says 172.21.15.16:37526) 2023-09-30T22:33:48.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.591+0000 7fa72a7fc700 1 -- 172.21.15.16:0/2200599219 learned_addr learned my addr 172.21.15.16:0/2200599219 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:48.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.591+0000 7fa72affd700 1 --2- 172.21.15.16:0/2200599219 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa72c09d6f0 0x7fa72c1c0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:48.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.592+0000 7fa72a7fc700 1 -- 172.21.15.16:0/2200599219 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa72c09e410 msgr2=0x7fa72c1c3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:48.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.592+0000 7fa72a7fc700 1 --2- 172.21.15.16:0/2200599219 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa72c09e410 0x7fa72c1c3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:48.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.592+0000 7fa72a7fc700 1 -- 172.21.15.16:0/2200599219 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa72c09d6f0 msgr2=0x7fa72c1c0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:48.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.592+0000 7fa72a7fc700 1 --2- 172.21.15.16:0/2200599219 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa72c09d6f0 0x7fa72c1c0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:48.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.592+0000 7fa72a7fc700 1 -- 172.21.15.16:0/2200599219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa71c009550 con 0x7fa72c09e000 2023-09-30T22:33:48.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.592+0000 7fa72a7fc700 1 --2- 172.21.15.16:0/2200599219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa72c09e000 0x7fa72c1c1150 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fa72000c550 tx=0x7fa72000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:48.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.592+0000 7fa7137fe700 1 -- 172.21.15.16:0/2200599219 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa720003680 con 0x7fa72c09e000 2023-09-30T22:33:48.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.592+0000 7fa7137fe700 1 -- 172.21.15.16:0/2200599219 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa720003800 con 0x7fa72c09e000 2023-09-30T22:33:48.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.592+0000 7fa73130c700 1 -- 172.21.15.16:0/2200599219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa72c1a34b0 con 0x7fa72c09e000 2023-09-30T22:33:48.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.592+0000 7fa7137fe700 1 -- 172.21.15.16:0/2200599219 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa720003bb0 con 0x7fa72c09e000 2023-09-30T22:33:48.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.592+0000 7fa73130c700 1 -- 172.21.15.16:0/2200599219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa72c1a50e0 con 0x7fa72c09e000 2023-09-30T22:33:48.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.593+0000 7fa7137fe700 1 -- 172.21.15.16:0/2200599219 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa720003d10 con 0x7fa72c09e000 2023-09-30T22:33:48.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.593+0000 7fa73130c700 1 -- 172.21.15.16:0/2200599219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa72c074ea0 con 0x7fa72c09e000 2023-09-30T22:33:48.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.594+0000 7fa7137fe700 1 -- 172.21.15.16:0/2200599219 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fa72003eb80 con 0x7fa72c09e000 2023-09-30T22:33:48.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.599+0000 7fa7137fe700 1 -- 172.21.15.16:0/2200599219 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa72001bac0 con 0x7fa72c09e000 2023-09-30T22:33:48.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.687+0000 7f8b60cd5700 1 -- 172.21.15.16:0/4069376502 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8b5c0713a0 con 0x7f8b5c130260 2023-09-30T22:33:48.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.688+0000 7f8b4affd700 1 -- 172.21.15.16:0/4069376502 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8b54017030 con 0x7f8b5c130260 2023-09-30T22:33:48.688 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:33:48.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.690+0000 7f8b60cd5700 1 -- 172.21.15.16:0/4069376502 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8b5c130260 msgr2=0x7f8b5c1c82c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:48.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.690+0000 7f8b60cd5700 1 --2- 172.21.15.16:0/4069376502 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8b5c130260 0x7f8b5c1c82c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f8b5400cbc0 tx=0x7f8b5400ced0 comp rx=0 tx=0).stop 2023-09-30T22:33:48.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.690+0000 7f8b60cd5700 1 -- 172.21.15.16:0/4069376502 shutdown_connections 2023-09-30T22:33:48.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.690+0000 7f8b60cd5700 1 --2- 172.21.15.16:0/4069376502 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8b5c130260 0x7f8b5c1c82c0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:48.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.690+0000 7f8b60cd5700 1 --2- 172.21.15.16:0/4069376502 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8b5c12cf60 0x7f8b5c1c5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:48.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.690+0000 7f8b60cd5700 1 --2- 172.21.15.16:0/4069376502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b5c12f8b0 0x7f8b5c1c57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:48.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.691+0000 7f8b60cd5700 1 -- 172.21.15.16:0/4069376502 >> 172.21.15.16:0/4069376502 conn(0x7f8b5c0a2aa0 msgr2=0x7f8b5c1b7990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:48.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.691+0000 7f8b60cd5700 1 -- 172.21.15.16:0/4069376502 shutdown_connections 2023-09-30T22:33:48.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.691+0000 7f8b60cd5700 1 -- 172.21.15.16:0/4069376502 wait complete. 2023-09-30T22:33:48.705 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:33:48.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.736+0000 7f704dfd6700 1 -- 172.21.15.16:0/916816468 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7028005590 con 0x7f70480a0a30 2023-09-30T22:33:48.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.736+0000 7f702ffff700 1 -- 172.21.15.16:0/916816468 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f703c024020 con 0x7f70480a0a30 2023-09-30T22:33:48.737 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:33:48.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.739+0000 7f704dfd6700 1 -- 172.21.15.16:0/916816468 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f70480a0a30 msgr2=0x7f704812ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:48.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.739+0000 7f704dfd6700 1 --2- 172.21.15.16:0/916816468 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f70480a0a30 0x7f704812ca70 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f703c00e8e0 tx=0x7f703c00ebf0 comp rx=0 tx=0).stop 2023-09-30T22:33:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.739+0000 7f704dfd6700 1 -- 172.21.15.16:0/916816468 shutdown_connections 2023-09-30T22:33:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.739+0000 7f704dfd6700 1 --2- 172.21.15.16:0/916816468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7048138d80 0x7f704812cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.739+0000 7f704dfd6700 1 --2- 172.21.15.16:0/916816468 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f704809d630 0x7f704812c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.739+0000 7f704dfd6700 1 --2- 172.21.15.16:0/916816468 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f70480a0a30 0x7f704812ca70 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.739+0000 7f704dfd6700 1 -- 172.21.15.16:0/916816468 >> 172.21.15.16:0/916816468 conn(0x7f70480a3050 msgr2=0x7f704812ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.739+0000 7f704dfd6700 1 -- 172.21.15.16:0/916816468 shutdown_connections 2023-09-30T22:33:48.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.739+0000 7f704dfd6700 1 -- 172.21.15.16:0/916816468 wait complete. 2023-09-30T22:33:48.753 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:33:48.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.769+0000 7f72294d8700 1 -- 172.21.15.16:0/2404853321 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7204005590 con 0x7f722409e760 2023-09-30T22:33:48.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.770+0000 7f720b7fe700 1 -- 172.21.15.16:0/2404853321 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7218021090 con 0x7f722409e760 2023-09-30T22:33:48.770 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:33:48.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.772+0000 7f72294d8700 1 -- 172.21.15.16:0/2404853321 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f722409e760 msgr2=0x7f722409a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:48.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.772+0000 7f72294d8700 1 --2- 172.21.15.16:0/2404853321 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f722409e760 0x7f722409a220 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f721800e870 tx=0x7f721800eb80 comp rx=0 tx=0).stop 2023-09-30T22:33:48.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.772+0000 7f72294d8700 1 -- 172.21.15.16:0/2404853321 shutdown_connections 2023-09-30T22:33:48.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.772+0000 7f72294d8700 1 --2- 172.21.15.16:0/2404853321 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72241338a0 0x7f72241cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:48.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.772+0000 7f72294d8700 1 --2- 172.21.15.16:0/2404853321 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f722409de50 0x7f7224099ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:48.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.772+0000 7f72294d8700 1 --2- 172.21.15.16:0/2404853321 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f722409e760 0x7f722409a220 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:48.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.772+0000 7f72294d8700 1 -- 172.21.15.16:0/2404853321 >> 172.21.15.16:0/2404853321 conn(0x7f72240a2aa0 msgr2=0x7f722409ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:48.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.772+0000 7f72294d8700 1 -- 172.21.15.16:0/2404853321 shutdown_connections 2023-09-30T22:33:48.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.772+0000 7f72294d8700 1 -- 172.21.15.16:0/2404853321 wait complete. 2023-09-30T22:33:48.786 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:33:48.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.806+0000 7fa73130c700 1 -- 172.21.15.16:0/2200599219 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa72c06c360 con 0x7fa72c09e000 2023-09-30T22:33:48.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.806+0000 7fa7137fe700 1 -- 172.21.15.16:0/2200599219 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa720017030 con 0x7fa72c09e000 2023-09-30T22:33:48.807 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:33:48.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.808+0000 7fa73130c700 1 -- 172.21.15.16:0/2200599219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa72c09e000 msgr2=0x7fa72c1c1150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:48.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.808+0000 7fa73130c700 1 --2- 172.21.15.16:0/2200599219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa72c09e000 0x7fa72c1c1150 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fa72000c550 tx=0x7fa72000c860 comp rx=0 tx=0).stop 2023-09-30T22:33:48.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.808+0000 7fa73130c700 1 -- 172.21.15.16:0/2200599219 shutdown_connections 2023-09-30T22:33:48.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.808+0000 7fa73130c700 1 --2- 172.21.15.16:0/2200599219 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa72c09d6f0 0x7fa72c1c0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:48.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.808+0000 7fa73130c700 1 --2- 172.21.15.16:0/2200599219 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa72c09e410 0x7fa72c1c3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:48.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.808+0000 7fa73130c700 1 --2- 172.21.15.16:0/2200599219 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa72c09e000 0x7fa72c1c1150 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:48.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.808+0000 7fa73130c700 1 -- 172.21.15.16:0/2200599219 >> 172.21.15.16:0/2200599219 conn(0x7fa72c066f50 msgr2=0x7fa72c1b6a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:48.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.808+0000 7fa73130c700 1 -- 172.21.15.16:0/2200599219 shutdown_connections 2023-09-30T22:33:48.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:48.808+0000 7fa73130c700 1 -- 172.21.15.16:0/2200599219 wait complete. 2023-09-30T22:33:48.822 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:33:49.707 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:33:49.754 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:33:49.787 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:33:49.823 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:33:49.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.859+0000 7faf0ab22700 1 Processor -- start 2023-09-30T22:33:49.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.859+0000 7faf0ab22700 1 -- start start 2023-09-30T22:33:49.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.859+0000 7faf0ab22700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf04130540 0x7faf04130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:49.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.859+0000 7faf0ab22700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faf04130e50 0x7faf0406f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:49.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.859+0000 7faf0ab22700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf040716a0 con 0x7faf0406fb20 2023-09-30T22:33:49.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.859+0000 7faf0ab22700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf0409a390 con 0x7faf04130e50 2023-09-30T22:33:49.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.859+0000 7faf0ab22700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faf04071290 con 0x7faf04130540 2023-09-30T22:33:49.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.860+0000 7faf03fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faf04130e50 0x7faf0406f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:49.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.860+0000 7faf03fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faf04130e50 0x7faf0406f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43488/0 (socket says 172.21.15.16:43488) 2023-09-30T22:33:49.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.860+0000 7faf03fff700 1 -- 172.21.15.16:0/1135030736 learned_addr learned my addr 172.21.15.16:0/1135030736 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:49.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.860+0000 7faf090bf700 1 --1- 172.21.15.16:0/1135030736 >> v1:172.21.15.16:6789/0 conn(0x7faf0406fb20 0x7faf04137560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50014/0 (socket says 172.21.15.16:50014) 2023-09-30T22:33:49.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.860+0000 7faf03fff700 1 -- 172.21.15.16:0/1135030736 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf04130540 msgr2=0x7faf04130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:49.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.860+0000 7faf03fff700 1 --2- 172.21.15.16:0/1135030736 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf04130540 0x7faf04130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:49.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.860+0000 7faf03fff700 1 -- 172.21.15.16:0/1135030736 >> v1:172.21.15.16:6789/0 conn(0x7faf0406fb20 legacy=0x7faf04137560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:49.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.860+0000 7faf03fff700 1 -- 172.21.15.16:0/1135030736 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf04130340 con 0x7faf04130e50 2023-09-30T22:33:49.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.860+0000 7faf03fff700 1 --2- 172.21.15.16:0/1135030736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf04130e50 0x7faf0406f5e0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7faef00098f0 tx=0x7faef0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7922f4b8e7572cfa server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:49.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.860+0000 7faf037fe700 1 -- 172.21.15.16:0/1135030736 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef000b8a0 con 0x7faf04130e50 2023-09-30T22:33:49.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.861+0000 7faf037fe700 1 -- 172.21.15.16:0/1135030736 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faef000ba00 con 0x7faf04130e50 2023-09-30T22:33:49.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.861+0000 7faf037fe700 1 -- 172.21.15.16:0/1135030736 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef000bd90 con 0x7faf04130e50 2023-09-30T22:33:49.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.862+0000 7faf0ab22700 1 -- 172.21.15.16:0/1135030736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf04130e50 msgr2=0x7faf0406f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:49.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.862+0000 7faf0ab22700 1 --2- 172.21.15.16:0/1135030736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf04130e50 0x7faf0406f5e0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7faef00098f0 tx=0x7faef0009c00 comp rx=0 tx=0).stop 2023-09-30T22:33:49.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.862+0000 7faf0ab22700 1 -- 172.21.15.16:0/1135030736 shutdown_connections 2023-09-30T22:33:49.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.862+0000 7faf0ab22700 1 --2- 172.21.15.16:0/1135030736 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf04130540 0x7faf04130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:49.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.862+0000 7faf0ab22700 1 --2- 172.21.15.16:0/1135030736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf04130e50 0x7faf0406f5e0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:49.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.862+0000 7faf0ab22700 1 -- 172.21.15.16:0/1135030736 >> 172.21.15.16:0/1135030736 conn(0x7faf040a2aa0 msgr2=0x7faf040a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:49.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.862+0000 7faf0ab22700 1 -- 172.21.15.16:0/1135030736 shutdown_connections 2023-09-30T22:33:49.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.862+0000 7faf0ab22700 1 -- 172.21.15.16:0/1135030736 wait complete. 2023-09-30T22:33:49.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.863+0000 7faf0ab22700 1 Processor -- start 2023-09-30T22:33:49.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.863+0000 7faf0ab22700 1 -- start start 2023-09-30T22:33:49.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.864+0000 7faf0ab22700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf0406fb20 0x7faf0413aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:49.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.864+0000 7faf0ab22700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf04130540 0x7faf04137b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:49.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.864+0000 7faf0ab22700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf04130e50 0x7faf04138070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:49.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.864+0000 7faf0ab22700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf040a05f0 con 0x7faf04130540 2023-09-30T22:33:49.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.864+0000 7faf0ab22700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf040718e0 con 0x7faf0406fb20 2023-09-30T22:33:49.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.864+0000 7faf0ab22700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faf0413b7b0 con 0x7faf04130e50 2023-09-30T22:33:49.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.864+0000 7faf088be700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf0406fb20 0x7faf0413aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:49.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.864+0000 7faf088be700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf0406fb20 0x7faf0413aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37490/0 (socket says 172.21.15.16:37490) 2023-09-30T22:33:49.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.864+0000 7faf088be700 1 -- 172.21.15.16:0/1621748967 learned_addr learned my addr 172.21.15.16:0/1621748967 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:49.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.865+0000 7faf03fff700 1 --2- 172.21.15.16:0/1621748967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf04130540 0x7faf04137b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:49.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.865+0000 7faf090bf700 1 --2- 172.21.15.16:0/1621748967 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf04130e50 0x7faf04138070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:49.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.865+0000 7faf088be700 1 -- 172.21.15.16:0/1621748967 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf04130e50 msgr2=0x7faf04138070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:49.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.865+0000 7faf088be700 1 --2- 172.21.15.16:0/1621748967 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf04130e50 0x7faf04138070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.865+0000 7faf088be700 1 -- 172.21.15.16:0/1621748967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf04130540 msgr2=0x7faf04137b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.865+0000 7faf088be700 1 --2- 172.21.15.16:0/1621748967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf04130540 0x7faf04137b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.865+0000 7faf088be700 1 -- 172.21.15.16:0/1621748967 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faef0009550 con 0x7faf0406fb20 2023-09-30T22:33:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.865+0000 7faf03fff700 1 --2- 172.21.15.16:0/1621748967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf04130540 0x7faf04137b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:33:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.865+0000 7faf090bf700 1 --2- 172.21.15.16:0/1621748967 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf04130e50 0x7faf04138070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:33:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.865+0000 7faf088be700 1 --2- 172.21.15.16:0/1621748967 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf0406fb20 0x7faf0413aca0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7faef8013f70 tx=0x7faef8011e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.865+0000 7faf017fa700 1 -- 172.21.15.16:0/1621748967 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef8014d30 con 0x7faf0406fb20 2023-09-30T22:33:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.865+0000 7faf017fa700 1 -- 172.21.15.16:0/1621748967 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faef8014e90 con 0x7faf0406fb20 2023-09-30T22:33:49.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.865+0000 7faf0ab22700 1 -- 172.21.15.16:0/1621748967 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf04136ce0 con 0x7faf0406fb20 2023-09-30T22:33:49.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.865+0000 7faf017fa700 1 -- 172.21.15.16:0/1621748967 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef8011310 con 0x7faf0406fb20 2023-09-30T22:33:49.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.865+0000 7faf0ab22700 1 -- 172.21.15.16:0/1621748967 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf04135cb0 con 0x7faf0406fb20 2023-09-30T22:33:49.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.866+0000 7faf017fa700 1 -- 172.21.15.16:0/1621748967 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7faef8011470 con 0x7faf0406fb20 2023-09-30T22:33:49.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.867+0000 7faf017fa700 1 -- 172.21.15.16:0/1621748967 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7faef8047300 con 0x7faf0406fb20 2023-09-30T22:33:49.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.867+0000 7faf0ab22700 1 -- 172.21.15.16:0/1621748967 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf04136b20 con 0x7faf0406fb20 2023-09-30T22:33:49.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.872+0000 7faf017fa700 1 -- 172.21.15.16:0/1621748967 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7faef8019890 con 0x7faf0406fb20 2023-09-30T22:33:49.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.902+0000 7ff005ce9700 1 Processor -- start 2023-09-30T22:33:49.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.902+0000 7ff005ce9700 1 -- start start 2023-09-30T22:33:49.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.903+0000 7ff005ce9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff00009d6f0 0x7ff00009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:49.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.903+0000 7ff005ce9700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff00009e000 0x7ff0001339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:49.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.903+0000 7ff005ce9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff00009e410 0x7ff000133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:49.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.903+0000 7ff005ce9700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff00006b460 con 0x7ff00009e000 2023-09-30T22:33:49.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.903+0000 7ff005ce9700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff00009b120 con 0x7ff00009d6f0 2023-09-30T22:33:49.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.903+0000 7ff005ce9700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff00009a7e0 con 0x7ff00009e410 2023-09-30T22:33:49.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.903+0000 7fefff7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff00009d6f0 0x7ff00009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:49.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.903+0000 7fefff7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff00009d6f0 0x7ff00009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37500/0 (socket says 172.21.15.16:37500) 2023-09-30T22:33:49.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.903+0000 7fefff7fe700 1 -- 172.21.15.16:0/97706915 learned_addr learned my addr 172.21.15.16:0/97706915 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:49.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.903+0000 7feffffff700 1 --2- 172.21.15.16:0/97706915 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff00009e410 0x7ff000133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:49.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.903+0000 7feffffff700 1 -- 172.21.15.16:0/97706915 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff00009d6f0 msgr2=0x7ff00009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:49.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.903+0000 7feffffff700 1 --2- 172.21.15.16:0/97706915 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff00009d6f0 0x7ff00009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:49.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.903+0000 7feffffff700 1 -- 172.21.15.16:0/97706915 >> v2:172.21.15.16:3300/0 conn(0x7ff00009e000 msgr2=0x7ff0001339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:49.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.903+0000 7feffffff700 1 --2- 172.21.15.16:0/97706915 >> v2:172.21.15.16:3300/0 conn(0x7ff00009e000 0x7ff0001339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:49.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.903+0000 7feffffff700 1 -- 172.21.15.16:0/97706915 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff00009a500 con 0x7ff00009e410 2023-09-30T22:33:49.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.904+0000 7fefff7fe700 1 --2- 172.21.15.16:0/97706915 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff00009d6f0 0x7ff00009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:49.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.904+0000 7feffffff700 1 --2- 172.21.15.16:0/97706915 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff00009e410 0x7ff000133f30 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7feff8009dc0 tx=0x7feff800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afed6f6b5aa8f939 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:49.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.904+0000 7feffe7fc700 1 -- 172.21.15.16:0/97706915 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff800e070 con 0x7ff00009e410 2023-09-30T22:33:49.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.904+0000 7feffe7fc700 1 -- 172.21.15.16:0/97706915 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feff800bae0 con 0x7ff00009e410 2023-09-30T22:33:49.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.904+0000 7feffe7fc700 1 -- 172.21.15.16:0/97706915 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff800be90 con 0x7ff00009e410 2023-09-30T22:33:49.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.905+0000 7ff005ce9700 1 -- 172.21.15.16:0/97706915 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff00009e410 msgr2=0x7ff000133f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:49.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.905+0000 7ff005ce9700 1 --2- 172.21.15.16:0/97706915 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff00009e410 0x7ff000133f30 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7feff8009dc0 tx=0x7feff800b330 comp rx=0 tx=0).stop 2023-09-30T22:33:49.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.905+0000 7ff005ce9700 1 -- 172.21.15.16:0/97706915 shutdown_connections 2023-09-30T22:33:49.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.905+0000 7ff005ce9700 1 --2- 172.21.15.16:0/97706915 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff00009d6f0 0x7ff00009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:49.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.905+0000 7ff005ce9700 1 --2- 172.21.15.16:0/97706915 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff00009e410 0x7ff000133f30 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:49.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.905+0000 7ff005ce9700 1 --2- 172.21.15.16:0/97706915 >> v2:172.21.15.16:3300/0 conn(0x7ff00009e000 0x7ff0001339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:49.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.905+0000 7ff005ce9700 1 -- 172.21.15.16:0/97706915 >> 172.21.15.16:0/97706915 conn(0x7ff000066f50 msgr2=0x7ff00009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:49.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.905+0000 7ff005ce9700 1 -- 172.21.15.16:0/97706915 shutdown_connections 2023-09-30T22:33:49.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.905+0000 7ff005ce9700 1 -- 172.21.15.16:0/97706915 wait complete. 2023-09-30T22:33:49.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.906+0000 7ff005ce9700 1 Processor -- start 2023-09-30T22:33:49.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.906+0000 7ff005ce9700 1 -- start start 2023-09-30T22:33:49.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.907+0000 7ff005ce9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff00009d6f0 0x7ff0001a5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:49.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.907+0000 7ff005ce9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff00009e000 0x7ff0001a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:49.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.907+0000 7ff005ce9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff00009e410 0x7ff0001a67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:49.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.907+0000 7ff005ce9700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0000a1f00 con 0x7ff00009d6f0 2023-09-30T22:33:49.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.907+0000 7ff005ce9700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0001a92d0 con 0x7ff00009e410 2023-09-30T22:33:49.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.907+0000 7ff005ce9700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff0001a7a70 con 0x7ff00009e000 2023-09-30T22:33:49.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.907+0000 7feffeffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff00009e000 0x7ff0001a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:49.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.907+0000 7feffeffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff00009e000 0x7ff0001a6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37560/0 (socket says 172.21.15.16:37560) 2023-09-30T22:33:49.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.907+0000 7feffeffd700 1 -- 172.21.15.16:0/501764724 learned_addr learned my addr 172.21.15.16:0/501764724 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:49.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.907+0000 7fefff7fe700 1 --2- 172.21.15.16:0/501764724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff00009d6f0 0x7ff0001a5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:49.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.907+0000 7feffffff700 1 --2- 172.21.15.16:0/501764724 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff00009e410 0x7ff0001a67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:49.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.907+0000 7feffeffd700 1 -- 172.21.15.16:0/501764724 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff00009e410 msgr2=0x7ff0001a67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:49.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.907+0000 7feffeffd700 1 --2- 172.21.15.16:0/501764724 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff00009e410 0x7ff0001a67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:49.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.907+0000 7feffeffd700 1 -- 172.21.15.16:0/501764724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff00009d6f0 msgr2=0x7ff0001a5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:49.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.907+0000 7feffeffd700 1 --2- 172.21.15.16:0/501764724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff00009d6f0 0x7ff0001a5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:49.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.907+0000 7feffeffd700 1 -- 172.21.15.16:0/501764724 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff0009550 con 0x7ff00009e000 2023-09-30T22:33:49.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.908+0000 7feffeffd700 1 --2- 172.21.15.16:0/501764724 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff00009e000 0x7ff0001a6290 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7feff400b500 tx=0x7feff400b810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:49.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.908+0000 7feffffff700 1 --2- 172.21.15.16:0/501764724 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff00009e410 0x7ff0001a67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:33:49.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.908+0000 7fefe7fff700 1 -- 172.21.15.16:0/501764724 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff4011830 con 0x7ff00009e000 2023-09-30T22:33:49.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.908+0000 7fefe7fff700 1 -- 172.21.15.16:0/501764724 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feff4011990 con 0x7ff00009e000 2023-09-30T22:33:49.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.908+0000 7fefe7fff700 1 -- 172.21.15.16:0/501764724 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff4011d20 con 0x7ff00009e000 2023-09-30T22:33:49.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.908+0000 7ff005ce9700 1 -- 172.21.15.16:0/501764724 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feff8009a70 con 0x7ff00009e000 2023-09-30T22:33:49.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.908+0000 7ff005ce9700 1 -- 172.21.15.16:0/501764724 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0001a8760 con 0x7ff00009e000 2023-09-30T22:33:49.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.909+0000 7ff005ce9700 1 -- 172.21.15.16:0/501764724 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0001a9410 con 0x7ff00009e000 2023-09-30T22:33:49.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.909+0000 7fefe7fff700 1 -- 172.21.15.16:0/501764724 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7feff401a400 con 0x7ff00009e000 2023-09-30T22:33:49.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.910+0000 7fefe7fff700 1 -- 172.21.15.16:0/501764724 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7feff403f830 con 0x7ff00009e000 2023-09-30T22:33:49.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.914+0000 7fefe7fff700 1 -- 172.21.15.16:0/501764724 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7feff400f8d0 con 0x7ff00009e000 2023-09-30T22:33:49.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.976+0000 7fd6e5446700 1 Processor -- start 2023-09-30T22:33:49.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.976+0000 7fd6e5446700 1 -- start start 2023-09-30T22:33:49.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.977+0000 7fd6e5446700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6e009d6f0 0x7fd6e009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:49.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.977+0000 7fd6e5446700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd6e009e410 0x7fd6e0134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:49.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.977+0000 7fd6e5446700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6e009b120 con 0x7fd6e009e000 2023-09-30T22:33:49.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.977+0000 7fd6e5446700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6e006b460 con 0x7fd6e009e410 2023-09-30T22:33:49.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.977+0000 7fd6e5446700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e009a7e0 con 0x7fd6e009d6f0 2023-09-30T22:33:49.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.977+0000 7fd6deffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6e009d6f0 0x7fd6e009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:49.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.977+0000 7fd6de7fc700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd6e009e000 0x7fd6e01339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50020/0 (socket says 172.21.15.16:50020) 2023-09-30T22:33:49.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.977+0000 7fd6de7fc700 1 -- 172.21.15.16:0/2241730868 learned_addr learned my addr 172.21.15.16:0/2241730868 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:49.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.977+0000 7fd6deffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6e009d6f0 0x7fd6e009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37516/0 (socket says 172.21.15.16:37516) 2023-09-30T22:33:49.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.977+0000 7fd6df7fe700 1 --2- 172.21.15.16:0/2241730868 >> v2:172.21.15.16:3300/0 conn(0x7fd6e009e410 0x7fd6e0134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:49.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.977+0000 7fd6df7fe700 1 -- 172.21.15.16:0/2241730868 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6e009d6f0 msgr2=0x7fd6e009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:49.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.977+0000 7fd6df7fe700 1 --2- 172.21.15.16:0/2241730868 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6e009d6f0 0x7fd6e009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:49.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.977+0000 7fd6df7fe700 1 -- 172.21.15.16:0/2241730868 >> v1:172.21.15.16:6789/0 conn(0x7fd6e009e000 legacy=0x7fd6e01339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:49.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.977+0000 7fd6df7fe700 1 -- 172.21.15.16:0/2241730868 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e009a500 con 0x7fd6e009e410 2023-09-30T22:33:49.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.978+0000 7fd6df7fe700 1 --2- 172.21.15.16:0/2241730868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6e009e410 0x7fd6e0134100 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8009d90 tx=0x7fd6d800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50d1259e62b654d0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:49.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.978+0000 7fd6ddffb700 1 -- 172.21.15.16:0/2241730868 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d800e070 con 0x7fd6e009e410 2023-09-30T22:33:49.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.978+0000 7fd6ddffb700 1 -- 172.21.15.16:0/2241730868 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6d800bb20 con 0x7fd6e009e410 2023-09-30T22:33:49.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.978+0000 7fd6ddffb700 1 -- 172.21.15.16:0/2241730868 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d8012430 con 0x7fd6e009e410 2023-09-30T22:33:49.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.979+0000 7fd6e5446700 1 -- 172.21.15.16:0/2241730868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6e009e410 msgr2=0x7fd6e0134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:49.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.979+0000 7fd6e5446700 1 --2- 172.21.15.16:0/2241730868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6e009e410 0x7fd6e0134100 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8009d90 tx=0x7fd6d800b330 comp rx=0 tx=0).stop 2023-09-30T22:33:49.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.979+0000 7fd6e5446700 1 -- 172.21.15.16:0/2241730868 shutdown_connections 2023-09-30T22:33:49.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.979+0000 7fd6e5446700 1 --2- 172.21.15.16:0/2241730868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6e009e410 0x7fd6e0134100 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:49.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.979+0000 7fd6e5446700 1 --2- 172.21.15.16:0/2241730868 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6e009d6f0 0x7fd6e009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:49.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.979+0000 7fd6e5446700 1 -- 172.21.15.16:0/2241730868 >> 172.21.15.16:0/2241730868 conn(0x7fd6e0066f50 msgr2=0x7fd6e009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:49.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.979+0000 7fd6e5446700 1 -- 172.21.15.16:0/2241730868 shutdown_connections 2023-09-30T22:33:49.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.979+0000 7fd6e5446700 1 -- 172.21.15.16:0/2241730868 wait complete. 2023-09-30T22:33:49.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.980+0000 7fd6e5446700 1 Processor -- start 2023-09-30T22:33:49.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.980+0000 7fd6e5446700 1 -- start start 2023-09-30T22:33:49.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.981+0000 7fd6e5446700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6e009d6f0 0x7fd6e01a5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:49.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.981+0000 7fd6e5446700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6e009e000 0x7fd6e01a64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:49.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.981+0000 7fd6e5446700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd6e009e410 0x7fd6e01a6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:49.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.981+0000 7fd6e5446700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e01a8090 con 0x7fd6e009d6f0 2023-09-30T22:33:49.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.981+0000 7fd6e5446700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e01a9150 con 0x7fd6e009e000 2023-09-30T22:33:49.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.981+0000 7fd6e5446700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e009c710 con 0x7fd6e009e410 2023-09-30T22:33:49.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.981+0000 7fd6de7fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6e009e000 0x7fd6e01a64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:49.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.981+0000 7fd6de7fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6e009e000 0x7fd6e01a64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37526/0 (socket says 172.21.15.16:37526) 2023-09-30T22:33:49.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.981+0000 7fd6de7fc700 1 -- 172.21.15.16:0/1233591102 learned_addr learned my addr 172.21.15.16:0/1233591102 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:49.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.981+0000 7fd6df7fe700 1 --2- 172.21.15.16:0/1233591102 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd6e009e410 0x7fd6e01a6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:49.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.981+0000 7fd6deffd700 1 --2- 172.21.15.16:0/1233591102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6e009d6f0 0x7fd6e01a5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:49.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.982+0000 7fd6df7fe700 1 -- 172.21.15.16:0/1233591102 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6e009e000 msgr2=0x7fd6e01a64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:49.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.982+0000 7fd6df7fe700 1 --2- 172.21.15.16:0/1233591102 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6e009e000 0x7fd6e01a64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:49.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.982+0000 7fd6df7fe700 1 -- 172.21.15.16:0/1233591102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6e009d6f0 msgr2=0x7fd6e01a5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:49.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.982+0000 7fd6df7fe700 1 --2- 172.21.15.16:0/1233591102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6e009d6f0 0x7fd6e01a5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:49.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.982+0000 7fd6df7fe700 1 -- 172.21.15.16:0/1233591102 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d8009a70 con 0x7fd6e009e410 2023-09-30T22:33:49.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.982+0000 7fd6deffd700 1 --2- 172.21.15.16:0/1233591102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6e009d6f0 0x7fd6e01a5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:49.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.982+0000 7fd6df7fe700 1 --2- 172.21.15.16:0/1233591102 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd6e009e410 0x7fd6e01a6a00 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8009d90 tx=0x7fd6d8004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:49.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.982+0000 7fd6c77fe700 1 -- 172.21.15.16:0/1233591102 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d8012430 con 0x7fd6e009e410 2023-09-30T22:33:49.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.982+0000 7fd6c77fe700 1 -- 172.21.15.16:0/1233591102 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6d8020070 con 0x7fd6e009e410 2023-09-30T22:33:49.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.982+0000 7fd6e5446700 1 -- 172.21.15.16:0/1233591102 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6e00a30d0 con 0x7fd6e009e410 2023-09-30T22:33:49.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.982+0000 7fd6c77fe700 1 -- 172.21.15.16:0/1233591102 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d8004280 con 0x7fd6e009e410 2023-09-30T22:33:49.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.982+0000 7fd6e5446700 1 -- 172.21.15.16:0/1233591102 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6e01a8d00 con 0x7fd6e009e410 2023-09-30T22:33:49.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.983+0000 7fd6c77fe700 1 -- 172.21.15.16:0/1233591102 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd6d80043e0 con 0x7fd6e009e410 2023-09-30T22:33:49.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.984+0000 7fd6e5446700 1 -- 172.21.15.16:0/1233591102 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6e0074ea0 con 0x7fd6e009e410 2023-09-30T22:33:49.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.984+0000 7fd6c77fe700 1 -- 172.21.15.16:0/1233591102 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd6d8022070 con 0x7fd6e009e410 2023-09-30T22:33:49.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.991+0000 7f31470e1700 1 Processor -- start 2023-09-30T22:33:49.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.992+0000 7f31470e1700 1 -- start start 2023-09-30T22:33:49.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.992+0000 7f31470e1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f314009e850 0x7f314009ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:49.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.992+0000 7f31470e1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f314009f160 0x7f3140131790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:49.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.992+0000 7f31470e1700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3140131cd0 0x7f3140132170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:49.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.992+0000 7f31470e1700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f314009b560 con 0x7f3140131cd0 2023-09-30T22:33:49.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.992+0000 7f31470e1700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31400a0040 con 0x7f314009e850 2023-09-30T22:33:49.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.992+0000 7f31470e1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f31400a02e0 con 0x7f314009f160 2023-09-30T22:33:49.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.992+0000 7f314567e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3140131cd0 0x7f3140132170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:49.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.992+0000 7f3144e7d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f314009e850 0x7f314009ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:49.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.992+0000 7f314567e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3140131cd0 0x7f3140132170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43542/0 (socket says 172.21.15.16:43542) 2023-09-30T22:33:49.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.992+0000 7f314567e700 1 -- 172.21.15.16:0/2974728306 learned_addr learned my addr 172.21.15.16:0/2974728306 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:49.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.993+0000 7f313ffff700 1 --2- 172.21.15.16:0/2974728306 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f314009f160 0x7f3140131790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:49.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.993+0000 7f314567e700 1 -- 172.21.15.16:0/2974728306 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f314009f160 msgr2=0x7f3140131790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:49.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.993+0000 7f314567e700 1 --2- 172.21.15.16:0/2974728306 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f314009f160 0x7f3140131790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:49.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.993+0000 7f314567e700 1 -- 172.21.15.16:0/2974728306 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f314009e850 msgr2=0x7f314009ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:49.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.993+0000 7f314567e700 1 --2- 172.21.15.16:0/2974728306 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f314009e850 0x7f314009ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:49.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.993+0000 7f314567e700 1 -- 172.21.15.16:0/2974728306 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f314009fab0 con 0x7f3140131cd0 2023-09-30T22:33:49.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.993+0000 7f3144e7d700 1 --2- 172.21.15.16:0/2974728306 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f314009e850 0x7f314009ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:49.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.993+0000 7f314567e700 1 --2- 172.21.15.16:0/2974728306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3140131cd0 0x7f3140132170 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f3138009e50 tx=0x7f313800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20c1333b5510a4f3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:49.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.993+0000 7f313f7fe700 1 -- 172.21.15.16:0/2974728306 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313800e070 con 0x7f3140131cd0 2023-09-30T22:33:49.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.993+0000 7f313f7fe700 1 -- 172.21.15.16:0/2974728306 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f313800b910 con 0x7f3140131cd0 2023-09-30T22:33:49.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.993+0000 7f313f7fe700 1 -- 172.21.15.16:0/2974728306 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f313800bcc0 con 0x7f3140131cd0 2023-09-30T22:33:49.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.993+0000 7fd6c77fe700 1 -- 172.21.15.16:0/1233591102 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd6d801b400 con 0x7fd6e009e410 2023-09-30T22:33:49.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.994+0000 7f31470e1700 1 -- 172.21.15.16:0/2974728306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3140131cd0 msgr2=0x7f3140132170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:49.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.994+0000 7f31470e1700 1 --2- 172.21.15.16:0/2974728306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3140131cd0 0x7f3140132170 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f3138009e50 tx=0x7f313800b330 comp rx=0 tx=0).stop 2023-09-30T22:33:49.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.994+0000 7f31470e1700 1 -- 172.21.15.16:0/2974728306 shutdown_connections 2023-09-30T22:33:49.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.994+0000 7f31470e1700 1 --2- 172.21.15.16:0/2974728306 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f314009e850 0x7f314009ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:49.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.994+0000 7f31470e1700 1 --2- 172.21.15.16:0/2974728306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3140131cd0 0x7f3140132170 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:49.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.994+0000 7f31470e1700 1 --2- 172.21.15.16:0/2974728306 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f314009f160 0x7f3140131790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:49.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.994+0000 7f31470e1700 1 -- 172.21.15.16:0/2974728306 >> 172.21.15.16:0/2974728306 conn(0x7f31400a3f70 msgr2=0x7f31400a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:49.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.994+0000 7f31470e1700 1 -- 172.21.15.16:0/2974728306 shutdown_connections 2023-09-30T22:33:49.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.994+0000 7f31470e1700 1 -- 172.21.15.16:0/2974728306 wait complete. 2023-09-30T22:33:49.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.995+0000 7f31470e1700 1 Processor -- start 2023-09-30T22:33:49.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.995+0000 7f31470e1700 1 -- start start 2023-09-30T22:33:49.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.996+0000 7f31470e1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f314009e850 0x7f314011d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:49.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.996+0000 7f31470e1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f314009f160 0x7f314011d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:49.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.996+0000 7f31470e1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3140131cd0 0x7f314011ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:49.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.996+0000 7f31470e1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3140120b70 con 0x7f3140131cd0 2023-09-30T22:33:49.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.996+0000 7f31470e1700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3140131790 con 0x7f314009e850 2023-09-30T22:33:49.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.996+0000 7f31470e1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f314011f050 con 0x7f314009f160 2023-09-30T22:33:49.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.996+0000 7f314567e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3140131cd0 0x7f314011ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:49.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.996+0000 7f3144e7d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f314009e850 0x7f314011d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:49.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.996+0000 7f314567e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3140131cd0 0x7f314011ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43548/0 (socket says 172.21.15.16:43548) 2023-09-30T22:33:49.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.996+0000 7f314567e700 1 -- 172.21.15.16:0/3787593278 learned_addr learned my addr 172.21.15.16:0/3787593278 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:50.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.996+0000 7f313ffff700 1 --2- 172.21.15.16:0/3787593278 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f314009f160 0x7f314011d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:50.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.997+0000 7f314567e700 1 -- 172.21.15.16:0/3787593278 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f314009f160 msgr2=0x7f314011d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:50.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.997+0000 7f314567e700 1 --2- 172.21.15.16:0/3787593278 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f314009f160 0x7f314011d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:50.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.997+0000 7f314567e700 1 -- 172.21.15.16:0/3787593278 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f314009e850 msgr2=0x7f314011d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:50.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.997+0000 7f314567e700 1 --2- 172.21.15.16:0/3787593278 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f314009e850 0x7f314011d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:50.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.997+0000 7f314567e700 1 -- 172.21.15.16:0/3787593278 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31340094a0 con 0x7f3140131cd0 2023-09-30T22:33:50.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.997+0000 7f314567e700 1 --2- 172.21.15.16:0/3787593278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3140131cd0 0x7f314011ddb0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f314009b6a0 tx=0x7f3138004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:50.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.997+0000 7f313d7fa700 1 -- 172.21.15.16:0/3787593278 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3138004240 con 0x7f3140131cd0 2023-09-30T22:33:50.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.997+0000 7f313d7fa700 1 -- 172.21.15.16:0/3787593278 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31380043e0 con 0x7f3140131cd0 2023-09-30T22:33:50.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.997+0000 7f31470e1700 1 -- 172.21.15.16:0/3787593278 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3138009b30 con 0x7f3140131cd0 2023-09-30T22:33:50.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.997+0000 7f313d7fa700 1 -- 172.21.15.16:0/3787593278 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3138004790 con 0x7f3140131cd0 2023-09-30T22:33:50.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.997+0000 7f31470e1700 1 -- 172.21.15.16:0/3787593278 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f314012dea0 con 0x7f3140131cd0 2023-09-30T22:33:50.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:49.998+0000 7f31470e1700 1 -- 172.21.15.16:0/3787593278 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31400747f0 con 0x7f3140131cd0 2023-09-30T22:33:50.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:50.003+0000 7f313d7fa700 1 -- 172.21.15.16:0/3787593278 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f313801c410 con 0x7f3140131cd0 2023-09-30T22:33:50.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:50.003+0000 7f313d7fa700 1 -- 172.21.15.16:0/3787593278 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f31380412f0 con 0x7f3140131cd0 2023-09-30T22:33:50.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:50.008+0000 7f313d7fa700 1 -- 172.21.15.16:0/3787593278 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3138023360 con 0x7f3140131cd0 2023-09-30T22:33:50.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:50.130+0000 7faf0ab22700 1 -- 172.21.15.16:0/1621748967 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7faf04136910 con 0x7faf0406fb20 2023-09-30T22:33:50.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:50.131+0000 7ff005ce9700 1 -- 172.21.15.16:0/501764724 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff000074ea0 con 0x7ff00009e000 2023-09-30T22:33:50.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:50.131+0000 7faf017fa700 1 -- 172.21.15.16:0/1621748967 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7faef8024360 con 0x7faf0406fb20 2023-09-30T22:33:50.132 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:33:50.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:50.131+0000 7fefe7fff700 1 -- 172.21.15.16:0/501764724 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7feff400f2e0 con 0x7ff00009e000 2023-09-30T22:33:50.132 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:33:50.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:50.133+0000 7faf0ab22700 1 -- 172.21.15.16:0/1621748967 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf0406fb20 msgr2=0x7faf0413aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:50.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:50.133+0000 7faf0ab22700 1 --2- 172.21.15.16:0/1621748967 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf0406fb20 0x7faf0413aca0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7faef8013f70 tx=0x7faef8011e30 comp rx=0 tx=0).stop 2023-09-30T22:33:50.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:50.134+0000 7faf0ab22700 1 -- 172.21.15.16:0/1621748967 shutdown_connections 2023-09-30T22:33:50.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:50.134+0000 7faf0ab22700 1 --2- 172.21.15.16:0/1621748967 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf0406fb20 0x7faf0413aca0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:50.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:50.134+0000 7faf0ab22700 1 --2- 172.21.15.16:0/1621748967 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf04130e50 0x7faf04138070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:50.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:50.134+0000 7faf0ab22700 1 --2- 172.21.15.16:0/1621748967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf04130540 0x7faf04137b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:50.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:50.134+0000 7faf0ab22700 1 -- 172.21.15.16:0/1621748967 >> 172.21.15.16:0/1621748967 conn(0x7faf040a2aa0 msgr2=0x7faf0406e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:50.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:50.134+0000 7ff005ce9700 1 -- 172.21.15.16:0/501764724 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff00009e000 msgr2=0x7ff0001a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:50.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:50.134+0000 7ff005ce9700 1 --2- 172.21.15.16:0/501764724 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff00009e000 0x7ff0001a6290 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7feff400b500 tx=0x7feff400b810 comp rx=0 tx=0).stop 2023-09-30T22:33:50.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:50.134+0000 7faf0ab22700 1 -- 172.21.15.16:0/1621748967 shutdown_connections 2023-09-30T22:33:50.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:50.134+0000 7faf0ab22700 1 -- 172.21.15.16:0/1621748967 wait complete. 2023-09-30T22:33:50.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:50.134+0000 7ff005ce9700 1 -- 172.21.15.16:0/501764724 shutdown_connections 2023-09-30T22:33:50.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:50.134+0000 7ff005ce9700 1 --2- 172.21.15.16:0/501764724 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff00009e410 0x7ff0001a67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:50.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:50.134+0000 7ff005ce9700 1 --2- 172.21.15.16:0/501764724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff00009d6f0 0x7ff0001a5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:50.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:50.134+0000 7ff005ce9700 1 --2- 172.21.15.16:0/501764724 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff00009e000 0x7ff0001a6290 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:50.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:50.134+0000 7ff005ce9700 1 -- 172.21.15.16:0/501764724 >> 172.21.15.16:0/501764724 conn(0x7ff000066f50 msgr2=0x7ff0001bae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:50.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:50.135+0000 7ff005ce9700 1 -- 172.21.15.16:0/501764724 shutdown_connections 2023-09-30T22:33:50.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:50.135+0000 7ff005ce9700 1 -- 172.21.15.16:0/501764724 wait complete. 2023-09-30T22:33:50.149 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:33:50.156 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:33:50.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:50.200+0000 7fd6e5446700 1 -- 172.21.15.16:0/1233591102 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd6e006c360 con 0x7fd6e009e410 2023-09-30T22:33:50.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:50.201+0000 7fd6c77fe700 1 -- 172.21.15.16:0/1233591102 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd6d80263e0 con 0x7fd6e009e410 2023-09-30T22:33:50.202 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:33:50.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:50.203+0000 7fd6e5446700 1 -- 172.21.15.16:0/1233591102 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd6e009e410 msgr2=0x7fd6e01a6a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:50.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:50.203+0000 7fd6e5446700 1 --2- 172.21.15.16:0/1233591102 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd6e009e410 0x7fd6e01a6a00 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8009d90 tx=0x7fd6d8004ad0 comp rx=0 tx=0).stop 2023-09-30T22:33:50.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:50.203+0000 7fd6e5446700 1 -- 172.21.15.16:0/1233591102 shutdown_connections 2023-09-30T22:33:50.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:50.203+0000 7fd6e5446700 1 --2- 172.21.15.16:0/1233591102 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd6e009e410 0x7fd6e01a6a00 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:50.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:50.203+0000 7fd6e5446700 1 --2- 172.21.15.16:0/1233591102 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6e009d6f0 0x7fd6e01a5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:50.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:50.203+0000 7fd6e5446700 1 --2- 172.21.15.16:0/1233591102 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6e009e000 0x7fd6e01a64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:50.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:50.203+0000 7fd6e5446700 1 -- 172.21.15.16:0/1233591102 >> 172.21.15.16:0/1233591102 conn(0x7fd6e0066f50 msgr2=0x7fd6e01baf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:50.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:50.203+0000 7fd6e5446700 1 -- 172.21.15.16:0/1233591102 shutdown_connections 2023-09-30T22:33:50.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:50.203+0000 7fd6e5446700 1 -- 172.21.15.16:0/1233591102 wait complete. 2023-09-30T22:33:50.217 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:33:50.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:50.227+0000 7f31470e1700 1 -- 172.21.15.16:0/3787593278 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f314009b780 con 0x7f3140131cd0 2023-09-30T22:33:50.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:50.227+0000 7f313d7fa700 1 -- 172.21.15.16:0/3787593278 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3138032090 con 0x7f3140131cd0 2023-09-30T22:33:50.228 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:33:50.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:50.230+0000 7f31470e1700 1 -- 172.21.15.16:0/3787593278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3140131cd0 msgr2=0x7f314011ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:50.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:50.230+0000 7f31470e1700 1 --2- 172.21.15.16:0/3787593278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3140131cd0 0x7f314011ddb0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f314009b6a0 tx=0x7f3138004ad0 comp rx=0 tx=0).stop 2023-09-30T22:33:50.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:50.230+0000 7f31470e1700 1 -- 172.21.15.16:0/3787593278 shutdown_connections 2023-09-30T22:33:50.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:50.230+0000 7f31470e1700 1 --2- 172.21.15.16:0/3787593278 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f314009e850 0x7f314011d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:50.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:50.230+0000 7f31470e1700 1 --2- 172.21.15.16:0/3787593278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3140131cd0 0x7f314011ddb0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:50.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:50.230+0000 7f31470e1700 1 --2- 172.21.15.16:0/3787593278 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f314009f160 0x7f314011d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:50.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:50.230+0000 7f31470e1700 1 -- 172.21.15.16:0/3787593278 >> 172.21.15.16:0/3787593278 conn(0x7f31400a3f70 msgr2=0x7f31400a1e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:50.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:50.230+0000 7f31470e1700 1 -- 172.21.15.16:0/3787593278 shutdown_connections 2023-09-30T22:33:50.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:50.230+0000 7f31470e1700 1 -- 172.21.15.16:0/3787593278 wait complete. 2023-09-30T22:33:50.244 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:33:51.150 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:33:51.156 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:33:51.218 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:33:51.244 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:33:51.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.308+0000 7fcf3352d700 1 Processor -- start 2023-09-30T22:33:51.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.308+0000 7fcf3352d700 1 -- start start 2023-09-30T22:33:51.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.309+0000 7fcf3352d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcf2c06e570 0x7fcf2c06e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:51.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.309+0000 7fcf3352d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcf2c06ee80 0x7fcf2c134d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:51.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.309+0000 7fcf3352d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf2c070ee0 con 0x7fcf2c09eca0 2023-09-30T22:33:51.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.309+0000 7fcf3352d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf2c071480 con 0x7fcf2c06ee80 2023-09-30T22:33:51.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.309+0000 7fcf3352d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2c071b90 con 0x7fcf2c06e570 2023-09-30T22:33:51.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.309+0000 7fcf312c9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcf2c06e570 0x7fcf2c06e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:51.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.309+0000 7fcf312c9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcf2c06e570 0x7fcf2c06e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37590/0 (socket says 172.21.15.16:37590) 2023-09-30T22:33:51.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.309+0000 7fcf312c9700 1 -- 172.21.15.16:0/3200548937 learned_addr learned my addr 172.21.15.16:0/3200548937 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:51.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.309+0000 7fcf31aca700 1 --2- 172.21.15.16:0/3200548937 >> v2:172.21.15.16:3300/0 conn(0x7fcf2c06ee80 0x7fcf2c134d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:51.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.309+0000 7fcf312c9700 1 -- 172.21.15.16:0/3200548937 >> v2:172.21.15.16:3300/0 conn(0x7fcf2c06ee80 msgr2=0x7fcf2c134d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:51.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.309+0000 7fcf312c9700 1 --2- 172.21.15.16:0/3200548937 >> v2:172.21.15.16:3300/0 conn(0x7fcf2c06ee80 0x7fcf2c134d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:51.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.309+0000 7fcf312c9700 1 -- 172.21.15.16:0/3200548937 >> v1:172.21.15.16:6789/0 conn(0x7fcf2c09eca0 legacy=0x7fcf2c134610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:51.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.309+0000 7fcf312c9700 1 -- 172.21.15.16:0/3200548937 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf2c0711a0 con 0x7fcf2c06e570 2023-09-30T22:33:51.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.310+0000 7fcf312c9700 1 --2- 172.21.15.16:0/3200548937 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcf2c06e570 0x7fcf2c06e940 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fcf280098f0 tx=0x7fcf28009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c75f1a13cbd8e121 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:51.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.310+0000 7fcf1ffff700 1 -- 172.21.15.16:0/3200548937 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf28011830 con 0x7fcf2c06e570 2023-09-30T22:33:51.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.310+0000 7fcf1ffff700 1 -- 172.21.15.16:0/3200548937 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf28011990 con 0x7fcf2c06e570 2023-09-30T22:33:51.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.310+0000 7fcf1ffff700 1 -- 172.21.15.16:0/3200548937 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf28011d20 con 0x7fcf2c06e570 2023-09-30T22:33:51.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.311+0000 7fcf3352d700 1 -- 172.21.15.16:0/3200548937 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcf2c06e570 msgr2=0x7fcf2c06e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:51.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.311+0000 7fcf3352d700 1 --2- 172.21.15.16:0/3200548937 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcf2c06e570 0x7fcf2c06e940 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fcf280098f0 tx=0x7fcf28009c00 comp rx=0 tx=0).stop 2023-09-30T22:33:51.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.311+0000 7fcf3352d700 1 -- 172.21.15.16:0/3200548937 shutdown_connections 2023-09-30T22:33:51.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.311+0000 7fcf3352d700 1 --2- 172.21.15.16:0/3200548937 >> v2:172.21.15.16:3300/0 conn(0x7fcf2c06ee80 0x7fcf2c134d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:51.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.311+0000 7fcf3352d700 1 --2- 172.21.15.16:0/3200548937 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcf2c06e570 0x7fcf2c06e940 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:51.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.311+0000 7fcf3352d700 1 -- 172.21.15.16:0/3200548937 >> 172.21.15.16:0/3200548937 conn(0x7fcf2c0a2aa0 msgr2=0x7fcf2c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:51.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.311+0000 7fcf3352d700 1 -- 172.21.15.16:0/3200548937 shutdown_connections 2023-09-30T22:33:51.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.311+0000 7fcf3352d700 1 -- 172.21.15.16:0/3200548937 wait complete. 2023-09-30T22:33:51.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.313+0000 7fcf3352d700 1 Processor -- start 2023-09-30T22:33:51.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.313+0000 7fcf3352d700 1 -- start start 2023-09-30T22:33:51.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.313+0000 7fcf3352d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcf2c06e570 0x7fcf2c1232e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:51.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.313+0000 7fcf3352d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcf2c06ee80 0x7fcf2c123820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:51.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.313+0000 7fcf3352d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf2c123d60 0x7fcf2c1cca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:51.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.313+0000 7fcf3352d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2c1247b0 con 0x7fcf2c123d60 2023-09-30T22:33:51.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.313+0000 7fcf3352d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2c126a90 con 0x7fcf2c06e570 2023-09-30T22:33:51.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.313+0000 7fcf3352d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2c124f70 con 0x7fcf2c06ee80 2023-09-30T22:33:51.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.313+0000 7fcf312c9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcf2c06e570 0x7fcf2c1232e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:51.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.313+0000 7fcf30ac8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcf2c06ee80 0x7fcf2c123820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:51.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.313+0000 7fcf312c9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcf2c06e570 0x7fcf2c1232e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37542/0 (socket says 172.21.15.16:37542) 2023-09-30T22:33:51.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.313+0000 7fcf312c9700 1 -- 172.21.15.16:0/877840359 learned_addr learned my addr 172.21.15.16:0/877840359 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:51.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.314+0000 7fcf31aca700 1 --2- 172.21.15.16:0/877840359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf2c123d60 0x7fcf2c1cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:51.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.314+0000 7fcf312c9700 1 -- 172.21.15.16:0/877840359 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcf2c06ee80 msgr2=0x7fcf2c123820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:51.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.314+0000 7fcf312c9700 1 --2- 172.21.15.16:0/877840359 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcf2c06ee80 0x7fcf2c123820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:51.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.314+0000 7fcf312c9700 1 -- 172.21.15.16:0/877840359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf2c123d60 msgr2=0x7fcf2c1cca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:51.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.314+0000 7fcf312c9700 1 --2- 172.21.15.16:0/877840359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf2c123d60 0x7fcf2c1cca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:51.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.314+0000 7fcf312c9700 1 -- 172.21.15.16:0/877840359 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf28009550 con 0x7fcf2c06e570 2023-09-30T22:33:51.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.314+0000 7fcf30ac8700 1 --2- 172.21.15.16:0/877840359 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcf2c06ee80 0x7fcf2c123820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:51.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.314+0000 7fcf31aca700 1 --2- 172.21.15.16:0/877840359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf2c123d60 0x7fcf2c1cca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:33:51.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.314+0000 7fcf312c9700 1 --2- 172.21.15.16:0/877840359 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcf2c06e570 0x7fcf2c1232e0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fcf28000c00 tx=0x7fcf28004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:51.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.314+0000 7fcf1dffb700 1 -- 172.21.15.16:0/877840359 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf28004480 con 0x7fcf2c06e570 2023-09-30T22:33:51.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.314+0000 7fcf1dffb700 1 -- 172.21.15.16:0/877840359 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf2801f070 con 0x7fcf2c06e570 2023-09-30T22:33:51.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.314+0000 7fcf3352d700 1 -- 172.21.15.16:0/877840359 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf2c135be0 con 0x7fcf2c06e570 2023-09-30T22:33:51.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.314+0000 7fcf1dffb700 1 -- 172.21.15.16:0/877840359 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf280038c0 con 0x7fcf2c06e570 2023-09-30T22:33:51.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.315+0000 7fcf3352d700 1 -- 172.21.15.16:0/877840359 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf2c070670 con 0x7fcf2c06e570 2023-09-30T22:33:51.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.316+0000 7fcf3352d700 1 -- 172.21.15.16:0/877840359 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf2c074ea0 con 0x7fcf2c06e570 2023-09-30T22:33:51.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.316+0000 7fcf1dffb700 1 -- 172.21.15.16:0/877840359 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcf28021030 con 0x7fcf2c06e570 2023-09-30T22:33:51.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.316+0000 7fcf1dffb700 1 -- 172.21.15.16:0/877840359 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fcf28040bc0 con 0x7fcf2c06e570 2023-09-30T22:33:51.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.325+0000 7fcf1dffb700 1 -- 172.21.15.16:0/877840359 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcf280199a0 con 0x7fcf2c06e570 2023-09-30T22:33:51.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.361+0000 7f05c871b700 1 Processor -- start 2023-09-30T22:33:51.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.361+0000 7f05c871b700 1 -- start start 2023-09-30T22:33:51.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.362+0000 7f05c871b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f05c0130750 0x7f05c0130b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:51.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.362+0000 7f05c871b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f05c009d6f0 0x7f05c009db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:51.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.362+0000 7f05c871b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05c009c190 con 0x7f05c009e080 2023-09-30T22:33:51.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.362+0000 7f05c871b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05c009b090 con 0x7f05c0130750 2023-09-30T22:33:51.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.362+0000 7f05c871b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05c009a6c0 con 0x7f05c009d6f0 2023-09-30T22:33:51.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.362+0000 7f05c5cb6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f05c009d6f0 0x7f05c009db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:51.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.362+0000 7f05c64b7700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f05c0130750 0x7f05c0130b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:51.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.362+0000 7f05c6cb8700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f05c009e080 0x7f05c01374b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50042/0 (socket says 172.21.15.16:50042) 2023-09-30T22:33:51.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.362+0000 7f05c5cb6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f05c009d6f0 0x7f05c009db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37558/0 (socket says 172.21.15.16:37558) 2023-09-30T22:33:51.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.362+0000 7f05c6cb8700 1 -- 172.21.15.16:0/3620270720 learned_addr learned my addr 172.21.15.16:0/3620270720 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:51.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.362+0000 7f05c64b7700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f05c0130750 0x7f05c0130b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43562/0 (socket says 172.21.15.16:43562) 2023-09-30T22:33:51.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.362+0000 7f05c54b5700 1 -- 172.21.15.16:0/3620270720 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3500140734 0 0) 0x7f05c009c190 con 0x7f05c009e080 2023-09-30T22:33:51.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.362+0000 7f05c64b7700 1 -- 172.21.15.16:0/3620270720 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f05c009d6f0 msgr2=0x7f05c009db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:51.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.363+0000 7f05c64b7700 1 --2- 172.21.15.16:0/3620270720 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f05c009d6f0 0x7f05c009db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:51.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.363+0000 7f05c64b7700 1 -- 172.21.15.16:0/3620270720 >> v1:172.21.15.16:6789/0 conn(0x7f05c009e080 legacy=0x7f05c01374b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:51.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.363+0000 7f05c64b7700 1 -- 172.21.15.16:0/3620270720 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05c009e500 con 0x7f05c0130750 2023-09-30T22:33:51.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.363+0000 7f05c64b7700 1 --2- 172.21.15.16:0/3620270720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05c0130750 0x7f05c0130b20 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f05bc009840 tx=0x7f05bc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e96cd3de29e5e8a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:51.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.363+0000 7f05c54b5700 1 -- 172.21.15.16:0/3620270720 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05bc011830 con 0x7f05c0130750 2023-09-30T22:33:51.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.363+0000 7f05c54b5700 1 -- 172.21.15.16:0/3620270720 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05bc011990 con 0x7f05c0130750 2023-09-30T22:33:51.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.363+0000 7f05c54b5700 1 -- 172.21.15.16:0/3620270720 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05bc011d20 con 0x7f05c0130750 2023-09-30T22:33:51.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.364+0000 7f05c871b700 1 -- 172.21.15.16:0/3620270720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05c0130750 msgr2=0x7f05c0130b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:51.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.364+0000 7f05c871b700 1 --2- 172.21.15.16:0/3620270720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05c0130750 0x7f05c0130b20 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f05bc009840 tx=0x7f05bc009b50 comp rx=0 tx=0).stop 2023-09-30T22:33:51.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.364+0000 7f05c871b700 1 -- 172.21.15.16:0/3620270720 shutdown_connections 2023-09-30T22:33:51.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.364+0000 7f05c871b700 1 --2- 172.21.15.16:0/3620270720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05c0130750 0x7f05c0130b20 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:51.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.364+0000 7f05c871b700 1 --2- 172.21.15.16:0/3620270720 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f05c009d6f0 0x7f05c009db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:51.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.364+0000 7f05c871b700 1 -- 172.21.15.16:0/3620270720 >> 172.21.15.16:0/3620270720 conn(0x7f05c009f470 msgr2=0x7f05c00a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:51.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.364+0000 7f05c871b700 1 -- 172.21.15.16:0/3620270720 shutdown_connections 2023-09-30T22:33:51.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.364+0000 7f05c871b700 1 -- 172.21.15.16:0/3620270720 wait complete. 2023-09-30T22:33:51.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.365+0000 7f05c871b700 1 Processor -- start 2023-09-30T22:33:51.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.365+0000 7f05c871b700 1 -- start start 2023-09-30T22:33:51.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.365+0000 7f05c871b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05c009d6f0 0x7f05c00711d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:51.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.365+0000 7f05c871b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f05c009e080 0x7f05c0071710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:51.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.365+0000 7f05c871b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f05c0130750 0x7f05c01c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:51.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.365+0000 7f05c871b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05c009d010 con 0x7f05c009d6f0 2023-09-30T22:33:51.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.365+0000 7f05c871b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05c0070df0 con 0x7f05c009e080 2023-09-30T22:33:51.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.365+0000 7f05c871b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f05c0071c50 con 0x7f05c0130750 2023-09-30T22:33:51.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.366+0000 7f05c5cb6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f05c009e080 0x7f05c0071710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:51.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.366+0000 7f05c5cb6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f05c009e080 0x7f05c0071710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37566/0 (socket says 172.21.15.16:37566) 2023-09-30T22:33:51.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.366+0000 7f05c5cb6700 1 -- 172.21.15.16:0/1565858686 learned_addr learned my addr 172.21.15.16:0/1565858686 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:51.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.366+0000 7f05c6cb8700 1 --2- 172.21.15.16:0/1565858686 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f05c0130750 0x7f05c01c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:51.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.366+0000 7f05c5cb6700 1 -- 172.21.15.16:0/1565858686 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f05c0130750 msgr2=0x7f05c01c8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:51.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.366+0000 7f05c5cb6700 1 --2- 172.21.15.16:0/1565858686 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f05c0130750 0x7f05c01c8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:51.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.366+0000 7f05c5cb6700 1 -- 172.21.15.16:0/1565858686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05c009d6f0 msgr2=0x7f05c00711d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:51.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.366+0000 7f05c5cb6700 1 --2- 172.21.15.16:0/1565858686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05c009d6f0 0x7f05c00711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:51.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.366+0000 7f05c5cb6700 1 -- 172.21.15.16:0/1565858686 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05bc0094a0 con 0x7f05c009e080 2023-09-30T22:33:51.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.366+0000 7f05c5cb6700 1 --2- 172.21.15.16:0/1565858686 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f05c009e080 0x7f05c0071710 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f05b000e870 tx=0x7f05b000ec30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:51.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.366+0000 7f05b6ffd700 1 -- 172.21.15.16:0/1565858686 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b0009590 con 0x7f05c009e080 2023-09-30T22:33:51.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.366+0000 7f05b6ffd700 1 -- 172.21.15.16:0/1565858686 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05b000ee70 con 0x7f05c009e080 2023-09-30T22:33:51.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.366+0000 7f05c871b700 1 -- 172.21.15.16:0/1565858686 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05c0127d50 con 0x7f05c009e080 2023-09-30T22:33:51.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.366+0000 7f05b6ffd700 1 -- 172.21.15.16:0/1565858686 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b000ca50 con 0x7f05c009e080 2023-09-30T22:33:51.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.367+0000 7f05c871b700 1 -- 172.21.15.16:0/1565858686 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05c0127180 con 0x7f05c009e080 2023-09-30T22:33:51.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.367+0000 7f05b6ffd700 1 -- 172.21.15.16:0/1565858686 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f05b000cbb0 con 0x7f05c009e080 2023-09-30T22:33:51.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.368+0000 7f05b6ffd700 1 -- 172.21.15.16:0/1565858686 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f05b003fbf0 con 0x7f05c009e080 2023-09-30T22:33:51.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.368+0000 7f05c871b700 1 -- 172.21.15.16:0/1565858686 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05c0131020 con 0x7f05c009e080 2023-09-30T22:33:51.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.373+0000 7f05b6ffd700 1 -- 172.21.15.16:0/1565858686 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f05b0017080 con 0x7f05c009e080 2023-09-30T22:33:51.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.415+0000 7f2b664d9700 1 Processor -- start 2023-09-30T22:33:51.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.415+0000 7f2b664d9700 1 -- start start 2023-09-30T22:33:51.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.416+0000 7f2b664d9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2b60136710 0x7f2b60138af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:51.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.416+0000 7f2b664d9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2b60139030 0x7f2b6013b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:51.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.416+0000 7f2b664d9700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b6006ef30 con 0x7f2b601338a0 2023-09-30T22:33:51.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.416+0000 7f2b664d9700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6006fef0 con 0x7f2b60139030 2023-09-30T22:33:51.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.416+0000 7f2b664d9700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2b601334b0 con 0x7f2b60136710 2023-09-30T22:33:51.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.416+0000 7f2b5f7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2b60136710 0x7f2b60138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:51.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.416+0000 7f2b5f7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2b60136710 0x7f2b60138af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37614/0 (socket says 172.21.15.16:37614) 2023-09-30T22:33:51.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.416+0000 7f2b5f7fe700 1 -- 172.21.15.16:0/26009245 learned_addr learned my addr 172.21.15.16:0/26009245 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:51.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.416+0000 7f2b64a76700 1 --2- 172.21.15.16:0/26009245 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2b60139030 0x7f2b6013b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:51.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.417+0000 7f2b5effd700 1 -- 172.21.15.16:0/26009245 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2604045464 0 0) 0x7f2b6006ef30 con 0x7f2b601338a0 2023-09-30T22:33:51.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.417+0000 7f2b5effd700 1 -- 172.21.15.16:0/26009245 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b480035b0 con 0x7f2b601338a0 2023-09-30T22:33:51.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.417+0000 7f2b5f7fe700 1 -- 172.21.15.16:0/26009245 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2b60139030 msgr2=0x7f2b6013b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:51.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.417+0000 7f2b5f7fe700 1 --2- 172.21.15.16:0/26009245 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2b60139030 0x7f2b6013b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:51.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.417+0000 7f2b5f7fe700 1 -- 172.21.15.16:0/26009245 >> v1:172.21.15.16:6789/0 conn(0x7f2b601338a0 legacy=0x7f2b60133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:51.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.417+0000 7f2b5f7fe700 1 -- 172.21.15.16:0/26009245 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b6006fc30 con 0x7f2b60136710 2023-09-30T22:33:51.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.417+0000 7f2b64a76700 1 --2- 172.21.15.16:0/26009245 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2b60139030 0x7f2b6013b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:51.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.417+0000 7f2b5f7fe700 1 --2- 172.21.15.16:0/26009245 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2b60136710 0x7f2b60138af0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f2b54009920 tx=0x7f2b54009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d4f8577ca496cd1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:51.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.417+0000 7f2b5effd700 1 -- 172.21.15.16:0/26009245 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b54011830 con 0x7f2b60136710 2023-09-30T22:33:51.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.417+0000 7f2b5effd700 1 -- 172.21.15.16:0/26009245 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b54011990 con 0x7f2b60136710 2023-09-30T22:33:51.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.417+0000 7f2b5effd700 1 -- 172.21.15.16:0/26009245 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b54011d20 con 0x7f2b60136710 2023-09-30T22:33:51.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.418+0000 7f2b664d9700 1 -- 172.21.15.16:0/26009245 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2b60136710 msgr2=0x7f2b60138af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:51.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.418+0000 7f2b664d9700 1 --2- 172.21.15.16:0/26009245 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2b60136710 0x7f2b60138af0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f2b54009920 tx=0x7f2b54009c30 comp rx=0 tx=0).stop 2023-09-30T22:33:51.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.418+0000 7f2b664d9700 1 -- 172.21.15.16:0/26009245 shutdown_connections 2023-09-30T22:33:51.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.418+0000 7f2b664d9700 1 --2- 172.21.15.16:0/26009245 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2b60139030 0x7f2b6013b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:51.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.418+0000 7f2b664d9700 1 --2- 172.21.15.16:0/26009245 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2b60136710 0x7f2b60138af0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:51.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.418+0000 7f2b664d9700 1 -- 172.21.15.16:0/26009245 >> 172.21.15.16:0/26009245 conn(0x7f2b600a2aa0 msgr2=0x7f2b600a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:51.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.418+0000 7f2b664d9700 1 -- 172.21.15.16:0/26009245 shutdown_connections 2023-09-30T22:33:51.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.418+0000 7f2b664d9700 1 -- 172.21.15.16:0/26009245 wait complete. 2023-09-30T22:33:51.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.419+0000 7f2b664d9700 1 Processor -- start 2023-09-30T22:33:51.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.419+0000 7f2b664d9700 1 -- start start 2023-09-30T22:33:51.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.420+0000 7f2b664d9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2b601338a0 0x7f2b6011ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:51.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.420+0000 7f2b664d9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b60136710 0x7f2b6011b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:51.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.420+0000 7f2b664d9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2b60139030 0x7f2b6011b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:51.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.420+0000 7f2b664d9700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b60130620 con 0x7f2b60136710 2023-09-30T22:33:51.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.420+0000 7f2b664d9700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6006f640 con 0x7f2b601338a0 2023-09-30T22:33:51.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.420+0000 7f2b664d9700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2b60138d90 con 0x7f2b60139030 2023-09-30T22:33:51.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.420+0000 7f2b64a76700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2b60139030 0x7f2b6011b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:51.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.420+0000 7f2b5f7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b60136710 0x7f2b6011b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:51.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.420+0000 7f2b5ffff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2b601338a0 0x7f2b6011ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:51.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.420+0000 7f2b64a76700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2b60139030 0x7f2b6011b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37624/0 (socket says 172.21.15.16:37624) 2023-09-30T22:33:51.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.420+0000 7f2b64a76700 1 -- 172.21.15.16:0/365604693 learned_addr learned my addr 172.21.15.16:0/365604693 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:51.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.420+0000 7f2b64a76700 1 -- 172.21.15.16:0/365604693 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2b601338a0 msgr2=0x7f2b6011ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:51.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.420+0000 7f2b64a76700 1 --2- 172.21.15.16:0/365604693 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2b601338a0 0x7f2b6011ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:51.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.420+0000 7f2b64a76700 1 -- 172.21.15.16:0/365604693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b60136710 msgr2=0x7f2b6011b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:51.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.420+0000 7f2b64a76700 1 --2- 172.21.15.16:0/365604693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b60136710 0x7f2b6011b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:51.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.420+0000 7f2b64a76700 1 -- 172.21.15.16:0/365604693 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b58009a70 con 0x7f2b60139030 2023-09-30T22:33:51.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.421+0000 7f2b64a76700 1 --2- 172.21.15.16:0/365604693 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2b60139030 0x7f2b6011b800 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f2b5800efc0 tx=0x7f2b5800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:51.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.421+0000 7f2b5cff9700 1 -- 172.21.15.16:0/365604693 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5800ca90 con 0x7f2b60139030 2023-09-30T22:33:51.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.421+0000 7f2b5cff9700 1 -- 172.21.15.16:0/365604693 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b5800f040 con 0x7f2b60139030 2023-09-30T22:33:51.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.421+0000 7f2b664d9700 1 -- 172.21.15.16:0/365604693 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b54009550 con 0x7f2b60139030 2023-09-30T22:33:51.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.421+0000 7f2b5cff9700 1 -- 172.21.15.16:0/365604693 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5800ce50 con 0x7f2b60139030 2023-09-30T22:33:51.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.421+0000 7f2b664d9700 1 -- 172.21.15.16:0/365604693 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b6011e530 con 0x7f2b60139030 2023-09-30T22:33:51.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.422+0000 7f2b5cff9700 1 -- 172.21.15.16:0/365604693 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2b58014480 con 0x7f2b60139030 2023-09-30T22:33:51.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.422+0000 7f2b5cff9700 1 -- 172.21.15.16:0/365604693 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f2b5803fd30 con 0x7f2b60139030 2023-09-30T22:33:51.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.422+0000 7f2b664d9700 1 -- 172.21.15.16:0/365604693 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b6011e180 con 0x7f2b60139030 2023-09-30T22:33:51.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.427+0000 7f2b5cff9700 1 -- 172.21.15.16:0/365604693 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2b58012930 con 0x7f2b60139030 2023-09-30T22:33:51.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.471+0000 7fc289fbe700 1 Processor -- start 2023-09-30T22:33:51.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.471+0000 7fc289fbe700 1 -- start start 2023-09-30T22:33:51.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.472+0000 7fc289fbe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc28409e850 0x7fc28409ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:51.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.472+0000 7fc289fbe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc28409f160 0x7fc28413c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:51.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.472+0000 7fc289fbe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc28409f590 0x7fc28413e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:51.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.472+0000 7fc289fbe700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc28406c930 con 0x7fc28409e850 2023-09-30T22:33:51.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.472+0000 7fc289fbe700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc28409aa30 con 0x7fc28409f160 2023-09-30T22:33:51.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.472+0000 7fc289fbe700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc28406bf30 con 0x7fc28409f590 2023-09-30T22:33:51.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.472+0000 7fc2837fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc28409e850 0x7fc28409ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:51.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.472+0000 7fc283fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc28409f590 0x7fc28413e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:51.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.472+0000 7fc2837fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc28409e850 0x7fc28409ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43578/0 (socket says 172.21.15.16:43578) 2023-09-30T22:33:51.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.472+0000 7fc2837fe700 1 -- 172.21.15.16:0/196062129 learned_addr learned my addr 172.21.15.16:0/196062129 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:51.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.472+0000 7fc283fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc28409f590 0x7fc28413e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37632/0 (socket says 172.21.15.16:37632) 2023-09-30T22:33:51.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.473+0000 7fc282ffd700 1 --2- 172.21.15.16:0/196062129 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc28409f160 0x7fc28413c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:51.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.473+0000 7fc2837fe700 1 -- 172.21.15.16:0/196062129 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc28409f590 msgr2=0x7fc28413e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:51.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.473+0000 7fc2837fe700 1 --2- 172.21.15.16:0/196062129 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc28409f590 0x7fc28413e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:51.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.473+0000 7fc2837fe700 1 -- 172.21.15.16:0/196062129 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc28409f160 msgr2=0x7fc28413c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:51.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.473+0000 7fc2837fe700 1 --2- 172.21.15.16:0/196062129 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc28409f160 0x7fc28413c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:51.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.473+0000 7fc2837fe700 1 -- 172.21.15.16:0/196062129 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2840a0250 con 0x7fc28409e850 2023-09-30T22:33:51.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.473+0000 7fc283fff700 1 --2- 172.21.15.16:0/196062129 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc28409f590 0x7fc28413e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:51.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.473+0000 7fc2837fe700 1 --2- 172.21.15.16:0/196062129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc28409e850 0x7fc28409ec20 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fc2740098f0 tx=0x7fc274009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2b3a118735e8d04 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:51.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.473+0000 7fc2827fc700 1 -- 172.21.15.16:0/196062129 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc274011830 con 0x7fc28409e850 2023-09-30T22:33:51.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.473+0000 7fc2827fc700 1 -- 172.21.15.16:0/196062129 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc274011990 con 0x7fc28409e850 2023-09-30T22:33:51.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.473+0000 7fc2827fc700 1 -- 172.21.15.16:0/196062129 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc274011d20 con 0x7fc28409e850 2023-09-30T22:33:51.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.474+0000 7fc289fbe700 1 -- 172.21.15.16:0/196062129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc28409e850 msgr2=0x7fc28409ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:51.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.474+0000 7fc289fbe700 1 --2- 172.21.15.16:0/196062129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc28409e850 0x7fc28409ec20 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fc2740098f0 tx=0x7fc274009c00 comp rx=0 tx=0).stop 2023-09-30T22:33:51.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.474+0000 7fc289fbe700 1 -- 172.21.15.16:0/196062129 shutdown_connections 2023-09-30T22:33:51.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.474+0000 7fc289fbe700 1 --2- 172.21.15.16:0/196062129 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc28409f590 0x7fc28413e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:51.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.474+0000 7fc289fbe700 1 --2- 172.21.15.16:0/196062129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc28409e850 0x7fc28409ec20 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:51.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.474+0000 7fc289fbe700 1 --2- 172.21.15.16:0/196062129 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc28409f160 0x7fc28413c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:51.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.474+0000 7fc289fbe700 1 -- 172.21.15.16:0/196062129 >> 172.21.15.16:0/196062129 conn(0x7fc2840a3f70 msgr2=0x7fc2840a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:51.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.474+0000 7fc289fbe700 1 -- 172.21.15.16:0/196062129 shutdown_connections 2023-09-30T22:33:51.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.474+0000 7fc289fbe700 1 -- 172.21.15.16:0/196062129 wait complete. 2023-09-30T22:33:51.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.475+0000 7fc289fbe700 1 Processor -- start 2023-09-30T22:33:51.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.476+0000 7fc289fbe700 1 -- start start 2023-09-30T22:33:51.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.476+0000 7fc289fbe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc28409e850 0x7fc28411a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:51.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.476+0000 7fc289fbe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc28409f160 0x7fc28411ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:51.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.476+0000 7fc289fbe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc28409f590 0x7fc28411b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:51.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.476+0000 7fc289fbe700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc28411b6e0 con 0x7fc28409f160 2023-09-30T22:33:51.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.476+0000 7fc289fbe700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc28411dcc0 con 0x7fc28409e850 2023-09-30T22:33:51.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.476+0000 7fc289fbe700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc28411ccf0 con 0x7fc28409f590 2023-09-30T22:33:51.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.477+0000 7fc2837fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc28409e850 0x7fc28411a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:51.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.477+0000 7fc283fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc28409f590 0x7fc28411b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:51.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.477+0000 7fc2837fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc28409e850 0x7fc28411a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37610/0 (socket says 172.21.15.16:37610) 2023-09-30T22:33:51.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.477+0000 7fc2837fe700 1 -- 172.21.15.16:0/1378853142 learned_addr learned my addr 172.21.15.16:0/1378853142 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:51.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.477+0000 7fc282ffd700 1 --2- 172.21.15.16:0/1378853142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc28409f160 0x7fc28411ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:51.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.477+0000 7fc283fff700 1 -- 172.21.15.16:0/1378853142 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc28409e850 msgr2=0x7fc28411a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:51.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.477+0000 7fc283fff700 1 --2- 172.21.15.16:0/1378853142 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc28409e850 0x7fc28411a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:51.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.477+0000 7fc283fff700 1 -- 172.21.15.16:0/1378853142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc28409f160 msgr2=0x7fc28411ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:51.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.477+0000 7fc283fff700 1 --2- 172.21.15.16:0/1378853142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc28409f160 0x7fc28411ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:51.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.477+0000 7fc283fff700 1 -- 172.21.15.16:0/1378853142 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc27c009a70 con 0x7fc28409f590 2023-09-30T22:33:51.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.477+0000 7fc282ffd700 1 --2- 172.21.15.16:0/1378853142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc28409f160 0x7fc28411ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:33:51.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.477+0000 7fc2837fe700 1 --2- 172.21.15.16:0/1378853142 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc28409e850 0x7fc28411a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:51.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.477+0000 7fc283fff700 1 --2- 172.21.15.16:0/1378853142 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc28409f590 0x7fc28411b0a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fc27c00ef90 tx=0x7fc27c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:51.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.477+0000 7fc26bfff700 1 -- 172.21.15.16:0/1378853142 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27c00cd40 con 0x7fc28409f590 2023-09-30T22:33:51.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.478+0000 7fc26bfff700 1 -- 172.21.15.16:0/1378853142 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc27c00f040 con 0x7fc28409f590 2023-09-30T22:33:51.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.478+0000 7fc26bfff700 1 -- 172.21.15.16:0/1378853142 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27c0146c0 con 0x7fc28409f590 2023-09-30T22:33:51.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.478+0000 7fc289fbe700 1 -- 172.21.15.16:0/1378853142 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc274009550 con 0x7fc28409f590 2023-09-30T22:33:51.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.478+0000 7fc289fbe700 1 -- 172.21.15.16:0/1378853142 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc28406f8e0 con 0x7fc28409f590 2023-09-30T22:33:51.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.479+0000 7fc289fbe700 1 -- 172.21.15.16:0/1378853142 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc28411cb30 con 0x7fc28409f590 2023-09-30T22:33:51.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.479+0000 7fc26bfff700 1 -- 172.21.15.16:0/1378853142 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc27c014820 con 0x7fc28409f590 2023-09-30T22:33:51.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.480+0000 7fc26bfff700 1 -- 172.21.15.16:0/1378853142 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fc27c0402e0 con 0x7fc28409f590 2023-09-30T22:33:51.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.489+0000 7fc26bfff700 1 -- 172.21.15.16:0/1378853142 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc27c007920 con 0x7fc28409f590 2023-09-30T22:33:51.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.603+0000 7fcf3352d700 1 -- 172.21.15.16:0/877840359 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fcf2c06b9e0 con 0x7fcf2c06e570 2023-09-30T22:33:51.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.603+0000 7fcf1dffb700 1 -- 172.21.15.16:0/877840359 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcf28015070 con 0x7fcf2c06e570 2023-09-30T22:33:51.604 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:33:51.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.605+0000 7fcf3352d700 1 -- 172.21.15.16:0/877840359 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcf2c06e570 msgr2=0x7fcf2c1232e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:51.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.606+0000 7fcf3352d700 1 --2- 172.21.15.16:0/877840359 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcf2c06e570 0x7fcf2c1232e0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fcf28000c00 tx=0x7fcf28004750 comp rx=0 tx=0).stop 2023-09-30T22:33:51.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.606+0000 7fcf3352d700 1 -- 172.21.15.16:0/877840359 shutdown_connections 2023-09-30T22:33:51.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.606+0000 7fcf3352d700 1 --2- 172.21.15.16:0/877840359 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcf2c06e570 0x7fcf2c1232e0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:51.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.606+0000 7fcf3352d700 1 --2- 172.21.15.16:0/877840359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf2c123d60 0x7fcf2c1cca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:51.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.606+0000 7fcf3352d700 1 --2- 172.21.15.16:0/877840359 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcf2c06ee80 0x7fcf2c123820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:51.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.606+0000 7fcf3352d700 1 -- 172.21.15.16:0/877840359 >> 172.21.15.16:0/877840359 conn(0x7fcf2c0a2aa0 msgr2=0x7fcf2c0a1410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:51.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.607+0000 7fcf3352d700 1 -- 172.21.15.16:0/877840359 shutdown_connections 2023-09-30T22:33:51.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.607+0000 7fcf3352d700 1 -- 172.21.15.16:0/877840359 wait complete. 2023-09-30T22:33:51.621 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:33:51.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.690+0000 7f05c871b700 1 -- 172.21.15.16:0/1565858686 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f05c00747f0 con 0x7f05c009e080 2023-09-30T22:33:51.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.691+0000 7f05b6ffd700 1 -- 172.21.15.16:0/1565858686 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f05b0021020 con 0x7f05c009e080 2023-09-30T22:33:51.691 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:33:51.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.693+0000 7f2b664d9700 1 -- 172.21.15.16:0/365604693 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f2b60074ea0 con 0x7f2b60139030 2023-09-30T22:33:51.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.693+0000 7f05c871b700 1 -- 172.21.15.16:0/1565858686 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f05c009e080 msgr2=0x7f05c0071710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:51.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.693+0000 7f05c871b700 1 --2- 172.21.15.16:0/1565858686 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f05c009e080 0x7f05c0071710 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f05b000e870 tx=0x7f05b000ec30 comp rx=0 tx=0).stop 2023-09-30T22:33:51.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.693+0000 7f2b5cff9700 1 -- 172.21.15.16:0/365604693 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2b5802b090 con 0x7f2b60139030 2023-09-30T22:33:51.694 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:33:51.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.693+0000 7f05c871b700 1 -- 172.21.15.16:0/1565858686 shutdown_connections 2023-09-30T22:33:51.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.693+0000 7f05c871b700 1 --2- 172.21.15.16:0/1565858686 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f05c0130750 0x7f05c01c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:51.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.693+0000 7f05c871b700 1 --2- 172.21.15.16:0/1565858686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05c009d6f0 0x7f05c00711d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:51.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.693+0000 7f05c871b700 1 --2- 172.21.15.16:0/1565858686 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f05c009e080 0x7f05c0071710 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:51.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.693+0000 7f05c871b700 1 -- 172.21.15.16:0/1565858686 >> 172.21.15.16:0/1565858686 conn(0x7f05c009f470 msgr2=0x7f05c0138a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:51.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.694+0000 7f05c871b700 1 -- 172.21.15.16:0/1565858686 shutdown_connections 2023-09-30T22:33:51.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.694+0000 7f05c871b700 1 -- 172.21.15.16:0/1565858686 wait complete. 2023-09-30T22:33:51.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.695+0000 7f2b664d9700 1 -- 172.21.15.16:0/365604693 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2b60139030 msgr2=0x7f2b6011b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:51.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.695+0000 7f2b664d9700 1 --2- 172.21.15.16:0/365604693 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2b60139030 0x7f2b6011b800 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f2b5800efc0 tx=0x7f2b5800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:33:51.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.695+0000 7f2b664d9700 1 -- 172.21.15.16:0/365604693 shutdown_connections 2023-09-30T22:33:51.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.695+0000 7f2b664d9700 1 --2- 172.21.15.16:0/365604693 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2b601338a0 0x7f2b6011ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:51.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.695+0000 7f2b664d9700 1 --2- 172.21.15.16:0/365604693 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2b60139030 0x7f2b6011b800 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:51.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.695+0000 7f2b664d9700 1 --2- 172.21.15.16:0/365604693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b60136710 0x7f2b6011b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:51.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.695+0000 7f2b664d9700 1 -- 172.21.15.16:0/365604693 >> 172.21.15.16:0/365604693 conn(0x7f2b600a2aa0 msgr2=0x7f2b601344f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:51.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.696+0000 7f2b664d9700 1 -- 172.21.15.16:0/365604693 shutdown_connections 2023-09-30T22:33:51.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.696+0000 7f2b664d9700 1 -- 172.21.15.16:0/365604693 wait complete. 2023-09-30T22:33:51.709 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:33:51.711 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:33:51.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.722+0000 7fc289fbe700 1 -- 172.21.15.16:0/1378853142 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc2840747f0 con 0x7fc28409f590 2023-09-30T22:33:51.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.723+0000 7fc26bfff700 1 -- 172.21.15.16:0/1378853142 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc27c007330 con 0x7fc28409f590 2023-09-30T22:33:51.723 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:33:51.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.725+0000 7fc289fbe700 1 -- 172.21.15.16:0/1378853142 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc28409f590 msgr2=0x7fc28411b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:51.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.725+0000 7fc289fbe700 1 --2- 172.21.15.16:0/1378853142 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc28409f590 0x7fc28411b0a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fc27c00ef90 tx=0x7fc27c00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:33:51.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.725+0000 7fc289fbe700 1 -- 172.21.15.16:0/1378853142 shutdown_connections 2023-09-30T22:33:51.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.725+0000 7fc289fbe700 1 --2- 172.21.15.16:0/1378853142 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc28409e850 0x7fc28411a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:51.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.725+0000 7fc289fbe700 1 --2- 172.21.15.16:0/1378853142 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc28409f590 0x7fc28411b0a0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:51.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.725+0000 7fc289fbe700 1 --2- 172.21.15.16:0/1378853142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc28409f160 0x7fc28411ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:51.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.725+0000 7fc289fbe700 1 -- 172.21.15.16:0/1378853142 >> 172.21.15.16:0/1378853142 conn(0x7fc2840a3f70 msgr2=0x7fc2840a2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:51.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.725+0000 7fc289fbe700 1 -- 172.21.15.16:0/1378853142 shutdown_connections 2023-09-30T22:33:51.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:51.725+0000 7fc289fbe700 1 -- 172.21.15.16:0/1378853142 wait complete. 2023-09-30T22:33:51.739 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:33:52.622 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:33:52.710 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:33:52.712 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:33:52.740 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:33:52.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.779+0000 7fa7b8e65700 1 Processor -- start 2023-09-30T22:33:52.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.780+0000 7fa7b8e65700 1 -- start start 2023-09-30T22:33:52.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.781+0000 7fa7b8e65700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa7b409d6f0 0x7fa7b409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:52.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.781+0000 7fa7b8e65700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa7b409e000 0x7fa7b41339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:52.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.781+0000 7fa7b8e65700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa7b409e410 0x7fa7b4133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:52.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.781+0000 7fa7b8e65700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7b406b460 con 0x7fa7b409d6f0 2023-09-30T22:33:52.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.781+0000 7fa7b8e65700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b409b120 con 0x7fa7b409e000 2023-09-30T22:33:52.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.781+0000 7fa7b8e65700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b409a7e0 con 0x7fa7b409e410 2023-09-30T22:33:52.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.781+0000 7fa7b2d9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa7b409e410 0x7fa7b4133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:52.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.781+0000 7fa7b1d9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa7b409e000 0x7fa7b41339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:52.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.781+0000 7fa7b1d9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa7b409e000 0x7fa7b41339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37612/0 (socket says 172.21.15.16:37612) 2023-09-30T22:33:52.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.781+0000 7fa7b2d9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa7b409e410 0x7fa7b4133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37654/0 (socket says 172.21.15.16:37654) 2023-09-30T22:33:52.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.781+0000 7fa7b1d9b700 1 -- 172.21.15.16:0/1115854692 learned_addr learned my addr 172.21.15.16:0/1115854692 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:52.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.781+0000 7fa7b259c700 1 --2- 172.21.15.16:0/1115854692 >> v2:172.21.15.16:3300/0 conn(0x7fa7b409d6f0 0x7fa7b409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:52.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.781+0000 7fa7b2d9d700 1 -- 172.21.15.16:0/1115854692 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa7b409e000 msgr2=0x7fa7b41339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:52.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.781+0000 7fa7b2d9d700 1 --2- 172.21.15.16:0/1115854692 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa7b409e000 0x7fa7b41339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:52.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.781+0000 7fa7b2d9d700 1 -- 172.21.15.16:0/1115854692 >> v2:172.21.15.16:3300/0 conn(0x7fa7b409d6f0 msgr2=0x7fa7b409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:52.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.781+0000 7fa7b2d9d700 1 --2- 172.21.15.16:0/1115854692 >> v2:172.21.15.16:3300/0 conn(0x7fa7b409d6f0 0x7fa7b409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:52.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.781+0000 7fa7b2d9d700 1 -- 172.21.15.16:0/1115854692 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7b409a500 con 0x7fa7b409e410 2023-09-30T22:33:52.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.781+0000 7fa7b1d9b700 1 --2- 172.21.15.16:0/1115854692 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa7b409e000 0x7fa7b41339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:52.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.782+0000 7fa7b2d9d700 1 --2- 172.21.15.16:0/1115854692 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa7b409e410 0x7fa7b4133f30 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac009dc0 tx=0x7fa7ac00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=594eeeb5b4f984d1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:52.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.782+0000 7fa7b159a700 1 -- 172.21.15.16:0/1115854692 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac00e070 con 0x7fa7b409e410 2023-09-30T22:33:52.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.782+0000 7fa7b159a700 1 -- 172.21.15.16:0/1115854692 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7ac00bae0 con 0x7fa7b409e410 2023-09-30T22:33:52.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.782+0000 7fa7b159a700 1 -- 172.21.15.16:0/1115854692 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac00be90 con 0x7fa7b409e410 2023-09-30T22:33:52.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.783+0000 7fa7b8e65700 1 -- 172.21.15.16:0/1115854692 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa7b409e410 msgr2=0x7fa7b4133f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:52.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.783+0000 7fa7b8e65700 1 --2- 172.21.15.16:0/1115854692 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa7b409e410 0x7fa7b4133f30 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac009dc0 tx=0x7fa7ac00b330 comp rx=0 tx=0).stop 2023-09-30T22:33:52.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.783+0000 7fa7b8e65700 1 -- 172.21.15.16:0/1115854692 shutdown_connections 2023-09-30T22:33:52.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.783+0000 7fa7b8e65700 1 --2- 172.21.15.16:0/1115854692 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa7b409e410 0x7fa7b4133f30 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:52.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.783+0000 7fa7b8e65700 1 --2- 172.21.15.16:0/1115854692 >> v2:172.21.15.16:3300/0 conn(0x7fa7b409d6f0 0x7fa7b409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:52.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.783+0000 7fa7b8e65700 1 --2- 172.21.15.16:0/1115854692 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa7b409e000 0x7fa7b41339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:52.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.783+0000 7fa7b8e65700 1 -- 172.21.15.16:0/1115854692 >> 172.21.15.16:0/1115854692 conn(0x7fa7b4066f50 msgr2=0x7fa7b409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:52.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.783+0000 7fa7b8e65700 1 -- 172.21.15.16:0/1115854692 shutdown_connections 2023-09-30T22:33:52.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.783+0000 7fa7b8e65700 1 -- 172.21.15.16:0/1115854692 wait complete. 2023-09-30T22:33:52.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.785+0000 7fa7b8e65700 1 Processor -- start 2023-09-30T22:33:52.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.785+0000 7fa7b8e65700 1 -- start start 2023-09-30T22:33:52.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.785+0000 7fa7b8e65700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa7b409d6f0 0x7fa7b41c9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:52.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.786+0000 7fa7b8e65700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa7b409e000 0x7fa7b41c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:52.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.786+0000 7fa7b8e65700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7b409e410 0x7fa7b41cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:52.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.786+0000 7fa7b8e65700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b411a850 con 0x7fa7b409e410 2023-09-30T22:33:52.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.786+0000 7fa7b8e65700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b411be00 con 0x7fa7b409d6f0 2023-09-30T22:33:52.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.786+0000 7fa7b8e65700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b41c9df0 con 0x7fa7b409e000 2023-09-30T22:33:52.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.786+0000 7fa7b2d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7b409e410 0x7fa7b41cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:52.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.786+0000 7fa7b259c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa7b409d6f0 0x7fa7b41c9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:52.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.786+0000 7fa7b2d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7b409e410 0x7fa7b41cc3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43604/0 (socket says 172.21.15.16:43604) 2023-09-30T22:33:52.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.786+0000 7fa7b259c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa7b409d6f0 0x7fa7b41c9370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37626/0 (socket says 172.21.15.16:37626) 2023-09-30T22:33:52.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.786+0000 7fa7b2d9d700 1 -- 172.21.15.16:0/1106912595 learned_addr learned my addr 172.21.15.16:0/1106912595 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:52.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.786+0000 7fa7b1d9b700 1 --2- 172.21.15.16:0/1106912595 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa7b409e000 0x7fa7b41c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:52.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.786+0000 7fa7b259c700 1 -- 172.21.15.16:0/1106912595 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa7b409e000 msgr2=0x7fa7b41c98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:52.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.786+0000 7fa7b259c700 1 --2- 172.21.15.16:0/1106912595 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa7b409e000 0x7fa7b41c98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:52.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.786+0000 7fa7b259c700 1 -- 172.21.15.16:0/1106912595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7b409e410 msgr2=0x7fa7b41cc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:52.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.786+0000 7fa7b259c700 1 --2- 172.21.15.16:0/1106912595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7b409e410 0x7fa7b41cc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:52.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.786+0000 7fa7b259c700 1 -- 172.21.15.16:0/1106912595 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a8009550 con 0x7fa7b409d6f0 2023-09-30T22:33:52.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.786+0000 7fa7b2d9d700 1 --2- 172.21.15.16:0/1106912595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7b409e410 0x7fa7b41cc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:52.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.787+0000 7fa7b259c700 1 --2- 172.21.15.16:0/1106912595 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa7b409d6f0 0x7fa7b41c9370 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fa7a400e900 tx=0x7fa7a400ec10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:52.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.787+0000 7fa7a2ffd700 1 -- 172.21.15.16:0/1106912595 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4024690 con 0x7fa7b409d6f0 2023-09-30T22:33:52.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.787+0000 7fa7a2ffd700 1 -- 172.21.15.16:0/1106912595 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7a40247f0 con 0x7fa7b409d6f0 2023-09-30T22:33:52.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.787+0000 7fa7a2ffd700 1 -- 172.21.15.16:0/1106912595 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a4024b60 con 0x7fa7b409d6f0 2023-09-30T22:33:52.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.787+0000 7fa7b8e65700 1 -- 172.21.15.16:0/1106912595 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7ac009a70 con 0x7fa7b409d6f0 2023-09-30T22:33:52.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.787+0000 7fa7b8e65700 1 -- 172.21.15.16:0/1106912595 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7b411cfa0 con 0x7fa7b409d6f0 2023-09-30T22:33:52.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.788+0000 7fa7b8e65700 1 -- 172.21.15.16:0/1106912595 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7b4074ea0 con 0x7fa7b409d6f0 2023-09-30T22:33:52.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.789+0000 7fa7a2ffd700 1 -- 172.21.15.16:0/1106912595 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa7a4024cc0 con 0x7fa7b409d6f0 2023-09-30T22:33:52.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.789+0000 7fa7a2ffd700 1 -- 172.21.15.16:0/1106912595 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fa7a4050590 con 0x7fa7b409d6f0 2023-09-30T22:33:52.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.794+0000 7fa7a2ffd700 1 -- 172.21.15.16:0/1106912595 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa7a400c3f0 con 0x7fa7b409d6f0 2023-09-30T22:33:52.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.861+0000 7fb5591d2700 1 Processor -- start 2023-09-30T22:33:52.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.861+0000 7f449c70f700 1 Processor -- start 2023-09-30T22:33:52.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.861+0000 7fb5591d2700 1 -- start start 2023-09-30T22:33:52.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.861+0000 7fb5591d2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb55409d6f0 0x7fb55409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:52.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.861+0000 7fb5591d2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb55409e000 0x7fb554130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:52.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.861+0000 7fb5591d2700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb55409e410 0x7fb554130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:52.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.861+0000 7fb5591d2700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb55409f520 con 0x7fb55409e410 2023-09-30T22:33:52.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.861+0000 7fb5591d2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb55409f130 con 0x7fb55409e000 2023-09-30T22:33:52.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.861+0000 7fb5591d2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb55409e930 con 0x7fb55409d6f0 2023-09-30T22:33:52.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.861+0000 7f449c70f700 1 -- start start 2023-09-30T22:33:52.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.861+0000 7f449c70f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f44940a0a30 0x7f44940a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:52.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.861+0000 7f449c70f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f449409d630 0x7f44940a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:52.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.861+0000 7fb55259c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb55409e000 0x7fb554130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:52.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.861+0000 7fb55259c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb55409e000 0x7fb554130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37630/0 (socket says 172.21.15.16:37630) 2023-09-30T22:33:52.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.861+0000 7fb55259c700 1 -- 172.21.15.16:0/2801583754 learned_addr learned my addr 172.21.15.16:0/2801583754 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:52.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.862+0000 7fb55359e700 1 --2- 172.21.15.16:0/2801583754 >> v2:172.21.15.16:3300/0 conn(0x7fb55409e410 0x7fb554130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:52.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.861+0000 7f449c70f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4494133930 0x7f4494133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:52.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.861+0000 7f449c70f700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44940a2a70 con 0x7f449409d630 2023-09-30T22:33:52.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.861+0000 7f449c70f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44940a01d0 con 0x7f44940a0a30 2023-09-30T22:33:52.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.861+0000 7f449c70f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f449409e2b0 con 0x7f4494133930 2023-09-30T22:33:52.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.861+0000 7f4499caa700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f449409d630 0x7f44940a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:52.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.862+0000 7f4499caa700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f449409d630 0x7f44940a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43620/0 (socket says 172.21.15.16:43620) 2023-09-30T22:33:52.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.862+0000 7f4499caa700 1 -- 172.21.15.16:0/3290297038 learned_addr learned my addr 172.21.15.16:0/3290297038 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:52.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.862+0000 7f449a4ab700 1 --2- 172.21.15.16:0/3290297038 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f44940a0a30 0x7f44940a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:52.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.862+0000 7f449acac700 1 --2- 172.21.15.16:0/3290297038 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4494133930 0x7f4494133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:52.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.862+0000 7fb552d9d700 1 --2- 172.21.15.16:0/2801583754 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb55409d6f0 0x7fb55409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:52.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.862+0000 7fb55359e700 1 -- 172.21.15.16:0/2801583754 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb55409d6f0 msgr2=0x7fb55409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:52.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.862+0000 7fb55359e700 1 --2- 172.21.15.16:0/2801583754 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb55409d6f0 0x7fb55409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:52.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.862+0000 7fb55359e700 1 -- 172.21.15.16:0/2801583754 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb55409e000 msgr2=0x7fb554130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:52.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.862+0000 7fb55359e700 1 --2- 172.21.15.16:0/2801583754 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb55409e000 0x7fb554130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:52.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.862+0000 7fb55359e700 1 -- 172.21.15.16:0/2801583754 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb55409b2d0 con 0x7fb55409e410 2023-09-30T22:33:52.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.862+0000 7f4499caa700 1 -- 172.21.15.16:0/3290297038 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4494133930 msgr2=0x7f4494133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:52.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.862+0000 7f4499caa700 1 --2- 172.21.15.16:0/3290297038 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4494133930 0x7f4494133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:52.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.862+0000 7f4499caa700 1 -- 172.21.15.16:0/3290297038 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f44940a0a30 msgr2=0x7f44940a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:52.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.862+0000 7f4499caa700 1 --2- 172.21.15.16:0/3290297038 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f44940a0a30 0x7f44940a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:52.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.862+0000 7f4499caa700 1 -- 172.21.15.16:0/3290297038 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f449409dc50 con 0x7f449409d630 2023-09-30T22:33:52.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.862+0000 7f4499caa700 1 --2- 172.21.15.16:0/3290297038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f449409d630 0x7f44940a1340 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f44840098f0 tx=0x7f4484009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b5bf8c178e35756 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:52.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.862+0000 7fb55359e700 1 --2- 172.21.15.16:0/2801583754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb55409e410 0x7fb554130ba0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fb54c00f370 tx=0x7fb54c00f680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=934f7c4d4197f98d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:52.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.863+0000 7f44994a9700 1 -- 172.21.15.16:0/3290297038 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4484011830 con 0x7f449409d630 2023-09-30T22:33:52.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.863+0000 7f44994a9700 1 -- 172.21.15.16:0/3290297038 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4484011990 con 0x7f449409d630 2023-09-30T22:33:52.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.863+0000 7f44994a9700 1 -- 172.21.15.16:0/3290297038 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4484011d20 con 0x7f449409d630 2023-09-30T22:33:52.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.863+0000 7fb551d9b700 1 -- 172.21.15.16:0/2801583754 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54c012070 con 0x7fb55409e410 2023-09-30T22:33:52.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.863+0000 7fb551d9b700 1 -- 172.21.15.16:0/2801583754 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb54c00fe70 con 0x7fb55409e410 2023-09-30T22:33:52.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.863+0000 7fb551d9b700 1 -- 172.21.15.16:0/2801583754 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54c00d2f0 con 0x7fb55409e410 2023-09-30T22:33:52.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.863+0000 7f449c70f700 1 -- 172.21.15.16:0/3290297038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f449409d630 msgr2=0x7f44940a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:52.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.863+0000 7f449c70f700 1 --2- 172.21.15.16:0/3290297038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f449409d630 0x7f44940a1340 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f44840098f0 tx=0x7f4484009c00 comp rx=0 tx=0).stop 2023-09-30T22:33:52.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.864+0000 7f449c70f700 1 -- 172.21.15.16:0/3290297038 shutdown_connections 2023-09-30T22:33:52.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.864+0000 7f449c70f700 1 --2- 172.21.15.16:0/3290297038 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f44940a0a30 0x7f44940a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:52.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.864+0000 7f449c70f700 1 --2- 172.21.15.16:0/3290297038 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4494133930 0x7f4494133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:52.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.864+0000 7f449c70f700 1 --2- 172.21.15.16:0/3290297038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f449409d630 0x7f44940a1340 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:52.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.864+0000 7f449c70f700 1 -- 172.21.15.16:0/3290297038 >> 172.21.15.16:0/3290297038 conn(0x7f44940a3050 msgr2=0x7f44940a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:52.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.864+0000 7f449c70f700 1 -- 172.21.15.16:0/3290297038 shutdown_connections 2023-09-30T22:33:52.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.864+0000 7f449c70f700 1 -- 172.21.15.16:0/3290297038 wait complete. 2023-09-30T22:33:52.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.864+0000 7fb5591d2700 1 -- 172.21.15.16:0/2801583754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb55409e410 msgr2=0x7fb554130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:52.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.864+0000 7fb5591d2700 1 --2- 172.21.15.16:0/2801583754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb55409e410 0x7fb554130ba0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fb54c00f370 tx=0x7fb54c00f680 comp rx=0 tx=0).stop 2023-09-30T22:33:52.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.864+0000 7fb5591d2700 1 -- 172.21.15.16:0/2801583754 shutdown_connections 2023-09-30T22:33:52.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.864+0000 7fb5591d2700 1 --2- 172.21.15.16:0/2801583754 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb55409e410 0x7fb554130ba0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:52.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.864+0000 7fb5591d2700 1 --2- 172.21.15.16:0/2801583754 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb55409d6f0 0x7fb55409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:52.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.864+0000 7fb5591d2700 1 --2- 172.21.15.16:0/2801583754 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb55409e000 0x7fb554130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:52.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.864+0000 7fb5591d2700 1 -- 172.21.15.16:0/2801583754 >> 172.21.15.16:0/2801583754 conn(0x7fb554066f50 msgr2=0x7fb55409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:52.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.864+0000 7fb5591d2700 1 -- 172.21.15.16:0/2801583754 shutdown_connections 2023-09-30T22:33:52.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.864+0000 7fb5591d2700 1 -- 172.21.15.16:0/2801583754 wait complete. 2023-09-30T22:33:52.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.865+0000 7f449c70f700 1 Processor -- start 2023-09-30T22:33:52.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.865+0000 7f449c70f700 1 -- start start 2023-09-30T22:33:52.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.865+0000 7f449c70f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f449409d630 0x7f449411f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:52.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.865+0000 7f449c70f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f44940a0a30 0x7f449411fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:52.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.865+0000 7f449c70f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4494133930 0x7f44941c7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:52.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.865+0000 7f449c70f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44941223d0 con 0x7f449409d630 2023-09-30T22:33:52.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.865+0000 7f449c70f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44941225b0 con 0x7f4494133930 2023-09-30T22:33:52.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.865+0000 7f449c70f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4494133420 con 0x7f44940a0a30 2023-09-30T22:33:52.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.866+0000 7f449a4ab700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f449409d630 0x7f449411f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:52.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.866+0000 7f449a4ab700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f449409d630 0x7f449411f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43626/0 (socket says 172.21.15.16:43626) 2023-09-30T22:33:52.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.866+0000 7f449a4ab700 1 -- 172.21.15.16:0/2060081639 learned_addr learned my addr 172.21.15.16:0/2060081639 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:52.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.866+0000 7f449acac700 1 --2- 172.21.15.16:0/2060081639 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4494133930 0x7f44941c7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:52.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.866+0000 7f4499caa700 1 --2- 172.21.15.16:0/2060081639 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f44940a0a30 0x7f449411fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:52.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.866+0000 7fb5591d2700 1 Processor -- start 2023-09-30T22:33:52.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.866+0000 7f449acac700 1 -- 172.21.15.16:0/2060081639 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f44940a0a30 msgr2=0x7f449411fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:52.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.866+0000 7f449acac700 1 --2- 172.21.15.16:0/2060081639 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f44940a0a30 0x7f449411fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:52.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.866+0000 7f449acac700 1 -- 172.21.15.16:0/2060081639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f449409d630 msgr2=0x7f449411f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:52.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.866+0000 7f449acac700 1 --2- 172.21.15.16:0/2060081639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f449409d630 0x7f449411f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:52.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.866+0000 7f449acac700 1 -- 172.21.15.16:0/2060081639 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4484009550 con 0x7f4494133930 2023-09-30T22:33:52.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.867+0000 7f449acac700 1 --2- 172.21.15.16:0/2060081639 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4494133930 0x7f44941c7ff0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f448c00efc0 tx=0x7f448c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:52.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.867+0000 7f448affd700 1 -- 172.21.15.16:0/2060081639 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448c00cc90 con 0x7f4494133930 2023-09-30T22:33:52.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.867+0000 7f448affd700 1 -- 172.21.15.16:0/2060081639 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f448c00f040 con 0x7f4494133930 2023-09-30T22:33:52.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.867+0000 7f449c70f700 1 -- 172.21.15.16:0/2060081639 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44940a4030 con 0x7f4494133930 2023-09-30T22:33:52.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.867+0000 7f448affd700 1 -- 172.21.15.16:0/2060081639 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448c0146c0 con 0x7f4494133930 2023-09-30T22:33:52.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.866+0000 7fb5591d2700 1 -- start start 2023-09-30T22:33:52.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.867+0000 7fb5591d2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb55409d6f0 0x7fb554127140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:52.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.867+0000 7fb5591d2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb55409e000 0x7fb554127680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:52.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.867+0000 7fb5591d2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb55409e410 0x7fb5541cc740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:52.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.867+0000 7fb5591d2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb55412a190 con 0x7fb55409e410 2023-09-30T22:33:52.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.867+0000 7fb5591d2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb55412a390 con 0x7fb55409e000 2023-09-30T22:33:52.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.867+0000 7fb5591d2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb554128480 con 0x7fb55409d6f0 2023-09-30T22:33:52.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.867+0000 7fb552d9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb55409d6f0 0x7fb554127140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:52.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.868+0000 7fb552d9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb55409d6f0 0x7fb554127140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37714/0 (socket says 172.21.15.16:37714) 2023-09-30T22:33:52.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.868+0000 7fb552d9d700 1 -- 172.21.15.16:0/1735818463 learned_addr learned my addr 172.21.15.16:0/1735818463 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:52.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.867+0000 7f449c70f700 1 -- 172.21.15.16:0/2060081639 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4494120e90 con 0x7f4494133930 2023-09-30T22:33:52.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.868+0000 7fb55359e700 1 --2- 172.21.15.16:0/1735818463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb55409e410 0x7fb5541cc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:52.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.868+0000 7fb55259c700 1 --2- 172.21.15.16:0/1735818463 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb55409e000 0x7fb554127680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:52.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.868+0000 7fb55259c700 1 -- 172.21.15.16:0/1735818463 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb55409d6f0 msgr2=0x7fb554127140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:52.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.868+0000 7fb55259c700 1 --2- 172.21.15.16:0/1735818463 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb55409d6f0 0x7fb554127140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:52.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.868+0000 7fb55259c700 1 -- 172.21.15.16:0/1735818463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb55409e410 msgr2=0x7fb5541cc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:52.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.868+0000 7fb55259c700 1 --2- 172.21.15.16:0/1735818463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb55409e410 0x7fb5541cc740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:52.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.868+0000 7fb55259c700 1 -- 172.21.15.16:0/1735818463 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb54c00f050 con 0x7fb55409e000 2023-09-30T22:33:52.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.868+0000 7fb55359e700 1 --2- 172.21.15.16:0/1735818463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb55409e410 0x7fb5541cc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:52.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.868+0000 7f449c70f700 1 -- 172.21.15.16:0/2060081639 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44941206a0 con 0x7f4494133930 2023-09-30T22:33:52.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.869+0000 7fb55259c700 1 --2- 172.21.15.16:0/1735818463 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb55409e000 0x7fb554127680 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fb54800d970 tx=0x7fb54800dc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:52.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.869+0000 7f448affd700 1 -- 172.21.15.16:0/2060081639 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f448c014820 con 0x7f4494133930 2023-09-30T22:33:52.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.869+0000 7f448affd700 1 -- 172.21.15.16:0/2060081639 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f448c040650 con 0x7f4494133930 2023-09-30T22:33:52.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.873+0000 7fb53b7fe700 1 -- 172.21.15.16:0/1735818463 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb548004440 con 0x7fb55409e000 2023-09-30T22:33:52.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.874+0000 7fb53b7fe700 1 -- 172.21.15.16:0/1735818463 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5480045a0 con 0x7fb55409e000 2023-09-30T22:33:52.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.874+0000 7fb5591d2700 1 -- 172.21.15.16:0/1735818463 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb55409c700 con 0x7fb55409e000 2023-09-30T22:33:52.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.874+0000 7fb53b7fe700 1 -- 172.21.15.16:0/1735818463 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5480038c0 con 0x7fb55409e000 2023-09-30T22:33:52.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.874+0000 7fb5591d2700 1 -- 172.21.15.16:0/1735818463 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5540a3ac0 con 0x7fb55409e000 2023-09-30T22:33:52.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.874+0000 7f448affd700 1 -- 172.21.15.16:0/2060081639 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f448c014ad0 con 0x7f4494133930 2023-09-30T22:33:52.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.875+0000 7fb53b7fe700 1 -- 172.21.15.16:0/1735818463 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb548003a20 con 0x7fb55409e000 2023-09-30T22:33:52.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.875+0000 7fb5591d2700 1 -- 172.21.15.16:0/1735818463 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb554074ea0 con 0x7fb55409e000 2023-09-30T22:33:52.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.875+0000 7fb53b7fe700 1 -- 172.21.15.16:0/1735818463 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fb54803f6e0 con 0x7fb55409e000 2023-09-30T22:33:52.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.880+0000 7fb53b7fe700 1 -- 172.21.15.16:0/1735818463 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb54800f490 con 0x7fb55409e000 2023-09-30T22:33:52.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.903+0000 7f1293135700 1 Processor -- start 2023-09-30T22:33:52.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.903+0000 7f1293135700 1 -- start start 2023-09-30T22:33:52.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.904+0000 7f1293135700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f128c09e1d0 0x7f128c135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:52.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.904+0000 7f1293135700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f128c135e50 0x7f128c138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:52.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.904+0000 7f1293135700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f128c09f130 con 0x7f128c09d6f0 2023-09-30T22:33:52.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.904+0000 7f1293135700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f128c09f520 con 0x7f128c09e1d0 2023-09-30T22:33:52.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.904+0000 7f1293135700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f128c09e930 con 0x7f128c135e50 2023-09-30T22:33:52.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.904+0000 7f1290ed1700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f128c09d6f0 0x7f128c09dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50068/0 (socket says 172.21.15.16:50068) 2023-09-30T22:33:52.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.904+0000 7f12916d2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f128c135e50 0x7f128c138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:52.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.904+0000 7f1290ed1700 1 -- 172.21.15.16:0/2798236353 learned_addr learned my addr 172.21.15.16:0/2798236353 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:52.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.904+0000 7f128bfff700 1 --2- 172.21.15.16:0/2798236353 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f128c09e1d0 0x7f128c135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:52.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.904+0000 7f128b7fe700 1 -- 172.21.15.16:0/2798236353 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3123724055 0 0) 0x7f128c09f130 con 0x7f128c09d6f0 2023-09-30T22:33:52.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.904+0000 7f128b7fe700 1 -- 172.21.15.16:0/2798236353 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f127c0035b0 con 0x7f128c09d6f0 2023-09-30T22:33:52.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.904+0000 7f12916d2700 1 -- 172.21.15.16:0/2798236353 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f128c09e1d0 msgr2=0x7f128c135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:52.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.904+0000 7f12916d2700 1 --2- 172.21.15.16:0/2798236353 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f128c09e1d0 0x7f128c135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:52.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.904+0000 7f12916d2700 1 -- 172.21.15.16:0/2798236353 >> v1:172.21.15.16:6789/0 conn(0x7f128c09d6f0 legacy=0x7f128c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:52.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.904+0000 7f12916d2700 1 -- 172.21.15.16:0/2798236353 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f128c09b2d0 con 0x7f128c135e50 2023-09-30T22:33:52.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.905+0000 7f12916d2700 1 --2- 172.21.15.16:0/2798236353 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f128c135e50 0x7f128c138230 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f128400be60 tx=0x7f12840092f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e7236373ec4809d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:52.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.905+0000 7f128b7fe700 1 -- 172.21.15.16:0/2798236353 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128400c890 con 0x7f128c135e50 2023-09-30T22:33:52.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.905+0000 7f128b7fe700 1 -- 172.21.15.16:0/2798236353 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f128400c9f0 con 0x7f128c135e50 2023-09-30T22:33:52.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.905+0000 7f128b7fe700 1 -- 172.21.15.16:0/2798236353 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128400cda0 con 0x7f128c135e50 2023-09-30T22:33:52.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.906+0000 7f1293135700 1 -- 172.21.15.16:0/2798236353 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f128c135e50 msgr2=0x7f128c138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:52.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.906+0000 7f1293135700 1 --2- 172.21.15.16:0/2798236353 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f128c135e50 0x7f128c138230 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f128400be60 tx=0x7f12840092f0 comp rx=0 tx=0).stop 2023-09-30T22:33:52.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.906+0000 7f1293135700 1 -- 172.21.15.16:0/2798236353 shutdown_connections 2023-09-30T22:33:52.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.906+0000 7f1293135700 1 --2- 172.21.15.16:0/2798236353 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f128c135e50 0x7f128c138230 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:52.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.906+0000 7f1293135700 1 --2- 172.21.15.16:0/2798236353 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f128c09e1d0 0x7f128c135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:52.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.906+0000 7f1293135700 1 -- 172.21.15.16:0/2798236353 >> 172.21.15.16:0/2798236353 conn(0x7f128c066f50 msgr2=0x7f128c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:52.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.906+0000 7f1293135700 1 -- 172.21.15.16:0/2798236353 shutdown_connections 2023-09-30T22:33:52.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.906+0000 7f1293135700 1 -- 172.21.15.16:0/2798236353 wait complete. 2023-09-30T22:33:52.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.907+0000 7f1293135700 1 Processor -- start 2023-09-30T22:33:52.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.907+0000 7f1293135700 1 -- start start 2023-09-30T22:33:52.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.908+0000 7f1293135700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f128c09d6f0 0x7f128c11c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:52.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.908+0000 7f1293135700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f128c09e1d0 0x7f128c11c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:52.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.908+0000 7f1293135700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f128c135e50 0x7f128c11cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:52.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.908+0000 7f1293135700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128c0a20b0 con 0x7f128c09e1d0 2023-09-30T22:33:52.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.908+0000 7f1293135700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f128c11f510 con 0x7f128c09d6f0 2023-09-30T22:33:52.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.908+0000 7f1293135700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f128c11faa0 con 0x7f128c135e50 2023-09-30T22:33:52.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.908+0000 7f12916d2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f128c135e50 0x7f128c11cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:52.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.908+0000 7f12916d2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f128c135e50 0x7f128c11cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37722/0 (socket says 172.21.15.16:37722) 2023-09-30T22:33:52.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.908+0000 7f12916d2700 1 -- 172.21.15.16:0/3763121292 learned_addr learned my addr 172.21.15.16:0/3763121292 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:52.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.908+0000 7f1290ed1700 1 --2- 172.21.15.16:0/3763121292 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f128c09d6f0 0x7f128c11c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:52.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.908+0000 7f128bfff700 1 --2- 172.21.15.16:0/3763121292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f128c09e1d0 0x7f128c11c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:52.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.908+0000 7f12916d2700 1 -- 172.21.15.16:0/3763121292 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f128c09d6f0 msgr2=0x7f128c11c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:52.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.908+0000 7f12916d2700 1 --2- 172.21.15.16:0/3763121292 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f128c09d6f0 0x7f128c11c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:52.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.908+0000 7f12916d2700 1 -- 172.21.15.16:0/3763121292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f128c09e1d0 msgr2=0x7f128c11c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:52.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.908+0000 7f12916d2700 1 --2- 172.21.15.16:0/3763121292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f128c09e1d0 0x7f128c11c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:52.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.908+0000 7f12916d2700 1 -- 172.21.15.16:0/3763121292 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f128400ba90 con 0x7f128c135e50 2023-09-30T22:33:52.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.909+0000 7f12916d2700 1 --2- 172.21.15.16:0/3763121292 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f128c135e50 0x7f128c11cce0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f128400bfd0 tx=0x7f1284004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:52.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.909+0000 7f12897fa700 1 -- 172.21.15.16:0/3763121292 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128400c890 con 0x7f128c135e50 2023-09-30T22:33:52.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.909+0000 7f12897fa700 1 -- 172.21.15.16:0/3763121292 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1284003e30 con 0x7f128c135e50 2023-09-30T22:33:52.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.909+0000 7f1293135700 1 -- 172.21.15.16:0/3763121292 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f128c11e660 con 0x7f128c135e50 2023-09-30T22:33:52.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.909+0000 7f12897fa700 1 -- 172.21.15.16:0/3763121292 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1284009b70 con 0x7f128c135e50 2023-09-30T22:33:52.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.909+0000 7f1293135700 1 -- 172.21.15.16:0/3763121292 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f128c131a70 con 0x7f128c135e50 2023-09-30T22:33:52.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.910+0000 7f12897fa700 1 -- 172.21.15.16:0/3763121292 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f128401a070 con 0x7f128c135e50 2023-09-30T22:33:52.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.910+0000 7f12897fa700 1 -- 172.21.15.16:0/3763121292 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f1284041080 con 0x7f128c135e50 2023-09-30T22:33:52.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.911+0000 7f1293135700 1 -- 172.21.15.16:0/3763121292 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f128c11f6d0 con 0x7f128c135e50 2023-09-30T22:33:52.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:52.915+0000 7f12897fa700 1 -- 172.21.15.16:0/3763121292 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1284018080 con 0x7f128c135e50 2023-09-30T22:33:53.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:53.007+0000 7fa7b8e65700 1 -- 172.21.15.16:0/1106912595 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa7b406c360 con 0x7fa7b409d6f0 2023-09-30T22:33:53.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:53.008+0000 7fa7a2ffd700 1 -- 172.21.15.16:0/1106912595 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa7a4028070 con 0x7fa7b409d6f0 2023-09-30T22:33:53.009 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:33:53.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:53.010+0000 7fa7b8e65700 1 -- 172.21.15.16:0/1106912595 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa7b409d6f0 msgr2=0x7fa7b41c9370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:53.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:53.010+0000 7fa7b8e65700 1 --2- 172.21.15.16:0/1106912595 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa7b409d6f0 0x7fa7b41c9370 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fa7a400e900 tx=0x7fa7a400ec10 comp rx=0 tx=0).stop 2023-09-30T22:33:53.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:53.010+0000 7fa7b8e65700 1 -- 172.21.15.16:0/1106912595 shutdown_connections 2023-09-30T22:33:53.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:53.010+0000 7fa7b8e65700 1 --2- 172.21.15.16:0/1106912595 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa7b409d6f0 0x7fa7b41c9370 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:53.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:53.010+0000 7fa7b8e65700 1 --2- 172.21.15.16:0/1106912595 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7b409e410 0x7fa7b41cc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:53.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:53.010+0000 7fa7b8e65700 1 --2- 172.21.15.16:0/1106912595 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa7b409e000 0x7fa7b41c98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:53.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:53.010+0000 7fa7b8e65700 1 -- 172.21.15.16:0/1106912595 >> 172.21.15.16:0/1106912595 conn(0x7fa7b4066f50 msgr2=0x7fa7b40a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:53.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:53.010+0000 7fa7b8e65700 1 -- 172.21.15.16:0/1106912595 shutdown_connections 2023-09-30T22:33:53.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:53.010+0000 7fa7b8e65700 1 -- 172.21.15.16:0/1106912595 wait complete. 2023-09-30T22:33:53.025 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:33:53.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:53.098+0000 7f449c70f700 1 -- 172.21.15.16:0/2060081639 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4494074770 con 0x7f4494133930 2023-09-30T22:33:53.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:53.099+0000 7f448affd700 1 -- 172.21.15.16:0/2060081639 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f448c02a020 con 0x7f4494133930 2023-09-30T22:33:53.099 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:33:53.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:53.101+0000 7f449c70f700 1 -- 172.21.15.16:0/2060081639 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4494133930 msgr2=0x7f44941c7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:53.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:53.101+0000 7f449c70f700 1 --2- 172.21.15.16:0/2060081639 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4494133930 0x7f44941c7ff0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f448c00efc0 tx=0x7f448c00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:33:53.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:53.101+0000 7f449c70f700 1 -- 172.21.15.16:0/2060081639 shutdown_connections 2023-09-30T22:33:53.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:53.101+0000 7f449c70f700 1 --2- 172.21.15.16:0/2060081639 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4494133930 0x7f44941c7ff0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:53.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:53.101+0000 7f449c70f700 1 --2- 172.21.15.16:0/2060081639 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f449409d630 0x7f449411f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:53.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:53.101+0000 7f449c70f700 1 --2- 172.21.15.16:0/2060081639 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f44940a0a30 0x7f449411fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:53.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:53.101+0000 7f449c70f700 1 -- 172.21.15.16:0/2060081639 >> 172.21.15.16:0/2060081639 conn(0x7f44940a3050 msgr2=0x7f44941bba60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:53.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:53.101+0000 7f449c70f700 1 -- 172.21.15.16:0/2060081639 shutdown_connections 2023-09-30T22:33:53.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:53.101+0000 7f449c70f700 1 -- 172.21.15.16:0/2060081639 wait complete. 2023-09-30T22:33:53.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:53.103+0000 7fb5591d2700 1 -- 172.21.15.16:0/1735818463 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb554071d40 con 0x7fb55409e000 2023-09-30T22:33:53.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:53.104+0000 7fb53b7fe700 1 -- 172.21.15.16:0/1735818463 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb54801f020 con 0x7fb55409e000 2023-09-30T22:33:53.104 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:33:53.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:53.106+0000 7fb5591d2700 1 -- 172.21.15.16:0/1735818463 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb55409e000 msgr2=0x7fb554127680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:53.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:53.106+0000 7fb5591d2700 1 --2- 172.21.15.16:0/1735818463 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb55409e000 0x7fb554127680 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fb54800d970 tx=0x7fb54800dc80 comp rx=0 tx=0).stop 2023-09-30T22:33:53.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:53.106+0000 7fb5591d2700 1 -- 172.21.15.16:0/1735818463 shutdown_connections 2023-09-30T22:33:53.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:53.106+0000 7fb5591d2700 1 --2- 172.21.15.16:0/1735818463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb55409e410 0x7fb5541cc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:53.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:53.106+0000 7fb5591d2700 1 --2- 172.21.15.16:0/1735818463 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb55409d6f0 0x7fb554127140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:53.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:53.106+0000 7fb5591d2700 1 --2- 172.21.15.16:0/1735818463 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb55409e000 0x7fb554127680 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:53.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:53.106+0000 7fb5591d2700 1 -- 172.21.15.16:0/1735818463 >> 172.21.15.16:0/1735818463 conn(0x7fb554066f50 msgr2=0x7fb5541309b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:53.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:53.106+0000 7fb5591d2700 1 -- 172.21.15.16:0/1735818463 shutdown_connections 2023-09-30T22:33:53.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:53.106+0000 7fb5591d2700 1 -- 172.21.15.16:0/1735818463 wait complete. 2023-09-30T22:33:53.117 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:33:53.121 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:33:53.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:53.123+0000 7f1293135700 1 -- 172.21.15.16:0/3763121292 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f128c074ea0 con 0x7f128c135e50 2023-09-30T22:33:53.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:53.123+0000 7f12897fa700 1 -- 172.21.15.16:0/3763121292 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f128402b030 con 0x7f128c135e50 2023-09-30T22:33:53.124 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:33:53.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:53.125+0000 7f1293135700 1 -- 172.21.15.16:0/3763121292 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f128c135e50 msgr2=0x7f128c11cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:53.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:53.125+0000 7f1293135700 1 --2- 172.21.15.16:0/3763121292 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f128c135e50 0x7f128c11cce0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f128400bfd0 tx=0x7f1284004ad0 comp rx=0 tx=0).stop 2023-09-30T22:33:53.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:53.125+0000 7f1293135700 1 -- 172.21.15.16:0/3763121292 shutdown_connections 2023-09-30T22:33:53.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:53.125+0000 7f1293135700 1 --2- 172.21.15.16:0/3763121292 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f128c09d6f0 0x7f128c11c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:53.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:53.125+0000 7f1293135700 1 --2- 172.21.15.16:0/3763121292 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f128c135e50 0x7f128c11cce0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:53.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:53.125+0000 7f1293135700 1 --2- 172.21.15.16:0/3763121292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f128c09e1d0 0x7f128c11c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:53.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:53.125+0000 7f1293135700 1 -- 172.21.15.16:0/3763121292 >> 172.21.15.16:0/3763121292 conn(0x7f128c066f50 msgr2=0x7f128c06ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:53.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:53.126+0000 7f1293135700 1 -- 172.21.15.16:0/3763121292 shutdown_connections 2023-09-30T22:33:53.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:53.126+0000 7f1293135700 1 -- 172.21.15.16:0/3763121292 wait complete. 2023-09-30T22:33:53.139 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:33:54.026 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:33:54.118 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:33:54.122 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:33:54.142 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:33:54.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.175+0000 7f685d640700 1 Processor -- start 2023-09-30T22:33:54.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.175+0000 7f685d640700 1 -- start start 2023-09-30T22:33:54.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.176+0000 7f685d640700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f68581338a0 0x7f6858133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:54.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.176+0000 7f685d640700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f685809de50 0x7f685809e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:54.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.176+0000 7f685d640700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f685809e760 0x7f685813a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:54.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.176+0000 7f685d640700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f685806fef0 con 0x7f685809e760 2023-09-30T22:33:54.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.176+0000 7f685d640700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f685806ef30 con 0x7f685809de50 2023-09-30T22:33:54.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.176+0000 7f685d640700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f68581334b0 con 0x7f68581338a0 2023-09-30T22:33:54.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.176+0000 7f6856ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f68581338a0 0x7f6858133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:54.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.176+0000 7f6856ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f68581338a0 0x7f6858133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37738/0 (socket says 172.21.15.16:37738) 2023-09-30T22:33:54.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.176+0000 7f6856ffd700 1 -- 172.21.15.16:0/3121887834 learned_addr learned my addr 172.21.15.16:0/3121887834 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:54.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.176+0000 7f68567fc700 1 --2- 172.21.15.16:0/3121887834 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f685809de50 0x7f685809e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:54.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.176+0000 7f68577fe700 1 --2- 172.21.15.16:0/3121887834 >> v2:172.21.15.16:3300/0 conn(0x7f685809e760 0x7f685813a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:54.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.176+0000 7f68577fe700 1 -- 172.21.15.16:0/3121887834 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f68581338a0 msgr2=0x7f6858133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:54.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.177+0000 7f68577fe700 1 --2- 172.21.15.16:0/3121887834 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f68581338a0 0x7f6858133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:54.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.177+0000 7f68577fe700 1 -- 172.21.15.16:0/3121887834 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f685809de50 msgr2=0x7f685809e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:54.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.177+0000 7f68577fe700 1 --2- 172.21.15.16:0/3121887834 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f685809de50 0x7f685809e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:54.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.177+0000 7f68577fe700 1 -- 172.21.15.16:0/3121887834 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f685806fc30 con 0x7f685809e760 2023-09-30T22:33:54.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.177+0000 7f68577fe700 1 --2- 172.21.15.16:0/3121887834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f685809e760 0x7f685813a730 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f6850009d90 tx=0x7f685000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=857f74a94f8c8854 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:54.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.177+0000 7f6855ffb700 1 -- 172.21.15.16:0/3121887834 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685000e070 con 0x7f685809e760 2023-09-30T22:33:54.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.177+0000 7f6855ffb700 1 -- 172.21.15.16:0/3121887834 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f685000bb20 con 0x7f685809e760 2023-09-30T22:33:54.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.177+0000 7f6855ffb700 1 -- 172.21.15.16:0/3121887834 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6850012430 con 0x7f685809e760 2023-09-30T22:33:54.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.178+0000 7f685d640700 1 -- 172.21.15.16:0/3121887834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f685809e760 msgr2=0x7f685813a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:54.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.178+0000 7f685d640700 1 --2- 172.21.15.16:0/3121887834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f685809e760 0x7f685813a730 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f6850009d90 tx=0x7f685000b330 comp rx=0 tx=0).stop 2023-09-30T22:33:54.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.178+0000 7f685d640700 1 -- 172.21.15.16:0/3121887834 shutdown_connections 2023-09-30T22:33:54.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.178+0000 7f685d640700 1 --2- 172.21.15.16:0/3121887834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f685809e760 0x7f685813a730 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:54.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.178+0000 7f685d640700 1 --2- 172.21.15.16:0/3121887834 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f68581338a0 0x7f6858133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:54.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.178+0000 7f685d640700 1 --2- 172.21.15.16:0/3121887834 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f685809de50 0x7f685809e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:54.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.178+0000 7f685d640700 1 -- 172.21.15.16:0/3121887834 >> 172.21.15.16:0/3121887834 conn(0x7f68580a2aa0 msgr2=0x7f68580a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:54.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.178+0000 7f685d640700 1 -- 172.21.15.16:0/3121887834 shutdown_connections 2023-09-30T22:33:54.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.178+0000 7f685d640700 1 -- 172.21.15.16:0/3121887834 wait complete. 2023-09-30T22:33:54.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.179+0000 7f685d640700 1 Processor -- start 2023-09-30T22:33:54.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.179+0000 7f685d640700 1 -- start start 2023-09-30T22:33:54.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.180+0000 7f685d640700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f685809de50 0x7f685811ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:54.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.180+0000 7f685d640700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f685809e760 0x7f685811ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:54.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.180+0000 7f685d640700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f68581338a0 0x7f685811f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:54.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.180+0000 7f685d640700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6858121f30 con 0x7f685809e760 2023-09-30T22:33:54.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.180+0000 7f685d640700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68581230b0 con 0x7f68581338a0 2023-09-30T22:33:54.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.180+0000 7f685d640700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f68580a0f40 con 0x7f685809de50 2023-09-30T22:33:54.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.180+0000 7f68577fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f68581338a0 0x7f685811f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:54.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.181+0000 7f68577fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f68581338a0 0x7f685811f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37722/0 (socket says 172.21.15.16:37722) 2023-09-30T22:33:54.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.181+0000 7f68577fe700 1 -- 172.21.15.16:0/857937844 learned_addr learned my addr 172.21.15.16:0/857937844 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:54.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.181+0000 7f6856ffd700 1 --2- 172.21.15.16:0/857937844 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f685809de50 0x7f685811ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:54.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.181+0000 7f68567fc700 1 --2- 172.21.15.16:0/857937844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f685809e760 0x7f685811ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:54.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.181+0000 7f68577fe700 1 -- 172.21.15.16:0/857937844 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f685809de50 msgr2=0x7f685811ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:54.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.181+0000 7f68577fe700 1 --2- 172.21.15.16:0/857937844 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f685809de50 0x7f685811ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:54.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.181+0000 7f68577fe700 1 -- 172.21.15.16:0/857937844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f685809e760 msgr2=0x7f685811ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:54.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.181+0000 7f68577fe700 1 --2- 172.21.15.16:0/857937844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f685809e760 0x7f685811ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:54.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.181+0000 7f68577fe700 1 -- 172.21.15.16:0/857937844 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6850009a70 con 0x7f68581338a0 2023-09-30T22:33:54.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.181+0000 7f6856ffd700 1 --2- 172.21.15.16:0/857937844 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f685809de50 0x7f685811ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:54.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.181+0000 7f68577fe700 1 --2- 172.21.15.16:0/857937844 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f68581338a0 0x7f685811f4b0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f6850014040 tx=0x7f6850004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:54.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.182+0000 7f683f7fe700 1 -- 172.21.15.16:0/857937844 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6850004840 con 0x7f68581338a0 2023-09-30T22:33:54.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.182+0000 7f685d640700 1 -- 172.21.15.16:0/857937844 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f685806f640 con 0x7f68581338a0 2023-09-30T22:33:54.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.182+0000 7f685d640700 1 -- 172.21.15.16:0/857937844 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6858120550 con 0x7f68581338a0 2023-09-30T22:33:54.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.182+0000 7f683f7fe700 1 -- 172.21.15.16:0/857937844 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6850020070 con 0x7f68581338a0 2023-09-30T22:33:54.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.182+0000 7f683f7fe700 1 -- 172.21.15.16:0/857937844 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6850027680 con 0x7f68581338a0 2023-09-30T22:33:54.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.183+0000 7f683f7fe700 1 -- 172.21.15.16:0/857937844 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f68500277e0 con 0x7f68581338a0 2023-09-30T22:33:54.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.183+0000 7f685d640700 1 -- 172.21.15.16:0/857937844 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68380052e0 con 0x7f68581338a0 2023-09-30T22:33:54.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.183+0000 7f683f7fe700 1 -- 172.21.15.16:0/857937844 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f6850022070 con 0x7f68581338a0 2023-09-30T22:33:54.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.192+0000 7f683f7fe700 1 -- 172.21.15.16:0/857937844 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f685002f980 con 0x7f68581338a0 2023-09-30T22:33:54.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.267+0000 7f67d9f59700 1 Processor -- start 2023-09-30T22:33:54.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.267+0000 7f67d9f59700 1 -- start start 2023-09-30T22:33:54.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.268+0000 7f67d9f59700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f67d409d6f0 0x7f67d409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:54.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.268+0000 7f67d9f59700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f67d409e410 0x7f67d4130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:54.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.268+0000 7f67d9f59700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67d409f130 con 0x7f67d409e000 2023-09-30T22:33:54.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.268+0000 7f67d9f59700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67d409f520 con 0x7f67d409e410 2023-09-30T22:33:54.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.268+0000 7f67d9f59700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f67d409e930 con 0x7f67d409d6f0 2023-09-30T22:33:54.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.268+0000 7f67d2ffd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f67d409e000 0x7f67d4130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50074/0 (socket says 172.21.15.16:50074) 2023-09-30T22:33:54.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.268+0000 7f67d2ffd700 1 -- 172.21.15.16:0/897712350 learned_addr learned my addr 172.21.15.16:0/897712350 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:54.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.268+0000 7f67d37fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f67d409d6f0 0x7f67d409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:54.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.268+0000 7f67d3fff700 1 --2- 172.21.15.16:0/897712350 >> v2:172.21.15.16:3300/0 conn(0x7f67d409e410 0x7f67d4130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:54.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.269+0000 7f67d3fff700 1 -- 172.21.15.16:0/897712350 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f67d409d6f0 msgr2=0x7f67d409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:54.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.269+0000 7f67d3fff700 1 --2- 172.21.15.16:0/897712350 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f67d409d6f0 0x7f67d409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:54.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.269+0000 7f67d3fff700 1 -- 172.21.15.16:0/897712350 >> v1:172.21.15.16:6789/0 conn(0x7f67d409e000 legacy=0x7f67d4130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:54.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.269+0000 7f67d3fff700 1 -- 172.21.15.16:0/897712350 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d409b2d0 con 0x7f67d409e410 2023-09-30T22:33:54.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.269+0000 7f67d37fe700 1 --2- 172.21.15.16:0/897712350 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f67d409d6f0 0x7f67d409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:33:54.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.269+0000 7f67d3fff700 1 --2- 172.21.15.16:0/897712350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67d409e410 0x7f67d4130d70 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f67cc009dc0 tx=0x7f67cc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c464910c3b4e6b68 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:54.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.269+0000 7f67d27fc700 1 -- 172.21.15.16:0/897712350 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67cc00e070 con 0x7f67d409e410 2023-09-30T22:33:54.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.269+0000 7f67d27fc700 1 -- 172.21.15.16:0/897712350 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67cc00bb90 con 0x7f67d409e410 2023-09-30T22:33:54.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.269+0000 7f67d27fc700 1 -- 172.21.15.16:0/897712350 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67cc012430 con 0x7f67d409e410 2023-09-30T22:33:54.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.270+0000 7f67d9f59700 1 -- 172.21.15.16:0/897712350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67d409e410 msgr2=0x7f67d4130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:54.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.270+0000 7f67d9f59700 1 --2- 172.21.15.16:0/897712350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67d409e410 0x7f67d4130d70 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f67cc009dc0 tx=0x7f67cc00b330 comp rx=0 tx=0).stop 2023-09-30T22:33:54.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.270+0000 7f67d9f59700 1 -- 172.21.15.16:0/897712350 shutdown_connections 2023-09-30T22:33:54.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.270+0000 7f67d9f59700 1 --2- 172.21.15.16:0/897712350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67d409e410 0x7f67d4130d70 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:54.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.270+0000 7f67d9f59700 1 --2- 172.21.15.16:0/897712350 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f67d409d6f0 0x7f67d409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:54.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.270+0000 7f67d9f59700 1 -- 172.21.15.16:0/897712350 >> 172.21.15.16:0/897712350 conn(0x7f67d4066f50 msgr2=0x7f67d409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:54.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.270+0000 7f67d9f59700 1 -- 172.21.15.16:0/897712350 shutdown_connections 2023-09-30T22:33:54.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.270+0000 7f67d9f59700 1 -- 172.21.15.16:0/897712350 wait complete. 2023-09-30T22:33:54.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.271+0000 7f67d9f59700 1 Processor -- start 2023-09-30T22:33:54.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.271+0000 7f67d9f59700 1 -- start start 2023-09-30T22:33:54.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.272+0000 7f67d9f59700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f67d409d6f0 0x7f67d4131630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:54.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.272+0000 7f67d9f59700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f67d409e000 0x7f67d4131b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:54.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.272+0000 7f67d9f59700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67d409e410 0x7f67d41320b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:54.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.272+0000 7f67d9f59700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d41347c0 con 0x7f67d409e410 2023-09-30T22:33:54.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.272+0000 7f67d9f59700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67d4135380 con 0x7f67d409e000 2023-09-30T22:33:54.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.272+0000 7f67d9f59700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f67d4072550 con 0x7f67d409d6f0 2023-09-30T22:33:54.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.272+0000 7f67d37fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f67d409d6f0 0x7f67d4131630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:54.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.272+0000 7f67d37fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f67d409d6f0 0x7f67d4131630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37762/0 (socket says 172.21.15.16:37762) 2023-09-30T22:33:54.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.272+0000 7f67d37fe700 1 -- 172.21.15.16:0/415311472 learned_addr learned my addr 172.21.15.16:0/415311472 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:54.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.272+0000 7f67d3fff700 1 --2- 172.21.15.16:0/415311472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67d409e410 0x7f67d41320b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:54.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.272+0000 7f67d37fe700 1 -- 172.21.15.16:0/415311472 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f67d409e000 msgr2=0x7f67d4131b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:54.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.272+0000 7f67d37fe700 1 --2- 172.21.15.16:0/415311472 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f67d409e000 0x7f67d4131b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:54.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.272+0000 7f67d37fe700 1 -- 172.21.15.16:0/415311472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67d409e410 msgr2=0x7f67d41320b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:54.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.272+0000 7f67d37fe700 1 --2- 172.21.15.16:0/415311472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67d409e410 0x7f67d41320b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:54.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.273+0000 7f67d37fe700 1 -- 172.21.15.16:0/415311472 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67cc009a70 con 0x7f67d409d6f0 2023-09-30T22:33:54.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.273+0000 7f67d3fff700 1 --2- 172.21.15.16:0/415311472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67d409e410 0x7f67d41320b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:54.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.273+0000 7f67d37fe700 1 --2- 172.21.15.16:0/415311472 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f67d409d6f0 0x7f67d4131630 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f67c400e840 tx=0x7f67c400eb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:54.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.273+0000 7f67bbfff700 1 -- 172.21.15.16:0/415311472 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c4004230 con 0x7f67d409d6f0 2023-09-30T22:33:54.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.273+0000 7f67bbfff700 1 -- 172.21.15.16:0/415311472 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67c4004390 con 0x7f67d409d6f0 2023-09-30T22:33:54.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.273+0000 7f67d9f59700 1 -- 172.21.15.16:0/415311472 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67d4132b00 con 0x7f67d409d6f0 2023-09-30T22:33:54.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.273+0000 7f67bbfff700 1 -- 172.21.15.16:0/415311472 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c40036a0 con 0x7f67d409d6f0 2023-09-30T22:33:54.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.273+0000 7f67d9f59700 1 -- 172.21.15.16:0/415311472 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67d4134a10 con 0x7f67d409d6f0 2023-09-30T22:33:54.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.274+0000 7f67bbfff700 1 -- 172.21.15.16:0/415311472 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f67c4003800 con 0x7f67d409d6f0 2023-09-30T22:33:54.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.274+0000 7f67bbfff700 1 -- 172.21.15.16:0/415311472 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f67c403eff0 con 0x7f67d409d6f0 2023-09-30T22:33:54.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.275+0000 7f67d9f59700 1 -- 172.21.15.16:0/415311472 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67d4134a10 con 0x7f67d409d6f0 2023-09-30T22:33:54.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.280+0000 7f67bbfff700 1 -- 172.21.15.16:0/415311472 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f67d4134a10 con 0x7f67d409d6f0 2023-09-30T22:33:54.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.309+0000 7fac25884700 1 Processor -- start 2023-09-30T22:33:54.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.309+0000 7fac25884700 1 -- start start 2023-09-30T22:33:54.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.309+0000 7fac25884700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fac2009ac90 0x7fac2013cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:54.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.309+0000 7fac25884700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fac2013d240 0x7fac2013f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:54.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.309+0000 7fac25884700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac20134980 con 0x7fac2006c0e0 2023-09-30T22:33:54.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.309+0000 7fac25884700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac2006c4b0 con 0x7fac2009ac90 2023-09-30T22:33:54.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.309+0000 7fac25884700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fac2009b0e0 con 0x7fac2013d240 2023-09-30T22:33:54.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.310+0000 7fac1effd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fac2006c0e0 0x7fac20134ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50086/0 (socket says 172.21.15.16:50086) 2023-09-30T22:33:54.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.310+0000 7fac1effd700 1 -- 172.21.15.16:0/4060336985 learned_addr learned my addr 172.21.15.16:0/4060336985 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:54.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.310+0000 7fac1e7fc700 1 --2- 172.21.15.16:0/4060336985 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fac2009ac90 0x7fac2013cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:54.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.310+0000 7fac1f7fe700 1 --2- 172.21.15.16:0/4060336985 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fac2013d240 0x7fac2013f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:54.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.310+0000 7fac1e7fc700 1 -- 172.21.15.16:0/4060336985 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fac2013d240 msgr2=0x7fac2013f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:54.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.310+0000 7fac1e7fc700 1 --2- 172.21.15.16:0/4060336985 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fac2013d240 0x7fac2013f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:54.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.310+0000 7fac1e7fc700 1 -- 172.21.15.16:0/4060336985 >> v1:172.21.15.16:6789/0 conn(0x7fac2006c0e0 legacy=0x7fac20134ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:54.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.310+0000 7fac1e7fc700 1 -- 172.21.15.16:0/4060336985 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac200a0600 con 0x7fac2009ac90 2023-09-30T22:33:54.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.310+0000 7fac1e7fc700 1 --2- 172.21.15.16:0/4060336985 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fac2009ac90 0x7fac2013cd00 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fac14009870 tx=0x7fac14009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69c92daf8689b7fc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:54.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.311+0000 7fac1dffb700 1 -- 172.21.15.16:0/4060336985 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac14011830 con 0x7fac2009ac90 2023-09-30T22:33:54.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.311+0000 7fac1dffb700 1 -- 172.21.15.16:0/4060336985 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac14011990 con 0x7fac2009ac90 2023-09-30T22:33:54.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.311+0000 7fac1dffb700 1 -- 172.21.15.16:0/4060336985 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac14011d20 con 0x7fac2009ac90 2023-09-30T22:33:54.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.311+0000 7fac25884700 1 -- 172.21.15.16:0/4060336985 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fac2009ac90 msgr2=0x7fac2013cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:54.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.311+0000 7fac25884700 1 --2- 172.21.15.16:0/4060336985 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fac2009ac90 0x7fac2013cd00 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fac14009870 tx=0x7fac14009b80 comp rx=0 tx=0).stop 2023-09-30T22:33:54.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.311+0000 7fac25884700 1 -- 172.21.15.16:0/4060336985 shutdown_connections 2023-09-30T22:33:54.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.311+0000 7fac25884700 1 --2- 172.21.15.16:0/4060336985 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fac2013d240 0x7fac2013f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:54.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.311+0000 7fac25884700 1 --2- 172.21.15.16:0/4060336985 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fac2009ac90 0x7fac2013cd00 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:54.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.311+0000 7fac25884700 1 -- 172.21.15.16:0/4060336985 >> 172.21.15.16:0/4060336985 conn(0x7fac200a4210 msgr2=0x7fac200a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:54.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.311+0000 7fac25884700 1 -- 172.21.15.16:0/4060336985 shutdown_connections 2023-09-30T22:33:54.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.311+0000 7fac25884700 1 -- 172.21.15.16:0/4060336985 wait complete. 2023-09-30T22:33:54.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.313+0000 7fac25884700 1 Processor -- start 2023-09-30T22:33:54.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.313+0000 7fac25884700 1 -- start start 2023-09-30T22:33:54.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.314+0000 7fac25884700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac2006c0e0 0x7fac2009b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:54.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.314+0000 7fac25884700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fac2009ac90 0x7fac2009b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:54.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.314+0000 7fac25884700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fac2013d240 0x7fac2009beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:54.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.314+0000 7fac25884700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac201383a0 con 0x7fac2006c0e0 2023-09-30T22:33:54.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.314+0000 7fac25884700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac2013ca30 con 0x7fac2013d240 2023-09-30T22:33:54.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.314+0000 7fac25884700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fac2013f630 con 0x7fac2009ac90 2023-09-30T22:33:54.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.314+0000 7fac1e7fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fac2009ac90 0x7fac2009b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:54.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.314+0000 7fac1e7fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fac2009ac90 0x7fac2009b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37784/0 (socket says 172.21.15.16:37784) 2023-09-30T22:33:54.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.314+0000 7fac1e7fc700 1 -- 172.21.15.16:0/3659464457 learned_addr learned my addr 172.21.15.16:0/3659464457 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:54.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.314+0000 7fac1f7fe700 1 --2- 172.21.15.16:0/3659464457 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fac2013d240 0x7fac2009beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:54.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.314+0000 7fac1effd700 1 --2- 172.21.15.16:0/3659464457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac2006c0e0 0x7fac2009b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:54.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.314+0000 7fac1e7fc700 1 -- 172.21.15.16:0/3659464457 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fac2013d240 msgr2=0x7fac2009beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:54.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.314+0000 7fac1e7fc700 1 --2- 172.21.15.16:0/3659464457 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fac2013d240 0x7fac2009beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:54.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.314+0000 7fac1e7fc700 1 -- 172.21.15.16:0/3659464457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac2006c0e0 msgr2=0x7fac2009b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:54.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.314+0000 7fac1e7fc700 1 --2- 172.21.15.16:0/3659464457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac2006c0e0 0x7fac2009b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:54.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.314+0000 7fac1e7fc700 1 -- 172.21.15.16:0/3659464457 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac140094a0 con 0x7fac2009ac90 2023-09-30T22:33:54.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.314+0000 7fac1f7fe700 1 --2- 172.21.15.16:0/3659464457 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fac2013d240 0x7fac2009beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:54.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.315+0000 7fac1e7fc700 1 --2- 172.21.15.16:0/3659464457 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fac2009ac90 0x7fac2009b970 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fac14009840 tx=0x7fac14004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:54.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.316+0000 7fac07fff700 1 -- 172.21.15.16:0/3659464457 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac14004590 con 0x7fac2009ac90 2023-09-30T22:33:54.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.316+0000 7fac25884700 1 -- 172.21.15.16:0/3659464457 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac200a03f0 con 0x7fac2009ac90 2023-09-30T22:33:54.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.316+0000 7fac25884700 1 -- 172.21.15.16:0/3659464457 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac2009c4d0 con 0x7fac2009ac90 2023-09-30T22:33:54.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.316+0000 7fac07fff700 1 -- 172.21.15.16:0/3659464457 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac1401f070 con 0x7fac2009ac90 2023-09-30T22:33:54.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.316+0000 7fac07fff700 1 -- 172.21.15.16:0/3659464457 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac140038c0 con 0x7fac2009ac90 2023-09-30T22:33:54.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.317+0000 7fac07fff700 1 -- 172.21.15.16:0/3659464457 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fac14003a20 con 0x7fac2009ac90 2023-09-30T22:33:54.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.317+0000 7fac25884700 1 -- 172.21.15.16:0/3659464457 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac000052e0 con 0x7fac2009ac90 2023-09-30T22:33:54.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.317+0000 7fac07fff700 1 -- 172.21.15.16:0/3659464457 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fac14021070 con 0x7fac2009ac90 2023-09-30T22:33:54.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.319+0000 7f29e6c17700 1 Processor -- start 2023-09-30T22:33:54.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.319+0000 7f29e6c17700 1 -- start start 2023-09-30T22:33:54.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.320+0000 7f29e6c17700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f29e01305f0 0x7f29e01309c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:54.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.320+0000 7f29e6c17700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f29e006f660 0x7f29e006fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:54.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.320+0000 7f29e6c17700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29e01301c0 con 0x7f29e006ff90 2023-09-30T22:33:54.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.320+0000 7f29e6c17700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29e0071ff0 con 0x7f29e006f660 2023-09-30T22:33:54.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.320+0000 7f29e6c17700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f29e0070e70 con 0x7f29e01305f0 2023-09-30T22:33:54.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.321+0000 7f29dffff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f29e006f660 0x7f29e006fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:54.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.321+0000 7f29e51b4700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f29e006ff90 0x7f29e0137530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50092/0 (socket says 172.21.15.16:50092) 2023-09-30T22:33:54.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.321+0000 7f29dffff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f29e006f660 0x7f29e006fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37748/0 (socket says 172.21.15.16:37748) 2023-09-30T22:33:54.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.321+0000 7f29e51b4700 1 -- 172.21.15.16:0/759456433 learned_addr learned my addr 172.21.15.16:0/759456433 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:54.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.321+0000 7f29df7fe700 1 -- 172.21.15.16:0/759456433 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 878740233 0 0) 0x7f29e01301c0 con 0x7f29e006ff90 2023-09-30T22:33:54.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.321+0000 7f29df7fe700 1 -- 172.21.15.16:0/759456433 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29d00035b0 con 0x7f29e006ff90 2023-09-30T22:33:54.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.321+0000 7f29dffff700 1 -- 172.21.15.16:0/759456433 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f29e01305f0 msgr2=0x7f29e01309c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:54.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.321+0000 7f29dffff700 1 --2- 172.21.15.16:0/759456433 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f29e01305f0 0x7f29e01309c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:54.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.321+0000 7f29dffff700 1 -- 172.21.15.16:0/759456433 >> v1:172.21.15.16:6789/0 conn(0x7f29e006ff90 legacy=0x7f29e0137530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:54.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.321+0000 7f29dffff700 1 -- 172.21.15.16:0/759456433 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e00722b0 con 0x7f29e006f660 2023-09-30T22:33:54.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.321+0000 7f29dffff700 1 --2- 172.21.15.16:0/759456433 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f29e006f660 0x7f29e006fa50 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f29cc0098a0 tx=0x7f29cc009bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de8f1cda11259b71 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:54.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.322+0000 7f29df7fe700 1 -- 172.21.15.16:0/759456433 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc011830 con 0x7f29e006f660 2023-09-30T22:33:54.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.322+0000 7f29df7fe700 1 -- 172.21.15.16:0/759456433 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29cc011990 con 0x7f29e006f660 2023-09-30T22:33:54.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.322+0000 7f29df7fe700 1 -- 172.21.15.16:0/759456433 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29cc011d20 con 0x7f29e006f660 2023-09-30T22:33:54.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.322+0000 7f29e6c17700 1 -- 172.21.15.16:0/759456433 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f29e006f660 msgr2=0x7f29e006fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:54.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.322+0000 7f29e6c17700 1 --2- 172.21.15.16:0/759456433 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f29e006f660 0x7f29e006fa50 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f29cc0098a0 tx=0x7f29cc009bb0 comp rx=0 tx=0).stop 2023-09-30T22:33:54.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.322+0000 7f29e6c17700 1 -- 172.21.15.16:0/759456433 shutdown_connections 2023-09-30T22:33:54.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.322+0000 7f29e6c17700 1 --2- 172.21.15.16:0/759456433 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f29e01305f0 0x7f29e01309c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:54.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.322+0000 7f29e6c17700 1 --2- 172.21.15.16:0/759456433 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f29e006f660 0x7f29e006fa50 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:54.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.322+0000 7f29e6c17700 1 -- 172.21.15.16:0/759456433 >> 172.21.15.16:0/759456433 conn(0x7f29e00a2aa0 msgr2=0x7f29e00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:54.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.323+0000 7f29e6c17700 1 -- 172.21.15.16:0/759456433 shutdown_connections 2023-09-30T22:33:54.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.323+0000 7f29e6c17700 1 -- 172.21.15.16:0/759456433 wait complete. 2023-09-30T22:33:54.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.323+0000 7f29e6c17700 1 Processor -- start 2023-09-30T22:33:54.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.324+0000 7f29e6c17700 1 -- start start 2023-09-30T22:33:54.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.324+0000 7f29e6c17700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29e006f660 0x7f29e01c8610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:54.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.324+0000 7f29e6c17700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f29e006ff90 0x7f29e01c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:54.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.324+0000 7f29e6c17700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f29e01305f0 0x7f29e01c4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:54.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.324+0000 7f29e6c17700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e0134c10 con 0x7f29e006f660 2023-09-30T22:33:54.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.324+0000 7f29e6c17700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29e0137080 con 0x7f29e01305f0 2023-09-30T22:33:54.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.324+0000 7f29e6c17700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f29e01c90c0 con 0x7f29e006ff90 2023-09-30T22:33:54.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.324+0000 7f29e49b3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29e006f660 0x7f29e01c8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:54.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.324+0000 7f29e49b3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29e006f660 0x7f29e01c8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43698/0 (socket says 172.21.15.16:43698) 2023-09-30T22:33:54.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.324+0000 7f29e49b3700 1 -- 172.21.15.16:0/512915782 learned_addr learned my addr 172.21.15.16:0/512915782 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:54.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.325+0000 7f29dffff700 1 --2- 172.21.15.16:0/512915782 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f29e006ff90 0x7f29e01c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:54.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.325+0000 7f29e49b3700 1 -- 172.21.15.16:0/512915782 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f29e006ff90 msgr2=0x7f29e01c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:54.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.325+0000 7f29e49b3700 1 --2- 172.21.15.16:0/512915782 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f29e006ff90 0x7f29e01c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:54.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.325+0000 7f29e49b3700 1 -- 172.21.15.16:0/512915782 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f29e01305f0 msgr2=0x7f29e01c4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:54.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.325+0000 7f29e49b3700 1 --2- 172.21.15.16:0/512915782 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f29e01305f0 0x7f29e01c4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:54.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.325+0000 7f29e49b3700 1 -- 172.21.15.16:0/512915782 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29cc0094a0 con 0x7f29e006f660 2023-09-30T22:33:54.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.325+0000 7f29e49b3700 1 --2- 172.21.15.16:0/512915782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29e006f660 0x7f29e01c8610 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f29d400b530 tx=0x7f29d400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:54.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.326+0000 7f29dd7fa700 1 -- 172.21.15.16:0/512915782 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d4011830 con 0x7f29e006f660 2023-09-30T22:33:54.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.326+0000 7f29dd7fa700 1 -- 172.21.15.16:0/512915782 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29d40119b0 con 0x7f29e006f660 2023-09-30T22:33:54.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.326+0000 7f29e6c17700 1 -- 172.21.15.16:0/512915782 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29e01362f0 con 0x7f29e006f660 2023-09-30T22:33:54.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.326+0000 7f29dd7fa700 1 -- 172.21.15.16:0/512915782 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d4011d60 con 0x7f29e006f660 2023-09-30T22:33:54.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.326+0000 7f29e6c17700 1 -- 172.21.15.16:0/512915782 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29e0137280 con 0x7f29e006f660 2023-09-30T22:33:54.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.327+0000 7fac07fff700 1 -- 172.21.15.16:0/3659464457 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fac140193b0 con 0x7fac2009ac90 2023-09-30T22:33:54.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.327+0000 7f29e6c17700 1 -- 172.21.15.16:0/512915782 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29e0074ea0 con 0x7f29e006f660 2023-09-30T22:33:54.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.328+0000 7f29dd7fa700 1 -- 172.21.15.16:0/512915782 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f29d401a400 con 0x7f29e006f660 2023-09-30T22:33:54.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.328+0000 7f29dd7fa700 1 -- 172.21.15.16:0/512915782 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f29d403f8a0 con 0x7f29e006f660 2023-09-30T22:33:54.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.337+0000 7f29dd7fa700 1 -- 172.21.15.16:0/512915782 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f29d400f8d0 con 0x7f29e006f660 2023-09-30T22:33:54.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.414+0000 7f685d640700 1 -- 172.21.15.16:0/857937844 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6838005590 con 0x7f68581338a0 2023-09-30T22:33:54.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.415+0000 7f683f7fe700 1 -- 172.21.15.16:0/857937844 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f685001c400 con 0x7f68581338a0 2023-09-30T22:33:54.415 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:33:54.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.417+0000 7f685d640700 1 -- 172.21.15.16:0/857937844 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f68581338a0 msgr2=0x7f685811f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:54.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.417+0000 7f685d640700 1 --2- 172.21.15.16:0/857937844 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f68581338a0 0x7f685811f4b0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f6850014040 tx=0x7f6850004ad0 comp rx=0 tx=0).stop 2023-09-30T22:33:54.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.417+0000 7f685d640700 1 -- 172.21.15.16:0/857937844 shutdown_connections 2023-09-30T22:33:54.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.417+0000 7f685d640700 1 --2- 172.21.15.16:0/857937844 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f68581338a0 0x7f685811f4b0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:54.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.417+0000 7f685d640700 1 --2- 172.21.15.16:0/857937844 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f685809de50 0x7f685811ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:54.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.417+0000 7f685d640700 1 --2- 172.21.15.16:0/857937844 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f685809e760 0x7f685811ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:54.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.417+0000 7f685d640700 1 -- 172.21.15.16:0/857937844 >> 172.21.15.16:0/857937844 conn(0x7f68580a2aa0 msgr2=0x7f685809ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:54.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.417+0000 7f685d640700 1 -- 172.21.15.16:0/857937844 shutdown_connections 2023-09-30T22:33:54.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.417+0000 7f685d640700 1 -- 172.21.15.16:0/857937844 wait complete. 2023-09-30T22:33:54.432 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:33:54.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.503+0000 7f67d9f59700 1 -- 172.21.15.16:0/415311472 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f67d4134a10 con 0x7f67d409d6f0 2023-09-30T22:33:54.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.504+0000 7f67bbfff700 1 -- 172.21.15.16:0/415311472 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f67d4134a10 con 0x7f67d409d6f0 2023-09-30T22:33:54.504 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:33:54.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.506+0000 7f67d9f59700 1 -- 172.21.15.16:0/415311472 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f67d409d6f0 msgr2=0x7f67d4131630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:54.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.506+0000 7f67d9f59700 1 --2- 172.21.15.16:0/415311472 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f67d409d6f0 0x7f67d4131630 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f67c400e840 tx=0x7f67c400eb50 comp rx=0 tx=0).stop 2023-09-30T22:33:54.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.506+0000 7f67d9f59700 1 -- 172.21.15.16:0/415311472 shutdown_connections 2023-09-30T22:33:54.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.506+0000 7f67d9f59700 1 --2- 172.21.15.16:0/415311472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67d409e410 0x7f67d41320b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:54.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.506+0000 7f67d9f59700 1 --2- 172.21.15.16:0/415311472 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f67d409d6f0 0x7f67d4131630 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:54.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.506+0000 7f67d9f59700 1 --2- 172.21.15.16:0/415311472 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f67d409e000 0x7f67d4131b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:54.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.506+0000 7f67d9f59700 1 -- 172.21.15.16:0/415311472 >> 172.21.15.16:0/415311472 conn(0x7f67d4066f50 msgr2=0x7f67d40a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:54.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.506+0000 7f67d9f59700 1 -- 172.21.15.16:0/415311472 shutdown_connections 2023-09-30T22:33:54.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.506+0000 7f67d9f59700 1 -- 172.21.15.16:0/415311472 wait complete. 2023-09-30T22:33:54.520 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:33:54.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.534+0000 7fac25884700 1 -- 172.21.15.16:0/3659464457 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fac00005590 con 0x7fac2009ac90 2023-09-30T22:33:54.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.534+0000 7fac07fff700 1 -- 172.21.15.16:0/3659464457 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fac1402b070 con 0x7fac2009ac90 2023-09-30T22:33:54.535 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:33:54.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.536+0000 7fac25884700 1 -- 172.21.15.16:0/3659464457 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fac2009ac90 msgr2=0x7fac2009b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:54.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.536+0000 7fac25884700 1 --2- 172.21.15.16:0/3659464457 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fac2009ac90 0x7fac2009b970 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fac14009840 tx=0x7fac14004750 comp rx=0 tx=0).stop 2023-09-30T22:33:54.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.536+0000 7fac25884700 1 -- 172.21.15.16:0/3659464457 shutdown_connections 2023-09-30T22:33:54.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.536+0000 7fac25884700 1 --2- 172.21.15.16:0/3659464457 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fac2013d240 0x7fac2009beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:54.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.536+0000 7fac25884700 1 --2- 172.21.15.16:0/3659464457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fac2006c0e0 0x7fac2009b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:54.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.536+0000 7fac25884700 1 --2- 172.21.15.16:0/3659464457 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fac2009ac90 0x7fac2009b970 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:54.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.536+0000 7fac25884700 1 -- 172.21.15.16:0/3659464457 >> 172.21.15.16:0/3659464457 conn(0x7fac200a4210 msgr2=0x7fac200a0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:54.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.536+0000 7fac25884700 1 -- 172.21.15.16:0/3659464457 shutdown_connections 2023-09-30T22:33:54.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.536+0000 7fac25884700 1 -- 172.21.15.16:0/3659464457 wait complete. 2023-09-30T22:33:54.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.545+0000 7f29e6c17700 1 -- 172.21.15.16:0/512915782 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f29e006b9e0 con 0x7f29e006f660 2023-09-30T22:33:54.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.546+0000 7f29dd7fa700 1 -- 172.21.15.16:0/512915782 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f29d4015030 con 0x7f29e006f660 2023-09-30T22:33:54.546 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:33:54.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.548+0000 7f29e6c17700 1 -- 172.21.15.16:0/512915782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29e006f660 msgr2=0x7f29e01c8610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:54.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.548+0000 7f29e6c17700 1 --2- 172.21.15.16:0/512915782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29e006f660 0x7f29e01c8610 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f29d400b530 tx=0x7f29d400b8f0 comp rx=0 tx=0).stop 2023-09-30T22:33:54.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.548+0000 7f29e6c17700 1 -- 172.21.15.16:0/512915782 shutdown_connections 2023-09-30T22:33:54.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.548+0000 7f29e6c17700 1 --2- 172.21.15.16:0/512915782 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f29e01305f0 0x7f29e01c4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:54.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.548+0000 7f29e6c17700 1 --2- 172.21.15.16:0/512915782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f29e006f660 0x7f29e01c8610 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:54.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.548+0000 7f29e6c17700 1 --2- 172.21.15.16:0/512915782 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f29e006ff90 0x7f29e01c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:54.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.549+0000 7f29e6c17700 1 -- 172.21.15.16:0/512915782 >> 172.21.15.16:0/512915782 conn(0x7f29e00a2aa0 msgr2=0x7f29e012f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:54.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.549+0000 7f29e6c17700 1 -- 172.21.15.16:0/512915782 shutdown_connections 2023-09-30T22:33:54.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:54.549+0000 7f29e6c17700 1 -- 172.21.15.16:0/512915782 wait complete. 2023-09-30T22:33:54.551 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:33:54.562 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:33:55.433 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:33:55.521 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:33:55.552 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:33:55.564 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:33:55.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.583+0000 7fb3072ed700 1 Processor -- start 2023-09-30T22:33:55.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.583+0000 7fb3072ed700 1 -- start start 2023-09-30T22:33:55.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.584+0000 7fb3072ed700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb30009d6f0 0x7fb30009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:55.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.584+0000 7fb3072ed700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb30009e440 0x7fb300134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:55.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.584+0000 7fb3072ed700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb30009b120 con 0x7fb30009e000 2023-09-30T22:33:55.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.584+0000 7fb3072ed700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb30006b460 con 0x7fb30009d6f0 2023-09-30T22:33:55.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.584+0000 7fb3072ed700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb30009a7e0 con 0x7fb30009e440 2023-09-30T22:33:55.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.584+0000 7fb305089700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb30009d6f0 0x7fb30009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:55.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.584+0000 7fb305089700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb30009d6f0 0x7fb30009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43700/0 (socket says 172.21.15.16:43700) 2023-09-30T22:33:55.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.584+0000 7fb305089700 1 -- 172.21.15.16:0/1745230258 learned_addr learned my addr 172.21.15.16:0/1745230258 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:55.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.584+0000 7fb30588a700 1 --2- 172.21.15.16:0/1745230258 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb30009e440 0x7fb300134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:55.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.584+0000 7fb305089700 1 -- 172.21.15.16:0/1745230258 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb30009e440 msgr2=0x7fb300134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:55.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.584+0000 7fb305089700 1 --2- 172.21.15.16:0/1745230258 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb30009e440 0x7fb300134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:55.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.584+0000 7fb305089700 1 -- 172.21.15.16:0/1745230258 >> v1:172.21.15.16:6789/0 conn(0x7fb30009e000 legacy=0x7fb3001339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:55.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.585+0000 7fb305089700 1 -- 172.21.15.16:0/1745230258 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb30009a500 con 0x7fb30009d6f0 2023-09-30T22:33:55.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.585+0000 7fb30588a700 1 --2- 172.21.15.16:0/1745230258 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb30009e440 0x7fb300134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:55.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.585+0000 7fb305089700 1 --2- 172.21.15.16:0/1745230258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb30009d6f0 0x7fb30009dac0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc009920 tx=0x7fb2fc009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31607af324ce9930 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:55.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.585+0000 7fb2f7fff700 1 -- 172.21.15.16:0/1745230258 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc011830 con 0x7fb30009d6f0 2023-09-30T22:33:55.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.585+0000 7fb2f7fff700 1 -- 172.21.15.16:0/1745230258 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2fc011990 con 0x7fb30009d6f0 2023-09-30T22:33:55.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.585+0000 7fb2f7fff700 1 -- 172.21.15.16:0/1745230258 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc011d20 con 0x7fb30009d6f0 2023-09-30T22:33:55.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.586+0000 7fb3072ed700 1 -- 172.21.15.16:0/1745230258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb30009d6f0 msgr2=0x7fb30009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:55.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.586+0000 7fb3072ed700 1 --2- 172.21.15.16:0/1745230258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb30009d6f0 0x7fb30009dac0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc009920 tx=0x7fb2fc009c30 comp rx=0 tx=0).stop 2023-09-30T22:33:55.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.586+0000 7fb3072ed700 1 -- 172.21.15.16:0/1745230258 shutdown_connections 2023-09-30T22:33:55.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.586+0000 7fb3072ed700 1 --2- 172.21.15.16:0/1745230258 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb30009e440 0x7fb300134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:55.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.586+0000 7fb3072ed700 1 --2- 172.21.15.16:0/1745230258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb30009d6f0 0x7fb30009dac0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:55.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.586+0000 7fb3072ed700 1 -- 172.21.15.16:0/1745230258 >> 172.21.15.16:0/1745230258 conn(0x7fb300066f50 msgr2=0x7fb30009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:55.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.586+0000 7fb3072ed700 1 -- 172.21.15.16:0/1745230258 shutdown_connections 2023-09-30T22:33:55.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.586+0000 7fb3072ed700 1 -- 172.21.15.16:0/1745230258 wait complete. 2023-09-30T22:33:55.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.587+0000 7fb3072ed700 1 Processor -- start 2023-09-30T22:33:55.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.588+0000 7fb3072ed700 1 -- start start 2023-09-30T22:33:55.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.588+0000 7fb3072ed700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb30009d6f0 0x7fb3001ab8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:55.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.588+0000 7fb3072ed700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb30009e000 0x7fb3001abde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:55.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.588+0000 7fb3072ed700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb30009e440 0x7fb3001ac320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:55.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.588+0000 7fb3072ed700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3001aee20 con 0x7fb30009e000 2023-09-30T22:33:55.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.588+0000 7fb3072ed700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3001bad50 con 0x7fb30009e440 2023-09-30T22:33:55.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.588+0000 7fb3072ed700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb3001acc70 con 0x7fb30009d6f0 2023-09-30T22:33:55.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.588+0000 7fb304888700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb30009e000 0x7fb3001abde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:55.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.588+0000 7fb304888700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb30009e000 0x7fb3001abde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43714/0 (socket says 172.21.15.16:43714) 2023-09-30T22:33:55.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.588+0000 7fb304888700 1 -- 172.21.15.16:0/2290358173 learned_addr learned my addr 172.21.15.16:0/2290358173 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:55.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.588+0000 7fb30588a700 1 --2- 172.21.15.16:0/2290358173 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb30009e440 0x7fb3001ac320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:55.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.589+0000 7fb304888700 1 -- 172.21.15.16:0/2290358173 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb30009d6f0 msgr2=0x7fb3001ab8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:55.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.589+0000 7fb304888700 1 --2- 172.21.15.16:0/2290358173 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb30009d6f0 0x7fb3001ab8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:55.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.589+0000 7fb304888700 1 -- 172.21.15.16:0/2290358173 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb30009e440 msgr2=0x7fb3001ac320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:55.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.589+0000 7fb304888700 1 --2- 172.21.15.16:0/2290358173 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb30009e440 0x7fb3001ac320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:55.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.589+0000 7fb304888700 1 -- 172.21.15.16:0/2290358173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2f8009a70 con 0x7fb30009e000 2023-09-30T22:33:55.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.589+0000 7fb304888700 1 --2- 172.21.15.16:0/2290358173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb30009e000 0x7fb3001abde0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fb2f000c4e0 tx=0x7fb2f000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:55.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.589+0000 7fb2f5ffb700 1 -- 172.21.15.16:0/2290358173 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f00052d0 con 0x7fb30009e000 2023-09-30T22:33:55.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.589+0000 7fb2f5ffb700 1 -- 172.21.15.16:0/2290358173 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2f0005430 con 0x7fb30009e000 2023-09-30T22:33:55.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.589+0000 7fb3072ed700 1 -- 172.21.15.16:0/2290358173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2fc009550 con 0x7fb30009e000 2023-09-30T22:33:55.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.589+0000 7fb2f5ffb700 1 -- 172.21.15.16:0/2290358173 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f00038c0 con 0x7fb30009e000 2023-09-30T22:33:55.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.590+0000 7fb3072ed700 1 -- 172.21.15.16:0/2290358173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3001af0e0 con 0x7fb30009e000 2023-09-30T22:33:55.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.590+0000 7fb2f5ffb700 1 -- 172.21.15.16:0/2290358173 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb2f0003a20 con 0x7fb30009e000 2023-09-30T22:33:55.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.590+0000 7fb2f5ffb700 1 -- 172.21.15.16:0/2290358173 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fb2f003ead0 con 0x7fb30009e000 2023-09-30T22:33:55.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.592+0000 7fb3072ed700 1 -- 172.21.15.16:0/2290358173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb300074ea0 con 0x7fb30009e000 2023-09-30T22:33:55.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.597+0000 7fb2f5ffb700 1 -- 172.21.15.16:0/2290358173 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb2f0003cd0 con 0x7fb30009e000 2023-09-30T22:33:55.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.670+0000 7f14b9577700 1 Processor -- start 2023-09-30T22:33:55.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.670+0000 7f14b9577700 1 -- start start 2023-09-30T22:33:55.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.670+0000 7f14b9577700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f14b409d6f0 0x7f14b409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:55.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.670+0000 7f14b9577700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f14b409e000 0x7f14b41339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:55.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.670+0000 7f14b9577700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14b409b120 con 0x7f14b409e410 2023-09-30T22:33:55.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.670+0000 7f14b9577700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14b406b460 con 0x7f14b409d6f0 2023-09-30T22:33:55.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.670+0000 7f14b9577700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f14b409a7e0 con 0x7f14b409e000 2023-09-30T22:33:55.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.671+0000 7f14b27fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f14b409e000 0x7f14b41339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:55.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.671+0000 7f14b2ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f14b409d6f0 0x7f14b409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:55.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.671+0000 7f14b27fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f14b409e000 0x7f14b41339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37814/0 (socket says 172.21.15.16:37814) 2023-09-30T22:33:55.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.671+0000 7f14b27fc700 1 -- 172.21.15.16:0/4116097726 learned_addr learned my addr 172.21.15.16:0/4116097726 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:55.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.671+0000 7f14b2ffd700 1 --2- 172.21.15.16:0/4116097726 >> v2:172.21.15.16:3300/0 conn(0x7f14b409d6f0 0x7f14b409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43730/0 (socket says 172.21.15.16:43730) 2023-09-30T22:33:55.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.671+0000 7f14b27fc700 1 -- 172.21.15.16:0/4116097726 >> v2:172.21.15.16:3300/0 conn(0x7f14b409d6f0 msgr2=0x7f14b409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:55.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.671+0000 7f14b27fc700 1 --2- 172.21.15.16:0/4116097726 >> v2:172.21.15.16:3300/0 conn(0x7f14b409d6f0 0x7f14b409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:55.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.671+0000 7f14b27fc700 1 -- 172.21.15.16:0/4116097726 >> v1:172.21.15.16:6789/0 conn(0x7f14b409e410 legacy=0x7f14b4133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:55.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.671+0000 7f14b1ffb700 1 -- 172.21.15.16:0/4116097726 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 305867837 0 0) 0x7f14b409b120 con 0x7f14b409e410 2023-09-30T22:33:55.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.671+0000 7f14b27fc700 1 -- 172.21.15.16:0/4116097726 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b409a500 con 0x7f14b409e000 2023-09-30T22:33:55.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.671+0000 7f14b27fc700 1 --2- 172.21.15.16:0/4116097726 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f14b409e000 0x7f14b41339f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f14a8009920 tx=0x7f14a8009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e16c1e577ccb4528 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:55.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.672+0000 7f14b1ffb700 1 -- 172.21.15.16:0/4116097726 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a800ba20 con 0x7f14b409e000 2023-09-30T22:33:55.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.672+0000 7f14b1ffb700 1 -- 172.21.15.16:0/4116097726 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a800bb80 con 0x7f14b409e000 2023-09-30T22:33:55.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.672+0000 7f14b1ffb700 1 -- 172.21.15.16:0/4116097726 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a8009e70 con 0x7f14b409e000 2023-09-30T22:33:55.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.673+0000 7f14b9577700 1 -- 172.21.15.16:0/4116097726 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f14b409e000 msgr2=0x7f14b41339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:55.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.673+0000 7f14b9577700 1 --2- 172.21.15.16:0/4116097726 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f14b409e000 0x7f14b41339f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f14a8009920 tx=0x7f14a8009c30 comp rx=0 tx=0).stop 2023-09-30T22:33:55.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.673+0000 7f14b9577700 1 -- 172.21.15.16:0/4116097726 shutdown_connections 2023-09-30T22:33:55.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.673+0000 7f14b9577700 1 --2- 172.21.15.16:0/4116097726 >> v2:172.21.15.16:3300/0 conn(0x7f14b409d6f0 0x7f14b409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:55.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.673+0000 7f14b9577700 1 --2- 172.21.15.16:0/4116097726 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f14b409e000 0x7f14b41339f0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:55.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.673+0000 7f14b9577700 1 -- 172.21.15.16:0/4116097726 >> 172.21.15.16:0/4116097726 conn(0x7f14b4066f50 msgr2=0x7f14b409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:55.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.673+0000 7f14b9577700 1 -- 172.21.15.16:0/4116097726 shutdown_connections 2023-09-30T22:33:55.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.673+0000 7f14b9577700 1 -- 172.21.15.16:0/4116097726 wait complete. 2023-09-30T22:33:55.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.675+0000 7f14b9577700 1 Processor -- start 2023-09-30T22:33:55.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.675+0000 7f14b9577700 1 -- start start 2023-09-30T22:33:55.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.675+0000 7f14b9577700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14b409d6f0 0x7f14b41a42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:55.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.675+0000 7f14b9577700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f14b409e000 0x7f14b41a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:55.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.675+0000 7f14b9577700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f14b409e410 0x7f14b41c8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:55.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.675+0000 7f14b9577700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b41a5620 con 0x7f14b409d6f0 2023-09-30T22:33:55.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.675+0000 7f14b9577700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14b41a5430 con 0x7f14b409e410 2023-09-30T22:33:55.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.675+0000 7f14b9577700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f14b41a5a50 con 0x7f14b409e000 2023-09-30T22:33:55.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.676+0000 7f14b2ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14b409d6f0 0x7f14b41a42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:55.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.676+0000 7f14b2ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14b409d6f0 0x7f14b41a42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43734/0 (socket says 172.21.15.16:43734) 2023-09-30T22:33:55.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.676+0000 7f14b2ffd700 1 -- 172.21.15.16:0/228478877 learned_addr learned my addr 172.21.15.16:0/228478877 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:55.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.676+0000 7f14b37fe700 1 --2- 172.21.15.16:0/228478877 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f14b409e410 0x7f14b41c8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:55.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.676+0000 7f14b27fc700 1 --2- 172.21.15.16:0/228478877 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f14b409e000 0x7f14b41a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:55.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.676+0000 7f14b37fe700 1 -- 172.21.15.16:0/228478877 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f14b409e000 msgr2=0x7f14b41a47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:55.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.676+0000 7f14b37fe700 1 --2- 172.21.15.16:0/228478877 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f14b409e000 0x7f14b41a47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:55.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.676+0000 7f14b37fe700 1 -- 172.21.15.16:0/228478877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14b409d6f0 msgr2=0x7f14b41a42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:55.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.676+0000 7f14b37fe700 1 --2- 172.21.15.16:0/228478877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14b409d6f0 0x7f14b41a42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:55.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.676+0000 7f14b37fe700 1 -- 172.21.15.16:0/228478877 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14a8009550 con 0x7f14b409e410 2023-09-30T22:33:55.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.676+0000 7f14b37fe700 1 --2- 172.21.15.16:0/228478877 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f14b409e410 0x7f14b41c8200 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f14ac00c7f0 tx=0x7f14ac00cb00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:55.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.677+0000 7f149b7fe700 1 -- 172.21.15.16:0/228478877 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ac003e30 con 0x7f14b409e410 2023-09-30T22:33:55.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.677+0000 7f149b7fe700 1 -- 172.21.15.16:0/228478877 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14ac005070 con 0x7f14b409e410 2023-09-30T22:33:55.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.677+0000 7f14b9577700 1 -- 172.21.15.16:0/228478877 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14b41b81b0 con 0x7f14b409e410 2023-09-30T22:33:55.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.677+0000 7f149b7fe700 1 -- 172.21.15.16:0/228478877 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ac005420 con 0x7f14b409e410 2023-09-30T22:33:55.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.677+0000 7f14b9577700 1 -- 172.21.15.16:0/228478877 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14b409c710 con 0x7f14b409e410 2023-09-30T22:33:55.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.678+0000 7f14b9577700 1 -- 172.21.15.16:0/228478877 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14b4072440 con 0x7f14b409e410 2023-09-30T22:33:55.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.678+0000 7f149b7fe700 1 -- 172.21.15.16:0/228478877 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f14ac005580 con 0x7f14b409e410 2023-09-30T22:33:55.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.678+0000 7f149b7fe700 1 -- 172.21.15.16:0/228478877 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f14ac005830 con 0x7f14b409e410 2023-09-30T22:33:55.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.687+0000 7f149b7fe700 1 -- 172.21.15.16:0/228478877 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f14ac044060 con 0x7f14b409e410 2023-09-30T22:33:55.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.713+0000 7f38c02a2700 1 Processor -- start 2023-09-30T22:33:55.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.713+0000 7f38c02a2700 1 -- start start 2023-09-30T22:33:55.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.714+0000 7f38c02a2700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f38b80a0a30 0x7f38b80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:55.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.714+0000 7f38c02a2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38b80a1340 0x7f38b8133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:55.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.714+0000 7f38c02a2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f38b8133fa0 0x7f38b8134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:55.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.714+0000 7f38c02a2700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38b809edd0 con 0x7f38b80a0a30 2023-09-30T22:33:55.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.714+0000 7f38c02a2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38b809e1f0 con 0x7f38b80a1340 2023-09-30T22:33:55.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.714+0000 7f38c02a2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f38b809dc50 con 0x7f38b8133fa0 2023-09-30T22:33:55.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.714+0000 7f38be83f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f38b8133fa0 0x7f38b8134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:55.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.714+0000 7f38bd83d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38b80a1340 0x7f38b8133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:55.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.715+0000 7f38be83f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f38b8133fa0 0x7f38b8134410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37828/0 (socket says 172.21.15.16:37828) 2023-09-30T22:33:55.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.715+0000 7f38be83f700 1 -- 172.21.15.16:0/343436835 learned_addr learned my addr 172.21.15.16:0/343436835 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:55.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.715+0000 7f38bd83d700 1 --2- 172.21.15.16:0/343436835 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38b80a1340 0x7f38b8133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37782/0 (socket says 172.21.15.16:37782) 2023-09-30T22:33:55.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.715+0000 7f38be03e700 1 --2- 172.21.15.16:0/343436835 >> v2:172.21.15.16:3300/0 conn(0x7f38b80a0a30 0x7f38b80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:55.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.715+0000 7f38bd83d700 1 -- 172.21.15.16:0/343436835 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f38b8133fa0 msgr2=0x7f38b8134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:55.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.715+0000 7f38bd83d700 1 --2- 172.21.15.16:0/343436835 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f38b8133fa0 0x7f38b8134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:55.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.715+0000 7f38bd83d700 1 -- 172.21.15.16:0/343436835 >> v2:172.21.15.16:3300/0 conn(0x7f38b80a0a30 msgr2=0x7f38b80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:55.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.715+0000 7f38bd83d700 1 --2- 172.21.15.16:0/343436835 >> v2:172.21.15.16:3300/0 conn(0x7f38b80a0a30 0x7f38b80a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:55.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.715+0000 7f38bd83d700 1 -- 172.21.15.16:0/343436835 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b80a1fe0 con 0x7f38b80a1340 2023-09-30T22:33:55.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.715+0000 7f38be03e700 1 --2- 172.21.15.16:0/343436835 >> v2:172.21.15.16:3300/0 conn(0x7f38b80a0a30 0x7f38b80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:55.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.715+0000 7f38bd83d700 1 --2- 172.21.15.16:0/343436835 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38b80a1340 0x7f38b8133a60 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f38a8009870 tx=0x7f38a8009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=141fa95104cbbab5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:55.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.715+0000 7f38bd03c700 1 -- 172.21.15.16:0/343436835 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a8011830 con 0x7f38b80a1340 2023-09-30T22:33:55.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.715+0000 7f38bd03c700 1 -- 172.21.15.16:0/343436835 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38a8011990 con 0x7f38b80a1340 2023-09-30T22:33:55.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.715+0000 7f38bd03c700 1 -- 172.21.15.16:0/343436835 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38a8011d20 con 0x7f38b80a1340 2023-09-30T22:33:55.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.716+0000 7f38c02a2700 1 -- 172.21.15.16:0/343436835 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38b80a1340 msgr2=0x7f38b8133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:55.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.716+0000 7f38c02a2700 1 --2- 172.21.15.16:0/343436835 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38b80a1340 0x7f38b8133a60 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f38a8009870 tx=0x7f38a8009b80 comp rx=0 tx=0).stop 2023-09-30T22:33:55.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.716+0000 7f38c02a2700 1 -- 172.21.15.16:0/343436835 shutdown_connections 2023-09-30T22:33:55.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.716+0000 7f38c02a2700 1 --2- 172.21.15.16:0/343436835 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f38b8133fa0 0x7f38b8134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:55.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.716+0000 7f38c02a2700 1 --2- 172.21.15.16:0/343436835 >> v2:172.21.15.16:3300/0 conn(0x7f38b80a0a30 0x7f38b80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:55.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.716+0000 7f38c02a2700 1 --2- 172.21.15.16:0/343436835 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38b80a1340 0x7f38b8133a60 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:55.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.716+0000 7f38c02a2700 1 -- 172.21.15.16:0/343436835 >> 172.21.15.16:0/343436835 conn(0x7f38b80a6550 msgr2=0x7f38b80a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:55.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.716+0000 7f38c02a2700 1 -- 172.21.15.16:0/343436835 shutdown_connections 2023-09-30T22:33:55.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.716+0000 7f38c02a2700 1 -- 172.21.15.16:0/343436835 wait complete. 2023-09-30T22:33:55.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.718+0000 7f38c02a2700 1 Processor -- start 2023-09-30T22:33:55.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.718+0000 7f38c02a2700 1 -- start start 2023-09-30T22:33:55.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.718+0000 7f38c02a2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38b80a0a30 0x7f38b8120170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:55.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.719+0000 7f38c02a2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f38b80a1340 0x7f38b81206b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:55.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.719+0000 7f38c02a2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38b8133fa0 0x7f38b8120bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:55.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.719+0000 7f38c02a2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b81223d0 con 0x7f38b8133fa0 2023-09-30T22:33:55.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.719+0000 7f38c02a2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38b8123c90 con 0x7f38b80a0a30 2023-09-30T22:33:55.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.719+0000 7f38c02a2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f38b8123060 con 0x7f38b80a1340 2023-09-30T22:33:55.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.719+0000 7f38be03e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38b80a0a30 0x7f38b8120170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:55.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.719+0000 7f38be03e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38b80a0a30 0x7f38b8120170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37788/0 (socket says 172.21.15.16:37788) 2023-09-30T22:33:55.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.719+0000 7f38be03e700 1 -- 172.21.15.16:0/3357310578 learned_addr learned my addr 172.21.15.16:0/3357310578 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:55.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.719+0000 7f38be83f700 1 --2- 172.21.15.16:0/3357310578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38b8133fa0 0x7f38b8120bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:55.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.719+0000 7f38bd83d700 1 --2- 172.21.15.16:0/3357310578 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f38b80a1340 0x7f38b81206b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:55.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.719+0000 7f38be03e700 1 -- 172.21.15.16:0/3357310578 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f38b80a1340 msgr2=0x7f38b81206b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:55.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.719+0000 7f38be03e700 1 --2- 172.21.15.16:0/3357310578 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f38b80a1340 0x7f38b81206b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:55.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.719+0000 7f38be03e700 1 -- 172.21.15.16:0/3357310578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38b8133fa0 msgr2=0x7f38b8120bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:55.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.719+0000 7f38be03e700 1 --2- 172.21.15.16:0/3357310578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38b8133fa0 0x7f38b8120bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:55.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.719+0000 7f38be03e700 1 -- 172.21.15.16:0/3357310578 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b40094a0 con 0x7f38b80a0a30 2023-09-30T22:33:55.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.719+0000 7f38be83f700 1 --2- 172.21.15.16:0/3357310578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38b8133fa0 0x7f38b8120bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:33:55.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.719+0000 7f38be03e700 1 --2- 172.21.15.16:0/3357310578 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38b80a0a30 0x7f38b8120170 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f38b400ebe0 tx=0x7f38b4009ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:55.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.719+0000 7f38aeffd700 1 -- 172.21.15.16:0/3357310578 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b400c910 con 0x7f38b80a0a30 2023-09-30T22:33:55.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.720+0000 7f38aeffd700 1 -- 172.21.15.16:0/3357310578 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38b400ca70 con 0x7f38b80a0a30 2023-09-30T22:33:55.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.720+0000 7f38c02a2700 1 -- 172.21.15.16:0/3357310578 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38a80094a0 con 0x7f38b80a0a30 2023-09-30T22:33:55.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.720+0000 7f38aeffd700 1 -- 172.21.15.16:0/3357310578 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38b400cde0 con 0x7f38b80a0a30 2023-09-30T22:33:55.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.720+0000 7f38c02a2700 1 -- 172.21.15.16:0/3357310578 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38b80a0220 con 0x7f38b80a0a30 2023-09-30T22:33:55.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.720+0000 7f38aeffd700 1 -- 172.21.15.16:0/3357310578 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f38b4013470 con 0x7f38b80a0a30 2023-09-30T22:33:55.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.721+0000 7f38aeffd700 1 -- 172.21.15.16:0/3357310578 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f38b403f930 con 0x7f38b80a0a30 2023-09-30T22:33:55.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.721+0000 7f38c02a2700 1 -- 172.21.15.16:0/3357310578 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38b8074ea0 con 0x7f38b80a0a30 2023-09-30T22:33:55.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.724+0000 7f4350c22700 1 Processor -- start 2023-09-30T22:33:55.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.725+0000 7f4350c22700 1 -- start start 2023-09-30T22:33:55.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.726+0000 7f4350c22700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f434c12cf60 0x7f434c06f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:55.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.726+0000 7f4350c22700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f434c130430 0x7f434c137c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:55.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.726+0000 7f4350c22700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f434c12cd90 con 0x7f434c12f8b0 2023-09-30T22:33:55.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.726+0000 7f4350c22700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f434c12d330 con 0x7f434c130430 2023-09-30T22:33:55.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.726+0000 7f4350c22700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f434c071780 con 0x7f434c12cf60 2023-09-30T22:33:55.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.726+0000 7f4349d9b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f434c12f8b0 0x7f434c12fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50118/0 (socket says 172.21.15.16:50118) 2023-09-30T22:33:55.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.726+0000 7f4349d9b700 1 -- 172.21.15.16:0/1874920655 learned_addr learned my addr 172.21.15.16:0/1874920655 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:55.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.726+0000 7f434ad9d700 1 --2- 172.21.15.16:0/1874920655 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f434c130430 0x7f434c137c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:55.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.726+0000 7f434959a700 1 -- 172.21.15.16:0/1874920655 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1100925665 0 0) 0x7f434c12cd90 con 0x7f434c12f8b0 2023-09-30T22:33:55.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.726+0000 7f434959a700 1 -- 172.21.15.16:0/1874920655 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43340035b0 con 0x7f434c12f8b0 2023-09-30T22:33:55.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.726+0000 7f434ad9d700 1 -- 172.21.15.16:0/1874920655 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f434c12cf60 msgr2=0x7f434c06f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:55.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.726+0000 7f434ad9d700 1 --2- 172.21.15.16:0/1874920655 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f434c12cf60 0x7f434c06f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:55.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.726+0000 7f434ad9d700 1 -- 172.21.15.16:0/1874920655 >> v1:172.21.15.16:6789/0 conn(0x7f434c12f8b0 legacy=0x7f434c12fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:55.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.726+0000 7f434ad9d700 1 -- 172.21.15.16:0/1874920655 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f434c071a40 con 0x7f434c130430 2023-09-30T22:33:55.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.726+0000 7f434ad9d700 1 --2- 172.21.15.16:0/1874920655 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f434c130430 0x7f434c137c30 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4344009dc0 tx=0x7f434400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60eb2e875ccc33ae server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:55.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.730+0000 7f434959a700 1 -- 172.21.15.16:0/1874920655 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434400e070 con 0x7f434c130430 2023-09-30T22:33:55.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.730+0000 7f434959a700 1 -- 172.21.15.16:0/1874920655 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f434400bb20 con 0x7f434c130430 2023-09-30T22:33:55.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.730+0000 7f434959a700 1 -- 172.21.15.16:0/1874920655 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434400be90 con 0x7f434c130430 2023-09-30T22:33:55.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.730+0000 7f38aeffd700 1 -- 172.21.15.16:0/3357310578 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f38b40232f0 con 0x7f38b80a0a30 2023-09-30T22:33:55.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.731+0000 7f4350c22700 1 -- 172.21.15.16:0/1874920655 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f434c130430 msgr2=0x7f434c137c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:55.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.731+0000 7f4350c22700 1 --2- 172.21.15.16:0/1874920655 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f434c130430 0x7f434c137c30 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4344009dc0 tx=0x7f434400b330 comp rx=0 tx=0).stop 2023-09-30T22:33:55.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.731+0000 7f4350c22700 1 -- 172.21.15.16:0/1874920655 shutdown_connections 2023-09-30T22:33:55.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.731+0000 7f4350c22700 1 --2- 172.21.15.16:0/1874920655 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f434c130430 0x7f434c137c30 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:55.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.731+0000 7f4350c22700 1 --2- 172.21.15.16:0/1874920655 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f434c12cf60 0x7f434c06f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:55.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.731+0000 7f4350c22700 1 -- 172.21.15.16:0/1874920655 >> 172.21.15.16:0/1874920655 conn(0x7f434c0a2aa0 msgr2=0x7f434c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:55.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.731+0000 7f4350c22700 1 -- 172.21.15.16:0/1874920655 shutdown_connections 2023-09-30T22:33:55.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.731+0000 7f4350c22700 1 -- 172.21.15.16:0/1874920655 wait complete. 2023-09-30T22:33:55.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.733+0000 7f4350c22700 1 Processor -- start 2023-09-30T22:33:55.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.733+0000 7f4350c22700 1 -- start start 2023-09-30T22:33:55.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.733+0000 7f4350c22700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f434c12cf60 0x7f434c1c85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:55.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.733+0000 7f4350c22700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f434c12f8b0 0x7f434c1c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:55.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.734+0000 7f4350c22700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f434c130430 0x7f434c1c9070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:55.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.734+0000 7f4350c22700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f434c09ea30 con 0x7f434c12f8b0 2023-09-30T22:33:55.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.734+0000 7f4350c22700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f434c13b760 con 0x7f434c12cf60 2023-09-30T22:33:55.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.734+0000 7f4350c22700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f434c129fa0 con 0x7f434c130430 2023-09-30T22:33:55.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.734+0000 7f434a59c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f434c12cf60 0x7f434c1c85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:55.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.734+0000 7f434ad9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f434c130430 0x7f434c1c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:55.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.734+0000 7f434a59c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f434c12cf60 0x7f434c1c85f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37804/0 (socket says 172.21.15.16:37804) 2023-09-30T22:33:55.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.734+0000 7f434a59c700 1 -- 172.21.15.16:0/595535668 learned_addr learned my addr 172.21.15.16:0/595535668 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:55.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.734+0000 7f4349d9b700 1 --2- 172.21.15.16:0/595535668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f434c12f8b0 0x7f434c1c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:55.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.734+0000 7f434a59c700 1 -- 172.21.15.16:0/595535668 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f434c130430 msgr2=0x7f434c1c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:55.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.734+0000 7f434a59c700 1 --2- 172.21.15.16:0/595535668 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f434c130430 0x7f434c1c9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:55.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.734+0000 7f434a59c700 1 -- 172.21.15.16:0/595535668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f434c12f8b0 msgr2=0x7f434c1c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:55.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.734+0000 7f434a59c700 1 --2- 172.21.15.16:0/595535668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f434c12f8b0 0x7f434c1c8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:55.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.734+0000 7f434a59c700 1 -- 172.21.15.16:0/595535668 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4344009a70 con 0x7f434c12cf60 2023-09-30T22:33:55.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.734+0000 7f4349d9b700 1 --2- 172.21.15.16:0/595535668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f434c12f8b0 0x7f434c1c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:55.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.734+0000 7f434a59c700 1 --2- 172.21.15.16:0/595535668 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f434c12cf60 0x7f434c1c85f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f433c00b530 tx=0x7f433c00b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:55.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.735+0000 7f433affd700 1 -- 172.21.15.16:0/595535668 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f433c011830 con 0x7f434c12cf60 2023-09-30T22:33:55.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.735+0000 7f433affd700 1 -- 172.21.15.16:0/595535668 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f433c011990 con 0x7f434c12cf60 2023-09-30T22:33:55.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.735+0000 7f433affd700 1 -- 172.21.15.16:0/595535668 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f433c011d40 con 0x7f434c12cf60 2023-09-30T22:33:55.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.735+0000 7f4350c22700 1 -- 172.21.15.16:0/595535668 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f434c132d70 con 0x7f434c12cf60 2023-09-30T22:33:55.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.735+0000 7f4350c22700 1 -- 172.21.15.16:0/595535668 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f434c09df20 con 0x7f434c12cf60 2023-09-30T22:33:55.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.736+0000 7f433affd700 1 -- 172.21.15.16:0/595535668 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f433c01a400 con 0x7f434c12cf60 2023-09-30T22:33:55.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.736+0000 7f433affd700 1 -- 172.21.15.16:0/595535668 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f433c03f710 con 0x7f434c12cf60 2023-09-30T22:33:55.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.737+0000 7f4350c22700 1 -- 172.21.15.16:0/595535668 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f434c074ea0 con 0x7f434c12cf60 2023-09-30T22:33:55.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.747+0000 7f433affd700 1 -- 172.21.15.16:0/595535668 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f433c015080 con 0x7f434c12cf60 2023-09-30T22:33:55.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.863+0000 7fb3072ed700 1 -- 172.21.15.16:0/2290358173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb30006c360 con 0x7fb30009e000 2023-09-30T22:33:55.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.864+0000 7fb2f5ffb700 1 -- 172.21.15.16:0/2290358173 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb2f0017030 con 0x7fb30009e000 2023-09-30T22:33:55.864 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:33:55.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.867+0000 7fb3072ed700 1 -- 172.21.15.16:0/2290358173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb30009e000 msgr2=0x7fb3001abde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:55.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.867+0000 7fb3072ed700 1 --2- 172.21.15.16:0/2290358173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb30009e000 0x7fb3001abde0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fb2f000c4e0 tx=0x7fb2f000c8a0 comp rx=0 tx=0).stop 2023-09-30T22:33:55.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.867+0000 7fb3072ed700 1 -- 172.21.15.16:0/2290358173 shutdown_connections 2023-09-30T22:33:55.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.867+0000 7fb3072ed700 1 --2- 172.21.15.16:0/2290358173 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb30009e440 0x7fb3001ac320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:55.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.867+0000 7fb3072ed700 1 --2- 172.21.15.16:0/2290358173 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb30009d6f0 0x7fb3001ab8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:55.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.867+0000 7fb3072ed700 1 --2- 172.21.15.16:0/2290358173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb30009e000 0x7fb3001abde0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:55.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.867+0000 7fb3072ed700 1 -- 172.21.15.16:0/2290358173 >> 172.21.15.16:0/2290358173 conn(0x7fb300066f50 msgr2=0x7fb30006f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:55.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.867+0000 7fb3072ed700 1 -- 172.21.15.16:0/2290358173 shutdown_connections 2023-09-30T22:33:55.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.867+0000 7fb3072ed700 1 -- 172.21.15.16:0/2290358173 wait complete. 2023-09-30T22:33:55.886 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:33:55.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.916+0000 7f14b9577700 1 -- 172.21.15.16:0/228478877 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f14b4074ea0 con 0x7f14b409e410 2023-09-30T22:33:55.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.918+0000 7f149b7fe700 1 -- 172.21.15.16:0/228478877 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f14ac017030 con 0x7f14b409e410 2023-09-30T22:33:55.919 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:33:55.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.921+0000 7f14b9577700 1 -- 172.21.15.16:0/228478877 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f14b409e410 msgr2=0x7f14b41c8200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:55.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.921+0000 7f14b9577700 1 --2- 172.21.15.16:0/228478877 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f14b409e410 0x7f14b41c8200 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f14ac00c7f0 tx=0x7f14ac00cb00 comp rx=0 tx=0).stop 2023-09-30T22:33:55.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.921+0000 7f14b9577700 1 -- 172.21.15.16:0/228478877 shutdown_connections 2023-09-30T22:33:55.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.921+0000 7f14b9577700 1 --2- 172.21.15.16:0/228478877 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f14b409e410 0x7f14b41c8200 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:55.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.921+0000 7f14b9577700 1 --2- 172.21.15.16:0/228478877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14b409d6f0 0x7f14b41a42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:55.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.921+0000 7f14b9577700 1 --2- 172.21.15.16:0/228478877 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f14b409e000 0x7f14b41a47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:55.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.921+0000 7f14b9577700 1 -- 172.21.15.16:0/228478877 >> 172.21.15.16:0/228478877 conn(0x7f14b4066f50 msgr2=0x7f14b406f040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:55.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.921+0000 7f14b9577700 1 -- 172.21.15.16:0/228478877 shutdown_connections 2023-09-30T22:33:55.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.921+0000 7f14b9577700 1 -- 172.21.15.16:0/228478877 wait complete. 2023-09-30T22:33:55.935 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:33:55.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.955+0000 7f4350c22700 1 -- 172.21.15.16:0/595535668 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f434c071380 con 0x7f434c12cf60 2023-09-30T22:33:55.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.956+0000 7f433affd700 1 -- 172.21.15.16:0/595535668 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f433c00f8d0 con 0x7f434c12cf60 2023-09-30T22:33:55.956 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:33:55.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.958+0000 7f4350c22700 1 -- 172.21.15.16:0/595535668 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f434c12cf60 msgr2=0x7f434c1c85f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:55.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.958+0000 7f4350c22700 1 --2- 172.21.15.16:0/595535668 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f434c12cf60 0x7f434c1c85f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f433c00b530 tx=0x7f433c00b840 comp rx=0 tx=0).stop 2023-09-30T22:33:55.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.958+0000 7f4350c22700 1 -- 172.21.15.16:0/595535668 shutdown_connections 2023-09-30T22:33:55.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.958+0000 7f4350c22700 1 --2- 172.21.15.16:0/595535668 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f434c12cf60 0x7f434c1c85f0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:55.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.958+0000 7f4350c22700 1 --2- 172.21.15.16:0/595535668 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f434c130430 0x7f434c1c9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:55.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.958+0000 7f4350c22700 1 --2- 172.21.15.16:0/595535668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f434c12f8b0 0x7f434c1c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:55.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.958+0000 7f4350c22700 1 -- 172.21.15.16:0/595535668 >> 172.21.15.16:0/595535668 conn(0x7f434c0a2aa0 msgr2=0x7f434c06e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:55.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.958+0000 7f4350c22700 1 -- 172.21.15.16:0/595535668 shutdown_connections 2023-09-30T22:33:55.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.958+0000 7f4350c22700 1 -- 172.21.15.16:0/595535668 wait complete. 2023-09-30T22:33:55.972 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:33:55.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.978+0000 7f38c02a2700 1 -- 172.21.15.16:0/3357310578 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f38b80a28e0 con 0x7f38b80a0a30 2023-09-30T22:33:55.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.979+0000 7f38aeffd700 1 -- 172.21.15.16:0/3357310578 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f38b4017020 con 0x7f38b80a0a30 2023-09-30T22:33:55.979 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:33:55.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.981+0000 7f38c02a2700 1 -- 172.21.15.16:0/3357310578 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38b80a0a30 msgr2=0x7f38b8120170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:55.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.981+0000 7f38c02a2700 1 --2- 172.21.15.16:0/3357310578 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38b80a0a30 0x7f38b8120170 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f38b400ebe0 tx=0x7f38b4009ec0 comp rx=0 tx=0).stop 2023-09-30T22:33:55.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.981+0000 7f38c02a2700 1 -- 172.21.15.16:0/3357310578 shutdown_connections 2023-09-30T22:33:55.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.981+0000 7f38c02a2700 1 --2- 172.21.15.16:0/3357310578 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38b80a0a30 0x7f38b8120170 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:55.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.981+0000 7f38c02a2700 1 --2- 172.21.15.16:0/3357310578 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38b8133fa0 0x7f38b8120bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:55.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.981+0000 7f38c02a2700 1 --2- 172.21.15.16:0/3357310578 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f38b80a1340 0x7f38b81206b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:55.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.981+0000 7f38c02a2700 1 -- 172.21.15.16:0/3357310578 >> 172.21.15.16:0/3357310578 conn(0x7f38b80a6550 msgr2=0x7f38b809ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:55.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.981+0000 7f38c02a2700 1 -- 172.21.15.16:0/3357310578 shutdown_connections 2023-09-30T22:33:55.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:55.981+0000 7f38c02a2700 1 -- 172.21.15.16:0/3357310578 wait complete. 2023-09-30T22:33:55.995 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:33:56.887 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:33:56.935 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:33:56.973 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:33:56.996 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:33:57.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.040+0000 7f2c45165700 1 Processor -- start 2023-09-30T22:33:57.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.040+0000 7f2c45165700 1 -- start start 2023-09-30T22:33:57.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.040+0000 7f2c45165700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c4009de00 0x7f2c4009e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:57.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.040+0000 7f2c45165700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2c4013d1f0 0x7f2c4013f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:57.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.040+0000 7f2c45165700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c4009a810 con 0x7f2c4006a770 2023-09-30T22:33:57.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.040+0000 7f2c45165700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c4006ab40 con 0x7f2c4013d1f0 2023-09-30T22:33:57.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.040+0000 7f2c45165700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4009ebc0 con 0x7f2c4009de00 2023-09-30T22:33:57.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.041+0000 7f2c3e59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c4009de00 0x7f2c4009e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:57.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.041+0000 7f2c3e59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c4009de00 0x7f2c4009e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37858/0 (socket says 172.21.15.16:37858) 2023-09-30T22:33:57.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.041+0000 7f2c3e59c700 1 -- 172.21.15.16:0/1481623723 learned_addr learned my addr 172.21.15.16:0/1481623723 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:57.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.041+0000 7f2c3f59e700 1 --2- 172.21.15.16:0/1481623723 >> v2:172.21.15.16:3300/0 conn(0x7f2c4013d1f0 0x7f2c4013f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:57.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.041+0000 7f2c3e59c700 1 -- 172.21.15.16:0/1481623723 >> v2:172.21.15.16:3300/0 conn(0x7f2c4013d1f0 msgr2=0x7f2c4013f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:57.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.041+0000 7f2c3e59c700 1 --2- 172.21.15.16:0/1481623723 >> v2:172.21.15.16:3300/0 conn(0x7f2c4013d1f0 0x7f2c4013f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:57.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.041+0000 7f2c3e59c700 1 -- 172.21.15.16:0/1481623723 >> v1:172.21.15.16:6789/0 conn(0x7f2c4006a770 legacy=0x7f2c4009d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:57.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.041+0000 7f2c3e59c700 1 -- 172.21.15.16:0/1481623723 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c4009e920 con 0x7f2c4009de00 2023-09-30T22:33:57.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.041+0000 7f2c3f59e700 1 --2- 172.21.15.16:0/1481623723 >> v2:172.21.15.16:3300/0 conn(0x7f2c4013d1f0 0x7f2c4013f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:57.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.041+0000 7f2c3e59c700 1 --2- 172.21.15.16:0/1481623723 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c4009de00 0x7f2c4009e250 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f2c340098f0 tx=0x7f2c34009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf42e4d0246ff936 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:57.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.042+0000 7f2c3dd9b700 1 -- 172.21.15.16:0/1481623723 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c34011830 con 0x7f2c4009de00 2023-09-30T22:33:57.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.042+0000 7f2c3dd9b700 1 -- 172.21.15.16:0/1481623723 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c34011990 con 0x7f2c4009de00 2023-09-30T22:33:57.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.042+0000 7f2c3dd9b700 1 -- 172.21.15.16:0/1481623723 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c34011d20 con 0x7f2c4009de00 2023-09-30T22:33:57.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.042+0000 7f2c45165700 1 -- 172.21.15.16:0/1481623723 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c4009de00 msgr2=0x7f2c4009e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:57.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.042+0000 7f2c45165700 1 --2- 172.21.15.16:0/1481623723 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c4009de00 0x7f2c4009e250 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f2c340098f0 tx=0x7f2c34009c00 comp rx=0 tx=0).stop 2023-09-30T22:33:57.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.042+0000 7f2c45165700 1 -- 172.21.15.16:0/1481623723 shutdown_connections 2023-09-30T22:33:57.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.042+0000 7f2c45165700 1 --2- 172.21.15.16:0/1481623723 >> v2:172.21.15.16:3300/0 conn(0x7f2c4013d1f0 0x7f2c4013f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:57.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.042+0000 7f2c45165700 1 --2- 172.21.15.16:0/1481623723 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c4009de00 0x7f2c4009e250 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:57.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.042+0000 7f2c45165700 1 -- 172.21.15.16:0/1481623723 >> 172.21.15.16:0/1481623723 conn(0x7f2c40066f50 msgr2=0x7f2c4009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:57.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.043+0000 7f2c45165700 1 -- 172.21.15.16:0/1481623723 shutdown_connections 2023-09-30T22:33:57.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.043+0000 7f2c45165700 1 -- 172.21.15.16:0/1481623723 wait complete. 2023-09-30T22:33:57.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.044+0000 7f2c45165700 1 Processor -- start 2023-09-30T22:33:57.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.045+0000 7f2c45165700 1 -- start start 2023-09-30T22:33:57.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.045+0000 7f2c45165700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c4006a770 0x7f2c4013b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:57.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.045+0000 7f2c45165700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c4013d1f0 0x7f2c4013bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:57.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.045+0000 7f2c45165700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c4013c470 0x7f2c401350b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:57.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.045+0000 7f2c45165700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4011c900 con 0x7f2c4013d1f0 2023-09-30T22:33:57.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.045+0000 7f2c45165700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4011d230 con 0x7f2c4013c470 2023-09-30T22:33:57.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.045+0000 7f2c45165700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4011d040 con 0x7f2c4006a770 2023-09-30T22:33:57.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.045+0000 7f2c3ed9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c4006a770 0x7f2c4013b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:57.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.045+0000 7f2c3ed9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c4006a770 0x7f2c4013b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37866/0 (socket says 172.21.15.16:37866) 2023-09-30T22:33:57.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.045+0000 7f2c3ed9d700 1 -- 172.21.15.16:0/23075042 learned_addr learned my addr 172.21.15.16:0/23075042 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:57.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.046+0000 7f2c3e59c700 1 --2- 172.21.15.16:0/23075042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c4013d1f0 0x7f2c4013bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:57.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.046+0000 7f2c3f59e700 1 --2- 172.21.15.16:0/23075042 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c4013c470 0x7f2c401350b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:57.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.046+0000 7f2c3ed9d700 1 -- 172.21.15.16:0/23075042 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c4013c470 msgr2=0x7f2c401350b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:57.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.046+0000 7f2c3ed9d700 1 --2- 172.21.15.16:0/23075042 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c4013c470 0x7f2c401350b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:57.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.046+0000 7f2c3ed9d700 1 -- 172.21.15.16:0/23075042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c4013d1f0 msgr2=0x7f2c4013bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:57.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.046+0000 7f2c3ed9d700 1 --2- 172.21.15.16:0/23075042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c4013d1f0 0x7f2c4013bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:57.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.046+0000 7f2c3ed9d700 1 -- 172.21.15.16:0/23075042 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c38009a70 con 0x7f2c4006a770 2023-09-30T22:33:57.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.046+0000 7f2c3ed9d700 1 --2- 172.21.15.16:0/23075042 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c4006a770 0x7f2c4013b9f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f2c3000c4e0 tx=0x7f2c3000c7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:57.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.047+0000 7f2c277fe700 1 -- 172.21.15.16:0/23075042 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c300052d0 con 0x7f2c4006a770 2023-09-30T22:33:57.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.047+0000 7f2c277fe700 1 -- 172.21.15.16:0/23075042 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c30005430 con 0x7f2c4006a770 2023-09-30T22:33:57.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.047+0000 7f2c45165700 1 -- 172.21.15.16:0/23075042 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c34009550 con 0x7f2c4006a770 2023-09-30T22:33:57.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.047+0000 7f2c277fe700 1 -- 172.21.15.16:0/23075042 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c300038c0 con 0x7f2c4006a770 2023-09-30T22:33:57.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.047+0000 7f2c45165700 1 -- 172.21.15.16:0/23075042 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c40136150 con 0x7f2c4006a770 2023-09-30T22:33:57.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.048+0000 7f2c277fe700 1 -- 172.21.15.16:0/23075042 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2c30003a20 con 0x7f2c4006a770 2023-09-30T22:33:57.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.048+0000 7f2c45165700 1 -- 172.21.15.16:0/23075042 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c4011eb10 con 0x7f2c4006a770 2023-09-30T22:33:57.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.048+0000 7f2c277fe700 1 -- 172.21.15.16:0/23075042 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f2c3003e9f0 con 0x7f2c4006a770 2023-09-30T22:33:57.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.057+0000 7f2c277fe700 1 -- 172.21.15.16:0/23075042 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2c30018260 con 0x7f2c4006a770 2023-09-30T22:33:57.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.094+0000 7fa4b7562700 1 Processor -- start 2023-09-30T22:33:57.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.095+0000 7fa4b7562700 1 -- start start 2023-09-30T22:33:57.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.095+0000 7fa4b7562700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa4b00a0a30 0x7fa4b00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:57.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.095+0000 7fa4b7562700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa4b00a1340 0x7fa4b0133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:57.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.096+0000 7fa4b7562700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4b009edd0 con 0x7fa4b0133fa0 2023-09-30T22:33:57.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.096+0000 7fa4b7562700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4b009e1f0 con 0x7fa4b00a1340 2023-09-30T22:33:57.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.096+0000 7fa4b7562700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b009dc50 con 0x7fa4b00a0a30 2023-09-30T22:33:57.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.096+0000 7fa4b4afd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa4b00a1340 0x7fa4b0133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:57.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.096+0000 7fa4b4afd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa4b00a1340 0x7fa4b0133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43784/0 (socket says 172.21.15.16:43784) 2023-09-30T22:33:57.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.096+0000 7fa4b4afd700 1 -- 172.21.15.16:0/2740185767 learned_addr learned my addr 172.21.15.16:0/2740185767 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:57.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.096+0000 7fa4b52fe700 1 --2- 172.21.15.16:0/2740185767 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa4b00a0a30 0x7fa4b00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:57.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.096+0000 7fa4b4afd700 1 -- 172.21.15.16:0/2740185767 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa4b00a0a30 msgr2=0x7fa4b00a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:57.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.096+0000 7fa4b4afd700 1 --2- 172.21.15.16:0/2740185767 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa4b00a0a30 0x7fa4b00a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:57.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.096+0000 7fa4b4afd700 1 -- 172.21.15.16:0/2740185767 >> v1:172.21.15.16:6789/0 conn(0x7fa4b0133fa0 legacy=0x7fa4b013ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:57.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.096+0000 7fa4b4afd700 1 -- 172.21.15.16:0/2740185767 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4b00a1fe0 con 0x7fa4b00a1340 2023-09-30T22:33:57.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.096+0000 7fa4b4afd700 1 --2- 172.21.15.16:0/2740185767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4b00a1340 0x7fa4b0133a60 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fa4a00098f0 tx=0x7fa4a0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=569edd35d51ff30f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:57.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.097+0000 7fa4a7fff700 1 -- 172.21.15.16:0/2740185767 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a0011830 con 0x7fa4b00a1340 2023-09-30T22:33:57.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.097+0000 7fa4a7fff700 1 -- 172.21.15.16:0/2740185767 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4a0011990 con 0x7fa4b00a1340 2023-09-30T22:33:57.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.097+0000 7fa4a7fff700 1 -- 172.21.15.16:0/2740185767 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a0011d20 con 0x7fa4b00a1340 2023-09-30T22:33:57.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.097+0000 7fa4b7562700 1 -- 172.21.15.16:0/2740185767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4b00a1340 msgr2=0x7fa4b0133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:57.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.097+0000 7fa4b7562700 1 --2- 172.21.15.16:0/2740185767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4b00a1340 0x7fa4b0133a60 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fa4a00098f0 tx=0x7fa4a0009c00 comp rx=0 tx=0).stop 2023-09-30T22:33:57.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.098+0000 7fa4b7562700 1 -- 172.21.15.16:0/2740185767 shutdown_connections 2023-09-30T22:33:57.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.098+0000 7fa4b7562700 1 --2- 172.21.15.16:0/2740185767 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa4b00a0a30 0x7fa4b00a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:57.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.098+0000 7fa4b7562700 1 --2- 172.21.15.16:0/2740185767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4b00a1340 0x7fa4b0133a60 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:57.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.098+0000 7fa4b7562700 1 -- 172.21.15.16:0/2740185767 >> 172.21.15.16:0/2740185767 conn(0x7fa4b00a6550 msgr2=0x7fa4b00a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:57.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.098+0000 7fa4b7562700 1 -- 172.21.15.16:0/2740185767 shutdown_connections 2023-09-30T22:33:57.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.098+0000 7fa4b7562700 1 -- 172.21.15.16:0/2740185767 wait complete. 2023-09-30T22:33:57.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.099+0000 7fa4b7562700 1 Processor -- start 2023-09-30T22:33:57.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.099+0000 7fa4b7562700 1 -- start start 2023-09-30T22:33:57.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.100+0000 7fa4b7562700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa4b00a0a30 0x7fa4b01c9780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:57.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.100+0000 7fa4b7562700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa4b00a1340 0x7fa4b01c9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:57.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.100+0000 7fa4b7562700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4b0133fa0 0x7fa4b01cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:57.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.100+0000 7fa4b7562700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b01293b0 con 0x7fa4b0133fa0 2023-09-30T22:33:57.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.100+0000 7fa4b7562700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b01ca200 con 0x7fa4b00a0a30 2023-09-30T22:33:57.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.100+0000 7fa4b7562700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b012a120 con 0x7fa4b00a1340 2023-09-30T22:33:57.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.100+0000 7fa4b4afd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa4b00a1340 0x7fa4b01c9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:57.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.100+0000 7fa4b4afd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa4b00a1340 0x7fa4b01c9cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37868/0 (socket says 172.21.15.16:37868) 2023-09-30T22:33:57.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.100+0000 7fa4b4afd700 1 -- 172.21.15.16:0/2902035003 learned_addr learned my addr 172.21.15.16:0/2902035003 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:57.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.101+0000 7fa4b4afd700 1 -- 172.21.15.16:0/2902035003 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa4b00a0a30 msgr2=0x7fa4b01c9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:57.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.101+0000 7fa4b4afd700 1 --2- 172.21.15.16:0/2902035003 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa4b00a0a30 0x7fa4b01c9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:57.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.101+0000 7fa4b4afd700 1 -- 172.21.15.16:0/2902035003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4b0133fa0 msgr2=0x7fa4b01cc7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:57.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.101+0000 7fa4b4afd700 1 --2- 172.21.15.16:0/2902035003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4b0133fa0 0x7fa4b01cc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:57.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.101+0000 7fa4b4afd700 1 -- 172.21.15.16:0/2902035003 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4a0009550 con 0x7fa4b00a1340 2023-09-30T22:33:57.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.101+0000 7fa4b4afd700 1 --2- 172.21.15.16:0/2902035003 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa4b00a1340 0x7fa4b01c9cc0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fa4a0009c40 tx=0x7fa4a0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:57.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.101+0000 7fa4a5ffb700 1 -- 172.21.15.16:0/2902035003 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a0004530 con 0x7fa4b00a1340 2023-09-30T22:33:57.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.101+0000 7fa4a5ffb700 1 -- 172.21.15.16:0/2902035003 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4a001f070 con 0x7fa4b00a1340 2023-09-30T22:33:57.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.101+0000 7fa4b7562700 1 -- 172.21.15.16:0/2902035003 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4b00a3710 con 0x7fa4b00a1340 2023-09-30T22:33:57.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.101+0000 7fa4a5ffb700 1 -- 172.21.15.16:0/2902035003 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a00038c0 con 0x7fa4b00a1340 2023-09-30T22:33:57.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.102+0000 7fa4b7562700 1 -- 172.21.15.16:0/2902035003 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4b0128210 con 0x7fa4b00a1340 2023-09-30T22:33:57.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.103+0000 7fa4a5ffb700 1 -- 172.21.15.16:0/2902035003 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa4a0021030 con 0x7fa4b00a1340 2023-09-30T22:33:57.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.103+0000 7fa4b7562700 1 -- 172.21.15.16:0/2902035003 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4b00a1d40 con 0x7fa4b00a1340 2023-09-30T22:33:57.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.103+0000 7fa4a5ffb700 1 -- 172.21.15.16:0/2902035003 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fa4a0040c40 con 0x7fa4b00a1340 2023-09-30T22:33:57.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.112+0000 7fa4a5ffb700 1 -- 172.21.15.16:0/2902035003 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa4a00199a0 con 0x7fa4b00a1340 2023-09-30T22:33:57.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.132+0000 7fd6e940e700 1 Processor -- start 2023-09-30T22:33:57.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.132+0000 7fd6e940e700 1 -- start start 2023-09-30T22:33:57.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.132+0000 7fd6e940e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd6e409d6f0 0x7fd6e409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:57.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.132+0000 7fd6e940e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd6e409e000 0x7fd6e4130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:57.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.132+0000 7fd6e940e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6e409e410 0x7fd6e4130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:57.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.132+0000 7fd6e940e700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6e409f520 con 0x7fd6e409d6f0 2023-09-30T22:33:57.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.132+0000 7fd6e940e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e409f130 con 0x7fd6e409e410 2023-09-30T22:33:57.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.132+0000 7fd6e940e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e409e930 con 0x7fd6e409e000 2023-09-30T22:33:57.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.133+0000 7fd6e37fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6e409e410 0x7fd6e4130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:57.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.133+0000 7fd6e37fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6e409e410 0x7fd6e4130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37838/0 (socket says 172.21.15.16:37838) 2023-09-30T22:33:57.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.133+0000 7fd6e37fe700 1 -- 172.21.15.16:0/3122248307 learned_addr learned my addr 172.21.15.16:0/3122248307 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:57.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.133+0000 7fd6e2ffd700 1 --2- 172.21.15.16:0/3122248307 >> v2:172.21.15.16:3300/0 conn(0x7fd6e409d6f0 0x7fd6e409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:57.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.133+0000 7fd6e27fc700 1 --2- 172.21.15.16:0/3122248307 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd6e409e000 0x7fd6e4130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:57.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.133+0000 7fd6e37fe700 1 -- 172.21.15.16:0/3122248307 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd6e409e000 msgr2=0x7fd6e4130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:57.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.133+0000 7fd6e37fe700 1 --2- 172.21.15.16:0/3122248307 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd6e409e000 0x7fd6e4130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:57.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.133+0000 7fd6e37fe700 1 -- 172.21.15.16:0/3122248307 >> v2:172.21.15.16:3300/0 conn(0x7fd6e409d6f0 msgr2=0x7fd6e409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:57.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.133+0000 7fd6e37fe700 1 --2- 172.21.15.16:0/3122248307 >> v2:172.21.15.16:3300/0 conn(0x7fd6e409d6f0 0x7fd6e409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:57.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.133+0000 7fd6e37fe700 1 -- 172.21.15.16:0/3122248307 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e409b2d0 con 0x7fd6e409e410 2023-09-30T22:33:57.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.133+0000 7fd6e2ffd700 1 --2- 172.21.15.16:0/3122248307 >> v2:172.21.15.16:3300/0 conn(0x7fd6e409d6f0 0x7fd6e409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:57.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.133+0000 7fd6e37fe700 1 --2- 172.21.15.16:0/3122248307 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6e409e410 0x7fd6e4130ba0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc00d150 tx=0x7fd6dc00d460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e71e23913b13f31 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:57.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.134+0000 7fd6e1ffb700 1 -- 172.21.15.16:0/3122248307 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc010070 con 0x7fd6e409e410 2023-09-30T22:33:57.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.134+0000 7fd6e1ffb700 1 -- 172.21.15.16:0/3122248307 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6dc00dd10 con 0x7fd6e409e410 2023-09-30T22:33:57.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.134+0000 7fd6e1ffb700 1 -- 172.21.15.16:0/3122248307 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc00b310 con 0x7fd6e409e410 2023-09-30T22:33:57.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.134+0000 7fd6e940e700 1 -- 172.21.15.16:0/3122248307 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6e409e410 msgr2=0x7fd6e4130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:57.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.134+0000 7fd6e940e700 1 --2- 172.21.15.16:0/3122248307 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6e409e410 0x7fd6e4130ba0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc00d150 tx=0x7fd6dc00d460 comp rx=0 tx=0).stop 2023-09-30T22:33:57.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.134+0000 7fd6e940e700 1 -- 172.21.15.16:0/3122248307 shutdown_connections 2023-09-30T22:33:57.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.134+0000 7fd6e940e700 1 --2- 172.21.15.16:0/3122248307 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6e409e410 0x7fd6e4130ba0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:57.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.134+0000 7fd6e940e700 1 --2- 172.21.15.16:0/3122248307 >> v2:172.21.15.16:3300/0 conn(0x7fd6e409d6f0 0x7fd6e409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:57.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.134+0000 7fd6e940e700 1 --2- 172.21.15.16:0/3122248307 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd6e409e000 0x7fd6e4130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:57.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.134+0000 7fd6e940e700 1 -- 172.21.15.16:0/3122248307 >> 172.21.15.16:0/3122248307 conn(0x7fd6e4066f50 msgr2=0x7fd6e409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:57.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.135+0000 7fd6e940e700 1 -- 172.21.15.16:0/3122248307 shutdown_connections 2023-09-30T22:33:57.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.135+0000 7fd6e940e700 1 -- 172.21.15.16:0/3122248307 wait complete. 2023-09-30T22:33:57.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.136+0000 7fd6e940e700 1 Processor -- start 2023-09-30T22:33:57.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.136+0000 7fd6e940e700 1 -- start start 2023-09-30T22:33:57.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.136+0000 7fd6e940e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd6e409d6f0 0x7fd6e411dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:57.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.136+0000 7fd6e940e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6e409e000 0x7fd6e411e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:57.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.136+0000 7fd6e940e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6e409e410 0x7fd6e411e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:57.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.136+0000 7fd6e940e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e41210f0 con 0x7fd6e409e000 2023-09-30T22:33:57.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.136+0000 7fd6e940e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e409c700 con 0x7fd6e409e410 2023-09-30T22:33:57.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.136+0000 7fd6e940e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e40a3ad0 con 0x7fd6e409d6f0 2023-09-30T22:33:57.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.136+0000 7fd6e37fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6e409e410 0x7fd6e411e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:57.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.136+0000 7fd6e2ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd6e409d6f0 0x7fd6e411dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:57.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.136+0000 7fd6e37fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6e409e410 0x7fd6e411e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37842/0 (socket says 172.21.15.16:37842) 2023-09-30T22:33:57.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.136+0000 7fd6e37fe700 1 -- 172.21.15.16:0/4115369268 learned_addr learned my addr 172.21.15.16:0/4115369268 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:57.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.137+0000 7fd6e27fc700 1 --2- 172.21.15.16:0/4115369268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6e409e000 0x7fd6e411e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:57.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.137+0000 7fd6e2ffd700 1 -- 172.21.15.16:0/4115369268 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6e409e410 msgr2=0x7fd6e411e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:57.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.137+0000 7fd6e2ffd700 1 --2- 172.21.15.16:0/4115369268 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6e409e410 0x7fd6e411e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:57.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.137+0000 7fd6e2ffd700 1 -- 172.21.15.16:0/4115369268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6e409e000 msgr2=0x7fd6e411e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:57.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.137+0000 7fd6e2ffd700 1 --2- 172.21.15.16:0/4115369268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6e409e000 0x7fd6e411e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:57.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.137+0000 7fd6e2ffd700 1 -- 172.21.15.16:0/4115369268 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d40094a0 con 0x7fd6e409d6f0 2023-09-30T22:33:57.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.137+0000 7fd6e2ffd700 1 --2- 172.21.15.16:0/4115369268 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd6e409d6f0 0x7fd6e411dee0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fd6d400e980 tx=0x7fd6d400ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:57.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.137+0000 7fd6cb7fe700 1 -- 172.21.15.16:0/4115369268 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d4004030 con 0x7fd6e409d6f0 2023-09-30T22:33:57.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.137+0000 7fd6cb7fe700 1 -- 172.21.15.16:0/4115369268 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6d40041b0 con 0x7fd6e409d6f0 2023-09-30T22:33:57.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.137+0000 7fd6e940e700 1 -- 172.21.15.16:0/4115369268 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6dc00bd40 con 0x7fd6e409d6f0 2023-09-30T22:33:57.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.137+0000 7fd6cb7fe700 1 -- 172.21.15.16:0/4115369268 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d4004560 con 0x7fd6e409d6f0 2023-09-30T22:33:57.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.137+0000 7fd6e940e700 1 -- 172.21.15.16:0/4115369268 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6e4120ca0 con 0x7fd6e409d6f0 2023-09-30T22:33:57.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.139+0000 7fd6e940e700 1 -- 172.21.15.16:0/4115369268 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6e411ac20 con 0x7fd6e409d6f0 2023-09-30T22:33:57.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.139+0000 7fd6cb7fe700 1 -- 172.21.15.16:0/4115369268 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd6d40036a0 con 0x7fd6e409d6f0 2023-09-30T22:33:57.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.140+0000 7fd6cb7fe700 1 -- 172.21.15.16:0/4115369268 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd6d403fca0 con 0x7fd6e409d6f0 2023-09-30T22:33:57.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.144+0000 7fd6cb7fe700 1 -- 172.21.15.16:0/4115369268 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd6d4003950 con 0x7fd6e409d6f0 2023-09-30T22:33:57.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.172+0000 7f48b5ed9700 1 Processor -- start 2023-09-30T22:33:57.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.172+0000 7f48b5ed9700 1 -- start start 2023-09-30T22:33:57.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.172+0000 7f48b5ed9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f48b009d630 0x7f48b00a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:57.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.172+0000 7f48b5ed9700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f48b0138d80 0x7f48b013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:57.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.173+0000 7f48b5ed9700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48b00a2a70 con 0x7f48b00a0a30 2023-09-30T22:33:57.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.173+0000 7f48b5ed9700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48b00a01d0 con 0x7f48b0138d80 2023-09-30T22:33:57.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.173+0000 7f48b5ed9700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48b009e2b0 con 0x7f48b009d630 2023-09-30T22:33:57.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.173+0000 7f48affff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f48b0138d80 0x7f48b013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:57.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.173+0000 7f48af7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f48b00a0a30 0x7f48b00a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50148/0 (socket says 172.21.15.16:50148) 2023-09-30T22:33:57.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.173+0000 7f48affff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f48b0138d80 0x7f48b013b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43824/0 (socket says 172.21.15.16:43824) 2023-09-30T22:33:57.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.173+0000 7f48af7fe700 1 -- 172.21.15.16:0/4103945366 learned_addr learned my addr 172.21.15.16:0/4103945366 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:57.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.173+0000 7f48aeffd700 1 --2- 172.21.15.16:0/4103945366 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f48b009d630 0x7f48b00a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:57.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.173+0000 7f48ae7fc700 1 -- 172.21.15.16:0/4103945366 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3935138423 0 0) 0x7f48b00a2a70 con 0x7f48b00a0a30 2023-09-30T22:33:57.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.173+0000 7f48ae7fc700 1 -- 172.21.15.16:0/4103945366 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48980035b0 con 0x7f48b00a0a30 2023-09-30T22:33:57.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.173+0000 7f48aeffd700 1 -- 172.21.15.16:0/4103945366 >> v2:172.21.15.16:3300/0 conn(0x7f48b0138d80 msgr2=0x7f48b013b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:57.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.173+0000 7f48ae7fc700 1 -- 172.21.15.16:0/4103945366 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3797561842 0 0) 0x7f48980035b0 con 0x7f48b00a0a30 2023-09-30T22:33:57.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.173+0000 7f48aeffd700 1 --2- 172.21.15.16:0/4103945366 >> v2:172.21.15.16:3300/0 conn(0x7f48b0138d80 0x7f48b013b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:57.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.173+0000 7f48aeffd700 1 -- 172.21.15.16:0/4103945366 >> v1:172.21.15.16:6789/0 conn(0x7f48b00a0a30 legacy=0x7f48b00a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:57.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.173+0000 7f48aeffd700 1 -- 172.21.15.16:0/4103945366 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b009dc50 con 0x7f48b009d630 2023-09-30T22:33:57.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.174+0000 7f48aeffd700 1 --2- 172.21.15.16:0/4103945366 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f48b009d630 0x7f48b00a1510 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f48a4009840 tx=0x7f48a4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80cd36043df7f8d1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:57.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.174+0000 7f48ae7fc700 1 -- 172.21.15.16:0/4103945366 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a0001670 con 0x7f48b009d630 2023-09-30T22:33:57.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.174+0000 7f48ae7fc700 1 -- 172.21.15.16:0/4103945366 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48a4004030 con 0x7f48b009d630 2023-09-30T22:33:57.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.174+0000 7f48ae7fc700 1 -- 172.21.15.16:0/4103945366 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a40043c0 con 0x7f48b009d630 2023-09-30T22:33:57.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.175+0000 7f48b5ed9700 1 -- 172.21.15.16:0/4103945366 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f48b009d630 msgr2=0x7f48b00a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:57.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.175+0000 7f48b5ed9700 1 --2- 172.21.15.16:0/4103945366 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f48b009d630 0x7f48b00a1510 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f48a4009840 tx=0x7f48a4009b50 comp rx=0 tx=0).stop 2023-09-30T22:33:57.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.175+0000 7f48b5ed9700 1 -- 172.21.15.16:0/4103945366 shutdown_connections 2023-09-30T22:33:57.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.175+0000 7f48b5ed9700 1 --2- 172.21.15.16:0/4103945366 >> v2:172.21.15.16:3300/0 conn(0x7f48b0138d80 0x7f48b013b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:57.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.175+0000 7f48b5ed9700 1 --2- 172.21.15.16:0/4103945366 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f48b009d630 0x7f48b00a1510 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:57.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.175+0000 7f48b5ed9700 1 -- 172.21.15.16:0/4103945366 >> 172.21.15.16:0/4103945366 conn(0x7f48b00a3050 msgr2=0x7f48b00a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:57.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.175+0000 7f48b5ed9700 1 -- 172.21.15.16:0/4103945366 shutdown_connections 2023-09-30T22:33:57.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.175+0000 7f48b5ed9700 1 -- 172.21.15.16:0/4103945366 wait complete. 2023-09-30T22:33:57.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.176+0000 7f48b5ed9700 1 Processor -- start 2023-09-30T22:33:57.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.176+0000 7f48b5ed9700 1 -- start start 2023-09-30T22:33:57.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.176+0000 7f48b5ed9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f48b009d630 0x7f48b01208f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:57.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.177+0000 7f48b5ed9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f48b00a0a30 0x7f48b0120e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:57.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.177+0000 7f48b5ed9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48b0138d80 0x7f48b0121370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:57.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.177+0000 7f48b5ed9700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48b0124380 con 0x7f48b0138d80 2023-09-30T22:33:57.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.177+0000 7f48b5ed9700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48b00a4c70 con 0x7f48b009d630 2023-09-30T22:33:57.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.177+0000 7f48b5ed9700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f48b00a0410 con 0x7f48b00a0a30 2023-09-30T22:33:57.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.177+0000 7f48af7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f48b009d630 0x7f48b01208f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:57.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.177+0000 7f48aeffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f48b00a0a30 0x7f48b0120e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:57.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.177+0000 7f48affff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48b0138d80 0x7f48b0121370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:57.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.177+0000 7f48af7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f48b009d630 0x7f48b01208f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37860/0 (socket says 172.21.15.16:37860) 2023-09-30T22:33:57.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.177+0000 7f48aeffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f48b00a0a30 0x7f48b0120e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37902/0 (socket says 172.21.15.16:37902) 2023-09-30T22:33:57.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.177+0000 7f48af7fe700 1 -- 172.21.15.16:0/505914984 learned_addr learned my addr 172.21.15.16:0/505914984 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:57.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.177+0000 7f48affff700 1 --2- 172.21.15.16:0/505914984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48b0138d80 0x7f48b0121370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43834/0 (socket says 172.21.15.16:43834) 2023-09-30T22:33:57.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.177+0000 7f48af7fe700 1 -- 172.21.15.16:0/505914984 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f48b00a0a30 msgr2=0x7f48b0120e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:57.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.177+0000 7f48af7fe700 1 --2- 172.21.15.16:0/505914984 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f48b00a0a30 0x7f48b0120e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:57.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.177+0000 7f48af7fe700 1 -- 172.21.15.16:0/505914984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48b0138d80 msgr2=0x7f48b0121370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:57.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.177+0000 7f48af7fe700 1 --2- 172.21.15.16:0/505914984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48b0138d80 0x7f48b0121370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:57.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.177+0000 7f48af7fe700 1 -- 172.21.15.16:0/505914984 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a40094a0 con 0x7f48b009d630 2023-09-30T22:33:57.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.177+0000 7f48affff700 1 --2- 172.21.15.16:0/505914984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48b0138d80 0x7f48b0121370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:57.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.177+0000 7f48af7fe700 1 --2- 172.21.15.16:0/505914984 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f48b009d630 0x7f48b01208f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f48a00011e0 tx=0x7f48a000c9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:57.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.178+0000 7f4897fff700 1 -- 172.21.15.16:0/505914984 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a0004c80 con 0x7f48b009d630 2023-09-30T22:33:57.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.178+0000 7f4897fff700 1 -- 172.21.15.16:0/505914984 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48a0004e00 con 0x7f48b009d630 2023-09-30T22:33:57.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.178+0000 7f48b5ed9700 1 -- 172.21.15.16:0/505914984 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48b0123de0 con 0x7f48b009d630 2023-09-30T22:33:57.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.178+0000 7f4897fff700 1 -- 172.21.15.16:0/505914984 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a0007310 con 0x7f48b009d630 2023-09-30T22:33:57.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.178+0000 7f48b5ed9700 1 -- 172.21.15.16:0/505914984 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48b0122c60 con 0x7f48b009d630 2023-09-30T22:33:57.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.178+0000 7f4897fff700 1 -- 172.21.15.16:0/505914984 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f48a0007470 con 0x7f48b009d630 2023-09-30T22:33:57.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.179+0000 7f4897fff700 1 -- 172.21.15.16:0/505914984 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f48a003f4a0 con 0x7f48b009d630 2023-09-30T22:33:57.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.180+0000 7f48b5ed9700 1 -- 172.21.15.16:0/505914984 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48b0123750 con 0x7f48b009d630 2023-09-30T22:33:57.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.185+0000 7f4897fff700 1 -- 172.21.15.16:0/505914984 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f48a00123b0 con 0x7f48b009d630 2023-09-30T22:33:57.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.352+0000 7fd6e940e700 1 -- 172.21.15.16:0/4115369268 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd6e4121320 con 0x7fd6e409d6f0 2023-09-30T22:33:57.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.352+0000 7fd6cb7fe700 1 -- 172.21.15.16:0/4115369268 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd6e4121320 con 0x7fd6e409d6f0 2023-09-30T22:33:57.353 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:33:57.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.353+0000 7f2c45165700 1 -- 172.21.15.16:0/23075042 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2c4009f1a0 con 0x7f2c4006a770 2023-09-30T22:33:57.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.353+0000 7f2c277fe700 1 -- 172.21.15.16:0/23075042 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2c30017030 con 0x7f2c4006a770 2023-09-30T22:33:57.354 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:33:57.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.356+0000 7fd6e940e700 1 -- 172.21.15.16:0/4115369268 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd6e409d6f0 msgr2=0x7fd6e411dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:57.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.356+0000 7fd6e940e700 1 --2- 172.21.15.16:0/4115369268 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd6e409d6f0 0x7fd6e411dee0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fd6d400e980 tx=0x7fd6d400ec90 comp rx=0 tx=0).stop 2023-09-30T22:33:57.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.356+0000 7fd6e940e700 1 -- 172.21.15.16:0/4115369268 shutdown_connections 2023-09-30T22:33:57.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.356+0000 7fd6e940e700 1 --2- 172.21.15.16:0/4115369268 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6e409e410 0x7fd6e411e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:57.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.356+0000 7fd6e940e700 1 --2- 172.21.15.16:0/4115369268 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd6e409d6f0 0x7fd6e411dee0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:57.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.356+0000 7fd6e940e700 1 --2- 172.21.15.16:0/4115369268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6e409e000 0x7fd6e411e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:57.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.356+0000 7fd6e940e700 1 -- 172.21.15.16:0/4115369268 >> 172.21.15.16:0/4115369268 conn(0x7fd6e4066f50 msgr2=0x7fd6e4130920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:57.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.356+0000 7fd6e940e700 1 -- 172.21.15.16:0/4115369268 shutdown_connections 2023-09-30T22:33:57.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.356+0000 7fd6e940e700 1 -- 172.21.15.16:0/4115369268 wait complete. 2023-09-30T22:33:57.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.357+0000 7f2c45165700 1 -- 172.21.15.16:0/23075042 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c4006a770 msgr2=0x7f2c4013b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:57.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.357+0000 7f2c45165700 1 --2- 172.21.15.16:0/23075042 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c4006a770 0x7f2c4013b9f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f2c3000c4e0 tx=0x7f2c3000c7f0 comp rx=0 tx=0).stop 2023-09-30T22:33:57.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.357+0000 7f2c45165700 1 -- 172.21.15.16:0/23075042 shutdown_connections 2023-09-30T22:33:57.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.357+0000 7f2c45165700 1 --2- 172.21.15.16:0/23075042 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c4013c470 0x7f2c401350b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:57.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.357+0000 7f2c45165700 1 --2- 172.21.15.16:0/23075042 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c4006a770 0x7f2c4013b9f0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:57.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.357+0000 7f2c45165700 1 --2- 172.21.15.16:0/23075042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c4013d1f0 0x7f2c4013bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:57.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.357+0000 7f2c45165700 1 -- 172.21.15.16:0/23075042 >> 172.21.15.16:0/23075042 conn(0x7f2c40066f50 msgr2=0x7f2c4009b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:57.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.357+0000 7f2c45165700 1 -- 172.21.15.16:0/23075042 shutdown_connections 2023-09-30T22:33:57.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.357+0000 7f2c45165700 1 -- 172.21.15.16:0/23075042 wait complete. 2023-09-30T22:33:57.372 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:33:57.372 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:33:57.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.391+0000 7fa4b7562700 1 -- 172.21.15.16:0/2902035003 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa4b0074ea0 con 0x7fa4b00a1340 2023-09-30T22:33:57.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.391+0000 7fa4a5ffb700 1 -- 172.21.15.16:0/2902035003 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa4a0015070 con 0x7fa4b00a1340 2023-09-30T22:33:57.392 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:33:57.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.394+0000 7fa4b7562700 1 -- 172.21.15.16:0/2902035003 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa4b00a1340 msgr2=0x7fa4b01c9cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:57.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.394+0000 7fa4b7562700 1 --2- 172.21.15.16:0/2902035003 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa4b00a1340 0x7fa4b01c9cc0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fa4a0009c40 tx=0x7fa4a0004750 comp rx=0 tx=0).stop 2023-09-30T22:33:57.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.394+0000 7f48b5ed9700 1 -- 172.21.15.16:0/505914984 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f48b00747f0 con 0x7f48b009d630 2023-09-30T22:33:57.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.394+0000 7fa4b7562700 1 -- 172.21.15.16:0/2902035003 shutdown_connections 2023-09-30T22:33:57.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.394+0000 7fa4b7562700 1 --2- 172.21.15.16:0/2902035003 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa4b00a0a30 0x7fa4b01c9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:57.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.394+0000 7fa4b7562700 1 --2- 172.21.15.16:0/2902035003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa4b0133fa0 0x7fa4b01cc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:57.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.394+0000 7fa4b7562700 1 --2- 172.21.15.16:0/2902035003 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa4b00a1340 0x7fa4b01c9cc0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:57.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.394+0000 7fa4b7562700 1 -- 172.21.15.16:0/2902035003 >> 172.21.15.16:0/2902035003 conn(0x7fa4b00a6550 msgr2=0x7fa4b01332a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:57.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.394+0000 7fa4b7562700 1 -- 172.21.15.16:0/2902035003 shutdown_connections 2023-09-30T22:33:57.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.394+0000 7fa4b7562700 1 -- 172.21.15.16:0/2902035003 wait complete. 2023-09-30T22:33:57.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.394+0000 7f4897fff700 1 -- 172.21.15.16:0/505914984 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f48a001b340 con 0x7f48b009d630 2023-09-30T22:33:57.397 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:33:57.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.396+0000 7f48b5ed9700 1 -- 172.21.15.16:0/505914984 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f48b009d630 msgr2=0x7f48b01208f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:57.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.396+0000 7f48b5ed9700 1 --2- 172.21.15.16:0/505914984 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f48b009d630 0x7f48b01208f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f48a00011e0 tx=0x7f48a000c9e0 comp rx=0 tx=0).stop 2023-09-30T22:33:57.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.396+0000 7f48b5ed9700 1 -- 172.21.15.16:0/505914984 shutdown_connections 2023-09-30T22:33:57.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.396+0000 7f48b5ed9700 1 --2- 172.21.15.16:0/505914984 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f48b009d630 0x7f48b01208f0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:57.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.396+0000 7f48b5ed9700 1 --2- 172.21.15.16:0/505914984 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48b0138d80 0x7f48b0121370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:57.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.396+0000 7f48b5ed9700 1 --2- 172.21.15.16:0/505914984 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f48b00a0a30 0x7f48b0120e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:57.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.396+0000 7f48b5ed9700 1 -- 172.21.15.16:0/505914984 >> 172.21.15.16:0/505914984 conn(0x7f48b00a3050 msgr2=0x7f48b009ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:57.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.396+0000 7f48b5ed9700 1 -- 172.21.15.16:0/505914984 shutdown_connections 2023-09-30T22:33:57.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:57.396+0000 7f48b5ed9700 1 -- 172.21.15.16:0/505914984 wait complete. 2023-09-30T22:33:57.409 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:33:57.411 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:33:58.372 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:33:58.373 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:33:58.410 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:33:58.412 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:33:58.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.527+0000 7f55b8aab700 1 Processor -- start 2023-09-30T22:33:58.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.527+0000 7f55b8aab700 1 -- start start 2023-09-30T22:33:58.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.528+0000 7f55b8aab700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55b409e1d0 0x7f55b4138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:58.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.528+0000 7f55b8aab700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f55b4139170 0x7f55b413b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:58.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.528+0000 7f55b8aab700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55b409b120 con 0x7f55b409d6f0 2023-09-30T22:33:58.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.528+0000 7f55b8aab700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55b406b460 con 0x7f55b409e1d0 2023-09-30T22:33:58.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.528+0000 7f55b8aab700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f55b409a7e0 con 0x7f55b4139170 2023-09-30T22:33:58.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.529+0000 7f55b2d9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f55b4139170 0x7f55b413b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:58.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.529+0000 7f55b1d9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55b409e1d0 0x7f55b4138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:58.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.529+0000 7f55b259c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f55b409d6f0 0x7f55b409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47380/0 (socket says 172.21.15.16:47380) 2023-09-30T22:33:58.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.529+0000 7f55b259c700 1 -- 172.21.15.16:0/1402046384 learned_addr learned my addr 172.21.15.16:0/1402046384 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:58.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.529+0000 7f55b159a700 1 -- 172.21.15.16:0/1402046384 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2626090861 0 0) 0x7f55b409b120 con 0x7f55b409d6f0 2023-09-30T22:33:58.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.529+0000 7f55b159a700 1 -- 172.21.15.16:0/1402046384 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f559c0035b0 con 0x7f55b409d6f0 2023-09-30T22:33:58.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.529+0000 7f55b1d9b700 1 -- 172.21.15.16:0/1402046384 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f55b4139170 msgr2=0x7f55b413b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:58.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.529+0000 7f55b1d9b700 1 --2- 172.21.15.16:0/1402046384 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f55b4139170 0x7f55b413b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:58.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.529+0000 7f55b1d9b700 1 -- 172.21.15.16:0/1402046384 >> v1:172.21.15.16:6789/0 conn(0x7f55b409d6f0 legacy=0x7f55b409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:58.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.529+0000 7f55b1d9b700 1 -- 172.21.15.16:0/1402046384 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b409a500 con 0x7f55b409e1d0 2023-09-30T22:33:58.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.529+0000 7f55b2d9d700 1 --2- 172.21.15.16:0/1402046384 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f55b4139170 0x7f55b413b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:58.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.529+0000 7f55b1d9b700 1 --2- 172.21.15.16:0/1402046384 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55b409e1d0 0x7f55b4138c30 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f55a8009870 tx=0x7f55a8009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88da1e162c744c47 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:58.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.529+0000 7f55b159a700 1 -- 172.21.15.16:0/1402046384 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a8004230 con 0x7f55b409e1d0 2023-09-30T22:33:58.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.530+0000 7f55b159a700 1 -- 172.21.15.16:0/1402046384 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55a8004390 con 0x7f55b409e1d0 2023-09-30T22:33:58.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.530+0000 7f55b159a700 1 -- 172.21.15.16:0/1402046384 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a80036a0 con 0x7f55b409e1d0 2023-09-30T22:33:58.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.531+0000 7f55b8aab700 1 -- 172.21.15.16:0/1402046384 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55b409e1d0 msgr2=0x7f55b4138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:58.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.531+0000 7f55b8aab700 1 --2- 172.21.15.16:0/1402046384 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55b409e1d0 0x7f55b4138c30 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f55a8009870 tx=0x7f55a8009b80 comp rx=0 tx=0).stop 2023-09-30T22:33:58.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.531+0000 7f55b8aab700 1 -- 172.21.15.16:0/1402046384 shutdown_connections 2023-09-30T22:33:58.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.531+0000 7f55b8aab700 1 --2- 172.21.15.16:0/1402046384 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f55b4139170 0x7f55b413b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:58.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.531+0000 7f55b8aab700 1 --2- 172.21.15.16:0/1402046384 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55b409e1d0 0x7f55b4138c30 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:58.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.531+0000 7f55b8aab700 1 -- 172.21.15.16:0/1402046384 >> 172.21.15.16:0/1402046384 conn(0x7f55b4066f50 msgr2=0x7f55b409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:58.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.531+0000 7f55b8aab700 1 -- 172.21.15.16:0/1402046384 shutdown_connections 2023-09-30T22:33:58.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.531+0000 7f55b8aab700 1 -- 172.21.15.16:0/1402046384 wait complete. 2023-09-30T22:33:58.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.532+0000 7f55b8aab700 1 Processor -- start 2023-09-30T22:33:58.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.533+0000 7f55b8aab700 1 -- start start 2023-09-30T22:33:58.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.533+0000 7f55b8aab700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55b409d6f0 0x7f55b411e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:58.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.533+0000 7f55b8aab700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55b409e1d0 0x7f55b411ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:58.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.533+0000 7f55b8aab700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f55b4139170 0x7f55b411f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:58.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.533+0000 7f55b8aab700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b40a3a60 con 0x7f55b409e1d0 2023-09-30T22:33:58.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.533+0000 7f55b8aab700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55b409c710 con 0x7f55b409d6f0 2023-09-30T22:33:58.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.533+0000 7f55b8aab700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f55b41205f0 con 0x7f55b4139170 2023-09-30T22:33:58.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.533+0000 7f55b1d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55b409e1d0 0x7f55b411ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:58.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.533+0000 7f55b259c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55b409d6f0 0x7f55b411e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:58.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.534+0000 7f55b1d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55b409e1d0 0x7f55b411ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37120/0 (socket says 172.21.15.16:37120) 2023-09-30T22:33:58.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.534+0000 7f55b1d9b700 1 -- 172.21.15.16:0/340694109 learned_addr learned my addr 172.21.15.16:0/340694109 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:58.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.534+0000 7f55b259c700 1 --2- 172.21.15.16:0/340694109 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55b409d6f0 0x7f55b411e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38456/0 (socket says 172.21.15.16:38456) 2023-09-30T22:33:58.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.534+0000 7f55b2d9d700 1 --2- 172.21.15.16:0/340694109 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f55b4139170 0x7f55b411f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:58.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.534+0000 7f55b259c700 1 -- 172.21.15.16:0/340694109 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f55b4139170 msgr2=0x7f55b411f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:58.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.534+0000 7f55b259c700 1 --2- 172.21.15.16:0/340694109 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f55b4139170 0x7f55b411f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:58.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.534+0000 7f55b259c700 1 -- 172.21.15.16:0/340694109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55b409e1d0 msgr2=0x7f55b411ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:58.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.534+0000 7f55b259c700 1 --2- 172.21.15.16:0/340694109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55b409e1d0 0x7f55b411ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:58.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.534+0000 7f55b259c700 1 -- 172.21.15.16:0/340694109 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55ac009a70 con 0x7f55b409d6f0 2023-09-30T22:33:58.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.534+0000 7f55b259c700 1 --2- 172.21.15.16:0/340694109 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55b409d6f0 0x7f55b411e8d0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f55a400c4b0 tx=0x7f55a400c7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:58.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.534+0000 7f55a2ffd700 1 -- 172.21.15.16:0/340694109 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a4004c80 con 0x7f55b409d6f0 2023-09-30T22:33:58.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.534+0000 7f55a2ffd700 1 -- 172.21.15.16:0/340694109 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55a4004de0 con 0x7f55b409d6f0 2023-09-30T22:33:58.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.534+0000 7f55b8aab700 1 -- 172.21.15.16:0/340694109 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55a80094a0 con 0x7f55b409d6f0 2023-09-30T22:33:58.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.534+0000 7f55b8aab700 1 -- 172.21.15.16:0/340694109 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55b4122160 con 0x7f55b409d6f0 2023-09-30T22:33:58.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.535+0000 7f55a2ffd700 1 -- 172.21.15.16:0/340694109 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a40072f0 con 0x7f55b409d6f0 2023-09-30T22:33:58.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.536+0000 7f55b8aab700 1 -- 172.21.15.16:0/340694109 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55980052e0 con 0x7f55b409d6f0 2023-09-30T22:33:58.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.536+0000 7f55a2ffd700 1 -- 172.21.15.16:0/340694109 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f55a4007450 con 0x7f55b409d6f0 2023-09-30T22:33:58.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.536+0000 7f55a2ffd700 1 -- 172.21.15.16:0/340694109 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f55a403f170 con 0x7f55b409d6f0 2023-09-30T22:33:58.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.536+0000 7f0da9844700 1 Processor -- start 2023-09-30T22:33:58.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.536+0000 7f0da9844700 1 -- start start 2023-09-30T22:33:58.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.537+0000 7f0da9844700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0da409e1d0 0x7f0da4138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:58.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.537+0000 7f0da9844700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0da4139170 0x7f0da413b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:58.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.537+0000 7f0da9844700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0da409b120 con 0x7f0da409d6f0 2023-09-30T22:33:58.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.537+0000 7f0da9844700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0da406b460 con 0x7f0da4139170 2023-09-30T22:33:58.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.537+0000 7f0da9844700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0da409a7e0 con 0x7f0da409e1d0 2023-09-30T22:33:58.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.537+0000 7f0d9a7fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0da409e1d0 0x7f0da4138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:58.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.537+0000 7f0d9a7fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0da409e1d0 0x7f0da4138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:57224/0 (socket says 172.21.15.16:57224) 2023-09-30T22:33:58.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.537+0000 7f0d9a7fc700 1 -- 172.21.15.16:0/2094765063 learned_addr learned my addr 172.21.15.16:0/2094765063 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:58.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.537+0000 7f0d9a7fc700 1 -- 172.21.15.16:0/2094765063 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0da4139170 msgr2=0x7f0da413b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:58.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.537+0000 7f0d9a7fc700 1 --2- 172.21.15.16:0/2094765063 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0da4139170 0x7f0da413b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:58.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.537+0000 7f0d9a7fc700 1 -- 172.21.15.16:0/2094765063 >> v1:172.21.15.16:6789/0 conn(0x7f0da409d6f0 legacy=0x7f0da409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:58.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.538+0000 7f0d9a7fc700 1 -- 172.21.15.16:0/2094765063 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0da409a500 con 0x7f0da409e1d0 2023-09-30T22:33:58.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.538+0000 7f0da27fc700 1 -- 172.21.15.16:0/2094765063 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1451321923 0 0) 0x7f0da409b120 con 0x7f0da409d6f0 2023-09-30T22:33:58.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.538+0000 7f0d9a7fc700 1 --2- 172.21.15.16:0/2094765063 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0da409e1d0 0x7f0da4138c30 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c009950 tx=0x7f0d8c009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26dee1e94a083385 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:58.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.538+0000 7f0da27fc700 1 -- 172.21.15.16:0/2094765063 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c009ea0 con 0x7f0da409e1d0 2023-09-30T22:33:58.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.538+0000 7f0da27fc700 1 -- 172.21.15.16:0/2094765063 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d8c00ba20 con 0x7f0da409e1d0 2023-09-30T22:33:58.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.538+0000 7f0da27fc700 1 -- 172.21.15.16:0/2094765063 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c00bdb0 con 0x7f0da409e1d0 2023-09-30T22:33:58.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.539+0000 7f0da9844700 1 -- 172.21.15.16:0/2094765063 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0da409e1d0 msgr2=0x7f0da4138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:58.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.539+0000 7f0da9844700 1 --2- 172.21.15.16:0/2094765063 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0da409e1d0 0x7f0da4138c30 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c009950 tx=0x7f0d8c009c60 comp rx=0 tx=0).stop 2023-09-30T22:33:58.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.539+0000 7f0da9844700 1 -- 172.21.15.16:0/2094765063 shutdown_connections 2023-09-30T22:33:58.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.539+0000 7f0da9844700 1 --2- 172.21.15.16:0/2094765063 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0da4139170 0x7f0da413b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:58.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.539+0000 7f0da9844700 1 --2- 172.21.15.16:0/2094765063 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0da409e1d0 0x7f0da4138c30 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:58.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.539+0000 7f0da9844700 1 -- 172.21.15.16:0/2094765063 >> 172.21.15.16:0/2094765063 conn(0x7f0da4066f50 msgr2=0x7f0da409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:58.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.539+0000 7f0da9844700 1 -- 172.21.15.16:0/2094765063 shutdown_connections 2023-09-30T22:33:58.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.539+0000 7f0da9844700 1 -- 172.21.15.16:0/2094765063 wait complete. 2023-09-30T22:33:58.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.540+0000 7f0da9844700 1 Processor -- start 2023-09-30T22:33:58.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.540+0000 7f0da9844700 1 -- start start 2023-09-30T22:33:58.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.541+0000 7f0da9844700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0da409d6f0 0x7f0da411e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:58.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.541+0000 7f0da9844700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0da409e1d0 0x7f0da411ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:58.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.541+0000 7f0da9844700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0da4139170 0x7f0da411f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:58.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.541+0000 7f0da9844700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da40a3b40 con 0x7f0da409d6f0 2023-09-30T22:33:58.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.541+0000 7f0da9844700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0da4122360 con 0x7f0da4139170 2023-09-30T22:33:58.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.541+0000 7f0da9844700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0da4121a00 con 0x7f0da409e1d0 2023-09-30T22:33:58.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.541+0000 7f0da2ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0da409d6f0 0x7f0da411e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:58.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.541+0000 7f0da2ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0da409d6f0 0x7f0da411e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37130/0 (socket says 172.21.15.16:37130) 2023-09-30T22:33:58.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.541+0000 7f0da2ffd700 1 -- 172.21.15.16:0/1659016086 learned_addr learned my addr 172.21.15.16:0/1659016086 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:58.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.541+0000 7f0da37fe700 1 --2- 172.21.15.16:0/1659016086 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0da4139170 0x7f0da411f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:58.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.541+0000 7f0d9a7fc700 1 --2- 172.21.15.16:0/1659016086 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0da409e1d0 0x7f0da411ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:58.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.541+0000 7f0da2ffd700 1 -- 172.21.15.16:0/1659016086 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0da409e1d0 msgr2=0x7f0da411ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:58.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.541+0000 7f0da2ffd700 1 --2- 172.21.15.16:0/1659016086 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0da409e1d0 0x7f0da411ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:58.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.541+0000 7f0da2ffd700 1 -- 172.21.15.16:0/1659016086 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0da4139170 msgr2=0x7f0da411f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:58.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.541+0000 7f0da2ffd700 1 --2- 172.21.15.16:0/1659016086 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0da4139170 0x7f0da411f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:58.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.541+0000 7f0da2ffd700 1 -- 172.21.15.16:0/1659016086 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d8c009550 con 0x7f0da409d6f0 2023-09-30T22:33:58.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.542+0000 7f0da2ffd700 1 --2- 172.21.15.16:0/1659016086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0da409d6f0 0x7f0da411e730 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f0d9400c8c0 tx=0x7f0d9400cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:58.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.542+0000 7f0da0ff9700 1 -- 172.21.15.16:0/1659016086 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d94003e30 con 0x7f0da409d6f0 2023-09-30T22:33:58.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.542+0000 7f0da0ff9700 1 -- 172.21.15.16:0/1659016086 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d94005070 con 0x7f0da409d6f0 2023-09-30T22:33:58.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.542+0000 7f0da9844700 1 -- 172.21.15.16:0/1659016086 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0da4121cb0 con 0x7f0da409d6f0 2023-09-30T22:33:58.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.542+0000 7f0da0ff9700 1 -- 172.21.15.16:0/1659016086 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d94005400 con 0x7f0da409d6f0 2023-09-30T22:33:58.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.542+0000 7f0da9844700 1 -- 172.21.15.16:0/1659016086 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0da4120af0 con 0x7f0da409d6f0 2023-09-30T22:33:58.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.543+0000 7f0da0ff9700 1 -- 172.21.15.16:0/1659016086 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0d94005560 con 0x7f0da409d6f0 2023-09-30T22:33:58.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.543+0000 7f0da0ff9700 1 -- 172.21.15.16:0/1659016086 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f0d9403ed70 con 0x7f0da409d6f0 2023-09-30T22:33:58.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.543+0000 7f0da9844700 1 -- 172.21.15.16:0/1659016086 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0da4074ea0 con 0x7f0da409d6f0 2023-09-30T22:33:58.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.545+0000 7f55a2ffd700 1 -- 172.21.15.16:0/340694109 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f55a401bda0 con 0x7f55b409d6f0 2023-09-30T22:33:58.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.549+0000 7f0da0ff9700 1 -- 172.21.15.16:0/1659016086 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0d94017070 con 0x7f0da409d6f0 2023-09-30T22:33:58.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.571+0000 7f5a69ffe700 1 Processor -- start 2023-09-30T22:33:58.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.571+0000 7f5a69ffe700 1 -- start start 2023-09-30T22:33:58.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.571+0000 7f5a69ffe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5a6409eca0 0x7f5a6413cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:58.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.571+0000 7f5a69ffe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a6406f050 0x7f5a6413f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:58.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.571+0000 7f5a69ffe700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a64070ee0 con 0x7f5a6406e570 2023-09-30T22:33:58.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.571+0000 7f5a69ffe700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a64071980 con 0x7f5a6409eca0 2023-09-30T22:33:58.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.571+0000 7f5a69ffe700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5a64071480 con 0x7f5a6406f050 2023-09-30T22:33:58.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.571+0000 7f5a637fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5a6406e570 0x7f5a6406e940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47402/0 (socket says 172.21.15.16:47402) 2023-09-30T22:33:58.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.572+0000 7f5a637fe700 1 -- 172.21.15.16:0/262149087 learned_addr learned my addr 172.21.15.16:0/262149087 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:58.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.572+0000 7f5a62ffd700 1 --2- 172.21.15.16:0/262149087 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5a6409eca0 0x7f5a6413cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:58.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.572+0000 7f5a627fc700 1 -- 172.21.15.16:0/262149087 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 490744001 0 0) 0x7f5a64070ee0 con 0x7f5a6406e570 2023-09-30T22:33:58.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.572+0000 7f5a627fc700 1 -- 172.21.15.16:0/262149087 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a4c0035b0 con 0x7f5a6406e570 2023-09-30T22:33:58.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.572+0000 7f5a63fff700 1 --2- 172.21.15.16:0/262149087 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a6406f050 0x7f5a6413f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:58.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.572+0000 7f83bfb33700 1 Processor -- start 2023-09-30T22:33:58.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.572+0000 7f5a62ffd700 1 -- 172.21.15.16:0/262149087 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a6406f050 msgr2=0x7f5a6413f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:58.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.572+0000 7f5a62ffd700 1 --2- 172.21.15.16:0/262149087 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a6406f050 0x7f5a6413f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:58.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.572+0000 7f5a62ffd700 1 -- 172.21.15.16:0/262149087 >> v1:172.21.15.16:6789/0 conn(0x7f5a6406e570 legacy=0x7f5a6406e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:58.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.572+0000 7f5a62ffd700 1 -- 172.21.15.16:0/262149087 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a64071b90 con 0x7f5a6409eca0 2023-09-30T22:33:58.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.572+0000 7f83bfb33700 1 -- start start 2023-09-30T22:33:58.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.573+0000 7f5a62ffd700 1 --2- 172.21.15.16:0/262149087 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5a6409eca0 0x7f5a6413cea0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f5a58009870 tx=0x7f5a58009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ff0717fe5970a11 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:58.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.573+0000 7f5a627fc700 1 -- 172.21.15.16:0/262149087 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a58011830 con 0x7f5a6409eca0 2023-09-30T22:33:58.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.573+0000 7f5a627fc700 1 -- 172.21.15.16:0/262149087 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a58011990 con 0x7f5a6409eca0 2023-09-30T22:33:58.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.573+0000 7f5a627fc700 1 -- 172.21.15.16:0/262149087 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a58011d20 con 0x7f5a6409eca0 2023-09-30T22:33:58.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.573+0000 7f83bfb33700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83b8095690 0x7f83b8134cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:58.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.573+0000 7f83bfb33700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f83b8138430 0x7f83b813fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:58.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.573+0000 7f83bfb33700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83b809e800 con 0x7f83b81378f0 2023-09-30T22:33:58.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.573+0000 7f83bfb33700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83b8095a60 con 0x7f83b8095690 2023-09-30T22:33:58.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.573+0000 7f83bfb33700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f83b8094bd0 con 0x7f83b8138430 2023-09-30T22:33:58.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.573+0000 7f83bd8cf700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83b8095690 0x7f83b8134cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:58.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.573+0000 7f83be0d0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f83b8138430 0x7f83b813fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:58.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.573+0000 7f83bd8cf700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83b8095690 0x7f83b8134cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38486/0 (socket says 172.21.15.16:38486) 2023-09-30T22:33:58.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.573+0000 7f83bd8cf700 1 -- 172.21.15.16:0/2564831006 learned_addr learned my addr 172.21.15.16:0/2564831006 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:58.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.574+0000 7f83be0d0700 1 -- 172.21.15.16:0/2564831006 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83b8095690 msgr2=0x7f83b8134cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:58.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.574+0000 7f83be0d0700 1 --2- 172.21.15.16:0/2564831006 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83b8095690 0x7f83b8134cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:58.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.574+0000 7f83be0d0700 1 -- 172.21.15.16:0/2564831006 >> v1:172.21.15.16:6789/0 conn(0x7f83b81378f0 legacy=0x7f83b8137d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:58.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.574+0000 7f83be0d0700 1 -- 172.21.15.16:0/2564831006 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b80940e0 con 0x7f83b8138430 2023-09-30T22:33:58.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.574+0000 7f83bd8cf700 1 --2- 172.21.15.16:0/2564831006 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83b8095690 0x7f83b8134cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:58.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.574+0000 7f83be0d0700 1 --2- 172.21.15.16:0/2564831006 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f83b8138430 0x7f83b813fc30 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f83b0009dc0 tx=0x7f83b000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f30c82e1289ac37b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:58.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.574+0000 7f83bc8cd700 1 -- 172.21.15.16:0/2564831006 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b000e070 con 0x7f83b8138430 2023-09-30T22:33:58.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.574+0000 7f83bc8cd700 1 -- 172.21.15.16:0/2564831006 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83b000bae0 con 0x7f83b8138430 2023-09-30T22:33:58.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.574+0000 7f83bc8cd700 1 -- 172.21.15.16:0/2564831006 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b000be90 con 0x7f83b8138430 2023-09-30T22:33:58.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.574+0000 7f5a69ffe700 1 -- 172.21.15.16:0/262149087 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5a6409eca0 msgr2=0x7f5a6413cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:58.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.574+0000 7f5a69ffe700 1 --2- 172.21.15.16:0/262149087 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5a6409eca0 0x7f5a6413cea0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f5a58009870 tx=0x7f5a58009b80 comp rx=0 tx=0).stop 2023-09-30T22:33:58.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.574+0000 7f5a69ffe700 1 -- 172.21.15.16:0/262149087 shutdown_connections 2023-09-30T22:33:58.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.574+0000 7f5a69ffe700 1 --2- 172.21.15.16:0/262149087 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a6406f050 0x7f5a6413f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:58.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.574+0000 7f5a69ffe700 1 --2- 172.21.15.16:0/262149087 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5a6409eca0 0x7f5a6413cea0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:58.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.574+0000 7f5a69ffe700 1 -- 172.21.15.16:0/262149087 >> 172.21.15.16:0/262149087 conn(0x7f5a640a2aa0 msgr2=0x7f5a640a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:58.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.575+0000 7f5a69ffe700 1 -- 172.21.15.16:0/262149087 shutdown_connections 2023-09-30T22:33:58.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.575+0000 7f5a69ffe700 1 -- 172.21.15.16:0/262149087 wait complete. 2023-09-30T22:33:58.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.575+0000 7f83bfb33700 1 -- 172.21.15.16:0/2564831006 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f83b8138430 msgr2=0x7f83b813fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:58.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.575+0000 7f83bfb33700 1 --2- 172.21.15.16:0/2564831006 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f83b8138430 0x7f83b813fc30 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f83b0009dc0 tx=0x7f83b000b330 comp rx=0 tx=0).stop 2023-09-30T22:33:58.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.575+0000 7f83bfb33700 1 -- 172.21.15.16:0/2564831006 shutdown_connections 2023-09-30T22:33:58.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.575+0000 7f83bfb33700 1 --2- 172.21.15.16:0/2564831006 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f83b8138430 0x7f83b813fc30 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:58.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.575+0000 7f83bfb33700 1 --2- 172.21.15.16:0/2564831006 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83b8095690 0x7f83b8134cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:58.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.575+0000 7f83bfb33700 1 -- 172.21.15.16:0/2564831006 >> 172.21.15.16:0/2564831006 conn(0x7f83b80a2360 msgr2=0x7f83b80a2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:58.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.575+0000 7f83bfb33700 1 -- 172.21.15.16:0/2564831006 shutdown_connections 2023-09-30T22:33:58.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.575+0000 7f83bfb33700 1 -- 172.21.15.16:0/2564831006 wait complete. 2023-09-30T22:33:58.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.576+0000 7f5a69ffe700 1 Processor -- start 2023-09-30T22:33:58.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.576+0000 7f83bfb33700 1 Processor -- start 2023-09-30T22:33:58.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.577+0000 7f83bfb33700 1 -- start start 2023-09-30T22:33:58.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.576+0000 7f5a69ffe700 1 -- start start 2023-09-30T22:33:58.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.576+0000 7f5a69ffe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a6406e570 0x7f5a641231b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:58.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.576+0000 7f5a69ffe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5a6406f050 0x7f5a641236f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:58.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.576+0000 7f5a69ffe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a64123c30 0x7f5a641d2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:58.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.576+0000 7f5a69ffe700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a641269b0 con 0x7f5a64123c30 2023-09-30T22:33:58.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.576+0000 7f5a69ffe700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6413b470 con 0x7f5a6406f050 2023-09-30T22:33:58.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.576+0000 7f5a69ffe700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5a64126bb0 con 0x7f5a6406e570 2023-09-30T22:33:58.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.577+0000 7f5a63fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a64123c30 0x7f5a641d2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:58.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.577+0000 7f5a63fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a64123c30 0x7f5a641d2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37132/0 (socket says 172.21.15.16:37132) 2023-09-30T22:33:58.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.577+0000 7f5a63fff700 1 -- 172.21.15.16:0/3366700714 learned_addr learned my addr 172.21.15.16:0/3366700714 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:58.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.577+0000 7f5a62ffd700 1 --2- 172.21.15.16:0/3366700714 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5a6406f050 0x7f5a641236f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:58.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.577+0000 7f5a637fe700 1 --2- 172.21.15.16:0/3366700714 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a6406e570 0x7f5a641231b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:58.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.577+0000 7f5a63fff700 1 -- 172.21.15.16:0/3366700714 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a6406e570 msgr2=0x7f5a641231b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:58.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.577+0000 7f5a63fff700 1 --2- 172.21.15.16:0/3366700714 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a6406e570 0x7f5a641231b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:58.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.577+0000 7f5a63fff700 1 -- 172.21.15.16:0/3366700714 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5a6406f050 msgr2=0x7f5a641236f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:58.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.577+0000 7f5a63fff700 1 --2- 172.21.15.16:0/3366700714 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5a6406f050 0x7f5a641236f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:58.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.577+0000 7f5a63fff700 1 -- 172.21.15.16:0/3366700714 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a580094a0 con 0x7f5a64123c30 2023-09-30T22:33:58.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.577+0000 7f83bfb33700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83b8095690 0x7f83b81ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:58.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.577+0000 7f83bfb33700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83b81378f0 0x7f83b81cd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:58.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.577+0000 7f83bfb33700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f83b8138430 0x7f83b806eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:58.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.577+0000 7f83bfb33700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b8127240 con 0x7f83b8095690 2023-09-30T22:33:58.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.577+0000 7f83bfb33700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83b8072250 con 0x7f83b81378f0 2023-09-30T22:33:58.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.577+0000 7f83bfb33700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f83b80946d0 con 0x7f83b8138430 2023-09-30T22:33:58.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.577+0000 7f83bd8cf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83b8095690 0x7f83b81ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:58.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.577+0000 7f83bd0ce700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83b81378f0 0x7f83b81cd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:58.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.577+0000 7f83bd8cf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83b8095690 0x7f83b81ccc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37134/0 (socket says 172.21.15.16:37134) 2023-09-30T22:33:58.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.577+0000 7f83bd8cf700 1 -- 172.21.15.16:0/2254893631 learned_addr learned my addr 172.21.15.16:0/2254893631 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:58.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.578+0000 7f83be0d0700 1 --2- 172.21.15.16:0/2254893631 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f83b8138430 0x7f83b806eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:58.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.578+0000 7f5a63fff700 1 --2- 172.21.15.16:0/3366700714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a64123c30 0x7f5a641d2a70 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c00efc0 tx=0x7f5a5c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:58.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.578+0000 7f83bd8cf700 1 -- 172.21.15.16:0/2254893631 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f83b8138430 msgr2=0x7f83b806eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:58.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.578+0000 7f83bd8cf700 1 --2- 172.21.15.16:0/2254893631 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f83b8138430 0x7f83b806eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:58.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.578+0000 7f83bd8cf700 1 -- 172.21.15.16:0/2254893631 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83b81378f0 msgr2=0x7f83b81cd1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:58.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.578+0000 7f83bd8cf700 1 --2- 172.21.15.16:0/2254893631 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83b81378f0 0x7f83b81cd1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:58.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.578+0000 7f83bd8cf700 1 -- 172.21.15.16:0/2254893631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b4009550 con 0x7f83b8095690 2023-09-30T22:33:58.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.578+0000 7f5a4bfff700 1 -- 172.21.15.16:0/3366700714 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c009ea0 con 0x7f5a64123c30 2023-09-30T22:33:58.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.578+0000 7f5a4bfff700 1 -- 172.21.15.16:0/3366700714 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a5c00f040 con 0x7f5a64123c30 2023-09-30T22:33:58.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.578+0000 7f5a4bfff700 1 -- 172.21.15.16:0/3366700714 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a5c00ced0 con 0x7f5a64123c30 2023-09-30T22:33:58.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.578+0000 7f5a69ffe700 1 -- 172.21.15.16:0/3366700714 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a6409a0a0 con 0x7f5a64123c30 2023-09-30T22:33:58.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.578+0000 7f5a69ffe700 1 -- 172.21.15.16:0/3366700714 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a641253d0 con 0x7f5a64123c30 2023-09-30T22:33:58.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.578+0000 7f83bd8cf700 1 --2- 172.21.15.16:0/2254893631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83b8095690 0x7f83b81ccc60 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f83b400e9b0 tx=0x7f83b400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:58.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.579+0000 7f83ae7fc700 1 -- 172.21.15.16:0/2254893631 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b400ee50 con 0x7f83b8095690 2023-09-30T22:33:58.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.579+0000 7f83ae7fc700 1 -- 172.21.15.16:0/2254893631 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83b400c7f0 con 0x7f83b8095690 2023-09-30T22:33:58.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.579+0000 7f83bfb33700 1 -- 172.21.15.16:0/2254893631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83b0009a70 con 0x7f83b8095690 2023-09-30T22:33:58.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.579+0000 7f83ae7fc700 1 -- 172.21.15.16:0/2254893631 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b400cb60 con 0x7f83b8095690 2023-09-30T22:33:58.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.580+0000 7f83bfb33700 1 -- 172.21.15.16:0/2254893631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83b8071f00 con 0x7f83b8095690 2023-09-30T22:33:58.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.580+0000 7f5a4bfff700 1 -- 172.21.15.16:0/3366700714 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5a5c014480 con 0x7f5a64123c30 2023-09-30T22:33:58.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.580+0000 7f5a69ffe700 1 -- 172.21.15.16:0/3366700714 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a64074ea0 con 0x7f5a64123c30 2023-09-30T22:33:58.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.580+0000 7f5a4bfff700 1 -- 172.21.15.16:0/3366700714 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f5a5c040170 con 0x7f5a64123c30 2023-09-30T22:33:58.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.581+0000 7f839bfff700 1 -- 172.21.15.16:0/2254893631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83b8074ea0 con 0x7f83b8095690 2023-09-30T22:33:58.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.586+0000 7f83ae7fc700 1 -- 172.21.15.16:0/2254893631 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f83b400ccc0 con 0x7f83b8095690 2023-09-30T22:33:58.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.586+0000 7f83ae7fc700 1 -- 172.21.15.16:0/2254893631 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f83b403fcc0 con 0x7f83b8095690 2023-09-30T22:33:58.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.587+0000 7f5a4bfff700 1 -- 172.21.15.16:0/3366700714 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5a5c0246c0 con 0x7f5a64123c30 2023-09-30T22:33:58.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.593+0000 7f83ae7fc700 1 -- 172.21.15.16:0/2254893631 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f83b40036a0 con 0x7f83b8095690 2023-09-30T22:33:58.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.758+0000 7f55b8aab700 1 -- 172.21.15.16:0/340694109 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5598005590 con 0x7f55b409d6f0 2023-09-30T22:33:58.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.758+0000 7f55a2ffd700 1 -- 172.21.15.16:0/340694109 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f55a401b340 con 0x7f55b409d6f0 2023-09-30T22:33:58.759 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:33:58.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.762+0000 7f55b8aab700 1 -- 172.21.15.16:0/340694109 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55b409d6f0 msgr2=0x7f55b411e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:58.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.762+0000 7f55b8aab700 1 --2- 172.21.15.16:0/340694109 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55b409d6f0 0x7f55b411e8d0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f55a400c4b0 tx=0x7f55a400c7c0 comp rx=0 tx=0).stop 2023-09-30T22:33:58.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.762+0000 7f55b8aab700 1 -- 172.21.15.16:0/340694109 shutdown_connections 2023-09-30T22:33:58.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.762+0000 7f55b8aab700 1 --2- 172.21.15.16:0/340694109 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55b409d6f0 0x7f55b411e8d0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:58.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.762+0000 7f55b8aab700 1 --2- 172.21.15.16:0/340694109 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f55b4139170 0x7f55b411f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:58.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.762+0000 7f55b8aab700 1 --2- 172.21.15.16:0/340694109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55b409e1d0 0x7f55b411ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:58.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.762+0000 7f55b8aab700 1 -- 172.21.15.16:0/340694109 >> 172.21.15.16:0/340694109 conn(0x7f55b4066f50 msgr2=0x7f55b4136c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:58.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.762+0000 7f55b8aab700 1 -- 172.21.15.16:0/340694109 shutdown_connections 2023-09-30T22:33:58.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.762+0000 7f55b8aab700 1 -- 172.21.15.16:0/340694109 wait complete. 2023-09-30T22:33:58.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.763+0000 7f0da9844700 1 -- 172.21.15.16:0/1659016086 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f0da409c690 con 0x7f0da409d6f0 2023-09-30T22:33:58.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.764+0000 7f0da0ff9700 1 -- 172.21.15.16:0/1659016086 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0d9400ad60 con 0x7f0da409d6f0 2023-09-30T22:33:58.765 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:33:58.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.766+0000 7f0da9844700 1 -- 172.21.15.16:0/1659016086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0da409d6f0 msgr2=0x7f0da411e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:58.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.766+0000 7f0da9844700 1 --2- 172.21.15.16:0/1659016086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0da409d6f0 0x7f0da411e730 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f0d9400c8c0 tx=0x7f0d9400cbd0 comp rx=0 tx=0).stop 2023-09-30T22:33:58.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.768+0000 7f0da9844700 1 -- 172.21.15.16:0/1659016086 shutdown_connections 2023-09-30T22:33:58.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.768+0000 7f0da9844700 1 --2- 172.21.15.16:0/1659016086 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0da4139170 0x7f0da411f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:58.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.768+0000 7f0da9844700 1 --2- 172.21.15.16:0/1659016086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0da409d6f0 0x7f0da411e730 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:58.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.768+0000 7f0da9844700 1 --2- 172.21.15.16:0/1659016086 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0da409e1d0 0x7f0da411ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:58.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.768+0000 7f0da9844700 1 -- 172.21.15.16:0/1659016086 >> 172.21.15.16:0/1659016086 conn(0x7f0da4066f50 msgr2=0x7f0da4138090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:58.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.768+0000 7f0da9844700 1 -- 172.21.15.16:0/1659016086 shutdown_connections 2023-09-30T22:33:58.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.768+0000 7f0da9844700 1 -- 172.21.15.16:0/1659016086 wait complete. 2023-09-30T22:33:58.783 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:33:58.784 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:33:58.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.798+0000 7f5a69ffe700 1 -- 172.21.15.16:0/3366700714 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f5a6406b9e0 con 0x7f5a64123c30 2023-09-30T22:33:58.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.798+0000 7f5a4bfff700 1 -- 172.21.15.16:0/3366700714 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5a5c024300 con 0x7f5a64123c30 2023-09-30T22:33:58.799 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:33:58.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.801+0000 7f5a69ffe700 1 -- 172.21.15.16:0/3366700714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a64123c30 msgr2=0x7f5a641d2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:58.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.801+0000 7f5a69ffe700 1 --2- 172.21.15.16:0/3366700714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a64123c30 0x7f5a641d2a70 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f5a5c00efc0 tx=0x7f5a5c00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:33:58.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.801+0000 7f5a69ffe700 1 -- 172.21.15.16:0/3366700714 shutdown_connections 2023-09-30T22:33:58.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.801+0000 7f5a69ffe700 1 --2- 172.21.15.16:0/3366700714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a64123c30 0x7f5a641d2a70 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:58.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.801+0000 7f5a69ffe700 1 --2- 172.21.15.16:0/3366700714 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a6406e570 0x7f5a641231b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:58.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.801+0000 7f5a69ffe700 1 --2- 172.21.15.16:0/3366700714 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5a6406f050 0x7f5a641236f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:58.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.801+0000 7f5a69ffe700 1 -- 172.21.15.16:0/3366700714 >> 172.21.15.16:0/3366700714 conn(0x7f5a640a2aa0 msgr2=0x7f5a640a1410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:58.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.801+0000 7f5a69ffe700 1 -- 172.21.15.16:0/3366700714 shutdown_connections 2023-09-30T22:33:58.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.801+0000 7f5a69ffe700 1 -- 172.21.15.16:0/3366700714 wait complete. 2023-09-30T22:33:58.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.813+0000 7f839bfff700 1 -- 172.21.15.16:0/2254893631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f83b8095e70 con 0x7f83b8095690 2023-09-30T22:33:58.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.813+0000 7f83ae7fc700 1 -- 172.21.15.16:0/2254893631 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f83b4017030 con 0x7f83b8095690 2023-09-30T22:33:58.814 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:33:58.815 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:33:58.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.815+0000 7f839bfff700 1 -- 172.21.15.16:0/2254893631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83b8095690 msgr2=0x7f83b81ccc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:58.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.815+0000 7f839bfff700 1 --2- 172.21.15.16:0/2254893631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83b8095690 0x7f83b81ccc60 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f83b400e9b0 tx=0x7f83b400ecc0 comp rx=0 tx=0).stop 2023-09-30T22:33:58.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.815+0000 7f839bfff700 1 -- 172.21.15.16:0/2254893631 shutdown_connections 2023-09-30T22:33:58.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.815+0000 7f839bfff700 1 --2- 172.21.15.16:0/2254893631 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f83b8138430 0x7f83b806eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:58.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.815+0000 7f839bfff700 1 --2- 172.21.15.16:0/2254893631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83b8095690 0x7f83b81ccc60 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:58.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.815+0000 7f839bfff700 1 --2- 172.21.15.16:0/2254893631 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83b81378f0 0x7f83b81cd1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:58.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.815+0000 7f839bfff700 1 -- 172.21.15.16:0/2254893631 >> 172.21.15.16:0/2254893631 conn(0x7f83b80a2360 msgr2=0x7f83b809b690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:58.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.816+0000 7f839bfff700 1 -- 172.21.15.16:0/2254893631 shutdown_connections 2023-09-30T22:33:58.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:58.816+0000 7f839bfff700 1 -- 172.21.15.16:0/2254893631 wait complete. 2023-09-30T22:33:58.829 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:33:59.784 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:33:59.785 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:33:59.816 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:33:59.830 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:33:59.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.937+0000 7f9fd37a4700 1 Processor -- start 2023-09-30T22:33:59.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.938+0000 7f9fd37a4700 1 -- start start 2023-09-30T22:33:59.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.938+0000 7f9fd37a4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9fcc0a0a30 0x7f9fcc0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:59.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.938+0000 7f9fd37a4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9fcc138d80 0x7f9fcc13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:59.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.938+0000 7f9fd37a4700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fcc0a2a70 con 0x7f9fcc09d630 2023-09-30T22:33:59.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.938+0000 7f9fd37a4700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fcc0a01d0 con 0x7f9fcc0a0a30 2023-09-30T22:33:59.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.939+0000 7f9fd37a4700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9fcc09e2b0 con 0x7f9fcc138d80 2023-09-30T22:33:59.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.939+0000 7f9fd0d3f700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9fcc09d630 0x7f9fcc0a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47430/0 (socket says 172.21.15.16:47430) 2023-09-30T22:33:59.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.939+0000 7f9fd0d3f700 1 -- 172.21.15.16:0/1390684995 learned_addr learned my addr 172.21.15.16:0/1390684995 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:33:59.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.939+0000 7f9fd1d41700 1 --2- 172.21.15.16:0/1390684995 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9fcc138d80 0x7f9fcc13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:59.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.939+0000 7f9fd1540700 1 --2- 172.21.15.16:0/1390684995 >> v2:172.21.15.16:3300/0 conn(0x7f9fcc0a0a30 0x7f9fcc0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:59.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.939+0000 7f9fc3fff700 1 -- 172.21.15.16:0/1390684995 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2274384759 0 0) 0x7f9fcc0a2a70 con 0x7f9fcc09d630 2023-09-30T22:33:59.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.939+0000 7f9fc3fff700 1 -- 172.21.15.16:0/1390684995 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fb80035b0 con 0x7f9fcc09d630 2023-09-30T22:33:59.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.939+0000 7f9fd1540700 1 -- 172.21.15.16:0/1390684995 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9fcc138d80 msgr2=0x7f9fcc13b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:59.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.939+0000 7f9fd1540700 1 --2- 172.21.15.16:0/1390684995 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9fcc138d80 0x7f9fcc13b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:59.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.939+0000 7f9fd1540700 1 -- 172.21.15.16:0/1390684995 >> v1:172.21.15.16:6789/0 conn(0x7f9fcc09d630 legacy=0x7f9fcc0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:59.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.939+0000 7f9fd1540700 1 -- 172.21.15.16:0/1390684995 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fcc09dc50 con 0x7f9fcc0a0a30 2023-09-30T22:33:59.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.939+0000 7f9fd1d41700 1 --2- 172.21.15.16:0/1390684995 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9fcc138d80 0x7f9fcc13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:59.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.939+0000 7f9fd1540700 1 --2- 172.21.15.16:0/1390684995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fcc0a0a30 0x7f9fcc0a0e00 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f9fc8009840 tx=0x7f9fc8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aed7fa5d0fb4272 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:59.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.940+0000 7f9fc3fff700 1 -- 172.21.15.16:0/1390684995 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc8011830 con 0x7f9fcc0a0a30 2023-09-30T22:33:59.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.940+0000 7f9fc3fff700 1 -- 172.21.15.16:0/1390684995 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fc8011990 con 0x7f9fcc0a0a30 2023-09-30T22:33:59.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.940+0000 7f9fc3fff700 1 -- 172.21.15.16:0/1390684995 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc8011d20 con 0x7f9fcc0a0a30 2023-09-30T22:33:59.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.940+0000 7f9fd37a4700 1 -- 172.21.15.16:0/1390684995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fcc0a0a30 msgr2=0x7f9fcc0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:59.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.940+0000 7f9fd37a4700 1 --2- 172.21.15.16:0/1390684995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fcc0a0a30 0x7f9fcc0a0e00 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f9fc8009840 tx=0x7f9fc8009b50 comp rx=0 tx=0).stop 2023-09-30T22:33:59.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.940+0000 7f9fd37a4700 1 -- 172.21.15.16:0/1390684995 shutdown_connections 2023-09-30T22:33:59.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.940+0000 7f9fd37a4700 1 --2- 172.21.15.16:0/1390684995 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9fcc138d80 0x7f9fcc13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:59.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.940+0000 7f9fd37a4700 1 --2- 172.21.15.16:0/1390684995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fcc0a0a30 0x7f9fcc0a0e00 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:59.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.940+0000 7f9fd37a4700 1 -- 172.21.15.16:0/1390684995 >> 172.21.15.16:0/1390684995 conn(0x7f9fcc0a3050 msgr2=0x7f9fcc0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:59.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.941+0000 7f9fd37a4700 1 -- 172.21.15.16:0/1390684995 shutdown_connections 2023-09-30T22:33:59.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.941+0000 7f9fd37a4700 1 -- 172.21.15.16:0/1390684995 wait complete. 2023-09-30T22:33:59.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.942+0000 7f9fd37a4700 1 Processor -- start 2023-09-30T22:33:59.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.942+0000 7f9fd37a4700 1 -- start start 2023-09-30T22:33:59.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.942+0000 7f9fd37a4700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9fcc09d630 0x7f9fcc133690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:59.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.942+0000 7f9fd37a4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9fcc0a0a30 0x7f9fcc133bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:59.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.942+0000 7f9fd37a4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fcc138d80 0x7f9fcc134110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:59.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.942+0000 7f9fd37a4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fcc134b60 con 0x7f9fcc138d80 2023-09-30T22:33:59.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.942+0000 7f9fd37a4700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fcc1361b0 con 0x7f9fcc09d630 2023-09-30T22:33:59.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.942+0000 7f9fd37a4700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9fcc136ed0 con 0x7f9fcc0a0a30 2023-09-30T22:33:59.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.942+0000 7f9fd1d41700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fcc138d80 0x7f9fcc134110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:59.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.942+0000 7f9fd1d41700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fcc138d80 0x7f9fcc134110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37148/0 (socket says 172.21.15.16:37148) 2023-09-30T22:33:59.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.942+0000 7f9fd1d41700 1 -- 172.21.15.16:0/349438439 learned_addr learned my addr 172.21.15.16:0/349438439 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:59.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.943+0000 7f9fd1540700 1 --2- 172.21.15.16:0/349438439 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9fcc09d630 0x7f9fcc133690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:59.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.943+0000 7f9fd1d41700 1 -- 172.21.15.16:0/349438439 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9fcc0a0a30 msgr2=0x7f9fcc133bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:59.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.943+0000 7f9fd1d41700 1 --2- 172.21.15.16:0/349438439 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9fcc0a0a30 0x7f9fcc133bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:59.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.943+0000 7f9fd1d41700 1 -- 172.21.15.16:0/349438439 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9fcc09d630 msgr2=0x7f9fcc133690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:59.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.943+0000 7f9fd1d41700 1 --2- 172.21.15.16:0/349438439 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9fcc09d630 0x7f9fcc133690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:59.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.943+0000 7f9fd1d41700 1 -- 172.21.15.16:0/349438439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc4009a70 con 0x7f9fcc138d80 2023-09-30T22:33:59.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.943+0000 7f9fd1540700 1 --2- 172.21.15.16:0/349438439 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9fcc09d630 0x7f9fcc133690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:33:59.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.943+0000 7f9fd1d41700 1 --2- 172.21.15.16:0/349438439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fcc138d80 0x7f9fcc134110 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f9fc400ef90 tx=0x7f9fc400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:59.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.943+0000 7f9fc1ffb700 1 -- 172.21.15.16:0/349438439 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc4009d70 con 0x7f9fcc138d80 2023-09-30T22:33:59.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.943+0000 7f9fc1ffb700 1 -- 172.21.15.16:0/349438439 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fc400f040 con 0x7f9fcc138d80 2023-09-30T22:33:59.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.943+0000 7f9fd37a4700 1 -- 172.21.15.16:0/349438439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fc80094a0 con 0x7f9fcc138d80 2023-09-30T22:33:59.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.943+0000 7f9fc1ffb700 1 -- 172.21.15.16:0/349438439 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc4014480 con 0x7f9fcc138d80 2023-09-30T22:33:59.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.943+0000 7f9fd37a4700 1 -- 172.21.15.16:0/349438439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fcc0a0410 con 0x7f9fcc138d80 2023-09-30T22:33:59.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.944+0000 7f9fc1ffb700 1 -- 172.21.15.16:0/349438439 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9fc40145e0 con 0x7f9fcc138d80 2023-09-30T22:33:59.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.944+0000 7f9fd37a4700 1 -- 172.21.15.16:0/349438439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fac0052e0 con 0x7f9fcc138d80 2023-09-30T22:33:59.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.945+0000 7f9fc1ffb700 1 -- 172.21.15.16:0/349438439 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f9fc4040440 con 0x7f9fcc138d80 2023-09-30T22:33:59.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.950+0000 7f5dbb105700 1 Processor -- start 2023-09-30T22:33:59.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.950+0000 7f5dbb105700 1 -- start start 2023-09-30T22:33:59.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.951+0000 7f5dbb105700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5db4071b40 0x7f5db4130420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:59.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.951+0000 7f5dbb105700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5db4130960 0x7f5db4133570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:59.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.951+0000 7f5dbb105700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5db4130d50 0x7f5db4133ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:59.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.951+0000 7f5dbb105700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5db4099ef0 con 0x7f5db4071b40 2023-09-30T22:33:59.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.951+0000 7f5dbb105700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5db4099500 con 0x7f5db4130d50 2023-09-30T22:33:59.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.951+0000 7f5dbb105700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5db4099c10 con 0x7f5db4130960 2023-09-30T22:33:59.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.951+0000 7f5db8ea1700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5db4071b40 0x7f5db4130420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:59.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.951+0000 7f5db8ea1700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5db4071b40 0x7f5db4130420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37164/0 (socket says 172.21.15.16:37164) 2023-09-30T22:33:59.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.951+0000 7f5db8ea1700 1 -- 172.21.15.16:0/2457424467 learned_addr learned my addr 172.21.15.16:0/2457424467 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:59.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.951+0000 7f5dabfff700 1 --2- 172.21.15.16:0/2457424467 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5db4130960 0x7f5db4133570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:59.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.951+0000 7f5db8ea1700 1 -- 172.21.15.16:0/2457424467 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5db4130960 msgr2=0x7f5db4133570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:59.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.951+0000 7f5db8ea1700 1 --2- 172.21.15.16:0/2457424467 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5db4130960 0x7f5db4133570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:59.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.951+0000 7f5db8ea1700 1 -- 172.21.15.16:0/2457424467 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5db4130d50 msgr2=0x7f5db4133ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:59.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.951+0000 7f5db8ea1700 1 --2- 172.21.15.16:0/2457424467 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5db4130d50 0x7f5db4133ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:59.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.951+0000 7f5db8ea1700 1 -- 172.21.15.16:0/2457424467 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db409e370 con 0x7f5db4071b40 2023-09-30T22:33:59.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.951+0000 7f5db8ea1700 1 --2- 172.21.15.16:0/2457424467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5db4071b40 0x7f5db4130420 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f5db00098f0 tx=0x7f5db0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0a515fe69fc0764 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:59.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.953+0000 7f5dab7fe700 1 -- 172.21.15.16:0/2457424467 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db0011830 con 0x7f5db4071b40 2023-09-30T22:33:59.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.954+0000 7f5dab7fe700 1 -- 172.21.15.16:0/2457424467 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5db0011990 con 0x7f5db4071b40 2023-09-30T22:33:59.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.954+0000 7f5dab7fe700 1 -- 172.21.15.16:0/2457424467 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db0011d20 con 0x7f5db4071b40 2023-09-30T22:33:59.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.954+0000 7f9fc1ffb700 1 -- 172.21.15.16:0/349438439 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9fc4007920 con 0x7f9fcc138d80 2023-09-30T22:33:59.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.954+0000 7f5dbb105700 1 -- 172.21.15.16:0/2457424467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5db4071b40 msgr2=0x7f5db4130420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:59.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.954+0000 7f5dbb105700 1 --2- 172.21.15.16:0/2457424467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5db4071b40 0x7f5db4130420 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f5db00098f0 tx=0x7f5db0009c00 comp rx=0 tx=0).stop 2023-09-30T22:33:59.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.954+0000 7f5dbb105700 1 -- 172.21.15.16:0/2457424467 shutdown_connections 2023-09-30T22:33:59.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.954+0000 7f5dbb105700 1 --2- 172.21.15.16:0/2457424467 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5db4130d50 0x7f5db4133ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:59.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.954+0000 7f5dbb105700 1 --2- 172.21.15.16:0/2457424467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5db4071b40 0x7f5db4130420 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:59.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.954+0000 7f5dbb105700 1 --2- 172.21.15.16:0/2457424467 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5db4130960 0x7f5db4133570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:59.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.954+0000 7f5dbb105700 1 -- 172.21.15.16:0/2457424467 >> 172.21.15.16:0/2457424467 conn(0x7f5db409e7e0 msgr2=0x7f5db409ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:59.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.955+0000 7f5dbb105700 1 -- 172.21.15.16:0/2457424467 shutdown_connections 2023-09-30T22:33:59.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.955+0000 7f5dbb105700 1 -- 172.21.15.16:0/2457424467 wait complete. 2023-09-30T22:33:59.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.956+0000 7f5dbb105700 1 Processor -- start 2023-09-30T22:33:59.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.956+0000 7f5dbb105700 1 -- start start 2023-09-30T22:33:59.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.957+0000 7f5dbb105700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5db4071b40 0x7f5db412a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:59.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.957+0000 7f5dbb105700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5db4130960 0x7f5db412ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:59.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.957+0000 7f5dbb105700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5db4130d50 0x7f5db412b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:59.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.957+0000 7f5dbb105700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db412e0e0 con 0x7f5db4130d50 2023-09-30T22:33:59.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.957+0000 7f5dbb105700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5db41328f0 con 0x7f5db4130960 2023-09-30T22:33:59.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.957+0000 7f5dbb105700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5db409c150 con 0x7f5db4071b40 2023-09-30T22:33:59.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.957+0000 7f5db96a2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5db4130d50 0x7f5db412b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:59.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.957+0000 7f5db96a2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5db4130d50 0x7f5db412b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37178/0 (socket says 172.21.15.16:37178) 2023-09-30T22:33:59.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.957+0000 7f5db96a2700 1 -- 172.21.15.16:0/1291581670 learned_addr learned my addr 172.21.15.16:0/1291581670 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:59.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.957+0000 7f5dabfff700 1 --2- 172.21.15.16:0/1291581670 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5db4130960 0x7f5db412ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:59.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.957+0000 7f5db8ea1700 1 --2- 172.21.15.16:0/1291581670 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5db4071b40 0x7f5db412a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:59.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.958+0000 7f5dabfff700 1 -- 172.21.15.16:0/1291581670 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5db4071b40 msgr2=0x7f5db412a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:59.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.958+0000 7f5dabfff700 1 --2- 172.21.15.16:0/1291581670 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5db4071b40 0x7f5db412a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:59.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.958+0000 7f5dabfff700 1 -- 172.21.15.16:0/1291581670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5db4130d50 msgr2=0x7f5db412b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:59.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.958+0000 7f5dabfff700 1 --2- 172.21.15.16:0/1291581670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5db4130d50 0x7f5db412b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:59.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.958+0000 7f5dabfff700 1 -- 172.21.15.16:0/1291581670 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db0009550 con 0x7f5db4130960 2023-09-30T22:33:59.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.958+0000 7f5dabfff700 1 --2- 172.21.15.16:0/1291581670 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5db4130960 0x7f5db412ae20 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f5da400e900 tx=0x7f5da400ec10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:59.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.958+0000 7f5da97fa700 1 -- 172.21.15.16:0/1291581670 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da4013680 con 0x7f5db4130960 2023-09-30T22:33:59.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.958+0000 7f5da97fa700 1 -- 172.21.15.16:0/1291581670 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5da40137e0 con 0x7f5db4130960 2023-09-30T22:33:59.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.958+0000 7f5dbb105700 1 -- 172.21.15.16:0/1291581670 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5db412de20 con 0x7f5db4130960 2023-09-30T22:33:59.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.958+0000 7f5da97fa700 1 -- 172.21.15.16:0/1291581670 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da4013b50 con 0x7f5db4130960 2023-09-30T22:33:59.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.958+0000 7f5dbb105700 1 -- 172.21.15.16:0/1291581670 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5db412db50 con 0x7f5db4130960 2023-09-30T22:33:59.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.960+0000 7f5dbb105700 1 -- 172.21.15.16:0/1291581670 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5db412efa0 con 0x7f5db4130960 2023-09-30T22:33:59.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.960+0000 7f5da97fa700 1 -- 172.21.15.16:0/1291581670 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5da4013cb0 con 0x7f5db4130960 2023-09-30T22:33:59.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.960+0000 7f5da97fa700 1 -- 172.21.15.16:0/1291581670 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f5da4040720 con 0x7f5db4130960 2023-09-30T22:33:59.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.970+0000 7f5da97fa700 1 -- 172.21.15.16:0/1291581670 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5da401be10 con 0x7f5db4130960 2023-09-30T22:33:59.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.979+0000 7effc987e700 1 Processor -- start 2023-09-30T22:33:59.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.979+0000 7effc987e700 1 -- start start 2023-09-30T22:33:59.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.980+0000 7effc987e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7effc41338a0 0x7effc4133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:59.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.980+0000 7effc987e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7effc409de50 0x7effc409e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:59.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.980+0000 7effc987e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7effc409e760 0x7effc413a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:59.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.980+0000 7effc987e700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effc406fef0 con 0x7effc409e760 2023-09-30T22:33:59.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.980+0000 7effc987e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effc406ef30 con 0x7effc409de50 2023-09-30T22:33:59.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.980+0000 7effc987e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7effc41334b0 con 0x7effc41338a0 2023-09-30T22:33:59.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.980+0000 7effc27fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7effc409de50 0x7effc409e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:59.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.980+0000 7effc27fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7effc409de50 0x7effc409e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38514/0 (socket says 172.21.15.16:38514) 2023-09-30T22:33:59.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.980+0000 7effc27fc700 1 -- 172.21.15.16:0/3174019005 learned_addr learned my addr 172.21.15.16:0/3174019005 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:59.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.980+0000 7effc2ffd700 1 --2- 172.21.15.16:0/3174019005 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7effc41338a0 0x7effc4133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:59.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.980+0000 7effc37fe700 1 --2- 172.21.15.16:0/3174019005 >> v2:172.21.15.16:3300/0 conn(0x7effc409e760 0x7effc413a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:59.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.980+0000 7effc27fc700 1 -- 172.21.15.16:0/3174019005 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7effc41338a0 msgr2=0x7effc4133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:59.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.980+0000 7effc27fc700 1 --2- 172.21.15.16:0/3174019005 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7effc41338a0 0x7effc4133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:59.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.980+0000 7effc27fc700 1 -- 172.21.15.16:0/3174019005 >> v2:172.21.15.16:3300/0 conn(0x7effc409e760 msgr2=0x7effc413a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:59.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.980+0000 7effc27fc700 1 --2- 172.21.15.16:0/3174019005 >> v2:172.21.15.16:3300/0 conn(0x7effc409e760 0x7effc413a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:59.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.980+0000 7effc27fc700 1 -- 172.21.15.16:0/3174019005 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effc406fc30 con 0x7effc409de50 2023-09-30T22:33:59.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.980+0000 7effc2ffd700 1 --2- 172.21.15.16:0/3174019005 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7effc41338a0 0x7effc4133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:33:59.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.980+0000 7effc37fe700 1 --2- 172.21.15.16:0/3174019005 >> v2:172.21.15.16:3300/0 conn(0x7effc409e760 0x7effc413a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:33:59.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.981+0000 7effc27fc700 1 --2- 172.21.15.16:0/3174019005 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7effc409de50 0x7effc409e220 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7effb8009920 tx=0x7effb8009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5d78c9a5d447be0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:59.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.981+0000 7effc1ffb700 1 -- 172.21.15.16:0/3174019005 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb8011830 con 0x7effc409de50 2023-09-30T22:33:59.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.981+0000 7effc1ffb700 1 -- 172.21.15.16:0/3174019005 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effb8011990 con 0x7effc409de50 2023-09-30T22:33:59.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.981+0000 7effc1ffb700 1 -- 172.21.15.16:0/3174019005 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effb8011d20 con 0x7effc409de50 2023-09-30T22:33:59.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.981+0000 7effc987e700 1 -- 172.21.15.16:0/3174019005 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7effc409de50 msgr2=0x7effc409e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:59.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.981+0000 7effc987e700 1 --2- 172.21.15.16:0/3174019005 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7effc409de50 0x7effc409e220 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7effb8009920 tx=0x7effb8009c30 comp rx=0 tx=0).stop 2023-09-30T22:33:59.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.981+0000 7effc987e700 1 -- 172.21.15.16:0/3174019005 shutdown_connections 2023-09-30T22:33:59.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.982+0000 7effc987e700 1 --2- 172.21.15.16:0/3174019005 >> v2:172.21.15.16:3300/0 conn(0x7effc409e760 0x7effc413a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:59.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.982+0000 7effc987e700 1 --2- 172.21.15.16:0/3174019005 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7effc41338a0 0x7effc4133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:59.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.982+0000 7effc987e700 1 --2- 172.21.15.16:0/3174019005 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7effc409de50 0x7effc409e220 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:59.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.982+0000 7effc987e700 1 -- 172.21.15.16:0/3174019005 >> 172.21.15.16:0/3174019005 conn(0x7effc40a2aa0 msgr2=0x7effc40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:59.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.982+0000 7effc987e700 1 -- 172.21.15.16:0/3174019005 shutdown_connections 2023-09-30T22:33:59.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.982+0000 7effc987e700 1 -- 172.21.15.16:0/3174019005 wait complete. 2023-09-30T22:33:59.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.983+0000 7effc987e700 1 Processor -- start 2023-09-30T22:33:59.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.983+0000 7effc987e700 1 -- start start 2023-09-30T22:33:59.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.983+0000 7effc987e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effc409de50 0x7effc411ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:59.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.983+0000 7effc987e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7effc409e760 0x7effc411efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:59.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.983+0000 7effc987e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7effc41338a0 0x7effc411f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:59.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.983+0000 7effc987e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc411fec0 con 0x7effc409de50 2023-09-30T22:33:59.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.983+0000 7effc987e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effc40a1110 con 0x7effc41338a0 2023-09-30T22:33:59.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.983+0000 7effc987e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7effc4120780 con 0x7effc409e760 2023-09-30T22:33:59.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.983+0000 7effc37fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7effc41338a0 0x7effc411f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:59.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.984+0000 7effc27fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7effc409e760 0x7effc411efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:59.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.984+0000 7effc37fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7effc41338a0 0x7effc411f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38528/0 (socket says 172.21.15.16:38528) 2023-09-30T22:33:59.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.984+0000 7effc37fe700 1 -- 172.21.15.16:0/1367832260 learned_addr learned my addr 172.21.15.16:0/1367832260 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:59.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.984+0000 7effc37fe700 1 -- 172.21.15.16:0/1367832260 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7effc409e760 msgr2=0x7effc411efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:59.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.984+0000 7effc37fe700 1 --2- 172.21.15.16:0/1367832260 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7effc409e760 0x7effc411efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:59.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.984+0000 7effc37fe700 1 -- 172.21.15.16:0/1367832260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effc409de50 msgr2=0x7effc411ea60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:33:59.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.984+0000 7effc37fe700 1 --2- 172.21.15.16:0/1367832260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effc409de50 0x7effc411ea60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:59.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.984+0000 7effc37fe700 1 -- 172.21.15.16:0/1367832260 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effbc009a70 con 0x7effc41338a0 2023-09-30T22:33:59.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.984+0000 7effc37fe700 1 --2- 172.21.15.16:0/1367832260 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7effc41338a0 0x7effc411f4e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7effbc00ef00 tx=0x7effbc00b390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:59.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.984+0000 7effabfff700 1 -- 172.21.15.16:0/1367832260 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effbc010040 con 0x7effc41338a0 2023-09-30T22:33:59.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.984+0000 7effabfff700 1 -- 172.21.15.16:0/1367832260 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effbc00bb30 con 0x7effc41338a0 2023-09-30T22:33:59.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.984+0000 7effc987e700 1 -- 172.21.15.16:0/1367832260 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effb8009550 con 0x7effc41338a0 2023-09-30T22:33:59.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.984+0000 7effabfff700 1 -- 172.21.15.16:0/1367832260 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effbc003a20 con 0x7effc41338a0 2023-09-30T22:33:59.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.984+0000 7effc987e700 1 -- 172.21.15.16:0/1367832260 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effc412f580 con 0x7effc41338a0 2023-09-30T22:33:59.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.986+0000 7f1f1b247700 1 Processor -- start 2023-09-30T22:33:59.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.986+0000 7f1f1b247700 1 -- start start 2023-09-30T22:33:59.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.986+0000 7f1f1b247700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f1412fc70 0x7f1f14130040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:59.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.986+0000 7f1f1b247700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1f1412c060 0x7f1f1412c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:59.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.986+0000 7f1f1b247700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f1412c970 0x7f1f14136940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:33:59.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.986+0000 7f1f1b247700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f14130580 con 0x7f1f1412c060 2023-09-30T22:33:59.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.986+0000 7f1f1b247700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1409be70 con 0x7f1f1412fc70 2023-09-30T22:33:59.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.986+0000 7f1f1b247700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1f1409a6e0 con 0x7f1f1412c970 2023-09-30T22:33:59.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.987+0000 7f1f18fe3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f1412fc70 0x7f1f14130040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:59.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.987+0000 7f1f18fe3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f1412fc70 0x7f1f14130040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38540/0 (socket says 172.21.15.16:38540) 2023-09-30T22:33:59.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.987+0000 7f1f18fe3700 1 -- 172.21.15.16:0/627983155 learned_addr learned my addr 172.21.15.16:0/627983155 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:33:59.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.986+0000 7effc987e700 1 -- 172.21.15.16:0/1367832260 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effc4074ea0 con 0x7effc41338a0 2023-09-30T22:33:59.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.987+0000 7effabfff700 1 -- 172.21.15.16:0/1367832260 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7effbc026430 con 0x7effc41338a0 2023-09-30T22:33:59.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.987+0000 7f1f197e4700 1 --2- 172.21.15.16:0/627983155 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f1412c970 0x7f1f14136940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:33:59.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.987+0000 7effabfff700 1 -- 172.21.15.16:0/1367832260 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7effbc0404b0 con 0x7effc41338a0 2023-09-30T22:33:59.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.987+0000 7f1f18fe3700 1 -- 172.21.15.16:0/627983155 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f1412c970 msgr2=0x7f1f14136940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:33:59.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.987+0000 7f1f18fe3700 1 --2- 172.21.15.16:0/627983155 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f1412c970 0x7f1f14136940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:59.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.987+0000 7f1f18fe3700 1 -- 172.21.15.16:0/627983155 >> v2:172.21.15.16:3300/0 conn(0x7f1f1412c060 msgr2=0x7f1f1412c430 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T22:33:59.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.987+0000 7f1f18fe3700 1 --2- 172.21.15.16:0/627983155 >> v2:172.21.15.16:3300/0 conn(0x7f1f1412c060 0x7f1f1412c430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:59.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.987+0000 7f1f18fe3700 1 -- 172.21.15.16:0/627983155 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f1409ac80 con 0x7f1f1412fc70 2023-09-30T22:33:59.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.987+0000 7f1f18fe3700 1 --2- 172.21.15.16:0/627983155 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f1412fc70 0x7f1f14130040 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f1f080098f0 tx=0x7f1f08009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18d21bd430fa240a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:33:59.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.991+0000 7f1f137fe700 1 -- 172.21.15.16:0/627983155 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f08011830 con 0x7f1f1412fc70 2023-09-30T22:33:59.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.991+0000 7f1f137fe700 1 -- 172.21.15.16:0/627983155 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f08011990 con 0x7f1f1412fc70 2023-09-30T22:33:59.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.991+0000 7f1f137fe700 1 -- 172.21.15.16:0/627983155 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f08011d20 con 0x7f1f1412fc70 2023-09-30T22:33:59.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.992+0000 7effabfff700 1 -- 172.21.15.16:0/1367832260 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7effbc026e40 con 0x7effc41338a0 2023-09-30T22:33:59.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.993+0000 7f1f1b247700 1 -- 172.21.15.16:0/627983155 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f1412fc70 msgr2=0x7f1f14130040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:33:59.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.993+0000 7f1f1b247700 1 --2- 172.21.15.16:0/627983155 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f1412fc70 0x7f1f14130040 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f1f080098f0 tx=0x7f1f08009c00 comp rx=0 tx=0).stop 2023-09-30T22:33:59.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.993+0000 7f1f1b247700 1 -- 172.21.15.16:0/627983155 shutdown_connections 2023-09-30T22:33:59.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.993+0000 7f1f1b247700 1 --2- 172.21.15.16:0/627983155 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f1412fc70 0x7f1f14130040 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:59.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.993+0000 7f1f1b247700 1 --2- 172.21.15.16:0/627983155 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f1412c970 0x7f1f14136940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:59.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.993+0000 7f1f1b247700 1 --2- 172.21.15.16:0/627983155 >> v2:172.21.15.16:3300/0 conn(0x7f1f1412c060 0x7f1f1412c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:33:59.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.993+0000 7f1f1b247700 1 -- 172.21.15.16:0/627983155 >> 172.21.15.16:0/627983155 conn(0x7f1f1409e7e0 msgr2=0x7f1f1409ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:33:59.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.993+0000 7f1f1b247700 1 -- 172.21.15.16:0/627983155 shutdown_connections 2023-09-30T22:33:59.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.993+0000 7f1f1b247700 1 -- 172.21.15.16:0/627983155 wait complete. 2023-09-30T22:33:59.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.994+0000 7f1f1b247700 1 Processor -- start 2023-09-30T22:33:59.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.994+0000 7f1f1b247700 1 -- start start 2023-09-30T22:34:00.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.995+0000 7f1f1b247700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f1412c060 0x7f1f14070990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:00.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.995+0000 7f1f1b247700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f1412c970 0x7f1f14070ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:00.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.995+0000 7f1f1b247700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f1412fc70 0x7f1f14071410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:00.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.995+0000 7f1f1b247700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f14131930 con 0x7f1f1412c970 2023-09-30T22:34:00.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.995+0000 7f1f1b247700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f14071e60 con 0x7f1f1412fc70 2023-09-30T22:34:00.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.995+0000 7f1f1b247700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1f14131030 con 0x7f1f1412c060 2023-09-30T22:34:00.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.996+0000 7f1f13fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f1412c970 0x7f1f14070ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:00.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.996+0000 7f1f18fe3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f1412c060 0x7f1f14070990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:00.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.996+0000 7f1f18fe3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f1412c060 0x7f1f14070990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:57320/0 (socket says 172.21.15.16:57320) 2023-09-30T22:34:00.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.996+0000 7f1f13fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f1412c970 0x7f1f14070ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37190/0 (socket says 172.21.15.16:37190) 2023-09-30T22:34:00.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.996+0000 7f1f18fe3700 1 -- 172.21.15.16:0/1359779261 learned_addr learned my addr 172.21.15.16:0/1359779261 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:00.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.996+0000 7f1f197e4700 1 --2- 172.21.15.16:0/1359779261 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f1412fc70 0x7f1f14071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:00.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.996+0000 7f1f18fe3700 1 -- 172.21.15.16:0/1359779261 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f1412fc70 msgr2=0x7f1f14071410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:00.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.996+0000 7f1f18fe3700 1 --2- 172.21.15.16:0/1359779261 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f1412fc70 0x7f1f14071410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:00.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.996+0000 7f1f18fe3700 1 -- 172.21.15.16:0/1359779261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f1412c970 msgr2=0x7f1f14070ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:00.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.996+0000 7f1f18fe3700 1 --2- 172.21.15.16:0/1359779261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f1412c970 0x7f1f14070ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:00.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.996+0000 7f1f18fe3700 1 -- 172.21.15.16:0/1359779261 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f08009550 con 0x7f1f1412c060 2023-09-30T22:34:00.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.996+0000 7f1f197e4700 1 --2- 172.21.15.16:0/1359779261 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f1412fc70 0x7f1f14071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:34:00.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.996+0000 7f1f13fff700 1 --2- 172.21.15.16:0/1359779261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f1412c970 0x7f1f14070ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:00.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.997+0000 7f1f18fe3700 1 --2- 172.21.15.16:0/1359779261 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f1412c060 0x7f1f14070990 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f1f08009f50 tx=0x7f1f08004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:00.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.997+0000 7f1f117fa700 1 -- 172.21.15.16:0/1359779261 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0801a830 con 0x7f1f1412c060 2023-09-30T22:34:00.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.997+0000 7f1f117fa700 1 -- 172.21.15.16:0/1359779261 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f0801a990 con 0x7f1f1412c060 2023-09-30T22:34:00.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.997+0000 7f1f1b247700 1 -- 172.21.15.16:0/1359779261 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f1409d0c0 con 0x7f1f1412c060 2023-09-30T22:34:00.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.997+0000 7f1f117fa700 1 -- 172.21.15.16:0/1359779261 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0801ad20 con 0x7f1f1412c060 2023-09-30T22:34:00.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.997+0000 7f1f1b247700 1 -- 172.21.15.16:0/1359779261 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f14138060 con 0x7f1f1412c060 2023-09-30T22:34:00.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.998+0000 7f1f1b247700 1 -- 172.21.15.16:0/1359779261 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f14130780 con 0x7f1f1412c060 2023-09-30T22:34:00.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:33:59.999+0000 7f1f117fa700 1 -- 172.21.15.16:0/1359779261 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1f0802f3f0 con 0x7f1f1412c060 2023-09-30T22:34:00.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:00.000+0000 7f1f117fa700 1 -- 172.21.15.16:0/1359779261 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f1f08040e60 con 0x7f1f1412c060 2023-09-30T22:34:00.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:00.008+0000 7f1f117fa700 1 -- 172.21.15.16:0/1359779261 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1f08024760 con 0x7f1f1412c060 2023-09-30T22:34:00.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:00.177+0000 7f9fd37a4700 1 -- 172.21.15.16:0/349438439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f9fac005150 con 0x7f9fcc138d80 2023-09-30T22:34:00.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:00.178+0000 7f9fc1ffb700 1 -- 172.21.15.16:0/349438439 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9fc4007330 con 0x7f9fcc138d80 2023-09-30T22:34:00.178 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:34:00.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:00.180+0000 7f9fd37a4700 1 -- 172.21.15.16:0/349438439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fcc138d80 msgr2=0x7f9fcc134110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:00.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:00.180+0000 7f9fd37a4700 1 --2- 172.21.15.16:0/349438439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fcc138d80 0x7f9fcc134110 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f9fc400ef90 tx=0x7f9fc400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:34:00.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:00.180+0000 7f9fd37a4700 1 -- 172.21.15.16:0/349438439 shutdown_connections 2023-09-30T22:34:00.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:00.180+0000 7f9fd37a4700 1 --2- 172.21.15.16:0/349438439 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9fcc09d630 0x7f9fcc133690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:00.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:00.180+0000 7f9fd37a4700 1 --2- 172.21.15.16:0/349438439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9fcc138d80 0x7f9fcc134110 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:00.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:00.180+0000 7f9fd37a4700 1 --2- 172.21.15.16:0/349438439 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9fcc0a0a30 0x7f9fcc133bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:00.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:00.180+0000 7f9fd37a4700 1 -- 172.21.15.16:0/349438439 >> 172.21.15.16:0/349438439 conn(0x7f9fcc0a3050 msgr2=0x7f9fcc13ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:00.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:00.180+0000 7f9fd37a4700 1 -- 172.21.15.16:0/349438439 shutdown_connections 2023-09-30T22:34:00.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:00.180+0000 7f9fd37a4700 1 -- 172.21.15.16:0/349438439 wait complete. 2023-09-30T22:34:00.195 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:34:00.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:00.197+0000 7f5dbb105700 1 -- 172.21.15.16:0/1291581670 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5db4133f70 con 0x7f5db4130960 2023-09-30T22:34:00.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:00.197+0000 7f5da97fa700 1 -- 172.21.15.16:0/1291581670 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5da401b3b0 con 0x7f5db4130960 2023-09-30T22:34:00.198 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:34:00.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:00.199+0000 7effc987e700 1 -- 172.21.15.16:0/1367832260 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7effc406b9e0 con 0x7effc41338a0 2023-09-30T22:34:00.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:00.199+0000 7effabfff700 1 -- 172.21.15.16:0/1367832260 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7effbc01b360 con 0x7effc41338a0 2023-09-30T22:34:00.200 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:34:00.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:00.200+0000 7f5dbb105700 1 -- 172.21.15.16:0/1291581670 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5db4130960 msgr2=0x7f5db412ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:00.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:00.200+0000 7f5dbb105700 1 --2- 172.21.15.16:0/1291581670 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5db4130960 0x7f5db412ae20 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f5da400e900 tx=0x7f5da400ec10 comp rx=0 tx=0).stop 2023-09-30T22:34:00.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:00.200+0000 7f5dbb105700 1 -- 172.21.15.16:0/1291581670 shutdown_connections 2023-09-30T22:34:00.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:00.200+0000 7f5dbb105700 1 --2- 172.21.15.16:0/1291581670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5db4130d50 0x7f5db412b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:00.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:00.200+0000 7f5dbb105700 1 --2- 172.21.15.16:0/1291581670 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5db4071b40 0x7f5db412a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:00.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:00.200+0000 7f5dbb105700 1 --2- 172.21.15.16:0/1291581670 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5db4130960 0x7f5db412ae20 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:00.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:00.200+0000 7f5dbb105700 1 -- 172.21.15.16:0/1291581670 >> 172.21.15.16:0/1291581670 conn(0x7f5db409e7e0 msgr2=0x7f5db4131f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:00.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:00.200+0000 7f5dbb105700 1 -- 172.21.15.16:0/1291581670 shutdown_connections 2023-09-30T22:34:00.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:00.200+0000 7f5dbb105700 1 -- 172.21.15.16:0/1291581670 wait complete. 2023-09-30T22:34:00.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:00.202+0000 7effc987e700 1 -- 172.21.15.16:0/1367832260 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7effc41338a0 msgr2=0x7effc411f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:00.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:00.202+0000 7effc987e700 1 --2- 172.21.15.16:0/1367832260 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7effc41338a0 0x7effc411f4e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7effbc00ef00 tx=0x7effbc00b390 comp rx=0 tx=0).stop 2023-09-30T22:34:00.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:00.202+0000 7effc987e700 1 -- 172.21.15.16:0/1367832260 shutdown_connections 2023-09-30T22:34:00.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:00.202+0000 7effc987e700 1 --2- 172.21.15.16:0/1367832260 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7effc41338a0 0x7effc411f4e0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:00.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:00.202+0000 7effc987e700 1 --2- 172.21.15.16:0/1367832260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7effc409de50 0x7effc411ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:00.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:00.202+0000 7effc987e700 1 --2- 172.21.15.16:0/1367832260 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7effc409e760 0x7effc411efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:00.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:00.202+0000 7effc987e700 1 -- 172.21.15.16:0/1367832260 >> 172.21.15.16:0/1367832260 conn(0x7effc40a2aa0 msgr2=0x7effc409ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:00.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:00.202+0000 7effc987e700 1 -- 172.21.15.16:0/1367832260 shutdown_connections 2023-09-30T22:34:00.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:00.202+0000 7effc987e700 1 -- 172.21.15.16:0/1367832260 wait complete. 2023-09-30T22:34:00.216 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:34:00.218 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:34:00.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:00.231+0000 7f1f1b247700 1 -- 172.21.15.16:0/1359779261 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1f14099b60 con 0x7f1f1412c060 2023-09-30T22:34:00.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:00.232+0000 7f1f117fa700 1 -- 172.21.15.16:0/1359779261 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1f08034090 con 0x7f1f1412c060 2023-09-30T22:34:00.232 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:34:00.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:00.234+0000 7f1f1b247700 1 -- 172.21.15.16:0/1359779261 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f1412c060 msgr2=0x7f1f14070990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:00.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:00.234+0000 7f1f1b247700 1 --2- 172.21.15.16:0/1359779261 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f1412c060 0x7f1f14070990 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f1f08009f50 tx=0x7f1f08004750 comp rx=0 tx=0).stop 2023-09-30T22:34:00.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:00.234+0000 7f1f1b247700 1 -- 172.21.15.16:0/1359779261 shutdown_connections 2023-09-30T22:34:00.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:00.234+0000 7f1f1b247700 1 --2- 172.21.15.16:0/1359779261 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f1412fc70 0x7f1f14071410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:00.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:00.234+0000 7f1f1b247700 1 --2- 172.21.15.16:0/1359779261 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f1412c060 0x7f1f14070990 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:00.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:00.234+0000 7f1f1b247700 1 --2- 172.21.15.16:0/1359779261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f1412c970 0x7f1f14070ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:00.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:00.234+0000 7f1f1b247700 1 -- 172.21.15.16:0/1359779261 >> 172.21.15.16:0/1359779261 conn(0x7f1f1409e7e0 msgr2=0x7f1f1412ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:00.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:00.234+0000 7f1f1b247700 1 -- 172.21.15.16:0/1359779261 shutdown_connections 2023-09-30T22:34:00.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:00.234+0000 7f1f1b247700 1 -- 172.21.15.16:0/1359779261 wait complete. 2023-09-30T22:34:00.248 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:34:01.196 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:34:01.217 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:34:01.219 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:34:01.249 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:34:01.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.355+0000 7f819abe6700 1 Processor -- start 2023-09-30T22:34:01.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.355+0000 7f819abe6700 1 -- start start 2023-09-30T22:34:01.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.355+0000 7f819abe6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f819409d6f0 0x7f819409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:01.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.355+0000 7f819abe6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f819409e000 0x7f81941339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:01.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.355+0000 7f819abe6700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f819409b120 con 0x7f819409e410 2023-09-30T22:34:01.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.355+0000 7f819abe6700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f819406b460 con 0x7f819409d6f0 2023-09-30T22:34:01.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.355+0000 7f819abe6700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f819409a7e0 con 0x7f819409e000 2023-09-30T22:34:01.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.356+0000 7f8193fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f819409e000 0x7f81941339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:01.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.356+0000 7f8198982700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f819409d6f0 0x7f819409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:01.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.356+0000 7f8193fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f819409e000 0x7f81941339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:57336/0 (socket says 172.21.15.16:57336) 2023-09-30T22:34:01.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.356+0000 7f8193fff700 1 -- 172.21.15.16:0/168620614 learned_addr learned my addr 172.21.15.16:0/168620614 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:01.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.356+0000 7f8193fff700 1 -- 172.21.15.16:0/168620614 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f819409d6f0 msgr2=0x7f819409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:01.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.356+0000 7f8193fff700 1 --2- 172.21.15.16:0/168620614 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f819409d6f0 0x7f819409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:01.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.356+0000 7f8193fff700 1 -- 172.21.15.16:0/168620614 >> v1:172.21.15.16:6789/0 conn(0x7f819409e410 legacy=0x7f8194133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:01.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.356+0000 7f8193fff700 1 -- 172.21.15.16:0/168620614 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f819409a500 con 0x7f819409e000 2023-09-30T22:34:01.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.356+0000 7f8193fff700 1 --2- 172.21.15.16:0/168620614 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f819409e000 0x7f81941339f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f8180009950 tx=0x7f8180009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9805bb3ae0b1e09 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:01.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.356+0000 7f81937fe700 1 -- 172.21.15.16:0/168620614 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8180011830 con 0x7f819409e000 2023-09-30T22:34:01.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.357+0000 7f81937fe700 1 -- 172.21.15.16:0/168620614 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8180011990 con 0x7f819409e000 2023-09-30T22:34:01.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.357+0000 7f81937fe700 1 -- 172.21.15.16:0/168620614 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8180011d40 con 0x7f819409e000 2023-09-30T22:34:01.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.358+0000 7f819abe6700 1 -- 172.21.15.16:0/168620614 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f819409e000 msgr2=0x7f81941339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:01.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.358+0000 7f819abe6700 1 --2- 172.21.15.16:0/168620614 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f819409e000 0x7f81941339f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f8180009950 tx=0x7f8180009c60 comp rx=0 tx=0).stop 2023-09-30T22:34:01.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.358+0000 7f819abe6700 1 -- 172.21.15.16:0/168620614 shutdown_connections 2023-09-30T22:34:01.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.358+0000 7f819abe6700 1 --2- 172.21.15.16:0/168620614 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f819409d6f0 0x7f819409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:01.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.358+0000 7f819abe6700 1 --2- 172.21.15.16:0/168620614 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f819409e000 0x7f81941339f0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:01.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.358+0000 7f819abe6700 1 -- 172.21.15.16:0/168620614 >> 172.21.15.16:0/168620614 conn(0x7f8194066f50 msgr2=0x7f819409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:01.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.358+0000 7f819abe6700 1 -- 172.21.15.16:0/168620614 shutdown_connections 2023-09-30T22:34:01.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.358+0000 7f819abe6700 1 -- 172.21.15.16:0/168620614 wait complete. 2023-09-30T22:34:01.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.359+0000 7f819abe6700 1 Processor -- start 2023-09-30T22:34:01.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.359+0000 7f819abe6700 1 -- start start 2023-09-30T22:34:01.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.360+0000 7f819abe6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f819409d6f0 0x7f81941a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:01.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.360+0000 7f819abe6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f819409e000 0x7f81941a65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:01.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.360+0000 7f819abe6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f819409e410 0x7f81941a6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:01.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.360+0000 7f819abe6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81941a8250 con 0x7f819409e410 2023-09-30T22:34:01.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.360+0000 7f819abe6700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81941b93e0 con 0x7f819409d6f0 2023-09-30T22:34:01.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.360+0000 7f819abe6700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f81941a7460 con 0x7f819409e000 2023-09-30T22:34:01.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.361+0000 7f8198982700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f819409d6f0 0x7f81941a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:01.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.361+0000 7f8198982700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f819409d6f0 0x7f81941a6070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38580/0 (socket says 172.21.15.16:38580) 2023-09-30T22:34:01.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.361+0000 7f8198982700 1 -- 172.21.15.16:0/1844653421 learned_addr learned my addr 172.21.15.16:0/1844653421 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:01.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.361+0000 7f8193fff700 1 --2- 172.21.15.16:0/1844653421 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f819409e000 0x7f81941a65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:01.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.361+0000 7f8199183700 1 --2- 172.21.15.16:0/1844653421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f819409e410 0x7f81941a6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:01.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.361+0000 7f8193fff700 1 -- 172.21.15.16:0/1844653421 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f819409d6f0 msgr2=0x7f81941a6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:01.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.361+0000 7f8193fff700 1 --2- 172.21.15.16:0/1844653421 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f819409d6f0 0x7f81941a6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:01.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.361+0000 7f8193fff700 1 -- 172.21.15.16:0/1844653421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f819409e410 msgr2=0x7f81941a6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:01.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.361+0000 7f8193fff700 1 --2- 172.21.15.16:0/1844653421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f819409e410 0x7f81941a6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:01.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.361+0000 7f8193fff700 1 -- 172.21.15.16:0/1844653421 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8180009550 con 0x7f819409e000 2023-09-30T22:34:01.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.361+0000 7f8199183700 1 --2- 172.21.15.16:0/1844653421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f819409e410 0x7f81941a6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:01.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.361+0000 7f8198982700 1 --2- 172.21.15.16:0/1844653421 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f819409d6f0 0x7f81941a6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:01.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.361+0000 7f8193fff700 1 --2- 172.21.15.16:0/1844653421 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f819409e000 0x7f81941a65b0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f819409e9b0 tx=0x7f8180004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:01.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.362+0000 7f81917fa700 1 -- 172.21.15.16:0/1844653421 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8180004240 con 0x7f819409e000 2023-09-30T22:34:01.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.362+0000 7f81917fa700 1 -- 172.21.15.16:0/1844653421 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f818001f070 con 0x7f819409e000 2023-09-30T22:34:01.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.362+0000 7f819abe6700 1 -- 172.21.15.16:0/1844653421 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f819409c710 con 0x7f819409e000 2023-09-30T22:34:01.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.362+0000 7f81917fa700 1 -- 172.21.15.16:0/1844653421 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81800045e0 con 0x7f819409e000 2023-09-30T22:34:01.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.362+0000 7f819abe6700 1 -- 172.21.15.16:0/1844653421 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81941a8e50 con 0x7f819409e000 2023-09-30T22:34:01.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.363+0000 7f81917fa700 1 -- 172.21.15.16:0/1844653421 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8180021030 con 0x7f819409e000 2023-09-30T22:34:01.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.363+0000 7f81917fa700 1 -- 172.21.15.16:0/1844653421 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f81800409b0 con 0x7f819409e000 2023-09-30T22:34:01.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.363+0000 7f819abe6700 1 -- 172.21.15.16:0/1844653421 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8194074ea0 con 0x7f819409e000 2023-09-30T22:34:01.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.368+0000 7f81917fa700 1 -- 172.21.15.16:0/1844653421 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8180015080 con 0x7f819409e000 2023-09-30T22:34:01.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.390+0000 7f8c2f9c1700 1 Processor -- start 2023-09-30T22:34:01.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.390+0000 7f2743e44700 1 Processor -- start 2023-09-30T22:34:01.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.390+0000 7f8c2f9c1700 1 -- start start 2023-09-30T22:34:01.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.390+0000 7f2743e44700 1 -- start start 2023-09-30T22:34:01.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.391+0000 7f8c2f9c1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8c2809e1d0 0x7f8c28138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:01.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.391+0000 7f8c2f9c1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8c28139170 0x7f8c2813b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:01.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.391+0000 7f8c2f9c1700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c2809b120 con 0x7f8c2809d6f0 2023-09-30T22:34:01.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.391+0000 7f8c2f9c1700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2806b460 con 0x7f8c28139170 2023-09-30T22:34:01.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.391+0000 7f8c2f9c1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2809a7e0 con 0x7f8c2809e1d0 2023-09-30T22:34:01.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.391+0000 7f8c2d75d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8c2809d6f0 0x7f8c2809dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47444/0 (socket says 172.21.15.16:47444) 2023-09-30T22:34:01.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.391+0000 7f8c2d75d700 1 -- 172.21.15.16:0/3580683694 learned_addr learned my addr 172.21.15.16:0/3580683694 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:34:01.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.391+0000 7f8c2df5e700 1 --2- 172.21.15.16:0/3580683694 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8c28139170 0x7f8c2813b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:01.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.391+0000 7f2743e44700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f273c0a09c0 0x7f273c0a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:01.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.391+0000 7f2743e44700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f273c09d5c0 0x7f273c0a12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:01.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.391+0000 7f2743e44700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f273c0a1810 con 0x7f273c133920 2023-09-30T22:34:01.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.391+0000 7f2743e44700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f273c09ea10 con 0x7f273c0a09c0 2023-09-30T22:34:01.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.391+0000 7f2743e44700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f273c09e210 con 0x7f273c09d5c0 2023-09-30T22:34:01.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.391+0000 7f2741be0700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f273c0a09c0 0x7f273c0a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:01.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.391+0000 7f2741be0700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f273c0a09c0 0x7f273c0a0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37210/0 (socket says 172.21.15.16:37210) 2023-09-30T22:34:01.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.391+0000 7f2741be0700 1 -- 172.21.15.16:0/1169081519 learned_addr learned my addr 172.21.15.16:0/1169081519 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:01.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.392+0000 7f8c2df5e700 1 -- 172.21.15.16:0/3580683694 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8c2809e1d0 msgr2=0x7f8c28138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:01.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.392+0000 7f8c2df5e700 1 --2- 172.21.15.16:0/3580683694 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8c2809e1d0 0x7f8c28138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:01.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.392+0000 7f8c2df5e700 1 -- 172.21.15.16:0/3580683694 >> v1:172.21.15.16:6789/0 conn(0x7f8c2809d6f0 legacy=0x7f8c2809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:01.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.392+0000 7f8c1ffff700 1 -- 172.21.15.16:0/3580683694 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3433752818 0 0) 0x7f8c2809b120 con 0x7f8c2809d6f0 2023-09-30T22:34:01.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.392+0000 7f8c2df5e700 1 -- 172.21.15.16:0/3580683694 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c2809a500 con 0x7f8c28139170 2023-09-30T22:34:01.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.392+0000 7f2741be0700 1 -- 172.21.15.16:0/1169081519 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f273c09d5c0 msgr2=0x7f273c0a12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:01.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.392+0000 7f2741be0700 1 --2- 172.21.15.16:0/1169081519 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f273c09d5c0 0x7f273c0a12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:01.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.392+0000 7f2741be0700 1 -- 172.21.15.16:0/1169081519 >> v1:172.21.15.16:6789/0 conn(0x7f273c133920 legacy=0x7f273c133d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:01.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.392+0000 7f2741be0700 1 -- 172.21.15.16:0/1169081519 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f273c0a26f0 con 0x7f273c0a09c0 2023-09-30T22:34:01.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.392+0000 7f2741be0700 1 --2- 172.21.15.16:0/1169081519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f273c0a09c0 0x7f273c0a0d90 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f27380098f0 tx=0x7f2738009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1f12e055a47e866 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:01.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.392+0000 7f8c2df5e700 1 --2- 172.21.15.16:0/3580683694 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8c28139170 0x7f8c2813b550 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f8c20009dc0 tx=0x7f8c2000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e247b65e9b10cd0f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:01.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.392+0000 7f2740bde700 1 -- 172.21.15.16:0/1169081519 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273800b8a0 con 0x7f273c0a09c0 2023-09-30T22:34:01.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.392+0000 7f2740bde700 1 -- 172.21.15.16:0/1169081519 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f273800ba00 con 0x7f273c0a09c0 2023-09-30T22:34:01.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.392+0000 7f2740bde700 1 -- 172.21.15.16:0/1169081519 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f273800bd70 con 0x7f273c0a09c0 2023-09-30T22:34:01.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.392+0000 7f8c1ffff700 1 -- 172.21.15.16:0/3580683694 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c2000e070 con 0x7f8c28139170 2023-09-30T22:34:01.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.392+0000 7f8c1ffff700 1 -- 172.21.15.16:0/3580683694 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c2000bbd0 con 0x7f8c28139170 2023-09-30T22:34:01.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.392+0000 7f8c1ffff700 1 -- 172.21.15.16:0/3580683694 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c20012430 con 0x7f8c28139170 2023-09-30T22:34:01.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.393+0000 7f8c2f9c1700 1 -- 172.21.15.16:0/3580683694 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8c28139170 msgr2=0x7f8c2813b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:01.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.393+0000 7f2743e44700 1 -- 172.21.15.16:0/1169081519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f273c0a09c0 msgr2=0x7f273c0a0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:01.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.393+0000 7f2743e44700 1 --2- 172.21.15.16:0/1169081519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f273c0a09c0 0x7f273c0a0d90 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f27380098f0 tx=0x7f2738009c00 comp rx=0 tx=0).stop 2023-09-30T22:34:01.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.393+0000 7f8c2f9c1700 1 --2- 172.21.15.16:0/3580683694 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8c28139170 0x7f8c2813b550 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f8c20009dc0 tx=0x7f8c2000b330 comp rx=0 tx=0).stop 2023-09-30T22:34:01.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.393+0000 7f8c2f9c1700 1 -- 172.21.15.16:0/3580683694 shutdown_connections 2023-09-30T22:34:01.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.393+0000 7f8c2f9c1700 1 --2- 172.21.15.16:0/3580683694 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8c28139170 0x7f8c2813b550 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:01.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.393+0000 7f8c2f9c1700 1 --2- 172.21.15.16:0/3580683694 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8c2809e1d0 0x7f8c28138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:01.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.393+0000 7f8c2f9c1700 1 -- 172.21.15.16:0/3580683694 >> 172.21.15.16:0/3580683694 conn(0x7f8c28066f50 msgr2=0x7f8c2809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:01.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.393+0000 7f2743e44700 1 -- 172.21.15.16:0/1169081519 shutdown_connections 2023-09-30T22:34:01.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.393+0000 7f2743e44700 1 --2- 172.21.15.16:0/1169081519 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f273c0a09c0 0x7f273c0a0d90 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:01.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.393+0000 7f2743e44700 1 --2- 172.21.15.16:0/1169081519 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f273c09d5c0 0x7f273c0a12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:01.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.393+0000 7f2743e44700 1 -- 172.21.15.16:0/1169081519 >> 172.21.15.16:0/1169081519 conn(0x7f273c0a2fe0 msgr2=0x7f273c0a33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:01.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.393+0000 7f8c2f9c1700 1 -- 172.21.15.16:0/3580683694 shutdown_connections 2023-09-30T22:34:01.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.394+0000 7f8c2f9c1700 1 -- 172.21.15.16:0/3580683694 wait complete. 2023-09-30T22:34:01.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.394+0000 7f2743e44700 1 -- 172.21.15.16:0/1169081519 shutdown_connections 2023-09-30T22:34:01.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.394+0000 7f2743e44700 1 -- 172.21.15.16:0/1169081519 wait complete. 2023-09-30T22:34:01.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.395+0000 7f8c2f9c1700 1 Processor -- start 2023-09-30T22:34:01.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.395+0000 7f8c2f9c1700 1 -- start start 2023-09-30T22:34:01.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.395+0000 7f2743e44700 1 Processor -- start 2023-09-30T22:34:01.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.395+0000 7f2743e44700 1 -- start start 2023-09-30T22:34:01.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.395+0000 7f8c2f9c1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c2809d6f0 0x7f8c28120640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:01.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.395+0000 7f8c2f9c1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8c2809e1d0 0x7f8c28120b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:01.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.395+0000 7f8c2f9c1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8c28139170 0x7f8c281210c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:01.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.395+0000 7f8c2f9c1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c28123160 con 0x7f8c2809d6f0 2023-09-30T22:34:01.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.395+0000 7f8c2f9c1700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c28123510 con 0x7f8c28139170 2023-09-30T22:34:01.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.395+0000 7f8c2f9c1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8c28123e80 con 0x7f8c2809e1d0 2023-09-30T22:34:01.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.395+0000 7f8c2cf5c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8c2809e1d0 0x7f8c28120b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:01.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.396+0000 7f8c2cf5c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8c2809e1d0 0x7f8c28120b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:57388/0 (socket says 172.21.15.16:57388) 2023-09-30T22:34:01.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.396+0000 7f8c2cf5c700 1 -- 172.21.15.16:0/4234221904 learned_addr learned my addr 172.21.15.16:0/4234221904 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:01.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.396+0000 7f8c2d75d700 1 --2- 172.21.15.16:0/4234221904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c2809d6f0 0x7f8c28120640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:01.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.396+0000 7f2743e44700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f273c09d5c0 0x7f273c11e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:01.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.396+0000 7f2743e44700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f273c0a09c0 0x7f273c11ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:01.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.396+0000 7f2743e44700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f273c133920 0x7f273c1c8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:01.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.396+0000 7f2743e44700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f273c1316c0 con 0x7f273c09d5c0 2023-09-30T22:34:01.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.396+0000 7f2743e44700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f273c1206c0 con 0x7f273c133920 2023-09-30T22:34:01.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.396+0000 7f2743e44700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f273c11e3f0 con 0x7f273c0a09c0 2023-09-30T22:34:01.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.396+0000 7f8c2cf5c700 1 -- 172.21.15.16:0/4234221904 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8c28139170 msgr2=0x7f8c281210c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:01.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.396+0000 7f8c2cf5c700 1 --2- 172.21.15.16:0/4234221904 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8c28139170 0x7f8c281210c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:01.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.396+0000 7f8c2cf5c700 1 -- 172.21.15.16:0/4234221904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c2809d6f0 msgr2=0x7f8c28120640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:01.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.396+0000 7f8c2cf5c700 1 --2- 172.21.15.16:0/4234221904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c2809d6f0 0x7f8c28120640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:01.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.396+0000 7f8c2cf5c700 1 -- 172.21.15.16:0/4234221904 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c20009a70 con 0x7f8c2809e1d0 2023-09-30T22:34:01.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.396+0000 7f2741be0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f273c09d5c0 0x7f273c11e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:01.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.396+0000 7f27413df700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f273c0a09c0 0x7f273c11ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:01.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.396+0000 7f2741be0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f273c09d5c0 0x7f273c11e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37224/0 (socket says 172.21.15.16:37224) 2023-09-30T22:34:01.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.396+0000 7f2741be0700 1 -- 172.21.15.16:0/3271511462 learned_addr learned my addr 172.21.15.16:0/3271511462 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:01.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.396+0000 7f27423e1700 1 --2- 172.21.15.16:0/3271511462 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f273c133920 0x7f273c1c8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:01.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.397+0000 7f8c2cf5c700 1 --2- 172.21.15.16:0/4234221904 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8c2809e1d0 0x7f8c28120b80 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f8c1800b530 tx=0x7f8c1800b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:01.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.397+0000 7f27413df700 1 -- 172.21.15.16:0/3271511462 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f273c133920 msgr2=0x7f273c1c8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:01.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.397+0000 7f27413df700 1 --2- 172.21.15.16:0/3271511462 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f273c133920 0x7f273c1c8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:01.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.397+0000 7f27413df700 1 -- 172.21.15.16:0/3271511462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f273c09d5c0 msgr2=0x7f273c11e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:01.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.397+0000 7f27413df700 1 --2- 172.21.15.16:0/3271511462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f273c09d5c0 0x7f273c11e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:01.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.397+0000 7f27413df700 1 -- 172.21.15.16:0/3271511462 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2738009550 con 0x7f273c0a09c0 2023-09-30T22:34:01.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.397+0000 7f27413df700 1 --2- 172.21.15.16:0/3271511462 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f273c0a09c0 0x7f273c11ee60 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f272c00b4b0 tx=0x7f272c00b7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:01.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.397+0000 7f8c1dffb700 1 -- 172.21.15.16:0/4234221904 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c18011830 con 0x7f8c2809e1d0 2023-09-30T22:34:01.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.397+0000 7f8c1dffb700 1 -- 172.21.15.16:0/4234221904 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c18011990 con 0x7f8c2809e1d0 2023-09-30T22:34:01.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.397+0000 7f8c1dffb700 1 -- 172.21.15.16:0/4234221904 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c18011d40 con 0x7f8c2809e1d0 2023-09-30T22:34:01.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.397+0000 7f8c2f9c1700 1 -- 172.21.15.16:0/4234221904 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c28071f20 con 0x7f8c2809e1d0 2023-09-30T22:34:01.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.397+0000 7f27327fc700 1 -- 172.21.15.16:0/3271511462 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f272c011830 con 0x7f273c0a09c0 2023-09-30T22:34:01.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.397+0000 7f8c2f9c1700 1 -- 172.21.15.16:0/4234221904 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c281248f0 con 0x7f8c2809e1d0 2023-09-30T22:34:01.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.397+0000 7f27327fc700 1 -- 172.21.15.16:0/3271511462 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f272c011990 con 0x7f273c0a09c0 2023-09-30T22:34:01.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.397+0000 7f27327fc700 1 -- 172.21.15.16:0/3271511462 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f272c011d20 con 0x7f273c0a09c0 2023-09-30T22:34:01.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.397+0000 7f2743e44700 1 -- 172.21.15.16:0/3271511462 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f273c120a60 con 0x7f273c0a09c0 2023-09-30T22:34:01.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.397+0000 7f2743e44700 1 -- 172.21.15.16:0/3271511462 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f273c121580 con 0x7f273c0a09c0 2023-09-30T22:34:01.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.398+0000 7f2743e44700 1 -- 172.21.15.16:0/3271511462 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f273c0747f0 con 0x7f273c0a09c0 2023-09-30T22:34:01.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.399+0000 7f27327fc700 1 -- 172.21.15.16:0/3271511462 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f272c01a400 con 0x7f273c0a09c0 2023-09-30T22:34:01.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.399+0000 7f27327fc700 1 -- 172.21.15.16:0/3271511462 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f272c03f5e0 con 0x7f273c0a09c0 2023-09-30T22:34:01.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.399+0000 7f8c1dffb700 1 -- 172.21.15.16:0/4234221904 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8c1801a400 con 0x7f8c2809e1d0 2023-09-30T22:34:01.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.400+0000 7f8c1dffb700 1 -- 172.21.15.16:0/4234221904 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f8c1803f710 con 0x7f8c2809e1d0 2023-09-30T22:34:01.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.404+0000 7f8c2f9c1700 1 -- 172.21.15.16:0/4234221904 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c28123b20 con 0x7f8c2809e1d0 2023-09-30T22:34:01.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.408+0000 7f27327fc700 1 -- 172.21.15.16:0/3271511462 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f272c00f8d0 con 0x7f273c0a09c0 2023-09-30T22:34:01.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.417+0000 7f8c1dffb700 1 -- 172.21.15.16:0/4234221904 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8c18015020 con 0x7f8c2809e1d0 2023-09-30T22:34:01.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.440+0000 7f5810c09700 1 Processor -- start 2023-09-30T22:34:01.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.440+0000 7f5810c09700 1 -- start start 2023-09-30T22:34:01.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.440+0000 7f5810c09700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f580c09d6f0 0x7f580c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:01.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.440+0000 7f5810c09700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f580c09e410 0x7f580c130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:01.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.440+0000 7f5810c09700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f580c09f130 con 0x7f580c09e000 2023-09-30T22:34:01.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.440+0000 7f5810c09700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f580c09f520 con 0x7f580c09d6f0 2023-09-30T22:34:01.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.440+0000 7f5810c09700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f580c09e930 con 0x7f580c09e410 2023-09-30T22:34:01.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.441+0000 7f580ad9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f580c09e410 0x7f580c130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:01.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.441+0000 7f580ad9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f580c09e410 0x7f580c130d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38614/0 (socket says 172.21.15.16:38614) 2023-09-30T22:34:01.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.441+0000 7f580ad9d700 1 -- 172.21.15.16:0/4012311139 learned_addr learned my addr 172.21.15.16:0/4012311139 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:01.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.441+0000 7f5809d9b700 1 --1- 172.21.15.16:0/4012311139 >> v1:172.21.15.16:6789/0 conn(0x7f580c09e000 0x7f580c130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47472/0 (socket says 172.21.15.16:47472) 2023-09-30T22:34:01.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.441+0000 7f580a59c700 1 --2- 172.21.15.16:0/4012311139 >> v2:172.21.15.16:3300/0 conn(0x7f580c09d6f0 0x7f580c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:01.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.441+0000 7f580ad9d700 1 -- 172.21.15.16:0/4012311139 >> v2:172.21.15.16:3300/0 conn(0x7f580c09d6f0 msgr2=0x7f580c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:01.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.441+0000 7f580ad9d700 1 --2- 172.21.15.16:0/4012311139 >> v2:172.21.15.16:3300/0 conn(0x7f580c09d6f0 0x7f580c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:01.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.441+0000 7f580ad9d700 1 -- 172.21.15.16:0/4012311139 >> v1:172.21.15.16:6789/0 conn(0x7f580c09e000 legacy=0x7f580c130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:01.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.441+0000 7f580ad9d700 1 -- 172.21.15.16:0/4012311139 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f580c09b2d0 con 0x7f580c09e410 2023-09-30T22:34:01.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.441+0000 7f580ad9d700 1 --2- 172.21.15.16:0/4012311139 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f580c09e410 0x7f580c130d70 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f580400d150 tx=0x7f580400d460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f8bf749de010d76 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:01.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.442+0000 7f580959a700 1 -- 172.21.15.16:0/4012311139 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5804010070 con 0x7f580c09e410 2023-09-30T22:34:01.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.442+0000 7f580959a700 1 -- 172.21.15.16:0/4012311139 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f580400dd10 con 0x7f580c09e410 2023-09-30T22:34:01.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.442+0000 7f580959a700 1 -- 172.21.15.16:0/4012311139 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580400b2d0 con 0x7f580c09e410 2023-09-30T22:34:01.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.442+0000 7f5810c09700 1 -- 172.21.15.16:0/4012311139 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f580c09e410 msgr2=0x7f580c130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:01.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.442+0000 7f5810c09700 1 --2- 172.21.15.16:0/4012311139 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f580c09e410 0x7f580c130d70 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f580400d150 tx=0x7f580400d460 comp rx=0 tx=0).stop 2023-09-30T22:34:01.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.442+0000 7f5810c09700 1 -- 172.21.15.16:0/4012311139 shutdown_connections 2023-09-30T22:34:01.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.442+0000 7f5810c09700 1 --2- 172.21.15.16:0/4012311139 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f580c09e410 0x7f580c130d70 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:01.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.442+0000 7f5810c09700 1 --2- 172.21.15.16:0/4012311139 >> v2:172.21.15.16:3300/0 conn(0x7f580c09d6f0 0x7f580c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:01.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.442+0000 7f5810c09700 1 -- 172.21.15.16:0/4012311139 >> 172.21.15.16:0/4012311139 conn(0x7f580c066f50 msgr2=0x7f580c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:01.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.443+0000 7f5810c09700 1 -- 172.21.15.16:0/4012311139 shutdown_connections 2023-09-30T22:34:01.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.443+0000 7f5810c09700 1 -- 172.21.15.16:0/4012311139 wait complete. 2023-09-30T22:34:01.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.444+0000 7f5810c09700 1 Processor -- start 2023-09-30T22:34:01.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.444+0000 7f5810c09700 1 -- start start 2023-09-30T22:34:01.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.444+0000 7f5810c09700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f580c09d6f0 0x7f580c122840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:01.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.444+0000 7f5810c09700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f580c09e000 0x7f580c122d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:01.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.444+0000 7f5810c09700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f580c09e410 0x7f580c1232c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:01.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.444+0000 7f5810c09700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f580c126080 con 0x7f580c09d6f0 2023-09-30T22:34:01.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.444+0000 7f5810c09700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f580c124560 con 0x7f580c09e410 2023-09-30T22:34:01.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.444+0000 7f5810c09700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f580c072150 con 0x7f580c09e000 2023-09-30T22:34:01.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.445+0000 7f580ad9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f580c09e410 0x7f580c1232c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:01.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.445+0000 7f580ad9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f580c09e410 0x7f580c1232c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38616/0 (socket says 172.21.15.16:38616) 2023-09-30T22:34:01.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.445+0000 7f580ad9d700 1 -- 172.21.15.16:0/1212903893 learned_addr learned my addr 172.21.15.16:0/1212903893 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:01.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.445+0000 7f580a59c700 1 --2- 172.21.15.16:0/1212903893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f580c09d6f0 0x7f580c122840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:01.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.445+0000 7f580ad9d700 1 -- 172.21.15.16:0/1212903893 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f580c09e000 msgr2=0x7f580c122d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:01.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.445+0000 7f580ad9d700 1 --2- 172.21.15.16:0/1212903893 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f580c09e000 0x7f580c122d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:01.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.445+0000 7f580ad9d700 1 -- 172.21.15.16:0/1212903893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f580c09d6f0 msgr2=0x7f580c122840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:01.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.445+0000 7f580ad9d700 1 --2- 172.21.15.16:0/1212903893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f580c09d6f0 0x7f580c122840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:01.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.445+0000 7f580ad9d700 1 -- 172.21.15.16:0/1212903893 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f580400bd40 con 0x7f580c09e410 2023-09-30T22:34:01.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.445+0000 7f580a59c700 1 --2- 172.21.15.16:0/1212903893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f580c09d6f0 0x7f580c122840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:01.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.445+0000 7f580ad9d700 1 --2- 172.21.15.16:0/1212903893 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f580c09e410 0x7f580c1232c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f580400d7b0 tx=0x7f5804004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:01.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.445+0000 7f57faffd700 1 -- 172.21.15.16:0/1212903893 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580400b6e0 con 0x7f580c09e410 2023-09-30T22:34:01.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.445+0000 7f57faffd700 1 -- 172.21.15.16:0/1212903893 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5804009810 con 0x7f580c09e410 2023-09-30T22:34:01.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.445+0000 7f5810c09700 1 -- 172.21.15.16:0/1212903893 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f580c09c700 con 0x7f580c09e410 2023-09-30T22:34:01.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.446+0000 7f5810c09700 1 -- 172.21.15.16:0/1212903893 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f580c126af0 con 0x7f580c09e410 2023-09-30T22:34:01.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.446+0000 7f57faffd700 1 -- 172.21.15.16:0/1212903893 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5804009bc0 con 0x7f580c09e410 2023-09-30T22:34:01.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.447+0000 7f57faffd700 1 -- 172.21.15.16:0/1212903893 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5804004040 con 0x7f580c09e410 2023-09-30T22:34:01.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.447+0000 7f57faffd700 1 -- 172.21.15.16:0/1212903893 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f5804040d60 con 0x7f580c09e410 2023-09-30T22:34:01.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.448+0000 7f5810c09700 1 -- 172.21.15.16:0/1212903893 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f580c074ea0 con 0x7f580c09e410 2023-09-30T22:34:01.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.453+0000 7f57faffd700 1 -- 172.21.15.16:0/1212903893 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f580401d430 con 0x7f580c09e410 2023-09-30T22:34:01.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.581+0000 7f819abe6700 1 -- 172.21.15.16:0/1844653421 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f819406c360 con 0x7f819409e000 2023-09-30T22:34:01.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.581+0000 7f81917fa700 1 -- 172.21.15.16:0/1844653421 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8180019770 con 0x7f819409e000 2023-09-30T22:34:01.582 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:34:01.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.584+0000 7f819abe6700 1 -- 172.21.15.16:0/1844653421 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f819409e000 msgr2=0x7f81941a65b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:01.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.584+0000 7f819abe6700 1 --2- 172.21.15.16:0/1844653421 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f819409e000 0x7f81941a65b0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f819409e9b0 tx=0x7f8180004750 comp rx=0 tx=0).stop 2023-09-30T22:34:01.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.584+0000 7f819abe6700 1 -- 172.21.15.16:0/1844653421 shutdown_connections 2023-09-30T22:34:01.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.584+0000 7f819abe6700 1 --2- 172.21.15.16:0/1844653421 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f819409d6f0 0x7f81941a6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:01.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.584+0000 7f819abe6700 1 --2- 172.21.15.16:0/1844653421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f819409e410 0x7f81941a6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:01.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.584+0000 7f819abe6700 1 --2- 172.21.15.16:0/1844653421 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f819409e000 0x7f81941a65b0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:01.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.584+0000 7f819abe6700 1 -- 172.21.15.16:0/1844653421 >> 172.21.15.16:0/1844653421 conn(0x7f8194066f50 msgr2=0x7f81941bad40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:01.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.585+0000 7f819abe6700 1 -- 172.21.15.16:0/1844653421 shutdown_connections 2023-09-30T22:34:01.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.585+0000 7f819abe6700 1 -- 172.21.15.16:0/1844653421 wait complete. 2023-09-30T22:34:01.604 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:34:01.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.624+0000 7f8c2f9c1700 1 -- 172.21.15.16:0/4234221904 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8c28074ea0 con 0x7f8c2809e1d0 2023-09-30T22:34:01.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.624+0000 7f8c1dffb700 1 -- 172.21.15.16:0/4234221904 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8c18029090 con 0x7f8c2809e1d0 2023-09-30T22:34:01.625 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:34:01.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.626+0000 7f2743e44700 1 -- 172.21.15.16:0/3271511462 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f273c072980 con 0x7f273c0a09c0 2023-09-30T22:34:01.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.626+0000 7f27327fc700 1 -- 172.21.15.16:0/3271511462 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f272c015030 con 0x7f273c0a09c0 2023-09-30T22:34:01.626 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:34:01.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.627+0000 7f8c2f9c1700 1 -- 172.21.15.16:0/4234221904 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8c2809e1d0 msgr2=0x7f8c28120b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:01.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.627+0000 7f8c2f9c1700 1 --2- 172.21.15.16:0/4234221904 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8c2809e1d0 0x7f8c28120b80 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f8c1800b530 tx=0x7f8c1800b8f0 comp rx=0 tx=0).stop 2023-09-30T22:34:01.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.627+0000 7f8c2f9c1700 1 -- 172.21.15.16:0/4234221904 shutdown_connections 2023-09-30T22:34:01.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.627+0000 7f8c2f9c1700 1 --2- 172.21.15.16:0/4234221904 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8c28139170 0x7f8c281210c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:01.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.627+0000 7f8c2f9c1700 1 --2- 172.21.15.16:0/4234221904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8c2809d6f0 0x7f8c28120640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:01.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.627+0000 7f8c2f9c1700 1 --2- 172.21.15.16:0/4234221904 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8c2809e1d0 0x7f8c28120b80 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:01.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.627+0000 7f8c2f9c1700 1 -- 172.21.15.16:0/4234221904 >> 172.21.15.16:0/4234221904 conn(0x7f8c28066f50 msgr2=0x7f8c28070ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:01.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.627+0000 7f8c2f9c1700 1 -- 172.21.15.16:0/4234221904 shutdown_connections 2023-09-30T22:34:01.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.627+0000 7f8c2f9c1700 1 -- 172.21.15.16:0/4234221904 wait complete. 2023-09-30T22:34:01.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.628+0000 7f2743e44700 1 -- 172.21.15.16:0/3271511462 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f273c0a09c0 msgr2=0x7f273c11ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:01.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.628+0000 7f2743e44700 1 --2- 172.21.15.16:0/3271511462 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f273c0a09c0 0x7f273c11ee60 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f272c00b4b0 tx=0x7f272c00b7c0 comp rx=0 tx=0).stop 2023-09-30T22:34:01.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.629+0000 7f2743e44700 1 -- 172.21.15.16:0/3271511462 shutdown_connections 2023-09-30T22:34:01.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.629+0000 7f2743e44700 1 --2- 172.21.15.16:0/3271511462 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f273c133920 0x7f273c1c8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:01.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.629+0000 7f2743e44700 1 --2- 172.21.15.16:0/3271511462 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f273c09d5c0 0x7f273c11e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:01.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.629+0000 7f2743e44700 1 --2- 172.21.15.16:0/3271511462 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f273c0a09c0 0x7f273c11ee60 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:01.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.629+0000 7f2743e44700 1 -- 172.21.15.16:0/3271511462 >> 172.21.15.16:0/3271511462 conn(0x7f273c0a2fe0 msgr2=0x7f273c132e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:01.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.629+0000 7f2743e44700 1 -- 172.21.15.16:0/3271511462 shutdown_connections 2023-09-30T22:34:01.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.629+0000 7f2743e44700 1 -- 172.21.15.16:0/3271511462 wait complete. 2023-09-30T22:34:01.644 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:34:01.645 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:34:01.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.662+0000 7f5810c09700 1 -- 172.21.15.16:0/1212903893 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f580c071f30 con 0x7f580c09e410 2023-09-30T22:34:01.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.663+0000 7f57faffd700 1 -- 172.21.15.16:0/1212903893 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5804034090 con 0x7f580c09e410 2023-09-30T22:34:01.663 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:34:01.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.665+0000 7f5810c09700 1 -- 172.21.15.16:0/1212903893 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f580c09e410 msgr2=0x7f580c1232c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:01.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.665+0000 7f5810c09700 1 --2- 172.21.15.16:0/1212903893 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f580c09e410 0x7f580c1232c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f580400d7b0 tx=0x7f5804004ad0 comp rx=0 tx=0).stop 2023-09-30T22:34:01.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.665+0000 7f5810c09700 1 -- 172.21.15.16:0/1212903893 shutdown_connections 2023-09-30T22:34:01.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.665+0000 7f5810c09700 1 --2- 172.21.15.16:0/1212903893 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f580c09e410 0x7f580c1232c0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:01.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.665+0000 7f5810c09700 1 --2- 172.21.15.16:0/1212903893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f580c09d6f0 0x7f580c122840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:01.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.665+0000 7f5810c09700 1 --2- 172.21.15.16:0/1212903893 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f580c09e000 0x7f580c122d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:01.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.665+0000 7f5810c09700 1 -- 172.21.15.16:0/1212903893 >> 172.21.15.16:0/1212903893 conn(0x7f580c066f50 msgr2=0x7f580c070f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:01.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.665+0000 7f5810c09700 1 -- 172.21.15.16:0/1212903893 shutdown_connections 2023-09-30T22:34:01.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:01.665+0000 7f5810c09700 1 -- 172.21.15.16:0/1212903893 wait complete. 2023-09-30T22:34:01.679 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:34:02.605 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:34:02.645 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:34:02.647 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:34:02.680 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:34:02.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.762+0000 7ff70419a700 1 Processor -- start 2023-09-30T22:34:02.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.763+0000 7ff70419a700 1 -- start start 2023-09-30T22:34:02.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.763+0000 7ff70419a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff6fc06a770 0x7ff6fc09d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:02.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.763+0000 7ff70419a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff6fc09dc30 0x7ff6fc09e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:02.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.763+0000 7ff70419a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff6fc13c1e0 0x7ff6fc13e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:02.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.763+0000 7ff70419a700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6fc06ab40 con 0x7ff6fc09dc30 2023-09-30T22:34:02.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.763+0000 7ff70419a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6fc09ebc0 con 0x7ff6fc06a770 2023-09-30T22:34:02.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.763+0000 7ff70419a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff6fc09a810 con 0x7ff6fc13c1e0 2023-09-30T22:34:02.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.763+0000 7ff702737700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff6fc13c1e0 0x7ff6fc13e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:02.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.763+0000 7ff701735700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff6fc09dc30 0x7ff6fc09e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:02.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.764+0000 7ff702737700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff6fc13c1e0 0x7ff6fc13e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:57410/0 (socket says 172.21.15.16:57410) 2023-09-30T22:34:02.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.764+0000 7ff702737700 1 -- 172.21.15.16:0/3292869628 learned_addr learned my addr 172.21.15.16:0/3292869628 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:02.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.764+0000 7ff701f36700 1 --2- 172.21.15.16:0/3292869628 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff6fc06a770 0x7ff6fc09d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:02.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.764+0000 7ff702737700 1 -- 172.21.15.16:0/3292869628 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff6fc06a770 msgr2=0x7ff6fc09d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:02.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.764+0000 7ff702737700 1 --2- 172.21.15.16:0/3292869628 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff6fc06a770 0x7ff6fc09d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:02.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.764+0000 7ff702737700 1 -- 172.21.15.16:0/3292869628 >> v2:172.21.15.16:3300/0 conn(0x7ff6fc09dc30 msgr2=0x7ff6fc09e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:02.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.764+0000 7ff702737700 1 --2- 172.21.15.16:0/3292869628 >> v2:172.21.15.16:3300/0 conn(0x7ff6fc09dc30 0x7ff6fc09e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:02.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.764+0000 7ff702737700 1 -- 172.21.15.16:0/3292869628 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6fc09e920 con 0x7ff6fc13c1e0 2023-09-30T22:34:02.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.764+0000 7ff701735700 1 --2- 172.21.15.16:0/3292869628 >> v2:172.21.15.16:3300/0 conn(0x7ff6fc09dc30 0x7ff6fc09e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:02.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.764+0000 7ff702737700 1 --2- 172.21.15.16:0/3292869628 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff6fc13c1e0 0x7ff6fc13e5c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7ff6f4009e80 tx=0x7ff6f400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c61f69ab583ad1b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:02.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.764+0000 7ff700f34700 1 -- 172.21.15.16:0/3292869628 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f400e070 con 0x7ff6fc13c1e0 2023-09-30T22:34:02.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.765+0000 7ff700f34700 1 -- 172.21.15.16:0/3292869628 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6f400b8d0 con 0x7ff6fc13c1e0 2023-09-30T22:34:02.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.765+0000 7ff700f34700 1 -- 172.21.15.16:0/3292869628 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f400bc80 con 0x7ff6fc13c1e0 2023-09-30T22:34:02.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.765+0000 7ff70419a700 1 -- 172.21.15.16:0/3292869628 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff6fc13c1e0 msgr2=0x7ff6fc13e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:02.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.765+0000 7ff70419a700 1 --2- 172.21.15.16:0/3292869628 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff6fc13c1e0 0x7ff6fc13e5c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7ff6f4009e80 tx=0x7ff6f400b330 comp rx=0 tx=0).stop 2023-09-30T22:34:02.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.765+0000 7ff70419a700 1 -- 172.21.15.16:0/3292869628 shutdown_connections 2023-09-30T22:34:02.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.765+0000 7ff70419a700 1 --2- 172.21.15.16:0/3292869628 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff6fc06a770 0x7ff6fc09d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:02.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.765+0000 7ff70419a700 1 --2- 172.21.15.16:0/3292869628 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff6fc13c1e0 0x7ff6fc13e5c0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:02.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.765+0000 7ff70419a700 1 --2- 172.21.15.16:0/3292869628 >> v2:172.21.15.16:3300/0 conn(0x7ff6fc09dc30 0x7ff6fc09e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:02.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.765+0000 7ff70419a700 1 -- 172.21.15.16:0/3292869628 >> 172.21.15.16:0/3292869628 conn(0x7ff6fc066f50 msgr2=0x7ff6fc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:02.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.765+0000 7ff70419a700 1 -- 172.21.15.16:0/3292869628 shutdown_connections 2023-09-30T22:34:02.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.766+0000 7ff70419a700 1 -- 172.21.15.16:0/3292869628 wait complete. 2023-09-30T22:34:02.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.766+0000 7ff70419a700 1 Processor -- start 2023-09-30T22:34:02.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.767+0000 7ff70419a700 1 -- start start 2023-09-30T22:34:02.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.767+0000 7ff70419a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff6fc06a770 0x7ff6fc11cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:02.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.767+0000 7ff70419a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff6fc09dc30 0x7ff6fc12dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:02.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.767+0000 7ff70419a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6fc13c1e0 0x7ff6fc12e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:02.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.767+0000 7ff70419a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6fc12fe40 con 0x7ff6fc13c1e0 2023-09-30T22:34:02.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.767+0000 7ff70419a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6fc12fc50 con 0x7ff6fc09dc30 2023-09-30T22:34:02.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.767+0000 7ff70419a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff6fc12f510 con 0x7ff6fc06a770 2023-09-30T22:34:02.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.767+0000 7ff701f36700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff6fc06a770 0x7ff6fc11cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:02.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.767+0000 7ff701f36700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff6fc06a770 0x7ff6fc11cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:57412/0 (socket says 172.21.15.16:57412) 2023-09-30T22:34:02.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.767+0000 7ff701f36700 1 -- 172.21.15.16:0/1882245343 learned_addr learned my addr 172.21.15.16:0/1882245343 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:02.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.767+0000 7ff702737700 1 --2- 172.21.15.16:0/1882245343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6fc13c1e0 0x7ff6fc12e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:02.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.768+0000 7ff701735700 1 --2- 172.21.15.16:0/1882245343 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff6fc09dc30 0x7ff6fc12dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:02.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.768+0000 7ff701f36700 1 -- 172.21.15.16:0/1882245343 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff6fc09dc30 msgr2=0x7ff6fc12dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:02.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.768+0000 7ff701f36700 1 --2- 172.21.15.16:0/1882245343 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff6fc09dc30 0x7ff6fc12dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:02.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.768+0000 7ff701f36700 1 -- 172.21.15.16:0/1882245343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6fc13c1e0 msgr2=0x7ff6fc12e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:02.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.768+0000 7ff701f36700 1 --2- 172.21.15.16:0/1882245343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6fc13c1e0 0x7ff6fc12e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:02.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.768+0000 7ff701f36700 1 -- 172.21.15.16:0/1882245343 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6ec0094a0 con 0x7ff6fc06a770 2023-09-30T22:34:02.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.768+0000 7ff701f36700 1 --2- 172.21.15.16:0/1882245343 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff6fc06a770 0x7ff6fc11cae0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff6f800e9b0 tx=0x7ff6f800ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:02.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.768+0000 7ff6f27fc700 1 -- 172.21.15.16:0/1882245343 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f8009710 con 0x7ff6fc06a770 2023-09-30T22:34:02.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.768+0000 7ff70419a700 1 -- 172.21.15.16:0/1882245343 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6f4009b30 con 0x7ff6fc06a770 2023-09-30T22:34:02.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.768+0000 7ff6f27fc700 1 -- 172.21.15.16:0/1882245343 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6f800c850 con 0x7ff6fc06a770 2023-09-30T22:34:02.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.768+0000 7ff6f27fc700 1 -- 172.21.15.16:0/1882245343 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f800cbc0 con 0x7ff6fc06a770 2023-09-30T22:34:02.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.768+0000 7ff70419a700 1 -- 172.21.15.16:0/1882245343 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6fc09f1a0 con 0x7ff6fc06a770 2023-09-30T22:34:02.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.769+0000 7ff6f27fc700 1 -- 172.21.15.16:0/1882245343 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff6f800cd20 con 0x7ff6fc06a770 2023-09-30T22:34:02.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.769+0000 7ff6f27fc700 1 -- 172.21.15.16:0/1882245343 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7ff6f803f820 con 0x7ff6fc06a770 2023-09-30T22:34:02.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.770+0000 7ff70419a700 1 -- 172.21.15.16:0/1882245343 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6fc071ff0 con 0x7ff6fc06a770 2023-09-30T22:34:02.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.779+0000 7ff6f27fc700 1 -- 172.21.15.16:0/1882245343 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff6f80232f0 con 0x7ff6fc06a770 2023-09-30T22:34:02.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.800+0000 7f4d46b08700 1 Processor -- start 2023-09-30T22:34:02.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.800+0000 7f4d46b08700 1 -- start start 2023-09-30T22:34:02.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.800+0000 7fb9375fd700 1 Processor -- start 2023-09-30T22:34:02.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.800+0000 7fb9375fd700 1 -- start start 2023-09-30T22:34:02.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.801+0000 7f4d46b08700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4d401316d0 0x7f4d40131aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:02.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.801+0000 7f4d46b08700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4d4006ec60 0x7f4d40139740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:02.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.801+0000 7f4d46b08700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d40131510 con 0x7f4d40131fe0 2023-09-30T22:34:02.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.801+0000 7f4d46b08700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d40099e70 con 0x7f4d401316d0 2023-09-30T22:34:02.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.801+0000 7f4d46b08700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4d40066680 con 0x7f4d4006ec60 2023-09-30T22:34:02.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.801+0000 7f4d448a4700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4d401316d0 0x7f4d40131aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:02.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.801+0000 7f4d448a4700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4d401316d0 0x7f4d40131aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38642/0 (socket says 172.21.15.16:38642) 2023-09-30T22:34:02.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.801+0000 7f4d448a4700 1 -- 172.21.15.16:0/2488980050 learned_addr learned my addr 172.21.15.16:0/2488980050 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:02.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.801+0000 7f4d450a5700 1 --2- 172.21.15.16:0/2488980050 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4d4006ec60 0x7f4d40139740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:02.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.801+0000 7fb9375fd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb9301305f0 0x7fb9301309c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:02.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.801+0000 7fb9375fd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb930133710 0x7fb930135af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:02.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.801+0000 7fb9375fd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb930131500 0x7fb9301319a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:02.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.801+0000 7fb9375fd700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb93009a2e0 con 0x7fb930131500 2023-09-30T22:34:02.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.801+0000 7fb9375fd700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9301301c0 con 0x7fb930133710 2023-09-30T22:34:02.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.801+0000 7fb9375fd700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb930071ff0 con 0x7fb9301305f0 2023-09-30T22:34:02.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.802+0000 7fb935399700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb9301305f0 0x7fb9301309c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:02.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.801+0000 7f4d448a4700 1 -- 172.21.15.16:0/2488980050 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4d4006ec60 msgr2=0x7f4d40139740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:02.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.801+0000 7f4d448a4700 1 --2- 172.21.15.16:0/2488980050 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4d4006ec60 0x7f4d40139740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:02.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.801+0000 7f4d448a4700 1 -- 172.21.15.16:0/2488980050 >> v1:172.21.15.16:6789/0 conn(0x7f4d40131fe0 legacy=0x7f4d4006e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:02.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.801+0000 7f4d448a4700 1 -- 172.21.15.16:0/2488980050 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d4009ee00 con 0x7f4d401316d0 2023-09-30T22:34:02.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.802+0000 7f4d448a4700 1 --2- 172.21.15.16:0/2488980050 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4d401316d0 0x7f4d40131aa0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f4d34009920 tx=0x7f4d34009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a5796e1ef89ad30 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:02.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.802+0000 7fb935399700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb9301305f0 0x7fb9301309c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:57438/0 (socket says 172.21.15.16:57438) 2023-09-30T22:34:02.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.802+0000 7fb935399700 1 -- 172.21.15.16:0/3189018999 learned_addr learned my addr 172.21.15.16:0/3189018999 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:02.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.802+0000 7fb934b98700 1 --2- 172.21.15.16:0/3189018999 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb930133710 0x7fb930135af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:02.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.802+0000 7fb935399700 1 -- 172.21.15.16:0/3189018999 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb930133710 msgr2=0x7fb930135af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:02.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.802+0000 7fb935399700 1 --2- 172.21.15.16:0/3189018999 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb930133710 0x7fb930135af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:02.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.802+0000 7fb935399700 1 -- 172.21.15.16:0/3189018999 >> v2:172.21.15.16:3300/0 conn(0x7fb930131500 msgr2=0x7fb9301319a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:02.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.802+0000 7fb935399700 1 --2- 172.21.15.16:0/3189018999 >> v2:172.21.15.16:3300/0 conn(0x7fb930131500 0x7fb9301319a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:02.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.802+0000 7fb935399700 1 -- 172.21.15.16:0/3189018999 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb930070e70 con 0x7fb9301305f0 2023-09-30T22:34:02.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.802+0000 7f4d3f7fe700 1 -- 172.21.15.16:0/2488980050 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d34004230 con 0x7f4d401316d0 2023-09-30T22:34:02.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.802+0000 7f4d3f7fe700 1 -- 172.21.15.16:0/2488980050 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d34004390 con 0x7f4d401316d0 2023-09-30T22:34:02.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.802+0000 7f4d3f7fe700 1 -- 172.21.15.16:0/2488980050 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d340036a0 con 0x7f4d401316d0 2023-09-30T22:34:02.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.802+0000 7fb935399700 1 --2- 172.21.15.16:0/3189018999 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb9301305f0 0x7fb9301309c0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fb92c009920 tx=0x7fb92c009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d7462a2f21f422a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:02.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.802+0000 7fb927fff700 1 -- 172.21.15.16:0/3189018999 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92c011830 con 0x7fb9301305f0 2023-09-30T22:34:02.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.802+0000 7fb927fff700 1 -- 172.21.15.16:0/3189018999 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb92c011990 con 0x7fb9301305f0 2023-09-30T22:34:02.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.802+0000 7fb927fff700 1 -- 172.21.15.16:0/3189018999 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92c011d20 con 0x7fb9301305f0 2023-09-30T22:34:02.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.803+0000 7f4d46b08700 1 -- 172.21.15.16:0/2488980050 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4d401316d0 msgr2=0x7f4d40131aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:02.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.803+0000 7f4d46b08700 1 --2- 172.21.15.16:0/2488980050 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4d401316d0 0x7f4d40131aa0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f4d34009920 tx=0x7f4d34009c30 comp rx=0 tx=0).stop 2023-09-30T22:34:02.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.803+0000 7f4d46b08700 1 -- 172.21.15.16:0/2488980050 shutdown_connections 2023-09-30T22:34:02.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.803+0000 7f4d46b08700 1 --2- 172.21.15.16:0/2488980050 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4d4006ec60 0x7f4d40139740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:02.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.803+0000 7f4d46b08700 1 --2- 172.21.15.16:0/2488980050 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4d401316d0 0x7f4d40131aa0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:02.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.803+0000 7f4d46b08700 1 -- 172.21.15.16:0/2488980050 >> 172.21.15.16:0/2488980050 conn(0x7f4d400a2aa0 msgr2=0x7f4d400a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:02.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.803+0000 7f4d46b08700 1 -- 172.21.15.16:0/2488980050 shutdown_connections 2023-09-30T22:34:02.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.803+0000 7f4d46b08700 1 -- 172.21.15.16:0/2488980050 wait complete. 2023-09-30T22:34:02.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.803+0000 7fb9375fd700 1 -- 172.21.15.16:0/3189018999 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb9301305f0 msgr2=0x7fb9301309c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:02.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.803+0000 7fb9375fd700 1 --2- 172.21.15.16:0/3189018999 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb9301305f0 0x7fb9301309c0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fb92c009920 tx=0x7fb92c009c30 comp rx=0 tx=0).stop 2023-09-30T22:34:02.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.803+0000 7fb9375fd700 1 -- 172.21.15.16:0/3189018999 shutdown_connections 2023-09-30T22:34:02.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.803+0000 7fb9375fd700 1 --2- 172.21.15.16:0/3189018999 >> v2:172.21.15.16:3300/0 conn(0x7fb930131500 0x7fb9301319a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:02.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.803+0000 7fb9375fd700 1 --2- 172.21.15.16:0/3189018999 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb9301305f0 0x7fb9301309c0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:02.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.803+0000 7fb9375fd700 1 --2- 172.21.15.16:0/3189018999 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb930133710 0x7fb930135af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:02.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.803+0000 7fb9375fd700 1 -- 172.21.15.16:0/3189018999 >> 172.21.15.16:0/3189018999 conn(0x7fb9300a2aa0 msgr2=0x7fb9300a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:02.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.804+0000 7fb9375fd700 1 -- 172.21.15.16:0/3189018999 shutdown_connections 2023-09-30T22:34:02.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.804+0000 7fb9375fd700 1 -- 172.21.15.16:0/3189018999 wait complete. 2023-09-30T22:34:02.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.805+0000 7fb9375fd700 1 Processor -- start 2023-09-30T22:34:02.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.805+0000 7f4d46b08700 1 Processor -- start 2023-09-30T22:34:02.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.805+0000 7f4d46b08700 1 -- start start 2023-09-30T22:34:02.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.805+0000 7f4d46b08700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4d4006ec60 0x7f4d401c8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:02.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.805+0000 7f4d46b08700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d401316d0 0x7f4d401c8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:02.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.805+0000 7f4d46b08700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4d40131fe0 0x7f4d401c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:02.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.805+0000 7f4d46b08700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d401367b0 con 0x7f4d401316d0 2023-09-30T22:34:02.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.805+0000 7f4d46b08700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d400a08c0 con 0x7f4d40131fe0 2023-09-30T22:34:02.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.805+0000 7f4d46b08700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4009a0b0 con 0x7f4d4006ec60 2023-09-30T22:34:02.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.805+0000 7fb9375fd700 1 -- start start 2023-09-30T22:34:02.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.805+0000 7fb9375fd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb9301305f0 0x7fb930134200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:02.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.805+0000 7fb9375fd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb930131500 0x7fb930134740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:02.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.805+0000 7fb9375fd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb930133710 0x7fb930134c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:02.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.805+0000 7fb9375fd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb930135f20 con 0x7fb930133710 2023-09-30T22:34:02.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.805+0000 7fb9375fd700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb930137a40 con 0x7fb9301305f0 2023-09-30T22:34:02.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.805+0000 7fb9375fd700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb93013bd80 con 0x7fb930131500 2023-09-30T22:34:02.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.806+0000 7fb935399700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb9301305f0 0x7fb930134200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:02.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.806+0000 7fb935399700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb9301305f0 0x7fb930134200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38658/0 (socket says 172.21.15.16:38658) 2023-09-30T22:34:02.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.806+0000 7fb935399700 1 -- 172.21.15.16:0/3636570107 learned_addr learned my addr 172.21.15.16:0/3636570107 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:02.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.806+0000 7fb934b98700 1 --2- 172.21.15.16:0/3636570107 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb930131500 0x7fb930134740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:02.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.806+0000 7fb935b9a700 1 --2- 172.21.15.16:0/3636570107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb930133710 0x7fb930134c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:02.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.807+0000 7fb935399700 1 -- 172.21.15.16:0/3636570107 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb930131500 msgr2=0x7fb930134740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:02.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.807+0000 7fb935399700 1 --2- 172.21.15.16:0/3636570107 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb930131500 0x7fb930134740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:02.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.807+0000 7fb935399700 1 -- 172.21.15.16:0/3636570107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb930133710 msgr2=0x7fb930134c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:02.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.807+0000 7fb935399700 1 --2- 172.21.15.16:0/3636570107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb930133710 0x7fb930134c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:02.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.807+0000 7fb935399700 1 -- 172.21.15.16:0/3636570107 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb92c009550 con 0x7fb9301305f0 2023-09-30T22:34:02.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.807+0000 7fb935399700 1 --2- 172.21.15.16:0/3636570107 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb9301305f0 0x7fb930134200 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fb92c0098f0 tx=0x7fb92c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:02.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.807+0000 7fb925ffb700 1 -- 172.21.15.16:0/3636570107 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92c004590 con 0x7fb9301305f0 2023-09-30T22:34:02.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.805+0000 7f4d448a4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4d4006ec60 0x7f4d401c8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:02.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.806+0000 7f4d448a4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4d4006ec60 0x7f4d401c8250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:57452/0 (socket says 172.21.15.16:57452) 2023-09-30T22:34:02.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.806+0000 7f4d448a4700 1 -- 172.21.15.16:0/656273948 learned_addr learned my addr 172.21.15.16:0/656273948 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:02.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.806+0000 7f4d450a5700 1 --2- 172.21.15.16:0/656273948 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4d40131fe0 0x7f4d401c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:02.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.806+0000 7f4d3ffff700 1 --2- 172.21.15.16:0/656273948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d401316d0 0x7f4d401c8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:02.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.806+0000 7f4d3ffff700 1 -- 172.21.15.16:0/656273948 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4d4006ec60 msgr2=0x7f4d401c8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:02.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.806+0000 7f4d3ffff700 1 --2- 172.21.15.16:0/656273948 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4d4006ec60 0x7f4d401c8250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:02.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.807+0000 7fb9375fd700 1 -- 172.21.15.16:0/3636570107 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb930071820 con 0x7fb9301305f0 2023-09-30T22:34:02.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.806+0000 7f4d3ffff700 1 -- 172.21.15.16:0/656273948 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4d40131fe0 msgr2=0x7f4d401c8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:02.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.806+0000 7f4d3ffff700 1 --2- 172.21.15.16:0/656273948 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4d40131fe0 0x7f4d401c8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:02.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.806+0000 7f4d3ffff700 1 -- 172.21.15.16:0/656273948 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d34009550 con 0x7f4d401316d0 2023-09-30T22:34:02.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.807+0000 7f4d3ffff700 1 --2- 172.21.15.16:0/656273948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d401316d0 0x7f4d401c8790 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f4d4009ebc0 tx=0x7f4d2c00c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:02.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.807+0000 7f4d3d7fa700 1 -- 172.21.15.16:0/656273948 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c0052d0 con 0x7f4d401316d0 2023-09-30T22:34:02.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.807+0000 7f4d46b08700 1 -- 172.21.15.16:0/656273948 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d40136310 con 0x7f4d401316d0 2023-09-30T22:34:02.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.807+0000 7f4d3d7fa700 1 -- 172.21.15.16:0/656273948 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d2c005430 con 0x7f4d401316d0 2023-09-30T22:34:02.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.807+0000 7f4d3d7fa700 1 -- 172.21.15.16:0/656273948 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c0038c0 con 0x7f4d401316d0 2023-09-30T22:34:02.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.807+0000 7f4d46b08700 1 -- 172.21.15.16:0/656273948 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d40137d40 con 0x7f4d401316d0 2023-09-30T22:34:02.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.807+0000 7fb9375fd700 1 -- 172.21.15.16:0/3636570107 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb930137d20 con 0x7fb9301305f0 2023-09-30T22:34:02.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.808+0000 7fb925ffb700 1 -- 172.21.15.16:0/3636570107 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb92c0036a0 con 0x7fb9301305f0 2023-09-30T22:34:02.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.808+0000 7fb925ffb700 1 -- 172.21.15.16:0/3636570107 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92c003a30 con 0x7fb9301305f0 2023-09-30T22:34:02.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.809+0000 7fb925ffb700 1 -- 172.21.15.16:0/3636570107 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb92c021070 con 0x7fb9301305f0 2023-09-30T22:34:02.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.809+0000 7fb9375fd700 1 -- 172.21.15.16:0/3636570107 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9100052e0 con 0x7fb9301305f0 2023-09-30T22:34:02.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.809+0000 7fb925ffb700 1 -- 172.21.15.16:0/3636570107 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fb92c040b10 con 0x7fb9301305f0 2023-09-30T22:34:02.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.809+0000 7f4d46b08700 1 -- 172.21.15.16:0/656273948 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d40074ea0 con 0x7f4d401316d0 2023-09-30T22:34:02.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.810+0000 7f4d3d7fa700 1 -- 172.21.15.16:0/656273948 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4d2c003c30 con 0x7f4d401316d0 2023-09-30T22:34:02.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.811+0000 7f4d3d7fa700 1 -- 172.21.15.16:0/656273948 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f4d2c03f520 con 0x7f4d401316d0 2023-09-30T22:34:02.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.815+0000 7fb925ffb700 1 -- 172.21.15.16:0/3636570107 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb92c00b3e0 con 0x7fb9301305f0 2023-09-30T22:34:02.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.819+0000 7f4d3d7fa700 1 -- 172.21.15.16:0/656273948 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4d2c023d00 con 0x7f4d401316d0 2023-09-30T22:34:02.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.852+0000 7fabe1dd8700 1 Processor -- start 2023-09-30T22:34:02.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.852+0000 7fabe1dd8700 1 -- start start 2023-09-30T22:34:03.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.852+0000 7fabe1dd8700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fabdc09eca0 0x7fabdc13cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:03.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.852+0000 7fabe1dd8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fabdc06f050 0x7fabdc13f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:03.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.852+0000 7fabe1dd8700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabdc070ee0 con 0x7fabdc06e570 2023-09-30T22:34:03.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.852+0000 7fabe1dd8700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabdc071980 con 0x7fabdc09eca0 2023-09-30T22:34:03.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.852+0000 7fabe1dd8700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc071480 con 0x7fabdc06f050 2023-09-30T22:34:03.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.853+0000 7fabdaffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fabdc09eca0 0x7fabdc13cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:03.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.853+0000 7fabdaffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fabdc09eca0 0x7fabdc13cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37282/0 (socket says 172.21.15.16:37282) 2023-09-30T22:34:03.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.853+0000 7fabdaffd700 1 -- 172.21.15.16:0/4068399209 learned_addr learned my addr 172.21.15.16:0/4068399209 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:03.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.853+0000 7fabdbfff700 1 --2- 172.21.15.16:0/4068399209 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fabdc06f050 0x7fabdc13f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:03.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.853+0000 7fabdbfff700 1 -- 172.21.15.16:0/4068399209 >> v2:172.21.15.16:3300/0 conn(0x7fabdc09eca0 msgr2=0x7fabdc13cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:03.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.853+0000 7fabdbfff700 1 --2- 172.21.15.16:0/4068399209 >> v2:172.21.15.16:3300/0 conn(0x7fabdc09eca0 0x7fabdc13cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:03.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.853+0000 7fabdbfff700 1 -- 172.21.15.16:0/4068399209 >> v1:172.21.15.16:6789/0 conn(0x7fabdc06e570 legacy=0x7fabdc06e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:03.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.853+0000 7fabdbfff700 1 -- 172.21.15.16:0/4068399209 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabdc071b90 con 0x7fabdc06f050 2023-09-30T22:34:03.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.853+0000 7fabdbfff700 1 --2- 172.21.15.16:0/4068399209 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fabdc06f050 0x7fabdc13f3f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fabd4009d90 tx=0x7fabd400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e9ac5bbabedd465 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:03.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.854+0000 7fabda7fc700 1 -- 172.21.15.16:0/4068399209 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd400e070 con 0x7fabdc06f050 2023-09-30T22:34:03.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.854+0000 7fabda7fc700 1 -- 172.21.15.16:0/4068399209 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd400bb20 con 0x7fabdc06f050 2023-09-30T22:34:03.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.854+0000 7fabda7fc700 1 -- 172.21.15.16:0/4068399209 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd4012430 con 0x7fabdc06f050 2023-09-30T22:34:03.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.854+0000 7fabe1dd8700 1 -- 172.21.15.16:0/4068399209 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fabdc06f050 msgr2=0x7fabdc13f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:03.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.855+0000 7fabe1dd8700 1 --2- 172.21.15.16:0/4068399209 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fabdc06f050 0x7fabdc13f3f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fabd4009d90 tx=0x7fabd400b330 comp rx=0 tx=0).stop 2023-09-30T22:34:03.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.855+0000 7fabe1dd8700 1 -- 172.21.15.16:0/4068399209 shutdown_connections 2023-09-30T22:34:03.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.855+0000 7fabe1dd8700 1 --2- 172.21.15.16:0/4068399209 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fabdc06f050 0x7fabdc13f3f0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:03.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.855+0000 7fabe1dd8700 1 --2- 172.21.15.16:0/4068399209 >> v2:172.21.15.16:3300/0 conn(0x7fabdc09eca0 0x7fabdc13cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:03.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.855+0000 7fabe1dd8700 1 -- 172.21.15.16:0/4068399209 >> 172.21.15.16:0/4068399209 conn(0x7fabdc0a2aa0 msgr2=0x7fabdc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:03.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.855+0000 7fabe1dd8700 1 -- 172.21.15.16:0/4068399209 shutdown_connections 2023-09-30T22:34:03.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.855+0000 7fabe1dd8700 1 -- 172.21.15.16:0/4068399209 wait complete. 2023-09-30T22:34:03.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.856+0000 7fabe1dd8700 1 Processor -- start 2023-09-30T22:34:03.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.856+0000 7fabe1dd8700 1 -- start start 2023-09-30T22:34:03.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.856+0000 7fabe1dd8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fabdc06e570 0x7fabdc13ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:03.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.856+0000 7fabe1dd8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabdc06f050 0x7fabdc13d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:03.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.856+0000 7fabe1dd8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fabdc13d8b0 0x7fabdc12adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:03.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.856+0000 7fabe1dd8700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc12c010 con 0x7fabdc06f050 2023-09-30T22:34:03.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.856+0000 7fabe1dd8700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc12de70 con 0x7fabdc13d8b0 2023-09-30T22:34:03.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.856+0000 7fabe1dd8700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fabdc0a2350 con 0x7fabdc06e570 2023-09-30T22:34:03.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.857+0000 7fabdaffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabdc06f050 0x7fabdc13d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:03.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.857+0000 7fabdaffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabdc06f050 0x7fabdc13d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37294/0 (socket says 172.21.15.16:37294) 2023-09-30T22:34:03.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.857+0000 7fabdaffd700 1 -- 172.21.15.16:0/240608454 learned_addr learned my addr 172.21.15.16:0/240608454 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:03.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.857+0000 7fabdbfff700 1 --2- 172.21.15.16:0/240608454 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fabdc13d8b0 0x7fabdc12adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:03.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.857+0000 7fabdb7fe700 1 --2- 172.21.15.16:0/240608454 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fabdc06e570 0x7fabdc13ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:03.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.857+0000 7fabdbfff700 1 -- 172.21.15.16:0/240608454 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fabdc06e570 msgr2=0x7fabdc13ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:03.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.857+0000 7fabdbfff700 1 --2- 172.21.15.16:0/240608454 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fabdc06e570 0x7fabdc13ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:03.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.857+0000 7fabdbfff700 1 -- 172.21.15.16:0/240608454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabdc06f050 msgr2=0x7fabdc13d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:03.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.857+0000 7fabdbfff700 1 --2- 172.21.15.16:0/240608454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabdc06f050 0x7fabdc13d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:03.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.857+0000 7fabdbfff700 1 -- 172.21.15.16:0/240608454 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd4009a70 con 0x7fabdc13d8b0 2023-09-30T22:34:03.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.857+0000 7fabdaffd700 1 --2- 172.21.15.16:0/240608454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabdc06f050 0x7fabdc13d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:03.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.857+0000 7fabdbfff700 1 --2- 172.21.15.16:0/240608454 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fabdc13d8b0 0x7fabdc12adf0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fabd4016000 tx=0x7fabd4004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:03.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.858+0000 7fabc3fff700 1 -- 172.21.15.16:0/240608454 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd4012430 con 0x7fabdc13d8b0 2023-09-30T22:34:03.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.858+0000 7fabc3fff700 1 -- 172.21.15.16:0/240608454 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd4020070 con 0x7fabdc13d8b0 2023-09-30T22:34:03.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.858+0000 7fabe1dd8700 1 -- 172.21.15.16:0/240608454 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabdc12d110 con 0x7fabdc13d8b0 2023-09-30T22:34:03.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.858+0000 7fabc3fff700 1 -- 172.21.15.16:0/240608454 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd4004280 con 0x7fabdc13d8b0 2023-09-30T22:34:03.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.858+0000 7fabe1dd8700 1 -- 172.21.15.16:0/240608454 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabdc12c640 con 0x7fabdc13d8b0 2023-09-30T22:34:03.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.859+0000 7fabc3fff700 1 -- 172.21.15.16:0/240608454 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fabd40043e0 con 0x7fabdc13d8b0 2023-09-30T22:34:03.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.859+0000 7fabe1dd8700 1 -- 172.21.15.16:0/240608454 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabdc074ea0 con 0x7fabdc13d8b0 2023-09-30T22:34:03.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.859+0000 7fabc3fff700 1 -- 172.21.15.16:0/240608454 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fabd4022070 con 0x7fabdc13d8b0 2023-09-30T22:34:03.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.864+0000 7fabc3fff700 1 -- 172.21.15.16:0/240608454 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fabd401c780 con 0x7fabdc13d8b0 2023-09-30T22:34:03.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.997+0000 7ff70419a700 1 -- 172.21.15.16:0/1882245343 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff6fc131670 con 0x7ff6fc06a770 2023-09-30T22:34:03.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:02.998+0000 7ff6f27fc700 1 -- 172.21.15.16:0/1882245343 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff6fc131670 con 0x7ff6fc06a770 2023-09-30T22:34:03.102 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:34:03.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:03.000+0000 7ff70419a700 1 -- 172.21.15.16:0/1882245343 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff6fc06a770 msgr2=0x7ff6fc11cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:03.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:03.000+0000 7ff70419a700 1 --2- 172.21.15.16:0/1882245343 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff6fc06a770 0x7ff6fc11cae0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff6f800e9b0 tx=0x7ff6f800ecc0 comp rx=0 tx=0).stop 2023-09-30T22:34:03.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:03.000+0000 7ff70419a700 1 -- 172.21.15.16:0/1882245343 shutdown_connections 2023-09-30T22:34:03.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:03.000+0000 7ff70419a700 1 --2- 172.21.15.16:0/1882245343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff6fc13c1e0 0x7ff6fc12e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:03.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:03.000+0000 7ff70419a700 1 --2- 172.21.15.16:0/1882245343 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff6fc06a770 0x7ff6fc11cae0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:03.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:03.000+0000 7ff70419a700 1 --2- 172.21.15.16:0/1882245343 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff6fc09dc30 0x7ff6fc12dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:03.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:03.000+0000 7ff70419a700 1 -- 172.21.15.16:0/1882245343 >> 172.21.15.16:0/1882245343 conn(0x7ff6fc066f50 msgr2=0x7ff6fc13dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:03.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:03.000+0000 7ff70419a700 1 -- 172.21.15.16:0/1882245343 shutdown_connections 2023-09-30T22:34:03.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:03.000+0000 7ff70419a700 1 -- 172.21.15.16:0/1882245343 wait complete. 2023-09-30T22:34:03.104 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:34:03.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:03.027+0000 7fb9375fd700 1 -- 172.21.15.16:0/3636570107 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb910005590 con 0x7fb9301305f0 2023-09-30T22:34:03.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:03.027+0000 7fb925ffb700 1 -- 172.21.15.16:0/3636570107 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb92c032020 con 0x7fb9301305f0 2023-09-30T22:34:03.105 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:34:03.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:03.030+0000 7fb9375fd700 1 -- 172.21.15.16:0/3636570107 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb9301305f0 msgr2=0x7fb930134200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:03.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:03.030+0000 7fb9375fd700 1 --2- 172.21.15.16:0/3636570107 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb9301305f0 0x7fb930134200 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fb92c0098f0 tx=0x7fb92c004750 comp rx=0 tx=0).stop 2023-09-30T22:34:03.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:03.030+0000 7fb9375fd700 1 -- 172.21.15.16:0/3636570107 shutdown_connections 2023-09-30T22:34:03.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:03.030+0000 7fb9375fd700 1 --2- 172.21.15.16:0/3636570107 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb9301305f0 0x7fb930134200 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:03.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:03.030+0000 7fb9375fd700 1 --2- 172.21.15.16:0/3636570107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb930133710 0x7fb930134c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:03.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:03.030+0000 7fb9375fd700 1 --2- 172.21.15.16:0/3636570107 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb930131500 0x7fb930134740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:03.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:03.030+0000 7fb9375fd700 1 -- 172.21.15.16:0/3636570107 >> 172.21.15.16:0/3636570107 conn(0x7fb9300a2aa0 msgr2=0x7fb9301319a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:03.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:03.030+0000 7fb9375fd700 1 -- 172.21.15.16:0/3636570107 shutdown_connections 2023-09-30T22:34:03.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:03.030+0000 7fb9375fd700 1 -- 172.21.15.16:0/3636570107 wait complete. 2023-09-30T22:34:03.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:03.031+0000 7f4d46b08700 1 -- 172.21.15.16:0/656273948 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4d40136f70 con 0x7f4d401316d0 2023-09-30T22:34:03.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:03.031+0000 7f4d3d7fa700 1 -- 172.21.15.16:0/656273948 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4d2c017070 con 0x7f4d401316d0 2023-09-30T22:34:03.107 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:34:03.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:03.035+0000 7f4d46b08700 1 -- 172.21.15.16:0/656273948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d401316d0 msgr2=0x7f4d401c8790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:03.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:03.035+0000 7f4d46b08700 1 --2- 172.21.15.16:0/656273948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d401316d0 0x7f4d401c8790 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f4d4009ebc0 tx=0x7f4d2c00c6e0 comp rx=0 tx=0).stop 2023-09-30T22:34:03.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:03.035+0000 7f4d46b08700 1 -- 172.21.15.16:0/656273948 shutdown_connections 2023-09-30T22:34:03.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:03.035+0000 7f4d46b08700 1 --2- 172.21.15.16:0/656273948 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4d40131fe0 0x7f4d401c8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:03.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:03.035+0000 7f4d46b08700 1 --2- 172.21.15.16:0/656273948 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4d4006ec60 0x7f4d401c8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:03.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:03.035+0000 7f4d46b08700 1 --2- 172.21.15.16:0/656273948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d401316d0 0x7f4d401c8790 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:03.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:03.035+0000 7f4d46b08700 1 -- 172.21.15.16:0/656273948 >> 172.21.15.16:0/656273948 conn(0x7f4d400a2aa0 msgr2=0x7f4d401307e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:03.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:03.035+0000 7f4d46b08700 1 -- 172.21.15.16:0/656273948 shutdown_connections 2023-09-30T22:34:03.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:03.035+0000 7f4d46b08700 1 -- 172.21.15.16:0/656273948 wait complete. 2023-09-30T22:34:03.109 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:34:03.110 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:34:03.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:03.072+0000 7fabe1dd8700 1 -- 172.21.15.16:0/240608454 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fabdc06b9e0 con 0x7fabdc13d8b0 2023-09-30T22:34:03.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:03.072+0000 7fabc3fff700 1 -- 172.21.15.16:0/240608454 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fabd402c030 con 0x7fabdc13d8b0 2023-09-30T22:34:03.111 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:34:03.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:03.074+0000 7fabe1dd8700 1 -- 172.21.15.16:0/240608454 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fabdc13d8b0 msgr2=0x7fabdc12adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:03.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:03.074+0000 7fabe1dd8700 1 --2- 172.21.15.16:0/240608454 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fabdc13d8b0 0x7fabdc12adf0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fabd4016000 tx=0x7fabd4004ad0 comp rx=0 tx=0).stop 2023-09-30T22:34:03.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:03.075+0000 7fabe1dd8700 1 -- 172.21.15.16:0/240608454 shutdown_connections 2023-09-30T22:34:03.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:03.075+0000 7fabe1dd8700 1 --2- 172.21.15.16:0/240608454 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fabdc13d8b0 0x7fabdc12adf0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:03.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:03.075+0000 7fabe1dd8700 1 --2- 172.21.15.16:0/240608454 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fabdc06e570 0x7fabdc13ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:03.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:03.075+0000 7fabe1dd8700 1 --2- 172.21.15.16:0/240608454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fabdc06f050 0x7fabdc13d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:03.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:03.075+0000 7fabe1dd8700 1 -- 172.21.15.16:0/240608454 >> 172.21.15.16:0/240608454 conn(0x7fabdc0a2aa0 msgr2=0x7fabdc09cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:03.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:03.075+0000 7fabe1dd8700 1 -- 172.21.15.16:0/240608454 shutdown_connections 2023-09-30T22:34:03.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:03.075+0000 7fabe1dd8700 1 -- 172.21.15.16:0/240608454 wait complete. 2023-09-30T22:34:03.113 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:34:04.105 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:34:04.110 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:34:04.111 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:34:04.114 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:34:04.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.263+0000 7fadebd33700 1 Processor -- start 2023-09-30T22:34:04.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.263+0000 7fadebd33700 1 -- start start 2023-09-30T22:34:04.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.263+0000 7fadebd33700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fade41338a0 0x7fade4133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:04.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.263+0000 7fadebd33700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fade409de50 0x7fade409e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:04.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.263+0000 7fadebd33700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fade409e760 0x7fade413a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:04.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.263+0000 7fadebd33700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fade406fef0 con 0x7fade41338a0 2023-09-30T22:34:04.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.264+0000 7fadebd33700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fade406ef30 con 0x7fade409de50 2023-09-30T22:34:04.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.264+0000 7fadebd33700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fade41334b0 con 0x7fade409e760 2023-09-30T22:34:04.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.264+0000 7fade9acf700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fade41338a0 0x7fade4133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:04.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.264+0000 7fade9acf700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fade41338a0 0x7fade4133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37308/0 (socket says 172.21.15.16:37308) 2023-09-30T22:34:04.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.264+0000 7fade9acf700 1 -- 172.21.15.16:0/564734541 learned_addr learned my addr 172.21.15.16:0/564734541 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:04.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.264+0000 7fadea2d0700 1 --2- 172.21.15.16:0/564734541 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fade409e760 0x7fade413a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:04.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.264+0000 7fade92ce700 1 --2- 172.21.15.16:0/564734541 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fade409de50 0x7fade409e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:04.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.264+0000 7fade9acf700 1 -- 172.21.15.16:0/564734541 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fade409e760 msgr2=0x7fade413a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:04.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.264+0000 7fade9acf700 1 --2- 172.21.15.16:0/564734541 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fade409e760 0x7fade413a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:04.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.264+0000 7fade9acf700 1 -- 172.21.15.16:0/564734541 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fade409de50 msgr2=0x7fade409e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:04.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.264+0000 7fade9acf700 1 --2- 172.21.15.16:0/564734541 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fade409de50 0x7fade409e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:04.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.264+0000 7fade9acf700 1 -- 172.21.15.16:0/564734541 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade406fc30 con 0x7fade41338a0 2023-09-30T22:34:04.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.264+0000 7fade9acf700 1 --2- 172.21.15.16:0/564734541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fade41338a0 0x7fade4133c70 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fade00098f0 tx=0x7fade0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f82d2c9d5ae74ebb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:04.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.265+0000 7fade8acd700 1 -- 172.21.15.16:0/564734541 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade0011830 con 0x7fade41338a0 2023-09-30T22:34:04.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.265+0000 7fade8acd700 1 -- 172.21.15.16:0/564734541 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fade0011990 con 0x7fade41338a0 2023-09-30T22:34:04.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.265+0000 7fade8acd700 1 -- 172.21.15.16:0/564734541 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade0011d20 con 0x7fade41338a0 2023-09-30T22:34:04.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.265+0000 7fadebd33700 1 -- 172.21.15.16:0/564734541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fade41338a0 msgr2=0x7fade4133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:04.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.265+0000 7fadebd33700 1 --2- 172.21.15.16:0/564734541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fade41338a0 0x7fade4133c70 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fade00098f0 tx=0x7fade0009c00 comp rx=0 tx=0).stop 2023-09-30T22:34:04.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.266+0000 7fadebd33700 1 -- 172.21.15.16:0/564734541 shutdown_connections 2023-09-30T22:34:04.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.266+0000 7fadebd33700 1 --2- 172.21.15.16:0/564734541 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fade409e760 0x7fade413a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:04.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.266+0000 7fadebd33700 1 --2- 172.21.15.16:0/564734541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fade41338a0 0x7fade4133c70 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:04.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.266+0000 7fadebd33700 1 --2- 172.21.15.16:0/564734541 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fade409de50 0x7fade409e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:04.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.266+0000 7fadebd33700 1 -- 172.21.15.16:0/564734541 >> 172.21.15.16:0/564734541 conn(0x7fade40a2aa0 msgr2=0x7fade40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:04.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.266+0000 7fadebd33700 1 -- 172.21.15.16:0/564734541 shutdown_connections 2023-09-30T22:34:04.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.266+0000 7fadebd33700 1 -- 172.21.15.16:0/564734541 wait complete. 2023-09-30T22:34:04.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.267+0000 7fadebd33700 1 Processor -- start 2023-09-30T22:34:04.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.267+0000 7fadebd33700 1 -- start start 2023-09-30T22:34:04.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.268+0000 7fadebd33700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fade409de50 0x7fade411b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:04.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.268+0000 7fadebd33700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fade409e760 0x7fade411bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:04.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.268+0000 7fadebd33700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fade41338a0 0x7fade411c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:04.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.268+0000 7fadebd33700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade411cbe0 con 0x7fade409de50 2023-09-30T22:34:04.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.268+0000 7fadebd33700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fade411d2f0 con 0x7fade409e760 2023-09-30T22:34:04.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.268+0000 7fadebd33700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fade411d910 con 0x7fade41338a0 2023-09-30T22:34:04.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.268+0000 7fadea2d0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fade41338a0 0x7fade411c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:04.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.268+0000 7fadea2d0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fade41338a0 0x7fade411c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:57508/0 (socket says 172.21.15.16:57508) 2023-09-30T22:34:04.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.268+0000 7fadea2d0700 1 -- 172.21.15.16:0/14004404 learned_addr learned my addr 172.21.15.16:0/14004404 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:04.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.268+0000 7fade9acf700 1 --2- 172.21.15.16:0/14004404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fade409de50 0x7fade411b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:04.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.268+0000 7fade92ce700 1 --2- 172.21.15.16:0/14004404 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fade409e760 0x7fade411bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:04.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.269+0000 7fadea2d0700 1 -- 172.21.15.16:0/14004404 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fade409e760 msgr2=0x7fade411bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:04.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.269+0000 7fadea2d0700 1 --2- 172.21.15.16:0/14004404 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fade409e760 0x7fade411bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:04.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.269+0000 7fadea2d0700 1 -- 172.21.15.16:0/14004404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fade409de50 msgr2=0x7fade411b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:04.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.269+0000 7fadea2d0700 1 --2- 172.21.15.16:0/14004404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fade409de50 0x7fade411b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:04.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.269+0000 7fadea2d0700 1 -- 172.21.15.16:0/14004404 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade0009550 con 0x7fade41338a0 2023-09-30T22:34:04.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.269+0000 7fadea2d0700 1 --2- 172.21.15.16:0/14004404 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fade41338a0 0x7fade411c1b0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7faddc009dd0 tx=0x7faddc00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:04.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.269+0000 7fadda7fc700 1 -- 172.21.15.16:0/14004404 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faddc009c40 con 0x7fade41338a0 2023-09-30T22:34:04.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.269+0000 7fadda7fc700 1 -- 172.21.15.16:0/14004404 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faddc00f040 con 0x7fade41338a0 2023-09-30T22:34:04.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.269+0000 7fadda7fc700 1 -- 172.21.15.16:0/14004404 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faddc00ceb0 con 0x7fade41338a0 2023-09-30T22:34:04.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.269+0000 7fadebd33700 1 -- 172.21.15.16:0/14004404 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fade411ef20 con 0x7fade41338a0 2023-09-30T22:34:04.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.269+0000 7fadebd33700 1 -- 172.21.15.16:0/14004404 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fade40a0f90 con 0x7fade41338a0 2023-09-30T22:34:04.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.270+0000 7fadebd33700 1 -- 172.21.15.16:0/14004404 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fade4074ea0 con 0x7fade41338a0 2023-09-30T22:34:04.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.271+0000 7fadda7fc700 1 -- 172.21.15.16:0/14004404 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7faddc014480 con 0x7fade41338a0 2023-09-30T22:34:04.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.271+0000 7fadda7fc700 1 -- 172.21.15.16:0/14004404 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7faddc03fdd0 con 0x7fade41338a0 2023-09-30T22:34:04.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.280+0000 7fadda7fc700 1 -- 172.21.15.16:0/14004404 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7faddc01a8b0 con 0x7fade41338a0 2023-09-30T22:34:04.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.299+0000 7f6d9cdf2700 1 Processor -- start 2023-09-30T22:34:04.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.299+0000 7f6d9cdf2700 1 -- start start 2023-09-30T22:34:04.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.299+0000 7f2ccec5f700 1 Processor -- start 2023-09-30T22:34:04.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.300+0000 7f6d9cdf2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d980a0a30 0x7f6d980a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:04.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.300+0000 7f6d9cdf2700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6d9809d630 0x7f6d980a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:04.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.300+0000 7f6d9cdf2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d98133930 0x7f6d98133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:04.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.300+0000 7f2ccec5f700 1 -- start start 2023-09-30T22:34:04.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.300+0000 7f2ccec5f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2cc80a1510 0x7f2cc8138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:04.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.300+0000 7f2ccec5f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2cc8139250 0x7f2cc813b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:04.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.300+0000 7f2ccec5f700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cc809edd0 con 0x7f2cc80a0a30 2023-09-30T22:34:04.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.300+0000 7f2ccec5f700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cc80a18e0 con 0x7f2cc80a1510 2023-09-30T22:34:04.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.300+0000 7f2ccec5f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc809e1f0 con 0x7f2cc8139250 2023-09-30T22:34:04.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.300+0000 7f6d9cdf2700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d980a2a70 con 0x7f6d9809d630 2023-09-30T22:34:04.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.300+0000 7f6d9cdf2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d980a01d0 con 0x7f6d980a0a30 2023-09-30T22:34:04.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.300+0000 7f6d9cdf2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9809e2b0 con 0x7f6d98133930 2023-09-30T22:34:04.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.300+0000 7f6d9659c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d980a0a30 0x7f6d980a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:04.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.300+0000 7f6d9659c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d980a0a30 0x7f6d980a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38696/0 (socket says 172.21.15.16:38696) 2023-09-30T22:34:04.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.300+0000 7f6d9659c700 1 -- 172.21.15.16:0/3079430137 learned_addr learned my addr 172.21.15.16:0/3079430137 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:04.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.300+0000 7f6d95d9b700 1 --2- 172.21.15.16:0/3079430137 >> v2:172.21.15.16:3300/0 conn(0x7f6d9809d630 0x7f6d980a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:04.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.300+0000 7f2cc7fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2cc80a1510 0x7f2cc8138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:04.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.301+0000 7f2cc7fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2cc80a1510 0x7f2cc8138d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37354/0 (socket says 172.21.15.16:37354) 2023-09-30T22:34:04.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.301+0000 7f2cc7fff700 1 -- 172.21.15.16:0/3434075337 learned_addr learned my addr 172.21.15.16:0/3434075337 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:04.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.301+0000 7f6d9659c700 1 -- 172.21.15.16:0/3079430137 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d98133930 msgr2=0x7f6d98133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:04.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.301+0000 7f6d9659c700 1 --2- 172.21.15.16:0/3079430137 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d98133930 0x7f6d98133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:04.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.301+0000 7f6d9659c700 1 -- 172.21.15.16:0/3079430137 >> v2:172.21.15.16:3300/0 conn(0x7f6d9809d630 msgr2=0x7f6d980a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:04.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.301+0000 7f6d9659c700 1 --2- 172.21.15.16:0/3079430137 >> v2:172.21.15.16:3300/0 conn(0x7f6d9809d630 0x7f6d980a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:04.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.301+0000 7f6d9659c700 1 -- 172.21.15.16:0/3079430137 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d9809dc50 con 0x7f6d980a0a30 2023-09-30T22:34:04.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.301+0000 7f2cc7fff700 1 -- 172.21.15.16:0/3434075337 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2cc8139250 msgr2=0x7f2cc813b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:04.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.301+0000 7f2cc7fff700 1 --2- 172.21.15.16:0/3434075337 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2cc8139250 0x7f2cc813b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:04.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.301+0000 7f2cc7fff700 1 -- 172.21.15.16:0/3434075337 >> v1:172.21.15.16:6789/0 conn(0x7f2cc80a0a30 legacy=0x7f2cc80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:04.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.301+0000 7f2cc7fff700 1 -- 172.21.15.16:0/3434075337 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cc809dc50 con 0x7f2cc80a1510 2023-09-30T22:34:04.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.301+0000 7f6d9659c700 1 --2- 172.21.15.16:0/3079430137 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d980a0a30 0x7f6d980a0e00 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f6d880098f0 tx=0x7f6d88009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7fabc83dcf48201 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:04.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.301+0000 7f2cc7fff700 1 --2- 172.21.15.16:0/3434075337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cc80a1510 0x7f2cc8138d10 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f2cb40098f0 tx=0x7f2cb4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b17a147cc5e8f62 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:04.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.302+0000 7f2cc77fe700 1 -- 172.21.15.16:0/3434075337 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb4004230 con 0x7f2cc80a1510 2023-09-30T22:34:04.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.302+0000 7f2cc77fe700 1 -- 172.21.15.16:0/3434075337 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cb4004390 con 0x7f2cc80a1510 2023-09-30T22:34:04.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.302+0000 7f2cc77fe700 1 -- 172.21.15.16:0/3434075337 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb40036a0 con 0x7f2cc80a1510 2023-09-30T22:34:04.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.303+0000 7f2ccec5f700 1 -- 172.21.15.16:0/3434075337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cc80a1510 msgr2=0x7f2cc8138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:04.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.303+0000 7f2ccec5f700 1 --2- 172.21.15.16:0/3434075337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cc80a1510 0x7f2cc8138d10 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f2cb40098f0 tx=0x7f2cb4009c00 comp rx=0 tx=0).stop 2023-09-30T22:34:04.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.303+0000 7f6d9559a700 1 -- 172.21.15.16:0/3079430137 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d88011830 con 0x7f6d980a0a30 2023-09-30T22:34:04.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.303+0000 7f6d9559a700 1 -- 172.21.15.16:0/3079430137 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d88011990 con 0x7f6d980a0a30 2023-09-30T22:34:04.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.303+0000 7f6d9559a700 1 -- 172.21.15.16:0/3079430137 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d88011d20 con 0x7f6d980a0a30 2023-09-30T22:34:04.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.303+0000 7f6d9cdf2700 1 -- 172.21.15.16:0/3079430137 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d980a0a30 msgr2=0x7f6d980a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:04.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.303+0000 7f6d9cdf2700 1 --2- 172.21.15.16:0/3079430137 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d980a0a30 0x7f6d980a0e00 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f6d880098f0 tx=0x7f6d88009c00 comp rx=0 tx=0).stop 2023-09-30T22:34:04.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.303+0000 7f2ccec5f700 1 -- 172.21.15.16:0/3434075337 shutdown_connections 2023-09-30T22:34:04.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.303+0000 7f2ccec5f700 1 --2- 172.21.15.16:0/3434075337 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2cc8139250 0x7f2cc813b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:04.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.303+0000 7f2ccec5f700 1 --2- 172.21.15.16:0/3434075337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cc80a1510 0x7f2cc8138d10 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:04.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.303+0000 7f2ccec5f700 1 -- 172.21.15.16:0/3434075337 >> 172.21.15.16:0/3434075337 conn(0x7f2cc80a6550 msgr2=0x7f2cc80a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:04.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.304+0000 7f2ccec5f700 1 -- 172.21.15.16:0/3434075337 shutdown_connections 2023-09-30T22:34:04.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.304+0000 7f2ccec5f700 1 -- 172.21.15.16:0/3434075337 wait complete. 2023-09-30T22:34:04.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.304+0000 7f6d9cdf2700 1 -- 172.21.15.16:0/3079430137 shutdown_connections 2023-09-30T22:34:04.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.304+0000 7f6d9cdf2700 1 --2- 172.21.15.16:0/3079430137 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d980a0a30 0x7f6d980a0e00 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:04.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.304+0000 7f6d9cdf2700 1 --2- 172.21.15.16:0/3079430137 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d98133930 0x7f6d98133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:04.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.304+0000 7f6d9cdf2700 1 --2- 172.21.15.16:0/3079430137 >> v2:172.21.15.16:3300/0 conn(0x7f6d9809d630 0x7f6d980a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:04.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.304+0000 7f6d9cdf2700 1 -- 172.21.15.16:0/3079430137 >> 172.21.15.16:0/3079430137 conn(0x7f6d980a3050 msgr2=0x7f6d980a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:04.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.305+0000 7f2ccec5f700 1 Processor -- start 2023-09-30T22:34:04.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.305+0000 7f2ccec5f700 1 -- start start 2023-09-30T22:34:04.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.305+0000 7f2ccec5f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2cc80a0a30 0x7f2cc81c8880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:04.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.305+0000 7f2ccec5f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cc80a1510 0x7f2cc81c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:04.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.305+0000 7f2ccec5f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2cc8139250 0x7f2cc81217c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:04.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.305+0000 7f2ccec5f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc81c9300 con 0x7f2cc80a1510 2023-09-30T22:34:04.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.305+0000 7f2ccec5f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc8124dd0 con 0x7f2cc8139250 2023-09-30T22:34:04.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.305+0000 7f2ccec5f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc81be3d0 con 0x7f2cc80a0a30 2023-09-30T22:34:04.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.306+0000 7f6d9cdf2700 1 -- 172.21.15.16:0/3079430137 shutdown_connections 2023-09-30T22:34:04.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.305+0000 7f2ccc9fb700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2cc80a0a30 0x7f2cc81c8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:04.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.306+0000 7f2ccc9fb700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2cc80a0a30 0x7f2cc81c8880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:57534/0 (socket says 172.21.15.16:57534) 2023-09-30T22:34:04.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.306+0000 7f2ccc9fb700 1 -- 172.21.15.16:0/3499074128 learned_addr learned my addr 172.21.15.16:0/3499074128 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:04.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.306+0000 7f2ccd1fc700 1 --2- 172.21.15.16:0/3499074128 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2cc8139250 0x7f2cc81217c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:04.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.306+0000 7f2cc7fff700 1 --2- 172.21.15.16:0/3499074128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cc80a1510 0x7f2cc81c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:04.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.306+0000 7f2ccd1fc700 1 -- 172.21.15.16:0/3499074128 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2cc80a0a30 msgr2=0x7f2cc81c8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:04.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.306+0000 7f2ccd1fc700 1 --2- 172.21.15.16:0/3499074128 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2cc80a0a30 0x7f2cc81c8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:04.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.306+0000 7f2ccd1fc700 1 -- 172.21.15.16:0/3499074128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cc80a1510 msgr2=0x7f2cc81c8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:04.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.306+0000 7f2ccd1fc700 1 --2- 172.21.15.16:0/3499074128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cc80a1510 0x7f2cc81c8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:04.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.306+0000 7f2ccd1fc700 1 -- 172.21.15.16:0/3499074128 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cb4009550 con 0x7f2cc8139250 2023-09-30T22:34:04.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.306+0000 7f2ccc9fb700 1 --2- 172.21.15.16:0/3499074128 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2cc80a0a30 0x7f2cc81c8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:04.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.306+0000 7f2ccd1fc700 1 --2- 172.21.15.16:0/3499074128 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2cc8139250 0x7f2cc81217c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f2cc000bdf0 tx=0x7f2cc0009ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:04.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.306+0000 7f6d9cdf2700 1 -- 172.21.15.16:0/3079430137 wait complete. 2023-09-30T22:34:04.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.307+0000 7f2cc57fa700 1 -- 172.21.15.16:0/3499074128 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc000cc10 con 0x7f2cc8139250 2023-09-30T22:34:04.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.307+0000 7f2cc57fa700 1 -- 172.21.15.16:0/3499074128 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cc000cd90 con 0x7f2cc8139250 2023-09-30T22:34:04.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.307+0000 7f2ccec5f700 1 -- 172.21.15.16:0/3499074128 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cc80a0220 con 0x7f2cc8139250 2023-09-30T22:34:04.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.307+0000 7f2cc57fa700 1 -- 172.21.15.16:0/3499074128 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0014680 con 0x7f2cc8139250 2023-09-30T22:34:04.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.307+0000 7f2ccec5f700 1 -- 172.21.15.16:0/3499074128 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cc81252e0 con 0x7f2cc8139250 2023-09-30T22:34:04.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.308+0000 7f6d9cdf2700 1 Processor -- start 2023-09-30T22:34:04.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.308+0000 7f6d9cdf2700 1 -- start start 2023-09-30T22:34:04.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.308+0000 7f6d9cdf2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d9809d630 0x7f6d98070820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:04.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.309+0000 7f6d9cdf2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d980a0a30 0x7f6d98070d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:04.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.308+0000 7f2cc57fa700 1 -- 172.21.15.16:0/3499074128 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2cc001c360 con 0x7f2cc8139250 2023-09-30T22:34:04.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.308+0000 7f2cc57fa700 1 -- 172.21.15.16:0/3499074128 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f2cc0006e40 con 0x7f2cc8139250 2023-09-30T22:34:04.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.309+0000 7f6d9cdf2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d98133930 0x7f6d981cc400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:04.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.309+0000 7f6d9cdf2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d980a4230 con 0x7f6d98133930 2023-09-30T22:34:04.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.309+0000 7f6d9cdf2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9811c7c0 con 0x7f6d980a0a30 2023-09-30T22:34:04.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.309+0000 7f6d9cdf2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9811e030 con 0x7f6d9809d630 2023-09-30T22:34:04.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.308+0000 7f2ccec5f700 1 -- 172.21.15.16:0/3499074128 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cc8124f10 con 0x7f2cc8139250 2023-09-30T22:34:04.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.309+0000 7f6d96d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d98133930 0x7f6d981cc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:04.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.309+0000 7f6d9659c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d9809d630 0x7f6d98070820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:04.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.309+0000 7f6d96d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d98133930 0x7f6d981cc400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37368/0 (socket says 172.21.15.16:37368) 2023-09-30T22:34:04.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.309+0000 7f6d96d9d700 1 -- 172.21.15.16:0/3817617006 learned_addr learned my addr 172.21.15.16:0/3817617006 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:04.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.309+0000 7f6d95d9b700 1 --2- 172.21.15.16:0/3817617006 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d980a0a30 0x7f6d98070d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:04.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.309+0000 7f6d96d9d700 1 -- 172.21.15.16:0/3817617006 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d9809d630 msgr2=0x7f6d98070820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:04.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.309+0000 7f6d96d9d700 1 --2- 172.21.15.16:0/3817617006 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d9809d630 0x7f6d98070820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:04.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.309+0000 7f6d96d9d700 1 -- 172.21.15.16:0/3817617006 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d980a0a30 msgr2=0x7f6d98070d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:04.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.309+0000 7f6d96d9d700 1 --2- 172.21.15.16:0/3817617006 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d980a0a30 0x7f6d98070d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:04.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.309+0000 7f6d96d9d700 1 -- 172.21.15.16:0/3817617006 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d88009550 con 0x7f6d98133930 2023-09-30T22:34:04.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.309+0000 7f6d96d9d700 1 --2- 172.21.15.16:0/3817617006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d98133930 0x7f6d981cc400 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f6d9000b970 tx=0x7f6d9000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:04.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.310+0000 7f6d86ffd700 1 -- 172.21.15.16:0/3817617006 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9000c930 con 0x7f6d98133930 2023-09-30T22:34:04.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.310+0000 7f6d86ffd700 1 -- 172.21.15.16:0/3817617006 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d9000cab0 con 0x7f6d98133930 2023-09-30T22:34:04.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.310+0000 7f6d86ffd700 1 -- 172.21.15.16:0/3817617006 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9000ce60 con 0x7f6d98133930 2023-09-30T22:34:04.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.310+0000 7f6d9cdf2700 1 -- 172.21.15.16:0/3817617006 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d980702a0 con 0x7f6d98133930 2023-09-30T22:34:04.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.310+0000 7f6d9cdf2700 1 -- 172.21.15.16:0/3817617006 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d9811ead0 con 0x7f6d98133930 2023-09-30T22:34:04.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.311+0000 7f6d9cdf2700 1 -- 172.21.15.16:0/3817617006 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d980747f0 con 0x7f6d98133930 2023-09-30T22:34:04.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.311+0000 7f6d86ffd700 1 -- 172.21.15.16:0/3817617006 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6d9001d3b0 con 0x7f6d98133930 2023-09-30T22:34:04.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.312+0000 7f6d86ffd700 1 -- 172.21.15.16:0/3817617006 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f6d90006e40 con 0x7f6d98133930 2023-09-30T22:34:04.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.314+0000 7f2c1f72a700 1 Processor -- start 2023-09-30T22:34:04.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.314+0000 7f2c1f72a700 1 -- start start 2023-09-30T22:34:04.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.314+0000 7f2c1f72a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c1809e930 0x7f2c1809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:04.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.314+0000 7f2c1f72a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2c1809f240 0x7f2c181317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:04.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.314+0000 7f2c1f72a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c18070940 con 0x7f2c1809f650 2023-09-30T22:34:04.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.314+0000 7f2c1f72a700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c1806fd60 con 0x7f2c1809f240 2023-09-30T22:34:04.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.314+0000 7f2c1f72a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2c1806fa80 con 0x7f2c1809e930 2023-09-30T22:34:04.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.315+0000 7f2c1ccc5700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2c1809f240 0x7f2c181317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:04.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.315+0000 7f2c1ccc5700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2c1809f240 0x7f2c181317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37380/0 (socket says 172.21.15.16:37380) 2023-09-30T22:34:04.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.315+0000 7f2c1ccc5700 1 -- 172.21.15.16:0/2130814583 learned_addr learned my addr 172.21.15.16:0/2130814583 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:04.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.315+0000 7f2c1dcc7700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2c1809f650 0x7f2c18131d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47498/0 (socket says 172.21.15.16:47498) 2023-09-30T22:34:04.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.315+0000 7f2c1d4c6700 1 --2- 172.21.15.16:0/2130814583 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c1809e930 0x7f2c1809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:04.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.315+0000 7f2c1ccc5700 1 -- 172.21.15.16:0/2130814583 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c1809e930 msgr2=0x7f2c1809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:04.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.315+0000 7f2c1ccc5700 1 --2- 172.21.15.16:0/2130814583 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c1809e930 0x7f2c1809ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:04.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.315+0000 7f2c1ccc5700 1 -- 172.21.15.16:0/2130814583 >> v1:172.21.15.16:6789/0 conn(0x7f2c1809f650 legacy=0x7f2c18131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:04.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.315+0000 7f2c1ccc5700 1 -- 172.21.15.16:0/2130814583 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c1809fb60 con 0x7f2c1809f240 2023-09-30T22:34:04.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.315+0000 7f2c1d4c6700 1 --2- 172.21.15.16:0/2130814583 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c1809e930 0x7f2c1809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:04.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.315+0000 7f2c1ccc5700 1 --2- 172.21.15.16:0/2130814583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c1809f240 0x7f2c181317f0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f2c080098f0 tx=0x7f2c08009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff3cea5a738be59b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:04.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.316+0000 7f2cc57fa700 1 -- 172.21.15.16:0/3499074128 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2cc0019070 con 0x7f2cc8139250 2023-09-30T22:34:04.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.318+0000 7f6d86ffd700 1 -- 172.21.15.16:0/3817617006 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6d90007370 con 0x7f6d98133930 2023-09-30T22:34:04.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.320+0000 7f2c0ffff700 1 -- 172.21.15.16:0/2130814583 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c08011830 con 0x7f2c1809f240 2023-09-30T22:34:04.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.320+0000 7f2c0ffff700 1 -- 172.21.15.16:0/2130814583 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c08011990 con 0x7f2c1809f240 2023-09-30T22:34:04.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.320+0000 7f2c0ffff700 1 -- 172.21.15.16:0/2130814583 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c08011d20 con 0x7f2c1809f240 2023-09-30T22:34:04.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.320+0000 7f2c1f72a700 1 -- 172.21.15.16:0/2130814583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c1809f240 msgr2=0x7f2c181317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:04.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.320+0000 7f2c1f72a700 1 --2- 172.21.15.16:0/2130814583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c1809f240 0x7f2c181317f0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f2c080098f0 tx=0x7f2c08009c00 comp rx=0 tx=0).stop 2023-09-30T22:34:04.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.320+0000 7f2c1f72a700 1 -- 172.21.15.16:0/2130814583 shutdown_connections 2023-09-30T22:34:04.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.320+0000 7f2c1f72a700 1 --2- 172.21.15.16:0/2130814583 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c1809e930 0x7f2c1809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:04.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.320+0000 7f2c1f72a700 1 --2- 172.21.15.16:0/2130814583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c1809f240 0x7f2c181317f0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:04.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.320+0000 7f2c1f72a700 1 -- 172.21.15.16:0/2130814583 >> 172.21.15.16:0/2130814583 conn(0x7f2c180a4210 msgr2=0x7f2c180a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:04.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.321+0000 7f2c1f72a700 1 -- 172.21.15.16:0/2130814583 shutdown_connections 2023-09-30T22:34:04.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.321+0000 7f2c1f72a700 1 -- 172.21.15.16:0/2130814583 wait complete. 2023-09-30T22:34:04.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.322+0000 7f2c1f72a700 1 Processor -- start 2023-09-30T22:34:04.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.322+0000 7f2c1f72a700 1 -- start start 2023-09-30T22:34:04.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.323+0000 7f2c1f72a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c1809e930 0x7f2c181d53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:04.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.323+0000 7f2c1f72a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c1809f240 0x7f2c181d5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:04.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.323+0000 7f2c1f72a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c1809f650 0x7f2c181d16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:04.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.323+0000 7f2c1f72a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c181305a0 con 0x7f2c1809e930 2023-09-30T22:34:04.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.323+0000 7f2c1f72a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c181b76f0 con 0x7f2c1809f240 2023-09-30T22:34:04.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.323+0000 7f2c1f72a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2c18071d20 con 0x7f2c1809f650 2023-09-30T22:34:04.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.323+0000 7f2c1ccc5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c1809f240 0x7f2c181d5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:04.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.323+0000 7f2c1ccc5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c1809f240 0x7f2c181d5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38722/0 (socket says 172.21.15.16:38722) 2023-09-30T22:34:04.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.323+0000 7f2c1ccc5700 1 -- 172.21.15.16:0/3772997525 learned_addr learned my addr 172.21.15.16:0/3772997525 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:04.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.323+0000 7f2c1dcc7700 1 --2- 172.21.15.16:0/3772997525 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c1809f650 0x7f2c181d16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:04.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.323+0000 7f2c1d4c6700 1 --2- 172.21.15.16:0/3772997525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c1809e930 0x7f2c181d53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:04.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.324+0000 7f2c1ccc5700 1 -- 172.21.15.16:0/3772997525 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c1809f650 msgr2=0x7f2c181d16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:04.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.324+0000 7f2c1ccc5700 1 --2- 172.21.15.16:0/3772997525 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c1809f650 0x7f2c181d16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:04.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.324+0000 7f2c1ccc5700 1 -- 172.21.15.16:0/3772997525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c1809e930 msgr2=0x7f2c181d53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:04.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.324+0000 7f2c1ccc5700 1 --2- 172.21.15.16:0/3772997525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c1809e930 0x7f2c181d53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:04.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.324+0000 7f2c1ccc5700 1 -- 172.21.15.16:0/3772997525 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c140094a0 con 0x7f2c1809f240 2023-09-30T22:34:04.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.324+0000 7f2c1ccc5700 1 --2- 172.21.15.16:0/3772997525 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c1809f240 0x7f2c181d5910 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f2c180a0b00 tx=0x7f2c08004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:04.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.324+0000 7f2c0dffb700 1 -- 172.21.15.16:0/3772997525 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c08004450 con 0x7f2c1809f240 2023-09-30T22:34:04.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.324+0000 7f2c0dffb700 1 -- 172.21.15.16:0/3772997525 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c080045d0 con 0x7f2c1809f240 2023-09-30T22:34:04.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.324+0000 7f2c0dffb700 1 -- 172.21.15.16:0/3772997525 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c080038e0 con 0x7f2c1809f240 2023-09-30T22:34:04.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.325+0000 7f2c1f72a700 1 -- 172.21.15.16:0/3772997525 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c08009550 con 0x7f2c1809f240 2023-09-30T22:34:04.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.325+0000 7f2c1f72a700 1 -- 172.21.15.16:0/3772997525 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c180a0190 con 0x7f2c1809f240 2023-09-30T22:34:04.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.326+0000 7f2c0dffb700 1 -- 172.21.15.16:0/3772997525 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2c08003a40 con 0x7f2c1809f240 2023-09-30T22:34:04.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.326+0000 7f2c0dffb700 1 -- 172.21.15.16:0/3772997525 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f2c080409e0 con 0x7f2c1809f240 2023-09-30T22:34:04.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.327+0000 7f2c1f72a700 1 -- 172.21.15.16:0/3772997525 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c18074ea0 con 0x7f2c1809f240 2023-09-30T22:34:04.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.332+0000 7f2c0dffb700 1 -- 172.21.15.16:0/3772997525 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2c0801a420 con 0x7f2c1809f240 2023-09-30T22:34:04.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.505+0000 7fadebd33700 1 -- 172.21.15.16:0/14004404 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fade406b9e0 con 0x7fade41338a0 2023-09-30T22:34:04.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.505+0000 7fadda7fc700 1 -- 172.21.15.16:0/14004404 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7faddc018020 con 0x7fade41338a0 2023-09-30T22:34:04.506 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:34:04.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.508+0000 7fadebd33700 1 -- 172.21.15.16:0/14004404 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fade41338a0 msgr2=0x7fade411c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:04.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.508+0000 7fadebd33700 1 --2- 172.21.15.16:0/14004404 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fade41338a0 0x7fade411c1b0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7faddc009dd0 tx=0x7faddc00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:34:04.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.508+0000 7fadebd33700 1 -- 172.21.15.16:0/14004404 shutdown_connections 2023-09-30T22:34:04.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.508+0000 7fadebd33700 1 --2- 172.21.15.16:0/14004404 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fade41338a0 0x7fade411c1b0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:04.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.508+0000 7fadebd33700 1 --2- 172.21.15.16:0/14004404 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fade409de50 0x7fade411b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:04.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.508+0000 7fadebd33700 1 --2- 172.21.15.16:0/14004404 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fade409e760 0x7fade411bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:04.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.508+0000 7fadebd33700 1 -- 172.21.15.16:0/14004404 >> 172.21.15.16:0/14004404 conn(0x7fade40a2aa0 msgr2=0x7fade41306c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:04.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.508+0000 7fadebd33700 1 -- 172.21.15.16:0/14004404 shutdown_connections 2023-09-30T22:34:04.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.508+0000 7fadebd33700 1 -- 172.21.15.16:0/14004404 wait complete. 2023-09-30T22:34:04.527 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:34:04.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.534+0000 7f6d9cdf2700 1 -- 172.21.15.16:0/3817617006 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6d98072980 con 0x7f6d98133930 2023-09-30T22:34:04.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.534+0000 7f6d86ffd700 1 -- 172.21.15.16:0/3817617006 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6d90019030 con 0x7f6d98133930 2023-09-30T22:34:04.535 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:34:04.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.537+0000 7f6d9cdf2700 1 -- 172.21.15.16:0/3817617006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d98133930 msgr2=0x7f6d981cc400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:04.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.537+0000 7f6d9cdf2700 1 --2- 172.21.15.16:0/3817617006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d98133930 0x7f6d981cc400 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f6d9000b970 tx=0x7f6d9000bc80 comp rx=0 tx=0).stop 2023-09-30T22:34:04.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.537+0000 7f6d9cdf2700 1 -- 172.21.15.16:0/3817617006 shutdown_connections 2023-09-30T22:34:04.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.537+0000 7f6d9cdf2700 1 --2- 172.21.15.16:0/3817617006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d98133930 0x7f6d981cc400 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:04.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.537+0000 7f6d9cdf2700 1 --2- 172.21.15.16:0/3817617006 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d9809d630 0x7f6d98070820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:04.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.537+0000 7f6d9cdf2700 1 --2- 172.21.15.16:0/3817617006 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d980a0a30 0x7f6d98070d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:04.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.537+0000 7f6d9cdf2700 1 -- 172.21.15.16:0/3817617006 >> 172.21.15.16:0/3817617006 conn(0x7f6d980a3050 msgr2=0x7f6d9809ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:04.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.537+0000 7f6d9cdf2700 1 -- 172.21.15.16:0/3817617006 shutdown_connections 2023-09-30T22:34:04.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.537+0000 7f6d9cdf2700 1 -- 172.21.15.16:0/3817617006 wait complete. 2023-09-30T22:34:04.551 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:34:04.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.562+0000 7f2ccec5f700 1 -- 172.21.15.16:0/3499074128 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f2cc8074ea0 con 0x7f2cc8139250 2023-09-30T22:34:04.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.562+0000 7f2cc57fa700 1 -- 172.21.15.16:0/3499074128 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2cc001d3b0 con 0x7f2cc8139250 2023-09-30T22:34:04.563 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:34:04.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.565+0000 7f2ccec5f700 1 -- 172.21.15.16:0/3499074128 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2cc8139250 msgr2=0x7f2cc81217c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:04.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.565+0000 7f2ccec5f700 1 --2- 172.21.15.16:0/3499074128 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2cc8139250 0x7f2cc81217c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f2cc000bdf0 tx=0x7f2cc0009ed0 comp rx=0 tx=0).stop 2023-09-30T22:34:04.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.565+0000 7f2ccec5f700 1 -- 172.21.15.16:0/3499074128 shutdown_connections 2023-09-30T22:34:04.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.565+0000 7f2ccec5f700 1 --2- 172.21.15.16:0/3499074128 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2cc8139250 0x7f2cc81217c0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:04.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.565+0000 7f2ccec5f700 1 --2- 172.21.15.16:0/3499074128 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2cc80a0a30 0x7f2cc81c8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:04.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.565+0000 7f2ccec5f700 1 --2- 172.21.15.16:0/3499074128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cc80a1510 0x7f2cc81c8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:04.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.565+0000 7f2ccec5f700 1 -- 172.21.15.16:0/3499074128 >> 172.21.15.16:0/3499074128 conn(0x7f2cc80a6550 msgr2=0x7f2cc8137b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:04.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.565+0000 7f2ccec5f700 1 -- 172.21.15.16:0/3499074128 shutdown_connections 2023-09-30T22:34:04.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.565+0000 7f2ccec5f700 1 -- 172.21.15.16:0/3499074128 wait complete. 2023-09-30T22:34:04.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.574+0000 7f2c1f72a700 1 -- 172.21.15.16:0/3772997525 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f2c1806b250 con 0x7f2c1809f240 2023-09-30T22:34:04.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.575+0000 7f2c0dffb700 1 -- 172.21.15.16:0/3772997525 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2c08036020 con 0x7f2c1809f240 2023-09-30T22:34:04.575 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:34:04.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.577+0000 7f2c1f72a700 1 -- 172.21.15.16:0/3772997525 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c1809f240 msgr2=0x7f2c181d5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:04.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.577+0000 7f2c1f72a700 1 --2- 172.21.15.16:0/3772997525 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c1809f240 0x7f2c181d5910 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f2c180a0b00 tx=0x7f2c08004750 comp rx=0 tx=0).stop 2023-09-30T22:34:04.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.577+0000 7f2c1f72a700 1 -- 172.21.15.16:0/3772997525 shutdown_connections 2023-09-30T22:34:04.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.577+0000 7f2c1f72a700 1 --2- 172.21.15.16:0/3772997525 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c1809f650 0x7f2c181d16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:04.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.577+0000 7f2c1f72a700 1 --2- 172.21.15.16:0/3772997525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c1809e930 0x7f2c181d53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:04.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.577+0000 7f2c1f72a700 1 --2- 172.21.15.16:0/3772997525 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c1809f240 0x7f2c181d5910 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:04.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.577+0000 7f2c1f72a700 1 -- 172.21.15.16:0/3772997525 >> 172.21.15.16:0/3772997525 conn(0x7f2c180a4210 msgr2=0x7f2c180a3770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:04.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.577+0000 7f2c1f72a700 1 -- 172.21.15.16:0/3772997525 shutdown_connections 2023-09-30T22:34:04.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:04.577+0000 7f2c1f72a700 1 -- 172.21.15.16:0/3772997525 wait complete. 2023-09-30T22:34:04.579 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:34:04.591 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:34:05.528 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:34:05.552 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:34:05.580 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:34:05.592 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:34:05.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.685+0000 7f97d6cee700 1 Processor -- start 2023-09-30T22:34:05.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.685+0000 7f97d6cee700 1 -- start start 2023-09-30T22:34:05.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.685+0000 7f97d6cee700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f97d0134630 0x7f97d0136a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:05.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.685+0000 7f97d6cee700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f97d0136f50 0x7f97d0139330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:05.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.685+0000 7f97d6cee700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97d009b4f0 con 0x7f97d0131810 2023-09-30T22:34:05.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.685+0000 7f97d6cee700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97d01313c0 con 0x7f97d0136f50 2023-09-30T22:34:05.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.685+0000 7f97d6cee700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f97d006e8c0 con 0x7f97d0134630 2023-09-30T22:34:05.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.686+0000 7f97d528b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f97d0136f50 0x7f97d0139330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:05.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.686+0000 7f97d528b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f97d0136f50 0x7f97d0139330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38734/0 (socket says 172.21.15.16:38734) 2023-09-30T22:34:05.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.686+0000 7f97d528b700 1 -- 172.21.15.16:0/1685317813 learned_addr learned my addr 172.21.15.16:0/1685317813 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:05.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.686+0000 7f97cffff700 1 --2- 172.21.15.16:0/1685317813 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f97d0134630 0x7f97d0136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:05.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.686+0000 7f97d4a8a700 1 --1- 172.21.15.16:0/1685317813 >> v1:172.21.15.16:6789/0 conn(0x7f97d0131810 0x7f97d0131be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47512/0 (socket says 172.21.15.16:47512) 2023-09-30T22:34:05.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.686+0000 7f97cffff700 1 -- 172.21.15.16:0/1685317813 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f97d0136f50 msgr2=0x7f97d0139330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:05.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.686+0000 7f97cffff700 1 --2- 172.21.15.16:0/1685317813 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f97d0136f50 0x7f97d0139330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:05.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.686+0000 7f97cffff700 1 -- 172.21.15.16:0/1685317813 >> v1:172.21.15.16:6789/0 conn(0x7f97d0131810 legacy=0x7f97d0131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:05.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.686+0000 7f97cffff700 1 -- 172.21.15.16:0/1685317813 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97d0071d20 con 0x7f97d0134630 2023-09-30T22:34:05.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.686+0000 7f97d528b700 1 --2- 172.21.15.16:0/1685317813 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f97d0136f50 0x7f97d0139330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:05.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.686+0000 7f97cffff700 1 --2- 172.21.15.16:0/1685317813 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f97d0134630 0x7f97d0136a10 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f97bc009870 tx=0x7f97bc009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12e8a4728369063 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:05.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.686+0000 7f97cf7fe700 1 -- 172.21.15.16:0/1685317813 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97bc011830 con 0x7f97d0134630 2023-09-30T22:34:05.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.687+0000 7f97cf7fe700 1 -- 172.21.15.16:0/1685317813 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97bc011990 con 0x7f97d0134630 2023-09-30T22:34:05.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.687+0000 7f97cf7fe700 1 -- 172.21.15.16:0/1685317813 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97bc011d20 con 0x7f97d0134630 2023-09-30T22:34:05.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.687+0000 7f97d6cee700 1 -- 172.21.15.16:0/1685317813 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f97d0134630 msgr2=0x7f97d0136a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:05.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.687+0000 7f97d6cee700 1 --2- 172.21.15.16:0/1685317813 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f97d0134630 0x7f97d0136a10 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f97bc009870 tx=0x7f97bc009b80 comp rx=0 tx=0).stop 2023-09-30T22:34:05.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.687+0000 7f97d6cee700 1 -- 172.21.15.16:0/1685317813 shutdown_connections 2023-09-30T22:34:05.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.687+0000 7f97d6cee700 1 --2- 172.21.15.16:0/1685317813 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f97d0136f50 0x7f97d0139330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:05.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.687+0000 7f97d6cee700 1 --2- 172.21.15.16:0/1685317813 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f97d0134630 0x7f97d0136a10 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:05.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.687+0000 7f97d6cee700 1 -- 172.21.15.16:0/1685317813 >> 172.21.15.16:0/1685317813 conn(0x7f97d00a2aa0 msgr2=0x7f97d00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:05.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.687+0000 7f97d6cee700 1 -- 172.21.15.16:0/1685317813 shutdown_connections 2023-09-30T22:34:05.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.687+0000 7f97d6cee700 1 -- 172.21.15.16:0/1685317813 wait complete. 2023-09-30T22:34:05.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.688+0000 7f97d6cee700 1 Processor -- start 2023-09-30T22:34:05.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.689+0000 7f97d6cee700 1 -- start start 2023-09-30T22:34:05.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.689+0000 7f97d6cee700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97d0131810 0x7f97d012dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:05.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.689+0000 7f97d6cee700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f97d0134630 0x7f97d012e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:05.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.689+0000 7f97d6cee700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f97d0136f50 0x7f97d01c4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:05.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.689+0000 7f97d6cee700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d009e450 con 0x7f97d0131810 2023-09-30T22:34:05.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.689+0000 7f97d6cee700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97d0136c60 con 0x7f97d0134630 2023-09-30T22:34:05.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.689+0000 7f97d6cee700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f97d01196c0 con 0x7f97d0136f50 2023-09-30T22:34:05.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.689+0000 7f97d528b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f97d0136f50 0x7f97d01c4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:05.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.689+0000 7f97d4a8a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97d0131810 0x7f97d012dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:05.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.689+0000 7f97d528b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f97d0136f50 0x7f97d01c4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:57574/0 (socket says 172.21.15.16:57574) 2023-09-30T22:34:05.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.689+0000 7f97d528b700 1 -- 172.21.15.16:0/3396638632 learned_addr learned my addr 172.21.15.16:0/3396638632 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:05.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.690+0000 7f97cffff700 1 --2- 172.21.15.16:0/3396638632 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f97d0134630 0x7f97d012e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:05.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.690+0000 7f97d528b700 1 -- 172.21.15.16:0/3396638632 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f97d0134630 msgr2=0x7f97d012e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:05.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.690+0000 7f97d528b700 1 --2- 172.21.15.16:0/3396638632 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f97d0134630 0x7f97d012e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:05.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.690+0000 7f97d528b700 1 -- 172.21.15.16:0/3396638632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97d0131810 msgr2=0x7f97d012dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:05.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.690+0000 7f97d528b700 1 --2- 172.21.15.16:0/3396638632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97d0131810 0x7f97d012dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:05.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.690+0000 7f97d528b700 1 -- 172.21.15.16:0/3396638632 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97c8009b30 con 0x7f97d0136f50 2023-09-30T22:34:05.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.690+0000 7f97d528b700 1 --2- 172.21.15.16:0/3396638632 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f97d0136f50 0x7f97d01c4120 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f97c800b370 tx=0x7f97c800b680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:05.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.690+0000 7f97cd7fa700 1 -- 172.21.15.16:0/3396638632 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c8015070 con 0x7f97d0136f50 2023-09-30T22:34:05.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.690+0000 7f97cd7fa700 1 -- 172.21.15.16:0/3396638632 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97c800bba0 con 0x7f97d0136f50 2023-09-30T22:34:05.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.691+0000 7f97cd7fa700 1 -- 172.21.15.16:0/3396638632 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c8003a20 con 0x7f97d0136f50 2023-09-30T22:34:05.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.691+0000 7f97d6cee700 1 -- 172.21.15.16:0/3396638632 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97bc0094a0 con 0x7f97d0136f50 2023-09-30T22:34:05.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.691+0000 7f97d6cee700 1 -- 172.21.15.16:0/3396638632 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97d006eb50 con 0x7f97d0136f50 2023-09-30T22:34:05.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.692+0000 7f97d6cee700 1 -- 172.21.15.16:0/3396638632 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97d00a2470 con 0x7f97d0136f50 2023-09-30T22:34:05.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.692+0000 7f97cd7fa700 1 -- 172.21.15.16:0/3396638632 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f97c80073f0 con 0x7f97d0136f50 2023-09-30T22:34:05.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.692+0000 7f97cd7fa700 1 -- 172.21.15.16:0/3396638632 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f97c80434f0 con 0x7f97d0136f50 2023-09-30T22:34:05.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.701+0000 7f97cd7fa700 1 -- 172.21.15.16:0/3396638632 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f97c80176e0 con 0x7f97d0136f50 2023-09-30T22:34:05.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.726+0000 7f841bb6b700 1 Processor -- start 2023-09-30T22:34:05.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.727+0000 7f841bb6b700 1 -- start start 2023-09-30T22:34:05.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.727+0000 7f841bb6b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f841409d6f0 0x7f841409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:05.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.727+0000 7f841bb6b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f841409e000 0x7f84141339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:05.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.727+0000 7f841bb6b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f841409b120 con 0x7f841409e410 2023-09-30T22:34:05.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.727+0000 7f841bb6b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f841406b460 con 0x7f841409e000 2023-09-30T22:34:05.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.727+0000 7f841bb6b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f841409a7e0 con 0x7f841409d6f0 2023-09-30T22:34:05.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.727+0000 7f8419907700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f841409d6f0 0x7f841409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:05.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.727+0000 7f8419907700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f841409d6f0 0x7f841409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:57576/0 (socket says 172.21.15.16:57576) 2023-09-30T22:34:05.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.727+0000 7f8419907700 1 -- 172.21.15.16:0/4144002047 learned_addr learned my addr 172.21.15.16:0/4144002047 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:05.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.727+0000 7f8419106700 1 --2- 172.21.15.16:0/4144002047 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f841409e000 0x7f84141339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:05.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.728+0000 7f8419907700 1 -- 172.21.15.16:0/4144002047 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f841409e000 msgr2=0x7f84141339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:05.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.728+0000 7f8419907700 1 --2- 172.21.15.16:0/4144002047 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f841409e000 0x7f84141339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:05.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.728+0000 7f8419907700 1 -- 172.21.15.16:0/4144002047 >> v1:172.21.15.16:6789/0 conn(0x7f841409e410 legacy=0x7f8414133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:05.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.728+0000 7f8419907700 1 -- 172.21.15.16:0/4144002047 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f841409a500 con 0x7f841409d6f0 2023-09-30T22:34:05.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.728+0000 7f8419907700 1 --2- 172.21.15.16:0/4144002047 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f841409d6f0 0x7f841409dac0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f8410009950 tx=0x7f8410009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27d7904ed3288e5b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:05.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.728+0000 7f8418905700 1 -- 172.21.15.16:0/4144002047 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8410009ea0 con 0x7f841409d6f0 2023-09-30T22:34:05.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.728+0000 7f8418905700 1 -- 172.21.15.16:0/4144002047 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f841000ba20 con 0x7f841409d6f0 2023-09-30T22:34:05.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.728+0000 7f8418905700 1 -- 172.21.15.16:0/4144002047 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841000bdb0 con 0x7f841409d6f0 2023-09-30T22:34:05.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.729+0000 7f841bb6b700 1 -- 172.21.15.16:0/4144002047 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f841409d6f0 msgr2=0x7f841409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:05.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.729+0000 7f841bb6b700 1 --2- 172.21.15.16:0/4144002047 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f841409d6f0 0x7f841409dac0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f8410009950 tx=0x7f8410009c60 comp rx=0 tx=0).stop 2023-09-30T22:34:05.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.729+0000 7f841bb6b700 1 -- 172.21.15.16:0/4144002047 shutdown_connections 2023-09-30T22:34:05.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.729+0000 7f841bb6b700 1 --2- 172.21.15.16:0/4144002047 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f841409d6f0 0x7f841409dac0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:05.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.729+0000 7f841bb6b700 1 --2- 172.21.15.16:0/4144002047 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f841409e000 0x7f84141339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:05.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.729+0000 7f841bb6b700 1 -- 172.21.15.16:0/4144002047 >> 172.21.15.16:0/4144002047 conn(0x7f8414066f50 msgr2=0x7f841409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:05.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.730+0000 7f841bb6b700 1 -- 172.21.15.16:0/4144002047 shutdown_connections 2023-09-30T22:34:05.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.730+0000 7f841bb6b700 1 -- 172.21.15.16:0/4144002047 wait complete. 2023-09-30T22:34:05.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.731+0000 7f841bb6b700 1 Processor -- start 2023-09-30T22:34:05.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.731+0000 7f841bb6b700 1 -- start start 2023-09-30T22:34:05.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.732+0000 7f841bb6b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f841409d6f0 0x7f84141a5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:05.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.732+0000 7f841bb6b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f841409e000 0x7f84141a63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:05.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.732+0000 7f841bb6b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f841409e410 0x7f84141a6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:05.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.732+0000 7f841bb6b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84141a93c0 con 0x7f841409e410 2023-09-30T22:34:05.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.732+0000 7f841bb6b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f841409c710 con 0x7f841409e000 2023-09-30T22:34:05.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.732+0000 7f841bb6b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f84141a8d10 con 0x7f841409d6f0 2023-09-30T22:34:05.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.732+0000 7f8419106700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f841409e000 0x7f84141a63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:05.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.732+0000 7f841a108700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f841409e410 0x7f84141a6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:05.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.732+0000 7f8419106700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f841409e000 0x7f84141a63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38774/0 (socket says 172.21.15.16:38774) 2023-09-30T22:34:05.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.732+0000 7f8419106700 1 -- 172.21.15.16:0/3411607806 learned_addr learned my addr 172.21.15.16:0/3411607806 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:05.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.732+0000 7f8419907700 1 --2- 172.21.15.16:0/3411607806 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f841409d6f0 0x7f84141a5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:05.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.733+0000 7f8419907700 1 -- 172.21.15.16:0/3411607806 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f841409e000 msgr2=0x7f84141a63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:05.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.733+0000 7f8419907700 1 --2- 172.21.15.16:0/3411607806 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f841409e000 0x7f84141a63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:05.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.733+0000 7f8419907700 1 -- 172.21.15.16:0/3411607806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f841409e410 msgr2=0x7f84141a6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:05.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.733+0000 7f8419907700 1 --2- 172.21.15.16:0/3411607806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f841409e410 0x7f84141a6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:05.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.733+0000 7f8419907700 1 -- 172.21.15.16:0/3411607806 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8410009550 con 0x7f841409d6f0 2023-09-30T22:34:05.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.733+0000 7f8419106700 1 --2- 172.21.15.16:0/3411607806 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f841409e000 0x7f84141a63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:05.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.733+0000 7f841a108700 1 --2- 172.21.15.16:0/3411607806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f841409e410 0x7f84141a6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:05.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.733+0000 7f8419907700 1 --2- 172.21.15.16:0/3411607806 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f841409d6f0 0x7f84141a5e80 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f8410016000 tx=0x7f8410004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:05.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.734+0000 7f840a7fc700 1 -- 172.21.15.16:0/3411607806 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841000be90 con 0x7f841409d6f0 2023-09-30T22:34:05.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.734+0000 7f840a7fc700 1 -- 172.21.15.16:0/3411607806 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f841001f070 con 0x7f841409d6f0 2023-09-30T22:34:05.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.734+0000 7f841bb6b700 1 -- 172.21.15.16:0/3411607806 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84141a7b60 con 0x7f841409d6f0 2023-09-30T22:34:05.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.734+0000 7f840a7fc700 1 -- 172.21.15.16:0/3411607806 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84100036a0 con 0x7f841409d6f0 2023-09-30T22:34:05.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.734+0000 7f841bb6b700 1 -- 172.21.15.16:0/3411607806 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84141a9960 con 0x7f841409d6f0 2023-09-30T22:34:05.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.735+0000 7f841bb6b700 1 -- 172.21.15.16:0/3411607806 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84141a9530 con 0x7f841409d6f0 2023-09-30T22:34:05.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.735+0000 7f840a7fc700 1 -- 172.21.15.16:0/3411607806 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8410021030 con 0x7f841409d6f0 2023-09-30T22:34:05.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.735+0000 7f840a7fc700 1 -- 172.21.15.16:0/3411607806 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f8410040be0 con 0x7f841409d6f0 2023-09-30T22:34:05.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.744+0000 7f840a7fc700 1 -- 172.21.15.16:0/3411607806 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f84100199a0 con 0x7f841409d6f0 2023-09-30T22:34:05.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.766+0000 7f867d4cf700 1 Processor -- start 2023-09-30T22:34:05.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.766+0000 7f867d4cf700 1 -- start start 2023-09-30T22:34:05.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.766+0000 7f867d4cf700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f86780a0a30 0x7f86780a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:05.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.766+0000 7f867d4cf700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f867809d630 0x7f86780a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:05.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.766+0000 7f867d4cf700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8678133930 0x7f8678133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:05.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.766+0000 7f867d4cf700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86780a2a70 con 0x7f8678133930 2023-09-30T22:34:05.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.766+0000 7f867d4cf700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86780a01d0 con 0x7f867809d630 2023-09-30T22:34:05.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.766+0000 7f867d4cf700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f867809e2b0 con 0x7f86780a0a30 2023-09-30T22:34:05.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.766+0000 7f86777fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8678133930 0x7f8678133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:05.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.767+0000 7f86777fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8678133930 0x7f8678133dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37408/0 (socket says 172.21.15.16:37408) 2023-09-30T22:34:05.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.767+0000 7f86777fe700 1 -- 172.21.15.16:0/4092026712 learned_addr learned my addr 172.21.15.16:0/4092026712 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:05.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.767+0000 7f86767fc700 1 --2- 172.21.15.16:0/4092026712 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f867809d630 0x7f86780a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:05.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.767+0000 7f8676ffd700 1 --2- 172.21.15.16:0/4092026712 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f86780a0a30 0x7f86780a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:05.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.767+0000 7f86777fe700 1 -- 172.21.15.16:0/4092026712 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f86780a0a30 msgr2=0x7f86780a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:05.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.767+0000 7f86777fe700 1 --2- 172.21.15.16:0/4092026712 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f86780a0a30 0x7f86780a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:05.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.767+0000 7f86777fe700 1 -- 172.21.15.16:0/4092026712 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f867809d630 msgr2=0x7f86780a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:05.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.767+0000 7f86777fe700 1 --2- 172.21.15.16:0/4092026712 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f867809d630 0x7f86780a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:05.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.767+0000 7f86777fe700 1 -- 172.21.15.16:0/4092026712 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f867809dc50 con 0x7f8678133930 2023-09-30T22:34:05.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.767+0000 7f86777fe700 1 --2- 172.21.15.16:0/4092026712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8678133930 0x7f8678133dd0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f8670009e50 tx=0x7f867000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2488542cd27bc322 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:05.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.767+0000 7f8675ffb700 1 -- 172.21.15.16:0/4092026712 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867000e070 con 0x7f8678133930 2023-09-30T22:34:05.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.767+0000 7f8675ffb700 1 -- 172.21.15.16:0/4092026712 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f867000b910 con 0x7f8678133930 2023-09-30T22:34:05.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.767+0000 7f8675ffb700 1 -- 172.21.15.16:0/4092026712 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867000bcc0 con 0x7f8678133930 2023-09-30T22:34:05.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.768+0000 7f867d4cf700 1 -- 172.21.15.16:0/4092026712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8678133930 msgr2=0x7f8678133dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:05.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.768+0000 7f867d4cf700 1 --2- 172.21.15.16:0/4092026712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8678133930 0x7f8678133dd0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f8670009e50 tx=0x7f867000b330 comp rx=0 tx=0).stop 2023-09-30T22:34:05.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.769+0000 7f867d4cf700 1 -- 172.21.15.16:0/4092026712 shutdown_connections 2023-09-30T22:34:05.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.769+0000 7f867d4cf700 1 --2- 172.21.15.16:0/4092026712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8678133930 0x7f8678133dd0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:05.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.769+0000 7f867d4cf700 1 --2- 172.21.15.16:0/4092026712 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f86780a0a30 0x7f86780a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:05.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.769+0000 7f867d4cf700 1 --2- 172.21.15.16:0/4092026712 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f867809d630 0x7f86780a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:05.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.769+0000 7f867d4cf700 1 -- 172.21.15.16:0/4092026712 >> 172.21.15.16:0/4092026712 conn(0x7f86780a3050 msgr2=0x7f86780a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:05.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.769+0000 7f867d4cf700 1 -- 172.21.15.16:0/4092026712 shutdown_connections 2023-09-30T22:34:05.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.769+0000 7f867d4cf700 1 -- 172.21.15.16:0/4092026712 wait complete. 2023-09-30T22:34:05.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.770+0000 7f867d4cf700 1 Processor -- start 2023-09-30T22:34:05.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.770+0000 7f867d4cf700 1 -- start start 2023-09-30T22:34:05.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.771+0000 7f867d4cf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f867809d630 0x7f8678126020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:05.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.771+0000 7f867d4cf700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f86780a0a30 0x7f8678126560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:05.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.771+0000 7f867d4cf700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8678133930 0x7f8678126aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:05.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.771+0000 7f867d4cf700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86780a50b0 con 0x7f867809d630 2023-09-30T22:34:05.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.771+0000 7f867d4cf700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8678129a90 con 0x7f86780a0a30 2023-09-30T22:34:05.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.771+0000 7f867d4cf700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8678127380 con 0x7f8678133930 2023-09-30T22:34:05.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.771+0000 7f8676ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f867809d630 0x7f8678126020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:05.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.771+0000 7f86777fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8678133930 0x7f8678126aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:05.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.771+0000 7f86767fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f86780a0a30 0x7f8678126560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:05.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.771+0000 7f8676ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f867809d630 0x7f8678126020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37422/0 (socket says 172.21.15.16:37422) 2023-09-30T22:34:05.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.771+0000 7f8676ffd700 1 -- 172.21.15.16:0/132622760 learned_addr learned my addr 172.21.15.16:0/132622760 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:05.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.771+0000 7f86777fe700 1 -- 172.21.15.16:0/132622760 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f86780a0a30 msgr2=0x7f8678126560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:05.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.771+0000 7f86777fe700 1 --2- 172.21.15.16:0/132622760 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f86780a0a30 0x7f8678126560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:05.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.771+0000 7f86777fe700 1 -- 172.21.15.16:0/132622760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f867809d630 msgr2=0x7f8678126020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:05.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.771+0000 7f86777fe700 1 --2- 172.21.15.16:0/132622760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f867809d630 0x7f8678126020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:05.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.771+0000 7f86777fe700 1 -- 172.21.15.16:0/132622760 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8670009b30 con 0x7f8678133930 2023-09-30T22:34:05.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.772+0000 7f8676ffd700 1 --2- 172.21.15.16:0/132622760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f867809d630 0x7f8678126020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:05.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.772+0000 7f86767fc700 1 --2- 172.21.15.16:0/132622760 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f86780a0a30 0x7f8678126560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:05.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.772+0000 7f86777fe700 1 --2- 172.21.15.16:0/132622760 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8678133930 0x7f8678126aa0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f8670015000 tx=0x7f8670004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:05.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.772+0000 7f865f7fe700 1 -- 172.21.15.16:0/132622760 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8670004450 con 0x7f8678133930 2023-09-30T22:34:05.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.772+0000 7f867d4cf700 1 -- 172.21.15.16:0/132622760 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86781283d0 con 0x7f8678133930 2023-09-30T22:34:05.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.772+0000 7f865f7fe700 1 -- 172.21.15.16:0/132622760 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86700092e0 con 0x7f8678133930 2023-09-30T22:34:05.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.772+0000 7f867d4cf700 1 -- 172.21.15.16:0/132622760 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8678128ff0 con 0x7f8678133930 2023-09-30T22:34:05.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.772+0000 7f865f7fe700 1 -- 172.21.15.16:0/132622760 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8670004810 con 0x7f8678133930 2023-09-30T22:34:05.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.773+0000 7f865f7fe700 1 -- 172.21.15.16:0/132622760 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f867001c410 con 0x7f8678133930 2023-09-30T22:34:05.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.773+0000 7f865f7fe700 1 -- 172.21.15.16:0/132622760 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f86700414c0 con 0x7f8678133930 2023-09-30T22:34:05.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.776+0000 7f867d4cf700 1 -- 172.21.15.16:0/132622760 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86580052e0 con 0x7f8678133930 2023-09-30T22:34:05.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.778+0000 7fe618afe700 1 Processor -- start 2023-09-30T22:34:05.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.778+0000 7fe618afe700 1 -- start start 2023-09-30T22:34:05.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.779+0000 7fe618afe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe61409d6f0 0x7fe61409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:05.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.779+0000 7fe618afe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe61409e000 0x7fe6141339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:05.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.779+0000 7fe618afe700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe61409b120 con 0x7fe61409e410 2023-09-30T22:34:05.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.779+0000 7fe618afe700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe61406b460 con 0x7fe61409d6f0 2023-09-30T22:34:05.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.779+0000 7fe618afe700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe61409a7e0 con 0x7fe61409e000 2023-09-30T22:34:05.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.779+0000 7fe612d9d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe61409e410 0x7fe614133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47516/0 (socket says 172.21.15.16:47516) 2023-09-30T22:34:05.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.779+0000 7fe612d9d700 1 -- 172.21.15.16:0/3715478498 learned_addr learned my addr 172.21.15.16:0/3715478498 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:34:05.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.779+0000 7fe61159a700 1 -- 172.21.15.16:0/3715478498 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2832346114 0 0) 0x7fe61409b120 con 0x7fe61409e410 2023-09-30T22:34:05.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.779+0000 7fe61159a700 1 -- 172.21.15.16:0/3715478498 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5fc0035b0 con 0x7fe61409e410 2023-09-30T22:34:05.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.780+0000 7fe61159a700 1 -- 172.21.15.16:0/3715478498 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 580121685 0 0) 0x7fe5fc0035b0 con 0x7fe61409e410 2023-09-30T22:34:05.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.780+0000 7fe61159a700 1 -- 172.21.15.16:0/3715478498 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe61409e000 msgr2=0x7fe6141339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:05.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.780+0000 7fe61159a700 1 --2- 172.21.15.16:0/3715478498 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe61409e000 0x7fe6141339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:05.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.780+0000 7fe61159a700 1 -- 172.21.15.16:0/3715478498 >> v2:172.21.15.16:3300/0 conn(0x7fe61409d6f0 msgr2=0x7fe61409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:05.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.780+0000 7fe61159a700 1 --2- 172.21.15.16:0/3715478498 >> v2:172.21.15.16:3300/0 conn(0x7fe61409d6f0 0x7fe61409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:05.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.780+0000 7fe61159a700 1 -- 172.21.15.16:0/3715478498 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe61409a500 con 0x7fe61409e410 2023-09-30T22:34:05.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.780+0000 7fe61159a700 1 -- 172.21.15.16:0/3715478498 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1074163509 0 0) 0x7fe60c0019f0 con 0x7fe61409e410 2023-09-30T22:34:05.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.780+0000 7fe61159a700 1 -- 172.21.15.16:0/3715478498 >> v1:172.21.15.16:6789/0 conn(0x7fe61409e410 legacy=0x7fe614133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:05.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.780+0000 7fe61159a700 1 --2- 172.21.15.16:0/3715478498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5fc003a30 0x7fe5fc003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:05.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.780+0000 7fe61159a700 1 --2- 172.21.15.16:0/3715478498 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe5fc004850 0x7fe5fc004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:05.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.780+0000 7fe61159a700 1 --2- 172.21.15.16:0/3715478498 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe61409e000 0x7fe5fc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:05.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.780+0000 7fe61159a700 1 -- 172.21.15.16:0/3715478498 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe61406b460 con 0x7fe5fc003a30 2023-09-30T22:34:05.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.780+0000 7fe61159a700 1 -- 172.21.15.16:0/3715478498 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe61409a7e0 con 0x7fe5fc004850 2023-09-30T22:34:05.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.780+0000 7fe61159a700 1 -- 172.21.15.16:0/3715478498 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe5fc005610 con 0x7fe61409e000 2023-09-30T22:34:05.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.780+0000 7fe611d9b700 1 --2- 172.21.15.16:0/3715478498 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe5fc004850 0x7fe5fc004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:05.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.780+0000 7fe612d9d700 1 --2- 172.21.15.16:0/3715478498 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe61409e000 0x7fe5fc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:05.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.780+0000 7fe61259c700 1 --2- 172.21.15.16:0/3715478498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5fc003a30 0x7fe5fc003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:05.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.780+0000 7fe61259c700 1 -- 172.21.15.16:0/3715478498 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe61409e000 msgr2=0x7fe5fc00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:05.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.780+0000 7fe61259c700 1 --2- 172.21.15.16:0/3715478498 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe61409e000 0x7fe5fc00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:05.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.780+0000 7fe61259c700 1 -- 172.21.15.16:0/3715478498 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe5fc004850 msgr2=0x7fe5fc004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:05.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.780+0000 7fe61259c700 1 --2- 172.21.15.16:0/3715478498 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe5fc004850 0x7fe5fc004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:05.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.781+0000 7fe61259c700 1 -- 172.21.15.16:0/3715478498 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5fc0035b0 con 0x7fe5fc003a30 2023-09-30T22:34:05.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.781+0000 7fe611d9b700 1 --2- 172.21.15.16:0/3715478498 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe5fc004850 0x7fe5fc004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:05.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.781+0000 7fe61259c700 1 --2- 172.21.15.16:0/3715478498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5fc003a30 0x7fe5fc003e40 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fe604009910 tx=0x7fe5fc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8788389be29126d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:05.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.781+0000 7fe61159a700 1 -- 172.21.15.16:0/3715478498 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe604011620 con 0x7fe5fc003a30 2023-09-30T22:34:05.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.781+0000 7fe61159a700 1 -- 172.21.15.16:0/3715478498 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe604011780 con 0x7fe5fc003a30 2023-09-30T22:34:05.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.781+0000 7fe61159a700 1 -- 172.21.15.16:0/3715478498 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe604011b10 con 0x7fe5fc003a30 2023-09-30T22:34:05.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.782+0000 7fe618afe700 1 -- 172.21.15.16:0/3715478498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5fc003a30 msgr2=0x7fe5fc003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:05.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.782+0000 7fe618afe700 1 --2- 172.21.15.16:0/3715478498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5fc003a30 0x7fe5fc003e40 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fe604009910 tx=0x7fe5fc005880 comp rx=0 tx=0).stop 2023-09-30T22:34:05.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.782+0000 7fe612d9d700 1 -- 172.21.15.16:0/3715478498 reap_dead start 2023-09-30T22:34:05.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.782+0000 7fe618afe700 1 -- 172.21.15.16:0/3715478498 shutdown_connections 2023-09-30T22:34:05.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.782+0000 7fe618afe700 1 -- 172.21.15.16:0/3715478498 >> 172.21.15.16:0/3715478498 conn(0x7fe614066f50 msgr2=0x7fe61409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:05.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.782+0000 7fe618afe700 1 -- 172.21.15.16:0/3715478498 shutdown_connections 2023-09-30T22:34:05.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.782+0000 7fe618afe700 1 -- 172.21.15.16:0/3715478498 wait complete. 2023-09-30T22:34:05.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.783+0000 7fe618afe700 1 Processor -- start 2023-09-30T22:34:05.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.784+0000 7fe618afe700 1 -- start start 2023-09-30T22:34:05.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.784+0000 7fe618afe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5fc003a30 0x7fe6141a5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:05.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.784+0000 7fe618afe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe6141a9f30 0x7fe6141a6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:05.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.784+0000 7fe618afe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe6141a6990 0x7fe6141cea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:05.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.784+0000 7fe618afe700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6141a6de0 con 0x7fe5fc003a30 2023-09-30T22:34:05.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.784+0000 7fe618afe700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6141a73c0 con 0x7fe6141a9f30 2023-09-30T22:34:05.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.784+0000 7fe618afe700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe6141a7b20 con 0x7fe6141a6990 2023-09-30T22:34:05.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.784+0000 7fe611d9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe6141a9f30 0x7fe6141a6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:05.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.784+0000 7fe611d9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe6141a9f30 0x7fe6141a6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38820/0 (socket says 172.21.15.16:38820) 2023-09-30T22:34:05.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.784+0000 7fe611d9b700 1 -- 172.21.15.16:0/1074435384 learned_addr learned my addr 172.21.15.16:0/1074435384 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:05.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.784+0000 7fe612d9d700 1 --2- 172.21.15.16:0/1074435384 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe6141a6990 0x7fe6141cea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:05.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.785+0000 7fe612d9d700 1 -- 172.21.15.16:0/1074435384 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe6141a9f30 msgr2=0x7fe6141a6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:05.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.785+0000 7fe612d9d700 1 --2- 172.21.15.16:0/1074435384 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe6141a9f30 0x7fe6141a6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:05.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.785+0000 7fe612d9d700 1 -- 172.21.15.16:0/1074435384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5fc003a30 msgr2=0x7fe6141a5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:05.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.785+0000 7fe612d9d700 1 --2- 172.21.15.16:0/1074435384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5fc003a30 0x7fe6141a5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:05.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.785+0000 7fe612d9d700 1 -- 172.21.15.16:0/1074435384 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6141a8d70 con 0x7fe6141a6990 2023-09-30T22:34:05.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.785+0000 7fe612d9d700 1 --2- 172.21.15.16:0/1074435384 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe6141a6990 0x7fe6141cea40 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fe61406b830 tx=0x7fe60c00a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:05.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.786+0000 7fe602ffd700 1 -- 172.21.15.16:0/1074435384 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe60c00ad40 con 0x7fe6141a6990 2023-09-30T22:34:05.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.786+0000 7fe618afe700 1 -- 172.21.15.16:0/1074435384 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6141a8100 con 0x7fe6141a6990 2023-09-30T22:34:05.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.786+0000 7f865f7fe700 1 -- 172.21.15.16:0/132622760 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f867002d7a0 con 0x7f8678133930 2023-09-30T22:34:05.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.786+0000 7fe618afe700 1 -- 172.21.15.16:0/1074435384 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6140a2090 con 0x7fe6141a6990 2023-09-30T22:34:05.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.786+0000 7fe602ffd700 1 -- 172.21.15.16:0/1074435384 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe60c01d460 con 0x7fe6141a6990 2023-09-30T22:34:05.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.786+0000 7fe602ffd700 1 -- 172.21.15.16:0/1074435384 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe60c00aea0 con 0x7fe6141a6990 2023-09-30T22:34:05.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.787+0000 7fe618afe700 1 -- 172.21.15.16:0/1074435384 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5f80052e0 con 0x7fe6141a6990 2023-09-30T22:34:05.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.790+0000 7fe602ffd700 1 -- 172.21.15.16:0/1074435384 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe60c01d7d0 con 0x7fe6141a6990 2023-09-30T22:34:05.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.790+0000 7fe602ffd700 1 -- 172.21.15.16:0/1074435384 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fe60c03fe50 con 0x7fe6141a6990 2023-09-30T22:34:05.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.797+0000 7fe602ffd700 1 -- 172.21.15.16:0/1074435384 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe60c01a310 con 0x7fe6141a6990 2023-09-30T22:34:05.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.924+0000 7f97d6cee700 1 -- 172.21.15.16:0/3396638632 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f97d0074ea0 con 0x7f97d0136f50 2023-09-30T22:34:05.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.924+0000 7f97cd7fa700 1 -- 172.21.15.16:0/3396638632 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f97c8028090 con 0x7f97d0136f50 2023-09-30T22:34:05.925 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:34:05.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.927+0000 7f97d6cee700 1 -- 172.21.15.16:0/3396638632 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f97d0136f50 msgr2=0x7f97d01c4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:05.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.927+0000 7f97d6cee700 1 --2- 172.21.15.16:0/3396638632 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f97d0136f50 0x7f97d01c4120 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f97c800b370 tx=0x7f97c800b680 comp rx=0 tx=0).stop 2023-09-30T22:34:05.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.927+0000 7f97d6cee700 1 -- 172.21.15.16:0/3396638632 shutdown_connections 2023-09-30T22:34:05.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.927+0000 7f97d6cee700 1 --2- 172.21.15.16:0/3396638632 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f97d0136f50 0x7f97d01c4120 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:05.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.927+0000 7f97d6cee700 1 --2- 172.21.15.16:0/3396638632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f97d0131810 0x7f97d012dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:05.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.927+0000 7f97d6cee700 1 --2- 172.21.15.16:0/3396638632 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f97d0134630 0x7f97d012e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:05.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.927+0000 7f97d6cee700 1 -- 172.21.15.16:0/3396638632 >> 172.21.15.16:0/3396638632 conn(0x7f97d00a2aa0 msgr2=0x7f97d0134fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:05.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.927+0000 7f97d6cee700 1 -- 172.21.15.16:0/3396638632 shutdown_connections 2023-09-30T22:34:05.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.927+0000 7f97d6cee700 1 -- 172.21.15.16:0/3396638632 wait complete. 2023-09-30T22:34:05.942 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:34:05.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.979+0000 7f841bb6b700 1 -- 172.21.15.16:0/3411607806 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8414074ea0 con 0x7f841409d6f0 2023-09-30T22:34:05.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.979+0000 7f840a7fc700 1 -- 172.21.15.16:0/3411607806 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8410015030 con 0x7f841409d6f0 2023-09-30T22:34:05.980 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:34:05.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.982+0000 7f841bb6b700 1 -- 172.21.15.16:0/3411607806 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f841409d6f0 msgr2=0x7f84141a5e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:05.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.982+0000 7f841bb6b700 1 --2- 172.21.15.16:0/3411607806 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f841409d6f0 0x7f84141a5e80 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f8410016000 tx=0x7f8410004750 comp rx=0 tx=0).stop 2023-09-30T22:34:05.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.983+0000 7f841bb6b700 1 -- 172.21.15.16:0/3411607806 shutdown_connections 2023-09-30T22:34:05.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.983+0000 7f841bb6b700 1 --2- 172.21.15.16:0/3411607806 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f841409e410 0x7f84141a6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:05.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.983+0000 7f841bb6b700 1 --2- 172.21.15.16:0/3411607806 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f841409d6f0 0x7f84141a5e80 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:05.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.983+0000 7f841bb6b700 1 --2- 172.21.15.16:0/3411607806 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f841409e000 0x7f84141a63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:05.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.983+0000 7f841bb6b700 1 -- 172.21.15.16:0/3411607806 >> 172.21.15.16:0/3411607806 conn(0x7f8414066f50 msgr2=0x7f84141baed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:05.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.983+0000 7f841bb6b700 1 -- 172.21.15.16:0/3411607806 shutdown_connections 2023-09-30T22:34:05.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.983+0000 7f841bb6b700 1 -- 172.21.15.16:0/3411607806 wait complete. 2023-09-30T22:34:05.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.994+0000 7f867d4cf700 1 -- 172.21.15.16:0/132622760 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8658005590 con 0x7f8678133930 2023-09-30T22:34:05.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.995+0000 7f865f7fe700 1 -- 172.21.15.16:0/132622760 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f867001a3c0 con 0x7f8678133930 2023-09-30T22:34:05.995 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:34:05.997 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:34:05.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.997+0000 7f867d4cf700 1 -- 172.21.15.16:0/132622760 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8678133930 msgr2=0x7f8678126aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:05.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.997+0000 7f867d4cf700 1 --2- 172.21.15.16:0/132622760 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8678133930 0x7f8678126aa0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f8670015000 tx=0x7f8670004ad0 comp rx=0 tx=0).stop 2023-09-30T22:34:05.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.997+0000 7f867d4cf700 1 -- 172.21.15.16:0/132622760 shutdown_connections 2023-09-30T22:34:05.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.997+0000 7f867d4cf700 1 --2- 172.21.15.16:0/132622760 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8678133930 0x7f8678126aa0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:05.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.997+0000 7f867d4cf700 1 --2- 172.21.15.16:0/132622760 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f867809d630 0x7f8678126020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:05.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.997+0000 7f867d4cf700 1 --2- 172.21.15.16:0/132622760 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f86780a0a30 0x7f8678126560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:05.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.997+0000 7f867d4cf700 1 -- 172.21.15.16:0/132622760 >> 172.21.15.16:0/132622760 conn(0x7f86780a3050 msgr2=0x7f86780a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:05.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.997+0000 7f867d4cf700 1 -- 172.21.15.16:0/132622760 shutdown_connections 2023-09-30T22:34:05.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:05.997+0000 7f867d4cf700 1 -- 172.21.15.16:0/132622760 wait complete. 2023-09-30T22:34:06.011 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:34:06.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:06.020+0000 7fe618afe700 1 -- 172.21.15.16:0/1074435384 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe5f8005590 con 0x7fe6141a6990 2023-09-30T22:34:06.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:06.020+0000 7fe602ffd700 1 -- 172.21.15.16:0/1074435384 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe60c041020 con 0x7fe6141a6990 2023-09-30T22:34:06.021 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:34:06.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:06.023+0000 7fe618afe700 1 -- 172.21.15.16:0/1074435384 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe6141a6990 msgr2=0x7fe6141cea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:06.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:06.023+0000 7fe618afe700 1 --2- 172.21.15.16:0/1074435384 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe6141a6990 0x7fe6141cea40 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fe61406b830 tx=0x7fe60c00a5f0 comp rx=0 tx=0).stop 2023-09-30T22:34:06.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:06.023+0000 7fe618afe700 1 -- 172.21.15.16:0/1074435384 shutdown_connections 2023-09-30T22:34:06.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:06.023+0000 7fe618afe700 1 --2- 172.21.15.16:0/1074435384 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe6141a6990 0x7fe6141cea40 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:06.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:06.023+0000 7fe618afe700 1 --2- 172.21.15.16:0/1074435384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe5fc003a30 0x7fe6141a5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:06.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:06.023+0000 7fe618afe700 1 --2- 172.21.15.16:0/1074435384 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe6141a9f30 0x7fe6141a6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:06.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:06.023+0000 7fe618afe700 1 -- 172.21.15.16:0/1074435384 >> 172.21.15.16:0/1074435384 conn(0x7fe614066f50 msgr2=0x7fe6141baf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:06.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:06.023+0000 7fe618afe700 1 -- 172.21.15.16:0/1074435384 shutdown_connections 2023-09-30T22:34:06.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:06.023+0000 7fe618afe700 1 -- 172.21.15.16:0/1074435384 wait complete. 2023-09-30T22:34:06.036 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:34:06.942 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:34:06.998 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:34:07.012 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:34:07.038 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:34:07.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.094+0000 7f86a805a700 1 Processor -- start 2023-09-30T22:34:07.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.095+0000 7f86a805a700 1 -- start start 2023-09-30T22:34:07.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.095+0000 7f86a805a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f86a00a0a30 0x7f86a00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:07.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.095+0000 7f86a805a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f86a00a1340 0x7f86a0133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:07.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.095+0000 7f86a805a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86a009edd0 con 0x7f86a0133fa0 2023-09-30T22:34:07.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.095+0000 7f86a805a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86a009e1f0 con 0x7f86a00a1340 2023-09-30T22:34:07.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.095+0000 7f86a805a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f86a009dc50 con 0x7f86a00a0a30 2023-09-30T22:34:07.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.095+0000 7f86a55f5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f86a00a1340 0x7f86a0133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:07.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.095+0000 7f86a5df6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f86a00a0a30 0x7f86a00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:07.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.095+0000 7f86a55f5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f86a00a1340 0x7f86a0133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38828/0 (socket says 172.21.15.16:38828) 2023-09-30T22:34:07.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.095+0000 7f86a55f5700 1 -- 172.21.15.16:0/454885719 learned_addr learned my addr 172.21.15.16:0/454885719 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:07.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.095+0000 7f86a5df6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f86a00a0a30 0x7f86a00a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:57632/0 (socket says 172.21.15.16:57632) 2023-09-30T22:34:07.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.096+0000 7f86a4df4700 1 -- 172.21.15.16:0/454885719 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1163121318 0 0) 0x7f86a009edd0 con 0x7f86a0133fa0 2023-09-30T22:34:07.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.096+0000 7f86a4df4700 1 -- 172.21.15.16:0/454885719 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f868c0035b0 con 0x7f86a0133fa0 2023-09-30T22:34:07.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.096+0000 7f86a5df6700 1 -- 172.21.15.16:0/454885719 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f86a00a1340 msgr2=0x7f86a0133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:07.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.096+0000 7f86a5df6700 1 --2- 172.21.15.16:0/454885719 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f86a00a1340 0x7f86a0133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:07.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.096+0000 7f86a5df6700 1 -- 172.21.15.16:0/454885719 >> v1:172.21.15.16:6789/0 conn(0x7f86a0133fa0 legacy=0x7f86a013ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:07.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.096+0000 7f86a5df6700 1 -- 172.21.15.16:0/454885719 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86a00a1fe0 con 0x7f86a00a0a30 2023-09-30T22:34:07.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.096+0000 7f86a5df6700 1 --2- 172.21.15.16:0/454885719 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f86a00a0a30 0x7f86a00a0e00 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f869c009870 tx=0x7f869c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39083097ad3c91bd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:07.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.097+0000 7f86a4df4700 1 -- 172.21.15.16:0/454885719 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869c011830 con 0x7f86a00a0a30 2023-09-30T22:34:07.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.097+0000 7f86a4df4700 1 -- 172.21.15.16:0/454885719 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f869c011990 con 0x7f86a00a0a30 2023-09-30T22:34:07.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.097+0000 7f86a4df4700 1 -- 172.21.15.16:0/454885719 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869c011d00 con 0x7f86a00a0a30 2023-09-30T22:34:07.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.097+0000 7f86a805a700 1 -- 172.21.15.16:0/454885719 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f86a00a0a30 msgr2=0x7f86a00a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:07.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.097+0000 7f86a805a700 1 --2- 172.21.15.16:0/454885719 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f86a00a0a30 0x7f86a00a0e00 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f869c009870 tx=0x7f869c009b80 comp rx=0 tx=0).stop 2023-09-30T22:34:07.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.098+0000 7f86a805a700 1 -- 172.21.15.16:0/454885719 shutdown_connections 2023-09-30T22:34:07.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.098+0000 7f86a805a700 1 --2- 172.21.15.16:0/454885719 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f86a00a0a30 0x7f86a00a0e00 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:07.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.098+0000 7f86a805a700 1 --2- 172.21.15.16:0/454885719 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f86a00a1340 0x7f86a0133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:07.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.098+0000 7f86a805a700 1 -- 172.21.15.16:0/454885719 >> 172.21.15.16:0/454885719 conn(0x7f86a00a6550 msgr2=0x7f86a00a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:07.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.098+0000 7f86a805a700 1 -- 172.21.15.16:0/454885719 shutdown_connections 2023-09-30T22:34:07.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.098+0000 7f86a805a700 1 -- 172.21.15.16:0/454885719 wait complete. 2023-09-30T22:34:07.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.099+0000 7f86a805a700 1 Processor -- start 2023-09-30T22:34:07.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.099+0000 7f86a805a700 1 -- start start 2023-09-30T22:34:07.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.099+0000 7f86a805a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f86a00a0a30 0x7f86a01c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:07.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.099+0000 7f86a805a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86a00a1340 0x7f86a01c8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:07.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.099+0000 7f86a805a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f86a0133fa0 0x7f86a01c8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:07.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.099+0000 7f86a805a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a00a0220 con 0x7f86a00a1340 2023-09-30T22:34:07.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.099+0000 7f86a805a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86a01263b0 con 0x7f86a00a0a30 2023-09-30T22:34:07.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.099+0000 7f86a805a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f86a01246a0 con 0x7f86a0133fa0 2023-09-30T22:34:07.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.099+0000 7f86a5df6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f86a00a0a30 0x7f86a01c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:07.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.099+0000 7f86a5df6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f86a00a0a30 0x7f86a01c83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38842/0 (socket says 172.21.15.16:38842) 2023-09-30T22:34:07.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.100+0000 7f86a5df6700 1 -- 172.21.15.16:0/1574712007 learned_addr learned my addr 172.21.15.16:0/1574712007 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:07.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.100+0000 7f86a65f7700 1 --2- 172.21.15.16:0/1574712007 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f86a0133fa0 0x7f86a01c8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:07.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.100+0000 7f86a55f5700 1 --2- 172.21.15.16:0/1574712007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86a00a1340 0x7f86a01c8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:07.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.100+0000 7f86a5df6700 1 -- 172.21.15.16:0/1574712007 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f86a0133fa0 msgr2=0x7f86a01c8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:07.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.100+0000 7f86a5df6700 1 --2- 172.21.15.16:0/1574712007 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f86a0133fa0 0x7f86a01c8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:07.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.100+0000 7f86a5df6700 1 -- 172.21.15.16:0/1574712007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86a00a1340 msgr2=0x7f86a01c8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:07.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.100+0000 7f86a5df6700 1 --2- 172.21.15.16:0/1574712007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86a00a1340 0x7f86a01c8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:07.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.100+0000 7f86a5df6700 1 -- 172.21.15.16:0/1574712007 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f869c0094a0 con 0x7f86a00a0a30 2023-09-30T22:34:07.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.100+0000 7f86a5df6700 1 --2- 172.21.15.16:0/1574712007 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f86a00a0a30 0x7f86a01c83d0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f869c009bc0 tx=0x7f869c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:07.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.100+0000 7f86967fc700 1 -- 172.21.15.16:0/1574712007 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869c004460 con 0x7f86a00a0a30 2023-09-30T22:34:07.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.100+0000 7f86967fc700 1 -- 172.21.15.16:0/1574712007 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f869c0045c0 con 0x7f86a00a0a30 2023-09-30T22:34:07.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.100+0000 7f86a805a700 1 -- 172.21.15.16:0/1574712007 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86a00a25e0 con 0x7f86a00a0a30 2023-09-30T22:34:07.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.100+0000 7f86967fc700 1 -- 172.21.15.16:0/1574712007 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f869c0038c0 con 0x7f86a00a0a30 2023-09-30T22:34:07.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.102+0000 7f86a805a700 1 -- 172.21.15.16:0/1574712007 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86a00a3630 con 0x7f86a00a0a30 2023-09-30T22:34:07.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.102+0000 7f86967fc700 1 -- 172.21.15.16:0/1574712007 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f869c021070 con 0x7f86a00a0a30 2023-09-30T22:34:07.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.102+0000 7f86967fc700 1 -- 172.21.15.16:0/1574712007 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f869c041450 con 0x7f86a00a0a30 2023-09-30T22:34:07.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.103+0000 7f86a805a700 1 -- 172.21.15.16:0/1574712007 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86a0074ea0 con 0x7f86a00a0a30 2023-09-30T22:34:07.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.108+0000 7f86967fc700 1 -- 172.21.15.16:0/1574712007 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f869c015030 con 0x7f86a00a0a30 2023-09-30T22:34:07.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.150+0000 7fb37d0de700 1 Processor -- start 2023-09-30T22:34:07.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.150+0000 7fb37d0de700 1 -- start start 2023-09-30T22:34:07.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.150+0000 7fb37d0de700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb3781316c0 0x7fb378131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:07.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.150+0000 7fb37d0de700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb37809d9f0 0x7fb37806a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:07.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.150+0000 7fb37d0de700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb37806acb0 0x7fb37806b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:07.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.150+0000 7fb37d0de700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb37806b630 con 0x7fb37809d9f0 2023-09-30T22:34:07.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.151+0000 7fb37d0de700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb37809e130 con 0x7fb3781316c0 2023-09-30T22:34:07.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.151+0000 7fb37d0de700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb3781312b0 con 0x7fb37806acb0 2023-09-30T22:34:07.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.151+0000 7fb376d9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb3781316c0 0x7fb378131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:07.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.151+0000 7fb376d9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb3781316c0 0x7fb378131a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38854/0 (socket says 172.21.15.16:38854) 2023-09-30T22:34:07.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.151+0000 7fb376d9d700 1 -- 172.21.15.16:0/30786925 learned_addr learned my addr 172.21.15.16:0/30786925 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:07.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.151+0000 7fb37659c700 1 --2- 172.21.15.16:0/30786925 >> v2:172.21.15.16:3300/0 conn(0x7fb37809d9f0 0x7fb37806a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:07.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.151+0000 7fb37759e700 1 --2- 172.21.15.16:0/30786925 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb37806acb0 0x7fb37806b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:07.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.151+0000 7fb376d9d700 1 -- 172.21.15.16:0/30786925 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb37806acb0 msgr2=0x7fb37806b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:07.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.151+0000 7fb376d9d700 1 --2- 172.21.15.16:0/30786925 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb37806acb0 0x7fb37806b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:07.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.151+0000 7fb376d9d700 1 -- 172.21.15.16:0/30786925 >> v2:172.21.15.16:3300/0 conn(0x7fb37809d9f0 msgr2=0x7fb37806a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:07.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.151+0000 7fb376d9d700 1 --2- 172.21.15.16:0/30786925 >> v2:172.21.15.16:3300/0 conn(0x7fb37809d9f0 0x7fb37806a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:07.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.151+0000 7fb376d9d700 1 -- 172.21.15.16:0/30786925 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb37809f6e0 con 0x7fb3781316c0 2023-09-30T22:34:07.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.152+0000 7fb376d9d700 1 --2- 172.21.15.16:0/30786925 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb3781316c0 0x7fb378131a90 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fb368009920 tx=0x7fb368009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91301eb1179fd2a2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:07.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.152+0000 7fb375d9b700 1 -- 172.21.15.16:0/30786925 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb368011830 con 0x7fb3781316c0 2023-09-30T22:34:07.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.152+0000 7fb375d9b700 1 -- 172.21.15.16:0/30786925 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb368011990 con 0x7fb3781316c0 2023-09-30T22:34:07.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.152+0000 7fb375d9b700 1 -- 172.21.15.16:0/30786925 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb368011d20 con 0x7fb3781316c0 2023-09-30T22:34:07.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.152+0000 7fb37d0de700 1 -- 172.21.15.16:0/30786925 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb3781316c0 msgr2=0x7fb378131a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:07.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.152+0000 7fb37d0de700 1 --2- 172.21.15.16:0/30786925 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb3781316c0 0x7fb378131a90 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fb368009920 tx=0x7fb368009c30 comp rx=0 tx=0).stop 2023-09-30T22:34:07.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.153+0000 7fb37d0de700 1 -- 172.21.15.16:0/30786925 shutdown_connections 2023-09-30T22:34:07.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.153+0000 7fb37d0de700 1 --2- 172.21.15.16:0/30786925 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb3781316c0 0x7fb378131a90 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:07.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.153+0000 7fb37d0de700 1 --2- 172.21.15.16:0/30786925 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb37806acb0 0x7fb37806b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:07.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.153+0000 7fb37d0de700 1 --2- 172.21.15.16:0/30786925 >> v2:172.21.15.16:3300/0 conn(0x7fb37809d9f0 0x7fb37806a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:07.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.153+0000 7fb37d0de700 1 -- 172.21.15.16:0/30786925 >> 172.21.15.16:0/30786925 conn(0x7fb378066f50 msgr2=0x7fb37809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:07.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.153+0000 7fb37d0de700 1 -- 172.21.15.16:0/30786925 shutdown_connections 2023-09-30T22:34:07.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.153+0000 7fb37d0de700 1 -- 172.21.15.16:0/30786925 wait complete. 2023-09-30T22:34:07.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.155+0000 7fb37d0de700 1 Processor -- start 2023-09-30T22:34:07.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.155+0000 7fb37d0de700 1 -- start start 2023-09-30T22:34:07.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.155+0000 7fb37d0de700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb37806acb0 0x7fb378070ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:07.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.155+0000 7fb37d0de700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb37809d9f0 0x7fb378071010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:07.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.155+0000 7fb37d0de700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb3781316c0 0x7fb378071550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:07.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.155+0000 7fb37d0de700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb378128610 con 0x7fb37806acb0 2023-09-30T22:34:07.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.155+0000 7fb37d0de700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb37811b400 con 0x7fb3781316c0 2023-09-30T22:34:07.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.155+0000 7fb37d0de700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb37811a830 con 0x7fb37809d9f0 2023-09-30T22:34:07.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.156+0000 7fb37659c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb37809d9f0 0x7fb378071010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:07.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.156+0000 7fb376d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb37806acb0 0x7fb378070ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:07.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.156+0000 7fb37659c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb37809d9f0 0x7fb378071010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:57646/0 (socket says 172.21.15.16:57646) 2023-09-30T22:34:07.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.156+0000 7fb37659c700 1 -- 172.21.15.16:0/3925527325 learned_addr learned my addr 172.21.15.16:0/3925527325 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:07.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.156+0000 7fb37759e700 1 --2- 172.21.15.16:0/3925527325 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb3781316c0 0x7fb378071550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:07.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.156+0000 7fb37759e700 1 -- 172.21.15.16:0/3925527325 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb37809d9f0 msgr2=0x7fb378071010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:07.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.156+0000 7fb37759e700 1 --2- 172.21.15.16:0/3925527325 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb37809d9f0 0x7fb378071010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:07.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.156+0000 7fb37759e700 1 -- 172.21.15.16:0/3925527325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb37806acb0 msgr2=0x7fb378070ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:07.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.156+0000 7fb37759e700 1 --2- 172.21.15.16:0/3925527325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb37806acb0 0x7fb378070ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:07.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.156+0000 7fb37759e700 1 -- 172.21.15.16:0/3925527325 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb368009550 con 0x7fb3781316c0 2023-09-30T22:34:07.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.156+0000 7fb37659c700 1 --2- 172.21.15.16:0/3925527325 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb37809d9f0 0x7fb378071010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:07.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.156+0000 7fb37759e700 1 --2- 172.21.15.16:0/3925527325 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb3781316c0 0x7fb378071550 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fb37000efc0 tx=0x7fb37000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:07.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.157+0000 7fb35f7fe700 1 -- 172.21.15.16:0/3925527325 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37000cc90 con 0x7fb3781316c0 2023-09-30T22:34:07.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.157+0000 7fb35f7fe700 1 -- 172.21.15.16:0/3925527325 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb37000f040 con 0x7fb3781316c0 2023-09-30T22:34:07.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.157+0000 7fb37d0de700 1 -- 172.21.15.16:0/3925527325 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb378072760 con 0x7fb3781316c0 2023-09-30T22:34:07.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.157+0000 7fb35f7fe700 1 -- 172.21.15.16:0/3925527325 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3700146c0 con 0x7fb3781316c0 2023-09-30T22:34:07.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.157+0000 7fb37d0de700 1 -- 172.21.15.16:0/3925527325 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb378119480 con 0x7fb3781316c0 2023-09-30T22:34:07.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.158+0000 7fb35f7fe700 1 -- 172.21.15.16:0/3925527325 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb370014820 con 0x7fb3781316c0 2023-09-30T22:34:07.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.158+0000 7fb35f7fe700 1 -- 172.21.15.16:0/3925527325 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fb370040280 con 0x7fb3781316c0 2023-09-30T22:34:07.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.158+0000 7fb37d0de700 1 -- 172.21.15.16:0/3925527325 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3781196d0 con 0x7fb3781316c0 2023-09-30T22:34:07.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.163+0000 7fb35f7fe700 1 -- 172.21.15.16:0/3925527325 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb370018020 con 0x7fb3781316c0 2023-09-30T22:34:07.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.193+0000 7ff8a9c26700 1 Processor -- start 2023-09-30T22:34:07.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.193+0000 7ff8a9c26700 1 -- start start 2023-09-30T22:34:07.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.193+0000 7f20f71e5700 1 Processor -- start 2023-09-30T22:34:07.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.193+0000 7f20f71e5700 1 -- start start 2023-09-30T22:34:07.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.194+0000 7ff8a9c26700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff8a409d6f0 0x7ff8a409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:07.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.194+0000 7ff8a9c26700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff8a409e000 0x7ff8a4130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:07.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.194+0000 7ff8a9c26700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8a409f130 con 0x7ff8a409e410 2023-09-30T22:34:07.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.194+0000 7ff8a9c26700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8a409f520 con 0x7ff8a409d6f0 2023-09-30T22:34:07.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.194+0000 7ff8a9c26700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a409e930 con 0x7ff8a409e000 2023-09-30T22:34:07.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.194+0000 7f20f71e5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f20f00a0a30 0x7f20f00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:07.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.194+0000 7ff8a3fff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff8a409e410 0x7ff8a4130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47524/0 (socket says 172.21.15.16:47524) 2023-09-30T22:34:07.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.194+0000 7ff8a3fff700 1 -- 172.21.15.16:0/2071009545 learned_addr learned my addr 172.21.15.16:0/2071009545 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:34:07.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.194+0000 7ff8a37fe700 1 --2- 172.21.15.16:0/2071009545 >> v2:172.21.15.16:3300/0 conn(0x7ff8a409d6f0 0x7ff8a409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:07.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.194+0000 7ff8a2ffd700 1 --2- 172.21.15.16:0/2071009545 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff8a409e000 0x7ff8a4130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:07.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.194+0000 7f20f71e5700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f20f0138d80 0x7f20f013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:07.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.194+0000 7f20f71e5700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20f00a2a70 con 0x7f20f009d630 2023-09-30T22:34:07.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.194+0000 7f20f71e5700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20f00a01d0 con 0x7f20f0138d80 2023-09-30T22:34:07.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.194+0000 7f20f71e5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20f009e2b0 con 0x7f20f00a0a30 2023-09-30T22:34:07.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.194+0000 7f20effff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f20f009d630 0x7f20f00a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47526/0 (socket says 172.21.15.16:47526) 2023-09-30T22:34:07.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.194+0000 7f20effff700 1 -- 172.21.15.16:0/214596838 learned_addr learned my addr 172.21.15.16:0/214596838 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:34:07.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.194+0000 7ff8a2ffd700 1 -- 172.21.15.16:0/2071009545 >> v2:172.21.15.16:3300/0 conn(0x7ff8a409d6f0 msgr2=0x7ff8a409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:07.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.194+0000 7ff8a2ffd700 1 --2- 172.21.15.16:0/2071009545 >> v2:172.21.15.16:3300/0 conn(0x7ff8a409d6f0 0x7ff8a409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:07.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.194+0000 7ff8a2ffd700 1 -- 172.21.15.16:0/2071009545 >> v1:172.21.15.16:6789/0 conn(0x7ff8a409e410 legacy=0x7ff8a4130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:07.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.194+0000 7ff8a2ffd700 1 -- 172.21.15.16:0/2071009545 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8a409b2d0 con 0x7ff8a409e000 2023-09-30T22:34:07.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.195+0000 7ff8a2ffd700 1 --2- 172.21.15.16:0/2071009545 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff8a409e000 0x7ff8a4130660 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ff898009840 tx=0x7ff898009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=892a2af93a302f37 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:07.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.195+0000 7ff8a27fc700 1 -- 172.21.15.16:0/2071009545 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff898004230 con 0x7ff8a409e000 2023-09-30T22:34:07.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.195+0000 7ff8a27fc700 1 -- 172.21.15.16:0/2071009545 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff898004390 con 0x7ff8a409e000 2023-09-30T22:34:07.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.195+0000 7ff8a27fc700 1 -- 172.21.15.16:0/2071009545 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8980036a0 con 0x7ff8a409e000 2023-09-30T22:34:07.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.195+0000 7f20f4f81700 1 --2- 172.21.15.16:0/214596838 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f20f00a0a30 0x7f20f00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:07.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.195+0000 7f20f4f81700 1 -- 172.21.15.16:0/214596838 >> v2:172.21.15.16:3300/0 conn(0x7f20f0138d80 msgr2=0x7f20f013b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:07.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.195+0000 7f20f4f81700 1 --2- 172.21.15.16:0/214596838 >> v2:172.21.15.16:3300/0 conn(0x7f20f0138d80 0x7f20f013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:07.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.195+0000 7f20f4f81700 1 -- 172.21.15.16:0/214596838 >> v1:172.21.15.16:6789/0 conn(0x7f20f009d630 legacy=0x7f20f00a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:07.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.195+0000 7f20f4f81700 1 -- 172.21.15.16:0/214596838 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f009dc50 con 0x7f20f00a0a30 2023-09-30T22:34:07.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.196+0000 7f20f4f81700 1 --2- 172.21.15.16:0/214596838 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f20f00a0a30 0x7f20f00a0e00 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f20e0009840 tx=0x7f20e0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fc321e208c12265 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:07.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.196+0000 7ff8a9c26700 1 -- 172.21.15.16:0/2071009545 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff8a409e000 msgr2=0x7ff8a4130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:07.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.196+0000 7ff8a9c26700 1 --2- 172.21.15.16:0/2071009545 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff8a409e000 0x7ff8a4130660 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ff898009840 tx=0x7ff898009b50 comp rx=0 tx=0).stop 2023-09-30T22:34:07.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.196+0000 7ff8a9c26700 1 -- 172.21.15.16:0/2071009545 shutdown_connections 2023-09-30T22:34:07.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.196+0000 7ff8a9c26700 1 --2- 172.21.15.16:0/2071009545 >> v2:172.21.15.16:3300/0 conn(0x7ff8a409d6f0 0x7ff8a409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:07.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.196+0000 7ff8a9c26700 1 --2- 172.21.15.16:0/2071009545 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff8a409e000 0x7ff8a4130660 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:07.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.196+0000 7ff8a9c26700 1 -- 172.21.15.16:0/2071009545 >> 172.21.15.16:0/2071009545 conn(0x7ff8a4066f50 msgr2=0x7ff8a409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:07.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.196+0000 7ff8a9c26700 1 -- 172.21.15.16:0/2071009545 shutdown_connections 2023-09-30T22:34:07.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.196+0000 7ff8a9c26700 1 -- 172.21.15.16:0/2071009545 wait complete. 2023-09-30T22:34:07.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.197+0000 7ff8a9c26700 1 Processor -- start 2023-09-30T22:34:07.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.198+0000 7f20ef7fe700 1 -- 172.21.15.16:0/214596838 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e0011830 con 0x7f20f00a0a30 2023-09-30T22:34:07.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.197+0000 7ff8a9c26700 1 -- start start 2023-09-30T22:34:07.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.198+0000 7ff8a9c26700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff8a409d6f0 0x7ff8a41ceae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:07.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.198+0000 7ff8a9c26700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8a409e000 0x7ff8a41cf020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:07.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.198+0000 7ff8a9c26700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff8a409e410 0x7ff8a41b6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:07.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.198+0000 7ff8a9c26700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a41b7c20 con 0x7ff8a409e000 2023-09-30T22:34:07.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.198+0000 7ff8a9c26700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a412fad0 con 0x7ff8a409d6f0 2023-09-30T22:34:07.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.198+0000 7ff8a9c26700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a409c700 con 0x7ff8a409e410 2023-09-30T22:34:07.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.198+0000 7f20ef7fe700 1 -- 172.21.15.16:0/214596838 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e0011990 con 0x7f20f00a0a30 2023-09-30T22:34:07.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.198+0000 7f20ef7fe700 1 -- 172.21.15.16:0/214596838 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e0011d20 con 0x7f20f00a0a30 2023-09-30T22:34:07.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.198+0000 7ff8a37fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff8a409d6f0 0x7ff8a41ceae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:07.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.198+0000 7ff8a37fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff8a409d6f0 0x7ff8a41ceae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38888/0 (socket says 172.21.15.16:38888) 2023-09-30T22:34:07.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.198+0000 7ff8a37fe700 1 -- 172.21.15.16:0/4166632944 learned_addr learned my addr 172.21.15.16:0/4166632944 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:07.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.198+0000 7ff8a3fff700 1 --2- 172.21.15.16:0/4166632944 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff8a409e410 0x7ff8a41b6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:07.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.198+0000 7ff8a2ffd700 1 --2- 172.21.15.16:0/4166632944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8a409e000 0x7ff8a41cf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:07.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.198+0000 7ff8a3fff700 1 -- 172.21.15.16:0/4166632944 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff8a409d6f0 msgr2=0x7ff8a41ceae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:07.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.198+0000 7ff8a3fff700 1 --2- 172.21.15.16:0/4166632944 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff8a409d6f0 0x7ff8a41ceae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:07.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.198+0000 7ff8a3fff700 1 -- 172.21.15.16:0/4166632944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8a409e000 msgr2=0x7ff8a41cf020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:07.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.199+0000 7ff8a3fff700 1 --2- 172.21.15.16:0/4166632944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8a409e000 0x7ff8a41cf020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:07.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.199+0000 7ff8a3fff700 1 -- 172.21.15.16:0/4166632944 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8980094a0 con 0x7ff8a409e410 2023-09-30T22:34:07.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.199+0000 7f20f71e5700 1 -- 172.21.15.16:0/214596838 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f20f00a0a30 msgr2=0x7f20f00a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:07.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.199+0000 7f20f71e5700 1 --2- 172.21.15.16:0/214596838 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f20f00a0a30 0x7f20f00a0e00 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f20e0009840 tx=0x7f20e0009b50 comp rx=0 tx=0).stop 2023-09-30T22:34:07.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.199+0000 7ff8a37fe700 1 --2- 172.21.15.16:0/4166632944 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff8a409d6f0 0x7ff8a41ceae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:07.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.199+0000 7f20f71e5700 1 -- 172.21.15.16:0/214596838 shutdown_connections 2023-09-30T22:34:07.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.199+0000 7f20f71e5700 1 --2- 172.21.15.16:0/214596838 >> v2:172.21.15.16:3300/0 conn(0x7f20f0138d80 0x7f20f013b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:07.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.199+0000 7f20f71e5700 1 --2- 172.21.15.16:0/214596838 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f20f00a0a30 0x7f20f00a0e00 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:07.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.199+0000 7f20f71e5700 1 -- 172.21.15.16:0/214596838 >> 172.21.15.16:0/214596838 conn(0x7f20f00a3050 msgr2=0x7f20f00a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:07.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.199+0000 7ff8a3fff700 1 --2- 172.21.15.16:0/4166632944 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff8a409e410 0x7ff8a41b6340 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ff89c00cb20 tx=0x7ff89c00ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:07.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.199+0000 7f20f71e5700 1 -- 172.21.15.16:0/214596838 shutdown_connections 2023-09-30T22:34:07.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.199+0000 7ff88bfff700 1 -- 172.21.15.16:0/4166632944 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89c00a9a0 con 0x7ff8a409e410 2023-09-30T22:34:07.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.199+0000 7f20f71e5700 1 -- 172.21.15.16:0/214596838 wait complete. 2023-09-30T22:34:07.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.199+0000 7ff88bfff700 1 -- 172.21.15.16:0/4166632944 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff89c00ab00 con 0x7ff8a409e410 2023-09-30T22:34:07.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.199+0000 7ff8a9c26700 1 -- 172.21.15.16:0/4166632944 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8a41b8940 con 0x7ff8a409e410 2023-09-30T22:34:07.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.200+0000 7f20f71e5700 1 Processor -- start 2023-09-30T22:34:07.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.200+0000 7f20f71e5700 1 -- start start 2023-09-30T22:34:07.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.199+0000 7ff88bfff700 1 -- 172.21.15.16:0/4166632944 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89c012400 con 0x7ff8a409e410 2023-09-30T22:34:07.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.199+0000 7ff8a9c26700 1 -- 172.21.15.16:0/4166632944 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8a41cf560 con 0x7ff8a409e410 2023-09-30T22:34:07.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.200+0000 7ff8a9c26700 1 -- 172.21.15.16:0/4166632944 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8a4074ea0 con 0x7ff8a409e410 2023-09-30T22:34:07.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.200+0000 7ff88bfff700 1 -- 172.21.15.16:0/4166632944 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff89c012560 con 0x7ff8a409e410 2023-09-30T22:34:07.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.201+0000 7ff88bfff700 1 -- 172.21.15.16:0/4166632944 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7ff89c03f900 con 0x7ff8a409e410 2023-09-30T22:34:07.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.201+0000 7f20f71e5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20f009d630 0x7f20f006dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:07.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.201+0000 7f20f71e5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f20f0138d80 0x7f20f006e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:07.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.201+0000 7f20f71e5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f20f01cc8e0 0x7f20f01ccd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:07.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.201+0000 7f20f71e5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f0071290 con 0x7f20f009d630 2023-09-30T22:34:07.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.201+0000 7f20f71e5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20f0138930 con 0x7f20f01cc8e0 2023-09-30T22:34:07.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.201+0000 7f20f71e5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f20f0071990 con 0x7f20f0138d80 2023-09-30T22:34:07.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.201+0000 7f20effff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f20f0138d80 0x7f20f006e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:07.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.201+0000 7f20effff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f20f0138d80 0x7f20f006e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:57670/0 (socket says 172.21.15.16:57670) 2023-09-30T22:34:07.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.201+0000 7f20effff700 1 -- 172.21.15.16:0/206493786 learned_addr learned my addr 172.21.15.16:0/206493786 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:07.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.202+0000 7f20effff700 1 -- 172.21.15.16:0/206493786 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f20f01cc8e0 msgr2=0x7f20f01ccd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:07.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.202+0000 7f20effff700 1 --2- 172.21.15.16:0/206493786 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f20f01cc8e0 0x7f20f01ccd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:07.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.202+0000 7f20effff700 1 -- 172.21.15.16:0/206493786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20f009d630 msgr2=0x7f20f006dca0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:34:07.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.202+0000 7f20effff700 1 --2- 172.21.15.16:0/206493786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20f009d630 0x7f20f006dca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:07.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.202+0000 7f20effff700 1 -- 172.21.15.16:0/206493786 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e00094a0 con 0x7f20f0138d80 2023-09-30T22:34:07.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.202+0000 7f20effff700 1 --2- 172.21.15.16:0/206493786 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f20f0138d80 0x7f20f006e1e0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f20e400c500 tx=0x7f20e400c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:07.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.210+0000 7f20ed7fa700 1 -- 172.21.15.16:0/206493786 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e4005d70 con 0x7f20f0138d80 2023-09-30T22:34:07.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.210+0000 7f20ed7fa700 1 -- 172.21.15.16:0/206493786 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e40070d0 con 0x7f20f0138d80 2023-09-30T22:34:07.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.210+0000 7f20f71e5700 1 -- 172.21.15.16:0/206493786 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20f0071490 con 0x7f20f0138d80 2023-09-30T22:34:07.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.210+0000 7f20ed7fa700 1 -- 172.21.15.16:0/206493786 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e4007440 con 0x7f20f0138d80 2023-09-30T22:34:07.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.210+0000 7f20f71e5700 1 -- 172.21.15.16:0/206493786 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20f00a4f70 con 0x7f20f0138d80 2023-09-30T22:34:07.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.210+0000 7ff88bfff700 1 -- 172.21.15.16:0/4166632944 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff89c012810 con 0x7ff8a409e410 2023-09-30T22:34:07.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.211+0000 7f20f71e5700 1 -- 172.21.15.16:0/206493786 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20f00a0410 con 0x7f20f0138d80 2023-09-30T22:34:07.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.211+0000 7f20ed7fa700 1 -- 172.21.15.16:0/206493786 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f20e40075a0 con 0x7f20f0138d80 2023-09-30T22:34:07.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.211+0000 7f20ed7fa700 1 -- 172.21.15.16:0/206493786 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f20e403edb0 con 0x7f20f0138d80 2023-09-30T22:34:07.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.221+0000 7f20ed7fa700 1 -- 172.21.15.16:0/206493786 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f20e4019b40 con 0x7f20f0138d80 2023-09-30T22:34:07.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.347+0000 7f86a805a700 1 -- 172.21.15.16:0/1574712007 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f86a009bcc0 con 0x7f86a00a0a30 2023-09-30T22:34:07.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.347+0000 7f86967fc700 1 -- 172.21.15.16:0/1574712007 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f869c010de0 con 0x7f86a00a0a30 2023-09-30T22:34:07.348 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:34:07.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.350+0000 7f86a805a700 1 -- 172.21.15.16:0/1574712007 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f86a00a0a30 msgr2=0x7f86a01c83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:07.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.350+0000 7f86a805a700 1 --2- 172.21.15.16:0/1574712007 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f86a00a0a30 0x7f86a01c83d0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f869c009bc0 tx=0x7f869c004750 comp rx=0 tx=0).stop 2023-09-30T22:34:07.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.350+0000 7f86a805a700 1 -- 172.21.15.16:0/1574712007 shutdown_connections 2023-09-30T22:34:07.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.350+0000 7f86a805a700 1 --2- 172.21.15.16:0/1574712007 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f86a00a0a30 0x7f86a01c83d0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:07.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.350+0000 7f86a805a700 1 --2- 172.21.15.16:0/1574712007 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f86a0133fa0 0x7f86a01c8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:07.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.350+0000 7f86a805a700 1 --2- 172.21.15.16:0/1574712007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f86a00a1340 0x7f86a01c8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:07.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.350+0000 7f86a805a700 1 -- 172.21.15.16:0/1574712007 >> 172.21.15.16:0/1574712007 conn(0x7f86a00a6550 msgr2=0x7f86a011c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:07.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.351+0000 7f86a805a700 1 -- 172.21.15.16:0/1574712007 shutdown_connections 2023-09-30T22:34:07.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.351+0000 7f86a805a700 1 -- 172.21.15.16:0/1574712007 wait complete. 2023-09-30T22:34:07.370 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:34:07.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.384+0000 7fb37d0de700 1 -- 172.21.15.16:0/3925527325 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb378074ea0 con 0x7fb3781316c0 2023-09-30T22:34:07.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.384+0000 7fb35f7fe700 1 -- 172.21.15.16:0/3925527325 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb37002b090 con 0x7fb3781316c0 2023-09-30T22:34:07.385 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:34:07.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.387+0000 7fb37d0de700 1 -- 172.21.15.16:0/3925527325 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb3781316c0 msgr2=0x7fb378071550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:07.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.387+0000 7fb37d0de700 1 --2- 172.21.15.16:0/3925527325 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb3781316c0 0x7fb378071550 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fb37000efc0 tx=0x7fb37000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:34:07.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.387+0000 7fb37d0de700 1 -- 172.21.15.16:0/3925527325 shutdown_connections 2023-09-30T22:34:07.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.387+0000 7fb37d0de700 1 --2- 172.21.15.16:0/3925527325 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb3781316c0 0x7fb378071550 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:07.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.387+0000 7fb37d0de700 1 --2- 172.21.15.16:0/3925527325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb37806acb0 0x7fb378070ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:07.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.387+0000 7fb37d0de700 1 --2- 172.21.15.16:0/3925527325 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb37809d9f0 0x7fb378071010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:07.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.387+0000 7fb37d0de700 1 -- 172.21.15.16:0/3925527325 >> 172.21.15.16:0/3925527325 conn(0x7fb378066f50 msgr2=0x7fb3780a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:07.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.387+0000 7fb37d0de700 1 -- 172.21.15.16:0/3925527325 shutdown_connections 2023-09-30T22:34:07.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.387+0000 7fb37d0de700 1 -- 172.21.15.16:0/3925527325 wait complete. 2023-09-30T22:34:07.401 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:34:07.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.436+0000 7ff8a9c26700 1 -- 172.21.15.16:0/4166632944 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff8a406c360 con 0x7ff8a409e410 2023-09-30T22:34:07.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.436+0000 7ff88bfff700 1 -- 172.21.15.16:0/4166632944 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff89c019300 con 0x7ff8a409e410 2023-09-30T22:34:07.437 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:34:07.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.439+0000 7ff8a9c26700 1 -- 172.21.15.16:0/4166632944 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff8a409e410 msgr2=0x7ff8a41b6340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:07.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.439+0000 7ff8a9c26700 1 --2- 172.21.15.16:0/4166632944 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff8a409e410 0x7ff8a41b6340 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ff89c00cb20 tx=0x7ff89c00ce30 comp rx=0 tx=0).stop 2023-09-30T22:34:07.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.439+0000 7ff8a9c26700 1 -- 172.21.15.16:0/4166632944 shutdown_connections 2023-09-30T22:34:07.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.439+0000 7ff8a9c26700 1 --2- 172.21.15.16:0/4166632944 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff8a409d6f0 0x7ff8a41ceae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:07.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.439+0000 7ff8a9c26700 1 --2- 172.21.15.16:0/4166632944 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff8a409e410 0x7ff8a41b6340 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:07.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.439+0000 7ff8a9c26700 1 --2- 172.21.15.16:0/4166632944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8a409e000 0x7ff8a41cf020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:07.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.439+0000 7ff8a9c26700 1 -- 172.21.15.16:0/4166632944 >> 172.21.15.16:0/4166632944 conn(0x7ff8a4066f50 msgr2=0x7ff8a40a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:07.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.439+0000 7ff8a9c26700 1 -- 172.21.15.16:0/4166632944 shutdown_connections 2023-09-30T22:34:07.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.439+0000 7ff8a9c26700 1 -- 172.21.15.16:0/4166632944 wait complete. 2023-09-30T22:34:07.453 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:34:07.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.456+0000 7f20f71e5700 1 -- 172.21.15.16:0/206493786 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f20f00747f0 con 0x7f20f0138d80 2023-09-30T22:34:07.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.456+0000 7f20ed7fa700 1 -- 172.21.15.16:0/206493786 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f20e4017070 con 0x7f20f0138d80 2023-09-30T22:34:07.457 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:34:07.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.459+0000 7f20f71e5700 1 -- 172.21.15.16:0/206493786 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f20f0138d80 msgr2=0x7f20f006e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:07.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.459+0000 7f20f71e5700 1 --2- 172.21.15.16:0/206493786 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f20f0138d80 0x7f20f006e1e0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f20e400c500 tx=0x7f20e400c8c0 comp rx=0 tx=0).stop 2023-09-30T22:34:07.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.459+0000 7f20f71e5700 1 -- 172.21.15.16:0/206493786 shutdown_connections 2023-09-30T22:34:07.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.459+0000 7f20f71e5700 1 --2- 172.21.15.16:0/206493786 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f20f01cc8e0 0x7f20f01ccd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:07.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.459+0000 7f20f71e5700 1 --2- 172.21.15.16:0/206493786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20f009d630 0x7f20f006dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:07.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.459+0000 7f20f71e5700 1 --2- 172.21.15.16:0/206493786 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f20f0138d80 0x7f20f006e1e0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:07.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.459+0000 7f20f71e5700 1 -- 172.21.15.16:0/206493786 >> 172.21.15.16:0/206493786 conn(0x7f20f00a3050 msgr2=0x7f20f00a54a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:07.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.459+0000 7f20f71e5700 1 -- 172.21.15.16:0/206493786 shutdown_connections 2023-09-30T22:34:07.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:07.459+0000 7f20f71e5700 1 -- 172.21.15.16:0/206493786 wait complete. 2023-09-30T22:34:07.472 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:34:08.371 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:34:08.402 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:34:08.453 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:34:08.474 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:34:08.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.532+0000 7fa3e1142700 1 Processor -- start 2023-09-30T22:34:08.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.532+0000 7fa3e1142700 1 -- start start 2023-09-30T22:34:08.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.533+0000 7fa3e1142700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa3dc09d6f0 0x7fa3dc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:08.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.533+0000 7fa3e1142700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa3dc09e000 0x7fa3dc130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:08.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.533+0000 7fa3e1142700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa3dc09e410 0x7fa3dc130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:08.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.533+0000 7fa3e1142700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3dc09f520 con 0x7fa3dc09d6f0 2023-09-30T22:34:08.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.533+0000 7fa3e1142700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc09f130 con 0x7fa3dc09e000 2023-09-30T22:34:08.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.533+0000 7fa3e1142700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc09e930 con 0x7fa3dc09e410 2023-09-30T22:34:08.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.533+0000 7fa3dad9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa3dc09d6f0 0x7fa3dc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:08.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.533+0000 7fa3dad9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa3dc09d6f0 0x7fa3dc09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33298/0 (socket says 172.21.15.16:33298) 2023-09-30T22:34:08.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.533+0000 7fa3dad9d700 1 -- 172.21.15.16:0/3893876082 learned_addr learned my addr 172.21.15.16:0/3893876082 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:08.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.533+0000 7fa3dad9d700 1 -- 172.21.15.16:0/3893876082 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa3dc09e410 msgr2=0x7fa3dc130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:08.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.533+0000 7fa3dad9d700 1 --2- 172.21.15.16:0/3893876082 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa3dc09e410 0x7fa3dc130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:08.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.533+0000 7fa3dad9d700 1 -- 172.21.15.16:0/3893876082 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa3dc09e000 msgr2=0x7fa3dc130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:08.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.533+0000 7fa3dad9d700 1 --2- 172.21.15.16:0/3893876082 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa3dc09e000 0x7fa3dc130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:08.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.533+0000 7fa3dad9d700 1 -- 172.21.15.16:0/3893876082 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3dc09b2d0 con 0x7fa3dc09d6f0 2023-09-30T22:34:08.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.534+0000 7fa3dad9d700 1 --2- 172.21.15.16:0/3893876082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3dc09d6f0 0x7fa3dc09dac0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc00bbc0 tx=0x7fa3cc00bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46e2d5c390f62452 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:08.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.534+0000 7fa3d9d9b700 1 -- 172.21.15.16:0/3893876082 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc00dd50 con 0x7fa3dc09d6f0 2023-09-30T22:34:08.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.534+0000 7fa3d9d9b700 1 -- 172.21.15.16:0/3893876082 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3cc010470 con 0x7fa3dc09d6f0 2023-09-30T22:34:08.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.534+0000 7fa3d9d9b700 1 -- 172.21.15.16:0/3893876082 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc010800 con 0x7fa3dc09d6f0 2023-09-30T22:34:08.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.534+0000 7fa3e1142700 1 -- 172.21.15.16:0/3893876082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3dc09d6f0 msgr2=0x7fa3dc09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:08.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.535+0000 7fa3e1142700 1 --2- 172.21.15.16:0/3893876082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3dc09d6f0 0x7fa3dc09dac0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fa3cc00bbc0 tx=0x7fa3cc00bed0 comp rx=0 tx=0).stop 2023-09-30T22:34:08.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.535+0000 7fa3e1142700 1 -- 172.21.15.16:0/3893876082 shutdown_connections 2023-09-30T22:34:08.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.535+0000 7fa3e1142700 1 --2- 172.21.15.16:0/3893876082 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa3dc09e410 0x7fa3dc130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:08.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.535+0000 7fa3e1142700 1 --2- 172.21.15.16:0/3893876082 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3dc09d6f0 0x7fa3dc09dac0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:08.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.535+0000 7fa3e1142700 1 --2- 172.21.15.16:0/3893876082 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa3dc09e000 0x7fa3dc130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:08.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.535+0000 7fa3e1142700 1 -- 172.21.15.16:0/3893876082 >> 172.21.15.16:0/3893876082 conn(0x7fa3dc066f50 msgr2=0x7fa3dc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:08.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.535+0000 7fa3e1142700 1 -- 172.21.15.16:0/3893876082 shutdown_connections 2023-09-30T22:34:08.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.535+0000 7fa3e1142700 1 -- 172.21.15.16:0/3893876082 wait complete. 2023-09-30T22:34:08.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.536+0000 7fa3e1142700 1 Processor -- start 2023-09-30T22:34:08.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.536+0000 7fa3e1142700 1 -- start start 2023-09-30T22:34:08.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.537+0000 7fa3e1142700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3dc09d6f0 0x7fa3dc11e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:08.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.537+0000 7fa3e1142700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa3dc09e000 0x7fa3dc11e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:08.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.537+0000 7fa3e1142700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa3dc09e410 0x7fa3dc11eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:08.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.537+0000 7fa3e1142700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc122b50 con 0x7fa3dc09d6f0 2023-09-30T22:34:08.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.537+0000 7fa3e1142700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc11f890 con 0x7fa3dc09e000 2023-09-30T22:34:08.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.537+0000 7fa3e1142700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc09c700 con 0x7fa3dc09e410 2023-09-30T22:34:08.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.537+0000 7fa3db59e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa3dc09e410 0x7fa3dc11eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:08.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.537+0000 7fa3da59c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa3dc09e000 0x7fa3dc11e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:08.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.537+0000 7fa3db59e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa3dc09e410 0x7fa3dc11eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:34992/0 (socket says 172.21.15.16:34992) 2023-09-30T22:34:08.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.537+0000 7fa3db59e700 1 -- 172.21.15.16:0/1091320069 learned_addr learned my addr 172.21.15.16:0/1091320069 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:08.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.537+0000 7fa3dad9d700 1 --2- 172.21.15.16:0/1091320069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3dc09d6f0 0x7fa3dc11e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:08.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.537+0000 7fa3db59e700 1 -- 172.21.15.16:0/1091320069 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa3dc09e000 msgr2=0x7fa3dc11e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:08.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.537+0000 7fa3db59e700 1 --2- 172.21.15.16:0/1091320069 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa3dc09e000 0x7fa3dc11e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:08.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.537+0000 7fa3db59e700 1 -- 172.21.15.16:0/1091320069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3dc09d6f0 msgr2=0x7fa3dc11e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:08.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.537+0000 7fa3db59e700 1 --2- 172.21.15.16:0/1091320069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3dc09d6f0 0x7fa3dc11e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:08.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.537+0000 7fa3db59e700 1 -- 172.21.15.16:0/1091320069 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3cc00b7f0 con 0x7fa3dc09e410 2023-09-30T22:34:08.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.538+0000 7fa3db59e700 1 --2- 172.21.15.16:0/1091320069 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa3dc09e410 0x7fa3dc11eeb0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fa3d400b970 tx=0x7fa3d400bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:08.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.538+0000 7fa3c37fe700 1 -- 172.21.15.16:0/1091320069 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d400d860 con 0x7fa3dc09e410 2023-09-30T22:34:08.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.538+0000 7fa3c37fe700 1 -- 172.21.15.16:0/1091320069 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3d400d9c0 con 0x7fa3dc09e410 2023-09-30T22:34:08.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.538+0000 7fa3c37fe700 1 -- 172.21.15.16:0/1091320069 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d400dd70 con 0x7fa3dc09e410 2023-09-30T22:34:08.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.538+0000 7fa3e1142700 1 -- 172.21.15.16:0/1091320069 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3dc121bf0 con 0x7fa3dc09e410 2023-09-30T22:34:08.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.538+0000 7fa3e1142700 1 -- 172.21.15.16:0/1091320069 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3dc122660 con 0x7fa3dc09e410 2023-09-30T22:34:08.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.540+0000 7fa3e1142700 1 -- 172.21.15.16:0/1091320069 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3dc074ea0 con 0x7fa3dc09e410 2023-09-30T22:34:08.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.540+0000 7fa3c37fe700 1 -- 172.21.15.16:0/1091320069 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa3d4017400 con 0x7fa3dc09e410 2023-09-30T22:34:08.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.541+0000 7fa3c37fe700 1 -- 172.21.15.16:0/1091320069 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fa3d4006e40 con 0x7fa3dc09e410 2023-09-30T22:34:08.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.545+0000 7fa3c37fe700 1 -- 172.21.15.16:0/1091320069 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa3d4015710 con 0x7fa3dc09e410 2023-09-30T22:34:08.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.568+0000 7fdec243e700 1 Processor -- start 2023-09-30T22:34:08.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.568+0000 7fdec243e700 1 -- start start 2023-09-30T22:34:08.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.569+0000 7fdec243e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdebc09d6f0 0x7fdebc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:08.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.569+0000 7fdec243e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdebc09e410 0x7fdebc134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:08.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.569+0000 7fdec243e700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdebc09b120 con 0x7fdebc09e000 2023-09-30T22:34:08.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.569+0000 7fdec243e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdebc06b460 con 0x7fdebc09e410 2023-09-30T22:34:08.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.569+0000 7fdec243e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdebc09a7e0 con 0x7fdebc09d6f0 2023-09-30T22:34:08.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.569+0000 7fdebbfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdebc09d6f0 0x7fdebc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:08.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.569+0000 7fdebbfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdebc09d6f0 0x7fdebc09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35000/0 (socket says 172.21.15.16:35000) 2023-09-30T22:34:08.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.569+0000 7fdebbfff700 1 -- 172.21.15.16:0/3701270512 learned_addr learned my addr 172.21.15.16:0/3701270512 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:08.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.569+0000 7fdec09db700 1 --2- 172.21.15.16:0/3701270512 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdebc09e410 0x7fdebc134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:08.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.569+0000 7fdebbfff700 1 -- 172.21.15.16:0/3701270512 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdebc09e410 msgr2=0x7fdebc134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:08.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.569+0000 7fdebbfff700 1 --2- 172.21.15.16:0/3701270512 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdebc09e410 0x7fdebc134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:08.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.569+0000 7fdebbfff700 1 -- 172.21.15.16:0/3701270512 >> v1:172.21.15.16:6789/0 conn(0x7fdebc09e000 legacy=0x7fdebc1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:08.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.569+0000 7fdebbfff700 1 -- 172.21.15.16:0/3701270512 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdebc09a500 con 0x7fdebc09d6f0 2023-09-30T22:34:08.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.569+0000 7fdebaffd700 1 -- 172.21.15.16:0/3701270512 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 638626342 0 0) 0x7fdebc09b120 con 0x7fdebc09e000 2023-09-30T22:34:08.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.570+0000 7fdebbfff700 1 --2- 172.21.15.16:0/3701270512 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdebc09d6f0 0x7fdebc09dac0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fdeac009950 tx=0x7fdeac009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=188380d060ee4891 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:08.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.570+0000 7fdebaffd700 1 -- 172.21.15.16:0/3701270512 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeac011830 con 0x7fdebc09d6f0 2023-09-30T22:34:08.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.570+0000 7fdebaffd700 1 -- 172.21.15.16:0/3701270512 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeac011990 con 0x7fdebc09d6f0 2023-09-30T22:34:08.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.570+0000 7fdebaffd700 1 -- 172.21.15.16:0/3701270512 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeac011d40 con 0x7fdebc09d6f0 2023-09-30T22:34:08.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.571+0000 7fdec243e700 1 -- 172.21.15.16:0/3701270512 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdebc09d6f0 msgr2=0x7fdebc09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:08.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.571+0000 7fdec243e700 1 --2- 172.21.15.16:0/3701270512 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdebc09d6f0 0x7fdebc09dac0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fdeac009950 tx=0x7fdeac009c60 comp rx=0 tx=0).stop 2023-09-30T22:34:08.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.571+0000 7fdec243e700 1 -- 172.21.15.16:0/3701270512 shutdown_connections 2023-09-30T22:34:08.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.571+0000 7fdec243e700 1 --2- 172.21.15.16:0/3701270512 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdebc09e410 0x7fdebc134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:08.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.571+0000 7fdec243e700 1 --2- 172.21.15.16:0/3701270512 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdebc09d6f0 0x7fdebc09dac0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:08.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.571+0000 7fdec243e700 1 -- 172.21.15.16:0/3701270512 >> 172.21.15.16:0/3701270512 conn(0x7fdebc066f50 msgr2=0x7fdebc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:08.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.571+0000 7fdec243e700 1 -- 172.21.15.16:0/3701270512 shutdown_connections 2023-09-30T22:34:08.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.571+0000 7fdec243e700 1 -- 172.21.15.16:0/3701270512 wait complete. 2023-09-30T22:34:08.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.572+0000 7fdec243e700 1 Processor -- start 2023-09-30T22:34:08.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.572+0000 7fdec243e700 1 -- start start 2023-09-30T22:34:08.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.572+0000 7fdec243e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdebc09d6f0 0x7fdebc1222e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:08.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.572+0000 7fdec243e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdebc09e000 0x7fdebc122820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:08.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.573+0000 7fdec243e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdebc09e410 0x7fdebc122d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:08.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.573+0000 7fdec243e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdebc125bc0 con 0x7fdebc09e410 2023-09-30T22:34:08.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.573+0000 7fdec243e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdebc1336b0 con 0x7fdebc09d6f0 2023-09-30T22:34:08.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.573+0000 7fdec243e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdebc125860 con 0x7fdebc09e000 2023-09-30T22:34:08.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.573+0000 7fdebbfff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdebc09d6f0 0x7fdebc1222e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:08.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.573+0000 7fdebbfff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdebc09d6f0 0x7fdebc1222e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:52768/0 (socket says 172.21.15.16:52768) 2023-09-30T22:34:08.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.573+0000 7fdebbfff700 1 -- 172.21.15.16:0/3186693149 learned_addr learned my addr 172.21.15.16:0/3186693149 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:08.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.573+0000 7fdebb7fe700 1 --2- 172.21.15.16:0/3186693149 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdebc09e000 0x7fdebc122820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:08.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.573+0000 7fdec09db700 1 --2- 172.21.15.16:0/3186693149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdebc09e410 0x7fdebc122d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:08.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.573+0000 7fdebbfff700 1 -- 172.21.15.16:0/3186693149 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdebc09e000 msgr2=0x7fdebc122820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:08.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.573+0000 7fdebbfff700 1 --2- 172.21.15.16:0/3186693149 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdebc09e000 0x7fdebc122820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:08.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.573+0000 7fdebbfff700 1 -- 172.21.15.16:0/3186693149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdebc09e410 msgr2=0x7fdebc122d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:08.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.573+0000 7fdebbfff700 1 --2- 172.21.15.16:0/3186693149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdebc09e410 0x7fdebc122d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:08.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.573+0000 7fdebbfff700 1 -- 172.21.15.16:0/3186693149 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeac009550 con 0x7fdebc09d6f0 2023-09-30T22:34:08.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.573+0000 7fdec09db700 1 --2- 172.21.15.16:0/3186693149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdebc09e410 0x7fdebc122d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:08.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.573+0000 7fdebb7fe700 1 --2- 172.21.15.16:0/3186693149 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdebc09e000 0x7fdebc122820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:08.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.573+0000 7fdebbfff700 1 --2- 172.21.15.16:0/3186693149 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdebc09d6f0 0x7fdebc1222e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fdeac009130 tx=0x7fdeac004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:08.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.574+0000 7fdeb8ff9700 1 -- 172.21.15.16:0/3186693149 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeac01a840 con 0x7fdebc09d6f0 2023-09-30T22:34:08.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.574+0000 7fdeb8ff9700 1 -- 172.21.15.16:0/3186693149 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeac01f070 con 0x7fdebc09d6f0 2023-09-30T22:34:08.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.574+0000 7fdec243e700 1 -- 172.21.15.16:0/3186693149 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdebc124d80 con 0x7fdebc09d6f0 2023-09-30T22:34:08.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.574+0000 7fdeb8ff9700 1 -- 172.21.15.16:0/3186693149 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeac01ac00 con 0x7fdebc09d6f0 2023-09-30T22:34:08.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.574+0000 7fdec243e700 1 -- 172.21.15.16:0/3186693149 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdebc125590 con 0x7fdebc09d6f0 2023-09-30T22:34:08.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.575+0000 7fdeb8ff9700 1 -- 172.21.15.16:0/3186693149 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdeac00b420 con 0x7fdebc09d6f0 2023-09-30T22:34:08.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.575+0000 7fdeb8ff9700 1 -- 172.21.15.16:0/3186693149 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fdeac021070 con 0x7fdebc09d6f0 2023-09-30T22:34:08.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.575+0000 7fdea27fc700 1 -- 172.21.15.16:0/3186693149 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde940052e0 con 0x7fdebc09d6f0 2023-09-30T22:34:08.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.585+0000 7fdeb8ff9700 1 -- 172.21.15.16:0/3186693149 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdeac023330 con 0x7fdebc09d6f0 2023-09-30T22:34:08.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.631+0000 7fa8db290700 1 Processor -- start 2023-09-30T22:34:08.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.631+0000 7fa8db290700 1 -- start start 2023-09-30T22:34:08.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.632+0000 7fa8db290700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa8d41338c0 0x7fa8d4133c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:08.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.632+0000 7fa8db290700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa8d41341d0 0x7fa8d409de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:08.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.632+0000 7fa8db290700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa8d409e390 0x7fa8d409e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:08.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.632+0000 7fa8db290700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8d4070c80 con 0x7fa8d409e390 2023-09-30T22:34:08.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.632+0000 7fa8db290700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d406f720 con 0x7fa8d41341d0 2023-09-30T22:34:08.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.632+0000 7fa8db290700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d406fce0 con 0x7fa8d41338c0 2023-09-30T22:34:08.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.632+0000 7fa8d902c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa8d41338c0 0x7fa8d4133c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:08.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.632+0000 7fa8d882b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa8d41341d0 0x7fa8d409de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:08.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.632+0000 7fa8d902c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa8d41338c0 0x7fa8d4133c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35006/0 (socket says 172.21.15.16:35006) 2023-09-30T22:34:08.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.632+0000 7fa8d902c700 1 -- 172.21.15.16:0/2184590278 learned_addr learned my addr 172.21.15.16:0/2184590278 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:08.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.632+0000 7fa8d982d700 1 --2- 172.21.15.16:0/2184590278 >> v2:172.21.15.16:3300/0 conn(0x7fa8d409e390 0x7fa8d409e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:08.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.632+0000 7fa8d982d700 1 -- 172.21.15.16:0/2184590278 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa8d41338c0 msgr2=0x7fa8d4133c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:08.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.632+0000 7fa8d982d700 1 --2- 172.21.15.16:0/2184590278 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa8d41338c0 0x7fa8d4133c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:08.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.632+0000 7fa8d982d700 1 -- 172.21.15.16:0/2184590278 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa8d41341d0 msgr2=0x7fa8d409de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:08.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.632+0000 7fa8d982d700 1 --2- 172.21.15.16:0/2184590278 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa8d41341d0 0x7fa8d409de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:08.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.632+0000 7fa8d982d700 1 -- 172.21.15.16:0/2184590278 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d41336a0 con 0x7fa8d409e390 2023-09-30T22:34:08.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.633+0000 7fa8d982d700 1 --2- 172.21.15.16:0/2184590278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8d409e390 0x7fa8d409e800 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc009d90 tx=0x7fa8cc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df8fad6b25499962 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:08.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.633+0000 7fa8cbfff700 1 -- 172.21.15.16:0/2184590278 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc00e070 con 0x7fa8d409e390 2023-09-30T22:34:08.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.633+0000 7fa8cbfff700 1 -- 172.21.15.16:0/2184590278 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8cc00bb20 con 0x7fa8d409e390 2023-09-30T22:34:08.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.633+0000 7fa8cbfff700 1 -- 172.21.15.16:0/2184590278 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc012430 con 0x7fa8d409e390 2023-09-30T22:34:08.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.634+0000 7fa8db290700 1 -- 172.21.15.16:0/2184590278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8d409e390 msgr2=0x7fa8d409e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:08.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.634+0000 7fa8db290700 1 --2- 172.21.15.16:0/2184590278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8d409e390 0x7fa8d409e800 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc009d90 tx=0x7fa8cc00b330 comp rx=0 tx=0).stop 2023-09-30T22:34:08.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.634+0000 7fa8db290700 1 -- 172.21.15.16:0/2184590278 shutdown_connections 2023-09-30T22:34:08.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.634+0000 7fa8db290700 1 --2- 172.21.15.16:0/2184590278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8d409e390 0x7fa8d409e800 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:08.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.634+0000 7fa8db290700 1 --2- 172.21.15.16:0/2184590278 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa8d41338c0 0x7fa8d4133c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:08.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.634+0000 7fa8db290700 1 --2- 172.21.15.16:0/2184590278 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa8d41341d0 0x7fa8d409de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:08.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.634+0000 7fa8db290700 1 -- 172.21.15.16:0/2184590278 >> 172.21.15.16:0/2184590278 conn(0x7fa8d40a2aa0 msgr2=0x7fa8d40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:08.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.634+0000 7fa8db290700 1 -- 172.21.15.16:0/2184590278 shutdown_connections 2023-09-30T22:34:08.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.634+0000 7fa8db290700 1 -- 172.21.15.16:0/2184590278 wait complete. 2023-09-30T22:34:08.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.635+0000 7fa8db290700 1 Processor -- start 2023-09-30T22:34:08.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.635+0000 7fa8db290700 1 -- start start 2023-09-30T22:34:08.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.636+0000 7fa8db290700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa8d409e390 0x7fa8d411e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:08.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.636+0000 7fa8db290700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8d41338c0 0x7fa8d411ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:08.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.636+0000 7fa8db290700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa8d41341d0 0x7fa8d411f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:08.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.636+0000 7fa8db290700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d411fda0 con 0x7fa8d41338c0 2023-09-30T22:34:08.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.636+0000 7fa8db290700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d412fc10 con 0x7fa8d41341d0 2023-09-30T22:34:08.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.636+0000 7fa8db290700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d41212e0 con 0x7fa8d409e390 2023-09-30T22:34:08.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.636+0000 7fa8d902c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa8d409e390 0x7fa8d411e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:08.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.636+0000 7fa8d902c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa8d409e390 0x7fa8d411e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35016/0 (socket says 172.21.15.16:35016) 2023-09-30T22:34:08.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.636+0000 7fa8d902c700 1 -- 172.21.15.16:0/1155877442 learned_addr learned my addr 172.21.15.16:0/1155877442 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:08.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.636+0000 7fa8d882b700 1 --2- 172.21.15.16:0/1155877442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8d41338c0 0x7fa8d411ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:08.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.636+0000 7fa8d982d700 1 --2- 172.21.15.16:0/1155877442 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa8d41341d0 0x7fa8d411f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:08.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.636+0000 7fa8d882b700 1 -- 172.21.15.16:0/1155877442 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa8d409e390 msgr2=0x7fa8d411e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.636+0000 7fa8d882b700 1 --2- 172.21.15.16:0/1155877442 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa8d409e390 0x7fa8d411e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.636+0000 7fa8d882b700 1 -- 172.21.15.16:0/1155877442 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa8d41341d0 msgr2=0x7fa8d411f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.636+0000 7fa8d882b700 1 --2- 172.21.15.16:0/1155877442 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa8d41341d0 0x7fa8d411f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.636+0000 7fa8d882b700 1 -- 172.21.15.16:0/1155877442 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8cc009a70 con 0x7fa8d41338c0 2023-09-30T22:34:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.636+0000 7fa8d982d700 1 --2- 172.21.15.16:0/1155877442 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa8d41341d0 0x7fa8d411f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:34:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.637+0000 7fa8d882b700 1 --2- 172.21.15.16:0/1155877442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8d41338c0 0x7fa8d411ee10 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fa8c400e900 tx=0x7fa8c400ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.637+0000 7fa8c9ffb700 1 -- 172.21.15.16:0/1155877442 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c400ee50 con 0x7fa8d41338c0 2023-09-30T22:34:08.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.637+0000 7fa8c9ffb700 1 -- 172.21.15.16:0/1155877442 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8c400c850 con 0x7fa8d41338c0 2023-09-30T22:34:08.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.637+0000 7fa8c9ffb700 1 -- 172.21.15.16:0/1155877442 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c400cbc0 con 0x7fa8d41338c0 2023-09-30T22:34:08.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.637+0000 7fa8db290700 1 -- 172.21.15.16:0/1155877442 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8d4121dc0 con 0x7fa8d41338c0 2023-09-30T22:34:08.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.637+0000 7fa8db290700 1 -- 172.21.15.16:0/1155877442 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8d406f960 con 0x7fa8d41338c0 2023-09-30T22:34:08.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.638+0000 7fa8c9ffb700 1 -- 172.21.15.16:0/1155877442 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa8c400cd20 con 0x7fa8d41338c0 2023-09-30T22:34:08.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.638+0000 7fa8db290700 1 -- 172.21.15.16:0/1155877442 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8d411bb90 con 0x7fa8d41338c0 2023-09-30T22:34:08.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.639+0000 7fa8c9ffb700 1 -- 172.21.15.16:0/1155877442 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fa8c403f9f0 con 0x7fa8d41338c0 2023-09-30T22:34:08.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.648+0000 7fa8c9ffb700 1 -- 172.21.15.16:0/1155877442 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa8c401b3b0 con 0x7fa8d41338c0 2023-09-30T22:34:08.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.683+0000 7f2229b34700 1 Processor -- start 2023-09-30T22:34:08.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.683+0000 7f2229b34700 1 -- start start 2023-09-30T22:34:08.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.684+0000 7f2229b34700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f22240a0a30 0x7f22240a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:08.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.684+0000 7f2229b34700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f222409d630 0x7f22240a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:08.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.684+0000 7f2229b34700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22240a2a70 con 0x7f2224133930 2023-09-30T22:34:08.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.684+0000 7f2229b34700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22240a01d0 con 0x7f222409d630 2023-09-30T22:34:08.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.684+0000 7f2229b34700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f222409e2b0 con 0x7f22240a0a30 2023-09-30T22:34:08.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.684+0000 7f2222ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f222409d630 0x7f22240a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:08.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.684+0000 7f2223fff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2224133930 0x7f2224133db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59740/0 (socket says 172.21.15.16:59740) 2023-09-30T22:34:08.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.684+0000 7f2223fff700 1 -- 172.21.15.16:0/3639942658 learned_addr learned my addr 172.21.15.16:0/3639942658 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:34:08.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.685+0000 7f22237fe700 1 --2- 172.21.15.16:0/3639942658 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f22240a0a30 0x7f22240a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:08.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.685+0000 7f2222ffd700 1 -- 172.21.15.16:0/3639942658 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f22240a0a30 msgr2=0x7f22240a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:08.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.685+0000 7f2222ffd700 1 --2- 172.21.15.16:0/3639942658 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f22240a0a30 0x7f22240a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:08.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.685+0000 7f2222ffd700 1 -- 172.21.15.16:0/3639942658 >> v1:172.21.15.16:6789/0 conn(0x7f2224133930 legacy=0x7f2224133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:08.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.685+0000 7f2222ffd700 1 -- 172.21.15.16:0/3639942658 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f222409dc50 con 0x7f222409d630 2023-09-30T22:34:08.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.685+0000 7f2222ffd700 1 --2- 172.21.15.16:0/3639942658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f222409d630 0x7f22240a1340 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f2218009840 tx=0x7f2218009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61c32fa1a3cd15b5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:08.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.685+0000 7f22227fc700 1 -- 172.21.15.16:0/3639942658 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2218004230 con 0x7f222409d630 2023-09-30T22:34:08.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.685+0000 7f22227fc700 1 -- 172.21.15.16:0/3639942658 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2218004390 con 0x7f222409d630 2023-09-30T22:34:08.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.685+0000 7f22227fc700 1 -- 172.21.15.16:0/3639942658 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22180036a0 con 0x7f222409d630 2023-09-30T22:34:08.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.686+0000 7f2229b34700 1 -- 172.21.15.16:0/3639942658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f222409d630 msgr2=0x7f22240a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:08.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.686+0000 7f2229b34700 1 --2- 172.21.15.16:0/3639942658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f222409d630 0x7f22240a1340 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f2218009840 tx=0x7f2218009b50 comp rx=0 tx=0).stop 2023-09-30T22:34:08.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.686+0000 7f2229b34700 1 -- 172.21.15.16:0/3639942658 shutdown_connections 2023-09-30T22:34:08.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.686+0000 7f2229b34700 1 --2- 172.21.15.16:0/3639942658 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f22240a0a30 0x7f22240a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:08.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.686+0000 7f2229b34700 1 --2- 172.21.15.16:0/3639942658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f222409d630 0x7f22240a1340 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:08.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.686+0000 7f2229b34700 1 -- 172.21.15.16:0/3639942658 >> 172.21.15.16:0/3639942658 conn(0x7f22240a3050 msgr2=0x7f22240a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:08.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.687+0000 7f2229b34700 1 -- 172.21.15.16:0/3639942658 shutdown_connections 2023-09-30T22:34:08.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.687+0000 7f2229b34700 1 -- 172.21.15.16:0/3639942658 wait complete. 2023-09-30T22:34:08.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.688+0000 7f2229b34700 1 Processor -- start 2023-09-30T22:34:08.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.688+0000 7f2229b34700 1 -- start start 2023-09-30T22:34:08.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.689+0000 7f2229b34700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f222409d630 0x7f222411f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:08.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.689+0000 7f2229b34700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f22240a0a30 0x7f222411fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:08.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.689+0000 7f2229b34700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2224133930 0x7f22241c83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:08.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.689+0000 7f2229b34700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2224122540 con 0x7f222409d630 2023-09-30T22:34:08.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.689+0000 7f2229b34700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22241209b0 con 0x7f2224133930 2023-09-30T22:34:08.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.689+0000 7f2229b34700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2224121a20 con 0x7f22240a0a30 2023-09-30T22:34:08.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.689+0000 7f22237fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f222409d630 0x7f222411f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:08.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.689+0000 7f22237fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f222409d630 0x7f222411f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33354/0 (socket says 172.21.15.16:33354) 2023-09-30T22:34:08.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.689+0000 7f22237fe700 1 -- 172.21.15.16:0/3441814302 learned_addr learned my addr 172.21.15.16:0/3441814302 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:08.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.689+0000 7f2222ffd700 1 --2- 172.21.15.16:0/3441814302 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f22240a0a30 0x7f222411fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:08.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.690+0000 7f2223fff700 1 --2- 172.21.15.16:0/3441814302 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2224133930 0x7f22241c83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:08.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.690+0000 7f22237fe700 1 -- 172.21.15.16:0/3441814302 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f22240a0a30 msgr2=0x7f222411fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:08.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.690+0000 7f22237fe700 1 --2- 172.21.15.16:0/3441814302 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f22240a0a30 0x7f222411fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:08.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.690+0000 7f22237fe700 1 -- 172.21.15.16:0/3441814302 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2224133930 msgr2=0x7f22241c83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:08.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.690+0000 7f22237fe700 1 --2- 172.21.15.16:0/3441814302 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2224133930 0x7f22241c83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:08.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.690+0000 7f22237fe700 1 -- 172.21.15.16:0/3441814302 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22180094a0 con 0x7f222409d630 2023-09-30T22:34:08.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.690+0000 7f22237fe700 1 --2- 172.21.15.16:0/3441814302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f222409d630 0x7f222411f670 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f221400e9b0 tx=0x7f221400ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:08.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.690+0000 7f220bfff700 1 -- 172.21.15.16:0/3441814302 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2214009710 con 0x7f222409d630 2023-09-30T22:34:08.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.690+0000 7f220bfff700 1 -- 172.21.15.16:0/3441814302 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f221400ee50 con 0x7f222409d630 2023-09-30T22:34:08.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.690+0000 7f2229b34700 1 -- 172.21.15.16:0/3441814302 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22241222e0 con 0x7f222409d630 2023-09-30T22:34:08.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.691+0000 7f220bfff700 1 -- 172.21.15.16:0/3441814302 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f221400ca50 con 0x7f222409d630 2023-09-30T22:34:08.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.691+0000 7f2229b34700 1 -- 172.21.15.16:0/3441814302 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2224121580 con 0x7f222409d630 2023-09-30T22:34:08.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.692+0000 7f220bfff700 1 -- 172.21.15.16:0/3441814302 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f221400cbb0 con 0x7f222409d630 2023-09-30T22:34:08.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.692+0000 7f2229b34700 1 -- 172.21.15.16:0/3441814302 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2224122780 con 0x7f222409d630 2023-09-30T22:34:08.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.692+0000 7f220bfff700 1 -- 172.21.15.16:0/3441814302 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f221403f880 con 0x7f222409d630 2023-09-30T22:34:08.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.701+0000 7f220bfff700 1 -- 172.21.15.16:0/3441814302 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2214021070 con 0x7f222409d630 2023-09-30T22:34:08.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.772+0000 7fa3e1142700 1 -- 172.21.15.16:0/1091320069 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa3dc06c360 con 0x7fa3dc09e410 2023-09-30T22:34:08.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.773+0000 7fa3c37fe700 1 -- 172.21.15.16:0/1091320069 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa3d400c380 con 0x7fa3dc09e410 2023-09-30T22:34:08.773 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:34:08.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.776+0000 7fa3e1142700 1 -- 172.21.15.16:0/1091320069 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa3dc09e410 msgr2=0x7fa3dc11eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:08.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.776+0000 7fa3e1142700 1 --2- 172.21.15.16:0/1091320069 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa3dc09e410 0x7fa3dc11eeb0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fa3d400b970 tx=0x7fa3d400bc80 comp rx=0 tx=0).stop 2023-09-30T22:34:08.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.776+0000 7fa3e1142700 1 -- 172.21.15.16:0/1091320069 shutdown_connections 2023-09-30T22:34:08.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.776+0000 7fa3e1142700 1 --2- 172.21.15.16:0/1091320069 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa3dc09e410 0x7fa3dc11eeb0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:08.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.776+0000 7fa3e1142700 1 --2- 172.21.15.16:0/1091320069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3dc09d6f0 0x7fa3dc11e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:08.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.776+0000 7fa3e1142700 1 --2- 172.21.15.16:0/1091320069 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa3dc09e000 0x7fa3dc11e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:08.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.776+0000 7fa3e1142700 1 -- 172.21.15.16:0/1091320069 >> 172.21.15.16:0/1091320069 conn(0x7fa3dc066f50 msgr2=0x7fa3dc0710c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:08.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.776+0000 7fa3e1142700 1 -- 172.21.15.16:0/1091320069 shutdown_connections 2023-09-30T22:34:08.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.776+0000 7fa3e1142700 1 -- 172.21.15.16:0/1091320069 wait complete. 2023-09-30T22:34:08.792 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:34:08.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.820+0000 7fdea27fc700 1 -- 172.21.15.16:0/3186693149 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fde94005590 con 0x7fdebc09d6f0 2023-09-30T22:34:08.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.821+0000 7fdeb8ff9700 1 -- 172.21.15.16:0/3186693149 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdeac025bd0 con 0x7fdebc09d6f0 2023-09-30T22:34:08.821 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:34:08.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.823+0000 7fdea27fc700 1 -- 172.21.15.16:0/3186693149 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdebc09d6f0 msgr2=0x7fdebc1222e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:08.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.823+0000 7fdea27fc700 1 --2- 172.21.15.16:0/3186693149 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdebc09d6f0 0x7fdebc1222e0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fdeac009130 tx=0x7fdeac004750 comp rx=0 tx=0).stop 2023-09-30T22:34:08.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.823+0000 7fdea27fc700 1 -- 172.21.15.16:0/3186693149 shutdown_connections 2023-09-30T22:34:08.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.823+0000 7fdea27fc700 1 --2- 172.21.15.16:0/3186693149 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdebc09d6f0 0x7fdebc1222e0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:08.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.823+0000 7fdea27fc700 1 --2- 172.21.15.16:0/3186693149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdebc09e410 0x7fdebc122d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:08.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.823+0000 7fdea27fc700 1 --2- 172.21.15.16:0/3186693149 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdebc09e000 0x7fdebc122820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:08.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.823+0000 7fdea27fc700 1 -- 172.21.15.16:0/3186693149 >> 172.21.15.16:0/3186693149 conn(0x7fdebc066f50 msgr2=0x7fdebc133e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:08.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.823+0000 7fdea27fc700 1 -- 172.21.15.16:0/3186693149 shutdown_connections 2023-09-30T22:34:08.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.824+0000 7fdea27fc700 1 -- 172.21.15.16:0/3186693149 wait complete. 2023-09-30T22:34:08.838 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:34:08.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.866+0000 7fa8db290700 1 -- 172.21.15.16:0/1155877442 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa8d4074ea0 con 0x7fa8d41338c0 2023-09-30T22:34:08.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.867+0000 7fa8c9ffb700 1 -- 172.21.15.16:0/1155877442 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa8c401ab10 con 0x7fa8d41338c0 2023-09-30T22:34:08.867 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:34:08.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.869+0000 7fa8db290700 1 -- 172.21.15.16:0/1155877442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8d41338c0 msgr2=0x7fa8d411ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:08.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.869+0000 7fa8db290700 1 --2- 172.21.15.16:0/1155877442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8d41338c0 0x7fa8d411ee10 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fa8c400e900 tx=0x7fa8c400ec10 comp rx=0 tx=0).stop 2023-09-30T22:34:08.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.869+0000 7fa8db290700 1 -- 172.21.15.16:0/1155877442 shutdown_connections 2023-09-30T22:34:08.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.869+0000 7fa8db290700 1 --2- 172.21.15.16:0/1155877442 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa8d41341d0 0x7fa8d411f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:08.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.869+0000 7fa8db290700 1 --2- 172.21.15.16:0/1155877442 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa8d409e390 0x7fa8d411e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:08.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.869+0000 7fa8db290700 1 --2- 172.21.15.16:0/1155877442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8d41338c0 0x7fa8d411ee10 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:08.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.869+0000 7fa8db290700 1 -- 172.21.15.16:0/1155877442 >> 172.21.15.16:0/1155877442 conn(0x7fa8d40a2aa0 msgr2=0x7fa8d4118cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:08.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.869+0000 7fa8db290700 1 -- 172.21.15.16:0/1155877442 shutdown_connections 2023-09-30T22:34:08.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.869+0000 7fa8db290700 1 -- 172.21.15.16:0/1155877442 wait complete. 2023-09-30T22:34:08.883 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:34:08.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.915+0000 7f2229b34700 1 -- 172.21.15.16:0/3441814302 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f22240747f0 con 0x7f222409d630 2023-09-30T22:34:08.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.915+0000 7f220bfff700 1 -- 172.21.15.16:0/3441814302 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f221402b090 con 0x7f222409d630 2023-09-30T22:34:08.915 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:34:08.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.917+0000 7f2229b34700 1 -- 172.21.15.16:0/3441814302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f222409d630 msgr2=0x7f222411f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:08.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.917+0000 7f2229b34700 1 --2- 172.21.15.16:0/3441814302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f222409d630 0x7f222411f670 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f221400e9b0 tx=0x7f221400ecc0 comp rx=0 tx=0).stop 2023-09-30T22:34:08.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.917+0000 7f2229b34700 1 -- 172.21.15.16:0/3441814302 shutdown_connections 2023-09-30T22:34:08.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.917+0000 7f2229b34700 1 --2- 172.21.15.16:0/3441814302 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2224133930 0x7f22241c83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:08.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.917+0000 7f2229b34700 1 --2- 172.21.15.16:0/3441814302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f222409d630 0x7f222411f670 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:08.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.917+0000 7f2229b34700 1 --2- 172.21.15.16:0/3441814302 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f22240a0a30 0x7f222411fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:08.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.917+0000 7f2229b34700 1 -- 172.21.15.16:0/3441814302 >> 172.21.15.16:0/3441814302 conn(0x7f22240a3050 msgr2=0x7f22241bbb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:08.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.917+0000 7f2229b34700 1 -- 172.21.15.16:0/3441814302 shutdown_connections 2023-09-30T22:34:08.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:08.917+0000 7f2229b34700 1 -- 172.21.15.16:0/3441814302 wait complete. 2023-09-30T22:34:08.931 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:34:09.792 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:34:09.839 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:34:09.884 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:34:09.932 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:34:09.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:09.947+0000 7f427ad9d700 1 Processor -- start 2023-09-30T22:34:09.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:09.947+0000 7f427ad9d700 1 -- start start 2023-09-30T22:34:09.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:09.948+0000 7f427ad9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4274134630 0x7f4274136a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:09.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:09.948+0000 7f427ad9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4274136f50 0x7f4274139330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:09.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:09.948+0000 7f427ad9d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4274071a00 con 0x7f4274131630 2023-09-30T22:34:09.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:09.948+0000 7f427ad9d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4274131430 con 0x7f4274136f50 2023-09-30T22:34:09.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:09.948+0000 7f427ad9d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f427409a510 con 0x7f4274134630 2023-09-30T22:34:09.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:09.948+0000 7f427933a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4274136f50 0x7f4274139330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:09.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:09.948+0000 7f4278b39700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4274131630 0x7f4274131a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59748/0 (socket says 172.21.15.16:59748) 2023-09-30T22:34:09.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:09.948+0000 7f4278b39700 1 -- 172.21.15.16:0/1055282081 learned_addr learned my addr 172.21.15.16:0/1055282081 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:34:09.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:09.948+0000 7f42737fe700 1 -- 172.21.15.16:0/1055282081 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2406275612 0 0) 0x7f4274071a00 con 0x7f4274131630 2023-09-30T22:34:09.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:09.948+0000 7f42737fe700 1 -- 172.21.15.16:0/1055282081 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42640035b0 con 0x7f4274131630 2023-09-30T22:34:09.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:09.948+0000 7f427933a700 1 -- 172.21.15.16:0/1055282081 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4274134630 msgr2=0x7f4274136a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:09.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:09.948+0000 7f427933a700 1 --2- 172.21.15.16:0/1055282081 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4274134630 0x7f4274136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:09.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:09.948+0000 7f427933a700 1 -- 172.21.15.16:0/1055282081 >> v1:172.21.15.16:6789/0 conn(0x7f4274131630 legacy=0x7f4274131a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:09.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:09.948+0000 7f427933a700 1 -- 172.21.15.16:0/1055282081 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42740721c0 con 0x7f4274136f50 2023-09-30T22:34:09.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:09.949+0000 7f427933a700 1 --2- 172.21.15.16:0/1055282081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4274136f50 0x7f4274139330 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f426c009d90 tx=0x7f426c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41beef7b35be7497 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:09.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:09.949+0000 7f42737fe700 1 -- 172.21.15.16:0/1055282081 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c00e070 con 0x7f4274136f50 2023-09-30T22:34:09.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:09.949+0000 7f42737fe700 1 -- 172.21.15.16:0/1055282081 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f426c00bb20 con 0x7f4274136f50 2023-09-30T22:34:09.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:09.949+0000 7f42737fe700 1 -- 172.21.15.16:0/1055282081 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c012430 con 0x7f4274136f50 2023-09-30T22:34:09.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:09.950+0000 7f427ad9d700 1 -- 172.21.15.16:0/1055282081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4274136f50 msgr2=0x7f4274139330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:09.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:09.950+0000 7f427ad9d700 1 --2- 172.21.15.16:0/1055282081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4274136f50 0x7f4274139330 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f426c009d90 tx=0x7f426c00b330 comp rx=0 tx=0).stop 2023-09-30T22:34:09.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:09.950+0000 7f427ad9d700 1 -- 172.21.15.16:0/1055282081 shutdown_connections 2023-09-30T22:34:09.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:09.950+0000 7f427ad9d700 1 --2- 172.21.15.16:0/1055282081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4274136f50 0x7f4274139330 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:09.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:09.950+0000 7f427ad9d700 1 --2- 172.21.15.16:0/1055282081 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4274134630 0x7f4274136a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:09.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:09.950+0000 7f427ad9d700 1 -- 172.21.15.16:0/1055282081 >> 172.21.15.16:0/1055282081 conn(0x7f42740a2aa0 msgr2=0x7f42740a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:09.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:09.950+0000 7f427ad9d700 1 -- 172.21.15.16:0/1055282081 shutdown_connections 2023-09-30T22:34:09.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:09.950+0000 7f427ad9d700 1 -- 172.21.15.16:0/1055282081 wait complete. 2023-09-30T22:34:09.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:09.951+0000 7f427ad9d700 1 Processor -- start 2023-09-30T22:34:09.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:09.951+0000 7f427ad9d700 1 -- start start 2023-09-30T22:34:09.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:09.952+0000 7f427ad9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4274131630 0x7f42741c85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:09.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:09.952+0000 7f427ad9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4274134630 0x7f42741c8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:09.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:09.952+0000 7f427ad9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4274136f50 0x7f427406f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:09.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:09.952+0000 7f427ad9d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f427411e240 con 0x7f4274131630 2023-09-30T22:34:09.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:09.952+0000 7f427ad9d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4274099f20 con 0x7f4274136f50 2023-09-30T22:34:09.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:09.952+0000 7f427ad9d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f427412ec90 con 0x7f4274134630 2023-09-30T22:34:09.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:09.952+0000 7f4278b39700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4274131630 0x7f42741c85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:09.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:09.952+0000 7f4278b39700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4274131630 0x7f42741c85e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33370/0 (socket says 172.21.15.16:33370) 2023-09-30T22:34:09.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:09.952+0000 7f4278b39700 1 -- 172.21.15.16:0/2763850592 learned_addr learned my addr 172.21.15.16:0/2763850592 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:09.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:09.952+0000 7f4278b39700 1 -- 172.21.15.16:0/2763850592 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4274134630 msgr2=0x7f42741c8b20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:34:09.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:09.952+0000 7f4278b39700 1 --2- 172.21.15.16:0/2763850592 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4274134630 0x7f42741c8b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:09.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:09.952+0000 7f4278b39700 1 -- 172.21.15.16:0/2763850592 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4274136f50 msgr2=0x7f427406f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:09.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:09.952+0000 7f4278b39700 1 --2- 172.21.15.16:0/2763850592 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4274136f50 0x7f427406f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:09.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:09.952+0000 7f4278b39700 1 -- 172.21.15.16:0/2763850592 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f426c009a70 con 0x7f4274131630 2023-09-30T22:34:09.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:09.953+0000 7f4278b39700 1 --2- 172.21.15.16:0/2763850592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4274131630 0x7f42741c85e0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f426800c700 tx=0x7f426800cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:09.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:09.953+0000 7f42717fa700 1 -- 172.21.15.16:0/2763850592 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4268005d70 con 0x7f4274131630 2023-09-30T22:34:09.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:09.953+0000 7f42717fa700 1 -- 172.21.15.16:0/2763850592 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42680070d0 con 0x7f4274131630 2023-09-30T22:34:09.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:09.953+0000 7f427ad9d700 1 -- 172.21.15.16:0/2763850592 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4274071080 con 0x7f4274131630 2023-09-30T22:34:09.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:09.953+0000 7f42717fa700 1 -- 172.21.15.16:0/2763850592 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4268007460 con 0x7f4274131630 2023-09-30T22:34:09.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:09.953+0000 7f427ad9d700 1 -- 172.21.15.16:0/2763850592 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f427411ece0 con 0x7f4274131630 2023-09-30T22:34:09.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:09.954+0000 7f427ad9d700 1 -- 172.21.15.16:0/2763850592 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4274074ea0 con 0x7f4274131630 2023-09-30T22:34:09.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:09.954+0000 7f42717fa700 1 -- 172.21.15.16:0/2763850592 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f42680075c0 con 0x7f4274131630 2023-09-30T22:34:09.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:09.955+0000 7f42717fa700 1 -- 172.21.15.16:0/2763850592 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f426803edc0 con 0x7f4274131630 2023-09-30T22:34:09.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:09.964+0000 7f42717fa700 1 -- 172.21.15.16:0/2763850592 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f426800e280 con 0x7f4274131630 2023-09-30T22:34:10.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.002+0000 7f32bbbd5700 1 Processor -- start 2023-09-30T22:34:10.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.002+0000 7f32bbbd5700 1 -- start start 2023-09-30T22:34:10.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.003+0000 7f32bbbd5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f32b4136710 0x7f32b4138af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:10.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.003+0000 7f32bbbd5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f32b4139030 0x7f32b413b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:10.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.003+0000 7f32bbbd5700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32b406ef30 con 0x7f32b41338a0 2023-09-30T22:34:10.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.003+0000 7f32bbbd5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32b406fef0 con 0x7f32b4136710 2023-09-30T22:34:10.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.003+0000 7f32bbbd5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f32b41334b0 con 0x7f32b4139030 2023-09-30T22:34:10.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.003+0000 7f32b9971700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f32b41338a0 0x7f32b4133c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59764/0 (socket says 172.21.15.16:59764) 2023-09-30T22:34:10.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.003+0000 7f32b9971700 1 -- 172.21.15.16:0/2830555046 learned_addr learned my addr 172.21.15.16:0/2830555046 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:34:10.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.003+0000 7f32ba172700 1 --2- 172.21.15.16:0/2830555046 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f32b4139030 0x7f32b413b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:10.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.003+0000 7f32b9170700 1 --2- 172.21.15.16:0/2830555046 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f32b4136710 0x7f32b4138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:10.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.003+0000 7f32b896f700 1 -- 172.21.15.16:0/2830555046 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 861803969 0 0) 0x7f32b406ef30 con 0x7f32b41338a0 2023-09-30T22:34:10.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.004+0000 7f32b896f700 1 -- 172.21.15.16:0/2830555046 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32a00035b0 con 0x7f32b41338a0 2023-09-30T22:34:10.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.004+0000 7f32ba172700 1 -- 172.21.15.16:0/2830555046 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f32b4136710 msgr2=0x7f32b4138af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:10.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.004+0000 7f32ba172700 1 --2- 172.21.15.16:0/2830555046 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f32b4136710 0x7f32b4138af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:10.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.004+0000 7f32ba172700 1 -- 172.21.15.16:0/2830555046 >> v1:172.21.15.16:6789/0 conn(0x7f32b41338a0 legacy=0x7f32b4133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:10.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.004+0000 7f32ba172700 1 -- 172.21.15.16:0/2830555046 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b406fc30 con 0x7f32b4139030 2023-09-30T22:34:10.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.004+0000 7f32ba172700 1 --2- 172.21.15.16:0/2830555046 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f32b4139030 0x7f32b413b410 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f32ac009dc0 tx=0x7f32ac00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a972fca8d2ada26 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:10.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.004+0000 7f32b896f700 1 -- 172.21.15.16:0/2830555046 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ac00e070 con 0x7f32b4139030 2023-09-30T22:34:10.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.004+0000 7f32b896f700 1 -- 172.21.15.16:0/2830555046 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32ac00bb20 con 0x7f32b4139030 2023-09-30T22:34:10.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.004+0000 7f32b896f700 1 -- 172.21.15.16:0/2830555046 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ac00be90 con 0x7f32b4139030 2023-09-30T22:34:10.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.005+0000 7f32bbbd5700 1 -- 172.21.15.16:0/2830555046 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f32b4139030 msgr2=0x7f32b413b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:10.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.005+0000 7f32bbbd5700 1 --2- 172.21.15.16:0/2830555046 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f32b4139030 0x7f32b413b410 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f32ac009dc0 tx=0x7f32ac00b330 comp rx=0 tx=0).stop 2023-09-30T22:34:10.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.005+0000 7f32bbbd5700 1 -- 172.21.15.16:0/2830555046 shutdown_connections 2023-09-30T22:34:10.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.005+0000 7f32bbbd5700 1 --2- 172.21.15.16:0/2830555046 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f32b4139030 0x7f32b413b410 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:10.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.005+0000 7f32bbbd5700 1 --2- 172.21.15.16:0/2830555046 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f32b4136710 0x7f32b4138af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:10.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.005+0000 7f32bbbd5700 1 -- 172.21.15.16:0/2830555046 >> 172.21.15.16:0/2830555046 conn(0x7f32b40a2aa0 msgr2=0x7f32b40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:10.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.005+0000 7f32bbbd5700 1 -- 172.21.15.16:0/2830555046 shutdown_connections 2023-09-30T22:34:10.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.005+0000 7f32bbbd5700 1 -- 172.21.15.16:0/2830555046 wait complete. 2023-09-30T22:34:10.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.007+0000 7f32bbbd5700 1 Processor -- start 2023-09-30T22:34:10.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.007+0000 7f32bbbd5700 1 -- start start 2023-09-30T22:34:10.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.007+0000 7f32bbbd5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32b41338a0 0x7f32b411ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:10.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.007+0000 7f32bbbd5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f32b4136710 0x7f32b411b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:10.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.007+0000 7f32bbbd5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f32b4139030 0x7f32b411b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:10.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.007+0000 7f32bbbd5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b411deb0 con 0x7f32b41338a0 2023-09-30T22:34:10.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.007+0000 7f32bbbd5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32b411e720 con 0x7f32b4136710 2023-09-30T22:34:10.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.007+0000 7f32bbbd5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f32b4130a50 con 0x7f32b4139030 2023-09-30T22:34:10.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.008+0000 7f32ba172700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f32b4139030 0x7f32b411b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:10.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.008+0000 7f32b9971700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32b41338a0 0x7f32b411ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:10.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.008+0000 7f32ba172700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f32b4139030 0x7f32b411b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35060/0 (socket says 172.21.15.16:35060) 2023-09-30T22:34:10.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.008+0000 7f32ba172700 1 -- 172.21.15.16:0/8358323 learned_addr learned my addr 172.21.15.16:0/8358323 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:10.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.008+0000 7f32b9971700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32b41338a0 0x7f32b411ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33386/0 (socket says 172.21.15.16:33386) 2023-09-30T22:34:10.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.008+0000 7f32b9170700 1 --2- 172.21.15.16:0/8358323 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f32b4136710 0x7f32b411b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:10.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.008+0000 7f32b9971700 1 -- 172.21.15.16:0/8358323 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f32b4139030 msgr2=0x7f32b411b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:10.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.008+0000 7f32b9971700 1 --2- 172.21.15.16:0/8358323 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f32b4139030 0x7f32b411b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:10.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.008+0000 7f32b9971700 1 -- 172.21.15.16:0/8358323 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f32b4136710 msgr2=0x7f32b411b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:10.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.008+0000 7f32b9971700 1 --2- 172.21.15.16:0/8358323 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f32b4136710 0x7f32b411b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:10.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.008+0000 7f32b9971700 1 -- 172.21.15.16:0/8358323 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32ac009a70 con 0x7f32b41338a0 2023-09-30T22:34:10.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.008+0000 7f32b9971700 1 --2- 172.21.15.16:0/8358323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32b41338a0 0x7f32b411ab60 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f32b000c4c0 tx=0x7f32b000c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:10.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.008+0000 7f32aa7fc700 1 -- 172.21.15.16:0/8358323 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b0005d70 con 0x7f32b41338a0 2023-09-30T22:34:10.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.008+0000 7f32aa7fc700 1 -- 172.21.15.16:0/8358323 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32b00070d0 con 0x7f32b41338a0 2023-09-30T22:34:10.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.008+0000 7f32aa7fc700 1 -- 172.21.15.16:0/8358323 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b0007440 con 0x7f32b41338a0 2023-09-30T22:34:10.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.008+0000 7f32bbbd5700 1 -- 172.21.15.16:0/8358323 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32b411cfd0 con 0x7f32b41338a0 2023-09-30T22:34:10.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.009+0000 7f32bbbd5700 1 -- 172.21.15.16:0/8358323 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32b411e3a0 con 0x7f32b41338a0 2023-09-30T22:34:10.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.010+0000 7f32bbbd5700 1 -- 172.21.15.16:0/8358323 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32b4074ea0 con 0x7f32b41338a0 2023-09-30T22:34:10.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.010+0000 7f32aa7fc700 1 -- 172.21.15.16:0/8358323 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f32b00075a0 con 0x7f32b41338a0 2023-09-30T22:34:10.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.010+0000 7f32aa7fc700 1 -- 172.21.15.16:0/8358323 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f32b003e720 con 0x7f32b41338a0 2023-09-30T22:34:10.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.015+0000 7f32aa7fc700 1 -- 172.21.15.16:0/8358323 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f32b0018af0 con 0x7f32b41338a0 2023-09-30T22:34:10.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.035+0000 7f9f215dc700 1 Processor -- start 2023-09-30T22:34:10.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.035+0000 7f9f215dc700 1 -- start start 2023-09-30T22:34:10.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.036+0000 7f9f215dc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9f1c09d6f0 0x7f9f1c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:10.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.036+0000 7f9f215dc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9f1c09e000 0x7f9f1c130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:10.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.036+0000 7f9f215dc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9f1c09e410 0x7f9f1c130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:10.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.036+0000 7f9f215dc700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f1c09f520 con 0x7f9f1c09d6f0 2023-09-30T22:34:10.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.036+0000 7f9f215dc700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1c09f130 con 0x7f9f1c09e410 2023-09-30T22:34:10.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.036+0000 7f9f215dc700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1c09e930 con 0x7f9f1c09e000 2023-09-30T22:34:10.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.037+0000 7f9f1affd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9f1c09d6f0 0x7f9f1c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:10.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.037+0000 7f9f1affd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9f1c09d6f0 0x7f9f1c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33390/0 (socket says 172.21.15.16:33390) 2023-09-30T22:34:10.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.037+0000 7f9f1affd700 1 -- 172.21.15.16:0/3327357149 learned_addr learned my addr 172.21.15.16:0/3327357149 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:10.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.037+0000 7f9f1a7fc700 1 --2- 172.21.15.16:0/3327357149 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9f1c09e000 0x7f9f1c130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:10.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.037+0000 7f9f1b7fe700 1 --2- 172.21.15.16:0/3327357149 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9f1c09e410 0x7f9f1c130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:10.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.037+0000 7f9f1affd700 1 -- 172.21.15.16:0/3327357149 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9f1c09e000 msgr2=0x7f9f1c130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:10.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.037+0000 7f9f1affd700 1 --2- 172.21.15.16:0/3327357149 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9f1c09e000 0x7f9f1c130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:10.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.037+0000 7f9f1affd700 1 -- 172.21.15.16:0/3327357149 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9f1c09e410 msgr2=0x7f9f1c130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:10.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.037+0000 7f9f1affd700 1 --2- 172.21.15.16:0/3327357149 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9f1c09e410 0x7f9f1c130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:10.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.037+0000 7f9f1affd700 1 -- 172.21.15.16:0/3327357149 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f1c09b2d0 con 0x7f9f1c09d6f0 2023-09-30T22:34:10.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.037+0000 7f9f1affd700 1 --2- 172.21.15.16:0/3327357149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f1c09d6f0 0x7f9f1c09dac0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c009920 tx=0x7f9f0c009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b289a4ba8b17d1bc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:10.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.038+0000 7f9f19ffb700 1 -- 172.21.15.16:0/3327357149 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f0c00ba20 con 0x7f9f1c09d6f0 2023-09-30T22:34:10.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.038+0000 7f9f19ffb700 1 -- 172.21.15.16:0/3327357149 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f0c00bb80 con 0x7f9f1c09d6f0 2023-09-30T22:34:10.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.038+0000 7f9f19ffb700 1 -- 172.21.15.16:0/3327357149 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f0c009e70 con 0x7f9f1c09d6f0 2023-09-30T22:34:10.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.039+0000 7f9f215dc700 1 -- 172.21.15.16:0/3327357149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f1c09d6f0 msgr2=0x7f9f1c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:10.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.039+0000 7f9f215dc700 1 --2- 172.21.15.16:0/3327357149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f1c09d6f0 0x7f9f1c09dac0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c009920 tx=0x7f9f0c009c30 comp rx=0 tx=0).stop 2023-09-30T22:34:10.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.039+0000 7f9f215dc700 1 -- 172.21.15.16:0/3327357149 shutdown_connections 2023-09-30T22:34:10.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.039+0000 7f9f215dc700 1 --2- 172.21.15.16:0/3327357149 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9f1c09e410 0x7f9f1c130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:10.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.039+0000 7f9f215dc700 1 --2- 172.21.15.16:0/3327357149 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f1c09d6f0 0x7f9f1c09dac0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:10.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.039+0000 7f9f215dc700 1 --2- 172.21.15.16:0/3327357149 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9f1c09e000 0x7f9f1c130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:10.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.039+0000 7f9f215dc700 1 -- 172.21.15.16:0/3327357149 >> 172.21.15.16:0/3327357149 conn(0x7f9f1c066f50 msgr2=0x7f9f1c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:10.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.039+0000 7f9f215dc700 1 -- 172.21.15.16:0/3327357149 shutdown_connections 2023-09-30T22:34:10.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.039+0000 7f9f215dc700 1 -- 172.21.15.16:0/3327357149 wait complete. 2023-09-30T22:34:10.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.041+0000 7f9f215dc700 1 Processor -- start 2023-09-30T22:34:10.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.041+0000 7f9f215dc700 1 -- start start 2023-09-30T22:34:10.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.041+0000 7f9f215dc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f1c09d6f0 0x7f9f1c122430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:10.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.041+0000 7f9f215dc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9f1c09e000 0x7f9f1c122970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:10.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.041+0000 7f9f215dc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9f1c09e410 0x7f9f1c122eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:10.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.041+0000 7f9f215dc700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1c123fc0 con 0x7f9f1c09d6f0 2023-09-30T22:34:10.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.041+0000 7f9f215dc700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1c125b50 con 0x7f9f1c09e000 2023-09-30T22:34:10.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.041+0000 7f9f215dc700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1c0a16b0 con 0x7f9f1c09e410 2023-09-30T22:34:10.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.042+0000 7f9f1affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f1c09d6f0 0x7f9f1c122430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:10.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.042+0000 7f9f1affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f1c09d6f0 0x7f9f1c122430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33396/0 (socket says 172.21.15.16:33396) 2023-09-30T22:34:10.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.042+0000 7f9f1affd700 1 -- 172.21.15.16:0/1779596391 learned_addr learned my addr 172.21.15.16:0/1779596391 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:10.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.042+0000 7f9f1a7fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9f1c09e000 0x7f9f1c122970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:10.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.042+0000 7f9f1b7fe700 1 --2- 172.21.15.16:0/1779596391 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9f1c09e410 0x7f9f1c122eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:10.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.042+0000 7f9f1affd700 1 -- 172.21.15.16:0/1779596391 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9f1c09e410 msgr2=0x7f9f1c122eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:10.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.042+0000 7f9f1affd700 1 --2- 172.21.15.16:0/1779596391 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9f1c09e410 0x7f9f1c122eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:10.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.042+0000 7f9f1affd700 1 -- 172.21.15.16:0/1779596391 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9f1c09e000 msgr2=0x7f9f1c122970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:10.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.042+0000 7f9f1affd700 1 --2- 172.21.15.16:0/1779596391 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9f1c09e000 0x7f9f1c122970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:10.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.042+0000 7f9f1affd700 1 -- 172.21.15.16:0/1779596391 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f0c009550 con 0x7f9f1c09d6f0 2023-09-30T22:34:10.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.042+0000 7f9f1a7fc700 1 --2- 172.21.15.16:0/1779596391 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9f1c09e000 0x7f9f1c122970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:10.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.042+0000 7f9f1affd700 1 --2- 172.21.15.16:0/1779596391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f1c09d6f0 0x7f9f1c122430 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c009c70 tx=0x7f9f0c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:10.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.043+0000 7f9f037fe700 1 -- 172.21.15.16:0/1779596391 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f0c004510 con 0x7f9f1c09d6f0 2023-09-30T22:34:10.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.043+0000 7f9f037fe700 1 -- 172.21.15.16:0/1779596391 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f0c0036a0 con 0x7f9f1c09d6f0 2023-09-30T22:34:10.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.043+0000 7f9f215dc700 1 -- 172.21.15.16:0/1779596391 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f1c09c700 con 0x7f9f1c09d6f0 2023-09-30T22:34:10.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.043+0000 7f9f037fe700 1 -- 172.21.15.16:0/1779596391 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f0c003a50 con 0x7f9f1c09d6f0 2023-09-30T22:34:10.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.043+0000 7f9f215dc700 1 -- 172.21.15.16:0/1779596391 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f1c1265c0 con 0x7f9f1c09d6f0 2023-09-30T22:34:10.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.044+0000 7f9f037fe700 1 -- 172.21.15.16:0/1779596391 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9f0c003bb0 con 0x7f9f1c09d6f0 2023-09-30T22:34:10.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.044+0000 7f9f037fe700 1 -- 172.21.15.16:0/1779596391 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f9f0c021030 con 0x7f9f1c09d6f0 2023-09-30T22:34:10.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.045+0000 7f9f215dc700 1 -- 172.21.15.16:0/1779596391 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f1c074ea0 con 0x7f9f1c09d6f0 2023-09-30T22:34:10.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.054+0000 7f9f037fe700 1 -- 172.21.15.16:0/1779596391 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9f0c015080 con 0x7f9f1c09d6f0 2023-09-30T22:34:10.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.087+0000 7f1a8dc45700 1 Processor -- start 2023-09-30T22:34:10.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.087+0000 7f1a8dc45700 1 -- start start 2023-09-30T22:34:10.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.087+0000 7f1a8dc45700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1a8806e570 0x7f1a8806e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:10.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.087+0000 7f1a8dc45700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1a8809eca0 0x7f1a8806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:10.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.087+0000 7f1a8dc45700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1a88134610 0x7f1a88134a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:10.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.087+0000 7f1a8dc45700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a88070ee0 con 0x7f1a8806e570 2023-09-30T22:34:10.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.087+0000 7f1a8dc45700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a88071480 con 0x7f1a8809eca0 2023-09-30T22:34:10.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.087+0000 7f1a8dc45700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8806f3c0 con 0x7f1a88134610 2023-09-30T22:34:10.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.087+0000 7f1a87fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1a88134610 0x7f1a88134a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:10.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.087+0000 7f1a87fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1a88134610 0x7f1a88134a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35088/0 (socket says 172.21.15.16:35088) 2023-09-30T22:34:10.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.087+0000 7f1a87fff700 1 -- 172.21.15.16:0/3832495505 learned_addr learned my addr 172.21.15.16:0/3832495505 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:10.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.087+0000 7f1a86ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1a8809eca0 0x7f1a8806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:10.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.088+0000 7f1a877fe700 1 --2- 172.21.15.16:0/3832495505 >> v2:172.21.15.16:3300/0 conn(0x7f1a8806e570 0x7f1a8806e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:10.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.088+0000 7f1a87fff700 1 -- 172.21.15.16:0/3832495505 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1a8809eca0 msgr2=0x7f1a8806ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:10.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.088+0000 7f1a87fff700 1 --2- 172.21.15.16:0/3832495505 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1a8809eca0 0x7f1a8806ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:10.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.088+0000 7f1a87fff700 1 -- 172.21.15.16:0/3832495505 >> v2:172.21.15.16:3300/0 conn(0x7f1a8806e570 msgr2=0x7f1a8806e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:10.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.088+0000 7f1a87fff700 1 --2- 172.21.15.16:0/3832495505 >> v2:172.21.15.16:3300/0 conn(0x7f1a8806e570 0x7f1a8806e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:10.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.088+0000 7f1a87fff700 1 -- 172.21.15.16:0/3832495505 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a88071b90 con 0x7f1a88134610 2023-09-30T22:34:10.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.088+0000 7f1a87fff700 1 --2- 172.21.15.16:0/3832495505 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1a88134610 0x7f1a88134a90 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f1a80009e50 tx=0x7f1a8000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7cea87ec1f92ffa server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:10.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.088+0000 7f1a867fc700 1 -- 172.21.15.16:0/3832495505 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8000e070 con 0x7f1a88134610 2023-09-30T22:34:10.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.088+0000 7f1a867fc700 1 -- 172.21.15.16:0/3832495505 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a8000b910 con 0x7f1a88134610 2023-09-30T22:34:10.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.089+0000 7f1a867fc700 1 -- 172.21.15.16:0/3832495505 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8000bcc0 con 0x7f1a88134610 2023-09-30T22:34:10.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.089+0000 7f1a8dc45700 1 -- 172.21.15.16:0/3832495505 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1a88134610 msgr2=0x7f1a88134a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:10.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.089+0000 7f1a8dc45700 1 --2- 172.21.15.16:0/3832495505 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1a88134610 0x7f1a88134a90 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f1a80009e50 tx=0x7f1a8000b330 comp rx=0 tx=0).stop 2023-09-30T22:34:10.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.089+0000 7f1a8dc45700 1 -- 172.21.15.16:0/3832495505 shutdown_connections 2023-09-30T22:34:10.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.089+0000 7f1a8dc45700 1 --2- 172.21.15.16:0/3832495505 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1a88134610 0x7f1a88134a90 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:10.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.089+0000 7f1a8dc45700 1 --2- 172.21.15.16:0/3832495505 >> v2:172.21.15.16:3300/0 conn(0x7f1a8806e570 0x7f1a8806e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:10.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.089+0000 7f1a8dc45700 1 --2- 172.21.15.16:0/3832495505 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1a8809eca0 0x7f1a8806ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:10.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.089+0000 7f1a8dc45700 1 -- 172.21.15.16:0/3832495505 >> 172.21.15.16:0/3832495505 conn(0x7f1a880a2aa0 msgr2=0x7f1a880a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:10.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.089+0000 7f1a8dc45700 1 -- 172.21.15.16:0/3832495505 shutdown_connections 2023-09-30T22:34:10.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.089+0000 7f1a8dc45700 1 -- 172.21.15.16:0/3832495505 wait complete. 2023-09-30T22:34:10.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.090+0000 7f1a8dc45700 1 Processor -- start 2023-09-30T22:34:10.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.091+0000 7f1a8dc45700 1 -- start start 2023-09-30T22:34:10.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.091+0000 7f1a8dc45700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1a8806e570 0x7f1a88127570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:10.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.091+0000 7f1a8dc45700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a88134610 0x7f1a8809bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:10.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.091+0000 7f1a8dc45700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1a8809c1f0 0x7f1a8809c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:10.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.091+0000 7f1a8dc45700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8809d880 con 0x7f1a88134610 2023-09-30T22:34:10.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.091+0000 7f1a8dc45700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a88070610 con 0x7f1a8806e570 2023-09-30T22:34:10.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.091+0000 7f1a8dc45700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1a88139130 con 0x7f1a8809c1f0 2023-09-30T22:34:10.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.091+0000 7f1a87fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1a8809c1f0 0x7f1a8809c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:10.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.091+0000 7f1a87fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1a8809c1f0 0x7f1a8809c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35096/0 (socket says 172.21.15.16:35096) 2023-09-30T22:34:10.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.091+0000 7f1a87fff700 1 -- 172.21.15.16:0/148456782 learned_addr learned my addr 172.21.15.16:0/148456782 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:10.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.091+0000 7f1a877fe700 1 --2- 172.21.15.16:0/148456782 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1a8806e570 0x7f1a88127570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:10.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.092+0000 7f1a87fff700 1 -- 172.21.15.16:0/148456782 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1a8806e570 msgr2=0x7f1a88127570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:10.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.092+0000 7f1a87fff700 1 --2- 172.21.15.16:0/148456782 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1a8806e570 0x7f1a88127570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:10.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.092+0000 7f1a87fff700 1 -- 172.21.15.16:0/148456782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a88134610 msgr2=0x7f1a8809bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:10.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.092+0000 7f1a87fff700 1 --2- 172.21.15.16:0/148456782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a88134610 0x7f1a8809bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:10.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.092+0000 7f1a87fff700 1 -- 172.21.15.16:0/148456782 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a80009b30 con 0x7f1a8809c1f0 2023-09-30T22:34:10.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.092+0000 7f1a877fe700 1 --2- 172.21.15.16:0/148456782 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1a8806e570 0x7f1a88127570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:34:10.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.092+0000 7f1a87fff700 1 --2- 172.21.15.16:0/148456782 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1a8809c1f0 0x7f1a8809c670 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f1a8806f500 tx=0x7f1a80004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:10.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.092+0000 7f1a6ffff700 1 -- 172.21.15.16:0/148456782 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a80004450 con 0x7f1a8809c1f0 2023-09-30T22:34:10.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.092+0000 7f1a6ffff700 1 -- 172.21.15.16:0/148456782 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a800045b0 con 0x7f1a8809c1f0 2023-09-30T22:34:10.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.092+0000 7f1a8dc45700 1 -- 172.21.15.16:0/148456782 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a8809de00 con 0x7f1a8809c1f0 2023-09-30T22:34:10.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.092+0000 7f1a6ffff700 1 -- 172.21.15.16:0/148456782 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a80004960 con 0x7f1a8809c1f0 2023-09-30T22:34:10.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.092+0000 7f1a8dc45700 1 -- 172.21.15.16:0/148456782 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a88139e60 con 0x7f1a8809c1f0 2023-09-30T22:34:10.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.093+0000 7f1a6ffff700 1 -- 172.21.15.16:0/148456782 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1a8001c410 con 0x7f1a8809c1f0 2023-09-30T22:34:10.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.093+0000 7f1a6ffff700 1 -- 172.21.15.16:0/148456782 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f1a80040f50 con 0x7f1a8809c1f0 2023-09-30T22:34:10.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.094+0000 7f1a8dc45700 1 -- 172.21.15.16:0/148456782 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a88074ea0 con 0x7f1a8809c1f0 2023-09-30T22:34:10.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.099+0000 7f1a6ffff700 1 -- 172.21.15.16:0/148456782 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1a8002a080 con 0x7f1a8809c1f0 2023-09-30T22:34:10.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.194+0000 7f427ad9d700 1 -- 172.21.15.16:0/2763850592 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f427406b9e0 con 0x7f4274131630 2023-09-30T22:34:10.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.194+0000 7f42717fa700 1 -- 172.21.15.16:0/2763850592 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4268017020 con 0x7f4274131630 2023-09-30T22:34:10.195 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:34:10.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.197+0000 7f427ad9d700 1 -- 172.21.15.16:0/2763850592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4274131630 msgr2=0x7f42741c85e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:10.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.197+0000 7f427ad9d700 1 --2- 172.21.15.16:0/2763850592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4274131630 0x7f42741c85e0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f426800c700 tx=0x7f426800cac0 comp rx=0 tx=0).stop 2023-09-30T22:34:10.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.197+0000 7f427ad9d700 1 -- 172.21.15.16:0/2763850592 shutdown_connections 2023-09-30T22:34:10.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.197+0000 7f427ad9d700 1 --2- 172.21.15.16:0/2763850592 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4274136f50 0x7f427406f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:10.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.197+0000 7f427ad9d700 1 --2- 172.21.15.16:0/2763850592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4274131630 0x7f42741c85e0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:10.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.197+0000 7f427ad9d700 1 --2- 172.21.15.16:0/2763850592 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4274134630 0x7f42741c8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:10.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.197+0000 7f427ad9d700 1 -- 172.21.15.16:0/2763850592 >> 172.21.15.16:0/2763850592 conn(0x7f42740a2aa0 msgr2=0x7f42740a0890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:10.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.197+0000 7f427ad9d700 1 -- 172.21.15.16:0/2763850592 shutdown_connections 2023-09-30T22:34:10.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.197+0000 7f427ad9d700 1 -- 172.21.15.16:0/2763850592 wait complete. 2023-09-30T22:34:10.217 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:34:10.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.238+0000 7f32bbbd5700 1 -- 172.21.15.16:0/8358323 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f32b406b9e0 con 0x7f32b41338a0 2023-09-30T22:34:10.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.238+0000 7f32aa7fc700 1 -- 172.21.15.16:0/8358323 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f32b0017020 con 0x7f32b41338a0 2023-09-30T22:34:10.239 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:34:10.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.241+0000 7f32bbbd5700 1 -- 172.21.15.16:0/8358323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32b41338a0 msgr2=0x7f32b411ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:10.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.241+0000 7f32bbbd5700 1 --2- 172.21.15.16:0/8358323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32b41338a0 0x7f32b411ab60 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f32b000c4c0 tx=0x7f32b000c7d0 comp rx=0 tx=0).stop 2023-09-30T22:34:10.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.241+0000 7f32bbbd5700 1 -- 172.21.15.16:0/8358323 shutdown_connections 2023-09-30T22:34:10.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.241+0000 7f32bbbd5700 1 --2- 172.21.15.16:0/8358323 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f32b4139030 0x7f32b411b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:10.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.241+0000 7f32bbbd5700 1 --2- 172.21.15.16:0/8358323 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32b41338a0 0x7f32b411ab60 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:10.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.241+0000 7f32bbbd5700 1 --2- 172.21.15.16:0/8358323 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f32b4136710 0x7f32b411b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:10.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.241+0000 7f32bbbd5700 1 -- 172.21.15.16:0/8358323 >> 172.21.15.16:0/8358323 conn(0x7f32b40a2aa0 msgr2=0x7f32b41344f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:10.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.241+0000 7f32bbbd5700 1 -- 172.21.15.16:0/8358323 shutdown_connections 2023-09-30T22:34:10.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.241+0000 7f32bbbd5700 1 -- 172.21.15.16:0/8358323 wait complete. 2023-09-30T22:34:10.255 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:34:10.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.266+0000 7f9f215dc700 1 -- 172.21.15.16:0/1779596391 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9f1c06c360 con 0x7f9f1c09d6f0 2023-09-30T22:34:10.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.266+0000 7f9f037fe700 1 -- 172.21.15.16:0/1779596391 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9f0c035090 con 0x7f9f1c09d6f0 2023-09-30T22:34:10.267 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:34:10.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.268+0000 7f9f215dc700 1 -- 172.21.15.16:0/1779596391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f1c09d6f0 msgr2=0x7f9f1c122430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:10.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.268+0000 7f9f215dc700 1 --2- 172.21.15.16:0/1779596391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f1c09d6f0 0x7f9f1c122430 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c009c70 tx=0x7f9f0c004750 comp rx=0 tx=0).stop 2023-09-30T22:34:10.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.268+0000 7f9f215dc700 1 -- 172.21.15.16:0/1779596391 shutdown_connections 2023-09-30T22:34:10.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.268+0000 7f9f215dc700 1 --2- 172.21.15.16:0/1779596391 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9f1c09e410 0x7f9f1c122eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:10.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.268+0000 7f9f215dc700 1 --2- 172.21.15.16:0/1779596391 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f1c09d6f0 0x7f9f1c122430 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:10.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.268+0000 7f9f215dc700 1 --2- 172.21.15.16:0/1779596391 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9f1c09e000 0x7f9f1c122970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:10.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.268+0000 7f9f215dc700 1 -- 172.21.15.16:0/1779596391 >> 172.21.15.16:0/1779596391 conn(0x7f9f1c066f50 msgr2=0x7f9f1c130920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:10.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.268+0000 7f9f215dc700 1 -- 172.21.15.16:0/1779596391 shutdown_connections 2023-09-30T22:34:10.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.269+0000 7f9f215dc700 1 -- 172.21.15.16:0/1779596391 wait complete. 2023-09-30T22:34:10.282 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:34:10.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.305+0000 7f1a8dc45700 1 -- 172.21.15.16:0/148456782 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1a88071100 con 0x7f1a8809c1f0 2023-09-30T22:34:10.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.306+0000 7f1a6ffff700 1 -- 172.21.15.16:0/148456782 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1a8001c6c0 con 0x7f1a8809c1f0 2023-09-30T22:34:10.306 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:34:10.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.308+0000 7f1a8dc45700 1 -- 172.21.15.16:0/148456782 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1a8809c1f0 msgr2=0x7f1a8809c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:10.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.308+0000 7f1a8dc45700 1 --2- 172.21.15.16:0/148456782 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1a8809c1f0 0x7f1a8809c670 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f1a8806f500 tx=0x7f1a80004ad0 comp rx=0 tx=0).stop 2023-09-30T22:34:10.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.308+0000 7f1a8dc45700 1 -- 172.21.15.16:0/148456782 shutdown_connections 2023-09-30T22:34:10.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.308+0000 7f1a8dc45700 1 --2- 172.21.15.16:0/148456782 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1a8806e570 0x7f1a88127570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:10.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.308+0000 7f1a8dc45700 1 --2- 172.21.15.16:0/148456782 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1a8809c1f0 0x7f1a8809c670 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:10.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.308+0000 7f1a8dc45700 1 --2- 172.21.15.16:0/148456782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1a88134610 0x7f1a8809bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:10.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.308+0000 7f1a8dc45700 1 -- 172.21.15.16:0/148456782 >> 172.21.15.16:0/148456782 conn(0x7f1a880a2aa0 msgr2=0x7f1a88136660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:10.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.308+0000 7f1a8dc45700 1 -- 172.21.15.16:0/148456782 shutdown_connections 2023-09-30T22:34:10.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:10.308+0000 7f1a8dc45700 1 -- 172.21.15.16:0/148456782 wait complete. 2023-09-30T22:34:10.322 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:34:11.218 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:34:11.256 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:34:11.284 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:34:11.322 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:34:11.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.369+0000 7f18daf7a700 1 Processor -- start 2023-09-30T22:34:11.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.369+0000 7f18daf7a700 1 -- start start 2023-09-30T22:34:11.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.370+0000 7f18daf7a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f18d41338a0 0x7f18d4133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:11.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.370+0000 7f18daf7a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f18d409e930 0x7f18d413b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:11.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.370+0000 7f18daf7a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18d406ef30 con 0x7f18d409de50 2023-09-30T22:34:11.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.370+0000 7f18daf7a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18d406fef0 con 0x7f18d409e930 2023-09-30T22:34:11.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.370+0000 7f18daf7a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f18d41334b0 con 0x7f18d41338a0 2023-09-30T22:34:11.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.370+0000 7f18d8d16700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f18d41338a0 0x7f18d4133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:11.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.370+0000 7f18d8d16700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f18d41338a0 0x7f18d4133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35112/0 (socket says 172.21.15.16:35112) 2023-09-30T22:34:11.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.370+0000 7f18d8d16700 1 -- 172.21.15.16:0/1718607896 learned_addr learned my addr 172.21.15.16:0/1718607896 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:11.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.370+0000 7f18d9517700 1 --2- 172.21.15.16:0/1718607896 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f18d409e930 0x7f18d413b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:11.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.370+0000 7f18d8d16700 1 -- 172.21.15.16:0/1718607896 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f18d409e930 msgr2=0x7f18d413b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:11.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.370+0000 7f18d8d16700 1 --2- 172.21.15.16:0/1718607896 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f18d409e930 0x7f18d413b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:11.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.370+0000 7f18d8d16700 1 -- 172.21.15.16:0/1718607896 >> v1:172.21.15.16:6789/0 conn(0x7f18d409de50 legacy=0x7f18d409e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:11.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.370+0000 7f18d8d16700 1 -- 172.21.15.16:0/1718607896 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d406fc30 con 0x7f18d41338a0 2023-09-30T22:34:11.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.371+0000 7f18d8d16700 1 --2- 172.21.15.16:0/1718607896 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f18d41338a0 0x7f18d4133c70 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f18c8009920 tx=0x7f18c8009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=612348512238553 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:11.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.371+0000 7f18d37fe700 1 -- 172.21.15.16:0/1718607896 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c8004230 con 0x7f18d41338a0 2023-09-30T22:34:11.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.371+0000 7f18d37fe700 1 -- 172.21.15.16:0/1718607896 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18c8004390 con 0x7f18d41338a0 2023-09-30T22:34:11.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.371+0000 7f18d37fe700 1 -- 172.21.15.16:0/1718607896 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c80036a0 con 0x7f18d41338a0 2023-09-30T22:34:11.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.372+0000 7f18daf7a700 1 -- 172.21.15.16:0/1718607896 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f18d41338a0 msgr2=0x7f18d4133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:11.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.372+0000 7f18daf7a700 1 --2- 172.21.15.16:0/1718607896 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f18d41338a0 0x7f18d4133c70 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f18c8009920 tx=0x7f18c8009c30 comp rx=0 tx=0).stop 2023-09-30T22:34:11.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.372+0000 7f18daf7a700 1 -- 172.21.15.16:0/1718607896 shutdown_connections 2023-09-30T22:34:11.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.372+0000 7f18daf7a700 1 --2- 172.21.15.16:0/1718607896 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f18d409e930 0x7f18d413b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:11.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.372+0000 7f18daf7a700 1 --2- 172.21.15.16:0/1718607896 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f18d41338a0 0x7f18d4133c70 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:11.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.372+0000 7f18daf7a700 1 -- 172.21.15.16:0/1718607896 >> 172.21.15.16:0/1718607896 conn(0x7f18d40a2aa0 msgr2=0x7f18d40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:11.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.372+0000 7f18daf7a700 1 -- 172.21.15.16:0/1718607896 shutdown_connections 2023-09-30T22:34:11.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.372+0000 7f18daf7a700 1 -- 172.21.15.16:0/1718607896 wait complete. 2023-09-30T22:34:11.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.373+0000 7f18daf7a700 1 Processor -- start 2023-09-30T22:34:11.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.373+0000 7f18daf7a700 1 -- start start 2023-09-30T22:34:11.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.374+0000 7f18daf7a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18d409de50 0x7f18d411d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:11.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.374+0000 7f18daf7a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f18d409e930 0x7f18d411dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:11.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.374+0000 7f18daf7a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f18d41338a0 0x7f18d411e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:11.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.374+0000 7f18daf7a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d411f310 con 0x7f18d409de50 2023-09-30T22:34:11.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.374+0000 7f18daf7a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18d406f640 con 0x7f18d409e930 2023-09-30T22:34:11.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.374+0000 7f18daf7a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f18d4120560 con 0x7f18d41338a0 2023-09-30T22:34:11.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.374+0000 7f18d3fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f18d409e930 0x7f18d411dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:11.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.374+0000 7f18d3fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f18d409e930 0x7f18d411dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:52846/0 (socket says 172.21.15.16:52846) 2023-09-30T22:34:11.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.374+0000 7f18d3fff700 1 -- 172.21.15.16:0/919026232 learned_addr learned my addr 172.21.15.16:0/919026232 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:11.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.374+0000 7f18d8d16700 1 --2- 172.21.15.16:0/919026232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18d409de50 0x7f18d411d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:11.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.374+0000 7f18d9517700 1 --2- 172.21.15.16:0/919026232 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f18d41338a0 0x7f18d411e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:11.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.374+0000 7f18d8d16700 1 -- 172.21.15.16:0/919026232 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f18d41338a0 msgr2=0x7f18d411e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:11.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.374+0000 7f18d8d16700 1 --2- 172.21.15.16:0/919026232 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f18d41338a0 0x7f18d411e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:11.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.374+0000 7f18d8d16700 1 -- 172.21.15.16:0/919026232 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f18d409e930 msgr2=0x7f18d411dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:11.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.374+0000 7f18d8d16700 1 --2- 172.21.15.16:0/919026232 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f18d409e930 0x7f18d411dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:11.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.374+0000 7f18d8d16700 1 -- 172.21.15.16:0/919026232 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c8009550 con 0x7f18d409de50 2023-09-30T22:34:11.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.374+0000 7f18d3fff700 1 --2- 172.21.15.16:0/919026232 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f18d409e930 0x7f18d411dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:11.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.375+0000 7f18d8d16700 1 --2- 172.21.15.16:0/919026232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18d409de50 0x7f18d411d740 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f18c8015000 tx=0x7f18c8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:11.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.375+0000 7f18d17fa700 1 -- 172.21.15.16:0/919026232 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c8003df0 con 0x7f18d409de50 2023-09-30T22:34:11.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.375+0000 7f18d17fa700 1 -- 172.21.15.16:0/919026232 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18c8011630 con 0x7f18d409de50 2023-09-30T22:34:11.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.375+0000 7f18d17fa700 1 -- 172.21.15.16:0/919026232 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c80119e0 con 0x7f18d409de50 2023-09-30T22:34:11.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.375+0000 7f18daf7a700 1 -- 172.21.15.16:0/919026232 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18d4120f40 con 0x7f18d409de50 2023-09-30T22:34:11.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.375+0000 7f18daf7a700 1 -- 172.21.15.16:0/919026232 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18d41209b0 con 0x7f18d409de50 2023-09-30T22:34:11.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.376+0000 7f18daf7a700 1 -- 172.21.15.16:0/919026232 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18d411a470 con 0x7f18d409de50 2023-09-30T22:34:11.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.377+0000 7f18d17fa700 1 -- 172.21.15.16:0/919026232 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f18c8021070 con 0x7f18d409de50 2023-09-30T22:34:11.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.377+0000 7f18d17fa700 1 -- 172.21.15.16:0/919026232 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f18c8040900 con 0x7f18d409de50 2023-09-30T22:34:11.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.386+0000 7f18d17fa700 1 -- 172.21.15.16:0/919026232 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f18c802c3a0 con 0x7f18d409de50 2023-09-30T22:34:11.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.421+0000 7f5d4a43d700 1 Processor -- start 2023-09-30T22:34:11.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.421+0000 7f5d4a43d700 1 -- start start 2023-09-30T22:34:11.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.422+0000 7f5d4a43d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d4409e1d0 0x7f5d44135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:11.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.422+0000 7f5d4a43d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d44135e50 0x7f5d44138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:11.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.422+0000 7f5d4a43d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d4409f130 con 0x7f5d4409d6f0 2023-09-30T22:34:11.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.422+0000 7f5d4a43d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4409f520 con 0x7f5d44135e50 2023-09-30T22:34:11.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.422+0000 7f5d4a43d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4409e930 con 0x7f5d4409e1d0 2023-09-30T22:34:11.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.422+0000 7f5d437fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d4409e1d0 0x7f5d44135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:11.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.422+0000 7f5d437fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d4409e1d0 0x7f5d44135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35142/0 (socket says 172.21.15.16:35142) 2023-09-30T22:34:11.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.422+0000 7f5d437fe700 1 -- 172.21.15.16:0/2471928509 learned_addr learned my addr 172.21.15.16:0/2471928509 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:11.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.422+0000 7f5d489da700 1 --2- 172.21.15.16:0/2471928509 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d44135e50 0x7f5d44138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:11.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.423+0000 7f5d437fe700 1 -- 172.21.15.16:0/2471928509 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d44135e50 msgr2=0x7f5d44138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:11.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.423+0000 7f5d437fe700 1 --2- 172.21.15.16:0/2471928509 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d44135e50 0x7f5d44138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:11.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.423+0000 7f5d437fe700 1 -- 172.21.15.16:0/2471928509 >> v1:172.21.15.16:6789/0 conn(0x7f5d4409d6f0 legacy=0x7f5d4409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:11.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.423+0000 7f5d437fe700 1 -- 172.21.15.16:0/2471928509 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d4409b2d0 con 0x7f5d4409e1d0 2023-09-30T22:34:11.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.423+0000 7f5d437fe700 1 --2- 172.21.15.16:0/2471928509 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d4409e1d0 0x7f5d44135910 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f5d38009920 tx=0x7f5d38009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b9d74c10e44edf4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:11.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.423+0000 7f5d42ffd700 1 -- 172.21.15.16:0/2471928509 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d38011830 con 0x7f5d4409e1d0 2023-09-30T22:34:11.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.423+0000 7f5d42ffd700 1 -- 172.21.15.16:0/2471928509 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d38011990 con 0x7f5d4409e1d0 2023-09-30T22:34:11.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.423+0000 7f5d42ffd700 1 -- 172.21.15.16:0/2471928509 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d38011d20 con 0x7f5d4409e1d0 2023-09-30T22:34:11.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.424+0000 7f5d4a43d700 1 -- 172.21.15.16:0/2471928509 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d4409e1d0 msgr2=0x7f5d44135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:11.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.424+0000 7f5d4a43d700 1 --2- 172.21.15.16:0/2471928509 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d4409e1d0 0x7f5d44135910 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f5d38009920 tx=0x7f5d38009c30 comp rx=0 tx=0).stop 2023-09-30T22:34:11.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.424+0000 7f5d4a43d700 1 -- 172.21.15.16:0/2471928509 shutdown_connections 2023-09-30T22:34:11.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.424+0000 7f5d4a43d700 1 --2- 172.21.15.16:0/2471928509 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d44135e50 0x7f5d44138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:11.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.424+0000 7f5d4a43d700 1 --2- 172.21.15.16:0/2471928509 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d4409e1d0 0x7f5d44135910 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:11.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.424+0000 7f5d4a43d700 1 -- 172.21.15.16:0/2471928509 >> 172.21.15.16:0/2471928509 conn(0x7f5d44066f50 msgr2=0x7f5d4409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:11.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.424+0000 7f5d4a43d700 1 -- 172.21.15.16:0/2471928509 shutdown_connections 2023-09-30T22:34:11.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.424+0000 7f5d4a43d700 1 -- 172.21.15.16:0/2471928509 wait complete. 2023-09-30T22:34:11.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.425+0000 7f5d4a43d700 1 Processor -- start 2023-09-30T22:34:11.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.426+0000 7f5d4a43d700 1 -- start start 2023-09-30T22:34:11.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.426+0000 7f5d4a43d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d4409d6f0 0x7f5d44134d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:11.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.426+0000 7f5d4a43d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d4409e1d0 0x7f5d44135280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:11.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.426+0000 7f5d4a43d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d44135e50 0x7f5d441357c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:11.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.426+0000 7f5d4a43d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d441331e0 con 0x7f5d4409e1d0 2023-09-30T22:34:11.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.426+0000 7f5d4a43d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d44134160 con 0x7f5d44135e50 2023-09-30T22:34:11.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.426+0000 7f5d4a43d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5d44132ec0 con 0x7f5d4409d6f0 2023-09-30T22:34:11.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.426+0000 7f5d489da700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d44135e50 0x7f5d441357c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:11.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.426+0000 7f5d489da700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d44135e50 0x7f5d441357c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:52868/0 (socket says 172.21.15.16:52868) 2023-09-30T22:34:11.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.426+0000 7f5d489da700 1 -- 172.21.15.16:0/4240898929 learned_addr learned my addr 172.21.15.16:0/4240898929 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:11.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.426+0000 7f5d437fe700 1 --2- 172.21.15.16:0/4240898929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d4409e1d0 0x7f5d44135280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:11.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.426+0000 7f5d43fff700 1 --2- 172.21.15.16:0/4240898929 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d4409d6f0 0x7f5d44134d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:11.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.427+0000 7f5d489da700 1 -- 172.21.15.16:0/4240898929 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d4409d6f0 msgr2=0x7f5d44134d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:11.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.427+0000 7f5d489da700 1 --2- 172.21.15.16:0/4240898929 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d4409d6f0 0x7f5d44134d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:11.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.427+0000 7f5d489da700 1 -- 172.21.15.16:0/4240898929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d4409e1d0 msgr2=0x7f5d44135280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:11.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.427+0000 7f5d489da700 1 --2- 172.21.15.16:0/4240898929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d4409e1d0 0x7f5d44135280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:11.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.427+0000 7f5d489da700 1 -- 172.21.15.16:0/4240898929 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d38009550 con 0x7f5d44135e50 2023-09-30T22:34:11.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.427+0000 7f5d489da700 1 --2- 172.21.15.16:0/4240898929 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d44135e50 0x7f5d441357c0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c009e30 tx=0x7f5d3c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:11.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.427+0000 7f5d40ff9700 1 -- 172.21.15.16:0/4240898929 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c009ca0 con 0x7f5d44135e50 2023-09-30T22:34:11.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.427+0000 7f5d40ff9700 1 -- 172.21.15.16:0/4240898929 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d3c00f040 con 0x7f5d44135e50 2023-09-30T22:34:11.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.427+0000 7f5d4a43d700 1 -- 172.21.15.16:0/4240898929 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d440a2220 con 0x7f5d44135e50 2023-09-30T22:34:11.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.427+0000 7f5d40ff9700 1 -- 172.21.15.16:0/4240898929 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d3c014480 con 0x7f5d44135e50 2023-09-30T22:34:11.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.427+0000 7f5d4a43d700 1 -- 172.21.15.16:0/4240898929 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d44133bc0 con 0x7f5d44135e50 2023-09-30T22:34:11.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.428+0000 7f5d40ff9700 1 -- 172.21.15.16:0/4240898929 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5d3c0145e0 con 0x7f5d44135e50 2023-09-30T22:34:11.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.429+0000 7f5d40ff9700 1 -- 172.21.15.16:0/4240898929 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f5d3c040260 con 0x7f5d44135e50 2023-09-30T22:34:11.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.429+0000 7f5d4a43d700 1 -- 172.21.15.16:0/4240898929 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d44074ea0 con 0x7f5d44135e50 2023-09-30T22:34:11.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.438+0000 7f5d40ff9700 1 -- 172.21.15.16:0/4240898929 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5d3c007920 con 0x7f5d44135e50 2023-09-30T22:34:11.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.462+0000 7f2f4aeae700 1 Processor -- start 2023-09-30T22:34:11.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.463+0000 7f2f4aeae700 1 -- start start 2023-09-30T22:34:11.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.463+0000 7f2f4aeae700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f441338a0 0x7f2f44133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:11.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.463+0000 7f2f4aeae700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f4409e930 0x7f2f4413b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:11.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.463+0000 7f2f4aeae700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f4406ef30 con 0x7f2f4409de50 2023-09-30T22:34:11.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.463+0000 7f2f4aeae700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4406fef0 con 0x7f2f4409e930 2023-09-30T22:34:11.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.463+0000 7f2f4aeae700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2f441334b0 con 0x7f2f441338a0 2023-09-30T22:34:11.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.463+0000 7f2f43fff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2f4409de50 0x7f2f4409e220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59790/0 (socket says 172.21.15.16:59790) 2023-09-30T22:34:11.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.463+0000 7f2f43fff700 1 -- 172.21.15.16:0/3292010659 learned_addr learned my addr 172.21.15.16:0/3292010659 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:34:11.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.464+0000 7f2f4944b700 1 --2- 172.21.15.16:0/3292010659 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f4409e930 0x7f2f4413b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:11.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.464+0000 7f2f48c4a700 1 --2- 172.21.15.16:0/3292010659 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f441338a0 0x7f2f44133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:11.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.464+0000 7f2f437fe700 1 -- 172.21.15.16:0/3292010659 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3420218899 0 0) 0x7f2f4406ef30 con 0x7f2f4409de50 2023-09-30T22:34:11.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.464+0000 7f2f437fe700 1 -- 172.21.15.16:0/3292010659 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f340035b0 con 0x7f2f4409de50 2023-09-30T22:34:11.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.464+0000 7f2f4944b700 1 -- 172.21.15.16:0/3292010659 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f441338a0 msgr2=0x7f2f44133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:11.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.464+0000 7f2f4944b700 1 --2- 172.21.15.16:0/3292010659 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f441338a0 0x7f2f44133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:11.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.464+0000 7f2f4944b700 1 -- 172.21.15.16:0/3292010659 >> v1:172.21.15.16:6789/0 conn(0x7f2f4409de50 legacy=0x7f2f4409e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:11.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.464+0000 7f2f4944b700 1 -- 172.21.15.16:0/3292010659 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f4406fc30 con 0x7f2f4409e930 2023-09-30T22:34:11.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.464+0000 7f2f4944b700 1 --2- 172.21.15.16:0/3292010659 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f4409e930 0x7f2f4413b740 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c009dc0 tx=0x7f2f3c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a03989a5c228f17 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:11.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.464+0000 7f2f437fe700 1 -- 172.21.15.16:0/3292010659 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c00e070 con 0x7f2f4409e930 2023-09-30T22:34:11.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.465+0000 7f2f437fe700 1 -- 172.21.15.16:0/3292010659 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f3c00bb20 con 0x7f2f4409e930 2023-09-30T22:34:11.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.465+0000 7f2f437fe700 1 -- 172.21.15.16:0/3292010659 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c012430 con 0x7f2f4409e930 2023-09-30T22:34:11.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.465+0000 7f2f4aeae700 1 -- 172.21.15.16:0/3292010659 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f4409e930 msgr2=0x7f2f4413b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:11.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.465+0000 7f2f4aeae700 1 --2- 172.21.15.16:0/3292010659 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f4409e930 0x7f2f4413b740 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c009dc0 tx=0x7f2f3c00b330 comp rx=0 tx=0).stop 2023-09-30T22:34:11.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.465+0000 7f2f4aeae700 1 -- 172.21.15.16:0/3292010659 shutdown_connections 2023-09-30T22:34:11.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.465+0000 7f2f4aeae700 1 --2- 172.21.15.16:0/3292010659 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f4409e930 0x7f2f4413b740 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:11.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.465+0000 7f2f4aeae700 1 --2- 172.21.15.16:0/3292010659 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f441338a0 0x7f2f44133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:11.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.465+0000 7f2f4aeae700 1 -- 172.21.15.16:0/3292010659 >> 172.21.15.16:0/3292010659 conn(0x7f2f440a2aa0 msgr2=0x7f2f440a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:11.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.465+0000 7f2f4aeae700 1 -- 172.21.15.16:0/3292010659 shutdown_connections 2023-09-30T22:34:11.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.466+0000 7f2f4aeae700 1 -- 172.21.15.16:0/3292010659 wait complete. 2023-09-30T22:34:11.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.466+0000 7f2f4aeae700 1 Processor -- start 2023-09-30T22:34:11.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.467+0000 7f2f4aeae700 1 -- start start 2023-09-30T22:34:11.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.467+0000 7f2f4aeae700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f4409de50 0x7f2f4411d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:11.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.467+0000 7f2f4aeae700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f4409e930 0x7f2f4411dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:11.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.467+0000 7f2f4aeae700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f441338a0 0x7f2f4411e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:11.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.467+0000 7f2f4aeae700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f44120920 con 0x7f2f441338a0 2023-09-30T22:34:11.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.467+0000 7f2f4aeae700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4406f640 con 0x7f2f4409e930 2023-09-30T22:34:11.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.467+0000 7f2f4aeae700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2f440a0c00 con 0x7f2f4409de50 2023-09-30T22:34:11.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.467+0000 7f2f48c4a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f4409de50 0x7f2f4411d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:11.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.467+0000 7f2f48c4a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f4409de50 0x7f2f4411d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35172/0 (socket says 172.21.15.16:35172) 2023-09-30T22:34:11.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.467+0000 7f2f48c4a700 1 -- 172.21.15.16:0/3475350278 learned_addr learned my addr 172.21.15.16:0/3475350278 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:11.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.468+0000 7f2f4944b700 1 --2- 172.21.15.16:0/3475350278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f441338a0 0x7f2f4411e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:11.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.468+0000 7f2f43fff700 1 --2- 172.21.15.16:0/3475350278 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f4409e930 0x7f2f4411dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:11.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.468+0000 7f2f48c4a700 1 -- 172.21.15.16:0/3475350278 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f4409e930 msgr2=0x7f2f4411dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:11.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.468+0000 7f2f48c4a700 1 --2- 172.21.15.16:0/3475350278 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f4409e930 0x7f2f4411dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:11.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.468+0000 7f2f48c4a700 1 -- 172.21.15.16:0/3475350278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f441338a0 msgr2=0x7f2f4411e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:11.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.468+0000 7f2f48c4a700 1 --2- 172.21.15.16:0/3475350278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f441338a0 0x7f2f4411e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:11.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.468+0000 7f2f48c4a700 1 -- 172.21.15.16:0/3475350278 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f3c009a70 con 0x7f2f4409de50 2023-09-30T22:34:11.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.468+0000 7f2f4944b700 1 --2- 172.21.15.16:0/3475350278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f441338a0 0x7f2f4411e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:11.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.468+0000 7f2f48c4a700 1 --2- 172.21.15.16:0/3475350278 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f4409de50 0x7f2f4411d670 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f2f3800e9b0 tx=0x7f2f3800ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:11.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.468+0000 7f2f417fa700 1 -- 172.21.15.16:0/3475350278 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f38009710 con 0x7f2f4409de50 2023-09-30T22:34:11.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.468+0000 7f2f417fa700 1 -- 172.21.15.16:0/3475350278 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f3800ee50 con 0x7f2f4409de50 2023-09-30T22:34:11.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.468+0000 7f2f417fa700 1 -- 172.21.15.16:0/3475350278 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3800ca50 con 0x7f2f4409de50 2023-09-30T22:34:11.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.469+0000 7f2f4aeae700 1 -- 172.21.15.16:0/3475350278 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f4411f280 con 0x7f2f4409de50 2023-09-30T22:34:11.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.469+0000 7f2f4aeae700 1 -- 172.21.15.16:0/3475350278 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f44121690 con 0x7f2f4409de50 2023-09-30T22:34:11.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.470+0000 7f2f417fa700 1 -- 172.21.15.16:0/3475350278 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2f3800cbb0 con 0x7f2f4409de50 2023-09-30T22:34:11.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.471+0000 7f2f4aeae700 1 -- 172.21.15.16:0/3475350278 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f4411a470 con 0x7f2f4409de50 2023-09-30T22:34:11.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.471+0000 7f2f417fa700 1 -- 172.21.15.16:0/3475350278 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f2f3803f770 con 0x7f2f4409de50 2023-09-30T22:34:11.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.480+0000 7f2f417fa700 1 -- 172.21.15.16:0/3475350278 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2f3801be10 con 0x7f2f4409de50 2023-09-30T22:34:11.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.512+0000 7fc60873c700 1 Processor -- start 2023-09-30T22:34:11.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.512+0000 7fc60873c700 1 -- start start 2023-09-30T22:34:11.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.512+0000 7fc60873c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc60009d6f0 0x7fc60009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:11.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.512+0000 7fc60873c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc60009e000 0x7fc6001339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:11.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.512+0000 7fc60873c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc60009e410 0x7fc600133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:11.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.512+0000 7fc60873c700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc60006b460 con 0x7fc60009e000 2023-09-30T22:34:11.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.512+0000 7fc60873c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc60009b120 con 0x7fc60009d6f0 2023-09-30T22:34:11.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.512+0000 7fc60873c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc60009a7e0 con 0x7fc60009e410 2023-09-30T22:34:11.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.513+0000 7fc606cd9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc60009e410 0x7fc600133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:11.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.513+0000 7fc605cd7700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc60009e000 0x7fc6001339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:11.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.513+0000 7fc606cd9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc60009e410 0x7fc600133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35184/0 (socket says 172.21.15.16:35184) 2023-09-30T22:34:11.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.513+0000 7fc605cd7700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc60009e000 0x7fc6001339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33438/0 (socket says 172.21.15.16:33438) 2023-09-30T22:34:11.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.513+0000 7fc606cd9700 1 -- 172.21.15.16:0/3029679058 learned_addr learned my addr 172.21.15.16:0/3029679058 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:11.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.513+0000 7fc6064d8700 1 --2- 172.21.15.16:0/3029679058 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc60009d6f0 0x7fc60009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:11.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.513+0000 7fc606cd9700 1 -- 172.21.15.16:0/3029679058 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc60009d6f0 msgr2=0x7fc60009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:11.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.513+0000 7fc606cd9700 1 --2- 172.21.15.16:0/3029679058 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc60009d6f0 0x7fc60009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:11.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.513+0000 7fc606cd9700 1 -- 172.21.15.16:0/3029679058 >> v2:172.21.15.16:3300/0 conn(0x7fc60009e000 msgr2=0x7fc6001339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:11.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.513+0000 7fc606cd9700 1 --2- 172.21.15.16:0/3029679058 >> v2:172.21.15.16:3300/0 conn(0x7fc60009e000 0x7fc6001339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:11.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.513+0000 7fc606cd9700 1 -- 172.21.15.16:0/3029679058 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc60009a500 con 0x7fc60009e410 2023-09-30T22:34:11.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.513+0000 7fc606cd9700 1 --2- 172.21.15.16:0/3029679058 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc60009e410 0x7fc600133f30 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8009e80 tx=0x7fc5f800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b92c4ad9311dba81 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:11.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.514+0000 7fc6054d6700 1 -- 172.21.15.16:0/3029679058 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f800e070 con 0x7fc60009e410 2023-09-30T22:34:11.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.514+0000 7fc6054d6700 1 -- 172.21.15.16:0/3029679058 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5f800b8d0 con 0x7fc60009e410 2023-09-30T22:34:11.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.514+0000 7fc6054d6700 1 -- 172.21.15.16:0/3029679058 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f800bc80 con 0x7fc60009e410 2023-09-30T22:34:11.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.515+0000 7fc60873c700 1 -- 172.21.15.16:0/3029679058 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc60009e410 msgr2=0x7fc600133f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:11.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.515+0000 7fc60873c700 1 --2- 172.21.15.16:0/3029679058 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc60009e410 0x7fc600133f30 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fc5f8009e80 tx=0x7fc5f800b330 comp rx=0 tx=0).stop 2023-09-30T22:34:11.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.515+0000 7fc60873c700 1 -- 172.21.15.16:0/3029679058 shutdown_connections 2023-09-30T22:34:11.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.515+0000 7fc60873c700 1 --2- 172.21.15.16:0/3029679058 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc60009d6f0 0x7fc60009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:11.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.515+0000 7fc60873c700 1 --2- 172.21.15.16:0/3029679058 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc60009e410 0x7fc600133f30 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:11.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.515+0000 7fc60873c700 1 --2- 172.21.15.16:0/3029679058 >> v2:172.21.15.16:3300/0 conn(0x7fc60009e000 0x7fc6001339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:11.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.515+0000 7fc60873c700 1 -- 172.21.15.16:0/3029679058 >> 172.21.15.16:0/3029679058 conn(0x7fc600066f50 msgr2=0x7fc60009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:11.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.515+0000 7fc60873c700 1 -- 172.21.15.16:0/3029679058 shutdown_connections 2023-09-30T22:34:11.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.515+0000 7fc60873c700 1 -- 172.21.15.16:0/3029679058 wait complete. 2023-09-30T22:34:11.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.516+0000 7fc60873c700 1 Processor -- start 2023-09-30T22:34:11.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.516+0000 7fc60873c700 1 -- start start 2023-09-30T22:34:11.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.517+0000 7fc60873c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc60009d6f0 0x7fc6001a5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:11.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.517+0000 7fc60873c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc60009e000 0x7fc6001a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:11.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.517+0000 7fc60873c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc60009e410 0x7fc6001a6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:11.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.517+0000 7fc60873c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6001aa220 con 0x7fc60009e000 2023-09-30T22:34:11.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.517+0000 7fc60873c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6001a7950 con 0x7fc60009e410 2023-09-30T22:34:11.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.517+0000 7fc60873c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc6001a7f70 con 0x7fc60009d6f0 2023-09-30T22:34:11.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.517+0000 7fc606cd9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc60009e410 0x7fc6001a6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:11.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.517+0000 7fc606cd9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc60009e410 0x7fc6001a6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:52914/0 (socket says 172.21.15.16:52914) 2023-09-30T22:34:11.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.517+0000 7fc606cd9700 1 -- 172.21.15.16:0/3699668046 learned_addr learned my addr 172.21.15.16:0/3699668046 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:11.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.517+0000 7fc605cd7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc60009e000 0x7fc6001a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:11.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.518+0000 7fc6064d8700 1 --2- 172.21.15.16:0/3699668046 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc60009d6f0 0x7fc6001a5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:11.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.518+0000 7fc606cd9700 1 -- 172.21.15.16:0/3699668046 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc60009d6f0 msgr2=0x7fc6001a5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:11.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.518+0000 7fc606cd9700 1 --2- 172.21.15.16:0/3699668046 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc60009d6f0 0x7fc6001a5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:11.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.518+0000 7fc606cd9700 1 -- 172.21.15.16:0/3699668046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc60009e000 msgr2=0x7fc6001a6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:11.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.518+0000 7fc606cd9700 1 --2- 172.21.15.16:0/3699668046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc60009e000 0x7fc6001a6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:11.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.518+0000 7fc606cd9700 1 -- 172.21.15.16:0/3699668046 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f8009b30 con 0x7fc60009e410 2023-09-30T22:34:11.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.518+0000 7fc606cd9700 1 --2- 172.21.15.16:0/3699668046 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc60009e410 0x7fc6001a6860 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fc5f800b680 tx=0x7fc5f8004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:11.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.518+0000 7fc5f6ffd700 1 -- 172.21.15.16:0/3699668046 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f8004450 con 0x7fc60009e410 2023-09-30T22:34:11.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.518+0000 7fc5f6ffd700 1 -- 172.21.15.16:0/3699668046 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5f80045d0 con 0x7fc60009e410 2023-09-30T22:34:11.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.518+0000 7fc5f6ffd700 1 -- 172.21.15.16:0/3699668046 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f8003e10 con 0x7fc60009e410 2023-09-30T22:34:11.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.518+0000 7fc60873c700 1 -- 172.21.15.16:0/3699668046 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6001a8ba0 con 0x7fc60009e410 2023-09-30T22:34:11.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.519+0000 7fc60873c700 1 -- 172.21.15.16:0/3699668046 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6001a8ff0 con 0x7fc60009e410 2023-09-30T22:34:11.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.520+0000 7fc60873c700 1 -- 172.21.15.16:0/3699668046 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc600074ea0 con 0x7fc60009e410 2023-09-30T22:34:11.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.520+0000 7fc5f6ffd700 1 -- 172.21.15.16:0/3699668046 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc5f801c410 con 0x7fc60009e410 2023-09-30T22:34:11.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.520+0000 7fc5f6ffd700 1 -- 172.21.15.16:0/3699668046 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fc5f8040e20 con 0x7fc60009e410 2023-09-30T22:34:11.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.525+0000 7fc5f6ffd700 1 -- 172.21.15.16:0/3699668046 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc5f802d350 con 0x7fc60009e410 2023-09-30T22:34:11.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.611+0000 7f18daf7a700 1 -- 172.21.15.16:0/919026232 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f18d4074ea0 con 0x7f18d409de50 2023-09-30T22:34:11.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.612+0000 7f18d17fa700 1 -- 172.21.15.16:0/919026232 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f18c8028030 con 0x7f18d409de50 2023-09-30T22:34:11.612 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:34:11.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.615+0000 7f18daf7a700 1 -- 172.21.15.16:0/919026232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18d409de50 msgr2=0x7f18d411d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:11.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.615+0000 7f18daf7a700 1 --2- 172.21.15.16:0/919026232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18d409de50 0x7f18d411d740 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f18c8015000 tx=0x7f18c8004750 comp rx=0 tx=0).stop 2023-09-30T22:34:11.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.615+0000 7f18daf7a700 1 -- 172.21.15.16:0/919026232 shutdown_connections 2023-09-30T22:34:11.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.615+0000 7f18daf7a700 1 --2- 172.21.15.16:0/919026232 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f18d41338a0 0x7f18d411e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:11.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.615+0000 7f18daf7a700 1 --2- 172.21.15.16:0/919026232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18d409de50 0x7f18d411d740 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:11.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.615+0000 7f18daf7a700 1 --2- 172.21.15.16:0/919026232 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f18d409e930 0x7f18d411dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:11.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.615+0000 7f18daf7a700 1 -- 172.21.15.16:0/919026232 >> 172.21.15.16:0/919026232 conn(0x7f18d40a2aa0 msgr2=0x7f18d409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:11.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.615+0000 7f18daf7a700 1 -- 172.21.15.16:0/919026232 shutdown_connections 2023-09-30T22:34:11.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.615+0000 7f18daf7a700 1 -- 172.21.15.16:0/919026232 wait complete. 2023-09-30T22:34:11.635 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:34:11.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.652+0000 7f5d4a43d700 1 -- 172.21.15.16:0/4240898929 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5d44132a60 con 0x7f5d44135e50 2023-09-30T22:34:11.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.653+0000 7f5d40ff9700 1 -- 172.21.15.16:0/4240898929 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5d3c02c090 con 0x7f5d44135e50 2023-09-30T22:34:11.653 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:34:11.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.655+0000 7f5d4a43d700 1 -- 172.21.15.16:0/4240898929 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d44135e50 msgr2=0x7f5d441357c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:11.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.655+0000 7f5d4a43d700 1 --2- 172.21.15.16:0/4240898929 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d44135e50 0x7f5d441357c0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f5d3c009e30 tx=0x7f5d3c00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:34:11.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.655+0000 7f5d4a43d700 1 -- 172.21.15.16:0/4240898929 shutdown_connections 2023-09-30T22:34:11.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.655+0000 7f5d4a43d700 1 --2- 172.21.15.16:0/4240898929 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d44135e50 0x7f5d441357c0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:11.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.655+0000 7f5d4a43d700 1 --2- 172.21.15.16:0/4240898929 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d4409d6f0 0x7f5d44134d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:11.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.655+0000 7f5d4a43d700 1 --2- 172.21.15.16:0/4240898929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d4409e1d0 0x7f5d44135280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:11.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.655+0000 7f5d4a43d700 1 -- 172.21.15.16:0/4240898929 >> 172.21.15.16:0/4240898929 conn(0x7f5d44066f50 msgr2=0x7f5d4406e840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:11.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.655+0000 7f5d4a43d700 1 -- 172.21.15.16:0/4240898929 shutdown_connections 2023-09-30T22:34:11.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.655+0000 7f5d4a43d700 1 -- 172.21.15.16:0/4240898929 wait complete. 2023-09-30T22:34:11.671 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:34:11.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.702+0000 7f2f4aeae700 1 -- 172.21.15.16:0/3475350278 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f2f44074ea0 con 0x7f2f4409de50 2023-09-30T22:34:11.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.702+0000 7f2f417fa700 1 -- 172.21.15.16:0/3475350278 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2f38021020 con 0x7f2f4409de50 2023-09-30T22:34:11.703 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:34:11.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.705+0000 7f2f4aeae700 1 -- 172.21.15.16:0/3475350278 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f4409de50 msgr2=0x7f2f4411d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:11.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.705+0000 7f2f4aeae700 1 --2- 172.21.15.16:0/3475350278 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f4409de50 0x7f2f4411d670 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f2f3800e9b0 tx=0x7f2f3800ecc0 comp rx=0 tx=0).stop 2023-09-30T22:34:11.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.705+0000 7f2f4aeae700 1 -- 172.21.15.16:0/3475350278 shutdown_connections 2023-09-30T22:34:11.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.705+0000 7f2f4aeae700 1 --2- 172.21.15.16:0/3475350278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f441338a0 0x7f2f4411e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:11.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.705+0000 7f2f4aeae700 1 --2- 172.21.15.16:0/3475350278 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f4409de50 0x7f2f4411d670 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:11.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.705+0000 7f2f4aeae700 1 --2- 172.21.15.16:0/3475350278 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f4409e930 0x7f2f4411dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:11.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.705+0000 7f2f4aeae700 1 -- 172.21.15.16:0/3475350278 >> 172.21.15.16:0/3475350278 conn(0x7f2f440a2aa0 msgr2=0x7f2f441bac80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:11.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.705+0000 7f2f4aeae700 1 -- 172.21.15.16:0/3475350278 shutdown_connections 2023-09-30T22:34:11.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.705+0000 7f2f4aeae700 1 -- 172.21.15.16:0/3475350278 wait complete. 2023-09-30T22:34:11.719 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:34:11.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.736+0000 7fc60873c700 1 -- 172.21.15.16:0/3699668046 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc60009c690 con 0x7fc60009e410 2023-09-30T22:34:11.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.737+0000 7fc5f6ffd700 1 -- 172.21.15.16:0/3699668046 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc5f802a090 con 0x7fc60009e410 2023-09-30T22:34:11.737 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:34:11.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.739+0000 7fc60873c700 1 -- 172.21.15.16:0/3699668046 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc60009e410 msgr2=0x7fc6001a6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:11.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.739+0000 7fc60873c700 1 --2- 172.21.15.16:0/3699668046 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc60009e410 0x7fc6001a6860 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fc5f800b680 tx=0x7fc5f8004ad0 comp rx=0 tx=0).stop 2023-09-30T22:34:11.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.739+0000 7fc60873c700 1 -- 172.21.15.16:0/3699668046 shutdown_connections 2023-09-30T22:34:11.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.739+0000 7fc60873c700 1 --2- 172.21.15.16:0/3699668046 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc60009e410 0x7fc6001a6860 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:11.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.739+0000 7fc60873c700 1 --2- 172.21.15.16:0/3699668046 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc60009d6f0 0x7fc6001a5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:11.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.739+0000 7fc60873c700 1 --2- 172.21.15.16:0/3699668046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc60009e000 0x7fc6001a6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:11.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.739+0000 7fc60873c700 1 -- 172.21.15.16:0/3699668046 >> 172.21.15.16:0/3699668046 conn(0x7fc600066f50 msgr2=0x7fc6000a16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:11.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.739+0000 7fc60873c700 1 -- 172.21.15.16:0/3699668046 shutdown_connections 2023-09-30T22:34:11.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:11.739+0000 7fc60873c700 1 -- 172.21.15.16:0/3699668046 wait complete. 2023-09-30T22:34:11.752 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:34:12.636 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:34:12.672 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:34:12.719 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:34:12.754 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:34:12.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.787+0000 7f73e2b2a700 1 Processor -- start 2023-09-30T22:34:12.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.787+0000 7f73e2b2a700 1 -- start start 2023-09-30T22:34:12.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.788+0000 7f73e2b2a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f73dc06c0e0 0x7f73dc09ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:12.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.788+0000 7f73e2b2a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f73dc1380f0 0x7f73dc13fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:12.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.788+0000 7f73e2b2a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73dc134980 con 0x7f73dc134ec0 2023-09-30T22:34:12.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.788+0000 7f73e2b2a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73dc06c4b0 con 0x7f73dc1380f0 2023-09-30T22:34:12.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.788+0000 7f73e2b2a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f73dc0a09c0 con 0x7f73dc06c0e0 2023-09-30T22:34:12.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.788+0000 7f73dbfff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f73dc134ec0 0x7f73dc1379e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59796/0 (socket says 172.21.15.16:59796) 2023-09-30T22:34:12.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.788+0000 7f73dbfff700 1 -- 172.21.15.16:0/2164125567 learned_addr learned my addr 172.21.15.16:0/2164125567 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:34:12.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.789+0000 7f73e10c7700 1 --2- 172.21.15.16:0/2164125567 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f73dc1380f0 0x7f73dc13fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:12.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.789+0000 7f73db7fe700 1 -- 172.21.15.16:0/2164125567 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1897768771 0 0) 0x7f73dc134980 con 0x7f73dc134ec0 2023-09-30T22:34:12.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.789+0000 7f73db7fe700 1 -- 172.21.15.16:0/2164125567 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73cc0035b0 con 0x7f73dc134ec0 2023-09-30T22:34:12.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.789+0000 7f73e10c7700 1 -- 172.21.15.16:0/2164125567 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f73dc06c0e0 msgr2=0x7f73dc09ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:12.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.789+0000 7f73e10c7700 1 --2- 172.21.15.16:0/2164125567 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f73dc06c0e0 0x7f73dc09ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:12.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.789+0000 7f73e10c7700 1 -- 172.21.15.16:0/2164125567 >> v1:172.21.15.16:6789/0 conn(0x7f73dc134ec0 legacy=0x7f73dc1379e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:12.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.789+0000 7f73e10c7700 1 -- 172.21.15.16:0/2164125567 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73dc0a0600 con 0x7f73dc1380f0 2023-09-30T22:34:12.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.789+0000 7f73e10c7700 1 --2- 172.21.15.16:0/2164125567 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f73dc1380f0 0x7f73dc13fd20 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f73d4009dc0 tx=0x7f73d400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b9a14c7532c7622 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:12.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.789+0000 7f73db7fe700 1 -- 172.21.15.16:0/2164125567 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d400e070 con 0x7f73dc1380f0 2023-09-30T22:34:12.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.789+0000 7f73db7fe700 1 -- 172.21.15.16:0/2164125567 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73d400bb20 con 0x7f73dc1380f0 2023-09-30T22:34:12.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.789+0000 7f73db7fe700 1 -- 172.21.15.16:0/2164125567 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d400be90 con 0x7f73dc1380f0 2023-09-30T22:34:12.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.790+0000 7f73e2b2a700 1 -- 172.21.15.16:0/2164125567 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f73dc1380f0 msgr2=0x7f73dc13fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:12.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.790+0000 7f73e2b2a700 1 --2- 172.21.15.16:0/2164125567 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f73dc1380f0 0x7f73dc13fd20 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f73d4009dc0 tx=0x7f73d400b330 comp rx=0 tx=0).stop 2023-09-30T22:34:12.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.790+0000 7f73e2b2a700 1 -- 172.21.15.16:0/2164125567 shutdown_connections 2023-09-30T22:34:12.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.790+0000 7f73e2b2a700 1 --2- 172.21.15.16:0/2164125567 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f73dc1380f0 0x7f73dc13fd20 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:12.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.790+0000 7f73e2b2a700 1 --2- 172.21.15.16:0/2164125567 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f73dc06c0e0 0x7f73dc09ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:12.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.790+0000 7f73e2b2a700 1 -- 172.21.15.16:0/2164125567 >> 172.21.15.16:0/2164125567 conn(0x7f73dc0a4210 msgr2=0x7f73dc0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:12.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.790+0000 7f73e2b2a700 1 -- 172.21.15.16:0/2164125567 shutdown_connections 2023-09-30T22:34:12.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.790+0000 7f73e2b2a700 1 -- 172.21.15.16:0/2164125567 wait complete. 2023-09-30T22:34:12.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.792+0000 7f73e2b2a700 1 Processor -- start 2023-09-30T22:34:12.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.792+0000 7f73e2b2a700 1 -- start start 2023-09-30T22:34:12.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.793+0000 7f73e2b2a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73dc06c0e0 0x7f73dc1ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:12.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.793+0000 7f73e2b2a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f73dc134ec0 0x7f73dc1cd290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:12.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.793+0000 7f73e2b2a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f73dc1380f0 0x7f73dc11a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:12.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.793+0000 7f73e2b2a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73dc127420 con 0x7f73dc06c0e0 2023-09-30T22:34:12.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.793+0000 7f73e2b2a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73dc0a03f0 con 0x7f73dc1380f0 2023-09-30T22:34:12.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.793+0000 7f73e2b2a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f73dc11d8e0 con 0x7f73dc134ec0 2023-09-30T22:34:12.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.793+0000 7f73dbfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f73dc134ec0 0x7f73dc1cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:12.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.793+0000 7f73e08c6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73dc06c0e0 0x7f73dc1ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:12.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.793+0000 7f73dbfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f73dc134ec0 0x7f73dc1cd290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35212/0 (socket says 172.21.15.16:35212) 2023-09-30T22:34:12.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.793+0000 7f73dbfff700 1 -- 172.21.15.16:0/2173007410 learned_addr learned my addr 172.21.15.16:0/2173007410 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:12.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.793+0000 7f73e10c7700 1 --2- 172.21.15.16:0/2173007410 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f73dc1380f0 0x7f73dc11a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:12.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.793+0000 7f73dbfff700 1 -- 172.21.15.16:0/2173007410 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f73dc1380f0 msgr2=0x7f73dc11a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:12.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.793+0000 7f73dbfff700 1 --2- 172.21.15.16:0/2173007410 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f73dc1380f0 0x7f73dc11a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:12.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.793+0000 7f73dbfff700 1 -- 172.21.15.16:0/2173007410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73dc06c0e0 msgr2=0x7f73dc1ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:12.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.793+0000 7f73dbfff700 1 --2- 172.21.15.16:0/2173007410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73dc06c0e0 0x7f73dc1ccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:12.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.793+0000 7f73dbfff700 1 -- 172.21.15.16:0/2173007410 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73d4009a70 con 0x7f73dc134ec0 2023-09-30T22:34:12.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.794+0000 7f73e10c7700 1 --2- 172.21.15.16:0/2173007410 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f73dc1380f0 0x7f73dc11a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:12.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.794+0000 7f73dbfff700 1 --2- 172.21.15.16:0/2173007410 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f73dc134ec0 0x7f73dc1cd290 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f73c8005910 tx=0x7f73c800c9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:12.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.794+0000 7f73d97fa700 1 -- 172.21.15.16:0/2173007410 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c8005d70 con 0x7f73dc134ec0 2023-09-30T22:34:12.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.794+0000 7f73d97fa700 1 -- 172.21.15.16:0/2173007410 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73c80070d0 con 0x7f73dc134ec0 2023-09-30T22:34:12.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.794+0000 7f73e2b2a700 1 -- 172.21.15.16:0/2173007410 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73dc11d5d0 con 0x7f73dc134ec0 2023-09-30T22:34:12.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.794+0000 7f73d97fa700 1 -- 172.21.15.16:0/2173007410 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73c8007460 con 0x7f73dc134ec0 2023-09-30T22:34:12.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.794+0000 7f73e2b2a700 1 -- 172.21.15.16:0/2173007410 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73dc1cd7d0 con 0x7f73dc134ec0 2023-09-30T22:34:12.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.795+0000 7f73e2b2a700 1 -- 172.21.15.16:0/2173007410 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73dc09fc20 con 0x7f73dc134ec0 2023-09-30T22:34:12.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.795+0000 7f73d97fa700 1 -- 172.21.15.16:0/2173007410 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f73c80075c0 con 0x7f73dc134ec0 2023-09-30T22:34:12.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.796+0000 7f73d97fa700 1 -- 172.21.15.16:0/2173007410 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f73c803efc0 con 0x7f73dc134ec0 2023-09-30T22:34:12.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.804+0000 7f73d97fa700 1 -- 172.21.15.16:0/2173007410 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f73c801bb10 con 0x7f73dc134ec0 2023-09-30T22:34:12.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.840+0000 7fb08dbb1700 1 Processor -- start 2023-09-30T22:34:12.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.840+0000 7fb08dbb1700 1 -- start start 2023-09-30T22:34:12.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.841+0000 7fb08dbb1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb08809d6f0 0x7fb08809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:12.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.841+0000 7fb08dbb1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb08809e410 0x7fb088134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:12.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.841+0000 7fb08dbb1700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb08809b120 con 0x7fb08809e000 2023-09-30T22:34:12.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.841+0000 7fb08dbb1700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb08806b460 con 0x7fb08809e410 2023-09-30T22:34:12.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.841+0000 7fb08dbb1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb08809a7e0 con 0x7fb08809d6f0 2023-09-30T22:34:12.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.841+0000 7fb0877fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb08809d6f0 0x7fb08809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:12.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.841+0000 7fb0877fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb08809d6f0 0x7fb08809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35214/0 (socket says 172.21.15.16:35214) 2023-09-30T22:34:12.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.841+0000 7fb0877fe700 1 -- 172.21.15.16:0/122125753 learned_addr learned my addr 172.21.15.16:0/122125753 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:12.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.841+0000 7fb087fff700 1 --2- 172.21.15.16:0/122125753 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb08809e410 0x7fb088134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:12.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.841+0000 7fb0867fc700 1 -- 172.21.15.16:0/122125753 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1045857378 0 0) 0x7fb08809b120 con 0x7fb08809e000 2023-09-30T22:34:12.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.841+0000 7fb0867fc700 1 -- 172.21.15.16:0/122125753 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0700035b0 con 0x7fb08809e000 2023-09-30T22:34:12.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.841+0000 7fb087fff700 1 -- 172.21.15.16:0/122125753 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb08809d6f0 msgr2=0x7fb08809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:12.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.841+0000 7fb087fff700 1 --2- 172.21.15.16:0/122125753 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb08809d6f0 0x7fb08809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:12.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.841+0000 7fb087fff700 1 -- 172.21.15.16:0/122125753 >> v1:172.21.15.16:6789/0 conn(0x7fb08809e000 legacy=0x7fb0881339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:12.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.841+0000 7fb087fff700 1 -- 172.21.15.16:0/122125753 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb08809a500 con 0x7fb08809e410 2023-09-30T22:34:12.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.842+0000 7fb0877fe700 1 --2- 172.21.15.16:0/122125753 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb08809d6f0 0x7fb08809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:12.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.842+0000 7fb087fff700 1 --2- 172.21.15.16:0/122125753 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb08809e410 0x7fb088134100 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fb080009dc0 tx=0x7fb08000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b5052823f7267f5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:12.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.842+0000 7fb0867fc700 1 -- 172.21.15.16:0/122125753 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08000e070 con 0x7fb08809e410 2023-09-30T22:34:12.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.842+0000 7fb0867fc700 1 -- 172.21.15.16:0/122125753 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb08000bb20 con 0x7fb08809e410 2023-09-30T22:34:12.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.842+0000 7fb0867fc700 1 -- 172.21.15.16:0/122125753 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb080012430 con 0x7fb08809e410 2023-09-30T22:34:12.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.843+0000 7fb08dbb1700 1 -- 172.21.15.16:0/122125753 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb08809e410 msgr2=0x7fb088134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:12.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.843+0000 7fb08dbb1700 1 --2- 172.21.15.16:0/122125753 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb08809e410 0x7fb088134100 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fb080009dc0 tx=0x7fb08000b330 comp rx=0 tx=0).stop 2023-09-30T22:34:12.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.843+0000 7fb08dbb1700 1 -- 172.21.15.16:0/122125753 shutdown_connections 2023-09-30T22:34:12.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.843+0000 7fb08dbb1700 1 --2- 172.21.15.16:0/122125753 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb08809e410 0x7fb088134100 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:12.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.843+0000 7fb08dbb1700 1 --2- 172.21.15.16:0/122125753 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb08809d6f0 0x7fb08809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:12.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.843+0000 7fb08dbb1700 1 -- 172.21.15.16:0/122125753 >> 172.21.15.16:0/122125753 conn(0x7fb088066f50 msgr2=0x7fb08809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:12.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.843+0000 7fb08dbb1700 1 -- 172.21.15.16:0/122125753 shutdown_connections 2023-09-30T22:34:12.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.843+0000 7fb08dbb1700 1 -- 172.21.15.16:0/122125753 wait complete. 2023-09-30T22:34:12.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.844+0000 7fb08dbb1700 1 Processor -- start 2023-09-30T22:34:12.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.845+0000 7fb08dbb1700 1 -- start start 2023-09-30T22:34:12.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.845+0000 7fb08dbb1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb08809d6f0 0x7fb088122230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:12.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.845+0000 7fb08dbb1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb08809e000 0x7fb088122770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:12.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.845+0000 7fb08dbb1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb08809e410 0x7fb088122cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:12.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.845+0000 7fb08dbb1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb088123e40 con 0x7fb08809d6f0 2023-09-30T22:34:12.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.845+0000 7fb08dbb1700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb088124cd0 con 0x7fb08809e410 2023-09-30T22:34:12.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.845+0000 7fb08dbb1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb0881257b0 con 0x7fb08809e000 2023-09-30T22:34:12.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.846+0000 7fb087fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb08809e410 0x7fb088122cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:12.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.846+0000 7fb087fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb08809e410 0x7fb088122cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:52958/0 (socket says 172.21.15.16:52958) 2023-09-30T22:34:12.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.846+0000 7fb087fff700 1 -- 172.21.15.16:0/2932177479 learned_addr learned my addr 172.21.15.16:0/2932177479 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:12.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.846+0000 7fb0877fe700 1 --2- 172.21.15.16:0/2932177479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb08809d6f0 0x7fb088122230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:12.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.846+0000 7fb086ffd700 1 --2- 172.21.15.16:0/2932177479 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb08809e000 0x7fb088122770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:12.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.846+0000 7fb087fff700 1 -- 172.21.15.16:0/2932177479 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb08809e000 msgr2=0x7fb088122770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:12.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.846+0000 7fb087fff700 1 --2- 172.21.15.16:0/2932177479 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb08809e000 0x7fb088122770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:12.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.846+0000 7fb087fff700 1 -- 172.21.15.16:0/2932177479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb08809d6f0 msgr2=0x7fb088122230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:12.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.846+0000 7fb087fff700 1 --2- 172.21.15.16:0/2932177479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb08809d6f0 0x7fb088122230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:12.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.846+0000 7fb087fff700 1 -- 172.21.15.16:0/2932177479 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb078009550 con 0x7fb08809e410 2023-09-30T22:34:12.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.846+0000 7fb087fff700 1 --2- 172.21.15.16:0/2932177479 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb08809e410 0x7fb088122cb0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fb08000b680 tx=0x7fb080004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:12.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.847+0000 7fb06ffff700 1 -- 172.21.15.16:0/2932177479 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb080004920 con 0x7fb08809e410 2023-09-30T22:34:12.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.847+0000 7fb06ffff700 1 -- 172.21.15.16:0/2932177479 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb080020070 con 0x7fb08809e410 2023-09-30T22:34:12.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.847+0000 7fb08dbb1700 1 -- 172.21.15.16:0/2932177479 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb080009a70 con 0x7fb08809e410 2023-09-30T22:34:12.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.847+0000 7fb06ffff700 1 -- 172.21.15.16:0/2932177479 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0800276c0 con 0x7fb08809e410 2023-09-30T22:34:12.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.847+0000 7fb08dbb1700 1 -- 172.21.15.16:0/2932177479 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb08809c710 con 0x7fb08809e410 2023-09-30T22:34:12.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.848+0000 7fb06ffff700 1 -- 172.21.15.16:0/2932177479 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb080012430 con 0x7fb08809e410 2023-09-30T22:34:12.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.848+0000 7fb08dbb1700 1 -- 172.21.15.16:0/2932177479 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb08811f030 con 0x7fb08809e410 2023-09-30T22:34:12.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.848+0000 7fb06ffff700 1 -- 172.21.15.16:0/2932177479 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fb080022070 con 0x7fb08809e410 2023-09-30T22:34:12.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.857+0000 7fb06ffff700 1 -- 172.21.15.16:0/2932177479 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb0800126e0 con 0x7fb08809e410 2023-09-30T22:34:12.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.873+0000 7fc854eeb700 1 Processor -- start 2023-09-30T22:34:12.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.873+0000 7fc854eeb700 1 -- start start 2023-09-30T22:34:12.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.874+0000 7fc854eeb700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc85009d6f0 0x7fc85009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:12.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.874+0000 7fc854eeb700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc85009e000 0x7fc8501339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:12.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.874+0000 7fc854eeb700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc85009e410 0x7fc850133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:12.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.874+0000 7fc854eeb700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc85006b460 con 0x7fc85009d6f0 2023-09-30T22:34:12.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.874+0000 7fc854eeb700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc85009b120 con 0x7fc85009e000 2023-09-30T22:34:12.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.874+0000 7fc854eeb700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc85009a7e0 con 0x7fc85009e410 2023-09-30T22:34:12.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.874+0000 7fc84e59c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc85009d6f0 0x7fc85009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:12.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.874+0000 7fc84e59c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc85009d6f0 0x7fc85009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33470/0 (socket says 172.21.15.16:33470) 2023-09-30T22:34:12.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.874+0000 7fc84e59c700 1 -- 172.21.15.16:0/277569304 learned_addr learned my addr 172.21.15.16:0/277569304 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:12.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.874+0000 7fc84ed9d700 1 --2- 172.21.15.16:0/277569304 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc85009e410 0x7fc850133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:12.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.874+0000 7fc84dd9b700 1 --2- 172.21.15.16:0/277569304 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc85009e000 0x7fc8501339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:12.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.874+0000 7fc84e59c700 1 -- 172.21.15.16:0/277569304 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc85009e410 msgr2=0x7fc850133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:12.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.874+0000 7fc84e59c700 1 --2- 172.21.15.16:0/277569304 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc85009e410 0x7fc850133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:12.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.874+0000 7fc84e59c700 1 -- 172.21.15.16:0/277569304 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc85009e000 msgr2=0x7fc8501339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:12.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.874+0000 7fc84e59c700 1 --2- 172.21.15.16:0/277569304 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc85009e000 0x7fc8501339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:12.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.874+0000 7fc84e59c700 1 -- 172.21.15.16:0/277569304 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc85009a500 con 0x7fc85009d6f0 2023-09-30T22:34:12.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.874+0000 7fc84dd9b700 1 --2- 172.21.15.16:0/277569304 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc85009e000 0x7fc8501339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:12.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.875+0000 7fc84e59c700 1 --2- 172.21.15.16:0/277569304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc85009d6f0 0x7fc85009dac0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fc8400098f0 tx=0x7fc840009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc4e0912dd3626d6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:12.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.875+0000 7fc84d59a700 1 -- 172.21.15.16:0/277569304 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc840011830 con 0x7fc85009d6f0 2023-09-30T22:34:12.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.875+0000 7fc84d59a700 1 -- 172.21.15.16:0/277569304 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc840011990 con 0x7fc85009d6f0 2023-09-30T22:34:12.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.875+0000 7fc84d59a700 1 -- 172.21.15.16:0/277569304 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc840011d20 con 0x7fc85009d6f0 2023-09-30T22:34:12.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.876+0000 7fc854eeb700 1 -- 172.21.15.16:0/277569304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc85009d6f0 msgr2=0x7fc85009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:12.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.876+0000 7fc854eeb700 1 --2- 172.21.15.16:0/277569304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc85009d6f0 0x7fc85009dac0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fc8400098f0 tx=0x7fc840009c00 comp rx=0 tx=0).stop 2023-09-30T22:34:12.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.876+0000 7fc854eeb700 1 -- 172.21.15.16:0/277569304 shutdown_connections 2023-09-30T22:34:12.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.876+0000 7fc854eeb700 1 --2- 172.21.15.16:0/277569304 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc85009e410 0x7fc850133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:12.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.876+0000 7fc854eeb700 1 --2- 172.21.15.16:0/277569304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc85009d6f0 0x7fc85009dac0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:12.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.876+0000 7fc854eeb700 1 --2- 172.21.15.16:0/277569304 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc85009e000 0x7fc8501339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:12.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.876+0000 7fc854eeb700 1 -- 172.21.15.16:0/277569304 >> 172.21.15.16:0/277569304 conn(0x7fc850066f50 msgr2=0x7fc85009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:12.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.876+0000 7fc854eeb700 1 -- 172.21.15.16:0/277569304 shutdown_connections 2023-09-30T22:34:12.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.876+0000 7fc854eeb700 1 -- 172.21.15.16:0/277569304 wait complete. 2023-09-30T22:34:12.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.878+0000 7fc854eeb700 1 Processor -- start 2023-09-30T22:34:12.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.878+0000 7fc854eeb700 1 -- start start 2023-09-30T22:34:12.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.878+0000 7fc854eeb700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc85009d6f0 0x7fc8501a5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:12.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.878+0000 7fc854eeb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc85009e000 0x7fc8501a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:12.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.878+0000 7fc854eeb700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc85009e410 0x7fc8501a67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:12.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.878+0000 7fc854eeb700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8501a9840 con 0x7fc85009e000 2023-09-30T22:34:12.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.878+0000 7fc854eeb700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8501a8c10 con 0x7fc85009d6f0 2023-09-30T22:34:12.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.879+0000 7fc854eeb700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc8501a7180 con 0x7fc85009e410 2023-09-30T22:34:12.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.879+0000 7fc84ed9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc85009e410 0x7fc8501a67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:12.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.879+0000 7fc84ed9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc85009e410 0x7fc8501a67a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35234/0 (socket says 172.21.15.16:35234) 2023-09-30T22:34:12.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.879+0000 7fc84ed9d700 1 -- 172.21.15.16:0/1197226298 learned_addr learned my addr 172.21.15.16:0/1197226298 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:12.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.879+0000 7fc84dd9b700 1 --2- 172.21.15.16:0/1197226298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc85009e000 0x7fc8501a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:12.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.879+0000 7fc84e59c700 1 --2- 172.21.15.16:0/1197226298 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc85009d6f0 0x7fc8501a5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:12.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.879+0000 7fc84ed9d700 1 -- 172.21.15.16:0/1197226298 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc85009d6f0 msgr2=0x7fc8501a5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:12.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.879+0000 7fc84ed9d700 1 --2- 172.21.15.16:0/1197226298 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc85009d6f0 0x7fc8501a5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:12.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.879+0000 7fc84ed9d700 1 -- 172.21.15.16:0/1197226298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc85009e000 msgr2=0x7fc8501a6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:12.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.879+0000 7fc84ed9d700 1 --2- 172.21.15.16:0/1197226298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc85009e000 0x7fc8501a6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:12.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.879+0000 7fc84ed9d700 1 -- 172.21.15.16:0/1197226298 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8440094a0 con 0x7fc85009e410 2023-09-30T22:34:12.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.879+0000 7fc84e59c700 1 --2- 172.21.15.16:0/1197226298 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc85009d6f0 0x7fc8501a5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:12.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.880+0000 7fc84ed9d700 1 --2- 172.21.15.16:0/1197226298 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc85009e410 0x7fc8501a67a0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fc84800efc0 tx=0x7fc84800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:12.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.880+0000 7fc83effd700 1 -- 172.21.15.16:0/1197226298 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc848009ea0 con 0x7fc85009e410 2023-09-30T22:34:12.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.880+0000 7fc83effd700 1 -- 172.21.15.16:0/1197226298 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc84800f040 con 0x7fc85009e410 2023-09-30T22:34:12.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.880+0000 7fc854eeb700 1 -- 172.21.15.16:0/1197226298 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc840009550 con 0x7fc85009e410 2023-09-30T22:34:12.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.880+0000 7fc83effd700 1 -- 172.21.15.16:0/1197226298 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84800ced0 con 0x7fc85009e410 2023-09-30T22:34:12.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.880+0000 7fc854eeb700 1 -- 172.21.15.16:0/1197226298 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8500a20d0 con 0x7fc85009e410 2023-09-30T22:34:12.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.881+0000 7fc854eeb700 1 -- 172.21.15.16:0/1197226298 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8501a9200 con 0x7fc85009e410 2023-09-30T22:34:12.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.882+0000 7fc83effd700 1 -- 172.21.15.16:0/1197226298 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc848014480 con 0x7fc85009e410 2023-09-30T22:34:12.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.882+0000 7fc83effd700 1 -- 172.21.15.16:0/1197226298 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fc848040250 con 0x7fc85009e410 2023-09-30T22:34:12.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.891+0000 7fc83effd700 1 -- 172.21.15.16:0/1197226298 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc8501a9200 con 0x7fc85009e410 2023-09-30T22:34:12.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.923+0000 7f645823f700 1 Processor -- start 2023-09-30T22:34:12.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.923+0000 7f645823f700 1 -- start start 2023-09-30T22:34:12.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.924+0000 7f645823f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f645009d6f0 0x7f645009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:12.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.924+0000 7f645823f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f645009e000 0x7f6450130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:12.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.924+0000 7f645823f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f645009e410 0x7f6450130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:12.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.924+0000 7f645823f700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f645009f520 con 0x7f645009e000 2023-09-30T22:34:12.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.924+0000 7f645823f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f645009f130 con 0x7f645009e410 2023-09-30T22:34:12.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.924+0000 7f645823f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f645009e930 con 0x7f645009d6f0 2023-09-30T22:34:12.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.924+0000 7f64557da700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f645009e000 0x7f6450130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:12.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.924+0000 7f6455fdb700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f645009d6f0 0x7f645009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:12.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.924+0000 7f64557da700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f645009e000 0x7f6450130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33476/0 (socket says 172.21.15.16:33476) 2023-09-30T22:34:12.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.924+0000 7f64557da700 1 -- 172.21.15.16:0/195524223 learned_addr learned my addr 172.21.15.16:0/195524223 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:12.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.924+0000 7f6455fdb700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f645009d6f0 0x7f645009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35248/0 (socket says 172.21.15.16:35248) 2023-09-30T22:34:12.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.924+0000 7f64567dc700 1 --2- 172.21.15.16:0/195524223 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f645009e410 0x7f6450130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:12.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.924+0000 7f6455fdb700 1 -- 172.21.15.16:0/195524223 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f645009e410 msgr2=0x7f6450130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:12.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.924+0000 7f6455fdb700 1 --2- 172.21.15.16:0/195524223 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f645009e410 0x7f6450130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:12.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.924+0000 7f6455fdb700 1 -- 172.21.15.16:0/195524223 >> v2:172.21.15.16:3300/0 conn(0x7f645009e000 msgr2=0x7f6450130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:12.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.924+0000 7f6455fdb700 1 --2- 172.21.15.16:0/195524223 >> v2:172.21.15.16:3300/0 conn(0x7f645009e000 0x7f6450130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:12.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.924+0000 7f6455fdb700 1 -- 172.21.15.16:0/195524223 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f645009b2d0 con 0x7f645009d6f0 2023-09-30T22:34:12.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.924+0000 7f64557da700 1 --2- 172.21.15.16:0/195524223 >> v2:172.21.15.16:3300/0 conn(0x7f645009e000 0x7f6450130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:12.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.925+0000 7f6455fdb700 1 --2- 172.21.15.16:0/195524223 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f645009d6f0 0x7f645009dac0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f644c009870 tx=0x7f644c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5288c135d667d6de server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:12.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.925+0000 7f6454fd9700 1 -- 172.21.15.16:0/195524223 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644c011830 con 0x7f645009d6f0 2023-09-30T22:34:12.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.925+0000 7f6454fd9700 1 -- 172.21.15.16:0/195524223 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f644c011990 con 0x7f645009d6f0 2023-09-30T22:34:12.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.925+0000 7f6454fd9700 1 -- 172.21.15.16:0/195524223 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644c011d20 con 0x7f645009d6f0 2023-09-30T22:34:12.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.926+0000 7f645823f700 1 -- 172.21.15.16:0/195524223 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f645009d6f0 msgr2=0x7f645009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:12.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.926+0000 7f645823f700 1 --2- 172.21.15.16:0/195524223 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f645009d6f0 0x7f645009dac0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f644c009870 tx=0x7f644c009b80 comp rx=0 tx=0).stop 2023-09-30T22:34:12.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.926+0000 7f645823f700 1 -- 172.21.15.16:0/195524223 shutdown_connections 2023-09-30T22:34:12.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.926+0000 7f645823f700 1 --2- 172.21.15.16:0/195524223 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f645009e410 0x7f6450130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:12.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.926+0000 7f645823f700 1 --2- 172.21.15.16:0/195524223 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f645009d6f0 0x7f645009dac0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:12.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.926+0000 7f645823f700 1 --2- 172.21.15.16:0/195524223 >> v2:172.21.15.16:3300/0 conn(0x7f645009e000 0x7f6450130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:12.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.926+0000 7f645823f700 1 -- 172.21.15.16:0/195524223 >> 172.21.15.16:0/195524223 conn(0x7f6450066f50 msgr2=0x7f645009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:12.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.926+0000 7f645823f700 1 -- 172.21.15.16:0/195524223 shutdown_connections 2023-09-30T22:34:12.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.926+0000 7f645823f700 1 -- 172.21.15.16:0/195524223 wait complete. 2023-09-30T22:34:12.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.927+0000 7f645823f700 1 Processor -- start 2023-09-30T22:34:12.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.927+0000 7f645823f700 1 -- start start 2023-09-30T22:34:12.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.927+0000 7f645823f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f645009d6f0 0x7f645011e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:12.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.927+0000 7f645823f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f645009e000 0x7f645011e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:12.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.927+0000 7f645823f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f645009e410 0x7f645011ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:12.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.927+0000 7f645823f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645009c770 con 0x7f645009d6f0 2023-09-30T22:34:12.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.927+0000 7f645823f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64500a3b10 con 0x7f645009e000 2023-09-30T22:34:12.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.927+0000 7f645823f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f64501205a0 con 0x7f645009e410 2023-09-30T22:34:12.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.927+0000 7f6455fdb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f645009d6f0 0x7f645011e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:12.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.927+0000 7f6455fdb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f645009d6f0 0x7f645011e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33482/0 (socket says 172.21.15.16:33482) 2023-09-30T22:34:12.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.927+0000 7f6455fdb700 1 -- 172.21.15.16:0/3734091635 learned_addr learned my addr 172.21.15.16:0/3734091635 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:12.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.928+0000 7f64557da700 1 --2- 172.21.15.16:0/3734091635 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f645009e000 0x7f645011e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:12.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.928+0000 7f6455fdb700 1 -- 172.21.15.16:0/3734091635 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f645009e410 msgr2=0x7f645011ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:12.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.928+0000 7f6455fdb700 1 --2- 172.21.15.16:0/3734091635 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f645009e410 0x7f645011ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:12.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.928+0000 7f6455fdb700 1 -- 172.21.15.16:0/3734091635 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f645009e000 msgr2=0x7f645011e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:12.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.928+0000 7f6455fdb700 1 --2- 172.21.15.16:0/3734091635 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f645009e000 0x7f645011e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:12.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.928+0000 7f6455fdb700 1 -- 172.21.15.16:0/3734091635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f644000b7f0 con 0x7f645009d6f0 2023-09-30T22:34:12.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.928+0000 7f6455fdb700 1 --2- 172.21.15.16:0/3734091635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f645009d6f0 0x7f645011e130 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f644c000c00 tx=0x7f644c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:12.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.928+0000 7f64467fc700 1 -- 172.21.15.16:0/3734091635 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644c004590 con 0x7f645009d6f0 2023-09-30T22:34:12.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.928+0000 7f64467fc700 1 -- 172.21.15.16:0/3734091635 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f644c01f070 con 0x7f645009d6f0 2023-09-30T22:34:12.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.928+0000 7f645823f700 1 -- 172.21.15.16:0/3734091635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f644c0094a0 con 0x7f645009d6f0 2023-09-30T22:34:12.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.928+0000 7f64467fc700 1 -- 172.21.15.16:0/3734091635 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644c0038c0 con 0x7f645009d6f0 2023-09-30T22:34:12.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.928+0000 7f645823f700 1 -- 172.21.15.16:0/3734091635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64501216b0 con 0x7f645009d6f0 2023-09-30T22:34:12.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.929+0000 7f645823f700 1 -- 172.21.15.16:0/3734091635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f645011ac20 con 0x7f645009d6f0 2023-09-30T22:34:12.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.930+0000 7f64467fc700 1 -- 172.21.15.16:0/3734091635 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f644c003a20 con 0x7f645009d6f0 2023-09-30T22:34:12.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.930+0000 7f64467fc700 1 -- 172.21.15.16:0/3734091635 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f644c021070 con 0x7f645009d6f0 2023-09-30T22:34:12.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:12.934+0000 7f64467fc700 1 -- 172.21.15.16:0/3734091635 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f644c010380 con 0x7f645009d6f0 2023-09-30T22:34:13.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:13.062+0000 7f73e2b2a700 1 -- 172.21.15.16:0/2173007410 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f73dc11daa0 con 0x7f73dc134ec0 2023-09-30T22:34:13.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:13.062+0000 7f73d97fa700 1 -- 172.21.15.16:0/2173007410 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f73c80192c0 con 0x7f73dc134ec0 2023-09-30T22:34:13.063 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:34:13.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:13.065+0000 7f73e2b2a700 1 -- 172.21.15.16:0/2173007410 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f73dc134ec0 msgr2=0x7f73dc1cd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:13.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:13.065+0000 7f73e2b2a700 1 --2- 172.21.15.16:0/2173007410 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f73dc134ec0 0x7f73dc1cd290 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f73c8005910 tx=0x7f73c800c9e0 comp rx=0 tx=0).stop 2023-09-30T22:34:13.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:13.065+0000 7f73e2b2a700 1 -- 172.21.15.16:0/2173007410 shutdown_connections 2023-09-30T22:34:13.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:13.065+0000 7f73e2b2a700 1 --2- 172.21.15.16:0/2173007410 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f73dc1380f0 0x7f73dc11a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:13.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:13.065+0000 7f73e2b2a700 1 --2- 172.21.15.16:0/2173007410 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73dc06c0e0 0x7f73dc1ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:13.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:13.065+0000 7f73e2b2a700 1 --2- 172.21.15.16:0/2173007410 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f73dc134ec0 0x7f73dc1cd290 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:13.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:13.065+0000 7f73e2b2a700 1 -- 172.21.15.16:0/2173007410 >> 172.21.15.16:0/2173007410 conn(0x7f73dc0a4210 msgr2=0x7f73dc09a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:13.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:13.065+0000 7f73e2b2a700 1 -- 172.21.15.16:0/2173007410 shutdown_connections 2023-09-30T22:34:13.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:13.065+0000 7f73e2b2a700 1 -- 172.21.15.16:0/2173007410 wait complete. 2023-09-30T22:34:13.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:13.073+0000 7fb08dbb1700 1 -- 172.21.15.16:0/2932177479 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb088074ea0 con 0x7fb08809e410 2023-09-30T22:34:13.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:13.073+0000 7fb06ffff700 1 -- 172.21.15.16:0/2932177479 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb0800303e0 con 0x7fb08809e410 2023-09-30T22:34:13.074 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:34:13.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:13.076+0000 7fb08dbb1700 1 -- 172.21.15.16:0/2932177479 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb08809e410 msgr2=0x7fb088122cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:13.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:13.076+0000 7fb08dbb1700 1 --2- 172.21.15.16:0/2932177479 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb08809e410 0x7fb088122cb0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fb08000b680 tx=0x7fb080004ad0 comp rx=0 tx=0).stop 2023-09-30T22:34:13.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:13.076+0000 7fb08dbb1700 1 -- 172.21.15.16:0/2932177479 shutdown_connections 2023-09-30T22:34:13.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:13.076+0000 7fb08dbb1700 1 --2- 172.21.15.16:0/2932177479 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb08809e410 0x7fb088122cb0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:13.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:13.076+0000 7fb08dbb1700 1 --2- 172.21.15.16:0/2932177479 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb08809d6f0 0x7fb088122230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:13.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:13.076+0000 7fb08dbb1700 1 --2- 172.21.15.16:0/2932177479 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb08809e000 0x7fb088122770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:13.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:13.076+0000 7fb08dbb1700 1 -- 172.21.15.16:0/2932177479 >> 172.21.15.16:0/2932177479 conn(0x7fb088066f50 msgr2=0x7fb088133db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:13.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:13.076+0000 7fb08dbb1700 1 -- 172.21.15.16:0/2932177479 shutdown_connections 2023-09-30T22:34:13.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:13.076+0000 7fb08dbb1700 1 -- 172.21.15.16:0/2932177479 wait complete. 2023-09-30T22:34:13.086 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:34:13.091 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:34:13.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:13.128+0000 7fc854eeb700 1 -- 172.21.15.16:0/1197226298 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc8501a9200 con 0x7fc85009e410 2023-09-30T22:34:13.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:13.128+0000 7fc83effd700 1 -- 172.21.15.16:0/1197226298 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc8501a9200 con 0x7fc85009e410 2023-09-30T22:34:13.129 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:34:13.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:13.131+0000 7fc854eeb700 1 -- 172.21.15.16:0/1197226298 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc85009e410 msgr2=0x7fc8501a67a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:13.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:13.131+0000 7fc854eeb700 1 --2- 172.21.15.16:0/1197226298 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc85009e410 0x7fc8501a67a0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fc84800efc0 tx=0x7fc84800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:34:13.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:13.131+0000 7fc854eeb700 1 -- 172.21.15.16:0/1197226298 shutdown_connections 2023-09-30T22:34:13.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:13.131+0000 7fc854eeb700 1 --2- 172.21.15.16:0/1197226298 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc85009d6f0 0x7fc8501a5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:13.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:13.131+0000 7fc854eeb700 1 --2- 172.21.15.16:0/1197226298 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc85009e410 0x7fc8501a67a0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:13.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:13.131+0000 7fc854eeb700 1 --2- 172.21.15.16:0/1197226298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc85009e000 0x7fc8501a6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:13.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:13.131+0000 7fc854eeb700 1 -- 172.21.15.16:0/1197226298 >> 172.21.15.16:0/1197226298 conn(0x7fc850066f50 msgr2=0x7fc8500a2620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:13.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:13.131+0000 7fc854eeb700 1 -- 172.21.15.16:0/1197226298 shutdown_connections 2023-09-30T22:34:13.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:13.131+0000 7fc854eeb700 1 -- 172.21.15.16:0/1197226298 wait complete. 2023-09-30T22:34:13.145 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:34:13.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:13.152+0000 7f645823f700 1 -- 172.21.15.16:0/3734091635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f6450074ea0 con 0x7f645009d6f0 2023-09-30T22:34:13.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:13.152+0000 7f64467fc700 1 -- 172.21.15.16:0/3734091635 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f644c015030 con 0x7f645009d6f0 2023-09-30T22:34:13.152 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:34:13.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:13.154+0000 7f645823f700 1 -- 172.21.15.16:0/3734091635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f645009d6f0 msgr2=0x7f645011e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:13.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:13.154+0000 7f645823f700 1 --2- 172.21.15.16:0/3734091635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f645009d6f0 0x7f645011e130 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f644c000c00 tx=0x7f644c004750 comp rx=0 tx=0).stop 2023-09-30T22:34:13.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:13.154+0000 7f645823f700 1 -- 172.21.15.16:0/3734091635 shutdown_connections 2023-09-30T22:34:13.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:13.154+0000 7f645823f700 1 --2- 172.21.15.16:0/3734091635 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f645009e410 0x7f645011ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:13.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:13.154+0000 7f645823f700 1 --2- 172.21.15.16:0/3734091635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f645009d6f0 0x7f645011e130 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:13.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:13.154+0000 7f645823f700 1 --2- 172.21.15.16:0/3734091635 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f645009e000 0x7f645011e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:13.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:13.154+0000 7f645823f700 1 -- 172.21.15.16:0/3734091635 >> 172.21.15.16:0/3734091635 conn(0x7f6450066f50 msgr2=0x7f6450130920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:13.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:13.154+0000 7f645823f700 1 -- 172.21.15.16:0/3734091635 shutdown_connections 2023-09-30T22:34:13.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:13.154+0000 7f645823f700 1 -- 172.21.15.16:0/3734091635 wait complete. 2023-09-30T22:34:13.168 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:34:14.087 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:34:14.092 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:34:14.146 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:34:14.169 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:34:14.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.250+0000 7fd1b1cc2700 1 Processor -- start 2023-09-30T22:34:14.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.250+0000 7fd1b1cc2700 1 -- start start 2023-09-30T22:34:14.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.251+0000 7fd1b1cc2700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd1ac09e930 0x7fd1ac09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:14.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.251+0000 7fd1b1cc2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd1ac09f650 0x7fd1ac131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:14.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.251+0000 7fd1b1cc2700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1ac070940 con 0x7fd1ac09f240 2023-09-30T22:34:14.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.251+0000 7fd1b1cc2700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1ac06fd60 con 0x7fd1ac09e930 2023-09-30T22:34:14.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.251+0000 7fd1b1cc2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ac06fa80 con 0x7fd1ac09f650 2023-09-30T22:34:14.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.251+0000 7fd1ab7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd1ac09e930 0x7fd1ac09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:14.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.251+0000 7fd1aaffd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd1ac09f240 0x7fd1ac1317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59800/0 (socket says 172.21.15.16:59800) 2023-09-30T22:34:14.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.251+0000 7fd1aaffd700 1 -- 172.21.15.16:0/3162372625 learned_addr learned my addr 172.21.15.16:0/3162372625 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:34:14.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.252+0000 7fd1abfff700 1 --2- 172.21.15.16:0/3162372625 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd1ac09f650 0x7fd1ac131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:14.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.252+0000 7fd1ab7fe700 1 -- 172.21.15.16:0/3162372625 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd1ac09f650 msgr2=0x7fd1ac131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:14.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.252+0000 7fd1ab7fe700 1 --2- 172.21.15.16:0/3162372625 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd1ac09f650 0x7fd1ac131f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:14.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.252+0000 7fd1ab7fe700 1 -- 172.21.15.16:0/3162372625 >> v1:172.21.15.16:6789/0 conn(0x7fd1ac09f240 legacy=0x7fd1ac1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:14.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.252+0000 7fd1ab7fe700 1 -- 172.21.15.16:0/3162372625 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1ac09fb60 con 0x7fd1ac09e930 2023-09-30T22:34:14.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.252+0000 7fd1abfff700 1 --2- 172.21.15.16:0/3162372625 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd1ac09f650 0x7fd1ac131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:14.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.252+0000 7fd1ab7fe700 1 --2- 172.21.15.16:0/3162372625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1ac09e930 0x7fd1ac09ed00 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fd19c009840 tx=0x7fd19c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7596b7981dbab01 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:14.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.252+0000 7fd1aa7fc700 1 -- 172.21.15.16:0/3162372625 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19c011830 con 0x7fd1ac09e930 2023-09-30T22:34:14.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.252+0000 7fd1aa7fc700 1 -- 172.21.15.16:0/3162372625 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd19c011990 con 0x7fd1ac09e930 2023-09-30T22:34:14.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.252+0000 7fd1aa7fc700 1 -- 172.21.15.16:0/3162372625 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19c011d20 con 0x7fd1ac09e930 2023-09-30T22:34:14.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.253+0000 7fd1b1cc2700 1 -- 172.21.15.16:0/3162372625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1ac09e930 msgr2=0x7fd1ac09ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:14.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.253+0000 7fd1b1cc2700 1 --2- 172.21.15.16:0/3162372625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1ac09e930 0x7fd1ac09ed00 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fd19c009840 tx=0x7fd19c009b50 comp rx=0 tx=0).stop 2023-09-30T22:34:14.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.254+0000 7fd1b1cc2700 1 -- 172.21.15.16:0/3162372625 shutdown_connections 2023-09-30T22:34:14.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.254+0000 7fd1b1cc2700 1 --2- 172.21.15.16:0/3162372625 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd1ac09f650 0x7fd1ac131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:14.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.254+0000 7fd1b1cc2700 1 --2- 172.21.15.16:0/3162372625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1ac09e930 0x7fd1ac09ed00 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:14.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.254+0000 7fd1b1cc2700 1 -- 172.21.15.16:0/3162372625 >> 172.21.15.16:0/3162372625 conn(0x7fd1ac0a4210 msgr2=0x7fd1ac0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:14.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.254+0000 7fd1b1cc2700 1 -- 172.21.15.16:0/3162372625 shutdown_connections 2023-09-30T22:34:14.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.254+0000 7fd1b1cc2700 1 -- 172.21.15.16:0/3162372625 wait complete. 2023-09-30T22:34:14.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.255+0000 7fd1b1cc2700 1 Processor -- start 2023-09-30T22:34:14.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.256+0000 7fd1b1cc2700 1 -- start start 2023-09-30T22:34:14.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.256+0000 7fd1b1cc2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd1ac09e930 0x7fd1ac1a3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:14.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.256+0000 7fd1b1cc2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1ac09f240 0x7fd1ac1a40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:14.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.256+0000 7fd1b1cc2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd1ac09f650 0x7fd1ac1a4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:14.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.256+0000 7fd1b1cc2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ac1a8020 con 0x7fd1ac09f240 2023-09-30T22:34:14.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.256+0000 7fd1b1cc2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ac1a7660 con 0x7fd1ac09e930 2023-09-30T22:34:14.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.256+0000 7fd1b1cc2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ac1a6a30 con 0x7fd1ac09f650 2023-09-30T22:34:14.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.257+0000 7fd1aaffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1ac09f240 0x7fd1ac1a40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:14.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.257+0000 7fd1aaffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1ac09f240 0x7fd1ac1a40f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33492/0 (socket says 172.21.15.16:33492) 2023-09-30T22:34:14.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.257+0000 7fd1aaffd700 1 -- 172.21.15.16:0/3608300931 learned_addr learned my addr 172.21.15.16:0/3608300931 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:14.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.257+0000 7fd1abfff700 1 --2- 172.21.15.16:0/3608300931 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd1ac09f650 0x7fd1ac1a4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:14.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.257+0000 7fd1aaffd700 1 -- 172.21.15.16:0/3608300931 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd1ac09f650 msgr2=0x7fd1ac1a4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:14.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.257+0000 7fd1aaffd700 1 --2- 172.21.15.16:0/3608300931 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd1ac09f650 0x7fd1ac1a4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:14.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.257+0000 7fd1aaffd700 1 -- 172.21.15.16:0/3608300931 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd1ac09e930 msgr2=0x7fd1ac1a3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:14.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.257+0000 7fd1aaffd700 1 --2- 172.21.15.16:0/3608300931 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd1ac09e930 0x7fd1ac1a3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:14.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.257+0000 7fd1aaffd700 1 -- 172.21.15.16:0/3608300931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1a4009a70 con 0x7fd1ac09f240 2023-09-30T22:34:14.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.257+0000 7fd1aaffd700 1 --2- 172.21.15.16:0/3608300931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1ac09f240 0x7fd1ac1a40f0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fd1a000c550 tx=0x7fd1a000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:14.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.257+0000 7fd193fff700 1 -- 172.21.15.16:0/3608300931 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a0005d70 con 0x7fd1ac09f240 2023-09-30T22:34:14.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.257+0000 7fd193fff700 1 -- 172.21.15.16:0/3608300931 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1a00070d0 con 0x7fd1ac09f240 2023-09-30T22:34:14.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.257+0000 7fd1b1cc2700 1 -- 172.21.15.16:0/3608300931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd19c0094a0 con 0x7fd1ac09f240 2023-09-30T22:34:14.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.257+0000 7fd193fff700 1 -- 172.21.15.16:0/3608300931 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a0007440 con 0x7fd1ac09f240 2023-09-30T22:34:14.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.258+0000 7fd1b1cc2700 1 -- 172.21.15.16:0/3608300931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1ac0a0190 con 0x7fd1ac09f240 2023-09-30T22:34:14.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.259+0000 7fd193fff700 1 -- 172.21.15.16:0/3608300931 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd1a00075a0 con 0x7fd1ac09f240 2023-09-30T22:34:14.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.259+0000 7fd193fff700 1 -- 172.21.15.16:0/3608300931 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd1a003ec30 con 0x7fd1ac09f240 2023-09-30T22:34:14.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.259+0000 7fd1b1cc2700 1 -- 172.21.15.16:0/3608300931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1ac1a5d30 con 0x7fd1ac09f240 2023-09-30T22:34:14.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.268+0000 7fd193fff700 1 -- 172.21.15.16:0/3608300931 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd1a0017030 con 0x7fd1ac09f240 2023-09-30T22:34:14.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.294+0000 7f574f7e2700 1 Processor -- start 2023-09-30T22:34:14.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.295+0000 7f574f7e2700 1 -- start start 2023-09-30T22:34:14.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.295+0000 7f574f7e2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f574809e1d0 0x7f5748138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:14.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.295+0000 7f574f7e2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5748139170 0x7f574813b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:14.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.295+0000 7f574f7e2700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f574809b120 con 0x7f574809d6f0 2023-09-30T22:34:14.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.295+0000 7f574f7e2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f574806b460 con 0x7f574809e1d0 2023-09-30T22:34:14.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.295+0000 7f574f7e2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f574809a7e0 con 0x7f5748139170 2023-09-30T22:34:14.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.295+0000 7f574cd7d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f574809e1d0 0x7f5748138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:14.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.295+0000 7f574cd7d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f574809e1d0 0x7f5748138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:53002/0 (socket says 172.21.15.16:53002) 2023-09-30T22:34:14.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.295+0000 7f574cd7d700 1 -- 172.21.15.16:0/1454565594 learned_addr learned my addr 172.21.15.16:0/1454565594 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:14.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.295+0000 7f574dd7f700 1 --2- 172.21.15.16:0/1454565594 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5748139170 0x7f574813b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:14.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.296+0000 7f574cd7d700 1 -- 172.21.15.16:0/1454565594 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5748139170 msgr2=0x7f574813b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:14.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.296+0000 7f574cd7d700 1 --2- 172.21.15.16:0/1454565594 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5748139170 0x7f574813b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:14.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.296+0000 7f574cd7d700 1 -- 172.21.15.16:0/1454565594 >> v1:172.21.15.16:6789/0 conn(0x7f574809d6f0 legacy=0x7f574809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:14.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.296+0000 7f574cd7d700 1 -- 172.21.15.16:0/1454565594 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f574809a500 con 0x7f574809e1d0 2023-09-30T22:34:14.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.296+0000 7f574dd7f700 1 --2- 172.21.15.16:0/1454565594 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5748139170 0x7f574813b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:34:14.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.296+0000 7f574cd7d700 1 --2- 172.21.15.16:0/1454565594 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f574809e1d0 0x7f5748138c30 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f5738009920 tx=0x7f5738009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbeb7cf253060a3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:14.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.296+0000 7f573ffff700 1 -- 172.21.15.16:0/1454565594 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573800ba60 con 0x7f574809e1d0 2023-09-30T22:34:14.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.296+0000 7f573ffff700 1 -- 172.21.15.16:0/1454565594 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f573800bbc0 con 0x7f574809e1d0 2023-09-30T22:34:14.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.296+0000 7f573ffff700 1 -- 172.21.15.16:0/1454565594 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5738009e70 con 0x7f574809e1d0 2023-09-30T22:34:14.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.297+0000 7f574f7e2700 1 -- 172.21.15.16:0/1454565594 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f574809e1d0 msgr2=0x7f5748138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:14.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.297+0000 7f574f7e2700 1 --2- 172.21.15.16:0/1454565594 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f574809e1d0 0x7f5748138c30 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f5738009920 tx=0x7f5738009c30 comp rx=0 tx=0).stop 2023-09-30T22:34:14.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.297+0000 7f574f7e2700 1 -- 172.21.15.16:0/1454565594 shutdown_connections 2023-09-30T22:34:14.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.297+0000 7f574f7e2700 1 --2- 172.21.15.16:0/1454565594 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5748139170 0x7f574813b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:14.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.297+0000 7f574f7e2700 1 --2- 172.21.15.16:0/1454565594 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f574809e1d0 0x7f5748138c30 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:14.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.297+0000 7f574f7e2700 1 -- 172.21.15.16:0/1454565594 >> 172.21.15.16:0/1454565594 conn(0x7f5748066f50 msgr2=0x7f574809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:14.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.298+0000 7f574f7e2700 1 -- 172.21.15.16:0/1454565594 shutdown_connections 2023-09-30T22:34:14.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.298+0000 7f574f7e2700 1 -- 172.21.15.16:0/1454565594 wait complete. 2023-09-30T22:34:14.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.299+0000 7f574f7e2700 1 Processor -- start 2023-09-30T22:34:14.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.299+0000 7f574f7e2700 1 -- start start 2023-09-30T22:34:14.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.299+0000 7f574f7e2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f574809d6f0 0x7f5748120820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:14.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.299+0000 7f574f7e2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f574809e1d0 0x7f5748120d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:14.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.299+0000 7f574f7e2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5748139170 0x7f57481212a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:14.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.299+0000 7f574f7e2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5748124340 con 0x7f574809e1d0 2023-09-30T22:34:14.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.299+0000 7f574f7e2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f574809c710 con 0x7f574809d6f0 2023-09-30T22:34:14.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.299+0000 7f574f7e2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f57480a1880 con 0x7f5748139170 2023-09-30T22:34:14.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.299+0000 7f574d57e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f574809d6f0 0x7f5748120820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:14.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.299+0000 7f574dd7f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5748139170 0x7f57481212a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:14.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.299+0000 7f574d57e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f574809d6f0 0x7f5748120820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:53008/0 (socket says 172.21.15.16:53008) 2023-09-30T22:34:14.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.299+0000 7f574d57e700 1 -- 172.21.15.16:0/2091863949 learned_addr learned my addr 172.21.15.16:0/2091863949 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:14.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.299+0000 7f574cd7d700 1 --2- 172.21.15.16:0/2091863949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f574809e1d0 0x7f5748120d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:14.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.300+0000 7f574d57e700 1 -- 172.21.15.16:0/2091863949 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5748139170 msgr2=0x7f57481212a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:14.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.300+0000 7f574d57e700 1 --2- 172.21.15.16:0/2091863949 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5748139170 0x7f57481212a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:14.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.300+0000 7f574d57e700 1 -- 172.21.15.16:0/2091863949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f574809e1d0 msgr2=0x7f5748120d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:14.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.300+0000 7f574d57e700 1 --2- 172.21.15.16:0/2091863949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f574809e1d0 0x7f5748120d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:14.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.300+0000 7f574d57e700 1 -- 172.21.15.16:0/2091863949 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5738009550 con 0x7f574809d6f0 2023-09-30T22:34:14.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.300+0000 7f574d57e700 1 --2- 172.21.15.16:0/2091863949 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f574809d6f0 0x7f5748120820 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f574400c4e0 tx=0x7f574400c7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:14.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.300+0000 7f573dffb700 1 -- 172.21.15.16:0/2091863949 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57440052d0 con 0x7f574809d6f0 2023-09-30T22:34:14.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.300+0000 7f573dffb700 1 -- 172.21.15.16:0/2091863949 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5744005430 con 0x7f574809d6f0 2023-09-30T22:34:14.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.300+0000 7f574f7e2700 1 -- 172.21.15.16:0/2091863949 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5748122a00 con 0x7f574809d6f0 2023-09-30T22:34:14.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.300+0000 7f573dffb700 1 -- 172.21.15.16:0/2091863949 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57440038c0 con 0x7f574809d6f0 2023-09-30T22:34:14.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.301+0000 7f574f7e2700 1 -- 172.21.15.16:0/2091863949 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5748122c20 con 0x7f574809d6f0 2023-09-30T22:34:14.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.302+0000 7f574f7e2700 1 -- 172.21.15.16:0/2091863949 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57481367a0 con 0x7f574809d6f0 2023-09-30T22:34:14.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.302+0000 7f573dffb700 1 -- 172.21.15.16:0/2091863949 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5744003a20 con 0x7f574809d6f0 2023-09-30T22:34:14.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.303+0000 7f573dffb700 1 -- 172.21.15.16:0/2091863949 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f574403ecd0 con 0x7f574809d6f0 2023-09-30T22:34:14.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.311+0000 7f573dffb700 1 -- 172.21.15.16:0/2091863949 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5744003cd0 con 0x7f574809d6f0 2023-09-30T22:34:14.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.327+0000 7f6d7477f700 1 Processor -- start 2023-09-30T22:34:14.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.327+0000 7f6d7477f700 1 -- start start 2023-09-30T22:34:14.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.328+0000 7f6d7477f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d6c06a770 0x7f6d6c09d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:14.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.328+0000 7f6d7477f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d6c09dc30 0x7f6d6c09e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:14.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.328+0000 7f6d7477f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6d6c13c1e0 0x7f6d6c13e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:14.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.328+0000 7f6d7477f700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d6c06ab40 con 0x7f6d6c13c1e0 2023-09-30T22:34:14.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.328+0000 7f6d7477f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6c09ebc0 con 0x7f6d6c09dc30 2023-09-30T22:34:14.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.328+0000 7f6d7477f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6c09a810 con 0x7f6d6c06a770 2023-09-30T22:34:14.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.328+0000 7f6d71d1a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d6c09dc30 0x7f6d6c09e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:14.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.328+0000 7f6d71d1a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d6c09dc30 0x7f6d6c09e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:53024/0 (socket says 172.21.15.16:53024) 2023-09-30T22:34:14.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.328+0000 7f6d72d1c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6d6c13c1e0 0x7f6d6c13e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:14.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.328+0000 7f6d71d1a700 1 -- 172.21.15.16:0/643037669 learned_addr learned my addr 172.21.15.16:0/643037669 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:14.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.328+0000 7f6d7251b700 1 --2- 172.21.15.16:0/643037669 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d6c06a770 0x7f6d6c09d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:14.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.328+0000 7f6d71d1a700 1 -- 172.21.15.16:0/643037669 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d6c06a770 msgr2=0x7f6d6c09d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:14.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.328+0000 7f6d71d1a700 1 --2- 172.21.15.16:0/643037669 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d6c06a770 0x7f6d6c09d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:14.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.328+0000 7f6d71d1a700 1 -- 172.21.15.16:0/643037669 >> v2:172.21.15.16:3300/0 conn(0x7f6d6c13c1e0 msgr2=0x7f6d6c13e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:14.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.328+0000 7f6d71d1a700 1 --2- 172.21.15.16:0/643037669 >> v2:172.21.15.16:3300/0 conn(0x7f6d6c13c1e0 0x7f6d6c13e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:14.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.329+0000 7f6d71d1a700 1 -- 172.21.15.16:0/643037669 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d6c09e920 con 0x7f6d6c09dc30 2023-09-30T22:34:14.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.329+0000 7f6d7251b700 1 --2- 172.21.15.16:0/643037669 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d6c06a770 0x7f6d6c09d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:34:14.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.329+0000 7f6d71d1a700 1 --2- 172.21.15.16:0/643037669 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d6c09dc30 0x7f6d6c09e080 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c009920 tx=0x7f6d5c009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b5fb863c5078b2c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:14.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.329+0000 7f6d71519700 1 -- 172.21.15.16:0/643037669 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c011830 con 0x7f6d6c09dc30 2023-09-30T22:34:14.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.329+0000 7f6d71519700 1 -- 172.21.15.16:0/643037669 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d5c011990 con 0x7f6d6c09dc30 2023-09-30T22:34:14.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.329+0000 7f6d71519700 1 -- 172.21.15.16:0/643037669 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c011d20 con 0x7f6d6c09dc30 2023-09-30T22:34:14.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.330+0000 7f1b4d2a0700 1 Processor -- start 2023-09-30T22:34:14.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.330+0000 7f1b4d2a0700 1 -- start start 2023-09-30T22:34:14.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.330+0000 7f6d7477f700 1 -- 172.21.15.16:0/643037669 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d6c09dc30 msgr2=0x7f6d6c09e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:14.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.330+0000 7f6d7477f700 1 --2- 172.21.15.16:0/643037669 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d6c09dc30 0x7f6d6c09e080 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c009920 tx=0x7f6d5c009c30 comp rx=0 tx=0).stop 2023-09-30T22:34:14.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.330+0000 7f1b4d2a0700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1b48130540 0x7f1b48130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:14.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.330+0000 7f1b4d2a0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1b48130e50 0x7f1b48135750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:14.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.330+0000 7f1b4d2a0700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b480716a0 con 0x7f1b48135c90 2023-09-30T22:34:14.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.330+0000 7f1b4d2a0700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b4809a390 con 0x7f1b48130540 2023-09-30T22:34:14.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.330+0000 7f1b4d2a0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b48071290 con 0x7f1b48130e50 2023-09-30T22:34:14.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.330+0000 7f6d7477f700 1 -- 172.21.15.16:0/643037669 shutdown_connections 2023-09-30T22:34:14.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.330+0000 7f6d7477f700 1 --2- 172.21.15.16:0/643037669 >> v2:172.21.15.16:3300/0 conn(0x7f6d6c13c1e0 0x7f6d6c13e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:14.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.330+0000 7f6d7477f700 1 --2- 172.21.15.16:0/643037669 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d6c06a770 0x7f6d6c09d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:14.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.330+0000 7f6d7477f700 1 --2- 172.21.15.16:0/643037669 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d6c09dc30 0x7f6d6c09e080 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:14.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.330+0000 7f6d7477f700 1 -- 172.21.15.16:0/643037669 >> 172.21.15.16:0/643037669 conn(0x7f6d6c066f50 msgr2=0x7f6d6c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:14.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.331+0000 7f1b46ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1b48130540 0x7f1b48130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:14.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.331+0000 7f1b467fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1b48130e50 0x7f1b48135750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:14.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.331+0000 7f1b46ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1b48130540 0x7f1b48130910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33518/0 (socket says 172.21.15.16:33518) 2023-09-30T22:34:14.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.331+0000 7f1b46ffd700 1 -- 172.21.15.16:0/3041440440 learned_addr learned my addr 172.21.15.16:0/3041440440 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:14.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.331+0000 7f6d7477f700 1 -- 172.21.15.16:0/643037669 shutdown_connections 2023-09-30T22:34:14.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.331+0000 7f6d7477f700 1 -- 172.21.15.16:0/643037669 wait complete. 2023-09-30T22:34:14.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.331+0000 7f1b46ffd700 1 -- 172.21.15.16:0/3041440440 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1b48130e50 msgr2=0x7f1b48135750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:14.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.331+0000 7f1b46ffd700 1 --2- 172.21.15.16:0/3041440440 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1b48130e50 0x7f1b48135750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:14.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.331+0000 7f1b46ffd700 1 -- 172.21.15.16:0/3041440440 >> v1:172.21.15.16:6789/0 conn(0x7f1b48135c90 legacy=0x7f1b48138070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:14.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.331+0000 7f1b46ffd700 1 -- 172.21.15.16:0/3041440440 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b48130340 con 0x7f1b48130540 2023-09-30T22:34:14.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.331+0000 7f1b46ffd700 1 --2- 172.21.15.16:0/3041440440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b48130540 0x7f1b48130910 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f1b380098f0 tx=0x7f1b38009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a8b340987df9cc7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:14.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.331+0000 7f1b45ffb700 1 -- 172.21.15.16:0/3041440440 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b38004230 con 0x7f1b48130540 2023-09-30T22:34:14.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.331+0000 7f1b45ffb700 1 -- 172.21.15.16:0/3041440440 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b38004390 con 0x7f1b48130540 2023-09-30T22:34:14.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.331+0000 7f1b45ffb700 1 -- 172.21.15.16:0/3041440440 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b380036a0 con 0x7f1b48130540 2023-09-30T22:34:14.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.332+0000 7f1b4d2a0700 1 -- 172.21.15.16:0/3041440440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b48130540 msgr2=0x7f1b48130910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:14.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.332+0000 7f1b4d2a0700 1 --2- 172.21.15.16:0/3041440440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b48130540 0x7f1b48130910 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f1b380098f0 tx=0x7f1b38009c00 comp rx=0 tx=0).stop 2023-09-30T22:34:14.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.332+0000 7f1b4d2a0700 1 -- 172.21.15.16:0/3041440440 shutdown_connections 2023-09-30T22:34:14.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.332+0000 7f1b4d2a0700 1 --2- 172.21.15.16:0/3041440440 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b48130540 0x7f1b48130910 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:14.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.332+0000 7f1b4d2a0700 1 --2- 172.21.15.16:0/3041440440 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1b48130e50 0x7f1b48135750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:14.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.332+0000 7f1b4d2a0700 1 -- 172.21.15.16:0/3041440440 >> 172.21.15.16:0/3041440440 conn(0x7f1b480a2aa0 msgr2=0x7f1b480a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:14.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.332+0000 7f6d7477f700 1 Processor -- start 2023-09-30T22:34:14.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.332+0000 7f6d7477f700 1 -- start start 2023-09-30T22:34:14.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.333+0000 7f6d7477f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d6c06a770 0x7f6d6c126d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:14.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.333+0000 7f6d7477f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d6c09dc30 0x7f6d6c127290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:14.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.333+0000 7f6d7477f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d6c13c1e0 0x7f6d6c1cc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:14.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.333+0000 7f6d7477f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6c13ad90 con 0x7f6d6c13c1e0 2023-09-30T22:34:14.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.333+0000 7f6d7477f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6c126850 con 0x7f6d6c06a770 2023-09-30T22:34:14.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.333+0000 7f6d7477f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6c129a10 con 0x7f6d6c09dc30 2023-09-30T22:34:14.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.333+0000 7f1b4d2a0700 1 -- 172.21.15.16:0/3041440440 shutdown_connections 2023-09-30T22:34:14.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.333+0000 7f1b4d2a0700 1 -- 172.21.15.16:0/3041440440 wait complete. 2023-09-30T22:34:14.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.333+0000 7f6d72d1c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d6c13c1e0 0x7f6d6c1cc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:14.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.333+0000 7f6d72d1c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d6c13c1e0 0x7f6d6c1cc3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33528/0 (socket says 172.21.15.16:33528) 2023-09-30T22:34:14.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.333+0000 7f6d72d1c700 1 -- 172.21.15.16:0/3247499746 learned_addr learned my addr 172.21.15.16:0/3247499746 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:14.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.333+0000 7f6d71d1a700 1 --2- 172.21.15.16:0/3247499746 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d6c09dc30 0x7f6d6c127290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:14.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.334+0000 7f6d72d1c700 1 -- 172.21.15.16:0/3247499746 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d6c09dc30 msgr2=0x7f6d6c127290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:14.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.334+0000 7f6d72d1c700 1 --2- 172.21.15.16:0/3247499746 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d6c09dc30 0x7f6d6c127290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:14.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.334+0000 7f6d72d1c700 1 -- 172.21.15.16:0/3247499746 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d6c06a770 msgr2=0x7f6d6c126d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:14.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.334+0000 7f6d72d1c700 1 --2- 172.21.15.16:0/3247499746 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d6c06a770 0x7f6d6c126d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:14.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.334+0000 7f6d72d1c700 1 -- 172.21.15.16:0/3247499746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d5c009550 con 0x7f6d6c13c1e0 2023-09-30T22:34:14.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.334+0000 7f6d72d1c700 1 --2- 172.21.15.16:0/3247499746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d6c13c1e0 0x7f6d6c1cc3e0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f6d6400efc0 tx=0x7f6d6400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:14.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.335+0000 7f6d62ffd700 1 -- 172.21.15.16:0/3247499746 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d64009ea0 con 0x7f6d6c13c1e0 2023-09-30T22:34:14.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.335+0000 7f6d62ffd700 1 -- 172.21.15.16:0/3247499746 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d6400f040 con 0x7f6d6c13c1e0 2023-09-30T22:34:14.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.335+0000 7f6d62ffd700 1 -- 172.21.15.16:0/3247499746 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d64014480 con 0x7f6d6c13c1e0 2023-09-30T22:34:14.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.335+0000 7f6d7477f700 1 -- 172.21.15.16:0/3247499746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d6c128390 con 0x7f6d6c13c1e0 2023-09-30T22:34:14.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.335+0000 7f6d7477f700 1 -- 172.21.15.16:0/3247499746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d6c1277d0 con 0x7f6d6c13c1e0 2023-09-30T22:34:14.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.335+0000 7f1b4d2a0700 1 Processor -- start 2023-09-30T22:34:14.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.336+0000 7f1b4d2a0700 1 -- start start 2023-09-30T22:34:14.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.336+0000 7f1b4d2a0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b48130540 0x7f1b4809c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:14.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.336+0000 7f1b4d2a0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1b48130e50 0x7f1b4809cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:14.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.336+0000 7f1b4d2a0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1b48135c90 0x7f1b4809d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:14.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.336+0000 7f1b4d2a0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4809a7b0 con 0x7f1b48130540 2023-09-30T22:34:14.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.336+0000 7f1b4d2a0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b48134ae0 con 0x7f1b48130e50 2023-09-30T22:34:14.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.336+0000 7f1b4d2a0700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4809d7a0 con 0x7f1b48135c90 2023-09-30T22:34:14.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.336+0000 7f1b46ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b48130540 0x7f1b4809c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:14.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.336+0000 7f1b46ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b48130540 0x7f1b4809c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33542/0 (socket says 172.21.15.16:33542) 2023-09-30T22:34:14.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.336+0000 7f1b46ffd700 1 -- 172.21.15.16:0/197553426 learned_addr learned my addr 172.21.15.16:0/197553426 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:14.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.336+0000 7f1b477fe700 1 --2- 172.21.15.16:0/197553426 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1b48135c90 0x7f1b4809d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:14.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.337+0000 7f1b467fc700 1 --2- 172.21.15.16:0/197553426 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1b48130e50 0x7f1b4809cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:14.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.337+0000 7f1b46ffd700 1 -- 172.21.15.16:0/197553426 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1b48135c90 msgr2=0x7f1b4809d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:14.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.337+0000 7f1b46ffd700 1 --2- 172.21.15.16:0/197553426 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1b48135c90 0x7f1b4809d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:14.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.337+0000 7f1b46ffd700 1 -- 172.21.15.16:0/197553426 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1b48130e50 msgr2=0x7f1b4809cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:14.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.337+0000 7f1b46ffd700 1 --2- 172.21.15.16:0/197553426 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1b48130e50 0x7f1b4809cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:14.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.337+0000 7f1b46ffd700 1 -- 172.21.15.16:0/197553426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b38009550 con 0x7f1b48130540 2023-09-30T22:34:14.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.337+0000 7f1b467fc700 1 --2- 172.21.15.16:0/197553426 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1b48130e50 0x7f1b4809cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:34:14.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.337+0000 7f1b46ffd700 1 --2- 172.21.15.16:0/197553426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b48130540 0x7f1b4809c7e0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f1b48070570 tx=0x7f1b38004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:14.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.337+0000 7f6d7477f700 1 -- 172.21.15.16:0/3247499746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d6c129850 con 0x7f6d6c13c1e0 2023-09-30T22:34:14.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.337+0000 7f1b2f7fe700 1 -- 172.21.15.16:0/197553426 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b38003b90 con 0x7f1b48130540 2023-09-30T22:34:14.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.337+0000 7f1b2f7fe700 1 -- 172.21.15.16:0/197553426 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b3801f070 con 0x7f1b48130540 2023-09-30T22:34:14.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.337+0000 7f1b4d2a0700 1 -- 172.21.15.16:0/197553426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b4813a0a0 con 0x7f1b48130540 2023-09-30T22:34:14.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.337+0000 7f1b2f7fe700 1 -- 172.21.15.16:0/197553426 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b38011630 con 0x7f1b48130540 2023-09-30T22:34:14.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.337+0000 7f1b4d2a0700 1 -- 172.21.15.16:0/197553426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b480718e0 con 0x7f1b48130540 2023-09-30T22:34:14.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.337+0000 7f6d62ffd700 1 -- 172.21.15.16:0/3247499746 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6d640145e0 con 0x7f6d6c13c1e0 2023-09-30T22:34:14.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.337+0000 7f6d62ffd700 1 -- 172.21.15.16:0/3247499746 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f6d64040080 con 0x7f6d6c13c1e0 2023-09-30T22:34:14.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.338+0000 7f1b4d2a0700 1 -- 172.21.15.16:0/197553426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b4809b4b0 con 0x7f1b48130540 2023-09-30T22:34:14.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.347+0000 7f6d62ffd700 1 -- 172.21.15.16:0/3247499746 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6d64071750 con 0x7f6d6c13c1e0 2023-09-30T22:34:14.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.347+0000 7f1b2f7fe700 1 -- 172.21.15.16:0/197553426 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1b38021070 con 0x7f1b48130540 2023-09-30T22:34:14.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.347+0000 7f1b2f7fe700 1 -- 172.21.15.16:0/197553426 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f1b38040d40 con 0x7f1b48130540 2023-09-30T22:34:14.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.356+0000 7f1b2f7fe700 1 -- 172.21.15.16:0/197553426 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1b3800b9d0 con 0x7f1b48130540 2023-09-30T22:34:14.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.501+0000 7fd1b1cc2700 1 -- 172.21.15.16:0/3608300931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd1ac074ea0 con 0x7fd1ac09f240 2023-09-30T22:34:14.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.502+0000 7fd193fff700 1 -- 172.21.15.16:0/3608300931 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd1a0019d20 con 0x7fd1ac09f240 2023-09-30T22:34:14.502 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:34:14.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.504+0000 7fd1b1cc2700 1 -- 172.21.15.16:0/3608300931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1ac09f240 msgr2=0x7fd1ac1a40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:14.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.504+0000 7fd1b1cc2700 1 --2- 172.21.15.16:0/3608300931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1ac09f240 0x7fd1ac1a40f0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fd1a000c550 tx=0x7fd1a000c910 comp rx=0 tx=0).stop 2023-09-30T22:34:14.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.504+0000 7fd1b1cc2700 1 -- 172.21.15.16:0/3608300931 shutdown_connections 2023-09-30T22:34:14.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.504+0000 7fd1b1cc2700 1 --2- 172.21.15.16:0/3608300931 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd1ac09e930 0x7fd1ac1a3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:14.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.504+0000 7fd1b1cc2700 1 --2- 172.21.15.16:0/3608300931 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd1ac09f650 0x7fd1ac1a4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:14.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.504+0000 7fd1b1cc2700 1 --2- 172.21.15.16:0/3608300931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1ac09f240 0x7fd1ac1a40f0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:14.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.504+0000 7fd1b1cc2700 1 -- 172.21.15.16:0/3608300931 >> 172.21.15.16:0/3608300931 conn(0x7fd1ac0a4210 msgr2=0x7fd1ac1b8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:14.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.504+0000 7fd1b1cc2700 1 -- 172.21.15.16:0/3608300931 shutdown_connections 2023-09-30T22:34:14.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.504+0000 7fd1b1cc2700 1 -- 172.21.15.16:0/3608300931 wait complete. 2023-09-30T22:34:14.525 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:34:14.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.529+0000 7f574f7e2700 1 -- 172.21.15.16:0/2091863949 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5748074ea0 con 0x7f574809d6f0 2023-09-30T22:34:14.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.529+0000 7f573dffb700 1 -- 172.21.15.16:0/2091863949 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5744017020 con 0x7f574809d6f0 2023-09-30T22:34:14.530 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:34:14.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.532+0000 7f574f7e2700 1 -- 172.21.15.16:0/2091863949 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f574809d6f0 msgr2=0x7f5748120820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:14.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.532+0000 7f574f7e2700 1 --2- 172.21.15.16:0/2091863949 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f574809d6f0 0x7f5748120820 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f574400c4e0 tx=0x7f574400c7f0 comp rx=0 tx=0).stop 2023-09-30T22:34:14.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.532+0000 7f574f7e2700 1 -- 172.21.15.16:0/2091863949 shutdown_connections 2023-09-30T22:34:14.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.532+0000 7f574f7e2700 1 --2- 172.21.15.16:0/2091863949 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f574809d6f0 0x7f5748120820 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:14.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.532+0000 7f574f7e2700 1 --2- 172.21.15.16:0/2091863949 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5748139170 0x7f57481212a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:14.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.532+0000 7f574f7e2700 1 --2- 172.21.15.16:0/2091863949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f574809e1d0 0x7f5748120d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:14.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.532+0000 7f574f7e2700 1 -- 172.21.15.16:0/2091863949 >> 172.21.15.16:0/2091863949 conn(0x7f5748066f50 msgr2=0x7f574809b410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:14.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.532+0000 7f574f7e2700 1 -- 172.21.15.16:0/2091863949 shutdown_connections 2023-09-30T22:34:14.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.532+0000 7f574f7e2700 1 -- 172.21.15.16:0/2091863949 wait complete. 2023-09-30T22:34:14.547 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:34:14.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.555+0000 7f6d7477f700 1 -- 172.21.15.16:0/3247499746 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f6d6c074ea0 con 0x7f6d6c13c1e0 2023-09-30T22:34:14.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.556+0000 7f6d62ffd700 1 -- 172.21.15.16:0/3247499746 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6d64018030 con 0x7f6d6c13c1e0 2023-09-30T22:34:14.557 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:34:14.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.558+0000 7f6d7477f700 1 -- 172.21.15.16:0/3247499746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d6c13c1e0 msgr2=0x7f6d6c1cc3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:14.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.558+0000 7f6d7477f700 1 --2- 172.21.15.16:0/3247499746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d6c13c1e0 0x7f6d6c1cc3e0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f6d6400efc0 tx=0x7f6d6400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:34:14.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.558+0000 7f6d7477f700 1 -- 172.21.15.16:0/3247499746 shutdown_connections 2023-09-30T22:34:14.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.558+0000 7f6d7477f700 1 --2- 172.21.15.16:0/3247499746 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d6c06a770 0x7f6d6c126d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:14.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.558+0000 7f6d7477f700 1 --2- 172.21.15.16:0/3247499746 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d6c13c1e0 0x7f6d6c1cc3e0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:14.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.558+0000 7f6d7477f700 1 --2- 172.21.15.16:0/3247499746 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d6c09dc30 0x7f6d6c127290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:14.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.558+0000 7f6d7477f700 1 -- 172.21.15.16:0/3247499746 >> 172.21.15.16:0/3247499746 conn(0x7f6d6c066f50 msgr2=0x7f6d6c13b400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:14.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.558+0000 7f6d7477f700 1 -- 172.21.15.16:0/3247499746 shutdown_connections 2023-09-30T22:34:14.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.558+0000 7f6d7477f700 1 -- 172.21.15.16:0/3247499746 wait complete. 2023-09-30T22:34:14.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.563+0000 7f1b4d2a0700 1 -- 172.21.15.16:0/197553426 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1b48074ea0 con 0x7f1b48130540 2023-09-30T22:34:14.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.563+0000 7f1b2f7fe700 1 -- 172.21.15.16:0/197553426 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1b38029020 con 0x7f1b48130540 2023-09-30T22:34:14.564 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:34:14.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.566+0000 7f1b4d2a0700 1 -- 172.21.15.16:0/197553426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b48130540 msgr2=0x7f1b4809c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:14.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.566+0000 7f1b4d2a0700 1 --2- 172.21.15.16:0/197553426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b48130540 0x7f1b4809c7e0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f1b48070570 tx=0x7f1b38004750 comp rx=0 tx=0).stop 2023-09-30T22:34:14.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.566+0000 7f1b4d2a0700 1 -- 172.21.15.16:0/197553426 shutdown_connections 2023-09-30T22:34:14.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.566+0000 7f1b4d2a0700 1 --2- 172.21.15.16:0/197553426 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1b48135c90 0x7f1b4809d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:14.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.566+0000 7f1b4d2a0700 1 --2- 172.21.15.16:0/197553426 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b48130540 0x7f1b4809c7e0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:14.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.566+0000 7f1b4d2a0700 1 --2- 172.21.15.16:0/197553426 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1b48130e50 0x7f1b4809cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:14.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.566+0000 7f1b4d2a0700 1 -- 172.21.15.16:0/197553426 >> 172.21.15.16:0/197553426 conn(0x7f1b480a2aa0 msgr2=0x7f1b4806e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:14.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.566+0000 7f1b4d2a0700 1 -- 172.21.15.16:0/197553426 shutdown_connections 2023-09-30T22:34:14.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:14.566+0000 7f1b4d2a0700 1 -- 172.21.15.16:0/197553426 wait complete. 2023-09-30T22:34:14.573 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:34:14.580 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:34:15.525 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:34:15.548 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:34:15.574 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:34:15.582 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:34:15.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.683+0000 7f326cf4c700 1 Processor -- start 2023-09-30T22:34:15.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.683+0000 7f326cf4c700 1 -- start start 2023-09-30T22:34:15.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.684+0000 7f326cf4c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3268133540 0x7f3268135920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:15.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.684+0000 7f326cf4c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3268135e60 0x7f3268138240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:15.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.684+0000 7f326cf4c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32680716a0 con 0x7f3268130540 2023-09-30T22:34:15.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.684+0000 7f326cf4c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3268071290 con 0x7f3268135e60 2023-09-30T22:34:15.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.684+0000 7f326cf4c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3268130340 con 0x7f3268133540 2023-09-30T22:34:15.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.684+0000 7f326659c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3268130540 0x7f3268130910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59840/0 (socket says 172.21.15.16:59840) 2023-09-30T22:34:15.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.684+0000 7f3266d9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3268135e60 0x7f3268138240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:15.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.684+0000 7f326659c700 1 -- 172.21.15.16:0/3873907449 learned_addr learned my addr 172.21.15.16:0/3873907449 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:34:15.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.685+0000 7f3265d9b700 1 --2- 172.21.15.16:0/3873907449 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3268133540 0x7f3268135920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:15.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.685+0000 7f3266d9d700 1 -- 172.21.15.16:0/3873907449 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3268133540 msgr2=0x7f3268135920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:15.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.685+0000 7f3266d9d700 1 --2- 172.21.15.16:0/3873907449 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3268133540 0x7f3268135920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:15.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.685+0000 7f3266d9d700 1 -- 172.21.15.16:0/3873907449 >> v1:172.21.15.16:6789/0 conn(0x7f3268130540 legacy=0x7f3268130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:15.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.685+0000 7f3266d9d700 1 -- 172.21.15.16:0/3873907449 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3268070b80 con 0x7f3268135e60 2023-09-30T22:34:15.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.685+0000 7f3266d9d700 1 --2- 172.21.15.16:0/3873907449 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3268135e60 0x7f3268138240 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f3260009dc0 tx=0x7f326000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36148dca26801529 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:15.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.685+0000 7f326559a700 1 -- 172.21.15.16:0/3873907449 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326000e070 con 0x7f3268135e60 2023-09-30T22:34:15.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.685+0000 7f326559a700 1 -- 172.21.15.16:0/3873907449 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f326000bb20 con 0x7f3268135e60 2023-09-30T22:34:15.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.685+0000 7f326559a700 1 -- 172.21.15.16:0/3873907449 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3260012430 con 0x7f3268135e60 2023-09-30T22:34:15.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.686+0000 7f326cf4c700 1 -- 172.21.15.16:0/3873907449 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3268135e60 msgr2=0x7f3268138240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:15.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.686+0000 7f326cf4c700 1 --2- 172.21.15.16:0/3873907449 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3268135e60 0x7f3268138240 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f3260009dc0 tx=0x7f326000b330 comp rx=0 tx=0).stop 2023-09-30T22:34:15.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.686+0000 7f326cf4c700 1 -- 172.21.15.16:0/3873907449 shutdown_connections 2023-09-30T22:34:15.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.686+0000 7f326cf4c700 1 --2- 172.21.15.16:0/3873907449 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3268135e60 0x7f3268138240 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:15.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.686+0000 7f326cf4c700 1 --2- 172.21.15.16:0/3873907449 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3268133540 0x7f3268135920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:15.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.686+0000 7f326cf4c700 1 -- 172.21.15.16:0/3873907449 >> 172.21.15.16:0/3873907449 conn(0x7f32680a2aa0 msgr2=0x7f32680a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:15.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.686+0000 7f326cf4c700 1 -- 172.21.15.16:0/3873907449 shutdown_connections 2023-09-30T22:34:15.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.686+0000 7f326cf4c700 1 -- 172.21.15.16:0/3873907449 wait complete. 2023-09-30T22:34:15.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.688+0000 7f326cf4c700 1 Processor -- start 2023-09-30T22:34:15.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.688+0000 7f326cf4c700 1 -- start start 2023-09-30T22:34:15.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.688+0000 7f326cf4c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3268130540 0x7f326809c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:15.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.688+0000 7f326cf4c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3268133540 0x7f326809cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:15.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.688+0000 7f326cf4c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3268135e60 0x7f326809d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:15.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.688+0000 7f326cf4c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32680718e0 con 0x7f3268133540 2023-09-30T22:34:15.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.688+0000 7f326cf4c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f326811a5f0 con 0x7f3268135e60 2023-09-30T22:34:15.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.688+0000 7f326cf4c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3268135ac0 con 0x7f3268130540 2023-09-30T22:34:15.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.688+0000 7f326659c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3268130540 0x7f326809c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:15.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.688+0000 7f326659c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3268130540 0x7f326809c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35326/0 (socket says 172.21.15.16:35326) 2023-09-30T22:34:15.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.688+0000 7f326659c700 1 -- 172.21.15.16:0/1575639289 learned_addr learned my addr 172.21.15.16:0/1575639289 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:15.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.688+0000 7f3265d9b700 1 --2- 172.21.15.16:0/1575639289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3268133540 0x7f326809cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:15.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.688+0000 7f3266d9d700 1 --2- 172.21.15.16:0/1575639289 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3268135e60 0x7f326809d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:15.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.689+0000 7f326659c700 1 -- 172.21.15.16:0/1575639289 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3268135e60 msgr2=0x7f326809d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:15.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.689+0000 7f326659c700 1 --2- 172.21.15.16:0/1575639289 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3268135e60 0x7f326809d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:15.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.689+0000 7f326659c700 1 -- 172.21.15.16:0/1575639289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3268133540 msgr2=0x7f326809cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:15.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.689+0000 7f326659c700 1 --2- 172.21.15.16:0/1575639289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3268133540 0x7f326809cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:15.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.689+0000 7f326659c700 1 -- 172.21.15.16:0/1575639289 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3260009a70 con 0x7f3268130540 2023-09-30T22:34:15.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.689+0000 7f326659c700 1 --2- 172.21.15.16:0/1575639289 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3268130540 0x7f326809c760 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f325800c480 tx=0x7f325800c790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:15.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.689+0000 7f3256ffd700 1 -- 172.21.15.16:0/1575639289 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3258005d70 con 0x7f3268130540 2023-09-30T22:34:15.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.689+0000 7f3256ffd700 1 -- 172.21.15.16:0/1575639289 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f325800ce90 con 0x7f3268130540 2023-09-30T22:34:15.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.689+0000 7f326cf4c700 1 -- 172.21.15.16:0/1575639289 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f326809e480 con 0x7f3268130540 2023-09-30T22:34:15.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.689+0000 7f326cf4c700 1 -- 172.21.15.16:0/1575639289 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f326811a030 con 0x7f3268130540 2023-09-30T22:34:15.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.689+0000 7f3256ffd700 1 -- 172.21.15.16:0/1575639289 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32580072d0 con 0x7f3268130540 2023-09-30T22:34:15.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.690+0000 7f3256ffd700 1 -- 172.21.15.16:0/1575639289 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3258017020 con 0x7f3268130540 2023-09-30T22:34:15.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.691+0000 7f3256ffd700 1 -- 172.21.15.16:0/1575639289 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f3258042a70 con 0x7f3268130540 2023-09-30T22:34:15.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.691+0000 7f326cf4c700 1 -- 172.21.15.16:0/1575639289 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f326811a210 con 0x7f3268130540 2023-09-30T22:34:15.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.700+0000 7f3256ffd700 1 -- 172.21.15.16:0/1575639289 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f325801f070 con 0x7f3268130540 2023-09-30T22:34:15.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.724+0000 7fcb57850700 1 Processor -- start 2023-09-30T22:34:15.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.724+0000 7fcb57850700 1 -- start start 2023-09-30T22:34:15.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.725+0000 7fcb57850700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcb5009d6f0 0x7fcb5009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:15.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.725+0000 7fcb57850700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcb5009e410 0x7fcb50134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:15.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.725+0000 7fcb57850700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb5009b120 con 0x7fcb5009e000 2023-09-30T22:34:15.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.725+0000 7fcb57850700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb5006b460 con 0x7fcb5009e410 2023-09-30T22:34:15.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.725+0000 7fcb57850700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5009a7e0 con 0x7fcb5009d6f0 2023-09-30T22:34:15.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.725+0000 7fcb555ec700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcb5009d6f0 0x7fcb5009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:15.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.725+0000 7fcb555ec700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcb5009d6f0 0x7fcb5009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:53056/0 (socket says 172.21.15.16:53056) 2023-09-30T22:34:15.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.725+0000 7fcb555ec700 1 -- 172.21.15.16:0/3323420667 learned_addr learned my addr 172.21.15.16:0/3323420667 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:15.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.725+0000 7fcb54deb700 1 --1- 172.21.15.16:0/3323420667 >> v1:172.21.15.16:6789/0 conn(0x7fcb5009e000 0x7fcb501339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59852/0 (socket says 172.21.15.16:59852) 2023-09-30T22:34:15.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.725+0000 7fcb55ded700 1 --2- 172.21.15.16:0/3323420667 >> v2:172.21.15.16:3300/0 conn(0x7fcb5009e410 0x7fcb50134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:15.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.726+0000 7fcb555ec700 1 -- 172.21.15.16:0/3323420667 >> v2:172.21.15.16:3300/0 conn(0x7fcb5009e410 msgr2=0x7fcb50134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:15.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.726+0000 7fcb555ec700 1 --2- 172.21.15.16:0/3323420667 >> v2:172.21.15.16:3300/0 conn(0x7fcb5009e410 0x7fcb50134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:15.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.726+0000 7fcb555ec700 1 -- 172.21.15.16:0/3323420667 >> v1:172.21.15.16:6789/0 conn(0x7fcb5009e000 legacy=0x7fcb501339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:15.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.726+0000 7fcb555ec700 1 -- 172.21.15.16:0/3323420667 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb5009a500 con 0x7fcb5009d6f0 2023-09-30T22:34:15.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.726+0000 7fcb555ec700 1 --2- 172.21.15.16:0/3323420667 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcb5009d6f0 0x7fcb5009dac0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c009920 tx=0x7fcb4c009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ae0909bceeb1552 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:15.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.726+0000 7fcb47fff700 1 -- 172.21.15.16:0/3323420667 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c011830 con 0x7fcb5009d6f0 2023-09-30T22:34:15.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.726+0000 7fcb47fff700 1 -- 172.21.15.16:0/3323420667 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb4c011990 con 0x7fcb5009d6f0 2023-09-30T22:34:15.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.726+0000 7fcb47fff700 1 -- 172.21.15.16:0/3323420667 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c011d20 con 0x7fcb5009d6f0 2023-09-30T22:34:15.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.727+0000 7fcb57850700 1 -- 172.21.15.16:0/3323420667 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcb5009d6f0 msgr2=0x7fcb5009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:15.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.727+0000 7fcb57850700 1 --2- 172.21.15.16:0/3323420667 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcb5009d6f0 0x7fcb5009dac0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c009920 tx=0x7fcb4c009c30 comp rx=0 tx=0).stop 2023-09-30T22:34:15.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.727+0000 7fcb57850700 1 -- 172.21.15.16:0/3323420667 shutdown_connections 2023-09-30T22:34:15.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.727+0000 7fcb57850700 1 --2- 172.21.15.16:0/3323420667 >> v2:172.21.15.16:3300/0 conn(0x7fcb5009e410 0x7fcb50134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:15.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.727+0000 7fcb57850700 1 --2- 172.21.15.16:0/3323420667 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcb5009d6f0 0x7fcb5009dac0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:15.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.727+0000 7fcb57850700 1 -- 172.21.15.16:0/3323420667 >> 172.21.15.16:0/3323420667 conn(0x7fcb50066f50 msgr2=0x7fcb5009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:15.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.727+0000 7fcb57850700 1 -- 172.21.15.16:0/3323420667 shutdown_connections 2023-09-30T22:34:15.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.727+0000 7fcb57850700 1 -- 172.21.15.16:0/3323420667 wait complete. 2023-09-30T22:34:15.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.728+0000 7fcb57850700 1 Processor -- start 2023-09-30T22:34:15.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.728+0000 7fcb57850700 1 -- start start 2023-09-30T22:34:15.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.729+0000 7fcb57850700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcb5009d6f0 0x7fcb501a8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:15.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.729+0000 7fcb57850700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb5009e000 0x7fcb501a85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:15.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.729+0000 7fcb57850700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcb5009e410 0x7fcb501a8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:15.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.729+0000 7fcb57850700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb501ab880 con 0x7fcb5009e000 2023-09-30T22:34:15.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.729+0000 7fcb57850700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5009c710 con 0x7fcb5009e410 2023-09-30T22:34:15.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.729+0000 7fcb57850700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcb501a9ba0 con 0x7fcb5009d6f0 2023-09-30T22:34:15.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.729+0000 7fcb55ded700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcb5009e410 0x7fcb501a8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:15.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.729+0000 7fcb55ded700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcb5009e410 0x7fcb501a8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:53064/0 (socket says 172.21.15.16:53064) 2023-09-30T22:34:15.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.729+0000 7fcb55ded700 1 -- 172.21.15.16:0/3460350495 learned_addr learned my addr 172.21.15.16:0/3460350495 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:15.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.729+0000 7fcb555ec700 1 --2- 172.21.15.16:0/3460350495 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcb5009d6f0 0x7fcb501a8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:15.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.729+0000 7fcb54deb700 1 --2- 172.21.15.16:0/3460350495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb5009e000 0x7fcb501a85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:15.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.729+0000 7fcb55ded700 1 -- 172.21.15.16:0/3460350495 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcb5009d6f0 msgr2=0x7fcb501a8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:15.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.729+0000 7fcb55ded700 1 --2- 172.21.15.16:0/3460350495 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcb5009d6f0 0x7fcb501a8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:15.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.729+0000 7fcb55ded700 1 -- 172.21.15.16:0/3460350495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb5009e000 msgr2=0x7fcb501a85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:15.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.729+0000 7fcb55ded700 1 --2- 172.21.15.16:0/3460350495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb5009e000 0x7fcb501a85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:15.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.729+0000 7fcb55ded700 1 -- 172.21.15.16:0/3460350495 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb4c009550 con 0x7fcb5009e410 2023-09-30T22:34:15.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.729+0000 7fcb54deb700 1 --2- 172.21.15.16:0/3460350495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb5009e000 0x7fcb501a85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:34:15.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.729+0000 7fcb55ded700 1 --2- 172.21.15.16:0/3460350495 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcb5009e410 0x7fcb501a8af0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fcb48009dd0 tx=0x7fcb4800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:15.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.730+0000 7fcb45ffb700 1 -- 172.21.15.16:0/3460350495 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb48009c40 con 0x7fcb5009e410 2023-09-30T22:34:15.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.730+0000 7fcb45ffb700 1 -- 172.21.15.16:0/3460350495 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb4800f040 con 0x7fcb5009e410 2023-09-30T22:34:15.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.730+0000 7fcb45ffb700 1 -- 172.21.15.16:0/3460350495 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb48014480 con 0x7fcb5009e410 2023-09-30T22:34:15.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.730+0000 7fcb57850700 1 -- 172.21.15.16:0/3460350495 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb501abae0 con 0x7fcb5009e410 2023-09-30T22:34:15.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.730+0000 7fcb57850700 1 -- 172.21.15.16:0/3460350495 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb501aa290 con 0x7fcb5009e410 2023-09-30T22:34:15.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.731+0000 7fcb45ffb700 1 -- 172.21.15.16:0/3460350495 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcb480145e0 con 0x7fcb5009e410 2023-09-30T22:34:15.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.731+0000 7fcb45ffb700 1 -- 172.21.15.16:0/3460350495 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fcb4803ff00 con 0x7fcb5009e410 2023-09-30T22:34:15.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.732+0000 7fcb57850700 1 -- 172.21.15.16:0/3460350495 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb50074ea0 con 0x7fcb5009e410 2023-09-30T22:34:15.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.741+0000 7fcb45ffb700 1 -- 172.21.15.16:0/3460350495 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcb48018020 con 0x7fcb5009e410 2023-09-30T22:34:15.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.765+0000 7f3517293700 1 Processor -- start 2023-09-30T22:34:15.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.765+0000 7f3517293700 1 -- start start 2023-09-30T22:34:15.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.766+0000 7f3517293700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f35100a0a30 0x7f35100a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:15.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.766+0000 7f3517293700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f351009d630 0x7f35100a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:15.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.766+0000 7f3517293700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3510133930 0x7f3510133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:15.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.766+0000 7f3517293700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35100a2a70 con 0x7f351009d630 2023-09-30T22:34:15.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.766+0000 7f3517293700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35100a01d0 con 0x7f35100a0a30 2023-09-30T22:34:15.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.766+0000 7f3517293700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f351009e2b0 con 0x7f3510133930 2023-09-30T22:34:15.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.766+0000 7f3515830700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3510133930 0x7f3510133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:15.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.766+0000 7f3515830700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3510133930 0x7f3510133dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35340/0 (socket says 172.21.15.16:35340) 2023-09-30T22:34:15.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.766+0000 7f3515830700 1 -- 172.21.15.16:0/820786298 learned_addr learned my addr 172.21.15.16:0/820786298 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:15.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.766+0000 7f351502f700 1 --2- 172.21.15.16:0/820786298 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f35100a0a30 0x7f35100a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:15.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.766+0000 7f351482e700 1 --2- 172.21.15.16:0/820786298 >> v2:172.21.15.16:3300/0 conn(0x7f351009d630 0x7f35100a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:15.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.766+0000 7f3515830700 1 -- 172.21.15.16:0/820786298 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f35100a0a30 msgr2=0x7f35100a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:15.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.766+0000 7f3515830700 1 --2- 172.21.15.16:0/820786298 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f35100a0a30 0x7f35100a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:15.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.766+0000 7f3515830700 1 -- 172.21.15.16:0/820786298 >> v2:172.21.15.16:3300/0 conn(0x7f351009d630 msgr2=0x7f35100a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:15.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.766+0000 7f3515830700 1 --2- 172.21.15.16:0/820786298 >> v2:172.21.15.16:3300/0 conn(0x7f351009d630 0x7f35100a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:15.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.766+0000 7f3515830700 1 -- 172.21.15.16:0/820786298 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f351009dc50 con 0x7f3510133930 2023-09-30T22:34:15.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.767+0000 7f3515830700 1 --2- 172.21.15.16:0/820786298 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3510133930 0x7f3510133dd0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f3508009e50 tx=0x7f350800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9146febb52e1f543 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:15.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.767+0000 7f3507fff700 1 -- 172.21.15.16:0/820786298 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350800e070 con 0x7f3510133930 2023-09-30T22:34:15.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.767+0000 7f3507fff700 1 -- 172.21.15.16:0/820786298 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f350800b910 con 0x7f3510133930 2023-09-30T22:34:15.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.767+0000 7f3507fff700 1 -- 172.21.15.16:0/820786298 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350800bcc0 con 0x7f3510133930 2023-09-30T22:34:15.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.768+0000 7f3517293700 1 -- 172.21.15.16:0/820786298 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3510133930 msgr2=0x7f3510133dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:15.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.768+0000 7f3517293700 1 --2- 172.21.15.16:0/820786298 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3510133930 0x7f3510133dd0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f3508009e50 tx=0x7f350800b330 comp rx=0 tx=0).stop 2023-09-30T22:34:15.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.768+0000 7f3517293700 1 -- 172.21.15.16:0/820786298 shutdown_connections 2023-09-30T22:34:15.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.768+0000 7f3517293700 1 --2- 172.21.15.16:0/820786298 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f35100a0a30 0x7f35100a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:15.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.768+0000 7f3517293700 1 --2- 172.21.15.16:0/820786298 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3510133930 0x7f3510133dd0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:15.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.768+0000 7f3517293700 1 --2- 172.21.15.16:0/820786298 >> v2:172.21.15.16:3300/0 conn(0x7f351009d630 0x7f35100a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:15.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.768+0000 7f3517293700 1 -- 172.21.15.16:0/820786298 >> 172.21.15.16:0/820786298 conn(0x7f35100a3050 msgr2=0x7f35100a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:15.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.768+0000 7f6249a11700 1 Processor -- start 2023-09-30T22:34:15.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.768+0000 7f3517293700 1 -- 172.21.15.16:0/820786298 shutdown_connections 2023-09-30T22:34:15.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.768+0000 7f6249a11700 1 -- start start 2023-09-30T22:34:15.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.769+0000 7f3517293700 1 -- 172.21.15.16:0/820786298 wait complete. 2023-09-30T22:34:15.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.769+0000 7f6249a11700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6244130540 0x7f6244130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:15.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.769+0000 7f6249a11700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f624406fcf0 0x7f6244138570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:15.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.769+0000 7f6249a11700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62440716a0 con 0x7f6244130e50 2023-09-30T22:34:15.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.769+0000 7f6249a11700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f624409a390 con 0x7f624406fcf0 2023-09-30T22:34:15.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.769+0000 7f6249a11700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6244071290 con 0x7f6244130540 2023-09-30T22:34:15.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.769+0000 7f6242ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6244130540 0x7f6244130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:15.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.769+0000 7f6242ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6244130540 0x7f6244130910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35346/0 (socket says 172.21.15.16:35346) 2023-09-30T22:34:15.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.769+0000 7f6242ffd700 1 -- 172.21.15.16:0/537633681 learned_addr learned my addr 172.21.15.16:0/537633681 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:15.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.770+0000 7f6242ffd700 1 -- 172.21.15.16:0/537633681 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f624406fcf0 msgr2=0x7f6244138570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:15.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.770+0000 7f6242ffd700 1 --2- 172.21.15.16:0/537633681 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f624406fcf0 0x7f6244138570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:15.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.770+0000 7f6242ffd700 1 -- 172.21.15.16:0/537633681 >> v1:172.21.15.16:6789/0 conn(0x7f6244130e50 legacy=0x7f624406f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:15.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.770+0000 7f6242ffd700 1 -- 172.21.15.16:0/537633681 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6244130340 con 0x7f6244130540 2023-09-30T22:34:15.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.770+0000 7f6241ffb700 1 -- 172.21.15.16:0/537633681 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3066064956 0 0) 0x7f62440716a0 con 0x7f6244130e50 2023-09-30T22:34:15.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.770+0000 7f6242ffd700 1 --2- 172.21.15.16:0/537633681 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6244130540 0x7f6244130910 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f6234009920 tx=0x7f6234009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45faa4517c8a8270 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:15.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.770+0000 7f3517293700 1 Processor -- start 2023-09-30T22:34:15.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.770+0000 7f6241ffb700 1 -- 172.21.15.16:0/537633681 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623400ba60 con 0x7f6244130540 2023-09-30T22:34:15.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.770+0000 7f6241ffb700 1 -- 172.21.15.16:0/537633681 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f623400bbc0 con 0x7f6244130540 2023-09-30T22:34:15.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.770+0000 7f6241ffb700 1 -- 172.21.15.16:0/537633681 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6234009e70 con 0x7f6244130540 2023-09-30T22:34:15.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.770+0000 7f3517293700 1 -- start start 2023-09-30T22:34:15.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.771+0000 7f3517293700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f351009d630 0x7f351011f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:15.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.771+0000 7f3517293700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f35100a0a30 0x7f351011f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:15.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.771+0000 7f3517293700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3510133930 0x7f35101c7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:15.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.771+0000 7f3517293700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3510122230 con 0x7f351009d630 2023-09-30T22:34:15.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.771+0000 7f3517293700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3510122410 con 0x7f35100a0a30 2023-09-30T22:34:15.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.771+0000 7f3517293700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3510133280 con 0x7f3510133930 2023-09-30T22:34:15.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.771+0000 7f351502f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f351009d630 0x7f351011f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:15.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.771+0000 7f3515830700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3510133930 0x7f35101c7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:15.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.771+0000 7f351502f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f351009d630 0x7f351011f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33574/0 (socket says 172.21.15.16:33574) 2023-09-30T22:34:15.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.771+0000 7f351502f700 1 -- 172.21.15.16:0/4003599518 learned_addr learned my addr 172.21.15.16:0/4003599518 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:15.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.771+0000 7f6249a11700 1 -- 172.21.15.16:0/537633681 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6244130540 msgr2=0x7f6244130910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:15.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.771+0000 7f6249a11700 1 --2- 172.21.15.16:0/537633681 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6244130540 0x7f6244130910 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f6234009920 tx=0x7f6234009c30 comp rx=0 tx=0).stop 2023-09-30T22:34:15.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.771+0000 7f6249a11700 1 -- 172.21.15.16:0/537633681 shutdown_connections 2023-09-30T22:34:15.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.771+0000 7f6249a11700 1 --2- 172.21.15.16:0/537633681 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f624406fcf0 0x7f6244138570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:15.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.771+0000 7f6249a11700 1 --2- 172.21.15.16:0/537633681 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6244130540 0x7f6244130910 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:15.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.771+0000 7f6249a11700 1 -- 172.21.15.16:0/537633681 >> 172.21.15.16:0/537633681 conn(0x7f62440a2aa0 msgr2=0x7f62440a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:15.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.771+0000 7f351502f700 1 -- 172.21.15.16:0/4003599518 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3510133930 msgr2=0x7f35101c7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:15.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.771+0000 7f351502f700 1 --2- 172.21.15.16:0/4003599518 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3510133930 0x7f35101c7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:15.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.771+0000 7f351502f700 1 -- 172.21.15.16:0/4003599518 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f35100a0a30 msgr2=0x7f351011f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:15.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.771+0000 7f351502f700 1 --2- 172.21.15.16:0/4003599518 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f35100a0a30 0x7f351011f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:15.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.771+0000 7f351502f700 1 -- 172.21.15.16:0/4003599518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3508009b30 con 0x7f351009d630 2023-09-30T22:34:15.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.771+0000 7f3515830700 1 --2- 172.21.15.16:0/4003599518 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3510133930 0x7f35101c7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:15.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.771+0000 7f6249a11700 1 -- 172.21.15.16:0/537633681 shutdown_connections 2023-09-30T22:34:15.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.772+0000 7f6249a11700 1 -- 172.21.15.16:0/537633681 wait complete. 2023-09-30T22:34:15.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.772+0000 7f351502f700 1 --2- 172.21.15.16:0/4003599518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f351009d630 0x7f351011f3f0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f350c00e9e0 tx=0x7f350c00ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:15.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.772+0000 7f3505ffb700 1 -- 172.21.15.16:0/4003599518 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350c009710 con 0x7f351009d630 2023-09-30T22:34:15.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.772+0000 7f3505ffb700 1 -- 172.21.15.16:0/4003599518 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f350c00c850 con 0x7f351009d630 2023-09-30T22:34:15.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.772+0000 7f3517293700 1 -- 172.21.15.16:0/4003599518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f351011fe70 con 0x7f351009d630 2023-09-30T22:34:15.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.772+0000 7f3505ffb700 1 -- 172.21.15.16:0/4003599518 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350c00cbc0 con 0x7f351009d630 2023-09-30T22:34:15.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.772+0000 7f3517293700 1 -- 172.21.15.16:0/4003599518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3510120cf0 con 0x7f351009d630 2023-09-30T22:34:15.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.773+0000 7f3505ffb700 1 -- 172.21.15.16:0/4003599518 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f350c00cd20 con 0x7f351009d630 2023-09-30T22:34:15.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.773+0000 7f3505ffb700 1 -- 172.21.15.16:0/4003599518 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f350c03f800 con 0x7f351009d630 2023-09-30T22:34:15.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.773+0000 7f6249a11700 1 Processor -- start 2023-09-30T22:34:15.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.773+0000 7f6249a11700 1 -- start start 2023-09-30T22:34:15.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.774+0000 7f3517293700 1 -- 172.21.15.16:0/4003599518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35100747f0 con 0x7f351009d630 2023-09-30T22:34:15.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.774+0000 7f6249a11700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f624406fcf0 0x7f624409b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:15.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.774+0000 7f6249a11700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6244130540 0x7f624409bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:15.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.774+0000 7f6249a11700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6244130e50 0x7f624409c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:15.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.774+0000 7f6249a11700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62440a0a50 con 0x7f6244130e50 2023-09-30T22:34:15.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.774+0000 7f6249a11700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62440718e0 con 0x7f6244130540 2023-09-30T22:34:15.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.774+0000 7f6249a11700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f624411dc90 con 0x7f624406fcf0 2023-09-30T22:34:15.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.774+0000 7f6242ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f624406fcf0 0x7f624409b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:15.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.774+0000 7f6242ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f624406fcf0 0x7f624409b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35360/0 (socket says 172.21.15.16:35360) 2023-09-30T22:34:15.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.774+0000 7f6242ffd700 1 -- 172.21.15.16:0/2670816973 learned_addr learned my addr 172.21.15.16:0/2670816973 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:15.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.774+0000 7f62427fc700 1 --2- 172.21.15.16:0/2670816973 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6244130540 0x7f624409bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:15.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.774+0000 7f62437fe700 1 --2- 172.21.15.16:0/2670816973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6244130e50 0x7f624409c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:15.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.775+0000 7f62427fc700 1 -- 172.21.15.16:0/2670816973 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f624406fcf0 msgr2=0x7f624409b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:15.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.775+0000 7f62427fc700 1 --2- 172.21.15.16:0/2670816973 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f624406fcf0 0x7f624409b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:15.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.775+0000 7f62427fc700 1 -- 172.21.15.16:0/2670816973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6244130e50 msgr2=0x7f624409c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:15.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.775+0000 7f62427fc700 1 --2- 172.21.15.16:0/2670816973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6244130e50 0x7f624409c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:15.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.775+0000 7f62427fc700 1 -- 172.21.15.16:0/2670816973 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6234009550 con 0x7f6244130540 2023-09-30T22:34:15.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.775+0000 7f62427fc700 1 --2- 172.21.15.16:0/2670816973 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6244130540 0x7f624409bd60 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f62440a35d0 tx=0x7f623800c710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:15.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.778+0000 7f622bfff700 1 -- 172.21.15.16:0/2670816973 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62380054e0 con 0x7f6244130540 2023-09-30T22:34:15.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.778+0000 7f622bfff700 1 -- 172.21.15.16:0/2670816973 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6238003680 con 0x7f6244130540 2023-09-30T22:34:15.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.778+0000 7f622bfff700 1 -- 172.21.15.16:0/2670816973 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6238003a30 con 0x7f6244130540 2023-09-30T22:34:15.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.778+0000 7f6249a11700 1 -- 172.21.15.16:0/2670816973 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f624409d1a0 con 0x7f6244130540 2023-09-30T22:34:15.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.778+0000 7f6249a11700 1 -- 172.21.15.16:0/2670816973 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f624411d580 con 0x7f6244130540 2023-09-30T22:34:15.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.780+0000 7f3505ffb700 1 -- 172.21.15.16:0/4003599518 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f350c01b9c0 con 0x7f351009d630 2023-09-30T22:34:15.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.780+0000 7f622bfff700 1 -- 172.21.15.16:0/2670816973 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6238003b90 con 0x7f6244130540 2023-09-30T22:34:15.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.780+0000 7f6249a11700 1 -- 172.21.15.16:0/2670816973 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6244074ea0 con 0x7f6244130540 2023-09-30T22:34:15.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.780+0000 7f622bfff700 1 -- 172.21.15.16:0/2670816973 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f6238019910 con 0x7f6244130540 2023-09-30T22:34:15.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.790+0000 7f622bfff700 1 -- 172.21.15.16:0/2670816973 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6238017030 con 0x7f6244130540 2023-09-30T22:34:15.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.926+0000 7f326cf4c700 1 -- 172.21.15.16:0/1575639289 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3268074ea0 con 0x7f3268130540 2023-09-30T22:34:15.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.927+0000 7f3256ffd700 1 -- 172.21.15.16:0/1575639289 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3258028090 con 0x7f3268130540 2023-09-30T22:34:15.927 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:34:15.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.929+0000 7f326cf4c700 1 -- 172.21.15.16:0/1575639289 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3268130540 msgr2=0x7f326809c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:15.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.929+0000 7f326cf4c700 1 --2- 172.21.15.16:0/1575639289 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3268130540 0x7f326809c760 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f325800c480 tx=0x7f325800c790 comp rx=0 tx=0).stop 2023-09-30T22:34:15.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.929+0000 7f326cf4c700 1 -- 172.21.15.16:0/1575639289 shutdown_connections 2023-09-30T22:34:15.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.929+0000 7f326cf4c700 1 --2- 172.21.15.16:0/1575639289 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3268135e60 0x7f326809d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:15.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.929+0000 7f326cf4c700 1 --2- 172.21.15.16:0/1575639289 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3268130540 0x7f326809c760 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:15.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.929+0000 7f326cf4c700 1 --2- 172.21.15.16:0/1575639289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3268133540 0x7f326809cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:15.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.929+0000 7f326cf4c700 1 -- 172.21.15.16:0/1575639289 >> 172.21.15.16:0/1575639289 conn(0x7f32680a2aa0 msgr2=0x7f3268131320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:15.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.930+0000 7f326cf4c700 1 -- 172.21.15.16:0/1575639289 shutdown_connections 2023-09-30T22:34:15.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.930+0000 7f326cf4c700 1 -- 172.21.15.16:0/1575639289 wait complete. 2023-09-30T22:34:15.944 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:34:15.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.953+0000 7fcb57850700 1 -- 172.21.15.16:0/3460350495 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fcb5006c360 con 0x7fcb5009e410 2023-09-30T22:34:15.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.953+0000 7fcb45ffb700 1 -- 172.21.15.16:0/3460350495 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcb480114a0 con 0x7fcb5009e410 2023-09-30T22:34:15.954 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:34:15.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.956+0000 7fcb57850700 1 -- 172.21.15.16:0/3460350495 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcb5009e410 msgr2=0x7fcb501a8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:15.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.956+0000 7fcb57850700 1 --2- 172.21.15.16:0/3460350495 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcb5009e410 0x7fcb501a8af0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fcb48009dd0 tx=0x7fcb4800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:34:15.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.956+0000 7fcb57850700 1 -- 172.21.15.16:0/3460350495 shutdown_connections 2023-09-30T22:34:15.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.956+0000 7fcb57850700 1 --2- 172.21.15.16:0/3460350495 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcb5009e410 0x7fcb501a8af0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:15.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.956+0000 7fcb57850700 1 --2- 172.21.15.16:0/3460350495 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcb5009d6f0 0x7fcb501a8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:15.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.956+0000 7fcb57850700 1 --2- 172.21.15.16:0/3460350495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcb5009e000 0x7fcb501a85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:15.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.956+0000 7fcb57850700 1 -- 172.21.15.16:0/3460350495 >> 172.21.15.16:0/3460350495 conn(0x7fcb50066f50 msgr2=0x7fcb5006f470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:15.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.956+0000 7fcb57850700 1 -- 172.21.15.16:0/3460350495 shutdown_connections 2023-09-30T22:34:15.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.956+0000 7fcb57850700 1 -- 172.21.15.16:0/3460350495 wait complete. 2023-09-30T22:34:15.970 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:34:15.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.990+0000 7f3517293700 1 -- 172.21.15.16:0/4003599518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3510072980 con 0x7f351009d630 2023-09-30T22:34:15.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.990+0000 7f3505ffb700 1 -- 172.21.15.16:0/4003599518 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f350c021030 con 0x7f351009d630 2023-09-30T22:34:15.991 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:34:15.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.992+0000 7f3517293700 1 -- 172.21.15.16:0/4003599518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f351009d630 msgr2=0x7f351011f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:15.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.992+0000 7f3517293700 1 --2- 172.21.15.16:0/4003599518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f351009d630 0x7f351011f3f0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f350c00e9e0 tx=0x7f350c00ecf0 comp rx=0 tx=0).stop 2023-09-30T22:34:15.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.992+0000 7f3517293700 1 -- 172.21.15.16:0/4003599518 shutdown_connections 2023-09-30T22:34:15.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.992+0000 7f3517293700 1 --2- 172.21.15.16:0/4003599518 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3510133930 0x7f35101c7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:15.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.992+0000 7f3517293700 1 --2- 172.21.15.16:0/4003599518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f351009d630 0x7f351011f3f0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:15.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.992+0000 7f3517293700 1 --2- 172.21.15.16:0/4003599518 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f35100a0a30 0x7f351011f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:15.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.992+0000 7f3517293700 1 -- 172.21.15.16:0/4003599518 >> 172.21.15.16:0/4003599518 conn(0x7f35100a3050 msgr2=0x7f351009ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:15.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.993+0000 7f3517293700 1 -- 172.21.15.16:0/4003599518 shutdown_connections 2023-09-30T22:34:15.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.993+0000 7f3517293700 1 -- 172.21.15.16:0/4003599518 wait complete. 2023-09-30T22:34:15.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.999+0000 7f6249a11700 1 -- 172.21.15.16:0/2670816973 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f624406b9e0 con 0x7f6244130540 2023-09-30T22:34:16.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:15.999+0000 7f622bfff700 1 -- 172.21.15.16:0/2670816973 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6238003e40 con 0x7f6244130540 2023-09-30T22:34:16.000 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:34:16.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:16.002+0000 7f6249a11700 1 -- 172.21.15.16:0/2670816973 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6244130540 msgr2=0x7f624409bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:16.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:16.002+0000 7f6249a11700 1 --2- 172.21.15.16:0/2670816973 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6244130540 0x7f624409bd60 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f62440a35d0 tx=0x7f623800c710 comp rx=0 tx=0).stop 2023-09-30T22:34:16.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:16.002+0000 7f6249a11700 1 -- 172.21.15.16:0/2670816973 shutdown_connections 2023-09-30T22:34:16.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:16.002+0000 7f6249a11700 1 --2- 172.21.15.16:0/2670816973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6244130e50 0x7f624409c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:16.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:16.002+0000 7f6249a11700 1 --2- 172.21.15.16:0/2670816973 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f624406fcf0 0x7f624409b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:16.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:16.002+0000 7f6249a11700 1 --2- 172.21.15.16:0/2670816973 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6244130540 0x7f624409bd60 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:16.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:16.002+0000 7f6249a11700 1 -- 172.21.15.16:0/2670816973 >> 172.21.15.16:0/2670816973 conn(0x7f62440a2aa0 msgr2=0x7f62441b6240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:16.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:16.002+0000 7f6249a11700 1 -- 172.21.15.16:0/2670816973 shutdown_connections 2023-09-30T22:34:16.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:16.002+0000 7f6249a11700 1 -- 172.21.15.16:0/2670816973 wait complete. 2023-09-30T22:34:16.007 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:34:16.016 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:34:16.945 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:34:16.970 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:34:17.008 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:34:17.017 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:34:17.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.111+0000 7f3ea7c08700 1 Processor -- start 2023-09-30T22:34:17.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.111+0000 7f3ea7c08700 1 -- start start 2023-09-30T22:34:17.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.112+0000 7f3ea7c08700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3ea006a770 0x7f3ea006ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:17.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.112+0000 7f3ea7c08700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3ea006b080 0x7f3ea0131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:17.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.112+0000 7f3ea7c08700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3ea0131b40 0x7f3ea0131fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:17.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.112+0000 7f3ea7c08700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ea009ece0 con 0x7f3ea006b080 2023-09-30T22:34:17.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.112+0000 7f3ea7c08700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea009e760 con 0x7f3ea006a770 2023-09-30T22:34:17.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.112+0000 7f3ea7c08700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea009e040 con 0x7f3ea0131b40 2023-09-30T22:34:17.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.112+0000 7f3ea59a4700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3ea006a770 0x7f3ea006ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:17.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.112+0000 7f3ea61a5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3ea0131b40 0x7f3ea0131fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:17.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.112+0000 7f3ea51a3700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3ea006b080 0x7f3ea0131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:17.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.112+0000 7f3ea59a4700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3ea006a770 0x7f3ea006ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:53106/0 (socket says 172.21.15.16:53106) 2023-09-30T22:34:17.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.112+0000 7f3ea59a4700 1 -- 172.21.15.16:0/3135465355 learned_addr learned my addr 172.21.15.16:0/3135465355 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:17.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.112+0000 7f3ea61a5700 1 -- 172.21.15.16:0/3135465355 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3ea006a770 msgr2=0x7f3ea006ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:17.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.112+0000 7f3ea61a5700 1 --2- 172.21.15.16:0/3135465355 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3ea006a770 0x7f3ea006ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:17.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.112+0000 7f3ea61a5700 1 -- 172.21.15.16:0/3135465355 >> v2:172.21.15.16:3300/0 conn(0x7f3ea006b080 msgr2=0x7f3ea0131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:17.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.113+0000 7f3ea61a5700 1 --2- 172.21.15.16:0/3135465355 >> v2:172.21.15.16:3300/0 conn(0x7f3ea006b080 0x7f3ea0131600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:17.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.113+0000 7f3ea61a5700 1 -- 172.21.15.16:0/3135465355 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ea009f630 con 0x7f3ea0131b40 2023-09-30T22:34:17.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.113+0000 7f3ea61a5700 1 --2- 172.21.15.16:0/3135465355 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3ea0131b40 0x7f3ea0131fc0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f3e98009d90 tx=0x7f3e9800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c48fc80364ecb3fa server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:17.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.113+0000 7f3ea49a2700 1 -- 172.21.15.16:0/3135465355 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9800e070 con 0x7f3ea0131b40 2023-09-30T22:34:17.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.113+0000 7f3ea49a2700 1 -- 172.21.15.16:0/3135465355 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e9800bb20 con 0x7f3ea0131b40 2023-09-30T22:34:17.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.113+0000 7f3ea49a2700 1 -- 172.21.15.16:0/3135465355 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e98012430 con 0x7f3ea0131b40 2023-09-30T22:34:17.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.114+0000 7f3ea7c08700 1 -- 172.21.15.16:0/3135465355 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3ea0131b40 msgr2=0x7f3ea0131fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:17.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.114+0000 7f3ea7c08700 1 --2- 172.21.15.16:0/3135465355 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3ea0131b40 0x7f3ea0131fc0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f3e98009d90 tx=0x7f3e9800b330 comp rx=0 tx=0).stop 2023-09-30T22:34:17.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.114+0000 7f3ea7c08700 1 -- 172.21.15.16:0/3135465355 shutdown_connections 2023-09-30T22:34:17.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.114+0000 7f3ea7c08700 1 --2- 172.21.15.16:0/3135465355 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3ea006a770 0x7f3ea006ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:17.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.114+0000 7f3ea7c08700 1 --2- 172.21.15.16:0/3135465355 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3ea0131b40 0x7f3ea0131fc0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:17.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.114+0000 7f3ea7c08700 1 --2- 172.21.15.16:0/3135465355 >> v2:172.21.15.16:3300/0 conn(0x7f3ea006b080 0x7f3ea0131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:17.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.114+0000 7f3ea7c08700 1 -- 172.21.15.16:0/3135465355 >> 172.21.15.16:0/3135465355 conn(0x7f3ea0066f50 msgr2=0x7f3ea009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:17.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.114+0000 7f3ea7c08700 1 -- 172.21.15.16:0/3135465355 shutdown_connections 2023-09-30T22:34:17.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.114+0000 7f3ea7c08700 1 -- 172.21.15.16:0/3135465355 wait complete. 2023-09-30T22:34:17.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.116+0000 7f3ea7c08700 1 Processor -- start 2023-09-30T22:34:17.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.116+0000 7f3ea7c08700 1 -- start start 2023-09-30T22:34:17.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.117+0000 7f3ea7c08700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3ea006a770 0x7f3ea011c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:17.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.117+0000 7f3ea7c08700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3ea006b080 0x7f3ea011c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:17.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.117+0000 7f3ea7c08700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ea0131b40 0x7f3ea011cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:17.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.117+0000 7f3ea7c08700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea011f8c0 con 0x7f3ea0131b40 2023-09-30T22:34:17.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.117+0000 7f3ea7c08700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea011e560 con 0x7f3ea006a770 2023-09-30T22:34:17.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.117+0000 7f3ea7c08700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea00a3b80 con 0x7f3ea006b080 2023-09-30T22:34:17.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.117+0000 7f3ea61a5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ea0131b40 0x7f3ea011cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:17.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.117+0000 7f3ea51a3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3ea006b080 0x7f3ea011c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:17.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.117+0000 7f3ea61a5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ea0131b40 0x7f3ea011cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33600/0 (socket says 172.21.15.16:33600) 2023-09-30T22:34:17.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.117+0000 7f3ea51a3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3ea006b080 0x7f3ea011c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35372/0 (socket says 172.21.15.16:35372) 2023-09-30T22:34:17.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.117+0000 7f3ea59a4700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3ea006a770 0x7f3ea011c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:17.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.117+0000 7f3ea61a5700 1 -- 172.21.15.16:0/1088563721 learned_addr learned my addr 172.21.15.16:0/1088563721 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:17.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.117+0000 7f3ea59a4700 1 -- 172.21.15.16:0/1088563721 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3ea006b080 msgr2=0x7f3ea011c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:17.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.117+0000 7f3ea59a4700 1 --2- 172.21.15.16:0/1088563721 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3ea006b080 0x7f3ea011c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:17.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.117+0000 7f3ea59a4700 1 -- 172.21.15.16:0/1088563721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ea0131b40 msgr2=0x7f3ea011cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:17.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.117+0000 7f3ea59a4700 1 --2- 172.21.15.16:0/1088563721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ea0131b40 0x7f3ea011cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:17.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.117+0000 7f3ea59a4700 1 -- 172.21.15.16:0/1088563721 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e98009a70 con 0x7f3ea006a770 2023-09-30T22:34:17.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.117+0000 7f3ea59a4700 1 --2- 172.21.15.16:0/1088563721 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3ea006a770 0x7f3ea011c080 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c00e910 tx=0x7f3e9c00ec20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:17.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.118+0000 7f3e967fc700 1 -- 172.21.15.16:0/1088563721 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c00ee60 con 0x7f3ea006a770 2023-09-30T22:34:17.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.118+0000 7f3e967fc700 1 -- 172.21.15.16:0/1088563721 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e9c00c850 con 0x7f3ea006a770 2023-09-30T22:34:17.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.118+0000 7f3e967fc700 1 -- 172.21.15.16:0/1088563721 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9c00cbe0 con 0x7f3ea006a770 2023-09-30T22:34:17.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.118+0000 7f3ea7c08700 1 -- 172.21.15.16:0/1088563721 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ea011e240 con 0x7f3ea006a770 2023-09-30T22:34:17.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.118+0000 7f3ea7c08700 1 -- 172.21.15.16:0/1088563721 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ea0071ec0 con 0x7f3ea006a770 2023-09-30T22:34:17.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.119+0000 7f3e967fc700 1 -- 172.21.15.16:0/1088563721 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3e9c00cd40 con 0x7f3ea006a770 2023-09-30T22:34:17.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.119+0000 7f3ea7c08700 1 -- 172.21.15.16:0/1088563721 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ea011f450 con 0x7f3ea006a770 2023-09-30T22:34:17.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.120+0000 7f3e967fc700 1 -- 172.21.15.16:0/1088563721 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f3e9c03f760 con 0x7f3ea006a770 2023-09-30T22:34:17.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.124+0000 7f3e967fc700 1 -- 172.21.15.16:0/1088563721 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3e9c0232f0 con 0x7f3ea006a770 2023-09-30T22:34:17.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.146+0000 7fe28eb97700 1 Processor -- start 2023-09-30T22:34:17.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.146+0000 7fe28eb97700 1 -- start start 2023-09-30T22:34:17.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.146+0000 7fe28eb97700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe28809d6f0 0x7fe28809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:17.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.146+0000 7fe28eb97700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe28809e000 0x7fe288130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:17.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.147+0000 7fe28eb97700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe28809e410 0x7fe288130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:17.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.147+0000 7fe28eb97700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe28809f520 con 0x7fe28809e000 2023-09-30T22:34:17.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.147+0000 7fe28eb97700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe28809f130 con 0x7fe28809d6f0 2023-09-30T22:34:17.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.147+0000 7fe28eb97700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe28809e930 con 0x7fe28809e410 2023-09-30T22:34:17.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.147+0000 7fe28d134700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe28809e410 0x7fe288130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:17.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.147+0000 7fe28d134700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe28809e410 0x7fe288130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35386/0 (socket says 172.21.15.16:35386) 2023-09-30T22:34:17.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.147+0000 7fe28d134700 1 -- 172.21.15.16:0/1812906018 learned_addr learned my addr 172.21.15.16:0/1812906018 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:17.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.147+0000 7fe28c933700 1 --2- 172.21.15.16:0/1812906018 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe28809d6f0 0x7fe28809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:17.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.147+0000 7fe287fff700 1 --2- 172.21.15.16:0/1812906018 >> v2:172.21.15.16:3300/0 conn(0x7fe28809e000 0x7fe288130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:17.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.147+0000 7fe287fff700 1 -- 172.21.15.16:0/1812906018 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe28809e410 msgr2=0x7fe288130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:17.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.147+0000 7fe287fff700 1 --2- 172.21.15.16:0/1812906018 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe28809e410 0x7fe288130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:17.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.147+0000 7fe287fff700 1 -- 172.21.15.16:0/1812906018 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe28809d6f0 msgr2=0x7fe28809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:17.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.147+0000 7fe287fff700 1 --2- 172.21.15.16:0/1812906018 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe28809d6f0 0x7fe28809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:17.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.147+0000 7fe287fff700 1 -- 172.21.15.16:0/1812906018 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe28809b2d0 con 0x7fe28809e000 2023-09-30T22:34:17.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.147+0000 7fe28d134700 1 --2- 172.21.15.16:0/1812906018 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe28809e410 0x7fe288130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:17.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.148+0000 7fe287fff700 1 --2- 172.21.15.16:0/1812906018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe28809e000 0x7fe288130660 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fe274009870 tx=0x7fe274009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7558ed0e6d3985f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:17.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.148+0000 7fe2877fe700 1 -- 172.21.15.16:0/1812906018 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe274011830 con 0x7fe28809e000 2023-09-30T22:34:17.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.148+0000 7fe2877fe700 1 -- 172.21.15.16:0/1812906018 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe274011990 con 0x7fe28809e000 2023-09-30T22:34:17.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.148+0000 7fe2877fe700 1 -- 172.21.15.16:0/1812906018 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe274011d20 con 0x7fe28809e000 2023-09-30T22:34:17.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.149+0000 7fe28eb97700 1 -- 172.21.15.16:0/1812906018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe28809e000 msgr2=0x7fe288130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:17.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.149+0000 7fe28eb97700 1 --2- 172.21.15.16:0/1812906018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe28809e000 0x7fe288130660 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fe274009870 tx=0x7fe274009b80 comp rx=0 tx=0).stop 2023-09-30T22:34:17.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.149+0000 7fe28eb97700 1 -- 172.21.15.16:0/1812906018 shutdown_connections 2023-09-30T22:34:17.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.149+0000 7fe28eb97700 1 --2- 172.21.15.16:0/1812906018 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe28809d6f0 0x7fe28809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:17.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.149+0000 7fe28eb97700 1 --2- 172.21.15.16:0/1812906018 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe28809e410 0x7fe288130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:17.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.149+0000 7fe28eb97700 1 --2- 172.21.15.16:0/1812906018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe28809e000 0x7fe288130660 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:17.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.149+0000 7fe28eb97700 1 -- 172.21.15.16:0/1812906018 >> 172.21.15.16:0/1812906018 conn(0x7fe288066f50 msgr2=0x7fe28809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:17.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.149+0000 7fe28eb97700 1 -- 172.21.15.16:0/1812906018 shutdown_connections 2023-09-30T22:34:17.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.149+0000 7fe28eb97700 1 -- 172.21.15.16:0/1812906018 wait complete. 2023-09-30T22:34:17.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.151+0000 7fe28eb97700 1 Processor -- start 2023-09-30T22:34:17.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.151+0000 7fe28eb97700 1 -- start start 2023-09-30T22:34:17.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.152+0000 7fe28eb97700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe28809d6f0 0x7fe28811e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:17.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.152+0000 7fe28eb97700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe28809e000 0x7fe28811e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:17.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.152+0000 7fe28eb97700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe28809e410 0x7fe28811eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:17.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.152+0000 7fe28eb97700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe288121af0 con 0x7fe28809e410 2023-09-30T22:34:17.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.152+0000 7fe28eb97700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2881201c0 con 0x7fe28809e000 2023-09-30T22:34:17.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.152+0000 7fe28eb97700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe288121280 con 0x7fe28809d6f0 2023-09-30T22:34:17.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.152+0000 7fe28c933700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe28809d6f0 0x7fe28811e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:17.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.152+0000 7fe28c933700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe28809d6f0 0x7fe28811e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35398/0 (socket says 172.21.15.16:35398) 2023-09-30T22:34:17.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.152+0000 7fe28c933700 1 -- 172.21.15.16:0/1231342959 learned_addr learned my addr 172.21.15.16:0/1231342959 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:17.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.152+0000 7fe287fff700 1 --2- 172.21.15.16:0/1231342959 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe28809e000 0x7fe28811e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:17.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.152+0000 7fe28d134700 1 --2- 172.21.15.16:0/1231342959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe28809e410 0x7fe28811eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:17.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.152+0000 7fe28c933700 1 -- 172.21.15.16:0/1231342959 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe28809e000 msgr2=0x7fe28811e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:17.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.152+0000 7fe28c933700 1 --2- 172.21.15.16:0/1231342959 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe28809e000 0x7fe28811e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:17.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.152+0000 7fe28c933700 1 -- 172.21.15.16:0/1231342959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe28809e410 msgr2=0x7fe28811eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:17.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.152+0000 7fe28c933700 1 --2- 172.21.15.16:0/1231342959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe28809e410 0x7fe28811eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:17.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.152+0000 7fe28c933700 1 -- 172.21.15.16:0/1231342959 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe280009b30 con 0x7fe28809d6f0 2023-09-30T22:34:17.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.152+0000 7fe287fff700 1 --2- 172.21.15.16:0/1231342959 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe28809e000 0x7fe28811e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:17.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.153+0000 7fe28c933700 1 --2- 172.21.15.16:0/1231342959 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe28809d6f0 0x7fe28811e020 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fe27c010e20 tx=0x7fe27c00e640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:17.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.153+0000 7fe2857fa700 1 -- 172.21.15.16:0/1231342959 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27c00b930 con 0x7fe28809d6f0 2023-09-30T22:34:17.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.153+0000 7fe2857fa700 1 -- 172.21.15.16:0/1231342959 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe27c011450 con 0x7fe28809d6f0 2023-09-30T22:34:17.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.153+0000 7fe28eb97700 1 -- 172.21.15.16:0/1231342959 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2740094a0 con 0x7fe28809d6f0 2023-09-30T22:34:17.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.153+0000 7fe2857fa700 1 -- 172.21.15.16:0/1231342959 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe27c0117e0 con 0x7fe28809d6f0 2023-09-30T22:34:17.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.153+0000 7fe28eb97700 1 -- 172.21.15.16:0/1231342959 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe28809c750 con 0x7fe28809d6f0 2023-09-30T22:34:17.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.154+0000 7fe2857fa700 1 -- 172.21.15.16:0/1231342959 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe27c011940 con 0x7fe28809d6f0 2023-09-30T22:34:17.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.154+0000 7fe2857fa700 1 -- 172.21.15.16:0/1231342959 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fe27c042350 con 0x7fe28809d6f0 2023-09-30T22:34:17.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.154+0000 7fe28eb97700 1 -- 172.21.15.16:0/1231342959 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe28811ac20 con 0x7fe28809d6f0 2023-09-30T22:34:17.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.159+0000 7fe2857fa700 1 -- 172.21.15.16:0/1231342959 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe27c017da0 con 0x7fe28809d6f0 2023-09-30T22:34:17.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.190+0000 7f8d3eae0700 1 Processor -- start 2023-09-30T22:34:17.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.190+0000 7f8d3eae0700 1 -- start start 2023-09-30T22:34:17.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.191+0000 7f8d3eae0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8d3809e1d0 0x7f8d38138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:17.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.191+0000 7f8d3eae0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8d38139170 0x7f8d3813b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:17.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.191+0000 7f8d3eae0700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d3809b120 con 0x7f8d3809d6f0 2023-09-30T22:34:17.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.191+0000 7f8d3eae0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3806b460 con 0x7f8d38139170 2023-09-30T22:34:17.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.191+0000 7f8d3eae0700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3809a7e0 con 0x7f8d3809e1d0 2023-09-30T22:34:17.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.191+0000 7f8d3c87c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8d3809d6f0 0x7f8d3809dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:59876/0 (socket says 172.21.15.16:59876) 2023-09-30T22:34:17.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.192+0000 7f8d3c87c700 1 -- 172.21.15.16:0/2670390832 learned_addr learned my addr 172.21.15.16:0/2670390832 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:34:17.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.192+0000 7f8d37fff700 1 --2- 172.21.15.16:0/2670390832 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8d3809e1d0 0x7f8d38138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:17.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.192+0000 7f8d3d07d700 1 --2- 172.21.15.16:0/2670390832 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8d38139170 0x7f8d3813b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:17.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.192+0000 7f8d37fff700 1 -- 172.21.15.16:0/2670390832 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8d38139170 msgr2=0x7f8d3813b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:17.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.192+0000 7f8d37fff700 1 --2- 172.21.15.16:0/2670390832 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8d38139170 0x7f8d3813b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:17.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.192+0000 7f8d37fff700 1 -- 172.21.15.16:0/2670390832 >> v1:172.21.15.16:6789/0 conn(0x7f8d3809d6f0 legacy=0x7f8d3809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:17.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.192+0000 7f8d37fff700 1 -- 172.21.15.16:0/2670390832 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d3809a500 con 0x7f8d3809e1d0 2023-09-30T22:34:17.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.192+0000 7f8d37fff700 1 --2- 172.21.15.16:0/2670390832 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8d3809e1d0 0x7f8d38138c30 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f8d240098a0 tx=0x7f8d24009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6d2855cf31bdc31 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:17.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.192+0000 7f8d377fe700 1 -- 172.21.15.16:0/2670390832 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d24004230 con 0x7f8d3809e1d0 2023-09-30T22:34:17.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.192+0000 7f8d377fe700 1 -- 172.21.15.16:0/2670390832 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d24004390 con 0x7f8d3809e1d0 2023-09-30T22:34:17.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.192+0000 7f8d377fe700 1 -- 172.21.15.16:0/2670390832 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d240036a0 con 0x7f8d3809e1d0 2023-09-30T22:34:17.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.193+0000 7f8d3eae0700 1 -- 172.21.15.16:0/2670390832 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8d3809e1d0 msgr2=0x7f8d38138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:17.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.193+0000 7f8d3eae0700 1 --2- 172.21.15.16:0/2670390832 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8d3809e1d0 0x7f8d38138c30 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f8d240098a0 tx=0x7f8d24009bb0 comp rx=0 tx=0).stop 2023-09-30T22:34:17.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.194+0000 7f8d3eae0700 1 -- 172.21.15.16:0/2670390832 shutdown_connections 2023-09-30T22:34:17.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.194+0000 7f8d3eae0700 1 --2- 172.21.15.16:0/2670390832 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8d38139170 0x7f8d3813b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:17.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.194+0000 7f8d3eae0700 1 --2- 172.21.15.16:0/2670390832 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8d3809e1d0 0x7f8d38138c30 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:17.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.194+0000 7f8d3eae0700 1 -- 172.21.15.16:0/2670390832 >> 172.21.15.16:0/2670390832 conn(0x7f8d38066f50 msgr2=0x7f8d3809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:17.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.194+0000 7f8d3eae0700 1 -- 172.21.15.16:0/2670390832 shutdown_connections 2023-09-30T22:34:17.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.194+0000 7f8d3eae0700 1 -- 172.21.15.16:0/2670390832 wait complete. 2023-09-30T22:34:17.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.195+0000 7f8d3eae0700 1 Processor -- start 2023-09-30T22:34:17.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.195+0000 7f8d3eae0700 1 -- start start 2023-09-30T22:34:17.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.195+0000 7f8d3eae0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d3809d6f0 0x7f8d3811f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:17.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.195+0000 7f8d3eae0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8d3809e1d0 0x7f8d3811f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:17.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.195+0000 7f8d3eae0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8d38139170 0x7f8d381c8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:17.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.195+0000 7f8d3eae0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d38122390 con 0x7f8d3809d6f0 2023-09-30T22:34:17.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.195+0000 7f8d3eae0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d38122570 con 0x7f8d3809e1d0 2023-09-30T22:34:17.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.195+0000 7f8d3eae0700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8d38121fd0 con 0x7f8d38139170 2023-09-30T22:34:17.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.196+0000 7f8d37fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8d3809e1d0 0x7f8d3811f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:17.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.196+0000 7f8d37fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8d3809e1d0 0x7f8d3811f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:53146/0 (socket says 172.21.15.16:53146) 2023-09-30T22:34:17.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.196+0000 7f8d37fff700 1 -- 172.21.15.16:0/1574549789 learned_addr learned my addr 172.21.15.16:0/1574549789 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:17.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.196+0000 7f8d3d07d700 1 --2- 172.21.15.16:0/1574549789 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8d38139170 0x7f8d381c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:17.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.196+0000 7f8d3c87c700 1 --2- 172.21.15.16:0/1574549789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d3809d6f0 0x7f8d3811f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:17.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.196+0000 7f8d3d07d700 1 -- 172.21.15.16:0/1574549789 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8d3809e1d0 msgr2=0x7f8d3811f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:17.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.196+0000 7f8d3d07d700 1 --2- 172.21.15.16:0/1574549789 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8d3809e1d0 0x7f8d3811f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:17.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.196+0000 7f8d3d07d700 1 -- 172.21.15.16:0/1574549789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d3809d6f0 msgr2=0x7f8d3811f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:17.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.196+0000 7f8d3d07d700 1 --2- 172.21.15.16:0/1574549789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d3809d6f0 0x7f8d3811f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:17.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.196+0000 7f8d3d07d700 1 -- 172.21.15.16:0/1574549789 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d240094a0 con 0x7f8d38139170 2023-09-30T22:34:17.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.196+0000 7f8d37fff700 1 --2- 172.21.15.16:0/1574549789 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8d3809e1d0 0x7f8d3811f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:17.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.196+0000 7f8d3d07d700 1 --2- 172.21.15.16:0/1574549789 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8d38139170 0x7f8d381c8460 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f8d30009d90 tx=0x7f8d3000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:17.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.197+0000 7f8d357fa700 1 -- 172.21.15.16:0/1574549789 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d30009c00 con 0x7f8d38139170 2023-09-30T22:34:17.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.197+0000 7f8d357fa700 1 -- 172.21.15.16:0/1574549789 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d3000f040 con 0x7f8d38139170 2023-09-30T22:34:17.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.197+0000 7f8d3eae0700 1 -- 172.21.15.16:0/1574549789 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d3811ef50 con 0x7f8d38139170 2023-09-30T22:34:17.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.197+0000 7f8d357fa700 1 -- 172.21.15.16:0/1574549789 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d30014480 con 0x7f8d38139170 2023-09-30T22:34:17.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.197+0000 7f8d3eae0700 1 -- 172.21.15.16:0/1574549789 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d38120660 con 0x7f8d38139170 2023-09-30T22:34:17.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.198+0000 7f8d3eae0700 1 -- 172.21.15.16:0/1574549789 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d38074ea0 con 0x7f8d38139170 2023-09-30T22:34:17.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.198+0000 7f8d357fa700 1 -- 172.21.15.16:0/1574549789 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8d300145e0 con 0x7f8d38139170 2023-09-30T22:34:17.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.198+0000 7f8d357fa700 1 -- 172.21.15.16:0/1574549789 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f8d3001d400 con 0x7f8d38139170 2023-09-30T22:34:17.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.200+0000 7fbc1cecc700 1 Processor -- start 2023-09-30T22:34:17.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.200+0000 7fbc1cecc700 1 -- start start 2023-09-30T22:34:17.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.201+0000 7fbc1cecc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbc181305f0 0x7fbc181309c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:17.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.201+0000 7fbc1cecc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbc1806f660 0x7fbc1806fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:17.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.201+0000 7fbc1cecc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbc1806ff90 0x7fbc18137530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:17.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.202+0000 7fbc1cecc700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc181301c0 con 0x7fbc1806ff90 2023-09-30T22:34:17.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.202+0000 7fbc1cecc700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc18071ff0 con 0x7fbc181305f0 2023-09-30T22:34:17.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.202+0000 7fbc1cecc700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbc18070e70 con 0x7fbc1806f660 2023-09-30T22:34:17.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.202+0000 7fbc15d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbc1806f660 0x7fbc1806fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:17.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.202+0000 7fbc16d9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbc1806ff90 0x7fbc18137530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:17.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.202+0000 7fbc15d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbc1806f660 0x7fbc1806fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35426/0 (socket says 172.21.15.16:35426) 2023-09-30T22:34:17.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.202+0000 7fbc15d9b700 1 -- 172.21.15.16:0/2473392682 learned_addr learned my addr 172.21.15.16:0/2473392682 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:17.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.202+0000 7fbc1659c700 1 --2- 172.21.15.16:0/2473392682 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbc181305f0 0x7fbc181309c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:17.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.202+0000 7fbc15d9b700 1 -- 172.21.15.16:0/2473392682 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbc181305f0 msgr2=0x7fbc181309c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:17.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.202+0000 7fbc15d9b700 1 --2- 172.21.15.16:0/2473392682 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbc181305f0 0x7fbc181309c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:17.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.202+0000 7fbc15d9b700 1 -- 172.21.15.16:0/2473392682 >> v2:172.21.15.16:3300/0 conn(0x7fbc1806ff90 msgr2=0x7fbc18137530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:17.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.202+0000 7fbc15d9b700 1 --2- 172.21.15.16:0/2473392682 >> v2:172.21.15.16:3300/0 conn(0x7fbc1806ff90 0x7fbc18137530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:17.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.202+0000 7fbc15d9b700 1 -- 172.21.15.16:0/2473392682 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc180722b0 con 0x7fbc1806f660 2023-09-30T22:34:17.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.202+0000 7fbc15d9b700 1 --2- 172.21.15.16:0/2473392682 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbc1806f660 0x7fbc1806fa50 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c00b9a0 tx=0x7fbc0c00bcb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb769026dc0d7459 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:17.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.204+0000 7fbc1559a700 1 -- 172.21.15.16:0/2473392682 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c00caa0 con 0x7fbc1806f660 2023-09-30T22:34:17.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.204+0000 7f8d357fa700 1 -- 172.21.15.16:0/1574549789 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8d30045060 con 0x7f8d38139170 2023-09-30T22:34:17.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.204+0000 7fbc1559a700 1 -- 172.21.15.16:0/2473392682 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc0c00cc00 con 0x7fbc1806f660 2023-09-30T22:34:17.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.204+0000 7fbc1559a700 1 -- 172.21.15.16:0/2473392682 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c00f460 con 0x7fbc1806f660 2023-09-30T22:34:17.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.204+0000 7fbc1cecc700 1 -- 172.21.15.16:0/2473392682 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbc1806f660 msgr2=0x7fbc1806fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:17.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.204+0000 7fbc1cecc700 1 --2- 172.21.15.16:0/2473392682 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbc1806f660 0x7fbc1806fa50 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c00b9a0 tx=0x7fbc0c00bcb0 comp rx=0 tx=0).stop 2023-09-30T22:34:17.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.205+0000 7fbc1cecc700 1 -- 172.21.15.16:0/2473392682 shutdown_connections 2023-09-30T22:34:17.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.205+0000 7fbc1cecc700 1 --2- 172.21.15.16:0/2473392682 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbc181305f0 0x7fbc181309c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:17.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.205+0000 7fbc1cecc700 1 --2- 172.21.15.16:0/2473392682 >> v2:172.21.15.16:3300/0 conn(0x7fbc1806ff90 0x7fbc18137530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:17.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.205+0000 7fbc1cecc700 1 --2- 172.21.15.16:0/2473392682 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbc1806f660 0x7fbc1806fa50 secure :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c00b9a0 tx=0x7fbc0c00bcb0 comp rx=0 tx=0).stop 2023-09-30T22:34:17.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.205+0000 7fbc1cecc700 1 -- 172.21.15.16:0/2473392682 >> 172.21.15.16:0/2473392682 conn(0x7fbc180a2aa0 msgr2=0x7fbc180a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:17.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.209+0000 7fbc1cecc700 1 -- 172.21.15.16:0/2473392682 shutdown_connections 2023-09-30T22:34:17.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.209+0000 7fbc1cecc700 1 -- 172.21.15.16:0/2473392682 wait complete. 2023-09-30T22:34:17.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.210+0000 7fbc1cecc700 1 Processor -- start 2023-09-30T22:34:17.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.210+0000 7fbc1cecc700 1 -- start start 2023-09-30T22:34:17.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.211+0000 7fbc1cecc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbc1806ff90 0x7fbc1811ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:17.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.211+0000 7fbc1cecc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc181305f0 0x7fbc1811bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:17.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.211+0000 7fbc1cecc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbc1811fa40 0x7fbc1811c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:17.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.211+0000 7fbc1cecc700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1811f260 con 0x7fbc181305f0 2023-09-30T22:34:17.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.211+0000 7fbc1cecc700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1811d740 con 0x7fbc1811fa40 2023-09-30T22:34:17.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.211+0000 7fbc1cecc700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1811dbe0 con 0x7fbc1806ff90 2023-09-30T22:34:17.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.211+0000 7fbc15d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc181305f0 0x7fbc1811bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:17.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.211+0000 7fbc15d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc181305f0 0x7fbc1811bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33654/0 (socket says 172.21.15.16:33654) 2023-09-30T22:34:17.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.211+0000 7fbc15d9b700 1 -- 172.21.15.16:0/4050662514 learned_addr learned my addr 172.21.15.16:0/4050662514 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:17.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.212+0000 7fbc1659c700 1 --2- 172.21.15.16:0/4050662514 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbc1806ff90 0x7fbc1811ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:17.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.212+0000 7fbc16d9d700 1 --2- 172.21.15.16:0/4050662514 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbc1811fa40 0x7fbc1811c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:17.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.212+0000 7fbc15d9b700 1 -- 172.21.15.16:0/4050662514 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbc1806ff90 msgr2=0x7fbc1811ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:17.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.212+0000 7fbc15d9b700 1 --2- 172.21.15.16:0/4050662514 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbc1806ff90 0x7fbc1811ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:17.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.212+0000 7fbc15d9b700 1 -- 172.21.15.16:0/4050662514 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbc1811fa40 msgr2=0x7fbc1811c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:17.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.212+0000 7fbc15d9b700 1 --2- 172.21.15.16:0/4050662514 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbc1811fa40 0x7fbc1811c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:17.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.212+0000 7fbc15d9b700 1 -- 172.21.15.16:0/4050662514 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc0c00b600 con 0x7fbc181305f0 2023-09-30T22:34:17.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.212+0000 7fbc15d9b700 1 --2- 172.21.15.16:0/4050662514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc181305f0 0x7fbc1811bf60 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c0090b0 tx=0x7fbc0c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:17.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.212+0000 7fbc06ffd700 1 -- 172.21.15.16:0/4050662514 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c01b070 con 0x7fbc181305f0 2023-09-30T22:34:17.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.212+0000 7fbc06ffd700 1 -- 172.21.15.16:0/4050662514 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc0c01c040 con 0x7fbc181305f0 2023-09-30T22:34:17.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.213+0000 7fbc1cecc700 1 -- 172.21.15.16:0/4050662514 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc1812ed60 con 0x7fbc181305f0 2023-09-30T22:34:17.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.213+0000 7fbc06ffd700 1 -- 172.21.15.16:0/4050662514 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c003ac0 con 0x7fbc181305f0 2023-09-30T22:34:17.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.213+0000 7fbc1cecc700 1 -- 172.21.15.16:0/4050662514 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc181c42f0 con 0x7fbc181305f0 2023-09-30T22:34:17.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.214+0000 7fbc06ffd700 1 -- 172.21.15.16:0/4050662514 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbc0c003c20 con 0x7fbc181305f0 2023-09-30T22:34:17.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.214+0000 7fbc1cecc700 1 -- 172.21.15.16:0/4050662514 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc18074ea0 con 0x7fbc181305f0 2023-09-30T22:34:17.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.214+0000 7fbc06ffd700 1 -- 172.21.15.16:0/4050662514 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fbc0c040aa0 con 0x7fbc181305f0 2023-09-30T22:34:17.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.223+0000 7fbc06ffd700 1 -- 172.21.15.16:0/4050662514 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbc0c00f460 con 0x7fbc181305f0 2023-09-30T22:34:17.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.346+0000 7f3ea7c08700 1 -- 172.21.15.16:0/1088563721 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3ea0074ea0 con 0x7f3ea006a770 2023-09-30T22:34:17.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.347+0000 7f3e967fc700 1 -- 172.21.15.16:0/1088563721 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3e9c021090 con 0x7f3ea006a770 2023-09-30T22:34:17.350 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:34:17.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.350+0000 7f3ea7c08700 1 -- 172.21.15.16:0/1088563721 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3ea006a770 msgr2=0x7f3ea011c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:17.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.350+0000 7f3ea7c08700 1 --2- 172.21.15.16:0/1088563721 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3ea006a770 0x7f3ea011c080 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c00e910 tx=0x7f3e9c00ec20 comp rx=0 tx=0).stop 2023-09-30T22:34:17.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.350+0000 7f3ea7c08700 1 -- 172.21.15.16:0/1088563721 shutdown_connections 2023-09-30T22:34:17.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.350+0000 7f3ea7c08700 1 --2- 172.21.15.16:0/1088563721 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3ea006a770 0x7f3ea011c080 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:17.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.350+0000 7f3ea7c08700 1 --2- 172.21.15.16:0/1088563721 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ea0131b40 0x7f3ea011cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:17.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.350+0000 7f3ea7c08700 1 --2- 172.21.15.16:0/1088563721 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3ea006b080 0x7f3ea011c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:17.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.350+0000 7f3ea7c08700 1 -- 172.21.15.16:0/1088563721 >> 172.21.15.16:0/1088563721 conn(0x7f3ea0066f50 msgr2=0x7f3ea0130ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:17.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.350+0000 7f3ea7c08700 1 -- 172.21.15.16:0/1088563721 shutdown_connections 2023-09-30T22:34:17.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.350+0000 7f3ea7c08700 1 -- 172.21.15.16:0/1088563721 wait complete. 2023-09-30T22:34:17.370 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:34:17.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.387+0000 7fe28eb97700 1 -- 172.21.15.16:0/1231342959 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe288074ea0 con 0x7fe28809d6f0 2023-09-30T22:34:17.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.388+0000 7fe2857fa700 1 -- 172.21.15.16:0/1231342959 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe27c01e020 con 0x7fe28809d6f0 2023-09-30T22:34:17.388 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:34:17.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.390+0000 7fe28eb97700 1 -- 172.21.15.16:0/1231342959 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe28809d6f0 msgr2=0x7fe28811e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:17.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.390+0000 7fe28eb97700 1 --2- 172.21.15.16:0/1231342959 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe28809d6f0 0x7fe28811e020 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fe27c010e20 tx=0x7fe27c00e640 comp rx=0 tx=0).stop 2023-09-30T22:34:17.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.390+0000 7fe28eb97700 1 -- 172.21.15.16:0/1231342959 shutdown_connections 2023-09-30T22:34:17.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.390+0000 7fe28eb97700 1 --2- 172.21.15.16:0/1231342959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe28809e410 0x7fe28811eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:17.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.390+0000 7fe28eb97700 1 --2- 172.21.15.16:0/1231342959 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe28809d6f0 0x7fe28811e020 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:17.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.390+0000 7fe28eb97700 1 --2- 172.21.15.16:0/1231342959 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe28809e000 0x7fe28811e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:17.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.390+0000 7fe28eb97700 1 -- 172.21.15.16:0/1231342959 >> 172.21.15.16:0/1231342959 conn(0x7fe288066f50 msgr2=0x7fe288130920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:17.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.390+0000 7fe28eb97700 1 -- 172.21.15.16:0/1231342959 shutdown_connections 2023-09-30T22:34:17.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.390+0000 7fe28eb97700 1 -- 172.21.15.16:0/1231342959 wait complete. 2023-09-30T22:34:17.404 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:34:17.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.415+0000 7f8d3eae0700 1 -- 172.21.15.16:0/1574549789 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8d3806c360 con 0x7f8d38139170 2023-09-30T22:34:17.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.415+0000 7f8d357fa700 1 -- 172.21.15.16:0/1574549789 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8d30018020 con 0x7f8d38139170 2023-09-30T22:34:17.415 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:34:17.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.418+0000 7f8d3eae0700 1 -- 172.21.15.16:0/1574549789 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8d38139170 msgr2=0x7f8d381c8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:17.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.418+0000 7f8d3eae0700 1 --2- 172.21.15.16:0/1574549789 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8d38139170 0x7f8d381c8460 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f8d30009d90 tx=0x7f8d3000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:34:17.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.419+0000 7f8d3eae0700 1 -- 172.21.15.16:0/1574549789 shutdown_connections 2023-09-30T22:34:17.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.419+0000 7f8d3eae0700 1 --2- 172.21.15.16:0/1574549789 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8d38139170 0x7f8d381c8460 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:17.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.419+0000 7f8d3eae0700 1 --2- 172.21.15.16:0/1574549789 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d3809d6f0 0x7f8d3811f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:17.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.419+0000 7f8d3eae0700 1 --2- 172.21.15.16:0/1574549789 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8d3809e1d0 0x7f8d3811f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:17.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.419+0000 7f8d3eae0700 1 -- 172.21.15.16:0/1574549789 >> 172.21.15.16:0/1574549789 conn(0x7f8d38066f50 msgr2=0x7f8d381381e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:17.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.419+0000 7f8d3eae0700 1 -- 172.21.15.16:0/1574549789 shutdown_connections 2023-09-30T22:34:17.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.419+0000 7f8d3eae0700 1 -- 172.21.15.16:0/1574549789 wait complete. 2023-09-30T22:34:17.437 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:34:17.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.451+0000 7fbc1cecc700 1 -- 172.21.15.16:0/4050662514 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fbc1806b9e0 con 0x7fbc181305f0 2023-09-30T22:34:17.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.451+0000 7fbc06ffd700 1 -- 172.21.15.16:0/4050662514 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbc0c0293c0 con 0x7fbc181305f0 2023-09-30T22:34:17.452 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:34:17.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.454+0000 7fbc1cecc700 1 -- 172.21.15.16:0/4050662514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc181305f0 msgr2=0x7fbc1811bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:17.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.454+0000 7fbc1cecc700 1 --2- 172.21.15.16:0/4050662514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc181305f0 0x7fbc1811bf60 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c0090b0 tx=0x7fbc0c004750 comp rx=0 tx=0).stop 2023-09-30T22:34:17.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.454+0000 7fbc1cecc700 1 -- 172.21.15.16:0/4050662514 shutdown_connections 2023-09-30T22:34:17.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.454+0000 7fbc1cecc700 1 --2- 172.21.15.16:0/4050662514 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbc1811fa40 0x7fbc1811c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:17.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.454+0000 7fbc1cecc700 1 --2- 172.21.15.16:0/4050662514 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbc1806ff90 0x7fbc1811ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:17.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.454+0000 7fbc1cecc700 1 --2- 172.21.15.16:0/4050662514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc181305f0 0x7fbc1811bf60 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:17.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.454+0000 7fbc1cecc700 1 -- 172.21.15.16:0/4050662514 >> 172.21.15.16:0/4050662514 conn(0x7fbc180a2aa0 msgr2=0x7fbc1806e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:17.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.454+0000 7fbc1cecc700 1 -- 172.21.15.16:0/4050662514 shutdown_connections 2023-09-30T22:34:17.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:17.454+0000 7fbc1cecc700 1 -- 172.21.15.16:0/4050662514 wait complete. 2023-09-30T22:34:17.467 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:34:18.371 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:34:18.405 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:34:18.438 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:34:18.469 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:34:18.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.522+0000 7f12eb6a0700 1 Processor -- start 2023-09-30T22:34:18.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.523+0000 7f12eb6a0700 1 -- start start 2023-09-30T22:34:18.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.524+0000 7f12eb6a0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12e41338c0 0x7f12e4133c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:18.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.524+0000 7f12eb6a0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12e409e560 0x7f12e413b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:18.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.524+0000 7f12eb6a0700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12e406f720 con 0x7f12e41341d0 2023-09-30T22:34:18.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.524+0000 7f12eb6a0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12e4070c80 con 0x7f12e41338c0 2023-09-30T22:34:18.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.524+0000 7f12eb6a0700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f12e406fce0 con 0x7f12e409e560 2023-09-30T22:34:18.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.524+0000 7f12e943c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12e41338c0 0x7f12e4133c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:18.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.524+0000 7f12e943c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12e41338c0 0x7f12e4133c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:55250/0 (socket says 172.21.15.16:55250) 2023-09-30T22:34:18.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.524+0000 7f12e8c3b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f12e41341d0 0x7f12e409de50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54840/0 (socket says 172.21.15.16:54840) 2023-09-30T22:34:18.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.524+0000 7f12e943c700 1 -- 172.21.15.16:0/905424974 learned_addr learned my addr 172.21.15.16:0/905424974 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:18.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.524+0000 7f12e9c3d700 1 --2- 172.21.15.16:0/905424974 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12e409e560 0x7f12e413b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:18.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.524+0000 7f12dbfff700 1 -- 172.21.15.16:0/905424974 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3379328557 0 0) 0x7f12e406f720 con 0x7f12e41341d0 2023-09-30T22:34:18.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.524+0000 7f12e9c3d700 1 -- 172.21.15.16:0/905424974 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12e41338c0 msgr2=0x7f12e4133c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:18.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.524+0000 7f12e9c3d700 1 --2- 172.21.15.16:0/905424974 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12e41338c0 0x7f12e4133c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:18.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.524+0000 7f12e9c3d700 1 -- 172.21.15.16:0/905424974 >> v1:172.21.15.16:6789/0 conn(0x7f12e41341d0 legacy=0x7f12e409de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:18.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.524+0000 7f12e9c3d700 1 -- 172.21.15.16:0/905424974 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12e41336a0 con 0x7f12e409e560 2023-09-30T22:34:18.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.525+0000 7f12e9c3d700 1 --2- 172.21.15.16:0/905424974 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12e409e560 0x7f12e413b860 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f12dc009dc0 tx=0x7f12dc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cedc3cca7ec896b6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:18.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.525+0000 7f12dbfff700 1 -- 172.21.15.16:0/905424974 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12dc00e070 con 0x7f12e409e560 2023-09-30T22:34:18.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.525+0000 7f12dbfff700 1 -- 172.21.15.16:0/905424974 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12dc00bb20 con 0x7f12e409e560 2023-09-30T22:34:18.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.525+0000 7f12dbfff700 1 -- 172.21.15.16:0/905424974 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12dc012430 con 0x7f12e409e560 2023-09-30T22:34:18.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.526+0000 7f12eb6a0700 1 -- 172.21.15.16:0/905424974 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12e409e560 msgr2=0x7f12e413b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:18.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.526+0000 7f12eb6a0700 1 --2- 172.21.15.16:0/905424974 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12e409e560 0x7f12e413b860 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f12dc009dc0 tx=0x7f12dc00b330 comp rx=0 tx=0).stop 2023-09-30T22:34:18.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.526+0000 7f12eb6a0700 1 -- 172.21.15.16:0/905424974 shutdown_connections 2023-09-30T22:34:18.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.526+0000 7f12eb6a0700 1 --2- 172.21.15.16:0/905424974 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12e409e560 0x7f12e413b860 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:18.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.526+0000 7f12eb6a0700 1 --2- 172.21.15.16:0/905424974 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12e41338c0 0x7f12e4133c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:18.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.526+0000 7f12eb6a0700 1 -- 172.21.15.16:0/905424974 >> 172.21.15.16:0/905424974 conn(0x7f12e40a2aa0 msgr2=0x7f12e40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:18.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.527+0000 7f12eb6a0700 1 -- 172.21.15.16:0/905424974 shutdown_connections 2023-09-30T22:34:18.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.527+0000 7f12eb6a0700 1 -- 172.21.15.16:0/905424974 wait complete. 2023-09-30T22:34:18.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.528+0000 7f12eb6a0700 1 Processor -- start 2023-09-30T22:34:18.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.528+0000 7f12eb6a0700 1 -- start start 2023-09-30T22:34:18.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.529+0000 7f12eb6a0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12e409e560 0x7f12e411f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:18.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.529+0000 7f12eb6a0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12e41338c0 0x7f12e411fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:18.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.529+0000 7f12eb6a0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12e41341d0 0x7f12e41203a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:18.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.529+0000 7f12eb6a0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e4123360 con 0x7f12e41338c0 2023-09-30T22:34:18.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.529+0000 7f12eb6a0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12e4122300 con 0x7f12e41341d0 2023-09-30T22:34:18.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.529+0000 7f12eb6a0700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f12e4131f00 con 0x7f12e409e560 2023-09-30T22:34:18.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.529+0000 7f12e9c3d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12e41341d0 0x7f12e41203a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:18.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.529+0000 7f12e9c3d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12e41341d0 0x7f12e41203a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:55266/0 (socket says 172.21.15.16:55266) 2023-09-30T22:34:18.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.529+0000 7f12e9c3d700 1 -- 172.21.15.16:0/1388662499 learned_addr learned my addr 172.21.15.16:0/1388662499 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:18.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.529+0000 7f12e943c700 1 --2- 172.21.15.16:0/1388662499 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12e409e560 0x7f12e411f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:18.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.530+0000 7f12e8c3b700 1 --2- 172.21.15.16:0/1388662499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12e41338c0 0x7f12e411fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:18.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.530+0000 7f12e9c3d700 1 -- 172.21.15.16:0/1388662499 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12e409e560 msgr2=0x7f12e411f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:18.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.530+0000 7f12e9c3d700 1 --2- 172.21.15.16:0/1388662499 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12e409e560 0x7f12e411f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:18.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.530+0000 7f12e9c3d700 1 -- 172.21.15.16:0/1388662499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12e41338c0 msgr2=0x7f12e411fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:18.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.530+0000 7f12e9c3d700 1 --2- 172.21.15.16:0/1388662499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12e41338c0 0x7f12e411fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:18.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.530+0000 7f12e9c3d700 1 -- 172.21.15.16:0/1388662499 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12dc009a70 con 0x7f12e41341d0 2023-09-30T22:34:18.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.530+0000 7f12e943c700 1 --2- 172.21.15.16:0/1388662499 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12e409e560 0x7f12e411f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:18.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.530+0000 7f12e8c3b700 1 --2- 172.21.15.16:0/1388662499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12e41338c0 0x7f12e411fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:34:18.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.530+0000 7f12e9c3d700 1 --2- 172.21.15.16:0/1388662499 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12e41341d0 0x7f12e41203a0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f12dc014040 tx=0x7f12dc004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:18.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.530+0000 7f12d9ffb700 1 -- 172.21.15.16:0/1388662499 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12dc004930 con 0x7f12e41341d0 2023-09-30T22:34:18.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.530+0000 7f12d9ffb700 1 -- 172.21.15.16:0/1388662499 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12dc020070 con 0x7f12e41341d0 2023-09-30T22:34:18.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.530+0000 7f12d9ffb700 1 -- 172.21.15.16:0/1388662499 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12dc0276c0 con 0x7f12e41341d0 2023-09-30T22:34:18.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.530+0000 7f12eb6a0700 1 -- 172.21.15.16:0/1388662499 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12e4122af0 con 0x7f12e41341d0 2023-09-30T22:34:18.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.531+0000 7f12eb6a0700 1 -- 172.21.15.16:0/1388662499 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12e406f9c0 con 0x7f12e41341d0 2023-09-30T22:34:18.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.532+0000 7f12d9ffb700 1 -- 172.21.15.16:0/1388662499 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f12dc012430 con 0x7f12e41341d0 2023-09-30T22:34:18.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.532+0000 7f12d9ffb700 1 -- 172.21.15.16:0/1388662499 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f12dc022070 con 0x7f12e41341d0 2023-09-30T22:34:18.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.532+0000 7f12eb6a0700 1 -- 172.21.15.16:0/1388662499 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12e4074ea0 con 0x7f12e41341d0 2023-09-30T22:34:18.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.538+0000 7f12d9ffb700 1 -- 172.21.15.16:0/1388662499 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f12dc01b400 con 0x7f12e41341d0 2023-09-30T22:34:18.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.572+0000 7f7662f90700 1 Processor -- start 2023-09-30T22:34:18.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.572+0000 7f7662f90700 1 -- start start 2023-09-30T22:34:18.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.573+0000 7f7662f90700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f765c09e1d0 0x7f765c135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:18.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.573+0000 7f7662f90700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f765c135e50 0x7f765c138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:18.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.573+0000 7f7662f90700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f765c09f130 con 0x7f765c09d6f0 2023-09-30T22:34:18.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.573+0000 7f7662f90700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f765c09f520 con 0x7f765c135e50 2023-09-30T22:34:18.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.573+0000 7f7662f90700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f765c09e930 con 0x7f765c09e1d0 2023-09-30T22:34:18.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.573+0000 7f7660d2c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f765c09d6f0 0x7f765c09dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54850/0 (socket says 172.21.15.16:54850) 2023-09-30T22:34:18.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.573+0000 7f7660d2c700 1 -- 172.21.15.16:0/3491336688 learned_addr learned my addr 172.21.15.16:0/3491336688 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:34:18.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.573+0000 7f766152d700 1 --2- 172.21.15.16:0/3491336688 >> v2:172.21.15.16:3300/0 conn(0x7f765c135e50 0x7f765c138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:18.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.573+0000 7f765bfff700 1 --2- 172.21.15.16:0/3491336688 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f765c09e1d0 0x7f765c135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:18.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.573+0000 7f765b7fe700 1 -- 172.21.15.16:0/3491336688 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 211250011 0 0) 0x7f765c09f130 con 0x7f765c09d6f0 2023-09-30T22:34:18.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.573+0000 7f765b7fe700 1 -- 172.21.15.16:0/3491336688 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f764c0035b0 con 0x7f765c09d6f0 2023-09-30T22:34:18.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.574+0000 7f765bfff700 1 -- 172.21.15.16:0/3491336688 >> v2:172.21.15.16:3300/0 conn(0x7f765c135e50 msgr2=0x7f765c138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:18.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.574+0000 7f765bfff700 1 --2- 172.21.15.16:0/3491336688 >> v2:172.21.15.16:3300/0 conn(0x7f765c135e50 0x7f765c138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:18.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.574+0000 7f765bfff700 1 -- 172.21.15.16:0/3491336688 >> v1:172.21.15.16:6789/0 conn(0x7f765c09d6f0 legacy=0x7f765c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:18.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.574+0000 7f765bfff700 1 -- 172.21.15.16:0/3491336688 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f765c09b2d0 con 0x7f765c09e1d0 2023-09-30T22:34:18.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.574+0000 7f765bfff700 1 --2- 172.21.15.16:0/3491336688 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f765c09e1d0 0x7f765c135910 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f7648009870 tx=0x7f7648009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7020919813ec9932 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:18.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.574+0000 7f765b7fe700 1 -- 172.21.15.16:0/3491336688 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7648011830 con 0x7f765c09e1d0 2023-09-30T22:34:18.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.574+0000 7f765b7fe700 1 -- 172.21.15.16:0/3491336688 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7648011990 con 0x7f765c09e1d0 2023-09-30T22:34:18.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.574+0000 7f765b7fe700 1 -- 172.21.15.16:0/3491336688 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7648011d20 con 0x7f765c09e1d0 2023-09-30T22:34:18.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.575+0000 7f7662f90700 1 -- 172.21.15.16:0/3491336688 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f765c09e1d0 msgr2=0x7f765c135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:18.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.575+0000 7f7662f90700 1 --2- 172.21.15.16:0/3491336688 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f765c09e1d0 0x7f765c135910 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f7648009870 tx=0x7f7648009b80 comp rx=0 tx=0).stop 2023-09-30T22:34:18.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.575+0000 7f7662f90700 1 -- 172.21.15.16:0/3491336688 shutdown_connections 2023-09-30T22:34:18.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.575+0000 7f7662f90700 1 --2- 172.21.15.16:0/3491336688 >> v2:172.21.15.16:3300/0 conn(0x7f765c135e50 0x7f765c138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:18.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.575+0000 7f7662f90700 1 --2- 172.21.15.16:0/3491336688 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f765c09e1d0 0x7f765c135910 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:18.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.575+0000 7f7662f90700 1 -- 172.21.15.16:0/3491336688 >> 172.21.15.16:0/3491336688 conn(0x7f765c066f50 msgr2=0x7f765c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:18.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.575+0000 7f7662f90700 1 -- 172.21.15.16:0/3491336688 shutdown_connections 2023-09-30T22:34:18.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.575+0000 7f7662f90700 1 -- 172.21.15.16:0/3491336688 wait complete. 2023-09-30T22:34:18.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.577+0000 7f7662f90700 1 Processor -- start 2023-09-30T22:34:18.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.577+0000 7f7662f90700 1 -- start start 2023-09-30T22:34:18.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.578+0000 7f7662f90700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f765c09d6f0 0x7f765c11bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:18.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.578+0000 7f7662f90700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f765c09e1d0 0x7f765c11c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:18.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.578+0000 7f7662f90700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f765c135e50 0x7f765c1c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:18.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.578+0000 7f7662f90700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f765c122170 con 0x7f765c09e1d0 2023-09-30T22:34:18.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.578+0000 7f7662f90700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f765c11b620 con 0x7f765c09d6f0 2023-09-30T22:34:18.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.578+0000 7f7662f90700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f765c1311c0 con 0x7f765c135e50 2023-09-30T22:34:18.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.578+0000 7f7660d2c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f765c09d6f0 0x7f765c11bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:18.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.578+0000 7f765bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f765c09e1d0 0x7f765c11c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:18.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.578+0000 7f765bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f765c09e1d0 0x7f765c11c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47388/0 (socket says 172.21.15.16:47388) 2023-09-30T22:34:18.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.578+0000 7f7660d2c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f765c09d6f0 0x7f765c11bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:55282/0 (socket says 172.21.15.16:55282) 2023-09-30T22:34:18.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.578+0000 7f765bfff700 1 -- 172.21.15.16:0/4209747275 learned_addr learned my addr 172.21.15.16:0/4209747275 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:18.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.578+0000 7f766152d700 1 --2- 172.21.15.16:0/4209747275 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f765c135e50 0x7f765c1c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:18.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.579+0000 7f765bfff700 1 -- 172.21.15.16:0/4209747275 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f765c135e50 msgr2=0x7f765c1c82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:18.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.579+0000 7f765bfff700 1 --2- 172.21.15.16:0/4209747275 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f765c135e50 0x7f765c1c82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:18.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.579+0000 7f765bfff700 1 -- 172.21.15.16:0/4209747275 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f765c09d6f0 msgr2=0x7f765c11bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:18.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.579+0000 7f765bfff700 1 --2- 172.21.15.16:0/4209747275 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f765c09d6f0 0x7f765c11bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:18.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.579+0000 7f765bfff700 1 -- 172.21.15.16:0/4209747275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76480094a0 con 0x7f765c09e1d0 2023-09-30T22:34:18.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.579+0000 7f7660d2c700 1 --2- 172.21.15.16:0/4209747275 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f765c09d6f0 0x7f765c11bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:18.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.579+0000 7f765bfff700 1 --2- 172.21.15.16:0/4209747275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f765c09e1d0 0x7f765c11c020 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f7648000c00 tx=0x7f7648004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:18.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.579+0000 7f76597fa700 1 -- 172.21.15.16:0/4209747275 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76480036a0 con 0x7f765c09e1d0 2023-09-30T22:34:18.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.579+0000 7f76597fa700 1 -- 172.21.15.16:0/4209747275 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f764801f070 con 0x7f765c09e1d0 2023-09-30T22:34:18.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.579+0000 7f7662f90700 1 -- 172.21.15.16:0/4209747275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f765c06b1f0 con 0x7f765c09e1d0 2023-09-30T22:34:18.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.579+0000 7f7662f90700 1 -- 172.21.15.16:0/4209747275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f765c11c560 con 0x7f765c09e1d0 2023-09-30T22:34:18.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.580+0000 7f76597fa700 1 -- 172.21.15.16:0/4209747275 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7648003a20 con 0x7f765c09e1d0 2023-09-30T22:34:18.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.581+0000 7f7662f90700 1 -- 172.21.15.16:0/4209747275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f763c0052e0 con 0x7f765c09e1d0 2023-09-30T22:34:18.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.581+0000 7f76597fa700 1 -- 172.21.15.16:0/4209747275 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7648003b80 con 0x7f765c09e1d0 2023-09-30T22:34:18.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.581+0000 7f76597fa700 1 -- 172.21.15.16:0/4209747275 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f7648021070 con 0x7f765c09e1d0 2023-09-30T22:34:18.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.586+0000 7f76597fa700 1 -- 172.21.15.16:0/4209747275 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7648010380 con 0x7f765c09e1d0 2023-09-30T22:34:18.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.625+0000 7f5bb7ca4700 1 Processor -- start 2023-09-30T22:34:18.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.625+0000 7f5bb7ca4700 1 -- start start 2023-09-30T22:34:18.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.626+0000 7f5bb7ca4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5bb009d6f0 0x7f5bb009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:18.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.626+0000 7f5bb7ca4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5bb009e000 0x7f5bb0130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:18.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.626+0000 7f5bb7ca4700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5bb009e410 0x7f5bb0130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:18.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.626+0000 7f5bb7ca4700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bb009f520 con 0x7f5bb009d6f0 2023-09-30T22:34:18.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.626+0000 7f5bb7ca4700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb009f130 con 0x7f5bb009e410 2023-09-30T22:34:18.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.626+0000 7f5bb7ca4700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb009e930 con 0x7f5bb009e000 2023-09-30T22:34:18.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.626+0000 7f5bb6241700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5bb009e410 0x7f5bb0130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:18.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.626+0000 7f5bb523f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5bb009e000 0x7f5bb0130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:18.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.626+0000 7f5bb6241700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5bb009e410 0x7f5bb0130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:55298/0 (socket says 172.21.15.16:55298) 2023-09-30T22:34:18.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.626+0000 7f5bb6241700 1 -- 172.21.15.16:0/1789474461 learned_addr learned my addr 172.21.15.16:0/1789474461 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:18.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.626+0000 7f5bb5a40700 1 --2- 172.21.15.16:0/1789474461 >> v2:172.21.15.16:3300/0 conn(0x7f5bb009d6f0 0x7f5bb009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:18.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.626+0000 7f5bb523f700 1 -- 172.21.15.16:0/1789474461 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5bb009e410 msgr2=0x7f5bb0130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:18.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.626+0000 7f5bb523f700 1 --2- 172.21.15.16:0/1789474461 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5bb009e410 0x7f5bb0130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:18.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.626+0000 7f5bb523f700 1 -- 172.21.15.16:0/1789474461 >> v2:172.21.15.16:3300/0 conn(0x7f5bb009d6f0 msgr2=0x7f5bb009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:18.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.627+0000 7f5bb523f700 1 --2- 172.21.15.16:0/1789474461 >> v2:172.21.15.16:3300/0 conn(0x7f5bb009d6f0 0x7f5bb009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:18.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.627+0000 7f5bb523f700 1 -- 172.21.15.16:0/1789474461 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bb009b2d0 con 0x7f5bb009e000 2023-09-30T22:34:18.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.627+0000 7f5bb5a40700 1 --2- 172.21.15.16:0/1789474461 >> v2:172.21.15.16:3300/0 conn(0x7f5bb009d6f0 0x7f5bb009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:18.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.627+0000 7f5bb523f700 1 --2- 172.21.15.16:0/1789474461 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5bb009e000 0x7f5bb0130660 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f5ba0009870 tx=0x7f5ba0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfb189efe2eb14e8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:18.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.627+0000 7f5bb4a3e700 1 -- 172.21.15.16:0/1789474461 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba0011830 con 0x7f5bb009e000 2023-09-30T22:34:18.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.627+0000 7f5bb4a3e700 1 -- 172.21.15.16:0/1789474461 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ba0011990 con 0x7f5bb009e000 2023-09-30T22:34:18.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.627+0000 7f5bb4a3e700 1 -- 172.21.15.16:0/1789474461 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba0011d20 con 0x7f5bb009e000 2023-09-30T22:34:18.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.628+0000 7f5bb7ca4700 1 -- 172.21.15.16:0/1789474461 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5bb009e000 msgr2=0x7f5bb0130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:18.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.628+0000 7f5bb7ca4700 1 --2- 172.21.15.16:0/1789474461 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5bb009e000 0x7f5bb0130660 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f5ba0009870 tx=0x7f5ba0009b80 comp rx=0 tx=0).stop 2023-09-30T22:34:18.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.628+0000 7f5bb7ca4700 1 -- 172.21.15.16:0/1789474461 shutdown_connections 2023-09-30T22:34:18.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.628+0000 7f5bb7ca4700 1 --2- 172.21.15.16:0/1789474461 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5bb009e410 0x7f5bb0130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:18.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.628+0000 7f5bb7ca4700 1 --2- 172.21.15.16:0/1789474461 >> v2:172.21.15.16:3300/0 conn(0x7f5bb009d6f0 0x7f5bb009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:18.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.628+0000 7f5bb7ca4700 1 --2- 172.21.15.16:0/1789474461 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5bb009e000 0x7f5bb0130660 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:18.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.628+0000 7f5bb7ca4700 1 -- 172.21.15.16:0/1789474461 >> 172.21.15.16:0/1789474461 conn(0x7f5bb0066f50 msgr2=0x7f5bb009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:18.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.628+0000 7f5bb7ca4700 1 -- 172.21.15.16:0/1789474461 shutdown_connections 2023-09-30T22:34:18.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.628+0000 7f5bb7ca4700 1 -- 172.21.15.16:0/1789474461 wait complete. 2023-09-30T22:34:18.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.629+0000 7f5bb7ca4700 1 Processor -- start 2023-09-30T22:34:18.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.630+0000 7f5bb7ca4700 1 -- start start 2023-09-30T22:34:18.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.630+0000 7f5bb7ca4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bb009d6f0 0x7f5bb0137920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:18.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.630+0000 7f5bb7ca4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5bb009e000 0x7f5bb0137e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:18.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.630+0000 7f5bb7ca4700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5bb009e410 0x7f5bb01383a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:18.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.630+0000 7f5bb7ca4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb0138f60 con 0x7f5bb009d6f0 2023-09-30T22:34:18.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.630+0000 7f5bb7ca4700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb0130f60 con 0x7f5bb009e410 2023-09-30T22:34:18.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.630+0000 7f5bb7ca4700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb011e2e0 con 0x7f5bb009e000 2023-09-30T22:34:18.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.630+0000 7f5bb6241700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5bb009e410 0x7f5bb01383a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:18.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.630+0000 7f5bb6241700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5bb009e410 0x7f5bb01383a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:55306/0 (socket says 172.21.15.16:55306) 2023-09-30T22:34:18.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.630+0000 7f5bb6241700 1 -- 172.21.15.16:0/2999760768 learned_addr learned my addr 172.21.15.16:0/2999760768 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:18.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.630+0000 7f5bb523f700 1 --2- 172.21.15.16:0/2999760768 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5bb009e000 0x7f5bb0137e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:18.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.631+0000 7f5bb5a40700 1 --2- 172.21.15.16:0/2999760768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bb009d6f0 0x7f5bb0137920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:18.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.631+0000 7f5bb6241700 1 -- 172.21.15.16:0/2999760768 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5bb009e000 msgr2=0x7f5bb0137e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:18.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.631+0000 7f5bb6241700 1 --2- 172.21.15.16:0/2999760768 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5bb009e000 0x7f5bb0137e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:18.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.631+0000 7f5bb6241700 1 -- 172.21.15.16:0/2999760768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bb009d6f0 msgr2=0x7f5bb0137920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:18.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.631+0000 7f5bb6241700 1 --2- 172.21.15.16:0/2999760768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bb009d6f0 0x7f5bb0137920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:18.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.631+0000 7f5bb6241700 1 -- 172.21.15.16:0/2999760768 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bac0094a0 con 0x7f5bb009e410 2023-09-30T22:34:18.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.631+0000 7f5bb6241700 1 --2- 172.21.15.16:0/2999760768 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5bb009e410 0x7f5bb01383a0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f5ba800efc0 tx=0x7f5ba800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:18.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.631+0000 7f5ba67fc700 1 -- 172.21.15.16:0/2999760768 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba8003c20 con 0x7f5bb009e410 2023-09-30T22:34:18.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.631+0000 7f5bb7ca4700 1 -- 172.21.15.16:0/2999760768 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ba00094a0 con 0x7f5bb009e410 2023-09-30T22:34:18.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.631+0000 7f5bb7ca4700 1 -- 172.21.15.16:0/2999760768 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bb01391b0 con 0x7f5bb009e410 2023-09-30T22:34:18.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.632+0000 7f5ba67fc700 1 -- 172.21.15.16:0/2999760768 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ba800f040 con 0x7f5bb009e410 2023-09-30T22:34:18.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.632+0000 7f5ba67fc700 1 -- 172.21.15.16:0/2999760768 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba8009e60 con 0x7f5bb009e410 2023-09-30T22:34:18.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.633+0000 7f5bb7ca4700 1 -- 172.21.15.16:0/2999760768 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b980052e0 con 0x7f5bb009e410 2023-09-30T22:34:18.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.634+0000 7f5ba67fc700 1 -- 172.21.15.16:0/2999760768 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5ba800c880 con 0x7f5bb009e410 2023-09-30T22:34:18.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.635+0000 7f5ba67fc700 1 -- 172.21.15.16:0/2999760768 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f5ba8026d60 con 0x7f5bb009e410 2023-09-30T22:34:18.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.643+0000 7f5ba67fc700 1 -- 172.21.15.16:0/2999760768 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5ba80133e0 con 0x7f5bb009e410 2023-09-30T22:34:18.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.653+0000 7fd4e774b700 1 Processor -- start 2023-09-30T22:34:18.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.653+0000 7fd4e774b700 1 -- start start 2023-09-30T22:34:18.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.654+0000 7fd4e774b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd4e0130540 0x7fd4e0130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:18.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.654+0000 7fd4e774b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4e006fcf0 0x7fd4e0138570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:18.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.654+0000 7fd4e774b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4e00716a0 con 0x7fd4e0130e50 2023-09-30T22:34:18.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.654+0000 7fd4e774b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4e009a390 con 0x7fd4e0130540 2023-09-30T22:34:18.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.654+0000 7fd4e774b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e0071290 con 0x7fd4e006fcf0 2023-09-30T22:34:18.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.654+0000 7fd4e54e7700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd4e0130540 0x7fd4e0130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:18.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.654+0000 7fd4e54e7700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd4e0130540 0x7fd4e0130910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47418/0 (socket says 172.21.15.16:47418) 2023-09-30T22:34:18.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.654+0000 7fd4e54e7700 1 -- 172.21.15.16:0/3503913242 learned_addr learned my addr 172.21.15.16:0/3503913242 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:18.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.654+0000 7fd4e5ce8700 1 --2- 172.21.15.16:0/3503913242 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4e006fcf0 0x7fd4e0138570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:18.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.654+0000 7fd4e5ce8700 1 -- 172.21.15.16:0/3503913242 >> v2:172.21.15.16:3300/0 conn(0x7fd4e0130540 msgr2=0x7fd4e0130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:18.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.654+0000 7fd4e5ce8700 1 --2- 172.21.15.16:0/3503913242 >> v2:172.21.15.16:3300/0 conn(0x7fd4e0130540 0x7fd4e0130910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:18.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.654+0000 7fd4e5ce8700 1 -- 172.21.15.16:0/3503913242 >> v1:172.21.15.16:6789/0 conn(0x7fd4e0130e50 legacy=0x7fd4e006f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:18.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.654+0000 7fd4e5ce8700 1 -- 172.21.15.16:0/3503913242 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e0130340 con 0x7fd4e006fcf0 2023-09-30T22:34:18.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.654+0000 7fd4e54e7700 1 --2- 172.21.15.16:0/3503913242 >> v2:172.21.15.16:3300/0 conn(0x7fd4e0130540 0x7fd4e0130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:18.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.655+0000 7fd4e5ce8700 1 --2- 172.21.15.16:0/3503913242 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4e006fcf0 0x7fd4e0138570 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8009dc0 tx=0x7fd4d800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4240f49163ee2f78 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:18.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.655+0000 7fd4d7fff700 1 -- 172.21.15.16:0/3503913242 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d800e070 con 0x7fd4e006fcf0 2023-09-30T22:34:18.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.655+0000 7fd4d7fff700 1 -- 172.21.15.16:0/3503913242 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4d800bae0 con 0x7fd4e006fcf0 2023-09-30T22:34:18.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.655+0000 7fd4d7fff700 1 -- 172.21.15.16:0/3503913242 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d800be90 con 0x7fd4e006fcf0 2023-09-30T22:34:18.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.656+0000 7fd4e774b700 1 -- 172.21.15.16:0/3503913242 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4e006fcf0 msgr2=0x7fd4e0138570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:18.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.656+0000 7fd4e774b700 1 --2- 172.21.15.16:0/3503913242 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4e006fcf0 0x7fd4e0138570 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8009dc0 tx=0x7fd4d800b330 comp rx=0 tx=0).stop 2023-09-30T22:34:18.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.656+0000 7fd4e774b700 1 -- 172.21.15.16:0/3503913242 shutdown_connections 2023-09-30T22:34:18.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.656+0000 7fd4e774b700 1 --2- 172.21.15.16:0/3503913242 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4e006fcf0 0x7fd4e0138570 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:18.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.656+0000 7fd4e774b700 1 --2- 172.21.15.16:0/3503913242 >> v2:172.21.15.16:3300/0 conn(0x7fd4e0130540 0x7fd4e0130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:18.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.656+0000 7fd4e774b700 1 -- 172.21.15.16:0/3503913242 >> 172.21.15.16:0/3503913242 conn(0x7fd4e00a2aa0 msgr2=0x7fd4e00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:18.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.656+0000 7fd4e774b700 1 -- 172.21.15.16:0/3503913242 shutdown_connections 2023-09-30T22:34:18.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.656+0000 7fd4e774b700 1 -- 172.21.15.16:0/3503913242 wait complete. 2023-09-30T22:34:18.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.657+0000 7fd4e774b700 1 Processor -- start 2023-09-30T22:34:18.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.657+0000 7fd4e774b700 1 -- start start 2023-09-30T22:34:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.657+0000 7fd4e774b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4e006fcf0 0x7fd4e011ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.657+0000 7fd4e774b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4e0130540 0x7fd4e011f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.657+0000 7fd4e774b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd4e0130e50 0x7fd4e011f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.657+0000 7fd4e774b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e01226f0 con 0x7fd4e006fcf0 2023-09-30T22:34:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.658+0000 7fd4e774b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e012c6a0 con 0x7fd4e0130540 2023-09-30T22:34:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.658+0000 7fd4e774b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e0120880 con 0x7fd4e0130e50 2023-09-30T22:34:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.658+0000 7fd4e5ce8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd4e0130e50 0x7fd4e011f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.658+0000 7fd4e5ce8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd4e0130e50 0x7fd4e011f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:47970/0 (socket says 172.21.15.16:47970) 2023-09-30T22:34:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.658+0000 7fd4e5ce8700 1 -- 172.21.15.16:0/3454872156 learned_addr learned my addr 172.21.15.16:0/3454872156 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.658+0000 7fd4e54e7700 1 --2- 172.21.15.16:0/3454872156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4e006fcf0 0x7fd4e011ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:18.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.658+0000 7fd4e4ce6700 1 --2- 172.21.15.16:0/3454872156 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4e0130540 0x7fd4e011f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:18.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.658+0000 7fd4e5ce8700 1 -- 172.21.15.16:0/3454872156 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4e0130540 msgr2=0x7fd4e011f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:18.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.658+0000 7fd4e5ce8700 1 --2- 172.21.15.16:0/3454872156 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4e0130540 0x7fd4e011f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:18.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.658+0000 7fd4e5ce8700 1 -- 172.21.15.16:0/3454872156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4e006fcf0 msgr2=0x7fd4e011ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:18.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.658+0000 7fd4e5ce8700 1 --2- 172.21.15.16:0/3454872156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4e006fcf0 0x7fd4e011ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:18.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.658+0000 7fd4e5ce8700 1 -- 172.21.15.16:0/3454872156 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4dc00b7f0 con 0x7fd4e0130e50 2023-09-30T22:34:18.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.658+0000 7fd4e54e7700 1 --2- 172.21.15.16:0/3454872156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4e006fcf0 0x7fd4e011ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:34:18.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.658+0000 7fd4e5ce8700 1 --2- 172.21.15.16:0/3454872156 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd4e0130e50 0x7fd4e011f6e0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fd4d800b370 tx=0x7fd4d8004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:18.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.659+0000 7fd4d5ffb700 1 -- 172.21.15.16:0/3454872156 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d80048b0 con 0x7fd4e0130e50 2023-09-30T22:34:18.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.659+0000 7fd4d5ffb700 1 -- 172.21.15.16:0/3454872156 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4d8020070 con 0x7fd4e0130e50 2023-09-30T22:34:18.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.659+0000 7fd4e774b700 1 -- 172.21.15.16:0/3454872156 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4d8009a70 con 0x7fd4e0130e50 2023-09-30T22:34:18.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.659+0000 7fd4d5ffb700 1 -- 172.21.15.16:0/3454872156 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d8026670 con 0x7fd4e0130e50 2023-09-30T22:34:18.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.659+0000 7fd4e774b700 1 -- 172.21.15.16:0/3454872156 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4e00718e0 con 0x7fd4e0130e50 2023-09-30T22:34:18.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.660+0000 7fd4d5ffb700 1 -- 172.21.15.16:0/3454872156 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd4d8012430 con 0x7fd4e0130e50 2023-09-30T22:34:18.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.660+0000 7fd4d5ffb700 1 -- 172.21.15.16:0/3454872156 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd4d8022070 con 0x7fd4e0130e50 2023-09-30T22:34:18.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.660+0000 7fd4e774b700 1 -- 172.21.15.16:0/3454872156 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4e0131320 con 0x7fd4e0130e50 2023-09-30T22:34:18.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.665+0000 7fd4d5ffb700 1 -- 172.21.15.16:0/3454872156 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd4d80126e0 con 0x7fd4e0130e50 2023-09-30T22:34:18.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.806+0000 7f12eb6a0700 1 -- 172.21.15.16:0/1388662499 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f12e406b9e0 con 0x7f12e41341d0 2023-09-30T22:34:18.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.806+0000 7f12d9ffb700 1 -- 172.21.15.16:0/1388662499 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f12dc02c030 con 0x7f12e41341d0 2023-09-30T22:34:18.807 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:34:18.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.810+0000 7f12eb6a0700 1 -- 172.21.15.16:0/1388662499 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12e41341d0 msgr2=0x7f12e41203a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:18.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.810+0000 7f12eb6a0700 1 --2- 172.21.15.16:0/1388662499 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12e41341d0 0x7f12e41203a0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f12dc014040 tx=0x7f12dc004ad0 comp rx=0 tx=0).stop 2023-09-30T22:34:18.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.810+0000 7f12eb6a0700 1 -- 172.21.15.16:0/1388662499 shutdown_connections 2023-09-30T22:34:18.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.810+0000 7f12eb6a0700 1 --2- 172.21.15.16:0/1388662499 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12e41341d0 0x7f12e41203a0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:18.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.810+0000 7f12eb6a0700 1 --2- 172.21.15.16:0/1388662499 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12e409e560 0x7f12e411f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:18.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.810+0000 7f12eb6a0700 1 --2- 172.21.15.16:0/1388662499 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12e41338c0 0x7f12e411fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:18.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.810+0000 7f12eb6a0700 1 -- 172.21.15.16:0/1388662499 >> 172.21.15.16:0/1388662499 conn(0x7f12e40a2aa0 msgr2=0x7f12e40a0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:18.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.810+0000 7f12eb6a0700 1 -- 172.21.15.16:0/1388662499 shutdown_connections 2023-09-30T22:34:18.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.810+0000 7f12eb6a0700 1 -- 172.21.15.16:0/1388662499 wait complete. 2023-09-30T22:34:18.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.818+0000 7f7662f90700 1 -- 172.21.15.16:0/4209747275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f763c005590 con 0x7f765c09e1d0 2023-09-30T22:34:18.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.818+0000 7f76597fa700 1 -- 172.21.15.16:0/4209747275 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7648015020 con 0x7f765c09e1d0 2023-09-30T22:34:18.819 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:34:18.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.822+0000 7f7662f90700 1 -- 172.21.15.16:0/4209747275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f765c09e1d0 msgr2=0x7f765c11c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:18.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.822+0000 7f7662f90700 1 --2- 172.21.15.16:0/4209747275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f765c09e1d0 0x7f765c11c020 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f7648000c00 tx=0x7f7648004750 comp rx=0 tx=0).stop 2023-09-30T22:34:18.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.822+0000 7f7662f90700 1 -- 172.21.15.16:0/4209747275 shutdown_connections 2023-09-30T22:34:18.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.822+0000 7f7662f90700 1 --2- 172.21.15.16:0/4209747275 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f765c09d6f0 0x7f765c11bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:18.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.822+0000 7f7662f90700 1 --2- 172.21.15.16:0/4209747275 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f765c135e50 0x7f765c1c82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:18.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.822+0000 7f7662f90700 1 --2- 172.21.15.16:0/4209747275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f765c09e1d0 0x7f765c11c020 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:18.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.822+0000 7f7662f90700 1 -- 172.21.15.16:0/4209747275 >> 172.21.15.16:0/4209747275 conn(0x7f765c066f50 msgr2=0x7f765c0a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:18.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.822+0000 7f7662f90700 1 -- 172.21.15.16:0/4209747275 shutdown_connections 2023-09-30T22:34:18.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.822+0000 7f7662f90700 1 -- 172.21.15.16:0/4209747275 wait complete. 2023-09-30T22:34:18.831 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:34:18.841 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:34:18.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.897+0000 7f5bb7ca4700 1 -- 172.21.15.16:0/2999760768 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5b98005860 con 0x7f5bb009e410 2023-09-30T22:34:18.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.897+0000 7f5ba67fc700 1 -- 172.21.15.16:0/2999760768 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5ba8014e00 con 0x7f5bb009e410 2023-09-30T22:34:18.898 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:34:18.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.900+0000 7f5bb7ca4700 1 -- 172.21.15.16:0/2999760768 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5bb009e410 msgr2=0x7f5bb01383a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:18.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.900+0000 7f5bb7ca4700 1 --2- 172.21.15.16:0/2999760768 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5bb009e410 0x7f5bb01383a0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f5ba800efc0 tx=0x7f5ba800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:34:18.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.900+0000 7f5bb7ca4700 1 -- 172.21.15.16:0/2999760768 shutdown_connections 2023-09-30T22:34:18.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.900+0000 7f5bb7ca4700 1 --2- 172.21.15.16:0/2999760768 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5bb009e410 0x7f5bb01383a0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:18.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.900+0000 7f5bb7ca4700 1 --2- 172.21.15.16:0/2999760768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5bb009d6f0 0x7f5bb0137920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:18.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.900+0000 7f5bb7ca4700 1 --2- 172.21.15.16:0/2999760768 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5bb009e000 0x7f5bb0137e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:18.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.900+0000 7f5bb7ca4700 1 -- 172.21.15.16:0/2999760768 >> 172.21.15.16:0/2999760768 conn(0x7f5bb0066f50 msgr2=0x7f5bb006a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:18.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.900+0000 7f5bb7ca4700 1 -- 172.21.15.16:0/2999760768 shutdown_connections 2023-09-30T22:34:18.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.900+0000 7f5bb7ca4700 1 -- 172.21.15.16:0/2999760768 wait complete. 2023-09-30T22:34:18.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.901+0000 7fd4e774b700 1 -- 172.21.15.16:0/3454872156 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd4e0074ea0 con 0x7fd4e0130e50 2023-09-30T22:34:18.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.902+0000 7fd4d5ffb700 1 -- 172.21.15.16:0/3454872156 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd4d802e390 con 0x7fd4e0130e50 2023-09-30T22:34:18.903 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:34:18.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.904+0000 7fd4e774b700 1 -- 172.21.15.16:0/3454872156 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd4e0130e50 msgr2=0x7fd4e011f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:18.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.904+0000 7fd4e774b700 1 --2- 172.21.15.16:0/3454872156 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd4e0130e50 0x7fd4e011f6e0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fd4d800b370 tx=0x7fd4d8004ad0 comp rx=0 tx=0).stop 2023-09-30T22:34:18.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.904+0000 7fd4e774b700 1 -- 172.21.15.16:0/3454872156 shutdown_connections 2023-09-30T22:34:18.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.904+0000 7fd4e774b700 1 --2- 172.21.15.16:0/3454872156 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd4e0130e50 0x7fd4e011f6e0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:18.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.904+0000 7fd4e774b700 1 --2- 172.21.15.16:0/3454872156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4e006fcf0 0x7fd4e011ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:18.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.904+0000 7fd4e774b700 1 --2- 172.21.15.16:0/3454872156 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4e0130540 0x7fd4e011f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:18.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.904+0000 7fd4e774b700 1 -- 172.21.15.16:0/3454872156 >> 172.21.15.16:0/3454872156 conn(0x7fd4e00a2aa0 msgr2=0x7fd4e012d550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:18.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.904+0000 7fd4e774b700 1 -- 172.21.15.16:0/3454872156 shutdown_connections 2023-09-30T22:34:18.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:18.904+0000 7fd4e774b700 1 -- 172.21.15.16:0/3454872156 wait complete. 2023-09-30T22:34:18.916 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:34:18.919 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:34:19.832 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:34:19.842 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:34:19.917 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:34:19.920 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:34:19.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:19.995+0000 7fb4291b5700 1 Processor -- start 2023-09-30T22:34:19.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:19.995+0000 7fb4291b5700 1 -- start start 2023-09-30T22:34:19.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:19.996+0000 7fb4291b5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb42409e1d0 0x7fb424135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:19.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:19.996+0000 7fb4291b5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb424135e50 0x7fb424138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:19.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:19.996+0000 7fb4291b5700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb42409f130 con 0x7fb42409d6f0 2023-09-30T22:34:19.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:19.996+0000 7fb4291b5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb42409f520 con 0x7fb424135e50 2023-09-30T22:34:19.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:19.996+0000 7fb4291b5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb42409e930 con 0x7fb42409e1d0 2023-09-30T22:34:19.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:19.996+0000 7fb422d9d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb42409d6f0 0x7fb42409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54858/0 (socket says 172.21.15.16:54858) 2023-09-30T22:34:19.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:19.996+0000 7fb422d9d700 1 -- 172.21.15.16:0/1795946845 learned_addr learned my addr 172.21.15.16:0/1795946845 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:34:19.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:19.996+0000 7fb42259c700 1 --2- 172.21.15.16:0/1795946845 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb42409e1d0 0x7fb424135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:19.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:19.996+0000 7fb42359e700 1 --2- 172.21.15.16:0/1795946845 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb424135e50 0x7fb424138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:20.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:19.997+0000 7fb421d9b700 1 -- 172.21.15.16:0/1795946845 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3433454823 0 0) 0x7fb42409f130 con 0x7fb42409d6f0 2023-09-30T22:34:20.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:19.997+0000 7fb421d9b700 1 -- 172.21.15.16:0/1795946845 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb40c0035b0 con 0x7fb42409d6f0 2023-09-30T22:34:20.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:19.997+0000 7fb42259c700 1 -- 172.21.15.16:0/1795946845 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb424135e50 msgr2=0x7fb424138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:20.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:19.997+0000 7fb42259c700 1 --2- 172.21.15.16:0/1795946845 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb424135e50 0x7fb424138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:20.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:19.997+0000 7fb42259c700 1 -- 172.21.15.16:0/1795946845 >> v1:172.21.15.16:6789/0 conn(0x7fb42409d6f0 legacy=0x7fb42409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:20.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:19.997+0000 7fb42259c700 1 -- 172.21.15.16:0/1795946845 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb42409b2d0 con 0x7fb42409e1d0 2023-09-30T22:34:20.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:19.997+0000 7fb42359e700 1 --2- 172.21.15.16:0/1795946845 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb424135e50 0x7fb424138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:20.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:19.997+0000 7fb42259c700 1 --2- 172.21.15.16:0/1795946845 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb42409e1d0 0x7fb424135910 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fb418009870 tx=0x7fb418009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ba69d45c0a6b370 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:20.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:19.997+0000 7fb421d9b700 1 -- 172.21.15.16:0/1795946845 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb418011830 con 0x7fb42409e1d0 2023-09-30T22:34:20.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:19.997+0000 7fb421d9b700 1 -- 172.21.15.16:0/1795946845 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb418011990 con 0x7fb42409e1d0 2023-09-30T22:34:20.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:19.997+0000 7fb421d9b700 1 -- 172.21.15.16:0/1795946845 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb418011d00 con 0x7fb42409e1d0 2023-09-30T22:34:20.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:19.998+0000 7fb4291b5700 1 -- 172.21.15.16:0/1795946845 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb42409e1d0 msgr2=0x7fb424135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:20.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:19.998+0000 7fb4291b5700 1 --2- 172.21.15.16:0/1795946845 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb42409e1d0 0x7fb424135910 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fb418009870 tx=0x7fb418009b80 comp rx=0 tx=0).stop 2023-09-30T22:34:20.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:19.999+0000 7fb4291b5700 1 -- 172.21.15.16:0/1795946845 shutdown_connections 2023-09-30T22:34:20.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:19.999+0000 7fb4291b5700 1 --2- 172.21.15.16:0/1795946845 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb424135e50 0x7fb424138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:20.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:19.999+0000 7fb4291b5700 1 --2- 172.21.15.16:0/1795946845 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb42409e1d0 0x7fb424135910 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:20.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:19.999+0000 7fb4291b5700 1 -- 172.21.15.16:0/1795946845 >> 172.21.15.16:0/1795946845 conn(0x7fb424066f50 msgr2=0x7fb42409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:20.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:19.999+0000 7fb4291b5700 1 -- 172.21.15.16:0/1795946845 shutdown_connections 2023-09-30T22:34:20.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:19.999+0000 7fb4291b5700 1 -- 172.21.15.16:0/1795946845 wait complete. 2023-09-30T22:34:20.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.000+0000 7fb4291b5700 1 Processor -- start 2023-09-30T22:34:20.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.000+0000 7fb4291b5700 1 -- start start 2023-09-30T22:34:20.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.001+0000 7fb4291b5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb42409d6f0 0x7fb4241397b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:20.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.001+0000 7fb4291b5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb42409e1d0 0x7fb424139cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:20.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.001+0000 7fb4291b5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb424135e50 0x7fb42413a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:20.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.001+0000 7fb4291b5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb42411fdc0 con 0x7fb42409d6f0 2023-09-30T22:34:20.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.001+0000 7fb4291b5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4241211f0 con 0x7fb424135e50 2023-09-30T22:34:20.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.001+0000 7fb4291b5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb424120980 con 0x7fb42409e1d0 2023-09-30T22:34:20.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.001+0000 7fb42259c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb42409e1d0 0x7fb424139cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:20.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.001+0000 7fb42259c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb42409e1d0 0x7fb424139cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:47994/0 (socket says 172.21.15.16:47994) 2023-09-30T22:34:20.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.001+0000 7fb42259c700 1 -- 172.21.15.16:0/1467486801 learned_addr learned my addr 172.21.15.16:0/1467486801 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:20.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.001+0000 7fb422d9d700 1 --2- 172.21.15.16:0/1467486801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb42409d6f0 0x7fb4241397b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:20.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.001+0000 7fb42359e700 1 --2- 172.21.15.16:0/1467486801 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb424135e50 0x7fb42413a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:20.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.001+0000 7fb42259c700 1 -- 172.21.15.16:0/1467486801 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb424135e50 msgr2=0x7fb42413a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:20.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.001+0000 7fb42259c700 1 --2- 172.21.15.16:0/1467486801 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb424135e50 0x7fb42413a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:20.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.001+0000 7fb42259c700 1 -- 172.21.15.16:0/1467486801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb42409d6f0 msgr2=0x7fb4241397b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:20.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.001+0000 7fb42259c700 1 --2- 172.21.15.16:0/1467486801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb42409d6f0 0x7fb4241397b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:20.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.001+0000 7fb42259c700 1 -- 172.21.15.16:0/1467486801 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb41c009a70 con 0x7fb42409e1d0 2023-09-30T22:34:20.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.002+0000 7fb42259c700 1 --2- 172.21.15.16:0/1467486801 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb42409e1d0 0x7fb424139cf0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb4180090b0 tx=0x7fb418004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:20.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.002+0000 7fb40b7fe700 1 -- 172.21.15.16:0/1467486801 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb418004450 con 0x7fb42409e1d0 2023-09-30T22:34:20.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.002+0000 7fb40b7fe700 1 -- 172.21.15.16:0/1467486801 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb41801f070 con 0x7fb42409e1d0 2023-09-30T22:34:20.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.002+0000 7fb4291b5700 1 -- 172.21.15.16:0/1467486801 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4180094a0 con 0x7fb42409e1d0 2023-09-30T22:34:20.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.002+0000 7fb40b7fe700 1 -- 172.21.15.16:0/1467486801 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4180038e0 con 0x7fb42409e1d0 2023-09-30T22:34:20.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.002+0000 7fb4291b5700 1 -- 172.21.15.16:0/1467486801 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb424120580 con 0x7fb42409e1d0 2023-09-30T22:34:20.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.003+0000 7fb40b7fe700 1 -- 172.21.15.16:0/1467486801 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb418003a40 con 0x7fb42409e1d0 2023-09-30T22:34:20.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.003+0000 7fb40b7fe700 1 -- 172.21.15.16:0/1467486801 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fb418021070 con 0x7fb42409e1d0 2023-09-30T22:34:20.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.003+0000 7fb4291b5700 1 -- 172.21.15.16:0/1467486801 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb424071700 con 0x7fb42409e1d0 2023-09-30T22:34:20.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.009+0000 7fb40b7fe700 1 -- 172.21.15.16:0/1467486801 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb418024380 con 0x7fb42409e1d0 2023-09-30T22:34:20.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.028+0000 7f1c049fd700 1 Processor -- start 2023-09-30T22:34:20.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.028+0000 7f1c049fd700 1 -- start start 2023-09-30T22:34:20.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.029+0000 7f1c049fd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1c0012cf60 0x7f1c0006f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:20.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.029+0000 7f1c049fd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1c0012f8b0 0x7f1c0012fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:20.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.029+0000 7f1c049fd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1c00130260 0x7f1c00136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:20.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.029+0000 7f1c049fd700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c0012d330 con 0x7f1c00130260 2023-09-30T22:34:20.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.029+0000 7f1c049fd700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0012cd90 con 0x7f1c0012f8b0 2023-09-30T22:34:20.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.029+0000 7f1c049fd700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1c001306d0 con 0x7f1c0012cf60 2023-09-30T22:34:20.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.029+0000 7f1bfdd9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1c0012f8b0 0x7f1c0012fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:20.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.029+0000 7f1bfed9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1c00130260 0x7f1c00136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:20.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.029+0000 7f1bfe59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1c0012cf60 0x7f1c0006f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:20.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.029+0000 7f1bfdd9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1c0012f8b0 0x7f1c0012fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:55358/0 (socket says 172.21.15.16:55358) 2023-09-30T22:34:20.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.029+0000 7f1bfdd9b700 1 -- 172.21.15.16:0/2280599377 learned_addr learned my addr 172.21.15.16:0/2280599377 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:20.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.029+0000 7f1bfed9d700 1 -- 172.21.15.16:0/2280599377 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1c0012cf60 msgr2=0x7f1c0006f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:20.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.029+0000 7f1bfed9d700 1 --2- 172.21.15.16:0/2280599377 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1c0012cf60 0x7f1c0006f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:20.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.029+0000 7f1bfed9d700 1 -- 172.21.15.16:0/2280599377 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1c0012f8b0 msgr2=0x7f1c0012fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:20.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.029+0000 7f1bfed9d700 1 --2- 172.21.15.16:0/2280599377 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1c0012f8b0 0x7f1c0012fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:20.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.029+0000 7f1bfed9d700 1 -- 172.21.15.16:0/2280599377 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c00071780 con 0x7f1c00130260 2023-09-30T22:34:20.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.029+0000 7f1bfdd9b700 1 --2- 172.21.15.16:0/2280599377 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1c0012f8b0 0x7f1c0012fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:20.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.029+0000 7f1bfe59c700 1 --2- 172.21.15.16:0/2280599377 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1c0012cf60 0x7f1c0006f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:20.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.030+0000 7f1bfed9d700 1 --2- 172.21.15.16:0/2280599377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c00130260 0x7f1c00136c20 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8009d90 tx=0x7f1bf800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5d77dccd5ad0bf9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:20.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.030+0000 7f1bfd59a700 1 -- 172.21.15.16:0/2280599377 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf800e070 con 0x7f1c00130260 2023-09-30T22:34:20.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.030+0000 7f1bfd59a700 1 -- 172.21.15.16:0/2280599377 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bf800bb20 con 0x7f1c00130260 2023-09-30T22:34:20.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.030+0000 7f1bfd59a700 1 -- 172.21.15.16:0/2280599377 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf8012430 con 0x7f1c00130260 2023-09-30T22:34:20.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.031+0000 7f1c049fd700 1 -- 172.21.15.16:0/2280599377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c00130260 msgr2=0x7f1c00136c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:20.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.031+0000 7f1c049fd700 1 --2- 172.21.15.16:0/2280599377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c00130260 0x7f1c00136c20 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8009d90 tx=0x7f1bf800b330 comp rx=0 tx=0).stop 2023-09-30T22:34:20.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.031+0000 7f1c049fd700 1 -- 172.21.15.16:0/2280599377 shutdown_connections 2023-09-30T22:34:20.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.031+0000 7f1c049fd700 1 --2- 172.21.15.16:0/2280599377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c00130260 0x7f1c00136c20 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:20.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.031+0000 7f1c049fd700 1 --2- 172.21.15.16:0/2280599377 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1c0012cf60 0x7f1c0006f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:20.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.031+0000 7f1c049fd700 1 --2- 172.21.15.16:0/2280599377 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1c0012f8b0 0x7f1c0012fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:20.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.031+0000 7f1c049fd700 1 -- 172.21.15.16:0/2280599377 >> 172.21.15.16:0/2280599377 conn(0x7f1c000a2aa0 msgr2=0x7f1c000a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:20.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.031+0000 7f1c049fd700 1 -- 172.21.15.16:0/2280599377 shutdown_connections 2023-09-30T22:34:20.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.031+0000 7f1c049fd700 1 -- 172.21.15.16:0/2280599377 wait complete. 2023-09-30T22:34:20.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.032+0000 7f1c049fd700 1 Processor -- start 2023-09-30T22:34:20.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.032+0000 7f1c049fd700 1 -- start start 2023-09-30T22:34:20.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.033+0000 7f1c049fd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1c0012cf60 0x7f1c001c3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:20.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.033+0000 7f1c049fd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1c0012f8b0 0x7f1c001c3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:20.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.033+0000 7f1c049fd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c00130260 0x7f1c001c6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:20.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.033+0000 7f1c049fd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0011dd60 con 0x7f1c00130260 2023-09-30T22:34:20.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.033+0000 7f1c049fd700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0011df40 con 0x7f1c0012f8b0 2023-09-30T22:34:20.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.033+0000 7f1c049fd700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1c001c3aa0 con 0x7f1c0012cf60 2023-09-30T22:34:20.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.033+0000 7f1bfe59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1c0012cf60 0x7f1c001c3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:20.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.033+0000 7f1bfed9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c00130260 0x7f1c001c6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:20.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.033+0000 7f1bfe59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1c0012cf60 0x7f1c001c3020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:48010/0 (socket says 172.21.15.16:48010) 2023-09-30T22:34:20.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.033+0000 7f1bfed9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c00130260 0x7f1c001c6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47452/0 (socket says 172.21.15.16:47452) 2023-09-30T22:34:20.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.033+0000 7f1bfe59c700 1 -- 172.21.15.16:0/1550047724 learned_addr learned my addr 172.21.15.16:0/1550047724 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:20.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.033+0000 7f1bfdd9b700 1 --2- 172.21.15.16:0/1550047724 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1c0012f8b0 0x7f1c001c3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:20.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.033+0000 7f1bfed9d700 1 -- 172.21.15.16:0/1550047724 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1c0012cf60 msgr2=0x7f1c001c3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:20.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.033+0000 7f1bfed9d700 1 --2- 172.21.15.16:0/1550047724 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1c0012cf60 0x7f1c001c3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:20.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.033+0000 7f1bfed9d700 1 -- 172.21.15.16:0/1550047724 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1c0012f8b0 msgr2=0x7f1c001c3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:20.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.033+0000 7f1bfed9d700 1 --2- 172.21.15.16:0/1550047724 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1c0012f8b0 0x7f1c001c3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:20.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.033+0000 7f1bfed9d700 1 -- 172.21.15.16:0/1550047724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bf00094a0 con 0x7f1c00130260 2023-09-30T22:34:20.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.034+0000 7f1bfed9d700 1 --2- 172.21.15.16:0/1550047724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c00130260 0x7f1c001c6040 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8016000 tx=0x7f1bf8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:20.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.034+0000 7f1beeffd700 1 -- 172.21.15.16:0/1550047724 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf8012430 con 0x7f1c00130260 2023-09-30T22:34:20.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.034+0000 7f1beeffd700 1 -- 172.21.15.16:0/1550047724 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bf8020070 con 0x7f1c00130260 2023-09-30T22:34:20.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.034+0000 7f1c049fd700 1 -- 172.21.15.16:0/1550047724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bf400a6f0 con 0x7f1c00130260 2023-09-30T22:34:20.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.034+0000 7f1beeffd700 1 -- 172.21.15.16:0/1550047724 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf8004280 con 0x7f1c00130260 2023-09-30T22:34:20.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.034+0000 7f1c049fd700 1 -- 172.21.15.16:0/1550047724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bf8009a70 con 0x7f1c00130260 2023-09-30T22:34:20.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.035+0000 7f1beeffd700 1 -- 172.21.15.16:0/1550047724 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1bf80043e0 con 0x7f1c00130260 2023-09-30T22:34:20.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.035+0000 7f1c049fd700 1 -- 172.21.15.16:0/1550047724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c00074ea0 con 0x7f1c00130260 2023-09-30T22:34:20.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.035+0000 7f1beeffd700 1 -- 172.21.15.16:0/1550047724 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f1bf8022070 con 0x7f1c00130260 2023-09-30T22:34:20.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.044+0000 7f1beeffd700 1 -- 172.21.15.16:0/1550047724 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1bf801c780 con 0x7f1c00130260 2023-09-30T22:34:20.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.074+0000 7ff8f4016700 1 Processor -- start 2023-09-30T22:34:20.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.074+0000 7ff8f4016700 1 -- start start 2023-09-30T22:34:20.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.074+0000 7ff8f4016700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff8ec09e930 0x7ff8ec09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:20.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.074+0000 7ff8f4016700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff8ec09f650 0x7ff8ec131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:20.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.074+0000 7ff8f4016700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8ec070940 con 0x7ff8ec09f240 2023-09-30T22:34:20.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.074+0000 7ff8f4016700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8ec06fd60 con 0x7ff8ec09e930 2023-09-30T22:34:20.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.074+0000 7ff8f4016700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ec06fa80 con 0x7ff8ec09f650 2023-09-30T22:34:20.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.075+0000 7ff8f25b3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff8ec09f650 0x7ff8ec131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:20.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.075+0000 7ff8f25b3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff8ec09f650 0x7ff8ec131f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:48024/0 (socket says 172.21.15.16:48024) 2023-09-30T22:34:20.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.075+0000 7ff8f25b3700 1 -- 172.21.15.16:0/35485040 learned_addr learned my addr 172.21.15.16:0/35485040 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:20.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.075+0000 7ff8f1db2700 1 --2- 172.21.15.16:0/35485040 >> v2:172.21.15.16:3300/0 conn(0x7ff8ec09e930 0x7ff8ec09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:20.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.075+0000 7ff8f25b3700 1 -- 172.21.15.16:0/35485040 >> v2:172.21.15.16:3300/0 conn(0x7ff8ec09e930 msgr2=0x7ff8ec09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:20.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.075+0000 7ff8f25b3700 1 --2- 172.21.15.16:0/35485040 >> v2:172.21.15.16:3300/0 conn(0x7ff8ec09e930 0x7ff8ec09ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:20.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.075+0000 7ff8f25b3700 1 -- 172.21.15.16:0/35485040 >> v1:172.21.15.16:6789/0 conn(0x7ff8ec09f240 legacy=0x7ff8ec1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:20.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.075+0000 7ff8f0db0700 1 -- 172.21.15.16:0/35485040 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 861540918 0 0) 0x7ff8ec070940 con 0x7ff8ec09f240 2023-09-30T22:34:20.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.075+0000 7ff8f25b3700 1 -- 172.21.15.16:0/35485040 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8ec09fb60 con 0x7ff8ec09f650 2023-09-30T22:34:20.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.075+0000 7ff8f25b3700 1 --2- 172.21.15.16:0/35485040 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff8ec09f650 0x7ff8ec131f00 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4009e50 tx=0x7ff8e400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18acc2dc09a7428e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:20.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.076+0000 7ff8f0db0700 1 -- 172.21.15.16:0/35485040 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e400e070 con 0x7ff8ec09f650 2023-09-30T22:34:20.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.076+0000 7ff8f0db0700 1 -- 172.21.15.16:0/35485040 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8e400b910 con 0x7ff8ec09f650 2023-09-30T22:34:20.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.076+0000 7ff8f0db0700 1 -- 172.21.15.16:0/35485040 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e400bcc0 con 0x7ff8ec09f650 2023-09-30T22:34:20.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.076+0000 7ff8f4016700 1 -- 172.21.15.16:0/35485040 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff8ec09f650 msgr2=0x7ff8ec131f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:20.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.076+0000 7ff8f4016700 1 --2- 172.21.15.16:0/35485040 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff8ec09f650 0x7ff8ec131f00 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4009e50 tx=0x7ff8e400b330 comp rx=0 tx=0).stop 2023-09-30T22:34:20.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.076+0000 7ff8f4016700 1 -- 172.21.15.16:0/35485040 shutdown_connections 2023-09-30T22:34:20.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.076+0000 7ff8f4016700 1 --2- 172.21.15.16:0/35485040 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff8ec09f650 0x7ff8ec131f00 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:20.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.076+0000 7ff8f4016700 1 --2- 172.21.15.16:0/35485040 >> v2:172.21.15.16:3300/0 conn(0x7ff8ec09e930 0x7ff8ec09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:20.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.076+0000 7ff8f4016700 1 -- 172.21.15.16:0/35485040 >> 172.21.15.16:0/35485040 conn(0x7ff8ec0a4210 msgr2=0x7ff8ec0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:20.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.077+0000 7ff8f4016700 1 -- 172.21.15.16:0/35485040 shutdown_connections 2023-09-30T22:34:20.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.077+0000 7ff8f4016700 1 -- 172.21.15.16:0/35485040 wait complete. 2023-09-30T22:34:20.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.078+0000 7ff8f4016700 1 Processor -- start 2023-09-30T22:34:20.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.078+0000 7ff8f4016700 1 -- start start 2023-09-30T22:34:20.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.078+0000 7f5529eb5700 1 Processor -- start 2023-09-30T22:34:20.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.078+0000 7f5529eb5700 1 -- start start 2023-09-30T22:34:20.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.079+0000 7ff8f4016700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff8ec09e930 0x7ff8ec1a3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:20.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.079+0000 7ff8f4016700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8ec09f240 0x7ff8ec1a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:20.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.079+0000 7ff8f4016700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff8ec09f650 0x7ff8ec1c5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:20.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.079+0000 7ff8f4016700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ec1b7000 con 0x7ff8ec09f240 2023-09-30T22:34:20.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.079+0000 7ff8f4016700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ec1a5f40 con 0x7ff8ec09e930 2023-09-30T22:34:20.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.079+0000 7ff8f4016700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff8ec1a2b70 con 0x7ff8ec09f650 2023-09-30T22:34:20.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.079+0000 7ff8f25b3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff8ec09f650 0x7ff8ec1c5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:20.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.079+0000 7ff8f25b3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff8ec09f650 0x7ff8ec1c5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:48036/0 (socket says 172.21.15.16:48036) 2023-09-30T22:34:20.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.079+0000 7ff8f25b3700 1 -- 172.21.15.16:0/653429954 learned_addr learned my addr 172.21.15.16:0/653429954 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:20.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.079+0000 7f5529eb5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55241338c0 0x7f5524133c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:20.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.079+0000 7f5529eb5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f55241341d0 0x7f552409de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:20.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.079+0000 7f5529eb5700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f552406f720 con 0x7f552409e390 2023-09-30T22:34:20.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.079+0000 7f5529eb5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5524070c80 con 0x7f55241338c0 2023-09-30T22:34:20.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.079+0000 7f5529eb5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f552406fce0 con 0x7f55241341d0 2023-09-30T22:34:20.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.079+0000 7ff8f25b3700 1 -- 172.21.15.16:0/653429954 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff8ec09e930 msgr2=0x7ff8ec1a3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:20.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.079+0000 7ff8f25b3700 1 --2- 172.21.15.16:0/653429954 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff8ec09e930 0x7ff8ec1a3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:20.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.079+0000 7ff8f25b3700 1 -- 172.21.15.16:0/653429954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8ec09f240 msgr2=0x7ff8ec1a3550 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:34:20.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.079+0000 7ff8f25b3700 1 --2- 172.21.15.16:0/653429954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8ec09f240 0x7ff8ec1a3550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:20.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.079+0000 7ff8f25b3700 1 -- 172.21.15.16:0/653429954 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e4009b30 con 0x7ff8ec09f650 2023-09-30T22:34:20.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.079+0000 7f5522ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f55241341d0 0x7f552409de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:20.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.080+0000 7f5522ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f55241341d0 0x7f552409de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:48048/0 (socket says 172.21.15.16:48048) 2023-09-30T22:34:20.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.080+0000 7f5522ffd700 1 -- 172.21.15.16:0/2911780722 learned_addr learned my addr 172.21.15.16:0/2911780722 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:20.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.080+0000 7ff8f25b3700 1 --2- 172.21.15.16:0/653429954 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff8ec09f650 0x7ff8ec1c5e40 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4004270 tx=0x7ff8e4004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:20.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.080+0000 7f5522ffd700 1 -- 172.21.15.16:0/2911780722 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55241338c0 msgr2=0x7f5524133c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:20.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.080+0000 7f5522ffd700 1 --2- 172.21.15.16:0/2911780722 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55241338c0 0x7f5524133c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:20.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.080+0000 7f5522ffd700 1 -- 172.21.15.16:0/2911780722 >> v1:172.21.15.16:6789/0 conn(0x7f552409e390 legacy=0x7f552413a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:20.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.080+0000 7f5522ffd700 1 -- 172.21.15.16:0/2911780722 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55241336a0 con 0x7f55241341d0 2023-09-30T22:34:20.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.080+0000 7ff8e27fc700 1 -- 172.21.15.16:0/653429954 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e40092e0 con 0x7ff8ec09f650 2023-09-30T22:34:20.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.080+0000 7ff8e27fc700 1 -- 172.21.15.16:0/653429954 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8e401c440 con 0x7ff8ec09f650 2023-09-30T22:34:20.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.080+0000 7ff8f4016700 1 -- 172.21.15.16:0/653429954 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8ec0a0190 con 0x7ff8ec09f650 2023-09-30T22:34:20.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.080+0000 7ff8e27fc700 1 -- 172.21.15.16:0/653429954 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e401c7f0 con 0x7ff8ec09f650 2023-09-30T22:34:20.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.080+0000 7ff8f4016700 1 -- 172.21.15.16:0/653429954 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8ec1a3af0 con 0x7ff8ec09f650 2023-09-30T22:34:20.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.081+0000 7f5522ffd700 1 --2- 172.21.15.16:0/2911780722 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f55241341d0 0x7f552409de50 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f5518009920 tx=0x7f5518009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e93447d729fcedb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:20.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.081+0000 7f55227fc700 1 -- 172.21.15.16:0/2911780722 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5518004230 con 0x7f55241341d0 2023-09-30T22:34:20.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.081+0000 7f55227fc700 1 -- 172.21.15.16:0/2911780722 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5518004390 con 0x7f55241341d0 2023-09-30T22:34:20.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.081+0000 7f55227fc700 1 -- 172.21.15.16:0/2911780722 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5518009e70 con 0x7f55241341d0 2023-09-30T22:34:20.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.081+0000 7ff8f4016700 1 -- 172.21.15.16:0/653429954 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8ec1a6080 con 0x7ff8ec09f650 2023-09-30T22:34:20.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.082+0000 7ff8e27fc700 1 -- 172.21.15.16:0/653429954 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff8e401c950 con 0x7ff8ec09f650 2023-09-30T22:34:20.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.082+0000 7ff8e27fc700 1 -- 172.21.15.16:0/653429954 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7ff8e40415b0 con 0x7ff8ec09f650 2023-09-30T22:34:20.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.082+0000 7f5529eb5700 1 -- 172.21.15.16:0/2911780722 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f55241341d0 msgr2=0x7f552409de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:20.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.082+0000 7f5529eb5700 1 --2- 172.21.15.16:0/2911780722 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f55241341d0 0x7f552409de50 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f5518009920 tx=0x7f5518009c30 comp rx=0 tx=0).stop 2023-09-30T22:34:20.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.082+0000 7f5529eb5700 1 -- 172.21.15.16:0/2911780722 shutdown_connections 2023-09-30T22:34:20.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.082+0000 7f5529eb5700 1 --2- 172.21.15.16:0/2911780722 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55241338c0 0x7f5524133c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:20.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.082+0000 7f5529eb5700 1 --2- 172.21.15.16:0/2911780722 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f55241341d0 0x7f552409de50 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:20.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.082+0000 7f5529eb5700 1 -- 172.21.15.16:0/2911780722 >> 172.21.15.16:0/2911780722 conn(0x7f55240a2aa0 msgr2=0x7f55240a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:20.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.082+0000 7f5529eb5700 1 -- 172.21.15.16:0/2911780722 shutdown_connections 2023-09-30T22:34:20.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.082+0000 7f5529eb5700 1 -- 172.21.15.16:0/2911780722 wait complete. 2023-09-30T22:34:20.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.084+0000 7f5529eb5700 1 Processor -- start 2023-09-30T22:34:20.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.084+0000 7f5529eb5700 1 -- start start 2023-09-30T22:34:20.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.084+0000 7f5529eb5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f552409e390 0x7f5524130470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:20.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.085+0000 7f5529eb5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55241338c0 0x7f55241309b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:20.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.085+0000 7f5529eb5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f55241341d0 0x7f55241cc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:20.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.085+0000 7f5529eb5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55241395d0 con 0x7f552409e390 2023-09-30T22:34:20.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.085+0000 7f5529eb5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55240a1fd0 con 0x7f55241338c0 2023-09-30T22:34:20.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.085+0000 7f5529eb5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5524139e10 con 0x7f55241341d0 2023-09-30T22:34:20.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.085+0000 7f55237fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f552409e390 0x7f5524130470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:20.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.085+0000 7f55237fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f552409e390 0x7f5524130470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47470/0 (socket says 172.21.15.16:47470) 2023-09-30T22:34:20.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.085+0000 7f55237fe700 1 -- 172.21.15.16:0/2176148434 learned_addr learned my addr 172.21.15.16:0/2176148434 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:20.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.085+0000 7f5523fff700 1 --2- 172.21.15.16:0/2176148434 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f55241341d0 0x7f55241cc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:20.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.085+0000 7f55237fe700 1 -- 172.21.15.16:0/2176148434 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f55241341d0 msgr2=0x7f55241cc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:20.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.085+0000 7f55237fe700 1 --2- 172.21.15.16:0/2176148434 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f55241341d0 0x7f55241cc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:20.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.085+0000 7f55237fe700 1 -- 172.21.15.16:0/2176148434 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55241338c0 msgr2=0x7f55241309b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:20.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.085+0000 7f55237fe700 1 --2- 172.21.15.16:0/2176148434 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55241338c0 0x7f55241309b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:20.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.085+0000 7f55237fe700 1 -- 172.21.15.16:0/2176148434 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5518009550 con 0x7f552409e390 2023-09-30T22:34:20.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.086+0000 7f55237fe700 1 --2- 172.21.15.16:0/2176148434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f552409e390 0x7f5524130470 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f551400b560 tx=0x7f551400b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:20.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.086+0000 7f550bfff700 1 -- 172.21.15.16:0/2176148434 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5514011830 con 0x7f552409e390 2023-09-30T22:34:20.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.086+0000 7f550bfff700 1 -- 172.21.15.16:0/2176148434 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5514011990 con 0x7f552409e390 2023-09-30T22:34:20.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.086+0000 7f550bfff700 1 -- 172.21.15.16:0/2176148434 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5514011d20 con 0x7f552409e390 2023-09-30T22:34:20.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.086+0000 7f5529eb5700 1 -- 172.21.15.16:0/2176148434 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f552406f960 con 0x7f552409e390 2023-09-30T22:34:20.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.086+0000 7f5529eb5700 1 -- 172.21.15.16:0/2176148434 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5524130ef0 con 0x7f552409e390 2023-09-30T22:34:20.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.087+0000 7f550bfff700 1 -- 172.21.15.16:0/2176148434 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f551401a400 con 0x7f552409e390 2023-09-30T22:34:20.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.087+0000 7f5529eb5700 1 -- 172.21.15.16:0/2176148434 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f552413a880 con 0x7f552409e390 2023-09-30T22:34:20.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.087+0000 7f550bfff700 1 -- 172.21.15.16:0/2176148434 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f551403f800 con 0x7f552409e390 2023-09-30T22:34:20.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.091+0000 7ff8e27fc700 1 -- 172.21.15.16:0/653429954 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff8e402e9b0 con 0x7ff8ec09f650 2023-09-30T22:34:20.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.092+0000 7f550bfff700 1 -- 172.21.15.16:0/2176148434 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f551400f8d0 con 0x7f552409e390 2023-09-30T22:34:20.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.229+0000 7fb4291b5700 1 -- 172.21.15.16:0/1467486801 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb424074ea0 con 0x7fb42409e1d0 2023-09-30T22:34:20.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.230+0000 7fb40b7fe700 1 -- 172.21.15.16:0/1467486801 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb41802eb90 con 0x7fb42409e1d0 2023-09-30T22:34:20.230 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:34:20.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.233+0000 7fb4291b5700 1 -- 172.21.15.16:0/1467486801 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb42409e1d0 msgr2=0x7fb424139cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:20.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.233+0000 7fb4291b5700 1 --2- 172.21.15.16:0/1467486801 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb42409e1d0 0x7fb424139cf0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fb4180090b0 tx=0x7fb418004750 comp rx=0 tx=0).stop 2023-09-30T22:34:20.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.233+0000 7fb4291b5700 1 -- 172.21.15.16:0/1467486801 shutdown_connections 2023-09-30T22:34:20.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.233+0000 7fb4291b5700 1 --2- 172.21.15.16:0/1467486801 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb424135e50 0x7fb42413a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:20.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.233+0000 7fb4291b5700 1 --2- 172.21.15.16:0/1467486801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb42409d6f0 0x7fb4241397b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:20.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.233+0000 7fb4291b5700 1 --2- 172.21.15.16:0/1467486801 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb42409e1d0 0x7fb424139cf0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:20.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.233+0000 7fb4291b5700 1 -- 172.21.15.16:0/1467486801 >> 172.21.15.16:0/1467486801 conn(0x7fb424066f50 msgr2=0x7fb42406e840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:20.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.234+0000 7fb4291b5700 1 -- 172.21.15.16:0/1467486801 shutdown_connections 2023-09-30T22:34:20.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.234+0000 7fb4291b5700 1 -- 172.21.15.16:0/1467486801 wait complete. 2023-09-30T22:34:20.253 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:34:20.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.277+0000 7f1c049fd700 1 -- 172.21.15.16:0/1550047724 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1c000713a0 con 0x7f1c00130260 2023-09-30T22:34:20.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.278+0000 7f1beeffd700 1 -- 172.21.15.16:0/1550047724 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1bf801a7f0 con 0x7f1c00130260 2023-09-30T22:34:20.278 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:34:20.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.280+0000 7f1c049fd700 1 -- 172.21.15.16:0/1550047724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c00130260 msgr2=0x7f1c001c6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:20.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.280+0000 7f1c049fd700 1 --2- 172.21.15.16:0/1550047724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c00130260 0x7f1c001c6040 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f1bf8016000 tx=0x7f1bf8004ad0 comp rx=0 tx=0).stop 2023-09-30T22:34:20.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.280+0000 7f1c049fd700 1 -- 172.21.15.16:0/1550047724 shutdown_connections 2023-09-30T22:34:20.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.280+0000 7f1c049fd700 1 --2- 172.21.15.16:0/1550047724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c00130260 0x7f1c001c6040 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:20.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.280+0000 7f1c049fd700 1 --2- 172.21.15.16:0/1550047724 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1c0012cf60 0x7f1c001c3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:20.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.280+0000 7f1c049fd700 1 --2- 172.21.15.16:0/1550047724 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1c0012f8b0 0x7f1c001c3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:20.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.280+0000 7f1c049fd700 1 -- 172.21.15.16:0/1550047724 >> 172.21.15.16:0/1550047724 conn(0x7f1c000a2aa0 msgr2=0x7f1c0009e320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:20.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.280+0000 7f1c049fd700 1 -- 172.21.15.16:0/1550047724 shutdown_connections 2023-09-30T22:34:20.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.280+0000 7f1c049fd700 1 -- 172.21.15.16:0/1550047724 wait complete. 2023-09-30T22:34:20.294 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:34:20.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.297+0000 7f5529eb5700 1 -- 172.21.15.16:0/2176148434 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5524074ea0 con 0x7f552409e390 2023-09-30T22:34:20.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.298+0000 7f550bfff700 1 -- 172.21.15.16:0/2176148434 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5514015030 con 0x7f552409e390 2023-09-30T22:34:20.298 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:34:20.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.300+0000 7f5529eb5700 1 -- 172.21.15.16:0/2176148434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f552409e390 msgr2=0x7f5524130470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:20.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.300+0000 7f5529eb5700 1 --2- 172.21.15.16:0/2176148434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f552409e390 0x7f5524130470 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f551400b560 tx=0x7f551400b920 comp rx=0 tx=0).stop 2023-09-30T22:34:20.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.300+0000 7f5529eb5700 1 -- 172.21.15.16:0/2176148434 shutdown_connections 2023-09-30T22:34:20.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.300+0000 7f5529eb5700 1 --2- 172.21.15.16:0/2176148434 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f55241341d0 0x7f55241cc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:20.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.300+0000 7f5529eb5700 1 --2- 172.21.15.16:0/2176148434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f552409e390 0x7f5524130470 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:20.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.300+0000 7f5529eb5700 1 --2- 172.21.15.16:0/2176148434 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55241338c0 0x7f55241309b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:20.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.300+0000 7f5529eb5700 1 -- 172.21.15.16:0/2176148434 >> 172.21.15.16:0/2176148434 conn(0x7f55240a2aa0 msgr2=0x7f5524132920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:20.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.300+0000 7f5529eb5700 1 -- 172.21.15.16:0/2176148434 shutdown_connections 2023-09-30T22:34:20.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.300+0000 7f5529eb5700 1 -- 172.21.15.16:0/2176148434 wait complete. 2023-09-30T22:34:20.314 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:34:20.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.326+0000 7ff8f4016700 1 -- 172.21.15.16:0/653429954 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff8ec1a5b50 con 0x7ff8ec09f650 2023-09-30T22:34:20.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.326+0000 7ff8e27fc700 1 -- 172.21.15.16:0/653429954 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff8ec1a5b50 con 0x7ff8ec09f650 2023-09-30T22:34:20.326 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:34:20.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.328+0000 7ff8f4016700 1 -- 172.21.15.16:0/653429954 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff8ec09f650 msgr2=0x7ff8ec1c5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:20.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.328+0000 7ff8f4016700 1 --2- 172.21.15.16:0/653429954 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff8ec09f650 0x7ff8ec1c5e40 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4004270 tx=0x7ff8e4004ad0 comp rx=0 tx=0).stop 2023-09-30T22:34:20.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.328+0000 7ff8f4016700 1 -- 172.21.15.16:0/653429954 shutdown_connections 2023-09-30T22:34:20.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.328+0000 7ff8f4016700 1 --2- 172.21.15.16:0/653429954 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff8ec09e930 0x7ff8ec1a3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:20.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.328+0000 7ff8f4016700 1 --2- 172.21.15.16:0/653429954 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff8ec09f650 0x7ff8ec1c5e40 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:20.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.328+0000 7ff8f4016700 1 --2- 172.21.15.16:0/653429954 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8ec09f240 0x7ff8ec1a3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:20.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.328+0000 7ff8f4016700 1 -- 172.21.15.16:0/653429954 >> 172.21.15.16:0/653429954 conn(0x7ff8ec0a4210 msgr2=0x7ff8ec09bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:20.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.328+0000 7ff8f4016700 1 -- 172.21.15.16:0/653429954 shutdown_connections 2023-09-30T22:34:20.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:20.328+0000 7ff8f4016700 1 -- 172.21.15.16:0/653429954 wait complete. 2023-09-30T22:34:20.342 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:34:21.254 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:34:21.295 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:34:21.318 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:34:21.342 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:34:21.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.407+0000 7efc5639e700 1 Processor -- start 2023-09-30T22:34:21.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.407+0000 7efc5639e700 1 -- start start 2023-09-30T22:34:21.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.408+0000 7efc5639e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efc5009d800 0x7efc5009dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:21.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.408+0000 7efc5639e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc5009e110 0x7efc50133860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:21.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.408+0000 7efc5639e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efc5009e4e0 0x7efc50133da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:21.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.408+0000 7efc5639e700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc5009afe0 con 0x7efc5009e4e0 2023-09-30T22:34:21.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.408+0000 7efc5639e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc5009b1e0 con 0x7efc5009d800 2023-09-30T22:34:21.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.408+0000 7efc5639e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efc5009a600 con 0x7efc5009e110 2023-09-30T22:34:21.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.408+0000 7efc54b9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc5009e110 0x7efc50133860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:21.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.408+0000 7efc54b9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc5009e110 0x7efc50133860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:48064/0 (socket says 172.21.15.16:48064) 2023-09-30T22:34:21.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.408+0000 7efc54b9b700 1 -- 172.21.15.16:0/1768702437 learned_addr learned my addr 172.21.15.16:0/1768702437 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:21.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.408+0000 7efc5539c700 1 --2- 172.21.15.16:0/1768702437 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efc5009d800 0x7efc5009dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:21.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.408+0000 7efc55b9d700 1 --2- 172.21.15.16:0/1768702437 >> v2:172.21.15.16:3300/0 conn(0x7efc5009e4e0 0x7efc50133da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:21.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.409+0000 7efc54b9b700 1 -- 172.21.15.16:0/1768702437 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efc5009d800 msgr2=0x7efc5009dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:21.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.409+0000 7efc54b9b700 1 --2- 172.21.15.16:0/1768702437 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efc5009d800 0x7efc5009dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:21.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.409+0000 7efc54b9b700 1 -- 172.21.15.16:0/1768702437 >> v2:172.21.15.16:3300/0 conn(0x7efc5009e4e0 msgr2=0x7efc50133da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:21.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.409+0000 7efc54b9b700 1 --2- 172.21.15.16:0/1768702437 >> v2:172.21.15.16:3300/0 conn(0x7efc5009e4e0 0x7efc50133da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:21.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.409+0000 7efc54b9b700 1 -- 172.21.15.16:0/1768702437 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc50099c50 con 0x7efc5009e110 2023-09-30T22:34:21.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.409+0000 7efc54b9b700 1 --2- 172.21.15.16:0/1768702437 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc5009e110 0x7efc50133860 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7efc40006fd0 tx=0x7efc40009bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1f9b7441af978fe server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:21.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.410+0000 7efc47fff700 1 -- 172.21.15.16:0/1768702437 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc40005e30 con 0x7efc5009e110 2023-09-30T22:34:21.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.410+0000 7efc47fff700 1 -- 172.21.15.16:0/1768702437 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc40004030 con 0x7efc5009e110 2023-09-30T22:34:21.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.410+0000 7efc47fff700 1 -- 172.21.15.16:0/1768702437 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc400043e0 con 0x7efc5009e110 2023-09-30T22:34:21.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.411+0000 7efc5639e700 1 -- 172.21.15.16:0/1768702437 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc5009e110 msgr2=0x7efc50133860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:21.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.411+0000 7efc5639e700 1 --2- 172.21.15.16:0/1768702437 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc5009e110 0x7efc50133860 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7efc40006fd0 tx=0x7efc40009bd0 comp rx=0 tx=0).stop 2023-09-30T22:34:21.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.411+0000 7efc5639e700 1 -- 172.21.15.16:0/1768702437 shutdown_connections 2023-09-30T22:34:21.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.411+0000 7efc5639e700 1 --2- 172.21.15.16:0/1768702437 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efc5009d800 0x7efc5009dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:21.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.411+0000 7efc5639e700 1 --2- 172.21.15.16:0/1768702437 >> v2:172.21.15.16:3300/0 conn(0x7efc5009e4e0 0x7efc50133da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:21.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.411+0000 7efc5639e700 1 --2- 172.21.15.16:0/1768702437 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc5009e110 0x7efc50133860 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:21.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.411+0000 7efc5639e700 1 -- 172.21.15.16:0/1768702437 >> 172.21.15.16:0/1768702437 conn(0x7efc500a2e20 msgr2=0x7efc500a3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:21.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.411+0000 7efc5639e700 1 -- 172.21.15.16:0/1768702437 shutdown_connections 2023-09-30T22:34:21.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.411+0000 7efc5639e700 1 -- 172.21.15.16:0/1768702437 wait complete. 2023-09-30T22:34:21.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.413+0000 7efc5639e700 1 Processor -- start 2023-09-30T22:34:21.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.413+0000 7efc5639e700 1 -- start start 2023-09-30T22:34:21.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.414+0000 7efc5639e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc5009d800 0x7efc50122550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:21.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.414+0000 7efc5639e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc5009e110 0x7efc50122a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:21.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.414+0000 7efc5639e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efc5009e4e0 0x7efc50122fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:21.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.414+0000 7efc5639e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc50125770 con 0x7efc5009e110 2023-09-30T22:34:21.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.414+0000 7efc5639e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc50124160 con 0x7efc5009e4e0 2023-09-30T22:34:21.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.414+0000 7efc5639e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efc50133120 con 0x7efc5009d800 2023-09-30T22:34:21.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.414+0000 7efc5539c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc5009d800 0x7efc50122550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:21.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.414+0000 7efc5539c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc5009d800 0x7efc50122550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:48078/0 (socket says 172.21.15.16:48078) 2023-09-30T22:34:21.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.414+0000 7efc5539c700 1 -- 172.21.15.16:0/1046443457 learned_addr learned my addr 172.21.15.16:0/1046443457 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:21.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.414+0000 7efc54b9b700 1 --2- 172.21.15.16:0/1046443457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc5009e110 0x7efc50122a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:21.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.414+0000 7efc55b9d700 1 --2- 172.21.15.16:0/1046443457 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efc5009e4e0 0x7efc50122fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:21.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.414+0000 7efc55b9d700 1 -- 172.21.15.16:0/1046443457 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc5009d800 msgr2=0x7efc50122550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:21.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.414+0000 7efc55b9d700 1 --2- 172.21.15.16:0/1046443457 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc5009d800 0x7efc50122550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:21.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.414+0000 7efc55b9d700 1 -- 172.21.15.16:0/1046443457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc5009e110 msgr2=0x7efc50122a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:21.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.414+0000 7efc55b9d700 1 --2- 172.21.15.16:0/1046443457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc5009e110 0x7efc50122a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:21.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.414+0000 7efc55b9d700 1 -- 172.21.15.16:0/1046443457 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc40009500 con 0x7efc5009e4e0 2023-09-30T22:34:21.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.414+0000 7efc54b9b700 1 --2- 172.21.15.16:0/1046443457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc5009e110 0x7efc50122a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:21.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.414+0000 7efc5539c700 1 --2- 172.21.15.16:0/1046443457 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc5009d800 0x7efc50122550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:21.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.415+0000 7efc55b9d700 1 --2- 172.21.15.16:0/1046443457 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efc5009e4e0 0x7efc50122fd0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7efc4800efc0 tx=0x7efc4800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:21.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.415+0000 7efc45ffb700 1 -- 172.21.15.16:0/1046443457 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc4800cc90 con 0x7efc5009e4e0 2023-09-30T22:34:21.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.415+0000 7efc45ffb700 1 -- 172.21.15.16:0/1046443457 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc4800f040 con 0x7efc5009e4e0 2023-09-30T22:34:21.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.415+0000 7efc5639e700 1 -- 172.21.15.16:0/1046443457 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc50126070 con 0x7efc5009e4e0 2023-09-30T22:34:21.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.415+0000 7efc45ffb700 1 -- 172.21.15.16:0/1046443457 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc480146c0 con 0x7efc5009e4e0 2023-09-30T22:34:21.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.415+0000 7efc5639e700 1 -- 172.21.15.16:0/1046443457 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc50125d90 con 0x7efc5009e4e0 2023-09-30T22:34:21.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.416+0000 7efc45ffb700 1 -- 172.21.15.16:0/1046443457 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7efc48014820 con 0x7efc5009e4e0 2023-09-30T22:34:21.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.416+0000 7efc45ffb700 1 -- 172.21.15.16:0/1046443457 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7efc480400e0 con 0x7efc5009e4e0 2023-09-30T22:34:21.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.417+0000 7efc5639e700 1 -- 172.21.15.16:0/1046443457 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc5004f2a0 con 0x7efc5009e4e0 2023-09-30T22:34:21.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.422+0000 7efc45ffb700 1 -- 172.21.15.16:0/1046443457 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7efc4801d400 con 0x7efc5009e4e0 2023-09-30T22:34:21.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.462+0000 7f415c0ec700 1 Processor -- start 2023-09-30T22:34:21.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.463+0000 7f415c0ec700 1 -- start start 2023-09-30T22:34:21.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.463+0000 7f415c0ec700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f415409eca0 0x7f415413cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:21.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.463+0000 7f415c0ec700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f415406f050 0x7f415413f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:21.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.463+0000 7f415c0ec700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4154070ee0 con 0x7f415406e570 2023-09-30T22:34:21.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.463+0000 7f415c0ec700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4154071980 con 0x7f415406f050 2023-09-30T22:34:21.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.463+0000 7f415c0ec700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4154071480 con 0x7f415409eca0 2023-09-30T22:34:21.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.464+0000 7f4159687700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f415409eca0 0x7f415413cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:21.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.464+0000 7f4159e88700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f415406e570 0x7f415406e940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54886/0 (socket says 172.21.15.16:54886) 2023-09-30T22:34:21.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.464+0000 7f415a689700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f415406f050 0x7f415413f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:21.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.464+0000 7f4159687700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f415409eca0 0x7f415413cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:48080/0 (socket says 172.21.15.16:48080) 2023-09-30T22:34:21.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.464+0000 7f4159e88700 1 -- 172.21.15.16:0/3701341216 learned_addr learned my addr 172.21.15.16:0/3701341216 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:34:21.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.464+0000 7f4159687700 1 -- 172.21.15.16:0/3701341216 >> v2:172.21.15.16:3300/0 conn(0x7f415406f050 msgr2=0x7f415413f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:21.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.464+0000 7f4159687700 1 --2- 172.21.15.16:0/3701341216 >> v2:172.21.15.16:3300/0 conn(0x7f415406f050 0x7f415413f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:21.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.464+0000 7f4159687700 1 -- 172.21.15.16:0/3701341216 >> v1:172.21.15.16:6789/0 conn(0x7f415406e570 legacy=0x7f415406e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:21.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.464+0000 7f4159687700 1 -- 172.21.15.16:0/3701341216 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4154071b90 con 0x7f415409eca0 2023-09-30T22:34:21.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.464+0000 7f4158e86700 1 -- 172.21.15.16:0/3701341216 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2929903752 0 0) 0x7f4154070ee0 con 0x7f415406e570 2023-09-30T22:34:21.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.464+0000 7f4159687700 1 --2- 172.21.15.16:0/3701341216 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f415409eca0 0x7f415413cea0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f4144009840 tx=0x7f4144009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ed42ea2dd35422b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:21.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.464+0000 7f4158e86700 1 -- 172.21.15.16:0/3701341216 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4144011830 con 0x7f415409eca0 2023-09-30T22:34:21.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.464+0000 7f4158e86700 1 -- 172.21.15.16:0/3701341216 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4144011990 con 0x7f415409eca0 2023-09-30T22:34:21.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.464+0000 7f4158e86700 1 -- 172.21.15.16:0/3701341216 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4144011d20 con 0x7f415409eca0 2023-09-30T22:34:21.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.465+0000 7f415c0ec700 1 -- 172.21.15.16:0/3701341216 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f415409eca0 msgr2=0x7f415413cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:21.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.465+0000 7f415c0ec700 1 --2- 172.21.15.16:0/3701341216 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f415409eca0 0x7f415413cea0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f4144009840 tx=0x7f4144009b50 comp rx=0 tx=0).stop 2023-09-30T22:34:21.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.465+0000 7f415c0ec700 1 -- 172.21.15.16:0/3701341216 shutdown_connections 2023-09-30T22:34:21.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.465+0000 7f415c0ec700 1 --2- 172.21.15.16:0/3701341216 >> v2:172.21.15.16:3300/0 conn(0x7f415406f050 0x7f415413f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:21.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.465+0000 7f415c0ec700 1 --2- 172.21.15.16:0/3701341216 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f415409eca0 0x7f415413cea0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:21.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.465+0000 7f415c0ec700 1 -- 172.21.15.16:0/3701341216 >> 172.21.15.16:0/3701341216 conn(0x7f41540a2aa0 msgr2=0x7f41540a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:21.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.465+0000 7f415c0ec700 1 -- 172.21.15.16:0/3701341216 shutdown_connections 2023-09-30T22:34:21.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.465+0000 7f415c0ec700 1 -- 172.21.15.16:0/3701341216 wait complete. 2023-09-30T22:34:21.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.466+0000 7f415c0ec700 1 Processor -- start 2023-09-30T22:34:21.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.466+0000 7f415c0ec700 1 -- start start 2023-09-30T22:34:21.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.467+0000 7f415c0ec700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f415406e570 0x7f4154136fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:21.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.467+0000 7f415c0ec700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f415406f050 0x7f41541374f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:21.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.467+0000 7f415c0ec700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f415411ae40 0x7f4154137a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:21.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.467+0000 7f415c0ec700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f415411a700 con 0x7f415411ae40 2023-09-30T22:34:21.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.467+0000 7f415c0ec700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41541387d0 con 0x7f415406f050 2023-09-30T22:34:21.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.467+0000 7f415c0ec700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4154131710 con 0x7f415406e570 2023-09-30T22:34:21.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.467+0000 7f4159e88700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f415406e570 0x7f4154136fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:21.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.467+0000 7f4159e88700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f415406e570 0x7f4154136fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:48096/0 (socket says 172.21.15.16:48096) 2023-09-30T22:34:21.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.467+0000 7f4159e88700 1 -- 172.21.15.16:0/52814142 learned_addr learned my addr 172.21.15.16:0/52814142 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:21.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.467+0000 7f415a689700 1 --2- 172.21.15.16:0/52814142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f415411ae40 0x7f4154137a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:21.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.467+0000 7f4159e88700 1 -- 172.21.15.16:0/52814142 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f415406f050 msgr2=0x7f41541374f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:21.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.467+0000 7f4159e88700 1 --2- 172.21.15.16:0/52814142 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f415406f050 0x7f41541374f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:21.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.467+0000 7f4159e88700 1 -- 172.21.15.16:0/52814142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f415411ae40 msgr2=0x7f4154137a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:21.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.467+0000 7f4159e88700 1 --2- 172.21.15.16:0/52814142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f415411ae40 0x7f4154137a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:21.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.468+0000 7f4159e88700 1 -- 172.21.15.16:0/52814142 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41440094a0 con 0x7f415406e570 2023-09-30T22:34:21.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.468+0000 7f4159e88700 1 --2- 172.21.15.16:0/52814142 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f415406e570 0x7f4154136fb0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f415000c520 tx=0x7f415000c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:21.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.468+0000 7f414a7fc700 1 -- 172.21.15.16:0/52814142 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4150004c80 con 0x7f415406e570 2023-09-30T22:34:21.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.468+0000 7f414a7fc700 1 -- 172.21.15.16:0/52814142 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4150004de0 con 0x7f415406e570 2023-09-30T22:34:21.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.468+0000 7f415c0ec700 1 -- 172.21.15.16:0/52814142 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f415413e820 con 0x7f415406e570 2023-09-30T22:34:21.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.468+0000 7f414a7fc700 1 -- 172.21.15.16:0/52814142 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4150007310 con 0x7f415406e570 2023-09-30T22:34:21.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.468+0000 7f415c0ec700 1 -- 172.21.15.16:0/52814142 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4154119d10 con 0x7f415406e570 2023-09-30T22:34:21.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.469+0000 7f414a7fc700 1 -- 172.21.15.16:0/52814142 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4150007470 con 0x7f415406e570 2023-09-30T22:34:21.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.469+0000 7f414a7fc700 1 -- 172.21.15.16:0/52814142 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f415003f410 con 0x7f415406e570 2023-09-30T22:34:21.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.469+0000 7f415c0ec700 1 -- 172.21.15.16:0/52814142 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41540a2220 con 0x7f415406e570 2023-09-30T22:34:21.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.479+0000 7f414a7fc700 1 -- 172.21.15.16:0/52814142 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f41500123b0 con 0x7f415406e570 2023-09-30T22:34:21.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.496+0000 7f917d615700 1 Processor -- start 2023-09-30T22:34:21.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.496+0000 7f917d615700 1 -- start start 2023-09-30T22:34:21.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.496+0000 7f917d615700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f917809f410 0x7f9178136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:21.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.496+0000 7f917d615700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9178136fe0 0x7f91781393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:21.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.496+0000 7f917d615700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9178070940 con 0x7f917809e930 2023-09-30T22:34:21.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.496+0000 7f917d615700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f917806fd60 con 0x7f9178136fe0 2023-09-30T22:34:21.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.496+0000 7f917d615700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f917806fa80 con 0x7f917809f410 2023-09-30T22:34:21.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.496+0000 7f91777fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9178136fe0 0x7f91781393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:21.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.497+0000 7f91777fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9178136fe0 0x7f91781393c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47494/0 (socket says 172.21.15.16:47494) 2023-09-30T22:34:21.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.497+0000 7f91777fe700 1 -- 172.21.15.16:0/1021399339 learned_addr learned my addr 172.21.15.16:0/1021399339 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:21.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.497+0000 7f91767fc700 1 --2- 172.21.15.16:0/1021399339 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f917809f410 0x7f9178136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:21.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.497+0000 7f91777fe700 1 -- 172.21.15.16:0/1021399339 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f917809f410 msgr2=0x7f9178136aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:21.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.497+0000 7f91777fe700 1 --2- 172.21.15.16:0/1021399339 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f917809f410 0x7f9178136aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:21.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.497+0000 7f91777fe700 1 -- 172.21.15.16:0/1021399339 >> v1:172.21.15.16:6789/0 conn(0x7f917809e930 legacy=0x7f917809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:21.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.497+0000 7f91777fe700 1 -- 172.21.15.16:0/1021399339 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f917809fb60 con 0x7f9178136fe0 2023-09-30T22:34:21.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.497+0000 7f91777fe700 1 --2- 172.21.15.16:0/1021399339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9178136fe0 0x7f91781393c0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f9170009e50 tx=0x7f917000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9cef32e3c4634ca server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:21.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.497+0000 7f9175ffb700 1 -- 172.21.15.16:0/1021399339 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917000e070 con 0x7f9178136fe0 2023-09-30T22:34:21.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.497+0000 7f9175ffb700 1 -- 172.21.15.16:0/1021399339 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f917000b910 con 0x7f9178136fe0 2023-09-30T22:34:21.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.497+0000 7f9175ffb700 1 -- 172.21.15.16:0/1021399339 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917000bcc0 con 0x7f9178136fe0 2023-09-30T22:34:21.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.498+0000 7f917d615700 1 -- 172.21.15.16:0/1021399339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9178136fe0 msgr2=0x7f91781393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:21.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.498+0000 7f917d615700 1 --2- 172.21.15.16:0/1021399339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9178136fe0 0x7f91781393c0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f9170009e50 tx=0x7f917000b330 comp rx=0 tx=0).stop 2023-09-30T22:34:21.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.498+0000 7f917d615700 1 -- 172.21.15.16:0/1021399339 shutdown_connections 2023-09-30T22:34:21.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.498+0000 7f917d615700 1 --2- 172.21.15.16:0/1021399339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9178136fe0 0x7f91781393c0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:21.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.498+0000 7f917d615700 1 --2- 172.21.15.16:0/1021399339 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f917809f410 0x7f9178136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:21.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.498+0000 7f917d615700 1 -- 172.21.15.16:0/1021399339 >> 172.21.15.16:0/1021399339 conn(0x7f91780a4210 msgr2=0x7f91780a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:21.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.498+0000 7f917d615700 1 -- 172.21.15.16:0/1021399339 shutdown_connections 2023-09-30T22:34:21.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.499+0000 7f917d615700 1 -- 172.21.15.16:0/1021399339 wait complete. 2023-09-30T22:34:21.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.500+0000 7f917d615700 1 Processor -- start 2023-09-30T22:34:21.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.500+0000 7f917d615700 1 -- start start 2023-09-30T22:34:21.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.501+0000 7f917d615700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f917809e930 0x7f917811c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:21.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.501+0000 7f917d615700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f917809f410 0x7f917811cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:21.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.501+0000 7f917d615700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9178136fe0 0x7f917811d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:21.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.501+0000 7f917d615700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9178131e70 con 0x7f917809e930 2023-09-30T22:34:21.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.501+0000 7f917d615700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f917811e150 con 0x7f917809f410 2023-09-30T22:34:21.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.501+0000 7f917d615700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f917811e9e0 con 0x7f9178136fe0 2023-09-30T22:34:21.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.501+0000 7f91767fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f917809f410 0x7f917811cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:21.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.501+0000 7f91767fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f917809f410 0x7f917811cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:55440/0 (socket says 172.21.15.16:55440) 2023-09-30T22:34:21.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.501+0000 7f91767fc700 1 -- 172.21.15.16:0/2937914407 learned_addr learned my addr 172.21.15.16:0/2937914407 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:21.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.501+0000 7f91777fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9178136fe0 0x7f917811d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:21.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.501+0000 7f9176ffd700 1 --2- 172.21.15.16:0/2937914407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f917809e930 0x7f917811c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:21.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.502+0000 7f91767fc700 1 -- 172.21.15.16:0/2937914407 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9178136fe0 msgr2=0x7f917811d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:21.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.502+0000 7f91767fc700 1 --2- 172.21.15.16:0/2937914407 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9178136fe0 0x7f917811d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:21.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.502+0000 7f91767fc700 1 -- 172.21.15.16:0/2937914407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f917809e930 msgr2=0x7f917811c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:21.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.502+0000 7f91767fc700 1 --2- 172.21.15.16:0/2937914407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f917809e930 0x7f917811c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:21.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.502+0000 7f91767fc700 1 -- 172.21.15.16:0/2937914407 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9170009b30 con 0x7f917809f410 2023-09-30T22:34:21.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.502+0000 7f91767fc700 1 --2- 172.21.15.16:0/2937914407 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f917809f410 0x7f917811cac0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f916c00e9e0 tx=0x7f916c00ecf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:21.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.502+0000 7f915f7fe700 1 -- 172.21.15.16:0/2937914407 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916c009710 con 0x7f917809f410 2023-09-30T22:34:21.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.502+0000 7f915f7fe700 1 -- 172.21.15.16:0/2937914407 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f916c00ee80 con 0x7f917809f410 2023-09-30T22:34:21.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.502+0000 7f917d615700 1 -- 172.21.15.16:0/2937914407 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9178071d20 con 0x7f917809f410 2023-09-30T22:34:21.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.502+0000 7f915f7fe700 1 -- 172.21.15.16:0/2937914407 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916c00c9f0 con 0x7f917809f410 2023-09-30T22:34:21.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.502+0000 7f917d615700 1 -- 172.21.15.16:0/2937914407 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91780a2f40 con 0x7f917809f410 2023-09-30T22:34:21.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.503+0000 7f915f7fe700 1 -- 172.21.15.16:0/2937914407 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f916c00cb50 con 0x7f917809f410 2023-09-30T22:34:21.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.503+0000 7f915f7fe700 1 -- 172.21.15.16:0/2937914407 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f916c03f770 con 0x7f917809f410 2023-09-30T22:34:21.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.503+0000 7f917d615700 1 -- 172.21.15.16:0/2937914407 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9178074ea0 con 0x7f917809f410 2023-09-30T22:34:21.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.506+0000 7f4c73771700 1 Processor -- start 2023-09-30T22:34:21.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.507+0000 7f4c73771700 1 -- start start 2023-09-30T22:34:21.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.507+0000 7f4c73771700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4c6c09d6f0 0x7f4c6c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:21.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.507+0000 7f4c73771700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4c6c09e410 0x7f4c6c130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:21.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.507+0000 7f4c73771700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c6c09f130 con 0x7f4c6c09e000 2023-09-30T22:34:21.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.507+0000 7f4c73771700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c6c09f520 con 0x7f4c6c09d6f0 2023-09-30T22:34:21.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.507+0000 7f4c73771700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6c09e930 con 0x7f4c6c09e410 2023-09-30T22:34:21.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.507+0000 7f4c71d0e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4c6c09e410 0x7f4c6c130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:21.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.507+0000 7f4c71d0e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4c6c09e410 0x7f4c6c130d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:48108/0 (socket says 172.21.15.16:48108) 2023-09-30T22:34:21.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.507+0000 7f4c71d0e700 1 -- 172.21.15.16:0/3664115073 learned_addr learned my addr 172.21.15.16:0/3664115073 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:21.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.508+0000 7f4c71d0e700 1 -- 172.21.15.16:0/3664115073 >> v2:172.21.15.16:3300/0 conn(0x7f4c6c09d6f0 msgr2=0x7f4c6c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:21.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.508+0000 7f4c71d0e700 1 --2- 172.21.15.16:0/3664115073 >> v2:172.21.15.16:3300/0 conn(0x7f4c6c09d6f0 0x7f4c6c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:21.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.508+0000 7f4c71d0e700 1 -- 172.21.15.16:0/3664115073 >> v1:172.21.15.16:6789/0 conn(0x7f4c6c09e000 legacy=0x7f4c6c130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:21.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.508+0000 7f4c71d0e700 1 -- 172.21.15.16:0/3664115073 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c6c09b2d0 con 0x7f4c6c09e410 2023-09-30T22:34:21.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.508+0000 7f4c71d0e700 1 --2- 172.21.15.16:0/3664115073 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4c6c09e410 0x7f4c6c130d70 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f4c6400d150 tx=0x7f4c6400d460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a46c1ce2da0af6d1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:21.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.508+0000 7f4c63fff700 1 -- 172.21.15.16:0/3664115073 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c64010070 con 0x7f4c6c09e410 2023-09-30T22:34:21.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.508+0000 7f4c63fff700 1 -- 172.21.15.16:0/3664115073 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c6400dd80 con 0x7f4c6c09e410 2023-09-30T22:34:21.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.508+0000 7f4c63fff700 1 -- 172.21.15.16:0/3664115073 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6400b310 con 0x7f4c6c09e410 2023-09-30T22:34:21.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.509+0000 7f4c73771700 1 -- 172.21.15.16:0/3664115073 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4c6c09e410 msgr2=0x7f4c6c130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:21.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.509+0000 7f4c73771700 1 --2- 172.21.15.16:0/3664115073 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4c6c09e410 0x7f4c6c130d70 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f4c6400d150 tx=0x7f4c6400d460 comp rx=0 tx=0).stop 2023-09-30T22:34:21.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.509+0000 7f915f7fe700 1 -- 172.21.15.16:0/2937914407 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f916c017080 con 0x7f917809f410 2023-09-30T22:34:21.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.509+0000 7f4c73771700 1 -- 172.21.15.16:0/3664115073 shutdown_connections 2023-09-30T22:34:21.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.509+0000 7f4c73771700 1 --2- 172.21.15.16:0/3664115073 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4c6c09e410 0x7f4c6c130d70 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:21.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.509+0000 7f4c73771700 1 --2- 172.21.15.16:0/3664115073 >> v2:172.21.15.16:3300/0 conn(0x7f4c6c09d6f0 0x7f4c6c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:21.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.509+0000 7f4c73771700 1 -- 172.21.15.16:0/3664115073 >> 172.21.15.16:0/3664115073 conn(0x7f4c6c066f50 msgr2=0x7f4c6c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:21.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.509+0000 7f4c73771700 1 -- 172.21.15.16:0/3664115073 shutdown_connections 2023-09-30T22:34:21.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.510+0000 7f4c73771700 1 -- 172.21.15.16:0/3664115073 wait complete. 2023-09-30T22:34:21.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.511+0000 7f4c73771700 1 Processor -- start 2023-09-30T22:34:21.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.511+0000 7f4c73771700 1 -- start start 2023-09-30T22:34:21.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.511+0000 7f4c73771700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4c6c09d6f0 0x7f4c6c1b6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:21.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.511+0000 7f4c73771700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4c6c09e000 0x7f4c6c1b68a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:21.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.511+0000 7f4c73771700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c6c09e410 0x7f4c6c1b6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:21.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.511+0000 7f4c73771700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6c09c790 con 0x7f4c6c09e410 2023-09-30T22:34:21.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.511+0000 7f4c73771700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6c1b7ed0 con 0x7f4c6c09e000 2023-09-30T22:34:21.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.511+0000 7f4c73771700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6c1b9480 con 0x7f4c6c09d6f0 2023-09-30T22:34:21.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.511+0000 7f4c7150d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4c6c09d6f0 0x7f4c6c1b6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:21.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.511+0000 7f4c7150d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4c6c09d6f0 0x7f4c6c1b6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:48122/0 (socket says 172.21.15.16:48122) 2023-09-30T22:34:21.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.511+0000 7f4c7150d700 1 -- 172.21.15.16:0/3780023834 learned_addr learned my addr 172.21.15.16:0/3780023834 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:21.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.511+0000 7f4c70d0c700 1 --2- 172.21.15.16:0/3780023834 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4c6c09e000 0x7f4c6c1b68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:21.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.511+0000 7f4c71d0e700 1 --2- 172.21.15.16:0/3780023834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c6c09e410 0x7f4c6c1b6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:21.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.512+0000 7f4c71d0e700 1 -- 172.21.15.16:0/3780023834 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4c6c09d6f0 msgr2=0x7f4c6c1b6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:21.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.512+0000 7f4c71d0e700 1 --2- 172.21.15.16:0/3780023834 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4c6c09d6f0 0x7f4c6c1b6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:21.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.512+0000 7f4c71d0e700 1 -- 172.21.15.16:0/3780023834 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4c6c09e000 msgr2=0x7f4c6c1b68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:21.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.512+0000 7f4c71d0e700 1 --2- 172.21.15.16:0/3780023834 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4c6c09e000 0x7f4c6c1b68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:21.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.512+0000 7f4c71d0e700 1 -- 172.21.15.16:0/3780023834 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c6400bd40 con 0x7f4c6c09e410 2023-09-30T22:34:21.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.512+0000 7f4c70d0c700 1 --2- 172.21.15.16:0/3780023834 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4c6c09e000 0x7f4c6c1b68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:21.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.512+0000 7f4c71d0e700 1 --2- 172.21.15.16:0/3780023834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c6c09e410 0x7f4c6c1b6de0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c072a80 tx=0x7f4c64004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:21.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.512+0000 7f4c61ffb700 1 -- 172.21.15.16:0/3780023834 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c64009a10 con 0x7f4c6c09e410 2023-09-30T22:34:21.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.512+0000 7f4c61ffb700 1 -- 172.21.15.16:0/3780023834 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c64009b70 con 0x7f4c6c09e410 2023-09-30T22:34:21.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.512+0000 7f4c73771700 1 -- 172.21.15.16:0/3780023834 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c6c1b91b0 con 0x7f4c6c09e410 2023-09-30T22:34:21.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.512+0000 7f4c61ffb700 1 -- 172.21.15.16:0/3780023834 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c64003e50 con 0x7f4c6c09e410 2023-09-30T22:34:21.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.512+0000 7f4c73771700 1 -- 172.21.15.16:0/3780023834 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c6c1b8300 con 0x7f4c6c09e410 2023-09-30T22:34:21.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.514+0000 7f4c73771700 1 -- 172.21.15.16:0/3780023834 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c6c09ed60 con 0x7f4c6c09e410 2023-09-30T22:34:21.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.514+0000 7f4c61ffb700 1 -- 172.21.15.16:0/3780023834 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4c6401b050 con 0x7f4c6c09e410 2023-09-30T22:34:21.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.514+0000 7f4c61ffb700 1 -- 172.21.15.16:0/3780023834 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f4c64040b80 con 0x7f4c6c09e410 2023-09-30T22:34:21.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.519+0000 7f4c61ffb700 1 -- 172.21.15.16:0/3780023834 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4c64013390 con 0x7f4c6c09e410 2023-09-30T22:34:21.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.646+0000 7efc5639e700 1 -- 172.21.15.16:0/1046443457 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7efc500a1be0 con 0x7efc5009e4e0 2023-09-30T22:34:21.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.646+0000 7efc45ffb700 1 -- 172.21.15.16:0/1046443457 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7efc4802c090 con 0x7efc5009e4e0 2023-09-30T22:34:21.647 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:34:21.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.650+0000 7efc5639e700 1 -- 172.21.15.16:0/1046443457 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efc5009e4e0 msgr2=0x7efc50122fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:21.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.650+0000 7efc5639e700 1 --2- 172.21.15.16:0/1046443457 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efc5009e4e0 0x7efc50122fd0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7efc4800efc0 tx=0x7efc4800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:34:21.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.650+0000 7efc5639e700 1 -- 172.21.15.16:0/1046443457 shutdown_connections 2023-09-30T22:34:21.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.650+0000 7efc5639e700 1 --2- 172.21.15.16:0/1046443457 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efc5009e4e0 0x7efc50122fd0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:21.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.650+0000 7efc5639e700 1 --2- 172.21.15.16:0/1046443457 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc5009d800 0x7efc50122550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:21.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.650+0000 7efc5639e700 1 --2- 172.21.15.16:0/1046443457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc5009e110 0x7efc50122a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:21.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.650+0000 7efc5639e700 1 -- 172.21.15.16:0/1046443457 >> 172.21.15.16:0/1046443457 conn(0x7efc500a2e20 msgr2=0x7efc50070d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:21.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.650+0000 7efc5639e700 1 -- 172.21.15.16:0/1046443457 shutdown_connections 2023-09-30T22:34:21.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.650+0000 7efc5639e700 1 -- 172.21.15.16:0/1046443457 wait complete. 2023-09-30T22:34:21.669 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:34:21.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.735+0000 7f4c73771700 1 -- 172.21.15.16:0/3780023834 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4c6c074ea0 con 0x7f4c6c09e410 2023-09-30T22:34:21.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.735+0000 7f4c61ffb700 1 -- 172.21.15.16:0/3780023834 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4c6402b070 con 0x7f4c6c09e410 2023-09-30T22:34:21.736 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:34:21.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.738+0000 7f4c73771700 1 -- 172.21.15.16:0/3780023834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c6c09e410 msgr2=0x7f4c6c1b6de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:21.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.738+0000 7f4c73771700 1 --2- 172.21.15.16:0/3780023834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c6c09e410 0x7f4c6c1b6de0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c072a80 tx=0x7f4c64004ad0 comp rx=0 tx=0).stop 2023-09-30T22:34:21.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.738+0000 7f4c73771700 1 -- 172.21.15.16:0/3780023834 shutdown_connections 2023-09-30T22:34:21.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.738+0000 7f4c73771700 1 --2- 172.21.15.16:0/3780023834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c6c09e410 0x7f4c6c1b6de0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:21.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.738+0000 7f4c73771700 1 --2- 172.21.15.16:0/3780023834 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4c6c09d6f0 0x7f4c6c1b6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:21.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.738+0000 7f4c73771700 1 --2- 172.21.15.16:0/3780023834 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4c6c09e000 0x7f4c6c1b68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:21.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.738+0000 7f4c73771700 1 -- 172.21.15.16:0/3780023834 >> 172.21.15.16:0/3780023834 conn(0x7f4c6c066f50 msgr2=0x7f4c6c12d960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:21.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.738+0000 7f4c73771700 1 -- 172.21.15.16:0/3780023834 shutdown_connections 2023-09-30T22:34:21.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.738+0000 7f4c73771700 1 -- 172.21.15.16:0/3780023834 wait complete. 2023-09-30T22:34:21.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.744+0000 7f415c0ec700 1 -- 172.21.15.16:0/52814142 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4154074ea0 con 0x7f415406e570 2023-09-30T22:34:21.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.744+0000 7f414a7fc700 1 -- 172.21.15.16:0/52814142 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f41500102c0 con 0x7f415406e570 2023-09-30T22:34:21.744 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:34:21.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.746+0000 7f415c0ec700 1 -- 172.21.15.16:0/52814142 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f415406e570 msgr2=0x7f4154136fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:21.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.746+0000 7f415c0ec700 1 --2- 172.21.15.16:0/52814142 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f415406e570 0x7f4154136fb0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f415000c520 tx=0x7f415000c830 comp rx=0 tx=0).stop 2023-09-30T22:34:21.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.747+0000 7f415c0ec700 1 -- 172.21.15.16:0/52814142 shutdown_connections 2023-09-30T22:34:21.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.747+0000 7f415c0ec700 1 --2- 172.21.15.16:0/52814142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f415411ae40 0x7f4154137a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:21.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.747+0000 7f415c0ec700 1 --2- 172.21.15.16:0/52814142 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f415406e570 0x7f4154136fb0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:21.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.747+0000 7f415c0ec700 1 --2- 172.21.15.16:0/52814142 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f415406f050 0x7f41541374f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:21.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.747+0000 7f415c0ec700 1 -- 172.21.15.16:0/52814142 >> 172.21.15.16:0/52814142 conn(0x7f41540a2aa0 msgr2=0x7f415409c970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:21.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.747+0000 7f415c0ec700 1 -- 172.21.15.16:0/52814142 shutdown_connections 2023-09-30T22:34:21.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.747+0000 7f415c0ec700 1 -- 172.21.15.16:0/52814142 wait complete. 2023-09-30T22:34:21.754 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:34:21.762 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:34:21.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.771+0000 7f917d615700 1 -- 172.21.15.16:0/2937914407 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f917811fa20 con 0x7f917809f410 2023-09-30T22:34:21.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.771+0000 7f915f7fe700 1 -- 172.21.15.16:0/2937914407 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f916c0232f0 con 0x7f917809f410 2023-09-30T22:34:21.772 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:34:21.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.774+0000 7f917d615700 1 -- 172.21.15.16:0/2937914407 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f917809f410 msgr2=0x7f917811cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:21.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.774+0000 7f917d615700 1 --2- 172.21.15.16:0/2937914407 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f917809f410 0x7f917811cac0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f916c00e9e0 tx=0x7f916c00ecf0 comp rx=0 tx=0).stop 2023-09-30T22:34:21.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.774+0000 7f917d615700 1 -- 172.21.15.16:0/2937914407 shutdown_connections 2023-09-30T22:34:21.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.774+0000 7f917d615700 1 --2- 172.21.15.16:0/2937914407 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9178136fe0 0x7f917811d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:21.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.774+0000 7f917d615700 1 --2- 172.21.15.16:0/2937914407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f917809e930 0x7f917811c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:21.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.774+0000 7f917d615700 1 --2- 172.21.15.16:0/2937914407 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f917809f410 0x7f917811cac0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:21.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.774+0000 7f917d615700 1 -- 172.21.15.16:0/2937914407 >> 172.21.15.16:0/2937914407 conn(0x7f91780a4210 msgr2=0x7f91780a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:21.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.774+0000 7f917d615700 1 -- 172.21.15.16:0/2937914407 shutdown_connections 2023-09-30T22:34:21.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:21.774+0000 7f917d615700 1 -- 172.21.15.16:0/2937914407 wait complete. 2023-09-30T22:34:21.787 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:34:22.670 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:34:22.754 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:34:22.762 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:34:22.789 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:34:22.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.820+0000 7f3c84b68700 1 Processor -- start 2023-09-30T22:34:22.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.820+0000 7f3c84b68700 1 -- start start 2023-09-30T22:34:22.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.821+0000 7f3c84b68700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3c80136830 0x7f3c80138c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:22.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.821+0000 7f3c84b68700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3c80139150 0x7f3c8013b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:22.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.821+0000 7f3c84b68700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c8006f720 con 0x7f3c801338c0 2023-09-30T22:34:22.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.821+0000 7f3c84b68700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c80070c80 con 0x7f3c80139150 2023-09-30T22:34:22.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.821+0000 7f3c84b68700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8006fce0 con 0x7f3c80136830 2023-09-30T22:34:22.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.821+0000 7f3c7e59c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3c801338c0 0x7f3c80133c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54906/0 (socket says 172.21.15.16:54906) 2023-09-30T22:34:22.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.821+0000 7f3c7e59c700 1 -- 172.21.15.16:0/2889426689 learned_addr learned my addr 172.21.15.16:0/2889426689 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:34:22.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.822+0000 7f3c7ed9d700 1 --2- 172.21.15.16:0/2889426689 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3c80139150 0x7f3c8013b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:22.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.822+0000 7f3c7d59a700 1 -- 172.21.15.16:0/2889426689 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2884314617 0 0) 0x7f3c8006f720 con 0x7f3c801338c0 2023-09-30T22:34:22.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.822+0000 7f3c7d59a700 1 -- 172.21.15.16:0/2889426689 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c680035b0 con 0x7f3c801338c0 2023-09-30T22:34:22.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.822+0000 7f3c7ed9d700 1 -- 172.21.15.16:0/2889426689 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3c80136830 msgr2=0x7f3c80138c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:22.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.822+0000 7f3c7ed9d700 1 --2- 172.21.15.16:0/2889426689 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3c80136830 0x7f3c80138c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:22.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.822+0000 7f3c7ed9d700 1 -- 172.21.15.16:0/2889426689 >> v1:172.21.15.16:6789/0 conn(0x7f3c801338c0 legacy=0x7f3c80133c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:22.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.822+0000 7f3c7ed9d700 1 -- 172.21.15.16:0/2889426689 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c801336a0 con 0x7f3c80139150 2023-09-30T22:34:22.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.822+0000 7f3c7ed9d700 1 --2- 172.21.15.16:0/2889426689 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3c80139150 0x7f3c8013b530 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f3c78009dc0 tx=0x7f3c7800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5e7be000b3aa9f6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:22.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.822+0000 7f3c7d59a700 1 -- 172.21.15.16:0/2889426689 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c7800e070 con 0x7f3c80139150 2023-09-30T22:34:22.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.823+0000 7f3c7d59a700 1 -- 172.21.15.16:0/2889426689 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c7800bb20 con 0x7f3c80139150 2023-09-30T22:34:22.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.823+0000 7f3c7d59a700 1 -- 172.21.15.16:0/2889426689 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c78012430 con 0x7f3c80139150 2023-09-30T22:34:22.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.823+0000 7f3c84b68700 1 -- 172.21.15.16:0/2889426689 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3c80139150 msgr2=0x7f3c8013b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:22.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.823+0000 7f3c84b68700 1 --2- 172.21.15.16:0/2889426689 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3c80139150 0x7f3c8013b530 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f3c78009dc0 tx=0x7f3c7800b330 comp rx=0 tx=0).stop 2023-09-30T22:34:22.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.824+0000 7f3c84b68700 1 -- 172.21.15.16:0/2889426689 shutdown_connections 2023-09-30T22:34:22.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.824+0000 7f3c84b68700 1 --2- 172.21.15.16:0/2889426689 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3c80139150 0x7f3c8013b530 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:22.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.824+0000 7f3c84b68700 1 --2- 172.21.15.16:0/2889426689 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3c80136830 0x7f3c80138c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:22.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.824+0000 7f3c84b68700 1 -- 172.21.15.16:0/2889426689 >> 172.21.15.16:0/2889426689 conn(0x7f3c800a2aa0 msgr2=0x7f3c800a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:22.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.824+0000 7f3c84b68700 1 -- 172.21.15.16:0/2889426689 shutdown_connections 2023-09-30T22:34:22.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.824+0000 7f3c84b68700 1 -- 172.21.15.16:0/2889426689 wait complete. 2023-09-30T22:34:22.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.826+0000 7f3c84b68700 1 Processor -- start 2023-09-30T22:34:22.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.827+0000 7f3c84b68700 1 -- start start 2023-09-30T22:34:22.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.827+0000 7f3c84b68700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c801338c0 0x7f3c801c8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:22.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.827+0000 7f3c84b68700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3c80136830 0x7f3c801c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:22.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.827+0000 7f3c84b68700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3c80139150 0x7f3c80119e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:22.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.827+0000 7f3c84b68700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8011d6a0 con 0x7f3c801338c0 2023-09-30T22:34:22.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.827+0000 7f3c84b68700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8011d880 con 0x7f3c80136830 2023-09-30T22:34:22.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.827+0000 7f3c84b68700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3c801be230 con 0x7f3c80139150 2023-09-30T22:34:22.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.828+0000 7f3c7e59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c801338c0 0x7f3c801c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:22.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.828+0000 7f3c7dd9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3c80136830 0x7f3c801c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:22.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.828+0000 7f3c7e59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c801338c0 0x7f3c801c8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47534/0 (socket says 172.21.15.16:47534) 2023-09-30T22:34:22.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.828+0000 7f3c7dd9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3c80136830 0x7f3c801c8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:55472/0 (socket says 172.21.15.16:55472) 2023-09-30T22:34:22.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.828+0000 7f3c7e59c700 1 -- 172.21.15.16:0/3169360877 learned_addr learned my addr 172.21.15.16:0/3169360877 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:22.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.828+0000 7f3c7ed9d700 1 --2- 172.21.15.16:0/3169360877 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3c80139150 0x7f3c80119e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:22.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.828+0000 7f3c7e59c700 1 -- 172.21.15.16:0/3169360877 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3c80139150 msgr2=0x7f3c80119e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:22.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.828+0000 7f3c7e59c700 1 --2- 172.21.15.16:0/3169360877 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3c80139150 0x7f3c80119e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:22.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.828+0000 7f3c7e59c700 1 -- 172.21.15.16:0/3169360877 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3c80136830 msgr2=0x7f3c801c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:22.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.828+0000 7f3c7e59c700 1 --2- 172.21.15.16:0/3169360877 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3c80136830 0x7f3c801c8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:22.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.828+0000 7f3c7e59c700 1 -- 172.21.15.16:0/3169360877 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c78009a70 con 0x7f3c801338c0 2023-09-30T22:34:22.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.828+0000 7f3c7dd9b700 1 --2- 172.21.15.16:0/3169360877 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3c80136830 0x7f3c801c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:22.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.829+0000 7f3c7e59c700 1 --2- 172.21.15.16:0/3169360877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c801338c0 0x7f3c801c8560 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f3c7000c700 tx=0x7f3c7000ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:22.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.829+0000 7f3c6effd700 1 -- 172.21.15.16:0/3169360877 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c70005d70 con 0x7f3c801338c0 2023-09-30T22:34:22.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.829+0000 7f3c6effd700 1 -- 172.21.15.16:0/3169360877 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c700070d0 con 0x7f3c801338c0 2023-09-30T22:34:22.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.829+0000 7f3c6effd700 1 -- 172.21.15.16:0/3169360877 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c70007460 con 0x7f3c801338c0 2023-09-30T22:34:22.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.829+0000 7f3c84b68700 1 -- 172.21.15.16:0/3169360877 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c80130000 con 0x7f3c801338c0 2023-09-30T22:34:22.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.829+0000 7f3c84b68700 1 -- 172.21.15.16:0/3169360877 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c8011b970 con 0x7f3c801338c0 2023-09-30T22:34:22.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.830+0000 7f3c6effd700 1 -- 172.21.15.16:0/3169360877 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3c700075c0 con 0x7f3c801338c0 2023-09-30T22:34:22.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.830+0000 7f3c84b68700 1 -- 172.21.15.16:0/3169360877 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c80131140 con 0x7f3c801338c0 2023-09-30T22:34:22.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.830+0000 7f3c6effd700 1 -- 172.21.15.16:0/3169360877 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f3c7003ec10 con 0x7f3c801338c0 2023-09-30T22:34:22.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.836+0000 7f3c6effd700 1 -- 172.21.15.16:0/3169360877 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3c7000e280 con 0x7f3c801338c0 2023-09-30T22:34:22.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.905+0000 7f93c48a1700 1 Processor -- start 2023-09-30T22:34:22.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.905+0000 7f93c48a1700 1 -- start start 2023-09-30T22:34:22.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.905+0000 7f93c48a1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f93c009f410 0x7f93c0136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:22.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.905+0000 7f93c48a1700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f93c0136fe0 0x7f93c01393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:22.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.905+0000 7f93c48a1700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93c0070940 con 0x7f93c009e930 2023-09-30T22:34:22.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.905+0000 7f93c48a1700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93c006fd60 con 0x7f93c0136fe0 2023-09-30T22:34:22.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.905+0000 7f93c48a1700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93c006fa80 con 0x7f93c009f410 2023-09-30T22:34:22.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.906+0000 7f93bed9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f93c0136fe0 0x7f93c01393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:22.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.906+0000 7f93bed9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f93c0136fe0 0x7f93c01393c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47548/0 (socket says 172.21.15.16:47548) 2023-09-30T22:34:22.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.906+0000 7f93bed9d700 1 -- 172.21.15.16:0/2490442001 learned_addr learned my addr 172.21.15.16:0/2490442001 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:22.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.906+0000 7f93bdd9b700 1 --2- 172.21.15.16:0/2490442001 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f93c009f410 0x7f93c0136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:22.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.906+0000 7f93bed9d700 1 -- 172.21.15.16:0/2490442001 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f93c009f410 msgr2=0x7f93c0136aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:22.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.906+0000 7f93bed9d700 1 --2- 172.21.15.16:0/2490442001 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f93c009f410 0x7f93c0136aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:22.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.906+0000 7f93bed9d700 1 -- 172.21.15.16:0/2490442001 >> v1:172.21.15.16:6789/0 conn(0x7f93c009e930 legacy=0x7f93c009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:22.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.906+0000 7f93bed9d700 1 -- 172.21.15.16:0/2490442001 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93c009fb60 con 0x7f93c0136fe0 2023-09-30T22:34:22.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.906+0000 7f93bdd9b700 1 --2- 172.21.15.16:0/2490442001 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f93c009f410 0x7f93c0136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:34:22.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.906+0000 7f93bed9d700 1 --2- 172.21.15.16:0/2490442001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93c0136fe0 0x7f93c01393c0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f93b8009e50 tx=0x7f93b800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89b66c0aa806f356 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:22.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.906+0000 7f93bd59a700 1 -- 172.21.15.16:0/2490442001 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b800e070 con 0x7f93c0136fe0 2023-09-30T22:34:22.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.906+0000 7f93bd59a700 1 -- 172.21.15.16:0/2490442001 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93b800b9c0 con 0x7f93c0136fe0 2023-09-30T22:34:22.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.906+0000 7f93bd59a700 1 -- 172.21.15.16:0/2490442001 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b800bd70 con 0x7f93c0136fe0 2023-09-30T22:34:22.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.907+0000 7f93c48a1700 1 -- 172.21.15.16:0/2490442001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93c0136fe0 msgr2=0x7f93c01393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:22.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.907+0000 7f93c48a1700 1 --2- 172.21.15.16:0/2490442001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93c0136fe0 0x7f93c01393c0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f93b8009e50 tx=0x7f93b800b330 comp rx=0 tx=0).stop 2023-09-30T22:34:22.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.907+0000 7f93c48a1700 1 -- 172.21.15.16:0/2490442001 shutdown_connections 2023-09-30T22:34:22.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.907+0000 7f93c48a1700 1 --2- 172.21.15.16:0/2490442001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93c0136fe0 0x7f93c01393c0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:22.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.907+0000 7f93c48a1700 1 --2- 172.21.15.16:0/2490442001 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f93c009f410 0x7f93c0136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:22.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.907+0000 7f93c48a1700 1 -- 172.21.15.16:0/2490442001 >> 172.21.15.16:0/2490442001 conn(0x7f93c00a4210 msgr2=0x7f93c00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:22.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.907+0000 7f93c48a1700 1 -- 172.21.15.16:0/2490442001 shutdown_connections 2023-09-30T22:34:22.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.908+0000 7f93c48a1700 1 -- 172.21.15.16:0/2490442001 wait complete. 2023-09-30T22:34:22.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.909+0000 7f93c48a1700 1 Processor -- start 2023-09-30T22:34:22.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.909+0000 7f93c48a1700 1 -- start start 2023-09-30T22:34:22.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.909+0000 7f93c48a1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f93c009e930 0x7f93c011e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:22.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.909+0000 7f93c48a1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f93c009f410 0x7f93c011eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:22.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.909+0000 7f93c48a1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93c0136fe0 0x7f93c011f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:22.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.909+0000 7f93c48a1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c0133ef0 con 0x7f93c0136fe0 2023-09-30T22:34:22.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.909+0000 7f93c48a1700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93c01201d0 con 0x7f93c009f410 2023-09-30T22:34:22.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.909+0000 7f93c48a1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f93c01344b0 con 0x7f93c009e930 2023-09-30T22:34:22.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.909+0000 7f93bed9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93c0136fe0 0x7f93c011f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:22.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.910+0000 7f93bed9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93c0136fe0 0x7f93c011f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47552/0 (socket says 172.21.15.16:47552) 2023-09-30T22:34:22.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.910+0000 7f93bed9d700 1 -- 172.21.15.16:0/2710102294 learned_addr learned my addr 172.21.15.16:0/2710102294 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:22.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.910+0000 7f93be59c700 1 --2- 172.21.15.16:0/2710102294 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f93c009e930 0x7f93c011e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:22.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.910+0000 7f93bdd9b700 1 --2- 172.21.15.16:0/2710102294 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f93c009f410 0x7f93c011eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:22.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.910+0000 7f93bed9d700 1 -- 172.21.15.16:0/2710102294 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f93c009e930 msgr2=0x7f93c011e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:22.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.910+0000 7f93bed9d700 1 --2- 172.21.15.16:0/2710102294 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f93c009e930 0x7f93c011e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:22.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.910+0000 7f93bed9d700 1 -- 172.21.15.16:0/2710102294 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f93c009f410 msgr2=0x7f93c011eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:22.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.910+0000 7f93bed9d700 1 --2- 172.21.15.16:0/2710102294 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f93c009f410 0x7f93c011eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:22.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.910+0000 7f93bed9d700 1 -- 172.21.15.16:0/2710102294 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b8009b30 con 0x7f93c0136fe0 2023-09-30T22:34:22.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.910+0000 7f93bdd9b700 1 --2- 172.21.15.16:0/2710102294 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f93c009f410 0x7f93c011eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:34:22.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.910+0000 7f93bed9d700 1 --2- 172.21.15.16:0/2710102294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93c0136fe0 0x7f93c011f040 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f93c006fea0 tx=0x7f93b8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:22.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.910+0000 7f93aeffd700 1 -- 172.21.15.16:0/2710102294 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b80048b0 con 0x7f93c0136fe0 2023-09-30T22:34:22.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.910+0000 7f93aeffd700 1 -- 172.21.15.16:0/2710102294 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93b80092e0 con 0x7f93c0136fe0 2023-09-30T22:34:22.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.910+0000 7f93c48a1700 1 -- 172.21.15.16:0/2710102294 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93c0071d20 con 0x7f93c0136fe0 2023-09-30T22:34:22.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.911+0000 7f93aeffd700 1 -- 172.21.15.16:0/2710102294 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b801c660 con 0x7f93c0136fe0 2023-09-30T22:34:22.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.911+0000 7f93c48a1700 1 -- 172.21.15.16:0/2710102294 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93c0121870 con 0x7f93c0136fe0 2023-09-30T22:34:22.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.912+0000 7f93aeffd700 1 -- 172.21.15.16:0/2710102294 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f93b801c7c0 con 0x7f93c0136fe0 2023-09-30T22:34:22.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.912+0000 7f93c48a1700 1 -- 172.21.15.16:0/2710102294 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93c00a2f40 con 0x7f93c0136fe0 2023-09-30T22:34:22.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.912+0000 7f93aeffd700 1 -- 172.21.15.16:0/2710102294 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f93b80412c0 con 0x7f93c0136fe0 2023-09-30T22:34:22.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.921+0000 7f93aeffd700 1 -- 172.21.15.16:0/2710102294 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f93b80257d0 con 0x7f93c0136fe0 2023-09-30T22:34:22.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.945+0000 7f05e1925700 1 Processor -- start 2023-09-30T22:34:22.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.945+0000 7f05e1925700 1 -- start start 2023-09-30T22:34:22.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.946+0000 7f05e1925700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f05dc09d6f0 0x7f05dc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:22.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.946+0000 7f05e1925700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f05dc09e410 0x7f05dc134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:22.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.946+0000 7f05e1925700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05dc09b120 con 0x7f05dc09e000 2023-09-30T22:34:22.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.946+0000 7f05e1925700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05dc06b460 con 0x7f05dc09d6f0 2023-09-30T22:34:22.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.946+0000 7f05e1925700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f05dc09a7e0 con 0x7f05dc09e410 2023-09-30T22:34:22.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.946+0000 7f05da7fc700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f05dc09e000 0x7f05dc1339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54922/0 (socket says 172.21.15.16:54922) 2023-09-30T22:34:22.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.946+0000 7f05da7fc700 1 -- 172.21.15.16:0/2456222925 learned_addr learned my addr 172.21.15.16:0/2456222925 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:34:22.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.946+0000 7f05d9ffb700 1 -- 172.21.15.16:0/2456222925 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 535823311 0 0) 0x7f05dc09b120 con 0x7f05dc09e000 2023-09-30T22:34:22.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.946+0000 7f05d9ffb700 1 -- 172.21.15.16:0/2456222925 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05c40035b0 con 0x7f05dc09e000 2023-09-30T22:34:22.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.947+0000 7f05daffd700 1 --2- 172.21.15.16:0/2456222925 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f05dc09d6f0 0x7f05dc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:22.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.947+0000 7f05db7fe700 1 --2- 172.21.15.16:0/2456222925 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f05dc09e410 0x7f05dc134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:22.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.947+0000 7f05d9ffb700 1 -- 172.21.15.16:0/2456222925 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 942834457 0 0) 0x7f05c40035b0 con 0x7f05dc09e000 2023-09-30T22:34:22.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.947+0000 7f05d9ffb700 1 -- 172.21.15.16:0/2456222925 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f05dc09e410 msgr2=0x7f05dc134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:22.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.947+0000 7f05d9ffb700 1 --2- 172.21.15.16:0/2456222925 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f05dc09e410 0x7f05dc134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:22.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.947+0000 7f05d9ffb700 1 -- 172.21.15.16:0/2456222925 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f05dc09d6f0 msgr2=0x7f05dc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:22.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.947+0000 7f05d9ffb700 1 --2- 172.21.15.16:0/2456222925 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f05dc09d6f0 0x7f05dc09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:22.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.947+0000 7f05d9ffb700 1 -- 172.21.15.16:0/2456222925 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05dc09a500 con 0x7f05dc09e000 2023-09-30T22:34:22.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.947+0000 7f05d9ffb700 1 -- 172.21.15.16:0/2456222925 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1074163509 0 0) 0x7f05d0001670 con 0x7f05dc09e000 2023-09-30T22:34:22.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.947+0000 7f05d9ffb700 1 -- 172.21.15.16:0/2456222925 >> v1:172.21.15.16:6789/0 conn(0x7f05dc09e000 legacy=0x7f05dc1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:22.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.947+0000 7f05d9ffb700 1 --2- 172.21.15.16:0/2456222925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05c4003a30 0x7f05c4003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:22.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.947+0000 7f05d9ffb700 1 --2- 172.21.15.16:0/2456222925 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f05dc09e410 0x7f05c4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:22.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.947+0000 7f05d9ffb700 1 --2- 172.21.15.16:0/2456222925 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f05dc09d6f0 0x7f05c4004d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:22.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.947+0000 7f05d9ffb700 1 -- 172.21.15.16:0/2456222925 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05dc06b460 con 0x7f05c4003a30 2023-09-30T22:34:22.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.947+0000 7f05d9ffb700 1 -- 172.21.15.16:0/2456222925 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05dc09a7e0 con 0x7f05dc09d6f0 2023-09-30T22:34:22.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.947+0000 7f05d9ffb700 1 -- 172.21.15.16:0/2456222925 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f05c4005420 con 0x7f05dc09e410 2023-09-30T22:34:22.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.947+0000 7f05da7fc700 1 --2- 172.21.15.16:0/2456222925 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f05dc09e410 0x7f05c4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:22.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.947+0000 7f05db7fe700 1 --2- 172.21.15.16:0/2456222925 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f05dc09d6f0 0x7f05c4004d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:22.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.947+0000 7f05daffd700 1 --2- 172.21.15.16:0/2456222925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05c4003a30 0x7f05c4003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:22.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.947+0000 7f05da7fc700 1 -- 172.21.15.16:0/2456222925 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f05dc09d6f0 msgr2=0x7f05c4004d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:22.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.947+0000 7f05da7fc700 1 --2- 172.21.15.16:0/2456222925 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f05dc09d6f0 0x7f05c4004d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:22.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.947+0000 7f05da7fc700 1 -- 172.21.15.16:0/2456222925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05c4003a30 msgr2=0x7f05c4003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:22.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.947+0000 7f05da7fc700 1 --2- 172.21.15.16:0/2456222925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05c4003a30 0x7f05c4003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:22.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.947+0000 7f05da7fc700 1 -- 172.21.15.16:0/2456222925 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05c40035b0 con 0x7f05dc09e410 2023-09-30T22:34:22.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.948+0000 7f05daffd700 1 --2- 172.21.15.16:0/2456222925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05c4003a30 0x7f05c4003e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:34:22.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.948+0000 7f05da7fc700 1 --2- 172.21.15.16:0/2456222925 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f05dc09e410 0x7f05c4004850 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f05d000c160 tx=0x7f05c4005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63ca05abb94e14ad server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:22.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.948+0000 7f05d9ffb700 1 -- 172.21.15.16:0/2456222925 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d0007b30 con 0x7f05dc09e410 2023-09-30T22:34:22.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.948+0000 7f05d9ffb700 1 -- 172.21.15.16:0/2456222925 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05d000cce0 con 0x7f05dc09e410 2023-09-30T22:34:22.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.948+0000 7f05d9ffb700 1 -- 172.21.15.16:0/2456222925 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d0005ba0 con 0x7f05dc09e410 2023-09-30T22:34:22.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.949+0000 7f05e1925700 1 -- 172.21.15.16:0/2456222925 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f05dc09e410 msgr2=0x7f05c4004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:22.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.949+0000 7f05e1925700 1 --2- 172.21.15.16:0/2456222925 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f05dc09e410 0x7f05c4004850 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f05d000c160 tx=0x7f05c4005880 comp rx=0 tx=0).stop 2023-09-30T22:34:22.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.949+0000 7f05e1925700 1 -- 172.21.15.16:0/2456222925 shutdown_connections 2023-09-30T22:34:22.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.949+0000 7f05e1925700 1 --2- 172.21.15.16:0/2456222925 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f05dc09d6f0 0x7f05c4004d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:22.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.949+0000 7f05e1925700 1 --2- 172.21.15.16:0/2456222925 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f05dc09e410 0x7f05c4004850 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:22.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.949+0000 7f05e1925700 1 --2- 172.21.15.16:0/2456222925 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05c4003a30 0x7f05c4003e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:22.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.949+0000 7f05e1925700 1 -- 172.21.15.16:0/2456222925 >> 172.21.15.16:0/2456222925 conn(0x7f05dc066f50 msgr2=0x7f05dc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:22.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.949+0000 7f05e1925700 1 -- 172.21.15.16:0/2456222925 shutdown_connections 2023-09-30T22:34:22.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.949+0000 7f05e1925700 1 -- 172.21.15.16:0/2456222925 wait complete. 2023-09-30T22:34:22.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.950+0000 7f05e1925700 1 Processor -- start 2023-09-30T22:34:22.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.950+0000 7f0a0a4ca700 1 Processor -- start 2023-09-30T22:34:22.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.950+0000 7f05e1925700 1 -- start start 2023-09-30T22:34:22.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.950+0000 7f0a0a4ca700 1 -- start start 2023-09-30T22:34:22.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.951+0000 7f05e1925700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05c4003a30 0x7f05dc1a7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:22.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.951+0000 7f05e1925700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f05dc09d6f0 0x7f05dc1a78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:22.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.951+0000 7f05e1925700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f05dc09e000 0x7f05dc1c7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:22.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.951+0000 7f05e1925700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05dc1aa540 con 0x7f05c4003a30 2023-09-30T22:34:22.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.951+0000 7f05e1925700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05dc1bb3b0 con 0x7f05dc09d6f0 2023-09-30T22:34:22.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.951+0000 7f05e1925700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f05dc1a6f00 con 0x7f05dc09e000 2023-09-30T22:34:22.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.951+0000 7f0a0a4ca700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0a0406f5d0 0x7f0a0412cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:22.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.951+0000 7f0a0a4ca700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0a04066410 0x7f0a0412f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:22.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.951+0000 7f0a0a4ca700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0a04066880 0x7f0a0412fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:22.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.951+0000 7f0a0a4ca700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a0412d360 con 0x7f0a04066880 2023-09-30T22:34:22.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.951+0000 7f0a0a4ca700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a04071780 con 0x7f0a0406f5d0 2023-09-30T22:34:22.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.951+0000 7f0a0a4ca700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0406f9a0 con 0x7f0a04066410 2023-09-30T22:34:22.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.951+0000 7f05da7fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f05dc09d6f0 0x7f05dc1a78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:22.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.951+0000 7f05da7fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f05dc09d6f0 0x7f05dc1a78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:55526/0 (socket says 172.21.15.16:55526) 2023-09-30T22:34:22.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.951+0000 7f05da7fc700 1 -- 172.21.15.16:0/2252661864 learned_addr learned my addr 172.21.15.16:0/2252661864 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:22.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.951+0000 7f0a03fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0a0406f5d0 0x7f0a0412cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:22.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.951+0000 7f0a03fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0a0406f5d0 0x7f0a0412cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:55524/0 (socket says 172.21.15.16:55524) 2023-09-30T22:34:22.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.951+0000 7f0a03fff700 1 -- 172.21.15.16:0/3923305923 learned_addr learned my addr 172.21.15.16:0/3923305923 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:22.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.951+0000 7f0a037fe700 1 --2- 172.21.15.16:0/3923305923 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0a04066410 0x7f0a0412f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:22.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.951+0000 7f0a08a67700 1 --2- 172.21.15.16:0/3923305923 >> v2:172.21.15.16:3300/0 conn(0x7f0a04066880 0x7f0a0412fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:22.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.951+0000 7f05db7fe700 1 --2- 172.21.15.16:0/2252661864 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f05dc09e000 0x7f05dc1c7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:22.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.951+0000 7f05da7fc700 1 -- 172.21.15.16:0/2252661864 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f05dc09e000 msgr2=0x7f05dc1c7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:22.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.951+0000 7f05da7fc700 1 --2- 172.21.15.16:0/2252661864 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f05dc09e000 0x7f05dc1c7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:22.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.951+0000 7f05da7fc700 1 -- 172.21.15.16:0/2252661864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05c4003a30 msgr2=0x7f05dc1a7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:22.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.951+0000 7f05da7fc700 1 --2- 172.21.15.16:0/2252661864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05c4003a30 0x7f05dc1a7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:22.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.951+0000 7f05da7fc700 1 -- 172.21.15.16:0/2252661864 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05dc1a9590 con 0x7f05dc09d6f0 2023-09-30T22:34:22.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.952+0000 7f0a08a67700 1 -- 172.21.15.16:0/3923305923 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0a04066410 msgr2=0x7f0a0412f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:22.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.952+0000 7f0a08a67700 1 --2- 172.21.15.16:0/3923305923 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0a04066410 0x7f0a0412f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:22.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.952+0000 7f0a08a67700 1 -- 172.21.15.16:0/3923305923 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0a0406f5d0 msgr2=0x7f0a0412cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:22.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.952+0000 7f0a08a67700 1 --2- 172.21.15.16:0/3923305923 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0a0406f5d0 0x7f0a0412cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:22.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.952+0000 7f0a08a67700 1 -- 172.21.15.16:0/3923305923 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a04071a40 con 0x7f0a04066880 2023-09-30T22:34:22.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.952+0000 7f0a08a67700 1 --2- 172.21.15.16:0/3923305923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a04066880 0x7f0a0412fdf0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f09fc009d90 tx=0x7f09fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1e4ab504fc62860 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:22.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.952+0000 7f05da7fc700 1 --2- 172.21.15.16:0/2252661864 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f05dc09d6f0 0x7f05dc1a78c0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f05d000c750 tx=0x7f05dc09b3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:22.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.952+0000 7f05c3fff700 1 -- 172.21.15.16:0/2252661864 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d0004d40 con 0x7f05dc09d6f0 2023-09-30T22:34:22.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.952+0000 7f05c3fff700 1 -- 172.21.15.16:0/2252661864 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05d00070d0 con 0x7f05dc09d6f0 2023-09-30T22:34:22.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.952+0000 7f05e1925700 1 -- 172.21.15.16:0/2252661864 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05dc072250 con 0x7f05dc09d6f0 2023-09-30T22:34:22.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.952+0000 7f05c3fff700 1 -- 172.21.15.16:0/2252661864 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d0004ea0 con 0x7f05dc09d6f0 2023-09-30T22:34:22.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.952+0000 7f05e1925700 1 -- 172.21.15.16:0/2252661864 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05dc1a7e00 con 0x7f05dc09d6f0 2023-09-30T22:34:22.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.953+0000 7f0a02ffd700 1 -- 172.21.15.16:0/3923305923 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc00e070 con 0x7f0a04066880 2023-09-30T22:34:22.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.953+0000 7f05c3fff700 1 -- 172.21.15.16:0/2252661864 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f05d0007440 con 0x7f05dc09d6f0 2023-09-30T22:34:22.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.953+0000 7f05c3fff700 1 -- 172.21.15.16:0/2252661864 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f05d0040b30 con 0x7f05dc09d6f0 2023-09-30T22:34:22.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.953+0000 7f05c1ffb700 1 -- 172.21.15.16:0/2252661864 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05b40052e0 con 0x7f05dc09d6f0 2023-09-30T22:34:22.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.953+0000 7f0a02ffd700 1 -- 172.21.15.16:0/3923305923 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09fc00bb20 con 0x7f0a04066880 2023-09-30T22:34:22.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.953+0000 7f0a02ffd700 1 -- 172.21.15.16:0/3923305923 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc012430 con 0x7f0a04066880 2023-09-30T22:34:22.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.954+0000 7f0a0a4ca700 1 -- 172.21.15.16:0/3923305923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a04066880 msgr2=0x7f0a0412fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:22.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.954+0000 7f0a0a4ca700 1 --2- 172.21.15.16:0/3923305923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a04066880 0x7f0a0412fdf0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f09fc009d90 tx=0x7f09fc00b330 comp rx=0 tx=0).stop 2023-09-30T22:34:22.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.954+0000 7f0a0a4ca700 1 -- 172.21.15.16:0/3923305923 shutdown_connections 2023-09-30T22:34:22.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.954+0000 7f0a0a4ca700 1 --2- 172.21.15.16:0/3923305923 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0a0406f5d0 0x7f0a0412cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:22.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.954+0000 7f0a0a4ca700 1 --2- 172.21.15.16:0/3923305923 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a04066880 0x7f0a0412fdf0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:22.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.954+0000 7f0a0a4ca700 1 --2- 172.21.15.16:0/3923305923 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0a04066410 0x7f0a0412f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:22.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.954+0000 7f0a0a4ca700 1 -- 172.21.15.16:0/3923305923 >> 172.21.15.16:0/3923305923 conn(0x7f0a040a2aa0 msgr2=0x7f0a040a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:22.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.955+0000 7f0a0a4ca700 1 -- 172.21.15.16:0/3923305923 shutdown_connections 2023-09-30T22:34:22.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.955+0000 7f0a0a4ca700 1 -- 172.21.15.16:0/3923305923 wait complete. 2023-09-30T22:34:22.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.956+0000 7f0a0a4ca700 1 Processor -- start 2023-09-30T22:34:22.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.956+0000 7f0a0a4ca700 1 -- start start 2023-09-30T22:34:22.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.957+0000 7f0a0a4ca700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0a04066410 0x7f0a0412f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:22.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.957+0000 7f0a0a4ca700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0a04066880 0x7f0a0412fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:22.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.957+0000 7f0a0a4ca700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a0406f5d0 0x7f0a041302b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:22.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.957+0000 7f0a0a4ca700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a041332c0 con 0x7f0a0406f5d0 2023-09-30T22:34:22.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.957+0000 7f0a0a4ca700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a04131450 con 0x7f0a04066410 2023-09-30T22:34:22.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.957+0000 7f0a0a4ca700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0413dcc0 con 0x7f0a04066880 2023-09-30T22:34:22.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.957+0000 7f0a037fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0a04066880 0x7f0a0412fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:22.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.957+0000 7f0a037fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0a04066880 0x7f0a0412fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:48188/0 (socket says 172.21.15.16:48188) 2023-09-30T22:34:22.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.957+0000 7f0a037fe700 1 -- 172.21.15.16:0/1613364223 learned_addr learned my addr 172.21.15.16:0/1613364223 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:22.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.957+0000 7f0a08a67700 1 --2- 172.21.15.16:0/1613364223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a0406f5d0 0x7f0a041302b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:22.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.957+0000 7f0a03fff700 1 --2- 172.21.15.16:0/1613364223 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0a04066410 0x7f0a0412f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:22.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.957+0000 7f0a037fe700 1 -- 172.21.15.16:0/1613364223 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0a04066410 msgr2=0x7f0a0412f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:22.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.957+0000 7f0a037fe700 1 --2- 172.21.15.16:0/1613364223 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0a04066410 0x7f0a0412f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:22.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.957+0000 7f0a037fe700 1 -- 172.21.15.16:0/1613364223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a0406f5d0 msgr2=0x7f0a041302b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:22.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.958+0000 7f0a037fe700 1 --2- 172.21.15.16:0/1613364223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a0406f5d0 0x7f0a041302b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:22.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.958+0000 7f0a037fe700 1 -- 172.21.15.16:0/1613364223 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09fc009a70 con 0x7f0a04066880 2023-09-30T22:34:22.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.958+0000 7f0a037fe700 1 --2- 172.21.15.16:0/1613364223 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0a04066880 0x7f0a0412fd70 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f09f800e7e0 tx=0x7f09f800eaf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:22.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.958+0000 7f0a00ff9700 1 -- 172.21.15.16:0/1613364223 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f800c070 con 0x7f0a04066880 2023-09-30T22:34:22.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.958+0000 7f0a00ff9700 1 -- 172.21.15.16:0/1613364223 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09f8013610 con 0x7f0a04066880 2023-09-30T22:34:22.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.958+0000 7f0a0a4ca700 1 -- 172.21.15.16:0/1613364223 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a0412a000 con 0x7f0a04066880 2023-09-30T22:34:22.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.958+0000 7f0a00ff9700 1 -- 172.21.15.16:0/1613364223 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f8013980 con 0x7f0a04066880 2023-09-30T22:34:22.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.958+0000 7f0a0a4ca700 1 -- 172.21.15.16:0/1613364223 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a041339c0 con 0x7f0a04066880 2023-09-30T22:34:22.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.959+0000 7f0a00ff9700 1 -- 172.21.15.16:0/1613364223 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f09f8013ae0 con 0x7f0a04066880 2023-09-30T22:34:22.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.959+0000 7f0a0a4ca700 1 -- 172.21.15.16:0/1613364223 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a04074ea0 con 0x7f0a04066880 2023-09-30T22:34:22.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.959+0000 7f0a00ff9700 1 -- 172.21.15.16:0/1613364223 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f09f803f3b0 con 0x7f0a04066880 2023-09-30T22:34:22.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.959+0000 7f05c3fff700 1 -- 172.21.15.16:0/2252661864 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f05d00076f0 con 0x7f05dc09d6f0 2023-09-30T22:34:22.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:22.964+0000 7f0a00ff9700 1 -- 172.21.15.16:0/1613364223 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f09f80104f0 con 0x7f0a04066880 2023-09-30T22:34:23.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:23.051+0000 7f3c84b68700 1 -- 172.21.15.16:0/3169360877 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3c80074ea0 con 0x7f3c801338c0 2023-09-30T22:34:23.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:23.051+0000 7f3c6effd700 1 -- 172.21.15.16:0/3169360877 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3c70017070 con 0x7f3c801338c0 2023-09-30T22:34:23.052 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:34:23.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:23.054+0000 7f3c84b68700 1 -- 172.21.15.16:0/3169360877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c801338c0 msgr2=0x7f3c801c8560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:23.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:23.054+0000 7f3c84b68700 1 --2- 172.21.15.16:0/3169360877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c801338c0 0x7f3c801c8560 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f3c7000c700 tx=0x7f3c7000ca10 comp rx=0 tx=0).stop 2023-09-30T22:34:23.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:23.054+0000 7f3c84b68700 1 -- 172.21.15.16:0/3169360877 shutdown_connections 2023-09-30T22:34:23.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:23.054+0000 7f3c84b68700 1 --2- 172.21.15.16:0/3169360877 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3c80139150 0x7f3c80119e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:23.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:23.054+0000 7f3c84b68700 1 --2- 172.21.15.16:0/3169360877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c801338c0 0x7f3c801c8560 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:23.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:23.054+0000 7f3c84b68700 1 --2- 172.21.15.16:0/3169360877 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3c80136830 0x7f3c801c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:23.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:23.054+0000 7f3c84b68700 1 -- 172.21.15.16:0/3169360877 >> 172.21.15.16:0/3169360877 conn(0x7f3c800a2aa0 msgr2=0x7f3c80134d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:23.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:23.054+0000 7f3c84b68700 1 -- 172.21.15.16:0/3169360877 shutdown_connections 2023-09-30T22:34:23.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:23.054+0000 7f3c84b68700 1 -- 172.21.15.16:0/3169360877 wait complete. 2023-09-30T22:34:23.068 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:34:23.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:23.132+0000 7f93c48a1700 1 -- 172.21.15.16:0/2710102294 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f93c0074ea0 con 0x7f93c0136fe0 2023-09-30T22:34:23.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:23.132+0000 7f93aeffd700 1 -- 172.21.15.16:0/2710102294 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f93b8034090 con 0x7f93c0136fe0 2023-09-30T22:34:23.133 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:34:23.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:23.134+0000 7f93c48a1700 1 -- 172.21.15.16:0/2710102294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93c0136fe0 msgr2=0x7f93c011f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:23.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:23.134+0000 7f93c48a1700 1 --2- 172.21.15.16:0/2710102294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93c0136fe0 0x7f93c011f040 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f93c006fea0 tx=0x7f93b8004ad0 comp rx=0 tx=0).stop 2023-09-30T22:34:23.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:23.135+0000 7f93c48a1700 1 -- 172.21.15.16:0/2710102294 shutdown_connections 2023-09-30T22:34:23.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:23.135+0000 7f93c48a1700 1 --2- 172.21.15.16:0/2710102294 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93c0136fe0 0x7f93c011f040 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:23.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:23.135+0000 7f93c48a1700 1 --2- 172.21.15.16:0/2710102294 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f93c009e930 0x7f93c011e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:23.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:23.135+0000 7f93c48a1700 1 --2- 172.21.15.16:0/2710102294 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f93c009f410 0x7f93c011eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:23.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:23.135+0000 7f93c48a1700 1 -- 172.21.15.16:0/2710102294 >> 172.21.15.16:0/2710102294 conn(0x7f93c00a4210 msgr2=0x7f93c00a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:23.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:23.135+0000 7f93c48a1700 1 -- 172.21.15.16:0/2710102294 shutdown_connections 2023-09-30T22:34:23.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:23.135+0000 7f93c48a1700 1 -- 172.21.15.16:0/2710102294 wait complete. 2023-09-30T22:34:23.149 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:34:23.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:23.187+0000 7f05c1ffb700 1 -- 172.21.15.16:0/2252661864 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f05b4005590 con 0x7f05dc09d6f0 2023-09-30T22:34:23.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:23.188+0000 7f05c3fff700 1 -- 172.21.15.16:0/2252661864 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f05d002b030 con 0x7f05dc09d6f0 2023-09-30T22:34:23.189 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:34:23.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:23.190+0000 7f05c1ffb700 1 -- 172.21.15.16:0/2252661864 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f05dc09d6f0 msgr2=0x7f05dc1a78c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:23.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:23.190+0000 7f05c1ffb700 1 --2- 172.21.15.16:0/2252661864 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f05dc09d6f0 0x7f05dc1a78c0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f05d000c750 tx=0x7f05dc09b3c0 comp rx=0 tx=0).stop 2023-09-30T22:34:23.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:23.190+0000 7f05c1ffb700 1 -- 172.21.15.16:0/2252661864 shutdown_connections 2023-09-30T22:34:23.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:23.190+0000 7f05c1ffb700 1 --2- 172.21.15.16:0/2252661864 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f05dc09e000 0x7f05dc1c7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:23.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:23.190+0000 7f05c1ffb700 1 --2- 172.21.15.16:0/2252661864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05c4003a30 0x7f05dc1a7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:23.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:23.190+0000 7f05c1ffb700 1 --2- 172.21.15.16:0/2252661864 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f05dc09d6f0 0x7f05dc1a78c0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:23.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:23.190+0000 7f05c1ffb700 1 -- 172.21.15.16:0/2252661864 >> 172.21.15.16:0/2252661864 conn(0x7f05dc066f50 msgr2=0x7f05dc1bb7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:23.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:23.190+0000 7f05c1ffb700 1 -- 172.21.15.16:0/2252661864 shutdown_connections 2023-09-30T22:34:23.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:23.191+0000 7f05c1ffb700 1 -- 172.21.15.16:0/2252661864 wait complete. 2023-09-30T22:34:23.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:23.192+0000 7f0a0a4ca700 1 -- 172.21.15.16:0/1613364223 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f0a04132c80 con 0x7f0a04066880 2023-09-30T22:34:23.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:23.192+0000 7f0a00ff9700 1 -- 172.21.15.16:0/1613364223 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f09f8010310 con 0x7f0a04066880 2023-09-30T22:34:23.193 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:34:23.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:23.195+0000 7f0a0a4ca700 1 -- 172.21.15.16:0/1613364223 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0a04066880 msgr2=0x7f0a0412fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:23.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:23.195+0000 7f0a0a4ca700 1 --2- 172.21.15.16:0/1613364223 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0a04066880 0x7f0a0412fd70 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f09f800e7e0 tx=0x7f09f800eaf0 comp rx=0 tx=0).stop 2023-09-30T22:34:23.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:23.195+0000 7f0a0a4ca700 1 -- 172.21.15.16:0/1613364223 shutdown_connections 2023-09-30T22:34:23.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:23.195+0000 7f0a0a4ca700 1 --2- 172.21.15.16:0/1613364223 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0a04066410 0x7f0a0412f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:23.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:23.195+0000 7f0a0a4ca700 1 --2- 172.21.15.16:0/1613364223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a0406f5d0 0x7f0a041302b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:23.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:23.195+0000 7f0a0a4ca700 1 --2- 172.21.15.16:0/1613364223 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0a04066880 0x7f0a0412fd70 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:23.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:23.195+0000 7f0a0a4ca700 1 -- 172.21.15.16:0/1613364223 >> 172.21.15.16:0/1613364223 conn(0x7f0a040a2aa0 msgr2=0x7f0a0409f180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:23.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:23.195+0000 7f0a0a4ca700 1 -- 172.21.15.16:0/1613364223 shutdown_connections 2023-09-30T22:34:23.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:23.195+0000 7f0a0a4ca700 1 -- 172.21.15.16:0/1613364223 wait complete. 2023-09-30T22:34:23.206 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:34:23.210 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:34:24.069 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:34:24.150 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:34:24.207 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:34:24.211 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:34:24.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.217+0000 7f70dfc4d700 1 Processor -- start 2023-09-30T22:34:24.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.218+0000 7f70dfc4d700 1 -- start start 2023-09-30T22:34:24.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.218+0000 7f70dfc4d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f70d809d6f0 0x7f70d809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:24.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.218+0000 7f70dfc4d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f70d809e440 0x7f70d8134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:24.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.219+0000 7f70dfc4d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70d809b120 con 0x7f70d809e000 2023-09-30T22:34:24.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.219+0000 7f70dfc4d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70d806b460 con 0x7f70d809d6f0 2023-09-30T22:34:24.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.219+0000 7f70dfc4d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f70d809a7e0 con 0x7f70d809e440 2023-09-30T22:34:24.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.219+0000 7f70dd9e9700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f70d809d6f0 0x7f70d809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:24.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.219+0000 7f70dd9e9700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f70d809d6f0 0x7f70d809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47598/0 (socket says 172.21.15.16:47598) 2023-09-30T22:34:24.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.219+0000 7f70dd1e8700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f70d809e000 0x7f70d81339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54928/0 (socket says 172.21.15.16:54928) 2023-09-30T22:34:24.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.219+0000 7f70dd9e9700 1 -- 172.21.15.16:0/654742398 learned_addr learned my addr 172.21.15.16:0/654742398 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:24.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.219+0000 7f70dd9e9700 1 -- 172.21.15.16:0/654742398 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f70d809e440 msgr2=0x7f70d8134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:24.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.219+0000 7f70dd9e9700 1 --2- 172.21.15.16:0/654742398 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f70d809e440 0x7f70d8134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:24.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.219+0000 7f70dd9e9700 1 -- 172.21.15.16:0/654742398 >> v1:172.21.15.16:6789/0 conn(0x7f70d809e000 legacy=0x7f70d81339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:24.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.219+0000 7f70dc9e7700 1 -- 172.21.15.16:0/654742398 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4176606013 0 0) 0x7f70d809b120 con 0x7f70d809e000 2023-09-30T22:34:24.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.219+0000 7f70dd9e9700 1 -- 172.21.15.16:0/654742398 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d809a500 con 0x7f70d809d6f0 2023-09-30T22:34:24.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.219+0000 7f70dd9e9700 1 --2- 172.21.15.16:0/654742398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70d809d6f0 0x7f70d809dac0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f70d4009920 tx=0x7f70d4009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea5cccca3ba252ec server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:24.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.220+0000 7f70dc9e7700 1 -- 172.21.15.16:0/654742398 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d400ba20 con 0x7f70d809d6f0 2023-09-30T22:34:24.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.220+0000 7f70dc9e7700 1 -- 172.21.15.16:0/654742398 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d400bb80 con 0x7f70d809d6f0 2023-09-30T22:34:24.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.220+0000 7f70dc9e7700 1 -- 172.21.15.16:0/654742398 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d4009e70 con 0x7f70d809d6f0 2023-09-30T22:34:24.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.221+0000 7f70dfc4d700 1 -- 172.21.15.16:0/654742398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70d809d6f0 msgr2=0x7f70d809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:24.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.221+0000 7f70dfc4d700 1 --2- 172.21.15.16:0/654742398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70d809d6f0 0x7f70d809dac0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f70d4009920 tx=0x7f70d4009c30 comp rx=0 tx=0).stop 2023-09-30T22:34:24.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.221+0000 7f70dfc4d700 1 -- 172.21.15.16:0/654742398 shutdown_connections 2023-09-30T22:34:24.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.221+0000 7f70dfc4d700 1 --2- 172.21.15.16:0/654742398 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f70d809e440 0x7f70d8134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:24.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.221+0000 7f70dfc4d700 1 --2- 172.21.15.16:0/654742398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70d809d6f0 0x7f70d809dac0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:24.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.221+0000 7f70dfc4d700 1 -- 172.21.15.16:0/654742398 >> 172.21.15.16:0/654742398 conn(0x7f70d8066f50 msgr2=0x7f70d809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:24.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.221+0000 7f70dfc4d700 1 -- 172.21.15.16:0/654742398 shutdown_connections 2023-09-30T22:34:24.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.221+0000 7f70dfc4d700 1 -- 172.21.15.16:0/654742398 wait complete. 2023-09-30T22:34:24.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.223+0000 7f70dfc4d700 1 Processor -- start 2023-09-30T22:34:24.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.223+0000 7f70dfc4d700 1 -- start start 2023-09-30T22:34:24.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.223+0000 7f70dfc4d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f70d809d6f0 0x7f70d81a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:24.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.223+0000 7f70dfc4d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f70d809e000 0x7f70d81a5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:24.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.223+0000 7f70dfc4d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70d809e440 0x7f70d81c7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:24.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.223+0000 7f70dfc4d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d81a8200 con 0x7f70d809e440 2023-09-30T22:34:24.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.223+0000 7f70dfc4d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70d81a8370 con 0x7f70d809e000 2023-09-30T22:34:24.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.223+0000 7f70dfc4d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f70d81a7060 con 0x7f70d809d6f0 2023-09-30T22:34:24.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.223+0000 7f70dd9e9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f70d809d6f0 0x7f70d81a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:24.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.224+0000 7f70dd9e9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f70d809d6f0 0x7f70d81a51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:48210/0 (socket says 172.21.15.16:48210) 2023-09-30T22:34:24.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.224+0000 7f70dd9e9700 1 -- 172.21.15.16:0/132943733 learned_addr learned my addr 172.21.15.16:0/132943733 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:24.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.224+0000 7f70dd1e8700 1 --2- 172.21.15.16:0/132943733 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f70d809e000 0x7f70d81a5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:24.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.224+0000 7f70de1ea700 1 --2- 172.21.15.16:0/132943733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70d809e440 0x7f70d81c7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:24.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.224+0000 7f70dd9e9700 1 -- 172.21.15.16:0/132943733 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f70d809e000 msgr2=0x7f70d81a5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:24.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.224+0000 7f70dd9e9700 1 --2- 172.21.15.16:0/132943733 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f70d809e000 0x7f70d81a5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:24.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.224+0000 7f70dd9e9700 1 -- 172.21.15.16:0/132943733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70d809e440 msgr2=0x7f70d81c7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:24.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.224+0000 7f70dd9e9700 1 --2- 172.21.15.16:0/132943733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70d809e440 0x7f70d81c7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:24.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.224+0000 7f70dd9e9700 1 -- 172.21.15.16:0/132943733 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d4009550 con 0x7f70d809d6f0 2023-09-30T22:34:24.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.224+0000 7f70de1ea700 1 --2- 172.21.15.16:0/132943733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70d809e440 0x7f70d81c7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:34:24.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.224+0000 7f70dd9e9700 1 --2- 172.21.15.16:0/132943733 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f70d809d6f0 0x7f70d81a51e0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f70d400b3c0 tx=0x7f70d4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:24.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.224+0000 7f70ce7fc700 1 -- 172.21.15.16:0/132943733 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d40036a0 con 0x7f70d809d6f0 2023-09-30T22:34:24.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.224+0000 7f70ce7fc700 1 -- 172.21.15.16:0/132943733 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d4003820 con 0x7f70d809d6f0 2023-09-30T22:34:24.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.225+0000 7f70dfc4d700 1 -- 172.21.15.16:0/132943733 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70d81a4d00 con 0x7f70d809d6f0 2023-09-30T22:34:24.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.225+0000 7f70ce7fc700 1 -- 172.21.15.16:0/132943733 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d4003bd0 con 0x7f70d809d6f0 2023-09-30T22:34:24.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.225+0000 7f70dfc4d700 1 -- 172.21.15.16:0/132943733 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70d81a8990 con 0x7f70d809d6f0 2023-09-30T22:34:24.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.226+0000 7f70ce7fc700 1 -- 172.21.15.16:0/132943733 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f70d4003d30 con 0x7f70d809d6f0 2023-09-30T22:34:24.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.226+0000 7f70dfc4d700 1 -- 172.21.15.16:0/132943733 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70d8074ea0 con 0x7f70d809d6f0 2023-09-30T22:34:24.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.226+0000 7f70ce7fc700 1 -- 172.21.15.16:0/132943733 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f70d4040a20 con 0x7f70d809d6f0 2023-09-30T22:34:24.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.233+0000 7f70ce7fc700 1 -- 172.21.15.16:0/132943733 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f70d402b360 con 0x7f70d809d6f0 2023-09-30T22:34:24.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.295+0000 7f5b44d15700 1 Processor -- start 2023-09-30T22:34:24.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.295+0000 7f5b44d15700 1 -- start start 2023-09-30T22:34:24.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.296+0000 7f5b44d15700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5b40134630 0x7f5b40136a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:24.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.296+0000 7f5b44d15700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5b40136f50 0x7f5b40139330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:24.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.296+0000 7f5b44d15700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b4009b4f0 con 0x7f5b40131810 2023-09-30T22:34:24.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.296+0000 7f5b44d15700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b401313c0 con 0x7f5b40134630 2023-09-30T22:34:24.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.296+0000 7f5b44d15700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4006e8c0 con 0x7f5b40136f50 2023-09-30T22:34:24.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.296+0000 7f5b3dd9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5b40134630 0x7f5b40136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:24.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.296+0000 7f5b3ed9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5b40136f50 0x7f5b40139330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:24.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.296+0000 7f5b3dd9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5b40134630 0x7f5b40136a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:55542/0 (socket says 172.21.15.16:55542) 2023-09-30T22:34:24.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.296+0000 7f5b3dd9b700 1 -- 172.21.15.16:0/3893756255 learned_addr learned my addr 172.21.15.16:0/3893756255 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:24.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.297+0000 7f5b3dd9b700 1 -- 172.21.15.16:0/3893756255 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5b40136f50 msgr2=0x7f5b40139330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:24.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.297+0000 7f5b3dd9b700 1 --2- 172.21.15.16:0/3893756255 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5b40136f50 0x7f5b40139330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:24.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.297+0000 7f5b3dd9b700 1 -- 172.21.15.16:0/3893756255 >> v1:172.21.15.16:6789/0 conn(0x7f5b40131810 legacy=0x7f5b40131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:24.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.297+0000 7f5b3dd9b700 1 -- 172.21.15.16:0/3893756255 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b40071d20 con 0x7f5b40134630 2023-09-30T22:34:24.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.297+0000 7f5b3dd9b700 1 --2- 172.21.15.16:0/3893756255 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5b40134630 0x7f5b40136a10 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f5b34009920 tx=0x7f5b34009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4eb4df8f4ee5fde server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:24.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.297+0000 7f5b3d59a700 1 -- 172.21.15.16:0/3893756255 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b34011830 con 0x7f5b40134630 2023-09-30T22:34:24.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.297+0000 7f5b3d59a700 1 -- 172.21.15.16:0/3893756255 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b34011990 con 0x7f5b40134630 2023-09-30T22:34:24.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.297+0000 7f5b3d59a700 1 -- 172.21.15.16:0/3893756255 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b34011d20 con 0x7f5b40134630 2023-09-30T22:34:24.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.298+0000 7f5b44d15700 1 -- 172.21.15.16:0/3893756255 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5b40134630 msgr2=0x7f5b40136a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:24.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.298+0000 7f5b44d15700 1 --2- 172.21.15.16:0/3893756255 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5b40134630 0x7f5b40136a10 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f5b34009920 tx=0x7f5b34009c30 comp rx=0 tx=0).stop 2023-09-30T22:34:24.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.299+0000 7f5b44d15700 1 -- 172.21.15.16:0/3893756255 shutdown_connections 2023-09-30T22:34:24.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.299+0000 7f5b44d15700 1 --2- 172.21.15.16:0/3893756255 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5b40136f50 0x7f5b40139330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:24.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.299+0000 7f5b44d15700 1 --2- 172.21.15.16:0/3893756255 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5b40134630 0x7f5b40136a10 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:24.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.299+0000 7f5b44d15700 1 -- 172.21.15.16:0/3893756255 >> 172.21.15.16:0/3893756255 conn(0x7f5b400a2aa0 msgr2=0x7f5b400a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:24.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.299+0000 7f5b44d15700 1 -- 172.21.15.16:0/3893756255 shutdown_connections 2023-09-30T22:34:24.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.299+0000 7f5b44d15700 1 -- 172.21.15.16:0/3893756255 wait complete. 2023-09-30T22:34:24.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.300+0000 7f5b44d15700 1 Processor -- start 2023-09-30T22:34:24.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.301+0000 7f5b44d15700 1 -- start start 2023-09-30T22:34:24.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.301+0000 7f5b44d15700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5b40131810 0x7f5b4009ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:24.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.301+0000 7f5b44d15700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5b40134630 0x7f5b4009cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:24.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.301+0000 7f5b44d15700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b40136f50 0x7f5b4009d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:24.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.301+0000 7f5b44d15700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4009e290 con 0x7f5b40136f50 2023-09-30T22:34:24.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.301+0000 7f5b44d15700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b40118d80 con 0x7f5b40131810 2023-09-30T22:34:24.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.301+0000 7f5b44d15700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4006eb70 con 0x7f5b40134630 2023-09-30T22:34:24.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.301+0000 7f5b3ed9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b40136f50 0x7f5b4009d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:24.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.301+0000 7f5b3ed9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b40136f50 0x7f5b4009d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47610/0 (socket says 172.21.15.16:47610) 2023-09-30T22:34:24.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.301+0000 7f5b3ed9d700 1 -- 172.21.15.16:0/3357286567 learned_addr learned my addr 172.21.15.16:0/3357286567 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:24.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.302+0000 7f5b3dd9b700 1 --2- 172.21.15.16:0/3357286567 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5b40134630 0x7f5b4009cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:24.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.302+0000 7f5b3e59c700 1 --2- 172.21.15.16:0/3357286567 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5b40131810 0x7f5b4009ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:24.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.302+0000 7f5b3e59c700 1 -- 172.21.15.16:0/3357286567 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5b40134630 msgr2=0x7f5b4009cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:24.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.302+0000 7f5b3e59c700 1 --2- 172.21.15.16:0/3357286567 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5b40134630 0x7f5b4009cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:24.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.302+0000 7f5b3e59c700 1 -- 172.21.15.16:0/3357286567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b40136f50 msgr2=0x7f5b4009d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:24.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.302+0000 7f5b3e59c700 1 --2- 172.21.15.16:0/3357286567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b40136f50 0x7f5b4009d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:24.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.302+0000 7f5b3e59c700 1 -- 172.21.15.16:0/3357286567 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b34009550 con 0x7f5b40131810 2023-09-30T22:34:24.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.303+0000 7f5b3e59c700 1 --2- 172.21.15.16:0/3357286567 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5b40131810 0x7f5b4009ca50 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f5b3000c450 tx=0x7f5b3000c760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:24.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.303+0000 7f5b2effd700 1 -- 172.21.15.16:0/3357286567 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b300052d0 con 0x7f5b40131810 2023-09-30T22:34:24.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.303+0000 7f5b2effd700 1 -- 172.21.15.16:0/3357286567 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b30005430 con 0x7f5b40131810 2023-09-30T22:34:24.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.303+0000 7f5b44d15700 1 -- 172.21.15.16:0/3357286567 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b40136c40 con 0x7f5b40131810 2023-09-30T22:34:24.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.303+0000 7f5b2effd700 1 -- 172.21.15.16:0/3357286567 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b300038c0 con 0x7f5b40131810 2023-09-30T22:34:24.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.303+0000 7f5b44d15700 1 -- 172.21.15.16:0/3357286567 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b4011a400 con 0x7f5b40131810 2023-09-30T22:34:24.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.304+0000 7f5b44d15700 1 -- 172.21.15.16:0/3357286567 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b40118fb0 con 0x7f5b40131810 2023-09-30T22:34:24.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.305+0000 7f5b2effd700 1 -- 172.21.15.16:0/3357286567 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5b30003a20 con 0x7f5b40131810 2023-09-30T22:34:24.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.306+0000 7f5b2effd700 1 -- 172.21.15.16:0/3357286567 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f5b3003eec0 con 0x7f5b40131810 2023-09-30T22:34:24.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.310+0000 7f5b2effd700 1 -- 172.21.15.16:0/3357286567 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5b30019d20 con 0x7f5b40131810 2023-09-30T22:34:24.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.355+0000 7fc2e8062700 1 Processor -- start 2023-09-30T22:34:24.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.355+0000 7fc2e8062700 1 -- start start 2023-09-30T22:34:24.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.356+0000 7fc2e8062700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc2e01338a0 0x7fc2e0133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:24.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.356+0000 7fc2e8062700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2e009de50 0x7fc2e009e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:24.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.356+0000 7fc2e8062700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2e009e760 0x7fc2e013a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:24.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.356+0000 7fc2e8062700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2e006fef0 con 0x7fc2e01338a0 2023-09-30T22:34:24.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.356+0000 7fc2e8062700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e006ef30 con 0x7fc2e009e760 2023-09-30T22:34:24.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.356+0000 7fc2e8062700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e01334b0 con 0x7fc2e009de50 2023-09-30T22:34:24.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.356+0000 7fc2e55fd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2e009de50 0x7fc2e009e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:24.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.356+0000 7fc2e55fd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2e009de50 0x7fc2e009e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:48232/0 (socket says 172.21.15.16:48232) 2023-09-30T22:34:24.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.356+0000 7fc2e55fd700 1 -- 172.21.15.16:0/3627388887 learned_addr learned my addr 172.21.15.16:0/3627388887 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:24.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.356+0000 7fc2e65ff700 1 --2- 172.21.15.16:0/3627388887 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2e009e760 0x7fc2e013a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:24.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.356+0000 7fc2e5dfe700 1 --2- 172.21.15.16:0/3627388887 >> v2:172.21.15.16:3300/0 conn(0x7fc2e01338a0 0x7fc2e0133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:24.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.356+0000 7fc2e55fd700 1 -- 172.21.15.16:0/3627388887 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2e009e760 msgr2=0x7fc2e013a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:24.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.356+0000 7fc2e55fd700 1 --2- 172.21.15.16:0/3627388887 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2e009e760 0x7fc2e013a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:24.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.356+0000 7fc2e55fd700 1 -- 172.21.15.16:0/3627388887 >> v2:172.21.15.16:3300/0 conn(0x7fc2e01338a0 msgr2=0x7fc2e0133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:24.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.356+0000 7fc2e55fd700 1 --2- 172.21.15.16:0/3627388887 >> v2:172.21.15.16:3300/0 conn(0x7fc2e01338a0 0x7fc2e0133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:24.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.356+0000 7fc2e55fd700 1 -- 172.21.15.16:0/3627388887 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2e006fc30 con 0x7fc2e009de50 2023-09-30T22:34:24.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.357+0000 7fc2e55fd700 1 --2- 172.21.15.16:0/3627388887 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2e009de50 0x7fc2e009e220 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fc2d00098f0 tx=0x7fc2d0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a555e90841172b18 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:24.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.357+0000 7fc2e4dfc700 1 -- 172.21.15.16:0/3627388887 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d0011830 con 0x7fc2e009de50 2023-09-30T22:34:24.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.357+0000 7fc2e4dfc700 1 -- 172.21.15.16:0/3627388887 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d0011990 con 0x7fc2e009de50 2023-09-30T22:34:24.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.357+0000 7fc2e4dfc700 1 -- 172.21.15.16:0/3627388887 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d0011d20 con 0x7fc2e009de50 2023-09-30T22:34:24.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.358+0000 7fc2e8062700 1 -- 172.21.15.16:0/3627388887 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2e009de50 msgr2=0x7fc2e009e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:24.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.358+0000 7fc2e8062700 1 --2- 172.21.15.16:0/3627388887 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2e009de50 0x7fc2e009e220 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fc2d00098f0 tx=0x7fc2d0009c00 comp rx=0 tx=0).stop 2023-09-30T22:34:24.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.358+0000 7fc2e8062700 1 -- 172.21.15.16:0/3627388887 shutdown_connections 2023-09-30T22:34:24.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.358+0000 7fc2e8062700 1 --2- 172.21.15.16:0/3627388887 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2e009e760 0x7fc2e013a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:24.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.358+0000 7fc2e8062700 1 --2- 172.21.15.16:0/3627388887 >> v2:172.21.15.16:3300/0 conn(0x7fc2e01338a0 0x7fc2e0133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:24.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.358+0000 7fc2e8062700 1 --2- 172.21.15.16:0/3627388887 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2e009de50 0x7fc2e009e220 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:24.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.358+0000 7fc2e8062700 1 -- 172.21.15.16:0/3627388887 >> 172.21.15.16:0/3627388887 conn(0x7fc2e00a2aa0 msgr2=0x7fc2e00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:24.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.358+0000 7fc2e8062700 1 -- 172.21.15.16:0/3627388887 shutdown_connections 2023-09-30T22:34:24.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.358+0000 7fc2e8062700 1 -- 172.21.15.16:0/3627388887 wait complete. 2023-09-30T22:34:24.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.359+0000 7fc2e8062700 1 Processor -- start 2023-09-30T22:34:24.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.359+0000 7fc2e8062700 1 -- start start 2023-09-30T22:34:24.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.359+0000 7fc2e8062700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2e009de50 0x7fc2e011b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:24.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.359+0000 7fc2e8062700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2e009e760 0x7fc2e011bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:24.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.359+0000 7fc2e8062700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2e01338a0 0x7fc2e011c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:24.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.359+0000 7fc2e8062700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e011cb50 con 0x7fc2e009de50 2023-09-30T22:34:24.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.359+0000 7fc2e8062700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e011d2b0 con 0x7fc2e01338a0 2023-09-30T22:34:24.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.359+0000 7fc2e8062700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e011d8d0 con 0x7fc2e009e760 2023-09-30T22:34:24.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.360+0000 7fc2e55fd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2e009e760 0x7fc2e011bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:24.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.360+0000 7fc2e55fd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2e009e760 0x7fc2e011bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:48234/0 (socket says 172.21.15.16:48234) 2023-09-30T22:34:24.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.360+0000 7fc2e55fd700 1 -- 172.21.15.16:0/1445793487 learned_addr learned my addr 172.21.15.16:0/1445793487 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:24.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.360+0000 7fc2e5dfe700 1 --2- 172.21.15.16:0/1445793487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2e009de50 0x7fc2e011b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:24.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.360+0000 7fc2e65ff700 1 --2- 172.21.15.16:0/1445793487 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2e01338a0 0x7fc2e011c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:24.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.360+0000 7fc2e55fd700 1 -- 172.21.15.16:0/1445793487 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2e01338a0 msgr2=0x7fc2e011c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:24.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.360+0000 7fc2e55fd700 1 --2- 172.21.15.16:0/1445793487 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2e01338a0 0x7fc2e011c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:24.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.360+0000 7fc2e55fd700 1 -- 172.21.15.16:0/1445793487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2e009de50 msgr2=0x7fc2e011b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:24.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.360+0000 7fc2e55fd700 1 --2- 172.21.15.16:0/1445793487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2e009de50 0x7fc2e011b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:24.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.360+0000 7fc2e55fd700 1 -- 172.21.15.16:0/1445793487 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d0009550 con 0x7fc2e009e760 2023-09-30T22:34:24.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.360+0000 7fc2e55fd700 1 --2- 172.21.15.16:0/1445793487 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2e009e760 0x7fc2e011bbe0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0000c00 tx=0x7fc2d0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:24.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.360+0000 7fc2d67fc700 1 -- 172.21.15.16:0/1445793487 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d0004450 con 0x7fc2e009e760 2023-09-30T22:34:24.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.361+0000 7fc2d67fc700 1 -- 172.21.15.16:0/1445793487 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d00045b0 con 0x7fc2e009e760 2023-09-30T22:34:24.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.361+0000 7fc2e8062700 1 -- 172.21.15.16:0/1445793487 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2e011eee0 con 0x7fc2e009e760 2023-09-30T22:34:24.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.361+0000 7fc2d67fc700 1 -- 172.21.15.16:0/1445793487 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d00038c0 con 0x7fc2e009e760 2023-09-30T22:34:24.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.361+0000 7fc2e8062700 1 -- 172.21.15.16:0/1445793487 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2e011e9a0 con 0x7fc2e009e760 2023-09-30T22:34:24.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.362+0000 7fc2e8062700 1 -- 172.21.15.16:0/1445793487 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2e0074ea0 con 0x7fc2e009e760 2023-09-30T22:34:24.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.362+0000 7fc2d67fc700 1 -- 172.21.15.16:0/1445793487 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc2d0003a20 con 0x7fc2e009e760 2023-09-30T22:34:24.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.363+0000 7fc2d67fc700 1 -- 172.21.15.16:0/1445793487 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fc2d0040ab0 con 0x7fc2e009e760 2023-09-30T22:34:24.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.365+0000 7fc12ec8e700 1 Processor -- start 2023-09-30T22:34:24.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.365+0000 7fc12ec8e700 1 -- start start 2023-09-30T22:34:24.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.366+0000 7fc12ec8e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc12806a770 0x7fc12809d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:24.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.366+0000 7fc12ec8e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc12809dc30 0x7fc12809e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:24.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.366+0000 7fc12ec8e700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc12809a810 con 0x7fc12813c1e0 2023-09-30T22:34:24.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.366+0000 7fc12ec8e700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc12806ab40 con 0x7fc12809dc30 2023-09-30T22:34:24.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.366+0000 7fc12ec8e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc12809ebc0 con 0x7fc12806a770 2023-09-30T22:34:24.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.366+0000 7fc12ca2a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc12806a770 0x7fc12809d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:24.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.366+0000 7fc127fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc12809dc30 0x7fc12809e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:24.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.366+0000 7fc12d22b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc12813c1e0 0x7fc12813e5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54940/0 (socket says 172.21.15.16:54940) 2023-09-30T22:34:24.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.366+0000 7fc12d22b700 1 -- 172.21.15.16:0/3058390390 learned_addr learned my addr 172.21.15.16:0/3058390390 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:34:24.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.367+0000 7fc127fff700 1 -- 172.21.15.16:0/3058390390 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc12806a770 msgr2=0x7fc12809d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:24.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.367+0000 7fc1277fe700 1 -- 172.21.15.16:0/3058390390 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4042423444 0 0) 0x7fc12809a810 con 0x7fc12813c1e0 2023-09-30T22:34:24.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.367+0000 7fc127fff700 1 --2- 172.21.15.16:0/3058390390 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc12806a770 0x7fc12809d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:24.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.367+0000 7fc127fff700 1 -- 172.21.15.16:0/3058390390 >> v1:172.21.15.16:6789/0 conn(0x7fc12813c1e0 legacy=0x7fc12813e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:24.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.367+0000 7fc127fff700 1 -- 172.21.15.16:0/3058390390 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc12809e920 con 0x7fc12809dc30 2023-09-30T22:34:24.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.367+0000 7fc127fff700 1 --2- 172.21.15.16:0/3058390390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc12809dc30 0x7fc12809e080 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fc114009870 tx=0x7fc114009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa11946f480dc960 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:24.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.367+0000 7fc1277fe700 1 -- 172.21.15.16:0/3058390390 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc114011830 con 0x7fc12809dc30 2023-09-30T22:34:24.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.367+0000 7fc1277fe700 1 -- 172.21.15.16:0/3058390390 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc114011990 con 0x7fc12809dc30 2023-09-30T22:34:24.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.367+0000 7fc1277fe700 1 -- 172.21.15.16:0/3058390390 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc114011d20 con 0x7fc12809dc30 2023-09-30T22:34:24.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.368+0000 7fc2d67fc700 1 -- 172.21.15.16:0/1445793487 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc2d000b7f0 con 0x7fc2e009e760 2023-09-30T22:34:24.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.368+0000 7fc12ec8e700 1 -- 172.21.15.16:0/3058390390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc12809dc30 msgr2=0x7fc12809e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:24.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.368+0000 7fc12ec8e700 1 --2- 172.21.15.16:0/3058390390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc12809dc30 0x7fc12809e080 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fc114009870 tx=0x7fc114009b80 comp rx=0 tx=0).stop 2023-09-30T22:34:24.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.368+0000 7fc12ec8e700 1 -- 172.21.15.16:0/3058390390 shutdown_connections 2023-09-30T22:34:24.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.368+0000 7fc12ec8e700 1 --2- 172.21.15.16:0/3058390390 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc12806a770 0x7fc12809d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:24.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.368+0000 7fc12ec8e700 1 --2- 172.21.15.16:0/3058390390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc12809dc30 0x7fc12809e080 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:24.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.368+0000 7fc12ec8e700 1 -- 172.21.15.16:0/3058390390 >> 172.21.15.16:0/3058390390 conn(0x7fc128066f50 msgr2=0x7fc12809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:24.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.368+0000 7fc12ec8e700 1 -- 172.21.15.16:0/3058390390 shutdown_connections 2023-09-30T22:34:24.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.368+0000 7fc12ec8e700 1 -- 172.21.15.16:0/3058390390 wait complete. 2023-09-30T22:34:24.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.370+0000 7fc12ec8e700 1 Processor -- start 2023-09-30T22:34:24.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.370+0000 7fc12ec8e700 1 -- start start 2023-09-30T22:34:24.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.371+0000 7fc12ec8e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc12806a770 0x7fc12813bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:24.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.371+0000 7fc12ec8e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc12809dc30 0x7fc12813a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:24.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.371+0000 7fc12ec8e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc12813c1e0 0x7fc12813a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:24.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.371+0000 7fc12ec8e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc12813c070 con 0x7fc12806a770 2023-09-30T22:34:24.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.371+0000 7fc12ec8e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc128134b20 con 0x7fc12809dc30 2023-09-30T22:34:24.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.371+0000 7fc12ec8e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc12809f1a0 con 0x7fc12813c1e0 2023-09-30T22:34:24.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.371+0000 7fc127fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc12809dc30 0x7fc12813a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:24.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.371+0000 7fc127fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc12809dc30 0x7fc12813a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:55580/0 (socket says 172.21.15.16:55580) 2023-09-30T22:34:24.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.371+0000 7fc127fff700 1 -- 172.21.15.16:0/3795306619 learned_addr learned my addr 172.21.15.16:0/3795306619 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:24.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.372+0000 7fc127fff700 1 -- 172.21.15.16:0/3795306619 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc12813c1e0 msgr2=0x7fc12813a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:24.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.372+0000 7fc127fff700 1 --2- 172.21.15.16:0/3795306619 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc12813c1e0 0x7fc12813a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:24.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.372+0000 7fc127fff700 1 -- 172.21.15.16:0/3795306619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc12806a770 msgr2=0x7fc12813bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:24.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.372+0000 7fc127fff700 1 --2- 172.21.15.16:0/3795306619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc12806a770 0x7fc12813bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:24.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.372+0000 7fc127fff700 1 -- 172.21.15.16:0/3795306619 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1140094a0 con 0x7fc12809dc30 2023-09-30T22:34:24.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.372+0000 7fc127fff700 1 --2- 172.21.15.16:0/3795306619 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc12809dc30 0x7fc12813a3c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fc114000c00 tx=0x7fc114004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:24.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.372+0000 7fc1257fa700 1 -- 172.21.15.16:0/3795306619 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1140036a0 con 0x7fc12809dc30 2023-09-30T22:34:24.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.372+0000 7fc1257fa700 1 -- 172.21.15.16:0/3795306619 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc11401f070 con 0x7fc12809dc30 2023-09-30T22:34:24.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.372+0000 7fc12ec8e700 1 -- 172.21.15.16:0/3795306619 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc12811c360 con 0x7fc12809dc30 2023-09-30T22:34:24.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.372+0000 7fc12ec8e700 1 -- 172.21.15.16:0/3795306619 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc12811bc30 con 0x7fc12809dc30 2023-09-30T22:34:24.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.372+0000 7fc1257fa700 1 -- 172.21.15.16:0/3795306619 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc114003a00 con 0x7fc12809dc30 2023-09-30T22:34:24.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.374+0000 7fc12ec8e700 1 -- 172.21.15.16:0/3795306619 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1080052e0 con 0x7fc12809dc30 2023-09-30T22:34:24.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.374+0000 7fc1257fa700 1 -- 172.21.15.16:0/3795306619 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc114003b60 con 0x7fc12809dc30 2023-09-30T22:34:24.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.375+0000 7fc1257fa700 1 -- 172.21.15.16:0/3795306619 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fc114021070 con 0x7fc12809dc30 2023-09-30T22:34:24.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.380+0000 7fc1257fa700 1 -- 172.21.15.16:0/3795306619 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc11402ebe0 con 0x7fc12809dc30 2023-09-30T22:34:24.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.451+0000 7f70dfc4d700 1 -- 172.21.15.16:0/132943733 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f70d809f420 con 0x7f70d809d6f0 2023-09-30T22:34:24.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.451+0000 7f70ce7fc700 1 -- 172.21.15.16:0/132943733 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f70d4028090 con 0x7f70d809d6f0 2023-09-30T22:34:24.452 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:34:24.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.455+0000 7f70dfc4d700 1 -- 172.21.15.16:0/132943733 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f70d809d6f0 msgr2=0x7f70d81a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:24.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.455+0000 7f70dfc4d700 1 --2- 172.21.15.16:0/132943733 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f70d809d6f0 0x7f70d81a51e0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f70d400b3c0 tx=0x7f70d4004750 comp rx=0 tx=0).stop 2023-09-30T22:34:24.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.455+0000 7f70dfc4d700 1 -- 172.21.15.16:0/132943733 shutdown_connections 2023-09-30T22:34:24.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.455+0000 7f70dfc4d700 1 --2- 172.21.15.16:0/132943733 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70d809e440 0x7f70d81c7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:24.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.455+0000 7f70dfc4d700 1 --2- 172.21.15.16:0/132943733 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f70d809d6f0 0x7f70d81a51e0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:24.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.455+0000 7f70dfc4d700 1 --2- 172.21.15.16:0/132943733 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f70d809e000 0x7f70d81a5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:24.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.455+0000 7f70dfc4d700 1 -- 172.21.15.16:0/132943733 >> 172.21.15.16:0/132943733 conn(0x7f70d8066f50 msgr2=0x7f70d81bbb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:24.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.455+0000 7f70dfc4d700 1 -- 172.21.15.16:0/132943733 shutdown_connections 2023-09-30T22:34:24.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.455+0000 7f70dfc4d700 1 -- 172.21.15.16:0/132943733 wait complete. 2023-09-30T22:34:24.475 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:34:24.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.524+0000 7f5b44d15700 1 -- 172.21.15.16:0/3357286567 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5b40074ea0 con 0x7f5b40131810 2023-09-30T22:34:24.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.524+0000 7f5b2effd700 1 -- 172.21.15.16:0/3357286567 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5b30017070 con 0x7f5b40131810 2023-09-30T22:34:24.525 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:34:24.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.528+0000 7f5b44d15700 1 -- 172.21.15.16:0/3357286567 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5b40131810 msgr2=0x7f5b4009ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:24.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.528+0000 7f5b44d15700 1 --2- 172.21.15.16:0/3357286567 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5b40131810 0x7f5b4009ca50 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f5b3000c450 tx=0x7f5b3000c760 comp rx=0 tx=0).stop 2023-09-30T22:34:24.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.528+0000 7f5b44d15700 1 -- 172.21.15.16:0/3357286567 shutdown_connections 2023-09-30T22:34:24.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.528+0000 7f5b44d15700 1 --2- 172.21.15.16:0/3357286567 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5b40131810 0x7f5b4009ca50 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:24.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.528+0000 7f5b44d15700 1 --2- 172.21.15.16:0/3357286567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b40136f50 0x7f5b4009d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:24.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.528+0000 7f5b44d15700 1 --2- 172.21.15.16:0/3357286567 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5b40134630 0x7f5b4009cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:24.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.528+0000 7f5b44d15700 1 -- 172.21.15.16:0/3357286567 >> 172.21.15.16:0/3357286567 conn(0x7f5b400a2aa0 msgr2=0x7f5b4006f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:24.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.528+0000 7f5b44d15700 1 -- 172.21.15.16:0/3357286567 shutdown_connections 2023-09-30T22:34:24.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.528+0000 7f5b44d15700 1 -- 172.21.15.16:0/3357286567 wait complete. 2023-09-30T22:34:24.547 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:34:24.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.609+0000 7fc12ec8e700 1 -- 172.21.15.16:0/3795306619 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc108005590 con 0x7fc12809dc30 2023-09-30T22:34:24.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.610+0000 7fc1257fa700 1 -- 172.21.15.16:0/3795306619 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc114015020 con 0x7fc12809dc30 2023-09-30T22:34:24.610 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:34:24.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.612+0000 7fc12ec8e700 1 -- 172.21.15.16:0/3795306619 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc12809dc30 msgr2=0x7fc12813a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:24.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.612+0000 7fc12ec8e700 1 --2- 172.21.15.16:0/3795306619 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc12809dc30 0x7fc12813a3c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fc114000c00 tx=0x7fc114004750 comp rx=0 tx=0).stop 2023-09-30T22:34:24.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.612+0000 7fc12ec8e700 1 -- 172.21.15.16:0/3795306619 shutdown_connections 2023-09-30T22:34:24.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.612+0000 7fc12ec8e700 1 --2- 172.21.15.16:0/3795306619 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc12813c1e0 0x7fc12813a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:24.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.612+0000 7fc12ec8e700 1 --2- 172.21.15.16:0/3795306619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc12806a770 0x7fc12813bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:24.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.612+0000 7fc12ec8e700 1 --2- 172.21.15.16:0/3795306619 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc12809dc30 0x7fc12813a3c0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:24.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.612+0000 7fc12ec8e700 1 -- 172.21.15.16:0/3795306619 >> 172.21.15.16:0/3795306619 conn(0x7fc128066f50 msgr2=0x7fc128072270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:24.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.612+0000 7fc12ec8e700 1 -- 172.21.15.16:0/3795306619 shutdown_connections 2023-09-30T22:34:24.613 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.612+0000 7fc12ec8e700 1 -- 172.21.15.16:0/3795306619 wait complete. 2023-09-30T22:34:24.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.623+0000 7fc2e8062700 1 -- 172.21.15.16:0/1445793487 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc2e006b9e0 con 0x7fc2e009e760 2023-09-30T22:34:24.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.623+0000 7fc2d67fc700 1 -- 172.21.15.16:0/1445793487 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc2d0015020 con 0x7fc2e009e760 2023-09-30T22:34:24.624 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:34:24.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.625+0000 7fc2e8062700 1 -- 172.21.15.16:0/1445793487 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2e009e760 msgr2=0x7fc2e011bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:24.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.625+0000 7fc2e8062700 1 --2- 172.21.15.16:0/1445793487 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2e009e760 0x7fc2e011bbe0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0000c00 tx=0x7fc2d0004750 comp rx=0 tx=0).stop 2023-09-30T22:34:24.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.626+0000 7fc2e8062700 1 -- 172.21.15.16:0/1445793487 shutdown_connections 2023-09-30T22:34:24.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.626+0000 7fc2e8062700 1 --2- 172.21.15.16:0/1445793487 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2e01338a0 0x7fc2e011c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:24.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.626+0000 7fc2e8062700 1 --2- 172.21.15.16:0/1445793487 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2e009de50 0x7fc2e011b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:24.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.626+0000 7fc2e8062700 1 --2- 172.21.15.16:0/1445793487 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2e009e760 0x7fc2e011bbe0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:24.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.626+0000 7fc2e8062700 1 -- 172.21.15.16:0/1445793487 >> 172.21.15.16:0/1445793487 conn(0x7fc2e00a2aa0 msgr2=0x7fc2e01306c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:24.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.626+0000 7fc2e8062700 1 -- 172.21.15.16:0/1445793487 shutdown_connections 2023-09-30T22:34:24.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:24.626+0000 7fc2e8062700 1 -- 172.21.15.16:0/1445793487 wait complete. 2023-09-30T22:34:24.627 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:34:24.639 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:34:25.476 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:34:25.547 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:34:25.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.620+0000 7f789f61a700 1 Processor -- start 2023-09-30T22:34:25.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.621+0000 7f789f61a700 1 -- start start 2023-09-30T22:34:25.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.621+0000 7f789f61a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f78980a0a30 0x7f78980a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:25.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.621+0000 7f789f61a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f78980a1340 0x7f7898133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:25.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.621+0000 7f789f61a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7898133fa0 0x7f7898134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:25.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.621+0000 7f789f61a700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f789809edd0 con 0x7f78980a0a30 2023-09-30T22:34:25.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.621+0000 7f789f61a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f789809e1f0 con 0x7f78980a1340 2023-09-30T22:34:25.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.621+0000 7f789f61a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f789809dc50 con 0x7f7898133fa0 2023-09-30T22:34:25.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.621+0000 7f789d3b6700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f78980a0a30 0x7f78980a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:25.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.621+0000 7f789cbb5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f78980a1340 0x7f7898133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:25.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.621+0000 7f789d3b6700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f78980a0a30 0x7f78980a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47666/0 (socket says 172.21.15.16:47666) 2023-09-30T22:34:25.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.621+0000 7f789cbb5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f78980a1340 0x7f7898133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:55586/0 (socket says 172.21.15.16:55586) 2023-09-30T22:34:25.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.621+0000 7f789d3b6700 1 -- 172.21.15.16:0/1447500695 learned_addr learned my addr 172.21.15.16:0/1447500695 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:25.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.621+0000 7f789dbb7700 1 --2- 172.21.15.16:0/1447500695 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7898133fa0 0x7f7898134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:25.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.622+0000 7f789d3b6700 1 -- 172.21.15.16:0/1447500695 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7898133fa0 msgr2=0x7f7898134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:25.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.622+0000 7f789d3b6700 1 --2- 172.21.15.16:0/1447500695 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7898133fa0 0x7f7898134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:25.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.622+0000 7f789d3b6700 1 -- 172.21.15.16:0/1447500695 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f78980a1340 msgr2=0x7f7898133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:25.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.622+0000 7f789d3b6700 1 --2- 172.21.15.16:0/1447500695 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f78980a1340 0x7f7898133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:25.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.622+0000 7f789d3b6700 1 -- 172.21.15.16:0/1447500695 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78980a1fe0 con 0x7f78980a0a30 2023-09-30T22:34:25.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.622+0000 7f789dbb7700 1 --2- 172.21.15.16:0/1447500695 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7898133fa0 0x7f7898134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:25.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.622+0000 7f789cbb5700 1 --2- 172.21.15.16:0/1447500695 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f78980a1340 0x7f7898133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:25.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.622+0000 7f789d3b6700 1 --2- 172.21.15.16:0/1447500695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78980a0a30 0x7f78980a0e00 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f78940098f0 tx=0x7f7894009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22255d547d8fde40 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:25.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.622+0000 7f788ffff700 1 -- 172.21.15.16:0/1447500695 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7894011830 con 0x7f78980a0a30 2023-09-30T22:34:25.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.622+0000 7f788ffff700 1 -- 172.21.15.16:0/1447500695 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7894011990 con 0x7f78980a0a30 2023-09-30T22:34:25.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.622+0000 7f788ffff700 1 -- 172.21.15.16:0/1447500695 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7894011d20 con 0x7f78980a0a30 2023-09-30T22:34:25.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.623+0000 7f789f61a700 1 -- 172.21.15.16:0/1447500695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78980a0a30 msgr2=0x7f78980a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:25.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.623+0000 7f789f61a700 1 --2- 172.21.15.16:0/1447500695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78980a0a30 0x7f78980a0e00 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f78940098f0 tx=0x7f7894009c00 comp rx=0 tx=0).stop 2023-09-30T22:34:25.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.623+0000 7f789f61a700 1 -- 172.21.15.16:0/1447500695 shutdown_connections 2023-09-30T22:34:25.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.623+0000 7f789f61a700 1 --2- 172.21.15.16:0/1447500695 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7898133fa0 0x7f7898134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:25.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.623+0000 7f789f61a700 1 --2- 172.21.15.16:0/1447500695 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78980a0a30 0x7f78980a0e00 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:25.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.623+0000 7f789f61a700 1 --2- 172.21.15.16:0/1447500695 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f78980a1340 0x7f7898133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:25.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.623+0000 7f789f61a700 1 -- 172.21.15.16:0/1447500695 >> 172.21.15.16:0/1447500695 conn(0x7f78980a6550 msgr2=0x7f78980a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:25.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.623+0000 7f789f61a700 1 -- 172.21.15.16:0/1447500695 shutdown_connections 2023-09-30T22:34:25.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.623+0000 7f789f61a700 1 -- 172.21.15.16:0/1447500695 wait complete. 2023-09-30T22:34:25.628 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:34:25.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.624+0000 7f789f61a700 1 Processor -- start 2023-09-30T22:34:25.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.624+0000 7f789f61a700 1 -- start start 2023-09-30T22:34:25.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.624+0000 7f789f61a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78980a0a30 0x7f7898120120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:25.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.624+0000 7f789f61a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f78980a1340 0x7f7898120660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:25.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.624+0000 7f789f61a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7898133fa0 0x7f7898120ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:25.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.624+0000 7f789f61a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7898123c40 con 0x7f78980a0a30 2023-09-30T22:34:25.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.624+0000 7f789f61a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7898123010 con 0x7f78980a1340 2023-09-30T22:34:25.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.624+0000 7f789f61a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f78980a35a0 con 0x7f7898133fa0 2023-09-30T22:34:25.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.624+0000 7f789d3b6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78980a0a30 0x7f7898120120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:25.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.625+0000 7f789cbb5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f78980a1340 0x7f7898120660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:25.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.625+0000 7f789d3b6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78980a0a30 0x7f7898120120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47676/0 (socket says 172.21.15.16:47676) 2023-09-30T22:34:25.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.625+0000 7f789d3b6700 1 -- 172.21.15.16:0/935951517 learned_addr learned my addr 172.21.15.16:0/935951517 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:25.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.625+0000 7f789dbb7700 1 --2- 172.21.15.16:0/935951517 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7898133fa0 0x7f7898120ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:25.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.625+0000 7f789d3b6700 1 -- 172.21.15.16:0/935951517 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7898133fa0 msgr2=0x7f7898120ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:25.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.625+0000 7f789d3b6700 1 --2- 172.21.15.16:0/935951517 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7898133fa0 0x7f7898120ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:25.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.625+0000 7f789d3b6700 1 -- 172.21.15.16:0/935951517 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f78980a1340 msgr2=0x7f7898120660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:25.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.625+0000 7f789d3b6700 1 --2- 172.21.15.16:0/935951517 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f78980a1340 0x7f7898120660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:25.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.625+0000 7f789d3b6700 1 -- 172.21.15.16:0/935951517 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78880094a0 con 0x7f78980a0a30 2023-09-30T22:34:25.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.625+0000 7f789dbb7700 1 --2- 172.21.15.16:0/935951517 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7898133fa0 0x7f7898120ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:25.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.625+0000 7f789cbb5700 1 --2- 172.21.15.16:0/935951517 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f78980a1340 0x7f7898120660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:25.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.625+0000 7f789d3b6700 1 --2- 172.21.15.16:0/935951517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78980a0a30 0x7f7898120120 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f7894009f50 tx=0x7f7894004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:25.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.625+0000 7f788dffb700 1 -- 172.21.15.16:0/935951517 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7894004240 con 0x7f78980a0a30 2023-09-30T22:34:25.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.625+0000 7f788dffb700 1 -- 172.21.15.16:0/935951517 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f789401f070 con 0x7f78980a0a30 2023-09-30T22:34:25.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.625+0000 7f789f61a700 1 -- 172.21.15.16:0/935951517 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7890009a70 con 0x7f78980a0a30 2023-09-30T22:34:25.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.625+0000 7f788dffb700 1 -- 172.21.15.16:0/935951517 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78940045c0 con 0x7f78980a0a30 2023-09-30T22:34:25.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.626+0000 7f789f61a700 1 -- 172.21.15.16:0/935951517 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7894009550 con 0x7f78980a0a30 2023-09-30T22:34:25.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.626+0000 7f788dffb700 1 -- 172.21.15.16:0/935951517 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7894021030 con 0x7f78980a0a30 2023-09-30T22:34:25.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.627+0000 7f789f61a700 1 -- 172.21.15.16:0/935951517 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7898074ea0 con 0x7f78980a0a30 2023-09-30T22:34:25.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.627+0000 7f788dffb700 1 -- 172.21.15.16:0/935951517 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f78940406b0 con 0x7f78980a0a30 2023-09-30T22:34:25.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.632+0000 7f788dffb700 1 -- 172.21.15.16:0/935951517 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7894015070 con 0x7f78980a0a30 2023-09-30T22:34:25.640 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:34:25.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.693+0000 7f6ff14c7700 1 Processor -- start 2023-09-30T22:34:25.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.693+0000 7f6ff14c7700 1 -- start start 2023-09-30T22:34:25.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.694+0000 7f6ff14c7700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6fec09d6f0 0x7f6fec09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:25.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.694+0000 7f6ff14c7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6fec09e000 0x7f6fec130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:25.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.694+0000 7f6ff14c7700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fec09f130 con 0x7f6fec09e410 2023-09-30T22:34:25.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.694+0000 7f6ff14c7700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fec09f520 con 0x7f6fec09d6f0 2023-09-30T22:34:25.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.694+0000 7f6ff14c7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fec09e930 con 0x7f6fec09e000 2023-09-30T22:34:25.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.694+0000 7f6feaffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6fec09d6f0 0x7f6fec09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:25.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.694+0000 7f6feaffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6fec09d6f0 0x7f6fec09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47690/0 (socket says 172.21.15.16:47690) 2023-09-30T22:34:25.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.694+0000 7f6feaffd700 1 -- 172.21.15.16:0/397575518 learned_addr learned my addr 172.21.15.16:0/397575518 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:25.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.694+0000 7f6fea7fc700 1 --2- 172.21.15.16:0/397575518 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6fec09e000 0x7f6fec130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:25.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.694+0000 7f6feaffd700 1 -- 172.21.15.16:0/397575518 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6fec09e000 msgr2=0x7f6fec130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:25.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.694+0000 7f6feaffd700 1 --2- 172.21.15.16:0/397575518 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6fec09e000 0x7f6fec130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:25.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.694+0000 7f6feaffd700 1 -- 172.21.15.16:0/397575518 >> v1:172.21.15.16:6789/0 conn(0x7f6fec09e410 legacy=0x7f6fec130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:25.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.694+0000 7f6feaffd700 1 -- 172.21.15.16:0/397575518 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fec09b2d0 con 0x7f6fec09d6f0 2023-09-30T22:34:25.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.695+0000 7f6feaffd700 1 --2- 172.21.15.16:0/397575518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fec09d6f0 0x7f6fec09dac0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc0098f0 tx=0x7f6fdc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d29ef1cc37b0f326 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:25.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.695+0000 7f6fe9ffb700 1 -- 172.21.15.16:0/397575518 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc004230 con 0x7f6fec09d6f0 2023-09-30T22:34:25.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.695+0000 7f6fe9ffb700 1 -- 172.21.15.16:0/397575518 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fdc004390 con 0x7f6fec09d6f0 2023-09-30T22:34:25.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.695+0000 7f6fe9ffb700 1 -- 172.21.15.16:0/397575518 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc0036a0 con 0x7f6fec09d6f0 2023-09-30T22:34:25.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.696+0000 7f6ff14c7700 1 -- 172.21.15.16:0/397575518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fec09d6f0 msgr2=0x7f6fec09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:25.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.696+0000 7f6ff14c7700 1 --2- 172.21.15.16:0/397575518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fec09d6f0 0x7f6fec09dac0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc0098f0 tx=0x7f6fdc009c00 comp rx=0 tx=0).stop 2023-09-30T22:34:25.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.696+0000 7f6ff14c7700 1 -- 172.21.15.16:0/397575518 shutdown_connections 2023-09-30T22:34:25.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.696+0000 7f6ff14c7700 1 --2- 172.21.15.16:0/397575518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fec09d6f0 0x7f6fec09dac0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:25.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.696+0000 7f6ff14c7700 1 --2- 172.21.15.16:0/397575518 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6fec09e000 0x7f6fec130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:25.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.696+0000 7f6ff14c7700 1 -- 172.21.15.16:0/397575518 >> 172.21.15.16:0/397575518 conn(0x7f6fec066f50 msgr2=0x7f6fec09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:25.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.696+0000 7f6ff14c7700 1 -- 172.21.15.16:0/397575518 shutdown_connections 2023-09-30T22:34:25.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.696+0000 7f6ff14c7700 1 -- 172.21.15.16:0/397575518 wait complete. 2023-09-30T22:34:25.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.698+0000 7f6ff14c7700 1 Processor -- start 2023-09-30T22:34:25.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.698+0000 7f6ff14c7700 1 -- start start 2023-09-30T22:34:25.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.699+0000 7f6ff14c7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6fec09d6f0 0x7f6fec12fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:25.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.699+0000 7f6ff14c7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fec09e000 0x7f6fec130170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:25.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.699+0000 7f6ff14c7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6fec09e410 0x7f6fec1306b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:25.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.699+0000 7f6ff14c7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fec0a18e0 con 0x7f6fec09e000 2023-09-30T22:34:25.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.699+0000 7f6ff14c7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fec13a060 con 0x7f6fec09d6f0 2023-09-30T22:34:25.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.699+0000 7f6ff14c7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6fec131630 con 0x7f6fec09e410 2023-09-30T22:34:25.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.699+0000 7f6feaffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6fec09d6f0 0x7f6fec12fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:25.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.699+0000 7f6feaffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6fec09d6f0 0x7f6fec12fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:55612/0 (socket says 172.21.15.16:55612) 2023-09-30T22:34:25.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.699+0000 7f6feaffd700 1 -- 172.21.15.16:0/1186222822 learned_addr learned my addr 172.21.15.16:0/1186222822 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:25.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.699+0000 7f6feaffd700 1 -- 172.21.15.16:0/1186222822 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6fec09e410 msgr2=0x7f6fec1306b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:25.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.699+0000 7f6feaffd700 1 --2- 172.21.15.16:0/1186222822 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6fec09e410 0x7f6fec1306b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:25.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.699+0000 7f6feaffd700 1 -- 172.21.15.16:0/1186222822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fec09e000 msgr2=0x7f6fec130170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:25.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.699+0000 7f6feaffd700 1 --2- 172.21.15.16:0/1186222822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fec09e000 0x7f6fec130170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:25.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.699+0000 7f6feaffd700 1 -- 172.21.15.16:0/1186222822 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fdc009550 con 0x7f6fec09d6f0 2023-09-30T22:34:25.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.700+0000 7f6feaffd700 1 --2- 172.21.15.16:0/1186222822 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6fec09d6f0 0x7f6fec12fc30 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f6fec09b9e0 tx=0x7f6fdc004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:25.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.700+0000 7f6fd37fe700 1 -- 172.21.15.16:0/1186222822 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc003de0 con 0x7f6fec09d6f0 2023-09-30T22:34:25.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.700+0000 7f6fd37fe700 1 -- 172.21.15.16:0/1186222822 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fdc006bd0 con 0x7f6fec09d6f0 2023-09-30T22:34:25.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.700+0000 7f6ff14c7700 1 -- 172.21.15.16:0/1186222822 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fec13b850 con 0x7f6fec09d6f0 2023-09-30T22:34:25.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.700+0000 7f6fd37fe700 1 -- 172.21.15.16:0/1186222822 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc011870 con 0x7f6fec09d6f0 2023-09-30T22:34:25.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.700+0000 7f6ff14c7700 1 -- 172.21.15.16:0/1186222822 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fec12a1f0 con 0x7f6fec09d6f0 2023-09-30T22:34:25.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.701+0000 7f6fd17fa700 1 -- 172.21.15.16:0/1186222822 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fc40052e0 con 0x7f6fec09d6f0 2023-09-30T22:34:25.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.703+0000 7f6fd37fe700 1 -- 172.21.15.16:0/1186222822 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6fdc021070 con 0x7f6fec09d6f0 2023-09-30T22:34:25.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.703+0000 7f6fd37fe700 1 -- 172.21.15.16:0/1186222822 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f6fdc0410f0 con 0x7f6fec09d6f0 2023-09-30T22:34:25.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.711+0000 7f6fd37fe700 1 -- 172.21.15.16:0/1186222822 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6fdc02d7b0 con 0x7f6fec09d6f0 2023-09-30T22:34:25.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.782+0000 7fd4f563e700 1 Processor -- start 2023-09-30T22:34:25.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.782+0000 7fd4f563e700 1 -- start start 2023-09-30T22:34:25.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.783+0000 7fd4f563e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4f009d6f0 0x7fd4f009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:25.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.783+0000 7fd4f563e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd4f009e410 0x7fd4f0134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:25.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.783+0000 7fd4f563e700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4f009b120 con 0x7fd4f009e000 2023-09-30T22:34:25.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.783+0000 7fd4f563e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f006b460 con 0x7fd4f009d6f0 2023-09-30T22:34:25.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.783+0000 7fd4f563e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f009a7e0 con 0x7fd4f009e410 2023-09-30T22:34:25.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.783+0000 7fd4ee7fc700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd4f009e000 0x7fd4f01339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54956/0 (socket says 172.21.15.16:54956) 2023-09-30T22:34:25.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.783+0000 7fd4ee7fc700 1 -- 172.21.15.16:0/2209414548 learned_addr learned my addr 172.21.15.16:0/2209414548 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:34:25.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.784+0000 7fd4ef7fe700 1 --2- 172.21.15.16:0/2209414548 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd4f009e410 0x7fd4f0134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:25.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.784+0000 7fd4eeffd700 1 --2- 172.21.15.16:0/2209414548 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4f009d6f0 0x7fd4f009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:25.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.784+0000 7fd4edffb700 1 -- 172.21.15.16:0/2209414548 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2742615691 0 0) 0x7fd4f009b120 con 0x7fd4f009e000 2023-09-30T22:34:25.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.784+0000 7fd4edffb700 1 -- 172.21.15.16:0/2209414548 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4d80035b0 con 0x7fd4f009e000 2023-09-30T22:34:25.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.784+0000 7fd4ef7fe700 1 -- 172.21.15.16:0/2209414548 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4f009d6f0 msgr2=0x7fd4f009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:25.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.784+0000 7fd4ef7fe700 1 --2- 172.21.15.16:0/2209414548 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4f009d6f0 0x7fd4f009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:25.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.784+0000 7fd4ef7fe700 1 -- 172.21.15.16:0/2209414548 >> v1:172.21.15.16:6789/0 conn(0x7fd4f009e000 legacy=0x7fd4f01339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:25.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.784+0000 7fd4ef7fe700 1 -- 172.21.15.16:0/2209414548 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f009a500 con 0x7fd4f009e410 2023-09-30T22:34:25.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.784+0000 7fd4ef7fe700 1 --2- 172.21.15.16:0/2209414548 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd4f009e410 0x7fd4f0134100 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8009df0 tx=0x7fd4e800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b76b2e7cddcdd4c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:25.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.784+0000 7fd4edffb700 1 -- 172.21.15.16:0/2209414548 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e800e070 con 0x7fd4f009e410 2023-09-30T22:34:25.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.785+0000 7fd4edffb700 1 -- 172.21.15.16:0/2209414548 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4e800bae0 con 0x7fd4f009e410 2023-09-30T22:34:25.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.785+0000 7fd4edffb700 1 -- 172.21.15.16:0/2209414548 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e800be90 con 0x7fd4f009e410 2023-09-30T22:34:25.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.786+0000 7fd4f563e700 1 -- 172.21.15.16:0/2209414548 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd4f009e410 msgr2=0x7fd4f0134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:25.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.786+0000 7fd4f563e700 1 --2- 172.21.15.16:0/2209414548 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd4f009e410 0x7fd4f0134100 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8009df0 tx=0x7fd4e800b330 comp rx=0 tx=0).stop 2023-09-30T22:34:25.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.786+0000 7fd4f563e700 1 -- 172.21.15.16:0/2209414548 shutdown_connections 2023-09-30T22:34:25.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.786+0000 7fd4f563e700 1 --2- 172.21.15.16:0/2209414548 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd4f009e410 0x7fd4f0134100 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:25.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.786+0000 7fd4f563e700 1 --2- 172.21.15.16:0/2209414548 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4f009d6f0 0x7fd4f009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:25.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.786+0000 7fd4f563e700 1 -- 172.21.15.16:0/2209414548 >> 172.21.15.16:0/2209414548 conn(0x7fd4f0066f50 msgr2=0x7fd4f009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:25.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.786+0000 7fd4f563e700 1 -- 172.21.15.16:0/2209414548 shutdown_connections 2023-09-30T22:34:25.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.786+0000 7fd4f563e700 1 -- 172.21.15.16:0/2209414548 wait complete. 2023-09-30T22:34:25.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.787+0000 7fd4f563e700 1 Processor -- start 2023-09-30T22:34:25.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.788+0000 7fd4f563e700 1 -- start start 2023-09-30T22:34:25.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.788+0000 7fd4f563e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4f009d6f0 0x7fd4f0122650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:25.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.788+0000 7fd4f563e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f009e000 0x7fd4f0122b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:25.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.788+0000 7fd4f563e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd4f009e410 0x7fd4f01230d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:25.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.788+0000 7fd4f563e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f0124a10 con 0x7fd4f009e000 2023-09-30T22:34:25.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.788+0000 7fd4f563e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f01260e0 con 0x7fd4f009d6f0 2023-09-30T22:34:25.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.788+0000 7fd4f563e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f01241d0 con 0x7fd4f009e410 2023-09-30T22:34:25.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.788+0000 7fd4eeffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4f009d6f0 0x7fd4f0122650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:25.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.788+0000 7fd4ee7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f009e000 0x7fd4f0122b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:25.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.788+0000 7fd4eeffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4f009d6f0 0x7fd4f0122650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:55636/0 (socket says 172.21.15.16:55636) 2023-09-30T22:34:25.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.788+0000 7fd4eeffd700 1 -- 172.21.15.16:0/2393792409 learned_addr learned my addr 172.21.15.16:0/2393792409 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:25.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.789+0000 7fd4ef7fe700 1 --2- 172.21.15.16:0/2393792409 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd4f009e410 0x7fd4f01230d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:25.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.789+0000 7fd4eeffd700 1 -- 172.21.15.16:0/2393792409 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd4f009e410 msgr2=0x7fd4f01230d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:25.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.789+0000 7fd4eeffd700 1 --2- 172.21.15.16:0/2393792409 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd4f009e410 0x7fd4f01230d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:25.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.789+0000 7fd4eeffd700 1 -- 172.21.15.16:0/2393792409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f009e000 msgr2=0x7fd4f0122b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:25.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.789+0000 7fd4eeffd700 1 --2- 172.21.15.16:0/2393792409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f009e000 0x7fd4f0122b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:25.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.789+0000 7fd4eeffd700 1 -- 172.21.15.16:0/2393792409 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e8009a70 con 0x7fd4f009d6f0 2023-09-30T22:34:25.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.789+0000 7fd4ee7fc700 1 --2- 172.21.15.16:0/2393792409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f009e000 0x7fd4f0122b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:25.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.789+0000 7fd4eeffd700 1 --2- 172.21.15.16:0/2393792409 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4f009d6f0 0x7fd4f0122650 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fd4e000e9e0 tx=0x7fd4e000ecf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:25.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.789+0000 7fd4d77fe700 1 -- 172.21.15.16:0/2393792409 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e0009710 con 0x7fd4f009d6f0 2023-09-30T22:34:25.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.789+0000 7fd4d77fe700 1 -- 172.21.15.16:0/2393792409 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4e000c850 con 0x7fd4f009d6f0 2023-09-30T22:34:25.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.789+0000 7fd4d77fe700 1 -- 172.21.15.16:0/2393792409 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e000cbc0 con 0x7fd4f009d6f0 2023-09-30T22:34:25.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.789+0000 7fd4f563e700 1 -- 172.21.15.16:0/2393792409 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4f0071fd0 con 0x7fd4f009d6f0 2023-09-30T22:34:25.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.790+0000 7fd4f563e700 1 -- 172.21.15.16:0/2393792409 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4f009c760 con 0x7fd4f009d6f0 2023-09-30T22:34:25.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.790+0000 7fd4d77fe700 1 -- 172.21.15.16:0/2393792409 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd4e000cd20 con 0x7fd4f009d6f0 2023-09-30T22:34:25.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.791+0000 7fd4d77fe700 1 -- 172.21.15.16:0/2393792409 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd4e003f7f0 con 0x7fd4f009d6f0 2023-09-30T22:34:25.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.791+0000 7fd4f563e700 1 -- 172.21.15.16:0/2393792409 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4f009c9d0 con 0x7fd4f009d6f0 2023-09-30T22:34:25.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.800+0000 7fd4d77fe700 1 -- 172.21.15.16:0/2393792409 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd4e001bdc0 con 0x7fd4f009d6f0 2023-09-30T22:34:25.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.835+0000 7f80f22a2700 1 Processor -- start 2023-09-30T22:34:25.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.835+0000 7f80f22a2700 1 -- start start 2023-09-30T22:34:25.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.836+0000 7f80f22a2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f80ec09d6f0 0x7f80ec09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:25.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.836+0000 7f80f22a2700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f80ec09e000 0x7f80ec134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:25.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.836+0000 7f80f22a2700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80ec09b150 con 0x7f80ec09ace0 2023-09-30T22:34:25.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.836+0000 7f80f22a2700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80ec09a500 con 0x7f80ec09e000 2023-09-30T22:34:25.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.836+0000 7f80f22a2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f80ec09f720 con 0x7f80ec09d6f0 2023-09-30T22:34:25.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.836+0000 7f80ebfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f80ec09d6f0 0x7f80ec09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:25.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.836+0000 7f80ebfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f80ec09d6f0 0x7f80ec09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:48282/0 (socket says 172.21.15.16:48282) 2023-09-30T22:34:25.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.836+0000 7f80ebfff700 1 -- 172.21.15.16:0/5580711 learned_addr learned my addr 172.21.15.16:0/5580711 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:25.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.836+0000 7f80f083f700 1 --2- 172.21.15.16:0/5580711 >> v2:172.21.15.16:3300/0 conn(0x7f80ec09e000 0x7f80ec134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:25.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.836+0000 7f80ebfff700 1 -- 172.21.15.16:0/5580711 >> v2:172.21.15.16:3300/0 conn(0x7f80ec09e000 msgr2=0x7f80ec134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:25.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.836+0000 7f80ebfff700 1 --2- 172.21.15.16:0/5580711 >> v2:172.21.15.16:3300/0 conn(0x7f80ec09e000 0x7f80ec134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:25.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.836+0000 7f80eaffd700 1 -- 172.21.15.16:0/5580711 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3377024970 0 0) 0x7f80ec09b150 con 0x7f80ec09ace0 2023-09-30T22:34:25.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.836+0000 7f80ebfff700 1 -- 172.21.15.16:0/5580711 >> v1:172.21.15.16:6789/0 conn(0x7f80ec09ace0 legacy=0x7f80ec1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:25.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.836+0000 7f80ebfff700 1 -- 172.21.15.16:0/5580711 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80ec06b490 con 0x7f80ec09d6f0 2023-09-30T22:34:25.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.837+0000 7f80ebfff700 1 --2- 172.21.15.16:0/5580711 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f80ec09d6f0 0x7f80ec09dac0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f80dc0098f0 tx=0x7f80dc009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fb5a2c61ccd030f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:25.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.837+0000 7f80eaffd700 1 -- 172.21.15.16:0/5580711 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80dc00b8a0 con 0x7f80ec09d6f0 2023-09-30T22:34:25.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.837+0000 7f80eaffd700 1 -- 172.21.15.16:0/5580711 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80dc00ba00 con 0x7f80ec09d6f0 2023-09-30T22:34:25.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.837+0000 7f80eaffd700 1 -- 172.21.15.16:0/5580711 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80dc00bd90 con 0x7f80ec09d6f0 2023-09-30T22:34:25.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.838+0000 7f80f22a2700 1 -- 172.21.15.16:0/5580711 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f80ec09d6f0 msgr2=0x7f80ec09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:25.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.838+0000 7f80f22a2700 1 --2- 172.21.15.16:0/5580711 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f80ec09d6f0 0x7f80ec09dac0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f80dc0098f0 tx=0x7f80dc009c00 comp rx=0 tx=0).stop 2023-09-30T22:34:25.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.838+0000 7f80f22a2700 1 -- 172.21.15.16:0/5580711 shutdown_connections 2023-09-30T22:34:25.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.838+0000 7f80f22a2700 1 --2- 172.21.15.16:0/5580711 >> v2:172.21.15.16:3300/0 conn(0x7f80ec09e000 0x7f80ec134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:25.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.838+0000 7f80f22a2700 1 --2- 172.21.15.16:0/5580711 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f80ec09d6f0 0x7f80ec09dac0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:25.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.838+0000 7f80f22a2700 1 -- 172.21.15.16:0/5580711 >> 172.21.15.16:0/5580711 conn(0x7f80ec066f50 msgr2=0x7f80ec09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:25.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.838+0000 7f80f22a2700 1 -- 172.21.15.16:0/5580711 shutdown_connections 2023-09-30T22:34:25.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.838+0000 7f80f22a2700 1 -- 172.21.15.16:0/5580711 wait complete. 2023-09-30T22:34:25.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.839+0000 7f80f22a2700 1 Processor -- start 2023-09-30T22:34:25.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.840+0000 7f80f22a2700 1 -- start start 2023-09-30T22:34:25.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.840+0000 7f80f22a2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f80ec09ace0 0x7f80ec1a5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:25.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.840+0000 7f80f22a2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80ec09d6f0 0x7f80ec1a6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:25.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.840+0000 7f80f22a2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f80ec09e000 0x7f80ec1a6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:25.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.840+0000 7f80f22a2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80ec1a7fe0 con 0x7f80ec09d6f0 2023-09-30T22:34:25.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.840+0000 7f80f22a2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80ec1a98a0 con 0x7f80ec09e000 2023-09-30T22:34:25.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.840+0000 7f80f22a2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f80ec1a8c70 con 0x7f80ec09ace0 2023-09-30T22:34:25.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.841+0000 7f80eb7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80ec09d6f0 0x7f80ec1a6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:25.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.841+0000 7f80eb7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80ec09d6f0 0x7f80ec1a6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47724/0 (socket says 172.21.15.16:47724) 2023-09-30T22:34:25.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.841+0000 7f80eb7fe700 1 -- 172.21.15.16:0/2480785474 learned_addr learned my addr 172.21.15.16:0/2480785474 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:25.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.841+0000 7f80ebfff700 1 --2- 172.21.15.16:0/2480785474 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f80ec09ace0 0x7f80ec1a5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:25.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.841+0000 7f80f083f700 1 --2- 172.21.15.16:0/2480785474 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f80ec09e000 0x7f80ec1a6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:25.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.841+0000 7f80ebfff700 1 -- 172.21.15.16:0/2480785474 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f80ec09e000 msgr2=0x7f80ec1a6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:25.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.841+0000 7f80ebfff700 1 --2- 172.21.15.16:0/2480785474 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f80ec09e000 0x7f80ec1a6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:25.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.841+0000 7f80ebfff700 1 -- 172.21.15.16:0/2480785474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80ec09d6f0 msgr2=0x7f80ec1a6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:25.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.841+0000 7f80ebfff700 1 --2- 172.21.15.16:0/2480785474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80ec09d6f0 0x7f80ec1a6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:25.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.841+0000 7f80ebfff700 1 -- 172.21.15.16:0/2480785474 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80dc009550 con 0x7f80ec09ace0 2023-09-30T22:34:25.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.841+0000 7f80f083f700 1 --2- 172.21.15.16:0/2480785474 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f80ec09e000 0x7f80ec1a6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:25.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.841+0000 7f80eb7fe700 1 --2- 172.21.15.16:0/2480785474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80ec09d6f0 0x7f80ec1a6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:25.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.841+0000 7f80ebfff700 1 --2- 172.21.15.16:0/2480785474 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f80ec09ace0 0x7f80ec1a5e10 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f80dc016000 tx=0x7f80dc004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:25.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.842+0000 7f80e8ff9700 1 -- 172.21.15.16:0/2480785474 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80dc004510 con 0x7f80ec09ace0 2023-09-30T22:34:25.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.842+0000 7f80e8ff9700 1 -- 172.21.15.16:0/2480785474 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80dc0036a0 con 0x7f80ec09ace0 2023-09-30T22:34:25.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.842+0000 7f80f22a2700 1 -- 172.21.15.16:0/2480785474 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80ec1a9030 con 0x7f80ec09ace0 2023-09-30T22:34:25.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.842+0000 7f80f22a2700 1 -- 172.21.15.16:0/2480785474 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80ec09c6f0 con 0x7f80ec09ace0 2023-09-30T22:34:25.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.842+0000 7f80e8ff9700 1 -- 172.21.15.16:0/2480785474 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80dc003a50 con 0x7f80ec09ace0 2023-09-30T22:34:25.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.843+0000 7f80e8ff9700 1 -- 172.21.15.16:0/2480785474 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f80dc021070 con 0x7f80ec09ace0 2023-09-30T22:34:25.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.843+0000 7f80e8ff9700 1 -- 172.21.15.16:0/2480785474 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f80dc040a10 con 0x7f80ec09ace0 2023-09-30T22:34:25.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.843+0000 7f80f22a2700 1 -- 172.21.15.16:0/2480785474 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80cc0052e0 con 0x7f80ec09ace0 2023-09-30T22:34:25.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.848+0000 7f80e8ff9700 1 -- 172.21.15.16:0/2480785474 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f80dc015080 con 0x7f80ec09ace0 2023-09-30T22:34:25.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.862+0000 7f789f61a700 1 -- 172.21.15.16:0/935951517 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f7898121d30 con 0x7f78980a0a30 2023-09-30T22:34:25.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.862+0000 7f788dffb700 1 -- 172.21.15.16:0/935951517 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f789401a450 con 0x7f78980a0a30 2023-09-30T22:34:25.864 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:34:25.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.865+0000 7f789f61a700 1 -- 172.21.15.16:0/935951517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78980a0a30 msgr2=0x7f7898120120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:25.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.865+0000 7f789f61a700 1 --2- 172.21.15.16:0/935951517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78980a0a30 0x7f7898120120 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f7894009f50 tx=0x7f7894004750 comp rx=0 tx=0).stop 2023-09-30T22:34:25.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.865+0000 7f789f61a700 1 -- 172.21.15.16:0/935951517 shutdown_connections 2023-09-30T22:34:25.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.865+0000 7f789f61a700 1 --2- 172.21.15.16:0/935951517 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7898133fa0 0x7f7898120ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:25.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.865+0000 7f789f61a700 1 --2- 172.21.15.16:0/935951517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78980a0a30 0x7f7898120120 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:25.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.865+0000 7f789f61a700 1 --2- 172.21.15.16:0/935951517 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f78980a1340 0x7f7898120660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:25.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.865+0000 7f789f61a700 1 -- 172.21.15.16:0/935951517 >> 172.21.15.16:0/935951517 conn(0x7f78980a6550 msgr2=0x7f789809ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:25.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.865+0000 7f789f61a700 1 -- 172.21.15.16:0/935951517 shutdown_connections 2023-09-30T22:34:25.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.865+0000 7f789f61a700 1 -- 172.21.15.16:0/935951517 wait complete. 2023-09-30T22:34:25.880 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:34:25.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.927+0000 7f6fd17fa700 1 -- 172.21.15.16:0/1186222822 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f6fc4005860 con 0x7f6fec09d6f0 2023-09-30T22:34:25.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.928+0000 7f6fd37fe700 1 -- 172.21.15.16:0/1186222822 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6fdc033090 con 0x7f6fec09d6f0 2023-09-30T22:34:25.929 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:34:25.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.930+0000 7f6fd17fa700 1 -- 172.21.15.16:0/1186222822 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6fec09d6f0 msgr2=0x7f6fec12fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:25.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.930+0000 7f6fd17fa700 1 --2- 172.21.15.16:0/1186222822 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6fec09d6f0 0x7f6fec12fc30 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f6fec09b9e0 tx=0x7f6fdc004750 comp rx=0 tx=0).stop 2023-09-30T22:34:25.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.930+0000 7f6fd17fa700 1 -- 172.21.15.16:0/1186222822 shutdown_connections 2023-09-30T22:34:25.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.930+0000 7f6fd17fa700 1 --2- 172.21.15.16:0/1186222822 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6fec09d6f0 0x7f6fec12fc30 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:25.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.930+0000 7f6fd17fa700 1 --2- 172.21.15.16:0/1186222822 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6fec09e410 0x7f6fec1306b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:25.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.930+0000 7f6fd17fa700 1 --2- 172.21.15.16:0/1186222822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6fec09e000 0x7f6fec130170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:25.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.930+0000 7f6fd17fa700 1 -- 172.21.15.16:0/1186222822 >> 172.21.15.16:0/1186222822 conn(0x7f6fec066f50 msgr2=0x7f6fec06a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:25.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.930+0000 7f6fd17fa700 1 -- 172.21.15.16:0/1186222822 shutdown_connections 2023-09-30T22:34:25.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:25.930+0000 7f6fd17fa700 1 -- 172.21.15.16:0/1186222822 wait complete. 2023-09-30T22:34:25.945 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:34:26.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:26.058+0000 7fd4f563e700 1 -- 172.21.15.16:0/2393792409 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd4f0125aa0 con 0x7fd4f009d6f0 2023-09-30T22:34:26.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:26.059+0000 7fd4d77fe700 1 -- 172.21.15.16:0/2393792409 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd4f0125aa0 con 0x7fd4f009d6f0 2023-09-30T22:34:26.060 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:34:26.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:26.061+0000 7fd4f563e700 1 -- 172.21.15.16:0/2393792409 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4f009d6f0 msgr2=0x7fd4f0122650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:26.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:26.061+0000 7fd4f563e700 1 --2- 172.21.15.16:0/2393792409 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4f009d6f0 0x7fd4f0122650 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fd4e000e9e0 tx=0x7fd4e000ecf0 comp rx=0 tx=0).stop 2023-09-30T22:34:26.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:26.061+0000 7fd4f563e700 1 -- 172.21.15.16:0/2393792409 shutdown_connections 2023-09-30T22:34:26.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:26.061+0000 7fd4f563e700 1 --2- 172.21.15.16:0/2393792409 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4f009d6f0 0x7fd4f0122650 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:26.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:26.061+0000 7fd4f563e700 1 --2- 172.21.15.16:0/2393792409 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd4f009e410 0x7fd4f01230d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:26.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:26.061+0000 7fd4f563e700 1 --2- 172.21.15.16:0/2393792409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f009e000 0x7fd4f0122b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:26.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:26.061+0000 7fd4f563e700 1 -- 172.21.15.16:0/2393792409 >> 172.21.15.16:0/2393792409 conn(0x7fd4f0066f50 msgr2=0x7fd4f0070d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:26.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:26.061+0000 7fd4f563e700 1 -- 172.21.15.16:0/2393792409 shutdown_connections 2023-09-30T22:34:26.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:26.061+0000 7fd4f563e700 1 -- 172.21.15.16:0/2393792409 wait complete. 2023-09-30T22:34:26.076 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:34:26.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:26.091+0000 7f80f22a2700 1 -- 172.21.15.16:0/2480785474 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f80cc005150 con 0x7f80ec09ace0 2023-09-30T22:34:26.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:26.091+0000 7f80e8ff9700 1 -- 172.21.15.16:0/2480785474 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f80dc024470 con 0x7f80ec09ace0 2023-09-30T22:34:26.093 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:34:26.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:26.093+0000 7f80f22a2700 1 -- 172.21.15.16:0/2480785474 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f80ec09ace0 msgr2=0x7f80ec1a5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:26.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:26.093+0000 7f80f22a2700 1 --2- 172.21.15.16:0/2480785474 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f80ec09ace0 0x7f80ec1a5e10 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f80dc016000 tx=0x7f80dc004750 comp rx=0 tx=0).stop 2023-09-30T22:34:26.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:26.093+0000 7f80f22a2700 1 -- 172.21.15.16:0/2480785474 shutdown_connections 2023-09-30T22:34:26.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:26.093+0000 7f80f22a2700 1 --2- 172.21.15.16:0/2480785474 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f80ec09e000 0x7f80ec1a6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:26.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:26.093+0000 7f80f22a2700 1 --2- 172.21.15.16:0/2480785474 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f80ec09ace0 0x7f80ec1a5e10 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:26.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:26.093+0000 7f80f22a2700 1 --2- 172.21.15.16:0/2480785474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f80ec09d6f0 0x7f80ec1a6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:26.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:26.093+0000 7f80f22a2700 1 -- 172.21.15.16:0/2480785474 >> 172.21.15.16:0/2480785474 conn(0x7f80ec066f50 msgr2=0x7f80ec1bae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:26.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:26.094+0000 7f80f22a2700 1 -- 172.21.15.16:0/2480785474 shutdown_connections 2023-09-30T22:34:26.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:26.094+0000 7f80f22a2700 1 -- 172.21.15.16:0/2480785474 wait complete. 2023-09-30T22:34:26.108 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:34:26.882 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:34:26.946 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:34:27.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.031+0000 7fe94dfbd700 1 Processor -- start 2023-09-30T22:34:27.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.031+0000 7fe94dfbd700 1 -- start start 2023-09-30T22:34:27.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.031+0000 7fe94dfbd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe9480a0a30 0x7fe9480a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:27.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.032+0000 7fe94dfbd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe94809d630 0x7fe9480a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:27.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.032+0000 7fe94dfbd700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9480a2a70 con 0x7fe948133930 2023-09-30T22:34:27.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.032+0000 7fe94dfbd700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9480a01d0 con 0x7fe94809d630 2023-09-30T22:34:27.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.032+0000 7fe94dfbd700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe94809e2b0 con 0x7fe9480a0a30 2023-09-30T22:34:27.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.032+0000 7fe9477fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe9480a0a30 0x7fe9480a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:27.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.032+0000 7fe946ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe94809d630 0x7fe9480a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:27.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.032+0000 7fe9477fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe9480a0a30 0x7fe9480a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:48302/0 (socket says 172.21.15.16:48302) 2023-09-30T22:34:27.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.032+0000 7fe9477fe700 1 -- 172.21.15.16:0/2694846523 learned_addr learned my addr 172.21.15.16:0/2694846523 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:27.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.032+0000 7fe946ffd700 1 -- 172.21.15.16:0/2694846523 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe9480a0a30 msgr2=0x7fe9480a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:27.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.032+0000 7fe946ffd700 1 --2- 172.21.15.16:0/2694846523 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe9480a0a30 0x7fe9480a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:27.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.032+0000 7fe946ffd700 1 -- 172.21.15.16:0/2694846523 >> v1:172.21.15.16:6789/0 conn(0x7fe948133930 legacy=0x7fe948133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:27.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.032+0000 7fe946ffd700 1 -- 172.21.15.16:0/2694846523 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe94809dc50 con 0x7fe94809d630 2023-09-30T22:34:27.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.032+0000 7fe9467fc700 1 -- 172.21.15.16:0/2694846523 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3951414380 0 0) 0x7fe9480a2a70 con 0x7fe948133930 2023-09-30T22:34:27.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.032+0000 7fe9477fe700 1 --2- 172.21.15.16:0/2694846523 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe9480a0a30 0x7fe9480a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:27.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.032+0000 7fe946ffd700 1 --2- 172.21.15.16:0/2694846523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe94809d630 0x7fe9480a1340 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fe930009840 tx=0x7fe930009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98557fc4c91b25bf server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:27.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.033+0000 7fe9467fc700 1 -- 172.21.15.16:0/2694846523 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe930011830 con 0x7fe94809d630 2023-09-30T22:34:27.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.033+0000 7fe9467fc700 1 -- 172.21.15.16:0/2694846523 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe930011990 con 0x7fe94809d630 2023-09-30T22:34:27.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.033+0000 7fe9467fc700 1 -- 172.21.15.16:0/2694846523 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe930011d20 con 0x7fe94809d630 2023-09-30T22:34:27.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.033+0000 7fe94dfbd700 1 -- 172.21.15.16:0/2694846523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe94809d630 msgr2=0x7fe9480a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:27.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.033+0000 7fe94dfbd700 1 --2- 172.21.15.16:0/2694846523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe94809d630 0x7fe9480a1340 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fe930009840 tx=0x7fe930009b50 comp rx=0 tx=0).stop 2023-09-30T22:34:27.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.033+0000 7fe94dfbd700 1 -- 172.21.15.16:0/2694846523 shutdown_connections 2023-09-30T22:34:27.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.033+0000 7fe94dfbd700 1 --2- 172.21.15.16:0/2694846523 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe9480a0a30 0x7fe9480a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:27.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.033+0000 7fe94dfbd700 1 --2- 172.21.15.16:0/2694846523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe94809d630 0x7fe9480a1340 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:27.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.033+0000 7fe94dfbd700 1 -- 172.21.15.16:0/2694846523 >> 172.21.15.16:0/2694846523 conn(0x7fe9480a3050 msgr2=0x7fe9480a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:27.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.033+0000 7fe94dfbd700 1 -- 172.21.15.16:0/2694846523 shutdown_connections 2023-09-30T22:34:27.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.033+0000 7fe94dfbd700 1 -- 172.21.15.16:0/2694846523 wait complete. 2023-09-30T22:34:27.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.035+0000 7fe94dfbd700 1 Processor -- start 2023-09-30T22:34:27.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.035+0000 7fe94dfbd700 1 -- start start 2023-09-30T22:34:27.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.036+0000 7fe94dfbd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe94809d630 0x7fe94811f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:27.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.036+0000 7fe94dfbd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe9480a0a30 0x7fe9481c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:27.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.036+0000 7fe94dfbd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe948133930 0x7fe9481c84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:27.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.036+0000 7fe94dfbd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe948121f30 con 0x7fe94809d630 2023-09-30T22:34:27.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.036+0000 7fe94dfbd700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9480a0410 con 0x7fe948133930 2023-09-30T22:34:27.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.036+0000 7fe94dfbd700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe94811f1e0 con 0x7fe9480a0a30 2023-09-30T22:34:27.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.036+0000 7fe947fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe948133930 0x7fe9481c84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:27.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.036+0000 7fe947fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe948133930 0x7fe9481c84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:55658/0 (socket says 172.21.15.16:55658) 2023-09-30T22:34:27.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.036+0000 7fe9477fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe94809d630 0x7fe94811f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:27.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.036+0000 7fe947fff700 1 -- 172.21.15.16:0/451650506 learned_addr learned my addr 172.21.15.16:0/451650506 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:27.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.036+0000 7fe946ffd700 1 --2- 172.21.15.16:0/451650506 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe9480a0a30 0x7fe9481c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:27.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.036+0000 7fe947fff700 1 -- 172.21.15.16:0/451650506 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe9480a0a30 msgr2=0x7fe9481c7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:27.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.036+0000 7fe947fff700 1 --2- 172.21.15.16:0/451650506 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe9480a0a30 0x7fe9481c7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:27.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.036+0000 7fe947fff700 1 -- 172.21.15.16:0/451650506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe94809d630 msgr2=0x7fe94811f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:27.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.036+0000 7fe947fff700 1 --2- 172.21.15.16:0/451650506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe94809d630 0x7fe94811f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:27.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.036+0000 7fe947fff700 1 -- 172.21.15.16:0/451650506 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe940009b30 con 0x7fe948133930 2023-09-30T22:34:27.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.036+0000 7fe9477fe700 1 --2- 172.21.15.16:0/451650506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe94809d630 0x7fe94811f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:27.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.036+0000 7fe947fff700 1 --2- 172.21.15.16:0/451650506 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe948133930 0x7fe9481c84d0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fe93800c4d0 tx=0x7fe93800c7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:27.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.037+0000 7fe93ffff700 1 -- 172.21.15.16:0/451650506 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9380054e0 con 0x7fe948133930 2023-09-30T22:34:27.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.037+0000 7fe93ffff700 1 -- 172.21.15.16:0/451650506 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe938003680 con 0x7fe948133930 2023-09-30T22:34:27.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.037+0000 7fe94dfbd700 1 -- 172.21.15.16:0/451650506 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9300094a0 con 0x7fe948133930 2023-09-30T22:34:27.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.037+0000 7fe93ffff700 1 -- 172.21.15.16:0/451650506 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe938003a30 con 0x7fe948133930 2023-09-30T22:34:27.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.037+0000 7fe94dfbd700 1 -- 172.21.15.16:0/451650506 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe948120550 con 0x7fe948133930 2023-09-30T22:34:27.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.038+0000 7fe94dfbd700 1 -- 172.21.15.16:0/451650506 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe948120740 con 0x7fe948133930 2023-09-30T22:34:27.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.038+0000 7fe93ffff700 1 -- 172.21.15.16:0/451650506 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe938003b90 con 0x7fe948133930 2023-09-30T22:34:27.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.038+0000 7fe93ffff700 1 -- 172.21.15.16:0/451650506 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fe93803eb80 con 0x7fe948133930 2023-09-30T22:34:27.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.043+0000 7fe93ffff700 1 -- 172.21.15.16:0/451650506 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe938003e40 con 0x7fe948133930 2023-09-30T22:34:27.078 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:34:27.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.088+0000 7f0a0d27a700 1 Processor -- start 2023-09-30T22:34:27.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.088+0000 7f0a0d27a700 1 -- start start 2023-09-30T22:34:27.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.089+0000 7f0a0d27a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0a08130670 0x7f0a08130a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:27.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.089+0000 7f0a0d27a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0a080701a0 0x7f0a08138540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:27.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.089+0000 7f0a0d27a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a08130260 con 0x7f0a0806f680 2023-09-30T22:34:27.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.089+0000 7f0a0d27a700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a08071ff0 con 0x7f0a080701a0 2023-09-30T22:34:27.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.089+0000 7f0a0d27a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0809ecf0 con 0x7f0a08130670 2023-09-30T22:34:27.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.089+0000 7f0a06ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0a08130670 0x7f0a08130a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:27.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.090+0000 7f0a06ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0a08130670 0x7f0a08130a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:48322/0 (socket says 172.21.15.16:48322) 2023-09-30T22:34:27.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.090+0000 7f0a06ffd700 1 -- 172.21.15.16:0/4130716071 learned_addr learned my addr 172.21.15.16:0/4130716071 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:27.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.090+0000 7f0a077fe700 1 --2- 172.21.15.16:0/4130716071 >> v2:172.21.15.16:3300/0 conn(0x7f0a080701a0 0x7f0a08138540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:27.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.090+0000 7f0a06ffd700 1 -- 172.21.15.16:0/4130716071 >> v2:172.21.15.16:3300/0 conn(0x7f0a080701a0 msgr2=0x7f0a08138540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:27.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.090+0000 7f0a06ffd700 1 --2- 172.21.15.16:0/4130716071 >> v2:172.21.15.16:3300/0 conn(0x7f0a080701a0 0x7f0a08138540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:27.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.090+0000 7f0a06ffd700 1 -- 172.21.15.16:0/4130716071 >> v1:172.21.15.16:6789/0 conn(0x7f0a0806f680 legacy=0x7f0a0806fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:27.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.090+0000 7f0a06ffd700 1 -- 172.21.15.16:0/4130716071 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a08070e70 con 0x7f0a08130670 2023-09-30T22:34:27.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.090+0000 7f0a06ffd700 1 --2- 172.21.15.16:0/4130716071 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0a08130670 0x7f0a08130a40 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f09f80098f0 tx=0x7f09f8009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c58c6593f299183 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:27.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.090+0000 7f0a05ffb700 1 -- 172.21.15.16:0/4130716071 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f8011830 con 0x7f0a08130670 2023-09-30T22:34:27.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.090+0000 7f0a05ffb700 1 -- 172.21.15.16:0/4130716071 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09f8011990 con 0x7f0a08130670 2023-09-30T22:34:27.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.090+0000 7f0a05ffb700 1 -- 172.21.15.16:0/4130716071 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f8011d20 con 0x7f0a08130670 2023-09-30T22:34:27.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.091+0000 7f0a0d27a700 1 -- 172.21.15.16:0/4130716071 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0a08130670 msgr2=0x7f0a08130a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:27.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.091+0000 7f0a0d27a700 1 --2- 172.21.15.16:0/4130716071 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0a08130670 0x7f0a08130a40 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f09f80098f0 tx=0x7f09f8009c00 comp rx=0 tx=0).stop 2023-09-30T22:34:27.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.091+0000 7f0a0d27a700 1 -- 172.21.15.16:0/4130716071 shutdown_connections 2023-09-30T22:34:27.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.091+0000 7f0a0d27a700 1 --2- 172.21.15.16:0/4130716071 >> v2:172.21.15.16:3300/0 conn(0x7f0a080701a0 0x7f0a08138540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:27.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.091+0000 7f0a0d27a700 1 --2- 172.21.15.16:0/4130716071 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0a08130670 0x7f0a08130a40 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:27.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.091+0000 7f0a0d27a700 1 -- 172.21.15.16:0/4130716071 >> 172.21.15.16:0/4130716071 conn(0x7f0a080a2aa0 msgr2=0x7f0a080a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:27.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.091+0000 7f0a0d27a700 1 -- 172.21.15.16:0/4130716071 shutdown_connections 2023-09-30T22:34:27.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.091+0000 7f0a0d27a700 1 -- 172.21.15.16:0/4130716071 wait complete. 2023-09-30T22:34:27.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.093+0000 7f0a0d27a700 1 Processor -- start 2023-09-30T22:34:27.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.093+0000 7f0a0d27a700 1 -- start start 2023-09-30T22:34:27.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.093+0000 7f0a0d27a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0a0806f680 0x7f0a08133e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:27.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.094+0000 7f0a0d27a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a080701a0 0x7f0a081343d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:27.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.094+0000 7f0a0d27a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0a08130670 0x7f0a08134910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:27.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.094+0000 7f0a0d27a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a08136200 con 0x7f0a080701a0 2023-09-30T22:34:27.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.094+0000 7f0a0d27a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0809f8b0 con 0x7f0a0806f680 2023-09-30T22:34:27.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.094+0000 7f0a0d27a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0a081378b0 con 0x7f0a08130670 2023-09-30T22:34:27.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.094+0000 7f0a067fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a080701a0 0x7f0a081343d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:27.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.094+0000 7f0a067fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a080701a0 0x7f0a081343d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47770/0 (socket says 172.21.15.16:47770) 2023-09-30T22:34:27.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.094+0000 7f0a067fc700 1 -- 172.21.15.16:0/2604781934 learned_addr learned my addr 172.21.15.16:0/2604781934 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:27.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.094+0000 7f0a06ffd700 1 --2- 172.21.15.16:0/2604781934 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0a0806f680 0x7f0a08133e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:27.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.094+0000 7f0a077fe700 1 --2- 172.21.15.16:0/2604781934 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0a08130670 0x7f0a08134910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:27.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.094+0000 7f0a067fc700 1 -- 172.21.15.16:0/2604781934 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0a08130670 msgr2=0x7f0a08134910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:27.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.094+0000 7f0a067fc700 1 --2- 172.21.15.16:0/2604781934 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0a08130670 0x7f0a08134910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:27.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.094+0000 7f0a067fc700 1 -- 172.21.15.16:0/2604781934 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0a0806f680 msgr2=0x7f0a08133e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:27.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.094+0000 7f0a067fc700 1 --2- 172.21.15.16:0/2604781934 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0a0806f680 0x7f0a08133e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:27.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.094+0000 7f0a067fc700 1 -- 172.21.15.16:0/2604781934 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09f8009550 con 0x7f0a080701a0 2023-09-30T22:34:27.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.094+0000 7f0a067fc700 1 --2- 172.21.15.16:0/2604781934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a080701a0 0x7f0a081343d0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f09fc00c4b0 tx=0x7f09fc00c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:27.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.095+0000 7f09ef7fe700 1 -- 172.21.15.16:0/2604781934 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc0052d0 con 0x7f0a080701a0 2023-09-30T22:34:27.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.095+0000 7f09ef7fe700 1 -- 172.21.15.16:0/2604781934 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09fc005430 con 0x7f0a080701a0 2023-09-30T22:34:27.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.095+0000 7f0a0d27a700 1 -- 172.21.15.16:0/2604781934 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a08135930 con 0x7f0a080701a0 2023-09-30T22:34:27.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.095+0000 7f09ef7fe700 1 -- 172.21.15.16:0/2604781934 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc0038c0 con 0x7f0a080701a0 2023-09-30T22:34:27.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.095+0000 7f0a0d27a700 1 -- 172.21.15.16:0/2604781934 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a081375d0 con 0x7f0a080701a0 2023-09-30T22:34:27.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.096+0000 7f09ef7fe700 1 -- 172.21.15.16:0/2604781934 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f09fc003a20 con 0x7f0a080701a0 2023-09-30T22:34:27.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.096+0000 7f0a0d27a700 1 -- 172.21.15.16:0/2604781934 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a0809dd00 con 0x7f0a080701a0 2023-09-30T22:34:27.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.096+0000 7f09ef7fe700 1 -- 172.21.15.16:0/2604781934 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f09fc03eb00 con 0x7f0a080701a0 2023-09-30T22:34:27.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.101+0000 7f09ef7fe700 1 -- 172.21.15.16:0/2604781934 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f09fc019e60 con 0x7f0a080701a0 2023-09-30T22:34:27.109 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:34:27.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.219+0000 7ff07ed94700 1 Processor -- start 2023-09-30T22:34:27.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.219+0000 7ff07ed94700 1 -- start start 2023-09-30T22:34:27.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.220+0000 7ff07ed94700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff07806a770 0x7ff07806ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:27.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.220+0000 7ff07ed94700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff07806b080 0x7ff078131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:27.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.220+0000 7ff07ed94700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff07809ece0 con 0x7ff078131b40 2023-09-30T22:34:27.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.220+0000 7ff07ed94700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff07809e760 con 0x7ff07806b080 2023-09-30T22:34:27.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.220+0000 7ff07ed94700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff07809e040 con 0x7ff07806a770 2023-09-30T22:34:27.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.220+0000 7ff07d331700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff078131b40 0x7ff078138850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:54986/0 (socket says 172.21.15.16:54986) 2023-09-30T22:34:27.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.220+0000 7ff07d331700 1 -- 172.21.15.16:0/908841974 learned_addr learned my addr 172.21.15.16:0/908841974 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:34:27.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.220+0000 7ff077fff700 1 --2- 172.21.15.16:0/908841974 >> v2:172.21.15.16:3300/0 conn(0x7ff07806b080 0x7ff078131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:27.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.220+0000 7ff07cb30700 1 --2- 172.21.15.16:0/908841974 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff07806a770 0x7ff07806ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:27.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.220+0000 7ff077fff700 1 -- 172.21.15.16:0/908841974 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff07806a770 msgr2=0x7ff07806ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:27.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.220+0000 7ff077fff700 1 --2- 172.21.15.16:0/908841974 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff07806a770 0x7ff07806ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:27.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.220+0000 7ff077fff700 1 -- 172.21.15.16:0/908841974 >> v1:172.21.15.16:6789/0 conn(0x7ff078131b40 legacy=0x7ff078138850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:27.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.220+0000 7ff077fff700 1 -- 172.21.15.16:0/908841974 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff07809f630 con 0x7ff07806b080 2023-09-30T22:34:27.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.220+0000 7ff07cb30700 1 --2- 172.21.15.16:0/908841974 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff07806a770 0x7ff07806ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:34:27.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.221+0000 7ff077fff700 1 --2- 172.21.15.16:0/908841974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff07806b080 0x7ff078131600 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7ff064009840 tx=0x7ff064009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13ef673a45071c83 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:27.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.221+0000 7ff0777fe700 1 -- 172.21.15.16:0/908841974 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff064011830 con 0x7ff07806b080 2023-09-30T22:34:27.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.221+0000 7ff0777fe700 1 -- 172.21.15.16:0/908841974 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff064011990 con 0x7ff07806b080 2023-09-30T22:34:27.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.221+0000 7ff0777fe700 1 -- 172.21.15.16:0/908841974 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff064011d20 con 0x7ff07806b080 2023-09-30T22:34:27.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.222+0000 7ff07ed94700 1 -- 172.21.15.16:0/908841974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff07806b080 msgr2=0x7ff078131600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:27.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.222+0000 7ff07ed94700 1 --2- 172.21.15.16:0/908841974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff07806b080 0x7ff078131600 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7ff064009840 tx=0x7ff064009b50 comp rx=0 tx=0).stop 2023-09-30T22:34:27.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.222+0000 7ff07ed94700 1 -- 172.21.15.16:0/908841974 shutdown_connections 2023-09-30T22:34:27.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.222+0000 7ff07ed94700 1 --2- 172.21.15.16:0/908841974 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff07806a770 0x7ff07806ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:27.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.222+0000 7ff07ed94700 1 --2- 172.21.15.16:0/908841974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff07806b080 0x7ff078131600 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:27.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.222+0000 7ff07ed94700 1 -- 172.21.15.16:0/908841974 >> 172.21.15.16:0/908841974 conn(0x7ff078066f50 msgr2=0x7ff07809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:27.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.222+0000 7ff07ed94700 1 -- 172.21.15.16:0/908841974 shutdown_connections 2023-09-30T22:34:27.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.222+0000 7ff07ed94700 1 -- 172.21.15.16:0/908841974 wait complete. 2023-09-30T22:34:27.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.223+0000 7ff07ed94700 1 Processor -- start 2023-09-30T22:34:27.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.223+0000 7ff07ed94700 1 -- start start 2023-09-30T22:34:27.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.224+0000 7ff07ed94700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff07806a770 0x7ff07811c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:27.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.224+0000 7ff07ed94700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff07806b080 0x7ff07811ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:27.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.224+0000 7ff07ed94700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff078131b40 0x7ff07811cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:27.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.224+0000 7ff07ed94700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff07811e110 con 0x7ff07806b080 2023-09-30T22:34:27.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.224+0000 7ff07ed94700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff078071920 con 0x7ff078131b40 2023-09-30T22:34:27.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.224+0000 7ff07ed94700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff078071e60 con 0x7ff07806a770 2023-09-30T22:34:27.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.224+0000 7ff077fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff07806b080 0x7ff07811ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:27.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.224+0000 7ff077fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff07806b080 0x7ff07811ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47794/0 (socket says 172.21.15.16:47794) 2023-09-30T22:34:27.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.224+0000 7ff077fff700 1 -- 172.21.15.16:0/69979607 learned_addr learned my addr 172.21.15.16:0/69979607 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:27.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.224+0000 7ff07cb30700 1 --2- 172.21.15.16:0/69979607 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff07806a770 0x7ff07811c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:27.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.224+0000 7ff07d331700 1 --2- 172.21.15.16:0/69979607 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff078131b40 0x7ff07811cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:27.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.224+0000 7ff077fff700 1 -- 172.21.15.16:0/69979607 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff07806a770 msgr2=0x7ff07811c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:27.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.224+0000 7ff077fff700 1 --2- 172.21.15.16:0/69979607 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff07806a770 0x7ff07811c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:27.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.224+0000 7ff077fff700 1 -- 172.21.15.16:0/69979607 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff078131b40 msgr2=0x7ff07811cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:27.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.224+0000 7ff077fff700 1 --2- 172.21.15.16:0/69979607 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff078131b40 0x7ff07811cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:27.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.224+0000 7ff077fff700 1 -- 172.21.15.16:0/69979607 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0640094a0 con 0x7ff07806b080 2023-09-30T22:34:27.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.224+0000 7ff07d331700 1 --2- 172.21.15.16:0/69979607 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff078131b40 0x7ff07811cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:34:27.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.225+0000 7ff077fff700 1 --2- 172.21.15.16:0/69979607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff07806b080 0x7ff07811ca30 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7ff064009b90 tx=0x7ff064004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:27.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.225+0000 7ff0757fa700 1 -- 172.21.15.16:0/69979607 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0640045d0 con 0x7ff07806b080 2023-09-30T22:34:27.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.225+0000 7ff0757fa700 1 -- 172.21.15.16:0/69979607 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff06401f070 con 0x7ff07806b080 2023-09-30T22:34:27.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.225+0000 7ff07ed94700 1 -- 172.21.15.16:0/69979607 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff07811e860 con 0x7ff07806b080 2023-09-30T22:34:27.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.225+0000 7ff0757fa700 1 -- 172.21.15.16:0/69979607 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0640038c0 con 0x7ff07806b080 2023-09-30T22:34:27.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.225+0000 7ff07ed94700 1 -- 172.21.15.16:0/69979607 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff078130f10 con 0x7ff07806b080 2023-09-30T22:34:27.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.226+0000 7ff0757fa700 1 -- 172.21.15.16:0/69979607 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff064003a20 con 0x7ff07806b080 2023-09-30T22:34:27.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.226+0000 7ff07ed94700 1 -- 172.21.15.16:0/69979607 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff07811fae0 con 0x7ff07806b080 2023-09-30T22:34:27.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.226+0000 7ff0757fa700 1 -- 172.21.15.16:0/69979607 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7ff064021070 con 0x7ff07806b080 2023-09-30T22:34:27.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.236+0000 7ff0757fa700 1 -- 172.21.15.16:0/69979607 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff06402f3a0 con 0x7ff07806b080 2023-09-30T22:34:27.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.253+0000 7fa574165700 1 Processor -- start 2023-09-30T22:34:27.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.253+0000 7fa574165700 1 -- start start 2023-09-30T22:34:27.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.254+0000 7fa574165700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa56c09d6f0 0x7fa56c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:27.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.254+0000 7fa574165700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa56c09e000 0x7fa56c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:27.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.254+0000 7fa574165700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa56c09e410 0x7fa56c133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:27.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.254+0000 7fa574165700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa56c06b460 con 0x7fa56c09e410 2023-09-30T22:34:27.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.254+0000 7fa574165700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa56c09b120 con 0x7fa56c09e000 2023-09-30T22:34:27.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.254+0000 7fa574165700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa56c09a7e0 con 0x7fa56c09d6f0 2023-09-30T22:34:27.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.254+0000 7fa571700700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa56c09e000 0x7fa56c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:27.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.254+0000 7fa572702700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa56c09e410 0x7fa56c133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:27.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.254+0000 7fa571700700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa56c09e000 0x7fa56c1339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:55678/0 (socket says 172.21.15.16:55678) 2023-09-30T22:34:27.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.254+0000 7fa571700700 1 -- 172.21.15.16:0/2468737877 learned_addr learned my addr 172.21.15.16:0/2468737877 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:27.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.254+0000 7fa571f01700 1 --2- 172.21.15.16:0/2468737877 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa56c09d6f0 0x7fa56c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:27.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.255+0000 7fa572702700 1 -- 172.21.15.16:0/2468737877 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa56c09d6f0 msgr2=0x7fa56c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:27.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.255+0000 7fa572702700 1 --2- 172.21.15.16:0/2468737877 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa56c09d6f0 0x7fa56c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:27.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.255+0000 7fa572702700 1 -- 172.21.15.16:0/2468737877 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa56c09e000 msgr2=0x7fa56c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:27.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.255+0000 7fa572702700 1 --2- 172.21.15.16:0/2468737877 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa56c09e000 0x7fa56c1339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:27.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.255+0000 7fa572702700 1 -- 172.21.15.16:0/2468737877 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa56c09a500 con 0x7fa56c09e410 2023-09-30T22:34:27.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.255+0000 7fa571700700 1 --2- 172.21.15.16:0/2468737877 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa56c09e000 0x7fa56c1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:27.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.255+0000 7fa572702700 1 --2- 172.21.15.16:0/2468737877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa56c09e410 0x7fa56c133f30 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fa564009d90 tx=0x7fa56400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b567c1e28837ba3d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:27.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.255+0000 7fa570eff700 1 -- 172.21.15.16:0/2468737877 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56400e070 con 0x7fa56c09e410 2023-09-30T22:34:27.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.255+0000 7fa570eff700 1 -- 172.21.15.16:0/2468737877 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa56400bb20 con 0x7fa56c09e410 2023-09-30T22:34:27.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.255+0000 7fa570eff700 1 -- 172.21.15.16:0/2468737877 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa564012430 con 0x7fa56c09e410 2023-09-30T22:34:27.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.256+0000 7fa574165700 1 -- 172.21.15.16:0/2468737877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa56c09e410 msgr2=0x7fa56c133f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:27.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.256+0000 7fa574165700 1 --2- 172.21.15.16:0/2468737877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa56c09e410 0x7fa56c133f30 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fa564009d90 tx=0x7fa56400b330 comp rx=0 tx=0).stop 2023-09-30T22:34:27.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.256+0000 7fa574165700 1 -- 172.21.15.16:0/2468737877 shutdown_connections 2023-09-30T22:34:27.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.256+0000 7fa574165700 1 --2- 172.21.15.16:0/2468737877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa56c09e410 0x7fa56c133f30 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:27.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.256+0000 7fa574165700 1 --2- 172.21.15.16:0/2468737877 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa56c09d6f0 0x7fa56c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:27.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.256+0000 7fa574165700 1 --2- 172.21.15.16:0/2468737877 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa56c09e000 0x7fa56c1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:27.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.256+0000 7fa574165700 1 -- 172.21.15.16:0/2468737877 >> 172.21.15.16:0/2468737877 conn(0x7fa56c066f50 msgr2=0x7fa56c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:27.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.256+0000 7fa574165700 1 -- 172.21.15.16:0/2468737877 shutdown_connections 2023-09-30T22:34:27.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.256+0000 7fa574165700 1 -- 172.21.15.16:0/2468737877 wait complete. 2023-09-30T22:34:27.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.257+0000 7fa574165700 1 Processor -- start 2023-09-30T22:34:27.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.257+0000 7fa574165700 1 -- start start 2023-09-30T22:34:27.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.258+0000 7fa574165700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa56c09d6f0 0x7fa56c1a5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:27.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.258+0000 7fa574165700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa56c09e000 0x7fa56c1a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:27.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.258+0000 7fa574165700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa56c09e410 0x7fa56c1a6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:27.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.258+0000 7fa574165700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56c1a9680 con 0x7fa56c09e410 2023-09-30T22:34:27.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.258+0000 7fa574165700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa56c09c710 con 0x7fa56c09e000 2023-09-30T22:34:27.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.258+0000 7fa574165700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa56c1a79a0 con 0x7fa56c09d6f0 2023-09-30T22:34:27.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.258+0000 7fa571f01700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa56c09d6f0 0x7fa56c1a5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:27.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.258+0000 7fa571700700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa56c09e000 0x7fa56c1a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:27.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.258+0000 7fa571f01700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa56c09d6f0 0x7fa56c1a5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:48352/0 (socket says 172.21.15.16:48352) 2023-09-30T22:34:27.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.258+0000 7fa571700700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa56c09e000 0x7fa56c1a6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:55690/0 (socket says 172.21.15.16:55690) 2023-09-30T22:34:27.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.258+0000 7fa571f01700 1 -- 172.21.15.16:0/3075272554 learned_addr learned my addr 172.21.15.16:0/3075272554 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:27.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.258+0000 7fa572702700 1 --2- 172.21.15.16:0/3075272554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa56c09e410 0x7fa56c1a6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:27.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.259+0000 7fa571f01700 1 -- 172.21.15.16:0/3075272554 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa56c09e000 msgr2=0x7fa56c1a6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:27.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.259+0000 7fa571f01700 1 --2- 172.21.15.16:0/3075272554 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa56c09e000 0x7fa56c1a6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:27.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.259+0000 7fa571f01700 1 -- 172.21.15.16:0/3075272554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa56c09e410 msgr2=0x7fa56c1a6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:27.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.259+0000 7fa571f01700 1 --2- 172.21.15.16:0/3075272554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa56c09e410 0x7fa56c1a6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:27.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.259+0000 7fa571f01700 1 -- 172.21.15.16:0/3075272554 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa55c009550 con 0x7fa56c09d6f0 2023-09-30T22:34:27.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.259+0000 7fa572702700 1 --2- 172.21.15.16:0/3075272554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa56c09e410 0x7fa56c1a6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:34:27.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.259+0000 7fa571f01700 1 --2- 172.21.15.16:0/3075272554 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa56c09d6f0 0x7fa56c1a5de0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fa56800e9e0 tx=0x7fa56800ecf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:27.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.259+0000 7fa5627fc700 1 -- 172.21.15.16:0/3075272554 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa568009710 con 0x7fa56c09d6f0 2023-09-30T22:34:27.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.259+0000 7fa574165700 1 -- 172.21.15.16:0/3075272554 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa564009a70 con 0x7fa56c09d6f0 2023-09-30T22:34:27.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.259+0000 7fa574165700 1 -- 172.21.15.16:0/3075272554 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa56c0a30e0 con 0x7fa56c09d6f0 2023-09-30T22:34:27.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.259+0000 7fa5627fc700 1 -- 172.21.15.16:0/3075272554 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa56800c850 con 0x7fa56c09d6f0 2023-09-30T22:34:27.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.259+0000 7fa5627fc700 1 -- 172.21.15.16:0/3075272554 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56800cbc0 con 0x7fa56c09d6f0 2023-09-30T22:34:27.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.260+0000 7fa574165700 1 -- 172.21.15.16:0/3075272554 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5540052e0 con 0x7fa56c09d6f0 2023-09-30T22:34:27.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.260+0000 7fa5627fc700 1 -- 172.21.15.16:0/3075272554 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa56800cd20 con 0x7fa56c09d6f0 2023-09-30T22:34:27.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.261+0000 7fa5627fc700 1 -- 172.21.15.16:0/3075272554 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fa56803fa00 con 0x7fa56c09d6f0 2023-09-30T22:34:27.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.269+0000 7fe94dfbd700 1 -- 172.21.15.16:0/451650506 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe948122340 con 0x7fe948133930 2023-09-30T22:34:27.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.269+0000 7fe93ffff700 1 -- 172.21.15.16:0/451650506 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe938017070 con 0x7fe948133930 2023-09-30T22:34:27.270 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:34:27.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.269+0000 7fa5627fc700 1 -- 172.21.15.16:0/3075272554 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa568024d20 con 0x7fa56c09d6f0 2023-09-30T22:34:27.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.272+0000 7fe94dfbd700 1 -- 172.21.15.16:0/451650506 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe948133930 msgr2=0x7fe9481c84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:27.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.272+0000 7fe94dfbd700 1 --2- 172.21.15.16:0/451650506 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe948133930 0x7fe9481c84d0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fe93800c4d0 tx=0x7fe93800c7e0 comp rx=0 tx=0).stop 2023-09-30T22:34:27.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.272+0000 7fe94dfbd700 1 -- 172.21.15.16:0/451650506 shutdown_connections 2023-09-30T22:34:27.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.272+0000 7fe94dfbd700 1 --2- 172.21.15.16:0/451650506 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe948133930 0x7fe9481c84d0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:27.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.272+0000 7fe94dfbd700 1 --2- 172.21.15.16:0/451650506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe94809d630 0x7fe94811f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:27.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.272+0000 7fe94dfbd700 1 --2- 172.21.15.16:0/451650506 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe9480a0a30 0x7fe9481c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:27.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.272+0000 7fe94dfbd700 1 -- 172.21.15.16:0/451650506 >> 172.21.15.16:0/451650506 conn(0x7fe9480a3050 msgr2=0x7fe9481bbb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:27.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.273+0000 7fe94dfbd700 1 -- 172.21.15.16:0/451650506 shutdown_connections 2023-09-30T22:34:27.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.273+0000 7fe94dfbd700 1 -- 172.21.15.16:0/451650506 wait complete. 2023-09-30T22:34:27.294 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:34:27.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.322+0000 7f0a0d27a700 1 -- 172.21.15.16:0/2604781934 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0a08074ea0 con 0x7f0a080701a0 2023-09-30T22:34:27.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.323+0000 7f09ef7fe700 1 -- 172.21.15.16:0/2604781934 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f09fc019a70 con 0x7f0a080701a0 2023-09-30T22:34:27.324 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:34:27.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.325+0000 7f0a0d27a700 1 -- 172.21.15.16:0/2604781934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a080701a0 msgr2=0x7f0a081343d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:27.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.325+0000 7f0a0d27a700 1 --2- 172.21.15.16:0/2604781934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a080701a0 0x7f0a081343d0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f09fc00c4b0 tx=0x7f09fc00c7c0 comp rx=0 tx=0).stop 2023-09-30T22:34:27.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.325+0000 7f0a0d27a700 1 -- 172.21.15.16:0/2604781934 shutdown_connections 2023-09-30T22:34:27.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.325+0000 7f0a0d27a700 1 --2- 172.21.15.16:0/2604781934 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0a0806f680 0x7f0a08133e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:27.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.325+0000 7f0a0d27a700 1 --2- 172.21.15.16:0/2604781934 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0a08130670 0x7f0a08134910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:27.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.325+0000 7f0a0d27a700 1 --2- 172.21.15.16:0/2604781934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a080701a0 0x7f0a081343d0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:27.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.325+0000 7f0a0d27a700 1 -- 172.21.15.16:0/2604781934 >> 172.21.15.16:0/2604781934 conn(0x7f0a080a2aa0 msgr2=0x7f0a080a1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:27.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.325+0000 7f0a0d27a700 1 -- 172.21.15.16:0/2604781934 shutdown_connections 2023-09-30T22:34:27.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.325+0000 7f0a0d27a700 1 -- 172.21.15.16:0/2604781934 wait complete. 2023-09-30T22:34:27.341 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:34:27.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.454+0000 7ff07ed94700 1 -- 172.21.15.16:0/69979607 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff07809e3f0 con 0x7ff07806b080 2023-09-30T22:34:27.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.455+0000 7ff0757fa700 1 -- 172.21.15.16:0/69979607 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff07809e3f0 con 0x7ff07806b080 2023-09-30T22:34:27.456 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:34:27.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.457+0000 7ff07ed94700 1 -- 172.21.15.16:0/69979607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff07806b080 msgr2=0x7ff07811ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:27.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.457+0000 7ff07ed94700 1 --2- 172.21.15.16:0/69979607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff07806b080 0x7ff07811ca30 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7ff064009b90 tx=0x7ff064004750 comp rx=0 tx=0).stop 2023-09-30T22:34:27.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.457+0000 7ff07ed94700 1 -- 172.21.15.16:0/69979607 shutdown_connections 2023-09-30T22:34:27.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.457+0000 7ff07ed94700 1 --2- 172.21.15.16:0/69979607 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff078131b40 0x7ff07811cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:27.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.457+0000 7ff07ed94700 1 --2- 172.21.15.16:0/69979607 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff07806a770 0x7ff07811c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:27.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.457+0000 7ff07ed94700 1 --2- 172.21.15.16:0/69979607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff07806b080 0x7ff07811ca30 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:27.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.457+0000 7ff07ed94700 1 -- 172.21.15.16:0/69979607 >> 172.21.15.16:0/69979607 conn(0x7ff078066f50 msgr2=0x7ff0780a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:27.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.457+0000 7ff07ed94700 1 -- 172.21.15.16:0/69979607 shutdown_connections 2023-09-30T22:34:27.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.457+0000 7ff07ed94700 1 -- 172.21.15.16:0/69979607 wait complete. 2023-09-30T22:34:27.472 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:34:27.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.480+0000 7fa574165700 1 -- 172.21.15.16:0/3075272554 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa554005590 con 0x7fa56c09d6f0 2023-09-30T22:34:27.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.480+0000 7fa5627fc700 1 -- 172.21.15.16:0/3075272554 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa568017030 con 0x7fa56c09d6f0 2023-09-30T22:34:27.481 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:34:27.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.482+0000 7fa574165700 1 -- 172.21.15.16:0/3075272554 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa56c09d6f0 msgr2=0x7fa56c1a5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:27.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.482+0000 7fa574165700 1 --2- 172.21.15.16:0/3075272554 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa56c09d6f0 0x7fa56c1a5de0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fa56800e9e0 tx=0x7fa56800ecf0 comp rx=0 tx=0).stop 2023-09-30T22:34:27.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.482+0000 7fa574165700 1 -- 172.21.15.16:0/3075272554 shutdown_connections 2023-09-30T22:34:27.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.482+0000 7fa574165700 1 --2- 172.21.15.16:0/3075272554 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa56c09e410 0x7fa56c1a6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:27.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.482+0000 7fa574165700 1 --2- 172.21.15.16:0/3075272554 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa56c09d6f0 0x7fa56c1a5de0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:27.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.482+0000 7fa574165700 1 --2- 172.21.15.16:0/3075272554 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa56c09e000 0x7fa56c1a6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:27.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.482+0000 7fa574165700 1 -- 172.21.15.16:0/3075272554 >> 172.21.15.16:0/3075272554 conn(0x7fa56c066f50 msgr2=0x7fa56c1badf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:27.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.482+0000 7fa574165700 1 -- 172.21.15.16:0/3075272554 shutdown_connections 2023-09-30T22:34:27.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:27.482+0000 7fa574165700 1 -- 172.21.15.16:0/3075272554 wait complete. 2023-09-30T22:34:27.497 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:34:28.294 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:34:28.342 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:34:28.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.442+0000 7f0af5b8b700 1 Processor -- start 2023-09-30T22:34:28.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.442+0000 7f0af5b8b700 1 -- start start 2023-09-30T22:34:28.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.443+0000 7f0af5b8b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0af00a0a30 0x7f0af00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:28.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.443+0000 7f0af5b8b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0af00a1340 0x7f0af0133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:28.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.443+0000 7f0af5b8b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0af009edd0 con 0x7f0af0133fa0 2023-09-30T22:34:28.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.443+0000 7f0af5b8b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0af009e1f0 con 0x7f0af00a1340 2023-09-30T22:34:28.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.443+0000 7f0af5b8b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0af009dc50 con 0x7f0af00a0a30 2023-09-30T22:34:28.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.443+0000 7f0aeeffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0af00a1340 0x7f0af0133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:28.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.443+0000 7f0aeffff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0af0133fa0 0x7f0af013ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51028/0 (socket says 172.21.15.16:51028) 2023-09-30T22:34:28.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.443+0000 7f0aeffff700 1 -- 172.21.15.16:0/1421836545 learned_addr learned my addr 172.21.15.16:0/1421836545 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:34:28.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.443+0000 7f0aef7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0af00a0a30 0x7f0af00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:28.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.443+0000 7f0aeeffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0af00a1340 0x7f0af0133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39938/0 (socket says 172.21.15.16:39938) 2023-09-30T22:34:28.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.443+0000 7f0aeeffd700 1 -- 172.21.15.16:0/1421836545 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0af00a0a30 msgr2=0x7f0af00a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:28.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.443+0000 7f0aeeffd700 1 --2- 172.21.15.16:0/1421836545 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0af00a0a30 0x7f0af00a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:28.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.443+0000 7f0aeeffd700 1 -- 172.21.15.16:0/1421836545 >> v1:172.21.15.16:6789/0 conn(0x7f0af0133fa0 legacy=0x7f0af013ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:28.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.443+0000 7f0aeeffd700 1 -- 172.21.15.16:0/1421836545 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af00a1fe0 con 0x7f0af00a1340 2023-09-30T22:34:28.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.443+0000 7f0aee7fc700 1 -- 172.21.15.16:0/1421836545 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 754985055 0 0) 0x7f0af009edd0 con 0x7f0af0133fa0 2023-09-30T22:34:28.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.443+0000 7f0aef7fe700 1 --2- 172.21.15.16:0/1421836545 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0af00a0a30 0x7f0af00a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:28.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.444+0000 7f0aeeffd700 1 --2- 172.21.15.16:0/1421836545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0af00a1340 0x7f0af0133a60 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8009840 tx=0x7f0ad8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=330493ce16f5db2d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:28.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.444+0000 7f0aee7fc700 1 -- 172.21.15.16:0/1421836545 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad8011830 con 0x7f0af00a1340 2023-09-30T22:34:28.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.444+0000 7f0aee7fc700 1 -- 172.21.15.16:0/1421836545 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ad8011990 con 0x7f0af00a1340 2023-09-30T22:34:28.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.444+0000 7f0aee7fc700 1 -- 172.21.15.16:0/1421836545 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad8011d20 con 0x7f0af00a1340 2023-09-30T22:34:28.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.444+0000 7f0af5b8b700 1 -- 172.21.15.16:0/1421836545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0af00a1340 msgr2=0x7f0af0133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:28.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.444+0000 7f0af5b8b700 1 --2- 172.21.15.16:0/1421836545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0af00a1340 0x7f0af0133a60 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8009840 tx=0x7f0ad8009b50 comp rx=0 tx=0).stop 2023-09-30T22:34:28.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.445+0000 7f0af5b8b700 1 -- 172.21.15.16:0/1421836545 shutdown_connections 2023-09-30T22:34:28.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.445+0000 7f0af5b8b700 1 --2- 172.21.15.16:0/1421836545 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0af00a0a30 0x7f0af00a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:28.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.445+0000 7f0af5b8b700 1 --2- 172.21.15.16:0/1421836545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0af00a1340 0x7f0af0133a60 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:28.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.445+0000 7f0af5b8b700 1 -- 172.21.15.16:0/1421836545 >> 172.21.15.16:0/1421836545 conn(0x7f0af00a6550 msgr2=0x7f0af00a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:28.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.445+0000 7f0af5b8b700 1 -- 172.21.15.16:0/1421836545 shutdown_connections 2023-09-30T22:34:28.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.445+0000 7f0af5b8b700 1 -- 172.21.15.16:0/1421836545 wait complete. 2023-09-30T22:34:28.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.446+0000 7f0af5b8b700 1 Processor -- start 2023-09-30T22:34:28.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.446+0000 7f0af5b8b700 1 -- start start 2023-09-30T22:34:28.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.446+0000 7f0af5b8b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0af00a0a30 0x7f0af0122820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:28.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.446+0000 7f0af5b8b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0af00a1340 0x7f0af0122d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:28.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.446+0000 7f0af5b8b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0af0133fa0 0x7f0af01232a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:28.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.446+0000 7f0af5b8b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af0126f80 con 0x7f0af0133fa0 2023-09-30T22:34:28.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.446+0000 7f0af5b8b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0af0125d60 con 0x7f0af00a1340 2023-09-30T22:34:28.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.446+0000 7f0af5b8b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0af00a3580 con 0x7f0af00a0a30 2023-09-30T22:34:28.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.446+0000 7f0aeeffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0af00a1340 0x7f0af0122d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:28.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.447+0000 7f0aeeffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0af00a1340 0x7f0af0122d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:40862/0 (socket says 172.21.15.16:40862) 2023-09-30T22:34:28.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.447+0000 7f0aeeffd700 1 -- 172.21.15.16:0/819778943 learned_addr learned my addr 172.21.15.16:0/819778943 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:28.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.447+0000 7f0aeffff700 1 --2- 172.21.15.16:0/819778943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0af0133fa0 0x7f0af01232a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:28.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.447+0000 7f0aef7fe700 1 --2- 172.21.15.16:0/819778943 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0af00a0a30 0x7f0af0122820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:28.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.447+0000 7f0aeeffd700 1 -- 172.21.15.16:0/819778943 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0af00a0a30 msgr2=0x7f0af0122820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:28.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.447+0000 7f0aeeffd700 1 --2- 172.21.15.16:0/819778943 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0af00a0a30 0x7f0af0122820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:28.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.447+0000 7f0aeeffd700 1 -- 172.21.15.16:0/819778943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0af0133fa0 msgr2=0x7f0af01232a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:28.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.447+0000 7f0aeeffd700 1 --2- 172.21.15.16:0/819778943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0af0133fa0 0x7f0af01232a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:28.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.447+0000 7f0aeeffd700 1 -- 172.21.15.16:0/819778943 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae0009a70 con 0x7f0af00a1340 2023-09-30T22:34:28.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.447+0000 7f0aeffff700 1 --2- 172.21.15.16:0/819778943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0af0133fa0 0x7f0af01232a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:28.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.447+0000 7f0aeeffd700 1 --2- 172.21.15.16:0/819778943 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0af00a1340 0x7f0af0122d60 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f0ad80090b0 tx=0x7f0ad8004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:28.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.447+0000 7f0ae7fff700 1 -- 172.21.15.16:0/819778943 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad80036a0 con 0x7f0af00a1340 2023-09-30T22:34:28.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.447+0000 7f0ae7fff700 1 -- 172.21.15.16:0/819778943 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ad801f070 con 0x7f0af00a1340 2023-09-30T22:34:28.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.447+0000 7f0af5b8b700 1 -- 172.21.15.16:0/819778943 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ad80094a0 con 0x7f0af00a1340 2023-09-30T22:34:28.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.447+0000 7f0ae7fff700 1 -- 172.21.15.16:0/819778943 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad8003a60 con 0x7f0af00a1340 2023-09-30T22:34:28.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.447+0000 7f0af5b8b700 1 -- 172.21.15.16:0/819778943 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0af0124bf0 con 0x7f0af00a1340 2023-09-30T22:34:28.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.448+0000 7f0af5b8b700 1 -- 172.21.15.16:0/819778943 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0af0125ed0 con 0x7f0af00a1340 2023-09-30T22:34:28.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.449+0000 7f0ae7fff700 1 -- 172.21.15.16:0/819778943 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0ad8003bc0 con 0x7f0af00a1340 2023-09-30T22:34:28.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.449+0000 7f0ae7fff700 1 -- 172.21.15.16:0/819778943 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f0ad8021070 con 0x7f0af00a1340 2023-09-30T22:34:28.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.453+0000 7f0ae7fff700 1 -- 172.21.15.16:0/819778943 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0ad8024d50 con 0x7f0af00a1340 2023-09-30T22:34:28.473 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:34:28.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.484+0000 7f138c071700 1 Processor -- start 2023-09-30T22:34:28.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.484+0000 7f138c071700 1 -- start start 2023-09-30T22:34:28.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.484+0000 7f138c071700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f138409d6f0 0x7f138409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:28.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.484+0000 7f138c071700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f138409e000 0x7f13841339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:28.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.484+0000 7f138c071700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f138409e410 0x7f1384133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:28.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.484+0000 7f138c071700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f138406b460 con 0x7f138409e410 2023-09-30T22:34:28.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.484+0000 7f138c071700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f138409b120 con 0x7f138409d6f0 2023-09-30T22:34:28.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.484+0000 7f138c071700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f138409a7e0 con 0x7f138409e000 2023-09-30T22:34:28.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.485+0000 7f1389e0d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f138409d6f0 0x7f138409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:28.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.485+0000 7f138960c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f138409e000 0x7f13841339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:28.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.485+0000 7f1389e0d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f138409d6f0 0x7f138409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:40878/0 (socket says 172.21.15.16:40878) 2023-09-30T22:34:28.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.485+0000 7f1389e0d700 1 -- 172.21.15.16:0/4057572533 learned_addr learned my addr 172.21.15.16:0/4057572533 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:28.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.485+0000 7f138a60e700 1 --2- 172.21.15.16:0/4057572533 >> v2:172.21.15.16:3300/0 conn(0x7f138409e410 0x7f1384133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:28.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.485+0000 7f1389e0d700 1 -- 172.21.15.16:0/4057572533 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f138409e000 msgr2=0x7f13841339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:28.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.485+0000 7f1389e0d700 1 --2- 172.21.15.16:0/4057572533 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f138409e000 0x7f13841339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:28.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.485+0000 7f1389e0d700 1 -- 172.21.15.16:0/4057572533 >> v2:172.21.15.16:3300/0 conn(0x7f138409e410 msgr2=0x7f1384133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:28.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.485+0000 7f1389e0d700 1 --2- 172.21.15.16:0/4057572533 >> v2:172.21.15.16:3300/0 conn(0x7f138409e410 0x7f1384133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:28.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.485+0000 7f1389e0d700 1 -- 172.21.15.16:0/4057572533 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f138409a500 con 0x7f138409d6f0 2023-09-30T22:34:28.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.485+0000 7f1389e0d700 1 --2- 172.21.15.16:0/4057572533 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f138409d6f0 0x7f138409dac0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f1380009920 tx=0x7f1380009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff81815520a1fb70 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:28.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.485+0000 7f1388e0b700 1 -- 172.21.15.16:0/4057572533 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1380011830 con 0x7f138409d6f0 2023-09-30T22:34:28.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.485+0000 7f1388e0b700 1 -- 172.21.15.16:0/4057572533 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1380011990 con 0x7f138409d6f0 2023-09-30T22:34:28.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.486+0000 7f1388e0b700 1 -- 172.21.15.16:0/4057572533 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1380011d20 con 0x7f138409d6f0 2023-09-30T22:34:28.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.486+0000 7f138c071700 1 -- 172.21.15.16:0/4057572533 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f138409d6f0 msgr2=0x7f138409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:28.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.486+0000 7f138c071700 1 --2- 172.21.15.16:0/4057572533 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f138409d6f0 0x7f138409dac0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f1380009920 tx=0x7f1380009c30 comp rx=0 tx=0).stop 2023-09-30T22:34:28.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.486+0000 7f138c071700 1 -- 172.21.15.16:0/4057572533 shutdown_connections 2023-09-30T22:34:28.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.486+0000 7f138c071700 1 --2- 172.21.15.16:0/4057572533 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f138409d6f0 0x7f138409dac0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:28.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.486+0000 7f138c071700 1 --2- 172.21.15.16:0/4057572533 >> v2:172.21.15.16:3300/0 conn(0x7f138409e410 0x7f1384133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:28.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.486+0000 7f138c071700 1 --2- 172.21.15.16:0/4057572533 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f138409e000 0x7f13841339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:28.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.486+0000 7f138c071700 1 -- 172.21.15.16:0/4057572533 >> 172.21.15.16:0/4057572533 conn(0x7f1384066f50 msgr2=0x7f138409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:28.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.486+0000 7f138c071700 1 -- 172.21.15.16:0/4057572533 shutdown_connections 2023-09-30T22:34:28.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.486+0000 7f138c071700 1 -- 172.21.15.16:0/4057572533 wait complete. 2023-09-30T22:34:28.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.488+0000 7f138c071700 1 Processor -- start 2023-09-30T22:34:28.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.488+0000 7f138c071700 1 -- start start 2023-09-30T22:34:28.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.488+0000 7f138c071700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f138409d6f0 0x7f13841a5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:28.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.488+0000 7f138c071700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f138409e000 0x7f13841a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:28.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.488+0000 7f138c071700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f138409e410 0x7f13841a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:28.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.489+0000 7f138c071700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13841aa250 con 0x7f138409e410 2023-09-30T22:34:28.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.489+0000 7f138c071700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13841a9890 con 0x7f138409e000 2023-09-30T22:34:28.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.489+0000 7f138c071700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f13841a8c60 con 0x7f138409d6f0 2023-09-30T22:34:28.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.489+0000 7f138960c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f138409e000 0x7f13841a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:28.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.489+0000 7f138960c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f138409e000 0x7f13841a62b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:40886/0 (socket says 172.21.15.16:40886) 2023-09-30T22:34:28.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.489+0000 7f138960c700 1 -- 172.21.15.16:0/1751323493 learned_addr learned my addr 172.21.15.16:0/1751323493 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:28.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.489+0000 7f138a60e700 1 --2- 172.21.15.16:0/1751323493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f138409e410 0x7f13841a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:28.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.489+0000 7f1389e0d700 1 --2- 172.21.15.16:0/1751323493 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f138409d6f0 0x7f13841a5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:28.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.489+0000 7f138960c700 1 -- 172.21.15.16:0/1751323493 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f138409d6f0 msgr2=0x7f13841a5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:28.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.489+0000 7f138960c700 1 --2- 172.21.15.16:0/1751323493 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f138409d6f0 0x7f13841a5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:28.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.489+0000 7f138960c700 1 -- 172.21.15.16:0/1751323493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f138409e410 msgr2=0x7f13841a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:28.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.489+0000 7f138960c700 1 --2- 172.21.15.16:0/1751323493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f138409e410 0x7f13841a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:28.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.489+0000 7f138960c700 1 -- 172.21.15.16:0/1751323493 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1380009550 con 0x7f138409e000 2023-09-30T22:34:28.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.489+0000 7f138960c700 1 --2- 172.21.15.16:0/1751323493 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f138409e000 0x7f13841a62b0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f137400e810 tx=0x7f137400eb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:28.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.489+0000 7f137a7fc700 1 -- 172.21.15.16:0/1751323493 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137400c070 con 0x7f138409e000 2023-09-30T22:34:28.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.490+0000 7f137a7fc700 1 -- 172.21.15.16:0/1751323493 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1374013610 con 0x7f138409e000 2023-09-30T22:34:28.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.490+0000 7f138c071700 1 -- 172.21.15.16:0/1751323493 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13841a9020 con 0x7f138409e000 2023-09-30T22:34:28.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.490+0000 7f137a7fc700 1 -- 172.21.15.16:0/1751323493 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1374013980 con 0x7f138409e000 2023-09-30T22:34:28.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.490+0000 7f138c071700 1 -- 172.21.15.16:0/1751323493 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f138409c710 con 0x7f138409e000 2023-09-30T22:34:28.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.490+0000 7f137a7fc700 1 -- 172.21.15.16:0/1751323493 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1374013ae0 con 0x7f138409e000 2023-09-30T22:34:28.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.491+0000 7f137a7fc700 1 -- 172.21.15.16:0/1751323493 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f137403f370 con 0x7f138409e000 2023-09-30T22:34:28.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.491+0000 7f138c071700 1 -- 172.21.15.16:0/1751323493 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1384074ea0 con 0x7f138409e000 2023-09-30T22:34:28.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.496+0000 7f137a7fc700 1 -- 172.21.15.16:0/1751323493 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f13740108e0 con 0x7f138409e000 2023-09-30T22:34:28.498 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:34:28.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.626+0000 7f3961647700 1 Processor -- start 2023-09-30T22:34:28.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.626+0000 7f3961647700 1 -- start start 2023-09-30T22:34:28.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.627+0000 7f3961647700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f395c06a770 0x7f395c09d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:28.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.627+0000 7f3961647700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f395c09dc30 0x7f395c09e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:28.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.627+0000 7f3961647700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f395c09a810 con 0x7f395c13c1e0 2023-09-30T22:34:28.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.627+0000 7f3961647700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f395c06ab40 con 0x7f395c09dc30 2023-09-30T22:34:28.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.627+0000 7f3961647700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f395c09ebc0 con 0x7f395c06a770 2023-09-30T22:34:28.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.627+0000 7f395affd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f395c06a770 0x7f395c09d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:28.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.627+0000 7f395affd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f395c06a770 0x7f395c09d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:40902/0 (socket says 172.21.15.16:40902) 2023-09-30T22:34:28.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.627+0000 7f395affd700 1 -- 172.21.15.16:0/2359548349 learned_addr learned my addr 172.21.15.16:0/2359548349 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:28.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.627+0000 7f395a7fc700 1 --2- 172.21.15.16:0/2359548349 >> v2:172.21.15.16:3300/0 conn(0x7f395c09dc30 0x7f395c09e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:28.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.628+0000 7f395a7fc700 1 -- 172.21.15.16:0/2359548349 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f395c06a770 msgr2=0x7f395c09d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:28.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.628+0000 7f395a7fc700 1 --2- 172.21.15.16:0/2359548349 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f395c06a770 0x7f395c09d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:28.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.628+0000 7f395a7fc700 1 -- 172.21.15.16:0/2359548349 >> v1:172.21.15.16:6789/0 conn(0x7f395c13c1e0 legacy=0x7f395c13e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:28.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.628+0000 7f395a7fc700 1 -- 172.21.15.16:0/2359548349 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f395c09e920 con 0x7f395c09dc30 2023-09-30T22:34:28.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.628+0000 7f395a7fc700 1 --2- 172.21.15.16:0/2359548349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f395c09dc30 0x7f395c09e080 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f3950009870 tx=0x7f3950009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99c46433720dbaa8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:28.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.628+0000 7f3959ffb700 1 -- 172.21.15.16:0/2359548349 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3950011830 con 0x7f395c09dc30 2023-09-30T22:34:28.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.628+0000 7f3959ffb700 1 -- 172.21.15.16:0/2359548349 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3950011990 con 0x7f395c09dc30 2023-09-30T22:34:28.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.628+0000 7f3959ffb700 1 -- 172.21.15.16:0/2359548349 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3950011d20 con 0x7f395c09dc30 2023-09-30T22:34:28.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.629+0000 7f3961647700 1 -- 172.21.15.16:0/2359548349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f395c09dc30 msgr2=0x7f395c09e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:28.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.629+0000 7f3961647700 1 --2- 172.21.15.16:0/2359548349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f395c09dc30 0x7f395c09e080 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f3950009870 tx=0x7f3950009b80 comp rx=0 tx=0).stop 2023-09-30T22:34:28.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.629+0000 7f3961647700 1 -- 172.21.15.16:0/2359548349 shutdown_connections 2023-09-30T22:34:28.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.629+0000 7f3961647700 1 --2- 172.21.15.16:0/2359548349 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f395c06a770 0x7f395c09d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:28.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.629+0000 7f3961647700 1 --2- 172.21.15.16:0/2359548349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f395c09dc30 0x7f395c09e080 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:28.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.629+0000 7f3961647700 1 -- 172.21.15.16:0/2359548349 >> 172.21.15.16:0/2359548349 conn(0x7f395c066f50 msgr2=0x7f395c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:28.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.629+0000 7f3961647700 1 -- 172.21.15.16:0/2359548349 shutdown_connections 2023-09-30T22:34:28.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.629+0000 7f3961647700 1 -- 172.21.15.16:0/2359548349 wait complete. 2023-09-30T22:34:28.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.631+0000 7f3961647700 1 Processor -- start 2023-09-30T22:34:28.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.631+0000 7f3961647700 1 -- start start 2023-09-30T22:34:28.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.631+0000 7f3961647700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f395c06a770 0x7f395c13a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:28.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.631+0000 7f3961647700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f395c09dc30 0x7f395c13a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:28.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.631+0000 7f3961647700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f395c13c1e0 0x7f395c1cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:28.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.631+0000 7f3961647700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f395c129bd0 con 0x7f395c06a770 2023-09-30T22:34:28.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.631+0000 7f3961647700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f395c127f30 con 0x7f395c13c1e0 2023-09-30T22:34:28.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.631+0000 7f3961647700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f395c0a2d20 con 0x7f395c09dc30 2023-09-30T22:34:28.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.631+0000 7f395affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f395c06a770 0x7f395c13a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:28.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.631+0000 7f395affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f395c06a770 0x7f395c13a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39982/0 (socket says 172.21.15.16:39982) 2023-09-30T22:34:28.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.631+0000 7f395affd700 1 -- 172.21.15.16:0/2082400001 learned_addr learned my addr 172.21.15.16:0/2082400001 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:28.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.631+0000 7f395a7fc700 1 --2- 172.21.15.16:0/2082400001 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f395c09dc30 0x7f395c13a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:28.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.632+0000 7f395affd700 1 -- 172.21.15.16:0/2082400001 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f395c09dc30 msgr2=0x7f395c13a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:28.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.632+0000 7f395b7fe700 1 --2- 172.21.15.16:0/2082400001 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f395c13c1e0 0x7f395c1cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:28.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.632+0000 7f395affd700 1 --2- 172.21.15.16:0/2082400001 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f395c09dc30 0x7f395c13a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:28.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.632+0000 7f395affd700 1 -- 172.21.15.16:0/2082400001 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f395c13c1e0 msgr2=0x7f395c1cc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:28.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.632+0000 7f395affd700 1 --2- 172.21.15.16:0/2082400001 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f395c13c1e0 0x7f395c1cc4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:28.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.632+0000 7f395affd700 1 -- 172.21.15.16:0/2082400001 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39500094a0 con 0x7f395c06a770 2023-09-30T22:34:28.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.632+0000 7f395a7fc700 1 --2- 172.21.15.16:0/2082400001 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f395c09dc30 0x7f395c13a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:34:28.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.632+0000 7f395b7fe700 1 --2- 172.21.15.16:0/2082400001 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f395c13c1e0 0x7f395c1cc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:34:28.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.632+0000 7f395affd700 1 --2- 172.21.15.16:0/2082400001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f395c06a770 0x7f395c13a170 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f394c00e910 tx=0x7f394c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:28.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.632+0000 7f39437fe700 1 -- 172.21.15.16:0/2082400001 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394c0096a0 con 0x7f395c06a770 2023-09-30T22:34:28.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.632+0000 7f39437fe700 1 -- 172.21.15.16:0/2082400001 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f394c00c850 con 0x7f395c06a770 2023-09-30T22:34:28.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.632+0000 7f39437fe700 1 -- 172.21.15.16:0/2082400001 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394c00cc00 con 0x7f395c06a770 2023-09-30T22:34:28.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.633+0000 7f3961647700 1 -- 172.21.15.16:0/2082400001 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f395c09f1a0 con 0x7f395c06a770 2023-09-30T22:34:28.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.633+0000 7f3961647700 1 -- 172.21.15.16:0/2082400001 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f395c13abf0 con 0x7f395c06a770 2023-09-30T22:34:28.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.633+0000 7f39437fe700 1 -- 172.21.15.16:0/2082400001 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f394c00cd60 con 0x7f395c06a770 2023-09-30T22:34:28.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.634+0000 7f39437fe700 1 -- 172.21.15.16:0/2082400001 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f394c03fb30 con 0x7f395c06a770 2023-09-30T22:34:28.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.634+0000 7f3961647700 1 -- 172.21.15.16:0/2082400001 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f395c074ea0 con 0x7f395c06a770 2023-09-30T22:34:28.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.639+0000 7f39437fe700 1 -- 172.21.15.16:0/2082400001 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f394c017080 con 0x7f395c06a770 2023-09-30T22:34:28.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.643+0000 7fb8105c3700 1 Processor -- start 2023-09-30T22:34:28.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.644+0000 7fb8105c3700 1 -- start start 2023-09-30T22:34:28.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.644+0000 7fb8105c3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb8080a0a30 0x7fb8080a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:28.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.644+0000 7fb8105c3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb8080a1340 0x7fb808133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:28.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.644+0000 7fb8105c3700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb80809edd0 con 0x7fb808133fa0 2023-09-30T22:34:28.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.644+0000 7fb8105c3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb80809e1f0 con 0x7fb8080a0a30 2023-09-30T22:34:28.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.644+0000 7fb8105c3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb80809dc50 con 0x7fb8080a1340 2023-09-30T22:34:28.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.644+0000 7fb80e35f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb8080a0a30 0x7fb8080a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:28.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.645+0000 7fb80e35f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb8080a0a30 0x7fb8080a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:40914/0 (socket says 172.21.15.16:40914) 2023-09-30T22:34:28.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.645+0000 7fb80e35f700 1 -- 172.21.15.16:0/2023508222 learned_addr learned my addr 172.21.15.16:0/2023508222 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:28.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.645+0000 7fb80db5e700 1 --2- 172.21.15.16:0/2023508222 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb8080a1340 0x7fb808133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:28.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.645+0000 7fb80e35f700 1 -- 172.21.15.16:0/2023508222 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb8080a1340 msgr2=0x7fb808133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:28.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.645+0000 7fb80e35f700 1 --2- 172.21.15.16:0/2023508222 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb8080a1340 0x7fb808133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:28.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.645+0000 7fb80e35f700 1 -- 172.21.15.16:0/2023508222 >> v1:172.21.15.16:6789/0 conn(0x7fb808133fa0 legacy=0x7fb80813ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:28.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.645+0000 7fb80e35f700 1 -- 172.21.15.16:0/2023508222 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8080a1fe0 con 0x7fb8080a0a30 2023-09-30T22:34:28.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.645+0000 7fb80e35f700 1 --2- 172.21.15.16:0/2023508222 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb8080a0a30 0x7fb8080a0e00 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fb804009920 tx=0x7fb804009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acb9967ae5a92eeb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:28.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.645+0000 7fb80d35d700 1 -- 172.21.15.16:0/2023508222 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb804011830 con 0x7fb8080a0a30 2023-09-30T22:34:28.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.645+0000 7fb80d35d700 1 -- 172.21.15.16:0/2023508222 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb804011990 con 0x7fb8080a0a30 2023-09-30T22:34:28.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.645+0000 7fb80d35d700 1 -- 172.21.15.16:0/2023508222 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb804011d20 con 0x7fb8080a0a30 2023-09-30T22:34:28.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.646+0000 7fb8105c3700 1 -- 172.21.15.16:0/2023508222 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb8080a0a30 msgr2=0x7fb8080a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:28.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.646+0000 7fb8105c3700 1 --2- 172.21.15.16:0/2023508222 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb8080a0a30 0x7fb8080a0e00 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fb804009920 tx=0x7fb804009c30 comp rx=0 tx=0).stop 2023-09-30T22:34:28.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.646+0000 7fb8105c3700 1 -- 172.21.15.16:0/2023508222 shutdown_connections 2023-09-30T22:34:28.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.646+0000 7fb8105c3700 1 --2- 172.21.15.16:0/2023508222 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb8080a0a30 0x7fb8080a0e00 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:28.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.646+0000 7fb8105c3700 1 --2- 172.21.15.16:0/2023508222 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb8080a1340 0x7fb808133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:28.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.646+0000 7fb8105c3700 1 -- 172.21.15.16:0/2023508222 >> 172.21.15.16:0/2023508222 conn(0x7fb8080a6550 msgr2=0x7fb8080a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:28.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.646+0000 7fb8105c3700 1 -- 172.21.15.16:0/2023508222 shutdown_connections 2023-09-30T22:34:28.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.646+0000 7fb8105c3700 1 -- 172.21.15.16:0/2023508222 wait complete. 2023-09-30T22:34:28.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.647+0000 7fb8105c3700 1 Processor -- start 2023-09-30T22:34:28.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.647+0000 7fb8105c3700 1 -- start start 2023-09-30T22:34:28.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.648+0000 7fb8105c3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8080a0a30 0x7fb80811e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:28.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.648+0000 7fb8105c3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb8080a1340 0x7fb80811ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:28.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.648+0000 7fb8105c3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb808133fa0 0x7fb80811f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:28.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.648+0000 7fb8105c3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb808120b50 con 0x7fb8080a0a30 2023-09-30T22:34:28.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.648+0000 7fb8105c3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb808122440 con 0x7fb808133fa0 2023-09-30T22:34:28.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.648+0000 7fb8105c3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb8081206b0 con 0x7fb8080a1340 2023-09-30T22:34:28.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.648+0000 7fb80e35f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8080a0a30 0x7fb80811e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:28.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.648+0000 7fb80e35f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8080a0a30 0x7fb80811e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39992/0 (socket says 172.21.15.16:39992) 2023-09-30T22:34:28.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.648+0000 7fb80eb60700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb808133fa0 0x7fb80811f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:28.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.648+0000 7fb80e35f700 1 -- 172.21.15.16:0/1630738029 learned_addr learned my addr 172.21.15.16:0/1630738029 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:28.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.648+0000 7fb80db5e700 1 --2- 172.21.15.16:0/1630738029 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb8080a1340 0x7fb80811ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:28.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.648+0000 7fb80e35f700 1 -- 172.21.15.16:0/1630738029 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb8080a1340 msgr2=0x7fb80811ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:28.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.648+0000 7fb80e35f700 1 --2- 172.21.15.16:0/1630738029 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb8080a1340 0x7fb80811ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:28.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.648+0000 7fb80e35f700 1 -- 172.21.15.16:0/1630738029 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb808133fa0 msgr2=0x7fb80811f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:28.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.648+0000 7fb80e35f700 1 --2- 172.21.15.16:0/1630738029 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb808133fa0 0x7fb80811f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:28.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.648+0000 7fb80e35f700 1 -- 172.21.15.16:0/1630738029 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb804009550 con 0x7fb8080a0a30 2023-09-30T22:34:28.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.649+0000 7fb80e35f700 1 --2- 172.21.15.16:0/1630738029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8080a0a30 0x7fb80811e920 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fb804000c00 tx=0x7fb804004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:28.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.649+0000 7fb7feffd700 1 -- 172.21.15.16:0/1630738029 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb804004450 con 0x7fb8080a0a30 2023-09-30T22:34:28.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.649+0000 7fb7feffd700 1 -- 172.21.15.16:0/1630738029 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8040045b0 con 0x7fb8080a0a30 2023-09-30T22:34:28.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.649+0000 7fb7feffd700 1 -- 172.21.15.16:0/1630738029 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8040038c0 con 0x7fb8080a0a30 2023-09-30T22:34:28.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.649+0000 7fb8105c3700 1 -- 172.21.15.16:0/1630738029 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8080a25e0 con 0x7fb8080a0a30 2023-09-30T22:34:28.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.649+0000 7fb8105c3700 1 -- 172.21.15.16:0/1630738029 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb808122160 con 0x7fb8080a0a30 2023-09-30T22:34:28.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.650+0000 7fb8105c3700 1 -- 172.21.15.16:0/1630738029 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb808074ea0 con 0x7fb8080a0a30 2023-09-30T22:34:28.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.650+0000 7fb7feffd700 1 -- 172.21.15.16:0/1630738029 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb804003a20 con 0x7fb8080a0a30 2023-09-30T22:34:28.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.650+0000 7fb7feffd700 1 -- 172.21.15.16:0/1630738029 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fb804021030 con 0x7fb8080a0a30 2023-09-30T22:34:28.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.655+0000 7fb7feffd700 1 -- 172.21.15.16:0/1630738029 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb80401a430 con 0x7fb8080a0a30 2023-09-30T22:34:28.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.716+0000 7f138c071700 1 -- 172.21.15.16:0/1751323493 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f138409f420 con 0x7f138409e000 2023-09-30T22:34:28.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.717+0000 7f137a7fc700 1 -- 172.21.15.16:0/1751323493 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f137402b090 con 0x7f138409e000 2023-09-30T22:34:28.718 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:34:28.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.719+0000 7f138c071700 1 -- 172.21.15.16:0/1751323493 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f138409e000 msgr2=0x7f13841a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:28.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.719+0000 7f138c071700 1 --2- 172.21.15.16:0/1751323493 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f138409e000 0x7f13841a62b0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f137400e810 tx=0x7f137400eb20 comp rx=0 tx=0).stop 2023-09-30T22:34:28.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.719+0000 7f138c071700 1 -- 172.21.15.16:0/1751323493 shutdown_connections 2023-09-30T22:34:28.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.719+0000 7f138c071700 1 --2- 172.21.15.16:0/1751323493 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f138409e410 0x7f13841a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:28.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.719+0000 7f138c071700 1 --2- 172.21.15.16:0/1751323493 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f138409d6f0 0x7f13841a5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:28.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.719+0000 7f138c071700 1 --2- 172.21.15.16:0/1751323493 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f138409e000 0x7f13841a62b0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:28.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.719+0000 7f138c071700 1 -- 172.21.15.16:0/1751323493 >> 172.21.15.16:0/1751323493 conn(0x7f1384066f50 msgr2=0x7f13841bad60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:28.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.719+0000 7f138c071700 1 -- 172.21.15.16:0/1751323493 shutdown_connections 2023-09-30T22:34:28.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.719+0000 7f138c071700 1 -- 172.21.15.16:0/1751323493 wait complete. 2023-09-30T22:34:28.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.730+0000 7f0af5b8b700 1 -- 172.21.15.16:0/819778943 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0af0126b50 con 0x7f0af00a1340 2023-09-30T22:34:28.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.730+0000 7f0ae7fff700 1 -- 172.21.15.16:0/819778943 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0ad8015020 con 0x7f0af00a1340 2023-09-30T22:34:28.732 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:34:28.735 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:34:28.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.734+0000 7f0af5b8b700 1 -- 172.21.15.16:0/819778943 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0af00a1340 msgr2=0x7f0af0122d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:28.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.734+0000 7f0af5b8b700 1 --2- 172.21.15.16:0/819778943 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0af00a1340 0x7f0af0122d60 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f0ad80090b0 tx=0x7f0ad8004750 comp rx=0 tx=0).stop 2023-09-30T22:34:28.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.734+0000 7f0af5b8b700 1 -- 172.21.15.16:0/819778943 shutdown_connections 2023-09-30T22:34:28.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.734+0000 7f0af5b8b700 1 --2- 172.21.15.16:0/819778943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0af0133fa0 0x7f0af01232a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:28.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.734+0000 7f0af5b8b700 1 --2- 172.21.15.16:0/819778943 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0af00a0a30 0x7f0af0122820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:28.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.734+0000 7f0af5b8b700 1 --2- 172.21.15.16:0/819778943 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0af00a1340 0x7f0af0122d60 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:28.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.734+0000 7f0af5b8b700 1 -- 172.21.15.16:0/819778943 >> 172.21.15.16:0/819778943 conn(0x7f0af00a6550 msgr2=0x7f0af011c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:28.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.734+0000 7f0af5b8b700 1 -- 172.21.15.16:0/819778943 shutdown_connections 2023-09-30T22:34:28.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.734+0000 7f0af5b8b700 1 -- 172.21.15.16:0/819778943 wait complete. 2023-09-30T22:34:28.755 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:34:28.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.874+0000 7fb8105c3700 1 -- 172.21.15.16:0/1630738029 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb8080a28e0 con 0x7fb8080a0a30 2023-09-30T22:34:28.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.874+0000 7fb7feffd700 1 -- 172.21.15.16:0/1630738029 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb804015030 con 0x7fb8080a0a30 2023-09-30T22:34:28.876 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:34:28.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.876+0000 7fb8105c3700 1 -- 172.21.15.16:0/1630738029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8080a0a30 msgr2=0x7fb80811e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:28.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.876+0000 7fb8105c3700 1 --2- 172.21.15.16:0/1630738029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8080a0a30 0x7fb80811e920 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fb804000c00 tx=0x7fb804004750 comp rx=0 tx=0).stop 2023-09-30T22:34:28.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.876+0000 7fb8105c3700 1 -- 172.21.15.16:0/1630738029 shutdown_connections 2023-09-30T22:34:28.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.876+0000 7fb8105c3700 1 --2- 172.21.15.16:0/1630738029 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb808133fa0 0x7fb80811f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:28.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.876+0000 7fb8105c3700 1 --2- 172.21.15.16:0/1630738029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8080a0a30 0x7fb80811e920 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:28.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.876+0000 7fb8105c3700 1 --2- 172.21.15.16:0/1630738029 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb8080a1340 0x7fb80811ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:28.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.876+0000 7fb8105c3700 1 -- 172.21.15.16:0/1630738029 >> 172.21.15.16:0/1630738029 conn(0x7fb8080a6550 msgr2=0x7fb8081b9120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:28.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.876+0000 7fb8105c3700 1 -- 172.21.15.16:0/1630738029 shutdown_connections 2023-09-30T22:34:28.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.876+0000 7fb8105c3700 1 -- 172.21.15.16:0/1630738029 wait complete. 2023-09-30T22:34:28.891 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:34:28.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.899+0000 7f3961647700 1 -- 172.21.15.16:0/2082400001 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f395c129a10 con 0x7f395c06a770 2023-09-30T22:34:28.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.899+0000 7f39437fe700 1 -- 172.21.15.16:0/2082400001 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f395c129a10 con 0x7f395c06a770 2023-09-30T22:34:28.900 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:34:28.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.901+0000 7f3961647700 1 -- 172.21.15.16:0/2082400001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f395c06a770 msgr2=0x7f395c13a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:28.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.901+0000 7f3961647700 1 --2- 172.21.15.16:0/2082400001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f395c06a770 0x7f395c13a170 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f394c00e910 tx=0x7f394c00ecd0 comp rx=0 tx=0).stop 2023-09-30T22:34:28.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.901+0000 7f3961647700 1 -- 172.21.15.16:0/2082400001 shutdown_connections 2023-09-30T22:34:28.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.901+0000 7f3961647700 1 --2- 172.21.15.16:0/2082400001 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f395c13c1e0 0x7f395c1cc4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:28.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.901+0000 7f3961647700 1 --2- 172.21.15.16:0/2082400001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f395c06a770 0x7f395c13a170 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:28.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.901+0000 7f3961647700 1 --2- 172.21.15.16:0/2082400001 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f395c09dc30 0x7f395c13a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:28.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.901+0000 7f3961647700 1 -- 172.21.15.16:0/2082400001 >> 172.21.15.16:0/2082400001 conn(0x7f395c066f50 msgr2=0x7f395c13b400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:28.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.902+0000 7f3961647700 1 -- 172.21.15.16:0/2082400001 shutdown_connections 2023-09-30T22:34:28.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:28.902+0000 7f3961647700 1 -- 172.21.15.16:0/2082400001 wait complete. 2023-09-30T22:34:28.916 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:34:29.736 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:34:29.756 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:34:29.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.884+0000 7f4bad543700 1 Processor -- start 2023-09-30T22:34:29.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.884+0000 7f4bad543700 1 -- start start 2023-09-30T22:34:29.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.885+0000 7f4bad543700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4ba80a09c0 0x7f4ba80a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:29.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.885+0000 7f4bad543700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4ba809d5c0 0x7f4ba80a12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:29.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.885+0000 7f4bad543700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ba80a1810 con 0x7f4ba8133920 2023-09-30T22:34:29.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.885+0000 7f4bad543700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba809ea10 con 0x7f4ba809d5c0 2023-09-30T22:34:29.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.885+0000 7f4bad543700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba809e210 con 0x7f4ba80a09c0 2023-09-30T22:34:29.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.885+0000 7f4ba77fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4ba8133920 0x7f4ba8133d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51062/0 (socket says 172.21.15.16:51062) 2023-09-30T22:34:29.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.885+0000 7f4ba77fe700 1 -- 172.21.15.16:0/2341846818 learned_addr learned my addr 172.21.15.16:0/2341846818 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:34:29.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.885+0000 7f4ba6ffd700 1 --2- 172.21.15.16:0/2341846818 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4ba80a09c0 0x7f4ba80a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:29.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.885+0000 7f4ba67fc700 1 --2- 172.21.15.16:0/2341846818 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4ba809d5c0 0x7f4ba80a12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:29.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.885+0000 7f4ba5ffb700 1 -- 172.21.15.16:0/2341846818 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1843814782 0 0) 0x7f4ba80a1810 con 0x7f4ba8133920 2023-09-30T22:34:29.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.886+0000 7f4ba5ffb700 1 -- 172.21.15.16:0/2341846818 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b940035b0 con 0x7f4ba8133920 2023-09-30T22:34:29.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.886+0000 7f4ba6ffd700 1 -- 172.21.15.16:0/2341846818 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4ba809d5c0 msgr2=0x7f4ba80a12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:29.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.886+0000 7f4ba6ffd700 1 --2- 172.21.15.16:0/2341846818 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4ba809d5c0 0x7f4ba80a12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:29.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.886+0000 7f4ba6ffd700 1 -- 172.21.15.16:0/2341846818 >> v1:172.21.15.16:6789/0 conn(0x7f4ba8133920 legacy=0x7f4ba8133d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:29.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.886+0000 7f4ba6ffd700 1 -- 172.21.15.16:0/2341846818 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ba80a26f0 con 0x7f4ba80a09c0 2023-09-30T22:34:29.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.886+0000 7f4ba67fc700 1 --2- 172.21.15.16:0/2341846818 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4ba809d5c0 0x7f4ba80a12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:29.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.886+0000 7f4ba6ffd700 1 --2- 172.21.15.16:0/2341846818 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4ba80a09c0 0x7f4ba80a0d90 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f4b98009dc0 tx=0x7f4b9800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56f9bec670d218b1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:29.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.886+0000 7f4ba5ffb700 1 -- 172.21.15.16:0/2341846818 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b9800e070 con 0x7f4ba80a09c0 2023-09-30T22:34:29.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.886+0000 7f4ba5ffb700 1 -- 172.21.15.16:0/2341846818 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b9800bb20 con 0x7f4ba80a09c0 2023-09-30T22:34:29.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.886+0000 7f4ba5ffb700 1 -- 172.21.15.16:0/2341846818 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b98012430 con 0x7f4ba80a09c0 2023-09-30T22:34:29.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.887+0000 7f4bad543700 1 -- 172.21.15.16:0/2341846818 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4ba80a09c0 msgr2=0x7f4ba80a0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:29.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.887+0000 7f4bad543700 1 --2- 172.21.15.16:0/2341846818 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4ba80a09c0 0x7f4ba80a0d90 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f4b98009dc0 tx=0x7f4b9800b330 comp rx=0 tx=0).stop 2023-09-30T22:34:29.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.887+0000 7f4bad543700 1 -- 172.21.15.16:0/2341846818 shutdown_connections 2023-09-30T22:34:29.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.887+0000 7f4bad543700 1 --2- 172.21.15.16:0/2341846818 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4ba80a09c0 0x7f4ba80a0d90 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:29.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.887+0000 7f4bad543700 1 --2- 172.21.15.16:0/2341846818 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4ba809d5c0 0x7f4ba80a12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:29.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.887+0000 7f4bad543700 1 -- 172.21.15.16:0/2341846818 >> 172.21.15.16:0/2341846818 conn(0x7f4ba80a2fe0 msgr2=0x7f4ba80a33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:29.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.887+0000 7f4bad543700 1 -- 172.21.15.16:0/2341846818 shutdown_connections 2023-09-30T22:34:29.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.887+0000 7f4bad543700 1 -- 172.21.15.16:0/2341846818 wait complete. 2023-09-30T22:34:29.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.888+0000 7f4bad543700 1 Processor -- start 2023-09-30T22:34:29.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.888+0000 7f4bad543700 1 -- start start 2023-09-30T22:34:29.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.888+0000 7f4bad543700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4ba809d5c0 0x7f4ba81c7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:29.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.888+0000 7f4bad543700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4ba80a09c0 0x7f4ba81c84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:29.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.888+0000 7f4bad543700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ba8133920 0x7f4ba81c89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:29.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.888+0000 7f4bad543700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba8120610 con 0x7f4ba8133920 2023-09-30T22:34:29.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.888+0000 7f4bad543700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba80a4300 con 0x7f4ba809d5c0 2023-09-30T22:34:29.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.888+0000 7f4bad543700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba809cec0 con 0x7f4ba80a09c0 2023-09-30T22:34:29.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.889+0000 7f4ba6ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4ba809d5c0 0x7f4ba81c7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:29.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.889+0000 7f4ba6ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4ba809d5c0 0x7f4ba81c7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:40948/0 (socket says 172.21.15.16:40948) 2023-09-30T22:34:29.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.889+0000 7f4ba67fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4ba80a09c0 0x7f4ba81c84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:29.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.889+0000 7f4ba6ffd700 1 -- 172.21.15.16:0/2484780496 learned_addr learned my addr 172.21.15.16:0/2484780496 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:29.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.889+0000 7f4ba77fe700 1 --2- 172.21.15.16:0/2484780496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ba8133920 0x7f4ba81c89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:29.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.889+0000 7f4ba67fc700 1 -- 172.21.15.16:0/2484780496 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4ba809d5c0 msgr2=0x7f4ba81c7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:29.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.889+0000 7f4ba67fc700 1 --2- 172.21.15.16:0/2484780496 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4ba809d5c0 0x7f4ba81c7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:29.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.889+0000 7f4ba67fc700 1 -- 172.21.15.16:0/2484780496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ba8133920 msgr2=0x7f4ba81c89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:29.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.889+0000 7f4ba67fc700 1 --2- 172.21.15.16:0/2484780496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ba8133920 0x7f4ba81c89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:29.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.889+0000 7f4ba67fc700 1 -- 172.21.15.16:0/2484780496 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b900094a0 con 0x7f4ba80a09c0 2023-09-30T22:34:29.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.889+0000 7f4ba6ffd700 1 --2- 172.21.15.16:0/2484780496 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4ba809d5c0 0x7f4ba81c7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:29.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.889+0000 7f4ba77fe700 1 --2- 172.21.15.16:0/2484780496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ba8133920 0x7f4ba81c89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:34:29.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.889+0000 7f4ba67fc700 1 --2- 172.21.15.16:0/2484780496 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4ba80a09c0 0x7f4ba81c84b0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f4b9000e900 tx=0x7f4b9000ec10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:29.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.889+0000 7f4b9f7fe700 1 -- 172.21.15.16:0/2484780496 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b9000ee50 con 0x7f4ba80a09c0 2023-09-30T22:34:29.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.889+0000 7f4b9f7fe700 1 -- 172.21.15.16:0/2484780496 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b9000c850 con 0x7f4ba80a09c0 2023-09-30T22:34:29.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.889+0000 7f4b9f7fe700 1 -- 172.21.15.16:0/2484780496 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b9000cbe0 con 0x7f4ba80a09c0 2023-09-30T22:34:29.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.889+0000 7f4bad543700 1 -- 172.21.15.16:0/2484780496 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b98009a70 con 0x7f4ba80a09c0 2023-09-30T22:34:29.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.890+0000 7f4bad543700 1 -- 172.21.15.16:0/2484780496 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ba81334b0 con 0x7f4ba80a09c0 2023-09-30T22:34:29.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.890+0000 7f4b9f7fe700 1 -- 172.21.15.16:0/2484780496 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4b9000cd40 con 0x7f4ba80a09c0 2023-09-30T22:34:29.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.890+0000 7f4b9f7fe700 1 -- 172.21.15.16:0/2484780496 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f4b9003fa80 con 0x7f4ba80a09c0 2023-09-30T22:34:29.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.891+0000 7f4bad543700 1 -- 172.21.15.16:0/2484780496 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ba80747f0 con 0x7f4ba80a09c0 2023-09-30T22:34:29.898 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:34:29.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.896+0000 7f4b9f7fe700 1 -- 172.21.15.16:0/2484780496 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4b9001b3b0 con 0x7f4ba80a09c0 2023-09-30T22:34:29.917 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:34:29.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.921+0000 7ffbc775a700 1 Processor -- start 2023-09-30T22:34:29.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.921+0000 7ffbc775a700 1 -- start start 2023-09-30T22:34:29.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.922+0000 7ffbc775a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffbc009d6f0 0x7ffbc009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:29.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.922+0000 7ffbc775a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ffbc009e000 0x7ffbc01339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:29.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.922+0000 7ffbc775a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbc009b120 con 0x7ffbc009e410 2023-09-30T22:34:29.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.922+0000 7ffbc775a700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbc006b460 con 0x7ffbc009e000 2023-09-30T22:34:29.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.922+0000 7ffbc775a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc009a7e0 con 0x7ffbc009d6f0 2023-09-30T22:34:29.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.922+0000 7ffbc5cf7700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ffbc009e410 0x7ffbc0133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51070/0 (socket says 172.21.15.16:51070) 2023-09-30T22:34:29.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.922+0000 7ffbc5cf7700 1 -- 172.21.15.16:0/3958437216 learned_addr learned my addr 172.21.15.16:0/3958437216 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:34:29.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.922+0000 7ffbc4cf5700 1 --2- 172.21.15.16:0/3958437216 >> v2:172.21.15.16:3300/0 conn(0x7ffbc009e000 0x7ffbc01339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:29.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.922+0000 7ffbb7fff700 1 -- 172.21.15.16:0/3958437216 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4045346176 0 0) 0x7ffbc009b120 con 0x7ffbc009e410 2023-09-30T22:34:29.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.923+0000 7ffbb7fff700 1 -- 172.21.15.16:0/3958437216 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffbac0035b0 con 0x7ffbc009e410 2023-09-30T22:34:29.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.923+0000 7ffbc4cf5700 1 -- 172.21.15.16:0/3958437216 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffbc009d6f0 msgr2=0x7ffbc009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:29.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.923+0000 7ffbc4cf5700 1 --2- 172.21.15.16:0/3958437216 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffbc009d6f0 0x7ffbc009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:29.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.923+0000 7ffbc4cf5700 1 -- 172.21.15.16:0/3958437216 >> v1:172.21.15.16:6789/0 conn(0x7ffbc009e410 legacy=0x7ffbc0133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:29.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.923+0000 7ffbc4cf5700 1 -- 172.21.15.16:0/3958437216 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc009a500 con 0x7ffbc009e000 2023-09-30T22:34:29.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.923+0000 7ffbc4cf5700 1 --2- 172.21.15.16:0/3958437216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbc009e000 0x7ffbc01339f0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0009840 tx=0x7ffbb0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=380b647e87b884da server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:29.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.923+0000 7ffbb7fff700 1 -- 172.21.15.16:0/3958437216 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb0011830 con 0x7ffbc009e000 2023-09-30T22:34:29.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.923+0000 7ffbb7fff700 1 -- 172.21.15.16:0/3958437216 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbb0011990 con 0x7ffbc009e000 2023-09-30T22:34:29.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.923+0000 7ffbb7fff700 1 -- 172.21.15.16:0/3958437216 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb0011d20 con 0x7ffbc009e000 2023-09-30T22:34:29.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.924+0000 7ffbc775a700 1 -- 172.21.15.16:0/3958437216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbc009e000 msgr2=0x7ffbc01339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:29.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.924+0000 7ffbc775a700 1 --2- 172.21.15.16:0/3958437216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbc009e000 0x7ffbc01339f0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0009840 tx=0x7ffbb0009b50 comp rx=0 tx=0).stop 2023-09-30T22:34:29.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.924+0000 7ffbc775a700 1 -- 172.21.15.16:0/3958437216 shutdown_connections 2023-09-30T22:34:29.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.924+0000 7ffbc775a700 1 --2- 172.21.15.16:0/3958437216 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffbc009d6f0 0x7ffbc009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:29.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.924+0000 7ffbc775a700 1 --2- 172.21.15.16:0/3958437216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbc009e000 0x7ffbc01339f0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:29.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.924+0000 7ffbc775a700 1 -- 172.21.15.16:0/3958437216 >> 172.21.15.16:0/3958437216 conn(0x7ffbc0066f50 msgr2=0x7ffbc009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:29.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.924+0000 7ffbc775a700 1 -- 172.21.15.16:0/3958437216 shutdown_connections 2023-09-30T22:34:29.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.924+0000 7ffbc775a700 1 -- 172.21.15.16:0/3958437216 wait complete. 2023-09-30T22:34:29.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.925+0000 7ffbc775a700 1 Processor -- start 2023-09-30T22:34:29.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.925+0000 7ffbc775a700 1 -- start start 2023-09-30T22:34:29.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.926+0000 7ffbc775a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffbc009d6f0 0x7ffbc01a2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:29.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.926+0000 7ffbc775a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbc009e000 0x7ffbc01a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:29.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.926+0000 7ffbc775a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffbc009e410 0x7ffbc01a39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:29.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.926+0000 7ffbc775a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc01a6390 con 0x7ffbc009e000 2023-09-30T22:34:29.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.926+0000 7ffbc775a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc00a2f20 con 0x7ffbc009d6f0 2023-09-30T22:34:29.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.926+0000 7ffbc775a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc009c710 con 0x7ffbc009e410 2023-09-30T22:34:29.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.926+0000 7ffbc5cf7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffbc009e410 0x7ffbc01a39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:29.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.926+0000 7ffbc5cf7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffbc009e410 0x7ffbc01a39a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:52796/0 (socket says 172.21.15.16:52796) 2023-09-30T22:34:29.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.926+0000 7ffbc5cf7700 1 -- 172.21.15.16:0/3592378922 learned_addr learned my addr 172.21.15.16:0/3592378922 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:29.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.926+0000 7ffbc4cf5700 1 --2- 172.21.15.16:0/3592378922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbc009e000 0x7ffbc01a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:29.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.926+0000 7ffbc54f6700 1 --2- 172.21.15.16:0/3592378922 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffbc009d6f0 0x7ffbc01a2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:29.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.927+0000 7ffbc5cf7700 1 -- 172.21.15.16:0/3592378922 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffbc009d6f0 msgr2=0x7ffbc01a2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:29.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.927+0000 7ffbc5cf7700 1 --2- 172.21.15.16:0/3592378922 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffbc009d6f0 0x7ffbc01a2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:29.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.927+0000 7ffbc5cf7700 1 -- 172.21.15.16:0/3592378922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbc009e000 msgr2=0x7ffbc01a3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:29.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.927+0000 7ffbc5cf7700 1 --2- 172.21.15.16:0/3592378922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbc009e000 0x7ffbc01a3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:29.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.927+0000 7ffbc5cf7700 1 -- 172.21.15.16:0/3592378922 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb00094a0 con 0x7ffbc009e410 2023-09-30T22:34:29.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.927+0000 7ffbc5cf7700 1 --2- 172.21.15.16:0/3592378922 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffbc009e410 0x7ffbc01a39a0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7ffbb800cb70 tx=0x7ffbb800ce80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:29.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.927+0000 7ffbb5ffb700 1 -- 172.21.15.16:0/3592378922 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb800aa00 con 0x7ffbc009e410 2023-09-30T22:34:29.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.927+0000 7ffbb5ffb700 1 -- 172.21.15.16:0/3592378922 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbb800ab60 con 0x7ffbc009e410 2023-09-30T22:34:29.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.927+0000 7ffbc775a700 1 -- 172.21.15.16:0/3592378922 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbc01a5c70 con 0x7ffbc009e410 2023-09-30T22:34:29.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.927+0000 7ffbb5ffb700 1 -- 172.21.15.16:0/3592378922 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb8012400 con 0x7ffbc009e410 2023-09-30T22:34:29.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.928+0000 7ffbc775a700 1 -- 172.21.15.16:0/3592378922 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbc01a4b30 con 0x7ffbc009e410 2023-09-30T22:34:29.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.929+0000 7ffbc775a700 1 -- 172.21.15.16:0/3592378922 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbc0074ea0 con 0x7ffbc009e410 2023-09-30T22:34:29.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.929+0000 7ffbb5ffb700 1 -- 172.21.15.16:0/3592378922 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ffbb801b3b0 con 0x7ffbc009e410 2023-09-30T22:34:29.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.930+0000 7ffbb5ffb700 1 -- 172.21.15.16:0/3592378922 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7ffbb801a760 con 0x7ffbc009e410 2023-09-30T22:34:29.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:29.935+0000 7ffbb5ffb700 1 -- 172.21.15.16:0/3592378922 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ffbb80198b0 con 0x7ffbc009e410 2023-09-30T22:34:30.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.042+0000 7f9ee492c700 1 Processor -- start 2023-09-30T22:34:30.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.042+0000 7f9ee492c700 1 -- start start 2023-09-30T22:34:30.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.043+0000 7f9ee492c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9ee009d6f0 0x7f9ee009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:30.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.043+0000 7f9ee492c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9ee009e000 0x7f9ee01339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:30.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.043+0000 7f9ee492c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9ee009e410 0x7f9ee0133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:30.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.043+0000 7f9ee492c700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ee006b460 con 0x7f9ee009e410 2023-09-30T22:34:30.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.043+0000 7f9ee492c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee009b120 con 0x7f9ee009d6f0 2023-09-30T22:34:30.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.043+0000 7f9ee492c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee009a7e0 con 0x7f9ee009e000 2023-09-30T22:34:30.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.043+0000 7f9eded9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9ee009e410 0x7f9ee0133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:30.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.043+0000 7f9eded9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9ee009e410 0x7f9ee0133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40010/0 (socket says 172.21.15.16:40010) 2023-09-30T22:34:30.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.043+0000 7f9eded9d700 1 -- 172.21.15.16:0/1854367003 learned_addr learned my addr 172.21.15.16:0/1854367003 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:30.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.044+0000 7f9ede59c700 1 --2- 172.21.15.16:0/1854367003 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9ee009d6f0 0x7f9ee009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:30.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.044+0000 7f9eddd9b700 1 --2- 172.21.15.16:0/1854367003 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9ee009e000 0x7f9ee01339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:30.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.044+0000 7f9eded9d700 1 -- 172.21.15.16:0/1854367003 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9ee009e000 msgr2=0x7f9ee01339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:30.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.044+0000 7f9eded9d700 1 --2- 172.21.15.16:0/1854367003 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9ee009e000 0x7f9ee01339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:30.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.044+0000 7f9eded9d700 1 -- 172.21.15.16:0/1854367003 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9ee009d6f0 msgr2=0x7f9ee009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:30.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.044+0000 7f9eded9d700 1 --2- 172.21.15.16:0/1854367003 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9ee009d6f0 0x7f9ee009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:30.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.044+0000 7f9eded9d700 1 -- 172.21.15.16:0/1854367003 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ee009a500 con 0x7f9ee009e410 2023-09-30T22:34:30.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.044+0000 7f9ede59c700 1 --2- 172.21.15.16:0/1854367003 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9ee009d6f0 0x7f9ee009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:30.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.044+0000 7f9eded9d700 1 --2- 172.21.15.16:0/1854367003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ee009e410 0x7f9ee0133f30 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8009e50 tx=0x7f9ed800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ce0f9da58e564b6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:30.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.044+0000 7f9edd59a700 1 -- 172.21.15.16:0/1854367003 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed800e070 con 0x7f9ee009e410 2023-09-30T22:34:30.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.044+0000 7f9edd59a700 1 -- 172.21.15.16:0/1854367003 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ed800b910 con 0x7f9ee009e410 2023-09-30T22:34:30.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.044+0000 7f9edd59a700 1 -- 172.21.15.16:0/1854367003 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed800bcc0 con 0x7f9ee009e410 2023-09-30T22:34:30.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.045+0000 7f9ee492c700 1 -- 172.21.15.16:0/1854367003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ee009e410 msgr2=0x7f9ee0133f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:30.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.045+0000 7f9ee492c700 1 --2- 172.21.15.16:0/1854367003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ee009e410 0x7f9ee0133f30 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f9ed8009e50 tx=0x7f9ed800b330 comp rx=0 tx=0).stop 2023-09-30T22:34:30.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.046+0000 7f9ee492c700 1 -- 172.21.15.16:0/1854367003 shutdown_connections 2023-09-30T22:34:30.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.046+0000 7f9ee492c700 1 --2- 172.21.15.16:0/1854367003 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9ee009d6f0 0x7f9ee009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:30.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.046+0000 7f9ee492c700 1 --2- 172.21.15.16:0/1854367003 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ee009e410 0x7f9ee0133f30 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:30.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.046+0000 7f9ee492c700 1 --2- 172.21.15.16:0/1854367003 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9ee009e000 0x7f9ee01339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:30.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.046+0000 7f9ee492c700 1 -- 172.21.15.16:0/1854367003 >> 172.21.15.16:0/1854367003 conn(0x7f9ee0066f50 msgr2=0x7f9ee009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:30.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.046+0000 7f9ee492c700 1 -- 172.21.15.16:0/1854367003 shutdown_connections 2023-09-30T22:34:30.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.046+0000 7f9ee492c700 1 -- 172.21.15.16:0/1854367003 wait complete. 2023-09-30T22:34:30.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.047+0000 7f9ee492c700 1 Processor -- start 2023-09-30T22:34:30.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.047+0000 7f9ee492c700 1 -- start start 2023-09-30T22:34:30.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.048+0000 7f9ee492c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ee009d6f0 0x7f9ee01c5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:30.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.048+0000 7f9ee492c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9ee009e000 0x7f9ee01c5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:30.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.048+0000 7f9ee492c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9ee009e410 0x7f9ee01c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:30.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.048+0000 7f9ee492c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee01a9600 con 0x7f9ee009d6f0 2023-09-30T22:34:30.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.048+0000 7f9ee492c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee01a97e0 con 0x7f9ee009e410 2023-09-30T22:34:30.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.048+0000 7f9ee492c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee01c5aa0 con 0x7f9ee009e000 2023-09-30T22:34:30.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.048+0000 7f9eddd9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9ee009e000 0x7f9ee01c5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:30.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.048+0000 7f9eddd9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9ee009e000 0x7f9ee01c5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:52818/0 (socket says 172.21.15.16:52818) 2023-09-30T22:34:30.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.048+0000 7f9eddd9b700 1 -- 172.21.15.16:0/555001053 learned_addr learned my addr 172.21.15.16:0/555001053 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:30.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.048+0000 7f9ede59c700 1 --2- 172.21.15.16:0/555001053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ee009d6f0 0x7f9ee01c5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:30.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.049+0000 7f9eded9d700 1 --2- 172.21.15.16:0/555001053 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9ee009e410 0x7f9ee01c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:30.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.049+0000 7f9eddd9b700 1 -- 172.21.15.16:0/555001053 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9ee009e410 msgr2=0x7f9ee01c8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:30.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.049+0000 7f9eddd9b700 1 --2- 172.21.15.16:0/555001053 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9ee009e410 0x7f9ee01c8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:30.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.049+0000 7f9eddd9b700 1 -- 172.21.15.16:0/555001053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ee009d6f0 msgr2=0x7f9ee01c5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:30.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.049+0000 7f9eddd9b700 1 --2- 172.21.15.16:0/555001053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ee009d6f0 0x7f9ee01c5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:30.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.049+0000 7f9eddd9b700 1 -- 172.21.15.16:0/555001053 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed00094a0 con 0x7f9ee009e000 2023-09-30T22:34:30.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.049+0000 7f9eded9d700 1 --2- 172.21.15.16:0/555001053 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9ee009e410 0x7f9ee01c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:30.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.049+0000 7f9ede59c700 1 --2- 172.21.15.16:0/555001053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ee009d6f0 0x7f9ee01c5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:34:30.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.049+0000 7f9eddd9b700 1 --2- 172.21.15.16:0/555001053 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9ee009e000 0x7f9ee01c5560 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f9ed400e980 tx=0x7f9ed400ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:30.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.049+0000 7f9eceffd700 1 -- 172.21.15.16:0/555001053 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed4009710 con 0x7f9ee009e000 2023-09-30T22:34:30.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.050+0000 7f9eceffd700 1 -- 172.21.15.16:0/555001053 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ed400eed0 con 0x7f9ee009e000 2023-09-30T22:34:30.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.050+0000 7f9eceffd700 1 -- 172.21.15.16:0/555001053 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed400ca70 con 0x7f9ee009e000 2023-09-30T22:34:30.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.050+0000 7f9ee492c700 1 -- 172.21.15.16:0/555001053 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ed8009b30 con 0x7f9ee009e000 2023-09-30T22:34:30.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.050+0000 7f9ee492c700 1 -- 172.21.15.16:0/555001053 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ee01a9240 con 0x7f9ee009e000 2023-09-30T22:34:30.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.051+0000 7f9eceffd700 1 -- 172.21.15.16:0/555001053 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9ed400cbd0 con 0x7f9ee009e000 2023-09-30T22:34:30.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.051+0000 7f9eceffd700 1 -- 172.21.15.16:0/555001053 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f9ed403f710 con 0x7f9ee009e000 2023-09-30T22:34:30.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.053+0000 7f9ee492c700 1 -- 172.21.15.16:0/555001053 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ee0074ea0 con 0x7f9ee009e000 2023-09-30T22:34:30.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.058+0000 7f9eceffd700 1 -- 172.21.15.16:0/555001053 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9ed4017080 con 0x7f9ee009e000 2023-09-30T22:34:30.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.068+0000 7f33f25e7700 1 Processor -- start 2023-09-30T22:34:30.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.068+0000 7f33f25e7700 1 -- start start 2023-09-30T22:34:30.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.068+0000 7f33f25e7700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f33ec09d6f0 0x7f33ec09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:30.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.068+0000 7f33f25e7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f33ec09e000 0x7f33ec130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:30.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.068+0000 7f33f25e7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f33ec09e410 0x7f33ec130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:30.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.068+0000 7f33f25e7700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33ec09f520 con 0x7f33ec09d6f0 2023-09-30T22:34:30.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.068+0000 7f33f25e7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33ec09f130 con 0x7f33ec09e410 2023-09-30T22:34:30.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.068+0000 7f33f25e7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f33ec09e930 con 0x7f33ec09e000 2023-09-30T22:34:30.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.069+0000 7f33eb7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f33ec09e000 0x7f33ec130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:30.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.069+0000 7f33ebfff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f33ec09d6f0 0x7f33ec09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:30.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.069+0000 7f33eb7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f33ec09e000 0x7f33ec130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:52832/0 (socket says 172.21.15.16:52832) 2023-09-30T22:34:30.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.069+0000 7f33eb7fe700 1 -- 172.21.15.16:0/3713014375 learned_addr learned my addr 172.21.15.16:0/3713014375 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:30.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.069+0000 7f33f0b84700 1 --2- 172.21.15.16:0/3713014375 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f33ec09e410 0x7f33ec130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:30.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.069+0000 7f33eb7fe700 1 -- 172.21.15.16:0/3713014375 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f33ec09e410 msgr2=0x7f33ec130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:30.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.069+0000 7f33eb7fe700 1 --2- 172.21.15.16:0/3713014375 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f33ec09e410 0x7f33ec130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:30.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.069+0000 7f33eb7fe700 1 -- 172.21.15.16:0/3713014375 >> v2:172.21.15.16:3300/0 conn(0x7f33ec09d6f0 msgr2=0x7f33ec09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:30.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.069+0000 7f33eb7fe700 1 --2- 172.21.15.16:0/3713014375 >> v2:172.21.15.16:3300/0 conn(0x7f33ec09d6f0 0x7f33ec09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:30.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.069+0000 7f33eb7fe700 1 -- 172.21.15.16:0/3713014375 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33ec09b2d0 con 0x7f33ec09e000 2023-09-30T22:34:30.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.069+0000 7f33eb7fe700 1 --2- 172.21.15.16:0/3713014375 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f33ec09e000 0x7f33ec130660 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f33e0009920 tx=0x7f33e0009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0d453af643613de server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:30.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.070+0000 7f33eaffd700 1 -- 172.21.15.16:0/3713014375 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e0011830 con 0x7f33ec09e000 2023-09-30T22:34:30.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.070+0000 7f33eaffd700 1 -- 172.21.15.16:0/3713014375 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33e0011990 con 0x7f33ec09e000 2023-09-30T22:34:30.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.070+0000 7f33eaffd700 1 -- 172.21.15.16:0/3713014375 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e0011d20 con 0x7f33ec09e000 2023-09-30T22:34:30.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.070+0000 7f33f25e7700 1 -- 172.21.15.16:0/3713014375 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f33ec09e000 msgr2=0x7f33ec130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:30.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.070+0000 7f33f25e7700 1 --2- 172.21.15.16:0/3713014375 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f33ec09e000 0x7f33ec130660 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f33e0009920 tx=0x7f33e0009c30 comp rx=0 tx=0).stop 2023-09-30T22:34:30.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.070+0000 7f33f25e7700 1 -- 172.21.15.16:0/3713014375 shutdown_connections 2023-09-30T22:34:30.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.070+0000 7f33f25e7700 1 --2- 172.21.15.16:0/3713014375 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f33ec09e410 0x7f33ec130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:30.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.070+0000 7f33f25e7700 1 --2- 172.21.15.16:0/3713014375 >> v2:172.21.15.16:3300/0 conn(0x7f33ec09d6f0 0x7f33ec09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:30.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.070+0000 7f33f25e7700 1 --2- 172.21.15.16:0/3713014375 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f33ec09e000 0x7f33ec130660 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:30.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.070+0000 7f33f25e7700 1 -- 172.21.15.16:0/3713014375 >> 172.21.15.16:0/3713014375 conn(0x7f33ec066f50 msgr2=0x7f33ec09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:30.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.071+0000 7f33f25e7700 1 -- 172.21.15.16:0/3713014375 shutdown_connections 2023-09-30T22:34:30.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.071+0000 7f33f25e7700 1 -- 172.21.15.16:0/3713014375 wait complete. 2023-09-30T22:34:30.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.072+0000 7f33f25e7700 1 Processor -- start 2023-09-30T22:34:30.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.072+0000 7f33f25e7700 1 -- start start 2023-09-30T22:34:30.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.073+0000 7f33f25e7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33ec09d6f0 0x7f33ec11f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:30.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.073+0000 7f33f25e7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f33ec09e000 0x7f33ec11f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:30.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.073+0000 7f33f25e7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f33ec09e410 0x7f33ec11fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:30.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.073+0000 7f33f25e7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33ec130450 con 0x7f33ec09d6f0 2023-09-30T22:34:30.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.073+0000 7f33f25e7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33ec121b20 con 0x7f33ec09e410 2023-09-30T22:34:30.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.073+0000 7f33f25e7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f33ec122600 con 0x7f33ec09e000 2023-09-30T22:34:30.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.073+0000 7f33ebfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33ec09d6f0 0x7f33ec11f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:30.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.073+0000 7f33ebfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33ec09d6f0 0x7f33ec11f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40030/0 (socket says 172.21.15.16:40030) 2023-09-30T22:34:30.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.073+0000 7f33ebfff700 1 -- 172.21.15.16:0/2344670654 learned_addr learned my addr 172.21.15.16:0/2344670654 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:30.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.073+0000 7f33f0b84700 1 --2- 172.21.15.16:0/2344670654 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f33ec09e410 0x7f33ec11fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:30.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.073+0000 7f33eb7fe700 1 --2- 172.21.15.16:0/2344670654 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f33ec09e000 0x7f33ec11f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:30.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.073+0000 7f33ebfff700 1 -- 172.21.15.16:0/2344670654 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f33ec09e000 msgr2=0x7f33ec11f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:30.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.073+0000 7f33ebfff700 1 --2- 172.21.15.16:0/2344670654 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f33ec09e000 0x7f33ec11f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:30.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.073+0000 7f33ebfff700 1 -- 172.21.15.16:0/2344670654 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f33ec09e410 msgr2=0x7f33ec11fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:30.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.073+0000 7f33ebfff700 1 --2- 172.21.15.16:0/2344670654 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f33ec09e410 0x7f33ec11fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:30.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.073+0000 7f33ebfff700 1 -- 172.21.15.16:0/2344670654 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e0009550 con 0x7f33ec09d6f0 2023-09-30T22:34:30.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.073+0000 7f33f0b84700 1 --2- 172.21.15.16:0/2344670654 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f33ec09e410 0x7f33ec11fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:34:30.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.074+0000 7f33eb7fe700 1 --2- 172.21.15.16:0/2344670654 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f33ec09e000 0x7f33ec11f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:34:30.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.074+0000 7f33ebfff700 1 --2- 172.21.15.16:0/2344670654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33ec09d6f0 0x7f33ec11f080 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f33dc00e960 tx=0x7f33dc00ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:30.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.074+0000 7f33e8ff9700 1 -- 172.21.15.16:0/2344670654 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33dc009690 con 0x7f33ec09d6f0 2023-09-30T22:34:30.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.074+0000 7f33e8ff9700 1 -- 172.21.15.16:0/2344670654 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33dc00c850 con 0x7f33ec09d6f0 2023-09-30T22:34:30.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.074+0000 7f33e8ff9700 1 -- 172.21.15.16:0/2344670654 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33dc00cbc0 con 0x7f33ec09d6f0 2023-09-30T22:34:30.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.074+0000 7f33f25e7700 1 -- 172.21.15.16:0/2344670654 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33ec1228c0 con 0x7f33ec09d6f0 2023-09-30T22:34:30.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.074+0000 7f33f25e7700 1 -- 172.21.15.16:0/2344670654 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33ec09c700 con 0x7f33ec09d6f0 2023-09-30T22:34:30.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.075+0000 7f33f25e7700 1 -- 172.21.15.16:0/2344670654 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33ec121ee0 con 0x7f33ec09d6f0 2023-09-30T22:34:30.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.076+0000 7f33e8ff9700 1 -- 172.21.15.16:0/2344670654 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f33dc00cd20 con 0x7f33ec09d6f0 2023-09-30T22:34:30.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.076+0000 7f33e8ff9700 1 -- 172.21.15.16:0/2344670654 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f33dc03fb10 con 0x7f33ec09d6f0 2023-09-30T22:34:30.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.085+0000 7f33e8ff9700 1 -- 172.21.15.16:0/2344670654 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f33dc01adc0 con 0x7f33ec09d6f0 2023-09-30T22:34:30.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.115+0000 7f4bad543700 1 -- 172.21.15.16:0/2484780496 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4ba8072980 con 0x7f4ba80a09c0 2023-09-30T22:34:30.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.116+0000 7f4b9f7fe700 1 -- 172.21.15.16:0/2484780496 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4b90011b40 con 0x7f4ba80a09c0 2023-09-30T22:34:30.117 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:34:30.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.119+0000 7f4bad543700 1 -- 172.21.15.16:0/2484780496 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4ba80a09c0 msgr2=0x7f4ba81c84b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:30.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.119+0000 7f4bad543700 1 --2- 172.21.15.16:0/2484780496 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4ba80a09c0 0x7f4ba81c84b0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f4b9000e900 tx=0x7f4b9000ec10 comp rx=0 tx=0).stop 2023-09-30T22:34:30.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.119+0000 7f4bad543700 1 -- 172.21.15.16:0/2484780496 shutdown_connections 2023-09-30T22:34:30.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.119+0000 7f4bad543700 1 --2- 172.21.15.16:0/2484780496 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4ba809d5c0 0x7f4ba81c7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:30.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.119+0000 7f4bad543700 1 --2- 172.21.15.16:0/2484780496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ba8133920 0x7f4ba81c89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:30.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.119+0000 7f4bad543700 1 --2- 172.21.15.16:0/2484780496 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4ba80a09c0 0x7f4ba81c84b0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:30.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.119+0000 7f4bad543700 1 -- 172.21.15.16:0/2484780496 >> 172.21.15.16:0/2484780496 conn(0x7f4ba80a2fe0 msgr2=0x7f4ba809ecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:30.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.119+0000 7f4bad543700 1 -- 172.21.15.16:0/2484780496 shutdown_connections 2023-09-30T22:34:30.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.119+0000 7f4bad543700 1 -- 172.21.15.16:0/2484780496 wait complete. 2023-09-30T22:34:30.140 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:34:30.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.166+0000 7ffbc775a700 1 -- 172.21.15.16:0/3592378922 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ffbc006c360 con 0x7ffbc009e410 2023-09-30T22:34:30.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.167+0000 7ffbb5ffb700 1 -- 172.21.15.16:0/3592378922 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ffbb8015e80 con 0x7ffbc009e410 2023-09-30T22:34:30.168 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:34:30.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.170+0000 7ffbc775a700 1 -- 172.21.15.16:0/3592378922 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffbc009e410 msgr2=0x7ffbc01a39a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:30.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.170+0000 7ffbc775a700 1 --2- 172.21.15.16:0/3592378922 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffbc009e410 0x7ffbc01a39a0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7ffbb800cb70 tx=0x7ffbb800ce80 comp rx=0 tx=0).stop 2023-09-30T22:34:30.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.170+0000 7ffbc775a700 1 -- 172.21.15.16:0/3592378922 shutdown_connections 2023-09-30T22:34:30.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.170+0000 7ffbc775a700 1 --2- 172.21.15.16:0/3592378922 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffbc009d6f0 0x7ffbc01a2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:30.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.170+0000 7ffbc775a700 1 --2- 172.21.15.16:0/3592378922 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffbc009e410 0x7ffbc01a39a0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:30.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.170+0000 7ffbc775a700 1 --2- 172.21.15.16:0/3592378922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffbc009e000 0x7ffbc01a3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:30.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.170+0000 7ffbc775a700 1 -- 172.21.15.16:0/3592378922 >> 172.21.15.16:0/3592378922 conn(0x7ffbc0066f50 msgr2=0x7ffbc01b9d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:30.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.170+0000 7ffbc775a700 1 -- 172.21.15.16:0/3592378922 shutdown_connections 2023-09-30T22:34:30.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.170+0000 7ffbc775a700 1 -- 172.21.15.16:0/3592378922 wait complete. 2023-09-30T22:34:30.191 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:34:30.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.269+0000 7f9ee492c700 1 -- 172.21.15.16:0/555001053 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9ee01a93d0 con 0x7f9ee009e000 2023-09-30T22:34:30.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.270+0000 7f9eceffd700 1 -- 172.21.15.16:0/555001053 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9ed4021020 con 0x7f9ee009e000 2023-09-30T22:34:30.271 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:34:30.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.272+0000 7f9ee492c700 1 -- 172.21.15.16:0/555001053 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9ee009e000 msgr2=0x7f9ee01c5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:30.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.272+0000 7f9ee492c700 1 --2- 172.21.15.16:0/555001053 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9ee009e000 0x7f9ee01c5560 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f9ed400e980 tx=0x7f9ed400ec90 comp rx=0 tx=0).stop 2023-09-30T22:34:30.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.272+0000 7f9ee492c700 1 -- 172.21.15.16:0/555001053 shutdown_connections 2023-09-30T22:34:30.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.272+0000 7f9ee492c700 1 --2- 172.21.15.16:0/555001053 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9ee009e410 0x7f9ee01c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:30.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.272+0000 7f9ee492c700 1 --2- 172.21.15.16:0/555001053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ee009d6f0 0x7f9ee01c5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:30.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.272+0000 7f9ee492c700 1 --2- 172.21.15.16:0/555001053 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9ee009e000 0x7f9ee01c5560 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:30.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.272+0000 7f9ee492c700 1 -- 172.21.15.16:0/555001053 >> 172.21.15.16:0/555001053 conn(0x7f9ee0066f50 msgr2=0x7f9ee01bab50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:30.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.272+0000 7f9ee492c700 1 -- 172.21.15.16:0/555001053 shutdown_connections 2023-09-30T22:34:30.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.272+0000 7f9ee492c700 1 -- 172.21.15.16:0/555001053 wait complete. 2023-09-30T22:34:30.288 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:34:30.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.302+0000 7f33f25e7700 1 -- 172.21.15.16:0/2344670654 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f33ec074ea0 con 0x7f33ec09d6f0 2023-09-30T22:34:30.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.302+0000 7f33e8ff9700 1 -- 172.21.15.16:0/2344670654 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f33dc021070 con 0x7f33ec09d6f0 2023-09-30T22:34:30.304 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:34:30.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.305+0000 7f33f25e7700 1 -- 172.21.15.16:0/2344670654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33ec09d6f0 msgr2=0x7f33ec11f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:30.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.305+0000 7f33f25e7700 1 --2- 172.21.15.16:0/2344670654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33ec09d6f0 0x7f33ec11f080 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f33dc00e960 tx=0x7f33dc00ed20 comp rx=0 tx=0).stop 2023-09-30T22:34:30.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.305+0000 7f33f25e7700 1 -- 172.21.15.16:0/2344670654 shutdown_connections 2023-09-30T22:34:30.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.305+0000 7f33f25e7700 1 --2- 172.21.15.16:0/2344670654 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f33ec09e410 0x7f33ec11fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:30.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.305+0000 7f33f25e7700 1 --2- 172.21.15.16:0/2344670654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33ec09d6f0 0x7f33ec11f080 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:30.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.305+0000 7f33f25e7700 1 --2- 172.21.15.16:0/2344670654 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f33ec09e000 0x7f33ec11f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:30.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.305+0000 7f33f25e7700 1 -- 172.21.15.16:0/2344670654 >> 172.21.15.16:0/2344670654 conn(0x7f33ec066f50 msgr2=0x7f33ec130920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:30.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.305+0000 7f33f25e7700 1 -- 172.21.15.16:0/2344670654 shutdown_connections 2023-09-30T22:34:30.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:30.305+0000 7f33f25e7700 1 -- 172.21.15.16:0/2344670654 wait complete. 2023-09-30T22:34:30.320 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:34:31.141 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:34:31.191 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:34:31.289 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:34:31.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.290+0000 7f1870547700 1 Processor -- start 2023-09-30T22:34:31.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.291+0000 7f1870547700 1 -- start start 2023-09-30T22:34:31.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.291+0000 7f1870547700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f186809d6f0 0x7f186809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:31.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.291+0000 7f1870547700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f186809e000 0x7f18681339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:31.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.291+0000 7f1870547700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f186809e410 0x7f1868133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:31.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.291+0000 7f1870547700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f186806b460 con 0x7f186809e410 2023-09-30T22:34:31.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.291+0000 7f1870547700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f186809b120 con 0x7f186809e000 2023-09-30T22:34:31.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.291+0000 7f1870547700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f186809a7e0 con 0x7f186809d6f0 2023-09-30T22:34:31.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.291+0000 7f186eae4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f186809e410 0x7f1868133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:31.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.292+0000 7f186e2e3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f186809d6f0 0x7f186809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:31.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.292+0000 7f186eae4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f186809e410 0x7f1868133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40042/0 (socket says 172.21.15.16:40042) 2023-09-30T22:34:31.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.292+0000 7f186dae2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f186809e000 0x7f18681339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:31.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.292+0000 7f186eae4700 1 -- 172.21.15.16:0/1297714883 learned_addr learned my addr 172.21.15.16:0/1297714883 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:31.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.292+0000 7f186dae2700 1 -- 172.21.15.16:0/1297714883 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f186809d6f0 msgr2=0x7f186809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:31.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.292+0000 7f186dae2700 1 --2- 172.21.15.16:0/1297714883 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f186809d6f0 0x7f186809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:31.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.292+0000 7f186dae2700 1 -- 172.21.15.16:0/1297714883 >> v2:172.21.15.16:3300/0 conn(0x7f186809e410 msgr2=0x7f1868133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:31.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.292+0000 7f186dae2700 1 --2- 172.21.15.16:0/1297714883 >> v2:172.21.15.16:3300/0 conn(0x7f186809e410 0x7f1868133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:31.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.292+0000 7f186dae2700 1 -- 172.21.15.16:0/1297714883 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f186809a500 con 0x7f186809e000 2023-09-30T22:34:31.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.292+0000 7f186eae4700 1 --2- 172.21.15.16:0/1297714883 >> v2:172.21.15.16:3300/0 conn(0x7f186809e410 0x7f1868133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:31.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.292+0000 7f186dae2700 1 --2- 172.21.15.16:0/1297714883 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f186809e000 0x7f18681339f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f1858009870 tx=0x7f1858009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13a20167138c7290 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:31.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.292+0000 7f186d2e1700 1 -- 172.21.15.16:0/1297714883 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1858011830 con 0x7f186809e000 2023-09-30T22:34:31.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.292+0000 7f186d2e1700 1 -- 172.21.15.16:0/1297714883 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1858011990 con 0x7f186809e000 2023-09-30T22:34:31.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.292+0000 7f186d2e1700 1 -- 172.21.15.16:0/1297714883 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1858011d20 con 0x7f186809e000 2023-09-30T22:34:31.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.293+0000 7f1870547700 1 -- 172.21.15.16:0/1297714883 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f186809e000 msgr2=0x7f18681339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:31.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.293+0000 7f1870547700 1 --2- 172.21.15.16:0/1297714883 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f186809e000 0x7f18681339f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f1858009870 tx=0x7f1858009b80 comp rx=0 tx=0).stop 2023-09-30T22:34:31.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.293+0000 7f1870547700 1 -- 172.21.15.16:0/1297714883 shutdown_connections 2023-09-30T22:34:31.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.293+0000 7f1870547700 1 --2- 172.21.15.16:0/1297714883 >> v2:172.21.15.16:3300/0 conn(0x7f186809e410 0x7f1868133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:31.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.293+0000 7f1870547700 1 --2- 172.21.15.16:0/1297714883 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f186809d6f0 0x7f186809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:31.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.293+0000 7f1870547700 1 --2- 172.21.15.16:0/1297714883 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f186809e000 0x7f18681339f0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:31.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.293+0000 7f1870547700 1 -- 172.21.15.16:0/1297714883 >> 172.21.15.16:0/1297714883 conn(0x7f1868066f50 msgr2=0x7f186809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:31.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.293+0000 7f1870547700 1 -- 172.21.15.16:0/1297714883 shutdown_connections 2023-09-30T22:34:31.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.293+0000 7f1870547700 1 -- 172.21.15.16:0/1297714883 wait complete. 2023-09-30T22:34:31.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.294+0000 7f1870547700 1 Processor -- start 2023-09-30T22:34:31.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.294+0000 7f1870547700 1 -- start start 2023-09-30T22:34:31.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.295+0000 7f1870547700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f186809d6f0 0x7f18681a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:31.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.295+0000 7f1870547700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f186809e000 0x7f18681a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:31.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.295+0000 7f1870547700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f186809e410 0x7f18681c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:31.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.295+0000 7f1870547700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18681a7010 con 0x7f186809d6f0 2023-09-30T22:34:31.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.295+0000 7f1870547700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18681a3c80 con 0x7f186809e410 2023-09-30T22:34:31.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.295+0000 7f1870547700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f186809c710 con 0x7f186809e000 2023-09-30T22:34:31.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.295+0000 7f186dae2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f186809e000 0x7f18681a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:31.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.295+0000 7f186dae2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f186809e000 0x7f18681a4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:52852/0 (socket says 172.21.15.16:52852) 2023-09-30T22:34:31.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.295+0000 7f186dae2700 1 -- 172.21.15.16:0/3191283009 learned_addr learned my addr 172.21.15.16:0/3191283009 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:31.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.295+0000 7f186e2e3700 1 --2- 172.21.15.16:0/3191283009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f186809d6f0 0x7f18681a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:31.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.295+0000 7f186eae4700 1 --2- 172.21.15.16:0/3191283009 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f186809e410 0x7f18681c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:31.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.296+0000 7f186dae2700 1 -- 172.21.15.16:0/3191283009 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f186809e410 msgr2=0x7f18681c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:31.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.296+0000 7f186dae2700 1 --2- 172.21.15.16:0/3191283009 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f186809e410 0x7f18681c8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:31.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.296+0000 7f186dae2700 1 -- 172.21.15.16:0/3191283009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f186809d6f0 msgr2=0x7f18681a4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:31.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.296+0000 7f186dae2700 1 --2- 172.21.15.16:0/3191283009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f186809d6f0 0x7f18681a4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:31.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.296+0000 7f186dae2700 1 -- 172.21.15.16:0/3191283009 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1860009b30 con 0x7f186809e000 2023-09-30T22:34:31.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.296+0000 7f186dae2700 1 --2- 172.21.15.16:0/3191283009 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f186809e000 0x7f18681a4680 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f18580090b0 tx=0x7f1858004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:31.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.296+0000 7f185effd700 1 -- 172.21.15.16:0/3191283009 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1858004510 con 0x7f186809e000 2023-09-30T22:34:31.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.296+0000 7f185effd700 1 -- 172.21.15.16:0/3191283009 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f185801f070 con 0x7f186809e000 2023-09-30T22:34:31.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.296+0000 7f1870547700 1 -- 172.21.15.16:0/3191283009 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18580094a0 con 0x7f186809e000 2023-09-30T22:34:31.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.296+0000 7f185effd700 1 -- 172.21.15.16:0/3191283009 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18580038c0 con 0x7f186809e000 2023-09-30T22:34:31.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.296+0000 7f1870547700 1 -- 172.21.15.16:0/3191283009 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18681a5f50 con 0x7f186809e000 2023-09-30T22:34:31.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.297+0000 7f185effd700 1 -- 172.21.15.16:0/3191283009 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1858003a20 con 0x7f186809e000 2023-09-30T22:34:31.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.297+0000 7f185effd700 1 -- 172.21.15.16:0/3191283009 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f1858021070 con 0x7f186809e000 2023-09-30T22:34:31.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.297+0000 7f1870547700 1 -- 172.21.15.16:0/3191283009 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1868074ea0 con 0x7f186809e000 2023-09-30T22:34:31.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.303+0000 7f185effd700 1 -- 172.21.15.16:0/3191283009 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1858015070 con 0x7f186809e000 2023-09-30T22:34:31.320 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:34:31.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.337+0000 7fe70adb6700 1 Processor -- start 2023-09-30T22:34:31.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.337+0000 7fe70adb6700 1 -- start start 2023-09-30T22:34:31.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.337+0000 7fe70adb6700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe70409d6f0 0x7fe70409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:31.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.337+0000 7fe70adb6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe70409e000 0x7fe704130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:31.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.337+0000 7fe70adb6700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe70409f130 con 0x7fe70409e410 2023-09-30T22:34:31.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.337+0000 7fe70adb6700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe70409f520 con 0x7fe70409d6f0 2023-09-30T22:34:31.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.337+0000 7fe70adb6700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe70409e930 con 0x7fe70409e000 2023-09-30T22:34:31.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.338+0000 7fe703fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe70409e000 0x7fe704130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:31.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.338+0000 7fe703fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe70409e000 0x7fe704130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41006/0 (socket says 172.21.15.16:41006) 2023-09-30T22:34:31.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.338+0000 7fe703fff700 1 -- 172.21.15.16:0/2732452095 learned_addr learned my addr 172.21.15.16:0/2732452095 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:31.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.338+0000 7fe708b52700 1 --2- 172.21.15.16:0/2732452095 >> v2:172.21.15.16:3300/0 conn(0x7fe70409d6f0 0x7fe70409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:31.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.338+0000 7fe703fff700 1 -- 172.21.15.16:0/2732452095 >> v2:172.21.15.16:3300/0 conn(0x7fe70409d6f0 msgr2=0x7fe70409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:31.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.338+0000 7fe703fff700 1 --2- 172.21.15.16:0/2732452095 >> v2:172.21.15.16:3300/0 conn(0x7fe70409d6f0 0x7fe70409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:31.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.338+0000 7fe703fff700 1 -- 172.21.15.16:0/2732452095 >> v1:172.21.15.16:6789/0 conn(0x7fe70409e410 legacy=0x7fe704130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:31.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.338+0000 7fe703fff700 1 -- 172.21.15.16:0/2732452095 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe70409b2d0 con 0x7fe70409e000 2023-09-30T22:34:31.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.338+0000 7fe703fff700 1 --2- 172.21.15.16:0/2732452095 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe70409e000 0x7fe704130660 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fe6f00098f0 tx=0x7fe6f0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=483f0c4ba5858107 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:31.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.339+0000 7fe7037fe700 1 -- 172.21.15.16:0/2732452095 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f000b8a0 con 0x7fe70409e000 2023-09-30T22:34:31.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.339+0000 7fe7037fe700 1 -- 172.21.15.16:0/2732452095 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6f000ba00 con 0x7fe70409e000 2023-09-30T22:34:31.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.339+0000 7fe7037fe700 1 -- 172.21.15.16:0/2732452095 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f000bd90 con 0x7fe70409e000 2023-09-30T22:34:31.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.339+0000 7fe70adb6700 1 -- 172.21.15.16:0/2732452095 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe70409e000 msgr2=0x7fe704130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:31.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.339+0000 7fe70adb6700 1 --2- 172.21.15.16:0/2732452095 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe70409e000 0x7fe704130660 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fe6f00098f0 tx=0x7fe6f0009c00 comp rx=0 tx=0).stop 2023-09-30T22:34:31.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.339+0000 7fe70adb6700 1 -- 172.21.15.16:0/2732452095 shutdown_connections 2023-09-30T22:34:31.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.339+0000 7fe70adb6700 1 --2- 172.21.15.16:0/2732452095 >> v2:172.21.15.16:3300/0 conn(0x7fe70409d6f0 0x7fe70409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:31.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.339+0000 7fe70adb6700 1 --2- 172.21.15.16:0/2732452095 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe70409e000 0x7fe704130660 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:31.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.339+0000 7fe70adb6700 1 -- 172.21.15.16:0/2732452095 >> 172.21.15.16:0/2732452095 conn(0x7fe704066f50 msgr2=0x7fe70409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:31.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.339+0000 7fe70adb6700 1 -- 172.21.15.16:0/2732452095 shutdown_connections 2023-09-30T22:34:31.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.339+0000 7fe70adb6700 1 -- 172.21.15.16:0/2732452095 wait complete. 2023-09-30T22:34:31.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.340+0000 7fe70adb6700 1 Processor -- start 2023-09-30T22:34:31.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.341+0000 7fe70adb6700 1 -- start start 2023-09-30T22:34:31.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.341+0000 7fe70adb6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe70409d6f0 0x7fe7041388f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:31.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.341+0000 7fe70adb6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe70409e000 0x7fe704138e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:31.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.341+0000 7fe70adb6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe70409e410 0x7fe70412fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:31.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.341+0000 7fe70adb6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe704139e30 con 0x7fe70409e410 2023-09-30T22:34:31.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.341+0000 7fe70adb6700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe70409c770 con 0x7fe70409e000 2023-09-30T22:34:31.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.341+0000 7fe70adb6700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe70413a690 con 0x7fe70409d6f0 2023-09-30T22:34:31.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.341+0000 7fe709353700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe70409e410 0x7fe70412fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:31.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.341+0000 7fe709353700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe70409e410 0x7fe70412fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40084/0 (socket says 172.21.15.16:40084) 2023-09-30T22:34:31.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.341+0000 7fe709353700 1 -- 172.21.15.16:0/2820311743 learned_addr learned my addr 172.21.15.16:0/2820311743 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:31.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.342+0000 7fe703fff700 1 --2- 172.21.15.16:0/2820311743 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe70409e000 0x7fe704138e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:31.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.342+0000 7fe709353700 1 -- 172.21.15.16:0/2820311743 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe70409d6f0 msgr2=0x7fe7041388f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:31.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.342+0000 7fe709353700 1 --2- 172.21.15.16:0/2820311743 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe70409d6f0 0x7fe7041388f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:31.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.342+0000 7fe709353700 1 -- 172.21.15.16:0/2820311743 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe70409e000 msgr2=0x7fe704138e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:31.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.342+0000 7fe709353700 1 --2- 172.21.15.16:0/2820311743 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe70409e000 0x7fe704138e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:31.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.342+0000 7fe709353700 1 -- 172.21.15.16:0/2820311743 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f0009550 con 0x7fe70409e410 2023-09-30T22:34:31.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.342+0000 7fe709353700 1 --2- 172.21.15.16:0/2820311743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe70409e410 0x7fe70412fb80 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc00cc80 tx=0x7fe6fc00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:31.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.342+0000 7fe7017fa700 1 -- 172.21.15.16:0/2820311743 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc003c00 con 0x7fe70409e410 2023-09-30T22:34:31.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.342+0000 7fe7017fa700 1 -- 172.21.15.16:0/2820311743 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6fc003d60 con 0x7fe70409e410 2023-09-30T22:34:31.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.342+0000 7fe70adb6700 1 -- 172.21.15.16:0/2820311743 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe704131880 con 0x7fe70409e410 2023-09-30T22:34:31.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.342+0000 7fe7017fa700 1 -- 172.21.15.16:0/2820311743 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc00aaf0 con 0x7fe70409e410 2023-09-30T22:34:31.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.342+0000 7fe70adb6700 1 -- 172.21.15.16:0/2820311743 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe70413b7a0 con 0x7fe70409e410 2023-09-30T22:34:31.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.344+0000 7fe70adb6700 1 -- 172.21.15.16:0/2820311743 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe704074ea0 con 0x7fe70409e410 2023-09-30T22:34:31.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.345+0000 7fe7017fa700 1 -- 172.21.15.16:0/2820311743 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe6fc00ac50 con 0x7fe70409e410 2023-09-30T22:34:31.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.345+0000 7fe7017fa700 1 -- 172.21.15.16:0/2820311743 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fe6fc0401e0 con 0x7fe70409e410 2023-09-30T22:34:31.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.349+0000 7fe7017fa700 1 -- 172.21.15.16:0/2820311743 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe6fc0050d0 con 0x7fe70409e410 2023-09-30T22:34:31.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.441+0000 7f116d46b700 1 Processor -- start 2023-09-30T22:34:31.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.441+0000 7f116d46b700 1 -- start start 2023-09-30T22:34:31.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.442+0000 7f116d46b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f11680a0a30 0x7f11680a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:31.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.442+0000 7f116d46b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f116809d630 0x7f11680a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:31.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.442+0000 7f116d46b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11680a2a70 con 0x7f1168133930 2023-09-30T22:34:31.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.442+0000 7f116d46b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11680a01d0 con 0x7f116809d630 2023-09-30T22:34:31.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.442+0000 7f116d46b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f116809e2b0 con 0x7f11680a0a30 2023-09-30T22:34:31.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.442+0000 7f1166ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f11680a0a30 0x7f11680a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:31.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.442+0000 7f11667fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f116809d630 0x7f11680a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:31.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.442+0000 7f11667fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f116809d630 0x7f11680a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40092/0 (socket says 172.21.15.16:40092) 2023-09-30T22:34:31.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.442+0000 7f1166ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f11680a0a30 0x7f11680a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41028/0 (socket says 172.21.15.16:41028) 2023-09-30T22:34:31.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.442+0000 7f11667fc700 1 -- 172.21.15.16:0/3715486854 learned_addr learned my addr 172.21.15.16:0/3715486854 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:31.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.443+0000 7f1166ffd700 1 -- 172.21.15.16:0/3715486854 >> v2:172.21.15.16:3300/0 conn(0x7f116809d630 msgr2=0x7f11680a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:31.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.443+0000 7f1166ffd700 1 --2- 172.21.15.16:0/3715486854 >> v2:172.21.15.16:3300/0 conn(0x7f116809d630 0x7f11680a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:31.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.443+0000 7f1166ffd700 1 -- 172.21.15.16:0/3715486854 >> v1:172.21.15.16:6789/0 conn(0x7f1168133930 legacy=0x7f1168133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:31.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.443+0000 7f1166ffd700 1 -- 172.21.15.16:0/3715486854 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f116809dc50 con 0x7f11680a0a30 2023-09-30T22:34:31.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.443+0000 7f11667fc700 1 --2- 172.21.15.16:0/3715486854 >> v2:172.21.15.16:3300/0 conn(0x7f116809d630 0x7f11680a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:31.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.443+0000 7f1166ffd700 1 --2- 172.21.15.16:0/3715486854 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f11680a0a30 0x7f11680a0e00 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f1158009840 tx=0x7f1158009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8541dd9639702397 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:31.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.443+0000 7f1165ffb700 1 -- 172.21.15.16:0/3715486854 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1158011830 con 0x7f11680a0a30 2023-09-30T22:34:31.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.443+0000 7f1165ffb700 1 -- 172.21.15.16:0/3715486854 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1158011990 con 0x7f11680a0a30 2023-09-30T22:34:31.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.443+0000 7f1165ffb700 1 -- 172.21.15.16:0/3715486854 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1158011d20 con 0x7f11680a0a30 2023-09-30T22:34:31.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.444+0000 7f116d46b700 1 -- 172.21.15.16:0/3715486854 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f11680a0a30 msgr2=0x7f11680a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:31.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.444+0000 7f116d46b700 1 --2- 172.21.15.16:0/3715486854 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f11680a0a30 0x7f11680a0e00 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f1158009840 tx=0x7f1158009b50 comp rx=0 tx=0).stop 2023-09-30T22:34:31.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.444+0000 7f116d46b700 1 -- 172.21.15.16:0/3715486854 shutdown_connections 2023-09-30T22:34:31.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.444+0000 7f116d46b700 1 --2- 172.21.15.16:0/3715486854 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f11680a0a30 0x7f11680a0e00 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:31.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.444+0000 7f116d46b700 1 --2- 172.21.15.16:0/3715486854 >> v2:172.21.15.16:3300/0 conn(0x7f116809d630 0x7f11680a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:31.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.444+0000 7f116d46b700 1 -- 172.21.15.16:0/3715486854 >> 172.21.15.16:0/3715486854 conn(0x7f11680a3050 msgr2=0x7f11680a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:31.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.444+0000 7f116d46b700 1 -- 172.21.15.16:0/3715486854 shutdown_connections 2023-09-30T22:34:31.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.444+0000 7f116d46b700 1 -- 172.21.15.16:0/3715486854 wait complete. 2023-09-30T22:34:31.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.446+0000 7f116d46b700 1 Processor -- start 2023-09-30T22:34:31.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.446+0000 7f116d46b700 1 -- start start 2023-09-30T22:34:31.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.447+0000 7f116d46b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f116809d630 0x7f116811efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:31.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.447+0000 7f116d46b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f11680a0a30 0x7f116811f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:31.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.447+0000 7f116d46b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1168133930 0x7f116811fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:31.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.447+0000 7f116d46b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11680a50b0 con 0x7f116809d630 2023-09-30T22:34:31.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.447+0000 7f116d46b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1168120cb0 con 0x7f1168133930 2023-09-30T22:34:31.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.447+0000 7f116d46b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f11680a0410 con 0x7f11680a0a30 2023-09-30T22:34:31.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.447+0000 7f1166ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f116809d630 0x7f116811efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:31.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.447+0000 7f1166ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f116809d630 0x7f116811efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40098/0 (socket says 172.21.15.16:40098) 2023-09-30T22:34:31.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.447+0000 7f1166ffd700 1 -- 172.21.15.16:0/1968962768 learned_addr learned my addr 172.21.15.16:0/1968962768 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:31.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.447+0000 7f11667fc700 1 --2- 172.21.15.16:0/1968962768 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f11680a0a30 0x7f116811f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:31.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.447+0000 7f11677fe700 1 --2- 172.21.15.16:0/1968962768 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1168133930 0x7f116811fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:31.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.447+0000 7f1166ffd700 1 -- 172.21.15.16:0/1968962768 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f11680a0a30 msgr2=0x7f116811f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:31.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.447+0000 7f1166ffd700 1 --2- 172.21.15.16:0/1968962768 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f11680a0a30 0x7f116811f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:31.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.447+0000 7f1166ffd700 1 -- 172.21.15.16:0/1968962768 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1168133930 msgr2=0x7f116811fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:31.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.447+0000 7f1166ffd700 1 --2- 172.21.15.16:0/1968962768 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1168133930 0x7f116811fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:31.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.447+0000 7f1166ffd700 1 -- 172.21.15.16:0/1968962768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f115c009550 con 0x7f116809d630 2023-09-30T22:34:31.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.447+0000 7f11677fe700 1 --2- 172.21.15.16:0/1968962768 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1168133930 0x7f116811fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:31.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.448+0000 7f1166ffd700 1 --2- 172.21.15.16:0/1968962768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f116809d630 0x7f116811efe0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f1158009fd0 tx=0x7f1158004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:31.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.448+0000 7f114f7fe700 1 -- 172.21.15.16:0/1968962768 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1158004400 con 0x7f116809d630 2023-09-30T22:34:31.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.448+0000 7f114f7fe700 1 -- 172.21.15.16:0/1968962768 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f115801f070 con 0x7f116809d630 2023-09-30T22:34:31.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.448+0000 7f114f7fe700 1 -- 172.21.15.16:0/1968962768 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11580038e0 con 0x7f116809d630 2023-09-30T22:34:31.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.448+0000 7f116d46b700 1 -- 172.21.15.16:0/1968962768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11580094a0 con 0x7f116809d630 2023-09-30T22:34:31.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.448+0000 7f116d46b700 1 -- 172.21.15.16:0/1968962768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11681219a0 con 0x7f116809d630 2023-09-30T22:34:31.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.449+0000 7f116d46b700 1 -- 172.21.15.16:0/1968962768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f116809c900 con 0x7f116809d630 2023-09-30T22:34:31.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.450+0000 7f114f7fe700 1 -- 172.21.15.16:0/1968962768 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1158021030 con 0x7f116809d630 2023-09-30T22:34:31.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.453+0000 7f114f7fe700 1 -- 172.21.15.16:0/1968962768 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f1158004580 con 0x7f116809d630 2023-09-30T22:34:31.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.458+0000 7f114f7fe700 1 -- 172.21.15.16:0/1968962768 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f11580239c0 con 0x7f116809d630 2023-09-30T22:34:31.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.484+0000 7f72bfd3d700 1 Processor -- start 2023-09-30T22:34:31.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.484+0000 7f72bfd3d700 1 -- start start 2023-09-30T22:34:31.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.485+0000 7f72bfd3d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72b809e930 0x7f72b809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:31.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.485+0000 7f72bfd3d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72b809f240 0x7f72b81317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:31.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.485+0000 7f72bfd3d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72b8070940 con 0x7f72b809f650 2023-09-30T22:34:31.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.485+0000 7f72bfd3d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72b806fd60 con 0x7f72b809f240 2023-09-30T22:34:31.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.485+0000 7f72bfd3d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f72b806fa80 con 0x7f72b809e930 2023-09-30T22:34:31.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.485+0000 7f72be2da700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f72b809f650 0x7f72b8131d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51094/0 (socket says 172.21.15.16:51094) 2023-09-30T22:34:31.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.485+0000 7f72bdad9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72b809e930 0x7f72b809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:31.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.485+0000 7f72be2da700 1 -- 172.21.15.16:0/1909402016 learned_addr learned my addr 172.21.15.16:0/1909402016 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:34:31.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.485+0000 7f72bd2d8700 1 --2- 172.21.15.16:0/1909402016 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72b809f240 0x7f72b81317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:31.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.485+0000 7f72bcad7700 1 -- 172.21.15.16:0/1909402016 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1329131819 0 0) 0x7f72b8070940 con 0x7f72b809f650 2023-09-30T22:34:31.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.485+0000 7f72bcad7700 1 -- 172.21.15.16:0/1909402016 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72a40035b0 con 0x7f72b809f650 2023-09-30T22:34:31.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.486+0000 7f72bdad9700 1 -- 172.21.15.16:0/1909402016 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72b809f240 msgr2=0x7f72b81317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:31.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.486+0000 7f72bdad9700 1 --2- 172.21.15.16:0/1909402016 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72b809f240 0x7f72b81317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:31.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.486+0000 7f72bdad9700 1 -- 172.21.15.16:0/1909402016 >> v1:172.21.15.16:6789/0 conn(0x7f72b809f650 legacy=0x7f72b8131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:31.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.486+0000 7f72bdad9700 1 -- 172.21.15.16:0/1909402016 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b809fb60 con 0x7f72b809e930 2023-09-30T22:34:31.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.486+0000 7f72bdad9700 1 --2- 172.21.15.16:0/1909402016 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72b809e930 0x7f72b809ed00 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f72b4009870 tx=0x7f72b4009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fa708121f6c3c28 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:31.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.486+0000 7f72bcad7700 1 -- 172.21.15.16:0/1909402016 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b4011830 con 0x7f72b809e930 2023-09-30T22:34:31.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.486+0000 7f72bcad7700 1 -- 172.21.15.16:0/1909402016 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72b4011990 con 0x7f72b809e930 2023-09-30T22:34:31.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.486+0000 7f72bcad7700 1 -- 172.21.15.16:0/1909402016 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b4011d20 con 0x7f72b809e930 2023-09-30T22:34:31.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.487+0000 7f72bfd3d700 1 -- 172.21.15.16:0/1909402016 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72b809e930 msgr2=0x7f72b809ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:31.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.487+0000 7f72bfd3d700 1 --2- 172.21.15.16:0/1909402016 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72b809e930 0x7f72b809ed00 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f72b4009870 tx=0x7f72b4009b80 comp rx=0 tx=0).stop 2023-09-30T22:34:31.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.487+0000 7f72bfd3d700 1 -- 172.21.15.16:0/1909402016 shutdown_connections 2023-09-30T22:34:31.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.487+0000 7f72bfd3d700 1 --2- 172.21.15.16:0/1909402016 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72b809e930 0x7f72b809ed00 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:31.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.487+0000 7f72bfd3d700 1 --2- 172.21.15.16:0/1909402016 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72b809f240 0x7f72b81317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:31.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.487+0000 7f72bfd3d700 1 -- 172.21.15.16:0/1909402016 >> 172.21.15.16:0/1909402016 conn(0x7f72b80a4210 msgr2=0x7f72b80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:31.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.487+0000 7f72bfd3d700 1 -- 172.21.15.16:0/1909402016 shutdown_connections 2023-09-30T22:34:31.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.487+0000 7f72bfd3d700 1 -- 172.21.15.16:0/1909402016 wait complete. 2023-09-30T22:34:31.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.489+0000 7f72bfd3d700 1 Processor -- start 2023-09-30T22:34:31.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.489+0000 7f72bfd3d700 1 -- start start 2023-09-30T22:34:31.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.489+0000 7f72bfd3d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72b809e930 0x7f72b81c6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:31.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.489+0000 7f72bfd3d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72b809f240 0x7f72b81c6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:31.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.489+0000 7f72bfd3d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72b809f650 0x7f72b81c27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:31.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.489+0000 7f72bfd3d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72b81a7730 con 0x7f72b809e930 2023-09-30T22:34:31.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.489+0000 7f72bfd3d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72b81a4ef0 con 0x7f72b809f240 2023-09-30T22:34:31.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.489+0000 7f72bfd3d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f72b80a32d0 con 0x7f72b809f650 2023-09-30T22:34:31.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.490+0000 7f72bdad9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72b809e930 0x7f72b81c6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:31.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.490+0000 7f72bd2d8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72b809f240 0x7f72b81c6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:31.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.490+0000 7f72bdad9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72b809e930 0x7f72b81c6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40100/0 (socket says 172.21.15.16:40100) 2023-09-30T22:34:31.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.490+0000 7f72bdad9700 1 -- 172.21.15.16:0/2970531394 learned_addr learned my addr 172.21.15.16:0/2970531394 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:31.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.490+0000 7f72be2da700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72b809f650 0x7f72b81c27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:31.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.490+0000 7f72bd2d8700 1 -- 172.21.15.16:0/2970531394 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72b809f650 msgr2=0x7f72b81c27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:31.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.490+0000 7f72bd2d8700 1 --2- 172.21.15.16:0/2970531394 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72b809f650 0x7f72b81c27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:31.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.490+0000 7f72bd2d8700 1 -- 172.21.15.16:0/2970531394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72b809e930 msgr2=0x7f72b81c6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:31.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.490+0000 7f72bd2d8700 1 --2- 172.21.15.16:0/2970531394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72b809e930 0x7f72b81c6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:31.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.490+0000 7f72bd2d8700 1 -- 172.21.15.16:0/2970531394 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b40094a0 con 0x7f72b809f240 2023-09-30T22:34:31.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.490+0000 7f72bdad9700 1 --2- 172.21.15.16:0/2970531394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72b809e930 0x7f72b81c6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:31.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.490+0000 7f72be2da700 1 --2- 172.21.15.16:0/2970531394 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72b809f650 0x7f72b81c27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:34:31.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.490+0000 7f72bd2d8700 1 --2- 172.21.15.16:0/2970531394 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72b809f240 0x7f72b81c6750 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f72a800c420 tx=0x7f72a800c730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:31.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.490+0000 7f72ae7fc700 1 -- 172.21.15.16:0/2970531394 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a8004020 con 0x7f72b809f240 2023-09-30T22:34:31.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.491+0000 7f72ae7fc700 1 -- 172.21.15.16:0/2970531394 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72a8004180 con 0x7f72b809f240 2023-09-30T22:34:31.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.491+0000 7f72bfd3d700 1 -- 172.21.15.16:0/2970531394 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72b80a0190 con 0x7f72b809f240 2023-09-30T22:34:31.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.491+0000 7f72ae7fc700 1 -- 172.21.15.16:0/2970531394 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a8004510 con 0x7f72b809f240 2023-09-30T22:34:31.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.491+0000 7f72bfd3d700 1 -- 172.21.15.16:0/2970531394 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72b81c6c90 con 0x7f72b809f240 2023-09-30T22:34:31.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.492+0000 7f72ae7fc700 1 -- 172.21.15.16:0/2970531394 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f72a80036a0 con 0x7f72b809f240 2023-09-30T22:34:31.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.492+0000 7f72bfd3d700 1 -- 172.21.15.16:0/2970531394 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72b81a7190 con 0x7f72b809f240 2023-09-30T22:34:31.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.492+0000 7f72ae7fc700 1 -- 172.21.15.16:0/2970531394 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f72a8047410 con 0x7f72b809f240 2023-09-30T22:34:31.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.497+0000 7f72ae7fc700 1 -- 172.21.15.16:0/2970531394 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f72a8003950 con 0x7f72b809f240 2023-09-30T22:34:31.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.544+0000 7f1870547700 1 -- 172.21.15.16:0/3191283009 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f186806c360 con 0x7f186809e000 2023-09-30T22:34:31.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.544+0000 7f185effd700 1 -- 172.21.15.16:0/3191283009 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1858024380 con 0x7f186809e000 2023-09-30T22:34:31.546 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:34:31.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.546+0000 7f1870547700 1 -- 172.21.15.16:0/3191283009 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f186809e000 msgr2=0x7f18681a4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:31.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.546+0000 7f1870547700 1 --2- 172.21.15.16:0/3191283009 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f186809e000 0x7f18681a4680 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f18580090b0 tx=0x7f1858004750 comp rx=0 tx=0).stop 2023-09-30T22:34:31.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.547+0000 7f1870547700 1 -- 172.21.15.16:0/3191283009 shutdown_connections 2023-09-30T22:34:31.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.547+0000 7f1870547700 1 --2- 172.21.15.16:0/3191283009 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f186809e410 0x7f18681c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:31.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.547+0000 7f1870547700 1 --2- 172.21.15.16:0/3191283009 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f186809d6f0 0x7f18681a4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:31.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.547+0000 7f1870547700 1 --2- 172.21.15.16:0/3191283009 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f186809e000 0x7f18681a4680 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:31.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.547+0000 7f1870547700 1 -- 172.21.15.16:0/3191283009 >> 172.21.15.16:0/3191283009 conn(0x7f1868066f50 msgr2=0x7f18681bbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:31.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.547+0000 7f1870547700 1 -- 172.21.15.16:0/3191283009 shutdown_connections 2023-09-30T22:34:31.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.547+0000 7f1870547700 1 -- 172.21.15.16:0/3191283009 wait complete. 2023-09-30T22:34:31.563 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:34:31.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.580+0000 7fe70adb6700 1 -- 172.21.15.16:0/2820311743 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe70406c360 con 0x7fe70409e410 2023-09-30T22:34:31.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.581+0000 7fe7017fa700 1 -- 172.21.15.16:0/2820311743 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe6fc011380 con 0x7fe70409e410 2023-09-30T22:34:31.582 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:34:31.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.583+0000 7fe70adb6700 1 -- 172.21.15.16:0/2820311743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe70409e410 msgr2=0x7fe70412fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:31.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.583+0000 7fe70adb6700 1 --2- 172.21.15.16:0/2820311743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe70409e410 0x7fe70412fb80 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc00cc80 tx=0x7fe6fc00cf90 comp rx=0 tx=0).stop 2023-09-30T22:34:31.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.583+0000 7fe70adb6700 1 -- 172.21.15.16:0/2820311743 shutdown_connections 2023-09-30T22:34:31.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.583+0000 7fe70adb6700 1 --2- 172.21.15.16:0/2820311743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe70409e410 0x7fe70412fb80 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:31.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.583+0000 7fe70adb6700 1 --2- 172.21.15.16:0/2820311743 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe70409d6f0 0x7fe7041388f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:31.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.583+0000 7fe70adb6700 1 --2- 172.21.15.16:0/2820311743 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe70409e000 0x7fe704138e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:31.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.583+0000 7fe70adb6700 1 -- 172.21.15.16:0/2820311743 >> 172.21.15.16:0/2820311743 conn(0x7fe704066f50 msgr2=0x7fe7040a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:31.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.583+0000 7fe70adb6700 1 -- 172.21.15.16:0/2820311743 shutdown_connections 2023-09-30T22:34:31.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.584+0000 7fe70adb6700 1 -- 172.21.15.16:0/2820311743 wait complete. 2023-09-30T22:34:31.601 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:34:31.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.674+0000 7f116d46b700 1 -- 172.21.15.16:0/1968962768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f11680747f0 con 0x7f116809d630 2023-09-30T22:34:31.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.674+0000 7f114f7fe700 1 -- 172.21.15.16:0/1968962768 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1158015070 con 0x7f116809d630 2023-09-30T22:34:31.675 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:34:31.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.676+0000 7f116d46b700 1 -- 172.21.15.16:0/1968962768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f116809d630 msgr2=0x7f116811efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:31.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.676+0000 7f116d46b700 1 --2- 172.21.15.16:0/1968962768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f116809d630 0x7f116811efe0 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f1158009fd0 tx=0x7f1158004750 comp rx=0 tx=0).stop 2023-09-30T22:34:31.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.676+0000 7f116d46b700 1 -- 172.21.15.16:0/1968962768 shutdown_connections 2023-09-30T22:34:31.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.676+0000 7f116d46b700 1 --2- 172.21.15.16:0/1968962768 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1168133930 0x7f116811fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:31.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.676+0000 7f116d46b700 1 --2- 172.21.15.16:0/1968962768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f116809d630 0x7f116811efe0 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:31.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.676+0000 7f116d46b700 1 --2- 172.21.15.16:0/1968962768 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f11680a0a30 0x7f116811f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:31.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.676+0000 7f116d46b700 1 -- 172.21.15.16:0/1968962768 >> 172.21.15.16:0/1968962768 conn(0x7f11680a3050 msgr2=0x7f1168099a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:31.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.676+0000 7f116d46b700 1 -- 172.21.15.16:0/1968962768 shutdown_connections 2023-09-30T22:34:31.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.676+0000 7f116d46b700 1 -- 172.21.15.16:0/1968962768 wait complete. 2023-09-30T22:34:31.691 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:34:31.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.707+0000 7f72bfd3d700 1 -- 172.21.15.16:0/2970531394 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f72b81a6ee0 con 0x7f72b809f240 2023-09-30T22:34:31.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.708+0000 7f72ae7fc700 1 -- 172.21.15.16:0/2970531394 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f72a801c310 con 0x7f72b809f240 2023-09-30T22:34:31.709 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:34:31.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.710+0000 7f72bfd3d700 1 -- 172.21.15.16:0/2970531394 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72b809f240 msgr2=0x7f72b81c6750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:31.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.710+0000 7f72bfd3d700 1 --2- 172.21.15.16:0/2970531394 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72b809f240 0x7f72b81c6750 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f72a800c420 tx=0x7f72a800c730 comp rx=0 tx=0).stop 2023-09-30T22:34:31.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.710+0000 7f72bfd3d700 1 -- 172.21.15.16:0/2970531394 shutdown_connections 2023-09-30T22:34:31.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.710+0000 7f72bfd3d700 1 --2- 172.21.15.16:0/2970531394 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72b809f650 0x7f72b81c27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:31.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.710+0000 7f72bfd3d700 1 --2- 172.21.15.16:0/2970531394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72b809e930 0x7f72b81c6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:31.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.710+0000 7f72bfd3d700 1 --2- 172.21.15.16:0/2970531394 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72b809f240 0x7f72b81c6750 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:31.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.710+0000 7f72bfd3d700 1 -- 172.21.15.16:0/2970531394 >> 172.21.15.16:0/2970531394 conn(0x7f72b80a4210 msgr2=0x7f72b81b8a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:31.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.710+0000 7f72bfd3d700 1 -- 172.21.15.16:0/2970531394 shutdown_connections 2023-09-30T22:34:31.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:31.710+0000 7f72bfd3d700 1 -- 172.21.15.16:0/2970531394 wait complete. 2023-09-30T22:34:31.725 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:34:32.563 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:34:32.602 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:34:32.692 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:34:32.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.712+0000 7fd8a8ae4700 1 Processor -- start 2023-09-30T22:34:32.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.712+0000 7fd8a8ae4700 1 -- start start 2023-09-30T22:34:32.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.713+0000 7fd8a8ae4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd8a409f410 0x7fd8a4136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:32.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.713+0000 7fd8a8ae4700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd8a4136fe0 0x7fd8a41393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:32.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.713+0000 7fd8a8ae4700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8a4070940 con 0x7fd8a409e930 2023-09-30T22:34:32.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.713+0000 7fd8a8ae4700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8a406fd60 con 0x7fd8a409f410 2023-09-30T22:34:32.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.713+0000 7fd8a8ae4700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a406fa80 con 0x7fd8a4136fe0 2023-09-30T22:34:32.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.713+0000 7fd8a2d9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd8a4136fe0 0x7fd8a41393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:32.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.713+0000 7fd8a259c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd8a409e930 0x7fd8a409ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51104/0 (socket says 172.21.15.16:51104) 2023-09-30T22:34:32.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.713+0000 7fd8a259c700 1 -- 172.21.15.16:0/4250827202 learned_addr learned my addr 172.21.15.16:0/4250827202 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:34:32.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.713+0000 7fd8a1d9b700 1 --2- 172.21.15.16:0/4250827202 >> v2:172.21.15.16:3300/0 conn(0x7fd8a409f410 0x7fd8a4136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:32.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.714+0000 7fd8a1d9b700 1 -- 172.21.15.16:0/4250827202 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd8a4136fe0 msgr2=0x7fd8a41393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:32.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.714+0000 7fd8a1d9b700 1 --2- 172.21.15.16:0/4250827202 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd8a4136fe0 0x7fd8a41393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:32.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.714+0000 7fd8a1d9b700 1 -- 172.21.15.16:0/4250827202 >> v1:172.21.15.16:6789/0 conn(0x7fd8a409e930 legacy=0x7fd8a409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:32.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.714+0000 7fd8a159a700 1 -- 172.21.15.16:0/4250827202 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3956355185 0 0) 0x7fd8a4070940 con 0x7fd8a409e930 2023-09-30T22:34:32.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.714+0000 7fd8a1d9b700 1 -- 172.21.15.16:0/4250827202 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a409fb60 con 0x7fd8a409f410 2023-09-30T22:34:32.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.714+0000 7fd8a1d9b700 1 --2- 172.21.15.16:0/4250827202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8a409f410 0x7fd8a4136aa0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fd898009840 tx=0x7fd898009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bba0476940e3b6f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:32.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.714+0000 7fd8a159a700 1 -- 172.21.15.16:0/4250827202 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd898011830 con 0x7fd8a409f410 2023-09-30T22:34:32.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.714+0000 7fd8a159a700 1 -- 172.21.15.16:0/4250827202 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd898011990 con 0x7fd8a409f410 2023-09-30T22:34:32.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.714+0000 7fd8a159a700 1 -- 172.21.15.16:0/4250827202 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd898011d20 con 0x7fd8a409f410 2023-09-30T22:34:32.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.715+0000 7fd8a8ae4700 1 -- 172.21.15.16:0/4250827202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8a409f410 msgr2=0x7fd8a4136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:32.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.715+0000 7fd8a8ae4700 1 --2- 172.21.15.16:0/4250827202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8a409f410 0x7fd8a4136aa0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fd898009840 tx=0x7fd898009b50 comp rx=0 tx=0).stop 2023-09-30T22:34:32.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.715+0000 7fd8a8ae4700 1 -- 172.21.15.16:0/4250827202 shutdown_connections 2023-09-30T22:34:32.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.715+0000 7fd8a8ae4700 1 --2- 172.21.15.16:0/4250827202 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd8a4136fe0 0x7fd8a41393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:32.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.715+0000 7fd8a8ae4700 1 --2- 172.21.15.16:0/4250827202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8a409f410 0x7fd8a4136aa0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:32.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.715+0000 7fd8a8ae4700 1 -- 172.21.15.16:0/4250827202 >> 172.21.15.16:0/4250827202 conn(0x7fd8a40a4210 msgr2=0x7fd8a40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:32.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.715+0000 7fd8a8ae4700 1 -- 172.21.15.16:0/4250827202 shutdown_connections 2023-09-30T22:34:32.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.715+0000 7fd8a8ae4700 1 -- 172.21.15.16:0/4250827202 wait complete. 2023-09-30T22:34:32.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.716+0000 7fd8a8ae4700 1 Processor -- start 2023-09-30T22:34:32.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.716+0000 7fd8a8ae4700 1 -- start start 2023-09-30T22:34:32.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.717+0000 7fd8a8ae4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8a409e930 0x7fd8a41c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:32.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.717+0000 7fd8a8ae4700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd8a409f410 0x7fd8a41c8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:32.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.717+0000 7fd8a8ae4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd8a4136fe0 0x7fd8a406b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:32.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.717+0000 7fd8a8ae4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a409dec0 con 0x7fd8a409e930 2023-09-30T22:34:32.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.717+0000 7fd8a8ae4700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a4071d20 con 0x7fd8a409f410 2023-09-30T22:34:32.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.717+0000 7fd8a8ae4700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a4121f00 con 0x7fd8a4136fe0 2023-09-30T22:34:32.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.717+0000 7fd8a259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8a409e930 0x7fd8a41c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:32.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.717+0000 7fd8a1d9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd8a409f410 0x7fd8a41c8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:32.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.717+0000 7fd8a259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8a409e930 0x7fd8a41c84e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40126/0 (socket says 172.21.15.16:40126) 2023-09-30T22:34:32.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.717+0000 7fd8a259c700 1 -- 172.21.15.16:0/2686422703 learned_addr learned my addr 172.21.15.16:0/2686422703 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:32.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.717+0000 7fd8a2d9d700 1 --2- 172.21.15.16:0/2686422703 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd8a4136fe0 0x7fd8a406b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:32.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.717+0000 7fd8a1d9b700 1 -- 172.21.15.16:0/2686422703 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd8a4136fe0 msgr2=0x7fd8a406b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:32.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.717+0000 7fd8a1d9b700 1 --2- 172.21.15.16:0/2686422703 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd8a4136fe0 0x7fd8a406b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:32.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.717+0000 7fd8a1d9b700 1 -- 172.21.15.16:0/2686422703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8a409e930 msgr2=0x7fd8a41c84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:32.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.717+0000 7fd8a1d9b700 1 --2- 172.21.15.16:0/2686422703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8a409e930 0x7fd8a41c84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:32.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.717+0000 7fd8a1d9b700 1 -- 172.21.15.16:0/2686422703 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8980094a0 con 0x7fd8a409f410 2023-09-30T22:34:32.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.717+0000 7fd8a259c700 1 --2- 172.21.15.16:0/2686422703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8a409e930 0x7fd8a41c84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:32.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.718+0000 7fd8a1d9b700 1 --2- 172.21.15.16:0/2686422703 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd8a409f410 0x7fd8a41c8a20 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fd898016000 tx=0x7fd898004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:32.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.718+0000 7fd892ffd700 1 -- 172.21.15.16:0/2686422703 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd898004550 con 0x7fd8a409f410 2023-09-30T22:34:32.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.718+0000 7fd892ffd700 1 -- 172.21.15.16:0/2686422703 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd89801f070 con 0x7fd8a409f410 2023-09-30T22:34:32.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.718+0000 7fd8a8ae4700 1 -- 172.21.15.16:0/2686422703 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8a4122970 con 0x7fd8a409f410 2023-09-30T22:34:32.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.718+0000 7fd892ffd700 1 -- 172.21.15.16:0/2686422703 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8980038e0 con 0x7fd8a409f410 2023-09-30T22:34:32.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.718+0000 7fd8a8ae4700 1 -- 172.21.15.16:0/2686422703 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8a4120ba0 con 0x7fd8a409f410 2023-09-30T22:34:32.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.719+0000 7fd892ffd700 1 -- 172.21.15.16:0/2686422703 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd898003a40 con 0x7fd8a409f410 2023-09-30T22:34:32.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.719+0000 7fd892ffd700 1 -- 172.21.15.16:0/2686422703 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd898021070 con 0x7fd8a409f410 2023-09-30T22:34:32.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.719+0000 7fd8a8ae4700 1 -- 172.21.15.16:0/2686422703 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8a4074ea0 con 0x7fd8a409f410 2023-09-30T22:34:32.727 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:34:32.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.729+0000 7fd892ffd700 1 -- 172.21.15.16:0/2686422703 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd898015070 con 0x7fd8a409f410 2023-09-30T22:34:32.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.768+0000 7f9568634700 1 Processor -- start 2023-09-30T22:34:32.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.768+0000 7f9568634700 1 -- start start 2023-09-30T22:34:32.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.769+0000 7f9568634700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f956009f410 0x7f9560136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:32.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.769+0000 7f9568634700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9560136fe0 0x7f95601393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:32.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.769+0000 7f9568634700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9560070940 con 0x7f956009e930 2023-09-30T22:34:32.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.769+0000 7f9568634700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f956006fd60 con 0x7f956009f410 2023-09-30T22:34:32.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.769+0000 7f9568634700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f956006fa80 con 0x7f9560136fe0 2023-09-30T22:34:32.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.769+0000 7f9565bcf700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f956009f410 0x7f9560136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:32.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.769+0000 7f95663d0700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f956009e930 0x7f956009ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51112/0 (socket says 172.21.15.16:51112) 2023-09-30T22:34:32.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.769+0000 7f95663d0700 1 -- 172.21.15.16:0/3255961265 learned_addr learned my addr 172.21.15.16:0/3255961265 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:34:32.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.769+0000 7f95653ce700 1 -- 172.21.15.16:0/3255961265 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3270258938 0 0) 0x7f9560070940 con 0x7f956009e930 2023-09-30T22:34:32.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.769+0000 7f95653ce700 1 -- 172.21.15.16:0/3255961265 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f954c0035b0 con 0x7f956009e930 2023-09-30T22:34:32.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.769+0000 7f9565bcf700 1 -- 172.21.15.16:0/3255961265 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9560136fe0 msgr2=0x7f95601393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:32.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.769+0000 7f9565bcf700 1 --2- 172.21.15.16:0/3255961265 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9560136fe0 0x7f95601393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:32.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.769+0000 7f9565bcf700 1 -- 172.21.15.16:0/3255961265 >> v1:172.21.15.16:6789/0 conn(0x7f956009e930 legacy=0x7f956009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:32.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.769+0000 7f9565bcf700 1 -- 172.21.15.16:0/3255961265 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f956009fb60 con 0x7f956009f410 2023-09-30T22:34:32.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.770+0000 7f9565bcf700 1 --2- 172.21.15.16:0/3255961265 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f956009f410 0x7f9560136aa0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f9550009870 tx=0x7f9550009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9151460a9880005 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:32.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.770+0000 7f95653ce700 1 -- 172.21.15.16:0/3255961265 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9550011830 con 0x7f956009f410 2023-09-30T22:34:32.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.770+0000 7f95653ce700 1 -- 172.21.15.16:0/3255961265 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9550011990 con 0x7f956009f410 2023-09-30T22:34:32.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.770+0000 7f95653ce700 1 -- 172.21.15.16:0/3255961265 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9550011d20 con 0x7f956009f410 2023-09-30T22:34:32.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.771+0000 7f9568634700 1 -- 172.21.15.16:0/3255961265 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f956009f410 msgr2=0x7f9560136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:32.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.771+0000 7f9568634700 1 --2- 172.21.15.16:0/3255961265 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f956009f410 0x7f9560136aa0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f9550009870 tx=0x7f9550009b80 comp rx=0 tx=0).stop 2023-09-30T22:34:32.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.771+0000 7f9568634700 1 -- 172.21.15.16:0/3255961265 shutdown_connections 2023-09-30T22:34:32.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.771+0000 7f9568634700 1 --2- 172.21.15.16:0/3255961265 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9560136fe0 0x7f95601393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:32.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.771+0000 7f9568634700 1 --2- 172.21.15.16:0/3255961265 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f956009f410 0x7f9560136aa0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:32.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.771+0000 7f9568634700 1 -- 172.21.15.16:0/3255961265 >> 172.21.15.16:0/3255961265 conn(0x7f95600a4210 msgr2=0x7f95600a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:32.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.771+0000 7f9568634700 1 -- 172.21.15.16:0/3255961265 shutdown_connections 2023-09-30T22:34:32.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.771+0000 7f9568634700 1 -- 172.21.15.16:0/3255961265 wait complete. 2023-09-30T22:34:32.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.772+0000 7f9568634700 1 Processor -- start 2023-09-30T22:34:32.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.772+0000 7f9568634700 1 -- start start 2023-09-30T22:34:32.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.773+0000 7f9568634700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f956009e930 0x7f956011c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:32.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.773+0000 7f9568634700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f956009f410 0x7f956011caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:32.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.773+0000 7f9568634700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9560136fe0 0x7f956011d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:32.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.773+0000 7f9568634700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f956011e1c0 con 0x7f956009f410 2023-09-30T22:34:32.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.773+0000 7f9568634700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f956011e7e0 con 0x7f956009e930 2023-09-30T22:34:32.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.773+0000 7f9568634700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9560071d90 con 0x7f9560136fe0 2023-09-30T22:34:32.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.773+0000 7f9565bcf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f956009f410 0x7f956011caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:32.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.773+0000 7f9565bcf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f956009f410 0x7f956011caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40136/0 (socket says 172.21.15.16:40136) 2023-09-30T22:34:32.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.773+0000 7f9565bcf700 1 -- 172.21.15.16:0/3018579708 learned_addr learned my addr 172.21.15.16:0/3018579708 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:32.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.774+0000 7f95663d0700 1 --2- 172.21.15.16:0/3018579708 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f956009e930 0x7f956011c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:32.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.774+0000 7f9565bcf700 1 -- 172.21.15.16:0/3018579708 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9560136fe0 msgr2=0x7f956011d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:32.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.774+0000 7f9565bcf700 1 --2- 172.21.15.16:0/3018579708 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9560136fe0 0x7f956011d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:32.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.774+0000 7f9565bcf700 1 -- 172.21.15.16:0/3018579708 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f956009e930 msgr2=0x7f956011c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:32.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.774+0000 7f9565bcf700 1 --2- 172.21.15.16:0/3018579708 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f956009e930 0x7f956011c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:32.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.774+0000 7f9565bcf700 1 -- 172.21.15.16:0/3018579708 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95500094a0 con 0x7f956009f410 2023-09-30T22:34:32.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.774+0000 7f9565bcf700 1 --2- 172.21.15.16:0/3018579708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f956009f410 0x7f956011caf0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f9550009840 tx=0x7f9550004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:32.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.774+0000 7f9556ffd700 1 -- 172.21.15.16:0/3018579708 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9550004590 con 0x7f956009f410 2023-09-30T22:34:32.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.774+0000 7f9556ffd700 1 -- 172.21.15.16:0/3018579708 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f955001f070 con 0x7f956009f410 2023-09-30T22:34:32.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.774+0000 7f9556ffd700 1 -- 172.21.15.16:0/3018579708 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95500038e0 con 0x7f956009f410 2023-09-30T22:34:32.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.774+0000 7f9568634700 1 -- 172.21.15.16:0/3018579708 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f956011fdf0 con 0x7f956009f410 2023-09-30T22:34:32.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.775+0000 7f9568634700 1 -- 172.21.15.16:0/3018579708 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f956009df30 con 0x7f956009f410 2023-09-30T22:34:32.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.775+0000 7f9556ffd700 1 -- 172.21.15.16:0/3018579708 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9550003a40 con 0x7f956009f410 2023-09-30T22:34:32.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.776+0000 7f9556ffd700 1 -- 172.21.15.16:0/3018579708 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f9550021070 con 0x7f956009f410 2023-09-30T22:34:32.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.776+0000 7f9568634700 1 -- 172.21.15.16:0/3018579708 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9560074ea0 con 0x7f956009f410 2023-09-30T22:34:32.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.781+0000 7f9556ffd700 1 -- 172.21.15.16:0/3018579708 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9550015030 con 0x7f956009f410 2023-09-30T22:34:32.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.871+0000 7f932d89e700 1 Processor -- start 2023-09-30T22:34:32.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.871+0000 7f932d89e700 1 -- start start 2023-09-30T22:34:32.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.872+0000 7f932d89e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f932809e930 0x7f932809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:32.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.872+0000 7f932d89e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f932809f240 0x7f93281317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:32.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.872+0000 7f932d89e700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9328070940 con 0x7f932809f650 2023-09-30T22:34:32.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.872+0000 7f932d89e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f932806fd60 con 0x7f932809f240 2023-09-30T22:34:32.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.872+0000 7f932d89e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f932806fa80 con 0x7f932809e930 2023-09-30T22:34:32.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.872+0000 7f93267fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f932809f240 0x7f93281317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:32.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.872+0000 7f93267fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f932809f240 0x7f93281317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41090/0 (socket says 172.21.15.16:41090) 2023-09-30T22:34:32.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.872+0000 7f93267fc700 1 -- 172.21.15.16:0/2755670423 learned_addr learned my addr 172.21.15.16:0/2755670423 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:32.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.872+0000 7f9326ffd700 1 --2- 172.21.15.16:0/2755670423 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f932809e930 0x7f932809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:32.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.872+0000 7f93267fc700 1 -- 172.21.15.16:0/2755670423 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f932809e930 msgr2=0x7f932809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:32.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.872+0000 7f93267fc700 1 --2- 172.21.15.16:0/2755670423 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f932809e930 0x7f932809ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:32.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.872+0000 7f93267fc700 1 -- 172.21.15.16:0/2755670423 >> v1:172.21.15.16:6789/0 conn(0x7f932809f650 legacy=0x7f9328131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:32.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.872+0000 7f93267fc700 1 -- 172.21.15.16:0/2755670423 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f932809fb60 con 0x7f932809f240 2023-09-30T22:34:32.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.873+0000 7f93267fc700 1 --2- 172.21.15.16:0/2755670423 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f932809f240 0x7f93281317f0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f931c009920 tx=0x7f931c009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3af138a279b8952 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:32.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.873+0000 7f9325ffb700 1 -- 172.21.15.16:0/2755670423 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931c011830 con 0x7f932809f240 2023-09-30T22:34:32.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.873+0000 7f9325ffb700 1 -- 172.21.15.16:0/2755670423 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f931c011990 con 0x7f932809f240 2023-09-30T22:34:32.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.873+0000 7f9325ffb700 1 -- 172.21.15.16:0/2755670423 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931c011d20 con 0x7f932809f240 2023-09-30T22:34:32.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.874+0000 7f932d89e700 1 -- 172.21.15.16:0/2755670423 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f932809f240 msgr2=0x7f93281317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:32.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.874+0000 7f932d89e700 1 --2- 172.21.15.16:0/2755670423 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f932809f240 0x7f93281317f0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f931c009920 tx=0x7f931c009c30 comp rx=0 tx=0).stop 2023-09-30T22:34:32.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.874+0000 7f932d89e700 1 -- 172.21.15.16:0/2755670423 shutdown_connections 2023-09-30T22:34:32.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.874+0000 7f932d89e700 1 --2- 172.21.15.16:0/2755670423 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f932809e930 0x7f932809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:32.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.874+0000 7f932d89e700 1 --2- 172.21.15.16:0/2755670423 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f932809f240 0x7f93281317f0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:32.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.874+0000 7f932d89e700 1 -- 172.21.15.16:0/2755670423 >> 172.21.15.16:0/2755670423 conn(0x7f93280a4210 msgr2=0x7f93280a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:32.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.874+0000 7f932d89e700 1 -- 172.21.15.16:0/2755670423 shutdown_connections 2023-09-30T22:34:32.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.874+0000 7f932d89e700 1 -- 172.21.15.16:0/2755670423 wait complete. 2023-09-30T22:34:32.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.875+0000 7f932d89e700 1 Processor -- start 2023-09-30T22:34:32.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.875+0000 7f932d89e700 1 -- start start 2023-09-30T22:34:32.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.876+0000 7f932d89e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f932809e930 0x7f93281a3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:32.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.876+0000 7f932d89e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f932809f240 0x7f93281a4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:32.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.876+0000 7f932d89e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f932809f650 0x7f93281a4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:32.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.876+0000 7f932d89e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93281a5210 con 0x7f932809e930 2023-09-30T22:34:32.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.876+0000 7f932d89e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93281a5f30 con 0x7f932809f240 2023-09-30T22:34:32.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.876+0000 7f932d89e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f93281a6ff0 con 0x7f932809f650 2023-09-30T22:34:32.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.876+0000 7f93277fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f932809f650 0x7f93281a4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:32.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.876+0000 7f93267fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f932809f240 0x7f93281a4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:32.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.876+0000 7f93277fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f932809f650 0x7f93281a4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:52910/0 (socket says 172.21.15.16:52910) 2023-09-30T22:34:32.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.876+0000 7f93277fe700 1 -- 172.21.15.16:0/3746399786 learned_addr learned my addr 172.21.15.16:0/3746399786 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:32.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.876+0000 7f93267fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f932809f240 0x7f93281a4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41096/0 (socket says 172.21.15.16:41096) 2023-09-30T22:34:32.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.877+0000 7f93267fc700 1 -- 172.21.15.16:0/3746399786 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f932809f650 msgr2=0x7f93281a4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:32.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.877+0000 7f93267fc700 1 --2- 172.21.15.16:0/3746399786 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f932809f650 0x7f93281a4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:32.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.877+0000 7f93267fc700 1 -- 172.21.15.16:0/3746399786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f932809e930 msgr2=0x7f93281a3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:32.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.877+0000 7f93267fc700 1 --2- 172.21.15.16:0/3746399786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f932809e930 0x7f93281a3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:32.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.877+0000 7f93267fc700 1 -- 172.21.15.16:0/3746399786 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f931c009550 con 0x7f932809f240 2023-09-30T22:34:32.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.877+0000 7f93267fc700 1 --2- 172.21.15.16:0/3746399786 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f932809f240 0x7f93281a4310 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f931c016000 tx=0x7f931c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:32.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.877+0000 7f930ffff700 1 -- 172.21.15.16:0/3746399786 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931c004510 con 0x7f932809f240 2023-09-30T22:34:32.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.877+0000 7f930ffff700 1 -- 172.21.15.16:0/3746399786 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f931c01f070 con 0x7f932809f240 2023-09-30T22:34:32.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.877+0000 7f932d89e700 1 -- 172.21.15.16:0/3746399786 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93280a0190 con 0x7f932809f240 2023-09-30T22:34:32.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.877+0000 7f930ffff700 1 -- 172.21.15.16:0/3746399786 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931c0038c0 con 0x7f932809f240 2023-09-30T22:34:32.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.877+0000 7f932d89e700 1 -- 172.21.15.16:0/3746399786 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93280a36e0 con 0x7f932809f240 2023-09-30T22:34:32.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.879+0000 7f932d89e700 1 -- 172.21.15.16:0/3746399786 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93281a0b30 con 0x7f932809f240 2023-09-30T22:34:32.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.879+0000 7f930ffff700 1 -- 172.21.15.16:0/3746399786 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f931c003a20 con 0x7f932809f240 2023-09-30T22:34:32.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.879+0000 7f930ffff700 1 -- 172.21.15.16:0/3746399786 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f931c021070 con 0x7f932809f240 2023-09-30T22:34:32.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.885+0000 7fea14b0d700 1 Processor -- start 2023-09-30T22:34:32.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.885+0000 7fea14b0d700 1 -- start start 2023-09-30T22:34:32.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.885+0000 7fea14b0d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea1009d6f0 0x7fea1009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:32.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.885+0000 7fea14b0d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea1009e000 0x7fea10130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:32.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.885+0000 7fea14b0d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea1009f130 con 0x7fea1009e410 2023-09-30T22:34:32.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.885+0000 7fea14b0d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea1009f520 con 0x7fea1009d6f0 2023-09-30T22:34:32.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.885+0000 7fea14b0d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fea1009e930 con 0x7fea1009e000 2023-09-30T22:34:32.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.886+0000 7fea0e59c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea1009d6f0 0x7fea1009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:32.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.886+0000 7fea0e59c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea1009d6f0 0x7fea1009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41112/0 (socket says 172.21.15.16:41112) 2023-09-30T22:34:32.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.886+0000 7fea0e59c700 1 -- 172.21.15.16:0/697222943 learned_addr learned my addr 172.21.15.16:0/697222943 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:32.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.886+0000 7fea0dd9b700 1 --2- 172.21.15.16:0/697222943 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea1009e000 0x7fea10130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:32.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.886+0000 7fea0e59c700 1 -- 172.21.15.16:0/697222943 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea1009e000 msgr2=0x7fea10130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:32.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.886+0000 7fea0e59c700 1 --2- 172.21.15.16:0/697222943 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea1009e000 0x7fea10130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:32.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.886+0000 7fea0e59c700 1 -- 172.21.15.16:0/697222943 >> v1:172.21.15.16:6789/0 conn(0x7fea1009e410 legacy=0x7fea10130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:32.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.886+0000 7fea0d59a700 1 -- 172.21.15.16:0/697222943 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 967807639 0 0) 0x7fea1009f130 con 0x7fea1009e410 2023-09-30T22:34:32.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.886+0000 7fea0e59c700 1 -- 172.21.15.16:0/697222943 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea1009b2d0 con 0x7fea1009d6f0 2023-09-30T22:34:32.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.886+0000 7fea0dd9b700 1 --2- 172.21.15.16:0/697222943 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea1009e000 0x7fea10130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:32.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.886+0000 7fea0e59c700 1 --2- 172.21.15.16:0/697222943 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea1009d6f0 0x7fea1009dac0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fea0000de80 tx=0x7fea0000f330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33b08fdbcaa7e1ea server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:32.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.888+0000 7fea0d59a700 1 -- 172.21.15.16:0/697222943 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea00012070 con 0x7fea1009d6f0 2023-09-30T22:34:32.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.888+0000 7fea0d59a700 1 -- 172.21.15.16:0/697222943 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea0000f910 con 0x7fea1009d6f0 2023-09-30T22:34:32.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.888+0000 7fea0d59a700 1 -- 172.21.15.16:0/697222943 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0000fc80 con 0x7fea1009d6f0 2023-09-30T22:34:32.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.888+0000 7f930ffff700 1 -- 172.21.15.16:0/3746399786 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f931c024390 con 0x7f932809f240 2023-09-30T22:34:32.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.889+0000 7fea14b0d700 1 -- 172.21.15.16:0/697222943 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea1009d6f0 msgr2=0x7fea1009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:32.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.889+0000 7fea14b0d700 1 --2- 172.21.15.16:0/697222943 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea1009d6f0 0x7fea1009dac0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fea0000de80 tx=0x7fea0000f330 comp rx=0 tx=0).stop 2023-09-30T22:34:32.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.889+0000 7fea14b0d700 1 -- 172.21.15.16:0/697222943 shutdown_connections 2023-09-30T22:34:32.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.889+0000 7fea14b0d700 1 --2- 172.21.15.16:0/697222943 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea1009d6f0 0x7fea1009dac0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:32.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.889+0000 7fea14b0d700 1 --2- 172.21.15.16:0/697222943 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea1009e000 0x7fea10130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:32.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.889+0000 7fea14b0d700 1 -- 172.21.15.16:0/697222943 >> 172.21.15.16:0/697222943 conn(0x7fea10066f50 msgr2=0x7fea1009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:32.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.890+0000 7fea14b0d700 1 -- 172.21.15.16:0/697222943 shutdown_connections 2023-09-30T22:34:32.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.890+0000 7fea14b0d700 1 -- 172.21.15.16:0/697222943 wait complete. 2023-09-30T22:34:32.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.891+0000 7fea14b0d700 1 Processor -- start 2023-09-30T22:34:32.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.891+0000 7fea14b0d700 1 -- start start 2023-09-30T22:34:32.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.891+0000 7fea14b0d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea1009d6f0 0x7fea1011ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:32.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.891+0000 7fea14b0d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea1009e000 0x7fea1011e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:32.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.891+0000 7fea14b0d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea1009e410 0x7fea1011e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:32.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.891+0000 7fea14b0d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea10122290 con 0x7fea1009d6f0 2023-09-30T22:34:32.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.891+0000 7fea14b0d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea101218d0 con 0x7fea1009e000 2023-09-30T22:34:32.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.891+0000 7fea14b0d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fea10120ca0 con 0x7fea1009e410 2023-09-30T22:34:32.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.892+0000 7fea0e59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea1009d6f0 0x7fea1011ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:32.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.892+0000 7fea0e59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea1009d6f0 0x7fea1011ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40152/0 (socket says 172.21.15.16:40152) 2023-09-30T22:34:32.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.892+0000 7fea0e59c700 1 -- 172.21.15.16:0/875286833 learned_addr learned my addr 172.21.15.16:0/875286833 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:32.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.892+0000 7fea0dd9b700 1 --2- 172.21.15.16:0/875286833 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea1009e000 0x7fea1011e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:32.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.892+0000 7fea0e59c700 1 -- 172.21.15.16:0/875286833 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea1009e410 msgr2=0x7fea1011e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:32.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.892+0000 7fea0e59c700 1 --2- 172.21.15.16:0/875286833 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea1009e410 0x7fea1011e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:32.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.892+0000 7fea0e59c700 1 -- 172.21.15.16:0/875286833 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea1009e000 msgr2=0x7fea1011e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:32.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.892+0000 7fea0e59c700 1 --2- 172.21.15.16:0/875286833 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea1009e000 0x7fea1011e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:32.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.892+0000 7fea0e59c700 1 -- 172.21.15.16:0/875286833 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea040094a0 con 0x7fea1009d6f0 2023-09-30T22:34:32.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.892+0000 7fea0e59c700 1 --2- 172.21.15.16:0/875286833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea1009d6f0 0x7fea1011ddb0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fea00004620 tx=0x7fea00004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:32.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.892+0000 7fe9feffd700 1 -- 172.21.15.16:0/875286833 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0000d2e0 con 0x7fea1009d6f0 2023-09-30T22:34:32.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.892+0000 7fe9feffd700 1 -- 172.21.15.16:0/875286833 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea000038b0 con 0x7fea1009d6f0 2023-09-30T22:34:32.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.892+0000 7fea14b0d700 1 -- 172.21.15.16:0/875286833 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea0000db30 con 0x7fea1009d6f0 2023-09-30T22:34:32.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.893+0000 7fea14b0d700 1 -- 172.21.15.16:0/875286833 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea1011f9c0 con 0x7fea1009d6f0 2023-09-30T22:34:32.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.893+0000 7fe9feffd700 1 -- 172.21.15.16:0/875286833 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea00003c20 con 0x7fea1009d6f0 2023-09-30T22:34:32.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.893+0000 7fe9feffd700 1 -- 172.21.15.16:0/875286833 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fea00020440 con 0x7fea1009d6f0 2023-09-30T22:34:32.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.894+0000 7fe9feffd700 1 -- 172.21.15.16:0/875286833 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fea00044be0 con 0x7fea1009d6f0 2023-09-30T22:34:32.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.894+0000 7fea14b0d700 1 -- 172.21.15.16:0/875286833 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9f40052e0 con 0x7fea1009d6f0 2023-09-30T22:34:32.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.899+0000 7fe9feffd700 1 -- 172.21.15.16:0/875286833 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fea0001d350 con 0x7fea1009d6f0 2023-09-30T22:34:32.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.968+0000 7fd8a8ae4700 1 -- 172.21.15.16:0/2686422703 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd8a406b250 con 0x7fd8a409f410 2023-09-30T22:34:32.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.968+0000 7fd892ffd700 1 -- 172.21.15.16:0/2686422703 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd898003cf0 con 0x7fd8a409f410 2023-09-30T22:34:32.970 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:34:32.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.971+0000 7fd8a8ae4700 1 -- 172.21.15.16:0/2686422703 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd8a409f410 msgr2=0x7fd8a41c8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:32.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.971+0000 7fd8a8ae4700 1 --2- 172.21.15.16:0/2686422703 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd8a409f410 0x7fd8a41c8a20 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fd898016000 tx=0x7fd898004750 comp rx=0 tx=0).stop 2023-09-30T22:34:32.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.972+0000 7fd8a8ae4700 1 -- 172.21.15.16:0/2686422703 shutdown_connections 2023-09-30T22:34:32.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.972+0000 7fd8a8ae4700 1 --2- 172.21.15.16:0/2686422703 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd8a4136fe0 0x7fd8a406b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:32.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.972+0000 7fd8a8ae4700 1 --2- 172.21.15.16:0/2686422703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8a409e930 0x7fd8a41c84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:32.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.972+0000 7fd8a8ae4700 1 --2- 172.21.15.16:0/2686422703 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd8a409f410 0x7fd8a41c8a20 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:32.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.972+0000 7fd8a8ae4700 1 -- 172.21.15.16:0/2686422703 >> 172.21.15.16:0/2686422703 conn(0x7fd8a40a4210 msgr2=0x7fd8a4134ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:32.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.972+0000 7fd8a8ae4700 1 -- 172.21.15.16:0/2686422703 shutdown_connections 2023-09-30T22:34:32.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:32.972+0000 7fd8a8ae4700 1 -- 172.21.15.16:0/2686422703 wait complete. 2023-09-30T22:34:32.987 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:34:33.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:33.052+0000 7f9568634700 1 -- 172.21.15.16:0/3018579708 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f956006b250 con 0x7f956009f410 2023-09-30T22:34:33.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:33.053+0000 7f9556ffd700 1 -- 172.21.15.16:0/3018579708 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9550010380 con 0x7f956009f410 2023-09-30T22:34:33.054 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:34:33.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:33.055+0000 7f9568634700 1 -- 172.21.15.16:0/3018579708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f956009f410 msgr2=0x7f956011caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:33.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:33.055+0000 7f9568634700 1 --2- 172.21.15.16:0/3018579708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f956009f410 0x7f956011caf0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f9550009840 tx=0x7f9550004750 comp rx=0 tx=0).stop 2023-09-30T22:34:33.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:33.055+0000 7f9568634700 1 -- 172.21.15.16:0/3018579708 shutdown_connections 2023-09-30T22:34:33.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:33.055+0000 7f9568634700 1 --2- 172.21.15.16:0/3018579708 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f956009e930 0x7f956011c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:33.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:33.055+0000 7f9568634700 1 --2- 172.21.15.16:0/3018579708 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9560136fe0 0x7f956011d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:33.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:33.055+0000 7f9568634700 1 --2- 172.21.15.16:0/3018579708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f956009f410 0x7f956011caf0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:33.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:33.055+0000 7f9568634700 1 -- 172.21.15.16:0/3018579708 >> 172.21.15.16:0/3018579708 conn(0x7f95600a4210 msgr2=0x7f9560134990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:33.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:33.055+0000 7f9568634700 1 -- 172.21.15.16:0/3018579708 shutdown_connections 2023-09-30T22:34:33.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:33.055+0000 7f9568634700 1 -- 172.21.15.16:0/3018579708 wait complete. 2023-09-30T22:34:33.070 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:34:33.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:33.109+0000 7f932d89e700 1 -- 172.21.15.16:0/3746399786 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f93281a59b0 con 0x7f932809f240 2023-09-30T22:34:33.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:33.109+0000 7f930ffff700 1 -- 172.21.15.16:0/3746399786 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f931c010b70 con 0x7f932809f240 2023-09-30T22:34:33.111 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:34:33.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:33.111+0000 7f932d89e700 1 -- 172.21.15.16:0/3746399786 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f932809f240 msgr2=0x7f93281a4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:33.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:33.111+0000 7f932d89e700 1 --2- 172.21.15.16:0/3746399786 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f932809f240 0x7f93281a4310 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f931c016000 tx=0x7f931c004750 comp rx=0 tx=0).stop 2023-09-30T22:34:33.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:33.112+0000 7f932d89e700 1 -- 172.21.15.16:0/3746399786 shutdown_connections 2023-09-30T22:34:33.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:33.112+0000 7f932d89e700 1 --2- 172.21.15.16:0/3746399786 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f932809f650 0x7f93281a4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:33.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:33.112+0000 7f932d89e700 1 --2- 172.21.15.16:0/3746399786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f932809e930 0x7f93281a3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:33.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:33.112+0000 7f932d89e700 1 --2- 172.21.15.16:0/3746399786 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f932809f240 0x7f93281a4310 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:33.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:33.112+0000 7f932d89e700 1 -- 172.21.15.16:0/3746399786 >> 172.21.15.16:0/3746399786 conn(0x7f93280a4210 msgr2=0x7f93281b8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:33.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:33.112+0000 7f932d89e700 1 -- 172.21.15.16:0/3746399786 shutdown_connections 2023-09-30T22:34:33.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:33.112+0000 7f932d89e700 1 -- 172.21.15.16:0/3746399786 wait complete. 2023-09-30T22:34:33.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:33.119+0000 7fea14b0d700 1 -- 172.21.15.16:0/875286833 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe9f4005590 con 0x7fea1009d6f0 2023-09-30T22:34:33.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:33.119+0000 7fe9feffd700 1 -- 172.21.15.16:0/875286833 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fea00015b80 con 0x7fea1009d6f0 2023-09-30T22:34:33.121 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:34:33.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:33.122+0000 7fea14b0d700 1 -- 172.21.15.16:0/875286833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea1009d6f0 msgr2=0x7fea1011ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:33.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:33.122+0000 7fea14b0d700 1 --2- 172.21.15.16:0/875286833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea1009d6f0 0x7fea1011ddb0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fea00004620 tx=0x7fea00004750 comp rx=0 tx=0).stop 2023-09-30T22:34:33.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:33.122+0000 7fea14b0d700 1 -- 172.21.15.16:0/875286833 shutdown_connections 2023-09-30T22:34:33.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:33.122+0000 7fea14b0d700 1 --2- 172.21.15.16:0/875286833 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea1009e410 0x7fea1011e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:33.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:33.122+0000 7fea14b0d700 1 --2- 172.21.15.16:0/875286833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea1009d6f0 0x7fea1011ddb0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:33.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:33.122+0000 7fea14b0d700 1 --2- 172.21.15.16:0/875286833 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea1009e000 0x7fea1011e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:33.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:33.122+0000 7fea14b0d700 1 -- 172.21.15.16:0/875286833 >> 172.21.15.16:0/875286833 conn(0x7fea10066f50 msgr2=0x7fea10130a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:33.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:33.122+0000 7fea14b0d700 1 -- 172.21.15.16:0/875286833 shutdown_connections 2023-09-30T22:34:33.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:33.122+0000 7fea14b0d700 1 -- 172.21.15.16:0/875286833 wait complete. 2023-09-30T22:34:33.127 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:34:33.137 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:34:33.988 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:34:34.071 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:34:34.128 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:34:34.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.135+0000 7f023fefd700 1 Processor -- start 2023-09-30T22:34:34.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.136+0000 7f023fefd700 1 -- start start 2023-09-30T22:34:34.137 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:34:34.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.136+0000 7f023fefd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0238131810 0x7f0238131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:34.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.136+0000 7f023fefd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0238099a40 0x7f0238099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:34.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.136+0000 7f023fefd700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02381313c0 con 0x7f023809a3b0 2023-09-30T22:34:34.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.136+0000 7f023fefd700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f023809b580 con 0x7f0238131810 2023-09-30T22:34:34.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.136+0000 7f023fefd700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f023806e8c0 con 0x7f0238099a40 2023-09-30T22:34:34.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.136+0000 7f023dc99700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0238131810 0x7f0238131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:34.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.137+0000 7f023e49a700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f023809a3b0 0x7f0238138650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51132/0 (socket says 172.21.15.16:51132) 2023-09-30T22:34:34.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.137+0000 7f023dc99700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0238131810 0x7f0238131be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40160/0 (socket says 172.21.15.16:40160) 2023-09-30T22:34:34.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.137+0000 7f023e49a700 1 -- 172.21.15.16:0/3363921092 learned_addr learned my addr 172.21.15.16:0/3363921092 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:34:34.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.137+0000 7f023d498700 1 --2- 172.21.15.16:0/3363921092 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0238099a40 0x7f0238099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:34.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.137+0000 7f023dc99700 1 -- 172.21.15.16:0/3363921092 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0238099a40 msgr2=0x7f0238099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:34.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.137+0000 7f023dc99700 1 --2- 172.21.15.16:0/3363921092 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0238099a40 0x7f0238099e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:34.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.137+0000 7f023dc99700 1 -- 172.21.15.16:0/3363921092 >> v1:172.21.15.16:6789/0 conn(0x7f023809a3b0 legacy=0x7f0238138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:34.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.137+0000 7f023dc99700 1 -- 172.21.15.16:0/3363921092 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f023809a830 con 0x7f0238131810 2023-09-30T22:34:34.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.137+0000 7f023cc97700 1 -- 172.21.15.16:0/3363921092 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2261386478 0 0) 0x7f02381313c0 con 0x7f023809a3b0 2023-09-30T22:34:34.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.137+0000 7f023dc99700 1 --2- 172.21.15.16:0/3363921092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0238131810 0x7f0238131be0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f0234009840 tx=0x7f0234009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f530221a8167606 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:34.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.137+0000 7f023cc97700 1 -- 172.21.15.16:0/3363921092 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0234011830 con 0x7f0238131810 2023-09-30T22:34:34.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.137+0000 7f023cc97700 1 -- 172.21.15.16:0/3363921092 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0234011990 con 0x7f0238131810 2023-09-30T22:34:34.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.137+0000 7f023cc97700 1 -- 172.21.15.16:0/3363921092 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0234011d20 con 0x7f0238131810 2023-09-30T22:34:34.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.138+0000 7f023fefd700 1 -- 172.21.15.16:0/3363921092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0238131810 msgr2=0x7f0238131be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:34.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.138+0000 7f023fefd700 1 --2- 172.21.15.16:0/3363921092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0238131810 0x7f0238131be0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f0234009840 tx=0x7f0234009b50 comp rx=0 tx=0).stop 2023-09-30T22:34:34.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.138+0000 7f023fefd700 1 -- 172.21.15.16:0/3363921092 shutdown_connections 2023-09-30T22:34:34.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.138+0000 7f023fefd700 1 --2- 172.21.15.16:0/3363921092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0238131810 0x7f0238131be0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:34.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.138+0000 7f023fefd700 1 --2- 172.21.15.16:0/3363921092 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0238099a40 0x7f0238099e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:34.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.138+0000 7f023fefd700 1 -- 172.21.15.16:0/3363921092 >> 172.21.15.16:0/3363921092 conn(0x7f02380a2aa0 msgr2=0x7f02380a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:34.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.138+0000 7f023fefd700 1 -- 172.21.15.16:0/3363921092 shutdown_connections 2023-09-30T22:34:34.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.138+0000 7f023fefd700 1 -- 172.21.15.16:0/3363921092 wait complete. 2023-09-30T22:34:34.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.139+0000 7f023fefd700 1 Processor -- start 2023-09-30T22:34:34.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.139+0000 7f023fefd700 1 -- start start 2023-09-30T22:34:34.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.140+0000 7f023fefd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0238099a40 0x7f023806f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:34.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.140+0000 7f023fefd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f023809a3b0 0x7f023806fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:34.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.140+0000 7f023fefd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0238131810 0x7f0238070090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:34.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.140+0000 7f023fefd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0238070a70 con 0x7f0238099a40 2023-09-30T22:34:34.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.140+0000 7f023fefd700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f023811f180 con 0x7f023809a3b0 2023-09-30T22:34:34.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.140+0000 7f023fefd700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f023811e070 con 0x7f0238131810 2023-09-30T22:34:34.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.140+0000 7f023e49a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0238131810 0x7f0238070090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:34.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.140+0000 7f023e49a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0238131810 0x7f0238070090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:52954/0 (socket says 172.21.15.16:52954) 2023-09-30T22:34:34.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.140+0000 7f023e49a700 1 -- 172.21.15.16:0/3204183358 learned_addr learned my addr 172.21.15.16:0/3204183358 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:34.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.141+0000 7f023e49a700 1 -- 172.21.15.16:0/3204183358 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f023809a3b0 msgr2=0x7f023806fb50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:34:34.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.141+0000 7f023e49a700 1 --2- 172.21.15.16:0/3204183358 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f023809a3b0 0x7f023806fb50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:34.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.141+0000 7f023e49a700 1 -- 172.21.15.16:0/3204183358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0238099a40 msgr2=0x7f023806f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:34.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.141+0000 7f023e49a700 1 --2- 172.21.15.16:0/3204183358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0238099a40 0x7f023806f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:34.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.141+0000 7f023e49a700 1 -- 172.21.15.16:0/3204183358 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02340094a0 con 0x7f0238131810 2023-09-30T22:34:34.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.141+0000 7f023e49a700 1 --2- 172.21.15.16:0/3204183358 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0238131810 0x7f0238070090 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f0230002230 tx=0x7f023000a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:34.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.141+0000 7f022e7fc700 1 -- 172.21.15.16:0/3204183358 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023000ac20 con 0x7f0238131810 2023-09-30T22:34:34.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.141+0000 7f023fefd700 1 -- 172.21.15.16:0/3204183358 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02380712c0 con 0x7f0238131810 2023-09-30T22:34:34.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.141+0000 7f022e7fc700 1 -- 172.21.15.16:0/3204183358 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f023000ad80 con 0x7f0238131810 2023-09-30T22:34:34.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.141+0000 7f023fefd700 1 -- 172.21.15.16:0/3204183358 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02380a17b0 con 0x7f0238131810 2023-09-30T22:34:34.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.141+0000 7f022e7fc700 1 -- 172.21.15.16:0/3204183358 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0230007310 con 0x7f0238131810 2023-09-30T22:34:34.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.142+0000 7f023fefd700 1 -- 172.21.15.16:0/3204183358 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02200052e0 con 0x7f0238131810 2023-09-30T22:34:34.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.143+0000 7f022e7fc700 1 -- 172.21.15.16:0/3204183358 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0230007470 con 0x7f0238131810 2023-09-30T22:34:34.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.143+0000 7f022e7fc700 1 -- 172.21.15.16:0/3204183358 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f023003f380 con 0x7f0238131810 2023-09-30T22:34:34.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.148+0000 7f022e7fc700 1 -- 172.21.15.16:0/3204183358 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0230012400 con 0x7f0238131810 2023-09-30T22:34:34.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.225+0000 7faa10115700 1 Processor -- start 2023-09-30T22:34:34.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.226+0000 7faa10115700 1 -- start start 2023-09-30T22:34:34.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.226+0000 7faa10115700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faa08133420 0x7faa081337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:34.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.226+0000 7faa10115700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faa08133e30 0x7faa0812ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:34.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.226+0000 7faa10115700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faa08134260 0x7faa081304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:34.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.226+0000 7faa10115700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa0806b990 con 0x7faa08133420 2023-09-30T22:34:34.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.226+0000 7faa10115700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa080a0590 con 0x7faa08133e30 2023-09-30T22:34:34.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.226+0000 7faa10115700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faa0809fb80 con 0x7faa08134260 2023-09-30T22:34:34.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.227+0000 7faa0e6b2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faa08134260 0x7faa081304d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:34.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.227+0000 7faa0d6b0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faa08133e30 0x7faa0812ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:34.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.227+0000 7faa0e6b2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faa08134260 0x7faa081304d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:52968/0 (socket says 172.21.15.16:52968) 2023-09-30T22:34:34.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.227+0000 7faa0e6b2700 1 -- 172.21.15.16:0/1440560321 learned_addr learned my addr 172.21.15.16:0/1440560321 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:34.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.227+0000 7faa0deb1700 1 --2- 172.21.15.16:0/1440560321 >> v2:172.21.15.16:3300/0 conn(0x7faa08133420 0x7faa081337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:34.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.227+0000 7faa0d6b0700 1 -- 172.21.15.16:0/1440560321 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faa08134260 msgr2=0x7faa081304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:34.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.227+0000 7faa0d6b0700 1 --2- 172.21.15.16:0/1440560321 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faa08134260 0x7faa081304d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:34.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.227+0000 7faa0d6b0700 1 -- 172.21.15.16:0/1440560321 >> v2:172.21.15.16:3300/0 conn(0x7faa08133420 msgr2=0x7faa081337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:34.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.227+0000 7faa0d6b0700 1 --2- 172.21.15.16:0/1440560321 >> v2:172.21.15.16:3300/0 conn(0x7faa08133420 0x7faa081337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:34.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.227+0000 7faa0d6b0700 1 -- 172.21.15.16:0/1440560321 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa080a0030 con 0x7faa08133e30 2023-09-30T22:34:34.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.227+0000 7faa0d6b0700 1 --2- 172.21.15.16:0/1440560321 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faa08133e30 0x7faa0812ff90 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8009840 tx=0x7fa9f8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e421ea394cb2819 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:34.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.227+0000 7faa0ceaf700 1 -- 172.21.15.16:0/1440560321 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8011830 con 0x7faa08133e30 2023-09-30T22:34:34.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.228+0000 7faa0ceaf700 1 -- 172.21.15.16:0/1440560321 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9f8011990 con 0x7faa08133e30 2023-09-30T22:34:34.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.228+0000 7faa0ceaf700 1 -- 172.21.15.16:0/1440560321 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8011d20 con 0x7faa08133e30 2023-09-30T22:34:34.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.228+0000 7faa10115700 1 -- 172.21.15.16:0/1440560321 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faa08133e30 msgr2=0x7faa0812ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:34.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.228+0000 7faa10115700 1 --2- 172.21.15.16:0/1440560321 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faa08133e30 0x7faa0812ff90 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8009840 tx=0x7fa9f8009b50 comp rx=0 tx=0).stop 2023-09-30T22:34:34.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.228+0000 7faa10115700 1 -- 172.21.15.16:0/1440560321 shutdown_connections 2023-09-30T22:34:34.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.228+0000 7faa10115700 1 --2- 172.21.15.16:0/1440560321 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faa08134260 0x7faa081304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:34.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.228+0000 7faa10115700 1 --2- 172.21.15.16:0/1440560321 >> v2:172.21.15.16:3300/0 conn(0x7faa08133420 0x7faa081337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:34.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.228+0000 7faa10115700 1 --2- 172.21.15.16:0/1440560321 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faa08133e30 0x7faa0812ff90 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:34.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.228+0000 7faa10115700 1 -- 172.21.15.16:0/1440560321 >> 172.21.15.16:0/1440560321 conn(0x7faa080a4210 msgr2=0x7faa080a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:34.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.228+0000 7faa10115700 1 -- 172.21.15.16:0/1440560321 shutdown_connections 2023-09-30T22:34:34.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.229+0000 7faa10115700 1 -- 172.21.15.16:0/1440560321 wait complete. 2023-09-30T22:34:34.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.230+0000 7faa10115700 1 Processor -- start 2023-09-30T22:34:34.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.230+0000 7faa10115700 1 -- start start 2023-09-30T22:34:34.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.230+0000 7faa10115700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa08133420 0x7faa0809c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:34.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.230+0000 7faa10115700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faa08133e30 0x7faa0809c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:34.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.230+0000 7faa10115700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faa08134260 0x7faa0809cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:34.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.230+0000 7faa10115700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa0809d8e0 con 0x7faa08133420 2023-09-30T22:34:34.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.230+0000 7faa10115700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa0811f0d0 con 0x7faa08134260 2023-09-30T22:34:34.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.230+0000 7faa10115700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faa0812d990 con 0x7faa08133e30 2023-09-30T22:34:34.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.230+0000 7faa0e6b2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faa08134260 0x7faa0809cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:34.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.230+0000 7faa0e6b2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faa08134260 0x7faa0809cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41124/0 (socket says 172.21.15.16:41124) 2023-09-30T22:34:34.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.230+0000 7faa0e6b2700 1 -- 172.21.15.16:0/1482059197 learned_addr learned my addr 172.21.15.16:0/1482059197 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:34.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.230+0000 7faa0deb1700 1 --2- 172.21.15.16:0/1482059197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa08133420 0x7faa0809c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:34.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.231+0000 7faa0d6b0700 1 --2- 172.21.15.16:0/1482059197 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faa08133e30 0x7faa0809c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:34.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.231+0000 7faa0e6b2700 1 -- 172.21.15.16:0/1482059197 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faa08133e30 msgr2=0x7faa0809c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:34.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.231+0000 7faa0e6b2700 1 --2- 172.21.15.16:0/1482059197 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faa08133e30 0x7faa0809c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:34.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.231+0000 7faa0e6b2700 1 -- 172.21.15.16:0/1482059197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa08133420 msgr2=0x7faa0809c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:34.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.231+0000 7faa0e6b2700 1 --2- 172.21.15.16:0/1482059197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa08133420 0x7faa0809c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:34.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.231+0000 7faa0e6b2700 1 -- 172.21.15.16:0/1482059197 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f80094a0 con 0x7faa08134260 2023-09-30T22:34:34.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.231+0000 7faa0e6b2700 1 --2- 172.21.15.16:0/1482059197 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faa08134260 0x7faa0809cf00 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7faa0000efc0 tx=0x7faa0000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:34.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.231+0000 7fa9fe7fc700 1 -- 172.21.15.16:0/1482059197 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa00003c20 con 0x7faa08134260 2023-09-30T22:34:34.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.231+0000 7fa9fe7fc700 1 -- 172.21.15.16:0/1482059197 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa0000f040 con 0x7faa08134260 2023-09-30T22:34:34.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.231+0000 7fa9fe7fc700 1 -- 172.21.15.16:0/1482059197 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa00009e30 con 0x7faa08134260 2023-09-30T22:34:34.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.232+0000 7faa10115700 1 -- 172.21.15.16:0/1482059197 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa08131400 con 0x7faa08134260 2023-09-30T22:34:34.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.232+0000 7faa10115700 1 -- 172.21.15.16:0/1482059197 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa0809dd40 con 0x7faa08134260 2023-09-30T22:34:34.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.233+0000 7faa10115700 1 -- 172.21.15.16:0/1482059197 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa08074ea0 con 0x7faa08134260 2023-09-30T22:34:34.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.233+0000 7fa9fe7fc700 1 -- 172.21.15.16:0/1482059197 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7faa0000c930 con 0x7faa08134260 2023-09-30T22:34:34.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.234+0000 7fa9fe7fc700 1 -- 172.21.15.16:0/1482059197 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7faa0003f9e0 con 0x7faa08134260 2023-09-30T22:34:34.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.239+0000 7fa9fe7fc700 1 -- 172.21.15.16:0/1482059197 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7faa00013db0 con 0x7faa08134260 2023-09-30T22:34:34.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.273+0000 7fbae6320700 1 Processor -- start 2023-09-30T22:34:34.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.273+0000 7fbae6320700 1 -- start start 2023-09-30T22:34:34.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.274+0000 7fbae6320700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbae009e1d0 0x7fbae0135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:34.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.274+0000 7fbae6320700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbae0135e50 0x7fbae0138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:34.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.274+0000 7fbae6320700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbae009f130 con 0x7fbae009d6f0 2023-09-30T22:34:34.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.274+0000 7fbae6320700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbae009f520 con 0x7fbae0135e50 2023-09-30T22:34:34.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.274+0000 7fbae6320700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbae009e930 con 0x7fbae009e1d0 2023-09-30T22:34:34.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.274+0000 7fbae48bd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbae0135e50 0x7fbae0138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:34.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.274+0000 7fbadffff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fbae009d6f0 0x7fbae009dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51138/0 (socket says 172.21.15.16:51138) 2023-09-30T22:34:34.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.274+0000 7fbae48bd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbae0135e50 0x7fbae0138230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40200/0 (socket says 172.21.15.16:40200) 2023-09-30T22:34:34.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.274+0000 7fbadffff700 1 -- 172.21.15.16:0/657288288 learned_addr learned my addr 172.21.15.16:0/657288288 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:34:34.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.275+0000 7fbadf7fe700 1 --2- 172.21.15.16:0/657288288 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbae009e1d0 0x7fbae0135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:34.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.275+0000 7fbae48bd700 1 -- 172.21.15.16:0/657288288 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbae009e1d0 msgr2=0x7fbae0135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:34.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.275+0000 7fbae48bd700 1 --2- 172.21.15.16:0/657288288 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbae009e1d0 0x7fbae0135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:34.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.275+0000 7fbae48bd700 1 -- 172.21.15.16:0/657288288 >> v1:172.21.15.16:6789/0 conn(0x7fbae009d6f0 legacy=0x7fbae009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:34.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.275+0000 7fbae48bd700 1 -- 172.21.15.16:0/657288288 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbae009b2d0 con 0x7fbae0135e50 2023-09-30T22:34:34.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.275+0000 7fbadf7fe700 1 --2- 172.21.15.16:0/657288288 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbae009e1d0 0x7fbae0135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:34:34.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.275+0000 7fbae48bd700 1 --2- 172.21.15.16:0/657288288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbae0135e50 0x7fbae0138230 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fbad8009dc0 tx=0x7fbad800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcaaab279eee6467 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:34.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.275+0000 7fbadeffd700 1 -- 172.21.15.16:0/657288288 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad800e070 con 0x7fbae0135e50 2023-09-30T22:34:34.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.275+0000 7fbadeffd700 1 -- 172.21.15.16:0/657288288 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbad800bb90 con 0x7fbae0135e50 2023-09-30T22:34:34.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.275+0000 7fbadeffd700 1 -- 172.21.15.16:0/657288288 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad8012430 con 0x7fbae0135e50 2023-09-30T22:34:34.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.276+0000 7fbae6320700 1 -- 172.21.15.16:0/657288288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbae0135e50 msgr2=0x7fbae0138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:34.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.276+0000 7fbae6320700 1 --2- 172.21.15.16:0/657288288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbae0135e50 0x7fbae0138230 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fbad8009dc0 tx=0x7fbad800b330 comp rx=0 tx=0).stop 2023-09-30T22:34:34.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.276+0000 7fbae6320700 1 -- 172.21.15.16:0/657288288 shutdown_connections 2023-09-30T22:34:34.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.276+0000 7fbae6320700 1 --2- 172.21.15.16:0/657288288 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbae0135e50 0x7fbae0138230 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:34.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.276+0000 7fbae6320700 1 --2- 172.21.15.16:0/657288288 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbae009e1d0 0x7fbae0135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:34.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.276+0000 7fbae6320700 1 -- 172.21.15.16:0/657288288 >> 172.21.15.16:0/657288288 conn(0x7fbae0066f50 msgr2=0x7fbae009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:34.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.276+0000 7fbae6320700 1 -- 172.21.15.16:0/657288288 shutdown_connections 2023-09-30T22:34:34.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.276+0000 7fbae6320700 1 -- 172.21.15.16:0/657288288 wait complete. 2023-09-30T22:34:34.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.277+0000 7fbae6320700 1 Processor -- start 2023-09-30T22:34:34.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.277+0000 7fbae6320700 1 -- start start 2023-09-30T22:34:34.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.278+0000 7fbae6320700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbae009d6f0 0x7fbae0120880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:34.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.278+0000 7fbae6320700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbae009e1d0 0x7fbae0120dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:34.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.278+0000 7fbae6320700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbae0135e50 0x7fbae0121300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:34.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.278+0000 7fbae6320700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbae0122c80 con 0x7fbae0135e50 2023-09-30T22:34:34.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.278+0000 7fbae6320700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbae01225a0 con 0x7fbae009e1d0 2023-09-30T22:34:34.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.278+0000 7fbae6320700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbae0135810 con 0x7fbae009d6f0 2023-09-30T22:34:34.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.278+0000 7fbadffff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbae009d6f0 0x7fbae0120880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:34.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.278+0000 7fbadffff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbae009d6f0 0x7fbae0120880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:53004/0 (socket says 172.21.15.16:53004) 2023-09-30T22:34:34.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.278+0000 7fbae48bd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbae0135e50 0x7fbae0121300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:34.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.278+0000 7fbadffff700 1 -- 172.21.15.16:0/2781382091 learned_addr learned my addr 172.21.15.16:0/2781382091 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:34.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.278+0000 7fbae48bd700 1 -- 172.21.15.16:0/2781382091 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbae009d6f0 msgr2=0x7fbae0120880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:34.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.278+0000 7fbae48bd700 1 --2- 172.21.15.16:0/2781382091 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbae009d6f0 0x7fbae0120880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:34.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.278+0000 7fbae48bd700 1 -- 172.21.15.16:0/2781382091 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbae009e1d0 msgr2=0x7fbae0120dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:34.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.278+0000 7fbae48bd700 1 --2- 172.21.15.16:0/2781382091 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbae009e1d0 0x7fbae0120dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:34.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.278+0000 7fbae48bd700 1 -- 172.21.15.16:0/2781382091 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad8009a70 con 0x7fbae0135e50 2023-09-30T22:34:34.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.279+0000 7fbae48bd700 1 --2- 172.21.15.16:0/2781382091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbae0135e50 0x7fbae0121300 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fbad8016000 tx=0x7fbad8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:34.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.279+0000 7fbadcff9700 1 -- 172.21.15.16:0/2781382091 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad8012430 con 0x7fbae0135e50 2023-09-30T22:34:34.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.279+0000 7fbadcff9700 1 -- 172.21.15.16:0/2781382091 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbad8020070 con 0x7fbae0135e50 2023-09-30T22:34:34.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.279+0000 7fbadcff9700 1 -- 172.21.15.16:0/2781382091 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbad8004280 con 0x7fbae0135e50 2023-09-30T22:34:34.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.279+0000 7fbae6320700 1 -- 172.21.15.16:0/2781382091 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbae0123e00 con 0x7fbae0135e50 2023-09-30T22:34:34.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.279+0000 7fbae6320700 1 -- 172.21.15.16:0/2781382091 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbae01229e0 con 0x7fbae0135e50 2023-09-30T22:34:34.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.280+0000 7fbae6320700 1 -- 172.21.15.16:0/2781382091 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbae0074ea0 con 0x7fbae0135e50 2023-09-30T22:34:34.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.280+0000 7fbadcff9700 1 -- 172.21.15.16:0/2781382091 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbad80043e0 con 0x7fbae0135e50 2023-09-30T22:34:34.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.282+0000 7fbadcff9700 1 -- 172.21.15.16:0/2781382091 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fbad8022070 con 0x7fbae0135e50 2023-09-30T22:34:34.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.290+0000 7fbadcff9700 1 -- 172.21.15.16:0/2781382091 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbad8027480 con 0x7fbae0135e50 2023-09-30T22:34:34.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.294+0000 7fa68aca2700 1 Processor -- start 2023-09-30T22:34:34.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.294+0000 7fa68aca2700 1 -- start start 2023-09-30T22:34:34.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.295+0000 7fa68aca2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa6841338c0 0x7fa684133c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:34.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.295+0000 7fa68aca2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa68409e560 0x7fa68413b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:34.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.295+0000 7fa68aca2700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa68406f720 con 0x7fa6841341d0 2023-09-30T22:34:34.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.295+0000 7fa68aca2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa684070c80 con 0x7fa68409e560 2023-09-30T22:34:34.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.295+0000 7fa68aca2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa68406fce0 con 0x7fa6841338c0 2023-09-30T22:34:34.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.295+0000 7fa683fff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa6841341d0 0x7fa68409de50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51140/0 (socket says 172.21.15.16:51140) 2023-09-30T22:34:34.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.295+0000 7fa683fff700 1 -- 172.21.15.16:0/192714856 learned_addr learned my addr 172.21.15.16:0/192714856 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:34:34.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.295+0000 7fa688a3e700 1 --2- 172.21.15.16:0/192714856 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa6841338c0 0x7fa684133c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:34.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.295+0000 7fa68923f700 1 --2- 172.21.15.16:0/192714856 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa68409e560 0x7fa68413b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:34.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.295+0000 7fa6837fe700 1 -- 172.21.15.16:0/192714856 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4256194911 0 0) 0x7fa68406f720 con 0x7fa6841341d0 2023-09-30T22:34:34.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.295+0000 7fa6837fe700 1 -- 172.21.15.16:0/192714856 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6740035b0 con 0x7fa6841341d0 2023-09-30T22:34:34.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.295+0000 7fa688a3e700 1 -- 172.21.15.16:0/192714856 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa68409e560 msgr2=0x7fa68413b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:34.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.295+0000 7fa688a3e700 1 --2- 172.21.15.16:0/192714856 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa68409e560 0x7fa68413b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:34.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.295+0000 7fa688a3e700 1 -- 172.21.15.16:0/192714856 >> v1:172.21.15.16:6789/0 conn(0x7fa6841341d0 legacy=0x7fa68409de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:34.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.295+0000 7fa688a3e700 1 -- 172.21.15.16:0/192714856 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6841336a0 con 0x7fa6841338c0 2023-09-30T22:34:34.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.296+0000 7fa688a3e700 1 --2- 172.21.15.16:0/192714856 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa6841338c0 0x7fa684133c90 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fa678013180 tx=0x7fa678013490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=901a2ea78d739246 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:34.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.296+0000 7fa6837fe700 1 -- 172.21.15.16:0/192714856 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa678016070 con 0x7fa6841338c0 2023-09-30T22:34:34.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.296+0000 7fa6837fe700 1 -- 172.21.15.16:0/192714856 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa678013d40 con 0x7fa6841338c0 2023-09-30T22:34:34.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.296+0000 7fa6837fe700 1 -- 172.21.15.16:0/192714856 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6780112f0 con 0x7fa6841338c0 2023-09-30T22:34:34.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.297+0000 7fa68aca2700 1 -- 172.21.15.16:0/192714856 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa6841338c0 msgr2=0x7fa684133c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:34.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.297+0000 7fa68aca2700 1 --2- 172.21.15.16:0/192714856 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa6841338c0 0x7fa684133c90 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fa678013180 tx=0x7fa678013490 comp rx=0 tx=0).stop 2023-09-30T22:34:34.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.297+0000 7fa68aca2700 1 -- 172.21.15.16:0/192714856 shutdown_connections 2023-09-30T22:34:34.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.297+0000 7fa68aca2700 1 --2- 172.21.15.16:0/192714856 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa68409e560 0x7fa68413b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:34.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.297+0000 7fa68aca2700 1 --2- 172.21.15.16:0/192714856 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa6841338c0 0x7fa684133c90 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:34.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.297+0000 7fa68aca2700 1 -- 172.21.15.16:0/192714856 >> 172.21.15.16:0/192714856 conn(0x7fa6840a2aa0 msgr2=0x7fa6840a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:34.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.297+0000 7fa68aca2700 1 -- 172.21.15.16:0/192714856 shutdown_connections 2023-09-30T22:34:34.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.297+0000 7fa68aca2700 1 -- 172.21.15.16:0/192714856 wait complete. 2023-09-30T22:34:34.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.298+0000 7fa68aca2700 1 Processor -- start 2023-09-30T22:34:34.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.298+0000 7fa68aca2700 1 -- start start 2023-09-30T22:34:34.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.299+0000 7fa68aca2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa68409e560 0x7fa68411b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:34.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.299+0000 7fa68aca2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa6841338c0 0x7fa68411bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:34.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.299+0000 7fa68aca2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6841341d0 0x7fa68411c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:34.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.299+0000 7fa68aca2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa68411ca40 con 0x7fa6841341d0 2023-09-30T22:34:34.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.299+0000 7fa68aca2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa68411f0b0 con 0x7fa68409e560 2023-09-30T22:34:34.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.299+0000 7fa68aca2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa68411e460 con 0x7fa6841338c0 2023-09-30T22:34:34.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.299+0000 7fa688a3e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa68409e560 0x7fa68411b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:34.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.299+0000 7fa688a3e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa68409e560 0x7fa68411b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41142/0 (socket says 172.21.15.16:41142) 2023-09-30T22:34:34.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.299+0000 7fa688a3e700 1 -- 172.21.15.16:0/175283680 learned_addr learned my addr 172.21.15.16:0/175283680 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:34.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.299+0000 7fa683fff700 1 --2- 172.21.15.16:0/175283680 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa6841338c0 0x7fa68411bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:34.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.299+0000 7fa68923f700 1 --2- 172.21.15.16:0/175283680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6841341d0 0x7fa68411c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:34.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.300+0000 7fa688a3e700 1 -- 172.21.15.16:0/175283680 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa6841338c0 msgr2=0x7fa68411bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:34.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.300+0000 7fa688a3e700 1 --2- 172.21.15.16:0/175283680 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa6841338c0 0x7fa68411bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:34.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.300+0000 7fa688a3e700 1 -- 172.21.15.16:0/175283680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6841341d0 msgr2=0x7fa68411c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:34.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.300+0000 7fa688a3e700 1 --2- 172.21.15.16:0/175283680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6841341d0 0x7fa68411c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:34.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.300+0000 7fa688a3e700 1 -- 172.21.15.16:0/175283680 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa678011d40 con 0x7fa68409e560 2023-09-30T22:34:34.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.300+0000 7fa688a3e700 1 --2- 172.21.15.16:0/175283680 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa68409e560 0x7fa68411b590 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fa678000c00 tx=0x7fa678004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:34.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.300+0000 7fa6817fa700 1 -- 172.21.15.16:0/175283680 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6780110d0 con 0x7fa68409e560 2023-09-30T22:34:34.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.300+0000 7fa6817fa700 1 -- 172.21.15.16:0/175283680 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa678011230 con 0x7fa68409e560 2023-09-30T22:34:34.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.300+0000 7fa68aca2700 1 -- 172.21.15.16:0/175283680 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa68411d240 con 0x7fa68409e560 2023-09-30T22:34:34.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.300+0000 7fa6817fa700 1 -- 172.21.15.16:0/175283680 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6780115e0 con 0x7fa68409e560 2023-09-30T22:34:34.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.300+0000 7fa68aca2700 1 -- 172.21.15.16:0/175283680 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa68411dfa0 con 0x7fa68409e560 2023-09-30T22:34:34.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.301+0000 7fa6817fa700 1 -- 172.21.15.16:0/175283680 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa6780036a0 con 0x7fa68409e560 2023-09-30T22:34:34.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.301+0000 7fa6817fa700 1 -- 172.21.15.16:0/175283680 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fa678048e20 con 0x7fa68409e560 2023-09-30T22:34:34.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.302+0000 7fa68aca2700 1 -- 172.21.15.16:0/175283680 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa68409d1f0 con 0x7fa68409e560 2023-09-30T22:34:34.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.308+0000 7fa6817fa700 1 -- 172.21.15.16:0/175283680 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa678024440 con 0x7fa68409e560 2023-09-30T22:34:34.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.367+0000 7f023fefd700 1 -- 172.21.15.16:0/3204183358 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0220005590 con 0x7f0238131810 2023-09-30T22:34:34.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.368+0000 7f022e7fc700 1 -- 172.21.15.16:0/3204183358 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0230011360 con 0x7f0238131810 2023-09-30T22:34:34.369 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:34:34.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.370+0000 7f023fefd700 1 -- 172.21.15.16:0/3204183358 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0238131810 msgr2=0x7f0238070090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:34.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.370+0000 7f023fefd700 1 --2- 172.21.15.16:0/3204183358 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0238131810 0x7f0238070090 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f0230002230 tx=0x7f023000a060 comp rx=0 tx=0).stop 2023-09-30T22:34:34.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.370+0000 7f023fefd700 1 -- 172.21.15.16:0/3204183358 shutdown_connections 2023-09-30T22:34:34.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.370+0000 7f023fefd700 1 --2- 172.21.15.16:0/3204183358 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0238131810 0x7f0238070090 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:34.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.370+0000 7f023fefd700 1 --2- 172.21.15.16:0/3204183358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0238099a40 0x7f023806f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:34.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.370+0000 7f023fefd700 1 --2- 172.21.15.16:0/3204183358 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f023809a3b0 0x7f023806fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:34.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.370+0000 7f023fefd700 1 -- 172.21.15.16:0/3204183358 >> 172.21.15.16:0/3204183358 conn(0x7f02380a2aa0 msgr2=0x7f023812e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:34.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.370+0000 7f023fefd700 1 -- 172.21.15.16:0/3204183358 shutdown_connections 2023-09-30T22:34:34.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.370+0000 7f023fefd700 1 -- 172.21.15.16:0/3204183358 wait complete. 2023-09-30T22:34:34.386 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:34:34.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.454+0000 7faa10115700 1 -- 172.21.15.16:0/1482059197 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7faa0811f490 con 0x7faa08134260 2023-09-30T22:34:34.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.455+0000 7fa9fe7fc700 1 -- 172.21.15.16:0/1482059197 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7faa00014e00 con 0x7faa08134260 2023-09-30T22:34:34.456 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:34:34.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.457+0000 7faa10115700 1 -- 172.21.15.16:0/1482059197 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faa08134260 msgr2=0x7faa0809cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:34.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.457+0000 7faa10115700 1 --2- 172.21.15.16:0/1482059197 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faa08134260 0x7faa0809cf00 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7faa0000efc0 tx=0x7faa0000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:34:34.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.457+0000 7faa10115700 1 -- 172.21.15.16:0/1482059197 shutdown_connections 2023-09-30T22:34:34.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.457+0000 7faa10115700 1 --2- 172.21.15.16:0/1482059197 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faa08134260 0x7faa0809cf00 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:34.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.457+0000 7faa10115700 1 --2- 172.21.15.16:0/1482059197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa08133420 0x7faa0809c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:34.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.457+0000 7faa10115700 1 --2- 172.21.15.16:0/1482059197 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faa08133e30 0x7faa0809c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:34.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.457+0000 7faa10115700 1 -- 172.21.15.16:0/1482059197 >> 172.21.15.16:0/1482059197 conn(0x7faa080a4210 msgr2=0x7faa080a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:34.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.457+0000 7faa10115700 1 -- 172.21.15.16:0/1482059197 shutdown_connections 2023-09-30T22:34:34.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.457+0000 7faa10115700 1 -- 172.21.15.16:0/1482059197 wait complete. 2023-09-30T22:34:34.472 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:34:34.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.524+0000 7fbae6320700 1 -- 172.21.15.16:0/2781382091 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fbae006c360 con 0x7fbae0135e50 2023-09-30T22:34:34.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.524+0000 7fbadcff9700 1 -- 172.21.15.16:0/2781382091 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbad802c070 con 0x7fbae0135e50 2023-09-30T22:34:34.526 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:34:34.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.526+0000 7fbae6320700 1 -- 172.21.15.16:0/2781382091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbae0135e50 msgr2=0x7fbae0121300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:34.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.526+0000 7fbae6320700 1 --2- 172.21.15.16:0/2781382091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbae0135e50 0x7fbae0121300 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fbad8016000 tx=0x7fbad8004ad0 comp rx=0 tx=0).stop 2023-09-30T22:34:34.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.526+0000 7fbae6320700 1 -- 172.21.15.16:0/2781382091 shutdown_connections 2023-09-30T22:34:34.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.526+0000 7fbae6320700 1 --2- 172.21.15.16:0/2781382091 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbae0135e50 0x7fbae0121300 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:34.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.526+0000 7fbae6320700 1 --2- 172.21.15.16:0/2781382091 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbae009d6f0 0x7fbae0120880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:34.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.526+0000 7fbae6320700 1 --2- 172.21.15.16:0/2781382091 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbae009e1d0 0x7fbae0120dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:34.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.526+0000 7fbae6320700 1 -- 172.21.15.16:0/2781382091 >> 172.21.15.16:0/2781382091 conn(0x7fbae0066f50 msgr2=0x7fbae006a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:34.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.527+0000 7fbae6320700 1 -- 172.21.15.16:0/2781382091 shutdown_connections 2023-09-30T22:34:34.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.527+0000 7fbae6320700 1 -- 172.21.15.16:0/2781382091 wait complete. 2023-09-30T22:34:34.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.536+0000 7fa68aca2700 1 -- 172.21.15.16:0/175283680 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa68411ed10 con 0x7fa68409e560 2023-09-30T22:34:34.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.537+0000 7fa6817fa700 1 -- 172.21.15.16:0/175283680 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa68411ed10 con 0x7fa68409e560 2023-09-30T22:34:34.538 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:34:34.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.539+0000 7fa68aca2700 1 -- 172.21.15.16:0/175283680 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa68409e560 msgr2=0x7fa68411b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:34.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.539+0000 7fa68aca2700 1 --2- 172.21.15.16:0/175283680 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa68409e560 0x7fa68411b590 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fa678000c00 tx=0x7fa678004750 comp rx=0 tx=0).stop 2023-09-30T22:34:34.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.539+0000 7fa68aca2700 1 -- 172.21.15.16:0/175283680 shutdown_connections 2023-09-30T22:34:34.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.539+0000 7fa68aca2700 1 --2- 172.21.15.16:0/175283680 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa68409e560 0x7fa68411b590 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:34.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.539+0000 7fa68aca2700 1 --2- 172.21.15.16:0/175283680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa6841341d0 0x7fa68411c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:34.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.539+0000 7fa68aca2700 1 --2- 172.21.15.16:0/175283680 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa6841338c0 0x7fa68411bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:34.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.539+0000 7fa68aca2700 1 -- 172.21.15.16:0/175283680 >> 172.21.15.16:0/175283680 conn(0x7fa6840a2aa0 msgr2=0x7fa68409dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:34.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.539+0000 7fa68aca2700 1 -- 172.21.15.16:0/175283680 shutdown_connections 2023-09-30T22:34:34.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:34.539+0000 7fa68aca2700 1 -- 172.21.15.16:0/175283680 wait complete. 2023-09-30T22:34:34.543 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:34:34.554 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:34:35.387 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:34:35.473 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:34:35.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.532+0000 7f759b96d700 1 Processor -- start 2023-09-30T22:34:35.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.533+0000 7f759b96d700 1 -- start start 2023-09-30T22:34:35.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.534+0000 7f759b96d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f759412c2f0 0x7f7594130040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:35.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.534+0000 7f759b96d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f759412cff0 0x7f7594137cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:35.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.534+0000 7f759b96d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f759409a510 con 0x7f759412cc20 2023-09-30T22:34:35.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.534+0000 7f759b96d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f759412c7d0 con 0x7f759412c2f0 2023-09-30T22:34:35.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.534+0000 7f759b96d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f759409c190 con 0x7f759412cff0 2023-09-30T22:34:35.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.534+0000 7f7599f0a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f759412cff0 0x7f7594137cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:35.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.534+0000 7f7598f08700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f759412c2f0 0x7f7594130040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:35.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.534+0000 7f7599709700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f759412cc20 0x7f759412f930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51146/0 (socket says 172.21.15.16:51146) 2023-09-30T22:34:35.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.534+0000 7f7599709700 1 -- 172.21.15.16:0/2113911389 learned_addr learned my addr 172.21.15.16:0/2113911389 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:34:35.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.534+0000 7f7599f0a700 1 --2- 172.21.15.16:0/2113911389 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f759412cff0 0x7f7594137cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41150/0 (socket says 172.21.15.16:41150) 2023-09-30T22:34:35.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.534+0000 7f7599f0a700 1 -- 172.21.15.16:0/2113911389 >> v2:172.21.15.16:3300/0 conn(0x7f759412c2f0 msgr2=0x7f7594130040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:35.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.534+0000 7f7599f0a700 1 --2- 172.21.15.16:0/2113911389 >> v2:172.21.15.16:3300/0 conn(0x7f759412c2f0 0x7f7594130040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:35.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.534+0000 7f7599f0a700 1 -- 172.21.15.16:0/2113911389 >> v1:172.21.15.16:6789/0 conn(0x7f759412cc20 legacy=0x7f759412f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:35.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.534+0000 7f7593fff700 1 -- 172.21.15.16:0/2113911389 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 531111140 0 0) 0x7f759409a510 con 0x7f759412cc20 2023-09-30T22:34:35.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.534+0000 7f7599f0a700 1 -- 172.21.15.16:0/2113911389 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f759409b540 con 0x7f759412cff0 2023-09-30T22:34:35.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.534+0000 7f7598f08700 1 --2- 172.21.15.16:0/2113911389 >> v2:172.21.15.16:3300/0 conn(0x7f759412c2f0 0x7f7594130040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:35.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.534+0000 7f7599f0a700 1 --2- 172.21.15.16:0/2113911389 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f759412cff0 0x7f7594137cd0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f758400bfa0 tx=0x7f758400d330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89dbbd44cb611c40 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:35.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.535+0000 7f7593fff700 1 -- 172.21.15.16:0/2113911389 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7584010070 con 0x7f759412cff0 2023-09-30T22:34:35.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.535+0000 7f7593fff700 1 -- 172.21.15.16:0/2113911389 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f758400da00 con 0x7f759412cff0 2023-09-30T22:34:35.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.535+0000 7f7593fff700 1 -- 172.21.15.16:0/2113911389 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758400ddb0 con 0x7f759412cff0 2023-09-30T22:34:35.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.535+0000 7f759b96d700 1 -- 172.21.15.16:0/2113911389 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f759412cff0 msgr2=0x7f7594137cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:35.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.535+0000 7f759b96d700 1 --2- 172.21.15.16:0/2113911389 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f759412cff0 0x7f7594137cd0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f758400bfa0 tx=0x7f758400d330 comp rx=0 tx=0).stop 2023-09-30T22:34:35.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.535+0000 7f759b96d700 1 -- 172.21.15.16:0/2113911389 shutdown_connections 2023-09-30T22:34:35.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.535+0000 7f759b96d700 1 --2- 172.21.15.16:0/2113911389 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f759412cff0 0x7f7594137cd0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:35.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.535+0000 7f759b96d700 1 --2- 172.21.15.16:0/2113911389 >> v2:172.21.15.16:3300/0 conn(0x7f759412c2f0 0x7f7594130040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:35.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.535+0000 7f759b96d700 1 -- 172.21.15.16:0/2113911389 >> 172.21.15.16:0/2113911389 conn(0x7f759409f470 msgr2=0x7f75940a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:35.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.535+0000 7f759b96d700 1 -- 172.21.15.16:0/2113911389 shutdown_connections 2023-09-30T22:34:35.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.535+0000 7f759b96d700 1 -- 172.21.15.16:0/2113911389 wait complete. 2023-09-30T22:34:35.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.537+0000 7f759b96d700 1 Processor -- start 2023-09-30T22:34:35.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.537+0000 7f759b96d700 1 -- start start 2023-09-30T22:34:35.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.538+0000 7f759b96d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f759412c2f0 0x7f75941c51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:35.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.538+0000 7f759b96d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f759412cc20 0x7f75941c5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:35.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.538+0000 7f759b96d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f759412cff0 0x7f75941c7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:35.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.538+0000 7f759b96d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7594129f10 con 0x7f759412c2f0 2023-09-30T22:34:35.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.538+0000 7f759b96d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7594072560 con 0x7f759412cc20 2023-09-30T22:34:35.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.538+0000 7f759b96d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f759408b8a0 con 0x7f759412cff0 2023-09-30T22:34:35.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.538+0000 7f7599f0a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f759412cff0 0x7f75941c7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:35.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.538+0000 7f7599709700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f759412c2f0 0x7f75941c51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:35.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.538+0000 7f7599f0a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f759412cff0 0x7f75941c7fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:53026/0 (socket says 172.21.15.16:53026) 2023-09-30T22:34:35.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.538+0000 7f7599f0a700 1 -- 172.21.15.16:0/1906660459 learned_addr learned my addr 172.21.15.16:0/1906660459 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:35.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.538+0000 7f7598f08700 1 --2- 172.21.15.16:0/1906660459 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f759412cc20 0x7f75941c5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:35.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.538+0000 7f7599f0a700 1 -- 172.21.15.16:0/1906660459 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f759412cc20 msgr2=0x7f75941c5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:35.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.538+0000 7f7599f0a700 1 --2- 172.21.15.16:0/1906660459 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f759412cc20 0x7f75941c5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:35.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.538+0000 7f7599f0a700 1 -- 172.21.15.16:0/1906660459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f759412c2f0 msgr2=0x7f75941c51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:35.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.538+0000 7f7599f0a700 1 --2- 172.21.15.16:0/1906660459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f759412c2f0 0x7f75941c51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:35.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.538+0000 7f7599f0a700 1 -- 172.21.15.16:0/1906660459 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75880094a0 con 0x7f759412cff0 2023-09-30T22:34:35.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.538+0000 7f7599709700 1 --2- 172.21.15.16:0/1906660459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f759412c2f0 0x7f75941c51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:35.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.538+0000 7f7599f0a700 1 --2- 172.21.15.16:0/1906660459 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f759412cff0 0x7f75941c7fc0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f7584003f30 tx=0x7f7584004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:35.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.539+0000 7f7591ffb700 1 -- 172.21.15.16:0/1906660459 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758400b450 con 0x7f759412cff0 2023-09-30T22:34:35.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.539+0000 7f7591ffb700 1 -- 172.21.15.16:0/1906660459 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75840040c0 con 0x7f759412cff0 2023-09-30T22:34:35.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.539+0000 7f7591ffb700 1 -- 172.21.15.16:0/1906660459 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7584004450 con 0x7f759412cff0 2023-09-30T22:34:35.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.539+0000 7f759b96d700 1 -- 172.21.15.16:0/1906660459 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f758400bc80 con 0x7f759412cff0 2023-09-30T22:34:35.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.539+0000 7f759b96d700 1 -- 172.21.15.16:0/1906660459 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f759408bb40 con 0x7f759412cff0 2023-09-30T22:34:35.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.540+0000 7f7591ffb700 1 -- 172.21.15.16:0/1906660459 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f75840045b0 con 0x7f759412cff0 2023-09-30T22:34:35.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.540+0000 7f759b96d700 1 -- 172.21.15.16:0/1906660459 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75940747f0 con 0x7f759412cff0 2023-09-30T22:34:35.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.540+0000 7f7591ffb700 1 -- 172.21.15.16:0/1906660459 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f75840418e0 con 0x7f759412cff0 2023-09-30T22:34:35.548 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:34:35.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.545+0000 7f7591ffb700 1 -- 172.21.15.16:0/1906660459 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7584023020 con 0x7f759412cff0 2023-09-30T22:34:35.555 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:34:35.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.624+0000 7f96b5d5a700 1 Processor -- start 2023-09-30T22:34:35.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.624+0000 7f96b5d5a700 1 -- start start 2023-09-30T22:34:35.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.625+0000 7f96b5d5a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f96b006e570 0x7f96b006e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:35.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.625+0000 7f96b5d5a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f96b009eca0 0x7f96b006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:35.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.625+0000 7f96b5d5a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96b0070ee0 con 0x7f96b0134610 2023-09-30T22:34:35.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.625+0000 7f96b5d5a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96b0071480 con 0x7f96b009eca0 2023-09-30T22:34:35.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.625+0000 7f96b5d5a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f96b006f3c0 con 0x7f96b006e570 2023-09-30T22:34:35.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.625+0000 7f96af7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f96b006e570 0x7f96b006e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:35.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.625+0000 7f96af7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f96b006e570 0x7f96b006e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:53040/0 (socket says 172.21.15.16:53040) 2023-09-30T22:34:35.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.625+0000 7f96af7fe700 1 -- 172.21.15.16:0/336166344 learned_addr learned my addr 172.21.15.16:0/336166344 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:35.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.625+0000 7f96aeffd700 1 --2- 172.21.15.16:0/336166344 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f96b009eca0 0x7f96b006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:35.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.625+0000 7f96aeffd700 1 -- 172.21.15.16:0/336166344 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f96b006e570 msgr2=0x7f96b006e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:35.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.625+0000 7f96aeffd700 1 --2- 172.21.15.16:0/336166344 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f96b006e570 0x7f96b006e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:35.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.625+0000 7f96aeffd700 1 -- 172.21.15.16:0/336166344 >> v1:172.21.15.16:6789/0 conn(0x7f96b0134610 legacy=0x7f96b0134ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:35.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.625+0000 7f96aeffd700 1 -- 172.21.15.16:0/336166344 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b0071b90 con 0x7f96b009eca0 2023-09-30T22:34:35.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.626+0000 7f96aeffd700 1 --2- 172.21.15.16:0/336166344 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f96b009eca0 0x7f96b006ee80 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f96a4009870 tx=0x7f96a4009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2869f454e6a8cc53 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:35.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.626+0000 7f96ae7fc700 1 -- 172.21.15.16:0/336166344 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a4011830 con 0x7f96b009eca0 2023-09-30T22:34:35.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.626+0000 7f96ae7fc700 1 -- 172.21.15.16:0/336166344 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96a4011990 con 0x7f96b009eca0 2023-09-30T22:34:35.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.626+0000 7f96ae7fc700 1 -- 172.21.15.16:0/336166344 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a4011d20 con 0x7f96b009eca0 2023-09-30T22:34:35.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.627+0000 7f96b5d5a700 1 -- 172.21.15.16:0/336166344 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f96b009eca0 msgr2=0x7f96b006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:35.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.627+0000 7f96b5d5a700 1 --2- 172.21.15.16:0/336166344 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f96b009eca0 0x7f96b006ee80 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f96a4009870 tx=0x7f96a4009b80 comp rx=0 tx=0).stop 2023-09-30T22:34:35.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.627+0000 7f96b5d5a700 1 -- 172.21.15.16:0/336166344 shutdown_connections 2023-09-30T22:34:35.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.627+0000 7f96b5d5a700 1 --2- 172.21.15.16:0/336166344 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f96b006e570 0x7f96b006e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:35.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.627+0000 7f96b5d5a700 1 --2- 172.21.15.16:0/336166344 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f96b009eca0 0x7f96b006ee80 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:35.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.627+0000 7f96b5d5a700 1 -- 172.21.15.16:0/336166344 >> 172.21.15.16:0/336166344 conn(0x7f96b00a2aa0 msgr2=0x7f96b00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:35.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.627+0000 7f96b5d5a700 1 -- 172.21.15.16:0/336166344 shutdown_connections 2023-09-30T22:34:35.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.627+0000 7f96b5d5a700 1 -- 172.21.15.16:0/336166344 wait complete. 2023-09-30T22:34:35.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.628+0000 7f96b5d5a700 1 Processor -- start 2023-09-30T22:34:35.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.629+0000 7f96b5d5a700 1 -- start start 2023-09-30T22:34:35.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.629+0000 7f96b5d5a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f96b006e570 0x7f96b013ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:35.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.629+0000 7f96b5d5a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f96b0134610 0x7f96b013f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:35.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.629+0000 7f96b5d5a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96b013f8c0 0x7f96b0132e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:35.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.629+0000 7f96b5d5a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b0140960 con 0x7f96b013f8c0 2023-09-30T22:34:35.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.629+0000 7f96b5d5a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96b013dcc0 con 0x7f96b0134610 2023-09-30T22:34:35.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.629+0000 7f96b5d5a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f96b00a2100 con 0x7f96b006e570 2023-09-30T22:34:35.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.629+0000 7f96af7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f96b006e570 0x7f96b013ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:35.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.629+0000 7f96af7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f96b006e570 0x7f96b013ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:53052/0 (socket says 172.21.15.16:53052) 2023-09-30T22:34:35.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.629+0000 7f96af7fe700 1 -- 172.21.15.16:0/194931101 learned_addr learned my addr 172.21.15.16:0/194931101 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:35.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.629+0000 7f96affff700 1 --2- 172.21.15.16:0/194931101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96b013f8c0 0x7f96b0132e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:35.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.630+0000 7f96aeffd700 1 --2- 172.21.15.16:0/194931101 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f96b0134610 0x7f96b013f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:35.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.630+0000 7f96affff700 1 -- 172.21.15.16:0/194931101 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f96b006e570 msgr2=0x7f96b013ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:35.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.630+0000 7f96affff700 1 --2- 172.21.15.16:0/194931101 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f96b006e570 0x7f96b013ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:35.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.630+0000 7f96affff700 1 -- 172.21.15.16:0/194931101 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f96b0134610 msgr2=0x7f96b013f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:35.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.630+0000 7f96affff700 1 --2- 172.21.15.16:0/194931101 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f96b0134610 0x7f96b013f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:35.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.630+0000 7f96affff700 1 -- 172.21.15.16:0/194931101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96a40094a0 con 0x7f96b013f8c0 2023-09-30T22:34:35.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.630+0000 7f96af7fe700 1 --2- 172.21.15.16:0/194931101 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f96b006e570 0x7f96b013ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:35.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.630+0000 7f96aeffd700 1 --2- 172.21.15.16:0/194931101 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f96b0134610 0x7f96b013f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:34:35.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.630+0000 7f96affff700 1 --2- 172.21.15.16:0/194931101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96b013f8c0 0x7f96b0132e70 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f96a800cb90 tx=0x7f96a800cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:35.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.630+0000 7f9697fff700 1 -- 172.21.15.16:0/194931101 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a8003c00 con 0x7f96b013f8c0 2023-09-30T22:34:35.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.630+0000 7f9697fff700 1 -- 172.21.15.16:0/194931101 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96a8003d60 con 0x7f96b013f8c0 2023-09-30T22:34:35.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.630+0000 7f96b5d5a700 1 -- 172.21.15.16:0/194931101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96b013e260 con 0x7f96b013f8c0 2023-09-30T22:34:35.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.630+0000 7f9697fff700 1 -- 172.21.15.16:0/194931101 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a800aaa0 con 0x7f96b013f8c0 2023-09-30T22:34:35.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.630+0000 7f96b5d5a700 1 -- 172.21.15.16:0/194931101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96b013caf0 con 0x7f96b013f8c0 2023-09-30T22:34:35.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.631+0000 7f9697fff700 1 -- 172.21.15.16:0/194931101 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f96a800ac00 con 0x7f96b013f8c0 2023-09-30T22:34:35.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.631+0000 7f9697fff700 1 -- 172.21.15.16:0/194931101 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f96a803f680 con 0x7f96b013f8c0 2023-09-30T22:34:35.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.632+0000 7f96b5d5a700 1 -- 172.21.15.16:0/194931101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96b013fd30 con 0x7f96b013f8c0 2023-09-30T22:34:35.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.637+0000 7f9697fff700 1 -- 172.21.15.16:0/194931101 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f96a80198f0 con 0x7f96b013f8c0 2023-09-30T22:34:35.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.710+0000 7f51c5e81700 1 Processor -- start 2023-09-30T22:34:35.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.710+0000 7f51c5e81700 1 -- start start 2023-09-30T22:34:35.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.711+0000 7f51c5e81700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f51c012cf60 0x7f51c006f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:35.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.711+0000 7f51c5e81700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f51c0130430 0x7f51c0137c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:35.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.711+0000 7f51c5e81700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51c012cd90 con 0x7f51c012f8b0 2023-09-30T22:34:35.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.711+0000 7f51c5e81700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51c012d330 con 0x7f51c012cf60 2023-09-30T22:34:35.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.711+0000 7f51c5e81700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f51c0071780 con 0x7f51c0130430 2023-09-30T22:34:35.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.711+0000 7f51beffd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f51c012f8b0 0x7f51c012fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51168/0 (socket says 172.21.15.16:51168) 2023-09-30T22:34:35.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.711+0000 7f51beffd700 1 -- 172.21.15.16:0/2730942351 learned_addr learned my addr 172.21.15.16:0/2730942351 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:34:35.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.711+0000 7f51bffff700 1 --2- 172.21.15.16:0/2730942351 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f51c0130430 0x7f51c0137c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:35.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.711+0000 7f51bf7fe700 1 --2- 172.21.15.16:0/2730942351 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f51c012cf60 0x7f51c006f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:35.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.711+0000 7f51bf7fe700 1 -- 172.21.15.16:0/2730942351 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f51c0130430 msgr2=0x7f51c0137c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:35.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.711+0000 7f51bf7fe700 1 --2- 172.21.15.16:0/2730942351 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f51c0130430 0x7f51c0137c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:35.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.711+0000 7f51bf7fe700 1 -- 172.21.15.16:0/2730942351 >> v1:172.21.15.16:6789/0 conn(0x7f51c012f8b0 legacy=0x7f51c012fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:35.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.711+0000 7f51bf7fe700 1 -- 172.21.15.16:0/2730942351 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c0071a40 con 0x7f51c012cf60 2023-09-30T22:34:35.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.712+0000 7f51bffff700 1 --2- 172.21.15.16:0/2730942351 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f51c0130430 0x7f51c0137c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:35.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.712+0000 7f51bf7fe700 1 --2- 172.21.15.16:0/2730942351 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f51c012cf60 0x7f51c006f5d0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f51b0009870 tx=0x7f51b0009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91191ba3ca6949c9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:35.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.712+0000 7f51be7fc700 1 -- 172.21.15.16:0/2730942351 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b0011830 con 0x7f51c012cf60 2023-09-30T22:34:35.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.712+0000 7f51be7fc700 1 -- 172.21.15.16:0/2730942351 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51b0011990 con 0x7f51c012cf60 2023-09-30T22:34:35.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.712+0000 7f51be7fc700 1 -- 172.21.15.16:0/2730942351 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b0011d20 con 0x7f51c012cf60 2023-09-30T22:34:35.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.713+0000 7f51c5e81700 1 -- 172.21.15.16:0/2730942351 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f51c012cf60 msgr2=0x7f51c006f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:35.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.713+0000 7f51c5e81700 1 --2- 172.21.15.16:0/2730942351 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f51c012cf60 0x7f51c006f5d0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f51b0009870 tx=0x7f51b0009b80 comp rx=0 tx=0).stop 2023-09-30T22:34:35.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.713+0000 7f51c5e81700 1 -- 172.21.15.16:0/2730942351 shutdown_connections 2023-09-30T22:34:35.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.713+0000 7f51c5e81700 1 --2- 172.21.15.16:0/2730942351 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f51c0130430 0x7f51c0137c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:35.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.713+0000 7f51c5e81700 1 --2- 172.21.15.16:0/2730942351 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f51c012cf60 0x7f51c006f5d0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:35.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.713+0000 7f51c5e81700 1 -- 172.21.15.16:0/2730942351 >> 172.21.15.16:0/2730942351 conn(0x7f51c00a2aa0 msgr2=0x7f51c00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:35.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.713+0000 7f51c5e81700 1 -- 172.21.15.16:0/2730942351 shutdown_connections 2023-09-30T22:34:35.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.713+0000 7f51c5e81700 1 -- 172.21.15.16:0/2730942351 wait complete. 2023-09-30T22:34:35.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.714+0000 7f51c5e81700 1 Processor -- start 2023-09-30T22:34:35.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.714+0000 7f51c5e81700 1 -- start start 2023-09-30T22:34:35.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.715+0000 7f51c5e81700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f51c012cf60 0x7f51c0099d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:35.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.715+0000 7f51c5e81700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f51c012f8b0 0x7f51c009a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:35.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.715+0000 7f51beffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f51c012f8b0 0x7f51c009a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:35.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.715+0000 7f51beffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f51c012f8b0 0x7f51c009a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41182/0 (socket says 172.21.15.16:41182) 2023-09-30T22:34:35.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.715+0000 7f51bf7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f51c012cf60 0x7f51c0099d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:35.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.715+0000 7f51c5e81700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51c0130430 0x7f51c009a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:35.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.715+0000 7f51bf7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f51c012cf60 0x7f51c0099d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:53060/0 (socket says 172.21.15.16:53060) 2023-09-30T22:34:35.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.715+0000 7f51c5e81700 1 -- 172.21.15.16:0/3675972268 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c0120010 con 0x7f51c0130430 2023-09-30T22:34:35.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.715+0000 7f51bf7fe700 1 -- 172.21.15.16:0/3675972268 learned_addr learned my addr 172.21.15.16:0/3675972268 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:35.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.715+0000 7f51c5e81700 1 -- 172.21.15.16:0/3675972268 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51c011f6a0 con 0x7f51c012f8b0 2023-09-30T22:34:35.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.715+0000 7f51c5e81700 1 -- 172.21.15.16:0/3675972268 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f51c011fd50 con 0x7f51c012cf60 2023-09-30T22:34:35.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.715+0000 7f51bffff700 1 --2- 172.21.15.16:0/3675972268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51c0130430 0x7f51c009a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:35.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.715+0000 7f51beffd700 1 -- 172.21.15.16:0/3675972268 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f51c012cf60 msgr2=0x7f51c0099d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:35.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.715+0000 7f51beffd700 1 --2- 172.21.15.16:0/3675972268 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f51c012cf60 0x7f51c0099d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:35.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.715+0000 7f51beffd700 1 -- 172.21.15.16:0/3675972268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51c0130430 msgr2=0x7f51c009a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:35.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.715+0000 7f51beffd700 1 --2- 172.21.15.16:0/3675972268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51c0130430 0x7f51c009a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:35.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.715+0000 7f51beffd700 1 -- 172.21.15.16:0/3675972268 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b800d040 con 0x7f51c012f8b0 2023-09-30T22:34:35.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.715+0000 7f51bf7fe700 1 --2- 172.21.15.16:0/3675972268 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f51c012cf60 0x7f51c0099d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:35.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.715+0000 7f51bffff700 1 --2- 172.21.15.16:0/3675972268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51c0130430 0x7f51c009a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:35.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.716+0000 7f51beffd700 1 --2- 172.21.15.16:0/3675972268 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f51c012f8b0 0x7f51c009a2c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f51b400c4d0 tx=0x7f51b400c7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:35.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.716+0000 7f51a7fff700 1 -- 172.21.15.16:0/3675972268 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b4005d70 con 0x7f51c012f8b0 2023-09-30T22:34:35.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.716+0000 7f51a7fff700 1 -- 172.21.15.16:0/3675972268 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51b400ce80 con 0x7f51c012f8b0 2023-09-30T22:34:35.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.716+0000 7f51c5e81700 1 -- 172.21.15.16:0/3675972268 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51b00094a0 con 0x7f51c012f8b0 2023-09-30T22:34:35.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.716+0000 7f51a7fff700 1 -- 172.21.15.16:0/3675972268 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b40072d0 con 0x7f51c012f8b0 2023-09-30T22:34:35.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.716+0000 7f51c5e81700 1 -- 172.21.15.16:0/3675972268 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51c012d7b0 con 0x7f51c012f8b0 2023-09-30T22:34:35.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.717+0000 7f51a7fff700 1 -- 172.21.15.16:0/3675972268 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f51b4007430 con 0x7f51c012f8b0 2023-09-30T22:34:35.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.717+0000 7f51a7fff700 1 -- 172.21.15.16:0/3675972268 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f51b403e940 con 0x7f51c012f8b0 2023-09-30T22:34:35.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.717+0000 7f51c5e81700 1 -- 172.21.15.16:0/3675972268 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51c0120250 con 0x7f51c012f8b0 2023-09-30T22:34:35.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.726+0000 7f51a7fff700 1 -- 172.21.15.16:0/3675972268 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f51b4017080 con 0x7f51c012f8b0 2023-09-30T22:34:35.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.756+0000 7fdb6e9b1700 1 Processor -- start 2023-09-30T22:34:35.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.756+0000 7fdb6e9b1700 1 -- start start 2023-09-30T22:34:35.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.757+0000 7fdb6e9b1700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdb6809d6f0 0x7fdb6809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:35.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.757+0000 7fdb6e9b1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdb6809e000 0x7fdb68130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:35.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.757+0000 7fdb6e9b1700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb6809f130 con 0x7fdb6809e410 2023-09-30T22:34:35.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.757+0000 7fdb6e9b1700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb6809f520 con 0x7fdb6809d6f0 2023-09-30T22:34:35.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.757+0000 7fdb6e9b1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdb6809e930 con 0x7fdb6809e000 2023-09-30T22:34:35.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.757+0000 7fdb677fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdb6809e000 0x7fdb68130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:35.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.757+0000 7fdb677fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdb6809e000 0x7fdb68130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:53062/0 (socket says 172.21.15.16:53062) 2023-09-30T22:34:35.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.757+0000 7fdb677fe700 1 -- 172.21.15.16:0/159209323 learned_addr learned my addr 172.21.15.16:0/159209323 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:35.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.757+0000 7fdb67fff700 1 --2- 172.21.15.16:0/159209323 >> v2:172.21.15.16:3300/0 conn(0x7fdb6809d6f0 0x7fdb6809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:35.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.757+0000 7fdb677fe700 1 -- 172.21.15.16:0/159209323 >> v2:172.21.15.16:3300/0 conn(0x7fdb6809d6f0 msgr2=0x7fdb6809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:35.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.757+0000 7fdb677fe700 1 --2- 172.21.15.16:0/159209323 >> v2:172.21.15.16:3300/0 conn(0x7fdb6809d6f0 0x7fdb6809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:35.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.758+0000 7fdb677fe700 1 -- 172.21.15.16:0/159209323 >> v1:172.21.15.16:6789/0 conn(0x7fdb6809e410 legacy=0x7fdb68130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:35.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.758+0000 7fdb66ffd700 1 -- 172.21.15.16:0/159209323 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 275032533 0 0) 0x7fdb6809f130 con 0x7fdb6809e410 2023-09-30T22:34:35.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.758+0000 7fdb677fe700 1 -- 172.21.15.16:0/159209323 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb6809b2d0 con 0x7fdb6809e000 2023-09-30T22:34:35.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.758+0000 7fdb67fff700 1 --2- 172.21.15.16:0/159209323 >> v2:172.21.15.16:3300/0 conn(0x7fdb6809d6f0 0x7fdb6809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:35.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.758+0000 7fdb677fe700 1 --2- 172.21.15.16:0/159209323 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdb6809e000 0x7fdb68130660 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c00bb90 tx=0x7fdb5c00bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=363d4f9e8d77da40 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:35.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.758+0000 7fdb66ffd700 1 -- 172.21.15.16:0/159209323 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb5c00dd30 con 0x7fdb6809e000 2023-09-30T22:34:35.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.758+0000 7fdb66ffd700 1 -- 172.21.15.16:0/159209323 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb5c00de90 con 0x7fdb6809e000 2023-09-30T22:34:35.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.758+0000 7fdb66ffd700 1 -- 172.21.15.16:0/159209323 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb5c010690 con 0x7fdb6809e000 2023-09-30T22:34:35.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.759+0000 7fdb6e9b1700 1 -- 172.21.15.16:0/159209323 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdb6809e000 msgr2=0x7fdb68130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:35.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.759+0000 7fdb6e9b1700 1 --2- 172.21.15.16:0/159209323 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdb6809e000 0x7fdb68130660 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c00bb90 tx=0x7fdb5c00bea0 comp rx=0 tx=0).stop 2023-09-30T22:34:35.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.759+0000 7fdb6e9b1700 1 -- 172.21.15.16:0/159209323 shutdown_connections 2023-09-30T22:34:35.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.759+0000 7fdb6e9b1700 1 --2- 172.21.15.16:0/159209323 >> v2:172.21.15.16:3300/0 conn(0x7fdb6809d6f0 0x7fdb6809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:35.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.759+0000 7fdb6e9b1700 1 --2- 172.21.15.16:0/159209323 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdb6809e000 0x7fdb68130660 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:35.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.759+0000 7fdb6e9b1700 1 -- 172.21.15.16:0/159209323 >> 172.21.15.16:0/159209323 conn(0x7fdb68066f50 msgr2=0x7fdb6809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:35.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.759+0000 7fdb6e9b1700 1 -- 172.21.15.16:0/159209323 shutdown_connections 2023-09-30T22:34:35.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.759+0000 7fdb6e9b1700 1 -- 172.21.15.16:0/159209323 wait complete. 2023-09-30T22:34:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.761+0000 7fdb6e9b1700 1 Processor -- start 2023-09-30T22:34:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.761+0000 7fdb6e9b1700 1 -- start start 2023-09-30T22:34:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.762+0000 7fdb6e9b1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb6809d6f0 0x7fdb68122430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.762+0000 7fdb6e9b1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdb6809e000 0x7fdb68122970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.762+0000 7fdb6e9b1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdb6809e410 0x7fdb68122eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.762+0000 7fdb6e9b1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb68123890 con 0x7fdb6809d6f0 2023-09-30T22:34:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.762+0000 7fdb6e9b1700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb68124150 con 0x7fdb6809e410 2023-09-30T22:34:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.762+0000 7fdb6e9b1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdb6809c700 con 0x7fdb6809e000 2023-09-30T22:34:35.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.762+0000 7fdb67fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb6809d6f0 0x7fdb68122430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.762+0000 7fdb67fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb6809d6f0 0x7fdb68122430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40270/0 (socket says 172.21.15.16:40270) 2023-09-30T22:34:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.762+0000 7fdb67fff700 1 -- 172.21.15.16:0/2703676873 learned_addr learned my addr 172.21.15.16:0/2703676873 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.762+0000 7fdb677fe700 1 --2- 172.21.15.16:0/2703676873 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdb6809e000 0x7fdb68122970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.762+0000 7fdb6cf4e700 1 --2- 172.21.15.16:0/2703676873 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdb6809e410 0x7fdb68122eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.762+0000 7fdb67fff700 1 -- 172.21.15.16:0/2703676873 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdb6809e000 msgr2=0x7fdb68122970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.762+0000 7fdb67fff700 1 --2- 172.21.15.16:0/2703676873 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdb6809e000 0x7fdb68122970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.762+0000 7fdb67fff700 1 -- 172.21.15.16:0/2703676873 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdb6809e410 msgr2=0x7fdb68122eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.762+0000 7fdb67fff700 1 --2- 172.21.15.16:0/2703676873 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdb6809e410 0x7fdb68122eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.762+0000 7fdb67fff700 1 -- 172.21.15.16:0/2703676873 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb580094a0 con 0x7fdb6809d6f0 2023-09-30T22:34:35.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.762+0000 7fdb67fff700 1 --2- 172.21.15.16:0/2703676873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb6809d6f0 0x7fdb68122430 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fdb5800ec10 tx=0x7fdb5800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:35.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.763+0000 7fdb64ff9700 1 -- 172.21.15.16:0/2703676873 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb5800c910 con 0x7fdb6809d6f0 2023-09-30T22:34:35.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.763+0000 7fdb64ff9700 1 -- 172.21.15.16:0/2703676873 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb5800ca70 con 0x7fdb6809d6f0 2023-09-30T22:34:35.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.763+0000 7fdb6e9b1700 1 -- 172.21.15.16:0/2703676873 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb5c00b7f0 con 0x7fdb6809d6f0 2023-09-30T22:34:35.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.763+0000 7fdb64ff9700 1 -- 172.21.15.16:0/2703676873 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb5800cde0 con 0x7fdb6809d6f0 2023-09-30T22:34:35.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.763+0000 7fdb6e9b1700 1 -- 172.21.15.16:0/2703676873 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb68124800 con 0x7fdb6809d6f0 2023-09-30T22:34:35.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.764+0000 7fdb6e9b1700 1 -- 172.21.15.16:0/2703676873 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb680a1f40 con 0x7fdb6809d6f0 2023-09-30T22:34:35.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.765+0000 7fdb64ff9700 1 -- 172.21.15.16:0/2703676873 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdb58013470 con 0x7fdb6809d6f0 2023-09-30T22:34:35.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.765+0000 7fdb64ff9700 1 -- 172.21.15.16:0/2703676873 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fdb58040050 con 0x7fdb6809d6f0 2023-09-30T22:34:35.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.767+0000 7f759b96d700 1 -- 172.21.15.16:0/1906660459 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f759406b250 con 0x7f759412cff0 2023-09-30T22:34:35.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.772+0000 7f7591ffb700 1 -- 172.21.15.16:0/1906660459 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f75840083a0 con 0x7f759412cff0 2023-09-30T22:34:35.773 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:34:35.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.773+0000 7fdb64ff9700 1 -- 172.21.15.16:0/2703676873 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdb58011760 con 0x7fdb6809d6f0 2023-09-30T22:34:35.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.774+0000 7f759b96d700 1 -- 172.21.15.16:0/1906660459 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f759412cff0 msgr2=0x7f75941c7fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:35.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.774+0000 7f759b96d700 1 --2- 172.21.15.16:0/1906660459 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f759412cff0 0x7f75941c7fc0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f7584003f30 tx=0x7f7584004ad0 comp rx=0 tx=0).stop 2023-09-30T22:34:35.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.774+0000 7f759b96d700 1 -- 172.21.15.16:0/1906660459 shutdown_connections 2023-09-30T22:34:35.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.774+0000 7f759b96d700 1 --2- 172.21.15.16:0/1906660459 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f759412cff0 0x7f75941c7fc0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:35.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.774+0000 7f759b96d700 1 --2- 172.21.15.16:0/1906660459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f759412c2f0 0x7f75941c51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:35.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.774+0000 7f759b96d700 1 --2- 172.21.15.16:0/1906660459 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f759412cc20 0x7f75941c5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:35.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.774+0000 7f759b96d700 1 -- 172.21.15.16:0/1906660459 >> 172.21.15.16:0/1906660459 conn(0x7f759409f470 msgr2=0x7f759412a120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:35.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.774+0000 7f759b96d700 1 -- 172.21.15.16:0/1906660459 shutdown_connections 2023-09-30T22:34:35.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.774+0000 7f759b96d700 1 -- 172.21.15.16:0/1906660459 wait complete. 2023-09-30T22:34:35.790 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:34:35.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.852+0000 7f96b5d5a700 1 -- 172.21.15.16:0/194931101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f96b0074ea0 con 0x7f96b013f8c0 2023-09-30T22:34:35.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.853+0000 7f9697fff700 1 -- 172.21.15.16:0/194931101 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f96a8019300 con 0x7f96b013f8c0 2023-09-30T22:34:35.854 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:34:35.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.856+0000 7f96b5d5a700 1 -- 172.21.15.16:0/194931101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96b013f8c0 msgr2=0x7f96b0132e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:35.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.856+0000 7f96b5d5a700 1 --2- 172.21.15.16:0/194931101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96b013f8c0 0x7f96b0132e70 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f96a800cb90 tx=0x7f96a800cea0 comp rx=0 tx=0).stop 2023-09-30T22:34:35.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.856+0000 7f96b5d5a700 1 -- 172.21.15.16:0/194931101 shutdown_connections 2023-09-30T22:34:35.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.856+0000 7f96b5d5a700 1 --2- 172.21.15.16:0/194931101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96b013f8c0 0x7f96b0132e70 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:35.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.856+0000 7f96b5d5a700 1 --2- 172.21.15.16:0/194931101 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f96b006e570 0x7f96b013ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:35.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.856+0000 7f96b5d5a700 1 --2- 172.21.15.16:0/194931101 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f96b0134610 0x7f96b013f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:35.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.857+0000 7f96b5d5a700 1 -- 172.21.15.16:0/194931101 >> 172.21.15.16:0/194931101 conn(0x7f96b00a2aa0 msgr2=0x7f96b00a0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:35.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.857+0000 7f96b5d5a700 1 -- 172.21.15.16:0/194931101 shutdown_connections 2023-09-30T22:34:35.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.857+0000 7f96b5d5a700 1 -- 172.21.15.16:0/194931101 wait complete. 2023-09-30T22:34:35.877 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:34:35.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.945+0000 7f51c5e81700 1 -- 172.21.15.16:0/3675972268 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f51c0074ea0 con 0x7f51c012f8b0 2023-09-30T22:34:35.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.945+0000 7f51a7fff700 1 -- 172.21.15.16:0/3675972268 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f51b401b3b0 con 0x7f51c012f8b0 2023-09-30T22:34:35.947 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:34:35.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.947+0000 7f51c5e81700 1 -- 172.21.15.16:0/3675972268 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f51c012f8b0 msgr2=0x7f51c009a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:35.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.947+0000 7f51c5e81700 1 --2- 172.21.15.16:0/3675972268 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f51c012f8b0 0x7f51c009a2c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f51b400c4d0 tx=0x7f51b400c7e0 comp rx=0 tx=0).stop 2023-09-30T22:34:35.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.947+0000 7f51c5e81700 1 -- 172.21.15.16:0/3675972268 shutdown_connections 2023-09-30T22:34:35.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.947+0000 7f51c5e81700 1 --2- 172.21.15.16:0/3675972268 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51c0130430 0x7f51c009a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:35.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.947+0000 7f51c5e81700 1 --2- 172.21.15.16:0/3675972268 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f51c012cf60 0x7f51c0099d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:35.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.947+0000 7f51c5e81700 1 --2- 172.21.15.16:0/3675972268 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f51c012f8b0 0x7f51c009a2c0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:35.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.947+0000 7f51c5e81700 1 -- 172.21.15.16:0/3675972268 >> 172.21.15.16:0/3675972268 conn(0x7f51c00a2aa0 msgr2=0x7f51c009e650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:35.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.948+0000 7f51c5e81700 1 -- 172.21.15.16:0/3675972268 shutdown_connections 2023-09-30T22:34:35.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.948+0000 7f51c5e81700 1 -- 172.21.15.16:0/3675972268 wait complete. 2023-09-30T22:34:35.962 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:34:35.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.993+0000 7fdb6e9b1700 1 -- 172.21.15.16:0/2703676873 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fdb68074ea0 con 0x7fdb6809d6f0 2023-09-30T22:34:35.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.994+0000 7fdb64ff9700 1 -- 172.21.15.16:0/2703676873 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdb58021020 con 0x7fdb6809d6f0 2023-09-30T22:34:35.995 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:34:35.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.996+0000 7fdb6e9b1700 1 -- 172.21.15.16:0/2703676873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb6809d6f0 msgr2=0x7fdb68122430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:35.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.996+0000 7fdb6e9b1700 1 --2- 172.21.15.16:0/2703676873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb6809d6f0 0x7fdb68122430 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fdb5800ec10 tx=0x7fdb5800ef20 comp rx=0 tx=0).stop 2023-09-30T22:34:35.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.996+0000 7fdb6e9b1700 1 -- 172.21.15.16:0/2703676873 shutdown_connections 2023-09-30T22:34:35.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.996+0000 7fdb6e9b1700 1 --2- 172.21.15.16:0/2703676873 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdb6809e410 0x7fdb68122eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:35.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.996+0000 7fdb6e9b1700 1 --2- 172.21.15.16:0/2703676873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdb6809d6f0 0x7fdb68122430 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:35.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.996+0000 7fdb6e9b1700 1 --2- 172.21.15.16:0/2703676873 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdb6809e000 0x7fdb68122970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:35.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.996+0000 7fdb6e9b1700 1 -- 172.21.15.16:0/2703676873 >> 172.21.15.16:0/2703676873 conn(0x7fdb68066f50 msgr2=0x7fdb68070f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:35.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.996+0000 7fdb6e9b1700 1 -- 172.21.15.16:0/2703676873 shutdown_connections 2023-09-30T22:34:35.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:35.996+0000 7fdb6e9b1700 1 -- 172.21.15.16:0/2703676873 wait complete. 2023-09-30T22:34:36.011 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:34:36.791 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:34:36.878 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:34:36.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.938+0000 7f2d79c5e700 1 Processor -- start 2023-09-30T22:34:36.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.938+0000 7f2d79c5e700 1 -- start start 2023-09-30T22:34:36.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.939+0000 7f2d79c5e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2d740a0a30 0x7f2d740a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:36.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.939+0000 7f2d79c5e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2d740a1340 0x7f2d74133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:36.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.939+0000 7f2d79c5e700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d7409edd0 con 0x7f2d74133fa0 2023-09-30T22:34:36.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.939+0000 7f2d79c5e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7409e1f0 con 0x7f2d740a0a30 2023-09-30T22:34:36.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.939+0000 7f2d79c5e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7409dc50 con 0x7f2d740a1340 2023-09-30T22:34:36.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.939+0000 7f2d737fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2d740a0a30 0x7f2d740a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:36.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.939+0000 7f2d737fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2d740a0a30 0x7f2d740a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41200/0 (socket says 172.21.15.16:41200) 2023-09-30T22:34:36.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.939+0000 7f2d737fe700 1 -- 172.21.15.16:0/2236301039 learned_addr learned my addr 172.21.15.16:0/2236301039 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:36.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.939+0000 7f2d73fff700 1 --1- 172.21.15.16:0/2236301039 >> v1:172.21.15.16:6789/0 conn(0x7f2d74133fa0 0x7f2d7413ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51186/0 (socket says 172.21.15.16:51186) 2023-09-30T22:34:36.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.939+0000 7f2d72ffd700 1 --2- 172.21.15.16:0/2236301039 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2d740a1340 0x7f2d74133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:36.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.939+0000 7f2d72ffd700 1 -- 172.21.15.16:0/2236301039 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2d740a0a30 msgr2=0x7f2d740a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:36.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.939+0000 7f2d727fc700 1 -- 172.21.15.16:0/2236301039 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1211818871 0 0) 0x7f2d7409edd0 con 0x7f2d74133fa0 2023-09-30T22:34:36.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.939+0000 7f2d72ffd700 1 --2- 172.21.15.16:0/2236301039 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2d740a0a30 0x7f2d740a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:36.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.939+0000 7f2d72ffd700 1 -- 172.21.15.16:0/2236301039 >> v1:172.21.15.16:6789/0 conn(0x7f2d74133fa0 legacy=0x7f2d7413ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:36.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.939+0000 7f2d72ffd700 1 -- 172.21.15.16:0/2236301039 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d740a1fe0 con 0x7f2d740a1340 2023-09-30T22:34:36.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.939+0000 7f2d737fe700 1 --2- 172.21.15.16:0/2236301039 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2d740a0a30 0x7f2d740a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:36.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.940+0000 7f2d72ffd700 1 --2- 172.21.15.16:0/2236301039 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2d740a1340 0x7f2d74133a60 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f2d68009870 tx=0x7f2d68009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d475118551f570b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:36.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.940+0000 7f2d727fc700 1 -- 172.21.15.16:0/2236301039 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d68011830 con 0x7f2d740a1340 2023-09-30T22:34:36.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.940+0000 7f2d727fc700 1 -- 172.21.15.16:0/2236301039 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d68011990 con 0x7f2d740a1340 2023-09-30T22:34:36.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.940+0000 7f2d727fc700 1 -- 172.21.15.16:0/2236301039 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d68011d20 con 0x7f2d740a1340 2023-09-30T22:34:36.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.941+0000 7f2d79c5e700 1 -- 172.21.15.16:0/2236301039 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2d740a1340 msgr2=0x7f2d74133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:36.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.941+0000 7f2d79c5e700 1 --2- 172.21.15.16:0/2236301039 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2d740a1340 0x7f2d74133a60 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f2d68009870 tx=0x7f2d68009b80 comp rx=0 tx=0).stop 2023-09-30T22:34:36.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.941+0000 7f2d79c5e700 1 -- 172.21.15.16:0/2236301039 shutdown_connections 2023-09-30T22:34:36.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.941+0000 7f2d79c5e700 1 --2- 172.21.15.16:0/2236301039 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2d740a0a30 0x7f2d740a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:36.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.941+0000 7f2d79c5e700 1 --2- 172.21.15.16:0/2236301039 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2d740a1340 0x7f2d74133a60 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:36.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.941+0000 7f2d79c5e700 1 -- 172.21.15.16:0/2236301039 >> 172.21.15.16:0/2236301039 conn(0x7f2d740a6550 msgr2=0x7f2d740a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:36.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.941+0000 7f2d79c5e700 1 -- 172.21.15.16:0/2236301039 shutdown_connections 2023-09-30T22:34:36.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.941+0000 7f2d79c5e700 1 -- 172.21.15.16:0/2236301039 wait complete. 2023-09-30T22:34:36.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.942+0000 7f2d79c5e700 1 Processor -- start 2023-09-30T22:34:36.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.942+0000 7f2d79c5e700 1 -- start start 2023-09-30T22:34:36.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.943+0000 7f2d79c5e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2d740a0a30 0x7f2d741c8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:36.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.943+0000 7f2d79c5e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d740a1340 0x7f2d741c87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:36.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.943+0000 7f2d79c5e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2d74133fa0 0x7f2d741c8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:36.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.943+0000 7f2d79c5e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d741244c0 con 0x7f2d740a1340 2023-09-30T22:34:36.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.943+0000 7f2d79c5e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d74124d90 con 0x7f2d74133fa0 2023-09-30T22:34:36.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.943+0000 7f2d79c5e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2d74122d60 con 0x7f2d740a0a30 2023-09-30T22:34:36.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.943+0000 7f2d737fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2d740a0a30 0x7f2d741c8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:36.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.943+0000 7f2d72ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d740a1340 0x7f2d741c87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:36.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.943+0000 7f2d737fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2d740a0a30 0x7f2d741c8280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:53094/0 (socket says 172.21.15.16:53094) 2023-09-30T22:34:36.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.943+0000 7f2d737fe700 1 -- 172.21.15.16:0/1213437000 learned_addr learned my addr 172.21.15.16:0/1213437000 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:36.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.943+0000 7f2d72ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d740a1340 0x7f2d741c87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40286/0 (socket says 172.21.15.16:40286) 2023-09-30T22:34:36.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.943+0000 7f2d73fff700 1 --2- 172.21.15.16:0/1213437000 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2d74133fa0 0x7f2d741c8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:36.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.943+0000 7f2d737fe700 1 -- 172.21.15.16:0/1213437000 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2d74133fa0 msgr2=0x7f2d741c8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:36.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.943+0000 7f2d737fe700 1 --2- 172.21.15.16:0/1213437000 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2d74133fa0 0x7f2d741c8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:36.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.943+0000 7f2d737fe700 1 -- 172.21.15.16:0/1213437000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d740a1340 msgr2=0x7f2d741c87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:36.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.943+0000 7f2d737fe700 1 --2- 172.21.15.16:0/1213437000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d740a1340 0x7f2d741c87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:36.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.943+0000 7f2d737fe700 1 -- 172.21.15.16:0/1213437000 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d64009550 con 0x7f2d740a0a30 2023-09-30T22:34:36.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.943+0000 7f2d72ffd700 1 --2- 172.21.15.16:0/1213437000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d740a1340 0x7f2d741c87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:36.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.943+0000 7f2d737fe700 1 --2- 172.21.15.16:0/1213437000 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2d740a0a30 0x7f2d741c8280 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f2d6400ebe0 tx=0x7f2d6400eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:36.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.944+0000 7f2d5bfff700 1 -- 172.21.15.16:0/1213437000 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6400c850 con 0x7f2d740a0a30 2023-09-30T22:34:36.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.944+0000 7f2d5bfff700 1 -- 172.21.15.16:0/1213437000 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d6400c9b0 con 0x7f2d740a0a30 2023-09-30T22:34:36.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.944+0000 7f2d79c5e700 1 -- 172.21.15.16:0/1213437000 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d680094a0 con 0x7f2d740a0a30 2023-09-30T22:34:36.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.944+0000 7f2d5bfff700 1 -- 172.21.15.16:0/1213437000 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6400cd40 con 0x7f2d740a0a30 2023-09-30T22:34:36.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.944+0000 7f2d79c5e700 1 -- 172.21.15.16:0/1213437000 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d740a25e0 con 0x7f2d740a0a30 2023-09-30T22:34:36.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.945+0000 7f2d5bfff700 1 -- 172.21.15.16:0/1213437000 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2d64013470 con 0x7f2d740a0a30 2023-09-30T22:34:36.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.945+0000 7f2d79c5e700 1 -- 172.21.15.16:0/1213437000 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d740a0220 con 0x7f2d740a0a30 2023-09-30T22:34:36.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.945+0000 7f2d5bfff700 1 -- 172.21.15.16:0/1213437000 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f2d6403f850 con 0x7f2d740a0a30 2023-09-30T22:34:36.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:36.950+0000 7f2d5bfff700 1 -- 172.21.15.16:0/1213437000 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2d640232f0 con 0x7f2d740a0a30 2023-09-30T22:34:36.963 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:34:37.012 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:34:37.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.020+0000 7f8d84d96700 1 Processor -- start 2023-09-30T22:34:37.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.020+0000 7f8d84d96700 1 -- start start 2023-09-30T22:34:37.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.021+0000 7f8d84d96700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8d8009d6f0 0x7f8d8009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:37.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.021+0000 7f8d84d96700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8d8009e000 0x7f8d801339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:37.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.021+0000 7f8d84d96700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8d8009e410 0x7f8d80133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:37.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.021+0000 7f8d84d96700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d8006b460 con 0x7f8d8009e000 2023-09-30T22:34:37.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.021+0000 7f8d84d96700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8009b120 con 0x7f8d8009e410 2023-09-30T22:34:37.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.021+0000 7f8d84d96700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8009a7e0 con 0x7f8d8009d6f0 2023-09-30T22:34:37.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.021+0000 7f8d7dd9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8d8009e000 0x7f8d801339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:37.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.021+0000 7f8d7dd9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8d8009e000 0x7f8d801339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40298/0 (socket says 172.21.15.16:40298) 2023-09-30T22:34:37.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.021+0000 7f8d7dd9b700 1 -- 172.21.15.16:0/422557506 learned_addr learned my addr 172.21.15.16:0/422557506 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:37.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.021+0000 7f8d7e59c700 1 --2- 172.21.15.16:0/422557506 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8d8009d6f0 0x7f8d8009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:37.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.022+0000 7f8d7dd9b700 1 -- 172.21.15.16:0/422557506 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8d8009d6f0 msgr2=0x7f8d8009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:37.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.022+0000 7f8d7dd9b700 1 --2- 172.21.15.16:0/422557506 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8d8009d6f0 0x7f8d8009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:37.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.022+0000 7f8d7dd9b700 1 -- 172.21.15.16:0/422557506 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8d8009e410 msgr2=0x7f8d80133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:37.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.022+0000 7f8d7dd9b700 1 --2- 172.21.15.16:0/422557506 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8d8009e410 0x7f8d80133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:37.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.022+0000 7f8d7dd9b700 1 -- 172.21.15.16:0/422557506 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d8009a500 con 0x7f8d8009e000 2023-09-30T22:34:37.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.022+0000 7f8d7dd9b700 1 --2- 172.21.15.16:0/422557506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d8009e000 0x7f8d801339f0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f8d740098f0 tx=0x7f8d74009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f290499473600d5f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:37.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.023+0000 7f8d7d59a700 1 -- 172.21.15.16:0/422557506 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d74011830 con 0x7f8d8009e000 2023-09-30T22:34:37.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.023+0000 7f8d7d59a700 1 -- 172.21.15.16:0/422557506 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d74011990 con 0x7f8d8009e000 2023-09-30T22:34:37.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.023+0000 7f8d7d59a700 1 -- 172.21.15.16:0/422557506 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d74011d20 con 0x7f8d8009e000 2023-09-30T22:34:37.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.023+0000 7f8d84d96700 1 -- 172.21.15.16:0/422557506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d8009e000 msgr2=0x7f8d801339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:37.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.023+0000 7f8d84d96700 1 --2- 172.21.15.16:0/422557506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d8009e000 0x7f8d801339f0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f8d740098f0 tx=0x7f8d74009c00 comp rx=0 tx=0).stop 2023-09-30T22:34:37.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.024+0000 7f8d84d96700 1 -- 172.21.15.16:0/422557506 shutdown_connections 2023-09-30T22:34:37.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.024+0000 7f8d84d96700 1 --2- 172.21.15.16:0/422557506 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8d8009e410 0x7f8d80133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:37.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.024+0000 7f8d84d96700 1 --2- 172.21.15.16:0/422557506 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8d8009d6f0 0x7f8d8009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:37.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.024+0000 7f8d84d96700 1 --2- 172.21.15.16:0/422557506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d8009e000 0x7f8d801339f0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:37.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.024+0000 7f8d84d96700 1 -- 172.21.15.16:0/422557506 >> 172.21.15.16:0/422557506 conn(0x7f8d80066f50 msgr2=0x7f8d8009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:37.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.024+0000 7f8d84d96700 1 -- 172.21.15.16:0/422557506 shutdown_connections 2023-09-30T22:34:37.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.024+0000 7f8d84d96700 1 -- 172.21.15.16:0/422557506 wait complete. 2023-09-30T22:34:37.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.026+0000 7f8d84d96700 1 Processor -- start 2023-09-30T22:34:37.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.026+0000 7f8d84d96700 1 -- start start 2023-09-30T22:34:37.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.026+0000 7f8d84d96700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8d8009d6f0 0x7f8d80122730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:37.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.026+0000 7f8d84d96700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d8009e000 0x7f8d80122c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:37.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.026+0000 7f8d84d96700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8d8009e410 0x7f8d801231b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:37.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.027+0000 7f8d84d96700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d801258a0 con 0x7f8d8009e000 2023-09-30T22:34:37.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.027+0000 7f8d84d96700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d80133250 con 0x7f8d8009d6f0 2023-09-30T22:34:37.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.027+0000 7f8d84d96700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8d801242b0 con 0x7f8d8009e410 2023-09-30T22:34:37.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.027+0000 7f8d7e59c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8d8009d6f0 0x7f8d80122730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:37.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.027+0000 7f8d7e59c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8d8009d6f0 0x7f8d80122730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41226/0 (socket says 172.21.15.16:41226) 2023-09-30T22:34:37.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.027+0000 7f8d7e59c700 1 -- 172.21.15.16:0/4103855139 learned_addr learned my addr 172.21.15.16:0/4103855139 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:37.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.027+0000 7f8d7dd9b700 1 --2- 172.21.15.16:0/4103855139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d8009e000 0x7f8d80122c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:37.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.027+0000 7f8d7ed9d700 1 --2- 172.21.15.16:0/4103855139 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8d8009e410 0x7f8d801231b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:37.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.027+0000 7f8d7e59c700 1 -- 172.21.15.16:0/4103855139 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8d8009e410 msgr2=0x7f8d801231b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:37.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.027+0000 7f8d7e59c700 1 --2- 172.21.15.16:0/4103855139 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8d8009e410 0x7f8d801231b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:37.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.027+0000 7f8d7e59c700 1 -- 172.21.15.16:0/4103855139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d8009e000 msgr2=0x7f8d80122c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:37.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.027+0000 7f8d7e59c700 1 --2- 172.21.15.16:0/4103855139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d8009e000 0x7f8d80122c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:37.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.027+0000 7f8d7e59c700 1 -- 172.21.15.16:0/4103855139 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d74009550 con 0x7f8d8009d6f0 2023-09-30T22:34:37.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.027+0000 7f8d7e59c700 1 --2- 172.21.15.16:0/4103855139 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8d8009d6f0 0x7f8d80122730 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f8d7000ea10 tx=0x7f8d7000ed20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:37.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.028+0000 7f8d6effd700 1 -- 172.21.15.16:0/4103855139 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d70009710 con 0x7f8d8009d6f0 2023-09-30T22:34:37.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.028+0000 7f8d6effd700 1 -- 172.21.15.16:0/4103855139 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d7000eeb0 con 0x7f8d8009d6f0 2023-09-30T22:34:37.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.028+0000 7f8d6effd700 1 -- 172.21.15.16:0/4103855139 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7000ca50 con 0x7f8d8009d6f0 2023-09-30T22:34:37.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.029+0000 7f8d84d96700 1 -- 172.21.15.16:0/4103855139 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d800a1d00 con 0x7f8d8009d6f0 2023-09-30T22:34:37.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.029+0000 7f8d84d96700 1 -- 172.21.15.16:0/4103855139 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d8009c710 con 0x7f8d8009d6f0 2023-09-30T22:34:37.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.030+0000 7f8d6effd700 1 -- 172.21.15.16:0/4103855139 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8d7000cbb0 con 0x7f8d8009d6f0 2023-09-30T22:34:37.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.030+0000 7f8d6effd700 1 -- 172.21.15.16:0/4103855139 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f8d7001a360 con 0x7f8d8009d6f0 2023-09-30T22:34:37.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.030+0000 7f8d84d96700 1 -- 172.21.15.16:0/4103855139 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d80074ea0 con 0x7f8d8009d6f0 2023-09-30T22:34:37.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.035+0000 7f8d6effd700 1 -- 172.21.15.16:0/4103855139 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8d70044060 con 0x7f8d8009d6f0 2023-09-30T22:34:37.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.105+0000 7f952c631700 1 Processor -- start 2023-09-30T22:34:37.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.105+0000 7f952c631700 1 -- start start 2023-09-30T22:34:37.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.106+0000 7f952c631700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f952406f5d0 0x7f952412ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:37.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.106+0000 7f952c631700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9524135c20 0x7f9524138000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:37.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.106+0000 7f952c631700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f952406fa40 con 0x7f952412cf60 2023-09-30T22:34:37.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.106+0000 7f952c631700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f952412cd90 con 0x7f9524135c20 2023-09-30T22:34:37.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.106+0000 7f952c631700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9524071780 con 0x7f952406f5d0 2023-09-30T22:34:37.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.106+0000 7f9529bcc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f952406f5d0 0x7f952412ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:37.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.106+0000 7f9529bcc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f952406f5d0 0x7f952412ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41234/0 (socket says 172.21.15.16:41234) 2023-09-30T22:34:37.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.106+0000 7f952a3cd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f952412cf60 0x7f952412f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51198/0 (socket says 172.21.15.16:51198) 2023-09-30T22:34:37.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.106+0000 7f9529bcc700 1 -- 172.21.15.16:0/2759058972 learned_addr learned my addr 172.21.15.16:0/2759058972 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:37.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.106+0000 7f952abce700 1 --2- 172.21.15.16:0/2759058972 >> v2:172.21.15.16:3300/0 conn(0x7f9524135c20 0x7f9524138000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:37.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.106+0000 7f952abce700 1 -- 172.21.15.16:0/2759058972 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f952406f5d0 msgr2=0x7f952412ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:37.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.106+0000 7f952abce700 1 --2- 172.21.15.16:0/2759058972 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f952406f5d0 0x7f952412ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:37.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.106+0000 7f952abce700 1 -- 172.21.15.16:0/2759058972 >> v1:172.21.15.16:6789/0 conn(0x7f952412cf60 legacy=0x7f952412f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:37.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.106+0000 7f952abce700 1 -- 172.21.15.16:0/2759058972 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9524071a40 con 0x7f9524135c20 2023-09-30T22:34:37.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.107+0000 7f952abce700 1 --2- 172.21.15.16:0/2759058972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9524135c20 0x7f9524138000 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f951c009d90 tx=0x7f951c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ba65b0d9eb0e740 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:37.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.107+0000 7f95293cb700 1 -- 172.21.15.16:0/2759058972 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951c00bd20 con 0x7f9524135c20 2023-09-30T22:34:37.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.107+0000 7f95293cb700 1 -- 172.21.15.16:0/2759058972 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f951c00be80 con 0x7f9524135c20 2023-09-30T22:34:37.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.107+0000 7f95293cb700 1 -- 172.21.15.16:0/2759058972 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951c0116a0 con 0x7f9524135c20 2023-09-30T22:34:37.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.108+0000 7f952c631700 1 -- 172.21.15.16:0/2759058972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9524135c20 msgr2=0x7f9524138000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:37.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.108+0000 7f952c631700 1 --2- 172.21.15.16:0/2759058972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9524135c20 0x7f9524138000 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f951c009d90 tx=0x7f951c00b330 comp rx=0 tx=0).stop 2023-09-30T22:34:37.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.108+0000 7f952c631700 1 -- 172.21.15.16:0/2759058972 shutdown_connections 2023-09-30T22:34:37.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.108+0000 7f952c631700 1 --2- 172.21.15.16:0/2759058972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9524135c20 0x7f9524138000 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:37.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.108+0000 7f952c631700 1 --2- 172.21.15.16:0/2759058972 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f952406f5d0 0x7f952412ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:37.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.108+0000 7f952c631700 1 -- 172.21.15.16:0/2759058972 >> 172.21.15.16:0/2759058972 conn(0x7f95240a2aa0 msgr2=0x7f95240a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:37.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.108+0000 7f952c631700 1 -- 172.21.15.16:0/2759058972 shutdown_connections 2023-09-30T22:34:37.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.108+0000 7f952c631700 1 -- 172.21.15.16:0/2759058972 wait complete. 2023-09-30T22:34:37.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.110+0000 7f952c631700 1 Processor -- start 2023-09-30T22:34:37.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.110+0000 7f952c631700 1 -- start start 2023-09-30T22:34:37.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.110+0000 7f952c631700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f952406f5d0 0x7f9524133a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:37.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.110+0000 7f952c631700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f952412cf60 0x7f9524133f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:37.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.110+0000 7f952c631700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9524135c20 0x7f95241c4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:37.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.110+0000 7f952c631700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952412c0a0 con 0x7f952406f5d0 2023-09-30T22:34:37.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.110+0000 7f952c631700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9524134480 con 0x7f952412cf60 2023-09-30T22:34:37.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.110+0000 7f952c631700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9524129fa0 con 0x7f9524135c20 2023-09-30T22:34:37.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.110+0000 7f952a3cd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f952406f5d0 0x7f9524133a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:37.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.111+0000 7f952a3cd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f952406f5d0 0x7f9524133a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40334/0 (socket says 172.21.15.16:40334) 2023-09-30T22:34:37.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.111+0000 7f952a3cd700 1 -- 172.21.15.16:0/3013933193 learned_addr learned my addr 172.21.15.16:0/3013933193 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:37.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.111+0000 7f9529bcc700 1 --2- 172.21.15.16:0/3013933193 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f952412cf60 0x7f9524133f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:37.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.111+0000 7f952abce700 1 --2- 172.21.15.16:0/3013933193 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9524135c20 0x7f95241c4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:37.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.111+0000 7f952a3cd700 1 -- 172.21.15.16:0/3013933193 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9524135c20 msgr2=0x7f95241c4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:37.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.111+0000 7f952a3cd700 1 --2- 172.21.15.16:0/3013933193 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9524135c20 0x7f95241c4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:37.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.111+0000 7f952a3cd700 1 -- 172.21.15.16:0/3013933193 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f952412cf60 msgr2=0x7f9524133f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:37.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.111+0000 7f952a3cd700 1 --2- 172.21.15.16:0/3013933193 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f952412cf60 0x7f9524133f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:37.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.111+0000 7f952a3cd700 1 -- 172.21.15.16:0/3013933193 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f951c009a70 con 0x7f952406f5d0 2023-09-30T22:34:37.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.111+0000 7f952abce700 1 --2- 172.21.15.16:0/3013933193 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9524135c20 0x7f95241c4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:34:37.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.111+0000 7f952a3cd700 1 --2- 172.21.15.16:0/3013933193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f952406f5d0 0x7f9524133a00 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f952000c700 tx=0x7f952000ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:37.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.111+0000 7f951affd700 1 -- 172.21.15.16:0/3013933193 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9520005b80 con 0x7f952406f5d0 2023-09-30T22:34:37.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.111+0000 7f951affd700 1 -- 172.21.15.16:0/3013933193 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9520005ce0 con 0x7f952406f5d0 2023-09-30T22:34:37.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.111+0000 7f952c631700 1 -- 172.21.15.16:0/3013933193 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f952409a890 con 0x7f952406f5d0 2023-09-30T22:34:37.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.111+0000 7f951affd700 1 -- 172.21.15.16:0/3013933193 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95200072d0 con 0x7f952406f5d0 2023-09-30T22:34:37.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.111+0000 7f952c631700 1 -- 172.21.15.16:0/3013933193 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f952409a4d0 con 0x7f952406f5d0 2023-09-30T22:34:37.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.112+0000 7f951affd700 1 -- 172.21.15.16:0/3013933193 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9520007430 con 0x7f952406f5d0 2023-09-30T22:34:37.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.113+0000 7f951affd700 1 -- 172.21.15.16:0/3013933193 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f952003eb40 con 0x7f952406f5d0 2023-09-30T22:34:37.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.113+0000 7f952c631700 1 -- 172.21.15.16:0/3013933193 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9524074ea0 con 0x7f952406f5d0 2023-09-30T22:34:37.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.122+0000 7f951affd700 1 -- 172.21.15.16:0/3013933193 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9520017080 con 0x7f952406f5d0 2023-09-30T22:34:37.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.158+0000 7f22f335e700 1 Processor -- start 2023-09-30T22:34:37.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.159+0000 7f22f335e700 1 -- start start 2023-09-30T22:34:37.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.159+0000 7f22f335e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f22ec131810 0x7f22ec131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:37.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.159+0000 7f22f335e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f22ec09a580 0x7f22ec139660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:37.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.159+0000 7f22f335e700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22ec1313c0 con 0x7f22ec099a40 2023-09-30T22:34:37.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.159+0000 7f22f335e700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22ec06e8c0 con 0x7f22ec131810 2023-09-30T22:34:37.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.159+0000 7f22f335e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22ec071d20 con 0x7f22ec09a580 2023-09-30T22:34:37.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.159+0000 7f22f18fb700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f22ec09a580 0x7f22ec139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:37.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.161+0000 7f22f18fb700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f22ec09a580 0x7f22ec139660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41238/0 (socket says 172.21.15.16:41238) 2023-09-30T22:34:37.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.161+0000 7f22f18fb700 1 -- 172.21.15.16:0/1794980110 learned_addr learned my addr 172.21.15.16:0/1794980110 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:37.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.161+0000 7f22f10fa700 1 --2- 172.21.15.16:0/1794980110 >> v2:172.21.15.16:3300/0 conn(0x7f22ec131810 0x7f22ec131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:37.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.161+0000 7f22f18fb700 1 -- 172.21.15.16:0/1794980110 >> v2:172.21.15.16:3300/0 conn(0x7f22ec131810 msgr2=0x7f22ec131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:37.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.161+0000 7f22f18fb700 1 --2- 172.21.15.16:0/1794980110 >> v2:172.21.15.16:3300/0 conn(0x7f22ec131810 0x7f22ec131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:37.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.161+0000 7f22f18fb700 1 -- 172.21.15.16:0/1794980110 >> v1:172.21.15.16:6789/0 conn(0x7f22ec099a40 legacy=0x7f22ec099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:37.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.161+0000 7f22f18fb700 1 -- 172.21.15.16:0/1794980110 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22ec0721a0 con 0x7f22ec09a580 2023-09-30T22:34:37.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.161+0000 7f22f18fb700 1 --2- 172.21.15.16:0/1794980110 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f22ec09a580 0x7f22ec139660 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f22e4009e50 tx=0x7f22e400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e2e4b3829e06aea server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:37.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.161+0000 7f22e3fff700 1 -- 172.21.15.16:0/1794980110 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e400e070 con 0x7f22ec09a580 2023-09-30T22:34:37.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.161+0000 7f22e3fff700 1 -- 172.21.15.16:0/1794980110 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22e400b910 con 0x7f22ec09a580 2023-09-30T22:34:37.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.161+0000 7f22e3fff700 1 -- 172.21.15.16:0/1794980110 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e400bcc0 con 0x7f22ec09a580 2023-09-30T22:34:37.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.161+0000 7f22f335e700 1 -- 172.21.15.16:0/1794980110 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f22ec09a580 msgr2=0x7f22ec139660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:37.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.161+0000 7f22f335e700 1 --2- 172.21.15.16:0/1794980110 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f22ec09a580 0x7f22ec139660 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f22e4009e50 tx=0x7f22e400b330 comp rx=0 tx=0).stop 2023-09-30T22:34:37.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.161+0000 7f22f335e700 1 -- 172.21.15.16:0/1794980110 shutdown_connections 2023-09-30T22:34:37.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.161+0000 7f22f335e700 1 --2- 172.21.15.16:0/1794980110 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f22ec09a580 0x7f22ec139660 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:37.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.161+0000 7f22f335e700 1 --2- 172.21.15.16:0/1794980110 >> v2:172.21.15.16:3300/0 conn(0x7f22ec131810 0x7f22ec131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:37.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.161+0000 7f22f335e700 1 -- 172.21.15.16:0/1794980110 >> 172.21.15.16:0/1794980110 conn(0x7f22ec0a2aa0 msgr2=0x7f22ec0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:37.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.161+0000 7f22f335e700 1 -- 172.21.15.16:0/1794980110 shutdown_connections 2023-09-30T22:34:37.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.161+0000 7f22f335e700 1 -- 172.21.15.16:0/1794980110 wait complete. 2023-09-30T22:34:37.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.162+0000 7f22f335e700 1 Processor -- start 2023-09-30T22:34:37.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.162+0000 7f22f335e700 1 -- start start 2023-09-30T22:34:37.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.163+0000 7f22f335e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22ec099a40 0x7f22ec1197c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:37.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.163+0000 7f22f335e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f22ec09a580 0x7f22ec119d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:37.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.163+0000 7f22f335e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f22ec131810 0x7f22ec11a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:37.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.163+0000 7f22f335e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ec11d0a0 con 0x7f22ec099a40 2023-09-30T22:34:37.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.163+0000 7f22f335e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22ec12cbb0 con 0x7f22ec09a580 2023-09-30T22:34:37.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.163+0000 7f22f335e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f22ec11c620 con 0x7f22ec131810 2023-09-30T22:34:37.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.163+0000 7f22f08f9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f22ec09a580 0x7f22ec119d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:37.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.163+0000 7f22f08f9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f22ec09a580 0x7f22ec119d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:41244/0 (socket says 172.21.15.16:41244) 2023-09-30T22:34:37.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.163+0000 7f22f08f9700 1 -- 172.21.15.16:0/34674029 learned_addr learned my addr 172.21.15.16:0/34674029 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:37.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.163+0000 7f22f18fb700 1 --2- 172.21.15.16:0/34674029 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f22ec131810 0x7f22ec11a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:37.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.163+0000 7f22f10fa700 1 --2- 172.21.15.16:0/34674029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22ec099a40 0x7f22ec1197c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:37.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.163+0000 7f22f08f9700 1 -- 172.21.15.16:0/34674029 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f22ec131810 msgr2=0x7f22ec11a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:37.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.163+0000 7f22f08f9700 1 --2- 172.21.15.16:0/34674029 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f22ec131810 0x7f22ec11a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:37.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.163+0000 7f22f08f9700 1 -- 172.21.15.16:0/34674029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22ec099a40 msgr2=0x7f22ec1197c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:37.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.163+0000 7f22f08f9700 1 --2- 172.21.15.16:0/34674029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22ec099a40 0x7f22ec1197c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:37.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.163+0000 7f22f08f9700 1 -- 172.21.15.16:0/34674029 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e4009b30 con 0x7f22ec09a580 2023-09-30T22:34:37.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.164+0000 7f22f08f9700 1 --2- 172.21.15.16:0/34674029 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f22ec09a580 0x7f22ec119d00 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f22dc00c4b0 tx=0x7f22dc00c7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:37.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.164+0000 7f22e1ffb700 1 -- 172.21.15.16:0/34674029 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc0052d0 con 0x7f22ec09a580 2023-09-30T22:34:37.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.164+0000 7f22e1ffb700 1 -- 172.21.15.16:0/34674029 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22dc005430 con 0x7f22ec09a580 2023-09-30T22:34:37.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.164+0000 7f22f335e700 1 -- 172.21.15.16:0/34674029 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22ec11b9b0 con 0x7f22ec09a580 2023-09-30T22:34:37.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.164+0000 7f22f335e700 1 -- 172.21.15.16:0/34674029 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22ec06eb00 con 0x7f22ec09a580 2023-09-30T22:34:37.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.165+0000 7f22e1ffb700 1 -- 172.21.15.16:0/34674029 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22dc0038a0 con 0x7f22ec09a580 2023-09-30T22:34:37.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.165+0000 7f22e1ffb700 1 -- 172.21.15.16:0/34674029 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f22dc003a00 con 0x7f22ec09a580 2023-09-30T22:34:37.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.166+0000 7f22f335e700 1 -- 172.21.15.16:0/34674029 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22cc0052e0 con 0x7f22ec09a580 2023-09-30T22:34:37.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.166+0000 7f22e1ffb700 1 -- 172.21.15.16:0/34674029 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f22dc03eb60 con 0x7f22ec09a580 2023-09-30T22:34:37.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.175+0000 7f22e1ffb700 1 -- 172.21.15.16:0/34674029 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f22dc019d20 con 0x7f22ec09a580 2023-09-30T22:34:37.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.200+0000 7f2d79c5e700 1 -- 172.21.15.16:0/1213437000 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f2d74074ea0 con 0x7f2d740a0a30 2023-09-30T22:34:37.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.200+0000 7f2d5bfff700 1 -- 172.21.15.16:0/1213437000 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2d64021070 con 0x7f2d740a0a30 2023-09-30T22:34:37.202 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:34:37.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.202+0000 7f2d79c5e700 1 -- 172.21.15.16:0/1213437000 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2d740a0a30 msgr2=0x7f2d741c8280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:37.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.202+0000 7f2d79c5e700 1 --2- 172.21.15.16:0/1213437000 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2d740a0a30 0x7f2d741c8280 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f2d6400ebe0 tx=0x7f2d6400eef0 comp rx=0 tx=0).stop 2023-09-30T22:34:37.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.203+0000 7f2d79c5e700 1 -- 172.21.15.16:0/1213437000 shutdown_connections 2023-09-30T22:34:37.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.203+0000 7f2d79c5e700 1 --2- 172.21.15.16:0/1213437000 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2d74133fa0 0x7f2d741c8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:37.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.203+0000 7f2d79c5e700 1 --2- 172.21.15.16:0/1213437000 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2d740a0a30 0x7f2d741c8280 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:37.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.203+0000 7f2d79c5e700 1 --2- 172.21.15.16:0/1213437000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d740a1340 0x7f2d741c87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:37.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.203+0000 7f2d79c5e700 1 -- 172.21.15.16:0/1213437000 >> 172.21.15.16:0/1213437000 conn(0x7f2d740a6550 msgr2=0x7f2d7411c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:37.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.203+0000 7f2d79c5e700 1 -- 172.21.15.16:0/1213437000 shutdown_connections 2023-09-30T22:34:37.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.203+0000 7f2d79c5e700 1 -- 172.21.15.16:0/1213437000 wait complete. 2023-09-30T22:34:37.218 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:34:37.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.295+0000 7f8d84d96700 1 -- 172.21.15.16:0/4103855139 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f8d8006c360 con 0x7f8d8009d6f0 2023-09-30T22:34:37.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.295+0000 7f8d6effd700 1 -- 172.21.15.16:0/4103855139 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8d70017020 con 0x7f8d8009d6f0 2023-09-30T22:34:37.298 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:34:37.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.298+0000 7f8d84d96700 1 -- 172.21.15.16:0/4103855139 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8d8009d6f0 msgr2=0x7f8d80122730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:37.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.298+0000 7f8d84d96700 1 --2- 172.21.15.16:0/4103855139 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8d8009d6f0 0x7f8d80122730 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f8d7000ea10 tx=0x7f8d7000ed20 comp rx=0 tx=0).stop 2023-09-30T22:34:37.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.298+0000 7f8d84d96700 1 -- 172.21.15.16:0/4103855139 shutdown_connections 2023-09-30T22:34:37.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.298+0000 7f8d84d96700 1 --2- 172.21.15.16:0/4103855139 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8d8009d6f0 0x7f8d80122730 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:37.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.298+0000 7f8d84d96700 1 --2- 172.21.15.16:0/4103855139 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8d8009e410 0x7f8d801231b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:37.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.298+0000 7f8d84d96700 1 --2- 172.21.15.16:0/4103855139 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d8009e000 0x7f8d80122c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:37.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.298+0000 7f8d84d96700 1 -- 172.21.15.16:0/4103855139 >> 172.21.15.16:0/4103855139 conn(0x7f8d80066f50 msgr2=0x7f8d800a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:37.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.299+0000 7f8d84d96700 1 -- 172.21.15.16:0/4103855139 shutdown_connections 2023-09-30T22:34:37.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.299+0000 7f8d84d96700 1 -- 172.21.15.16:0/4103855139 wait complete. 2023-09-30T22:34:37.318 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:34:37.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.355+0000 7f952c631700 1 -- 172.21.15.16:0/3013933193 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f95240713a0 con 0x7f952406f5d0 2023-09-30T22:34:37.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.357+0000 7f951affd700 1 -- 172.21.15.16:0/3013933193 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f952003ce60 con 0x7f952406f5d0 2023-09-30T22:34:37.357 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:34:37.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.358+0000 7f952c631700 1 -- 172.21.15.16:0/3013933193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f952406f5d0 msgr2=0x7f9524133a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:37.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.358+0000 7f952c631700 1 --2- 172.21.15.16:0/3013933193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f952406f5d0 0x7f9524133a00 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f952000c700 tx=0x7f952000ca10 comp rx=0 tx=0).stop 2023-09-30T22:34:37.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.358+0000 7f952c631700 1 -- 172.21.15.16:0/3013933193 shutdown_connections 2023-09-30T22:34:37.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.358+0000 7f952c631700 1 --2- 172.21.15.16:0/3013933193 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9524135c20 0x7f95241c4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:37.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.358+0000 7f952c631700 1 --2- 172.21.15.16:0/3013933193 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f952406f5d0 0x7f9524133a00 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:37.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.358+0000 7f952c631700 1 --2- 172.21.15.16:0/3013933193 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f952412cf60 0x7f9524133f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:37.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.358+0000 7f952c631700 1 -- 172.21.15.16:0/3013933193 >> 172.21.15.16:0/3013933193 conn(0x7f95240a2aa0 msgr2=0x7f952406e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:37.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.358+0000 7f952c631700 1 -- 172.21.15.16:0/3013933193 shutdown_connections 2023-09-30T22:34:37.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.358+0000 7f952c631700 1 -- 172.21.15.16:0/3013933193 wait complete. 2023-09-30T22:34:37.373 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:34:37.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.385+0000 7f22f335e700 1 -- 172.21.15.16:0/34674029 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f22cc005590 con 0x7f22ec09a580 2023-09-30T22:34:37.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.385+0000 7f22e1ffb700 1 -- 172.21.15.16:0/34674029 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f22dc0192c0 con 0x7f22ec09a580 2023-09-30T22:34:37.385 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:34:37.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.387+0000 7f22f335e700 1 -- 172.21.15.16:0/34674029 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f22ec09a580 msgr2=0x7f22ec119d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:37.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.387+0000 7f22f335e700 1 --2- 172.21.15.16:0/34674029 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f22ec09a580 0x7f22ec119d00 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f22dc00c4b0 tx=0x7f22dc00c7c0 comp rx=0 tx=0).stop 2023-09-30T22:34:37.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.388+0000 7f22f335e700 1 -- 172.21.15.16:0/34674029 shutdown_connections 2023-09-30T22:34:37.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.388+0000 7f22f335e700 1 --2- 172.21.15.16:0/34674029 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f22ec131810 0x7f22ec11a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:37.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.388+0000 7f22f335e700 1 --2- 172.21.15.16:0/34674029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22ec099a40 0x7f22ec1197c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:37.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.388+0000 7f22f335e700 1 --2- 172.21.15.16:0/34674029 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f22ec09a580 0x7f22ec119d00 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:37.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.388+0000 7f22f335e700 1 -- 172.21.15.16:0/34674029 >> 172.21.15.16:0/34674029 conn(0x7f22ec0a2aa0 msgr2=0x7f22ec12e680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:37.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.388+0000 7f22f335e700 1 -- 172.21.15.16:0/34674029 shutdown_connections 2023-09-30T22:34:37.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:37.388+0000 7f22f335e700 1 -- 172.21.15.16:0/34674029 wait complete. 2023-09-30T22:34:37.401 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:34:38.219 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:34:38.320 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:34:38.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.367+0000 7ffa04edc700 1 Processor -- start 2023-09-30T22:34:38.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.367+0000 7ffa04edc700 1 -- start start 2023-09-30T22:34:38.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.368+0000 7ffa04edc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa0009d6f0 0x7ffa0009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:38.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.368+0000 7ffa04edc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa0009e410 0x7ffa00130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:38.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.368+0000 7ffa04edc700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa0009f130 con 0x7ffa0009e000 2023-09-30T22:34:38.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.368+0000 7ffa04edc700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0009f520 con 0x7ffa0009d6f0 2023-09-30T22:34:38.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.368+0000 7ffa04edc700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0009e930 con 0x7ffa0009e410 2023-09-30T22:34:38.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.368+0000 7ff9fe59c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa0009d6f0 0x7ffa0009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:38.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.368+0000 7ff9fed9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa0009e410 0x7ffa00130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:38.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.368+0000 7ff9fe59c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa0009d6f0 0x7ffa0009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:51402/0 (socket says 172.21.15.16:51402) 2023-09-30T22:34:38.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.368+0000 7ff9fe59c700 1 -- 172.21.15.16:0/2874226134 learned_addr learned my addr 172.21.15.16:0/2874226134 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:38.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.369+0000 7ff9fd59a700 1 -- 172.21.15.16:0/2874226134 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2966587506 0 0) 0x7ffa0009f130 con 0x7ffa0009e000 2023-09-30T22:34:38.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.369+0000 7ff9fe59c700 1 -- 172.21.15.16:0/2874226134 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa0009e410 msgr2=0x7ffa00130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:38.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.369+0000 7ff9fe59c700 1 --2- 172.21.15.16:0/2874226134 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa0009e410 0x7ffa00130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:38.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.369+0000 7ff9fe59c700 1 -- 172.21.15.16:0/2874226134 >> v1:172.21.15.16:6789/0 conn(0x7ffa0009e000 legacy=0x7ffa00130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:38.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.369+0000 7ff9fe59c700 1 -- 172.21.15.16:0/2874226134 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa0009b2d0 con 0x7ffa0009d6f0 2023-09-30T22:34:38.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.369+0000 7ff9fed9d700 1 --2- 172.21.15.16:0/2874226134 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa0009e410 0x7ffa00130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:38.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.369+0000 7ff9fe59c700 1 --2- 172.21.15.16:0/2874226134 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa0009d6f0 0x7ffa0009dac0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7ff9f0009920 tx=0x7ff9f0009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e12b01979842b565 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:38.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.369+0000 7ff9fd59a700 1 -- 172.21.15.16:0/2874226134 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f0011830 con 0x7ffa0009d6f0 2023-09-30T22:34:38.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.369+0000 7ff9fd59a700 1 -- 172.21.15.16:0/2874226134 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9f0011990 con 0x7ffa0009d6f0 2023-09-30T22:34:38.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.369+0000 7ff9fd59a700 1 -- 172.21.15.16:0/2874226134 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f0011d20 con 0x7ffa0009d6f0 2023-09-30T22:34:38.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.370+0000 7ffa04edc700 1 -- 172.21.15.16:0/2874226134 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa0009d6f0 msgr2=0x7ffa0009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:38.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.370+0000 7ffa04edc700 1 --2- 172.21.15.16:0/2874226134 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa0009d6f0 0x7ffa0009dac0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7ff9f0009920 tx=0x7ff9f0009c30 comp rx=0 tx=0).stop 2023-09-30T22:34:38.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.370+0000 7ffa04edc700 1 -- 172.21.15.16:0/2874226134 shutdown_connections 2023-09-30T22:34:38.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.370+0000 7ffa04edc700 1 --2- 172.21.15.16:0/2874226134 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa0009e410 0x7ffa00130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:38.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.370+0000 7ffa04edc700 1 --2- 172.21.15.16:0/2874226134 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa0009d6f0 0x7ffa0009dac0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:38.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.370+0000 7ffa04edc700 1 -- 172.21.15.16:0/2874226134 >> 172.21.15.16:0/2874226134 conn(0x7ffa00066f50 msgr2=0x7ffa0009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:38.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.370+0000 7ffa04edc700 1 -- 172.21.15.16:0/2874226134 shutdown_connections 2023-09-30T22:34:38.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.370+0000 7ffa04edc700 1 -- 172.21.15.16:0/2874226134 wait complete. 2023-09-30T22:34:38.375 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:34:38.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.372+0000 7ffa04edc700 1 Processor -- start 2023-09-30T22:34:38.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.372+0000 7ffa04edc700 1 -- start start 2023-09-30T22:34:38.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.372+0000 7ffa04edc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa0009d6f0 0x7ffa0011e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:38.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.372+0000 7ffa04edc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa0009e000 0x7ffa0011e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:38.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.372+0000 7ffa04edc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa0009e410 0x7ffa0011ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:38.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.372+0000 7ffa04edc700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0009c770 con 0x7ffa0009e410 2023-09-30T22:34:38.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.372+0000 7ffa04edc700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa000a20d0 con 0x7ffa0009d6f0 2023-09-30T22:34:38.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.372+0000 7ffa04edc700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffa001204c0 con 0x7ffa0009e000 2023-09-30T22:34:38.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.373+0000 7ff9fdd9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa0009e000 0x7ffa0011e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:38.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.373+0000 7ff9fe59c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa0009d6f0 0x7ffa0011e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:38.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.373+0000 7ff9fed9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa0009e410 0x7ffa0011ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:38.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.373+0000 7ff9fe59c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa0009d6f0 0x7ffa0011e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:51418/0 (socket says 172.21.15.16:51418) 2023-09-30T22:34:38.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.373+0000 7ff9fdd9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa0009e000 0x7ffa0011e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:59152/0 (socket says 172.21.15.16:59152) 2023-09-30T22:34:38.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.373+0000 7ff9fe59c700 1 -- 172.21.15.16:0/1092688681 learned_addr learned my addr 172.21.15.16:0/1092688681 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:38.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.373+0000 7ff9fdd9b700 1 -- 172.21.15.16:0/1092688681 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa0009d6f0 msgr2=0x7ffa0011e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:38.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.373+0000 7ff9fdd9b700 1 --2- 172.21.15.16:0/1092688681 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa0009d6f0 0x7ffa0011e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:38.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.373+0000 7ff9fdd9b700 1 -- 172.21.15.16:0/1092688681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa0009e410 msgr2=0x7ffa0011ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:38.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.373+0000 7ff9fdd9b700 1 --2- 172.21.15.16:0/1092688681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa0009e410 0x7ffa0011ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:38.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.373+0000 7ff9fdd9b700 1 -- 172.21.15.16:0/1092688681 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f800bc80 con 0x7ffa0009e000 2023-09-30T22:34:38.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.373+0000 7ff9fe59c700 1 --2- 172.21.15.16:0/1092688681 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa0009d6f0 0x7ffa0011e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:38.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.373+0000 7ff9fed9d700 1 --2- 172.21.15.16:0/1092688681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa0009e410 0x7ffa0011ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:38.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.373+0000 7ff9fdd9b700 1 --2- 172.21.15.16:0/1092688681 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa0009e000 0x7ffa0011e590 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7ffa000a3e20 tx=0x7ff9e800c710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:38.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.373+0000 7ff9f6ffd700 1 -- 172.21.15.16:0/1092688681 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e80052e0 con 0x7ffa0009e000 2023-09-30T22:34:38.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.373+0000 7ff9f6ffd700 1 -- 172.21.15.16:0/1092688681 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9e8005460 con 0x7ffa0009e000 2023-09-30T22:34:38.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.373+0000 7ff9f6ffd700 1 -- 172.21.15.16:0/1092688681 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e80038c0 con 0x7ffa0009e000 2023-09-30T22:34:38.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.373+0000 7ffa04edc700 1 -- 172.21.15.16:0/1092688681 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9f0009550 con 0x7ffa0009e000 2023-09-30T22:34:38.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.373+0000 7ffa04edc700 1 -- 172.21.15.16:0/1092688681 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa001215d0 con 0x7ffa0009e000 2023-09-30T22:34:38.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.374+0000 7ff9f6ffd700 1 -- 172.21.15.16:0/1092688681 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff9e8003a20 con 0x7ffa0009e000 2023-09-30T22:34:38.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.374+0000 7ff9f6ffd700 1 -- 172.21.15.16:0/1092688681 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7ff9e803e970 con 0x7ffa0009e000 2023-09-30T22:34:38.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.374+0000 7ffa04edc700 1 -- 172.21.15.16:0/1092688681 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa0011ac20 con 0x7ffa0009e000 2023-09-30T22:34:38.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.383+0000 7ff9f6ffd700 1 -- 172.21.15.16:0/1092688681 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff9e8017070 con 0x7ffa0009e000 2023-09-30T22:34:38.402 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:34:38.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.491+0000 7fc7588f1700 1 Processor -- start 2023-09-30T22:34:38.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.491+0000 7fc7588f1700 1 -- start start 2023-09-30T22:34:38.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.492+0000 7fc7588f1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc75409e930 0x7fc75409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:38.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.492+0000 7fc7588f1700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc75409f240 0x7fc7541317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:38.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.492+0000 7fc7588f1700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc754070940 con 0x7fc75409f650 2023-09-30T22:34:38.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.492+0000 7fc7588f1700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc75406fd60 con 0x7fc75409f240 2023-09-30T22:34:38.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.492+0000 7fc7588f1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc75406fa80 con 0x7fc75409e930 2023-09-30T22:34:38.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.492+0000 7fc751d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc75409f240 0x7fc7541317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:38.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.492+0000 7fc752d9d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc75409f650 0x7fc754131d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58292/0 (socket says 172.21.15.16:58292) 2023-09-30T22:34:38.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.492+0000 7fc752d9d700 1 -- 172.21.15.16:0/454972962 learned_addr learned my addr 172.21.15.16:0/454972962 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:34:38.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.492+0000 7fc75259c700 1 --2- 172.21.15.16:0/454972962 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc75409e930 0x7fc75409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:38.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.493+0000 7fc751d9b700 1 -- 172.21.15.16:0/454972962 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc75409e930 msgr2=0x7fc75409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:38.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.493+0000 7fc751d9b700 1 --2- 172.21.15.16:0/454972962 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc75409e930 0x7fc75409ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:38.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.493+0000 7fc751d9b700 1 -- 172.21.15.16:0/454972962 >> v1:172.21.15.16:6789/0 conn(0x7fc75409f650 legacy=0x7fc754131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:38.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.493+0000 7fc751d9b700 1 -- 172.21.15.16:0/454972962 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc75409fb60 con 0x7fc75409f240 2023-09-30T22:34:38.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.493+0000 7fc751d9b700 1 --2- 172.21.15.16:0/454972962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc75409f240 0x7fc7541317f0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fc748009840 tx=0x7fc748009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b37122564348f5c2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:38.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.494+0000 7fc75159a700 1 -- 172.21.15.16:0/454972962 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc748011830 con 0x7fc75409f240 2023-09-30T22:34:38.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.494+0000 7fc75159a700 1 -- 172.21.15.16:0/454972962 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc748011990 con 0x7fc75409f240 2023-09-30T22:34:38.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.494+0000 7fc75159a700 1 -- 172.21.15.16:0/454972962 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc748011d20 con 0x7fc75409f240 2023-09-30T22:34:38.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.494+0000 7fc7588f1700 1 -- 172.21.15.16:0/454972962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc75409f240 msgr2=0x7fc7541317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:38.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.494+0000 7fc7588f1700 1 --2- 172.21.15.16:0/454972962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc75409f240 0x7fc7541317f0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7fc748009840 tx=0x7fc748009b50 comp rx=0 tx=0).stop 2023-09-30T22:34:38.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.495+0000 7fc7588f1700 1 -- 172.21.15.16:0/454972962 shutdown_connections 2023-09-30T22:34:38.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.495+0000 7fc7588f1700 1 --2- 172.21.15.16:0/454972962 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc75409e930 0x7fc75409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:38.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.495+0000 7fc7588f1700 1 --2- 172.21.15.16:0/454972962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc75409f240 0x7fc7541317f0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:38.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.495+0000 7fc7588f1700 1 -- 172.21.15.16:0/454972962 >> 172.21.15.16:0/454972962 conn(0x7fc7540a4210 msgr2=0x7fc7540a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:38.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.495+0000 7fc7588f1700 1 -- 172.21.15.16:0/454972962 shutdown_connections 2023-09-30T22:34:38.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.495+0000 7fc7588f1700 1 -- 172.21.15.16:0/454972962 wait complete. 2023-09-30T22:34:38.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.496+0000 7fc7588f1700 1 Processor -- start 2023-09-30T22:34:38.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.496+0000 7fc7588f1700 1 -- start start 2023-09-30T22:34:38.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.497+0000 7fc7588f1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc75409e930 0x7fc7541a3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:38.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.497+0000 7fc7588f1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc75409f240 0x7fc7541a43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:38.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.497+0000 7fc7588f1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc75409f650 0x7fc7541a48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:38.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.497+0000 7fc7588f1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7541a5310 con 0x7fc75409f240 2023-09-30T22:34:38.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.497+0000 7fc7588f1700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7541a59d0 con 0x7fc75409f650 2023-09-30T22:34:38.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.497+0000 7fc7588f1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc7541a6c20 con 0x7fc75409e930 2023-09-30T22:34:38.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.497+0000 7fc75259c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc75409e930 0x7fc7541a3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:38.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.497+0000 7fc751d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc75409f240 0x7fc7541a43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:38.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.497+0000 7fc75259c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc75409e930 0x7fc7541a3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:59172/0 (socket says 172.21.15.16:59172) 2023-09-30T22:34:38.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.497+0000 7fc75259c700 1 -- 172.21.15.16:0/1640703282 learned_addr learned my addr 172.21.15.16:0/1640703282 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:38.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.498+0000 7fc75259c700 1 -- 172.21.15.16:0/1640703282 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc75409f650 msgr2=0x7fc7541a48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:38.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.498+0000 7fc75259c700 1 --2- 172.21.15.16:0/1640703282 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc75409f650 0x7fc7541a48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:38.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.498+0000 7fc75259c700 1 -- 172.21.15.16:0/1640703282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc75409f240 msgr2=0x7fc7541a43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:38.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.498+0000 7fc75259c700 1 --2- 172.21.15.16:0/1640703282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc75409f240 0x7fc7541a43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:38.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.498+0000 7fc75259c700 1 -- 172.21.15.16:0/1640703282 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7480094a0 con 0x7fc75409e930 2023-09-30T22:34:38.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.498+0000 7fc751d9b700 1 --2- 172.21.15.16:0/1640703282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc75409f240 0x7fc7541a43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:38.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.498+0000 7fc75259c700 1 --2- 172.21.15.16:0/1640703282 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc75409e930 0x7fc7541a3e60 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc74400e9e0 tx=0x7fc74400ecf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:38.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.498+0000 7fc742ffd700 1 -- 172.21.15.16:0/1640703282 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc744009710 con 0x7fc75409e930 2023-09-30T22:34:38.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.498+0000 7fc742ffd700 1 -- 172.21.15.16:0/1640703282 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc74400c850 con 0x7fc75409e930 2023-09-30T22:34:38.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.498+0000 7fc7588f1700 1 -- 172.21.15.16:0/1640703282 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7540a0190 con 0x7fc75409e930 2023-09-30T22:34:38.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.498+0000 7fc742ffd700 1 -- 172.21.15.16:0/1640703282 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74400cbc0 con 0x7fc75409e930 2023-09-30T22:34:38.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.498+0000 7fc7588f1700 1 -- 172.21.15.16:0/1640703282 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7540a3630 con 0x7fc75409e930 2023-09-30T22:34:38.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.500+0000 7fc7588f1700 1 -- 172.21.15.16:0/1640703282 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc754071d20 con 0x7fc75409e930 2023-09-30T22:34:38.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.501+0000 7fc742ffd700 1 -- 172.21.15.16:0/1640703282 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc74400cd20 con 0x7fc75409e930 2023-09-30T22:34:38.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.501+0000 7fc742ffd700 1 -- 172.21.15.16:0/1640703282 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fc74403f8d0 con 0x7fc75409e930 2023-09-30T22:34:38.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.506+0000 7fc742ffd700 1 -- 172.21.15.16:0/1640703282 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc7440232f0 con 0x7fc75409e930 2023-09-30T22:34:38.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.553+0000 7ff217847700 1 Processor -- start 2023-09-30T22:34:38.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.553+0000 7ff217847700 1 -- start start 2023-09-30T22:34:38.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.554+0000 7ff217847700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff21009d6f0 0x7ff21009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:38.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.554+0000 7ff217847700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff21009e410 0x7ff210134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:38.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.554+0000 7ff217847700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff21009b120 con 0x7ff21009e000 2023-09-30T22:34:38.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.554+0000 7ff217847700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff21006b460 con 0x7ff21009e410 2023-09-30T22:34:38.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.554+0000 7ff217847700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff21009a7e0 con 0x7ff21009d6f0 2023-09-30T22:34:38.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.554+0000 7ff2155e3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff21009d6f0 0x7ff21009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:38.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.555+0000 7ff2155e3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff21009d6f0 0x7ff21009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:59180/0 (socket says 172.21.15.16:59180) 2023-09-30T22:34:38.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.555+0000 7ff2155e3700 1 -- 172.21.15.16:0/2349297236 learned_addr learned my addr 172.21.15.16:0/2349297236 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:38.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.555+0000 7ff214de2700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff21009e000 0x7ff2101339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58300/0 (socket says 172.21.15.16:58300) 2023-09-30T22:34:38.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.555+0000 7ff2155e3700 1 -- 172.21.15.16:0/2349297236 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff21009e410 msgr2=0x7ff210134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:38.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.555+0000 7ff2155e3700 1 --2- 172.21.15.16:0/2349297236 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff21009e410 0x7ff210134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:38.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.555+0000 7ff2155e3700 1 -- 172.21.15.16:0/2349297236 >> v1:172.21.15.16:6789/0 conn(0x7ff21009e000 legacy=0x7ff2101339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:38.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.555+0000 7ff207fff700 1 -- 172.21.15.16:0/2349297236 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 454767628 0 0) 0x7ff21009b120 con 0x7ff21009e000 2023-09-30T22:34:38.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.555+0000 7ff2155e3700 1 -- 172.21.15.16:0/2349297236 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff21009a500 con 0x7ff21009d6f0 2023-09-30T22:34:38.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.555+0000 7ff2155e3700 1 --2- 172.21.15.16:0/2349297236 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff21009d6f0 0x7ff21009dac0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7ff20c009950 tx=0x7ff20c009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdf9b3b7b128f1d7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:38.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.555+0000 7ff207fff700 1 -- 172.21.15.16:0/2349297236 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff20c009ea0 con 0x7ff21009d6f0 2023-09-30T22:34:38.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.555+0000 7ff207fff700 1 -- 172.21.15.16:0/2349297236 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff20c00ba20 con 0x7ff21009d6f0 2023-09-30T22:34:38.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.555+0000 7ff207fff700 1 -- 172.21.15.16:0/2349297236 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff20c00bdb0 con 0x7ff21009d6f0 2023-09-30T22:34:38.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.556+0000 7ff217847700 1 -- 172.21.15.16:0/2349297236 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff21009d6f0 msgr2=0x7ff21009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:38.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.556+0000 7ff217847700 1 --2- 172.21.15.16:0/2349297236 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff21009d6f0 0x7ff21009dac0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7ff20c009950 tx=0x7ff20c009c60 comp rx=0 tx=0).stop 2023-09-30T22:34:38.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.556+0000 7ff217847700 1 -- 172.21.15.16:0/2349297236 shutdown_connections 2023-09-30T22:34:38.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.556+0000 7ff217847700 1 --2- 172.21.15.16:0/2349297236 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff21009e410 0x7ff210134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:38.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.556+0000 7ff217847700 1 --2- 172.21.15.16:0/2349297236 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff21009d6f0 0x7ff21009dac0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:38.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.556+0000 7ff217847700 1 -- 172.21.15.16:0/2349297236 >> 172.21.15.16:0/2349297236 conn(0x7ff210066f50 msgr2=0x7ff21009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:38.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.557+0000 7ff217847700 1 -- 172.21.15.16:0/2349297236 shutdown_connections 2023-09-30T22:34:38.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.557+0000 7ff217847700 1 -- 172.21.15.16:0/2349297236 wait complete. 2023-09-30T22:34:38.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.558+0000 7ff217847700 1 Processor -- start 2023-09-30T22:34:38.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.558+0000 7ff217847700 1 -- start start 2023-09-30T22:34:38.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.559+0000 7ff217847700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff21009d6f0 0x7ff210122200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:38.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.559+0000 7ff217847700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff21009e000 0x7ff210122740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:38.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.559+0000 7ff217847700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff21009e410 0x7ff210122c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:38.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.559+0000 7ff217847700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff210125c90 con 0x7ff21009e000 2023-09-30T22:34:38.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.559+0000 7ff217847700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2101256f0 con 0x7ff21009e410 2023-09-30T22:34:38.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.559+0000 7ff217847700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff210123e20 con 0x7ff21009d6f0 2023-09-30T22:34:38.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.559+0000 7ff214de2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff21009e000 0x7ff210122740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:38.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.559+0000 7ff214de2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff21009e000 0x7ff210122740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56474/0 (socket says 172.21.15.16:56474) 2023-09-30T22:34:38.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.559+0000 7ff214de2700 1 -- 172.21.15.16:0/3919208384 learned_addr learned my addr 172.21.15.16:0/3919208384 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:38.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.559+0000 7ff215de4700 1 --2- 172.21.15.16:0/3919208384 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff21009e410 0x7ff210122c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:38.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.560+0000 7ff214de2700 1 -- 172.21.15.16:0/3919208384 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff21009d6f0 msgr2=0x7ff210122200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:38.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.560+0000 7ff214de2700 1 --2- 172.21.15.16:0/3919208384 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff21009d6f0 0x7ff210122200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:38.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.560+0000 7ff214de2700 1 -- 172.21.15.16:0/3919208384 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff21009e410 msgr2=0x7ff210122c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:38.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.560+0000 7ff214de2700 1 --2- 172.21.15.16:0/3919208384 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff21009e410 0x7ff210122c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:38.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.560+0000 7ff214de2700 1 -- 172.21.15.16:0/3919208384 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff20c009550 con 0x7ff21009e000 2023-09-30T22:34:38.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.560+0000 7ff215de4700 1 --2- 172.21.15.16:0/3919208384 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff21009e410 0x7ff210122c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:38.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.560+0000 7ff214de2700 1 --2- 172.21.15.16:0/3919208384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff21009e000 0x7ff210122740 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7ff20000c540 tx=0x7ff20000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:38.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.560+0000 7ff205ffb700 1 -- 172.21.15.16:0/3919208384 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff200004c80 con 0x7ff21009e000 2023-09-30T22:34:38.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.560+0000 7ff205ffb700 1 -- 172.21.15.16:0/3919208384 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff200004de0 con 0x7ff21009e000 2023-09-30T22:34:38.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.560+0000 7ff217847700 1 -- 172.21.15.16:0/3919208384 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2100a19d0 con 0x7ff21009e000 2023-09-30T22:34:38.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.560+0000 7ff205ffb700 1 -- 172.21.15.16:0/3919208384 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff200007310 con 0x7ff21009e000 2023-09-30T22:34:38.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.560+0000 7ff217847700 1 -- 172.21.15.16:0/3919208384 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff210126190 con 0x7ff21009e000 2023-09-30T22:34:38.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.561+0000 7ff205ffb700 1 -- 172.21.15.16:0/3919208384 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff200007470 con 0x7ff21009e000 2023-09-30T22:34:38.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.561+0000 7ff205ffb700 1 -- 172.21.15.16:0/3919208384 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7ff20003f400 con 0x7ff21009e000 2023-09-30T22:34:38.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.561+0000 7ff217847700 1 -- 172.21.15.16:0/3919208384 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff21011f030 con 0x7ff21009e000 2023-09-30T22:34:38.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.564+0000 7f5fcc97b700 1 Processor -- start 2023-09-30T22:34:38.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.564+0000 7f5fcc97b700 1 -- start start 2023-09-30T22:34:38.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.565+0000 7f5fcc97b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5fc809d6f0 0x7f5fc809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:38.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.565+0000 7f5fcc97b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5fc809e000 0x7f5fc81339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:38.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.565+0000 7f5fcc97b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5fc809e410 0x7f5fc8133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:38.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.565+0000 7f5fcc97b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fc806b460 con 0x7f5fc809e000 2023-09-30T22:34:38.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.565+0000 7f5fcc97b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc809b120 con 0x7f5fc809e410 2023-09-30T22:34:38.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.565+0000 7f5fcc97b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc809a7e0 con 0x7f5fc809d6f0 2023-09-30T22:34:38.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.565+0000 7f5fc659c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5fc809d6f0 0x7f5fc809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:38.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.565+0000 7f5fc659c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5fc809d6f0 0x7f5fc809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:59198/0 (socket says 172.21.15.16:59198) 2023-09-30T22:34:38.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.565+0000 7f5fc659c700 1 -- 172.21.15.16:0/907601463 learned_addr learned my addr 172.21.15.16:0/907601463 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:38.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.565+0000 7f5fc6d9d700 1 --2- 172.21.15.16:0/907601463 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5fc809e410 0x7f5fc8133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:38.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.565+0000 7f5fc5d9b700 1 --2- 172.21.15.16:0/907601463 >> v2:172.21.15.16:3300/0 conn(0x7f5fc809e000 0x7f5fc81339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:38.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.566+0000 7f5fc659c700 1 -- 172.21.15.16:0/907601463 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5fc809e410 msgr2=0x7f5fc8133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:38.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.566+0000 7f5fc659c700 1 --2- 172.21.15.16:0/907601463 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5fc809e410 0x7f5fc8133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:38.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.566+0000 7f5fc659c700 1 -- 172.21.15.16:0/907601463 >> v2:172.21.15.16:3300/0 conn(0x7f5fc809e000 msgr2=0x7f5fc81339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:38.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.566+0000 7f5fc659c700 1 --2- 172.21.15.16:0/907601463 >> v2:172.21.15.16:3300/0 conn(0x7f5fc809e000 0x7f5fc81339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:38.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.566+0000 7f5fc659c700 1 -- 172.21.15.16:0/907601463 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc809a500 con 0x7f5fc809d6f0 2023-09-30T22:34:38.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.566+0000 7f5fc659c700 1 --2- 172.21.15.16:0/907601463 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5fc809d6f0 0x7f5fc809dac0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f5fb8009920 tx=0x7f5fb8009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1368f678d2777f8a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:38.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.566+0000 7f5fc559a700 1 -- 172.21.15.16:0/907601463 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb8011830 con 0x7f5fc809d6f0 2023-09-30T22:34:38.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.566+0000 7f5fc559a700 1 -- 172.21.15.16:0/907601463 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fb8011990 con 0x7f5fc809d6f0 2023-09-30T22:34:38.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.566+0000 7f5fc559a700 1 -- 172.21.15.16:0/907601463 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb8011d20 con 0x7f5fc809d6f0 2023-09-30T22:34:38.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.567+0000 7f5fcc97b700 1 -- 172.21.15.16:0/907601463 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5fc809d6f0 msgr2=0x7f5fc809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:38.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.567+0000 7f5fcc97b700 1 --2- 172.21.15.16:0/907601463 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5fc809d6f0 0x7f5fc809dac0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f5fb8009920 tx=0x7f5fb8009c30 comp rx=0 tx=0).stop 2023-09-30T22:34:38.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.567+0000 7f5fcc97b700 1 -- 172.21.15.16:0/907601463 shutdown_connections 2023-09-30T22:34:38.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.567+0000 7f5fcc97b700 1 --2- 172.21.15.16:0/907601463 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5fc809e410 0x7f5fc8133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:38.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.567+0000 7f5fcc97b700 1 --2- 172.21.15.16:0/907601463 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5fc809d6f0 0x7f5fc809dac0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:38.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.567+0000 7f5fcc97b700 1 --2- 172.21.15.16:0/907601463 >> v2:172.21.15.16:3300/0 conn(0x7f5fc809e000 0x7f5fc81339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:38.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.567+0000 7f5fcc97b700 1 -- 172.21.15.16:0/907601463 >> 172.21.15.16:0/907601463 conn(0x7f5fc8066f50 msgr2=0x7f5fc809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:38.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.567+0000 7f5fcc97b700 1 -- 172.21.15.16:0/907601463 shutdown_connections 2023-09-30T22:34:38.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.567+0000 7f5fcc97b700 1 -- 172.21.15.16:0/907601463 wait complete. 2023-09-30T22:34:38.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.567+0000 7ff205ffb700 1 -- 172.21.15.16:0/3919208384 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff2000123b0 con 0x7ff21009e000 2023-09-30T22:34:38.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.569+0000 7f5fcc97b700 1 Processor -- start 2023-09-30T22:34:38.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.569+0000 7f5fcc97b700 1 -- start start 2023-09-30T22:34:38.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.569+0000 7f5fcc97b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5fc809d6f0 0x7f5fc81a5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:38.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.570+0000 7f5fcc97b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fc809e000 0x7f5fc81a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:38.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.570+0000 7f5fcc97b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5fc809e410 0x7f5fc81a68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:38.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.570+0000 7f5fcc97b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc81aa2a0 con 0x7f5fc809e000 2023-09-30T22:34:38.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.570+0000 7f5fcc97b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc81a79d0 con 0x7f5fc809d6f0 2023-09-30T22:34:38.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.570+0000 7f5fcc97b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc81a8c20 con 0x7f5fc809e410 2023-09-30T22:34:38.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.570+0000 7f5fc6d9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5fc809e410 0x7f5fc81a68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:38.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.570+0000 7f5fc6d9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5fc809e410 0x7f5fc81a68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:59200/0 (socket says 172.21.15.16:59200) 2023-09-30T22:34:38.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.570+0000 7f5fc6d9d700 1 -- 172.21.15.16:0/684500348 learned_addr learned my addr 172.21.15.16:0/684500348 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:38.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.570+0000 7f5fc5d9b700 1 --2- 172.21.15.16:0/684500348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fc809e000 0x7f5fc81a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:38.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.570+0000 7f5fc6d9d700 1 -- 172.21.15.16:0/684500348 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5fc809d6f0 msgr2=0x7f5fc81a5e50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:34:38.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.570+0000 7f5fc6d9d700 1 --2- 172.21.15.16:0/684500348 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5fc809d6f0 0x7f5fc81a5e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:38.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.570+0000 7f5fc6d9d700 1 -- 172.21.15.16:0/684500348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fc809e000 msgr2=0x7f5fc81a6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:38.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.570+0000 7f5fc6d9d700 1 --2- 172.21.15.16:0/684500348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fc809e000 0x7f5fc81a6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:38.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.570+0000 7f5fc6d9d700 1 -- 172.21.15.16:0/684500348 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fb8009550 con 0x7f5fc809e410 2023-09-30T22:34:38.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.570+0000 7f5fc6d9d700 1 --2- 172.21.15.16:0/684500348 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5fc809e410 0x7f5fc81a68d0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0009dd0 tx=0x7f5fc000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:38.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.571+0000 7f5fb6ffd700 1 -- 172.21.15.16:0/684500348 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc0009c40 con 0x7f5fc809e410 2023-09-30T22:34:38.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.571+0000 7f5fb6ffd700 1 -- 172.21.15.16:0/684500348 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fc000f040 con 0x7f5fc809e410 2023-09-30T22:34:38.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.571+0000 7f5fb6ffd700 1 -- 172.21.15.16:0/684500348 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc0014480 con 0x7f5fc809e410 2023-09-30T22:34:38.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.571+0000 7f5fcc97b700 1 -- 172.21.15.16:0/684500348 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fc81a7fc0 con 0x7f5fc809e410 2023-09-30T22:34:38.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.571+0000 7f5fcc97b700 1 -- 172.21.15.16:0/684500348 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fc809c710 con 0x7f5fc809e410 2023-09-30T22:34:38.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.572+0000 7f5fb6ffd700 1 -- 172.21.15.16:0/684500348 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5fc00145e0 con 0x7f5fc809e410 2023-09-30T22:34:38.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.572+0000 7f5fcc97b700 1 -- 172.21.15.16:0/684500348 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fac0052e0 con 0x7f5fc809e410 2023-09-30T22:34:38.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.572+0000 7f5fb6ffd700 1 -- 172.21.15.16:0/684500348 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f5fc003ff40 con 0x7f5fc809e410 2023-09-30T22:34:38.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.577+0000 7f5fb6ffd700 1 -- 172.21.15.16:0/684500348 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5fc00118b0 con 0x7f5fc809e410 2023-09-30T22:34:38.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.608+0000 7ffa04edc700 1 -- 172.21.15.16:0/1092688681 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ffa00074ea0 con 0x7ffa0009e000 2023-09-30T22:34:38.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.608+0000 7ff9f6ffd700 1 -- 172.21.15.16:0/1092688681 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff9e8019b20 con 0x7ffa0009e000 2023-09-30T22:34:38.609 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:34:38.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.611+0000 7ffa04edc700 1 -- 172.21.15.16:0/1092688681 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa0009e000 msgr2=0x7ffa0011e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:38.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.611+0000 7ffa04edc700 1 --2- 172.21.15.16:0/1092688681 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa0009e000 0x7ffa0011e590 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7ffa000a3e20 tx=0x7ff9e800c710 comp rx=0 tx=0).stop 2023-09-30T22:34:38.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.611+0000 7ffa04edc700 1 -- 172.21.15.16:0/1092688681 shutdown_connections 2023-09-30T22:34:38.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.611+0000 7ffa04edc700 1 --2- 172.21.15.16:0/1092688681 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa0009d6f0 0x7ffa0011e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:38.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.611+0000 7ffa04edc700 1 --2- 172.21.15.16:0/1092688681 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa0009e410 0x7ffa0011ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:38.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.611+0000 7ffa04edc700 1 --2- 172.21.15.16:0/1092688681 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa0009e000 0x7ffa0011e590 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:38.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.611+0000 7ffa04edc700 1 -- 172.21.15.16:0/1092688681 >> 172.21.15.16:0/1092688681 conn(0x7ffa00066f50 msgr2=0x7ffa00130b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:38.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.611+0000 7ffa04edc700 1 -- 172.21.15.16:0/1092688681 shutdown_connections 2023-09-30T22:34:38.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.611+0000 7ffa04edc700 1 -- 172.21.15.16:0/1092688681 wait complete. 2023-09-30T22:34:38.626 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:34:38.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.756+0000 7fc7588f1700 1 -- 172.21.15.16:0/1640703282 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc754074ea0 con 0x7fc75409e930 2023-09-30T22:34:38.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.756+0000 7fc742ffd700 1 -- 172.21.15.16:0/1640703282 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc744021020 con 0x7fc75409e930 2023-09-30T22:34:38.757 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:34:38.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.760+0000 7fc7588f1700 1 -- 172.21.15.16:0/1640703282 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc75409e930 msgr2=0x7fc7541a3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:38.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.760+0000 7fc7588f1700 1 --2- 172.21.15.16:0/1640703282 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc75409e930 0x7fc7541a3e60 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc74400e9e0 tx=0x7fc74400ecf0 comp rx=0 tx=0).stop 2023-09-30T22:34:38.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.760+0000 7fc7588f1700 1 -- 172.21.15.16:0/1640703282 shutdown_connections 2023-09-30T22:34:38.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.760+0000 7fc7588f1700 1 --2- 172.21.15.16:0/1640703282 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc75409f650 0x7fc7541a48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:38.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.760+0000 7fc7588f1700 1 --2- 172.21.15.16:0/1640703282 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc75409e930 0x7fc7541a3e60 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:38.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.760+0000 7fc7588f1700 1 --2- 172.21.15.16:0/1640703282 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc75409f240 0x7fc7541a43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:38.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.760+0000 7fc7588f1700 1 -- 172.21.15.16:0/1640703282 >> 172.21.15.16:0/1640703282 conn(0x7fc7540a4210 msgr2=0x7fc7541b8d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:38.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.760+0000 7fc7588f1700 1 -- 172.21.15.16:0/1640703282 shutdown_connections 2023-09-30T22:34:38.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.760+0000 7fc7588f1700 1 -- 172.21.15.16:0/1640703282 wait complete. 2023-09-30T22:34:38.775 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:34:38.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.783+0000 7ff217847700 1 -- 172.21.15.16:0/3919208384 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff210125910 con 0x7ff21009e000 2023-09-30T22:34:38.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.783+0000 7ff205ffb700 1 -- 172.21.15.16:0/3919208384 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff200011360 con 0x7ff21009e000 2023-09-30T22:34:38.784 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:34:38.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.785+0000 7ff217847700 1 -- 172.21.15.16:0/3919208384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff21009e000 msgr2=0x7ff210122740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:38.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.785+0000 7ff217847700 1 --2- 172.21.15.16:0/3919208384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff21009e000 0x7ff210122740 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7ff20000c540 tx=0x7ff20000c850 comp rx=0 tx=0).stop 2023-09-30T22:34:38.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.785+0000 7ff217847700 1 -- 172.21.15.16:0/3919208384 shutdown_connections 2023-09-30T22:34:38.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.785+0000 7ff217847700 1 --2- 172.21.15.16:0/3919208384 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff21009e410 0x7ff210122c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:38.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.785+0000 7ff217847700 1 --2- 172.21.15.16:0/3919208384 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff21009d6f0 0x7ff210122200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:38.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.785+0000 7ff217847700 1 --2- 172.21.15.16:0/3919208384 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff21009e000 0x7ff210122740 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:38.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.785+0000 7ff217847700 1 -- 172.21.15.16:0/3919208384 >> 172.21.15.16:0/3919208384 conn(0x7ff210066f50 msgr2=0x7ff210133e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:38.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.785+0000 7ff217847700 1 -- 172.21.15.16:0/3919208384 shutdown_connections 2023-09-30T22:34:38.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.785+0000 7ff217847700 1 -- 172.21.15.16:0/3919208384 wait complete. 2023-09-30T22:34:38.799 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:34:38.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.804+0000 7f5fcc97b700 1 -- 172.21.15.16:0/684500348 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5fac005590 con 0x7f5fc809e410 2023-09-30T22:34:38.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.805+0000 7f5fb6ffd700 1 -- 172.21.15.16:0/684500348 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5fc0018070 con 0x7f5fc809e410 2023-09-30T22:34:38.805 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:34:38.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.807+0000 7f5fcc97b700 1 -- 172.21.15.16:0/684500348 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5fc809e410 msgr2=0x7f5fc81a68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:38.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.807+0000 7f5fcc97b700 1 --2- 172.21.15.16:0/684500348 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5fc809e410 0x7f5fc81a68d0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0009dd0 tx=0x7f5fc000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:34:38.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.807+0000 7f5fcc97b700 1 -- 172.21.15.16:0/684500348 shutdown_connections 2023-09-30T22:34:38.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.807+0000 7f5fcc97b700 1 --2- 172.21.15.16:0/684500348 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5fc809d6f0 0x7f5fc81a5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:38.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.807+0000 7f5fcc97b700 1 --2- 172.21.15.16:0/684500348 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5fc809e410 0x7f5fc81a68d0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:38.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.807+0000 7f5fcc97b700 1 --2- 172.21.15.16:0/684500348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fc809e000 0x7f5fc81a6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:38.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.807+0000 7f5fcc97b700 1 -- 172.21.15.16:0/684500348 >> 172.21.15.16:0/684500348 conn(0x7f5fc8066f50 msgr2=0x7f5fc81bae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:38.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.807+0000 7f5fcc97b700 1 -- 172.21.15.16:0/684500348 shutdown_connections 2023-09-30T22:34:38.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:38.807+0000 7f5fcc97b700 1 -- 172.21.15.16:0/684500348 wait complete. 2023-09-30T22:34:38.820 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:34:39.627 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:34:39.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.774+0000 7f2d20585700 1 Processor -- start 2023-09-30T22:34:39.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.774+0000 7f2d20585700 1 -- start start 2023-09-30T22:34:39.775 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:34:39.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.775+0000 7f2d20585700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2d1809d6f0 0x7f2d1809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:39.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.775+0000 7f2d20585700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2d1809e000 0x7f2d181339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:39.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.775+0000 7f2d20585700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2d1809e410 0x7f2d18133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:39.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.775+0000 7f2d20585700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d1806b460 con 0x7f2d1809d6f0 2023-09-30T22:34:39.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.775+0000 7f2d20585700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1809b120 con 0x7f2d1809e410 2023-09-30T22:34:39.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.775+0000 7f2d20585700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1809a7e0 con 0x7f2d1809e000 2023-09-30T22:34:39.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.775+0000 7f2d1e321700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2d1809d6f0 0x7f2d1809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:39.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.775+0000 7f2d1db20700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2d1809e000 0x7f2d181339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:39.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.775+0000 7f2d1eb22700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2d1809e410 0x7f2d18133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:39.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.775+0000 7f2d1db20700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2d1809e000 0x7f2d181339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:59202/0 (socket says 172.21.15.16:59202) 2023-09-30T22:34:39.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.775+0000 7f2d1e321700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2d1809d6f0 0x7f2d1809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56490/0 (socket says 172.21.15.16:56490) 2023-09-30T22:34:39.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.775+0000 7f2d1db20700 1 -- 172.21.15.16:0/646499120 learned_addr learned my addr 172.21.15.16:0/646499120 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:39.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.776+0000 7f2d1db20700 1 -- 172.21.15.16:0/646499120 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2d1809e410 msgr2=0x7f2d18133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:39.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.776+0000 7f2d1db20700 1 --2- 172.21.15.16:0/646499120 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2d1809e410 0x7f2d18133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:39.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.776+0000 7f2d1db20700 1 -- 172.21.15.16:0/646499120 >> v2:172.21.15.16:3300/0 conn(0x7f2d1809d6f0 msgr2=0x7f2d1809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:39.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.776+0000 7f2d1db20700 1 --2- 172.21.15.16:0/646499120 >> v2:172.21.15.16:3300/0 conn(0x7f2d1809d6f0 0x7f2d1809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:39.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.776+0000 7f2d1db20700 1 -- 172.21.15.16:0/646499120 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d1809a500 con 0x7f2d1809e000 2023-09-30T22:34:39.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.776+0000 7f2d1eb22700 1 --2- 172.21.15.16:0/646499120 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2d1809e410 0x7f2d18133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:39.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.776+0000 7f2d1e321700 1 --2- 172.21.15.16:0/646499120 >> v2:172.21.15.16:3300/0 conn(0x7f2d1809d6f0 0x7f2d1809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:39.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.776+0000 7f2d1db20700 1 --2- 172.21.15.16:0/646499120 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2d1809e000 0x7f2d181339f0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f2d08009920 tx=0x7f2d08009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b90aa3f3eefc0b87 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:39.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.776+0000 7f2d1d31f700 1 -- 172.21.15.16:0/646499120 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d08011830 con 0x7f2d1809e000 2023-09-30T22:34:39.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.776+0000 7f2d1d31f700 1 -- 172.21.15.16:0/646499120 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d08011990 con 0x7f2d1809e000 2023-09-30T22:34:39.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.776+0000 7f2d1d31f700 1 -- 172.21.15.16:0/646499120 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d08011d20 con 0x7f2d1809e000 2023-09-30T22:34:39.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.777+0000 7f2d20585700 1 -- 172.21.15.16:0/646499120 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2d1809e000 msgr2=0x7f2d181339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:39.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.777+0000 7f2d20585700 1 --2- 172.21.15.16:0/646499120 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2d1809e000 0x7f2d181339f0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f2d08009920 tx=0x7f2d08009c30 comp rx=0 tx=0).stop 2023-09-30T22:34:39.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.777+0000 7f2d20585700 1 -- 172.21.15.16:0/646499120 shutdown_connections 2023-09-30T22:34:39.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.777+0000 7f2d20585700 1 --2- 172.21.15.16:0/646499120 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2d1809e410 0x7f2d18133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:39.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.777+0000 7f2d20585700 1 --2- 172.21.15.16:0/646499120 >> v2:172.21.15.16:3300/0 conn(0x7f2d1809d6f0 0x7f2d1809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:39.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.777+0000 7f2d20585700 1 --2- 172.21.15.16:0/646499120 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2d1809e000 0x7f2d181339f0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:39.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.777+0000 7f2d20585700 1 -- 172.21.15.16:0/646499120 >> 172.21.15.16:0/646499120 conn(0x7f2d18066f50 msgr2=0x7f2d1809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:39.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.777+0000 7f2d20585700 1 -- 172.21.15.16:0/646499120 shutdown_connections 2023-09-30T22:34:39.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.777+0000 7f2d20585700 1 -- 172.21.15.16:0/646499120 wait complete. 2023-09-30T22:34:39.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.778+0000 7f2d20585700 1 Processor -- start 2023-09-30T22:34:39.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.778+0000 7f2d20585700 1 -- start start 2023-09-30T22:34:39.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.779+0000 7f2d20585700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d1809d6f0 0x7f2d181a5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:39.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.779+0000 7f2d20585700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2d1809e000 0x7f2d181a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:39.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.779+0000 7f2d20585700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2d1809e410 0x7f2d181a6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:39.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.779+0000 7f2d20585700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d181aa240 con 0x7f2d1809d6f0 2023-09-30T22:34:39.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.779+0000 7f2d20585700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d181a7970 con 0x7f2d1809e000 2023-09-30T22:34:39.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.779+0000 7f2d20585700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2d181a7f90 con 0x7f2d1809e410 2023-09-30T22:34:39.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.779+0000 7f2d1db20700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2d1809e000 0x7f2d181a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:39.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.779+0000 7f2d1eb22700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2d1809e410 0x7f2d181a6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:39.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.779+0000 7f2d1e321700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d1809d6f0 0x7f2d181a5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:39.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.779+0000 7f2d1db20700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2d1809e000 0x7f2d181a62f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:51478/0 (socket says 172.21.15.16:51478) 2023-09-30T22:34:39.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.779+0000 7f2d1db20700 1 -- 172.21.15.16:0/3336108904 learned_addr learned my addr 172.21.15.16:0/3336108904 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:39.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.779+0000 7f2d1eb22700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2d1809e410 0x7f2d181a6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:59204/0 (socket says 172.21.15.16:59204) 2023-09-30T22:34:39.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.780+0000 7f2d1e321700 1 -- 172.21.15.16:0/3336108904 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2d1809e410 msgr2=0x7f2d181a6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:39.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.780+0000 7f2d1e321700 1 --2- 172.21.15.16:0/3336108904 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2d1809e410 0x7f2d181a6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:39.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.780+0000 7f2d1e321700 1 -- 172.21.15.16:0/3336108904 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2d1809e000 msgr2=0x7f2d181a62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:39.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.780+0000 7f2d1e321700 1 --2- 172.21.15.16:0/3336108904 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2d1809e000 0x7f2d181a62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:39.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.780+0000 7f2d1e321700 1 -- 172.21.15.16:0/3336108904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d140094a0 con 0x7f2d1809d6f0 2023-09-30T22:34:39.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.780+0000 7f2d1db20700 1 --2- 172.21.15.16:0/3336108904 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2d1809e000 0x7f2d181a62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:39.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.780+0000 7f2d1eb22700 1 --2- 172.21.15.16:0/3336108904 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2d1809e410 0x7f2d181a6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:39.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.780+0000 7f2d1e321700 1 --2- 172.21.15.16:0/3336108904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d1809d6f0 0x7f2d181a5db0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400e9e0 tx=0x7f2d1400ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:39.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.780+0000 7f2d0effd700 1 -- 172.21.15.16:0/3336108904 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1400ee70 con 0x7f2d1809d6f0 2023-09-30T22:34:39.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.780+0000 7f2d0effd700 1 -- 172.21.15.16:0/3336108904 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d1400c850 con 0x7f2d1809d6f0 2023-09-30T22:34:39.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.780+0000 7f2d20585700 1 -- 172.21.15.16:0/3336108904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d10009a70 con 0x7f2d1809d6f0 2023-09-30T22:34:39.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.780+0000 7f2d0effd700 1 -- 172.21.15.16:0/3336108904 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d1400cc00 con 0x7f2d1809d6f0 2023-09-30T22:34:39.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.780+0000 7f2d20585700 1 -- 172.21.15.16:0/3336108904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d08009550 con 0x7f2d1809d6f0 2023-09-30T22:34:39.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.781+0000 7f2d0effd700 1 -- 172.21.15.16:0/3336108904 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2d1400cd60 con 0x7f2d1809d6f0 2023-09-30T22:34:39.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.781+0000 7f2d0effd700 1 -- 172.21.15.16:0/3336108904 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f2d1403fa40 con 0x7f2d1809d6f0 2023-09-30T22:34:39.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.781+0000 7f2d20585700 1 -- 172.21.15.16:0/3336108904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d1809c710 con 0x7f2d1809d6f0 2023-09-30T22:34:39.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.786+0000 7f2d0effd700 1 -- 172.21.15.16:0/3336108904 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2d1401b3b0 con 0x7f2d1809d6f0 2023-09-30T22:34:39.800 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:34:39.821 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:34:39.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.929+0000 7f3f8fc20700 1 Processor -- start 2023-09-30T22:34:39.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.930+0000 7f3f8fc20700 1 -- start start 2023-09-30T22:34:39.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.931+0000 7f3f8fc20700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3f881338c0 0x7f3f88133c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:39.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.931+0000 7f3f8fc20700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3f881341d0 0x7f3f8809de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:39.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.931+0000 7f3f8fc20700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f8806f720 con 0x7f3f8809e390 2023-09-30T22:34:39.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.931+0000 7f3f8fc20700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f88070c80 con 0x7f3f881341d0 2023-09-30T22:34:39.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.931+0000 7f3f8fc20700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8806fce0 con 0x7f3f881338c0 2023-09-30T22:34:39.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.931+0000 7f3f8d1bb700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3f881341d0 0x7f3f8809de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:39.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.931+0000 7f3f8d9bc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3f881338c0 0x7f3f88133c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:39.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.931+0000 7f3f8d1bb700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3f881341d0 0x7f3f8809de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56516/0 (socket says 172.21.15.16:56516) 2023-09-30T22:34:39.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.931+0000 7f3f8d9bc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3f881338c0 0x7f3f88133c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:59212/0 (socket says 172.21.15.16:59212) 2023-09-30T22:34:39.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.931+0000 7f3f8d1bb700 1 -- 172.21.15.16:0/2991780776 learned_addr learned my addr 172.21.15.16:0/2991780776 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:39.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.931+0000 7f3f8e1bd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3f8809e390 0x7f3f8813a850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58308/0 (socket says 172.21.15.16:58308) 2023-09-30T22:34:39.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.931+0000 7f3f8d9bc700 1 -- 172.21.15.16:0/2991780776 >> v2:172.21.15.16:3300/0 conn(0x7f3f881341d0 msgr2=0x7f3f8809de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:39.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.931+0000 7f3f8d9bc700 1 --2- 172.21.15.16:0/2991780776 >> v2:172.21.15.16:3300/0 conn(0x7f3f881341d0 0x7f3f8809de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:39.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.931+0000 7f3f8d9bc700 1 -- 172.21.15.16:0/2991780776 >> v1:172.21.15.16:6789/0 conn(0x7f3f8809e390 legacy=0x7f3f8813a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:39.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.931+0000 7f3f8d9bc700 1 -- 172.21.15.16:0/2991780776 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f881336a0 con 0x7f3f881338c0 2023-09-30T22:34:39.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.932+0000 7f3f8d9bc700 1 --2- 172.21.15.16:0/2991780776 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3f881338c0 0x7f3f88133c90 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f3f84009840 tx=0x7f3f84009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1754e0f4ada6bf68 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:39.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.932+0000 7f3f8c9ba700 1 -- 172.21.15.16:0/2991780776 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f84011830 con 0x7f3f881338c0 2023-09-30T22:34:39.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.932+0000 7f3f8c9ba700 1 -- 172.21.15.16:0/2991780776 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f84011990 con 0x7f3f881338c0 2023-09-30T22:34:39.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.932+0000 7f3f8c9ba700 1 -- 172.21.15.16:0/2991780776 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f84011d20 con 0x7f3f881338c0 2023-09-30T22:34:39.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.933+0000 7f3f8fc20700 1 -- 172.21.15.16:0/2991780776 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3f881338c0 msgr2=0x7f3f88133c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:39.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.933+0000 7f3f8fc20700 1 --2- 172.21.15.16:0/2991780776 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3f881338c0 0x7f3f88133c90 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f3f84009840 tx=0x7f3f84009b50 comp rx=0 tx=0).stop 2023-09-30T22:34:39.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.933+0000 7f3f8fc20700 1 -- 172.21.15.16:0/2991780776 shutdown_connections 2023-09-30T22:34:39.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.933+0000 7f3f8fc20700 1 --2- 172.21.15.16:0/2991780776 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3f881338c0 0x7f3f88133c90 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:39.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.933+0000 7f3f8fc20700 1 --2- 172.21.15.16:0/2991780776 >> v2:172.21.15.16:3300/0 conn(0x7f3f881341d0 0x7f3f8809de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:39.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.933+0000 7f3f8fc20700 1 -- 172.21.15.16:0/2991780776 >> 172.21.15.16:0/2991780776 conn(0x7f3f880a2aa0 msgr2=0x7f3f880a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:39.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.935+0000 7f3f8fc20700 1 -- 172.21.15.16:0/2991780776 shutdown_connections 2023-09-30T22:34:39.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.935+0000 7f3f8fc20700 1 -- 172.21.15.16:0/2991780776 wait complete. 2023-09-30T22:34:39.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.936+0000 7f3f8fc20700 1 Processor -- start 2023-09-30T22:34:39.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.936+0000 7f3f8fc20700 1 -- start start 2023-09-30T22:34:39.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.937+0000 7f3f8fc20700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3f8809e390 0x7f3f8811f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:39.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.937+0000 7f3f8fc20700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f881338c0 0x7f3f8811fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:39.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.937+0000 7f3f8fc20700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3f881341d0 0x7f3f881202f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:39.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.937+0000 7f3f8fc20700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f881230b0 con 0x7f3f881338c0 2023-09-30T22:34:39.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.937+0000 7f3f8fc20700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f88122390 con 0x7f3f881341d0 2023-09-30T22:34:39.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.937+0000 7f3f8fc20700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3f880a1260 con 0x7f3f8809e390 2023-09-30T22:34:39.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.937+0000 7f3f8e1bd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3f881341d0 0x7f3f881202f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:39.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.937+0000 7f3f8e1bd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3f881341d0 0x7f3f881202f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:51488/0 (socket says 172.21.15.16:51488) 2023-09-30T22:34:39.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.937+0000 7f3f8e1bd700 1 -- 172.21.15.16:0/1878464059 learned_addr learned my addr 172.21.15.16:0/1878464059 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:39.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.937+0000 7f3f8d9bc700 1 --2- 172.21.15.16:0/1878464059 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3f8809e390 0x7f3f8811f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:39.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.938+0000 7f3f8d1bb700 1 --2- 172.21.15.16:0/1878464059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f881338c0 0x7f3f8811fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:39.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.938+0000 7f3f8e1bd700 1 -- 172.21.15.16:0/1878464059 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3f8809e390 msgr2=0x7f3f8811f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:39.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.938+0000 7f3f8e1bd700 1 --2- 172.21.15.16:0/1878464059 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3f8809e390 0x7f3f8811f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:39.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.938+0000 7f3f8e1bd700 1 -- 172.21.15.16:0/1878464059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f881338c0 msgr2=0x7f3f8811fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:39.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.938+0000 7f3f8e1bd700 1 --2- 172.21.15.16:0/1878464059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f881338c0 0x7f3f8811fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:39.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.938+0000 7f3f8e1bd700 1 -- 172.21.15.16:0/1878464059 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f840094a0 con 0x7f3f881341d0 2023-09-30T22:34:39.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.938+0000 7f3f8e1bd700 1 --2- 172.21.15.16:0/1878464059 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3f881341d0 0x7f3f881202f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f3f8000cbd0 tx=0x7f3f8000cee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:39.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.938+0000 7f3f7e7fc700 1 -- 172.21.15.16:0/1878464059 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8000ab00 con 0x7f3f881341d0 2023-09-30T22:34:39.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.938+0000 7f3f8fc20700 1 -- 172.21.15.16:0/1878464059 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f8806f960 con 0x7f3f881341d0 2023-09-30T22:34:39.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.938+0000 7f3f8fc20700 1 -- 172.21.15.16:0/1878464059 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f88123390 con 0x7f3f881341d0 2023-09-30T22:34:39.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.938+0000 7f3f7e7fc700 1 -- 172.21.15.16:0/1878464059 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f8000ac60 con 0x7f3f881341d0 2023-09-30T22:34:39.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.938+0000 7f3f7e7fc700 1 -- 172.21.15.16:0/1878464059 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f80007070 con 0x7f3f881341d0 2023-09-30T22:34:39.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.939+0000 7f3f7e7fc700 1 -- 172.21.15.16:0/1878464059 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3f800071d0 con 0x7f3f881341d0 2023-09-30T22:34:39.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.939+0000 7f3f7e7fc700 1 -- 172.21.15.16:0/1878464059 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f3f8003f5a0 con 0x7f3f881341d0 2023-09-30T22:34:39.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.941+0000 7f3f8fc20700 1 -- 172.21.15.16:0/1878464059 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f8811c480 con 0x7f3f881341d0 2023-09-30T22:34:39.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.943+0000 7f3af76a1700 1 Processor -- start 2023-09-30T22:34:39.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.943+0000 7f3af76a1700 1 -- start start 2023-09-30T22:34:39.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.944+0000 7f3af76a1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3af006a770 0x7f3af009d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:39.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.944+0000 7f3af76a1700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3af013d1f0 0x7f3af013f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:39.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.944+0000 7f3af76a1700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3af009a810 con 0x7f3af009dc30 2023-09-30T22:34:39.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.944+0000 7f3af76a1700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3af006ab40 con 0x7f3af013d1f0 2023-09-30T22:34:39.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.944+0000 7f3af76a1700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3af009ebc0 con 0x7f3af006a770 2023-09-30T22:34:39.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.944+0000 7f3af4c3c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3af009dc30 0x7f3af009e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58318/0 (socket says 172.21.15.16:58318) 2023-09-30T22:34:39.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.944+0000 7f3af4c3c700 1 -- 172.21.15.16:0/1828669529 learned_addr learned my addr 172.21.15.16:0/1828669529 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:34:39.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.944+0000 7f3af5c3e700 1 --2- 172.21.15.16:0/1828669529 >> v2:172.21.15.16:3300/0 conn(0x7f3af013d1f0 0x7f3af013f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:39.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.944+0000 7f3af543d700 1 --2- 172.21.15.16:0/1828669529 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3af006a770 0x7f3af009d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:39.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.944+0000 7f3ae7fff700 1 -- 172.21.15.16:0/1828669529 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2406247559 0 0) 0x7f3af009a810 con 0x7f3af009dc30 2023-09-30T22:34:39.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.944+0000 7f3ae7fff700 1 -- 172.21.15.16:0/1828669529 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3adc0035b0 con 0x7f3af009dc30 2023-09-30T22:34:39.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.945+0000 7f3af5c3e700 1 -- 172.21.15.16:0/1828669529 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3af006a770 msgr2=0x7f3af009d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:39.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.945+0000 7f3af5c3e700 1 --2- 172.21.15.16:0/1828669529 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3af006a770 0x7f3af009d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:39.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.945+0000 7f3af5c3e700 1 -- 172.21.15.16:0/1828669529 >> v1:172.21.15.16:6789/0 conn(0x7f3af009dc30 legacy=0x7f3af009e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:39.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.945+0000 7f3af5c3e700 1 -- 172.21.15.16:0/1828669529 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3af009e920 con 0x7f3af013d1f0 2023-09-30T22:34:39.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.945+0000 7f3af5c3e700 1 --2- 172.21.15.16:0/1828669529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3af013d1f0 0x7f3af013f5d0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f3ae8009dc0 tx=0x7f3ae800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab54632735245b34 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:39.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.945+0000 7f3ae7fff700 1 -- 172.21.15.16:0/1828669529 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae800e070 con 0x7f3af013d1f0 2023-09-30T22:34:39.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.945+0000 7f3ae7fff700 1 -- 172.21.15.16:0/1828669529 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ae800bae0 con 0x7f3af013d1f0 2023-09-30T22:34:39.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.945+0000 7f3ae7fff700 1 -- 172.21.15.16:0/1828669529 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae800be90 con 0x7f3af013d1f0 2023-09-30T22:34:39.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.946+0000 7f3af76a1700 1 -- 172.21.15.16:0/1828669529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3af013d1f0 msgr2=0x7f3af013f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:39.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.946+0000 7f3af76a1700 1 --2- 172.21.15.16:0/1828669529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3af013d1f0 0x7f3af013f5d0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f3ae8009dc0 tx=0x7f3ae800b330 comp rx=0 tx=0).stop 2023-09-30T22:34:39.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.946+0000 7f3af76a1700 1 -- 172.21.15.16:0/1828669529 shutdown_connections 2023-09-30T22:34:39.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.946+0000 7f3af76a1700 1 --2- 172.21.15.16:0/1828669529 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3af013d1f0 0x7f3af013f5d0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:39.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.946+0000 7f3af76a1700 1 --2- 172.21.15.16:0/1828669529 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3af006a770 0x7f3af009d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:39.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.946+0000 7f3af76a1700 1 -- 172.21.15.16:0/1828669529 >> 172.21.15.16:0/1828669529 conn(0x7f3af0066f50 msgr2=0x7f3af009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:39.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.946+0000 7f3af76a1700 1 -- 172.21.15.16:0/1828669529 shutdown_connections 2023-09-30T22:34:39.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.946+0000 7f3af76a1700 1 -- 172.21.15.16:0/1828669529 wait complete. 2023-09-30T22:34:39.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.947+0000 7f3f7e7fc700 1 -- 172.21.15.16:0/1878464059 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3f80017020 con 0x7f3f881341d0 2023-09-30T22:34:39.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.947+0000 7f3af76a1700 1 Processor -- start 2023-09-30T22:34:39.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.947+0000 7f3af76a1700 1 -- start start 2023-09-30T22:34:39.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.948+0000 7f3af76a1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3af006a770 0x7f3af011bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:39.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.948+0000 7f3af76a1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3af009dc30 0x7f3af013b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:39.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.948+0000 7f3af76a1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3af013d1f0 0x7f3af013ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:39.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.948+0000 7f3af76a1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af009f210 con 0x7f3af006a770 2023-09-30T22:34:39.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.948+0000 7f3af76a1700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3af011cd60 con 0x7f3af009dc30 2023-09-30T22:34:39.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.948+0000 7f3af76a1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3af011e6c0 con 0x7f3af013d1f0 2023-09-30T22:34:39.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.948+0000 7f3af543d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3af006a770 0x7f3af011bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:39.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.948+0000 7f3af543d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3af006a770 0x7f3af011bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56542/0 (socket says 172.21.15.16:56542) 2023-09-30T22:34:39.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.948+0000 7f3af543d700 1 -- 172.21.15.16:0/2441847852 learned_addr learned my addr 172.21.15.16:0/2441847852 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:39.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.948+0000 7f3af4c3c700 1 --2- 172.21.15.16:0/2441847852 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3af009dc30 0x7f3af013b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:39.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.948+0000 7f3af543d700 1 -- 172.21.15.16:0/2441847852 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3af013d1f0 msgr2=0x7f3af013ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:34:39.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.948+0000 7f3af543d700 1 --2- 172.21.15.16:0/2441847852 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3af013d1f0 0x7f3af013ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:39.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.948+0000 7f3af543d700 1 -- 172.21.15.16:0/2441847852 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3af009dc30 msgr2=0x7f3af013b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:39.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.948+0000 7f3af543d700 1 --2- 172.21.15.16:0/2441847852 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3af009dc30 0x7f3af013b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:39.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.948+0000 7f3af543d700 1 -- 172.21.15.16:0/2441847852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ae8009a70 con 0x7f3af006a770 2023-09-30T22:34:39.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.948+0000 7f3af543d700 1 --2- 172.21.15.16:0/2441847852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3af006a770 0x7f3af011bfe0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f3aec00e870 tx=0x7f3aec00eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:39.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.949+0000 7f3ae5ffb700 1 -- 172.21.15.16:0/2441847852 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aec013680 con 0x7f3af006a770 2023-09-30T22:34:39.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.949+0000 7f3ae5ffb700 1 -- 172.21.15.16:0/2441847852 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aec0137e0 con 0x7f3af006a770 2023-09-30T22:34:39.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.949+0000 7f3ae5ffb700 1 -- 172.21.15.16:0/2441847852 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aec013b50 con 0x7f3af006a770 2023-09-30T22:34:39.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.949+0000 7f3af76a1700 1 -- 172.21.15.16:0/2441847852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3af011eec0 con 0x7f3af006a770 2023-09-30T22:34:39.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.949+0000 7f3af76a1700 1 -- 172.21.15.16:0/2441847852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3af011e200 con 0x7f3af006a770 2023-09-30T22:34:39.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.950+0000 7f3af76a1700 1 -- 172.21.15.16:0/2441847852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3af0074ea0 con 0x7f3af006a770 2023-09-30T22:34:39.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.951+0000 7f3ae5ffb700 1 -- 172.21.15.16:0/2441847852 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3aec013cb0 con 0x7f3af006a770 2023-09-30T22:34:39.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.951+0000 7f3ae5ffb700 1 -- 172.21.15.16:0/2441847852 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f3aec03f6f0 con 0x7f3af006a770 2023-09-30T22:34:39.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:39.956+0000 7f3ae5ffb700 1 -- 172.21.15.16:0/2441847852 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3aec00c970 con 0x7f3af006a770 2023-09-30T22:34:40.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.006+0000 7f2d20585700 1 -- 172.21.15.16:0/3336108904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f2d18074ea0 con 0x7f2d1809d6f0 2023-09-30T22:34:40.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.007+0000 7f2d0effd700 1 -- 172.21.15.16:0/3336108904 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2d14021030 con 0x7f2d1809d6f0 2023-09-30T22:34:40.007 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:34:40.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.009+0000 7f2d20585700 1 -- 172.21.15.16:0/3336108904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d1809d6f0 msgr2=0x7f2d181a5db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:40.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.009+0000 7f2d20585700 1 --2- 172.21.15.16:0/3336108904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d1809d6f0 0x7f2d181a5db0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400e9e0 tx=0x7f2d1400ecf0 comp rx=0 tx=0).stop 2023-09-30T22:34:40.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.009+0000 7f2d20585700 1 -- 172.21.15.16:0/3336108904 shutdown_connections 2023-09-30T22:34:40.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.009+0000 7f2d20585700 1 --2- 172.21.15.16:0/3336108904 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2d1809e410 0x7f2d181a6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:40.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.009+0000 7f2d20585700 1 --2- 172.21.15.16:0/3336108904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d1809d6f0 0x7f2d181a5db0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:40.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.009+0000 7f2d20585700 1 --2- 172.21.15.16:0/3336108904 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2d1809e000 0x7f2d181a62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:40.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.009+0000 7f2d20585700 1 -- 172.21.15.16:0/3336108904 >> 172.21.15.16:0/3336108904 conn(0x7f2d18066f50 msgr2=0x7f2d181bad40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:40.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.010+0000 7f2d20585700 1 -- 172.21.15.16:0/3336108904 shutdown_connections 2023-09-30T22:34:40.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.012+0000 7f2d20585700 1 -- 172.21.15.16:0/3336108904 wait complete. 2023-09-30T22:34:40.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.014+0000 7ff763c8a700 1 Processor -- start 2023-09-30T22:34:40.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.014+0000 7ff763c8a700 1 -- start start 2023-09-30T22:34:40.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.015+0000 7ff763c8a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff75c09d6f0 0x7ff75c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:40.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.015+0000 7ff763c8a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff75c09e000 0x7ff75c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:40.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.015+0000 7ff763c8a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff75c09b120 con 0x7ff75c09e410 2023-09-30T22:34:40.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.015+0000 7ff763c8a700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff75c06b460 con 0x7ff75c09e000 2023-09-30T22:34:40.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.015+0000 7ff763c8a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c09a7e0 con 0x7ff75c09d6f0 2023-09-30T22:34:40.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.015+0000 7ff761a26700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff75c09d6f0 0x7ff75c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:40.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.015+0000 7ff761a26700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff75c09d6f0 0x7ff75c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:51502/0 (socket says 172.21.15.16:51502) 2023-09-30T22:34:40.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.015+0000 7ff761a26700 1 -- 172.21.15.16:0/2771942900 learned_addr learned my addr 172.21.15.16:0/2771942900 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:40.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.015+0000 7ff761225700 1 --2- 172.21.15.16:0/2771942900 >> v2:172.21.15.16:3300/0 conn(0x7ff75c09e000 0x7ff75c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:40.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.015+0000 7ff761a26700 1 -- 172.21.15.16:0/2771942900 >> v2:172.21.15.16:3300/0 conn(0x7ff75c09e000 msgr2=0x7ff75c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:40.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.015+0000 7ff761a26700 1 --2- 172.21.15.16:0/2771942900 >> v2:172.21.15.16:3300/0 conn(0x7ff75c09e000 0x7ff75c1339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:40.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.015+0000 7ff761a26700 1 -- 172.21.15.16:0/2771942900 >> v1:172.21.15.16:6789/0 conn(0x7ff75c09e410 legacy=0x7ff75c133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:40.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.015+0000 7ff761a26700 1 -- 172.21.15.16:0/2771942900 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff75c09a500 con 0x7ff75c09d6f0 2023-09-30T22:34:40.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.016+0000 7ff761a26700 1 --2- 172.21.15.16:0/2771942900 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff75c09d6f0 0x7ff75c09dac0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7ff758009920 tx=0x7ff758009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a271cbadb0215c78 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:40.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.016+0000 7ff760a24700 1 -- 172.21.15.16:0/2771942900 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff758011830 con 0x7ff75c09d6f0 2023-09-30T22:34:40.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.016+0000 7ff760a24700 1 -- 172.21.15.16:0/2771942900 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff758011990 con 0x7ff75c09d6f0 2023-09-30T22:34:40.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.016+0000 7ff760a24700 1 -- 172.21.15.16:0/2771942900 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff758011d20 con 0x7ff75c09d6f0 2023-09-30T22:34:40.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.017+0000 7ff763c8a700 1 -- 172.21.15.16:0/2771942900 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff75c09d6f0 msgr2=0x7ff75c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:40.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.017+0000 7ff763c8a700 1 --2- 172.21.15.16:0/2771942900 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff75c09d6f0 0x7ff75c09dac0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7ff758009920 tx=0x7ff758009c30 comp rx=0 tx=0).stop 2023-09-30T22:34:40.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.017+0000 7ff763c8a700 1 -- 172.21.15.16:0/2771942900 shutdown_connections 2023-09-30T22:34:40.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.017+0000 7ff763c8a700 1 --2- 172.21.15.16:0/2771942900 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff75c09d6f0 0x7ff75c09dac0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:40.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.017+0000 7ff763c8a700 1 --2- 172.21.15.16:0/2771942900 >> v2:172.21.15.16:3300/0 conn(0x7ff75c09e000 0x7ff75c1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:40.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.017+0000 7ff763c8a700 1 -- 172.21.15.16:0/2771942900 >> 172.21.15.16:0/2771942900 conn(0x7ff75c066f50 msgr2=0x7ff75c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:40.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.017+0000 7ff763c8a700 1 -- 172.21.15.16:0/2771942900 shutdown_connections 2023-09-30T22:34:40.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.017+0000 7ff763c8a700 1 -- 172.21.15.16:0/2771942900 wait complete. 2023-09-30T22:34:40.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.018+0000 7ff763c8a700 1 Processor -- start 2023-09-30T22:34:40.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.018+0000 7ff763c8a700 1 -- start start 2023-09-30T22:34:40.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.019+0000 7ff763c8a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff75c09d6f0 0x7ff75c1a5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:40.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.019+0000 7ff763c8a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff75c09e000 0x7ff75c1a6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:40.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.019+0000 7ff763c8a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff75c09e410 0x7ff75c1a6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:40.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.019+0000 7ff763c8a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c1aa270 con 0x7ff75c09d6f0 2023-09-30T22:34:40.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.019+0000 7ff763c8a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c1a87a0 con 0x7ff75c09e410 2023-09-30T22:34:40.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.019+0000 7ff763c8a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c1a7a40 con 0x7ff75c09e000 2023-09-30T22:34:40.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.019+0000 7ff761a26700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff75c09d6f0 0x7ff75c1a5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:40.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.019+0000 7ff761a26700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff75c09d6f0 0x7ff75c1a5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56554/0 (socket says 172.21.15.16:56554) 2023-09-30T22:34:40.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.019+0000 7ff761a26700 1 -- 172.21.15.16:0/3804261957 learned_addr learned my addr 172.21.15.16:0/3804261957 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:40.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.019+0000 7ff761225700 1 --2- 172.21.15.16:0/3804261957 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff75c09e000 0x7ff75c1a6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:40.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.020+0000 7ff762227700 1 --2- 172.21.15.16:0/3804261957 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff75c09e410 0x7ff75c1a6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:40.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.020+0000 7ff761225700 1 -- 172.21.15.16:0/3804261957 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff75c09e410 msgr2=0x7ff75c1a6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:40.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.020+0000 7ff761225700 1 --2- 172.21.15.16:0/3804261957 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff75c09e410 0x7ff75c1a6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:40.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.020+0000 7ff761225700 1 -- 172.21.15.16:0/3804261957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff75c09d6f0 msgr2=0x7ff75c1a5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:40.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.020+0000 7ff761225700 1 --2- 172.21.15.16:0/3804261957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff75c09d6f0 0x7ff75c1a5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:40.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.020+0000 7ff761225700 1 -- 172.21.15.16:0/3804261957 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff758009550 con 0x7ff75c09e000 2023-09-30T22:34:40.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.020+0000 7ff761a26700 1 --2- 172.21.15.16:0/3804261957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff75c09d6f0 0x7ff75c1a5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:40.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.020+0000 7ff761225700 1 --2- 172.21.15.16:0/3804261957 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff75c09e000 0x7ff75c1a6400 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7ff74c00e900 tx=0x7ff74c00ec10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:40.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.020+0000 7ff7527fc700 1 -- 172.21.15.16:0/3804261957 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74c013680 con 0x7ff75c09e000 2023-09-30T22:34:40.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.020+0000 7ff7527fc700 1 -- 172.21.15.16:0/3804261957 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff74c0137e0 con 0x7ff75c09e000 2023-09-30T22:34:40.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.020+0000 7ff763c8a700 1 -- 172.21.15.16:0/3804261957 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff75c0a3190 con 0x7ff75c09e000 2023-09-30T22:34:40.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.020+0000 7ff7527fc700 1 -- 172.21.15.16:0/3804261957 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74c013b50 con 0x7ff75c09e000 2023-09-30T22:34:40.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.020+0000 7ff763c8a700 1 -- 172.21.15.16:0/3804261957 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff75c1a98b0 con 0x7ff75c09e000 2023-09-30T22:34:40.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.021+0000 7ff763c8a700 1 -- 172.21.15.16:0/3804261957 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff75c074ea0 con 0x7ff75c09e000 2023-09-30T22:34:40.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.022+0000 7ff7527fc700 1 -- 172.21.15.16:0/3804261957 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff74c013cb0 con 0x7ff75c09e000 2023-09-30T22:34:40.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.022+0000 7ff7527fc700 1 -- 172.21.15.16:0/3804261957 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7ff74c03f660 con 0x7ff75c09e000 2023-09-30T22:34:40.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.027+0000 7ff7527fc700 1 -- 172.21.15.16:0/3804261957 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff74c01bde0 con 0x7ff75c09e000 2023-09-30T22:34:40.030 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:34:40.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.165+0000 7f3af76a1700 1 -- 172.21.15.16:0/2441847852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3af009a3d0 con 0x7f3af006a770 2023-09-30T22:34:40.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.165+0000 7f3ae5ffb700 1 -- 172.21.15.16:0/2441847852 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3aec017020 con 0x7f3af006a770 2023-09-30T22:34:40.166 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:34:40.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.168+0000 7f3af76a1700 1 -- 172.21.15.16:0/2441847852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3af006a770 msgr2=0x7f3af011bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:40.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.168+0000 7f3af76a1700 1 --2- 172.21.15.16:0/2441847852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3af006a770 0x7f3af011bfe0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f3aec00e870 tx=0x7f3aec00eb80 comp rx=0 tx=0).stop 2023-09-30T22:34:40.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.168+0000 7f3af76a1700 1 -- 172.21.15.16:0/2441847852 shutdown_connections 2023-09-30T22:34:40.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.168+0000 7f3af76a1700 1 --2- 172.21.15.16:0/2441847852 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3af013d1f0 0x7f3af013ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:40.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.168+0000 7f3af76a1700 1 --2- 172.21.15.16:0/2441847852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3af006a770 0x7f3af011bfe0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:40.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.168+0000 7f3af76a1700 1 --2- 172.21.15.16:0/2441847852 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3af009dc30 0x7f3af013b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:40.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.168+0000 7f3af76a1700 1 -- 172.21.15.16:0/2441847852 >> 172.21.15.16:0/2441847852 conn(0x7f3af0066f50 msgr2=0x7f3af00a0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:40.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.168+0000 7f3af76a1700 1 -- 172.21.15.16:0/2441847852 shutdown_connections 2023-09-30T22:34:40.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.168+0000 7f3af76a1700 1 -- 172.21.15.16:0/2441847852 wait complete. 2023-09-30T22:34:40.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.177+0000 7f3f8fc20700 1 -- 172.21.15.16:0/1878464059 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3f88074ea0 con 0x7f3f881341d0 2023-09-30T22:34:40.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.177+0000 7f3f7e7fc700 1 -- 172.21.15.16:0/1878464059 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3f8001b3b0 con 0x7f3f881341d0 2023-09-30T22:34:40.177 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:34:40.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.179+0000 7f3f8fc20700 1 -- 172.21.15.16:0/1878464059 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3f881341d0 msgr2=0x7f3f881202f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:40.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.179+0000 7f3f8fc20700 1 --2- 172.21.15.16:0/1878464059 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3f881341d0 0x7f3f881202f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f3f8000cbd0 tx=0x7f3f8000cee0 comp rx=0 tx=0).stop 2023-09-30T22:34:40.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.179+0000 7f3f8fc20700 1 -- 172.21.15.16:0/1878464059 shutdown_connections 2023-09-30T22:34:40.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.179+0000 7f3f8fc20700 1 --2- 172.21.15.16:0/1878464059 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3f881341d0 0x7f3f881202f0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:40.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.179+0000 7f3f8fc20700 1 --2- 172.21.15.16:0/1878464059 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3f8809e390 0x7f3f8811f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:40.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.179+0000 7f3f8fc20700 1 --2- 172.21.15.16:0/1878464059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f881338c0 0x7f3f8811fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:40.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.179+0000 7f3f8fc20700 1 -- 172.21.15.16:0/1878464059 >> 172.21.15.16:0/1878464059 conn(0x7f3f880a2aa0 msgr2=0x7f3f88131ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:40.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.179+0000 7f3f8fc20700 1 -- 172.21.15.16:0/1878464059 shutdown_connections 2023-09-30T22:34:40.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.179+0000 7f3f8fc20700 1 -- 172.21.15.16:0/1878464059 wait complete. 2023-09-30T22:34:40.182 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:34:40.194 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:34:40.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.236+0000 7ff763c8a700 1 -- 172.21.15.16:0/3804261957 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff75c09c710 con 0x7ff75c09e000 2023-09-30T22:34:40.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.236+0000 7ff7527fc700 1 -- 172.21.15.16:0/3804261957 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff74c01b3b0 con 0x7ff75c09e000 2023-09-30T22:34:40.237 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:34:40.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.239+0000 7ff763c8a700 1 -- 172.21.15.16:0/3804261957 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff75c09e000 msgr2=0x7ff75c1a6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:40.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.239+0000 7ff763c8a700 1 --2- 172.21.15.16:0/3804261957 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff75c09e000 0x7ff75c1a6400 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7ff74c00e900 tx=0x7ff74c00ec10 comp rx=0 tx=0).stop 2023-09-30T22:34:40.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.239+0000 7ff763c8a700 1 -- 172.21.15.16:0/3804261957 shutdown_connections 2023-09-30T22:34:40.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.239+0000 7ff763c8a700 1 --2- 172.21.15.16:0/3804261957 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff75c09e410 0x7ff75c1a6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:40.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.239+0000 7ff763c8a700 1 --2- 172.21.15.16:0/3804261957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff75c09d6f0 0x7ff75c1a5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:40.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.239+0000 7ff763c8a700 1 --2- 172.21.15.16:0/3804261957 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff75c09e000 0x7ff75c1a6400 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:40.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.239+0000 7ff763c8a700 1 -- 172.21.15.16:0/3804261957 >> 172.21.15.16:0/3804261957 conn(0x7ff75c066f50 msgr2=0x7ff75c1bad60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:40.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.239+0000 7ff763c8a700 1 -- 172.21.15.16:0/3804261957 shutdown_connections 2023-09-30T22:34:40.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:40.239+0000 7ff763c8a700 1 -- 172.21.15.16:0/3804261957 wait complete. 2023-09-30T22:34:40.253 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:34:41.031 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:34:41.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.176+0000 7f0a9453e700 1 Processor -- start 2023-09-30T22:34:41.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.176+0000 7f0a9453e700 1 -- start start 2023-09-30T22:34:41.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.177+0000 7f0a9453e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0a8c1367c0 0x7f0a8c138ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:41.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.177+0000 7f0a9453e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0a8c1390e0 0x7f0a8c13b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:41.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.177+0000 7f0a9453e700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a8c133550 con 0x7f0a8c133960 2023-09-30T22:34:41.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.177+0000 7f0a9453e700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a8c06fab0 con 0x7f0a8c1367c0 2023-09-30T22:34:41.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.177+0000 7f0a9453e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8c134440 con 0x7f0a8c1390e0 2023-09-30T22:34:41.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.177+0000 7f0a91ad9700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0a8c1367c0 0x7f0a8c138ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:41.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.177+0000 7f0a92adb700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0a8c1390e0 0x7f0a8c13b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:41.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.177+0000 7f0a922da700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0a8c133960 0x7f0a8c133d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58336/0 (socket says 172.21.15.16:58336) 2023-09-30T22:34:41.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.177+0000 7f0a922da700 1 -- 172.21.15.16:0/4010081531 learned_addr learned my addr 172.21.15.16:0/4010081531 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:34:41.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.177+0000 7f0a91ad9700 1 -- 172.21.15.16:0/4010081531 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0a8c1390e0 msgr2=0x7f0a8c13b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:41.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.177+0000 7f0a91ad9700 1 --2- 172.21.15.16:0/4010081531 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0a8c1390e0 0x7f0a8c13b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:41.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.177+0000 7f0a91ad9700 1 -- 172.21.15.16:0/4010081531 >> v1:172.21.15.16:6789/0 conn(0x7f0a8c133960 legacy=0x7f0a8c133d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:41.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.177+0000 7f0a91ad9700 1 -- 172.21.15.16:0/4010081531 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a8c071e20 con 0x7f0a8c1367c0 2023-09-30T22:34:41.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.177+0000 7f0a912d8700 1 -- 172.21.15.16:0/4010081531 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4121726177 0 0) 0x7f0a8c133550 con 0x7f0a8c133960 2023-09-30T22:34:41.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.177+0000 7f0a92adb700 1 --2- 172.21.15.16:0/4010081531 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0a8c1390e0 0x7f0a8c13b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:41.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.177+0000 7f0a91ad9700 1 --2- 172.21.15.16:0/4010081531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a8c1367c0 0x7f0a8c138ba0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f0a80009840 tx=0x7f0a80009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a471a6b2b12a3cf7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:41.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.177+0000 7f0a912d8700 1 -- 172.21.15.16:0/4010081531 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a80011830 con 0x7f0a8c1367c0 2023-09-30T22:34:41.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.178+0000 7f0a912d8700 1 -- 172.21.15.16:0/4010081531 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a80011990 con 0x7f0a8c1367c0 2023-09-30T22:34:41.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.178+0000 7f0a912d8700 1 -- 172.21.15.16:0/4010081531 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a80011d20 con 0x7f0a8c1367c0 2023-09-30T22:34:41.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.178+0000 7f0a9453e700 1 -- 172.21.15.16:0/4010081531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a8c1367c0 msgr2=0x7f0a8c138ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:41.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.178+0000 7f0a9453e700 1 --2- 172.21.15.16:0/4010081531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a8c1367c0 0x7f0a8c138ba0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f0a80009840 tx=0x7f0a80009b50 comp rx=0 tx=0).stop 2023-09-30T22:34:41.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.178+0000 7f0a9453e700 1 -- 172.21.15.16:0/4010081531 shutdown_connections 2023-09-30T22:34:41.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.178+0000 7f0a9453e700 1 --2- 172.21.15.16:0/4010081531 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0a8c1390e0 0x7f0a8c13b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:41.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.178+0000 7f0a9453e700 1 --2- 172.21.15.16:0/4010081531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a8c1367c0 0x7f0a8c138ba0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:41.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.178+0000 7f0a9453e700 1 -- 172.21.15.16:0/4010081531 >> 172.21.15.16:0/4010081531 conn(0x7f0a8c0a2610 msgr2=0x7f0a8c0a2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:41.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.178+0000 7f0a9453e700 1 -- 172.21.15.16:0/4010081531 shutdown_connections 2023-09-30T22:34:41.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.178+0000 7f0a9453e700 1 -- 172.21.15.16:0/4010081531 wait complete. 2023-09-30T22:34:41.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.179+0000 7f0a9453e700 1 Processor -- start 2023-09-30T22:34:41.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.179+0000 7f0a9453e700 1 -- start start 2023-09-30T22:34:41.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.180+0000 7f0a9453e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0a8c133960 0x7f0a8c1c8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:41.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.180+0000 7f0a9453e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0a8c1367c0 0x7f0a8c1c8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:41.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.180+0000 7f0a9453e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a8c1390e0 0x7f0a8c119aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:41.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.180+0000 7f0a9453e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8c11b660 con 0x7f0a8c1390e0 2023-09-30T22:34:41.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.180+0000 7f0a9453e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8c06f420 con 0x7f0a8c133960 2023-09-30T22:34:41.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.180+0000 7f0a9453e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8c11d300 con 0x7f0a8c1367c0 2023-09-30T22:34:41.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.180+0000 7f0a91ad9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0a8c1367c0 0x7f0a8c1c8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:41.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.180+0000 7f0a91ad9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0a8c1367c0 0x7f0a8c1c8980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:59264/0 (socket says 172.21.15.16:59264) 2023-09-30T22:34:41.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.180+0000 7f0a91ad9700 1 -- 172.21.15.16:0/2083853427 learned_addr learned my addr 172.21.15.16:0/2083853427 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:41.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.180+0000 7f0a92adb700 1 --2- 172.21.15.16:0/2083853427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a8c1390e0 0x7f0a8c119aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:41.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.180+0000 7f0a922da700 1 --2- 172.21.15.16:0/2083853427 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0a8c133960 0x7f0a8c1c8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:41.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.180+0000 7f0a91ad9700 1 -- 172.21.15.16:0/2083853427 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0a8c133960 msgr2=0x7f0a8c1c8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:41.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.180+0000 7f0a91ad9700 1 --2- 172.21.15.16:0/2083853427 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0a8c133960 0x7f0a8c1c8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:41.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.180+0000 7f0a91ad9700 1 -- 172.21.15.16:0/2083853427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a8c1390e0 msgr2=0x7f0a8c119aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:41.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.180+0000 7f0a91ad9700 1 --2- 172.21.15.16:0/2083853427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a8c1390e0 0x7f0a8c119aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:41.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.180+0000 7f0a91ad9700 1 -- 172.21.15.16:0/2083853427 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a84009a70 con 0x7f0a8c1367c0 2023-09-30T22:34:41.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.180+0000 7f0a91ad9700 1 --2- 172.21.15.16:0/2083853427 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0a8c1367c0 0x7f0a8c1c8980 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f0a80009fd0 tx=0x7f0a80004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:41.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.181+0000 7f0a7effd700 1 -- 172.21.15.16:0/2083853427 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a800045d0 con 0x7f0a8c1367c0 2023-09-30T22:34:41.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.181+0000 7f0a7effd700 1 -- 172.21.15.16:0/2083853427 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a8001f070 con 0x7f0a8c1367c0 2023-09-30T22:34:41.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.181+0000 7f0a9453e700 1 -- 172.21.15.16:0/2083853427 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a800094a0 con 0x7f0a8c1367c0 2023-09-30T22:34:41.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.181+0000 7f0a7effd700 1 -- 172.21.15.16:0/2083853427 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a800038a0 con 0x7f0a8c1367c0 2023-09-30T22:34:41.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.181+0000 7f0a9453e700 1 -- 172.21.15.16:0/2083853427 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a8c12f920 con 0x7f0a8c1367c0 2023-09-30T22:34:41.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.181+0000 7f0a7effd700 1 -- 172.21.15.16:0/2083853427 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0a80003a00 con 0x7f0a8c1367c0 2023-09-30T22:34:41.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.182+0000 7f0a7effd700 1 -- 172.21.15.16:0/2083853427 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f0a80021070 con 0x7f0a8c1367c0 2023-09-30T22:34:41.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.182+0000 7f0a9453e700 1 -- 172.21.15.16:0/2083853427 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a8c0747f0 con 0x7f0a8c1367c0 2023-09-30T22:34:41.187 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:34:41.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.187+0000 7f0a7effd700 1 -- 172.21.15.16:0/2083853427 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0a80003cb0 con 0x7f0a8c1367c0 2023-09-30T22:34:41.194 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:34:41.254 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:34:41.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.333+0000 7fb9c111a700 1 Processor -- start 2023-09-30T22:34:41.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.333+0000 7fb9c111a700 1 -- start start 2023-09-30T22:34:41.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.334+0000 7fb9c111a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb9bc09e1d0 0x7fb9bc135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:41.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.334+0000 7fb9c111a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb9bc135e50 0x7fb9bc138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:41.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.334+0000 7fb9c111a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9bc09f130 con 0x7fb9bc09d6f0 2023-09-30T22:34:41.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.334+0000 7fb9c111a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9bc09f520 con 0x7fb9bc135e50 2023-09-30T22:34:41.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.334+0000 7fb9c111a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb9bc09e930 con 0x7fb9bc09e1d0 2023-09-30T22:34:41.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.334+0000 7fb9bad9d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb9bc09d6f0 0x7fb9bc09dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58350/0 (socket says 172.21.15.16:58350) 2023-09-30T22:34:41.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.334+0000 7fb9bad9d700 1 -- 172.21.15.16:0/2725182791 learned_addr learned my addr 172.21.15.16:0/2725182791 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:34:41.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.334+0000 7fb9bb59e700 1 --2- 172.21.15.16:0/2725182791 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb9bc135e50 0x7fb9bc138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:41.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.334+0000 7fb9ba59c700 1 --2- 172.21.15.16:0/2725182791 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb9bc09e1d0 0x7fb9bc135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:41.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.335+0000 7fb9b9d9b700 1 -- 172.21.15.16:0/2725182791 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3986191535 0 0) 0x7fb9bc09f130 con 0x7fb9bc09d6f0 2023-09-30T22:34:41.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.335+0000 7fb9b9d9b700 1 -- 172.21.15.16:0/2725182791 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9a40035b0 con 0x7fb9bc09d6f0 2023-09-30T22:34:41.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.335+0000 7fb9ba59c700 1 -- 172.21.15.16:0/2725182791 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb9bc135e50 msgr2=0x7fb9bc138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:41.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.335+0000 7fb9ba59c700 1 --2- 172.21.15.16:0/2725182791 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb9bc135e50 0x7fb9bc138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:41.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.335+0000 7fb9ba59c700 1 -- 172.21.15.16:0/2725182791 >> v1:172.21.15.16:6789/0 conn(0x7fb9bc09d6f0 legacy=0x7fb9bc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:41.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.335+0000 7fb9ba59c700 1 -- 172.21.15.16:0/2725182791 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9bc09b2d0 con 0x7fb9bc09e1d0 2023-09-30T22:34:41.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.335+0000 7fb9ba59c700 1 --2- 172.21.15.16:0/2725182791 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb9bc09e1d0 0x7fb9bc135910 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0009870 tx=0x7fb9b0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3107f0653c8224f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:41.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.335+0000 7fb9b9d9b700 1 -- 172.21.15.16:0/2725182791 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b0011830 con 0x7fb9bc09e1d0 2023-09-30T22:34:41.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.335+0000 7fb9b9d9b700 1 -- 172.21.15.16:0/2725182791 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9b0011990 con 0x7fb9bc09e1d0 2023-09-30T22:34:41.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.335+0000 7fb9b9d9b700 1 -- 172.21.15.16:0/2725182791 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b0011d20 con 0x7fb9bc09e1d0 2023-09-30T22:34:41.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.336+0000 7fb9c111a700 1 -- 172.21.15.16:0/2725182791 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb9bc09e1d0 msgr2=0x7fb9bc135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:41.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.336+0000 7fb9c111a700 1 --2- 172.21.15.16:0/2725182791 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb9bc09e1d0 0x7fb9bc135910 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0009870 tx=0x7fb9b0009b80 comp rx=0 tx=0).stop 2023-09-30T22:34:41.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.336+0000 7fb9c111a700 1 -- 172.21.15.16:0/2725182791 shutdown_connections 2023-09-30T22:34:41.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.336+0000 7fb9c111a700 1 --2- 172.21.15.16:0/2725182791 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb9bc135e50 0x7fb9bc138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:41.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.336+0000 7fb9c111a700 1 --2- 172.21.15.16:0/2725182791 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb9bc09e1d0 0x7fb9bc135910 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:41.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.336+0000 7fb9c111a700 1 -- 172.21.15.16:0/2725182791 >> 172.21.15.16:0/2725182791 conn(0x7fb9bc066f50 msgr2=0x7fb9bc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:41.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.337+0000 7fb9c111a700 1 -- 172.21.15.16:0/2725182791 shutdown_connections 2023-09-30T22:34:41.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.337+0000 7fb9c111a700 1 -- 172.21.15.16:0/2725182791 wait complete. 2023-09-30T22:34:41.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.338+0000 7fb9c111a700 1 Processor -- start 2023-09-30T22:34:41.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.338+0000 7fb9c111a700 1 -- start start 2023-09-30T22:34:41.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.339+0000 7fb9c111a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb9bc09d6f0 0x7fb9bc11b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:41.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.339+0000 7fb9c111a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9bc09e1d0 0x7fb9bc11ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:41.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.339+0000 7fb9c111a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb9bc135e50 0x7fb9bc11bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:41.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.339+0000 7fb9c111a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9bc11e830 con 0x7fb9bc09e1d0 2023-09-30T22:34:41.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.339+0000 7fb9c111a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9bc11f980 con 0x7fb9bc09d6f0 2023-09-30T22:34:41.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.339+0000 7fb9c111a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb9bc11d150 con 0x7fb9bc135e50 2023-09-30T22:34:41.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.339+0000 7fb9bb59e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb9bc135e50 0x7fb9bc11bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:41.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.339+0000 7fb9bb59e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb9bc135e50 0x7fb9bc11bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:59282/0 (socket says 172.21.15.16:59282) 2023-09-30T22:34:41.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.339+0000 7fb9bb59e700 1 -- 172.21.15.16:0/3745704862 learned_addr learned my addr 172.21.15.16:0/3745704862 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:41.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.339+0000 7fb9bad9d700 1 --2- 172.21.15.16:0/3745704862 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb9bc09d6f0 0x7fb9bc11b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:41.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.339+0000 7fb9ba59c700 1 --2- 172.21.15.16:0/3745704862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9bc09e1d0 0x7fb9bc11ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:41.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.340+0000 7fb9bb59e700 1 -- 172.21.15.16:0/3745704862 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb9bc09d6f0 msgr2=0x7fb9bc11b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:41.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.340+0000 7fb9bb59e700 1 --2- 172.21.15.16:0/3745704862 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb9bc09d6f0 0x7fb9bc11b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:41.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.340+0000 7fb9bb59e700 1 -- 172.21.15.16:0/3745704862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9bc09e1d0 msgr2=0x7fb9bc11ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:41.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.340+0000 7fb9bb59e700 1 --2- 172.21.15.16:0/3745704862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9bc09e1d0 0x7fb9bc11ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:41.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.340+0000 7fb9bb59e700 1 -- 172.21.15.16:0/3745704862 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9b00094a0 con 0x7fb9bc135e50 2023-09-30T22:34:41.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.340+0000 7fb9ba59c700 1 --2- 172.21.15.16:0/3745704862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9bc09e1d0 0x7fb9bc11ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:41.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.340+0000 7fb9bb59e700 1 --2- 172.21.15.16:0/3745704862 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb9bc135e50 0x7fb9bc11bfc0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fb9b400efc0 tx=0x7fb9b400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:41.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.340+0000 7fb9a37fe700 1 -- 172.21.15.16:0/3745704862 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b400cc90 con 0x7fb9bc135e50 2023-09-30T22:34:41.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.340+0000 7fb9a37fe700 1 -- 172.21.15.16:0/3745704862 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9b400f040 con 0x7fb9bc135e50 2023-09-30T22:34:41.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.340+0000 7fb9a37fe700 1 -- 172.21.15.16:0/3745704862 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b40146c0 con 0x7fb9bc135e50 2023-09-30T22:34:41.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.340+0000 7fb9c111a700 1 -- 172.21.15.16:0/3745704862 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9bc11ece0 con 0x7fb9bc135e50 2023-09-30T22:34:41.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.341+0000 7fb9c111a700 1 -- 172.21.15.16:0/3745704862 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9bc09c700 con 0x7fb9bc135e50 2023-09-30T22:34:41.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.342+0000 7fb9c111a700 1 -- 172.21.15.16:0/3745704862 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9bc074ea0 con 0x7fb9bc135e50 2023-09-30T22:34:41.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.342+0000 7fb9a37fe700 1 -- 172.21.15.16:0/3745704862 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb9b4014820 con 0x7fb9bc135e50 2023-09-30T22:34:41.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.343+0000 7fb9a37fe700 1 -- 172.21.15.16:0/3745704862 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fb9b4040790 con 0x7fb9bc135e50 2023-09-30T22:34:41.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.347+0000 7fb9a37fe700 1 -- 172.21.15.16:0/3745704862 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb9b4012930 con 0x7fb9bc135e50 2023-09-30T22:34:41.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.377+0000 7f1d021a2700 1 Processor -- start 2023-09-30T22:34:41.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.378+0000 7f1d021a2700 1 -- start start 2023-09-30T22:34:41.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.378+0000 7f1d021a2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1cfc1368c0 0x7f1cfc138ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:41.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.378+0000 7f1d021a2700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1cfc1391e0 0x7f1cfc13b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:41.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.378+0000 7f1d021a2700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cfc099780 con 0x7f1cfc133870 2023-09-30T22:34:41.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.378+0000 7f1d021a2700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cfc09aa80 con 0x7f1cfc1391e0 2023-09-30T22:34:41.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.378+0000 7f1d021a2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1cfc133650 con 0x7f1cfc1368c0 2023-09-30T22:34:41.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.378+0000 7f1d011a0700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1cfc133870 0x7f1cfc133c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58364/0 (socket says 172.21.15.16:58364) 2023-09-30T22:34:41.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.378+0000 7f1d011a0700 1 -- 172.21.15.16:0/427541601 learned_addr learned my addr 172.21.15.16:0/427541601 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:34:41.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.378+0000 7f1d019a1700 1 --2- 172.21.15.16:0/427541601 >> v2:172.21.15.16:3300/0 conn(0x7f1cfc1391e0 0x7f1cfc13b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:41.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.379+0000 7f1d0099f700 1 --2- 172.21.15.16:0/427541601 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1cfc1368c0 0x7f1cfc138ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:41.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.379+0000 7f1cf3fff700 1 -- 172.21.15.16:0/427541601 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1799001919 0 0) 0x7f1cfc099780 con 0x7f1cfc133870 2023-09-30T22:34:41.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.379+0000 7f1cf3fff700 1 -- 172.21.15.16:0/427541601 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ce80035b0 con 0x7f1cfc133870 2023-09-30T22:34:41.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.379+0000 7f1d019a1700 1 -- 172.21.15.16:0/427541601 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1cfc1368c0 msgr2=0x7f1cfc138ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:41.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.379+0000 7f1d019a1700 1 --2- 172.21.15.16:0/427541601 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1cfc1368c0 0x7f1cfc138ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:41.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.379+0000 7f1d019a1700 1 -- 172.21.15.16:0/427541601 >> v1:172.21.15.16:6789/0 conn(0x7f1cfc133870 legacy=0x7f1cfc133c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:41.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.379+0000 7f1d019a1700 1 -- 172.21.15.16:0/427541601 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cfc09ebc0 con 0x7f1cfc1391e0 2023-09-30T22:34:41.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.379+0000 7f1d0099f700 1 --2- 172.21.15.16:0/427541601 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1cfc1368c0 0x7f1cfc138ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:34:41.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.379+0000 7f1d019a1700 1 --2- 172.21.15.16:0/427541601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cfc1391e0 0x7f1cfc13b5c0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4009d90 tx=0x7f1cf400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9adc41dc796149a1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:41.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.379+0000 7f1cf3fff700 1 -- 172.21.15.16:0/427541601 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf400bd20 con 0x7f1cfc1391e0 2023-09-30T22:34:41.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.379+0000 7f1cf3fff700 1 -- 172.21.15.16:0/427541601 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf400be80 con 0x7f1cfc1391e0 2023-09-30T22:34:41.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.379+0000 7f1cf3fff700 1 -- 172.21.15.16:0/427541601 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf4014ad0 con 0x7f1cfc1391e0 2023-09-30T22:34:41.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.380+0000 7f1d021a2700 1 -- 172.21.15.16:0/427541601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cfc1391e0 msgr2=0x7f1cfc13b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:41.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.380+0000 7f1d021a2700 1 --2- 172.21.15.16:0/427541601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cfc1391e0 0x7f1cfc13b5c0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4009d90 tx=0x7f1cf400b330 comp rx=0 tx=0).stop 2023-09-30T22:34:41.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.380+0000 7f1d021a2700 1 -- 172.21.15.16:0/427541601 shutdown_connections 2023-09-30T22:34:41.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.380+0000 7f1d021a2700 1 --2- 172.21.15.16:0/427541601 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cfc1391e0 0x7f1cfc13b5c0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:41.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.380+0000 7f1d021a2700 1 --2- 172.21.15.16:0/427541601 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1cfc1368c0 0x7f1cfc138ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:41.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.380+0000 7f1d021a2700 1 -- 172.21.15.16:0/427541601 >> 172.21.15.16:0/427541601 conn(0x7f1cfc0a2540 msgr2=0x7f1cfc0a2940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:41.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.381+0000 7f1d021a2700 1 -- 172.21.15.16:0/427541601 shutdown_connections 2023-09-30T22:34:41.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.381+0000 7f1d021a2700 1 -- 172.21.15.16:0/427541601 wait complete. 2023-09-30T22:34:41.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.382+0000 7f1d021a2700 1 Processor -- start 2023-09-30T22:34:41.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.382+0000 7f1d021a2700 1 -- start start 2023-09-30T22:34:41.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.382+0000 7f1d021a2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1cfc133870 0x7f1cfc1276e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:41.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.382+0000 7f1d021a2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1cfc1368c0 0x7f1cfc127c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:41.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.382+0000 7f1d021a2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cfc1391e0 0x7f1cfc128160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:41.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.382+0000 7f1d021a2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cfc128ab0 con 0x7f1cfc1391e0 2023-09-30T22:34:41.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.382+0000 7f1d021a2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cfc09bfd0 con 0x7f1cfc1368c0 2023-09-30T22:34:41.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.382+0000 7f1d021a2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1cfc12afd0 con 0x7f1cfc133870 2023-09-30T22:34:41.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.382+0000 7f1d019a1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cfc1391e0 0x7f1cfc128160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:41.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.382+0000 7f1d019a1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cfc1391e0 0x7f1cfc128160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56608/0 (socket says 172.21.15.16:56608) 2023-09-30T22:34:41.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.382+0000 7f1d019a1700 1 -- 172.21.15.16:0/3969925878 learned_addr learned my addr 172.21.15.16:0/3969925878 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:41.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.382+0000 7f1d0099f700 1 --2- 172.21.15.16:0/3969925878 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1cfc1368c0 0x7f1cfc127c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:41.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.383+0000 7f1d019a1700 1 -- 172.21.15.16:0/3969925878 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1cfc133870 msgr2=0x7f1cfc1276e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:41.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.383+0000 7f1d019a1700 1 --2- 172.21.15.16:0/3969925878 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1cfc133870 0x7f1cfc1276e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:41.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.383+0000 7f1d019a1700 1 -- 172.21.15.16:0/3969925878 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1cfc1368c0 msgr2=0x7f1cfc127c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:41.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.383+0000 7f1d019a1700 1 --2- 172.21.15.16:0/3969925878 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1cfc1368c0 0x7f1cfc127c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:41.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.383+0000 7f1d019a1700 1 -- 172.21.15.16:0/3969925878 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf4009a70 con 0x7f1cfc1391e0 2023-09-30T22:34:41.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.383+0000 7f1d019a1700 1 --2- 172.21.15.16:0/3969925878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cfc1391e0 0x7f1cfc128160 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f1cf401b000 tx=0x7f1cf4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:41.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.383+0000 7f1cf1ffb700 1 -- 172.21.15.16:0/3969925878 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf4004450 con 0x7f1cfc1391e0 2023-09-30T22:34:41.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.383+0000 7f1cf1ffb700 1 -- 172.21.15.16:0/3969925878 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf40045b0 con 0x7f1cfc1391e0 2023-09-30T22:34:41.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.383+0000 7f1d021a2700 1 -- 172.21.15.16:0/3969925878 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cfc12b3d0 con 0x7f1cfc1391e0 2023-09-30T22:34:41.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.383+0000 7f1cf1ffb700 1 -- 172.21.15.16:0/3969925878 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf4004960 con 0x7f1cfc1391e0 2023-09-30T22:34:41.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.383+0000 7f1d021a2700 1 -- 172.21.15.16:0/3969925878 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cfc12a930 con 0x7f1cfc1391e0 2023-09-30T22:34:41.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.385+0000 7f1d021a2700 1 -- 172.21.15.16:0/3969925878 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cfc04f2a0 con 0x7f1cfc1391e0 2023-09-30T22:34:41.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.385+0000 7f1cf1ffb700 1 -- 172.21.15.16:0/3969925878 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1cf401c440 con 0x7f1cfc1391e0 2023-09-30T22:34:41.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.385+0000 7f1cf1ffb700 1 -- 172.21.15.16:0/3969925878 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f1cf4049f20 con 0x7f1cfc1391e0 2023-09-30T22:34:41.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.394+0000 7f1cf1ffb700 1 -- 172.21.15.16:0/3969925878 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1cf4003e10 con 0x7f1cfc1391e0 2023-09-30T22:34:41.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.421+0000 7f0a9453e700 1 -- 172.21.15.16:0/2083853427 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0a8c06ba90 con 0x7f0a8c1367c0 2023-09-30T22:34:41.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.422+0000 7f0a7effd700 1 -- 172.21.15.16:0/2083853427 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0a8002b070 con 0x7f0a8c1367c0 2023-09-30T22:34:41.423 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:34:41.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.425+0000 7f0a9453e700 1 -- 172.21.15.16:0/2083853427 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0a8c1367c0 msgr2=0x7f0a8c1c8980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:41.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.425+0000 7f0a9453e700 1 --2- 172.21.15.16:0/2083853427 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0a8c1367c0 0x7f0a8c1c8980 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f0a80009fd0 tx=0x7f0a80004750 comp rx=0 tx=0).stop 2023-09-30T22:34:41.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.425+0000 7f0a9453e700 1 -- 172.21.15.16:0/2083853427 shutdown_connections 2023-09-30T22:34:41.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.425+0000 7f0a9453e700 1 --2- 172.21.15.16:0/2083853427 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0a8c133960 0x7f0a8c1c8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:41.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.425+0000 7f0a9453e700 1 --2- 172.21.15.16:0/2083853427 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a8c1390e0 0x7f0a8c119aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:41.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.425+0000 7f0a9453e700 1 --2- 172.21.15.16:0/2083853427 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0a8c1367c0 0x7f0a8c1c8980 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:41.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.425+0000 7f0a9453e700 1 -- 172.21.15.16:0/2083853427 >> 172.21.15.16:0/2083853427 conn(0x7f0a8c0a2610 msgr2=0x7f0a8c070970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:41.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.425+0000 7f0a9453e700 1 -- 172.21.15.16:0/2083853427 shutdown_connections 2023-09-30T22:34:41.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.425+0000 7f0a9453e700 1 -- 172.21.15.16:0/2083853427 wait complete. 2023-09-30T22:34:41.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.428+0000 7f1efd5ed700 1 Processor -- start 2023-09-30T22:34:41.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.428+0000 7f1efd5ed700 1 -- start start 2023-09-30T22:34:41.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.429+0000 7f1efd5ed700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1ef809d6f0 0x7f1ef809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:41.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.429+0000 7f1efd5ed700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ef809e000 0x7f1ef81339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:41.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.429+0000 7f1efd5ed700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ef809e410 0x7f1ef8133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:41.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.429+0000 7f1efd5ed700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ef806b460 con 0x7f1ef809d6f0 2023-09-30T22:34:41.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.429+0000 7f1efd5ed700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef809b120 con 0x7f1ef809e410 2023-09-30T22:34:41.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.429+0000 7f1efd5ed700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef809a7e0 con 0x7f1ef809e000 2023-09-30T22:34:41.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.429+0000 7f1ef6ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1ef809d6f0 0x7f1ef809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:41.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.429+0000 7f1ef6ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1ef809d6f0 0x7f1ef809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56622/0 (socket says 172.21.15.16:56622) 2023-09-30T22:34:41.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.429+0000 7f1ef6ffd700 1 -- 172.21.15.16:0/1114129290 learned_addr learned my addr 172.21.15.16:0/1114129290 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:41.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.429+0000 7f1ef6ffd700 1 -- 172.21.15.16:0/1114129290 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ef809e000 msgr2=0x7f1ef81339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:41.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.429+0000 7f1ef6ffd700 1 --2- 172.21.15.16:0/1114129290 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ef809e000 0x7f1ef81339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:41.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.429+0000 7f1ef6ffd700 1 -- 172.21.15.16:0/1114129290 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ef809e410 msgr2=0x7f1ef8133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:41.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.429+0000 7f1ef6ffd700 1 --2- 172.21.15.16:0/1114129290 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ef809e410 0x7f1ef8133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:41.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.429+0000 7f1ef6ffd700 1 -- 172.21.15.16:0/1114129290 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef809a500 con 0x7f1ef809d6f0 2023-09-30T22:34:41.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.430+0000 7f1ef6ffd700 1 --2- 172.21.15.16:0/1114129290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ef809d6f0 0x7f1ef809dac0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f1ee80098f0 tx=0x7f1ee8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f22733930ebcec04 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:41.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.430+0000 7f1ef5ffb700 1 -- 172.21.15.16:0/1114129290 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee800b8a0 con 0x7f1ef809d6f0 2023-09-30T22:34:41.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.430+0000 7f1ef5ffb700 1 -- 172.21.15.16:0/1114129290 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ee800ba00 con 0x7f1ef809d6f0 2023-09-30T22:34:41.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.430+0000 7f1ef5ffb700 1 -- 172.21.15.16:0/1114129290 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee800bd90 con 0x7f1ef809d6f0 2023-09-30T22:34:41.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.431+0000 7f1efd5ed700 1 -- 172.21.15.16:0/1114129290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ef809d6f0 msgr2=0x7f1ef809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:41.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.431+0000 7f1efd5ed700 1 --2- 172.21.15.16:0/1114129290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ef809d6f0 0x7f1ef809dac0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f1ee80098f0 tx=0x7f1ee8009c00 comp rx=0 tx=0).stop 2023-09-30T22:34:41.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.431+0000 7f1efd5ed700 1 -- 172.21.15.16:0/1114129290 shutdown_connections 2023-09-30T22:34:41.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.431+0000 7f1efd5ed700 1 --2- 172.21.15.16:0/1114129290 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ef809e410 0x7f1ef8133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:41.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.431+0000 7f1efd5ed700 1 --2- 172.21.15.16:0/1114129290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ef809d6f0 0x7f1ef809dac0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:41.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.431+0000 7f1efd5ed700 1 --2- 172.21.15.16:0/1114129290 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ef809e000 0x7f1ef81339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:41.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.431+0000 7f1efd5ed700 1 -- 172.21.15.16:0/1114129290 >> 172.21.15.16:0/1114129290 conn(0x7f1ef8066f50 msgr2=0x7f1ef809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:41.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.431+0000 7f1efd5ed700 1 -- 172.21.15.16:0/1114129290 shutdown_connections 2023-09-30T22:34:41.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.431+0000 7f1efd5ed700 1 -- 172.21.15.16:0/1114129290 wait complete. 2023-09-30T22:34:41.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.433+0000 7f1efd5ed700 1 Processor -- start 2023-09-30T22:34:41.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.433+0000 7f1efd5ed700 1 -- start start 2023-09-30T22:34:41.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.434+0000 7f1efd5ed700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ef809d6f0 0x7f1ef81a5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:41.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.434+0000 7f1efd5ed700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ef809e000 0x7f1ef81a62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:41.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.434+0000 7f1efd5ed700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ef809e410 0x7f1ef81a6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:41.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.434+0000 7f1efd5ed700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef80a2320 con 0x7f1ef809e000 2023-09-30T22:34:41.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.434+0000 7f1efd5ed700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef81a7ab0 con 0x7f1ef809e410 2023-09-30T22:34:41.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.434+0000 7f1efd5ed700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef809c710 con 0x7f1ef809d6f0 2023-09-30T22:34:41.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.434+0000 7f1ef67fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ef809e000 0x7f1ef81a62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:41.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.434+0000 7f1ef67fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ef809e000 0x7f1ef81a62d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56630/0 (socket says 172.21.15.16:56630) 2023-09-30T22:34:41.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.434+0000 7f1ef67fc700 1 -- 172.21.15.16:0/1821358968 learned_addr learned my addr 172.21.15.16:0/1821358968 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:41.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.434+0000 7f1ef6ffd700 1 --2- 172.21.15.16:0/1821358968 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ef809d6f0 0x7f1ef81a5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:41.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.434+0000 7f1ef77fe700 1 --2- 172.21.15.16:0/1821358968 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ef809e410 0x7f1ef81a6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:41.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.434+0000 7f1ef67fc700 1 -- 172.21.15.16:0/1821358968 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ef809d6f0 msgr2=0x7f1ef81a5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:41.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.434+0000 7f1ef67fc700 1 --2- 172.21.15.16:0/1821358968 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ef809d6f0 0x7f1ef81a5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:41.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.434+0000 7f1ef67fc700 1 -- 172.21.15.16:0/1821358968 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ef809e410 msgr2=0x7f1ef81a6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:41.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.434+0000 7f1ef67fc700 1 --2- 172.21.15.16:0/1821358968 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ef809e410 0x7f1ef81a6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:41.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.434+0000 7f1ef67fc700 1 -- 172.21.15.16:0/1821358968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee8009550 con 0x7f1ef809e000 2023-09-30T22:34:41.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.434+0000 7f1ef77fe700 1 --2- 172.21.15.16:0/1821358968 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ef809e410 0x7f1ef81a6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:41.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.434+0000 7f1ef67fc700 1 --2- 172.21.15.16:0/1821358968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ef809e000 0x7f1ef81a62d0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f1eec00b530 tx=0x7f1eec00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:41.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.435+0000 7f1edf7fe700 1 -- 172.21.15.16:0/1821358968 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eec011830 con 0x7f1ef809e000 2023-09-30T22:34:41.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.435+0000 7f1edf7fe700 1 -- 172.21.15.16:0/1821358968 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eec011990 con 0x7f1ef809e000 2023-09-30T22:34:41.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.435+0000 7f1edf7fe700 1 -- 172.21.15.16:0/1821358968 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eec011d40 con 0x7f1ef809e000 2023-09-30T22:34:41.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.435+0000 7f1efd5ed700 1 -- 172.21.15.16:0/1821358968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ef81a8c60 con 0x7f1ef809e000 2023-09-30T22:34:41.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.435+0000 7f1efd5ed700 1 -- 172.21.15.16:0/1821358968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ef81a98b0 con 0x7f1ef809e000 2023-09-30T22:34:41.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.436+0000 7f1efd5ed700 1 -- 172.21.15.16:0/1821358968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ef8074ea0 con 0x7f1ef809e000 2023-09-30T22:34:41.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.437+0000 7f1edf7fe700 1 -- 172.21.15.16:0/1821358968 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1eec01a400 con 0x7f1ef809e000 2023-09-30T22:34:41.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.437+0000 7f1edf7fe700 1 -- 172.21.15.16:0/1821358968 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f1eec03fd80 con 0x7f1ef809e000 2023-09-30T22:34:41.442 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:34:41.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.445+0000 7f1edf7fe700 1 -- 172.21.15.16:0/1821358968 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1eec01ae60 con 0x7f1ef809e000 2023-09-30T22:34:41.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.566+0000 7fb9c111a700 1 -- 172.21.15.16:0/3745704862 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb9bc06c360 con 0x7fb9bc135e50 2023-09-30T22:34:41.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.567+0000 7fb9a37fe700 1 -- 172.21.15.16:0/3745704862 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb9b402a070 con 0x7fb9bc135e50 2023-09-30T22:34:41.567 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:34:41.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.569+0000 7fb9c111a700 1 -- 172.21.15.16:0/3745704862 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb9bc135e50 msgr2=0x7fb9bc11bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:41.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.569+0000 7fb9c111a700 1 --2- 172.21.15.16:0/3745704862 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb9bc135e50 0x7fb9bc11bfc0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fb9b400efc0 tx=0x7fb9b400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:34:41.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.569+0000 7fb9c111a700 1 -- 172.21.15.16:0/3745704862 shutdown_connections 2023-09-30T22:34:41.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.569+0000 7fb9c111a700 1 --2- 172.21.15.16:0/3745704862 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb9bc09d6f0 0x7fb9bc11b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:41.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.569+0000 7fb9c111a700 1 --2- 172.21.15.16:0/3745704862 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb9bc135e50 0x7fb9bc11bfc0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:41.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.569+0000 7fb9c111a700 1 --2- 172.21.15.16:0/3745704862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9bc09e1d0 0x7fb9bc11ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:41.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.569+0000 7fb9c111a700 1 -- 172.21.15.16:0/3745704862 >> 172.21.15.16:0/3745704862 conn(0x7fb9bc066f50 msgr2=0x7fb9bc0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:41.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.569+0000 7fb9c111a700 1 -- 172.21.15.16:0/3745704862 shutdown_connections 2023-09-30T22:34:41.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.569+0000 7fb9c111a700 1 -- 172.21.15.16:0/3745704862 wait complete. 2023-09-30T22:34:41.583 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:34:41.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.614+0000 7f1d021a2700 1 -- 172.21.15.16:0/3969925878 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1cfc06b7c0 con 0x7f1cfc1391e0 2023-09-30T22:34:41.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.614+0000 7f1cf1ffb700 1 -- 172.21.15.16:0/3969925878 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1cf4037b40 con 0x7f1cfc1391e0 2023-09-30T22:34:41.615 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:34:41.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.617+0000 7f1d021a2700 1 -- 172.21.15.16:0/3969925878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cfc1391e0 msgr2=0x7f1cfc128160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:41.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.617+0000 7f1d021a2700 1 --2- 172.21.15.16:0/3969925878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cfc1391e0 0x7f1cfc128160 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f1cf401b000 tx=0x7f1cf4004ad0 comp rx=0 tx=0).stop 2023-09-30T22:34:41.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.617+0000 7f1d021a2700 1 -- 172.21.15.16:0/3969925878 shutdown_connections 2023-09-30T22:34:41.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.617+0000 7f1d021a2700 1 --2- 172.21.15.16:0/3969925878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1cfc1391e0 0x7f1cfc128160 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:41.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.617+0000 7f1d021a2700 1 --2- 172.21.15.16:0/3969925878 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1cfc133870 0x7f1cfc1276e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:41.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.617+0000 7f1d021a2700 1 --2- 172.21.15.16:0/3969925878 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1cfc1368c0 0x7f1cfc127c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:41.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.617+0000 7f1d021a2700 1 -- 172.21.15.16:0/3969925878 >> 172.21.15.16:0/3969925878 conn(0x7f1cfc0a2540 msgr2=0x7f1cfc0a2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:41.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.617+0000 7f1d021a2700 1 -- 172.21.15.16:0/3969925878 shutdown_connections 2023-09-30T22:34:41.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.617+0000 7f1d021a2700 1 -- 172.21.15.16:0/3969925878 wait complete. 2023-09-30T22:34:41.631 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:34:41.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.652+0000 7f1efd5ed700 1 -- 172.21.15.16:0/1821358968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1ef81a9530 con 0x7f1ef809e000 2023-09-30T22:34:41.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.652+0000 7f1edf7fe700 1 -- 172.21.15.16:0/1821358968 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1eec028070 con 0x7f1ef809e000 2023-09-30T22:34:41.653 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:34:41.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.654+0000 7f1efd5ed700 1 -- 172.21.15.16:0/1821358968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ef809e000 msgr2=0x7f1ef81a62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:41.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.654+0000 7f1efd5ed700 1 --2- 172.21.15.16:0/1821358968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ef809e000 0x7f1ef81a62d0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f1eec00b530 tx=0x7f1eec00b840 comp rx=0 tx=0).stop 2023-09-30T22:34:41.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.654+0000 7f1efd5ed700 1 -- 172.21.15.16:0/1821358968 shutdown_connections 2023-09-30T22:34:41.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.655+0000 7f1efd5ed700 1 --2- 172.21.15.16:0/1821358968 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ef809e410 0x7f1ef81a6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:41.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.655+0000 7f1efd5ed700 1 --2- 172.21.15.16:0/1821358968 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ef809d6f0 0x7f1ef81a5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:41.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.655+0000 7f1efd5ed700 1 --2- 172.21.15.16:0/1821358968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ef809e000 0x7f1ef81a62d0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:41.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.655+0000 7f1efd5ed700 1 -- 172.21.15.16:0/1821358968 >> 172.21.15.16:0/1821358968 conn(0x7f1ef8066f50 msgr2=0x7f1ef81bac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:41.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.655+0000 7f1efd5ed700 1 -- 172.21.15.16:0/1821358968 shutdown_connections 2023-09-30T22:34:41.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:41.655+0000 7f1efd5ed700 1 -- 172.21.15.16:0/1821358968 wait complete. 2023-09-30T22:34:41.669 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:34:42.444 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:34:42.584 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:34:42.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.589+0000 7f1505e3f700 1 Processor -- start 2023-09-30T22:34:42.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.589+0000 7f1505e3f700 1 -- start start 2023-09-30T22:34:42.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.589+0000 7f1505e3f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1500131810 0x7f1500131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:42.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.589+0000 7f1505e3f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f150009a580 0x7f1500139660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:42.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.589+0000 7f1505e3f700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15001313c0 con 0x7f1500099a40 2023-09-30T22:34:42.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.589+0000 7f1505e3f700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f150006e8c0 con 0x7f1500131810 2023-09-30T22:34:42.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.589+0000 7f1505e3f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1500071d20 con 0x7f150009a580 2023-09-30T22:34:42.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.589+0000 7f14fffff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f150009a580 0x7f1500139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:42.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.589+0000 7f14ff7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1500131810 0x7f1500131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:42.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.589+0000 7f14fffff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f150009a580 0x7f1500139660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:51568/0 (socket says 172.21.15.16:51568) 2023-09-30T22:34:42.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.590+0000 7f14fffff700 1 -- 172.21.15.16:0/1131128591 learned_addr learned my addr 172.21.15.16:0/1131128591 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:42.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.590+0000 7f14fffff700 1 -- 172.21.15.16:0/1131128591 >> v2:172.21.15.16:3300/0 conn(0x7f1500131810 msgr2=0x7f1500131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:42.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.590+0000 7f14fffff700 1 --2- 172.21.15.16:0/1131128591 >> v2:172.21.15.16:3300/0 conn(0x7f1500131810 0x7f1500131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:42.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.590+0000 7f14fffff700 1 -- 172.21.15.16:0/1131128591 >> v1:172.21.15.16:6789/0 conn(0x7f1500099a40 legacy=0x7f1500099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:42.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.590+0000 7f14fffff700 1 -- 172.21.15.16:0/1131128591 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15000721a0 con 0x7f150009a580 2023-09-30T22:34:42.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.590+0000 7f14fffff700 1 --2- 172.21.15.16:0/1131128591 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f150009a580 0x7f1500139660 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f14f8009e50 tx=0x7f14f800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6726680b7859a7e0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:42.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.590+0000 7f14fe7fc700 1 -- 172.21.15.16:0/1131128591 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f800e070 con 0x7f150009a580 2023-09-30T22:34:42.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.590+0000 7f14fe7fc700 1 -- 172.21.15.16:0/1131128591 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14f800b910 con 0x7f150009a580 2023-09-30T22:34:42.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.590+0000 7f14fe7fc700 1 -- 172.21.15.16:0/1131128591 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f800bcc0 con 0x7f150009a580 2023-09-30T22:34:42.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.591+0000 7f1505e3f700 1 -- 172.21.15.16:0/1131128591 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f150009a580 msgr2=0x7f1500139660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:42.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.591+0000 7f1505e3f700 1 --2- 172.21.15.16:0/1131128591 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f150009a580 0x7f1500139660 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f14f8009e50 tx=0x7f14f800b330 comp rx=0 tx=0).stop 2023-09-30T22:34:42.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.591+0000 7f1505e3f700 1 -- 172.21.15.16:0/1131128591 shutdown_connections 2023-09-30T22:34:42.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.591+0000 7f1505e3f700 1 --2- 172.21.15.16:0/1131128591 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f150009a580 0x7f1500139660 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:42.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.591+0000 7f1505e3f700 1 --2- 172.21.15.16:0/1131128591 >> v2:172.21.15.16:3300/0 conn(0x7f1500131810 0x7f1500131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:42.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.591+0000 7f1505e3f700 1 -- 172.21.15.16:0/1131128591 >> 172.21.15.16:0/1131128591 conn(0x7f15000a2aa0 msgr2=0x7f15000a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:42.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.591+0000 7f1505e3f700 1 -- 172.21.15.16:0/1131128591 shutdown_connections 2023-09-30T22:34:42.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.591+0000 7f1505e3f700 1 -- 172.21.15.16:0/1131128591 wait complete. 2023-09-30T22:34:42.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.592+0000 7f1505e3f700 1 Processor -- start 2023-09-30T22:34:42.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.592+0000 7f1505e3f700 1 -- start start 2023-09-30T22:34:42.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.593+0000 7f1505e3f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1500099a40 0x7f150006f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:42.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.593+0000 7f1505e3f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f150009a580 0x7f150006fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:42.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.593+0000 7f1505e3f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1500131810 0x7f1500070090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:42.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.593+0000 7f1505e3f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1500121400 con 0x7f1500131810 2023-09-30T22:34:42.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.593+0000 7f1505e3f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1500120b90 con 0x7f1500099a40 2023-09-30T22:34:42.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.593+0000 7f1505e3f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f150012ecb0 con 0x7f150009a580 2023-09-30T22:34:42.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.593+0000 7f14ff7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1500099a40 0x7f150006f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:42.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.593+0000 7f14ff7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1500099a40 0x7f150006f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:51580/0 (socket says 172.21.15.16:51580) 2023-09-30T22:34:42.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.593+0000 7f14ff7fe700 1 -- 172.21.15.16:0/1483833130 learned_addr learned my addr 172.21.15.16:0/1483833130 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:42.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.593+0000 7f14feffd700 1 --2- 172.21.15.16:0/1483833130 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f150009a580 0x7f150006fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:42.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.593+0000 7f14fffff700 1 --2- 172.21.15.16:0/1483833130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1500131810 0x7f1500070090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:42.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.593+0000 7f14ff7fe700 1 -- 172.21.15.16:0/1483833130 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f150009a580 msgr2=0x7f150006fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:42.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.593+0000 7f14ff7fe700 1 --2- 172.21.15.16:0/1483833130 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f150009a580 0x7f150006fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:42.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.593+0000 7f14ff7fe700 1 -- 172.21.15.16:0/1483833130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1500131810 msgr2=0x7f1500070090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:42.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.593+0000 7f14ff7fe700 1 --2- 172.21.15.16:0/1483833130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1500131810 0x7f1500070090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:42.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.593+0000 7f14ff7fe700 1 -- 172.21.15.16:0/1483833130 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14f8009b30 con 0x7f1500099a40 2023-09-30T22:34:42.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.593+0000 7f14feffd700 1 --2- 172.21.15.16:0/1483833130 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f150009a580 0x7f150006fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:42.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.594+0000 7f14ff7fe700 1 --2- 172.21.15.16:0/1483833130 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1500099a40 0x7f150006f610 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f14f000e9e0 tx=0x7f14f000ecf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:42.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.594+0000 7f14e7fff700 1 -- 172.21.15.16:0/1483833130 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f0009710 con 0x7f1500099a40 2023-09-30T22:34:42.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.594+0000 7f14e7fff700 1 -- 172.21.15.16:0/1483833130 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14f000c850 con 0x7f1500099a40 2023-09-30T22:34:42.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.594+0000 7f1505e3f700 1 -- 172.21.15.16:0/1483833130 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1500120e60 con 0x7f1500099a40 2023-09-30T22:34:42.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.594+0000 7f14e7fff700 1 -- 172.21.15.16:0/1483833130 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f000cbc0 con 0x7f1500099a40 2023-09-30T22:34:42.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.594+0000 7f1505e3f700 1 -- 172.21.15.16:0/1483833130 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f150006eb00 con 0x7f1500099a40 2023-09-30T22:34:42.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.595+0000 7f14e7fff700 1 -- 172.21.15.16:0/1483833130 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f14f000cd20 con 0x7f1500099a40 2023-09-30T22:34:42.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.595+0000 7f14e7fff700 1 -- 172.21.15.16:0/1483833130 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f14f003fc20 con 0x7f1500099a40 2023-09-30T22:34:42.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.595+0000 7f1505e3f700 1 -- 172.21.15.16:0/1483833130 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1500074ea0 con 0x7f1500099a40 2023-09-30T22:34:42.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.600+0000 7f14e7fff700 1 -- 172.21.15.16:0/1483833130 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f14f0021080 con 0x7f1500099a40 2023-09-30T22:34:42.631 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:34:42.669 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:34:42.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.729+0000 7f193b919700 1 Processor -- start 2023-09-30T22:34:42.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.729+0000 7f193b919700 1 -- start start 2023-09-30T22:34:42.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.730+0000 7f193b919700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f193409e930 0x7f193409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:42.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.730+0000 7f193b919700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f193409f240 0x7f19341317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:42.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.730+0000 7f193b919700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1934070940 con 0x7f193409f650 2023-09-30T22:34:42.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.730+0000 7f193b919700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f193406fd60 con 0x7f193409f240 2023-09-30T22:34:42.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.730+0000 7f193b919700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f193406fa80 con 0x7f193409e930 2023-09-30T22:34:42.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.730+0000 7f1938eb4700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f193409f240 0x7f19341317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:42.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.730+0000 7f19396b5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f193409e930 0x7f193409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:42.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.730+0000 7f1938eb4700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f193409f240 0x7f19341317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:51588/0 (socket says 172.21.15.16:51588) 2023-09-30T22:34:42.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.730+0000 7f1938eb4700 1 -- 172.21.15.16:0/2601011524 learned_addr learned my addr 172.21.15.16:0/2601011524 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:42.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.731+0000 7f1938eb4700 1 -- 172.21.15.16:0/2601011524 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f193409e930 msgr2=0x7f193409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:42.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.731+0000 7f192bfff700 1 -- 172.21.15.16:0/2601011524 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1082724075 0 0) 0x7f1934070940 con 0x7f193409f650 2023-09-30T22:34:42.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.731+0000 7f1938eb4700 1 --2- 172.21.15.16:0/2601011524 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f193409e930 0x7f193409ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:42.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.731+0000 7f1938eb4700 1 -- 172.21.15.16:0/2601011524 >> v1:172.21.15.16:6789/0 conn(0x7f193409f650 legacy=0x7f1934131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:42.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.731+0000 7f1938eb4700 1 -- 172.21.15.16:0/2601011524 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f193409fb60 con 0x7f193409f240 2023-09-30T22:34:42.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.731+0000 7f1938eb4700 1 --2- 172.21.15.16:0/2601011524 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f193409f240 0x7f19341317f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f1924009920 tx=0x7f1924009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fcb179bb5c9bf809 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:42.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.731+0000 7f192bfff700 1 -- 172.21.15.16:0/2601011524 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1924011830 con 0x7f193409f240 2023-09-30T22:34:42.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.731+0000 7f192bfff700 1 -- 172.21.15.16:0/2601011524 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1924011990 con 0x7f193409f240 2023-09-30T22:34:42.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.731+0000 7f192bfff700 1 -- 172.21.15.16:0/2601011524 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1924011d20 con 0x7f193409f240 2023-09-30T22:34:42.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.732+0000 7f193b919700 1 -- 172.21.15.16:0/2601011524 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f193409f240 msgr2=0x7f19341317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:42.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.732+0000 7f193b919700 1 --2- 172.21.15.16:0/2601011524 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f193409f240 0x7f19341317f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f1924009920 tx=0x7f1924009c30 comp rx=0 tx=0).stop 2023-09-30T22:34:42.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.732+0000 7f193b919700 1 -- 172.21.15.16:0/2601011524 shutdown_connections 2023-09-30T22:34:42.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.732+0000 7f193b919700 1 --2- 172.21.15.16:0/2601011524 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f193409e930 0x7f193409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:42.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.732+0000 7f193b919700 1 --2- 172.21.15.16:0/2601011524 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f193409f240 0x7f19341317f0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:42.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.732+0000 7f193b919700 1 -- 172.21.15.16:0/2601011524 >> 172.21.15.16:0/2601011524 conn(0x7f19340a4210 msgr2=0x7f19340a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:42.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.732+0000 7f193b919700 1 -- 172.21.15.16:0/2601011524 shutdown_connections 2023-09-30T22:34:42.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.732+0000 7f193b919700 1 -- 172.21.15.16:0/2601011524 wait complete. 2023-09-30T22:34:42.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.733+0000 7f193b919700 1 Processor -- start 2023-09-30T22:34:42.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.733+0000 7f193b919700 1 -- start start 2023-09-30T22:34:42.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.734+0000 7f193b919700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f193409e930 0x7f19341a1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:42.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.734+0000 7f193b919700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f193409f240 0x7f19341a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:42.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.734+0000 7f193b919700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f193409f650 0x7f19341c3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:42.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.734+0000 7f193b919700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19341b4f50 con 0x7f193409e930 2023-09-30T22:34:42.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.734+0000 7f193b919700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19341a41e0 con 0x7f193409f650 2023-09-30T22:34:42.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.734+0000 7f193b919700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f19341a0b30 con 0x7f193409f240 2023-09-30T22:34:42.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.734+0000 7f1938eb4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f193409f240 0x7f19341a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:42.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.734+0000 7f1939eb6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f193409f650 0x7f19341c3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:42.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.734+0000 7f1938eb4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f193409f240 0x7f19341a16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:59342/0 (socket says 172.21.15.16:59342) 2023-09-30T22:34:42.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.734+0000 7f1939eb6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f193409f650 0x7f19341c3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:51596/0 (socket says 172.21.15.16:51596) 2023-09-30T22:34:42.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.734+0000 7f1938eb4700 1 -- 172.21.15.16:0/2868559577 learned_addr learned my addr 172.21.15.16:0/2868559577 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:42.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.734+0000 7f19396b5700 1 --2- 172.21.15.16:0/2868559577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f193409e930 0x7f19341a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:42.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.735+0000 7f1939eb6700 1 -- 172.21.15.16:0/2868559577 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f193409f240 msgr2=0x7f19341a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:42.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.735+0000 7f1939eb6700 1 --2- 172.21.15.16:0/2868559577 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f193409f240 0x7f19341a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:42.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.735+0000 7f1939eb6700 1 -- 172.21.15.16:0/2868559577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f193409e930 msgr2=0x7f19341a1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:42.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.735+0000 7f1939eb6700 1 --2- 172.21.15.16:0/2868559577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f193409e930 0x7f19341a1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:42.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.735+0000 7f1939eb6700 1 -- 172.21.15.16:0/2868559577 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1924009550 con 0x7f193409f650 2023-09-30T22:34:42.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.735+0000 7f1939eb6700 1 --2- 172.21.15.16:0/2868559577 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f193409f650 0x7f19341c3d90 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f193409faf0 tx=0x7f192c00caa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:42.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.735+0000 7f1929ffb700 1 -- 172.21.15.16:0/2868559577 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192c015e90 con 0x7f193409f650 2023-09-30T22:34:42.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.735+0000 7f1929ffb700 1 -- 172.21.15.16:0/2868559577 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f192c003a00 con 0x7f193409f650 2023-09-30T22:34:42.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.735+0000 7f1929ffb700 1 -- 172.21.15.16:0/2868559577 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192c003d90 con 0x7f193409f650 2023-09-30T22:34:42.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.735+0000 7f193b919700 1 -- 172.21.15.16:0/2868559577 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19340a0190 con 0x7f193409f650 2023-09-30T22:34:42.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.736+0000 7f193b919700 1 -- 172.21.15.16:0/2868559577 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1934071d20 con 0x7f193409f650 2023-09-30T22:34:42.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.737+0000 7f1929ffb700 1 -- 172.21.15.16:0/2868559577 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f192c01b070 con 0x7f193409f650 2023-09-30T22:34:42.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.737+0000 7f1929ffb700 1 -- 172.21.15.16:0/2868559577 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f192c047540 con 0x7f193409f650 2023-09-30T22:34:42.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.737+0000 7f193b919700 1 -- 172.21.15.16:0/2868559577 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1934074ea0 con 0x7f193409f650 2023-09-30T22:34:42.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.746+0000 7f1929ffb700 1 -- 172.21.15.16:0/2868559577 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f192c023070 con 0x7f193409f650 2023-09-30T22:34:42.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.779+0000 7f81e12a7700 1 Processor -- start 2023-09-30T22:34:42.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.779+0000 7f81e12a7700 1 -- start start 2023-09-30T22:34:42.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.780+0000 7f81e12a7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f81dc09d6f0 0x7f81dc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:42.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.780+0000 7f81e12a7700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f81dc09e410 0x7f81dc130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:42.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.780+0000 7f81e12a7700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81dc09f130 con 0x7f81dc09e000 2023-09-30T22:34:42.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.780+0000 7f81e12a7700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81dc09f520 con 0x7f81dc09e410 2023-09-30T22:34:42.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.780+0000 7f81e12a7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81dc09e930 con 0x7f81dc09d6f0 2023-09-30T22:34:42.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.780+0000 7f81daffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f81dc09d6f0 0x7f81dc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:42.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.780+0000 7f81daffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f81dc09d6f0 0x7f81dc09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:51610/0 (socket says 172.21.15.16:51610) 2023-09-30T22:34:42.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.780+0000 7f81daffd700 1 -- 172.21.15.16:0/1956395562 learned_addr learned my addr 172.21.15.16:0/1956395562 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:42.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.780+0000 7f81db7fe700 1 --2- 172.21.15.16:0/1956395562 >> v2:172.21.15.16:3300/0 conn(0x7f81dc09e410 0x7f81dc130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:42.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.780+0000 7f81da7fc700 1 --1- 172.21.15.16:0/1956395562 >> v1:172.21.15.16:6789/0 conn(0x7f81dc09e000 0x7f81dc130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58398/0 (socket says 172.21.15.16:58398) 2023-09-30T22:34:42.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.780+0000 7f81daffd700 1 -- 172.21.15.16:0/1956395562 >> v2:172.21.15.16:3300/0 conn(0x7f81dc09e410 msgr2=0x7f81dc130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:42.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.780+0000 7f81daffd700 1 --2- 172.21.15.16:0/1956395562 >> v2:172.21.15.16:3300/0 conn(0x7f81dc09e410 0x7f81dc130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:42.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.780+0000 7f81daffd700 1 -- 172.21.15.16:0/1956395562 >> v1:172.21.15.16:6789/0 conn(0x7f81dc09e000 legacy=0x7f81dc130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:42.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.780+0000 7f81daffd700 1 -- 172.21.15.16:0/1956395562 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81dc09b2d0 con 0x7f81dc09d6f0 2023-09-30T22:34:42.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.781+0000 7f81daffd700 1 --2- 172.21.15.16:0/1956395562 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f81dc09d6f0 0x7f81dc09dac0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f81cc00bb90 tx=0x7f81cc00bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5932828aa1683f46 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:42.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.781+0000 7f81d9ffb700 1 -- 172.21.15.16:0/1956395562 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81cc00dd50 con 0x7f81dc09d6f0 2023-09-30T22:34:42.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.781+0000 7f81d9ffb700 1 -- 172.21.15.16:0/1956395562 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81cc010470 con 0x7f81dc09d6f0 2023-09-30T22:34:42.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.781+0000 7f81d9ffb700 1 -- 172.21.15.16:0/1956395562 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81cc010800 con 0x7f81dc09d6f0 2023-09-30T22:34:42.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.782+0000 7f81e12a7700 1 -- 172.21.15.16:0/1956395562 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f81dc09d6f0 msgr2=0x7f81dc09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:42.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.782+0000 7f81e12a7700 1 --2- 172.21.15.16:0/1956395562 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f81dc09d6f0 0x7f81dc09dac0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f81cc00bb90 tx=0x7f81cc00bea0 comp rx=0 tx=0).stop 2023-09-30T22:34:42.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.782+0000 7f81e12a7700 1 -- 172.21.15.16:0/1956395562 shutdown_connections 2023-09-30T22:34:42.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.782+0000 7f81e12a7700 1 --2- 172.21.15.16:0/1956395562 >> v2:172.21.15.16:3300/0 conn(0x7f81dc09e410 0x7f81dc130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:42.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.782+0000 7f81e12a7700 1 --2- 172.21.15.16:0/1956395562 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f81dc09d6f0 0x7f81dc09dac0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:42.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.782+0000 7f81e12a7700 1 -- 172.21.15.16:0/1956395562 >> 172.21.15.16:0/1956395562 conn(0x7f81dc066f50 msgr2=0x7f81dc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:42.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.782+0000 7f81e12a7700 1 -- 172.21.15.16:0/1956395562 shutdown_connections 2023-09-30T22:34:42.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.782+0000 7f81e12a7700 1 -- 172.21.15.16:0/1956395562 wait complete. 2023-09-30T22:34:42.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.784+0000 7f81e12a7700 1 Processor -- start 2023-09-30T22:34:42.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.784+0000 7f81e12a7700 1 -- start start 2023-09-30T22:34:42.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.785+0000 7f81e12a7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f81dc09d6f0 0x7f81dc11e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:42.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.785+0000 7f81e12a7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f81dc09e000 0x7f81dc11e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:42.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.785+0000 7f81e12a7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81dc09e410 0x7f81dc11eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:42.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.785+0000 7f81e12a7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81dc11f510 con 0x7f81dc09e410 2023-09-30T22:34:42.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.785+0000 7f81e12a7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81dc121b10 con 0x7f81dc09e000 2023-09-30T22:34:42.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.785+0000 7f81e12a7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f81dc1303d0 con 0x7f81dc09d6f0 2023-09-30T22:34:42.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.785+0000 7f81db7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81dc09e410 0x7f81dc11eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:42.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.785+0000 7f81da7fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f81dc09e000 0x7f81dc11e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:42.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.785+0000 7f81db7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81dc09e410 0x7f81dc11eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56664/0 (socket says 172.21.15.16:56664) 2023-09-30T22:34:42.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.785+0000 7f81db7fe700 1 -- 172.21.15.16:0/880231832 learned_addr learned my addr 172.21.15.16:0/880231832 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:42.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.785+0000 7f81db7fe700 1 -- 172.21.15.16:0/880231832 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f81dc09d6f0 msgr2=0x7f81dc11e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:42.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.785+0000 7f81db7fe700 1 --2- 172.21.15.16:0/880231832 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f81dc09d6f0 0x7f81dc11e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:42.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.785+0000 7f81db7fe700 1 -- 172.21.15.16:0/880231832 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f81dc09e000 msgr2=0x7f81dc11e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:42.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.785+0000 7f81db7fe700 1 --2- 172.21.15.16:0/880231832 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f81dc09e000 0x7f81dc11e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:42.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.785+0000 7f81db7fe700 1 -- 172.21.15.16:0/880231832 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81cc00b7f0 con 0x7f81dc09e410 2023-09-30T22:34:42.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.785+0000 7f81da7fc700 1 --2- 172.21.15.16:0/880231832 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f81dc09e000 0x7f81dc11e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:42.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.786+0000 7f81db7fe700 1 --2- 172.21.15.16:0/880231832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81dc09e410 0x7f81dc11eac0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f81d4007e60 tx=0x7f81d400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:42.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.786+0000 7f81c37fe700 1 -- 172.21.15.16:0/880231832 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d400ede0 con 0x7f81dc09e410 2023-09-30T22:34:42.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.786+0000 7f81c37fe700 1 -- 172.21.15.16:0/880231832 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81d4014420 con 0x7f81dc09e410 2023-09-30T22:34:42.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.786+0000 7f81c37fe700 1 -- 172.21.15.16:0/880231832 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d40147d0 con 0x7f81dc09e410 2023-09-30T22:34:42.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.786+0000 7f81e12a7700 1 -- 172.21.15.16:0/880231832 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81dc1201e0 con 0x7f81dc09e410 2023-09-30T22:34:42.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.786+0000 7f81e12a7700 1 -- 172.21.15.16:0/880231832 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81dc121570 con 0x7f81dc09e410 2023-09-30T22:34:42.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.787+0000 7f81c37fe700 1 -- 172.21.15.16:0/880231832 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f81d4014930 con 0x7f81dc09e410 2023-09-30T22:34:42.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.787+0000 7f81e12a7700 1 -- 172.21.15.16:0/880231832 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81dc074ea0 con 0x7f81dc09e410 2023-09-30T22:34:42.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.787+0000 7f81c37fe700 1 -- 172.21.15.16:0/880231832 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f81d403fe10 con 0x7f81dc09e410 2023-09-30T22:34:42.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.793+0000 7f81c37fe700 1 -- 172.21.15.16:0/880231832 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f81d4014be0 con 0x7f81dc09e410 2023-09-30T22:34:42.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.823+0000 7f2de1401700 1 Processor -- start 2023-09-30T22:34:42.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.823+0000 7f2de1401700 1 -- start start 2023-09-30T22:34:42.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.823+0000 7f2de1401700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2ddc06a770 0x7f2ddc133830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:42.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.823+0000 7f2de1401700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2ddc133280 0x7f2ddc133d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:42.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.823+0000 7f2de1401700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ddc09f440 con 0x7f2ddc12fcc0 2023-09-30T22:34:42.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.823+0000 7f2de1401700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ddc06ab40 con 0x7f2ddc06a770 2023-09-30T22:34:42.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.823+0000 7f2de1401700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2ddc09a2c0 con 0x7f2ddc133280 2023-09-30T22:34:42.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.823+0000 7f2ddaffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2ddc06a770 0x7f2ddc133830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:42.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.823+0000 7f2dda7fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2ddc133280 0x7f2ddc133d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:42.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.823+0000 7f2ddaffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2ddc06a770 0x7f2ddc133830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56674/0 (socket says 172.21.15.16:56674) 2023-09-30T22:34:42.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.823+0000 7f2ddaffd700 1 -- 172.21.15.16:0/484845792 learned_addr learned my addr 172.21.15.16:0/484845792 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:42.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.824+0000 7f2ddaffd700 1 -- 172.21.15.16:0/484845792 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2ddc133280 msgr2=0x7f2ddc133d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:42.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.824+0000 7f2ddaffd700 1 --2- 172.21.15.16:0/484845792 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2ddc133280 0x7f2ddc133d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:42.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.824+0000 7f2ddaffd700 1 -- 172.21.15.16:0/484845792 >> v1:172.21.15.16:6789/0 conn(0x7f2ddc12fcc0 legacy=0x7f2ddc130120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:42.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.824+0000 7f2ddaffd700 1 -- 172.21.15.16:0/484845792 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ddc06b250 con 0x7f2ddc06a770 2023-09-30T22:34:42.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.824+0000 7f2ddaffd700 1 --2- 172.21.15.16:0/484845792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ddc06a770 0x7f2ddc133830 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc0098f0 tx=0x7f2dcc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad46ea4be0675ce6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:42.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.824+0000 7f2dd9ffb700 1 -- 172.21.15.16:0/484845792 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc011830 con 0x7f2ddc06a770 2023-09-30T22:34:42.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.824+0000 7f2dd9ffb700 1 -- 172.21.15.16:0/484845792 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dcc011990 con 0x7f2ddc06a770 2023-09-30T22:34:42.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.824+0000 7f2dd9ffb700 1 -- 172.21.15.16:0/484845792 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc011d20 con 0x7f2ddc06a770 2023-09-30T22:34:42.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.825+0000 7f1505e3f700 1 -- 172.21.15.16:0/1483833130 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f150006b9e0 con 0x7f1500099a40 2023-09-30T22:34:42.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.825+0000 7f14e7fff700 1 -- 172.21.15.16:0/1483833130 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f14f001a9b0 con 0x7f1500099a40 2023-09-30T22:34:42.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.825+0000 7f2de1401700 1 -- 172.21.15.16:0/484845792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ddc06a770 msgr2=0x7f2ddc133830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:42.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.825+0000 7f2de1401700 1 --2- 172.21.15.16:0/484845792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ddc06a770 0x7f2ddc133830 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc0098f0 tx=0x7f2dcc009c00 comp rx=0 tx=0).stop 2023-09-30T22:34:42.827 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:34:42.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.825+0000 7f2de1401700 1 -- 172.21.15.16:0/484845792 shutdown_connections 2023-09-30T22:34:42.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.825+0000 7f2de1401700 1 --2- 172.21.15.16:0/484845792 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ddc06a770 0x7f2ddc133830 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:42.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.825+0000 7f2de1401700 1 --2- 172.21.15.16:0/484845792 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2ddc133280 0x7f2ddc133d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:42.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.825+0000 7f2de1401700 1 -- 172.21.15.16:0/484845792 >> 172.21.15.16:0/484845792 conn(0x7f2ddc0a2cc0 msgr2=0x7f2ddc0a30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:42.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.825+0000 7f2de1401700 1 -- 172.21.15.16:0/484845792 shutdown_connections 2023-09-30T22:34:42.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.825+0000 7f2de1401700 1 -- 172.21.15.16:0/484845792 wait complete. 2023-09-30T22:34:42.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.827+0000 7f2de1401700 1 Processor -- start 2023-09-30T22:34:42.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.827+0000 7f2de1401700 1 -- start start 2023-09-30T22:34:42.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.828+0000 7f2de1401700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2ddc06a770 0x7f2ddc11d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:42.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.828+0000 7f2de1401700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2ddc12fcc0 0x7f2ddc11dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:42.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.828+0000 7f2de1401700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ddc133280 0x7f2ddc11e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:42.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.828+0000 7f2de1401700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ddc12d0c0 con 0x7f2ddc133280 2023-09-30T22:34:42.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.828+0000 7f2de1401700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ddc11f6c0 con 0x7f2ddc06a770 2023-09-30T22:34:42.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.828+0000 7f2de1401700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2ddc0720e0 con 0x7f2ddc12fcc0 2023-09-30T22:34:42.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.828+0000 7f2ddaffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2ddc06a770 0x7f2ddc11d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:42.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.828+0000 7f2ddaffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2ddc06a770 0x7f2ddc11d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:51640/0 (socket says 172.21.15.16:51640) 2023-09-30T22:34:42.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.828+0000 7f2ddaffd700 1 -- 172.21.15.16:0/693664808 learned_addr learned my addr 172.21.15.16:0/693664808 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:42.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.828+0000 7f2ddb7fe700 1 --2- 172.21.15.16:0/693664808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ddc133280 0x7f2ddc11e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:42.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.828+0000 7f2dda7fc700 1 --2- 172.21.15.16:0/693664808 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2ddc12fcc0 0x7f2ddc11dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:42.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.829+0000 7f1505e3f700 1 -- 172.21.15.16:0/1483833130 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1500099a40 msgr2=0x7f150006f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:42.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.829+0000 7f1505e3f700 1 --2- 172.21.15.16:0/1483833130 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1500099a40 0x7f150006f610 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f14f000e9e0 tx=0x7f14f000ecf0 comp rx=0 tx=0).stop 2023-09-30T22:34:42.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.829+0000 7f1505e3f700 1 -- 172.21.15.16:0/1483833130 shutdown_connections 2023-09-30T22:34:42.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.829+0000 7f1505e3f700 1 --2- 172.21.15.16:0/1483833130 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1500099a40 0x7f150006f610 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:42.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.829+0000 7f1505e3f700 1 --2- 172.21.15.16:0/1483833130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1500131810 0x7f1500070090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:42.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.829+0000 7f1505e3f700 1 --2- 172.21.15.16:0/1483833130 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f150009a580 0x7f150006fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:42.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.829+0000 7f1505e3f700 1 -- 172.21.15.16:0/1483833130 >> 172.21.15.16:0/1483833130 conn(0x7f15000a2aa0 msgr2=0x7f15001b8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:42.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.829+0000 7f2ddaffd700 1 -- 172.21.15.16:0/693664808 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2ddc12fcc0 msgr2=0x7f2ddc11dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:42.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.829+0000 7f2ddaffd700 1 --2- 172.21.15.16:0/693664808 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2ddc12fcc0 0x7f2ddc11dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:42.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.829+0000 7f2ddaffd700 1 -- 172.21.15.16:0/693664808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ddc133280 msgr2=0x7f2ddc11e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:42.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.829+0000 7f2ddaffd700 1 --2- 172.21.15.16:0/693664808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ddc133280 0x7f2ddc11e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:42.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.829+0000 7f2ddaffd700 1 -- 172.21.15.16:0/693664808 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dcc009550 con 0x7f2ddc06a770 2023-09-30T22:34:42.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.829+0000 7f1505e3f700 1 -- 172.21.15.16:0/1483833130 shutdown_connections 2023-09-30T22:34:42.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.829+0000 7f1505e3f700 1 -- 172.21.15.16:0/1483833130 wait complete. 2023-09-30T22:34:42.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.829+0000 7f2ddaffd700 1 --2- 172.21.15.16:0/693664808 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2ddc06a770 0x7f2ddc11d720 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc009c40 tx=0x7f2dcc004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:42.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.829+0000 7f2dc37fe700 1 -- 172.21.15.16:0/693664808 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc004400 con 0x7f2ddc06a770 2023-09-30T22:34:42.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.830+0000 7f2dc37fe700 1 -- 172.21.15.16:0/693664808 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dcc01f070 con 0x7f2ddc06a770 2023-09-30T22:34:42.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.830+0000 7f2de1401700 1 -- 172.21.15.16:0/693664808 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ddc120040 con 0x7f2ddc06a770 2023-09-30T22:34:42.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.830+0000 7f2dc37fe700 1 -- 172.21.15.16:0/693664808 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc0038e0 con 0x7f2ddc06a770 2023-09-30T22:34:42.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.830+0000 7f2de1401700 1 -- 172.21.15.16:0/693664808 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ddc11fa40 con 0x7f2ddc06a770 2023-09-30T22:34:42.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.831+0000 7f2de1401700 1 -- 172.21.15.16:0/693664808 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ddc04f2a0 con 0x7f2ddc06a770 2023-09-30T22:34:42.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.832+0000 7f2dc37fe700 1 -- 172.21.15.16:0/693664808 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2dcc021030 con 0x7f2ddc06a770 2023-09-30T22:34:42.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.832+0000 7f2dc37fe700 1 -- 172.21.15.16:0/693664808 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f2dcc040df0 con 0x7f2ddc06a770 2023-09-30T22:34:42.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:42.836+0000 7f2dc37fe700 1 -- 172.21.15.16:0/693664808 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2dcc0199a0 con 0x7f2ddc06a770 2023-09-30T22:34:42.846 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:34:43.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.003+0000 7f193b919700 1 -- 172.21.15.16:0/2868559577 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f19341a2e40 con 0x7f193409f650 2023-09-30T22:34:43.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.003+0000 7f1929ffb700 1 -- 172.21.15.16:0/2868559577 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f192c0273a0 con 0x7f193409f650 2023-09-30T22:34:43.020 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:34:43.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.005+0000 7f193b919700 1 -- 172.21.15.16:0/2868559577 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f193409f650 msgr2=0x7f19341c3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:43.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.005+0000 7f193b919700 1 --2- 172.21.15.16:0/2868559577 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f193409f650 0x7f19341c3d90 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f193409faf0 tx=0x7f192c00caa0 comp rx=0 tx=0).stop 2023-09-30T22:34:43.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.005+0000 7f193b919700 1 -- 172.21.15.16:0/2868559577 shutdown_connections 2023-09-30T22:34:43.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.005+0000 7f193b919700 1 --2- 172.21.15.16:0/2868559577 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f193409f650 0x7f19341c3d90 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:43.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.005+0000 7f193b919700 1 --2- 172.21.15.16:0/2868559577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f193409e930 0x7f19341a1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:43.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.005+0000 7f193b919700 1 --2- 172.21.15.16:0/2868559577 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f193409f240 0x7f19341a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:43.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.005+0000 7f193b919700 1 -- 172.21.15.16:0/2868559577 >> 172.21.15.16:0/2868559577 conn(0x7f19340a4210 msgr2=0x7f193406c460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:43.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.006+0000 7f193b919700 1 -- 172.21.15.16:0/2868559577 shutdown_connections 2023-09-30T22:34:43.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.006+0000 7f193b919700 1 -- 172.21.15.16:0/2868559577 wait complete. 2023-09-30T22:34:43.022 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:34:43.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.044+0000 7f81e12a7700 1 -- 172.21.15.16:0/880231832 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f81dc06c360 con 0x7f81dc09e410 2023-09-30T22:34:43.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.045+0000 7f81c37fe700 1 -- 172.21.15.16:0/880231832 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f81d401bb00 con 0x7f81dc09e410 2023-09-30T22:34:43.126 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:34:43.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.047+0000 7f2de1401700 1 -- 172.21.15.16:0/693664808 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2ddc06c0c0 con 0x7f2ddc06a770 2023-09-30T22:34:43.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.047+0000 7f81e12a7700 1 -- 172.21.15.16:0/880231832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81dc09e410 msgr2=0x7f81dc11eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:43.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.047+0000 7f81e12a7700 1 --2- 172.21.15.16:0/880231832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81dc09e410 0x7f81dc11eac0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f81d4007e60 tx=0x7f81d400e3c0 comp rx=0 tx=0).stop 2023-09-30T22:34:43.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.047+0000 7f81e12a7700 1 -- 172.21.15.16:0/880231832 shutdown_connections 2023-09-30T22:34:43.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.047+0000 7f81e12a7700 1 --2- 172.21.15.16:0/880231832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81dc09e410 0x7f81dc11eac0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:43.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.047+0000 7f81e12a7700 1 --2- 172.21.15.16:0/880231832 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f81dc09d6f0 0x7f81dc11e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:43.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.047+0000 7f81e12a7700 1 --2- 172.21.15.16:0/880231832 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f81dc09e000 0x7f81dc11e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:43.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.047+0000 7f81e12a7700 1 -- 172.21.15.16:0/880231832 >> 172.21.15.16:0/880231832 conn(0x7f81dc066f50 msgr2=0x7f81dc0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:43.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.047+0000 7f81e12a7700 1 -- 172.21.15.16:0/880231832 shutdown_connections 2023-09-30T22:34:43.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.047+0000 7f81e12a7700 1 -- 172.21.15.16:0/880231832 wait complete. 2023-09-30T22:34:43.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.047+0000 7f2dc37fe700 1 -- 172.21.15.16:0/693664808 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2dcc015070 con 0x7f2ddc06a770 2023-09-30T22:34:43.128 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:34:43.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.050+0000 7f2de1401700 1 -- 172.21.15.16:0/693664808 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2ddc06a770 msgr2=0x7f2ddc11d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:43.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.050+0000 7f2de1401700 1 --2- 172.21.15.16:0/693664808 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2ddc06a770 0x7f2ddc11d720 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc009c40 tx=0x7f2dcc004750 comp rx=0 tx=0).stop 2023-09-30T22:34:43.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.050+0000 7f2de1401700 1 -- 172.21.15.16:0/693664808 shutdown_connections 2023-09-30T22:34:43.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.050+0000 7f2de1401700 1 --2- 172.21.15.16:0/693664808 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2ddc06a770 0x7f2ddc11d720 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:43.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.050+0000 7f2de1401700 1 --2- 172.21.15.16:0/693664808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ddc133280 0x7f2ddc11e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:43.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.050+0000 7f2de1401700 1 --2- 172.21.15.16:0/693664808 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2ddc12fcc0 0x7f2ddc11dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:43.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.050+0000 7f2de1401700 1 -- 172.21.15.16:0/693664808 >> 172.21.15.16:0/693664808 conn(0x7f2ddc0a2cc0 msgr2=0x7f2ddc09d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:43.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.050+0000 7f2de1401700 1 -- 172.21.15.16:0/693664808 shutdown_connections 2023-09-30T22:34:43.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.050+0000 7f2de1401700 1 -- 172.21.15.16:0/693664808 wait complete. 2023-09-30T22:34:43.131 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:34:43.132 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:34:43.847 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:34:43.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.993+0000 7f1eaf76b700 1 Processor -- start 2023-09-30T22:34:43.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.994+0000 7f1eaf76b700 1 -- start start 2023-09-30T22:34:43.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.994+0000 7f1eaf76b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1ea81337f0 0x7f1ea8138880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:43.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.994+0000 7f1eaf76b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ea8138dc0 0x7f1ea813b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:43.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.994+0000 7f1eaf76b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ea806b990 con 0x7f1ea8133420 2023-09-30T22:34:43.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.994+0000 7f1eaf76b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ea809b0e0 con 0x7f1ea81337f0 2023-09-30T22:34:43.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.994+0000 7f1eaf76b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea8133c20 con 0x7f1ea8138dc0 2023-09-30T22:34:43.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.994+0000 7f1eacd06700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1ea81337f0 0x7f1ea8138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:43.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.994+0000 7f1eadd08700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ea8138dc0 0x7f1ea813b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:43.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.994+0000 7f1eacd06700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1ea81337f0 0x7f1ea8138880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56700/0 (socket says 172.21.15.16:56700) 2023-09-30T22:34:43.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.994+0000 7f1eadd08700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ea8138dc0 0x7f1ea813b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:59390/0 (socket says 172.21.15.16:59390) 2023-09-30T22:34:43.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.994+0000 7f1eacd06700 1 -- 172.21.15.16:0/3320316369 learned_addr learned my addr 172.21.15.16:0/3320316369 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:43.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.994+0000 7f1ead507700 1 --1- 172.21.15.16:0/3320316369 >> v1:172.21.15.16:6789/0 conn(0x7f1ea8133420 0x7f1ea8133e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58404/0 (socket says 172.21.15.16:58404) 2023-09-30T22:34:43.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.995+0000 7f1eadd08700 1 -- 172.21.15.16:0/3320316369 >> v2:172.21.15.16:3300/0 conn(0x7f1ea81337f0 msgr2=0x7f1ea8138880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:43.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.995+0000 7f1eadd08700 1 --2- 172.21.15.16:0/3320316369 >> v2:172.21.15.16:3300/0 conn(0x7f1ea81337f0 0x7f1ea8138880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:43.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.995+0000 7f1eadd08700 1 -- 172.21.15.16:0/3320316369 >> v1:172.21.15.16:6789/0 conn(0x7f1ea8133420 legacy=0x7f1ea8133e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:43.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.995+0000 7f1eadd08700 1 -- 172.21.15.16:0/3320316369 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ea80a0590 con 0x7f1ea8138dc0 2023-09-30T22:34:43.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.995+0000 7f1e9ffff700 1 -- 172.21.15.16:0/3320316369 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3394567150 0 0) 0x7f1ea806b990 con 0x7f1ea8133420 2023-09-30T22:34:43.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.995+0000 7f1eacd06700 1 --2- 172.21.15.16:0/3320316369 >> v2:172.21.15.16:3300/0 conn(0x7f1ea81337f0 0x7f1ea8138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:43.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.995+0000 7f1eadd08700 1 --2- 172.21.15.16:0/3320316369 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ea8138dc0 0x7f1ea813b1a0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0009d90 tx=0x7f1ea000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a47b24ec8fa61a2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:43.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.995+0000 7f1e9ffff700 1 -- 172.21.15.16:0/3320316369 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea000e070 con 0x7f1ea8138dc0 2023-09-30T22:34:43.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.995+0000 7f1e9ffff700 1 -- 172.21.15.16:0/3320316369 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ea000bb20 con 0x7f1ea8138dc0 2023-09-30T22:34:43.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.995+0000 7f1e9ffff700 1 -- 172.21.15.16:0/3320316369 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea0012430 con 0x7f1ea8138dc0 2023-09-30T22:34:43.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.996+0000 7f1eaf76b700 1 -- 172.21.15.16:0/3320316369 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ea8138dc0 msgr2=0x7f1ea813b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:43.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.996+0000 7f1eaf76b700 1 --2- 172.21.15.16:0/3320316369 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ea8138dc0 0x7f1ea813b1a0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0009d90 tx=0x7f1ea000b330 comp rx=0 tx=0).stop 2023-09-30T22:34:43.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.996+0000 7f1eaf76b700 1 -- 172.21.15.16:0/3320316369 shutdown_connections 2023-09-30T22:34:43.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.996+0000 7f1eaf76b700 1 --2- 172.21.15.16:0/3320316369 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ea8138dc0 0x7f1ea813b1a0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:43.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.996+0000 7f1eaf76b700 1 --2- 172.21.15.16:0/3320316369 >> v2:172.21.15.16:3300/0 conn(0x7f1ea81337f0 0x7f1ea8138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:43.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.996+0000 7f1eaf76b700 1 -- 172.21.15.16:0/3320316369 >> 172.21.15.16:0/3320316369 conn(0x7f1ea80a4210 msgr2=0x7f1ea80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:43.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.996+0000 7f1eaf76b700 1 -- 172.21.15.16:0/3320316369 shutdown_connections 2023-09-30T22:34:43.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.996+0000 7f1eaf76b700 1 -- 172.21.15.16:0/3320316369 wait complete. 2023-09-30T22:34:44.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.997+0000 7f1eaf76b700 1 Processor -- start 2023-09-30T22:34:44.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.997+0000 7f1eaf76b700 1 -- start start 2023-09-30T22:34:44.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.998+0000 7f1eaf76b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ea8133420 0x7f1ea812eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:44.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.998+0000 7f1eaf76b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ea81337f0 0x7f1ea812f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:44.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.998+0000 7f1eaf76b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ea8138dc0 0x7f1ea812f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:44.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.998+0000 7f1eaf76b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea809c660 con 0x7f1ea8138dc0 2023-09-30T22:34:44.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.998+0000 7f1eaf76b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea809b9d0 con 0x7f1ea8133420 2023-09-30T22:34:44.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.998+0000 7f1eaf76b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea8130820 con 0x7f1ea81337f0 2023-09-30T22:34:44.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.998+0000 7f1eacd06700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ea81337f0 0x7f1ea812f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:44.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.998+0000 7f1ead507700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ea8133420 0x7f1ea812eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:44.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.998+0000 7f1eadd08700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ea8138dc0 0x7f1ea812f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:44.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.998+0000 7f1eacd06700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ea81337f0 0x7f1ea812f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:59392/0 (socket says 172.21.15.16:59392) 2023-09-30T22:34:44.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.998+0000 7f1ead507700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ea8133420 0x7f1ea812eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:51650/0 (socket says 172.21.15.16:51650) 2023-09-30T22:34:44.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.998+0000 7f1eacd06700 1 -- 172.21.15.16:0/2644915197 learned_addr learned my addr 172.21.15.16:0/2644915197 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:44.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.998+0000 7f1eadd08700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ea8138dc0 0x7f1ea812f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56702/0 (socket says 172.21.15.16:56702) 2023-09-30T22:34:44.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.999+0000 7f1eacd06700 1 -- 172.21.15.16:0/2644915197 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ea8133420 msgr2=0x7f1ea812eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:44.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.999+0000 7f1eacd06700 1 --2- 172.21.15.16:0/2644915197 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ea8133420 0x7f1ea812eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:44.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.999+0000 7f1eacd06700 1 -- 172.21.15.16:0/2644915197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ea8138dc0 msgr2=0x7f1ea812f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:44.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.999+0000 7f1eacd06700 1 --2- 172.21.15.16:0/2644915197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ea8138dc0 0x7f1ea812f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:44.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.999+0000 7f1eacd06700 1 -- 172.21.15.16:0/2644915197 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ea4009550 con 0x7f1ea81337f0 2023-09-30T22:34:44.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.999+0000 7f1ead507700 1 --2- 172.21.15.16:0/2644915197 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ea8133420 0x7f1ea812eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:44.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.999+0000 7f1eacd06700 1 --2- 172.21.15.16:0/2644915197 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ea81337f0 0x7f1ea812f0c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f1ea400ebe0 tx=0x7f1ea400eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:44.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.999+0000 7f1e9dffb700 1 -- 172.21.15.16:0/2644915197 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea400c850 con 0x7f1ea81337f0 2023-09-30T22:34:44.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.999+0000 7f1e9dffb700 1 -- 172.21.15.16:0/2644915197 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ea400c9b0 con 0x7f1ea81337f0 2023-09-30T22:34:44.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.999+0000 7f1eaf76b700 1 -- 172.21.15.16:0/2644915197 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ea0009a70 con 0x7f1ea81337f0 2023-09-30T22:34:44.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.999+0000 7f1e9dffb700 1 -- 172.21.15.16:0/2644915197 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea400cd40 con 0x7f1ea81337f0 2023-09-30T22:34:44.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:43.999+0000 7f1eaf76b700 1 -- 172.21.15.16:0/2644915197 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ea809c920 con 0x7f1ea81337f0 2023-09-30T22:34:44.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.000+0000 7f1e9dffb700 1 -- 172.21.15.16:0/2644915197 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1ea4013470 con 0x7f1ea81337f0 2023-09-30T22:34:44.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.000+0000 7f1e9dffb700 1 -- 172.21.15.16:0/2644915197 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f1ea403fc00 con 0x7f1ea81337f0 2023-09-30T22:34:44.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.000+0000 7f1eaf76b700 1 -- 172.21.15.16:0/2644915197 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ea809cb10 con 0x7f1ea81337f0 2023-09-30T22:34:44.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.005+0000 7f1e9dffb700 1 -- 172.21.15.16:0/2644915197 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1ea401b3b0 con 0x7f1ea81337f0 2023-09-30T22:34:44.023 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:34:44.131 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:34:44.132 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:34:44.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.163+0000 7f4ac93aa700 1 Processor -- start 2023-09-30T22:34:44.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.163+0000 7f4ac93aa700 1 -- start start 2023-09-30T22:34:44.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.163+0000 7f4ac93aa700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4ac409de00 0x7f4ac409e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:44.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.164+0000 7f4ac93aa700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4ac413d1f0 0x7f4ac413f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:44.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.164+0000 7f4ac93aa700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ac409a810 con 0x7f4ac406a770 2023-09-30T22:34:44.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.164+0000 7f4ac93aa700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ac406ab40 con 0x7f4ac409de00 2023-09-30T22:34:44.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.164+0000 7f4ac93aa700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac409ebc0 con 0x7f4ac413d1f0 2023-09-30T22:34:44.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.164+0000 7f4ac37fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4ac413d1f0 0x7f4ac413f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:44.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.164+0000 7f4ac2ffd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4ac406a770 0x7f4ac409d6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58410/0 (socket says 172.21.15.16:58410) 2023-09-30T22:34:44.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.164+0000 7f4ac2ffd700 1 -- 172.21.15.16:0/1327375136 learned_addr learned my addr 172.21.15.16:0/1327375136 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:34:44.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.164+0000 7f4ac37fe700 1 -- 172.21.15.16:0/1327375136 >> v2:172.21.15.16:3300/0 conn(0x7f4ac409de00 msgr2=0x7f4ac409e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:44.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.164+0000 7f4ac37fe700 1 --2- 172.21.15.16:0/1327375136 >> v2:172.21.15.16:3300/0 conn(0x7f4ac409de00 0x7f4ac409e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:44.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.164+0000 7f4ac37fe700 1 -- 172.21.15.16:0/1327375136 >> v1:172.21.15.16:6789/0 conn(0x7f4ac406a770 legacy=0x7f4ac409d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:44.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.164+0000 7f4ac37fe700 1 -- 172.21.15.16:0/1327375136 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac409e920 con 0x7f4ac413d1f0 2023-09-30T22:34:44.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.164+0000 7f4ac1ffb700 1 -- 172.21.15.16:0/1327375136 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1737160783 0 0) 0x7f4ac409a810 con 0x7f4ac406a770 2023-09-30T22:34:44.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.164+0000 7f4ac37fe700 1 --2- 172.21.15.16:0/1327375136 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4ac413d1f0 0x7f4ac413f5d0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f4abc009d90 tx=0x7f4abc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc614c04de09ea94 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:44.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.165+0000 7f4ac1ffb700 1 -- 172.21.15.16:0/1327375136 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4abc00e070 con 0x7f4ac413d1f0 2023-09-30T22:34:44.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.165+0000 7f4ac1ffb700 1 -- 172.21.15.16:0/1327375136 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4abc00bbd0 con 0x7f4ac413d1f0 2023-09-30T22:34:44.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.165+0000 7f4ac1ffb700 1 -- 172.21.15.16:0/1327375136 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4abc012430 con 0x7f4ac413d1f0 2023-09-30T22:34:44.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.166+0000 7f4ac93aa700 1 -- 172.21.15.16:0/1327375136 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4ac413d1f0 msgr2=0x7f4ac413f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:44.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.166+0000 7f4ac93aa700 1 --2- 172.21.15.16:0/1327375136 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4ac413d1f0 0x7f4ac413f5d0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f4abc009d90 tx=0x7f4abc00b330 comp rx=0 tx=0).stop 2023-09-30T22:34:44.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.166+0000 7f4ac93aa700 1 -- 172.21.15.16:0/1327375136 shutdown_connections 2023-09-30T22:34:44.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.166+0000 7f4ac93aa700 1 --2- 172.21.15.16:0/1327375136 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4ac413d1f0 0x7f4ac413f5d0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:44.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.166+0000 7f4ac93aa700 1 --2- 172.21.15.16:0/1327375136 >> v2:172.21.15.16:3300/0 conn(0x7f4ac409de00 0x7f4ac409e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:44.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.166+0000 7f4ac93aa700 1 -- 172.21.15.16:0/1327375136 >> 172.21.15.16:0/1327375136 conn(0x7f4ac4066f50 msgr2=0x7f4ac409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:44.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.166+0000 7f4ac93aa700 1 -- 172.21.15.16:0/1327375136 shutdown_connections 2023-09-30T22:34:44.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.166+0000 7f4ac93aa700 1 -- 172.21.15.16:0/1327375136 wait complete. 2023-09-30T22:34:44.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.167+0000 7f4ac93aa700 1 Processor -- start 2023-09-30T22:34:44.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.167+0000 7f4ac93aa700 1 -- start start 2023-09-30T22:34:44.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.168+0000 7f4ac93aa700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4ac406a770 0x7f4ac4071510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:44.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.168+0000 7f4ac93aa700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4ac409de00 0x7f4ac413b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:44.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.168+0000 7f4ac93aa700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ac413d1f0 0x7f4ac413bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:44.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.168+0000 7f4ac93aa700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac413d070 con 0x7f4ac413d1f0 2023-09-30T22:34:44.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.168+0000 7f4ac93aa700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac409f230 con 0x7f4ac406a770 2023-09-30T22:34:44.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.168+0000 7f4ac93aa700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac411efc0 con 0x7f4ac409de00 2023-09-30T22:34:44.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.168+0000 7f4ac27fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4ac409de00 0x7f4ac413b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:44.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.168+0000 7f4ac27fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4ac409de00 0x7f4ac413b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:59404/0 (socket says 172.21.15.16:59404) 2023-09-30T22:34:44.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.168+0000 7f4ac27fc700 1 -- 172.21.15.16:0/3243791558 learned_addr learned my addr 172.21.15.16:0/3243791558 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:44.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.168+0000 7f4ac2ffd700 1 --2- 172.21.15.16:0/3243791558 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4ac406a770 0x7f4ac4071510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:44.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.168+0000 7f4ac27fc700 1 -- 172.21.15.16:0/3243791558 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4ac406a770 msgr2=0x7f4ac4071510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:44.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.168+0000 7f4ac27fc700 1 --2- 172.21.15.16:0/3243791558 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4ac406a770 0x7f4ac4071510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:44.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.168+0000 7f4ac27fc700 1 -- 172.21.15.16:0/3243791558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ac413d1f0 msgr2=0x7f4ac413bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:44.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.168+0000 7f4ac27fc700 1 --2- 172.21.15.16:0/3243791558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ac413d1f0 0x7f4ac413bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:44.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.168+0000 7f4ac27fc700 1 -- 172.21.15.16:0/3243791558 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4abc009a70 con 0x7f4ac409de00 2023-09-30T22:34:44.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.169+0000 7f4ac27fc700 1 --2- 172.21.15.16:0/3243791558 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4ac409de00 0x7f4ac413b630 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f4ab800b560 tx=0x7f4ab800b920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:44.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.169+0000 7f4aab7fe700 1 -- 172.21.15.16:0/3243791558 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab8011830 con 0x7f4ac409de00 2023-09-30T22:34:44.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.169+0000 7f4aab7fe700 1 -- 172.21.15.16:0/3243791558 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ab8011990 con 0x7f4ac409de00 2023-09-30T22:34:44.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.169+0000 7f4ac93aa700 1 -- 172.21.15.16:0/3243791558 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ac411e300 con 0x7f4ac409de00 2023-09-30T22:34:44.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.169+0000 7f4aab7fe700 1 -- 172.21.15.16:0/3243791558 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab8011d40 con 0x7f4ac409de00 2023-09-30T22:34:44.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.169+0000 7f4ac93aa700 1 -- 172.21.15.16:0/3243791558 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ac413cd50 con 0x7f4ac409de00 2023-09-30T22:34:44.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.170+0000 7f4aab7fe700 1 -- 172.21.15.16:0/3243791558 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4ab801a400 con 0x7f4ac409de00 2023-09-30T22:34:44.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.170+0000 7f4aab7fe700 1 -- 172.21.15.16:0/3243791558 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f4ab803f6c0 con 0x7f4ac409de00 2023-09-30T22:34:44.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.170+0000 7f4ac93aa700 1 -- 172.21.15.16:0/3243791558 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ac411ead0 con 0x7f4ac409de00 2023-09-30T22:34:44.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.180+0000 7f4aab7fe700 1 -- 172.21.15.16:0/3243791558 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4ab8015020 con 0x7f4ac409de00 2023-09-30T22:34:44.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.220+0000 7f1eaf76b700 1 -- 172.21.15.16:0/2644915197 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1ea8074ea0 con 0x7f1ea81337f0 2023-09-30T22:34:44.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.221+0000 7f1e9dffb700 1 -- 172.21.15.16:0/2644915197 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1ea4011b00 con 0x7f1ea81337f0 2023-09-30T22:34:44.222 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:34:44.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.223+0000 7f1eaf76b700 1 -- 172.21.15.16:0/2644915197 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ea81337f0 msgr2=0x7f1ea812f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:44.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.224+0000 7f1eaf76b700 1 --2- 172.21.15.16:0/2644915197 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ea81337f0 0x7f1ea812f0c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f1ea400ebe0 tx=0x7f1ea400eef0 comp rx=0 tx=0).stop 2023-09-30T22:34:44.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.224+0000 7f1eaf76b700 1 -- 172.21.15.16:0/2644915197 shutdown_connections 2023-09-30T22:34:44.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.224+0000 7f1eaf76b700 1 --2- 172.21.15.16:0/2644915197 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ea8133420 0x7f1ea812eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:44.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.224+0000 7f1eaf76b700 1 --2- 172.21.15.16:0/2644915197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ea8138dc0 0x7f1ea812f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:44.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.224+0000 7f1eaf76b700 1 --2- 172.21.15.16:0/2644915197 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ea81337f0 0x7f1ea812f0c0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:44.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.224+0000 7f1eaf76b700 1 -- 172.21.15.16:0/2644915197 >> 172.21.15.16:0/2644915197 conn(0x7f1ea80a4210 msgr2=0x7f1ea813aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:44.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.225+0000 7f1eaf76b700 1 -- 172.21.15.16:0/2644915197 shutdown_connections 2023-09-30T22:34:44.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.225+0000 7f1eaf76b700 1 -- 172.21.15.16:0/2644915197 wait complete. 2023-09-30T22:34:44.241 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:34:44.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.277+0000 7f6c83c0e700 1 Processor -- start 2023-09-30T22:34:44.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.277+0000 7f6c83c0e700 1 -- start start 2023-09-30T22:34:44.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.278+0000 7f6c83c0e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6c7c1338a0 0x7f6c7c133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:44.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.278+0000 7f6c83c0e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6c7c09e930 0x7f6c7c13b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:44.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.278+0000 7f6c83c0e700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c7c06ef30 con 0x7f6c7c09de50 2023-09-30T22:34:44.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.278+0000 7f6c83c0e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7c06fef0 con 0x7f6c7c09e930 2023-09-30T22:34:44.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.278+0000 7f6c83c0e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7c1334b0 con 0x7f6c7c1338a0 2023-09-30T22:34:44.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.278+0000 7f6c819aa700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6c7c1338a0 0x7f6c7c133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:44.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.278+0000 7f6c821ab700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6c7c09e930 0x7f6c7c13b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:44.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.278+0000 7f6c811a9700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6c7c09de50 0x7f6c7c09e220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58426/0 (socket says 172.21.15.16:58426) 2023-09-30T22:34:44.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.278+0000 7f6c819aa700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6c7c1338a0 0x7f6c7c133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:59418/0 (socket says 172.21.15.16:59418) 2023-09-30T22:34:44.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.278+0000 7f6c821ab700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6c7c09e930 0x7f6c7c13b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:51678/0 (socket says 172.21.15.16:51678) 2023-09-30T22:34:44.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.278+0000 7f6c811a9700 1 -- 172.21.15.16:0/893515311 learned_addr learned my addr 172.21.15.16:0/893515311 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:34:44.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.278+0000 7f6c821ab700 1 -- 172.21.15.16:0/893515311 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6c7c1338a0 msgr2=0x7f6c7c133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:44.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.278+0000 7f6c821ab700 1 --2- 172.21.15.16:0/893515311 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6c7c1338a0 0x7f6c7c133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:44.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.278+0000 7f6c821ab700 1 -- 172.21.15.16:0/893515311 >> v1:172.21.15.16:6789/0 conn(0x7f6c7c09de50 legacy=0x7f6c7c09e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:44.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.278+0000 7f6c809a8700 1 -- 172.21.15.16:0/893515311 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1259311368 0 0) 0x7f6c7c06ef30 con 0x7f6c7c09de50 2023-09-30T22:34:44.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.278+0000 7f6c821ab700 1 -- 172.21.15.16:0/893515311 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c7c06fc30 con 0x7f6c7c09e930 2023-09-30T22:34:44.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.279+0000 7f6c821ab700 1 --2- 172.21.15.16:0/893515311 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6c7c09e930 0x7f6c7c13b740 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f6c74009dc0 tx=0x7f6c7400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84f4c6c6b07b5ccd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:44.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.279+0000 7f6c809a8700 1 -- 172.21.15.16:0/893515311 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7400e070 con 0x7f6c7c09e930 2023-09-30T22:34:44.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.279+0000 7f6c809a8700 1 -- 172.21.15.16:0/893515311 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c7400bb20 con 0x7f6c7c09e930 2023-09-30T22:34:44.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.279+0000 7f6c809a8700 1 -- 172.21.15.16:0/893515311 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c74012430 con 0x7f6c7c09e930 2023-09-30T22:34:44.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.279+0000 7f1f75be6700 1 Processor -- start 2023-09-30T22:34:44.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.279+0000 7f1f75be6700 1 -- start start 2023-09-30T22:34:44.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.280+0000 7f6c83c0e700 1 -- 172.21.15.16:0/893515311 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6c7c09e930 msgr2=0x7f6c7c13b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:44.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.280+0000 7f6c83c0e700 1 --2- 172.21.15.16:0/893515311 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6c7c09e930 0x7f6c7c13b740 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f6c74009dc0 tx=0x7f6c7400b330 comp rx=0 tx=0).stop 2023-09-30T22:34:44.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.280+0000 7f6c83c0e700 1 -- 172.21.15.16:0/893515311 shutdown_connections 2023-09-30T22:34:44.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.280+0000 7f6c83c0e700 1 --2- 172.21.15.16:0/893515311 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6c7c09e930 0x7f6c7c13b740 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:44.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.280+0000 7f6c83c0e700 1 --2- 172.21.15.16:0/893515311 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6c7c1338a0 0x7f6c7c133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:44.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.280+0000 7f6c83c0e700 1 -- 172.21.15.16:0/893515311 >> 172.21.15.16:0/893515311 conn(0x7f6c7c0a2aa0 msgr2=0x7f6c7c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:44.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.280+0000 7f6c83c0e700 1 -- 172.21.15.16:0/893515311 shutdown_connections 2023-09-30T22:34:44.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.280+0000 7f6c83c0e700 1 -- 172.21.15.16:0/893515311 wait complete. 2023-09-30T22:34:44.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.280+0000 7f1f75be6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f701316c0 0x7f1f70131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:44.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.280+0000 7f1f75be6700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1f7009d9f0 0x7f1f7006a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:44.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.280+0000 7f1f75be6700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f7009e130 con 0x7f1f7006acb0 2023-09-30T22:34:44.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.280+0000 7f1f75be6700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f701312b0 con 0x7f1f7009d9f0 2023-09-30T22:34:44.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.280+0000 7f1f75be6700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7009f6e0 con 0x7f1f701316c0 2023-09-30T22:34:44.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.280+0000 7f1f6f7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f701316c0 0x7f1f70131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:44.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.280+0000 7f1f6f7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f701316c0 0x7f1f70131a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:59422/0 (socket says 172.21.15.16:59422) 2023-09-30T22:34:44.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.280+0000 7f1f6f7fe700 1 -- 172.21.15.16:0/2316232276 learned_addr learned my addr 172.21.15.16:0/2316232276 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:44.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.280+0000 7f1f6effd700 1 --2- 172.21.15.16:0/2316232276 >> v2:172.21.15.16:3300/0 conn(0x7f1f7009d9f0 0x7f1f7006a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:44.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.281+0000 7f1f6effd700 1 -- 172.21.15.16:0/2316232276 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f701316c0 msgr2=0x7f1f70131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:44.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.281+0000 7f1f6effd700 1 --2- 172.21.15.16:0/2316232276 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f701316c0 0x7f1f70131a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:44.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.281+0000 7f1f6effd700 1 -- 172.21.15.16:0/2316232276 >> v1:172.21.15.16:6789/0 conn(0x7f1f7006acb0 legacy=0x7f1f70138540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:44.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.281+0000 7f1f6effd700 1 -- 172.21.15.16:0/2316232276 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f7009e760 con 0x7f1f7009d9f0 2023-09-30T22:34:44.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.281+0000 7f6c83c0e700 1 Processor -- start 2023-09-30T22:34:44.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.281+0000 7f1f6effd700 1 --2- 172.21.15.16:0/2316232276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f7009d9f0 0x7f1f7006a770 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f1f64009840 tx=0x7f1f64009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7285b5b606a46653 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:44.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.281+0000 7f1f6e7fc700 1 -- 172.21.15.16:0/2316232276 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f64011830 con 0x7f1f7009d9f0 2023-09-30T22:34:44.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.281+0000 7f1f6e7fc700 1 -- 172.21.15.16:0/2316232276 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f64011990 con 0x7f1f7009d9f0 2023-09-30T22:34:44.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.281+0000 7f1f6e7fc700 1 -- 172.21.15.16:0/2316232276 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f64011d20 con 0x7f1f7009d9f0 2023-09-30T22:34:44.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.281+0000 7f6c83c0e700 1 -- start start 2023-09-30T22:34:44.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.282+0000 7f6c83c0e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6c7c09de50 0x7f6c7c1214e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:44.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.282+0000 7f6c83c0e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6c7c09e930 0x7f6c7c121a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:44.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.282+0000 7f6c83c0e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c7c1338a0 0x7f6c7c121f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:44.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.282+0000 7f6c83c0e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7c130320 con 0x7f6c7c1338a0 2023-09-30T22:34:44.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.282+0000 7f6c83c0e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7c0a0d40 con 0x7f6c7c09de50 2023-09-30T22:34:44.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.282+0000 7f6c83c0e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7c06f640 con 0x7f6c7c09e930 2023-09-30T22:34:44.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.282+0000 7f6c819aa700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6c7c09de50 0x7f6c7c1214e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:44.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.282+0000 7f6c811a9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6c7c09e930 0x7f6c7c121a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:44.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.282+0000 7f6c819aa700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6c7c09de50 0x7f6c7c1214e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:51686/0 (socket says 172.21.15.16:51686) 2023-09-30T22:34:44.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.282+0000 7f6c819aa700 1 -- 172.21.15.16:0/4124711334 learned_addr learned my addr 172.21.15.16:0/4124711334 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:44.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.282+0000 7f6c821ab700 1 --2- 172.21.15.16:0/4124711334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c7c1338a0 0x7f6c7c121f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:44.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.282+0000 7f1f75be6700 1 -- 172.21.15.16:0/2316232276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f7009d9f0 msgr2=0x7f1f7006a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:44.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.282+0000 7f1f75be6700 1 --2- 172.21.15.16:0/2316232276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f7009d9f0 0x7f1f7006a770 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f1f64009840 tx=0x7f1f64009b50 comp rx=0 tx=0).stop 2023-09-30T22:34:44.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.282+0000 7f6c811a9700 1 -- 172.21.15.16:0/4124711334 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6c7c09de50 msgr2=0x7f6c7c1214e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:44.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.282+0000 7f6c811a9700 1 --2- 172.21.15.16:0/4124711334 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6c7c09de50 0x7f6c7c1214e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:44.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.282+0000 7f1f75be6700 1 -- 172.21.15.16:0/2316232276 shutdown_connections 2023-09-30T22:34:44.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.282+0000 7f6c811a9700 1 -- 172.21.15.16:0/4124711334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c7c1338a0 msgr2=0x7f6c7c121f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:44.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.282+0000 7f6c811a9700 1 --2- 172.21.15.16:0/4124711334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c7c1338a0 0x7f6c7c121f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:44.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.282+0000 7f1f75be6700 1 --2- 172.21.15.16:0/2316232276 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f701316c0 0x7f1f70131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:44.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.282+0000 7f1f75be6700 1 --2- 172.21.15.16:0/2316232276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f7009d9f0 0x7f1f7006a770 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:44.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.282+0000 7f1f75be6700 1 -- 172.21.15.16:0/2316232276 >> 172.21.15.16:0/2316232276 conn(0x7f1f70066f50 msgr2=0x7f1f7009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:44.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.282+0000 7f6c811a9700 1 -- 172.21.15.16:0/4124711334 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c74009a70 con 0x7f6c7c09e930 2023-09-30T22:34:44.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.282+0000 7f1f75be6700 1 -- 172.21.15.16:0/2316232276 shutdown_connections 2023-09-30T22:34:44.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.282+0000 7f1f75be6700 1 -- 172.21.15.16:0/2316232276 wait complete. 2023-09-30T22:34:44.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.282+0000 7f6c811a9700 1 --2- 172.21.15.16:0/4124711334 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6c7c09e930 0x7f6c7c121a20 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c06d750 tx=0x7f6c6c00c750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:44.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.283+0000 7f6c727fc700 1 -- 172.21.15.16:0/4124711334 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c004c80 con 0x7f6c7c09e930 2023-09-30T22:34:44.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.283+0000 7f6c727fc700 1 -- 172.21.15.16:0/4124711334 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c6c004de0 con 0x7f6c7c09e930 2023-09-30T22:34:44.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.283+0000 7f6c83c0e700 1 -- 172.21.15.16:0/4124711334 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c7c1245d0 con 0x7f6c7c09e930 2023-09-30T22:34:44.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.283+0000 7f6c727fc700 1 -- 172.21.15.16:0/4124711334 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c007310 con 0x7f6c7c09e930 2023-09-30T22:34:44.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.283+0000 7f6c83c0e700 1 -- 172.21.15.16:0/4124711334 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c7c125070 con 0x7f6c7c09e930 2023-09-30T22:34:44.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.283+0000 7f1f75be6700 1 Processor -- start 2023-09-30T22:34:44.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.284+0000 7f1f75be6700 1 -- start start 2023-09-30T22:34:44.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.284+0000 7f6c727fc700 1 -- 172.21.15.16:0/4124711334 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6c6c007470 con 0x7f6c7c09e930 2023-09-30T22:34:44.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.284+0000 7f6c727fc700 1 -- 172.21.15.16:0/4124711334 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f6c6c03f240 con 0x7f6c7c09e930 2023-09-30T22:34:44.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.284+0000 7f6c83c0e700 1 -- 172.21.15.16:0/4124711334 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c640052e0 con 0x7f6c7c09e930 2023-09-30T22:34:44.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.284+0000 7f1f75be6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f7006acb0 0x7f1f70129550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:44.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.284+0000 7f1f75be6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f701316c0 0x7f1f70129a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:44.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.284+0000 7f1f75be6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f70129fd0 0x7f1f701c9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:44.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.284+0000 7f1f75be6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7012a420 con 0x7f1f701316c0 2023-09-30T22:34:44.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.284+0000 7f1f75be6700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7012a7c0 con 0x7f1f7006acb0 2023-09-30T22:34:44.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.284+0000 7f1f75be6700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7006f4e0 con 0x7f1f70129fd0 2023-09-30T22:34:44.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.285+0000 7f1f6effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f701316c0 0x7f1f70129a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:44.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.285+0000 7f1f6f7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f7006acb0 0x7f1f70129550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:44.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.285+0000 7f1f6effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f701316c0 0x7f1f70129a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56754/0 (socket says 172.21.15.16:56754) 2023-09-30T22:34:44.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.285+0000 7f1f6effd700 1 -- 172.21.15.16:0/3547392153 learned_addr learned my addr 172.21.15.16:0/3547392153 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:44.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.285+0000 7f1f6ffff700 1 --2- 172.21.15.16:0/3547392153 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f70129fd0 0x7f1f701c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:44.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.285+0000 7f1f6effd700 1 -- 172.21.15.16:0/3547392153 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f70129fd0 msgr2=0x7f1f701c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:44.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.285+0000 7f1f6effd700 1 --2- 172.21.15.16:0/3547392153 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f70129fd0 0x7f1f701c9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:44.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.285+0000 7f1f6effd700 1 -- 172.21.15.16:0/3547392153 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f7006acb0 msgr2=0x7f1f70129550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:44.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.285+0000 7f1f6effd700 1 --2- 172.21.15.16:0/3547392153 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f7006acb0 0x7f1f70129550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:44.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.285+0000 7f1f6effd700 1 -- 172.21.15.16:0/3547392153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f640094a0 con 0x7f1f701316c0 2023-09-30T22:34:44.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.285+0000 7f1f6effd700 1 --2- 172.21.15.16:0/3547392153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f701316c0 0x7f1f70129a90 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f1f64013000 tx=0x7f1f64004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:44.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.285+0000 7f1f57fff700 1 -- 172.21.15.16:0/3547392153 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f64004550 con 0x7f1f701316c0 2023-09-30T22:34:44.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.285+0000 7f1f57fff700 1 -- 172.21.15.16:0/3547392153 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f6401f070 con 0x7f1f701316c0 2023-09-30T22:34:44.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.285+0000 7f1f57fff700 1 -- 172.21.15.16:0/3547392153 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f640038e0 con 0x7f1f701316c0 2023-09-30T22:34:44.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.285+0000 7f1f75be6700 1 -- 172.21.15.16:0/3547392153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f7012ad70 con 0x7f1f701316c0 2023-09-30T22:34:44.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.286+0000 7f1f57fff700 1 -- 172.21.15.16:0/3547392153 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1f64003a40 con 0x7f1f701316c0 2023-09-30T22:34:44.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.287+0000 7f1f75be6700 1 -- 172.21.15.16:0/3547392153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f7012e830 con 0x7f1f701316c0 2023-09-30T22:34:44.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.287+0000 7f1f57fff700 1 -- 172.21.15.16:0/3547392153 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f1f64021070 con 0x7f1f701316c0 2023-09-30T22:34:44.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.290+0000 7f6c727fc700 1 -- 172.21.15.16:0/4124711334 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6c6c007720 con 0x7f6c7c09e930 2023-09-30T22:34:44.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.291+0000 7f1f75be6700 1 -- 172.21.15.16:0/3547392153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f70074ea0 con 0x7f1f701316c0 2023-09-30T22:34:44.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.297+0000 7f1f57fff700 1 -- 172.21.15.16:0/3547392153 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1f640253e0 con 0x7f1f701316c0 2023-09-30T22:34:44.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.391+0000 7f4ac93aa700 1 -- 172.21.15.16:0/3243791558 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f4ac4074ea0 con 0x7f4ac409de00 2023-09-30T22:34:44.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.392+0000 7f4aab7fe700 1 -- 172.21.15.16:0/3243791558 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4ab8029090 con 0x7f4ac409de00 2023-09-30T22:34:44.392 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:34:44.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.394+0000 7f4ac93aa700 1 -- 172.21.15.16:0/3243791558 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4ac409de00 msgr2=0x7f4ac413b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:44.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.394+0000 7f4ac93aa700 1 --2- 172.21.15.16:0/3243791558 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4ac409de00 0x7f4ac413b630 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f4ab800b560 tx=0x7f4ab800b920 comp rx=0 tx=0).stop 2023-09-30T22:34:44.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.394+0000 7f4ac93aa700 1 -- 172.21.15.16:0/3243791558 shutdown_connections 2023-09-30T22:34:44.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.394+0000 7f4ac93aa700 1 --2- 172.21.15.16:0/3243791558 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4ac406a770 0x7f4ac4071510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:44.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.394+0000 7f4ac93aa700 1 --2- 172.21.15.16:0/3243791558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ac413d1f0 0x7f4ac413bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:44.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.394+0000 7f4ac93aa700 1 --2- 172.21.15.16:0/3243791558 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4ac409de00 0x7f4ac413b630 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:44.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.394+0000 7f4ac93aa700 1 -- 172.21.15.16:0/3243791558 >> 172.21.15.16:0/3243791558 conn(0x7f4ac4066f50 msgr2=0x7f4ac409b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:44.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.394+0000 7f4ac93aa700 1 -- 172.21.15.16:0/3243791558 shutdown_connections 2023-09-30T22:34:44.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.394+0000 7f4ac93aa700 1 -- 172.21.15.16:0/3243791558 wait complete. 2023-09-30T22:34:44.408 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:34:44.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.509+0000 7f6c83c0e700 1 -- 172.21.15.16:0/4124711334 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6c64005590 con 0x7f6c7c09e930 2023-09-30T22:34:44.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.510+0000 7f6c727fc700 1 -- 172.21.15.16:0/4124711334 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6c6c01b340 con 0x7f6c7c09e930 2023-09-30T22:34:44.511 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:34:44.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.512+0000 7f6c83c0e700 1 -- 172.21.15.16:0/4124711334 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6c7c09e930 msgr2=0x7f6c7c121a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:44.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.512+0000 7f6c83c0e700 1 --2- 172.21.15.16:0/4124711334 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6c7c09e930 0x7f6c7c121a20 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c06d750 tx=0x7f6c6c00c750 comp rx=0 tx=0).stop 2023-09-30T22:34:44.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.512+0000 7f6c83c0e700 1 -- 172.21.15.16:0/4124711334 shutdown_connections 2023-09-30T22:34:44.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.512+0000 7f6c83c0e700 1 --2- 172.21.15.16:0/4124711334 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6c7c09de50 0x7f6c7c1214e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:44.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.512+0000 7f6c83c0e700 1 --2- 172.21.15.16:0/4124711334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c7c1338a0 0x7f6c7c121f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:44.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.512+0000 7f6c83c0e700 1 --2- 172.21.15.16:0/4124711334 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6c7c09e930 0x7f6c7c121a20 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:44.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.512+0000 7f6c83c0e700 1 -- 172.21.15.16:0/4124711334 >> 172.21.15.16:0/4124711334 conn(0x7f6c7c0a2aa0 msgr2=0x7f6c7c09ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:44.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.512+0000 7f6c83c0e700 1 -- 172.21.15.16:0/4124711334 shutdown_connections 2023-09-30T22:34:44.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.512+0000 7f6c83c0e700 1 -- 172.21.15.16:0/4124711334 wait complete. 2023-09-30T22:34:44.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.517+0000 7f1f75be6700 1 -- 172.21.15.16:0/3547392153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1f7006c360 con 0x7f1f701316c0 2023-09-30T22:34:44.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.518+0000 7f1f57fff700 1 -- 172.21.15.16:0/3547392153 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1f6402b020 con 0x7f1f701316c0 2023-09-30T22:34:44.518 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:34:44.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.522+0000 7f1f75be6700 1 -- 172.21.15.16:0/3547392153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f701316c0 msgr2=0x7f1f70129a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:44.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.522+0000 7f1f75be6700 1 --2- 172.21.15.16:0/3547392153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f701316c0 0x7f1f70129a90 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f1f64013000 tx=0x7f1f64004750 comp rx=0 tx=0).stop 2023-09-30T22:34:44.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.522+0000 7f1f75be6700 1 -- 172.21.15.16:0/3547392153 shutdown_connections 2023-09-30T22:34:44.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.522+0000 7f1f75be6700 1 --2- 172.21.15.16:0/3547392153 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1f7006acb0 0x7f1f70129550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:44.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.522+0000 7f1f75be6700 1 --2- 172.21.15.16:0/3547392153 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1f70129fd0 0x7f1f701c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:44.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.522+0000 7f1f75be6700 1 --2- 172.21.15.16:0/3547392153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f701316c0 0x7f1f70129a90 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:44.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.522+0000 7f1f75be6700 1 -- 172.21.15.16:0/3547392153 >> 172.21.15.16:0/3547392153 conn(0x7f1f70066f50 msgr2=0x7f1f700a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:44.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.522+0000 7f1f75be6700 1 -- 172.21.15.16:0/3547392153 shutdown_connections 2023-09-30T22:34:44.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:44.522+0000 7f1f75be6700 1 -- 172.21.15.16:0/3547392153 wait complete. 2023-09-30T22:34:44.529 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:34:44.538 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:34:45.242 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:34:45.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.388+0000 7f0dcf4f8700 1 Processor -- start 2023-09-30T22:34:45.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.388+0000 7f0dcf4f8700 1 -- start start 2023-09-30T22:34:45.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.388+0000 7f0dcf4f8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0dc809d6f0 0x7f0dc809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:45.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.388+0000 7f0dcf4f8700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0dc809e000 0x7f0dc81339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:45.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.388+0000 7f0dcf4f8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0dc809e410 0x7f0dc8133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:45.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.388+0000 7f0dcf4f8700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dc806b460 con 0x7f0dc809e000 2023-09-30T22:34:45.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.388+0000 7f0dcf4f8700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc809b120 con 0x7f0dc809d6f0 2023-09-30T22:34:45.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.388+0000 7f0dcf4f8700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc809a7e0 con 0x7f0dc809e410 2023-09-30T22:34:45.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.389+0000 7f0dcd294700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0dc809d6f0 0x7f0dc809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:45.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.389+0000 7f0dcca93700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0dc809e000 0x7f0dc81339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:45.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.389+0000 7f0dcda95700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0dc809e410 0x7f0dc8133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:45.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.389+0000 7f0dcca93700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0dc809e000 0x7f0dc81339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56770/0 (socket says 172.21.15.16:56770) 2023-09-30T22:34:45.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.389+0000 7f0dcda95700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0dc809e410 0x7f0dc8133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:59440/0 (socket says 172.21.15.16:59440) 2023-09-30T22:34:45.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.389+0000 7f0dcca93700 1 -- 172.21.15.16:0/3397030928 learned_addr learned my addr 172.21.15.16:0/3397030928 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:45.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.389+0000 7f0dcd294700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0dc809d6f0 0x7f0dc809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:51694/0 (socket says 172.21.15.16:51694) 2023-09-30T22:34:45.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.389+0000 7f0dcd294700 1 -- 172.21.15.16:0/3397030928 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0dc809e410 msgr2=0x7f0dc8133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:45.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.389+0000 7f0dcd294700 1 --2- 172.21.15.16:0/3397030928 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0dc809e410 0x7f0dc8133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:45.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.389+0000 7f0dcd294700 1 -- 172.21.15.16:0/3397030928 >> v2:172.21.15.16:3300/0 conn(0x7f0dc809e000 msgr2=0x7f0dc81339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:45.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.389+0000 7f0dcd294700 1 --2- 172.21.15.16:0/3397030928 >> v2:172.21.15.16:3300/0 conn(0x7f0dc809e000 0x7f0dc81339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:45.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.389+0000 7f0dcd294700 1 -- 172.21.15.16:0/3397030928 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dc809a500 con 0x7f0dc809d6f0 2023-09-30T22:34:45.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.389+0000 7f0dcca93700 1 --2- 172.21.15.16:0/3397030928 >> v2:172.21.15.16:3300/0 conn(0x7f0dc809e000 0x7f0dc81339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:45.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.389+0000 7f0dcda95700 1 --2- 172.21.15.16:0/3397030928 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0dc809e410 0x7f0dc8133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:45.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.389+0000 7f0dcd294700 1 --2- 172.21.15.16:0/3397030928 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0dc809d6f0 0x7f0dc809dac0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4009870 tx=0x7f0dc4009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db2550af71169282 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:45.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.389+0000 7f0dbffff700 1 -- 172.21.15.16:0/3397030928 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc4011830 con 0x7f0dc809d6f0 2023-09-30T22:34:45.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.389+0000 7f0dbffff700 1 -- 172.21.15.16:0/3397030928 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dc4011990 con 0x7f0dc809d6f0 2023-09-30T22:34:45.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.389+0000 7f0dbffff700 1 -- 172.21.15.16:0/3397030928 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dc4011d20 con 0x7f0dc809d6f0 2023-09-30T22:34:45.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.390+0000 7f0dcf4f8700 1 -- 172.21.15.16:0/3397030928 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0dc809d6f0 msgr2=0x7f0dc809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:45.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.390+0000 7f0dcf4f8700 1 --2- 172.21.15.16:0/3397030928 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0dc809d6f0 0x7f0dc809dac0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4009870 tx=0x7f0dc4009b80 comp rx=0 tx=0).stop 2023-09-30T22:34:45.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.390+0000 7f0dcf4f8700 1 -- 172.21.15.16:0/3397030928 shutdown_connections 2023-09-30T22:34:45.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.390+0000 7f0dcf4f8700 1 --2- 172.21.15.16:0/3397030928 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0dc809d6f0 0x7f0dc809dac0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:45.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.390+0000 7f0dcf4f8700 1 --2- 172.21.15.16:0/3397030928 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0dc809e410 0x7f0dc8133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:45.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.390+0000 7f0dcf4f8700 1 --2- 172.21.15.16:0/3397030928 >> v2:172.21.15.16:3300/0 conn(0x7f0dc809e000 0x7f0dc81339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:45.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.390+0000 7f0dcf4f8700 1 -- 172.21.15.16:0/3397030928 >> 172.21.15.16:0/3397030928 conn(0x7f0dc8066f50 msgr2=0x7f0dc809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:45.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.390+0000 7f0dcf4f8700 1 -- 172.21.15.16:0/3397030928 shutdown_connections 2023-09-30T22:34:45.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.390+0000 7f0dcf4f8700 1 -- 172.21.15.16:0/3397030928 wait complete. 2023-09-30T22:34:45.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.391+0000 7f0dcf4f8700 1 Processor -- start 2023-09-30T22:34:45.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.391+0000 7f0dcf4f8700 1 -- start start 2023-09-30T22:34:45.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.392+0000 7f0dcf4f8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0dc809d6f0 0x7f0dc8122550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:45.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.392+0000 7f0dcf4f8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dc809e000 0x7f0dc8122a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:45.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.392+0000 7f0dcf4f8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0dc809e410 0x7f0dc8122fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:45.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.392+0000 7f0dcf4f8700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc81239b0 con 0x7f0dc809e000 2023-09-30T22:34:45.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.392+0000 7f0dcf4f8700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc8133050 con 0x7f0dc809e410 2023-09-30T22:34:45.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.392+0000 7f0dcf4f8700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc8125ad0 con 0x7f0dc809d6f0 2023-09-30T22:34:45.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.392+0000 7f0dcca93700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dc809e000 0x7f0dc8122a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:45.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.392+0000 7f0dcd294700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0dc809d6f0 0x7f0dc8122550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:45.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.392+0000 7f0dcca93700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dc809e000 0x7f0dc8122a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56772/0 (socket says 172.21.15.16:56772) 2023-09-30T22:34:45.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.392+0000 7f0dcca93700 1 -- 172.21.15.16:0/3320598523 learned_addr learned my addr 172.21.15.16:0/3320598523 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:45.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.392+0000 7f0dcda95700 1 --2- 172.21.15.16:0/3320598523 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0dc809e410 0x7f0dc8122fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:45.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.392+0000 7f0dcca93700 1 -- 172.21.15.16:0/3320598523 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0dc809d6f0 msgr2=0x7f0dc8122550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:45.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.392+0000 7f0dcca93700 1 --2- 172.21.15.16:0/3320598523 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0dc809d6f0 0x7f0dc8122550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:45.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.392+0000 7f0dcca93700 1 -- 172.21.15.16:0/3320598523 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0dc809e410 msgr2=0x7f0dc8122fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:45.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.392+0000 7f0dcca93700 1 --2- 172.21.15.16:0/3320598523 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0dc809e410 0x7f0dc8122fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:45.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.392+0000 7f0dcca93700 1 -- 172.21.15.16:0/3320598523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0db8009550 con 0x7f0dc809e000 2023-09-30T22:34:45.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.392+0000 7f0dcca93700 1 --2- 172.21.15.16:0/3320598523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dc809e000 0x7f0dc8122a90 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f0db800ebe0 tx=0x7f0db800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:45.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.392+0000 7f0dbdffb700 1 -- 172.21.15.16:0/3320598523 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db800c850 con 0x7f0dc809e000 2023-09-30T22:34:45.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.392+0000 7f0dbdffb700 1 -- 172.21.15.16:0/3320598523 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0db800c9b0 con 0x7f0dc809e000 2023-09-30T22:34:45.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.392+0000 7f0dcf4f8700 1 -- 172.21.15.16:0/3320598523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dc0009a70 con 0x7f0dc809e000 2023-09-30T22:34:45.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.392+0000 7f0dbdffb700 1 -- 172.21.15.16:0/3320598523 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db800cd60 con 0x7f0dc809e000 2023-09-30T22:34:45.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.393+0000 7f0dcf4f8700 1 -- 172.21.15.16:0/3320598523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dc40094a0 con 0x7f0dc809e000 2023-09-30T22:34:45.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.393+0000 7f0dbdffb700 1 -- 172.21.15.16:0/3320598523 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0db8013470 con 0x7f0dc809e000 2023-09-30T22:34:45.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.394+0000 7f0dbdffb700 1 -- 172.21.15.16:0/3320598523 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f0db803fb30 con 0x7f0dc809e000 2023-09-30T22:34:45.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.394+0000 7f0dcf4f8700 1 -- 172.21.15.16:0/3320598523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dc8125fd0 con 0x7f0dc809e000 2023-09-30T22:34:45.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.398+0000 7f0dbdffb700 1 -- 172.21.15.16:0/3320598523 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0db8017020 con 0x7f0dc809e000 2023-09-30T22:34:45.409 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:34:45.531 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:34:45.538 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:34:45.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.546+0000 7f2c7ecf8700 1 Processor -- start 2023-09-30T22:34:45.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.546+0000 7f2c7ecf8700 1 -- start start 2023-09-30T22:34:45.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.546+0000 7f2c7ecf8700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2c781338a0 0x7f2c78133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:45.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.546+0000 7f2c7ecf8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c7809e930 0x7f2c7813b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:45.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.546+0000 7f2c7ecf8700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c7806ef30 con 0x7f2c7809de50 2023-09-30T22:34:45.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.546+0000 7f2c7ecf8700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c7806fef0 con 0x7f2c781338a0 2023-09-30T22:34:45.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.546+0000 7f2c7ecf8700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2c781334b0 con 0x7f2c7809e930 2023-09-30T22:34:45.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.546+0000 7f2c77fff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2c7809de50 0x7f2c7809e220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58430/0 (socket says 172.21.15.16:58430) 2023-09-30T22:34:45.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.546+0000 7f2c77fff700 1 -- 172.21.15.16:0/49987450 learned_addr learned my addr 172.21.15.16:0/49987450 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:34:45.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.546+0000 7f2c7ca94700 1 --2- 172.21.15.16:0/49987450 >> v2:172.21.15.16:3300/0 conn(0x7f2c781338a0 0x7f2c78133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:45.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.547+0000 7f2c7d295700 1 --2- 172.21.15.16:0/49987450 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c7809e930 0x7f2c7813b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:45.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.547+0000 7f2c777fe700 1 -- 172.21.15.16:0/49987450 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3375252074 0 0) 0x7f2c7806ef30 con 0x7f2c7809de50 2023-09-30T22:34:45.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.547+0000 7f2c777fe700 1 -- 172.21.15.16:0/49987450 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c680035b0 con 0x7f2c7809de50 2023-09-30T22:34:45.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.547+0000 7f2c7d295700 1 -- 172.21.15.16:0/49987450 >> v2:172.21.15.16:3300/0 conn(0x7f2c781338a0 msgr2=0x7f2c78133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:45.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.547+0000 7f2c7d295700 1 --2- 172.21.15.16:0/49987450 >> v2:172.21.15.16:3300/0 conn(0x7f2c781338a0 0x7f2c78133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:45.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.547+0000 7f2c7d295700 1 -- 172.21.15.16:0/49987450 >> v1:172.21.15.16:6789/0 conn(0x7f2c7809de50 legacy=0x7f2c7809e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:45.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.547+0000 7f2c7d295700 1 -- 172.21.15.16:0/49987450 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c7806fc30 con 0x7f2c7809e930 2023-09-30T22:34:45.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.547+0000 7f2c7ca94700 1 --2- 172.21.15.16:0/49987450 >> v2:172.21.15.16:3300/0 conn(0x7f2c781338a0 0x7f2c78133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:45.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.547+0000 7f2c7d295700 1 --2- 172.21.15.16:0/49987450 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c7809e930 0x7f2c7813b740 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f2c70009d90 tx=0x7f2c7000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b150d92c20830dca server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:45.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.547+0000 7f2c777fe700 1 -- 172.21.15.16:0/49987450 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c7000e070 con 0x7f2c7809e930 2023-09-30T22:34:45.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.547+0000 7f2c777fe700 1 -- 172.21.15.16:0/49987450 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c7000bb20 con 0x7f2c7809e930 2023-09-30T22:34:45.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.547+0000 7f2c777fe700 1 -- 172.21.15.16:0/49987450 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c70012430 con 0x7f2c7809e930 2023-09-30T22:34:45.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.548+0000 7f2c7ecf8700 1 -- 172.21.15.16:0/49987450 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c7809e930 msgr2=0x7f2c7813b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:45.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.548+0000 7f2c7ecf8700 1 --2- 172.21.15.16:0/49987450 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c7809e930 0x7f2c7813b740 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f2c70009d90 tx=0x7f2c7000b330 comp rx=0 tx=0).stop 2023-09-30T22:34:45.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.548+0000 7f2c7ecf8700 1 -- 172.21.15.16:0/49987450 shutdown_connections 2023-09-30T22:34:45.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.548+0000 7f2c7ecf8700 1 --2- 172.21.15.16:0/49987450 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c7809e930 0x7f2c7813b740 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:45.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.548+0000 7f2c7ecf8700 1 --2- 172.21.15.16:0/49987450 >> v2:172.21.15.16:3300/0 conn(0x7f2c781338a0 0x7f2c78133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:45.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.548+0000 7f2c7ecf8700 1 -- 172.21.15.16:0/49987450 >> 172.21.15.16:0/49987450 conn(0x7f2c780a2aa0 msgr2=0x7f2c780a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:45.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.548+0000 7f2c7ecf8700 1 -- 172.21.15.16:0/49987450 shutdown_connections 2023-09-30T22:34:45.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.548+0000 7f2c7ecf8700 1 -- 172.21.15.16:0/49987450 wait complete. 2023-09-30T22:34:45.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.549+0000 7f2c7ecf8700 1 Processor -- start 2023-09-30T22:34:45.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.549+0000 7f2c7ecf8700 1 -- start start 2023-09-30T22:34:45.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.550+0000 7f2c7ecf8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c7809de50 0x7f2c7811d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:45.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.550+0000 7f2c7ecf8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c7809e930 0x7f2c7811dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:45.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.550+0000 7f2c7ecf8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c781338a0 0x7f2c7811e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:45.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.550+0000 7f2c7ecf8700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c781208f0 con 0x7f2c7809de50 2023-09-30T22:34:45.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.550+0000 7f2c7ecf8700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7806f640 con 0x7f2c7809e930 2023-09-30T22:34:45.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.550+0000 7f2c7ecf8700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2c780a0c40 con 0x7f2c781338a0 2023-09-30T22:34:45.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.551+0000 7f2c77fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c7809e930 0x7f2c7811dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:45.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.551+0000 7f2c77fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c7809e930 0x7f2c7811dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:51718/0 (socket says 172.21.15.16:51718) 2023-09-30T22:34:45.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.551+0000 7f2c77fff700 1 -- 172.21.15.16:0/1041382483 learned_addr learned my addr 172.21.15.16:0/1041382483 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:45.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.551+0000 7f2c7d295700 1 --2- 172.21.15.16:0/1041382483 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c781338a0 0x7f2c7811e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:45.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.551+0000 7f2c7ca94700 1 --2- 172.21.15.16:0/1041382483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c7809de50 0x7f2c7811d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:45.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.551+0000 7f2c77fff700 1 -- 172.21.15.16:0/1041382483 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c781338a0 msgr2=0x7f2c7811e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:45.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.551+0000 7f2c77fff700 1 --2- 172.21.15.16:0/1041382483 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c781338a0 0x7f2c7811e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:45.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.551+0000 7f2c77fff700 1 -- 172.21.15.16:0/1041382483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c7809de50 msgr2=0x7f2c7811d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:45.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.551+0000 7f2c77fff700 1 --2- 172.21.15.16:0/1041382483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c7809de50 0x7f2c7811d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:45.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.551+0000 7f2c77fff700 1 -- 172.21.15.16:0/1041382483 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c6c0094a0 con 0x7f2c7809e930 2023-09-30T22:34:45.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.551+0000 7f2c7ca94700 1 --2- 172.21.15.16:0/1041382483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c7809de50 0x7f2c7811d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:34:45.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.551+0000 7f2c77fff700 1 --2- 172.21.15.16:0/1041382483 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c7809e930 0x7f2c7811dcb0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f2c64001eb0 tx=0x7f2c6400c7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:45.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.552+0000 7f2c757fa700 1 -- 172.21.15.16:0/1041382483 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c64005d70 con 0x7f2c7809e930 2023-09-30T22:34:45.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.552+0000 7f2c757fa700 1 -- 172.21.15.16:0/1041382483 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c640070d0 con 0x7f2c7809e930 2023-09-30T22:34:45.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.552+0000 7f2c7ecf8700 1 -- 172.21.15.16:0/1041382483 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c70009a70 con 0x7f2c7809e930 2023-09-30T22:34:45.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.552+0000 7f2c757fa700 1 -- 172.21.15.16:0/1041382483 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c64007460 con 0x7f2c7809e930 2023-09-30T22:34:45.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.552+0000 7f2c7ecf8700 1 -- 172.21.15.16:0/1041382483 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c781204a0 con 0x7f2c7809e930 2023-09-30T22:34:45.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.553+0000 7f2c757fa700 1 -- 172.21.15.16:0/1041382483 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2c640075c0 con 0x7f2c7809e930 2023-09-30T22:34:45.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.553+0000 7f2c7ecf8700 1 -- 172.21.15.16:0/1041382483 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c7811a470 con 0x7f2c7809e930 2023-09-30T22:34:45.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.553+0000 7f2c757fa700 1 -- 172.21.15.16:0/1041382483 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f2c6403eb70 con 0x7f2c7809e930 2023-09-30T22:34:45.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.560+0000 7f2c757fa700 1 -- 172.21.15.16:0/1041382483 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2c64017020 con 0x7f2c7809e930 2023-09-30T22:34:45.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.614+0000 7f0dcf4f8700 1 -- 172.21.15.16:0/3320598523 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0dc8074ea0 con 0x7f0dc809e000 2023-09-30T22:34:45.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.615+0000 7f0dbdffb700 1 -- 172.21.15.16:0/3320598523 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0db801b3b0 con 0x7f0dc809e000 2023-09-30T22:34:45.616 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:34:45.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.617+0000 7f0dcf4f8700 1 -- 172.21.15.16:0/3320598523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dc809e000 msgr2=0x7f0dc8122a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:45.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.617+0000 7f0dcf4f8700 1 --2- 172.21.15.16:0/3320598523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dc809e000 0x7f0dc8122a90 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f0db800ebe0 tx=0x7f0db800eef0 comp rx=0 tx=0).stop 2023-09-30T22:34:45.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.618+0000 7f0dcf4f8700 1 -- 172.21.15.16:0/3320598523 shutdown_connections 2023-09-30T22:34:45.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.618+0000 7f0dcf4f8700 1 --2- 172.21.15.16:0/3320598523 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0dc809e410 0x7f0dc8122fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:45.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.618+0000 7f0dcf4f8700 1 --2- 172.21.15.16:0/3320598523 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0dc809d6f0 0x7f0dc8122550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:45.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.618+0000 7f0dcf4f8700 1 --2- 172.21.15.16:0/3320598523 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0dc809e000 0x7f0dc8122a90 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:45.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.618+0000 7f0dcf4f8700 1 -- 172.21.15.16:0/3320598523 >> 172.21.15.16:0/3320598523 conn(0x7f0dc8066f50 msgr2=0x7f0dc80a19f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:45.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.618+0000 7f0dcf4f8700 1 -- 172.21.15.16:0/3320598523 shutdown_connections 2023-09-30T22:34:45.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.618+0000 7f0dcf4f8700 1 -- 172.21.15.16:0/3320598523 wait complete. 2023-09-30T22:34:45.639 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:34:45.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.677+0000 7f2d644e5700 1 Processor -- start 2023-09-30T22:34:45.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.677+0000 7f2d644e5700 1 -- start start 2023-09-30T22:34:45.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.678+0000 7f2d644e5700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2d5c09e930 0x7f2d5c09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:45.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.678+0000 7f2d644e5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2d5c09f240 0x7f2d5c1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:45.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.678+0000 7f2d644e5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2d5c09f650 0x7f2d5c131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:45.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.678+0000 7f2d644e5700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d5c070940 con 0x7f2d5c09e930 2023-09-30T22:34:45.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.678+0000 7f2d644e5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5c06fd60 con 0x7f2d5c09f650 2023-09-30T22:34:45.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.678+0000 7f2d644e5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5c06fa80 con 0x7f2d5c09f240 2023-09-30T22:34:45.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.678+0000 7f2d61a80700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2d5c09f240 0x7f2d5c1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:45.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.678+0000 7f2d61a80700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2d5c09f240 0x7f2d5c1317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:59474/0 (socket says 172.21.15.16:59474) 2023-09-30T22:34:45.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.678+0000 7f2d62281700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2d5c09e930 0x7f2d5c09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:45.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.678+0000 7f2d61a80700 1 -- 172.21.15.16:0/3454710334 learned_addr learned my addr 172.21.15.16:0/3454710334 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:45.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.678+0000 7f2d62a82700 1 --2- 172.21.15.16:0/3454710334 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2d5c09f650 0x7f2d5c131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:45.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.678+0000 7f2d61a80700 1 -- 172.21.15.16:0/3454710334 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2d5c09f650 msgr2=0x7f2d5c131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:45.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.678+0000 7f2d61a80700 1 --2- 172.21.15.16:0/3454710334 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2d5c09f650 0x7f2d5c131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:45.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.678+0000 7f2d61a80700 1 -- 172.21.15.16:0/3454710334 >> v2:172.21.15.16:3300/0 conn(0x7f2d5c09e930 msgr2=0x7f2d5c09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:45.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.678+0000 7f2d61a80700 1 --2- 172.21.15.16:0/3454710334 >> v2:172.21.15.16:3300/0 conn(0x7f2d5c09e930 0x7f2d5c09ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:45.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.678+0000 7f2d61a80700 1 -- 172.21.15.16:0/3454710334 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d5c09fb60 con 0x7f2d5c09f240 2023-09-30T22:34:45.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.678+0000 7f2d62281700 1 --2- 172.21.15.16:0/3454710334 >> v2:172.21.15.16:3300/0 conn(0x7f2d5c09e930 0x7f2d5c09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:45.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.679+0000 7f2d61a80700 1 --2- 172.21.15.16:0/3454710334 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2d5c09f240 0x7f2d5c1317f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c0098f0 tx=0x7f2d4c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9942d3837f8fd743 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:45.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.679+0000 7f2d6127f700 1 -- 172.21.15.16:0/3454710334 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c010a90 con 0x7f2d5c09f240 2023-09-30T22:34:45.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.679+0000 7f2d6127f700 1 -- 172.21.15.16:0/3454710334 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d4c010bf0 con 0x7f2d5c09f240 2023-09-30T22:34:45.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.679+0000 7f2d6127f700 1 -- 172.21.15.16:0/3454710334 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c018440 con 0x7f2d5c09f240 2023-09-30T22:34:45.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.680+0000 7f2d644e5700 1 -- 172.21.15.16:0/3454710334 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2d5c09f240 msgr2=0x7f2d5c1317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:45.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.680+0000 7f2d644e5700 1 --2- 172.21.15.16:0/3454710334 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2d5c09f240 0x7f2d5c1317f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c0098f0 tx=0x7f2d4c009c00 comp rx=0 tx=0).stop 2023-09-30T22:34:45.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.680+0000 7f2d644e5700 1 -- 172.21.15.16:0/3454710334 shutdown_connections 2023-09-30T22:34:45.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.680+0000 7f2d644e5700 1 --2- 172.21.15.16:0/3454710334 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2d5c09f650 0x7f2d5c131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:45.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.680+0000 7f2d644e5700 1 --2- 172.21.15.16:0/3454710334 >> v2:172.21.15.16:3300/0 conn(0x7f2d5c09e930 0x7f2d5c09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:45.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.680+0000 7f2d644e5700 1 --2- 172.21.15.16:0/3454710334 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2d5c09f240 0x7f2d5c1317f0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:45.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.680+0000 7f2d644e5700 1 -- 172.21.15.16:0/3454710334 >> 172.21.15.16:0/3454710334 conn(0x7f2d5c0a4210 msgr2=0x7f2d5c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:45.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.680+0000 7f2d644e5700 1 -- 172.21.15.16:0/3454710334 shutdown_connections 2023-09-30T22:34:45.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.680+0000 7f2d644e5700 1 -- 172.21.15.16:0/3454710334 wait complete. 2023-09-30T22:34:45.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.681+0000 7f2d644e5700 1 Processor -- start 2023-09-30T22:34:45.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.681+0000 7f2d644e5700 1 -- start start 2023-09-30T22:34:45.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.681+0000 7f2d644e5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2d5c09e930 0x7f2d5c11d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:45.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.681+0000 7f2d644e5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2d5c09f240 0x7f2d5c11d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:45.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.681+0000 7f2d644e5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d5c09f650 0x7f2d5c11de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:45.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.681+0000 7f2d644e5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5c120e90 con 0x7f2d5c09f650 2023-09-30T22:34:45.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.681+0000 7f2d644e5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5c1206c0 con 0x7f2d5c09e930 2023-09-30T22:34:45.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.681+0000 7f2d644e5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5c071d20 con 0x7f2d5c09f240 2023-09-30T22:34:45.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.681+0000 7f2d61a80700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2d5c09f240 0x7f2d5c11d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:45.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.681+0000 7f2d62281700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2d5c09e930 0x7f2d5c11d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:45.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.681+0000 7f2d61a80700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2d5c09f240 0x7f2d5c11d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:59484/0 (socket says 172.21.15.16:59484) 2023-09-30T22:34:45.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.681+0000 7f2d61a80700 1 -- 172.21.15.16:0/3604553496 learned_addr learned my addr 172.21.15.16:0/3604553496 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:45.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.682+0000 7f2d62a82700 1 --2- 172.21.15.16:0/3604553496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d5c09f650 0x7f2d5c11de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:45.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.682+0000 7f2d62281700 1 -- 172.21.15.16:0/3604553496 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2d5c09f240 msgr2=0x7f2d5c11d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:45.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.682+0000 7f2d62281700 1 --2- 172.21.15.16:0/3604553496 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2d5c09f240 0x7f2d5c11d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:45.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.682+0000 7f2d62281700 1 -- 172.21.15.16:0/3604553496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d5c09f650 msgr2=0x7f2d5c11de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:45.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.682+0000 7f2d62281700 1 --2- 172.21.15.16:0/3604553496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d5c09f650 0x7f2d5c11de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:45.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.682+0000 7f2d62281700 1 -- 172.21.15.16:0/3604553496 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d580094a0 con 0x7f2d5c09e930 2023-09-30T22:34:45.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.682+0000 7f2d62281700 1 --2- 172.21.15.16:0/3604553496 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2d5c09e930 0x7f2d5c11d400 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f2d5800e9b0 tx=0x7f2d5800ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:45.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.682+0000 7f2d52ffd700 1 -- 172.21.15.16:0/3604553496 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5800ee50 con 0x7f2d5c09e930 2023-09-30T22:34:45.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.682+0000 7f2d52ffd700 1 -- 172.21.15.16:0/3604553496 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d58004030 con 0x7f2d5c09e930 2023-09-30T22:34:45.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.682+0000 7f2d644e5700 1 -- 172.21.15.16:0/3604553496 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d4c009550 con 0x7f2d5c09e930 2023-09-30T22:34:45.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.682+0000 7f2d52ffd700 1 -- 172.21.15.16:0/3604553496 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d580043c0 con 0x7f2d5c09e930 2023-09-30T22:34:45.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.683+0000 7f2d644e5700 1 -- 172.21.15.16:0/3604553496 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d5c0a0190 con 0x7f2d5c09e930 2023-09-30T22:34:45.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.683+0000 7f2d52ffd700 1 -- 172.21.15.16:0/3604553496 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2d5800c7f0 con 0x7f2d5c09e930 2023-09-30T22:34:45.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.684+0000 7f2d644e5700 1 -- 172.21.15.16:0/3604553496 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d5c074ea0 con 0x7f2d5c09e930 2023-09-30T22:34:45.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.684+0000 7f2d52ffd700 1 -- 172.21.15.16:0/3604553496 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f2d5803f4d0 con 0x7f2d5c09e930 2023-09-30T22:34:45.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.693+0000 7f2d52ffd700 1 -- 172.21.15.16:0/3604553496 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2d58019d60 con 0x7f2d5c09e930 2023-09-30T22:34:45.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.700+0000 7fa1340aa700 1 Processor -- start 2023-09-30T22:34:45.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.700+0000 7fa1340aa700 1 -- start start 2023-09-30T22:34:45.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.701+0000 7fa1340aa700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa12c06c0e0 0x7fa12c09ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:45.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.701+0000 7fa1340aa700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa12c134ec0 0x7fa12c1379e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:45.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.701+0000 7fa1340aa700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa12c137f20 0x7fa12c138390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:45.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.701+0000 7fa1340aa700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa12c06c4b0 con 0x7fa12c137f20 2023-09-30T22:34:45.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.701+0000 7fa1340aa700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa12c0a09c0 con 0x7fa12c134ec0 2023-09-30T22:34:45.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.701+0000 7fa1340aa700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa12c134980 con 0x7fa12c06c0e0 2023-09-30T22:34:45.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.701+0000 7fa131645700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa12c134ec0 0x7fa12c1379e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:45.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.701+0000 7fa131645700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa12c134ec0 0x7fa12c1379e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:51750/0 (socket says 172.21.15.16:51750) 2023-09-30T22:34:45.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.701+0000 7fa131645700 1 -- 172.21.15.16:0/3039598782 learned_addr learned my addr 172.21.15.16:0/3039598782 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:45.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.701+0000 7fa132647700 1 --2- 172.21.15.16:0/3039598782 >> v2:172.21.15.16:3300/0 conn(0x7fa12c137f20 0x7fa12c138390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:45.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.701+0000 7fa131e46700 1 --2- 172.21.15.16:0/3039598782 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa12c06c0e0 0x7fa12c09ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:45.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.702+0000 7fa131645700 1 -- 172.21.15.16:0/3039598782 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa12c06c0e0 msgr2=0x7fa12c09ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:45.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.702+0000 7fa131645700 1 --2- 172.21.15.16:0/3039598782 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa12c06c0e0 0x7fa12c09ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:45.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.702+0000 7fa131645700 1 -- 172.21.15.16:0/3039598782 >> v2:172.21.15.16:3300/0 conn(0x7fa12c137f20 msgr2=0x7fa12c138390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:45.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.702+0000 7fa131645700 1 --2- 172.21.15.16:0/3039598782 >> v2:172.21.15.16:3300/0 conn(0x7fa12c137f20 0x7fa12c138390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:45.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.702+0000 7fa131645700 1 -- 172.21.15.16:0/3039598782 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa12c0a0600 con 0x7fa12c134ec0 2023-09-30T22:34:45.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.702+0000 7fa132647700 1 --2- 172.21.15.16:0/3039598782 >> v2:172.21.15.16:3300/0 conn(0x7fa12c137f20 0x7fa12c138390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:45.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.702+0000 7fa131e46700 1 --2- 172.21.15.16:0/3039598782 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa12c06c0e0 0x7fa12c09ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:45.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.702+0000 7fa131645700 1 --2- 172.21.15.16:0/3039598782 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa12c134ec0 0x7fa12c1379e0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fa11c009920 tx=0x7fa11c009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be380dd8ab4cdc09 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:45.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.702+0000 7fa130e44700 1 -- 172.21.15.16:0/3039598782 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11c011830 con 0x7fa12c134ec0 2023-09-30T22:34:45.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.702+0000 7fa130e44700 1 -- 172.21.15.16:0/3039598782 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa11c011990 con 0x7fa12c134ec0 2023-09-30T22:34:45.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.702+0000 7fa130e44700 1 -- 172.21.15.16:0/3039598782 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11c011d20 con 0x7fa12c134ec0 2023-09-30T22:34:45.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.703+0000 7fa1340aa700 1 -- 172.21.15.16:0/3039598782 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa12c134ec0 msgr2=0x7fa12c1379e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:45.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.703+0000 7fa1340aa700 1 --2- 172.21.15.16:0/3039598782 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa12c134ec0 0x7fa12c1379e0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fa11c009920 tx=0x7fa11c009c30 comp rx=0 tx=0).stop 2023-09-30T22:34:45.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.703+0000 7fa1340aa700 1 -- 172.21.15.16:0/3039598782 shutdown_connections 2023-09-30T22:34:45.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.703+0000 7fa1340aa700 1 --2- 172.21.15.16:0/3039598782 >> v2:172.21.15.16:3300/0 conn(0x7fa12c137f20 0x7fa12c138390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:45.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.703+0000 7fa1340aa700 1 --2- 172.21.15.16:0/3039598782 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa12c06c0e0 0x7fa12c09ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:45.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.703+0000 7fa1340aa700 1 --2- 172.21.15.16:0/3039598782 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa12c134ec0 0x7fa12c1379e0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:45.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.703+0000 7fa1340aa700 1 -- 172.21.15.16:0/3039598782 >> 172.21.15.16:0/3039598782 conn(0x7fa12c0a4210 msgr2=0x7fa12c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:45.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.703+0000 7fa1340aa700 1 -- 172.21.15.16:0/3039598782 shutdown_connections 2023-09-30T22:34:45.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.703+0000 7fa1340aa700 1 -- 172.21.15.16:0/3039598782 wait complete. 2023-09-30T22:34:45.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.704+0000 7fa1340aa700 1 Processor -- start 2023-09-30T22:34:45.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.704+0000 7fa1340aa700 1 -- start start 2023-09-30T22:34:45.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.704+0000 7fa1340aa700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa12c06c0e0 0x7fa12c1ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:45.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.704+0000 7fa1340aa700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa12c134ec0 0x7fa12c1cd290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:45.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.704+0000 7fa1340aa700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa12c137f20 0x7fa12c11a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:45.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.704+0000 7fa1340aa700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa12c11e720 con 0x7fa12c134ec0 2023-09-30T22:34:45.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.704+0000 7fa1340aa700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa12c11d3b0 con 0x7fa12c137f20 2023-09-30T22:34:45.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.704+0000 7fa1340aa700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa12c0a1580 con 0x7fa12c06c0e0 2023-09-30T22:34:45.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.704+0000 7fa131e46700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa12c06c0e0 0x7fa12c1ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:45.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.704+0000 7fa131e46700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa12c06c0e0 0x7fa12c1ccd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:59500/0 (socket says 172.21.15.16:59500) 2023-09-30T22:34:45.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.704+0000 7fa131e46700 1 -- 172.21.15.16:0/3709785112 learned_addr learned my addr 172.21.15.16:0/3709785112 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:45.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.705+0000 7fa131645700 1 --2- 172.21.15.16:0/3709785112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa12c134ec0 0x7fa12c1cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:45.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.705+0000 7fa132647700 1 --2- 172.21.15.16:0/3709785112 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa12c137f20 0x7fa12c11a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:45.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.705+0000 7fa131e46700 1 -- 172.21.15.16:0/3709785112 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa12c137f20 msgr2=0x7fa12c11a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:45.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.705+0000 7fa131e46700 1 --2- 172.21.15.16:0/3709785112 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa12c137f20 0x7fa12c11a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:45.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.705+0000 7fa131e46700 1 -- 172.21.15.16:0/3709785112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa12c134ec0 msgr2=0x7fa12c1cd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:45.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.705+0000 7fa131e46700 1 --2- 172.21.15.16:0/3709785112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa12c134ec0 0x7fa12c1cd290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:45.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.705+0000 7fa131e46700 1 -- 172.21.15.16:0/3709785112 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa128009a70 con 0x7fa12c06c0e0 2023-09-30T22:34:45.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.705+0000 7fa132647700 1 --2- 172.21.15.16:0/3709785112 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa12c137f20 0x7fa12c11a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:45.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.705+0000 7fa131645700 1 --2- 172.21.15.16:0/3709785112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa12c134ec0 0x7fa12c1cd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:45.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.705+0000 7fa131e46700 1 --2- 172.21.15.16:0/3709785112 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa12c06c0e0 0x7fa12c1ccd50 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fa128009ec0 tx=0x7fa12800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:45.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.705+0000 7fa1227fc700 1 -- 172.21.15.16:0/3709785112 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12800cdc0 con 0x7fa12c06c0e0 2023-09-30T22:34:45.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.705+0000 7fa1227fc700 1 -- 172.21.15.16:0/3709785112 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa12800f040 con 0x7fa12c06c0e0 2023-09-30T22:34:45.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.705+0000 7fa1340aa700 1 -- 172.21.15.16:0/3709785112 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1240094a0 con 0x7fa12c06c0e0 2023-09-30T22:34:45.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.705+0000 7fa1227fc700 1 -- 172.21.15.16:0/3709785112 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1280146a0 con 0x7fa12c06c0e0 2023-09-30T22:34:45.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.706+0000 7fa1340aa700 1 -- 172.21.15.16:0/3709785112 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa11c009550 con 0x7fa12c06c0e0 2023-09-30T22:34:45.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.706+0000 7fa1227fc700 1 -- 172.21.15.16:0/3709785112 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa128014800 con 0x7fa12c06c0e0 2023-09-30T22:34:45.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.706+0000 7fa1340aa700 1 -- 172.21.15.16:0/3709785112 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa12c11ccd0 con 0x7fa12c06c0e0 2023-09-30T22:34:45.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.707+0000 7fa1227fc700 1 -- 172.21.15.16:0/3709785112 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fa1280401d0 con 0x7fa12c06c0e0 2023-09-30T22:34:45.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.712+0000 7fa1227fc700 1 -- 172.21.15.16:0/3709785112 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa128007920 con 0x7fa12c06c0e0 2023-09-30T22:34:45.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.782+0000 7f2c7ecf8700 1 -- 172.21.15.16:0/1041382483 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f2c78120d00 con 0x7f2c7809e930 2023-09-30T22:34:45.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.782+0000 7f2c757fa700 1 -- 172.21.15.16:0/1041382483 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2c6400f260 con 0x7f2c7809e930 2023-09-30T22:34:45.783 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:34:45.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.785+0000 7f2c7ecf8700 1 -- 172.21.15.16:0/1041382483 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c7809e930 msgr2=0x7f2c7811dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:45.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.785+0000 7f2c7ecf8700 1 --2- 172.21.15.16:0/1041382483 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c7809e930 0x7f2c7811dcb0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f2c64001eb0 tx=0x7f2c6400c7b0 comp rx=0 tx=0).stop 2023-09-30T22:34:45.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.785+0000 7f2c7ecf8700 1 -- 172.21.15.16:0/1041382483 shutdown_connections 2023-09-30T22:34:45.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.785+0000 7f2c7ecf8700 1 --2- 172.21.15.16:0/1041382483 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c781338a0 0x7f2c7811e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:45.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.785+0000 7f2c7ecf8700 1 --2- 172.21.15.16:0/1041382483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c7809de50 0x7f2c7811d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:45.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.785+0000 7f2c7ecf8700 1 --2- 172.21.15.16:0/1041382483 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c7809e930 0x7f2c7811dcb0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:45.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.785+0000 7f2c7ecf8700 1 -- 172.21.15.16:0/1041382483 >> 172.21.15.16:0/1041382483 conn(0x7f2c780a2aa0 msgr2=0x7f2c781baac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:45.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.785+0000 7f2c7ecf8700 1 -- 172.21.15.16:0/1041382483 shutdown_connections 2023-09-30T22:34:45.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.785+0000 7f2c7ecf8700 1 -- 172.21.15.16:0/1041382483 wait complete. 2023-09-30T22:34:45.799 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:34:45.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.903+0000 7f2d644e5700 1 -- 172.21.15.16:0/3604553496 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2d5c11ef40 con 0x7f2d5c09e930 2023-09-30T22:34:45.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.904+0000 7f2d52ffd700 1 -- 172.21.15.16:0/3604553496 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2d58019300 con 0x7f2d5c09e930 2023-09-30T22:34:45.904 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:34:45.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.908+0000 7f2d644e5700 1 -- 172.21.15.16:0/3604553496 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2d5c09e930 msgr2=0x7f2d5c11d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:45.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.908+0000 7f2d644e5700 1 --2- 172.21.15.16:0/3604553496 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2d5c09e930 0x7f2d5c11d400 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f2d5800e9b0 tx=0x7f2d5800ecc0 comp rx=0 tx=0).stop 2023-09-30T22:34:45.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.908+0000 7f2d644e5700 1 -- 172.21.15.16:0/3604553496 shutdown_connections 2023-09-30T22:34:45.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.908+0000 7f2d644e5700 1 --2- 172.21.15.16:0/3604553496 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2d5c09e930 0x7f2d5c11d400 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:45.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.908+0000 7f2d644e5700 1 --2- 172.21.15.16:0/3604553496 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d5c09f650 0x7f2d5c11de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:45.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.908+0000 7f2d644e5700 1 --2- 172.21.15.16:0/3604553496 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2d5c09f240 0x7f2d5c11d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:45.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.908+0000 7f2d644e5700 1 -- 172.21.15.16:0/3604553496 >> 172.21.15.16:0/3604553496 conn(0x7f2d5c0a4210 msgr2=0x7f2d5c0a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:45.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.908+0000 7f2d644e5700 1 -- 172.21.15.16:0/3604553496 shutdown_connections 2023-09-30T22:34:45.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.908+0000 7f2d644e5700 1 -- 172.21.15.16:0/3604553496 wait complete. 2023-09-30T22:34:45.927 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:34:45.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.927+0000 7fa1340aa700 1 -- 172.21.15.16:0/3709785112 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa12c074ea0 con 0x7fa12c06c0e0 2023-09-30T22:34:45.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.927+0000 7fa1227fc700 1 -- 172.21.15.16:0/3709785112 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa128007330 con 0x7fa12c06c0e0 2023-09-30T22:34:45.928 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:34:45.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.929+0000 7fa1340aa700 1 -- 172.21.15.16:0/3709785112 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa12c06c0e0 msgr2=0x7fa12c1ccd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:45.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.929+0000 7fa1340aa700 1 --2- 172.21.15.16:0/3709785112 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa12c06c0e0 0x7fa12c1ccd50 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fa128009ec0 tx=0x7fa12800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:34:45.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.929+0000 7fa1340aa700 1 -- 172.21.15.16:0/3709785112 shutdown_connections 2023-09-30T22:34:45.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.930+0000 7fa1340aa700 1 --2- 172.21.15.16:0/3709785112 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa12c137f20 0x7fa12c11a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:45.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.930+0000 7fa1340aa700 1 --2- 172.21.15.16:0/3709785112 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa12c06c0e0 0x7fa12c1ccd50 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:45.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.930+0000 7fa1340aa700 1 --2- 172.21.15.16:0/3709785112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa12c134ec0 0x7fa12c1cd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:45.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.930+0000 7fa1340aa700 1 -- 172.21.15.16:0/3709785112 >> 172.21.15.16:0/3709785112 conn(0x7fa12c0a4210 msgr2=0x7fa12c099c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:45.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.930+0000 7fa1340aa700 1 -- 172.21.15.16:0/3709785112 shutdown_connections 2023-09-30T22:34:45.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:45.930+0000 7fa1340aa700 1 -- 172.21.15.16:0/3709785112 wait complete. 2023-09-30T22:34:45.943 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:34:46.640 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:34:46.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.785+0000 7ff272748700 1 Processor -- start 2023-09-30T22:34:46.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.785+0000 7ff272748700 1 -- start start 2023-09-30T22:34:46.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.785+0000 7ff272748700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff26c09d6f0 0x7ff26c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:46.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.786+0000 7ff272748700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff26c09e000 0x7ff26c130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:46.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.786+0000 7ff272748700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff26c09f130 con 0x7ff26c09e410 2023-09-30T22:34:46.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.786+0000 7ff272748700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff26c09f520 con 0x7ff26c09e000 2023-09-30T22:34:46.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.786+0000 7ff272748700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff26c09e930 con 0x7ff26c09d6f0 2023-09-30T22:34:46.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.786+0000 7ff263fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff26c09e000 0x7ff26c130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:46.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.786+0000 7ff26bfff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff26c09d6f0 0x7ff26c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:46.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.786+0000 7ff270ce5700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff26c09e410 0x7ff26c130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58446/0 (socket says 172.21.15.16:58446) 2023-09-30T22:34:46.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.786+0000 7ff270ce5700 1 -- 172.21.15.16:0/2776468341 learned_addr learned my addr 172.21.15.16:0/2776468341 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:34:46.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.786+0000 7ff263fff700 1 -- 172.21.15.16:0/2776468341 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff26c09d6f0 msgr2=0x7ff26c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:46.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.786+0000 7ff263fff700 1 --2- 172.21.15.16:0/2776468341 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff26c09d6f0 0x7ff26c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:46.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.786+0000 7ff263fff700 1 -- 172.21.15.16:0/2776468341 >> v1:172.21.15.16:6789/0 conn(0x7ff26c09e410 legacy=0x7ff26c130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:46.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.786+0000 7ff263fff700 1 -- 172.21.15.16:0/2776468341 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff26c09b2d0 con 0x7ff26c09e000 2023-09-30T22:34:46.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.786+0000 7ff26bfff700 1 --2- 172.21.15.16:0/2776468341 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff26c09d6f0 0x7ff26c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:46.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.786+0000 7ff263fff700 1 --2- 172.21.15.16:0/2776468341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff26c09e000 0x7ff26c130660 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7ff254009840 tx=0x7ff254009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5ca1910f742c02b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:46.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.786+0000 7ff26b7fe700 1 -- 172.21.15.16:0/2776468341 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff254011830 con 0x7ff26c09e000 2023-09-30T22:34:46.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.786+0000 7ff26b7fe700 1 -- 172.21.15.16:0/2776468341 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff254011990 con 0x7ff26c09e000 2023-09-30T22:34:46.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.787+0000 7ff26b7fe700 1 -- 172.21.15.16:0/2776468341 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff254011d20 con 0x7ff26c09e000 2023-09-30T22:34:46.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.787+0000 7ff272748700 1 -- 172.21.15.16:0/2776468341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff26c09e000 msgr2=0x7ff26c130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:46.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.787+0000 7ff272748700 1 --2- 172.21.15.16:0/2776468341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff26c09e000 0x7ff26c130660 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7ff254009840 tx=0x7ff254009b50 comp rx=0 tx=0).stop 2023-09-30T22:34:46.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.787+0000 7ff272748700 1 -- 172.21.15.16:0/2776468341 shutdown_connections 2023-09-30T22:34:46.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.787+0000 7ff272748700 1 --2- 172.21.15.16:0/2776468341 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff26c09d6f0 0x7ff26c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:46.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.787+0000 7ff272748700 1 --2- 172.21.15.16:0/2776468341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff26c09e000 0x7ff26c130660 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:46.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.787+0000 7ff272748700 1 -- 172.21.15.16:0/2776468341 >> 172.21.15.16:0/2776468341 conn(0x7ff26c066f50 msgr2=0x7ff26c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:46.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.787+0000 7ff272748700 1 -- 172.21.15.16:0/2776468341 shutdown_connections 2023-09-30T22:34:46.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.787+0000 7ff272748700 1 -- 172.21.15.16:0/2776468341 wait complete. 2023-09-30T22:34:46.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.788+0000 7ff272748700 1 Processor -- start 2023-09-30T22:34:46.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.788+0000 7ff272748700 1 -- start start 2023-09-30T22:34:46.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.789+0000 7ff272748700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff26c09d6f0 0x7ff26c11df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:46.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.789+0000 7ff272748700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff26c09e000 0x7ff26c11e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:46.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.789+0000 7ff272748700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff26c09e410 0x7ff26c11ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:46.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.789+0000 7ff272748700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff26c11f450 con 0x7ff26c09e000 2023-09-30T22:34:46.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.789+0000 7ff272748700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff26c121aa0 con 0x7ff26c09d6f0 2023-09-30T22:34:46.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.789+0000 7ff272748700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff26c1201b0 con 0x7ff26c09e410 2023-09-30T22:34:46.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.789+0000 7ff26bfff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff26c09d6f0 0x7ff26c11df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:46.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.789+0000 7ff263fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff26c09e000 0x7ff26c11e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:46.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.789+0000 7ff270ce5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff26c09e410 0x7ff26c11ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:46.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.789+0000 7ff26bfff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff26c09d6f0 0x7ff26c11df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:51794/0 (socket says 172.21.15.16:51794) 2023-09-30T22:34:46.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.789+0000 7ff26bfff700 1 -- 172.21.15.16:0/2359651632 learned_addr learned my addr 172.21.15.16:0/2359651632 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:46.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.789+0000 7ff263fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff26c09e000 0x7ff26c11e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56848/0 (socket says 172.21.15.16:56848) 2023-09-30T22:34:46.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.789+0000 7ff26bfff700 1 -- 172.21.15.16:0/2359651632 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff26c09e410 msgr2=0x7ff26c11ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:46.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.789+0000 7ff26bfff700 1 --2- 172.21.15.16:0/2359651632 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff26c09e410 0x7ff26c11ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:46.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.789+0000 7ff26bfff700 1 -- 172.21.15.16:0/2359651632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff26c09e000 msgr2=0x7ff26c11e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:46.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.789+0000 7ff26bfff700 1 --2- 172.21.15.16:0/2359651632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff26c09e000 0x7ff26c11e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:46.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.789+0000 7ff26bfff700 1 -- 172.21.15.16:0/2359651632 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff25c0094a0 con 0x7ff26c09d6f0 2023-09-30T22:34:46.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.789+0000 7ff26bfff700 1 --2- 172.21.15.16:0/2359651632 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff26c09d6f0 0x7ff26c11df80 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7ff25c00ebe0 tx=0x7ff25c00eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:46.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.790+0000 7ff2697fa700 1 -- 172.21.15.16:0/2359651632 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25c009760 con 0x7ff26c09d6f0 2023-09-30T22:34:46.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.790+0000 7ff2697fa700 1 -- 172.21.15.16:0/2359651632 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff25c00c850 con 0x7ff26c09d6f0 2023-09-30T22:34:46.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.790+0000 7ff272748700 1 -- 172.21.15.16:0/2359651632 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2540094a0 con 0x7ff26c09d6f0 2023-09-30T22:34:46.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.790+0000 7ff2697fa700 1 -- 172.21.15.16:0/2359651632 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25c00cbe0 con 0x7ff26c09d6f0 2023-09-30T22:34:46.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.790+0000 7ff272748700 1 -- 172.21.15.16:0/2359651632 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff26c0a1d70 con 0x7ff26c09d6f0 2023-09-30T22:34:46.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.790+0000 7ff2697fa700 1 -- 172.21.15.16:0/2359651632 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff25c00cd40 con 0x7ff26c09d6f0 2023-09-30T22:34:46.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.791+0000 7ff2697fa700 1 -- 172.21.15.16:0/2359651632 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7ff25c03f730 con 0x7ff26c09d6f0 2023-09-30T22:34:46.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.791+0000 7ff272748700 1 -- 172.21.15.16:0/2359651632 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff26c120e70 con 0x7ff26c09d6f0 2023-09-30T22:34:46.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.796+0000 7ff2697fa700 1 -- 172.21.15.16:0/2359651632 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff25c0232f0 con 0x7ff26c09d6f0 2023-09-30T22:34:46.800 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:34:46.927 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:34:46.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.937+0000 7f16ce568700 1 Processor -- start 2023-09-30T22:34:46.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.937+0000 7f16ce568700 1 -- start start 2023-09-30T22:34:46.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.937+0000 7f16ce568700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f16c81338a0 0x7f16c8133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:46.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.938+0000 7f16ce568700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f16c809de50 0x7f16c809e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:46.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.938+0000 7f16ce568700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f16c809e760 0x7f16c813a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:46.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.938+0000 7f16ce568700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16c806fef0 con 0x7f16c809de50 2023-09-30T22:34:46.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.938+0000 7f16ce568700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16c806ef30 con 0x7f16c809e760 2023-09-30T22:34:46.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.938+0000 7f16ce568700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f16c81334b0 con 0x7f16c81338a0 2023-09-30T22:34:46.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.938+0000 7f16c7fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f16c81338a0 0x7f16c8133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:46.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.938+0000 7f16c7fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f16c81338a0 0x7f16c8133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:59518/0 (socket says 172.21.15.16:59518) 2023-09-30T22:34:46.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.938+0000 7f16c7fff700 1 -- 172.21.15.16:0/1142896638 learned_addr learned my addr 172.21.15.16:0/1142896638 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:46.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.938+0000 7f16c77fe700 1 --2- 172.21.15.16:0/1142896638 >> v2:172.21.15.16:3300/0 conn(0x7f16c809de50 0x7f16c809e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:46.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.938+0000 7f16ccb05700 1 --2- 172.21.15.16:0/1142896638 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f16c809e760 0x7f16c813a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:46.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.938+0000 7f16c7fff700 1 -- 172.21.15.16:0/1142896638 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f16c809e760 msgr2=0x7f16c813a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:46.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.938+0000 7f16c7fff700 1 --2- 172.21.15.16:0/1142896638 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f16c809e760 0x7f16c813a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:46.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.938+0000 7f16c7fff700 1 -- 172.21.15.16:0/1142896638 >> v2:172.21.15.16:3300/0 conn(0x7f16c809de50 msgr2=0x7f16c809e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:46.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.938+0000 7f16c7fff700 1 --2- 172.21.15.16:0/1142896638 >> v2:172.21.15.16:3300/0 conn(0x7f16c809de50 0x7f16c809e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:46.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.938+0000 7f16c7fff700 1 -- 172.21.15.16:0/1142896638 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c806fc30 con 0x7f16c81338a0 2023-09-30T22:34:46.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.938+0000 7f16c7fff700 1 --2- 172.21.15.16:0/1142896638 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f16c81338a0 0x7f16c8133c70 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f16b80098f0 tx=0x7f16b8009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f734558fbced3b55 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:46.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.939+0000 7f16c6ffd700 1 -- 172.21.15.16:0/1142896638 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b8011830 con 0x7f16c81338a0 2023-09-30T22:34:46.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.939+0000 7f16c6ffd700 1 -- 172.21.15.16:0/1142896638 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16b8011990 con 0x7f16c81338a0 2023-09-30T22:34:46.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.939+0000 7f16c6ffd700 1 -- 172.21.15.16:0/1142896638 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b8011d20 con 0x7f16c81338a0 2023-09-30T22:34:46.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.939+0000 7f16ce568700 1 -- 172.21.15.16:0/1142896638 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f16c81338a0 msgr2=0x7f16c8133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:46.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.939+0000 7f16ce568700 1 --2- 172.21.15.16:0/1142896638 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f16c81338a0 0x7f16c8133c70 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f16b80098f0 tx=0x7f16b8009c00 comp rx=0 tx=0).stop 2023-09-30T22:34:46.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.939+0000 7f16ce568700 1 -- 172.21.15.16:0/1142896638 shutdown_connections 2023-09-30T22:34:46.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.939+0000 7f16ce568700 1 --2- 172.21.15.16:0/1142896638 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f16c809e760 0x7f16c813a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:46.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.939+0000 7f16ce568700 1 --2- 172.21.15.16:0/1142896638 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f16c81338a0 0x7f16c8133c70 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:46.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.939+0000 7f16ce568700 1 --2- 172.21.15.16:0/1142896638 >> v2:172.21.15.16:3300/0 conn(0x7f16c809de50 0x7f16c809e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:46.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.939+0000 7f16ce568700 1 -- 172.21.15.16:0/1142896638 >> 172.21.15.16:0/1142896638 conn(0x7f16c80a2aa0 msgr2=0x7f16c80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:46.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.940+0000 7f16ce568700 1 -- 172.21.15.16:0/1142896638 shutdown_connections 2023-09-30T22:34:46.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.940+0000 7f16ce568700 1 -- 172.21.15.16:0/1142896638 wait complete. 2023-09-30T22:34:46.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.941+0000 7f16ce568700 1 Processor -- start 2023-09-30T22:34:46.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.941+0000 7f16ce568700 1 -- start start 2023-09-30T22:34:46.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.941+0000 7f16ce568700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f16c809de50 0x7f16c811a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:46.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.941+0000 7f16ce568700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f16c809e760 0x7f16c811adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:46.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.942+0000 7f16ce568700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16c81338a0 0x7f16c811b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:46.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.942+0000 7f16ce568700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c80a1000 con 0x7f16c81338a0 2023-09-30T22:34:46.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.942+0000 7f16ce568700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16c811e2b0 con 0x7f16c809e760 2023-09-30T22:34:46.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.942+0000 7f16ce568700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f16c806f640 con 0x7f16c809de50 2023-09-30T22:34:46.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.942+0000 7f16c77fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f16c809e760 0x7f16c811adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:46.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.942+0000 7f16c77fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f16c809e760 0x7f16c811adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:51814/0 (socket says 172.21.15.16:51814) 2023-09-30T22:34:46.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.942+0000 7f16c77fe700 1 -- 172.21.15.16:0/3941828724 learned_addr learned my addr 172.21.15.16:0/3941828724 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:46.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.942+0000 7f16ccb05700 1 --2- 172.21.15.16:0/3941828724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16c81338a0 0x7f16c811b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:46.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.942+0000 7f16c7fff700 1 --2- 172.21.15.16:0/3941828724 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f16c809de50 0x7f16c811a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:46.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.942+0000 7f16c77fe700 1 -- 172.21.15.16:0/3941828724 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f16c809de50 msgr2=0x7f16c811a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:46.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.942+0000 7f16c77fe700 1 --2- 172.21.15.16:0/3941828724 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f16c809de50 0x7f16c811a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:46.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.942+0000 7f16c77fe700 1 -- 172.21.15.16:0/3941828724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16c81338a0 msgr2=0x7f16c811b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:46.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.942+0000 7f16c77fe700 1 --2- 172.21.15.16:0/3941828724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16c81338a0 0x7f16c811b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:46.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.942+0000 7f16c77fe700 1 -- 172.21.15.16:0/3941828724 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b8009550 con 0x7f16c809e760 2023-09-30T22:34:46.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.942+0000 7f16c7fff700 1 --2- 172.21.15.16:0/3941828724 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f16c809de50 0x7f16c811a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:34:46.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.942+0000 7f16c77fe700 1 --2- 172.21.15.16:0/3941828724 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f16c809e760 0x7f16c811adc0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f16bc00e840 tx=0x7f16bc00ec00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:46.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.942+0000 7f16c4ff9700 1 -- 172.21.15.16:0/3941828724 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16bc009590 con 0x7f16c809e760 2023-09-30T22:34:46.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.943+0000 7f16c4ff9700 1 -- 172.21.15.16:0/3941828724 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16bc013680 con 0x7f16c809e760 2023-09-30T22:34:46.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.943+0000 7f16ce568700 1 -- 172.21.15.16:0/3941828724 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16c812dc80 con 0x7f16c809e760 2023-09-30T22:34:46.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.943+0000 7f16c4ff9700 1 -- 172.21.15.16:0/3941828724 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16bc013a10 con 0x7f16c809e760 2023-09-30T22:34:46.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.943+0000 7f16ce568700 1 -- 172.21.15.16:0/3941828724 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16c812d910 con 0x7f16c809e760 2023-09-30T22:34:46.947 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:34:46.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.944+0000 7f16ce568700 1 -- 172.21.15.16:0/3941828724 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16a80052e0 con 0x7f16c809e760 2023-09-30T22:34:46.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.944+0000 7f16c4ff9700 1 -- 172.21.15.16:0/3941828724 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f16bc013b70 con 0x7f16c809e760 2023-09-30T22:34:46.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.944+0000 7f16c4ff9700 1 -- 172.21.15.16:0/3941828724 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f16bc03faf0 con 0x7f16c809e760 2023-09-30T22:34:46.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:46.953+0000 7f16c4ff9700 1 -- 172.21.15.16:0/3941828724 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f16bc011d90 con 0x7f16c809e760 2023-09-30T22:34:47.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.015+0000 7ff272748700 1 -- 172.21.15.16:0/2359651632 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff26c121620 con 0x7ff26c09d6f0 2023-09-30T22:34:47.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.016+0000 7ff2697fa700 1 -- 172.21.15.16:0/2359651632 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff26c121620 con 0x7ff26c09d6f0 2023-09-30T22:34:47.017 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:34:47.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.019+0000 7ff272748700 1 -- 172.21.15.16:0/2359651632 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff26c09d6f0 msgr2=0x7ff26c11df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:47.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.019+0000 7ff272748700 1 --2- 172.21.15.16:0/2359651632 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff26c09d6f0 0x7ff26c11df80 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7ff25c00ebe0 tx=0x7ff25c00eef0 comp rx=0 tx=0).stop 2023-09-30T22:34:47.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.020+0000 7ff272748700 1 -- 172.21.15.16:0/2359651632 shutdown_connections 2023-09-30T22:34:47.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.020+0000 7ff272748700 1 --2- 172.21.15.16:0/2359651632 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff26c09d6f0 0x7ff26c11df80 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:47.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.020+0000 7ff272748700 1 --2- 172.21.15.16:0/2359651632 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff26c09e410 0x7ff26c11ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:47.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.020+0000 7ff272748700 1 --2- 172.21.15.16:0/2359651632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff26c09e000 0x7ff26c11e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:47.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.020+0000 7ff272748700 1 -- 172.21.15.16:0/2359651632 >> 172.21.15.16:0/2359651632 conn(0x7ff26c066f50 msgr2=0x7ff26c0710c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:47.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.022+0000 7ff272748700 1 -- 172.21.15.16:0/2359651632 shutdown_connections 2023-09-30T22:34:47.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.022+0000 7ff272748700 1 -- 172.21.15.16:0/2359651632 wait complete. 2023-09-30T22:34:47.043 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:34:47.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.070+0000 7fc938aff700 1 Processor -- start 2023-09-30T22:34:47.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.071+0000 7fc938aff700 1 -- start start 2023-09-30T22:34:47.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.071+0000 7fc938aff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc93406a770 0x7fc93406ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:47.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.071+0000 7fc938aff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc934131d10 0x7fc934139860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:47.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.071+0000 7fc938aff700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc93409ece0 con 0x7fc93406b080 2023-09-30T22:34:47.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.071+0000 7fc938aff700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc93409e760 con 0x7fc93406a770 2023-09-30T22:34:47.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.071+0000 7fc938aff700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc93409e040 con 0x7fc934131d10 2023-09-30T22:34:47.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.071+0000 7fc93259c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc93406a770 0x7fc93406ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:47.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.071+0000 7fc929d9b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc93406b080 0x7fc934131600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58448/0 (socket says 172.21.15.16:58448) 2023-09-30T22:34:47.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.071+0000 7fc929d9b700 1 -- 172.21.15.16:0/67220421 learned_addr learned my addr 172.21.15.16:0/67220421 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:34:47.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.071+0000 7fc932d9d700 1 --2- 172.21.15.16:0/67220421 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc934131d10 0x7fc934139860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:47.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.072+0000 7fc93259c700 1 -- 172.21.15.16:0/67220421 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc934131d10 msgr2=0x7fc934139860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:47.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.072+0000 7fc93259c700 1 --2- 172.21.15.16:0/67220421 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc934131d10 0x7fc934139860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:47.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.072+0000 7fc93259c700 1 -- 172.21.15.16:0/67220421 >> v1:172.21.15.16:6789/0 conn(0x7fc93406b080 legacy=0x7fc934131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:47.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.072+0000 7fc931d9b700 1 -- 172.21.15.16:0/67220421 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2364583288 0 0) 0x7fc93409ece0 con 0x7fc93406b080 2023-09-30T22:34:47.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.072+0000 7fc93259c700 1 -- 172.21.15.16:0/67220421 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc93409f630 con 0x7fc93406a770 2023-09-30T22:34:47.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.072+0000 7fc932d9d700 1 --2- 172.21.15.16:0/67220421 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc934131d10 0x7fc934139860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:47.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.072+0000 7fc93259c700 1 --2- 172.21.15.16:0/67220421 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc93406a770 0x7fc93406ab40 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fc924009870 tx=0x7fc924009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=205090ab91523885 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:47.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.072+0000 7fc931d9b700 1 -- 172.21.15.16:0/67220421 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc924011830 con 0x7fc93406a770 2023-09-30T22:34:47.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.072+0000 7fc931d9b700 1 -- 172.21.15.16:0/67220421 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc924011990 con 0x7fc93406a770 2023-09-30T22:34:47.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.072+0000 7fc931d9b700 1 -- 172.21.15.16:0/67220421 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc924011d20 con 0x7fc93406a770 2023-09-30T22:34:47.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.073+0000 7fc938aff700 1 -- 172.21.15.16:0/67220421 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc93406a770 msgr2=0x7fc93406ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:47.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.073+0000 7fc938aff700 1 --2- 172.21.15.16:0/67220421 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc93406a770 0x7fc93406ab40 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fc924009870 tx=0x7fc924009b80 comp rx=0 tx=0).stop 2023-09-30T22:34:47.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.073+0000 7fc938aff700 1 -- 172.21.15.16:0/67220421 shutdown_connections 2023-09-30T22:34:47.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.073+0000 7fc938aff700 1 --2- 172.21.15.16:0/67220421 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc934131d10 0x7fc934139860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:47.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.073+0000 7fc938aff700 1 --2- 172.21.15.16:0/67220421 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc93406a770 0x7fc93406ab40 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:47.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.073+0000 7fc938aff700 1 -- 172.21.15.16:0/67220421 >> 172.21.15.16:0/67220421 conn(0x7fc934066f50 msgr2=0x7fc93409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:47.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.073+0000 7fc938aff700 1 -- 172.21.15.16:0/67220421 shutdown_connections 2023-09-30T22:34:47.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.073+0000 7fc938aff700 1 -- 172.21.15.16:0/67220421 wait complete. 2023-09-30T22:34:47.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.074+0000 7fc938aff700 1 Processor -- start 2023-09-30T22:34:47.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.074+0000 7fc938aff700 1 -- start start 2023-09-30T22:34:47.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.075+0000 7fc938aff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc93406a770 0x7fc93411d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:47.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.075+0000 7fc938aff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc93406b080 0x7fc93411ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:47.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.075+0000 7fc938aff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc934131d10 0x7fc93411e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:47.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.075+0000 7fc938aff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc93411ffc0 con 0x7fc93406b080 2023-09-30T22:34:47.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.075+0000 7fc938aff700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc934130fa0 con 0x7fc93406a770 2023-09-30T22:34:47.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.075+0000 7fc938aff700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc9341201a0 con 0x7fc934131d10 2023-09-30T22:34:47.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.075+0000 7fc929d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc93406b080 0x7fc93411ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:47.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.075+0000 7fc93259c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc93406a770 0x7fc93411d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:47.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.075+0000 7fc93259c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc93406a770 0x7fc93411d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:51834/0 (socket says 172.21.15.16:51834) 2023-09-30T22:34:47.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.075+0000 7fc93259c700 1 -- 172.21.15.16:0/2340250737 learned_addr learned my addr 172.21.15.16:0/2340250737 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:47.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.075+0000 7fc929d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc93406b080 0x7fc93411ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56882/0 (socket says 172.21.15.16:56882) 2023-09-30T22:34:47.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.075+0000 7fc932d9d700 1 --2- 172.21.15.16:0/2340250737 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc934131d10 0x7fc93411e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:47.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.075+0000 7fc93259c700 1 -- 172.21.15.16:0/2340250737 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc934131d10 msgr2=0x7fc93411e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:47.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.075+0000 7fc93259c700 1 --2- 172.21.15.16:0/2340250737 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc934131d10 0x7fc93411e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:47.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.075+0000 7fc93259c700 1 -- 172.21.15.16:0/2340250737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc93406b080 msgr2=0x7fc93411ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:47.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.075+0000 7fc93259c700 1 --2- 172.21.15.16:0/2340250737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc93406b080 0x7fc93411ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:47.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.075+0000 7fc93259c700 1 -- 172.21.15.16:0/2340250737 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc92c009a70 con 0x7fc93406a770 2023-09-30T22:34:47.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.075+0000 7fc932d9d700 1 --2- 172.21.15.16:0/2340250737 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc934131d10 0x7fc93411e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:47.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.076+0000 7fc929d9b700 1 --2- 172.21.15.16:0/2340250737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc93406b080 0x7fc93411ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:47.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.076+0000 7fc93259c700 1 --2- 172.21.15.16:0/2340250737 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc93406a770 0x7fc93411d890 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fc9240090b0 tx=0x7fc924004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:47.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.076+0000 7fc92b7fe700 1 -- 172.21.15.16:0/2340250737 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9240044d0 con 0x7fc93406a770 2023-09-30T22:34:47.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.076+0000 7fc92b7fe700 1 -- 172.21.15.16:0/2340250737 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc92401f070 con 0x7fc93406a770 2023-09-30T22:34:47.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.076+0000 7fc938aff700 1 -- 172.21.15.16:0/2340250737 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9240094a0 con 0x7fc93406a770 2023-09-30T22:34:47.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.076+0000 7fc92b7fe700 1 -- 172.21.15.16:0/2340250737 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9240038e0 con 0x7fc93406a770 2023-09-30T22:34:47.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.076+0000 7fc938aff700 1 -- 172.21.15.16:0/2340250737 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc93411fc00 con 0x7fc93406a770 2023-09-30T22:34:47.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.077+0000 7fc938aff700 1 -- 172.21.15.16:0/2340250737 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc934074ea0 con 0x7fc93406a770 2023-09-30T22:34:47.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.078+0000 7fc92b7fe700 1 -- 172.21.15.16:0/2340250737 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc924003a40 con 0x7fc93406a770 2023-09-30T22:34:47.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.078+0000 7fc92b7fe700 1 -- 172.21.15.16:0/2340250737 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fc924021070 con 0x7fc93406a770 2023-09-30T22:34:47.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.086+0000 7fc92b7fe700 1 -- 172.21.15.16:0/2340250737 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc924010380 con 0x7fc93406a770 2023-09-30T22:34:47.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.095+0000 7f6c27824700 1 Processor -- start 2023-09-30T22:34:47.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.095+0000 7f6c27824700 1 -- start start 2023-09-30T22:34:47.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.095+0000 7f6c27824700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6c2006a770 0x7f6c2009d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:47.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.095+0000 7f6c27824700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6c2009dc30 0x7f6c2009e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:47.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.095+0000 7f6c27824700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c2009a810 con 0x7f6c2013c1e0 2023-09-30T22:34:47.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.095+0000 7f6c27824700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c2006ab40 con 0x7f6c2006a770 2023-09-30T22:34:47.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.095+0000 7f6c27824700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2009ebc0 con 0x7f6c2009dc30 2023-09-30T22:34:47.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.096+0000 7f6c255c0700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6c2006a770 0x7f6c2009d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:47.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.096+0000 7f6c25dc1700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6c2013c1e0 0x7f6c2013e5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58458/0 (socket says 172.21.15.16:58458) 2023-09-30T22:34:47.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.096+0000 7f6c25dc1700 1 -- 172.21.15.16:0/2978589937 learned_addr learned my addr 172.21.15.16:0/2978589937 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:34:47.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.096+0000 7f6c24dbf700 1 --2- 172.21.15.16:0/2978589937 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6c2009dc30 0x7f6c2009e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:47.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.096+0000 7f6c255c0700 1 -- 172.21.15.16:0/2978589937 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6c2009dc30 msgr2=0x7f6c2009e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:47.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.096+0000 7f6c255c0700 1 --2- 172.21.15.16:0/2978589937 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6c2009dc30 0x7f6c2009e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:47.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.096+0000 7f6c255c0700 1 -- 172.21.15.16:0/2978589937 >> v1:172.21.15.16:6789/0 conn(0x7f6c2013c1e0 legacy=0x7f6c2013e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:47.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.096+0000 7f6c255c0700 1 -- 172.21.15.16:0/2978589937 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c2009e920 con 0x7f6c2006a770 2023-09-30T22:34:47.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.096+0000 7f6c255c0700 1 --2- 172.21.15.16:0/2978589937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c2006a770 0x7f6c2009d6f0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c009840 tx=0x7f6c1c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15befa99837154d4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:47.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.096+0000 7f6c17fff700 1 -- 172.21.15.16:0/2978589937 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c011830 con 0x7f6c2006a770 2023-09-30T22:34:47.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.096+0000 7f6c17fff700 1 -- 172.21.15.16:0/2978589937 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c1c011990 con 0x7f6c2006a770 2023-09-30T22:34:47.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.097+0000 7f6c17fff700 1 -- 172.21.15.16:0/2978589937 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c011d20 con 0x7f6c2006a770 2023-09-30T22:34:47.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.097+0000 7f6c27824700 1 -- 172.21.15.16:0/2978589937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c2006a770 msgr2=0x7f6c2009d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:47.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.097+0000 7f6c27824700 1 --2- 172.21.15.16:0/2978589937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c2006a770 0x7f6c2009d6f0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c009840 tx=0x7f6c1c009b50 comp rx=0 tx=0).stop 2023-09-30T22:34:47.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.097+0000 7f6c27824700 1 -- 172.21.15.16:0/2978589937 shutdown_connections 2023-09-30T22:34:47.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.097+0000 7f6c27824700 1 --2- 172.21.15.16:0/2978589937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c2006a770 0x7f6c2009d6f0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:47.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.097+0000 7f6c27824700 1 --2- 172.21.15.16:0/2978589937 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6c2009dc30 0x7f6c2009e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:47.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.097+0000 7f6c27824700 1 -- 172.21.15.16:0/2978589937 >> 172.21.15.16:0/2978589937 conn(0x7f6c20066f50 msgr2=0x7f6c2009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:47.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.097+0000 7f6c27824700 1 -- 172.21.15.16:0/2978589937 shutdown_connections 2023-09-30T22:34:47.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.097+0000 7f6c27824700 1 -- 172.21.15.16:0/2978589937 wait complete. 2023-09-30T22:34:47.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.098+0000 7f6c27824700 1 Processor -- start 2023-09-30T22:34:47.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.098+0000 7f6c27824700 1 -- start start 2023-09-30T22:34:47.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.099+0000 7f6c27824700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6c2006a770 0x7f6c2011b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:47.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.099+0000 7f6c27824700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c2009dc30 0x7f6c2013a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:47.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.099+0000 7f6c27824700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6c2013c1e0 0x7f6c2013aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:47.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.099+0000 7f6c27824700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c20134cb0 con 0x7f6c2009dc30 2023-09-30T22:34:47.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.099+0000 7f6c27824700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2011dc00 con 0x7f6c2006a770 2023-09-30T22:34:47.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.099+0000 7f6c27824700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2011e120 con 0x7f6c2013c1e0 2023-09-30T22:34:47.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.099+0000 7f6c255c0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6c2006a770 0x7f6c2011b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:47.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.099+0000 7f6c255c0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6c2006a770 0x7f6c2011b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:51854/0 (socket says 172.21.15.16:51854) 2023-09-30T22:34:47.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.099+0000 7f6c255c0700 1 -- 172.21.15.16:0/3125220530 learned_addr learned my addr 172.21.15.16:0/3125220530 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:47.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.099+0000 7f6c24dbf700 1 --2- 172.21.15.16:0/3125220530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c2009dc30 0x7f6c2013a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:47.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.099+0000 7f6c25dc1700 1 --2- 172.21.15.16:0/3125220530 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6c2013c1e0 0x7f6c2013aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:47.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.099+0000 7f6c255c0700 1 -- 172.21.15.16:0/3125220530 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6c2013c1e0 msgr2=0x7f6c2013aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:47.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.099+0000 7f6c255c0700 1 --2- 172.21.15.16:0/3125220530 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6c2013c1e0 0x7f6c2013aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:47.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.099+0000 7f6c255c0700 1 -- 172.21.15.16:0/3125220530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c2009dc30 msgr2=0x7f6c2013a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:47.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.099+0000 7f6c255c0700 1 --2- 172.21.15.16:0/3125220530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c2009dc30 0x7f6c2013a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:47.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.099+0000 7f6c255c0700 1 -- 172.21.15.16:0/3125220530 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c1c0094a0 con 0x7f6c2006a770 2023-09-30T22:34:47.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.099+0000 7f6c24dbf700 1 --2- 172.21.15.16:0/3125220530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c2009dc30 0x7f6c2013a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:47.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.100+0000 7f6c255c0700 1 --2- 172.21.15.16:0/3125220530 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6c2006a770 0x7f6c2011b340 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c0090b0 tx=0x7f6c1c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:47.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.100+0000 7f6c15ffb700 1 -- 172.21.15.16:0/3125220530 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c0036a0 con 0x7f6c2006a770 2023-09-30T22:34:47.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.100+0000 7f6c15ffb700 1 -- 172.21.15.16:0/3125220530 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c1c01f070 con 0x7f6c2006a770 2023-09-30T22:34:47.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.100+0000 7f6c27824700 1 -- 172.21.15.16:0/3125220530 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c20071e40 con 0x7f6c2006a770 2023-09-30T22:34:47.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.100+0000 7f6c15ffb700 1 -- 172.21.15.16:0/3125220530 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c003a60 con 0x7f6c2006a770 2023-09-30T22:34:47.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.100+0000 7f6c27824700 1 -- 172.21.15.16:0/3125220530 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c2011c730 con 0x7f6c2006a770 2023-09-30T22:34:47.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.101+0000 7f6c15ffb700 1 -- 172.21.15.16:0/3125220530 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6c1c003bc0 con 0x7f6c2006a770 2023-09-30T22:34:47.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.101+0000 7f6c15ffb700 1 -- 172.21.15.16:0/3125220530 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f6c1c021070 con 0x7f6c2006a770 2023-09-30T22:34:47.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.101+0000 7f6c27824700 1 -- 172.21.15.16:0/3125220530 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c2013bcc0 con 0x7f6c2006a770 2023-09-30T22:34:47.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.106+0000 7f6c15ffb700 1 -- 172.21.15.16:0/3125220530 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6c1c015080 con 0x7f6c2006a770 2023-09-30T22:34:47.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.180+0000 7f16ce568700 1 -- 172.21.15.16:0/3941828724 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f16a8005590 con 0x7f16c809e760 2023-09-30T22:34:47.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.181+0000 7f16c4ff9700 1 -- 172.21.15.16:0/3941828724 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f16bc03fe60 con 0x7f16c809e760 2023-09-30T22:34:47.182 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:34:47.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.183+0000 7f16ce568700 1 -- 172.21.15.16:0/3941828724 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f16c809e760 msgr2=0x7f16c811adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:47.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.183+0000 7f16ce568700 1 --2- 172.21.15.16:0/3941828724 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f16c809e760 0x7f16c811adc0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f16bc00e840 tx=0x7f16bc00ec00 comp rx=0 tx=0).stop 2023-09-30T22:34:47.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.183+0000 7f16ce568700 1 -- 172.21.15.16:0/3941828724 shutdown_connections 2023-09-30T22:34:47.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.183+0000 7f16ce568700 1 --2- 172.21.15.16:0/3941828724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16c81338a0 0x7f16c811b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:47.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.183+0000 7f16ce568700 1 --2- 172.21.15.16:0/3941828724 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f16c809de50 0x7f16c811a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:47.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.183+0000 7f16ce568700 1 --2- 172.21.15.16:0/3941828724 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f16c809e760 0x7f16c811adc0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:47.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.183+0000 7f16ce568700 1 -- 172.21.15.16:0/3941828724 >> 172.21.15.16:0/3941828724 conn(0x7f16c80a2aa0 msgr2=0x7f16c81b9410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:47.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.183+0000 7f16ce568700 1 -- 172.21.15.16:0/3941828724 shutdown_connections 2023-09-30T22:34:47.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.183+0000 7f16ce568700 1 -- 172.21.15.16:0/3941828724 wait complete. 2023-09-30T22:34:47.198 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:34:47.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.296+0000 7fc938aff700 1 -- 172.21.15.16:0/2340250737 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc93406c360 con 0x7fc93406a770 2023-09-30T22:34:47.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.297+0000 7fc92b7fe700 1 -- 172.21.15.16:0/2340250737 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc924034090 con 0x7fc93406a770 2023-09-30T22:34:47.297 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:34:47.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.299+0000 7fc938aff700 1 -- 172.21.15.16:0/2340250737 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc93406a770 msgr2=0x7fc93411d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:47.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.299+0000 7fc938aff700 1 --2- 172.21.15.16:0/2340250737 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc93406a770 0x7fc93411d890 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fc9240090b0 tx=0x7fc924004750 comp rx=0 tx=0).stop 2023-09-30T22:34:47.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.299+0000 7fc938aff700 1 -- 172.21.15.16:0/2340250737 shutdown_connections 2023-09-30T22:34:47.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.299+0000 7fc938aff700 1 --2- 172.21.15.16:0/2340250737 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc93406a770 0x7fc93411d890 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:47.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.299+0000 7fc938aff700 1 --2- 172.21.15.16:0/2340250737 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc934131d10 0x7fc93411e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:47.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.299+0000 7fc938aff700 1 --2- 172.21.15.16:0/2340250737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc93406b080 0x7fc93411ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:47.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.299+0000 7fc938aff700 1 -- 172.21.15.16:0/2340250737 >> 172.21.15.16:0/2340250737 conn(0x7fc934066f50 msgr2=0x7fc9340a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:47.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.299+0000 7fc938aff700 1 -- 172.21.15.16:0/2340250737 shutdown_connections 2023-09-30T22:34:47.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.299+0000 7fc938aff700 1 -- 172.21.15.16:0/2340250737 wait complete. 2023-09-30T22:34:47.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.310+0000 7f6c27824700 1 -- 172.21.15.16:0/3125220530 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6c2011dd40 con 0x7f6c2006a770 2023-09-30T22:34:47.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.310+0000 7f6c15ffb700 1 -- 172.21.15.16:0/3125220530 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6c2011dd40 con 0x7f6c2006a770 2023-09-30T22:34:47.311 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:34:47.313 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:34:47.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.313+0000 7f6c27824700 1 -- 172.21.15.16:0/3125220530 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6c2006a770 msgr2=0x7f6c2011b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:47.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.313+0000 7f6c27824700 1 --2- 172.21.15.16:0/3125220530 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6c2006a770 0x7f6c2011b340 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c0090b0 tx=0x7f6c1c004750 comp rx=0 tx=0).stop 2023-09-30T22:34:47.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.313+0000 7f6c27824700 1 -- 172.21.15.16:0/3125220530 shutdown_connections 2023-09-30T22:34:47.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.313+0000 7f6c27824700 1 --2- 172.21.15.16:0/3125220530 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6c2006a770 0x7f6c2011b340 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:47.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.313+0000 7f6c27824700 1 --2- 172.21.15.16:0/3125220530 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6c2013c1e0 0x7f6c2013aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:47.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.313+0000 7f6c27824700 1 --2- 172.21.15.16:0/3125220530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c2009dc30 0x7f6c2013a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:47.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.313+0000 7f6c27824700 1 -- 172.21.15.16:0/3125220530 >> 172.21.15.16:0/3125220530 conn(0x7f6c20066f50 msgr2=0x7f6c200a1550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:47.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.313+0000 7f6c27824700 1 -- 172.21.15.16:0/3125220530 shutdown_connections 2023-09-30T22:34:47.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:47.313+0000 7f6c27824700 1 -- 172.21.15.16:0/3125220530 wait complete. 2023-09-30T22:34:47.327 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:34:48.044 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:34:48.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.188+0000 7f7b63f1e700 1 Processor -- start 2023-09-30T22:34:48.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.188+0000 7f7b63f1e700 1 -- start start 2023-09-30T22:34:48.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.189+0000 7f7b63f1e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7b5c09d6f0 0x7f7b5c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:48.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.189+0000 7f7b63f1e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7b5c09e000 0x7f7b5c130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:48.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.189+0000 7f7b63f1e700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b5c09f130 con 0x7f7b5c09e410 2023-09-30T22:34:48.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.189+0000 7f7b63f1e700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b5c09f520 con 0x7f7b5c09e000 2023-09-30T22:34:48.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.189+0000 7f7b63f1e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5c09e930 con 0x7f7b5c09d6f0 2023-09-30T22:34:48.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.189+0000 7f7b61cba700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7b5c09d6f0 0x7f7b5c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:48.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.189+0000 7f7b614b9700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7b5c09e000 0x7f7b5c130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:48.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.189+0000 7f7b624bb700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7b5c09e410 0x7f7b5c130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60812/0 (socket says 172.21.15.16:60812) 2023-09-30T22:34:48.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.189+0000 7f7b624bb700 1 -- 172.21.15.16:0/3492617903 learned_addr learned my addr 172.21.15.16:0/3492617903 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:34:48.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.189+0000 7f7b61cba700 1 -- 172.21.15.16:0/3492617903 >> v2:172.21.15.16:3300/0 conn(0x7f7b5c09e000 msgr2=0x7f7b5c130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:48.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.189+0000 7f7b61cba700 1 --2- 172.21.15.16:0/3492617903 >> v2:172.21.15.16:3300/0 conn(0x7f7b5c09e000 0x7f7b5c130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:48.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.189+0000 7f7b61cba700 1 -- 172.21.15.16:0/3492617903 >> v1:172.21.15.16:6789/0 conn(0x7f7b5c09e410 legacy=0x7f7b5c130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:48.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.189+0000 7f7b61cba700 1 -- 172.21.15.16:0/3492617903 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b5c09b2d0 con 0x7f7b5c09d6f0 2023-09-30T22:34:48.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.189+0000 7f7b614b9700 1 --2- 172.21.15.16:0/3492617903 >> v2:172.21.15.16:3300/0 conn(0x7f7b5c09e000 0x7f7b5c130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:48.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.190+0000 7f7b61cba700 1 --2- 172.21.15.16:0/3492617903 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7b5c09d6f0 0x7f7b5c09dac0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f7b58009840 tx=0x7f7b58009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bacb517809915fc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:48.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.190+0000 7f7b60cb8700 1 -- 172.21.15.16:0/3492617903 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b58011830 con 0x7f7b5c09d6f0 2023-09-30T22:34:48.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.190+0000 7f7b60cb8700 1 -- 172.21.15.16:0/3492617903 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b58011990 con 0x7f7b5c09d6f0 2023-09-30T22:34:48.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.190+0000 7f7b60cb8700 1 -- 172.21.15.16:0/3492617903 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b58011d20 con 0x7f7b5c09d6f0 2023-09-30T22:34:48.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.190+0000 7f7b63f1e700 1 -- 172.21.15.16:0/3492617903 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7b5c09d6f0 msgr2=0x7f7b5c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:48.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.190+0000 7f7b63f1e700 1 --2- 172.21.15.16:0/3492617903 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7b5c09d6f0 0x7f7b5c09dac0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f7b58009840 tx=0x7f7b58009b50 comp rx=0 tx=0).stop 2023-09-30T22:34:48.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.191+0000 7f7b63f1e700 1 -- 172.21.15.16:0/3492617903 shutdown_connections 2023-09-30T22:34:48.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.191+0000 7f7b63f1e700 1 --2- 172.21.15.16:0/3492617903 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7b5c09d6f0 0x7f7b5c09dac0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:48.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.191+0000 7f7b63f1e700 1 --2- 172.21.15.16:0/3492617903 >> v2:172.21.15.16:3300/0 conn(0x7f7b5c09e000 0x7f7b5c130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:48.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.191+0000 7f7b63f1e700 1 -- 172.21.15.16:0/3492617903 >> 172.21.15.16:0/3492617903 conn(0x7f7b5c066f50 msgr2=0x7f7b5c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:48.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.191+0000 7f7b63f1e700 1 -- 172.21.15.16:0/3492617903 shutdown_connections 2023-09-30T22:34:48.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.191+0000 7f7b63f1e700 1 -- 172.21.15.16:0/3492617903 wait complete. 2023-09-30T22:34:48.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.192+0000 7f7b63f1e700 1 Processor -- start 2023-09-30T22:34:48.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.192+0000 7f7b63f1e700 1 -- start start 2023-09-30T22:34:48.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.192+0000 7f7b63f1e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7b5c09d6f0 0x7f7b5c06e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:48.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.192+0000 7f7b63f1e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b5c09e000 0x7f7b5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:48.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.192+0000 7f7b63f1e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7b5c09e410 0x7f7b5c06f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:48.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.192+0000 7f7b63f1e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5c070330 con 0x7f7b5c09e000 2023-09-30T22:34:48.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.192+0000 7f7b63f1e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5c09c770 con 0x7f7b5c09d6f0 2023-09-30T22:34:48.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.192+0000 7f7b63f1e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5c12ff10 con 0x7f7b5c09e410 2023-09-30T22:34:48.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.192+0000 7f7b61cba700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7b5c09d6f0 0x7f7b5c06e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:48.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.192+0000 7f7b614b9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b5c09e000 0x7f7b5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:48.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.192+0000 7f7b61cba700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7b5c09d6f0 0x7f7b5c06e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:42018/0 (socket says 172.21.15.16:42018) 2023-09-30T22:34:48.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.192+0000 7f7b614b9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b5c09e000 0x7f7b5c06edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34348/0 (socket says 172.21.15.16:34348) 2023-09-30T22:34:48.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.192+0000 7f7b61cba700 1 -- 172.21.15.16:0/3508215932 learned_addr learned my addr 172.21.15.16:0/3508215932 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:48.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.192+0000 7f7b624bb700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7b5c09e410 0x7f7b5c06f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:48.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.193+0000 7f7b61cba700 1 -- 172.21.15.16:0/3508215932 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7b5c09e410 msgr2=0x7f7b5c06f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:48.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.193+0000 7f7b61cba700 1 --2- 172.21.15.16:0/3508215932 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7b5c09e410 0x7f7b5c06f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:48.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.193+0000 7f7b61cba700 1 -- 172.21.15.16:0/3508215932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b5c09e000 msgr2=0x7f7b5c06edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:48.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.193+0000 7f7b61cba700 1 --2- 172.21.15.16:0/3508215932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b5c09e000 0x7f7b5c06edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:48.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.193+0000 7f7b61cba700 1 -- 172.21.15.16:0/3508215932 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b500094a0 con 0x7f7b5c09d6f0 2023-09-30T22:34:48.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.193+0000 7f7b614b9700 1 --2- 172.21.15.16:0/3508215932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b5c09e000 0x7f7b5c06edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:48.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.193+0000 7f7b61cba700 1 --2- 172.21.15.16:0/3508215932 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7b5c09d6f0 0x7f7b5c06e870 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f7b58009fd0 tx=0x7f7b58004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:48.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.193+0000 7f7b4e7fc700 1 -- 172.21.15.16:0/3508215932 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b580036a0 con 0x7f7b5c09d6f0 2023-09-30T22:34:48.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.193+0000 7f7b4e7fc700 1 -- 172.21.15.16:0/3508215932 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b5801e070 con 0x7f7b5c09d6f0 2023-09-30T22:34:48.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.193+0000 7f7b4e7fc700 1 -- 172.21.15.16:0/3508215932 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b58003a40 con 0x7f7b5c09d6f0 2023-09-30T22:34:48.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.193+0000 7f7b63f1e700 1 -- 172.21.15.16:0/3508215932 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b580094a0 con 0x7f7b5c09d6f0 2023-09-30T22:34:48.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.193+0000 7f7b63f1e700 1 -- 172.21.15.16:0/3508215932 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b5c0a1f10 con 0x7f7b5c09d6f0 2023-09-30T22:34:48.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.194+0000 7f7b4e7fc700 1 -- 172.21.15.16:0/3508215932 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7b58003ba0 con 0x7f7b5c09d6f0 2023-09-30T22:34:48.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.194+0000 7f7b4e7fc700 1 -- 172.21.15.16:0/3508215932 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f7b58040620 con 0x7f7b5c09d6f0 2023-09-30T22:34:48.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.194+0000 7f7b63f1e700 1 -- 172.21.15.16:0/3508215932 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b5c074ea0 con 0x7f7b5c09d6f0 2023-09-30T22:34:48.199 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:34:48.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.199+0000 7f7b4e7fc700 1 -- 172.21.15.16:0/3508215932 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7b58015030 con 0x7f7b5c09d6f0 2023-09-30T22:34:48.314 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:34:48.328 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:34:48.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.336+0000 7fcccf53d700 1 Processor -- start 2023-09-30T22:34:48.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.337+0000 7fcccf53d700 1 -- start start 2023-09-30T22:34:48.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.337+0000 7fcccf53d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fccc809d6f0 0x7fccc809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:48.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.337+0000 7fcccf53d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fccc809e440 0x7fccc8134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:48.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.337+0000 7fcccf53d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccc809b120 con 0x7fccc809e000 2023-09-30T22:34:48.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.337+0000 7fcccf53d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccc806b460 con 0x7fccc809d6f0 2023-09-30T22:34:48.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.337+0000 7fcccf53d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccc809a7e0 con 0x7fccc809e440 2023-09-30T22:34:48.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.337+0000 7fccccad8700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fccc809e000 0x7fccc81339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60818/0 (socket says 172.21.15.16:60818) 2023-09-30T22:34:48.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.337+0000 7fccccad8700 1 -- 172.21.15.16:0/1103213189 learned_addr learned my addr 172.21.15.16:0/1103213189 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:34:48.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.337+0000 7fcccdada700 1 --2- 172.21.15.16:0/1103213189 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fccc809e440 0x7fccc8134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:48.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.338+0000 7fcccd2d9700 1 --2- 172.21.15.16:0/1103213189 >> v2:172.21.15.16:3300/0 conn(0x7fccc809d6f0 0x7fccc809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:48.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.338+0000 7fcccdada700 1 -- 172.21.15.16:0/1103213189 >> v2:172.21.15.16:3300/0 conn(0x7fccc809d6f0 msgr2=0x7fccc809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:48.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.338+0000 7fcccdada700 1 --2- 172.21.15.16:0/1103213189 >> v2:172.21.15.16:3300/0 conn(0x7fccc809d6f0 0x7fccc809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:48.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.338+0000 7fcccdada700 1 -- 172.21.15.16:0/1103213189 >> v1:172.21.15.16:6789/0 conn(0x7fccc809e000 legacy=0x7fccc81339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:48.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.338+0000 7fcccdada700 1 -- 172.21.15.16:0/1103213189 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc809a500 con 0x7fccc809e440 2023-09-30T22:34:48.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.338+0000 7fccbffff700 1 -- 172.21.15.16:0/1103213189 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3430267859 0 0) 0x7fccc809b120 con 0x7fccc809e000 2023-09-30T22:34:48.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.338+0000 7fcccdada700 1 --2- 172.21.15.16:0/1103213189 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fccc809e440 0x7fccc8134100 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fccc0009dc0 tx=0x7fccc000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2543088412ef722 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:48.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.338+0000 7fccbffff700 1 -- 172.21.15.16:0/1103213189 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc000e070 con 0x7fccc809e440 2023-09-30T22:34:48.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.338+0000 7fccbffff700 1 -- 172.21.15.16:0/1103213189 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccc000bb90 con 0x7fccc809e440 2023-09-30T22:34:48.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.338+0000 7fccbffff700 1 -- 172.21.15.16:0/1103213189 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc0012430 con 0x7fccc809e440 2023-09-30T22:34:48.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.339+0000 7fcccf53d700 1 -- 172.21.15.16:0/1103213189 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fccc809e440 msgr2=0x7fccc8134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:48.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.339+0000 7fcccf53d700 1 --2- 172.21.15.16:0/1103213189 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fccc809e440 0x7fccc8134100 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fccc0009dc0 tx=0x7fccc000b330 comp rx=0 tx=0).stop 2023-09-30T22:34:48.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.339+0000 7fcccf53d700 1 -- 172.21.15.16:0/1103213189 shutdown_connections 2023-09-30T22:34:48.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.339+0000 7fcccf53d700 1 --2- 172.21.15.16:0/1103213189 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fccc809e440 0x7fccc8134100 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:48.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.340+0000 7fcccf53d700 1 --2- 172.21.15.16:0/1103213189 >> v2:172.21.15.16:3300/0 conn(0x7fccc809d6f0 0x7fccc809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:48.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.340+0000 7fcccf53d700 1 -- 172.21.15.16:0/1103213189 >> 172.21.15.16:0/1103213189 conn(0x7fccc8066f50 msgr2=0x7fccc809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:48.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.340+0000 7fcccf53d700 1 -- 172.21.15.16:0/1103213189 shutdown_connections 2023-09-30T22:34:48.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.340+0000 7fcccf53d700 1 -- 172.21.15.16:0/1103213189 wait complete. 2023-09-30T22:34:48.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.341+0000 7fcccf53d700 1 Processor -- start 2023-09-30T22:34:48.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.341+0000 7fcccf53d700 1 -- start start 2023-09-30T22:34:48.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.342+0000 7fcccf53d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccc809d6f0 0x7fccc81a4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:48.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.342+0000 7fcccf53d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fccc809e000 0x7fccc81a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:48.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.342+0000 7fcccf53d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fccc809e440 0x7fccc81c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:48.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.342+0000 7fcccf53d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc81a52c0 con 0x7fccc809d6f0 2023-09-30T22:34:48.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.342+0000 7fcccf53d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccc81b8040 con 0x7fccc809e440 2023-09-30T22:34:48.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.342+0000 7fcccf53d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fccc809c710 con 0x7fccc809e000 2023-09-30T22:34:48.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.342+0000 7fcccdada700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fccc809e440 0x7fccc81c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:48.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.342+0000 7fcccd2d9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccc809d6f0 0x7fccc81a4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:48.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.342+0000 7fcccdada700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fccc809e440 0x7fccc81c8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:42036/0 (socket says 172.21.15.16:42036) 2023-09-30T22:34:48.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.342+0000 7fcccdada700 1 -- 172.21.15.16:0/3892306701 learned_addr learned my addr 172.21.15.16:0/3892306701 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:48.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.342+0000 7fcccd2d9700 1 --2- 172.21.15.16:0/3892306701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccc809d6f0 0x7fccc81a4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34378/0 (socket says 172.21.15.16:34378) 2023-09-30T22:34:48.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.343+0000 7fccccad8700 1 --2- 172.21.15.16:0/3892306701 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fccc809e000 0x7fccc81a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:48.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.343+0000 7fcccdada700 1 -- 172.21.15.16:0/3892306701 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fccc809e000 msgr2=0x7fccc81a46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:48.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.343+0000 7fcccdada700 1 --2- 172.21.15.16:0/3892306701 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fccc809e000 0x7fccc81a46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:48.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.343+0000 7fcccdada700 1 -- 172.21.15.16:0/3892306701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccc809d6f0 msgr2=0x7fccc81a4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:48.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.343+0000 7fcccdada700 1 --2- 172.21.15.16:0/3892306701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccc809d6f0 0x7fccc81a4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:48.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.343+0000 7fcccdada700 1 -- 172.21.15.16:0/3892306701 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc0009a70 con 0x7fccc809e440 2023-09-30T22:34:48.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.343+0000 7fcccdada700 1 --2- 172.21.15.16:0/3892306701 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fccc809e440 0x7fccc81c8090 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fccc809a920 tx=0x7fccc0004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:48.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.343+0000 7fccbdffb700 1 -- 172.21.15.16:0/3892306701 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc0012430 con 0x7fccc809e440 2023-09-30T22:34:48.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.343+0000 7fccbdffb700 1 -- 172.21.15.16:0/3892306701 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccc0020070 con 0x7fccc809e440 2023-09-30T22:34:48.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.343+0000 7fcccf53d700 1 -- 172.21.15.16:0/3892306701 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccc81a4be0 con 0x7fccc809e440 2023-09-30T22:34:48.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.343+0000 7fccbdffb700 1 -- 172.21.15.16:0/3892306701 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc0004280 con 0x7fccc809e440 2023-09-30T22:34:48.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.344+0000 7fcccf53d700 1 -- 172.21.15.16:0/3892306701 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccc81a6c80 con 0x7fccc809e440 2023-09-30T22:34:48.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.345+0000 7fcccf53d700 1 -- 172.21.15.16:0/3892306701 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccc80a2c80 con 0x7fccc809e440 2023-09-30T22:34:48.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.346+0000 7fccbdffb700 1 -- 172.21.15.16:0/3892306701 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fccc00043e0 con 0x7fccc809e440 2023-09-30T22:34:48.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.346+0000 7fccbdffb700 1 -- 172.21.15.16:0/3892306701 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fccc0022070 con 0x7fccc809e440 2023-09-30T22:34:48.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.354+0000 7fccbdffb700 1 -- 172.21.15.16:0/3892306701 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fccc00048d0 con 0x7fccc809e440 2023-09-30T22:34:48.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.415+0000 7f7b63f1e700 1 -- 172.21.15.16:0/3508215932 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7b5c124770 con 0x7f7b5c09d6f0 2023-09-30T22:34:48.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.415+0000 7f7b4e7fc700 1 -- 172.21.15.16:0/3508215932 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7b5802d7d0 con 0x7f7b5c09d6f0 2023-09-30T22:34:48.416 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:34:48.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.418+0000 7f7b63f1e700 1 -- 172.21.15.16:0/3508215932 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7b5c09d6f0 msgr2=0x7f7b5c06e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:48.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.418+0000 7f7b63f1e700 1 --2- 172.21.15.16:0/3508215932 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7b5c09d6f0 0x7f7b5c06e870 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f7b58009fd0 tx=0x7f7b58004750 comp rx=0 tx=0).stop 2023-09-30T22:34:48.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.418+0000 7f7b63f1e700 1 -- 172.21.15.16:0/3508215932 shutdown_connections 2023-09-30T22:34:48.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.418+0000 7f7b63f1e700 1 --2- 172.21.15.16:0/3508215932 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7b5c09d6f0 0x7f7b5c06e870 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:48.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.418+0000 7f7b63f1e700 1 --2- 172.21.15.16:0/3508215932 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7b5c09e410 0x7f7b5c06f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:48.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.418+0000 7f7b63f1e700 1 --2- 172.21.15.16:0/3508215932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b5c09e000 0x7f7b5c06edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:48.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.418+0000 7f7b63f1e700 1 -- 172.21.15.16:0/3508215932 >> 172.21.15.16:0/3508215932 conn(0x7f7b5c066f50 msgr2=0x7f7b5c06a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:48.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.418+0000 7f7b63f1e700 1 -- 172.21.15.16:0/3508215932 shutdown_connections 2023-09-30T22:34:48.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.418+0000 7f7b63f1e700 1 -- 172.21.15.16:0/3508215932 wait complete. 2023-09-30T22:34:48.433 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:34:48.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.459+0000 7f347ba22700 1 Processor -- start 2023-09-30T22:34:48.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.459+0000 7f347ba22700 1 -- start start 2023-09-30T22:34:48.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.460+0000 7f347ba22700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f34741338a0 0x7f3474133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:48.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.460+0000 7f347ba22700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f347409de50 0x7f347409e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:48.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.460+0000 7f347ba22700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f347406ef30 con 0x7f347409e760 2023-09-30T22:34:48.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.460+0000 7f347ba22700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f347406fef0 con 0x7f34741338a0 2023-09-30T22:34:48.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.460+0000 7f347ba22700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34741334b0 con 0x7f347409de50 2023-09-30T22:34:48.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.460+0000 7f34797be700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f34741338a0 0x7f3474133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:48.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.460+0000 7f34797be700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f34741338a0 0x7f3474133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34390/0 (socket says 172.21.15.16:34390) 2023-09-30T22:34:48.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.460+0000 7f3479fbf700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f347409e760 0x7f347413a730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60824/0 (socket says 172.21.15.16:60824) 2023-09-30T22:34:48.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.460+0000 7f34797be700 1 -- 172.21.15.16:0/2492478396 learned_addr learned my addr 172.21.15.16:0/2492478396 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:48.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.460+0000 7f3478fbd700 1 --2- 172.21.15.16:0/2492478396 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f347409de50 0x7f347409e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:48.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.460+0000 7f34797be700 1 -- 172.21.15.16:0/2492478396 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f347409de50 msgr2=0x7f347409e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:48.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.460+0000 7f34797be700 1 --2- 172.21.15.16:0/2492478396 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f347409de50 0x7f347409e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:48.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.460+0000 7f34797be700 1 -- 172.21.15.16:0/2492478396 >> v1:172.21.15.16:6789/0 conn(0x7f347409e760 legacy=0x7f347413a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:48.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.460+0000 7f34797be700 1 -- 172.21.15.16:0/2492478396 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f347406fc30 con 0x7f34741338a0 2023-09-30T22:34:48.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.461+0000 7f34797be700 1 --2- 172.21.15.16:0/2492478396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34741338a0 0x7f3474133c70 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f34700098f0 tx=0x7f3470009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f73292db55c895a6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:48.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.461+0000 7f346bfff700 1 -- 172.21.15.16:0/2492478396 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3470011830 con 0x7f34741338a0 2023-09-30T22:34:48.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.461+0000 7f346bfff700 1 -- 172.21.15.16:0/2492478396 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3470011990 con 0x7f34741338a0 2023-09-30T22:34:48.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.461+0000 7f346bfff700 1 -- 172.21.15.16:0/2492478396 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3470011d20 con 0x7f34741338a0 2023-09-30T22:34:48.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.461+0000 7f347ba22700 1 -- 172.21.15.16:0/2492478396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34741338a0 msgr2=0x7f3474133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:48.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.461+0000 7f347ba22700 1 --2- 172.21.15.16:0/2492478396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34741338a0 0x7f3474133c70 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f34700098f0 tx=0x7f3470009c00 comp rx=0 tx=0).stop 2023-09-30T22:34:48.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.461+0000 7f347ba22700 1 -- 172.21.15.16:0/2492478396 shutdown_connections 2023-09-30T22:34:48.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.462+0000 7f347ba22700 1 --2- 172.21.15.16:0/2492478396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34741338a0 0x7f3474133c70 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:48.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.462+0000 7f347ba22700 1 --2- 172.21.15.16:0/2492478396 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f347409de50 0x7f347409e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:48.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.462+0000 7f347ba22700 1 -- 172.21.15.16:0/2492478396 >> 172.21.15.16:0/2492478396 conn(0x7f34740a2aa0 msgr2=0x7f34740a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:48.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.462+0000 7f347ba22700 1 -- 172.21.15.16:0/2492478396 shutdown_connections 2023-09-30T22:34:48.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.462+0000 7f347ba22700 1 -- 172.21.15.16:0/2492478396 wait complete. 2023-09-30T22:34:48.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.463+0000 7f347ba22700 1 Processor -- start 2023-09-30T22:34:48.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.463+0000 7f347ba22700 1 -- start start 2023-09-30T22:34:48.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.463+0000 7f347ba22700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f347409de50 0x7f347411eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:48.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.463+0000 7f347ba22700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f347409e760 0x7f347411f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:48.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.463+0000 7f347ba22700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f34741338a0 0x7f347411f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:48.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.463+0000 7f347ba22700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f347406f640 con 0x7f347409e760 2023-09-30T22:34:48.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.463+0000 7f347ba22700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34741222b0 con 0x7f347409de50 2023-09-30T22:34:48.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.463+0000 7f347ba22700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3474120790 con 0x7f34741338a0 2023-09-30T22:34:48.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.463+0000 7f3478fbd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f347409e760 0x7f347411f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:48.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.463+0000 7f3478fbd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f347409e760 0x7f347411f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34406/0 (socket says 172.21.15.16:34406) 2023-09-30T22:34:48.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.463+0000 7f3478fbd700 1 -- 172.21.15.16:0/299917112 learned_addr learned my addr 172.21.15.16:0/299917112 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.463+0000 7f3479fbf700 1 --2- 172.21.15.16:0/299917112 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f34741338a0 0x7f347411f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.464+0000 7f34797be700 1 --2- 172.21.15.16:0/299917112 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f347409de50 0x7f347411eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.464+0000 7f3478fbd700 1 -- 172.21.15.16:0/299917112 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f34741338a0 msgr2=0x7f347411f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.464+0000 7f3478fbd700 1 --2- 172.21.15.16:0/299917112 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f34741338a0 0x7f347411f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.464+0000 7f3478fbd700 1 -- 172.21.15.16:0/299917112 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f347409de50 msgr2=0x7f347411eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.464+0000 7f3478fbd700 1 --2- 172.21.15.16:0/299917112 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f347409de50 0x7f347411eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.464+0000 7f3478fbd700 1 -- 172.21.15.16:0/299917112 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3470009550 con 0x7f347409e760 2023-09-30T22:34:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.464+0000 7f34797be700 1 --2- 172.21.15.16:0/299917112 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f347409de50 0x7f347411eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:34:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.464+0000 7f3478fbd700 1 --2- 172.21.15.16:0/299917112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f347409e760 0x7f347411f040 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f346400e870 tx=0x7f346400eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.464+0000 7f3469ffb700 1 -- 172.21.15.16:0/299917112 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3464009590 con 0x7f347409e760 2023-09-30T22:34:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.464+0000 7f3469ffb700 1 -- 172.21.15.16:0/299917112 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3464013680 con 0x7f347409e760 2023-09-30T22:34:48.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.464+0000 7f347ba22700 1 -- 172.21.15.16:0/299917112 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34740a1020 con 0x7f347409e760 2023-09-30T22:34:48.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.464+0000 7f3469ffb700 1 -- 172.21.15.16:0/299917112 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34640139f0 con 0x7f347409e760 2023-09-30T22:34:48.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.464+0000 7f347ba22700 1 -- 172.21.15.16:0/299917112 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3474122d20 con 0x7f347409e760 2023-09-30T22:34:48.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.465+0000 7f3469ffb700 1 -- 172.21.15.16:0/299917112 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3464013b50 con 0x7f347409e760 2023-09-30T22:34:48.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.465+0000 7f347ba22700 1 -- 172.21.15.16:0/299917112 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3474134130 con 0x7f347409e760 2023-09-30T22:34:48.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.466+0000 7f3469ffb700 1 -- 172.21.15.16:0/299917112 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f346403f9b0 con 0x7f347409e760 2023-09-30T22:34:48.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.471+0000 7f3469ffb700 1 -- 172.21.15.16:0/299917112 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f346400c7d0 con 0x7f347409e760 2023-09-30T22:34:48.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.483+0000 7fe406075700 1 Processor -- start 2023-09-30T22:34:48.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.483+0000 7fe406075700 1 -- start start 2023-09-30T22:34:48.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.483+0000 7fe406075700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe4000a0a30 0x7fe4000a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:48.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.483+0000 7fe406075700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe400134170 0x7fe40013bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:48.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.483+0000 7fe406075700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe40009edd0 con 0x7fe4000a1340 2023-09-30T22:34:48.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.483+0000 7fe406075700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe40009e1f0 con 0x7fe4000a0a30 2023-09-30T22:34:48.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.483+0000 7fe406075700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe40009dc50 con 0x7fe400134170 2023-09-30T22:34:48.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.483+0000 7fe3ff7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe4000a0a30 0x7fe4000a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:48.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.484+0000 7fe3ff7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe4000a0a30 0x7fe4000a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34408/0 (socket says 172.21.15.16:34408) 2023-09-30T22:34:48.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.484+0000 7fe3ff7fe700 1 -- 172.21.15.16:0/2922560452 learned_addr learned my addr 172.21.15.16:0/2922560452 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:48.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.484+0000 7fe3fffff700 1 --2- 172.21.15.16:0/2922560452 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe400134170 0x7fe40013bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:48.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.484+0000 7fe3ff7fe700 1 -- 172.21.15.16:0/2922560452 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe400134170 msgr2=0x7fe40013bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:48.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.484+0000 7fe3ff7fe700 1 --2- 172.21.15.16:0/2922560452 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe400134170 0x7fe40013bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:48.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.484+0000 7fe3ff7fe700 1 -- 172.21.15.16:0/2922560452 >> v1:172.21.15.16:6789/0 conn(0x7fe4000a1340 legacy=0x7fe400133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:48.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.484+0000 7fe3ff7fe700 1 -- 172.21.15.16:0/2922560452 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4000a1fe0 con 0x7fe4000a0a30 2023-09-30T22:34:48.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.484+0000 7fe3feffd700 1 -- 172.21.15.16:0/2922560452 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 604366737 0 0) 0x7fe40009edd0 con 0x7fe4000a1340 2023-09-30T22:34:48.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.484+0000 7fe3ff7fe700 1 --2- 172.21.15.16:0/2922560452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4000a0a30 0x7fe4000a0e00 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fe3f00098f0 tx=0x7fe3f0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af1d645428d8bd18 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:48.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.484+0000 7fe3feffd700 1 -- 172.21.15.16:0/2922560452 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f0011830 con 0x7fe4000a0a30 2023-09-30T22:34:48.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.484+0000 7fe3feffd700 1 -- 172.21.15.16:0/2922560452 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f0011990 con 0x7fe4000a0a30 2023-09-30T22:34:48.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.484+0000 7fe3feffd700 1 -- 172.21.15.16:0/2922560452 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f0011d20 con 0x7fe4000a0a30 2023-09-30T22:34:48.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.485+0000 7fe406075700 1 -- 172.21.15.16:0/2922560452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4000a0a30 msgr2=0x7fe4000a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:48.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.485+0000 7fe406075700 1 --2- 172.21.15.16:0/2922560452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4000a0a30 0x7fe4000a0e00 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fe3f00098f0 tx=0x7fe3f0009c00 comp rx=0 tx=0).stop 2023-09-30T22:34:48.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.485+0000 7fe406075700 1 -- 172.21.15.16:0/2922560452 shutdown_connections 2023-09-30T22:34:48.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.485+0000 7fe406075700 1 --2- 172.21.15.16:0/2922560452 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe400134170 0x7fe40013bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:48.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.485+0000 7fe406075700 1 --2- 172.21.15.16:0/2922560452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4000a0a30 0x7fe4000a0e00 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:48.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.485+0000 7fe406075700 1 -- 172.21.15.16:0/2922560452 >> 172.21.15.16:0/2922560452 conn(0x7fe4000a6550 msgr2=0x7fe4000a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:48.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.485+0000 7fe406075700 1 -- 172.21.15.16:0/2922560452 shutdown_connections 2023-09-30T22:34:48.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.485+0000 7fe406075700 1 -- 172.21.15.16:0/2922560452 wait complete. 2023-09-30T22:34:48.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.486+0000 7fe406075700 1 Processor -- start 2023-09-30T22:34:48.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.486+0000 7fe406075700 1 -- start start 2023-09-30T22:34:48.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.487+0000 7fe406075700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4000a0a30 0x7fe400122870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:48.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.487+0000 7fe406075700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe4000a1340 0x7fe400122db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:48.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.487+0000 7fe406075700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe400134170 0x7fe4001232f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:48.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.487+0000 7fe406075700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe400125fd0 con 0x7fe4000a0a30 2023-09-30T22:34:48.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.487+0000 7fe406075700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4001244b0 con 0x7fe400134170 2023-09-30T22:34:48.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.487+0000 7fe406075700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe4000a4e10 con 0x7fe4000a1340 2023-09-30T22:34:48.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.487+0000 7fe3ff7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4000a0a30 0x7fe400122870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:48.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.487+0000 7fe3ff7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4000a0a30 0x7fe400122870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34418/0 (socket says 172.21.15.16:34418) 2023-09-30T22:34:48.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.487+0000 7fe3ff7fe700 1 -- 172.21.15.16:0/1885999145 learned_addr learned my addr 172.21.15.16:0/1885999145 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:48.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.487+0000 7fe3fffff700 1 --2- 172.21.15.16:0/1885999145 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe400134170 0x7fe4001232f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:48.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.487+0000 7fe3f6ffd700 1 --2- 172.21.15.16:0/1885999145 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe4000a1340 0x7fe400122db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:48.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.487+0000 7fe3ff7fe700 1 -- 172.21.15.16:0/1885999145 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe4000a1340 msgr2=0x7fe400122db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:48.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.487+0000 7fe3ff7fe700 1 --2- 172.21.15.16:0/1885999145 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe4000a1340 0x7fe400122db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:48.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.487+0000 7fe3ff7fe700 1 -- 172.21.15.16:0/1885999145 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe400134170 msgr2=0x7fe4001232f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:48.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.487+0000 7fe3ff7fe700 1 --2- 172.21.15.16:0/1885999145 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe400134170 0x7fe4001232f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:48.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.487+0000 7fe3ff7fe700 1 -- 172.21.15.16:0/1885999145 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f0009550 con 0x7fe4000a0a30 2023-09-30T22:34:48.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.487+0000 7fe3fffff700 1 --2- 172.21.15.16:0/1885999145 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe400134170 0x7fe4001232f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:48.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.488+0000 7fe3ff7fe700 1 --2- 172.21.15.16:0/1885999145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4000a0a30 0x7fe400122870 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0009c40 tx=0x7fe3f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:48.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.488+0000 7fe3fcff9700 1 -- 172.21.15.16:0/1885999145 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f0004550 con 0x7fe4000a0a30 2023-09-30T22:34:48.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.488+0000 7fe3fcff9700 1 -- 172.21.15.16:0/1885999145 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f00036a0 con 0x7fe4000a0a30 2023-09-30T22:34:48.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.488+0000 7fe406075700 1 -- 172.21.15.16:0/1885999145 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4000a25e0 con 0x7fe4000a0a30 2023-09-30T22:34:48.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.488+0000 7fe3fcff9700 1 -- 172.21.15.16:0/1885999145 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f0003a30 con 0x7fe4000a0a30 2023-09-30T22:34:48.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.488+0000 7fe406075700 1 -- 172.21.15.16:0/1885999145 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4000a0280 con 0x7fe4000a0a30 2023-09-30T22:34:48.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.489+0000 7fe3fcff9700 1 -- 172.21.15.16:0/1885999145 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe3f0021070 con 0x7fe4000a0a30 2023-09-30T22:34:48.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.489+0000 7fe3fcff9700 1 -- 172.21.15.16:0/1885999145 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fe3f0040c90 con 0x7fe4000a0a30 2023-09-30T22:34:48.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.489+0000 7fe3f5ffb700 1 -- 172.21.15.16:0/1885999145 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3d80052e0 con 0x7fe4000a0a30 2023-09-30T22:34:48.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.494+0000 7fe3fcff9700 1 -- 172.21.15.16:0/1885999145 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe3f0029080 con 0x7fe4000a0a30 2023-09-30T22:34:48.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.569+0000 7fcccf53d700 1 -- 172.21.15.16:0/3892306701 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fccc809f420 con 0x7fccc809e440 2023-09-30T22:34:48.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.569+0000 7fccbdffb700 1 -- 172.21.15.16:0/3892306701 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fccc002c020 con 0x7fccc809e440 2023-09-30T22:34:48.570 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:34:48.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.572+0000 7fcccf53d700 1 -- 172.21.15.16:0/3892306701 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fccc809e440 msgr2=0x7fccc81c8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:48.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.572+0000 7fcccf53d700 1 --2- 172.21.15.16:0/3892306701 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fccc809e440 0x7fccc81c8090 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fccc809a920 tx=0x7fccc0004ad0 comp rx=0 tx=0).stop 2023-09-30T22:34:48.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.573+0000 7fcccf53d700 1 -- 172.21.15.16:0/3892306701 shutdown_connections 2023-09-30T22:34:48.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.573+0000 7fcccf53d700 1 --2- 172.21.15.16:0/3892306701 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fccc809e440 0x7fccc81c8090 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:48.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.573+0000 7fcccf53d700 1 --2- 172.21.15.16:0/3892306701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fccc809d6f0 0x7fccc81a4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:48.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.573+0000 7fcccf53d700 1 --2- 172.21.15.16:0/3892306701 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fccc809e000 0x7fccc81a46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:48.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.573+0000 7fcccf53d700 1 -- 172.21.15.16:0/3892306701 >> 172.21.15.16:0/3892306701 conn(0x7fccc8066f50 msgr2=0x7fccc80a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:48.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.573+0000 7fcccf53d700 1 -- 172.21.15.16:0/3892306701 shutdown_connections 2023-09-30T22:34:48.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.573+0000 7fcccf53d700 1 -- 172.21.15.16:0/3892306701 wait complete. 2023-09-30T22:34:48.587 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:34:48.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.679+0000 7f347ba22700 1 -- 172.21.15.16:0/299917112 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f3474074ea0 con 0x7f347409e760 2023-09-30T22:34:48.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.679+0000 7f3469ffb700 1 -- 172.21.15.16:0/299917112 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3464017070 con 0x7f347409e760 2023-09-30T22:34:48.680 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:34:48.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.681+0000 7f347ba22700 1 -- 172.21.15.16:0/299917112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f347409e760 msgr2=0x7f347411f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:48.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.681+0000 7f347ba22700 1 --2- 172.21.15.16:0/299917112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f347409e760 0x7f347411f040 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f346400e870 tx=0x7f346400eb80 comp rx=0 tx=0).stop 2023-09-30T22:34:48.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.682+0000 7f347ba22700 1 -- 172.21.15.16:0/299917112 shutdown_connections 2023-09-30T22:34:48.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.682+0000 7f347ba22700 1 --2- 172.21.15.16:0/299917112 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f347409de50 0x7f347411eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:48.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.682+0000 7f347ba22700 1 --2- 172.21.15.16:0/299917112 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f34741338a0 0x7f347411f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:48.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.682+0000 7f347ba22700 1 --2- 172.21.15.16:0/299917112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f347409e760 0x7f347411f040 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:48.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.682+0000 7f347ba22700 1 -- 172.21.15.16:0/299917112 >> 172.21.15.16:0/299917112 conn(0x7f34740a2aa0 msgr2=0x7f347409ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:48.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.682+0000 7f347ba22700 1 -- 172.21.15.16:0/299917112 shutdown_connections 2023-09-30T22:34:48.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.682+0000 7f347ba22700 1 -- 172.21.15.16:0/299917112 wait complete. 2023-09-30T22:34:48.696 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:34:48.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.704+0000 7fe3f5ffb700 1 -- 172.21.15.16:0/1885999145 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe3d8005590 con 0x7fe4000a0a30 2023-09-30T22:34:48.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.704+0000 7fe3fcff9700 1 -- 172.21.15.16:0/1885999145 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe3f0015020 con 0x7fe4000a0a30 2023-09-30T22:34:48.705 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:34:48.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.707+0000 7fe3f5ffb700 1 -- 172.21.15.16:0/1885999145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4000a0a30 msgr2=0x7fe400122870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:48.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.707+0000 7fe3f5ffb700 1 --2- 172.21.15.16:0/1885999145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4000a0a30 0x7fe400122870 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0009c40 tx=0x7fe3f0004750 comp rx=0 tx=0).stop 2023-09-30T22:34:48.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.707+0000 7fe3f5ffb700 1 -- 172.21.15.16:0/1885999145 shutdown_connections 2023-09-30T22:34:48.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.707+0000 7fe3f5ffb700 1 --2- 172.21.15.16:0/1885999145 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe400134170 0x7fe4001232f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:48.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.707+0000 7fe3f5ffb700 1 --2- 172.21.15.16:0/1885999145 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4000a0a30 0x7fe400122870 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:48.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.707+0000 7fe3f5ffb700 1 --2- 172.21.15.16:0/1885999145 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe4000a1340 0x7fe400122db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:48.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.707+0000 7fe3f5ffb700 1 -- 172.21.15.16:0/1885999145 >> 172.21.15.16:0/1885999145 conn(0x7fe4000a6550 msgr2=0x7fe40009edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:48.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.707+0000 7fe3f5ffb700 1 -- 172.21.15.16:0/1885999145 shutdown_connections 2023-09-30T22:34:48.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:48.707+0000 7fe3f5ffb700 1 -- 172.21.15.16:0/1885999145 wait complete. 2023-09-30T22:34:48.721 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:34:49.434 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:34:49.588 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:34:49.770 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:34:49.771 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:34:49.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.604+0000 7f2e2e2c3700 1 Processor -- start 2023-09-30T22:34:49.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.604+0000 7f2e2e2c3700 1 -- start start 2023-09-30T22:34:49.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.605+0000 7f2e2e2c3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2e2809d6f0 0x7f2e2809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:49.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.605+0000 7f2e2e2c3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2e2809e410 0x7f2e28134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:49.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.605+0000 7f2e2e2c3700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e2809b120 con 0x7f2e2809e000 2023-09-30T22:34:49.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.605+0000 7f2e2e2c3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2806b460 con 0x7f2e2809e410 2023-09-30T22:34:49.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.605+0000 7f2e2e2c3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2809a7e0 con 0x7f2e2809d6f0 2023-09-30T22:34:49.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.605+0000 7f2e2c860700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2e2809e410 0x7f2e28134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:49.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.605+0000 7f2e277fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2e2809e000 0x7f2e281339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60854/0 (socket says 172.21.15.16:60854) 2023-09-30T22:34:49.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.605+0000 7f2e277fe700 1 -- 172.21.15.16:0/2617954573 learned_addr learned my addr 172.21.15.16:0/2617954573 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:34:49.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.605+0000 7f2e27fff700 1 --2- 172.21.15.16:0/2617954573 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2e2809d6f0 0x7f2e2809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:49.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.605+0000 7f2e2c860700 1 -- 172.21.15.16:0/2617954573 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2e2809d6f0 msgr2=0x7f2e2809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:49.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.605+0000 7f2e2c860700 1 --2- 172.21.15.16:0/2617954573 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2e2809d6f0 0x7f2e2809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:49.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.605+0000 7f2e26ffd700 1 -- 172.21.15.16:0/2617954573 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2609236691 0 0) 0x7f2e2809b120 con 0x7f2e2809e000 2023-09-30T22:34:49.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.605+0000 7f2e2c860700 1 -- 172.21.15.16:0/2617954573 >> v1:172.21.15.16:6789/0 conn(0x7f2e2809e000 legacy=0x7f2e281339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:49.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.605+0000 7f2e2c860700 1 -- 172.21.15.16:0/2617954573 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e2809a500 con 0x7f2e2809e410 2023-09-30T22:34:49.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.605+0000 7f2e2c860700 1 --2- 172.21.15.16:0/2617954573 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2e2809e410 0x7f2e28134100 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f2e20009dc0 tx=0x7f2e2000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b436eb4c326b811 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:49.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.605+0000 7f2e26ffd700 1 -- 172.21.15.16:0/2617954573 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e2000e070 con 0x7f2e2809e410 2023-09-30T22:34:49.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.605+0000 7f2e26ffd700 1 -- 172.21.15.16:0/2617954573 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e2000bb20 con 0x7f2e2809e410 2023-09-30T22:34:49.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.606+0000 7f2e26ffd700 1 -- 172.21.15.16:0/2617954573 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e20012430 con 0x7f2e2809e410 2023-09-30T22:34:49.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.606+0000 7f2e2e2c3700 1 -- 172.21.15.16:0/2617954573 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2e2809e410 msgr2=0x7f2e28134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:49.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.606+0000 7f2e2e2c3700 1 --2- 172.21.15.16:0/2617954573 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2e2809e410 0x7f2e28134100 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f2e20009dc0 tx=0x7f2e2000b330 comp rx=0 tx=0).stop 2023-09-30T22:34:49.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.606+0000 7f2e2e2c3700 1 -- 172.21.15.16:0/2617954573 shutdown_connections 2023-09-30T22:34:49.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.606+0000 7f2e2e2c3700 1 --2- 172.21.15.16:0/2617954573 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2e2809e410 0x7f2e28134100 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:49.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.606+0000 7f2e2e2c3700 1 --2- 172.21.15.16:0/2617954573 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2e2809d6f0 0x7f2e2809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:49.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.606+0000 7f2e2e2c3700 1 -- 172.21.15.16:0/2617954573 >> 172.21.15.16:0/2617954573 conn(0x7f2e28066f50 msgr2=0x7f2e2809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:49.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.606+0000 7f2e2e2c3700 1 -- 172.21.15.16:0/2617954573 shutdown_connections 2023-09-30T22:34:49.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.606+0000 7f2e2e2c3700 1 -- 172.21.15.16:0/2617954573 wait complete. 2023-09-30T22:34:49.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.607+0000 7f2e2e2c3700 1 Processor -- start 2023-09-30T22:34:49.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.607+0000 7f2e2e2c3700 1 -- start start 2023-09-30T22:34:49.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.608+0000 7f2e2e2c3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2e2809d6f0 0x7f2e281c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:49.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.608+0000 7f2e2e2c3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e2809e000 0x7f2e281c8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:49.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.608+0000 7f2e2e2c3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2e2809e410 0x7f2e281c89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:49.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.608+0000 7f2e2e2c3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e281a6f20 con 0x7f2e2809e000 2023-09-30T22:34:49.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.608+0000 7f2e2e2c3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e280a1d00 con 0x7f2e2809d6f0 2023-09-30T22:34:49.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.608+0000 7f2e2e2c3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2e281a6350 con 0x7f2e2809e410 2023-09-30T22:34:49.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.608+0000 7f2e27fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2e2809d6f0 0x7f2e281c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:49.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.608+0000 7f2e27fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2e2809d6f0 0x7f2e281c7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:42094/0 (socket says 172.21.15.16:42094) 2023-09-30T22:34:49.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.608+0000 7f2e277fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e2809e000 0x7f2e281c8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:49.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.608+0000 7f2e27fff700 1 -- 172.21.15.16:0/669944241 learned_addr learned my addr 172.21.15.16:0/669944241 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:49.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.608+0000 7f2e2c860700 1 --2- 172.21.15.16:0/669944241 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2e2809e410 0x7f2e281c89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:49.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.608+0000 7f2e27fff700 1 -- 172.21.15.16:0/669944241 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2e2809e410 msgr2=0x7f2e281c89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:49.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.608+0000 7f2e27fff700 1 --2- 172.21.15.16:0/669944241 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2e2809e410 0x7f2e281c89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:49.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.608+0000 7f2e27fff700 1 -- 172.21.15.16:0/669944241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e2809e000 msgr2=0x7f2e281c8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:49.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.608+0000 7f2e27fff700 1 --2- 172.21.15.16:0/669944241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e2809e000 0x7f2e281c8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:49.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.608+0000 7f2e27fff700 1 -- 172.21.15.16:0/669944241 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e20009a70 con 0x7f2e2809d6f0 2023-09-30T22:34:49.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.608+0000 7f2e277fe700 1 --2- 172.21.15.16:0/669944241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e2809e000 0x7f2e281c8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:49.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.608+0000 7f2e27fff700 1 --2- 172.21.15.16:0/669944241 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2e2809d6f0 0x7f2e281c7f40 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f2e1800e9b0 tx=0x7f2e1800ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:49.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.609+0000 7f2e24ff9700 1 -- 172.21.15.16:0/669944241 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e18009710 con 0x7f2e2809d6f0 2023-09-30T22:34:49.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.609+0000 7f2e24ff9700 1 -- 172.21.15.16:0/669944241 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e1800ee50 con 0x7f2e2809d6f0 2023-09-30T22:34:49.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.609+0000 7f2e2e2c3700 1 -- 172.21.15.16:0/669944241 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e281a7e70 con 0x7f2e2809d6f0 2023-09-30T22:34:49.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.609+0000 7f2e24ff9700 1 -- 172.21.15.16:0/669944241 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1800ca50 con 0x7f2e2809d6f0 2023-09-30T22:34:49.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.609+0000 7f2e2e2c3700 1 -- 172.21.15.16:0/669944241 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e281b90e0 con 0x7f2e2809d6f0 2023-09-30T22:34:49.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.610+0000 7f2e2e2c3700 1 -- 172.21.15.16:0/669944241 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e28074ea0 con 0x7f2e2809d6f0 2023-09-30T22:34:49.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.610+0000 7f2e24ff9700 1 -- 172.21.15.16:0/669944241 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2e1800cbb0 con 0x7f2e2809d6f0 2023-09-30T22:34:49.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.610+0000 7f2e24ff9700 1 -- 172.21.15.16:0/669944241 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f2e18040070 con 0x7f2e2809d6f0 2023-09-30T22:34:49.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.615+0000 7f2e24ff9700 1 -- 172.21.15.16:0/669944241 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2e1801a360 con 0x7f2e2809d6f0 2023-09-30T22:34:49.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.831+0000 7f2e2e2c3700 1 -- 172.21.15.16:0/669944241 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f2e2806c360 con 0x7f2e2809d6f0 2023-09-30T22:34:49.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.832+0000 7f2e24ff9700 1 -- 172.21.15.16:0/669944241 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2e18017020 con 0x7f2e2809d6f0 2023-09-30T22:34:49.832 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:34:49.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.834+0000 7f2e2e2c3700 1 -- 172.21.15.16:0/669944241 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2e2809d6f0 msgr2=0x7f2e281c7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:49.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.834+0000 7f2e2e2c3700 1 --2- 172.21.15.16:0/669944241 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2e2809d6f0 0x7f2e281c7f40 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f2e1800e9b0 tx=0x7f2e1800ecc0 comp rx=0 tx=0).stop 2023-09-30T22:34:49.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.834+0000 7f2e2e2c3700 1 -- 172.21.15.16:0/669944241 shutdown_connections 2023-09-30T22:34:49.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.834+0000 7f2e2e2c3700 1 --2- 172.21.15.16:0/669944241 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2e2809d6f0 0x7f2e281c7f40 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:49.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.834+0000 7f2e2e2c3700 1 --2- 172.21.15.16:0/669944241 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2e2809e410 0x7f2e281c89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:49.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.834+0000 7f2e2e2c3700 1 --2- 172.21.15.16:0/669944241 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2e2809e000 0x7f2e281c8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:49.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.834+0000 7f2e2e2c3700 1 -- 172.21.15.16:0/669944241 >> 172.21.15.16:0/669944241 conn(0x7f2e28066f50 msgr2=0x7f2e281bbb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:49.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.834+0000 7f2e2e2c3700 1 -- 172.21.15.16:0/669944241 shutdown_connections 2023-09-30T22:34:49.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.834+0000 7f2e2e2c3700 1 -- 172.21.15.16:0/669944241 wait complete. 2023-09-30T22:34:49.850 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:34:49.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.926+0000 7f3ddb2be700 1 Processor -- start 2023-09-30T22:34:49.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.926+0000 7f3ddb2be700 1 -- start start 2023-09-30T22:34:49.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.927+0000 7f3ddb2be700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3dd4130540 0x7f3dd4130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:49.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.927+0000 7f3ddb2be700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3dd4130e50 0x7f3dd406f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:49.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.927+0000 7f3ddb2be700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dd40716a0 con 0x7f3dd406fb20 2023-09-30T22:34:49.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.927+0000 7f3ddb2be700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dd409a390 con 0x7f3dd4130540 2023-09-30T22:34:49.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.927+0000 7f3ddb2be700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd4071290 con 0x7f3dd4130e50 2023-09-30T22:34:49.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.927+0000 7f3dd905a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3dd4130540 0x7f3dd4130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:49.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.927+0000 7f3dd905a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3dd4130540 0x7f3dd4130910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34428/0 (socket says 172.21.15.16:34428) 2023-09-30T22:34:49.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.927+0000 7f3dd8859700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3dd4130e50 0x7f3dd406f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:49.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.927+0000 7f3dd905a700 1 -- 172.21.15.16:0/1961074382 learned_addr learned my addr 172.21.15.16:0/1961074382 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:49.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.928+0000 7f3dd905a700 1 -- 172.21.15.16:0/1961074382 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3dd4130e50 msgr2=0x7f3dd406f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:49.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.928+0000 7f3dd905a700 1 --2- 172.21.15.16:0/1961074382 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3dd4130e50 0x7f3dd406f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:49.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.928+0000 7f3dd905a700 1 -- 172.21.15.16:0/1961074382 >> v1:172.21.15.16:6789/0 conn(0x7f3dd406fb20 legacy=0x7f3dd4137560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:49.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.928+0000 7f3dd905a700 1 -- 172.21.15.16:0/1961074382 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd4130340 con 0x7f3dd4130540 2023-09-30T22:34:49.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.928+0000 7f3dd8859700 1 --2- 172.21.15.16:0/1961074382 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3dd4130e50 0x7f3dd406f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:49.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.928+0000 7f3dd905a700 1 --2- 172.21.15.16:0/1961074382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dd4130540 0x7f3dd4130910 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f3dd000b9a0 tx=0x7f3dd000bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f606cf5c9a1b289 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:49.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.928+0000 7f3dcbfff700 1 -- 172.21.15.16:0/1961074382 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd000caa0 con 0x7f3dd4130540 2023-09-30T22:34:49.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.928+0000 7f3dcbfff700 1 -- 172.21.15.16:0/1961074382 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dd000cc00 con 0x7f3dd4130540 2023-09-30T22:34:49.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.928+0000 7f3dcbfff700 1 -- 172.21.15.16:0/1961074382 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd000f460 con 0x7f3dd4130540 2023-09-30T22:34:49.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.929+0000 7f3ddb2be700 1 -- 172.21.15.16:0/1961074382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dd4130540 msgr2=0x7f3dd4130910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:49.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.929+0000 7f3ddb2be700 1 --2- 172.21.15.16:0/1961074382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dd4130540 0x7f3dd4130910 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f3dd000b9a0 tx=0x7f3dd000bcb0 comp rx=0 tx=0).stop 2023-09-30T22:34:49.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.929+0000 7f3ddb2be700 1 -- 172.21.15.16:0/1961074382 shutdown_connections 2023-09-30T22:34:49.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.929+0000 7f3ddb2be700 1 --2- 172.21.15.16:0/1961074382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dd4130540 0x7f3dd4130910 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:49.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.929+0000 7f3ddb2be700 1 --2- 172.21.15.16:0/1961074382 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3dd4130e50 0x7f3dd406f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:49.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.929+0000 7f3ddb2be700 1 -- 172.21.15.16:0/1961074382 >> 172.21.15.16:0/1961074382 conn(0x7f3dd40a2aa0 msgr2=0x7f3dd40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:49.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.929+0000 7f3ddb2be700 1 -- 172.21.15.16:0/1961074382 shutdown_connections 2023-09-30T22:34:49.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.929+0000 7f3ddb2be700 1 -- 172.21.15.16:0/1961074382 wait complete. 2023-09-30T22:34:49.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.930+0000 7f3ddb2be700 1 Processor -- start 2023-09-30T22:34:49.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.930+0000 7f3ddb2be700 1 -- start start 2023-09-30T22:34:49.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.930+0000 7f3ddb2be700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3dd406fb20 0x7f3dd4119a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:49.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.930+0000 7f3ddb2be700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3dd4130540 0x7f3dd4119f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:49.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.930+0000 7f3ddb2be700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dd4130e50 0x7f3dd411a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:49.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.930+0000 7f3ddb2be700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd411bdb0 con 0x7f3dd4130e50 2023-09-30T22:34:49.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.930+0000 7f3ddb2be700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd40a0b50 con 0x7f3dd406fb20 2023-09-30T22:34:49.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.930+0000 7f3ddb2be700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd411d3f0 con 0x7f3dd4130540 2023-09-30T22:34:49.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.931+0000 7f3dd8859700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3dd4130540 0x7f3dd4119f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:49.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.931+0000 7f3dd985b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dd4130e50 0x7f3dd411a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:49.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.931+0000 7f3dd8859700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3dd4130540 0x7f3dd4119f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:38816/0 (socket says 172.21.15.16:38816) 2023-09-30T22:34:49.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.931+0000 7f3dd8859700 1 -- 172.21.15.16:0/3754499412 learned_addr learned my addr 172.21.15.16:0/3754499412 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:49.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.931+0000 7f3dd905a700 1 --2- 172.21.15.16:0/3754499412 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3dd406fb20 0x7f3dd4119a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:49.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.931+0000 7f3dd985b700 1 -- 172.21.15.16:0/3754499412 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3dd4130540 msgr2=0x7f3dd4119f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:49.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.931+0000 7f3dd985b700 1 --2- 172.21.15.16:0/3754499412 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3dd4130540 0x7f3dd4119f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:49.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.931+0000 7f3dd985b700 1 -- 172.21.15.16:0/3754499412 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3dd406fb20 msgr2=0x7f3dd4119a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:49.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.931+0000 7f3dd985b700 1 --2- 172.21.15.16:0/3754499412 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3dd406fb20 0x7f3dd4119a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:49.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.931+0000 7f3dd985b700 1 -- 172.21.15.16:0/3754499412 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dc400d940 con 0x7f3dd4130e50 2023-09-30T22:34:49.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.931+0000 7f3dd8859700 1 --2- 172.21.15.16:0/3754499412 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3dd4130540 0x7f3dd4119f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:49.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.931+0000 7f3dd985b700 1 --2- 172.21.15.16:0/3754499412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dd4130e50 0x7f3dd411a480 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc00cb90 tx=0x7f3dcc00cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:49.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.931+0000 7f3dc9ffb700 1 -- 172.21.15.16:0/3754499412 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc003c00 con 0x7f3dd4130e50 2023-09-30T22:34:49.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.932+0000 7f3dc9ffb700 1 -- 172.21.15.16:0/3754499412 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dcc003d60 con 0x7f3dd4130e50 2023-09-30T22:34:49.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.932+0000 7f3ddb2be700 1 -- 172.21.15.16:0/3754499412 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dd000b600 con 0x7f3dd4130e50 2023-09-30T22:34:49.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.932+0000 7f3ddb2be700 1 -- 172.21.15.16:0/3754499412 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dd411ce50 con 0x7f3dd4130e50 2023-09-30T22:34:49.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.932+0000 7f3dc9ffb700 1 -- 172.21.15.16:0/3754499412 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc00aa80 con 0x7f3dd4130e50 2023-09-30T22:34:49.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.933+0000 7f3dc9ffb700 1 -- 172.21.15.16:0/3754499412 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3dcc00abe0 con 0x7f3dd4130e50 2023-09-30T22:34:49.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.933+0000 7f3dc9ffb700 1 -- 172.21.15.16:0/3754499412 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f3dcc03f5c0 con 0x7f3dd4130e50 2023-09-30T22:34:49.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.933+0000 7f3ddb2be700 1 -- 172.21.15.16:0/3754499412 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3db40052e0 con 0x7f3dd4130e50 2023-09-30T22:34:49.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.938+0000 7f3dc9ffb700 1 -- 172.21.15.16:0/3754499412 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3dcc0198f0 con 0x7f3dd4130e50 2023-09-30T22:34:49.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.954+0000 7f4ec4e41700 1 Processor -- start 2023-09-30T22:34:49.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.954+0000 7ff68c10a700 1 Processor -- start 2023-09-30T22:34:49.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.954+0000 7f4ec4e41700 1 -- start start 2023-09-30T22:34:49.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.954+0000 7ff68c10a700 1 -- start start 2023-09-30T22:34:49.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.955+0000 7f4ec4e41700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4ec00a1510 0x7f4ec0138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:49.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.955+0000 7f4ec4e41700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4ec0139250 0x7f4ec013b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:49.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.955+0000 7f4ec4e41700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ec009edd0 con 0x7f4ec00a0a30 2023-09-30T22:34:49.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.955+0000 7f4ec4e41700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ec00a18e0 con 0x7f4ec0139250 2023-09-30T22:34:49.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.955+0000 7f4ec4e41700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec009e1f0 con 0x7f4ec00a1510 2023-09-30T22:34:49.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.955+0000 7ff68c10a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff6840a0a30 0x7ff6840a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:49.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.955+0000 7f4ebed9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4ec0139250 0x7f4ec013b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:49.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.955+0000 7f4ebed9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4ec0139250 0x7f4ec013b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34448/0 (socket says 172.21.15.16:34448) 2023-09-30T22:34:49.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.955+0000 7f4ebed9d700 1 -- 172.21.15.16:0/3185545025 learned_addr learned my addr 172.21.15.16:0/3185545025 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:49.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.955+0000 7ff68c10a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff68409d630 0x7ff6840a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:49.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.955+0000 7f4eb5d9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4ec00a1510 0x7f4ec0138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:49.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.955+0000 7ff68c10a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff684133930 0x7ff684133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:49.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.955+0000 7ff68c10a700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6840a2a70 con 0x7ff68409d630 2023-09-30T22:34:49.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.955+0000 7ff68c10a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6840a01d0 con 0x7ff684133930 2023-09-30T22:34:49.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.955+0000 7ff68c10a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff68409e2b0 con 0x7ff6840a0a30 2023-09-30T22:34:49.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.955+0000 7ff68a6a7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff684133930 0x7ff684133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:49.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.955+0000 7ff689ea6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff6840a0a30 0x7ff6840a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:49.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.955+0000 7f4eb5d9b700 1 -- 172.21.15.16:0/3185545025 >> v2:172.21.15.16:3300/0 conn(0x7f4ec0139250 msgr2=0x7f4ec013b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:49.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.955+0000 7f4eb5d9b700 1 --2- 172.21.15.16:0/3185545025 >> v2:172.21.15.16:3300/0 conn(0x7f4ec0139250 0x7f4ec013b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:49.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.955+0000 7f4eb5d9b700 1 -- 172.21.15.16:0/3185545025 >> v1:172.21.15.16:6789/0 conn(0x7f4ec00a0a30 legacy=0x7f4ec00a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:49.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.955+0000 7f4eb5d9b700 1 -- 172.21.15.16:0/3185545025 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec009dc50 con 0x7f4ec00a1510 2023-09-30T22:34:49.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.955+0000 7ff689ea6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff6840a0a30 0x7ff6840a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:38822/0 (socket says 172.21.15.16:38822) 2023-09-30T22:34:49.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.955+0000 7ff68a6a7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff684133930 0x7ff684133dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:42110/0 (socket says 172.21.15.16:42110) 2023-09-30T22:34:49.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.955+0000 7ff689ea6700 1 -- 172.21.15.16:0/584951014 learned_addr learned my addr 172.21.15.16:0/584951014 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:49.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.955+0000 7ff6896a5700 1 --2- 172.21.15.16:0/584951014 >> v2:172.21.15.16:3300/0 conn(0x7ff68409d630 0x7ff6840a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:49.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.956+0000 7ff6896a5700 1 -- 172.21.15.16:0/584951014 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff6840a0a30 msgr2=0x7ff6840a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:49.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.956+0000 7ff6896a5700 1 --2- 172.21.15.16:0/584951014 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff6840a0a30 0x7ff6840a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:49.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.956+0000 7ff6896a5700 1 -- 172.21.15.16:0/584951014 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff684133930 msgr2=0x7ff684133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:49.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.956+0000 7ff6896a5700 1 --2- 172.21.15.16:0/584951014 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff684133930 0x7ff684133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:49.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.956+0000 7ff6896a5700 1 -- 172.21.15.16:0/584951014 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff68409dc50 con 0x7ff68409d630 2023-09-30T22:34:49.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.956+0000 7ff68a6a7700 1 --2- 172.21.15.16:0/584951014 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff684133930 0x7ff684133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:49.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.956+0000 7f4eb5d9b700 1 --2- 172.21.15.16:0/3185545025 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4ec00a1510 0x7f4ec0138d10 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8009870 tx=0x7f4ea8009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eded8dfd565789f8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:49.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.956+0000 7ff6896a5700 1 --2- 172.21.15.16:0/584951014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff68409d630 0x7ff6840a1340 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7ff674009840 tx=0x7ff674009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d61ff53a3fc2a13 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:49.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.956+0000 7f4ebdd9b700 1 -- 172.21.15.16:0/3185545025 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea8004230 con 0x7f4ec00a1510 2023-09-30T22:34:49.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.956+0000 7ff688ea4700 1 -- 172.21.15.16:0/584951014 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff674011830 con 0x7ff68409d630 2023-09-30T22:34:49.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.956+0000 7ff688ea4700 1 -- 172.21.15.16:0/584951014 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff674011990 con 0x7ff68409d630 2023-09-30T22:34:49.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.956+0000 7ff688ea4700 1 -- 172.21.15.16:0/584951014 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff674011d20 con 0x7ff68409d630 2023-09-30T22:34:49.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.956+0000 7f4ebdd9b700 1 -- 172.21.15.16:0/3185545025 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ea8004390 con 0x7f4ec00a1510 2023-09-30T22:34:49.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.956+0000 7f4ebdd9b700 1 -- 172.21.15.16:0/3185545025 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea80036a0 con 0x7f4ec00a1510 2023-09-30T22:34:49.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.957+0000 7ff68c10a700 1 -- 172.21.15.16:0/584951014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff68409d630 msgr2=0x7ff6840a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:49.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.957+0000 7ff68c10a700 1 --2- 172.21.15.16:0/584951014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff68409d630 0x7ff6840a1340 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7ff674009840 tx=0x7ff674009b50 comp rx=0 tx=0).stop 2023-09-30T22:34:49.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.957+0000 7f4ec4e41700 1 -- 172.21.15.16:0/3185545025 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4ec00a1510 msgr2=0x7f4ec0138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:49.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.957+0000 7f4ec4e41700 1 --2- 172.21.15.16:0/3185545025 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4ec00a1510 0x7f4ec0138d10 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8009870 tx=0x7f4ea8009b80 comp rx=0 tx=0).stop 2023-09-30T22:34:49.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.957+0000 7f4ec4e41700 1 -- 172.21.15.16:0/3185545025 shutdown_connections 2023-09-30T22:34:49.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.957+0000 7ff68c10a700 1 -- 172.21.15.16:0/584951014 shutdown_connections 2023-09-30T22:34:49.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.957+0000 7ff68c10a700 1 --2- 172.21.15.16:0/584951014 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff684133930 0x7ff684133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:49.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.957+0000 7ff68c10a700 1 --2- 172.21.15.16:0/584951014 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff6840a0a30 0x7ff6840a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:49.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.957+0000 7f4ec4e41700 1 --2- 172.21.15.16:0/3185545025 >> v2:172.21.15.16:3300/0 conn(0x7f4ec0139250 0x7f4ec013b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:49.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.957+0000 7f4ec4e41700 1 --2- 172.21.15.16:0/3185545025 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4ec00a1510 0x7f4ec0138d10 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:49.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.957+0000 7f4ec4e41700 1 -- 172.21.15.16:0/3185545025 >> 172.21.15.16:0/3185545025 conn(0x7f4ec00a6550 msgr2=0x7f4ec00a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:49.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.957+0000 7ff68c10a700 1 --2- 172.21.15.16:0/584951014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff68409d630 0x7ff6840a1340 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:49.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.957+0000 7ff68c10a700 1 -- 172.21.15.16:0/584951014 >> 172.21.15.16:0/584951014 conn(0x7ff6840a3050 msgr2=0x7ff6840a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:49.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.957+0000 7f4ec4e41700 1 -- 172.21.15.16:0/3185545025 shutdown_connections 2023-09-30T22:34:49.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.957+0000 7f4ec4e41700 1 -- 172.21.15.16:0/3185545025 wait complete. 2023-09-30T22:34:49.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.957+0000 7ff68c10a700 1 -- 172.21.15.16:0/584951014 shutdown_connections 2023-09-30T22:34:49.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.957+0000 7ff68c10a700 1 -- 172.21.15.16:0/584951014 wait complete. 2023-09-30T22:34:49.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.958+0000 7ff68c10a700 1 Processor -- start 2023-09-30T22:34:49.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.958+0000 7ff68c10a700 1 -- start start 2023-09-30T22:34:49.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.959+0000 7f4ec4e41700 1 Processor -- start 2023-09-30T22:34:49.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.959+0000 7f4ec4e41700 1 -- start start 2023-09-30T22:34:49.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.959+0000 7ff68c10a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff68409d630 0x7ff68411f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:49.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.959+0000 7ff68c10a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff6840a0a30 0x7ff68411fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:49.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.959+0000 7ff68c10a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff684133930 0x7ff6841c7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:49.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.959+0000 7ff68c10a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6841222e0 con 0x7ff68409d630 2023-09-30T22:34:49.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.959+0000 7ff68c10a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff684122800 con 0x7ff6840a0a30 2023-09-30T22:34:49.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.959+0000 7ff68c10a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff684133430 con 0x7ff684133930 2023-09-30T22:34:49.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.959+0000 7ff6896a5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff6840a0a30 0x7ff68411fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:49.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.959+0000 7ff6896a5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff6840a0a30 0x7ff68411fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:42116/0 (socket says 172.21.15.16:42116) 2023-09-30T22:34:49.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.959+0000 7ff6896a5700 1 -- 172.21.15.16:0/2389170351 learned_addr learned my addr 172.21.15.16:0/2389170351 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:49.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.959+0000 7ff68a6a7700 1 --2- 172.21.15.16:0/2389170351 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff684133930 0x7ff6841c7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:49.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.959+0000 7ff689ea6700 1 --2- 172.21.15.16:0/2389170351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff68409d630 0x7ff68411f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:49.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.959+0000 7f4ec4e41700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4ec00a0a30 0x7f4ec01c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:49.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.959+0000 7f4ec4e41700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4ec00a1510 0x7f4ec01c8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:49.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.959+0000 7f4ec4e41700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ec0139250 0x7f4ec01217e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:49.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.959+0000 7f4ec4e41700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec0124cd0 con 0x7f4ec0139250 2023-09-30T22:34:49.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.959+0000 7f4ec4e41700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec0138c40 con 0x7f4ec00a1510 2023-09-30T22:34:49.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.959+0000 7f4ec4e41700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec01be3d0 con 0x7f4ec00a0a30 2023-09-30T22:34:49.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.959+0000 7f4ebed9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ec0139250 0x7f4ec01217e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:49.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.960+0000 7f4ebed9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ec0139250 0x7f4ec01217e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34482/0 (socket says 172.21.15.16:34482) 2023-09-30T22:34:49.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.960+0000 7f4ebed9d700 1 -- 172.21.15.16:0/914520852 learned_addr learned my addr 172.21.15.16:0/914520852 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:49.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.960+0000 7f4eb5d9b700 1 --2- 172.21.15.16:0/914520852 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4ec00a1510 0x7f4ec01c8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:49.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.959+0000 7ff6896a5700 1 -- 172.21.15.16:0/2389170351 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff684133930 msgr2=0x7ff6841c7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:49.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.959+0000 7ff6896a5700 1 --2- 172.21.15.16:0/2389170351 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff684133930 0x7ff6841c7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:49.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.959+0000 7ff6896a5700 1 -- 172.21.15.16:0/2389170351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff68409d630 msgr2=0x7ff68411f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:49.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.959+0000 7ff6896a5700 1 --2- 172.21.15.16:0/2389170351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff68409d630 0x7ff68411f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:49.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.959+0000 7ff6896a5700 1 -- 172.21.15.16:0/2389170351 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff67c009a70 con 0x7ff6840a0a30 2023-09-30T22:34:49.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.960+0000 7ff6896a5700 1 --2- 172.21.15.16:0/2389170351 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff6840a0a30 0x7ff68411fa40 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7ff674000c00 tx=0x7ff674004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:49.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.960+0000 7f4ebe59c700 1 --2- 172.21.15.16:0/914520852 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4ec00a0a30 0x7f4ec01c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:49.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.960+0000 7f4ebed9d700 1 -- 172.21.15.16:0/914520852 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4ec00a0a30 msgr2=0x7f4ec01c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:49.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.960+0000 7f4ebed9d700 1 --2- 172.21.15.16:0/914520852 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4ec00a0a30 0x7f4ec01c86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:49.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.960+0000 7f4ebed9d700 1 -- 172.21.15.16:0/914520852 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4ec00a1510 msgr2=0x7f4ec01c8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:49.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.960+0000 7f4ebed9d700 1 --2- 172.21.15.16:0/914520852 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4ec00a1510 0x7f4ec01c8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:49.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.960+0000 7f4ebed9d700 1 -- 172.21.15.16:0/914520852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea80094a0 con 0x7f4ec0139250 2023-09-30T22:34:49.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.960+0000 7f4ebed9d700 1 --2- 172.21.15.16:0/914520852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ec0139250 0x7f4ec01217e0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f4eb800efc0 tx=0x7f4eb800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:49.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.960+0000 7ff67a7fc700 1 -- 172.21.15.16:0/2389170351 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff674004240 con 0x7ff6840a0a30 2023-09-30T22:34:49.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.960+0000 7ff67a7fc700 1 -- 172.21.15.16:0/2389170351 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff67401f070 con 0x7ff6840a0a30 2023-09-30T22:34:49.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.960+0000 7ff68c10a700 1 -- 172.21.15.16:0/2389170351 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6740094a0 con 0x7ff6840a0a30 2023-09-30T22:34:49.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.960+0000 7ff68c10a700 1 -- 172.21.15.16:0/2389170351 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff684120ea0 con 0x7ff6840a0a30 2023-09-30T22:34:49.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.960+0000 7f4eb77fe700 1 -- 172.21.15.16:0/914520852 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb8003c20 con 0x7f4ec0139250 2023-09-30T22:34:49.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.960+0000 7f4eb77fe700 1 -- 172.21.15.16:0/914520852 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eb800f040 con 0x7f4ec0139250 2023-09-30T22:34:49.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.960+0000 7f4eb77fe700 1 -- 172.21.15.16:0/914520852 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb8009e30 con 0x7f4ec0139250 2023-09-30T22:34:49.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.961+0000 7f4ec4e41700 1 -- 172.21.15.16:0/914520852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ec0137620 con 0x7f4ec0139250 2023-09-30T22:34:49.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.961+0000 7f4ec4e41700 1 -- 172.21.15.16:0/914520852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ec0125a00 con 0x7f4ec0139250 2023-09-30T22:34:49.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.960+0000 7ff67a7fc700 1 -- 172.21.15.16:0/2389170351 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6740045e0 con 0x7ff6840a0a30 2023-09-30T22:34:49.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.961+0000 7ff67a7fc700 1 -- 172.21.15.16:0/2389170351 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff674021030 con 0x7ff6840a0a30 2023-09-30T22:34:49.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.961+0000 7ff67a7fc700 1 -- 172.21.15.16:0/2389170351 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7ff6740406b0 con 0x7ff6840a0a30 2023-09-30T22:34:49.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.962+0000 7ff68c10a700 1 -- 172.21.15.16:0/2389170351 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff66c0052e0 con 0x7ff6840a0a30 2023-09-30T22:34:49.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.962+0000 7f4eb77fe700 1 -- 172.21.15.16:0/914520852 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4eb800c930 con 0x7f4ec0139250 2023-09-30T22:34:49.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.963+0000 7f4eb77fe700 1 -- 172.21.15.16:0/914520852 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f4eb803f9e0 con 0x7f4ec0139250 2023-09-30T22:34:49.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.963+0000 7f4ec4e41700 1 -- 172.21.15.16:0/914520852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ec01251d0 con 0x7f4ec0139250 2023-09-30T22:34:49.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.970+0000 7ff67a7fc700 1 -- 172.21.15.16:0/2389170351 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff67401a450 con 0x7ff6840a0a30 2023-09-30T22:34:49.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:49.971+0000 7f4eb77fe700 1 -- 172.21.15.16:0/914520852 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4eb800cbe0 con 0x7f4ec0139250 2023-09-30T22:34:50.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:50.147+0000 7f3ddb2be700 1 -- 172.21.15.16:0/3754499412 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3db4005590 con 0x7f3dd4130e50 2023-09-30T22:34:50.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:50.147+0000 7f3dc9ffb700 1 -- 172.21.15.16:0/3754499412 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3dcc001c20 con 0x7f3dd4130e50 2023-09-30T22:34:50.148 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:34:50.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:50.150+0000 7f3ddb2be700 1 -- 172.21.15.16:0/3754499412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dd4130e50 msgr2=0x7f3dd411a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:50.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:50.150+0000 7f3ddb2be700 1 --2- 172.21.15.16:0/3754499412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dd4130e50 0x7f3dd411a480 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc00cb90 tx=0x7f3dcc00cea0 comp rx=0 tx=0).stop 2023-09-30T22:34:50.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:50.150+0000 7f3ddb2be700 1 -- 172.21.15.16:0/3754499412 shutdown_connections 2023-09-30T22:34:50.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:50.150+0000 7f3ddb2be700 1 --2- 172.21.15.16:0/3754499412 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3dd406fb20 0x7f3dd4119a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:50.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:50.150+0000 7f3ddb2be700 1 --2- 172.21.15.16:0/3754499412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3dd4130e50 0x7f3dd411a480 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:50.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:50.150+0000 7f3ddb2be700 1 --2- 172.21.15.16:0/3754499412 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3dd4130540 0x7f3dd4119f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:50.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:50.150+0000 7f3ddb2be700 1 -- 172.21.15.16:0/3754499412 >> 172.21.15.16:0/3754499412 conn(0x7f3dd40a2aa0 msgr2=0x7f3dd412d700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:50.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:50.150+0000 7f3ddb2be700 1 -- 172.21.15.16:0/3754499412 shutdown_connections 2023-09-30T22:34:50.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:50.150+0000 7f3ddb2be700 1 -- 172.21.15.16:0/3754499412 wait complete. 2023-09-30T22:34:50.164 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:34:50.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:50.176+0000 7f4ec4e41700 1 -- 172.21.15.16:0/914520852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f4ec0074ea0 con 0x7f4ec0139250 2023-09-30T22:34:50.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:50.177+0000 7f4eb77fe700 1 -- 172.21.15.16:0/914520852 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4eb8013b90 con 0x7f4ec0139250 2023-09-30T22:34:50.177 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:34:50.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:50.179+0000 7f4ec4e41700 1 -- 172.21.15.16:0/914520852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ec0139250 msgr2=0x7f4ec01217e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:50.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:50.179+0000 7f4ec4e41700 1 --2- 172.21.15.16:0/914520852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ec0139250 0x7f4ec01217e0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f4eb800efc0 tx=0x7f4eb800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:34:50.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:50.179+0000 7ff68c10a700 1 -- 172.21.15.16:0/2389170351 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff66c005590 con 0x7ff6840a0a30 2023-09-30T22:34:50.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:50.179+0000 7f4ec4e41700 1 -- 172.21.15.16:0/914520852 shutdown_connections 2023-09-30T22:34:50.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:50.179+0000 7f4ec4e41700 1 --2- 172.21.15.16:0/914520852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ec0139250 0x7f4ec01217e0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:50.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:50.179+0000 7f4ec4e41700 1 --2- 172.21.15.16:0/914520852 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4ec00a0a30 0x7f4ec01c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:50.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:50.179+0000 7f4ec4e41700 1 --2- 172.21.15.16:0/914520852 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4ec00a1510 0x7f4ec01c8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:50.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:50.179+0000 7f4ec4e41700 1 -- 172.21.15.16:0/914520852 >> 172.21.15.16:0/914520852 conn(0x7f4ec00a6550 msgr2=0x7f4ec0137b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:50.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:50.179+0000 7f4ec4e41700 1 -- 172.21.15.16:0/914520852 shutdown_connections 2023-09-30T22:34:50.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:50.179+0000 7f4ec4e41700 1 -- 172.21.15.16:0/914520852 wait complete. 2023-09-30T22:34:50.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:50.179+0000 7ff67a7fc700 1 -- 172.21.15.16:0/2389170351 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff6740233d0 con 0x7ff6840a0a30 2023-09-30T22:34:50.181 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:34:50.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:50.182+0000 7ff68c10a700 1 -- 172.21.15.16:0/2389170351 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff6840a0a30 msgr2=0x7ff68411fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:50.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:50.182+0000 7ff68c10a700 1 --2- 172.21.15.16:0/2389170351 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff6840a0a30 0x7ff68411fa40 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7ff674000c00 tx=0x7ff674004750 comp rx=0 tx=0).stop 2023-09-30T22:34:50.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:50.182+0000 7ff68c10a700 1 -- 172.21.15.16:0/2389170351 shutdown_connections 2023-09-30T22:34:50.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:50.182+0000 7ff68c10a700 1 --2- 172.21.15.16:0/2389170351 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff684133930 0x7ff6841c7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:50.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:50.182+0000 7ff68c10a700 1 --2- 172.21.15.16:0/2389170351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff68409d630 0x7ff68411f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:50.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:50.182+0000 7ff68c10a700 1 --2- 172.21.15.16:0/2389170351 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff6840a0a30 0x7ff68411fa40 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:50.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:50.182+0000 7ff68c10a700 1 -- 172.21.15.16:0/2389170351 >> 172.21.15.16:0/2389170351 conn(0x7ff6840a3050 msgr2=0x7ff6841bba60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:50.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:50.182+0000 7ff68c10a700 1 -- 172.21.15.16:0/2389170351 shutdown_connections 2023-09-30T22:34:50.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:50.182+0000 7ff68c10a700 1 -- 172.21.15.16:0/2389170351 wait complete. 2023-09-30T22:34:50.195 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:34:50.197 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:34:50.851 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:34:51.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:50.997+0000 7faa55f66700 1 Processor -- start 2023-09-30T22:34:51.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:50.997+0000 7faa55f66700 1 -- start start 2023-09-30T22:34:51.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:50.998+0000 7faa55f66700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faa5009d6f0 0x7faa5009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:51.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:50.998+0000 7faa55f66700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faa5009e440 0x7faa50134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:51.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:50.998+0000 7faa55f66700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa5009b120 con 0x7faa5009e000 2023-09-30T22:34:51.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:50.998+0000 7faa55f66700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa5006b460 con 0x7faa5009d6f0 2023-09-30T22:34:51.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:50.998+0000 7faa55f66700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa5009a7e0 con 0x7faa5009e440 2023-09-30T22:34:51.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:50.998+0000 7faa4f7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faa5009d6f0 0x7faa5009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:51.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:50.998+0000 7faa4effd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7faa5009e000 0x7faa501339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60882/0 (socket says 172.21.15.16:60882) 2023-09-30T22:34:51.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:50.998+0000 7faa4effd700 1 -- 172.21.15.16:0/2123110040 learned_addr learned my addr 172.21.15.16:0/2123110040 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:34:51.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:50.998+0000 7faa4ffff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faa5009e440 0x7faa50134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:51.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:50.998+0000 7faa4f7fe700 1 -- 172.21.15.16:0/2123110040 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faa5009e440 msgr2=0x7faa50134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:51.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:50.999+0000 7faa4f7fe700 1 --2- 172.21.15.16:0/2123110040 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faa5009e440 0x7faa50134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:51.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:50.999+0000 7faa4f7fe700 1 -- 172.21.15.16:0/2123110040 >> v1:172.21.15.16:6789/0 conn(0x7faa5009e000 legacy=0x7faa501339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:51.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:50.999+0000 7faa4f7fe700 1 -- 172.21.15.16:0/2123110040 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa5009a500 con 0x7faa5009d6f0 2023-09-30T22:34:51.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:50.999+0000 7faa4e7fc700 1 -- 172.21.15.16:0/2123110040 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2299271822 0 0) 0x7faa5009b120 con 0x7faa5009e000 2023-09-30T22:34:51.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:50.999+0000 7faa4ffff700 1 --2- 172.21.15.16:0/2123110040 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faa5009e440 0x7faa50134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:51.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:50.999+0000 7faa4f7fe700 1 --2- 172.21.15.16:0/2123110040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa5009d6f0 0x7faa5009dac0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7faa38009870 tx=0x7faa38009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bc4cb581af223c1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:51.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:50.999+0000 7faa4e7fc700 1 -- 172.21.15.16:0/2123110040 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa38011830 con 0x7faa5009d6f0 2023-09-30T22:34:51.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:50.999+0000 7faa4e7fc700 1 -- 172.21.15.16:0/2123110040 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa38011990 con 0x7faa5009d6f0 2023-09-30T22:34:51.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:50.999+0000 7faa4e7fc700 1 -- 172.21.15.16:0/2123110040 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa38011d20 con 0x7faa5009d6f0 2023-09-30T22:34:51.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:50.999+0000 7faa55f66700 1 -- 172.21.15.16:0/2123110040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa5009d6f0 msgr2=0x7faa5009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:51.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:50.999+0000 7faa55f66700 1 --2- 172.21.15.16:0/2123110040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa5009d6f0 0x7faa5009dac0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7faa38009870 tx=0x7faa38009b80 comp rx=0 tx=0).stop 2023-09-30T22:34:51.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.000+0000 7faa55f66700 1 -- 172.21.15.16:0/2123110040 shutdown_connections 2023-09-30T22:34:51.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.000+0000 7faa55f66700 1 --2- 172.21.15.16:0/2123110040 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faa5009e440 0x7faa50134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:51.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.000+0000 7faa55f66700 1 --2- 172.21.15.16:0/2123110040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa5009d6f0 0x7faa5009dac0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:51.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.000+0000 7faa55f66700 1 -- 172.21.15.16:0/2123110040 >> 172.21.15.16:0/2123110040 conn(0x7faa50066f50 msgr2=0x7faa5009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:51.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.000+0000 7faa55f66700 1 -- 172.21.15.16:0/2123110040 shutdown_connections 2023-09-30T22:34:51.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.000+0000 7faa55f66700 1 -- 172.21.15.16:0/2123110040 wait complete. 2023-09-30T22:34:51.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.001+0000 7faa55f66700 1 Processor -- start 2023-09-30T22:34:51.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.001+0000 7faa55f66700 1 -- start start 2023-09-30T22:34:51.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.001+0000 7faa55f66700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faa5009d6f0 0x7faa501c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:51.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.001+0000 7faa55f66700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faa5009e000 0x7faa501c8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:51.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.001+0000 7faa55f66700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa5009e440 0x7faa501c89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:51.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.001+0000 7faa55f66700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa501a6f70 con 0x7faa5009e440 2023-09-30T22:34:51.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.001+0000 7faa55f66700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa501a6410 con 0x7faa5009d6f0 2023-09-30T22:34:51.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.001+0000 7faa55f66700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faa5009c710 con 0x7faa5009e000 2023-09-30T22:34:51.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.001+0000 7faa4f7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faa5009d6f0 0x7faa501c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:51.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.001+0000 7faa4f7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faa5009d6f0 0x7faa501c7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:42140/0 (socket says 172.21.15.16:42140) 2023-09-30T22:34:51.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.001+0000 7faa4f7fe700 1 -- 172.21.15.16:0/587673214 learned_addr learned my addr 172.21.15.16:0/587673214 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:51.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.001+0000 7faa4ffff700 1 --2- 172.21.15.16:0/587673214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa5009e440 0x7faa501c89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:51.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.001+0000 7faa4effd700 1 --2- 172.21.15.16:0/587673214 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faa5009e000 0x7faa501c8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:51.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.002+0000 7faa4f7fe700 1 -- 172.21.15.16:0/587673214 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faa5009e000 msgr2=0x7faa501c8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:51.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.002+0000 7faa4f7fe700 1 --2- 172.21.15.16:0/587673214 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faa5009e000 0x7faa501c8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:51.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.002+0000 7faa4f7fe700 1 -- 172.21.15.16:0/587673214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa5009e440 msgr2=0x7faa501c89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:51.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.002+0000 7faa4f7fe700 1 --2- 172.21.15.16:0/587673214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa5009e440 0x7faa501c89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:51.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.002+0000 7faa4f7fe700 1 -- 172.21.15.16:0/587673214 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa48009a70 con 0x7faa5009d6f0 2023-09-30T22:34:51.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.002+0000 7faa4ffff700 1 --2- 172.21.15.16:0/587673214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa5009e440 0x7faa501c89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:51.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.002+0000 7faa4f7fe700 1 --2- 172.21.15.16:0/587673214 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faa5009d6f0 0x7faa501c7f40 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7faa380090b0 tx=0x7faa38004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:51.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.002+0000 7faa47fff700 1 -- 172.21.15.16:0/587673214 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3801a960 con 0x7faa5009d6f0 2023-09-30T22:34:51.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.002+0000 7faa47fff700 1 -- 172.21.15.16:0/587673214 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa3801aac0 con 0x7faa5009d6f0 2023-09-30T22:34:51.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.002+0000 7faa55f66700 1 -- 172.21.15.16:0/587673214 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa380094a0 con 0x7faa5009d6f0 2023-09-30T22:34:51.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.002+0000 7faa47fff700 1 -- 172.21.15.16:0/587673214 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3801ae50 con 0x7faa5009d6f0 2023-09-30T22:34:51.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.002+0000 7faa55f66700 1 -- 172.21.15.16:0/587673214 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa501a6a80 con 0x7faa5009d6f0 2023-09-30T22:34:51.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.003+0000 7faa47fff700 1 -- 172.21.15.16:0/587673214 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7faa380243f0 con 0x7faa5009d6f0 2023-09-30T22:34:51.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.003+0000 7faa47fff700 1 -- 172.21.15.16:0/587673214 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7faa38040bc0 con 0x7faa5009d6f0 2023-09-30T22:34:51.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.003+0000 7faa55f66700 1 -- 172.21.15.16:0/587673214 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa501b91f0 con 0x7faa5009d6f0 2023-09-30T22:34:51.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.008+0000 7faa47fff700 1 -- 172.21.15.16:0/587673214 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7faa501b91f0 con 0x7faa5009d6f0 2023-09-30T22:34:51.165 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:34:51.196 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:34:51.197 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:34:51.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.223+0000 7faa55f66700 1 -- 172.21.15.16:0/587673214 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7faa501b91f0 con 0x7faa5009d6f0 2023-09-30T22:34:51.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.223+0000 7faa47fff700 1 -- 172.21.15.16:0/587673214 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7faa501b91f0 con 0x7faa5009d6f0 2023-09-30T22:34:51.224 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:34:51.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.226+0000 7faa55f66700 1 -- 172.21.15.16:0/587673214 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faa5009d6f0 msgr2=0x7faa501c7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:51.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.226+0000 7faa55f66700 1 --2- 172.21.15.16:0/587673214 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faa5009d6f0 0x7faa501c7f40 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7faa380090b0 tx=0x7faa38004750 comp rx=0 tx=0).stop 2023-09-30T22:34:51.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.226+0000 7faa55f66700 1 -- 172.21.15.16:0/587673214 shutdown_connections 2023-09-30T22:34:51.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.226+0000 7faa55f66700 1 --2- 172.21.15.16:0/587673214 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faa5009d6f0 0x7faa501c7f40 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:51.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.226+0000 7faa55f66700 1 --2- 172.21.15.16:0/587673214 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faa5009e440 0x7faa501c89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:51.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.226+0000 7faa55f66700 1 --2- 172.21.15.16:0/587673214 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faa5009e000 0x7faa501c8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:51.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.226+0000 7faa55f66700 1 -- 172.21.15.16:0/587673214 >> 172.21.15.16:0/587673214 conn(0x7faa50066f50 msgr2=0x7faa501bbb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:51.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.227+0000 7faa55f66700 1 -- 172.21.15.16:0/587673214 shutdown_connections 2023-09-30T22:34:51.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.227+0000 7faa55f66700 1 -- 172.21.15.16:0/587673214 wait complete. 2023-09-30T22:34:51.244 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:34:51.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.315+0000 7f169ccb4700 1 Processor -- start 2023-09-30T22:34:51.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.315+0000 7f169ccb4700 1 -- start start 2023-09-30T22:34:51.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.316+0000 7f169ccb4700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1698131810 0x7f1698131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:51.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.316+0000 7f169ccb4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1698099a40 0x7f1698099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:51.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.316+0000 7f169ccb4700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16981313c0 con 0x7f169809a3b0 2023-09-30T22:34:51.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.316+0000 7f169ccb4700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f169809b580 con 0x7f1698131810 2023-09-30T22:34:51.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.316+0000 7f169ccb4700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f169806e8c0 con 0x7f1698099a40 2023-09-30T22:34:51.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.316+0000 7f1695d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1698099a40 0x7f1698099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:51.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.316+0000 7f169659c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1698131810 0x7f1698131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:51.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.316+0000 7f1696d9d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f169809a3b0 0x7f1698138650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60886/0 (socket says 172.21.15.16:60886) 2023-09-30T22:34:51.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.316+0000 7f1696d9d700 1 -- 172.21.15.16:0/466905557 learned_addr learned my addr 172.21.15.16:0/466905557 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:34:51.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.316+0000 7f1695d9b700 1 -- 172.21.15.16:0/466905557 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1698131810 msgr2=0x7f1698131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:51.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.316+0000 7f1695d9b700 1 --2- 172.21.15.16:0/466905557 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1698131810 0x7f1698131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:51.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.316+0000 7f1695d9b700 1 -- 172.21.15.16:0/466905557 >> v1:172.21.15.16:6789/0 conn(0x7f169809a3b0 legacy=0x7f1698138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:51.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.316+0000 7f169559a700 1 -- 172.21.15.16:0/466905557 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3874974964 0 0) 0x7f16981313c0 con 0x7f169809a3b0 2023-09-30T22:34:51.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.316+0000 7f1695d9b700 1 -- 172.21.15.16:0/466905557 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f169809a820 con 0x7f1698099a40 2023-09-30T22:34:51.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.316+0000 7f169659c700 1 --2- 172.21.15.16:0/466905557 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1698131810 0x7f1698131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:51.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.317+0000 7f1695d9b700 1 --2- 172.21.15.16:0/466905557 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1698099a40 0x7f1698099e70 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f168c009870 tx=0x7f168c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b261990c018a25a2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:51.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.317+0000 7f169559a700 1 -- 172.21.15.16:0/466905557 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168c011830 con 0x7f1698099a40 2023-09-30T22:34:51.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.317+0000 7f169559a700 1 -- 172.21.15.16:0/466905557 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f168c011990 con 0x7f1698099a40 2023-09-30T22:34:51.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.317+0000 7f169559a700 1 -- 172.21.15.16:0/466905557 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168c011d20 con 0x7f1698099a40 2023-09-30T22:34:51.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.318+0000 7f169ccb4700 1 -- 172.21.15.16:0/466905557 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1698099a40 msgr2=0x7f1698099e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:51.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.318+0000 7f169ccb4700 1 --2- 172.21.15.16:0/466905557 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1698099a40 0x7f1698099e70 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f168c009870 tx=0x7f168c009b80 comp rx=0 tx=0).stop 2023-09-30T22:34:51.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.318+0000 7f169ccb4700 1 -- 172.21.15.16:0/466905557 shutdown_connections 2023-09-30T22:34:51.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.318+0000 7f169ccb4700 1 --2- 172.21.15.16:0/466905557 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1698131810 0x7f1698131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:51.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.318+0000 7f169ccb4700 1 --2- 172.21.15.16:0/466905557 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1698099a40 0x7f1698099e70 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:51.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.318+0000 7f169ccb4700 1 -- 172.21.15.16:0/466905557 >> 172.21.15.16:0/466905557 conn(0x7f16980a2aa0 msgr2=0x7f16980a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:51.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.318+0000 7f169ccb4700 1 -- 172.21.15.16:0/466905557 shutdown_connections 2023-09-30T22:34:51.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.318+0000 7f169ccb4700 1 -- 172.21.15.16:0/466905557 wait complete. 2023-09-30T22:34:51.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.319+0000 7f169ccb4700 1 Processor -- start 2023-09-30T22:34:51.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.319+0000 7f169ccb4700 1 -- start start 2023-09-30T22:34:51.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.320+0000 7f169ccb4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1698099a40 0x7f16981195d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:51.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.320+0000 7f169ccb4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f169809a3b0 0x7f1698119b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:51.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.320+0000 7f169ccb4700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1698131810 0x7f169811a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:51.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.320+0000 7f169ccb4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f169811aa80 con 0x7f169809a3b0 2023-09-30T22:34:51.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.320+0000 7f169ccb4700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f169811b190 con 0x7f1698131810 2023-09-30T22:34:51.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.320+0000 7f169ccb4700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f169811c3e0 con 0x7f1698099a40 2023-09-30T22:34:51.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.320+0000 7f1695d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f169809a3b0 0x7f1698119b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:51.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.320+0000 7f1695d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f169809a3b0 0x7f1698119b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34506/0 (socket says 172.21.15.16:34506) 2023-09-30T22:34:51.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.320+0000 7f1695d9b700 1 -- 172.21.15.16:0/2691144785 learned_addr learned my addr 172.21.15.16:0/2691144785 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:51.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.320+0000 7f169659c700 1 --2- 172.21.15.16:0/2691144785 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1698099a40 0x7f16981195d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:51.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.320+0000 7f1696d9d700 1 --2- 172.21.15.16:0/2691144785 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1698131810 0x7f169811a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:51.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.320+0000 7f169659c700 1 -- 172.21.15.16:0/2691144785 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1698131810 msgr2=0x7f169811a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:51.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.320+0000 7f169659c700 1 --2- 172.21.15.16:0/2691144785 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1698131810 0x7f169811a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:51.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.320+0000 7f169659c700 1 -- 172.21.15.16:0/2691144785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f169809a3b0 msgr2=0x7f1698119b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:51.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.320+0000 7f169659c700 1 --2- 172.21.15.16:0/2691144785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f169809a3b0 0x7f1698119b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:51.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.320+0000 7f169659c700 1 -- 172.21.15.16:0/2691144785 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16880094a0 con 0x7f1698099a40 2023-09-30T22:34:51.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.320+0000 7f1695d9b700 1 --2- 172.21.15.16:0/2691144785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f169809a3b0 0x7f1698119b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:34:51.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.320+0000 7f169659c700 1 --2- 172.21.15.16:0/2691144785 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1698099a40 0x7f16981195d0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f168800e9e0 tx=0x7f168800ecf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:51.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.321+0000 7f1686ffd700 1 -- 172.21.15.16:0/2691144785 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168800c850 con 0x7f1698099a40 2023-09-30T22:34:51.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.321+0000 7f1686ffd700 1 -- 172.21.15.16:0/2691144785 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f168800c9b0 con 0x7f1698099a40 2023-09-30T22:34:51.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.321+0000 7f169ccb4700 1 -- 172.21.15.16:0/2691144785 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f168c0094a0 con 0x7f1698099a40 2023-09-30T22:34:51.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.321+0000 7f1686ffd700 1 -- 172.21.15.16:0/2691144785 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168800cd60 con 0x7f1698099a40 2023-09-30T22:34:51.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.321+0000 7f169ccb4700 1 -- 172.21.15.16:0/2691144785 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16981307c0 con 0x7f1698099a40 2023-09-30T22:34:51.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.322+0000 7f169ccb4700 1 -- 172.21.15.16:0/2691144785 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f169806eb00 con 0x7f1698099a40 2023-09-30T22:34:51.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.322+0000 7f1686ffd700 1 -- 172.21.15.16:0/2691144785 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1688013470 con 0x7f1698099a40 2023-09-30T22:34:51.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.322+0000 7f1686ffd700 1 -- 172.21.15.16:0/2691144785 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f1688040110 con 0x7f1698099a40 2023-09-30T22:34:51.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.327+0000 7f1686ffd700 1 -- 172.21.15.16:0/2691144785 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1688011760 con 0x7f1698099a40 2023-09-30T22:34:51.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.354+0000 7fdef18be700 1 Processor -- start 2023-09-30T22:34:51.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.354+0000 7fdef18be700 1 -- start start 2023-09-30T22:34:51.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.354+0000 7fdef18be700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdeec09e1d0 0x7fdeec138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:51.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.354+0000 7fdef18be700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdeec139170 0x7fdeec13b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:51.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.354+0000 7fdef18be700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdeec09b120 con 0x7fdeec09d6f0 2023-09-30T22:34:51.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.354+0000 7fdef18be700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdeec06b460 con 0x7fdeec139170 2023-09-30T22:34:51.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.354+0000 7fdef18be700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdeec09a7e0 con 0x7fdeec09e1d0 2023-09-30T22:34:51.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.354+0000 7fdeea7fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdeec09e1d0 0x7fdeec138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:51.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.354+0000 7fdeeb7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdeec139170 0x7fdeec13b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:51.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.354+0000 7fdeea7fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdeec09e1d0 0x7fdeec138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:42166/0 (socket says 172.21.15.16:42166) 2023-09-30T22:34:51.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.354+0000 7fdeea7fc700 1 -- 172.21.15.16:0/2288679563 learned_addr learned my addr 172.21.15.16:0/2288679563 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:51.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.355+0000 7fdeeb7fe700 1 -- 172.21.15.16:0/2288679563 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdeec09e1d0 msgr2=0x7fdeec138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:51.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.355+0000 7fdeeb7fe700 1 --2- 172.21.15.16:0/2288679563 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdeec09e1d0 0x7fdeec138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:51.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.355+0000 7fdeeb7fe700 1 -- 172.21.15.16:0/2288679563 >> v1:172.21.15.16:6789/0 conn(0x7fdeec09d6f0 legacy=0x7fdeec09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:51.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.355+0000 7fdeeb7fe700 1 -- 172.21.15.16:0/2288679563 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeec09a500 con 0x7fdeec139170 2023-09-30T22:34:51.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.355+0000 7fdeea7fc700 1 --2- 172.21.15.16:0/2288679563 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdeec09e1d0 0x7fdeec138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:51.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.355+0000 7fdeeb7fe700 1 --2- 172.21.15.16:0/2288679563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdeec139170 0x7fdeec13b550 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fdee4009d90 tx=0x7fdee400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=230b59d7f041e609 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:51.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.355+0000 7fdee9ffb700 1 -- 172.21.15.16:0/2288679563 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee400e070 con 0x7fdeec139170 2023-09-30T22:34:51.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.355+0000 7fdee9ffb700 1 -- 172.21.15.16:0/2288679563 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdee400bb20 con 0x7fdeec139170 2023-09-30T22:34:51.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.355+0000 7fdee9ffb700 1 -- 172.21.15.16:0/2288679563 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee4012430 con 0x7fdeec139170 2023-09-30T22:34:51.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.356+0000 7fdef18be700 1 -- 172.21.15.16:0/2288679563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdeec139170 msgr2=0x7fdeec13b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:51.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.356+0000 7fdef18be700 1 --2- 172.21.15.16:0/2288679563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdeec139170 0x7fdeec13b550 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fdee4009d90 tx=0x7fdee400b330 comp rx=0 tx=0).stop 2023-09-30T22:34:51.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.356+0000 7fdef18be700 1 -- 172.21.15.16:0/2288679563 shutdown_connections 2023-09-30T22:34:51.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.356+0000 7fdef18be700 1 --2- 172.21.15.16:0/2288679563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdeec139170 0x7fdeec13b550 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:51.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.356+0000 7fdef18be700 1 --2- 172.21.15.16:0/2288679563 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdeec09e1d0 0x7fdeec138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:51.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.356+0000 7fdef18be700 1 -- 172.21.15.16:0/2288679563 >> 172.21.15.16:0/2288679563 conn(0x7fdeec066f50 msgr2=0x7fdeec09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:51.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.356+0000 7fdef18be700 1 -- 172.21.15.16:0/2288679563 shutdown_connections 2023-09-30T22:34:51.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.356+0000 7fdef18be700 1 -- 172.21.15.16:0/2288679563 wait complete. 2023-09-30T22:34:51.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.357+0000 7fdef18be700 1 Processor -- start 2023-09-30T22:34:51.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.357+0000 7fdef18be700 1 -- start start 2023-09-30T22:34:51.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.358+0000 7fdef18be700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdeec09d6f0 0x7fdeec1c8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:51.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.358+0000 7fdef18be700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdeec09e1d0 0x7fdeec1c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:51.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.358+0000 7fdef18be700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdeec139170 0x7fdeec11e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:51.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.358+0000 7fdef18be700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdeec120560 con 0x7fdeec139170 2023-09-30T22:34:51.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.358+0000 7fdef18be700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdeec120370 con 0x7fdeec09d6f0 2023-09-30T22:34:51.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.358+0000 7fdef18be700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdeec121ad0 con 0x7fdeec09e1d0 2023-09-30T22:34:51.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.358+0000 7fdeeb7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdeec139170 0x7fdeec11e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:51.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.358+0000 7fdeeaffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdeec09d6f0 0x7fdeec1c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:51.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.358+0000 7fdeeb7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdeec139170 0x7fdeec11e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34526/0 (socket says 172.21.15.16:34526) 2023-09-30T22:34:51.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.358+0000 7fdeeb7fe700 1 -- 172.21.15.16:0/2245730226 learned_addr learned my addr 172.21.15.16:0/2245730226 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:51.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.358+0000 7fdeeb7fe700 1 -- 172.21.15.16:0/2245730226 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdeec09e1d0 msgr2=0x7fdeec1c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:51.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.358+0000 7fdeeb7fe700 1 --2- 172.21.15.16:0/2245730226 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdeec09e1d0 0x7fdeec1c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:51.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.358+0000 7fdeeb7fe700 1 -- 172.21.15.16:0/2245730226 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdeec09d6f0 msgr2=0x7fdeec1c8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:51.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.358+0000 7fdeeb7fe700 1 --2- 172.21.15.16:0/2245730226 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdeec09d6f0 0x7fdeec1c8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:51.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.358+0000 7fdeeb7fe700 1 -- 172.21.15.16:0/2245730226 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee0009550 con 0x7fdeec139170 2023-09-30T22:34:51.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.358+0000 7fdeeaffd700 1 --2- 172.21.15.16:0/2245730226 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdeec09d6f0 0x7fdeec1c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:51.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.359+0000 7fdeeb7fe700 1 --2- 172.21.15.16:0/2245730226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdeec139170 0x7fdeec11e8e0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fdee400b680 tx=0x7fdee4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:51.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.359+0000 7fded3fff700 1 -- 172.21.15.16:0/2245730226 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee4012430 con 0x7fdeec139170 2023-09-30T22:34:51.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.359+0000 7fded3fff700 1 -- 172.21.15.16:0/2245730226 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdee4020070 con 0x7fdeec139170 2023-09-30T22:34:51.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.359+0000 7fdef18be700 1 -- 172.21.15.16:0/2245730226 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdee4009a70 con 0x7fdeec139170 2023-09-30T22:34:51.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.359+0000 7fded3fff700 1 -- 172.21.15.16:0/2245730226 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee4004280 con 0x7fdeec139170 2023-09-30T22:34:51.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.359+0000 7fdef18be700 1 -- 172.21.15.16:0/2245730226 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdeec121fa0 con 0x7fdeec139170 2023-09-30T22:34:51.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.360+0000 7fded3fff700 1 -- 172.21.15.16:0/2245730226 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdee40043e0 con 0x7fdeec139170 2023-09-30T22:34:51.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.360+0000 7fded3fff700 1 -- 172.21.15.16:0/2245730226 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fdee4022070 con 0x7fdeec139170 2023-09-30T22:34:51.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.360+0000 7fdef18be700 1 -- 172.21.15.16:0/2245730226 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdeec074ea0 con 0x7fdeec139170 2023-09-30T22:34:51.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.363+0000 7f818b8a2700 1 Processor -- start 2023-09-30T22:34:51.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.363+0000 7f818b8a2700 1 -- start start 2023-09-30T22:34:51.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.364+0000 7f818b8a2700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f818406a770 0x7f818406ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:51.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.364+0000 7f818b8a2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8184131d10 0x7f8184139860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:51.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.364+0000 7f818b8a2700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f818409ece0 con 0x7f818406b080 2023-09-30T22:34:51.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.364+0000 7f818b8a2700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f818409e760 con 0x7f818406a770 2023-09-30T22:34:51.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.364+0000 7f818b8a2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f818409e040 con 0x7f8184131d10 2023-09-30T22:34:51.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.364+0000 7f8189e3f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8184131d10 0x7f8184139860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:51.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.364+0000 7f818963e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f818406a770 0x7f818406ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:51.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.364+0000 7f8188e3d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f818406b080 0x7f8184131600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60904/0 (socket says 172.21.15.16:60904) 2023-09-30T22:34:51.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.364+0000 7f8188e3d700 1 -- 172.21.15.16:0/3579927439 learned_addr learned my addr 172.21.15.16:0/3579927439 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:34:51.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.364+0000 7f818963e700 1 -- 172.21.15.16:0/3579927439 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8184131d10 msgr2=0x7f8184139860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:51.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.364+0000 7f818963e700 1 --2- 172.21.15.16:0/3579927439 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8184131d10 0x7f8184139860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:51.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.364+0000 7f818963e700 1 -- 172.21.15.16:0/3579927439 >> v1:172.21.15.16:6789/0 conn(0x7f818406b080 legacy=0x7f8184131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:51.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.364+0000 7f818963e700 1 -- 172.21.15.16:0/3579927439 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f818409f630 con 0x7f818406a770 2023-09-30T22:34:51.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.364+0000 7f8189e3f700 1 --2- 172.21.15.16:0/3579927439 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8184131d10 0x7f8184139860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:51.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.364+0000 7f818963e700 1 --2- 172.21.15.16:0/3579927439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f818406a770 0x7f818406ab40 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f8180009840 tx=0x7f8180009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b46ddd62600e2c3d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:51.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.365+0000 7f817bfff700 1 -- 172.21.15.16:0/3579927439 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8180011830 con 0x7f818406a770 2023-09-30T22:34:51.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.365+0000 7f817bfff700 1 -- 172.21.15.16:0/3579927439 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8180011990 con 0x7f818406a770 2023-09-30T22:34:51.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.365+0000 7f817bfff700 1 -- 172.21.15.16:0/3579927439 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8180011d20 con 0x7f818406a770 2023-09-30T22:34:51.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.365+0000 7fded3fff700 1 -- 172.21.15.16:0/2245730226 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdee401b400 con 0x7fdeec139170 2023-09-30T22:34:51.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.366+0000 7f818b8a2700 1 -- 172.21.15.16:0/3579927439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f818406a770 msgr2=0x7f818406ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:51.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.366+0000 7f818b8a2700 1 --2- 172.21.15.16:0/3579927439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f818406a770 0x7f818406ab40 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f8180009840 tx=0x7f8180009b50 comp rx=0 tx=0).stop 2023-09-30T22:34:51.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.366+0000 7f818b8a2700 1 -- 172.21.15.16:0/3579927439 shutdown_connections 2023-09-30T22:34:51.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.366+0000 7f818b8a2700 1 --2- 172.21.15.16:0/3579927439 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8184131d10 0x7f8184139860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:51.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.366+0000 7f818b8a2700 1 --2- 172.21.15.16:0/3579927439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f818406a770 0x7f818406ab40 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:51.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.366+0000 7f818b8a2700 1 -- 172.21.15.16:0/3579927439 >> 172.21.15.16:0/3579927439 conn(0x7f8184066f50 msgr2=0x7f818409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:51.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.366+0000 7f818b8a2700 1 -- 172.21.15.16:0/3579927439 shutdown_connections 2023-09-30T22:34:51.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.366+0000 7f818b8a2700 1 -- 172.21.15.16:0/3579927439 wait complete. 2023-09-30T22:34:51.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.367+0000 7f818b8a2700 1 Processor -- start 2023-09-30T22:34:51.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.367+0000 7f818b8a2700 1 -- start start 2023-09-30T22:34:51.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.368+0000 7f818b8a2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f818406a770 0x7f81841c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:51.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.368+0000 7f818b8a2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f818406b080 0x7f81841c8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:51.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.368+0000 7f818b8a2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8184131d10 0x7f81841c8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:51.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.368+0000 7f818b8a2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f818409c010 con 0x7f8184131d10 2023-09-30T22:34:51.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.368+0000 7f818b8a2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f818409a850 con 0x7f818406b080 2023-09-30T22:34:51.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.368+0000 7f818b8a2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8184125440 con 0x7f818406a770 2023-09-30T22:34:51.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.368+0000 7f818963e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f818406a770 0x7f81841c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:51.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.368+0000 7f818963e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f818406a770 0x7f81841c82f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:38888/0 (socket says 172.21.15.16:38888) 2023-09-30T22:34:51.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.368+0000 7f818963e700 1 -- 172.21.15.16:0/3583670495 learned_addr learned my addr 172.21.15.16:0/3583670495 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:51.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.368+0000 7f8188e3d700 1 --2- 172.21.15.16:0/3583670495 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f818406b080 0x7f81841c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:51.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.368+0000 7f8189e3f700 1 --2- 172.21.15.16:0/3583670495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8184131d10 0x7f81841c8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:51.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.368+0000 7f818963e700 1 -- 172.21.15.16:0/3583670495 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f818406b080 msgr2=0x7f81841c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:51.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.368+0000 7f818963e700 1 --2- 172.21.15.16:0/3583670495 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f818406b080 0x7f81841c8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:51.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.368+0000 7f818963e700 1 -- 172.21.15.16:0/3583670495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8184131d10 msgr2=0x7f81841c8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:51.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.368+0000 7f818963e700 1 --2- 172.21.15.16:0/3583670495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8184131d10 0x7f81841c8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:51.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.368+0000 7f818963e700 1 -- 172.21.15.16:0/3583670495 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f817c009a70 con 0x7f818406a770 2023-09-30T22:34:51.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.369+0000 7f818963e700 1 --2- 172.21.15.16:0/3583670495 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f818406a770 0x7f81841c82f0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f8180009fd0 tx=0x7f8180004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:51.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.369+0000 7f8179ffb700 1 -- 172.21.15.16:0/3583670495 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81800045d0 con 0x7f818406a770 2023-09-30T22:34:51.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.369+0000 7f8179ffb700 1 -- 172.21.15.16:0/3583670495 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f818001f070 con 0x7f818406a770 2023-09-30T22:34:51.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.369+0000 7f818b8a2700 1 -- 172.21.15.16:0/3583670495 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81800094a0 con 0x7f818406a770 2023-09-30T22:34:51.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.369+0000 7f8179ffb700 1 -- 172.21.15.16:0/3583670495 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81800038e0 con 0x7f818406a770 2023-09-30T22:34:51.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.369+0000 7f818b8a2700 1 -- 172.21.15.16:0/3583670495 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8184125e30 con 0x7f818406a770 2023-09-30T22:34:51.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.370+0000 7f8179ffb700 1 -- 172.21.15.16:0/3583670495 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8180003a40 con 0x7f818406a770 2023-09-30T22:34:51.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.370+0000 7f8179ffb700 1 -- 172.21.15.16:0/3583670495 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f8180021070 con 0x7f818406a770 2023-09-30T22:34:51.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.374+0000 7f818b8a2700 1 -- 172.21.15.16:0/3583670495 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8184074ea0 con 0x7f818406a770 2023-09-30T22:34:51.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.379+0000 7f8179ffb700 1 -- 172.21.15.16:0/3583670495 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f818002b020 con 0x7f818406a770 2023-09-30T22:34:51.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.550+0000 7f169ccb4700 1 -- 172.21.15.16:0/2691144785 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1698074ea0 con 0x7f1698099a40 2023-09-30T22:34:51.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.550+0000 7f1686ffd700 1 -- 172.21.15.16:0/2691144785 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1688021020 con 0x7f1698099a40 2023-09-30T22:34:51.551 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:34:51.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.552+0000 7f169ccb4700 1 -- 172.21.15.16:0/2691144785 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1698099a40 msgr2=0x7f16981195d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:51.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.552+0000 7f169ccb4700 1 --2- 172.21.15.16:0/2691144785 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1698099a40 0x7f16981195d0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f168800e9e0 tx=0x7f168800ecf0 comp rx=0 tx=0).stop 2023-09-30T22:34:51.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.552+0000 7f169ccb4700 1 -- 172.21.15.16:0/2691144785 shutdown_connections 2023-09-30T22:34:51.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.552+0000 7f169ccb4700 1 --2- 172.21.15.16:0/2691144785 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1698131810 0x7f169811a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:51.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.552+0000 7f169ccb4700 1 --2- 172.21.15.16:0/2691144785 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1698099a40 0x7f16981195d0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:51.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.552+0000 7f169ccb4700 1 --2- 172.21.15.16:0/2691144785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f169809a3b0 0x7f1698119b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:51.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.552+0000 7f169ccb4700 1 -- 172.21.15.16:0/2691144785 >> 172.21.15.16:0/2691144785 conn(0x7f16980a2aa0 msgr2=0x7f169812e560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:51.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.553+0000 7f169ccb4700 1 -- 172.21.15.16:0/2691144785 shutdown_connections 2023-09-30T22:34:51.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.553+0000 7f169ccb4700 1 -- 172.21.15.16:0/2691144785 wait complete. 2023-09-30T22:34:51.567 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:34:51.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.586+0000 7fdef18be700 1 -- 172.21.15.16:0/2245730226 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fdeec0a29d0 con 0x7fdeec139170 2023-09-30T22:34:51.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.587+0000 7fded3fff700 1 -- 172.21.15.16:0/2245730226 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdee402c030 con 0x7fdeec139170 2023-09-30T22:34:51.587 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:34:51.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.589+0000 7fdef18be700 1 -- 172.21.15.16:0/2245730226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdeec139170 msgr2=0x7fdeec11e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:51.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.589+0000 7fdef18be700 1 --2- 172.21.15.16:0/2245730226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdeec139170 0x7fdeec11e8e0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fdee400b680 tx=0x7fdee4004ad0 comp rx=0 tx=0).stop 2023-09-30T22:34:51.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.589+0000 7fdef18be700 1 -- 172.21.15.16:0/2245730226 shutdown_connections 2023-09-30T22:34:51.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.589+0000 7fdef18be700 1 --2- 172.21.15.16:0/2245730226 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdeec09d6f0 0x7fdeec1c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:51.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.589+0000 7fdef18be700 1 --2- 172.21.15.16:0/2245730226 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdeec139170 0x7fdeec11e8e0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:51.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.589+0000 7fdef18be700 1 --2- 172.21.15.16:0/2245730226 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdeec09e1d0 0x7fdeec1c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:51.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.589+0000 7fdef18be700 1 -- 172.21.15.16:0/2245730226 >> 172.21.15.16:0/2245730226 conn(0x7fdeec066f50 msgr2=0x7fdeec0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:51.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.589+0000 7fdef18be700 1 -- 172.21.15.16:0/2245730226 shutdown_connections 2023-09-30T22:34:51.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.590+0000 7fdef18be700 1 -- 172.21.15.16:0/2245730226 wait complete. 2023-09-30T22:34:51.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.600+0000 7f818b8a2700 1 -- 172.21.15.16:0/3583670495 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f818406ed90 con 0x7f818406a770 2023-09-30T22:34:51.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.601+0000 7f8179ffb700 1 -- 172.21.15.16:0/3583670495 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8180015070 con 0x7f818406a770 2023-09-30T22:34:51.601 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:34:51.604 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:34:51.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.603+0000 7f818b8a2700 1 -- 172.21.15.16:0/3583670495 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f818406a770 msgr2=0x7f81841c82f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:51.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.603+0000 7f818b8a2700 1 --2- 172.21.15.16:0/3583670495 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f818406a770 0x7f81841c82f0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f8180009fd0 tx=0x7f8180004750 comp rx=0 tx=0).stop 2023-09-30T22:34:51.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.603+0000 7f818b8a2700 1 -- 172.21.15.16:0/3583670495 shutdown_connections 2023-09-30T22:34:51.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.603+0000 7f818b8a2700 1 --2- 172.21.15.16:0/3583670495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8184131d10 0x7f81841c8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:51.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.603+0000 7f818b8a2700 1 --2- 172.21.15.16:0/3583670495 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f818406a770 0x7f81841c82f0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:51.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.603+0000 7f818b8a2700 1 --2- 172.21.15.16:0/3583670495 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f818406b080 0x7f81841c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:51.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.603+0000 7f818b8a2700 1 -- 172.21.15.16:0/3583670495 >> 172.21.15.16:0/3583670495 conn(0x7f8184066f50 msgr2=0x7f8184070110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:51.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.603+0000 7f818b8a2700 1 -- 172.21.15.16:0/3583670495 shutdown_connections 2023-09-30T22:34:51.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:51.603+0000 7f818b8a2700 1 -- 172.21.15.16:0/3583670495 wait complete. 2023-09-30T22:34:51.617 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:34:52.245 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:34:52.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.390+0000 7f5d29a7d700 1 Processor -- start 2023-09-30T22:34:52.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.390+0000 7f5d29a7d700 1 -- start start 2023-09-30T22:34:52.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.391+0000 7f5d29a7d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d2409e1d0 0x7f5d24138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:52.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.391+0000 7f5d29a7d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5d24139170 0x7f5d2413b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:52.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.391+0000 7f5d29a7d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d2409b120 con 0x7f5d2409d6f0 2023-09-30T22:34:52.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.391+0000 7f5d29a7d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d2406b460 con 0x7f5d24139170 2023-09-30T22:34:52.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.391+0000 7f5d29a7d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2409a7e0 con 0x7f5d2409e1d0 2023-09-30T22:34:52.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.391+0000 7f5d23fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5d24139170 0x7f5d2413b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:52.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.391+0000 7f5d22ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d2409e1d0 0x7f5d24138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:52.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.391+0000 7f5d237fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5d2409d6f0 0x7f5d2409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60908/0 (socket says 172.21.15.16:60908) 2023-09-30T22:34:52.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.391+0000 7f5d237fe700 1 -- 172.21.15.16:0/3314945546 learned_addr learned my addr 172.21.15.16:0/3314945546 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:34:52.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.391+0000 7f5d22ffd700 1 -- 172.21.15.16:0/3314945546 >> v2:172.21.15.16:3300/0 conn(0x7f5d24139170 msgr2=0x7f5d2413b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:52.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.391+0000 7f5d22ffd700 1 --2- 172.21.15.16:0/3314945546 >> v2:172.21.15.16:3300/0 conn(0x7f5d24139170 0x7f5d2413b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:52.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.391+0000 7f5d22ffd700 1 -- 172.21.15.16:0/3314945546 >> v1:172.21.15.16:6789/0 conn(0x7f5d2409d6f0 legacy=0x7f5d2409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:52.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.391+0000 7f5d227fc700 1 -- 172.21.15.16:0/3314945546 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3948178640 0 0) 0x7f5d2409b120 con 0x7f5d2409d6f0 2023-09-30T22:34:52.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.391+0000 7f5d22ffd700 1 -- 172.21.15.16:0/3314945546 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d2409a500 con 0x7f5d2409e1d0 2023-09-30T22:34:52.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.391+0000 7f5d23fff700 1 --2- 172.21.15.16:0/3314945546 >> v2:172.21.15.16:3300/0 conn(0x7f5d24139170 0x7f5d2413b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:52.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.391+0000 7f5d22ffd700 1 --2- 172.21.15.16:0/3314945546 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d2409e1d0 0x7f5d24138c30 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c009870 tx=0x7f5d0c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a990ec8bacf27df2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:52.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.392+0000 7f5d227fc700 1 -- 172.21.15.16:0/3314945546 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c011830 con 0x7f5d2409e1d0 2023-09-30T22:34:52.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.392+0000 7f5d227fc700 1 -- 172.21.15.16:0/3314945546 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d0c011990 con 0x7f5d2409e1d0 2023-09-30T22:34:52.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.392+0000 7f5d227fc700 1 -- 172.21.15.16:0/3314945546 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c011d20 con 0x7f5d2409e1d0 2023-09-30T22:34:52.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.392+0000 7f5d29a7d700 1 -- 172.21.15.16:0/3314945546 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d2409e1d0 msgr2=0x7f5d24138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:52.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.392+0000 7f5d29a7d700 1 --2- 172.21.15.16:0/3314945546 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d2409e1d0 0x7f5d24138c30 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c009870 tx=0x7f5d0c009b80 comp rx=0 tx=0).stop 2023-09-30T22:34:52.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.392+0000 7f5d29a7d700 1 -- 172.21.15.16:0/3314945546 shutdown_connections 2023-09-30T22:34:52.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.392+0000 7f5d29a7d700 1 --2- 172.21.15.16:0/3314945546 >> v2:172.21.15.16:3300/0 conn(0x7f5d24139170 0x7f5d2413b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:52.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.392+0000 7f5d29a7d700 1 --2- 172.21.15.16:0/3314945546 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d2409e1d0 0x7f5d24138c30 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:52.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.392+0000 7f5d29a7d700 1 -- 172.21.15.16:0/3314945546 >> 172.21.15.16:0/3314945546 conn(0x7f5d24066f50 msgr2=0x7f5d2409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:52.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.392+0000 7f5d29a7d700 1 -- 172.21.15.16:0/3314945546 shutdown_connections 2023-09-30T22:34:52.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.392+0000 7f5d29a7d700 1 -- 172.21.15.16:0/3314945546 wait complete. 2023-09-30T22:34:52.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.393+0000 7f5d29a7d700 1 Processor -- start 2023-09-30T22:34:52.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.394+0000 7f5d29a7d700 1 -- start start 2023-09-30T22:34:52.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.394+0000 7f5d29a7d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d2409d6f0 0x7f5d2411e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:52.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.394+0000 7f5d29a7d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d2409e1d0 0x7f5d2411eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:52.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.394+0000 7f5d29a7d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d24139170 0x7f5d2411f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:52.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.394+0000 7f5d29a7d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d24072110 con 0x7f5d2409e1d0 2023-09-30T22:34:52.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.394+0000 7f5d29a7d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d24122100 con 0x7f5d24139170 2023-09-30T22:34:52.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.394+0000 7f5d29a7d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5d24120a80 con 0x7f5d2409d6f0 2023-09-30T22:34:52.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.394+0000 7f5d22ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d2409e1d0 0x7f5d2411eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:52.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.394+0000 7f5d22ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d2409e1d0 0x7f5d2411eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34544/0 (socket says 172.21.15.16:34544) 2023-09-30T22:34:52.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.394+0000 7f5d22ffd700 1 -- 172.21.15.16:0/643123607 learned_addr learned my addr 172.21.15.16:0/643123607 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:52.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.394+0000 7f5d237fe700 1 --2- 172.21.15.16:0/643123607 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d2409d6f0 0x7f5d2411e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:52.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.394+0000 7f5d23fff700 1 --2- 172.21.15.16:0/643123607 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d24139170 0x7f5d2411f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:52.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.395+0000 7f5d237fe700 1 -- 172.21.15.16:0/643123607 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d24139170 msgr2=0x7f5d2411f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:52.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.395+0000 7f5d237fe700 1 --2- 172.21.15.16:0/643123607 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d24139170 0x7f5d2411f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:52.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.395+0000 7f5d237fe700 1 -- 172.21.15.16:0/643123607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d2409e1d0 msgr2=0x7f5d2411eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:52.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.395+0000 7f5d237fe700 1 --2- 172.21.15.16:0/643123607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d2409e1d0 0x7f5d2411eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:52.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.395+0000 7f5d237fe700 1 -- 172.21.15.16:0/643123607 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d1c009a70 con 0x7f5d2409d6f0 2023-09-30T22:34:52.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.395+0000 7f5d22ffd700 1 --2- 172.21.15.16:0/643123607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d2409e1d0 0x7f5d2411eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:52.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.395+0000 7f5d23fff700 1 --2- 172.21.15.16:0/643123607 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d24139170 0x7f5d2411f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:52.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.395+0000 7f5d237fe700 1 --2- 172.21.15.16:0/643123607 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d2409d6f0 0x7f5d2411e960 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f5d1400c470 tx=0x7f5d1400c780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:52.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.395+0000 7f5d1bfff700 1 -- 172.21.15.16:0/643123607 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d14004c80 con 0x7f5d2409d6f0 2023-09-30T22:34:52.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.395+0000 7f5d1bfff700 1 -- 172.21.15.16:0/643123607 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d14004de0 con 0x7f5d2409d6f0 2023-09-30T22:34:52.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.395+0000 7f5d29a7d700 1 -- 172.21.15.16:0/643123607 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d0c0094a0 con 0x7f5d2409d6f0 2023-09-30T22:34:52.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.395+0000 7f5d1bfff700 1 -- 172.21.15.16:0/643123607 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d14007310 con 0x7f5d2409d6f0 2023-09-30T22:34:52.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.395+0000 7f5d29a7d700 1 -- 172.21.15.16:0/643123607 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d24120cc0 con 0x7f5d2409d6f0 2023-09-30T22:34:52.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.396+0000 7f5d1bfff700 1 -- 172.21.15.16:0/643123607 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5d14007470 con 0x7f5d2409d6f0 2023-09-30T22:34:52.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.396+0000 7f5d29a7d700 1 -- 172.21.15.16:0/643123607 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d24134810 con 0x7f5d2409d6f0 2023-09-30T22:34:52.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.396+0000 7f5d1bfff700 1 -- 172.21.15.16:0/643123607 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f5d1403f260 con 0x7f5d2409d6f0 2023-09-30T22:34:52.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.401+0000 7f5d1bfff700 1 -- 172.21.15.16:0/643123607 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5d14007720 con 0x7f5d2409d6f0 2023-09-30T22:34:52.568 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:34:52.604 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:34:52.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.607+0000 7f5d29a7d700 1 -- 172.21.15.16:0/643123607 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5d24074ea0 con 0x7f5d2409d6f0 2023-09-30T22:34:52.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.608+0000 7f5d1bfff700 1 -- 172.21.15.16:0/643123607 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5d14011360 con 0x7f5d2409d6f0 2023-09-30T22:34:52.608 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:34:52.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.611+0000 7f5d29a7d700 1 -- 172.21.15.16:0/643123607 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d2409d6f0 msgr2=0x7f5d2411e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:52.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.611+0000 7f5d29a7d700 1 --2- 172.21.15.16:0/643123607 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d2409d6f0 0x7f5d2411e960 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f5d1400c470 tx=0x7f5d1400c780 comp rx=0 tx=0).stop 2023-09-30T22:34:52.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.611+0000 7f5d29a7d700 1 -- 172.21.15.16:0/643123607 shutdown_connections 2023-09-30T22:34:52.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.611+0000 7f5d29a7d700 1 --2- 172.21.15.16:0/643123607 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d24139170 0x7f5d2411f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:52.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.611+0000 7f5d29a7d700 1 --2- 172.21.15.16:0/643123607 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d2409d6f0 0x7f5d2411e960 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:52.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.611+0000 7f5d29a7d700 1 --2- 172.21.15.16:0/643123607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d2409e1d0 0x7f5d2411eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:52.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.611+0000 7f5d29a7d700 1 -- 172.21.15.16:0/643123607 >> 172.21.15.16:0/643123607 conn(0x7f5d24066f50 msgr2=0x7f5d24136b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:52.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.611+0000 7f5d29a7d700 1 -- 172.21.15.16:0/643123607 shutdown_connections 2023-09-30T22:34:52.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.611+0000 7f5d29a7d700 1 -- 172.21.15.16:0/643123607 wait complete. 2023-09-30T22:34:52.618 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:34:52.628 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:34:52.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.717+0000 7fcafdbe5700 1 Processor -- start 2023-09-30T22:34:52.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.718+0000 7fcafdbe5700 1 -- start start 2023-09-30T22:34:52.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.718+0000 7fcafdbe5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcaf812cf60 0x7fcaf806f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:52.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.718+0000 7fcafdbe5700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcaf812f8b0 0x7fcaf812fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:52.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.718+0000 7fcafdbe5700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaf812cd90 con 0x7fcaf8130260 2023-09-30T22:34:52.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.718+0000 7fcafdbe5700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaf812d330 con 0x7fcaf812f8b0 2023-09-30T22:34:52.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.718+0000 7fcafdbe5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf81306e0 con 0x7fcaf812cf60 2023-09-30T22:34:52.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.719+0000 7fcaf77fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcaf812cf60 0x7fcaf806f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:52.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.719+0000 7fcaf77fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcaf812cf60 0x7fcaf806f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:42210/0 (socket says 172.21.15.16:42210) 2023-09-30T22:34:52.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.719+0000 7fcaf77fe700 1 -- 172.21.15.16:0/2161277804 learned_addr learned my addr 172.21.15.16:0/2161277804 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:52.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.719+0000 7fcaeffff700 1 --2- 172.21.15.16:0/2161277804 >> v2:172.21.15.16:3300/0 conn(0x7fcaf812f8b0 0x7fcaf812fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:52.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.719+0000 7fcaeffff700 1 -- 172.21.15.16:0/2161277804 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcaf812cf60 msgr2=0x7fcaf806f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:52.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.719+0000 7fcaeffff700 1 --2- 172.21.15.16:0/2161277804 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcaf812cf60 0x7fcaf806f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:52.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.719+0000 7fcaeffff700 1 -- 172.21.15.16:0/2161277804 >> v1:172.21.15.16:6789/0 conn(0x7fcaf8130260 legacy=0x7fcaf8136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:52.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.719+0000 7fcaeffff700 1 -- 172.21.15.16:0/2161277804 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf8071780 con 0x7fcaf812f8b0 2023-09-30T22:34:52.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.719+0000 7fcaf77fe700 1 --2- 172.21.15.16:0/2161277804 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcaf812cf60 0x7fcaf806f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:52.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.719+0000 7fcaf6ffd700 1 -- 172.21.15.16:0/2161277804 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3948413584 0 0) 0x7fcaf812cd90 con 0x7fcaf8130260 2023-09-30T22:34:52.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.719+0000 7fcaeffff700 1 --2- 172.21.15.16:0/2161277804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcaf812f8b0 0x7fcaf812fd20 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fcae000dc60 tx=0x7fcae000df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6229b865b999065 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:52.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.719+0000 7fcaf6ffd700 1 -- 172.21.15.16:0/2161277804 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae0012070 con 0x7fcaf812f8b0 2023-09-30T22:34:52.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.719+0000 7fcaf6ffd700 1 -- 172.21.15.16:0/2161277804 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcae000d2e0 con 0x7fcaf812f8b0 2023-09-30T22:34:52.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.719+0000 7fcaf6ffd700 1 -- 172.21.15.16:0/2161277804 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae000f900 con 0x7fcaf812f8b0 2023-09-30T22:34:52.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.721+0000 7fcafdbe5700 1 -- 172.21.15.16:0/2161277804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcaf812f8b0 msgr2=0x7fcaf812fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:52.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.721+0000 7fcafdbe5700 1 --2- 172.21.15.16:0/2161277804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcaf812f8b0 0x7fcaf812fd20 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fcae000dc60 tx=0x7fcae000df70 comp rx=0 tx=0).stop 2023-09-30T22:34:52.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.721+0000 7fcafdbe5700 1 -- 172.21.15.16:0/2161277804 shutdown_connections 2023-09-30T22:34:52.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.721+0000 7fcafdbe5700 1 --2- 172.21.15.16:0/2161277804 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcaf812cf60 0x7fcaf806f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:52.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.721+0000 7fcafdbe5700 1 --2- 172.21.15.16:0/2161277804 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcaf812f8b0 0x7fcaf812fd20 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:52.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.721+0000 7fcafdbe5700 1 -- 172.21.15.16:0/2161277804 >> 172.21.15.16:0/2161277804 conn(0x7fcaf80a2aa0 msgr2=0x7fcaf80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:52.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.721+0000 7fcafdbe5700 1 -- 172.21.15.16:0/2161277804 shutdown_connections 2023-09-30T22:34:52.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.721+0000 7fcafdbe5700 1 -- 172.21.15.16:0/2161277804 wait complete. 2023-09-30T22:34:52.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.722+0000 7fcafdbe5700 1 Processor -- start 2023-09-30T22:34:52.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.722+0000 7fcafdbe5700 1 -- start start 2023-09-30T22:34:52.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.722+0000 7fcafdbe5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcaf812cf60 0x7fcaf809bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:52.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.722+0000 7fcafdbe5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcaf812f8b0 0x7fcaf809c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:52.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.722+0000 7fcafdbe5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcaf8130260 0x7fcaf81c3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:52.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.722+0000 7fcafdbe5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf8129c20 con 0x7fcaf812cf60 2023-09-30T22:34:52.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.722+0000 7fcafdbe5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf809e880 con 0x7fcaf812f8b0 2023-09-30T22:34:52.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.722+0000 7fcafdbe5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf81299c0 con 0x7fcaf8130260 2023-09-30T22:34:52.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.722+0000 7fcaeffff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcaf812f8b0 0x7fcaf809c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:52.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.722+0000 7fcaeffff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcaf812f8b0 0x7fcaf809c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:42212/0 (socket says 172.21.15.16:42212) 2023-09-30T22:34:52.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.722+0000 7fcaeffff700 1 -- 172.21.15.16:0/1955846379 learned_addr learned my addr 172.21.15.16:0/1955846379 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:52.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.722+0000 7fcaf7fff700 1 --2- 172.21.15.16:0/1955846379 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcaf8130260 0x7fcaf81c3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:52.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.723+0000 7fcaf77fe700 1 --2- 172.21.15.16:0/1955846379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcaf812cf60 0x7fcaf809bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:52.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.723+0000 7fcaf7fff700 1 -- 172.21.15.16:0/1955846379 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcaf812f8b0 msgr2=0x7fcaf809c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:52.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.723+0000 7fcaf7fff700 1 --2- 172.21.15.16:0/1955846379 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcaf812f8b0 0x7fcaf809c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:52.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.723+0000 7fcaf7fff700 1 -- 172.21.15.16:0/1955846379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcaf812cf60 msgr2=0x7fcaf809bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:52.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.723+0000 7fcaf7fff700 1 --2- 172.21.15.16:0/1955846379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcaf812cf60 0x7fcaf809bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:52.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.723+0000 7fcaf7fff700 1 -- 172.21.15.16:0/1955846379 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcae8009550 con 0x7fcaf8130260 2023-09-30T22:34:52.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.723+0000 7fcaf77fe700 1 --2- 172.21.15.16:0/1955846379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcaf812cf60 0x7fcaf809bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:34:52.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.723+0000 7fcaeffff700 1 --2- 172.21.15.16:0/1955846379 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcaf812f8b0 0x7fcaf809c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:52.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.723+0000 7fcaf7fff700 1 --2- 172.21.15.16:0/1955846379 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcaf8130260 0x7fcaf81c3f50 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fcaf000c7c0 tx=0x7fcaf000cad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:52.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.723+0000 7fcaf4ff9700 1 -- 172.21.15.16:0/1955846379 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf0003e30 con 0x7fcaf8130260 2023-09-30T22:34:52.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.723+0000 7fcaf4ff9700 1 -- 172.21.15.16:0/1955846379 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaf0005070 con 0x7fcaf8130260 2023-09-30T22:34:52.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.723+0000 7fcaf4ff9700 1 -- 172.21.15.16:0/1955846379 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf0005420 con 0x7fcaf8130260 2023-09-30T22:34:52.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.723+0000 7fcafdbe5700 1 -- 172.21.15.16:0/1955846379 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcae000d940 con 0x7fcaf8130260 2023-09-30T22:34:52.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.723+0000 7fcafdbe5700 1 -- 172.21.15.16:0/1955846379 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaf809b780 con 0x7fcaf8130260 2023-09-30T22:34:52.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.724+0000 7fcaf4ff9700 1 -- 172.21.15.16:0/1955846379 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcaf0005580 con 0x7fcaf8130260 2023-09-30T22:34:52.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.724+0000 7fcaf4ff9700 1 -- 172.21.15.16:0/1955846379 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fcaf003ecc0 con 0x7fcaf8130260 2023-09-30T22:34:52.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.725+0000 7fcafdbe5700 1 -- 172.21.15.16:0/1955846379 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaf812d9d0 con 0x7fcaf8130260 2023-09-30T22:34:52.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.734+0000 7fcaf4ff9700 1 -- 172.21.15.16:0/1955846379 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcaf0017070 con 0x7fcaf8130260 2023-09-30T22:34:52.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.757+0000 7f8a0453a700 1 Processor -- start 2023-09-30T22:34:52.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.757+0000 7f8a0453a700 1 -- start start 2023-09-30T22:34:52.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.758+0000 7f8a0453a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f89fc06a770 0x7f89fc06ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:52.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.758+0000 7f8a0453a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f89fc131d10 0x7f89fc139860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:52.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.758+0000 7f8a0453a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89fc09ece0 con 0x7f89fc06b080 2023-09-30T22:34:52.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.758+0000 7f8a0453a700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89fc09e760 con 0x7f89fc131d10 2023-09-30T22:34:52.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.758+0000 7f8a0453a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc09e040 con 0x7f89fc06a770 2023-09-30T22:34:52.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.758+0000 7f8a02ad7700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f89fc131d10 0x7f89fc139860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:52.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.758+0000 7f8a022d6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f89fc06a770 0x7f89fc06ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:52.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.758+0000 7f8a01ad5700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f89fc06b080 0x7f89fc131600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60932/0 (socket says 172.21.15.16:60932) 2023-09-30T22:34:52.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.758+0000 7f8a01ad5700 1 -- 172.21.15.16:0/484765779 learned_addr learned my addr 172.21.15.16:0/484765779 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:34:52.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.758+0000 7f8a022d6700 1 -- 172.21.15.16:0/484765779 >> v2:172.21.15.16:3300/0 conn(0x7f89fc131d10 msgr2=0x7f89fc139860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:52.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.758+0000 7f8a022d6700 1 --2- 172.21.15.16:0/484765779 >> v2:172.21.15.16:3300/0 conn(0x7f89fc131d10 0x7f89fc139860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:52.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.758+0000 7f8a022d6700 1 -- 172.21.15.16:0/484765779 >> v1:172.21.15.16:6789/0 conn(0x7f89fc06b080 legacy=0x7f89fc131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:52.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.758+0000 7f8a012d4700 1 -- 172.21.15.16:0/484765779 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3665386197 0 0) 0x7f89fc09ece0 con 0x7f89fc06b080 2023-09-30T22:34:52.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.758+0000 7f8a022d6700 1 -- 172.21.15.16:0/484765779 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89fc09f630 con 0x7f89fc06a770 2023-09-30T22:34:52.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.758+0000 7f8a02ad7700 1 --2- 172.21.15.16:0/484765779 >> v2:172.21.15.16:3300/0 conn(0x7f89fc131d10 0x7f89fc139860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:52.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.759+0000 7f8a022d6700 1 --2- 172.21.15.16:0/484765779 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f89fc06a770 0x7f89fc06ab40 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f89f8009840 tx=0x7f89f8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afa2383747ab67e4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:52.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.759+0000 7f8a012d4700 1 -- 172.21.15.16:0/484765779 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f8011830 con 0x7f89fc06a770 2023-09-30T22:34:52.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.759+0000 7f8a012d4700 1 -- 172.21.15.16:0/484765779 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89f8011990 con 0x7f89fc06a770 2023-09-30T22:34:52.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.759+0000 7f8a012d4700 1 -- 172.21.15.16:0/484765779 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f8011d20 con 0x7f89fc06a770 2023-09-30T22:34:52.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.759+0000 7f8a0453a700 1 -- 172.21.15.16:0/484765779 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f89fc06a770 msgr2=0x7f89fc06ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:52.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.759+0000 7f8a0453a700 1 --2- 172.21.15.16:0/484765779 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f89fc06a770 0x7f89fc06ab40 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f89f8009840 tx=0x7f89f8009b50 comp rx=0 tx=0).stop 2023-09-30T22:34:52.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.760+0000 7f8a0453a700 1 -- 172.21.15.16:0/484765779 shutdown_connections 2023-09-30T22:34:52.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.760+0000 7f8a0453a700 1 --2- 172.21.15.16:0/484765779 >> v2:172.21.15.16:3300/0 conn(0x7f89fc131d10 0x7f89fc139860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:52.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.760+0000 7f8a0453a700 1 --2- 172.21.15.16:0/484765779 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f89fc06a770 0x7f89fc06ab40 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:52.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.760+0000 7f8a0453a700 1 -- 172.21.15.16:0/484765779 >> 172.21.15.16:0/484765779 conn(0x7f89fc066f50 msgr2=0x7f89fc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:52.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.760+0000 7f8a0453a700 1 -- 172.21.15.16:0/484765779 shutdown_connections 2023-09-30T22:34:52.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.760+0000 7f8a0453a700 1 -- 172.21.15.16:0/484765779 wait complete. 2023-09-30T22:34:52.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.761+0000 7f8a0453a700 1 Processor -- start 2023-09-30T22:34:52.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.761+0000 7f8a0453a700 1 -- start start 2023-09-30T22:34:52.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.762+0000 7f8a0453a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f89fc06a770 0x7f89fc09abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:52.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.762+0000 7f8a0453a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89fc06b080 0x7f89fc09b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:52.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.762+0000 7f8a0453a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f89fc131d10 0x7f89fc1c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:52.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.762+0000 7f8a0453a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc1311d0 con 0x7f89fc06b080 2023-09-30T22:34:52.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.762+0000 7f8a0453a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc071e60 con 0x7f89fc131d10 2023-09-30T22:34:52.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.762+0000 7f8a0453a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc124ed0 con 0x7f89fc06a770 2023-09-30T22:34:52.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.762+0000 7f8a02ad7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f89fc131d10 0x7f89fc1c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:52.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.762+0000 7f8a02ad7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f89fc131d10 0x7f89fc1c8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:42226/0 (socket says 172.21.15.16:42226) 2023-09-30T22:34:52.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.762+0000 7f8a01ad5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89fc06b080 0x7f89fc09b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:52.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.762+0000 7f8a02ad7700 1 -- 172.21.15.16:0/4026136609 learned_addr learned my addr 172.21.15.16:0/4026136609 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:52.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.762+0000 7f8a01ad5700 1 -- 172.21.15.16:0/4026136609 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f89fc06a770 msgr2=0x7f89fc09abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:52.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.762+0000 7f8a01ad5700 1 --2- 172.21.15.16:0/4026136609 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f89fc06a770 0x7f89fc09abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:52.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.763+0000 7f8a01ad5700 1 -- 172.21.15.16:0/4026136609 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f89fc131d10 msgr2=0x7f89fc1c8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:52.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.763+0000 7f8a01ad5700 1 --2- 172.21.15.16:0/4026136609 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f89fc131d10 0x7f89fc1c8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:52.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.763+0000 7f8a01ad5700 1 -- 172.21.15.16:0/4026136609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f4009a70 con 0x7f89fc06b080 2023-09-30T22:34:52.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.763+0000 7f8a01ad5700 1 --2- 172.21.15.16:0/4026136609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89fc06b080 0x7f89fc09b0f0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f89fc09f8c0 tx=0x7f89ec00c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:52.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.763+0000 7f89f2ffd700 1 -- 172.21.15.16:0/4026136609 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ec004c80 con 0x7f89fc06b080 2023-09-30T22:34:52.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.763+0000 7f89f2ffd700 1 -- 172.21.15.16:0/4026136609 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89ec004de0 con 0x7f89fc06b080 2023-09-30T22:34:52.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.763+0000 7f8a0453a700 1 -- 172.21.15.16:0/4026136609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89f80094a0 con 0x7f89fc06b080 2023-09-30T22:34:52.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.763+0000 7f89f2ffd700 1 -- 172.21.15.16:0/4026136609 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89ec007310 con 0x7f89fc06b080 2023-09-30T22:34:52.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.763+0000 7f8a0453a700 1 -- 172.21.15.16:0/4026136609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89fc125c30 con 0x7f89fc06b080 2023-09-30T22:34:52.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.764+0000 7f89f2ffd700 1 -- 172.21.15.16:0/4026136609 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f89ec007470 con 0x7f89fc06b080 2023-09-30T22:34:52.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.764+0000 7f89f2ffd700 1 -- 172.21.15.16:0/4026136609 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f89ec03f060 con 0x7f89fc06b080 2023-09-30T22:34:52.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.764+0000 7f8a0453a700 1 -- 172.21.15.16:0/4026136609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89fc131580 con 0x7f89fc06b080 2023-09-30T22:34:52.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.768+0000 7fd1e08ce700 1 Processor -- start 2023-09-30T22:34:52.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.768+0000 7fd1e08ce700 1 -- start start 2023-09-30T22:34:52.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.769+0000 7fd1e08ce700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd1dc06f5d0 0x7fd1dc12ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:52.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.769+0000 7fd1e08ce700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd1dc135c20 0x7fd1dc138000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:52.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.769+0000 7fd1e08ce700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1dc06fa40 con 0x7fd1dc12cf60 2023-09-30T22:34:52.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.769+0000 7fd1e08ce700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1dc12cd90 con 0x7fd1dc06f5d0 2023-09-30T22:34:52.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.769+0000 7fd1e08ce700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1dc071780 con 0x7fd1dc135c20 2023-09-30T22:34:52.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.769+0000 7fd1dad9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd1dc135c20 0x7fd1dc138000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:52.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.769+0000 7fd1d9d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd1dc06f5d0 0x7fd1dc12ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:52.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.769+0000 7fd1dad9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd1dc135c20 0x7fd1dc138000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:42242/0 (socket says 172.21.15.16:42242) 2023-09-30T22:34:52.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.769+0000 7fd1dad9d700 1 -- 172.21.15.16:0/805418931 learned_addr learned my addr 172.21.15.16:0/805418931 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:52.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.769+0000 7fd1da59c700 1 --1- 172.21.15.16:0/805418931 >> v1:172.21.15.16:6789/0 conn(0x7fd1dc12cf60 0x7fd1dc12f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60934/0 (socket says 172.21.15.16:60934) 2023-09-30T22:34:52.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.769+0000 7fd1d9d9b700 1 --2- 172.21.15.16:0/805418931 >> v2:172.21.15.16:3300/0 conn(0x7fd1dc06f5d0 0x7fd1dc12ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34586/0 (socket says 172.21.15.16:34586) 2023-09-30T22:34:52.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.769+0000 7f89f2ffd700 1 -- 172.21.15.16:0/4026136609 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f89ec007720 con 0x7f89fc06b080 2023-09-30T22:34:52.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.769+0000 7fd1dad9d700 1 -- 172.21.15.16:0/805418931 >> v2:172.21.15.16:3300/0 conn(0x7fd1dc06f5d0 msgr2=0x7fd1dc12ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:52.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.769+0000 7fd1dad9d700 1 --2- 172.21.15.16:0/805418931 >> v2:172.21.15.16:3300/0 conn(0x7fd1dc06f5d0 0x7fd1dc12ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:52.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.769+0000 7fd1dad9d700 1 -- 172.21.15.16:0/805418931 >> v1:172.21.15.16:6789/0 conn(0x7fd1dc12cf60 legacy=0x7fd1dc12f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:52.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.769+0000 7fd1d959a700 1 -- 172.21.15.16:0/805418931 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 569341462 0 0) 0x7fd1dc06fa40 con 0x7fd1dc12cf60 2023-09-30T22:34:52.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.769+0000 7fd1dad9d700 1 -- 172.21.15.16:0/805418931 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1dc071a40 con 0x7fd1dc135c20 2023-09-30T22:34:52.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.769+0000 7fd1d9d9b700 1 --2- 172.21.15.16:0/805418931 >> v2:172.21.15.16:3300/0 conn(0x7fd1dc06f5d0 0x7fd1dc12ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:52.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.770+0000 7fd1dad9d700 1 --2- 172.21.15.16:0/805418931 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd1dc135c20 0x7fd1dc138000 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fd1d400bef0 tx=0x7fd1d400a4c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3af5d097d4ccd3b7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:52.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.770+0000 7fd1d959a700 1 -- 172.21.15.16:0/805418931 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d400a2e0 con 0x7fd1dc135c20 2023-09-30T22:34:52.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.770+0000 7fd1d959a700 1 -- 172.21.15.16:0/805418931 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1d400cb50 con 0x7fd1dc135c20 2023-09-30T22:34:52.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.770+0000 7fd1d959a700 1 -- 172.21.15.16:0/805418931 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d4013420 con 0x7fd1dc135c20 2023-09-30T22:34:52.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.771+0000 7fd1e08ce700 1 -- 172.21.15.16:0/805418931 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd1dc135c20 msgr2=0x7fd1dc138000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:52.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.771+0000 7fd1e08ce700 1 --2- 172.21.15.16:0/805418931 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd1dc135c20 0x7fd1dc138000 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fd1d400bef0 tx=0x7fd1d400a4c0 comp rx=0 tx=0).stop 2023-09-30T22:34:52.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.771+0000 7fd1e08ce700 1 -- 172.21.15.16:0/805418931 shutdown_connections 2023-09-30T22:34:52.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.771+0000 7fd1e08ce700 1 --2- 172.21.15.16:0/805418931 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd1dc135c20 0x7fd1dc138000 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:52.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.771+0000 7fd1e08ce700 1 --2- 172.21.15.16:0/805418931 >> v2:172.21.15.16:3300/0 conn(0x7fd1dc06f5d0 0x7fd1dc12ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:52.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.771+0000 7fd1e08ce700 1 -- 172.21.15.16:0/805418931 >> 172.21.15.16:0/805418931 conn(0x7fd1dc0a2aa0 msgr2=0x7fd1dc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:52.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.771+0000 7fd1e08ce700 1 -- 172.21.15.16:0/805418931 shutdown_connections 2023-09-30T22:34:52.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.771+0000 7fd1e08ce700 1 -- 172.21.15.16:0/805418931 wait complete. 2023-09-30T22:34:52.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.772+0000 7fd1e08ce700 1 Processor -- start 2023-09-30T22:34:52.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.772+0000 7fd1e08ce700 1 -- start start 2023-09-30T22:34:52.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.773+0000 7fd1e08ce700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1dc06f5d0 0x7fd1dc127200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:52.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.773+0000 7fd1e08ce700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd1dc12cf60 0x7fd1dc127740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:52.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.773+0000 7fd1e08ce700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd1dc135c20 0x7fd1dc09c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:52.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.773+0000 7fd1e08ce700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1dc131800 con 0x7fd1dc06f5d0 2023-09-30T22:34:52.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.773+0000 7fd1e08ce700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1dc0a1600 con 0x7fd1dc12cf60 2023-09-30T22:34:52.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.773+0000 7fd1e08ce700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd1dc129fa0 con 0x7fd1dc135c20 2023-09-30T22:34:52.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.773+0000 7fd1da59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1dc06f5d0 0x7fd1dc127200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:52.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.773+0000 7fd1d9d9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd1dc12cf60 0x7fd1dc127740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:52.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.773+0000 7fd1dad9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd1dc135c20 0x7fd1dc09c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:52.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.773+0000 7fd1d9d9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd1dc12cf60 0x7fd1dc127740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:42244/0 (socket says 172.21.15.16:42244) 2023-09-30T22:34:52.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.773+0000 7fd1da59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1dc06f5d0 0x7fd1dc127200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34588/0 (socket says 172.21.15.16:34588) 2023-09-30T22:34:52.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.773+0000 7fd1d9d9b700 1 -- 172.21.15.16:0/1075159524 learned_addr learned my addr 172.21.15.16:0/1075159524 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:52.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.774+0000 7fd1dad9d700 1 -- 172.21.15.16:0/1075159524 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd1dc12cf60 msgr2=0x7fd1dc127740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:52.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.774+0000 7fd1dad9d700 1 --2- 172.21.15.16:0/1075159524 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd1dc12cf60 0x7fd1dc127740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:52.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.774+0000 7fd1dad9d700 1 -- 172.21.15.16:0/1075159524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1dc06f5d0 msgr2=0x7fd1dc127200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:52.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.774+0000 7fd1dad9d700 1 --2- 172.21.15.16:0/1075159524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1dc06f5d0 0x7fd1dc127200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:52.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.774+0000 7fd1dad9d700 1 -- 172.21.15.16:0/1075159524 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1d000d940 con 0x7fd1dc135c20 2023-09-30T22:34:52.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.774+0000 7fd1da59c700 1 --2- 172.21.15.16:0/1075159524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1dc06f5d0 0x7fd1dc127200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:52.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.774+0000 7fd1dad9d700 1 --2- 172.21.15.16:0/1075159524 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd1dc135c20 0x7fd1dc09c4f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc070520 tx=0x7fd1d4004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:52.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.774+0000 7fd1caffd700 1 -- 172.21.15.16:0/1075159524 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d4013420 con 0x7fd1dc135c20 2023-09-30T22:34:52.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.774+0000 7fd1caffd700 1 -- 172.21.15.16:0/1075159524 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1d4007160 con 0x7fd1dc135c20 2023-09-30T22:34:52.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.774+0000 7fd1caffd700 1 -- 172.21.15.16:0/1075159524 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d400b470 con 0x7fd1dc135c20 2023-09-30T22:34:52.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.774+0000 7fd1e08ce700 1 -- 172.21.15.16:0/1075159524 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1d400bb50 con 0x7fd1dc135c20 2023-09-30T22:34:52.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.774+0000 7fd1e08ce700 1 -- 172.21.15.16:0/1075159524 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1dc127c80 con 0x7fd1dc135c20 2023-09-30T22:34:52.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.775+0000 7fd1caffd700 1 -- 172.21.15.16:0/1075159524 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd1d4013620 con 0x7fd1dc135c20 2023-09-30T22:34:52.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.775+0000 7fd1caffd700 1 -- 172.21.15.16:0/1075159524 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd1d4040b90 con 0x7fd1dc135c20 2023-09-30T22:34:52.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.775+0000 7fd1e08ce700 1 -- 172.21.15.16:0/1075159524 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1dc074ea0 con 0x7fd1dc135c20 2023-09-30T22:34:52.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.780+0000 7fd1caffd700 1 -- 172.21.15.16:0/1075159524 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd1d402b080 con 0x7fd1dc135c20 2023-09-30T22:34:52.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.949+0000 7fcafdbe5700 1 -- 172.21.15.16:0/1955846379 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fcaf8074ea0 con 0x7fcaf8130260 2023-09-30T22:34:52.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.949+0000 7fcaf4ff9700 1 -- 172.21.15.16:0/1955846379 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcaf0005830 con 0x7fcaf8130260 2023-09-30T22:34:52.950 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:34:52.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.953+0000 7fcafdbe5700 1 -- 172.21.15.16:0/1955846379 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcaf8130260 msgr2=0x7fcaf81c3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:52.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.953+0000 7fcafdbe5700 1 --2- 172.21.15.16:0/1955846379 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcaf8130260 0x7fcaf81c3f50 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fcaf000c7c0 tx=0x7fcaf000cad0 comp rx=0 tx=0).stop 2023-09-30T22:34:52.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.953+0000 7fcafdbe5700 1 -- 172.21.15.16:0/1955846379 shutdown_connections 2023-09-30T22:34:52.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.953+0000 7fcafdbe5700 1 --2- 172.21.15.16:0/1955846379 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcaf8130260 0x7fcaf81c3f50 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:52.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.953+0000 7fcafdbe5700 1 --2- 172.21.15.16:0/1955846379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcaf812cf60 0x7fcaf809bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:52.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.953+0000 7fcafdbe5700 1 --2- 172.21.15.16:0/1955846379 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcaf812f8b0 0x7fcaf809c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:52.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.953+0000 7fcafdbe5700 1 -- 172.21.15.16:0/1955846379 >> 172.21.15.16:0/1955846379 conn(0x7fcaf80a2aa0 msgr2=0x7fcaf812a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:52.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.954+0000 7fcafdbe5700 1 -- 172.21.15.16:0/1955846379 shutdown_connections 2023-09-30T22:34:52.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.954+0000 7fcafdbe5700 1 -- 172.21.15.16:0/1955846379 wait complete. 2023-09-30T22:34:52.968 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:34:52.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.977+0000 7f8a0453a700 1 -- 172.21.15.16:0/4026136609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f89fc074ea0 con 0x7f89fc06b080 2023-09-30T22:34:52.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.978+0000 7f89f2ffd700 1 -- 172.21.15.16:0/4026136609 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f89ec01b340 con 0x7f89fc06b080 2023-09-30T22:34:52.978 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:34:52.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.980+0000 7f8a0453a700 1 -- 172.21.15.16:0/4026136609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89fc06b080 msgr2=0x7f89fc09b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:52.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.980+0000 7f8a0453a700 1 --2- 172.21.15.16:0/4026136609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89fc06b080 0x7f89fc09b0f0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f89fc09f8c0 tx=0x7f89ec00c750 comp rx=0 tx=0).stop 2023-09-30T22:34:52.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.980+0000 7f8a0453a700 1 -- 172.21.15.16:0/4026136609 shutdown_connections 2023-09-30T22:34:52.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.980+0000 7f8a0453a700 1 --2- 172.21.15.16:0/4026136609 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f89fc131d10 0x7f89fc1c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:52.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.980+0000 7f8a0453a700 1 --2- 172.21.15.16:0/4026136609 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f89fc06a770 0x7f89fc09abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:52.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.980+0000 7f8a0453a700 1 --2- 172.21.15.16:0/4026136609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89fc06b080 0x7f89fc09b0f0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:52.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.980+0000 7f8a0453a700 1 -- 172.21.15.16:0/4026136609 >> 172.21.15.16:0/4026136609 conn(0x7f89fc066f50 msgr2=0x7f89fc070000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:52.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.980+0000 7f8a0453a700 1 -- 172.21.15.16:0/4026136609 shutdown_connections 2023-09-30T22:34:52.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.980+0000 7f8a0453a700 1 -- 172.21.15.16:0/4026136609 wait complete. 2023-09-30T22:34:52.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.990+0000 7fd1e08ce700 1 -- 172.21.15.16:0/1075159524 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd1dc0722f0 con 0x7fd1dc135c20 2023-09-30T22:34:52.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.991+0000 7fd1caffd700 1 -- 172.21.15.16:0/1075159524 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd1dc0722f0 con 0x7fd1dc135c20 2023-09-30T22:34:52.991 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:34:52.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.994+0000 7fd1e08ce700 1 -- 172.21.15.16:0/1075159524 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd1dc135c20 msgr2=0x7fd1dc09c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:52.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.994+0000 7fd1e08ce700 1 --2- 172.21.15.16:0/1075159524 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd1dc135c20 0x7fd1dc09c4f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc070520 tx=0x7fd1d4004ad0 comp rx=0 tx=0).stop 2023-09-30T22:34:52.995 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:34:52.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.994+0000 7fd1e08ce700 1 -- 172.21.15.16:0/1075159524 shutdown_connections 2023-09-30T22:34:52.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.994+0000 7fd1e08ce700 1 --2- 172.21.15.16:0/1075159524 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd1dc135c20 0x7fd1dc09c4f0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:52.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.994+0000 7fd1e08ce700 1 --2- 172.21.15.16:0/1075159524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd1dc06f5d0 0x7fd1dc127200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:52.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.994+0000 7fd1e08ce700 1 --2- 172.21.15.16:0/1075159524 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd1dc12cf60 0x7fd1dc127740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:52.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.994+0000 7fd1e08ce700 1 -- 172.21.15.16:0/1075159524 >> 172.21.15.16:0/1075159524 conn(0x7fd1dc0a2aa0 msgr2=0x7fd1dc12a560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:52.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.994+0000 7fd1e08ce700 1 -- 172.21.15.16:0/1075159524 shutdown_connections 2023-09-30T22:34:52.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:52.994+0000 7fd1e08ce700 1 -- 172.21.15.16:0/1075159524 wait complete. 2023-09-30T22:34:53.008 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:34:53.629 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:34:53.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.775+0000 7f798ff09700 1 Processor -- start 2023-09-30T22:34:53.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.776+0000 7f798ff09700 1 -- start start 2023-09-30T22:34:53.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.776+0000 7f798ff09700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f798809d6f0 0x7f798809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:53.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.776+0000 7f798ff09700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f798809e000 0x7f79881339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:53.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.776+0000 7f798ff09700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f798809e410 0x7f7988133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:53.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.776+0000 7f798ff09700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f798806b460 con 0x7f798809e000 2023-09-30T22:34:53.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.776+0000 7f798ff09700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f798809b120 con 0x7f798809e410 2023-09-30T22:34:53.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.776+0000 7f798ff09700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f798809a7e0 con 0x7f798809d6f0 2023-09-30T22:34:53.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.776+0000 7f798d4a4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f798809e000 0x7f79881339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:53.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.776+0000 7f798dca5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f798809d6f0 0x7f798809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:53.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.776+0000 7f798e4a6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f798809e410 0x7f7988133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:53.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.776+0000 7f798d4a4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f798809e000 0x7f79881339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34592/0 (socket says 172.21.15.16:34592) 2023-09-30T22:34:53.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.776+0000 7f798e4a6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f798809e410 0x7f7988133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:42250/0 (socket says 172.21.15.16:42250) 2023-09-30T22:34:53.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.776+0000 7f798dca5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f798809d6f0 0x7f798809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:38914/0 (socket says 172.21.15.16:38914) 2023-09-30T22:34:53.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.776+0000 7f798d4a4700 1 -- 172.21.15.16:0/849303037 learned_addr learned my addr 172.21.15.16:0/849303037 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:53.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.777+0000 7f798d4a4700 1 -- 172.21.15.16:0/849303037 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f798809d6f0 msgr2=0x7f798809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:53.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.777+0000 7f798d4a4700 1 --2- 172.21.15.16:0/849303037 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f798809d6f0 0x7f798809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:53.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.777+0000 7f798d4a4700 1 -- 172.21.15.16:0/849303037 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f798809e410 msgr2=0x7f7988133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:53.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.777+0000 7f798d4a4700 1 --2- 172.21.15.16:0/849303037 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f798809e410 0x7f7988133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:53.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.777+0000 7f798d4a4700 1 -- 172.21.15.16:0/849303037 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f798809a500 con 0x7f798809e000 2023-09-30T22:34:53.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.777+0000 7f798e4a6700 1 --2- 172.21.15.16:0/849303037 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f798809e410 0x7f7988133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:53.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.777+0000 7f798d4a4700 1 --2- 172.21.15.16:0/849303037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f798809e000 0x7f79881339f0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f79780098f0 tx=0x7f7978009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7546184ac4b32738 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:53.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.777+0000 7f798cca3700 1 -- 172.21.15.16:0/849303037 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7978011830 con 0x7f798809e000 2023-09-30T22:34:53.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.777+0000 7f798cca3700 1 -- 172.21.15.16:0/849303037 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7978011990 con 0x7f798809e000 2023-09-30T22:34:53.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.777+0000 7f798cca3700 1 -- 172.21.15.16:0/849303037 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7978011d20 con 0x7f798809e000 2023-09-30T22:34:53.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.778+0000 7f798ff09700 1 -- 172.21.15.16:0/849303037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f798809e000 msgr2=0x7f79881339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:53.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.778+0000 7f798ff09700 1 --2- 172.21.15.16:0/849303037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f798809e000 0x7f79881339f0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f79780098f0 tx=0x7f7978009c00 comp rx=0 tx=0).stop 2023-09-30T22:34:53.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.778+0000 7f798ff09700 1 -- 172.21.15.16:0/849303037 shutdown_connections 2023-09-30T22:34:53.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.778+0000 7f798ff09700 1 --2- 172.21.15.16:0/849303037 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f798809e410 0x7f7988133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:53.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.778+0000 7f798ff09700 1 --2- 172.21.15.16:0/849303037 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f798809d6f0 0x7f798809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:53.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.778+0000 7f798ff09700 1 --2- 172.21.15.16:0/849303037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f798809e000 0x7f79881339f0 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:53.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.778+0000 7f798ff09700 1 -- 172.21.15.16:0/849303037 >> 172.21.15.16:0/849303037 conn(0x7f7988066f50 msgr2=0x7f798809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:53.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.778+0000 7f798ff09700 1 -- 172.21.15.16:0/849303037 shutdown_connections 2023-09-30T22:34:53.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.778+0000 7f798ff09700 1 -- 172.21.15.16:0/849303037 wait complete. 2023-09-30T22:34:53.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.779+0000 7f798ff09700 1 Processor -- start 2023-09-30T22:34:53.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.779+0000 7f798ff09700 1 -- start start 2023-09-30T22:34:53.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.779+0000 7f798ff09700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f798809d6f0 0x7f79881c8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:53.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.779+0000 7f798ff09700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f798809e000 0x7f79881c86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:53.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.779+0000 7f798ff09700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f798809e410 0x7f79881c8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:53.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.779+0000 7f798ff09700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79881a5590 con 0x7f798809d6f0 2023-09-30T22:34:53.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.779+0000 7f798ff09700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79881a53a0 con 0x7f798809e410 2023-09-30T22:34:53.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.779+0000 7f798ff09700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f79880a1fe0 con 0x7f798809e000 2023-09-30T22:34:53.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.779+0000 7f798dca5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f798809d6f0 0x7f79881c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:53.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.779+0000 7f798d4a4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f798809e000 0x7f79881c86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:53.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.779+0000 7f798dca5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f798809d6f0 0x7f79881c8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34602/0 (socket says 172.21.15.16:34602) 2023-09-30T22:34:53.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.779+0000 7f798dca5700 1 -- 172.21.15.16:0/3488303662 learned_addr learned my addr 172.21.15.16:0/3488303662 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:53.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.780+0000 7f798e4a6700 1 --2- 172.21.15.16:0/3488303662 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f798809e410 0x7f79881c8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:53.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.780+0000 7f798d4a4700 1 -- 172.21.15.16:0/3488303662 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f798809e410 msgr2=0x7f79881c8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:53.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.780+0000 7f798d4a4700 1 --2- 172.21.15.16:0/3488303662 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f798809e410 0x7f79881c8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:53.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.780+0000 7f798d4a4700 1 -- 172.21.15.16:0/3488303662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f798809d6f0 msgr2=0x7f79881c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:53.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.780+0000 7f798d4a4700 1 --2- 172.21.15.16:0/3488303662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f798809d6f0 0x7f79881c8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:53.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.780+0000 7f798d4a4700 1 -- 172.21.15.16:0/3488303662 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7980009a70 con 0x7f798809e000 2023-09-30T22:34:53.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.780+0000 7f798dca5700 1 --2- 172.21.15.16:0/3488303662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f798809d6f0 0x7f79881c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:53.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.780+0000 7f798d4a4700 1 --2- 172.21.15.16:0/3488303662 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f798809e000 0x7f79881c86d0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f798806e510 tx=0x7f7978004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:53.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.780+0000 7f797e7fc700 1 -- 172.21.15.16:0/3488303662 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7978004240 con 0x7f798809e000 2023-09-30T22:34:53.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.780+0000 7f797e7fc700 1 -- 172.21.15.16:0/3488303662 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f797801f070 con 0x7f798809e000 2023-09-30T22:34:53.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.780+0000 7f798ff09700 1 -- 172.21.15.16:0/3488303662 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7978009550 con 0x7f798809e000 2023-09-30T22:34:53.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.780+0000 7f797e7fc700 1 -- 172.21.15.16:0/3488303662 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79780045c0 con 0x7f798809e000 2023-09-30T22:34:53.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.780+0000 7f798ff09700 1 -- 172.21.15.16:0/3488303662 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79881a7320 con 0x7f798809e000 2023-09-30T22:34:53.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.781+0000 7f797e7fc700 1 -- 172.21.15.16:0/3488303662 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7978021030 con 0x7f798809e000 2023-09-30T22:34:53.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.781+0000 7f797e7fc700 1 -- 172.21.15.16:0/3488303662 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f7978040810 con 0x7f798809e000 2023-09-30T22:34:53.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.781+0000 7f798ff09700 1 -- 172.21.15.16:0/3488303662 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79881a6ef0 con 0x7f798809e000 2023-09-30T22:34:53.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.786+0000 7f797e7fc700 1 -- 172.21.15.16:0/3488303662 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f797801a450 con 0x7f798809e000 2023-09-30T22:34:53.969 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:34:53.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.991+0000 7f798ff09700 1 -- 172.21.15.16:0/3488303662 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7988074ea0 con 0x7f798809e000 2023-09-30T22:34:53.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.991+0000 7f797e7fc700 1 -- 172.21.15.16:0/3488303662 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7978015030 con 0x7f798809e000 2023-09-30T22:34:53.992 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:34:53.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.994+0000 7f798ff09700 1 -- 172.21.15.16:0/3488303662 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f798809e000 msgr2=0x7f79881c86d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:53.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.994+0000 7f798ff09700 1 --2- 172.21.15.16:0/3488303662 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f798809e000 0x7f79881c86d0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f798806e510 tx=0x7f7978004750 comp rx=0 tx=0).stop 2023-09-30T22:34:53.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.994+0000 7f798ff09700 1 -- 172.21.15.16:0/3488303662 shutdown_connections 2023-09-30T22:34:53.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.994+0000 7f798ff09700 1 --2- 172.21.15.16:0/3488303662 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f798809e410 0x7f79881c8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:53.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.994+0000 7f798ff09700 1 --2- 172.21.15.16:0/3488303662 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f798809d6f0 0x7f79881c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:53.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.994+0000 7f798ff09700 1 --2- 172.21.15.16:0/3488303662 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f798809e000 0x7f79881c86d0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:53.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.994+0000 7f798ff09700 1 -- 172.21.15.16:0/3488303662 >> 172.21.15.16:0/3488303662 conn(0x7f7988066f50 msgr2=0x7f79881bbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:53.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.994+0000 7f798ff09700 1 -- 172.21.15.16:0/3488303662 shutdown_connections 2023-09-30T22:34:53.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:53.994+0000 7f798ff09700 1 -- 172.21.15.16:0/3488303662 wait complete. 2023-09-30T22:34:53.996 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:34:54.009 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:34:54.010 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:34:54.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.114+0000 7f1ff7827700 1 Processor -- start 2023-09-30T22:34:54.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.114+0000 7f1ff7827700 1 -- start start 2023-09-30T22:34:54.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.115+0000 7f1ff7827700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ff0131810 0x7f1ff0131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:54.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.115+0000 7f1ff7827700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1ff009a580 0x7f1ff0139660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:54.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.115+0000 7f1ff7827700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ff01313c0 con 0x7f1ff0099a40 2023-09-30T22:34:54.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.115+0000 7f1ff7827700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ff009b580 con 0x7f1ff009a580 2023-09-30T22:34:54.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.115+0000 7f1ff7827700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff006e8c0 con 0x7f1ff0131810 2023-09-30T22:34:54.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.115+0000 7f1ff55c3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ff0131810 0x7f1ff0131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:54.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.115+0000 7f1ff5dc4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1ff009a580 0x7f1ff0139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:54.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.115+0000 7f1ff55c3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ff0131810 0x7f1ff0131be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:42272/0 (socket says 172.21.15.16:42272) 2023-09-30T22:34:54.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.115+0000 7f1ff55c3700 1 -- 172.21.15.16:0/1876062454 learned_addr learned my addr 172.21.15.16:0/1876062454 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:54.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.115+0000 7f1fe7fff700 1 -- 172.21.15.16:0/1876062454 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3059242340 0 0) 0x7f1ff01313c0 con 0x7f1ff0099a40 2023-09-30T22:34:54.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.115+0000 7f1ff5dc4700 1 -- 172.21.15.16:0/1876062454 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ff0131810 msgr2=0x7f1ff0131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:54.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.115+0000 7f1ff5dc4700 1 --2- 172.21.15.16:0/1876062454 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ff0131810 0x7f1ff0131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:54.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.115+0000 7f1ff5dc4700 1 -- 172.21.15.16:0/1876062454 >> v1:172.21.15.16:6789/0 conn(0x7f1ff0099a40 legacy=0x7f1ff0099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:54.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.115+0000 7f1ff5dc4700 1 -- 172.21.15.16:0/1876062454 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ff0071d20 con 0x7f1ff009a580 2023-09-30T22:34:54.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.115+0000 7f1ff55c3700 1 --2- 172.21.15.16:0/1876062454 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ff0131810 0x7f1ff0131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:54.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.115+0000 7f1ff5dc4700 1 --2- 172.21.15.16:0/1876062454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ff009a580 0x7f1ff0139660 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8009d90 tx=0x7f1fe800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d4acc973e5109fe server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:54.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.116+0000 7f1fe7fff700 1 -- 172.21.15.16:0/1876062454 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe800e070 con 0x7f1ff009a580 2023-09-30T22:34:54.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.116+0000 7f1fe7fff700 1 -- 172.21.15.16:0/1876062454 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fe800bb20 con 0x7f1ff009a580 2023-09-30T22:34:54.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.116+0000 7f1fe7fff700 1 -- 172.21.15.16:0/1876062454 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe8012430 con 0x7f1ff009a580 2023-09-30T22:34:54.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.116+0000 7f1ff7827700 1 -- 172.21.15.16:0/1876062454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ff009a580 msgr2=0x7f1ff0139660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:54.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.116+0000 7f1ff7827700 1 --2- 172.21.15.16:0/1876062454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ff009a580 0x7f1ff0139660 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8009d90 tx=0x7f1fe800b330 comp rx=0 tx=0).stop 2023-09-30T22:34:54.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.117+0000 7f1ff7827700 1 -- 172.21.15.16:0/1876062454 shutdown_connections 2023-09-30T22:34:54.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.117+0000 7f1ff7827700 1 --2- 172.21.15.16:0/1876062454 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ff009a580 0x7f1ff0139660 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:54.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.117+0000 7f1ff7827700 1 --2- 172.21.15.16:0/1876062454 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ff0131810 0x7f1ff0131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:54.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.117+0000 7f1ff7827700 1 -- 172.21.15.16:0/1876062454 >> 172.21.15.16:0/1876062454 conn(0x7f1ff00a2aa0 msgr2=0x7f1ff00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:54.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.117+0000 7f1ff7827700 1 -- 172.21.15.16:0/1876062454 shutdown_connections 2023-09-30T22:34:54.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.117+0000 7f1ff7827700 1 -- 172.21.15.16:0/1876062454 wait complete. 2023-09-30T22:34:54.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.118+0000 7f1ff7827700 1 Processor -- start 2023-09-30T22:34:54.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.118+0000 7f1ff7827700 1 -- start start 2023-09-30T22:34:54.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.118+0000 7f1ff7827700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ff0099a40 0x7f1ff006f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:54.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.118+0000 7f1ff7827700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ff009a580 0x7f1ff006fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:54.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.118+0000 7f1ff7827700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ff0131810 0x7f1ff0070090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:54.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.118+0000 7f1ff7827700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff0121290 con 0x7f1ff009a580 2023-09-30T22:34:54.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.118+0000 7f1ff7827700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff0120230 con 0x7f1ff0099a40 2023-09-30T22:34:54.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.118+0000 7f1ff7827700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff0070a70 con 0x7f1ff0131810 2023-09-30T22:34:54.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.118+0000 7f1ff55c3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ff0099a40 0x7f1ff006f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:54.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.119+0000 7f1ff55c3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ff0099a40 0x7f1ff006f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:42280/0 (socket says 172.21.15.16:42280) 2023-09-30T22:34:54.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.119+0000 7f1ff55c3700 1 -- 172.21.15.16:0/3604755047 learned_addr learned my addr 172.21.15.16:0/3604755047 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:54.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.119+0000 7f1ff4dc2700 1 --2- 172.21.15.16:0/3604755047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ff009a580 0x7f1ff006fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:54.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.119+0000 7f1ff5dc4700 1 --2- 172.21.15.16:0/3604755047 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ff0131810 0x7f1ff0070090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:54.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.119+0000 7f1ff4dc2700 1 -- 172.21.15.16:0/3604755047 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ff0131810 msgr2=0x7f1ff0070090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:54.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.119+0000 7f1ff4dc2700 1 --2- 172.21.15.16:0/3604755047 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ff0131810 0x7f1ff0070090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:54.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.119+0000 7f1ff4dc2700 1 -- 172.21.15.16:0/3604755047 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ff0099a40 msgr2=0x7f1ff006f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:54.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.119+0000 7f1ff4dc2700 1 --2- 172.21.15.16:0/3604755047 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ff0099a40 0x7f1ff006f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:54.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.119+0000 7f1ff4dc2700 1 -- 172.21.15.16:0/3604755047 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fec009550 con 0x7f1ff009a580 2023-09-30T22:34:54.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.119+0000 7f1ff4dc2700 1 --2- 172.21.15.16:0/3604755047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ff009a580 0x7f1ff006fb50 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0072eb0 tx=0x7f1fe000c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:54.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.119+0000 7f1fe5ffb700 1 -- 172.21.15.16:0/3604755047 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe00054e0 con 0x7f1ff009a580 2023-09-30T22:34:54.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.119+0000 7f1fe5ffb700 1 -- 172.21.15.16:0/3604755047 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fe000ce10 con 0x7f1ff009a580 2023-09-30T22:34:54.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.119+0000 7f1fe5ffb700 1 -- 172.21.15.16:0/3604755047 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe00038c0 con 0x7f1ff009a580 2023-09-30T22:34:54.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.119+0000 7f1ff7827700 1 -- 172.21.15.16:0/3604755047 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fe8009a70 con 0x7f1ff009a580 2023-09-30T22:34:54.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.120+0000 7f1ff7827700 1 -- 172.21.15.16:0/3604755047 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ff00a19f0 con 0x7f1ff009a580 2023-09-30T22:34:54.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.120+0000 7f1fe5ffb700 1 -- 172.21.15.16:0/3604755047 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1fe0003a20 con 0x7f1ff009a580 2023-09-30T22:34:54.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.121+0000 7f1fe5ffb700 1 -- 172.21.15.16:0/3604755047 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f1fe003ea50 con 0x7f1ff009a580 2023-09-30T22:34:54.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.121+0000 7f1ff7827700 1 -- 172.21.15.16:0/3604755047 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ff006eb60 con 0x7f1ff009a580 2023-09-30T22:34:54.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.130+0000 7f1fe5ffb700 1 -- 172.21.15.16:0/3604755047 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1fe0017080 con 0x7f1ff009a580 2023-09-30T22:34:54.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.137+0000 7ff69fb49700 1 Processor -- start 2023-09-30T22:34:54.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.138+0000 7ff69fb49700 1 -- start start 2023-09-30T22:34:54.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.138+0000 7ff69fb49700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff69809f410 0x7ff698136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:54.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.138+0000 7ff69fb49700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff698136fe0 0x7ff6981393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:54.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.138+0000 7ff69fb49700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff698070940 con 0x7ff69809e930 2023-09-30T22:34:54.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.138+0000 7ff69fb49700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff69806fd60 con 0x7ff69809f410 2023-09-30T22:34:54.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.138+0000 7ff69fb49700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff69806fa80 con 0x7ff698136fe0 2023-09-30T22:34:54.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.139+0000 7ff69e0e6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff698136fe0 0x7ff6981393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:54.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.139+0000 7ff69d0e4700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff69809f410 0x7ff698136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:54.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.139+0000 7ff69d8e5700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff69809e930 0x7ff69809ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60964/0 (socket says 172.21.15.16:60964) 2023-09-30T22:34:54.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.139+0000 7ff69d8e5700 1 -- 172.21.15.16:0/4214030884 learned_addr learned my addr 172.21.15.16:0/4214030884 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:34:54.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.139+0000 7ff69d0e4700 1 -- 172.21.15.16:0/4214030884 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff698136fe0 msgr2=0x7ff6981393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:54.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.139+0000 7ff69d0e4700 1 --2- 172.21.15.16:0/4214030884 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff698136fe0 0x7ff6981393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:54.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.139+0000 7ff69c8e3700 1 -- 172.21.15.16:0/4214030884 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2505206142 0 0) 0x7ff698070940 con 0x7ff69809e930 2023-09-30T22:34:54.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.139+0000 7ff69d0e4700 1 -- 172.21.15.16:0/4214030884 >> v1:172.21.15.16:6789/0 conn(0x7ff69809e930 legacy=0x7ff69809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:54.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.139+0000 7ff69d0e4700 1 -- 172.21.15.16:0/4214030884 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff69809fb60 con 0x7ff69809f410 2023-09-30T22:34:54.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.139+0000 7ff69d0e4700 1 --2- 172.21.15.16:0/4214030884 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff69809f410 0x7ff698136aa0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7ff688009870 tx=0x7ff688009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=777d329d1b18bb31 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:54.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.139+0000 7ff69c8e3700 1 -- 172.21.15.16:0/4214030884 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff688011830 con 0x7ff69809f410 2023-09-30T22:34:54.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.139+0000 7ff69c8e3700 1 -- 172.21.15.16:0/4214030884 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff688011990 con 0x7ff69809f410 2023-09-30T22:34:54.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.139+0000 7ff69c8e3700 1 -- 172.21.15.16:0/4214030884 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff688011d20 con 0x7ff69809f410 2023-09-30T22:34:54.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.140+0000 7ff69fb49700 1 -- 172.21.15.16:0/4214030884 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff69809f410 msgr2=0x7ff698136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:54.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.140+0000 7ff69fb49700 1 --2- 172.21.15.16:0/4214030884 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff69809f410 0x7ff698136aa0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7ff688009870 tx=0x7ff688009b80 comp rx=0 tx=0).stop 2023-09-30T22:34:54.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.140+0000 7ff69fb49700 1 -- 172.21.15.16:0/4214030884 shutdown_connections 2023-09-30T22:34:54.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.140+0000 7ff69fb49700 1 --2- 172.21.15.16:0/4214030884 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff698136fe0 0x7ff6981393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:54.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.140+0000 7ff69fb49700 1 --2- 172.21.15.16:0/4214030884 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff69809f410 0x7ff698136aa0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:54.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.140+0000 7ff69fb49700 1 -- 172.21.15.16:0/4214030884 >> 172.21.15.16:0/4214030884 conn(0x7ff6980a4210 msgr2=0x7ff6980a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:54.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.141+0000 7ff69fb49700 1 -- 172.21.15.16:0/4214030884 shutdown_connections 2023-09-30T22:34:54.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.141+0000 7ff69fb49700 1 -- 172.21.15.16:0/4214030884 wait complete. 2023-09-30T22:34:54.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.142+0000 7ff69fb49700 1 Processor -- start 2023-09-30T22:34:54.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.142+0000 7ff69fb49700 1 -- start start 2023-09-30T22:34:54.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.143+0000 7ff69fb49700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff69809e930 0x7ff69806c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:54.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.143+0000 7ff69fb49700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff69809f410 0x7ff69806c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:54.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.143+0000 7ff69fb49700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff698136fe0 0x7ff6981c8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:54.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.143+0000 7ff69fb49700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff698121160 con 0x7ff698136fe0 2023-09-30T22:34:54.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.143+0000 7ff69fb49700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff69806d5c0 con 0x7ff69809f410 2023-09-30T22:34:54.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.143+0000 7ff69fb49700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff69806cc40 con 0x7ff69809e930 2023-09-30T22:34:54.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.143+0000 7ff69d0e4700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff69809f410 0x7ff69806c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:54.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.143+0000 7ff69d0e4700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff69809f410 0x7ff69806c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:42298/0 (socket says 172.21.15.16:42298) 2023-09-30T22:34:54.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.143+0000 7ff69d0e4700 1 -- 172.21.15.16:0/1621254680 learned_addr learned my addr 172.21.15.16:0/1621254680 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:54.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.143+0000 7ff69e0e6700 1 --2- 172.21.15.16:0/1621254680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff698136fe0 0x7ff6981c8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:54.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.143+0000 7ff69d8e5700 1 --2- 172.21.15.16:0/1621254680 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff69809e930 0x7ff69806c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:54.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.143+0000 7ff69e0e6700 1 -- 172.21.15.16:0/1621254680 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff69809e930 msgr2=0x7ff69806c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:54.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.143+0000 7ff69e0e6700 1 --2- 172.21.15.16:0/1621254680 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff69809e930 0x7ff69806c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:54.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.143+0000 7ff69e0e6700 1 -- 172.21.15.16:0/1621254680 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff69809f410 msgr2=0x7ff69806c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:54.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.143+0000 7ff69e0e6700 1 --2- 172.21.15.16:0/1621254680 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff69809f410 0x7ff69806c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:54.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.143+0000 7ff69e0e6700 1 -- 172.21.15.16:0/1621254680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6880094a0 con 0x7ff698136fe0 2023-09-30T22:34:54.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.144+0000 7ff69e0e6700 1 --2- 172.21.15.16:0/1621254680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff698136fe0 0x7ff6981c8470 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7ff69000ef90 tx=0x7ff69000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:54.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.144+0000 7ff68e7fc700 1 -- 172.21.15.16:0/1621254680 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69000cd40 con 0x7ff698136fe0 2023-09-30T22:34:54.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.144+0000 7ff68e7fc700 1 -- 172.21.15.16:0/1621254680 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff69000f040 con 0x7ff698136fe0 2023-09-30T22:34:54.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.144+0000 7ff69fb49700 1 -- 172.21.15.16:0/1621254680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff698132640 con 0x7ff698136fe0 2023-09-30T22:34:54.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.144+0000 7ff68e7fc700 1 -- 172.21.15.16:0/1621254680 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6900146c0 con 0x7ff698136fe0 2023-09-30T22:34:54.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.144+0000 7ff69fb49700 1 -- 172.21.15.16:0/1621254680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff69809df70 con 0x7ff698136fe0 2023-09-30T22:34:54.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.145+0000 7ff68e7fc700 1 -- 172.21.15.16:0/1621254680 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff690014820 con 0x7ff698136fe0 2023-09-30T22:34:54.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.145+0000 7ff68e7fc700 1 -- 172.21.15.16:0/1621254680 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7ff690040270 con 0x7ff698136fe0 2023-09-30T22:34:54.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.146+0000 7ff69fb49700 1 -- 172.21.15.16:0/1621254680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff69809fdb0 con 0x7ff698136fe0 2023-09-30T22:34:54.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.151+0000 7ff68e7fc700 1 -- 172.21.15.16:0/1621254680 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff690018020 con 0x7ff698136fe0 2023-09-30T22:34:54.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.200+0000 7f34f5a05700 1 Processor -- start 2023-09-30T22:34:54.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.200+0000 7f34f5a05700 1 -- start start 2023-09-30T22:34:54.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.200+0000 7f34f5a05700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f34f00a0a30 0x7f34f00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:54.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.200+0000 7f34f5a05700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f34f00a1340 0x7f34f0133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:54.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.200+0000 7f34f5a05700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f34f0133fa0 0x7f34f0134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:54.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.200+0000 7f34f5a05700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34f009edd0 con 0x7f34f00a1340 2023-09-30T22:34:54.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.200+0000 7f34f5a05700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34f009e1f0 con 0x7f34f0133fa0 2023-09-30T22:34:54.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.200+0000 7f34f5a05700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f34f009dc50 con 0x7f34f00a0a30 2023-09-30T22:34:54.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.200+0000 7f34ee7fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f34f00a1340 0x7f34f0133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:54.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.201+0000 7f34ee7fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f34f00a1340 0x7f34f0133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34628/0 (socket says 172.21.15.16:34628) 2023-09-30T22:34:54.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.201+0000 7f34ee7fc700 1 -- 172.21.15.16:0/338265444 learned_addr learned my addr 172.21.15.16:0/338265444 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:54.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.201+0000 7f34eeffd700 1 --2- 172.21.15.16:0/338265444 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f34f00a0a30 0x7f34f00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:54.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.201+0000 7f34ef7fe700 1 --2- 172.21.15.16:0/338265444 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f34f0133fa0 0x7f34f0134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:54.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.201+0000 7f34eeffd700 1 -- 172.21.15.16:0/338265444 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f34f0133fa0 msgr2=0x7f34f0134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:54.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.201+0000 7f34eeffd700 1 --2- 172.21.15.16:0/338265444 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f34f0133fa0 0x7f34f0134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:54.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.201+0000 7f34eeffd700 1 -- 172.21.15.16:0/338265444 >> v2:172.21.15.16:3300/0 conn(0x7f34f00a1340 msgr2=0x7f34f0133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:54.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.201+0000 7f34eeffd700 1 --2- 172.21.15.16:0/338265444 >> v2:172.21.15.16:3300/0 conn(0x7f34f00a1340 0x7f34f0133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:54.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.201+0000 7f34eeffd700 1 -- 172.21.15.16:0/338265444 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f00a1fe0 con 0x7f34f00a0a30 2023-09-30T22:34:54.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.201+0000 7f34ee7fc700 1 --2- 172.21.15.16:0/338265444 >> v2:172.21.15.16:3300/0 conn(0x7f34f00a1340 0x7f34f0133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:54.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.201+0000 7f34eeffd700 1 --2- 172.21.15.16:0/338265444 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f34f00a0a30 0x7f34f00a0e00 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f34e0009870 tx=0x7f34e0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b223addf404c6506 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:54.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.201+0000 7f34edffb700 1 -- 172.21.15.16:0/338265444 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e0011830 con 0x7f34f00a0a30 2023-09-30T22:34:54.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.201+0000 7f34edffb700 1 -- 172.21.15.16:0/338265444 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34e0011990 con 0x7f34f00a0a30 2023-09-30T22:34:54.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.201+0000 7f34edffb700 1 -- 172.21.15.16:0/338265444 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e0011d20 con 0x7f34f00a0a30 2023-09-30T22:34:54.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.202+0000 7f34f5a05700 1 -- 172.21.15.16:0/338265444 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f34f00a0a30 msgr2=0x7f34f00a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:54.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.202+0000 7f34f5a05700 1 --2- 172.21.15.16:0/338265444 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f34f00a0a30 0x7f34f00a0e00 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f34e0009870 tx=0x7f34e0009b80 comp rx=0 tx=0).stop 2023-09-30T22:34:54.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.202+0000 7f34f5a05700 1 -- 172.21.15.16:0/338265444 shutdown_connections 2023-09-30T22:34:54.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.202+0000 7f34f5a05700 1 --2- 172.21.15.16:0/338265444 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f34f0133fa0 0x7f34f0134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:54.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.202+0000 7f34f5a05700 1 --2- 172.21.15.16:0/338265444 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f34f00a0a30 0x7f34f00a0e00 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:54.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.202+0000 7f34f5a05700 1 --2- 172.21.15.16:0/338265444 >> v2:172.21.15.16:3300/0 conn(0x7f34f00a1340 0x7f34f0133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:54.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.202+0000 7f34f5a05700 1 -- 172.21.15.16:0/338265444 >> 172.21.15.16:0/338265444 conn(0x7f34f00a6550 msgr2=0x7f34f00a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:54.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.202+0000 7f34f5a05700 1 -- 172.21.15.16:0/338265444 shutdown_connections 2023-09-30T22:34:54.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.202+0000 7f34f5a05700 1 -- 172.21.15.16:0/338265444 wait complete. 2023-09-30T22:34:54.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.203+0000 7f34f5a05700 1 Processor -- start 2023-09-30T22:34:54.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.203+0000 7f34f5a05700 1 -- start start 2023-09-30T22:34:54.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.204+0000 7f34f5a05700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34f00a0a30 0x7f34f01213d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:54.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.204+0000 7f34f5a05700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f34f00a1340 0x7f34f0121910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:54.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.204+0000 7f34f5a05700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f34f0133fa0 0x7f34f0121e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:54.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.204+0000 7f34f5a05700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f0124c10 con 0x7f34f00a0a30 2023-09-30T22:34:54.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.204+0000 7f34f5a05700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34f01238b0 con 0x7f34f0133fa0 2023-09-30T22:34:54.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.204+0000 7f34f5a05700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f34f00a37d0 con 0x7f34f00a1340 2023-09-30T22:34:54.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.204+0000 7f34ee7fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f34f00a1340 0x7f34f0121910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:54.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.204+0000 7f34ef7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f34f0133fa0 0x7f34f0121e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:54.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.204+0000 7f34ef7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f34f0133fa0 0x7f34f0121e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:42316/0 (socket says 172.21.15.16:42316) 2023-09-30T22:34:54.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.204+0000 7f34ee7fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f34f00a1340 0x7f34f0121910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:38966/0 (socket says 172.21.15.16:38966) 2023-09-30T22:34:54.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.204+0000 7f34ef7fe700 1 -- 172.21.15.16:0/2148810943 learned_addr learned my addr 172.21.15.16:0/2148810943 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:54.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.204+0000 7f34eeffd700 1 --2- 172.21.15.16:0/2148810943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34f00a0a30 0x7f34f01213d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:54.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.204+0000 7f34ee7fc700 1 -- 172.21.15.16:0/2148810943 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f34f0133fa0 msgr2=0x7f34f0121e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:54.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.204+0000 7f34ee7fc700 1 --2- 172.21.15.16:0/2148810943 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f34f0133fa0 0x7f34f0121e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:54.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.204+0000 7f34ee7fc700 1 -- 172.21.15.16:0/2148810943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34f00a0a30 msgr2=0x7f34f01213d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:54.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.204+0000 7f34ee7fc700 1 --2- 172.21.15.16:0/2148810943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34f00a0a30 0x7f34f01213d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:54.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.204+0000 7f34ee7fc700 1 -- 172.21.15.16:0/2148810943 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34e4009550 con 0x7f34f00a1340 2023-09-30T22:34:54.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.204+0000 7f34ef7fe700 1 --2- 172.21.15.16:0/2148810943 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f34f0133fa0 0x7f34f0121e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:54.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.204+0000 7f34eeffd700 1 --2- 172.21.15.16:0/2148810943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34f00a0a30 0x7f34f01213d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:54.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.205+0000 7f34ee7fc700 1 --2- 172.21.15.16:0/2148810943 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f34f00a1340 0x7f34f0121910 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f34e400e9b0 tx=0x7f34e400ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:54.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.205+0000 7f34d7fff700 1 -- 172.21.15.16:0/2148810943 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e4004030 con 0x7f34f00a1340 2023-09-30T22:34:54.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.205+0000 7f34d7fff700 1 -- 172.21.15.16:0/2148810943 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34e4004190 con 0x7f34f00a1340 2023-09-30T22:34:54.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.205+0000 7f34d7fff700 1 -- 172.21.15.16:0/2148810943 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e4004520 con 0x7f34f00a1340 2023-09-30T22:34:54.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.205+0000 7f34f5a05700 1 -- 172.21.15.16:0/2148810943 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34e00094a0 con 0x7f34f00a1340 2023-09-30T22:34:54.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.205+0000 7f34f5a05700 1 -- 172.21.15.16:0/2148810943 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34f0123e00 con 0x7f34f00a1340 2023-09-30T22:34:54.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.206+0000 7f34d7fff700 1 -- 172.21.15.16:0/2148810943 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f34e40036a0 con 0x7f34f00a1340 2023-09-30T22:34:54.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.206+0000 7f34d7fff700 1 -- 172.21.15.16:0/2148810943 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f34e403f430 con 0x7f34f00a1340 2023-09-30T22:34:54.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.206+0000 7f34f5a05700 1 -- 172.21.15.16:0/2148810943 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34f0074ea0 con 0x7f34f00a1340 2023-09-30T22:34:54.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.211+0000 7f34d7fff700 1 -- 172.21.15.16:0/2148810943 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f34e4019950 con 0x7f34f00a1340 2023-09-30T22:34:54.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.402+0000 7f1ff7827700 1 -- 172.21.15.16:0/3604755047 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1ff00721a0 con 0x7f1ff009a580 2023-09-30T22:34:54.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.403+0000 7f1fe5ffb700 1 -- 172.21.15.16:0/3604755047 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1fe0019b40 con 0x7f1ff009a580 2023-09-30T22:34:54.403 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:34:54.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.405+0000 7f1ff7827700 1 -- 172.21.15.16:0/3604755047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ff009a580 msgr2=0x7f1ff006fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:54.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.405+0000 7f1ff7827700 1 --2- 172.21.15.16:0/3604755047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ff009a580 0x7f1ff006fb50 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0072eb0 tx=0x7f1fe000c710 comp rx=0 tx=0).stop 2023-09-30T22:34:54.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.405+0000 7f1ff7827700 1 -- 172.21.15.16:0/3604755047 shutdown_connections 2023-09-30T22:34:54.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.405+0000 7f1ff7827700 1 --2- 172.21.15.16:0/3604755047 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1ff0099a40 0x7f1ff006f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:54.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.405+0000 7f1ff7827700 1 --2- 172.21.15.16:0/3604755047 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1ff0131810 0x7f1ff0070090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:54.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.405+0000 7f1ff7827700 1 --2- 172.21.15.16:0/3604755047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ff009a580 0x7f1ff006fb50 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:54.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.405+0000 7f1ff7827700 1 -- 172.21.15.16:0/3604755047 >> 172.21.15.16:0/3604755047 conn(0x7f1ff00a2aa0 msgr2=0x7f1ff01b8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:54.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.405+0000 7f1ff7827700 1 -- 172.21.15.16:0/3604755047 shutdown_connections 2023-09-30T22:34:54.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.405+0000 7f1ff7827700 1 -- 172.21.15.16:0/3604755047 wait complete. 2023-09-30T22:34:54.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.415+0000 7f34f5a05700 1 -- 172.21.15.16:0/2148810943 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f34f00a28e0 con 0x7f34f00a1340 2023-09-30T22:34:54.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.416+0000 7f34d7fff700 1 -- 172.21.15.16:0/2148810943 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f34e4019770 con 0x7f34f00a1340 2023-09-30T22:34:54.416 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:34:54.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.418+0000 7f34f5a05700 1 -- 172.21.15.16:0/2148810943 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f34f00a1340 msgr2=0x7f34f0121910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:54.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.418+0000 7f34f5a05700 1 --2- 172.21.15.16:0/2148810943 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f34f00a1340 0x7f34f0121910 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f34e400e9b0 tx=0x7f34e400ecc0 comp rx=0 tx=0).stop 2023-09-30T22:34:54.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.418+0000 7f34f5a05700 1 -- 172.21.15.16:0/2148810943 shutdown_connections 2023-09-30T22:34:54.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.418+0000 7f34f5a05700 1 --2- 172.21.15.16:0/2148810943 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f34f0133fa0 0x7f34f0121e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:54.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.418+0000 7f34f5a05700 1 --2- 172.21.15.16:0/2148810943 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34f00a0a30 0x7f34f01213d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:54.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.418+0000 7f34f5a05700 1 --2- 172.21.15.16:0/2148810943 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f34f00a1340 0x7f34f0121910 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:54.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.418+0000 7f34f5a05700 1 -- 172.21.15.16:0/2148810943 >> 172.21.15.16:0/2148810943 conn(0x7f34f00a6550 msgr2=0x7f34f00a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:54.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.418+0000 7f34f5a05700 1 -- 172.21.15.16:0/2148810943 shutdown_connections 2023-09-30T22:34:54.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.418+0000 7f34f5a05700 1 -- 172.21.15.16:0/2148810943 wait complete. 2023-09-30T22:34:54.420 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:34:54.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.420+0000 7ff69fb49700 1 -- 172.21.15.16:0/1621254680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff6980a3d20 con 0x7ff698136fe0 2023-09-30T22:34:54.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.420+0000 7ff68e7fc700 1 -- 172.21.15.16:0/1621254680 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff69002b090 con 0x7ff698136fe0 2023-09-30T22:34:54.421 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:34:54.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.422+0000 7ff69fb49700 1 -- 172.21.15.16:0/1621254680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff698136fe0 msgr2=0x7ff6981c8470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:54.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.422+0000 7ff69fb49700 1 --2- 172.21.15.16:0/1621254680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff698136fe0 0x7ff6981c8470 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7ff69000ef90 tx=0x7ff69000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:34:54.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.422+0000 7ff69fb49700 1 -- 172.21.15.16:0/1621254680 shutdown_connections 2023-09-30T22:34:54.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.422+0000 7ff69fb49700 1 --2- 172.21.15.16:0/1621254680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff698136fe0 0x7ff6981c8470 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:54.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.422+0000 7ff69fb49700 1 --2- 172.21.15.16:0/1621254680 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff69809e930 0x7ff69806c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:54.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.422+0000 7ff69fb49700 1 --2- 172.21.15.16:0/1621254680 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff69809f410 0x7ff69806c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:54.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.422+0000 7ff69fb49700 1 -- 172.21.15.16:0/1621254680 >> 172.21.15.16:0/1621254680 conn(0x7ff6980a4210 msgr2=0x7ff698135c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:54.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.423+0000 7ff69fb49700 1 -- 172.21.15.16:0/1621254680 shutdown_connections 2023-09-30T22:34:54.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:54.423+0000 7ff69fb49700 1 -- 172.21.15.16:0/1621254680 wait complete. 2023-09-30T22:34:54.434 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:34:54.437 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:34:55.011 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:34:55.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.157+0000 7f05a50eb700 1 Processor -- start 2023-09-30T22:34:55.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.157+0000 7f05a50eb700 1 -- start start 2023-09-30T22:34:55.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.158+0000 7f05a50eb700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f05a00a0a30 0x7f05a00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:55.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.158+0000 7f05a50eb700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f05a00a1340 0x7f05a0133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:55.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.158+0000 7f05a50eb700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f05a0133fa0 0x7f05a0134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:55.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.158+0000 7f05a50eb700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05a009edd0 con 0x7f05a0133fa0 2023-09-30T22:34:55.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.158+0000 7f05a50eb700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05a009e1f0 con 0x7f05a00a1340 2023-09-30T22:34:55.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.158+0000 7f05a50eb700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f05a009dc50 con 0x7f05a00a0a30 2023-09-30T22:34:55.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.158+0000 7f059e59c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f05a00a1340 0x7f05a0133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:55.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.158+0000 7f059f59e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f05a0133fa0 0x7f05a0134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:55.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.158+0000 7f059e59c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f05a00a1340 0x7f05a0133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:42322/0 (socket says 172.21.15.16:42322) 2023-09-30T22:34:55.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.158+0000 7f059ed9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f05a00a0a30 0x7f05a00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:55.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.158+0000 7f059e59c700 1 -- 172.21.15.16:0/2142745400 learned_addr learned my addr 172.21.15.16:0/2142745400 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:55.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.158+0000 7f059e59c700 1 -- 172.21.15.16:0/2142745400 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f05a00a0a30 msgr2=0x7f05a00a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:55.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.158+0000 7f059e59c700 1 --2- 172.21.15.16:0/2142745400 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f05a00a0a30 0x7f05a00a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:55.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.158+0000 7f059e59c700 1 -- 172.21.15.16:0/2142745400 >> v2:172.21.15.16:3300/0 conn(0x7f05a0133fa0 msgr2=0x7f05a0134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:55.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.158+0000 7f059e59c700 1 --2- 172.21.15.16:0/2142745400 >> v2:172.21.15.16:3300/0 conn(0x7f05a0133fa0 0x7f05a0134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:55.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.158+0000 7f059e59c700 1 -- 172.21.15.16:0/2142745400 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a00a1fe0 con 0x7f05a00a1340 2023-09-30T22:34:55.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.158+0000 7f059f59e700 1 --2- 172.21.15.16:0/2142745400 >> v2:172.21.15.16:3300/0 conn(0x7f05a0133fa0 0x7f05a0134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:55.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.159+0000 7f059e59c700 1 --2- 172.21.15.16:0/2142745400 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f05a00a1340 0x7f05a0133a60 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f05880098f0 tx=0x7f0588009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7914f5eeba25e8a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:55.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.159+0000 7f059dd9b700 1 -- 172.21.15.16:0/2142745400 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0588011830 con 0x7f05a00a1340 2023-09-30T22:34:55.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.159+0000 7f059dd9b700 1 -- 172.21.15.16:0/2142745400 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0588011990 con 0x7f05a00a1340 2023-09-30T22:34:55.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.159+0000 7f059dd9b700 1 -- 172.21.15.16:0/2142745400 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0588011d20 con 0x7f05a00a1340 2023-09-30T22:34:55.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.159+0000 7f05a50eb700 1 -- 172.21.15.16:0/2142745400 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f05a00a1340 msgr2=0x7f05a0133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:55.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.159+0000 7f05a50eb700 1 --2- 172.21.15.16:0/2142745400 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f05a00a1340 0x7f05a0133a60 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f05880098f0 tx=0x7f0588009c00 comp rx=0 tx=0).stop 2023-09-30T22:34:55.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.159+0000 7f05a50eb700 1 -- 172.21.15.16:0/2142745400 shutdown_connections 2023-09-30T22:34:55.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.159+0000 7f05a50eb700 1 --2- 172.21.15.16:0/2142745400 >> v2:172.21.15.16:3300/0 conn(0x7f05a0133fa0 0x7f05a0134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:55.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.159+0000 7f05a50eb700 1 --2- 172.21.15.16:0/2142745400 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f05a00a0a30 0x7f05a00a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:55.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.159+0000 7f05a50eb700 1 --2- 172.21.15.16:0/2142745400 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f05a00a1340 0x7f05a0133a60 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:55.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.159+0000 7f05a50eb700 1 -- 172.21.15.16:0/2142745400 >> 172.21.15.16:0/2142745400 conn(0x7f05a00a6550 msgr2=0x7f05a00a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:55.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.159+0000 7f05a50eb700 1 -- 172.21.15.16:0/2142745400 shutdown_connections 2023-09-30T22:34:55.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.159+0000 7f05a50eb700 1 -- 172.21.15.16:0/2142745400 wait complete. 2023-09-30T22:34:55.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.161+0000 7f05a50eb700 1 Processor -- start 2023-09-30T22:34:55.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.161+0000 7f05a50eb700 1 -- start start 2023-09-30T22:34:55.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.161+0000 7f05a50eb700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f05a00a0a30 0x7f05a0120130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:55.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.161+0000 7f05a50eb700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f05a00a1340 0x7f05a0120670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:55.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.161+0000 7f05a50eb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05a0133fa0 0x7f05a0120bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:55.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.161+0000 7f05a50eb700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a0123c50 con 0x7f05a0133fa0 2023-09-30T22:34:55.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.161+0000 7f05a50eb700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05a00a3530 con 0x7f05a00a1340 2023-09-30T22:34:55.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.161+0000 7f05a50eb700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f05a01233e0 con 0x7f05a00a0a30 2023-09-30T22:34:55.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.161+0000 7f059e59c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f05a00a1340 0x7f05a0120670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:55.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.161+0000 7f059f59e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05a0133fa0 0x7f05a0120bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:55.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.161+0000 7f059e59c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f05a00a1340 0x7f05a0120670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:42336/0 (socket says 172.21.15.16:42336) 2023-09-30T22:34:55.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.161+0000 7f059e59c700 1 -- 172.21.15.16:0/2179364159 learned_addr learned my addr 172.21.15.16:0/2179364159 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:55.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.161+0000 7f059ed9d700 1 --2- 172.21.15.16:0/2179364159 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f05a00a0a30 0x7f05a0120130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:55.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.162+0000 7f059f59e700 1 -- 172.21.15.16:0/2179364159 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f05a00a0a30 msgr2=0x7f05a0120130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:55.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.162+0000 7f059f59e700 1 --2- 172.21.15.16:0/2179364159 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f05a00a0a30 0x7f05a0120130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:55.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.162+0000 7f059f59e700 1 -- 172.21.15.16:0/2179364159 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f05a00a1340 msgr2=0x7f05a0120670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:55.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.162+0000 7f059f59e700 1 --2- 172.21.15.16:0/2179364159 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f05a00a1340 0x7f05a0120670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:55.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.162+0000 7f059f59e700 1 -- 172.21.15.16:0/2179364159 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0598009a70 con 0x7f05a0133fa0 2023-09-30T22:34:55.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.162+0000 7f059f59e700 1 --2- 172.21.15.16:0/2179364159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05a0133fa0 0x7f05a0120bb0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f059800efc0 tx=0x7f059800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:55.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.162+0000 7f05977fe700 1 -- 172.21.15.16:0/2179364159 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059800ca90 con 0x7f05a0133fa0 2023-09-30T22:34:55.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.162+0000 7f05977fe700 1 -- 172.21.15.16:0/2179364159 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f059800f040 con 0x7f05a0133fa0 2023-09-30T22:34:55.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.162+0000 7f05a50eb700 1 -- 172.21.15.16:0/2179364159 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0588009550 con 0x7f05a0133fa0 2023-09-30T22:34:55.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.162+0000 7f05977fe700 1 -- 172.21.15.16:0/2179364159 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059800ce30 con 0x7f05a0133fa0 2023-09-30T22:34:55.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.162+0000 7f05a50eb700 1 -- 172.21.15.16:0/2179364159 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05a0121d40 con 0x7f05a0133fa0 2023-09-30T22:34:55.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.163+0000 7f05a50eb700 1 -- 172.21.15.16:0/2179364159 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05a0122ff0 con 0x7f05a0133fa0 2023-09-30T22:34:55.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.163+0000 7f05977fe700 1 -- 172.21.15.16:0/2179364159 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0598003a20 con 0x7f05a0133fa0 2023-09-30T22:34:55.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.164+0000 7f05977fe700 1 -- 172.21.15.16:0/2179364159 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f05980405e0 con 0x7f05a0133fa0 2023-09-30T22:34:55.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.168+0000 7f05977fe700 1 -- 172.21.15.16:0/2179364159 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f059801c7c0 con 0x7f05a0133fa0 2023-09-30T22:34:55.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.371+0000 7f05a50eb700 1 -- 172.21.15.16:0/2179364159 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f05a0074ea0 con 0x7f05a0133fa0 2023-09-30T22:34:55.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.371+0000 7f05977fe700 1 -- 172.21.15.16:0/2179364159 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f059802b090 con 0x7f05a0133fa0 2023-09-30T22:34:55.372 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:34:55.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.373+0000 7f05a50eb700 1 -- 172.21.15.16:0/2179364159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05a0133fa0 msgr2=0x7f05a0120bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:55.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.373+0000 7f05a50eb700 1 --2- 172.21.15.16:0/2179364159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05a0133fa0 0x7f05a0120bb0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f059800efc0 tx=0x7f059800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:34:55.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.373+0000 7f05a50eb700 1 -- 172.21.15.16:0/2179364159 shutdown_connections 2023-09-30T22:34:55.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.373+0000 7f05a50eb700 1 --2- 172.21.15.16:0/2179364159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05a0133fa0 0x7f05a0120bb0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:55.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.373+0000 7f05a50eb700 1 --2- 172.21.15.16:0/2179364159 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f05a00a0a30 0x7f05a0120130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:55.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.373+0000 7f05a50eb700 1 --2- 172.21.15.16:0/2179364159 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f05a00a1340 0x7f05a0120670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:55.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.373+0000 7f05a50eb700 1 -- 172.21.15.16:0/2179364159 >> 172.21.15.16:0/2179364159 conn(0x7f05a00a6550 msgr2=0x7f05a01bb0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:55.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.373+0000 7f05a50eb700 1 -- 172.21.15.16:0/2179364159 shutdown_connections 2023-09-30T22:34:55.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.373+0000 7f05a50eb700 1 -- 172.21.15.16:0/2179364159 wait complete. 2023-09-30T22:34:55.388 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:34:55.421 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:34:55.435 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:34:55.438 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:34:55.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.566+0000 7f28fb032700 1 Processor -- start 2023-09-30T22:34:55.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.566+0000 7f28fb032700 1 -- start start 2023-09-30T22:34:55.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.566+0000 7f28fb032700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f28f40a0a30 0x7f28f40a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:55.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.566+0000 7f28fb032700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f28f40a1340 0x7f28f4133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:55.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.566+0000 7f28fb032700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f28f4133fa0 0x7f28f4134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:55.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.566+0000 7f28fb032700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28f409edd0 con 0x7f28f4133fa0 2023-09-30T22:34:55.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.566+0000 7f28fb032700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28f409e1f0 con 0x7f28f40a0a30 2023-09-30T22:34:55.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.566+0000 7f28fb032700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f28f409dc50 con 0x7f28f40a1340 2023-09-30T22:34:55.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.567+0000 7f28f8dce700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f28f40a0a30 0x7f28f40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:55.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.567+0000 7f28f3fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f28f40a1340 0x7f28f4133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:55.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.567+0000 7f28f3fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f28f40a1340 0x7f28f4133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:39004/0 (socket says 172.21.15.16:39004) 2023-09-30T22:34:55.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.567+0000 7f28f3fff700 1 -- 172.21.15.16:0/2604974913 learned_addr learned my addr 172.21.15.16:0/2604974913 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:55.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.567+0000 7f28f8dce700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f28f40a0a30 0x7f28f40a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:42342/0 (socket says 172.21.15.16:42342) 2023-09-30T22:34:55.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.567+0000 7f28f95cf700 1 --2- 172.21.15.16:0/2604974913 >> v2:172.21.15.16:3300/0 conn(0x7f28f4133fa0 0x7f28f4134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:55.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.567+0000 7f28f3fff700 1 -- 172.21.15.16:0/2604974913 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f28f40a0a30 msgr2=0x7f28f40a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:55.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.567+0000 7f28f3fff700 1 --2- 172.21.15.16:0/2604974913 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f28f40a0a30 0x7f28f40a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:55.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.567+0000 7f28f3fff700 1 -- 172.21.15.16:0/2604974913 >> v2:172.21.15.16:3300/0 conn(0x7f28f4133fa0 msgr2=0x7f28f4134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:55.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.567+0000 7f28f3fff700 1 --2- 172.21.15.16:0/2604974913 >> v2:172.21.15.16:3300/0 conn(0x7f28f4133fa0 0x7f28f4134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:55.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.567+0000 7f28f3fff700 1 -- 172.21.15.16:0/2604974913 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f40a1fe0 con 0x7f28f40a1340 2023-09-30T22:34:55.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.567+0000 7f28f3fff700 1 --2- 172.21.15.16:0/2604974913 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f28f40a1340 0x7f28f4133a60 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f28e00098f0 tx=0x7f28e0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=332472c1ef4329d6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:55.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.567+0000 7f28f37fe700 1 -- 172.21.15.16:0/2604974913 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e0011830 con 0x7f28f40a1340 2023-09-30T22:34:55.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.567+0000 7f28f37fe700 1 -- 172.21.15.16:0/2604974913 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28e0011990 con 0x7f28f40a1340 2023-09-30T22:34:55.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.567+0000 7f28f37fe700 1 -- 172.21.15.16:0/2604974913 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e0011d20 con 0x7f28f40a1340 2023-09-30T22:34:55.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.568+0000 7f28fb032700 1 -- 172.21.15.16:0/2604974913 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f28f40a1340 msgr2=0x7f28f4133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:55.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.568+0000 7f28fb032700 1 --2- 172.21.15.16:0/2604974913 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f28f40a1340 0x7f28f4133a60 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f28e00098f0 tx=0x7f28e0009c00 comp rx=0 tx=0).stop 2023-09-30T22:34:55.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.569+0000 7f28fb032700 1 -- 172.21.15.16:0/2604974913 shutdown_connections 2023-09-30T22:34:55.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.569+0000 7f28fb032700 1 --2- 172.21.15.16:0/2604974913 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f28f40a0a30 0x7f28f40a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:55.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.569+0000 7f28fb032700 1 --2- 172.21.15.16:0/2604974913 >> v2:172.21.15.16:3300/0 conn(0x7f28f4133fa0 0x7f28f4134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:55.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.569+0000 7f28fb032700 1 --2- 172.21.15.16:0/2604974913 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f28f40a1340 0x7f28f4133a60 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:55.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.569+0000 7f28fb032700 1 -- 172.21.15.16:0/2604974913 >> 172.21.15.16:0/2604974913 conn(0x7f28f40a6550 msgr2=0x7f28f40a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:55.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.569+0000 7f28fb032700 1 -- 172.21.15.16:0/2604974913 shutdown_connections 2023-09-30T22:34:55.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.569+0000 7f28fb032700 1 -- 172.21.15.16:0/2604974913 wait complete. 2023-09-30T22:34:55.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.570+0000 7f28fb032700 1 Processor -- start 2023-09-30T22:34:55.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.570+0000 7f28fb032700 1 -- start start 2023-09-30T22:34:55.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.571+0000 7f28fb032700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f28f40a0a30 0x7f28f41213e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:55.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.571+0000 7f28fb032700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28f40a1340 0x7f28f4121920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:55.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.571+0000 7f28fb032700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f28f4133fa0 0x7f28f4121e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:55.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.571+0000 7f28fb032700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f4124fa0 con 0x7f28f40a1340 2023-09-30T22:34:55.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.571+0000 7f28fb032700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28f41238c0 con 0x7f28f40a0a30 2023-09-30T22:34:55.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.571+0000 7f28fb032700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f28f40a0220 con 0x7f28f4133fa0 2023-09-30T22:34:55.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.571+0000 7f28f8dce700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f28f40a0a30 0x7f28f41213e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:55.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.571+0000 7f28f3fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28f40a1340 0x7f28f4121920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:55.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.571+0000 7f28f8dce700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f28f40a0a30 0x7f28f41213e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:42348/0 (socket says 172.21.15.16:42348) 2023-09-30T22:34:55.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.571+0000 7f28f8dce700 1 -- 172.21.15.16:0/4061515278 learned_addr learned my addr 172.21.15.16:0/4061515278 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:55.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.571+0000 7f28f95cf700 1 --2- 172.21.15.16:0/4061515278 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f28f4133fa0 0x7f28f4121e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:55.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.572+0000 7f28f8dce700 1 -- 172.21.15.16:0/4061515278 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f28f4133fa0 msgr2=0x7f28f4121e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:55.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.572+0000 7f28f8dce700 1 --2- 172.21.15.16:0/4061515278 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f28f4133fa0 0x7f28f4121e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:55.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.572+0000 7f28f8dce700 1 -- 172.21.15.16:0/4061515278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28f40a1340 msgr2=0x7f28f4121920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:55.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.572+0000 7f28f8dce700 1 --2- 172.21.15.16:0/4061515278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28f40a1340 0x7f28f4121920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:55.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.572+0000 7f28f8dce700 1 -- 172.21.15.16:0/4061515278 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e0009550 con 0x7f28f40a0a30 2023-09-30T22:34:55.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.572+0000 7f28f95cf700 1 --2- 172.21.15.16:0/4061515278 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f28f4133fa0 0x7f28f4121e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:55.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.572+0000 7f28f8dce700 1 --2- 172.21.15.16:0/4061515278 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f28f40a0a30 0x7f28f41213e0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f28e800e870 tx=0x7f28e800eb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:55.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.572+0000 7f28f17fa700 1 -- 172.21.15.16:0/4061515278 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e8013680 con 0x7f28f40a0a30 2023-09-30T22:34:55.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.572+0000 7f28f17fa700 1 -- 172.21.15.16:0/4061515278 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28e80137e0 con 0x7f28f40a0a30 2023-09-30T22:34:55.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.572+0000 7f28fb032700 1 -- 172.21.15.16:0/4061515278 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28f40a36e0 con 0x7f28f40a0a30 2023-09-30T22:34:55.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.572+0000 7f28f17fa700 1 -- 172.21.15.16:0/4061515278 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e8013b50 con 0x7f28f40a0a30 2023-09-30T22:34:55.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.572+0000 7f28fb032700 1 -- 172.21.15.16:0/4061515278 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28f4124c20 con 0x7f28f40a0a30 2023-09-30T22:34:55.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.573+0000 7f28fb032700 1 -- 172.21.15.16:0/4061515278 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28f4074ea0 con 0x7f28f40a0a30 2023-09-30T22:34:55.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.574+0000 7f28f17fa700 1 -- 172.21.15.16:0/4061515278 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f28e8013cb0 con 0x7f28f40a0a30 2023-09-30T22:34:55.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.574+0000 7f28f17fa700 1 -- 172.21.15.16:0/4061515278 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f28e803fd90 con 0x7f28f40a0a30 2023-09-30T22:34:55.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.583+0000 7f28f17fa700 1 -- 172.21.15.16:0/4061515278 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f28e8011d90 con 0x7f28f40a0a30 2023-09-30T22:34:55.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.610+0000 7fe81c9d2700 1 Processor -- start 2023-09-30T22:34:55.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.611+0000 7fe81c9d2700 1 -- start start 2023-09-30T22:34:55.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.612+0000 7fe81c9d2700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe81809d630 0x7fe8180a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:55.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.612+0000 7fe81c9d2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe818138d80 0x7fe81813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:55.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.612+0000 7fe81c9d2700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8180a2a70 con 0x7fe8180a0a30 2023-09-30T22:34:55.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.612+0000 7fe81c9d2700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8180a01d0 con 0x7fe81809d630 2023-09-30T22:34:55.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.612+0000 7fe81c9d2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe81809e2b0 con 0x7fe818138d80 2023-09-30T22:34:55.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.612+0000 7fe815d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe81809d630 0x7fe8180a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:55.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.612+0000 7fe81659c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe8180a0a30 0x7fe8180a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60970/0 (socket says 172.21.15.16:60970) 2023-09-30T22:34:55.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.612+0000 7fe815d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe81809d630 0x7fe8180a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34678/0 (socket says 172.21.15.16:34678) 2023-09-30T22:34:55.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.612+0000 7fe81659c700 1 -- 172.21.15.16:0/616597077 learned_addr learned my addr 172.21.15.16:0/616597077 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:34:55.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.612+0000 7fe816d9d700 1 --2- 172.21.15.16:0/616597077 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe818138d80 0x7fe81813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:55.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.612+0000 7fe815d9b700 1 -- 172.21.15.16:0/616597077 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe818138d80 msgr2=0x7fe81813b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:55.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.612+0000 7fe815d9b700 1 --2- 172.21.15.16:0/616597077 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe818138d80 0x7fe81813b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:55.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.612+0000 7fe815d9b700 1 -- 172.21.15.16:0/616597077 >> v1:172.21.15.16:6789/0 conn(0x7fe8180a0a30 legacy=0x7fe8180a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:55.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.612+0000 7fe81559a700 1 -- 172.21.15.16:0/616597077 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 23838270 0 0) 0x7fe8180a2a70 con 0x7fe8180a0a30 2023-09-30T22:34:55.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.612+0000 7fe815d9b700 1 -- 172.21.15.16:0/616597077 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe81809dc50 con 0x7fe81809d630 2023-09-30T22:34:55.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.612+0000 7fe816d9d700 1 --2- 172.21.15.16:0/616597077 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe818138d80 0x7fe81813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:55.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.612+0000 7fe815d9b700 1 --2- 172.21.15.16:0/616597077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe81809d630 0x7fe8180a1510 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fe80c009840 tx=0x7fe80c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c64ee91db420afb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:55.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.613+0000 7fe81559a700 1 -- 172.21.15.16:0/616597077 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80c011830 con 0x7fe81809d630 2023-09-30T22:34:55.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.613+0000 7fe81559a700 1 -- 172.21.15.16:0/616597077 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe80c011990 con 0x7fe81809d630 2023-09-30T22:34:55.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.613+0000 7fe81559a700 1 -- 172.21.15.16:0/616597077 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80c011d20 con 0x7fe81809d630 2023-09-30T22:34:55.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.613+0000 7fe81c9d2700 1 -- 172.21.15.16:0/616597077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe81809d630 msgr2=0x7fe8180a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:55.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.613+0000 7fe81c9d2700 1 --2- 172.21.15.16:0/616597077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe81809d630 0x7fe8180a1510 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fe80c009840 tx=0x7fe80c009b50 comp rx=0 tx=0).stop 2023-09-30T22:34:55.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.613+0000 7fe81c9d2700 1 -- 172.21.15.16:0/616597077 shutdown_connections 2023-09-30T22:34:55.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.613+0000 7fe81c9d2700 1 --2- 172.21.15.16:0/616597077 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe818138d80 0x7fe81813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:55.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.613+0000 7fe81c9d2700 1 --2- 172.21.15.16:0/616597077 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe81809d630 0x7fe8180a1510 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:55.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.613+0000 7fe81c9d2700 1 -- 172.21.15.16:0/616597077 >> 172.21.15.16:0/616597077 conn(0x7fe8180a3050 msgr2=0x7fe8180a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:55.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.613+0000 7fe81c9d2700 1 -- 172.21.15.16:0/616597077 shutdown_connections 2023-09-30T22:34:55.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.614+0000 7fe81c9d2700 1 -- 172.21.15.16:0/616597077 wait complete. 2023-09-30T22:34:55.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.614+0000 7f3760925700 1 Processor -- start 2023-09-30T22:34:55.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.614+0000 7f3760925700 1 -- start start 2023-09-30T22:34:55.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.615+0000 7f3760925700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f375c09d6f0 0x7f375c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:55.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.615+0000 7f3760925700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f375c09e000 0x7f375c130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:55.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.615+0000 7f3760925700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f375c09f130 con 0x7f375c09e410 2023-09-30T22:34:55.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.615+0000 7f3760925700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f375c09f520 con 0x7f375c09e000 2023-09-30T22:34:55.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.615+0000 7f3760925700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f375c09e930 con 0x7f375c09d6f0 2023-09-30T22:34:55.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.615+0000 7f375a59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f375c09d6f0 0x7f375c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:55.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.615+0000 7f375a59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f375c09d6f0 0x7f375c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:39028/0 (socket says 172.21.15.16:39028) 2023-09-30T22:34:55.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.615+0000 7f375a59c700 1 -- 172.21.15.16:0/1509923047 learned_addr learned my addr 172.21.15.16:0/1509923047 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:55.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.615+0000 7f375a59c700 1 -- 172.21.15.16:0/1509923047 >> v2:172.21.15.16:3300/0 conn(0x7f375c09e000 msgr2=0x7f375c130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:55.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.615+0000 7f375a59c700 1 --2- 172.21.15.16:0/1509923047 >> v2:172.21.15.16:3300/0 conn(0x7f375c09e000 0x7f375c130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:55.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.615+0000 7f375a59c700 1 -- 172.21.15.16:0/1509923047 >> v1:172.21.15.16:6789/0 conn(0x7f375c09e410 legacy=0x7f375c130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:55.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.615+0000 7f375a59c700 1 -- 172.21.15.16:0/1509923047 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f375c09b2d0 con 0x7f375c09d6f0 2023-09-30T22:34:55.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.615+0000 7fe81c9d2700 1 Processor -- start 2023-09-30T22:34:55.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.615+0000 7f375a59c700 1 --2- 172.21.15.16:0/1509923047 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f375c09d6f0 0x7f375c09dac0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f374c00bb90 tx=0x7f374c00bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ebf826503161372 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:55.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.615+0000 7fe81c9d2700 1 -- start start 2023-09-30T22:34:55.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.616+0000 7f375959a700 1 -- 172.21.15.16:0/1509923047 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374c00dd10 con 0x7f375c09d6f0 2023-09-30T22:34:55.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.616+0000 7f375959a700 1 -- 172.21.15.16:0/1509923047 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f374c00de70 con 0x7f375c09d6f0 2023-09-30T22:34:55.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.616+0000 7f375959a700 1 -- 172.21.15.16:0/1509923047 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374c004230 con 0x7f375c09d6f0 2023-09-30T22:34:55.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.616+0000 7fe81c9d2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe81809d630 0x7fe818133700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:55.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.616+0000 7fe81c9d2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8180a0a30 0x7fe818133c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:55.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.616+0000 7fe81c9d2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe818138d80 0x7fe818134180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:55.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.616+0000 7fe81c9d2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe818136190 con 0x7fe8180a0a30 2023-09-30T22:34:55.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.616+0000 7fe81c9d2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe818136eb0 con 0x7fe81809d630 2023-09-30T22:34:55.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.616+0000 7fe81c9d2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe8180a4ba0 con 0x7fe818138d80 2023-09-30T22:34:55.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.616+0000 7f3760925700 1 -- 172.21.15.16:0/1509923047 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f375c09d6f0 msgr2=0x7f375c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:55.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.616+0000 7f3760925700 1 --2- 172.21.15.16:0/1509923047 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f375c09d6f0 0x7f375c09dac0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f374c00bb90 tx=0x7f374c00bea0 comp rx=0 tx=0).stop 2023-09-30T22:34:55.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.616+0000 7fe815d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8180a0a30 0x7fe818133c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:55.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.616+0000 7fe816d9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe818138d80 0x7fe818134180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:55.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.616+0000 7fe815d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8180a0a30 0x7fe818133c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34692/0 (socket says 172.21.15.16:34692) 2023-09-30T22:34:55.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.616+0000 7fe815d9b700 1 -- 172.21.15.16:0/1387000416 learned_addr learned my addr 172.21.15.16:0/1387000416 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:55.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.617+0000 7f3760925700 1 -- 172.21.15.16:0/1509923047 shutdown_connections 2023-09-30T22:34:55.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.617+0000 7f3760925700 1 --2- 172.21.15.16:0/1509923047 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f375c09d6f0 0x7f375c09dac0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:55.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.617+0000 7f3760925700 1 --2- 172.21.15.16:0/1509923047 >> v2:172.21.15.16:3300/0 conn(0x7f375c09e000 0x7f375c130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:55.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.617+0000 7f3760925700 1 -- 172.21.15.16:0/1509923047 >> 172.21.15.16:0/1509923047 conn(0x7f375c066f50 msgr2=0x7f375c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:55.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.617+0000 7fe816d9d700 1 -- 172.21.15.16:0/1387000416 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe81809d630 msgr2=0x7fe818133700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:55.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.617+0000 7fe816d9d700 1 --2- 172.21.15.16:0/1387000416 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe81809d630 0x7fe818133700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:55.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.617+0000 7fe816d9d700 1 -- 172.21.15.16:0/1387000416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8180a0a30 msgr2=0x7fe818133c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:55.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.617+0000 7fe816d9d700 1 --2- 172.21.15.16:0/1387000416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8180a0a30 0x7fe818133c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:55.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.617+0000 7fe816d9d700 1 -- 172.21.15.16:0/1387000416 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe810009a70 con 0x7fe818138d80 2023-09-30T22:34:55.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.617+0000 7f3760925700 1 -- 172.21.15.16:0/1509923047 shutdown_connections 2023-09-30T22:34:55.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.617+0000 7f3760925700 1 -- 172.21.15.16:0/1509923047 wait complete. 2023-09-30T22:34:55.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.617+0000 7fe816d9d700 1 --2- 172.21.15.16:0/1387000416 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe818138d80 0x7fe818134180 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fe81000efc0 tx=0x7fe81000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:55.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.617+0000 7fe806ffd700 1 -- 172.21.15.16:0/1387000416 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81000cd40 con 0x7fe818138d80 2023-09-30T22:34:55.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.617+0000 7fe806ffd700 1 -- 172.21.15.16:0/1387000416 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe81000f040 con 0x7fe818138d80 2023-09-30T22:34:55.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.617+0000 7fe81c9d2700 1 -- 172.21.15.16:0/1387000416 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe80c0094a0 con 0x7fe818138d80 2023-09-30T22:34:55.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.617+0000 7fe806ffd700 1 -- 172.21.15.16:0/1387000416 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8100146c0 con 0x7fe818138d80 2023-09-30T22:34:55.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.617+0000 7fe81c9d2700 1 -- 172.21.15.16:0/1387000416 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe818136540 con 0x7fe818138d80 2023-09-30T22:34:55.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.618+0000 7f3760925700 1 Processor -- start 2023-09-30T22:34:55.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.618+0000 7f3760925700 1 -- start start 2023-09-30T22:34:55.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.618+0000 7fe806ffd700 1 -- 172.21.15.16:0/1387000416 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe810014820 con 0x7fe818138d80 2023-09-30T22:34:55.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.619+0000 7fe806ffd700 1 -- 172.21.15.16:0/1387000416 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fe810040360 con 0x7fe818138d80 2023-09-30T22:34:55.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.619+0000 7fe81c9d2700 1 -- 172.21.15.16:0/1387000416 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8180747f0 con 0x7fe818138d80 2023-09-30T22:34:55.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.619+0000 7f3760925700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f375c09d6f0 0x7f375c1b81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:55.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.619+0000 7f3760925700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f375c09e000 0x7f375c1b86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:55.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.619+0000 7f3760925700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f375c09e410 0x7f375c1cc760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:55.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.619+0000 7f3760925700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375c1b8c20 con 0x7f375c09d6f0 2023-09-30T22:34:55.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.619+0000 7f3760925700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f375c09c700 con 0x7f375c09e410 2023-09-30T22:34:55.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.619+0000 7f3760925700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f375c1b7c30 con 0x7f375c09e000 2023-09-30T22:34:55.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.619+0000 7f375a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f375c09d6f0 0x7f375c1b81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:55.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.619+0000 7f375a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f375c09d6f0 0x7f375c1b81a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34702/0 (socket says 172.21.15.16:34702) 2023-09-30T22:34:55.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.619+0000 7f375a59c700 1 -- 172.21.15.16:0/1803170706 learned_addr learned my addr 172.21.15.16:0/1803170706 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:55.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.619+0000 7f3759d9b700 1 --2- 172.21.15.16:0/1803170706 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f375c09e000 0x7f375c1b86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:55.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.619+0000 7f375a59c700 1 -- 172.21.15.16:0/1803170706 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f375c09e000 msgr2=0x7f375c1b86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:55.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.619+0000 7f375a59c700 1 --2- 172.21.15.16:0/1803170706 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f375c09e000 0x7f375c1b86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:55.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.619+0000 7f375a59c700 1 -- 172.21.15.16:0/1803170706 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f375c09e410 msgr2=0x7f375c1cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:55.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.619+0000 7f375a59c700 1 --2- 172.21.15.16:0/1803170706 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f375c09e410 0x7f375c1cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:55.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.619+0000 7f375a59c700 1 -- 172.21.15.16:0/1803170706 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f374c00b7f0 con 0x7f375c09d6f0 2023-09-30T22:34:55.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.620+0000 7f3759d9b700 1 --2- 172.21.15.16:0/1803170706 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f375c09e000 0x7f375c1b86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:55.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.620+0000 7f375a59c700 1 --2- 172.21.15.16:0/1803170706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f375c09d6f0 0x7f375c1b81a0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f375c09ea70 tx=0x7f374c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:55.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.620+0000 7f374affd700 1 -- 172.21.15.16:0/1803170706 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374c00dcc0 con 0x7f375c09d6f0 2023-09-30T22:34:55.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.620+0000 7f374affd700 1 -- 172.21.15.16:0/1803170706 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f374c010470 con 0x7f375c09d6f0 2023-09-30T22:34:55.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.620+0000 7f3760925700 1 -- 172.21.15.16:0/1803170706 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f375c1bab40 con 0x7f375c09d6f0 2023-09-30T22:34:55.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.620+0000 7f374affd700 1 -- 172.21.15.16:0/1803170706 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374c0107e0 con 0x7f375c09d6f0 2023-09-30T22:34:55.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.620+0000 7f3760925700 1 -- 172.21.15.16:0/1803170706 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f375c1b9480 con 0x7f375c09d6f0 2023-09-30T22:34:55.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.621+0000 7f374affd700 1 -- 172.21.15.16:0/1803170706 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f374c0193f0 con 0x7f375c09d6f0 2023-09-30T22:34:55.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.621+0000 7f374affd700 1 -- 172.21.15.16:0/1803170706 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f374c041620 con 0x7f375c09d6f0 2023-09-30T22:34:55.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.621+0000 7f3760925700 1 -- 172.21.15.16:0/1803170706 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f375c074ea0 con 0x7f375c09d6f0 2023-09-30T22:34:55.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.626+0000 7f374affd700 1 -- 172.21.15.16:0/1803170706 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f374c018990 con 0x7f375c09d6f0 2023-09-30T22:34:55.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.628+0000 7fe806ffd700 1 -- 172.21.15.16:0/1387000416 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe81001d400 con 0x7fe818138d80 2023-09-30T22:34:55.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.803+0000 7f28fb032700 1 -- 172.21.15.16:0/4061515278 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f28f40a28e0 con 0x7f28f40a0a30 2023-09-30T22:34:55.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.803+0000 7f28f17fa700 1 -- 172.21.15.16:0/4061515278 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f28e8021070 con 0x7f28f40a0a30 2023-09-30T22:34:55.804 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:34:55.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.805+0000 7f28fb032700 1 -- 172.21.15.16:0/4061515278 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f28f40a0a30 msgr2=0x7f28f41213e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:55.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.805+0000 7f28fb032700 1 --2- 172.21.15.16:0/4061515278 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f28f40a0a30 0x7f28f41213e0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f28e800e870 tx=0x7f28e800eb80 comp rx=0 tx=0).stop 2023-09-30T22:34:55.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.806+0000 7f28fb032700 1 -- 172.21.15.16:0/4061515278 shutdown_connections 2023-09-30T22:34:55.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.806+0000 7f28fb032700 1 --2- 172.21.15.16:0/4061515278 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f28f40a0a30 0x7f28f41213e0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:55.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.806+0000 7f28fb032700 1 --2- 172.21.15.16:0/4061515278 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f28f4133fa0 0x7f28f4121e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:55.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.806+0000 7f28fb032700 1 --2- 172.21.15.16:0/4061515278 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28f40a1340 0x7f28f4121920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:55.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.806+0000 7f28fb032700 1 -- 172.21.15.16:0/4061515278 >> 172.21.15.16:0/4061515278 conn(0x7f28f40a6550 msgr2=0x7f28f40a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:55.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.806+0000 7f28fb032700 1 -- 172.21.15.16:0/4061515278 shutdown_connections 2023-09-30T22:34:55.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.806+0000 7f28fb032700 1 -- 172.21.15.16:0/4061515278 wait complete. 2023-09-30T22:34:55.820 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:34:55.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.838+0000 7f3760925700 1 -- 172.21.15.16:0/1803170706 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f375c06c360 con 0x7f375c09d6f0 2023-09-30T22:34:55.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.838+0000 7f374affd700 1 -- 172.21.15.16:0/1803170706 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f374c00f3d0 con 0x7f375c09d6f0 2023-09-30T22:34:55.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.839+0000 7fe81c9d2700 1 -- 172.21.15.16:0/1387000416 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe818137980 con 0x7fe818138d80 2023-09-30T22:34:55.839 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:34:55.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.839+0000 7fe806ffd700 1 -- 172.21.15.16:0/1387000416 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe81002b090 con 0x7fe818138d80 2023-09-30T22:34:55.840 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:34:55.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.841+0000 7f3760925700 1 -- 172.21.15.16:0/1803170706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f375c09d6f0 msgr2=0x7f375c1b81a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:55.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.841+0000 7f3760925700 1 --2- 172.21.15.16:0/1803170706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f375c09d6f0 0x7f375c1b81a0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f375c09ea70 tx=0x7f374c004750 comp rx=0 tx=0).stop 2023-09-30T22:34:55.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.841+0000 7f3760925700 1 -- 172.21.15.16:0/1803170706 shutdown_connections 2023-09-30T22:34:55.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.841+0000 7f3760925700 1 --2- 172.21.15.16:0/1803170706 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f375c09e410 0x7f375c1cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:55.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.841+0000 7f3760925700 1 --2- 172.21.15.16:0/1803170706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f375c09d6f0 0x7f375c1b81a0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:55.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.841+0000 7f3760925700 1 --2- 172.21.15.16:0/1803170706 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f375c09e000 0x7f375c1b86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:55.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.841+0000 7f3760925700 1 -- 172.21.15.16:0/1803170706 >> 172.21.15.16:0/1803170706 conn(0x7f375c066f50 msgr2=0x7f375c06f140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:55.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.841+0000 7fe81c9d2700 1 -- 172.21.15.16:0/1387000416 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe818138d80 msgr2=0x7fe818134180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:55.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.841+0000 7fe81c9d2700 1 --2- 172.21.15.16:0/1387000416 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe818138d80 0x7fe818134180 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fe81000efc0 tx=0x7fe81000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:34:55.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.841+0000 7f3760925700 1 -- 172.21.15.16:0/1803170706 shutdown_connections 2023-09-30T22:34:55.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.841+0000 7f3760925700 1 -- 172.21.15.16:0/1803170706 wait complete. 2023-09-30T22:34:55.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.841+0000 7fe81c9d2700 1 -- 172.21.15.16:0/1387000416 shutdown_connections 2023-09-30T22:34:55.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.841+0000 7fe81c9d2700 1 --2- 172.21.15.16:0/1387000416 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe81809d630 0x7fe818133700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:55.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.841+0000 7fe81c9d2700 1 --2- 172.21.15.16:0/1387000416 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe818138d80 0x7fe818134180 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:55.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.841+0000 7fe81c9d2700 1 --2- 172.21.15.16:0/1387000416 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe8180a0a30 0x7fe818133c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:55.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.841+0000 7fe81c9d2700 1 -- 172.21.15.16:0/1387000416 >> 172.21.15.16:0/1387000416 conn(0x7fe8180a3050 msgr2=0x7fe81809ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:55.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.841+0000 7fe81c9d2700 1 -- 172.21.15.16:0/1387000416 shutdown_connections 2023-09-30T22:34:55.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:55.841+0000 7fe81c9d2700 1 -- 172.21.15.16:0/1387000416 wait complete. 2023-09-30T22:34:55.857 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:34:55.858 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:34:56.389 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:34:56.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.535+0000 7f37b20dd700 1 Processor -- start 2023-09-30T22:34:56.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.535+0000 7f37b20dd700 1 -- start start 2023-09-30T22:34:56.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.535+0000 7f37b20dd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f37ac133490 0x7f37ac135870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:56.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.535+0000 7f37b20dd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f37ac135db0 0x7f37ac138190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:56.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.535+0000 7f37b20dd700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37ac09c190 con 0x7f37ac130750 2023-09-30T22:34:56.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.535+0000 7f37b20dd700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37ac09b090 con 0x7f37ac133490 2023-09-30T22:34:56.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.535+0000 7f37b20dd700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f37ac09a6c0 con 0x7f37ac135db0 2023-09-30T22:34:56.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.535+0000 7f37ab7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f37ac130750 0x7f37ac130b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60988/0 (socket says 172.21.15.16:60988) 2023-09-30T22:34:56.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.535+0000 7f37ab7fe700 1 -- 172.21.15.16:0/983806399 learned_addr learned my addr 172.21.15.16:0/983806399 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:34:56.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.535+0000 7f37abfff700 1 --2- 172.21.15.16:0/983806399 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f37ac135db0 0x7f37ac138190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:56.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.535+0000 7f37aaffd700 1 --2- 172.21.15.16:0/983806399 >> v2:172.21.15.16:3300/0 conn(0x7f37ac133490 0x7f37ac135870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:56.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.536+0000 7f37aa7fc700 1 -- 172.21.15.16:0/983806399 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1566952907 0 0) 0x7f37ac09c190 con 0x7f37ac130750 2023-09-30T22:34:56.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.536+0000 7f37aa7fc700 1 -- 172.21.15.16:0/983806399 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37980035b0 con 0x7f37ac130750 2023-09-30T22:34:56.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.536+0000 7f37aaffd700 1 -- 172.21.15.16:0/983806399 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f37ac135db0 msgr2=0x7f37ac138190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:56.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.536+0000 7f37aaffd700 1 --2- 172.21.15.16:0/983806399 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f37ac135db0 0x7f37ac138190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:56.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.536+0000 7f37aaffd700 1 -- 172.21.15.16:0/983806399 >> v1:172.21.15.16:6789/0 conn(0x7f37ac130750 legacy=0x7f37ac130b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:56.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.536+0000 7f37aaffd700 1 -- 172.21.15.16:0/983806399 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37ac130300 con 0x7f37ac133490 2023-09-30T22:34:56.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.536+0000 7f37abfff700 1 --2- 172.21.15.16:0/983806399 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f37ac135db0 0x7f37ac138190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:56.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.536+0000 7f37aaffd700 1 --2- 172.21.15.16:0/983806399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37ac133490 0x7f37ac135870 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f3794009840 tx=0x7f3794009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1efd4bac93244d4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:56.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.536+0000 7f37aa7fc700 1 -- 172.21.15.16:0/983806399 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3794011830 con 0x7f37ac133490 2023-09-30T22:34:56.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.536+0000 7f37aa7fc700 1 -- 172.21.15.16:0/983806399 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3794011990 con 0x7f37ac133490 2023-09-30T22:34:56.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.536+0000 7f37aa7fc700 1 -- 172.21.15.16:0/983806399 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3794011d20 con 0x7f37ac133490 2023-09-30T22:34:56.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.537+0000 7f37b20dd700 1 -- 172.21.15.16:0/983806399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37ac133490 msgr2=0x7f37ac135870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:56.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.537+0000 7f37b20dd700 1 --2- 172.21.15.16:0/983806399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37ac133490 0x7f37ac135870 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f3794009840 tx=0x7f3794009b50 comp rx=0 tx=0).stop 2023-09-30T22:34:56.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.537+0000 7f37b20dd700 1 -- 172.21.15.16:0/983806399 shutdown_connections 2023-09-30T22:34:56.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.537+0000 7f37b20dd700 1 --2- 172.21.15.16:0/983806399 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f37ac135db0 0x7f37ac138190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:56.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.537+0000 7f37b20dd700 1 --2- 172.21.15.16:0/983806399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37ac133490 0x7f37ac135870 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:56.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.537+0000 7f37b20dd700 1 -- 172.21.15.16:0/983806399 >> 172.21.15.16:0/983806399 conn(0x7f37ac09f470 msgr2=0x7f37ac0a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:56.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.537+0000 7f37b20dd700 1 -- 172.21.15.16:0/983806399 shutdown_connections 2023-09-30T22:34:56.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.537+0000 7f37b20dd700 1 -- 172.21.15.16:0/983806399 wait complete. 2023-09-30T22:34:56.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.538+0000 7f37b20dd700 1 Processor -- start 2023-09-30T22:34:56.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.538+0000 7f37b20dd700 1 -- start start 2023-09-30T22:34:56.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.538+0000 7f37b20dd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f37ac130750 0x7f37ac128ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:56.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.538+0000 7f37b20dd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f37ac133490 0x7f37ac129530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:56.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.538+0000 7f37b20dd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37ac135db0 0x7f37ac129a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:56.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.538+0000 7f37b20dd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ac0a1d90 con 0x7f37ac135db0 2023-09-30T22:34:56.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.538+0000 7f37b20dd700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37ac09b2b0 con 0x7f37ac130750 2023-09-30T22:34:56.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.538+0000 7f37b20dd700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f37ac12f8b0 con 0x7f37ac133490 2023-09-30T22:34:56.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.538+0000 7f37abfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37ac135db0 0x7f37ac129a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:56.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.538+0000 7f37aaffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f37ac133490 0x7f37ac129530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:56.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.538+0000 7f37abfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37ac135db0 0x7f37ac129a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34714/0 (socket says 172.21.15.16:34714) 2023-09-30T22:34:56.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.538+0000 7f37abfff700 1 -- 172.21.15.16:0/1203231840 learned_addr learned my addr 172.21.15.16:0/1203231840 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:56.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.539+0000 7f37ab7fe700 1 --2- 172.21.15.16:0/1203231840 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f37ac130750 0x7f37ac128ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:56.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.539+0000 7f37abfff700 1 -- 172.21.15.16:0/1203231840 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f37ac133490 msgr2=0x7f37ac129530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:56.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.539+0000 7f37abfff700 1 --2- 172.21.15.16:0/1203231840 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f37ac133490 0x7f37ac129530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:56.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.539+0000 7f37abfff700 1 -- 172.21.15.16:0/1203231840 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f37ac130750 msgr2=0x7f37ac128ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:56.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.539+0000 7f37abfff700 1 --2- 172.21.15.16:0/1203231840 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f37ac130750 0x7f37ac128ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:56.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.539+0000 7f37abfff700 1 -- 172.21.15.16:0/1203231840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a4009a70 con 0x7f37ac135db0 2023-09-30T22:34:56.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.539+0000 7f37aaffd700 1 --2- 172.21.15.16:0/1203231840 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f37ac133490 0x7f37ac129530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:56.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.539+0000 7f37abfff700 1 --2- 172.21.15.16:0/1203231840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37ac135db0 0x7f37ac129a70 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f37a400ef90 tx=0x7f37a400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:56.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.539+0000 7f37a3fff700 1 -- 172.21.15.16:0/1203231840 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a400cc90 con 0x7f37ac135db0 2023-09-30T22:34:56.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.539+0000 7f37a3fff700 1 -- 172.21.15.16:0/1203231840 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37a400f040 con 0x7f37ac135db0 2023-09-30T22:34:56.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.539+0000 7f37b20dd700 1 -- 172.21.15.16:0/1203231840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37940094a0 con 0x7f37ac135db0 2023-09-30T22:34:56.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.539+0000 7f37a3fff700 1 -- 172.21.15.16:0/1203231840 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a40146c0 con 0x7f37ac135db0 2023-09-30T22:34:56.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.539+0000 7f37b20dd700 1 -- 172.21.15.16:0/1203231840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37ac12c700 con 0x7f37ac135db0 2023-09-30T22:34:56.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.540+0000 7f37a3fff700 1 -- 172.21.15.16:0/1203231840 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f37a4014820 con 0x7f37ac135db0 2023-09-30T22:34:56.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.540+0000 7f37a3fff700 1 -- 172.21.15.16:0/1203231840 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f37a4040440 con 0x7f37ac135db0 2023-09-30T22:34:56.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.540+0000 7f37b20dd700 1 -- 172.21.15.16:0/1203231840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37ac12c330 con 0x7f37ac135db0 2023-09-30T22:34:56.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.545+0000 7f37a3fff700 1 -- 172.21.15.16:0/1203231840 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f37a4012700 con 0x7f37ac135db0 2023-09-30T22:34:56.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.753+0000 7f37b20dd700 1 -- 172.21.15.16:0/1203231840 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f37ac12fd70 con 0x7f37ac135db0 2023-09-30T22:34:56.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.753+0000 7f37a3fff700 1 -- 172.21.15.16:0/1203231840 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f37a4012340 con 0x7f37ac135db0 2023-09-30T22:34:56.753 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:34:56.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.755+0000 7f37b20dd700 1 -- 172.21.15.16:0/1203231840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37ac135db0 msgr2=0x7f37ac129a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:56.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.755+0000 7f37b20dd700 1 --2- 172.21.15.16:0/1203231840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37ac135db0 0x7f37ac129a70 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f37a400ef90 tx=0x7f37a400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:34:56.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.756+0000 7f37b20dd700 1 -- 172.21.15.16:0/1203231840 shutdown_connections 2023-09-30T22:34:56.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.756+0000 7f37b20dd700 1 --2- 172.21.15.16:0/1203231840 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f37ac130750 0x7f37ac128ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:56.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.756+0000 7f37b20dd700 1 --2- 172.21.15.16:0/1203231840 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37ac135db0 0x7f37ac129a70 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:56.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.756+0000 7f37b20dd700 1 --2- 172.21.15.16:0/1203231840 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f37ac133490 0x7f37ac129530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:56.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.756+0000 7f37b20dd700 1 -- 172.21.15.16:0/1203231840 >> 172.21.15.16:0/1203231840 conn(0x7f37ac09f470 msgr2=0x7f37ac0a0300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:56.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.756+0000 7f37b20dd700 1 -- 172.21.15.16:0/1203231840 shutdown_connections 2023-09-30T22:34:56.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.756+0000 7f37b20dd700 1 -- 172.21.15.16:0/1203231840 wait complete. 2023-09-30T22:34:56.770 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:34:56.821 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:34:56.858 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:34:56.859 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:34:56.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.967+0000 7f0059580700 1 Processor -- start 2023-09-30T22:34:56.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.968+0000 7f0059580700 1 -- start start 2023-09-30T22:34:56.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.968+0000 7f0059580700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f005409d6f0 0x7f005409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:56.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.968+0000 7f0059580700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f005409e000 0x7f00541339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:56.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.968+0000 7f0059580700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f005409b120 con 0x7f005409e410 2023-09-30T22:34:56.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.968+0000 7f0059580700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f005406b460 con 0x7f005409d6f0 2023-09-30T22:34:56.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.968+0000 7f0059580700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f005409a7e0 con 0x7f005409e000 2023-09-30T22:34:56.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.969+0000 7f00527fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f005409e000 0x7f00541339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:56.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.969+0000 7f0052ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f005409d6f0 0x7f005409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:56.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.969+0000 7f0052ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f005409d6f0 0x7f005409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:42398/0 (socket says 172.21.15.16:42398) 2023-09-30T22:34:56.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.969+0000 7f0052ffd700 1 -- 172.21.15.16:0/2542855746 learned_addr learned my addr 172.21.15.16:0/2542855746 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:56.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.969+0000 7f00527fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f005409e000 0x7f00541339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:39084/0 (socket says 172.21.15.16:39084) 2023-09-30T22:34:56.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.969+0000 7f00537fe700 1 --1- 172.21.15.16:0/2542855746 >> v1:172.21.15.16:6789/0 conn(0x7f005409e410 0x7f0054133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60992/0 (socket says 172.21.15.16:60992) 2023-09-30T22:34:56.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.969+0000 7f0052ffd700 1 -- 172.21.15.16:0/2542855746 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f005409e000 msgr2=0x7f00541339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:56.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.969+0000 7f0052ffd700 1 --2- 172.21.15.16:0/2542855746 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f005409e000 0x7f00541339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:56.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.969+0000 7f0052ffd700 1 -- 172.21.15.16:0/2542855746 >> v1:172.21.15.16:6789/0 conn(0x7f005409e410 legacy=0x7f0054133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:56.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.969+0000 7f0051ffb700 1 -- 172.21.15.16:0/2542855746 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3054732257 0 0) 0x7f005409b120 con 0x7f005409e410 2023-09-30T22:34:56.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.969+0000 7f0052ffd700 1 -- 172.21.15.16:0/2542855746 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f005409a500 con 0x7f005409d6f0 2023-09-30T22:34:56.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.969+0000 7f00527fc700 1 --2- 172.21.15.16:0/2542855746 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f005409e000 0x7f00541339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:56.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.969+0000 7f0052ffd700 1 --2- 172.21.15.16:0/2542855746 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f005409d6f0 0x7f005409dac0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f0044009920 tx=0x7f0044009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37840fbf203c0674 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:56.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.969+0000 7f0051ffb700 1 -- 172.21.15.16:0/2542855746 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0044011830 con 0x7f005409d6f0 2023-09-30T22:34:56.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.970+0000 7f0051ffb700 1 -- 172.21.15.16:0/2542855746 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0044011990 con 0x7f005409d6f0 2023-09-30T22:34:56.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.970+0000 7f0051ffb700 1 -- 172.21.15.16:0/2542855746 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0044011d20 con 0x7f005409d6f0 2023-09-30T22:34:56.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.970+0000 7f0059580700 1 -- 172.21.15.16:0/2542855746 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f005409d6f0 msgr2=0x7f005409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:56.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.970+0000 7f0059580700 1 --2- 172.21.15.16:0/2542855746 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f005409d6f0 0x7f005409dac0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f0044009920 tx=0x7f0044009c30 comp rx=0 tx=0).stop 2023-09-30T22:34:56.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.970+0000 7f0059580700 1 -- 172.21.15.16:0/2542855746 shutdown_connections 2023-09-30T22:34:56.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.970+0000 7f0059580700 1 --2- 172.21.15.16:0/2542855746 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f005409d6f0 0x7f005409dac0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:56.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.970+0000 7f0059580700 1 --2- 172.21.15.16:0/2542855746 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f005409e000 0x7f00541339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:56.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.970+0000 7f0059580700 1 -- 172.21.15.16:0/2542855746 >> 172.21.15.16:0/2542855746 conn(0x7f0054066f50 msgr2=0x7f005409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:56.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.970+0000 7f0059580700 1 -- 172.21.15.16:0/2542855746 shutdown_connections 2023-09-30T22:34:56.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.970+0000 7f0059580700 1 -- 172.21.15.16:0/2542855746 wait complete. 2023-09-30T22:34:56.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.972+0000 7f0059580700 1 Processor -- start 2023-09-30T22:34:56.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.972+0000 7f0059580700 1 -- start start 2023-09-30T22:34:56.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.972+0000 7f0059580700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f005409d6f0 0x7f00541223c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:56.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.972+0000 7f0059580700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f005409e000 0x7f0054122900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:56.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.972+0000 7f0059580700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f005409e410 0x7f0054122e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:56.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.972+0000 7f0059580700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00541336f0 con 0x7f005409e410 2023-09-30T22:34:56.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.972+0000 7f0059580700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0054123820 con 0x7f005409d6f0 2023-09-30T22:34:56.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.972+0000 7f0059580700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0054124dc0 con 0x7f005409e000 2023-09-30T22:34:56.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.972+0000 7f00527fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f005409e000 0x7f0054122900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:56.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.972+0000 7f00537fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f005409e410 0x7f0054122e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:56.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.972+0000 7f00527fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f005409e000 0x7f0054122900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:39086/0 (socket says 172.21.15.16:39086) 2023-09-30T22:34:56.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.972+0000 7f00527fc700 1 -- 172.21.15.16:0/3449735566 learned_addr learned my addr 172.21.15.16:0/3449735566 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:56.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.972+0000 7f0052ffd700 1 --2- 172.21.15.16:0/3449735566 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f005409d6f0 0x7f00541223c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:56.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.973+0000 7f00527fc700 1 -- 172.21.15.16:0/3449735566 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f005409d6f0 msgr2=0x7f00541223c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:56.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.973+0000 7f00527fc700 1 --2- 172.21.15.16:0/3449735566 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f005409d6f0 0x7f00541223c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:56.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.973+0000 7f00527fc700 1 -- 172.21.15.16:0/3449735566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f005409e410 msgr2=0x7f0054122e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:56.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.973+0000 7f00527fc700 1 --2- 172.21.15.16:0/3449735566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f005409e410 0x7f0054122e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:56.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.973+0000 7f00527fc700 1 -- 172.21.15.16:0/3449735566 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00480094a0 con 0x7f005409e000 2023-09-30T22:34:56.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.973+0000 7f00537fe700 1 --2- 172.21.15.16:0/3449735566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f005409e410 0x7f0054122e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:56.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.973+0000 7f0052ffd700 1 --2- 172.21.15.16:0/3449735566 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f005409d6f0 0x7f00541223c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:56.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.973+0000 7f00527fc700 1 --2- 172.21.15.16:0/3449735566 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f005409e000 0x7f0054122900 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f004800ebe0 tx=0x7f004800eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:56.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.973+0000 7f003b7fe700 1 -- 172.21.15.16:0/3449735566 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004800c850 con 0x7f005409e000 2023-09-30T22:34:56.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.973+0000 7f003b7fe700 1 -- 172.21.15.16:0/3449735566 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f004800c9b0 con 0x7f005409e000 2023-09-30T22:34:56.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.973+0000 7f0059580700 1 -- 172.21.15.16:0/3449735566 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0044009550 con 0x7f005409e000 2023-09-30T22:34:56.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.973+0000 7f003b7fe700 1 -- 172.21.15.16:0/3449735566 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f004800cd40 con 0x7f005409e000 2023-09-30T22:34:56.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.973+0000 7f0059580700 1 -- 172.21.15.16:0/3449735566 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00540a2a10 con 0x7f005409e000 2023-09-30T22:34:56.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.974+0000 7f003b7fe700 1 -- 172.21.15.16:0/3449735566 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0048013470 con 0x7f005409e000 2023-09-30T22:34:56.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.974+0000 7f003b7fe700 1 -- 172.21.15.16:0/3449735566 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f004803f8d0 con 0x7f005409e000 2023-09-30T22:34:56.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.975+0000 7f0059580700 1 -- 172.21.15.16:0/3449735566 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0054125180 con 0x7f005409e000 2023-09-30T22:34:56.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:56.984+0000 7f003b7fe700 1 -- 172.21.15.16:0/3449735566 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0048013720 con 0x7f005409e000 2023-09-30T22:34:57.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.009+0000 7fef077af700 1 Processor -- start 2023-09-30T22:34:57.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.009+0000 7fef077af700 1 -- start start 2023-09-30T22:34:57.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.010+0000 7fef077af700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fef0009d6f0 0x7fef0009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:57.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.010+0000 7fef077af700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fef0009e000 0x7fef001339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:57.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.010+0000 7fef077af700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef0009b120 con 0x7fef0009e410 2023-09-30T22:34:57.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.010+0000 7fef077af700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef0006b460 con 0x7fef0009d6f0 2023-09-30T22:34:57.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.010+0000 7fef077af700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fef0009a7e0 con 0x7fef0009e000 2023-09-30T22:34:57.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.010+0000 7fef04d4a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fef0009e000 0x7fef001339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:57.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.010+0000 7fef05d4c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fef0009e410 0x7fef00133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:32774/0 (socket says 172.21.15.16:32774) 2023-09-30T22:34:57.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.010+0000 7fef05d4c700 1 -- 172.21.15.16:0/3251726595 learned_addr learned my addr 172.21.15.16:0/3251726595 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:34:57.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.010+0000 7fef0554b700 1 --2- 172.21.15.16:0/3251726595 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fef0009d6f0 0x7fef0009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:57.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.010+0000 7feef7fff700 1 -- 172.21.15.16:0/3251726595 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1315964486 0 0) 0x7fef0009b120 con 0x7fef0009e410 2023-09-30T22:34:57.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.010+0000 7fef04d4a700 1 -- 172.21.15.16:0/3251726595 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fef0009d6f0 msgr2=0x7fef0009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:57.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.010+0000 7fef04d4a700 1 --2- 172.21.15.16:0/3251726595 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fef0009d6f0 0x7fef0009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:57.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.010+0000 7fef04d4a700 1 -- 172.21.15.16:0/3251726595 >> v1:172.21.15.16:6789/0 conn(0x7fef0009e410 legacy=0x7fef00133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:57.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.010+0000 7fef04d4a700 1 -- 172.21.15.16:0/3251726595 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef0009a500 con 0x7fef0009e000 2023-09-30T22:34:57.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.010+0000 7fef0554b700 1 --2- 172.21.15.16:0/3251726595 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fef0009d6f0 0x7fef0009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:57.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.010+0000 7fef04d4a700 1 --2- 172.21.15.16:0/3251726595 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fef0009e000 0x7fef001339f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7feef00098a0 tx=0x7feef0009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=def416acda2c8a31 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:57.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.011+0000 7feef7fff700 1 -- 172.21.15.16:0/3251726595 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef0011830 con 0x7fef0009e000 2023-09-30T22:34:57.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.011+0000 7feef7fff700 1 -- 172.21.15.16:0/3251726595 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feef0011990 con 0x7fef0009e000 2023-09-30T22:34:57.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.011+0000 7feef7fff700 1 -- 172.21.15.16:0/3251726595 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef0011d20 con 0x7fef0009e000 2023-09-30T22:34:57.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.011+0000 7fef077af700 1 -- 172.21.15.16:0/3251726595 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fef0009e000 msgr2=0x7fef001339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:57.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.011+0000 7fef077af700 1 --2- 172.21.15.16:0/3251726595 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fef0009e000 0x7fef001339f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7feef00098a0 tx=0x7feef0009bb0 comp rx=0 tx=0).stop 2023-09-30T22:34:57.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.012+0000 7fef077af700 1 -- 172.21.15.16:0/3251726595 shutdown_connections 2023-09-30T22:34:57.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.012+0000 7fef077af700 1 --2- 172.21.15.16:0/3251726595 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fef0009d6f0 0x7fef0009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:57.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.012+0000 7fef077af700 1 --2- 172.21.15.16:0/3251726595 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fef0009e000 0x7fef001339f0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:57.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.012+0000 7fef077af700 1 -- 172.21.15.16:0/3251726595 >> 172.21.15.16:0/3251726595 conn(0x7fef00066f50 msgr2=0x7fef0009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:57.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.012+0000 7fef077af700 1 -- 172.21.15.16:0/3251726595 shutdown_connections 2023-09-30T22:34:57.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.012+0000 7fef077af700 1 -- 172.21.15.16:0/3251726595 wait complete. 2023-09-30T22:34:57.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.012+0000 7fa46e086700 1 Processor -- start 2023-09-30T22:34:57.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.013+0000 7fa46e086700 1 -- start start 2023-09-30T22:34:57.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.013+0000 7fef077af700 1 Processor -- start 2023-09-30T22:34:57.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.013+0000 7fef077af700 1 -- start start 2023-09-30T22:34:57.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.013+0000 7fa46e086700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa46809e1d0 0x7fa468138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:57.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.013+0000 7fa46e086700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa468139170 0x7fa46813b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:57.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.013+0000 7fa46e086700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa46809b120 con 0x7fa46809d6f0 2023-09-30T22:34:57.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.013+0000 7fa46e086700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa46806b460 con 0x7fa468139170 2023-09-30T22:34:57.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.013+0000 7fa46e086700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa46809a7e0 con 0x7fa46809e1d0 2023-09-30T22:34:57.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.013+0000 7fa467fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa468139170 0x7fa46813b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:57.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.013+0000 7fa4677fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa46809d6f0 0x7fa46809dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:32790/0 (socket says 172.21.15.16:32790) 2023-09-30T22:34:57.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.013+0000 7fa4677fe700 1 -- 172.21.15.16:0/4095619370 learned_addr learned my addr 172.21.15.16:0/4095619370 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:34:57.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.013+0000 7fa466ffd700 1 --2- 172.21.15.16:0/4095619370 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa46809e1d0 0x7fa468138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:57.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.014+0000 7fa467fff700 1 -- 172.21.15.16:0/4095619370 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa46809e1d0 msgr2=0x7fa468138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:57.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.014+0000 7fa467fff700 1 --2- 172.21.15.16:0/4095619370 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa46809e1d0 0x7fa468138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:57.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.014+0000 7fa467fff700 1 -- 172.21.15.16:0/4095619370 >> v1:172.21.15.16:6789/0 conn(0x7fa46809d6f0 legacy=0x7fa46809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:57.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.014+0000 7fa467fff700 1 -- 172.21.15.16:0/4095619370 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa46809a500 con 0x7fa468139170 2023-09-30T22:34:57.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.014+0000 7fa4667fc700 1 -- 172.21.15.16:0/4095619370 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4231384999 0 0) 0x7fa46809b120 con 0x7fa46809d6f0 2023-09-30T22:34:57.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.014+0000 7fef077af700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fef0009d6f0 0x7fef001a5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:57.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.014+0000 7fef077af700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fef0009e000 0x7fef001a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:57.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.014+0000 7fef077af700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef0009e410 0x7fef001c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:57.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.014+0000 7fef077af700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef001a4f70 con 0x7fef0009e410 2023-09-30T22:34:57.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.014+0000 7fef077af700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef001b93f0 con 0x7fef0009e000 2023-09-30T22:34:57.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.014+0000 7fef077af700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fef0009c710 con 0x7fef0009d6f0 2023-09-30T22:34:57.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.014+0000 7fef04d4a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fef0009e000 0x7fef001a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:57.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.014+0000 7fef0554b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fef0009d6f0 0x7fef001a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:57.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.014+0000 7fef04d4a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fef0009e000 0x7fef001a5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:42420/0 (socket says 172.21.15.16:42420) 2023-09-30T22:34:57.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.014+0000 7fef0554b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fef0009d6f0 0x7fef001a5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:39098/0 (socket says 172.21.15.16:39098) 2023-09-30T22:34:57.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.014+0000 7fef04d4a700 1 -- 172.21.15.16:0/2562967583 learned_addr learned my addr 172.21.15.16:0/2562967583 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:57.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.014+0000 7fa467fff700 1 --2- 172.21.15.16:0/4095619370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa468139170 0x7fa46813b550 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fa460009d90 tx=0x7fa46000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84a1b1dd94c4161e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:57.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.014+0000 7fef0554b700 1 -- 172.21.15.16:0/2562967583 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fef0009e000 msgr2=0x7fef001a5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:57.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.014+0000 7fa4667fc700 1 -- 172.21.15.16:0/4095619370 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa46000e070 con 0x7fa468139170 2023-09-30T22:34:57.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.014+0000 7fa4667fc700 1 -- 172.21.15.16:0/4095619370 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa46000bb20 con 0x7fa468139170 2023-09-30T22:34:57.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.014+0000 7fef0554b700 1 --2- 172.21.15.16:0/2562967583 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fef0009e000 0x7fef001a5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:57.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.014+0000 7fef0554b700 1 -- 172.21.15.16:0/2562967583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef0009e410 msgr2=0x7fef001c82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:57.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.014+0000 7fef0554b700 1 --2- 172.21.15.16:0/2562967583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef0009e410 0x7fef001c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:57.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.014+0000 7fa4667fc700 1 -- 172.21.15.16:0/4095619370 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa460012430 con 0x7fa468139170 2023-09-30T22:34:57.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.014+0000 7fef0554b700 1 -- 172.21.15.16:0/2562967583 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feefc0094a0 con 0x7fef0009d6f0 2023-09-30T22:34:57.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.014+0000 7fef04d4a700 1 --2- 172.21.15.16:0/2562967583 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fef0009e000 0x7fef001a5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:34:57.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.014+0000 7fef0554b700 1 --2- 172.21.15.16:0/2562967583 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fef0009d6f0 0x7fef001a5330 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7feefc00e9b0 tx=0x7feefc00ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:57.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.015+0000 7feef5ffb700 1 -- 172.21.15.16:0/2562967583 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feefc00c850 con 0x7fef0009d6f0 2023-09-30T22:34:57.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.015+0000 7feef5ffb700 1 -- 172.21.15.16:0/2562967583 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feefc00c9b0 con 0x7fef0009d6f0 2023-09-30T22:34:57.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.015+0000 7feef5ffb700 1 -- 172.21.15.16:0/2562967583 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feefc00cd40 con 0x7fef0009d6f0 2023-09-30T22:34:57.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.015+0000 7fef077af700 1 -- 172.21.15.16:0/2562967583 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feef00094a0 con 0x7fef0009d6f0 2023-09-30T22:34:57.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.015+0000 7fef077af700 1 -- 172.21.15.16:0/2562967583 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef001a6670 con 0x7fef0009d6f0 2023-09-30T22:34:57.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.015+0000 7fa46e086700 1 -- 172.21.15.16:0/4095619370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa468139170 msgr2=0x7fa46813b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:57.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.015+0000 7fa46e086700 1 --2- 172.21.15.16:0/4095619370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa468139170 0x7fa46813b550 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7fa460009d90 tx=0x7fa46000b330 comp rx=0 tx=0).stop 2023-09-30T22:34:57.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.016+0000 7fa46e086700 1 -- 172.21.15.16:0/4095619370 shutdown_connections 2023-09-30T22:34:57.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.016+0000 7fa46e086700 1 --2- 172.21.15.16:0/4095619370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa468139170 0x7fa46813b550 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:57.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.016+0000 7fa46e086700 1 --2- 172.21.15.16:0/4095619370 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa46809e1d0 0x7fa468138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:57.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.016+0000 7fa46e086700 1 -- 172.21.15.16:0/4095619370 >> 172.21.15.16:0/4095619370 conn(0x7fa468066f50 msgr2=0x7fa46809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:57.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.016+0000 7fa46e086700 1 -- 172.21.15.16:0/4095619370 shutdown_connections 2023-09-30T22:34:57.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.016+0000 7fa46e086700 1 -- 172.21.15.16:0/4095619370 wait complete. 2023-09-30T22:34:57.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.016+0000 7feef5ffb700 1 -- 172.21.15.16:0/2562967583 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7feefc013470 con 0x7fef0009d6f0 2023-09-30T22:34:57.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.016+0000 7feef5ffb700 1 -- 172.21.15.16:0/2562967583 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7feefc03f860 con 0x7fef0009d6f0 2023-09-30T22:34:57.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.017+0000 7fef077af700 1 -- 172.21.15.16:0/2562967583 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef00074ea0 con 0x7fef0009d6f0 2023-09-30T22:34:57.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.017+0000 7fa46e086700 1 Processor -- start 2023-09-30T22:34:57.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.017+0000 7fa46e086700 1 -- start start 2023-09-30T22:34:57.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.017+0000 7fa46e086700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa46809d6f0 0x7fa4681219d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:57.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.017+0000 7fa46e086700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa46809e1d0 0x7fa468121f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:57.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.017+0000 7fa46e086700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa468139170 0x7fa468122450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:57.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.017+0000 7fa46e086700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4681379a0 con 0x7fa468139170 2023-09-30T22:34:57.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.017+0000 7fa46e086700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4681236f0 con 0x7fa46809d6f0 2023-09-30T22:34:57.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.017+0000 7fa46e086700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa468071f20 con 0x7fa46809e1d0 2023-09-30T22:34:57.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.018+0000 7fa4677fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa46809d6f0 0x7fa4681219d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:57.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.018+0000 7fa466ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa46809e1d0 0x7fa468121f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:57.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.018+0000 7fa4677fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa46809d6f0 0x7fa4681219d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:42428/0 (socket says 172.21.15.16:42428) 2023-09-30T22:34:57.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.018+0000 7fa4677fe700 1 -- 172.21.15.16:0/2937396320 learned_addr learned my addr 172.21.15.16:0/2937396320 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:57.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.018+0000 7fa467fff700 1 --2- 172.21.15.16:0/2937396320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa468139170 0x7fa468122450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:57.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.018+0000 7fa4677fe700 1 -- 172.21.15.16:0/2937396320 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa46809e1d0 msgr2=0x7fa468121f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:57.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.018+0000 7fa4677fe700 1 --2- 172.21.15.16:0/2937396320 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa46809e1d0 0x7fa468121f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:57.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.018+0000 7fa4677fe700 1 -- 172.21.15.16:0/2937396320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa468139170 msgr2=0x7fa468122450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:57.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.018+0000 7fa4677fe700 1 --2- 172.21.15.16:0/2937396320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa468139170 0x7fa468122450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:57.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.018+0000 7fa4677fe700 1 -- 172.21.15.16:0/2937396320 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa460009a70 con 0x7fa46809d6f0 2023-09-30T22:34:57.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.018+0000 7fa466ffd700 1 --2- 172.21.15.16:0/2937396320 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa46809e1d0 0x7fa468121f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:57.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.018+0000 7fa467fff700 1 --2- 172.21.15.16:0/2937396320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa468139170 0x7fa468122450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:57.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.018+0000 7fa4677fe700 1 --2- 172.21.15.16:0/2937396320 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa46809d6f0 0x7fa4681219d0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fa45800c520 tx=0x7fa45800c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:57.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.018+0000 7fa44ffff700 1 -- 172.21.15.16:0/2937396320 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa458004c80 con 0x7fa46809d6f0 2023-09-30T22:34:57.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.019+0000 7fa44ffff700 1 -- 172.21.15.16:0/2937396320 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa458004e00 con 0x7fa46809d6f0 2023-09-30T22:34:57.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.019+0000 7fa46e086700 1 -- 172.21.15.16:0/2937396320 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa468124fb0 con 0x7fa46809d6f0 2023-09-30T22:34:57.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.019+0000 7fa44ffff700 1 -- 172.21.15.16:0/2937396320 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa458007310 con 0x7fa46809d6f0 2023-09-30T22:34:57.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.019+0000 7fa46e086700 1 -- 172.21.15.16:0/2937396320 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa468123b30 con 0x7fa46809d6f0 2023-09-30T22:34:57.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.020+0000 7fa46e086700 1 -- 172.21.15.16:0/2937396320 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa468074ea0 con 0x7fa46809d6f0 2023-09-30T22:34:57.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.020+0000 7fa44ffff700 1 -- 172.21.15.16:0/2937396320 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa458007470 con 0x7fa46809d6f0 2023-09-30T22:34:57.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.020+0000 7fa44ffff700 1 -- 172.21.15.16:0/2937396320 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fa45803f310 con 0x7fa46809d6f0 2023-09-30T22:34:57.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.024+0000 7feef5ffb700 1 -- 172.21.15.16:0/2562967583 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7feefc0232f0 con 0x7fef0009d6f0 2023-09-30T22:34:57.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.028+0000 7fa44ffff700 1 -- 172.21.15.16:0/2937396320 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa4580133c0 con 0x7fa46809d6f0 2023-09-30T22:34:57.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.194+0000 7f0059580700 1 -- 172.21.15.16:0/3449735566 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0054074ea0 con 0x7f005409e000 2023-09-30T22:34:57.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.194+0000 7f003b7fe700 1 -- 172.21.15.16:0/3449735566 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f004802b090 con 0x7f005409e000 2023-09-30T22:34:57.195 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:34:57.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.196+0000 7f0059580700 1 -- 172.21.15.16:0/3449735566 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f005409e000 msgr2=0x7f0054122900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:57.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.197+0000 7f0059580700 1 --2- 172.21.15.16:0/3449735566 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f005409e000 0x7f0054122900 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f004800ebe0 tx=0x7f004800eef0 comp rx=0 tx=0).stop 2023-09-30T22:34:57.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.197+0000 7f0059580700 1 -- 172.21.15.16:0/3449735566 shutdown_connections 2023-09-30T22:34:57.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.197+0000 7f0059580700 1 --2- 172.21.15.16:0/3449735566 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f005409d6f0 0x7f00541223c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:57.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.197+0000 7f0059580700 1 --2- 172.21.15.16:0/3449735566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f005409e410 0x7f0054122e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:57.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.197+0000 7f0059580700 1 --2- 172.21.15.16:0/3449735566 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f005409e000 0x7f0054122900 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:57.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.197+0000 7f0059580700 1 -- 172.21.15.16:0/3449735566 >> 172.21.15.16:0/3449735566 conn(0x7f0054066f50 msgr2=0x7f00540a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:57.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.197+0000 7f0059580700 1 -- 172.21.15.16:0/3449735566 shutdown_connections 2023-09-30T22:34:57.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.197+0000 7f0059580700 1 -- 172.21.15.16:0/3449735566 wait complete. 2023-09-30T22:34:57.211 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:34:57.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.234+0000 7fef077af700 1 -- 172.21.15.16:0/2562967583 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fef00072600 con 0x7fef0009d6f0 2023-09-30T22:34:57.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.234+0000 7fa46e086700 1 -- 172.21.15.16:0/2937396320 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa468071bc0 con 0x7fa46809d6f0 2023-09-30T22:34:57.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.235+0000 7fa44ffff700 1 -- 172.21.15.16:0/2937396320 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa458011360 con 0x7fa46809d6f0 2023-09-30T22:34:57.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.234+0000 7feef5ffb700 1 -- 172.21.15.16:0/2562967583 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7feefc021030 con 0x7fef0009d6f0 2023-09-30T22:34:57.236 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:34:57.237 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:34:57.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.237+0000 7fa46e086700 1 -- 172.21.15.16:0/2937396320 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa46809d6f0 msgr2=0x7fa4681219d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:57.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.237+0000 7fa46e086700 1 --2- 172.21.15.16:0/2937396320 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa46809d6f0 0x7fa4681219d0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fa45800c520 tx=0x7fa45800c830 comp rx=0 tx=0).stop 2023-09-30T22:34:57.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.237+0000 7fa46e086700 1 -- 172.21.15.16:0/2937396320 shutdown_connections 2023-09-30T22:34:57.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.237+0000 7fa46e086700 1 --2- 172.21.15.16:0/2937396320 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa46809d6f0 0x7fa4681219d0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:57.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.237+0000 7fa46e086700 1 --2- 172.21.15.16:0/2937396320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa468139170 0x7fa468122450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:57.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.237+0000 7fa46e086700 1 --2- 172.21.15.16:0/2937396320 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa46809e1d0 0x7fa468121f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:57.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.237+0000 7fa46e086700 1 -- 172.21.15.16:0/2937396320 >> 172.21.15.16:0/2937396320 conn(0x7fa468066f50 msgr2=0x7fa468070ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:57.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.237+0000 7fef077af700 1 -- 172.21.15.16:0/2562967583 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fef0009d6f0 msgr2=0x7fef001a5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:57.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.237+0000 7fef077af700 1 --2- 172.21.15.16:0/2562967583 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fef0009d6f0 0x7fef001a5330 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7feefc00e9b0 tx=0x7feefc00ed70 comp rx=0 tx=0).stop 2023-09-30T22:34:57.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.237+0000 7fa46e086700 1 -- 172.21.15.16:0/2937396320 shutdown_connections 2023-09-30T22:34:57.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.237+0000 7fa46e086700 1 -- 172.21.15.16:0/2937396320 wait complete. 2023-09-30T22:34:57.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.237+0000 7fef077af700 1 -- 172.21.15.16:0/2562967583 shutdown_connections 2023-09-30T22:34:57.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.237+0000 7fef077af700 1 --2- 172.21.15.16:0/2562967583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef0009e410 0x7fef001c82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:57.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.237+0000 7fef077af700 1 --2- 172.21.15.16:0/2562967583 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fef0009d6f0 0x7fef001a5330 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:57.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.237+0000 7fef077af700 1 --2- 172.21.15.16:0/2562967583 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fef0009e000 0x7fef001a5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:57.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.237+0000 7fef077af700 1 -- 172.21.15.16:0/2562967583 >> 172.21.15.16:0/2562967583 conn(0x7fef00066f50 msgr2=0x7fef0006f120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:57.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.237+0000 7fef077af700 1 -- 172.21.15.16:0/2562967583 shutdown_connections 2023-09-30T22:34:57.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.237+0000 7fef077af700 1 -- 172.21.15.16:0/2562967583 wait complete. 2023-09-30T22:34:57.252 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:34:57.253 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:34:57.770 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:34:57.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.916+0000 7fe16e727700 1 Processor -- start 2023-09-30T22:34:57.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.916+0000 7fe16e727700 1 -- start start 2023-09-30T22:34:57.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.917+0000 7fe16e727700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe168133420 0x7fe1681337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:57.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.917+0000 7fe16e727700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe168133e30 0x7fe16812ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:57.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.917+0000 7fe16e727700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe168134260 0x7fe1681304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:57.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.917+0000 7fe16e727700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe16806b990 con 0x7fe168133e30 2023-09-30T22:34:57.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.917+0000 7fe16e727700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1680a0590 con 0x7fe168134260 2023-09-30T22:34:57.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.917+0000 7fe16e727700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe16809fb80 con 0x7fe168133420 2023-09-30T22:34:57.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.917+0000 7fe167fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe168133420 0x7fe1681337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:57.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.917+0000 7fe1677fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe168133e30 0x7fe16812ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:57.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.917+0000 7fe16ccc4700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe168134260 0x7fe1681304d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:57.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.917+0000 7fe1677fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe168133e30 0x7fe16812ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43718/0 (socket says 172.21.15.16:43718) 2023-09-30T22:34:57.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.917+0000 7fe1677fe700 1 -- 172.21.15.16:0/2595555820 learned_addr learned my addr 172.21.15.16:0/2595555820 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:57.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.917+0000 7fe167fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe168133420 0x7fe1681337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:33852/0 (socket says 172.21.15.16:33852) 2023-09-30T22:34:57.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.917+0000 7fe167fff700 1 -- 172.21.15.16:0/2595555820 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe168134260 msgr2=0x7fe1681304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:57.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.917+0000 7fe167fff700 1 --2- 172.21.15.16:0/2595555820 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe168134260 0x7fe1681304d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:57.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.917+0000 7fe167fff700 1 -- 172.21.15.16:0/2595555820 >> v2:172.21.15.16:3300/0 conn(0x7fe168133e30 msgr2=0x7fe16812ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:57.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.917+0000 7fe167fff700 1 --2- 172.21.15.16:0/2595555820 >> v2:172.21.15.16:3300/0 conn(0x7fe168133e30 0x7fe16812ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:57.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.917+0000 7fe167fff700 1 -- 172.21.15.16:0/2595555820 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1680a0030 con 0x7fe168133420 2023-09-30T22:34:57.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.917+0000 7fe16ccc4700 1 --2- 172.21.15.16:0/2595555820 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe168134260 0x7fe1681304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:57.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.917+0000 7fe167fff700 1 --2- 172.21.15.16:0/2595555820 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe168133420 0x7fe1681337f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fe158009840 tx=0x7fe158009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90f3d346ab9a861b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:57.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.918+0000 7fe166ffd700 1 -- 172.21.15.16:0/2595555820 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe158011830 con 0x7fe168133420 2023-09-30T22:34:57.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.918+0000 7fe166ffd700 1 -- 172.21.15.16:0/2595555820 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe158011990 con 0x7fe168133420 2023-09-30T22:34:57.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.918+0000 7fe166ffd700 1 -- 172.21.15.16:0/2595555820 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe158011d20 con 0x7fe168133420 2023-09-30T22:34:57.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.918+0000 7fe16e727700 1 -- 172.21.15.16:0/2595555820 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe168133420 msgr2=0x7fe1681337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:57.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.918+0000 7fe16e727700 1 --2- 172.21.15.16:0/2595555820 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe168133420 0x7fe1681337f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fe158009840 tx=0x7fe158009b50 comp rx=0 tx=0).stop 2023-09-30T22:34:57.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.918+0000 7fe16e727700 1 -- 172.21.15.16:0/2595555820 shutdown_connections 2023-09-30T22:34:57.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.918+0000 7fe16e727700 1 --2- 172.21.15.16:0/2595555820 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe168134260 0x7fe1681304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:57.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.918+0000 7fe16e727700 1 --2- 172.21.15.16:0/2595555820 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe168133420 0x7fe1681337f0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:57.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.918+0000 7fe16e727700 1 --2- 172.21.15.16:0/2595555820 >> v2:172.21.15.16:3300/0 conn(0x7fe168133e30 0x7fe16812ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:57.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.918+0000 7fe16e727700 1 -- 172.21.15.16:0/2595555820 >> 172.21.15.16:0/2595555820 conn(0x7fe1680a4210 msgr2=0x7fe1680a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:57.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.918+0000 7fe16e727700 1 -- 172.21.15.16:0/2595555820 shutdown_connections 2023-09-30T22:34:57.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.918+0000 7fe16e727700 1 -- 172.21.15.16:0/2595555820 wait complete. 2023-09-30T22:34:57.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.920+0000 7fe16e727700 1 Processor -- start 2023-09-30T22:34:57.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.920+0000 7fe16e727700 1 -- start start 2023-09-30T22:34:57.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.920+0000 7fe16e727700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe168133420 0x7fe16809c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:57.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.921+0000 7fe16e727700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe168133e30 0x7fe16809c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:57.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.921+0000 7fe16e727700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe168134260 0x7fe16809cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:57.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.921+0000 7fe16e727700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16809dff0 con 0x7fe168133420 2023-09-30T22:34:57.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.921+0000 7fe16e727700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe16809de00 con 0x7fe168134260 2023-09-30T22:34:57.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.921+0000 7fe16e727700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe16811f150 con 0x7fe168133e30 2023-09-30T22:34:57.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.921+0000 7fe167fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe168133420 0x7fe16809c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:57.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.921+0000 7fe167fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe168133420 0x7fe16809c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43724/0 (socket says 172.21.15.16:43724) 2023-09-30T22:34:57.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.921+0000 7fe167fff700 1 -- 172.21.15.16:0/2866107106 learned_addr learned my addr 172.21.15.16:0/2866107106 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:57.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.921+0000 7fe1677fe700 1 --2- 172.21.15.16:0/2866107106 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe168133e30 0x7fe16809c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:57.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.921+0000 7fe16ccc4700 1 --2- 172.21.15.16:0/2866107106 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe168134260 0x7fe16809cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:57.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.921+0000 7fe167fff700 1 -- 172.21.15.16:0/2866107106 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe168133e30 msgr2=0x7fe16809c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:57.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.921+0000 7fe167fff700 1 --2- 172.21.15.16:0/2866107106 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe168133e30 0x7fe16809c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:57.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.921+0000 7fe167fff700 1 -- 172.21.15.16:0/2866107106 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe168134260 msgr2=0x7fe16809cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:57.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.921+0000 7fe167fff700 1 --2- 172.21.15.16:0/2866107106 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe168134260 0x7fe16809cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:57.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.921+0000 7fe167fff700 1 -- 172.21.15.16:0/2866107106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe160009a70 con 0x7fe168133420 2023-09-30T22:34:57.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.921+0000 7fe16ccc4700 1 --2- 172.21.15.16:0/2866107106 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe168134260 0x7fe16809cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:57.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.921+0000 7fe167fff700 1 --2- 172.21.15.16:0/2866107106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe168133420 0x7fe16809c480 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fe158000c00 tx=0x7fe158004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:57.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.921+0000 7fe164ff9700 1 -- 172.21.15.16:0/2866107106 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe158004450 con 0x7fe168133420 2023-09-30T22:34:57.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.921+0000 7fe164ff9700 1 -- 172.21.15.16:0/2866107106 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe15801f070 con 0x7fe168133420 2023-09-30T22:34:57.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.921+0000 7fe16e727700 1 -- 172.21.15.16:0/2866107106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1580094a0 con 0x7fe168133420 2023-09-30T22:34:57.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.921+0000 7fe164ff9700 1 -- 172.21.15.16:0/2866107106 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1580038e0 con 0x7fe168133420 2023-09-30T22:34:57.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.922+0000 7fe16e727700 1 -- 172.21.15.16:0/2866107106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe16811f4d0 con 0x7fe168133420 2023-09-30T22:34:57.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.922+0000 7fe16e727700 1 -- 172.21.15.16:0/2866107106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe168131490 con 0x7fe168133420 2023-09-30T22:34:57.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.923+0000 7fe164ff9700 1 -- 172.21.15.16:0/2866107106 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe158003a40 con 0x7fe168133420 2023-09-30T22:34:57.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.923+0000 7fe164ff9700 1 -- 172.21.15.16:0/2866107106 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fe158021070 con 0x7fe168133420 2023-09-30T22:34:57.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:57.927+0000 7fe164ff9700 1 -- 172.21.15.16:0/2866107106 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe15801a430 con 0x7fe168133420 2023-09-30T22:34:58.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.131+0000 7fe16e727700 1 -- 172.21.15.16:0/2866107106 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe168074ea0 con 0x7fe168133420 2023-09-30T22:34:58.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.132+0000 7fe164ff9700 1 -- 172.21.15.16:0/2866107106 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe158034090 con 0x7fe168133420 2023-09-30T22:34:58.132 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:34:58.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.134+0000 7fe16e727700 1 -- 172.21.15.16:0/2866107106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe168133420 msgr2=0x7fe16809c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:58.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.134+0000 7fe16e727700 1 --2- 172.21.15.16:0/2866107106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe168133420 0x7fe16809c480 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fe158000c00 tx=0x7fe158004750 comp rx=0 tx=0).stop 2023-09-30T22:34:58.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.134+0000 7fe16e727700 1 -- 172.21.15.16:0/2866107106 shutdown_connections 2023-09-30T22:34:58.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.134+0000 7fe16e727700 1 --2- 172.21.15.16:0/2866107106 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe168134260 0x7fe16809cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:58.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.134+0000 7fe16e727700 1 --2- 172.21.15.16:0/2866107106 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe168133420 0x7fe16809c480 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:58.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.134+0000 7fe16e727700 1 --2- 172.21.15.16:0/2866107106 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe168133e30 0x7fe16809c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:58.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.134+0000 7fe16e727700 1 -- 172.21.15.16:0/2866107106 >> 172.21.15.16:0/2866107106 conn(0x7fe1680a4210 msgr2=0x7fe1680a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:58.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.134+0000 7fe16e727700 1 -- 172.21.15.16:0/2866107106 shutdown_connections 2023-09-30T22:34:58.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.134+0000 7fe16e727700 1 -- 172.21.15.16:0/2866107106 wait complete. 2023-09-30T22:34:58.148 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:34:58.212 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:34:58.254 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:34:58.254 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:34:58.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.360+0000 7f5a3461e700 1 Processor -- start 2023-09-30T22:34:58.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.360+0000 7f5a3461e700 1 -- start start 2023-09-30T22:34:58.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.361+0000 7f5a3461e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5a2c133540 0x7f5a2c135920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:58.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.361+0000 7f5a3461e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a2c135e60 0x7f5a2c138240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:58.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.361+0000 7f5a3461e700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a2c0716a0 con 0x7f5a2c130540 2023-09-30T22:34:58.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.361+0000 7f5a3461e700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a2c071290 con 0x7f5a2c133540 2023-09-30T22:34:58.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.361+0000 7f5a3461e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2c130340 con 0x7f5a2c135e60 2023-09-30T22:34:58.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.361+0000 7f5a31bb9700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5a2c133540 0x7f5a2c135920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:58.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.361+0000 7f5a32bbb700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a2c135e60 0x7f5a2c138240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:58.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.361+0000 7f5a323ba700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f5a2c130540 0x7f5a2c130910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50762/0 (socket says 172.21.15.16:50762) 2023-09-30T22:34:58.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.361+0000 7f5a31bb9700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5a2c133540 0x7f5a2c135920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43740/0 (socket says 172.21.15.16:43740) 2023-09-30T22:34:58.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.361+0000 7f5a32bbb700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a2c135e60 0x7f5a2c138240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:33870/0 (socket says 172.21.15.16:33870) 2023-09-30T22:34:58.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.361+0000 7f5a323ba700 1 -- 172.21.15.16:0/481075889 learned_addr learned my addr 172.21.15.16:0/481075889 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:34:58.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.362+0000 7f5a31bb9700 1 -- 172.21.15.16:0/481075889 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a2c135e60 msgr2=0x7f5a2c138240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:58.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.362+0000 7f5a31bb9700 1 --2- 172.21.15.16:0/481075889 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a2c135e60 0x7f5a2c138240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:58.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.362+0000 7f5a31bb9700 1 -- 172.21.15.16:0/481075889 >> v1:172.21.15.16:6789/0 conn(0x7f5a2c130540 legacy=0x7f5a2c130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:58.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.362+0000 7f5a31bb9700 1 -- 172.21.15.16:0/481075889 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a2c070b80 con 0x7f5a2c133540 2023-09-30T22:34:58.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.362+0000 7f5a313b8700 1 -- 172.21.15.16:0/481075889 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 96356145 0 0) 0x7f5a2c0716a0 con 0x7f5a2c130540 2023-09-30T22:34:58.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.362+0000 7f5a32bbb700 1 --2- 172.21.15.16:0/481075889 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a2c135e60 0x7f5a2c138240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:58.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.362+0000 7f5a31bb9700 1 --2- 172.21.15.16:0/481075889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a2c133540 0x7f5a2c135920 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c009840 tx=0x7f5a1c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e324af4f6e53d4f6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:58.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.362+0000 7f5a313b8700 1 -- 172.21.15.16:0/481075889 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c011830 con 0x7f5a2c133540 2023-09-30T22:34:58.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.362+0000 7f5a313b8700 1 -- 172.21.15.16:0/481075889 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a1c011990 con 0x7f5a2c133540 2023-09-30T22:34:58.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.362+0000 7f5a313b8700 1 -- 172.21.15.16:0/481075889 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c011d20 con 0x7f5a2c133540 2023-09-30T22:34:58.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.363+0000 7f5a3461e700 1 -- 172.21.15.16:0/481075889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a2c133540 msgr2=0x7f5a2c135920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:58.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.363+0000 7f5a3461e700 1 --2- 172.21.15.16:0/481075889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a2c133540 0x7f5a2c135920 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c009840 tx=0x7f5a1c009b50 comp rx=0 tx=0).stop 2023-09-30T22:34:58.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.363+0000 7f5a3461e700 1 -- 172.21.15.16:0/481075889 shutdown_connections 2023-09-30T22:34:58.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.363+0000 7f5a3461e700 1 --2- 172.21.15.16:0/481075889 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a2c135e60 0x7f5a2c138240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:58.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.363+0000 7f5a3461e700 1 --2- 172.21.15.16:0/481075889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a2c133540 0x7f5a2c135920 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:58.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.363+0000 7f5a3461e700 1 -- 172.21.15.16:0/481075889 >> 172.21.15.16:0/481075889 conn(0x7f5a2c0a2aa0 msgr2=0x7f5a2c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:58.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.363+0000 7f5a3461e700 1 -- 172.21.15.16:0/481075889 shutdown_connections 2023-09-30T22:34:58.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.363+0000 7f5a3461e700 1 -- 172.21.15.16:0/481075889 wait complete. 2023-09-30T22:34:58.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.364+0000 7f5a3461e700 1 Processor -- start 2023-09-30T22:34:58.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.364+0000 7f5a3461e700 1 -- start start 2023-09-30T22:34:58.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.365+0000 7f5a3461e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a2c130540 0x7f5a2c09a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:58.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.365+0000 7f5a3461e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a2c133540 0x7f5a2c09aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:58.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.365+0000 7f5a3461e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5a2c135e60 0x7f5a2c09b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:58.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.365+0000 7f5a3461e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2c09d950 con 0x7f5a2c133540 2023-09-30T22:34:58.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.365+0000 7f5a3461e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2c09cb70 con 0x7f5a2c135e60 2023-09-30T22:34:58.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.365+0000 7f5a3461e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2c138ad0 con 0x7f5a2c130540 2023-09-30T22:34:58.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.365+0000 7f5a31bb9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a2c133540 0x7f5a2c09aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:58.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.365+0000 7f5a323ba700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a2c130540 0x7f5a2c09a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:58.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.365+0000 7f5a323ba700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a2c130540 0x7f5a2c09a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:33876/0 (socket says 172.21.15.16:33876) 2023-09-30T22:34:58.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.365+0000 7f5a323ba700 1 -- 172.21.15.16:0/4100454892 learned_addr learned my addr 172.21.15.16:0/4100454892 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:58.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.365+0000 7f5a32bbb700 1 --2- 172.21.15.16:0/4100454892 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5a2c135e60 0x7f5a2c09b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:58.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.365+0000 7f5a31bb9700 1 -- 172.21.15.16:0/4100454892 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a2c130540 msgr2=0x7f5a2c09a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:58.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.365+0000 7f5a31bb9700 1 --2- 172.21.15.16:0/4100454892 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a2c130540 0x7f5a2c09a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:58.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.365+0000 7f5a31bb9700 1 -- 172.21.15.16:0/4100454892 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5a2c135e60 msgr2=0x7f5a2c09b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:58.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.365+0000 7f5a31bb9700 1 --2- 172.21.15.16:0/4100454892 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5a2c135e60 0x7f5a2c09b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:58.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.365+0000 7f5a31bb9700 1 -- 172.21.15.16:0/4100454892 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a24009a70 con 0x7f5a2c133540 2023-09-30T22:34:58.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.365+0000 7f5a323ba700 1 --2- 172.21.15.16:0/4100454892 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a2c130540 0x7f5a2c09a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:58.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.366+0000 7f5a31bb9700 1 --2- 172.21.15.16:0/4100454892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a2c133540 0x7f5a2c09aad0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c016000 tx=0x7f5a1c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:58.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.366+0000 7f5a22ffd700 1 -- 172.21.15.16:0/4100454892 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c0045d0 con 0x7f5a2c133540 2023-09-30T22:34:58.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.366+0000 7f5a22ffd700 1 -- 172.21.15.16:0/4100454892 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a1c01f070 con 0x7f5a2c133540 2023-09-30T22:34:58.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.366+0000 7f5a3461e700 1 -- 172.21.15.16:0/4100454892 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a1c0094a0 con 0x7f5a2c133540 2023-09-30T22:34:58.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.366+0000 7f5a22ffd700 1 -- 172.21.15.16:0/4100454892 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c0038a0 con 0x7f5a2c133540 2023-09-30T22:34:58.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.366+0000 7f5a3461e700 1 -- 172.21.15.16:0/4100454892 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a2c0a23f0 con 0x7f5a2c133540 2023-09-30T22:34:58.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.367+0000 7f5a22ffd700 1 -- 172.21.15.16:0/4100454892 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5a1c003a00 con 0x7f5a2c133540 2023-09-30T22:34:58.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.367+0000 7f5a22ffd700 1 -- 172.21.15.16:0/4100454892 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f5a1c021070 con 0x7f5a2c133540 2023-09-30T22:34:58.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.367+0000 7f5a3461e700 1 -- 172.21.15.16:0/4100454892 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a2c09c820 con 0x7f5a2c133540 2023-09-30T22:34:58.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.372+0000 7f5a22ffd700 1 -- 172.21.15.16:0/4100454892 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5a1c015020 con 0x7f5a2c133540 2023-09-30T22:34:58.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.401+0000 7f9111051700 1 Processor -- start 2023-09-30T22:34:58.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.401+0000 7f9111051700 1 -- start start 2023-09-30T22:34:58.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.401+0000 7f9111051700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f910c0a0a30 0x7f910c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:58.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.401+0000 7f9111051700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f910c138d80 0x7f910c13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:58.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.401+0000 7f9111051700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f910c0a2a70 con 0x7f910c09d630 2023-09-30T22:34:58.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.401+0000 7f9111051700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f910c0a01d0 con 0x7f910c0a0a30 2023-09-30T22:34:58.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.401+0000 7f9111051700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f910c09e2b0 con 0x7f910c138d80 2023-09-30T22:34:58.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.401+0000 7f910b59e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f910c138d80 0x7f910c13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:58.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.402+0000 7f910a59c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f910c09d630 0x7f910c0a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50764/0 (socket says 172.21.15.16:50764) 2023-09-30T22:34:58.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.402+0000 7f910a59c700 1 -- 172.21.15.16:0/2427646017 learned_addr learned my addr 172.21.15.16:0/2427646017 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:34:58.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.402+0000 7f910ad9d700 1 --2- 172.21.15.16:0/2427646017 >> v2:172.21.15.16:3300/0 conn(0x7f910c0a0a30 0x7f910c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:58.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.402+0000 7f910b59e700 1 -- 172.21.15.16:0/2427646017 >> v2:172.21.15.16:3300/0 conn(0x7f910c0a0a30 msgr2=0x7f910c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:58.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.402+0000 7f910b59e700 1 --2- 172.21.15.16:0/2427646017 >> v2:172.21.15.16:3300/0 conn(0x7f910c0a0a30 0x7f910c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:58.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.402+0000 7f910b59e700 1 -- 172.21.15.16:0/2427646017 >> v1:172.21.15.16:6789/0 conn(0x7f910c09d630 legacy=0x7f910c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:58.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.402+0000 7f910b59e700 1 -- 172.21.15.16:0/2427646017 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f910c09dc50 con 0x7f910c138d80 2023-09-30T22:34:58.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.402+0000 7f910b59e700 1 --2- 172.21.15.16:0/2427646017 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f910c138d80 0x7f910c13b160 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f9104009d90 tx=0x7f910400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7eab725659131aae server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:58.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.402+0000 7f9109d9b700 1 -- 172.21.15.16:0/2427646017 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910400e070 con 0x7f910c138d80 2023-09-30T22:34:58.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.402+0000 7f9109d9b700 1 -- 172.21.15.16:0/2427646017 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f910400bb20 con 0x7f910c138d80 2023-09-30T22:34:58.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.402+0000 7f9109d9b700 1 -- 172.21.15.16:0/2427646017 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9104012430 con 0x7f910c138d80 2023-09-30T22:34:58.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.403+0000 7f9111051700 1 -- 172.21.15.16:0/2427646017 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f910c138d80 msgr2=0x7f910c13b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:58.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.403+0000 7f9111051700 1 --2- 172.21.15.16:0/2427646017 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f910c138d80 0x7f910c13b160 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f9104009d90 tx=0x7f910400b330 comp rx=0 tx=0).stop 2023-09-30T22:34:58.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.403+0000 7f9111051700 1 -- 172.21.15.16:0/2427646017 shutdown_connections 2023-09-30T22:34:58.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.403+0000 7f9111051700 1 --2- 172.21.15.16:0/2427646017 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f910c138d80 0x7f910c13b160 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:58.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.403+0000 7f9111051700 1 --2- 172.21.15.16:0/2427646017 >> v2:172.21.15.16:3300/0 conn(0x7f910c0a0a30 0x7f910c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:58.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.403+0000 7f9111051700 1 -- 172.21.15.16:0/2427646017 >> 172.21.15.16:0/2427646017 conn(0x7f910c0a3050 msgr2=0x7f910c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:58.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.403+0000 7f9111051700 1 -- 172.21.15.16:0/2427646017 shutdown_connections 2023-09-30T22:34:58.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.403+0000 7f9111051700 1 -- 172.21.15.16:0/2427646017 wait complete. 2023-09-30T22:34:58.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.404+0000 7f9111051700 1 Processor -- start 2023-09-30T22:34:58.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.405+0000 7f9111051700 1 -- start start 2023-09-30T22:34:58.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.405+0000 7f9111051700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f910c09d630 0x7f910c120620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:58.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.405+0000 7f9111051700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f910c0a0a30 0x7f910c120b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:58.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.405+0000 7f9111051700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f910c138d80 0x7f910c1210a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:58.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.405+0000 7f9111051700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f910c123e60 con 0x7f910c09d630 2023-09-30T22:34:58.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.405+0000 7f9111051700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f910c122ff0 con 0x7f910c0a0a30 2023-09-30T22:34:58.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.405+0000 7f9111051700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f910c124060 con 0x7f910c138d80 2023-09-30T22:34:58.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.405+0000 7f910b59e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f910c138d80 0x7f910c1210a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:58.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.405+0000 7f910b59e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f910c138d80 0x7f910c1210a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:33908/0 (socket says 172.21.15.16:33908) 2023-09-30T22:34:58.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.405+0000 7f910b59e700 1 -- 172.21.15.16:0/2828541291 learned_addr learned my addr 172.21.15.16:0/2828541291 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:58.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.405+0000 7f910ad9d700 1 --2- 172.21.15.16:0/2828541291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f910c09d630 0x7f910c120620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:58.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.405+0000 7f910a59c700 1 --2- 172.21.15.16:0/2828541291 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f910c0a0a30 0x7f910c120b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:58.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.406+0000 7f910b59e700 1 -- 172.21.15.16:0/2828541291 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f910c0a0a30 msgr2=0x7f910c120b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:58.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.406+0000 7f910b59e700 1 --2- 172.21.15.16:0/2828541291 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f910c0a0a30 0x7f910c120b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:58.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.406+0000 7f910b59e700 1 -- 172.21.15.16:0/2828541291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f910c09d630 msgr2=0x7f910c120620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:58.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.406+0000 7f910b59e700 1 --2- 172.21.15.16:0/2828541291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f910c09d630 0x7f910c120620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:58.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.406+0000 7f910b59e700 1 -- 172.21.15.16:0/2828541291 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9104009a70 con 0x7f910c138d80 2023-09-30T22:34:58.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.406+0000 7f910a59c700 1 --2- 172.21.15.16:0/2828541291 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f910c0a0a30 0x7f910c120b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:34:58.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.406+0000 7f910b59e700 1 --2- 172.21.15.16:0/2828541291 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f910c138d80 0x7f910c1210a0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f9104014040 tx=0x7f9104004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:58.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.406+0000 7f90f37fe700 1 -- 172.21.15.16:0/2828541291 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9104012430 con 0x7f910c138d80 2023-09-30T22:34:58.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.406+0000 7f90f37fe700 1 -- 172.21.15.16:0/2828541291 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9104020070 con 0x7f910c138d80 2023-09-30T22:34:58.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.406+0000 7f9111051700 1 -- 172.21.15.16:0/2828541291 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f910c123410 con 0x7f910c138d80 2023-09-30T22:34:58.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.406+0000 7f90f37fe700 1 -- 172.21.15.16:0/2828541291 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9104003ea0 con 0x7f910c138d80 2023-09-30T22:34:58.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.406+0000 7f9111051700 1 -- 172.21.15.16:0/2828541291 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f910c1389a0 con 0x7f910c138d80 2023-09-30T22:34:58.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.407+0000 7f90f37fe700 1 -- 172.21.15.16:0/2828541291 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9104004280 con 0x7f910c138d80 2023-09-30T22:34:58.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.407+0000 7f90f37fe700 1 -- 172.21.15.16:0/2828541291 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f9104022070 con 0x7f910c138d80 2023-09-30T22:34:58.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.408+0000 7f9111051700 1 -- 172.21.15.16:0/2828541291 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f910c13b620 con 0x7f910c138d80 2023-09-30T22:34:58.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.412+0000 7f90f37fe700 1 -- 172.21.15.16:0/2828541291 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f910401c400 con 0x7f910c138d80 2023-09-30T22:34:58.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.413+0000 7f6d96215700 1 Processor -- start 2023-09-30T22:34:58.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.413+0000 7f6d96215700 1 -- start start 2023-09-30T22:34:58.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.414+0000 7f6d96215700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d900a0a30 0x7f6d900a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:58.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.414+0000 7f6d96215700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d9009d630 0x7f6d900a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:58.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.414+0000 7f6d96215700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6d90133930 0x7f6d90133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:58.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.414+0000 7f6d96215700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d900a2a70 con 0x7f6d90133930 2023-09-30T22:34:58.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.414+0000 7f6d96215700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d900a01d0 con 0x7f6d900a0a30 2023-09-30T22:34:58.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.414+0000 7f6d96215700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9009e2b0 con 0x7f6d9009d630 2023-09-30T22:34:58.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.414+0000 7f6d8effd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d9009d630 0x7f6d900a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:58.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.414+0000 7f6d8effd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d9009d630 0x7f6d900a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:33910/0 (socket says 172.21.15.16:33910) 2023-09-30T22:34:58.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.414+0000 7f6d8effd700 1 -- 172.21.15.16:0/2947016948 learned_addr learned my addr 172.21.15.16:0/2947016948 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:58.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.414+0000 7f6d8f7fe700 1 --2- 172.21.15.16:0/2947016948 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d900a0a30 0x7f6d900a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:58.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.414+0000 7f6d8ffff700 1 --2- 172.21.15.16:0/2947016948 >> v2:172.21.15.16:3300/0 conn(0x7f6d90133930 0x7f6d90133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:58.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.415+0000 7f6d8effd700 1 -- 172.21.15.16:0/2947016948 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d900a0a30 msgr2=0x7f6d900a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:58.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.415+0000 7f6d8effd700 1 --2- 172.21.15.16:0/2947016948 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d900a0a30 0x7f6d900a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:58.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.415+0000 7f6d8effd700 1 -- 172.21.15.16:0/2947016948 >> v2:172.21.15.16:3300/0 conn(0x7f6d90133930 msgr2=0x7f6d90133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:58.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.415+0000 7f6d8effd700 1 --2- 172.21.15.16:0/2947016948 >> v2:172.21.15.16:3300/0 conn(0x7f6d90133930 0x7f6d90133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:58.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.415+0000 7f6d8effd700 1 -- 172.21.15.16:0/2947016948 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d9009dc50 con 0x7f6d9009d630 2023-09-30T22:34:58.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.415+0000 7f6d8f7fe700 1 --2- 172.21.15.16:0/2947016948 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d900a0a30 0x7f6d900a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:58.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.415+0000 7f6d8effd700 1 --2- 172.21.15.16:0/2947016948 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d9009d630 0x7f6d900a1340 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f6d840098f0 tx=0x7f6d84009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab3e365255caa831 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:58.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.415+0000 7f6d8e7fc700 1 -- 172.21.15.16:0/2947016948 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d84011830 con 0x7f6d9009d630 2023-09-30T22:34:58.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.415+0000 7f6d8e7fc700 1 -- 172.21.15.16:0/2947016948 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d84011990 con 0x7f6d9009d630 2023-09-30T22:34:58.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.415+0000 7f6d8e7fc700 1 -- 172.21.15.16:0/2947016948 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d84011d20 con 0x7f6d9009d630 2023-09-30T22:34:58.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.416+0000 7f6d96215700 1 -- 172.21.15.16:0/2947016948 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d9009d630 msgr2=0x7f6d900a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:58.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.416+0000 7f6d96215700 1 --2- 172.21.15.16:0/2947016948 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d9009d630 0x7f6d900a1340 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f6d840098f0 tx=0x7f6d84009c00 comp rx=0 tx=0).stop 2023-09-30T22:34:58.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.416+0000 7f6d96215700 1 -- 172.21.15.16:0/2947016948 shutdown_connections 2023-09-30T22:34:58.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.416+0000 7f6d96215700 1 --2- 172.21.15.16:0/2947016948 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d900a0a30 0x7f6d900a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:58.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.416+0000 7f6d96215700 1 --2- 172.21.15.16:0/2947016948 >> v2:172.21.15.16:3300/0 conn(0x7f6d90133930 0x7f6d90133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:58.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.416+0000 7f6d96215700 1 --2- 172.21.15.16:0/2947016948 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d9009d630 0x7f6d900a1340 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:58.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.416+0000 7f6d96215700 1 -- 172.21.15.16:0/2947016948 >> 172.21.15.16:0/2947016948 conn(0x7f6d900a3050 msgr2=0x7f6d900a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:58.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.416+0000 7f6d96215700 1 -- 172.21.15.16:0/2947016948 shutdown_connections 2023-09-30T22:34:58.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.416+0000 7f6d96215700 1 -- 172.21.15.16:0/2947016948 wait complete. 2023-09-30T22:34:58.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.417+0000 7f6d96215700 1 Processor -- start 2023-09-30T22:34:58.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.417+0000 7f6d96215700 1 -- start start 2023-09-30T22:34:58.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.417+0000 7f6d96215700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d9009d630 0x7f6d90121160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:58.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.417+0000 7f6d96215700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d900a0a30 0x7f6d901216a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:58.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.418+0000 7f6d96215700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d90133930 0x7f6d90121be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:58.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.418+0000 7f6d96215700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d90122610 con 0x7f6d9009d630 2023-09-30T22:34:58.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.418+0000 7f6d96215700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d901255b0 con 0x7f6d90133930 2023-09-30T22:34:58.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.418+0000 7f6d96215700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6d90122ce0 con 0x7f6d900a0a30 2023-09-30T22:34:58.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.418+0000 7f6d8effd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d900a0a30 0x7f6d901216a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:58.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.418+0000 7f6d8ffff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d90133930 0x7f6d90121be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:58.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.418+0000 7f6d8ffff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d90133930 0x7f6d90121be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:52148/0 (socket says 172.21.15.16:52148) 2023-09-30T22:34:58.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.418+0000 7f6d8effd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d900a0a30 0x7f6d901216a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:33926/0 (socket says 172.21.15.16:33926) 2023-09-30T22:34:58.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.418+0000 7f6d8ffff700 1 -- 172.21.15.16:0/3093449371 learned_addr learned my addr 172.21.15.16:0/3093449371 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:58.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.418+0000 7f6d8f7fe700 1 --2- 172.21.15.16:0/3093449371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d9009d630 0x7f6d90121160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:58.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.418+0000 7f6d8f7fe700 1 -- 172.21.15.16:0/3093449371 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d900a0a30 msgr2=0x7f6d901216a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:58.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.418+0000 7f6d8f7fe700 1 --2- 172.21.15.16:0/3093449371 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d900a0a30 0x7f6d901216a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:58.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.418+0000 7f6d8f7fe700 1 -- 172.21.15.16:0/3093449371 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d90133930 msgr2=0x7f6d90121be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:58.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.418+0000 7f6d8f7fe700 1 --2- 172.21.15.16:0/3093449371 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d90133930 0x7f6d90121be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:58.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.418+0000 7f6d8f7fe700 1 -- 172.21.15.16:0/3093449371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d800094a0 con 0x7f6d9009d630 2023-09-30T22:34:58.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.418+0000 7f6d8ffff700 1 --2- 172.21.15.16:0/3093449371 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d90133930 0x7f6d90121be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:58.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.418+0000 7f6d8f7fe700 1 --2- 172.21.15.16:0/3093449371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d9009d630 0x7f6d90121160 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f6d8000e9e0 tx=0x7f6d8000ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:58.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.419+0000 7f6d77fff700 1 -- 172.21.15.16:0/3093449371 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8000ee80 con 0x7f6d9009d630 2023-09-30T22:34:58.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.419+0000 7f6d77fff700 1 -- 172.21.15.16:0/3093449371 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d8000c850 con 0x7f6d9009d630 2023-09-30T22:34:58.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.419+0000 7f6d96215700 1 -- 172.21.15.16:0/3093449371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d84009550 con 0x7f6d9009d630 2023-09-30T22:34:58.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.419+0000 7f6d77fff700 1 -- 172.21.15.16:0/3093449371 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8000cc00 con 0x7f6d9009d630 2023-09-30T22:34:58.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.419+0000 7f6d96215700 1 -- 172.21.15.16:0/3093449371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d900a41e0 con 0x7f6d9009d630 2023-09-30T22:34:58.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.420+0000 7f6d96215700 1 -- 172.21.15.16:0/3093449371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d900a0470 con 0x7f6d9009d630 2023-09-30T22:34:58.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.421+0000 7f6d77fff700 1 -- 172.21.15.16:0/3093449371 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6d8000cd60 con 0x7f6d9009d630 2023-09-30T22:34:58.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.421+0000 7f6d77fff700 1 -- 172.21.15.16:0/3093449371 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f6d8003fec0 con 0x7f6d9009d630 2023-09-30T22:34:58.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.429+0000 7f6d77fff700 1 -- 172.21.15.16:0/3093449371 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6d80011350 con 0x7f6d9009d630 2023-09-30T22:34:58.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.618+0000 7f5a3461e700 1 -- 172.21.15.16:0/4100454892 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5a2c0718e0 con 0x7f5a2c133540 2023-09-30T22:34:58.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.619+0000 7f5a22ffd700 1 -- 172.21.15.16:0/4100454892 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5a1c01a950 con 0x7f5a2c133540 2023-09-30T22:34:58.623 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:34:58.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.620+0000 7f9111051700 1 -- 172.21.15.16:0/2828541291 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f910c0747f0 con 0x7f910c138d80 2023-09-30T22:34:58.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.621+0000 7f90f37fe700 1 -- 172.21.15.16:0/2828541291 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9104025b70 con 0x7f910c138d80 2023-09-30T22:34:58.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.621+0000 7f5a3461e700 1 -- 172.21.15.16:0/4100454892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a2c133540 msgr2=0x7f5a2c09aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:58.624 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:34:58.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.621+0000 7f5a3461e700 1 --2- 172.21.15.16:0/4100454892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a2c133540 0x7f5a2c09aad0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c016000 tx=0x7f5a1c004750 comp rx=0 tx=0).stop 2023-09-30T22:34:58.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.621+0000 7f5a3461e700 1 -- 172.21.15.16:0/4100454892 shutdown_connections 2023-09-30T22:34:58.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.621+0000 7f5a3461e700 1 --2- 172.21.15.16:0/4100454892 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5a2c135e60 0x7f5a2c09b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:58.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.621+0000 7f5a3461e700 1 --2- 172.21.15.16:0/4100454892 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5a2c130540 0x7f5a2c09a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:58.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.621+0000 7f5a3461e700 1 --2- 172.21.15.16:0/4100454892 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a2c133540 0x7f5a2c09aad0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:58.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.621+0000 7f5a3461e700 1 -- 172.21.15.16:0/4100454892 >> 172.21.15.16:0/4100454892 conn(0x7f5a2c0a2aa0 msgr2=0x7f5a2c131320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:58.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.621+0000 7f5a3461e700 1 -- 172.21.15.16:0/4100454892 shutdown_connections 2023-09-30T22:34:58.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.621+0000 7f5a3461e700 1 -- 172.21.15.16:0/4100454892 wait complete. 2023-09-30T22:34:58.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.623+0000 7f9111051700 1 -- 172.21.15.16:0/2828541291 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f910c138d80 msgr2=0x7f910c1210a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:58.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.623+0000 7f9111051700 1 --2- 172.21.15.16:0/2828541291 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f910c138d80 0x7f910c1210a0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f9104014040 tx=0x7f9104004ad0 comp rx=0 tx=0).stop 2023-09-30T22:34:58.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.624+0000 7f9111051700 1 -- 172.21.15.16:0/2828541291 shutdown_connections 2023-09-30T22:34:58.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.624+0000 7f9111051700 1 --2- 172.21.15.16:0/2828541291 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f910c138d80 0x7f910c1210a0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:58.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.624+0000 7f9111051700 1 --2- 172.21.15.16:0/2828541291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f910c09d630 0x7f910c120620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:58.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.624+0000 7f9111051700 1 --2- 172.21.15.16:0/2828541291 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f910c0a0a30 0x7f910c120b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:58.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.624+0000 7f9111051700 1 -- 172.21.15.16:0/2828541291 >> 172.21.15.16:0/2828541291 conn(0x7f910c0a3050 msgr2=0x7f910c09ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:58.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.624+0000 7f9111051700 1 -- 172.21.15.16:0/2828541291 shutdown_connections 2023-09-30T22:34:58.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.624+0000 7f9111051700 1 -- 172.21.15.16:0/2828541291 wait complete. 2023-09-30T22:34:58.637 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:34:58.638 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:34:58.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.652+0000 7f6d96215700 1 -- 172.21.15.16:0/3093449371 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6d901245b0 con 0x7f6d9009d630 2023-09-30T22:34:58.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.653+0000 7f6d77fff700 1 -- 172.21.15.16:0/3093449371 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6d80021020 con 0x7f6d9009d630 2023-09-30T22:34:58.653 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:34:58.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.655+0000 7f6d96215700 1 -- 172.21.15.16:0/3093449371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d9009d630 msgr2=0x7f6d90121160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:58.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.655+0000 7f6d96215700 1 --2- 172.21.15.16:0/3093449371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d9009d630 0x7f6d90121160 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f6d8000e9e0 tx=0x7f6d8000ecf0 comp rx=0 tx=0).stop 2023-09-30T22:34:58.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.655+0000 7f6d96215700 1 -- 172.21.15.16:0/3093449371 shutdown_connections 2023-09-30T22:34:58.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.655+0000 7f6d96215700 1 --2- 172.21.15.16:0/3093449371 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6d90133930 0x7f6d90121be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:58.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.655+0000 7f6d96215700 1 --2- 172.21.15.16:0/3093449371 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6d9009d630 0x7f6d90121160 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:58.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.655+0000 7f6d96215700 1 --2- 172.21.15.16:0/3093449371 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6d900a0a30 0x7f6d901216a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:58.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.655+0000 7f6d96215700 1 -- 172.21.15.16:0/3093449371 >> 172.21.15.16:0/3093449371 conn(0x7f6d900a3050 msgr2=0x7f6d90133140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:58.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.655+0000 7f6d96215700 1 -- 172.21.15.16:0/3093449371 shutdown_connections 2023-09-30T22:34:58.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:58.655+0000 7f6d96215700 1 -- 172.21.15.16:0/3093449371 wait complete. 2023-09-30T22:34:58.669 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:34:59.148 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:34:59.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.294+0000 7f486eba7700 1 Processor -- start 2023-09-30T22:34:59.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.294+0000 7f486eba7700 1 -- start start 2023-09-30T22:34:59.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.295+0000 7f486eba7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f486809d6f0 0x7f486809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:59.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.295+0000 7f486eba7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f486809e000 0x7f4868130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:59.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.295+0000 7f486eba7700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f486809e410 0x7f4868130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:59.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.295+0000 7f486eba7700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f486809f520 con 0x7f486809e410 2023-09-30T22:34:59.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.295+0000 7f486eba7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f486809f130 con 0x7f486809d6f0 2023-09-30T22:34:59.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.295+0000 7f486eba7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f486809e930 con 0x7f486809e000 2023-09-30T22:34:59.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.295+0000 7f486c943700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f486809d6f0 0x7f486809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:59.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.295+0000 7f486d144700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f486809e410 0x7f4868130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:59.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.295+0000 7f486c943700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f486809d6f0 0x7f486809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:52154/0 (socket says 172.21.15.16:52154) 2023-09-30T22:34:59.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.295+0000 7f486c943700 1 -- 172.21.15.16:0/553842857 learned_addr learned my addr 172.21.15.16:0/553842857 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:59.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.295+0000 7f4867fff700 1 --2- 172.21.15.16:0/553842857 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f486809e000 0x7f4868130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:59.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.295+0000 7f486c943700 1 -- 172.21.15.16:0/553842857 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f486809e000 msgr2=0x7f4868130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:59.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.295+0000 7f486c943700 1 --2- 172.21.15.16:0/553842857 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f486809e000 0x7f4868130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:59.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.295+0000 7f486c943700 1 -- 172.21.15.16:0/553842857 >> v2:172.21.15.16:3300/0 conn(0x7f486809e410 msgr2=0x7f4868130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:59.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.295+0000 7f486c943700 1 --2- 172.21.15.16:0/553842857 >> v2:172.21.15.16:3300/0 conn(0x7f486809e410 0x7f4868130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:59.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.295+0000 7f486c943700 1 -- 172.21.15.16:0/553842857 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f486809b2d0 con 0x7f486809d6f0 2023-09-30T22:34:59.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.295+0000 7f486d144700 1 --2- 172.21.15.16:0/553842857 >> v2:172.21.15.16:3300/0 conn(0x7f486809e410 0x7f4868130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:59.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.296+0000 7f486c943700 1 --2- 172.21.15.16:0/553842857 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f486809d6f0 0x7f486809dac0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f485c00bbc0 tx=0x7f485c00bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9a4c65edb26afaa server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:59.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.296+0000 7f48677fe700 1 -- 172.21.15.16:0/553842857 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f485c00dcf0 con 0x7f486809d6f0 2023-09-30T22:34:59.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.296+0000 7f48677fe700 1 -- 172.21.15.16:0/553842857 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f485c00de50 con 0x7f486809d6f0 2023-09-30T22:34:59.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.296+0000 7f48677fe700 1 -- 172.21.15.16:0/553842857 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f485c010690 con 0x7f486809d6f0 2023-09-30T22:34:59.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.296+0000 7f486eba7700 1 -- 172.21.15.16:0/553842857 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f486809d6f0 msgr2=0x7f486809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:59.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.296+0000 7f486eba7700 1 --2- 172.21.15.16:0/553842857 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f486809d6f0 0x7f486809dac0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f485c00bbc0 tx=0x7f485c00bed0 comp rx=0 tx=0).stop 2023-09-30T22:34:59.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.296+0000 7f486eba7700 1 -- 172.21.15.16:0/553842857 shutdown_connections 2023-09-30T22:34:59.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.296+0000 7f486eba7700 1 --2- 172.21.15.16:0/553842857 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f486809d6f0 0x7f486809dac0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:59.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.296+0000 7f486eba7700 1 --2- 172.21.15.16:0/553842857 >> v2:172.21.15.16:3300/0 conn(0x7f486809e410 0x7f4868130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:59.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.296+0000 7f486eba7700 1 --2- 172.21.15.16:0/553842857 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f486809e000 0x7f4868130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:59.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.296+0000 7f486eba7700 1 -- 172.21.15.16:0/553842857 >> 172.21.15.16:0/553842857 conn(0x7f4868066f50 msgr2=0x7f486809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:59.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.296+0000 7f486eba7700 1 -- 172.21.15.16:0/553842857 shutdown_connections 2023-09-30T22:34:59.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.296+0000 7f486eba7700 1 -- 172.21.15.16:0/553842857 wait complete. 2023-09-30T22:34:59.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.297+0000 7f486eba7700 1 Processor -- start 2023-09-30T22:34:59.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.297+0000 7f486eba7700 1 -- start start 2023-09-30T22:34:59.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.298+0000 7f486eba7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f486809d6f0 0x7f486811e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:59.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.298+0000 7f486eba7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f486809e000 0x7f486811e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:59.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.298+0000 7f486eba7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f486809e410 0x7f486811eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:59.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.298+0000 7f486eba7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4868120570 con 0x7f486809d6f0 2023-09-30T22:34:59.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.298+0000 7f486eba7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f486811f580 con 0x7f486809e410 2023-09-30T22:34:59.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.298+0000 7f486eba7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f486809c700 con 0x7f486809e000 2023-09-30T22:34:59.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.298+0000 7f486c943700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f486809d6f0 0x7f486811e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:59.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.298+0000 7f486c943700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f486809d6f0 0x7f486811e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43820/0 (socket says 172.21.15.16:43820) 2023-09-30T22:34:59.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.298+0000 7f486c943700 1 -- 172.21.15.16:0/1927779053 learned_addr learned my addr 172.21.15.16:0/1927779053 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:59.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.298+0000 7f486d144700 1 --2- 172.21.15.16:0/1927779053 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f486809e410 0x7f486811eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:59.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.298+0000 7f4867fff700 1 --2- 172.21.15.16:0/1927779053 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f486809e000 0x7f486811e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:59.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.299+0000 7f486c943700 1 -- 172.21.15.16:0/1927779053 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f486809e000 msgr2=0x7f486811e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:59.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.299+0000 7f486c943700 1 --2- 172.21.15.16:0/1927779053 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f486809e000 0x7f486811e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:59.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.299+0000 7f486c943700 1 -- 172.21.15.16:0/1927779053 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f486809e410 msgr2=0x7f486811eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:59.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.299+0000 7f486c943700 1 --2- 172.21.15.16:0/1927779053 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f486809e410 0x7f486811eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:59.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.299+0000 7f486c943700 1 -- 172.21.15.16:0/1927779053 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4860009a70 con 0x7f486809d6f0 2023-09-30T22:34:59.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.299+0000 7f486d144700 1 --2- 172.21.15.16:0/1927779053 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f486809e410 0x7f486811eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:59.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.299+0000 7f486c943700 1 --2- 172.21.15.16:0/1927779053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f486809d6f0 0x7f486811e120 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f486809b9e0 tx=0x7f485c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:59.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.299+0000 7f48657fa700 1 -- 172.21.15.16:0/1927779053 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f485c01f070 con 0x7f486809d6f0 2023-09-30T22:34:59.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.299+0000 7f48657fa700 1 -- 172.21.15.16:0/1927779053 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f485c0038b0 con 0x7f486809d6f0 2023-09-30T22:34:59.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.299+0000 7f48657fa700 1 -- 172.21.15.16:0/1927779053 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f485c003c40 con 0x7f486809d6f0 2023-09-30T22:34:59.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.299+0000 7f486eba7700 1 -- 172.21.15.16:0/1927779053 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f485c00b7f0 con 0x7f486809d6f0 2023-09-30T22:34:59.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.299+0000 7f486eba7700 1 -- 172.21.15.16:0/1927779053 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4868121340 con 0x7f486809d6f0 2023-09-30T22:34:59.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.300+0000 7f48657fa700 1 -- 172.21.15.16:0/1927779053 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f485c019400 con 0x7f486809d6f0 2023-09-30T22:34:59.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.300+0000 7f48657fa700 1 -- 172.21.15.16:0/1927779053 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f485c042760 con 0x7f486809d6f0 2023-09-30T22:34:59.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.300+0000 7f486eba7700 1 -- 172.21.15.16:0/1927779053 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4868121530 con 0x7f486809d6f0 2023-09-30T22:34:59.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.305+0000 7f48657fa700 1 -- 172.21.15.16:0/1927779053 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f485c003da0 con 0x7f486809d6f0 2023-09-30T22:34:59.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.508+0000 7f486eba7700 1 -- 172.21.15.16:0/1927779053 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4868121cb0 con 0x7f486809d6f0 2023-09-30T22:34:59.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.509+0000 7f48657fa700 1 -- 172.21.15.16:0/1927779053 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f485c02ab00 con 0x7f486809d6f0 2023-09-30T22:34:59.509 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:34:59.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.511+0000 7f486eba7700 1 -- 172.21.15.16:0/1927779053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f486809d6f0 msgr2=0x7f486811e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:59.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.511+0000 7f486eba7700 1 --2- 172.21.15.16:0/1927779053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f486809d6f0 0x7f486811e120 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f486809b9e0 tx=0x7f485c004750 comp rx=0 tx=0).stop 2023-09-30T22:34:59.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.511+0000 7f486eba7700 1 -- 172.21.15.16:0/1927779053 shutdown_connections 2023-09-30T22:34:59.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.511+0000 7f486eba7700 1 --2- 172.21.15.16:0/1927779053 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f486809e410 0x7f486811eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:59.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.511+0000 7f486eba7700 1 --2- 172.21.15.16:0/1927779053 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f486809d6f0 0x7f486811e120 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:59.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.511+0000 7f486eba7700 1 --2- 172.21.15.16:0/1927779053 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f486809e000 0x7f486811e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:59.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.511+0000 7f486eba7700 1 -- 172.21.15.16:0/1927779053 >> 172.21.15.16:0/1927779053 conn(0x7f4868066f50 msgr2=0x7f486806a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:59.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.511+0000 7f486eba7700 1 -- 172.21.15.16:0/1927779053 shutdown_connections 2023-09-30T22:34:59.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.511+0000 7f486eba7700 1 -- 172.21.15.16:0/1927779053 wait complete. 2023-09-30T22:34:59.525 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:34:59.638 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:34:59.639 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:34:59.670 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:34:59.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.794+0000 7f0e04874700 1 Processor -- start 2023-09-30T22:34:59.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.794+0000 7f7da49be700 1 Processor -- start 2023-09-30T22:34:59.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.794+0000 7f0e04874700 1 -- start start 2023-09-30T22:34:59.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.794+0000 7f7da49be700 1 -- start start 2023-09-30T22:34:59.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.795+0000 7f7da49be700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7da00a0a30 0x7f7da00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:59.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.795+0000 7f7da49be700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7da00a1340 0x7f7da0133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:59.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.795+0000 7f0e04874700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0e0009d630 0x7f0e000a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:59.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.795+0000 7f0e04874700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0e00138d80 0x7f0e0013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:59.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.795+0000 7f0e04874700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e000a2a70 con 0x7f0e000a0a30 2023-09-30T22:34:59.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.795+0000 7f0e04874700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e000a01d0 con 0x7f0e00138d80 2023-09-30T22:34:59.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.795+0000 7f0e04874700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0009e2b0 con 0x7f0e0009d630 2023-09-30T22:34:59.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.795+0000 7f0dfed9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0e00138d80 0x7f0e0013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:59.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.795+0000 7f0dfe59c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f0e000a0a30 0x7f0e000a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50770/0 (socket says 172.21.15.16:50770) 2023-09-30T22:34:59.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.795+0000 7f0dfe59c700 1 -- 172.21.15.16:0/3547669038 learned_addr learned my addr 172.21.15.16:0/3547669038 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:34:59.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.795+0000 7f0dfdd9b700 1 --2- 172.21.15.16:0/3547669038 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0e0009d630 0x7f0e000a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:59.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.795+0000 7f7da49be700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7da0133fa0 0x7f7da0134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:59.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.795+0000 7f7da49be700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7da009edd0 con 0x7f7da00a1340 2023-09-30T22:34:59.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.795+0000 7f7da49be700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7da009e1f0 con 0x7f7da0133fa0 2023-09-30T22:34:59.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.795+0000 7f7da49be700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7da009dc50 con 0x7f7da00a0a30 2023-09-30T22:34:59.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.795+0000 7f7d9e59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7da00a0a30 0x7f7da00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:59.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.795+0000 7f7d9e59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7da00a0a30 0x7f7da00a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:33938/0 (socket says 172.21.15.16:33938) 2023-09-30T22:34:59.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.795+0000 7f7d9e59c700 1 -- 172.21.15.16:0/3688931621 learned_addr learned my addr 172.21.15.16:0/3688931621 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:59.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.795+0000 7f0dfd59a700 1 -- 172.21.15.16:0/3547669038 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1987238191 0 0) 0x7f0e000a2a70 con 0x7f0e000a0a30 2023-09-30T22:34:59.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.795+0000 7f0dfd59a700 1 -- 172.21.15.16:0/3547669038 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0de80035b0 con 0x7f0e000a0a30 2023-09-30T22:34:59.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.795+0000 7f7d9dd9b700 1 --2- 172.21.15.16:0/3688931621 >> v2:172.21.15.16:3300/0 conn(0x7f7da00a1340 0x7f7da0133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:59.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.796+0000 7f0dfed9d700 1 -- 172.21.15.16:0/3547669038 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0e0009d630 msgr2=0x7f0e000a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:59.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.796+0000 7f0dfed9d700 1 --2- 172.21.15.16:0/3547669038 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0e0009d630 0x7f0e000a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:59.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.796+0000 7f0dfed9d700 1 -- 172.21.15.16:0/3547669038 >> v1:172.21.15.16:6789/0 conn(0x7f0e000a0a30 legacy=0x7f0e000a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:59.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.796+0000 7f0dfed9d700 1 -- 172.21.15.16:0/3547669038 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e0009dc50 con 0x7f0e00138d80 2023-09-30T22:34:59.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.796+0000 7f7d9e59c700 1 -- 172.21.15.16:0/3688931621 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7da0133fa0 msgr2=0x7f7da0134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:59.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.796+0000 7f7d9e59c700 1 --2- 172.21.15.16:0/3688931621 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7da0133fa0 0x7f7da0134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:59.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.796+0000 7f7d9e59c700 1 -- 172.21.15.16:0/3688931621 >> v2:172.21.15.16:3300/0 conn(0x7f7da00a1340 msgr2=0x7f7da0133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:59.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.796+0000 7f7d9e59c700 1 --2- 172.21.15.16:0/3688931621 >> v2:172.21.15.16:3300/0 conn(0x7f7da00a1340 0x7f7da0133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:59.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.796+0000 7f7d9e59c700 1 -- 172.21.15.16:0/3688931621 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7da00a1fe0 con 0x7f7da00a0a30 2023-09-30T22:34:59.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.796+0000 7f7d9e59c700 1 --2- 172.21.15.16:0/3688931621 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7da00a0a30 0x7f7da00a0e00 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f7d90009920 tx=0x7f7d90009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14eb81e18f8a46cb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:59.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.796+0000 7f0dfed9d700 1 --2- 172.21.15.16:0/3547669038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e00138d80 0x7f0e0013b160 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f0df8009d90 tx=0x7f0df800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b4606b897afaea4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:59.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.796+0000 7f0dfd59a700 1 -- 172.21.15.16:0/3547669038 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df800e070 con 0x7f0e00138d80 2023-09-30T22:34:59.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.796+0000 7f7d9d59a700 1 -- 172.21.15.16:0/3688931621 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d90011830 con 0x7f7da00a0a30 2023-09-30T22:34:59.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.796+0000 7f0dfd59a700 1 -- 172.21.15.16:0/3547669038 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0df800bb20 con 0x7f0e00138d80 2023-09-30T22:34:59.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.796+0000 7f0dfd59a700 1 -- 172.21.15.16:0/3547669038 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df8012430 con 0x7f0e00138d80 2023-09-30T22:34:59.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.796+0000 7f7d9d59a700 1 -- 172.21.15.16:0/3688931621 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d90011990 con 0x7f7da00a0a30 2023-09-30T22:34:59.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.796+0000 7f7d9d59a700 1 -- 172.21.15.16:0/3688931621 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d90011d20 con 0x7f7da00a0a30 2023-09-30T22:34:59.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.797+0000 7f0e04874700 1 -- 172.21.15.16:0/3547669038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e00138d80 msgr2=0x7f0e0013b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:59.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.797+0000 7f7da49be700 1 -- 172.21.15.16:0/3688931621 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7da00a0a30 msgr2=0x7f7da00a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:59.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.797+0000 7f7da49be700 1 --2- 172.21.15.16:0/3688931621 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7da00a0a30 0x7f7da00a0e00 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f7d90009920 tx=0x7f7d90009c30 comp rx=0 tx=0).stop 2023-09-30T22:34:59.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.797+0000 7f0e04874700 1 --2- 172.21.15.16:0/3547669038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e00138d80 0x7f0e0013b160 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f0df8009d90 tx=0x7f0df800b330 comp rx=0 tx=0).stop 2023-09-30T22:34:59.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.797+0000 7f7da49be700 1 -- 172.21.15.16:0/3688931621 shutdown_connections 2023-09-30T22:34:59.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.797+0000 7f7da49be700 1 --2- 172.21.15.16:0/3688931621 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7da0133fa0 0x7f7da0134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:59.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.797+0000 7f0e04874700 1 -- 172.21.15.16:0/3547669038 shutdown_connections 2023-09-30T22:34:59.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.797+0000 7f0e04874700 1 --2- 172.21.15.16:0/3547669038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e00138d80 0x7f0e0013b160 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:59.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.797+0000 7f0e04874700 1 --2- 172.21.15.16:0/3547669038 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0e0009d630 0x7f0e000a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:59.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.797+0000 7f7da49be700 1 --2- 172.21.15.16:0/3688931621 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7da00a0a30 0x7f7da00a0e00 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:59.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.797+0000 7f7da49be700 1 --2- 172.21.15.16:0/3688931621 >> v2:172.21.15.16:3300/0 conn(0x7f7da00a1340 0x7f7da0133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:59.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.797+0000 7f7da49be700 1 -- 172.21.15.16:0/3688931621 >> 172.21.15.16:0/3688931621 conn(0x7f7da00a6550 msgr2=0x7f7da00a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:59.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.797+0000 7f0e04874700 1 -- 172.21.15.16:0/3547669038 >> 172.21.15.16:0/3547669038 conn(0x7f0e000a3050 msgr2=0x7f0e000a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:59.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.797+0000 7f7da49be700 1 -- 172.21.15.16:0/3688931621 shutdown_connections 2023-09-30T22:34:59.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.797+0000 7f0e04874700 1 -- 172.21.15.16:0/3547669038 shutdown_connections 2023-09-30T22:34:59.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.797+0000 7f7da49be700 1 -- 172.21.15.16:0/3688931621 wait complete. 2023-09-30T22:34:59.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.799+0000 7f7da49be700 1 Processor -- start 2023-09-30T22:34:59.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.799+0000 7f0e04874700 1 -- 172.21.15.16:0/3547669038 wait complete. 2023-09-30T22:34:59.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.799+0000 7f7da49be700 1 -- start start 2023-09-30T22:34:59.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.799+0000 7f7da49be700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7da00a0a30 0x7f7da01201b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:59.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.799+0000 7f7da49be700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7da00a1340 0x7f7da01206f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:59.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.799+0000 7f7da49be700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7da0133fa0 0x7f7da0120c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:59.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.799+0000 7f7da49be700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7da0121dc0 con 0x7f7da00a0a30 2023-09-30T22:34:59.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.799+0000 7f7da49be700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7da0123010 con 0x7f7da0133fa0 2023-09-30T22:34:59.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.799+0000 7f7da49be700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7da01223b0 con 0x7f7da00a1340 2023-09-30T22:34:59.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.799+0000 7f7d9ed9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7da0133fa0 0x7f7da0120c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:59.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.799+0000 7f7d9e59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7da00a0a30 0x7f7da01201b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:59.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.799+0000 7f7d9ed9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7da0133fa0 0x7f7da0120c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:52174/0 (socket says 172.21.15.16:52174) 2023-09-30T22:34:59.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.799+0000 7f7d9ed9d700 1 -- 172.21.15.16:0/2091407132 learned_addr learned my addr 172.21.15.16:0/2091407132 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:59.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.799+0000 7f7d9e59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7da00a0a30 0x7f7da01201b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43846/0 (socket says 172.21.15.16:43846) 2023-09-30T22:34:59.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.799+0000 7f7d9dd9b700 1 --2- 172.21.15.16:0/2091407132 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7da00a1340 0x7f7da01206f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:59.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.800+0000 7f7d9ed9d700 1 -- 172.21.15.16:0/2091407132 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7da00a1340 msgr2=0x7f7da01206f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:59.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.800+0000 7f7d9ed9d700 1 --2- 172.21.15.16:0/2091407132 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7da00a1340 0x7f7da01206f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:59.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.800+0000 7f7d9ed9d700 1 -- 172.21.15.16:0/2091407132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7da00a0a30 msgr2=0x7f7da01201b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:59.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.800+0000 7f7d9ed9d700 1 --2- 172.21.15.16:0/2091407132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7da00a0a30 0x7f7da01201b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:59.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.800+0000 7f7d9ed9d700 1 -- 172.21.15.16:0/2091407132 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d90009550 con 0x7f7da0133fa0 2023-09-30T22:34:59.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.800+0000 7f0e04874700 1 Processor -- start 2023-09-30T22:34:59.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.800+0000 7f7d9ed9d700 1 --2- 172.21.15.16:0/2091407132 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7da0133fa0 0x7f7da0120c30 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f7d9800b970 tx=0x7f7d9800bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:59.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.800+0000 7f7d8effd700 1 -- 172.21.15.16:0/2091407132 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9800d860 con 0x7f7da0133fa0 2023-09-30T22:34:59.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.800+0000 7f7d8effd700 1 -- 172.21.15.16:0/2091407132 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d9800d9e0 con 0x7f7da0133fa0 2023-09-30T22:34:59.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.800+0000 7f7da49be700 1 -- 172.21.15.16:0/2091407132 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7da0123460 con 0x7f7da0133fa0 2023-09-30T22:34:59.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.800+0000 7f0e04874700 1 -- start start 2023-09-30T22:34:59.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.801+0000 7f0e04874700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0e0009d630 0x7f0e0012c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:59.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.801+0000 7f0e04874700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0e000a0a30 0x7f0e0012c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:59.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.801+0000 7f0e04874700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e00138d80 0x7f0e0012cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:59.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.801+0000 7f0e04874700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e00138a90 con 0x7f0e00138d80 2023-09-30T22:34:59.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.801+0000 7f0e04874700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0012d7e0 con 0x7f0e0009d630 2023-09-30T22:34:59.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.801+0000 7f0e04874700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0e001388a0 con 0x7f0e000a0a30 2023-09-30T22:34:59.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.800+0000 7f7d8effd700 1 -- 172.21.15.16:0/2091407132 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9800dd90 con 0x7f7da0133fa0 2023-09-30T22:34:59.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.800+0000 7f7da49be700 1 -- 172.21.15.16:0/2091407132 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7da01239f0 con 0x7f7da0133fa0 2023-09-30T22:34:59.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.801+0000 7f0dfed9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e00138d80 0x7f0e0012cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:59.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.801+0000 7f0dfed9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e00138d80 0x7f0e0012cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43848/0 (socket says 172.21.15.16:43848) 2023-09-30T22:34:59.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.801+0000 7f0dfed9d700 1 -- 172.21.15.16:0/2502363364 learned_addr learned my addr 172.21.15.16:0/2502363364 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:59.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.802+0000 7f7da49be700 1 -- 172.21.15.16:0/2091407132 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7da0074ea0 con 0x7f7da0133fa0 2023-09-30T22:34:59.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.802+0000 7f0dfdd9b700 1 --2- 172.21.15.16:0/2502363364 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0e000a0a30 0x7f0e0012c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:59.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.802+0000 7f0dfe59c700 1 --2- 172.21.15.16:0/2502363364 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0e0009d630 0x7f0e0012c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:59.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.802+0000 7f0dfed9d700 1 -- 172.21.15.16:0/2502363364 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0e000a0a30 msgr2=0x7f0e0012c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:59.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.802+0000 7f0dfed9d700 1 --2- 172.21.15.16:0/2502363364 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0e000a0a30 0x7f0e0012c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:59.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.802+0000 7f0dfed9d700 1 -- 172.21.15.16:0/2502363364 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0e0009d630 msgr2=0x7f0e0012c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:59.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.802+0000 7f0dfed9d700 1 --2- 172.21.15.16:0/2502363364 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0e0009d630 0x7f0e0012c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:59.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.802+0000 7f0dfed9d700 1 -- 172.21.15.16:0/2502363364 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0df8009a70 con 0x7f0e00138d80 2023-09-30T22:34:59.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.802+0000 7f0dfe59c700 1 --2- 172.21.15.16:0/2502363364 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0e0009d630 0x7f0e0012c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:34:59.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.802+0000 7f7d8effd700 1 -- 172.21.15.16:0/2091407132 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7d980203f0 con 0x7f7da0133fa0 2023-09-30T22:34:59.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.802+0000 7f0dfed9d700 1 --2- 172.21.15.16:0/2502363364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e00138d80 0x7f0e0012cc20 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f0df8013000 tx=0x7f0df8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:59.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.802+0000 7f7d8effd700 1 -- 172.21.15.16:0/2091407132 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f7d98006e40 con 0x7f7da0133fa0 2023-09-30T22:34:59.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.802+0000 7f0deeffd700 1 -- 172.21.15.16:0/2502363364 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df8012a90 con 0x7f0e00138d80 2023-09-30T22:34:59.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.803+0000 7f0deeffd700 1 -- 172.21.15.16:0/2502363364 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0df8028070 con 0x7f0e00138d80 2023-09-30T22:34:59.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.803+0000 7f0e04874700 1 -- 172.21.15.16:0/2502363364 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e0012e150 con 0x7f0e00138d80 2023-09-30T22:34:59.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.803+0000 7f0deeffd700 1 -- 172.21.15.16:0/2502363364 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df8012e90 con 0x7f0e00138d80 2023-09-30T22:34:59.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.803+0000 7f0e04874700 1 -- 172.21.15.16:0/2502363364 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e0012e8c0 con 0x7f0e00138d80 2023-09-30T22:34:59.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.804+0000 7f0deeffd700 1 -- 172.21.15.16:0/2502363364 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0df8004040 con 0x7f0e00138d80 2023-09-30T22:34:59.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.804+0000 7f0e04874700 1 -- 172.21.15.16:0/2502363364 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e000747f0 con 0x7f0e00138d80 2023-09-30T22:34:59.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.804+0000 7f0deeffd700 1 -- 172.21.15.16:0/2502363364 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f0df802a070 con 0x7f0e00138d80 2023-09-30T22:34:59.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.807+0000 7f7d8effd700 1 -- 172.21.15.16:0/2091407132 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7d980206a0 con 0x7f7da0133fa0 2023-09-30T22:34:59.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.812+0000 7f0deeffd700 1 -- 172.21.15.16:0/2502363364 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0df801a970 con 0x7f0e00138d80 2023-09-30T22:34:59.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.833+0000 7fe409bc4700 1 Processor -- start 2023-09-30T22:34:59.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.833+0000 7fe409bc4700 1 -- start start 2023-09-30T22:34:59.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.833+0000 7fe409bc4700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe40409e1d0 0x7fe404138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:59.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.833+0000 7fe409bc4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe404139170 0x7fe40413b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:59.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.833+0000 7fe409bc4700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe40409b120 con 0x7fe40409d6f0 2023-09-30T22:34:59.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.833+0000 7fe409bc4700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe40406b460 con 0x7fe404139170 2023-09-30T22:34:59.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.833+0000 7fe409bc4700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe40409a7e0 con 0x7fe40409e1d0 2023-09-30T22:34:59.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.834+0000 7fe402ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe40409e1d0 0x7fe404138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:59.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.834+0000 7fe402ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe40409e1d0 0x7fe404138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:52202/0 (socket says 172.21.15.16:52202) 2023-09-30T22:34:59.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.834+0000 7fe403fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe404139170 0x7fe40413b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:59.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.834+0000 7fe402ffd700 1 -- 172.21.15.16:0/3008260112 learned_addr learned my addr 172.21.15.16:0/3008260112 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:59.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.834+0000 7fe402ffd700 1 -- 172.21.15.16:0/3008260112 >> v2:172.21.15.16:3300/0 conn(0x7fe404139170 msgr2=0x7fe40413b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:34:59.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.834+0000 7fe402ffd700 1 --2- 172.21.15.16:0/3008260112 >> v2:172.21.15.16:3300/0 conn(0x7fe404139170 0x7fe40413b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:59.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.834+0000 7fe402ffd700 1 -- 172.21.15.16:0/3008260112 >> v1:172.21.15.16:6789/0 conn(0x7fe40409d6f0 legacy=0x7fe40409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:59.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.834+0000 7fe402ffd700 1 -- 172.21.15.16:0/3008260112 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe40409a500 con 0x7fe40409e1d0 2023-09-30T22:34:59.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.834+0000 7fe403fff700 1 --2- 172.21.15.16:0/3008260112 >> v2:172.21.15.16:3300/0 conn(0x7fe404139170 0x7fe40413b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:59.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.834+0000 7fe402ffd700 1 --2- 172.21.15.16:0/3008260112 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe40409e1d0 0x7fe404138c30 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8009920 tx=0x7fe3f8009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1308c4347cefe9da server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:59.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.835+0000 7fe4027fc700 1 -- 172.21.15.16:0/3008260112 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f8011830 con 0x7fe40409e1d0 2023-09-30T22:34:59.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.835+0000 7fe4027fc700 1 -- 172.21.15.16:0/3008260112 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f8011990 con 0x7fe40409e1d0 2023-09-30T22:34:59.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.835+0000 7fe4027fc700 1 -- 172.21.15.16:0/3008260112 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f8011d20 con 0x7fe40409e1d0 2023-09-30T22:34:59.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.835+0000 7fe409bc4700 1 -- 172.21.15.16:0/3008260112 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe40409e1d0 msgr2=0x7fe404138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:59.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.835+0000 7fe409bc4700 1 --2- 172.21.15.16:0/3008260112 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe40409e1d0 0x7fe404138c30 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8009920 tx=0x7fe3f8009c30 comp rx=0 tx=0).stop 2023-09-30T22:34:59.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.835+0000 7fe409bc4700 1 -- 172.21.15.16:0/3008260112 shutdown_connections 2023-09-30T22:34:59.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.835+0000 7fe409bc4700 1 --2- 172.21.15.16:0/3008260112 >> v2:172.21.15.16:3300/0 conn(0x7fe404139170 0x7fe40413b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:59.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.835+0000 7fe409bc4700 1 --2- 172.21.15.16:0/3008260112 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe40409e1d0 0x7fe404138c30 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:59.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.835+0000 7fe409bc4700 1 -- 172.21.15.16:0/3008260112 >> 172.21.15.16:0/3008260112 conn(0x7fe404066f50 msgr2=0x7fe40409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:34:59.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.836+0000 7fe409bc4700 1 -- 172.21.15.16:0/3008260112 shutdown_connections 2023-09-30T22:34:59.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.836+0000 7fe409bc4700 1 -- 172.21.15.16:0/3008260112 wait complete. 2023-09-30T22:34:59.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.836+0000 7fe409bc4700 1 Processor -- start 2023-09-30T22:34:59.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.837+0000 7fe409bc4700 1 -- start start 2023-09-30T22:34:59.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.837+0000 7fe409bc4700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe40409d6f0 0x7fe4041c8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:59.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.837+0000 7fe409bc4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe40409e1d0 0x7fe4041c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:59.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.837+0000 7fe409bc4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe404139170 0x7fe4041218e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:34:59.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.837+0000 7fe409bc4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe404125200 con 0x7fe40409e1d0 2023-09-30T22:34:59.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.837+0000 7fe409bc4700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4041234f0 con 0x7fe40409d6f0 2023-09-30T22:34:59.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.837+0000 7fe409bc4700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe4041377b0 con 0x7fe404139170 2023-09-30T22:34:59.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.837+0000 7fe4037fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe40409d6f0 0x7fe4041c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:59.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.837+0000 7fe4037fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe40409d6f0 0x7fe4041c8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:52216/0 (socket says 172.21.15.16:52216) 2023-09-30T22:34:59.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.837+0000 7fe4037fe700 1 -- 172.21.15.16:0/3475001187 learned_addr learned my addr 172.21.15.16:0/3475001187 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:34:59.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.837+0000 7fe402ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe40409e1d0 0x7fe4041c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:59.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.837+0000 7fe403fff700 1 --2- 172.21.15.16:0/3475001187 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe404139170 0x7fe4041218e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:34:59.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.838+0000 7fe402ffd700 1 -- 172.21.15.16:0/3475001187 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe404139170 msgr2=0x7fe4041218e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:59.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.838+0000 7fe402ffd700 1 --2- 172.21.15.16:0/3475001187 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe404139170 0x7fe4041218e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:59.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.838+0000 7fe402ffd700 1 -- 172.21.15.16:0/3475001187 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe40409d6f0 msgr2=0x7fe4041c8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:34:59.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.838+0000 7fe402ffd700 1 --2- 172.21.15.16:0/3475001187 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe40409d6f0 0x7fe4041c8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:34:59.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.838+0000 7fe402ffd700 1 -- 172.21.15.16:0/3475001187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3fc009a70 con 0x7fe40409e1d0 2023-09-30T22:34:59.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.838+0000 7fe4037fe700 1 --2- 172.21.15.16:0/3475001187 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe40409d6f0 0x7fe4041c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:59.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.838+0000 7fe403fff700 1 --2- 172.21.15.16:0/3475001187 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe404139170 0x7fe4041218e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:34:59.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.838+0000 7fe402ffd700 1 --2- 172.21.15.16:0/3475001187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe40409e1d0 0x7fe4041c8aa0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8016000 tx=0x7fe3f8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:34:59.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.838+0000 7fe3ebfff700 1 -- 172.21.15.16:0/3475001187 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f80044d0 con 0x7fe40409e1d0 2023-09-30T22:34:59.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.838+0000 7fe3ebfff700 1 -- 172.21.15.16:0/3475001187 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f80036a0 con 0x7fe40409e1d0 2023-09-30T22:34:59.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.838+0000 7fe409bc4700 1 -- 172.21.15.16:0/3475001187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3f8009550 con 0x7fe40409e1d0 2023-09-30T22:34:59.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.838+0000 7fe3ebfff700 1 -- 172.21.15.16:0/3475001187 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f8003a50 con 0x7fe40409e1d0 2023-09-30T22:34:59.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.838+0000 7fe409bc4700 1 -- 172.21.15.16:0/3475001187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe404125b90 con 0x7fe40409e1d0 2023-09-30T22:34:59.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.839+0000 7fe3ebfff700 1 -- 172.21.15.16:0/3475001187 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe3f8003bb0 con 0x7fe40409e1d0 2023-09-30T22:34:59.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.839+0000 7fe409bc4700 1 -- 172.21.15.16:0/3475001187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe404125060 con 0x7fe40409e1d0 2023-09-30T22:34:59.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.840+0000 7fe3ebfff700 1 -- 172.21.15.16:0/3475001187 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fe3f8021030 con 0x7fe40409e1d0 2023-09-30T22:34:59.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:34:59.844+0000 7fe3ebfff700 1 -- 172.21.15.16:0/3475001187 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe404125060 con 0x7fe40409e1d0 2023-09-30T22:35:00.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.019+0000 7f0e04874700 1 -- 172.21.15.16:0/2502363364 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0e0009c320 con 0x7f0e00138d80 2023-09-30T22:35:00.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.019+0000 7f7da49be700 1 -- 172.21.15.16:0/2091407132 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f7da01237e0 con 0x7f7da0133fa0 2023-09-30T22:35:00.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.020+0000 7f0deeffd700 1 -- 172.21.15.16:0/2502363364 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0df8034070 con 0x7f0e00138d80 2023-09-30T22:35:00.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.020+0000 7f7d8effd700 1 -- 172.21.15.16:0/2091407132 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7d98016360 con 0x7f7da0133fa0 2023-09-30T22:35:00.021 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:35:00.021 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:35:00.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.022+0000 7f0e04874700 1 -- 172.21.15.16:0/2502363364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e00138d80 msgr2=0x7f0e0012cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:00.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.022+0000 7f0e04874700 1 --2- 172.21.15.16:0/2502363364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e00138d80 0x7f0e0012cc20 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f0df8013000 tx=0x7f0df8004ad0 comp rx=0 tx=0).stop 2023-09-30T22:35:00.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.022+0000 7f0e04874700 1 -- 172.21.15.16:0/2502363364 shutdown_connections 2023-09-30T22:35:00.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.022+0000 7f0e04874700 1 --2- 172.21.15.16:0/2502363364 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0e0009d630 0x7f0e0012c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:00.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.022+0000 7f0e04874700 1 --2- 172.21.15.16:0/2502363364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0e00138d80 0x7f0e0012cc20 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:00.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.022+0000 7f0e04874700 1 --2- 172.21.15.16:0/2502363364 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0e000a0a30 0x7f0e0012c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:00.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.022+0000 7f0e04874700 1 -- 172.21.15.16:0/2502363364 >> 172.21.15.16:0/2502363364 conn(0x7f0e000a3050 msgr2=0x7f0e00139260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:00.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.022+0000 7f0e04874700 1 -- 172.21.15.16:0/2502363364 shutdown_connections 2023-09-30T22:35:00.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.022+0000 7f0e04874700 1 -- 172.21.15.16:0/2502363364 wait complete. 2023-09-30T22:35:00.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.022+0000 7f7da49be700 1 -- 172.21.15.16:0/2091407132 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7da0133fa0 msgr2=0x7f7da0120c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:00.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.022+0000 7f7da49be700 1 --2- 172.21.15.16:0/2091407132 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7da0133fa0 0x7f7da0120c30 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f7d9800b970 tx=0x7f7d9800bc80 comp rx=0 tx=0).stop 2023-09-30T22:35:00.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.022+0000 7f7da49be700 1 -- 172.21.15.16:0/2091407132 shutdown_connections 2023-09-30T22:35:00.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.022+0000 7f7da49be700 1 --2- 172.21.15.16:0/2091407132 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7da0133fa0 0x7f7da0120c30 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:00.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.022+0000 7f7da49be700 1 --2- 172.21.15.16:0/2091407132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7da00a0a30 0x7f7da01201b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:00.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.022+0000 7f7da49be700 1 --2- 172.21.15.16:0/2091407132 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7da00a1340 0x7f7da01206f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:00.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.022+0000 7f7da49be700 1 -- 172.21.15.16:0/2091407132 >> 172.21.15.16:0/2091407132 conn(0x7f7da00a6550 msgr2=0x7f7da009ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:00.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.023+0000 7f7da49be700 1 -- 172.21.15.16:0/2091407132 shutdown_connections 2023-09-30T22:35:00.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.023+0000 7f7da49be700 1 -- 172.21.15.16:0/2091407132 wait complete. 2023-09-30T22:35:00.038 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:35:00.038 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:35:00.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.049+0000 7fe409bc4700 1 -- 172.21.15.16:0/3475001187 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe404125060 con 0x7fe40409e1d0 2023-09-30T22:35:00.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.050+0000 7fe3ebfff700 1 -- 172.21.15.16:0/3475001187 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe404125060 con 0x7fe40409e1d0 2023-09-30T22:35:00.050 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:35:00.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.052+0000 7fe409bc4700 1 -- 172.21.15.16:0/3475001187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe40409e1d0 msgr2=0x7fe4041c8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:00.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.052+0000 7fe409bc4700 1 --2- 172.21.15.16:0/3475001187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe40409e1d0 0x7fe4041c8aa0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8016000 tx=0x7fe3f8004750 comp rx=0 tx=0).stop 2023-09-30T22:35:00.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.052+0000 7fe409bc4700 1 -- 172.21.15.16:0/3475001187 shutdown_connections 2023-09-30T22:35:00.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.052+0000 7fe409bc4700 1 --2- 172.21.15.16:0/3475001187 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe40409d6f0 0x7fe4041c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:00.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.052+0000 7fe409bc4700 1 --2- 172.21.15.16:0/3475001187 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe404139170 0x7fe4041218e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:00.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.052+0000 7fe409bc4700 1 --2- 172.21.15.16:0/3475001187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe40409e1d0 0x7fe4041c8aa0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:00.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.052+0000 7fe409bc4700 1 -- 172.21.15.16:0/3475001187 >> 172.21.15.16:0/3475001187 conn(0x7fe404066f50 msgr2=0x7fe4040a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:00.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.052+0000 7fe409bc4700 1 -- 172.21.15.16:0/3475001187 shutdown_connections 2023-09-30T22:35:00.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.052+0000 7fe409bc4700 1 -- 172.21.15.16:0/3475001187 wait complete. 2023-09-30T22:35:00.067 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:35:00.526 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:35:00.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.672+0000 7fdbe1071700 1 Processor -- start 2023-09-30T22:35:00.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.672+0000 7fdbe1071700 1 -- start start 2023-09-30T22:35:00.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.673+0000 7fdbe1071700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdbdc09d6f0 0x7fdbdc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:00.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.673+0000 7fdbe1071700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdbdc09e410 0x7fdbdc130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:00.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.673+0000 7fdbe1071700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbdc09f130 con 0x7fdbdc09e000 2023-09-30T22:35:00.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.673+0000 7fdbe1071700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbdc09f520 con 0x7fdbdc09d6f0 2023-09-30T22:35:00.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.673+0000 7fdbe1071700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdbdc09e930 con 0x7fdbdc09e410 2023-09-30T22:35:00.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.673+0000 7fdbdad9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdbdc09d6f0 0x7fdbdc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:00.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.673+0000 7fdbda59c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fdbdc09e000 0x7fdbdc130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50782/0 (socket says 172.21.15.16:50782) 2023-09-30T22:35:00.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.673+0000 7fdbda59c700 1 -- 172.21.15.16:0/3038583623 learned_addr learned my addr 172.21.15.16:0/3038583623 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:00.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.673+0000 7fdbdb59e700 1 --2- 172.21.15.16:0/3038583623 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdbdc09e410 0x7fdbdc130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:00.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.673+0000 7fdbdad9d700 1 -- 172.21.15.16:0/3038583623 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdbdc09e410 msgr2=0x7fdbdc130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:00.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.673+0000 7fdbdad9d700 1 --2- 172.21.15.16:0/3038583623 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdbdc09e410 0x7fdbdc130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:00.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.673+0000 7fdbdad9d700 1 -- 172.21.15.16:0/3038583623 >> v1:172.21.15.16:6789/0 conn(0x7fdbdc09e000 legacy=0x7fdbdc130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:00.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.673+0000 7fdbdad9d700 1 -- 172.21.15.16:0/3038583623 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbdc09b2d0 con 0x7fdbdc09d6f0 2023-09-30T22:35:00.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.673+0000 7fdbdb59e700 1 --2- 172.21.15.16:0/3038583623 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdbdc09e410 0x7fdbdc130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:00.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.673+0000 7fdbd9d9b700 1 -- 172.21.15.16:0/3038583623 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 774440250 0 0) 0x7fdbdc09f130 con 0x7fdbdc09e000 2023-09-30T22:35:00.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.674+0000 7fdbdad9d700 1 --2- 172.21.15.16:0/3038583623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbdc09d6f0 0x7fdbdc09dac0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc00bae0 tx=0x7fdbcc00bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4985a782255b4397 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:00.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.674+0000 7fdbd9d9b700 1 -- 172.21.15.16:0/3038583623 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc00dc70 con 0x7fdbdc09d6f0 2023-09-30T22:35:00.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.674+0000 7fdbd9d9b700 1 -- 172.21.15.16:0/3038583623 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbcc00ddd0 con 0x7fdbdc09d6f0 2023-09-30T22:35:00.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.674+0000 7fdbd9d9b700 1 -- 172.21.15.16:0/3038583623 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc010690 con 0x7fdbdc09d6f0 2023-09-30T22:35:00.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.674+0000 7fdbe1071700 1 -- 172.21.15.16:0/3038583623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbdc09d6f0 msgr2=0x7fdbdc09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:00.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.674+0000 7fdbe1071700 1 --2- 172.21.15.16:0/3038583623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbdc09d6f0 0x7fdbdc09dac0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc00bae0 tx=0x7fdbcc00bdf0 comp rx=0 tx=0).stop 2023-09-30T22:35:00.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.674+0000 7fdbe1071700 1 -- 172.21.15.16:0/3038583623 shutdown_connections 2023-09-30T22:35:00.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.674+0000 7fdbe1071700 1 --2- 172.21.15.16:0/3038583623 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdbdc09e410 0x7fdbdc130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:00.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.674+0000 7fdbe1071700 1 --2- 172.21.15.16:0/3038583623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbdc09d6f0 0x7fdbdc09dac0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:00.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.674+0000 7fdbe1071700 1 -- 172.21.15.16:0/3038583623 >> 172.21.15.16:0/3038583623 conn(0x7fdbdc066f50 msgr2=0x7fdbdc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:00.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.674+0000 7fdbe1071700 1 -- 172.21.15.16:0/3038583623 shutdown_connections 2023-09-30T22:35:00.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.674+0000 7fdbe1071700 1 -- 172.21.15.16:0/3038583623 wait complete. 2023-09-30T22:35:00.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.676+0000 7fdbe1071700 1 Processor -- start 2023-09-30T22:35:00.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.676+0000 7fdbe1071700 1 -- start start 2023-09-30T22:35:00.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.676+0000 7fdbe1071700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdbdc09d6f0 0x7fdbdc11df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:00.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.676+0000 7fdbe1071700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbdc09e000 0x7fdbdc11e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:00.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.676+0000 7fdbe1071700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdbdc09e410 0x7fdbdc11e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:00.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.676+0000 7fdbe1071700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbdc1302d0 con 0x7fdbdc09e000 2023-09-30T22:35:00.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.676+0000 7fdbe1071700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbdc120d50 con 0x7fdbdc09e410 2023-09-30T22:35:00.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.676+0000 7fdbe1071700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdbdc11f3a0 con 0x7fdbdc09d6f0 2023-09-30T22:35:00.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.676+0000 7fdbda59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbdc09e000 0x7fdbdc11e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:00.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.676+0000 7fdbdb59e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdbdc09e410 0x7fdbdc11e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:00.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.676+0000 7fdbdad9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdbdc09d6f0 0x7fdbdc11df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:00.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.676+0000 7fdbda59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbdc09e000 0x7fdbdc11e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43870/0 (socket says 172.21.15.16:43870) 2023-09-30T22:35:00.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.676+0000 7fdbda59c700 1 -- 172.21.15.16:0/2446813983 learned_addr learned my addr 172.21.15.16:0/2446813983 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:00.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.677+0000 7fdbdad9d700 1 -- 172.21.15.16:0/2446813983 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdbdc09e410 msgr2=0x7fdbdc11e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:00.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.677+0000 7fdbdad9d700 1 --2- 172.21.15.16:0/2446813983 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdbdc09e410 0x7fdbdc11e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:00.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.677+0000 7fdbdad9d700 1 -- 172.21.15.16:0/2446813983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbdc09e000 msgr2=0x7fdbdc11e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:00.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.677+0000 7fdbdad9d700 1 --2- 172.21.15.16:0/2446813983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbdc09e000 0x7fdbdc11e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:00.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.677+0000 7fdbdad9d700 1 -- 172.21.15.16:0/2446813983 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd4009a70 con 0x7fdbdc09d6f0 2023-09-30T22:35:00.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.677+0000 7fdbdb59e700 1 --2- 172.21.15.16:0/2446813983 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdbdc09e410 0x7fdbdc11e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:00.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.677+0000 7fdbda59c700 1 --2- 172.21.15.16:0/2446813983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbdc09e000 0x7fdbdc11e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:00.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.677+0000 7fdbdad9d700 1 --2- 172.21.15.16:0/2446813983 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdbdc09d6f0 0x7fdbdc11df40 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc018000 tx=0x7fdbcc004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:00.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.677+0000 7fdbd37fe700 1 -- 172.21.15.16:0/2446813983 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc01f070 con 0x7fdbdc09d6f0 2023-09-30T22:35:00.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.677+0000 7fdbd37fe700 1 -- 172.21.15.16:0/2446813983 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbcc0038b0 con 0x7fdbdc09d6f0 2023-09-30T22:35:00.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.677+0000 7fdbe1071700 1 -- 172.21.15.16:0/2446813983 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbcc00b740 con 0x7fdbdc09d6f0 2023-09-30T22:35:00.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.677+0000 7fdbd37fe700 1 -- 172.21.15.16:0/2446813983 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbcc003c40 con 0x7fdbdc09d6f0 2023-09-30T22:35:00.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.677+0000 7fdbe1071700 1 -- 172.21.15.16:0/2446813983 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbdc1211a0 con 0x7fdbdc09d6f0 2023-09-30T22:35:00.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.678+0000 7fdbd37fe700 1 -- 172.21.15.16:0/2446813983 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdbcc019400 con 0x7fdbdc09d6f0 2023-09-30T22:35:00.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.678+0000 7fdbd37fe700 1 -- 172.21.15.16:0/2446813983 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fdbcc042800 con 0x7fdbdc09d6f0 2023-09-30T22:35:00.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.678+0000 7fdbe1071700 1 -- 172.21.15.16:0/2446813983 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbdc0a1d60 con 0x7fdbdc09d6f0 2023-09-30T22:35:00.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.684+0000 7fdbd37fe700 1 -- 172.21.15.16:0/2446813983 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdbcc029020 con 0x7fdbdc09d6f0 2023-09-30T22:35:00.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.888+0000 7fdbe1071700 1 -- 172.21.15.16:0/2446813983 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fdbdc121690 con 0x7fdbdc09d6f0 2023-09-30T22:35:00.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.889+0000 7fdbd37fe700 1 -- 172.21.15.16:0/2446813983 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdbcc003da0 con 0x7fdbdc09d6f0 2023-09-30T22:35:00.889 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:35:00.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.891+0000 7fdbe1071700 1 -- 172.21.15.16:0/2446813983 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdbdc09d6f0 msgr2=0x7fdbdc11df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:00.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.891+0000 7fdbe1071700 1 --2- 172.21.15.16:0/2446813983 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdbdc09d6f0 0x7fdbdc11df40 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fdbcc018000 tx=0x7fdbcc004750 comp rx=0 tx=0).stop 2023-09-30T22:35:00.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.891+0000 7fdbe1071700 1 -- 172.21.15.16:0/2446813983 shutdown_connections 2023-09-30T22:35:00.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.891+0000 7fdbe1071700 1 --2- 172.21.15.16:0/2446813983 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdbdc09e410 0x7fdbdc11e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:00.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.891+0000 7fdbe1071700 1 --2- 172.21.15.16:0/2446813983 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdbdc09d6f0 0x7fdbdc11df40 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:00.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.891+0000 7fdbe1071700 1 --2- 172.21.15.16:0/2446813983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdbdc09e000 0x7fdbdc11e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:00.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.891+0000 7fdbe1071700 1 -- 172.21.15.16:0/2446813983 >> 172.21.15.16:0/2446813983 conn(0x7fdbdc066f50 msgr2=0x7fdbdc070fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:00.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.891+0000 7fdbe1071700 1 -- 172.21.15.16:0/2446813983 shutdown_connections 2023-09-30T22:35:00.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:00.891+0000 7fdbe1071700 1 -- 172.21.15.16:0/2446813983 wait complete. 2023-09-30T22:35:00.905 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:35:01.039 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:35:01.039 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:35:01.067 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:35:01.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.185+0000 7f814353e700 1 Processor -- start 2023-09-30T22:35:01.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.186+0000 7f814353e700 1 -- start start 2023-09-30T22:35:01.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.186+0000 7f814353e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f813c133420 0x7f813c1337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:01.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.186+0000 7f814353e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f813c133e30 0x7f813c12ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:01.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.186+0000 7f814353e700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f813c06b990 con 0x7f813c134260 2023-09-30T22:35:01.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.186+0000 7f814353e700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f813c0a0590 con 0x7f813c133420 2023-09-30T22:35:01.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.186+0000 7f814353e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f813c09fb80 con 0x7f813c133e30 2023-09-30T22:35:01.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.186+0000 7f81412da700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f813c133420 0x7f813c1337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:01.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.186+0000 7f81412da700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f813c133420 0x7f813c1337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43884/0 (socket says 172.21.15.16:43884) 2023-09-30T22:35:01.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.186+0000 7f8141adb700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f813c134260 0x7f813c1304d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50796/0 (socket says 172.21.15.16:50796) 2023-09-30T22:35:01.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.186+0000 7f81412da700 1 -- 172.21.15.16:0/2014334025 learned_addr learned my addr 172.21.15.16:0/2014334025 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:01.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.186+0000 7f8140ad9700 1 --2- 172.21.15.16:0/2014334025 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f813c133e30 0x7f813c12ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:01.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.187+0000 7f81412da700 1 -- 172.21.15.16:0/2014334025 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f813c133e30 msgr2=0x7f813c12ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:01.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.187+0000 7f81412da700 1 --2- 172.21.15.16:0/2014334025 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f813c133e30 0x7f813c12ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:01.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.187+0000 7f81412da700 1 -- 172.21.15.16:0/2014334025 >> v1:172.21.15.16:6789/0 conn(0x7f813c134260 legacy=0x7f813c1304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:01.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.187+0000 7f81412da700 1 -- 172.21.15.16:0/2014334025 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f813c0a0030 con 0x7f813c133420 2023-09-30T22:35:01.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.187+0000 7f8140ad9700 1 --2- 172.21.15.16:0/2014334025 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f813c133e30 0x7f813c12ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:01.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.187+0000 7f81412da700 1 --2- 172.21.15.16:0/2014334025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f813c133420 0x7f813c1337f0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f81380098f0 tx=0x7f8138009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aba9922964d239c5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:01.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.187+0000 7f8133fff700 1 -- 172.21.15.16:0/2014334025 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8138011830 con 0x7f813c133420 2023-09-30T22:35:01.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.187+0000 7f8133fff700 1 -- 172.21.15.16:0/2014334025 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8138011990 con 0x7f813c133420 2023-09-30T22:35:01.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.187+0000 7f8133fff700 1 -- 172.21.15.16:0/2014334025 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8138011d20 con 0x7f813c133420 2023-09-30T22:35:01.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.188+0000 7f814353e700 1 -- 172.21.15.16:0/2014334025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f813c133420 msgr2=0x7f813c1337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:01.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.188+0000 7f814353e700 1 --2- 172.21.15.16:0/2014334025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f813c133420 0x7f813c1337f0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f81380098f0 tx=0x7f8138009c00 comp rx=0 tx=0).stop 2023-09-30T22:35:01.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.188+0000 7f814353e700 1 -- 172.21.15.16:0/2014334025 shutdown_connections 2023-09-30T22:35:01.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.188+0000 7f814353e700 1 --2- 172.21.15.16:0/2014334025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f813c133420 0x7f813c1337f0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:01.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.188+0000 7f814353e700 1 --2- 172.21.15.16:0/2014334025 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f813c133e30 0x7f813c12ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:01.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.188+0000 7f814353e700 1 -- 172.21.15.16:0/2014334025 >> 172.21.15.16:0/2014334025 conn(0x7f813c0a4210 msgr2=0x7f813c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:01.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.188+0000 7f814353e700 1 -- 172.21.15.16:0/2014334025 shutdown_connections 2023-09-30T22:35:01.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.188+0000 7f814353e700 1 -- 172.21.15.16:0/2014334025 wait complete. 2023-09-30T22:35:01.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.189+0000 7f814353e700 1 Processor -- start 2023-09-30T22:35:01.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.189+0000 7f814353e700 1 -- start start 2023-09-30T22:35:01.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.190+0000 7f814353e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f813c133420 0x7f813c09c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:01.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.190+0000 7f814353e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f813c133e30 0x7f813c09c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:01.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.190+0000 7f814353e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f813c134260 0x7f813c09cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:01.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.190+0000 7f814353e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f813c120010 con 0x7f813c133420 2023-09-30T22:35:01.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.190+0000 7f814353e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f813c1201f0 con 0x7f813c134260 2023-09-30T22:35:01.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.190+0000 7f814353e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f813c12da70 con 0x7f813c133e30 2023-09-30T22:35:01.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.190+0000 7f8140ad9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f813c133e30 0x7f813c09c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:01.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.190+0000 7f81412da700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f813c133420 0x7f813c09c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:01.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.190+0000 7f8141adb700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f813c134260 0x7f813c09cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:01.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.190+0000 7f8140ad9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f813c133e30 0x7f813c09c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:33992/0 (socket says 172.21.15.16:33992) 2023-09-30T22:35:01.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.190+0000 7f81412da700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f813c133420 0x7f813c09c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43890/0 (socket says 172.21.15.16:43890) 2023-09-30T22:35:01.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.190+0000 7f8140ad9700 1 -- 172.21.15.16:0/933964667 learned_addr learned my addr 172.21.15.16:0/933964667 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:01.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.190+0000 7f8141adb700 1 -- 172.21.15.16:0/933964667 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f813c133e30 msgr2=0x7f813c09c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:01.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.190+0000 7f8141adb700 1 --2- 172.21.15.16:0/933964667 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f813c133e30 0x7f813c09c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:01.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.190+0000 7f8141adb700 1 -- 172.21.15.16:0/933964667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f813c133420 msgr2=0x7f813c09c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:01.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.190+0000 7f8141adb700 1 --2- 172.21.15.16:0/933964667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f813c133420 0x7f813c09c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:01.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.190+0000 7f8141adb700 1 -- 172.21.15.16:0/933964667 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f812c0094a0 con 0x7f813c134260 2023-09-30T22:35:01.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.190+0000 7f8140ad9700 1 --2- 172.21.15.16:0/933964667 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f813c133e30 0x7f813c09c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:01.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.190+0000 7f8141adb700 1 --2- 172.21.15.16:0/933964667 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f813c134260 0x7f813c09cc80 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f813400cb40 tx=0x7f813400ce50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:01.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.191+0000 7f8131ffb700 1 -- 172.21.15.16:0/933964667 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813400a960 con 0x7f813c134260 2023-09-30T22:35:01.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.191+0000 7f8131ffb700 1 -- 172.21.15.16:0/933964667 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f813400aac0 con 0x7f813c134260 2023-09-30T22:35:01.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.191+0000 7f814353e700 1 -- 172.21.15.16:0/933964667 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8138009550 con 0x7f813c134260 2023-09-30T22:35:01.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.191+0000 7f8131ffb700 1 -- 172.21.15.16:0/933964667 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813400ae30 con 0x7f813c134260 2023-09-30T22:35:01.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.191+0000 7f814353e700 1 -- 172.21.15.16:0/933964667 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f813c12d620 con 0x7f813c134260 2023-09-30T22:35:01.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.192+0000 7f8131ffb700 1 -- 172.21.15.16:0/933964667 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8134007070 con 0x7f813c134260 2023-09-30T22:35:01.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.192+0000 7f8131ffb700 1 -- 172.21.15.16:0/933964667 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f813403f2f0 con 0x7f813c134260 2023-09-30T22:35:01.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.192+0000 7f814353e700 1 -- 172.21.15.16:0/933964667 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f813c074ea0 con 0x7f813c134260 2023-09-30T22:35:01.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.196+0000 7f8411a40700 1 Processor -- start 2023-09-30T22:35:01.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.196+0000 7f8411a40700 1 -- start start 2023-09-30T22:35:01.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.196+0000 7f8411a40700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f840c130670 0x7f840c130a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:01.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.196+0000 7f8411a40700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f840c0701a0 0x7f840c138540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:01.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.197+0000 7f8411a40700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f840c130260 con 0x7f840c06f680 2023-09-30T22:35:01.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.197+0000 7f8411a40700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f840c09a350 con 0x7f840c0701a0 2023-09-30T22:35:01.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.197+0000 7f8411a40700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f840c071ff0 con 0x7f840c130670 2023-09-30T22:35:01.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.197+0000 7f840a7fc700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f840c06f680 0x7f840c06fa90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50804/0 (socket says 172.21.15.16:50804) 2023-09-30T22:35:01.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.197+0000 7f840a7fc700 1 -- 172.21.15.16:0/1135172017 learned_addr learned my addr 172.21.15.16:0/1135172017 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:01.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.197+0000 7f840b7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f840c0701a0 0x7f840c138540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:01.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.197+0000 7f840affd700 1 --2- 172.21.15.16:0/1135172017 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f840c130670 0x7f840c130a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:01.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.197+0000 7f840affd700 1 -- 172.21.15.16:0/1135172017 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f840c0701a0 msgr2=0x7f840c138540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:01.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.197+0000 7f840affd700 1 --2- 172.21.15.16:0/1135172017 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f840c0701a0 0x7f840c138540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:01.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.197+0000 7f840affd700 1 -- 172.21.15.16:0/1135172017 >> v1:172.21.15.16:6789/0 conn(0x7f840c06f680 legacy=0x7f840c06fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:01.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.197+0000 7f840affd700 1 -- 172.21.15.16:0/1135172017 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f840c09ecf0 con 0x7f840c130670 2023-09-30T22:35:01.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.197+0000 7f8131ffb700 1 -- 172.21.15.16:0/933964667 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8134017070 con 0x7f813c134260 2023-09-30T22:35:01.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.197+0000 7f8409ffb700 1 -- 172.21.15.16:0/1135172017 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2204122087 0 0) 0x7f840c130260 con 0x7f840c06f680 2023-09-30T22:35:01.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.197+0000 7f840b7fe700 1 --2- 172.21.15.16:0/1135172017 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f840c0701a0 0x7f840c138540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:01.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.198+0000 7f840affd700 1 --2- 172.21.15.16:0/1135172017 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f840c130670 0x7f840c130a40 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f83fc0098a0 tx=0x7f83fc009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91865ca35240d294 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:01.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.198+0000 7f8409ffb700 1 -- 172.21.15.16:0/1135172017 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83fc011830 con 0x7f840c130670 2023-09-30T22:35:01.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.198+0000 7f8409ffb700 1 -- 172.21.15.16:0/1135172017 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83fc011990 con 0x7f840c130670 2023-09-30T22:35:01.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.198+0000 7f8409ffb700 1 -- 172.21.15.16:0/1135172017 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83fc011d20 con 0x7f840c130670 2023-09-30T22:35:01.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.199+0000 7f8411a40700 1 -- 172.21.15.16:0/1135172017 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f840c130670 msgr2=0x7f840c130a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:01.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.199+0000 7f8411a40700 1 --2- 172.21.15.16:0/1135172017 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f840c130670 0x7f840c130a40 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f83fc0098a0 tx=0x7f83fc009bb0 comp rx=0 tx=0).stop 2023-09-30T22:35:01.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.199+0000 7f8411a40700 1 -- 172.21.15.16:0/1135172017 shutdown_connections 2023-09-30T22:35:01.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.199+0000 7f8411a40700 1 --2- 172.21.15.16:0/1135172017 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f840c0701a0 0x7f840c138540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:01.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.199+0000 7f8411a40700 1 --2- 172.21.15.16:0/1135172017 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f840c130670 0x7f840c130a40 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:01.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.199+0000 7f8411a40700 1 -- 172.21.15.16:0/1135172017 >> 172.21.15.16:0/1135172017 conn(0x7f840c0a2aa0 msgr2=0x7f840c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:01.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.199+0000 7f8411a40700 1 -- 172.21.15.16:0/1135172017 shutdown_connections 2023-09-30T22:35:01.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.199+0000 7f8411a40700 1 -- 172.21.15.16:0/1135172017 wait complete. 2023-09-30T22:35:01.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.200+0000 7f8411a40700 1 Processor -- start 2023-09-30T22:35:01.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.201+0000 7f8411a40700 1 -- start start 2023-09-30T22:35:01.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.201+0000 7f8411a40700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f840c06f680 0x7f840c09cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:01.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.201+0000 7f8411a40700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f840c0701a0 0x7f840c09d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:01.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.201+0000 7f8411a40700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f840c130670 0x7f840c09d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:01.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.201+0000 7f8411a40700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840c11f440 con 0x7f840c0701a0 2023-09-30T22:35:01.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.201+0000 7f8411a40700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f840c12e950 con 0x7f840c06f680 2023-09-30T22:35:01.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.201+0000 7f8411a40700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f840c071820 con 0x7f840c130670 2023-09-30T22:35:01.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.202+0000 7f840affd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f840c06f680 0x7f840c09cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.202+0000 7f840a7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f840c0701a0 0x7f840c09d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.202+0000 7f840affd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f840c06f680 0x7f840c09cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:52254/0 (socket says 172.21.15.16:52254) 2023-09-30T22:35:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.202+0000 7f840affd700 1 -- 172.21.15.16:0/2766144856 learned_addr learned my addr 172.21.15.16:0/2766144856 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.202+0000 7f840b7fe700 1 --2- 172.21.15.16:0/2766144856 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f840c130670 0x7f840c09d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.202+0000 7f840a7fc700 1 -- 172.21.15.16:0/2766144856 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f840c130670 msgr2=0x7f840c09d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.202+0000 7f840a7fc700 1 --2- 172.21.15.16:0/2766144856 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f840c130670 0x7f840c09d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.202+0000 7f840a7fc700 1 -- 172.21.15.16:0/2766144856 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f840c06f680 msgr2=0x7f840c09cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.202+0000 7f840a7fc700 1 --2- 172.21.15.16:0/2766144856 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f840c06f680 0x7f840c09cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.202+0000 7f840a7fc700 1 -- 172.21.15.16:0/2766144856 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8404009a70 con 0x7f840c0701a0 2023-09-30T22:35:01.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.202+0000 7f840a7fc700 1 --2- 172.21.15.16:0/2766144856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f840c0701a0 0x7f840c09d480 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f840c066940 tx=0x7f840000c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:01.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.202+0000 7f83f3fff700 1 -- 172.21.15.16:0/2766144856 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8400004c80 con 0x7f840c0701a0 2023-09-30T22:35:01.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.203+0000 7f83f3fff700 1 -- 172.21.15.16:0/2766144856 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8400004de0 con 0x7f840c0701a0 2023-09-30T22:35:01.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.203+0000 7f8411a40700 1 -- 172.21.15.16:0/2766144856 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83fc0094a0 con 0x7f840c0701a0 2023-09-30T22:35:01.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.203+0000 7f83f3fff700 1 -- 172.21.15.16:0/2766144856 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8400007310 con 0x7f840c0701a0 2023-09-30T22:35:01.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.203+0000 7f8411a40700 1 -- 172.21.15.16:0/2766144856 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f840c11e7d0 con 0x7f840c0701a0 2023-09-30T22:35:01.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.203+0000 7f83f3fff700 1 -- 172.21.15.16:0/2766144856 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8400007470 con 0x7f840c0701a0 2023-09-30T22:35:01.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.204+0000 7f83f3fff700 1 -- 172.21.15.16:0/2766144856 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f840003f220 con 0x7f840c0701a0 2023-09-30T22:35:01.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.204+0000 7f8411a40700 1 -- 172.21.15.16:0/2766144856 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f840c074ea0 con 0x7f840c0701a0 2023-09-30T22:35:01.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.209+0000 7f83f3fff700 1 -- 172.21.15.16:0/2766144856 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f840000ce40 con 0x7f840c0701a0 2023-09-30T22:35:01.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.245+0000 7f8032ff5700 1 Processor -- start 2023-09-30T22:35:01.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.245+0000 7f8032ff5700 1 -- start start 2023-09-30T22:35:01.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.246+0000 7f8032ff5700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f802c0a0a30 0x7f802c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:01.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.246+0000 7f8032ff5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f802c09d630 0x7f802c0a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:01.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.246+0000 7f8032ff5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f802c133930 0x7f802c133db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:01.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.246+0000 7f8032ff5700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f802c0a2a70 con 0x7f802c0a0a30 2023-09-30T22:35:01.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.246+0000 7f8032ff5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f802c0a01d0 con 0x7f802c133930 2023-09-30T22:35:01.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.246+0000 7f8032ff5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f802c09e2b0 con 0x7f802c09d630 2023-09-30T22:35:01.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.246+0000 7f8030d91700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f802c0a0a30 0x7f802c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:01.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.246+0000 7f802bfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f802c09d630 0x7f802c0a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:01.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.246+0000 7f8030d91700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f802c0a0a30 0x7f802c0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43920/0 (socket says 172.21.15.16:43920) 2023-09-30T22:35:01.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.246+0000 7f8030d91700 1 -- 172.21.15.16:0/442965118 learned_addr learned my addr 172.21.15.16:0/442965118 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:01.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.246+0000 7f8031592700 1 --2- 172.21.15.16:0/442965118 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f802c133930 0x7f802c133db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:01.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.246+0000 7f802bfff700 1 -- 172.21.15.16:0/442965118 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f802c133930 msgr2=0x7f802c133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:01.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.246+0000 7f802bfff700 1 --2- 172.21.15.16:0/442965118 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f802c133930 0x7f802c133db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:01.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.246+0000 7f802bfff700 1 -- 172.21.15.16:0/442965118 >> v2:172.21.15.16:3300/0 conn(0x7f802c0a0a30 msgr2=0x7f802c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:01.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.246+0000 7f802bfff700 1 --2- 172.21.15.16:0/442965118 >> v2:172.21.15.16:3300/0 conn(0x7f802c0a0a30 0x7f802c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:01.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.246+0000 7f802bfff700 1 -- 172.21.15.16:0/442965118 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f802c09dc50 con 0x7f802c09d630 2023-09-30T22:35:01.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.247+0000 7f8030d91700 1 --2- 172.21.15.16:0/442965118 >> v2:172.21.15.16:3300/0 conn(0x7f802c0a0a30 0x7f802c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:01.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.247+0000 7f8031592700 1 --2- 172.21.15.16:0/442965118 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f802c133930 0x7f802c133db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:35:01.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.247+0000 7f802bfff700 1 --2- 172.21.15.16:0/442965118 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f802c09d630 0x7f802c0a1340 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f8020009840 tx=0x7f8020009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3323f98c315165a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:01.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.247+0000 7f802b7fe700 1 -- 172.21.15.16:0/442965118 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8020011830 con 0x7f802c09d630 2023-09-30T22:35:01.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.247+0000 7f802b7fe700 1 -- 172.21.15.16:0/442965118 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8020011990 con 0x7f802c09d630 2023-09-30T22:35:01.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.247+0000 7f802b7fe700 1 -- 172.21.15.16:0/442965118 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8020011d20 con 0x7f802c09d630 2023-09-30T22:35:01.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.247+0000 7f8032ff5700 1 -- 172.21.15.16:0/442965118 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f802c09d630 msgr2=0x7f802c0a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:01.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.247+0000 7f8032ff5700 1 --2- 172.21.15.16:0/442965118 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f802c09d630 0x7f802c0a1340 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f8020009840 tx=0x7f8020009b50 comp rx=0 tx=0).stop 2023-09-30T22:35:01.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.248+0000 7f8032ff5700 1 -- 172.21.15.16:0/442965118 shutdown_connections 2023-09-30T22:35:01.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.248+0000 7f8032ff5700 1 --2- 172.21.15.16:0/442965118 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f802c133930 0x7f802c133db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:01.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.248+0000 7f8032ff5700 1 --2- 172.21.15.16:0/442965118 >> v2:172.21.15.16:3300/0 conn(0x7f802c0a0a30 0x7f802c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:01.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.248+0000 7f8032ff5700 1 --2- 172.21.15.16:0/442965118 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f802c09d630 0x7f802c0a1340 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:01.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.248+0000 7f8032ff5700 1 -- 172.21.15.16:0/442965118 >> 172.21.15.16:0/442965118 conn(0x7f802c0a3050 msgr2=0x7f802c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:01.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.248+0000 7f8032ff5700 1 -- 172.21.15.16:0/442965118 shutdown_connections 2023-09-30T22:35:01.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.248+0000 7f8032ff5700 1 -- 172.21.15.16:0/442965118 wait complete. 2023-09-30T22:35:01.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.249+0000 7f8032ff5700 1 Processor -- start 2023-09-30T22:35:01.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.249+0000 7f8032ff5700 1 -- start start 2023-09-30T22:35:01.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.249+0000 7f8032ff5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f802c09d630 0x7f802c11e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:01.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.249+0000 7f8032ff5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f802c0a0a30 0x7f802c11ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:01.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.249+0000 7f8032ff5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f802c133930 0x7f802c11f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:01.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.249+0000 7f8032ff5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f802c11fde0 con 0x7f802c133930 2023-09-30T22:35:01.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.249+0000 7f8032ff5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f802c120490 con 0x7f802c0a0a30 2023-09-30T22:35:01.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.249+0000 7f8032ff5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f802c1223a0 con 0x7f802c09d630 2023-09-30T22:35:01.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.249+0000 7f8030d91700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f802c09d630 0x7f802c11e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:01.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.249+0000 7f8030d91700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f802c09d630 0x7f802c11e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:34026/0 (socket says 172.21.15.16:34026) 2023-09-30T22:35:01.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.250+0000 7f8030d91700 1 -- 172.21.15.16:0/1996138252 learned_addr learned my addr 172.21.15.16:0/1996138252 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:01.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.250+0000 7f802bfff700 1 --2- 172.21.15.16:0/1996138252 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f802c0a0a30 0x7f802c11ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:01.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.250+0000 7f8031592700 1 --2- 172.21.15.16:0/1996138252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f802c133930 0x7f802c11f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:01.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.250+0000 7f8030d91700 1 -- 172.21.15.16:0/1996138252 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f802c0a0a30 msgr2=0x7f802c11ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:01.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.250+0000 7f8030d91700 1 --2- 172.21.15.16:0/1996138252 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f802c0a0a30 0x7f802c11ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:01.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.250+0000 7f8030d91700 1 -- 172.21.15.16:0/1996138252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f802c133930 msgr2=0x7f802c11f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:01.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.250+0000 7f8030d91700 1 --2- 172.21.15.16:0/1996138252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f802c133930 0x7f802c11f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:01.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.250+0000 7f8030d91700 1 -- 172.21.15.16:0/1996138252 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f801c009550 con 0x7f802c09d630 2023-09-30T22:35:01.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.250+0000 7f8030d91700 1 --2- 172.21.15.16:0/1996138252 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f802c09d630 0x7f802c11e910 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f801c00ebe0 tx=0x7f801c00eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:01.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.250+0000 7f80297fa700 1 -- 172.21.15.16:0/1996138252 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801c00c850 con 0x7f802c09d630 2023-09-30T22:35:01.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.250+0000 7f80297fa700 1 -- 172.21.15.16:0/1996138252 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f801c00c9b0 con 0x7f802c09d630 2023-09-30T22:35:01.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.250+0000 7f80297fa700 1 -- 172.21.15.16:0/1996138252 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801c00cd40 con 0x7f802c09d630 2023-09-30T22:35:01.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.250+0000 7f8032ff5700 1 -- 172.21.15.16:0/1996138252 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80200094a0 con 0x7f802c09d630 2023-09-30T22:35:01.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.250+0000 7f8032ff5700 1 -- 172.21.15.16:0/1996138252 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f802c1220c0 con 0x7f802c09d630 2023-09-30T22:35:01.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.251+0000 7f80297fa700 1 -- 172.21.15.16:0/1996138252 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f801c013470 con 0x7f802c09d630 2023-09-30T22:35:01.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.251+0000 7f80297fa700 1 -- 172.21.15.16:0/1996138252 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f801c03fc20 con 0x7f802c09d630 2023-09-30T22:35:01.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.252+0000 7f8032ff5700 1 -- 172.21.15.16:0/1996138252 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f802c0747f0 con 0x7f802c09d630 2023-09-30T22:35:01.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.257+0000 7f80297fa700 1 -- 172.21.15.16:0/1996138252 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f801c01b3b0 con 0x7f802c09d630 2023-09-30T22:35:01.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.408+0000 7f814353e700 1 -- 172.21.15.16:0/933964667 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f813c11fe20 con 0x7f813c134260 2023-09-30T22:35:01.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.409+0000 7f8131ffb700 1 -- 172.21.15.16:0/933964667 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8134010710 con 0x7f813c134260 2023-09-30T22:35:01.409 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:35:01.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.411+0000 7f814353e700 1 -- 172.21.15.16:0/933964667 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f813c134260 msgr2=0x7f813c09cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:01.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.411+0000 7f814353e700 1 --2- 172.21.15.16:0/933964667 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f813c134260 0x7f813c09cc80 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f813400cb40 tx=0x7f813400ce50 comp rx=0 tx=0).stop 2023-09-30T22:35:01.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.411+0000 7f814353e700 1 -- 172.21.15.16:0/933964667 shutdown_connections 2023-09-30T22:35:01.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.411+0000 7f814353e700 1 --2- 172.21.15.16:0/933964667 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f813c134260 0x7f813c09cc80 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:01.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.411+0000 7f814353e700 1 --2- 172.21.15.16:0/933964667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f813c133420 0x7f813c09c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:01.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.411+0000 7f814353e700 1 --2- 172.21.15.16:0/933964667 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f813c133e30 0x7f813c09c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:01.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.411+0000 7f814353e700 1 -- 172.21.15.16:0/933964667 >> 172.21.15.16:0/933964667 conn(0x7f813c0a4210 msgr2=0x7f813c12f690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:01.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.411+0000 7f814353e700 1 -- 172.21.15.16:0/933964667 shutdown_connections 2023-09-30T22:35:01.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.411+0000 7f814353e700 1 -- 172.21.15.16:0/933964667 wait complete. 2023-09-30T22:35:01.425 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:35:01.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.446+0000 7f8411a40700 1 -- 172.21.15.16:0/2766144856 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f840c072210 con 0x7f840c0701a0 2023-09-30T22:35:01.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.446+0000 7f83f3fff700 1 -- 172.21.15.16:0/2766144856 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f840001b340 con 0x7f840c0701a0 2023-09-30T22:35:01.446 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:35:01.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.448+0000 7f8411a40700 1 -- 172.21.15.16:0/2766144856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f840c0701a0 msgr2=0x7f840c09d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:01.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.448+0000 7f8411a40700 1 --2- 172.21.15.16:0/2766144856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f840c0701a0 0x7f840c09d480 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f840c066940 tx=0x7f840000c750 comp rx=0 tx=0).stop 2023-09-30T22:35:01.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.448+0000 7f8411a40700 1 -- 172.21.15.16:0/2766144856 shutdown_connections 2023-09-30T22:35:01.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.448+0000 7f8411a40700 1 --2- 172.21.15.16:0/2766144856 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f840c06f680 0x7f840c09cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:01.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.448+0000 7f8411a40700 1 --2- 172.21.15.16:0/2766144856 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f840c130670 0x7f840c09d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:01.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.448+0000 7f8411a40700 1 --2- 172.21.15.16:0/2766144856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f840c0701a0 0x7f840c09d480 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:01.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.448+0000 7f8411a40700 1 -- 172.21.15.16:0/2766144856 >> 172.21.15.16:0/2766144856 conn(0x7f840c0a2aa0 msgr2=0x7f840c12d700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:01.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.448+0000 7f8411a40700 1 -- 172.21.15.16:0/2766144856 shutdown_connections 2023-09-30T22:35:01.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.448+0000 7f8411a40700 1 -- 172.21.15.16:0/2766144856 wait complete. 2023-09-30T22:35:01.462 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:35:01.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.477+0000 7f8032ff5700 1 -- 172.21.15.16:0/1996138252 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f802c120ca0 con 0x7f802c09d630 2023-09-30T22:35:01.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.478+0000 7f80297fa700 1 -- 172.21.15.16:0/1996138252 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f801c024350 con 0x7f802c09d630 2023-09-30T22:35:01.478 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:35:01.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.480+0000 7f8032ff5700 1 -- 172.21.15.16:0/1996138252 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f802c09d630 msgr2=0x7f802c11e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:01.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.480+0000 7f8032ff5700 1 --2- 172.21.15.16:0/1996138252 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f802c09d630 0x7f802c11e910 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f801c00ebe0 tx=0x7f801c00eef0 comp rx=0 tx=0).stop 2023-09-30T22:35:01.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.480+0000 7f8032ff5700 1 -- 172.21.15.16:0/1996138252 shutdown_connections 2023-09-30T22:35:01.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.480+0000 7f8032ff5700 1 --2- 172.21.15.16:0/1996138252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f802c133930 0x7f802c11f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:01.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.480+0000 7f8032ff5700 1 --2- 172.21.15.16:0/1996138252 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f802c09d630 0x7f802c11e910 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:01.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.480+0000 7f8032ff5700 1 --2- 172.21.15.16:0/1996138252 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f802c0a0a30 0x7f802c11ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:01.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.480+0000 7f8032ff5700 1 -- 172.21.15.16:0/1996138252 >> 172.21.15.16:0/1996138252 conn(0x7f802c0a3050 msgr2=0x7f802c1bbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:01.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.480+0000 7f8032ff5700 1 -- 172.21.15.16:0/1996138252 shutdown_connections 2023-09-30T22:35:01.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:01.480+0000 7f8032ff5700 1 -- 172.21.15.16:0/1996138252 wait complete. 2023-09-30T22:35:01.494 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:35:01.906 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:35:02.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.052+0000 7f150e303700 1 Processor -- start 2023-09-30T22:35:02.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.052+0000 7f150e303700 1 -- start start 2023-09-30T22:35:02.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.052+0000 7f150e303700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1508133420 0x7f15081337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:02.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.053+0000 7f150e303700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1508134260 0x7f15081306a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:02.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.053+0000 7f150e303700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f150806b990 con 0x7f1508133e30 2023-09-30T22:35:02.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.053+0000 7f150e303700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15080a0590 con 0x7f1508134260 2023-09-30T22:35:02.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.053+0000 7f150e303700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f150809fb80 con 0x7f1508133420 2023-09-30T22:35:02.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.053+0000 7f150c8a0700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1508134260 0x7f15081306a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:02.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.053+0000 7f15077fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1508133e30 0x7f150812ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50818/0 (socket says 172.21.15.16:50818) 2023-09-30T22:35:02.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.053+0000 7f1507fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1508133420 0x7f15081337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:02.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.053+0000 7f15077fe700 1 -- 172.21.15.16:0/1787454543 learned_addr learned my addr 172.21.15.16:0/1787454543 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:02.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.053+0000 7f150c8a0700 1 -- 172.21.15.16:0/1787454543 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1508133420 msgr2=0x7f15081337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:02.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.053+0000 7f150c8a0700 1 --2- 172.21.15.16:0/1787454543 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1508133420 0x7f15081337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:02.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.053+0000 7f150c8a0700 1 -- 172.21.15.16:0/1787454543 >> v1:172.21.15.16:6789/0 conn(0x7f1508133e30 legacy=0x7f150812ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:02.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.053+0000 7f150c8a0700 1 -- 172.21.15.16:0/1787454543 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15080a0030 con 0x7f1508134260 2023-09-30T22:35:02.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.053+0000 7f1506ffd700 1 -- 172.21.15.16:0/1787454543 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2107173232 0 0) 0x7f150806b990 con 0x7f1508133e30 2023-09-30T22:35:02.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.053+0000 7f150c8a0700 1 --2- 172.21.15.16:0/1787454543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1508134260 0x7f15081306a0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f1500009d90 tx=0x7f150000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e634d373937cf828 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:02.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.053+0000 7f1506ffd700 1 -- 172.21.15.16:0/1787454543 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f150000e070 con 0x7f1508134260 2023-09-30T22:35:02.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.053+0000 7f1506ffd700 1 -- 172.21.15.16:0/1787454543 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f150000bb20 con 0x7f1508134260 2023-09-30T22:35:02.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.054+0000 7f1506ffd700 1 -- 172.21.15.16:0/1787454543 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1500012430 con 0x7f1508134260 2023-09-30T22:35:02.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.054+0000 7f150e303700 1 -- 172.21.15.16:0/1787454543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1508134260 msgr2=0x7f15081306a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:02.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.054+0000 7f150e303700 1 --2- 172.21.15.16:0/1787454543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1508134260 0x7f15081306a0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f1500009d90 tx=0x7f150000b330 comp rx=0 tx=0).stop 2023-09-30T22:35:02.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.054+0000 7f150e303700 1 -- 172.21.15.16:0/1787454543 shutdown_connections 2023-09-30T22:35:02.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.054+0000 7f150e303700 1 --2- 172.21.15.16:0/1787454543 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1508134260 0x7f15081306a0 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:02.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.054+0000 7f150e303700 1 --2- 172.21.15.16:0/1787454543 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1508133420 0x7f15081337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:02.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.054+0000 7f150e303700 1 -- 172.21.15.16:0/1787454543 >> 172.21.15.16:0/1787454543 conn(0x7f15080a4210 msgr2=0x7f15080a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:02.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.054+0000 7f150e303700 1 -- 172.21.15.16:0/1787454543 shutdown_connections 2023-09-30T22:35:02.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.054+0000 7f150e303700 1 -- 172.21.15.16:0/1787454543 wait complete. 2023-09-30T22:35:02.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.055+0000 7f150e303700 1 Processor -- start 2023-09-30T22:35:02.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.056+0000 7f150e303700 1 -- start start 2023-09-30T22:35:02.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.056+0000 7f150e303700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1508133420 0x7f150809c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:02.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.056+0000 7f150e303700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1508133e30 0x7f150809c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:02.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.056+0000 7f150e303700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1508134260 0x7f150809cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:02.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.056+0000 7f150e303700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f150811fad0 con 0x7f1508134260 2023-09-30T22:35:02.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.056+0000 7f150e303700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f150811f080 con 0x7f1508133e30 2023-09-30T22:35:02.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.056+0000 7f150e303700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f150809deb0 con 0x7f1508133420 2023-09-30T22:35:02.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.057+0000 7f1507fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1508133420 0x7f150809c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:02.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.057+0000 7f1507fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1508133420 0x7f150809c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:34044/0 (socket says 172.21.15.16:34044) 2023-09-30T22:35:02.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.057+0000 7f1507fff700 1 -- 172.21.15.16:0/2302194762 learned_addr learned my addr 172.21.15.16:0/2302194762 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:02.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.057+0000 7f150c8a0700 1 --2- 172.21.15.16:0/2302194762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1508134260 0x7f150809cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:02.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.057+0000 7f15077fe700 1 --2- 172.21.15.16:0/2302194762 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1508133e30 0x7f150809c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:02.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.057+0000 7f1507fff700 1 -- 172.21.15.16:0/2302194762 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1508133e30 msgr2=0x7f150809c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:02.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.057+0000 7f1507fff700 1 --2- 172.21.15.16:0/2302194762 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1508133e30 0x7f150809c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:02.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.057+0000 7f1507fff700 1 -- 172.21.15.16:0/2302194762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1508134260 msgr2=0x7f150809cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:02.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.057+0000 7f1507fff700 1 --2- 172.21.15.16:0/2302194762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1508134260 0x7f150809cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:02.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.057+0000 7f1507fff700 1 -- 172.21.15.16:0/2302194762 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1500009a70 con 0x7f1508133420 2023-09-30T22:35:02.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.057+0000 7f1507fff700 1 --2- 172.21.15.16:0/2302194762 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1508133420 0x7f150809c480 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f14f800e9b0 tx=0x7f14f800ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:02.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.057+0000 7f1504ff9700 1 -- 172.21.15.16:0/2302194762 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f8009710 con 0x7f1508133420 2023-09-30T22:35:02.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.057+0000 7f1504ff9700 1 -- 172.21.15.16:0/2302194762 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14f800ee50 con 0x7f1508133420 2023-09-30T22:35:02.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.057+0000 7f150e303700 1 -- 172.21.15.16:0/2302194762 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f150811edd0 con 0x7f1508133420 2023-09-30T22:35:02.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.057+0000 7f1504ff9700 1 -- 172.21.15.16:0/2302194762 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f800ca50 con 0x7f1508133420 2023-09-30T22:35:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.057+0000 7f150e303700 1 -- 172.21.15.16:0/2302194762 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1508120570 con 0x7f1508133420 2023-09-30T22:35:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.058+0000 7f1504ff9700 1 -- 172.21.15.16:0/2302194762 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f14f800cbb0 con 0x7f1508133420 2023-09-30T22:35:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.058+0000 7f1504ff9700 1 -- 172.21.15.16:0/2302194762 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f14f803fac0 con 0x7f1508133420 2023-09-30T22:35:02.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.058+0000 7f150e303700 1 -- 172.21.15.16:0/2302194762 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1508074ea0 con 0x7f1508133420 2023-09-30T22:35:02.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.063+0000 7f1504ff9700 1 -- 172.21.15.16:0/2302194762 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f14f8017080 con 0x7f1508133420 2023-09-30T22:35:02.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.267+0000 7f150e303700 1 -- 172.21.15.16:0/2302194762 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f150806b250 con 0x7f1508133420 2023-09-30T22:35:02.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.268+0000 7f1504ff9700 1 -- 172.21.15.16:0/2302194762 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f14f8021020 con 0x7f1508133420 2023-09-30T22:35:02.268 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:35:02.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.270+0000 7f150e303700 1 -- 172.21.15.16:0/2302194762 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1508133420 msgr2=0x7f150809c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:02.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.270+0000 7f150e303700 1 --2- 172.21.15.16:0/2302194762 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1508133420 0x7f150809c480 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f14f800e9b0 tx=0x7f14f800ecc0 comp rx=0 tx=0).stop 2023-09-30T22:35:02.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.270+0000 7f150e303700 1 -- 172.21.15.16:0/2302194762 shutdown_connections 2023-09-30T22:35:02.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.270+0000 7f150e303700 1 --2- 172.21.15.16:0/2302194762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1508134260 0x7f150809cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:02.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.270+0000 7f150e303700 1 --2- 172.21.15.16:0/2302194762 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1508133420 0x7f150809c480 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:02.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.270+0000 7f150e303700 1 --2- 172.21.15.16:0/2302194762 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1508133e30 0x7f150809c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:02.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.270+0000 7f150e303700 1 -- 172.21.15.16:0/2302194762 >> 172.21.15.16:0/2302194762 conn(0x7f15080a4210 msgr2=0x7f15080a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:02.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.270+0000 7f150e303700 1 -- 172.21.15.16:0/2302194762 shutdown_connections 2023-09-30T22:35:02.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.270+0000 7f150e303700 1 -- 172.21.15.16:0/2302194762 wait complete. 2023-09-30T22:35:02.284 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:35:02.426 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:35:02.463 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:35:02.495 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:35:02.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.581+0000 7fe94c5c3700 1 Processor -- start 2023-09-30T22:35:02.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.581+0000 7fe94c5c3700 1 -- start start 2023-09-30T22:35:02.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.582+0000 7fe94c5c3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe9441337f0 0x7fe944138880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:02.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.582+0000 7fe94c5c3700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe944138dc0 0x7fe94413b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:02.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.582+0000 7fe94c5c3700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe94406b990 con 0x7fe944133420 2023-09-30T22:35:02.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.582+0000 7fe94c5c3700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe94409b0e0 con 0x7fe944138dc0 2023-09-30T22:35:02.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.582+0000 7fe94c5c3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe944133c20 con 0x7fe9441337f0 2023-09-30T22:35:02.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.582+0000 7fe949b5e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe9441337f0 0x7fe944138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:02.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.582+0000 7fe94a35f700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe944133420 0x7fe944133e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50828/0 (socket says 172.21.15.16:50828) 2023-09-30T22:35:02.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.582+0000 7fe94a35f700 1 -- 172.21.15.16:0/157666050 learned_addr learned my addr 172.21.15.16:0/157666050 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:02.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.582+0000 7fe94ab60700 1 --2- 172.21.15.16:0/157666050 >> v2:172.21.15.16:3300/0 conn(0x7fe944138dc0 0x7fe94413b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:02.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.582+0000 7fe949b5e700 1 -- 172.21.15.16:0/157666050 >> v2:172.21.15.16:3300/0 conn(0x7fe944138dc0 msgr2=0x7fe94413b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:02.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.582+0000 7fe949b5e700 1 --2- 172.21.15.16:0/157666050 >> v2:172.21.15.16:3300/0 conn(0x7fe944138dc0 0x7fe94413b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:02.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.582+0000 7fe949b5e700 1 -- 172.21.15.16:0/157666050 >> v1:172.21.15.16:6789/0 conn(0x7fe944133420 legacy=0x7fe944133e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:02.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.582+0000 7fe949b5e700 1 -- 172.21.15.16:0/157666050 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9440a0590 con 0x7fe9441337f0 2023-09-30T22:35:02.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.582+0000 7fe94ab60700 1 --2- 172.21.15.16:0/157666050 >> v2:172.21.15.16:3300/0 conn(0x7fe944138dc0 0x7fe94413b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:02.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.582+0000 7fe94935d700 1 -- 172.21.15.16:0/157666050 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2755303256 0 0) 0x7fe94406b990 con 0x7fe944133420 2023-09-30T22:35:02.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.583+0000 7fe949b5e700 1 --2- 172.21.15.16:0/157666050 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe9441337f0 0x7fe944138880 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fe934009840 tx=0x7fe934009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0429729c69d9641 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:02.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.583+0000 7fe94935d700 1 -- 172.21.15.16:0/157666050 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe934011830 con 0x7fe9441337f0 2023-09-30T22:35:02.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.583+0000 7fe94935d700 1 -- 172.21.15.16:0/157666050 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe934011990 con 0x7fe9441337f0 2023-09-30T22:35:02.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.583+0000 7fe94935d700 1 -- 172.21.15.16:0/157666050 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe934011d20 con 0x7fe9441337f0 2023-09-30T22:35:02.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.584+0000 7fe94c5c3700 1 -- 172.21.15.16:0/157666050 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe9441337f0 msgr2=0x7fe944138880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:02.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.584+0000 7fe94c5c3700 1 --2- 172.21.15.16:0/157666050 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe9441337f0 0x7fe944138880 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fe934009840 tx=0x7fe934009b50 comp rx=0 tx=0).stop 2023-09-30T22:35:02.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.584+0000 7fe94c5c3700 1 -- 172.21.15.16:0/157666050 shutdown_connections 2023-09-30T22:35:02.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.584+0000 7fe94c5c3700 1 --2- 172.21.15.16:0/157666050 >> v2:172.21.15.16:3300/0 conn(0x7fe944138dc0 0x7fe94413b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:02.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.584+0000 7fe94c5c3700 1 --2- 172.21.15.16:0/157666050 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe9441337f0 0x7fe944138880 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:02.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.584+0000 7fe94c5c3700 1 -- 172.21.15.16:0/157666050 >> 172.21.15.16:0/157666050 conn(0x7fe9440a4210 msgr2=0x7fe9440a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:02.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.584+0000 7fe94c5c3700 1 -- 172.21.15.16:0/157666050 shutdown_connections 2023-09-30T22:35:02.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.584+0000 7fe94c5c3700 1 -- 172.21.15.16:0/157666050 wait complete. 2023-09-30T22:35:02.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.585+0000 7fe94c5c3700 1 Processor -- start 2023-09-30T22:35:02.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.585+0000 7fe94c5c3700 1 -- start start 2023-09-30T22:35:02.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.585+0000 7fe94c5c3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe944133420 0x7fe94412ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:02.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.585+0000 7fe94c5c3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9441337f0 0x7fe94412f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:02.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.585+0000 7fe94c5c3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe944138dc0 0x7fe94412f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:02.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.585+0000 7fe94c5c3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe94409da30 con 0x7fe9441337f0 2023-09-30T22:35:02.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.585+0000 7fe94c5c3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe94409b430 con 0x7fe944133420 2023-09-30T22:35:02.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.585+0000 7fe94c5c3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe94409c020 con 0x7fe944138dc0 2023-09-30T22:35:02.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.585+0000 7fe94a35f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe944133420 0x7fe94412ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:02.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.585+0000 7fe94a35f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe944133420 0x7fe94412ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:52288/0 (socket says 172.21.15.16:52288) 2023-09-30T22:35:02.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.585+0000 7fe94a35f700 1 -- 172.21.15.16:0/3766391896 learned_addr learned my addr 172.21.15.16:0/3766391896 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:02.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.586+0000 7fe949b5e700 1 --2- 172.21.15.16:0/3766391896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9441337f0 0x7fe94412f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:02.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.586+0000 7fe94ab60700 1 --2- 172.21.15.16:0/3766391896 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe944138dc0 0x7fe94412f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:02.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.586+0000 7fe949b5e700 1 -- 172.21.15.16:0/3766391896 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe944138dc0 msgr2=0x7fe94412f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:02.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.586+0000 7fe949b5e700 1 --2- 172.21.15.16:0/3766391896 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe944138dc0 0x7fe94412f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:02.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.586+0000 7fe949b5e700 1 -- 172.21.15.16:0/3766391896 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe944133420 msgr2=0x7fe94412ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:02.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.586+0000 7fe949b5e700 1 --2- 172.21.15.16:0/3766391896 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe944133420 0x7fe94412ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:02.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.586+0000 7fe949b5e700 1 -- 172.21.15.16:0/3766391896 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe93c009a70 con 0x7fe9441337f0 2023-09-30T22:35:02.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.586+0000 7fe94a35f700 1 --2- 172.21.15.16:0/3766391896 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe944133420 0x7fe94412ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:02.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.586+0000 7fe94ab60700 1 --2- 172.21.15.16:0/3766391896 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe944138dc0 0x7fe94412f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:02.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.586+0000 7fe949b5e700 1 --2- 172.21.15.16:0/3766391896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9441337f0 0x7fe94412f390 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fe934016000 tx=0x7fe934004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:02.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.586+0000 7fe93affd700 1 -- 172.21.15.16:0/3766391896 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9340045d0 con 0x7fe9441337f0 2023-09-30T22:35:02.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.586+0000 7fe93affd700 1 -- 172.21.15.16:0/3766391896 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe93401f070 con 0x7fe9441337f0 2023-09-30T22:35:02.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.586+0000 7fe93affd700 1 -- 172.21.15.16:0/3766391896 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9340038e0 con 0x7fe9441337f0 2023-09-30T22:35:02.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.586+0000 7fe94c5c3700 1 -- 172.21.15.16:0/3766391896 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9340094a0 con 0x7fe9441337f0 2023-09-30T22:35:02.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.587+0000 7fe94c5c3700 1 -- 172.21.15.16:0/3766391896 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe94409c7d0 con 0x7fe9441337f0 2023-09-30T22:35:02.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.587+0000 7fe93affd700 1 -- 172.21.15.16:0/3766391896 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe934003a40 con 0x7fe9441337f0 2023-09-30T22:35:02.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.587+0000 7fe93affd700 1 -- 172.21.15.16:0/3766391896 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fe934021070 con 0x7fe9441337f0 2023-09-30T22:35:02.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.588+0000 7fe94c5c3700 1 -- 172.21.15.16:0/3766391896 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe94409e4e0 con 0x7fe9441337f0 2023-09-30T22:35:02.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.597+0000 7fe93affd700 1 -- 172.21.15.16:0/3766391896 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe93402b070 con 0x7fe9441337f0 2023-09-30T22:35:02.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.624+0000 7fd642f0e700 1 Processor -- start 2023-09-30T22:35:02.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.624+0000 7fd642f0e700 1 -- start start 2023-09-30T22:35:02.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.624+0000 7fd642f0e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd63c133420 0x7fd63c1337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:02.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.624+0000 7fd642f0e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd63c133e30 0x7fd63c12ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:02.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.624+0000 7fd642f0e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd63c134260 0x7fd63c1304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:02.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.624+0000 7fd642f0e700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd63c06b990 con 0x7fd63c133420 2023-09-30T22:35:02.626 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.624+0000 7fd642f0e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c0a0590 con 0x7fd63c134260 2023-09-30T22:35:02.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.624+0000 7fd642f0e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c09fb80 con 0x7fd63c133e30 2023-09-30T22:35:02.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.625+0000 7fd640caa700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd63c133420 0x7fd63c1337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:02.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.625+0000 7fd640caa700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd63c133420 0x7fd63c1337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43964/0 (socket says 172.21.15.16:43964) 2023-09-30T22:35:02.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.625+0000 7fd640caa700 1 -- 172.21.15.16:0/955535405 learned_addr learned my addr 172.21.15.16:0/955535405 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:02.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.625+0000 7fd6414ab700 1 --2- 172.21.15.16:0/955535405 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd63c134260 0x7fd63c1304d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:02.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.625+0000 7fd633fff700 1 --2- 172.21.15.16:0/955535405 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd63c133e30 0x7fd63c12ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:02.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.625+0000 7fd640caa700 1 -- 172.21.15.16:0/955535405 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd63c133e30 msgr2=0x7fd63c12ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:02.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.625+0000 7fd640caa700 1 --2- 172.21.15.16:0/955535405 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd63c133e30 0x7fd63c12ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:02.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.625+0000 7fd640caa700 1 -- 172.21.15.16:0/955535405 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd63c134260 msgr2=0x7fd63c1304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:02.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.625+0000 7fd640caa700 1 --2- 172.21.15.16:0/955535405 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd63c134260 0x7fd63c1304d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:02.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.625+0000 7fd640caa700 1 -- 172.21.15.16:0/955535405 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd63c0a0030 con 0x7fd63c133420 2023-09-30T22:35:02.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.625+0000 7fd633fff700 1 --2- 172.21.15.16:0/955535405 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd63c133e30 0x7fd63c12ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:02.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.625+0000 7fd6414ab700 1 --2- 172.21.15.16:0/955535405 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd63c134260 0x7fd63c1304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:02.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.625+0000 7fd640caa700 1 --2- 172.21.15.16:0/955535405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd63c133420 0x7fd63c1337f0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fd6380098f0 tx=0x7fd638009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77eaf42a64de1709 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:02.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.626+0000 7fd6337fe700 1 -- 172.21.15.16:0/955535405 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd638011830 con 0x7fd63c133420 2023-09-30T22:35:02.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.626+0000 7fd6337fe700 1 -- 172.21.15.16:0/955535405 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd638011990 con 0x7fd63c133420 2023-09-30T22:35:02.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.626+0000 7fd6337fe700 1 -- 172.21.15.16:0/955535405 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd638011d20 con 0x7fd63c133420 2023-09-30T22:35:02.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.627+0000 7fd642f0e700 1 -- 172.21.15.16:0/955535405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd63c133420 msgr2=0x7fd63c1337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:02.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.627+0000 7fd642f0e700 1 --2- 172.21.15.16:0/955535405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd63c133420 0x7fd63c1337f0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fd6380098f0 tx=0x7fd638009c00 comp rx=0 tx=0).stop 2023-09-30T22:35:02.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.627+0000 7fd642f0e700 1 -- 172.21.15.16:0/955535405 shutdown_connections 2023-09-30T22:35:02.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.627+0000 7fd642f0e700 1 --2- 172.21.15.16:0/955535405 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd63c134260 0x7fd63c1304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:02.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.627+0000 7fd642f0e700 1 --2- 172.21.15.16:0/955535405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd63c133420 0x7fd63c1337f0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:02.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.627+0000 7fd642f0e700 1 --2- 172.21.15.16:0/955535405 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd63c133e30 0x7fd63c12ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:02.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.627+0000 7fd642f0e700 1 -- 172.21.15.16:0/955535405 >> 172.21.15.16:0/955535405 conn(0x7fd63c0a4210 msgr2=0x7fd63c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:02.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.627+0000 7fd642f0e700 1 -- 172.21.15.16:0/955535405 shutdown_connections 2023-09-30T22:35:02.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.627+0000 7fd642f0e700 1 -- 172.21.15.16:0/955535405 wait complete. 2023-09-30T22:35:02.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.628+0000 7fd642f0e700 1 Processor -- start 2023-09-30T22:35:02.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.628+0000 7fd642f0e700 1 -- start start 2023-09-30T22:35:02.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.629+0000 7fd642f0e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd63c133420 0x7fd63c09c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:02.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.629+0000 7fd642f0e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd63c133e30 0x7fd63c09c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:02.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.629+0000 7fd642f0e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd63c134260 0x7fd63c09cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:02.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.629+0000 7fd642f0e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c11f180 con 0x7fd63c133420 2023-09-30T22:35:02.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.629+0000 7fd642f0e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c09df60 con 0x7fd63c133e30 2023-09-30T22:35:02.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.629+0000 7fd642f0e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c09dd70 con 0x7fd63c134260 2023-09-30T22:35:02.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.629+0000 7fd640caa700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd63c133420 0x7fd63c09c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:02.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.629+0000 7fd640caa700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd63c133420 0x7fd63c09c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43974/0 (socket says 172.21.15.16:43974) 2023-09-30T22:35:02.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.629+0000 7fd640caa700 1 -- 172.21.15.16:0/2737073811 learned_addr learned my addr 172.21.15.16:0/2737073811 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:02.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.629+0000 7fd6414ab700 1 --2- 172.21.15.16:0/2737073811 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd63c134260 0x7fd63c09cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:02.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.629+0000 7fd633fff700 1 --2- 172.21.15.16:0/2737073811 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd63c133e30 0x7fd63c09c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:02.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.630+0000 7fd640caa700 1 -- 172.21.15.16:0/2737073811 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd63c134260 msgr2=0x7fd63c09cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:02.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.630+0000 7fd640caa700 1 --2- 172.21.15.16:0/2737073811 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd63c134260 0x7fd63c09cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:02.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.630+0000 7fd640caa700 1 -- 172.21.15.16:0/2737073811 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd63c133e30 msgr2=0x7fd63c09c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:02.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.630+0000 7fd640caa700 1 --2- 172.21.15.16:0/2737073811 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd63c133e30 0x7fd63c09c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:02.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.630+0000 7fd640caa700 1 -- 172.21.15.16:0/2737073811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd62c0094a0 con 0x7fd63c133420 2023-09-30T22:35:02.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.630+0000 7fd640caa700 1 --2- 172.21.15.16:0/2737073811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd63c133420 0x7fd63c09c480 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fd638000c00 tx=0x7fd638004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:02.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.630+0000 7fd6317fa700 1 -- 172.21.15.16:0/2737073811 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd638004450 con 0x7fd63c133420 2023-09-30T22:35:02.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.630+0000 7fd6317fa700 1 -- 172.21.15.16:0/2737073811 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6380045b0 con 0x7fd63c133420 2023-09-30T22:35:02.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.630+0000 7fd6317fa700 1 -- 172.21.15.16:0/2737073811 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6380038e0 con 0x7fd63c133420 2023-09-30T22:35:02.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.630+0000 7fd642f0e700 1 -- 172.21.15.16:0/2737073811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd634009a70 con 0x7fd63c133420 2023-09-30T22:35:02.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.630+0000 7fd642f0e700 1 -- 172.21.15.16:0/2737073811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd638009550 con 0x7fd63c133420 2023-09-30T22:35:02.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.631+0000 7fd6317fa700 1 -- 172.21.15.16:0/2737073811 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd638021070 con 0x7fd63c133420 2023-09-30T22:35:02.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.631+0000 7fd6317fa700 1 -- 172.21.15.16:0/2737073811 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd638040ac0 con 0x7fd63c133420 2023-09-30T22:35:02.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.632+0000 7fd642f0e700 1 -- 172.21.15.16:0/2737073811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd63c074ea0 con 0x7fd63c133420 2023-09-30T22:35:02.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.637+0000 7fd6317fa700 1 -- 172.21.15.16:0/2737073811 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd638029080 con 0x7fd63c133420 2023-09-30T22:35:02.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.661+0000 7fafc3d9c700 1 Processor -- start 2023-09-30T22:35:02.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.661+0000 7fafc3d9c700 1 -- start start 2023-09-30T22:35:02.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.662+0000 7fafc3d9c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fafbc0a0a30 0x7fafbc0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:02.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.662+0000 7fafc3d9c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fafbc0a1340 0x7fafbc133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:02.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.662+0000 7fafc3d9c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fafbc133fa0 0x7fafbc134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:02.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.662+0000 7fafc3d9c700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafbc09edd0 con 0x7fafbc0a1340 2023-09-30T22:35:02.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.662+0000 7fafc3d9c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafbc09e1f0 con 0x7fafbc133fa0 2023-09-30T22:35:02.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.662+0000 7fafc3d9c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fafbc09dc50 con 0x7fafbc0a0a30 2023-09-30T22:35:02.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.662+0000 7fafc1b38700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fafbc0a0a30 0x7fafbc0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:02.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.662+0000 7fafc1b38700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fafbc0a0a30 0x7fafbc0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:34086/0 (socket says 172.21.15.16:34086) 2023-09-30T22:35:02.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.662+0000 7fafc1b38700 1 -- 172.21.15.16:0/2949604047 learned_addr learned my addr 172.21.15.16:0/2949604047 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:02.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.662+0000 7fafc2339700 1 --2- 172.21.15.16:0/2949604047 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fafbc133fa0 0x7fafbc134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:02.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.662+0000 7fafc1337700 1 --2- 172.21.15.16:0/2949604047 >> v2:172.21.15.16:3300/0 conn(0x7fafbc0a1340 0x7fafbc133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:02.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.663+0000 7fafc1b38700 1 -- 172.21.15.16:0/2949604047 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fafbc133fa0 msgr2=0x7fafbc134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:02.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.663+0000 7fafc1b38700 1 --2- 172.21.15.16:0/2949604047 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fafbc133fa0 0x7fafbc134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:02.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.663+0000 7fafc1b38700 1 -- 172.21.15.16:0/2949604047 >> v2:172.21.15.16:3300/0 conn(0x7fafbc0a1340 msgr2=0x7fafbc133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:02.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.663+0000 7fafc1b38700 1 --2- 172.21.15.16:0/2949604047 >> v2:172.21.15.16:3300/0 conn(0x7fafbc0a1340 0x7fafbc133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:02.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.663+0000 7fafc1b38700 1 -- 172.21.15.16:0/2949604047 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafbc0a1fe0 con 0x7fafbc0a0a30 2023-09-30T22:35:02.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.663+0000 7fafc2339700 1 --2- 172.21.15.16:0/2949604047 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fafbc133fa0 0x7fafbc134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:02.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.663+0000 7fafc1b38700 1 --2- 172.21.15.16:0/2949604047 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fafbc0a0a30 0x7fafbc0a0e00 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fafb8009e80 tx=0x7fafb800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c55618841971fd10 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:02.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.663+0000 7fafc0b36700 1 -- 172.21.15.16:0/2949604047 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb800e070 con 0x7fafbc0a0a30 2023-09-30T22:35:02.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.663+0000 7fafc0b36700 1 -- 172.21.15.16:0/2949604047 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafb800b8d0 con 0x7fafbc0a0a30 2023-09-30T22:35:02.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.663+0000 7fafc0b36700 1 -- 172.21.15.16:0/2949604047 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb800bc80 con 0x7fafbc0a0a30 2023-09-30T22:35:02.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.664+0000 7fafc3d9c700 1 -- 172.21.15.16:0/2949604047 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fafbc0a0a30 msgr2=0x7fafbc0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:02.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.664+0000 7fafc3d9c700 1 --2- 172.21.15.16:0/2949604047 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fafbc0a0a30 0x7fafbc0a0e00 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fafb8009e80 tx=0x7fafb800b330 comp rx=0 tx=0).stop 2023-09-30T22:35:02.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.664+0000 7fafc3d9c700 1 -- 172.21.15.16:0/2949604047 shutdown_connections 2023-09-30T22:35:02.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.664+0000 7fafc3d9c700 1 --2- 172.21.15.16:0/2949604047 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fafbc133fa0 0x7fafbc134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:02.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.664+0000 7fafc3d9c700 1 --2- 172.21.15.16:0/2949604047 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fafbc0a0a30 0x7fafbc0a0e00 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:02.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.664+0000 7fafc3d9c700 1 --2- 172.21.15.16:0/2949604047 >> v2:172.21.15.16:3300/0 conn(0x7fafbc0a1340 0x7fafbc133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:02.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.664+0000 7fafc3d9c700 1 -- 172.21.15.16:0/2949604047 >> 172.21.15.16:0/2949604047 conn(0x7fafbc0a6550 msgr2=0x7fafbc0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:02.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.664+0000 7fafc3d9c700 1 -- 172.21.15.16:0/2949604047 shutdown_connections 2023-09-30T22:35:02.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.664+0000 7fafc3d9c700 1 -- 172.21.15.16:0/2949604047 wait complete. 2023-09-30T22:35:02.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.665+0000 7fafc3d9c700 1 Processor -- start 2023-09-30T22:35:02.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.665+0000 7fafc3d9c700 1 -- start start 2023-09-30T22:35:02.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.665+0000 7fafc3d9c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fafbc0a0a30 0x7fafbc11e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:02.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.665+0000 7fafc3d9c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafbc0a1340 0x7fafbc11e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:02.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.665+0000 7fafc3d9c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fafbc133fa0 0x7fafbc11ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:02.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.665+0000 7fafc3d9c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafbc120f90 con 0x7fafbc0a1340 2023-09-30T22:35:02.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.665+0000 7fafc3d9c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafbc0a0220 con 0x7fafbc133fa0 2023-09-30T22:35:02.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.665+0000 7fafc3d9c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fafbc120320 con 0x7fafbc0a0a30 2023-09-30T22:35:02.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.666+0000 7fafc1b38700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fafbc0a0a30 0x7fafbc11e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:02.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.666+0000 7fafc1b38700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fafbc0a0a30 0x7fafbc11e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:34100/0 (socket says 172.21.15.16:34100) 2023-09-30T22:35:02.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.666+0000 7fafc1b38700 1 -- 172.21.15.16:0/2178392303 learned_addr learned my addr 172.21.15.16:0/2178392303 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:02.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.666+0000 7fafc2339700 1 --2- 172.21.15.16:0/2178392303 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fafbc133fa0 0x7fafbc11ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:02.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.666+0000 7fafc1337700 1 --2- 172.21.15.16:0/2178392303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafbc0a1340 0x7fafbc11e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:02.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.666+0000 7fafc1b38700 1 -- 172.21.15.16:0/2178392303 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fafbc133fa0 msgr2=0x7fafbc11ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:02.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.666+0000 7fafc1b38700 1 --2- 172.21.15.16:0/2178392303 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fafbc133fa0 0x7fafbc11ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:02.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.666+0000 7fafc1b38700 1 -- 172.21.15.16:0/2178392303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafbc0a1340 msgr2=0x7fafbc11e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:02.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.666+0000 7fafc1b38700 1 --2- 172.21.15.16:0/2178392303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafbc0a1340 0x7fafbc11e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:02.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.666+0000 7fafc1b38700 1 -- 172.21.15.16:0/2178392303 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb40094a0 con 0x7fafbc0a0a30 2023-09-30T22:35:02.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.666+0000 7fafc1b38700 1 --2- 172.21.15.16:0/2178392303 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fafbc0a0a30 0x7fafbc11e300 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fafbc09dd90 tx=0x7fafb8004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:02.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.666+0000 7fafb27fc700 1 -- 172.21.15.16:0/2178392303 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb8004450 con 0x7fafbc0a0a30 2023-09-30T22:35:02.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.666+0000 7fafb27fc700 1 -- 172.21.15.16:0/2178392303 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafb80045b0 con 0x7fafbc0a0a30 2023-09-30T22:35:02.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.666+0000 7fafc3d9c700 1 -- 172.21.15.16:0/2178392303 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafb8009b30 con 0x7fafbc0a0a30 2023-09-30T22:35:02.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.666+0000 7fafb27fc700 1 -- 172.21.15.16:0/2178392303 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb8004960 con 0x7fafbc0a0a30 2023-09-30T22:35:02.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.666+0000 7fafc3d9c700 1 -- 172.21.15.16:0/2178392303 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafbc1219e0 con 0x7fafbc0a0a30 2023-09-30T22:35:02.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.667+0000 7fafb27fc700 1 -- 172.21.15.16:0/2178392303 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fafb801c440 con 0x7fafbc0a0a30 2023-09-30T22:35:02.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.668+0000 7fafb27fc700 1 -- 172.21.15.16:0/2178392303 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fafb8040ce0 con 0x7fafbc0a0a30 2023-09-30T22:35:02.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.668+0000 7fafc3d9c700 1 -- 172.21.15.16:0/2178392303 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafbc09b1e0 con 0x7fafbc0a0a30 2023-09-30T22:35:02.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.677+0000 7fafb27fc700 1 -- 172.21.15.16:0/2178392303 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fafb80249a0 con 0x7fafbc0a0a30 2023-09-30T22:35:02.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.817+0000 7fe94c5c3700 1 -- 172.21.15.16:0/3766391896 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe944074ea0 con 0x7fe9441337f0 2023-09-30T22:35:02.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.817+0000 7fe93affd700 1 -- 172.21.15.16:0/3766391896 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe934026480 con 0x7fe9441337f0 2023-09-30T22:35:02.818 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:35:02.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.821+0000 7fe94c5c3700 1 -- 172.21.15.16:0/3766391896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9441337f0 msgr2=0x7fe94412f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:02.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.821+0000 7fe94c5c3700 1 --2- 172.21.15.16:0/3766391896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9441337f0 0x7fe94412f390 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fe934016000 tx=0x7fe934004750 comp rx=0 tx=0).stop 2023-09-30T22:35:02.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.821+0000 7fe94c5c3700 1 -- 172.21.15.16:0/3766391896 shutdown_connections 2023-09-30T22:35:02.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.821+0000 7fe94c5c3700 1 --2- 172.21.15.16:0/3766391896 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe944133420 0x7fe94412ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:02.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.821+0000 7fe94c5c3700 1 --2- 172.21.15.16:0/3766391896 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe944138dc0 0x7fe94412f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:02.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.821+0000 7fe94c5c3700 1 --2- 172.21.15.16:0/3766391896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe9441337f0 0x7fe94412f390 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:02.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.821+0000 7fe94c5c3700 1 -- 172.21.15.16:0/3766391896 >> 172.21.15.16:0/3766391896 conn(0x7fe9440a4210 msgr2=0x7fe944131400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:02.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.821+0000 7fe94c5c3700 1 -- 172.21.15.16:0/3766391896 shutdown_connections 2023-09-30T22:35:02.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.821+0000 7fe94c5c3700 1 -- 172.21.15.16:0/3766391896 wait complete. 2023-09-30T22:35:02.840 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:35:02.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.847+0000 7fd642f0e700 1 -- 172.21.15.16:0/2737073811 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd63c11f4d0 con 0x7fd63c133420 2023-09-30T22:35:02.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.848+0000 7fd6317fa700 1 -- 172.21.15.16:0/2737073811 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd638015020 con 0x7fd63c133420 2023-09-30T22:35:02.848 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:35:02.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.850+0000 7fd642f0e700 1 -- 172.21.15.16:0/2737073811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd63c133420 msgr2=0x7fd63c09c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:02.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.850+0000 7fd642f0e700 1 --2- 172.21.15.16:0/2737073811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd63c133420 0x7fd63c09c480 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fd638000c00 tx=0x7fd638004750 comp rx=0 tx=0).stop 2023-09-30T22:35:02.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.850+0000 7fd642f0e700 1 -- 172.21.15.16:0/2737073811 shutdown_connections 2023-09-30T22:35:02.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.850+0000 7fd642f0e700 1 --2- 172.21.15.16:0/2737073811 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd63c134260 0x7fd63c09cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:02.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.850+0000 7fd642f0e700 1 --2- 172.21.15.16:0/2737073811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd63c133420 0x7fd63c09c480 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:02.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.850+0000 7fd642f0e700 1 --2- 172.21.15.16:0/2737073811 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd63c133e30 0x7fd63c09c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:02.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.850+0000 7fd642f0e700 1 -- 172.21.15.16:0/2737073811 >> 172.21.15.16:0/2737073811 conn(0x7fd63c0a4210 msgr2=0x7fd63c0a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:02.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.850+0000 7fd642f0e700 1 -- 172.21.15.16:0/2737073811 shutdown_connections 2023-09-30T22:35:02.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.850+0000 7fd642f0e700 1 -- 172.21.15.16:0/2737073811 wait complete. 2023-09-30T22:35:02.864 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:35:02.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.895+0000 7fafc3d9c700 1 -- 172.21.15.16:0/2178392303 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fafbc121680 con 0x7fafbc0a0a30 2023-09-30T22:35:02.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.895+0000 7fafb27fc700 1 -- 172.21.15.16:0/2178392303 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fafb80243b0 con 0x7fafbc0a0a30 2023-09-30T22:35:02.896 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:35:02.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.898+0000 7fafc3d9c700 1 -- 172.21.15.16:0/2178392303 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fafbc0a0a30 msgr2=0x7fafbc11e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:02.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.898+0000 7fafc3d9c700 1 --2- 172.21.15.16:0/2178392303 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fafbc0a0a30 0x7fafbc11e300 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fafbc09dd90 tx=0x7fafb8004ad0 comp rx=0 tx=0).stop 2023-09-30T22:35:02.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.898+0000 7fafc3d9c700 1 -- 172.21.15.16:0/2178392303 shutdown_connections 2023-09-30T22:35:02.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.898+0000 7fafc3d9c700 1 --2- 172.21.15.16:0/2178392303 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fafbc133fa0 0x7fafbc11ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:02.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.898+0000 7fafc3d9c700 1 --2- 172.21.15.16:0/2178392303 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fafbc0a0a30 0x7fafbc11e300 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:02.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.898+0000 7fafc3d9c700 1 --2- 172.21.15.16:0/2178392303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fafbc0a1340 0x7fafbc11e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:02.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.898+0000 7fafc3d9c700 1 -- 172.21.15.16:0/2178392303 >> 172.21.15.16:0/2178392303 conn(0x7fafbc0a6550 msgr2=0x7fafbc1331f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:02.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.898+0000 7fafc3d9c700 1 -- 172.21.15.16:0/2178392303 shutdown_connections 2023-09-30T22:35:02.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:02.898+0000 7fafc3d9c700 1 -- 172.21.15.16:0/2178392303 wait complete. 2023-09-30T22:35:02.912 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:35:03.285 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:35:03.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.432+0000 7f85cea1b700 1 Processor -- start 2023-09-30T22:35:03.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.432+0000 7f85cea1b700 1 -- start start 2023-09-30T22:35:03.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.432+0000 7f85cea1b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f85c809e1d0 0x7f85c8135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:03.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.432+0000 7f85cea1b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85c8135e50 0x7f85c8138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:03.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.432+0000 7f85cea1b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85c809f130 con 0x7f85c809d6f0 2023-09-30T22:35:03.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.432+0000 7f85cea1b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85c809f520 con 0x7f85c809e1d0 2023-09-30T22:35:03.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.432+0000 7f85cea1b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f85c809e930 con 0x7f85c8135e50 2023-09-30T22:35:03.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.433+0000 7f85ccfb8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85c8135e50 0x7f85c8138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:03.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.433+0000 7f85c77fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f85c809e1d0 0x7f85c8135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:03.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.433+0000 7f85ccfb8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85c8135e50 0x7f85c8138230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:34106/0 (socket says 172.21.15.16:34106) 2023-09-30T22:35:03.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.433+0000 7f85c77fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f85c809e1d0 0x7f85c8135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43996/0 (socket says 172.21.15.16:43996) 2023-09-30T22:35:03.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.433+0000 7f85ccfb8700 1 -- 172.21.15.16:0/2942741568 learned_addr learned my addr 172.21.15.16:0/2942741568 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:03.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.433+0000 7f85c77fe700 1 -- 172.21.15.16:0/2942741568 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85c8135e50 msgr2=0x7f85c8138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:03.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.433+0000 7f85c77fe700 1 --2- 172.21.15.16:0/2942741568 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85c8135e50 0x7f85c8138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:03.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.433+0000 7f85c77fe700 1 -- 172.21.15.16:0/2942741568 >> v1:172.21.15.16:6789/0 conn(0x7f85c809d6f0 legacy=0x7f85c809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:03.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.433+0000 7f85c77fe700 1 -- 172.21.15.16:0/2942741568 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c809b2d0 con 0x7f85c809e1d0 2023-09-30T22:35:03.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.433+0000 7f85ccfb8700 1 --2- 172.21.15.16:0/2942741568 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85c8135e50 0x7f85c8138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:03.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.433+0000 7f85c77fe700 1 --2- 172.21.15.16:0/2942741568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85c809e1d0 0x7f85c8135910 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f85bc009840 tx=0x7f85bc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c145cff2179a48b1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:03.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.433+0000 7f85c6ffd700 1 -- 172.21.15.16:0/2942741568 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85bc011830 con 0x7f85c809e1d0 2023-09-30T22:35:03.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.433+0000 7f85c6ffd700 1 -- 172.21.15.16:0/2942741568 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85bc011990 con 0x7f85c809e1d0 2023-09-30T22:35:03.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.433+0000 7f85c6ffd700 1 -- 172.21.15.16:0/2942741568 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85bc011d20 con 0x7f85c809e1d0 2023-09-30T22:35:03.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.434+0000 7f85cea1b700 1 -- 172.21.15.16:0/2942741568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85c809e1d0 msgr2=0x7f85c8135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:03.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.434+0000 7f85cea1b700 1 --2- 172.21.15.16:0/2942741568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85c809e1d0 0x7f85c8135910 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f85bc009840 tx=0x7f85bc009b50 comp rx=0 tx=0).stop 2023-09-30T22:35:03.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.434+0000 7f85cea1b700 1 -- 172.21.15.16:0/2942741568 shutdown_connections 2023-09-30T22:35:03.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.434+0000 7f85cea1b700 1 --2- 172.21.15.16:0/2942741568 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85c8135e50 0x7f85c8138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:03.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.434+0000 7f85cea1b700 1 --2- 172.21.15.16:0/2942741568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85c809e1d0 0x7f85c8135910 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:03.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.434+0000 7f85cea1b700 1 -- 172.21.15.16:0/2942741568 >> 172.21.15.16:0/2942741568 conn(0x7f85c8066f50 msgr2=0x7f85c809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:03.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.434+0000 7f85cea1b700 1 -- 172.21.15.16:0/2942741568 shutdown_connections 2023-09-30T22:35:03.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.434+0000 7f85cea1b700 1 -- 172.21.15.16:0/2942741568 wait complete. 2023-09-30T22:35:03.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.435+0000 7f85cea1b700 1 Processor -- start 2023-09-30T22:35:03.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.435+0000 7f85cea1b700 1 -- start start 2023-09-30T22:35:03.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.436+0000 7f85cea1b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85c809d6f0 0x7f85c812f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:03.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.436+0000 7f85cea1b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85c809e1d0 0x7f85c812fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:03.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.436+0000 7f85cea1b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85c8135e50 0x7f85c81397b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:03.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.436+0000 7f85cea1b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c8134150 con 0x7f85c809d6f0 2023-09-30T22:35:03.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.436+0000 7f85cea1b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85c8133770 con 0x7f85c8135e50 2023-09-30T22:35:03.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.436+0000 7f85cea1b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f85c81383d0 con 0x7f85c809e1d0 2023-09-30T22:35:03.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.436+0000 7f85c77fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85c809e1d0 0x7f85c812fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:03.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.436+0000 7f85c77fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85c809e1d0 0x7f85c812fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:34116/0 (socket says 172.21.15.16:34116) 2023-09-30T22:35:03.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.436+0000 7f85c77fe700 1 -- 172.21.15.16:0/1872486467 learned_addr learned my addr 172.21.15.16:0/1872486467 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:03.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.436+0000 7f85ccfb8700 1 --2- 172.21.15.16:0/1872486467 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85c8135e50 0x7f85c81397b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:03.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.436+0000 7f85c7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85c809d6f0 0x7f85c812f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:03.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.436+0000 7f85c77fe700 1 -- 172.21.15.16:0/1872486467 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85c8135e50 msgr2=0x7f85c81397b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:03.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.436+0000 7f85c77fe700 1 --2- 172.21.15.16:0/1872486467 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85c8135e50 0x7f85c81397b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:03.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.436+0000 7f85c77fe700 1 -- 172.21.15.16:0/1872486467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85c809d6f0 msgr2=0x7f85c812f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:03.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.436+0000 7f85c77fe700 1 --2- 172.21.15.16:0/1872486467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85c809d6f0 0x7f85c812f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:03.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.436+0000 7f85c77fe700 1 -- 172.21.15.16:0/1872486467 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c0009b30 con 0x7f85c809e1d0 2023-09-30T22:35:03.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.436+0000 7f85c77fe700 1 --2- 172.21.15.16:0/1872486467 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85c809e1d0 0x7f85c812fef0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f85bc0090b0 tx=0x7f85bc004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:03.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.436+0000 7f85c4ff9700 1 -- 172.21.15.16:0/1872486467 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85bc0036a0 con 0x7f85c809e1d0 2023-09-30T22:35:03.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.437+0000 7f85c4ff9700 1 -- 172.21.15.16:0/1872486467 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85bc01e070 con 0x7f85c809e1d0 2023-09-30T22:35:03.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.437+0000 7f85cea1b700 1 -- 172.21.15.16:0/1872486467 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85bc0094a0 con 0x7f85c809e1d0 2023-09-30T22:35:03.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.437+0000 7f85c4ff9700 1 -- 172.21.15.16:0/1872486467 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85bc003a40 con 0x7f85c809e1d0 2023-09-30T22:35:03.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.437+0000 7f85cea1b700 1 -- 172.21.15.16:0/1872486467 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85c8132d80 con 0x7f85c809e1d0 2023-09-30T22:35:03.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.437+0000 7f85c4ff9700 1 -- 172.21.15.16:0/1872486467 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f85bc003ba0 con 0x7f85c809e1d0 2023-09-30T22:35:03.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.438+0000 7f85c4ff9700 1 -- 172.21.15.16:0/1872486467 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f85bc0405a0 con 0x7f85c809e1d0 2023-09-30T22:35:03.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.438+0000 7f85cea1b700 1 -- 172.21.15.16:0/1872486467 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85c8074ea0 con 0x7f85c809e1d0 2023-09-30T22:35:03.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.442+0000 7f85c4ff9700 1 -- 172.21.15.16:0/1872486467 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f85bc015030 con 0x7f85c809e1d0 2023-09-30T22:35:03.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.646+0000 7f85cea1b700 1 -- 172.21.15.16:0/1872486467 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f85c80a29c0 con 0x7f85c809e1d0 2023-09-30T22:35:03.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.646+0000 7f85c4ff9700 1 -- 172.21.15.16:0/1872486467 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f85bc02d7d0 con 0x7f85c809e1d0 2023-09-30T22:35:03.647 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:35:03.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.648+0000 7f85cea1b700 1 -- 172.21.15.16:0/1872486467 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85c809e1d0 msgr2=0x7f85c812fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:03.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.648+0000 7f85cea1b700 1 --2- 172.21.15.16:0/1872486467 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85c809e1d0 0x7f85c812fef0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f85bc0090b0 tx=0x7f85bc004750 comp rx=0 tx=0).stop 2023-09-30T22:35:03.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.648+0000 7f85cea1b700 1 -- 172.21.15.16:0/1872486467 shutdown_connections 2023-09-30T22:35:03.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.648+0000 7f85cea1b700 1 --2- 172.21.15.16:0/1872486467 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85c8135e50 0x7f85c81397b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:03.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.648+0000 7f85cea1b700 1 --2- 172.21.15.16:0/1872486467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85c809d6f0 0x7f85c812f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:03.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.648+0000 7f85cea1b700 1 --2- 172.21.15.16:0/1872486467 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85c809e1d0 0x7f85c812fef0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:03.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.648+0000 7f85cea1b700 1 -- 172.21.15.16:0/1872486467 >> 172.21.15.16:0/1872486467 conn(0x7f85c8066f50 msgr2=0x7f85c806a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:03.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.648+0000 7f85cea1b700 1 -- 172.21.15.16:0/1872486467 shutdown_connections 2023-09-30T22:35:03.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.648+0000 7f85cea1b700 1 -- 172.21.15.16:0/1872486467 wait complete. 2023-09-30T22:35:03.662 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:35:03.841 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:35:03.865 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:35:03.913 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:35:03.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.993+0000 7f0a82baf700 1 Processor -- start 2023-09-30T22:35:03.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.993+0000 7f0a82baf700 1 -- start start 2023-09-30T22:35:03.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.993+0000 7f0a82baf700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0a7c136710 0x7f0a7c138af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:03.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.993+0000 7f0a82baf700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0a7c139030 0x7f0a7c13b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:03.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.993+0000 7f0a82baf700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a7c06ef30 con 0x7f0a7c1338a0 2023-09-30T22:35:03.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.993+0000 7f0a82baf700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a7c06fef0 con 0x7f0a7c139030 2023-09-30T22:35:03.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.993+0000 7f0a82baf700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7c1334b0 con 0x7f0a7c136710 2023-09-30T22:35:03.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.993+0000 7f0a8114c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0a7c139030 0x7f0a7c13b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:03.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.994+0000 7f0a8114c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0a7c139030 0x7f0a7c13b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44002/0 (socket says 172.21.15.16:44002) 2023-09-30T22:35:03.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.994+0000 7f0a8114c700 1 -- 172.21.15.16:0/1975262398 learned_addr learned my addr 172.21.15.16:0/1975262398 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:03.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.994+0000 7f0a8094b700 1 --1- 172.21.15.16:0/1975262398 >> v1:172.21.15.16:6789/0 conn(0x7f0a7c1338a0 0x7f0a7c133c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50836/0 (socket says 172.21.15.16:50836) 2023-09-30T22:35:03.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.994+0000 7f0a7bfff700 1 --2- 172.21.15.16:0/1975262398 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0a7c136710 0x7f0a7c138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:03.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.994+0000 7f0a7b7fe700 1 -- 172.21.15.16:0/1975262398 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3722848317 0 0) 0x7f0a7c06ef30 con 0x7f0a7c1338a0 2023-09-30T22:35:03.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.994+0000 7f0a7b7fe700 1 -- 172.21.15.16:0/1975262398 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a6c0035b0 con 0x7f0a7c1338a0 2023-09-30T22:35:03.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.994+0000 7f0a8114c700 1 -- 172.21.15.16:0/1975262398 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0a7c136710 msgr2=0x7f0a7c138af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:03.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.994+0000 7f0a8114c700 1 --2- 172.21.15.16:0/1975262398 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0a7c136710 0x7f0a7c138af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:03.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.994+0000 7f0a8114c700 1 -- 172.21.15.16:0/1975262398 >> v1:172.21.15.16:6789/0 conn(0x7f0a7c1338a0 legacy=0x7f0a7c133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:03.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.994+0000 7f0a8114c700 1 -- 172.21.15.16:0/1975262398 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a7c06fc30 con 0x7f0a7c139030 2023-09-30T22:35:03.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.994+0000 7f0a7bfff700 1 --2- 172.21.15.16:0/1975262398 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0a7c136710 0x7f0a7c138af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:03.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.994+0000 7f0a8114c700 1 --2- 172.21.15.16:0/1975262398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a7c139030 0x7f0a7c13b410 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f0a74009e50 tx=0x7f0a7400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=391214e13fea2b86 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:03.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.994+0000 7f0a7b7fe700 1 -- 172.21.15.16:0/1975262398 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7400e070 con 0x7f0a7c139030 2023-09-30T22:35:03.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.994+0000 7f0a7b7fe700 1 -- 172.21.15.16:0/1975262398 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a7400b910 con 0x7f0a7c139030 2023-09-30T22:35:03.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.994+0000 7f0a7b7fe700 1 -- 172.21.15.16:0/1975262398 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7400bcc0 con 0x7f0a7c139030 2023-09-30T22:35:03.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.995+0000 7f0a82baf700 1 -- 172.21.15.16:0/1975262398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a7c139030 msgr2=0x7f0a7c13b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:03.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.995+0000 7f0a82baf700 1 --2- 172.21.15.16:0/1975262398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a7c139030 0x7f0a7c13b410 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f0a74009e50 tx=0x7f0a7400b330 comp rx=0 tx=0).stop 2023-09-30T22:35:03.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.995+0000 7f0a82baf700 1 -- 172.21.15.16:0/1975262398 shutdown_connections 2023-09-30T22:35:03.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.995+0000 7f0a82baf700 1 --2- 172.21.15.16:0/1975262398 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a7c139030 0x7f0a7c13b410 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:03.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.995+0000 7f0a82baf700 1 --2- 172.21.15.16:0/1975262398 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0a7c136710 0x7f0a7c138af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:03.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.995+0000 7f0a82baf700 1 -- 172.21.15.16:0/1975262398 >> 172.21.15.16:0/1975262398 conn(0x7f0a7c0a2aa0 msgr2=0x7f0a7c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:04.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.995+0000 7f0a82baf700 1 -- 172.21.15.16:0/1975262398 shutdown_connections 2023-09-30T22:35:04.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.996+0000 7f0a82baf700 1 -- 172.21.15.16:0/1975262398 wait complete. 2023-09-30T22:35:04.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.996+0000 7f0a82baf700 1 Processor -- start 2023-09-30T22:35:04.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.997+0000 7f0a82baf700 1 -- start start 2023-09-30T22:35:04.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.997+0000 7f0a82baf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a7c1338a0 0x7f0a7c11aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:04.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.997+0000 7f0a82baf700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0a7c136710 0x7f0a7c11afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:04.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.997+0000 7f0a82baf700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0a7c139030 0x7f0a7c11b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:04.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.997+0000 7f0a82baf700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7c130330 con 0x7f0a7c1338a0 2023-09-30T22:35:04.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.997+0000 7f0a82baf700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7c11d5b0 con 0x7f0a7c136710 2023-09-30T22:35:04.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.997+0000 7f0a82baf700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7c06f640 con 0x7f0a7c139030 2023-09-30T22:35:04.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.997+0000 7f0a7bfff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0a7c136710 0x7f0a7c11afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:04.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.997+0000 7f0a8114c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0a7c139030 0x7f0a7c11b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:04.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.997+0000 7f0a7bfff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0a7c136710 0x7f0a7c11afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:52352/0 (socket says 172.21.15.16:52352) 2023-09-30T22:35:04.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.997+0000 7f0a7bfff700 1 -- 172.21.15.16:0/2992965119 learned_addr learned my addr 172.21.15.16:0/2992965119 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:04.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.998+0000 7f0a8094b700 1 --2- 172.21.15.16:0/2992965119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a7c1338a0 0x7f0a7c11aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:04.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.998+0000 7f0a8114c700 1 -- 172.21.15.16:0/2992965119 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0a7c136710 msgr2=0x7f0a7c11afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:04.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.998+0000 7f0a8114c700 1 --2- 172.21.15.16:0/2992965119 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0a7c136710 0x7f0a7c11afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:04.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.998+0000 7f0a8114c700 1 -- 172.21.15.16:0/2992965119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a7c1338a0 msgr2=0x7f0a7c11aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:04.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.998+0000 7f0a8114c700 1 --2- 172.21.15.16:0/2992965119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a7c1338a0 0x7f0a7c11aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:04.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.998+0000 7f0a8114c700 1 -- 172.21.15.16:0/2992965119 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a680094a0 con 0x7f0a7c139030 2023-09-30T22:35:04.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.998+0000 7f0a7bfff700 1 --2- 172.21.15.16:0/2992965119 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0a7c136710 0x7f0a7c11afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:04.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.998+0000 7f0a8094b700 1 --2- 172.21.15.16:0/2992965119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a7c1338a0 0x7f0a7c11aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:35:04.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.998+0000 7f0a8114c700 1 --2- 172.21.15.16:0/2992965119 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0a7c139030 0x7f0a7c11b510 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f0a74015000 tx=0x7f0a74004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:04.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.998+0000 7f0a797fa700 1 -- 172.21.15.16:0/2992965119 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a74004830 con 0x7f0a7c139030 2023-09-30T22:35:04.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.998+0000 7f0a797fa700 1 -- 172.21.15.16:0/2992965119 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a740092e0 con 0x7f0a7c139030 2023-09-30T22:35:04.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.998+0000 7f0a82baf700 1 -- 172.21.15.16:0/2992965119 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a74009b30 con 0x7f0a7c139030 2023-09-30T22:35:04.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.998+0000 7f0a797fa700 1 -- 172.21.15.16:0/2992965119 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a7401c650 con 0x7f0a7c139030 2023-09-30T22:35:04.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.998+0000 7f0a82baf700 1 -- 172.21.15.16:0/2992965119 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a7c11ce90 con 0x7f0a7c139030 2023-09-30T22:35:04.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:03.999+0000 7f0a82baf700 1 -- 172.21.15.16:0/2992965119 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a7c074ea0 con 0x7f0a7c139030 2023-09-30T22:35:04.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.000+0000 7f0a797fa700 1 -- 172.21.15.16:0/2992965119 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0a7401c7b0 con 0x7f0a7c139030 2023-09-30T22:35:04.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.000+0000 7f0a797fa700 1 -- 172.21.15.16:0/2992965119 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f0a74041220 con 0x7f0a7c139030 2023-09-30T22:35:04.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.005+0000 7f0a797fa700 1 -- 172.21.15.16:0/2992965119 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0a7401a940 con 0x7f0a7c139030 2023-09-30T22:35:04.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.031+0000 7fe3b08c9700 1 Processor -- start 2023-09-30T22:35:04.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.031+0000 7fe3b08c9700 1 -- start start 2023-09-30T22:35:04.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.032+0000 7fe3b08c9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe3ac0a0a30 0x7fe3ac0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:04.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.032+0000 7fe3b08c9700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe3ac134170 0x7fe3ac13bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:04.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.032+0000 7fe3b08c9700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3ac09edd0 con 0x7fe3ac0a1340 2023-09-30T22:35:04.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.032+0000 7fe3b08c9700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3ac09e1f0 con 0x7fe3ac134170 2023-09-30T22:35:04.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.032+0000 7fe3b08c9700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe3ac09dc50 con 0x7fe3ac0a0a30 2023-09-30T22:35:04.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.032+0000 7fe3aad9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe3ac134170 0x7fe3ac13bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:04.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.032+0000 7fe3aad9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe3ac134170 0x7fe3ac13bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44010/0 (socket says 172.21.15.16:44010) 2023-09-30T22:35:04.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.032+0000 7fe3aad9d700 1 -- 172.21.15.16:0/1838760851 learned_addr learned my addr 172.21.15.16:0/1838760851 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:04.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.032+0000 7fe3aa59c700 1 --2- 172.21.15.16:0/1838760851 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe3ac0a0a30 0x7fe3ac0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:04.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.033+0000 7fe3aad9d700 1 -- 172.21.15.16:0/1838760851 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe3ac0a0a30 msgr2=0x7fe3ac0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:04.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.033+0000 7fe3aad9d700 1 --2- 172.21.15.16:0/1838760851 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe3ac0a0a30 0x7fe3ac0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:04.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.033+0000 7fe3aad9d700 1 -- 172.21.15.16:0/1838760851 >> v1:172.21.15.16:6789/0 conn(0x7fe3ac0a1340 legacy=0x7fe3ac133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:04.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.033+0000 7fe3aad9d700 1 -- 172.21.15.16:0/1838760851 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3ac0a1fe0 con 0x7fe3ac134170 2023-09-30T22:35:04.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.033+0000 7fe3aad9d700 1 --2- 172.21.15.16:0/1838760851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3ac134170 0x7fe3ac13bd30 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fe3a40098f0 tx=0x7fe3a4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=662a5654326ba152 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:04.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.033+0000 7fe3a959a700 1 -- 172.21.15.16:0/1838760851 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a4011830 con 0x7fe3ac134170 2023-09-30T22:35:04.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.033+0000 7fe3a959a700 1 -- 172.21.15.16:0/1838760851 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3a4011990 con 0x7fe3ac134170 2023-09-30T22:35:04.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.033+0000 7fe3a959a700 1 -- 172.21.15.16:0/1838760851 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a4011d20 con 0x7fe3ac134170 2023-09-30T22:35:04.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.034+0000 7fe3b08c9700 1 -- 172.21.15.16:0/1838760851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3ac134170 msgr2=0x7fe3ac13bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:04.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.034+0000 7fe3b08c9700 1 --2- 172.21.15.16:0/1838760851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3ac134170 0x7fe3ac13bd30 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fe3a40098f0 tx=0x7fe3a4009c00 comp rx=0 tx=0).stop 2023-09-30T22:35:04.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.034+0000 7fe3b08c9700 1 -- 172.21.15.16:0/1838760851 shutdown_connections 2023-09-30T22:35:04.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.034+0000 7fe3b08c9700 1 --2- 172.21.15.16:0/1838760851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3ac134170 0x7fe3ac13bd30 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:04.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.034+0000 7fe3b08c9700 1 --2- 172.21.15.16:0/1838760851 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe3ac0a0a30 0x7fe3ac0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:04.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.034+0000 7fe3b08c9700 1 -- 172.21.15.16:0/1838760851 >> 172.21.15.16:0/1838760851 conn(0x7fe3ac0a6550 msgr2=0x7fe3ac0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:04.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.034+0000 7fe3b08c9700 1 -- 172.21.15.16:0/1838760851 shutdown_connections 2023-09-30T22:35:04.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.034+0000 7fe3b08c9700 1 -- 172.21.15.16:0/1838760851 wait complete. 2023-09-30T22:35:04.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.035+0000 7fe3b08c9700 1 Processor -- start 2023-09-30T22:35:04.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.035+0000 7fe3b08c9700 1 -- start start 2023-09-30T22:35:04.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.036+0000 7fe3b08c9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe3ac0a0a30 0x7fe3ac11e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:04.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.036+0000 7fe3b08c9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3ac0a1340 0x7fe3ac11ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:04.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.036+0000 7fe3b08c9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe3ac134170 0x7fe3ac11f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:04.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.036+0000 7fe3b08c9700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3ac122190 con 0x7fe3ac0a1340 2023-09-30T22:35:04.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.036+0000 7fe3b08c9700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3ac121bf0 con 0x7fe3ac0a0a30 2023-09-30T22:35:04.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.036+0000 7fe3b08c9700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe3ac120280 con 0x7fe3ac134170 2023-09-30T22:35:04.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.036+0000 7fe3a9d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3ac0a1340 0x7fe3ac11ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:04.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.036+0000 7fe3a9d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3ac0a1340 0x7fe3ac11ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44018/0 (socket says 172.21.15.16:44018) 2023-09-30T22:35:04.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.036+0000 7fe3a9d9b700 1 -- 172.21.15.16:0/2035406350 learned_addr learned my addr 172.21.15.16:0/2035406350 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:04.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.036+0000 7fe3aad9d700 1 --2- 172.21.15.16:0/2035406350 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe3ac134170 0x7fe3ac11f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:04.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.036+0000 7fe3aa59c700 1 --2- 172.21.15.16:0/2035406350 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe3ac0a0a30 0x7fe3ac11e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:04.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.036+0000 7fe3a9d9b700 1 -- 172.21.15.16:0/2035406350 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe3ac134170 msgr2=0x7fe3ac11f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:04.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.036+0000 7fe3a9d9b700 1 --2- 172.21.15.16:0/2035406350 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe3ac134170 0x7fe3ac11f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:04.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.036+0000 7fe3a9d9b700 1 -- 172.21.15.16:0/2035406350 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe3ac0a0a30 msgr2=0x7fe3ac11e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:04.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.036+0000 7fe3a9d9b700 1 --2- 172.21.15.16:0/2035406350 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe3ac0a0a30 0x7fe3ac11e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:04.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.036+0000 7fe3a9d9b700 1 -- 172.21.15.16:0/2035406350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a4009550 con 0x7fe3ac0a1340 2023-09-30T22:35:04.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.036+0000 7fe3aa59c700 1 --2- 172.21.15.16:0/2035406350 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe3ac0a0a30 0x7fe3ac11e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:35:04.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.036+0000 7fe3aad9d700 1 --2- 172.21.15.16:0/2035406350 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe3ac134170 0x7fe3ac11f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:04.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.037+0000 7fe3a9d9b700 1 --2- 172.21.15.16:0/2035406350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3ac0a1340 0x7fe3ac11ec40 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fe39400c4e0 tx=0x7fe39400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:04.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.037+0000 7fe3a2ffd700 1 -- 172.21.15.16:0/2035406350 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3940052d0 con 0x7fe3ac0a1340 2023-09-30T22:35:04.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.037+0000 7fe3a2ffd700 1 -- 172.21.15.16:0/2035406350 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe394005430 con 0x7fe3ac0a1340 2023-09-30T22:35:04.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.037+0000 7fe3a2ffd700 1 -- 172.21.15.16:0/2035406350 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3940038c0 con 0x7fe3ac0a1340 2023-09-30T22:35:04.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.037+0000 7fe3b08c9700 1 -- 172.21.15.16:0/2035406350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3ac0a4f10 con 0x7fe3ac0a1340 2023-09-30T22:35:04.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.037+0000 7fe3b08c9700 1 -- 172.21.15.16:0/2035406350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3ac0a0280 con 0x7fe3ac0a1340 2023-09-30T22:35:04.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.038+0000 7fe3a2ffd700 1 -- 172.21.15.16:0/2035406350 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe394003a20 con 0x7fe3ac0a1340 2023-09-30T22:35:04.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.038+0000 7fe3b08c9700 1 -- 172.21.15.16:0/2035406350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3ac074ea0 con 0x7fe3ac0a1340 2023-09-30T22:35:04.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.038+0000 7fe3a2ffd700 1 -- 172.21.15.16:0/2035406350 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fe39403eb00 con 0x7fe3ac0a1340 2023-09-30T22:35:04.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.048+0000 7fe3a2ffd700 1 -- 172.21.15.16:0/2035406350 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe394011ab0 con 0x7fe3ac0a1340 2023-09-30T22:35:04.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.072+0000 7f1d289c4700 1 Processor -- start 2023-09-30T22:35:04.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.072+0000 7f1d289c4700 1 -- start start 2023-09-30T22:35:04.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.073+0000 7f1d289c4700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d240a0090 0x7f1d24136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:04.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.073+0000 7f1d289c4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1d2409d130 0x7f1d24139170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:04.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.073+0000 7f1d289c4700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d2409ba70 con 0x7f1d2409c650 2023-09-30T22:35:04.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.073+0000 7f1d289c4700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2409e8f0 con 0x7f1d240a0090 2023-09-30T22:35:04.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.073+0000 7f1d289c4700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1d240a0760 con 0x7f1d2409d130 2023-09-30T22:35:04.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.073+0000 7f1d2259c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1d2409c650 0x7f1d2409ca20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50854/0 (socket says 172.21.15.16:50854) 2023-09-30T22:35:04.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.073+0000 7f1d2259c700 1 -- 172.21.15.16:0/742516131 learned_addr learned my addr 172.21.15.16:0/742516131 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:04.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.073+0000 7f1d21d9b700 1 --2- 172.21.15.16:0/742516131 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d240a0090 0x7f1d24136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:04.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.073+0000 7f1d22d9d700 1 --2- 172.21.15.16:0/742516131 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1d2409d130 0x7f1d24139170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:04.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.073+0000 7f1d2159a700 1 -- 172.21.15.16:0/742516131 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1825398235 0 0) 0x7f1d2409ba70 con 0x7f1d2409c650 2023-09-30T22:35:04.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.073+0000 7f1d2159a700 1 -- 172.21.15.16:0/742516131 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d0c0035b0 con 0x7f1d2409c650 2023-09-30T22:35:04.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.073+0000 7f1d21d9b700 1 -- 172.21.15.16:0/742516131 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1d2409d130 msgr2=0x7f1d24139170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:04.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.073+0000 7f1d21d9b700 1 --2- 172.21.15.16:0/742516131 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1d2409d130 0x7f1d24139170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:04.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.073+0000 7f1d21d9b700 1 -- 172.21.15.16:0/742516131 >> v1:172.21.15.16:6789/0 conn(0x7f1d2409c650 legacy=0x7f1d2409ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:04.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.073+0000 7f1d21d9b700 1 -- 172.21.15.16:0/742516131 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d2409a080 con 0x7f1d240a0090 2023-09-30T22:35:04.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.074+0000 7f1d21d9b700 1 --2- 172.21.15.16:0/742516131 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d240a0090 0x7f1d24136c20 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f1d18009870 tx=0x7f1d18009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b322ed32a77c2d09 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:04.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.074+0000 7f1d2159a700 1 -- 172.21.15.16:0/742516131 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d18011830 con 0x7f1d240a0090 2023-09-30T22:35:04.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.074+0000 7f1d2159a700 1 -- 172.21.15.16:0/742516131 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d18011990 con 0x7f1d240a0090 2023-09-30T22:35:04.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.074+0000 7f1d2159a700 1 -- 172.21.15.16:0/742516131 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d18011d20 con 0x7f1d240a0090 2023-09-30T22:35:04.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.075+0000 7f1d289c4700 1 -- 172.21.15.16:0/742516131 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d240a0090 msgr2=0x7f1d24136c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:04.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.075+0000 7f1d289c4700 1 --2- 172.21.15.16:0/742516131 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d240a0090 0x7f1d24136c20 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f1d18009870 tx=0x7f1d18009b80 comp rx=0 tx=0).stop 2023-09-30T22:35:04.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.075+0000 7f1d289c4700 1 -- 172.21.15.16:0/742516131 shutdown_connections 2023-09-30T22:35:04.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.075+0000 7f1d289c4700 1 --2- 172.21.15.16:0/742516131 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1d2409d130 0x7f1d24139170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:04.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.075+0000 7f1d289c4700 1 --2- 172.21.15.16:0/742516131 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d240a0090 0x7f1d24136c20 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:04.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.075+0000 7f1d289c4700 1 -- 172.21.15.16:0/742516131 >> 172.21.15.16:0/742516131 conn(0x7f1d240a3f70 msgr2=0x7f1d240a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:04.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.075+0000 7f1d289c4700 1 -- 172.21.15.16:0/742516131 shutdown_connections 2023-09-30T22:35:04.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.075+0000 7f1d289c4700 1 -- 172.21.15.16:0/742516131 wait complete. 2023-09-30T22:35:04.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.076+0000 7f1d289c4700 1 Processor -- start 2023-09-30T22:35:04.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.076+0000 7f1d289c4700 1 -- start start 2023-09-30T22:35:04.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.076+0000 7f1d289c4700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d2409c650 0x7f1d2411a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:04.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.076+0000 7f1d289c4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d2409d130 0x7f1d2411ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:04.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.076+0000 7f1d289c4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1d240a0090 0x7f1d2411b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:04.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.077+0000 7f1d289c4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2411cba0 con 0x7f1d2409d130 2023-09-30T22:35:04.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.077+0000 7f1d289c4700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2411c330 con 0x7f1d2409c650 2023-09-30T22:35:04.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.077+0000 7f1d289c4700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2411bee0 con 0x7f1d240a0090 2023-09-30T22:35:04.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.077+0000 7f1d2259c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d2409c650 0x7f1d2411a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:04.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.077+0000 7f1d22d9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1d240a0090 0x7f1d2411b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:04.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.077+0000 7f1d21d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d2409d130 0x7f1d2411ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:04.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.077+0000 7f1d2259c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d2409c650 0x7f1d2411a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:52400/0 (socket says 172.21.15.16:52400) 2023-09-30T22:35:04.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.077+0000 7f1d2259c700 1 -- 172.21.15.16:0/1810782439 learned_addr learned my addr 172.21.15.16:0/1810782439 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:04.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.077+0000 7f1d2259c700 1 -- 172.21.15.16:0/1810782439 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1d240a0090 msgr2=0x7f1d2411b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:04.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.077+0000 7f1d2259c700 1 --2- 172.21.15.16:0/1810782439 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1d240a0090 0x7f1d2411b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:04.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.077+0000 7f1d2259c700 1 -- 172.21.15.16:0/1810782439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d2409d130 msgr2=0x7f1d2411ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:04.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.077+0000 7f1d2259c700 1 --2- 172.21.15.16:0/1810782439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d2409d130 0x7f1d2411ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:04.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.077+0000 7f1d2259c700 1 -- 172.21.15.16:0/1810782439 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d180094a0 con 0x7f1d2409c650 2023-09-30T22:35:04.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.077+0000 7f1d21d9b700 1 --2- 172.21.15.16:0/1810782439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d2409d130 0x7f1d2411ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:04.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.077+0000 7f1d22d9d700 1 --2- 172.21.15.16:0/1810782439 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1d240a0090 0x7f1d2411b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:04.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.077+0000 7f1d2259c700 1 --2- 172.21.15.16:0/1810782439 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d2409c650 0x7f1d2411a620 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f1d1400c700 tx=0x7f1d1400ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:04.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.078+0000 7f1d12ffd700 1 -- 172.21.15.16:0/1810782439 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d14005d70 con 0x7f1d2409c650 2023-09-30T22:35:04.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.078+0000 7f1d12ffd700 1 -- 172.21.15.16:0/1810782439 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d140070d0 con 0x7f1d2409c650 2023-09-30T22:35:04.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.078+0000 7f1d289c4700 1 -- 172.21.15.16:0/1810782439 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d240a3d10 con 0x7f1d2409c650 2023-09-30T22:35:04.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.078+0000 7f1d12ffd700 1 -- 172.21.15.16:0/1810782439 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d14007460 con 0x7f1d2409c650 2023-09-30T22:35:04.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.078+0000 7f1d289c4700 1 -- 172.21.15.16:0/1810782439 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d240712e0 con 0x7f1d2409c650 2023-09-30T22:35:04.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.079+0000 7f1d289c4700 1 -- 172.21.15.16:0/1810782439 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d241374b0 con 0x7f1d2409c650 2023-09-30T22:35:04.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.079+0000 7f1d12ffd700 1 -- 172.21.15.16:0/1810782439 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1d140075c0 con 0x7f1d2409c650 2023-09-30T22:35:04.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.080+0000 7f1d12ffd700 1 -- 172.21.15.16:0/1810782439 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f1d1403eb60 con 0x7f1d2409c650 2023-09-30T22:35:04.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.085+0000 7f1d12ffd700 1 -- 172.21.15.16:0/1810782439 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1d14018260 con 0x7f1d2409c650 2023-09-30T22:35:04.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.226+0000 7f0a82baf700 1 -- 172.21.15.16:0/2992965119 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0a7c06b9e0 con 0x7f0a7c139030 2023-09-30T22:35:04.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.226+0000 7f0a797fa700 1 -- 172.21.15.16:0/2992965119 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0a7401a380 con 0x7f0a7c139030 2023-09-30T22:35:04.279 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:35:04.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.228+0000 7f0a82baf700 1 -- 172.21.15.16:0/2992965119 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0a7c139030 msgr2=0x7f0a7c11b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:04.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.228+0000 7f0a82baf700 1 --2- 172.21.15.16:0/2992965119 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0a7c139030 0x7f0a7c11b510 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f0a74015000 tx=0x7f0a74004ad0 comp rx=0 tx=0).stop 2023-09-30T22:35:04.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.228+0000 7f0a82baf700 1 -- 172.21.15.16:0/2992965119 shutdown_connections 2023-09-30T22:35:04.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.228+0000 7f0a82baf700 1 --2- 172.21.15.16:0/2992965119 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0a7c139030 0x7f0a7c11b510 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:04.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.228+0000 7f0a82baf700 1 --2- 172.21.15.16:0/2992965119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a7c1338a0 0x7f0a7c11aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:04.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.228+0000 7f0a82baf700 1 --2- 172.21.15.16:0/2992965119 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0a7c136710 0x7f0a7c11afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:04.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.228+0000 7f0a82baf700 1 -- 172.21.15.16:0/2992965119 >> 172.21.15.16:0/2992965119 conn(0x7f0a7c0a2aa0 msgr2=0x7f0a7c1344f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:04.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.228+0000 7f0a82baf700 1 -- 172.21.15.16:0/2992965119 shutdown_connections 2023-09-30T22:35:04.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.228+0000 7f0a82baf700 1 -- 172.21.15.16:0/2992965119 wait complete. 2023-09-30T22:35:04.281 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:35:04.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.259+0000 7fe3b08c9700 1 -- 172.21.15.16:0/2035406350 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe3ac0a28e0 con 0x7fe3ac0a1340 2023-09-30T22:35:04.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.260+0000 7fe3a2ffd700 1 -- 172.21.15.16:0/2035406350 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe394017070 con 0x7fe3ac0a1340 2023-09-30T22:35:04.282 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:35:04.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.262+0000 7fe3b08c9700 1 -- 172.21.15.16:0/2035406350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3ac0a1340 msgr2=0x7fe3ac11ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:04.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.262+0000 7fe3b08c9700 1 --2- 172.21.15.16:0/2035406350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3ac0a1340 0x7fe3ac11ec40 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fe39400c4e0 tx=0x7fe39400c7f0 comp rx=0 tx=0).stop 2023-09-30T22:35:04.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.262+0000 7fe3b08c9700 1 -- 172.21.15.16:0/2035406350 shutdown_connections 2023-09-30T22:35:04.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.262+0000 7fe3b08c9700 1 --2- 172.21.15.16:0/2035406350 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe3ac0a0a30 0x7fe3ac11e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:04.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.262+0000 7fe3b08c9700 1 --2- 172.21.15.16:0/2035406350 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe3ac134170 0x7fe3ac11f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:04.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.262+0000 7fe3b08c9700 1 --2- 172.21.15.16:0/2035406350 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe3ac0a1340 0x7fe3ac11ec40 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:04.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.262+0000 7fe3b08c9700 1 -- 172.21.15.16:0/2035406350 >> 172.21.15.16:0/2035406350 conn(0x7fe3ac0a6550 msgr2=0x7fe3ac09c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:04.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.262+0000 7fe3b08c9700 1 -- 172.21.15.16:0/2035406350 shutdown_connections 2023-09-30T22:35:04.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.262+0000 7fe3b08c9700 1 -- 172.21.15.16:0/2035406350 wait complete. 2023-09-30T22:35:04.284 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:35:04.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.294+0000 7f1d289c4700 1 -- 172.21.15.16:0/1810782439 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1d240747f0 con 0x7f1d2409c650 2023-09-30T22:35:04.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.294+0000 7f1d12ffd700 1 -- 172.21.15.16:0/1810782439 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1d14017070 con 0x7f1d2409c650 2023-09-30T22:35:04.295 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:35:04.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.297+0000 7f1d289c4700 1 -- 172.21.15.16:0/1810782439 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d2409c650 msgr2=0x7f1d2411a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:04.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.297+0000 7f1d289c4700 1 --2- 172.21.15.16:0/1810782439 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d2409c650 0x7f1d2411a620 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f1d1400c700 tx=0x7f1d1400ca10 comp rx=0 tx=0).stop 2023-09-30T22:35:04.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.297+0000 7f1d289c4700 1 -- 172.21.15.16:0/1810782439 shutdown_connections 2023-09-30T22:35:04.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.297+0000 7f1d289c4700 1 --2- 172.21.15.16:0/1810782439 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1d2409c650 0x7f1d2411a620 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:04.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.297+0000 7f1d289c4700 1 --2- 172.21.15.16:0/1810782439 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1d240a0090 0x7f1d2411b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:04.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.297+0000 7f1d289c4700 1 --2- 172.21.15.16:0/1810782439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d2409d130 0x7f1d2411ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:04.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.297+0000 7f1d289c4700 1 -- 172.21.15.16:0/1810782439 >> 172.21.15.16:0/1810782439 conn(0x7f1d240a3f70 msgr2=0x7f1d240a3340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:04.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.297+0000 7f1d289c4700 1 -- 172.21.15.16:0/1810782439 shutdown_connections 2023-09-30T22:35:04.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.297+0000 7f1d289c4700 1 -- 172.21.15.16:0/1810782439 wait complete. 2023-09-30T22:35:04.311 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:35:04.664 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:35:04.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.815+0000 7efca39e1700 1 Processor -- start 2023-09-30T22:35:04.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.815+0000 7efca39e1700 1 -- start start 2023-09-30T22:35:04.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.816+0000 7efca39e1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc9c09d6f0 0x7efc9c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:04.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.816+0000 7efca39e1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efc9c09e000 0x7efc9c130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:04.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.816+0000 7efca39e1700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efc9c09e410 0x7efc9c130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:04.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.816+0000 7efca39e1700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc9c09f520 con 0x7efc9c09e410 2023-09-30T22:35:04.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.816+0000 7efca39e1700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc9c09f130 con 0x7efc9c09e000 2023-09-30T22:35:04.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.816+0000 7efca39e1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efc9c09e930 con 0x7efc9c09d6f0 2023-09-30T22:35:04.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.816+0000 7efca177d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc9c09d6f0 0x7efc9c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:04.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.816+0000 7efca1f7e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efc9c09e410 0x7efc9c130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:04.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.816+0000 7efca0f7c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efc9c09e000 0x7efc9c130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:04.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.816+0000 7efca177d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc9c09d6f0 0x7efc9c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:34168/0 (socket says 172.21.15.16:34168) 2023-09-30T22:35:04.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.816+0000 7efca177d700 1 -- 172.21.15.16:0/3589925146 learned_addr learned my addr 172.21.15.16:0/3589925146 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:04.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.816+0000 7efca0f7c700 1 -- 172.21.15.16:0/3589925146 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc9c09d6f0 msgr2=0x7efc9c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:04.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.816+0000 7efca0f7c700 1 --2- 172.21.15.16:0/3589925146 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc9c09d6f0 0x7efc9c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:04.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.816+0000 7efca0f7c700 1 -- 172.21.15.16:0/3589925146 >> v2:172.21.15.16:3300/0 conn(0x7efc9c09e410 msgr2=0x7efc9c130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:04.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.816+0000 7efca0f7c700 1 --2- 172.21.15.16:0/3589925146 >> v2:172.21.15.16:3300/0 conn(0x7efc9c09e410 0x7efc9c130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:04.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.816+0000 7efca0f7c700 1 -- 172.21.15.16:0/3589925146 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc9c09b2d0 con 0x7efc9c09e000 2023-09-30T22:35:04.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.816+0000 7efca1f7e700 1 --2- 172.21.15.16:0/3589925146 >> v2:172.21.15.16:3300/0 conn(0x7efc9c09e410 0x7efc9c130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:04.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.816+0000 7efca0f7c700 1 --2- 172.21.15.16:0/3589925146 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efc9c09e000 0x7efc9c130660 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7efc98009870 tx=0x7efc98009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddcfc6e470d9df90 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:04.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.816+0000 7efc8ffff700 1 -- 172.21.15.16:0/3589925146 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc98011830 con 0x7efc9c09e000 2023-09-30T22:35:04.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.817+0000 7efc8ffff700 1 -- 172.21.15.16:0/3589925146 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc98011990 con 0x7efc9c09e000 2023-09-30T22:35:04.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.817+0000 7efc8ffff700 1 -- 172.21.15.16:0/3589925146 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc98011d20 con 0x7efc9c09e000 2023-09-30T22:35:04.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.817+0000 7efca39e1700 1 -- 172.21.15.16:0/3589925146 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efc9c09e000 msgr2=0x7efc9c130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:04.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.817+0000 7efca39e1700 1 --2- 172.21.15.16:0/3589925146 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efc9c09e000 0x7efc9c130660 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7efc98009870 tx=0x7efc98009b80 comp rx=0 tx=0).stop 2023-09-30T22:35:04.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.817+0000 7efca39e1700 1 -- 172.21.15.16:0/3589925146 shutdown_connections 2023-09-30T22:35:04.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.817+0000 7efca39e1700 1 --2- 172.21.15.16:0/3589925146 >> v2:172.21.15.16:3300/0 conn(0x7efc9c09e410 0x7efc9c130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:04.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.817+0000 7efca39e1700 1 --2- 172.21.15.16:0/3589925146 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc9c09d6f0 0x7efc9c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:04.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.817+0000 7efca39e1700 1 --2- 172.21.15.16:0/3589925146 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efc9c09e000 0x7efc9c130660 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:04.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.817+0000 7efca39e1700 1 -- 172.21.15.16:0/3589925146 >> 172.21.15.16:0/3589925146 conn(0x7efc9c066f50 msgr2=0x7efc9c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:04.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.817+0000 7efca39e1700 1 -- 172.21.15.16:0/3589925146 shutdown_connections 2023-09-30T22:35:04.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.817+0000 7efca39e1700 1 -- 172.21.15.16:0/3589925146 wait complete. 2023-09-30T22:35:04.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.819+0000 7efca39e1700 1 Processor -- start 2023-09-30T22:35:04.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.819+0000 7efca39e1700 1 -- start start 2023-09-30T22:35:04.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.819+0000 7efca39e1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc9c09d6f0 0x7efc9c1c8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:04.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.819+0000 7efca39e1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efc9c09e000 0x7efc9c1c89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:04.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.819+0000 7efca39e1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc9c09e410 0x7efc9c1c8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:04.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.819+0000 7efca39e1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc9c1210a0 con 0x7efc9c09d6f0 2023-09-30T22:35:04.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.819+0000 7efca39e1700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc9c125d90 con 0x7efc9c09e000 2023-09-30T22:35:04.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.819+0000 7efca39e1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efc9c06b280 con 0x7efc9c09e410 2023-09-30T22:35:04.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.819+0000 7efca177d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc9c09d6f0 0x7efc9c1c8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:04.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.819+0000 7efca177d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc9c09d6f0 0x7efc9c1c8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44058/0 (socket says 172.21.15.16:44058) 2023-09-30T22:35:04.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.819+0000 7efca1f7e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc9c09e410 0x7efc9c1c8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:04.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.819+0000 7efca177d700 1 -- 172.21.15.16:0/714179474 learned_addr learned my addr 172.21.15.16:0/714179474 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:04.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.820+0000 7efca0f7c700 1 --2- 172.21.15.16:0/714179474 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efc9c09e000 0x7efc9c1c89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:04.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.820+0000 7efca177d700 1 -- 172.21.15.16:0/714179474 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc9c09e410 msgr2=0x7efc9c1c8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:04.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.820+0000 7efca177d700 1 --2- 172.21.15.16:0/714179474 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc9c09e410 0x7efc9c1c8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:04.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.820+0000 7efca177d700 1 -- 172.21.15.16:0/714179474 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efc9c09e000 msgr2=0x7efc9c1c89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:04.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.820+0000 7efca177d700 1 --2- 172.21.15.16:0/714179474 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efc9c09e000 0x7efc9c1c89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:04.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.820+0000 7efca177d700 1 -- 172.21.15.16:0/714179474 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc94009a70 con 0x7efc9c09d6f0 2023-09-30T22:35:04.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.820+0000 7efca1f7e700 1 --2- 172.21.15.16:0/714179474 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc9c09e410 0x7efc9c1c8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:04.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.820+0000 7efca0f7c700 1 --2- 172.21.15.16:0/714179474 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efc9c09e000 0x7efc9c1c89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:04.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.820+0000 7efca177d700 1 --2- 172.21.15.16:0/714179474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc9c09d6f0 0x7efc9c1c8480 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7efc9000e990 tx=0x7efc9000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:04.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.820+0000 7efc8dffb700 1 -- 172.21.15.16:0/714179474 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc9000c640 con 0x7efc9c09d6f0 2023-09-30T22:35:04.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.820+0000 7efc8dffb700 1 -- 172.21.15.16:0/714179474 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc9000c7a0 con 0x7efc9c09d6f0 2023-09-30T22:35:04.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.820+0000 7efc8dffb700 1 -- 172.21.15.16:0/714179474 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc9000cb10 con 0x7efc9c09d6f0 2023-09-30T22:35:04.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.820+0000 7efca39e1700 1 -- 172.21.15.16:0/714179474 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc980094a0 con 0x7efc9c09d6f0 2023-09-30T22:35:04.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.821+0000 7efca39e1700 1 -- 172.21.15.16:0/714179474 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc9c09c700 con 0x7efc9c09d6f0 2023-09-30T22:35:04.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.821+0000 7efc8dffb700 1 -- 172.21.15.16:0/714179474 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7efc90013470 con 0x7efc9c09d6f0 2023-09-30T22:35:04.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.822+0000 7efc8dffb700 1 -- 172.21.15.16:0/714179474 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7efc9003f880 con 0x7efc9c09d6f0 2023-09-30T22:35:04.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.822+0000 7efca39e1700 1 -- 172.21.15.16:0/714179474 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc9c074ea0 con 0x7efc9c09d6f0 2023-09-30T22:35:04.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:04.826+0000 7efc8dffb700 1 -- 172.21.15.16:0/714179474 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7efc90021020 con 0x7efc9c09d6f0 2023-09-30T22:35:05.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.030+0000 7efca39e1700 1 -- 172.21.15.16:0/714179474 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7efc9c125880 con 0x7efc9c09d6f0 2023-09-30T22:35:05.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.030+0000 7efc8dffb700 1 -- 172.21.15.16:0/714179474 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7efc90013720 con 0x7efc9c09d6f0 2023-09-30T22:35:05.031 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:35:05.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.032+0000 7efca39e1700 1 -- 172.21.15.16:0/714179474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc9c09d6f0 msgr2=0x7efc9c1c8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:05.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.032+0000 7efca39e1700 1 --2- 172.21.15.16:0/714179474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc9c09d6f0 0x7efc9c1c8480 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7efc9000e990 tx=0x7efc9000eca0 comp rx=0 tx=0).stop 2023-09-30T22:35:05.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.032+0000 7efca39e1700 1 -- 172.21.15.16:0/714179474 shutdown_connections 2023-09-30T22:35:05.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.032+0000 7efca39e1700 1 --2- 172.21.15.16:0/714179474 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc9c09e410 0x7efc9c1c8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:05.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.032+0000 7efca39e1700 1 --2- 172.21.15.16:0/714179474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc9c09d6f0 0x7efc9c1c8480 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:05.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.032+0000 7efca39e1700 1 --2- 172.21.15.16:0/714179474 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efc9c09e000 0x7efc9c1c89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:05.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.032+0000 7efca39e1700 1 -- 172.21.15.16:0/714179474 >> 172.21.15.16:0/714179474 conn(0x7efc9c066f50 msgr2=0x7efc9c119480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:05.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.033+0000 7efca39e1700 1 -- 172.21.15.16:0/714179474 shutdown_connections 2023-09-30T22:35:05.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.033+0000 7efca39e1700 1 -- 172.21.15.16:0/714179474 wait complete. 2023-09-30T22:35:05.046 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:35:05.282 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:35:05.285 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:35:05.312 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:35:05.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.440+0000 7fea9360b700 1 Processor -- start 2023-09-30T22:35:05.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.440+0000 7f24dfaae700 1 Processor -- start 2023-09-30T22:35:05.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.440+0000 7fea9360b700 1 -- start start 2023-09-30T22:35:05.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.441+0000 7f24dfaae700 1 -- start start 2023-09-30T22:35:05.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.441+0000 7fea9360b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fea8c09d630 0x7fea8c0a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:05.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.441+0000 7fea9360b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea8c138d80 0x7fea8c13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:05.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.441+0000 7fea9360b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea8c0a2a70 con 0x7fea8c0a0a30 2023-09-30T22:35:05.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.441+0000 7fea9360b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea8c0a01d0 con 0x7fea8c09d630 2023-09-30T22:35:05.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.441+0000 7fea9360b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea8c09e2b0 con 0x7fea8c138d80 2023-09-30T22:35:05.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.441+0000 7f24dfaae700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24d806f5d0 0x7f24d812ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:05.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.441+0000 7f24dfaae700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f24d8135c20 0x7f24d8138000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:05.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.441+0000 7f24dfaae700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24d806fa40 con 0x7f24d812cf60 2023-09-30T22:35:05.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.441+0000 7fea90ba6700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fea8c09d630 0x7fea8c0a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:05.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.441+0000 7fea91ba8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea8c138d80 0x7fea8c13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:05.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.441+0000 7fea90ba6700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fea8c09d630 0x7fea8c0a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44072/0 (socket says 172.21.15.16:44072) 2023-09-30T22:35:05.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.441+0000 7fea90ba6700 1 -- 172.21.15.16:0/1295760914 learned_addr learned my addr 172.21.15.16:0/1295760914 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:05.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.441+0000 7f24dd84a700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f24d812cf60 0x7f24d812f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50866/0 (socket says 172.21.15.16:50866) 2023-09-30T22:35:05.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.441+0000 7f24dfaae700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24d812cd90 con 0x7f24d8135c20 2023-09-30T22:35:05.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.441+0000 7f24dd84a700 1 -- 172.21.15.16:0/3901108553 learned_addr learned my addr 172.21.15.16:0/3901108553 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:05.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.441+0000 7f24dfaae700 1 -- 172.21.15.16:0/3901108553 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f24d8071780 con 0x7f24d806f5d0 2023-09-30T22:35:05.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.442+0000 7f24dd049700 1 --2- 172.21.15.16:0/3901108553 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24d806f5d0 0x7f24d812ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:05.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.442+0000 7f24de04b700 1 --2- 172.21.15.16:0/3901108553 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f24d8135c20 0x7f24d8138000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:05.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.442+0000 7fea90ba6700 1 -- 172.21.15.16:0/1295760914 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea8c138d80 msgr2=0x7fea8c13b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:05.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.442+0000 7fea90ba6700 1 --2- 172.21.15.16:0/1295760914 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea8c138d80 0x7fea8c13b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:05.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.442+0000 7fea90ba6700 1 -- 172.21.15.16:0/1295760914 >> v1:172.21.15.16:6789/0 conn(0x7fea8c0a0a30 legacy=0x7fea8c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:05.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.442+0000 7fea90ba6700 1 -- 172.21.15.16:0/1295760914 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea8c09dc50 con 0x7fea8c09d630 2023-09-30T22:35:05.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.442+0000 7f24dc848700 1 -- 172.21.15.16:0/3901108553 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2971732335 0 0) 0x7f24d806fa40 con 0x7f24d812cf60 2023-09-30T22:35:05.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.442+0000 7f24dc848700 1 -- 172.21.15.16:0/3901108553 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24c40035b0 con 0x7f24d812cf60 2023-09-30T22:35:05.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.442+0000 7f24dd049700 1 -- 172.21.15.16:0/3901108553 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f24d8135c20 msgr2=0x7f24d8138000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:05.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.442+0000 7f24dd049700 1 --2- 172.21.15.16:0/3901108553 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f24d8135c20 0x7f24d8138000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:05.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.442+0000 7f24dd049700 1 -- 172.21.15.16:0/3901108553 >> v1:172.21.15.16:6789/0 conn(0x7f24d812cf60 legacy=0x7f24d812f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:05.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.442+0000 7f24dd049700 1 -- 172.21.15.16:0/3901108553 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24d8071a40 con 0x7f24d806f5d0 2023-09-30T22:35:05.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.442+0000 7fea90ba6700 1 --2- 172.21.15.16:0/1295760914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea8c09d630 0x7fea8c0a1510 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fea7c0098f0 tx=0x7fea7c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22fc7650cd860bf4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:05.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.442+0000 7f24dd049700 1 --2- 172.21.15.16:0/3901108553 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24d806f5d0 0x7f24d812ffc0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f24c80098a0 tx=0x7f24c8009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f528bdacb6de016 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:05.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.442+0000 7fea83fff700 1 -- 172.21.15.16:0/1295760914 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7c011830 con 0x7fea8c09d630 2023-09-30T22:35:05.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.442+0000 7fea83fff700 1 -- 172.21.15.16:0/1295760914 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea7c011990 con 0x7fea8c09d630 2023-09-30T22:35:05.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.442+0000 7fea83fff700 1 -- 172.21.15.16:0/1295760914 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7c011d20 con 0x7fea8c09d630 2023-09-30T22:35:05.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.443+0000 7f24dc848700 1 -- 172.21.15.16:0/3901108553 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c8011830 con 0x7f24d806f5d0 2023-09-30T22:35:05.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.443+0000 7f24dc848700 1 -- 172.21.15.16:0/3901108553 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24c8011990 con 0x7f24d806f5d0 2023-09-30T22:35:05.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.443+0000 7f24dc848700 1 -- 172.21.15.16:0/3901108553 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c8011d20 con 0x7f24d806f5d0 2023-09-30T22:35:05.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.443+0000 7fea9360b700 1 -- 172.21.15.16:0/1295760914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea8c09d630 msgr2=0x7fea8c0a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:05.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.443+0000 7fea9360b700 1 --2- 172.21.15.16:0/1295760914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea8c09d630 0x7fea8c0a1510 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fea7c0098f0 tx=0x7fea7c009c00 comp rx=0 tx=0).stop 2023-09-30T22:35:05.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.443+0000 7fea9360b700 1 -- 172.21.15.16:0/1295760914 shutdown_connections 2023-09-30T22:35:05.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.443+0000 7fea9360b700 1 --2- 172.21.15.16:0/1295760914 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea8c138d80 0x7fea8c13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:05.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.443+0000 7fea9360b700 1 --2- 172.21.15.16:0/1295760914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea8c09d630 0x7fea8c0a1510 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:05.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.443+0000 7fea9360b700 1 -- 172.21.15.16:0/1295760914 >> 172.21.15.16:0/1295760914 conn(0x7fea8c0a3050 msgr2=0x7fea8c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:05.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.443+0000 7f24dfaae700 1 -- 172.21.15.16:0/3901108553 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24d806f5d0 msgr2=0x7f24d812ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:05.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.443+0000 7f24dfaae700 1 --2- 172.21.15.16:0/3901108553 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24d806f5d0 0x7f24d812ffc0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f24c80098a0 tx=0x7f24c8009bb0 comp rx=0 tx=0).stop 2023-09-30T22:35:05.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.443+0000 7fea9360b700 1 -- 172.21.15.16:0/1295760914 shutdown_connections 2023-09-30T22:35:05.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.443+0000 7fea9360b700 1 -- 172.21.15.16:0/1295760914 wait complete. 2023-09-30T22:35:05.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.443+0000 7f24dfaae700 1 -- 172.21.15.16:0/3901108553 shutdown_connections 2023-09-30T22:35:05.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.443+0000 7f24dfaae700 1 --2- 172.21.15.16:0/3901108553 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f24d8135c20 0x7f24d8138000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:05.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.443+0000 7f24dfaae700 1 --2- 172.21.15.16:0/3901108553 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24d806f5d0 0x7f24d812ffc0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:05.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.443+0000 7f24dfaae700 1 -- 172.21.15.16:0/3901108553 >> 172.21.15.16:0/3901108553 conn(0x7f24d80a2aa0 msgr2=0x7f24d80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:05.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.444+0000 7f24dfaae700 1 -- 172.21.15.16:0/3901108553 shutdown_connections 2023-09-30T22:35:05.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.444+0000 7f24dfaae700 1 -- 172.21.15.16:0/3901108553 wait complete. 2023-09-30T22:35:05.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.445+0000 7fea9360b700 1 Processor -- start 2023-09-30T22:35:05.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.445+0000 7fea9360b700 1 -- start start 2023-09-30T22:35:05.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.445+0000 7f24dfaae700 1 Processor -- start 2023-09-30T22:35:05.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.445+0000 7f24dfaae700 1 -- start start 2023-09-30T22:35:05.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.446+0000 7fea9360b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea8c09d630 0x7fea8c133840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:05.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.446+0000 7fea9360b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea8c0a0a30 0x7fea8c133d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:05.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.446+0000 7fea9360b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea8c138d80 0x7fea8c1342c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:05.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.446+0000 7fea9360b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea8c134d10 con 0x7fea8c138d80 2023-09-30T22:35:05.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.446+0000 7fea9360b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea8c138020 con 0x7fea8c0a0a30 2023-09-30T22:35:05.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.446+0000 7fea9360b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fea8c137470 con 0x7fea8c09d630 2023-09-30T22:35:05.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.446+0000 7f24dfaae700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24d806f5d0 0x7f24d809c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:05.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.446+0000 7f24dfaae700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24d812cf60 0x7f24d809cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:05.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.446+0000 7f24dfaae700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f24d8135c20 0x7f24d809d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:05.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.446+0000 7fea90ba6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea8c0a0a30 0x7fea8c133d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:05.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.446+0000 7fea90ba6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea8c0a0a30 0x7fea8c133d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:52438/0 (socket says 172.21.15.16:52438) 2023-09-30T22:35:05.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.446+0000 7fea90ba6700 1 -- 172.21.15.16:0/2432399178 learned_addr learned my addr 172.21.15.16:0/2432399178 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:05.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.446+0000 7f24dfaae700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24d809e680 con 0x7f24d806f5d0 2023-09-30T22:35:05.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.446+0000 7f24dfaae700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24d809e490 con 0x7f24d8135c20 2023-09-30T22:35:05.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.446+0000 7f24dfaae700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f24d812dd00 con 0x7f24d812cf60 2023-09-30T22:35:05.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.446+0000 7f24dd049700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24d812cf60 0x7f24d809cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:05.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.446+0000 7f24dd84a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24d806f5d0 0x7f24d809c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:05.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.446+0000 7f24dd049700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24d812cf60 0x7f24d809cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:34192/0 (socket says 172.21.15.16:34192) 2023-09-30T22:35:05.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.446+0000 7f24dd049700 1 -- 172.21.15.16:0/3331416379 learned_addr learned my addr 172.21.15.16:0/3331416379 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:05.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.446+0000 7f24dd84a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24d806f5d0 0x7f24d809c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44094/0 (socket says 172.21.15.16:44094) 2023-09-30T22:35:05.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.446+0000 7fea91ba8700 1 --2- 172.21.15.16:0/2432399178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea8c138d80 0x7fea8c1342c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:05.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.446+0000 7fea913a7700 1 --2- 172.21.15.16:0/2432399178 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea8c09d630 0x7fea8c133840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:05.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.446+0000 7fea90ba6700 1 -- 172.21.15.16:0/2432399178 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea8c09d630 msgr2=0x7fea8c133840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:05.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.446+0000 7fea90ba6700 1 --2- 172.21.15.16:0/2432399178 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea8c09d630 0x7fea8c133840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:05.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.446+0000 7fea90ba6700 1 -- 172.21.15.16:0/2432399178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea8c138d80 msgr2=0x7fea8c1342c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:05.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.446+0000 7fea90ba6700 1 --2- 172.21.15.16:0/2432399178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea8c138d80 0x7fea8c1342c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:05.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.446+0000 7fea90ba6700 1 -- 172.21.15.16:0/2432399178 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea7c009550 con 0x7fea8c0a0a30 2023-09-30T22:35:05.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.446+0000 7f24de04b700 1 --2- 172.21.15.16:0/3331416379 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f24d8135c20 0x7f24d809d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:05.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.446+0000 7fea90ba6700 1 --2- 172.21.15.16:0/2432399178 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea8c0a0a30 0x7fea8c133d80 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fea7c009f50 tx=0x7fea7c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:05.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.447+0000 7f24dd84a700 1 -- 172.21.15.16:0/3331416379 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24d812cf60 msgr2=0x7f24d809cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:05.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.447+0000 7f24dd84a700 1 --2- 172.21.15.16:0/3331416379 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24d812cf60 0x7f24d809cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:05.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.447+0000 7f24dd84a700 1 -- 172.21.15.16:0/3331416379 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f24d8135c20 msgr2=0x7f24d809d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:05.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.447+0000 7f24dd84a700 1 --2- 172.21.15.16:0/3331416379 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f24d8135c20 0x7f24d809d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:05.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.447+0000 7f24dd84a700 1 -- 172.21.15.16:0/3331416379 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24c80094a0 con 0x7f24d806f5d0 2023-09-30T22:35:05.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.447+0000 7f24dd049700 1 --2- 172.21.15.16:0/3331416379 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24d812cf60 0x7f24d809cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:05.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.447+0000 7f24dd84a700 1 --2- 172.21.15.16:0/3331416379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24d806f5d0 0x7f24d809c880 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f24d400c4c0 tx=0x7f24d400c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:05.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.447+0000 7fea81ffb700 1 -- 172.21.15.16:0/2432399178 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7c0036a0 con 0x7fea8c0a0a30 2023-09-30T22:35:05.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.447+0000 7fea81ffb700 1 -- 172.21.15.16:0/2432399178 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea7c003800 con 0x7fea8c0a0a30 2023-09-30T22:35:05.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.447+0000 7fea9360b700 1 -- 172.21.15.16:0/2432399178 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea8c0a0410 con 0x7fea8c0a0a30 2023-09-30T22:35:05.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.447+0000 7fea81ffb700 1 -- 172.21.15.16:0/2432399178 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7c003b90 con 0x7fea8c0a0a30 2023-09-30T22:35:05.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.447+0000 7f24ce7fc700 1 -- 172.21.15.16:0/3331416379 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d4005d70 con 0x7f24d806f5d0 2023-09-30T22:35:05.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.447+0000 7f24ce7fc700 1 -- 172.21.15.16:0/3331416379 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24d40070d0 con 0x7f24d806f5d0 2023-09-30T22:35:05.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.447+0000 7f24dfaae700 1 -- 172.21.15.16:0/3331416379 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24d8129fa0 con 0x7f24d806f5d0 2023-09-30T22:35:05.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.447+0000 7f24ce7fc700 1 -- 172.21.15.16:0/3331416379 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24d4007440 con 0x7f24d806f5d0 2023-09-30T22:35:05.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.447+0000 7f24dfaae700 1 -- 172.21.15.16:0/3331416379 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24d80a0f70 con 0x7f24d806f5d0 2023-09-30T22:35:05.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.447+0000 7fea9360b700 1 -- 172.21.15.16:0/2432399178 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea8c137080 con 0x7fea8c0a0a30 2023-09-30T22:35:05.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.449+0000 7f24dfaae700 1 -- 172.21.15.16:0/3331416379 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24d812e700 con 0x7f24d806f5d0 2023-09-30T22:35:05.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.449+0000 7fea81ffb700 1 -- 172.21.15.16:0/2432399178 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fea7c029070 con 0x7fea8c0a0a30 2023-09-30T22:35:05.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.449+0000 7f24ce7fc700 1 -- 172.21.15.16:0/3331416379 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f24d40075a0 con 0x7f24d806f5d0 2023-09-30T22:35:05.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.449+0000 7fea777fe700 1 -- 172.21.15.16:0/2432399178 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea6c0052e0 con 0x7fea8c0a0a30 2023-09-30T22:35:05.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.449+0000 7fea81ffb700 1 -- 172.21.15.16:0/2432399178 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fea7c049840 con 0x7fea8c0a0a30 2023-09-30T22:35:05.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.449+0000 7f24ce7fc700 1 -- 172.21.15.16:0/3331416379 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f24d403edd0 con 0x7f24d806f5d0 2023-09-30T22:35:05.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.454+0000 7fea81ffb700 1 -- 172.21.15.16:0/2432399178 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fea7c015070 con 0x7fea8c0a0a30 2023-09-30T22:35:05.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.457+0000 7f24ce7fc700 1 -- 172.21.15.16:0/3331416379 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f24d4019910 con 0x7f24d806f5d0 2023-09-30T22:35:05.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.479+0000 7f988342b700 1 Processor -- start 2023-09-30T22:35:05.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.479+0000 7f988342b700 1 -- start start 2023-09-30T22:35:05.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.480+0000 7f988342b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f987c1337f0 0x7f987c138880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:05.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.480+0000 7f988342b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f987c138dc0 0x7f987c13b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:05.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.480+0000 7f988342b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f987c06b990 con 0x7f987c133420 2023-09-30T22:35:05.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.480+0000 7f988342b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f987c09b0e0 con 0x7f987c138dc0 2023-09-30T22:35:05.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.480+0000 7f988342b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f987c133c20 con 0x7f987c1337f0 2023-09-30T22:35:05.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.480+0000 7f98809c6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f987c1337f0 0x7f987c138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:05.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.480+0000 7f98809c6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f987c1337f0 0x7f987c138880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:34208/0 (socket says 172.21.15.16:34208) 2023-09-30T22:35:05.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.480+0000 7f98809c6700 1 -- 172.21.15.16:0/3571360373 learned_addr learned my addr 172.21.15.16:0/3571360373 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:05.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.480+0000 7f98819c8700 1 --2- 172.21.15.16:0/3571360373 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f987c138dc0 0x7f987c13b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:05.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.480+0000 7f98809c6700 1 -- 172.21.15.16:0/3571360373 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f987c138dc0 msgr2=0x7f987c13b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:05.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.480+0000 7f98809c6700 1 --2- 172.21.15.16:0/3571360373 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f987c138dc0 0x7f987c13b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:05.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.480+0000 7f98809c6700 1 -- 172.21.15.16:0/3571360373 >> v1:172.21.15.16:6789/0 conn(0x7f987c133420 legacy=0x7f987c133e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:05.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.480+0000 7f98809c6700 1 -- 172.21.15.16:0/3571360373 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f987c0a0590 con 0x7f987c1337f0 2023-09-30T22:35:05.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.480+0000 7f98809c6700 1 --2- 172.21.15.16:0/3571360373 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f987c1337f0 0x7f987c138880 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f986c009920 tx=0x7f986c009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=608375d12b4baca8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:05.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.481+0000 7f9873fff700 1 -- 172.21.15.16:0/3571360373 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986c011830 con 0x7f987c1337f0 2023-09-30T22:35:05.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.481+0000 7f9873fff700 1 -- 172.21.15.16:0/3571360373 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f986c011990 con 0x7f987c1337f0 2023-09-30T22:35:05.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.481+0000 7f9873fff700 1 -- 172.21.15.16:0/3571360373 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986c011d20 con 0x7f987c1337f0 2023-09-30T22:35:05.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.481+0000 7f988342b700 1 -- 172.21.15.16:0/3571360373 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f987c1337f0 msgr2=0x7f987c138880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:05.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.481+0000 7f988342b700 1 --2- 172.21.15.16:0/3571360373 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f987c1337f0 0x7f987c138880 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f986c009920 tx=0x7f986c009c30 comp rx=0 tx=0).stop 2023-09-30T22:35:05.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.481+0000 7f988342b700 1 -- 172.21.15.16:0/3571360373 shutdown_connections 2023-09-30T22:35:05.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.481+0000 7f988342b700 1 --2- 172.21.15.16:0/3571360373 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f987c138dc0 0x7f987c13b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:05.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.481+0000 7f988342b700 1 --2- 172.21.15.16:0/3571360373 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f987c1337f0 0x7f987c138880 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:05.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.481+0000 7f988342b700 1 -- 172.21.15.16:0/3571360373 >> 172.21.15.16:0/3571360373 conn(0x7f987c0a4210 msgr2=0x7f987c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:05.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.482+0000 7f988342b700 1 -- 172.21.15.16:0/3571360373 shutdown_connections 2023-09-30T22:35:05.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.482+0000 7f988342b700 1 -- 172.21.15.16:0/3571360373 wait complete. 2023-09-30T22:35:05.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.482+0000 7f988342b700 1 Processor -- start 2023-09-30T22:35:05.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.483+0000 7f988342b700 1 -- start start 2023-09-30T22:35:05.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.483+0000 7f988342b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f987c133420 0x7f987c12f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:05.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.483+0000 7f988342b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f987c1337f0 0x7f987c12f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:05.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.483+0000 7f988342b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f987c138dc0 0x7f987c12fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:05.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.483+0000 7f988342b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f987c130a90 con 0x7f987c1337f0 2023-09-30T22:35:05.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.483+0000 7f988342b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f987c130cc0 con 0x7f987c138dc0 2023-09-30T22:35:05.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.483+0000 7f988342b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f987c09c700 con 0x7f987c133420 2023-09-30T22:35:05.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.483+0000 7f98809c6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f987c1337f0 0x7f987c12f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:05.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.483+0000 7f98811c7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f987c133420 0x7f987c12f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:05.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.483+0000 7f98809c6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f987c1337f0 0x7f987c12f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44108/0 (socket says 172.21.15.16:44108) 2023-09-30T22:35:05.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.483+0000 7f98809c6700 1 -- 172.21.15.16:0/4254507021 learned_addr learned my addr 172.21.15.16:0/4254507021 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:05.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.483+0000 7f98819c8700 1 --2- 172.21.15.16:0/4254507021 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f987c138dc0 0x7f987c12fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:05.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.483+0000 7f98811c7700 1 -- 172.21.15.16:0/4254507021 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f987c138dc0 msgr2=0x7f987c12fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:05.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.484+0000 7f98811c7700 1 --2- 172.21.15.16:0/4254507021 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f987c138dc0 0x7f987c12fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:05.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.484+0000 7f98811c7700 1 -- 172.21.15.16:0/4254507021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f987c1337f0 msgr2=0x7f987c12f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:05.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.484+0000 7f98811c7700 1 --2- 172.21.15.16:0/4254507021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f987c1337f0 0x7f987c12f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:05.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.484+0000 7f98811c7700 1 -- 172.21.15.16:0/4254507021 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f986c009550 con 0x7f987c133420 2023-09-30T22:35:05.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.484+0000 7f98819c8700 1 --2- 172.21.15.16:0/4254507021 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f987c138dc0 0x7f987c12fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:05.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.484+0000 7f98809c6700 1 --2- 172.21.15.16:0/4254507021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f987c1337f0 0x7f987c12f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:05.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.484+0000 7f98811c7700 1 --2- 172.21.15.16:0/4254507021 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f987c133420 0x7f987c12f0b0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f9878000f60 tx=0x7f987800c730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:05.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.484+0000 7f9871ffb700 1 -- 172.21.15.16:0/4254507021 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98780052d0 con 0x7f987c133420 2023-09-30T22:35:05.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.484+0000 7f9871ffb700 1 -- 172.21.15.16:0/4254507021 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9878005430 con 0x7f987c133420 2023-09-30T22:35:05.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.484+0000 7f988342b700 1 -- 172.21.15.16:0/4254507021 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f987c138930 con 0x7f987c133420 2023-09-30T22:35:05.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.484+0000 7f9871ffb700 1 -- 172.21.15.16:0/4254507021 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98780038c0 con 0x7f987c133420 2023-09-30T22:35:05.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.484+0000 7f988342b700 1 -- 172.21.15.16:0/4254507021 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f987c09e290 con 0x7f987c133420 2023-09-30T22:35:05.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.485+0000 7f9871ffb700 1 -- 172.21.15.16:0/4254507021 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9878003a20 con 0x7f987c133420 2023-09-30T22:35:05.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.485+0000 7f988342b700 1 -- 172.21.15.16:0/4254507021 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f987c09cba0 con 0x7f987c133420 2023-09-30T22:35:05.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.485+0000 7f9871ffb700 1 -- 172.21.15.16:0/4254507021 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f987803ead0 con 0x7f987c133420 2023-09-30T22:35:05.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.490+0000 7f9871ffb700 1 -- 172.21.15.16:0/4254507021 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f987800ce30 con 0x7f987c133420 2023-09-30T22:35:05.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.696+0000 7f988342b700 1 -- 172.21.15.16:0/4254507021 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f987c074ea0 con 0x7f987c133420 2023-09-30T22:35:05.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.697+0000 7f9871ffb700 1 -- 172.21.15.16:0/4254507021 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9878017090 con 0x7f987c133420 2023-09-30T22:35:05.697 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:35:05.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.699+0000 7f988342b700 1 -- 172.21.15.16:0/4254507021 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f987c133420 msgr2=0x7f987c12f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:05.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.699+0000 7f988342b700 1 --2- 172.21.15.16:0/4254507021 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f987c133420 0x7f987c12f0b0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f9878000f60 tx=0x7f987800c730 comp rx=0 tx=0).stop 2023-09-30T22:35:05.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.699+0000 7f988342b700 1 -- 172.21.15.16:0/4254507021 shutdown_connections 2023-09-30T22:35:05.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.699+0000 7f988342b700 1 --2- 172.21.15.16:0/4254507021 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f987c138dc0 0x7f987c12fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:05.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.699+0000 7f988342b700 1 --2- 172.21.15.16:0/4254507021 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f987c133420 0x7f987c12f0b0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:05.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.699+0000 7f988342b700 1 --2- 172.21.15.16:0/4254507021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f987c1337f0 0x7f987c12f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:05.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.699+0000 7f988342b700 1 -- 172.21.15.16:0/4254507021 >> 172.21.15.16:0/4254507021 conn(0x7f987c0a4210 msgr2=0x7f987c131400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:05.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.700+0000 7f988342b700 1 -- 172.21.15.16:0/4254507021 shutdown_connections 2023-09-30T22:35:05.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.700+0000 7f988342b700 1 -- 172.21.15.16:0/4254507021 wait complete. 2023-09-30T22:35:05.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.704+0000 7f24dfaae700 1 -- 172.21.15.16:0/3331416379 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f24d812e700 con 0x7f24d806f5d0 2023-09-30T22:35:05.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.704+0000 7fea777fe700 1 -- 172.21.15.16:0/2432399178 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fea6c005590 con 0x7fea8c0a0a30 2023-09-30T22:35:05.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.704+0000 7f24ce7fc700 1 -- 172.21.15.16:0/3331416379 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f24d812e700 con 0x7f24d806f5d0 2023-09-30T22:35:05.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.704+0000 7fea81ffb700 1 -- 172.21.15.16:0/2432399178 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fea7c010380 con 0x7fea8c0a0a30 2023-09-30T22:35:05.705 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:35:05.705 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:35:05.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.707+0000 7fea777fe700 1 -- 172.21.15.16:0/2432399178 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea8c0a0a30 msgr2=0x7fea8c133d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:05.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.707+0000 7fea777fe700 1 --2- 172.21.15.16:0/2432399178 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea8c0a0a30 0x7fea8c133d80 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fea7c009f50 tx=0x7fea7c004750 comp rx=0 tx=0).stop 2023-09-30T22:35:05.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.707+0000 7f24dfaae700 1 -- 172.21.15.16:0/3331416379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24d806f5d0 msgr2=0x7f24d809c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:05.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.707+0000 7f24dfaae700 1 --2- 172.21.15.16:0/3331416379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24d806f5d0 0x7f24d809c880 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f24d400c4c0 tx=0x7f24d400c7d0 comp rx=0 tx=0).stop 2023-09-30T22:35:05.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.707+0000 7fea777fe700 1 -- 172.21.15.16:0/2432399178 shutdown_connections 2023-09-30T22:35:05.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.707+0000 7fea777fe700 1 --2- 172.21.15.16:0/2432399178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea8c138d80 0x7fea8c1342c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:05.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.707+0000 7fea777fe700 1 --2- 172.21.15.16:0/2432399178 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea8c09d630 0x7fea8c133840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:05.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.707+0000 7fea777fe700 1 --2- 172.21.15.16:0/2432399178 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea8c0a0a30 0x7fea8c133d80 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:05.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.707+0000 7fea777fe700 1 -- 172.21.15.16:0/2432399178 >> 172.21.15.16:0/2432399178 conn(0x7fea8c0a3050 msgr2=0x7fea8c09ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:05.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.707+0000 7f24dfaae700 1 -- 172.21.15.16:0/3331416379 shutdown_connections 2023-09-30T22:35:05.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.707+0000 7f24dfaae700 1 --2- 172.21.15.16:0/3331416379 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f24d8135c20 0x7f24d809d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:05.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.707+0000 7f24dfaae700 1 --2- 172.21.15.16:0/3331416379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24d806f5d0 0x7f24d809c880 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:05.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.707+0000 7f24dfaae700 1 --2- 172.21.15.16:0/3331416379 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24d812cf60 0x7f24d809cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:05.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.707+0000 7fea777fe700 1 -- 172.21.15.16:0/2432399178 shutdown_connections 2023-09-30T22:35:05.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.707+0000 7f24dfaae700 1 -- 172.21.15.16:0/3331416379 >> 172.21.15.16:0/3331416379 conn(0x7f24d80a2aa0 msgr2=0x7f24d806e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:05.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.707+0000 7fea777fe700 1 -- 172.21.15.16:0/2432399178 wait complete. 2023-09-30T22:35:05.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.707+0000 7f24dfaae700 1 -- 172.21.15.16:0/3331416379 shutdown_connections 2023-09-30T22:35:05.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:05.707+0000 7f24dfaae700 1 -- 172.21.15.16:0/3331416379 wait complete. 2023-09-30T22:35:05.719 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:35:05.723 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:35:05.725 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:35:06.048 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:35:06.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.193+0000 7fae83513700 1 Processor -- start 2023-09-30T22:35:06.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.193+0000 7fae83513700 1 -- start start 2023-09-30T22:35:06.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.194+0000 7fae83513700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fae7c133420 0x7fae7c1337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:06.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.194+0000 7fae83513700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fae7c134260 0x7fae7c1306a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:06.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.194+0000 7fae83513700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae7c06b990 con 0x7fae7c133e30 2023-09-30T22:35:06.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.194+0000 7fae83513700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae7c0a0590 con 0x7fae7c133420 2023-09-30T22:35:06.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.194+0000 7fae83513700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fae7c09fb80 con 0x7fae7c134260 2023-09-30T22:35:06.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.194+0000 7fae812af700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fae7c133420 0x7fae7c1337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:06.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.194+0000 7fae81ab0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fae7c134260 0x7fae7c1306a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:06.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.194+0000 7fae81ab0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fae7c134260 0x7fae7c1306a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:34224/0 (socket says 172.21.15.16:34224) 2023-09-30T22:35:06.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.194+0000 7fae812af700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fae7c133420 0x7fae7c1337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:52478/0 (socket says 172.21.15.16:52478) 2023-09-30T22:35:06.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.194+0000 7fae81ab0700 1 -- 172.21.15.16:0/4179431258 learned_addr learned my addr 172.21.15.16:0/4179431258 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:06.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.194+0000 7fae80aae700 1 --1- 172.21.15.16:0/4179431258 >> v1:172.21.15.16:6789/0 conn(0x7fae7c133e30 0x7fae7c12ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50890/0 (socket says 172.21.15.16:50890) 2023-09-30T22:35:06.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.194+0000 7fae81ab0700 1 -- 172.21.15.16:0/4179431258 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fae7c133420 msgr2=0x7fae7c1337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:06.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.194+0000 7fae81ab0700 1 --2- 172.21.15.16:0/4179431258 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fae7c133420 0x7fae7c1337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:06.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.194+0000 7fae81ab0700 1 -- 172.21.15.16:0/4179431258 >> v1:172.21.15.16:6789/0 conn(0x7fae7c133e30 legacy=0x7fae7c12ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:06.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.194+0000 7fae81ab0700 1 -- 172.21.15.16:0/4179431258 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae7c0a0030 con 0x7fae7c134260 2023-09-30T22:35:06.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.194+0000 7fae6ffff700 1 -- 172.21.15.16:0/4179431258 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 49242444 0 0) 0x7fae7c06b990 con 0x7fae7c133e30 2023-09-30T22:35:06.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.194+0000 7fae812af700 1 --2- 172.21.15.16:0/4179431258 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fae7c133420 0x7fae7c1337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:06.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.194+0000 7fae81ab0700 1 --2- 172.21.15.16:0/4179431258 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fae7c134260 0x7fae7c1306a0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fae74009e80 tx=0x7fae7400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77fe87eaef224e42 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:06.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.195+0000 7fae6ffff700 1 -- 172.21.15.16:0/4179431258 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7400e070 con 0x7fae7c134260 2023-09-30T22:35:06.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.195+0000 7fae6ffff700 1 -- 172.21.15.16:0/4179431258 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae7400b910 con 0x7fae7c134260 2023-09-30T22:35:06.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.195+0000 7fae6ffff700 1 -- 172.21.15.16:0/4179431258 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae7400bcc0 con 0x7fae7c134260 2023-09-30T22:35:06.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.195+0000 7fae83513700 1 -- 172.21.15.16:0/4179431258 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fae7c134260 msgr2=0x7fae7c1306a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:06.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.195+0000 7fae83513700 1 --2- 172.21.15.16:0/4179431258 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fae7c134260 0x7fae7c1306a0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fae74009e80 tx=0x7fae7400b330 comp rx=0 tx=0).stop 2023-09-30T22:35:06.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.195+0000 7fae83513700 1 -- 172.21.15.16:0/4179431258 shutdown_connections 2023-09-30T22:35:06.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.195+0000 7fae83513700 1 --2- 172.21.15.16:0/4179431258 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fae7c134260 0x7fae7c1306a0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:06.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.195+0000 7fae83513700 1 --2- 172.21.15.16:0/4179431258 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fae7c133420 0x7fae7c1337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:06.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.195+0000 7fae83513700 1 -- 172.21.15.16:0/4179431258 >> 172.21.15.16:0/4179431258 conn(0x7fae7c0a4210 msgr2=0x7fae7c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:06.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.195+0000 7fae83513700 1 -- 172.21.15.16:0/4179431258 shutdown_connections 2023-09-30T22:35:06.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.195+0000 7fae83513700 1 -- 172.21.15.16:0/4179431258 wait complete. 2023-09-30T22:35:06.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.196+0000 7fae83513700 1 Processor -- start 2023-09-30T22:35:06.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.196+0000 7fae83513700 1 -- start start 2023-09-30T22:35:06.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.197+0000 7fae83513700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae7c133420 0x7fae7c09c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:06.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.197+0000 7fae83513700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fae7c133e30 0x7fae7c09c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:06.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.197+0000 7fae83513700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fae7c134260 0x7fae7c09cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:06.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.197+0000 7fae83513700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae7c09de30 con 0x7fae7c133420 2023-09-30T22:35:06.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.197+0000 7fae83513700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae7c11edf0 con 0x7fae7c133e30 2023-09-30T22:35:06.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.197+0000 7fae83513700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fae7c11faf0 con 0x7fae7c134260 2023-09-30T22:35:06.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.197+0000 7fae80aae700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fae7c133e30 0x7fae7c09c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:06.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.197+0000 7fae81ab0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fae7c134260 0x7fae7c09cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:06.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.197+0000 7fae80aae700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fae7c133e30 0x7fae7c09c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:52494/0 (socket says 172.21.15.16:52494) 2023-09-30T22:35:06.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.197+0000 7fae80aae700 1 -- 172.21.15.16:0/898265488 learned_addr learned my addr 172.21.15.16:0/898265488 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:06.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.197+0000 7fae812af700 1 --2- 172.21.15.16:0/898265488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae7c133420 0x7fae7c09c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:06.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.197+0000 7fae80aae700 1 -- 172.21.15.16:0/898265488 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fae7c134260 msgr2=0x7fae7c09cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:06.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.197+0000 7fae80aae700 1 --2- 172.21.15.16:0/898265488 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fae7c134260 0x7fae7c09cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:06.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.197+0000 7fae80aae700 1 -- 172.21.15.16:0/898265488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae7c133420 msgr2=0x7fae7c09c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:06.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.197+0000 7fae80aae700 1 --2- 172.21.15.16:0/898265488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae7c133420 0x7fae7c09c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:06.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.197+0000 7fae80aae700 1 -- 172.21.15.16:0/898265488 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae780094a0 con 0x7fae7c133e30 2023-09-30T22:35:06.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.197+0000 7fae81ab0700 1 --2- 172.21.15.16:0/898265488 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fae7c134260 0x7fae7c09cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:06.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.197+0000 7fae80aae700 1 --2- 172.21.15.16:0/898265488 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fae7c133e30 0x7fae7c09c9c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fae70001eb0 tx=0x7fae7000c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:06.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.198+0000 7fae6dffb700 1 -- 172.21.15.16:0/898265488 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae70004c80 con 0x7fae7c133e30 2023-09-30T22:35:06.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.198+0000 7fae6dffb700 1 -- 172.21.15.16:0/898265488 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae70004de0 con 0x7fae7c133e30 2023-09-30T22:35:06.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.198+0000 7fae83513700 1 -- 172.21.15.16:0/898265488 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae74009b30 con 0x7fae7c133e30 2023-09-30T22:35:06.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.198+0000 7fae6dffb700 1 -- 172.21.15.16:0/898265488 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae70007310 con 0x7fae7c133e30 2023-09-30T22:35:06.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.198+0000 7fae83513700 1 -- 172.21.15.16:0/898265488 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae7c11fdb0 con 0x7fae7c133e30 2023-09-30T22:35:06.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.198+0000 7fae6dffb700 1 -- 172.21.15.16:0/898265488 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fae70007470 con 0x7fae7c133e30 2023-09-30T22:35:06.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.199+0000 7fae6dffb700 1 -- 172.21.15.16:0/898265488 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fae7003f2e0 con 0x7fae7c133e30 2023-09-30T22:35:06.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.199+0000 7fae83513700 1 -- 172.21.15.16:0/898265488 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae7c074ea0 con 0x7fae7c133e30 2023-09-30T22:35:06.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.204+0000 7fae6dffb700 1 -- 172.21.15.16:0/898265488 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fae700123b0 con 0x7fae7c133e30 2023-09-30T22:35:06.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.409+0000 7fae83513700 1 -- 172.21.15.16:0/898265488 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fae7c120880 con 0x7fae7c133e30 2023-09-30T22:35:06.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.410+0000 7fae6dffb700 1 -- 172.21.15.16:0/898265488 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fae700102c0 con 0x7fae7c133e30 2023-09-30T22:35:06.411 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:35:06.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.412+0000 7fae83513700 1 -- 172.21.15.16:0/898265488 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fae7c133e30 msgr2=0x7fae7c09c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:06.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.412+0000 7fae83513700 1 --2- 172.21.15.16:0/898265488 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fae7c133e30 0x7fae7c09c9c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fae70001eb0 tx=0x7fae7000c800 comp rx=0 tx=0).stop 2023-09-30T22:35:06.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.412+0000 7fae83513700 1 -- 172.21.15.16:0/898265488 shutdown_connections 2023-09-30T22:35:06.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.412+0000 7fae83513700 1 --2- 172.21.15.16:0/898265488 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fae7c134260 0x7fae7c09cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:06.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.412+0000 7fae83513700 1 --2- 172.21.15.16:0/898265488 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fae7c133420 0x7fae7c09c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:06.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.412+0000 7fae83513700 1 --2- 172.21.15.16:0/898265488 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fae7c133e30 0x7fae7c09c9c0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:06.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.412+0000 7fae83513700 1 -- 172.21.15.16:0/898265488 >> 172.21.15.16:0/898265488 conn(0x7fae7c0a4210 msgr2=0x7fae7c0a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:06.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.412+0000 7fae83513700 1 -- 172.21.15.16:0/898265488 shutdown_connections 2023-09-30T22:35:06.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.412+0000 7fae83513700 1 -- 172.21.15.16:0/898265488 wait complete. 2023-09-30T22:35:06.428 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:35:06.721 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:35:06.724 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:35:06.726 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:35:06.875 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.875+0000 7f7e35abb700 1 Processor -- start 2023-09-30T22:35:06.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.875+0000 7f94f5564700 1 Processor -- start 2023-09-30T22:35:06.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.875+0000 7f7e35abb700 1 -- start start 2023-09-30T22:35:06.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.875+0000 7f94f5564700 1 -- start start 2023-09-30T22:35:06.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.876+0000 7f94f5564700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f94f009d6f0 0x7f94f009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:06.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.876+0000 7f94f5564700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f94f009e000 0x7f94f0130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:06.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.876+0000 7f94f5564700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94f009f130 con 0x7f94f009e410 2023-09-30T22:35:06.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.876+0000 7f94f5564700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94f009f520 con 0x7f94f009d6f0 2023-09-30T22:35:06.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.876+0000 7f94f5564700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f94f009e930 con 0x7f94f009e000 2023-09-30T22:35:06.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.876+0000 7f94ef7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f94f009e410 0x7f94f0130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50906/0 (socket says 172.21.15.16:50906) 2023-09-30T22:35:06.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.876+0000 7f94ef7fe700 1 -- 172.21.15.16:0/1925080290 learned_addr learned my addr 172.21.15.16:0/1925080290 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:06.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.876+0000 7f7e35abb700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7e3009e1d0 0x7f7e30135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:06.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.876+0000 7f7e35abb700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7e30135e50 0x7f7e30138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:06.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.876+0000 7f7e35abb700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e3009f130 con 0x7f7e3009d6f0 2023-09-30T22:35:06.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.876+0000 7f7e35abb700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e3009f520 con 0x7f7e3009e1d0 2023-09-30T22:35:06.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.876+0000 7f7e35abb700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3009e930 con 0x7f7e30135e50 2023-09-30T22:35:06.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.876+0000 7f7e2ffff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7e30135e50 0x7f7e30138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:06.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.876+0000 7f7e2ffff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7e30135e50 0x7f7e30138230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:34238/0 (socket says 172.21.15.16:34238) 2023-09-30T22:35:06.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.876+0000 7f7e2ffff700 1 -- 172.21.15.16:0/110916074 learned_addr learned my addr 172.21.15.16:0/110916074 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:06.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.876+0000 7f7e2f7fe700 1 --1- 172.21.15.16:0/110916074 >> v1:172.21.15.16:6789/0 conn(0x7f7e3009d6f0 0x7f7e3009dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:50902/0 (socket says 172.21.15.16:50902) 2023-09-30T22:35:06.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.876+0000 7f94ee7fc700 1 --2- 172.21.15.16:0/1925080290 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f94f009e000 0x7f94f0130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:06.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.876+0000 7f94eeffd700 1 --2- 172.21.15.16:0/1925080290 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f94f009d6f0 0x7f94f009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:06.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.876+0000 7f7e2effd700 1 --2- 172.21.15.16:0/110916074 >> v2:172.21.15.16:3300/0 conn(0x7f7e3009e1d0 0x7f7e30135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:06.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.876+0000 7f94ee7fc700 1 -- 172.21.15.16:0/1925080290 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f94f009d6f0 msgr2=0x7f94f009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:06.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.876+0000 7f94ee7fc700 1 --2- 172.21.15.16:0/1925080290 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f94f009d6f0 0x7f94f009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:06.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.876+0000 7f94ee7fc700 1 -- 172.21.15.16:0/1925080290 >> v1:172.21.15.16:6789/0 conn(0x7f94f009e410 legacy=0x7f94f0130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:06.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.876+0000 7f94ee7fc700 1 -- 172.21.15.16:0/1925080290 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f009b2d0 con 0x7f94f009e000 2023-09-30T22:35:06.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.876+0000 7f94eeffd700 1 --2- 172.21.15.16:0/1925080290 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f94f009d6f0 0x7f94f009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:06.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.876+0000 7f7e2effd700 1 -- 172.21.15.16:0/110916074 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7e30135e50 msgr2=0x7f7e30138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:06.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.876+0000 7f7e2effd700 1 --2- 172.21.15.16:0/110916074 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7e30135e50 0x7f7e30138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:06.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.877+0000 7f7e2effd700 1 -- 172.21.15.16:0/110916074 >> v1:172.21.15.16:6789/0 conn(0x7f7e3009d6f0 legacy=0x7f7e3009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:06.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.877+0000 7f7e2effd700 1 -- 172.21.15.16:0/110916074 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e3009b2d0 con 0x7f7e3009e1d0 2023-09-30T22:35:06.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.877+0000 7f7e2effd700 1 --2- 172.21.15.16:0/110916074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e3009e1d0 0x7f7e30135910 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f7e24009840 tx=0x7f7e24009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a627ba285bc0ff2f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:06.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.877+0000 7f94ee7fc700 1 --2- 172.21.15.16:0/1925080290 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f94f009e000 0x7f94f0130660 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f94e400bb40 tx=0x7f94e400be50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70bd62a7003bb529 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:06.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.877+0000 7f94edffb700 1 -- 172.21.15.16:0/1925080290 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e400dc30 con 0x7f94f009e000 2023-09-30T22:35:06.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.877+0000 7f94edffb700 1 -- 172.21.15.16:0/1925080290 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94e400dd90 con 0x7f94f009e000 2023-09-30T22:35:06.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.877+0000 7f94edffb700 1 -- 172.21.15.16:0/1925080290 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e4010690 con 0x7f94f009e000 2023-09-30T22:35:06.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.877+0000 7f7e2e7fc700 1 -- 172.21.15.16:0/110916074 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e24011830 con 0x7f7e3009e1d0 2023-09-30T22:35:06.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.877+0000 7f7e2e7fc700 1 -- 172.21.15.16:0/110916074 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e24011990 con 0x7f7e3009e1d0 2023-09-30T22:35:06.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.877+0000 7f7e2e7fc700 1 -- 172.21.15.16:0/110916074 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e24011d20 con 0x7f7e3009e1d0 2023-09-30T22:35:06.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.878+0000 7f94f5564700 1 -- 172.21.15.16:0/1925080290 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f94f009e000 msgr2=0x7f94f0130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:06.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.878+0000 7f94f5564700 1 --2- 172.21.15.16:0/1925080290 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f94f009e000 0x7f94f0130660 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f94e400bb40 tx=0x7f94e400be50 comp rx=0 tx=0).stop 2023-09-30T22:35:06.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.878+0000 7f94f5564700 1 -- 172.21.15.16:0/1925080290 shutdown_connections 2023-09-30T22:35:06.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.878+0000 7f94f5564700 1 --2- 172.21.15.16:0/1925080290 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f94f009d6f0 0x7f94f009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:06.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.878+0000 7f94f5564700 1 --2- 172.21.15.16:0/1925080290 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f94f009e000 0x7f94f0130660 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:06.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.878+0000 7f94f5564700 1 -- 172.21.15.16:0/1925080290 >> 172.21.15.16:0/1925080290 conn(0x7f94f0066f50 msgr2=0x7f94f009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:06.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.878+0000 7f94f5564700 1 -- 172.21.15.16:0/1925080290 shutdown_connections 2023-09-30T22:35:06.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.878+0000 7f94f5564700 1 -- 172.21.15.16:0/1925080290 wait complete. 2023-09-30T22:35:06.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.878+0000 7f7e35abb700 1 -- 172.21.15.16:0/110916074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e3009e1d0 msgr2=0x7f7e30135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:06.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.878+0000 7f7e35abb700 1 --2- 172.21.15.16:0/110916074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e3009e1d0 0x7f7e30135910 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f7e24009840 tx=0x7f7e24009b50 comp rx=0 tx=0).stop 2023-09-30T22:35:06.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.878+0000 7f7e35abb700 1 -- 172.21.15.16:0/110916074 shutdown_connections 2023-09-30T22:35:06.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.878+0000 7f7e35abb700 1 --2- 172.21.15.16:0/110916074 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7e30135e50 0x7f7e30138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:06.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.878+0000 7f7e35abb700 1 --2- 172.21.15.16:0/110916074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e3009e1d0 0x7f7e30135910 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:06.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.878+0000 7f7e35abb700 1 -- 172.21.15.16:0/110916074 >> 172.21.15.16:0/110916074 conn(0x7f7e30066f50 msgr2=0x7f7e3009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:06.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.879+0000 7f7e35abb700 1 -- 172.21.15.16:0/110916074 shutdown_connections 2023-09-30T22:35:06.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.879+0000 7f7e35abb700 1 -- 172.21.15.16:0/110916074 wait complete. 2023-09-30T22:35:06.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.879+0000 7f94f5564700 1 Processor -- start 2023-09-30T22:35:06.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.879+0000 7f94f5564700 1 -- start start 2023-09-30T22:35:06.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.880+0000 7f94f5564700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f94f009d6f0 0x7f94f0122410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:06.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.880+0000 7f94f5564700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94f009e000 0x7f94f0122950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:06.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.880+0000 7f94f5564700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f94f009e410 0x7f94f0122e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:06.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.880+0000 7f94f5564700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f0125f30 con 0x7f94f009e000 2023-09-30T22:35:06.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.880+0000 7f94f5564700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94f00a1d90 con 0x7f94f009d6f0 2023-09-30T22:35:06.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.880+0000 7f94f5564700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f94f0124020 con 0x7f94f009e410 2023-09-30T22:35:06.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.880+0000 7f94ef7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f94f009e410 0x7f94f0122e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:06.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.880+0000 7f7e35abb700 1 Processor -- start 2023-09-30T22:35:06.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.880+0000 7f94ef7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f94f009e410 0x7f94f0122e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:34242/0 (socket says 172.21.15.16:34242) 2023-09-30T22:35:06.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.880+0000 7f94ef7fe700 1 -- 172.21.15.16:0/2669592917 learned_addr learned my addr 172.21.15.16:0/2669592917 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:06.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.880+0000 7f94eeffd700 1 --2- 172.21.15.16:0/2669592917 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f94f009d6f0 0x7f94f0122410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:06.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.880+0000 7f94ee7fc700 1 --2- 172.21.15.16:0/2669592917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94f009e000 0x7f94f0122950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:06.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.880+0000 7f94ef7fe700 1 -- 172.21.15.16:0/2669592917 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f94f009d6f0 msgr2=0x7f94f0122410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:06.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.880+0000 7f94ef7fe700 1 --2- 172.21.15.16:0/2669592917 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f94f009d6f0 0x7f94f0122410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:06.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.880+0000 7f94ef7fe700 1 -- 172.21.15.16:0/2669592917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94f009e000 msgr2=0x7f94f0122950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:06.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.880+0000 7f94ef7fe700 1 --2- 172.21.15.16:0/2669592917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94f009e000 0x7f94f0122950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:06.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.880+0000 7f94ef7fe700 1 -- 172.21.15.16:0/2669592917 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e00094a0 con 0x7f94f009e410 2023-09-30T22:35:06.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.880+0000 7f7e35abb700 1 -- start start 2023-09-30T22:35:06.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.881+0000 7f94ef7fe700 1 --2- 172.21.15.16:0/2669592917 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f94f009e410 0x7f94f0122e90 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f94e800cbc0 tx=0x7f94e800ced0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:06.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.881+0000 7f94d77fe700 1 -- 172.21.15.16:0/2669592917 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e800a9e0 con 0x7f94f009e410 2023-09-30T22:35:06.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.881+0000 7f94d77fe700 1 -- 172.21.15.16:0/2669592917 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94e800ab40 con 0x7f94f009e410 2023-09-30T22:35:06.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.881+0000 7f94d77fe700 1 -- 172.21.15.16:0/2669592917 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e8012400 con 0x7f94f009e410 2023-09-30T22:35:06.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.881+0000 7f94f5564700 1 -- 172.21.15.16:0/2669592917 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94e400b740 con 0x7f94f009e410 2023-09-30T22:35:06.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.881+0000 7f94f5564700 1 -- 172.21.15.16:0/2669592917 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94f0130330 con 0x7f94f009e410 2023-09-30T22:35:06.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.881+0000 7f7e35abb700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7e3009d6f0 0x7f7e3012e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:06.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.881+0000 7f7e35abb700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7e3009e1d0 0x7f7e3012ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:06.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.881+0000 7f7e35abb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e30135e50 0x7f7e3012f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:06.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.881+0000 7f7e35abb700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e30131fa0 con 0x7f7e30135e50 2023-09-30T22:35:06.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.881+0000 7f7e35abb700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e301302d0 con 0x7f7e3009e1d0 2023-09-30T22:35:06.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.881+0000 7f7e35abb700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7e30131520 con 0x7f7e3009d6f0 2023-09-30T22:35:06.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.882+0000 7f7e2ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e30135e50 0x7f7e3012f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:06.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.882+0000 7f7e2ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e30135e50 0x7f7e3012f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44140/0 (socket says 172.21.15.16:44140) 2023-09-30T22:35:06.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.882+0000 7f7e2ffff700 1 -- 172.21.15.16:0/591255782 learned_addr learned my addr 172.21.15.16:0/591255782 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:06.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.882+0000 7f7e2effd700 1 --2- 172.21.15.16:0/591255782 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7e3009e1d0 0x7f7e3012ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:06.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.882+0000 7f7e2f7fe700 1 --2- 172.21.15.16:0/591255782 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7e3009d6f0 0x7f7e3012e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:06.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.882+0000 7f7e2ffff700 1 -- 172.21.15.16:0/591255782 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7e3009d6f0 msgr2=0x7f7e3012e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:06.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.882+0000 7f7e2ffff700 1 --2- 172.21.15.16:0/591255782 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7e3009d6f0 0x7f7e3012e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:06.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.882+0000 7f7e2ffff700 1 -- 172.21.15.16:0/591255782 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7e3009e1d0 msgr2=0x7f7e3012ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:06.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.882+0000 7f7e2ffff700 1 --2- 172.21.15.16:0/591255782 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7e3009e1d0 0x7f7e3012ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:06.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.882+0000 7f7e2ffff700 1 -- 172.21.15.16:0/591255782 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e240094a0 con 0x7f7e30135e50 2023-09-30T22:35:06.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.882+0000 7f7e2ffff700 1 --2- 172.21.15.16:0/591255782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e30135e50 0x7f7e3012f260 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f7e28007b50 tx=0x7f7e28007e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:06.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.882+0000 7f94d77fe700 1 -- 172.21.15.16:0/2669592917 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f94e801b3b0 con 0x7f94f009e410 2023-09-30T22:35:06.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.882+0000 7f94d77fe700 1 -- 172.21.15.16:0/2669592917 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f94e803f3c0 con 0x7f94f009e410 2023-09-30T22:35:06.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.882+0000 7f94f5564700 1 -- 172.21.15.16:0/2669592917 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94f0074ea0 con 0x7f94f009e410 2023-09-30T22:35:06.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.883+0000 7f7e17fff700 1 -- 172.21.15.16:0/591255782 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e28017070 con 0x7f7e30135e50 2023-09-30T22:35:06.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.883+0000 7f7e17fff700 1 -- 172.21.15.16:0/591255782 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e2800f040 con 0x7f7e30135e50 2023-09-30T22:35:06.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.883+0000 7f7e17fff700 1 -- 172.21.15.16:0/591255782 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e28003c20 con 0x7f7e30135e50 2023-09-30T22:35:06.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.883+0000 7f7e35abb700 1 -- 172.21.15.16:0/591255782 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e30130b30 con 0x7f7e30135e50 2023-09-30T22:35:06.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.883+0000 7f7e35abb700 1 -- 172.21.15.16:0/591255782 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e301319d0 con 0x7f7e30135e50 2023-09-30T22:35:06.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.884+0000 7f7e17fff700 1 -- 172.21.15.16:0/591255782 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7e2800d870 con 0x7f7e30135e50 2023-09-30T22:35:06.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.884+0000 7f7e17fff700 1 -- 172.21.15.16:0/591255782 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f7e2803f060 con 0x7f7e30135e50 2023-09-30T22:35:06.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.884+0000 7f7e35abb700 1 -- 172.21.15.16:0/591255782 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e30131970 con 0x7f7e30135e50 2023-09-30T22:35:06.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.887+0000 7f94d77fe700 1 -- 172.21.15.16:0/2669592917 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f94e8017080 con 0x7f94f009e410 2023-09-30T22:35:06.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.890+0000 7f7e17fff700 1 -- 172.21.15.16:0/591255782 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7e2801c410 con 0x7f7e30135e50 2023-09-30T22:35:06.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.922+0000 7f24cab5b700 1 Processor -- start 2023-09-30T22:35:06.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.922+0000 7f24cab5b700 1 -- start start 2023-09-30T22:35:06.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.922+0000 7f24cab5b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f24c40a0a30 0x7f24c40a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:06.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.922+0000 7f24cab5b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24c40a1340 0x7f24c4133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:06.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.922+0000 7f24cab5b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24c409edd0 con 0x7f24c4133fa0 2023-09-30T22:35:06.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.922+0000 7f24cab5b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24c409e1f0 con 0x7f24c40a0a30 2023-09-30T22:35:06.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.923+0000 7f24cab5b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f24c409dc50 con 0x7f24c40a1340 2023-09-30T22:35:06.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.923+0000 7f24c3fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24c40a1340 0x7f24c4133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:06.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.923+0000 7f24c88f7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f24c40a0a30 0x7f24c40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:06.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.923+0000 7f24c3fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24c40a1340 0x7f24c4133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:34258/0 (socket says 172.21.15.16:34258) 2023-09-30T22:35:06.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.923+0000 7f24c88f7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f24c40a0a30 0x7f24c40a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:52538/0 (socket says 172.21.15.16:52538) 2023-09-30T22:35:06.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.923+0000 7f24c3fff700 1 -- 172.21.15.16:0/3895717864 learned_addr learned my addr 172.21.15.16:0/3895717864 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:06.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.923+0000 7f24c37fe700 1 -- 172.21.15.16:0/3895717864 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1097185750 0 0) 0x7f24c409edd0 con 0x7f24c4133fa0 2023-09-30T22:35:06.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.923+0000 7f24c37fe700 1 -- 172.21.15.16:0/3895717864 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24b40035b0 con 0x7f24c4133fa0 2023-09-30T22:35:06.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.923+0000 7f24c88f7700 1 -- 172.21.15.16:0/3895717864 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24c40a1340 msgr2=0x7f24c4133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:06.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.923+0000 7f24c88f7700 1 --2- 172.21.15.16:0/3895717864 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24c40a1340 0x7f24c4133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:06.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.923+0000 7f24c88f7700 1 -- 172.21.15.16:0/3895717864 >> v1:172.21.15.16:6789/0 conn(0x7f24c4133fa0 legacy=0x7f24c413ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:06.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.923+0000 7f24c88f7700 1 -- 172.21.15.16:0/3895717864 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24c40a1fe0 con 0x7f24c40a0a30 2023-09-30T22:35:06.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.923+0000 7f24c3fff700 1 --2- 172.21.15.16:0/3895717864 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24c40a1340 0x7f24c4133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:06.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.924+0000 7f24c88f7700 1 --2- 172.21.15.16:0/3895717864 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f24c40a0a30 0x7f24c40a0e00 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f24b8009870 tx=0x7f24b8009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32c7cabc6a7fc760 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:06.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.924+0000 7f24c37fe700 1 -- 172.21.15.16:0/3895717864 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b8011830 con 0x7f24c40a0a30 2023-09-30T22:35:06.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.924+0000 7f24c37fe700 1 -- 172.21.15.16:0/3895717864 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b8011990 con 0x7f24c40a0a30 2023-09-30T22:35:06.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.924+0000 7f24c37fe700 1 -- 172.21.15.16:0/3895717864 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b8011d00 con 0x7f24c40a0a30 2023-09-30T22:35:06.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.924+0000 7f24cab5b700 1 -- 172.21.15.16:0/3895717864 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f24c40a0a30 msgr2=0x7f24c40a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:06.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.924+0000 7f24cab5b700 1 --2- 172.21.15.16:0/3895717864 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f24c40a0a30 0x7f24c40a0e00 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f24b8009870 tx=0x7f24b8009b80 comp rx=0 tx=0).stop 2023-09-30T22:35:06.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.924+0000 7f24cab5b700 1 -- 172.21.15.16:0/3895717864 shutdown_connections 2023-09-30T22:35:06.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.924+0000 7f24cab5b700 1 --2- 172.21.15.16:0/3895717864 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f24c40a0a30 0x7f24c40a0e00 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:06.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.924+0000 7f24cab5b700 1 --2- 172.21.15.16:0/3895717864 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24c40a1340 0x7f24c4133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:06.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.924+0000 7f24cab5b700 1 -- 172.21.15.16:0/3895717864 >> 172.21.15.16:0/3895717864 conn(0x7f24c40a6550 msgr2=0x7f24c40a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:06.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.924+0000 7f24cab5b700 1 -- 172.21.15.16:0/3895717864 shutdown_connections 2023-09-30T22:35:06.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.924+0000 7f24cab5b700 1 -- 172.21.15.16:0/3895717864 wait complete. 2023-09-30T22:35:06.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.925+0000 7f24cab5b700 1 Processor -- start 2023-09-30T22:35:06.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.926+0000 7f24cab5b700 1 -- start start 2023-09-30T22:35:06.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.926+0000 7f24cab5b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24c40a0a30 0x7f24c411e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:06.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.926+0000 7f24cab5b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f24c40a1340 0x7f24c411ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:06.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.926+0000 7f24cab5b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24c4133fa0 0x7f24c411f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:06.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.926+0000 7f24cab5b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c41208b0 con 0x7f24c4133fa0 2023-09-30T22:35:06.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.926+0000 7f24cab5b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24c4122170 con 0x7f24c40a1340 2023-09-30T22:35:06.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.926+0000 7f24cab5b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f24c4120260 con 0x7f24c40a0a30 2023-09-30T22:35:06.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.926+0000 7f24c88f7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24c40a0a30 0x7f24c411e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:06.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.926+0000 7f24c90f8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24c4133fa0 0x7f24c411f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:06.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.926+0000 7f24c88f7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24c40a0a30 0x7f24c411e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:34260/0 (socket says 172.21.15.16:34260) 2023-09-30T22:35:06.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.926+0000 7f24c90f8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24c4133fa0 0x7f24c411f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44152/0 (socket says 172.21.15.16:44152) 2023-09-30T22:35:06.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.926+0000 7f24c88f7700 1 -- 172.21.15.16:0/1163615453 learned_addr learned my addr 172.21.15.16:0/1163615453 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:06.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.926+0000 7f24c3fff700 1 --2- 172.21.15.16:0/1163615453 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f24c40a1340 0x7f24c411ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:06.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.927+0000 7f24c88f7700 1 -- 172.21.15.16:0/1163615453 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f24c40a1340 msgr2=0x7f24c411ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:06.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.927+0000 7f24c88f7700 1 --2- 172.21.15.16:0/1163615453 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f24c40a1340 0x7f24c411ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:06.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.927+0000 7f24c88f7700 1 -- 172.21.15.16:0/1163615453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24c4133fa0 msgr2=0x7f24c411f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:06.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.927+0000 7f24c88f7700 1 --2- 172.21.15.16:0/1163615453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24c4133fa0 0x7f24c411f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:06.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.927+0000 7f24c88f7700 1 -- 172.21.15.16:0/1163615453 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b0009550 con 0x7f24c40a0a30 2023-09-30T22:35:06.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.927+0000 7f24c90f8700 1 --2- 172.21.15.16:0/1163615453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24c4133fa0 0x7f24c411f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:06.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.927+0000 7f24c3fff700 1 --2- 172.21.15.16:0/1163615453 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f24c40a1340 0x7f24c411ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:06.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.927+0000 7f24c88f7700 1 --2- 172.21.15.16:0/1163615453 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24c40a0a30 0x7f24c411e6e0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f24b8009bc0 tx=0x7f24b8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:06.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.927+0000 7f24c17fa700 1 -- 172.21.15.16:0/1163615453 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b8004550 con 0x7f24c40a0a30 2023-09-30T22:35:06.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.927+0000 7f24c17fa700 1 -- 172.21.15.16:0/1163615453 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b801f070 con 0x7f24c40a0a30 2023-09-30T22:35:06.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.927+0000 7f24cab5b700 1 -- 172.21.15.16:0/1163615453 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24b80094a0 con 0x7f24c40a0a30 2023-09-30T22:35:06.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.927+0000 7f24c17fa700 1 -- 172.21.15.16:0/1163615453 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b80038a0 con 0x7f24c40a0a30 2023-09-30T22:35:06.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.927+0000 7f24cab5b700 1 -- 172.21.15.16:0/1163615453 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24c40a25e0 con 0x7f24c40a0a30 2023-09-30T22:35:06.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.928+0000 7f24cab5b700 1 -- 172.21.15.16:0/1163615453 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24c4074ea0 con 0x7f24c40a0a30 2023-09-30T22:35:06.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.929+0000 7f24c17fa700 1 -- 172.21.15.16:0/1163615453 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f24b8003a00 con 0x7f24c40a0a30 2023-09-30T22:35:06.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.929+0000 7f24c17fa700 1 -- 172.21.15.16:0/1163615453 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f24b8021070 con 0x7f24c40a0a30 2023-09-30T22:35:06.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:06.934+0000 7f24c17fa700 1 -- 172.21.15.16:0/1163615453 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f24b8024970 con 0x7f24c40a0a30 2023-09-30T22:35:07.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.097+0000 7f94f5564700 1 -- 172.21.15.16:0/2669592917 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f94f0124810 con 0x7f94f009e410 2023-09-30T22:35:07.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.097+0000 7f94d77fe700 1 -- 172.21.15.16:0/2669592917 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f94e80192c0 con 0x7f94f009e410 2023-09-30T22:35:07.098 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:35:07.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.099+0000 7f94f5564700 1 -- 172.21.15.16:0/2669592917 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f94f009e410 msgr2=0x7f94f0122e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:07.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.099+0000 7f94f5564700 1 --2- 172.21.15.16:0/2669592917 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f94f009e410 0x7f94f0122e90 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f94e800cbc0 tx=0x7f94e800ced0 comp rx=0 tx=0).stop 2023-09-30T22:35:07.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.099+0000 7f94f5564700 1 -- 172.21.15.16:0/2669592917 shutdown_connections 2023-09-30T22:35:07.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.099+0000 7f94f5564700 1 --2- 172.21.15.16:0/2669592917 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f94f009d6f0 0x7f94f0122410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:07.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.099+0000 7f94f5564700 1 --2- 172.21.15.16:0/2669592917 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f94f009e410 0x7f94f0122e90 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:07.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.099+0000 7f94f5564700 1 --2- 172.21.15.16:0/2669592917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f94f009e000 0x7f94f0122950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:07.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.099+0000 7f94f5564700 1 -- 172.21.15.16:0/2669592917 >> 172.21.15.16:0/2669592917 conn(0x7f94f0066f50 msgr2=0x7f94f00a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:07.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.099+0000 7f94f5564700 1 -- 172.21.15.16:0/2669592917 shutdown_connections 2023-09-30T22:35:07.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.100+0000 7f94f5564700 1 -- 172.21.15.16:0/2669592917 wait complete. 2023-09-30T22:35:07.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.106+0000 7f7e35abb700 1 -- 172.21.15.16:0/591255782 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f7e30074ea0 con 0x7f7e30135e50 2023-09-30T22:35:07.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.106+0000 7f7e17fff700 1 -- 172.21.15.16:0/591255782 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7e28003d80 con 0x7f7e30135e50 2023-09-30T22:35:07.107 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:35:07.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.108+0000 7f7e35abb700 1 -- 172.21.15.16:0/591255782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e30135e50 msgr2=0x7f7e3012f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:07.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.108+0000 7f7e35abb700 1 --2- 172.21.15.16:0/591255782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e30135e50 0x7f7e3012f260 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f7e28007b50 tx=0x7f7e28007e60 comp rx=0 tx=0).stop 2023-09-30T22:35:07.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.108+0000 7f7e35abb700 1 -- 172.21.15.16:0/591255782 shutdown_connections 2023-09-30T22:35:07.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.108+0000 7f7e35abb700 1 --2- 172.21.15.16:0/591255782 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7e30135e50 0x7f7e3012f260 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:07.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.109+0000 7f7e35abb700 1 --2- 172.21.15.16:0/591255782 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7e3009d6f0 0x7f7e3012e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:07.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.109+0000 7f7e35abb700 1 --2- 172.21.15.16:0/591255782 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7e3009e1d0 0x7f7e3012ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:07.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.109+0000 7f7e35abb700 1 -- 172.21.15.16:0/591255782 >> 172.21.15.16:0/591255782 conn(0x7f7e30066f50 msgr2=0x7f7e300a1580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:07.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.109+0000 7f7e35abb700 1 -- 172.21.15.16:0/591255782 shutdown_connections 2023-09-30T22:35:07.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.109+0000 7f7e35abb700 1 -- 172.21.15.16:0/591255782 wait complete. 2023-09-30T22:35:07.115 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:35:07.123 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:35:07.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.142+0000 7f24cab5b700 1 -- 172.21.15.16:0/1163615453 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f24c40a28e0 con 0x7f24c40a0a30 2023-09-30T22:35:07.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.142+0000 7f24c17fa700 1 -- 172.21.15.16:0/1163615453 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f24b8034020 con 0x7f24c40a0a30 2023-09-30T22:35:07.143 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:35:07.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.145+0000 7f24cab5b700 1 -- 172.21.15.16:0/1163615453 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24c40a0a30 msgr2=0x7f24c411e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:07.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.145+0000 7f24cab5b700 1 --2- 172.21.15.16:0/1163615453 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24c40a0a30 0x7f24c411e6e0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f24b8009bc0 tx=0x7f24b8004750 comp rx=0 tx=0).stop 2023-09-30T22:35:07.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.145+0000 7f24cab5b700 1 -- 172.21.15.16:0/1163615453 shutdown_connections 2023-09-30T22:35:07.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.145+0000 7f24cab5b700 1 --2- 172.21.15.16:0/1163615453 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24c4133fa0 0x7f24c411f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:07.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.145+0000 7f24cab5b700 1 --2- 172.21.15.16:0/1163615453 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24c40a0a30 0x7f24c411e6e0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:07.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.145+0000 7f24cab5b700 1 --2- 172.21.15.16:0/1163615453 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f24c40a1340 0x7f24c411ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:07.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.145+0000 7f24cab5b700 1 -- 172.21.15.16:0/1163615453 >> 172.21.15.16:0/1163615453 conn(0x7f24c40a6550 msgr2=0x7f24c41b9120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:07.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.145+0000 7f24cab5b700 1 -- 172.21.15.16:0/1163615453 shutdown_connections 2023-09-30T22:35:07.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.145+0000 7f24cab5b700 1 -- 172.21.15.16:0/1163615453 wait complete. 2023-09-30T22:35:07.159 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:35:07.429 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:35:07.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.574+0000 7f1e33d1e700 1 Processor -- start 2023-09-30T22:35:07.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.575+0000 7f1e33d1e700 1 -- start start 2023-09-30T22:35:07.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.575+0000 7f1e33d1e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1e2c133420 0x7f1e2c1337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:07.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.575+0000 7f1e33d1e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1e2c133e30 0x7f1e2c12ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:07.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.575+0000 7f1e33d1e700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e2c06b990 con 0x7f1e2c134260 2023-09-30T22:35:07.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.575+0000 7f1e33d1e700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e2c0a0590 con 0x7f1e2c133e30 2023-09-30T22:35:07.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.575+0000 7f1e33d1e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2c09fb80 con 0x7f1e2c133420 2023-09-30T22:35:07.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.575+0000 7f1e312b9700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1e2c133e30 0x7f1e2c12ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:07.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.575+0000 7f1e31aba700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1e2c133420 0x7f1e2c1337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:07.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.575+0000 7f1e322bb700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1e2c134260 0x7f1e2c1304d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51664/0 (socket says 172.21.15.16:51664) 2023-09-30T22:35:07.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.575+0000 7f1e312b9700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1e2c133e30 0x7f1e2c12ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49098/0 (socket says 172.21.15.16:49098) 2023-09-30T22:35:07.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.575+0000 7f1e31aba700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1e2c133420 0x7f1e2c1337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37564/0 (socket says 172.21.15.16:37564) 2023-09-30T22:35:07.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.575+0000 7f1e322bb700 1 -- 172.21.15.16:0/3876163740 learned_addr learned my addr 172.21.15.16:0/3876163740 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:07.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.576+0000 7f1e31aba700 1 -- 172.21.15.16:0/3876163740 >> v2:172.21.15.16:3300/0 conn(0x7f1e2c133e30 msgr2=0x7f1e2c12ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:07.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.576+0000 7f1e31aba700 1 --2- 172.21.15.16:0/3876163740 >> v2:172.21.15.16:3300/0 conn(0x7f1e2c133e30 0x7f1e2c12ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:07.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.576+0000 7f1e31aba700 1 -- 172.21.15.16:0/3876163740 >> v1:172.21.15.16:6789/0 conn(0x7f1e2c134260 legacy=0x7f1e2c1304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:07.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.576+0000 7f1e31aba700 1 -- 172.21.15.16:0/3876163740 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e2c0a0030 con 0x7f1e2c133420 2023-09-30T22:35:07.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.576+0000 7f1e312b9700 1 --2- 172.21.15.16:0/3876163740 >> v2:172.21.15.16:3300/0 conn(0x7f1e2c133e30 0x7f1e2c12ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:07.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.576+0000 7f1e31aba700 1 --2- 172.21.15.16:0/3876163740 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1e2c133420 0x7f1e2c1337f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f1e28009840 tx=0x7f1e28009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea970bf61acc19e0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:07.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.576+0000 7f1e30ab8700 1 -- 172.21.15.16:0/3876163740 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e28011830 con 0x7f1e2c133420 2023-09-30T22:35:07.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.576+0000 7f1e30ab8700 1 -- 172.21.15.16:0/3876163740 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e28011990 con 0x7f1e2c133420 2023-09-30T22:35:07.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.576+0000 7f1e30ab8700 1 -- 172.21.15.16:0/3876163740 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e28011d20 con 0x7f1e2c133420 2023-09-30T22:35:07.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.577+0000 7f1e33d1e700 1 -- 172.21.15.16:0/3876163740 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1e2c133420 msgr2=0x7f1e2c1337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:07.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.577+0000 7f1e33d1e700 1 --2- 172.21.15.16:0/3876163740 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1e2c133420 0x7f1e2c1337f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f1e28009840 tx=0x7f1e28009b50 comp rx=0 tx=0).stop 2023-09-30T22:35:07.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.577+0000 7f1e33d1e700 1 -- 172.21.15.16:0/3876163740 shutdown_connections 2023-09-30T22:35:07.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.577+0000 7f1e33d1e700 1 --2- 172.21.15.16:0/3876163740 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1e2c133420 0x7f1e2c1337f0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:07.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.577+0000 7f1e33d1e700 1 --2- 172.21.15.16:0/3876163740 >> v2:172.21.15.16:3300/0 conn(0x7f1e2c133e30 0x7f1e2c12ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:07.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.577+0000 7f1e33d1e700 1 -- 172.21.15.16:0/3876163740 >> 172.21.15.16:0/3876163740 conn(0x7f1e2c0a4210 msgr2=0x7f1e2c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:07.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.577+0000 7f1e33d1e700 1 -- 172.21.15.16:0/3876163740 shutdown_connections 2023-09-30T22:35:07.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.577+0000 7f1e33d1e700 1 -- 172.21.15.16:0/3876163740 wait complete. 2023-09-30T22:35:07.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.578+0000 7f1e33d1e700 1 Processor -- start 2023-09-30T22:35:07.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.578+0000 7f1e33d1e700 1 -- start start 2023-09-30T22:35:07.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.578+0000 7f1e33d1e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1e2c133420 0x7f1e2c09c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:07.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.578+0000 7f1e33d1e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1e2c133e30 0x7f1e2c09c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:07.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.578+0000 7f1e33d1e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e2c134260 0x7f1e2c09cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:07.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.578+0000 7f1e33d1e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2c09dcd0 con 0x7f1e2c134260 2023-09-30T22:35:07.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.578+0000 7f1e33d1e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2c11fe90 con 0x7f1e2c133e30 2023-09-30T22:35:07.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.578+0000 7f1e33d1e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2c09df30 con 0x7f1e2c133420 2023-09-30T22:35:07.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.578+0000 7f1e31aba700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1e2c133420 0x7f1e2c09c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:07.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.578+0000 7f1e31aba700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1e2c133420 0x7f1e2c09c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:55146/0 (socket says 172.21.15.16:55146) 2023-09-30T22:35:07.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.578+0000 7f1e31aba700 1 -- 172.21.15.16:0/1237253680 learned_addr learned my addr 172.21.15.16:0/1237253680 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:07.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.578+0000 7f1e322bb700 1 --2- 172.21.15.16:0/1237253680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e2c134260 0x7f1e2c09cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:07.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.578+0000 7f1e312b9700 1 --2- 172.21.15.16:0/1237253680 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1e2c133e30 0x7f1e2c09c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:07.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.579+0000 7f1e322bb700 1 -- 172.21.15.16:0/1237253680 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1e2c133420 msgr2=0x7f1e2c09c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:07.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.579+0000 7f1e322bb700 1 --2- 172.21.15.16:0/1237253680 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1e2c133420 0x7f1e2c09c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:07.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.579+0000 7f1e322bb700 1 -- 172.21.15.16:0/1237253680 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1e2c133e30 msgr2=0x7f1e2c09c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:07.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.579+0000 7f1e322bb700 1 --2- 172.21.15.16:0/1237253680 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1e2c133e30 0x7f1e2c09c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:07.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.579+0000 7f1e322bb700 1 -- 172.21.15.16:0/1237253680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e1c0094a0 con 0x7f1e2c134260 2023-09-30T22:35:07.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.579+0000 7f1e31aba700 1 --2- 172.21.15.16:0/1237253680 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1e2c133420 0x7f1e2c09c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:07.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.579+0000 7f1e312b9700 1 --2- 172.21.15.16:0/1237253680 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1e2c133e30 0x7f1e2c09c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:07.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.579+0000 7f1e322bb700 1 --2- 172.21.15.16:0/1237253680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e2c134260 0x7f1e2c09cf00 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f1e2400caa0 tx=0x7f1e2400cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:07.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.579+0000 7f1e227fc700 1 -- 172.21.15.16:0/1237253680 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2400a960 con 0x7f1e2c134260 2023-09-30T22:35:07.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.579+0000 7f1e227fc700 1 -- 172.21.15.16:0/1237253680 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e2400aac0 con 0x7f1e2c134260 2023-09-30T22:35:07.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.579+0000 7f1e33d1e700 1 -- 172.21.15.16:0/1237253680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e280094a0 con 0x7f1e2c134260 2023-09-30T22:35:07.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.579+0000 7f1e227fc700 1 -- 172.21.15.16:0/1237253680 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2400ae30 con 0x7f1e2c134260 2023-09-30T22:35:07.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.579+0000 7f1e33d1e700 1 -- 172.21.15.16:0/1237253680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e2c11fb60 con 0x7f1e2c134260 2023-09-30T22:35:07.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.580+0000 7f1e227fc700 1 -- 172.21.15.16:0/1237253680 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1e2401b3b0 con 0x7f1e2c134260 2023-09-30T22:35:07.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.580+0000 7f1e33d1e700 1 -- 172.21.15.16:0/1237253680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e2c131990 con 0x7f1e2c134260 2023-09-30T22:35:07.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.580+0000 7f1e227fc700 1 -- 172.21.15.16:0/1237253680 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f1e2403f280 con 0x7f1e2c134260 2023-09-30T22:35:07.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.585+0000 7f1e227fc700 1 -- 172.21.15.16:0/1237253680 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1e24017020 con 0x7f1e2c134260 2023-09-30T22:35:07.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.794+0000 7f1e33d1e700 1 -- 172.21.15.16:0/1237253680 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1e2c074ea0 con 0x7f1e2c134260 2023-09-30T22:35:07.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.794+0000 7f1e227fc700 1 -- 172.21.15.16:0/1237253680 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1e240192c0 con 0x7f1e2c134260 2023-09-30T22:35:07.795 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:35:07.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.797+0000 7f1e33d1e700 1 -- 172.21.15.16:0/1237253680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e2c134260 msgr2=0x7f1e2c09cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:07.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.797+0000 7f1e33d1e700 1 --2- 172.21.15.16:0/1237253680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e2c134260 0x7f1e2c09cf00 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f1e2400caa0 tx=0x7f1e2400cdb0 comp rx=0 tx=0).stop 2023-09-30T22:35:07.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.797+0000 7f1e33d1e700 1 -- 172.21.15.16:0/1237253680 shutdown_connections 2023-09-30T22:35:07.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.797+0000 7f1e33d1e700 1 --2- 172.21.15.16:0/1237253680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1e2c134260 0x7f1e2c09cf00 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:07.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.797+0000 7f1e33d1e700 1 --2- 172.21.15.16:0/1237253680 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1e2c133420 0x7f1e2c09c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:07.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.797+0000 7f1e33d1e700 1 --2- 172.21.15.16:0/1237253680 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1e2c133e30 0x7f1e2c09c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:07.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.797+0000 7f1e33d1e700 1 -- 172.21.15.16:0/1237253680 >> 172.21.15.16:0/1237253680 conn(0x7f1e2c0a4210 msgr2=0x7f1e2c0a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:07.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.797+0000 7f1e33d1e700 1 -- 172.21.15.16:0/1237253680 shutdown_connections 2023-09-30T22:35:07.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:07.797+0000 7f1e33d1e700 1 -- 172.21.15.16:0/1237253680 wait complete. 2023-09-30T22:35:07.811 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:35:08.116 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:35:08.124 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:35:08.160 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:35:08.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.265+0000 7f44c171a700 1 Processor -- start 2023-09-30T22:35:08.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.265+0000 7f44c171a700 1 -- start start 2023-09-30T22:35:08.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.266+0000 7f44c171a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f44bc133ee0 0x7f44bc09e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:08.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.266+0000 7f44c171a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f44bc1397e0 0x7f44bc13bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:08.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.266+0000 7f44c171a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44bc1334b0 con 0x7f44bc133b10 2023-09-30T22:35:08.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.266+0000 7f44c171a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44bc070bc0 con 0x7f44bc133ee0 2023-09-30T22:35:08.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.266+0000 7f44c171a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f44bc070350 con 0x7f44bc1397e0 2023-09-30T22:35:08.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.266+0000 7f44ba7fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f44bc133ee0 0x7f44bc09e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:08.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.266+0000 7f44ba7fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f44bc133ee0 0x7f44bc09e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37572/0 (socket says 172.21.15.16:37572) 2023-09-30T22:35:08.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.266+0000 7f44ba7fc700 1 -- 172.21.15.16:0/1626413335 learned_addr learned my addr 172.21.15.16:0/1626413335 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:08.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.266+0000 7f44baffd700 1 --1- 172.21.15.16:0/1626413335 >> v1:172.21.15.16:6789/0 conn(0x7f44bc133b10 0x7f44bc09de50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51678/0 (socket says 172.21.15.16:51678) 2023-09-30T22:35:08.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.266+0000 7f44bb7fe700 1 --2- 172.21.15.16:0/1626413335 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f44bc1397e0 0x7f44bc13bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:08.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.267+0000 7f44b9ffb700 1 -- 172.21.15.16:0/1626413335 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1438364839 0 0) 0x7f44bc1334b0 con 0x7f44bc133b10 2023-09-30T22:35:08.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.267+0000 7f44ba7fc700 1 -- 172.21.15.16:0/1626413335 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f44bc1397e0 msgr2=0x7f44bc13bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:08.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.267+0000 7f44ba7fc700 1 --2- 172.21.15.16:0/1626413335 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f44bc1397e0 0x7f44bc13bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:08.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.267+0000 7f44ba7fc700 1 -- 172.21.15.16:0/1626413335 >> v1:172.21.15.16:6789/0 conn(0x7f44bc133b10 legacy=0x7f44bc09de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:08.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.267+0000 7f44ba7fc700 1 -- 172.21.15.16:0/1626413335 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44bc1336e0 con 0x7f44bc133ee0 2023-09-30T22:35:08.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.267+0000 7f44ba7fc700 1 --2- 172.21.15.16:0/1626413335 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f44bc133ee0 0x7f44bc09e560 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f44b0009920 tx=0x7f44b0009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5038b2e8341bd65 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:08.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.267+0000 7f44b9ffb700 1 -- 172.21.15.16:0/1626413335 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b0011830 con 0x7f44bc133ee0 2023-09-30T22:35:08.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.267+0000 7f44b9ffb700 1 -- 172.21.15.16:0/1626413335 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44b0011990 con 0x7f44bc133ee0 2023-09-30T22:35:08.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.267+0000 7f44b9ffb700 1 -- 172.21.15.16:0/1626413335 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b0011d20 con 0x7f44bc133ee0 2023-09-30T22:35:08.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.268+0000 7f44c171a700 1 -- 172.21.15.16:0/1626413335 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f44bc133ee0 msgr2=0x7f44bc09e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:08.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.268+0000 7f44c171a700 1 --2- 172.21.15.16:0/1626413335 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f44bc133ee0 0x7f44bc09e560 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f44b0009920 tx=0x7f44b0009c30 comp rx=0 tx=0).stop 2023-09-30T22:35:08.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.268+0000 7f44c171a700 1 -- 172.21.15.16:0/1626413335 shutdown_connections 2023-09-30T22:35:08.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.268+0000 7f44c171a700 1 --2- 172.21.15.16:0/1626413335 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f44bc1397e0 0x7f44bc13bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:08.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.268+0000 7f44c171a700 1 --2- 172.21.15.16:0/1626413335 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f44bc133ee0 0x7f44bc09e560 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:08.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.268+0000 7f44c171a700 1 -- 172.21.15.16:0/1626413335 >> 172.21.15.16:0/1626413335 conn(0x7f44bc0a2aa0 msgr2=0x7f44bc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:08.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.268+0000 7f44c171a700 1 -- 172.21.15.16:0/1626413335 shutdown_connections 2023-09-30T22:35:08.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.268+0000 7f44c171a700 1 -- 172.21.15.16:0/1626413335 wait complete. 2023-09-30T22:35:08.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.269+0000 7f44c171a700 1 Processor -- start 2023-09-30T22:35:08.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.270+0000 7f44c171a700 1 -- start start 2023-09-30T22:35:08.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.270+0000 7f44c171a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f44bc133b10 0x7f44bc11a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:08.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.270+0000 7f44c171a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f44bc133ee0 0x7f44bc11ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:08.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.270+0000 7f44c171a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44bc1397e0 0x7f44bc11b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:08.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.270+0000 7f44c171a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44bc11dec0 con 0x7f44bc1397e0 2023-09-30T22:35:08.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.270+0000 7f44c171a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44bc0a0ae0 con 0x7f44bc133b10 2023-09-30T22:35:08.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.270+0000 7f44c171a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f44bc11c3a0 con 0x7f44bc133ee0 2023-09-30T22:35:08.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.270+0000 7f44ba7fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f44bc133ee0 0x7f44bc11ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:08.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.270+0000 7f44ba7fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f44bc133ee0 0x7f44bc11ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:55170/0 (socket says 172.21.15.16:55170) 2023-09-30T22:35:08.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.270+0000 7f44ba7fc700 1 -- 172.21.15.16:0/3609691694 learned_addr learned my addr 172.21.15.16:0/3609691694 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:08.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.270+0000 7f44baffd700 1 --2- 172.21.15.16:0/3609691694 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f44bc133b10 0x7f44bc11a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:08.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.270+0000 7f44bb7fe700 1 --2- 172.21.15.16:0/3609691694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44bc1397e0 0x7f44bc11b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:08.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.271+0000 7f44ba7fc700 1 -- 172.21.15.16:0/3609691694 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f44bc133b10 msgr2=0x7f44bc11a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:08.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.271+0000 7f44ba7fc700 1 --2- 172.21.15.16:0/3609691694 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f44bc133b10 0x7f44bc11a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:08.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.271+0000 7f44ba7fc700 1 -- 172.21.15.16:0/3609691694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44bc1397e0 msgr2=0x7f44bc11b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:08.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.271+0000 7f44ba7fc700 1 --2- 172.21.15.16:0/3609691694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44bc1397e0 0x7f44bc11b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:08.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.271+0000 7f44ba7fc700 1 -- 172.21.15.16:0/3609691694 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44b0009550 con 0x7f44bc133ee0 2023-09-30T22:35:08.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.271+0000 7f44bb7fe700 1 --2- 172.21.15.16:0/3609691694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44bc1397e0 0x7f44bc11b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:08.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.271+0000 7f44ba7fc700 1 --2- 172.21.15.16:0/3609691694 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f44bc133ee0 0x7f44bc11ac40 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f44b00098f0 tx=0x7f44b0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:08.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.271+0000 7f44a37fe700 1 -- 172.21.15.16:0/3609691694 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b0004450 con 0x7f44bc133ee0 2023-09-30T22:35:08.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.271+0000 7f44a37fe700 1 -- 172.21.15.16:0/3609691694 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44b00045b0 con 0x7f44bc133ee0 2023-09-30T22:35:08.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.271+0000 7f44c171a700 1 -- 172.21.15.16:0/3609691694 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44bc06f910 con 0x7f44bc133ee0 2023-09-30T22:35:08.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.271+0000 7f44a37fe700 1 -- 172.21.15.16:0/3609691694 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44b00038a0 con 0x7f44bc133ee0 2023-09-30T22:35:08.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.271+0000 7f44c171a700 1 -- 172.21.15.16:0/3609691694 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44bc11e1a0 con 0x7f44bc133ee0 2023-09-30T22:35:08.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.273+0000 7f44c171a700 1 -- 172.21.15.16:0/3609691694 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44bc06eee0 con 0x7f44bc133ee0 2023-09-30T22:35:08.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.273+0000 7f44a37fe700 1 -- 172.21.15.16:0/3609691694 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f44b0021070 con 0x7f44bc133ee0 2023-09-30T22:35:08.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.273+0000 7f44a37fe700 1 -- 172.21.15.16:0/3609691694 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f44b0040c80 con 0x7f44bc133ee0 2023-09-30T22:35:08.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.278+0000 7f44a37fe700 1 -- 172.21.15.16:0/3609691694 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f44bc06eee0 con 0x7f44bc133ee0 2023-09-30T22:35:08.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.305+0000 7fdfb79f0700 1 Processor -- start 2023-09-30T22:35:08.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.305+0000 7fa33eba6700 1 Processor -- start 2023-09-30T22:35:08.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.305+0000 7fdfb79f0700 1 -- start start 2023-09-30T22:35:08.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.305+0000 7fa33eba6700 1 -- start start 2023-09-30T22:35:08.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.306+0000 7fa33eba6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa338131630 0x7fa338131a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:08.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.306+0000 7fdfb79f0700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdfb009d630 0x7fdfb00a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:08.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.306+0000 7fdfb79f0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdfb0138d80 0x7fdfb013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:08.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.306+0000 7fdfb79f0700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfb00a2a70 con 0x7fdfb00a0a30 2023-09-30T22:35:08.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.306+0000 7fdfb79f0700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfb00a01d0 con 0x7fdfb009d630 2023-09-30T22:35:08.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.306+0000 7fdfb79f0700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb009e2b0 con 0x7fdfb0138d80 2023-09-30T22:35:08.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.306+0000 7fa33eba6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa338131f40 0x7fa33806e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:08.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.306+0000 7fa33eba6700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa338071a00 con 0x7fa33806e9d0 2023-09-30T22:35:08.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.306+0000 7fa33eba6700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa338131430 con 0x7fa338131f40 2023-09-30T22:35:08.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.306+0000 7fa33eba6700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa33809a510 con 0x7fa338131630 2023-09-30T22:35:08.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.307+0000 7fa337fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa338131f40 0x7fa33806e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:08.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.306+0000 7fdfb4f8b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdfb009d630 0x7fdfb00a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:08.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.306+0000 7fdfb4f8b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdfb009d630 0x7fdfb00a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49142/0 (socket says 172.21.15.16:49142) 2023-09-30T22:35:08.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.306+0000 7fdfb4f8b700 1 -- 172.21.15.16:0/1601583841 learned_addr learned my addr 172.21.15.16:0/1601583841 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:08.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.307+0000 7fdfb5f8d700 1 --2- 172.21.15.16:0/1601583841 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdfb0138d80 0x7fdfb013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:08.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.307+0000 7fa33c942700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa338131630 0x7fa338131a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:08.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.307+0000 7fa337fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa338131f40 0x7fa33806e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37594/0 (socket says 172.21.15.16:37594) 2023-09-30T22:35:08.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.307+0000 7fa337fff700 1 -- 172.21.15.16:0/1537766416 learned_addr learned my addr 172.21.15.16:0/1537766416 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:08.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.307+0000 7fdfb5f8d700 1 -- 172.21.15.16:0/1601583841 >> v2:172.21.15.16:3300/0 conn(0x7fdfb009d630 msgr2=0x7fdfb00a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:08.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.307+0000 7fdfb5f8d700 1 --2- 172.21.15.16:0/1601583841 >> v2:172.21.15.16:3300/0 conn(0x7fdfb009d630 0x7fdfb00a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:08.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.307+0000 7fdfb5f8d700 1 -- 172.21.15.16:0/1601583841 >> v1:172.21.15.16:6789/0 conn(0x7fdfb00a0a30 legacy=0x7fdfb00a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:08.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.307+0000 7fdfb5f8d700 1 -- 172.21.15.16:0/1601583841 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfb009dc50 con 0x7fdfb0138d80 2023-09-30T22:35:08.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.307+0000 7fdfb4f8b700 1 --2- 172.21.15.16:0/1601583841 >> v2:172.21.15.16:3300/0 conn(0x7fdfb009d630 0x7fdfb00a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:08.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.307+0000 7fa337fff700 1 -- 172.21.15.16:0/1537766416 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa338131630 msgr2=0x7fa338131a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:08.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.307+0000 7fa337fff700 1 --2- 172.21.15.16:0/1537766416 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa338131630 0x7fa338131a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:08.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.307+0000 7fa337fff700 1 -- 172.21.15.16:0/1537766416 >> v1:172.21.15.16:6789/0 conn(0x7fa33806e9d0 legacy=0x7fa338138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:08.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.307+0000 7fa337fff700 1 -- 172.21.15.16:0/1537766416 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3380721c0 con 0x7fa338131f40 2023-09-30T22:35:08.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.307+0000 7fa33c942700 1 --2- 172.21.15.16:0/1537766416 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa338131630 0x7fa338131a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:08.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.307+0000 7fdfb5f8d700 1 --2- 172.21.15.16:0/1601583841 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdfb0138d80 0x7fdfb013b160 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8009d90 tx=0x7fdfa800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f399e2bca985cbff server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:08.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.307+0000 7fa337fff700 1 --2- 172.21.15.16:0/1537766416 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa338131f40 0x7fa33806e490 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fa324009920 tx=0x7fa324009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7b3c0979585fbea server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:08.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.307+0000 7fdfa7fff700 1 -- 172.21.15.16:0/1601583841 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa800e070 con 0x7fdfb0138d80 2023-09-30T22:35:08.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.307+0000 7fdfa7fff700 1 -- 172.21.15.16:0/1601583841 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfa800bae0 con 0x7fdfb0138d80 2023-09-30T22:35:08.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.307+0000 7fdfa7fff700 1 -- 172.21.15.16:0/1601583841 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa800be90 con 0x7fdfb0138d80 2023-09-30T22:35:08.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.308+0000 7fa3377fe700 1 -- 172.21.15.16:0/1537766416 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa324004230 con 0x7fa338131f40 2023-09-30T22:35:08.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.308+0000 7fa3377fe700 1 -- 172.21.15.16:0/1537766416 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa324004390 con 0x7fa338131f40 2023-09-30T22:35:08.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.308+0000 7fa3377fe700 1 -- 172.21.15.16:0/1537766416 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa324010890 con 0x7fa338131f40 2023-09-30T22:35:08.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.308+0000 7fdfb79f0700 1 -- 172.21.15.16:0/1601583841 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdfb0138d80 msgr2=0x7fdfb013b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:08.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.308+0000 7fdfb79f0700 1 --2- 172.21.15.16:0/1601583841 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdfb0138d80 0x7fdfb013b160 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8009d90 tx=0x7fdfa800b330 comp rx=0 tx=0).stop 2023-09-30T22:35:08.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.308+0000 7fdfb79f0700 1 -- 172.21.15.16:0/1601583841 shutdown_connections 2023-09-30T22:35:08.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.308+0000 7fdfb79f0700 1 --2- 172.21.15.16:0/1601583841 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdfb0138d80 0x7fdfb013b160 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:08.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.308+0000 7fdfb79f0700 1 --2- 172.21.15.16:0/1601583841 >> v2:172.21.15.16:3300/0 conn(0x7fdfb009d630 0x7fdfb00a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:08.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.308+0000 7fdfb79f0700 1 -- 172.21.15.16:0/1601583841 >> 172.21.15.16:0/1601583841 conn(0x7fdfb00a3050 msgr2=0x7fdfb00a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:08.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.308+0000 7fa33eba6700 1 -- 172.21.15.16:0/1537766416 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa338131f40 msgr2=0x7fa33806e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:08.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.308+0000 7fa33eba6700 1 --2- 172.21.15.16:0/1537766416 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa338131f40 0x7fa33806e490 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fa324009920 tx=0x7fa324009c30 comp rx=0 tx=0).stop 2023-09-30T22:35:08.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.308+0000 7fa33eba6700 1 -- 172.21.15.16:0/1537766416 shutdown_connections 2023-09-30T22:35:08.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.308+0000 7fa33eba6700 1 --2- 172.21.15.16:0/1537766416 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa338131630 0x7fa338131a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:08.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.308+0000 7fa33eba6700 1 --2- 172.21.15.16:0/1537766416 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa338131f40 0x7fa33806e490 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:08.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.308+0000 7fa33eba6700 1 -- 172.21.15.16:0/1537766416 >> 172.21.15.16:0/1537766416 conn(0x7fa3380a2aa0 msgr2=0x7fa3380a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:08.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.308+0000 7fdfb79f0700 1 -- 172.21.15.16:0/1601583841 shutdown_connections 2023-09-30T22:35:08.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.309+0000 7fdfb79f0700 1 -- 172.21.15.16:0/1601583841 wait complete. 2023-09-30T22:35:08.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.309+0000 7fa33eba6700 1 -- 172.21.15.16:0/1537766416 shutdown_connections 2023-09-30T22:35:08.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.310+0000 7fa33eba6700 1 -- 172.21.15.16:0/1537766416 wait complete. 2023-09-30T22:35:08.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.310+0000 7fdfb79f0700 1 Processor -- start 2023-09-30T22:35:08.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.310+0000 7fdfb79f0700 1 -- start start 2023-09-30T22:35:08.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.310+0000 7fa33eba6700 1 Processor -- start 2023-09-30T22:35:08.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.311+0000 7fa33eba6700 1 -- start start 2023-09-30T22:35:08.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.311+0000 7fdfb79f0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdfb009d630 0x7fdfb0133750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:08.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.311+0000 7fdfb79f0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdfb00a0a30 0x7fdfb0133c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:08.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.311+0000 7fdfb79f0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfb0138d80 0x7fdfb01341d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:08.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.311+0000 7fdfb79f0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb0138b90 con 0x7fdfb0138d80 2023-09-30T22:35:08.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.311+0000 7fdfb79f0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb0136f90 con 0x7fdfb00a0a30 2023-09-30T22:35:08.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.311+0000 7fdfb79f0700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdfb0136620 con 0x7fdfb009d630 2023-09-30T22:35:08.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.311+0000 7fdfb578c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdfb009d630 0x7fdfb0133750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:08.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.311+0000 7fdfb5f8d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfb0138d80 0x7fdfb01341d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:08.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.311+0000 7fdfb578c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdfb009d630 0x7fdfb0133750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:55178/0 (socket says 172.21.15.16:55178) 2023-09-30T22:35:08.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.311+0000 7fdfb5f8d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfb0138d80 0x7fdfb01341d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49152/0 (socket says 172.21.15.16:49152) 2023-09-30T22:35:08.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.311+0000 7fdfb578c700 1 -- 172.21.15.16:0/1385364564 learned_addr learned my addr 172.21.15.16:0/1385364564 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:08.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.311+0000 7fdfb4f8b700 1 --2- 172.21.15.16:0/1385364564 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdfb00a0a30 0x7fdfb0133c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:08.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.311+0000 7fa33eba6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa33806e9d0 0x7fa33811b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:08.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.311+0000 7fa33eba6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa338131630 0x7fa33811b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:08.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.311+0000 7fa33eba6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa338131f40 0x7fa33811bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:08.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.311+0000 7fa33eba6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa33811deb0 con 0x7fa338131f40 2023-09-30T22:35:08.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.311+0000 7fa33eba6700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa33811d150 con 0x7fa338131630 2023-09-30T22:35:08.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.311+0000 7fa33eba6700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa33809f940 con 0x7fa33806e9d0 2023-09-30T22:35:08.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.311+0000 7fa33c942700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa33806e9d0 0x7fa33811b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:08.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.312+0000 7fa33c942700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa33806e9d0 0x7fa33811b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:55194/0 (socket says 172.21.15.16:55194) 2023-09-30T22:35:08.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.311+0000 7fdfb5f8d700 1 -- 172.21.15.16:0/1385364564 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdfb009d630 msgr2=0x7fdfb0133750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:08.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.311+0000 7fdfb5f8d700 1 --2- 172.21.15.16:0/1385364564 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdfb009d630 0x7fdfb0133750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:08.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.311+0000 7fdfb5f8d700 1 -- 172.21.15.16:0/1385364564 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdfb00a0a30 msgr2=0x7fdfb0133c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:08.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.311+0000 7fdfb5f8d700 1 --2- 172.21.15.16:0/1385364564 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdfb00a0a30 0x7fdfb0133c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:08.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.311+0000 7fdfb5f8d700 1 -- 172.21.15.16:0/1385364564 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfa0009550 con 0x7fdfb0138d80 2023-09-30T22:35:08.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.312+0000 7fa33c942700 1 -- 172.21.15.16:0/3405862079 learned_addr learned my addr 172.21.15.16:0/3405862079 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:08.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.312+0000 7fa33d143700 1 --2- 172.21.15.16:0/3405862079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa338131f40 0x7fa33811bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:08.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.312+0000 7fa337fff700 1 --2- 172.21.15.16:0/3405862079 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa338131630 0x7fa33811b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:08.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.312+0000 7fdfb5f8d700 1 --2- 172.21.15.16:0/1385364564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfb0138d80 0x7fdfb01341d0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fdfb00a1fb0 tx=0x7fdfa8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:08.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.312+0000 7fa33d143700 1 -- 172.21.15.16:0/3405862079 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa33806e9d0 msgr2=0x7fa33811b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:08.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.312+0000 7fa33d143700 1 --2- 172.21.15.16:0/3405862079 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa33806e9d0 0x7fa33811b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:08.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.312+0000 7fa33d143700 1 -- 172.21.15.16:0/3405862079 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa338131630 msgr2=0x7fa33811b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:08.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.312+0000 7fa33d143700 1 --2- 172.21.15.16:0/3405862079 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa338131630 0x7fa33811b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:08.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.312+0000 7fa33d143700 1 -- 172.21.15.16:0/3405862079 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa32c0094a0 con 0x7fa338131f40 2023-09-30T22:35:08.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.312+0000 7fa33d143700 1 --2- 172.21.15.16:0/3405862079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa338131f40 0x7fa33811bf20 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fa33000cb10 tx=0x7fa33000ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:08.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.312+0000 7fdfa5ffb700 1 -- 172.21.15.16:0/1385364564 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa8004720 con 0x7fdfb0138d80 2023-09-30T22:35:08.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.312+0000 7fdfa5ffb700 1 -- 172.21.15.16:0/1385364564 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfa80048c0 con 0x7fdfb0138d80 2023-09-30T22:35:08.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.312+0000 7fdfb79f0700 1 -- 172.21.15.16:0/1385364564 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfa8009a70 con 0x7fdfb0138d80 2023-09-30T22:35:08.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.312+0000 7fa3357fa700 1 -- 172.21.15.16:0/3405862079 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa33000a9e0 con 0x7fa338131f40 2023-09-30T22:35:08.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.312+0000 7fa3357fa700 1 -- 172.21.15.16:0/3405862079 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa33000ab40 con 0x7fa338131f40 2023-09-30T22:35:08.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.312+0000 7fa33eba6700 1 -- 172.21.15.16:0/3405862079 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa324009550 con 0x7fa338131f40 2023-09-30T22:35:08.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.313+0000 7fa3357fa700 1 -- 172.21.15.16:0/3405862079 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa330012420 con 0x7fa338131f40 2023-09-30T22:35:08.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.312+0000 7fdfa5ffb700 1 -- 172.21.15.16:0/1385364564 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa8012670 con 0x7fdfb0138d80 2023-09-30T22:35:08.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.313+0000 7fdfb79f0700 1 -- 172.21.15.16:0/1385364564 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfb0137a00 con 0x7fdfb0138d80 2023-09-30T22:35:08.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.313+0000 7fa33eba6700 1 -- 172.21.15.16:0/3405862079 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa33811e370 con 0x7fa338131f40 2023-09-30T22:35:08.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.314+0000 7fdfb79f0700 1 -- 172.21.15.16:0/1385364564 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfb00747f0 con 0x7fdfb0138d80 2023-09-30T22:35:08.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.314+0000 7fa33eba6700 1 -- 172.21.15.16:0/3405862079 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa338074ea0 con 0x7fa338131f40 2023-09-30T22:35:08.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.314+0000 7fa3357fa700 1 -- 172.21.15.16:0/3405862079 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa330012580 con 0x7fa338131f40 2023-09-30T22:35:08.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.314+0000 7fdfa5ffb700 1 -- 172.21.15.16:0/1385364564 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdfa80127d0 con 0x7fdfb0138d80 2023-09-30T22:35:08.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.315+0000 7fa3357fa700 1 -- 172.21.15.16:0/3405862079 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fa33003fdc0 con 0x7fa338131f40 2023-09-30T22:35:08.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.315+0000 7fdfa5ffb700 1 -- 172.21.15.16:0/1385364564 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fdfa8041db0 con 0x7fdfb0138d80 2023-09-30T22:35:08.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.319+0000 7fdfa5ffb700 1 -- 172.21.15.16:0/1385364564 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdfa8012e60 con 0x7fdfb0138d80 2023-09-30T22:35:08.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.327+0000 7fa3357fa700 1 -- 172.21.15.16:0/3405862079 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa3300228f0 con 0x7fa338131f40 2023-09-30T22:35:08.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.489+0000 7f44c171a700 1 -- 172.21.15.16:0/3609691694 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f44bc06eee0 con 0x7f44bc133ee0 2023-09-30T22:35:08.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.489+0000 7f44a37fe700 1 -- 172.21.15.16:0/3609691694 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f44bc06eee0 con 0x7f44bc133ee0 2023-09-30T22:35:08.490 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:35:08.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.491+0000 7f44c171a700 1 -- 172.21.15.16:0/3609691694 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f44bc133ee0 msgr2=0x7f44bc11ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:08.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.492+0000 7f44c171a700 1 --2- 172.21.15.16:0/3609691694 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f44bc133ee0 0x7f44bc11ac40 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f44b00098f0 tx=0x7f44b0004750 comp rx=0 tx=0).stop 2023-09-30T22:35:08.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.492+0000 7f44c171a700 1 -- 172.21.15.16:0/3609691694 shutdown_connections 2023-09-30T22:35:08.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.492+0000 7f44c171a700 1 --2- 172.21.15.16:0/3609691694 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f44bc133b10 0x7f44bc11a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:08.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.492+0000 7f44c171a700 1 --2- 172.21.15.16:0/3609691694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f44bc1397e0 0x7f44bc11b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:08.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.492+0000 7f44c171a700 1 --2- 172.21.15.16:0/3609691694 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f44bc133ee0 0x7f44bc11ac40 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:08.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.492+0000 7f44c171a700 1 -- 172.21.15.16:0/3609691694 >> 172.21.15.16:0/3609691694 conn(0x7f44bc0a2aa0 msgr2=0x7f44bc132cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:08.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.492+0000 7f44c171a700 1 -- 172.21.15.16:0/3609691694 shutdown_connections 2023-09-30T22:35:08.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.492+0000 7f44c171a700 1 -- 172.21.15.16:0/3609691694 wait complete. 2023-09-30T22:35:08.506 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:35:08.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.527+0000 7fdfb79f0700 1 -- 172.21.15.16:0/1385364564 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fdfb0136c30 con 0x7fdfb0138d80 2023-09-30T22:35:08.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.528+0000 7fdfa5ffb700 1 -- 172.21.15.16:0/1385364564 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdfa802d800 con 0x7fdfb0138d80 2023-09-30T22:35:08.528 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:35:08.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.530+0000 7fdfb79f0700 1 -- 172.21.15.16:0/1385364564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfb0138d80 msgr2=0x7fdfb01341d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:08.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.530+0000 7fdfb79f0700 1 --2- 172.21.15.16:0/1385364564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfb0138d80 0x7fdfb01341d0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fdfb00a1fb0 tx=0x7fdfa8004ad0 comp rx=0 tx=0).stop 2023-09-30T22:35:08.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.530+0000 7fdfb79f0700 1 -- 172.21.15.16:0/1385364564 shutdown_connections 2023-09-30T22:35:08.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.530+0000 7fdfb79f0700 1 --2- 172.21.15.16:0/1385364564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfb0138d80 0x7fdfb01341d0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:08.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.530+0000 7fdfb79f0700 1 --2- 172.21.15.16:0/1385364564 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdfb009d630 0x7fdfb0133750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:08.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.530+0000 7fdfb79f0700 1 --2- 172.21.15.16:0/1385364564 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdfb00a0a30 0x7fdfb0133c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:08.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.530+0000 7fdfb79f0700 1 -- 172.21.15.16:0/1385364564 >> 172.21.15.16:0/1385364564 conn(0x7fdfb00a3050 msgr2=0x7fdfb00a68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:08.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.531+0000 7fdfb79f0700 1 -- 172.21.15.16:0/1385364564 shutdown_connections 2023-09-30T22:35:08.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.531+0000 7fdfb79f0700 1 -- 172.21.15.16:0/1385364564 wait complete. 2023-09-30T22:35:08.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.535+0000 7fa33eba6700 1 -- 172.21.15.16:0/3405862079 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa33806b9e0 con 0x7fa338131f40 2023-09-30T22:35:08.536 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.536+0000 7fa3357fa700 1 -- 172.21.15.16:0/3405862079 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa330016030 con 0x7fa338131f40 2023-09-30T22:35:08.536 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:35:08.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.538+0000 7fa33eba6700 1 -- 172.21.15.16:0/3405862079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa338131f40 msgr2=0x7fa33811bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:08.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.538+0000 7fa33eba6700 1 --2- 172.21.15.16:0/3405862079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa338131f40 0x7fa33811bf20 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fa33000cb10 tx=0x7fa33000ce20 comp rx=0 tx=0).stop 2023-09-30T22:35:08.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.538+0000 7fa33eba6700 1 -- 172.21.15.16:0/3405862079 shutdown_connections 2023-09-30T22:35:08.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.538+0000 7fa33eba6700 1 --2- 172.21.15.16:0/3405862079 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa338131f40 0x7fa33811bf20 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:08.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.538+0000 7fa33eba6700 1 --2- 172.21.15.16:0/3405862079 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa33806e9d0 0x7fa33811b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:08.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.538+0000 7fa33eba6700 1 --2- 172.21.15.16:0/3405862079 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa338131630 0x7fa33811b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:08.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.538+0000 7fa33eba6700 1 -- 172.21.15.16:0/3405862079 >> 172.21.15.16:0/3405862079 conn(0x7fa3380a2aa0 msgr2=0x7fa338070fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:08.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.538+0000 7fa33eba6700 1 -- 172.21.15.16:0/3405862079 shutdown_connections 2023-09-30T22:35:08.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.538+0000 7fa33eba6700 1 -- 172.21.15.16:0/3405862079 wait complete. 2023-09-30T22:35:08.547 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:35:08.553 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:35:08.812 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:35:08.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.959+0000 7f6859cb1700 1 Processor -- start 2023-09-30T22:35:08.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.960+0000 7f6859cb1700 1 -- start start 2023-09-30T22:35:08.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.960+0000 7f6859cb1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6854133420 0x7f68541337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:08.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.960+0000 7f6859cb1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6854133e30 0x7f685412ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:08.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.960+0000 7f6859cb1700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f685406b990 con 0x7f6854134260 2023-09-30T22:35:08.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.960+0000 7f6859cb1700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68540a0590 con 0x7f6854133e30 2023-09-30T22:35:08.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.960+0000 7f6859cb1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f685409fb80 con 0x7f6854133420 2023-09-30T22:35:08.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.960+0000 7f68537fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6854133420 0x7f68541337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:08.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.960+0000 7f6852ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6854133e30 0x7f685412ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:08.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.960+0000 7f6852ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6854133e30 0x7f685412ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37642/0 (socket says 172.21.15.16:37642) 2023-09-30T22:35:08.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.960+0000 7f68537fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6854133420 0x7f68541337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:55208/0 (socket says 172.21.15.16:55208) 2023-09-30T22:35:08.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.960+0000 7f6852ffd700 1 -- 172.21.15.16:0/2813034568 learned_addr learned my addr 172.21.15.16:0/2813034568 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:08.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.961+0000 7f6852ffd700 1 -- 172.21.15.16:0/2813034568 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6854133420 msgr2=0x7f68541337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:08.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.961+0000 7f6852ffd700 1 --2- 172.21.15.16:0/2813034568 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6854133420 0x7f68541337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:08.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.961+0000 7f6852ffd700 1 -- 172.21.15.16:0/2813034568 >> v1:172.21.15.16:6789/0 conn(0x7f6854134260 legacy=0x7f68541304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:08.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.961+0000 7f68527fc700 1 -- 172.21.15.16:0/2813034568 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 243122421 0 0) 0x7f685406b990 con 0x7f6854134260 2023-09-30T22:35:08.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.961+0000 7f6852ffd700 1 -- 172.21.15.16:0/2813034568 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68540a0030 con 0x7f6854133e30 2023-09-30T22:35:08.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.961+0000 7f6852ffd700 1 --2- 172.21.15.16:0/2813034568 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6854133e30 0x7f685412ff90 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f6848009920 tx=0x7f6848009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52250897b21b5d26 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:08.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.961+0000 7f68527fc700 1 -- 172.21.15.16:0/2813034568 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6848011830 con 0x7f6854133e30 2023-09-30T22:35:08.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.961+0000 7f68527fc700 1 -- 172.21.15.16:0/2813034568 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6848011990 con 0x7f6854133e30 2023-09-30T22:35:08.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.961+0000 7f68527fc700 1 -- 172.21.15.16:0/2813034568 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6848011d20 con 0x7f6854133e30 2023-09-30T22:35:08.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.962+0000 7f6859cb1700 1 -- 172.21.15.16:0/2813034568 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6854133e30 msgr2=0x7f685412ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:08.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.962+0000 7f6859cb1700 1 --2- 172.21.15.16:0/2813034568 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6854133e30 0x7f685412ff90 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f6848009920 tx=0x7f6848009c30 comp rx=0 tx=0).stop 2023-09-30T22:35:08.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.962+0000 7f6859cb1700 1 -- 172.21.15.16:0/2813034568 shutdown_connections 2023-09-30T22:35:08.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.962+0000 7f6859cb1700 1 --2- 172.21.15.16:0/2813034568 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6854133420 0x7f68541337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:08.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.962+0000 7f6859cb1700 1 --2- 172.21.15.16:0/2813034568 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6854133e30 0x7f685412ff90 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:08.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.962+0000 7f6859cb1700 1 -- 172.21.15.16:0/2813034568 >> 172.21.15.16:0/2813034568 conn(0x7f68540a4210 msgr2=0x7f68540a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:08.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.962+0000 7f6859cb1700 1 -- 172.21.15.16:0/2813034568 shutdown_connections 2023-09-30T22:35:08.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.962+0000 7f6859cb1700 1 -- 172.21.15.16:0/2813034568 wait complete. 2023-09-30T22:35:08.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.963+0000 7f6859cb1700 1 Processor -- start 2023-09-30T22:35:08.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.963+0000 7f6859cb1700 1 -- start start 2023-09-30T22:35:08.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.963+0000 7f6859cb1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6854133420 0x7f685409c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:08.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.963+0000 7f6859cb1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6854133e30 0x7f685409c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:08.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.963+0000 7f6859cb1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6854134260 0x7f685409cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:08.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.963+0000 7f6859cb1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685409d950 con 0x7f6854134260 2023-09-30T22:35:08.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.963+0000 7f6859cb1700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f685409ded0 con 0x7f6854133420 2023-09-30T22:35:08.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.963+0000 7f6859cb1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f685409dce0 con 0x7f6854133e30 2023-09-30T22:35:08.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.963+0000 7f68537fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6854133420 0x7f685409c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:08.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.963+0000 7f6852ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6854133e30 0x7f685409c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:08.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.963+0000 7f68537fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6854133420 0x7f685409c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37652/0 (socket says 172.21.15.16:37652) 2023-09-30T22:35:08.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.963+0000 7f6852ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6854133e30 0x7f685409c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:55224/0 (socket says 172.21.15.16:55224) 2023-09-30T22:35:08.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.963+0000 7f68537fe700 1 -- 172.21.15.16:0/1643525658 learned_addr learned my addr 172.21.15.16:0/1643525658 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:08.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.963+0000 7f6853fff700 1 --2- 172.21.15.16:0/1643525658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6854134260 0x7f685409cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:08.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.964+0000 7f68537fe700 1 -- 172.21.15.16:0/1643525658 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6854133e30 msgr2=0x7f685409c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:08.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.964+0000 7f68537fe700 1 --2- 172.21.15.16:0/1643525658 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6854133e30 0x7f685409c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:08.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.964+0000 7f68537fe700 1 -- 172.21.15.16:0/1643525658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6854134260 msgr2=0x7f685409cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:08.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.964+0000 7f68537fe700 1 --2- 172.21.15.16:0/1643525658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6854134260 0x7f685409cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:08.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.964+0000 7f68537fe700 1 -- 172.21.15.16:0/1643525658 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6848009550 con 0x7f6854133420 2023-09-30T22:35:08.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.964+0000 7f6853fff700 1 --2- 172.21.15.16:0/1643525658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6854134260 0x7f685409cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:08.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.964+0000 7f68537fe700 1 --2- 172.21.15.16:0/1643525658 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6854133420 0x7f685409c480 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f684400e9b0 tx=0x7f684400ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:08.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.964+0000 7f683bfff700 1 -- 172.21.15.16:0/1643525658 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6844009710 con 0x7f6854133420 2023-09-30T22:35:08.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.964+0000 7f683bfff700 1 -- 172.21.15.16:0/1643525658 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f684400ee50 con 0x7f6854133420 2023-09-30T22:35:08.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.964+0000 7f683bfff700 1 -- 172.21.15.16:0/1643525658 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684400ca50 con 0x7f6854133420 2023-09-30T22:35:08.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.964+0000 7f6859cb1700 1 -- 172.21.15.16:0/1643525658 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6854131590 con 0x7f6854133420 2023-09-30T22:35:08.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.964+0000 7f6859cb1700 1 -- 172.21.15.16:0/1643525658 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f685411fe20 con 0x7f6854133420 2023-09-30T22:35:08.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.965+0000 7f683bfff700 1 -- 172.21.15.16:0/1643525658 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f684400cbb0 con 0x7f6854133420 2023-09-30T22:35:08.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.965+0000 7f6859cb1700 1 -- 172.21.15.16:0/1643525658 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6854074ea0 con 0x7f6854133420 2023-09-30T22:35:08.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.965+0000 7f683bfff700 1 -- 172.21.15.16:0/1643525658 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f684403f810 con 0x7f6854133420 2023-09-30T22:35:08.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:08.970+0000 7f683bfff700 1 -- 172.21.15.16:0/1643525658 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f68440232f0 con 0x7f6854133420 2023-09-30T22:35:09.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.173+0000 7f6859cb1700 1 -- 172.21.15.16:0/1643525658 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f685406b250 con 0x7f6854133420 2023-09-30T22:35:09.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.173+0000 7f683bfff700 1 -- 172.21.15.16:0/1643525658 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f684401ae80 con 0x7f6854133420 2023-09-30T22:35:09.174 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:35:09.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.175+0000 7f6859cb1700 1 -- 172.21.15.16:0/1643525658 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6854133420 msgr2=0x7f685409c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:09.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.175+0000 7f6859cb1700 1 --2- 172.21.15.16:0/1643525658 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6854133420 0x7f685409c480 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f684400e9b0 tx=0x7f684400ecc0 comp rx=0 tx=0).stop 2023-09-30T22:35:09.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.175+0000 7f6859cb1700 1 -- 172.21.15.16:0/1643525658 shutdown_connections 2023-09-30T22:35:09.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.175+0000 7f6859cb1700 1 --2- 172.21.15.16:0/1643525658 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6854133420 0x7f685409c480 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:09.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.175+0000 7f6859cb1700 1 --2- 172.21.15.16:0/1643525658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6854134260 0x7f685409cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:09.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.175+0000 7f6859cb1700 1 --2- 172.21.15.16:0/1643525658 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6854133e30 0x7f685409c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:09.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.175+0000 7f6859cb1700 1 -- 172.21.15.16:0/1643525658 >> 172.21.15.16:0/1643525658 conn(0x7f68540a4210 msgr2=0x7f6854130460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:09.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.176+0000 7f6859cb1700 1 -- 172.21.15.16:0/1643525658 shutdown_connections 2023-09-30T22:35:09.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.176+0000 7f6859cb1700 1 -- 172.21.15.16:0/1643525658 wait complete. 2023-09-30T22:35:09.189 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:35:09.508 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:35:09.548 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:35:09.554 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:35:09.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.662+0000 7ff54c6cd700 1 Processor -- start 2023-09-30T22:35:09.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.662+0000 7ff54c6cd700 1 -- start start 2023-09-30T22:35:09.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.663+0000 7ff54c6cd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff54406a770 0x7ff54406ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:09.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.663+0000 7ff54c6cd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff544131d10 0x7ff544139860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:09.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.663+0000 7ff54c6cd700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff54409ece0 con 0x7ff54406b080 2023-09-30T22:35:09.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.663+0000 7ff54c6cd700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff54409e760 con 0x7ff54406a770 2023-09-30T22:35:09.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.663+0000 7ff54c6cd700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff54409e040 con 0x7ff544131d10 2023-09-30T22:35:09.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.663+0000 7ff54a469700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff54406a770 0x7ff54406ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:09.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.663+0000 7ff549c68700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff54406b080 0x7ff544131600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51718/0 (socket says 172.21.15.16:51718) 2023-09-30T22:35:09.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.663+0000 7ff549c68700 1 -- 172.21.15.16:0/1816099807 learned_addr learned my addr 172.21.15.16:0/1816099807 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:09.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.663+0000 7ff54ac6a700 1 --2- 172.21.15.16:0/1816099807 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff544131d10 0x7ff544139860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:09.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.663+0000 7ff54a469700 1 -- 172.21.15.16:0/1816099807 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff544131d10 msgr2=0x7ff544139860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:09.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.663+0000 7ff54a469700 1 --2- 172.21.15.16:0/1816099807 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff544131d10 0x7ff544139860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:09.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.663+0000 7ff54a469700 1 -- 172.21.15.16:0/1816099807 >> v1:172.21.15.16:6789/0 conn(0x7ff54406b080 legacy=0x7ff544131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:09.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.663+0000 7ff54a469700 1 -- 172.21.15.16:0/1816099807 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff54409f630 con 0x7ff54406a770 2023-09-30T22:35:09.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.664+0000 7ff54a469700 1 --2- 172.21.15.16:0/1816099807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff54406a770 0x7ff54406ab40 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7ff540009840 tx=0x7ff540009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cd911446a9777bb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:09.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.664+0000 7ff549467700 1 -- 172.21.15.16:0/1816099807 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff540004230 con 0x7ff54406a770 2023-09-30T22:35:09.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.664+0000 7ff549467700 1 -- 172.21.15.16:0/1816099807 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff540004390 con 0x7ff54406a770 2023-09-30T22:35:09.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.664+0000 7ff549467700 1 -- 172.21.15.16:0/1816099807 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5400036a0 con 0x7ff54406a770 2023-09-30T22:35:09.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.664+0000 7ff54c6cd700 1 -- 172.21.15.16:0/1816099807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff54406a770 msgr2=0x7ff54406ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:09.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.664+0000 7ff54c6cd700 1 --2- 172.21.15.16:0/1816099807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff54406a770 0x7ff54406ab40 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7ff540009840 tx=0x7ff540009b50 comp rx=0 tx=0).stop 2023-09-30T22:35:09.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.665+0000 7ff54c6cd700 1 -- 172.21.15.16:0/1816099807 shutdown_connections 2023-09-30T22:35:09.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.665+0000 7ff54c6cd700 1 --2- 172.21.15.16:0/1816099807 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff544131d10 0x7ff544139860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:09.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.665+0000 7ff54c6cd700 1 --2- 172.21.15.16:0/1816099807 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff54406a770 0x7ff54406ab40 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:09.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.665+0000 7ff54c6cd700 1 -- 172.21.15.16:0/1816099807 >> 172.21.15.16:0/1816099807 conn(0x7ff544066f50 msgr2=0x7ff54409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:09.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.665+0000 7ff54c6cd700 1 -- 172.21.15.16:0/1816099807 shutdown_connections 2023-09-30T22:35:09.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.665+0000 7ff54c6cd700 1 -- 172.21.15.16:0/1816099807 wait complete. 2023-09-30T22:35:09.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.666+0000 7ff54c6cd700 1 Processor -- start 2023-09-30T22:35:09.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.666+0000 7ff54c6cd700 1 -- start start 2023-09-30T22:35:09.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.666+0000 7ff54c6cd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff54406a770 0x7ff5441c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:09.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.666+0000 7ff54c6cd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff54406b080 0x7ff5441c6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:09.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.666+0000 7ff54c6cd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff544131d10 0x7ff5441c6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:09.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.666+0000 7ff54c6cd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff544121a50 con 0x7ff544131d10 2023-09-30T22:35:09.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.666+0000 7ff54c6cd700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff544123dc0 con 0x7ff54406b080 2023-09-30T22:35:09.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.666+0000 7ff54c6cd700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff544131180 con 0x7ff54406a770 2023-09-30T22:35:09.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.667+0000 7ff549c68700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff54406b080 0x7ff5441c6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:09.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.667+0000 7ff549c68700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff54406b080 0x7ff5441c6500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37678/0 (socket says 172.21.15.16:37678) 2023-09-30T22:35:09.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.667+0000 7ff549c68700 1 -- 172.21.15.16:0/3210303626 learned_addr learned my addr 172.21.15.16:0/3210303626 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:09.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.667+0000 7ff54ac6a700 1 --2- 172.21.15.16:0/3210303626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff544131d10 0x7ff5441c6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:09.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.667+0000 7ff54a469700 1 --2- 172.21.15.16:0/3210303626 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff54406a770 0x7ff5441c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:09.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.667+0000 7ff549c68700 1 -- 172.21.15.16:0/3210303626 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff54406a770 msgr2=0x7ff5441c5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:09.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.667+0000 7ff549c68700 1 --2- 172.21.15.16:0/3210303626 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff54406a770 0x7ff5441c5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:09.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.667+0000 7ff549c68700 1 -- 172.21.15.16:0/3210303626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff544131d10 msgr2=0x7ff5441c6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:09.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.667+0000 7ff549c68700 1 --2- 172.21.15.16:0/3210303626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff544131d10 0x7ff5441c6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:09.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.667+0000 7ff549c68700 1 -- 172.21.15.16:0/3210303626 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5400094a0 con 0x7ff54406b080 2023-09-30T22:35:09.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.667+0000 7ff549c68700 1 --2- 172.21.15.16:0/3210303626 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff54406b080 0x7ff5441c6500 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7ff53400c7e0 tx=0x7ff53400caf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:09.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.667+0000 7ff53affd700 1 -- 172.21.15.16:0/3210303626 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff534004c80 con 0x7ff54406b080 2023-09-30T22:35:09.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.667+0000 7ff53affd700 1 -- 172.21.15.16:0/3210303626 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff534004de0 con 0x7ff54406b080 2023-09-30T22:35:09.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.667+0000 7ff53affd700 1 -- 172.21.15.16:0/3210303626 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff534007310 con 0x7ff54406b080 2023-09-30T22:35:09.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.667+0000 7ff54c6cd700 1 -- 172.21.15.16:0/3210303626 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff544122310 con 0x7ff54406b080 2023-09-30T22:35:09.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.667+0000 7ff54c6cd700 1 -- 172.21.15.16:0/3210303626 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff544124830 con 0x7ff54406b080 2023-09-30T22:35:09.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.668+0000 7ff54c6cd700 1 -- 172.21.15.16:0/3210303626 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff544074ea0 con 0x7ff54406b080 2023-09-30T22:35:09.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.669+0000 7ff53affd700 1 -- 172.21.15.16:0/3210303626 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff534007470 con 0x7ff54406b080 2023-09-30T22:35:09.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.669+0000 7ff53affd700 1 -- 172.21.15.16:0/3210303626 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7ff53403fa60 con 0x7ff54406b080 2023-09-30T22:35:09.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.678+0000 7ff53affd700 1 -- 172.21.15.16:0/3210303626 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff534007720 con 0x7ff54406b080 2023-09-30T22:35:09.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.701+0000 7f05b4104700 1 Processor -- start 2023-09-30T22:35:09.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.701+0000 7f05b4104700 1 -- start start 2023-09-30T22:35:09.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.702+0000 7f05b4104700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f05ac12cf60 0x7f05ac06f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:09.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.702+0000 7f05b4104700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f05ac12f8b0 0x7f05ac12fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:09.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.702+0000 7f05b4104700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05ac12cd90 con 0x7f05ac130260 2023-09-30T22:35:09.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.702+0000 7f05b4104700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05ac12d330 con 0x7f05ac12f8b0 2023-09-30T22:35:09.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.702+0000 7f05b4104700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f05ac1306e0 con 0x7f05ac12cf60 2023-09-30T22:35:09.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.702+0000 7f05b1ea0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f05ac12cf60 0x7f05ac06f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:09.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.702+0000 7f05b1ea0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f05ac12cf60 0x7f05ac06f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:55234/0 (socket says 172.21.15.16:55234) 2023-09-30T22:35:09.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.702+0000 7f05b26a1700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f05ac130260 0x7f05ac136c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51726/0 (socket says 172.21.15.16:51726) 2023-09-30T22:35:09.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.702+0000 7f05b1ea0700 1 -- 172.21.15.16:0/296435883 learned_addr learned my addr 172.21.15.16:0/296435883 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:09.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.702+0000 7f05b169f700 1 --2- 172.21.15.16:0/296435883 >> v2:172.21.15.16:3300/0 conn(0x7f05ac12f8b0 0x7f05ac12fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:09.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.703+0000 7f05b169f700 1 -- 172.21.15.16:0/296435883 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f05ac12cf60 msgr2=0x7f05ac06f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:09.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.703+0000 7f05b169f700 1 --2- 172.21.15.16:0/296435883 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f05ac12cf60 0x7f05ac06f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:09.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.703+0000 7f05b0e9e700 1 -- 172.21.15.16:0/296435883 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2711128960 0 0) 0x7f05ac12cd90 con 0x7f05ac130260 2023-09-30T22:35:09.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.703+0000 7f05b169f700 1 -- 172.21.15.16:0/296435883 >> v1:172.21.15.16:6789/0 conn(0x7f05ac130260 legacy=0x7f05ac136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:09.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.703+0000 7f05b169f700 1 -- 172.21.15.16:0/296435883 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05ac071780 con 0x7f05ac12f8b0 2023-09-30T22:35:09.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.703+0000 7f05b1ea0700 1 --2- 172.21.15.16:0/296435883 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f05ac12cf60 0x7f05ac06f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:09.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.703+0000 7f05b169f700 1 --2- 172.21.15.16:0/296435883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05ac12f8b0 0x7f05ac12fd20 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f059c009840 tx=0x7f059c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1b2a64ca8bfeaaa server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:09.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.703+0000 7f102c07d700 1 Processor -- start 2023-09-30T22:35:09.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.703+0000 7f05b0e9e700 1 -- 172.21.15.16:0/296435883 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059c011830 con 0x7f05ac12f8b0 2023-09-30T22:35:09.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.703+0000 7f05b0e9e700 1 -- 172.21.15.16:0/296435883 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f059c011990 con 0x7f05ac12f8b0 2023-09-30T22:35:09.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.703+0000 7f05b0e9e700 1 -- 172.21.15.16:0/296435883 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059c011d20 con 0x7f05ac12f8b0 2023-09-30T22:35:09.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.704+0000 7f05b4104700 1 -- 172.21.15.16:0/296435883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05ac12f8b0 msgr2=0x7f05ac12fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:09.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.704+0000 7f05b4104700 1 --2- 172.21.15.16:0/296435883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05ac12f8b0 0x7f05ac12fd20 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f059c009840 tx=0x7f059c009b50 comp rx=0 tx=0).stop 2023-09-30T22:35:09.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.703+0000 7f102c07d700 1 -- start start 2023-09-30T22:35:09.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.704+0000 7f102c07d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f102406e570 0x7f102406e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:09.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.704+0000 7f102c07d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f102406ee80 0x7f1024134d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:09.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.704+0000 7f102c07d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1024070ee0 con 0x7f102409eca0 2023-09-30T22:35:09.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.704+0000 7f102c07d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1024071480 con 0x7f102406ee80 2023-09-30T22:35:09.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.704+0000 7f102c07d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1024071b90 con 0x7f102406e570 2023-09-30T22:35:09.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.704+0000 7f1029618700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f102409eca0 0x7f1024134610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51736/0 (socket says 172.21.15.16:51736) 2023-09-30T22:35:09.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.704+0000 7f102a61a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f102406ee80 0x7f1024134d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:09.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.704+0000 7f1029618700 1 -- 172.21.15.16:0/2561745878 learned_addr learned my addr 172.21.15.16:0/2561745878 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:09.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.704+0000 7f05b4104700 1 -- 172.21.15.16:0/296435883 shutdown_connections 2023-09-30T22:35:09.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.704+0000 7f05b4104700 1 --2- 172.21.15.16:0/296435883 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f05ac12cf60 0x7f05ac06f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:09.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.704+0000 7f05b4104700 1 --2- 172.21.15.16:0/296435883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05ac12f8b0 0x7f05ac12fd20 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:09.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.704+0000 7f05b4104700 1 -- 172.21.15.16:0/296435883 >> 172.21.15.16:0/296435883 conn(0x7f05ac0a2aa0 msgr2=0x7f05ac0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:09.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.704+0000 7f1029e19700 1 --2- 172.21.15.16:0/2561745878 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f102406e570 0x7f102406e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:09.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.704+0000 7f1028e17700 1 -- 172.21.15.16:0/2561745878 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2334609957 0 0) 0x7f1024070ee0 con 0x7f102409eca0 2023-09-30T22:35:09.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.704+0000 7f1028e17700 1 -- 172.21.15.16:0/2561745878 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10100035b0 con 0x7f102409eca0 2023-09-30T22:35:09.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.704+0000 7f05b4104700 1 -- 172.21.15.16:0/296435883 shutdown_connections 2023-09-30T22:35:09.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.704+0000 7f102a61a700 1 -- 172.21.15.16:0/2561745878 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f102406e570 msgr2=0x7f102406e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:09.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.704+0000 7f102a61a700 1 --2- 172.21.15.16:0/2561745878 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f102406e570 0x7f102406e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:09.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.705+0000 7f102a61a700 1 -- 172.21.15.16:0/2561745878 >> v1:172.21.15.16:6789/0 conn(0x7f102409eca0 legacy=0x7f1024134610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:09.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.705+0000 7f102a61a700 1 -- 172.21.15.16:0/2561745878 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10240711a0 con 0x7f102406ee80 2023-09-30T22:35:09.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.705+0000 7f05b4104700 1 -- 172.21.15.16:0/296435883 wait complete. 2023-09-30T22:35:09.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.705+0000 7f102a61a700 1 --2- 172.21.15.16:0/2561745878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f102406ee80 0x7f1024134d20 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f101c009d90 tx=0x7f101c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=507eb6eb9c3cbbbd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:09.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.705+0000 7f1028e17700 1 -- 172.21.15.16:0/2561745878 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101c00e070 con 0x7f102406ee80 2023-09-30T22:35:09.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.705+0000 7f1028e17700 1 -- 172.21.15.16:0/2561745878 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f101c00bb20 con 0x7f102406ee80 2023-09-30T22:35:09.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.705+0000 7f1028e17700 1 -- 172.21.15.16:0/2561745878 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101c012430 con 0x7f102406ee80 2023-09-30T22:35:09.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.706+0000 7f102c07d700 1 -- 172.21.15.16:0/2561745878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f102406ee80 msgr2=0x7f1024134d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:09.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.706+0000 7f102c07d700 1 --2- 172.21.15.16:0/2561745878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f102406ee80 0x7f1024134d20 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f101c009d90 tx=0x7f101c00b330 comp rx=0 tx=0).stop 2023-09-30T22:35:09.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.706+0000 7f102c07d700 1 -- 172.21.15.16:0/2561745878 shutdown_connections 2023-09-30T22:35:09.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.706+0000 7f102c07d700 1 --2- 172.21.15.16:0/2561745878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f102406ee80 0x7f1024134d20 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:09.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.706+0000 7f102c07d700 1 --2- 172.21.15.16:0/2561745878 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f102406e570 0x7f102406e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:09.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.706+0000 7f102c07d700 1 -- 172.21.15.16:0/2561745878 >> 172.21.15.16:0/2561745878 conn(0x7f10240a2aa0 msgr2=0x7f10240a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:09.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.706+0000 7f05b4104700 1 Processor -- start 2023-09-30T22:35:09.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.706+0000 7f05b4104700 1 -- start start 2023-09-30T22:35:09.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.706+0000 7f102c07d700 1 -- 172.21.15.16:0/2561745878 shutdown_connections 2023-09-30T22:35:09.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.706+0000 7f102c07d700 1 -- 172.21.15.16:0/2561745878 wait complete. 2023-09-30T22:35:09.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.707+0000 7f05b4104700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05ac12cf60 0x7f05ac1309e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:09.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.707+0000 7f05b4104700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f05ac12f8b0 0x7f05ac130f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:09.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.707+0000 7f05b4104700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f05ac130260 0x7f05ac131460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:09.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.707+0000 7f05b4104700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05ac067cd0 con 0x7f05ac12cf60 2023-09-30T22:35:09.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.707+0000 7f05b4104700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05ac133c10 con 0x7f05ac12f8b0 2023-09-30T22:35:09.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.707+0000 7f05b4104700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f05ac09e900 con 0x7f05ac130260 2023-09-30T22:35:09.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.707+0000 7f05b1ea0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05ac12cf60 0x7f05ac1309e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:09.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.707+0000 7f05b1ea0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05ac12cf60 0x7f05ac1309e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49226/0 (socket says 172.21.15.16:49226) 2023-09-30T22:35:09.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.707+0000 7f05b1ea0700 1 -- 172.21.15.16:0/2327735428 learned_addr learned my addr 172.21.15.16:0/2327735428 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:09.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.707+0000 7f05b169f700 1 --2- 172.21.15.16:0/2327735428 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f05ac12f8b0 0x7f05ac130f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:09.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.707+0000 7f05b26a1700 1 --2- 172.21.15.16:0/2327735428 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f05ac130260 0x7f05ac131460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:09.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.707+0000 7f05b1ea0700 1 -- 172.21.15.16:0/2327735428 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f05ac130260 msgr2=0x7f05ac131460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:09.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.707+0000 7f05b1ea0700 1 --2- 172.21.15.16:0/2327735428 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f05ac130260 0x7f05ac131460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:09.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.707+0000 7f05b1ea0700 1 -- 172.21.15.16:0/2327735428 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f05ac12f8b0 msgr2=0x7f05ac130f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:09.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.707+0000 7f05b1ea0700 1 --2- 172.21.15.16:0/2327735428 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f05ac12f8b0 0x7f05ac130f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:09.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.707+0000 7f05b1ea0700 1 -- 172.21.15.16:0/2327735428 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a8009550 con 0x7f05ac12cf60 2023-09-30T22:35:09.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.707+0000 7f05b1ea0700 1 --2- 172.21.15.16:0/2327735428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05ac12cf60 0x7f05ac1309e0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f05a800ebe0 tx=0x7f05a800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:09.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.708+0000 7f102c07d700 1 Processor -- start 2023-09-30T22:35:09.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.708+0000 7f05a27fc700 1 -- 172.21.15.16:0/2327735428 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a800c850 con 0x7f05ac12cf60 2023-09-30T22:35:09.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.708+0000 7f102c07d700 1 -- start start 2023-09-30T22:35:09.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.708+0000 7f05a27fc700 1 -- 172.21.15.16:0/2327735428 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05a800c9b0 con 0x7f05ac12cf60 2023-09-30T22:35:09.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.708+0000 7f05b4104700 1 -- 172.21.15.16:0/2327735428 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f059c0094a0 con 0x7f05ac12cf60 2023-09-30T22:35:09.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.708+0000 7f05a27fc700 1 -- 172.21.15.16:0/2327735428 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a800cd40 con 0x7f05ac12cf60 2023-09-30T22:35:09.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.708+0000 7f05b4104700 1 -- 172.21.15.16:0/2327735428 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05ac131b10 con 0x7f05ac12cf60 2023-09-30T22:35:09.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.708+0000 7f102c07d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f102406ee80 0x7f1024126650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:09.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.708+0000 7f102c07d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f102413dbc0 0x7f1024124e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:09.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.708+0000 7f102c07d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1024125340 0x7f10241257b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:09.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.708+0000 7f102c07d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f102413dfb0 con 0x7f102413dbc0 2023-09-30T22:35:09.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.708+0000 7f102c07d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f102413d730 con 0x7f1024125340 2023-09-30T22:35:09.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.708+0000 7f102c07d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1024070610 con 0x7f102406ee80 2023-09-30T22:35:09.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.708+0000 7f102a61a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1024125340 0x7f10241257b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:09.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.708+0000 7f102a61a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1024125340 0x7f10241257b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37698/0 (socket says 172.21.15.16:37698) 2023-09-30T22:35:09.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.708+0000 7f102a61a700 1 -- 172.21.15.16:0/464564583 learned_addr learned my addr 172.21.15.16:0/464564583 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:09.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.709+0000 7f1029618700 1 --2- 172.21.15.16:0/464564583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f102413dbc0 0x7f1024124e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:09.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.709+0000 7f1029e19700 1 --2- 172.21.15.16:0/464564583 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f102406ee80 0x7f1024126650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:09.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.709+0000 7f102a61a700 1 -- 172.21.15.16:0/464564583 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f102406ee80 msgr2=0x7f1024126650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:09.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.709+0000 7f102a61a700 1 --2- 172.21.15.16:0/464564583 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f102406ee80 0x7f1024126650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:09.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.709+0000 7f102a61a700 1 -- 172.21.15.16:0/464564583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f102413dbc0 msgr2=0x7f1024124e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:09.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.709+0000 7f102a61a700 1 --2- 172.21.15.16:0/464564583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f102413dbc0 0x7f1024124e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:09.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.709+0000 7f102a61a700 1 -- 172.21.15.16:0/464564583 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f101c009a70 con 0x7f1024125340 2023-09-30T22:35:09.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.709+0000 7f05a27fc700 1 -- 172.21.15.16:0/2327735428 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f05a8013470 con 0x7f05ac12cf60 2023-09-30T22:35:09.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.709+0000 7f102a61a700 1 --2- 172.21.15.16:0/464564583 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1024125340 0x7f10241257b0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f101c00b680 tx=0x7f101c004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:09.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.709+0000 7f05a27fc700 1 -- 172.21.15.16:0/2327735428 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f05a8011350 con 0x7f05ac12cf60 2023-09-30T22:35:09.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.709+0000 7f05b4104700 1 -- 172.21.15.16:0/2327735428 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05ac074ea0 con 0x7f05ac12cf60 2023-09-30T22:35:09.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.709+0000 7f101a7fc700 1 -- 172.21.15.16:0/464564583 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101c0129f0 con 0x7f1024125340 2023-09-30T22:35:09.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.709+0000 7f101a7fc700 1 -- 172.21.15.16:0/464564583 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f101c020070 con 0x7f1024125340 2023-09-30T22:35:09.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.709+0000 7f102c07d700 1 -- 172.21.15.16:0/464564583 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f102413b8c0 con 0x7f1024125340 2023-09-30T22:35:09.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.709+0000 7f101a7fc700 1 -- 172.21.15.16:0/464564583 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101c012db0 con 0x7f1024125340 2023-09-30T22:35:09.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.710+0000 7f101a7fc700 1 -- 172.21.15.16:0/464564583 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f101c004040 con 0x7f1024125340 2023-09-30T22:35:09.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.710+0000 7f102c07d700 1 -- 172.21.15.16:0/464564583 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f102413c3b0 con 0x7f1024125340 2023-09-30T22:35:09.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.710+0000 7f101a7fc700 1 -- 172.21.15.16:0/464564583 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f101c022070 con 0x7f1024125340 2023-09-30T22:35:09.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.711+0000 7f102c07d700 1 -- 172.21.15.16:0/464564583 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1024074ea0 con 0x7f1024125340 2023-09-30T22:35:09.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.714+0000 7f05a27fc700 1 -- 172.21.15.16:0/2327735428 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f05a8017020 con 0x7f05ac12cf60 2023-09-30T22:35:09.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.717+0000 7f101a7fc700 1 -- 172.21.15.16:0/464564583 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f101c046060 con 0x7f1024125340 2023-09-30T22:35:09.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.890+0000 7ff54c6cd700 1 -- 172.21.15.16:0/3210303626 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff54406c360 con 0x7ff54406b080 2023-09-30T22:35:09.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.891+0000 7ff53affd700 1 -- 172.21.15.16:0/3210303626 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff534011360 con 0x7ff54406b080 2023-09-30T22:35:09.891 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:35:09.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.893+0000 7ff54c6cd700 1 -- 172.21.15.16:0/3210303626 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff54406b080 msgr2=0x7ff5441c6500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:09.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.893+0000 7ff54c6cd700 1 --2- 172.21.15.16:0/3210303626 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff54406b080 0x7ff5441c6500 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7ff53400c7e0 tx=0x7ff53400caf0 comp rx=0 tx=0).stop 2023-09-30T22:35:09.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.893+0000 7ff54c6cd700 1 -- 172.21.15.16:0/3210303626 shutdown_connections 2023-09-30T22:35:09.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.893+0000 7ff54c6cd700 1 --2- 172.21.15.16:0/3210303626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff544131d10 0x7ff5441c6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:09.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.893+0000 7ff54c6cd700 1 --2- 172.21.15.16:0/3210303626 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff54406a770 0x7ff5441c5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:09.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.893+0000 7ff54c6cd700 1 --2- 172.21.15.16:0/3210303626 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff54406b080 0x7ff5441c6500 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:09.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.893+0000 7ff54c6cd700 1 -- 172.21.15.16:0/3210303626 >> 172.21.15.16:0/3210303626 conn(0x7ff544066f50 msgr2=0x7ff544070140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:09.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.893+0000 7ff54c6cd700 1 -- 172.21.15.16:0/3210303626 shutdown_connections 2023-09-30T22:35:09.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.893+0000 7ff54c6cd700 1 -- 172.21.15.16:0/3210303626 wait complete. 2023-09-30T22:35:09.914 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:35:09.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.961+0000 7f05b4104700 1 -- 172.21.15.16:0/2327735428 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f05ac132bf0 con 0x7f05ac12cf60 2023-09-30T22:35:09.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.962+0000 7f05a27fc700 1 -- 172.21.15.16:0/2327735428 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f05ac132bf0 con 0x7f05ac12cf60 2023-09-30T22:35:09.962 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:35:09.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.964+0000 7f05b4104700 1 -- 172.21.15.16:0/2327735428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05ac12cf60 msgr2=0x7f05ac1309e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:09.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.964+0000 7f05b4104700 1 --2- 172.21.15.16:0/2327735428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05ac12cf60 0x7f05ac1309e0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f05a800ebe0 tx=0x7f05a800eef0 comp rx=0 tx=0).stop 2023-09-30T22:35:09.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.964+0000 7f05b4104700 1 -- 172.21.15.16:0/2327735428 shutdown_connections 2023-09-30T22:35:09.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.964+0000 7f05b4104700 1 --2- 172.21.15.16:0/2327735428 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f05ac130260 0x7f05ac131460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:09.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.964+0000 7f05b4104700 1 --2- 172.21.15.16:0/2327735428 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f05ac12cf60 0x7f05ac1309e0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:09.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.964+0000 7f05b4104700 1 --2- 172.21.15.16:0/2327735428 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f05ac12f8b0 0x7f05ac130f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:09.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.964+0000 7f05b4104700 1 -- 172.21.15.16:0/2327735428 >> 172.21.15.16:0/2327735428 conn(0x7f05ac0a2aa0 msgr2=0x7f05ac0a0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:09.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.964+0000 7f05b4104700 1 -- 172.21.15.16:0/2327735428 shutdown_connections 2023-09-30T22:35:09.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.964+0000 7f05b4104700 1 -- 172.21.15.16:0/2327735428 wait complete. 2023-09-30T22:35:09.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.977+0000 7f102c07d700 1 -- 172.21.15.16:0/464564583 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f102413bf10 con 0x7f1024125340 2023-09-30T22:35:09.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.977+0000 7f101a7fc700 1 -- 172.21.15.16:0/464564583 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f101c01a740 con 0x7f1024125340 2023-09-30T22:35:09.978 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:35:09.979 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:35:09.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.979+0000 7f102c07d700 1 -- 172.21.15.16:0/464564583 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1024125340 msgr2=0x7f10241257b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:09.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.979+0000 7f102c07d700 1 --2- 172.21.15.16:0/464564583 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1024125340 0x7f10241257b0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f101c00b680 tx=0x7f101c004ad0 comp rx=0 tx=0).stop 2023-09-30T22:35:09.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.979+0000 7f102c07d700 1 -- 172.21.15.16:0/464564583 shutdown_connections 2023-09-30T22:35:09.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.979+0000 7f102c07d700 1 --2- 172.21.15.16:0/464564583 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1024125340 0x7f10241257b0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:09.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.979+0000 7f102c07d700 1 --2- 172.21.15.16:0/464564583 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f102406ee80 0x7f1024126650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:09.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.979+0000 7f102c07d700 1 --2- 172.21.15.16:0/464564583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f102413dbc0 0x7f1024124e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:09.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.979+0000 7f102c07d700 1 -- 172.21.15.16:0/464564583 >> 172.21.15.16:0/464564583 conn(0x7f10240a2aa0 msgr2=0x7f10240a1410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:09.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.979+0000 7f102c07d700 1 -- 172.21.15.16:0/464564583 shutdown_connections 2023-09-30T22:35:09.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:09.980+0000 7f102c07d700 1 -- 172.21.15.16:0/464564583 wait complete. 2023-09-30T22:35:09.993 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:35:10.191 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:35:10.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.337+0000 7f946fb47700 1 Processor -- start 2023-09-30T22:35:10.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.337+0000 7f946fb47700 1 -- start start 2023-09-30T22:35:10.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.338+0000 7f946fb47700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f94681338a0 0x7f9468133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:10.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.338+0000 7f946fb47700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f946809de50 0x7f946809e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:10.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.338+0000 7f946fb47700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f946809e760 0x7f946813a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:10.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.338+0000 7f946fb47700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f946806fef0 con 0x7f946809e760 2023-09-30T22:35:10.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.338+0000 7f946fb47700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f946806ef30 con 0x7f94681338a0 2023-09-30T22:35:10.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.338+0000 7f946fb47700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f94681334b0 con 0x7f946809de50 2023-09-30T22:35:10.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.338+0000 7f946d0e2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f946809de50 0x7f946809e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:10.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.338+0000 7f946d8e3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f94681338a0 0x7f9468133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:10.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.338+0000 7f946e0e4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f946809e760 0x7f946813a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:10.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.338+0000 7f946d0e2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f946809de50 0x7f946809e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:55262/0 (socket says 172.21.15.16:55262) 2023-09-30T22:35:10.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.338+0000 7f946d0e2700 1 -- 172.21.15.16:0/3024749922 learned_addr learned my addr 172.21.15.16:0/3024749922 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:10.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.338+0000 7f946d8e3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f94681338a0 0x7f9468133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37700/0 (socket says 172.21.15.16:37700) 2023-09-30T22:35:10.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.338+0000 7f946d0e2700 1 -- 172.21.15.16:0/3024749922 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f94681338a0 msgr2=0x7f9468133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:10.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.338+0000 7f946d0e2700 1 --2- 172.21.15.16:0/3024749922 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f94681338a0 0x7f9468133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:10.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.338+0000 7f946d0e2700 1 -- 172.21.15.16:0/3024749922 >> v2:172.21.15.16:3300/0 conn(0x7f946809e760 msgr2=0x7f946813a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:10.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.338+0000 7f946d0e2700 1 --2- 172.21.15.16:0/3024749922 >> v2:172.21.15.16:3300/0 conn(0x7f946809e760 0x7f946813a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:10.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.338+0000 7f946d0e2700 1 -- 172.21.15.16:0/3024749922 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f946806fc30 con 0x7f946809de50 2023-09-30T22:35:10.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.338+0000 7f946d8e3700 1 --2- 172.21.15.16:0/3024749922 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f94681338a0 0x7f9468133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:10.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.338+0000 7f946e0e4700 1 --2- 172.21.15.16:0/3024749922 >> v2:172.21.15.16:3300/0 conn(0x7f946809e760 0x7f946813a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:10.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.338+0000 7f946d0e2700 1 --2- 172.21.15.16:0/3024749922 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f946809de50 0x7f946809e220 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f9458009920 tx=0x7f9458009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7a282037d84cd30 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:10.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.338+0000 7f946c8e1700 1 -- 172.21.15.16:0/3024749922 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9458011830 con 0x7f946809de50 2023-09-30T22:35:10.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.339+0000 7f946c8e1700 1 -- 172.21.15.16:0/3024749922 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9458011990 con 0x7f946809de50 2023-09-30T22:35:10.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.339+0000 7f946c8e1700 1 -- 172.21.15.16:0/3024749922 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9458011d20 con 0x7f946809de50 2023-09-30T22:35:10.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.339+0000 7f946fb47700 1 -- 172.21.15.16:0/3024749922 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f946809de50 msgr2=0x7f946809e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:10.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.339+0000 7f946fb47700 1 --2- 172.21.15.16:0/3024749922 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f946809de50 0x7f946809e220 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f9458009920 tx=0x7f9458009c30 comp rx=0 tx=0).stop 2023-09-30T22:35:10.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.339+0000 7f946fb47700 1 -- 172.21.15.16:0/3024749922 shutdown_connections 2023-09-30T22:35:10.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.339+0000 7f946fb47700 1 --2- 172.21.15.16:0/3024749922 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f94681338a0 0x7f9468133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:10.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.339+0000 7f946fb47700 1 --2- 172.21.15.16:0/3024749922 >> v2:172.21.15.16:3300/0 conn(0x7f946809e760 0x7f946813a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:10.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.339+0000 7f946fb47700 1 --2- 172.21.15.16:0/3024749922 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f946809de50 0x7f946809e220 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:10.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.339+0000 7f946fb47700 1 -- 172.21.15.16:0/3024749922 >> 172.21.15.16:0/3024749922 conn(0x7f94680a2aa0 msgr2=0x7f94680a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:10.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.339+0000 7f946fb47700 1 -- 172.21.15.16:0/3024749922 shutdown_connections 2023-09-30T22:35:10.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.339+0000 7f946fb47700 1 -- 172.21.15.16:0/3024749922 wait complete. 2023-09-30T22:35:10.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.340+0000 7f946fb47700 1 Processor -- start 2023-09-30T22:35:10.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.340+0000 7f946fb47700 1 -- start start 2023-09-30T22:35:10.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.341+0000 7f946fb47700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f946809de50 0x7f946811eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:10.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.341+0000 7f946fb47700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f946809e760 0x7f946811f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:10.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.341+0000 7f946fb47700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f94681338a0 0x7f946811f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:10.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.341+0000 7f946fb47700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9468122370 con 0x7f946809de50 2023-09-30T22:35:10.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.341+0000 7f946fb47700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9468122570 con 0x7f94681338a0 2023-09-30T22:35:10.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.341+0000 7f946fb47700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9468120700 con 0x7f946809e760 2023-09-30T22:35:10.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.341+0000 7f946d8e3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f946809de50 0x7f946811eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:10.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.341+0000 7f946e0e4700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f94681338a0 0x7f946811f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:10.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.341+0000 7f946d8e3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f946809de50 0x7f946811eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49258/0 (socket says 172.21.15.16:49258) 2023-09-30T22:35:10.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.341+0000 7f946d8e3700 1 -- 172.21.15.16:0/3900170482 learned_addr learned my addr 172.21.15.16:0/3900170482 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:10.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.341+0000 7f946d0e2700 1 --2- 172.21.15.16:0/3900170482 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f946809e760 0x7f946811f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:10.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.341+0000 7f946e0e4700 1 -- 172.21.15.16:0/3900170482 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f946809e760 msgr2=0x7f946811f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:10.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.341+0000 7f946e0e4700 1 --2- 172.21.15.16:0/3900170482 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f946809e760 0x7f946811f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:10.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.341+0000 7f946e0e4700 1 -- 172.21.15.16:0/3900170482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f946809de50 msgr2=0x7f946811eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:10.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.341+0000 7f946e0e4700 1 --2- 172.21.15.16:0/3900170482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f946809de50 0x7f946811eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:10.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.341+0000 7f946e0e4700 1 -- 172.21.15.16:0/3900170482 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9460009a70 con 0x7f94681338a0 2023-09-30T22:35:10.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.341+0000 7f946d8e3700 1 --2- 172.21.15.16:0/3900170482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f946809de50 0x7f946811eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:10.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.341+0000 7f946d0e2700 1 --2- 172.21.15.16:0/3900170482 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f946809e760 0x7f946811f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:10.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.341+0000 7f946e0e4700 1 --2- 172.21.15.16:0/3900170482 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f94681338a0 0x7f946811f5b0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f946000efc0 tx=0x7f946000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:10.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.342+0000 7f945e7fc700 1 -- 172.21.15.16:0/3900170482 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946000ca90 con 0x7f94681338a0 2023-09-30T22:35:10.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.342+0000 7f945e7fc700 1 -- 172.21.15.16:0/3900170482 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f946000f040 con 0x7f94681338a0 2023-09-30T22:35:10.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.342+0000 7f946fb47700 1 -- 172.21.15.16:0/3900170482 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94640094a0 con 0x7f94681338a0 2023-09-30T22:35:10.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.342+0000 7f945e7fc700 1 -- 172.21.15.16:0/3900170482 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946000ce30 con 0x7f94681338a0 2023-09-30T22:35:10.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.342+0000 7f946fb47700 1 -- 172.21.15.16:0/3900170482 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9458009550 con 0x7f94681338a0 2023-09-30T22:35:10.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.342+0000 7f945e7fc700 1 -- 172.21.15.16:0/3900170482 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9460014480 con 0x7f94681338a0 2023-09-30T22:35:10.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.343+0000 7f945e7fc700 1 -- 172.21.15.16:0/3900170482 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f946003fc80 con 0x7f94681338a0 2023-09-30T22:35:10.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.343+0000 7f946fb47700 1 -- 172.21.15.16:0/3900170482 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9468074ea0 con 0x7f94681338a0 2023-09-30T22:35:10.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.348+0000 7f945e7fc700 1 -- 172.21.15.16:0/3900170482 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9460012930 con 0x7f94681338a0 2023-09-30T22:35:10.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.555+0000 7f946fb47700 1 -- 172.21.15.16:0/3900170482 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f946812f9a0 con 0x7f94681338a0 2023-09-30T22:35:10.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.556+0000 7f945e7fc700 1 -- 172.21.15.16:0/3900170482 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f946002a090 con 0x7f94681338a0 2023-09-30T22:35:10.556 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:35:10.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.558+0000 7f946fb47700 1 -- 172.21.15.16:0/3900170482 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f94681338a0 msgr2=0x7f946811f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:10.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.558+0000 7f946fb47700 1 --2- 172.21.15.16:0/3900170482 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f94681338a0 0x7f946811f5b0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f946000efc0 tx=0x7f946000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:35:10.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.558+0000 7f946fb47700 1 -- 172.21.15.16:0/3900170482 shutdown_connections 2023-09-30T22:35:10.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.558+0000 7f946fb47700 1 --2- 172.21.15.16:0/3900170482 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f94681338a0 0x7f946811f5b0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:10.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.558+0000 7f946fb47700 1 --2- 172.21.15.16:0/3900170482 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f946809de50 0x7f946811eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:10.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.558+0000 7f946fb47700 1 --2- 172.21.15.16:0/3900170482 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f946809e760 0x7f946811f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:10.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.558+0000 7f946fb47700 1 -- 172.21.15.16:0/3900170482 >> 172.21.15.16:0/3900170482 conn(0x7f94680a2aa0 msgr2=0x7f946809ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:10.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.558+0000 7f946fb47700 1 -- 172.21.15.16:0/3900170482 shutdown_connections 2023-09-30T22:35:10.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:10.558+0000 7f946fb47700 1 -- 172.21.15.16:0/3900170482 wait complete. 2023-09-30T22:35:10.572 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:35:10.914 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:35:10.980 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:35:10.995 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:35:11.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.064+0000 7fc5d5a80700 1 Processor -- start 2023-09-30T22:35:11.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.065+0000 7fc5d5a80700 1 -- start start 2023-09-30T22:35:11.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.065+0000 7fc5d5a80700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc5d009e1d0 0x7fc5d0135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:11.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.065+0000 7fc5d5a80700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc5d0135e50 0x7fc5d0138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:11.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.065+0000 7fc5d5a80700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5d009f130 con 0x7fc5d009d6f0 2023-09-30T22:35:11.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.065+0000 7fc5d5a80700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5d009f520 con 0x7fc5d009e1d0 2023-09-30T22:35:11.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.065+0000 7fc5d5a80700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d009e930 con 0x7fc5d0135e50 2023-09-30T22:35:11.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.065+0000 7fc5c7fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc5d009e1d0 0x7fc5d0135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:11.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.065+0000 7fc5cf7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc5d009d6f0 0x7fc5d009dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51748/0 (socket says 172.21.15.16:51748) 2023-09-30T22:35:11.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.065+0000 7fc5cf7fe700 1 -- 172.21.15.16:0/2272219497 learned_addr learned my addr 172.21.15.16:0/2272219497 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:11.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.065+0000 7fc5cffff700 1 --2- 172.21.15.16:0/2272219497 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc5d0135e50 0x7fc5d0138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:11.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.066+0000 7fc5ceffd700 1 -- 172.21.15.16:0/2272219497 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 845128129 0 0) 0x7fc5d009f130 con 0x7fc5d009d6f0 2023-09-30T22:35:11.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.066+0000 7fc5ceffd700 1 -- 172.21.15.16:0/2272219497 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5bc0035b0 con 0x7fc5d009d6f0 2023-09-30T22:35:11.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.066+0000 7fc5c7fff700 1 -- 172.21.15.16:0/2272219497 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc5d0135e50 msgr2=0x7fc5d0138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:11.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.066+0000 7fc5c7fff700 1 --2- 172.21.15.16:0/2272219497 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc5d0135e50 0x7fc5d0138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:11.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.066+0000 7fc5c7fff700 1 -- 172.21.15.16:0/2272219497 >> v1:172.21.15.16:6789/0 conn(0x7fc5d009d6f0 legacy=0x7fc5d009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:11.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.066+0000 7fc5c7fff700 1 -- 172.21.15.16:0/2272219497 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d009b2d0 con 0x7fc5d009e1d0 2023-09-30T22:35:11.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.066+0000 7fc5c7fff700 1 --2- 172.21.15.16:0/2272219497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5d009e1d0 0x7fc5d0135910 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8009840 tx=0x7fc5b8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f94b85bfb55231c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:11.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.066+0000 7fc5ceffd700 1 -- 172.21.15.16:0/2272219497 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b8011830 con 0x7fc5d009e1d0 2023-09-30T22:35:11.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.066+0000 7fc5ceffd700 1 -- 172.21.15.16:0/2272219497 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5b8011990 con 0x7fc5d009e1d0 2023-09-30T22:35:11.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.066+0000 7fc5ceffd700 1 -- 172.21.15.16:0/2272219497 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5b8011d20 con 0x7fc5d009e1d0 2023-09-30T22:35:11.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.067+0000 7fc5d5a80700 1 -- 172.21.15.16:0/2272219497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5d009e1d0 msgr2=0x7fc5d0135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:11.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.067+0000 7fc5d5a80700 1 --2- 172.21.15.16:0/2272219497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5d009e1d0 0x7fc5d0135910 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fc5b8009840 tx=0x7fc5b8009b50 comp rx=0 tx=0).stop 2023-09-30T22:35:11.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.067+0000 7fc5d5a80700 1 -- 172.21.15.16:0/2272219497 shutdown_connections 2023-09-30T22:35:11.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.067+0000 7fc5d5a80700 1 --2- 172.21.15.16:0/2272219497 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc5d0135e50 0x7fc5d0138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:11.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.067+0000 7fc5d5a80700 1 --2- 172.21.15.16:0/2272219497 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5d009e1d0 0x7fc5d0135910 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:11.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.067+0000 7fc5d5a80700 1 -- 172.21.15.16:0/2272219497 >> 172.21.15.16:0/2272219497 conn(0x7fc5d0066f50 msgr2=0x7fc5d009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:11.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.067+0000 7fc5d5a80700 1 -- 172.21.15.16:0/2272219497 shutdown_connections 2023-09-30T22:35:11.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.067+0000 7fc5d5a80700 1 -- 172.21.15.16:0/2272219497 wait complete. 2023-09-30T22:35:11.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.068+0000 7fc5d5a80700 1 Processor -- start 2023-09-30T22:35:11.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.068+0000 7fc5d5a80700 1 -- start start 2023-09-30T22:35:11.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.068+0000 7fc5d5a80700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc5d009d6f0 0x7fc5d01397b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:11.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.068+0000 7fc5d5a80700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5d009e1d0 0x7fc5d0139cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:11.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.069+0000 7fc5d5a80700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc5d0135e50 0x7fc5d013a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:11.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.069+0000 7fc5d5a80700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d0120f40 con 0x7fc5d009e1d0 2023-09-30T22:35:11.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.069+0000 7fc5d5a80700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d009c700 con 0x7fc5d0135e50 2023-09-30T22:35:11.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.069+0000 7fc5d5a80700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d0138960 con 0x7fc5d009d6f0 2023-09-30T22:35:11.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.069+0000 7fc5c7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5d009e1d0 0x7fc5d0139cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:11.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.069+0000 7fc5c7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5d009e1d0 0x7fc5d0139cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49262/0 (socket says 172.21.15.16:49262) 2023-09-30T22:35:11.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.069+0000 7fc5c7fff700 1 -- 172.21.15.16:0/3473148277 learned_addr learned my addr 172.21.15.16:0/3473148277 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:11.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.069+0000 7fc5cffff700 1 --2- 172.21.15.16:0/3473148277 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc5d0135e50 0x7fc5d013a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:11.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.069+0000 7fc5cf7fe700 1 --2- 172.21.15.16:0/3473148277 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc5d009d6f0 0x7fc5d01397b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:11.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.069+0000 7fc5cffff700 1 -- 172.21.15.16:0/3473148277 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc5d009d6f0 msgr2=0x7fc5d01397b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:11.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.069+0000 7fc5cffff700 1 --2- 172.21.15.16:0/3473148277 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc5d009d6f0 0x7fc5d01397b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:11.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.069+0000 7fc5cffff700 1 -- 172.21.15.16:0/3473148277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5d009e1d0 msgr2=0x7fc5d0139cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:11.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.069+0000 7fc5cffff700 1 --2- 172.21.15.16:0/3473148277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5d009e1d0 0x7fc5d0139cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:11.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.069+0000 7fc5cffff700 1 -- 172.21.15.16:0/3473148277 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5b80094a0 con 0x7fc5d0135e50 2023-09-30T22:35:11.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.069+0000 7fc5c7fff700 1 --2- 172.21.15.16:0/3473148277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5d009e1d0 0x7fc5d0139cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:11.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.069+0000 7fc5cf7fe700 1 --2- 172.21.15.16:0/3473148277 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc5d009d6f0 0x7fc5d01397b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:35:11.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.069+0000 7fc5cffff700 1 --2- 172.21.15.16:0/3473148277 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc5d0135e50 0x7fc5d013a230 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fc5c800efc0 tx=0x7fc5c800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:11.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.070+0000 7fc5ccff9700 1 -- 172.21.15.16:0/3473148277 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c800cd40 con 0x7fc5d0135e50 2023-09-30T22:35:11.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.070+0000 7fc5ccff9700 1 -- 172.21.15.16:0/3473148277 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5c800f040 con 0x7fc5d0135e50 2023-09-30T22:35:11.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.070+0000 7fc5ccff9700 1 -- 172.21.15.16:0/3473148277 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c80146c0 con 0x7fc5d0135e50 2023-09-30T22:35:11.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.070+0000 7fc5d5a80700 1 -- 172.21.15.16:0/3473148277 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5d0120b90 con 0x7fc5d0135e50 2023-09-30T22:35:11.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.070+0000 7fc5d5a80700 1 -- 172.21.15.16:0/3473148277 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5d011fcb0 con 0x7fc5d0135e50 2023-09-30T22:35:11.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.070+0000 7fc5ccff9700 1 -- 172.21.15.16:0/3473148277 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc5c8014820 con 0x7fc5d0135e50 2023-09-30T22:35:11.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.071+0000 7fc5ccff9700 1 -- 172.21.15.16:0/3473148277 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fc5c80402d0 con 0x7fc5d0135e50 2023-09-30T22:35:11.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.071+0000 7fc5d5a80700 1 -- 172.21.15.16:0/3473148277 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5d0071700 con 0x7fc5d0135e50 2023-09-30T22:35:11.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.076+0000 7fc5ccff9700 1 -- 172.21.15.16:0/3473148277 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc5c8018020 con 0x7fc5d0135e50 2023-09-30T22:35:11.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.129+0000 7fbed576c700 1 Processor -- start 2023-09-30T22:35:11.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.129+0000 7fbed576c700 1 -- start start 2023-09-30T22:35:11.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.129+0000 7fbed576c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbed009f410 0x7fbed0136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:11.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.129+0000 7fbed576c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbed0136fe0 0x7fbed01393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:11.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.129+0000 7fbed576c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbed0070940 con 0x7fbed009e930 2023-09-30T22:35:11.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.129+0000 7fbed576c700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbed006fd60 con 0x7fbed0136fe0 2023-09-30T22:35:11.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.129+0000 7fbed576c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbed006fa80 con 0x7fbed009f410 2023-09-30T22:35:11.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.129+0000 7fbecf7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbed0136fe0 0x7fbed01393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:11.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.129+0000 7fbece7fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbed009f410 0x7fbed0136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:11.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.129+0000 7fbecf7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fbed0136fe0 0x7fbed01393c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49274/0 (socket says 172.21.15.16:49274) 2023-09-30T22:35:11.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.129+0000 7fbecf7fe700 1 -- 172.21.15.16:0/1165450909 learned_addr learned my addr 172.21.15.16:0/1165450909 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:11.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.129+0000 7fbeceffd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fbed009e930 0x7fbed009ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51754/0 (socket says 172.21.15.16:51754) 2023-09-30T22:35:11.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.130+0000 7fbece7fc700 1 -- 172.21.15.16:0/1165450909 >> v2:172.21.15.16:3300/0 conn(0x7fbed0136fe0 msgr2=0x7fbed01393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:11.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.130+0000 7fbece7fc700 1 --2- 172.21.15.16:0/1165450909 >> v2:172.21.15.16:3300/0 conn(0x7fbed0136fe0 0x7fbed01393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:11.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.130+0000 7fbece7fc700 1 -- 172.21.15.16:0/1165450909 >> v1:172.21.15.16:6789/0 conn(0x7fbed009e930 legacy=0x7fbed009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:11.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.130+0000 7fbece7fc700 1 -- 172.21.15.16:0/1165450909 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed009fb60 con 0x7fbed009f410 2023-09-30T22:35:11.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.130+0000 7fbecf7fe700 1 --2- 172.21.15.16:0/1165450909 >> v2:172.21.15.16:3300/0 conn(0x7fbed0136fe0 0x7fbed01393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:11.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.130+0000 7fbece7fc700 1 --2- 172.21.15.16:0/1165450909 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbed009f410 0x7fbed0136aa0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fbec4009840 tx=0x7fbec4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=664ca878cc600d82 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:11.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.130+0000 7fbecdffb700 1 -- 172.21.15.16:0/1165450909 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec4011830 con 0x7fbed009f410 2023-09-30T22:35:11.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.130+0000 7fbecdffb700 1 -- 172.21.15.16:0/1165450909 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec4011990 con 0x7fbed009f410 2023-09-30T22:35:11.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.130+0000 7fbecdffb700 1 -- 172.21.15.16:0/1165450909 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec4011d20 con 0x7fbed009f410 2023-09-30T22:35:11.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.131+0000 7fbed576c700 1 -- 172.21.15.16:0/1165450909 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbed009f410 msgr2=0x7fbed0136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:11.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.131+0000 7fbed576c700 1 --2- 172.21.15.16:0/1165450909 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbed009f410 0x7fbed0136aa0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fbec4009840 tx=0x7fbec4009b50 comp rx=0 tx=0).stop 2023-09-30T22:35:11.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.131+0000 7fbed576c700 1 -- 172.21.15.16:0/1165450909 shutdown_connections 2023-09-30T22:35:11.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.131+0000 7fbed576c700 1 --2- 172.21.15.16:0/1165450909 >> v2:172.21.15.16:3300/0 conn(0x7fbed0136fe0 0x7fbed01393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:11.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.131+0000 7fbed576c700 1 --2- 172.21.15.16:0/1165450909 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbed009f410 0x7fbed0136aa0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:11.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.131+0000 7fbed576c700 1 -- 172.21.15.16:0/1165450909 >> 172.21.15.16:0/1165450909 conn(0x7fbed00a4210 msgr2=0x7fbed00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:11.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.131+0000 7fbed576c700 1 -- 172.21.15.16:0/1165450909 shutdown_connections 2023-09-30T22:35:11.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.131+0000 7fbed576c700 1 -- 172.21.15.16:0/1165450909 wait complete. 2023-09-30T22:35:11.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.132+0000 7fbed576c700 1 Processor -- start 2023-09-30T22:35:11.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.132+0000 7fbed576c700 1 -- start start 2023-09-30T22:35:11.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.133+0000 7fbed576c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbed009e930 0x7fbed011c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:11.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.133+0000 7fbed576c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbed009f410 0x7fbed011cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:11.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.133+0000 7fbed576c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbed0136fe0 0x7fbed011d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:11.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.133+0000 7fbed576c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed011fe50 con 0x7fbed009e930 2023-09-30T22:35:11.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.133+0000 7fbed576c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbed00a2f40 con 0x7fbed009f410 2023-09-30T22:35:11.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.133+0000 7fbed576c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbed011f8a0 con 0x7fbed0136fe0 2023-09-30T22:35:11.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.133+0000 7fbeceffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbed009e930 0x7fbed011c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:11.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.133+0000 7fbeceffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbed009e930 0x7fbed011c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49282/0 (socket says 172.21.15.16:49282) 2023-09-30T22:35:11.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.133+0000 7fbeceffd700 1 -- 172.21.15.16:0/2188210335 learned_addr learned my addr 172.21.15.16:0/2188210335 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:11.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.133+0000 7fbece7fc700 1 --2- 172.21.15.16:0/2188210335 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbed009f410 0x7fbed011cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:11.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.133+0000 7fbecf7fe700 1 --2- 172.21.15.16:0/2188210335 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbed0136fe0 0x7fbed011d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:11.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.133+0000 7fbeceffd700 1 -- 172.21.15.16:0/2188210335 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbed0136fe0 msgr2=0x7fbed011d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:11.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.133+0000 7fbeceffd700 1 --2- 172.21.15.16:0/2188210335 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbed0136fe0 0x7fbed011d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:11.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.133+0000 7fbeceffd700 1 -- 172.21.15.16:0/2188210335 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbed009f410 msgr2=0x7fbed011cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:11.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.133+0000 7fbeceffd700 1 --2- 172.21.15.16:0/2188210335 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbed009f410 0x7fbed011cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:11.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.133+0000 7fbeceffd700 1 -- 172.21.15.16:0/2188210335 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec8009b30 con 0x7fbed009e930 2023-09-30T22:35:11.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.134+0000 7fbeceffd700 1 --2- 172.21.15.16:0/2188210335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbed009e930 0x7fbed011c660 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fbec000c520 tx=0x7fbec000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:11.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.134+0000 7fbeb77fe700 1 -- 172.21.15.16:0/2188210335 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec0005cb0 con 0x7fbed009e930 2023-09-30T22:35:11.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.134+0000 7fbeb77fe700 1 -- 172.21.15.16:0/2188210335 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec0005e10 con 0x7fbed009e930 2023-09-30T22:35:11.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.134+0000 7fbeb77fe700 1 -- 172.21.15.16:0/2188210335 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec00072d0 con 0x7fbed009e930 2023-09-30T22:35:11.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.134+0000 7fbed576c700 1 -- 172.21.15.16:0/2188210335 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbec40094a0 con 0x7fbed009e930 2023-09-30T22:35:11.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.134+0000 7fbed576c700 1 -- 172.21.15.16:0/2188210335 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbed011e330 con 0x7fbed009e930 2023-09-30T22:35:11.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.135+0000 7fbeb77fe700 1 -- 172.21.15.16:0/2188210335 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbec0007430 con 0x7fbed009e930 2023-09-30T22:35:11.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.135+0000 7fbeb77fe700 1 -- 172.21.15.16:0/2188210335 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fbec003ea30 con 0x7fbed009e930 2023-09-30T22:35:11.139 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.135+0000 7fbed576c700 1 -- 172.21.15.16:0/2188210335 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbed0074ea0 con 0x7fbed009e930 2023-09-30T22:35:11.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.140+0000 7fbeb77fe700 1 -- 172.21.15.16:0/2188210335 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbec0017080 con 0x7fbed009e930 2023-09-30T22:35:11.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.166+0000 7f4891193700 1 Processor -- start 2023-09-30T22:35:11.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.166+0000 7f4891193700 1 -- start start 2023-09-30T22:35:11.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.166+0000 7f4891193700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f488c06c0e0 0x7f488c09ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:11.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.167+0000 7f4891193700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f488c1380f0 0x7f488c13fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:11.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.167+0000 7f4891193700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f488c134980 con 0x7f488c134ec0 2023-09-30T22:35:11.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.167+0000 7f4891193700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f488c06c4b0 con 0x7f488c1380f0 2023-09-30T22:35:11.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.167+0000 7f4891193700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f488c0a09c0 con 0x7f488c06c0e0 2023-09-30T22:35:11.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.167+0000 7f488b59e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f488c1380f0 0x7f488c13fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:11.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.167+0000 7f488b59e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f488c1380f0 0x7f488c13fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37760/0 (socket says 172.21.15.16:37760) 2023-09-30T22:35:11.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.167+0000 7f488b59e700 1 -- 172.21.15.16:0/1544996498 learned_addr learned my addr 172.21.15.16:0/1544996498 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:11.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.167+0000 7f488ad9d700 1 --2- 172.21.15.16:0/1544996498 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f488c06c0e0 0x7f488c09ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:11.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.167+0000 7f488b59e700 1 -- 172.21.15.16:0/1544996498 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f488c06c0e0 msgr2=0x7f488c09ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:11.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.167+0000 7f488b59e700 1 --2- 172.21.15.16:0/1544996498 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f488c06c0e0 0x7f488c09ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:11.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.167+0000 7f4889d9b700 1 -- 172.21.15.16:0/1544996498 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4270806281 0 0) 0x7f488c134980 con 0x7f488c134ec0 2023-09-30T22:35:11.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.167+0000 7f488b59e700 1 -- 172.21.15.16:0/1544996498 >> v1:172.21.15.16:6789/0 conn(0x7f488c134ec0 legacy=0x7f488c1379e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:11.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.167+0000 7f488b59e700 1 -- 172.21.15.16:0/1544996498 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f488c0a0600 con 0x7f488c1380f0 2023-09-30T22:35:11.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.167+0000 7f488b59e700 1 --2- 172.21.15.16:0/1544996498 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f488c1380f0 0x7f488c13fd20 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f4884009e80 tx=0x7f488400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b43072b2e0dbfe1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:11.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.168+0000 7f4889d9b700 1 -- 172.21.15.16:0/1544996498 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488400e070 con 0x7f488c1380f0 2023-09-30T22:35:11.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.168+0000 7f4889d9b700 1 -- 172.21.15.16:0/1544996498 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f488400b910 con 0x7f488c1380f0 2023-09-30T22:35:11.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.168+0000 7f4889d9b700 1 -- 172.21.15.16:0/1544996498 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488400bcc0 con 0x7f488c1380f0 2023-09-30T22:35:11.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.168+0000 7f4891193700 1 -- 172.21.15.16:0/1544996498 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f488c1380f0 msgr2=0x7f488c13fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:11.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.168+0000 7f4891193700 1 --2- 172.21.15.16:0/1544996498 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f488c1380f0 0x7f488c13fd20 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f4884009e80 tx=0x7f488400b330 comp rx=0 tx=0).stop 2023-09-30T22:35:11.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.169+0000 7f4891193700 1 -- 172.21.15.16:0/1544996498 shutdown_connections 2023-09-30T22:35:11.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.169+0000 7f4891193700 1 --2- 172.21.15.16:0/1544996498 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f488c1380f0 0x7f488c13fd20 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:11.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.169+0000 7f4891193700 1 --2- 172.21.15.16:0/1544996498 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f488c06c0e0 0x7f488c09ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:11.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.169+0000 7f4891193700 1 -- 172.21.15.16:0/1544996498 >> 172.21.15.16:0/1544996498 conn(0x7f488c0a4210 msgr2=0x7f488c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:11.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.169+0000 7f4891193700 1 -- 172.21.15.16:0/1544996498 shutdown_connections 2023-09-30T22:35:11.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.169+0000 7f4891193700 1 -- 172.21.15.16:0/1544996498 wait complete. 2023-09-30T22:35:11.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.170+0000 7f4891193700 1 Processor -- start 2023-09-30T22:35:11.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.170+0000 7f4891193700 1 -- start start 2023-09-30T22:35:11.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.171+0000 7f4891193700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f488c06c0e0 0x7f488c11a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:11.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.171+0000 7f4891193700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f488c134ec0 0x7f488c11ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:11.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.171+0000 7f4891193700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f488c1380f0 0x7f488c11b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:11.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.171+0000 7f4891193700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488c11c780 con 0x7f488c134ec0 2023-09-30T22:35:11.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.171+0000 7f4891193700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f488c06d2f0 con 0x7f488c06c0e0 2023-09-30T22:35:11.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.171+0000 7f4891193700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f488c11de30 con 0x7f488c1380f0 2023-09-30T22:35:11.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.171+0000 7f488ad9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f488c06c0e0 0x7f488c11a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:11.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.171+0000 7f488a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f488c134ec0 0x7f488c11ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:11.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.171+0000 7f488ad9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f488c06c0e0 0x7f488c11a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37770/0 (socket says 172.21.15.16:37770) 2023-09-30T22:35:11.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.171+0000 7f488ad9d700 1 -- 172.21.15.16:0/2470201836 learned_addr learned my addr 172.21.15.16:0/2470201836 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:11.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.171+0000 7f488b59e700 1 --2- 172.21.15.16:0/2470201836 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f488c1380f0 0x7f488c11b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:11.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.171+0000 7f488a59c700 1 -- 172.21.15.16:0/2470201836 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f488c1380f0 msgr2=0x7f488c11b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:11.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.171+0000 7f488a59c700 1 --2- 172.21.15.16:0/2470201836 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f488c1380f0 0x7f488c11b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:11.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.171+0000 7f488a59c700 1 -- 172.21.15.16:0/2470201836 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f488c06c0e0 msgr2=0x7f488c11a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:11.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.171+0000 7f488a59c700 1 --2- 172.21.15.16:0/2470201836 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f488c06c0e0 0x7f488c11a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:11.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.171+0000 7f488a59c700 1 -- 172.21.15.16:0/2470201836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4884009b30 con 0x7f488c134ec0 2023-09-30T22:35:11.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.171+0000 7f488a59c700 1 --2- 172.21.15.16:0/2470201836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f488c134ec0 0x7f488c11ae30 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f488c06f500 tx=0x7f488000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:11.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.172+0000 7f48737fe700 1 -- 172.21.15.16:0/2470201836 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488000cea0 con 0x7f488c134ec0 2023-09-30T22:35:11.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.172+0000 7f48737fe700 1 -- 172.21.15.16:0/2470201836 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4880005520 con 0x7f488c134ec0 2023-09-30T22:35:11.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.172+0000 7f4891193700 1 -- 172.21.15.16:0/2470201836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f488c0a0450 con 0x7f488c134ec0 2023-09-30T22:35:11.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.172+0000 7f48737fe700 1 -- 172.21.15.16:0/2470201836 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48800038c0 con 0x7f488c134ec0 2023-09-30T22:35:11.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.172+0000 7f4891193700 1 -- 172.21.15.16:0/2470201836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f488c11b8b0 con 0x7f488c134ec0 2023-09-30T22:35:11.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.173+0000 7f4891193700 1 -- 172.21.15.16:0/2470201836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f488c074ea0 con 0x7f488c134ec0 2023-09-30T22:35:11.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.174+0000 7f48737fe700 1 -- 172.21.15.16:0/2470201836 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4880003a20 con 0x7f488c134ec0 2023-09-30T22:35:11.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.174+0000 7f48737fe700 1 -- 172.21.15.16:0/2470201836 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f488003f260 con 0x7f488c134ec0 2023-09-30T22:35:11.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.178+0000 7f48737fe700 1 -- 172.21.15.16:0/2470201836 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4880012400 con 0x7f488c134ec0 2023-09-30T22:35:11.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.289+0000 7fc5d5a80700 1 -- 172.21.15.16:0/3473148277 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc5d0074ea0 con 0x7fc5d0135e50 2023-09-30T22:35:11.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.290+0000 7fc5ccff9700 1 -- 172.21.15.16:0/3473148277 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc5c802b090 con 0x7fc5d0135e50 2023-09-30T22:35:11.290 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:35:11.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.292+0000 7fc5d5a80700 1 -- 172.21.15.16:0/3473148277 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc5d0135e50 msgr2=0x7fc5d013a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:11.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.292+0000 7fc5d5a80700 1 --2- 172.21.15.16:0/3473148277 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc5d0135e50 0x7fc5d013a230 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fc5c800efc0 tx=0x7fc5c800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:35:11.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.292+0000 7fc5d5a80700 1 -- 172.21.15.16:0/3473148277 shutdown_connections 2023-09-30T22:35:11.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.292+0000 7fc5d5a80700 1 --2- 172.21.15.16:0/3473148277 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc5d0135e50 0x7fc5d013a230 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:11.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.292+0000 7fc5d5a80700 1 --2- 172.21.15.16:0/3473148277 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc5d009d6f0 0x7fc5d01397b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:11.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.292+0000 7fc5d5a80700 1 --2- 172.21.15.16:0/3473148277 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc5d009e1d0 0x7fc5d0139cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:11.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.292+0000 7fc5d5a80700 1 -- 172.21.15.16:0/3473148277 >> 172.21.15.16:0/3473148277 conn(0x7fc5d0066f50 msgr2=0x7fc5d006e840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:11.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.292+0000 7fc5d5a80700 1 -- 172.21.15.16:0/3473148277 shutdown_connections 2023-09-30T22:35:11.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.292+0000 7fc5d5a80700 1 -- 172.21.15.16:0/3473148277 wait complete. 2023-09-30T22:35:11.306 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:35:11.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.366+0000 7fbed576c700 1 -- 172.21.15.16:0/2188210335 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fbed0132500 con 0x7fbed009e930 2023-09-30T22:35:11.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.367+0000 7fbeb77fe700 1 -- 172.21.15.16:0/2188210335 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbec001b3b0 con 0x7fbed009e930 2023-09-30T22:35:11.367 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:35:11.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.369+0000 7fbed576c700 1 -- 172.21.15.16:0/2188210335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbed009e930 msgr2=0x7fbed011c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:11.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.369+0000 7fbed576c700 1 --2- 172.21.15.16:0/2188210335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbed009e930 0x7fbed011c660 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fbec000c520 tx=0x7fbec000c830 comp rx=0 tx=0).stop 2023-09-30T22:35:11.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.369+0000 7fbed576c700 1 -- 172.21.15.16:0/2188210335 shutdown_connections 2023-09-30T22:35:11.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.369+0000 7fbed576c700 1 --2- 172.21.15.16:0/2188210335 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbed0136fe0 0x7fbed011d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:11.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.369+0000 7fbed576c700 1 --2- 172.21.15.16:0/2188210335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbed009e930 0x7fbed011c660 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:11.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.369+0000 7fbed576c700 1 --2- 172.21.15.16:0/2188210335 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbed009f410 0x7fbed011cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:11.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.369+0000 7fbed576c700 1 -- 172.21.15.16:0/2188210335 >> 172.21.15.16:0/2188210335 conn(0x7fbed00a4210 msgr2=0x7fbed00a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:11.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.369+0000 7fbed576c700 1 -- 172.21.15.16:0/2188210335 shutdown_connections 2023-09-30T22:35:11.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.369+0000 7fbed576c700 1 -- 172.21.15.16:0/2188210335 wait complete. 2023-09-30T22:35:11.383 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:35:11.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.396+0000 7f4891193700 1 -- 172.21.15.16:0/2470201836 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f488c06b250 con 0x7f488c134ec0 2023-09-30T22:35:11.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.397+0000 7f48737fe700 1 -- 172.21.15.16:0/2470201836 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4880017030 con 0x7f488c134ec0 2023-09-30T22:35:11.397 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:35:11.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.399+0000 7f4891193700 1 -- 172.21.15.16:0/2470201836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f488c134ec0 msgr2=0x7f488c11ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:11.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.399+0000 7f4891193700 1 --2- 172.21.15.16:0/2470201836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f488c134ec0 0x7f488c11ae30 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f488c06f500 tx=0x7f488000c800 comp rx=0 tx=0).stop 2023-09-30T22:35:11.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.399+0000 7f4891193700 1 -- 172.21.15.16:0/2470201836 shutdown_connections 2023-09-30T22:35:11.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.399+0000 7f4891193700 1 --2- 172.21.15.16:0/2470201836 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f488c06c0e0 0x7f488c11a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:11.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.399+0000 7f4891193700 1 --2- 172.21.15.16:0/2470201836 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f488c1380f0 0x7f488c11b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:11.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.399+0000 7f4891193700 1 --2- 172.21.15.16:0/2470201836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f488c134ec0 0x7f488c11ae30 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:11.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.399+0000 7f4891193700 1 -- 172.21.15.16:0/2470201836 >> 172.21.15.16:0/2470201836 conn(0x7f488c0a4210 msgr2=0x7f488c1371a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:11.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.399+0000 7f4891193700 1 -- 172.21.15.16:0/2470201836 shutdown_connections 2023-09-30T22:35:11.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.399+0000 7f4891193700 1 -- 172.21.15.16:0/2470201836 wait complete. 2023-09-30T22:35:11.413 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:35:11.573 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:35:11.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.717+0000 7f8dde9b1700 1 Processor -- start 2023-09-30T22:35:11.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.718+0000 7f8dde9b1700 1 -- start start 2023-09-30T22:35:11.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.718+0000 7f8dde9b1700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8dd8134630 0x7f8dd8136a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:11.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.718+0000 7f8dde9b1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8dd8136f50 0x7f8dd8139330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:11.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.718+0000 7f8dde9b1700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dd809b4f0 con 0x7f8dd8131810 2023-09-30T22:35:11.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.718+0000 7f8dde9b1700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dd81313c0 con 0x7f8dd8134630 2023-09-30T22:35:11.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.718+0000 7f8dde9b1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd806e8c0 con 0x7f8dd8136f50 2023-09-30T22:35:11.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.718+0000 7f8dd77fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8dd8134630 0x7f8dd8136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:11.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.718+0000 7f8dd7fff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8dd8131810 0x7f8dd8131be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51768/0 (socket says 172.21.15.16:51768) 2023-09-30T22:35:11.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.718+0000 7f8ddcf4e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8dd8136f50 0x7f8dd8139330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:11.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.718+0000 7f8dd7fff700 1 -- 172.21.15.16:0/3655376730 learned_addr learned my addr 172.21.15.16:0/3655376730 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:11.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.719+0000 7f8ddcf4e700 1 -- 172.21.15.16:0/3655376730 >> v2:172.21.15.16:3300/0 conn(0x7f8dd8134630 msgr2=0x7f8dd8136a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:11.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.719+0000 7f8ddcf4e700 1 --2- 172.21.15.16:0/3655376730 >> v2:172.21.15.16:3300/0 conn(0x7f8dd8134630 0x7f8dd8136a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:11.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.719+0000 7f8ddcf4e700 1 -- 172.21.15.16:0/3655376730 >> v1:172.21.15.16:6789/0 conn(0x7f8dd8131810 legacy=0x7f8dd8131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:11.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.719+0000 7f8ddcf4e700 1 -- 172.21.15.16:0/3655376730 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dd8071d20 con 0x7f8dd8136f50 2023-09-30T22:35:11.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.719+0000 7f8dd77fe700 1 --2- 172.21.15.16:0/3655376730 >> v2:172.21.15.16:3300/0 conn(0x7f8dd8134630 0x7f8dd8136a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:11.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.719+0000 7f8ddcf4e700 1 --2- 172.21.15.16:0/3655376730 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8dd8136f50 0x7f8dd8139330 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f8dd0009d90 tx=0x7f8dd000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9579765e5cdc0ca server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:11.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.719+0000 7f8dd6ffd700 1 -- 172.21.15.16:0/3655376730 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd000e070 con 0x7f8dd8136f50 2023-09-30T22:35:11.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.719+0000 7f8dd6ffd700 1 -- 172.21.15.16:0/3655376730 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dd000bb20 con 0x7f8dd8136f50 2023-09-30T22:35:11.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.719+0000 7f8dd6ffd700 1 -- 172.21.15.16:0/3655376730 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd0012430 con 0x7f8dd8136f50 2023-09-30T22:35:11.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.719+0000 7f8dde9b1700 1 -- 172.21.15.16:0/3655376730 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8dd8136f50 msgr2=0x7f8dd8139330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:11.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.719+0000 7f8dde9b1700 1 --2- 172.21.15.16:0/3655376730 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8dd8136f50 0x7f8dd8139330 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f8dd0009d90 tx=0x7f8dd000b330 comp rx=0 tx=0).stop 2023-09-30T22:35:11.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.720+0000 7f8dde9b1700 1 -- 172.21.15.16:0/3655376730 shutdown_connections 2023-09-30T22:35:11.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.720+0000 7f8dde9b1700 1 --2- 172.21.15.16:0/3655376730 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8dd8136f50 0x7f8dd8139330 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:11.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.720+0000 7f8dde9b1700 1 --2- 172.21.15.16:0/3655376730 >> v2:172.21.15.16:3300/0 conn(0x7f8dd8134630 0x7f8dd8136a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:11.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.720+0000 7f8dde9b1700 1 -- 172.21.15.16:0/3655376730 >> 172.21.15.16:0/3655376730 conn(0x7f8dd80a2aa0 msgr2=0x7f8dd80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:11.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.720+0000 7f8dde9b1700 1 -- 172.21.15.16:0/3655376730 shutdown_connections 2023-09-30T22:35:11.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.720+0000 7f8dde9b1700 1 -- 172.21.15.16:0/3655376730 wait complete. 2023-09-30T22:35:11.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.721+0000 7f8dde9b1700 1 Processor -- start 2023-09-30T22:35:11.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.721+0000 7f8dde9b1700 1 -- start start 2023-09-30T22:35:11.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.722+0000 7f8dde9b1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8dd8131810 0x7f8dd8119b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:11.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.722+0000 7f8dde9b1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8dd8134630 0x7f8dd811a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:11.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.722+0000 7f8dde9b1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8dd8136f50 0x7f8dd81c63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:11.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.722+0000 7f8dde9b1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd811c670 con 0x7f8dd8136f50 2023-09-30T22:35:11.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.722+0000 7f8dde9b1700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd8136cb0 con 0x7f8dd8131810 2023-09-30T22:35:11.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.722+0000 7f8dde9b1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd806eb00 con 0x7f8dd8134630 2023-09-30T22:35:11.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.722+0000 7f8dd77fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8dd8134630 0x7f8dd811a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:11.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.722+0000 7f8dd7fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8dd8131810 0x7f8dd8119b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:11.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.722+0000 7f8ddcf4e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8dd8136f50 0x7f8dd81c63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:11.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.722+0000 7f8dd77fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8dd8134630 0x7f8dd811a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:55316/0 (socket says 172.21.15.16:55316) 2023-09-30T22:35:11.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.722+0000 7f8dd77fe700 1 -- 172.21.15.16:0/3182135152 learned_addr learned my addr 172.21.15.16:0/3182135152 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:11.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.722+0000 7f8dd77fe700 1 -- 172.21.15.16:0/3182135152 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8dd8131810 msgr2=0x7f8dd8119b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:11.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.722+0000 7f8dd77fe700 1 --2- 172.21.15.16:0/3182135152 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8dd8131810 0x7f8dd8119b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:11.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.722+0000 7f8dd77fe700 1 -- 172.21.15.16:0/3182135152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8dd8136f50 msgr2=0x7f8dd81c63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:11.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.722+0000 7f8dd77fe700 1 --2- 172.21.15.16:0/3182135152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8dd8136f50 0x7f8dd81c63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:11.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.722+0000 7f8dd77fe700 1 -- 172.21.15.16:0/3182135152 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dcc0094a0 con 0x7f8dd8134630 2023-09-30T22:35:11.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.722+0000 7f8ddcf4e700 1 --2- 172.21.15.16:0/3182135152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8dd8136f50 0x7f8dd81c63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:11.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.723+0000 7f8dd77fe700 1 --2- 172.21.15.16:0/3182135152 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8dd8134630 0x7f8dd811a090 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc00ebe0 tx=0x7f8dcc00eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:11.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.723+0000 7f8dd4ff9700 1 -- 172.21.15.16:0/3182135152 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc0097a0 con 0x7f8dd8134630 2023-09-30T22:35:11.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.723+0000 7f8dd4ff9700 1 -- 172.21.15.16:0/3182135152 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dcc00c850 con 0x7f8dd8134630 2023-09-30T22:35:11.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.723+0000 7f8dd4ff9700 1 -- 172.21.15.16:0/3182135152 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc00cbe0 con 0x7f8dd8134630 2023-09-30T22:35:11.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.723+0000 7f8dde9b1700 1 -- 172.21.15.16:0/3182135152 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dd0009a70 con 0x7f8dd8134630 2023-09-30T22:35:11.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.723+0000 7f8dde9b1700 1 -- 172.21.15.16:0/3182135152 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dd81195d0 con 0x7f8dd8134630 2023-09-30T22:35:11.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.724+0000 7f8dd4ff9700 1 -- 172.21.15.16:0/3182135152 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8dcc00cd40 con 0x7f8dd8134630 2023-09-30T22:35:11.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.724+0000 7f8dd4ff9700 1 -- 172.21.15.16:0/3182135152 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f8dcc03f3e0 con 0x7f8dd8134630 2023-09-30T22:35:11.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.724+0000 7f8dde9b1700 1 -- 172.21.15.16:0/3182135152 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dd8074ea0 con 0x7f8dd8134630 2023-09-30T22:35:11.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.729+0000 7f8dd4ff9700 1 -- 172.21.15.16:0/3182135152 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8dcc0102b0 con 0x7f8dd8134630 2023-09-30T22:35:11.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.934+0000 7f8dde9b1700 1 -- 172.21.15.16:0/3182135152 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8dd811c890 con 0x7f8dd8134630 2023-09-30T22:35:11.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.935+0000 7f8dd4ff9700 1 -- 172.21.15.16:0/3182135152 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8dcc017030 con 0x7f8dd8134630 2023-09-30T22:35:11.936 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:35:11.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.937+0000 7f8dde9b1700 1 -- 172.21.15.16:0/3182135152 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8dd8134630 msgr2=0x7f8dd811a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:11.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.937+0000 7f8dde9b1700 1 --2- 172.21.15.16:0/3182135152 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8dd8134630 0x7f8dd811a090 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc00ebe0 tx=0x7f8dcc00eef0 comp rx=0 tx=0).stop 2023-09-30T22:35:11.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.937+0000 7f8dde9b1700 1 -- 172.21.15.16:0/3182135152 shutdown_connections 2023-09-30T22:35:11.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.937+0000 7f8dde9b1700 1 --2- 172.21.15.16:0/3182135152 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8dd8131810 0x7f8dd8119b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:11.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.937+0000 7f8dde9b1700 1 --2- 172.21.15.16:0/3182135152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8dd8136f50 0x7f8dd81c63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:11.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.937+0000 7f8dde9b1700 1 --2- 172.21.15.16:0/3182135152 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8dd8134630 0x7f8dd811a090 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:11.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.937+0000 7f8dde9b1700 1 -- 172.21.15.16:0/3182135152 >> 172.21.15.16:0/3182135152 conn(0x7f8dd80a2aa0 msgr2=0x7f8dd812f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:11.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.937+0000 7f8dde9b1700 1 -- 172.21.15.16:0/3182135152 shutdown_connections 2023-09-30T22:35:11.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:11.937+0000 7f8dde9b1700 1 -- 172.21.15.16:0/3182135152 wait complete. 2023-09-30T22:35:11.954 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:35:12.308 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:35:12.383 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:35:12.414 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:35:12.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.457+0000 7f5e81ed8700 1 Processor -- start 2023-09-30T22:35:12.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.458+0000 7f5e81ed8700 1 -- start start 2023-09-30T22:35:12.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.458+0000 7f5e81ed8700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5e7c09e930 0x7f5e7c09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:12.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.458+0000 7f5e81ed8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5e7c09f240 0x7f5e7c1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:12.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.458+0000 7f5e81ed8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5e7c09f650 0x7f5e7c131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:12.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.458+0000 7f5e81ed8700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e7c070940 con 0x7f5e7c09e930 2023-09-30T22:35:12.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.458+0000 7f5e81ed8700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7c06fd60 con 0x7f5e7c09f650 2023-09-30T22:35:12.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.458+0000 7f5e81ed8700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7c06fa80 con 0x7f5e7c09f240 2023-09-30T22:35:12.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.458+0000 7f5e7b7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5e7c09e930 0x7f5e7c09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:12.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.458+0000 7f5e7b7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5e7c09e930 0x7f5e7c09ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49324/0 (socket says 172.21.15.16:49324) 2023-09-30T22:35:12.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.458+0000 7f5e7b7fe700 1 -- 172.21.15.16:0/3671470038 learned_addr learned my addr 172.21.15.16:0/3671470038 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:12.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.458+0000 7f5e7bfff700 1 --2- 172.21.15.16:0/3671470038 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5e7c09f650 0x7f5e7c131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:12.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.458+0000 7f5e7affd700 1 --2- 172.21.15.16:0/3671470038 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5e7c09f240 0x7f5e7c1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:12.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.459+0000 7f5e7b7fe700 1 -- 172.21.15.16:0/3671470038 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5e7c09f240 msgr2=0x7f5e7c1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:12.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.459+0000 7f5e7b7fe700 1 --2- 172.21.15.16:0/3671470038 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5e7c09f240 0x7f5e7c1317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:12.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.459+0000 7f5e7b7fe700 1 -- 172.21.15.16:0/3671470038 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5e7c09f650 msgr2=0x7f5e7c131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:12.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.459+0000 7f5e7b7fe700 1 --2- 172.21.15.16:0/3671470038 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5e7c09f650 0x7f5e7c131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:12.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.459+0000 7f5e7b7fe700 1 -- 172.21.15.16:0/3671470038 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e7c09fb60 con 0x7f5e7c09e930 2023-09-30T22:35:12.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.459+0000 7f5e7bfff700 1 --2- 172.21.15.16:0/3671470038 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5e7c09f650 0x7f5e7c131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:12.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.459+0000 7f5e7b7fe700 1 --2- 172.21.15.16:0/3671470038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e7c09e930 0x7f5e7c09ed00 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c0098f0 tx=0x7f5e6c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c750a4ede1b89216 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:12.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.459+0000 7f5e7a7fc700 1 -- 172.21.15.16:0/3671470038 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c011830 con 0x7f5e7c09e930 2023-09-30T22:35:12.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.459+0000 7f5e7a7fc700 1 -- 172.21.15.16:0/3671470038 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e6c011990 con 0x7f5e7c09e930 2023-09-30T22:35:12.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.459+0000 7f5e7a7fc700 1 -- 172.21.15.16:0/3671470038 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e6c011d20 con 0x7f5e7c09e930 2023-09-30T22:35:12.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.460+0000 7f5e81ed8700 1 -- 172.21.15.16:0/3671470038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e7c09e930 msgr2=0x7f5e7c09ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:12.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.460+0000 7f5e81ed8700 1 --2- 172.21.15.16:0/3671470038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e7c09e930 0x7f5e7c09ed00 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f5e6c0098f0 tx=0x7f5e6c009c00 comp rx=0 tx=0).stop 2023-09-30T22:35:12.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.460+0000 7f5e81ed8700 1 -- 172.21.15.16:0/3671470038 shutdown_connections 2023-09-30T22:35:12.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.460+0000 7f5e81ed8700 1 --2- 172.21.15.16:0/3671470038 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5e7c09f650 0x7f5e7c131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:12.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.460+0000 7f5e81ed8700 1 --2- 172.21.15.16:0/3671470038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e7c09e930 0x7f5e7c09ed00 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:12.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.460+0000 7f5e81ed8700 1 --2- 172.21.15.16:0/3671470038 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5e7c09f240 0x7f5e7c1317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:12.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.460+0000 7f5e81ed8700 1 -- 172.21.15.16:0/3671470038 >> 172.21.15.16:0/3671470038 conn(0x7f5e7c0a4210 msgr2=0x7f5e7c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:12.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.460+0000 7f5e81ed8700 1 -- 172.21.15.16:0/3671470038 shutdown_connections 2023-09-30T22:35:12.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.460+0000 7f5e81ed8700 1 -- 172.21.15.16:0/3671470038 wait complete. 2023-09-30T22:35:12.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.461+0000 7f5e81ed8700 1 Processor -- start 2023-09-30T22:35:12.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.461+0000 7f5e81ed8700 1 -- start start 2023-09-30T22:35:12.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.462+0000 7f5e81ed8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5e7c09e930 0x7f5e7c1c3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:12.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.462+0000 7f5e81ed8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5e7c09f240 0x7f5e7c1c42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:12.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.462+0000 7f5e81ed8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e7c09f650 0x7f5e7c1c4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:12.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.462+0000 7f5e81ed8700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7c1a40c0 con 0x7f5e7c09f650 2023-09-30T22:35:12.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.462+0000 7f5e81ed8700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7c1a2350 con 0x7f5e7c09f240 2023-09-30T22:35:12.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.462+0000 7f5e81ed8700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5e7c1a2120 con 0x7f5e7c09e930 2023-09-30T22:35:12.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.462+0000 7f5e7b7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5e7c09e930 0x7f5e7c1c3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:12.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.462+0000 7f5e7bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e7c09f650 0x7f5e7c1c4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:12.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.462+0000 7f5e7b7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5e7c09e930 0x7f5e7c1c3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:55336/0 (socket says 172.21.15.16:55336) 2023-09-30T22:35:12.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.462+0000 7f5e7b7fe700 1 -- 172.21.15.16:0/3561217346 learned_addr learned my addr 172.21.15.16:0/3561217346 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:12.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.462+0000 7f5e7affd700 1 --2- 172.21.15.16:0/3561217346 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5e7c09f240 0x7f5e7c1c42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:12.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.462+0000 7f5e7bfff700 1 -- 172.21.15.16:0/3561217346 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5e7c09e930 msgr2=0x7f5e7c1c3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:12.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.462+0000 7f5e7bfff700 1 --2- 172.21.15.16:0/3561217346 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5e7c09e930 0x7f5e7c1c3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:12.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.463+0000 7f5e7bfff700 1 -- 172.21.15.16:0/3561217346 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5e7c09f240 msgr2=0x7f5e7c1c42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:12.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.463+0000 7f5e7bfff700 1 --2- 172.21.15.16:0/3561217346 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5e7c09f240 0x7f5e7c1c42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:12.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.463+0000 7f5e7bfff700 1 -- 172.21.15.16:0/3561217346 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e74009a70 con 0x7f5e7c09f650 2023-09-30T22:35:12.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.463+0000 7f5e7affd700 1 --2- 172.21.15.16:0/3561217346 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5e7c09f240 0x7f5e7c1c42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:35:12.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.463+0000 7f5e7b7fe700 1 --2- 172.21.15.16:0/3561217346 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5e7c09e930 0x7f5e7c1c3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:12.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.463+0000 7f5e7bfff700 1 --2- 172.21.15.16:0/3561217346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e7c09f650 0x7f5e7c1c4810 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f5e74009e50 tx=0x7f5e7400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:12.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.463+0000 7f5e63fff700 1 -- 172.21.15.16:0/3561217346 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7400cb40 con 0x7f5e7c09f650 2023-09-30T22:35:12.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.463+0000 7f5e63fff700 1 -- 172.21.15.16:0/3561217346 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e7400f040 con 0x7f5e7c09f650 2023-09-30T22:35:12.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.463+0000 7f5e63fff700 1 -- 172.21.15.16:0/3561217346 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e74003a20 con 0x7f5e7c09f650 2023-09-30T22:35:12.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.463+0000 7f5e81ed8700 1 -- 172.21.15.16:0/3561217346 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e6c009550 con 0x7f5e7c09f650 2023-09-30T22:35:12.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.463+0000 7f5e81ed8700 1 -- 172.21.15.16:0/3561217346 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e7c0a0190 con 0x7f5e7c09f650 2023-09-30T22:35:12.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.464+0000 7f5e63fff700 1 -- 172.21.15.16:0/3561217346 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5e74014480 con 0x7f5e7c09f650 2023-09-30T22:35:12.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.464+0000 7f5e63fff700 1 -- 172.21.15.16:0/3561217346 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f5e7403ff20 con 0x7f5e7c09f650 2023-09-30T22:35:12.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.468+0000 7f5e81ed8700 1 -- 172.21.15.16:0/3561217346 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e7c071d20 con 0x7f5e7c09f650 2023-09-30T22:35:12.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.473+0000 7f5e63fff700 1 -- 172.21.15.16:0/3561217346 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5e74012930 con 0x7f5e7c09f650 2023-09-30T22:35:12.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.543+0000 7f460caee700 1 Processor -- start 2023-09-30T22:35:12.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.543+0000 7f460caee700 1 -- start start 2023-09-30T22:35:12.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.543+0000 7f460caee700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f46080a0a30 0x7f46080a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:12.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.543+0000 7f460caee700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f46080a1340 0x7f4608133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:12.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.544+0000 7f460caee700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f460809edd0 con 0x7f4608133fa0 2023-09-30T22:35:12.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.544+0000 7f460caee700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f460809e1f0 con 0x7f46080a0a30 2023-09-30T22:35:12.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.544+0000 7f460caee700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f460809dc50 con 0x7f46080a1340 2023-09-30T22:35:12.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.544+0000 7f4605d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f46080a1340 0x7f4608133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:12.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.544+0000 7f460659c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f46080a0a30 0x7f46080a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:12.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.544+0000 7f4605d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f46080a1340 0x7f4608133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:55342/0 (socket says 172.21.15.16:55342) 2023-09-30T22:35:12.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.544+0000 7f460659c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f46080a0a30 0x7f46080a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37812/0 (socket says 172.21.15.16:37812) 2023-09-30T22:35:12.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.544+0000 7f4605d9b700 1 -- 172.21.15.16:0/1295663133 learned_addr learned my addr 172.21.15.16:0/1295663133 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:12.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.544+0000 7f4605d9b700 1 -- 172.21.15.16:0/1295663133 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f46080a0a30 msgr2=0x7f46080a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:12.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.544+0000 7f4605d9b700 1 --2- 172.21.15.16:0/1295663133 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f46080a0a30 0x7f46080a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:12.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.544+0000 7f4605d9b700 1 -- 172.21.15.16:0/1295663133 >> v1:172.21.15.16:6789/0 conn(0x7f4608133fa0 legacy=0x7f460813ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:12.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.544+0000 7f4605d9b700 1 -- 172.21.15.16:0/1295663133 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46080a1fe0 con 0x7f46080a1340 2023-09-30T22:35:12.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.544+0000 7f460559a700 1 -- 172.21.15.16:0/1295663133 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 142878911 0 0) 0x7f460809edd0 con 0x7f4608133fa0 2023-09-30T22:35:12.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.544+0000 7f460659c700 1 --2- 172.21.15.16:0/1295663133 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f46080a0a30 0x7f46080a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:12.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.544+0000 7f4605d9b700 1 --2- 172.21.15.16:0/1295663133 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f46080a1340 0x7f4608133a60 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f45fc009920 tx=0x7f45fc009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6177c6daf762f6e1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:12.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.545+0000 7f460559a700 1 -- 172.21.15.16:0/1295663133 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45fc011830 con 0x7f46080a1340 2023-09-30T22:35:12.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.545+0000 7f460559a700 1 -- 172.21.15.16:0/1295663133 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45fc011990 con 0x7f46080a1340 2023-09-30T22:35:12.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.545+0000 7f460559a700 1 -- 172.21.15.16:0/1295663133 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45fc011d20 con 0x7f46080a1340 2023-09-30T22:35:12.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.545+0000 7f460caee700 1 -- 172.21.15.16:0/1295663133 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f46080a1340 msgr2=0x7f4608133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:12.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.545+0000 7f460caee700 1 --2- 172.21.15.16:0/1295663133 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f46080a1340 0x7f4608133a60 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f45fc009920 tx=0x7f45fc009c30 comp rx=0 tx=0).stop 2023-09-30T22:35:12.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.545+0000 7f460caee700 1 -- 172.21.15.16:0/1295663133 shutdown_connections 2023-09-30T22:35:12.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.545+0000 7f460caee700 1 --2- 172.21.15.16:0/1295663133 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f46080a0a30 0x7f46080a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:12.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.545+0000 7f460caee700 1 --2- 172.21.15.16:0/1295663133 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f46080a1340 0x7f4608133a60 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:12.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.545+0000 7f460caee700 1 -- 172.21.15.16:0/1295663133 >> 172.21.15.16:0/1295663133 conn(0x7f46080a6550 msgr2=0x7f46080a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:12.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.545+0000 7f460caee700 1 -- 172.21.15.16:0/1295663133 shutdown_connections 2023-09-30T22:35:12.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.546+0000 7f460caee700 1 -- 172.21.15.16:0/1295663133 wait complete. 2023-09-30T22:35:12.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.547+0000 7f460caee700 1 Processor -- start 2023-09-30T22:35:12.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.547+0000 7f460caee700 1 -- start start 2023-09-30T22:35:12.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.547+0000 7f460caee700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f46080a0a30 0x7f46081231b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:12.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.547+0000 7f460caee700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f46080a1340 0x7f46081236f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:12.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.547+0000 7f460caee700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4608133fa0 0x7f46081c8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:12.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.547+0000 7f460caee700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4608126160 con 0x7f4608133fa0 2023-09-30T22:35:12.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.547+0000 7f460caee700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4608122e30 con 0x7f46080a1340 2023-09-30T22:35:12.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.547+0000 7f460caee700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4608124e00 con 0x7f46080a0a30 2023-09-30T22:35:12.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.547+0000 7f460659c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f46080a0a30 0x7f46081231b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:12.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.547+0000 7f460659c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f46080a0a30 0x7f46081231b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:55358/0 (socket says 172.21.15.16:55358) 2023-09-30T22:35:12.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.547+0000 7f460659c700 1 -- 172.21.15.16:0/783396748 learned_addr learned my addr 172.21.15.16:0/783396748 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:12.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.548+0000 7f4605d9b700 1 --2- 172.21.15.16:0/783396748 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f46080a1340 0x7f46081236f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:12.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.548+0000 7f460659c700 1 -- 172.21.15.16:0/783396748 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f46080a1340 msgr2=0x7f46081236f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:12.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.548+0000 7f460659c700 1 --2- 172.21.15.16:0/783396748 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f46080a1340 0x7f46081236f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:12.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.548+0000 7f460659c700 1 -- 172.21.15.16:0/783396748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4608133fa0 msgr2=0x7f46081c8280 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:35:12.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.548+0000 7f460659c700 1 --2- 172.21.15.16:0/783396748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4608133fa0 0x7f46081c8280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:12.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.548+0000 7f460659c700 1 -- 172.21.15.16:0/783396748 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f80094a0 con 0x7f46080a0a30 2023-09-30T22:35:12.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.548+0000 7f4605d9b700 1 --2- 172.21.15.16:0/783396748 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f46080a1340 0x7f46081236f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:12.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.548+0000 7f460659c700 1 --2- 172.21.15.16:0/783396748 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f46080a0a30 0x7f46081231b0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f45f800ebe0 tx=0x7f45f800eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:12.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.548+0000 7f45f6ffd700 1 -- 172.21.15.16:0/783396748 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f8009800 con 0x7f46080a0a30 2023-09-30T22:35:12.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.548+0000 7f45f6ffd700 1 -- 172.21.15.16:0/783396748 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45f800c910 con 0x7f46080a0a30 2023-09-30T22:35:12.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.548+0000 7f460caee700 1 -- 172.21.15.16:0/783396748 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45fc009550 con 0x7f46080a0a30 2023-09-30T22:35:12.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.548+0000 7f45f6ffd700 1 -- 172.21.15.16:0/783396748 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f800cc80 con 0x7f46080a0a30 2023-09-30T22:35:12.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.548+0000 7f460caee700 1 -- 172.21.15.16:0/783396748 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46080a25e0 con 0x7f46080a0a30 2023-09-30T22:35:12.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.549+0000 7f45f6ffd700 1 -- 172.21.15.16:0/783396748 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f45f8013470 con 0x7f46080a0a30 2023-09-30T22:35:12.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.549+0000 7f45f6ffd700 1 -- 172.21.15.16:0/783396748 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f45f803f830 con 0x7f46080a0a30 2023-09-30T22:35:12.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.550+0000 7f460caee700 1 -- 172.21.15.16:0/783396748 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4608124590 con 0x7f46080a0a30 2023-09-30T22:35:12.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.555+0000 7f45f6ffd700 1 -- 172.21.15.16:0/783396748 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f45f8013720 con 0x7f46080a0a30 2023-09-30T22:35:12.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.574+0000 7f2bd5a2a700 1 Processor -- start 2023-09-30T22:35:12.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.574+0000 7f2bd5a2a700 1 -- start start 2023-09-30T22:35:12.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.574+0000 7f2bd5a2a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2bd009e930 0x7f2bd009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:12.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.574+0000 7f2bd5a2a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2bd009f650 0x7f2bd0131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:12.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.574+0000 7f2bd5a2a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bd0070940 con 0x7f2bd009f240 2023-09-30T22:35:12.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.574+0000 7f2bd5a2a700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bd006fd60 con 0x7f2bd009e930 2023-09-30T22:35:12.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.574+0000 7f2bd5a2a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd006fa80 con 0x7f2bd009f650 2023-09-30T22:35:12.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.575+0000 7f2bce7fc700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2bd009f240 0x7f2bd01317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51782/0 (socket says 172.21.15.16:51782) 2023-09-30T22:35:12.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.575+0000 7f2bce7fc700 1 -- 172.21.15.16:0/2791193587 learned_addr learned my addr 172.21.15.16:0/2791193587 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:12.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.575+0000 7f2bcf7fe700 1 --2- 172.21.15.16:0/2791193587 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2bd009f650 0x7f2bd0131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:12.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.575+0000 7f2bceffd700 1 --2- 172.21.15.16:0/2791193587 >> v2:172.21.15.16:3300/0 conn(0x7f2bd009e930 0x7f2bd009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:12.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.575+0000 7f2bcdffb700 1 -- 172.21.15.16:0/2791193587 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3185071344 0 0) 0x7f2bd0070940 con 0x7f2bd009f240 2023-09-30T22:35:12.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.575+0000 7f2bcdffb700 1 -- 172.21.15.16:0/2791193587 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bb80035b0 con 0x7f2bd009f240 2023-09-30T22:35:12.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.575+0000 7f2bcf7fe700 1 -- 172.21.15.16:0/2791193587 >> v2:172.21.15.16:3300/0 conn(0x7f2bd009e930 msgr2=0x7f2bd009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:12.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.575+0000 7f2bcf7fe700 1 --2- 172.21.15.16:0/2791193587 >> v2:172.21.15.16:3300/0 conn(0x7f2bd009e930 0x7f2bd009ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:12.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.575+0000 7f2bcf7fe700 1 -- 172.21.15.16:0/2791193587 >> v1:172.21.15.16:6789/0 conn(0x7f2bd009f240 legacy=0x7f2bd01317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:12.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.575+0000 7f2bcf7fe700 1 -- 172.21.15.16:0/2791193587 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd009fb60 con 0x7f2bd009f650 2023-09-30T22:35:12.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.575+0000 7f2bceffd700 1 --2- 172.21.15.16:0/2791193587 >> v2:172.21.15.16:3300/0 conn(0x7f2bd009e930 0x7f2bd009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:12.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.575+0000 7f2bcf7fe700 1 --2- 172.21.15.16:0/2791193587 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2bd009f650 0x7f2bd0131f00 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8009d90 tx=0x7f2bc800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=67651e0421986b75 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:12.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.575+0000 7f2bcdffb700 1 -- 172.21.15.16:0/2791193587 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc800e070 con 0x7f2bd009f650 2023-09-30T22:35:12.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.575+0000 7f2bcdffb700 1 -- 172.21.15.16:0/2791193587 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc800bb20 con 0x7f2bd009f650 2023-09-30T22:35:12.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.575+0000 7f2bcdffb700 1 -- 172.21.15.16:0/2791193587 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8012430 con 0x7f2bd009f650 2023-09-30T22:35:12.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.576+0000 7f2bd5a2a700 1 -- 172.21.15.16:0/2791193587 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2bd009f650 msgr2=0x7f2bd0131f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:12.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.576+0000 7f2bd5a2a700 1 --2- 172.21.15.16:0/2791193587 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2bd009f650 0x7f2bd0131f00 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8009d90 tx=0x7f2bc800b330 comp rx=0 tx=0).stop 2023-09-30T22:35:12.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.576+0000 7f2bd5a2a700 1 -- 172.21.15.16:0/2791193587 shutdown_connections 2023-09-30T22:35:12.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.576+0000 7f2bd5a2a700 1 --2- 172.21.15.16:0/2791193587 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2bd009f650 0x7f2bd0131f00 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:12.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.576+0000 7f2bd5a2a700 1 --2- 172.21.15.16:0/2791193587 >> v2:172.21.15.16:3300/0 conn(0x7f2bd009e930 0x7f2bd009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:12.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.576+0000 7f2bd5a2a700 1 -- 172.21.15.16:0/2791193587 >> 172.21.15.16:0/2791193587 conn(0x7f2bd00a4210 msgr2=0x7f2bd00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:12.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.577+0000 7f2bd5a2a700 1 -- 172.21.15.16:0/2791193587 shutdown_connections 2023-09-30T22:35:12.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.577+0000 7f2bd5a2a700 1 -- 172.21.15.16:0/2791193587 wait complete. 2023-09-30T22:35:12.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.578+0000 7f2bd5a2a700 1 Processor -- start 2023-09-30T22:35:12.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.578+0000 7f2bd5a2a700 1 -- start start 2023-09-30T22:35:12.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.579+0000 7f2bd5a2a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bd009e930 0x7f2bd01c61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:12.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.579+0000 7f2bd5a2a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2bd009f240 0x7f2bd01c6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:12.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.579+0000 7f2bd5a2a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2bd009f650 0x7f2bd01c27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:12.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.579+0000 7f2bd5a2a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd01b73c0 con 0x7f2bd009e930 2023-09-30T22:35:12.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.579+0000 7f2bd5a2a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd0071d20 con 0x7f2bd009f240 2023-09-30T22:35:12.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.579+0000 7f2bd5a2a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd01a4f40 con 0x7f2bd009f650 2023-09-30T22:35:12.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.579+0000 7f2bceffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bd009e930 0x7f2bd01c61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:12.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.579+0000 7f2bceffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bd009e930 0x7f2bd01c61f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49344/0 (socket says 172.21.15.16:49344) 2023-09-30T22:35:12.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.579+0000 7f2bcf7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2bd009f650 0x7f2bd01c27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:12.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.579+0000 7f2bceffd700 1 -- 172.21.15.16:0/3086029883 learned_addr learned my addr 172.21.15.16:0/3086029883 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:12.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.579+0000 7f2bce7fc700 1 --2- 172.21.15.16:0/3086029883 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2bd009f240 0x7f2bd01c6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:12.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.580+0000 7f2bceffd700 1 -- 172.21.15.16:0/3086029883 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2bd009f650 msgr2=0x7f2bd01c27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:12.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.580+0000 7f2bceffd700 1 --2- 172.21.15.16:0/3086029883 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2bd009f650 0x7f2bd01c27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:12.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.580+0000 7f2bceffd700 1 -- 172.21.15.16:0/3086029883 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2bd009f240 msgr2=0x7f2bd01c6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:12.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.580+0000 7f2bceffd700 1 --2- 172.21.15.16:0/3086029883 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2bd009f240 0x7f2bd01c6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:12.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.580+0000 7f2bceffd700 1 -- 172.21.15.16:0/3086029883 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc00094a0 con 0x7f2bd009e930 2023-09-30T22:35:12.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.580+0000 7f2bceffd700 1 --2- 172.21.15.16:0/3086029883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bd009e930 0x7f2bd01c61f0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f2bc000ebe0 tx=0x7f2bc000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:12.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.580+0000 7f2bb7fff700 1 -- 172.21.15.16:0/3086029883 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc000c7f0 con 0x7f2bd009e930 2023-09-30T22:35:12.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.580+0000 7f2bb7fff700 1 -- 172.21.15.16:0/3086029883 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc000c950 con 0x7f2bd009e930 2023-09-30T22:35:12.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.580+0000 7f2bd5a2a700 1 -- 172.21.15.16:0/3086029883 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bc8009a70 con 0x7f2bd009e930 2023-09-30T22:35:12.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.580+0000 7f2bb7fff700 1 -- 172.21.15.16:0/3086029883 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc000cce0 con 0x7f2bd009e930 2023-09-30T22:35:12.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.580+0000 7f2bd5a2a700 1 -- 172.21.15.16:0/3086029883 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bd00a0190 con 0x7f2bd009e930 2023-09-30T22:35:12.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.581+0000 7f2bb7fff700 1 -- 172.21.15.16:0/3086029883 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2bc0013470 con 0x7f2bd009e930 2023-09-30T22:35:12.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.581+0000 7f2bb7fff700 1 -- 172.21.15.16:0/3086029883 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f2bc003f780 con 0x7f2bd009e930 2023-09-30T22:35:12.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.581+0000 7f2bd5a2a700 1 -- 172.21.15.16:0/3086029883 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bd01c6c70 con 0x7f2bd009e930 2023-09-30T22:35:12.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.586+0000 7f2bb7fff700 1 -- 172.21.15.16:0/3086029883 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2bc0021070 con 0x7f2bd009e930 2023-09-30T22:35:12.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.704+0000 7f5e81ed8700 1 -- 172.21.15.16:0/3561217346 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5e7c1a3f20 con 0x7f5e7c09f650 2023-09-30T22:35:12.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.704+0000 7f5e63fff700 1 -- 172.21.15.16:0/3561217346 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5e7c1a3f20 con 0x7f5e7c09f650 2023-09-30T22:35:12.705 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:35:12.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.706+0000 7f5e81ed8700 1 -- 172.21.15.16:0/3561217346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e7c09f650 msgr2=0x7f5e7c1c4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:12.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.706+0000 7f5e81ed8700 1 --2- 172.21.15.16:0/3561217346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e7c09f650 0x7f5e7c1c4810 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f5e74009e50 tx=0x7f5e7400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:35:12.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.706+0000 7f5e81ed8700 1 -- 172.21.15.16:0/3561217346 shutdown_connections 2023-09-30T22:35:12.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.706+0000 7f5e81ed8700 1 --2- 172.21.15.16:0/3561217346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5e7c09f650 0x7f5e7c1c4810 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:12.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.706+0000 7f5e81ed8700 1 --2- 172.21.15.16:0/3561217346 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5e7c09e930 0x7f5e7c1c3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:12.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.706+0000 7f5e81ed8700 1 --2- 172.21.15.16:0/3561217346 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5e7c09f240 0x7f5e7c1c42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:12.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.706+0000 7f5e81ed8700 1 -- 172.21.15.16:0/3561217346 >> 172.21.15.16:0/3561217346 conn(0x7f5e7c0a4210 msgr2=0x7f5e7c06c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:12.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.707+0000 7f5e81ed8700 1 -- 172.21.15.16:0/3561217346 shutdown_connections 2023-09-30T22:35:12.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.707+0000 7f5e81ed8700 1 -- 172.21.15.16:0/3561217346 wait complete. 2023-09-30T22:35:12.721 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:35:12.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.794+0000 7f460caee700 1 -- 172.21.15.16:0/783396748 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f4608074ea0 con 0x7f46080a0a30 2023-09-30T22:35:12.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.794+0000 7f45f6ffd700 1 -- 172.21.15.16:0/783396748 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f45f8021030 con 0x7f46080a0a30 2023-09-30T22:35:12.795 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:35:12.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.796+0000 7f460caee700 1 -- 172.21.15.16:0/783396748 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f46080a0a30 msgr2=0x7f46081231b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:12.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.796+0000 7f460caee700 1 --2- 172.21.15.16:0/783396748 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f46080a0a30 0x7f46081231b0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f45f800ebe0 tx=0x7f45f800eef0 comp rx=0 tx=0).stop 2023-09-30T22:35:12.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.797+0000 7f460caee700 1 -- 172.21.15.16:0/783396748 shutdown_connections 2023-09-30T22:35:12.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.797+0000 7f460caee700 1 --2- 172.21.15.16:0/783396748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4608133fa0 0x7f46081c8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:12.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.797+0000 7f460caee700 1 --2- 172.21.15.16:0/783396748 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f46080a0a30 0x7f46081231b0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:12.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.797+0000 7f460caee700 1 --2- 172.21.15.16:0/783396748 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f46080a1340 0x7f46081236f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:12.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.797+0000 7f460caee700 1 -- 172.21.15.16:0/783396748 >> 172.21.15.16:0/783396748 conn(0x7f46080a6550 msgr2=0x7f460811c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:12.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.797+0000 7f460caee700 1 -- 172.21.15.16:0/783396748 shutdown_connections 2023-09-30T22:35:12.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.797+0000 7f460caee700 1 -- 172.21.15.16:0/783396748 wait complete. 2023-09-30T22:35:12.811 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:35:12.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.821+0000 7f2bd5a2a700 1 -- 172.21.15.16:0/3086029883 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2bd01a5c00 con 0x7f2bd009e930 2023-09-30T22:35:12.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.821+0000 7f2bb7fff700 1 -- 172.21.15.16:0/3086029883 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2bc001b970 con 0x7f2bd009e930 2023-09-30T22:35:12.821 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:35:12.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.823+0000 7f2bd5a2a700 1 -- 172.21.15.16:0/3086029883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bd009e930 msgr2=0x7f2bd01c61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:12.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.823+0000 7f2bd5a2a700 1 --2- 172.21.15.16:0/3086029883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bd009e930 0x7f2bd01c61f0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f2bc000ebe0 tx=0x7f2bc000eef0 comp rx=0 tx=0).stop 2023-09-30T22:35:12.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.823+0000 7f2bd5a2a700 1 -- 172.21.15.16:0/3086029883 shutdown_connections 2023-09-30T22:35:12.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.823+0000 7f2bd5a2a700 1 --2- 172.21.15.16:0/3086029883 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2bd009f650 0x7f2bd01c27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:12.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.823+0000 7f2bd5a2a700 1 --2- 172.21.15.16:0/3086029883 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bd009e930 0x7f2bd01c61f0 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:12.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.823+0000 7f2bd5a2a700 1 --2- 172.21.15.16:0/3086029883 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2bd009f240 0x7f2bd01c6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:12.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.823+0000 7f2bd5a2a700 1 -- 172.21.15.16:0/3086029883 >> 172.21.15.16:0/3086029883 conn(0x7f2bd00a4210 msgr2=0x7f2bd01b8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:12.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.823+0000 7f2bd5a2a700 1 -- 172.21.15.16:0/3086029883 shutdown_connections 2023-09-30T22:35:12.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:12.823+0000 7f2bd5a2a700 1 -- 172.21.15.16:0/3086029883 wait complete. 2023-09-30T22:35:12.837 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:35:12.955 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:35:13.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.099+0000 7fb6c364a700 1 Processor -- start 2023-09-30T22:35:13.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.099+0000 7fb6c364a700 1 -- start start 2023-09-30T22:35:13.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.099+0000 7fb6c364a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb6bc09e1d0 0x7fb6bc138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:13.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.099+0000 7fb6c364a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb6bc139170 0x7fb6bc13b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:13.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.100+0000 7fb6c364a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6bc09b120 con 0x7fb6bc09d6f0 2023-09-30T22:35:13.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.100+0000 7fb6c364a700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6bc06b460 con 0x7fb6bc139170 2023-09-30T22:35:13.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.100+0000 7fb6c0be5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb6bc09e1d0 0x7fb6bc138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:13.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.100+0000 7fb6c13e6700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb6bc09d6f0 0x7fb6bc09dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51790/0 (socket says 172.21.15.16:51790) 2023-09-30T22:35:13.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.100+0000 7fb6c1be7700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb6bc139170 0x7fb6bc13b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:13.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.100+0000 7fb6c364a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6bc09a7e0 con 0x7fb6bc09e1d0 2023-09-30T22:35:13.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.100+0000 7fb6c13e6700 1 -- 172.21.15.16:0/396454326 learned_addr learned my addr 172.21.15.16:0/396454326 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:13.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.100+0000 7fb6c0be5700 1 -- 172.21.15.16:0/396454326 >> v2:172.21.15.16:3300/0 conn(0x7fb6bc139170 msgr2=0x7fb6bc13b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:13.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.100+0000 7fb6c0be5700 1 --2- 172.21.15.16:0/396454326 >> v2:172.21.15.16:3300/0 conn(0x7fb6bc139170 0x7fb6bc13b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:13.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.100+0000 7fb6c0be5700 1 -- 172.21.15.16:0/396454326 >> v1:172.21.15.16:6789/0 conn(0x7fb6bc09d6f0 legacy=0x7fb6bc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:13.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.100+0000 7fb6c0be5700 1 -- 172.21.15.16:0/396454326 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6bc09a500 con 0x7fb6bc09e1d0 2023-09-30T22:35:13.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.100+0000 7fb6c1be7700 1 --2- 172.21.15.16:0/396454326 >> v2:172.21.15.16:3300/0 conn(0x7fb6bc139170 0x7fb6bc13b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:13.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.100+0000 7fb6c0be5700 1 --2- 172.21.15.16:0/396454326 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb6bc09e1d0 0x7fb6bc138c30 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac009870 tx=0x7fb6ac009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=652822a747572947 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:13.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.100+0000 7fb6b3fff700 1 -- 172.21.15.16:0/396454326 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ac011830 con 0x7fb6bc09e1d0 2023-09-30T22:35:13.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.100+0000 7fb6b3fff700 1 -- 172.21.15.16:0/396454326 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6ac011990 con 0x7fb6bc09e1d0 2023-09-30T22:35:13.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.100+0000 7fb6b3fff700 1 -- 172.21.15.16:0/396454326 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ac011d20 con 0x7fb6bc09e1d0 2023-09-30T22:35:13.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.101+0000 7fb6c364a700 1 -- 172.21.15.16:0/396454326 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb6bc09e1d0 msgr2=0x7fb6bc138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:13.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.101+0000 7fb6c364a700 1 --2- 172.21.15.16:0/396454326 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb6bc09e1d0 0x7fb6bc138c30 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac009870 tx=0x7fb6ac009b80 comp rx=0 tx=0).stop 2023-09-30T22:35:13.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.101+0000 7fb6c364a700 1 -- 172.21.15.16:0/396454326 shutdown_connections 2023-09-30T22:35:13.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.101+0000 7fb6c364a700 1 --2- 172.21.15.16:0/396454326 >> v2:172.21.15.16:3300/0 conn(0x7fb6bc139170 0x7fb6bc13b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:13.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.101+0000 7fb6c364a700 1 --2- 172.21.15.16:0/396454326 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb6bc09e1d0 0x7fb6bc138c30 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:13.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.101+0000 7fb6c364a700 1 -- 172.21.15.16:0/396454326 >> 172.21.15.16:0/396454326 conn(0x7fb6bc066f50 msgr2=0x7fb6bc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:13.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.101+0000 7fb6c364a700 1 -- 172.21.15.16:0/396454326 shutdown_connections 2023-09-30T22:35:13.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.101+0000 7fb6c364a700 1 -- 172.21.15.16:0/396454326 wait complete. 2023-09-30T22:35:13.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.102+0000 7fb6c364a700 1 Processor -- start 2023-09-30T22:35:13.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.102+0000 7fb6c364a700 1 -- start start 2023-09-30T22:35:13.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.103+0000 7fb6c364a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb6bc09d6f0 0x7fb6bc1c8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:13.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.103+0000 7fb6c364a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb6bc09e1d0 0x7fb6bc1c89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:13.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.103+0000 7fb6c364a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6bc139170 0x7fb6bc1c8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:13.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.103+0000 7fb6c364a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6bc0a2c50 con 0x7fb6bc139170 2023-09-30T22:35:13.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.103+0000 7fb6c364a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6bc120850 con 0x7fb6bc09d6f0 2023-09-30T22:35:13.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.103+0000 7fb6c364a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb6bc09c710 con 0x7fb6bc09e1d0 2023-09-30T22:35:13.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.103+0000 7fb6c13e6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb6bc09d6f0 0x7fb6bc1c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:13.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.103+0000 7fb6c13e6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb6bc09d6f0 0x7fb6bc1c8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37870/0 (socket says 172.21.15.16:37870) 2023-09-30T22:35:13.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.103+0000 7fb6c13e6700 1 -- 172.21.15.16:0/4149790618 learned_addr learned my addr 172.21.15.16:0/4149790618 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:13.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.103+0000 7fb6c1be7700 1 --2- 172.21.15.16:0/4149790618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6bc139170 0x7fb6bc1c8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:13.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.103+0000 7fb6c0be5700 1 --2- 172.21.15.16:0/4149790618 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb6bc09e1d0 0x7fb6bc1c89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:13.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.103+0000 7fb6c13e6700 1 -- 172.21.15.16:0/4149790618 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb6bc09e1d0 msgr2=0x7fb6bc1c89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:13.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.103+0000 7fb6c13e6700 1 --2- 172.21.15.16:0/4149790618 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb6bc09e1d0 0x7fb6bc1c89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:13.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.103+0000 7fb6c13e6700 1 -- 172.21.15.16:0/4149790618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6bc139170 msgr2=0x7fb6bc1c8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:13.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.103+0000 7fb6c13e6700 1 --2- 172.21.15.16:0/4149790618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6bc139170 0x7fb6bc1c8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:13.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.103+0000 7fb6c13e6700 1 -- 172.21.15.16:0/4149790618 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6b4009a70 con 0x7fb6bc09d6f0 2023-09-30T22:35:13.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.103+0000 7fb6c13e6700 1 --2- 172.21.15.16:0/4149790618 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb6bc09d6f0 0x7fb6bc1c8460 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fb6b800c580 tx=0x7fb6b800c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:13.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.103+0000 7fb6b1ffb700 1 -- 172.21.15.16:0/4149790618 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b8005d70 con 0x7fb6bc09d6f0 2023-09-30T22:35:13.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.103+0000 7fb6b1ffb700 1 -- 172.21.15.16:0/4149790618 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6b80070d0 con 0x7fb6bc09d6f0 2023-09-30T22:35:13.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.104+0000 7fb6c364a700 1 -- 172.21.15.16:0/4149790618 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6ac0094a0 con 0x7fb6bc09d6f0 2023-09-30T22:35:13.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.104+0000 7fb6b1ffb700 1 -- 172.21.15.16:0/4149790618 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b8007440 con 0x7fb6bc09d6f0 2023-09-30T22:35:13.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.104+0000 7fb6c364a700 1 -- 172.21.15.16:0/4149790618 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6bc122090 con 0x7fb6bc09d6f0 2023-09-30T22:35:13.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.104+0000 7fb6b1ffb700 1 -- 172.21.15.16:0/4149790618 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb6b80075a0 con 0x7fb6bc09d6f0 2023-09-30T22:35:13.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.105+0000 7fb6b1ffb700 1 -- 172.21.15.16:0/4149790618 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fb6b803ea00 con 0x7fb6bc09d6f0 2023-09-30T22:35:13.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.105+0000 7fb6c364a700 1 -- 172.21.15.16:0/4149790618 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6bc09e5a0 con 0x7fb6bc09d6f0 2023-09-30T22:35:13.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.110+0000 7fb6b1ffb700 1 -- 172.21.15.16:0/4149790618 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb6b8019b40 con 0x7fb6bc09d6f0 2023-09-30T22:35:13.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.316+0000 7fb6c364a700 1 -- 172.21.15.16:0/4149790618 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb6bc09f420 con 0x7fb6bc09d6f0 2023-09-30T22:35:13.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.317+0000 7fb6b1ffb700 1 -- 172.21.15.16:0/4149790618 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb6b8017020 con 0x7fb6bc09d6f0 2023-09-30T22:35:13.317 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:35:13.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.319+0000 7fb6c364a700 1 -- 172.21.15.16:0/4149790618 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb6bc09d6f0 msgr2=0x7fb6bc1c8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:13.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.319+0000 7fb6c364a700 1 --2- 172.21.15.16:0/4149790618 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb6bc09d6f0 0x7fb6bc1c8460 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fb6b800c580 tx=0x7fb6b800c890 comp rx=0 tx=0).stop 2023-09-30T22:35:13.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.319+0000 7fb6c364a700 1 -- 172.21.15.16:0/4149790618 shutdown_connections 2023-09-30T22:35:13.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.319+0000 7fb6c364a700 1 --2- 172.21.15.16:0/4149790618 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb6bc09d6f0 0x7fb6bc1c8460 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:13.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.319+0000 7fb6c364a700 1 --2- 172.21.15.16:0/4149790618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6bc139170 0x7fb6bc1c8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:13.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.319+0000 7fb6c364a700 1 --2- 172.21.15.16:0/4149790618 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb6bc09e1d0 0x7fb6bc1c89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:13.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.319+0000 7fb6c364a700 1 -- 172.21.15.16:0/4149790618 >> 172.21.15.16:0/4149790618 conn(0x7fb6bc066f50 msgr2=0x7fb6bc136c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:13.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.319+0000 7fb6c364a700 1 -- 172.21.15.16:0/4149790618 shutdown_connections 2023-09-30T22:35:13.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.319+0000 7fb6c364a700 1 -- 172.21.15.16:0/4149790618 wait complete. 2023-09-30T22:35:13.333 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:35:13.722 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:35:13.811 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:35:13.838 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:35:13.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.869+0000 7eff88dad700 1 Processor -- start 2023-09-30T22:35:13.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.869+0000 7eff88dad700 1 -- start start 2023-09-30T22:35:13.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.870+0000 7eff88dad700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7eff84136710 0x7eff84138af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:13.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.870+0000 7eff88dad700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7eff84139030 0x7eff8413b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:13.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.870+0000 7eff88dad700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff8406ef30 con 0x7eff841338a0 2023-09-30T22:35:13.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.870+0000 7eff88dad700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff8406fef0 con 0x7eff84139030 2023-09-30T22:35:13.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.870+0000 7eff88dad700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff841334b0 con 0x7eff84136710 2023-09-30T22:35:13.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.870+0000 7eff82d9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7eff84139030 0x7eff8413b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:13.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.870+0000 7eff82d9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7eff84139030 0x7eff8413b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49362/0 (socket says 172.21.15.16:49362) 2023-09-30T22:35:13.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.870+0000 7eff82d9d700 1 -- 172.21.15.16:0/3250264095 learned_addr learned my addr 172.21.15.16:0/3250264095 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:13.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.870+0000 7eff81d9b700 1 --2- 172.21.15.16:0/3250264095 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7eff84136710 0x7eff84138af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:13.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.870+0000 7eff81d9b700 1 -- 172.21.15.16:0/3250264095 >> v2:172.21.15.16:3300/0 conn(0x7eff84139030 msgr2=0x7eff8413b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:13.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.870+0000 7eff81d9b700 1 --2- 172.21.15.16:0/3250264095 >> v2:172.21.15.16:3300/0 conn(0x7eff84139030 0x7eff8413b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:13.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.870+0000 7eff81d9b700 1 -- 172.21.15.16:0/3250264095 >> v1:172.21.15.16:6789/0 conn(0x7eff841338a0 legacy=0x7eff84133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:13.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.870+0000 7eff81d9b700 1 -- 172.21.15.16:0/3250264095 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff8406fc30 con 0x7eff84136710 2023-09-30T22:35:13.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.871+0000 7eff81d9b700 1 --2- 172.21.15.16:0/3250264095 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7eff84136710 0x7eff84138af0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7eff78009840 tx=0x7eff78009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95004fc0bae9352c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:13.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.871+0000 7eff8159a700 1 -- 172.21.15.16:0/3250264095 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff78004230 con 0x7eff84136710 2023-09-30T22:35:13.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.871+0000 7eff8159a700 1 -- 172.21.15.16:0/3250264095 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff78004390 con 0x7eff84136710 2023-09-30T22:35:13.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.871+0000 7eff8159a700 1 -- 172.21.15.16:0/3250264095 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff780036a0 con 0x7eff84136710 2023-09-30T22:35:13.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.872+0000 7eff88dad700 1 -- 172.21.15.16:0/3250264095 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7eff84136710 msgr2=0x7eff84138af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:13.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.872+0000 7eff88dad700 1 --2- 172.21.15.16:0/3250264095 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7eff84136710 0x7eff84138af0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7eff78009840 tx=0x7eff78009b50 comp rx=0 tx=0).stop 2023-09-30T22:35:13.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.872+0000 7eff88dad700 1 -- 172.21.15.16:0/3250264095 shutdown_connections 2023-09-30T22:35:13.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.872+0000 7eff88dad700 1 --2- 172.21.15.16:0/3250264095 >> v2:172.21.15.16:3300/0 conn(0x7eff84139030 0x7eff8413b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:13.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.872+0000 7eff88dad700 1 --2- 172.21.15.16:0/3250264095 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7eff84136710 0x7eff84138af0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:13.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.872+0000 7eff88dad700 1 -- 172.21.15.16:0/3250264095 >> 172.21.15.16:0/3250264095 conn(0x7eff840a2aa0 msgr2=0x7eff840a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:13.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.872+0000 7eff88dad700 1 -- 172.21.15.16:0/3250264095 shutdown_connections 2023-09-30T22:35:13.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.872+0000 7eff88dad700 1 -- 172.21.15.16:0/3250264095 wait complete. 2023-09-30T22:35:13.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.873+0000 7eff88dad700 1 Processor -- start 2023-09-30T22:35:13.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.873+0000 7eff88dad700 1 -- start start 2023-09-30T22:35:13.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.874+0000 7eff88dad700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7eff841338a0 0x7eff841c8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:13.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.874+0000 7eff88dad700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7eff84136710 0x7eff841c8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:13.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.874+0000 7eff88dad700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff84139030 0x7eff8411ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:13.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.874+0000 7eff88dad700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff841c90d0 con 0x7eff84139030 2023-09-30T22:35:13.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.874+0000 7eff88dad700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff8411c9b0 con 0x7eff841338a0 2023-09-30T22:35:13.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.874+0000 7eff88dad700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7eff8411e9e0 con 0x7eff84136710 2023-09-30T22:35:13.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.874+0000 7eff81d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7eff84136710 0x7eff841c8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:13.876 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.874+0000 7eff8259c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7eff841338a0 0x7eff841c8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:13.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.874+0000 7eff81d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7eff84136710 0x7eff841c8b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:55378/0 (socket says 172.21.15.16:55378) 2023-09-30T22:35:13.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.874+0000 7eff81d9b700 1 -- 172.21.15.16:0/1356646886 learned_addr learned my addr 172.21.15.16:0/1356646886 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:13.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.874+0000 7eff82d9d700 1 --2- 172.21.15.16:0/1356646886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff84139030 0x7eff8411ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:13.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.874+0000 7eff8259c700 1 -- 172.21.15.16:0/1356646886 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7eff84136710 msgr2=0x7eff841c8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:13.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.874+0000 7eff8259c700 1 --2- 172.21.15.16:0/1356646886 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7eff84136710 0x7eff841c8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:13.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.874+0000 7eff8259c700 1 -- 172.21.15.16:0/1356646886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff84139030 msgr2=0x7eff8411ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:13.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.874+0000 7eff8259c700 1 --2- 172.21.15.16:0/1356646886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff84139030 0x7eff8411ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:13.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.874+0000 7eff8259c700 1 -- 172.21.15.16:0/1356646886 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff780094a0 con 0x7eff841338a0 2023-09-30T22:35:13.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.874+0000 7eff81d9b700 1 --2- 172.21.15.16:0/1356646886 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7eff84136710 0x7eff841c8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:13.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.874+0000 7eff8259c700 1 --2- 172.21.15.16:0/1356646886 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7eff841338a0 0x7eff841c8650 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7eff7400c780 tx=0x7eff7400ca90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:13.877 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.875+0000 7eff72ffd700 1 -- 172.21.15.16:0/1356646886 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff740054e0 con 0x7eff841338a0 2023-09-30T22:35:13.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.875+0000 7eff72ffd700 1 -- 172.21.15.16:0/1356646886 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff74003680 con 0x7eff841338a0 2023-09-30T22:35:13.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.875+0000 7eff88dad700 1 -- 172.21.15.16:0/1356646886 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff8411cf90 con 0x7eff841338a0 2023-09-30T22:35:13.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.875+0000 7eff72ffd700 1 -- 172.21.15.16:0/1356646886 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff740039f0 con 0x7eff841338a0 2023-09-30T22:35:13.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.875+0000 7eff88dad700 1 -- 172.21.15.16:0/1356646886 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff8406f640 con 0x7eff841338a0 2023-09-30T22:35:13.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.876+0000 7eff88dad700 1 -- 172.21.15.16:0/1356646886 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff84138e80 con 0x7eff841338a0 2023-09-30T22:35:13.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.877+0000 7eff72ffd700 1 -- 172.21.15.16:0/1356646886 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7eff74003b50 con 0x7eff841338a0 2023-09-30T22:35:13.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.877+0000 7eff72ffd700 1 -- 172.21.15.16:0/1356646886 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7eff7403ef50 con 0x7eff841338a0 2023-09-30T22:35:13.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.886+0000 7eff72ffd700 1 -- 172.21.15.16:0/1356646886 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7eff7401b3b0 con 0x7eff841338a0 2023-09-30T22:35:13.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.956+0000 7fbcd2ebf700 1 Processor -- start 2023-09-30T22:35:13.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.956+0000 7fbcd2ebf700 1 -- start start 2023-09-30T22:35:13.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.957+0000 7fbcd2ebf700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbccc0a1510 0x7fbccc138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:13.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.957+0000 7fbcd2ebf700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbccc139250 0x7fbccc13b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:13.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.957+0000 7fbcd2ebf700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbccc09edd0 con 0x7fbccc0a0a30 2023-09-30T22:35:13.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.957+0000 7fbcd2ebf700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc0a18e0 con 0x7fbccc0a1510 2023-09-30T22:35:13.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.957+0000 7fbcd2ebf700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc09e1f0 con 0x7fbccc139250 2023-09-30T22:35:13.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.957+0000 7fbcd0c5b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fbccc0a0a30 0x7fbccc0a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51804/0 (socket says 172.21.15.16:51804) 2023-09-30T22:35:13.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.957+0000 7fbcd0c5b700 1 -- 172.21.15.16:0/3720750843 learned_addr learned my addr 172.21.15.16:0/3720750843 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:13.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.957+0000 7fbccbfff700 1 --2- 172.21.15.16:0/3720750843 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbccc0a1510 0x7fbccc138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:13.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.957+0000 7fbcd145c700 1 --2- 172.21.15.16:0/3720750843 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbccc139250 0x7fbccc13b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:13.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.957+0000 7fbccbfff700 1 -- 172.21.15.16:0/3720750843 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbccc139250 msgr2=0x7fbccc13b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:13.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.957+0000 7fbccbfff700 1 --2- 172.21.15.16:0/3720750843 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbccc139250 0x7fbccc13b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:13.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.957+0000 7fbccbfff700 1 -- 172.21.15.16:0/3720750843 >> v1:172.21.15.16:6789/0 conn(0x7fbccc0a0a30 legacy=0x7fbccc0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:13.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.957+0000 7fbccbfff700 1 -- 172.21.15.16:0/3720750843 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbccc09dc50 con 0x7fbccc0a1510 2023-09-30T22:35:13.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.958+0000 7fbcd145c700 1 --2- 172.21.15.16:0/3720750843 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbccc139250 0x7fbccc13b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:13.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.958+0000 7fbccbfff700 1 --2- 172.21.15.16:0/3720750843 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbccc0a1510 0x7fbccc138d10 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8009870 tx=0x7fbcb8009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a142010a639f1d7d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:13.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.958+0000 7fbccb7fe700 1 -- 172.21.15.16:0/3720750843 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb8011830 con 0x7fbccc0a1510 2023-09-30T22:35:13.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.958+0000 7fbccb7fe700 1 -- 172.21.15.16:0/3720750843 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcb8011990 con 0x7fbccc0a1510 2023-09-30T22:35:13.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.958+0000 7fbccb7fe700 1 -- 172.21.15.16:0/3720750843 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb8011d20 con 0x7fbccc0a1510 2023-09-30T22:35:13.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.959+0000 7fbcd2ebf700 1 -- 172.21.15.16:0/3720750843 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbccc0a1510 msgr2=0x7fbccc138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:13.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.959+0000 7fbcd2ebf700 1 --2- 172.21.15.16:0/3720750843 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbccc0a1510 0x7fbccc138d10 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8009870 tx=0x7fbcb8009b80 comp rx=0 tx=0).stop 2023-09-30T22:35:13.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.959+0000 7fbcd2ebf700 1 -- 172.21.15.16:0/3720750843 shutdown_connections 2023-09-30T22:35:13.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.959+0000 7fbcd2ebf700 1 --2- 172.21.15.16:0/3720750843 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbccc139250 0x7fbccc13b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:13.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.959+0000 7fbcd2ebf700 1 --2- 172.21.15.16:0/3720750843 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbccc0a1510 0x7fbccc138d10 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:13.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.959+0000 7fbcd2ebf700 1 -- 172.21.15.16:0/3720750843 >> 172.21.15.16:0/3720750843 conn(0x7fbccc0a6550 msgr2=0x7fbccc0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:13.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.959+0000 7fbcd2ebf700 1 -- 172.21.15.16:0/3720750843 shutdown_connections 2023-09-30T22:35:13.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.959+0000 7fbcd2ebf700 1 -- 172.21.15.16:0/3720750843 wait complete. 2023-09-30T22:35:13.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.960+0000 7fbcd2ebf700 1 Processor -- start 2023-09-30T22:35:13.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.960+0000 7fbcd2ebf700 1 -- start start 2023-09-30T22:35:13.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.961+0000 7fbcd2ebf700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbccc0a0a30 0x7fbccc1223e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:13.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.961+0000 7fbcd2ebf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbccc0a1510 0x7fbccc122920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:13.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.961+0000 7fbcd2ebf700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbccc139250 0x7fbccc1c84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:13.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.961+0000 7fbcd2ebf700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc138e80 con 0x7fbccc0a1510 2023-09-30T22:35:13.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.961+0000 7fbcd2ebf700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc121f90 con 0x7fbccc0a0a30 2023-09-30T22:35:13.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.961+0000 7fbcd2ebf700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc125190 con 0x7fbccc139250 2023-09-30T22:35:13.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.961+0000 7fbccbfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbccc0a1510 0x7fbccc122920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:13.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.961+0000 7fbcd0c5b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbccc0a0a30 0x7fbccc1223e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:13.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.961+0000 7fbccbfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbccc0a1510 0x7fbccc122920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49376/0 (socket says 172.21.15.16:49376) 2023-09-30T22:35:13.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.961+0000 7fbccbfff700 1 -- 172.21.15.16:0/998993559 learned_addr learned my addr 172.21.15.16:0/998993559 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:13.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.961+0000 7fbcd145c700 1 --2- 172.21.15.16:0/998993559 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbccc139250 0x7fbccc1c84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:13.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.961+0000 7fbccbfff700 1 -- 172.21.15.16:0/998993559 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbccc139250 msgr2=0x7fbccc1c84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:13.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.961+0000 7fbccbfff700 1 --2- 172.21.15.16:0/998993559 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbccc139250 0x7fbccc1c84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:13.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.961+0000 7fbccbfff700 1 -- 172.21.15.16:0/998993559 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbccc0a0a30 msgr2=0x7fbccc1223e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:13.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.961+0000 7fbccbfff700 1 --2- 172.21.15.16:0/998993559 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbccc0a0a30 0x7fbccc1223e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:13.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.961+0000 7fbccbfff700 1 -- 172.21.15.16:0/998993559 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcc4009a70 con 0x7fbccc0a1510 2023-09-30T22:35:13.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.961+0000 7fbcd0c5b700 1 --2- 172.21.15.16:0/998993559 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbccc0a0a30 0x7fbccc1223e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:13.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.961+0000 7fbccbfff700 1 --2- 172.21.15.16:0/998993559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbccc0a1510 0x7fbccc122920 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fbcb80090b0 tx=0x7fbcb8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:13.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.962+0000 7fbcc97fa700 1 -- 172.21.15.16:0/998993559 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb8004450 con 0x7fbccc0a1510 2023-09-30T22:35:13.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.962+0000 7fbcc97fa700 1 -- 172.21.15.16:0/998993559 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcb801f070 con 0x7fbccc0a1510 2023-09-30T22:35:13.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.962+0000 7fbcc97fa700 1 -- 172.21.15.16:0/998993559 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb80038e0 con 0x7fbccc0a1510 2023-09-30T22:35:13.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.962+0000 7fbcd2ebf700 1 -- 172.21.15.16:0/998993559 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcb80094a0 con 0x7fbccc0a1510 2023-09-30T22:35:13.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.962+0000 7fbcd2ebf700 1 -- 172.21.15.16:0/998993559 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbccc0a0220 con 0x7fbccc0a1510 2023-09-30T22:35:13.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.963+0000 7fbcc97fa700 1 -- 172.21.15.16:0/998993559 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbcb8003a40 con 0x7fbccc0a1510 2023-09-30T22:35:13.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.963+0000 7fbcd2ebf700 1 -- 172.21.15.16:0/998993559 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbccc124ff0 con 0x7fbccc0a1510 2023-09-30T22:35:13.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.963+0000 7fbcc97fa700 1 -- 172.21.15.16:0/998993559 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fbcb8021070 con 0x7fbccc0a1510 2023-09-30T22:35:13.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.972+0000 7fbcc97fa700 1 -- 172.21.15.16:0/998993559 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbccc124ff0 con 0x7fbccc0a1510 2023-09-30T22:35:13.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.999+0000 7fc0378e2700 1 Processor -- start 2023-09-30T22:35:14.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:13.999+0000 7fc0378e2700 1 -- start start 2023-09-30T22:35:14.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.000+0000 7fc0378e2700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc030136820 0x7fc030138c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:14.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.000+0000 7fc0378e2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc030139140 0x7fc03013b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:14.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.000+0000 7fc0378e2700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc03006bc00 con 0x7fc030133a80 2023-09-30T22:35:14.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.000+0000 7fc0378e2700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc03006c570 con 0x7fc030136820 2023-09-30T22:35:14.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.000+0000 7fc0378e2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc0300a0040 con 0x7fc030139140 2023-09-30T22:35:14.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.000+0000 7fc034e7d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc030136820 0x7fc030138c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:14.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.000+0000 7fc035e7f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc030139140 0x7fc03013b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:14.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.000+0000 7fc034e7d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc030136820 0x7fc030138c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49378/0 (socket says 172.21.15.16:49378) 2023-09-30T22:35:14.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.000+0000 7fc034e7d700 1 -- 172.21.15.16:0/1534380936 learned_addr learned my addr 172.21.15.16:0/1534380936 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:14.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.001+0000 7fc034e7d700 1 -- 172.21.15.16:0/1534380936 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc030139140 msgr2=0x7fc03013b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:14.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.001+0000 7fc034e7d700 1 --2- 172.21.15.16:0/1534380936 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc030139140 0x7fc03013b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:14.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.001+0000 7fc034e7d700 1 -- 172.21.15.16:0/1534380936 >> v1:172.21.15.16:6789/0 conn(0x7fc030133a80 legacy=0x7fc030133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:14.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.001+0000 7fc034e7d700 1 -- 172.21.15.16:0/1534380936 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc030133860 con 0x7fc030136820 2023-09-30T22:35:14.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.001+0000 7fc027fff700 1 -- 172.21.15.16:0/1534380936 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2452187445 0 0) 0x7fc03006bc00 con 0x7fc030133a80 2023-09-30T22:35:14.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.001+0000 7fc034e7d700 1 --2- 172.21.15.16:0/1534380936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc030136820 0x7fc030138c00 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fc0200098f0 tx=0x7fc020009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=539d58c72feadae3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:14.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.001+0000 7fc027fff700 1 -- 172.21.15.16:0/1534380936 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc020011830 con 0x7fc030136820 2023-09-30T22:35:14.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.001+0000 7fc027fff700 1 -- 172.21.15.16:0/1534380936 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc020011990 con 0x7fc030136820 2023-09-30T22:35:14.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.001+0000 7fc027fff700 1 -- 172.21.15.16:0/1534380936 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc020011d20 con 0x7fc030136820 2023-09-30T22:35:14.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.002+0000 7fc0378e2700 1 -- 172.21.15.16:0/1534380936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc030136820 msgr2=0x7fc030138c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:14.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.002+0000 7fc0378e2700 1 --2- 172.21.15.16:0/1534380936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc030136820 0x7fc030138c00 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fc0200098f0 tx=0x7fc020009c00 comp rx=0 tx=0).stop 2023-09-30T22:35:14.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.002+0000 7fc0378e2700 1 -- 172.21.15.16:0/1534380936 shutdown_connections 2023-09-30T22:35:14.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.002+0000 7fc0378e2700 1 --2- 172.21.15.16:0/1534380936 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc030139140 0x7fc03013b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:14.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.002+0000 7fc0378e2700 1 --2- 172.21.15.16:0/1534380936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc030136820 0x7fc030138c00 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:14.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.002+0000 7fc0378e2700 1 -- 172.21.15.16:0/1534380936 >> 172.21.15.16:0/1534380936 conn(0x7fc0300a4210 msgr2=0x7fc0300a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:14.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.002+0000 7fc0378e2700 1 -- 172.21.15.16:0/1534380936 shutdown_connections 2023-09-30T22:35:14.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.002+0000 7fc0378e2700 1 -- 172.21.15.16:0/1534380936 wait complete. 2023-09-30T22:35:14.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.003+0000 7fc0378e2700 1 Processor -- start 2023-09-30T22:35:14.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.003+0000 7fc0378e2700 1 -- start start 2023-09-30T22:35:14.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.003+0000 7fc0378e2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc030133a80 0x7fc03006f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:14.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.003+0000 7fc0378e2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc030136820 0x7fc03006fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:14.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.003+0000 7fc0378e2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc030139140 0x7fc0300703d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:14.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.003+0000 7fc0378e2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc030070bd0 con 0x7fc030139140 2023-09-30T22:35:14.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.003+0000 7fc0378e2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc030136530 con 0x7fc030136820 2023-09-30T22:35:14.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.003+0000 7fc0378e2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc030071f50 con 0x7fc030133a80 2023-09-30T22:35:14.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.003+0000 7fc03567e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc030133a80 0x7fc03006f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:14.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.003+0000 7fc034e7d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc030136820 0x7fc03006fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:14.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.003+0000 7fc03567e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc030133a80 0x7fc03006f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:55404/0 (socket says 172.21.15.16:55404) 2023-09-30T22:35:14.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.003+0000 7fc03567e700 1 -- 172.21.15.16:0/459593730 learned_addr learned my addr 172.21.15.16:0/459593730 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:14.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.003+0000 7fc035e7f700 1 --2- 172.21.15.16:0/459593730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc030139140 0x7fc0300703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:14.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.004+0000 7fc03567e700 1 -- 172.21.15.16:0/459593730 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc030136820 msgr2=0x7fc03006fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:14.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.004+0000 7fc03567e700 1 --2- 172.21.15.16:0/459593730 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc030136820 0x7fc03006fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:14.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.004+0000 7fc03567e700 1 -- 172.21.15.16:0/459593730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc030139140 msgr2=0x7fc0300703d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:14.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.004+0000 7fc03567e700 1 --2- 172.21.15.16:0/459593730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc030139140 0x7fc0300703d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:14.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.004+0000 7fc03567e700 1 -- 172.21.15.16:0/459593730 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc020009550 con 0x7fc030133a80 2023-09-30T22:35:14.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.004+0000 7fc03567e700 1 --2- 172.21.15.16:0/459593730 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc030133a80 0x7fc03006f950 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fc02c001eb0 tx=0x7fc02c00c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:14.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.004+0000 7fc025ffb700 1 -- 172.21.15.16:0/459593730 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02c0054e0 con 0x7fc030133a80 2023-09-30T22:35:14.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.004+0000 7fc025ffb700 1 -- 172.21.15.16:0/459593730 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc02c003680 con 0x7fc030133a80 2023-09-30T22:35:14.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.004+0000 7fc0378e2700 1 -- 172.21.15.16:0/459593730 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0301309f0 con 0x7fc030133a80 2023-09-30T22:35:14.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.004+0000 7fc025ffb700 1 -- 172.21.15.16:0/459593730 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc02c0039f0 con 0x7fc030133a80 2023-09-30T22:35:14.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.004+0000 7fc0378e2700 1 -- 172.21.15.16:0/459593730 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc03009e330 con 0x7fc030133a80 2023-09-30T22:35:14.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.006+0000 7fc0378e2700 1 -- 172.21.15.16:0/459593730 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc030072450 con 0x7fc030133a80 2023-09-30T22:35:14.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.006+0000 7fc025ffb700 1 -- 172.21.15.16:0/459593730 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc02c003b50 con 0x7fc030133a80 2023-09-30T22:35:14.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.007+0000 7fc025ffb700 1 -- 172.21.15.16:0/459593730 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fc02c03efe0 con 0x7fc030133a80 2023-09-30T22:35:14.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.011+0000 7fc025ffb700 1 -- 172.21.15.16:0/459593730 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc02c01b3b0 con 0x7fc030133a80 2023-09-30T22:35:14.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.097+0000 7eff88dad700 1 -- 172.21.15.16:0/1356646886 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7eff84074ea0 con 0x7eff841338a0 2023-09-30T22:35:14.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.098+0000 7eff72ffd700 1 -- 172.21.15.16:0/1356646886 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7eff74017030 con 0x7eff841338a0 2023-09-30T22:35:14.098 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:35:14.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.100+0000 7eff88dad700 1 -- 172.21.15.16:0/1356646886 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7eff841338a0 msgr2=0x7eff841c8650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:14.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.100+0000 7eff88dad700 1 --2- 172.21.15.16:0/1356646886 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7eff841338a0 0x7eff841c8650 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7eff7400c780 tx=0x7eff7400ca90 comp rx=0 tx=0).stop 2023-09-30T22:35:14.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.100+0000 7eff88dad700 1 -- 172.21.15.16:0/1356646886 shutdown_connections 2023-09-30T22:35:14.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.100+0000 7eff88dad700 1 --2- 172.21.15.16:0/1356646886 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7eff841338a0 0x7eff841c8650 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:14.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.100+0000 7eff88dad700 1 --2- 172.21.15.16:0/1356646886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff84139030 0x7eff8411ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:14.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.100+0000 7eff88dad700 1 --2- 172.21.15.16:0/1356646886 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7eff84136710 0x7eff841c8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:14.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.100+0000 7eff88dad700 1 -- 172.21.15.16:0/1356646886 >> 172.21.15.16:0/1356646886 conn(0x7eff840a2aa0 msgr2=0x7eff841371c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:14.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.100+0000 7eff88dad700 1 -- 172.21.15.16:0/1356646886 shutdown_connections 2023-09-30T22:35:14.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.100+0000 7eff88dad700 1 -- 172.21.15.16:0/1356646886 wait complete. 2023-09-30T22:35:14.114 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:35:14.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.182+0000 7fbcd2ebf700 1 -- 172.21.15.16:0/998993559 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fbccc124ff0 con 0x7fbccc0a1510 2023-09-30T22:35:14.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.183+0000 7fbcc97fa700 1 -- 172.21.15.16:0/998993559 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbccc124ff0 con 0x7fbccc0a1510 2023-09-30T22:35:14.183 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:35:14.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.185+0000 7fbcd2ebf700 1 -- 172.21.15.16:0/998993559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbccc0a1510 msgr2=0x7fbccc122920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:14.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.185+0000 7fbcd2ebf700 1 --2- 172.21.15.16:0/998993559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbccc0a1510 0x7fbccc122920 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fbcb80090b0 tx=0x7fbcb8004750 comp rx=0 tx=0).stop 2023-09-30T22:35:14.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.185+0000 7fbcd2ebf700 1 -- 172.21.15.16:0/998993559 shutdown_connections 2023-09-30T22:35:14.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.185+0000 7fbcd2ebf700 1 --2- 172.21.15.16:0/998993559 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbccc0a0a30 0x7fbccc1223e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:14.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.185+0000 7fbcd2ebf700 1 --2- 172.21.15.16:0/998993559 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbccc139250 0x7fbccc1c84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:14.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.185+0000 7fbcd2ebf700 1 --2- 172.21.15.16:0/998993559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbccc0a1510 0x7fbccc122920 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:14.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.185+0000 7fbcd2ebf700 1 -- 172.21.15.16:0/998993559 >> 172.21.15.16:0/998993559 conn(0x7fbccc0a6550 msgr2=0x7fbccc137b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:14.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.185+0000 7fbcd2ebf700 1 -- 172.21.15.16:0/998993559 shutdown_connections 2023-09-30T22:35:14.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.185+0000 7fbcd2ebf700 1 -- 172.21.15.16:0/998993559 wait complete. 2023-09-30T22:35:14.199 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:35:14.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.217+0000 7fc0378e2700 1 -- 172.21.15.16:0/459593730 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc030074ea0 con 0x7fc030133a80 2023-09-30T22:35:14.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.218+0000 7fc025ffb700 1 -- 172.21.15.16:0/459593730 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc02c017030 con 0x7fc030133a80 2023-09-30T22:35:14.218 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:35:14.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.220+0000 7fc0378e2700 1 -- 172.21.15.16:0/459593730 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc030133a80 msgr2=0x7fc03006f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:14.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.220+0000 7fc0378e2700 1 --2- 172.21.15.16:0/459593730 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc030133a80 0x7fc03006f950 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fc02c001eb0 tx=0x7fc02c00c800 comp rx=0 tx=0).stop 2023-09-30T22:35:14.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.220+0000 7fc0378e2700 1 -- 172.21.15.16:0/459593730 shutdown_connections 2023-09-30T22:35:14.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.220+0000 7fc0378e2700 1 --2- 172.21.15.16:0/459593730 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc030139140 0x7fc0300703d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:14.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.220+0000 7fc0378e2700 1 --2- 172.21.15.16:0/459593730 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc030133a80 0x7fc03006f950 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:14.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.220+0000 7fc0378e2700 1 --2- 172.21.15.16:0/459593730 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc030136820 0x7fc03006fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:14.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.220+0000 7fc0378e2700 1 -- 172.21.15.16:0/459593730 >> 172.21.15.16:0/459593730 conn(0x7fc0300a4210 msgr2=0x7fc0300a0890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:14.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.220+0000 7fc0378e2700 1 -- 172.21.15.16:0/459593730 shutdown_connections 2023-09-30T22:35:14.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.220+0000 7fc0378e2700 1 -- 172.21.15.16:0/459593730 wait complete. 2023-09-30T22:35:14.234 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:35:14.334 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:35:14.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.474+0000 7f9e6a149700 1 Processor -- start 2023-09-30T22:35:14.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.474+0000 7f9e6a149700 1 -- start start 2023-09-30T22:35:14.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.475+0000 7f9e6a149700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e6409e930 0x7f9e6409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:14.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.475+0000 7f9e6a149700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9e6409f240 0x7f9e641317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:14.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.475+0000 7f9e6a149700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e6409f650 0x7f9e64131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:14.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.475+0000 7f9e6a149700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e64070940 con 0x7f9e6409f240 2023-09-30T22:35:14.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.475+0000 7f9e6a149700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6406fd60 con 0x7f9e6409e930 2023-09-30T22:35:14.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.475+0000 7f9e6a149700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6406fa80 con 0x7f9e6409f650 2023-09-30T22:35:14.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.475+0000 7f9e637fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e6409e930 0x7f9e6409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:14.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.475+0000 7f9e62ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9e6409f240 0x7f9e641317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:14.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.475+0000 7f9e62ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9e6409f240 0x7f9e641317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49400/0 (socket says 172.21.15.16:49400) 2023-09-30T22:35:14.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.475+0000 7f9e637fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e6409e930 0x7f9e6409ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37906/0 (socket says 172.21.15.16:37906) 2023-09-30T22:35:14.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.475+0000 7f9e63fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e6409f650 0x7f9e64131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:14.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.475+0000 7f9e637fe700 1 -- 172.21.15.16:0/2218743051 learned_addr learned my addr 172.21.15.16:0/2218743051 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:14.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.475+0000 7f9e637fe700 1 -- 172.21.15.16:0/2218743051 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e6409f650 msgr2=0x7f9e64131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:14.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.475+0000 7f9e637fe700 1 --2- 172.21.15.16:0/2218743051 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e6409f650 0x7f9e64131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:14.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.475+0000 7f9e637fe700 1 -- 172.21.15.16:0/2218743051 >> v2:172.21.15.16:3300/0 conn(0x7f9e6409f240 msgr2=0x7f9e641317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:14.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.475+0000 7f9e637fe700 1 --2- 172.21.15.16:0/2218743051 >> v2:172.21.15.16:3300/0 conn(0x7f9e6409f240 0x7f9e641317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:14.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.475+0000 7f9e637fe700 1 -- 172.21.15.16:0/2218743051 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e6409fb60 con 0x7f9e6409e930 2023-09-30T22:35:14.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.475+0000 7f9e62ffd700 1 --2- 172.21.15.16:0/2218743051 >> v2:172.21.15.16:3300/0 conn(0x7f9e6409f240 0x7f9e641317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:14.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.475+0000 7f9e63fff700 1 --2- 172.21.15.16:0/2218743051 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e6409f650 0x7f9e64131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:14.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.475+0000 7f9e637fe700 1 --2- 172.21.15.16:0/2218743051 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e6409e930 0x7f9e6409ed00 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c0098f0 tx=0x7f9e4c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9e250280a726f6c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:14.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.476+0000 7f9e627fc700 1 -- 172.21.15.16:0/2218743051 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c011830 con 0x7f9e6409e930 2023-09-30T22:35:14.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.476+0000 7f9e627fc700 1 -- 172.21.15.16:0/2218743051 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e4c011990 con 0x7f9e6409e930 2023-09-30T22:35:14.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.476+0000 7f9e627fc700 1 -- 172.21.15.16:0/2218743051 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c011d20 con 0x7f9e6409e930 2023-09-30T22:35:14.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.476+0000 7f9e6a149700 1 -- 172.21.15.16:0/2218743051 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e6409e930 msgr2=0x7f9e6409ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:14.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.476+0000 7f9e6a149700 1 --2- 172.21.15.16:0/2218743051 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e6409e930 0x7f9e6409ed00 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c0098f0 tx=0x7f9e4c009c00 comp rx=0 tx=0).stop 2023-09-30T22:35:14.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.476+0000 7f9e6a149700 1 -- 172.21.15.16:0/2218743051 shutdown_connections 2023-09-30T22:35:14.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.476+0000 7f9e6a149700 1 --2- 172.21.15.16:0/2218743051 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e6409e930 0x7f9e6409ed00 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:14.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.476+0000 7f9e6a149700 1 --2- 172.21.15.16:0/2218743051 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e6409f650 0x7f9e64131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:14.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.476+0000 7f9e6a149700 1 --2- 172.21.15.16:0/2218743051 >> v2:172.21.15.16:3300/0 conn(0x7f9e6409f240 0x7f9e641317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:14.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.476+0000 7f9e6a149700 1 -- 172.21.15.16:0/2218743051 >> 172.21.15.16:0/2218743051 conn(0x7f9e640a4210 msgr2=0x7f9e640a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:14.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.476+0000 7f9e6a149700 1 -- 172.21.15.16:0/2218743051 shutdown_connections 2023-09-30T22:35:14.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.476+0000 7f9e6a149700 1 -- 172.21.15.16:0/2218743051 wait complete. 2023-09-30T22:35:14.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.477+0000 7f9e6a149700 1 Processor -- start 2023-09-30T22:35:14.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.478+0000 7f9e6a149700 1 -- start start 2023-09-30T22:35:14.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.478+0000 7f9e6a149700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e6409e930 0x7f9e6406b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:14.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.478+0000 7f9e6a149700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e6409f240 0x7f9e6406bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:14.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.478+0000 7f9e6a149700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e6409f650 0x7f9e6406c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:14.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.478+0000 7f9e6a149700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e640a35c0 con 0x7f9e6409f240 2023-09-30T22:35:14.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.478+0000 7f9e6a149700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e64071d20 con 0x7f9e6409f650 2023-09-30T22:35:14.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.478+0000 7f9e6a149700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9e641aaa80 con 0x7f9e6409e930 2023-09-30T22:35:14.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.478+0000 7f9e637fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e6409e930 0x7f9e6406b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:14.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.478+0000 7f9e62ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e6409f240 0x7f9e6406bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:14.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.478+0000 7f9e637fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e6409e930 0x7f9e6406b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:55430/0 (socket says 172.21.15.16:55430) 2023-09-30T22:35:14.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.478+0000 7f9e637fe700 1 -- 172.21.15.16:0/730942297 learned_addr learned my addr 172.21.15.16:0/730942297 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:14.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.478+0000 7f9e63fff700 1 --2- 172.21.15.16:0/730942297 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e6409f650 0x7f9e6406c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:14.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.479+0000 7f9e637fe700 1 -- 172.21.15.16:0/730942297 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e6409f650 msgr2=0x7f9e6406c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:14.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.479+0000 7f9e637fe700 1 --2- 172.21.15.16:0/730942297 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e6409f650 0x7f9e6406c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:14.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.479+0000 7f9e637fe700 1 -- 172.21.15.16:0/730942297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e6409f240 msgr2=0x7f9e6406bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:14.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.479+0000 7f9e637fe700 1 --2- 172.21.15.16:0/730942297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e6409f240 0x7f9e6406bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:14.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.479+0000 7f9e637fe700 1 -- 172.21.15.16:0/730942297 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e540094a0 con 0x7f9e6409e930 2023-09-30T22:35:14.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.479+0000 7f9e62ffd700 1 --2- 172.21.15.16:0/730942297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e6409f240 0x7f9e6406bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:14.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.479+0000 7f9e63fff700 1 --2- 172.21.15.16:0/730942297 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e6409f650 0x7f9e6406c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:14.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.479+0000 7f9e637fe700 1 --2- 172.21.15.16:0/730942297 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e6409e930 0x7f9e6406b8b0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c000c00 tx=0x7f9e4c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:14.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.479+0000 7f9e5bfff700 1 -- 172.21.15.16:0/730942297 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c004240 con 0x7f9e6409e930 2023-09-30T22:35:14.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.479+0000 7f9e5bfff700 1 -- 172.21.15.16:0/730942297 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e4c01f070 con 0x7f9e6409e930 2023-09-30T22:35:14.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.479+0000 7f9e5bfff700 1 -- 172.21.15.16:0/730942297 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c0045e0 con 0x7f9e6409e930 2023-09-30T22:35:14.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.479+0000 7f9e6a149700 1 -- 172.21.15.16:0/730942297 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e5c009a70 con 0x7f9e6409e930 2023-09-30T22:35:14.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.479+0000 7f9e6a149700 1 -- 172.21.15.16:0/730942297 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e4c009550 con 0x7f9e6409e930 2023-09-30T22:35:14.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.480+0000 7f9e5bfff700 1 -- 172.21.15.16:0/730942297 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9e4c021030 con 0x7f9e6409e930 2023-09-30T22:35:14.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.480+0000 7f9e5bfff700 1 -- 172.21.15.16:0/730942297 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f9e4c0407a0 con 0x7f9e6409e930 2023-09-30T22:35:14.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.480+0000 7f9e6a149700 1 -- 172.21.15.16:0/730942297 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e641a2b40 con 0x7f9e6409e930 2023-09-30T22:35:14.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.485+0000 7f9e5bfff700 1 -- 172.21.15.16:0/730942297 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9e4c0093b0 con 0x7f9e6409e930 2023-09-30T22:35:14.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.689+0000 7f9e6a149700 1 -- 172.21.15.16:0/730942297 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9e640a1990 con 0x7f9e6409e930 2023-09-30T22:35:14.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.690+0000 7f9e5bfff700 1 -- 172.21.15.16:0/730942297 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9e4c0233d0 con 0x7f9e6409e930 2023-09-30T22:35:14.691 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:35:14.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.692+0000 7f9e6a149700 1 -- 172.21.15.16:0/730942297 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e6409e930 msgr2=0x7f9e6406b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:14.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.692+0000 7f9e6a149700 1 --2- 172.21.15.16:0/730942297 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e6409e930 0x7f9e6406b8b0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c000c00 tx=0x7f9e4c004750 comp rx=0 tx=0).stop 2023-09-30T22:35:14.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.692+0000 7f9e6a149700 1 -- 172.21.15.16:0/730942297 shutdown_connections 2023-09-30T22:35:14.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.692+0000 7f9e6a149700 1 --2- 172.21.15.16:0/730942297 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9e6409f650 0x7f9e6406c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:14.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.692+0000 7f9e6a149700 1 --2- 172.21.15.16:0/730942297 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9e6409e930 0x7f9e6406b8b0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:14.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.692+0000 7f9e6a149700 1 --2- 172.21.15.16:0/730942297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9e6409f240 0x7f9e6406bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:14.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.692+0000 7f9e6a149700 1 -- 172.21.15.16:0/730942297 >> 172.21.15.16:0/730942297 conn(0x7f9e640a4210 msgr2=0x7f9e6409dc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:14.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.692+0000 7f9e6a149700 1 -- 172.21.15.16:0/730942297 shutdown_connections 2023-09-30T22:35:14.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:14.692+0000 7f9e6a149700 1 -- 172.21.15.16:0/730942297 wait complete. 2023-09-30T22:35:14.708 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:35:15.116 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:35:15.200 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:35:15.235 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:35:15.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.265+0000 7ff26330c700 1 Processor -- start 2023-09-30T22:35:15.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.265+0000 7ff26330c700 1 -- start start 2023-09-30T22:35:15.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.266+0000 7ff26330c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff25c130560 0x7ff25c130930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:15.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.266+0000 7ff26330c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff25c09eca0 0x7ff25c06f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:15.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.266+0000 7ff26330c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff25c070a50 con 0x7ff25c130e70 2023-09-30T22:35:15.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.266+0000 7ff26330c700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff25c0717d0 con 0x7ff25c09eca0 2023-09-30T22:35:15.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.266+0000 7ff26330c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff25c130340 con 0x7ff25c130560 2023-09-30T22:35:15.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.266+0000 7ff2610a8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff25c130560 0x7ff25c130930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:15.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.266+0000 7ff2610a8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff25c130560 0x7ff25c130930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:55444/0 (socket says 172.21.15.16:55444) 2023-09-30T22:35:15.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.266+0000 7ff2610a8700 1 -- 172.21.15.16:0/3386752714 learned_addr learned my addr 172.21.15.16:0/3386752714 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:15.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.267+0000 7ff2608a7700 1 --2- 172.21.15.16:0/3386752714 >> v2:172.21.15.16:3300/0 conn(0x7ff25c09eca0 0x7ff25c06f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:15.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.267+0000 7ff2610a8700 1 -- 172.21.15.16:0/3386752714 >> v2:172.21.15.16:3300/0 conn(0x7ff25c09eca0 msgr2=0x7ff25c06f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:15.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.267+0000 7ff2610a8700 1 --2- 172.21.15.16:0/3386752714 >> v2:172.21.15.16:3300/0 conn(0x7ff25c09eca0 0x7ff25c06f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:15.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.267+0000 7ff2610a8700 1 -- 172.21.15.16:0/3386752714 >> v1:172.21.15.16:6789/0 conn(0x7ff25c130e70 legacy=0x7ff25c06fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:15.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.267+0000 7ff2610a8700 1 -- 172.21.15.16:0/3386752714 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff25c071060 con 0x7ff25c130560 2023-09-30T22:35:15.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.267+0000 7ff2610a8700 1 --2- 172.21.15.16:0/3386752714 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff25c130560 0x7ff25c130930 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7ff25800bb90 tx=0x7ff25800bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8074bdffdefaf9a9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:15.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.267+0000 7ff253fff700 1 -- 172.21.15.16:0/3386752714 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25800dc70 con 0x7ff25c130560 2023-09-30T22:35:15.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.267+0000 7ff253fff700 1 -- 172.21.15.16:0/3386752714 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff25800ddd0 con 0x7ff25c130560 2023-09-30T22:35:15.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.267+0000 7ff253fff700 1 -- 172.21.15.16:0/3386752714 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2580106a0 con 0x7ff25c130560 2023-09-30T22:35:15.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.268+0000 7ff26330c700 1 -- 172.21.15.16:0/3386752714 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff25c130560 msgr2=0x7ff25c130930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:15.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.268+0000 7ff26330c700 1 --2- 172.21.15.16:0/3386752714 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff25c130560 0x7ff25c130930 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7ff25800bb90 tx=0x7ff25800bea0 comp rx=0 tx=0).stop 2023-09-30T22:35:15.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.268+0000 7ff26330c700 1 -- 172.21.15.16:0/3386752714 shutdown_connections 2023-09-30T22:35:15.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.268+0000 7ff26330c700 1 --2- 172.21.15.16:0/3386752714 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff25c130560 0x7ff25c130930 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:15.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.268+0000 7ff26330c700 1 --2- 172.21.15.16:0/3386752714 >> v2:172.21.15.16:3300/0 conn(0x7ff25c09eca0 0x7ff25c06f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:15.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.268+0000 7ff26330c700 1 -- 172.21.15.16:0/3386752714 >> 172.21.15.16:0/3386752714 conn(0x7ff25c0a2aa0 msgr2=0x7ff25c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:15.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.268+0000 7ff26330c700 1 -- 172.21.15.16:0/3386752714 shutdown_connections 2023-09-30T22:35:15.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.268+0000 7ff26330c700 1 -- 172.21.15.16:0/3386752714 wait complete. 2023-09-30T22:35:15.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.269+0000 7ff26330c700 1 Processor -- start 2023-09-30T22:35:15.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.269+0000 7ff26330c700 1 -- start start 2023-09-30T22:35:15.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.270+0000 7ff26330c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff25c09eca0 0x7ff25c09bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:15.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.270+0000 7ff26330c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff25c130560 0x7ff25c09c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:15.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.270+0000 7ff26330c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff25c130e70 0x7ff25c09c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:15.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.270+0000 7ff26330c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff25c11a100 con 0x7ff25c130e70 2023-09-30T22:35:15.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.270+0000 7ff26330c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff25c119130 con 0x7ff25c09eca0 2023-09-30T22:35:15.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.270+0000 7ff26330c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff25c119560 con 0x7ff25c130560 2023-09-30T22:35:15.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.270+0000 7ff2608a7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff25c130560 0x7ff25c09c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:15.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.270+0000 7ff2608a7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff25c130560 0x7ff25c09c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:55454/0 (socket says 172.21.15.16:55454) 2023-09-30T22:35:15.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.270+0000 7ff2608a7700 1 -- 172.21.15.16:0/2412142302 learned_addr learned my addr 172.21.15.16:0/2412142302 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:15.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.270+0000 7ff2618a9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff25c130e70 0x7ff25c09c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:15.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.270+0000 7ff2608a7700 1 -- 172.21.15.16:0/2412142302 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff25c09eca0 msgr2=0x7ff25c09bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:15.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.270+0000 7ff2608a7700 1 --2- 172.21.15.16:0/2412142302 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff25c09eca0 0x7ff25c09bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:15.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.270+0000 7ff2608a7700 1 -- 172.21.15.16:0/2412142302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff25c130e70 msgr2=0x7ff25c09c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:15.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.270+0000 7ff2608a7700 1 --2- 172.21.15.16:0/2412142302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff25c130e70 0x7ff25c09c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:15.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.270+0000 7ff2608a7700 1 -- 172.21.15.16:0/2412142302 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff25800b7f0 con 0x7ff25c130560 2023-09-30T22:35:15.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.270+0000 7ff2618a9700 1 --2- 172.21.15.16:0/2412142302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff25c130e70 0x7ff25c09c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:15.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.271+0000 7ff2608a7700 1 --2- 172.21.15.16:0/2412142302 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff25c130560 0x7ff25c09c450 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7ff24c009d10 tx=0x7ff24c0133b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:15.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.271+0000 7ff251ffb700 1 -- 172.21.15.16:0/2412142302 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff24c013bc0 con 0x7ff25c130560 2023-09-30T22:35:15.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.271+0000 7ff251ffb700 1 -- 172.21.15.16:0/2412142302 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff24c015040 con 0x7ff25c130560 2023-09-30T22:35:15.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.271+0000 7ff26330c700 1 -- 172.21.15.16:0/2412142302 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff25c12eaf0 con 0x7ff25c130560 2023-09-30T22:35:15.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.271+0000 7ff251ffb700 1 -- 172.21.15.16:0/2412142302 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff24c01b400 con 0x7ff25c130560 2023-09-30T22:35:15.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.271+0000 7ff26330c700 1 -- 172.21.15.16:0/2412142302 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff25c119b60 con 0x7ff25c130560 2023-09-30T22:35:15.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.272+0000 7ff251ffb700 1 -- 172.21.15.16:0/2412142302 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff24c01b560 con 0x7ff25c130560 2023-09-30T22:35:15.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.272+0000 7ff251ffb700 1 -- 172.21.15.16:0/2412142302 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7ff24c043500 con 0x7ff25c130560 2023-09-30T22:35:15.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.272+0000 7ff26330c700 1 -- 172.21.15.16:0/2412142302 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff25c074ea0 con 0x7ff25c130560 2023-09-30T22:35:15.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.278+0000 7ff251ffb700 1 -- 172.21.15.16:0/2412142302 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff24c0243b0 con 0x7ff25c130560 2023-09-30T22:35:15.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.344+0000 7f73ee646700 1 Processor -- start 2023-09-30T22:35:15.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.344+0000 7f73ee646700 1 -- start start 2023-09-30T22:35:15.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.344+0000 7f73ee646700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f73e8131810 0x7f73e8131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:15.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.344+0000 7f73ee646700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f73e809a580 0x7f73e8139660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:15.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.344+0000 7f73ee646700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73e81313c0 con 0x7f73e8099a40 2023-09-30T22:35:15.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.344+0000 7f73ee646700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73e806e8c0 con 0x7f73e8131810 2023-09-30T22:35:15.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.344+0000 7f73ee646700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f73e8071d20 con 0x7f73e809a580 2023-09-30T22:35:15.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.344+0000 7f73e7fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f73e8131810 0x7f73e8131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:15.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.344+0000 7f73e7fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f73e8131810 0x7f73e8131be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49428/0 (socket says 172.21.15.16:49428) 2023-09-30T22:35:15.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.344+0000 7f73e7fff700 1 -- 172.21.15.16:0/943035134 learned_addr learned my addr 172.21.15.16:0/943035134 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:15.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.344+0000 7f73ecbe3700 1 --2- 172.21.15.16:0/943035134 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f73e809a580 0x7f73e8139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:15.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.345+0000 7f73e7fff700 1 -- 172.21.15.16:0/943035134 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f73e809a580 msgr2=0x7f73e8139660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:15.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.345+0000 7f73e7fff700 1 --2- 172.21.15.16:0/943035134 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f73e809a580 0x7f73e8139660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:15.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.345+0000 7f73e7fff700 1 -- 172.21.15.16:0/943035134 >> v1:172.21.15.16:6789/0 conn(0x7f73e8099a40 legacy=0x7f73e8099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:15.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.345+0000 7f73e7fff700 1 -- 172.21.15.16:0/943035134 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e80721a0 con 0x7f73e8131810 2023-09-30T22:35:15.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.345+0000 7f73e7fff700 1 --2- 172.21.15.16:0/943035134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73e8131810 0x7f73e8131be0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f73d80098f0 tx=0x7f73d8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4141994f6e6b075e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:15.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.345+0000 7f73e6ffd700 1 -- 172.21.15.16:0/943035134 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d8011830 con 0x7f73e8131810 2023-09-30T22:35:15.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.345+0000 7f73e6ffd700 1 -- 172.21.15.16:0/943035134 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73d8011990 con 0x7f73e8131810 2023-09-30T22:35:15.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.345+0000 7f73e6ffd700 1 -- 172.21.15.16:0/943035134 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d8011d20 con 0x7f73e8131810 2023-09-30T22:35:15.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.346+0000 7f73ee646700 1 -- 172.21.15.16:0/943035134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73e8131810 msgr2=0x7f73e8131be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:15.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.346+0000 7f73ee646700 1 --2- 172.21.15.16:0/943035134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73e8131810 0x7f73e8131be0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f73d80098f0 tx=0x7f73d8009c00 comp rx=0 tx=0).stop 2023-09-30T22:35:15.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.346+0000 7f73ee646700 1 -- 172.21.15.16:0/943035134 shutdown_connections 2023-09-30T22:35:15.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.346+0000 7f73ee646700 1 --2- 172.21.15.16:0/943035134 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f73e809a580 0x7f73e8139660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:15.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.346+0000 7f73ee646700 1 --2- 172.21.15.16:0/943035134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73e8131810 0x7f73e8131be0 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:15.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.346+0000 7f73ee646700 1 -- 172.21.15.16:0/943035134 >> 172.21.15.16:0/943035134 conn(0x7f73e80a2aa0 msgr2=0x7f73e80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:15.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.346+0000 7f73ee646700 1 -- 172.21.15.16:0/943035134 shutdown_connections 2023-09-30T22:35:15.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.346+0000 7f73ee646700 1 -- 172.21.15.16:0/943035134 wait complete. 2023-09-30T22:35:15.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.347+0000 7f73ee646700 1 Processor -- start 2023-09-30T22:35:15.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.348+0000 7f73ee646700 1 -- start start 2023-09-30T22:35:15.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.348+0000 7f73ee646700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f73e8099a40 0x7f73e811b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:15.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.348+0000 7f73ee646700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73e809a580 0x7f73e811bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:15.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.348+0000 7f73ee646700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f73e8131810 0x7f73e811c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:15.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.348+0000 7f73ee646700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e811f140 con 0x7f73e809a580 2023-09-30T22:35:15.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.348+0000 7f73ee646700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73e811d2d0 con 0x7f73e8131810 2023-09-30T22:35:15.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.348+0000 7f73ee646700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f73e811e4f0 con 0x7f73e8099a40 2023-09-30T22:35:15.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.348+0000 7f73e77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73e809a580 0x7f73e811bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:15.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.349+0000 7f73e77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73e809a580 0x7f73e811bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49442/0 (socket says 172.21.15.16:49442) 2023-09-30T22:35:15.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.349+0000 7f73e77fe700 1 -- 172.21.15.16:0/1575553778 learned_addr learned my addr 172.21.15.16:0/1575553778 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:15.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.349+0000 7f73e7fff700 1 --2- 172.21.15.16:0/1575553778 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f73e8099a40 0x7f73e811b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:15.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.349+0000 7f73ecbe3700 1 --2- 172.21.15.16:0/1575553778 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f73e8131810 0x7f73e811c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:15.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.349+0000 7f73e77fe700 1 -- 172.21.15.16:0/1575553778 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f73e8099a40 msgr2=0x7f73e811b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:15.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.349+0000 7f73e77fe700 1 --2- 172.21.15.16:0/1575553778 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f73e8099a40 0x7f73e811b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:15.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.349+0000 7f73e77fe700 1 -- 172.21.15.16:0/1575553778 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f73e8131810 msgr2=0x7f73e811c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:15.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.349+0000 7f73e77fe700 1 --2- 172.21.15.16:0/1575553778 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f73e8131810 0x7f73e811c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:15.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.349+0000 7f73e77fe700 1 -- 172.21.15.16:0/1575553778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73d8009550 con 0x7f73e809a580 2023-09-30T22:35:15.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.349+0000 7f73e77fe700 1 --2- 172.21.15.16:0/1575553778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73e809a580 0x7f73e811bb60 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f73dc00c4b0 tx=0x7f73dc00c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:15.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.349+0000 7f73e4ff9700 1 -- 172.21.15.16:0/1575553778 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73dc0052d0 con 0x7f73e809a580 2023-09-30T22:35:15.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.349+0000 7f73e4ff9700 1 -- 172.21.15.16:0/1575553778 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73dc005430 con 0x7f73e809a580 2023-09-30T22:35:15.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.349+0000 7f73e4ff9700 1 -- 172.21.15.16:0/1575553778 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73dc0038c0 con 0x7f73e809a580 2023-09-30T22:35:15.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.349+0000 7f73ee646700 1 -- 172.21.15.16:0/1575553778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73e80a1b60 con 0x7f73e809a580 2023-09-30T22:35:15.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.349+0000 7f73ee646700 1 -- 172.21.15.16:0/1575553778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73e811e030 con 0x7f73e809a580 2023-09-30T22:35:15.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.350+0000 7f73e4ff9700 1 -- 172.21.15.16:0/1575553778 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f73dc003a20 con 0x7f73e809a580 2023-09-30T22:35:15.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.350+0000 7f73e4ff9700 1 -- 172.21.15.16:0/1575553778 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f73dc03ea00 con 0x7f73e809a580 2023-09-30T22:35:15.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.351+0000 7f73ee646700 1 -- 172.21.15.16:0/1575553778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73e8074ea0 con 0x7f73e809a580 2023-09-30T22:35:15.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.356+0000 7f73e4ff9700 1 -- 172.21.15.16:0/1575553778 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f73dc018260 con 0x7f73e809a580 2023-09-30T22:35:15.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.385+0000 7f7d63936700 1 Processor -- start 2023-09-30T22:35:15.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.386+0000 7f7d63936700 1 -- start start 2023-09-30T22:35:15.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.386+0000 7f7d63936700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7d5c133a80 0x7f7d5c133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:15.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.386+0000 7f7d63936700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7d5c12ff20 0x7f7d5c130330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:15.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.386+0000 7f7d63936700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7d5c130870 0x7f7d5c13a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:15.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.386+0000 7f7d63936700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d5c06c570 con 0x7f7d5c130870 2023-09-30T22:35:15.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.386+0000 7f7d63936700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5c06bc00 con 0x7f7d5c12ff20 2023-09-30T22:35:15.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.386+0000 7f7d63936700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5c0a0040 con 0x7f7d5c133a80 2023-09-30T22:35:15.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.386+0000 7f7d60ed1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7d5c12ff20 0x7f7d5c130330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:15.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.386+0000 7f7d60ed1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7d5c12ff20 0x7f7d5c130330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37942/0 (socket says 172.21.15.16:37942) 2023-09-30T22:35:15.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.386+0000 7f7d60ed1700 1 -- 172.21.15.16:0/3553011052 learned_addr learned my addr 172.21.15.16:0/3553011052 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:15.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.386+0000 7f7d616d2700 1 --2- 172.21.15.16:0/3553011052 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7d5c133a80 0x7f7d5c133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:15.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.387+0000 7f7d61ed3700 1 --2- 172.21.15.16:0/3553011052 >> v2:172.21.15.16:3300/0 conn(0x7f7d5c130870 0x7f7d5c13a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:15.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.387+0000 7f7d60ed1700 1 -- 172.21.15.16:0/3553011052 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7d5c133a80 msgr2=0x7f7d5c133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:15.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.387+0000 7f7d60ed1700 1 --2- 172.21.15.16:0/3553011052 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7d5c133a80 0x7f7d5c133e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:15.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.387+0000 7f7d60ed1700 1 -- 172.21.15.16:0/3553011052 >> v2:172.21.15.16:3300/0 conn(0x7f7d5c130870 msgr2=0x7f7d5c13a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:15.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.387+0000 7f7d60ed1700 1 --2- 172.21.15.16:0/3553011052 >> v2:172.21.15.16:3300/0 conn(0x7f7d5c130870 0x7f7d5c13a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:15.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.387+0000 7f7d60ed1700 1 -- 172.21.15.16:0/3553011052 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d5c133860 con 0x7f7d5c12ff20 2023-09-30T22:35:15.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.387+0000 7f7d616d2700 1 --2- 172.21.15.16:0/3553011052 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7d5c133a80 0x7f7d5c133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:15.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.387+0000 7f7d60ed1700 1 --2- 172.21.15.16:0/3553011052 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7d5c12ff20 0x7f7d5c130330 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c0098f0 tx=0x7f7d4c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=532a1d77974206f2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:15.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.387+0000 7f7d53fff700 1 -- 172.21.15.16:0/3553011052 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c011830 con 0x7f7d5c12ff20 2023-09-30T22:35:15.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.387+0000 7f7d53fff700 1 -- 172.21.15.16:0/3553011052 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d4c011990 con 0x7f7d5c12ff20 2023-09-30T22:35:15.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.387+0000 7f7d53fff700 1 -- 172.21.15.16:0/3553011052 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c011d20 con 0x7f7d5c12ff20 2023-09-30T22:35:15.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.388+0000 7f7d63936700 1 -- 172.21.15.16:0/3553011052 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7d5c12ff20 msgr2=0x7f7d5c130330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:15.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.388+0000 7f7d63936700 1 --2- 172.21.15.16:0/3553011052 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7d5c12ff20 0x7f7d5c130330 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c0098f0 tx=0x7f7d4c009c00 comp rx=0 tx=0).stop 2023-09-30T22:35:15.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.388+0000 7f7d63936700 1 -- 172.21.15.16:0/3553011052 shutdown_connections 2023-09-30T22:35:15.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.388+0000 7f7d63936700 1 --2- 172.21.15.16:0/3553011052 >> v2:172.21.15.16:3300/0 conn(0x7f7d5c130870 0x7f7d5c13a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:15.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.388+0000 7f7d63936700 1 --2- 172.21.15.16:0/3553011052 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7d5c133a80 0x7f7d5c133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:15.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.388+0000 7f7d63936700 1 --2- 172.21.15.16:0/3553011052 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7d5c12ff20 0x7f7d5c130330 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:15.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.388+0000 7f7d63936700 1 -- 172.21.15.16:0/3553011052 >> 172.21.15.16:0/3553011052 conn(0x7f7d5c0a4210 msgr2=0x7f7d5c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:15.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.388+0000 7f7d63936700 1 -- 172.21.15.16:0/3553011052 shutdown_connections 2023-09-30T22:35:15.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.388+0000 7f7d63936700 1 -- 172.21.15.16:0/3553011052 wait complete. 2023-09-30T22:35:15.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.389+0000 7f7d63936700 1 Processor -- start 2023-09-30T22:35:15.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.389+0000 7f7d63936700 1 -- start start 2023-09-30T22:35:15.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.390+0000 7f7d63936700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7d5c12ff20 0x7f7d5c09c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:15.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.390+0000 7f7d63936700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d5c130870 0x7f7d5c09c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:15.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.390+0000 7f7d63936700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7d5c133a80 0x7f7d5c09cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:15.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.390+0000 7f7d63936700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5c09dd40 con 0x7f7d5c130870 2023-09-30T22:35:15.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.390+0000 7f7d63936700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5c11fc80 con 0x7f7d5c133a80 2023-09-30T22:35:15.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.390+0000 7f7d63936700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5c12d220 con 0x7f7d5c12ff20 2023-09-30T22:35:15.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.390+0000 7f7d616d2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7d5c12ff20 0x7f7d5c09c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:15.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.390+0000 7f7d616d2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7d5c12ff20 0x7f7d5c09c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:55492/0 (socket says 172.21.15.16:55492) 2023-09-30T22:35:15.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.390+0000 7f7d616d2700 1 -- 172.21.15.16:0/4165204442 learned_addr learned my addr 172.21.15.16:0/4165204442 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:15.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.390+0000 7f7d60ed1700 1 --2- 172.21.15.16:0/4165204442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d5c130870 0x7f7d5c09c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:15.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.390+0000 7f7d61ed3700 1 --2- 172.21.15.16:0/4165204442 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7d5c133a80 0x7f7d5c09cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:15.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.390+0000 7f7d616d2700 1 -- 172.21.15.16:0/4165204442 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7d5c133a80 msgr2=0x7f7d5c09cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:15.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.390+0000 7f7d616d2700 1 --2- 172.21.15.16:0/4165204442 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7d5c133a80 0x7f7d5c09cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:15.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.390+0000 7f7d616d2700 1 -- 172.21.15.16:0/4165204442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d5c130870 msgr2=0x7f7d5c09c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:15.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.390+0000 7f7d616d2700 1 --2- 172.21.15.16:0/4165204442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d5c130870 0x7f7d5c09c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:15.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.390+0000 7f7d616d2700 1 -- 172.21.15.16:0/4165204442 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d580094a0 con 0x7f7d5c12ff20 2023-09-30T22:35:15.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.390+0000 7f7d616d2700 1 --2- 172.21.15.16:0/4165204442 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7d5c12ff20 0x7f7d5c09c480 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f7d5800ebe0 tx=0x7f7d5800eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:15.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.390+0000 7f7d51ffb700 1 -- 172.21.15.16:0/4165204442 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5800c850 con 0x7f7d5c12ff20 2023-09-30T22:35:15.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.391+0000 7f7d51ffb700 1 -- 172.21.15.16:0/4165204442 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d5800c9b0 con 0x7f7d5c12ff20 2023-09-30T22:35:15.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.391+0000 7f7d63936700 1 -- 172.21.15.16:0/4165204442 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d4c009550 con 0x7f7d5c12ff20 2023-09-30T22:35:15.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.391+0000 7f7d51ffb700 1 -- 172.21.15.16:0/4165204442 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5800cd40 con 0x7f7d5c12ff20 2023-09-30T22:35:15.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.391+0000 7f7d63936700 1 -- 172.21.15.16:0/4165204442 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d5c0a2230 con 0x7f7d5c12ff20 2023-09-30T22:35:15.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.392+0000 7f7d51ffb700 1 -- 172.21.15.16:0/4165204442 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7d58013470 con 0x7f7d5c12ff20 2023-09-30T22:35:15.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.392+0000 7f7d63936700 1 -- 172.21.15.16:0/4165204442 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d5c120160 con 0x7f7d5c12ff20 2023-09-30T22:35:15.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.392+0000 7f7d51ffb700 1 -- 172.21.15.16:0/4165204442 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f7d5803f7e0 con 0x7f7d5c12ff20 2023-09-30T22:35:15.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.397+0000 7f7d51ffb700 1 -- 172.21.15.16:0/4165204442 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7d5c120160 con 0x7f7d5c12ff20 2023-09-30T22:35:15.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.490+0000 7ff26330c700 1 -- 172.21.15.16:0/2412142302 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff25c06b9e0 con 0x7ff25c130560 2023-09-30T22:35:15.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.491+0000 7ff251ffb700 1 -- 172.21.15.16:0/2412142302 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff24c020020 con 0x7ff25c130560 2023-09-30T22:35:15.492 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:35:15.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.493+0000 7ff26330c700 1 -- 172.21.15.16:0/2412142302 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff25c130560 msgr2=0x7ff25c09c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:15.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.493+0000 7ff26330c700 1 --2- 172.21.15.16:0/2412142302 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff25c130560 0x7ff25c09c450 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7ff24c009d10 tx=0x7ff24c0133b0 comp rx=0 tx=0).stop 2023-09-30T22:35:15.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.493+0000 7ff26330c700 1 -- 172.21.15.16:0/2412142302 shutdown_connections 2023-09-30T22:35:15.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.493+0000 7ff26330c700 1 --2- 172.21.15.16:0/2412142302 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff25c09eca0 0x7ff25c09bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:15.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.493+0000 7ff26330c700 1 --2- 172.21.15.16:0/2412142302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff25c130e70 0x7ff25c09c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:15.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.493+0000 7ff26330c700 1 --2- 172.21.15.16:0/2412142302 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff25c130560 0x7ff25c09c450 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:15.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.493+0000 7ff26330c700 1 -- 172.21.15.16:0/2412142302 >> 172.21.15.16:0/2412142302 conn(0x7ff25c0a2aa0 msgr2=0x7ff25c12b530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:15.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.493+0000 7ff26330c700 1 -- 172.21.15.16:0/2412142302 shutdown_connections 2023-09-30T22:35:15.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.493+0000 7ff26330c700 1 -- 172.21.15.16:0/2412142302 wait complete. 2023-09-30T22:35:15.508 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:35:15.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.564+0000 7f73ee646700 1 -- 172.21.15.16:0/1575553778 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f73e806b9e0 con 0x7f73e809a580 2023-09-30T22:35:15.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.564+0000 7f73e4ff9700 1 -- 172.21.15.16:0/1575553778 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f73dc017020 con 0x7f73e809a580 2023-09-30T22:35:15.566 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:35:15.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.566+0000 7f73ee646700 1 -- 172.21.15.16:0/1575553778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73e809a580 msgr2=0x7f73e811bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:15.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.566+0000 7f73ee646700 1 --2- 172.21.15.16:0/1575553778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73e809a580 0x7f73e811bb60 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f73dc00c4b0 tx=0x7f73dc00c7c0 comp rx=0 tx=0).stop 2023-09-30T22:35:15.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.566+0000 7f73ee646700 1 -- 172.21.15.16:0/1575553778 shutdown_connections 2023-09-30T22:35:15.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.566+0000 7f73ee646700 1 --2- 172.21.15.16:0/1575553778 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f73e8131810 0x7f73e811c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:15.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.566+0000 7f73ee646700 1 --2- 172.21.15.16:0/1575553778 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f73e8099a40 0x7f73e811b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:15.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.566+0000 7f73ee646700 1 --2- 172.21.15.16:0/1575553778 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f73e809a580 0x7f73e811bb60 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:15.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.566+0000 7f73ee646700 1 -- 172.21.15.16:0/1575553778 >> 172.21.15.16:0/1575553778 conn(0x7f73e80a2aa0 msgr2=0x7f73e81b8c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:15.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.567+0000 7f73ee646700 1 -- 172.21.15.16:0/1575553778 shutdown_connections 2023-09-30T22:35:15.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.567+0000 7f73ee646700 1 -- 172.21.15.16:0/1575553778 wait complete. 2023-09-30T22:35:15.581 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:35:15.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.603+0000 7f7d63936700 1 -- 172.21.15.16:0/4165204442 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f7d5c074ea0 con 0x7f7d5c12ff20 2023-09-30T22:35:15.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.603+0000 7f7d51ffb700 1 -- 172.21.15.16:0/4165204442 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7d58021070 con 0x7f7d5c12ff20 2023-09-30T22:35:15.604 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:35:15.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.605+0000 7f7d63936700 1 -- 172.21.15.16:0/4165204442 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7d5c12ff20 msgr2=0x7f7d5c09c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:15.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.605+0000 7f7d63936700 1 --2- 172.21.15.16:0/4165204442 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7d5c12ff20 0x7f7d5c09c480 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f7d5800ebe0 tx=0x7f7d5800eef0 comp rx=0 tx=0).stop 2023-09-30T22:35:15.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.605+0000 7f7d63936700 1 -- 172.21.15.16:0/4165204442 shutdown_connections 2023-09-30T22:35:15.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.605+0000 7f7d63936700 1 --2- 172.21.15.16:0/4165204442 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7d5c133a80 0x7f7d5c09cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:15.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.605+0000 7f7d63936700 1 --2- 172.21.15.16:0/4165204442 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7d5c12ff20 0x7f7d5c09c480 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:15.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.605+0000 7f7d63936700 1 --2- 172.21.15.16:0/4165204442 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d5c130870 0x7f7d5c09c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:15.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.605+0000 7f7d63936700 1 -- 172.21.15.16:0/4165204442 >> 172.21.15.16:0/4165204442 conn(0x7f7d5c0a4210 msgr2=0x7f7d5c09e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:15.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.605+0000 7f7d63936700 1 -- 172.21.15.16:0/4165204442 shutdown_connections 2023-09-30T22:35:15.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.605+0000 7f7d63936700 1 -- 172.21.15.16:0/4165204442 wait complete. 2023-09-30T22:35:15.619 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:35:15.709 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:35:15.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.853+0000 7feea31de700 1 Processor -- start 2023-09-30T22:35:15.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.853+0000 7feea31de700 1 -- start start 2023-09-30T22:35:15.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.854+0000 7feea31de700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fee9c133a80 0x7fee9c133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:15.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.854+0000 7feea31de700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fee9c12ff20 0x7fee9c130330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:15.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.854+0000 7feea31de700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fee9c130870 0x7fee9c13a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:15.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.854+0000 7feea31de700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee9c06c570 con 0x7fee9c12ff20 2023-09-30T22:35:15.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.854+0000 7feea31de700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee9c06bc00 con 0x7fee9c133a80 2023-09-30T22:35:15.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.854+0000 7feea31de700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fee9c0a0040 con 0x7fee9c130870 2023-09-30T22:35:15.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.854+0000 7fee9bfff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fee9c12ff20 0x7fee9c130330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:15.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.854+0000 7feea0f7a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fee9c133a80 0x7fee9c133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:15.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.854+0000 7fee9bfff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fee9c12ff20 0x7fee9c130330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49470/0 (socket says 172.21.15.16:49470) 2023-09-30T22:35:15.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.854+0000 7feea177b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fee9c130870 0x7fee9c13a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:15.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.854+0000 7fee9bfff700 1 -- 172.21.15.16:0/39388072 learned_addr learned my addr 172.21.15.16:0/39388072 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:15.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.854+0000 7fee9bfff700 1 -- 172.21.15.16:0/39388072 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fee9c130870 msgr2=0x7fee9c13a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:15.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.854+0000 7fee9bfff700 1 --2- 172.21.15.16:0/39388072 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fee9c130870 0x7fee9c13a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:15.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.854+0000 7fee9bfff700 1 -- 172.21.15.16:0/39388072 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fee9c133a80 msgr2=0x7fee9c133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:15.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.854+0000 7fee9bfff700 1 --2- 172.21.15.16:0/39388072 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fee9c133a80 0x7fee9c133e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:15.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.854+0000 7fee9bfff700 1 -- 172.21.15.16:0/39388072 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee9c133860 con 0x7fee9c12ff20 2023-09-30T22:35:15.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.854+0000 7feea0f7a700 1 --2- 172.21.15.16:0/39388072 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fee9c133a80 0x7fee9c133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:15.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.854+0000 7feea177b700 1 --2- 172.21.15.16:0/39388072 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fee9c130870 0x7fee9c13a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:15.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.854+0000 7fee9bfff700 1 --2- 172.21.15.16:0/39388072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee9c12ff20 0x7fee9c130330 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fee900098f0 tx=0x7fee90009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b85b178be774a65 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:15.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.855+0000 7fee9b7fe700 1 -- 172.21.15.16:0/39388072 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee90011830 con 0x7fee9c12ff20 2023-09-30T22:35:15.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.855+0000 7fee9b7fe700 1 -- 172.21.15.16:0/39388072 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee90011990 con 0x7fee9c12ff20 2023-09-30T22:35:15.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.855+0000 7fee9b7fe700 1 -- 172.21.15.16:0/39388072 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee90011d20 con 0x7fee9c12ff20 2023-09-30T22:35:15.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.855+0000 7feea31de700 1 -- 172.21.15.16:0/39388072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee9c12ff20 msgr2=0x7fee9c130330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:15.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.855+0000 7feea31de700 1 --2- 172.21.15.16:0/39388072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee9c12ff20 0x7fee9c130330 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fee900098f0 tx=0x7fee90009c00 comp rx=0 tx=0).stop 2023-09-30T22:35:15.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.855+0000 7feea31de700 1 -- 172.21.15.16:0/39388072 shutdown_connections 2023-09-30T22:35:15.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.855+0000 7feea31de700 1 --2- 172.21.15.16:0/39388072 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fee9c133a80 0x7fee9c133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:15.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.855+0000 7feea31de700 1 --2- 172.21.15.16:0/39388072 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fee9c130870 0x7fee9c13a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:15.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.855+0000 7feea31de700 1 --2- 172.21.15.16:0/39388072 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee9c12ff20 0x7fee9c130330 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:15.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.855+0000 7feea31de700 1 -- 172.21.15.16:0/39388072 >> 172.21.15.16:0/39388072 conn(0x7fee9c0a4210 msgr2=0x7fee9c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:15.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.855+0000 7feea31de700 1 -- 172.21.15.16:0/39388072 shutdown_connections 2023-09-30T22:35:15.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.855+0000 7feea31de700 1 -- 172.21.15.16:0/39388072 wait complete. 2023-09-30T22:35:15.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.856+0000 7feea31de700 1 Processor -- start 2023-09-30T22:35:15.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.856+0000 7feea31de700 1 -- start start 2023-09-30T22:35:15.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.857+0000 7feea31de700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee9c12ff20 0x7fee9c09c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:15.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.857+0000 7feea31de700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fee9c130870 0x7fee9c09c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:15.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.857+0000 7feea31de700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fee9c133a80 0x7fee9c09cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:15.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.857+0000 7feea31de700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9c09de20 con 0x7fee9c12ff20 2023-09-30T22:35:15.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.857+0000 7feea31de700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee9c12da80 con 0x7fee9c133a80 2023-09-30T22:35:15.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.857+0000 7feea31de700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fee9c11ff60 con 0x7fee9c130870 2023-09-30T22:35:15.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.857+0000 7feea0f7a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee9c12ff20 0x7fee9c09c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:15.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.857+0000 7feea0f7a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee9c12ff20 0x7fee9c09c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49474/0 (socket says 172.21.15.16:49474) 2023-09-30T22:35:15.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.857+0000 7feea0f7a700 1 -- 172.21.15.16:0/72484603 learned_addr learned my addr 172.21.15.16:0/72484603 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:15.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.857+0000 7feea177b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fee9c133a80 0x7fee9c09cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:15.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.857+0000 7fee9bfff700 1 --2- 172.21.15.16:0/72484603 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fee9c130870 0x7fee9c09c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:15.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.857+0000 7feea0f7a700 1 -- 172.21.15.16:0/72484603 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fee9c130870 msgr2=0x7fee9c09c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:15.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.857+0000 7feea0f7a700 1 --2- 172.21.15.16:0/72484603 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fee9c130870 0x7fee9c09c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:15.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.857+0000 7feea0f7a700 1 -- 172.21.15.16:0/72484603 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fee9c133a80 msgr2=0x7fee9c09cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:15.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.857+0000 7feea0f7a700 1 --2- 172.21.15.16:0/72484603 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fee9c133a80 0x7fee9c09cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:15.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.857+0000 7feea0f7a700 1 -- 172.21.15.16:0/72484603 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee94009a70 con 0x7fee9c12ff20 2023-09-30T22:35:15.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.857+0000 7feea177b700 1 --2- 172.21.15.16:0/72484603 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fee9c133a80 0x7fee9c09cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:15.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.858+0000 7feea0f7a700 1 --2- 172.21.15.16:0/72484603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee9c12ff20 0x7fee9c09c480 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fee8c00ec10 tx=0x7fee8c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:15.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.858+0000 7fee997fa700 1 -- 172.21.15.16:0/72484603 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8c00c850 con 0x7fee9c12ff20 2023-09-30T22:35:15.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.858+0000 7feea31de700 1 -- 172.21.15.16:0/72484603 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee8c0094a0 con 0x7fee9c12ff20 2023-09-30T22:35:15.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.858+0000 7fee997fa700 1 -- 172.21.15.16:0/72484603 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee8c00c9b0 con 0x7fee9c12ff20 2023-09-30T22:35:15.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.858+0000 7fee997fa700 1 -- 172.21.15.16:0/72484603 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee8c00cd20 con 0x7fee9c12ff20 2023-09-30T22:35:15.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.858+0000 7feea31de700 1 -- 172.21.15.16:0/72484603 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee90009550 con 0x7fee9c12ff20 2023-09-30T22:35:15.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.859+0000 7fee997fa700 1 -- 172.21.15.16:0/72484603 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fee8c013470 con 0x7fee9c12ff20 2023-09-30T22:35:15.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.859+0000 7feea31de700 1 -- 172.21.15.16:0/72484603 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee9c074ea0 con 0x7fee9c12ff20 2023-09-30T22:35:15.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.859+0000 7fee997fa700 1 -- 172.21.15.16:0/72484603 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fee8c03fae0 con 0x7fee9c12ff20 2023-09-30T22:35:15.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:15.864+0000 7fee997fa700 1 -- 172.21.15.16:0/72484603 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fee8c013720 con 0x7fee9c12ff20 2023-09-30T22:35:16.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.068+0000 7feea31de700 1 -- 172.21.15.16:0/72484603 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fee9c11f510 con 0x7fee9c12ff20 2023-09-30T22:35:16.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.068+0000 7fee997fa700 1 -- 172.21.15.16:0/72484603 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fee8c02a090 con 0x7fee9c12ff20 2023-09-30T22:35:16.068 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:35:16.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.070+0000 7feea31de700 1 -- 172.21.15.16:0/72484603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee9c12ff20 msgr2=0x7fee9c09c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:16.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.070+0000 7feea31de700 1 --2- 172.21.15.16:0/72484603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee9c12ff20 0x7fee9c09c480 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fee8c00ec10 tx=0x7fee8c00ef20 comp rx=0 tx=0).stop 2023-09-30T22:35:16.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.070+0000 7feea31de700 1 -- 172.21.15.16:0/72484603 shutdown_connections 2023-09-30T22:35:16.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.070+0000 7feea31de700 1 --2- 172.21.15.16:0/72484603 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fee9c133a80 0x7fee9c09cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:16.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.070+0000 7feea31de700 1 --2- 172.21.15.16:0/72484603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee9c12ff20 0x7fee9c09c480 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:16.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.070+0000 7feea31de700 1 --2- 172.21.15.16:0/72484603 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fee9c130870 0x7fee9c09c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:16.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.070+0000 7feea31de700 1 -- 172.21.15.16:0/72484603 >> 172.21.15.16:0/72484603 conn(0x7fee9c0a4210 msgr2=0x7fee9c0a0920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:16.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.070+0000 7feea31de700 1 -- 172.21.15.16:0/72484603 shutdown_connections 2023-09-30T22:35:16.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.070+0000 7feea31de700 1 -- 172.21.15.16:0/72484603 wait complete. 2023-09-30T22:35:16.085 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:35:16.509 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:35:16.583 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:35:16.620 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:35:16.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.657+0000 7f2c03407700 1 Processor -- start 2023-09-30T22:35:16.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.657+0000 7f2c03407700 1 -- start start 2023-09-30T22:35:16.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.658+0000 7f2c03407700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2bfc09fa10 0x7f2bfc136950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:16.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.658+0000 7f2c03407700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2bfc09f2c0 0x7f2bfc138ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:16.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.658+0000 7f2c03407700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bfc09b610 con 0x7f2bfc09e7e0 2023-09-30T22:35:16.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.658+0000 7f2c03407700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bfc09bbb0 con 0x7f2bfc09fa10 2023-09-30T22:35:16.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.658+0000 7f2c03407700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bfc0a0010 con 0x7f2bfc09f2c0 2023-09-30T22:35:16.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.658+0000 7f2c011a3700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2bfc09e7e0 0x7f2bfc09ebb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51836/0 (socket says 172.21.15.16:51836) 2023-09-30T22:35:16.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.658+0000 7f2c011a3700 1 -- 172.21.15.16:0/3633218434 learned_addr learned my addr 172.21.15.16:0/3633218434 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:16.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.658+0000 7f2c009a2700 1 --2- 172.21.15.16:0/3633218434 >> v2:172.21.15.16:3300/0 conn(0x7f2bfc09fa10 0x7f2bfc136950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:16.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.658+0000 7f2c019a4700 1 --2- 172.21.15.16:0/3633218434 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2bfc09f2c0 0x7f2bfc138ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:16.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.658+0000 7f2bf3fff700 1 -- 172.21.15.16:0/3633218434 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3262140899 0 0) 0x7f2bfc09b610 con 0x7f2bfc09e7e0 2023-09-30T22:35:16.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.658+0000 7f2bf3fff700 1 -- 172.21.15.16:0/3633218434 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2be80035b0 con 0x7f2bfc09e7e0 2023-09-30T22:35:16.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.659+0000 7f2c009a2700 1 -- 172.21.15.16:0/3633218434 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2bfc09f2c0 msgr2=0x7f2bfc138ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:16.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.659+0000 7f2c009a2700 1 --2- 172.21.15.16:0/3633218434 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2bfc09f2c0 0x7f2bfc138ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:16.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.659+0000 7f2c009a2700 1 -- 172.21.15.16:0/3633218434 >> v1:172.21.15.16:6789/0 conn(0x7f2bfc09e7e0 legacy=0x7f2bfc09ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:16.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.659+0000 7f2bf3fff700 1 -- 172.21.15.16:0/3633218434 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 701007145 0 0) 0x7f2be80035b0 con 0x7f2bfc09e7e0 2023-09-30T22:35:16.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.659+0000 7f2c009a2700 1 -- 172.21.15.16:0/3633218434 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bfc09b8d0 con 0x7f2bfc09fa10 2023-09-30T22:35:16.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.659+0000 7f2c009a2700 1 --2- 172.21.15.16:0/3633218434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bfc09fa10 0x7f2bfc136950 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f2bec009840 tx=0x7f2bec009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78ba002f13dddeaa server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:16.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.659+0000 7f2bf3fff700 1 -- 172.21.15.16:0/3633218434 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf8001670 con 0x7f2bfc09fa10 2023-09-30T22:35:16.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.659+0000 7f2bf3fff700 1 -- 172.21.15.16:0/3633218434 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bec004030 con 0x7f2bfc09fa10 2023-09-30T22:35:16.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.659+0000 7f2bf3fff700 1 -- 172.21.15.16:0/3633218434 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bec0043a0 con 0x7f2bfc09fa10 2023-09-30T22:35:16.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.660+0000 7f2c03407700 1 -- 172.21.15.16:0/3633218434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bfc09fa10 msgr2=0x7f2bfc136950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:16.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.660+0000 7f2c03407700 1 --2- 172.21.15.16:0/3633218434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bfc09fa10 0x7f2bfc136950 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f2bec009840 tx=0x7f2bec009b50 comp rx=0 tx=0).stop 2023-09-30T22:35:16.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.660+0000 7f2c03407700 1 -- 172.21.15.16:0/3633218434 shutdown_connections 2023-09-30T22:35:16.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.660+0000 7f2c03407700 1 --2- 172.21.15.16:0/3633218434 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2bfc09f2c0 0x7f2bfc138ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:16.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.660+0000 7f2c03407700 1 --2- 172.21.15.16:0/3633218434 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bfc09fa10 0x7f2bfc136950 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:16.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.660+0000 7f2c03407700 1 -- 172.21.15.16:0/3633218434 >> 172.21.15.16:0/3633218434 conn(0x7f2bfc0a4300 msgr2=0x7f2bfc0a4700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:16.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.660+0000 7f2c03407700 1 -- 172.21.15.16:0/3633218434 shutdown_connections 2023-09-30T22:35:16.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.661+0000 7f2c03407700 1 -- 172.21.15.16:0/3633218434 wait complete. 2023-09-30T22:35:16.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.661+0000 7f2c03407700 1 Processor -- start 2023-09-30T22:35:16.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.661+0000 7f2c03407700 1 -- start start 2023-09-30T22:35:16.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.662+0000 7f2c03407700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2bfc09e7e0 0x7f2bfc11a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:16.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.662+0000 7f2c03407700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2bfc09f2c0 0x7f2bfc11aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:16.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.662+0000 7f2c03407700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bfc09fa10 0x7f2bfc11afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:16.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.662+0000 7f2c03407700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bfc09d9c0 con 0x7f2bfc09fa10 2023-09-30T22:35:16.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.662+0000 7f2c03407700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bfc11d260 con 0x7f2bfc09e7e0 2023-09-30T22:35:16.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.662+0000 7f2c03407700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2bfc0a2a50 con 0x7f2bfc09f2c0 2023-09-30T22:35:16.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.662+0000 7f2c009a2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2bfc09f2c0 0x7f2bfc11aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:16.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.662+0000 7f2c009a2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2bfc09f2c0 0x7f2bfc11aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:55516/0 (socket says 172.21.15.16:55516) 2023-09-30T22:35:16.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.662+0000 7f2c009a2700 1 -- 172.21.15.16:0/821759295 learned_addr learned my addr 172.21.15.16:0/821759295 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:16.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.662+0000 7f2c019a4700 1 --2- 172.21.15.16:0/821759295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bfc09fa10 0x7f2bfc11afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:16.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.663+0000 7f2c009a2700 1 -- 172.21.15.16:0/821759295 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2bfc09e7e0 msgr2=0x7f2bfc11a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:16.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.663+0000 7f2c009a2700 1 --2- 172.21.15.16:0/821759295 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2bfc09e7e0 0x7f2bfc11a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:16.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.663+0000 7f2c009a2700 1 -- 172.21.15.16:0/821759295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bfc09fa10 msgr2=0x7f2bfc11afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:16.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.663+0000 7f2c009a2700 1 --2- 172.21.15.16:0/821759295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bfc09fa10 0x7f2bfc11afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:16.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.663+0000 7f2c009a2700 1 -- 172.21.15.16:0/821759295 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bec0094a0 con 0x7f2bfc09f2c0 2023-09-30T22:35:16.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.663+0000 7f2c009a2700 1 --2- 172.21.15.16:0/821759295 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2bfc09f2c0 0x7f2bfc11aa70 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f2bec000c00 tx=0x7f2bec004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:16.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.663+0000 7f2bf1ffb700 1 -- 172.21.15.16:0/821759295 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bec004590 con 0x7f2bfc09f2c0 2023-09-30T22:35:16.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.663+0000 7f2bf1ffb700 1 -- 172.21.15.16:0/821759295 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bec01f070 con 0x7f2bfc09f2c0 2023-09-30T22:35:16.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.663+0000 7f2c03407700 1 -- 172.21.15.16:0/821759295 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bfc1323f0 con 0x7f2bfc09f2c0 2023-09-30T22:35:16.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.663+0000 7f2bf1ffb700 1 -- 172.21.15.16:0/821759295 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bec011870 con 0x7f2bfc09f2c0 2023-09-30T22:35:16.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.663+0000 7f2c03407700 1 -- 172.21.15.16:0/821759295 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bfc11e260 con 0x7f2bfc09f2c0 2023-09-30T22:35:16.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.664+0000 7f2c03407700 1 -- 172.21.15.16:0/821759295 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bfc11d670 con 0x7f2bfc09f2c0 2023-09-30T22:35:16.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.665+0000 7f2bf1ffb700 1 -- 172.21.15.16:0/821759295 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2bec0119d0 con 0x7f2bfc09f2c0 2023-09-30T22:35:16.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.665+0000 7f2bf1ffb700 1 -- 172.21.15.16:0/821759295 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f2bec021070 con 0x7f2bfc09f2c0 2023-09-30T22:35:16.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.674+0000 7f2bf1ffb700 1 -- 172.21.15.16:0/821759295 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2bec06c750 con 0x7f2bfc09f2c0 2023-09-30T22:35:16.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.729+0000 7fb0f8a34700 1 Processor -- start 2023-09-30T22:35:16.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.729+0000 7fb0f8a34700 1 -- start start 2023-09-30T22:35:16.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.730+0000 7fb0f8a34700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb0f409d6f0 0x7fb0f409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:16.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.730+0000 7fb0f8a34700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb0f409e000 0x7fb0f41339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:16.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.730+0000 7fb0f8a34700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb0f409e410 0x7fb0f4133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:16.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.730+0000 7fb0f8a34700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0f406b460 con 0x7fb0f409e000 2023-09-30T22:35:16.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.730+0000 7fb0f8a34700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f409b120 con 0x7fb0f409d6f0 2023-09-30T22:35:16.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.730+0000 7fb0f8a34700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f409a7e0 con 0x7fb0f409e410 2023-09-30T22:35:16.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.730+0000 7fb0f1d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb0f409e000 0x7fb0f41339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:16.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.730+0000 7fb0f259c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb0f409d6f0 0x7fb0f409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:16.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.730+0000 7fb0f2d9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb0f409e410 0x7fb0f4133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:16.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.730+0000 7fb0f1d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb0f409e000 0x7fb0f41339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49490/0 (socket says 172.21.15.16:49490) 2023-09-30T22:35:16.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.730+0000 7fb0f259c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb0f409d6f0 0x7fb0f409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37992/0 (socket says 172.21.15.16:37992) 2023-09-30T22:35:16.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.730+0000 7fb0f1d9b700 1 -- 172.21.15.16:0/2531797924 learned_addr learned my addr 172.21.15.16:0/2531797924 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:16.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.731+0000 7fb0f259c700 1 -- 172.21.15.16:0/2531797924 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb0f409e410 msgr2=0x7fb0f4133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:16.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.731+0000 7fb0f259c700 1 --2- 172.21.15.16:0/2531797924 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb0f409e410 0x7fb0f4133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:16.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.731+0000 7fb0f259c700 1 -- 172.21.15.16:0/2531797924 >> v2:172.21.15.16:3300/0 conn(0x7fb0f409e000 msgr2=0x7fb0f41339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:16.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.731+0000 7fb0f259c700 1 --2- 172.21.15.16:0/2531797924 >> v2:172.21.15.16:3300/0 conn(0x7fb0f409e000 0x7fb0f41339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:16.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.731+0000 7fb0f259c700 1 -- 172.21.15.16:0/2531797924 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f409a500 con 0x7fb0f409d6f0 2023-09-30T22:35:16.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.731+0000 7fb0f2d9d700 1 --2- 172.21.15.16:0/2531797924 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb0f409e410 0x7fb0f4133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:16.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.731+0000 7fb0f1d9b700 1 --2- 172.21.15.16:0/2531797924 >> v2:172.21.15.16:3300/0 conn(0x7fb0f409e000 0x7fb0f41339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:16.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.731+0000 7fb0f259c700 1 --2- 172.21.15.16:0/2531797924 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb0f409d6f0 0x7fb0f409dac0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4009870 tx=0x7fb0e4009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9e7f3848eac9165 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:16.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.731+0000 7fb0f159a700 1 -- 172.21.15.16:0/2531797924 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e4011830 con 0x7fb0f409d6f0 2023-09-30T22:35:16.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.731+0000 7fb0f159a700 1 -- 172.21.15.16:0/2531797924 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0e4011990 con 0x7fb0f409d6f0 2023-09-30T22:35:16.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.731+0000 7fb0f159a700 1 -- 172.21.15.16:0/2531797924 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e4011d20 con 0x7fb0f409d6f0 2023-09-30T22:35:16.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.732+0000 7fb0f8a34700 1 -- 172.21.15.16:0/2531797924 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb0f409d6f0 msgr2=0x7fb0f409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:16.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.732+0000 7fb0f8a34700 1 --2- 172.21.15.16:0/2531797924 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb0f409d6f0 0x7fb0f409dac0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4009870 tx=0x7fb0e4009b80 comp rx=0 tx=0).stop 2023-09-30T22:35:16.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.732+0000 7fb0f8a34700 1 -- 172.21.15.16:0/2531797924 shutdown_connections 2023-09-30T22:35:16.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.732+0000 7fb0f8a34700 1 --2- 172.21.15.16:0/2531797924 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb0f409d6f0 0x7fb0f409dac0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:16.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.732+0000 7fb0f8a34700 1 --2- 172.21.15.16:0/2531797924 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb0f409e410 0x7fb0f4133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:16.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.732+0000 7fb0f8a34700 1 --2- 172.21.15.16:0/2531797924 >> v2:172.21.15.16:3300/0 conn(0x7fb0f409e000 0x7fb0f41339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:16.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.732+0000 7fb0f8a34700 1 -- 172.21.15.16:0/2531797924 >> 172.21.15.16:0/2531797924 conn(0x7fb0f4066f50 msgr2=0x7fb0f409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:16.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.732+0000 7fb0f8a34700 1 -- 172.21.15.16:0/2531797924 shutdown_connections 2023-09-30T22:35:16.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.732+0000 7fb0f8a34700 1 -- 172.21.15.16:0/2531797924 wait complete. 2023-09-30T22:35:16.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.733+0000 7fb0f8a34700 1 Processor -- start 2023-09-30T22:35:16.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.733+0000 7fb0f8a34700 1 -- start start 2023-09-30T22:35:16.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.734+0000 7fb0f8a34700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb0f409d6f0 0x7fb0f41c8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:16.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.734+0000 7fb0f8a34700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb0f409e000 0x7fb0f41c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:16.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.734+0000 7fb0f8a34700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0f409e410 0x7fb0f41c4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:16.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.734+0000 7fb0f8a34700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f41a9810 con 0x7fb0f409e410 2023-09-30T22:35:16.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.734+0000 7fb0f8a34700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f41aad00 con 0x7fb0f409d6f0 2023-09-30T22:35:16.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.734+0000 7fb0f8a34700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f409a0c0 con 0x7fb0f409e000 2023-09-30T22:35:16.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.734+0000 7fb0f259c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb0f409d6f0 0x7fb0f41c8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:16.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.734+0000 7fb0f1d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb0f409e000 0x7fb0f41c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:16.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.734+0000 7fb0f259c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb0f409d6f0 0x7fb0f41c8a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38002/0 (socket says 172.21.15.16:38002) 2023-09-30T22:35:16.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.734+0000 7fb0f1d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb0f409e000 0x7fb0f41c8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:55534/0 (socket says 172.21.15.16:55534) 2023-09-30T22:35:16.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.734+0000 7fb0f259c700 1 -- 172.21.15.16:0/3854530338 learned_addr learned my addr 172.21.15.16:0/3854530338 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:16.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.734+0000 7fb0f2d9d700 1 --2- 172.21.15.16:0/3854530338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0f409e410 0x7fb0f41c4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:16.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.734+0000 7fb0f1d9b700 1 -- 172.21.15.16:0/3854530338 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb0f409d6f0 msgr2=0x7fb0f41c8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:16.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.734+0000 7fb0f1d9b700 1 --2- 172.21.15.16:0/3854530338 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb0f409d6f0 0x7fb0f41c8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:16.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.734+0000 7fb0f1d9b700 1 -- 172.21.15.16:0/3854530338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0f409e410 msgr2=0x7fb0f41c4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:16.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.734+0000 7fb0f1d9b700 1 --2- 172.21.15.16:0/3854530338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0f409e410 0x7fb0f41c4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:16.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.734+0000 7fb0f1d9b700 1 -- 172.21.15.16:0/3854530338 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e8009550 con 0x7fb0f409e000 2023-09-30T22:35:16.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.735+0000 7fb0f259c700 1 --2- 172.21.15.16:0/3854530338 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb0f409d6f0 0x7fb0f41c8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:16.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.735+0000 7fb0f1d9b700 1 --2- 172.21.15.16:0/3854530338 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb0f409e000 0x7fb0f41c8f40 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fb0e800e9a0 tx=0x7fb0e800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:16.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.735+0000 7fb0e2ffd700 1 -- 172.21.15.16:0/3854530338 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e8004030 con 0x7fb0f409e000 2023-09-30T22:35:16.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.735+0000 7fb0e2ffd700 1 -- 172.21.15.16:0/3854530338 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0e8004190 con 0x7fb0f409e000 2023-09-30T22:35:16.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.735+0000 7fb0f8a34700 1 -- 172.21.15.16:0/3854530338 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0ec009a70 con 0x7fb0f409e000 2023-09-30T22:35:16.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.735+0000 7fb0e2ffd700 1 -- 172.21.15.16:0/3854530338 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e8004520 con 0x7fb0f409e000 2023-09-30T22:35:16.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.735+0000 7fb0f8a34700 1 -- 172.21.15.16:0/3854530338 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0e40094a0 con 0x7fb0f409e000 2023-09-30T22:35:16.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.736+0000 7fb0e2ffd700 1 -- 172.21.15.16:0/3854530338 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb0e80036a0 con 0x7fb0f409e000 2023-09-30T22:35:16.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.736+0000 7fb0f8a34700 1 -- 172.21.15.16:0/3854530338 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0f4074ea0 con 0x7fb0f409e000 2023-09-30T22:35:16.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.736+0000 7fb0e2ffd700 1 -- 172.21.15.16:0/3854530338 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fb0e803f840 con 0x7fb0f409e000 2023-09-30T22:35:16.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.741+0000 7fb0e2ffd700 1 -- 172.21.15.16:0/3854530338 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb0e801be10 con 0x7fb0f409e000 2023-09-30T22:35:16.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.776+0000 7f6b24650700 1 Processor -- start 2023-09-30T22:35:16.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.776+0000 7f6b24650700 1 -- start start 2023-09-30T22:35:16.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.777+0000 7f6b24650700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6b1c133420 0x7f6b1c1337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:16.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.777+0000 7f6b24650700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6b1c133e30 0x7f6b1c12ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:16.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.777+0000 7f6b24650700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6b1c134260 0x7f6b1c1304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:16.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.777+0000 7f6b24650700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b1c06b990 con 0x7f6b1c134260 2023-09-30T22:35:16.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.777+0000 7f6b24650700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c0a0590 con 0x7f6b1c133e30 2023-09-30T22:35:16.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.777+0000 7f6b24650700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c09fb80 con 0x7f6b1c133420 2023-09-30T22:35:16.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.777+0000 7f6b22bed700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6b1c134260 0x7f6b1c1304d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:16.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.777+0000 7f6b21beb700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6b1c133e30 0x7f6b1c12ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:16.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.777+0000 7f6b223ec700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6b1c133420 0x7f6b1c1337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:16.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.777+0000 7f6b21beb700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6b1c133e30 0x7f6b1c12ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38006/0 (socket says 172.21.15.16:38006) 2023-09-30T22:35:16.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.777+0000 7f6b223ec700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6b1c133420 0x7f6b1c1337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:55550/0 (socket says 172.21.15.16:55550) 2023-09-30T22:35:16.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.777+0000 7f6b22bed700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6b1c134260 0x7f6b1c1304d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49508/0 (socket says 172.21.15.16:49508) 2023-09-30T22:35:16.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.777+0000 7f6b21beb700 1 -- 172.21.15.16:0/2992655545 learned_addr learned my addr 172.21.15.16:0/2992655545 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:16.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.777+0000 7f6b22bed700 1 -- 172.21.15.16:0/2992655545 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6b1c133420 msgr2=0x7f6b1c1337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:16.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.777+0000 7f6b22bed700 1 --2- 172.21.15.16:0/2992655545 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6b1c133420 0x7f6b1c1337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:16.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.777+0000 7f6b22bed700 1 -- 172.21.15.16:0/2992655545 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6b1c133e30 msgr2=0x7f6b1c12ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:16.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.777+0000 7f6b22bed700 1 --2- 172.21.15.16:0/2992655545 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6b1c133e30 0x7f6b1c12ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:16.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.777+0000 7f6b22bed700 1 -- 172.21.15.16:0/2992655545 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b1c0a0030 con 0x7f6b1c134260 2023-09-30T22:35:16.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.777+0000 7f6b21beb700 1 --2- 172.21.15.16:0/2992655545 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6b1c133e30 0x7f6b1c12ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:16.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.777+0000 7f6b223ec700 1 --2- 172.21.15.16:0/2992655545 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6b1c133420 0x7f6b1c1337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:16.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.777+0000 7f6b22bed700 1 --2- 172.21.15.16:0/2992655545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b1c134260 0x7f6b1c1304d0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f6b14009d90 tx=0x7f6b1400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35b5cd570ea19a8a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:16.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.778+0000 7f6b213ea700 1 -- 172.21.15.16:0/2992655545 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1400e070 con 0x7f6b1c134260 2023-09-30T22:35:16.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.778+0000 7f6b213ea700 1 -- 172.21.15.16:0/2992655545 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b1400bb20 con 0x7f6b1c134260 2023-09-30T22:35:16.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.778+0000 7f6b213ea700 1 -- 172.21.15.16:0/2992655545 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b14012430 con 0x7f6b1c134260 2023-09-30T22:35:16.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.778+0000 7f6b24650700 1 -- 172.21.15.16:0/2992655545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b1c134260 msgr2=0x7f6b1c1304d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:16.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.778+0000 7f6b24650700 1 --2- 172.21.15.16:0/2992655545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b1c134260 0x7f6b1c1304d0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f6b14009d90 tx=0x7f6b1400b330 comp rx=0 tx=0).stop 2023-09-30T22:35:16.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.778+0000 7f6b24650700 1 -- 172.21.15.16:0/2992655545 shutdown_connections 2023-09-30T22:35:16.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.778+0000 7f6b24650700 1 --2- 172.21.15.16:0/2992655545 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b1c134260 0x7f6b1c1304d0 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:16.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.778+0000 7f6b24650700 1 --2- 172.21.15.16:0/2992655545 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6b1c133420 0x7f6b1c1337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:16.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.779+0000 7f6b24650700 1 --2- 172.21.15.16:0/2992655545 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6b1c133e30 0x7f6b1c12ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:16.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.779+0000 7f6b24650700 1 -- 172.21.15.16:0/2992655545 >> 172.21.15.16:0/2992655545 conn(0x7f6b1c0a4210 msgr2=0x7f6b1c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:16.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.779+0000 7f6b24650700 1 -- 172.21.15.16:0/2992655545 shutdown_connections 2023-09-30T22:35:16.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.779+0000 7f6b24650700 1 -- 172.21.15.16:0/2992655545 wait complete. 2023-09-30T22:35:16.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.780+0000 7f6b24650700 1 Processor -- start 2023-09-30T22:35:16.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.780+0000 7f6b24650700 1 -- start start 2023-09-30T22:35:16.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.781+0000 7f6b24650700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6b1c133420 0x7f6b1c09c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:16.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.781+0000 7f6b24650700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b1c133e30 0x7f6b1c09c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:16.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.781+0000 7f6b24650700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6b1c134260 0x7f6b1c09cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:16.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.781+0000 7f6b24650700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c120e40 con 0x7f6b1c133e30 2023-09-30T22:35:16.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.781+0000 7f6b24650700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c09df10 con 0x7f6b1c134260 2023-09-30T22:35:16.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.781+0000 7f6b24650700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c12d670 con 0x7f6b1c133420 2023-09-30T22:35:16.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.781+0000 7f6b223ec700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6b1c133420 0x7f6b1c09c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:16.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.781+0000 7f6b223ec700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6b1c133420 0x7f6b1c09c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:55566/0 (socket says 172.21.15.16:55566) 2023-09-30T22:35:16.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.781+0000 7f6b223ec700 1 -- 172.21.15.16:0/2119178129 learned_addr learned my addr 172.21.15.16:0/2119178129 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:16.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.781+0000 7f6b21beb700 1 --2- 172.21.15.16:0/2119178129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b1c133e30 0x7f6b1c09c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:16.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.781+0000 7f6b22bed700 1 --2- 172.21.15.16:0/2119178129 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6b1c134260 0x7f6b1c09cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:16.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.781+0000 7f6b223ec700 1 -- 172.21.15.16:0/2119178129 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6b1c134260 msgr2=0x7f6b1c09cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:16.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.781+0000 7f6b223ec700 1 --2- 172.21.15.16:0/2119178129 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6b1c134260 0x7f6b1c09cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:16.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.781+0000 7f6b223ec700 1 -- 172.21.15.16:0/2119178129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b1c133e30 msgr2=0x7f6b1c09c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:16.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.781+0000 7f6b223ec700 1 --2- 172.21.15.16:0/2119178129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b1c133e30 0x7f6b1c09c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:16.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.781+0000 7f6b223ec700 1 -- 172.21.15.16:0/2119178129 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b180094a0 con 0x7f6b1c133420 2023-09-30T22:35:16.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.782+0000 7f6b223ec700 1 --2- 172.21.15.16:0/2119178129 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6b1c133420 0x7f6b1c09c480 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f6b1800ebe0 tx=0x7f6b1800eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:16.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.782+0000 7f6b12ffd700 1 -- 172.21.15.16:0/2119178129 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1800c850 con 0x7f6b1c133420 2023-09-30T22:35:16.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.782+0000 7f6b24650700 1 -- 172.21.15.16:0/2119178129 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b0c009550 con 0x7f6b1c133420 2023-09-30T22:35:16.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.782+0000 7f6b12ffd700 1 -- 172.21.15.16:0/2119178129 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b1800c9b0 con 0x7f6b1c133420 2023-09-30T22:35:16.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.782+0000 7f6b12ffd700 1 -- 172.21.15.16:0/2119178129 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1800cd20 con 0x7f6b1c133420 2023-09-30T22:35:16.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.782+0000 7f6b24650700 1 -- 172.21.15.16:0/2119178129 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b14009a70 con 0x7f6b1c133420 2023-09-30T22:35:16.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.783+0000 7f6b12ffd700 1 -- 172.21.15.16:0/2119178129 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6b18013470 con 0x7f6b1c133420 2023-09-30T22:35:16.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.783+0000 7f6b24650700 1 -- 172.21.15.16:0/2119178129 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b1c06d2b0 con 0x7f6b1c133420 2023-09-30T22:35:16.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.783+0000 7f6b12ffd700 1 -- 172.21.15.16:0/2119178129 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f6b1803f7f0 con 0x7f6b1c133420 2023-09-30T22:35:16.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.788+0000 7f6b12ffd700 1 -- 172.21.15.16:0/2119178129 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6b18013720 con 0x7f6b1c133420 2023-09-30T22:35:16.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.884+0000 7f2c03407700 1 -- 172.21.15.16:0/821759295 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2bfc11d490 con 0x7f2bfc09f2c0 2023-09-30T22:35:16.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.885+0000 7f2bf1ffb700 1 -- 172.21.15.16:0/821759295 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2bfc11d490 con 0x7f2bfc09f2c0 2023-09-30T22:35:16.886 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:35:16.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.887+0000 7f2c03407700 1 -- 172.21.15.16:0/821759295 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2bfc09f2c0 msgr2=0x7f2bfc11aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:16.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.887+0000 7f2c03407700 1 --2- 172.21.15.16:0/821759295 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2bfc09f2c0 0x7f2bfc11aa70 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f2bec000c00 tx=0x7f2bec004750 comp rx=0 tx=0).stop 2023-09-30T22:35:16.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.887+0000 7f2c03407700 1 -- 172.21.15.16:0/821759295 shutdown_connections 2023-09-30T22:35:16.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.887+0000 7f2c03407700 1 --2- 172.21.15.16:0/821759295 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2bfc09e7e0 0x7f2bfc11a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:16.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.887+0000 7f2c03407700 1 --2- 172.21.15.16:0/821759295 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bfc09fa10 0x7f2bfc11afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:16.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.887+0000 7f2c03407700 1 --2- 172.21.15.16:0/821759295 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2bfc09f2c0 0x7f2bfc11aa70 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:16.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.887+0000 7f2c03407700 1 -- 172.21.15.16:0/821759295 >> 172.21.15.16:0/821759295 conn(0x7f2bfc0a4300 msgr2=0x7f2bfc09c790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:16.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.888+0000 7f2c03407700 1 -- 172.21.15.16:0/821759295 shutdown_connections 2023-09-30T22:35:16.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.888+0000 7f2c03407700 1 -- 172.21.15.16:0/821759295 wait complete. 2023-09-30T22:35:16.902 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:35:16.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.953+0000 7fb0f8a34700 1 -- 172.21.15.16:0/3854530338 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb0f40a2570 con 0x7fb0f409e000 2023-09-30T22:35:16.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.954+0000 7fb0e2ffd700 1 -- 172.21.15.16:0/3854530338 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb0e801b3b0 con 0x7fb0f409e000 2023-09-30T22:35:16.954 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:35:16.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.956+0000 7fb0f8a34700 1 -- 172.21.15.16:0/3854530338 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb0f409e000 msgr2=0x7fb0f41c8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:16.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.956+0000 7fb0f8a34700 1 --2- 172.21.15.16:0/3854530338 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb0f409e000 0x7fb0f41c8f40 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fb0e800e9a0 tx=0x7fb0e800ecb0 comp rx=0 tx=0).stop 2023-09-30T22:35:16.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.956+0000 7fb0f8a34700 1 -- 172.21.15.16:0/3854530338 shutdown_connections 2023-09-30T22:35:16.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.956+0000 7fb0f8a34700 1 --2- 172.21.15.16:0/3854530338 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb0f409d6f0 0x7fb0f41c8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:16.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.956+0000 7fb0f8a34700 1 --2- 172.21.15.16:0/3854530338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0f409e410 0x7fb0f41c4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:16.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.956+0000 7fb0f8a34700 1 --2- 172.21.15.16:0/3854530338 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb0f409e000 0x7fb0f41c8f40 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:16.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.956+0000 7fb0f8a34700 1 -- 172.21.15.16:0/3854530338 >> 172.21.15.16:0/3854530338 conn(0x7fb0f4066f50 msgr2=0x7fb0f406f390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:16.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.956+0000 7fb0f8a34700 1 -- 172.21.15.16:0/3854530338 shutdown_connections 2023-09-30T22:35:16.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.956+0000 7fb0f8a34700 1 -- 172.21.15.16:0/3854530338 wait complete. 2023-09-30T22:35:16.970 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:35:16.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.996+0000 7f6b24650700 1 -- 172.21.15.16:0/2119178129 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6b1c074ea0 con 0x7f6b1c133420 2023-09-30T22:35:16.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.997+0000 7f6b12ffd700 1 -- 172.21.15.16:0/2119178129 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6b18021020 con 0x7f6b1c133420 2023-09-30T22:35:16.997 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:35:16.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.999+0000 7f6b24650700 1 -- 172.21.15.16:0/2119178129 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6b1c133420 msgr2=0x7f6b1c09c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:16.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.999+0000 7f6b24650700 1 --2- 172.21.15.16:0/2119178129 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6b1c133420 0x7f6b1c09c480 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f6b1800ebe0 tx=0x7f6b1800eef0 comp rx=0 tx=0).stop 2023-09-30T22:35:17.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.999+0000 7f6b24650700 1 -- 172.21.15.16:0/2119178129 shutdown_connections 2023-09-30T22:35:17.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.999+0000 7f6b24650700 1 --2- 172.21.15.16:0/2119178129 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6b1c134260 0x7f6b1c09cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:17.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.999+0000 7f6b24650700 1 --2- 172.21.15.16:0/2119178129 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6b1c133420 0x7f6b1c09c480 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:17.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.999+0000 7f6b24650700 1 --2- 172.21.15.16:0/2119178129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b1c133e30 0x7f6b1c09c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:17.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.999+0000 7f6b24650700 1 -- 172.21.15.16:0/2119178129 >> 172.21.15.16:0/2119178129 conn(0x7f6b1c0a4210 msgr2=0x7f6b1c0a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:17.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.999+0000 7f6b24650700 1 -- 172.21.15.16:0/2119178129 shutdown_connections 2023-09-30T22:35:17.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:16.999+0000 7f6b24650700 1 -- 172.21.15.16:0/2119178129 wait complete. 2023-09-30T22:35:17.013 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:35:17.086 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:35:17.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.227+0000 7f1986ac1700 1 Processor -- start 2023-09-30T22:35:17.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.227+0000 7f1986ac1700 1 -- start start 2023-09-30T22:35:17.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.227+0000 7f1986ac1700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f198009d6d0 0x7f198009daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:17.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.227+0000 7f1986ac1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f198009dfe0 0x7f1980130600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:17.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.228+0000 7f1986ac1700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f198009b940 con 0x7f1980130b40 2023-09-30T22:35:17.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.228+0000 7f1986ac1700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f198009a920 con 0x7f198009d6d0 2023-09-30T22:35:17.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.228+0000 7f1986ac1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f198009ab10 con 0x7f198009dfe0 2023-09-30T22:35:17.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.228+0000 7f198485d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f198009d6d0 0x7f198009daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:17.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.228+0000 7f197ffff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f198009dfe0 0x7f1980130600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:17.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.228+0000 7f198505e700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1980130b40 0x7f19801378c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:51844/0 (socket says 172.21.15.16:51844) 2023-09-30T22:35:17.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.228+0000 7f198505e700 1 -- 172.21.15.16:0/3743496259 learned_addr learned my addr 172.21.15.16:0/3743496259 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:17.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.228+0000 7f197ffff700 1 -- 172.21.15.16:0/3743496259 >> v2:172.21.15.16:3300/0 conn(0x7f198009d6d0 msgr2=0x7f198009daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:17.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.228+0000 7f197ffff700 1 --2- 172.21.15.16:0/3743496259 >> v2:172.21.15.16:3300/0 conn(0x7f198009d6d0 0x7f198009daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:17.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.228+0000 7f197ffff700 1 -- 172.21.15.16:0/3743496259 >> v1:172.21.15.16:6789/0 conn(0x7f1980130b40 legacy=0x7f19801378c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:17.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.228+0000 7f197ffff700 1 -- 172.21.15.16:0/3743496259 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f198009c520 con 0x7f198009dfe0 2023-09-30T22:35:17.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.228+0000 7f198485d700 1 --2- 172.21.15.16:0/3743496259 >> v2:172.21.15.16:3300/0 conn(0x7f198009d6d0 0x7f198009daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:17.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.228+0000 7f197ffff700 1 --2- 172.21.15.16:0/3743496259 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f198009dfe0 0x7f1980130600 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f196c009870 tx=0x7f196c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af6dcaa36470fee6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:17.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.228+0000 7f197f7fe700 1 -- 172.21.15.16:0/3743496259 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196c011830 con 0x7f198009dfe0 2023-09-30T22:35:17.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.228+0000 7f197f7fe700 1 -- 172.21.15.16:0/3743496259 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f196c011990 con 0x7f198009dfe0 2023-09-30T22:35:17.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.228+0000 7f197f7fe700 1 -- 172.21.15.16:0/3743496259 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196c011d20 con 0x7f198009dfe0 2023-09-30T22:35:17.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.229+0000 7f1986ac1700 1 -- 172.21.15.16:0/3743496259 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f198009dfe0 msgr2=0x7f1980130600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:17.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.229+0000 7f1986ac1700 1 --2- 172.21.15.16:0/3743496259 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f198009dfe0 0x7f1980130600 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f196c009870 tx=0x7f196c009b80 comp rx=0 tx=0).stop 2023-09-30T22:35:17.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.229+0000 7f1986ac1700 1 -- 172.21.15.16:0/3743496259 shutdown_connections 2023-09-30T22:35:17.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.229+0000 7f1986ac1700 1 --2- 172.21.15.16:0/3743496259 >> v2:172.21.15.16:3300/0 conn(0x7f198009d6d0 0x7f198009daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:17.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.229+0000 7f1986ac1700 1 --2- 172.21.15.16:0/3743496259 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f198009dfe0 0x7f1980130600 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:17.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.229+0000 7f1986ac1700 1 -- 172.21.15.16:0/3743496259 >> 172.21.15.16:0/3743496259 conn(0x7f1980099c20 msgr2=0x7f19800a2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:17.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.229+0000 7f1986ac1700 1 -- 172.21.15.16:0/3743496259 shutdown_connections 2023-09-30T22:35:17.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.229+0000 7f1986ac1700 1 -- 172.21.15.16:0/3743496259 wait complete. 2023-09-30T22:35:17.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.230+0000 7f1986ac1700 1 Processor -- start 2023-09-30T22:35:17.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.230+0000 7f1986ac1700 1 -- start start 2023-09-30T22:35:17.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.231+0000 7f1986ac1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f198009d6d0 0x7f19801c61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:17.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.231+0000 7f1986ac1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f198009dfe0 0x7f19801c6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:17.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.231+0000 7f1986ac1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1980130b40 0x7f19801c6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:17.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.231+0000 7f1986ac1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1980119760 con 0x7f198009dfe0 2023-09-30T22:35:17.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.231+0000 7f1986ac1700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1980072190 con 0x7f1980130b40 2023-09-30T22:35:17.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.231+0000 7f1986ac1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f198009c740 con 0x7f198009d6d0 2023-09-30T22:35:17.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.231+0000 7f198485d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f198009d6d0 0x7f19801c61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:17.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.231+0000 7f198485d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f198009d6d0 0x7f19801c61e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:55590/0 (socket says 172.21.15.16:55590) 2023-09-30T22:35:17.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.231+0000 7f198485d700 1 -- 172.21.15.16:0/3545024333 learned_addr learned my addr 172.21.15.16:0/3545024333 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:17.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.231+0000 7f197ffff700 1 --2- 172.21.15.16:0/3545024333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f198009dfe0 0x7f19801c6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:17.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.232+0000 7f198505e700 1 --2- 172.21.15.16:0/3545024333 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1980130b40 0x7f19801c6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:17.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.232+0000 7f198485d700 1 -- 172.21.15.16:0/3545024333 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1980130b40 msgr2=0x7f19801c6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:17.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.232+0000 7f198485d700 1 --2- 172.21.15.16:0/3545024333 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1980130b40 0x7f19801c6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:17.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.232+0000 7f198485d700 1 -- 172.21.15.16:0/3545024333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f198009dfe0 msgr2=0x7f19801c6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:17.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.232+0000 7f198485d700 1 --2- 172.21.15.16:0/3545024333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f198009dfe0 0x7f19801c6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:17.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.232+0000 7f198485d700 1 -- 172.21.15.16:0/3545024333 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19740094a0 con 0x7f198009d6d0 2023-09-30T22:35:17.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.232+0000 7f197ffff700 1 --2- 172.21.15.16:0/3545024333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f198009dfe0 0x7f19801c6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:17.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.232+0000 7f198485d700 1 --2- 172.21.15.16:0/3545024333 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f198009d6d0 0x7f19801c61e0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f197400ebe0 tx=0x7f197400eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:17.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.232+0000 7f197d7fa700 1 -- 172.21.15.16:0/3545024333 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197400c850 con 0x7f198009d6d0 2023-09-30T22:35:17.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.232+0000 7f197d7fa700 1 -- 172.21.15.16:0/3545024333 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f197400c9b0 con 0x7f198009d6d0 2023-09-30T22:35:17.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.232+0000 7f1986ac1700 1 -- 172.21.15.16:0/3545024333 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f196c0094a0 con 0x7f198009d6d0 2023-09-30T22:35:17.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.232+0000 7f197d7fa700 1 -- 172.21.15.16:0/3545024333 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197400cd40 con 0x7f198009d6d0 2023-09-30T22:35:17.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.232+0000 7f1986ac1700 1 -- 172.21.15.16:0/3545024333 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1980137c60 con 0x7f198009d6d0 2023-09-30T22:35:17.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.233+0000 7f197d7fa700 1 -- 172.21.15.16:0/3545024333 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1974013470 con 0x7f198009d6d0 2023-09-30T22:35:17.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.233+0000 7f197d7fa700 1 -- 172.21.15.16:0/3545024333 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f197403fb50 con 0x7f198009d6d0 2023-09-30T22:35:17.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.233+0000 7f1986ac1700 1 -- 172.21.15.16:0/3545024333 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1980074ea0 con 0x7f198009d6d0 2023-09-30T22:35:17.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.238+0000 7f197d7fa700 1 -- 172.21.15.16:0/3545024333 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1974013720 con 0x7f198009d6d0 2023-09-30T22:35:17.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.441+0000 7f1986ac1700 1 -- 172.21.15.16:0/3545024333 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f198009b3f0 con 0x7f198009d6d0 2023-09-30T22:35:17.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.441+0000 7f197d7fa700 1 -- 172.21.15.16:0/3545024333 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1974017020 con 0x7f198009d6d0 2023-09-30T22:35:17.442 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:35:17.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.443+0000 7f1986ac1700 1 -- 172.21.15.16:0/3545024333 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f198009d6d0 msgr2=0x7f19801c61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:17.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.443+0000 7f1986ac1700 1 --2- 172.21.15.16:0/3545024333 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f198009d6d0 0x7f19801c61e0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f197400ebe0 tx=0x7f197400eef0 comp rx=0 tx=0).stop 2023-09-30T22:35:17.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.444+0000 7f1986ac1700 1 -- 172.21.15.16:0/3545024333 shutdown_connections 2023-09-30T22:35:17.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.444+0000 7f1986ac1700 1 --2- 172.21.15.16:0/3545024333 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1980130b40 0x7f19801c6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:17.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.444+0000 7f1986ac1700 1 --2- 172.21.15.16:0/3545024333 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f198009d6d0 0x7f19801c61e0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:17.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.444+0000 7f1986ac1700 1 --2- 172.21.15.16:0/3545024333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f198009dfe0 0x7f19801c6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:17.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.444+0000 7f1986ac1700 1 -- 172.21.15.16:0/3545024333 >> 172.21.15.16:0/3545024333 conn(0x7f1980099c20 msgr2=0x7f19801394e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:17.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.444+0000 7f1986ac1700 1 -- 172.21.15.16:0/3545024333 shutdown_connections 2023-09-30T22:35:17.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:17.444+0000 7f1986ac1700 1 -- 172.21.15.16:0/3545024333 wait complete. 2023-09-30T22:35:17.458 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:35:17.903 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:35:17.971 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:35:18.013 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:35:18.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.053+0000 7f19df4b7700 1 Processor -- start 2023-09-30T22:35:18.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.053+0000 7f19df4b7700 1 -- start start 2023-09-30T22:35:18.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.053+0000 7f19df4b7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f19d806b250 0x7f19d8136840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:18.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.053+0000 7f19df4b7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f19d8136d80 0x7f19d8139160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:18.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.053+0000 7f19df4b7700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19d809ece0 con 0x7f19d806a770 2023-09-30T22:35:18.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.053+0000 7f19df4b7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19d806b640 con 0x7f19d806b250 2023-09-30T22:35:18.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.053+0000 7f19df4b7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f19d809e760 con 0x7f19d8136d80 2023-09-30T22:35:18.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.054+0000 7f19dd253700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f19d806a770 0x7f19d806ab40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33472/0 (socket says 172.21.15.16:33472) 2023-09-30T22:35:18.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.054+0000 7f19dd253700 1 -- 172.21.15.16:0/1871886061 learned_addr learned my addr 172.21.15.16:0/1871886061 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:18.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.054+0000 7f19dca52700 1 --2- 172.21.15.16:0/1871886061 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f19d806b250 0x7f19d8136840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:18.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.054+0000 7f19dda54700 1 --2- 172.21.15.16:0/1871886061 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f19d8136d80 0x7f19d8139160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:18.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.054+0000 7f19cffff700 1 -- 172.21.15.16:0/1871886061 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3641895424 0 0) 0x7f19d809ece0 con 0x7f19d806a770 2023-09-30T22:35:18.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.054+0000 7f19dca52700 1 -- 172.21.15.16:0/1871886061 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f19d8136d80 msgr2=0x7f19d8139160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:18.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.054+0000 7f19dca52700 1 --2- 172.21.15.16:0/1871886061 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f19d8136d80 0x7f19d8139160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:18.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.054+0000 7f19dca52700 1 -- 172.21.15.16:0/1871886061 >> v1:172.21.15.16:6789/0 conn(0x7f19d806a770 legacy=0x7f19d806ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:18.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.054+0000 7f19dca52700 1 -- 172.21.15.16:0/1871886061 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d809e040 con 0x7f19d806b250 2023-09-30T22:35:18.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.054+0000 7f19dda54700 1 --2- 172.21.15.16:0/1871886061 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f19d8136d80 0x7f19d8139160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:18.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.054+0000 7f19dca52700 1 --2- 172.21.15.16:0/1871886061 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f19d806b250 0x7f19d8136840 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f19c8009870 tx=0x7f19c8009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90846cd809fc5e38 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:18.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.054+0000 7f19cffff700 1 -- 172.21.15.16:0/1871886061 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c8011830 con 0x7f19d806b250 2023-09-30T22:35:18.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.055+0000 7f19cffff700 1 -- 172.21.15.16:0/1871886061 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19c8011990 con 0x7f19d806b250 2023-09-30T22:35:18.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.055+0000 7f19cffff700 1 -- 172.21.15.16:0/1871886061 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c8011d20 con 0x7f19d806b250 2023-09-30T22:35:18.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.055+0000 7f19df4b7700 1 -- 172.21.15.16:0/1871886061 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f19d806b250 msgr2=0x7f19d8136840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:18.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.055+0000 7f19df4b7700 1 --2- 172.21.15.16:0/1871886061 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f19d806b250 0x7f19d8136840 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f19c8009870 tx=0x7f19c8009b80 comp rx=0 tx=0).stop 2023-09-30T22:35:18.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.055+0000 7f19df4b7700 1 -- 172.21.15.16:0/1871886061 shutdown_connections 2023-09-30T22:35:18.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.055+0000 7f19df4b7700 1 --2- 172.21.15.16:0/1871886061 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f19d8136d80 0x7f19d8139160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:18.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.055+0000 7f19df4b7700 1 --2- 172.21.15.16:0/1871886061 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f19d806b250 0x7f19d8136840 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:18.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.055+0000 7f19df4b7700 1 -- 172.21.15.16:0/1871886061 >> 172.21.15.16:0/1871886061 conn(0x7f19d8066f50 msgr2=0x7f19d809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:18.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.055+0000 7f19df4b7700 1 -- 172.21.15.16:0/1871886061 shutdown_connections 2023-09-30T22:35:18.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.055+0000 7f19df4b7700 1 -- 172.21.15.16:0/1871886061 wait complete. 2023-09-30T22:35:18.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.056+0000 7f19df4b7700 1 Processor -- start 2023-09-30T22:35:18.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.056+0000 7f19df4b7700 1 -- start start 2023-09-30T22:35:18.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.057+0000 7f19df4b7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19d806a770 0x7f19d81c6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:18.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.057+0000 7f19df4b7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f19d806b250 0x7f19d81c67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:18.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.057+0000 7f19df4b7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f19d8136d80 0x7f19d811c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:18.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.057+0000 7f19df4b7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d8120200 con 0x7f19d806a770 2023-09-30T22:35:18.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.057+0000 7f19df4b7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19d811fd50 con 0x7f19d8136d80 2023-09-30T22:35:18.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.057+0000 7f19df4b7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f19d81bbf70 con 0x7f19d806b250 2023-09-30T22:35:18.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.057+0000 7f19dd253700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19d806a770 0x7f19d81c6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:18.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.057+0000 7f19dd253700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19d806a770 0x7f19d81c6280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55306/0 (socket says 172.21.15.16:55306) 2023-09-30T22:35:18.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.057+0000 7f19dd253700 1 -- 172.21.15.16:0/3079166686 learned_addr learned my addr 172.21.15.16:0/3079166686 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:18.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.057+0000 7f19dca52700 1 --2- 172.21.15.16:0/3079166686 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f19d806b250 0x7f19d81c67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:18.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.057+0000 7f19dd253700 1 -- 172.21.15.16:0/3079166686 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f19d806b250 msgr2=0x7f19d81c67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:18.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.057+0000 7f19dd253700 1 --2- 172.21.15.16:0/3079166686 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f19d806b250 0x7f19d81c67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:18.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.057+0000 7f19dd253700 1 -- 172.21.15.16:0/3079166686 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f19d8136d80 msgr2=0x7f19d811c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:18.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.057+0000 7f19dd253700 1 --2- 172.21.15.16:0/3079166686 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f19d8136d80 0x7f19d811c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:18.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.057+0000 7f19dd253700 1 -- 172.21.15.16:0/3079166686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d0009a70 con 0x7f19d806a770 2023-09-30T22:35:18.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.057+0000 7f19dca52700 1 --2- 172.21.15.16:0/3079166686 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f19d806b250 0x7f19d81c67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:18.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.058+0000 7f19dd253700 1 --2- 172.21.15.16:0/3079166686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19d806a770 0x7f19d81c6280 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f19d400c550 tx=0x7f19d400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:18.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.058+0000 7f19cdffb700 1 -- 172.21.15.16:0/3079166686 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d4004c80 con 0x7f19d806a770 2023-09-30T22:35:18.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.058+0000 7f19cdffb700 1 -- 172.21.15.16:0/3079166686 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19d4004de0 con 0x7f19d806a770 2023-09-30T22:35:18.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.058+0000 7f19df4b7700 1 -- 172.21.15.16:0/3079166686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19c80094a0 con 0x7f19d806a770 2023-09-30T22:35:18.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.058+0000 7f19cdffb700 1 -- 172.21.15.16:0/3079166686 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d4007310 con 0x7f19d806a770 2023-09-30T22:35:18.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.058+0000 7f19df4b7700 1 -- 172.21.15.16:0/3079166686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19d81323d0 con 0x7f19d806a770 2023-09-30T22:35:18.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.059+0000 7f19cdffb700 1 -- 172.21.15.16:0/3079166686 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f19d4007470 con 0x7f19d806a770 2023-09-30T22:35:18.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.059+0000 7f19cdffb700 1 -- 172.21.15.16:0/3079166686 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f19d403f0a0 con 0x7f19d806a770 2023-09-30T22:35:18.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.059+0000 7f19df4b7700 1 -- 172.21.15.16:0/3079166686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19d8120810 con 0x7f19d806a770 2023-09-30T22:35:18.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.068+0000 7f19cdffb700 1 -- 172.21.15.16:0/3079166686 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f19d40102c0 con 0x7f19d806a770 2023-09-30T22:35:18.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.122+0000 7f25239f5700 1 Processor -- start 2023-09-30T22:35:18.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.122+0000 7f25239f5700 1 -- start start 2023-09-30T22:35:18.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.123+0000 7f25239f5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f251c09d6f0 0x7f251c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:18.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.123+0000 7f25239f5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f251c09e410 0x7f251c130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:18.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.123+0000 7f25239f5700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f251c09f130 con 0x7f251c09e000 2023-09-30T22:35:18.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.123+0000 7f25239f5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f251c09f520 con 0x7f251c09e410 2023-09-30T22:35:18.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.123+0000 7f25239f5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f251c09e930 con 0x7f251c09d6f0 2023-09-30T22:35:18.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.123+0000 7f2520f90700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f251c09e000 0x7f251c130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33486/0 (socket says 172.21.15.16:33486) 2023-09-30T22:35:18.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.123+0000 7f2520f90700 1 -- 172.21.15.16:0/1135992990 learned_addr learned my addr 172.21.15.16:0/1135992990 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:18.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.123+0000 7f2521f92700 1 --2- 172.21.15.16:0/1135992990 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f251c09e410 0x7f251c130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:18.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.123+0000 7f2521791700 1 --2- 172.21.15.16:0/1135992990 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f251c09d6f0 0x7f251c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:18.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.123+0000 7f2521f92700 1 -- 172.21.15.16:0/1135992990 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f251c09d6f0 msgr2=0x7f251c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:18.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.123+0000 7f2513fff700 1 -- 172.21.15.16:0/1135992990 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 114829008 0 0) 0x7f251c09f130 con 0x7f251c09e000 2023-09-30T22:35:18.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.123+0000 7f2521f92700 1 --2- 172.21.15.16:0/1135992990 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f251c09d6f0 0x7f251c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:18.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.123+0000 7f2521f92700 1 -- 172.21.15.16:0/1135992990 >> v1:172.21.15.16:6789/0 conn(0x7f251c09e000 legacy=0x7f251c130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:18.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.123+0000 7f2521f92700 1 -- 172.21.15.16:0/1135992990 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f251c09b2d0 con 0x7f251c09e410 2023-09-30T22:35:18.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.123+0000 7f2521791700 1 --2- 172.21.15.16:0/1135992990 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f251c09d6f0 0x7f251c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:18.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.124+0000 7f2521f92700 1 --2- 172.21.15.16:0/1135992990 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f251c09e410 0x7f251c130d70 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f2514009dc0 tx=0x7f251400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa5809016a1231dc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:18.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.124+0000 7f2513fff700 1 -- 172.21.15.16:0/1135992990 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251400bd20 con 0x7f251c09e410 2023-09-30T22:35:18.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.124+0000 7f2513fff700 1 -- 172.21.15.16:0/1135992990 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f251400be80 con 0x7f251c09e410 2023-09-30T22:35:18.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.124+0000 7f2513fff700 1 -- 172.21.15.16:0/1135992990 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25140116a0 con 0x7f251c09e410 2023-09-30T22:35:18.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.125+0000 7f25239f5700 1 -- 172.21.15.16:0/1135992990 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f251c09e410 msgr2=0x7f251c130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:18.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.125+0000 7f25239f5700 1 --2- 172.21.15.16:0/1135992990 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f251c09e410 0x7f251c130d70 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f2514009dc0 tx=0x7f251400b330 comp rx=0 tx=0).stop 2023-09-30T22:35:18.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.125+0000 7f25239f5700 1 -- 172.21.15.16:0/1135992990 shutdown_connections 2023-09-30T22:35:18.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.125+0000 7f25239f5700 1 --2- 172.21.15.16:0/1135992990 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f251c09e410 0x7f251c130d70 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:18.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.125+0000 7f25239f5700 1 --2- 172.21.15.16:0/1135992990 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f251c09d6f0 0x7f251c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:18.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.125+0000 7f25239f5700 1 -- 172.21.15.16:0/1135992990 >> 172.21.15.16:0/1135992990 conn(0x7f251c066f50 msgr2=0x7f251c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:18.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.125+0000 7f25239f5700 1 -- 172.21.15.16:0/1135992990 shutdown_connections 2023-09-30T22:35:18.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.125+0000 7f25239f5700 1 -- 172.21.15.16:0/1135992990 wait complete. 2023-09-30T22:35:18.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.126+0000 7f25239f5700 1 Processor -- start 2023-09-30T22:35:18.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.126+0000 7f25239f5700 1 -- start start 2023-09-30T22:35:18.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.127+0000 7f25239f5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f251c09d6f0 0x7f251c11e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:18.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.127+0000 7f25239f5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f251c09e000 0x7f251c11e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:18.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.127+0000 7f25239f5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f251c09e410 0x7f251c11ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:18.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.127+0000 7f25239f5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251c09c700 con 0x7f251c09d6f0 2023-09-30T22:35:18.130 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.127+0000 7f25239f5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f251c1202d0 con 0x7f251c09e410 2023-09-30T22:35:18.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.127+0000 7f25239f5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f251c0a3ba0 con 0x7f251c09e000 2023-09-30T22:35:18.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.127+0000 7f2521791700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f251c09d6f0 0x7f251c11e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:18.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.127+0000 7f2521791700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f251c09d6f0 0x7f251c11e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55316/0 (socket says 172.21.15.16:55316) 2023-09-30T22:35:18.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.127+0000 7f2521791700 1 -- 172.21.15.16:0/3573833572 learned_addr learned my addr 172.21.15.16:0/3573833572 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:18.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.128+0000 7f2520f90700 1 --2- 172.21.15.16:0/3573833572 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f251c09e000 0x7f251c11e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:18.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.128+0000 7f2521f92700 1 --2- 172.21.15.16:0/3573833572 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f251c09e410 0x7f251c11ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:18.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.128+0000 7f2520f90700 1 -- 172.21.15.16:0/3573833572 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f251c09e410 msgr2=0x7f251c11ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:18.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.128+0000 7f2520f90700 1 --2- 172.21.15.16:0/3573833572 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f251c09e410 0x7f251c11ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:18.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.128+0000 7f2520f90700 1 -- 172.21.15.16:0/3573833572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f251c09d6f0 msgr2=0x7f251c11e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:18.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.128+0000 7f2520f90700 1 --2- 172.21.15.16:0/3573833572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f251c09d6f0 0x7f251c11e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:18.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.128+0000 7f2520f90700 1 -- 172.21.15.16:0/3573833572 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25180094a0 con 0x7f251c09e000 2023-09-30T22:35:18.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.128+0000 7f2521791700 1 --2- 172.21.15.16:0/3573833572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f251c09d6f0 0x7f251c11e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:18.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.128+0000 7f2520f90700 1 --2- 172.21.15.16:0/3573833572 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f251c09e000 0x7f251c11e6e0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f251c0a3e20 tx=0x7f250c00c750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:18.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.128+0000 7f2511ffb700 1 -- 172.21.15.16:0/3573833572 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250c004c80 con 0x7f251c09e000 2023-09-30T22:35:18.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.128+0000 7f2511ffb700 1 -- 172.21.15.16:0/3573833572 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f250c004de0 con 0x7f251c09e000 2023-09-30T22:35:18.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.128+0000 7f25239f5700 1 -- 172.21.15.16:0/3573833572 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2514009a70 con 0x7f251c09e000 2023-09-30T22:35:18.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.128+0000 7f2511ffb700 1 -- 172.21.15.16:0/3573833572 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f250c007310 con 0x7f251c09e000 2023-09-30T22:35:18.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.129+0000 7f25239f5700 1 -- 172.21.15.16:0/3573833572 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f251c121950 con 0x7f251c09e000 2023-09-30T22:35:18.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.130+0000 7f2511ffb700 1 -- 172.21.15.16:0/3573833572 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f250c007470 con 0x7f251c09e000 2023-09-30T22:35:18.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.130+0000 7f2511ffb700 1 -- 172.21.15.16:0/3573833572 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f250c03f1f0 con 0x7f251c09e000 2023-09-30T22:35:18.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.130+0000 7f25239f5700 1 -- 172.21.15.16:0/3573833572 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f251c11ac20 con 0x7f251c09e000 2023-09-30T22:35:18.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.135+0000 7f2511ffb700 1 -- 172.21.15.16:0/3573833572 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f250c007720 con 0x7f251c09e000 2023-09-30T22:35:18.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.160+0000 7f4979f2f700 1 Processor -- start 2023-09-30T22:35:18.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.160+0000 7f4979f2f700 1 -- start start 2023-09-30T22:35:18.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.161+0000 7f4979f2f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f49740a0a30 0x7f49740a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:18.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.161+0000 7f4979f2f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4974134170 0x7f497413bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:18.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.161+0000 7f4979f2f700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f497409edd0 con 0x7f49740a1340 2023-09-30T22:35:18.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.161+0000 7f4979f2f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f497409e1f0 con 0x7f49740a0a30 2023-09-30T22:35:18.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.161+0000 7f4979f2f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f497409dc50 con 0x7f4974134170 2023-09-30T22:35:18.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.161+0000 7f4972ffd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f49740a1340 0x7f4974133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33488/0 (socket says 172.21.15.16:33488) 2023-09-30T22:35:18.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.161+0000 7f4972ffd700 1 -- 172.21.15.16:0/150122421 learned_addr learned my addr 172.21.15.16:0/150122421 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:18.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.161+0000 7f4973fff700 1 --2- 172.21.15.16:0/150122421 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4974134170 0x7f497413bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:18.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.161+0000 7f49737fe700 1 --2- 172.21.15.16:0/150122421 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f49740a0a30 0x7f49740a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:18.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.161+0000 7f4973fff700 1 -- 172.21.15.16:0/150122421 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f49740a0a30 msgr2=0x7f49740a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:18.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.161+0000 7f4973fff700 1 --2- 172.21.15.16:0/150122421 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f49740a0a30 0x7f49740a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:18.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.161+0000 7f4973fff700 1 -- 172.21.15.16:0/150122421 >> v1:172.21.15.16:6789/0 conn(0x7f49740a1340 legacy=0x7f4974133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:18.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.161+0000 7f4973fff700 1 -- 172.21.15.16:0/150122421 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49740a1fe0 con 0x7f4974134170 2023-09-30T22:35:18.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.162+0000 7f4973fff700 1 --2- 172.21.15.16:0/150122421 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4974134170 0x7f497413bd30 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f496c009dc0 tx=0x7f496c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e33b25a0f37f96c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:18.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.162+0000 7f49727fc700 1 -- 172.21.15.16:0/150122421 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496c00e070 con 0x7f4974134170 2023-09-30T22:35:18.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.162+0000 7f49727fc700 1 -- 172.21.15.16:0/150122421 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f496c003a20 con 0x7f4974134170 2023-09-30T22:35:18.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.162+0000 7f49727fc700 1 -- 172.21.15.16:0/150122421 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496c003dd0 con 0x7f4974134170 2023-09-30T22:35:18.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.162+0000 7f4979f2f700 1 -- 172.21.15.16:0/150122421 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4974134170 msgr2=0x7f497413bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:18.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.162+0000 7f4979f2f700 1 --2- 172.21.15.16:0/150122421 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4974134170 0x7f497413bd30 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f496c009dc0 tx=0x7f496c00b330 comp rx=0 tx=0).stop 2023-09-30T22:35:18.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.163+0000 7f4979f2f700 1 -- 172.21.15.16:0/150122421 shutdown_connections 2023-09-30T22:35:18.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.163+0000 7f4979f2f700 1 --2- 172.21.15.16:0/150122421 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4974134170 0x7f497413bd30 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:18.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.163+0000 7f4979f2f700 1 --2- 172.21.15.16:0/150122421 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f49740a0a30 0x7f49740a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:18.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.163+0000 7f4979f2f700 1 -- 172.21.15.16:0/150122421 >> 172.21.15.16:0/150122421 conn(0x7f49740a6550 msgr2=0x7f49740a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:18.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.163+0000 7f4979f2f700 1 -- 172.21.15.16:0/150122421 shutdown_connections 2023-09-30T22:35:18.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.163+0000 7f4979f2f700 1 -- 172.21.15.16:0/150122421 wait complete. 2023-09-30T22:35:18.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.164+0000 7f4979f2f700 1 Processor -- start 2023-09-30T22:35:18.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.164+0000 7f4979f2f700 1 -- start start 2023-09-30T22:35:18.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.164+0000 7f4979f2f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f49740a0a30 0x7f497411e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:18.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.164+0000 7f4979f2f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f49740a1340 0x7f497411ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:18.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.164+0000 7f4979f2f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4974134170 0x7f497411f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:18.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.164+0000 7f4979f2f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4974133010 con 0x7f4974134170 2023-09-30T22:35:18.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.164+0000 7f4979f2f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4974120270 con 0x7f49740a0a30 2023-09-30T22:35:18.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.164+0000 7f4979f2f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4974120890 con 0x7f49740a1340 2023-09-30T22:35:18.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.164+0000 7f4972ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f49740a1340 0x7f497411ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:18.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.164+0000 7f4972ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f49740a1340 0x7f497411ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:32986/0 (socket says 172.21.15.16:32986) 2023-09-30T22:35:18.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.164+0000 7f4972ffd700 1 -- 172.21.15.16:0/3101529345 learned_addr learned my addr 172.21.15.16:0/3101529345 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:18.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.164+0000 7f49737fe700 1 --2- 172.21.15.16:0/3101529345 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f49740a0a30 0x7f497411e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:18.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.165+0000 7f4973fff700 1 --2- 172.21.15.16:0/3101529345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4974134170 0x7f497411f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:18.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.165+0000 7f4972ffd700 1 -- 172.21.15.16:0/3101529345 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f49740a0a30 msgr2=0x7f497411e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:18.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.165+0000 7f4972ffd700 1 --2- 172.21.15.16:0/3101529345 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f49740a0a30 0x7f497411e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:18.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.165+0000 7f4972ffd700 1 -- 172.21.15.16:0/3101529345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4974134170 msgr2=0x7f497411f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:18.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.165+0000 7f4972ffd700 1 --2- 172.21.15.16:0/3101529345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4974134170 0x7f497411f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:18.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.165+0000 7f4972ffd700 1 -- 172.21.15.16:0/3101529345 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f496c009a70 con 0x7f49740a1340 2023-09-30T22:35:18.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.165+0000 7f4972ffd700 1 --2- 172.21.15.16:0/3101529345 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f49740a1340 0x7f497411ec30 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f496800c780 tx=0x7f496800ca90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:18.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.165+0000 7f495bfff700 1 -- 172.21.15.16:0/3101529345 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4968004c80 con 0x7f49740a1340 2023-09-30T22:35:18.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.165+0000 7f495bfff700 1 -- 172.21.15.16:0/3101529345 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4968004de0 con 0x7f49740a1340 2023-09-30T22:35:18.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.165+0000 7f495bfff700 1 -- 172.21.15.16:0/3101529345 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4968007310 con 0x7f49740a1340 2023-09-30T22:35:18.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.165+0000 7f4979f2f700 1 -- 172.21.15.16:0/3101529345 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49740a25e0 con 0x7f49740a1340 2023-09-30T22:35:18.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.165+0000 7f4979f2f700 1 -- 172.21.15.16:0/3101529345 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49740a4eb0 con 0x7f49740a1340 2023-09-30T22:35:18.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.166+0000 7f495bfff700 1 -- 172.21.15.16:0/3101529345 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4968007470 con 0x7f49740a1340 2023-09-30T22:35:18.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.166+0000 7f4979f2f700 1 -- 172.21.15.16:0/3101529345 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49741335f0 con 0x7f49740a1340 2023-09-30T22:35:18.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.167+0000 7f495bfff700 1 -- 172.21.15.16:0/3101529345 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f496803f450 con 0x7f49740a1340 2023-09-30T22:35:18.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.172+0000 7f495bfff700 1 -- 172.21.15.16:0/3101529345 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4968013bc0 con 0x7f49740a1340 2023-09-30T22:35:18.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.280+0000 7f19df4b7700 1 -- 172.21.15.16:0/3079166686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f19d809f6b0 con 0x7f19d806a770 2023-09-30T22:35:18.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.280+0000 7f19cdffb700 1 -- 172.21.15.16:0/3079166686 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f19d401bb30 con 0x7f19d806a770 2023-09-30T22:35:18.281 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:35:18.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.283+0000 7f19df4b7700 1 -- 172.21.15.16:0/3079166686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19d806a770 msgr2=0x7f19d81c6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:18.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.283+0000 7f19df4b7700 1 --2- 172.21.15.16:0/3079166686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19d806a770 0x7f19d81c6280 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f19d400c550 tx=0x7f19d400c860 comp rx=0 tx=0).stop 2023-09-30T22:35:18.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.283+0000 7f19df4b7700 1 -- 172.21.15.16:0/3079166686 shutdown_connections 2023-09-30T22:35:18.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.283+0000 7f19df4b7700 1 --2- 172.21.15.16:0/3079166686 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f19d8136d80 0x7f19d811c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:18.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.283+0000 7f19df4b7700 1 --2- 172.21.15.16:0/3079166686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19d806a770 0x7f19d81c6280 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:18.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.283+0000 7f19df4b7700 1 --2- 172.21.15.16:0/3079166686 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f19d806b250 0x7f19d81c67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:18.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.283+0000 7f19df4b7700 1 -- 172.21.15.16:0/3079166686 >> 172.21.15.16:0/3079166686 conn(0x7f19d8066f50 msgr2=0x7f19d80a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:18.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.283+0000 7f19df4b7700 1 -- 172.21.15.16:0/3079166686 shutdown_connections 2023-09-30T22:35:18.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.283+0000 7f19df4b7700 1 -- 172.21.15.16:0/3079166686 wait complete. 2023-09-30T22:35:18.298 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:35:18.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.345+0000 7f25239f5700 1 -- 172.21.15.16:0/3573833572 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f251c074ea0 con 0x7f251c09e000 2023-09-30T22:35:18.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.345+0000 7f2511ffb700 1 -- 172.21.15.16:0/3573833572 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f250c01b760 con 0x7f251c09e000 2023-09-30T22:35:18.346 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:35:18.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.347+0000 7f25239f5700 1 -- 172.21.15.16:0/3573833572 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f251c09e000 msgr2=0x7f251c11e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:18.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.347+0000 7f25239f5700 1 --2- 172.21.15.16:0/3573833572 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f251c09e000 0x7f251c11e6e0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f251c0a3e20 tx=0x7f250c00c750 comp rx=0 tx=0).stop 2023-09-30T22:35:18.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.347+0000 7f25239f5700 1 -- 172.21.15.16:0/3573833572 shutdown_connections 2023-09-30T22:35:18.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.347+0000 7f25239f5700 1 --2- 172.21.15.16:0/3573833572 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f251c09e410 0x7f251c11ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:18.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.347+0000 7f25239f5700 1 --2- 172.21.15.16:0/3573833572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f251c09d6f0 0x7f251c11e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:18.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.347+0000 7f25239f5700 1 --2- 172.21.15.16:0/3573833572 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f251c09e000 0x7f251c11e6e0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:18.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.347+0000 7f25239f5700 1 -- 172.21.15.16:0/3573833572 >> 172.21.15.16:0/3573833572 conn(0x7f251c066f50 msgr2=0x7f251c130b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:18.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.347+0000 7f25239f5700 1 -- 172.21.15.16:0/3573833572 shutdown_connections 2023-09-30T22:35:18.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.347+0000 7f25239f5700 1 -- 172.21.15.16:0/3573833572 wait complete. 2023-09-30T22:35:18.362 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:35:18.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.379+0000 7f4979f2f700 1 -- 172.21.15.16:0/3101529345 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4974121fc0 con 0x7f49740a1340 2023-09-30T22:35:18.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.379+0000 7f495bfff700 1 -- 172.21.15.16:0/3101529345 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4968011360 con 0x7f49740a1340 2023-09-30T22:35:18.380 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:35:18.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.381+0000 7f4979f2f700 1 -- 172.21.15.16:0/3101529345 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f49740a1340 msgr2=0x7f497411ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:18.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.381+0000 7f4979f2f700 1 --2- 172.21.15.16:0/3101529345 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f49740a1340 0x7f497411ec30 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f496800c780 tx=0x7f496800ca90 comp rx=0 tx=0).stop 2023-09-30T22:35:18.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.381+0000 7f4979f2f700 1 -- 172.21.15.16:0/3101529345 shutdown_connections 2023-09-30T22:35:18.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.381+0000 7f4979f2f700 1 --2- 172.21.15.16:0/3101529345 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f49740a0a30 0x7f497411e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:18.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.381+0000 7f4979f2f700 1 --2- 172.21.15.16:0/3101529345 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4974134170 0x7f497411f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:18.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.381+0000 7f4979f2f700 1 --2- 172.21.15.16:0/3101529345 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f49740a1340 0x7f497411ec30 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:18.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.381+0000 7f4979f2f700 1 -- 172.21.15.16:0/3101529345 >> 172.21.15.16:0/3101529345 conn(0x7f49740a6550 msgr2=0x7f497409ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:18.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.381+0000 7f4979f2f700 1 -- 172.21.15.16:0/3101529345 shutdown_connections 2023-09-30T22:35:18.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.381+0000 7f4979f2f700 1 -- 172.21.15.16:0/3101529345 wait complete. 2023-09-30T22:35:18.395 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:35:18.459 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:35:18.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.601+0000 7fa0ac8d3700 1 Processor -- start 2023-09-30T22:35:18.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.601+0000 7fa0ac8d3700 1 -- start start 2023-09-30T22:35:18.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.601+0000 7fa0ac8d3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa0a809e930 0x7fa0a809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:18.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.601+0000 7fa0ac8d3700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa0a809f240 0x7fa0a81317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:18.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.601+0000 7fa0ac8d3700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0a8070940 con 0x7fa0a809f650 2023-09-30T22:35:18.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.601+0000 7fa0ac8d3700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0a806fd60 con 0x7fa0a809f240 2023-09-30T22:35:18.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.601+0000 7fa0ac8d3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a806fa80 con 0x7fa0a809e930 2023-09-30T22:35:18.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.602+0000 7fa0a5d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa0a809f240 0x7fa0a81317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:18.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.602+0000 7fa0a659c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa0a809e930 0x7fa0a809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:18.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.602+0000 7fa0a6d9d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa0a809f650 0x7fa0a8131d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33498/0 (socket says 172.21.15.16:33498) 2023-09-30T22:35:18.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.602+0000 7fa0a6d9d700 1 -- 172.21.15.16:0/2948696357 learned_addr learned my addr 172.21.15.16:0/2948696357 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:18.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.602+0000 7fa0a5d9b700 1 -- 172.21.15.16:0/2948696357 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa0a809e930 msgr2=0x7fa0a809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:18.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.602+0000 7fa0a5d9b700 1 --2- 172.21.15.16:0/2948696357 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa0a809e930 0x7fa0a809ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:18.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.602+0000 7fa0a5d9b700 1 -- 172.21.15.16:0/2948696357 >> v1:172.21.15.16:6789/0 conn(0x7fa0a809f650 legacy=0x7fa0a8131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:18.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.602+0000 7fa0a5d9b700 1 -- 172.21.15.16:0/2948696357 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a809fb60 con 0x7fa0a809f240 2023-09-30T22:35:18.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.602+0000 7fa0a559a700 1 -- 172.21.15.16:0/2948696357 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3786124652 0 0) 0x7fa0a8070940 con 0x7fa0a809f650 2023-09-30T22:35:18.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.602+0000 7fa0a659c700 1 --2- 172.21.15.16:0/2948696357 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa0a809e930 0x7fa0a809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:18.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.602+0000 7fa0a5d9b700 1 --2- 172.21.15.16:0/2948696357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0a809f240 0x7fa0a81317f0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fa09c009840 tx=0x7fa09c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ed81d70018933aa server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:18.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.602+0000 7fa0a559a700 1 -- 172.21.15.16:0/2948696357 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09c011830 con 0x7fa0a809f240 2023-09-30T22:35:18.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.602+0000 7fa0a559a700 1 -- 172.21.15.16:0/2948696357 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa09c011990 con 0x7fa0a809f240 2023-09-30T22:35:18.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.602+0000 7fa0a559a700 1 -- 172.21.15.16:0/2948696357 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09c011d20 con 0x7fa0a809f240 2023-09-30T22:35:18.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.603+0000 7fa0ac8d3700 1 -- 172.21.15.16:0/2948696357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0a809f240 msgr2=0x7fa0a81317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:18.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.603+0000 7fa0ac8d3700 1 --2- 172.21.15.16:0/2948696357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0a809f240 0x7fa0a81317f0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fa09c009840 tx=0x7fa09c009b50 comp rx=0 tx=0).stop 2023-09-30T22:35:18.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.603+0000 7fa0ac8d3700 1 -- 172.21.15.16:0/2948696357 shutdown_connections 2023-09-30T22:35:18.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.603+0000 7fa0ac8d3700 1 --2- 172.21.15.16:0/2948696357 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa0a809e930 0x7fa0a809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:18.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.603+0000 7fa0ac8d3700 1 --2- 172.21.15.16:0/2948696357 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0a809f240 0x7fa0a81317f0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:18.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.603+0000 7fa0ac8d3700 1 -- 172.21.15.16:0/2948696357 >> 172.21.15.16:0/2948696357 conn(0x7fa0a80a4210 msgr2=0x7fa0a80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:18.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.603+0000 7fa0ac8d3700 1 -- 172.21.15.16:0/2948696357 shutdown_connections 2023-09-30T22:35:18.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.603+0000 7fa0ac8d3700 1 -- 172.21.15.16:0/2948696357 wait complete. 2023-09-30T22:35:18.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.605+0000 7fa0ac8d3700 1 Processor -- start 2023-09-30T22:35:18.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.605+0000 7fa0ac8d3700 1 -- start start 2023-09-30T22:35:18.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.605+0000 7fa0ac8d3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa0a809e930 0x7fa0a81a3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:18.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.605+0000 7fa0ac8d3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa0a809f240 0x7fa0a81a4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:18.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.605+0000 7fa0ac8d3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0a809f650 0x7fa0a81a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:18.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.605+0000 7fa0ac8d3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a81a5f20 con 0x7fa0a809f650 2023-09-30T22:35:18.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.605+0000 7fa0ac8d3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a81a6be0 con 0x7fa0a809f240 2023-09-30T22:35:18.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.605+0000 7fa0ac8d3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a80a32f0 con 0x7fa0a809e930 2023-09-30T22:35:18.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.606+0000 7fa0a659c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa0a809e930 0x7fa0a81a3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:18.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.606+0000 7fa0a659c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa0a809e930 0x7fa0a81a3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:32990/0 (socket says 172.21.15.16:32990) 2023-09-30T22:35:18.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.606+0000 7fa0a659c700 1 -- 172.21.15.16:0/1282194775 learned_addr learned my addr 172.21.15.16:0/1282194775 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:18.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.606+0000 7fa0a5d9b700 1 --2- 172.21.15.16:0/1282194775 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa0a809f240 0x7fa0a81a4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:18.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.606+0000 7fa0a6d9d700 1 --2- 172.21.15.16:0/1282194775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0a809f650 0x7fa0a81a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:18.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.606+0000 7fa0a659c700 1 -- 172.21.15.16:0/1282194775 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa0a809f240 msgr2=0x7fa0a81a4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:18.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.606+0000 7fa0a659c700 1 --2- 172.21.15.16:0/1282194775 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa0a809f240 0x7fa0a81a4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:18.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.606+0000 7fa0a659c700 1 -- 172.21.15.16:0/1282194775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0a809f650 msgr2=0x7fa0a81a4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:18.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.606+0000 7fa0a659c700 1 --2- 172.21.15.16:0/1282194775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0a809f650 0x7fa0a81a4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:18.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.606+0000 7fa0a659c700 1 -- 172.21.15.16:0/1282194775 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0980094a0 con 0x7fa0a809e930 2023-09-30T22:35:18.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.606+0000 7fa0a659c700 1 --2- 172.21.15.16:0/1282194775 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa0a809e930 0x7fa0a81a3cf0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fa09800ebe0 tx=0x7fa09800eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:18.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.606+0000 7fa096ffd700 1 -- 172.21.15.16:0/1282194775 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09800c850 con 0x7fa0a809e930 2023-09-30T22:35:18.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.606+0000 7fa096ffd700 1 -- 172.21.15.16:0/1282194775 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa09800c9b0 con 0x7fa0a809e930 2023-09-30T22:35:18.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.606+0000 7fa096ffd700 1 -- 172.21.15.16:0/1282194775 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa09800cd40 con 0x7fa0a809e930 2023-09-30T22:35:18.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.606+0000 7fa0ac8d3700 1 -- 172.21.15.16:0/1282194775 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa09c0094a0 con 0x7fa0a809e930 2023-09-30T22:35:18.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.606+0000 7fa0ac8d3700 1 -- 172.21.15.16:0/1282194775 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0a80a0190 con 0x7fa0a809e930 2023-09-30T22:35:18.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.607+0000 7fa096ffd700 1 -- 172.21.15.16:0/1282194775 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa098013470 con 0x7fa0a809e930 2023-09-30T22:35:18.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.607+0000 7fa096ffd700 1 -- 172.21.15.16:0/1282194775 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fa09803fb50 con 0x7fa0a809e930 2023-09-30T22:35:18.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.608+0000 7fa0ac8d3700 1 -- 172.21.15.16:0/1282194775 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0a81a0b30 con 0x7fa0a809e930 2023-09-30T22:35:18.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.612+0000 7fa096ffd700 1 -- 172.21.15.16:0/1282194775 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa098013720 con 0x7fa0a809e930 2023-09-30T22:35:18.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.819+0000 7fa0ac8d3700 1 -- 172.21.15.16:0/1282194775 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa0a8074ea0 con 0x7fa0a809e930 2023-09-30T22:35:18.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.819+0000 7fa096ffd700 1 -- 172.21.15.16:0/1282194775 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa098021030 con 0x7fa0a809e930 2023-09-30T22:35:18.820 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:35:18.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.821+0000 7fa0ac8d3700 1 -- 172.21.15.16:0/1282194775 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa0a809e930 msgr2=0x7fa0a81a3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:18.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.821+0000 7fa0ac8d3700 1 --2- 172.21.15.16:0/1282194775 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa0a809e930 0x7fa0a81a3cf0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fa09800ebe0 tx=0x7fa09800eef0 comp rx=0 tx=0).stop 2023-09-30T22:35:18.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.822+0000 7fa0ac8d3700 1 -- 172.21.15.16:0/1282194775 shutdown_connections 2023-09-30T22:35:18.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.822+0000 7fa0ac8d3700 1 --2- 172.21.15.16:0/1282194775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa0a809f650 0x7fa0a81a4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:18.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.822+0000 7fa0ac8d3700 1 --2- 172.21.15.16:0/1282194775 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa0a809e930 0x7fa0a81a3cf0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:18.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.822+0000 7fa0ac8d3700 1 --2- 172.21.15.16:0/1282194775 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa0a809f240 0x7fa0a81a4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:18.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.822+0000 7fa0ac8d3700 1 -- 172.21.15.16:0/1282194775 >> 172.21.15.16:0/1282194775 conn(0x7fa0a80a4210 msgr2=0x7fa0a81b8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:18.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.822+0000 7fa0ac8d3700 1 -- 172.21.15.16:0/1282194775 shutdown_connections 2023-09-30T22:35:18.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:18.822+0000 7fa0ac8d3700 1 -- 172.21.15.16:0/1282194775 wait complete. 2023-09-30T22:35:18.836 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:35:19.300 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:35:19.362 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:35:19.396 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:35:19.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.451+0000 7fe983b4f700 1 Processor -- start 2023-09-30T22:35:19.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.451+0000 7fe983b4f700 1 -- start start 2023-09-30T22:35:19.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.452+0000 7fe983b4f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe97c09d6f0 0x7fe97c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:19.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.452+0000 7fe983b4f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe97c09e000 0x7fe97c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:19.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.452+0000 7fe983b4f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe97c09e410 0x7fe97c133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:19.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.452+0000 7fe983b4f700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe97c06b460 con 0x7fe97c09e410 2023-09-30T22:35:19.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.452+0000 7fe983b4f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe97c09b120 con 0x7fe97c09e000 2023-09-30T22:35:19.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.452+0000 7fe983b4f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe97c09a7e0 con 0x7fe97c09d6f0 2023-09-30T22:35:19.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.452+0000 7fe9818eb700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe97c09d6f0 0x7fe97c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:19.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.452+0000 7fe9818eb700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe97c09d6f0 0x7fe97c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:32996/0 (socket says 172.21.15.16:32996) 2023-09-30T22:35:19.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.452+0000 7fe9818eb700 1 -- 172.21.15.16:0/1034043107 learned_addr learned my addr 172.21.15.16:0/1034043107 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:19.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.452+0000 7fe9810ea700 1 --2- 172.21.15.16:0/1034043107 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe97c09e000 0x7fe97c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:19.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.452+0000 7fe9820ec700 1 --2- 172.21.15.16:0/1034043107 >> v2:172.21.15.16:3300/0 conn(0x7fe97c09e410 0x7fe97c133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:19.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.452+0000 7fe9818eb700 1 -- 172.21.15.16:0/1034043107 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe97c09e000 msgr2=0x7fe97c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:19.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.452+0000 7fe9818eb700 1 --2- 172.21.15.16:0/1034043107 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe97c09e000 0x7fe97c1339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:19.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.452+0000 7fe9818eb700 1 -- 172.21.15.16:0/1034043107 >> v2:172.21.15.16:3300/0 conn(0x7fe97c09e410 msgr2=0x7fe97c133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:19.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.452+0000 7fe9818eb700 1 --2- 172.21.15.16:0/1034043107 >> v2:172.21.15.16:3300/0 conn(0x7fe97c09e410 0x7fe97c133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:19.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.452+0000 7fe9818eb700 1 -- 172.21.15.16:0/1034043107 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe97c09a500 con 0x7fe97c09d6f0 2023-09-30T22:35:19.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.452+0000 7fe9810ea700 1 --2- 172.21.15.16:0/1034043107 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe97c09e000 0x7fe97c1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:19.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.453+0000 7fe9818eb700 1 --2- 172.21.15.16:0/1034043107 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe97c09d6f0 0x7fe97c09dac0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fe978009920 tx=0x7fe978009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5dd4ea1c1359a896 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:19.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.453+0000 7fe9808e9700 1 -- 172.21.15.16:0/1034043107 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe978011830 con 0x7fe97c09d6f0 2023-09-30T22:35:19.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.453+0000 7fe9808e9700 1 -- 172.21.15.16:0/1034043107 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe978011990 con 0x7fe97c09d6f0 2023-09-30T22:35:19.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.453+0000 7fe9808e9700 1 -- 172.21.15.16:0/1034043107 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe978011d20 con 0x7fe97c09d6f0 2023-09-30T22:35:19.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.454+0000 7fe983b4f700 1 -- 172.21.15.16:0/1034043107 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe97c09d6f0 msgr2=0x7fe97c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:19.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.454+0000 7fe983b4f700 1 --2- 172.21.15.16:0/1034043107 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe97c09d6f0 0x7fe97c09dac0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fe978009920 tx=0x7fe978009c30 comp rx=0 tx=0).stop 2023-09-30T22:35:19.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.454+0000 7fe983b4f700 1 -- 172.21.15.16:0/1034043107 shutdown_connections 2023-09-30T22:35:19.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.454+0000 7fe983b4f700 1 --2- 172.21.15.16:0/1034043107 >> v2:172.21.15.16:3300/0 conn(0x7fe97c09e410 0x7fe97c133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:19.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.454+0000 7fe983b4f700 1 --2- 172.21.15.16:0/1034043107 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe97c09d6f0 0x7fe97c09dac0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:19.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.454+0000 7fe983b4f700 1 --2- 172.21.15.16:0/1034043107 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe97c09e000 0x7fe97c1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:19.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.454+0000 7fe983b4f700 1 -- 172.21.15.16:0/1034043107 >> 172.21.15.16:0/1034043107 conn(0x7fe97c066f50 msgr2=0x7fe97c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:19.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.454+0000 7fe983b4f700 1 -- 172.21.15.16:0/1034043107 shutdown_connections 2023-09-30T22:35:19.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.454+0000 7fe983b4f700 1 -- 172.21.15.16:0/1034043107 wait complete. 2023-09-30T22:35:19.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.455+0000 7fe983b4f700 1 Processor -- start 2023-09-30T22:35:19.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.455+0000 7fe983b4f700 1 -- start start 2023-09-30T22:35:19.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.455+0000 7fe983b4f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe97c09d6f0 0x7fe97c1a5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:19.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.455+0000 7fe983b4f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe97c09e000 0x7fe97c1a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:19.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.455+0000 7fe983b4f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe97c09e410 0x7fe97c1a68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:19.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.455+0000 7fe983b4f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe97c1a9170 con 0x7fe97c09d6f0 2023-09-30T22:35:19.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.455+0000 7fe983b4f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe97c1a7a90 con 0x7fe97c09e410 2023-09-30T22:35:19.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.455+0000 7fe983b4f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe97c1a9620 con 0x7fe97c09e000 2023-09-30T22:35:19.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.456+0000 7fe9810ea700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe97c09e000 0x7fe97c1a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:19.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.456+0000 7fe9810ea700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe97c09e000 0x7fe97c1a6370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:33006/0 (socket says 172.21.15.16:33006) 2023-09-30T22:35:19.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.456+0000 7fe9810ea700 1 -- 172.21.15.16:0/489283367 learned_addr learned my addr 172.21.15.16:0/489283367 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:19.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.456+0000 7fe9818eb700 1 --2- 172.21.15.16:0/489283367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe97c09d6f0 0x7fe97c1a5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:19.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.456+0000 7fe9820ec700 1 --2- 172.21.15.16:0/489283367 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe97c09e410 0x7fe97c1a68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:19.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.456+0000 7fe9810ea700 1 -- 172.21.15.16:0/489283367 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe97c09e410 msgr2=0x7fe97c1a68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:19.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.456+0000 7fe9810ea700 1 --2- 172.21.15.16:0/489283367 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe97c09e410 0x7fe97c1a68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:19.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.456+0000 7fe9810ea700 1 -- 172.21.15.16:0/489283367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe97c09d6f0 msgr2=0x7fe97c1a5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:19.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.456+0000 7fe9810ea700 1 --2- 172.21.15.16:0/489283367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe97c09d6f0 0x7fe97c1a5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:19.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.456+0000 7fe9810ea700 1 -- 172.21.15.16:0/489283367 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe96c0094a0 con 0x7fe97c09e000 2023-09-30T22:35:19.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.456+0000 7fe9818eb700 1 --2- 172.21.15.16:0/489283367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe97c09d6f0 0x7fe97c1a5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:19.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.456+0000 7fe9810ea700 1 --2- 172.21.15.16:0/489283367 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe97c09e000 0x7fe97c1a6370 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fe96c00ebe0 tx=0x7fe96c00eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:19.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.456+0000 7fe9727fc700 1 -- 172.21.15.16:0/489283367 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96c00c850 con 0x7fe97c09e000 2023-09-30T22:35:19.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.456+0000 7fe9727fc700 1 -- 172.21.15.16:0/489283367 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe96c00c9b0 con 0x7fe97c09e000 2023-09-30T22:35:19.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.457+0000 7fe983b4f700 1 -- 172.21.15.16:0/489283367 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe978009550 con 0x7fe97c09e000 2023-09-30T22:35:19.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.457+0000 7fe9727fc700 1 -- 172.21.15.16:0/489283367 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe96c00cd40 con 0x7fe97c09e000 2023-09-30T22:35:19.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.457+0000 7fe983b4f700 1 -- 172.21.15.16:0/489283367 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe97c1a8c40 con 0x7fe97c09e000 2023-09-30T22:35:19.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.457+0000 7fe9727fc700 1 -- 172.21.15.16:0/489283367 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe96c013470 con 0x7fe97c09e000 2023-09-30T22:35:19.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.458+0000 7fe9727fc700 1 -- 172.21.15.16:0/489283367 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fe96c03fb50 con 0x7fe97c09e000 2023-09-30T22:35:19.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.458+0000 7fe983b4f700 1 -- 172.21.15.16:0/489283367 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe97c074ea0 con 0x7fe97c09e000 2023-09-30T22:35:19.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.463+0000 7fe9727fc700 1 -- 172.21.15.16:0/489283367 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe96c013720 con 0x7fe97c09e000 2023-09-30T22:35:19.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.508+0000 7f85f4760700 1 Processor -- start 2023-09-30T22:35:19.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.508+0000 7f85f4760700 1 -- start start 2023-09-30T22:35:19.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.509+0000 7f85f4760700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85ec1337f0 0x7f85ec138880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:19.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.509+0000 7f85f4760700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85ec138dc0 0x7f85ec13b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:19.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.509+0000 7f85f4760700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85ec06b990 con 0x7f85ec133420 2023-09-30T22:35:19.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.509+0000 7f85f4760700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85ec09b0e0 con 0x7f85ec138dc0 2023-09-30T22:35:19.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.509+0000 7f85f4760700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f85ec133c20 con 0x7f85ec1337f0 2023-09-30T22:35:19.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.509+0000 7f85f24fc700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f85ec133420 0x7f85ec133e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33512/0 (socket says 172.21.15.16:33512) 2023-09-30T22:35:19.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.509+0000 7f85f24fc700 1 -- 172.21.15.16:0/333851539 learned_addr learned my addr 172.21.15.16:0/333851539 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:19.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.509+0000 7f85f2cfd700 1 --2- 172.21.15.16:0/333851539 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85ec138dc0 0x7f85ec13b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:19.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.509+0000 7f85f1cfb700 1 --2- 172.21.15.16:0/333851539 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85ec1337f0 0x7f85ec138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:19.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.509+0000 7f85f14fa700 1 -- 172.21.15.16:0/333851539 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 974863229 0 0) 0x7f85ec06b990 con 0x7f85ec133420 2023-09-30T22:35:19.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.509+0000 7f85f14fa700 1 -- 172.21.15.16:0/333851539 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85d80035b0 con 0x7f85ec133420 2023-09-30T22:35:19.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.509+0000 7f85f2cfd700 1 -- 172.21.15.16:0/333851539 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85ec1337f0 msgr2=0x7f85ec138880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:19.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.509+0000 7f85f2cfd700 1 --2- 172.21.15.16:0/333851539 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85ec1337f0 0x7f85ec138880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:19.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.509+0000 7f85f2cfd700 1 -- 172.21.15.16:0/333851539 >> v1:172.21.15.16:6789/0 conn(0x7f85ec133420 legacy=0x7f85ec133e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:19.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.509+0000 7f85f2cfd700 1 -- 172.21.15.16:0/333851539 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85ec0a0590 con 0x7f85ec138dc0 2023-09-30T22:35:19.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.509+0000 7f85f1cfb700 1 --2- 172.21.15.16:0/333851539 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85ec1337f0 0x7f85ec138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:19.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.510+0000 7f85f2cfd700 1 --2- 172.21.15.16:0/333851539 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85ec138dc0 0x7f85ec13b1a0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f85e4009dc0 tx=0x7f85e400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccbd7a6ba46dc813 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:19.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.510+0000 7f85f14fa700 1 -- 172.21.15.16:0/333851539 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e400e070 con 0x7f85ec138dc0 2023-09-30T22:35:19.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.510+0000 7f85f14fa700 1 -- 172.21.15.16:0/333851539 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85e400bb20 con 0x7f85ec138dc0 2023-09-30T22:35:19.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.510+0000 7f85f14fa700 1 -- 172.21.15.16:0/333851539 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85e4012430 con 0x7f85ec138dc0 2023-09-30T22:35:19.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.510+0000 7f85f4760700 1 -- 172.21.15.16:0/333851539 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85ec138dc0 msgr2=0x7f85ec13b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:19.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.510+0000 7f85f4760700 1 --2- 172.21.15.16:0/333851539 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85ec138dc0 0x7f85ec13b1a0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f85e4009dc0 tx=0x7f85e400b330 comp rx=0 tx=0).stop 2023-09-30T22:35:19.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.511+0000 7f85f4760700 1 -- 172.21.15.16:0/333851539 shutdown_connections 2023-09-30T22:35:19.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.511+0000 7f85f4760700 1 --2- 172.21.15.16:0/333851539 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85ec138dc0 0x7f85ec13b1a0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:19.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.511+0000 7f85f4760700 1 --2- 172.21.15.16:0/333851539 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85ec1337f0 0x7f85ec138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:19.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.511+0000 7f85f4760700 1 -- 172.21.15.16:0/333851539 >> 172.21.15.16:0/333851539 conn(0x7f85ec0a4210 msgr2=0x7f85ec0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:19.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.511+0000 7f85f4760700 1 -- 172.21.15.16:0/333851539 shutdown_connections 2023-09-30T22:35:19.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.511+0000 7f85f4760700 1 -- 172.21.15.16:0/333851539 wait complete. 2023-09-30T22:35:19.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.512+0000 7f85f4760700 1 Processor -- start 2023-09-30T22:35:19.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.512+0000 7f85f4760700 1 -- start start 2023-09-30T22:35:19.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.513+0000 7f85f4760700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85ec133420 0x7f85ec0709a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:19.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.513+0000 7f85f4760700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85ec1337f0 0x7f85ec070ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:19.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.513+0000 7f85f4760700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85ec138dc0 0x7f85ec071420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:19.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.513+0000 7f85f4760700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85ec072270 con 0x7f85ec1337f0 2023-09-30T22:35:19.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.513+0000 7f85f4760700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85ec09ccc0 con 0x7f85ec138dc0 2023-09-30T22:35:19.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.513+0000 7f85f4760700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f85ec09b430 con 0x7f85ec133420 2023-09-30T22:35:19.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.513+0000 7f85f1cfb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85ec1337f0 0x7f85ec070ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:19.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.513+0000 7f85f1cfb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85ec1337f0 0x7f85ec070ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55372/0 (socket says 172.21.15.16:55372) 2023-09-30T22:35:19.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.513+0000 7f85f1cfb700 1 -- 172.21.15.16:0/1865161522 learned_addr learned my addr 172.21.15.16:0/1865161522 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:19.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.513+0000 7f85f2cfd700 1 --2- 172.21.15.16:0/1865161522 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85ec138dc0 0x7f85ec071420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:19.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.513+0000 7f85f24fc700 1 --2- 172.21.15.16:0/1865161522 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85ec133420 0x7f85ec0709a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:19.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.513+0000 7f85f1cfb700 1 -- 172.21.15.16:0/1865161522 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85ec133420 msgr2=0x7f85ec0709a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:19.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.513+0000 7f85f1cfb700 1 --2- 172.21.15.16:0/1865161522 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85ec133420 0x7f85ec0709a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:19.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.513+0000 7f85f1cfb700 1 -- 172.21.15.16:0/1865161522 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85ec138dc0 msgr2=0x7f85ec071420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:19.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.513+0000 7f85f1cfb700 1 --2- 172.21.15.16:0/1865161522 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85ec138dc0 0x7f85ec071420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:19.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.513+0000 7f85f1cfb700 1 -- 172.21.15.16:0/1865161522 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85dc0094a0 con 0x7f85ec1337f0 2023-09-30T22:35:19.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.514+0000 7f85f1cfb700 1 --2- 172.21.15.16:0/1865161522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85ec1337f0 0x7f85ec070ee0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f85dc00ebe0 tx=0x7f85dc00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:19.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.514+0000 7f85e2ffd700 1 -- 172.21.15.16:0/1865161522 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85dc00c850 con 0x7f85ec1337f0 2023-09-30T22:35:19.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.514+0000 7f85e2ffd700 1 -- 172.21.15.16:0/1865161522 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85dc00c9b0 con 0x7f85ec1337f0 2023-09-30T22:35:19.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.514+0000 7f85f4760700 1 -- 172.21.15.16:0/1865161522 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85e4009a70 con 0x7f85ec1337f0 2023-09-30T22:35:19.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.514+0000 7f85e2ffd700 1 -- 172.21.15.16:0/1865161522 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85dc00cd40 con 0x7f85ec1337f0 2023-09-30T22:35:19.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.514+0000 7f85f4760700 1 -- 172.21.15.16:0/1865161522 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85ec09be70 con 0x7f85ec1337f0 2023-09-30T22:35:19.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.515+0000 7f85e2ffd700 1 -- 172.21.15.16:0/1865161522 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f85dc013470 con 0x7f85ec1337f0 2023-09-30T22:35:19.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.515+0000 7f85f4760700 1 -- 172.21.15.16:0/1865161522 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85ec074ea0 con 0x7f85ec1337f0 2023-09-30T22:35:19.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.515+0000 7f85e2ffd700 1 -- 172.21.15.16:0/1865161522 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f85dc03fb70 con 0x7f85ec1337f0 2023-09-30T22:35:19.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.520+0000 7f85e2ffd700 1 -- 172.21.15.16:0/1865161522 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f85dc013720 con 0x7f85ec1337f0 2023-09-30T22:35:19.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.551+0000 7fc2ad34b700 1 Processor -- start 2023-09-30T22:35:19.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.551+0000 7fc2ad34b700 1 -- start start 2023-09-30T22:35:19.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.551+0000 7fc2ad34b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2a80a0a30 0x7fc2a80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:19.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.551+0000 7fc2ad34b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2a8138d80 0x7fc2a813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:19.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.551+0000 7fc2ad34b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2a80a2a70 con 0x7fc2a809d630 2023-09-30T22:35:19.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.551+0000 7fc2ad34b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a80a01d0 con 0x7fc2a80a0a30 2023-09-30T22:35:19.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.551+0000 7fc2ad34b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a809e2b0 con 0x7fc2a8138d80 2023-09-30T22:35:19.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.552+0000 7fc2a77fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2a8138d80 0x7fc2a813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:19.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.552+0000 7fc2a67fc700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc2a809d630 0x7fc2a80a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33514/0 (socket says 172.21.15.16:33514) 2023-09-30T22:35:19.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.552+0000 7fc2a67fc700 1 -- 172.21.15.16:0/337443480 learned_addr learned my addr 172.21.15.16:0/337443480 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:19.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.552+0000 7fc2a77fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2a8138d80 0x7fc2a813b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:33042/0 (socket says 172.21.15.16:33042) 2023-09-30T22:35:19.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.552+0000 7fc2a6ffd700 1 --2- 172.21.15.16:0/337443480 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2a80a0a30 0x7fc2a80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:19.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.552+0000 7fc2a5ffb700 1 -- 172.21.15.16:0/337443480 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4212796984 0 0) 0x7fc2a80a2a70 con 0x7fc2a809d630 2023-09-30T22:35:19.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.552+0000 7fc2a5ffb700 1 -- 172.21.15.16:0/337443480 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2900035b0 con 0x7fc2a809d630 2023-09-30T22:35:19.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.552+0000 7fc2a77fe700 1 -- 172.21.15.16:0/337443480 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2a80a0a30 msgr2=0x7fc2a80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:19.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.552+0000 7fc2a77fe700 1 --2- 172.21.15.16:0/337443480 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2a80a0a30 0x7fc2a80a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:19.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.552+0000 7fc2a77fe700 1 -- 172.21.15.16:0/337443480 >> v1:172.21.15.16:6789/0 conn(0x7fc2a809d630 legacy=0x7fc2a80a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:19.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.552+0000 7fc2a77fe700 1 -- 172.21.15.16:0/337443480 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a809dc50 con 0x7fc2a8138d80 2023-09-30T22:35:19.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.552+0000 7fc2a77fe700 1 --2- 172.21.15.16:0/337443480 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2a8138d80 0x7fc2a813b160 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0009dc0 tx=0x7fc2a000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=880017f95c4db005 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:19.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.552+0000 7fc2a5ffb700 1 -- 172.21.15.16:0/337443480 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a000e070 con 0x7fc2a8138d80 2023-09-30T22:35:19.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.552+0000 7fc2a5ffb700 1 -- 172.21.15.16:0/337443480 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2a000bb20 con 0x7fc2a8138d80 2023-09-30T22:35:19.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.553+0000 7fc2a5ffb700 1 -- 172.21.15.16:0/337443480 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a000be90 con 0x7fc2a8138d80 2023-09-30T22:35:19.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.553+0000 7fc2ad34b700 1 -- 172.21.15.16:0/337443480 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2a8138d80 msgr2=0x7fc2a813b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:19.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.553+0000 7fc2ad34b700 1 --2- 172.21.15.16:0/337443480 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2a8138d80 0x7fc2a813b160 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0009dc0 tx=0x7fc2a000b330 comp rx=0 tx=0).stop 2023-09-30T22:35:19.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.553+0000 7fc2ad34b700 1 -- 172.21.15.16:0/337443480 shutdown_connections 2023-09-30T22:35:19.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.553+0000 7fc2ad34b700 1 --2- 172.21.15.16:0/337443480 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2a8138d80 0x7fc2a813b160 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:19.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.553+0000 7fc2ad34b700 1 --2- 172.21.15.16:0/337443480 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2a80a0a30 0x7fc2a80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:19.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.553+0000 7fc2ad34b700 1 -- 172.21.15.16:0/337443480 >> 172.21.15.16:0/337443480 conn(0x7fc2a80a3050 msgr2=0x7fc2a80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:19.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.553+0000 7fc2ad34b700 1 -- 172.21.15.16:0/337443480 shutdown_connections 2023-09-30T22:35:19.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.553+0000 7fc2ad34b700 1 -- 172.21.15.16:0/337443480 wait complete. 2023-09-30T22:35:19.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.554+0000 7fc2ad34b700 1 Processor -- start 2023-09-30T22:35:19.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.554+0000 7fc2ad34b700 1 -- start start 2023-09-30T22:35:19.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.555+0000 7fc2ad34b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2a809d630 0x7fc2a8133650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:19.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.555+0000 7fc2ad34b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2a80a0a30 0x7fc2a8133b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:19.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.555+0000 7fc2ad34b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2a8138d80 0x7fc2a81340d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:19.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.555+0000 7fc2ad34b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a8136440 con 0x7fc2a809d630 2023-09-30T22:35:19.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.555+0000 7fc2ad34b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a8136db0 con 0x7fc2a8138d80 2023-09-30T22:35:19.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.555+0000 7fc2ad34b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a8136090 con 0x7fc2a80a0a30 2023-09-30T22:35:19.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.555+0000 7fc2a77fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2a8138d80 0x7fc2a81340d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:19.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.555+0000 7fc2a77fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2a8138d80 0x7fc2a81340d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:52854/0 (socket says 172.21.15.16:52854) 2023-09-30T22:35:19.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.555+0000 7fc2a77fe700 1 -- 172.21.15.16:0/1968818616 learned_addr learned my addr 172.21.15.16:0/1968818616 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:19.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.555+0000 7fc2a67fc700 1 --2- 172.21.15.16:0/1968818616 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2a80a0a30 0x7fc2a8133b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:19.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.555+0000 7fc2a6ffd700 1 --2- 172.21.15.16:0/1968818616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2a809d630 0x7fc2a8133650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:19.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.555+0000 7fc2a77fe700 1 -- 172.21.15.16:0/1968818616 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2a80a0a30 msgr2=0x7fc2a8133b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:19.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.555+0000 7fc2a77fe700 1 --2- 172.21.15.16:0/1968818616 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2a80a0a30 0x7fc2a8133b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:19.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.555+0000 7fc2a77fe700 1 -- 172.21.15.16:0/1968818616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2a809d630 msgr2=0x7fc2a8133650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:19.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.555+0000 7fc2a77fe700 1 --2- 172.21.15.16:0/1968818616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2a809d630 0x7fc2a8133650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:19.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.555+0000 7fc2a77fe700 1 -- 172.21.15.16:0/1968818616 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a0009a70 con 0x7fc2a8138d80 2023-09-30T22:35:19.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.555+0000 7fc2a6ffd700 1 --2- 172.21.15.16:0/1968818616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2a809d630 0x7fc2a8133650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:35:19.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.556+0000 7fc2a77fe700 1 --2- 172.21.15.16:0/1968818616 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2a8138d80 0x7fc2a81340d0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0013000 tx=0x7fc2a0004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:19.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.556+0000 7fc28f7fe700 1 -- 172.21.15.16:0/1968818616 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a0007060 con 0x7fc2a8138d80 2023-09-30T22:35:19.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.556+0000 7fc28f7fe700 1 -- 172.21.15.16:0/1968818616 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2a0020070 con 0x7fc2a8138d80 2023-09-30T22:35:19.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.556+0000 7fc2ad34b700 1 -- 172.21.15.16:0/1968818616 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2a80a4920 con 0x7fc2a8138d80 2023-09-30T22:35:19.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.556+0000 7fc28f7fe700 1 -- 172.21.15.16:0/1968818616 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a00276c0 con 0x7fc2a8138d80 2023-09-30T22:35:19.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.556+0000 7fc2ad34b700 1 -- 172.21.15.16:0/1968818616 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2a8137590 con 0x7fc2a8138d80 2023-09-30T22:35:19.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.557+0000 7fc28f7fe700 1 -- 172.21.15.16:0/1968818616 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc2a0027820 con 0x7fc2a8138d80 2023-09-30T22:35:19.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.557+0000 7fc2ad34b700 1 -- 172.21.15.16:0/1968818616 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2a80747f0 con 0x7fc2a8138d80 2023-09-30T22:35:19.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.557+0000 7fc28f7fe700 1 -- 172.21.15.16:0/1968818616 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fc2a0022070 con 0x7fc2a8138d80 2023-09-30T22:35:19.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.562+0000 7fc28f7fe700 1 -- 172.21.15.16:0/1968818616 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc2a001a380 con 0x7fc2a8138d80 2023-09-30T22:35:19.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.676+0000 7fe983b4f700 1 -- 172.21.15.16:0/489283367 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe97c06c360 con 0x7fe97c09e000 2023-09-30T22:35:19.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.677+0000 7fe9727fc700 1 -- 172.21.15.16:0/489283367 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe96c011b40 con 0x7fe97c09e000 2023-09-30T22:35:19.677 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:35:19.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.680+0000 7fe983b4f700 1 -- 172.21.15.16:0/489283367 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe97c09e000 msgr2=0x7fe97c1a6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:19.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.680+0000 7fe983b4f700 1 --2- 172.21.15.16:0/489283367 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe97c09e000 0x7fe97c1a6370 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fe96c00ebe0 tx=0x7fe96c00eef0 comp rx=0 tx=0).stop 2023-09-30T22:35:19.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.680+0000 7fe983b4f700 1 -- 172.21.15.16:0/489283367 shutdown_connections 2023-09-30T22:35:19.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.680+0000 7fe983b4f700 1 --2- 172.21.15.16:0/489283367 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe97c09e410 0x7fe97c1a68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:19.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.680+0000 7fe983b4f700 1 --2- 172.21.15.16:0/489283367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe97c09d6f0 0x7fe97c1a5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:19.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.680+0000 7fe983b4f700 1 --2- 172.21.15.16:0/489283367 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe97c09e000 0x7fe97c1a6370 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:19.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.681+0000 7fe983b4f700 1 -- 172.21.15.16:0/489283367 >> 172.21.15.16:0/489283367 conn(0x7fe97c066f50 msgr2=0x7fe97c1bae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:19.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.681+0000 7fe983b4f700 1 -- 172.21.15.16:0/489283367 shutdown_connections 2023-09-30T22:35:19.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.681+0000 7fe983b4f700 1 -- 172.21.15.16:0/489283367 wait complete. 2023-09-30T22:35:19.700 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:35:19.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.728+0000 7f85f4760700 1 -- 172.21.15.16:0/1865161522 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f85ec0008d0 con 0x7f85ec1337f0 2023-09-30T22:35:19.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.728+0000 7f85e2ffd700 1 -- 172.21.15.16:0/1865161522 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f85dc021030 con 0x7f85ec1337f0 2023-09-30T22:35:19.730 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:35:19.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.730+0000 7f85f4760700 1 -- 172.21.15.16:0/1865161522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85ec1337f0 msgr2=0x7f85ec070ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:19.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.730+0000 7f85f4760700 1 --2- 172.21.15.16:0/1865161522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85ec1337f0 0x7f85ec070ee0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f85dc00ebe0 tx=0x7f85dc00eef0 comp rx=0 tx=0).stop 2023-09-30T22:35:19.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.730+0000 7f85f4760700 1 -- 172.21.15.16:0/1865161522 shutdown_connections 2023-09-30T22:35:19.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.730+0000 7f85f4760700 1 --2- 172.21.15.16:0/1865161522 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f85ec138dc0 0x7f85ec071420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:19.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.730+0000 7f85f4760700 1 --2- 172.21.15.16:0/1865161522 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f85ec133420 0x7f85ec0709a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:19.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.730+0000 7f85f4760700 1 --2- 172.21.15.16:0/1865161522 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85ec1337f0 0x7f85ec070ee0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:19.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.730+0000 7f85f4760700 1 -- 172.21.15.16:0/1865161522 >> 172.21.15.16:0/1865161522 conn(0x7f85ec0a4210 msgr2=0x7f85ec099c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:19.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.730+0000 7f85f4760700 1 -- 172.21.15.16:0/1865161522 shutdown_connections 2023-09-30T22:35:19.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.730+0000 7f85f4760700 1 -- 172.21.15.16:0/1865161522 wait complete. 2023-09-30T22:35:19.745 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:35:19.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.784+0000 7fc2ad34b700 1 -- 172.21.15.16:0/1968818616 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc2a8072980 con 0x7fc2a8138d80 2023-09-30T22:35:19.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.785+0000 7fc28f7fe700 1 -- 172.21.15.16:0/1968818616 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc2a0011b80 con 0x7fc2a8138d80 2023-09-30T22:35:19.785 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:35:19.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.787+0000 7fc2ad34b700 1 -- 172.21.15.16:0/1968818616 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2a8138d80 msgr2=0x7fc2a81340d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:19.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.787+0000 7fc2ad34b700 1 --2- 172.21.15.16:0/1968818616 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2a8138d80 0x7fc2a81340d0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0013000 tx=0x7fc2a0004ad0 comp rx=0 tx=0).stop 2023-09-30T22:35:19.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.787+0000 7fc2ad34b700 1 -- 172.21.15.16:0/1968818616 shutdown_connections 2023-09-30T22:35:19.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.787+0000 7fc2ad34b700 1 --2- 172.21.15.16:0/1968818616 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2a8138d80 0x7fc2a81340d0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:19.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.787+0000 7fc2ad34b700 1 --2- 172.21.15.16:0/1968818616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2a809d630 0x7fc2a8133650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:19.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.787+0000 7fc2ad34b700 1 --2- 172.21.15.16:0/1968818616 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2a80a0a30 0x7fc2a8133b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:19.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.787+0000 7fc2ad34b700 1 -- 172.21.15.16:0/1968818616 >> 172.21.15.16:0/1968818616 conn(0x7fc2a80a3050 msgr2=0x7fc2a813acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:19.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.787+0000 7fc2ad34b700 1 -- 172.21.15.16:0/1968818616 shutdown_connections 2023-09-30T22:35:19.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.787+0000 7fc2ad34b700 1 -- 172.21.15.16:0/1968818616 wait complete. 2023-09-30T22:35:19.801 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:35:19.837 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:35:19.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.985+0000 7fb7179e2700 1 Processor -- start 2023-09-30T22:35:19.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.985+0000 7fb7179e2700 1 -- start start 2023-09-30T22:35:19.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.986+0000 7fb7179e2700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb71009e930 0x7fb71009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:19.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.986+0000 7fb7179e2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb71009f240 0x7fb7101317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:19.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.986+0000 7fb7179e2700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb710070940 con 0x7fb71009f650 2023-09-30T22:35:19.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.986+0000 7fb7179e2700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb71006fd60 con 0x7fb71009e930 2023-09-30T22:35:19.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.986+0000 7fb7179e2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb71006fa80 con 0x7fb71009f240 2023-09-30T22:35:19.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.986+0000 7fb71577e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb71009e930 0x7fb71009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:19.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.986+0000 7fb715f7f700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb71009f650 0x7fb710131d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33520/0 (socket says 172.21.15.16:33520) 2023-09-30T22:35:19.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.986+0000 7fb715f7f700 1 -- 172.21.15.16:0/3496423599 learned_addr learned my addr 172.21.15.16:0/3496423599 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:19.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.986+0000 7fb714f7d700 1 --2- 172.21.15.16:0/3496423599 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb71009f240 0x7fb7101317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:19.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.986+0000 7fb71577e700 1 -- 172.21.15.16:0/3496423599 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb71009f240 msgr2=0x7fb7101317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:19.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.986+0000 7fb71577e700 1 --2- 172.21.15.16:0/3496423599 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb71009f240 0x7fb7101317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:19.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.986+0000 7fb71577e700 1 -- 172.21.15.16:0/3496423599 >> v1:172.21.15.16:6789/0 conn(0x7fb71009f650 legacy=0x7fb710131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:19.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.986+0000 7fb707fff700 1 -- 172.21.15.16:0/3496423599 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 262162190 0 0) 0x7fb710070940 con 0x7fb71009f650 2023-09-30T22:35:19.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.986+0000 7fb71577e700 1 -- 172.21.15.16:0/3496423599 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb71009fb60 con 0x7fb71009e930 2023-09-30T22:35:19.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.986+0000 7fb714f7d700 1 --2- 172.21.15.16:0/3496423599 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb71009f240 0x7fb7101317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:19.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.986+0000 7fb71577e700 1 --2- 172.21.15.16:0/3496423599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb71009e930 0x7fb71009ed00 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fb700009840 tx=0x7fb700009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b74acfab43e79347 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:19.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.986+0000 7fb707fff700 1 -- 172.21.15.16:0/3496423599 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb700011830 con 0x7fb71009e930 2023-09-30T22:35:19.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.987+0000 7fb707fff700 1 -- 172.21.15.16:0/3496423599 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb700011990 con 0x7fb71009e930 2023-09-30T22:35:19.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.987+0000 7fb707fff700 1 -- 172.21.15.16:0/3496423599 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb700011d20 con 0x7fb71009e930 2023-09-30T22:35:19.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.987+0000 7fb7179e2700 1 -- 172.21.15.16:0/3496423599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb71009e930 msgr2=0x7fb71009ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:19.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.987+0000 7fb7179e2700 1 --2- 172.21.15.16:0/3496423599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb71009e930 0x7fb71009ed00 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fb700009840 tx=0x7fb700009b50 comp rx=0 tx=0).stop 2023-09-30T22:35:19.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.987+0000 7fb7179e2700 1 -- 172.21.15.16:0/3496423599 shutdown_connections 2023-09-30T22:35:19.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.987+0000 7fb7179e2700 1 --2- 172.21.15.16:0/3496423599 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb71009e930 0x7fb71009ed00 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:19.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.987+0000 7fb7179e2700 1 --2- 172.21.15.16:0/3496423599 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb71009f240 0x7fb7101317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:19.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.987+0000 7fb7179e2700 1 -- 172.21.15.16:0/3496423599 >> 172.21.15.16:0/3496423599 conn(0x7fb7100a4210 msgr2=0x7fb7100a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:19.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.987+0000 7fb7179e2700 1 -- 172.21.15.16:0/3496423599 shutdown_connections 2023-09-30T22:35:19.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.987+0000 7fb7179e2700 1 -- 172.21.15.16:0/3496423599 wait complete. 2023-09-30T22:35:19.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.989+0000 7fb7179e2700 1 Processor -- start 2023-09-30T22:35:19.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.989+0000 7fb7179e2700 1 -- start start 2023-09-30T22:35:19.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.989+0000 7fb7179e2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb71009e930 0x7fb7101c6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:19.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.989+0000 7fb7179e2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb71009f240 0x7fb7101c6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:19.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.989+0000 7fb7179e2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb71009f650 0x7fb7101c27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:19.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.989+0000 7fb7179e2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7101a4fe0 con 0x7fb71009e930 2023-09-30T22:35:19.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.989+0000 7fb7179e2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7101a75d0 con 0x7fb71009f650 2023-09-30T22:35:19.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.989+0000 7fb7179e2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb7101c6ca0 con 0x7fb71009f240 2023-09-30T22:35:19.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.989+0000 7fb71577e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb71009e930 0x7fb7101c6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:19.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.990+0000 7fb71577e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb71009e930 0x7fb7101c6220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55396/0 (socket says 172.21.15.16:55396) 2023-09-30T22:35:19.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.990+0000 7fb71577e700 1 -- 172.21.15.16:0/3214051675 learned_addr learned my addr 172.21.15.16:0/3214051675 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:19.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.990+0000 7fb714f7d700 1 --2- 172.21.15.16:0/3214051675 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb71009f240 0x7fb7101c6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:19.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.990+0000 7fb715f7f700 1 --2- 172.21.15.16:0/3214051675 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb71009f650 0x7fb7101c27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:19.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.990+0000 7fb71577e700 1 -- 172.21.15.16:0/3214051675 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb71009f240 msgr2=0x7fb7101c6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:19.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.990+0000 7fb71577e700 1 --2- 172.21.15.16:0/3214051675 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb71009f240 0x7fb7101c6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:19.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.990+0000 7fb71577e700 1 -- 172.21.15.16:0/3214051675 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb71009f650 msgr2=0x7fb7101c27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:19.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.990+0000 7fb71577e700 1 --2- 172.21.15.16:0/3214051675 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb71009f650 0x7fb7101c27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:19.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.990+0000 7fb71577e700 1 -- 172.21.15.16:0/3214051675 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb70c0094a0 con 0x7fb71009e930 2023-09-30T22:35:19.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.990+0000 7fb71577e700 1 --2- 172.21.15.16:0/3214051675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb71009e930 0x7fb7101c6220 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fb700009fd0 tx=0x7fb700004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:19.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.990+0000 7fb705ffb700 1 -- 172.21.15.16:0/3214051675 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7000036a0 con 0x7fb71009e930 2023-09-30T22:35:19.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.990+0000 7fb705ffb700 1 -- 172.21.15.16:0/3214051675 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb70001f070 con 0x7fb71009e930 2023-09-30T22:35:19.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.990+0000 7fb7179e2700 1 -- 172.21.15.16:0/3214051675 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7000094a0 con 0x7fb71009e930 2023-09-30T22:35:19.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.990+0000 7fb705ffb700 1 -- 172.21.15.16:0/3214051675 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb700003a60 con 0x7fb71009e930 2023-09-30T22:35:19.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.991+0000 7fb7179e2700 1 -- 172.21.15.16:0/3214051675 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7100a0190 con 0x7fb71009e930 2023-09-30T22:35:19.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.991+0000 7fb705ffb700 1 -- 172.21.15.16:0/3214051675 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb700003bc0 con 0x7fb71009e930 2023-09-30T22:35:19.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.992+0000 7fb705ffb700 1 -- 172.21.15.16:0/3214051675 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fb700021070 con 0x7fb71009e930 2023-09-30T22:35:19.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.992+0000 7fb7179e2700 1 -- 172.21.15.16:0/3214051675 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7101a6f90 con 0x7fb71009e930 2023-09-30T22:35:19.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:19.997+0000 7fb705ffb700 1 -- 172.21.15.16:0/3214051675 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb700015080 con 0x7fb71009e930 2023-09-30T22:35:20.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.202+0000 7fb7179e2700 1 -- 172.21.15.16:0/3214051675 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb710074ea0 con 0x7fb71009e930 2023-09-30T22:35:20.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.202+0000 7fb705ffb700 1 -- 172.21.15.16:0/3214051675 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb70002b020 con 0x7fb71009e930 2023-09-30T22:35:20.203 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:35:20.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.204+0000 7fb7179e2700 1 -- 172.21.15.16:0/3214051675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb71009e930 msgr2=0x7fb7101c6220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:20.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.204+0000 7fb7179e2700 1 --2- 172.21.15.16:0/3214051675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb71009e930 0x7fb7101c6220 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fb700009fd0 tx=0x7fb700004750 comp rx=0 tx=0).stop 2023-09-30T22:35:20.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.204+0000 7fb7179e2700 1 -- 172.21.15.16:0/3214051675 shutdown_connections 2023-09-30T22:35:20.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.204+0000 7fb7179e2700 1 --2- 172.21.15.16:0/3214051675 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb71009f650 0x7fb7101c27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:20.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.204+0000 7fb7179e2700 1 --2- 172.21.15.16:0/3214051675 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb71009e930 0x7fb7101c6220 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:20.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.204+0000 7fb7179e2700 1 --2- 172.21.15.16:0/3214051675 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb71009f240 0x7fb7101c6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:20.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.204+0000 7fb7179e2700 1 -- 172.21.15.16:0/3214051675 >> 172.21.15.16:0/3214051675 conn(0x7fb7100a4210 msgr2=0x7fb7100a27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:20.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.204+0000 7fb7179e2700 1 -- 172.21.15.16:0/3214051675 shutdown_connections 2023-09-30T22:35:20.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.204+0000 7fb7179e2700 1 -- 172.21.15.16:0/3214051675 wait complete. 2023-09-30T22:35:20.218 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:35:20.701 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:35:20.746 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:35:21.149 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:35:21.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.850+0000 7f34d6fed700 1 Processor -- start 2023-09-30T22:35:21.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.850+0000 7f34d6fed700 1 -- start start 2023-09-30T22:35:21.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.851+0000 7f34d6fed700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f34d009d6f0 0x7f34d009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:21.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.851+0000 7f34d6fed700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f34d009e410 0x7f34d0134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:21.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.851+0000 7f34d6fed700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34d009b120 con 0x7f34d009e000 2023-09-30T22:35:21.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.851+0000 7f34d6fed700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34d006b460 con 0x7f34d009e410 2023-09-30T22:35:21.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.851+0000 7f34d6fed700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34d009a7e0 con 0x7f34d009d6f0 2023-09-30T22:35:21.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.851+0000 7f34d558a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f34d009e410 0x7f34d0134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:21.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.851+0000 7f34d4d89700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f34d009d6f0 0x7f34d009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:21.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.851+0000 7f34d558a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f34d009e410 0x7f34d0134100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55412/0 (socket says 172.21.15.16:55412) 2023-09-30T22:35:21.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.851+0000 7f34cffff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f34d009e000 0x7f34d01339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33524/0 (socket says 172.21.15.16:33524) 2023-09-30T22:35:21.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.851+0000 7f34d558a700 1 -- 172.21.15.16:0/2165236797 learned_addr learned my addr 172.21.15.16:0/2165236797 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:21.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.851+0000 7f34d558a700 1 -- 172.21.15.16:0/2165236797 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f34d009d6f0 msgr2=0x7f34d009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:21.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.851+0000 7f34d558a700 1 --2- 172.21.15.16:0/2165236797 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f34d009d6f0 0x7f34d009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:21.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.851+0000 7f34d558a700 1 -- 172.21.15.16:0/2165236797 >> v1:172.21.15.16:6789/0 conn(0x7f34d009e000 legacy=0x7f34d01339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:21.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.851+0000 7f34d558a700 1 -- 172.21.15.16:0/2165236797 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34d009a500 con 0x7f34d009e410 2023-09-30T22:35:21.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.851+0000 7f34cf7fe700 1 -- 172.21.15.16:0/2165236797 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3749815365 0 0) 0x7f34d009b120 con 0x7f34d009e000 2023-09-30T22:35:21.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.851+0000 7f34d4d89700 1 --2- 172.21.15.16:0/2165236797 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f34d009d6f0 0x7f34d009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:21.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.851+0000 7f34d558a700 1 --2- 172.21.15.16:0/2165236797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34d009e410 0x7f34d0134100 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f34c8009e50 tx=0x7f34c800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bb6814306e963c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:21.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.852+0000 7f34cf7fe700 1 -- 172.21.15.16:0/2165236797 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c800e070 con 0x7f34d009e410 2023-09-30T22:35:21.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.852+0000 7f34cf7fe700 1 -- 172.21.15.16:0/2165236797 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34c800b910 con 0x7f34d009e410 2023-09-30T22:35:21.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.852+0000 7f34cf7fe700 1 -- 172.21.15.16:0/2165236797 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c800bcc0 con 0x7f34d009e410 2023-09-30T22:35:21.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.852+0000 7f34d6fed700 1 -- 172.21.15.16:0/2165236797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34d009e410 msgr2=0x7f34d0134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:21.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.852+0000 7f34d6fed700 1 --2- 172.21.15.16:0/2165236797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34d009e410 0x7f34d0134100 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f34c8009e50 tx=0x7f34c800b330 comp rx=0 tx=0).stop 2023-09-30T22:35:21.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.852+0000 7f34d6fed700 1 -- 172.21.15.16:0/2165236797 shutdown_connections 2023-09-30T22:35:21.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.852+0000 7f34d6fed700 1 --2- 172.21.15.16:0/2165236797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34d009e410 0x7f34d0134100 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:21.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.852+0000 7f34d6fed700 1 --2- 172.21.15.16:0/2165236797 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f34d009d6f0 0x7f34d009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:21.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.852+0000 7f34d6fed700 1 -- 172.21.15.16:0/2165236797 >> 172.21.15.16:0/2165236797 conn(0x7f34d0066f50 msgr2=0x7f34d009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:21.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.852+0000 7f34d6fed700 1 -- 172.21.15.16:0/2165236797 shutdown_connections 2023-09-30T22:35:21.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.852+0000 7f34d6fed700 1 -- 172.21.15.16:0/2165236797 wait complete. 2023-09-30T22:35:21.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.853+0000 7f34d6fed700 1 Processor -- start 2023-09-30T22:35:21.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.853+0000 7f34d6fed700 1 -- start start 2023-09-30T22:35:21.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.854+0000 7f34d6fed700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f34d009d6f0 0x7f34d01c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:21.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.854+0000 7f34d6fed700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f34d009e000 0x7f34d01c8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:21.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.854+0000 7f34d6fed700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34d009e410 0x7f34d01c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:21.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.854+0000 7f34d6fed700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d01a4c80 con 0x7f34d009e410 2023-09-30T22:35:21.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.854+0000 7f34d6fed700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34d01a65d0 con 0x7f34d009e000 2023-09-30T22:35:21.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.854+0000 7f34d6fed700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f34d01a63e0 con 0x7f34d009d6f0 2023-09-30T22:35:21.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.854+0000 7f34cffff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f34d009e000 0x7f34d01c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:21.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.854+0000 7f34d558a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34d009e410 0x7f34d01c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:21.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.854+0000 7f34cffff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f34d009e000 0x7f34d01c8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:52866/0 (socket says 172.21.15.16:52866) 2023-09-30T22:35:21.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.854+0000 7f34cffff700 1 -- 172.21.15.16:0/101473433 learned_addr learned my addr 172.21.15.16:0/101473433 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:21.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.854+0000 7f34d4d89700 1 --2- 172.21.15.16:0/101473433 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f34d009d6f0 0x7f34d01c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:21.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.854+0000 7f34cffff700 1 -- 172.21.15.16:0/101473433 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f34d009d6f0 msgr2=0x7f34d01c8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:21.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.854+0000 7f34cffff700 1 --2- 172.21.15.16:0/101473433 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f34d009d6f0 0x7f34d01c8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:21.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.854+0000 7f34cffff700 1 -- 172.21.15.16:0/101473433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34d009e410 msgr2=0x7f34d01c8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:21.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.854+0000 7f34cffff700 1 --2- 172.21.15.16:0/101473433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34d009e410 0x7f34d01c8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:21.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.854+0000 7f34cffff700 1 -- 172.21.15.16:0/101473433 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34c00094a0 con 0x7f34d009e000 2023-09-30T22:35:21.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.855+0000 7f34cffff700 1 --2- 172.21.15.16:0/101473433 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f34d009e000 0x7f34d01c8580 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f34b8001eb0 tx=0x7f34b800c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:21.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.855+0000 7f34cd7fa700 1 -- 172.21.15.16:0/101473433 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b8004c80 con 0x7f34d009e000 2023-09-30T22:35:21.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.855+0000 7f34cd7fa700 1 -- 172.21.15.16:0/101473433 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34b8004de0 con 0x7f34d009e000 2023-09-30T22:35:21.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.855+0000 7f34d6fed700 1 -- 172.21.15.16:0/101473433 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34c8009b30 con 0x7f34d009e000 2023-09-30T22:35:21.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.855+0000 7f34cd7fa700 1 -- 172.21.15.16:0/101473433 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b8007310 con 0x7f34d009e000 2023-09-30T22:35:21.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.855+0000 7f34d6fed700 1 -- 172.21.15.16:0/101473433 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34d01a8010 con 0x7f34d009e000 2023-09-30T22:35:21.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.856+0000 7f34cd7fa700 1 -- 172.21.15.16:0/101473433 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f34b8007470 con 0x7f34d009e000 2023-09-30T22:35:21.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.856+0000 7f34cd7fa700 1 -- 172.21.15.16:0/101473433 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f34b803f240 con 0x7f34d009e000 2023-09-30T22:35:21.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.856+0000 7f34d6fed700 1 -- 172.21.15.16:0/101473433 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34d01a7cb0 con 0x7f34d009e000 2023-09-30T22:35:21.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:20.861+0000 7f34cd7fa700 1 -- 172.21.15.16:0/101473433 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f34d01a7cb0 con 0x7f34d009e000 2023-09-30T22:35:21.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.067+0000 7f34d6fed700 1 -- 172.21.15.16:0/101473433 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f34d01a7cb0 con 0x7f34d009e000 2023-09-30T22:35:21.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.068+0000 7f34cd7fa700 1 -- 172.21.15.16:0/101473433 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f34d01a7cb0 con 0x7f34d009e000 2023-09-30T22:35:21.161 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:35:21.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.070+0000 7f34d6fed700 1 -- 172.21.15.16:0/101473433 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f34d009e000 msgr2=0x7f34d01c8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:21.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.070+0000 7f34d6fed700 1 --2- 172.21.15.16:0/101473433 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f34d009e000 0x7f34d01c8580 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f34b8001eb0 tx=0x7f34b800c800 comp rx=0 tx=0).stop 2023-09-30T22:35:21.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.070+0000 7f34d6fed700 1 -- 172.21.15.16:0/101473433 shutdown_connections 2023-09-30T22:35:21.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.070+0000 7f34d6fed700 1 --2- 172.21.15.16:0/101473433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34d009e410 0x7f34d01c8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:21.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.070+0000 7f34d6fed700 1 --2- 172.21.15.16:0/101473433 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f34d009d6f0 0x7f34d01c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:21.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.070+0000 7f34d6fed700 1 --2- 172.21.15.16:0/101473433 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f34d009e000 0x7f34d01c8580 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:21.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.070+0000 7f34d6fed700 1 -- 172.21.15.16:0/101473433 >> 172.21.15.16:0/101473433 conn(0x7f34d0066f50 msgr2=0x7f34d00a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:21.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.070+0000 7f34d6fed700 1 -- 172.21.15.16:0/101473433 shutdown_connections 2023-09-30T22:35:21.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.070+0000 7f34d6fed700 1 -- 172.21.15.16:0/101473433 wait complete. 2023-09-30T22:35:21.163 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:35:21.219 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:35:21.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.345+0000 7f9ade810700 1 Processor -- start 2023-09-30T22:35:21.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.345+0000 7f2110a71700 1 Processor -- start 2023-09-30T22:35:21.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.345+0000 7f9ade810700 1 -- start start 2023-09-30T22:35:21.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.345+0000 7f2110a71700 1 -- start start 2023-09-30T22:35:21.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.345+0000 7f9ade810700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9ad80a0a30 0x7f9ad80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:21.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.345+0000 7f9ade810700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9ad809d630 0x7f9ad80a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:21.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.345+0000 7f9ade810700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ad80a2a70 con 0x7f9ad8133930 2023-09-30T22:35:21.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.345+0000 7f9ade810700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ad80a01d0 con 0x7f9ad80a0a30 2023-09-30T22:35:21.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.345+0000 7f9ade810700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad809e2b0 con 0x7f9ad809d630 2023-09-30T22:35:21.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.345+0000 7f9ad77fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9ad809d630 0x7f9ad80a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:21.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.346+0000 7f9ad7fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9ad80a0a30 0x7f9ad80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:21.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.346+0000 7f9ad77fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9ad809d630 0x7f9ad80a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:52878/0 (socket says 172.21.15.16:52878) 2023-09-30T22:35:21.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.346+0000 7f9ad7fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9ad80a0a30 0x7f9ad80a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55432/0 (socket says 172.21.15.16:55432) 2023-09-30T22:35:21.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.346+0000 7f9ad77fe700 1 -- 172.21.15.16:0/1781992409 learned_addr learned my addr 172.21.15.16:0/1781992409 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:21.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.346+0000 7f9ad77fe700 1 -- 172.21.15.16:0/1781992409 >> v2:172.21.15.16:3300/0 conn(0x7f9ad80a0a30 msgr2=0x7f9ad80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:21.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.346+0000 7f9ad77fe700 1 --2- 172.21.15.16:0/1781992409 >> v2:172.21.15.16:3300/0 conn(0x7f9ad80a0a30 0x7f9ad80a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:21.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.346+0000 7f9ad77fe700 1 -- 172.21.15.16:0/1781992409 >> v1:172.21.15.16:6789/0 conn(0x7f9ad8133930 legacy=0x7f9ad8133d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:21.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.346+0000 7f2110a71700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f210c130790 0x7f210c130b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:21.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.346+0000 7f2110a71700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f210c06f640 0x7f210c06fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:21.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.346+0000 7f2110a71700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f210c06ffb0 0x7f210c1374f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:21.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.346+0000 7f2110a71700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f210c09a370 con 0x7f210c06f640 2023-09-30T22:35:21.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.346+0000 7f2110a71700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f210c09ef50 con 0x7f210c06ffb0 2023-09-30T22:35:21.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.346+0000 7f2110a71700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f210c070860 con 0x7f210c130790 2023-09-30T22:35:21.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.346+0000 7f9ad6ffd700 1 -- 172.21.15.16:0/1781992409 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 69461329 0 0) 0x7f9ad80a2a70 con 0x7f9ad8133930 2023-09-30T22:35:21.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.346+0000 7f9ad77fe700 1 -- 172.21.15.16:0/1781992409 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad809dc50 con 0x7f9ad809d630 2023-09-30T22:35:21.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.346+0000 7f210a59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f210c130790 0x7f210c130b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:21.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.346+0000 7f210a59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f210c130790 0x7f210c130b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:33080/0 (socket says 172.21.15.16:33080) 2023-09-30T22:35:21.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.346+0000 7f210a59c700 1 -- 172.21.15.16:0/3615349030 learned_addr learned my addr 172.21.15.16:0/3615349030 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:21.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.346+0000 7f9ad77fe700 1 --2- 172.21.15.16:0/1781992409 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9ad809d630 0x7f9ad80a1340 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f9acc0098f0 tx=0x7f9acc009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d88e8cb58b42fbe server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:21.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.346+0000 7f2109d9b700 1 --2- 172.21.15.16:0/3615349030 >> v2:172.21.15.16:3300/0 conn(0x7f210c06f640 0x7f210c06fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:21.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.346+0000 7f210ad9d700 1 --2- 172.21.15.16:0/3615349030 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f210c06ffb0 0x7f210c1374f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:21.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.347+0000 7f210a59c700 1 -- 172.21.15.16:0/3615349030 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f210c06ffb0 msgr2=0x7f210c1374f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:21.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.347+0000 7f210a59c700 1 --2- 172.21.15.16:0/3615349030 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f210c06ffb0 0x7f210c1374f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:21.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.347+0000 7f210a59c700 1 -- 172.21.15.16:0/3615349030 >> v2:172.21.15.16:3300/0 conn(0x7f210c06f640 msgr2=0x7f210c06fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:21.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.347+0000 7f210a59c700 1 --2- 172.21.15.16:0/3615349030 >> v2:172.21.15.16:3300/0 conn(0x7f210c06f640 0x7f210c06fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:21.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.347+0000 7f210a59c700 1 -- 172.21.15.16:0/3615349030 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f210c070450 con 0x7f210c130790 2023-09-30T22:35:21.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.347+0000 7f210ad9d700 1 --2- 172.21.15.16:0/3615349030 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f210c06ffb0 0x7f210c1374f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:21.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.347+0000 7f2109d9b700 1 --2- 172.21.15.16:0/3615349030 >> v2:172.21.15.16:3300/0 conn(0x7f210c06f640 0x7f210c06fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:21.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.347+0000 7f9ad6ffd700 1 -- 172.21.15.16:0/1781992409 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9acc011830 con 0x7f9ad809d630 2023-09-30T22:35:21.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.347+0000 7f9ad6ffd700 1 -- 172.21.15.16:0/1781992409 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9acc011990 con 0x7f9ad809d630 2023-09-30T22:35:21.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.347+0000 7f9ad6ffd700 1 -- 172.21.15.16:0/1781992409 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9acc011d20 con 0x7f9ad809d630 2023-09-30T22:35:21.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.347+0000 7f210a59c700 1 --2- 172.21.15.16:0/3615349030 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f210c130790 0x7f210c130b60 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f20fc0098f0 tx=0x7f20fc009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a3b33ce60a0c6c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:21.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.347+0000 7f210959a700 1 -- 172.21.15.16:0/3615349030 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20fc011830 con 0x7f210c130790 2023-09-30T22:35:21.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.347+0000 7f210959a700 1 -- 172.21.15.16:0/3615349030 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20fc011990 con 0x7f210c130790 2023-09-30T22:35:21.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.347+0000 7f210959a700 1 -- 172.21.15.16:0/3615349030 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20fc011d20 con 0x7f210c130790 2023-09-30T22:35:21.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.347+0000 7f9ade810700 1 -- 172.21.15.16:0/1781992409 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9ad809d630 msgr2=0x7f9ad80a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:21.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.347+0000 7f9ade810700 1 --2- 172.21.15.16:0/1781992409 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9ad809d630 0x7f9ad80a1340 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f9acc0098f0 tx=0x7f9acc009c00 comp rx=0 tx=0).stop 2023-09-30T22:35:21.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.347+0000 7f9ade810700 1 -- 172.21.15.16:0/1781992409 shutdown_connections 2023-09-30T22:35:21.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.347+0000 7f9ade810700 1 --2- 172.21.15.16:0/1781992409 >> v2:172.21.15.16:3300/0 conn(0x7f9ad80a0a30 0x7f9ad80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:21.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.347+0000 7f9ade810700 1 --2- 172.21.15.16:0/1781992409 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9ad809d630 0x7f9ad80a1340 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:21.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.347+0000 7f9ade810700 1 -- 172.21.15.16:0/1781992409 >> 172.21.15.16:0/1781992409 conn(0x7f9ad80a3050 msgr2=0x7f9ad80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:21.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.348+0000 7f9ade810700 1 -- 172.21.15.16:0/1781992409 shutdown_connections 2023-09-30T22:35:21.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.348+0000 7f9ade810700 1 -- 172.21.15.16:0/1781992409 wait complete. 2023-09-30T22:35:21.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.348+0000 7f2110a71700 1 -- 172.21.15.16:0/3615349030 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f210c130790 msgr2=0x7f210c130b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:21.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.348+0000 7f2110a71700 1 --2- 172.21.15.16:0/3615349030 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f210c130790 0x7f210c130b60 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f20fc0098f0 tx=0x7f20fc009c00 comp rx=0 tx=0).stop 2023-09-30T22:35:21.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.348+0000 7f2110a71700 1 -- 172.21.15.16:0/3615349030 shutdown_connections 2023-09-30T22:35:21.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.348+0000 7f2110a71700 1 --2- 172.21.15.16:0/3615349030 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f210c06ffb0 0x7f210c1374f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:21.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.348+0000 7f2110a71700 1 --2- 172.21.15.16:0/3615349030 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f210c130790 0x7f210c130b60 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:21.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.348+0000 7f2110a71700 1 --2- 172.21.15.16:0/3615349030 >> v2:172.21.15.16:3300/0 conn(0x7f210c06f640 0x7f210c06fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:21.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.348+0000 7f2110a71700 1 -- 172.21.15.16:0/3615349030 >> 172.21.15.16:0/3615349030 conn(0x7f210c0a2aa0 msgr2=0x7f210c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:21.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.348+0000 7f2110a71700 1 -- 172.21.15.16:0/3615349030 shutdown_connections 2023-09-30T22:35:21.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.348+0000 7f2110a71700 1 -- 172.21.15.16:0/3615349030 wait complete. 2023-09-30T22:35:21.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.349+0000 7f9ade810700 1 Processor -- start 2023-09-30T22:35:21.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.349+0000 7f9ade810700 1 -- start start 2023-09-30T22:35:21.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.349+0000 7f9ade810700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ad809d630 0x7f9ad811f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:21.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.349+0000 7f9ade810700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9ad80a0a30 0x7f9ad811f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:21.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.349+0000 7f9ade810700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9ad8133930 0x7f9ad81c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:21.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.349+0000 7f9ade810700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad8121ea0 con 0x7f9ad809d630 2023-09-30T22:35:21.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.349+0000 7f9ade810700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad811eed0 con 0x7f9ad80a0a30 2023-09-30T22:35:21.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.349+0000 7f9ade810700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad8121210 con 0x7f9ad8133930 2023-09-30T22:35:21.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.349+0000 7f9ad77fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9ad80a0a30 0x7f9ad811f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:21.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.349+0000 7f9ad7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ad809d630 0x7f9ad811f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:21.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.349+0000 7f9adcdad700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9ad8133930 0x7f9ad81c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:21.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.349+0000 7f9ad7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ad809d630 0x7f9ad811f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55452/0 (socket says 172.21.15.16:55452) 2023-09-30T22:35:21.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.349+0000 7f9ad77fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9ad80a0a30 0x7f9ad811f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:52896/0 (socket says 172.21.15.16:52896) 2023-09-30T22:35:21.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.349+0000 7f9ad7fff700 1 -- 172.21.15.16:0/1756462300 learned_addr learned my addr 172.21.15.16:0/1756462300 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:21.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.350+0000 7f2110a71700 1 Processor -- start 2023-09-30T22:35:21.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.350+0000 7f9ad7fff700 1 -- 172.21.15.16:0/1756462300 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9ad8133930 msgr2=0x7f9ad81c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:21.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.350+0000 7f9ad7fff700 1 --2- 172.21.15.16:0/1756462300 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9ad8133930 0x7f9ad81c8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:21.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.350+0000 7f9ad7fff700 1 -- 172.21.15.16:0/1756462300 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9ad80a0a30 msgr2=0x7f9ad811f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:21.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.350+0000 7f9ad7fff700 1 --2- 172.21.15.16:0/1756462300 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9ad80a0a30 0x7f9ad811f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:21.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.350+0000 7f9ad7fff700 1 -- 172.21.15.16:0/1756462300 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9acc009550 con 0x7f9ad809d630 2023-09-30T22:35:21.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.350+0000 7f9ad77fe700 1 --2- 172.21.15.16:0/1756462300 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9ad80a0a30 0x7f9ad811f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:21.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.350+0000 7f9adcdad700 1 --2- 172.21.15.16:0/1756462300 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9ad8133930 0x7f9ad81c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:21.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.350+0000 7f9ad7fff700 1 --2- 172.21.15.16:0/1756462300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ad809d630 0x7f9ad811f400 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f9ac800e9b0 tx=0x7f9ac800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:21.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.350+0000 7f9ad4ff9700 1 -- 172.21.15.16:0/1756462300 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac8009710 con 0x7f9ad809d630 2023-09-30T22:35:21.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.350+0000 7f9ade810700 1 -- 172.21.15.16:0/1756462300 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ad8122160 con 0x7f9ad809d630 2023-09-30T22:35:21.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.350+0000 7f9ad4ff9700 1 -- 172.21.15.16:0/1756462300 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ac800ee50 con 0x7f9ad809d630 2023-09-30T22:35:21.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.350+0000 7f9ad4ff9700 1 -- 172.21.15.16:0/1756462300 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac800ca50 con 0x7f9ad809d630 2023-09-30T22:35:21.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.350+0000 7f2110a71700 1 -- start start 2023-09-30T22:35:21.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.350+0000 7f2110a71700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f210c06f640 0x7f210c139c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:21.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.350+0000 7f2110a71700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f210c06ffb0 0x7f210c13a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:21.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.351+0000 7f2110a71700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f210c130790 0x7f210c13a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:21.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.351+0000 7f2110a71700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210c13d590 con 0x7f210c06ffb0 2023-09-30T22:35:21.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.351+0000 7f2110a71700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f210c13dc90 con 0x7f210c06f640 2023-09-30T22:35:21.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.351+0000 7f2110a71700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f210c13d790 con 0x7f210c130790 2023-09-30T22:35:21.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.351+0000 7f2109d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f210c06ffb0 0x7f210c13a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:21.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.351+0000 7f2109d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f210c06ffb0 0x7f210c13a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55460/0 (socket says 172.21.15.16:55460) 2023-09-30T22:35:21.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.351+0000 7f2109d9b700 1 -- 172.21.15.16:0/845753979 learned_addr learned my addr 172.21.15.16:0/845753979 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:21.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.351+0000 7f210a59c700 1 --2- 172.21.15.16:0/845753979 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f210c06f640 0x7f210c139c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:21.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.351+0000 7f2109d9b700 1 -- 172.21.15.16:0/845753979 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f210c130790 msgr2=0x7f210c13a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:21.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.351+0000 7f2109d9b700 1 --2- 172.21.15.16:0/845753979 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f210c130790 0x7f210c13a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:21.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.351+0000 7f2109d9b700 1 -- 172.21.15.16:0/845753979 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f210c06f640 msgr2=0x7f210c139c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:21.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.351+0000 7f2109d9b700 1 --2- 172.21.15.16:0/845753979 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f210c06f640 0x7f210c139c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:21.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.351+0000 7f2109d9b700 1 -- 172.21.15.16:0/845753979 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f40094a0 con 0x7f210c06ffb0 2023-09-30T22:35:21.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.351+0000 7f2109d9b700 1 --2- 172.21.15.16:0/845753979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f210c06ffb0 0x7f210c13a1b0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f20f400eb30 tx=0x7f20f400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:21.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.350+0000 7f9ade810700 1 -- 172.21.15.16:0/1756462300 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ad8122440 con 0x7f9ad809d630 2023-09-30T22:35:21.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.351+0000 7f9ad4ff9700 1 -- 172.21.15.16:0/1756462300 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9ac800cbb0 con 0x7f9ad809d630 2023-09-30T22:35:21.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.352+0000 7f9ad4ff9700 1 -- 172.21.15.16:0/1756462300 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f9ac803f710 con 0x7f9ad809d630 2023-09-30T22:35:21.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.352+0000 7f2102ffd700 1 -- 172.21.15.16:0/845753979 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f400c850 con 0x7f210c06ffb0 2023-09-30T22:35:21.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.352+0000 7f2102ffd700 1 -- 172.21.15.16:0/845753979 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20f400c9b0 con 0x7f210c06ffb0 2023-09-30T22:35:21.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.352+0000 7f9ade810700 1 -- 172.21.15.16:0/1756462300 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ad80747f0 con 0x7f9ad809d630 2023-09-30T22:35:21.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.352+0000 7f2102ffd700 1 -- 172.21.15.16:0/845753979 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f400cd40 con 0x7f210c06ffb0 2023-09-30T22:35:21.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.352+0000 7f2110a71700 1 -- 172.21.15.16:0/845753979 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2104009a70 con 0x7f210c06ffb0 2023-09-30T22:35:21.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.352+0000 7f2110a71700 1 -- 172.21.15.16:0/845753979 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20fc009550 con 0x7f210c06ffb0 2023-09-30T22:35:21.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.353+0000 7f2110a71700 1 -- 172.21.15.16:0/845753979 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f210c074ea0 con 0x7f210c06ffb0 2023-09-30T22:35:21.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.357+0000 7f9ad4ff9700 1 -- 172.21.15.16:0/1756462300 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9ac8021070 con 0x7f9ad809d630 2023-09-30T22:35:21.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.357+0000 7f2102ffd700 1 -- 172.21.15.16:0/845753979 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f20f4013470 con 0x7f210c06ffb0 2023-09-30T22:35:21.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.357+0000 7f2102ffd700 1 -- 172.21.15.16:0/845753979 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f20f403ff50 con 0x7f210c06ffb0 2023-09-30T22:35:21.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.361+0000 7f2102ffd700 1 -- 172.21.15.16:0/845753979 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f20f4011760 con 0x7f210c06ffb0 2023-09-30T22:35:21.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.565+0000 7f9ade810700 1 -- 172.21.15.16:0/1756462300 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f9ad81333d0 con 0x7f9ad809d630 2023-09-30T22:35:21.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.566+0000 7f9ad4ff9700 1 -- 172.21.15.16:0/1756462300 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9ac80232f0 con 0x7f9ad809d630 2023-09-30T22:35:21.566 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:35:21.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.568+0000 7f9ade810700 1 -- 172.21.15.16:0/1756462300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ad809d630 msgr2=0x7f9ad811f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:21.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.568+0000 7f9ade810700 1 --2- 172.21.15.16:0/1756462300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ad809d630 0x7f9ad811f400 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f9ac800e9b0 tx=0x7f9ac800ecc0 comp rx=0 tx=0).stop 2023-09-30T22:35:21.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.568+0000 7f9ade810700 1 -- 172.21.15.16:0/1756462300 shutdown_connections 2023-09-30T22:35:21.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.568+0000 7f9ade810700 1 --2- 172.21.15.16:0/1756462300 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9ad8133930 0x7f9ad81c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:21.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.568+0000 7f9ade810700 1 --2- 172.21.15.16:0/1756462300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ad809d630 0x7f9ad811f400 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:21.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.568+0000 7f9ade810700 1 --2- 172.21.15.16:0/1756462300 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9ad80a0a30 0x7f9ad811f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:21.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.568+0000 7f9ade810700 1 -- 172.21.15.16:0/1756462300 >> 172.21.15.16:0/1756462300 conn(0x7f9ad80a3050 msgr2=0x7f9ad81bbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:21.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.568+0000 7f9ade810700 1 -- 172.21.15.16:0/1756462300 shutdown_connections 2023-09-30T22:35:21.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.568+0000 7f9ade810700 1 -- 172.21.15.16:0/1756462300 wait complete. 2023-09-30T22:35:21.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.569+0000 7f2110a71700 1 -- 172.21.15.16:0/845753979 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f210c072210 con 0x7f210c06ffb0 2023-09-30T22:35:21.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.569+0000 7f2102ffd700 1 -- 172.21.15.16:0/845753979 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f20f4021020 con 0x7f210c06ffb0 2023-09-30T22:35:21.572 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:35:21.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.573+0000 7f2110a71700 1 -- 172.21.15.16:0/845753979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f210c06ffb0 msgr2=0x7f210c13a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:21.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.573+0000 7f2110a71700 1 --2- 172.21.15.16:0/845753979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f210c06ffb0 0x7f210c13a1b0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f20f400eb30 tx=0x7f20f400ee40 comp rx=0 tx=0).stop 2023-09-30T22:35:21.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.573+0000 7f2110a71700 1 -- 172.21.15.16:0/845753979 shutdown_connections 2023-09-30T22:35:21.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.573+0000 7f2110a71700 1 --2- 172.21.15.16:0/845753979 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f210c06f640 0x7f210c139c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:21.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.573+0000 7f2110a71700 1 --2- 172.21.15.16:0/845753979 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f210c130790 0x7f210c13a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:21.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.573+0000 7f2110a71700 1 --2- 172.21.15.16:0/845753979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f210c06ffb0 0x7f210c13a1b0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:21.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.573+0000 7f2110a71700 1 -- 172.21.15.16:0/845753979 >> 172.21.15.16:0/845753979 conn(0x7f210c0a2aa0 msgr2=0x7f210c0a0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:21.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.573+0000 7f2110a71700 1 -- 172.21.15.16:0/845753979 shutdown_connections 2023-09-30T22:35:21.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.573+0000 7f2110a71700 1 -- 172.21.15.16:0/845753979 wait complete. 2023-09-30T22:35:21.587 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:35:21.590 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:35:21.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.685+0000 7f916f625700 1 Processor -- start 2023-09-30T22:35:21.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.685+0000 7f916f625700 1 -- start start 2023-09-30T22:35:21.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.686+0000 7f916f625700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f91681337f0 0x7f9168138880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:21.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.686+0000 7f916f625700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9168138dc0 0x7f916813b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:21.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.686+0000 7f916f625700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f916806b990 con 0x7f9168133420 2023-09-30T22:35:21.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.686+0000 7f916f625700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f916809b0e0 con 0x7f91681337f0 2023-09-30T22:35:21.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.686+0000 7f916f625700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9168133c20 con 0x7f9168138dc0 2023-09-30T22:35:21.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.686+0000 7f916d3c1700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9168133420 0x7f9168133e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33544/0 (socket says 172.21.15.16:33544) 2023-09-30T22:35:21.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.686+0000 7f916d3c1700 1 -- 172.21.15.16:0/2749240096 learned_addr learned my addr 172.21.15.16:0/2749240096 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:21.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.686+0000 7f916cbc0700 1 --2- 172.21.15.16:0/2749240096 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f91681337f0 0x7f9168138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:21.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.686+0000 7f916dbc2700 1 --2- 172.21.15.16:0/2749240096 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9168138dc0 0x7f916813b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:21.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.686+0000 7f915ffff700 1 -- 172.21.15.16:0/2749240096 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3366277115 0 0) 0x7f916806b990 con 0x7f9168133420 2023-09-30T22:35:21.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.686+0000 7f915ffff700 1 -- 172.21.15.16:0/2749240096 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91540035b0 con 0x7f9168133420 2023-09-30T22:35:21.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.687+0000 7f916dbc2700 1 -- 172.21.15.16:0/2749240096 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f91681337f0 msgr2=0x7f9168138880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:21.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.687+0000 7f916dbc2700 1 --2- 172.21.15.16:0/2749240096 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f91681337f0 0x7f9168138880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:21.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.687+0000 7f916dbc2700 1 -- 172.21.15.16:0/2749240096 >> v1:172.21.15.16:6789/0 conn(0x7f9168133420 legacy=0x7f9168133e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:21.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.687+0000 7f916dbc2700 1 -- 172.21.15.16:0/2749240096 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91680a0590 con 0x7f9168138dc0 2023-09-30T22:35:21.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.687+0000 7f916cbc0700 1 --2- 172.21.15.16:0/2749240096 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f91681337f0 0x7f9168138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:21.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.687+0000 7f916dbc2700 1 --2- 172.21.15.16:0/2749240096 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9168138dc0 0x7f916813b1a0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f9160009870 tx=0x7f9160009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4170a8ad911f9b1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:21.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.687+0000 7f915ffff700 1 -- 172.21.15.16:0/2749240096 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9160011830 con 0x7f9168138dc0 2023-09-30T22:35:21.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.687+0000 7f915ffff700 1 -- 172.21.15.16:0/2749240096 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9160011990 con 0x7f9168138dc0 2023-09-30T22:35:21.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.687+0000 7f915ffff700 1 -- 172.21.15.16:0/2749240096 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9160011d20 con 0x7f9168138dc0 2023-09-30T22:35:21.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.687+0000 7f916f625700 1 -- 172.21.15.16:0/2749240096 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9168138dc0 msgr2=0x7f916813b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:21.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.687+0000 7f916f625700 1 --2- 172.21.15.16:0/2749240096 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9168138dc0 0x7f916813b1a0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f9160009870 tx=0x7f9160009b80 comp rx=0 tx=0).stop 2023-09-30T22:35:21.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.688+0000 7f916f625700 1 -- 172.21.15.16:0/2749240096 shutdown_connections 2023-09-30T22:35:21.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.688+0000 7f916f625700 1 --2- 172.21.15.16:0/2749240096 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9168138dc0 0x7f916813b1a0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:21.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.688+0000 7f916f625700 1 --2- 172.21.15.16:0/2749240096 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f91681337f0 0x7f9168138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:21.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.688+0000 7f916f625700 1 -- 172.21.15.16:0/2749240096 >> 172.21.15.16:0/2749240096 conn(0x7f91680a4210 msgr2=0x7f91680a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:21.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.688+0000 7f916f625700 1 -- 172.21.15.16:0/2749240096 shutdown_connections 2023-09-30T22:35:21.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.688+0000 7f916f625700 1 -- 172.21.15.16:0/2749240096 wait complete. 2023-09-30T22:35:21.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.689+0000 7f916f625700 1 Processor -- start 2023-09-30T22:35:21.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.689+0000 7f916f625700 1 -- start start 2023-09-30T22:35:21.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.689+0000 7f916f625700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9168133420 0x7f916812f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:21.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.689+0000 7f916f625700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91681337f0 0x7f916812f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:21.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.689+0000 7f916f625700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9168138dc0 0x7f916812fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:21.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.689+0000 7f916f625700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9168130c10 con 0x7f91681337f0 2023-09-30T22:35:21.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.689+0000 7f916f625700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f916809e1c0 con 0x7f9168138dc0 2023-09-30T22:35:21.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.689+0000 7f916f625700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f916809c8d0 con 0x7f9168133420 2023-09-30T22:35:21.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.689+0000 7f916d3c1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9168133420 0x7f916812f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:21.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.689+0000 7f916dbc2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9168138dc0 0x7f916812fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:21.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.689+0000 7f916dbc2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9168138dc0 0x7f916812fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:52932/0 (socket says 172.21.15.16:52932) 2023-09-30T22:35:21.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.689+0000 7f916d3c1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9168133420 0x7f916812f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:33122/0 (socket says 172.21.15.16:33122) 2023-09-30T22:35:21.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.689+0000 7f916dbc2700 1 -- 172.21.15.16:0/3015949889 learned_addr learned my addr 172.21.15.16:0/3015949889 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:21.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.689+0000 7f916cbc0700 1 --2- 172.21.15.16:0/3015949889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91681337f0 0x7f916812f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:21.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.690+0000 7f916dbc2700 1 -- 172.21.15.16:0/3015949889 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9168133420 msgr2=0x7f916812f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:21.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.690+0000 7f916dbc2700 1 --2- 172.21.15.16:0/3015949889 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9168133420 0x7f916812f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:21.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.690+0000 7f916dbc2700 1 -- 172.21.15.16:0/3015949889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91681337f0 msgr2=0x7f916812f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:21.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.690+0000 7f916dbc2700 1 --2- 172.21.15.16:0/3015949889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91681337f0 0x7f916812f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:21.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.690+0000 7f916dbc2700 1 -- 172.21.15.16:0/3015949889 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91580094a0 con 0x7f9168138dc0 2023-09-30T22:35:21.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.690+0000 7f916d3c1700 1 --2- 172.21.15.16:0/3015949889 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9168133420 0x7f916812f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:21.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.690+0000 7f916dbc2700 1 --2- 172.21.15.16:0/3015949889 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9168138dc0 0x7f916812fd10 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f9160009bc0 tx=0x7f9160004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:21.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.690+0000 7f915dffb700 1 -- 172.21.15.16:0/3015949889 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9160004450 con 0x7f9168138dc0 2023-09-30T22:35:21.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.690+0000 7f915dffb700 1 -- 172.21.15.16:0/3015949889 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91600045b0 con 0x7f9168138dc0 2023-09-30T22:35:21.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.690+0000 7f916f625700 1 -- 172.21.15.16:0/3015949889 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91600094a0 con 0x7f9168138dc0 2023-09-30T22:35:21.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.690+0000 7f915dffb700 1 -- 172.21.15.16:0/3015949889 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91600038e0 con 0x7f9168138dc0 2023-09-30T22:35:21.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.690+0000 7f916f625700 1 -- 172.21.15.16:0/3015949889 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f916809da50 con 0x7f9168138dc0 2023-09-30T22:35:21.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.691+0000 7f915dffb700 1 -- 172.21.15.16:0/3015949889 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9160003a40 con 0x7f9168138dc0 2023-09-30T22:35:21.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.691+0000 7f915dffb700 1 -- 172.21.15.16:0/3015949889 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f9160040780 con 0x7f9168138dc0 2023-09-30T22:35:21.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.691+0000 7f916f625700 1 -- 172.21.15.16:0/3015949889 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9168134040 con 0x7f9168138dc0 2023-09-30T22:35:21.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.696+0000 7f915dffb700 1 -- 172.21.15.16:0/3015949889 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9160015080 con 0x7f9168138dc0 2023-09-30T22:35:21.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.901+0000 7f916f625700 1 -- 172.21.15.16:0/3015949889 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9168074ea0 con 0x7f9168138dc0 2023-09-30T22:35:21.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.901+0000 7f915dffb700 1 -- 172.21.15.16:0/3015949889 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f916002d3b0 con 0x7f9168138dc0 2023-09-30T22:35:21.902 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:35:21.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.903+0000 7f916f625700 1 -- 172.21.15.16:0/3015949889 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9168138dc0 msgr2=0x7f916812fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:21.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.903+0000 7f916f625700 1 --2- 172.21.15.16:0/3015949889 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9168138dc0 0x7f916812fd10 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f9160009bc0 tx=0x7f9160004750 comp rx=0 tx=0).stop 2023-09-30T22:35:21.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.903+0000 7f916f625700 1 -- 172.21.15.16:0/3015949889 shutdown_connections 2023-09-30T22:35:21.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.903+0000 7f916f625700 1 --2- 172.21.15.16:0/3015949889 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9168138dc0 0x7f916812fd10 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:21.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.903+0000 7f916f625700 1 --2- 172.21.15.16:0/3015949889 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9168133420 0x7f916812f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:21.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.903+0000 7f916f625700 1 --2- 172.21.15.16:0/3015949889 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91681337f0 0x7f916812f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:21.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.903+0000 7f916f625700 1 -- 172.21.15.16:0/3015949889 >> 172.21.15.16:0/3015949889 conn(0x7f91680a4210 msgr2=0x7f9168099c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:21.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.903+0000 7f916f625700 1 -- 172.21.15.16:0/3015949889 shutdown_connections 2023-09-30T22:35:21.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:21.903+0000 7f916f625700 1 -- 172.21.15.16:0/3015949889 wait complete. 2023-09-30T22:35:21.917 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:35:22.166 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:35:22.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.311+0000 7f71f0345700 1 Processor -- start 2023-09-30T22:35:22.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.311+0000 7f71f0345700 1 -- start start 2023-09-30T22:35:22.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.312+0000 7f71f0345700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f71e809f410 0x7f71e8136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:22.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.312+0000 7f71f0345700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f71e8136fe0 0x7f71e81393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:22.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.312+0000 7f71f0345700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71e8070940 con 0x7f71e809e930 2023-09-30T22:35:22.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.312+0000 7f71f0345700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71e806fd60 con 0x7f71e809f410 2023-09-30T22:35:22.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.312+0000 7f71f0345700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f71e806fa80 con 0x7f71e8136fe0 2023-09-30T22:35:22.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.312+0000 7f71ee8e2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f71e8136fe0 0x7f71e81393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:22.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.312+0000 7f71ed8e0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f71e809f410 0x7f71e8136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:22.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.312+0000 7f71ee8e2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f71e8136fe0 0x7f71e81393c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:33130/0 (socket says 172.21.15.16:33130) 2023-09-30T22:35:22.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.312+0000 7f71ee0e1700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f71e809e930 0x7f71e809ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33558/0 (socket says 172.21.15.16:33558) 2023-09-30T22:35:22.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.312+0000 7f71ee8e2700 1 -- 172.21.15.16:0/2456796141 learned_addr learned my addr 172.21.15.16:0/2456796141 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:22.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.312+0000 7f71ed0df700 1 -- 172.21.15.16:0/2456796141 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1342667639 0 0) 0x7f71e8070940 con 0x7f71e809e930 2023-09-30T22:35:22.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.313+0000 7f71ee8e2700 1 -- 172.21.15.16:0/2456796141 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f71e809f410 msgr2=0x7f71e8136aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:22.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.313+0000 7f71ee8e2700 1 --2- 172.21.15.16:0/2456796141 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f71e809f410 0x7f71e8136aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:22.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.313+0000 7f71ee8e2700 1 -- 172.21.15.16:0/2456796141 >> v1:172.21.15.16:6789/0 conn(0x7f71e809e930 legacy=0x7f71e809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:22.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.313+0000 7f71ee8e2700 1 -- 172.21.15.16:0/2456796141 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71e809fb60 con 0x7f71e8136fe0 2023-09-30T22:35:22.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.313+0000 7f71ed8e0700 1 --2- 172.21.15.16:0/2456796141 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f71e809f410 0x7f71e8136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:22.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.313+0000 7f71ee8e2700 1 --2- 172.21.15.16:0/2456796141 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f71e8136fe0 0x7f71e81393c0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f71e0009e80 tx=0x7f71e000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc0b4d3a23cc107d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:22.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.313+0000 7f71ed0df700 1 -- 172.21.15.16:0/2456796141 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e000e070 con 0x7f71e8136fe0 2023-09-30T22:35:22.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.313+0000 7f71ed0df700 1 -- 172.21.15.16:0/2456796141 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71e000b910 con 0x7f71e8136fe0 2023-09-30T22:35:22.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.313+0000 7f71ed0df700 1 -- 172.21.15.16:0/2456796141 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e000bcc0 con 0x7f71e8136fe0 2023-09-30T22:35:22.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.313+0000 7f71f0345700 1 -- 172.21.15.16:0/2456796141 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f71e8136fe0 msgr2=0x7f71e81393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:22.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.313+0000 7f71f0345700 1 --2- 172.21.15.16:0/2456796141 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f71e8136fe0 0x7f71e81393c0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f71e0009e80 tx=0x7f71e000b330 comp rx=0 tx=0).stop 2023-09-30T22:35:22.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.314+0000 7f71f0345700 1 -- 172.21.15.16:0/2456796141 shutdown_connections 2023-09-30T22:35:22.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.314+0000 7f71f0345700 1 --2- 172.21.15.16:0/2456796141 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f71e8136fe0 0x7f71e81393c0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:22.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.314+0000 7f71f0345700 1 --2- 172.21.15.16:0/2456796141 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f71e809f410 0x7f71e8136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:22.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.314+0000 7f71f0345700 1 -- 172.21.15.16:0/2456796141 >> 172.21.15.16:0/2456796141 conn(0x7f71e80a4210 msgr2=0x7f71e80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:22.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.314+0000 7f71f0345700 1 -- 172.21.15.16:0/2456796141 shutdown_connections 2023-09-30T22:35:22.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.314+0000 7f71f0345700 1 -- 172.21.15.16:0/2456796141 wait complete. 2023-09-30T22:35:22.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.315+0000 7f71f0345700 1 Processor -- start 2023-09-30T22:35:22.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.315+0000 7f71f0345700 1 -- start start 2023-09-30T22:35:22.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.315+0000 7f71f0345700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f71e809e930 0x7f71e811e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:22.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.315+0000 7f71f0345700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71e809f410 0x7f71e811e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:22.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.315+0000 7f71f0345700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f71e8136fe0 0x7f71e811eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:22.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.315+0000 7f71f0345700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e8121f90 con 0x7f71e809f410 2023-09-30T22:35:22.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.315+0000 7f71f0345700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71e81206a0 con 0x7f71e809e930 2023-09-30T22:35:22.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.315+0000 7f71f0345700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f71e8071d20 con 0x7f71e8136fe0 2023-09-30T22:35:22.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.315+0000 7f71ed8e0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71e809f410 0x7f71e811e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:22.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.315+0000 7f71ee8e2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f71e8136fe0 0x7f71e811eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:22.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.315+0000 7f71ed8e0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71e809f410 0x7f71e811e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55480/0 (socket says 172.21.15.16:55480) 2023-09-30T22:35:22.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.315+0000 7f71ee8e2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f71e8136fe0 0x7f71e811eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:33138/0 (socket says 172.21.15.16:33138) 2023-09-30T22:35:22.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.315+0000 7f71ed8e0700 1 -- 172.21.15.16:0/2526747822 learned_addr learned my addr 172.21.15.16:0/2526747822 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:22.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.315+0000 7f71ee0e1700 1 --2- 172.21.15.16:0/2526747822 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f71e809e930 0x7f71e811e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:22.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.316+0000 7f71ee8e2700 1 -- 172.21.15.16:0/2526747822 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f71e809e930 msgr2=0x7f71e811e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:22.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.316+0000 7f71ee8e2700 1 --2- 172.21.15.16:0/2526747822 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f71e809e930 0x7f71e811e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:22.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.316+0000 7f71ee8e2700 1 -- 172.21.15.16:0/2526747822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71e809f410 msgr2=0x7f71e811e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:22.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.316+0000 7f71ee8e2700 1 --2- 172.21.15.16:0/2526747822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71e809f410 0x7f71e811e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:22.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.316+0000 7f71ee8e2700 1 -- 172.21.15.16:0/2526747822 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71d80094a0 con 0x7f71e8136fe0 2023-09-30T22:35:22.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.316+0000 7f71ed8e0700 1 --2- 172.21.15.16:0/2526747822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71e809f410 0x7f71e811e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:22.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.316+0000 7f71ee8e2700 1 --2- 172.21.15.16:0/2526747822 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f71e8136fe0 0x7f71e811eef0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f71e0015000 tx=0x7f71e0004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:22.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.316+0000 7f71deffd700 1 -- 172.21.15.16:0/2526747822 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e0021070 con 0x7f71e8136fe0 2023-09-30T22:35:22.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.316+0000 7f71deffd700 1 -- 172.21.15.16:0/2526747822 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71e00092e0 con 0x7f71e8136fe0 2023-09-30T22:35:22.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.316+0000 7f71f0345700 1 -- 172.21.15.16:0/2526747822 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71e0009b30 con 0x7f71e8136fe0 2023-09-30T22:35:22.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.316+0000 7f71deffd700 1 -- 172.21.15.16:0/2526747822 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e001cad0 con 0x7f71e8136fe0 2023-09-30T22:35:22.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.316+0000 7f71f0345700 1 -- 172.21.15.16:0/2526747822 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71e81219f0 con 0x7f71e8136fe0 2023-09-30T22:35:22.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.317+0000 7f71deffd700 1 -- 172.21.15.16:0/2526747822 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f71e001cc30 con 0x7f71e8136fe0 2023-09-30T22:35:22.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.317+0000 7f71deffd700 1 -- 172.21.15.16:0/2526747822 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f71e0044570 con 0x7f71e8136fe0 2023-09-30T22:35:22.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.317+0000 7f71f0345700 1 -- 172.21.15.16:0/2526747822 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71e8074ea0 con 0x7f71e8136fe0 2023-09-30T22:35:22.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.322+0000 7f71deffd700 1 -- 172.21.15.16:0/2526747822 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f71e002b080 con 0x7f71e8136fe0 2023-09-30T22:35:22.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.526+0000 7f71f0345700 1 -- 172.21.15.16:0/2526747822 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f71e8120870 con 0x7f71e8136fe0 2023-09-30T22:35:22.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.527+0000 7f71deffd700 1 -- 172.21.15.16:0/2526747822 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f71e00123f0 con 0x7f71e8136fe0 2023-09-30T22:35:22.527 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:35:22.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.529+0000 7f71f0345700 1 -- 172.21.15.16:0/2526747822 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f71e8136fe0 msgr2=0x7f71e811eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:22.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.529+0000 7f71f0345700 1 --2- 172.21.15.16:0/2526747822 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f71e8136fe0 0x7f71e811eef0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f71e0015000 tx=0x7f71e0004ad0 comp rx=0 tx=0).stop 2023-09-30T22:35:22.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.529+0000 7f71f0345700 1 -- 172.21.15.16:0/2526747822 shutdown_connections 2023-09-30T22:35:22.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.529+0000 7f71f0345700 1 --2- 172.21.15.16:0/2526747822 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f71e809e930 0x7f71e811e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:22.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.529+0000 7f71f0345700 1 --2- 172.21.15.16:0/2526747822 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f71e8136fe0 0x7f71e811eef0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:22.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.529+0000 7f71f0345700 1 --2- 172.21.15.16:0/2526747822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f71e809f410 0x7f71e811e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:22.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.529+0000 7f71f0345700 1 -- 172.21.15.16:0/2526747822 >> 172.21.15.16:0/2526747822 conn(0x7f71e80a4210 msgr2=0x7f71e80a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:22.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.529+0000 7f71f0345700 1 -- 172.21.15.16:0/2526747822 shutdown_connections 2023-09-30T22:35:22.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.529+0000 7f71f0345700 1 -- 172.21.15.16:0/2526747822 wait complete. 2023-09-30T22:35:22.543 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:35:22.590 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:35:22.591 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:35:22.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.732+0000 7fd84e1ea700 1 Processor -- start 2023-09-30T22:35:22.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.732+0000 7f2bccaa6700 1 Processor -- start 2023-09-30T22:35:22.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.733+0000 7fd84e1ea700 1 -- start start 2023-09-30T22:35:22.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.733+0000 7f2bccaa6700 1 -- start start 2023-09-30T22:35:22.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.733+0000 7fd84e1ea700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd84809e930 0x7fd84809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:22.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.733+0000 7fd84e1ea700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd84809f650 0x7fd848131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:22.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.733+0000 7fd84e1ea700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd848070940 con 0x7fd84809f240 2023-09-30T22:35:22.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.733+0000 7fd84e1ea700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd84806fd60 con 0x7fd84809e930 2023-09-30T22:35:22.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.733+0000 7fd84e1ea700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd84806fa80 con 0x7fd84809f650 2023-09-30T22:35:22.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.733+0000 7fd8477fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd84809e930 0x7fd84809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:22.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.733+0000 7f2bccaa6700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2bc81338a0 0x7f2bc8133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:22.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.733+0000 7f2bccaa6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2bc809de50 0x7f2bc809e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:22.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.733+0000 7fd847fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd84809f650 0x7fd848131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:22.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.733+0000 7fd8477fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd84809e930 0x7fd84809ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:52960/0 (socket says 172.21.15.16:52960) 2023-09-30T22:35:22.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.733+0000 7fd8477fe700 1 -- 172.21.15.16:0/2167473071 learned_addr learned my addr 172.21.15.16:0/2167473071 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:22.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.733+0000 7f2bccaa6700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bc806ef30 con 0x7f2bc809e760 2023-09-30T22:35:22.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.733+0000 7f2bccaa6700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bc806fef0 con 0x7f2bc81338a0 2023-09-30T22:35:22.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.733+0000 7f2bccaa6700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc81334b0 con 0x7f2bc809de50 2023-09-30T22:35:22.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.734+0000 7f2bc5d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2bc809de50 0x7f2bc809e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:22.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.734+0000 7f2bc6d9d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2bc809e760 0x7f2bc813a730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33590/0 (socket says 172.21.15.16:33590) 2023-09-30T22:35:22.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.734+0000 7f2bc6d9d700 1 -- 172.21.15.16:0/2302820360 learned_addr learned my addr 172.21.15.16:0/2302820360 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:22.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.734+0000 7f2bc659c700 1 --2- 172.21.15.16:0/2302820360 >> v2:172.21.15.16:3300/0 conn(0x7f2bc81338a0 0x7f2bc8133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:22.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.734+0000 7fd8477fe700 1 -- 172.21.15.16:0/2167473071 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd84809f650 msgr2=0x7fd848131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:22.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.734+0000 7fd8477fe700 1 --2- 172.21.15.16:0/2167473071 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd84809f650 0x7fd848131f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:22.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.734+0000 7fd8477fe700 1 -- 172.21.15.16:0/2167473071 >> v1:172.21.15.16:6789/0 conn(0x7fd84809f240 legacy=0x7fd8481317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:22.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.734+0000 7fd8477fe700 1 -- 172.21.15.16:0/2167473071 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd84809fb60 con 0x7fd84809e930 2023-09-30T22:35:22.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.734+0000 7f2bc559a700 1 -- 172.21.15.16:0/2302820360 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2422955106 0 0) 0x7f2bc806ef30 con 0x7f2bc809e760 2023-09-30T22:35:22.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.734+0000 7f2bc659c700 1 -- 172.21.15.16:0/2302820360 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2bc809de50 msgr2=0x7f2bc809e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:22.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.734+0000 7f2bc659c700 1 --2- 172.21.15.16:0/2302820360 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2bc809de50 0x7f2bc809e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:22.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.734+0000 7f2bc659c700 1 -- 172.21.15.16:0/2302820360 >> v1:172.21.15.16:6789/0 conn(0x7f2bc809e760 legacy=0x7f2bc813a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:22.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.734+0000 7f2bc659c700 1 -- 172.21.15.16:0/2302820360 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc806fc30 con 0x7f2bc81338a0 2023-09-30T22:35:22.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.734+0000 7fd8477fe700 1 --2- 172.21.15.16:0/2167473071 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd84809e930 0x7fd84809ed00 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fd838009e80 tx=0x7fd83800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13151524cdea60ec server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:22.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.734+0000 7f2bc659c700 1 --2- 172.21.15.16:0/2302820360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bc81338a0 0x7f2bc8133c70 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8009840 tx=0x7f2bb8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=890c7ed68286c5a0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:22.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.734+0000 7fd8467fc700 1 -- 172.21.15.16:0/2167473071 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83800e070 con 0x7fd84809e930 2023-09-30T22:35:22.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.734+0000 7fd8467fc700 1 -- 172.21.15.16:0/2167473071 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd83800b910 con 0x7fd84809e930 2023-09-30T22:35:22.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.735+0000 7f2bc559a700 1 -- 172.21.15.16:0/2302820360 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8011830 con 0x7f2bc81338a0 2023-09-30T22:35:22.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.735+0000 7fd8467fc700 1 -- 172.21.15.16:0/2167473071 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd83800bcc0 con 0x7fd84809e930 2023-09-30T22:35:22.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.735+0000 7f2bc559a700 1 -- 172.21.15.16:0/2302820360 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bb8011990 con 0x7f2bc81338a0 2023-09-30T22:35:22.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.735+0000 7f2bc559a700 1 -- 172.21.15.16:0/2302820360 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8011d20 con 0x7f2bc81338a0 2023-09-30T22:35:22.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.735+0000 7f2bccaa6700 1 -- 172.21.15.16:0/2302820360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bc81338a0 msgr2=0x7f2bc8133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:22.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.735+0000 7fd84e1ea700 1 -- 172.21.15.16:0/2167473071 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd84809e930 msgr2=0x7fd84809ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:22.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.735+0000 7fd84e1ea700 1 --2- 172.21.15.16:0/2167473071 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd84809e930 0x7fd84809ed00 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fd838009e80 tx=0x7fd83800b330 comp rx=0 tx=0).stop 2023-09-30T22:35:22.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.735+0000 7f2bccaa6700 1 --2- 172.21.15.16:0/2302820360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bc81338a0 0x7f2bc8133c70 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8009840 tx=0x7f2bb8009b50 comp rx=0 tx=0).stop 2023-09-30T22:35:22.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.735+0000 7fd84e1ea700 1 -- 172.21.15.16:0/2167473071 shutdown_connections 2023-09-30T22:35:22.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.735+0000 7fd84e1ea700 1 --2- 172.21.15.16:0/2167473071 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd84809f650 0x7fd848131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:22.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.735+0000 7fd84e1ea700 1 --2- 172.21.15.16:0/2167473071 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd84809e930 0x7fd84809ed00 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:22.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.735+0000 7fd84e1ea700 1 -- 172.21.15.16:0/2167473071 >> 172.21.15.16:0/2167473071 conn(0x7fd8480a4210 msgr2=0x7fd8480a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:22.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.735+0000 7fd84e1ea700 1 -- 172.21.15.16:0/2167473071 shutdown_connections 2023-09-30T22:35:22.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.735+0000 7fd84e1ea700 1 -- 172.21.15.16:0/2167473071 wait complete. 2023-09-30T22:35:22.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.735+0000 7f2bccaa6700 1 -- 172.21.15.16:0/2302820360 shutdown_connections 2023-09-30T22:35:22.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.735+0000 7f2bccaa6700 1 --2- 172.21.15.16:0/2302820360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bc81338a0 0x7f2bc8133c70 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:22.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.735+0000 7f2bccaa6700 1 --2- 172.21.15.16:0/2302820360 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2bc809de50 0x7f2bc809e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:22.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.735+0000 7f2bccaa6700 1 -- 172.21.15.16:0/2302820360 >> 172.21.15.16:0/2302820360 conn(0x7f2bc80a2aa0 msgr2=0x7f2bc80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:22.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.736+0000 7f2bccaa6700 1 -- 172.21.15.16:0/2302820360 shutdown_connections 2023-09-30T22:35:22.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.736+0000 7f2bccaa6700 1 -- 172.21.15.16:0/2302820360 wait complete. 2023-09-30T22:35:22.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.736+0000 7fd84e1ea700 1 Processor -- start 2023-09-30T22:35:22.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.737+0000 7f2bccaa6700 1 Processor -- start 2023-09-30T22:35:22.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.737+0000 7fd84e1ea700 1 -- start start 2023-09-30T22:35:22.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.737+0000 7f2bccaa6700 1 -- start start 2023-09-30T22:35:22.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.737+0000 7fd84e1ea700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd84809e930 0x7fd8481a2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:22.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.737+0000 7fd84e1ea700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd84809f240 0x7fd8481a3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:22.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.737+0000 7fd84e1ea700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd84809f650 0x7fd8481a3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:22.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.737+0000 7fd84e1ea700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8481a4990 con 0x7fd84809f650 2023-09-30T22:35:22.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.737+0000 7fd84e1ea700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8481a6520 con 0x7fd84809e930 2023-09-30T22:35:22.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.737+0000 7fd84e1ea700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd8481a5b40 con 0x7fd84809f240 2023-09-30T22:35:22.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.737+0000 7f2bccaa6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2bc809de50 0x7f2bc811d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:22.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.737+0000 7f2bccaa6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bc809e760 0x7f2bc811db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:22.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.737+0000 7f2bccaa6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2bc81338a0 0x7f2bc811e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:22.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.737+0000 7f2bccaa6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc8121090 con 0x7f2bc809e760 2023-09-30T22:35:22.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.737+0000 7f2bccaa6700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc8120460 con 0x7f2bc809de50 2023-09-30T22:35:22.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.737+0000 7f2bccaa6700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc8121a30 con 0x7f2bc81338a0 2023-09-30T22:35:22.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.737+0000 7f2bc5d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bc809e760 0x7f2bc811db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:22.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.737+0000 7fd8477fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd84809e930 0x7fd8481a2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:22.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.737+0000 7fd8477fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd84809e930 0x7fd8481a2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:52970/0 (socket says 172.21.15.16:52970) 2023-09-30T22:35:22.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.737+0000 7fd8477fe700 1 -- 172.21.15.16:0/1473902660 learned_addr learned my addr 172.21.15.16:0/1473902660 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:22.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.737+0000 7fd847fff700 1 --2- 172.21.15.16:0/1473902660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd84809f650 0x7fd8481a3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:22.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.738+0000 7fd846ffd700 1 --2- 172.21.15.16:0/1473902660 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd84809f240 0x7fd8481a3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:22.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.737+0000 7f2bc5d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bc809e760 0x7f2bc811db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55496/0 (socket says 172.21.15.16:55496) 2023-09-30T22:35:22.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.737+0000 7f2bc5d9b700 1 -- 172.21.15.16:0/2927262647 learned_addr learned my addr 172.21.15.16:0/2927262647 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:22.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.737+0000 7f2bc6d9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2bc81338a0 0x7f2bc811e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:22.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.737+0000 7f2bc659c700 1 --2- 172.21.15.16:0/2927262647 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2bc809de50 0x7f2bc811d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:22.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.738+0000 7f2bc5d9b700 1 -- 172.21.15.16:0/2927262647 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2bc81338a0 msgr2=0x7f2bc811e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:22.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.738+0000 7f2bc5d9b700 1 --2- 172.21.15.16:0/2927262647 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2bc81338a0 0x7f2bc811e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:22.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.738+0000 7f2bc5d9b700 1 -- 172.21.15.16:0/2927262647 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2bc809de50 msgr2=0x7f2bc811d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:22.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.738+0000 7f2bc5d9b700 1 --2- 172.21.15.16:0/2927262647 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2bc809de50 0x7f2bc811d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:22.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.738+0000 7f2bc5d9b700 1 -- 172.21.15.16:0/2927262647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb80094a0 con 0x7f2bc809e760 2023-09-30T22:35:22.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.738+0000 7fd8477fe700 1 -- 172.21.15.16:0/1473902660 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd84809f240 msgr2=0x7fd8481a3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:22.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.738+0000 7fd8477fe700 1 --2- 172.21.15.16:0/1473902660 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd84809f240 0x7fd8481a3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:22.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.738+0000 7fd8477fe700 1 -- 172.21.15.16:0/1473902660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd84809f650 msgr2=0x7fd8481a3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:22.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.738+0000 7fd8477fe700 1 --2- 172.21.15.16:0/1473902660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd84809f650 0x7fd8481a3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:22.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.738+0000 7fd8477fe700 1 -- 172.21.15.16:0/1473902660 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd838009b30 con 0x7fd84809e930 2023-09-30T22:35:22.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.738+0000 7fd8477fe700 1 --2- 172.21.15.16:0/1473902660 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd84809e930 0x7fd8481a2ce0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fd838000f80 tx=0x7fd838004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:22.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.738+0000 7f2bc5d9b700 1 --2- 172.21.15.16:0/2927262647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bc809e760 0x7f2bc811db40 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc00e8a0 tx=0x7f2bbc00ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:22.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.738+0000 7f2bb6ffd700 1 -- 172.21.15.16:0/2927262647 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc009590 con 0x7f2bc809e760 2023-09-30T22:35:22.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.738+0000 7f2bb6ffd700 1 -- 172.21.15.16:0/2927262647 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bbc013680 con 0x7f2bc809e760 2023-09-30T22:35:22.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.738+0000 7f2bb6ffd700 1 -- 172.21.15.16:0/2927262647 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc0139f0 con 0x7f2bc809e760 2023-09-30T22:35:22.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.738+0000 7f2bccaa6700 1 -- 172.21.15.16:0/2927262647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bc8120820 con 0x7f2bc809e760 2023-09-30T22:35:22.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.738+0000 7fd83ffff700 1 -- 172.21.15.16:0/1473902660 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8380092e0 con 0x7fd84809e930 2023-09-30T22:35:22.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.738+0000 7fd83ffff700 1 -- 172.21.15.16:0/1473902660 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd838004250 con 0x7fd84809e930 2023-09-30T22:35:22.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.738+0000 7fd84e1ea700 1 -- 172.21.15.16:0/1473902660 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8480a0190 con 0x7fd84809e930 2023-09-30T22:35:22.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.738+0000 7f2bccaa6700 1 -- 172.21.15.16:0/2927262647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bc811f180 con 0x7f2bc809e760 2023-09-30T22:35:22.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.739+0000 7fd83ffff700 1 -- 172.21.15.16:0/1473902660 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd838004600 con 0x7fd84809e930 2023-09-30T22:35:22.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.739+0000 7fd84e1ea700 1 -- 172.21.15.16:0/1473902660 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd848071d20 con 0x7fd84809e930 2023-09-30T22:35:22.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.740+0000 7f2bccaa6700 1 -- 172.21.15.16:0/2927262647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bc8120a50 con 0x7f2bc809e760 2023-09-30T22:35:22.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.740+0000 7fd83ffff700 1 -- 172.21.15.16:0/1473902660 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd838004760 con 0x7fd84809e930 2023-09-30T22:35:22.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.740+0000 7fd84e1ea700 1 -- 172.21.15.16:0/1473902660 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8481a61c0 con 0x7fd84809e930 2023-09-30T22:35:22.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.740+0000 7f2bb6ffd700 1 -- 172.21.15.16:0/2927262647 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2bbc013b50 con 0x7f2bc809e760 2023-09-30T22:35:22.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.740+0000 7fd83ffff700 1 -- 172.21.15.16:0/1473902660 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd8380409c0 con 0x7fd84809e930 2023-09-30T22:35:22.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.740+0000 7f2bb6ffd700 1 -- 172.21.15.16:0/2927262647 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f2bbc03fad0 con 0x7f2bc809e760 2023-09-30T22:35:22.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.745+0000 7f2bb6ffd700 1 -- 172.21.15.16:0/2927262647 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2bc8120a50 con 0x7f2bc809e760 2023-09-30T22:35:22.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.745+0000 7fd83ffff700 1 -- 172.21.15.16:0/1473902660 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd838019350 con 0x7fd84809e930 2023-09-30T22:35:22.918 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:35:22.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.954+0000 7f2bccaa6700 1 -- 172.21.15.16:0/2927262647 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2bc8120a50 con 0x7f2bc809e760 2023-09-30T22:35:22.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.955+0000 7f2bb6ffd700 1 -- 172.21.15.16:0/2927262647 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2bc8120a50 con 0x7f2bc809e760 2023-09-30T22:35:22.955 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:35:22.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.956+0000 7fd84e1ea700 1 -- 172.21.15.16:0/1473902660 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd848074ea0 con 0x7fd84809e930 2023-09-30T22:35:22.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.957+0000 7fd83ffff700 1 -- 172.21.15.16:0/1473902660 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd838020070 con 0x7fd84809e930 2023-09-30T22:35:22.957 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:35:22.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.957+0000 7f2bccaa6700 1 -- 172.21.15.16:0/2927262647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bc809e760 msgr2=0x7f2bc811db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:22.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.957+0000 7f2bccaa6700 1 --2- 172.21.15.16:0/2927262647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bc809e760 0x7f2bc811db40 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc00e8a0 tx=0x7f2bbc00ebb0 comp rx=0 tx=0).stop 2023-09-30T22:35:22.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.957+0000 7f2bccaa6700 1 -- 172.21.15.16:0/2927262647 shutdown_connections 2023-09-30T22:35:22.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.957+0000 7f2bccaa6700 1 --2- 172.21.15.16:0/2927262647 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2bc809de50 0x7f2bc811d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:22.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.957+0000 7f2bccaa6700 1 --2- 172.21.15.16:0/2927262647 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2bc81338a0 0x7f2bc811e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:22.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.957+0000 7f2bccaa6700 1 --2- 172.21.15.16:0/2927262647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2bc809e760 0x7f2bc811db40 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:22.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.957+0000 7f2bccaa6700 1 -- 172.21.15.16:0/2927262647 >> 172.21.15.16:0/2927262647 conn(0x7f2bc80a2aa0 msgr2=0x7f2bc81bae30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:22.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.957+0000 7f2bccaa6700 1 -- 172.21.15.16:0/2927262647 shutdown_connections 2023-09-30T22:35:22.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.957+0000 7f2bccaa6700 1 -- 172.21.15.16:0/2927262647 wait complete. 2023-09-30T22:35:22.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.961+0000 7fd84e1ea700 1 -- 172.21.15.16:0/1473902660 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd84809e930 msgr2=0x7fd8481a2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:22.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.961+0000 7fd84e1ea700 1 --2- 172.21.15.16:0/1473902660 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd84809e930 0x7fd8481a2ce0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fd838000f80 tx=0x7fd838004ad0 comp rx=0 tx=0).stop 2023-09-30T22:35:22.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.961+0000 7fd84e1ea700 1 -- 172.21.15.16:0/1473902660 shutdown_connections 2023-09-30T22:35:22.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.961+0000 7fd84e1ea700 1 --2- 172.21.15.16:0/1473902660 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd84809e930 0x7fd8481a2ce0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:22.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.961+0000 7fd84e1ea700 1 --2- 172.21.15.16:0/1473902660 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd84809f650 0x7fd8481a3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:22.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.961+0000 7fd84e1ea700 1 --2- 172.21.15.16:0/1473902660 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd84809f240 0x7fd8481a3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:22.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.961+0000 7fd84e1ea700 1 -- 172.21.15.16:0/1473902660 >> 172.21.15.16:0/1473902660 conn(0x7fd8480a4210 msgr2=0x7fd8481b7a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:22.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.961+0000 7fd84e1ea700 1 -- 172.21.15.16:0/1473902660 shutdown_connections 2023-09-30T22:35:22.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:22.961+0000 7fd84e1ea700 1 -- 172.21.15.16:0/1473902660 wait complete. 2023-09-30T22:35:22.973 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:35:22.978 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:35:23.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.060+0000 7f3a59208700 1 Processor -- start 2023-09-30T22:35:23.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.060+0000 7f3a59208700 1 -- start start 2023-09-30T22:35:23.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.061+0000 7f3a59208700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3a5409e1d0 0x7f3a54138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:23.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.061+0000 7f3a59208700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3a54139170 0x7f3a5413b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:23.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.061+0000 7f3a59208700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a5409b120 con 0x7f3a5409d6f0 2023-09-30T22:35:23.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.061+0000 7f3a59208700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5406b460 con 0x7f3a54139170 2023-09-30T22:35:23.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.061+0000 7f3a59208700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3a5409a7e0 con 0x7f3a5409e1d0 2023-09-30T22:35:23.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.061+0000 7f3a52d9d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3a5409d6f0 0x7f3a5409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33598/0 (socket says 172.21.15.16:33598) 2023-09-30T22:35:23.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.061+0000 7f3a52d9d700 1 -- 172.21.15.16:0/584485541 learned_addr learned my addr 172.21.15.16:0/584485541 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:23.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.061+0000 7f3a5359e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3a54139170 0x7f3a5413b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:23.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.061+0000 7f3a5259c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3a5409e1d0 0x7f3a54138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:23.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.061+0000 7f3a51d9b700 1 -- 172.21.15.16:0/584485541 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 541423976 0 0) 0x7f3a5409b120 con 0x7f3a5409d6f0 2023-09-30T22:35:23.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.061+0000 7f3a5259c700 1 -- 172.21.15.16:0/584485541 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3a54139170 msgr2=0x7f3a5413b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:23.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.061+0000 7f3a5259c700 1 --2- 172.21.15.16:0/584485541 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3a54139170 0x7f3a5413b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:23.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.061+0000 7f3a5259c700 1 -- 172.21.15.16:0/584485541 >> v1:172.21.15.16:6789/0 conn(0x7f3a5409d6f0 legacy=0x7f3a5409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:23.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.061+0000 7f3a5259c700 1 -- 172.21.15.16:0/584485541 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a5409a500 con 0x7f3a5409e1d0 2023-09-30T22:35:23.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.061+0000 7f3a5359e700 1 --2- 172.21.15.16:0/584485541 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3a54139170 0x7f3a5413b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:23.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.061+0000 7f3a5259c700 1 --2- 172.21.15.16:0/584485541 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3a5409e1d0 0x7f3a54138c30 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c0098a0 tx=0x7f3a3c009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61bc35f4900562ac server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:23.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.062+0000 7f3a51d9b700 1 -- 172.21.15.16:0/584485541 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c011830 con 0x7f3a5409e1d0 2023-09-30T22:35:23.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.062+0000 7f3a51d9b700 1 -- 172.21.15.16:0/584485541 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a3c011990 con 0x7f3a5409e1d0 2023-09-30T22:35:23.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.062+0000 7f3a51d9b700 1 -- 172.21.15.16:0/584485541 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c011d20 con 0x7f3a5409e1d0 2023-09-30T22:35:23.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.062+0000 7f3a59208700 1 -- 172.21.15.16:0/584485541 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3a5409e1d0 msgr2=0x7f3a54138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:23.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.062+0000 7f3a59208700 1 --2- 172.21.15.16:0/584485541 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3a5409e1d0 0x7f3a54138c30 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c0098a0 tx=0x7f3a3c009bb0 comp rx=0 tx=0).stop 2023-09-30T22:35:23.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.062+0000 7f3a59208700 1 -- 172.21.15.16:0/584485541 shutdown_connections 2023-09-30T22:35:23.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.062+0000 7f3a59208700 1 --2- 172.21.15.16:0/584485541 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3a54139170 0x7f3a5413b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:23.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.062+0000 7f3a59208700 1 --2- 172.21.15.16:0/584485541 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3a5409e1d0 0x7f3a54138c30 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:23.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.062+0000 7f3a59208700 1 -- 172.21.15.16:0/584485541 >> 172.21.15.16:0/584485541 conn(0x7f3a54066f50 msgr2=0x7f3a5409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:23.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.062+0000 7f3a59208700 1 -- 172.21.15.16:0/584485541 shutdown_connections 2023-09-30T22:35:23.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.062+0000 7f3a59208700 1 -- 172.21.15.16:0/584485541 wait complete. 2023-09-30T22:35:23.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.064+0000 7f3a59208700 1 Processor -- start 2023-09-30T22:35:23.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.064+0000 7f3a59208700 1 -- start start 2023-09-30T22:35:23.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.064+0000 7f3a59208700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a5409d6f0 0x7f3a541c8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:23.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.064+0000 7f3a59208700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3a5409e1d0 0x7f3a541c89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:23.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.064+0000 7f3a59208700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3a54139170 0x7f3a541c8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:23.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.064+0000 7f3a59208700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a54072210 con 0x7f3a5409d6f0 2023-09-30T22:35:23.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.064+0000 7f3a59208700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a54120680 con 0x7f3a5409e1d0 2023-09-30T22:35:23.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.064+0000 7f3a59208700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3a54121d20 con 0x7f3a54139170 2023-09-30T22:35:23.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.064+0000 7f3a5259c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3a5409e1d0 0x7f3a541c89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:23.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.064+0000 7f3a5359e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3a54139170 0x7f3a541c8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:23.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.064+0000 7f3a5259c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3a5409e1d0 0x7f3a541c89a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:52992/0 (socket says 172.21.15.16:52992) 2023-09-30T22:35:23.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.064+0000 7f3a52d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a5409d6f0 0x7f3a541c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:23.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.064+0000 7f3a5259c700 1 -- 172.21.15.16:0/1509195922 learned_addr learned my addr 172.21.15.16:0/1509195922 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:23.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.065+0000 7f3a5259c700 1 -- 172.21.15.16:0/1509195922 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3a54139170 msgr2=0x7f3a541c8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:23.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.065+0000 7f3a5259c700 1 --2- 172.21.15.16:0/1509195922 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3a54139170 0x7f3a541c8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:23.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.065+0000 7f3a5259c700 1 -- 172.21.15.16:0/1509195922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a5409d6f0 msgr2=0x7f3a541c8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:23.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.065+0000 7f3a5259c700 1 --2- 172.21.15.16:0/1509195922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a5409d6f0 0x7f3a541c8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:23.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.065+0000 7f3a5259c700 1 -- 172.21.15.16:0/1509195922 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a4c009a70 con 0x7f3a5409e1d0 2023-09-30T22:35:23.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.065+0000 7f3a5359e700 1 --2- 172.21.15.16:0/1509195922 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3a54139170 0x7f3a541c8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:23.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.065+0000 7f3a52d9d700 1 --2- 172.21.15.16:0/1509195922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a5409d6f0 0x7f3a541c8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:23.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.065+0000 7f3a5259c700 1 --2- 172.21.15.16:0/1509195922 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3a5409e1d0 0x7f3a541c89a0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c000c00 tx=0x7f3a3c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:23.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.065+0000 7f3a4b7fe700 1 -- 172.21.15.16:0/1509195922 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c004550 con 0x7f3a5409e1d0 2023-09-30T22:35:23.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.065+0000 7f3a4b7fe700 1 -- 172.21.15.16:0/1509195922 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a3c01f070 con 0x7f3a5409e1d0 2023-09-30T22:35:23.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.065+0000 7f3a4b7fe700 1 -- 172.21.15.16:0/1509195922 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c0038e0 con 0x7f3a5409e1d0 2023-09-30T22:35:23.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.065+0000 7f3a59208700 1 -- 172.21.15.16:0/1509195922 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a3c0094a0 con 0x7f3a5409e1d0 2023-09-30T22:35:23.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.065+0000 7f3a59208700 1 -- 172.21.15.16:0/1509195922 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a541222b0 con 0x7f3a5409e1d0 2023-09-30T22:35:23.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.066+0000 7f3a4b7fe700 1 -- 172.21.15.16:0/1509195922 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3a3c003a40 con 0x7f3a5409e1d0 2023-09-30T22:35:23.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.066+0000 7f3a4b7fe700 1 -- 172.21.15.16:0/1509195922 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f3a3c021070 con 0x7f3a5409e1d0 2023-09-30T22:35:23.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.066+0000 7f3a59208700 1 -- 172.21.15.16:0/1509195922 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a541224d0 con 0x7f3a5409e1d0 2023-09-30T22:35:23.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.071+0000 7f3a4b7fe700 1 -- 172.21.15.16:0/1509195922 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3a541224d0 con 0x7f3a5409e1d0 2023-09-30T22:35:23.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.275+0000 7f3a59208700 1 -- 172.21.15.16:0/1509195922 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3a541224d0 con 0x7f3a5409e1d0 2023-09-30T22:35:23.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.275+0000 7f3a4b7fe700 1 -- 172.21.15.16:0/1509195922 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3a541224d0 con 0x7f3a5409e1d0 2023-09-30T22:35:23.276 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:35:23.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.277+0000 7f3a59208700 1 -- 172.21.15.16:0/1509195922 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3a5409e1d0 msgr2=0x7f3a541c89a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:23.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.277+0000 7f3a59208700 1 --2- 172.21.15.16:0/1509195922 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3a5409e1d0 0x7f3a541c89a0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c000c00 tx=0x7f3a3c004750 comp rx=0 tx=0).stop 2023-09-30T22:35:23.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.277+0000 7f3a59208700 1 -- 172.21.15.16:0/1509195922 shutdown_connections 2023-09-30T22:35:23.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.277+0000 7f3a59208700 1 --2- 172.21.15.16:0/1509195922 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3a54139170 0x7f3a541c8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:23.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.277+0000 7f3a59208700 1 --2- 172.21.15.16:0/1509195922 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3a5409d6f0 0x7f3a541c8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:23.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.277+0000 7f3a59208700 1 --2- 172.21.15.16:0/1509195922 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3a5409e1d0 0x7f3a541c89a0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:23.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.277+0000 7f3a59208700 1 -- 172.21.15.16:0/1509195922 >> 172.21.15.16:0/1509195922 conn(0x7f3a54066f50 msgr2=0x7f3a54136b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:23.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.277+0000 7f3a59208700 1 -- 172.21.15.16:0/1509195922 shutdown_connections 2023-09-30T22:35:23.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.277+0000 7f3a59208700 1 -- 172.21.15.16:0/1509195922 wait complete. 2023-09-30T22:35:23.291 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:35:23.544 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:35:23.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.691+0000 7f7036600700 1 Processor -- start 2023-09-30T22:35:23.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.691+0000 7f7036600700 1 -- start start 2023-09-30T22:35:23.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.691+0000 7f7036600700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f703009f410 0x7f7030136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:23.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.691+0000 7f7036600700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7030136fe0 0x7f70301393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:23.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.691+0000 7f7036600700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7030070940 con 0x7f703009e930 2023-09-30T22:35:23.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.691+0000 7f7036600700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f703006fd60 con 0x7f703009f410 2023-09-30T22:35:23.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.691+0000 7f7036600700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f703006fa80 con 0x7f7030136fe0 2023-09-30T22:35:23.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.692+0000 7f702f7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f703009f410 0x7f7030136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:23.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.692+0000 7f7034b9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7030136fe0 0x7f70301393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:23.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.692+0000 7f702ffff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f703009e930 0x7f703009ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33606/0 (socket says 172.21.15.16:33606) 2023-09-30T22:35:23.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.692+0000 7f702ffff700 1 -- 172.21.15.16:0/2666942522 learned_addr learned my addr 172.21.15.16:0/2666942522 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:23.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.692+0000 7f702f7fe700 1 -- 172.21.15.16:0/2666942522 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7030136fe0 msgr2=0x7f70301393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:23.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.692+0000 7f702effd700 1 -- 172.21.15.16:0/2666942522 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1900365499 0 0) 0x7f7030070940 con 0x7f703009e930 2023-09-30T22:35:23.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.692+0000 7f702f7fe700 1 --2- 172.21.15.16:0/2666942522 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7030136fe0 0x7f70301393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:23.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.692+0000 7f702f7fe700 1 -- 172.21.15.16:0/2666942522 >> v1:172.21.15.16:6789/0 conn(0x7f703009e930 legacy=0x7f703009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:23.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.692+0000 7f702f7fe700 1 -- 172.21.15.16:0/2666942522 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f703009fb60 con 0x7f703009f410 2023-09-30T22:35:23.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.692+0000 7f7034b9d700 1 --2- 172.21.15.16:0/2666942522 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7030136fe0 0x7f70301393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:23.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.692+0000 7f702f7fe700 1 --2- 172.21.15.16:0/2666942522 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f703009f410 0x7f7030136aa0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f7018009870 tx=0x7f7018009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfb4ea0d7e13623c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:23.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.692+0000 7f702effd700 1 -- 172.21.15.16:0/2666942522 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7018011830 con 0x7f703009f410 2023-09-30T22:35:23.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.692+0000 7f702effd700 1 -- 172.21.15.16:0/2666942522 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7018011990 con 0x7f703009f410 2023-09-30T22:35:23.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.692+0000 7f702effd700 1 -- 172.21.15.16:0/2666942522 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7018011d20 con 0x7f703009f410 2023-09-30T22:35:23.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.693+0000 7f7036600700 1 -- 172.21.15.16:0/2666942522 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f703009f410 msgr2=0x7f7030136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:23.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.693+0000 7f7036600700 1 --2- 172.21.15.16:0/2666942522 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f703009f410 0x7f7030136aa0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f7018009870 tx=0x7f7018009b80 comp rx=0 tx=0).stop 2023-09-30T22:35:23.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.693+0000 7f7036600700 1 -- 172.21.15.16:0/2666942522 shutdown_connections 2023-09-30T22:35:23.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.693+0000 7f7036600700 1 --2- 172.21.15.16:0/2666942522 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7030136fe0 0x7f70301393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:23.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.693+0000 7f7036600700 1 --2- 172.21.15.16:0/2666942522 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f703009f410 0x7f7030136aa0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:23.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.693+0000 7f7036600700 1 -- 172.21.15.16:0/2666942522 >> 172.21.15.16:0/2666942522 conn(0x7f70300a4210 msgr2=0x7f70300a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:23.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.693+0000 7f7036600700 1 -- 172.21.15.16:0/2666942522 shutdown_connections 2023-09-30T22:35:23.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.693+0000 7f7036600700 1 -- 172.21.15.16:0/2666942522 wait complete. 2023-09-30T22:35:23.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.694+0000 7f7036600700 1 Processor -- start 2023-09-30T22:35:23.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.694+0000 7f7036600700 1 -- start start 2023-09-30T22:35:23.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.695+0000 7f7036600700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f703009e930 0x7f703011d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:23.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.695+0000 7f7036600700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f703009f410 0x7f70301c6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:23.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.695+0000 7f7036600700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7030136fe0 0x7f70301c6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:23.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.695+0000 7f7036600700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f703011d0d0 con 0x7f703009e930 2023-09-30T22:35:23.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.695+0000 7f7036600700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f703011fe20 con 0x7f703009f410 2023-09-30T22:35:23.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.695+0000 7f7036600700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f70301200e0 con 0x7f7030136fe0 2023-09-30T22:35:23.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.695+0000 7f702ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f703009e930 0x7f703011d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:23.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.695+0000 7f7034b9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7030136fe0 0x7f70301c6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:23.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.695+0000 7f7034b9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7030136fe0 0x7f70301c6780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:33198/0 (socket says 172.21.15.16:33198) 2023-09-30T22:35:23.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.695+0000 7f702ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f703009e930 0x7f703011d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55520/0 (socket says 172.21.15.16:55520) 2023-09-30T22:35:23.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.695+0000 7f7034b9d700 1 -- 172.21.15.16:0/2839418424 learned_addr learned my addr 172.21.15.16:0/2839418424 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:23.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.695+0000 7f702f7fe700 1 --2- 172.21.15.16:0/2839418424 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f703009f410 0x7f70301c6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:23.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.695+0000 7f7034b9d700 1 -- 172.21.15.16:0/2839418424 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f703009f410 msgr2=0x7f70301c6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:23.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.695+0000 7f7034b9d700 1 --2- 172.21.15.16:0/2839418424 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f703009f410 0x7f70301c6240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:23.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.695+0000 7f7034b9d700 1 -- 172.21.15.16:0/2839418424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f703009e930 msgr2=0x7f703011d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:23.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.695+0000 7f7034b9d700 1 --2- 172.21.15.16:0/2839418424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f703009e930 0x7f703011d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:23.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.695+0000 7f7034b9d700 1 -- 172.21.15.16:0/2839418424 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7028009a70 con 0x7f7030136fe0 2023-09-30T22:35:23.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.695+0000 7f702ffff700 1 --2- 172.21.15.16:0/2839418424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f703009e930 0x7f703011d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:23.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.696+0000 7f7034b9d700 1 --2- 172.21.15.16:0/2839418424 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7030136fe0 0x7f70301c6780 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f702800efc0 tx=0x7f702800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:23.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.696+0000 7f702cff9700 1 -- 172.21.15.16:0/2839418424 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702800cc90 con 0x7f7030136fe0 2023-09-30T22:35:23.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.696+0000 7f702cff9700 1 -- 172.21.15.16:0/2839418424 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f702800f040 con 0x7f7030136fe0 2023-09-30T22:35:23.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.696+0000 7f7036600700 1 -- 172.21.15.16:0/2839418424 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70180094a0 con 0x7f7030136fe0 2023-09-30T22:35:23.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.696+0000 7f702cff9700 1 -- 172.21.15.16:0/2839418424 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70280146c0 con 0x7f7030136fe0 2023-09-30T22:35:23.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.696+0000 7f7036600700 1 -- 172.21.15.16:0/2839418424 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7030120b50 con 0x7f7030136fe0 2023-09-30T22:35:23.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.697+0000 7f702cff9700 1 -- 172.21.15.16:0/2839418424 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7028014820 con 0x7f7030136fe0 2023-09-30T22:35:23.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.697+0000 7f702cff9700 1 -- 172.21.15.16:0/2839418424 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f7028040430 con 0x7f7030136fe0 2023-09-30T22:35:23.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.697+0000 7f7036600700 1 -- 172.21.15.16:0/2839418424 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f703009fdb0 con 0x7f7030136fe0 2023-09-30T22:35:23.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.702+0000 7f702cff9700 1 -- 172.21.15.16:0/2839418424 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7028012700 con 0x7f7030136fe0 2023-09-30T22:35:23.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.909+0000 7f7036600700 1 -- 172.21.15.16:0/2839418424 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f70300a3d20 con 0x7f7030136fe0 2023-09-30T22:35:23.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.909+0000 7f702cff9700 1 -- 172.21.15.16:0/2839418424 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7028012340 con 0x7f7030136fe0 2023-09-30T22:35:23.910 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:35:23.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.912+0000 7f7036600700 1 -- 172.21.15.16:0/2839418424 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7030136fe0 msgr2=0x7f70301c6780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:23.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.912+0000 7f7036600700 1 --2- 172.21.15.16:0/2839418424 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7030136fe0 0x7f70301c6780 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f702800efc0 tx=0x7f702800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:35:23.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.912+0000 7f7036600700 1 -- 172.21.15.16:0/2839418424 shutdown_connections 2023-09-30T22:35:23.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.912+0000 7f7036600700 1 --2- 172.21.15.16:0/2839418424 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7030136fe0 0x7f70301c6780 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:23.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.912+0000 7f7036600700 1 --2- 172.21.15.16:0/2839418424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f703009e930 0x7f703011d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:23.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.912+0000 7f7036600700 1 --2- 172.21.15.16:0/2839418424 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f703009f410 0x7f70301c6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:23.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.912+0000 7f7036600700 1 -- 172.21.15.16:0/2839418424 >> 172.21.15.16:0/2839418424 conn(0x7f70300a4210 msgr2=0x7f7030135c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:23.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.912+0000 7f7036600700 1 -- 172.21.15.16:0/2839418424 shutdown_connections 2023-09-30T22:35:23.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:23.912+0000 7f7036600700 1 -- 172.21.15.16:0/2839418424 wait complete. 2023-09-30T22:35:23.926 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:35:23.974 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:35:23.979 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:35:24.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.117+0000 7f335206c700 1 Processor -- start 2023-09-30T22:35:24.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.117+0000 7f335206c700 1 -- start start 2023-09-30T22:35:24.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.118+0000 7f335206c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f334c0a0a30 0x7f334c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:24.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.118+0000 7f335206c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f334c0a1340 0x7f334c133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:24.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.118+0000 7f335206c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f334c133fa0 0x7f334c134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:24.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.118+0000 7f335206c700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f334c09edd0 con 0x7f334c133fa0 2023-09-30T22:35:24.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.118+0000 7f335206c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f334c09e1f0 con 0x7f334c0a1340 2023-09-30T22:35:24.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.118+0000 7f335206c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f334c09dc50 con 0x7f334c0a0a30 2023-09-30T22:35:24.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.118+0000 7f334affd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f334c0a1340 0x7f334c133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:24.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.118+0000 7f334bfff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f334c133fa0 0x7f334c134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:24.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.118+0000 7f334affd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f334c0a1340 0x7f334c133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:53036/0 (socket says 172.21.15.16:53036) 2023-09-30T22:35:24.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.118+0000 7f334affd700 1 -- 172.21.15.16:0/258977704 learned_addr learned my addr 172.21.15.16:0/258977704 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:24.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.118+0000 7f334b7fe700 1 --2- 172.21.15.16:0/258977704 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f334c0a0a30 0x7f334c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:24.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.119+0000 7f334bfff700 1 -- 172.21.15.16:0/258977704 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f334c0a0a30 msgr2=0x7f334c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:24.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.119+0000 7f334bfff700 1 --2- 172.21.15.16:0/258977704 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f334c0a0a30 0x7f334c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:24.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.119+0000 7f334bfff700 1 -- 172.21.15.16:0/258977704 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f334c0a1340 msgr2=0x7f334c133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:24.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.119+0000 7f334bfff700 1 --2- 172.21.15.16:0/258977704 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f334c0a1340 0x7f334c133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:24.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.119+0000 7f334bfff700 1 -- 172.21.15.16:0/258977704 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f334c0a1fe0 con 0x7f334c133fa0 2023-09-30T22:35:24.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.119+0000 7f334b7fe700 1 --2- 172.21.15.16:0/258977704 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f334c0a0a30 0x7f334c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:24.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.119+0000 7f334affd700 1 --2- 172.21.15.16:0/258977704 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f334c0a1340 0x7f334c133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:24.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.119+0000 7f334bfff700 1 --2- 172.21.15.16:0/258977704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f334c133fa0 0x7f334c134410 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f3344009840 tx=0x7f3344009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a27b176ff4ae688 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:24.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.119+0000 7f334a7fc700 1 -- 172.21.15.16:0/258977704 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3344011830 con 0x7f334c133fa0 2023-09-30T22:35:24.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.119+0000 7f334a7fc700 1 -- 172.21.15.16:0/258977704 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3344011990 con 0x7f334c133fa0 2023-09-30T22:35:24.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.119+0000 7f334a7fc700 1 -- 172.21.15.16:0/258977704 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3344011d20 con 0x7f334c133fa0 2023-09-30T22:35:24.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.120+0000 7f335206c700 1 -- 172.21.15.16:0/258977704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f334c133fa0 msgr2=0x7f334c134410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:24.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.120+0000 7f335206c700 1 --2- 172.21.15.16:0/258977704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f334c133fa0 0x7f334c134410 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f3344009840 tx=0x7f3344009b50 comp rx=0 tx=0).stop 2023-09-30T22:35:24.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.120+0000 7f335206c700 1 -- 172.21.15.16:0/258977704 shutdown_connections 2023-09-30T22:35:24.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.120+0000 7f335206c700 1 --2- 172.21.15.16:0/258977704 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f334c133fa0 0x7f334c134410 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:24.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.120+0000 7f335206c700 1 --2- 172.21.15.16:0/258977704 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f334c0a0a30 0x7f334c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:24.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.120+0000 7f335206c700 1 --2- 172.21.15.16:0/258977704 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f334c0a1340 0x7f334c133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:24.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.120+0000 7f335206c700 1 -- 172.21.15.16:0/258977704 >> 172.21.15.16:0/258977704 conn(0x7f334c0a6550 msgr2=0x7f334c0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:24.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.120+0000 7f335206c700 1 -- 172.21.15.16:0/258977704 shutdown_connections 2023-09-30T22:35:24.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.120+0000 7f335206c700 1 -- 172.21.15.16:0/258977704 wait complete. 2023-09-30T22:35:24.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.121+0000 7f335206c700 1 Processor -- start 2023-09-30T22:35:24.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.121+0000 7f335206c700 1 -- start start 2023-09-30T22:35:24.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.122+0000 7f335206c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f334c0a0a30 0x7f334c1200b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:24.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.122+0000 7f335206c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f334c0a1340 0x7f334c1205f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:24.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.122+0000 7f335206c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f334c133fa0 0x7f334c120b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:24.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.122+0000 7f335206c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f334c1238f0 con 0x7f334c0a0a30 2023-09-30T22:35:24.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.122+0000 7f335206c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f334c123af0 con 0x7f334c133fa0 2023-09-30T22:35:24.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.122+0000 7f335206c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f334c1229e0 con 0x7f334c0a1340 2023-09-30T22:35:24.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.122+0000 7f334affd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f334c0a1340 0x7f334c1205f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:24.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.122+0000 7f334bfff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f334c133fa0 0x7f334c120b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:24.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.122+0000 7f334affd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f334c0a1340 0x7f334c1205f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:33218/0 (socket says 172.21.15.16:33218) 2023-09-30T22:35:24.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.122+0000 7f334bfff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f334c133fa0 0x7f334c120b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:53044/0 (socket says 172.21.15.16:53044) 2023-09-30T22:35:24.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.122+0000 7f334affd700 1 -- 172.21.15.16:0/3590167569 learned_addr learned my addr 172.21.15.16:0/3590167569 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:24.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.122+0000 7f334b7fe700 1 --2- 172.21.15.16:0/3590167569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f334c0a0a30 0x7f334c1200b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:24.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.122+0000 7f334affd700 1 -- 172.21.15.16:0/3590167569 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f334c133fa0 msgr2=0x7f334c120b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:24.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.122+0000 7f334affd700 1 --2- 172.21.15.16:0/3590167569 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f334c133fa0 0x7f334c120b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:24.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.122+0000 7f334affd700 1 -- 172.21.15.16:0/3590167569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f334c0a0a30 msgr2=0x7f334c1200b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:24.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.122+0000 7f334affd700 1 --2- 172.21.15.16:0/3590167569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f334c0a0a30 0x7f334c1200b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:24.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.122+0000 7f334affd700 1 -- 172.21.15.16:0/3590167569 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f333c009a70 con 0x7f334c0a1340 2023-09-30T22:35:24.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.122+0000 7f334b7fe700 1 --2- 172.21.15.16:0/3590167569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f334c0a0a30 0x7f334c1200b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:24.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.122+0000 7f334affd700 1 --2- 172.21.15.16:0/3590167569 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f334c0a1340 0x7f334c1205f0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f334000ec10 tx=0x7f334000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:24.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.122+0000 7f3333fff700 1 -- 172.21.15.16:0/3590167569 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334000c850 con 0x7f334c0a1340 2023-09-30T22:35:24.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.123+0000 7f3333fff700 1 -- 172.21.15.16:0/3590167569 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f334000c9b0 con 0x7f334c0a1340 2023-09-30T22:35:24.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.123+0000 7f335206c700 1 -- 172.21.15.16:0/3590167569 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3340009550 con 0x7f334c0a1340 2023-09-30T22:35:24.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.123+0000 7f3333fff700 1 -- 172.21.15.16:0/3590167569 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334000cd40 con 0x7f334c0a1340 2023-09-30T22:35:24.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.123+0000 7f335206c700 1 -- 172.21.15.16:0/3590167569 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33440094a0 con 0x7f334c0a1340 2023-09-30T22:35:24.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.123+0000 7f3333fff700 1 -- 172.21.15.16:0/3590167569 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3340013470 con 0x7f334c0a1340 2023-09-30T22:35:24.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.124+0000 7f3333fff700 1 -- 172.21.15.16:0/3590167569 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f334003fb90 con 0x7f334c0a1340 2023-09-30T22:35:24.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.124+0000 7f335206c700 1 -- 172.21.15.16:0/3590167569 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f334c0a34a0 con 0x7f334c0a1340 2023-09-30T22:35:24.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.128+0000 7f3333fff700 1 -- 172.21.15.16:0/3590167569 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3340021020 con 0x7f334c0a1340 2023-09-30T22:35:24.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.159+0000 7f03f3e6c700 1 Processor -- start 2023-09-30T22:35:24.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.159+0000 7f03f3e6c700 1 -- start start 2023-09-30T22:35:24.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.159+0000 7f03f3e6c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f03ec0a0a30 0x7f03ec0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:24.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.159+0000 7f03f3e6c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f03ec0a1340 0x7f03ec133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:24.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.159+0000 7f03f3e6c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f03ec133fa0 0x7f03ec134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:24.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.159+0000 7f03f3e6c700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03ec09edd0 con 0x7f03ec0a0a30 2023-09-30T22:35:24.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.159+0000 7f03f3e6c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03ec09e1f0 con 0x7f03ec0a1340 2023-09-30T22:35:24.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.159+0000 7f03f3e6c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f03ec09dc50 con 0x7f03ec133fa0 2023-09-30T22:35:24.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.159+0000 7f03f2409700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f03ec133fa0 0x7f03ec134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:24.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.159+0000 7f03f1407700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f03ec0a1340 0x7f03ec133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:24.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.159+0000 7f03f1c08700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f03ec0a0a30 0x7f03ec0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:24.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.160+0000 7f03f1c08700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f03ec0a0a30 0x7f03ec0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55548/0 (socket says 172.21.15.16:55548) 2023-09-30T22:35:24.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.160+0000 7f03f2409700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f03ec133fa0 0x7f03ec134410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:33234/0 (socket says 172.21.15.16:33234) 2023-09-30T22:35:24.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.160+0000 7f03f1c08700 1 -- 172.21.15.16:0/3245900813 learned_addr learned my addr 172.21.15.16:0/3245900813 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:24.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.160+0000 7f03f1407700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f03ec0a1340 0x7f03ec133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:53052/0 (socket says 172.21.15.16:53052) 2023-09-30T22:35:24.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.160+0000 7f03f1407700 1 -- 172.21.15.16:0/3245900813 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f03ec133fa0 msgr2=0x7f03ec134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:24.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.160+0000 7f03f1407700 1 --2- 172.21.15.16:0/3245900813 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f03ec133fa0 0x7f03ec134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:24.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.160+0000 7f03f1407700 1 -- 172.21.15.16:0/3245900813 >> v2:172.21.15.16:3300/0 conn(0x7f03ec0a0a30 msgr2=0x7f03ec0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:24.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.160+0000 7f03f1407700 1 --2- 172.21.15.16:0/3245900813 >> v2:172.21.15.16:3300/0 conn(0x7f03ec0a0a30 0x7f03ec0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:24.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.160+0000 7f03f1407700 1 -- 172.21.15.16:0/3245900813 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03ec0a1fe0 con 0x7f03ec0a1340 2023-09-30T22:35:24.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.160+0000 7f03f1c08700 1 --2- 172.21.15.16:0/3245900813 >> v2:172.21.15.16:3300/0 conn(0x7f03ec0a0a30 0x7f03ec0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:24.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.160+0000 7f03f2409700 1 --2- 172.21.15.16:0/3245900813 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f03ec133fa0 0x7f03ec134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:24.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.160+0000 7f03f1407700 1 --2- 172.21.15.16:0/3245900813 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f03ec0a1340 0x7f03ec133a60 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f03dc009870 tx=0x7f03dc009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75241e0c57d2d584 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:24.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.160+0000 7f03f0c06700 1 -- 172.21.15.16:0/3245900813 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03dc011830 con 0x7f03ec0a1340 2023-09-30T22:35:24.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.160+0000 7f03f0c06700 1 -- 172.21.15.16:0/3245900813 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03dc011990 con 0x7f03ec0a1340 2023-09-30T22:35:24.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.160+0000 7f03f0c06700 1 -- 172.21.15.16:0/3245900813 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03dc011d20 con 0x7f03ec0a1340 2023-09-30T22:35:24.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.161+0000 7f03f3e6c700 1 -- 172.21.15.16:0/3245900813 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f03ec0a1340 msgr2=0x7f03ec133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:24.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.161+0000 7f03f3e6c700 1 --2- 172.21.15.16:0/3245900813 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f03ec0a1340 0x7f03ec133a60 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f03dc009870 tx=0x7f03dc009b80 comp rx=0 tx=0).stop 2023-09-30T22:35:24.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.161+0000 7f03f3e6c700 1 -- 172.21.15.16:0/3245900813 shutdown_connections 2023-09-30T22:35:24.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.161+0000 7f03f3e6c700 1 --2- 172.21.15.16:0/3245900813 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f03ec133fa0 0x7f03ec134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:24.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.161+0000 7f03f3e6c700 1 --2- 172.21.15.16:0/3245900813 >> v2:172.21.15.16:3300/0 conn(0x7f03ec0a0a30 0x7f03ec0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:24.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.161+0000 7f03f3e6c700 1 --2- 172.21.15.16:0/3245900813 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f03ec0a1340 0x7f03ec133a60 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:24.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.161+0000 7f03f3e6c700 1 -- 172.21.15.16:0/3245900813 >> 172.21.15.16:0/3245900813 conn(0x7f03ec0a6550 msgr2=0x7f03ec0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:24.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.161+0000 7f03f3e6c700 1 -- 172.21.15.16:0/3245900813 shutdown_connections 2023-09-30T22:35:24.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.161+0000 7f03f3e6c700 1 -- 172.21.15.16:0/3245900813 wait complete. 2023-09-30T22:35:24.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.162+0000 7f03f3e6c700 1 Processor -- start 2023-09-30T22:35:24.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.162+0000 7f03f3e6c700 1 -- start start 2023-09-30T22:35:24.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.163+0000 7f03f3e6c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f03ec0a0a30 0x7f03ec1206a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:24.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.163+0000 7f03f3e6c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f03ec0a1340 0x7f03ec120be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:24.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.163+0000 7f03f3e6c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03ec133fa0 0x7f03ec121120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:24.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.163+0000 7f03f3e6c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ec121ae0 con 0x7f03ec133fa0 2023-09-30T22:35:24.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.163+0000 7f03f3e6c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03ec122890 con 0x7f03ec0a1340 2023-09-30T22:35:24.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.163+0000 7f03f3e6c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f03ec123950 con 0x7f03ec0a0a30 2023-09-30T22:35:24.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.163+0000 7f03f1c08700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f03ec0a0a30 0x7f03ec1206a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:24.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.163+0000 7f03f2409700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03ec133fa0 0x7f03ec121120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:24.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.163+0000 7f03f1c08700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f03ec0a0a30 0x7f03ec1206a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:33250/0 (socket says 172.21.15.16:33250) 2023-09-30T22:35:24.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.163+0000 7f03f1c08700 1 -- 172.21.15.16:0/613977311 learned_addr learned my addr 172.21.15.16:0/613977311 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:24.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.163+0000 7f03f1407700 1 --2- 172.21.15.16:0/613977311 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f03ec0a1340 0x7f03ec120be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:24.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.163+0000 7f03f1c08700 1 -- 172.21.15.16:0/613977311 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f03ec0a1340 msgr2=0x7f03ec120be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:24.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.163+0000 7f03f1c08700 1 --2- 172.21.15.16:0/613977311 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f03ec0a1340 0x7f03ec120be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:24.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.163+0000 7f03f1c08700 1 -- 172.21.15.16:0/613977311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03ec133fa0 msgr2=0x7f03ec121120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:24.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.163+0000 7f03f1c08700 1 --2- 172.21.15.16:0/613977311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03ec133fa0 0x7f03ec121120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:24.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.163+0000 7f03f1c08700 1 -- 172.21.15.16:0/613977311 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e8009550 con 0x7f03ec0a0a30 2023-09-30T22:35:24.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.163+0000 7f03f2409700 1 --2- 172.21.15.16:0/613977311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03ec133fa0 0x7f03ec121120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:24.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.163+0000 7f03f1407700 1 --2- 172.21.15.16:0/613977311 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f03ec0a1340 0x7f03ec120be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:24.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.163+0000 7f03f1c08700 1 --2- 172.21.15.16:0/613977311 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f03ec0a0a30 0x7f03ec1206a0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f03e800ec10 tx=0x7f03e800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:24.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.163+0000 7f03e27fc700 1 -- 172.21.15.16:0/613977311 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e800c850 con 0x7f03ec0a0a30 2023-09-30T22:35:24.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.164+0000 7f03e27fc700 1 -- 172.21.15.16:0/613977311 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03e800c9b0 con 0x7f03ec0a0a30 2023-09-30T22:35:24.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.164+0000 7f03e27fc700 1 -- 172.21.15.16:0/613977311 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e800cd20 con 0x7f03ec0a0a30 2023-09-30T22:35:24.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.164+0000 7f03f3e6c700 1 -- 172.21.15.16:0/613977311 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03e4009a70 con 0x7f03ec0a0a30 2023-09-30T22:35:24.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.164+0000 7f03f3e6c700 1 -- 172.21.15.16:0/613977311 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03dc0094a0 con 0x7f03ec0a0a30 2023-09-30T22:35:24.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.164+0000 7f03e27fc700 1 -- 172.21.15.16:0/613977311 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f03e8013470 con 0x7f03ec0a0a30 2023-09-30T22:35:24.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.165+0000 7f03f3e6c700 1 -- 172.21.15.16:0/613977311 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03ec0a3610 con 0x7f03ec0a0a30 2023-09-30T22:35:24.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.165+0000 7f03e27fc700 1 -- 172.21.15.16:0/613977311 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f03e803f780 con 0x7f03ec0a0a30 2023-09-30T22:35:24.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.174+0000 7f03e27fc700 1 -- 172.21.15.16:0/613977311 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f03e8013720 con 0x7f03ec0a0a30 2023-09-30T22:35:24.293 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:35:24.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.344+0000 7f335206c700 1 -- 172.21.15.16:0/3590167569 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f334c0a0220 con 0x7f334c0a1340 2023-09-30T22:35:24.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.345+0000 7f3333fff700 1 -- 172.21.15.16:0/3590167569 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f334000cea0 con 0x7f334c0a1340 2023-09-30T22:35:24.345 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:35:24.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.347+0000 7f335206c700 1 -- 172.21.15.16:0/3590167569 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f334c0a1340 msgr2=0x7f334c1205f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:24.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.347+0000 7f335206c700 1 --2- 172.21.15.16:0/3590167569 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f334c0a1340 0x7f334c1205f0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f334000ec10 tx=0x7f334000ef20 comp rx=0 tx=0).stop 2023-09-30T22:35:24.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.347+0000 7f335206c700 1 -- 172.21.15.16:0/3590167569 shutdown_connections 2023-09-30T22:35:24.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.347+0000 7f335206c700 1 --2- 172.21.15.16:0/3590167569 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f334c133fa0 0x7f334c120b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:24.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.347+0000 7f335206c700 1 --2- 172.21.15.16:0/3590167569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f334c0a0a30 0x7f334c1200b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:24.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.347+0000 7f335206c700 1 --2- 172.21.15.16:0/3590167569 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f334c0a1340 0x7f334c1205f0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:24.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.347+0000 7f335206c700 1 -- 172.21.15.16:0/3590167569 >> 172.21.15.16:0/3590167569 conn(0x7f334c0a6550 msgr2=0x7f334c09ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:24.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.348+0000 7f335206c700 1 -- 172.21.15.16:0/3590167569 shutdown_connections 2023-09-30T22:35:24.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.348+0000 7f335206c700 1 -- 172.21.15.16:0/3590167569 wait complete. 2023-09-30T22:35:24.362 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:35:24.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.381+0000 7f03f3e6c700 1 -- 172.21.15.16:0/613977311 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f03ec074ea0 con 0x7f03ec0a0a30 2023-09-30T22:35:24.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.382+0000 7f03e27fc700 1 -- 172.21.15.16:0/613977311 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f03e8021020 con 0x7f03ec0a0a30 2023-09-30T22:35:24.382 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:35:24.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.386+0000 7f03f3e6c700 1 -- 172.21.15.16:0/613977311 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f03ec0a0a30 msgr2=0x7f03ec1206a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:24.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.386+0000 7f03f3e6c700 1 --2- 172.21.15.16:0/613977311 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f03ec0a0a30 0x7f03ec1206a0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f03e800ec10 tx=0x7f03e800ef20 comp rx=0 tx=0).stop 2023-09-30T22:35:24.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.386+0000 7f03f3e6c700 1 -- 172.21.15.16:0/613977311 shutdown_connections 2023-09-30T22:35:24.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.386+0000 7f03f3e6c700 1 --2- 172.21.15.16:0/613977311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03ec133fa0 0x7f03ec121120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:24.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.386+0000 7f03f3e6c700 1 --2- 172.21.15.16:0/613977311 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f03ec0a0a30 0x7f03ec1206a0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:24.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.386+0000 7f03f3e6c700 1 --2- 172.21.15.16:0/613977311 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f03ec0a1340 0x7f03ec120be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:24.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.386+0000 7f03f3e6c700 1 -- 172.21.15.16:0/613977311 >> 172.21.15.16:0/613977311 conn(0x7f03ec0a6550 msgr2=0x7f03ec09ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:24.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.386+0000 7f03f3e6c700 1 -- 172.21.15.16:0/613977311 shutdown_connections 2023-09-30T22:35:24.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.386+0000 7f03f3e6c700 1 -- 172.21.15.16:0/613977311 wait complete. 2023-09-30T22:35:24.400 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:35:24.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.434+0000 7f4907da6700 1 Processor -- start 2023-09-30T22:35:24.434 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.434+0000 7f4907da6700 1 -- start start 2023-09-30T22:35:24.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.434+0000 7f4907da6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4900130540 0x7f4900130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:24.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.434+0000 7f4907da6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f490006fcf0 0x7f4900138570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:24.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.434+0000 7f4907da6700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49000716a0 con 0x7f4900130e50 2023-09-30T22:35:24.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.434+0000 7f4907da6700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f490009a390 con 0x7f490006fcf0 2023-09-30T22:35:24.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.434+0000 7f4907da6700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4900071290 con 0x7f4900130540 2023-09-30T22:35:24.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.435+0000 7f4906343700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f490006fcf0 0x7f4900138570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:24.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.435+0000 7f4905341700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4900130e50 0x7f490006f5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33618/0 (socket says 172.21.15.16:33618) 2023-09-30T22:35:24.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.435+0000 7f4905341700 1 -- 172.21.15.16:0/2980487214 learned_addr learned my addr 172.21.15.16:0/2980487214 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:24.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.435+0000 7f4905b42700 1 --2- 172.21.15.16:0/2980487214 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4900130540 0x7f4900130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:24.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.435+0000 7f4906343700 1 -- 172.21.15.16:0/2980487214 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4900130540 msgr2=0x7f4900130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:24.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.435+0000 7f4906343700 1 --2- 172.21.15.16:0/2980487214 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4900130540 0x7f4900130910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:24.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.435+0000 7f4906343700 1 -- 172.21.15.16:0/2980487214 >> v1:172.21.15.16:6789/0 conn(0x7f4900130e50 legacy=0x7f490006f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:24.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.435+0000 7f4904b40700 1 -- 172.21.15.16:0/2980487214 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 44823189 0 0) 0x7f49000716a0 con 0x7f4900130e50 2023-09-30T22:35:24.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.435+0000 7f4906343700 1 -- 172.21.15.16:0/2980487214 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4900130340 con 0x7f490006fcf0 2023-09-30T22:35:24.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.435+0000 7f4905b42700 1 --2- 172.21.15.16:0/2980487214 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4900130540 0x7f4900130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:24.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.435+0000 7f4906343700 1 --2- 172.21.15.16:0/2980487214 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f490006fcf0 0x7f4900138570 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f48f8009dc0 tx=0x7f48f800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ecf221ce65721d3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:24.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.435+0000 7f4904b40700 1 -- 172.21.15.16:0/2980487214 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f800e070 con 0x7f490006fcf0 2023-09-30T22:35:24.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.435+0000 7f4904b40700 1 -- 172.21.15.16:0/2980487214 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48f800bb20 con 0x7f490006fcf0 2023-09-30T22:35:24.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.435+0000 7f4904b40700 1 -- 172.21.15.16:0/2980487214 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f8012430 con 0x7f490006fcf0 2023-09-30T22:35:24.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.436+0000 7f4907da6700 1 -- 172.21.15.16:0/2980487214 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f490006fcf0 msgr2=0x7f4900138570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:24.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.436+0000 7f4907da6700 1 --2- 172.21.15.16:0/2980487214 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f490006fcf0 0x7f4900138570 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f48f8009dc0 tx=0x7f48f800b330 comp rx=0 tx=0).stop 2023-09-30T22:35:24.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.436+0000 7f4907da6700 1 -- 172.21.15.16:0/2980487214 shutdown_connections 2023-09-30T22:35:24.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.436+0000 7f4907da6700 1 --2- 172.21.15.16:0/2980487214 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f490006fcf0 0x7f4900138570 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:24.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.436+0000 7f4907da6700 1 --2- 172.21.15.16:0/2980487214 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4900130540 0x7f4900130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:24.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.436+0000 7f4907da6700 1 -- 172.21.15.16:0/2980487214 >> 172.21.15.16:0/2980487214 conn(0x7f49000a2aa0 msgr2=0x7f49000a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:24.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.436+0000 7f4907da6700 1 -- 172.21.15.16:0/2980487214 shutdown_connections 2023-09-30T22:35:24.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.436+0000 7f4907da6700 1 -- 172.21.15.16:0/2980487214 wait complete. 2023-09-30T22:35:24.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.437+0000 7f4907da6700 1 Processor -- start 2023-09-30T22:35:24.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.437+0000 7f4907da6700 1 -- start start 2023-09-30T22:35:24.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.437+0000 7f4907da6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f490006fcf0 0x7f490011d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:24.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.437+0000 7f4907da6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4900130540 0x7f490011dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:24.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.437+0000 7f4907da6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4900130e50 0x7f490011e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:24.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.437+0000 7f4907da6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490011f350 con 0x7f4900130540 2023-09-30T22:35:24.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.437+0000 7f4907da6700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4900121260 con 0x7f4900130e50 2023-09-30T22:35:24.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.437+0000 7f4907da6700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4900120cc0 con 0x7f490006fcf0 2023-09-30T22:35:24.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.438+0000 7f4905341700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4900130540 0x7f490011dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:24.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.438+0000 7f4906343700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4900130e50 0x7f490011e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:24.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.438+0000 7f4905b42700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f490006fcf0 0x7f490011d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:24.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.438+0000 7f4905b42700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f490006fcf0 0x7f490011d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:33274/0 (socket says 172.21.15.16:33274) 2023-09-30T22:35:24.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.438+0000 7f4905341700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4900130540 0x7f490011dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55562/0 (socket says 172.21.15.16:55562) 2023-09-30T22:35:24.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.438+0000 7f4905b42700 1 -- 172.21.15.16:0/689317355 learned_addr learned my addr 172.21.15.16:0/689317355 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:24.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.438+0000 7f4906343700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4900130e50 0x7f490011e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:53072/0 (socket says 172.21.15.16:53072) 2023-09-30T22:35:24.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.438+0000 7f4906343700 1 -- 172.21.15.16:0/689317355 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f490006fcf0 msgr2=0x7f490011d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:24.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.438+0000 7f4906343700 1 --2- 172.21.15.16:0/689317355 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f490006fcf0 0x7f490011d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:24.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.438+0000 7f4906343700 1 -- 172.21.15.16:0/689317355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4900130540 msgr2=0x7f490011dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:24.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.438+0000 7f4906343700 1 --2- 172.21.15.16:0/689317355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4900130540 0x7f490011dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:24.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.438+0000 7f4906343700 1 -- 172.21.15.16:0/689317355 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48fc0094a0 con 0x7f4900130e50 2023-09-30T22:35:24.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.438+0000 7f4906343700 1 --2- 172.21.15.16:0/689317355 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4900130e50 0x7f490011e1c0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f48f8016000 tx=0x7f48f8004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:24.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.438+0000 7f48f67fc700 1 -- 172.21.15.16:0/689317355 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f8012430 con 0x7f4900130e50 2023-09-30T22:35:24.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.438+0000 7f48f67fc700 1 -- 172.21.15.16:0/689317355 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48f8020070 con 0x7f4900130e50 2023-09-30T22:35:24.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.438+0000 7f4907da6700 1 -- 172.21.15.16:0/689317355 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48f8009a70 con 0x7f4900130e50 2023-09-30T22:35:24.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.438+0000 7f48f67fc700 1 -- 172.21.15.16:0/689317355 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f8004280 con 0x7f4900130e50 2023-09-30T22:35:24.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.438+0000 7f4907da6700 1 -- 172.21.15.16:0/689317355 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49000718e0 con 0x7f4900130e50 2023-09-30T22:35:24.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.439+0000 7f48f67fc700 1 -- 172.21.15.16:0/689317355 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f48f80043e0 con 0x7f4900130e50 2023-09-30T22:35:24.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.439+0000 7f48f67fc700 1 -- 172.21.15.16:0/689317355 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f48f8022070 con 0x7f4900130e50 2023-09-30T22:35:24.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.439+0000 7f4907da6700 1 -- 172.21.15.16:0/689317355 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f490011a470 con 0x7f4900130e50 2023-09-30T22:35:24.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.445+0000 7f48f67fc700 1 -- 172.21.15.16:0/689317355 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f48f801b400 con 0x7f4900130e50 2023-09-30T22:35:24.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.648+0000 7f4907da6700 1 -- 172.21.15.16:0/689317355 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4900074ea0 con 0x7f4900130e50 2023-09-30T22:35:24.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.649+0000 7f48f67fc700 1 -- 172.21.15.16:0/689317355 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f48f801ab70 con 0x7f4900130e50 2023-09-30T22:35:24.649 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:35:24.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.651+0000 7f4907da6700 1 -- 172.21.15.16:0/689317355 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4900130e50 msgr2=0x7f490011e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:24.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.651+0000 7f4907da6700 1 --2- 172.21.15.16:0/689317355 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4900130e50 0x7f490011e1c0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f48f8016000 tx=0x7f48f8004ad0 comp rx=0 tx=0).stop 2023-09-30T22:35:24.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.651+0000 7f4907da6700 1 -- 172.21.15.16:0/689317355 shutdown_connections 2023-09-30T22:35:24.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.651+0000 7f4907da6700 1 --2- 172.21.15.16:0/689317355 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4900130e50 0x7f490011e1c0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:24.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.651+0000 7f4907da6700 1 --2- 172.21.15.16:0/689317355 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f490006fcf0 0x7f490011d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:24.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.651+0000 7f4907da6700 1 --2- 172.21.15.16:0/689317355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4900130540 0x7f490011dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:24.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.651+0000 7f4907da6700 1 -- 172.21.15.16:0/689317355 >> 172.21.15.16:0/689317355 conn(0x7f49000a2aa0 msgr2=0x7f490009d000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:24.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.651+0000 7f4907da6700 1 -- 172.21.15.16:0/689317355 shutdown_connections 2023-09-30T22:35:24.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:24.651+0000 7f4907da6700 1 -- 172.21.15.16:0/689317355 wait complete. 2023-09-30T22:35:24.666 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:35:24.927 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:35:25.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.074+0000 7f5b1e461700 1 Processor -- start 2023-09-30T22:35:25.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.074+0000 7f5b1e461700 1 -- start start 2023-09-30T22:35:25.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.075+0000 7f5b1e461700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5b18133490 0x7f5b18135870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:25.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.075+0000 7f5b1e461700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5b18135db0 0x7f5b18138190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:25.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.075+0000 7f5b1e461700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b1809c190 con 0x7f5b18130750 2023-09-30T22:35:25.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.075+0000 7f5b1e461700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b1809b090 con 0x7f5b18133490 2023-09-30T22:35:25.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.075+0000 7f5b1e461700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1809a6c0 con 0x7f5b18135db0 2023-09-30T22:35:25.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.075+0000 7f5b177fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5b18133490 0x7f5b18135870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:25.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.075+0000 7f5b1c9fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5b18135db0 0x7f5b18138190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:25.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.075+0000 7f5b177fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5b18133490 0x7f5b18135870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55572/0 (socket says 172.21.15.16:55572) 2023-09-30T22:35:25.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.075+0000 7f5b177fe700 1 -- 172.21.15.16:0/434347955 learned_addr learned my addr 172.21.15.16:0/434347955 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:25.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.075+0000 7f5b16ffd700 1 -- 172.21.15.16:0/434347955 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2405461202 0 0) 0x7f5b1809c190 con 0x7f5b18130750 2023-09-30T22:35:25.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.075+0000 7f5b16ffd700 1 -- 172.21.15.16:0/434347955 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b000035b0 con 0x7f5b18130750 2023-09-30T22:35:25.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.075+0000 7f5b1c9fe700 1 -- 172.21.15.16:0/434347955 >> v2:172.21.15.16:3300/0 conn(0x7f5b18133490 msgr2=0x7f5b18135870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:25.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.075+0000 7f5b1c9fe700 1 --2- 172.21.15.16:0/434347955 >> v2:172.21.15.16:3300/0 conn(0x7f5b18133490 0x7f5b18135870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:25.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.075+0000 7f5b1c9fe700 1 -- 172.21.15.16:0/434347955 >> v1:172.21.15.16:6789/0 conn(0x7f5b18130750 legacy=0x7f5b18130b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:25.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.075+0000 7f5b1c9fe700 1 -- 172.21.15.16:0/434347955 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b18130300 con 0x7f5b18135db0 2023-09-30T22:35:25.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.075+0000 7f5b177fe700 1 --2- 172.21.15.16:0/434347955 >> v2:172.21.15.16:3300/0 conn(0x7f5b18133490 0x7f5b18135870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:25.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.075+0000 7f5b1c9fe700 1 --2- 172.21.15.16:0/434347955 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5b18135db0 0x7f5b18138190 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f5b1000bfa0 tx=0x7f5b1000d330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df7b3d612a0c1869 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:25.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.076+0000 7f5b16ffd700 1 -- 172.21.15.16:0/434347955 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b10010070 con 0x7f5b18135db0 2023-09-30T22:35:25.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.076+0000 7f5b16ffd700 1 -- 172.21.15.16:0/434347955 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b1000b2e0 con 0x7f5b18135db0 2023-09-30T22:35:25.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.076+0000 7f5b16ffd700 1 -- 172.21.15.16:0/434347955 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1000dc40 con 0x7f5b18135db0 2023-09-30T22:35:25.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.076+0000 7f5b1e461700 1 -- 172.21.15.16:0/434347955 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5b18135db0 msgr2=0x7f5b18138190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:25.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.076+0000 7f5b1e461700 1 --2- 172.21.15.16:0/434347955 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5b18135db0 0x7f5b18138190 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f5b1000bfa0 tx=0x7f5b1000d330 comp rx=0 tx=0).stop 2023-09-30T22:35:25.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.076+0000 7f5b1e461700 1 -- 172.21.15.16:0/434347955 shutdown_connections 2023-09-30T22:35:25.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.076+0000 7f5b1e461700 1 --2- 172.21.15.16:0/434347955 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5b18135db0 0x7f5b18138190 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:25.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.076+0000 7f5b1e461700 1 --2- 172.21.15.16:0/434347955 >> v2:172.21.15.16:3300/0 conn(0x7f5b18133490 0x7f5b18135870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:25.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.076+0000 7f5b1e461700 1 -- 172.21.15.16:0/434347955 >> 172.21.15.16:0/434347955 conn(0x7f5b1809f470 msgr2=0x7f5b180a1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:25.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.076+0000 7f5b1e461700 1 -- 172.21.15.16:0/434347955 shutdown_connections 2023-09-30T22:35:25.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.076+0000 7f5b1e461700 1 -- 172.21.15.16:0/434347955 wait complete. 2023-09-30T22:35:25.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.077+0000 7f5b1e461700 1 Processor -- start 2023-09-30T22:35:25.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.078+0000 7f5b1e461700 1 -- start start 2023-09-30T22:35:25.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.078+0000 7f5b1e461700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5b18130750 0x7f5b18126e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:25.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.078+0000 7f5b1e461700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b18133490 0x7f5b18127340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:25.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.078+0000 7f5b1e461700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5b18135db0 0x7f5b18127880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:25.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.078+0000 7f5b1e461700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b181281f0 con 0x7f5b18133490 2023-09-30T22:35:25.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.078+0000 7f5b1e461700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b18128860 con 0x7f5b18130750 2023-09-30T22:35:25.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.078+0000 7f5b1e461700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1812a500 con 0x7f5b18135db0 2023-09-30T22:35:25.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.078+0000 7f5b177fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b18133490 0x7f5b18127340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:25.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.078+0000 7f5b177fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b18133490 0x7f5b18127340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55588/0 (socket says 172.21.15.16:55588) 2023-09-30T22:35:25.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.078+0000 7f5b1c9fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5b18135db0 0x7f5b18127880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:25.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.078+0000 7f5b177fe700 1 -- 172.21.15.16:0/1561056274 learned_addr learned my addr 172.21.15.16:0/1561056274 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:25.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.078+0000 7f5b17fff700 1 --2- 172.21.15.16:0/1561056274 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5b18130750 0x7f5b18126e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:25.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.078+0000 7f5b1c9fe700 1 -- 172.21.15.16:0/1561056274 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5b18130750 msgr2=0x7f5b18126e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:25.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.078+0000 7f5b1c9fe700 1 --2- 172.21.15.16:0/1561056274 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5b18130750 0x7f5b18126e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:25.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.078+0000 7f5b1c9fe700 1 -- 172.21.15.16:0/1561056274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b18133490 msgr2=0x7f5b18127340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:25.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.078+0000 7f5b1c9fe700 1 --2- 172.21.15.16:0/1561056274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b18133490 0x7f5b18127340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:25.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.079+0000 7f5b1c9fe700 1 -- 172.21.15.16:0/1561056274 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b0c009550 con 0x7f5b18135db0 2023-09-30T22:35:25.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.079+0000 7f5b177fe700 1 --2- 172.21.15.16:0/1561056274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b18133490 0x7f5b18127340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:25.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.079+0000 7f5b1c9fe700 1 --2- 172.21.15.16:0/1561056274 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5b18135db0 0x7f5b18127880 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f5b1806aeb0 tx=0x7f5b10004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:25.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.079+0000 7f5b14ff9700 1 -- 172.21.15.16:0/1561056274 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b10004450 con 0x7f5b18135db0 2023-09-30T22:35:25.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.079+0000 7f5b14ff9700 1 -- 172.21.15.16:0/1561056274 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b1000b450 con 0x7f5b18135db0 2023-09-30T22:35:25.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.079+0000 7f5b1e461700 1 -- 172.21.15.16:0/1561056274 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b1000bc80 con 0x7f5b18135db0 2023-09-30T22:35:25.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.079+0000 7f5b14ff9700 1 -- 172.21.15.16:0/1561056274 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b10004830 con 0x7f5b18135db0 2023-09-30T22:35:25.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.079+0000 7f5b1e461700 1 -- 172.21.15.16:0/1561056274 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b1812a1b0 con 0x7f5b18135db0 2023-09-30T22:35:25.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.080+0000 7f5b1e461700 1 -- 172.21.15.16:0/1561056274 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b180747f0 con 0x7f5b18135db0 2023-09-30T22:35:25.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.080+0000 7f5b14ff9700 1 -- 172.21.15.16:0/1561056274 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5b10009460 con 0x7f5b18135db0 2023-09-30T22:35:25.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.080+0000 7f5b14ff9700 1 -- 172.21.15.16:0/1561056274 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f5b100432b0 con 0x7f5b18135db0 2023-09-30T22:35:25.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.085+0000 7f5b14ff9700 1 -- 172.21.15.16:0/1561056274 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5b100137a0 con 0x7f5b18135db0 2023-09-30T22:35:25.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.287+0000 7f5b1e461700 1 -- 172.21.15.16:0/1561056274 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f5b1812fb00 con 0x7f5b18135db0 2023-09-30T22:35:25.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.287+0000 7f5b14ff9700 1 -- 172.21.15.16:0/1561056274 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5b10035090 con 0x7f5b18135db0 2023-09-30T22:35:25.288 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:35:25.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.289+0000 7f5b1e461700 1 -- 172.21.15.16:0/1561056274 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5b18135db0 msgr2=0x7f5b18127880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:25.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.289+0000 7f5b1e461700 1 --2- 172.21.15.16:0/1561056274 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5b18135db0 0x7f5b18127880 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f5b1806aeb0 tx=0x7f5b10004ad0 comp rx=0 tx=0).stop 2023-09-30T22:35:25.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.289+0000 7f5b1e461700 1 -- 172.21.15.16:0/1561056274 shutdown_connections 2023-09-30T22:35:25.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.289+0000 7f5b1e461700 1 --2- 172.21.15.16:0/1561056274 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5b18130750 0x7f5b18126e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:25.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.289+0000 7f5b1e461700 1 --2- 172.21.15.16:0/1561056274 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5b18135db0 0x7f5b18127880 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:25.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.289+0000 7f5b1e461700 1 --2- 172.21.15.16:0/1561056274 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5b18133490 0x7f5b18127340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:25.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.289+0000 7f5b1e461700 1 -- 172.21.15.16:0/1561056274 >> 172.21.15.16:0/1561056274 conn(0x7f5b1809f470 msgr2=0x7f5b1809c860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:25.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.289+0000 7f5b1e461700 1 -- 172.21.15.16:0/1561056274 shutdown_connections 2023-09-30T22:35:25.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.289+0000 7f5b1e461700 1 -- 172.21.15.16:0/1561056274 wait complete. 2023-09-30T22:35:25.303 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:35:25.364 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:35:25.401 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:35:25.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.506+0000 7f07345f8700 1 Processor -- start 2023-09-30T22:35:25.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.506+0000 7f07345f8700 1 -- start start 2023-09-30T22:35:25.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.507+0000 7f07345f8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f072c0a09c0 0x7f072c06c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:25.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.507+0000 7f07345f8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f072c0a0d90 0x7f072c0a11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:25.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.507+0000 7f07345f8700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f072c0a2a20 con 0x7f072c1338a0 2023-09-30T22:35:25.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.507+0000 7f07345f8700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f072c09e690 con 0x7f072c0a09c0 2023-09-30T22:35:25.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.507+0000 7f07345f8700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f072c0a1cc0 con 0x7f072c0a0d90 2023-09-30T22:35:25.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.507+0000 7f0732394700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f072c0a09c0 0x7f072c06c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:25.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.507+0000 7f0732b95700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f072c1338a0 0x7f072c133d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33636/0 (socket says 172.21.15.16:33636) 2023-09-30T22:35:25.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.507+0000 7f0732b95700 1 -- 172.21.15.16:0/3901897285 learned_addr learned my addr 172.21.15.16:0/3901897285 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:25.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.507+0000 7f0731b93700 1 --2- 172.21.15.16:0/3901897285 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f072c0a0d90 0x7f072c0a11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:25.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.508+0000 7f0731b93700 1 -- 172.21.15.16:0/3901897285 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f072c0a09c0 msgr2=0x7f072c06c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:25.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.508+0000 7f0731b93700 1 --2- 172.21.15.16:0/3901897285 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f072c0a09c0 0x7f072c06c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:25.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.508+0000 7f0731392700 1 -- 172.21.15.16:0/3901897285 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 371354670 0 0) 0x7f0724001490 con 0x7f072c1338a0 2023-09-30T22:35:25.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.508+0000 7f0731b93700 1 -- 172.21.15.16:0/3901897285 >> v1:172.21.15.16:6789/0 conn(0x7f072c1338a0 legacy=0x7f072c133d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:25.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.508+0000 7f0731b93700 1 -- 172.21.15.16:0/3901897285 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f072c0a1f40 con 0x7f072c0a0d90 2023-09-30T22:35:25.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.508+0000 7f0731b93700 1 --2- 172.21.15.16:0/3901897285 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f072c0a0d90 0x7f072c0a11c0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f071c009870 tx=0x7f071c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a4fb4b3a4831725 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:25.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.508+0000 7f0731392700 1 -- 172.21.15.16:0/3901897285 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071c011830 con 0x7f072c0a0d90 2023-09-30T22:35:25.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.508+0000 7f0731392700 1 -- 172.21.15.16:0/3901897285 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f071c011990 con 0x7f072c0a0d90 2023-09-30T22:35:25.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.508+0000 7f0731392700 1 -- 172.21.15.16:0/3901897285 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071c011d20 con 0x7f072c0a0d90 2023-09-30T22:35:25.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.509+0000 7f07345f8700 1 -- 172.21.15.16:0/3901897285 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f072c0a0d90 msgr2=0x7f072c0a11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:25.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.509+0000 7f07345f8700 1 --2- 172.21.15.16:0/3901897285 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f072c0a0d90 0x7f072c0a11c0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f071c009870 tx=0x7f071c009b80 comp rx=0 tx=0).stop 2023-09-30T22:35:25.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.509+0000 7f07345f8700 1 -- 172.21.15.16:0/3901897285 shutdown_connections 2023-09-30T22:35:25.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.509+0000 7f07345f8700 1 --2- 172.21.15.16:0/3901897285 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f072c0a09c0 0x7f072c06c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:25.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.509+0000 7f07345f8700 1 --2- 172.21.15.16:0/3901897285 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f072c0a0d90 0x7f072c0a11c0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:25.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.509+0000 7f07345f8700 1 -- 172.21.15.16:0/3901897285 >> 172.21.15.16:0/3901897285 conn(0x7f072c0a60c0 msgr2=0x7f072c0a64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:25.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.509+0000 7f07345f8700 1 -- 172.21.15.16:0/3901897285 shutdown_connections 2023-09-30T22:35:25.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.509+0000 7f07345f8700 1 -- 172.21.15.16:0/3901897285 wait complete. 2023-09-30T22:35:25.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.510+0000 7f07345f8700 1 Processor -- start 2023-09-30T22:35:25.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.510+0000 7f07345f8700 1 -- start start 2023-09-30T22:35:25.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.511+0000 7f07345f8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f072c0a09c0 0x7f072c11f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:25.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.511+0000 7f07345f8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f072c0a0d90 0x7f072c11f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:25.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.511+0000 7f07345f8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f072c1338a0 0x7f072c11fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:25.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.511+0000 7f07345f8700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072c121ab0 con 0x7f072c1338a0 2023-09-30T22:35:25.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.511+0000 7f07345f8700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f072c120d90 con 0x7f072c0a09c0 2023-09-30T22:35:25.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.511+0000 7f07345f8700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f072c121ed0 con 0x7f072c0a0d90 2023-09-30T22:35:25.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.511+0000 7f0732394700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f072c0a09c0 0x7f072c11f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:25.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.511+0000 7f0732394700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f072c0a09c0 0x7f072c11f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:53088/0 (socket says 172.21.15.16:53088) 2023-09-30T22:35:25.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.511+0000 7f0732394700 1 -- 172.21.15.16:0/2393725366 learned_addr learned my addr 172.21.15.16:0/2393725366 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:25.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.511+0000 7f0731b93700 1 --2- 172.21.15.16:0/2393725366 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f072c0a0d90 0x7f072c11f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:25.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.511+0000 7f0732b95700 1 --2- 172.21.15.16:0/2393725366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f072c1338a0 0x7f072c11fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:25.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.512+0000 7f0732394700 1 -- 172.21.15.16:0/2393725366 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f072c0a0d90 msgr2=0x7f072c11f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:25.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.512+0000 7f0732394700 1 --2- 172.21.15.16:0/2393725366 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f072c0a0d90 0x7f072c11f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:25.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.512+0000 7f0732394700 1 -- 172.21.15.16:0/2393725366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f072c1338a0 msgr2=0x7f072c11fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:25.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.512+0000 7f0732394700 1 --2- 172.21.15.16:0/2393725366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f072c1338a0 0x7f072c11fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:25.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.512+0000 7f0732394700 1 -- 172.21.15.16:0/2393725366 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f071c0094a0 con 0x7f072c0a09c0 2023-09-30T22:35:25.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.512+0000 7f0732394700 1 --2- 172.21.15.16:0/2393725366 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f072c0a09c0 0x7f072c11f090 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f072800e9b0 tx=0x7f072800ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:25.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.512+0000 7f0722ffd700 1 -- 172.21.15.16:0/2393725366 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0728009710 con 0x7f072c0a09c0 2023-09-30T22:35:25.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.512+0000 7f0722ffd700 1 -- 172.21.15.16:0/2393725366 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f072800ee50 con 0x7f072c0a09c0 2023-09-30T22:35:25.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.512+0000 7f07345f8700 1 -- 172.21.15.16:0/2393725366 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f072c09cea0 con 0x7f072c0a09c0 2023-09-30T22:35:25.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.512+0000 7f0722ffd700 1 -- 172.21.15.16:0/2393725366 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072800ca50 con 0x7f072c0a09c0 2023-09-30T22:35:25.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.512+0000 7f07345f8700 1 -- 172.21.15.16:0/2393725366 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f072c1225d0 con 0x7f072c0a09c0 2023-09-30T22:35:25.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.513+0000 7f0722ffd700 1 -- 172.21.15.16:0/2393725366 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f072800cbb0 con 0x7f072c0a09c0 2023-09-30T22:35:25.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.513+0000 7f0722ffd700 1 -- 172.21.15.16:0/2393725366 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f072803f740 con 0x7f072c0a09c0 2023-09-30T22:35:25.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.513+0000 7f07345f8700 1 -- 172.21.15.16:0/2393725366 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f072c09c900 con 0x7f072c0a09c0 2023-09-30T22:35:25.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.518+0000 7f0722ffd700 1 -- 172.21.15.16:0/2393725366 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f07280232f0 con 0x7f072c0a09c0 2023-09-30T22:35:25.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.547+0000 7fb9ca4cf700 1 Processor -- start 2023-09-30T22:35:25.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.547+0000 7fb9ca4cf700 1 -- start start 2023-09-30T22:35:25.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.548+0000 7fb9ca4cf700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb9c40a0a30 0x7fb9c40a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:25.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.548+0000 7fb9ca4cf700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb9c4138d80 0x7fb9c413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:25.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.548+0000 7fb9ca4cf700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9c40a2a70 con 0x7fb9c409d630 2023-09-30T22:35:25.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.548+0000 7fb9ca4cf700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9c40a01d0 con 0x7fb9c40a0a30 2023-09-30T22:35:25.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.548+0000 7fb9ca4cf700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c409e2b0 con 0x7fb9c4138d80 2023-09-30T22:35:25.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.548+0000 7fb9c3fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb9c40a0a30 0x7fb9c40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:25.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.548+0000 7fb9c37fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb9c409d630 0x7fb9c40a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33638/0 (socket says 172.21.15.16:33638) 2023-09-30T22:35:25.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.548+0000 7fb9c37fe700 1 -- 172.21.15.16:0/1770028123 learned_addr learned my addr 172.21.15.16:0/1770028123 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:25.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.548+0000 7fb9c8a6c700 1 --2- 172.21.15.16:0/1770028123 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb9c4138d80 0x7fb9c413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:25.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.548+0000 7fb9c3fff700 1 -- 172.21.15.16:0/1770028123 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb9c4138d80 msgr2=0x7fb9c413b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:25.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.548+0000 7fb9c3fff700 1 --2- 172.21.15.16:0/1770028123 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb9c4138d80 0x7fb9c413b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:25.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.548+0000 7fb9c3fff700 1 -- 172.21.15.16:0/1770028123 >> v1:172.21.15.16:6789/0 conn(0x7fb9c409d630 legacy=0x7fb9c40a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:25.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.548+0000 7fb9c3fff700 1 -- 172.21.15.16:0/1770028123 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c409dc50 con 0x7fb9c40a0a30 2023-09-30T22:35:25.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.548+0000 7fb9c8a6c700 1 --2- 172.21.15.16:0/1770028123 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb9c4138d80 0x7fb9c413b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:25.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.549+0000 7fb9c3fff700 1 --2- 172.21.15.16:0/1770028123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9c40a0a30 0x7fb9c40a0e00 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4009840 tx=0x7fb9b4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d35bf68527d2381 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:25.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.549+0000 7fb9c2ffd700 1 -- 172.21.15.16:0/1770028123 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b4011830 con 0x7fb9c40a0a30 2023-09-30T22:35:25.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.549+0000 7fb9c2ffd700 1 -- 172.21.15.16:0/1770028123 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9b4011990 con 0x7fb9c40a0a30 2023-09-30T22:35:25.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.549+0000 7fb9c2ffd700 1 -- 172.21.15.16:0/1770028123 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b4011d20 con 0x7fb9c40a0a30 2023-09-30T22:35:25.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.549+0000 7fb9ca4cf700 1 -- 172.21.15.16:0/1770028123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9c40a0a30 msgr2=0x7fb9c40a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:25.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.549+0000 7fb9ca4cf700 1 --2- 172.21.15.16:0/1770028123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9c40a0a30 0x7fb9c40a0e00 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fb9b4009840 tx=0x7fb9b4009b50 comp rx=0 tx=0).stop 2023-09-30T22:35:25.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.549+0000 7fb9ca4cf700 1 -- 172.21.15.16:0/1770028123 shutdown_connections 2023-09-30T22:35:25.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.549+0000 7fb9ca4cf700 1 --2- 172.21.15.16:0/1770028123 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb9c4138d80 0x7fb9c413b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:25.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.549+0000 7fb9ca4cf700 1 --2- 172.21.15.16:0/1770028123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9c40a0a30 0x7fb9c40a0e00 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:25.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.549+0000 7fb9ca4cf700 1 -- 172.21.15.16:0/1770028123 >> 172.21.15.16:0/1770028123 conn(0x7fb9c40a3050 msgr2=0x7fb9c40a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:25.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.549+0000 7fb9ca4cf700 1 -- 172.21.15.16:0/1770028123 shutdown_connections 2023-09-30T22:35:25.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.550+0000 7fb9ca4cf700 1 -- 172.21.15.16:0/1770028123 wait complete. 2023-09-30T22:35:25.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.550+0000 7fb9ca4cf700 1 Processor -- start 2023-09-30T22:35:25.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.551+0000 7fb9ca4cf700 1 -- start start 2023-09-30T22:35:25.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.551+0000 7fb9ca4cf700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb9c409d630 0x7fb9c4120700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:25.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.551+0000 7fb9ca4cf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9c40a0a30 0x7fb9c4120c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:25.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.551+0000 7fb9ca4cf700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb9c4138d80 0x7fb9c4121180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:25.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.551+0000 7fb9ca4cf700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c4123110 con 0x7fb9c40a0a30 2023-09-30T22:35:25.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.551+0000 7fb9ca4cf700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c4122be0 con 0x7fb9c4138d80 2023-09-30T22:35:25.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.551+0000 7fb9ca4cf700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c4122310 con 0x7fb9c409d630 2023-09-30T22:35:25.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.551+0000 7fb9c37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9c40a0a30 0x7fb9c4120c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:25.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.551+0000 7fb9c3fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb9c409d630 0x7fb9c4120700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:25.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.551+0000 7fb9c37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9c40a0a30 0x7fb9c4120c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55614/0 (socket says 172.21.15.16:55614) 2023-09-30T22:35:25.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.551+0000 7fb9c37fe700 1 -- 172.21.15.16:0/1221319213 learned_addr learned my addr 172.21.15.16:0/1221319213 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:25.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.551+0000 7fb9c8a6c700 1 --2- 172.21.15.16:0/1221319213 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb9c4138d80 0x7fb9c4121180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:25.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.552+0000 7fb9c37fe700 1 -- 172.21.15.16:0/1221319213 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb9c409d630 msgr2=0x7fb9c4120700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:25.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.552+0000 7fb9c37fe700 1 --2- 172.21.15.16:0/1221319213 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb9c409d630 0x7fb9c4120700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:25.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.552+0000 7fb9c37fe700 1 -- 172.21.15.16:0/1221319213 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb9c4138d80 msgr2=0x7fb9c4121180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:25.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.552+0000 7fb9c37fe700 1 --2- 172.21.15.16:0/1221319213 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb9c4138d80 0x7fb9c4121180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:25.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.552+0000 7fb9c37fe700 1 -- 172.21.15.16:0/1221319213 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9bc009a70 con 0x7fb9c40a0a30 2023-09-30T22:35:25.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.552+0000 7fb9c3fff700 1 --2- 172.21.15.16:0/1221319213 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb9c409d630 0x7fb9c4120700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:25.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.552+0000 7fb9c37fe700 1 --2- 172.21.15.16:0/1221319213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9c40a0a30 0x7fb9c4120c40 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fb9b800c550 tx=0x7fb9b800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:25.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.552+0000 7fb9c0ff9700 1 -- 172.21.15.16:0/1221319213 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b8005d70 con 0x7fb9c40a0a30 2023-09-30T22:35:25.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.552+0000 7fb9c0ff9700 1 -- 172.21.15.16:0/1221319213 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9b80070d0 con 0x7fb9c40a0a30 2023-09-30T22:35:25.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.552+0000 7fb9ca4cf700 1 -- 172.21.15.16:0/1221319213 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9b40094a0 con 0x7fb9c40a0a30 2023-09-30T22:35:25.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.552+0000 7fb9c0ff9700 1 -- 172.21.15.16:0/1221319213 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b8007440 con 0x7fb9c40a0a30 2023-09-30T22:35:25.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.552+0000 7fb9ca4cf700 1 -- 172.21.15.16:0/1221319213 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9c40a4ec0 con 0x7fb9c40a0a30 2023-09-30T22:35:25.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.553+0000 7fb9c0ff9700 1 -- 172.21.15.16:0/1221319213 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb9b80075a0 con 0x7fb9c40a0a30 2023-09-30T22:35:25.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.553+0000 7fb9c0ff9700 1 -- 172.21.15.16:0/1221319213 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fb9b803eaf0 con 0x7fb9c40a0a30 2023-09-30T22:35:25.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.553+0000 7fb9ca4cf700 1 -- 172.21.15.16:0/1221319213 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9c4123be0 con 0x7fb9c40a0a30 2023-09-30T22:35:25.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.558+0000 7fb9c0ff9700 1 -- 172.21.15.16:0/1221319213 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb9c4123be0 con 0x7fb9c40a0a30 2023-09-30T22:35:25.667 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:35:25.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.736+0000 7f07345f8700 1 -- 172.21.15.16:0/2393725366 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f072c122760 con 0x7f072c0a09c0 2023-09-30T22:35:25.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.737+0000 7f0722ffd700 1 -- 172.21.15.16:0/2393725366 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0728017020 con 0x7f072c0a09c0 2023-09-30T22:35:25.737 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:35:25.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.739+0000 7f07345f8700 1 -- 172.21.15.16:0/2393725366 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f072c0a09c0 msgr2=0x7f072c11f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:25.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.739+0000 7f07345f8700 1 --2- 172.21.15.16:0/2393725366 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f072c0a09c0 0x7f072c11f090 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f072800e9b0 tx=0x7f072800ecc0 comp rx=0 tx=0).stop 2023-09-30T22:35:25.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.739+0000 7f07345f8700 1 -- 172.21.15.16:0/2393725366 shutdown_connections 2023-09-30T22:35:25.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.739+0000 7f07345f8700 1 --2- 172.21.15.16:0/2393725366 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f072c0a09c0 0x7f072c11f090 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:25.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.739+0000 7f07345f8700 1 --2- 172.21.15.16:0/2393725366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f072c1338a0 0x7f072c11fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:25.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.739+0000 7f07345f8700 1 --2- 172.21.15.16:0/2393725366 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f072c0a0d90 0x7f072c11f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:25.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.739+0000 7f07345f8700 1 -- 172.21.15.16:0/2393725366 >> 172.21.15.16:0/2393725366 conn(0x7f072c0a60c0 msgr2=0x7f072c099a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:25.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.739+0000 7f07345f8700 1 -- 172.21.15.16:0/2393725366 shutdown_connections 2023-09-30T22:35:25.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.739+0000 7f07345f8700 1 -- 172.21.15.16:0/2393725366 wait complete. 2023-09-30T22:35:25.754 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:35:25.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.766+0000 7fb9ca4cf700 1 -- 172.21.15.16:0/1221319213 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb9c4123be0 con 0x7fb9c40a0a30 2023-09-30T22:35:25.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.767+0000 7fb9c0ff9700 1 -- 172.21.15.16:0/1221319213 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb9c4123be0 con 0x7fb9c40a0a30 2023-09-30T22:35:25.767 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:35:25.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.769+0000 7fb9ca4cf700 1 -- 172.21.15.16:0/1221319213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9c40a0a30 msgr2=0x7fb9c4120c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:25.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.769+0000 7fb9ca4cf700 1 --2- 172.21.15.16:0/1221319213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9c40a0a30 0x7fb9c4120c40 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fb9b800c550 tx=0x7fb9b800c860 comp rx=0 tx=0).stop 2023-09-30T22:35:25.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.769+0000 7fb9ca4cf700 1 -- 172.21.15.16:0/1221319213 shutdown_connections 2023-09-30T22:35:25.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.769+0000 7fb9ca4cf700 1 --2- 172.21.15.16:0/1221319213 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb9c4138d80 0x7fb9c4121180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:25.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.769+0000 7fb9ca4cf700 1 --2- 172.21.15.16:0/1221319213 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb9c409d630 0x7fb9c4120700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:25.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.769+0000 7fb9ca4cf700 1 --2- 172.21.15.16:0/1221319213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9c40a0a30 0x7fb9c4120c40 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:25.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.769+0000 7fb9ca4cf700 1 -- 172.21.15.16:0/1221319213 >> 172.21.15.16:0/1221319213 conn(0x7fb9c40a3050 msgr2=0x7fb9c409ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:25.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.769+0000 7fb9ca4cf700 1 -- 172.21.15.16:0/1221319213 shutdown_connections 2023-09-30T22:35:25.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.769+0000 7fb9ca4cf700 1 -- 172.21.15.16:0/1221319213 wait complete. 2023-09-30T22:35:25.783 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:35:25.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.809+0000 7fcece583700 1 Processor -- start 2023-09-30T22:35:25.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.809+0000 7fcece583700 1 -- start start 2023-09-30T22:35:25.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.810+0000 7fcece583700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcec80a09c0 0x7fcec80a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:25.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.810+0000 7fcece583700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcec80a12d0 0x7fcec8134030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:25.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.810+0000 7fcece583700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcec809ea10 con 0x7fcec809d5c0 2023-09-30T22:35:25.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.810+0000 7fcece583700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcec809e210 con 0x7fcec80a12d0 2023-09-30T22:35:25.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.810+0000 7fcece583700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcec80a26f0 con 0x7fcec80a09c0 2023-09-30T22:35:25.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.810+0000 7fcec77fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fcec809d5c0 0x7fcec8133920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33650/0 (socket says 172.21.15.16:33650) 2023-09-30T22:35:25.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.810+0000 7fcec7fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcec80a09c0 0x7fcec80a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:25.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.810+0000 7fcec77fe700 1 -- 172.21.15.16:0/2826321513 learned_addr learned my addr 172.21.15.16:0/2826321513 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:25.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.810+0000 7fceccb20700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcec80a12d0 0x7fcec8134030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:25.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.810+0000 7fcec6ffd700 1 -- 172.21.15.16:0/2826321513 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3777394938 0 0) 0x7fcec809ea10 con 0x7fcec809d5c0 2023-09-30T22:35:25.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.810+0000 7fcec6ffd700 1 -- 172.21.15.16:0/2826321513 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fceb00035b0 con 0x7fcec809d5c0 2023-09-30T22:35:25.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.810+0000 7fceccb20700 1 -- 172.21.15.16:0/2826321513 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcec80a09c0 msgr2=0x7fcec80a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:25.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.810+0000 7fceccb20700 1 --2- 172.21.15.16:0/2826321513 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcec80a09c0 0x7fcec80a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:25.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.810+0000 7fceccb20700 1 -- 172.21.15.16:0/2826321513 >> v1:172.21.15.16:6789/0 conn(0x7fcec809d5c0 legacy=0x7fcec8133920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:25.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.810+0000 7fceccb20700 1 -- 172.21.15.16:0/2826321513 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcec809db70 con 0x7fcec80a12d0 2023-09-30T22:35:25.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.810+0000 7fcec7fff700 1 --2- 172.21.15.16:0/2826321513 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcec80a09c0 0x7fcec80a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:25.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.810+0000 7fceccb20700 1 --2- 172.21.15.16:0/2826321513 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcec80a12d0 0x7fcec8134030 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fcec0009dc0 tx=0x7fcec000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75ad65923f1b14cd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:25.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.811+0000 7fcec6ffd700 1 -- 172.21.15.16:0/2826321513 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec000e070 con 0x7fcec80a12d0 2023-09-30T22:35:25.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.811+0000 7fcec6ffd700 1 -- 172.21.15.16:0/2826321513 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcec000bb20 con 0x7fcec80a12d0 2023-09-30T22:35:25.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.811+0000 7fcec6ffd700 1 -- 172.21.15.16:0/2826321513 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec0012430 con 0x7fcec80a12d0 2023-09-30T22:35:25.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.811+0000 7fcece583700 1 -- 172.21.15.16:0/2826321513 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcec80a12d0 msgr2=0x7fcec8134030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:25.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.811+0000 7fcece583700 1 --2- 172.21.15.16:0/2826321513 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcec80a12d0 0x7fcec8134030 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fcec0009dc0 tx=0x7fcec000b330 comp rx=0 tx=0).stop 2023-09-30T22:35:25.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.811+0000 7fcece583700 1 -- 172.21.15.16:0/2826321513 shutdown_connections 2023-09-30T22:35:25.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.811+0000 7fcece583700 1 --2- 172.21.15.16:0/2826321513 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcec80a12d0 0x7fcec8134030 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:25.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.811+0000 7fcece583700 1 --2- 172.21.15.16:0/2826321513 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcec80a09c0 0x7fcec80a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:25.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.811+0000 7fcece583700 1 -- 172.21.15.16:0/2826321513 >> 172.21.15.16:0/2826321513 conn(0x7fcec80a2fe0 msgr2=0x7fcec80a33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:25.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.811+0000 7fcece583700 1 -- 172.21.15.16:0/2826321513 shutdown_connections 2023-09-30T22:35:25.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.811+0000 7fcece583700 1 -- 172.21.15.16:0/2826321513 wait complete. 2023-09-30T22:35:25.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.812+0000 7fcece583700 1 Processor -- start 2023-09-30T22:35:25.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.813+0000 7fcece583700 1 -- start start 2023-09-30T22:35:25.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.813+0000 7fcece583700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcec809d5c0 0x7fcec81c8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:25.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.813+0000 7fcece583700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcec80a09c0 0x7fcec81c85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:25.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.813+0000 7fcece583700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcec80a12d0 0x7fcec81c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:25.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.813+0000 7fcece583700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec8121830 con 0x7fcec80a09c0 2023-09-30T22:35:25.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.813+0000 7fcece583700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcec809c340 con 0x7fcec80a12d0 2023-09-30T22:35:25.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.813+0000 7fcece583700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcec809cec0 con 0x7fcec809d5c0 2023-09-30T22:35:25.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.813+0000 7fcec77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcec80a09c0 0x7fcec81c85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:25.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.813+0000 7fceccb20700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcec80a12d0 0x7fcec81c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:25.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.813+0000 7fceccb20700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcec80a12d0 0x7fcec81c8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:53116/0 (socket says 172.21.15.16:53116) 2023-09-30T22:35:25.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.813+0000 7fcec7fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcec809d5c0 0x7fcec81c8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:25.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.813+0000 7fceccb20700 1 -- 172.21.15.16:0/4082507235 learned_addr learned my addr 172.21.15.16:0/4082507235 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:25.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.813+0000 7fcec77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcec80a09c0 0x7fcec81c85c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55618/0 (socket says 172.21.15.16:55618) 2023-09-30T22:35:25.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.813+0000 7fceccb20700 1 -- 172.21.15.16:0/4082507235 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcec809d5c0 msgr2=0x7fcec81c8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:25.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.813+0000 7fceccb20700 1 --2- 172.21.15.16:0/4082507235 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcec809d5c0 0x7fcec81c8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:25.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.814+0000 7fceccb20700 1 -- 172.21.15.16:0/4082507235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcec80a09c0 msgr2=0x7fcec81c85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:25.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.814+0000 7fceccb20700 1 --2- 172.21.15.16:0/4082507235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcec80a09c0 0x7fcec81c85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:25.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.814+0000 7fceccb20700 1 -- 172.21.15.16:0/4082507235 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceb80094a0 con 0x7fcec80a12d0 2023-09-30T22:35:25.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.814+0000 7fcec7fff700 1 --2- 172.21.15.16:0/4082507235 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcec809d5c0 0x7fcec81c8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:25.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.814+0000 7fceccb20700 1 --2- 172.21.15.16:0/4082507235 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcec80a12d0 0x7fcec81c8b00 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fcec0014040 tx=0x7fcec0004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:25.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.814+0000 7fcec4ff9700 1 -- 172.21.15.16:0/4082507235 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec0012430 con 0x7fcec80a12d0 2023-09-30T22:35:25.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.814+0000 7fcec4ff9700 1 -- 172.21.15.16:0/4082507235 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcec0020070 con 0x7fcec80a12d0 2023-09-30T22:35:25.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.814+0000 7fcece583700 1 -- 172.21.15.16:0/4082507235 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcec0009a70 con 0x7fcec80a12d0 2023-09-30T22:35:25.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.814+0000 7fcec4ff9700 1 -- 172.21.15.16:0/4082507235 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec0004280 con 0x7fcec80a12d0 2023-09-30T22:35:25.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.814+0000 7fcece583700 1 -- 172.21.15.16:0/4082507235 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcec8123390 con 0x7fcec80a12d0 2023-09-30T22:35:25.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.815+0000 7fcec4ff9700 1 -- 172.21.15.16:0/4082507235 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcec00043e0 con 0x7fcec80a12d0 2023-09-30T22:35:25.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.815+0000 7fcece583700 1 -- 172.21.15.16:0/4082507235 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcec81345b0 con 0x7fcec80a12d0 2023-09-30T22:35:25.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.815+0000 7fcec4ff9700 1 -- 172.21.15.16:0/4082507235 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fcec0022070 con 0x7fcec80a12d0 2023-09-30T22:35:25.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:25.820+0000 7fcec4ff9700 1 -- 172.21.15.16:0/4082507235 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcec0004690 con 0x7fcec80a12d0 2023-09-30T22:35:26.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.023+0000 7fcece583700 1 -- 172.21.15.16:0/4082507235 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fcec8123140 con 0x7fcec80a12d0 2023-09-30T22:35:26.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.024+0000 7fcec4ff9700 1 -- 172.21.15.16:0/4082507235 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcec002c030 con 0x7fcec80a12d0 2023-09-30T22:35:26.024 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:35:26.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.026+0000 7fcece583700 1 -- 172.21.15.16:0/4082507235 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcec80a12d0 msgr2=0x7fcec81c8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:26.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.026+0000 7fcece583700 1 --2- 172.21.15.16:0/4082507235 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcec80a12d0 0x7fcec81c8b00 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fcec0014040 tx=0x7fcec0004ad0 comp rx=0 tx=0).stop 2023-09-30T22:35:26.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.026+0000 7fcece583700 1 -- 172.21.15.16:0/4082507235 shutdown_connections 2023-09-30T22:35:26.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.026+0000 7fcece583700 1 --2- 172.21.15.16:0/4082507235 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcec80a12d0 0x7fcec81c8b00 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:26.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.026+0000 7fcece583700 1 --2- 172.21.15.16:0/4082507235 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcec809d5c0 0x7fcec81c8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:26.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.026+0000 7fcece583700 1 --2- 172.21.15.16:0/4082507235 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcec80a09c0 0x7fcec81c85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:26.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.026+0000 7fcece583700 1 -- 172.21.15.16:0/4082507235 >> 172.21.15.16:0/4082507235 conn(0x7fcec80a2fe0 msgr2=0x7fcec80a5550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:26.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.026+0000 7fcece583700 1 -- 172.21.15.16:0/4082507235 shutdown_connections 2023-09-30T22:35:26.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.026+0000 7fcece583700 1 -- 172.21.15.16:0/4082507235 wait complete. 2023-09-30T22:35:26.040 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:35:26.304 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:35:26.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.451+0000 7efcc392b700 1 Processor -- start 2023-09-30T22:35:26.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.452+0000 7efcc392b700 1 -- start start 2023-09-30T22:35:26.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.452+0000 7efcc392b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efcbc12c9a0 0x7efcbc12cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:26.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.452+0000 7efcc392b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efcbc12f880 0x7efcbc12fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:26.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.452+0000 7efcc392b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efcbc1301b0 0x7efcbc136c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:26.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.452+0000 7efcc392b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcbc09a420 con 0x7efcbc12f880 2023-09-30T22:35:26.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.452+0000 7efcc392b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcbc09a870 con 0x7efcbc1301b0 2023-09-30T22:35:26.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.452+0000 7efcc392b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efcbc09b890 con 0x7efcbc12c9a0 2023-09-30T22:35:26.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.452+0000 7efcc0ec6700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efcbc12f880 0x7efcbc12fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:26.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.452+0000 7efcc1ec8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efcbc1301b0 0x7efcbc136c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:26.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.452+0000 7efcc16c7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efcbc12c9a0 0x7efcbc12cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:26.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.452+0000 7efcc1ec8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efcbc1301b0 0x7efcbc136c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:53130/0 (socket says 172.21.15.16:53130) 2023-09-30T22:35:26.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.452+0000 7efcc0ec6700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efcbc12f880 0x7efcbc12fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55624/0 (socket says 172.21.15.16:55624) 2023-09-30T22:35:26.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.452+0000 7efcc1ec8700 1 -- 172.21.15.16:0/4146641364 learned_addr learned my addr 172.21.15.16:0/4146641364 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:26.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.453+0000 7efcc0ec6700 1 -- 172.21.15.16:0/4146641364 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efcbc12c9a0 msgr2=0x7efcbc12cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:26.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.453+0000 7efcc0ec6700 1 --2- 172.21.15.16:0/4146641364 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efcbc12c9a0 0x7efcbc12cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:26.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.453+0000 7efcc0ec6700 1 -- 172.21.15.16:0/4146641364 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efcbc1301b0 msgr2=0x7efcbc136c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:26.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.453+0000 7efcc0ec6700 1 --2- 172.21.15.16:0/4146641364 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efcbc1301b0 0x7efcbc136c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:26.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.453+0000 7efcc0ec6700 1 -- 172.21.15.16:0/4146641364 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcbc09b2f0 con 0x7efcbc12f880 2023-09-30T22:35:26.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.453+0000 7efcc1ec8700 1 --2- 172.21.15.16:0/4146641364 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efcbc1301b0 0x7efcbc136c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:26.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.453+0000 7efcc16c7700 1 --2- 172.21.15.16:0/4146641364 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efcbc12c9a0 0x7efcbc12cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:26.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.453+0000 7efcc0ec6700 1 --2- 172.21.15.16:0/4146641364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcbc12f880 0x7efcbc12fc70 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7efcac009840 tx=0x7efcac009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ff480929a792865 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:26.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.453+0000 7efcb3fff700 1 -- 172.21.15.16:0/4146641364 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcac00f8a0 con 0x7efcbc12f880 2023-09-30T22:35:26.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.453+0000 7efcb3fff700 1 -- 172.21.15.16:0/4146641364 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcac00fa00 con 0x7efcbc12f880 2023-09-30T22:35:26.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.453+0000 7efcb3fff700 1 -- 172.21.15.16:0/4146641364 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcac00fd90 con 0x7efcbc12f880 2023-09-30T22:35:26.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.454+0000 7efcc392b700 1 -- 172.21.15.16:0/4146641364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcbc12f880 msgr2=0x7efcbc12fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:26.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.454+0000 7efcc392b700 1 --2- 172.21.15.16:0/4146641364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcbc12f880 0x7efcbc12fc70 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7efcac009840 tx=0x7efcac009b50 comp rx=0 tx=0).stop 2023-09-30T22:35:26.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.454+0000 7efcc392b700 1 -- 172.21.15.16:0/4146641364 shutdown_connections 2023-09-30T22:35:26.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.454+0000 7efcc392b700 1 --2- 172.21.15.16:0/4146641364 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efcbc1301b0 0x7efcbc136c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:26.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.454+0000 7efcc392b700 1 --2- 172.21.15.16:0/4146641364 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efcbc12c9a0 0x7efcbc12cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:26.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.454+0000 7efcc392b700 1 --2- 172.21.15.16:0/4146641364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcbc12f880 0x7efcbc12fc70 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:26.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.454+0000 7efcc392b700 1 -- 172.21.15.16:0/4146641364 >> 172.21.15.16:0/4146641364 conn(0x7efcbc0a2360 msgr2=0x7efcbc0a2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:26.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.454+0000 7efcc392b700 1 -- 172.21.15.16:0/4146641364 shutdown_connections 2023-09-30T22:35:26.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.454+0000 7efcc392b700 1 -- 172.21.15.16:0/4146641364 wait complete. 2023-09-30T22:35:26.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.455+0000 7efcc392b700 1 Processor -- start 2023-09-30T22:35:26.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.455+0000 7efcc392b700 1 -- start start 2023-09-30T22:35:26.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.455+0000 7efcc392b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efcbc12c9a0 0x7efcbc1c4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:26.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.455+0000 7efcc392b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efcbc12f880 0x7efcbc1c54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:26.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.455+0000 7efcc392b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcbc1301b0 0x7efcbc1c82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:26.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.455+0000 7efcc392b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcbc098fd0 con 0x7efcbc1301b0 2023-09-30T22:35:26.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.455+0000 7efcc392b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcbc09ad80 con 0x7efcbc12c9a0 2023-09-30T22:35:26.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.455+0000 7efcc392b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efcbc1c4d40 con 0x7efcbc12f880 2023-09-30T22:35:26.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.455+0000 7efcc0ec6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efcbc12f880 0x7efcbc1c54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:26.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.455+0000 7efcc1ec8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcbc1301b0 0x7efcbc1c82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:26.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.455+0000 7efcc0ec6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efcbc12f880 0x7efcbc1c54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:33364/0 (socket says 172.21.15.16:33364) 2023-09-30T22:35:26.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.455+0000 7efcc0ec6700 1 -- 172.21.15.16:0/1777320784 learned_addr learned my addr 172.21.15.16:0/1777320784 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:26.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.455+0000 7efcc16c7700 1 --2- 172.21.15.16:0/1777320784 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efcbc12c9a0 0x7efcbc1c4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:26.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.456+0000 7efcc16c7700 1 -- 172.21.15.16:0/1777320784 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efcbc12f880 msgr2=0x7efcbc1c54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:26.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.456+0000 7efcc16c7700 1 --2- 172.21.15.16:0/1777320784 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efcbc12f880 0x7efcbc1c54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:26.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.456+0000 7efcc16c7700 1 -- 172.21.15.16:0/1777320784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcbc1301b0 msgr2=0x7efcbc1c82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:26.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.456+0000 7efcc16c7700 1 --2- 172.21.15.16:0/1777320784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcbc1301b0 0x7efcbc1c82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:26.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.456+0000 7efcc16c7700 1 -- 172.21.15.16:0/1777320784 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcb80094a0 con 0x7efcbc12c9a0 2023-09-30T22:35:26.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.456+0000 7efcc1ec8700 1 --2- 172.21.15.16:0/1777320784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcbc1301b0 0x7efcbc1c82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:26.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.456+0000 7efcc0ec6700 1 --2- 172.21.15.16:0/1777320784 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efcbc12f880 0x7efcbc1c54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:26.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.456+0000 7efcc16c7700 1 --2- 172.21.15.16:0/1777320784 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efcbc12c9a0 0x7efcbc1c4fa0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7efcb800b3f0 tx=0x7efcb800b700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:26.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.456+0000 7efcb1ffb700 1 -- 172.21.15.16:0/1777320784 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb8019070 con 0x7efcbc12c9a0 2023-09-30T22:35:26.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.456+0000 7efcb1ffb700 1 -- 172.21.15.16:0/1777320784 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcb800be70 con 0x7efcbc12c9a0 2023-09-30T22:35:26.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.456+0000 7efcc392b700 1 -- 172.21.15.16:0/1777320784 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcb400ac40 con 0x7efcbc12c9a0 2023-09-30T22:35:26.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.456+0000 7efcb1ffb700 1 -- 172.21.15.16:0/1777320784 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb801d610 con 0x7efcbc12c9a0 2023-09-30T22:35:26.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.456+0000 7efcc392b700 1 -- 172.21.15.16:0/1777320784 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcac0094a0 con 0x7efcbc12c9a0 2023-09-30T22:35:26.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.457+0000 7efcb1ffb700 1 -- 172.21.15.16:0/1777320784 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7efcb801d770 con 0x7efcbc12c9a0 2023-09-30T22:35:26.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.457+0000 7efcc392b700 1 -- 172.21.15.16:0/1777320784 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcbc09fb40 con 0x7efcbc12c9a0 2023-09-30T22:35:26.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.457+0000 7efcb1ffb700 1 -- 172.21.15.16:0/1777320784 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7efcb80473a0 con 0x7efcbc12c9a0 2023-09-30T22:35:26.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.462+0000 7efcb1ffb700 1 -- 172.21.15.16:0/1777320784 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7efcb801dcb0 con 0x7efcbc12c9a0 2023-09-30T22:35:26.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.666+0000 7efcc392b700 1 -- 172.21.15.16:0/1777320784 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7efcbc08bd80 con 0x7efcbc12c9a0 2023-09-30T22:35:26.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.666+0000 7efcb1ffb700 1 -- 172.21.15.16:0/1777320784 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7efcb8022070 con 0x7efcbc12c9a0 2023-09-30T22:35:26.667 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:35:26.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.668+0000 7efcc392b700 1 -- 172.21.15.16:0/1777320784 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efcbc12c9a0 msgr2=0x7efcbc1c4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:26.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.668+0000 7efcc392b700 1 --2- 172.21.15.16:0/1777320784 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efcbc12c9a0 0x7efcbc1c4fa0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7efcb800b3f0 tx=0x7efcb800b700 comp rx=0 tx=0).stop 2023-09-30T22:35:26.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.668+0000 7efcc392b700 1 -- 172.21.15.16:0/1777320784 shutdown_connections 2023-09-30T22:35:26.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.668+0000 7efcc392b700 1 --2- 172.21.15.16:0/1777320784 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efcbc12c9a0 0x7efcbc1c4fa0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:26.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.668+0000 7efcc392b700 1 --2- 172.21.15.16:0/1777320784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efcbc1301b0 0x7efcbc1c82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:26.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.668+0000 7efcc392b700 1 --2- 172.21.15.16:0/1777320784 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efcbc12f880 0x7efcbc1c54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:26.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.669+0000 7efcc392b700 1 -- 172.21.15.16:0/1777320784 >> 172.21.15.16:0/1777320784 conn(0x7efcbc0a2360 msgr2=0x7efcbc1b58f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:26.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.669+0000 7efcc392b700 1 -- 172.21.15.16:0/1777320784 shutdown_connections 2023-09-30T22:35:26.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.669+0000 7efcc392b700 1 -- 172.21.15.16:0/1777320784 wait complete. 2023-09-30T22:35:26.683 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:35:26.755 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:35:26.784 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:35:26.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.901+0000 7fd4f7925700 1 Processor -- start 2023-09-30T22:35:26.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.901+0000 7fd4f7925700 1 -- start start 2023-09-30T22:35:26.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.902+0000 7fd4f7925700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd4f0133a80 0x7fd4f0133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:26.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.902+0000 7fd4f7925700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd4f0130a70 0x7fd4f013b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:26.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.902+0000 7fd4f7925700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4f006bc00 con 0x7fd4f012ff20 2023-09-30T22:35:26.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.902+0000 7fd4f7925700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4f006c570 con 0x7fd4f0133a80 2023-09-30T22:35:26.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.902+0000 7fd4f7925700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f00a0040 con 0x7fd4f0130a70 2023-09-30T22:35:26.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.902+0000 7fd4f5ec2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd4f0130a70 0x7fd4f013b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:26.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.902+0000 7fd4f56c1700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd4f0133a80 0x7fd4f0133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:26.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.902+0000 7fd4f5ec2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd4f0130a70 0x7fd4f013b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:33366/0 (socket says 172.21.15.16:33366) 2023-09-30T22:35:26.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.902+0000 7fd4f5ec2700 1 -- 172.21.15.16:0/2829034047 learned_addr learned my addr 172.21.15.16:0/2829034047 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:26.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.902+0000 7fd4f56c1700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd4f0133a80 0x7fd4f0133e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55642/0 (socket says 172.21.15.16:55642) 2023-09-30T22:35:26.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.903+0000 7fd4f56c1700 1 -- 172.21.15.16:0/2829034047 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd4f0130a70 msgr2=0x7fd4f013b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:26.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.903+0000 7fd4f56c1700 1 --2- 172.21.15.16:0/2829034047 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd4f0130a70 0x7fd4f013b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:26.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.903+0000 7fd4f56c1700 1 -- 172.21.15.16:0/2829034047 >> v1:172.21.15.16:6789/0 conn(0x7fd4f012ff20 legacy=0x7fd4f0130360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:26.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.903+0000 7fd4f56c1700 1 -- 172.21.15.16:0/2829034047 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f0133860 con 0x7fd4f0133a80 2023-09-30T22:35:26.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.903+0000 7fd4e7fff700 1 -- 172.21.15.16:0/2829034047 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 157666625 0 0) 0x7fd4f006bc00 con 0x7fd4f012ff20 2023-09-30T22:35:26.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.903+0000 7fd4f5ec2700 1 --2- 172.21.15.16:0/2829034047 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd4f0130a70 0x7fd4f013b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:26.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.903+0000 7fd4f56c1700 1 --2- 172.21.15.16:0/2829034047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f0133a80 0x7fd4f0133e50 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec009840 tx=0x7fd4ec009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a345030a0283b38e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:26.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.903+0000 7fd4e7fff700 1 -- 172.21.15.16:0/2829034047 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec011830 con 0x7fd4f0133a80 2023-09-30T22:35:26.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.903+0000 7fd4e7fff700 1 -- 172.21.15.16:0/2829034047 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4ec011990 con 0x7fd4f0133a80 2023-09-30T22:35:26.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.903+0000 7fd4e7fff700 1 -- 172.21.15.16:0/2829034047 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4ec011d20 con 0x7fd4f0133a80 2023-09-30T22:35:26.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.904+0000 7fd4f7925700 1 -- 172.21.15.16:0/2829034047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f0133a80 msgr2=0x7fd4f0133e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:26.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.904+0000 7fd4f7925700 1 --2- 172.21.15.16:0/2829034047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f0133a80 0x7fd4f0133e50 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec009840 tx=0x7fd4ec009b50 comp rx=0 tx=0).stop 2023-09-30T22:35:26.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.904+0000 7fd4f7925700 1 -- 172.21.15.16:0/2829034047 shutdown_connections 2023-09-30T22:35:26.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.904+0000 7fd4f7925700 1 --2- 172.21.15.16:0/2829034047 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd4f0130a70 0x7fd4f013b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:26.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.904+0000 7fd4f7925700 1 --2- 172.21.15.16:0/2829034047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f0133a80 0x7fd4f0133e50 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:26.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.904+0000 7fd4f7925700 1 -- 172.21.15.16:0/2829034047 >> 172.21.15.16:0/2829034047 conn(0x7fd4f00a4210 msgr2=0x7fd4f00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:26.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.904+0000 7fd4f7925700 1 -- 172.21.15.16:0/2829034047 shutdown_connections 2023-09-30T22:35:26.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.904+0000 7fd4f7925700 1 -- 172.21.15.16:0/2829034047 wait complete. 2023-09-30T22:35:26.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.905+0000 7fd4f7925700 1 Processor -- start 2023-09-30T22:35:26.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.905+0000 7fd4f7925700 1 -- start start 2023-09-30T22:35:26.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.905+0000 7fd4f7925700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4f012ff20 0x7fd4f009c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:26.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.905+0000 7fd4f7925700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd4f0130a70 0x7fd4f009c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:26.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.905+0000 7fd4f7925700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f0133a80 0x7fd4f009cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:26.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.905+0000 7fd4f7925700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f011f180 con 0x7fd4f0133a80 2023-09-30T22:35:26.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.905+0000 7fd4f7925700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f012da40 con 0x7fd4f012ff20 2023-09-30T22:35:26.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.905+0000 7fd4f7925700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f012dc30 con 0x7fd4f0130a70 2023-09-30T22:35:26.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.905+0000 7fd4f5ec2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f0133a80 0x7fd4f009cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:26.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.905+0000 7fd4f5ec2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f0133a80 0x7fd4f009cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55658/0 (socket says 172.21.15.16:55658) 2023-09-30T22:35:26.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.906+0000 7fd4f5ec2700 1 -- 172.21.15.16:0/2207829130 learned_addr learned my addr 172.21.15.16:0/2207829130 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:26.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.906+0000 7fd4f4ec0700 1 --2- 172.21.15.16:0/2207829130 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd4f0130a70 0x7fd4f009c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:26.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.906+0000 7fd4f56c1700 1 --2- 172.21.15.16:0/2207829130 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4f012ff20 0x7fd4f009c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:26.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.906+0000 7fd4f5ec2700 1 -- 172.21.15.16:0/2207829130 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd4f0130a70 msgr2=0x7fd4f009c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:26.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.906+0000 7fd4f5ec2700 1 --2- 172.21.15.16:0/2207829130 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd4f0130a70 0x7fd4f009c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:26.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.906+0000 7fd4f5ec2700 1 -- 172.21.15.16:0/2207829130 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4f012ff20 msgr2=0x7fd4f009c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:26.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.906+0000 7fd4f5ec2700 1 --2- 172.21.15.16:0/2207829130 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4f012ff20 0x7fd4f009c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:26.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.906+0000 7fd4f5ec2700 1 -- 172.21.15.16:0/2207829130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e8009b30 con 0x7fd4f0133a80 2023-09-30T22:35:26.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.906+0000 7fd4f5ec2700 1 --2- 172.21.15.16:0/2207829130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f0133a80 0x7fd4f009cf00 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fd4e800efc0 tx=0x7fd4e800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:26.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.906+0000 7fd4e5ffb700 1 -- 172.21.15.16:0/2207829130 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e8018070 con 0x7fd4f0133a80 2023-09-30T22:35:26.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.906+0000 7fd4e5ffb700 1 -- 172.21.15.16:0/2207829130 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4e800f040 con 0x7fd4f0133a80 2023-09-30T22:35:26.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.906+0000 7fd4f7925700 1 -- 172.21.15.16:0/2207829130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4ec0094a0 con 0x7fd4f0133a80 2023-09-30T22:35:26.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.906+0000 7fd4e5ffb700 1 -- 172.21.15.16:0/2207829130 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e8003c20 con 0x7fd4f0133a80 2023-09-30T22:35:26.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.907+0000 7fd4f7925700 1 -- 172.21.15.16:0/2207829130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4f011fbf0 con 0x7fd4f0133a80 2023-09-30T22:35:26.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.907+0000 7fd4e5ffb700 1 -- 172.21.15.16:0/2207829130 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd4e800c880 con 0x7fd4f0133a80 2023-09-30T22:35:26.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.907+0000 7fd4e5ffb700 1 -- 172.21.15.16:0/2207829130 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd4e803f810 con 0x7fd4f0133a80 2023-09-30T22:35:26.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.908+0000 7fd4f7925700 1 -- 172.21.15.16:0/2207829130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4f0074ea0 con 0x7fd4f0133a80 2023-09-30T22:35:26.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.913+0000 7fd4e5ffb700 1 -- 172.21.15.16:0/2207829130 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd4e8014480 con 0x7fd4f0133a80 2023-09-30T22:35:26.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.940+0000 7f5250bbf700 1 Processor -- start 2023-09-30T22:35:26.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.940+0000 7f5250bbf700 1 -- start start 2023-09-30T22:35:26.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.941+0000 7f5250bbf700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f524c09d6f0 0x7f524c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:26.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.941+0000 7f5250bbf700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f524c09e000 0x7f524c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:26.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.941+0000 7f5250bbf700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f524c133f30 0x7f524c1343b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:26.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.941+0000 7f5250bbf700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f524c09b0f0 con 0x7f524c09e000 2023-09-30T22:35:26.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.941+0000 7f5250bbf700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f524c09d2f0 con 0x7f524c133f30 2023-09-30T22:35:26.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.941+0000 7f5250bbf700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f524c09a810 con 0x7f524c09d6f0 2023-09-30T22:35:26.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.941+0000 7f5249d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f524c09e000 0x7f524c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:26.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.941+0000 7f524a59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f524c09d6f0 0x7f524c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:26.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.941+0000 7f524a59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f524c09d6f0 0x7f524c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:33384/0 (socket says 172.21.15.16:33384) 2023-09-30T22:35:26.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.941+0000 7f5249d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f524c09e000 0x7f524c1339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55666/0 (socket says 172.21.15.16:55666) 2023-09-30T22:35:26.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.941+0000 7f524a59c700 1 -- 172.21.15.16:0/1812092367 learned_addr learned my addr 172.21.15.16:0/1812092367 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:26.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.941+0000 7f524ad9d700 1 --2- 172.21.15.16:0/1812092367 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f524c133f30 0x7f524c1343b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:26.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.941+0000 7f5249d9b700 1 -- 172.21.15.16:0/1812092367 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f524c09d6f0 msgr2=0x7f524c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:26.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.941+0000 7f5249d9b700 1 --2- 172.21.15.16:0/1812092367 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f524c09d6f0 0x7f524c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:26.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.941+0000 7f5249d9b700 1 -- 172.21.15.16:0/1812092367 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f524c133f30 msgr2=0x7f524c1343b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:26.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.941+0000 7f5249d9b700 1 --2- 172.21.15.16:0/1812092367 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f524c133f30 0x7f524c1343b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:26.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.941+0000 7f5249d9b700 1 -- 172.21.15.16:0/1812092367 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f524c09a470 con 0x7f524c09e000 2023-09-30T22:35:26.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.941+0000 7f524a59c700 1 --2- 172.21.15.16:0/1812092367 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f524c09d6f0 0x7f524c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:26.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.941+0000 7f5249d9b700 1 --2- 172.21.15.16:0/1812092367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f524c09e000 0x7f524c1339f0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f5234009840 tx=0x7f5234009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1c363bb3e86382c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:26.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.942+0000 7f524959a700 1 -- 172.21.15.16:0/1812092367 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5234011830 con 0x7f524c09e000 2023-09-30T22:35:26.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.942+0000 7f524959a700 1 -- 172.21.15.16:0/1812092367 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5234011990 con 0x7f524c09e000 2023-09-30T22:35:26.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.942+0000 7f524959a700 1 -- 172.21.15.16:0/1812092367 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5234011d20 con 0x7f524c09e000 2023-09-30T22:35:26.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.942+0000 7f5250bbf700 1 -- 172.21.15.16:0/1812092367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f524c09e000 msgr2=0x7f524c1339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:26.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.942+0000 7f5250bbf700 1 --2- 172.21.15.16:0/1812092367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f524c09e000 0x7f524c1339f0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f5234009840 tx=0x7f5234009b50 comp rx=0 tx=0).stop 2023-09-30T22:35:26.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.942+0000 7f5250bbf700 1 -- 172.21.15.16:0/1812092367 shutdown_connections 2023-09-30T22:35:26.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.942+0000 7f5250bbf700 1 --2- 172.21.15.16:0/1812092367 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f524c133f30 0x7f524c1343b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:26.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.942+0000 7f5250bbf700 1 --2- 172.21.15.16:0/1812092367 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f524c09d6f0 0x7f524c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:26.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.942+0000 7f5250bbf700 1 --2- 172.21.15.16:0/1812092367 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f524c09e000 0x7f524c1339f0 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:26.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.942+0000 7f5250bbf700 1 -- 172.21.15.16:0/1812092367 >> 172.21.15.16:0/1812092367 conn(0x7f524c066f50 msgr2=0x7f524c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:26.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.942+0000 7f5250bbf700 1 -- 172.21.15.16:0/1812092367 shutdown_connections 2023-09-30T22:35:26.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.942+0000 7f5250bbf700 1 -- 172.21.15.16:0/1812092367 wait complete. 2023-09-30T22:35:26.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.944+0000 7f5250bbf700 1 Processor -- start 2023-09-30T22:35:26.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.944+0000 7f5250bbf700 1 -- start start 2023-09-30T22:35:26.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.944+0000 7f5250bbf700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f524c09d6f0 0x7f524c121670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:26.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.944+0000 7f5250bbf700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f524c09e000 0x7f524c121bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:26.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.944+0000 7f5250bbf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f524c133f30 0x7f524c1220f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:26.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.944+0000 7f5250bbf700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f524c124bf0 con 0x7f524c133f30 2023-09-30T22:35:26.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.944+0000 7f5250bbf700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f524c132200 con 0x7f524c09e000 2023-09-30T22:35:26.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.944+0000 7f5250bbf700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f524c125190 con 0x7f524c09d6f0 2023-09-30T22:35:26.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.944+0000 7f5249d9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f524c09e000 0x7f524c121bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:26.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.944+0000 7f524a59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f524c09d6f0 0x7f524c121670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:26.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.944+0000 7f5249d9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f524c09e000 0x7f524c121bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:53178/0 (socket says 172.21.15.16:53178) 2023-09-30T22:35:26.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.944+0000 7f524a59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f524c09d6f0 0x7f524c121670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:33400/0 (socket says 172.21.15.16:33400) 2023-09-30T22:35:26.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.944+0000 7f5249d9b700 1 -- 172.21.15.16:0/3958944852 learned_addr learned my addr 172.21.15.16:0/3958944852 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:26.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.944+0000 7f524ad9d700 1 --2- 172.21.15.16:0/3958944852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f524c133f30 0x7f524c1220f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:26.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.945+0000 7f5249d9b700 1 -- 172.21.15.16:0/3958944852 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f524c09d6f0 msgr2=0x7f524c121670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:26.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.945+0000 7f5249d9b700 1 --2- 172.21.15.16:0/3958944852 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f524c09d6f0 0x7f524c121670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:26.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.945+0000 7f5249d9b700 1 -- 172.21.15.16:0/3958944852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f524c133f30 msgr2=0x7f524c1220f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:26.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.945+0000 7f5249d9b700 1 --2- 172.21.15.16:0/3958944852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f524c133f30 0x7f524c1220f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:26.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.945+0000 7f5249d9b700 1 -- 172.21.15.16:0/3958944852 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f523c009b30 con 0x7f524c09e000 2023-09-30T22:35:26.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.945+0000 7f524a59c700 1 --2- 172.21.15.16:0/3958944852 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f524c09d6f0 0x7f524c121670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:26.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.945+0000 7f5249d9b700 1 --2- 172.21.15.16:0/3958944852 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f524c09e000 0x7f524c121bb0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f52340090b0 tx=0x7f5234004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:26.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.945+0000 7f5242ffd700 1 -- 172.21.15.16:0/3958944852 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5234004450 con 0x7f524c09e000 2023-09-30T22:35:26.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.945+0000 7f5242ffd700 1 -- 172.21.15.16:0/3958944852 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f523401f070 con 0x7f524c09e000 2023-09-30T22:35:26.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.945+0000 7f5250bbf700 1 -- 172.21.15.16:0/3958944852 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52340094a0 con 0x7f524c09e000 2023-09-30T22:35:26.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.945+0000 7f5242ffd700 1 -- 172.21.15.16:0/3958944852 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52340038e0 con 0x7f524c09e000 2023-09-30T22:35:26.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.945+0000 7f5250bbf700 1 -- 172.21.15.16:0/3958944852 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f524c09c640 con 0x7f524c09e000 2023-09-30T22:35:26.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.946+0000 7f5242ffd700 1 -- 172.21.15.16:0/3958944852 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5234003a40 con 0x7f524c09e000 2023-09-30T22:35:26.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.946+0000 7f5242ffd700 1 -- 172.21.15.16:0/3958944852 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f5234021070 con 0x7f524c09e000 2023-09-30T22:35:26.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.946+0000 7f5250bbf700 1 -- 172.21.15.16:0/3958944852 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f524c074ea0 con 0x7f524c09e000 2023-09-30T22:35:26.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:26.951+0000 7f5242ffd700 1 -- 172.21.15.16:0/3958944852 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f523402b080 con 0x7f524c09e000 2023-09-30T22:35:27.041 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:35:27.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.119+0000 7fd4f7925700 1 -- 172.21.15.16:0/2207829130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd4f006b250 con 0x7fd4f0133a80 2023-09-30T22:35:27.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.119+0000 7fd4e5ffb700 1 -- 172.21.15.16:0/2207829130 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd4e801cbb0 con 0x7fd4f0133a80 2023-09-30T22:35:27.120 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:35:27.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.122+0000 7fd4f7925700 1 -- 172.21.15.16:0/2207829130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f0133a80 msgr2=0x7fd4f009cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:27.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.122+0000 7fd4f7925700 1 --2- 172.21.15.16:0/2207829130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f0133a80 0x7fd4f009cf00 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fd4e800efc0 tx=0x7fd4e800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:35:27.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.122+0000 7fd4f7925700 1 -- 172.21.15.16:0/2207829130 shutdown_connections 2023-09-30T22:35:27.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.122+0000 7fd4f7925700 1 --2- 172.21.15.16:0/2207829130 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4f012ff20 0x7fd4f009c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:27.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.122+0000 7fd4f7925700 1 --2- 172.21.15.16:0/2207829130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f0133a80 0x7fd4f009cf00 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:27.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.122+0000 7fd4f7925700 1 --2- 172.21.15.16:0/2207829130 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd4f0130a70 0x7fd4f009c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:27.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.122+0000 7fd4f7925700 1 -- 172.21.15.16:0/2207829130 >> 172.21.15.16:0/2207829130 conn(0x7fd4f00a4210 msgr2=0x7fd4f009e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:27.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.122+0000 7fd4f7925700 1 -- 172.21.15.16:0/2207829130 shutdown_connections 2023-09-30T22:35:27.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.122+0000 7fd4f7925700 1 -- 172.21.15.16:0/2207829130 wait complete. 2023-09-30T22:35:27.136 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:35:27.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.160+0000 7f5250bbf700 1 -- 172.21.15.16:0/3958944852 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f524c124f70 con 0x7f524c09e000 2023-09-30T22:35:27.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.161+0000 7f5242ffd700 1 -- 172.21.15.16:0/3958944852 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5234015020 con 0x7f524c09e000 2023-09-30T22:35:27.161 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:35:27.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.164+0000 7f5250bbf700 1 -- 172.21.15.16:0/3958944852 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f524c09e000 msgr2=0x7f524c121bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:27.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.164+0000 7f5250bbf700 1 --2- 172.21.15.16:0/3958944852 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f524c09e000 0x7f524c121bb0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f52340090b0 tx=0x7f5234004750 comp rx=0 tx=0).stop 2023-09-30T22:35:27.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.164+0000 7f5250bbf700 1 -- 172.21.15.16:0/3958944852 shutdown_connections 2023-09-30T22:35:27.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.164+0000 7f5250bbf700 1 --2- 172.21.15.16:0/3958944852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f524c133f30 0x7f524c1220f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:27.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.164+0000 7f5250bbf700 1 --2- 172.21.15.16:0/3958944852 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f524c09d6f0 0x7f524c121670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:27.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.164+0000 7f5250bbf700 1 --2- 172.21.15.16:0/3958944852 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f524c09e000 0x7f524c121bb0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:27.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.164+0000 7f5250bbf700 1 -- 172.21.15.16:0/3958944852 >> 172.21.15.16:0/3958944852 conn(0x7f524c066f50 msgr2=0x7f524c0a3680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:27.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.164+0000 7f5250bbf700 1 -- 172.21.15.16:0/3958944852 shutdown_connections 2023-09-30T22:35:27.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.164+0000 7f5250bbf700 1 -- 172.21.15.16:0/3958944852 wait complete. 2023-09-30T22:35:27.179 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:35:27.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.180+0000 7f82b53d5700 1 Processor -- start 2023-09-30T22:35:27.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.180+0000 7f82b53d5700 1 -- start start 2023-09-30T22:35:27.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.181+0000 7f82b53d5700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f82b009d6f0 0x7f82b009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:27.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.181+0000 7f82b53d5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f82b009e440 0x7f82b0134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:27.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.181+0000 7f82b53d5700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82b009b120 con 0x7f82b009e000 2023-09-30T22:35:27.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.181+0000 7f82b53d5700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82b006b460 con 0x7f82b009d6f0 2023-09-30T22:35:27.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.181+0000 7f82b53d5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82b009a7e0 con 0x7f82b009e440 2023-09-30T22:35:27.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.181+0000 7f82aeffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f82b009d6f0 0x7f82b009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:27.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.181+0000 7f82ae7fc700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f82b009e000 0x7f82b01339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33666/0 (socket says 172.21.15.16:33666) 2023-09-30T22:35:27.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.181+0000 7f82ae7fc700 1 -- 172.21.15.16:0/464484995 learned_addr learned my addr 172.21.15.16:0/464484995 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:27.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.181+0000 7f82af7fe700 1 --2- 172.21.15.16:0/464484995 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f82b009e440 0x7f82b0134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:27.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.181+0000 7f82aeffd700 1 -- 172.21.15.16:0/464484995 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f82b009e440 msgr2=0x7f82b0134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:27.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.181+0000 7f82aeffd700 1 --2- 172.21.15.16:0/464484995 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f82b009e440 0x7f82b0134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:27.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.181+0000 7f82aeffd700 1 -- 172.21.15.16:0/464484995 >> v1:172.21.15.16:6789/0 conn(0x7f82b009e000 legacy=0x7f82b01339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:27.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.181+0000 7f82aeffd700 1 -- 172.21.15.16:0/464484995 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b009a500 con 0x7f82b009d6f0 2023-09-30T22:35:27.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.181+0000 7f82af7fe700 1 --2- 172.21.15.16:0/464484995 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f82b009e440 0x7f82b0134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:27.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.181+0000 7f82aeffd700 1 --2- 172.21.15.16:0/464484995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82b009d6f0 0x7f82b009dac0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f82a0009870 tx=0x7f82a0009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=858be1942ee0454f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:27.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.181+0000 7f82adffb700 1 -- 172.21.15.16:0/464484995 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a0011830 con 0x7f82b009d6f0 2023-09-30T22:35:27.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.182+0000 7f82adffb700 1 -- 172.21.15.16:0/464484995 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82a0011990 con 0x7f82b009d6f0 2023-09-30T22:35:27.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.182+0000 7f82adffb700 1 -- 172.21.15.16:0/464484995 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a0011d20 con 0x7f82b009d6f0 2023-09-30T22:35:27.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.182+0000 7f82b53d5700 1 -- 172.21.15.16:0/464484995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82b009d6f0 msgr2=0x7f82b009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:27.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.182+0000 7f82b53d5700 1 --2- 172.21.15.16:0/464484995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82b009d6f0 0x7f82b009dac0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f82a0009870 tx=0x7f82a0009b80 comp rx=0 tx=0).stop 2023-09-30T22:35:27.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.182+0000 7f82b53d5700 1 -- 172.21.15.16:0/464484995 shutdown_connections 2023-09-30T22:35:27.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.182+0000 7f82b53d5700 1 --2- 172.21.15.16:0/464484995 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f82b009e440 0x7f82b0134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:27.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.182+0000 7f82b53d5700 1 --2- 172.21.15.16:0/464484995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82b009d6f0 0x7f82b009dac0 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:27.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.182+0000 7f82b53d5700 1 -- 172.21.15.16:0/464484995 >> 172.21.15.16:0/464484995 conn(0x7f82b0066f50 msgr2=0x7f82b009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:27.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.182+0000 7f82b53d5700 1 -- 172.21.15.16:0/464484995 shutdown_connections 2023-09-30T22:35:27.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.182+0000 7f82b53d5700 1 -- 172.21.15.16:0/464484995 wait complete. 2023-09-30T22:35:27.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.183+0000 7f82b53d5700 1 Processor -- start 2023-09-30T22:35:27.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.183+0000 7f82b53d5700 1 -- start start 2023-09-30T22:35:27.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.184+0000 7f82b53d5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f82b009d6f0 0x7f82b01a5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:27.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.184+0000 7f82b53d5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82b009e000 0x7f82b01a63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:27.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.184+0000 7f82b53d5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f82b009e440 0x7f82b01a6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:27.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.184+0000 7f82b53d5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b01a7a60 con 0x7f82b009e000 2023-09-30T22:35:27.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.184+0000 7f82b53d5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82b009c710 con 0x7f82b009e440 2023-09-30T22:35:27.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.184+0000 7f82b53d5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f82b01a9690 con 0x7f82b009d6f0 2023-09-30T22:35:27.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.184+0000 7f82ae7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82b009e000 0x7f82b01a63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:27.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.184+0000 7f82aeffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f82b009d6f0 0x7f82b01a5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:27.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.184+0000 7f82aeffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f82b009d6f0 0x7f82b01a5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:33404/0 (socket says 172.21.15.16:33404) 2023-09-30T22:35:27.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.184+0000 7f82ae7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82b009e000 0x7f82b01a63d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55690/0 (socket says 172.21.15.16:55690) 2023-09-30T22:35:27.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.184+0000 7f82aeffd700 1 -- 172.21.15.16:0/1124651762 learned_addr learned my addr 172.21.15.16:0/1124651762 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:27.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.184+0000 7f82af7fe700 1 --2- 172.21.15.16:0/1124651762 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f82b009e440 0x7f82b01a6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:27.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.184+0000 7f82af7fe700 1 -- 172.21.15.16:0/1124651762 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f82b009d6f0 msgr2=0x7f82b01a5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:27.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.184+0000 7f82af7fe700 1 --2- 172.21.15.16:0/1124651762 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f82b009d6f0 0x7f82b01a5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:27.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.184+0000 7f82af7fe700 1 -- 172.21.15.16:0/1124651762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82b009e000 msgr2=0x7f82b01a63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:27.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.184+0000 7f82af7fe700 1 --2- 172.21.15.16:0/1124651762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82b009e000 0x7f82b01a63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:27.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.184+0000 7f82af7fe700 1 -- 172.21.15.16:0/1124651762 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a8009a70 con 0x7f82b009e440 2023-09-30T22:35:27.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.184+0000 7f82ae7fc700 1 --2- 172.21.15.16:0/1124651762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82b009e000 0x7f82b01a63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:27.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.185+0000 7f82af7fe700 1 --2- 172.21.15.16:0/1124651762 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f82b009e440 0x7f82b01a6910 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f82a800efc0 tx=0x7f82a800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:27.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.185+0000 7f82a77fe700 1 -- 172.21.15.16:0/1124651762 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a800ca90 con 0x7f82b009e440 2023-09-30T22:35:27.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.185+0000 7f82a77fe700 1 -- 172.21.15.16:0/1124651762 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82a800f040 con 0x7f82b009e440 2023-09-30T22:35:27.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.185+0000 7f82b53d5700 1 -- 172.21.15.16:0/1124651762 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82a00094a0 con 0x7f82b009e440 2023-09-30T22:35:27.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.185+0000 7f82a77fe700 1 -- 172.21.15.16:0/1124651762 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a800ce30 con 0x7f82b009e440 2023-09-30T22:35:27.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.185+0000 7f82b53d5700 1 -- 172.21.15.16:0/1124651762 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82b01a8cb0 con 0x7f82b009e440 2023-09-30T22:35:27.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.186+0000 7f82a77fe700 1 -- 172.21.15.16:0/1124651762 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f82a8014480 con 0x7f82b009e440 2023-09-30T22:35:27.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.186+0000 7f82b53d5700 1 -- 172.21.15.16:0/1124651762 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82b0074ea0 con 0x7f82b009e440 2023-09-30T22:35:27.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.186+0000 7f82a77fe700 1 -- 172.21.15.16:0/1124651762 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f82a803fc20 con 0x7f82b009e440 2023-09-30T22:35:27.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.191+0000 7f82a77fe700 1 -- 172.21.15.16:0/1124651762 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f82a8012930 con 0x7f82b009e440 2023-09-30T22:35:27.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.398+0000 7f82b53d5700 1 -- 172.21.15.16:0/1124651762 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f82b009f420 con 0x7f82b009e440 2023-09-30T22:35:27.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.398+0000 7f82a77fe700 1 -- 172.21.15.16:0/1124651762 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f82a8012340 con 0x7f82b009e440 2023-09-30T22:35:27.399 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:35:27.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.401+0000 7f82b53d5700 1 -- 172.21.15.16:0/1124651762 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f82b009e440 msgr2=0x7f82b01a6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:27.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.401+0000 7f82b53d5700 1 --2- 172.21.15.16:0/1124651762 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f82b009e440 0x7f82b01a6910 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f82a800efc0 tx=0x7f82a800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:35:27.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.401+0000 7f82b53d5700 1 -- 172.21.15.16:0/1124651762 shutdown_connections 2023-09-30T22:35:27.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.401+0000 7f82b53d5700 1 --2- 172.21.15.16:0/1124651762 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f82b009e440 0x7f82b01a6910 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:27.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.401+0000 7f82b53d5700 1 --2- 172.21.15.16:0/1124651762 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f82b009d6f0 0x7f82b01a5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:27.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.401+0000 7f82b53d5700 1 --2- 172.21.15.16:0/1124651762 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82b009e000 0x7f82b01a63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:27.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.401+0000 7f82b53d5700 1 -- 172.21.15.16:0/1124651762 >> 172.21.15.16:0/1124651762 conn(0x7f82b0066f50 msgr2=0x7f82b01bae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:27.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.401+0000 7f82b53d5700 1 -- 172.21.15.16:0/1124651762 shutdown_connections 2023-09-30T22:35:27.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.401+0000 7f82b53d5700 1 -- 172.21.15.16:0/1124651762 wait complete. 2023-09-30T22:35:27.415 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:35:27.683 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:35:27.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.831+0000 7fd3cf6ff700 1 Processor -- start 2023-09-30T22:35:27.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.832+0000 7fd3cf6ff700 1 -- start start 2023-09-30T22:35:27.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.832+0000 7fd3cf6ff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd3c81337f0 0x7fd3c8138880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:27.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.832+0000 7fd3cf6ff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd3c8138dc0 0x7fd3c813b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:27.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.832+0000 7fd3cf6ff700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3c806b990 con 0x7fd3c8133420 2023-09-30T22:35:27.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.832+0000 7fd3cf6ff700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3c809b0e0 con 0x7fd3c81337f0 2023-09-30T22:35:27.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.832+0000 7fd3cf6ff700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c8133c20 con 0x7fd3c8138dc0 2023-09-30T22:35:27.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.832+0000 7fd3ccc9a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd3c81337f0 0x7fd3c8138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:27.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.832+0000 7fd3cdc9c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd3c8138dc0 0x7fd3c813b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:27.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.832+0000 7fd3cd49b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd3c8133420 0x7fd3c8133e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33758/0 (socket says 172.21.15.16:33758) 2023-09-30T22:35:27.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.832+0000 7fd3cd49b700 1 -- 172.21.15.16:0/1234607870 learned_addr learned my addr 172.21.15.16:0/1234607870 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:27.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.833+0000 7fd3ccc9a700 1 -- 172.21.15.16:0/1234607870 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd3c8138dc0 msgr2=0x7fd3c813b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:27.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.833+0000 7fd3ccc9a700 1 --2- 172.21.15.16:0/1234607870 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd3c8138dc0 0x7fd3c813b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:27.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.833+0000 7fd3ccc9a700 1 -- 172.21.15.16:0/1234607870 >> v1:172.21.15.16:6789/0 conn(0x7fd3c8133420 legacy=0x7fd3c8133e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:27.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.833+0000 7fd3bbfff700 1 -- 172.21.15.16:0/1234607870 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3517869214 0 0) 0x7fd3c806b990 con 0x7fd3c8133420 2023-09-30T22:35:27.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.833+0000 7fd3ccc9a700 1 -- 172.21.15.16:0/1234607870 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3c80a0590 con 0x7fd3c81337f0 2023-09-30T22:35:27.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.833+0000 7fd3cdc9c700 1 --2- 172.21.15.16:0/1234607870 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd3c8138dc0 0x7fd3c813b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:27.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.833+0000 7fd3ccc9a700 1 --2- 172.21.15.16:0/1234607870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3c81337f0 0x7fd3c8138880 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc009840 tx=0x7fd3bc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c651830b89a21fbd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:27.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.833+0000 7fd3bbfff700 1 -- 172.21.15.16:0/1234607870 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc011830 con 0x7fd3c81337f0 2023-09-30T22:35:27.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.833+0000 7fd3bbfff700 1 -- 172.21.15.16:0/1234607870 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3bc011990 con 0x7fd3c81337f0 2023-09-30T22:35:27.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.833+0000 7fd3bbfff700 1 -- 172.21.15.16:0/1234607870 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc011d20 con 0x7fd3c81337f0 2023-09-30T22:35:27.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.833+0000 7fd3cf6ff700 1 -- 172.21.15.16:0/1234607870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3c81337f0 msgr2=0x7fd3c8138880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:27.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.833+0000 7fd3cf6ff700 1 --2- 172.21.15.16:0/1234607870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3c81337f0 0x7fd3c8138880 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc009840 tx=0x7fd3bc009b50 comp rx=0 tx=0).stop 2023-09-30T22:35:27.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.834+0000 7fd3cf6ff700 1 -- 172.21.15.16:0/1234607870 shutdown_connections 2023-09-30T22:35:27.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.834+0000 7fd3cf6ff700 1 --2- 172.21.15.16:0/1234607870 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd3c8138dc0 0x7fd3c813b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:27.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.834+0000 7fd3cf6ff700 1 --2- 172.21.15.16:0/1234607870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3c81337f0 0x7fd3c8138880 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:27.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.834+0000 7fd3cf6ff700 1 -- 172.21.15.16:0/1234607870 >> 172.21.15.16:0/1234607870 conn(0x7fd3c80a4210 msgr2=0x7fd3c80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:27.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.834+0000 7fd3cf6ff700 1 -- 172.21.15.16:0/1234607870 shutdown_connections 2023-09-30T22:35:27.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.834+0000 7fd3cf6ff700 1 -- 172.21.15.16:0/1234607870 wait complete. 2023-09-30T22:35:27.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.835+0000 7fd3cf6ff700 1 Processor -- start 2023-09-30T22:35:27.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.835+0000 7fd3cf6ff700 1 -- start start 2023-09-30T22:35:27.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.835+0000 7fd3cf6ff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd3c8133420 0x7fd3c8137810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:27.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.835+0000 7fd3cf6ff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd3c81337f0 0x7fd3c8137d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:27.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.835+0000 7fd3cf6ff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3c8138dc0 0x7fd3c809c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:27.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.835+0000 7fd3cf6ff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c8134650 con 0x7fd3c8138dc0 2023-09-30T22:35:27.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.835+0000 7fd3cf6ff700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c809e020 con 0x7fd3c81337f0 2023-09-30T22:35:27.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.835+0000 7fd3cf6ff700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c8130ab0 con 0x7fd3c8133420 2023-09-30T22:35:27.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.835+0000 7fd3cd49b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd3c8133420 0x7fd3c8137810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:27.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.835+0000 7fd3cdc9c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3c8138dc0 0x7fd3c809c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:27.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.835+0000 7fd3cdc9c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3c8138dc0 0x7fd3c809c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54760/0 (socket says 172.21.15.16:54760) 2023-09-30T22:35:27.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.835+0000 7fd3cdc9c700 1 -- 172.21.15.16:0/1374965872 learned_addr learned my addr 172.21.15.16:0/1374965872 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:27.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.835+0000 7fd3ccc9a700 1 --2- 172.21.15.16:0/1374965872 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd3c81337f0 0x7fd3c8137d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:27.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.836+0000 7fd3ccc9a700 1 -- 172.21.15.16:0/1374965872 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd3c8133420 msgr2=0x7fd3c8137810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:27.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.836+0000 7fd3ccc9a700 1 --2- 172.21.15.16:0/1374965872 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd3c8133420 0x7fd3c8137810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:27.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.836+0000 7fd3ccc9a700 1 -- 172.21.15.16:0/1374965872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3c8138dc0 msgr2=0x7fd3c809c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:27.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.836+0000 7fd3ccc9a700 1 --2- 172.21.15.16:0/1374965872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3c8138dc0 0x7fd3c809c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:27.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.836+0000 7fd3ccc9a700 1 -- 172.21.15.16:0/1374965872 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3c0009a70 con 0x7fd3c81337f0 2023-09-30T22:35:27.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.836+0000 7fd3cd49b700 1 --2- 172.21.15.16:0/1374965872 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd3c8133420 0x7fd3c8137810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:27.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.836+0000 7fd3ccc9a700 1 --2- 172.21.15.16:0/1374965872 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd3c81337f0 0x7fd3c8137d50 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc016000 tx=0x7fd3bc004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:27.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.836+0000 7fd3b9ffb700 1 -- 172.21.15.16:0/1374965872 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc004480 con 0x7fd3c81337f0 2023-09-30T22:35:27.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.836+0000 7fd3b9ffb700 1 -- 172.21.15.16:0/1374965872 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3bc01f070 con 0x7fd3c81337f0 2023-09-30T22:35:27.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.836+0000 7fd3cf6ff700 1 -- 172.21.15.16:0/1374965872 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3bc0094a0 con 0x7fd3c81337f0 2023-09-30T22:35:27.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.836+0000 7fd3b9ffb700 1 -- 172.21.15.16:0/1374965872 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc0038a0 con 0x7fd3c81337f0 2023-09-30T22:35:27.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.836+0000 7fd3cf6ff700 1 -- 172.21.15.16:0/1374965872 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3c812f280 con 0x7fd3c81337f0 2023-09-30T22:35:27.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.837+0000 7fd3b9ffb700 1 -- 172.21.15.16:0/1374965872 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd3bc021030 con 0x7fd3c81337f0 2023-09-30T22:35:27.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.837+0000 7fd3b9ffb700 1 -- 172.21.15.16:0/1374965872 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd3bc040980 con 0x7fd3c81337f0 2023-09-30T22:35:27.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.837+0000 7fd3cf6ff700 1 -- 172.21.15.16:0/1374965872 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3c8130020 con 0x7fd3c81337f0 2023-09-30T22:35:27.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:27.842+0000 7fd3b9ffb700 1 -- 172.21.15.16:0/1374965872 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd3bc015080 con 0x7fd3c81337f0 2023-09-30T22:35:28.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.047+0000 7fd3cf6ff700 1 -- 172.21.15.16:0/1374965872 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd3c80a47a0 con 0x7fd3c81337f0 2023-09-30T22:35:28.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.048+0000 7fd3b9ffb700 1 -- 172.21.15.16:0/1374965872 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd3bc003a00 con 0x7fd3c81337f0 2023-09-30T22:35:28.048 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:35:28.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.050+0000 7fd3cf6ff700 1 -- 172.21.15.16:0/1374965872 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd3c81337f0 msgr2=0x7fd3c8137d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:28.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.050+0000 7fd3cf6ff700 1 --2- 172.21.15.16:0/1374965872 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd3c81337f0 0x7fd3c8137d50 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc016000 tx=0x7fd3bc004750 comp rx=0 tx=0).stop 2023-09-30T22:35:28.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.050+0000 7fd3cf6ff700 1 -- 172.21.15.16:0/1374965872 shutdown_connections 2023-09-30T22:35:28.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.050+0000 7fd3cf6ff700 1 --2- 172.21.15.16:0/1374965872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd3c8138dc0 0x7fd3c809c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:28.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.050+0000 7fd3cf6ff700 1 --2- 172.21.15.16:0/1374965872 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd3c8133420 0x7fd3c8137810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:28.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.050+0000 7fd3cf6ff700 1 --2- 172.21.15.16:0/1374965872 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd3c81337f0 0x7fd3c8137d50 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:28.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.050+0000 7fd3cf6ff700 1 -- 172.21.15.16:0/1374965872 >> 172.21.15.16:0/1374965872 conn(0x7fd3c80a4210 msgr2=0x7fd3c80a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:28.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.050+0000 7fd3cf6ff700 1 -- 172.21.15.16:0/1374965872 shutdown_connections 2023-09-30T22:35:28.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.050+0000 7fd3cf6ff700 1 -- 172.21.15.16:0/1374965872 wait complete. 2023-09-30T22:35:28.064 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:35:28.137 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:35:28.180 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:35:28.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.278+0000 7f432c92f700 1 Processor -- start 2023-09-30T22:35:28.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.278+0000 7f432c92f700 1 -- start start 2023-09-30T22:35:28.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.279+0000 7f432c92f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f43280a0a30 0x7f43280a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:28.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.279+0000 7f432c92f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f432809d630 0x7f43280a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:28.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.279+0000 7f432c92f700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43280a2a70 con 0x7f4328133930 2023-09-30T22:35:28.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.279+0000 7f432c92f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43280a01d0 con 0x7f432809d630 2023-09-30T22:35:28.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.279+0000 7f432c92f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f432809e2b0 con 0x7f43280a0a30 2023-09-30T22:35:28.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.279+0000 7f432659c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f43280a0a30 0x7f43280a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:28.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.279+0000 7f4325d9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f432809d630 0x7f43280a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:28.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.279+0000 7f432659c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f43280a0a30 0x7f43280a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:45076/0 (socket says 172.21.15.16:45076) 2023-09-30T22:35:28.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.279+0000 7f4325d9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f432809d630 0x7f43280a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:45870/0 (socket says 172.21.15.16:45870) 2023-09-30T22:35:28.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.279+0000 7f4326d9d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4328133930 0x7f4328133dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33762/0 (socket says 172.21.15.16:33762) 2023-09-30T22:35:28.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.279+0000 7f432659c700 1 -- 172.21.15.16:0/1068243864 learned_addr learned my addr 172.21.15.16:0/1068243864 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:28.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.279+0000 7f4325d9b700 1 -- 172.21.15.16:0/1068243864 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f43280a0a30 msgr2=0x7f43280a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:28.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.279+0000 7f4325d9b700 1 --2- 172.21.15.16:0/1068243864 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f43280a0a30 0x7f43280a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:28.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.279+0000 7f4325d9b700 1 -- 172.21.15.16:0/1068243864 >> v1:172.21.15.16:6789/0 conn(0x7f4328133930 legacy=0x7f4328133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:28.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.279+0000 7f432559a700 1 -- 172.21.15.16:0/1068243864 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1537089843 0 0) 0x7f43280a2a70 con 0x7f4328133930 2023-09-30T22:35:28.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.279+0000 7f4325d9b700 1 -- 172.21.15.16:0/1068243864 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f432809dc50 con 0x7f432809d630 2023-09-30T22:35:28.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.279+0000 7f432659c700 1 --2- 172.21.15.16:0/1068243864 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f43280a0a30 0x7f43280a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:28.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.280+0000 7f4325d9b700 1 --2- 172.21.15.16:0/1068243864 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f432809d630 0x7f43280a1340 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f4310009870 tx=0x7f4310009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=701229ec30b59d6b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:28.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.280+0000 7f432559a700 1 -- 172.21.15.16:0/1068243864 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4310011830 con 0x7f432809d630 2023-09-30T22:35:28.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.280+0000 7f432559a700 1 -- 172.21.15.16:0/1068243864 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4310011990 con 0x7f432809d630 2023-09-30T22:35:28.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.280+0000 7f432559a700 1 -- 172.21.15.16:0/1068243864 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4310011d20 con 0x7f432809d630 2023-09-30T22:35:28.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.280+0000 7f432c92f700 1 -- 172.21.15.16:0/1068243864 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f432809d630 msgr2=0x7f43280a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:28.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.280+0000 7f432c92f700 1 --2- 172.21.15.16:0/1068243864 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f432809d630 0x7f43280a1340 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f4310009870 tx=0x7f4310009b80 comp rx=0 tx=0).stop 2023-09-30T22:35:28.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.281+0000 7f432c92f700 1 -- 172.21.15.16:0/1068243864 shutdown_connections 2023-09-30T22:35:28.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.281+0000 7f432c92f700 1 --2- 172.21.15.16:0/1068243864 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f43280a0a30 0x7f43280a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:28.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.281+0000 7f432c92f700 1 --2- 172.21.15.16:0/1068243864 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f432809d630 0x7f43280a1340 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:28.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.281+0000 7f432c92f700 1 -- 172.21.15.16:0/1068243864 >> 172.21.15.16:0/1068243864 conn(0x7f43280a3050 msgr2=0x7f43280a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:28.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.281+0000 7f432c92f700 1 -- 172.21.15.16:0/1068243864 shutdown_connections 2023-09-30T22:35:28.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.281+0000 7f432c92f700 1 -- 172.21.15.16:0/1068243864 wait complete. 2023-09-30T22:35:28.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.282+0000 7f432c92f700 1 Processor -- start 2023-09-30T22:35:28.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.282+0000 7f432c92f700 1 -- start start 2023-09-30T22:35:28.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.283+0000 7f432c92f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f432809d630 0x7f43281c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:28.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.283+0000 7f432c92f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f43280a0a30 0x7f43281c84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:28.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.283+0000 7f432c92f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4328133930 0x7f43281c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:28.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.283+0000 7f432c92f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4328120650 con 0x7f432809d630 2023-09-30T22:35:28.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.283+0000 7f432c92f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43280a0410 con 0x7f4328133930 2023-09-30T22:35:28.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.283+0000 7f432c92f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4328121e40 con 0x7f43280a0a30 2023-09-30T22:35:28.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.283+0000 7f4326d9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4328133930 0x7f43281c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:28.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.283+0000 7f4326d9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4328133930 0x7f43281c8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:45872/0 (socket says 172.21.15.16:45872) 2023-09-30T22:35:28.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.283+0000 7f4326d9d700 1 -- 172.21.15.16:0/3236161928 learned_addr learned my addr 172.21.15.16:0/3236161928 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:28.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.283+0000 7f432659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f432809d630 0x7f43281c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:28.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.283+0000 7f4325d9b700 1 --2- 172.21.15.16:0/3236161928 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f43280a0a30 0x7f43281c84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:28.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.284+0000 7f4326d9d700 1 -- 172.21.15.16:0/3236161928 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f43280a0a30 msgr2=0x7f43281c84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:28.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.284+0000 7f4326d9d700 1 --2- 172.21.15.16:0/3236161928 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f43280a0a30 0x7f43281c84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:28.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.284+0000 7f4326d9d700 1 -- 172.21.15.16:0/3236161928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f432809d630 msgr2=0x7f43281c7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:28.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.284+0000 7f4326d9d700 1 --2- 172.21.15.16:0/3236161928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f432809d630 0x7f43281c7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:28.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.284+0000 7f4326d9d700 1 -- 172.21.15.16:0/3236161928 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4320009b30 con 0x7f4328133930 2023-09-30T22:35:28.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.284+0000 7f4325d9b700 1 --2- 172.21.15.16:0/3236161928 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f43280a0a30 0x7f43281c84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:28.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.284+0000 7f4326d9d700 1 --2- 172.21.15.16:0/3236161928 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4328133930 0x7f43281c8a10 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f43180011e0 tx=0x7f431800c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:28.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.284+0000 7f431effd700 1 -- 172.21.15.16:0/3236161928 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4318004c80 con 0x7f4328133930 2023-09-30T22:35:28.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.284+0000 7f431effd700 1 -- 172.21.15.16:0/3236161928 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4318004de0 con 0x7f4328133930 2023-09-30T22:35:28.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.284+0000 7f432c92f700 1 -- 172.21.15.16:0/3236161928 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43100094a0 con 0x7f4328133930 2023-09-30T22:35:28.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.284+0000 7f431effd700 1 -- 172.21.15.16:0/3236161928 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43180072d0 con 0x7f4328133930 2023-09-30T22:35:28.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.284+0000 7f432c92f700 1 -- 172.21.15.16:0/3236161928 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4328121140 con 0x7f4328133930 2023-09-30T22:35:28.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.285+0000 7f431effd700 1 -- 172.21.15.16:0/3236161928 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4318007430 con 0x7f4328133930 2023-09-30T22:35:28.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.285+0000 7f431effd700 1 -- 172.21.15.16:0/3236161928 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f431803f3e0 con 0x7f4328133930 2023-09-30T22:35:28.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.285+0000 7f432c92f700 1 -- 172.21.15.16:0/3236161928 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43280747f0 con 0x7f4328133930 2023-09-30T22:35:28.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.290+0000 7f431effd700 1 -- 172.21.15.16:0/3236161928 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f43180076e0 con 0x7f4328133930 2023-09-30T22:35:28.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.321+0000 7f5481a55700 1 Processor -- start 2023-09-30T22:35:28.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.321+0000 7f5481a55700 1 -- start start 2023-09-30T22:35:28.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.322+0000 7f5481a55700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f547c0a09c0 0x7f547c0a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:28.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.322+0000 7f5481a55700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f547c09d5c0 0x7f547c0a12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:28.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.322+0000 7f5481a55700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f547c133920 0x7f547c133dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:28.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.322+0000 7f5481a55700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f547c0a1810 con 0x7f547c09d5c0 2023-09-30T22:35:28.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.322+0000 7f5481a55700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f547c09ea10 con 0x7f547c133920 2023-09-30T22:35:28.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.322+0000 7f5481a55700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f547c09e210 con 0x7f547c0a09c0 2023-09-30T22:35:28.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.322+0000 7f547affd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f547c0a09c0 0x7f547c0a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:28.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.322+0000 7f547affd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f547c0a09c0 0x7f547c0a0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:45106/0 (socket says 172.21.15.16:45106) 2023-09-30T22:35:28.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.322+0000 7f547affd700 1 -- 172.21.15.16:0/3495258951 learned_addr learned my addr 172.21.15.16:0/3495258951 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:28.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.322+0000 7f547a7fc700 1 --2- 172.21.15.16:0/3495258951 >> v2:172.21.15.16:3300/0 conn(0x7f547c09d5c0 0x7f547c0a12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:28.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.322+0000 7f547b7fe700 1 --2- 172.21.15.16:0/3495258951 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f547c133920 0x7f547c133dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:28.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.322+0000 7f547affd700 1 -- 172.21.15.16:0/3495258951 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f547c133920 msgr2=0x7f547c133dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:28.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.322+0000 7f547affd700 1 --2- 172.21.15.16:0/3495258951 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f547c133920 0x7f547c133dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:28.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.322+0000 7f547affd700 1 -- 172.21.15.16:0/3495258951 >> v2:172.21.15.16:3300/0 conn(0x7f547c09d5c0 msgr2=0x7f547c0a12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:28.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.322+0000 7f547affd700 1 --2- 172.21.15.16:0/3495258951 >> v2:172.21.15.16:3300/0 conn(0x7f547c09d5c0 0x7f547c0a12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:28.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.322+0000 7f547affd700 1 -- 172.21.15.16:0/3495258951 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f547c0a26f0 con 0x7f547c0a09c0 2023-09-30T22:35:28.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.322+0000 7f547a7fc700 1 --2- 172.21.15.16:0/3495258951 >> v2:172.21.15.16:3300/0 conn(0x7f547c09d5c0 0x7f547c0a12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:28.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.323+0000 7f547affd700 1 --2- 172.21.15.16:0/3495258951 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f547c0a09c0 0x7f547c0a0d90 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f546c009e50 tx=0x7f546c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f59c1aa2e0c4ea30 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:28.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.323+0000 7f5479ffb700 1 -- 172.21.15.16:0/3495258951 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546c00e070 con 0x7f547c0a09c0 2023-09-30T22:35:28.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.323+0000 7f5479ffb700 1 -- 172.21.15.16:0/3495258951 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f546c00b910 con 0x7f547c0a09c0 2023-09-30T22:35:28.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.323+0000 7f5479ffb700 1 -- 172.21.15.16:0/3495258951 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546c00bcc0 con 0x7f547c0a09c0 2023-09-30T22:35:28.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.323+0000 7f5481a55700 1 -- 172.21.15.16:0/3495258951 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f547c0a09c0 msgr2=0x7f547c0a0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:28.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.323+0000 7f5481a55700 1 --2- 172.21.15.16:0/3495258951 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f547c0a09c0 0x7f547c0a0d90 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f546c009e50 tx=0x7f546c00b330 comp rx=0 tx=0).stop 2023-09-30T22:35:28.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.324+0000 7f5481a55700 1 -- 172.21.15.16:0/3495258951 shutdown_connections 2023-09-30T22:35:28.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.324+0000 7f5481a55700 1 --2- 172.21.15.16:0/3495258951 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f547c133920 0x7f547c133dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:28.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.324+0000 7f5481a55700 1 --2- 172.21.15.16:0/3495258951 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f547c0a09c0 0x7f547c0a0d90 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:28.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.324+0000 7f5481a55700 1 --2- 172.21.15.16:0/3495258951 >> v2:172.21.15.16:3300/0 conn(0x7f547c09d5c0 0x7f547c0a12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:28.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.324+0000 7f5481a55700 1 -- 172.21.15.16:0/3495258951 >> 172.21.15.16:0/3495258951 conn(0x7f547c0a2fe0 msgr2=0x7f547c0a33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:28.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.324+0000 7f5481a55700 1 -- 172.21.15.16:0/3495258951 shutdown_connections 2023-09-30T22:35:28.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.324+0000 7f5481a55700 1 -- 172.21.15.16:0/3495258951 wait complete. 2023-09-30T22:35:28.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.325+0000 7f5481a55700 1 Processor -- start 2023-09-30T22:35:28.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.325+0000 7f5481a55700 1 -- start start 2023-09-30T22:35:28.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.325+0000 7f5481a55700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f547c09d5c0 0x7f547c11f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:28.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.325+0000 7f5481a55700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f547c0a09c0 0x7f547c11f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:28.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.325+0000 7f5481a55700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f547c133920 0x7f547c1c80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:28.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.325+0000 7f5481a55700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f547c11ee50 con 0x7f547c133920 2023-09-30T22:35:28.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.325+0000 7f5481a55700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f547c133320 con 0x7f547c09d5c0 2023-09-30T22:35:28.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.325+0000 7f5481a55700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f547c121f10 con 0x7f547c0a09c0 2023-09-30T22:35:28.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.325+0000 7f547a7fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f547c0a09c0 0x7f547c11f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:28.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.325+0000 7f547a7fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f547c0a09c0 0x7f547c11f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:45116/0 (socket says 172.21.15.16:45116) 2023-09-30T22:35:28.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.325+0000 7f547a7fc700 1 -- 172.21.15.16:0/530330418 learned_addr learned my addr 172.21.15.16:0/530330418 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:28.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.325+0000 7f547b7fe700 1 --2- 172.21.15.16:0/530330418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f547c133920 0x7f547c1c80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:28.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.325+0000 7f547affd700 1 --2- 172.21.15.16:0/530330418 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f547c09d5c0 0x7f547c11f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:28.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.326+0000 7f547a7fc700 1 -- 172.21.15.16:0/530330418 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f547c09d5c0 msgr2=0x7f547c11f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:28.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.326+0000 7f547a7fc700 1 --2- 172.21.15.16:0/530330418 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f547c09d5c0 0x7f547c11f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:28.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.326+0000 7f547a7fc700 1 -- 172.21.15.16:0/530330418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f547c133920 msgr2=0x7f547c1c80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:28.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.326+0000 7f547a7fc700 1 --2- 172.21.15.16:0/530330418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f547c133920 0x7f547c1c80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:28.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.326+0000 7f547a7fc700 1 -- 172.21.15.16:0/530330418 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54640094a0 con 0x7f547c0a09c0 2023-09-30T22:35:28.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.326+0000 7f547affd700 1 --2- 172.21.15.16:0/530330418 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f547c09d5c0 0x7f547c11f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:28.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.326+0000 7f547b7fe700 1 --2- 172.21.15.16:0/530330418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f547c133920 0x7f547c1c80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:28.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.326+0000 7f547a7fc700 1 --2- 172.21.15.16:0/530330418 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f547c0a09c0 0x7f547c11f7f0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f546400ebb0 tx=0x7f546400eec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:28.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.326+0000 7f5473fff700 1 -- 172.21.15.16:0/530330418 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546400c850 con 0x7f547c0a09c0 2023-09-30T22:35:28.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.326+0000 7f5473fff700 1 -- 172.21.15.16:0/530330418 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f546400c9b0 con 0x7f547c0a09c0 2023-09-30T22:35:28.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.326+0000 7f5481a55700 1 -- 172.21.15.16:0/530330418 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f546c009b30 con 0x7f547c0a09c0 2023-09-30T22:35:28.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.326+0000 7f5473fff700 1 -- 172.21.15.16:0/530330418 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f546400cd40 con 0x7f547c0a09c0 2023-09-30T22:35:28.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.326+0000 7f5481a55700 1 -- 172.21.15.16:0/530330418 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f547c1205a0 con 0x7f547c0a09c0 2023-09-30T22:35:28.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.327+0000 7f5473fff700 1 -- 172.21.15.16:0/530330418 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5464013470 con 0x7f547c0a09c0 2023-09-30T22:35:28.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.327+0000 7f5481a55700 1 -- 172.21.15.16:0/530330418 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f547c0747f0 con 0x7f547c0a09c0 2023-09-30T22:35:28.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.327+0000 7f5473fff700 1 -- 172.21.15.16:0/530330418 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f546403f750 con 0x7f547c0a09c0 2023-09-30T22:35:28.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.337+0000 7f5473fff700 1 -- 172.21.15.16:0/530330418 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f54640232f0 con 0x7f547c0a09c0 2023-09-30T22:35:28.415 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:35:28.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.501+0000 7f432c92f700 1 -- 172.21.15.16:0/3236161928 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4328072980 con 0x7f4328133930 2023-09-30T22:35:28.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.502+0000 7f431effd700 1 -- 172.21.15.16:0/3236161928 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4318011e80 con 0x7f4328133930 2023-09-30T22:35:28.504 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:35:28.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.504+0000 7f432c92f700 1 -- 172.21.15.16:0/3236161928 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4328133930 msgr2=0x7f43281c8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:28.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.504+0000 7f432c92f700 1 --2- 172.21.15.16:0/3236161928 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4328133930 0x7f43281c8a10 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f43180011e0 tx=0x7f431800c800 comp rx=0 tx=0).stop 2023-09-30T22:35:28.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.504+0000 7f432c92f700 1 -- 172.21.15.16:0/3236161928 shutdown_connections 2023-09-30T22:35:28.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.504+0000 7f432c92f700 1 --2- 172.21.15.16:0/3236161928 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4328133930 0x7f43281c8a10 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:28.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.504+0000 7f432c92f700 1 --2- 172.21.15.16:0/3236161928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f432809d630 0x7f43281c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:28.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.504+0000 7f432c92f700 1 --2- 172.21.15.16:0/3236161928 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f43280a0a30 0x7f43281c84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:28.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.504+0000 7f432c92f700 1 -- 172.21.15.16:0/3236161928 >> 172.21.15.16:0/3236161928 conn(0x7f43280a3050 msgr2=0x7f43281bbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:28.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.504+0000 7f432c92f700 1 -- 172.21.15.16:0/3236161928 shutdown_connections 2023-09-30T22:35:28.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.504+0000 7f432c92f700 1 -- 172.21.15.16:0/3236161928 wait complete. 2023-09-30T22:35:28.519 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:35:28.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.549+0000 7f5481a55700 1 -- 172.21.15.16:0/530330418 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f547c121210 con 0x7f547c0a09c0 2023-09-30T22:35:28.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.550+0000 7f5473fff700 1 -- 172.21.15.16:0/530330418 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5464021070 con 0x7f547c0a09c0 2023-09-30T22:35:28.551 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:35:28.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.552+0000 7f5481a55700 1 -- 172.21.15.16:0/530330418 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f547c0a09c0 msgr2=0x7f547c11f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:28.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.552+0000 7f5481a55700 1 --2- 172.21.15.16:0/530330418 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f547c0a09c0 0x7f547c11f7f0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f546400ebb0 tx=0x7f546400eec0 comp rx=0 tx=0).stop 2023-09-30T22:35:28.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.553+0000 7f5481a55700 1 -- 172.21.15.16:0/530330418 shutdown_connections 2023-09-30T22:35:28.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.553+0000 7f5481a55700 1 --2- 172.21.15.16:0/530330418 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f547c09d5c0 0x7f547c11f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:28.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.553+0000 7f5481a55700 1 --2- 172.21.15.16:0/530330418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f547c133920 0x7f547c1c80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:28.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.553+0000 7f5481a55700 1 --2- 172.21.15.16:0/530330418 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f547c0a09c0 0x7f547c11f7f0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:28.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.553+0000 7f5481a55700 1 -- 172.21.15.16:0/530330418 >> 172.21.15.16:0/530330418 conn(0x7f547c0a2fe0 msgr2=0x7f547c1bbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:28.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.554+0000 7f5481a55700 1 -- 172.21.15.16:0/530330418 shutdown_connections 2023-09-30T22:35:28.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.554+0000 7f5481a55700 1 -- 172.21.15.16:0/530330418 wait complete. 2023-09-30T22:35:28.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.558+0000 7f4550e21700 1 Processor -- start 2023-09-30T22:35:28.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.558+0000 7f4550e21700 1 -- start start 2023-09-30T22:35:28.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.559+0000 7f4550e21700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f454c09d6f0 0x7f454c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:28.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.559+0000 7f4550e21700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f454c09e000 0x7f454c130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:28.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.559+0000 7f4550e21700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f454c09e410 0x7f454c130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:28.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.559+0000 7f4550e21700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f454c09f520 con 0x7f454c09d6f0 2023-09-30T22:35:28.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.559+0000 7f4550e21700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f454c09f130 con 0x7f454c09e410 2023-09-30T22:35:28.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.559+0000 7f4550e21700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f454c09e930 con 0x7f454c09e000 2023-09-30T22:35:28.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.559+0000 7f4543fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f454c09e000 0x7f454c130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:28.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.559+0000 7f454a59c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f454c09d6f0 0x7f454c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:28.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.559+0000 7f4543fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f454c09e000 0x7f454c130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:45118/0 (socket says 172.21.15.16:45118) 2023-09-30T22:35:28.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.559+0000 7f454a59c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f454c09d6f0 0x7f454c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54788/0 (socket says 172.21.15.16:54788) 2023-09-30T22:35:28.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.559+0000 7f4543fff700 1 -- 172.21.15.16:0/2905237971 learned_addr learned my addr 172.21.15.16:0/2905237971 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:28.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.559+0000 7f454ad9d700 1 --2- 172.21.15.16:0/2905237971 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f454c09e410 0x7f454c130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:28.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.560+0000 7f454a59c700 1 -- 172.21.15.16:0/2905237971 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f454c09e000 msgr2=0x7f454c130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:28.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.560+0000 7f454a59c700 1 --2- 172.21.15.16:0/2905237971 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f454c09e000 0x7f454c130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:28.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.560+0000 7f454a59c700 1 -- 172.21.15.16:0/2905237971 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f454c09e410 msgr2=0x7f454c130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:28.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.560+0000 7f454a59c700 1 --2- 172.21.15.16:0/2905237971 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f454c09e410 0x7f454c130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:28.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.560+0000 7f454a59c700 1 -- 172.21.15.16:0/2905237971 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f454c09b2d0 con 0x7f454c09d6f0 2023-09-30T22:35:28.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.560+0000 7f4543fff700 1 --2- 172.21.15.16:0/2905237971 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f454c09e000 0x7f454c130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:28.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.560+0000 7f454ad9d700 1 --2- 172.21.15.16:0/2905237971 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f454c09e410 0x7f454c130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:28.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.560+0000 7f454a59c700 1 --2- 172.21.15.16:0/2905237971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f454c09d6f0 0x7f454c09dac0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f453c00bb10 tx=0x7f453c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8e8f6e8a9b36537 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:28.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.560+0000 7f4549d9b700 1 -- 172.21.15.16:0/2905237971 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453c00dc30 con 0x7f454c09d6f0 2023-09-30T22:35:28.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.560+0000 7f4549d9b700 1 -- 172.21.15.16:0/2905237971 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f453c00dd90 con 0x7f454c09d6f0 2023-09-30T22:35:28.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.560+0000 7f4549d9b700 1 -- 172.21.15.16:0/2905237971 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453c010690 con 0x7f454c09d6f0 2023-09-30T22:35:28.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.561+0000 7f4550e21700 1 -- 172.21.15.16:0/2905237971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f454c09d6f0 msgr2=0x7f454c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:28.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.561+0000 7f4550e21700 1 --2- 172.21.15.16:0/2905237971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f454c09d6f0 0x7f454c09dac0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f453c00bb10 tx=0x7f453c00be20 comp rx=0 tx=0).stop 2023-09-30T22:35:28.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.561+0000 7f4550e21700 1 -- 172.21.15.16:0/2905237971 shutdown_connections 2023-09-30T22:35:28.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.561+0000 7f4550e21700 1 --2- 172.21.15.16:0/2905237971 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f454c09e410 0x7f454c130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:28.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.561+0000 7f4550e21700 1 --2- 172.21.15.16:0/2905237971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f454c09d6f0 0x7f454c09dac0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:28.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.561+0000 7f4550e21700 1 --2- 172.21.15.16:0/2905237971 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f454c09e000 0x7f454c130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:28.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.561+0000 7f4550e21700 1 -- 172.21.15.16:0/2905237971 >> 172.21.15.16:0/2905237971 conn(0x7f454c066f50 msgr2=0x7f454c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:28.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.561+0000 7f4550e21700 1 -- 172.21.15.16:0/2905237971 shutdown_connections 2023-09-30T22:35:28.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.561+0000 7f4550e21700 1 -- 172.21.15.16:0/2905237971 wait complete. 2023-09-30T22:35:28.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.562+0000 7f4550e21700 1 Processor -- start 2023-09-30T22:35:28.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.563+0000 7f4550e21700 1 -- start start 2023-09-30T22:35:28.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.563+0000 7f4550e21700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f454c09d6f0 0x7f454c122540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:28.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.563+0000 7f4550e21700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f454c09e000 0x7f454c122a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:28.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.563+0000 7f4550e21700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f454c09e410 0x7f454c122fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:28.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.563+0000 7f4550e21700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f454c125b00 con 0x7f454c09d6f0 2023-09-30T22:35:28.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.563+0000 7f4550e21700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f454c126bf0 con 0x7f454c09e410 2023-09-30T22:35:28.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.563+0000 7f4550e21700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f454c130050 con 0x7f454c09e000 2023-09-30T22:35:28.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.563+0000 7f454ad9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f454c09e410 0x7f454c122fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:28.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.563+0000 7f4543fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f454c09e000 0x7f454c122a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:28.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.563+0000 7f454a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f454c09d6f0 0x7f454c122540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:28.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.563+0000 7f454ad9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f454c09e410 0x7f454c122fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:45882/0 (socket says 172.21.15.16:45882) 2023-09-30T22:35:28.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.563+0000 7f454ad9d700 1 -- 172.21.15.16:0/2283710848 learned_addr learned my addr 172.21.15.16:0/2283710848 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:28.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.564+0000 7f454ad9d700 1 -- 172.21.15.16:0/2283710848 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f454c09e000 msgr2=0x7f454c122a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:28.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.564+0000 7f454ad9d700 1 --2- 172.21.15.16:0/2283710848 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f454c09e000 0x7f454c122a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:28.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.564+0000 7f454ad9d700 1 -- 172.21.15.16:0/2283710848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f454c09d6f0 msgr2=0x7f454c122540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:28.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.564+0000 7f454ad9d700 1 --2- 172.21.15.16:0/2283710848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f454c09d6f0 0x7f454c122540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:28.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.564+0000 7f454ad9d700 1 -- 172.21.15.16:0/2283710848 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4534009550 con 0x7f454c09e410 2023-09-30T22:35:28.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.564+0000 7f454a59c700 1 --2- 172.21.15.16:0/2283710848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f454c09d6f0 0x7f454c122540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:28.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.564+0000 7f4543fff700 1 --2- 172.21.15.16:0/2283710848 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f454c09e000 0x7f454c122a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:28.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.564+0000 7f454ad9d700 1 --2- 172.21.15.16:0/2283710848 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f454c09e410 0x7f454c122fc0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f4544009f00 tx=0x7f454400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:28.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.564+0000 7f4542ffd700 1 -- 172.21.15.16:0/2283710848 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4544009d70 con 0x7f454c09e410 2023-09-30T22:35:28.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.564+0000 7f4542ffd700 1 -- 172.21.15.16:0/2283710848 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f454400f040 con 0x7f454c09e410 2023-09-30T22:35:28.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.564+0000 7f4550e21700 1 -- 172.21.15.16:0/2283710848 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4544009a70 con 0x7f454c09e410 2023-09-30T22:35:28.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.564+0000 7f4542ffd700 1 -- 172.21.15.16:0/2283710848 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454400cd60 con 0x7f454c09e410 2023-09-30T22:35:28.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.564+0000 7f4550e21700 1 -- 172.21.15.16:0/2283710848 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f453c00b740 con 0x7f454c09e410 2023-09-30T22:35:28.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.565+0000 7f4542ffd700 1 -- 172.21.15.16:0/2283710848 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4544014480 con 0x7f454c09e410 2023-09-30T22:35:28.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.565+0000 7f4542ffd700 1 -- 172.21.15.16:0/2283710848 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f454403fd10 con 0x7f454c09e410 2023-09-30T22:35:28.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.565+0000 7f4550e21700 1 -- 172.21.15.16:0/2283710848 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f454c1246b0 con 0x7f454c09e410 2023-09-30T22:35:28.575 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:35:28.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.570+0000 7f4542ffd700 1 -- 172.21.15.16:0/2283710848 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4544012930 con 0x7f454c09e410 2023-09-30T22:35:28.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.780+0000 7f4550e21700 1 -- 172.21.15.16:0/2283710848 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f454c074ea0 con 0x7f454c09e410 2023-09-30T22:35:28.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.781+0000 7f4542ffd700 1 -- 172.21.15.16:0/2283710848 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f454402a090 con 0x7f454c09e410 2023-09-30T22:35:28.782 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:35:28.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.783+0000 7f4550e21700 1 -- 172.21.15.16:0/2283710848 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f454c09e410 msgr2=0x7f454c122fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:28.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.783+0000 7f4550e21700 1 --2- 172.21.15.16:0/2283710848 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f454c09e410 0x7f454c122fc0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f4544009f00 tx=0x7f454400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:35:28.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.783+0000 7f4550e21700 1 -- 172.21.15.16:0/2283710848 shutdown_connections 2023-09-30T22:35:28.784 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.783+0000 7f4550e21700 1 --2- 172.21.15.16:0/2283710848 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f454c09e410 0x7f454c122fc0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:28.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.783+0000 7f4550e21700 1 --2- 172.21.15.16:0/2283710848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f454c09d6f0 0x7f454c122540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:28.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.783+0000 7f4550e21700 1 --2- 172.21.15.16:0/2283710848 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f454c09e000 0x7f454c122a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:28.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.783+0000 7f4550e21700 1 -- 172.21.15.16:0/2283710848 >> 172.21.15.16:0/2283710848 conn(0x7f454c066f50 msgr2=0x7f454c06a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:28.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.783+0000 7f4550e21700 1 -- 172.21.15.16:0/2283710848 shutdown_connections 2023-09-30T22:35:28.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:28.783+0000 7f4550e21700 1 -- 172.21.15.16:0/2283710848 wait complete. 2023-09-30T22:35:28.797 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:35:29.065 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:35:29.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.211+0000 7f3c2b5a3700 1 Processor -- start 2023-09-30T22:35:29.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.212+0000 7f3c2b5a3700 1 -- start start 2023-09-30T22:35:29.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.212+0000 7f3c2b5a3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3c2406a770 0x7f3c2406ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:29.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.212+0000 7f3c2b5a3700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3c2406b080 0x7f3c24131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:29.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.212+0000 7f3c2b5a3700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c2409ece0 con 0x7f3c24131b40 2023-09-30T22:35:29.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.212+0000 7f3c2b5a3700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c2409e760 con 0x7f3c2406b080 2023-09-30T22:35:29.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.212+0000 7f3c2b5a3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2409e040 con 0x7f3c2406a770 2023-09-30T22:35:29.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.212+0000 7f3c2933f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3c2406a770 0x7f3c2406ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:29.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.212+0000 7f3c28b3e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3c2406b080 0x7f3c24131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:29.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.212+0000 7f3c2933f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3c2406a770 0x7f3c2406ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:45130/0 (socket says 172.21.15.16:45130) 2023-09-30T22:35:29.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.212+0000 7f3c28b3e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3c2406b080 0x7f3c24131600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54814/0 (socket says 172.21.15.16:54814) 2023-09-30T22:35:29.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.212+0000 7f3c2933f700 1 -- 172.21.15.16:0/1555993963 learned_addr learned my addr 172.21.15.16:0/1555993963 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:29.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.212+0000 7f3c29b40700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3c24131b40 0x7f3c24138850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33776/0 (socket says 172.21.15.16:33776) 2023-09-30T22:35:29.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.213+0000 7f3c2933f700 1 -- 172.21.15.16:0/1555993963 >> v2:172.21.15.16:3300/0 conn(0x7f3c2406b080 msgr2=0x7f3c24131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:29.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.213+0000 7f3c2933f700 1 --2- 172.21.15.16:0/1555993963 >> v2:172.21.15.16:3300/0 conn(0x7f3c2406b080 0x7f3c24131600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:29.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.213+0000 7f3c2933f700 1 -- 172.21.15.16:0/1555993963 >> v1:172.21.15.16:6789/0 conn(0x7f3c24131b40 legacy=0x7f3c24138850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:29.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.213+0000 7f3c2933f700 1 -- 172.21.15.16:0/1555993963 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c2409f630 con 0x7f3c2406a770 2023-09-30T22:35:29.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.213+0000 7f3c28b3e700 1 --2- 172.21.15.16:0/1555993963 >> v2:172.21.15.16:3300/0 conn(0x7f3c2406b080 0x7f3c24131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:29.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.213+0000 7f3c2933f700 1 --2- 172.21.15.16:0/1555993963 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3c2406a770 0x7f3c2406ab40 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f3c200098f0 tx=0x7f3c20009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92ace1905d688744 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:29.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.213+0000 7f3c17fff700 1 -- 172.21.15.16:0/1555993963 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c20011830 con 0x7f3c2406a770 2023-09-30T22:35:29.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.213+0000 7f3c17fff700 1 -- 172.21.15.16:0/1555993963 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c20011990 con 0x7f3c2406a770 2023-09-30T22:35:29.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.213+0000 7f3c17fff700 1 -- 172.21.15.16:0/1555993963 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c20011d20 con 0x7f3c2406a770 2023-09-30T22:35:29.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.214+0000 7f3c2b5a3700 1 -- 172.21.15.16:0/1555993963 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3c2406a770 msgr2=0x7f3c2406ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:29.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.214+0000 7f3c2b5a3700 1 --2- 172.21.15.16:0/1555993963 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3c2406a770 0x7f3c2406ab40 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f3c200098f0 tx=0x7f3c20009c00 comp rx=0 tx=0).stop 2023-09-30T22:35:29.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.214+0000 7f3c2b5a3700 1 -- 172.21.15.16:0/1555993963 shutdown_connections 2023-09-30T22:35:29.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.214+0000 7f3c2b5a3700 1 --2- 172.21.15.16:0/1555993963 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3c2406a770 0x7f3c2406ab40 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:29.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.214+0000 7f3c2b5a3700 1 --2- 172.21.15.16:0/1555993963 >> v2:172.21.15.16:3300/0 conn(0x7f3c2406b080 0x7f3c24131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:29.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.214+0000 7f3c2b5a3700 1 -- 172.21.15.16:0/1555993963 >> 172.21.15.16:0/1555993963 conn(0x7f3c24066f50 msgr2=0x7f3c2409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:29.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.214+0000 7f3c2b5a3700 1 -- 172.21.15.16:0/1555993963 shutdown_connections 2023-09-30T22:35:29.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.214+0000 7f3c2b5a3700 1 -- 172.21.15.16:0/1555993963 wait complete. 2023-09-30T22:35:29.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.215+0000 7f3c2b5a3700 1 Processor -- start 2023-09-30T22:35:29.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.215+0000 7f3c2b5a3700 1 -- start start 2023-09-30T22:35:29.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.216+0000 7f3c2b5a3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c2406a770 0x7f3c2411d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:29.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.216+0000 7f3c2b5a3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3c2406b080 0x7f3c2411d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:29.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.216+0000 7f3c2b5a3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3c24131b40 0x7f3c241c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:29.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.216+0000 7f3c2b5a3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2411fc40 con 0x7f3c2406a770 2023-09-30T22:35:29.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.216+0000 7f3c2b5a3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2411dbf0 con 0x7f3c24131b40 2023-09-30T22:35:29.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.216+0000 7f3c2b5a3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2411e2d0 con 0x7f3c2406b080 2023-09-30T22:35:29.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.216+0000 7f3c2933f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c2406a770 0x7f3c2411d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:29.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.216+0000 7f3c28b3e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3c2406b080 0x7f3c2411d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:29.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.216+0000 7f3c2933f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c2406a770 0x7f3c2411d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54830/0 (socket says 172.21.15.16:54830) 2023-09-30T22:35:29.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.216+0000 7f3c2933f700 1 -- 172.21.15.16:0/504510408 learned_addr learned my addr 172.21.15.16:0/504510408 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:29.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.216+0000 7f3c29b40700 1 --2- 172.21.15.16:0/504510408 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3c24131b40 0x7f3c241c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:29.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.217+0000 7f3c28b3e700 1 -- 172.21.15.16:0/504510408 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3c24131b40 msgr2=0x7f3c241c61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:29.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.217+0000 7f3c28b3e700 1 --2- 172.21.15.16:0/504510408 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3c24131b40 0x7f3c241c61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:29.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.217+0000 7f3c28b3e700 1 -- 172.21.15.16:0/504510408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c2406a770 msgr2=0x7f3c2411d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:29.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.217+0000 7f3c28b3e700 1 --2- 172.21.15.16:0/504510408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c2406a770 0x7f3c2411d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:29.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.217+0000 7f3c28b3e700 1 -- 172.21.15.16:0/504510408 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c180094a0 con 0x7f3c2406b080 2023-09-30T22:35:29.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.217+0000 7f3c2933f700 1 --2- 172.21.15.16:0/504510408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c2406a770 0x7f3c2411d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:29.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.217+0000 7f3c28b3e700 1 --2- 172.21.15.16:0/504510408 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3c2406b080 0x7f3c2411d6b0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f3c18010cc0 tx=0x7f3c1800b140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:29.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.217+0000 7f3c15ffb700 1 -- 172.21.15.16:0/504510408 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c18015070 con 0x7f3c2406b080 2023-09-30T22:35:29.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.217+0000 7f3c15ffb700 1 -- 172.21.15.16:0/504510408 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c1800b9a0 con 0x7f3c2406b080 2023-09-30T22:35:29.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.217+0000 7f3c2b5a3700 1 -- 172.21.15.16:0/504510408 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c20009550 con 0x7f3c2406b080 2023-09-30T22:35:29.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.217+0000 7f3c15ffb700 1 -- 172.21.15.16:0/504510408 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c1800bd30 con 0x7f3c2406b080 2023-09-30T22:35:29.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.217+0000 7f3c2b5a3700 1 -- 172.21.15.16:0/504510408 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c2409a0c0 con 0x7f3c2406b080 2023-09-30T22:35:29.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.218+0000 7f3c15ffb700 1 -- 172.21.15.16:0/504510408 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3c18019410 con 0x7f3c2406b080 2023-09-30T22:35:29.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.218+0000 7f3c15ffb700 1 -- 172.21.15.16:0/504510408 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f3c18042f20 con 0x7f3c2406b080 2023-09-30T22:35:29.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.218+0000 7f3c2b5a3700 1 -- 172.21.15.16:0/504510408 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c2411fe60 con 0x7f3c2406b080 2023-09-30T22:35:29.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.223+0000 7f3c15ffb700 1 -- 172.21.15.16:0/504510408 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3c1801e070 con 0x7f3c2406b080 2023-09-30T22:35:29.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.426+0000 7f3c2b5a3700 1 -- 172.21.15.16:0/504510408 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f3c24071e60 con 0x7f3c2406b080 2023-09-30T22:35:29.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.427+0000 7f3c15ffb700 1 -- 172.21.15.16:0/504510408 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3c180223b0 con 0x7f3c2406b080 2023-09-30T22:35:29.427 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:35:29.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.429+0000 7f3c2b5a3700 1 -- 172.21.15.16:0/504510408 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3c2406b080 msgr2=0x7f3c2411d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:29.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.429+0000 7f3c2b5a3700 1 --2- 172.21.15.16:0/504510408 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3c2406b080 0x7f3c2411d6b0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f3c18010cc0 tx=0x7f3c1800b140 comp rx=0 tx=0).stop 2023-09-30T22:35:29.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.429+0000 7f3c2b5a3700 1 -- 172.21.15.16:0/504510408 shutdown_connections 2023-09-30T22:35:29.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.429+0000 7f3c2b5a3700 1 --2- 172.21.15.16:0/504510408 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3c24131b40 0x7f3c241c61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:29.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.429+0000 7f3c2b5a3700 1 --2- 172.21.15.16:0/504510408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c2406a770 0x7f3c2411d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:29.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.429+0000 7f3c2b5a3700 1 --2- 172.21.15.16:0/504510408 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3c2406b080 0x7f3c2411d6b0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:29.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.429+0000 7f3c2b5a3700 1 -- 172.21.15.16:0/504510408 >> 172.21.15.16:0/504510408 conn(0x7f3c24066f50 msgr2=0x7f3c24070160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:29.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.429+0000 7f3c2b5a3700 1 -- 172.21.15.16:0/504510408 shutdown_connections 2023-09-30T22:35:29.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.429+0000 7f3c2b5a3700 1 -- 172.21.15.16:0/504510408 wait complete. 2023-09-30T22:35:29.444 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:35:29.519 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:35:29.576 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:35:29.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.663+0000 7f579526b700 1 Processor -- start 2023-09-30T22:35:29.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.663+0000 7f579526b700 1 -- start start 2023-09-30T22:35:29.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.663+0000 7f579526b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f57900a0a30 0x7f57900a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:29.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.663+0000 7f579526b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f579009d630 0x7f57900a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:29.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.663+0000 7f579526b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5790133930 0x7f5790133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:29.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.663+0000 7f579526b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57900a2a70 con 0x7f5790133930 2023-09-30T22:35:29.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.663+0000 7f579526b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57900a01d0 con 0x7f579009d630 2023-09-30T22:35:29.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.663+0000 7f579526b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f579009e2b0 con 0x7f57900a0a30 2023-09-30T22:35:29.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.664+0000 7f578f7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5790133930 0x7f5790133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:29.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.664+0000 7f578effd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f57900a0a30 0x7f57900a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:29.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.664+0000 7f578e7fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f579009d630 0x7f57900a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:29.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.664+0000 7f578f7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5790133930 0x7f5790133dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54844/0 (socket says 172.21.15.16:54844) 2023-09-30T22:35:29.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.664+0000 7f578f7fe700 1 -- 172.21.15.16:0/3174032796 learned_addr learned my addr 172.21.15.16:0/3174032796 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:29.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.664+0000 7f578effd700 1 -- 172.21.15.16:0/3174032796 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f579009d630 msgr2=0x7f57900a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:29.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.664+0000 7f578effd700 1 --2- 172.21.15.16:0/3174032796 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f579009d630 0x7f57900a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:29.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.664+0000 7f578effd700 1 -- 172.21.15.16:0/3174032796 >> v2:172.21.15.16:3300/0 conn(0x7f5790133930 msgr2=0x7f5790133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:29.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.664+0000 7f578effd700 1 --2- 172.21.15.16:0/3174032796 >> v2:172.21.15.16:3300/0 conn(0x7f5790133930 0x7f5790133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:29.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.664+0000 7f578effd700 1 -- 172.21.15.16:0/3174032796 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f579009dc50 con 0x7f57900a0a30 2023-09-30T22:35:29.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.664+0000 7f578f7fe700 1 --2- 172.21.15.16:0/3174032796 >> v2:172.21.15.16:3300/0 conn(0x7f5790133930 0x7f5790133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:29.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.664+0000 7f578effd700 1 --2- 172.21.15.16:0/3174032796 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f57900a0a30 0x7f57900a0e00 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f5780009840 tx=0x7f5780009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78b0c357d710437b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:29.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.664+0000 7f578dffb700 1 -- 172.21.15.16:0/3174032796 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5780011830 con 0x7f57900a0a30 2023-09-30T22:35:29.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.664+0000 7f578dffb700 1 -- 172.21.15.16:0/3174032796 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5780011990 con 0x7f57900a0a30 2023-09-30T22:35:29.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.664+0000 7f578dffb700 1 -- 172.21.15.16:0/3174032796 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5780011d20 con 0x7f57900a0a30 2023-09-30T22:35:29.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.665+0000 7f579526b700 1 -- 172.21.15.16:0/3174032796 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f57900a0a30 msgr2=0x7f57900a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:29.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.665+0000 7f579526b700 1 --2- 172.21.15.16:0/3174032796 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f57900a0a30 0x7f57900a0e00 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f5780009840 tx=0x7f5780009b50 comp rx=0 tx=0).stop 2023-09-30T22:35:29.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.665+0000 7f579526b700 1 -- 172.21.15.16:0/3174032796 shutdown_connections 2023-09-30T22:35:29.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.665+0000 7f579526b700 1 --2- 172.21.15.16:0/3174032796 >> v2:172.21.15.16:3300/0 conn(0x7f5790133930 0x7f5790133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:29.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.665+0000 7f579526b700 1 --2- 172.21.15.16:0/3174032796 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f57900a0a30 0x7f57900a0e00 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:29.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.665+0000 7f579526b700 1 --2- 172.21.15.16:0/3174032796 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f579009d630 0x7f57900a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:29.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.665+0000 7f579526b700 1 -- 172.21.15.16:0/3174032796 >> 172.21.15.16:0/3174032796 conn(0x7f57900a3050 msgr2=0x7f57900a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:29.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.665+0000 7f579526b700 1 -- 172.21.15.16:0/3174032796 shutdown_connections 2023-09-30T22:35:29.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.665+0000 7f579526b700 1 -- 172.21.15.16:0/3174032796 wait complete. 2023-09-30T22:35:29.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.666+0000 7f579526b700 1 Processor -- start 2023-09-30T22:35:29.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.666+0000 7f579526b700 1 -- start start 2023-09-30T22:35:29.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.667+0000 7f579526b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f579009d630 0x7f57901c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:29.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.667+0000 7f579526b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f57900a0a30 0x7f57901c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:29.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.667+0000 7f579526b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5790133930 0x7f57901c8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:29.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.667+0000 7f579526b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5790122430 con 0x7f579009d630 2023-09-30T22:35:29.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.667+0000 7f579526b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5790120720 con 0x7f57900a0a30 2023-09-30T22:35:29.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.667+0000 7f579526b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f57900a41c0 con 0x7f5790133930 2023-09-30T22:35:29.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.667+0000 7f578e7fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f57900a0a30 0x7f57901c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:29.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.667+0000 7f578effd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f579009d630 0x7f57901c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:29.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.667+0000 7f578e7fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f57900a0a30 0x7f57901c86e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:45906/0 (socket says 172.21.15.16:45906) 2023-09-30T22:35:29.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.667+0000 7f578e7fc700 1 -- 172.21.15.16:0/687417394 learned_addr learned my addr 172.21.15.16:0/687417394 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:29.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.667+0000 7f578f7fe700 1 --2- 172.21.15.16:0/687417394 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5790133930 0x7f57901c8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:29.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.667+0000 7f578effd700 1 -- 172.21.15.16:0/687417394 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5790133930 msgr2=0x7f57901c8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:29.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.667+0000 7f578effd700 1 --2- 172.21.15.16:0/687417394 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5790133930 0x7f57901c8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:29.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.667+0000 7f578effd700 1 -- 172.21.15.16:0/687417394 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f57900a0a30 msgr2=0x7f57901c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:29.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.667+0000 7f578effd700 1 --2- 172.21.15.16:0/687417394 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f57900a0a30 0x7f57901c86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:29.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.667+0000 7f578effd700 1 -- 172.21.15.16:0/687417394 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5788009b30 con 0x7f579009d630 2023-09-30T22:35:29.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.667+0000 7f578f7fe700 1 --2- 172.21.15.16:0/687417394 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5790133930 0x7f57901c8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:29.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.667+0000 7f578e7fc700 1 --2- 172.21.15.16:0/687417394 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f57900a0a30 0x7f57901c86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:29.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.668+0000 7f578effd700 1 --2- 172.21.15.16:0/687417394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f579009d630 0x7f57901c81a0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f57900a1f80 tx=0x7f5780004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:29.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.668+0000 7f57777fe700 1 -- 172.21.15.16:0/687417394 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5780004240 con 0x7f579009d630 2023-09-30T22:35:29.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.668+0000 7f57777fe700 1 -- 172.21.15.16:0/687417394 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f578001f070 con 0x7f579009d630 2023-09-30T22:35:29.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.668+0000 7f579526b700 1 -- 172.21.15.16:0/687417394 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57800094a0 con 0x7f579009d630 2023-09-30T22:35:29.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.668+0000 7f57777fe700 1 -- 172.21.15.16:0/687417394 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57800036a0 con 0x7f579009d630 2023-09-30T22:35:29.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.668+0000 7f579526b700 1 -- 172.21.15.16:0/687417394 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57901335c0 con 0x7f579009d630 2023-09-30T22:35:29.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.669+0000 7f579526b700 1 -- 172.21.15.16:0/687417394 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57900747f0 con 0x7f579009d630 2023-09-30T22:35:29.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.669+0000 7f57777fe700 1 -- 172.21.15.16:0/687417394 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5780021030 con 0x7f579009d630 2023-09-30T22:35:29.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.670+0000 7f57777fe700 1 -- 172.21.15.16:0/687417394 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f5780040a00 con 0x7f579009d630 2023-09-30T22:35:29.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.674+0000 7f57777fe700 1 -- 172.21.15.16:0/687417394 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f57800239c0 con 0x7f579009d630 2023-09-30T22:35:29.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.717+0000 7f5d4f923700 1 Processor -- start 2023-09-30T22:35:29.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.718+0000 7f5d4f923700 1 -- start start 2023-09-30T22:35:29.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.718+0000 7f5d4f923700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d4809e1d0 0x7f5d48138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:29.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.718+0000 7f5d4f923700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5d48139170 0x7f5d4813b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:29.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.718+0000 7f5d4f923700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d4809b120 con 0x7f5d4809d6f0 2023-09-30T22:35:29.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.718+0000 7f5d4f923700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d4806b460 con 0x7f5d48139170 2023-09-30T22:35:29.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.718+0000 7f5d4f923700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5d4809a7e0 con 0x7f5d4809e1d0 2023-09-30T22:35:29.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.718+0000 7f5d4cebe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d4809e1d0 0x7f5d48138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:29.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.718+0000 7f5d4dec0700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5d48139170 0x7f5d4813b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:29.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.719+0000 7f5d4cebe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d4809e1d0 0x7f5d48138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:45176/0 (socket says 172.21.15.16:45176) 2023-09-30T22:35:29.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.719+0000 7f5d4cebe700 1 -- 172.21.15.16:0/1893281654 learned_addr learned my addr 172.21.15.16:0/1893281654 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:29.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.719+0000 7f5d4d6bf700 1 --1- 172.21.15.16:0/1893281654 >> v1:172.21.15.16:6789/0 conn(0x7f5d4809d6f0 0x7f5d4809dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33786/0 (socket says 172.21.15.16:33786) 2023-09-30T22:35:29.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.719+0000 7f5d4dec0700 1 -- 172.21.15.16:0/1893281654 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d4809e1d0 msgr2=0x7f5d48138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:29.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.719+0000 7f5d4dec0700 1 --2- 172.21.15.16:0/1893281654 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d4809e1d0 0x7f5d48138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:29.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.719+0000 7f5d4dec0700 1 -- 172.21.15.16:0/1893281654 >> v1:172.21.15.16:6789/0 conn(0x7f5d4809d6f0 legacy=0x7f5d4809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:29.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.719+0000 7f5d4dec0700 1 -- 172.21.15.16:0/1893281654 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d4809a500 con 0x7f5d48139170 2023-09-30T22:35:29.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.719+0000 7f5d3ffff700 1 -- 172.21.15.16:0/1893281654 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 714775135 0 0) 0x7f5d4809b120 con 0x7f5d4809d6f0 2023-09-30T22:35:29.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.719+0000 7f5d4cebe700 1 --2- 172.21.15.16:0/1893281654 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d4809e1d0 0x7f5d48138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:29.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.719+0000 7f5d4dec0700 1 --2- 172.21.15.16:0/1893281654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d48139170 0x7f5d4813b550 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f5d40009d90 tx=0x7f5d4000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eba4d084745bd7a6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:29.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.719+0000 7f5d3ffff700 1 -- 172.21.15.16:0/1893281654 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d4000e070 con 0x7f5d48139170 2023-09-30T22:35:29.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.719+0000 7f5d3ffff700 1 -- 172.21.15.16:0/1893281654 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d4000bb20 con 0x7f5d48139170 2023-09-30T22:35:29.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.719+0000 7f5d3ffff700 1 -- 172.21.15.16:0/1893281654 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d40012430 con 0x7f5d48139170 2023-09-30T22:35:29.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.720+0000 7f5d4f923700 1 -- 172.21.15.16:0/1893281654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d48139170 msgr2=0x7f5d4813b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:29.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.720+0000 7f5d4f923700 1 --2- 172.21.15.16:0/1893281654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d48139170 0x7f5d4813b550 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f5d40009d90 tx=0x7f5d4000b330 comp rx=0 tx=0).stop 2023-09-30T22:35:29.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.720+0000 7f5d4f923700 1 -- 172.21.15.16:0/1893281654 shutdown_connections 2023-09-30T22:35:29.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.720+0000 7f5d4f923700 1 --2- 172.21.15.16:0/1893281654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d48139170 0x7f5d4813b550 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:29.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.720+0000 7f5d4f923700 1 --2- 172.21.15.16:0/1893281654 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d4809e1d0 0x7f5d48138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:29.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.720+0000 7f5d4f923700 1 -- 172.21.15.16:0/1893281654 >> 172.21.15.16:0/1893281654 conn(0x7f5d48066f50 msgr2=0x7f5d4809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:29.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.720+0000 7f5d4f923700 1 -- 172.21.15.16:0/1893281654 shutdown_connections 2023-09-30T22:35:29.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.720+0000 7f5d4f923700 1 -- 172.21.15.16:0/1893281654 wait complete. 2023-09-30T22:35:29.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.721+0000 7f5d4f923700 1 Processor -- start 2023-09-30T22:35:29.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.721+0000 7f5d4f923700 1 -- start start 2023-09-30T22:35:29.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.721+0000 7f5d4f923700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d4809d6f0 0x7f5d481c8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:29.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.721+0000 7f5d4f923700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d4809e1d0 0x7f5d481c89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:29.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.721+0000 7f5d4f923700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d48139170 0x7f5d481217b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:29.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.721+0000 7f5d4f923700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d48137680 con 0x7f5d4809d6f0 2023-09-30T22:35:29.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.721+0000 7f5d4f923700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d481233c0 con 0x7f5d4809e1d0 2023-09-30T22:35:29.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.721+0000 7f5d4f923700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5d480a20d0 con 0x7f5d48139170 2023-09-30T22:35:29.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.722+0000 7f5d4d6bf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d4809d6f0 0x7f5d481c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:29.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.722+0000 7f5d4dec0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d48139170 0x7f5d481217b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:29.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.722+0000 7f5d4dec0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d48139170 0x7f5d481217b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:45178/0 (socket says 172.21.15.16:45178) 2023-09-30T22:35:29.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.722+0000 7f5d4d6bf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d4809d6f0 0x7f5d481c8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54870/0 (socket says 172.21.15.16:54870) 2023-09-30T22:35:29.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.722+0000 7f5d4dec0700 1 -- 172.21.15.16:0/1905630105 learned_addr learned my addr 172.21.15.16:0/1905630105 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:29.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.722+0000 7f5d4cebe700 1 --2- 172.21.15.16:0/1905630105 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d4809e1d0 0x7f5d481c89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:29.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.722+0000 7f5d4d6bf700 1 -- 172.21.15.16:0/1905630105 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d48139170 msgr2=0x7f5d481217b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:29.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.722+0000 7f5d4d6bf700 1 --2- 172.21.15.16:0/1905630105 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d48139170 0x7f5d481217b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:29.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.722+0000 7f5d4d6bf700 1 -- 172.21.15.16:0/1905630105 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d4809e1d0 msgr2=0x7f5d481c89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:29.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.722+0000 7f5d4d6bf700 1 --2- 172.21.15.16:0/1905630105 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d4809e1d0 0x7f5d481c89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:29.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.722+0000 7f5d4d6bf700 1 -- 172.21.15.16:0/1905630105 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d38009550 con 0x7f5d4809d6f0 2023-09-30T22:35:29.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.722+0000 7f5d4cebe700 1 --2- 172.21.15.16:0/1905630105 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d4809e1d0 0x7f5d481c89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:29.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.722+0000 7f5d4d6bf700 1 --2- 172.21.15.16:0/1905630105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d4809d6f0 0x7f5d481c8460 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f5d4400c470 tx=0x7f5d4400c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:29.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.722+0000 7f5d3dffb700 1 -- 172.21.15.16:0/1905630105 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d44004c80 con 0x7f5d4809d6f0 2023-09-30T22:35:29.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.722+0000 7f5d3dffb700 1 -- 172.21.15.16:0/1905630105 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d44004de0 con 0x7f5d4809d6f0 2023-09-30T22:35:29.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.722+0000 7f5d4f923700 1 -- 172.21.15.16:0/1905630105 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d40009a70 con 0x7f5d4809d6f0 2023-09-30T22:35:29.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.722+0000 7f5d3dffb700 1 -- 172.21.15.16:0/1905630105 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d44007310 con 0x7f5d4809d6f0 2023-09-30T22:35:29.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.723+0000 7f5d4f923700 1 -- 172.21.15.16:0/1905630105 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d48125a60 con 0x7f5d4809d6f0 2023-09-30T22:35:29.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.723+0000 7f5d3dffb700 1 -- 172.21.15.16:0/1905630105 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5d44007470 con 0x7f5d4809d6f0 2023-09-30T22:35:29.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.724+0000 7f5d3dffb700 1 -- 172.21.15.16:0/1905630105 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f5d4403f1f0 con 0x7f5d4809d6f0 2023-09-30T22:35:29.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.724+0000 7f5d4f923700 1 -- 172.21.15.16:0/1905630105 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d48074ea0 con 0x7f5d4809d6f0 2023-09-30T22:35:29.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.729+0000 7f5d3dffb700 1 -- 172.21.15.16:0/1905630105 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5d44007720 con 0x7f5d4809d6f0 2023-09-30T22:35:29.798 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:35:29.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.888+0000 7f579526b700 1 -- 172.21.15.16:0/687417394 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5790121ff0 con 0x7f579009d630 2023-09-30T22:35:29.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.889+0000 7f57777fe700 1 -- 172.21.15.16:0/687417394 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5780015030 con 0x7f579009d630 2023-09-30T22:35:29.890 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:35:29.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.891+0000 7f579526b700 1 -- 172.21.15.16:0/687417394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f579009d630 msgr2=0x7f57901c81a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:29.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.891+0000 7f579526b700 1 --2- 172.21.15.16:0/687417394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f579009d630 0x7f57901c81a0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f57900a1f80 tx=0x7f5780004750 comp rx=0 tx=0).stop 2023-09-30T22:35:29.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.891+0000 7f579526b700 1 -- 172.21.15.16:0/687417394 shutdown_connections 2023-09-30T22:35:29.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.891+0000 7f579526b700 1 --2- 172.21.15.16:0/687417394 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5790133930 0x7f57901c8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:29.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.891+0000 7f579526b700 1 --2- 172.21.15.16:0/687417394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f579009d630 0x7f57901c81a0 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:29.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.891+0000 7f579526b700 1 --2- 172.21.15.16:0/687417394 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f57900a0a30 0x7f57901c86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:29.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.891+0000 7f579526b700 1 -- 172.21.15.16:0/687417394 >> 172.21.15.16:0/687417394 conn(0x7f57900a3050 msgr2=0x7f57901bbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:29.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.891+0000 7f579526b700 1 -- 172.21.15.16:0/687417394 shutdown_connections 2023-09-30T22:35:29.894 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.891+0000 7f579526b700 1 -- 172.21.15.16:0/687417394 wait complete. 2023-09-30T22:35:29.908 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:35:29.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.934+0000 7f5d4f923700 1 -- 172.21.15.16:0/1905630105 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f5d48124c90 con 0x7f5d4809d6f0 2023-09-30T22:35:29.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.935+0000 7f9721360700 1 Processor -- start 2023-09-30T22:35:29.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.935+0000 7f5d3dffb700 1 -- 172.21.15.16:0/1905630105 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5d44011360 con 0x7f5d4809d6f0 2023-09-30T22:35:29.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.935+0000 7f9721360700 1 -- start start 2023-09-30T22:35:29.937 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:35:29.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.936+0000 7f9721360700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f971c09d5c0 0x7f971c138d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:29.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.936+0000 7f9721360700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f971c0a14a0 0x7f971c13b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:29.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.936+0000 7f9721360700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f971c09ea10 con 0x7f971c0a09c0 2023-09-30T22:35:29.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.936+0000 7f9721360700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f971c09e210 con 0x7f971c09d5c0 2023-09-30T22:35:29.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.936+0000 7f9721360700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f971c0a26f0 con 0x7f971c0a14a0 2023-09-30T22:35:29.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.936+0000 7f971b7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f971c0a14a0 0x7f971c13b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:29.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.936+0000 7f971a7fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f971c09d5c0 0x7f971c138d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:29.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.936+0000 7f971b7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f971c0a14a0 0x7f971c13b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:45192/0 (socket says 172.21.15.16:45192) 2023-09-30T22:35:29.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.936+0000 7f971b7fe700 1 -- 172.21.15.16:0/3828019994 learned_addr learned my addr 172.21.15.16:0/3828019994 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:29.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.936+0000 7f971affd700 1 --1- 172.21.15.16:0/3828019994 >> v1:172.21.15.16:6789/0 conn(0x7f971c0a09c0 0x7f971c0a0d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33792/0 (socket says 172.21.15.16:33792) 2023-09-30T22:35:29.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.936+0000 7f971b7fe700 1 -- 172.21.15.16:0/3828019994 >> v2:172.21.15.16:3300/0 conn(0x7f971c09d5c0 msgr2=0x7f971c138d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:29.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.936+0000 7f971b7fe700 1 --2- 172.21.15.16:0/3828019994 >> v2:172.21.15.16:3300/0 conn(0x7f971c09d5c0 0x7f971c138d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:29.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.936+0000 7f971b7fe700 1 -- 172.21.15.16:0/3828019994 >> v1:172.21.15.16:6789/0 conn(0x7f971c0a09c0 legacy=0x7f971c0a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:29.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.936+0000 7f9719ffb700 1 -- 172.21.15.16:0/3828019994 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2614953088 0 0) 0x7f971c09ea10 con 0x7f971c0a09c0 2023-09-30T22:35:29.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.936+0000 7f971b7fe700 1 -- 172.21.15.16:0/3828019994 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f971c09db70 con 0x7f971c0a14a0 2023-09-30T22:35:29.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.936+0000 7f971a7fc700 1 --2- 172.21.15.16:0/3828019994 >> v2:172.21.15.16:3300/0 conn(0x7f971c09d5c0 0x7f971c138d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:29.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.936+0000 7f971b7fe700 1 --2- 172.21.15.16:0/3828019994 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f971c0a14a0 0x7f971c13b2c0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f9714009e50 tx=0x7f971400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e09686e0728b5541 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:29.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.937+0000 7f9719ffb700 1 -- 172.21.15.16:0/3828019994 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f971400e070 con 0x7f971c0a14a0 2023-09-30T22:35:29.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.937+0000 7f9719ffb700 1 -- 172.21.15.16:0/3828019994 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f971400b910 con 0x7f971c0a14a0 2023-09-30T22:35:29.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.937+0000 7f9719ffb700 1 -- 172.21.15.16:0/3828019994 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f971400bcc0 con 0x7f971c0a14a0 2023-09-30T22:35:29.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.937+0000 7f9721360700 1 -- 172.21.15.16:0/3828019994 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f971c0a14a0 msgr2=0x7f971c13b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:29.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.937+0000 7f9721360700 1 --2- 172.21.15.16:0/3828019994 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f971c0a14a0 0x7f971c13b2c0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f9714009e50 tx=0x7f971400b330 comp rx=0 tx=0).stop 2023-09-30T22:35:29.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.937+0000 7f9721360700 1 -- 172.21.15.16:0/3828019994 shutdown_connections 2023-09-30T22:35:29.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.937+0000 7f9721360700 1 --2- 172.21.15.16:0/3828019994 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f971c0a14a0 0x7f971c13b2c0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:29.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.937+0000 7f9721360700 1 --2- 172.21.15.16:0/3828019994 >> v2:172.21.15.16:3300/0 conn(0x7f971c09d5c0 0x7f971c138d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:29.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.937+0000 7f9721360700 1 -- 172.21.15.16:0/3828019994 >> 172.21.15.16:0/3828019994 conn(0x7f971c0a2fe0 msgr2=0x7f971c0a33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:29.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.937+0000 7f9721360700 1 -- 172.21.15.16:0/3828019994 shutdown_connections 2023-09-30T22:35:29.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.938+0000 7f9721360700 1 -- 172.21.15.16:0/3828019994 wait complete. 2023-09-30T22:35:29.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.938+0000 7f5d4f923700 1 -- 172.21.15.16:0/1905630105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d4809d6f0 msgr2=0x7f5d481c8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:29.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.938+0000 7f5d4f923700 1 --2- 172.21.15.16:0/1905630105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d4809d6f0 0x7f5d481c8460 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f5d4400c470 tx=0x7f5d4400c780 comp rx=0 tx=0).stop 2023-09-30T22:35:29.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.938+0000 7f5d4f923700 1 -- 172.21.15.16:0/1905630105 shutdown_connections 2023-09-30T22:35:29.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.938+0000 7f5d4f923700 1 --2- 172.21.15.16:0/1905630105 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d48139170 0x7f5d481217b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:29.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.938+0000 7f5d4f923700 1 --2- 172.21.15.16:0/1905630105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d4809d6f0 0x7f5d481c8460 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:29.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.938+0000 7f5d4f923700 1 --2- 172.21.15.16:0/1905630105 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d4809e1d0 0x7f5d481c89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:29.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.938+0000 7f5d4f923700 1 -- 172.21.15.16:0/1905630105 >> 172.21.15.16:0/1905630105 conn(0x7f5d48066f50 msgr2=0x7f5d48070f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:29.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.938+0000 7f5d4f923700 1 -- 172.21.15.16:0/1905630105 shutdown_connections 2023-09-30T22:35:29.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.938+0000 7f5d4f923700 1 -- 172.21.15.16:0/1905630105 wait complete. 2023-09-30T22:35:29.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.939+0000 7f9721360700 1 Processor -- start 2023-09-30T22:35:29.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.939+0000 7f9721360700 1 -- start start 2023-09-30T22:35:29.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.939+0000 7f9721360700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f971c09d5c0 0x7f971c11c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:29.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.939+0000 7f9721360700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f971c0a09c0 0x7f971c11ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:29.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.939+0000 7f9721360700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f971c0a14a0 0x7f971c11d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:29.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.939+0000 7f9721360700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f971c11faa0 con 0x7f971c0a14a0 2023-09-30T22:35:29.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.939+0000 7f9721360700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f971c11eb90 con 0x7f971c09d5c0 2023-09-30T22:35:29.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.939+0000 7f9721360700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f971c134680 con 0x7f971c0a09c0 2023-09-30T22:35:29.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.939+0000 7f971affd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f971c09d5c0 0x7f971c11c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:29.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.939+0000 7f971b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f971c0a14a0 0x7f971c11d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:29.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.940+0000 7f971affd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f971c09d5c0 0x7f971c11c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:45926/0 (socket says 172.21.15.16:45926) 2023-09-30T22:35:29.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.940+0000 7f971b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f971c0a14a0 0x7f971c11d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54890/0 (socket says 172.21.15.16:54890) 2023-09-30T22:35:29.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.940+0000 7f971affd700 1 -- 172.21.15.16:0/4212544096 learned_addr learned my addr 172.21.15.16:0/4212544096 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:29.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.940+0000 7f971a7fc700 1 --2- 172.21.15.16:0/4212544096 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f971c0a09c0 0x7f971c11ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:29.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.940+0000 7f971affd700 1 -- 172.21.15.16:0/4212544096 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f971c0a09c0 msgr2=0x7f971c11ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:29.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.940+0000 7f971affd700 1 --2- 172.21.15.16:0/4212544096 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f971c0a09c0 0x7f971c11ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:29.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.940+0000 7f971affd700 1 -- 172.21.15.16:0/4212544096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f971c0a14a0 msgr2=0x7f971c11d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:29.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.940+0000 7f971affd700 1 --2- 172.21.15.16:0/4212544096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f971c0a14a0 0x7f971c11d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:29.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.940+0000 7f971affd700 1 -- 172.21.15.16:0/4212544096 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97100094a0 con 0x7f971c09d5c0 2023-09-30T22:35:29.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.940+0000 7f971b7fe700 1 --2- 172.21.15.16:0/4212544096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f971c0a14a0 0x7f971c11d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:29.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.940+0000 7f971affd700 1 --2- 172.21.15.16:0/4212544096 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f971c09d5c0 0x7f971c11c7b0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f970c00c520 tx=0x7f970c00c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:29.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.940+0000 7f97037fe700 1 -- 172.21.15.16:0/4212544096 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f970c004c80 con 0x7f971c09d5c0 2023-09-30T22:35:29.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.940+0000 7f97037fe700 1 -- 172.21.15.16:0/4212544096 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f970c004e00 con 0x7f971c09d5c0 2023-09-30T22:35:29.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.940+0000 7f9721360700 1 -- 172.21.15.16:0/4212544096 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9714009b30 con 0x7f971c09d5c0 2023-09-30T22:35:29.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.940+0000 7f97037fe700 1 -- 172.21.15.16:0/4212544096 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f970c007310 con 0x7f971c09d5c0 2023-09-30T22:35:29.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.940+0000 7f9721360700 1 -- 172.21.15.16:0/4212544096 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f971c11f050 con 0x7f971c09d5c0 2023-09-30T22:35:29.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.941+0000 7f97037fe700 1 -- 172.21.15.16:0/4212544096 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f970c007470 con 0x7f971c09d5c0 2023-09-30T22:35:29.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.941+0000 7f97037fe700 1 -- 172.21.15.16:0/4212544096 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f970c03f320 con 0x7f971c09d5c0 2023-09-30T22:35:29.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.942+0000 7f9721360700 1 -- 172.21.15.16:0/4212544096 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f971c0747f0 con 0x7f971c09d5c0 2023-09-30T22:35:29.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:29.947+0000 7f97037fe700 1 -- 172.21.15.16:0/4212544096 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f970c0123b0 con 0x7f971c09d5c0 2023-09-30T22:35:29.954 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:35:30.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.151+0000 7f9721360700 1 -- 172.21.15.16:0/4212544096 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f971c072980 con 0x7f971c09d5c0 2023-09-30T22:35:30.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.151+0000 7f97037fe700 1 -- 172.21.15.16:0/4212544096 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f970c011360 con 0x7f971c09d5c0 2023-09-30T22:35:30.153 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:35:30.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.154+0000 7f9721360700 1 -- 172.21.15.16:0/4212544096 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f971c09d5c0 msgr2=0x7f971c11c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:30.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.154+0000 7f9721360700 1 --2- 172.21.15.16:0/4212544096 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f971c09d5c0 0x7f971c11c7b0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f970c00c520 tx=0x7f970c00c830 comp rx=0 tx=0).stop 2023-09-30T22:35:30.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.154+0000 7f9721360700 1 -- 172.21.15.16:0/4212544096 shutdown_connections 2023-09-30T22:35:30.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.154+0000 7f9721360700 1 --2- 172.21.15.16:0/4212544096 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f971c09d5c0 0x7f971c11c7b0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:30.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.154+0000 7f9721360700 1 --2- 172.21.15.16:0/4212544096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f971c0a14a0 0x7f971c11d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:30.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.154+0000 7f9721360700 1 --2- 172.21.15.16:0/4212544096 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f971c0a09c0 0x7f971c11ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:30.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.154+0000 7f9721360700 1 -- 172.21.15.16:0/4212544096 >> 172.21.15.16:0/4212544096 conn(0x7f971c0a2fe0 msgr2=0x7f971c136c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:30.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.154+0000 7f9721360700 1 -- 172.21.15.16:0/4212544096 shutdown_connections 2023-09-30T22:35:30.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.154+0000 7f9721360700 1 -- 172.21.15.16:0/4212544096 wait complete. 2023-09-30T22:35:30.169 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:35:30.444 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:35:30.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.589+0000 7fb8efffa700 1 Processor -- start 2023-09-30T22:35:30.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.589+0000 7fb8efffa700 1 -- start start 2023-09-30T22:35:30.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.590+0000 7fb8efffa700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb8e809d6f0 0x7fb8e809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:30.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.590+0000 7fb8efffa700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb8e809e000 0x7fb8e8130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:30.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.590+0000 7fb8efffa700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8e809f130 con 0x7fb8e809e410 2023-09-30T22:35:30.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.590+0000 7fb8efffa700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8e809f520 con 0x7fb8e809d6f0 2023-09-30T22:35:30.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.590+0000 7fb8efffa700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e809e930 con 0x7fb8e809e000 2023-09-30T22:35:30.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.590+0000 7fb8edd96700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb8e809d6f0 0x7fb8e809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:30.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.590+0000 7fb8ed595700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb8e809e000 0x7fb8e8130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:30.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.590+0000 7fb8ee597700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb8e809e410 0x7fb8e8130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33806/0 (socket says 172.21.15.16:33806) 2023-09-30T22:35:30.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.590+0000 7fb8ee597700 1 -- 172.21.15.16:0/1356716341 learned_addr learned my addr 172.21.15.16:0/1356716341 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:30.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.590+0000 7fb8ed595700 1 -- 172.21.15.16:0/1356716341 >> v2:172.21.15.16:3300/0 conn(0x7fb8e809d6f0 msgr2=0x7fb8e809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:30.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.590+0000 7fb8ed595700 1 --2- 172.21.15.16:0/1356716341 >> v2:172.21.15.16:3300/0 conn(0x7fb8e809d6f0 0x7fb8e809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:30.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.590+0000 7fb8ed595700 1 -- 172.21.15.16:0/1356716341 >> v1:172.21.15.16:6789/0 conn(0x7fb8e809e410 legacy=0x7fb8e8130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:30.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.590+0000 7fb8ed595700 1 -- 172.21.15.16:0/1356716341 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e809b2d0 con 0x7fb8e809e000 2023-09-30T22:35:30.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.590+0000 7fb8edd96700 1 --2- 172.21.15.16:0/1356716341 >> v2:172.21.15.16:3300/0 conn(0x7fb8e809d6f0 0x7fb8e809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:30.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.590+0000 7fb8ed595700 1 --2- 172.21.15.16:0/1356716341 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb8e809e000 0x7fb8e8130660 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc009840 tx=0x7fb8dc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2efa5b0eb0f7712d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:30.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.591+0000 7fb8ecd94700 1 -- 172.21.15.16:0/1356716341 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc00f8a0 con 0x7fb8e809e000 2023-09-30T22:35:30.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.591+0000 7fb8ecd94700 1 -- 172.21.15.16:0/1356716341 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8dc00fa00 con 0x7fb8e809e000 2023-09-30T22:35:30.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.591+0000 7fb8ecd94700 1 -- 172.21.15.16:0/1356716341 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc00fd90 con 0x7fb8e809e000 2023-09-30T22:35:30.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.591+0000 7fb8efffa700 1 -- 172.21.15.16:0/1356716341 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb8e809e000 msgr2=0x7fb8e8130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:30.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.591+0000 7fb8efffa700 1 --2- 172.21.15.16:0/1356716341 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb8e809e000 0x7fb8e8130660 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc009840 tx=0x7fb8dc009b50 comp rx=0 tx=0).stop 2023-09-30T22:35:30.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.591+0000 7fb8efffa700 1 -- 172.21.15.16:0/1356716341 shutdown_connections 2023-09-30T22:35:30.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.591+0000 7fb8efffa700 1 --2- 172.21.15.16:0/1356716341 >> v2:172.21.15.16:3300/0 conn(0x7fb8e809d6f0 0x7fb8e809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:30.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.591+0000 7fb8efffa700 1 --2- 172.21.15.16:0/1356716341 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb8e809e000 0x7fb8e8130660 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:30.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.591+0000 7fb8efffa700 1 -- 172.21.15.16:0/1356716341 >> 172.21.15.16:0/1356716341 conn(0x7fb8e8066f50 msgr2=0x7fb8e809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:30.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.591+0000 7fb8efffa700 1 -- 172.21.15.16:0/1356716341 shutdown_connections 2023-09-30T22:35:30.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.591+0000 7fb8efffa700 1 -- 172.21.15.16:0/1356716341 wait complete. 2023-09-30T22:35:30.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.592+0000 7fb8efffa700 1 Processor -- start 2023-09-30T22:35:30.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.592+0000 7fb8efffa700 1 -- start start 2023-09-30T22:35:30.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.593+0000 7fb8efffa700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb8e809d6f0 0x7fb8e811e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:30.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.593+0000 7fb8efffa700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb8e809e000 0x7fb8e811e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:30.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.593+0000 7fb8efffa700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8e809e410 0x7fb8e811eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:30.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.593+0000 7fb8efffa700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e8121930 con 0x7fb8e809e410 2023-09-30T22:35:30.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.593+0000 7fb8efffa700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e809c700 con 0x7fb8e809d6f0 2023-09-30T22:35:30.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.593+0000 7fb8efffa700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb8e8130390 con 0x7fb8e809e000 2023-09-30T22:35:30.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.593+0000 7fb8ee597700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8e809e410 0x7fb8e811eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:30.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.593+0000 7fb8edd96700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb8e809d6f0 0x7fb8e811e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:30.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.593+0000 7fb8ee597700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8e809e410 0x7fb8e811eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54906/0 (socket says 172.21.15.16:54906) 2023-09-30T22:35:30.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.593+0000 7fb8edd96700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb8e809d6f0 0x7fb8e811e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:45950/0 (socket says 172.21.15.16:45950) 2023-09-30T22:35:30.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.593+0000 7fb8ee597700 1 -- 172.21.15.16:0/1458859334 learned_addr learned my addr 172.21.15.16:0/1458859334 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:30.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.593+0000 7fb8ed595700 1 --2- 172.21.15.16:0/1458859334 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb8e809e000 0x7fb8e811e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:30.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.593+0000 7fb8edd96700 1 -- 172.21.15.16:0/1458859334 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb8e809e000 msgr2=0x7fb8e811e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:30.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.593+0000 7fb8edd96700 1 --2- 172.21.15.16:0/1458859334 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb8e809e000 0x7fb8e811e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:30.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.593+0000 7fb8edd96700 1 -- 172.21.15.16:0/1458859334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8e809e410 msgr2=0x7fb8e811eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:30.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.593+0000 7fb8edd96700 1 --2- 172.21.15.16:0/1458859334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8e809e410 0x7fb8e811eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:30.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.593+0000 7fb8edd96700 1 -- 172.21.15.16:0/1458859334 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e40094a0 con 0x7fb8e809d6f0 2023-09-30T22:35:30.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.593+0000 7fb8ee597700 1 --2- 172.21.15.16:0/1458859334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8e809e410 0x7fb8e811eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:30.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.594+0000 7fb8edd96700 1 --2- 172.21.15.16:0/1458859334 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb8e809d6f0 0x7fb8e811e090 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fb8e400e9b0 tx=0x7fb8e400ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:30.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.594+0000 7fb8da7fc700 1 -- 172.21.15.16:0/1458859334 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e400ee50 con 0x7fb8e809d6f0 2023-09-30T22:35:30.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.594+0000 7fb8da7fc700 1 -- 172.21.15.16:0/1458859334 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8e400c850 con 0x7fb8e809d6f0 2023-09-30T22:35:30.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.594+0000 7fb8efffa700 1 -- 172.21.15.16:0/1458859334 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8dc0094a0 con 0x7fb8e809d6f0 2023-09-30T22:35:30.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.594+0000 7fb8da7fc700 1 -- 172.21.15.16:0/1458859334 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e400cc00 con 0x7fb8e809d6f0 2023-09-30T22:35:30.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.594+0000 7fb8efffa700 1 -- 172.21.15.16:0/1458859334 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8e8120960 con 0x7fb8e809d6f0 2023-09-30T22:35:30.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.594+0000 7fb8da7fc700 1 -- 172.21.15.16:0/1458859334 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb8e4013470 con 0x7fb8e809d6f0 2023-09-30T22:35:30.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.595+0000 7fb8da7fc700 1 -- 172.21.15.16:0/1458859334 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fb8e403f6d0 con 0x7fb8e809d6f0 2023-09-30T22:35:30.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.595+0000 7fb8efffa700 1 -- 172.21.15.16:0/1458859334 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8e8074ea0 con 0x7fb8e809d6f0 2023-09-30T22:35:30.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.600+0000 7fb8da7fc700 1 -- 172.21.15.16:0/1458859334 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb8e4021080 con 0x7fb8e809d6f0 2023-09-30T22:35:30.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.803+0000 7fb8efffa700 1 -- 172.21.15.16:0/1458859334 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb8e8121a70 con 0x7fb8e809d6f0 2023-09-30T22:35:30.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.803+0000 7fb8da7fc700 1 -- 172.21.15.16:0/1458859334 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb8e4013720 con 0x7fb8e809d6f0 2023-09-30T22:35:30.805 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:35:30.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.805+0000 7fb8efffa700 1 -- 172.21.15.16:0/1458859334 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb8e809d6f0 msgr2=0x7fb8e811e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:30.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.805+0000 7fb8efffa700 1 --2- 172.21.15.16:0/1458859334 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb8e809d6f0 0x7fb8e811e090 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fb8e400e9b0 tx=0x7fb8e400ecc0 comp rx=0 tx=0).stop 2023-09-30T22:35:30.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.805+0000 7fb8efffa700 1 -- 172.21.15.16:0/1458859334 shutdown_connections 2023-09-30T22:35:30.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.805+0000 7fb8efffa700 1 --2- 172.21.15.16:0/1458859334 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb8e809d6f0 0x7fb8e811e090 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:30.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.805+0000 7fb8efffa700 1 --2- 172.21.15.16:0/1458859334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb8e809e410 0x7fb8e811eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:30.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.805+0000 7fb8efffa700 1 --2- 172.21.15.16:0/1458859334 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb8e809e000 0x7fb8e811e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:30.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.805+0000 7fb8efffa700 1 -- 172.21.15.16:0/1458859334 >> 172.21.15.16:0/1458859334 conn(0x7fb8e8066f50 msgr2=0x7fb8e806f090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:30.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.805+0000 7fb8efffa700 1 -- 172.21.15.16:0/1458859334 shutdown_connections 2023-09-30T22:35:30.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:30.805+0000 7fb8efffa700 1 -- 172.21.15.16:0/1458859334 wait complete. 2023-09-30T22:35:30.821 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:35:30.908 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:35:30.956 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:35:31.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.054+0000 7f53ddaae700 1 Processor -- start 2023-09-30T22:35:31.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.054+0000 7f53ddaae700 1 -- start start 2023-09-30T22:35:31.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.054+0000 7f53ddaae700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f53d80a09c0 0x7f53d806c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:31.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.054+0000 7f53ddaae700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f53d8138cf0 0x7f53d813b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:31.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.054+0000 7f53ddaae700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53d80a2a20 con 0x7f53d80a0d90 2023-09-30T22:35:31.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.054+0000 7f53ddaae700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53d809e690 con 0x7f53d8138cf0 2023-09-30T22:35:31.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.054+0000 7f53ddaae700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f53d80a1cc0 con 0x7f53d80a09c0 2023-09-30T22:35:31.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.054+0000 7f53d77fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f53d80a09c0 0x7f53d806c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:31.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.054+0000 7f53d7fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f53d8138cf0 0x7f53d813b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:31.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.054+0000 7f53d6ffd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f53d80a0d90 0x7f53d80a11c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33822/0 (socket says 172.21.15.16:33822) 2023-09-30T22:35:31.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.054+0000 7f53d6ffd700 1 -- 172.21.15.16:0/836978550 learned_addr learned my addr 172.21.15.16:0/836978550 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:31.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.055+0000 7f53d77fe700 1 -- 172.21.15.16:0/836978550 >> v2:172.21.15.16:3300/0 conn(0x7f53d8138cf0 msgr2=0x7f53d813b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:31.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.055+0000 7f53d77fe700 1 --2- 172.21.15.16:0/836978550 >> v2:172.21.15.16:3300/0 conn(0x7f53d8138cf0 0x7f53d813b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:31.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.055+0000 7f53d77fe700 1 -- 172.21.15.16:0/836978550 >> v1:172.21.15.16:6789/0 conn(0x7f53d80a0d90 legacy=0x7f53d80a11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:31.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.055+0000 7f53d77fe700 1 -- 172.21.15.16:0/836978550 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d80a1f40 con 0x7f53d80a09c0 2023-09-30T22:35:31.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.055+0000 7f53d67fc700 1 -- 172.21.15.16:0/836978550 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2457941233 0 0) 0x7f53cc001110 con 0x7f53d80a0d90 2023-09-30T22:35:31.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.055+0000 7f53d7fff700 1 --2- 172.21.15.16:0/836978550 >> v2:172.21.15.16:3300/0 conn(0x7f53d8138cf0 0x7f53d813b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:31.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.055+0000 7f53d77fe700 1 --2- 172.21.15.16:0/836978550 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f53d80a09c0 0x7f53d806c5d0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f53c8009d90 tx=0x7f53c800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=873baa8c1f8746ba server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:31.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.055+0000 7f53d67fc700 1 -- 172.21.15.16:0/836978550 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c800e070 con 0x7f53d80a09c0 2023-09-30T22:35:31.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.055+0000 7f53d67fc700 1 -- 172.21.15.16:0/836978550 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c800bb20 con 0x7f53d80a09c0 2023-09-30T22:35:31.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.055+0000 7f53d67fc700 1 -- 172.21.15.16:0/836978550 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c8012430 con 0x7f53d80a09c0 2023-09-30T22:35:31.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.056+0000 7f53ddaae700 1 -- 172.21.15.16:0/836978550 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f53d80a09c0 msgr2=0x7f53d806c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:31.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.056+0000 7f53ddaae700 1 --2- 172.21.15.16:0/836978550 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f53d80a09c0 0x7f53d806c5d0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f53c8009d90 tx=0x7f53c800b330 comp rx=0 tx=0).stop 2023-09-30T22:35:31.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.056+0000 7f53ddaae700 1 -- 172.21.15.16:0/836978550 shutdown_connections 2023-09-30T22:35:31.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.056+0000 7f53ddaae700 1 --2- 172.21.15.16:0/836978550 >> v2:172.21.15.16:3300/0 conn(0x7f53d8138cf0 0x7f53d813b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:31.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.056+0000 7f53ddaae700 1 --2- 172.21.15.16:0/836978550 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f53d80a09c0 0x7f53d806c5d0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:31.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.056+0000 7f53ddaae700 1 -- 172.21.15.16:0/836978550 >> 172.21.15.16:0/836978550 conn(0x7f53d80a60c0 msgr2=0x7f53d80a64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:31.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.056+0000 7f53ddaae700 1 -- 172.21.15.16:0/836978550 shutdown_connections 2023-09-30T22:35:31.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.056+0000 7f53ddaae700 1 -- 172.21.15.16:0/836978550 wait complete. 2023-09-30T22:35:31.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.057+0000 7f53ddaae700 1 Processor -- start 2023-09-30T22:35:31.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.057+0000 7f53ddaae700 1 -- start start 2023-09-30T22:35:31.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.058+0000 7f53ddaae700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53d80a09c0 0x7f53d81337b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:31.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.058+0000 7f53ddaae700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f53d80a0d90 0x7f53d8133cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:31.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.058+0000 7f53ddaae700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f53d8138cf0 0x7f53d8134230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:31.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.058+0000 7f53ddaae700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d81361c0 con 0x7f53d80a09c0 2023-09-30T22:35:31.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.058+0000 7f53ddaae700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53d8136680 con 0x7f53d8138cf0 2023-09-30T22:35:31.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.058+0000 7f53ddaae700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f53d8136ff0 con 0x7f53d80a0d90 2023-09-30T22:35:31.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.058+0000 7f53d77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53d80a09c0 0x7f53d81337b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:31.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.058+0000 7f53d77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53d80a09c0 0x7f53d81337b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54934/0 (socket says 172.21.15.16:54934) 2023-09-30T22:35:31.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.058+0000 7f53d77fe700 1 -- 172.21.15.16:0/2930471686 learned_addr learned my addr 172.21.15.16:0/2930471686 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:31.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.058+0000 7f53d6ffd700 1 --2- 172.21.15.16:0/2930471686 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f53d80a0d90 0x7f53d8133cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:31.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.058+0000 7f53d7fff700 1 --2- 172.21.15.16:0/2930471686 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f53d8138cf0 0x7f53d8134230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:31.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.058+0000 7f53d77fe700 1 -- 172.21.15.16:0/2930471686 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f53d80a0d90 msgr2=0x7f53d8133cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:31.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.058+0000 7f53d77fe700 1 --2- 172.21.15.16:0/2930471686 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f53d80a0d90 0x7f53d8133cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:31.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.058+0000 7f53d77fe700 1 -- 172.21.15.16:0/2930471686 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f53d8138cf0 msgr2=0x7f53d8134230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:31.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.058+0000 7f53d77fe700 1 --2- 172.21.15.16:0/2930471686 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f53d8138cf0 0x7f53d8134230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:31.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.058+0000 7f53d77fe700 1 -- 172.21.15.16:0/2930471686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d00094a0 con 0x7f53d80a09c0 2023-09-30T22:35:31.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.059+0000 7f53d77fe700 1 --2- 172.21.15.16:0/2930471686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53d80a09c0 0x7f53d81337b0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f53d80a1e00 tx=0x7f53c8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:31.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.059+0000 7f53bffff700 1 -- 172.21.15.16:0/2930471686 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c8012430 con 0x7f53d80a09c0 2023-09-30T22:35:31.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.059+0000 7f53bffff700 1 -- 172.21.15.16:0/2930471686 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c8020070 con 0x7f53d80a09c0 2023-09-30T22:35:31.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.059+0000 7f53bffff700 1 -- 172.21.15.16:0/2930471686 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c8004280 con 0x7f53d80a09c0 2023-09-30T22:35:31.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.059+0000 7f53ddaae700 1 -- 172.21.15.16:0/2930471686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53c8009a70 con 0x7f53d80a09c0 2023-09-30T22:35:31.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.059+0000 7f53ddaae700 1 -- 172.21.15.16:0/2930471686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53d809cea0 con 0x7f53d80a09c0 2023-09-30T22:35:31.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.060+0000 7f53bffff700 1 -- 172.21.15.16:0/2930471686 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f53c80043e0 con 0x7f53d80a09c0 2023-09-30T22:35:31.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.060+0000 7f53ddaae700 1 -- 172.21.15.16:0/2930471686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53d804f2a0 con 0x7f53d80a09c0 2023-09-30T22:35:31.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.060+0000 7f53bffff700 1 -- 172.21.15.16:0/2930471686 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f53c8022070 con 0x7f53d80a09c0 2023-09-30T22:35:31.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.065+0000 7f53bffff700 1 -- 172.21.15.16:0/2930471686 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f53c802c080 con 0x7f53d80a09c0 2023-09-30T22:35:31.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.098+0000 7f1ef23dd700 1 Processor -- start 2023-09-30T22:35:31.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.098+0000 7f1ef23dd700 1 -- start start 2023-09-30T22:35:31.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.098+0000 7f1ef23dd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1eec0a0a30 0x7f1eec0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:31.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.098+0000 7f1ef23dd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1eec134170 0x7f1eec13bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:31.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.098+0000 7f1ef23dd700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1eec09edd0 con 0x7f1eec0a1340 2023-09-30T22:35:31.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.098+0000 7f1ef23dd700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1eec09e1f0 con 0x7f1eec134170 2023-09-30T22:35:31.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.098+0000 7f1ef23dd700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1eec09dc50 con 0x7f1eec0a0a30 2023-09-30T22:35:31.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.098+0000 7f1eebfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1eec0a0a30 0x7f1eec0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:31.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.098+0000 7f1ef097a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1eec134170 0x7f1eec13bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:31.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.098+0000 7f1eeb7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1eec0a1340 0x7f1eec133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33836/0 (socket says 172.21.15.16:33836) 2023-09-30T22:35:31.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.098+0000 7f1eeb7fe700 1 -- 172.21.15.16:0/1194176412 learned_addr learned my addr 172.21.15.16:0/1194176412 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:31.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.099+0000 7f1ef097a700 1 -- 172.21.15.16:0/1194176412 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1eec0a0a30 msgr2=0x7f1eec0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:31.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.099+0000 7f1ef097a700 1 --2- 172.21.15.16:0/1194176412 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1eec0a0a30 0x7f1eec0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:31.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.099+0000 7f1ef097a700 1 -- 172.21.15.16:0/1194176412 >> v1:172.21.15.16:6789/0 conn(0x7f1eec0a1340 legacy=0x7f1eec133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:31.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.099+0000 7f1ef097a700 1 -- 172.21.15.16:0/1194176412 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eec0a1fe0 con 0x7f1eec134170 2023-09-30T22:35:31.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.099+0000 7f1eeaffd700 1 -- 172.21.15.16:0/1194176412 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 846882472 0 0) 0x7f1eec09edd0 con 0x7f1eec0a1340 2023-09-30T22:35:31.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.099+0000 7f1eebfff700 1 --2- 172.21.15.16:0/1194176412 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1eec0a0a30 0x7f1eec0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:31.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.099+0000 7f1ef097a700 1 --2- 172.21.15.16:0/1194176412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1eec134170 0x7f1eec13bd30 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f1ee4009d90 tx=0x7f1ee400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a1ac715ff9e8d42 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:31.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.099+0000 7f1eeaffd700 1 -- 172.21.15.16:0/1194176412 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee400e070 con 0x7f1eec134170 2023-09-30T22:35:31.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.099+0000 7f1eeaffd700 1 -- 172.21.15.16:0/1194176412 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ee400bb20 con 0x7f1eec134170 2023-09-30T22:35:31.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.099+0000 7f1eeaffd700 1 -- 172.21.15.16:0/1194176412 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee4012430 con 0x7f1eec134170 2023-09-30T22:35:31.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.100+0000 7f1ef23dd700 1 -- 172.21.15.16:0/1194176412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1eec134170 msgr2=0x7f1eec13bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:31.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.100+0000 7f1ef23dd700 1 --2- 172.21.15.16:0/1194176412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1eec134170 0x7f1eec13bd30 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f1ee4009d90 tx=0x7f1ee400b330 comp rx=0 tx=0).stop 2023-09-30T22:35:31.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.100+0000 7f1ef23dd700 1 -- 172.21.15.16:0/1194176412 shutdown_connections 2023-09-30T22:35:31.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.100+0000 7f1ef23dd700 1 --2- 172.21.15.16:0/1194176412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1eec134170 0x7f1eec13bd30 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:31.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.100+0000 7f1ef23dd700 1 --2- 172.21.15.16:0/1194176412 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1eec0a0a30 0x7f1eec0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:31.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.100+0000 7f1ef23dd700 1 -- 172.21.15.16:0/1194176412 >> 172.21.15.16:0/1194176412 conn(0x7f1eec0a6550 msgr2=0x7f1eec0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:31.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.100+0000 7f1ef23dd700 1 -- 172.21.15.16:0/1194176412 shutdown_connections 2023-09-30T22:35:31.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.100+0000 7f1ef23dd700 1 -- 172.21.15.16:0/1194176412 wait complete. 2023-09-30T22:35:31.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.101+0000 7f1ef23dd700 1 Processor -- start 2023-09-30T22:35:31.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.101+0000 7f1ef23dd700 1 -- start start 2023-09-30T22:35:31.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.101+0000 7f1ef23dd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1eec0a0a30 0x7f1eec122780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:31.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.102+0000 7f1ef23dd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1eec0a1340 0x7f1eec122cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:31.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.102+0000 7f1ef23dd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1eec134170 0x7f1eec123200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:31.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.102+0000 7f1ef23dd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eec125f30 con 0x7f1eec0a1340 2023-09-30T22:35:31.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.102+0000 7f1ef23dd700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eec0a4dc0 con 0x7f1eec0a0a30 2023-09-30T22:35:31.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.102+0000 7f1ef23dd700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1eec0a0220 con 0x7f1eec134170 2023-09-30T22:35:31.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.102+0000 7f1eeb7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1eec0a1340 0x7f1eec122cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:31.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.102+0000 7f1ef097a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1eec134170 0x7f1eec123200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:31.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.102+0000 7f1eeb7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1eec0a1340 0x7f1eec122cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54954/0 (socket says 172.21.15.16:54954) 2023-09-30T22:35:31.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.102+0000 7f1eeb7fe700 1 -- 172.21.15.16:0/2224273583 learned_addr learned my addr 172.21.15.16:0/2224273583 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:31.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.102+0000 7f1eebfff700 1 --2- 172.21.15.16:0/2224273583 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1eec0a0a30 0x7f1eec122780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:31.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.102+0000 7f1ef097a700 1 -- 172.21.15.16:0/2224273583 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1eec0a0a30 msgr2=0x7f1eec122780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:31.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.102+0000 7f1ef097a700 1 --2- 172.21.15.16:0/2224273583 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1eec0a0a30 0x7f1eec122780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:31.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.102+0000 7f1ef097a700 1 -- 172.21.15.16:0/2224273583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1eec0a1340 msgr2=0x7f1eec122cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:31.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.102+0000 7f1ef097a700 1 --2- 172.21.15.16:0/2224273583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1eec0a1340 0x7f1eec122cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:31.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.102+0000 7f1ef097a700 1 -- 172.21.15.16:0/2224273583 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1edc0094a0 con 0x7f1eec134170 2023-09-30T22:35:31.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.102+0000 7f1eeb7fe700 1 --2- 172.21.15.16:0/2224273583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1eec0a1340 0x7f1eec122cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:31.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.102+0000 7f1ef097a700 1 --2- 172.21.15.16:0/2224273583 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1eec134170 0x7f1eec123200 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f1ee4016000 tx=0x7f1ee4004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:31.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.102+0000 7f1ee8ff9700 1 -- 172.21.15.16:0/2224273583 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee4012430 con 0x7f1eec134170 2023-09-30T22:35:31.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.102+0000 7f1ee8ff9700 1 -- 172.21.15.16:0/2224273583 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ee4020070 con 0x7f1eec134170 2023-09-30T22:35:31.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.102+0000 7f1ef23dd700 1 -- 172.21.15.16:0/2224273583 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ee4009a70 con 0x7f1eec134170 2023-09-30T22:35:31.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.102+0000 7f1ee8ff9700 1 -- 172.21.15.16:0/2224273583 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee4004280 con 0x7f1eec134170 2023-09-30T22:35:31.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.103+0000 7f1ef23dd700 1 -- 172.21.15.16:0/2224273583 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eec1269a0 con 0x7f1eec134170 2023-09-30T22:35:31.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.103+0000 7f1ee8ff9700 1 -- 172.21.15.16:0/2224273583 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1ee40043e0 con 0x7f1eec134170 2023-09-30T22:35:31.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.103+0000 7f1ee8ff9700 1 -- 172.21.15.16:0/2224273583 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f1ee4022070 con 0x7f1eec134170 2023-09-30T22:35:31.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.104+0000 7f1ef23dd700 1 -- 172.21.15.16:0/2224273583 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1eec125bd0 con 0x7f1eec134170 2023-09-30T22:35:31.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.108+0000 7f1ee8ff9700 1 -- 172.21.15.16:0/2224273583 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1ee4012e60 con 0x7f1eec134170 2023-09-30T22:35:31.170 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:35:31.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.279+0000 7f53ddaae700 1 -- 172.21.15.16:0/2930471686 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f53d809dd50 con 0x7f53d80a09c0 2023-09-30T22:35:31.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.279+0000 7f53bffff700 1 -- 172.21.15.16:0/2930471686 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f53c801b400 con 0x7f53d80a09c0 2023-09-30T22:35:31.281 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:35:31.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.282+0000 7f53ddaae700 1 -- 172.21.15.16:0/2930471686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53d80a09c0 msgr2=0x7f53d81337b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:31.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.282+0000 7f53ddaae700 1 --2- 172.21.15.16:0/2930471686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53d80a09c0 0x7f53d81337b0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f53d80a1e00 tx=0x7f53c8004ad0 comp rx=0 tx=0).stop 2023-09-30T22:35:31.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.282+0000 7f53ddaae700 1 -- 172.21.15.16:0/2930471686 shutdown_connections 2023-09-30T22:35:31.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.282+0000 7f53ddaae700 1 --2- 172.21.15.16:0/2930471686 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f53d8138cf0 0x7f53d8134230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:31.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.282+0000 7f53ddaae700 1 --2- 172.21.15.16:0/2930471686 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f53d80a09c0 0x7f53d81337b0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:31.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.282+0000 7f53ddaae700 1 --2- 172.21.15.16:0/2930471686 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f53d80a0d90 0x7f53d8133cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:31.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.282+0000 7f53ddaae700 1 -- 172.21.15.16:0/2930471686 >> 172.21.15.16:0/2930471686 conn(0x7f53d80a60c0 msgr2=0x7f53d81392f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:31.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.282+0000 7f53ddaae700 1 -- 172.21.15.16:0/2930471686 shutdown_connections 2023-09-30T22:35:31.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.282+0000 7f53ddaae700 1 -- 172.21.15.16:0/2930471686 wait complete. 2023-09-30T22:35:31.298 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:35:31.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.311+0000 7fad6e35f700 1 Processor -- start 2023-09-30T22:35:31.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.311+0000 7fad6e35f700 1 -- start start 2023-09-30T22:35:31.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.312+0000 7fad6e35f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fad6812cf60 0x7fad6806f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:31.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.312+0000 7fad6e35f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fad68130430 0x7fad68137c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:31.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.312+0000 7fad6e35f700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad6812cd90 con 0x7fad6812f8b0 2023-09-30T22:35:31.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.312+0000 7fad6e35f700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad6812d330 con 0x7fad6812cf60 2023-09-30T22:35:31.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.312+0000 7fad6e35f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fad68071780 con 0x7fad68130430 2023-09-30T22:35:31.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.312+0000 7fad67fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fad6812cf60 0x7fad6806f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:31.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.312+0000 7fad67fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fad6812cf60 0x7fad6806f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54956/0 (socket says 172.21.15.16:54956) 2023-09-30T22:35:31.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.312+0000 7fad67fff700 1 -- 172.21.15.16:0/2180304262 learned_addr learned my addr 172.21.15.16:0/2180304262 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:31.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.312+0000 7fad677fe700 1 --1- 172.21.15.16:0/2180304262 >> v1:172.21.15.16:6789/0 conn(0x7fad6812f8b0 0x7fad6812fd20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33848/0 (socket says 172.21.15.16:33848) 2023-09-30T22:35:31.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.312+0000 7fad6c8fc700 1 --2- 172.21.15.16:0/2180304262 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fad68130430 0x7fad68137c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:31.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.312+0000 7fad67fff700 1 -- 172.21.15.16:0/2180304262 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fad68130430 msgr2=0x7fad68137c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:31.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.312+0000 7fad67fff700 1 --2- 172.21.15.16:0/2180304262 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fad68130430 0x7fad68137c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:31.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.312+0000 7fad67fff700 1 -- 172.21.15.16:0/2180304262 >> v1:172.21.15.16:6789/0 conn(0x7fad6812f8b0 legacy=0x7fad6812fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:31.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.312+0000 7fad66ffd700 1 -- 172.21.15.16:0/2180304262 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 596293612 0 0) 0x7fad6812cd90 con 0x7fad6812f8b0 2023-09-30T22:35:31.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.312+0000 7fad67fff700 1 -- 172.21.15.16:0/2180304262 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad68071a40 con 0x7fad6812cf60 2023-09-30T22:35:31.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.312+0000 7fad67fff700 1 --2- 172.21.15.16:0/2180304262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad6812cf60 0x7fad6806f5d0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fad5800aa90 tx=0x7fad5800ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71d6141c742e22c0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:31.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.313+0000 7fad66ffd700 1 -- 172.21.15.16:0/2180304262 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad5800cb70 con 0x7fad6812cf60 2023-09-30T22:35:31.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.313+0000 7fad66ffd700 1 -- 172.21.15.16:0/2180304262 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad5800ccd0 con 0x7fad6812cf60 2023-09-30T22:35:31.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.313+0000 7fad66ffd700 1 -- 172.21.15.16:0/2180304262 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad58018620 con 0x7fad6812cf60 2023-09-30T22:35:31.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.313+0000 7fad6e35f700 1 -- 172.21.15.16:0/2180304262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad6812cf60 msgr2=0x7fad6806f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:31.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.313+0000 7fad6e35f700 1 --2- 172.21.15.16:0/2180304262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad6812cf60 0x7fad6806f5d0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fad5800aa90 tx=0x7fad5800ada0 comp rx=0 tx=0).stop 2023-09-30T22:35:31.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.313+0000 7fad6e35f700 1 -- 172.21.15.16:0/2180304262 shutdown_connections 2023-09-30T22:35:31.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.313+0000 7fad6e35f700 1 --2- 172.21.15.16:0/2180304262 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fad68130430 0x7fad68137c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:31.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.313+0000 7fad6e35f700 1 --2- 172.21.15.16:0/2180304262 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad6812cf60 0x7fad6806f5d0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:31.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.313+0000 7fad6e35f700 1 -- 172.21.15.16:0/2180304262 >> 172.21.15.16:0/2180304262 conn(0x7fad680a2aa0 msgr2=0x7fad680a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:31.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.313+0000 7fad6e35f700 1 -- 172.21.15.16:0/2180304262 shutdown_connections 2023-09-30T22:35:31.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.313+0000 7fad6e35f700 1 -- 172.21.15.16:0/2180304262 wait complete. 2023-09-30T22:35:31.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.314+0000 7fad6e35f700 1 Processor -- start 2023-09-30T22:35:31.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.315+0000 7fad6e35f700 1 -- start start 2023-09-30T22:35:31.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.315+0000 7fad6e35f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fad6812cf60 0x7fad681c6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:31.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.315+0000 7fad6e35f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fad6812f8b0 0x7fad681c6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:31.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.315+0000 7fad6e35f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad68130430 0x7fad681c29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:31.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.315+0000 7fad6e35f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad6812dab0 con 0x7fad68130430 2023-09-30T22:35:31.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.315+0000 7fad6e35f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad68129fd0 con 0x7fad6812cf60 2023-09-30T22:35:31.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.315+0000 7fad6e35f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fad681c6e80 con 0x7fad6812f8b0 2023-09-30T22:35:31.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.315+0000 7fad677fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fad6812f8b0 0x7fad681c6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:31.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.315+0000 7fad677fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fad6812f8b0 0x7fad681c6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:45284/0 (socket says 172.21.15.16:45284) 2023-09-30T22:35:31.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.315+0000 7fad677fe700 1 -- 172.21.15.16:0/1799767838 learned_addr learned my addr 172.21.15.16:0/1799767838 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:31.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.315+0000 7fad67fff700 1 --2- 172.21.15.16:0/1799767838 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fad6812cf60 0x7fad681c6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:31.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.315+0000 7fad6c8fc700 1 --2- 172.21.15.16:0/1799767838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad68130430 0x7fad681c29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:31.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.316+0000 7fad67fff700 1 -- 172.21.15.16:0/1799767838 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fad6812f8b0 msgr2=0x7fad681c6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:31.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.316+0000 7fad67fff700 1 --2- 172.21.15.16:0/1799767838 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fad6812f8b0 0x7fad681c6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:31.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.316+0000 7fad67fff700 1 -- 172.21.15.16:0/1799767838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad68130430 msgr2=0x7fad681c29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:31.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.316+0000 7fad67fff700 1 --2- 172.21.15.16:0/1799767838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad68130430 0x7fad681c29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:31.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.316+0000 7fad67fff700 1 -- 172.21.15.16:0/1799767838 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad5800a6f0 con 0x7fad6812cf60 2023-09-30T22:35:31.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.316+0000 7fad6c8fc700 1 --2- 172.21.15.16:0/1799767838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad68130430 0x7fad681c29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:31.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.316+0000 7fad67fff700 1 --2- 172.21.15.16:0/1799767838 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fad6812cf60 0x7fad681c6400 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fad58017000 tx=0x7fad580047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:31.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.316+0000 7fad64ff9700 1 -- 172.21.15.16:0/1799767838 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad58003ab0 con 0x7fad6812cf60 2023-09-30T22:35:31.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.316+0000 7fad64ff9700 1 -- 172.21.15.16:0/1799767838 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad58003c10 con 0x7fad6812cf60 2023-09-30T22:35:31.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.316+0000 7fad6e35f700 1 -- 172.21.15.16:0/1799767838 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad6811d130 con 0x7fad6812cf60 2023-09-30T22:35:31.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.316+0000 7fad64ff9700 1 -- 172.21.15.16:0/1799767838 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad58020400 con 0x7fad6812cf60 2023-09-30T22:35:31.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.316+0000 7fad6e35f700 1 -- 172.21.15.16:0/1799767838 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad6811c090 con 0x7fad6812cf60 2023-09-30T22:35:31.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.317+0000 7fad64ff9700 1 -- 172.21.15.16:0/1799767838 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fad58018400 con 0x7fad6812cf60 2023-09-30T22:35:31.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.317+0000 7fad64ff9700 1 -- 172.21.15.16:0/1799767838 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fad58040800 con 0x7fad6812cf60 2023-09-30T22:35:31.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.317+0000 7fad6e35f700 1 -- 172.21.15.16:0/1799767838 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad68074ea0 con 0x7fad6812cf60 2023-09-30T22:35:31.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.319+0000 7f1ef23dd700 1 -- 172.21.15.16:0/2224273583 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1eec074ea0 con 0x7f1eec134170 2023-09-30T22:35:31.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.319+0000 7f1ee8ff9700 1 -- 172.21.15.16:0/2224273583 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1ee4011390 con 0x7f1eec134170 2023-09-30T22:35:31.323 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:35:31.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.321+0000 7f1ef23dd700 1 -- 172.21.15.16:0/2224273583 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1eec134170 msgr2=0x7f1eec123200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:31.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.321+0000 7f1ef23dd700 1 --2- 172.21.15.16:0/2224273583 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1eec134170 0x7f1eec123200 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f1ee4016000 tx=0x7f1ee4004ad0 comp rx=0 tx=0).stop 2023-09-30T22:35:31.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.321+0000 7f1ef23dd700 1 -- 172.21.15.16:0/2224273583 shutdown_connections 2023-09-30T22:35:31.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.321+0000 7f1ef23dd700 1 --2- 172.21.15.16:0/2224273583 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1eec0a0a30 0x7f1eec122780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:31.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.321+0000 7f1ef23dd700 1 --2- 172.21.15.16:0/2224273583 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1eec134170 0x7f1eec123200 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:31.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.321+0000 7f1ef23dd700 1 --2- 172.21.15.16:0/2224273583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1eec0a1340 0x7f1eec122cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:31.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.321+0000 7f1ef23dd700 1 -- 172.21.15.16:0/2224273583 >> 172.21.15.16:0/2224273583 conn(0x7f1eec0a6550 msgr2=0x7f1eec09edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:31.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.321+0000 7f1ef23dd700 1 -- 172.21.15.16:0/2224273583 shutdown_connections 2023-09-30T22:35:31.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.322+0000 7f1ef23dd700 1 -- 172.21.15.16:0/2224273583 wait complete. 2023-09-30T22:35:31.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.323+0000 7fad64ff9700 1 -- 172.21.15.16:0/1799767838 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fad58025080 con 0x7fad6812cf60 2023-09-30T22:35:31.337 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:35:31.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.528+0000 7fad6e35f700 1 -- 172.21.15.16:0/1799767838 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fad680713a0 con 0x7fad6812cf60 2023-09-30T22:35:31.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.528+0000 7fad64ff9700 1 -- 172.21.15.16:0/1799767838 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fad58016300 con 0x7fad6812cf60 2023-09-30T22:35:31.530 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:35:31.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.530+0000 7fad6e35f700 1 -- 172.21.15.16:0/1799767838 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fad6812cf60 msgr2=0x7fad681c6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:31.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.530+0000 7fad6e35f700 1 --2- 172.21.15.16:0/1799767838 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fad6812cf60 0x7fad681c6400 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fad58017000 tx=0x7fad580047c0 comp rx=0 tx=0).stop 2023-09-30T22:35:31.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.531+0000 7fad6e35f700 1 -- 172.21.15.16:0/1799767838 shutdown_connections 2023-09-30T22:35:31.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.531+0000 7fad6e35f700 1 --2- 172.21.15.16:0/1799767838 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fad6812cf60 0x7fad681c6400 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:31.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.531+0000 7fad6e35f700 1 --2- 172.21.15.16:0/1799767838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fad68130430 0x7fad681c29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:31.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.531+0000 7fad6e35f700 1 --2- 172.21.15.16:0/1799767838 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fad6812f8b0 0x7fad681c6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:31.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.531+0000 7fad6e35f700 1 -- 172.21.15.16:0/1799767838 >> 172.21.15.16:0/1799767838 conn(0x7fad680a2aa0 msgr2=0x7fad6809e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:31.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.531+0000 7fad6e35f700 1 -- 172.21.15.16:0/1799767838 shutdown_connections 2023-09-30T22:35:31.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.531+0000 7fad6e35f700 1 -- 172.21.15.16:0/1799767838 wait complete. 2023-09-30T22:35:31.546 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:35:31.821 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:35:31.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.967+0000 7f20a2c15700 1 Processor -- start 2023-09-30T22:35:31.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.967+0000 7f20a2c15700 1 -- start start 2023-09-30T22:35:31.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.967+0000 7f20a2c15700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f209c09e930 0x7f209c09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:31.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.967+0000 7f20a2c15700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f209c09f240 0x7f209c1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:31.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.967+0000 7f20a2c15700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f209c09f650 0x7f209c131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:31.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.968+0000 7f20a2c15700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f209c070940 con 0x7f209c09f240 2023-09-30T22:35:31.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.968+0000 7f20a2c15700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f209c06fd60 con 0x7f209c09e930 2023-09-30T22:35:31.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.968+0000 7f20a2c15700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f209c06fa80 con 0x7f209c09f650 2023-09-30T22:35:31.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.968+0000 7f20a09b1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f209c09e930 0x7f209c09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:31.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.968+0000 7f209bfff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f209c09f240 0x7f209c1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:31.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.968+0000 7f20a11b2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f209c09f650 0x7f209c131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:31.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.968+0000 7f20a09b1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f209c09e930 0x7f209c09ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:45986/0 (socket says 172.21.15.16:45986) 2023-09-30T22:35:31.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.968+0000 7f20a09b1700 1 -- 172.21.15.16:0/84184925 learned_addr learned my addr 172.21.15.16:0/84184925 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:31.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.968+0000 7f209bfff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f209c09f240 0x7f209c1317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54968/0 (socket says 172.21.15.16:54968) 2023-09-30T22:35:31.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.968+0000 7f20a11b2700 1 -- 172.21.15.16:0/84184925 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f209c09e930 msgr2=0x7f209c09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:31.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.968+0000 7f20a11b2700 1 --2- 172.21.15.16:0/84184925 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f209c09e930 0x7f209c09ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:31.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.968+0000 7f20a11b2700 1 -- 172.21.15.16:0/84184925 >> v2:172.21.15.16:3300/0 conn(0x7f209c09f240 msgr2=0x7f209c1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:31.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.968+0000 7f20a11b2700 1 --2- 172.21.15.16:0/84184925 >> v2:172.21.15.16:3300/0 conn(0x7f209c09f240 0x7f209c1317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:31.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.968+0000 7f20a11b2700 1 -- 172.21.15.16:0/84184925 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f209c09fb60 con 0x7f209c09f650 2023-09-30T22:35:31.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.968+0000 7f209bfff700 1 --2- 172.21.15.16:0/84184925 >> v2:172.21.15.16:3300/0 conn(0x7f209c09f240 0x7f209c1317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:31.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.968+0000 7f20a11b2700 1 --2- 172.21.15.16:0/84184925 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f209c09f650 0x7f209c131d30 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f2094009d90 tx=0x7f209400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63e0cb5267076a65 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:31.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.968+0000 7f209b7fe700 1 -- 172.21.15.16:0/84184925 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209400e070 con 0x7f209c09f650 2023-09-30T22:35:31.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.968+0000 7f209b7fe700 1 -- 172.21.15.16:0/84184925 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f209400bb20 con 0x7f209c09f650 2023-09-30T22:35:31.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.968+0000 7f209b7fe700 1 -- 172.21.15.16:0/84184925 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2094012430 con 0x7f209c09f650 2023-09-30T22:35:31.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.969+0000 7f20a2c15700 1 -- 172.21.15.16:0/84184925 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f209c09f650 msgr2=0x7f209c131d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:31.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.969+0000 7f20a2c15700 1 --2- 172.21.15.16:0/84184925 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f209c09f650 0x7f209c131d30 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f2094009d90 tx=0x7f209400b330 comp rx=0 tx=0).stop 2023-09-30T22:35:31.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.969+0000 7f20a2c15700 1 -- 172.21.15.16:0/84184925 shutdown_connections 2023-09-30T22:35:31.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.969+0000 7f20a2c15700 1 --2- 172.21.15.16:0/84184925 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f209c09e930 0x7f209c09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:31.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.969+0000 7f20a2c15700 1 --2- 172.21.15.16:0/84184925 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f209c09f650 0x7f209c131d30 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:31.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.969+0000 7f20a2c15700 1 --2- 172.21.15.16:0/84184925 >> v2:172.21.15.16:3300/0 conn(0x7f209c09f240 0x7f209c1317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:31.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.969+0000 7f20a2c15700 1 -- 172.21.15.16:0/84184925 >> 172.21.15.16:0/84184925 conn(0x7f209c0a4210 msgr2=0x7f209c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:31.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.969+0000 7f20a2c15700 1 -- 172.21.15.16:0/84184925 shutdown_connections 2023-09-30T22:35:31.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.969+0000 7f20a2c15700 1 -- 172.21.15.16:0/84184925 wait complete. 2023-09-30T22:35:31.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.970+0000 7f20a2c15700 1 Processor -- start 2023-09-30T22:35:31.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.970+0000 7f20a2c15700 1 -- start start 2023-09-30T22:35:31.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.971+0000 7f20a2c15700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f209c09e930 0x7f209c06b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:31.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.971+0000 7f20a2c15700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f209c09f240 0x7f209c06bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:31.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.971+0000 7f20a2c15700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f209c09f650 0x7f209c06c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:31.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.971+0000 7f20a2c15700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209c06d4c0 con 0x7f209c09f240 2023-09-30T22:35:31.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.971+0000 7f20a2c15700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f209c1a92f0 con 0x7f209c09f650 2023-09-30T22:35:31.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.971+0000 7f20a2c15700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f209c06cd10 con 0x7f209c09e930 2023-09-30T22:35:31.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.971+0000 7f209bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f209c09f240 0x7f209c06bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:31.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.971+0000 7f20a11b2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f209c09f650 0x7f209c06c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:31.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.971+0000 7f209bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f209c09f240 0x7f209c06bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54982/0 (socket says 172.21.15.16:54982) 2023-09-30T22:35:31.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.971+0000 7f209bfff700 1 -- 172.21.15.16:0/3962016653 learned_addr learned my addr 172.21.15.16:0/3962016653 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:31.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.971+0000 7f20a11b2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f209c09f650 0x7f209c06c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:46002/0 (socket says 172.21.15.16:46002) 2023-09-30T22:35:31.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.971+0000 7f20a09b1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f209c09e930 0x7f209c06b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:31.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.971+0000 7f20a11b2700 1 -- 172.21.15.16:0/3962016653 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f209c09e930 msgr2=0x7f209c06b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:31.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.971+0000 7f20a11b2700 1 --2- 172.21.15.16:0/3962016653 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f209c09e930 0x7f209c06b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:31.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.971+0000 7f20a11b2700 1 -- 172.21.15.16:0/3962016653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f209c09f240 msgr2=0x7f209c06bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:31.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.971+0000 7f20a11b2700 1 --2- 172.21.15.16:0/3962016653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f209c09f240 0x7f209c06bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:31.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.971+0000 7f20a11b2700 1 -- 172.21.15.16:0/3962016653 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20880094a0 con 0x7f209c09f650 2023-09-30T22:35:31.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.971+0000 7f209bfff700 1 --2- 172.21.15.16:0/3962016653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f209c09f240 0x7f209c06bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:31.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.972+0000 7f20a11b2700 1 --2- 172.21.15.16:0/3962016653 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f209c09f650 0x7f209c06c330 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f2094016000 tx=0x7f2094004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:31.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.972+0000 7f20997fa700 1 -- 172.21.15.16:0/3962016653 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2094004800 con 0x7f209c09f650 2023-09-30T22:35:31.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.972+0000 7f20997fa700 1 -- 172.21.15.16:0/3962016653 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2094020070 con 0x7f209c09f650 2023-09-30T22:35:31.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.972+0000 7f20a2c15700 1 -- 172.21.15.16:0/3962016653 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2094009a70 con 0x7f209c09f650 2023-09-30T22:35:31.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.972+0000 7f20997fa700 1 -- 172.21.15.16:0/3962016653 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20940276c0 con 0x7f209c09f650 2023-09-30T22:35:31.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.972+0000 7f20a2c15700 1 -- 172.21.15.16:0/3962016653 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f209c0a0190 con 0x7f209c09f650 2023-09-30T22:35:31.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.973+0000 7f20997fa700 1 -- 172.21.15.16:0/3962016653 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2094012430 con 0x7f209c09f650 2023-09-30T22:35:31.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.973+0000 7f20997fa700 1 -- 172.21.15.16:0/3962016653 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f2094022070 con 0x7f209c09f650 2023-09-30T22:35:31.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.973+0000 7f20a2c15700 1 -- 172.21.15.16:0/3962016653 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f209c1a9080 con 0x7f209c09f650 2023-09-30T22:35:31.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:31.978+0000 7f20997fa700 1 -- 172.21.15.16:0/3962016653 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f209c1a9080 con 0x7f209c09f650 2023-09-30T22:35:32.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.181+0000 7f20a2c15700 1 -- 172.21.15.16:0/3962016653 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f209c1a9080 con 0x7f209c09f650 2023-09-30T22:35:32.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.182+0000 7f20997fa700 1 -- 172.21.15.16:0/3962016653 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f209c1a9080 con 0x7f209c09f650 2023-09-30T22:35:32.183 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:35:32.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.184+0000 7f20a2c15700 1 -- 172.21.15.16:0/3962016653 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f209c09f650 msgr2=0x7f209c06c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:32.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.184+0000 7f20a2c15700 1 --2- 172.21.15.16:0/3962016653 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f209c09f650 0x7f209c06c330 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f2094016000 tx=0x7f2094004ad0 comp rx=0 tx=0).stop 2023-09-30T22:35:32.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.184+0000 7f20a2c15700 1 -- 172.21.15.16:0/3962016653 shutdown_connections 2023-09-30T22:35:32.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.184+0000 7f20a2c15700 1 --2- 172.21.15.16:0/3962016653 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f209c09f650 0x7f209c06c330 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:32.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.184+0000 7f20a2c15700 1 --2- 172.21.15.16:0/3962016653 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f209c09e930 0x7f209c06b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:32.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.184+0000 7f20a2c15700 1 --2- 172.21.15.16:0/3962016653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f209c09f240 0x7f209c06bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:32.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.184+0000 7f20a2c15700 1 -- 172.21.15.16:0/3962016653 >> 172.21.15.16:0/3962016653 conn(0x7f209c0a4210 msgr2=0x7f209c1b8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:32.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.184+0000 7f20a2c15700 1 -- 172.21.15.16:0/3962016653 shutdown_connections 2023-09-30T22:35:32.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.184+0000 7f20a2c15700 1 -- 172.21.15.16:0/3962016653 wait complete. 2023-09-30T22:35:32.199 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:35:32.299 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:35:32.338 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:35:32.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.444+0000 7f10fc8d6700 1 Processor -- start 2023-09-30T22:35:32.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.444+0000 7f10fc8d6700 1 -- start start 2023-09-30T22:35:32.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.444+0000 7f10fc8d6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f10f80a0a30 0x7f10f80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:32.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.444+0000 7f10fc8d6700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f10f80a1340 0x7f10f8133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:32.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.445+0000 7f10fc8d6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f10f8133fa0 0x7f10f8134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:32.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.445+0000 7f10fc8d6700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10f809edd0 con 0x7f10f80a1340 2023-09-30T22:35:32.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.445+0000 7f10fc8d6700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10f809e1f0 con 0x7f10f8133fa0 2023-09-30T22:35:32.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.445+0000 7f10fc8d6700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f10f809dc50 con 0x7f10f80a0a30 2023-09-30T22:35:32.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.445+0000 7f10f659c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f10f80a0a30 0x7f10f80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:32.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.445+0000 7f10f5d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f10f80a1340 0x7f10f8133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:32.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.445+0000 7f10f6d9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f10f8133fa0 0x7f10f8134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:32.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.445+0000 7f10f659c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f10f80a0a30 0x7f10f80a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:45312/0 (socket says 172.21.15.16:45312) 2023-09-30T22:35:32.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.445+0000 7f10f5d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f10f80a1340 0x7f10f8133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54998/0 (socket says 172.21.15.16:54998) 2023-09-30T22:35:32.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.445+0000 7f10f659c700 1 -- 172.21.15.16:0/118101537 learned_addr learned my addr 172.21.15.16:0/118101537 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:32.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.445+0000 7f10f6d9d700 1 -- 172.21.15.16:0/118101537 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f10f80a0a30 msgr2=0x7f10f80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:32.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.445+0000 7f10f6d9d700 1 --2- 172.21.15.16:0/118101537 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f10f80a0a30 0x7f10f80a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:32.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.445+0000 7f10f6d9d700 1 -- 172.21.15.16:0/118101537 >> v2:172.21.15.16:3300/0 conn(0x7f10f80a1340 msgr2=0x7f10f8133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:32.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.445+0000 7f10f6d9d700 1 --2- 172.21.15.16:0/118101537 >> v2:172.21.15.16:3300/0 conn(0x7f10f80a1340 0x7f10f8133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:32.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.445+0000 7f10f6d9d700 1 -- 172.21.15.16:0/118101537 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f80a1fe0 con 0x7f10f8133fa0 2023-09-30T22:35:32.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.445+0000 7f10f6d9d700 1 --2- 172.21.15.16:0/118101537 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f10f8133fa0 0x7f10f8134410 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f10f0009870 tx=0x7f10f0009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bab94528c7fd7da server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:32.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.445+0000 7f10f559a700 1 -- 172.21.15.16:0/118101537 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f0011830 con 0x7f10f8133fa0 2023-09-30T22:35:32.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.446+0000 7f10f559a700 1 -- 172.21.15.16:0/118101537 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10f0011990 con 0x7f10f8133fa0 2023-09-30T22:35:32.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.446+0000 7f10f559a700 1 -- 172.21.15.16:0/118101537 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f0011d20 con 0x7f10f8133fa0 2023-09-30T22:35:32.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.446+0000 7f10fc8d6700 1 -- 172.21.15.16:0/118101537 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f10f8133fa0 msgr2=0x7f10f8134410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:32.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.446+0000 7f10fc8d6700 1 --2- 172.21.15.16:0/118101537 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f10f8133fa0 0x7f10f8134410 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f10f0009870 tx=0x7f10f0009b80 comp rx=0 tx=0).stop 2023-09-30T22:35:32.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.446+0000 7f10fc8d6700 1 -- 172.21.15.16:0/118101537 shutdown_connections 2023-09-30T22:35:32.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.446+0000 7f10fc8d6700 1 --2- 172.21.15.16:0/118101537 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f10f8133fa0 0x7f10f8134410 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:32.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.446+0000 7f10fc8d6700 1 --2- 172.21.15.16:0/118101537 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f10f80a0a30 0x7f10f80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:32.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.446+0000 7f10fc8d6700 1 --2- 172.21.15.16:0/118101537 >> v2:172.21.15.16:3300/0 conn(0x7f10f80a1340 0x7f10f8133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:32.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.446+0000 7f10fc8d6700 1 -- 172.21.15.16:0/118101537 >> 172.21.15.16:0/118101537 conn(0x7f10f80a6550 msgr2=0x7f10f80a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:32.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.446+0000 7f10fc8d6700 1 -- 172.21.15.16:0/118101537 shutdown_connections 2023-09-30T22:35:32.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.446+0000 7f10fc8d6700 1 -- 172.21.15.16:0/118101537 wait complete. 2023-09-30T22:35:32.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.448+0000 7f10fc8d6700 1 Processor -- start 2023-09-30T22:35:32.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.448+0000 7f10fc8d6700 1 -- start start 2023-09-30T22:35:32.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.448+0000 7f10fc8d6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10f80a0a30 0x7f10f81200a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:32.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.448+0000 7f10fc8d6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f10f80a1340 0x7f10f81205e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:32.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.448+0000 7f10fc8d6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f10f8133fa0 0x7f10f8120b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:32.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.448+0000 7f10fc8d6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f81233e0 con 0x7f10f80a0a30 2023-09-30T22:35:32.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.448+0000 7f10fc8d6700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10f8122eb0 con 0x7f10f80a1340 2023-09-30T22:35:32.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.448+0000 7f10fc8d6700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f10f8121d00 con 0x7f10f8133fa0 2023-09-30T22:35:32.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.448+0000 7f10f659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10f80a0a30 0x7f10f81200a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:32.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.448+0000 7f10f5d9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f10f80a1340 0x7f10f81205e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:32.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.448+0000 7f10f659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10f80a0a30 0x7f10f81200a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55012/0 (socket says 172.21.15.16:55012) 2023-09-30T22:35:32.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.448+0000 7f10f659c700 1 -- 172.21.15.16:0/201314673 learned_addr learned my addr 172.21.15.16:0/201314673 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:32.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.448+0000 7f10f6d9d700 1 --2- 172.21.15.16:0/201314673 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f10f8133fa0 0x7f10f8120b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:32.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.449+0000 7f10f659c700 1 -- 172.21.15.16:0/201314673 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f10f8133fa0 msgr2=0x7f10f8120b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:32.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.449+0000 7f10f659c700 1 --2- 172.21.15.16:0/201314673 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f10f8133fa0 0x7f10f8120b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:32.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.449+0000 7f10f659c700 1 -- 172.21.15.16:0/201314673 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f10f80a1340 msgr2=0x7f10f81205e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:32.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.449+0000 7f10f659c700 1 --2- 172.21.15.16:0/201314673 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f10f80a1340 0x7f10f81205e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:32.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.449+0000 7f10f659c700 1 -- 172.21.15.16:0/201314673 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f00094a0 con 0x7f10f80a0a30 2023-09-30T22:35:32.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.449+0000 7f10f5d9b700 1 --2- 172.21.15.16:0/201314673 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f10f80a1340 0x7f10f81205e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:32.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.449+0000 7f10f659c700 1 --2- 172.21.15.16:0/201314673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10f80a0a30 0x7f10f81200a0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f10e8009d90 tx=0x7f10e800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:32.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.449+0000 7f10e6ffd700 1 -- 172.21.15.16:0/201314673 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e800cb00 con 0x7f10f80a0a30 2023-09-30T22:35:32.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.449+0000 7f10e6ffd700 1 -- 172.21.15.16:0/201314673 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10e800f040 con 0x7f10f80a0a30 2023-09-30T22:35:32.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.449+0000 7f10fc8d6700 1 -- 172.21.15.16:0/201314673 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10f8123890 con 0x7f10f80a0a30 2023-09-30T22:35:32.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.449+0000 7f10e6ffd700 1 -- 172.21.15.16:0/201314673 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e800ce60 con 0x7f10f80a0a30 2023-09-30T22:35:32.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.449+0000 7f10fc8d6700 1 -- 172.21.15.16:0/201314673 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10f80a0220 con 0x7f10f80a0a30 2023-09-30T22:35:32.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.450+0000 7f10e6ffd700 1 -- 172.21.15.16:0/201314673 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f10e8014480 con 0x7f10f80a0a30 2023-09-30T22:35:32.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.450+0000 7f10fc8d6700 1 -- 172.21.15.16:0/201314673 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10f8074ea0 con 0x7f10f80a0a30 2023-09-30T22:35:32.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.450+0000 7f10e6ffd700 1 -- 172.21.15.16:0/201314673 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f10e803fba0 con 0x7f10f80a0a30 2023-09-30T22:35:32.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.455+0000 7f10e6ffd700 1 -- 172.21.15.16:0/201314673 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f10e8012700 con 0x7f10f80a0a30 2023-09-30T22:35:32.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.483+0000 7f7150214700 1 Processor -- start 2023-09-30T22:35:32.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.483+0000 7f7150214700 1 -- start start 2023-09-30T22:35:32.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.483+0000 7f7150214700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f71480a0a30 0x7f71480a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:32.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.483+0000 7f7150214700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f714809d630 0x7f71480a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:32.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.483+0000 7f7150214700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7148133930 0x7f7148133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:32.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.483+0000 7f7150214700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71480a2a70 con 0x7f714809d630 2023-09-30T22:35:32.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.483+0000 7f7150214700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71480a01d0 con 0x7f7148133930 2023-09-30T22:35:32.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.483+0000 7f7150214700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f714809e2b0 con 0x7f71480a0a30 2023-09-30T22:35:32.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.484+0000 7f714dfb0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f71480a0a30 0x7f71480a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:32.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.484+0000 7f714e7b1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7148133930 0x7f7148133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:32.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.484+0000 7f714d7af700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f714809d630 0x7f71480a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:32.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.484+0000 7f714dfb0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f71480a0a30 0x7f71480a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:45322/0 (socket says 172.21.15.16:45322) 2023-09-30T22:35:32.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.484+0000 7f714dfb0700 1 -- 172.21.15.16:0/1848950775 learned_addr learned my addr 172.21.15.16:0/1848950775 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:32.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.484+0000 7f714dfb0700 1 -- 172.21.15.16:0/1848950775 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7148133930 msgr2=0x7f7148133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:32.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.484+0000 7f714dfb0700 1 --2- 172.21.15.16:0/1848950775 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7148133930 0x7f7148133dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:32.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.484+0000 7f714dfb0700 1 -- 172.21.15.16:0/1848950775 >> v2:172.21.15.16:3300/0 conn(0x7f714809d630 msgr2=0x7f71480a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:32.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.484+0000 7f714dfb0700 1 --2- 172.21.15.16:0/1848950775 >> v2:172.21.15.16:3300/0 conn(0x7f714809d630 0x7f71480a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:32.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.484+0000 7f714dfb0700 1 -- 172.21.15.16:0/1848950775 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f714809dc50 con 0x7f71480a0a30 2023-09-30T22:35:32.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.484+0000 7f714e7b1700 1 --2- 172.21.15.16:0/1848950775 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7148133930 0x7f7148133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:32.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.484+0000 7f714dfb0700 1 --2- 172.21.15.16:0/1848950775 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f71480a0a30 0x7f71480a0e00 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f71380098f0 tx=0x7f7138009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d1ef68c131d2729e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:32.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.484+0000 7f714cfae700 1 -- 172.21.15.16:0/1848950775 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7138011830 con 0x7f71480a0a30 2023-09-30T22:35:32.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.484+0000 7f714cfae700 1 -- 172.21.15.16:0/1848950775 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7138011990 con 0x7f71480a0a30 2023-09-30T22:35:32.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.484+0000 7f714cfae700 1 -- 172.21.15.16:0/1848950775 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7138011d20 con 0x7f71480a0a30 2023-09-30T22:35:32.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.485+0000 7f7150214700 1 -- 172.21.15.16:0/1848950775 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f71480a0a30 msgr2=0x7f71480a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:32.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.485+0000 7f7150214700 1 --2- 172.21.15.16:0/1848950775 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f71480a0a30 0x7f71480a0e00 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f71380098f0 tx=0x7f7138009c00 comp rx=0 tx=0).stop 2023-09-30T22:35:32.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.485+0000 7f7150214700 1 -- 172.21.15.16:0/1848950775 shutdown_connections 2023-09-30T22:35:32.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.485+0000 7f7150214700 1 --2- 172.21.15.16:0/1848950775 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7148133930 0x7f7148133dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:32.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.485+0000 7f7150214700 1 --2- 172.21.15.16:0/1848950775 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f71480a0a30 0x7f71480a0e00 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:32.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.485+0000 7f7150214700 1 --2- 172.21.15.16:0/1848950775 >> v2:172.21.15.16:3300/0 conn(0x7f714809d630 0x7f71480a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:32.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.485+0000 7f7150214700 1 -- 172.21.15.16:0/1848950775 >> 172.21.15.16:0/1848950775 conn(0x7f71480a3050 msgr2=0x7f71480a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:32.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.485+0000 7f7150214700 1 -- 172.21.15.16:0/1848950775 shutdown_connections 2023-09-30T22:35:32.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.485+0000 7f7150214700 1 -- 172.21.15.16:0/1848950775 wait complete. 2023-09-30T22:35:32.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.486+0000 7f7150214700 1 Processor -- start 2023-09-30T22:35:32.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.486+0000 7f7150214700 1 -- start start 2023-09-30T22:35:32.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.487+0000 7f7150214700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f714809d630 0x7f714811f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:32.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.487+0000 7f7150214700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f71480a0a30 0x7f714811f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:32.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.487+0000 7f7150214700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7148133930 0x7f71481c8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:32.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.487+0000 7f7150214700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7148120940 con 0x7f714809d630 2023-09-30T22:35:32.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.487+0000 7f7150214700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7148120750 con 0x7f7148133930 2023-09-30T22:35:32.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.487+0000 7f7150214700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f71481334d0 con 0x7f71480a0a30 2023-09-30T22:35:32.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.487+0000 7f714dfb0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f714809d630 0x7f714811f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:32.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.487+0000 7f714dfb0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f714809d630 0x7f714811f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55030/0 (socket says 172.21.15.16:55030) 2023-09-30T22:35:32.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.487+0000 7f714dfb0700 1 -- 172.21.15.16:0/1048308913 learned_addr learned my addr 172.21.15.16:0/1048308913 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:32.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.487+0000 7f714e7b1700 1 --2- 172.21.15.16:0/1048308913 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7148133930 0x7f71481c8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:32.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.487+0000 7f714d7af700 1 --2- 172.21.15.16:0/1048308913 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f71480a0a30 0x7f714811f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:32.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.487+0000 7f714dfb0700 1 -- 172.21.15.16:0/1048308913 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f71480a0a30 msgr2=0x7f714811f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:32.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.487+0000 7f714dfb0700 1 --2- 172.21.15.16:0/1048308913 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f71480a0a30 0x7f714811f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:32.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.487+0000 7f714dfb0700 1 -- 172.21.15.16:0/1048308913 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7148133930 msgr2=0x7f71481c8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:32.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.487+0000 7f714dfb0700 1 --2- 172.21.15.16:0/1048308913 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7148133930 0x7f71481c8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:32.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.487+0000 7f714dfb0700 1 -- 172.21.15.16:0/1048308913 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7140009a70 con 0x7f714809d630 2023-09-30T22:35:32.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.487+0000 7f714e7b1700 1 --2- 172.21.15.16:0/1048308913 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7148133930 0x7f71481c8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:32.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.487+0000 7f714dfb0700 1 --2- 172.21.15.16:0/1048308913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f714809d630 0x7f714811f480 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f7138000c00 tx=0x7f7138004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:32.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.488+0000 7f71467fc700 1 -- 172.21.15.16:0/1048308913 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7138004240 con 0x7f714809d630 2023-09-30T22:35:32.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.488+0000 7f71467fc700 1 -- 172.21.15.16:0/1048308913 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f713801f070 con 0x7f714809d630 2023-09-30T22:35:32.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.488+0000 7f71467fc700 1 -- 172.21.15.16:0/1048308913 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71380045c0 con 0x7f714809d630 2023-09-30T22:35:32.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.488+0000 7f7150214700 1 -- 172.21.15.16:0/1048308913 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7138009550 con 0x7f714809d630 2023-09-30T22:35:32.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.488+0000 7f7150214700 1 -- 172.21.15.16:0/1048308913 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7148122380 con 0x7f714809d630 2023-09-30T22:35:32.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.488+0000 7f71467fc700 1 -- 172.21.15.16:0/1048308913 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7138021030 con 0x7f714809d630 2023-09-30T22:35:32.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.489+0000 7f71467fc700 1 -- 172.21.15.16:0/1048308913 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f7138040640 con 0x7f714809d630 2023-09-30T22:35:32.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.489+0000 7f7150214700 1 -- 172.21.15.16:0/1048308913 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71481221e0 con 0x7f714809d630 2023-09-30T22:35:32.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.494+0000 7f71467fc700 1 -- 172.21.15.16:0/1048308913 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f71481221e0 con 0x7f714809d630 2023-09-30T22:35:32.547 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:35:32.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.667+0000 7f10fc8d6700 1 -- 172.21.15.16:0/201314673 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f10f80a28e0 con 0x7f10f80a0a30 2023-09-30T22:35:32.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.667+0000 7f10e6ffd700 1 -- 172.21.15.16:0/201314673 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f10e8012340 con 0x7f10f80a0a30 2023-09-30T22:35:32.669 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:35:32.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.669+0000 7f10fc8d6700 1 -- 172.21.15.16:0/201314673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10f80a0a30 msgr2=0x7f10f81200a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:32.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.669+0000 7f10fc8d6700 1 --2- 172.21.15.16:0/201314673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10f80a0a30 0x7f10f81200a0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f10e8009d90 tx=0x7f10e800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:35:32.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.670+0000 7f10fc8d6700 1 -- 172.21.15.16:0/201314673 shutdown_connections 2023-09-30T22:35:32.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.670+0000 7f10fc8d6700 1 --2- 172.21.15.16:0/201314673 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f10f8133fa0 0x7f10f8120b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:32.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.670+0000 7f10fc8d6700 1 --2- 172.21.15.16:0/201314673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f10f80a0a30 0x7f10f81200a0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:32.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.670+0000 7f10fc8d6700 1 --2- 172.21.15.16:0/201314673 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f10f80a1340 0x7f10f81205e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:32.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.670+0000 7f10fc8d6700 1 -- 172.21.15.16:0/201314673 >> 172.21.15.16:0/201314673 conn(0x7f10f80a6550 msgr2=0x7f10f81baff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:32.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.670+0000 7f10fc8d6700 1 -- 172.21.15.16:0/201314673 shutdown_connections 2023-09-30T22:35:32.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.670+0000 7f10fc8d6700 1 -- 172.21.15.16:0/201314673 wait complete. 2023-09-30T22:35:32.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.683+0000 7f67a8dfb700 1 Processor -- start 2023-09-30T22:35:32.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.683+0000 7f67a8dfb700 1 -- start start 2023-09-30T22:35:32.686 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:35:32.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.684+0000 7f67a8dfb700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f67a409f410 0x7f67a4136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:32.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.684+0000 7f67a8dfb700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f67a4136fe0 0x7f67a41393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:32.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.684+0000 7f67a8dfb700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67a4070940 con 0x7f67a409e930 2023-09-30T22:35:32.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.684+0000 7f67a8dfb700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67a406fd60 con 0x7f67a4136fe0 2023-09-30T22:35:32.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.684+0000 7f67a8dfb700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f67a406fa80 con 0x7f67a409f410 2023-09-30T22:35:32.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.684+0000 7f67a1d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f67a409f410 0x7f67a4136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:32.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.684+0000 7f67a1d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f67a409f410 0x7f67a4136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:45346/0 (socket says 172.21.15.16:45346) 2023-09-30T22:35:32.691 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.684+0000 7f67a2d9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f67a4136fe0 0x7f67a41393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:32.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.684+0000 7f67a1d9b700 1 -- 172.21.15.16:0/2702083736 learned_addr learned my addr 172.21.15.16:0/2702083736 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:32.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.684+0000 7f67a1d9b700 1 -- 172.21.15.16:0/2702083736 >> v2:172.21.15.16:3300/0 conn(0x7f67a4136fe0 msgr2=0x7f67a41393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:32.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.684+0000 7f67a1d9b700 1 --2- 172.21.15.16:0/2702083736 >> v2:172.21.15.16:3300/0 conn(0x7f67a4136fe0 0x7f67a41393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:32.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.684+0000 7f67a1d9b700 1 -- 172.21.15.16:0/2702083736 >> v1:172.21.15.16:6789/0 conn(0x7f67a409e930 legacy=0x7f67a409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:32.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.685+0000 7f67a1d9b700 1 -- 172.21.15.16:0/2702083736 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a409fb60 con 0x7f67a409f410 2023-09-30T22:35:32.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.685+0000 7f67a159a700 1 -- 172.21.15.16:0/2702083736 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1156994793 0 0) 0x7f67a4070940 con 0x7f67a409e930 2023-09-30T22:35:32.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.685+0000 7f67a2d9d700 1 --2- 172.21.15.16:0/2702083736 >> v2:172.21.15.16:3300/0 conn(0x7f67a4136fe0 0x7f67a41393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:32.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.685+0000 7f67a1d9b700 1 --2- 172.21.15.16:0/2702083736 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f67a409f410 0x7f67a4136aa0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f67980098f0 tx=0x7f6798009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=506fd0fbf844b5f3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:32.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.685+0000 7f67a159a700 1 -- 172.21.15.16:0/2702083736 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6798011830 con 0x7f67a409f410 2023-09-30T22:35:32.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.685+0000 7f67a159a700 1 -- 172.21.15.16:0/2702083736 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6798011990 con 0x7f67a409f410 2023-09-30T22:35:32.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.685+0000 7f67a159a700 1 -- 172.21.15.16:0/2702083736 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6798011d20 con 0x7f67a409f410 2023-09-30T22:35:32.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.686+0000 7f67a8dfb700 1 -- 172.21.15.16:0/2702083736 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f67a409f410 msgr2=0x7f67a4136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:32.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.686+0000 7f67a8dfb700 1 --2- 172.21.15.16:0/2702083736 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f67a409f410 0x7f67a4136aa0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f67980098f0 tx=0x7f6798009c00 comp rx=0 tx=0).stop 2023-09-30T22:35:32.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.686+0000 7f67a8dfb700 1 -- 172.21.15.16:0/2702083736 shutdown_connections 2023-09-30T22:35:32.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.686+0000 7f67a8dfb700 1 --2- 172.21.15.16:0/2702083736 >> v2:172.21.15.16:3300/0 conn(0x7f67a4136fe0 0x7f67a41393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:32.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.686+0000 7f67a8dfb700 1 --2- 172.21.15.16:0/2702083736 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f67a409f410 0x7f67a4136aa0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:32.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.686+0000 7f67a8dfb700 1 -- 172.21.15.16:0/2702083736 >> 172.21.15.16:0/2702083736 conn(0x7f67a40a4210 msgr2=0x7f67a40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:32.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.686+0000 7f67a8dfb700 1 -- 172.21.15.16:0/2702083736 shutdown_connections 2023-09-30T22:35:32.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.686+0000 7f67a8dfb700 1 -- 172.21.15.16:0/2702083736 wait complete. 2023-09-30T22:35:32.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.687+0000 7f67a8dfb700 1 Processor -- start 2023-09-30T22:35:32.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.687+0000 7f67a8dfb700 1 -- start start 2023-09-30T22:35:32.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.687+0000 7f67a8dfb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67a409e930 0x7f67a406c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:32.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.688+0000 7f67a8dfb700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f67a409f410 0x7f67a406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:32.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.688+0000 7f67a8dfb700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f67a4136fe0 0x7f67a41c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:32.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.688+0000 7f67a8dfb700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a4121bb0 con 0x7f67a409e930 2023-09-30T22:35:32.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.688+0000 7f67a8dfb700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67a4122200 con 0x7f67a4136fe0 2023-09-30T22:35:32.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.688+0000 7f67a8dfb700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f67a4132260 con 0x7f67a409f410 2023-09-30T22:35:32.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.688+0000 7f67a259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67a409e930 0x7f67a406c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:32.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.688+0000 7f67a1d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f67a409f410 0x7f67a406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:32.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.688+0000 7f67a259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67a409e930 0x7f67a406c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55040/0 (socket says 172.21.15.16:55040) 2023-09-30T22:35:32.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.688+0000 7f67a259c700 1 -- 172.21.15.16:0/3016370935 learned_addr learned my addr 172.21.15.16:0/3016370935 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:32.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.688+0000 7f67a1d9b700 1 -- 172.21.15.16:0/3016370935 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f67a4136fe0 msgr2=0x7f67a41c82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:32.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.688+0000 7f67a1d9b700 1 --2- 172.21.15.16:0/3016370935 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f67a4136fe0 0x7f67a41c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:32.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.688+0000 7f67a1d9b700 1 -- 172.21.15.16:0/3016370935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67a409e930 msgr2=0x7f67a406c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:32.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.688+0000 7f67a1d9b700 1 --2- 172.21.15.16:0/3016370935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67a409e930 0x7f67a406c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:32.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.688+0000 7f67a1d9b700 1 -- 172.21.15.16:0/3016370935 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f679c009a70 con 0x7f67a409f410 2023-09-30T22:35:32.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.688+0000 7f67a259c700 1 --2- 172.21.15.16:0/3016370935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67a409e930 0x7f67a406c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:32.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.688+0000 7f67a1d9b700 1 --2- 172.21.15.16:0/3016370935 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f67a409f410 0x7f67a406c8f0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f67a406f570 tx=0x7f6798004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:32.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.688+0000 7f6792ffd700 1 -- 172.21.15.16:0/3016370935 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6798004300 con 0x7f67a409f410 2023-09-30T22:35:32.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.688+0000 7f6792ffd700 1 -- 172.21.15.16:0/3016370935 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f679801f070 con 0x7f67a409f410 2023-09-30T22:35:32.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.688+0000 7f67a8dfb700 1 -- 172.21.15.16:0/3016370935 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6798009550 con 0x7f67a409f410 2023-09-30T22:35:32.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.688+0000 7f6792ffd700 1 -- 172.21.15.16:0/3016370935 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67980036a0 con 0x7f67a409f410 2023-09-30T22:35:32.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.689+0000 7f67a8dfb700 1 -- 172.21.15.16:0/3016370935 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67a406bd60 con 0x7f67a409f410 2023-09-30T22:35:32.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.689+0000 7f6792ffd700 1 -- 172.21.15.16:0/3016370935 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6798021030 con 0x7f67a409f410 2023-09-30T22:35:32.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.690+0000 7f6792ffd700 1 -- 172.21.15.16:0/3016370935 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f6798040920 con 0x7f67a409f410 2023-09-30T22:35:32.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.690+0000 7f67a8dfb700 1 -- 172.21.15.16:0/3016370935 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67a406d4c0 con 0x7f67a409f410 2023-09-30T22:35:32.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.695+0000 7f6792ffd700 1 -- 172.21.15.16:0/3016370935 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6798015080 con 0x7f67a409f410 2023-09-30T22:35:32.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.705+0000 7f7150214700 1 -- 172.21.15.16:0/1048308913 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f71481221e0 con 0x7f714809d630 2023-09-30T22:35:32.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.705+0000 7f71467fc700 1 -- 172.21.15.16:0/1048308913 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f71481221e0 con 0x7f714809d630 2023-09-30T22:35:32.706 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:35:32.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.707+0000 7f7150214700 1 -- 172.21.15.16:0/1048308913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f714809d630 msgr2=0x7f714811f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:32.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.707+0000 7f7150214700 1 --2- 172.21.15.16:0/1048308913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f714809d630 0x7f714811f480 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f7138000c00 tx=0x7f7138004750 comp rx=0 tx=0).stop 2023-09-30T22:35:32.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.707+0000 7f7150214700 1 -- 172.21.15.16:0/1048308913 shutdown_connections 2023-09-30T22:35:32.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.707+0000 7f7150214700 1 --2- 172.21.15.16:0/1048308913 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7148133930 0x7f71481c8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:32.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.707+0000 7f7150214700 1 --2- 172.21.15.16:0/1048308913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f714809d630 0x7f714811f480 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:32.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.707+0000 7f7150214700 1 --2- 172.21.15.16:0/1048308913 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f71480a0a30 0x7f714811f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:32.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.707+0000 7f7150214700 1 -- 172.21.15.16:0/1048308913 >> 172.21.15.16:0/1048308913 conn(0x7f71480a3050 msgr2=0x7f714809ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:32.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.707+0000 7f7150214700 1 -- 172.21.15.16:0/1048308913 shutdown_connections 2023-09-30T22:35:32.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.707+0000 7f7150214700 1 -- 172.21.15.16:0/1048308913 wait complete. 2023-09-30T22:35:32.723 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:35:32.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.900+0000 7f67a8dfb700 1 -- 172.21.15.16:0/3016370935 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f67a4071d20 con 0x7f67a409f410 2023-09-30T22:35:32.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.900+0000 7f6792ffd700 1 -- 172.21.15.16:0/3016370935 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f67980199a0 con 0x7f67a409f410 2023-09-30T22:35:32.902 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:35:32.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.903+0000 7f67a8dfb700 1 -- 172.21.15.16:0/3016370935 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f67a409f410 msgr2=0x7f67a406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:32.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.903+0000 7f67a8dfb700 1 --2- 172.21.15.16:0/3016370935 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f67a409f410 0x7f67a406c8f0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f67a406f570 tx=0x7f6798004750 comp rx=0 tx=0).stop 2023-09-30T22:35:32.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.903+0000 7f67a8dfb700 1 -- 172.21.15.16:0/3016370935 shutdown_connections 2023-09-30T22:35:32.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.903+0000 7f67a8dfb700 1 --2- 172.21.15.16:0/3016370935 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f67a4136fe0 0x7f67a41c82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:32.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.903+0000 7f67a8dfb700 1 --2- 172.21.15.16:0/3016370935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f67a409e930 0x7f67a406c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:32.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.903+0000 7f67a8dfb700 1 --2- 172.21.15.16:0/3016370935 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f67a409f410 0x7f67a406c8f0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:32.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.903+0000 7f67a8dfb700 1 -- 172.21.15.16:0/3016370935 >> 172.21.15.16:0/3016370935 conn(0x7f67a40a4210 msgr2=0x7f67a4134990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:32.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.903+0000 7f67a8dfb700 1 -- 172.21.15.16:0/3016370935 shutdown_connections 2023-09-30T22:35:32.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:32.903+0000 7f67a8dfb700 1 -- 172.21.15.16:0/3016370935 wait complete. 2023-09-30T22:35:32.918 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:35:33.200 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:35:33.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.346+0000 7f24d3b64700 1 Processor -- start 2023-09-30T22:35:33.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.346+0000 7f24d3b64700 1 -- start start 2023-09-30T22:35:33.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.346+0000 7f24d3b64700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24cc09d6f0 0x7f24cc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:33.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.346+0000 7f24d3b64700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f24cc134100 0x7f24cc13bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:33.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.346+0000 7f24d3b64700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24cc09d2f0 con 0x7f24cc09e000 2023-09-30T22:35:33.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.346+0000 7f24d3b64700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24cc09a810 con 0x7f24cc134100 2023-09-30T22:35:33.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.346+0000 7f24d3b64700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f24cc09a470 con 0x7f24cc09d6f0 2023-09-30T22:35:33.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.346+0000 7f24d1900700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24cc09d6f0 0x7f24cc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:33.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.346+0000 7f24d10ff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f24cc09e000 0x7f24cc1339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33864/0 (socket says 172.21.15.16:33864) 2023-09-30T22:35:33.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.346+0000 7f24d2101700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f24cc134100 0x7f24cc13bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:33.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.347+0000 7f24d10ff700 1 -- 172.21.15.16:0/927468627 learned_addr learned my addr 172.21.15.16:0/927468627 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:33.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.347+0000 7f24d2101700 1 -- 172.21.15.16:0/927468627 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24cc09d6f0 msgr2=0x7f24cc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:33.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.347+0000 7f24d2101700 1 --2- 172.21.15.16:0/927468627 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24cc09d6f0 0x7f24cc09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:33.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.347+0000 7f24d2101700 1 -- 172.21.15.16:0/927468627 >> v1:172.21.15.16:6789/0 conn(0x7f24cc09e000 legacy=0x7f24cc1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:33.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.347+0000 7f24d2101700 1 -- 172.21.15.16:0/927468627 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24cc09ec40 con 0x7f24cc134100 2023-09-30T22:35:33.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.347+0000 7f24d08fe700 1 -- 172.21.15.16:0/927468627 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1847264680 0 0) 0x7f24cc09d2f0 con 0x7f24cc09e000 2023-09-30T22:35:33.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.347+0000 7f24d1900700 1 --2- 172.21.15.16:0/927468627 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24cc09d6f0 0x7f24cc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:33.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.347+0000 7f24d2101700 1 --2- 172.21.15.16:0/927468627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24cc134100 0x7f24cc13bc50 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f24c4009d90 tx=0x7f24c400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a67e8d24e2d6116 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:33.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.347+0000 7f24d08fe700 1 -- 172.21.15.16:0/927468627 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c400e070 con 0x7f24cc134100 2023-09-30T22:35:33.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.347+0000 7f24d08fe700 1 -- 172.21.15.16:0/927468627 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24c400bb20 con 0x7f24cc134100 2023-09-30T22:35:33.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.347+0000 7f24d08fe700 1 -- 172.21.15.16:0/927468627 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c4012430 con 0x7f24cc134100 2023-09-30T22:35:33.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.348+0000 7f24d3b64700 1 -- 172.21.15.16:0/927468627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24cc134100 msgr2=0x7f24cc13bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:33.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.348+0000 7f24d3b64700 1 --2- 172.21.15.16:0/927468627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24cc134100 0x7f24cc13bc50 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f24c4009d90 tx=0x7f24c400b330 comp rx=0 tx=0).stop 2023-09-30T22:35:33.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.348+0000 7f24d3b64700 1 -- 172.21.15.16:0/927468627 shutdown_connections 2023-09-30T22:35:33.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.348+0000 7f24d3b64700 1 --2- 172.21.15.16:0/927468627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24cc134100 0x7f24cc13bc50 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:33.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.348+0000 7f24d3b64700 1 --2- 172.21.15.16:0/927468627 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24cc09d6f0 0x7f24cc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:33.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.348+0000 7f24d3b64700 1 -- 172.21.15.16:0/927468627 >> 172.21.15.16:0/927468627 conn(0x7f24cc066f50 msgr2=0x7f24cc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:33.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.348+0000 7f24d3b64700 1 -- 172.21.15.16:0/927468627 shutdown_connections 2023-09-30T22:35:33.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.348+0000 7f24d3b64700 1 -- 172.21.15.16:0/927468627 wait complete. 2023-09-30T22:35:33.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.349+0000 7f24d3b64700 1 Processor -- start 2023-09-30T22:35:33.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.349+0000 7f24d3b64700 1 -- start start 2023-09-30T22:35:33.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.349+0000 7f24d3b64700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24cc09d6f0 0x7f24cc1226c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:33.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.349+0000 7f24d3b64700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f24cc09e000 0x7f24cc122c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:33.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.349+0000 7f24d3b64700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24cc134100 0x7f24cc123140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:33.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.349+0000 7f24d3b64700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24cc1259f0 con 0x7f24cc09d6f0 2023-09-30T22:35:33.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.349+0000 7f24d3b64700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24cc133610 con 0x7f24cc09e000 2023-09-30T22:35:33.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.349+0000 7f24d3b64700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f24cc09c640 con 0x7f24cc134100 2023-09-30T22:35:33.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.349+0000 7f24d1900700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24cc09d6f0 0x7f24cc1226c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:33.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.349+0000 7f24d1900700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24cc09d6f0 0x7f24cc1226c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55054/0 (socket says 172.21.15.16:55054) 2023-09-30T22:35:33.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.350+0000 7f24d1900700 1 -- 172.21.15.16:0/2569438764 learned_addr learned my addr 172.21.15.16:0/2569438764 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:33.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.350+0000 7f24d2101700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24cc134100 0x7f24cc123140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:33.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.350+0000 7f24d10ff700 1 --2- 172.21.15.16:0/2569438764 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f24cc09e000 0x7f24cc122c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:33.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.350+0000 7f24d2101700 1 -- 172.21.15.16:0/2569438764 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f24cc09e000 msgr2=0x7f24cc122c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:33.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.350+0000 7f24d2101700 1 --2- 172.21.15.16:0/2569438764 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f24cc09e000 0x7f24cc122c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:33.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.350+0000 7f24d2101700 1 -- 172.21.15.16:0/2569438764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24cc09d6f0 msgr2=0x7f24cc1226c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:33.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.350+0000 7f24d2101700 1 --2- 172.21.15.16:0/2569438764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24cc09d6f0 0x7f24cc1226c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:33.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.350+0000 7f24d2101700 1 -- 172.21.15.16:0/2569438764 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24c80094a0 con 0x7f24cc134100 2023-09-30T22:35:33.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.350+0000 7f24d2101700 1 --2- 172.21.15.16:0/2569438764 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24cc134100 0x7f24cc123140 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f24c4016000 tx=0x7f24c4004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:33.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.350+0000 7f24c27fc700 1 -- 172.21.15.16:0/2569438764 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c4012430 con 0x7f24cc134100 2023-09-30T22:35:33.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.350+0000 7f24c27fc700 1 -- 172.21.15.16:0/2569438764 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24c4020070 con 0x7f24cc134100 2023-09-30T22:35:33.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.350+0000 7f24d3b64700 1 -- 172.21.15.16:0/2569438764 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24c4009a70 con 0x7f24cc134100 2023-09-30T22:35:33.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.350+0000 7f24c27fc700 1 -- 172.21.15.16:0/2569438764 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c4004280 con 0x7f24cc134100 2023-09-30T22:35:33.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.350+0000 7f24d3b64700 1 -- 172.21.15.16:0/2569438764 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24cc133950 con 0x7f24cc134100 2023-09-30T22:35:33.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.351+0000 7f24c27fc700 1 -- 172.21.15.16:0/2569438764 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f24c40043e0 con 0x7f24cc134100 2023-09-30T22:35:33.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.351+0000 7f24c27fc700 1 -- 172.21.15.16:0/2569438764 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f24c4022070 con 0x7f24cc134100 2023-09-30T22:35:33.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.351+0000 7f24d3b64700 1 -- 172.21.15.16:0/2569438764 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24cc074ea0 con 0x7f24cc134100 2023-09-30T22:35:33.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.356+0000 7f24c27fc700 1 -- 172.21.15.16:0/2569438764 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f24c401b400 con 0x7f24cc134100 2023-09-30T22:35:33.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.560+0000 7f24d3b64700 1 -- 172.21.15.16:0/2569438764 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f24cc06c360 con 0x7f24cc134100 2023-09-30T22:35:33.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.560+0000 7f24c27fc700 1 -- 172.21.15.16:0/2569438764 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f24c402c030 con 0x7f24cc134100 2023-09-30T22:35:33.562 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:35:33.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.563+0000 7f24d3b64700 1 -- 172.21.15.16:0/2569438764 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24cc134100 msgr2=0x7f24cc123140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:33.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.563+0000 7f24d3b64700 1 --2- 172.21.15.16:0/2569438764 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24cc134100 0x7f24cc123140 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f24c4016000 tx=0x7f24c4004ad0 comp rx=0 tx=0).stop 2023-09-30T22:35:33.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.563+0000 7f24d3b64700 1 -- 172.21.15.16:0/2569438764 shutdown_connections 2023-09-30T22:35:33.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.563+0000 7f24d3b64700 1 --2- 172.21.15.16:0/2569438764 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f24cc134100 0x7f24cc123140 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:33.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.563+0000 7f24d3b64700 1 --2- 172.21.15.16:0/2569438764 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f24cc09d6f0 0x7f24cc1226c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:33.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.563+0000 7f24d3b64700 1 --2- 172.21.15.16:0/2569438764 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f24cc09e000 0x7f24cc122c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:33.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.563+0000 7f24d3b64700 1 -- 172.21.15.16:0/2569438764 >> 172.21.15.16:0/2569438764 conn(0x7f24cc066f50 msgr2=0x7f24cc0a30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:33.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.563+0000 7f24d3b64700 1 -- 172.21.15.16:0/2569438764 shutdown_connections 2023-09-30T22:35:33.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.563+0000 7f24d3b64700 1 -- 172.21.15.16:0/2569438764 wait complete. 2023-09-30T22:35:33.578 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:35:33.687 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:35:33.724 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:35:33.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.833+0000 7f43eab03700 1 Processor -- start 2023-09-30T22:35:33.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.833+0000 7f43eab03700 1 -- start start 2023-09-30T22:35:33.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.834+0000 7f43eab03700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f43e409e930 0x7f43e409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:33.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.834+0000 7f43eab03700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f43e409f650 0x7f43e4131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:33.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.834+0000 7f43eab03700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43e4070940 con 0x7f43e409f240 2023-09-30T22:35:33.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.834+0000 7f43eab03700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43e406fd60 con 0x7f43e409f650 2023-09-30T22:35:33.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.834+0000 7f43eab03700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43e406fa80 con 0x7f43e409e930 2023-09-30T22:35:33.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.834+0000 7f43e889f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f43e409e930 0x7f43e409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:33.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.834+0000 7f43e90a0700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f43e409f650 0x7f43e4131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:33.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.834+0000 7f43e889f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f43e409e930 0x7f43e409ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:46078/0 (socket says 172.21.15.16:46078) 2023-09-30T22:35:33.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.834+0000 7f43e90a0700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f43e409f650 0x7f43e4131f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55064/0 (socket says 172.21.15.16:55064) 2023-09-30T22:35:33.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.834+0000 7f43e889f700 1 -- 172.21.15.16:0/1127196703 learned_addr learned my addr 172.21.15.16:0/1127196703 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:33.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.834+0000 7f43e90a0700 1 -- 172.21.15.16:0/1127196703 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f43e409e930 msgr2=0x7f43e409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:33.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.834+0000 7f43e90a0700 1 --2- 172.21.15.16:0/1127196703 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f43e409e930 0x7f43e409ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:33.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.834+0000 7f43e90a0700 1 -- 172.21.15.16:0/1127196703 >> v1:172.21.15.16:6789/0 conn(0x7f43e409f240 legacy=0x7f43e41317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:33.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.834+0000 7f43e90a0700 1 -- 172.21.15.16:0/1127196703 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e409fb60 con 0x7f43e409f650 2023-09-30T22:35:33.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.834+0000 7f43e889f700 1 --2- 172.21.15.16:0/1127196703 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f43e409e930 0x7f43e409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:33.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.834+0000 7f43e37fe700 1 -- 172.21.15.16:0/1127196703 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2124435290 0 0) 0x7f43e4070940 con 0x7f43e409f240 2023-09-30T22:35:33.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.835+0000 7f43e90a0700 1 --2- 172.21.15.16:0/1127196703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43e409f650 0x7f43e4131f00 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f43dc009d90 tx=0x7f43dc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54cb2afc79bb0010 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:33.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.835+0000 7f43e37fe700 1 -- 172.21.15.16:0/1127196703 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43dc00e070 con 0x7f43e409f650 2023-09-30T22:35:33.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.835+0000 7f43e37fe700 1 -- 172.21.15.16:0/1127196703 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43dc00bb20 con 0x7f43e409f650 2023-09-30T22:35:33.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.835+0000 7f43e37fe700 1 -- 172.21.15.16:0/1127196703 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43dc012430 con 0x7f43e409f650 2023-09-30T22:35:33.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.836+0000 7f43eab03700 1 -- 172.21.15.16:0/1127196703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43e409f650 msgr2=0x7f43e4131f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:33.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.836+0000 7f43eab03700 1 --2- 172.21.15.16:0/1127196703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43e409f650 0x7f43e4131f00 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f43dc009d90 tx=0x7f43dc00b330 comp rx=0 tx=0).stop 2023-09-30T22:35:33.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.836+0000 7f43eab03700 1 -- 172.21.15.16:0/1127196703 shutdown_connections 2023-09-30T22:35:33.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.836+0000 7f43eab03700 1 --2- 172.21.15.16:0/1127196703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43e409f650 0x7f43e4131f00 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:33.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.836+0000 7f43eab03700 1 --2- 172.21.15.16:0/1127196703 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f43e409e930 0x7f43e409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:33.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.836+0000 7f43eab03700 1 -- 172.21.15.16:0/1127196703 >> 172.21.15.16:0/1127196703 conn(0x7f43e40a4210 msgr2=0x7f43e40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:33.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.836+0000 7f43eab03700 1 -- 172.21.15.16:0/1127196703 shutdown_connections 2023-09-30T22:35:33.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.836+0000 7f43eab03700 1 -- 172.21.15.16:0/1127196703 wait complete. 2023-09-30T22:35:33.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.837+0000 7f43eab03700 1 Processor -- start 2023-09-30T22:35:33.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.837+0000 7f43eab03700 1 -- start start 2023-09-30T22:35:33.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.838+0000 7f43eab03700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43e409e930 0x7f43e41c8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:33.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.838+0000 7f43eab03700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f43e409f240 0x7f43e41c8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:33.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.838+0000 7f43eab03700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f43e409f650 0x7f43e41c8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:33.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.838+0000 7f43eab03700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e41a6cd0 con 0x7f43e409e930 2023-09-30T22:35:33.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.838+0000 7f43eab03700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43e406d6b0 con 0x7f43e409f240 2023-09-30T22:35:33.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.838+0000 7f43eab03700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f43e41a8310 con 0x7f43e409f650 2023-09-30T22:35:33.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.839+0000 7f43e3fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f43e409f240 0x7f43e41c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:33.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.839+0000 7f43e889f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43e409e930 0x7f43e41c8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:33.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.839+0000 7f43e3fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f43e409f240 0x7f43e41c8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:46090/0 (socket says 172.21.15.16:46090) 2023-09-30T22:35:33.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.839+0000 7f43e3fff700 1 -- 172.21.15.16:0/1365446626 learned_addr learned my addr 172.21.15.16:0/1365446626 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:33.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.839+0000 7f43e889f700 1 --2- 172.21.15.16:0/1365446626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43e409e930 0x7f43e41c8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55070/0 (socket says 172.21.15.16:55070) 2023-09-30T22:35:33.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.839+0000 7f43e90a0700 1 --2- 172.21.15.16:0/1365446626 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f43e409f650 0x7f43e41c8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:33.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.839+0000 7f43e889f700 1 -- 172.21.15.16:0/1365446626 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f43e409f650 msgr2=0x7f43e41c8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:33.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.839+0000 7f43e889f700 1 --2- 172.21.15.16:0/1365446626 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f43e409f650 0x7f43e41c8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:33.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.839+0000 7f43e889f700 1 -- 172.21.15.16:0/1365446626 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f43e409f240 msgr2=0x7f43e41c8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:33.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.839+0000 7f43e889f700 1 --2- 172.21.15.16:0/1365446626 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f43e409f240 0x7f43e41c8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:33.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.839+0000 7f43e889f700 1 -- 172.21.15.16:0/1365446626 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d8009550 con 0x7f43e409e930 2023-09-30T22:35:33.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.839+0000 7f43e3fff700 1 --2- 172.21.15.16:0/1365446626 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f43e409f240 0x7f43e41c8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:33.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.839+0000 7f43e889f700 1 --2- 172.21.15.16:0/1365446626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43e409e930 0x7f43e41c8000 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f43d800e9d0 tx=0x7f43d800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:33.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.839+0000 7f43e17fa700 1 -- 172.21.15.16:0/1365446626 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d800ee70 con 0x7f43e409e930 2023-09-30T22:35:33.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.839+0000 7f43e17fa700 1 -- 172.21.15.16:0/1365446626 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43d8004030 con 0x7f43e409e930 2023-09-30T22:35:33.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.839+0000 7f43e17fa700 1 -- 172.21.15.16:0/1365446626 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d80043c0 con 0x7f43e409e930 2023-09-30T22:35:33.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.839+0000 7f43eab03700 1 -- 172.21.15.16:0/1365446626 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43dc009a70 con 0x7f43e409e930 2023-09-30T22:35:33.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.839+0000 7f43eab03700 1 -- 172.21.15.16:0/1365446626 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43e40a0190 con 0x7f43e409e930 2023-09-30T22:35:33.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.840+0000 7f43e17fa700 1 -- 172.21.15.16:0/1365446626 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f43d800c850 con 0x7f43e409e930 2023-09-30T22:35:33.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.841+0000 7f43e17fa700 1 -- 172.21.15.16:0/1365446626 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f43d803f8b0 con 0x7f43e409e930 2023-09-30T22:35:33.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.841+0000 7f43eab03700 1 -- 172.21.15.16:0/1365446626 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43e4071d20 con 0x7f43e409e930 2023-09-30T22:35:33.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.846+0000 7f43e17fa700 1 -- 172.21.15.16:0/1365446626 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f43d8011d90 con 0x7f43e409e930 2023-09-30T22:35:33.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.883+0000 7fa9ced50700 1 Processor -- start 2023-09-30T22:35:33.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.883+0000 7fa9ced50700 1 -- start start 2023-09-30T22:35:33.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.883+0000 7fa9ced50700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa9c80a0a30 0x7fa9c80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:33.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.883+0000 7fa9ced50700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa9c8134170 0x7fa9c813bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:33.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.883+0000 7fa9ced50700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9c809edd0 con 0x7fa9c80a1340 2023-09-30T22:35:33.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.883+0000 7fa9ced50700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9c809e1f0 con 0x7fa9c80a0a30 2023-09-30T22:35:33.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.883+0000 7fa9ced50700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c809dc50 con 0x7fa9c8134170 2023-09-30T22:35:33.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.883+0000 7fa9ccaec700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa9c80a0a30 0x7fa9c80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:33.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.883+0000 7fa9cd2ed700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa9c8134170 0x7fa9c813bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:33.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.883+0000 7fa9ccaec700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa9c80a0a30 0x7fa9c80a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55080/0 (socket says 172.21.15.16:55080) 2023-09-30T22:35:33.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.883+0000 7fa9ccaec700 1 -- 172.21.15.16:0/777719923 learned_addr learned my addr 172.21.15.16:0/777719923 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:33.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.883+0000 7fa9c7fff700 1 --1- 172.21.15.16:0/777719923 >> v1:172.21.15.16:6789/0 conn(0x7fa9c80a1340 0x7fa9c8133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33884/0 (socket says 172.21.15.16:33884) 2023-09-30T22:35:33.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.884+0000 7fa9cd2ed700 1 -- 172.21.15.16:0/777719923 >> v2:172.21.15.16:3300/0 conn(0x7fa9c80a0a30 msgr2=0x7fa9c80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:33.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.884+0000 7fa9cd2ed700 1 --2- 172.21.15.16:0/777719923 >> v2:172.21.15.16:3300/0 conn(0x7fa9c80a0a30 0x7fa9c80a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:33.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.884+0000 7fa9cd2ed700 1 -- 172.21.15.16:0/777719923 >> v1:172.21.15.16:6789/0 conn(0x7fa9c80a1340 legacy=0x7fa9c8133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:33.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.884+0000 7fa9cd2ed700 1 -- 172.21.15.16:0/777719923 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9c80a1fe0 con 0x7fa9c8134170 2023-09-30T22:35:33.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.884+0000 7fa9c77fe700 1 -- 172.21.15.16:0/777719923 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1757749797 0 0) 0x7fa9c809edd0 con 0x7fa9c80a1340 2023-09-30T22:35:33.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.884+0000 7fa9cd2ed700 1 --2- 172.21.15.16:0/777719923 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa9c8134170 0x7fa9c813bd30 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0009d90 tx=0x7fa9c000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b6ed065e2ae3657 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:33.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.884+0000 7fa9c77fe700 1 -- 172.21.15.16:0/777719923 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c000e070 con 0x7fa9c8134170 2023-09-30T22:35:33.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.884+0000 7fa9c77fe700 1 -- 172.21.15.16:0/777719923 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9c000bb20 con 0x7fa9c8134170 2023-09-30T22:35:33.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.884+0000 7fa9c77fe700 1 -- 172.21.15.16:0/777719923 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c0012430 con 0x7fa9c8134170 2023-09-30T22:35:33.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.885+0000 7fa9ced50700 1 -- 172.21.15.16:0/777719923 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa9c8134170 msgr2=0x7fa9c813bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:33.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.885+0000 7fa9ced50700 1 --2- 172.21.15.16:0/777719923 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa9c8134170 0x7fa9c813bd30 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0009d90 tx=0x7fa9c000b330 comp rx=0 tx=0).stop 2023-09-30T22:35:33.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.885+0000 7fa9ced50700 1 -- 172.21.15.16:0/777719923 shutdown_connections 2023-09-30T22:35:33.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.885+0000 7fa9ced50700 1 --2- 172.21.15.16:0/777719923 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa9c8134170 0x7fa9c813bd30 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:33.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.885+0000 7fa9ced50700 1 --2- 172.21.15.16:0/777719923 >> v2:172.21.15.16:3300/0 conn(0x7fa9c80a0a30 0x7fa9c80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:33.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.885+0000 7fa9ced50700 1 -- 172.21.15.16:0/777719923 >> 172.21.15.16:0/777719923 conn(0x7fa9c80a6550 msgr2=0x7fa9c80a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:33.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.885+0000 7fa9ced50700 1 -- 172.21.15.16:0/777719923 shutdown_connections 2023-09-30T22:35:33.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.885+0000 7fa9ced50700 1 -- 172.21.15.16:0/777719923 wait complete. 2023-09-30T22:35:33.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.886+0000 7fa9ced50700 1 Processor -- start 2023-09-30T22:35:33.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.886+0000 7fa9ced50700 1 -- start start 2023-09-30T22:35:33.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.887+0000 7fa9ced50700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa9c80a0a30 0x7fa9c81226e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:33.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.887+0000 7fa9ced50700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9c80a1340 0x7fa9c8122c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:33.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.887+0000 7fa9ced50700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa9c8134170 0x7fa9c8123160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:33.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.887+0000 7fa9ced50700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c8124b80 con 0x7fa9c80a1340 2023-09-30T22:35:33.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.887+0000 7fa9ced50700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c8123b40 con 0x7fa9c80a0a30 2023-09-30T22:35:33.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.887+0000 7fa9ced50700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c8124400 con 0x7fa9c8134170 2023-09-30T22:35:33.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.887+0000 7fa9ccaec700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa9c80a0a30 0x7fa9c81226e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:33.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.887+0000 7fa9c7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9c80a1340 0x7fa9c8122c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:33.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.887+0000 7fa9ccaec700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa9c80a0a30 0x7fa9c81226e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:46092/0 (socket says 172.21.15.16:46092) 2023-09-30T22:35:33.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.887+0000 7fa9ccaec700 1 -- 172.21.15.16:0/467904094 learned_addr learned my addr 172.21.15.16:0/467904094 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:33.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.887+0000 7fa9c7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9c80a1340 0x7fa9c8122c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55082/0 (socket says 172.21.15.16:55082) 2023-09-30T22:35:33.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.887+0000 7fa9cd2ed700 1 --2- 172.21.15.16:0/467904094 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa9c8134170 0x7fa9c8123160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:33.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.887+0000 7fa9ccaec700 1 -- 172.21.15.16:0/467904094 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa9c8134170 msgr2=0x7fa9c8123160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:33.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.887+0000 7fa9ccaec700 1 --2- 172.21.15.16:0/467904094 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa9c8134170 0x7fa9c8123160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:33.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.887+0000 7fa9ccaec700 1 -- 172.21.15.16:0/467904094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9c80a1340 msgr2=0x7fa9c8122c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:33.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.887+0000 7fa9ccaec700 1 --2- 172.21.15.16:0/467904094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9c80a1340 0x7fa9c8122c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:33.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.887+0000 7fa9ccaec700 1 -- 172.21.15.16:0/467904094 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9c0009a70 con 0x7fa9c80a0a30 2023-09-30T22:35:33.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.887+0000 7fa9c7fff700 1 --2- 172.21.15.16:0/467904094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9c80a1340 0x7fa9c8122c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:33.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.887+0000 7fa9ccaec700 1 --2- 172.21.15.16:0/467904094 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa9c80a0a30 0x7fa9c81226e0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fa9b800e910 tx=0x7fa9b800ec20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:33.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.887+0000 7fa9c57fa700 1 -- 172.21.15.16:0/467904094 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b80096a0 con 0x7fa9c80a0a30 2023-09-30T22:35:33.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.887+0000 7fa9c57fa700 1 -- 172.21.15.16:0/467904094 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9b8013680 con 0x7fa9c80a0a30 2023-09-30T22:35:33.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.888+0000 7fa9ced50700 1 -- 172.21.15.16:0/467904094 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9c80a25e0 con 0x7fa9c80a0a30 2023-09-30T22:35:33.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.888+0000 7fa9c57fa700 1 -- 172.21.15.16:0/467904094 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b8013a30 con 0x7fa9c80a0a30 2023-09-30T22:35:33.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.888+0000 7fa9ced50700 1 -- 172.21.15.16:0/467904094 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9c8133200 con 0x7fa9c80a0a30 2023-09-30T22:35:33.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.888+0000 7fa9c57fa700 1 -- 172.21.15.16:0/467904094 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa9b8013b90 con 0x7fa9c80a0a30 2023-09-30T22:35:33.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.888+0000 7fa9c57fa700 1 -- 172.21.15.16:0/467904094 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fa9b803fa30 con 0x7fa9c80a0a30 2023-09-30T22:35:33.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.889+0000 7fa9ced50700 1 -- 172.21.15.16:0/467904094 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9c8074ea0 con 0x7fa9c80a0a30 2023-09-30T22:35:33.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:33.894+0000 7fa9c57fa700 1 -- 172.21.15.16:0/467904094 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa9b8021080 con 0x7fa9c80a0a30 2023-09-30T22:35:33.918 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:35:34.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.065+0000 7ff45ad82700 1 Processor -- start 2023-09-30T22:35:34.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.065+0000 7ff45ad82700 1 -- start start 2023-09-30T22:35:34.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.065+0000 7ff45ad82700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff4540a0a30 0x7ff4540a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:34.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.065+0000 7ff45ad82700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff45409d630 0x7ff4540a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:34.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.065+0000 7ff45ad82700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4540a2a70 con 0x7ff454133930 2023-09-30T22:35:34.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.065+0000 7ff45ad82700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4540a01d0 con 0x7ff4540a0a30 2023-09-30T22:35:34.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.065+0000 7ff45ad82700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff45409e2b0 con 0x7ff45409d630 2023-09-30T22:35:34.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.066+0000 7ff453fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff45409d630 0x7ff4540a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:34.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.066+0000 7ff453fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff45409d630 0x7ff4540a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:46100/0 (socket says 172.21.15.16:46100) 2023-09-30T22:35:34.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.066+0000 7ff453fff700 1 -- 172.21.15.16:0/1540687676 learned_addr learned my addr 172.21.15.16:0/1540687676 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:34.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.066+0000 7ff458b1e700 1 --2- 172.21.15.16:0/1540687676 >> v2:172.21.15.16:3300/0 conn(0x7ff4540a0a30 0x7ff4540a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:34.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.066+0000 7ff458b1e700 1 -- 172.21.15.16:0/1540687676 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff45409d630 msgr2=0x7ff4540a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:34.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.066+0000 7ff458b1e700 1 --2- 172.21.15.16:0/1540687676 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff45409d630 0x7ff4540a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:34.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.066+0000 7ff458b1e700 1 -- 172.21.15.16:0/1540687676 >> v1:172.21.15.16:6789/0 conn(0x7ff454133930 legacy=0x7ff454133d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:34.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.066+0000 7ff458b1e700 1 -- 172.21.15.16:0/1540687676 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff45409dc50 con 0x7ff4540a0a30 2023-09-30T22:35:34.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.066+0000 7ff453fff700 1 --2- 172.21.15.16:0/1540687676 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff45409d630 0x7ff4540a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:34.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.066+0000 7ff458b1e700 1 --2- 172.21.15.16:0/1540687676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4540a0a30 0x7ff4540a0e00 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7ff448009840 tx=0x7ff448009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f35a61359805a94 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:34.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.066+0000 7ff4537fe700 1 -- 172.21.15.16:0/1540687676 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff448011830 con 0x7ff4540a0a30 2023-09-30T22:35:34.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.066+0000 7ff4537fe700 1 -- 172.21.15.16:0/1540687676 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff448011990 con 0x7ff4540a0a30 2023-09-30T22:35:34.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.066+0000 7ff4537fe700 1 -- 172.21.15.16:0/1540687676 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff448011d20 con 0x7ff4540a0a30 2023-09-30T22:35:34.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.067+0000 7ff45ad82700 1 -- 172.21.15.16:0/1540687676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4540a0a30 msgr2=0x7ff4540a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:34.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.067+0000 7ff45ad82700 1 --2- 172.21.15.16:0/1540687676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4540a0a30 0x7ff4540a0e00 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7ff448009840 tx=0x7ff448009b50 comp rx=0 tx=0).stop 2023-09-30T22:35:34.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.067+0000 7ff45ad82700 1 -- 172.21.15.16:0/1540687676 shutdown_connections 2023-09-30T22:35:34.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.067+0000 7ff45ad82700 1 --2- 172.21.15.16:0/1540687676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff4540a0a30 0x7ff4540a0e00 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:34.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.067+0000 7ff45ad82700 1 --2- 172.21.15.16:0/1540687676 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff45409d630 0x7ff4540a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:34.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.067+0000 7ff45ad82700 1 -- 172.21.15.16:0/1540687676 >> 172.21.15.16:0/1540687676 conn(0x7ff4540a3050 msgr2=0x7ff4540a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:34.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.067+0000 7ff45ad82700 1 -- 172.21.15.16:0/1540687676 shutdown_connections 2023-09-30T22:35:34.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.067+0000 7ff45ad82700 1 -- 172.21.15.16:0/1540687676 wait complete. 2023-09-30T22:35:34.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.068+0000 7ff45ad82700 1 Processor -- start 2023-09-30T22:35:34.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.068+0000 7ff45ad82700 1 -- start start 2023-09-30T22:35:34.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.069+0000 7ff45ad82700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff45409d630 0x7ff4541c8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:34.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.069+0000 7ff45ad82700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff4540a0a30 0x7ff4541c8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:34.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.069+0000 7ff45ad82700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff454133930 0x7ff4541c8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:34.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.069+0000 7ff45ad82700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff45411eea0 con 0x7ff45409d630 2023-09-30T22:35:34.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.069+0000 7ff45ad82700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4541207f0 con 0x7ff4540a0a30 2023-09-30T22:35:34.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.069+0000 7ff45ad82700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff454120600 con 0x7ff454133930 2023-09-30T22:35:34.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.069+0000 7ff453fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff4540a0a30 0x7ff4541c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:34.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.069+0000 7ff458b1e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff45409d630 0x7ff4541c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:34.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.069+0000 7ff453fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff4540a0a30 0x7ff4541c8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:46108/0 (socket says 172.21.15.16:46108) 2023-09-30T22:35:34.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.069+0000 7ff453fff700 1 -- 172.21.15.16:0/675783731 learned_addr learned my addr 172.21.15.16:0/675783731 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:34.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.069+0000 7ff45931f700 1 --2- 172.21.15.16:0/675783731 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff454133930 0x7ff4541c8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:34.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.069+0000 7ff453fff700 1 -- 172.21.15.16:0/675783731 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff454133930 msgr2=0x7ff4541c8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:34.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.069+0000 7ff453fff700 1 --2- 172.21.15.16:0/675783731 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff454133930 0x7ff4541c8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:34.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.069+0000 7ff453fff700 1 -- 172.21.15.16:0/675783731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff45409d630 msgr2=0x7ff4541c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:34.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.069+0000 7ff453fff700 1 --2- 172.21.15.16:0/675783731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff45409d630 0x7ff4541c8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:34.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.070+0000 7ff453fff700 1 -- 172.21.15.16:0/675783731 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff440009550 con 0x7ff4540a0a30 2023-09-30T22:35:34.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.070+0000 7ff458b1e700 1 --2- 172.21.15.16:0/675783731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff45409d630 0x7ff4541c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:34.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.070+0000 7ff453fff700 1 --2- 172.21.15.16:0/675783731 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff4540a0a30 0x7ff4541c8660 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7ff44000ebe0 tx=0x7ff44000eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:34.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.070+0000 7ff4517fa700 1 -- 172.21.15.16:0/675783731 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44000c850 con 0x7ff4540a0a30 2023-09-30T22:35:34.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.070+0000 7ff4517fa700 1 -- 172.21.15.16:0/675783731 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff44000c9b0 con 0x7ff4540a0a30 2023-09-30T22:35:34.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.070+0000 7ff45ad82700 1 -- 172.21.15.16:0/675783731 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4480094a0 con 0x7ff4540a0a30 2023-09-30T22:35:34.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.070+0000 7ff4517fa700 1 -- 172.21.15.16:0/675783731 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff44000cd40 con 0x7ff4540a0a30 2023-09-30T22:35:34.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.070+0000 7ff45ad82700 1 -- 172.21.15.16:0/675783731 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff454122230 con 0x7ff4540a0a30 2023-09-30T22:35:34.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.071+0000 7ff4517fa700 1 -- 172.21.15.16:0/675783731 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff440013470 con 0x7ff4540a0a30 2023-09-30T22:35:34.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.071+0000 7ff45ad82700 1 -- 172.21.15.16:0/675783731 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff454122420 con 0x7ff4540a0a30 2023-09-30T22:35:34.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.071+0000 7ff4517fa700 1 -- 172.21.15.16:0/675783731 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7ff44003f990 con 0x7ff4540a0a30 2023-09-30T22:35:34.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.074+0000 7f43eab03700 1 -- 172.21.15.16:0/1365446626 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f43e4074ea0 con 0x7f43e409e930 2023-09-30T22:35:34.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.074+0000 7f43e17fa700 1 -- 172.21.15.16:0/1365446626 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f43d802a090 con 0x7f43e409e930 2023-09-30T22:35:34.083 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:35:34.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.076+0000 7f43eab03700 1 -- 172.21.15.16:0/1365446626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43e409e930 msgr2=0x7f43e41c8000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:34.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.076+0000 7f43eab03700 1 --2- 172.21.15.16:0/1365446626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43e409e930 0x7f43e41c8000 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f43d800e9d0 tx=0x7f43d800ece0 comp rx=0 tx=0).stop 2023-09-30T22:35:34.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.076+0000 7f43eab03700 1 -- 172.21.15.16:0/1365446626 shutdown_connections 2023-09-30T22:35:34.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.076+0000 7f43eab03700 1 --2- 172.21.15.16:0/1365446626 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f43e409f650 0x7f43e41c8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:34.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.076+0000 7f43eab03700 1 --2- 172.21.15.16:0/1365446626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43e409e930 0x7f43e41c8000 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:34.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.076+0000 7f43eab03700 1 --2- 172.21.15.16:0/1365446626 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f43e409f240 0x7f43e41c8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:34.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.076+0000 7f43eab03700 1 -- 172.21.15.16:0/1365446626 >> 172.21.15.16:0/1365446626 conn(0x7f43e40a4210 msgr2=0x7f43e40a2930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:34.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.076+0000 7f43eab03700 1 -- 172.21.15.16:0/1365446626 shutdown_connections 2023-09-30T22:35:34.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.076+0000 7f43eab03700 1 -- 172.21.15.16:0/1365446626 wait complete. 2023-09-30T22:35:34.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.081+0000 7ff4517fa700 1 -- 172.21.15.16:0/675783731 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff440021020 con 0x7ff4540a0a30 2023-09-30T22:35:34.092 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:35:34.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.117+0000 7fa9ced50700 1 -- 172.21.15.16:0/467904094 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa9c80a28e0 con 0x7fa9c80a0a30 2023-09-30T22:35:34.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.117+0000 7fa9c57fa700 1 -- 172.21.15.16:0/467904094 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa9b8011d90 con 0x7fa9c80a0a30 2023-09-30T22:35:34.119 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:35:34.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.120+0000 7fa9ced50700 1 -- 172.21.15.16:0/467904094 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa9c80a0a30 msgr2=0x7fa9c81226e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:34.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.120+0000 7fa9ced50700 1 --2- 172.21.15.16:0/467904094 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa9c80a0a30 0x7fa9c81226e0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fa9b800e910 tx=0x7fa9b800ec20 comp rx=0 tx=0).stop 2023-09-30T22:35:34.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.120+0000 7fa9ced50700 1 -- 172.21.15.16:0/467904094 shutdown_connections 2023-09-30T22:35:34.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.120+0000 7fa9ced50700 1 --2- 172.21.15.16:0/467904094 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa9c80a0a30 0x7fa9c81226e0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:34.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.120+0000 7fa9ced50700 1 --2- 172.21.15.16:0/467904094 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa9c8134170 0x7fa9c8123160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:34.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.120+0000 7fa9ced50700 1 --2- 172.21.15.16:0/467904094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9c80a1340 0x7fa9c8122c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:34.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.120+0000 7fa9ced50700 1 -- 172.21.15.16:0/467904094 >> 172.21.15.16:0/467904094 conn(0x7fa9c80a6550 msgr2=0x7fa9c809edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:34.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.120+0000 7fa9ced50700 1 -- 172.21.15.16:0/467904094 shutdown_connections 2023-09-30T22:35:34.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.120+0000 7fa9ced50700 1 -- 172.21.15.16:0/467904094 wait complete. 2023-09-30T22:35:34.136 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:35:34.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.289+0000 7ff45ad82700 1 -- 172.21.15.16:0/675783731 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff4540747f0 con 0x7ff4540a0a30 2023-09-30T22:35:34.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.289+0000 7ff4517fa700 1 -- 172.21.15.16:0/675783731 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff440017070 con 0x7ff4540a0a30 2023-09-30T22:35:34.291 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:35:34.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.291+0000 7ff45ad82700 1 -- 172.21.15.16:0/675783731 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff4540a0a30 msgr2=0x7ff4541c8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:34.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.291+0000 7ff45ad82700 1 --2- 172.21.15.16:0/675783731 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff4540a0a30 0x7ff4541c8660 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7ff44000ebe0 tx=0x7ff44000eef0 comp rx=0 tx=0).stop 2023-09-30T22:35:34.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.291+0000 7ff45ad82700 1 -- 172.21.15.16:0/675783731 shutdown_connections 2023-09-30T22:35:34.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.291+0000 7ff45ad82700 1 --2- 172.21.15.16:0/675783731 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff454133930 0x7ff4541c8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:34.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.291+0000 7ff45ad82700 1 --2- 172.21.15.16:0/675783731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff45409d630 0x7ff4541c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:34.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.291+0000 7ff45ad82700 1 --2- 172.21.15.16:0/675783731 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff4540a0a30 0x7ff4541c8660 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:34.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.291+0000 7ff45ad82700 1 -- 172.21.15.16:0/675783731 >> 172.21.15.16:0/675783731 conn(0x7ff4540a3050 msgr2=0x7ff45409ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:34.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.291+0000 7ff45ad82700 1 -- 172.21.15.16:0/675783731 shutdown_connections 2023-09-30T22:35:34.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.291+0000 7ff45ad82700 1 -- 172.21.15.16:0/675783731 wait complete. 2023-09-30T22:35:34.306 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:35:34.579 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:35:34.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.724+0000 7f3219602700 1 Processor -- start 2023-09-30T22:35:34.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.724+0000 7f3219602700 1 -- start start 2023-09-30T22:35:34.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.725+0000 7f3219602700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3214133960 0x7f3214133d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:34.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.725+0000 7f3219602700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f321409e990 0x7f321413b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:34.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.725+0000 7f3219602700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3214133550 con 0x7f321409de50 2023-09-30T22:35:34.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.725+0000 7f3219602700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f321406fab0 con 0x7f321409e990 2023-09-30T22:35:34.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.725+0000 7f3219602700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3214071e20 con 0x7f3214133960 2023-09-30T22:35:34.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.725+0000 7f3212ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3214133960 0x7f3214133d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:34.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.725+0000 7f32137fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f321409e990 0x7f321413b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:34.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.725+0000 7f3212ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3214133960 0x7f3214133d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:45420/0 (socket says 172.21.15.16:45420) 2023-09-30T22:35:34.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.725+0000 7f32137fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f321409e990 0x7f321413b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55114/0 (socket says 172.21.15.16:55114) 2023-09-30T22:35:34.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.725+0000 7f32127fc700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f321409de50 0x7f321409e280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33896/0 (socket says 172.21.15.16:33896) 2023-09-30T22:35:34.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.725+0000 7f3212ffd700 1 -- 172.21.15.16:0/2226585511 learned_addr learned my addr 172.21.15.16:0/2226585511 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:34.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.725+0000 7f3212ffd700 1 -- 172.21.15.16:0/2226585511 >> v2:172.21.15.16:3300/0 conn(0x7f321409e990 msgr2=0x7f321413b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:34.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.725+0000 7f3212ffd700 1 --2- 172.21.15.16:0/2226585511 >> v2:172.21.15.16:3300/0 conn(0x7f321409e990 0x7f321413b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:34.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.726+0000 7f3212ffd700 1 -- 172.21.15.16:0/2226585511 >> v1:172.21.15.16:6789/0 conn(0x7f321409de50 legacy=0x7f321409e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:34.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.726+0000 7f3212ffd700 1 -- 172.21.15.16:0/2226585511 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f321406f1e0 con 0x7f3214133960 2023-09-30T22:35:34.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.726+0000 7f3211ffb700 1 -- 172.21.15.16:0/2226585511 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3461214364 0 0) 0x7f3214133550 con 0x7f321409de50 2023-09-30T22:35:34.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.726+0000 7f32137fe700 1 --2- 172.21.15.16:0/2226585511 >> v2:172.21.15.16:3300/0 conn(0x7f321409e990 0x7f321413b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:34.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.726+0000 7f3212ffd700 1 --2- 172.21.15.16:0/2226585511 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3214133960 0x7f3214133d30 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f32040098f0 tx=0x7f3204009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13429b09a69702a7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:34.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.726+0000 7f3211ffb700 1 -- 172.21.15.16:0/2226585511 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3204011830 con 0x7f3214133960 2023-09-30T22:35:34.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.726+0000 7f3211ffb700 1 -- 172.21.15.16:0/2226585511 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3204011990 con 0x7f3214133960 2023-09-30T22:35:34.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.726+0000 7f3211ffb700 1 -- 172.21.15.16:0/2226585511 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3204011d20 con 0x7f3214133960 2023-09-30T22:35:34.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.726+0000 7f3219602700 1 -- 172.21.15.16:0/2226585511 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3214133960 msgr2=0x7f3214133d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:34.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.727+0000 7f3219602700 1 --2- 172.21.15.16:0/2226585511 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3214133960 0x7f3214133d30 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f32040098f0 tx=0x7f3204009c00 comp rx=0 tx=0).stop 2023-09-30T22:35:34.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.727+0000 7f3219602700 1 -- 172.21.15.16:0/2226585511 shutdown_connections 2023-09-30T22:35:34.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.727+0000 7f3219602700 1 --2- 172.21.15.16:0/2226585511 >> v2:172.21.15.16:3300/0 conn(0x7f321409e990 0x7f321413b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:34.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.727+0000 7f3219602700 1 --2- 172.21.15.16:0/2226585511 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3214133960 0x7f3214133d30 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:34.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.727+0000 7f3219602700 1 -- 172.21.15.16:0/2226585511 >> 172.21.15.16:0/2226585511 conn(0x7f32140a2610 msgr2=0x7f32140a2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:34.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.727+0000 7f3219602700 1 -- 172.21.15.16:0/2226585511 shutdown_connections 2023-09-30T22:35:34.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.727+0000 7f3219602700 1 -- 172.21.15.16:0/2226585511 wait complete. 2023-09-30T22:35:34.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.728+0000 7f3219602700 1 Processor -- start 2023-09-30T22:35:34.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.728+0000 7f3219602700 1 -- start start 2023-09-30T22:35:34.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.728+0000 7f3219602700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f321409de50 0x7f321411c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:34.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.728+0000 7f3219602700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f321409e990 0x7f321411cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:34.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.728+0000 7f3219602700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3214133960 0x7f321411d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:34.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.728+0000 7f3219602700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3214120220 con 0x7f321409de50 2023-09-30T22:35:34.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.728+0000 7f3219602700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f321411e930 con 0x7f3214133960 2023-09-30T22:35:34.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.728+0000 7f3219602700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f321411f5f0 con 0x7f321409e990 2023-09-30T22:35:34.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.728+0000 7f3212ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f321409de50 0x7f321411c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:34.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.728+0000 7f3212ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f321409de50 0x7f321411c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55122/0 (socket says 172.21.15.16:55122) 2023-09-30T22:35:34.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.728+0000 7f3212ffd700 1 -- 172.21.15.16:0/2410097615 learned_addr learned my addr 172.21.15.16:0/2410097615 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:34.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.728+0000 7f32127fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f321409e990 0x7f321411cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:34.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.728+0000 7f32137fe700 1 --2- 172.21.15.16:0/2410097615 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3214133960 0x7f321411d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:34.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.729+0000 7f3212ffd700 1 -- 172.21.15.16:0/2410097615 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f321409e990 msgr2=0x7f321411cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:34.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.729+0000 7f3212ffd700 1 --2- 172.21.15.16:0/2410097615 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f321409e990 0x7f321411cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:34.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.729+0000 7f3212ffd700 1 -- 172.21.15.16:0/2410097615 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3214133960 msgr2=0x7f321411d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:34.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.729+0000 7f3212ffd700 1 --2- 172.21.15.16:0/2410097615 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3214133960 0x7f321411d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:34.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.729+0000 7f3212ffd700 1 -- 172.21.15.16:0/2410097615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f320c009a70 con 0x7f321409de50 2023-09-30T22:35:34.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.729+0000 7f3212ffd700 1 --2- 172.21.15.16:0/2410097615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f321409de50 0x7f321411c750 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f3214071f60 tx=0x7f3204004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:34.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.729+0000 7f31fb7fe700 1 -- 172.21.15.16:0/2410097615 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3204004510 con 0x7f321409de50 2023-09-30T22:35:34.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.729+0000 7f31fb7fe700 1 -- 172.21.15.16:0/2410097615 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32040036a0 con 0x7f321409de50 2023-09-30T22:35:34.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.729+0000 7f31fb7fe700 1 -- 172.21.15.16:0/2410097615 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3204003a10 con 0x7f321409de50 2023-09-30T22:35:34.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.729+0000 7f3219602700 1 -- 172.21.15.16:0/2410097615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3204009550 con 0x7f321409de50 2023-09-30T22:35:34.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.729+0000 7f3219602700 1 -- 172.21.15.16:0/2410097615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f321411fa10 con 0x7f321409de50 2023-09-30T22:35:34.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.730+0000 7f31fb7fe700 1 -- 172.21.15.16:0/2410097615 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3204003b70 con 0x7f321409de50 2023-09-30T22:35:34.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.730+0000 7f31fb7fe700 1 -- 172.21.15.16:0/2410097615 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f3204040c40 con 0x7f321409de50 2023-09-30T22:35:34.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.730+0000 7f3219602700 1 -- 172.21.15.16:0/2410097615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f321411fda0 con 0x7f321409de50 2023-09-30T22:35:34.736 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.735+0000 7f31fb7fe700 1 -- 172.21.15.16:0/2410097615 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f321411fda0 con 0x7f321409de50 2023-09-30T22:35:34.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.940+0000 7f3219602700 1 -- 172.21.15.16:0/2410097615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f321411fda0 con 0x7f321409de50 2023-09-30T22:35:34.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.940+0000 7f31fb7fe700 1 -- 172.21.15.16:0/2410097615 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f321411fda0 con 0x7f321409de50 2023-09-30T22:35:34.942 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:35:34.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.942+0000 7f3219602700 1 -- 172.21.15.16:0/2410097615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f321409de50 msgr2=0x7f321411c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:34.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.942+0000 7f3219602700 1 --2- 172.21.15.16:0/2410097615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f321409de50 0x7f321411c750 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f3214071f60 tx=0x7f3204004750 comp rx=0 tx=0).stop 2023-09-30T22:35:34.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.942+0000 7f3219602700 1 -- 172.21.15.16:0/2410097615 shutdown_connections 2023-09-30T22:35:34.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.942+0000 7f3219602700 1 --2- 172.21.15.16:0/2410097615 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3214133960 0x7f321411d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:34.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.942+0000 7f3219602700 1 --2- 172.21.15.16:0/2410097615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f321409de50 0x7f321411c750 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:34.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.942+0000 7f3219602700 1 --2- 172.21.15.16:0/2410097615 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f321409e990 0x7f321411cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:34.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.942+0000 7f3219602700 1 -- 172.21.15.16:0/2410097615 >> 172.21.15.16:0/2410097615 conn(0x7f32140a2610 msgr2=0x7f32141bada0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:34.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.942+0000 7f3219602700 1 -- 172.21.15.16:0/2410097615 shutdown_connections 2023-09-30T22:35:34.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:34.942+0000 7f3219602700 1 -- 172.21.15.16:0/2410097615 wait complete. 2023-09-30T22:35:34.957 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:35:35.093 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:35:35.137 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:35:35.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.237+0000 7f9a02968700 1 Processor -- start 2023-09-30T22:35:35.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.237+0000 7f9a02968700 1 -- start start 2023-09-30T22:35:35.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.238+0000 7f9a02968700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f99fc09e930 0x7f99fc09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:35.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.238+0000 7f9a02968700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f99fc09f650 0x7f99fc131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:35.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.238+0000 7f9a02968700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99fc070940 con 0x7f99fc09f240 2023-09-30T22:35:35.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.238+0000 7f9a02968700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99fc06fd60 con 0x7f99fc09f650 2023-09-30T22:35:35.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.238+0000 7f9a02968700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99fc06fa80 con 0x7f99fc09e930 2023-09-30T22:35:35.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.238+0000 7f9a00f05700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f99fc09f650 0x7f99fc131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:35.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.238+0000 7f99fbfff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f99fc09e930 0x7f99fc09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:35.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.238+0000 7f99fb7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f99fc09f240 0x7f99fc1317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33902/0 (socket says 172.21.15.16:33902) 2023-09-30T22:35:35.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.238+0000 7f9a00f05700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f99fc09f650 0x7f99fc131f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55126/0 (socket says 172.21.15.16:55126) 2023-09-30T22:35:35.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.238+0000 7f99fbfff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f99fc09e930 0x7f99fc09ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:46112/0 (socket says 172.21.15.16:46112) 2023-09-30T22:35:35.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.238+0000 7f9a00f05700 1 -- 172.21.15.16:0/120958821 learned_addr learned my addr 172.21.15.16:0/120958821 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:35.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.238+0000 7f99faffd700 1 -- 172.21.15.16:0/120958821 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3017799736 0 0) 0x7f99fc070940 con 0x7f99fc09f240 2023-09-30T22:35:35.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.238+0000 7f99fbfff700 1 -- 172.21.15.16:0/120958821 >> v2:172.21.15.16:3300/0 conn(0x7f99fc09f650 msgr2=0x7f99fc131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:35.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.239+0000 7f99fbfff700 1 --2- 172.21.15.16:0/120958821 >> v2:172.21.15.16:3300/0 conn(0x7f99fc09f650 0x7f99fc131f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:35.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.239+0000 7f99fbfff700 1 -- 172.21.15.16:0/120958821 >> v1:172.21.15.16:6789/0 conn(0x7f99fc09f240 legacy=0x7f99fc1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:35.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.239+0000 7f99fbfff700 1 -- 172.21.15.16:0/120958821 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99fc09fb60 con 0x7f99fc09e930 2023-09-30T22:35:35.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.239+0000 7f9a00f05700 1 --2- 172.21.15.16:0/120958821 >> v2:172.21.15.16:3300/0 conn(0x7f99fc09f650 0x7f99fc131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:35.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.239+0000 7f99fbfff700 1 --2- 172.21.15.16:0/120958821 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f99fc09e930 0x7f99fc09ed00 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f99ec009840 tx=0x7f99ec009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc0bb2bd97dc5720 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:35.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.239+0000 7f99faffd700 1 -- 172.21.15.16:0/120958821 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ec011830 con 0x7f99fc09e930 2023-09-30T22:35:35.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.239+0000 7f99faffd700 1 -- 172.21.15.16:0/120958821 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99ec011990 con 0x7f99fc09e930 2023-09-30T22:35:35.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.239+0000 7f99faffd700 1 -- 172.21.15.16:0/120958821 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ec011d20 con 0x7f99fc09e930 2023-09-30T22:35:35.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.240+0000 7f9a02968700 1 -- 172.21.15.16:0/120958821 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f99fc09e930 msgr2=0x7f99fc09ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:35.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.240+0000 7f9a02968700 1 --2- 172.21.15.16:0/120958821 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f99fc09e930 0x7f99fc09ed00 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f99ec009840 tx=0x7f99ec009b50 comp rx=0 tx=0).stop 2023-09-30T22:35:35.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.240+0000 7f9a02968700 1 -- 172.21.15.16:0/120958821 shutdown_connections 2023-09-30T22:35:35.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.240+0000 7f9a02968700 1 --2- 172.21.15.16:0/120958821 >> v2:172.21.15.16:3300/0 conn(0x7f99fc09f650 0x7f99fc131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:35.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.240+0000 7f9a02968700 1 --2- 172.21.15.16:0/120958821 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f99fc09e930 0x7f99fc09ed00 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:35.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.240+0000 7f9a02968700 1 -- 172.21.15.16:0/120958821 >> 172.21.15.16:0/120958821 conn(0x7f99fc0a4210 msgr2=0x7f99fc0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:35.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.240+0000 7f9a02968700 1 -- 172.21.15.16:0/120958821 shutdown_connections 2023-09-30T22:35:35.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.240+0000 7f9a02968700 1 -- 172.21.15.16:0/120958821 wait complete. 2023-09-30T22:35:35.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.241+0000 7f9a02968700 1 Processor -- start 2023-09-30T22:35:35.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.241+0000 7f9a02968700 1 -- start start 2023-09-30T22:35:35.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.241+0000 7f9a02968700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f99fc09e930 0x7f99fc06c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:35.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.241+0000 7f9a02968700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99fc09f240 0x7f99fc06c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:35.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.241+0000 7f9a02968700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f99fc09f650 0x7f99fc1c8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:35.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.241+0000 7f9a02968700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99fc06bd70 con 0x7f99fc09f240 2023-09-30T22:35:35.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.241+0000 7f9a02968700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99fc0a3220 con 0x7f99fc09f650 2023-09-30T22:35:35.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.241+0000 7f9a02968700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f99fc1a8080 con 0x7f99fc09e930 2023-09-30T22:35:35.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.242+0000 7f99fb7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99fc09f240 0x7f99fc06c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:35.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.242+0000 7f9a00f05700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f99fc09f650 0x7f99fc1c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:35.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.242+0000 7f9a00f05700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f99fc09f650 0x7f99fc1c8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:46128/0 (socket says 172.21.15.16:46128) 2023-09-30T22:35:35.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.242+0000 7f99fb7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99fc09f240 0x7f99fc06c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55136/0 (socket says 172.21.15.16:55136) 2023-09-30T22:35:35.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.242+0000 7f9a00f05700 1 -- 172.21.15.16:0/1341745855 learned_addr learned my addr 172.21.15.16:0/1341745855 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:35.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.242+0000 7f99fbfff700 1 --2- 172.21.15.16:0/1341745855 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f99fc09e930 0x7f99fc06c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:35.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.242+0000 7f9a00f05700 1 -- 172.21.15.16:0/1341745855 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f99fc09e930 msgr2=0x7f99fc06c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:35.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.242+0000 7f9a00f05700 1 --2- 172.21.15.16:0/1341745855 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f99fc09e930 0x7f99fc06c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:35.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.242+0000 7f9a00f05700 1 -- 172.21.15.16:0/1341745855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99fc09f240 msgr2=0x7f99fc06c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:35.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.242+0000 7f9a00f05700 1 --2- 172.21.15.16:0/1341745855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99fc09f240 0x7f99fc06c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:35.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.242+0000 7f9a00f05700 1 -- 172.21.15.16:0/1341745855 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f4009b30 con 0x7f99fc09f650 2023-09-30T22:35:35.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.242+0000 7f99fb7fe700 1 --2- 172.21.15.16:0/1341745855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99fc09f240 0x7f99fc06c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:35.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.242+0000 7f9a00f05700 1 --2- 172.21.15.16:0/1341745855 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f99fc09f650 0x7f99fc1c8150 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f99f400efc0 tx=0x7f99f400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:35.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.242+0000 7f99f8ff9700 1 -- 172.21.15.16:0/1341745855 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f4018070 con 0x7f99fc09f650 2023-09-30T22:35:35.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.242+0000 7f99f8ff9700 1 -- 172.21.15.16:0/1341745855 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99f400f040 con 0x7f99fc09f650 2023-09-30T22:35:35.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.242+0000 7f99f8ff9700 1 -- 172.21.15.16:0/1341745855 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f4003c20 con 0x7f99fc09f650 2023-09-30T22:35:35.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.242+0000 7f9a02968700 1 -- 172.21.15.16:0/1341745855 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99ec0094a0 con 0x7f99fc09f650 2023-09-30T22:35:35.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.243+0000 7f9a02968700 1 -- 172.21.15.16:0/1341745855 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99fc0a0190 con 0x7f99fc09f650 2023-09-30T22:35:35.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.243+0000 7f99f8ff9700 1 -- 172.21.15.16:0/1341745855 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f99f400c880 con 0x7f99fc09f650 2023-09-30T22:35:35.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.244+0000 7f99f8ff9700 1 -- 172.21.15.16:0/1341745855 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f99f403f810 con 0x7f99fc09f650 2023-09-30T22:35:35.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.244+0000 7f9a02968700 1 -- 172.21.15.16:0/1341745855 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99fc1a7d00 con 0x7f99fc09f650 2023-09-30T22:35:35.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.249+0000 7f99f8ff9700 1 -- 172.21.15.16:0/1341745855 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f99fc1a7d00 con 0x7f99fc09f650 2023-09-30T22:35:35.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.278+0000 7fb09dd55700 1 Processor -- start 2023-09-30T22:35:35.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.279+0000 7fb09dd55700 1 -- start start 2023-09-30T22:35:35.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.279+0000 7fb09dd55700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb0980a0a30 0x7fb0980a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:35.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.279+0000 7fb09dd55700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb09809d630 0x7fb0980a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:35.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.279+0000 7fb09dd55700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0980a2a70 con 0x7fb098133930 2023-09-30T22:35:35.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.279+0000 7fb09dd55700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0980a01d0 con 0x7fb0980a0a30 2023-09-30T22:35:35.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.279+0000 7fb09dd55700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb09809e2b0 con 0x7fb09809d630 2023-09-30T22:35:35.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.279+0000 7fb0977fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb0980a0a30 0x7fb0980a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:35.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.279+0000 7fb097fff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb098133930 0x7fb098133dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33906/0 (socket says 172.21.15.16:33906) 2023-09-30T22:35:35.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.279+0000 7fb097fff700 1 -- 172.21.15.16:0/507207477 learned_addr learned my addr 172.21.15.16:0/507207477 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:35.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.279+0000 7fb096ffd700 1 --2- 172.21.15.16:0/507207477 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb09809d630 0x7fb0980a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:35.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.280+0000 7fb0967fc700 1 -- 172.21.15.16:0/507207477 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2839992337 0 0) 0x7fb0980a2a70 con 0x7fb098133930 2023-09-30T22:35:35.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.280+0000 7fb0977fe700 1 -- 172.21.15.16:0/507207477 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb09809d630 msgr2=0x7fb0980a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:35.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.280+0000 7fb0977fe700 1 --2- 172.21.15.16:0/507207477 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb09809d630 0x7fb0980a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:35.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.280+0000 7fb0977fe700 1 -- 172.21.15.16:0/507207477 >> v1:172.21.15.16:6789/0 conn(0x7fb098133930 legacy=0x7fb098133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:35.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.280+0000 7fb0977fe700 1 -- 172.21.15.16:0/507207477 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb09809dc50 con 0x7fb0980a0a30 2023-09-30T22:35:35.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.280+0000 7fb096ffd700 1 --2- 172.21.15.16:0/507207477 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb09809d630 0x7fb0980a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:35.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.280+0000 7fb0977fe700 1 --2- 172.21.15.16:0/507207477 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb0980a0a30 0x7fb0980a0e00 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fb088009dc0 tx=0x7fb08800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c1571d3b773d8bf server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:35.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.280+0000 7fb0967fc700 1 -- 172.21.15.16:0/507207477 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08800e070 con 0x7fb0980a0a30 2023-09-30T22:35:35.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.280+0000 7fb0967fc700 1 -- 172.21.15.16:0/507207477 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb08800bb20 con 0x7fb0980a0a30 2023-09-30T22:35:35.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.280+0000 7fb0967fc700 1 -- 172.21.15.16:0/507207477 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb088012430 con 0x7fb0980a0a30 2023-09-30T22:35:35.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.281+0000 7fb09dd55700 1 -- 172.21.15.16:0/507207477 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb0980a0a30 msgr2=0x7fb0980a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:35.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.281+0000 7fb09dd55700 1 --2- 172.21.15.16:0/507207477 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb0980a0a30 0x7fb0980a0e00 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fb088009dc0 tx=0x7fb08800b330 comp rx=0 tx=0).stop 2023-09-30T22:35:35.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.281+0000 7fb09dd55700 1 -- 172.21.15.16:0/507207477 shutdown_connections 2023-09-30T22:35:35.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.281+0000 7fb09dd55700 1 --2- 172.21.15.16:0/507207477 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb0980a0a30 0x7fb0980a0e00 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:35.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.281+0000 7fb09dd55700 1 --2- 172.21.15.16:0/507207477 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb09809d630 0x7fb0980a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:35.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.281+0000 7fb09dd55700 1 -- 172.21.15.16:0/507207477 >> 172.21.15.16:0/507207477 conn(0x7fb0980a3050 msgr2=0x7fb0980a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:35.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.281+0000 7fb09dd55700 1 -- 172.21.15.16:0/507207477 shutdown_connections 2023-09-30T22:35:35.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.281+0000 7fb09dd55700 1 -- 172.21.15.16:0/507207477 wait complete. 2023-09-30T22:35:35.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.282+0000 7fb09dd55700 1 Processor -- start 2023-09-30T22:35:35.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.282+0000 7fb09dd55700 1 -- start start 2023-09-30T22:35:35.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.283+0000 7fb09dd55700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb09809d630 0x7fb09811f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:35.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.283+0000 7fb09dd55700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb0980a0a30 0x7fb09811f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:35.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.283+0000 7fb09dd55700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb098133930 0x7fb0981c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:35.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.283+0000 7fb09dd55700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb098121ed0 con 0x7fb098133930 2023-09-30T22:35:35.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.283+0000 7fb09dd55700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb098133250 con 0x7fb09809d630 2023-09-30T22:35:35.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.283+0000 7fb09dd55700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb09811ee20 con 0x7fb0980a0a30 2023-09-30T22:35:35.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.283+0000 7fb0977fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb09809d630 0x7fb09811f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:35.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.283+0000 7fb096ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb0980a0a30 0x7fb09811f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:35.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.283+0000 7fb0977fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb09809d630 0x7fb09811f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:46154/0 (socket says 172.21.15.16:46154) 2023-09-30T22:35:35.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.283+0000 7fb0977fe700 1 -- 172.21.15.16:0/837759729 learned_addr learned my addr 172.21.15.16:0/837759729 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:35.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.283+0000 7fb096ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb0980a0a30 0x7fb09811f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:45456/0 (socket says 172.21.15.16:45456) 2023-09-30T22:35:35.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.283+0000 7fb097fff700 1 --2- 172.21.15.16:0/837759729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb098133930 0x7fb0981c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:35.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.283+0000 7fb0977fe700 1 -- 172.21.15.16:0/837759729 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb0980a0a30 msgr2=0x7fb09811f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:35.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.283+0000 7fb0977fe700 1 --2- 172.21.15.16:0/837759729 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb0980a0a30 0x7fb09811f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:35.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.283+0000 7fb0977fe700 1 -- 172.21.15.16:0/837759729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb098133930 msgr2=0x7fb0981c7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:35.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.283+0000 7fb0977fe700 1 --2- 172.21.15.16:0/837759729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb098133930 0x7fb0981c7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:35.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.283+0000 7fb0977fe700 1 -- 172.21.15.16:0/837759729 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb08c0094a0 con 0x7fb09809d630 2023-09-30T22:35:35.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.283+0000 7fb096ffd700 1 --2- 172.21.15.16:0/837759729 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb0980a0a30 0x7fb09811f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:35.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.283+0000 7fb0977fe700 1 --2- 172.21.15.16:0/837759729 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb09809d630 0x7fb09811f2c0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fb088014040 tx=0x7fb088004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:35.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.283+0000 7fb07ffff700 1 -- 172.21.15.16:0/837759729 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb088012430 con 0x7fb09809d630 2023-09-30T22:35:35.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.284+0000 7fb07ffff700 1 -- 172.21.15.16:0/837759729 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb088020070 con 0x7fb09809d630 2023-09-30T22:35:35.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.284+0000 7fb09dd55700 1 -- 172.21.15.16:0/837759729 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb088009a70 con 0x7fb09809d630 2023-09-30T22:35:35.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.284+0000 7fb07ffff700 1 -- 172.21.15.16:0/837759729 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb088004280 con 0x7fb09809d630 2023-09-30T22:35:35.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.284+0000 7fb09dd55700 1 -- 172.21.15.16:0/837759729 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0980a4250 con 0x7fb09809d630 2023-09-30T22:35:35.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.284+0000 7fb07ffff700 1 -- 172.21.15.16:0/837759729 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb0880043e0 con 0x7fb09809d630 2023-09-30T22:35:35.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.285+0000 7fb07ffff700 1 -- 172.21.15.16:0/837759729 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fb088022070 con 0x7fb09809d630 2023-09-30T22:35:35.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.285+0000 7fb09dd55700 1 -- 172.21.15.16:0/837759729 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb09811fda0 con 0x7fb09809d630 2023-09-30T22:35:35.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.290+0000 7fb07ffff700 1 -- 172.21.15.16:0/837759729 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb08801abc0 con 0x7fb09809d630 2023-09-30T22:35:35.307 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:35:35.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.445+0000 7f3c0e4d2700 1 Processor -- start 2023-09-30T22:35:35.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.445+0000 7f3c0e4d2700 1 -- start start 2023-09-30T22:35:35.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.445+0000 7f3c0e4d2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3c080a0a30 0x7f3c080a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:35.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.445+0000 7f3c0e4d2700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3c080a1340 0x7f3c08133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:35.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.445+0000 7f3c0e4d2700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c0809edd0 con 0x7f3c08133fa0 2023-09-30T22:35:35.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.445+0000 7f3c0e4d2700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c0809e1f0 con 0x7f3c080a1340 2023-09-30T22:35:35.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.445+0000 7f3c0e4d2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c0809dc50 con 0x7f3c080a0a30 2023-09-30T22:35:35.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.445+0000 7f3c07fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3c080a0a30 0x7f3c080a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:35.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.445+0000 7f3c07fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3c080a0a30 0x7f3c080a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:46170/0 (socket says 172.21.15.16:46170) 2023-09-30T22:35:35.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.445+0000 7f3c07fff700 1 -- 172.21.15.16:0/730289364 learned_addr learned my addr 172.21.15.16:0/730289364 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:35.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.446+0000 7f3c077fe700 1 --2- 172.21.15.16:0/730289364 >> v2:172.21.15.16:3300/0 conn(0x7f3c080a1340 0x7f3c08133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:35.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.446+0000 7f3c07fff700 1 -- 172.21.15.16:0/730289364 >> v2:172.21.15.16:3300/0 conn(0x7f3c080a1340 msgr2=0x7f3c08133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:35.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.446+0000 7f3c07fff700 1 --2- 172.21.15.16:0/730289364 >> v2:172.21.15.16:3300/0 conn(0x7f3c080a1340 0x7f3c08133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:35.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.446+0000 7f3c07fff700 1 -- 172.21.15.16:0/730289364 >> v1:172.21.15.16:6789/0 conn(0x7f3c08133fa0 legacy=0x7f3c0813ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:35.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.446+0000 7f3c07fff700 1 -- 172.21.15.16:0/730289364 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c080a1fe0 con 0x7f3c080a0a30 2023-09-30T22:35:35.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.446+0000 7f3c077fe700 1 --2- 172.21.15.16:0/730289364 >> v2:172.21.15.16:3300/0 conn(0x7f3c080a1340 0x7f3c08133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:35.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.446+0000 7f3c07fff700 1 --2- 172.21.15.16:0/730289364 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3c080a0a30 0x7f3c080a0e00 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f3bf80098f0 tx=0x7f3bf8009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26df703c93061229 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:35.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.446+0000 7f3c06ffd700 1 -- 172.21.15.16:0/730289364 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf8011830 con 0x7f3c080a0a30 2023-09-30T22:35:35.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.446+0000 7f3c06ffd700 1 -- 172.21.15.16:0/730289364 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bf8011990 con 0x7f3c080a0a30 2023-09-30T22:35:35.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.446+0000 7f3c06ffd700 1 -- 172.21.15.16:0/730289364 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bf8011d20 con 0x7f3c080a0a30 2023-09-30T22:35:35.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.447+0000 7f3c0e4d2700 1 -- 172.21.15.16:0/730289364 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3c080a0a30 msgr2=0x7f3c080a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:35.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.447+0000 7f3c0e4d2700 1 --2- 172.21.15.16:0/730289364 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3c080a0a30 0x7f3c080a0e00 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f3bf80098f0 tx=0x7f3bf8009c00 comp rx=0 tx=0).stop 2023-09-30T22:35:35.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.447+0000 7f3c0e4d2700 1 -- 172.21.15.16:0/730289364 shutdown_connections 2023-09-30T22:35:35.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.447+0000 7f3c0e4d2700 1 --2- 172.21.15.16:0/730289364 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3c080a0a30 0x7f3c080a0e00 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:35.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.447+0000 7f3c0e4d2700 1 --2- 172.21.15.16:0/730289364 >> v2:172.21.15.16:3300/0 conn(0x7f3c080a1340 0x7f3c08133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:35.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.447+0000 7f3c0e4d2700 1 -- 172.21.15.16:0/730289364 >> 172.21.15.16:0/730289364 conn(0x7f3c080a6550 msgr2=0x7f3c080a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:35.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.447+0000 7f3c0e4d2700 1 -- 172.21.15.16:0/730289364 shutdown_connections 2023-09-30T22:35:35.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.447+0000 7f3c0e4d2700 1 -- 172.21.15.16:0/730289364 wait complete. 2023-09-30T22:35:35.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.448+0000 7f3c0e4d2700 1 Processor -- start 2023-09-30T22:35:35.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.448+0000 7f3c0e4d2700 1 -- start start 2023-09-30T22:35:35.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.449+0000 7f3c0e4d2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c080a0a30 0x7f3c081c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:35.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.449+0000 7f3c0e4d2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3c080a1340 0x7f3c081c8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:35.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.449+0000 7f3c0e4d2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3c08133fa0 0x7f3c0811e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:35.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.449+0000 7f3c0e4d2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c081bd3a0 con 0x7f3c080a0a30 2023-09-30T22:35:35.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.449+0000 7f3c0e4d2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c081333d0 con 0x7f3c08133fa0 2023-09-30T22:35:35.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.449+0000 7f3c0e4d2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3c08120390 con 0x7f3c080a1340 2023-09-30T22:35:35.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.449+0000 7f3c077fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3c080a1340 0x7f3c081c8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:35.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.449+0000 7f3c077fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3c080a1340 0x7f3c081c8c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:45460/0 (socket says 172.21.15.16:45460) 2023-09-30T22:35:35.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.449+0000 7f3c077fe700 1 -- 172.21.15.16:0/3537322117 learned_addr learned my addr 172.21.15.16:0/3537322117 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:35.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.449+0000 7f3c0ca6f700 1 --2- 172.21.15.16:0/3537322117 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3c08133fa0 0x7f3c0811e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:35.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.449+0000 7f3c07fff700 1 --2- 172.21.15.16:0/3537322117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c080a0a30 0x7f3c081c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:35.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.449+0000 7f3c077fe700 1 -- 172.21.15.16:0/3537322117 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3c08133fa0 msgr2=0x7f3c0811e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:35.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.449+0000 7f3c077fe700 1 --2- 172.21.15.16:0/3537322117 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3c08133fa0 0x7f3c0811e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:35.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.449+0000 7f3c077fe700 1 -- 172.21.15.16:0/3537322117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c080a0a30 msgr2=0x7f3c081c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:35.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.449+0000 7f3c077fe700 1 --2- 172.21.15.16:0/3537322117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c080a0a30 0x7f3c081c86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:35.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.449+0000 7f3c077fe700 1 -- 172.21.15.16:0/3537322117 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bfc0094a0 con 0x7f3c080a1340 2023-09-30T22:35:35.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.450+0000 7f3c077fe700 1 --2- 172.21.15.16:0/3537322117 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3c080a1340 0x7f3c081c8c20 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc00ebe0 tx=0x7f3bfc00eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:35.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.450+0000 7f3c04ff9700 1 -- 172.21.15.16:0/3537322117 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc00c7f0 con 0x7f3c080a1340 2023-09-30T22:35:35.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.450+0000 7f3c04ff9700 1 -- 172.21.15.16:0/3537322117 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bfc00c950 con 0x7f3c080a1340 2023-09-30T22:35:35.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.450+0000 7f3c0e4d2700 1 -- 172.21.15.16:0/3537322117 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bf8009550 con 0x7f3c080a1340 2023-09-30T22:35:35.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.450+0000 7f3c04ff9700 1 -- 172.21.15.16:0/3537322117 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc00cce0 con 0x7f3c080a1340 2023-09-30T22:35:35.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.450+0000 7f3c0e4d2700 1 -- 172.21.15.16:0/3537322117 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c080a0270 con 0x7f3c080a1340 2023-09-30T22:35:35.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.451+0000 7f3c04ff9700 1 -- 172.21.15.16:0/3537322117 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3bfc013470 con 0x7f3c080a1340 2023-09-30T22:35:35.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.451+0000 7f3c04ff9700 1 -- 172.21.15.16:0/3537322117 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f3bfc03f7a0 con 0x7f3c080a1340 2023-09-30T22:35:35.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.451+0000 7f3c0e4d2700 1 -- 172.21.15.16:0/3537322117 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c080a1d00 con 0x7f3c080a1340 2023-09-30T22:35:35.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.456+0000 7f3c04ff9700 1 -- 172.21.15.16:0/3537322117 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3bfc021080 con 0x7f3c080a1340 2023-09-30T22:35:35.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.464+0000 7f9a02968700 1 -- 172.21.15.16:0/1341745855 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f99fc1a7d00 con 0x7f99fc09f650 2023-09-30T22:35:35.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.465+0000 7f99f8ff9700 1 -- 172.21.15.16:0/1341745855 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f99fc1a7d00 con 0x7f99fc09f650 2023-09-30T22:35:35.466 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:35:35.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.467+0000 7f9a02968700 1 -- 172.21.15.16:0/1341745855 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f99fc09f650 msgr2=0x7f99fc1c8150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:35.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.467+0000 7f9a02968700 1 --2- 172.21.15.16:0/1341745855 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f99fc09f650 0x7f99fc1c8150 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f99f400efc0 tx=0x7f99f400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:35:35.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.467+0000 7f9a02968700 1 -- 172.21.15.16:0/1341745855 shutdown_connections 2023-09-30T22:35:35.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.467+0000 7f9a02968700 1 --2- 172.21.15.16:0/1341745855 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f99fc09f650 0x7f99fc1c8150 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:35.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.467+0000 7f9a02968700 1 --2- 172.21.15.16:0/1341745855 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f99fc09e930 0x7f99fc06c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:35.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.467+0000 7f9a02968700 1 --2- 172.21.15.16:0/1341745855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99fc09f240 0x7f99fc06c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:35.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.467+0000 7f9a02968700 1 -- 172.21.15.16:0/1341745855 >> 172.21.15.16:0/1341745855 conn(0x7f99fc0a4210 msgr2=0x7f99fc1b9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:35.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.468+0000 7f9a02968700 1 -- 172.21.15.16:0/1341745855 shutdown_connections 2023-09-30T22:35:35.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.468+0000 7f9a02968700 1 -- 172.21.15.16:0/1341745855 wait complete. 2023-09-30T22:35:35.483 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:35:35.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.498+0000 7fb09dd55700 1 -- 172.21.15.16:0/837759729 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb098120a40 con 0x7fb09809d630 2023-09-30T22:35:35.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.498+0000 7fb07ffff700 1 -- 172.21.15.16:0/837759729 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb08802c030 con 0x7fb09809d630 2023-09-30T22:35:35.500 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:35:35.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.502+0000 7fb09dd55700 1 -- 172.21.15.16:0/837759729 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb09809d630 msgr2=0x7fb09811f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:35.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.502+0000 7fb09dd55700 1 --2- 172.21.15.16:0/837759729 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb09809d630 0x7fb09811f2c0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fb088014040 tx=0x7fb088004ad0 comp rx=0 tx=0).stop 2023-09-30T22:35:35.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.502+0000 7fb09dd55700 1 -- 172.21.15.16:0/837759729 shutdown_connections 2023-09-30T22:35:35.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.502+0000 7fb09dd55700 1 --2- 172.21.15.16:0/837759729 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb09809d630 0x7fb09811f2c0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:35.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.502+0000 7fb09dd55700 1 --2- 172.21.15.16:0/837759729 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb098133930 0x7fb0981c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:35.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.502+0000 7fb09dd55700 1 --2- 172.21.15.16:0/837759729 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb0980a0a30 0x7fb09811f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:35.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.502+0000 7fb09dd55700 1 -- 172.21.15.16:0/837759729 >> 172.21.15.16:0/837759729 conn(0x7fb0980a3050 msgr2=0x7fb0981bbb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:35.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.502+0000 7fb09dd55700 1 -- 172.21.15.16:0/837759729 shutdown_connections 2023-09-30T22:35:35.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.502+0000 7fb09dd55700 1 -- 172.21.15.16:0/837759729 wait complete. 2023-09-30T22:35:35.522 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:35:35.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.672+0000 7f3c0e4d2700 1 -- 172.21.15.16:0/3537322117 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3c08074ea0 con 0x7f3c080a1340 2023-09-30T22:35:35.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.672+0000 7f3c04ff9700 1 -- 172.21.15.16:0/3537322117 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3bfc01b970 con 0x7f3c080a1340 2023-09-30T22:35:35.674 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:35:35.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.674+0000 7f3c0e4d2700 1 -- 172.21.15.16:0/3537322117 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3c080a1340 msgr2=0x7f3c081c8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:35.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.674+0000 7f3c0e4d2700 1 --2- 172.21.15.16:0/3537322117 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3c080a1340 0x7f3c081c8c20 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc00ebe0 tx=0x7f3bfc00eef0 comp rx=0 tx=0).stop 2023-09-30T22:35:35.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.674+0000 7f3c0e4d2700 1 -- 172.21.15.16:0/3537322117 shutdown_connections 2023-09-30T22:35:35.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.674+0000 7f3c0e4d2700 1 --2- 172.21.15.16:0/3537322117 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3c08133fa0 0x7f3c0811e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:35.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.674+0000 7f3c0e4d2700 1 --2- 172.21.15.16:0/3537322117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c080a0a30 0x7f3c081c86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:35.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.674+0000 7f3c0e4d2700 1 --2- 172.21.15.16:0/3537322117 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3c080a1340 0x7f3c081c8c20 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:35.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.674+0000 7f3c0e4d2700 1 -- 172.21.15.16:0/3537322117 >> 172.21.15.16:0/3537322117 conn(0x7f3c080a6550 msgr2=0x7f3c081b9150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:35.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.675+0000 7f3c0e4d2700 1 -- 172.21.15.16:0/3537322117 shutdown_connections 2023-09-30T22:35:35.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:35.675+0000 7f3c0e4d2700 1 -- 172.21.15.16:0/3537322117 wait complete. 2023-09-30T22:35:35.689 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:35:35.958 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:35:36.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.102+0000 7fa5ee43f700 1 Processor -- start 2023-09-30T22:35:36.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.102+0000 7fa5ee43f700 1 -- start start 2023-09-30T22:35:36.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.103+0000 7fa5ee43f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa5e809d6f0 0x7fa5e809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:36.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.103+0000 7fa5ee43f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa5e809e410 0x7fa5e8134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:36.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.103+0000 7fa5ee43f700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5e809b120 con 0x7fa5e809e000 2023-09-30T22:35:36.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.103+0000 7fa5ee43f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e806b460 con 0x7fa5e809d6f0 2023-09-30T22:35:36.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.103+0000 7fa5ee43f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e809a7e0 con 0x7fa5e809e410 2023-09-30T22:35:36.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.103+0000 7fa5ec9dc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa5e809e410 0x7fa5e8134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:36.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.103+0000 7fa5e77fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa5e809e000 0x7fa5e81339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33922/0 (socket says 172.21.15.16:33922) 2023-09-30T22:35:36.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.103+0000 7fa5e77fe700 1 -- 172.21.15.16:0/1491298870 learned_addr learned my addr 172.21.15.16:0/1491298870 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:36.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.103+0000 7fa5e7fff700 1 --2- 172.21.15.16:0/1491298870 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa5e809d6f0 0x7fa5e809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:36.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.103+0000 7fa5ec9dc700 1 -- 172.21.15.16:0/1491298870 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa5e809d6f0 msgr2=0x7fa5e809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:36.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.103+0000 7fa5ec9dc700 1 --2- 172.21.15.16:0/1491298870 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa5e809d6f0 0x7fa5e809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:36.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.103+0000 7fa5e6ffd700 1 -- 172.21.15.16:0/1491298870 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3037842055 0 0) 0x7fa5e809b120 con 0x7fa5e809e000 2023-09-30T22:35:36.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.103+0000 7fa5ec9dc700 1 -- 172.21.15.16:0/1491298870 >> v1:172.21.15.16:6789/0 conn(0x7fa5e809e000 legacy=0x7fa5e81339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:36.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.103+0000 7fa5ec9dc700 1 -- 172.21.15.16:0/1491298870 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5e809a500 con 0x7fa5e809e410 2023-09-30T22:35:36.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.103+0000 7fa5ec9dc700 1 --2- 172.21.15.16:0/1491298870 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa5e809e410 0x7fa5e8134100 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0009df0 tx=0x7fa5e000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59f614186d6bbb75 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:36.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.104+0000 7fa5e6ffd700 1 -- 172.21.15.16:0/1491298870 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e000e070 con 0x7fa5e809e410 2023-09-30T22:35:36.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.104+0000 7fa5e6ffd700 1 -- 172.21.15.16:0/1491298870 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5e000bae0 con 0x7fa5e809e410 2023-09-30T22:35:36.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.104+0000 7fa5e6ffd700 1 -- 172.21.15.16:0/1491298870 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e000be90 con 0x7fa5e809e410 2023-09-30T22:35:36.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.104+0000 7fa5ee43f700 1 -- 172.21.15.16:0/1491298870 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa5e809e410 msgr2=0x7fa5e8134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:36.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.104+0000 7fa5ee43f700 1 --2- 172.21.15.16:0/1491298870 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa5e809e410 0x7fa5e8134100 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fa5e0009df0 tx=0x7fa5e000b330 comp rx=0 tx=0).stop 2023-09-30T22:35:36.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.104+0000 7fa5ee43f700 1 -- 172.21.15.16:0/1491298870 shutdown_connections 2023-09-30T22:35:36.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.104+0000 7fa5ee43f700 1 --2- 172.21.15.16:0/1491298870 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa5e809e410 0x7fa5e8134100 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:36.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.104+0000 7fa5ee43f700 1 --2- 172.21.15.16:0/1491298870 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa5e809d6f0 0x7fa5e809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:36.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.104+0000 7fa5ee43f700 1 -- 172.21.15.16:0/1491298870 >> 172.21.15.16:0/1491298870 conn(0x7fa5e8066f50 msgr2=0x7fa5e809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:36.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.104+0000 7fa5ee43f700 1 -- 172.21.15.16:0/1491298870 shutdown_connections 2023-09-30T22:35:36.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.104+0000 7fa5ee43f700 1 -- 172.21.15.16:0/1491298870 wait complete. 2023-09-30T22:35:36.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.105+0000 7fa5ee43f700 1 Processor -- start 2023-09-30T22:35:36.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.105+0000 7fa5ee43f700 1 -- start start 2023-09-30T22:35:36.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.106+0000 7fa5ee43f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa5e809d6f0 0x7fa5e8122320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:36.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.106+0000 7fa5ee43f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5e809e000 0x7fa5e8122860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:36.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.106+0000 7fa5ee43f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa5e809e410 0x7fa5e8122da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:36.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.106+0000 7fa5ee43f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e8123ea0 con 0x7fa5e809e000 2023-09-30T22:35:36.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.106+0000 7fa5ee43f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e8123780 con 0x7fa5e809e410 2023-09-30T22:35:36.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.106+0000 7fa5ee43f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa5e8124d30 con 0x7fa5e809d6f0 2023-09-30T22:35:36.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.106+0000 7fa5ec9dc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa5e809e410 0x7fa5e8122da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:36.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.106+0000 7fa5e7fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa5e809d6f0 0x7fa5e8122320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:36.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.106+0000 7fa5ec9dc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa5e809e410 0x7fa5e8122da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:46204/0 (socket says 172.21.15.16:46204) 2023-09-30T22:35:36.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.106+0000 7fa5ec9dc700 1 -- 172.21.15.16:0/1908303378 learned_addr learned my addr 172.21.15.16:0/1908303378 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:36.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.106+0000 7fa5e77fe700 1 --2- 172.21.15.16:0/1908303378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5e809e000 0x7fa5e8122860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:36.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.106+0000 7fa5e7fff700 1 -- 172.21.15.16:0/1908303378 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa5e809e410 msgr2=0x7fa5e8122da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:36.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.106+0000 7fa5e7fff700 1 --2- 172.21.15.16:0/1908303378 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa5e809e410 0x7fa5e8122da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:36.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.106+0000 7fa5e7fff700 1 -- 172.21.15.16:0/1908303378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5e809e000 msgr2=0x7fa5e8122860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:36.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.106+0000 7fa5e7fff700 1 --2- 172.21.15.16:0/1908303378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5e809e000 0x7fa5e8122860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:36.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.106+0000 7fa5e7fff700 1 -- 172.21.15.16:0/1908303378 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5e0009a70 con 0x7fa5e809d6f0 2023-09-30T22:35:36.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.106+0000 7fa5ec9dc700 1 --2- 172.21.15.16:0/1908303378 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa5e809e410 0x7fa5e8122da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:36.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.107+0000 7fa5e7fff700 1 --2- 172.21.15.16:0/1908303378 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa5e809d6f0 0x7fa5e8122320 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fa5d800e900 tx=0x7fa5d800ec10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:36.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.107+0000 7fa5e4ff9700 1 -- 172.21.15.16:0/1908303378 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d8013680 con 0x7fa5e809d6f0 2023-09-30T22:35:36.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.107+0000 7fa5e4ff9700 1 -- 172.21.15.16:0/1908303378 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5d80137e0 con 0x7fa5e809d6f0 2023-09-30T22:35:36.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.107+0000 7fa5ee43f700 1 -- 172.21.15.16:0/1908303378 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5e8125810 con 0x7fa5e809d6f0 2023-09-30T22:35:36.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.107+0000 7fa5e4ff9700 1 -- 172.21.15.16:0/1908303378 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5d8013b50 con 0x7fa5e809d6f0 2023-09-30T22:35:36.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.107+0000 7fa5ee43f700 1 -- 172.21.15.16:0/1908303378 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5e809c710 con 0x7fa5e809d6f0 2023-09-30T22:35:36.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.108+0000 7fa5e4ff9700 1 -- 172.21.15.16:0/1908303378 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa5d8013cb0 con 0x7fa5e809d6f0 2023-09-30T22:35:36.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.108+0000 7fa5e4ff9700 1 -- 172.21.15.16:0/1908303378 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fa5d803f770 con 0x7fa5e809d6f0 2023-09-30T22:35:36.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.108+0000 7fa5ee43f700 1 -- 172.21.15.16:0/1908303378 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5e811f030 con 0x7fa5e809d6f0 2023-09-30T22:35:36.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.113+0000 7fa5e4ff9700 1 -- 172.21.15.16:0/1908303378 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa5d801be10 con 0x7fa5e809d6f0 2023-09-30T22:35:36.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.319+0000 7fa5ee43f700 1 -- 172.21.15.16:0/1908303378 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa5e81259a0 con 0x7fa5e809d6f0 2023-09-30T22:35:36.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.320+0000 7fa5e4ff9700 1 -- 172.21.15.16:0/1908303378 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa5d802a090 con 0x7fa5e809d6f0 2023-09-30T22:35:36.321 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:35:36.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.322+0000 7fa5ee43f700 1 -- 172.21.15.16:0/1908303378 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa5e809d6f0 msgr2=0x7fa5e8122320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:36.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.322+0000 7fa5ee43f700 1 --2- 172.21.15.16:0/1908303378 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa5e809d6f0 0x7fa5e8122320 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fa5d800e900 tx=0x7fa5d800ec10 comp rx=0 tx=0).stop 2023-09-30T22:35:36.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.322+0000 7fa5ee43f700 1 -- 172.21.15.16:0/1908303378 shutdown_connections 2023-09-30T22:35:36.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.322+0000 7fa5ee43f700 1 --2- 172.21.15.16:0/1908303378 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa5e809e410 0x7fa5e8122da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:36.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.322+0000 7fa5ee43f700 1 --2- 172.21.15.16:0/1908303378 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa5e809d6f0 0x7fa5e8122320 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:36.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.322+0000 7fa5ee43f700 1 --2- 172.21.15.16:0/1908303378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa5e809e000 0x7fa5e8122860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:36.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.322+0000 7fa5ee43f700 1 -- 172.21.15.16:0/1908303378 >> 172.21.15.16:0/1908303378 conn(0x7fa5e8066f50 msgr2=0x7fa5e8133f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:36.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.322+0000 7fa5ee43f700 1 -- 172.21.15.16:0/1908303378 shutdown_connections 2023-09-30T22:35:36.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.322+0000 7fa5ee43f700 1 -- 172.21.15.16:0/1908303378 wait complete. 2023-09-30T22:35:36.337 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:35:36.485 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:35:36.523 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:35:36.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.631+0000 7fd5bc824700 1 Processor -- start 2023-09-30T22:35:36.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.631+0000 7fd5bc824700 1 -- start start 2023-09-30T22:35:36.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.632+0000 7fd5bc824700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd5b809d630 0x7fd5b80a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:36.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.632+0000 7fd5bc824700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd5b8138d80 0x7fd5b813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:36.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.632+0000 7fd5bc824700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5b80a2a70 con 0x7fd5b80a0a30 2023-09-30T22:35:36.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.632+0000 7fd5bc824700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5b80a01d0 con 0x7fd5b809d630 2023-09-30T22:35:36.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.632+0000 7fd5bc824700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b809e2b0 con 0x7fd5b8138d80 2023-09-30T22:35:36.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.632+0000 7fd5b6d9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd5b8138d80 0x7fd5b813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:36.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.632+0000 7fd5b5d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd5b809d630 0x7fd5b80a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:36.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.632+0000 7fd5b6d9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd5b8138d80 0x7fd5b813b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:46220/0 (socket says 172.21.15.16:46220) 2023-09-30T22:35:36.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.632+0000 7fd5b6d9d700 1 -- 172.21.15.16:0/3267814886 learned_addr learned my addr 172.21.15.16:0/3267814886 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:36.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.633+0000 7fd5b6d9d700 1 -- 172.21.15.16:0/3267814886 >> v2:172.21.15.16:3300/0 conn(0x7fd5b809d630 msgr2=0x7fd5b80a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:36.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.633+0000 7fd5b6d9d700 1 --2- 172.21.15.16:0/3267814886 >> v2:172.21.15.16:3300/0 conn(0x7fd5b809d630 0x7fd5b80a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:36.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.633+0000 7fd5b6d9d700 1 -- 172.21.15.16:0/3267814886 >> v1:172.21.15.16:6789/0 conn(0x7fd5b80a0a30 legacy=0x7fd5b80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:36.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.633+0000 7fd5b6d9d700 1 -- 172.21.15.16:0/3267814886 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5b809dc50 con 0x7fd5b8138d80 2023-09-30T22:35:36.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.633+0000 7fd5b559a700 1 -- 172.21.15.16:0/3267814886 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3576560235 0 0) 0x7fd5b80a2a70 con 0x7fd5b80a0a30 2023-09-30T22:35:36.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.633+0000 7fd5b5d9b700 1 --2- 172.21.15.16:0/3267814886 >> v2:172.21.15.16:3300/0 conn(0x7fd5b809d630 0x7fd5b80a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:36.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.633+0000 7fd5b6d9d700 1 --2- 172.21.15.16:0/3267814886 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd5b8138d80 0x7fd5b813b160 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fd5a80098f0 tx=0x7fd5a8009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=344463981abc4ad9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:36.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.633+0000 7fd5b559a700 1 -- 172.21.15.16:0/3267814886 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a8011830 con 0x7fd5b8138d80 2023-09-30T22:35:36.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.633+0000 7fd5b559a700 1 -- 172.21.15.16:0/3267814886 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5a8011990 con 0x7fd5b8138d80 2023-09-30T22:35:36.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.633+0000 7fd5b559a700 1 -- 172.21.15.16:0/3267814886 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5a8011d20 con 0x7fd5b8138d80 2023-09-30T22:35:36.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.634+0000 7fd5bc824700 1 -- 172.21.15.16:0/3267814886 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd5b8138d80 msgr2=0x7fd5b813b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:36.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.634+0000 7fd5bc824700 1 --2- 172.21.15.16:0/3267814886 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd5b8138d80 0x7fd5b813b160 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fd5a80098f0 tx=0x7fd5a8009c00 comp rx=0 tx=0).stop 2023-09-30T22:35:36.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.634+0000 7fd5bc824700 1 -- 172.21.15.16:0/3267814886 shutdown_connections 2023-09-30T22:35:36.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.634+0000 7fd5bc824700 1 --2- 172.21.15.16:0/3267814886 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd5b8138d80 0x7fd5b813b160 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:36.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.634+0000 7fd5bc824700 1 --2- 172.21.15.16:0/3267814886 >> v2:172.21.15.16:3300/0 conn(0x7fd5b809d630 0x7fd5b80a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:36.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.634+0000 7fd5bc824700 1 -- 172.21.15.16:0/3267814886 >> 172.21.15.16:0/3267814886 conn(0x7fd5b80a3050 msgr2=0x7fd5b80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:36.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.634+0000 7fd5bc824700 1 -- 172.21.15.16:0/3267814886 shutdown_connections 2023-09-30T22:35:36.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.634+0000 7fd5bc824700 1 -- 172.21.15.16:0/3267814886 wait complete. 2023-09-30T22:35:36.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.635+0000 7fd5bc824700 1 Processor -- start 2023-09-30T22:35:36.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.635+0000 7fd5bc824700 1 -- start start 2023-09-30T22:35:36.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.636+0000 7fd5bc824700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5b809d630 0x7fd5b8133700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:36.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.636+0000 7fd5bc824700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd5b80a0a30 0x7fd5b8133c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:36.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.636+0000 7fd5bc824700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd5b8138d80 0x7fd5b8134180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:36.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.636+0000 7fd5bc824700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b8137180 con 0x7fd5b809d630 2023-09-30T22:35:36.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.636+0000 7fd5bc824700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b8136070 con 0x7fd5b8138d80 2023-09-30T22:35:36.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.636+0000 7fd5bc824700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b8136530 con 0x7fd5b80a0a30 2023-09-30T22:35:36.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.636+0000 7fd5b659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5b809d630 0x7fd5b8133700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:36.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.636+0000 7fd5b6d9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd5b8138d80 0x7fd5b8134180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:36.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.636+0000 7fd5b659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5b809d630 0x7fd5b8133700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55178/0 (socket says 172.21.15.16:55178) 2023-09-30T22:35:36.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.636+0000 7fd5b659c700 1 -- 172.21.15.16:0/3217139320 learned_addr learned my addr 172.21.15.16:0/3217139320 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:36.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.636+0000 7fd5b5d9b700 1 --2- 172.21.15.16:0/3217139320 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd5b80a0a30 0x7fd5b8133c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:36.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.636+0000 7fd5b659c700 1 -- 172.21.15.16:0/3217139320 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd5b80a0a30 msgr2=0x7fd5b8133c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:36.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.636+0000 7fd5b659c700 1 --2- 172.21.15.16:0/3217139320 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd5b80a0a30 0x7fd5b8133c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:36.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.636+0000 7fd5b659c700 1 -- 172.21.15.16:0/3217139320 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd5b8138d80 msgr2=0x7fd5b8134180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:36.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.636+0000 7fd5b659c700 1 --2- 172.21.15.16:0/3217139320 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd5b8138d80 0x7fd5b8134180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:36.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.636+0000 7fd5b659c700 1 -- 172.21.15.16:0/3217139320 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a00094a0 con 0x7fd5b809d630 2023-09-30T22:35:36.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.636+0000 7fd5b6d9d700 1 --2- 172.21.15.16:0/3217139320 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd5b8138d80 0x7fd5b8134180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:36.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.636+0000 7fd5b659c700 1 --2- 172.21.15.16:0/3217139320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5b809d630 0x7fd5b8133700 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fd5b000c8c0 tx=0x7fd5b000cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:36.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.636+0000 7fd5aeffd700 1 -- 172.21.15.16:0/3217139320 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b0003e30 con 0x7fd5b809d630 2023-09-30T22:35:36.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.637+0000 7fd5aeffd700 1 -- 172.21.15.16:0/3217139320 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5b0005070 con 0x7fd5b809d630 2023-09-30T22:35:36.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.637+0000 7fd5bc824700 1 -- 172.21.15.16:0/3217139320 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5a8009550 con 0x7fd5b809d630 2023-09-30T22:35:36.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.637+0000 7fd5aeffd700 1 -- 172.21.15.16:0/3217139320 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b0005400 con 0x7fd5b809d630 2023-09-30T22:35:36.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.637+0000 7fd5bc824700 1 -- 172.21.15.16:0/3217139320 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5b8136be0 con 0x7fd5b809d630 2023-09-30T22:35:36.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.637+0000 7fd5aeffd700 1 -- 172.21.15.16:0/3217139320 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd5b0005560 con 0x7fd5b809d630 2023-09-30T22:35:36.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.638+0000 7fd5bc824700 1 -- 172.21.15.16:0/3217139320 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5b8137d60 con 0x7fd5b809d630 2023-09-30T22:35:36.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.638+0000 7fd5aeffd700 1 -- 172.21.15.16:0/3217139320 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd5b003eee0 con 0x7fd5b809d630 2023-09-30T22:35:36.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.642+0000 7fd5aeffd700 1 -- 172.21.15.16:0/3217139320 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd5b001b8d0 con 0x7fd5b809d630 2023-09-30T22:35:36.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.671+0000 7f50cbac8700 1 Processor -- start 2023-09-30T22:35:36.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.671+0000 7f50cbac8700 1 -- start start 2023-09-30T22:35:36.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.672+0000 7f50cbac8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f50c409f410 0x7f50c4136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:36.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.672+0000 7f50cbac8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f50c4136fe0 0x7f50c41393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:36.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.672+0000 7f50cbac8700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50c4070940 con 0x7f50c409e930 2023-09-30T22:35:36.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.672+0000 7f50cbac8700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50c406fd60 con 0x7f50c4136fe0 2023-09-30T22:35:36.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.672+0000 7f50cbac8700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f50c406fa80 con 0x7f50c409f410 2023-09-30T22:35:36.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.672+0000 7f50c9063700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f50c409f410 0x7f50c4136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:36.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.672+0000 7f50ca065700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f50c4136fe0 0x7f50c41393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:36.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.672+0000 7f50c9864700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f50c409e930 0x7f50c409ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:33926/0 (socket says 172.21.15.16:33926) 2023-09-30T22:35:36.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.672+0000 7f50c9864700 1 -- 172.21.15.16:0/1302245865 learned_addr learned my addr 172.21.15.16:0/1302245865 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:36.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.672+0000 7f50ca065700 1 -- 172.21.15.16:0/1302245865 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f50c409f410 msgr2=0x7f50c4136aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:36.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.672+0000 7f50ca065700 1 --2- 172.21.15.16:0/1302245865 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f50c409f410 0x7f50c4136aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:36.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.672+0000 7f50c8862700 1 -- 172.21.15.16:0/1302245865 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1266090371 0 0) 0x7f50c4070940 con 0x7f50c409e930 2023-09-30T22:35:36.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.672+0000 7f50ca065700 1 -- 172.21.15.16:0/1302245865 >> v1:172.21.15.16:6789/0 conn(0x7f50c409e930 legacy=0x7f50c409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:36.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.672+0000 7f50ca065700 1 -- 172.21.15.16:0/1302245865 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c409fb60 con 0x7f50c4136fe0 2023-09-30T22:35:36.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.672+0000 7f50c9063700 1 --2- 172.21.15.16:0/1302245865 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f50c409f410 0x7f50c4136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:36.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.672+0000 7f50ca065700 1 --2- 172.21.15.16:0/1302245865 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f50c4136fe0 0x7f50c41393c0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f50bc009dc0 tx=0x7f50bc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=684b49572c4c6b5f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:36.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.672+0000 7f50c8862700 1 -- 172.21.15.16:0/1302245865 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50bc00e070 con 0x7f50c4136fe0 2023-09-30T22:35:36.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.673+0000 7f50c8862700 1 -- 172.21.15.16:0/1302245865 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50bc00bb20 con 0x7f50c4136fe0 2023-09-30T22:35:36.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.673+0000 7f50c8862700 1 -- 172.21.15.16:0/1302245865 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50bc012430 con 0x7f50c4136fe0 2023-09-30T22:35:36.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.673+0000 7f50cbac8700 1 -- 172.21.15.16:0/1302245865 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f50c4136fe0 msgr2=0x7f50c41393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:36.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.673+0000 7f50cbac8700 1 --2- 172.21.15.16:0/1302245865 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f50c4136fe0 0x7f50c41393c0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f50bc009dc0 tx=0x7f50bc00b330 comp rx=0 tx=0).stop 2023-09-30T22:35:36.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.673+0000 7f50cbac8700 1 -- 172.21.15.16:0/1302245865 shutdown_connections 2023-09-30T22:35:36.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.673+0000 7f50cbac8700 1 --2- 172.21.15.16:0/1302245865 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f50c4136fe0 0x7f50c41393c0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:36.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.673+0000 7f50cbac8700 1 --2- 172.21.15.16:0/1302245865 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f50c409f410 0x7f50c4136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:36.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.673+0000 7f50cbac8700 1 -- 172.21.15.16:0/1302245865 >> 172.21.15.16:0/1302245865 conn(0x7f50c40a4210 msgr2=0x7f50c40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:36.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.673+0000 7f50cbac8700 1 -- 172.21.15.16:0/1302245865 shutdown_connections 2023-09-30T22:35:36.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.673+0000 7f50cbac8700 1 -- 172.21.15.16:0/1302245865 wait complete. 2023-09-30T22:35:36.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.674+0000 7f50cbac8700 1 Processor -- start 2023-09-30T22:35:36.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.674+0000 7f50cbac8700 1 -- start start 2023-09-30T22:35:36.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.675+0000 7f50cbac8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f50c409e930 0x7f50c406b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:36.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.675+0000 7f50cbac8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f50c409f410 0x7f50c406bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:36.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.675+0000 7f50cbac8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50c4136fe0 0x7f50c406c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:36.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.675+0000 7f50cbac8700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c4135080 con 0x7f50c4136fe0 2023-09-30T22:35:36.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.675+0000 7f50cbac8700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50c4124ed0 con 0x7f50c409e930 2023-09-30T22:35:36.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.675+0000 7f50cbac8700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f50c406d5d0 con 0x7f50c409f410 2023-09-30T22:35:36.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.675+0000 7f50c9063700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f50c409f410 0x7f50c406bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:36.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.675+0000 7f50ca065700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50c4136fe0 0x7f50c406c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:36.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.675+0000 7f50c9063700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f50c409f410 0x7f50c406bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:45498/0 (socket says 172.21.15.16:45498) 2023-09-30T22:35:36.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.675+0000 7f50c9063700 1 -- 172.21.15.16:0/1336697192 learned_addr learned my addr 172.21.15.16:0/1336697192 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:36.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.675+0000 7f50c9864700 1 --2- 172.21.15.16:0/1336697192 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f50c409e930 0x7f50c406b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:36.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.675+0000 7f50c9063700 1 -- 172.21.15.16:0/1336697192 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f50c409e930 msgr2=0x7f50c406b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:36.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.675+0000 7f50c9063700 1 --2- 172.21.15.16:0/1336697192 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f50c409e930 0x7f50c406b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:36.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.675+0000 7f50c9063700 1 -- 172.21.15.16:0/1336697192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50c4136fe0 msgr2=0x7f50c406c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:36.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.675+0000 7f50c9063700 1 --2- 172.21.15.16:0/1336697192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50c4136fe0 0x7f50c406c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:36.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.675+0000 7f50c9063700 1 -- 172.21.15.16:0/1336697192 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50b40094a0 con 0x7f50c409f410 2023-09-30T22:35:36.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.675+0000 7f50c9063700 1 --2- 172.21.15.16:0/1336697192 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f50c409f410 0x7f50c406bdf0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f50b400ec10 tx=0x7f50b400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:36.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.675+0000 7f50ba7fc700 1 -- 172.21.15.16:0/1336697192 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b400c850 con 0x7f50c409f410 2023-09-30T22:35:36.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.676+0000 7f50ba7fc700 1 -- 172.21.15.16:0/1336697192 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50b400c9b0 con 0x7f50c409f410 2023-09-30T22:35:36.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.676+0000 7f50cbac8700 1 -- 172.21.15.16:0/1336697192 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50bc009a70 con 0x7f50c409f410 2023-09-30T22:35:36.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.676+0000 7f50ba7fc700 1 -- 172.21.15.16:0/1336697192 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b400cd20 con 0x7f50c409f410 2023-09-30T22:35:36.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.676+0000 7f50cbac8700 1 -- 172.21.15.16:0/1336697192 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50c4123a90 con 0x7f50c409f410 2023-09-30T22:35:36.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.676+0000 7f50ba7fc700 1 -- 172.21.15.16:0/1336697192 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f50b4013470 con 0x7f50c409f410 2023-09-30T22:35:36.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.677+0000 7f50ba7fc700 1 -- 172.21.15.16:0/1336697192 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f50b403fb10 con 0x7f50c409f410 2023-09-30T22:35:36.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.677+0000 7f50cbac8700 1 -- 172.21.15.16:0/1336697192 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50c4125a00 con 0x7f50c409f410 2023-09-30T22:35:36.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.682+0000 7f50ba7fc700 1 -- 172.21.15.16:0/1336697192 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f50b4021020 con 0x7f50c409f410 2023-09-30T22:35:36.690 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:35:36.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.831+0000 7f1b9f8e7700 1 Processor -- start 2023-09-30T22:35:36.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.831+0000 7f1b9f8e7700 1 -- start start 2023-09-30T22:35:36.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.832+0000 7f1b9f8e7700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1b9809d6f0 0x7f1b9809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:36.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.832+0000 7f1b9f8e7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1b9809e000 0x7f1b981339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:36.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.832+0000 7f1b9f8e7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1b9809e410 0x7f1b98133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:36.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.832+0000 7f1b9f8e7700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b9806b460 con 0x7f1b9809d6f0 2023-09-30T22:35:36.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.832+0000 7f1b9f8e7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9809b120 con 0x7f1b9809e000 2023-09-30T22:35:36.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.832+0000 7f1b9f8e7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9809a7e0 con 0x7f1b9809e410 2023-09-30T22:35:36.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.832+0000 7f1b9d683700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1b9809d6f0 0x7f1b9809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:36.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.832+0000 7f1b9de84700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1b9809e410 0x7f1b98133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:36.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.832+0000 7f1b9d683700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1b9809d6f0 0x7f1b9809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55202/0 (socket says 172.21.15.16:55202) 2023-09-30T22:35:36.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.832+0000 7f1b9d683700 1 -- 172.21.15.16:0/653280676 learned_addr learned my addr 172.21.15.16:0/653280676 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:36.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.832+0000 7f1b9de84700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1b9809e410 0x7f1b98133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:45512/0 (socket says 172.21.15.16:45512) 2023-09-30T22:35:36.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.832+0000 7f1b9ce82700 1 --2- 172.21.15.16:0/653280676 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1b9809e000 0x7f1b981339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:36.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.832+0000 7f1b9d683700 1 -- 172.21.15.16:0/653280676 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1b9809e410 msgr2=0x7f1b98133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:36.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.832+0000 7f1b9d683700 1 --2- 172.21.15.16:0/653280676 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1b9809e410 0x7f1b98133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:36.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.832+0000 7f1b9d683700 1 -- 172.21.15.16:0/653280676 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1b9809e000 msgr2=0x7f1b981339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:36.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.832+0000 7f1b9d683700 1 --2- 172.21.15.16:0/653280676 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1b9809e000 0x7f1b981339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:36.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.833+0000 7f1b9d683700 1 -- 172.21.15.16:0/653280676 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b9809a500 con 0x7f1b9809d6f0 2023-09-30T22:35:36.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.833+0000 7f1b9de84700 1 --2- 172.21.15.16:0/653280676 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1b9809e410 0x7f1b98133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:36.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.833+0000 7f1b9d683700 1 --2- 172.21.15.16:0/653280676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b9809d6f0 0x7f1b9809dac0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f1b940098f0 tx=0x7f1b94009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53df25636ea55e3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:36.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.833+0000 7f1b8ffff700 1 -- 172.21.15.16:0/653280676 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b94011830 con 0x7f1b9809d6f0 2023-09-30T22:35:36.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.833+0000 7f1b8ffff700 1 -- 172.21.15.16:0/653280676 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b94011990 con 0x7f1b9809d6f0 2023-09-30T22:35:36.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.833+0000 7f1b8ffff700 1 -- 172.21.15.16:0/653280676 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b94011d20 con 0x7f1b9809d6f0 2023-09-30T22:35:36.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.834+0000 7f1b9f8e7700 1 -- 172.21.15.16:0/653280676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b9809d6f0 msgr2=0x7f1b9809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:36.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.834+0000 7f1b9f8e7700 1 --2- 172.21.15.16:0/653280676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b9809d6f0 0x7f1b9809dac0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f1b940098f0 tx=0x7f1b94009c00 comp rx=0 tx=0).stop 2023-09-30T22:35:36.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.834+0000 7f1b9f8e7700 1 -- 172.21.15.16:0/653280676 shutdown_connections 2023-09-30T22:35:36.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.834+0000 7f1b9f8e7700 1 --2- 172.21.15.16:0/653280676 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1b9809e410 0x7f1b98133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:36.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.834+0000 7f1b9f8e7700 1 --2- 172.21.15.16:0/653280676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b9809d6f0 0x7f1b9809dac0 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:36.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.834+0000 7f1b9f8e7700 1 --2- 172.21.15.16:0/653280676 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1b9809e000 0x7f1b981339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:36.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.834+0000 7f1b9f8e7700 1 -- 172.21.15.16:0/653280676 >> 172.21.15.16:0/653280676 conn(0x7f1b98066f50 msgr2=0x7f1b9809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:36.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.834+0000 7f1b9f8e7700 1 -- 172.21.15.16:0/653280676 shutdown_connections 2023-09-30T22:35:36.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.834+0000 7f1b9f8e7700 1 -- 172.21.15.16:0/653280676 wait complete. 2023-09-30T22:35:36.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.835+0000 7f1b9f8e7700 1 Processor -- start 2023-09-30T22:35:36.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.835+0000 7f1b9f8e7700 1 -- start start 2023-09-30T22:35:36.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.835+0000 7f1b9f8e7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1b9809d6f0 0x7f1b981c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:36.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.835+0000 7f1b9f8e7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b9809e000 0x7f1b981c85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:36.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.835+0000 7f1b9f8e7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1b9809e410 0x7f1b981c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:36.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.835+0000 7f1b9f8e7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b981a7190 con 0x7f1b9809e000 2023-09-30T22:35:36.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.835+0000 7f1b9f8e7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b981b9320 con 0x7f1b9809d6f0 2023-09-30T22:35:36.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.835+0000 7f1b9f8e7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1b981a6520 con 0x7f1b9809e410 2023-09-30T22:35:36.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.835+0000 7f1b9d683700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1b9809d6f0 0x7f1b981c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:36.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.835+0000 7f1b9d683700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1b9809d6f0 0x7f1b981c80b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:46276/0 (socket says 172.21.15.16:46276) 2023-09-30T22:35:36.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.835+0000 7f1b9d683700 1 -- 172.21.15.16:0/250645472 learned_addr learned my addr 172.21.15.16:0/250645472 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:36.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.835+0000 7f1b9ce82700 1 --2- 172.21.15.16:0/250645472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b9809e000 0x7f1b981c85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:36.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.836+0000 7f1b9d683700 1 -- 172.21.15.16:0/250645472 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1b9809e410 msgr2=0x7f1b981c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:36.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.836+0000 7f1b9d683700 1 --2- 172.21.15.16:0/250645472 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1b9809e410 0x7f1b981c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:36.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.836+0000 7f1b9d683700 1 -- 172.21.15.16:0/250645472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b9809e000 msgr2=0x7f1b981c85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:36.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.836+0000 7f1b9d683700 1 --2- 172.21.15.16:0/250645472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b9809e000 0x7f1b981c85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:36.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.836+0000 7f1b9d683700 1 -- 172.21.15.16:0/250645472 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b90009a70 con 0x7f1b9809d6f0 2023-09-30T22:35:36.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.836+0000 7f1b9ce82700 1 --2- 172.21.15.16:0/250645472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b9809e000 0x7f1b981c85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:36.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.836+0000 7f1b9d683700 1 --2- 172.21.15.16:0/250645472 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1b9809d6f0 0x7f1b981c80b0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f1b94011db0 tx=0x7f1b94004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:36.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.836+0000 7f1b8dffb700 1 -- 172.21.15.16:0/250645472 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b94004450 con 0x7f1b9809d6f0 2023-09-30T22:35:36.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.836+0000 7f1b8dffb700 1 -- 172.21.15.16:0/250645472 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b940045b0 con 0x7f1b9809d6f0 2023-09-30T22:35:36.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.836+0000 7f1b9f8e7700 1 -- 172.21.15.16:0/250645472 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b94009550 con 0x7f1b9809d6f0 2023-09-30T22:35:36.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.836+0000 7f1b8dffb700 1 -- 172.21.15.16:0/250645472 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b940038a0 con 0x7f1b9809d6f0 2023-09-30T22:35:36.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.836+0000 7f1b9f8e7700 1 -- 172.21.15.16:0/250645472 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b981a7d10 con 0x7f1b9809d6f0 2023-09-30T22:35:36.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.837+0000 7f1b8dffb700 1 -- 172.21.15.16:0/250645472 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1b94021070 con 0x7f1b9809d6f0 2023-09-30T22:35:36.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.837+0000 7f1b8dffb700 1 -- 172.21.15.16:0/250645472 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f1b94040a80 con 0x7f1b9809d6f0 2023-09-30T22:35:36.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.838+0000 7f1b9f8e7700 1 -- 172.21.15.16:0/250645472 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b98074ea0 con 0x7f1b9809d6f0 2023-09-30T22:35:36.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.842+0000 7f1b8dffb700 1 -- 172.21.15.16:0/250645472 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1b94015020 con 0x7f1b9809d6f0 2023-09-30T22:35:36.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.858+0000 7fd5bc824700 1 -- 172.21.15.16:0/3217139320 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd5b80747f0 con 0x7fd5b809d630 2023-09-30T22:35:36.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.858+0000 7fd5aeffd700 1 -- 172.21.15.16:0/3217139320 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd5b0017020 con 0x7fd5b809d630 2023-09-30T22:35:36.860 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:35:36.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.861+0000 7fd5bc824700 1 -- 172.21.15.16:0/3217139320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5b809d630 msgr2=0x7fd5b8133700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:36.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.861+0000 7fd5bc824700 1 --2- 172.21.15.16:0/3217139320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5b809d630 0x7fd5b8133700 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fd5b000c8c0 tx=0x7fd5b000cbd0 comp rx=0 tx=0).stop 2023-09-30T22:35:36.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.861+0000 7fd5bc824700 1 -- 172.21.15.16:0/3217139320 shutdown_connections 2023-09-30T22:35:36.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.861+0000 7fd5bc824700 1 --2- 172.21.15.16:0/3217139320 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd5b8138d80 0x7fd5b8134180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:36.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.861+0000 7fd5bc824700 1 --2- 172.21.15.16:0/3217139320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5b809d630 0x7fd5b8133700 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:36.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.861+0000 7fd5bc824700 1 --2- 172.21.15.16:0/3217139320 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd5b80a0a30 0x7fd5b8133c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:36.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.861+0000 7fd5bc824700 1 -- 172.21.15.16:0/3217139320 >> 172.21.15.16:0/3217139320 conn(0x7fd5b80a3050 msgr2=0x7fd5b809ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:36.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.861+0000 7fd5bc824700 1 -- 172.21.15.16:0/3217139320 shutdown_connections 2023-09-30T22:35:36.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.861+0000 7fd5bc824700 1 -- 172.21.15.16:0/3217139320 wait complete. 2023-09-30T22:35:36.876 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:35:36.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.900+0000 7f50cbac8700 1 -- 172.21.15.16:0/1336697192 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f50c4074ea0 con 0x7f50c409f410 2023-09-30T22:35:36.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.900+0000 7f50ba7fc700 1 -- 172.21.15.16:0/1336697192 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f50b400ce80 con 0x7f50c409f410 2023-09-30T22:35:36.902 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:35:36.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.903+0000 7f50cbac8700 1 -- 172.21.15.16:0/1336697192 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f50c409f410 msgr2=0x7f50c406bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:36.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.903+0000 7f50cbac8700 1 --2- 172.21.15.16:0/1336697192 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f50c409f410 0x7f50c406bdf0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f50b400ec10 tx=0x7f50b400ef20 comp rx=0 tx=0).stop 2023-09-30T22:35:36.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.903+0000 7f50cbac8700 1 -- 172.21.15.16:0/1336697192 shutdown_connections 2023-09-30T22:35:36.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.903+0000 7f50cbac8700 1 --2- 172.21.15.16:0/1336697192 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f50c409e930 0x7f50c406b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:36.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.903+0000 7f50cbac8700 1 --2- 172.21.15.16:0/1336697192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50c4136fe0 0x7f50c406c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:36.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.903+0000 7f50cbac8700 1 --2- 172.21.15.16:0/1336697192 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f50c409f410 0x7f50c406bdf0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:36.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.903+0000 7f50cbac8700 1 -- 172.21.15.16:0/1336697192 >> 172.21.15.16:0/1336697192 conn(0x7f50c40a4210 msgr2=0x7f50c40a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:36.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.903+0000 7f50cbac8700 1 -- 172.21.15.16:0/1336697192 shutdown_connections 2023-09-30T22:35:36.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:36.903+0000 7f50cbac8700 1 -- 172.21.15.16:0/1336697192 wait complete. 2023-09-30T22:35:36.918 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:35:37.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.046+0000 7f1b9f8e7700 1 -- 172.21.15.16:0/250645472 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1b981a7f00 con 0x7f1b9809d6f0 2023-09-30T22:35:37.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.047+0000 7f1b8dffb700 1 -- 172.21.15.16:0/250645472 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1b94019770 con 0x7f1b9809d6f0 2023-09-30T22:35:37.048 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:35:37.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.049+0000 7f1b9f8e7700 1 -- 172.21.15.16:0/250645472 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1b9809d6f0 msgr2=0x7f1b981c80b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:37.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.049+0000 7f1b9f8e7700 1 --2- 172.21.15.16:0/250645472 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1b9809d6f0 0x7f1b981c80b0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f1b94011db0 tx=0x7f1b94004750 comp rx=0 tx=0).stop 2023-09-30T22:35:37.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.049+0000 7f1b9f8e7700 1 -- 172.21.15.16:0/250645472 shutdown_connections 2023-09-30T22:35:37.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.049+0000 7f1b9f8e7700 1 --2- 172.21.15.16:0/250645472 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1b9809d6f0 0x7f1b981c80b0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:37.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.049+0000 7f1b9f8e7700 1 --2- 172.21.15.16:0/250645472 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1b9809e410 0x7f1b981c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:37.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.049+0000 7f1b9f8e7700 1 --2- 172.21.15.16:0/250645472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b9809e000 0x7f1b981c85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:37.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.049+0000 7f1b9f8e7700 1 -- 172.21.15.16:0/250645472 >> 172.21.15.16:0/250645472 conn(0x7f1b98066f50 msgr2=0x7f1b981bbb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:37.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.049+0000 7f1b9f8e7700 1 -- 172.21.15.16:0/250645472 shutdown_connections 2023-09-30T22:35:37.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.049+0000 7f1b9f8e7700 1 -- 172.21.15.16:0/250645472 wait complete. 2023-09-30T22:35:37.064 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:35:37.337 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:35:37.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.483+0000 7fb57ef49700 1 Processor -- start 2023-09-30T22:35:37.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.483+0000 7fb57ef49700 1 -- start start 2023-09-30T22:35:37.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.483+0000 7fb57ef49700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb57809d6f0 0x7fb57809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:37.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.483+0000 7fb57ef49700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb57809e000 0x7fb5781339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:37.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.483+0000 7fb57ef49700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb57809e410 0x7fb578133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:37.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.483+0000 7fb57ef49700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb57806b460 con 0x7fb57809e000 2023-09-30T22:35:37.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.483+0000 7fb57ef49700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb57809b120 con 0x7fb57809e410 2023-09-30T22:35:37.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.483+0000 7fb57ef49700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb57809a7e0 con 0x7fb57809d6f0 2023-09-30T22:35:37.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.484+0000 7fb57d4e6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb57809e410 0x7fb578133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:37.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.484+0000 7fb57cce5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb57809d6f0 0x7fb57809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:37.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.484+0000 7fb577fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb57809e000 0x7fb5781339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:37.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.484+0000 7fb57d4e6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb57809e410 0x7fb578133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47412/0 (socket says 172.21.15.16:47412) 2023-09-30T22:35:37.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.484+0000 7fb57d4e6700 1 -- 172.21.15.16:0/27279355 learned_addr learned my addr 172.21.15.16:0/27279355 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:37.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.484+0000 7fb57d4e6700 1 -- 172.21.15.16:0/27279355 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb57809d6f0 msgr2=0x7fb57809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:37.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.484+0000 7fb57d4e6700 1 --2- 172.21.15.16:0/27279355 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb57809d6f0 0x7fb57809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:37.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.484+0000 7fb57d4e6700 1 -- 172.21.15.16:0/27279355 >> v2:172.21.15.16:3300/0 conn(0x7fb57809e000 msgr2=0x7fb5781339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:37.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.484+0000 7fb57d4e6700 1 --2- 172.21.15.16:0/27279355 >> v2:172.21.15.16:3300/0 conn(0x7fb57809e000 0x7fb5781339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:37.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.484+0000 7fb57d4e6700 1 -- 172.21.15.16:0/27279355 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb57809a500 con 0x7fb57809e410 2023-09-30T22:35:37.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.484+0000 7fb57cce5700 1 --2- 172.21.15.16:0/27279355 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb57809d6f0 0x7fb57809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:37.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.484+0000 7fb577fff700 1 --2- 172.21.15.16:0/27279355 >> v2:172.21.15.16:3300/0 conn(0x7fb57809e000 0x7fb5781339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:37.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.484+0000 7fb57d4e6700 1 --2- 172.21.15.16:0/27279355 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb57809e410 0x7fb578133f30 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fb570009e80 tx=0x7fb57000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a5da4bdea613265 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:37.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.484+0000 7fb5777fe700 1 -- 172.21.15.16:0/27279355 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57000e070 con 0x7fb57809e410 2023-09-30T22:35:37.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.484+0000 7fb5777fe700 1 -- 172.21.15.16:0/27279355 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb57000b8d0 con 0x7fb57809e410 2023-09-30T22:35:37.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.484+0000 7fb5777fe700 1 -- 172.21.15.16:0/27279355 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57000bc80 con 0x7fb57809e410 2023-09-30T22:35:37.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.485+0000 7fb57ef49700 1 -- 172.21.15.16:0/27279355 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb57809e410 msgr2=0x7fb578133f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:37.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.485+0000 7fb57ef49700 1 --2- 172.21.15.16:0/27279355 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb57809e410 0x7fb578133f30 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fb570009e80 tx=0x7fb57000b330 comp rx=0 tx=0).stop 2023-09-30T22:35:37.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.485+0000 7fb57ef49700 1 -- 172.21.15.16:0/27279355 shutdown_connections 2023-09-30T22:35:37.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.485+0000 7fb57ef49700 1 --2- 172.21.15.16:0/27279355 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb57809e410 0x7fb578133f30 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:37.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.485+0000 7fb57ef49700 1 --2- 172.21.15.16:0/27279355 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb57809d6f0 0x7fb57809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:37.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.485+0000 7fb57ef49700 1 --2- 172.21.15.16:0/27279355 >> v2:172.21.15.16:3300/0 conn(0x7fb57809e000 0x7fb5781339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:37.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.485+0000 7fb57ef49700 1 -- 172.21.15.16:0/27279355 >> 172.21.15.16:0/27279355 conn(0x7fb578066f50 msgr2=0x7fb57809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:37.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.485+0000 7fb57ef49700 1 -- 172.21.15.16:0/27279355 shutdown_connections 2023-09-30T22:35:37.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.485+0000 7fb57ef49700 1 -- 172.21.15.16:0/27279355 wait complete. 2023-09-30T22:35:37.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.486+0000 7fb57ef49700 1 Processor -- start 2023-09-30T22:35:37.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.486+0000 7fb57ef49700 1 -- start start 2023-09-30T22:35:37.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.487+0000 7fb57ef49700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb57809d6f0 0x7fb5781a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:37.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.487+0000 7fb57ef49700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb57809e000 0x7fb5781a5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:37.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.487+0000 7fb57ef49700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb57809e410 0x7fb5781c8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:37.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.487+0000 7fb57ef49700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5781a8250 con 0x7fb57809e410 2023-09-30T22:35:37.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.487+0000 7fb57ef49700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5781a8430 con 0x7fb57809e000 2023-09-30T22:35:37.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.487+0000 7fb57ef49700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb5781b92a0 con 0x7fb57809d6f0 2023-09-30T22:35:37.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.487+0000 7fb577fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb57809e000 0x7fb5781a5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:37.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.487+0000 7fb57d4e6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb57809e410 0x7fb5781c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:37.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.487+0000 7fb577fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb57809e000 0x7fb5781a5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47416/0 (socket says 172.21.15.16:47416) 2023-09-30T22:35:37.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.487+0000 7fb57d4e6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb57809e410 0x7fb5781c8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40668/0 (socket says 172.21.15.16:40668) 2023-09-30T22:35:37.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.487+0000 7fb577fff700 1 -- 172.21.15.16:0/2198163822 learned_addr learned my addr 172.21.15.16:0/2198163822 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:37.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.487+0000 7fb57cce5700 1 --2- 172.21.15.16:0/2198163822 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb57809d6f0 0x7fb5781a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:37.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.487+0000 7fb577fff700 1 -- 172.21.15.16:0/2198163822 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb57809d6f0 msgr2=0x7fb5781a52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:37.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.487+0000 7fb577fff700 1 --2- 172.21.15.16:0/2198163822 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb57809d6f0 0x7fb5781a52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:37.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.487+0000 7fb577fff700 1 -- 172.21.15.16:0/2198163822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb57809e410 msgr2=0x7fb5781c8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:37.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.487+0000 7fb577fff700 1 --2- 172.21.15.16:0/2198163822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb57809e410 0x7fb5781c8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:37.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.487+0000 7fb577fff700 1 -- 172.21.15.16:0/2198163822 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb56c0094a0 con 0x7fb57809e000 2023-09-30T22:35:37.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.487+0000 7fb57d4e6700 1 --2- 172.21.15.16:0/2198163822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb57809e410 0x7fb5781c8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:37.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.487+0000 7fb577fff700 1 --2- 172.21.15.16:0/2198163822 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb57809e000 0x7fb5781a5820 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fb56400ec10 tx=0x7fb56400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:37.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.488+0000 7fb5757fa700 1 -- 172.21.15.16:0/2198163822 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb56400c850 con 0x7fb57809e000 2023-09-30T22:35:37.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.488+0000 7fb5757fa700 1 -- 172.21.15.16:0/2198163822 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb56400c9b0 con 0x7fb57809e000 2023-09-30T22:35:37.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.488+0000 7fb57ef49700 1 -- 172.21.15.16:0/2198163822 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5640094a0 con 0x7fb57809e000 2023-09-30T22:35:37.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.488+0000 7fb5757fa700 1 -- 172.21.15.16:0/2198163822 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb56400cd20 con 0x7fb57809e000 2023-09-30T22:35:37.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.488+0000 7fb57ef49700 1 -- 172.21.15.16:0/2198163822 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb570009b30 con 0x7fb57809e000 2023-09-30T22:35:37.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.489+0000 7fb5757fa700 1 -- 172.21.15.16:0/2198163822 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb564013470 con 0x7fb57809e000 2023-09-30T22:35:37.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.489+0000 7fb57ef49700 1 -- 172.21.15.16:0/2198163822 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb57809c710 con 0x7fb57809e000 2023-09-30T22:35:37.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.489+0000 7fb5757fa700 1 -- 172.21.15.16:0/2198163822 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fb56403f760 con 0x7fb57809e000 2023-09-30T22:35:37.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.494+0000 7fb5757fa700 1 -- 172.21.15.16:0/2198163822 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb564013720 con 0x7fb57809e000 2023-09-30T22:35:37.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.700+0000 7fb57ef49700 1 -- 172.21.15.16:0/2198163822 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb578074ea0 con 0x7fb57809e000 2023-09-30T22:35:37.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.700+0000 7fb5757fa700 1 -- 172.21.15.16:0/2198163822 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb56401bba0 con 0x7fb57809e000 2023-09-30T22:35:37.702 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:35:37.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.702+0000 7fb57ef49700 1 -- 172.21.15.16:0/2198163822 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb57809e000 msgr2=0x7fb5781a5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:37.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.702+0000 7fb57ef49700 1 --2- 172.21.15.16:0/2198163822 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb57809e000 0x7fb5781a5820 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fb56400ec10 tx=0x7fb56400ef20 comp rx=0 tx=0).stop 2023-09-30T22:35:37.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.703+0000 7fb57ef49700 1 -- 172.21.15.16:0/2198163822 shutdown_connections 2023-09-30T22:35:37.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.703+0000 7fb57ef49700 1 --2- 172.21.15.16:0/2198163822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb57809e410 0x7fb5781c8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:37.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.703+0000 7fb57ef49700 1 --2- 172.21.15.16:0/2198163822 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb57809d6f0 0x7fb5781a52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:37.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.703+0000 7fb57ef49700 1 --2- 172.21.15.16:0/2198163822 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb57809e000 0x7fb5781a5820 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:37.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.703+0000 7fb57ef49700 1 -- 172.21.15.16:0/2198163822 >> 172.21.15.16:0/2198163822 conn(0x7fb578066f50 msgr2=0x7fb5781bba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:37.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.703+0000 7fb57ef49700 1 -- 172.21.15.16:0/2198163822 shutdown_connections 2023-09-30T22:35:37.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:37.703+0000 7fb57ef49700 1 -- 172.21.15.16:0/2198163822 wait complete. 2023-09-30T22:35:37.718 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:35:37.877 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:35:37.919 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:35:38.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.024+0000 7fb019023700 1 Processor -- start 2023-09-30T22:35:38.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.024+0000 7fb019023700 1 -- start start 2023-09-30T22:35:38.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.024+0000 7fb019023700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb0140a0a30 0x7fb0140a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:38.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.024+0000 7fb019023700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb014138d80 0x7fb01413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:38.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.024+0000 7fb019023700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0140a2a70 con 0x7fb01409d630 2023-09-30T22:35:38.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.024+0000 7fb019023700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0140a01d0 con 0x7fb0140a0a30 2023-09-30T22:35:38.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.024+0000 7fb019023700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb01409e2b0 con 0x7fb014138d80 2023-09-30T22:35:38.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.025+0000 7fb012d9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb0140a0a30 0x7fb0140a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:38.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.025+0000 7fb01359e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb014138d80 0x7fb01413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:38.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.025+0000 7fb01259c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fb01409d630 0x7fb0140a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40494/0 (socket says 172.21.15.16:40494) 2023-09-30T22:35:38.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.025+0000 7fb01259c700 1 -- 172.21.15.16:0/3768629463 learned_addr learned my addr 172.21.15.16:0/3768629463 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:38.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.025+0000 7fb012d9d700 1 -- 172.21.15.16:0/3768629463 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb014138d80 msgr2=0x7fb01413b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:38.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.025+0000 7fb012d9d700 1 --2- 172.21.15.16:0/3768629463 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb014138d80 0x7fb01413b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:38.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.025+0000 7fb011d9b700 1 -- 172.21.15.16:0/3768629463 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2166759714 0 0) 0x7fb0140a2a70 con 0x7fb01409d630 2023-09-30T22:35:38.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.025+0000 7fb012d9d700 1 -- 172.21.15.16:0/3768629463 >> v1:172.21.15.16:6789/0 conn(0x7fb01409d630 legacy=0x7fb0140a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:38.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.025+0000 7fb012d9d700 1 -- 172.21.15.16:0/3768629463 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb01409dc50 con 0x7fb0140a0a30 2023-09-30T22:35:38.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.025+0000 7fb01359e700 1 --2- 172.21.15.16:0/3768629463 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb014138d80 0x7fb01413b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:38.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.025+0000 7fb012d9d700 1 --2- 172.21.15.16:0/3768629463 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb0140a0a30 0x7fb0140a0e00 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fb004009870 tx=0x7fb004009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f92994804d07870 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:38.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.025+0000 7fb011d9b700 1 -- 172.21.15.16:0/3768629463 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb004011830 con 0x7fb0140a0a30 2023-09-30T22:35:38.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.025+0000 7fb011d9b700 1 -- 172.21.15.16:0/3768629463 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb004011990 con 0x7fb0140a0a30 2023-09-30T22:35:38.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.025+0000 7fb011d9b700 1 -- 172.21.15.16:0/3768629463 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb004011d20 con 0x7fb0140a0a30 2023-09-30T22:35:38.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.026+0000 7fb019023700 1 -- 172.21.15.16:0/3768629463 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb0140a0a30 msgr2=0x7fb0140a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:38.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.026+0000 7fb019023700 1 --2- 172.21.15.16:0/3768629463 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb0140a0a30 0x7fb0140a0e00 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fb004009870 tx=0x7fb004009b80 comp rx=0 tx=0).stop 2023-09-30T22:35:38.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.026+0000 7fb019023700 1 -- 172.21.15.16:0/3768629463 shutdown_connections 2023-09-30T22:35:38.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.026+0000 7fb019023700 1 --2- 172.21.15.16:0/3768629463 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb014138d80 0x7fb01413b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:38.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.026+0000 7fb019023700 1 --2- 172.21.15.16:0/3768629463 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb0140a0a30 0x7fb0140a0e00 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:38.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.026+0000 7fb019023700 1 -- 172.21.15.16:0/3768629463 >> 172.21.15.16:0/3768629463 conn(0x7fb0140a3050 msgr2=0x7fb0140a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:38.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.026+0000 7fb019023700 1 -- 172.21.15.16:0/3768629463 shutdown_connections 2023-09-30T22:35:38.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.026+0000 7fb019023700 1 -- 172.21.15.16:0/3768629463 wait complete. 2023-09-30T22:35:38.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.028+0000 7fb019023700 1 Processor -- start 2023-09-30T22:35:38.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.028+0000 7fb019023700 1 -- start start 2023-09-30T22:35:38.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.029+0000 7fb019023700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb01409d630 0x7fb014133700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:38.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.029+0000 7fb019023700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0140a0a30 0x7fb014133c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:38.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.029+0000 7fb019023700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb014138d80 0x7fb014134180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:38.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.029+0000 7fb019023700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb014137d30 con 0x7fb0140a0a30 2023-09-30T22:35:38.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.029+0000 7fb019023700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0140a4bf0 con 0x7fb014138d80 2023-09-30T22:35:38.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.029+0000 7fb019023700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb0140a0410 con 0x7fb01409d630 2023-09-30T22:35:38.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.029+0000 7fb01359e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb014138d80 0x7fb014134180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:38.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.029+0000 7fb01259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0140a0a30 0x7fb014133c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:38.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.029+0000 7fb01359e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb014138d80 0x7fb014134180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47428/0 (socket says 172.21.15.16:47428) 2023-09-30T22:35:38.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.029+0000 7fb01359e700 1 -- 172.21.15.16:0/1195173847 learned_addr learned my addr 172.21.15.16:0/1195173847 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:38.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.029+0000 7fb012d9d700 1 --2- 172.21.15.16:0/1195173847 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb01409d630 0x7fb014133700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:38.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.030+0000 7fb01359e700 1 -- 172.21.15.16:0/1195173847 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb01409d630 msgr2=0x7fb014133700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:38.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.030+0000 7fb01359e700 1 --2- 172.21.15.16:0/1195173847 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb01409d630 0x7fb014133700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:38.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.030+0000 7fb01359e700 1 -- 172.21.15.16:0/1195173847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0140a0a30 msgr2=0x7fb014133c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:38.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.030+0000 7fb01359e700 1 --2- 172.21.15.16:0/1195173847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0140a0a30 0x7fb014133c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:38.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.030+0000 7fb01359e700 1 -- 172.21.15.16:0/1195173847 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb00c009a70 con 0x7fb014138d80 2023-09-30T22:35:38.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.030+0000 7fb01259c700 1 --2- 172.21.15.16:0/1195173847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0140a0a30 0x7fb014133c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:38.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.030+0000 7fb01359e700 1 --2- 172.21.15.16:0/1195173847 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb014138d80 0x7fb014134180 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fb00c00efc0 tx=0x7fb00c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:38.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.030+0000 7faffb7fe700 1 -- 172.21.15.16:0/1195173847 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00c00cc90 con 0x7fb014138d80 2023-09-30T22:35:38.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.030+0000 7faffb7fe700 1 -- 172.21.15.16:0/1195173847 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb00c00f040 con 0x7fb014138d80 2023-09-30T22:35:38.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.030+0000 7faffb7fe700 1 -- 172.21.15.16:0/1195173847 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00c014480 con 0x7fb014138d80 2023-09-30T22:35:38.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.030+0000 7fb019023700 1 -- 172.21.15.16:0/1195173847 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0040094a0 con 0x7fb014138d80 2023-09-30T22:35:38.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.030+0000 7fb019023700 1 -- 172.21.15.16:0/1195173847 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb014136500 con 0x7fb014138d80 2023-09-30T22:35:38.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.031+0000 7faffb7fe700 1 -- 172.21.15.16:0/1195173847 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb00c0145e0 con 0x7fb014138d80 2023-09-30T22:35:38.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.031+0000 7faffb7fe700 1 -- 172.21.15.16:0/1195173847 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fb00c03ff90 con 0x7fb014138d80 2023-09-30T22:35:38.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.031+0000 7fb019023700 1 -- 172.21.15.16:0/1195173847 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0140747f0 con 0x7fb014138d80 2023-09-30T22:35:38.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.036+0000 7faffb7fe700 1 -- 172.21.15.16:0/1195173847 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb00c018070 con 0x7fb014138d80 2023-09-30T22:35:38.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.063+0000 7f8682077700 1 Processor -- start 2023-09-30T22:35:38.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.063+0000 7f8682077700 1 -- start start 2023-09-30T22:35:38.065 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:35:38.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.064+0000 7f8682077700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f867c09d630 0x7f867c0a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:38.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.064+0000 7f8682077700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f867c138d80 0x7f867c13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:38.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.064+0000 7f8682077700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f867c0a2a70 con 0x7f867c0a0a30 2023-09-30T22:35:38.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.064+0000 7f8682077700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f867c0a01d0 con 0x7f867c138d80 2023-09-30T22:35:38.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.064+0000 7f8682077700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f867c09e2b0 con 0x7f867c09d630 2023-09-30T22:35:38.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.064+0000 7f867affd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f867c09d630 0x7f867c0a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:38.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.064+0000 7f867b7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f867c0a0a30 0x7f867c0a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40506/0 (socket says 172.21.15.16:40506) 2023-09-30T22:35:38.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.064+0000 7f867b7fe700 1 -- 172.21.15.16:0/2225583531 learned_addr learned my addr 172.21.15.16:0/2225583531 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:38.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.064+0000 7f867bfff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f867c138d80 0x7f867c13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:38.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.064+0000 7f867affd700 1 -- 172.21.15.16:0/2225583531 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f867c138d80 msgr2=0x7f867c13b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:38.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.064+0000 7f867affd700 1 --2- 172.21.15.16:0/2225583531 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f867c138d80 0x7f867c13b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:38.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.064+0000 7f867affd700 1 -- 172.21.15.16:0/2225583531 >> v1:172.21.15.16:6789/0 conn(0x7f867c0a0a30 legacy=0x7f867c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:38.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.064+0000 7f867affd700 1 -- 172.21.15.16:0/2225583531 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f867c09dc50 con 0x7f867c09d630 2023-09-30T22:35:38.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.064+0000 7f867a7fc700 1 -- 172.21.15.16:0/2225583531 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 217803381 0 0) 0x7f867c0a2a70 con 0x7f867c0a0a30 2023-09-30T22:35:38.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.064+0000 7f867bfff700 1 --2- 172.21.15.16:0/2225583531 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f867c138d80 0x7f867c13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:38.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.064+0000 7f867affd700 1 --2- 172.21.15.16:0/2225583531 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f867c09d630 0x7f867c0a1510 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f8664009870 tx=0x7f8664009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81c245dc6bea163c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:38.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.065+0000 7f867a7fc700 1 -- 172.21.15.16:0/2225583531 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8664011830 con 0x7f867c09d630 2023-09-30T22:35:38.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.065+0000 7f867a7fc700 1 -- 172.21.15.16:0/2225583531 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8664011990 con 0x7f867c09d630 2023-09-30T22:35:38.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.065+0000 7f867a7fc700 1 -- 172.21.15.16:0/2225583531 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8664011d20 con 0x7f867c09d630 2023-09-30T22:35:38.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.065+0000 7f8682077700 1 -- 172.21.15.16:0/2225583531 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f867c09d630 msgr2=0x7f867c0a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:38.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.065+0000 7f8682077700 1 --2- 172.21.15.16:0/2225583531 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f867c09d630 0x7f867c0a1510 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f8664009870 tx=0x7f8664009b80 comp rx=0 tx=0).stop 2023-09-30T22:35:38.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.065+0000 7f8682077700 1 -- 172.21.15.16:0/2225583531 shutdown_connections 2023-09-30T22:35:38.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.065+0000 7f8682077700 1 --2- 172.21.15.16:0/2225583531 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f867c138d80 0x7f867c13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:38.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.065+0000 7f8682077700 1 --2- 172.21.15.16:0/2225583531 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f867c09d630 0x7f867c0a1510 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:38.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.065+0000 7f8682077700 1 -- 172.21.15.16:0/2225583531 >> 172.21.15.16:0/2225583531 conn(0x7f867c0a3050 msgr2=0x7f867c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:38.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.065+0000 7f8682077700 1 -- 172.21.15.16:0/2225583531 shutdown_connections 2023-09-30T22:35:38.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.065+0000 7f8682077700 1 -- 172.21.15.16:0/2225583531 wait complete. 2023-09-30T22:35:38.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.067+0000 7f8682077700 1 Processor -- start 2023-09-30T22:35:38.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.067+0000 7f8682077700 1 -- start start 2023-09-30T22:35:38.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.068+0000 7f8682077700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f867c09d630 0x7f867c1337c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:38.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.068+0000 7f8682077700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f867c0a0a30 0x7f867c133d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:38.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.068+0000 7f8682077700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f867c138d80 0x7f867c134240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:38.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.068+0000 7f8682077700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f867c134c90 con 0x7f867c09d630 2023-09-30T22:35:38.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.068+0000 7f8682077700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f867c137310 con 0x7f867c138d80 2023-09-30T22:35:38.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.068+0000 7f8682077700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f867c136c60 con 0x7f867c0a0a30 2023-09-30T22:35:38.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.068+0000 7f867affd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f867c0a0a30 0x7f867c133d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:38.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.068+0000 7f867b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f867c09d630 0x7f867c1337c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:38.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.068+0000 7f867affd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f867c0a0a30 0x7f867c133d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:50510/0 (socket says 172.21.15.16:50510) 2023-09-30T22:35:38.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.068+0000 7f867affd700 1 -- 172.21.15.16:0/3225529206 learned_addr learned my addr 172.21.15.16:0/3225529206 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:38.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.068+0000 7f867bfff700 1 --2- 172.21.15.16:0/3225529206 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f867c138d80 0x7f867c134240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:38.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.068+0000 7f867b7fe700 1 -- 172.21.15.16:0/3225529206 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f867c0a0a30 msgr2=0x7f867c133d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:38.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.068+0000 7f867b7fe700 1 --2- 172.21.15.16:0/3225529206 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f867c0a0a30 0x7f867c133d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:38.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.068+0000 7f867b7fe700 1 -- 172.21.15.16:0/3225529206 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f867c138d80 msgr2=0x7f867c134240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:38.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.068+0000 7f867b7fe700 1 --2- 172.21.15.16:0/3225529206 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f867c138d80 0x7f867c134240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:38.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.068+0000 7f867b7fe700 1 -- 172.21.15.16:0/3225529206 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f866c009a70 con 0x7f867c09d630 2023-09-30T22:35:38.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.068+0000 7f867bfff700 1 --2- 172.21.15.16:0/3225529206 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f867c138d80 0x7f867c134240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:38.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.068+0000 7f867affd700 1 --2- 172.21.15.16:0/3225529206 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f867c0a0a30 0x7f867c133d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:38.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.069+0000 7f867b7fe700 1 --2- 172.21.15.16:0/3225529206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f867c09d630 0x7f867c1337c0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f8674001c40 tx=0x7f8674014330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:38.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.069+0000 7f8673fff700 1 -- 172.21.15.16:0/3225529206 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8674014980 con 0x7f867c09d630 2023-09-30T22:35:38.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.069+0000 7f8673fff700 1 -- 172.21.15.16:0/3225529206 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8674014ae0 con 0x7f867c09d630 2023-09-30T22:35:38.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.069+0000 7f8682077700 1 -- 172.21.15.16:0/3225529206 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86640094a0 con 0x7f867c09d630 2023-09-30T22:35:38.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.069+0000 7f8673fff700 1 -- 172.21.15.16:0/3225529206 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8674014e90 con 0x7f867c09d630 2023-09-30T22:35:38.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.069+0000 7f8682077700 1 -- 172.21.15.16:0/3225529206 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f867c136f20 con 0x7f867c09d630 2023-09-30T22:35:38.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.070+0000 7f8682077700 1 -- 172.21.15.16:0/3225529206 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f867c0747f0 con 0x7f867c09d630 2023-09-30T22:35:38.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.071+0000 7f8673fff700 1 -- 172.21.15.16:0/3225529206 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8674007170 con 0x7f867c09d630 2023-09-30T22:35:38.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.071+0000 7f8673fff700 1 -- 172.21.15.16:0/3225529206 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f867403ea30 con 0x7f867c09d630 2023-09-30T22:35:38.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.075+0000 7f8673fff700 1 -- 172.21.15.16:0/3225529206 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8674004870 con 0x7f867c09d630 2023-09-30T22:35:38.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.205+0000 7f5da7ab4700 1 Processor -- start 2023-09-30T22:35:38.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.205+0000 7f5da7ab4700 1 -- start start 2023-09-30T22:35:38.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.205+0000 7f5da7ab4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5da009e930 0x7f5da009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:38.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.205+0000 7f5da7ab4700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5da009f240 0x7f5da01317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:38.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.205+0000 7f5da7ab4700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5da0070940 con 0x7f5da009f650 2023-09-30T22:35:38.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.205+0000 7f5da7ab4700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5da006fd60 con 0x7f5da009e930 2023-09-30T22:35:38.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.205+0000 7f5da7ab4700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5da006fa80 con 0x7f5da009f240 2023-09-30T22:35:38.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.206+0000 7f5da5850700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5da009e930 0x7f5da009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:38.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.206+0000 7f5da504f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5da009f240 0x7f5da01317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:38.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.206+0000 7f5da504f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5da009f240 0x7f5da01317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47456/0 (socket says 172.21.15.16:47456) 2023-09-30T22:35:38.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.206+0000 7f5da5850700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5da009e930 0x7f5da009ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40692/0 (socket says 172.21.15.16:40692) 2023-09-30T22:35:38.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.206+0000 7f5da504f700 1 -- 172.21.15.16:0/331765811 learned_addr learned my addr 172.21.15.16:0/331765811 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:38.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.206+0000 7f5da504f700 1 -- 172.21.15.16:0/331765811 >> v2:172.21.15.16:3300/0 conn(0x7f5da009e930 msgr2=0x7f5da009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:38.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.206+0000 7f5da504f700 1 --2- 172.21.15.16:0/331765811 >> v2:172.21.15.16:3300/0 conn(0x7f5da009e930 0x7f5da009ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:38.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.206+0000 7f5da504f700 1 -- 172.21.15.16:0/331765811 >> v1:172.21.15.16:6789/0 conn(0x7f5da009f650 legacy=0x7f5da0131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:38.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.206+0000 7f5da504f700 1 -- 172.21.15.16:0/331765811 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5da009fb60 con 0x7f5da009f240 2023-09-30T22:35:38.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.206+0000 7f5da484e700 1 -- 172.21.15.16:0/331765811 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4100282151 0 0) 0x7f5da0070940 con 0x7f5da009f650 2023-09-30T22:35:38.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.206+0000 7f5da5850700 1 --2- 172.21.15.16:0/331765811 >> v2:172.21.15.16:3300/0 conn(0x7f5da009e930 0x7f5da009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:38.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.206+0000 7f5da504f700 1 --2- 172.21.15.16:0/331765811 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5da009f240 0x7f5da01317f0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f5d900098f0 tx=0x7f5d90009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f77d7f5d42777987 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:38.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.206+0000 7f5da484e700 1 -- 172.21.15.16:0/331765811 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d90011830 con 0x7f5da009f240 2023-09-30T22:35:38.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.206+0000 7f5da484e700 1 -- 172.21.15.16:0/331765811 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d90011990 con 0x7f5da009f240 2023-09-30T22:35:38.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.207+0000 7f5da484e700 1 -- 172.21.15.16:0/331765811 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d90011d20 con 0x7f5da009f240 2023-09-30T22:35:38.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.207+0000 7f5da7ab4700 1 -- 172.21.15.16:0/331765811 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5da009f240 msgr2=0x7f5da01317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:38.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.207+0000 7f5da7ab4700 1 --2- 172.21.15.16:0/331765811 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5da009f240 0x7f5da01317f0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f5d900098f0 tx=0x7f5d90009c00 comp rx=0 tx=0).stop 2023-09-30T22:35:38.217 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.207+0000 7f5da7ab4700 1 -- 172.21.15.16:0/331765811 shutdown_connections 2023-09-30T22:35:38.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.207+0000 7f5da7ab4700 1 --2- 172.21.15.16:0/331765811 >> v2:172.21.15.16:3300/0 conn(0x7f5da009e930 0x7f5da009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:38.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.207+0000 7f5da7ab4700 1 --2- 172.21.15.16:0/331765811 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5da009f240 0x7f5da01317f0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:38.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.207+0000 7f5da7ab4700 1 -- 172.21.15.16:0/331765811 >> 172.21.15.16:0/331765811 conn(0x7f5da00a4210 msgr2=0x7f5da00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:38.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.208+0000 7f5da7ab4700 1 -- 172.21.15.16:0/331765811 shutdown_connections 2023-09-30T22:35:38.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.208+0000 7f5da7ab4700 1 -- 172.21.15.16:0/331765811 wait complete. 2023-09-30T22:35:38.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.209+0000 7f5da7ab4700 1 Processor -- start 2023-09-30T22:35:38.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.209+0000 7f5da7ab4700 1 -- start start 2023-09-30T22:35:38.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.210+0000 7f5da7ab4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5da009e930 0x7f5da01c63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:38.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.210+0000 7f5da7ab4700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5da009f240 0x7f5da01c6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:38.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.210+0000 7f5da7ab4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5da009f650 0x7f5da01c2840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:38.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.210+0000 7f5da7ab4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5da01a7320 con 0x7f5da009f650 2023-09-30T22:35:38.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.210+0000 7f5da7ab4700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5da01a6540 con 0x7f5da009f240 2023-09-30T22:35:38.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.210+0000 7f5da7ab4700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5da01a6920 con 0x7f5da009e930 2023-09-30T22:35:38.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.210+0000 7f5da504f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5da009f240 0x7f5da01c6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:38.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.210+0000 7f5da504f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5da009f240 0x7f5da01c6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47458/0 (socket says 172.21.15.16:47458) 2023-09-30T22:35:38.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.210+0000 7f5da504f700 1 -- 172.21.15.16:0/2105220243 learned_addr learned my addr 172.21.15.16:0/2105220243 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:38.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.210+0000 7f5da5850700 1 --2- 172.21.15.16:0/2105220243 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5da009e930 0x7f5da01c63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:38.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.210+0000 7f5da6051700 1 --2- 172.21.15.16:0/2105220243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5da009f650 0x7f5da01c2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:38.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.210+0000 7f5da5850700 1 -- 172.21.15.16:0/2105220243 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5da009f240 msgr2=0x7f5da01c6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:38.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.210+0000 7f5da5850700 1 --2- 172.21.15.16:0/2105220243 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5da009f240 0x7f5da01c6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:38.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.210+0000 7f5da5850700 1 -- 172.21.15.16:0/2105220243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5da009f650 msgr2=0x7f5da01c2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:38.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.210+0000 7f5da5850700 1 --2- 172.21.15.16:0/2105220243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5da009f650 0x7f5da01c2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:38.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.210+0000 7f5da5850700 1 -- 172.21.15.16:0/2105220243 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d9c0094a0 con 0x7f5da009e930 2023-09-30T22:35:38.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.211+0000 7f5da6051700 1 --2- 172.21.15.16:0/2105220243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5da009f650 0x7f5da01c2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:38.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.211+0000 7f5da5850700 1 --2- 172.21.15.16:0/2105220243 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5da009e930 0x7f5da01c63d0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c010cf0 tx=0x7f5d9c00b140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:38.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.211+0000 7f5d967fc700 1 -- 172.21.15.16:0/2105220243 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c015070 con 0x7f5da009e930 2023-09-30T22:35:38.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.211+0000 7f5d967fc700 1 -- 172.21.15.16:0/2105220243 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d9c00b9d0 con 0x7f5da009e930 2023-09-30T22:35:38.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.211+0000 7f5d967fc700 1 -- 172.21.15.16:0/2105220243 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9c00bd60 con 0x7f5da009e930 2023-09-30T22:35:38.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.211+0000 7f5da7ab4700 1 -- 172.21.15.16:0/2105220243 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d90009550 con 0x7f5da009e930 2023-09-30T22:35:38.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.211+0000 7f5da7ab4700 1 -- 172.21.15.16:0/2105220243 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5da00a0190 con 0x7f5da009e930 2023-09-30T22:35:38.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.212+0000 7f5d967fc700 1 -- 172.21.15.16:0/2105220243 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5d9c019410 con 0x7f5da009e930 2023-09-30T22:35:38.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.212+0000 7f5d967fc700 1 -- 172.21.15.16:0/2105220243 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f5d9c043080 con 0x7f5da009e930 2023-09-30T22:35:38.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.212+0000 7f5da7ab4700 1 -- 172.21.15.16:0/2105220243 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5da0074ea0 con 0x7f5da009e930 2023-09-30T22:35:38.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.217+0000 7f5d967fc700 1 -- 172.21.15.16:0/2105220243 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5d9c01e020 con 0x7f5da009e930 2023-09-30T22:35:38.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.246+0000 7fb019023700 1 -- 172.21.15.16:0/1195173847 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb014072980 con 0x7fb014138d80 2023-09-30T22:35:38.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.247+0000 7faffb7fe700 1 -- 172.21.15.16:0/1195173847 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb00c011680 con 0x7fb014138d80 2023-09-30T22:35:38.249 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:35:38.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.249+0000 7fb019023700 1 -- 172.21.15.16:0/1195173847 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb014138d80 msgr2=0x7fb014134180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:38.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.249+0000 7fb019023700 1 --2- 172.21.15.16:0/1195173847 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb014138d80 0x7fb014134180 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fb00c00efc0 tx=0x7fb00c00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:35:38.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.249+0000 7fb019023700 1 -- 172.21.15.16:0/1195173847 shutdown_connections 2023-09-30T22:35:38.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.249+0000 7fb019023700 1 --2- 172.21.15.16:0/1195173847 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb014138d80 0x7fb014134180 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:38.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.249+0000 7fb019023700 1 --2- 172.21.15.16:0/1195173847 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb01409d630 0x7fb014133700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:38.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.249+0000 7fb019023700 1 --2- 172.21.15.16:0/1195173847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0140a0a30 0x7fb014133c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:38.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.249+0000 7fb019023700 1 -- 172.21.15.16:0/1195173847 >> 172.21.15.16:0/1195173847 conn(0x7fb0140a3050 msgr2=0x7fb01413add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:38.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.249+0000 7fb019023700 1 -- 172.21.15.16:0/1195173847 shutdown_connections 2023-09-30T22:35:38.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.249+0000 7fb019023700 1 -- 172.21.15.16:0/1195173847 wait complete. 2023-09-30T22:35:38.268 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:35:38.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.287+0000 7f8682077700 1 -- 172.21.15.16:0/3225529206 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f867c072980 con 0x7f867c09d630 2023-09-30T22:35:38.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.287+0000 7f8673fff700 1 -- 172.21.15.16:0/3225529206 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f867401d350 con 0x7f867c09d630 2023-09-30T22:35:38.288 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:35:38.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.289+0000 7f8682077700 1 -- 172.21.15.16:0/3225529206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f867c09d630 msgr2=0x7f867c1337c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:38.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.289+0000 7f8682077700 1 --2- 172.21.15.16:0/3225529206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f867c09d630 0x7f867c1337c0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f8674001c40 tx=0x7f8674014330 comp rx=0 tx=0).stop 2023-09-30T22:35:38.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.289+0000 7f8682077700 1 -- 172.21.15.16:0/3225529206 shutdown_connections 2023-09-30T22:35:38.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.289+0000 7f8682077700 1 --2- 172.21.15.16:0/3225529206 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f867c138d80 0x7f867c134240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:38.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.289+0000 7f8682077700 1 --2- 172.21.15.16:0/3225529206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f867c09d630 0x7f867c1337c0 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:38.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.289+0000 7f8682077700 1 --2- 172.21.15.16:0/3225529206 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f867c0a0a30 0x7f867c133d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:38.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.289+0000 7f8682077700 1 -- 172.21.15.16:0/3225529206 >> 172.21.15.16:0/3225529206 conn(0x7f867c0a3050 msgr2=0x7f867c09ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:38.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.289+0000 7f8682077700 1 -- 172.21.15.16:0/3225529206 shutdown_connections 2023-09-30T22:35:38.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.290+0000 7f8682077700 1 -- 172.21.15.16:0/3225529206 wait complete. 2023-09-30T22:35:38.305 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:35:38.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.434+0000 7f5da7ab4700 1 -- 172.21.15.16:0/2105220243 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5da006b250 con 0x7f5da009e930 2023-09-30T22:35:38.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.434+0000 7f5d967fc700 1 -- 172.21.15.16:0/2105220243 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5d9c0208b0 con 0x7f5da009e930 2023-09-30T22:35:38.436 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:35:38.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.436+0000 7f5da7ab4700 1 -- 172.21.15.16:0/2105220243 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5da009e930 msgr2=0x7f5da01c63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:38.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.436+0000 7f5da7ab4700 1 --2- 172.21.15.16:0/2105220243 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5da009e930 0x7f5da01c63d0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c010cf0 tx=0x7f5d9c00b140 comp rx=0 tx=0).stop 2023-09-30T22:35:38.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.437+0000 7f5da7ab4700 1 -- 172.21.15.16:0/2105220243 shutdown_connections 2023-09-30T22:35:38.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.437+0000 7f5da7ab4700 1 --2- 172.21.15.16:0/2105220243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5da009f650 0x7f5da01c2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:38.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.437+0000 7f5da7ab4700 1 --2- 172.21.15.16:0/2105220243 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5da009e930 0x7f5da01c63d0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:38.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.437+0000 7f5da7ab4700 1 --2- 172.21.15.16:0/2105220243 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5da009f240 0x7f5da01c6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:38.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.437+0000 7f5da7ab4700 1 -- 172.21.15.16:0/2105220243 >> 172.21.15.16:0/2105220243 conn(0x7f5da00a4210 msgr2=0x7f5da01b89c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:38.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.437+0000 7f5da7ab4700 1 -- 172.21.15.16:0/2105220243 shutdown_connections 2023-09-30T22:35:38.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.437+0000 7f5da7ab4700 1 -- 172.21.15.16:0/2105220243 wait complete. 2023-09-30T22:35:38.452 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:35:38.718 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:35:38.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.862+0000 7f80532b3700 1 Processor -- start 2023-09-30T22:35:38.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.863+0000 7f80532b3700 1 -- start start 2023-09-30T22:35:38.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.863+0000 7f80532b3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f804c0a1510 0x7f804c138d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:38.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.863+0000 7f80532b3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f804c139250 0x7f804c13b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:38.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.863+0000 7f80532b3700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f804c09edd0 con 0x7f804c0a0a30 2023-09-30T22:35:38.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.863+0000 7f80532b3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f804c0a18e0 con 0x7f804c0a1510 2023-09-30T22:35:38.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.863+0000 7f80532b3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f804c09e1f0 con 0x7f804c139250 2023-09-30T22:35:38.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.863+0000 7f805084e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f804c0a1510 0x7f804c138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:38.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.863+0000 7f8051850700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f804c139250 0x7f804c13b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:38.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.863+0000 7f805104f700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f804c0a0a30 0x7f804c0a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40524/0 (socket says 172.21.15.16:40524) 2023-09-30T22:35:38.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.863+0000 7f805104f700 1 -- 172.21.15.16:0/2961573137 learned_addr learned my addr 172.21.15.16:0/2961573137 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:38.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.864+0000 7f8051850700 1 -- 172.21.15.16:0/2961573137 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f804c0a1510 msgr2=0x7f804c138d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:38.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.864+0000 7f8051850700 1 --2- 172.21.15.16:0/2961573137 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f804c0a1510 0x7f804c138d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:38.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.864+0000 7f8043fff700 1 -- 172.21.15.16:0/2961573137 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2882553863 0 0) 0x7f804c09edd0 con 0x7f804c0a0a30 2023-09-30T22:35:38.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.864+0000 7f8051850700 1 -- 172.21.15.16:0/2961573137 >> v1:172.21.15.16:6789/0 conn(0x7f804c0a0a30 legacy=0x7f804c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:38.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.864+0000 7f8051850700 1 -- 172.21.15.16:0/2961573137 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f804c09dc50 con 0x7f804c139250 2023-09-30T22:35:38.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.864+0000 7f805084e700 1 --2- 172.21.15.16:0/2961573137 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f804c0a1510 0x7f804c138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:38.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.864+0000 7f8051850700 1 --2- 172.21.15.16:0/2961573137 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f804c139250 0x7f804c13b630 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f8044009dc0 tx=0x7f804400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f82130deb59e0e1e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:38.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.864+0000 7f8043fff700 1 -- 172.21.15.16:0/2961573137 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804400e070 con 0x7f804c139250 2023-09-30T22:35:38.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.864+0000 7f8043fff700 1 -- 172.21.15.16:0/2961573137 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f804400bb20 con 0x7f804c139250 2023-09-30T22:35:38.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.864+0000 7f8043fff700 1 -- 172.21.15.16:0/2961573137 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8044012430 con 0x7f804c139250 2023-09-30T22:35:38.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.865+0000 7f80532b3700 1 -- 172.21.15.16:0/2961573137 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f804c139250 msgr2=0x7f804c13b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:38.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.865+0000 7f80532b3700 1 --2- 172.21.15.16:0/2961573137 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f804c139250 0x7f804c13b630 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f8044009dc0 tx=0x7f804400b330 comp rx=0 tx=0).stop 2023-09-30T22:35:38.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.865+0000 7f80532b3700 1 -- 172.21.15.16:0/2961573137 shutdown_connections 2023-09-30T22:35:38.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.865+0000 7f80532b3700 1 --2- 172.21.15.16:0/2961573137 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f804c139250 0x7f804c13b630 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:38.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.865+0000 7f80532b3700 1 --2- 172.21.15.16:0/2961573137 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f804c0a1510 0x7f804c138d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:38.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.865+0000 7f80532b3700 1 -- 172.21.15.16:0/2961573137 >> 172.21.15.16:0/2961573137 conn(0x7f804c0a6550 msgr2=0x7f804c0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:38.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.865+0000 7f80532b3700 1 -- 172.21.15.16:0/2961573137 shutdown_connections 2023-09-30T22:35:38.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.865+0000 7f80532b3700 1 -- 172.21.15.16:0/2961573137 wait complete. 2023-09-30T22:35:38.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.866+0000 7f80532b3700 1 Processor -- start 2023-09-30T22:35:38.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.866+0000 7f80532b3700 1 -- start start 2023-09-30T22:35:38.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.866+0000 7f80532b3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f804c0a0a30 0x7f804c133680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:38.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.866+0000 7f80532b3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f804c0a1510 0x7f804c133bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:38.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.866+0000 7f80532b3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f804c139250 0x7f804c11e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:38.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.866+0000 7f80532b3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f804c11fd50 con 0x7f804c139250 2023-09-30T22:35:38.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.866+0000 7f80532b3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f804c09c4f0 con 0x7f804c0a1510 2023-09-30T22:35:38.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.866+0000 7f80532b3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f804c0a0220 con 0x7f804c0a0a30 2023-09-30T22:35:38.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.866+0000 7f805104f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f804c0a0a30 0x7f804c133680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:38.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.866+0000 7f8051850700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f804c139250 0x7f804c11e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:38.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.866+0000 7f805084e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f804c0a1510 0x7f804c133bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:38.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.866+0000 7f805104f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f804c0a0a30 0x7f804c133680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:50536/0 (socket says 172.21.15.16:50536) 2023-09-30T22:35:38.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.866+0000 7f805104f700 1 -- 172.21.15.16:0/121894076 learned_addr learned my addr 172.21.15.16:0/121894076 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:38.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.866+0000 7f8051850700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f804c139250 0x7f804c11e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40706/0 (socket says 172.21.15.16:40706) 2023-09-30T22:35:38.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.867+0000 7f805104f700 1 -- 172.21.15.16:0/121894076 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f804c0a1510 msgr2=0x7f804c133bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:38.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.867+0000 7f805104f700 1 --2- 172.21.15.16:0/121894076 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f804c0a1510 0x7f804c133bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:38.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.867+0000 7f805104f700 1 -- 172.21.15.16:0/121894076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f804c139250 msgr2=0x7f804c11e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:38.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.867+0000 7f805104f700 1 --2- 172.21.15.16:0/121894076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f804c139250 0x7f804c11e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:38.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.867+0000 7f805104f700 1 -- 172.21.15.16:0/121894076 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f803c0094a0 con 0x7f804c0a0a30 2023-09-30T22:35:38.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.867+0000 7f805084e700 1 --2- 172.21.15.16:0/121894076 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f804c0a1510 0x7f804c133bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:38.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.867+0000 7f805104f700 1 --2- 172.21.15.16:0/121894076 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f804c0a0a30 0x7f804c133680 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f804800c520 tx=0x7f804800c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:38.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.867+0000 7f8041ffb700 1 -- 172.21.15.16:0/121894076 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8048004c80 con 0x7f804c0a0a30 2023-09-30T22:35:38.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.867+0000 7f8041ffb700 1 -- 172.21.15.16:0/121894076 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8048004e00 con 0x7f804c0a0a30 2023-09-30T22:35:38.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.867+0000 7f80532b3700 1 -- 172.21.15.16:0/121894076 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8044009a70 con 0x7f804c0a0a30 2023-09-30T22:35:38.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.867+0000 7f8041ffb700 1 -- 172.21.15.16:0/121894076 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8048007310 con 0x7f804c0a0a30 2023-09-30T22:35:38.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.867+0000 7f80532b3700 1 -- 172.21.15.16:0/121894076 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f804c1220d0 con 0x7f804c0a0a30 2023-09-30T22:35:38.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.868+0000 7f8041ffb700 1 -- 172.21.15.16:0/121894076 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8048007470 con 0x7f804c0a0a30 2023-09-30T22:35:38.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.868+0000 7f8041ffb700 1 -- 172.21.15.16:0/121894076 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f804803f430 con 0x7f804c0a0a30 2023-09-30T22:35:38.873 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.868+0000 7f80532b3700 1 -- 172.21.15.16:0/121894076 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f804c121f30 con 0x7f804c0a0a30 2023-09-30T22:35:38.874 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:38.873+0000 7f8041ffb700 1 -- 172.21.15.16:0/121894076 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f804c121f30 con 0x7f804c0a0a30 2023-09-30T22:35:39.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.077+0000 7f80532b3700 1 -- 172.21.15.16:0/121894076 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f804c121f30 con 0x7f804c0a0a30 2023-09-30T22:35:39.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.077+0000 7f8041ffb700 1 -- 172.21.15.16:0/121894076 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f804c121f30 con 0x7f804c0a0a30 2023-09-30T22:35:39.079 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:35:39.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.079+0000 7f80532b3700 1 -- 172.21.15.16:0/121894076 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f804c0a0a30 msgr2=0x7f804c133680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:39.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.079+0000 7f80532b3700 1 --2- 172.21.15.16:0/121894076 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f804c0a0a30 0x7f804c133680 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f804800c520 tx=0x7f804800c830 comp rx=0 tx=0).stop 2023-09-30T22:35:39.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.079+0000 7f80532b3700 1 -- 172.21.15.16:0/121894076 shutdown_connections 2023-09-30T22:35:39.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.079+0000 7f80532b3700 1 --2- 172.21.15.16:0/121894076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f804c139250 0x7f804c11e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:39.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.079+0000 7f80532b3700 1 --2- 172.21.15.16:0/121894076 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f804c0a0a30 0x7f804c133680 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:39.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.079+0000 7f80532b3700 1 --2- 172.21.15.16:0/121894076 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f804c0a1510 0x7f804c133bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:39.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.079+0000 7f80532b3700 1 -- 172.21.15.16:0/121894076 >> 172.21.15.16:0/121894076 conn(0x7f804c0a6550 msgr2=0x7f804c0a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:39.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.079+0000 7f80532b3700 1 -- 172.21.15.16:0/121894076 shutdown_connections 2023-09-30T22:35:39.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.079+0000 7f80532b3700 1 -- 172.21.15.16:0/121894076 wait complete. 2023-09-30T22:35:39.094 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:35:39.269 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:35:39.306 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:35:39.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.415+0000 7f38a8cba700 1 Processor -- start 2023-09-30T22:35:39.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.415+0000 7f38a8cba700 1 -- start start 2023-09-30T22:35:39.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.416+0000 7f38a8cba700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f38a4133420 0x7f38a41337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:39.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.416+0000 7f38a8cba700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38a4134260 0x7f38a41306a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:39.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.416+0000 7f38a8cba700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38a406b990 con 0x7f38a4133e30 2023-09-30T22:35:39.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.416+0000 7f38a8cba700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38a40a0590 con 0x7f38a4133420 2023-09-30T22:35:39.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.416+0000 7f38a8cba700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38a409fb80 con 0x7f38a4134260 2023-09-30T22:35:39.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.416+0000 7f38a259c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f38a4133420 0x7f38a41337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:39.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.416+0000 7f38a259c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f38a4133420 0x7f38a41337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40720/0 (socket says 172.21.15.16:40720) 2023-09-30T22:35:39.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.416+0000 7f38a2d9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38a4134260 0x7f38a41306a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:39.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.416+0000 7f38a259c700 1 -- 172.21.15.16:0/2817806749 learned_addr learned my addr 172.21.15.16:0/2817806749 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:39.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.416+0000 7f38a2d9d700 1 -- 172.21.15.16:0/2817806749 >> v2:172.21.15.16:3300/0 conn(0x7f38a4133420 msgr2=0x7f38a41337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:39.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.416+0000 7f38a2d9d700 1 --2- 172.21.15.16:0/2817806749 >> v2:172.21.15.16:3300/0 conn(0x7f38a4133420 0x7f38a41337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:39.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.416+0000 7f38a2d9d700 1 -- 172.21.15.16:0/2817806749 >> v1:172.21.15.16:6789/0 conn(0x7f38a4133e30 legacy=0x7f38a412ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:39.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.416+0000 7f38a159a700 1 -- 172.21.15.16:0/2817806749 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3697493931 0 0) 0x7f38a406b990 con 0x7f38a4133e30 2023-09-30T22:35:39.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.416+0000 7f38a2d9d700 1 -- 172.21.15.16:0/2817806749 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a40a0030 con 0x7f38a4134260 2023-09-30T22:35:39.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.416+0000 7f38a259c700 1 --2- 172.21.15.16:0/2817806749 >> v2:172.21.15.16:3300/0 conn(0x7f38a4133420 0x7f38a41337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:39.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.417+0000 7f38a2d9d700 1 --2- 172.21.15.16:0/2817806749 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38a4134260 0x7f38a41306a0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f389c009840 tx=0x7f389c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=750b8098b8233788 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:39.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.417+0000 7f38a159a700 1 -- 172.21.15.16:0/2817806749 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c011830 con 0x7f38a4134260 2023-09-30T22:35:39.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.417+0000 7f38a159a700 1 -- 172.21.15.16:0/2817806749 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f389c011990 con 0x7f38a4134260 2023-09-30T22:35:39.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.417+0000 7f38a159a700 1 -- 172.21.15.16:0/2817806749 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c011d20 con 0x7f38a4134260 2023-09-30T22:35:39.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.417+0000 7f38a8cba700 1 -- 172.21.15.16:0/2817806749 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38a4134260 msgr2=0x7f38a41306a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:39.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.418+0000 7f38a8cba700 1 --2- 172.21.15.16:0/2817806749 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38a4134260 0x7f38a41306a0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f389c009840 tx=0x7f389c009b50 comp rx=0 tx=0).stop 2023-09-30T22:35:39.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.418+0000 7f38a8cba700 1 -- 172.21.15.16:0/2817806749 shutdown_connections 2023-09-30T22:35:39.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.418+0000 7f38a8cba700 1 --2- 172.21.15.16:0/2817806749 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38a4134260 0x7f38a41306a0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:39.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.418+0000 7f38a8cba700 1 --2- 172.21.15.16:0/2817806749 >> v2:172.21.15.16:3300/0 conn(0x7f38a4133420 0x7f38a41337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:39.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.418+0000 7f38a8cba700 1 -- 172.21.15.16:0/2817806749 >> 172.21.15.16:0/2817806749 conn(0x7f38a40a4210 msgr2=0x7f38a40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:39.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.418+0000 7f38a8cba700 1 -- 172.21.15.16:0/2817806749 shutdown_connections 2023-09-30T22:35:39.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.418+0000 7f38a8cba700 1 -- 172.21.15.16:0/2817806749 wait complete. 2023-09-30T22:35:39.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.419+0000 7f38a8cba700 1 Processor -- start 2023-09-30T22:35:39.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.420+0000 7f38a8cba700 1 -- start start 2023-09-30T22:35:39.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.420+0000 7f38a8cba700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f38a4133420 0x7f38a409c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:39.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.420+0000 7f38a8cba700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38a4133e30 0x7f38a409c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:39.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.420+0000 7f38a8cba700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38a4134260 0x7f38a409cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:39.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.420+0000 7f38a8cba700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a4120600 con 0x7f38a4133e30 2023-09-30T22:35:39.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.420+0000 7f38a8cba700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38a411fb60 con 0x7f38a4134260 2023-09-30T22:35:39.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.420+0000 7f38a8cba700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f38a4131590 con 0x7f38a4133420 2023-09-30T22:35:39.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.420+0000 7f38a2d9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38a4134260 0x7f38a409cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:39.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.420+0000 7f38a1d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38a4133e30 0x7f38a409c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:39.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.420+0000 7f38a2d9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38a4134260 0x7f38a409cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47494/0 (socket says 172.21.15.16:47494) 2023-09-30T22:35:39.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.420+0000 7f38a2d9d700 1 -- 172.21.15.16:0/2469231631 learned_addr learned my addr 172.21.15.16:0/2469231631 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:39.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.421+0000 7f38a259c700 1 --2- 172.21.15.16:0/2469231631 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f38a4133420 0x7f38a409c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:39.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.421+0000 7f38a2d9d700 1 -- 172.21.15.16:0/2469231631 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f38a4133420 msgr2=0x7f38a409c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:39.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.421+0000 7f38a2d9d700 1 --2- 172.21.15.16:0/2469231631 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f38a4133420 0x7f38a409c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:39.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.421+0000 7f38a2d9d700 1 -- 172.21.15.16:0/2469231631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38a4133e30 msgr2=0x7f38a409c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:39.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.421+0000 7f38a2d9d700 1 --2- 172.21.15.16:0/2469231631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38a4133e30 0x7f38a409c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:39.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.421+0000 7f38a2d9d700 1 -- 172.21.15.16:0/2469231631 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3894009b30 con 0x7f38a4134260 2023-09-30T22:35:39.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.421+0000 7f38a1d9b700 1 --2- 172.21.15.16:0/2469231631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38a4133e30 0x7f38a409c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:39.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.421+0000 7f38a2d9d700 1 --2- 172.21.15.16:0/2469231631 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38a4134260 0x7f38a409cf00 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f389c009fd0 tx=0x7f389c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:39.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.421+0000 7f3892ffd700 1 -- 172.21.15.16:0/2469231631 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c004400 con 0x7f38a4134260 2023-09-30T22:35:39.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.421+0000 7f3892ffd700 1 -- 172.21.15.16:0/2469231631 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f389c01f070 con 0x7f38a4134260 2023-09-30T22:35:39.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.421+0000 7f38a8cba700 1 -- 172.21.15.16:0/2469231631 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f389c0094a0 con 0x7f38a4134260 2023-09-30T22:35:39.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.421+0000 7f3892ffd700 1 -- 172.21.15.16:0/2469231631 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c0038e0 con 0x7f38a4134260 2023-09-30T22:35:39.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.421+0000 7f38a8cba700 1 -- 172.21.15.16:0/2469231631 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38a4120170 con 0x7f38a4134260 2023-09-30T22:35:39.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.422+0000 7f38a8cba700 1 -- 172.21.15.16:0/2469231631 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38a4074ea0 con 0x7f38a4134260 2023-09-30T22:35:39.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.423+0000 7f3892ffd700 1 -- 172.21.15.16:0/2469231631 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f389c021030 con 0x7f38a4134260 2023-09-30T22:35:39.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.423+0000 7f3892ffd700 1 -- 172.21.15.16:0/2469231631 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f389c040dc0 con 0x7f38a4134260 2023-09-30T22:35:39.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.428+0000 7f3892ffd700 1 -- 172.21.15.16:0/2469231631 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f389c023790 con 0x7f38a4134260 2023-09-30T22:35:39.452 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:35:39.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.456+0000 7fd88de97700 1 Processor -- start 2023-09-30T22:35:39.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.456+0000 7fd88de97700 1 -- start start 2023-09-30T22:35:39.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.456+0000 7fd88de97700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd88809d6f0 0x7fd88809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:39.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.456+0000 7fd88de97700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd88809e000 0x7fd8881339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:39.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.456+0000 7fd88de97700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd88809e410 0x7fd888133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:39.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.456+0000 7fd88de97700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd88806b460 con 0x7fd88809e000 2023-09-30T22:35:39.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.456+0000 7fd88de97700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd88809b120 con 0x7fd88809d6f0 2023-09-30T22:35:39.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.456+0000 7fd88de97700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd88809a7e0 con 0x7fd88809e410 2023-09-30T22:35:39.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.456+0000 7fd887fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd88809e410 0x7fd888133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:39.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.457+0000 7fd886ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd88809e000 0x7fd8881339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:39.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.457+0000 7fd887fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd88809e410 0x7fd888133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:50554/0 (socket says 172.21.15.16:50554) 2023-09-30T22:35:39.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.457+0000 7fd887fff700 1 -- 172.21.15.16:0/1021096048 learned_addr learned my addr 172.21.15.16:0/1021096048 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:39.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.457+0000 7fd8877fe700 1 --2- 172.21.15.16:0/1021096048 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd88809d6f0 0x7fd88809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:39.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.457+0000 7fd886ffd700 1 -- 172.21.15.16:0/1021096048 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd88809e410 msgr2=0x7fd888133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:39.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.457+0000 7fd886ffd700 1 --2- 172.21.15.16:0/1021096048 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd88809e410 0x7fd888133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:39.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.457+0000 7fd886ffd700 1 -- 172.21.15.16:0/1021096048 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd88809d6f0 msgr2=0x7fd88809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:39.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.457+0000 7fd886ffd700 1 --2- 172.21.15.16:0/1021096048 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd88809d6f0 0x7fd88809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:39.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.457+0000 7fd886ffd700 1 -- 172.21.15.16:0/1021096048 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd88809a500 con 0x7fd88809e000 2023-09-30T22:35:39.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.457+0000 7fd8877fe700 1 --2- 172.21.15.16:0/1021096048 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd88809d6f0 0x7fd88809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:35:39.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.457+0000 7fd886ffd700 1 --2- 172.21.15.16:0/1021096048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd88809e000 0x7fd8881339f0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fd87c009840 tx=0x7fd87c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12dae61e7708754f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:39.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.457+0000 7fd8867fc700 1 -- 172.21.15.16:0/1021096048 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87c011830 con 0x7fd88809e000 2023-09-30T22:35:39.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.457+0000 7fd8867fc700 1 -- 172.21.15.16:0/1021096048 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd87c011990 con 0x7fd88809e000 2023-09-30T22:35:39.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.457+0000 7fd8867fc700 1 -- 172.21.15.16:0/1021096048 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87c011d20 con 0x7fd88809e000 2023-09-30T22:35:39.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.458+0000 7fd88de97700 1 -- 172.21.15.16:0/1021096048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd88809e000 msgr2=0x7fd8881339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:39.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.458+0000 7fd88de97700 1 --2- 172.21.15.16:0/1021096048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd88809e000 0x7fd8881339f0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fd87c009840 tx=0x7fd87c009b50 comp rx=0 tx=0).stop 2023-09-30T22:35:39.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.458+0000 7fd88de97700 1 -- 172.21.15.16:0/1021096048 shutdown_connections 2023-09-30T22:35:39.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.458+0000 7fd88de97700 1 --2- 172.21.15.16:0/1021096048 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd88809d6f0 0x7fd88809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:39.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.458+0000 7fd88de97700 1 --2- 172.21.15.16:0/1021096048 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd88809e410 0x7fd888133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:39.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.458+0000 7fd88de97700 1 --2- 172.21.15.16:0/1021096048 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd88809e000 0x7fd8881339f0 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:39.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.458+0000 7fd88de97700 1 -- 172.21.15.16:0/1021096048 >> 172.21.15.16:0/1021096048 conn(0x7fd888066f50 msgr2=0x7fd88809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:39.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.458+0000 7fd88de97700 1 -- 172.21.15.16:0/1021096048 shutdown_connections 2023-09-30T22:35:39.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.458+0000 7fd88de97700 1 -- 172.21.15.16:0/1021096048 wait complete. 2023-09-30T22:35:39.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.459+0000 7fd88de97700 1 Processor -- start 2023-09-30T22:35:39.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.459+0000 7fd88de97700 1 -- start start 2023-09-30T22:35:39.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.460+0000 7fd88de97700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd88809d6f0 0x7fd8881a5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:39.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.460+0000 7fd88de97700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd88809e000 0x7fd8881a64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:39.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.460+0000 7fd88de97700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd88809e410 0x7fd8881a69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:39.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.460+0000 7fd88de97700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8881a7b70 con 0x7fd88809e000 2023-09-30T22:35:39.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.460+0000 7fd88de97700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8881a8dc0 con 0x7fd88809d6f0 2023-09-30T22:35:39.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.460+0000 7fd88de97700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd8881a8150 con 0x7fd88809e410 2023-09-30T22:35:39.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.460+0000 7fd887fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd88809e410 0x7fd8881a69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:39.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.460+0000 7fd886ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd88809e000 0x7fd8881a64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:39.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.460+0000 7fd887fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd88809e410 0x7fd8881a69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:50568/0 (socket says 172.21.15.16:50568) 2023-09-30T22:35:39.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.460+0000 7fd887fff700 1 -- 172.21.15.16:0/1685907576 learned_addr learned my addr 172.21.15.16:0/1685907576 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:39.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.460+0000 7fd8877fe700 1 --2- 172.21.15.16:0/1685907576 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd88809d6f0 0x7fd8881a5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:39.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.460+0000 7fd887fff700 1 -- 172.21.15.16:0/1685907576 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd88809d6f0 msgr2=0x7fd8881a5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:39.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.461+0000 7fd887fff700 1 --2- 172.21.15.16:0/1685907576 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd88809d6f0 0x7fd8881a5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:39.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.461+0000 7fd887fff700 1 -- 172.21.15.16:0/1685907576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd88809e000 msgr2=0x7fd8881a64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:39.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.461+0000 7fd887fff700 1 --2- 172.21.15.16:0/1685907576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd88809e000 0x7fd8881a64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:39.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.461+0000 7fd887fff700 1 -- 172.21.15.16:0/1685907576 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd87c0094a0 con 0x7fd88809e410 2023-09-30T22:35:39.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.461+0000 7fd887fff700 1 --2- 172.21.15.16:0/1685907576 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd88809e410 0x7fd8881a69e0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fd88000efc0 tx=0x7fd88000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:39.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.461+0000 7fd86ffff700 1 -- 172.21.15.16:0/1685907576 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd880003c20 con 0x7fd88809e410 2023-09-30T22:35:39.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.461+0000 7fd86ffff700 1 -- 172.21.15.16:0/1685907576 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd88000f040 con 0x7fd88809e410 2023-09-30T22:35:39.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.461+0000 7fd88de97700 1 -- 172.21.15.16:0/1685907576 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8881a97a0 con 0x7fd88809e410 2023-09-30T22:35:39.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.461+0000 7fd86ffff700 1 -- 172.21.15.16:0/1685907576 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd880009e60 con 0x7fd88809e410 2023-09-30T22:35:39.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.461+0000 7fd88de97700 1 -- 172.21.15.16:0/1685907576 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd88809c710 con 0x7fd88809e410 2023-09-30T22:35:39.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.462+0000 7fd86ffff700 1 -- 172.21.15.16:0/1685907576 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd88000c880 con 0x7fd88809e410 2023-09-30T22:35:39.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.462+0000 7fd88de97700 1 -- 172.21.15.16:0/1685907576 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd888074ea0 con 0x7fd88809e410 2023-09-30T22:35:39.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.462+0000 7fd86ffff700 1 -- 172.21.15.16:0/1685907576 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd88003f8c0 con 0x7fd88809e410 2023-09-30T22:35:39.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.467+0000 7fd86ffff700 1 -- 172.21.15.16:0/1685907576 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd880013bd0 con 0x7fd88809e410 2023-09-30T22:35:39.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.599+0000 7f8b993ae700 1 Processor -- start 2023-09-30T22:35:39.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.599+0000 7f8b993ae700 1 -- start start 2023-09-30T22:35:39.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.600+0000 7f8b993ae700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8b9409c650 0x7f8b9409ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:39.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.600+0000 7f8b993ae700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8b940a0090 0x7f8b9409cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:39.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.600+0000 7f8b993ae700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8b94131970 0x7f8b94131df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:39.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.600+0000 7f8b993ae700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b9409ba70 con 0x7f8b9409c650 2023-09-30T22:35:39.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.600+0000 7f8b993ae700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9409d4a0 con 0x7f8b94131970 2023-09-30T22:35:39.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.600+0000 7f8b993ae700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9409e8f0 con 0x7f8b940a0090 2023-09-30T22:35:39.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.600+0000 7f8b92ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8b9409c650 0x7f8b9409ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:39.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.600+0000 7f8b927fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8b940a0090 0x7f8b9409cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:39.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.600+0000 7f8b92ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8b9409c650 0x7f8b9409ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40740/0 (socket says 172.21.15.16:40740) 2023-09-30T22:35:39.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.600+0000 7f8b92ffd700 1 -- 172.21.15.16:0/2577253443 learned_addr learned my addr 172.21.15.16:0/2577253443 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:39.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.600+0000 7f8b937fe700 1 --2- 172.21.15.16:0/2577253443 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8b94131970 0x7f8b94131df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:39.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.600+0000 7f8b927fc700 1 -- 172.21.15.16:0/2577253443 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8b94131970 msgr2=0x7f8b94131df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:39.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.600+0000 7f8b927fc700 1 --2- 172.21.15.16:0/2577253443 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8b94131970 0x7f8b94131df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:39.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.600+0000 7f8b927fc700 1 -- 172.21.15.16:0/2577253443 >> v2:172.21.15.16:3300/0 conn(0x7f8b9409c650 msgr2=0x7f8b9409ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:39.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.600+0000 7f8b927fc700 1 --2- 172.21.15.16:0/2577253443 >> v2:172.21.15.16:3300/0 conn(0x7f8b9409c650 0x7f8b9409ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:39.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.600+0000 7f8b927fc700 1 -- 172.21.15.16:0/2577253443 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b940a0760 con 0x7f8b940a0090 2023-09-30T22:35:39.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.600+0000 7f8b92ffd700 1 --2- 172.21.15.16:0/2577253443 >> v2:172.21.15.16:3300/0 conn(0x7f8b9409c650 0x7f8b9409ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:39.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.600+0000 7f8b937fe700 1 --2- 172.21.15.16:0/2577253443 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8b94131970 0x7f8b94131df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:39.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.601+0000 7f8b927fc700 1 --2- 172.21.15.16:0/2577253443 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8b940a0090 0x7f8b9409cf60 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f8b8800dc60 tx=0x7f8b8800df70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27bbe21e210562ba server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:39.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.601+0000 7f8b91ffb700 1 -- 172.21.15.16:0/2577253443 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b88012070 con 0x7f8b940a0090 2023-09-30T22:35:39.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.601+0000 7f8b91ffb700 1 -- 172.21.15.16:0/2577253443 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b8800d2e0 con 0x7f8b940a0090 2023-09-30T22:35:39.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.601+0000 7f8b91ffb700 1 -- 172.21.15.16:0/2577253443 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8800f900 con 0x7f8b940a0090 2023-09-30T22:35:39.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.601+0000 7f8b993ae700 1 -- 172.21.15.16:0/2577253443 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8b940a0090 msgr2=0x7f8b9409cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:39.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.601+0000 7f8b993ae700 1 --2- 172.21.15.16:0/2577253443 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8b940a0090 0x7f8b9409cf60 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f8b8800dc60 tx=0x7f8b8800df70 comp rx=0 tx=0).stop 2023-09-30T22:35:39.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.601+0000 7f8b993ae700 1 -- 172.21.15.16:0/2577253443 shutdown_connections 2023-09-30T22:35:39.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.601+0000 7f8b993ae700 1 --2- 172.21.15.16:0/2577253443 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8b94131970 0x7f8b94131df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:39.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.601+0000 7f8b993ae700 1 --2- 172.21.15.16:0/2577253443 >> v2:172.21.15.16:3300/0 conn(0x7f8b9409c650 0x7f8b9409ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:39.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.601+0000 7f8b993ae700 1 --2- 172.21.15.16:0/2577253443 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8b940a0090 0x7f8b9409cf60 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:39.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.601+0000 7f8b993ae700 1 -- 172.21.15.16:0/2577253443 >> 172.21.15.16:0/2577253443 conn(0x7f8b940a3f70 msgr2=0x7f8b940a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:39.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.601+0000 7f8b993ae700 1 -- 172.21.15.16:0/2577253443 shutdown_connections 2023-09-30T22:35:39.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.602+0000 7f8b993ae700 1 -- 172.21.15.16:0/2577253443 wait complete. 2023-09-30T22:35:39.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.603+0000 7f8b993ae700 1 Processor -- start 2023-09-30T22:35:39.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.603+0000 7f8b993ae700 1 -- start start 2023-09-30T22:35:39.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.603+0000 7f8b993ae700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8b9409c650 0x7f8b9411a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:39.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.603+0000 7f8b993ae700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b940a0090 0x7f8b9411ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:39.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.603+0000 7f8b993ae700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8b94131970 0x7f8b9411b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:39.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.603+0000 7f8b993ae700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9411e2a0 con 0x7f8b940a0090 2023-09-30T22:35:39.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.603+0000 7f8b993ae700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9411b9d0 con 0x7f8b94131970 2023-09-30T22:35:39.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.603+0000 7f8b993ae700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9411bff0 con 0x7f8b9409c650 2023-09-30T22:35:39.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.603+0000 7f8b92ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8b9409c650 0x7f8b9411a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:39.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.603+0000 7f8b92ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8b9409c650 0x7f8b9411a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:50574/0 (socket says 172.21.15.16:50574) 2023-09-30T22:35:39.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.603+0000 7f8b92ffd700 1 -- 172.21.15.16:0/1953045625 learned_addr learned my addr 172.21.15.16:0/1953045625 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:39.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.603+0000 7f8b927fc700 1 --2- 172.21.15.16:0/1953045625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b940a0090 0x7f8b9411ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:39.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.603+0000 7f8b937fe700 1 --2- 172.21.15.16:0/1953045625 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8b94131970 0x7f8b9411b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:39.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.604+0000 7f8b92ffd700 1 -- 172.21.15.16:0/1953045625 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8b94131970 msgr2=0x7f8b9411b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:39.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.604+0000 7f8b92ffd700 1 --2- 172.21.15.16:0/1953045625 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8b94131970 0x7f8b9411b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:39.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.604+0000 7f8b92ffd700 1 -- 172.21.15.16:0/1953045625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b940a0090 msgr2=0x7f8b9411ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:39.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.604+0000 7f8b92ffd700 1 --2- 172.21.15.16:0/1953045625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b940a0090 0x7f8b9411ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:39.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.604+0000 7f8b92ffd700 1 -- 172.21.15.16:0/1953045625 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b84009550 con 0x7f8b9409c650 2023-09-30T22:35:39.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.604+0000 7f8b92ffd700 1 --2- 172.21.15.16:0/1953045625 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8b9409c650 0x7f8b9411a620 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f8b8400ec10 tx=0x7f8b8400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:39.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.604+0000 7f8b7b7fe700 1 -- 172.21.15.16:0/1953045625 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8400c850 con 0x7f8b9409c650 2023-09-30T22:35:39.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.604+0000 7f8b7b7fe700 1 -- 172.21.15.16:0/1953045625 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b8400c9b0 con 0x7f8b9409c650 2023-09-30T22:35:39.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.604+0000 7f8b993ae700 1 -- 172.21.15.16:0/1953045625 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b8c009a70 con 0x7f8b9409c650 2023-09-30T22:35:39.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.604+0000 7f8b7b7fe700 1 -- 172.21.15.16:0/1953045625 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8400cd20 con 0x7f8b9409c650 2023-09-30T22:35:39.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.604+0000 7f8b993ae700 1 -- 172.21.15.16:0/1953045625 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b8800d940 con 0x7f8b9409c650 2023-09-30T22:35:39.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.605+0000 7f8b7b7fe700 1 -- 172.21.15.16:0/1953045625 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8b84013470 con 0x7f8b9409c650 2023-09-30T22:35:39.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.605+0000 7f8b7b7fe700 1 -- 172.21.15.16:0/1953045625 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f8b8403fb00 con 0x7f8b9409c650 2023-09-30T22:35:39.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.606+0000 7f8b993ae700 1 -- 172.21.15.16:0/1953045625 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b9409d980 con 0x7f8b9409c650 2023-09-30T22:35:39.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.610+0000 7f8b7b7fe700 1 -- 172.21.15.16:0/1953045625 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8b84013720 con 0x7f8b9409c650 2023-09-30T22:35:39.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.638+0000 7f38a8cba700 1 -- 172.21.15.16:0/2469231631 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f38a406b250 con 0x7f38a4134260 2023-09-30T22:35:39.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.639+0000 7f3892ffd700 1 -- 172.21.15.16:0/2469231631 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f389c015030 con 0x7f38a4134260 2023-09-30T22:35:39.640 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:35:39.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.642+0000 7f38a8cba700 1 -- 172.21.15.16:0/2469231631 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38a4134260 msgr2=0x7f38a409cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:39.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.642+0000 7f38a8cba700 1 --2- 172.21.15.16:0/2469231631 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38a4134260 0x7f38a409cf00 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f389c009fd0 tx=0x7f389c004750 comp rx=0 tx=0).stop 2023-09-30T22:35:39.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.642+0000 7f38a8cba700 1 -- 172.21.15.16:0/2469231631 shutdown_connections 2023-09-30T22:35:39.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.642+0000 7f38a8cba700 1 --2- 172.21.15.16:0/2469231631 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f38a4134260 0x7f38a409cf00 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:39.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.642+0000 7f38a8cba700 1 --2- 172.21.15.16:0/2469231631 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f38a4133420 0x7f38a409c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:39.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.642+0000 7f38a8cba700 1 --2- 172.21.15.16:0/2469231631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38a4133e30 0x7f38a409c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:39.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.642+0000 7f38a8cba700 1 -- 172.21.15.16:0/2469231631 >> 172.21.15.16:0/2469231631 conn(0x7f38a40a4210 msgr2=0x7f38a4130480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:39.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.642+0000 7f38a8cba700 1 -- 172.21.15.16:0/2469231631 shutdown_connections 2023-09-30T22:35:39.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.642+0000 7f38a8cba700 1 -- 172.21.15.16:0/2469231631 wait complete. 2023-09-30T22:35:39.660 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:35:39.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.693+0000 7fd88de97700 1 -- 172.21.15.16:0/1685907576 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd8880725e0 con 0x7fd88809e410 2023-09-30T22:35:39.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.693+0000 7fd86ffff700 1 -- 172.21.15.16:0/1685907576 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd880014e00 con 0x7fd88809e410 2023-09-30T22:35:39.695 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:35:39.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.696+0000 7fd88de97700 1 -- 172.21.15.16:0/1685907576 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd88809e410 msgr2=0x7fd8881a69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:39.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.696+0000 7fd88de97700 1 --2- 172.21.15.16:0/1685907576 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd88809e410 0x7fd8881a69e0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fd88000efc0 tx=0x7fd88000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:35:39.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.696+0000 7fd88de97700 1 -- 172.21.15.16:0/1685907576 shutdown_connections 2023-09-30T22:35:39.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.696+0000 7fd88de97700 1 --2- 172.21.15.16:0/1685907576 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd88809d6f0 0x7fd8881a5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:39.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.696+0000 7fd88de97700 1 --2- 172.21.15.16:0/1685907576 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd88809e410 0x7fd8881a69e0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:39.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.696+0000 7fd88de97700 1 --2- 172.21.15.16:0/1685907576 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd88809e000 0x7fd8881a64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:39.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.696+0000 7fd88de97700 1 -- 172.21.15.16:0/1685907576 >> 172.21.15.16:0/1685907576 conn(0x7fd888066f50 msgr2=0x7fd8881bacf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:39.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.696+0000 7fd88de97700 1 -- 172.21.15.16:0/1685907576 shutdown_connections 2023-09-30T22:35:39.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.696+0000 7fd88de97700 1 -- 172.21.15.16:0/1685907576 wait complete. 2023-09-30T22:35:39.711 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:35:39.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.816+0000 7f8b993ae700 1 -- 172.21.15.16:0/1953045625 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8b940747f0 con 0x7f8b9409c650 2023-09-30T22:35:39.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.817+0000 7f8b7b7fe700 1 -- 172.21.15.16:0/1953045625 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8b84021020 con 0x7f8b9409c650 2023-09-30T22:35:39.818 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:35:39.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.819+0000 7f8b993ae700 1 -- 172.21.15.16:0/1953045625 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8b9409c650 msgr2=0x7f8b9411a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:39.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.819+0000 7f8b993ae700 1 --2- 172.21.15.16:0/1953045625 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8b9409c650 0x7f8b9411a620 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f8b8400ec10 tx=0x7f8b8400ef20 comp rx=0 tx=0).stop 2023-09-30T22:35:39.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.819+0000 7f8b993ae700 1 -- 172.21.15.16:0/1953045625 shutdown_connections 2023-09-30T22:35:39.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.819+0000 7f8b993ae700 1 --2- 172.21.15.16:0/1953045625 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8b94131970 0x7f8b9411b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:39.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.819+0000 7f8b993ae700 1 --2- 172.21.15.16:0/1953045625 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8b9409c650 0x7f8b9411a620 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:39.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.819+0000 7f8b993ae700 1 --2- 172.21.15.16:0/1953045625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b940a0090 0x7f8b9411ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:39.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.819+0000 7f8b993ae700 1 -- 172.21.15.16:0/1953045625 >> 172.21.15.16:0/1953045625 conn(0x7f8b940a3f70 msgr2=0x7f8b9412eda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:39.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.819+0000 7f8b993ae700 1 -- 172.21.15.16:0/1953045625 shutdown_connections 2023-09-30T22:35:39.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:39.819+0000 7f8b993ae700 1 -- 172.21.15.16:0/1953045625 wait complete. 2023-09-30T22:35:39.834 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:35:40.095 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:35:40.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.240+0000 7f2844893700 1 Processor -- start 2023-09-30T22:35:40.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.240+0000 7f2844893700 1 -- start start 2023-09-30T22:35:40.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.240+0000 7f2844893700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f28401316c0 0x7f2840131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:40.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.240+0000 7f2844893700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f284009d9f0 0x7f284006a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:40.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.240+0000 7f2844893700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f284009e130 con 0x7f284006acb0 2023-09-30T22:35:40.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.240+0000 7f2844893700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28401312b0 con 0x7f284009d9f0 2023-09-30T22:35:40.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.240+0000 7f2844893700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f284009f6e0 con 0x7f28401316c0 2023-09-30T22:35:40.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.241+0000 7f283dd9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f284009d9f0 0x7f284006a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:40.244 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.241+0000 7f283e59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f28401316c0 0x7f2840131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:40.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.241+0000 7f283e59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f28401316c0 0x7f2840131a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:50576/0 (socket says 172.21.15.16:50576) 2023-09-30T22:35:40.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.241+0000 7f283ed9d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f284006acb0 0x7f2840138540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40538/0 (socket says 172.21.15.16:40538) 2023-09-30T22:35:40.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.241+0000 7f283dd9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f284009d9f0 0x7f284006a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47546/0 (socket says 172.21.15.16:47546) 2023-09-30T22:35:40.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.241+0000 7f283e59c700 1 -- 172.21.15.16:0/3032003490 learned_addr learned my addr 172.21.15.16:0/3032003490 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:40.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.241+0000 7f283d59a700 1 -- 172.21.15.16:0/3032003490 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3516366692 0 0) 0x7f2838001490 con 0x7f284006acb0 2023-09-30T22:35:40.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.241+0000 7f283d59a700 1 -- 172.21.15.16:0/3032003490 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28280035b0 con 0x7f284006acb0 2023-09-30T22:35:40.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.241+0000 7f283e59c700 1 -- 172.21.15.16:0/3032003490 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f284009d9f0 msgr2=0x7f284006a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:40.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.241+0000 7f283e59c700 1 --2- 172.21.15.16:0/3032003490 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f284009d9f0 0x7f284006a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:40.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.241+0000 7f283e59c700 1 -- 172.21.15.16:0/3032003490 >> v1:172.21.15.16:6789/0 conn(0x7f284006acb0 legacy=0x7f2840138540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:40.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.241+0000 7f283e59c700 1 -- 172.21.15.16:0/3032003490 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f284009e760 con 0x7f28401316c0 2023-09-30T22:35:40.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.241+0000 7f283dd9b700 1 --2- 172.21.15.16:0/3032003490 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f284009d9f0 0x7f284006a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:40.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.241+0000 7f283e59c700 1 --2- 172.21.15.16:0/3032003490 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f28401316c0 0x7f2840131a90 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f2830009920 tx=0x7f2830009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d60579a0aae246b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:40.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.241+0000 7f283d59a700 1 -- 172.21.15.16:0/3032003490 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2830011830 con 0x7f28401316c0 2023-09-30T22:35:40.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.241+0000 7f283d59a700 1 -- 172.21.15.16:0/3032003490 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2830011990 con 0x7f28401316c0 2023-09-30T22:35:40.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.241+0000 7f283d59a700 1 -- 172.21.15.16:0/3032003490 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2830011d20 con 0x7f28401316c0 2023-09-30T22:35:40.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.242+0000 7f2844893700 1 -- 172.21.15.16:0/3032003490 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f28401316c0 msgr2=0x7f2840131a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:40.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.242+0000 7f2844893700 1 --2- 172.21.15.16:0/3032003490 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f28401316c0 0x7f2840131a90 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f2830009920 tx=0x7f2830009c30 comp rx=0 tx=0).stop 2023-09-30T22:35:40.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.242+0000 7f2844893700 1 -- 172.21.15.16:0/3032003490 shutdown_connections 2023-09-30T22:35:40.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.242+0000 7f2844893700 1 --2- 172.21.15.16:0/3032003490 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f28401316c0 0x7f2840131a90 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:40.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.242+0000 7f2844893700 1 --2- 172.21.15.16:0/3032003490 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f284009d9f0 0x7f284006a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:40.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.242+0000 7f2844893700 1 -- 172.21.15.16:0/3032003490 >> 172.21.15.16:0/3032003490 conn(0x7f2840066f50 msgr2=0x7f284009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:40.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.242+0000 7f2844893700 1 -- 172.21.15.16:0/3032003490 shutdown_connections 2023-09-30T22:35:40.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.242+0000 7f2844893700 1 -- 172.21.15.16:0/3032003490 wait complete. 2023-09-30T22:35:40.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.243+0000 7f2844893700 1 Processor -- start 2023-09-30T22:35:40.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.243+0000 7f2844893700 1 -- start start 2023-09-30T22:35:40.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.244+0000 7f2844893700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f284006acb0 0x7f28400709b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:40.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.244+0000 7f2844893700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f284009d9f0 0x7f2840070ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:40.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.244+0000 7f2844893700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f28401316c0 0x7f2840071430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:40.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.244+0000 7f2844893700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f284011a900 con 0x7f284006acb0 2023-09-30T22:35:40.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.244+0000 7f2844893700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f284012b900 con 0x7f28401316c0 2023-09-30T22:35:40.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.244+0000 7f2844893700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2840128520 con 0x7f284009d9f0 2023-09-30T22:35:40.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.244+0000 7f283e59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f284006acb0 0x7f28400709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:40.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.244+0000 7f283dd9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f284009d9f0 0x7f2840070ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:40.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.244+0000 7f283e59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f284006acb0 0x7f28400709b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40770/0 (socket says 172.21.15.16:40770) 2023-09-30T22:35:40.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.244+0000 7f283e59c700 1 -- 172.21.15.16:0/1339676267 learned_addr learned my addr 172.21.15.16:0/1339676267 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:40.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.244+0000 7f283ed9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f28401316c0 0x7f2840071430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:40.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.244+0000 7f283dd9b700 1 -- 172.21.15.16:0/1339676267 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f28401316c0 msgr2=0x7f2840071430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:40.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.244+0000 7f283dd9b700 1 --2- 172.21.15.16:0/1339676267 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f28401316c0 0x7f2840071430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:40.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.244+0000 7f283dd9b700 1 -- 172.21.15.16:0/1339676267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f284006acb0 msgr2=0x7f28400709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:40.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.244+0000 7f283dd9b700 1 --2- 172.21.15.16:0/1339676267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f284006acb0 0x7f28400709b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:40.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.244+0000 7f283dd9b700 1 -- 172.21.15.16:0/1339676267 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28340094a0 con 0x7f284009d9f0 2023-09-30T22:35:40.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.244+0000 7f283ed9d700 1 --2- 172.21.15.16:0/1339676267 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f28401316c0 0x7f2840071430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:40.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.244+0000 7f283e59c700 1 --2- 172.21.15.16:0/1339676267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f284006acb0 0x7f28400709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:40.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.244+0000 7f283dd9b700 1 --2- 172.21.15.16:0/1339676267 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f284009d9f0 0x7f2840070ef0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f284006b7d0 tx=0x7f283400ec60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:40.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.245+0000 7f282effd700 1 -- 172.21.15.16:0/1339676267 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2834013680 con 0x7f284009d9f0 2023-09-30T22:35:40.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.245+0000 7f282effd700 1 -- 172.21.15.16:0/1339676267 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28340137e0 con 0x7f284009d9f0 2023-09-30T22:35:40.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.245+0000 7f282effd700 1 -- 172.21.15.16:0/1339676267 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2834013b90 con 0x7f284009d9f0 2023-09-30T22:35:40.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.245+0000 7f2844893700 1 -- 172.21.15.16:0/1339676267 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2830009550 con 0x7f284009d9f0 2023-09-30T22:35:40.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.245+0000 7f2844893700 1 -- 172.21.15.16:0/1339676267 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2840071d10 con 0x7f284009d9f0 2023-09-30T22:35:40.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.245+0000 7f282effd700 1 -- 172.21.15.16:0/1339676267 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2834013cf0 con 0x7f284009d9f0 2023-09-30T22:35:40.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.246+0000 7f282effd700 1 -- 172.21.15.16:0/1339676267 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f283403fa20 con 0x7f284009d9f0 2023-09-30T22:35:40.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.246+0000 7f2844893700 1 -- 172.21.15.16:0/1339676267 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2840074ea0 con 0x7f284009d9f0 2023-09-30T22:35:40.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.251+0000 7f282effd700 1 -- 172.21.15.16:0/1339676267 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2834011bd0 con 0x7f284009d9f0 2023-09-30T22:35:40.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.457+0000 7f2844893700 1 -- 172.21.15.16:0/1339676267 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f284011b870 con 0x7f284009d9f0 2023-09-30T22:35:40.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.458+0000 7f282effd700 1 -- 172.21.15.16:0/1339676267 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2834021030 con 0x7f284009d9f0 2023-09-30T22:35:40.459 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:35:40.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.460+0000 7f2844893700 1 -- 172.21.15.16:0/1339676267 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f284009d9f0 msgr2=0x7f2840070ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:40.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.460+0000 7f2844893700 1 --2- 172.21.15.16:0/1339676267 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f284009d9f0 0x7f2840070ef0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f284006b7d0 tx=0x7f283400ec60 comp rx=0 tx=0).stop 2023-09-30T22:35:40.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.460+0000 7f2844893700 1 -- 172.21.15.16:0/1339676267 shutdown_connections 2023-09-30T22:35:40.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.460+0000 7f2844893700 1 --2- 172.21.15.16:0/1339676267 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f28401316c0 0x7f2840071430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:40.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.460+0000 7f2844893700 1 --2- 172.21.15.16:0/1339676267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f284006acb0 0x7f28400709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:40.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.460+0000 7f2844893700 1 --2- 172.21.15.16:0/1339676267 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f284009d9f0 0x7f2840070ef0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:40.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.460+0000 7f2844893700 1 -- 172.21.15.16:0/1339676267 >> 172.21.15.16:0/1339676267 conn(0x7f2840066f50 msgr2=0x7f28400a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:40.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.460+0000 7f2844893700 1 -- 172.21.15.16:0/1339676267 shutdown_connections 2023-09-30T22:35:40.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.460+0000 7f2844893700 1 -- 172.21.15.16:0/1339676267 wait complete. 2023-09-30T22:35:40.476 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:35:40.662 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:35:40.713 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:35:40.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.813+0000 7fa298473700 1 Processor -- start 2023-09-30T22:35:40.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.813+0000 7fa298473700 1 -- start start 2023-09-30T22:35:40.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.814+0000 7fa298473700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa29009d630 0x7fa2900a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:40.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.814+0000 7fa298473700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa290138d80 0x7fa29013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:40.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.814+0000 7fa298473700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2900a2a70 con 0x7fa2900a0a30 2023-09-30T22:35:40.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.814+0000 7fa298473700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2900a01d0 con 0x7fa290138d80 2023-09-30T22:35:40.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.814+0000 7fa298473700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa29009e2b0 con 0x7fa29009d630 2023-09-30T22:35:40.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.815+0000 7fa295a0e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa29009d630 0x7fa2900a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:40.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.815+0000 7fa296a10700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa290138d80 0x7fa29013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:40.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.815+0000 7fa295a0e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa29009d630 0x7fa2900a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:50604/0 (socket says 172.21.15.16:50604) 2023-09-30T22:35:40.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.815+0000 7fa29620f700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa2900a0a30 0x7fa2900a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40552/0 (socket says 172.21.15.16:40552) 2023-09-30T22:35:40.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.815+0000 7fa296a10700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa290138d80 0x7fa29013b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47566/0 (socket says 172.21.15.16:47566) 2023-09-30T22:35:40.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.815+0000 7fa295a0e700 1 -- 172.21.15.16:0/2996154929 learned_addr learned my addr 172.21.15.16:0/2996154929 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:40.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.815+0000 7fa295a0e700 1 -- 172.21.15.16:0/2996154929 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa290138d80 msgr2=0x7fa29013b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:40.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.815+0000 7fa295a0e700 1 --2- 172.21.15.16:0/2996154929 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa290138d80 0x7fa29013b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:40.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.815+0000 7fa295a0e700 1 -- 172.21.15.16:0/2996154929 >> v1:172.21.15.16:6789/0 conn(0x7fa2900a0a30 legacy=0x7fa2900a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:40.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.815+0000 7fa295a0e700 1 -- 172.21.15.16:0/2996154929 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa29009dc50 con 0x7fa29009d630 2023-09-30T22:35:40.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.815+0000 7fa29520d700 1 -- 172.21.15.16:0/2996154929 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3989207929 0 0) 0x7fa2900a2a70 con 0x7fa2900a0a30 2023-09-30T22:35:40.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.815+0000 7fa296a10700 1 --2- 172.21.15.16:0/2996154929 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa290138d80 0x7fa29013b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:40.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.815+0000 7fa295a0e700 1 --2- 172.21.15.16:0/2996154929 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa29009d630 0x7fa2900a1510 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fa280009920 tx=0x7fa280009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48778f7520f9a7c0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:40.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.815+0000 7fa29520d700 1 -- 172.21.15.16:0/2996154929 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa280011830 con 0x7fa29009d630 2023-09-30T22:35:40.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.815+0000 7fa29520d700 1 -- 172.21.15.16:0/2996154929 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa280011990 con 0x7fa29009d630 2023-09-30T22:35:40.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.815+0000 7fa29520d700 1 -- 172.21.15.16:0/2996154929 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa280011d20 con 0x7fa29009d630 2023-09-30T22:35:40.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.816+0000 7fa298473700 1 -- 172.21.15.16:0/2996154929 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa29009d630 msgr2=0x7fa2900a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:40.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.816+0000 7fa298473700 1 --2- 172.21.15.16:0/2996154929 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa29009d630 0x7fa2900a1510 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fa280009920 tx=0x7fa280009c30 comp rx=0 tx=0).stop 2023-09-30T22:35:40.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.816+0000 7fa298473700 1 -- 172.21.15.16:0/2996154929 shutdown_connections 2023-09-30T22:35:40.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.816+0000 7fa298473700 1 --2- 172.21.15.16:0/2996154929 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa290138d80 0x7fa29013b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:40.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.816+0000 7fa298473700 1 --2- 172.21.15.16:0/2996154929 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa29009d630 0x7fa2900a1510 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:40.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.816+0000 7fa298473700 1 -- 172.21.15.16:0/2996154929 >> 172.21.15.16:0/2996154929 conn(0x7fa2900a3050 msgr2=0x7fa2900a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:40.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.816+0000 7fa298473700 1 -- 172.21.15.16:0/2996154929 shutdown_connections 2023-09-30T22:35:40.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.816+0000 7fa298473700 1 -- 172.21.15.16:0/2996154929 wait complete. 2023-09-30T22:35:40.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.817+0000 7fa298473700 1 Processor -- start 2023-09-30T22:35:40.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.817+0000 7fa298473700 1 -- start start 2023-09-30T22:35:40.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.818+0000 7fa298473700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa29009d630 0x7fa290133780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:40.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.818+0000 7fa298473700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa2900a0a30 0x7fa290133cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:40.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.818+0000 7fa298473700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa290138d80 0x7fa290134200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:40.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.818+0000 7fa298473700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa290137df0 con 0x7fa290138d80 2023-09-30T22:35:40.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.818+0000 7fa298473700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa290136c70 con 0x7fa29009d630 2023-09-30T22:35:40.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.818+0000 7fa298473700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa2900a4bd0 con 0x7fa2900a0a30 2023-09-30T22:35:40.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.818+0000 7fa295a0e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa2900a0a30 0x7fa290133cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:40.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.818+0000 7fa295a0e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa2900a0a30 0x7fa290133cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:50608/0 (socket says 172.21.15.16:50608) 2023-09-30T22:35:40.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.818+0000 7fa295a0e700 1 -- 172.21.15.16:0/298934175 learned_addr learned my addr 172.21.15.16:0/298934175 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:40.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.818+0000 7fa296a10700 1 --2- 172.21.15.16:0/298934175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa290138d80 0x7fa290134200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:40.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.818+0000 7fa29620f700 1 --2- 172.21.15.16:0/298934175 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa29009d630 0x7fa290133780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:40.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.818+0000 7fa295a0e700 1 -- 172.21.15.16:0/298934175 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa29009d630 msgr2=0x7fa290133780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:40.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.818+0000 7fa295a0e700 1 --2- 172.21.15.16:0/298934175 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa29009d630 0x7fa290133780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:40.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.818+0000 7fa295a0e700 1 -- 172.21.15.16:0/298934175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa290138d80 msgr2=0x7fa290134200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:40.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.818+0000 7fa295a0e700 1 --2- 172.21.15.16:0/298934175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa290138d80 0x7fa290134200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:40.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.818+0000 7fa295a0e700 1 -- 172.21.15.16:0/298934175 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa288009a70 con 0x7fa2900a0a30 2023-09-30T22:35:40.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.818+0000 7fa296a10700 1 --2- 172.21.15.16:0/298934175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa290138d80 0x7fa290134200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:40.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.819+0000 7fa295a0e700 1 --2- 172.21.15.16:0/298934175 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa2900a0a30 0x7fa290133cc0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fa28000a010 tx=0x7fa280004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:40.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.819+0000 7fa286ffd700 1 -- 172.21.15.16:0/298934175 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa280004450 con 0x7fa2900a0a30 2023-09-30T22:35:40.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.819+0000 7fa286ffd700 1 -- 172.21.15.16:0/298934175 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2800045b0 con 0x7fa2900a0a30 2023-09-30T22:35:40.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.819+0000 7fa298473700 1 -- 172.21.15.16:0/298934175 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa280009550 con 0x7fa2900a0a30 2023-09-30T22:35:40.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.819+0000 7fa286ffd700 1 -- 172.21.15.16:0/298934175 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2800038c0 con 0x7fa2900a0a30 2023-09-30T22:35:40.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.820+0000 7fa286ffd700 1 -- 172.21.15.16:0/298934175 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa280021070 con 0x7fa2900a0a30 2023-09-30T22:35:40.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.819+0000 7fa298473700 1 -- 172.21.15.16:0/298934175 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa290137210 con 0x7fa2900a0a30 2023-09-30T22:35:40.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.820+0000 7fa286ffd700 1 -- 172.21.15.16:0/298934175 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fa28001a5d0 con 0x7fa2900a0a30 2023-09-30T22:35:40.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.821+0000 7fa298473700 1 -- 172.21.15.16:0/298934175 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2900747f0 con 0x7fa2900a0a30 2023-09-30T22:35:40.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.826+0000 7fa286ffd700 1 -- 172.21.15.16:0/298934175 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa280015070 con 0x7fa2900a0a30 2023-09-30T22:35:40.835 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:35:40.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.856+0000 7f1330329700 1 Processor -- start 2023-09-30T22:35:40.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.856+0000 7f1330329700 1 -- start start 2023-09-30T22:35:40.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.857+0000 7f1330329700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f132809e930 0x7f132809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:40.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.857+0000 7f1330329700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f132809f650 0x7f1328131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:40.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.857+0000 7f1330329700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1328070940 con 0x7f132809f240 2023-09-30T22:35:40.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.857+0000 7f1330329700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f132806fd60 con 0x7f132809e930 2023-09-30T22:35:40.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.857+0000 7f1330329700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f132806fa80 con 0x7f132809f650 2023-09-30T22:35:40.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.857+0000 7f132e0c5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f132809e930 0x7f132809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:40.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.857+0000 7f132e0c5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f132809e930 0x7f132809ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47592/0 (socket says 172.21.15.16:47592) 2023-09-30T22:35:40.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.857+0000 7f132e0c5700 1 -- 172.21.15.16:0/510713654 learned_addr learned my addr 172.21.15.16:0/510713654 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:40.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.857+0000 7f132e8c6700 1 --2- 172.21.15.16:0/510713654 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f132809f650 0x7f1328131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:40.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.857+0000 7f132e8c6700 1 -- 172.21.15.16:0/510713654 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f132809e930 msgr2=0x7f132809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:40.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.857+0000 7f132e8c6700 1 --2- 172.21.15.16:0/510713654 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f132809e930 0x7f132809ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:40.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.857+0000 7f132e8c6700 1 -- 172.21.15.16:0/510713654 >> v1:172.21.15.16:6789/0 conn(0x7f132809f240 legacy=0x7f13281317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:40.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.857+0000 7f132e8c6700 1 -- 172.21.15.16:0/510713654 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f132809fb60 con 0x7f132809f650 2023-09-30T22:35:40.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.857+0000 7f132e0c5700 1 --2- 172.21.15.16:0/510713654 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f132809e930 0x7f132809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:40.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.857+0000 7f132e8c6700 1 --2- 172.21.15.16:0/510713654 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f132809f650 0x7f1328131f00 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f1320009dc0 tx=0x7f132000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91742760fbbc65c8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:40.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.858+0000 7f132d0c3700 1 -- 172.21.15.16:0/510713654 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132000e070 con 0x7f132809f650 2023-09-30T22:35:40.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.858+0000 7f132d0c3700 1 -- 172.21.15.16:0/510713654 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1320003a20 con 0x7f132809f650 2023-09-30T22:35:40.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.858+0000 7f132d0c3700 1 -- 172.21.15.16:0/510713654 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1320003dd0 con 0x7f132809f650 2023-09-30T22:35:40.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.858+0000 7f1330329700 1 -- 172.21.15.16:0/510713654 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f132809f650 msgr2=0x7f1328131f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:40.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.858+0000 7f1330329700 1 --2- 172.21.15.16:0/510713654 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f132809f650 0x7f1328131f00 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f1320009dc0 tx=0x7f132000b330 comp rx=0 tx=0).stop 2023-09-30T22:35:40.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.859+0000 7f1330329700 1 -- 172.21.15.16:0/510713654 shutdown_connections 2023-09-30T22:35:40.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.859+0000 7f1330329700 1 --2- 172.21.15.16:0/510713654 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f132809f650 0x7f1328131f00 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:40.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.859+0000 7f1330329700 1 --2- 172.21.15.16:0/510713654 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f132809e930 0x7f132809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:40.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.859+0000 7f1330329700 1 -- 172.21.15.16:0/510713654 >> 172.21.15.16:0/510713654 conn(0x7f13280a4210 msgr2=0x7f13280a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:40.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.859+0000 7f1330329700 1 -- 172.21.15.16:0/510713654 shutdown_connections 2023-09-30T22:35:40.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.859+0000 7f1330329700 1 -- 172.21.15.16:0/510713654 wait complete. 2023-09-30T22:35:40.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.860+0000 7f1330329700 1 Processor -- start 2023-09-30T22:35:40.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.860+0000 7f1330329700 1 -- start start 2023-09-30T22:35:40.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.860+0000 7f1330329700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f132809e930 0x7f13281c8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:40.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.860+0000 7f1330329700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f132809f240 0x7f13281c8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:40.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.860+0000 7f1330329700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f132809f650 0x7f13281c4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:40.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.860+0000 7f1330329700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13281a9b20 con 0x7f132809f650 2023-09-30T22:35:40.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.860+0000 7f1330329700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13281a72e0 con 0x7f132809f240 2023-09-30T22:35:40.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.860+0000 7f1330329700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f13280a35a0 con 0x7f132809e930 2023-09-30T22:35:40.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.860+0000 7f132e0c5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f132809e930 0x7f13281c8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:40.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.861+0000 7f132e0c5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f132809e930 0x7f13281c8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:50620/0 (socket says 172.21.15.16:50620) 2023-09-30T22:35:40.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.861+0000 7f132e0c5700 1 -- 172.21.15.16:0/3369105664 learned_addr learned my addr 172.21.15.16:0/3369105664 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:40.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.861+0000 7f132e8c6700 1 --2- 172.21.15.16:0/3369105664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f132809f650 0x7f13281c4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:40.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.861+0000 7f132d8c4700 1 --2- 172.21.15.16:0/3369105664 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f132809f240 0x7f13281c8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:40.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.861+0000 7f132e0c5700 1 -- 172.21.15.16:0/3369105664 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f132809f240 msgr2=0x7f13281c8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:40.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.861+0000 7f132e0c5700 1 --2- 172.21.15.16:0/3369105664 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f132809f240 0x7f13281c8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:40.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.861+0000 7f132e0c5700 1 -- 172.21.15.16:0/3369105664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f132809f650 msgr2=0x7f13281c4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:40.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.861+0000 7f132e0c5700 1 --2- 172.21.15.16:0/3369105664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f132809f650 0x7f13281c4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:40.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.861+0000 7f132e0c5700 1 -- 172.21.15.16:0/3369105664 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1324009550 con 0x7f132809e930 2023-09-30T22:35:40.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.861+0000 7f132e0c5700 1 --2- 172.21.15.16:0/3369105664 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f132809e930 0x7f13281c8600 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f132400ec10 tx=0x7f132400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:40.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.861+0000 7f131effd700 1 -- 172.21.15.16:0/3369105664 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132400c850 con 0x7f132809e930 2023-09-30T22:35:40.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.861+0000 7f131effd700 1 -- 172.21.15.16:0/3369105664 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f132400c9b0 con 0x7f132809e930 2023-09-30T22:35:40.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.861+0000 7f131effd700 1 -- 172.21.15.16:0/3369105664 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132400cd40 con 0x7f132809e930 2023-09-30T22:35:40.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.861+0000 7f1330329700 1 -- 172.21.15.16:0/3369105664 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1320009a70 con 0x7f132809e930 2023-09-30T22:35:40.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.862+0000 7f1330329700 1 -- 172.21.15.16:0/3369105664 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13280a0190 con 0x7f132809e930 2023-09-30T22:35:40.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.863+0000 7f1330329700 1 -- 172.21.15.16:0/3369105664 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13281c9080 con 0x7f132809e930 2023-09-30T22:35:40.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.863+0000 7f131effd700 1 -- 172.21.15.16:0/3369105664 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1324013470 con 0x7f132809e930 2023-09-30T22:35:40.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.863+0000 7f131effd700 1 -- 172.21.15.16:0/3369105664 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f132403f860 con 0x7f132809e930 2023-09-30T22:35:40.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.869+0000 7f131effd700 1 -- 172.21.15.16:0/3369105664 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1324013720 con 0x7f132809e930 2023-09-30T22:35:40.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.975+0000 7f5081614700 1 Processor -- start 2023-09-30T22:35:40.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.975+0000 7f5081614700 1 -- start start 2023-09-30T22:35:40.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.976+0000 7f5081614700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f507c0a0a30 0x7f507c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:40.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.976+0000 7f5081614700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f507c138d80 0x7f507c13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:40.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.976+0000 7f5081614700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f507c0a2a70 con 0x7f507c09d630 2023-09-30T22:35:40.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.976+0000 7f5081614700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f507c0a01d0 con 0x7f507c138d80 2023-09-30T22:35:40.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.976+0000 7f5081614700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f507c09e2b0 con 0x7f507c0a0a30 2023-09-30T22:35:40.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.976+0000 7f507a7fc700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f507c09d630 0x7f507c0a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40570/0 (socket says 172.21.15.16:40570) 2023-09-30T22:35:40.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.976+0000 7f507b7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f507c138d80 0x7f507c13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:40.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.976+0000 7f507a7fc700 1 -- 172.21.15.16:0/2610166718 learned_addr learned my addr 172.21.15.16:0/2610166718 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:40.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.976+0000 7f507affd700 1 --2- 172.21.15.16:0/2610166718 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f507c0a0a30 0x7f507c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:40.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.976+0000 7f507b7fe700 1 -- 172.21.15.16:0/2610166718 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f507c0a0a30 msgr2=0x7f507c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:40.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.976+0000 7f507b7fe700 1 --2- 172.21.15.16:0/2610166718 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f507c0a0a30 0x7f507c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:40.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.976+0000 7f507b7fe700 1 -- 172.21.15.16:0/2610166718 >> v1:172.21.15.16:6789/0 conn(0x7f507c09d630 legacy=0x7f507c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:40.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.976+0000 7f507b7fe700 1 -- 172.21.15.16:0/2610166718 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f507c09dc50 con 0x7f507c138d80 2023-09-30T22:35:40.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.977+0000 7f507b7fe700 1 --2- 172.21.15.16:0/2610166718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f507c138d80 0x7f507c13b160 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f5074009840 tx=0x7f5074009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36ce18274defed6f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:40.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.977+0000 7f5079ffb700 1 -- 172.21.15.16:0/2610166718 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5074004230 con 0x7f507c138d80 2023-09-30T22:35:40.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.977+0000 7f5079ffb700 1 -- 172.21.15.16:0/2610166718 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5074004390 con 0x7f507c138d80 2023-09-30T22:35:40.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.977+0000 7f5079ffb700 1 -- 172.21.15.16:0/2610166718 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50740036a0 con 0x7f507c138d80 2023-09-30T22:35:40.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.977+0000 7f5081614700 1 -- 172.21.15.16:0/2610166718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f507c138d80 msgr2=0x7f507c13b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:40.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.977+0000 7f5081614700 1 --2- 172.21.15.16:0/2610166718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f507c138d80 0x7f507c13b160 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f5074009840 tx=0x7f5074009b50 comp rx=0 tx=0).stop 2023-09-30T22:35:40.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.977+0000 7f5081614700 1 -- 172.21.15.16:0/2610166718 shutdown_connections 2023-09-30T22:35:40.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.977+0000 7f5081614700 1 --2- 172.21.15.16:0/2610166718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f507c138d80 0x7f507c13b160 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:40.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.977+0000 7f5081614700 1 --2- 172.21.15.16:0/2610166718 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f507c0a0a30 0x7f507c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:40.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.977+0000 7f5081614700 1 -- 172.21.15.16:0/2610166718 >> 172.21.15.16:0/2610166718 conn(0x7f507c0a3050 msgr2=0x7f507c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:40.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.977+0000 7f5081614700 1 -- 172.21.15.16:0/2610166718 shutdown_connections 2023-09-30T22:35:40.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.977+0000 7f5081614700 1 -- 172.21.15.16:0/2610166718 wait complete. 2023-09-30T22:35:40.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.979+0000 7f5081614700 1 Processor -- start 2023-09-30T22:35:40.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.979+0000 7f5081614700 1 -- start start 2023-09-30T22:35:40.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.979+0000 7f5081614700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f507c09d630 0x7f507c1338c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:40.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.979+0000 7f5081614700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f507c0a0a30 0x7f507c133e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:40.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.979+0000 7f5081614700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f507c138d80 0x7f507c134340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:40.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.979+0000 7f5081614700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f507c136f20 con 0x7f507c09d630 2023-09-30T22:35:40.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.979+0000 7f5081614700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f507c1380d0 con 0x7f507c0a0a30 2023-09-30T22:35:40.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.979+0000 7f5081614700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f507c137100 con 0x7f507c138d80 2023-09-30T22:35:40.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.979+0000 7f507a7fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f507c0a0a30 0x7f507c133e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:40.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.979+0000 7f507a7fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f507c0a0a30 0x7f507c133e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47610/0 (socket says 172.21.15.16:47610) 2023-09-30T22:35:40.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.979+0000 7f507a7fc700 1 -- 172.21.15.16:0/1868050007 learned_addr learned my addr 172.21.15.16:0/1868050007 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:40.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.979+0000 7f507b7fe700 1 --2- 172.21.15.16:0/1868050007 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f507c138d80 0x7f507c134340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:40.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.980+0000 7f507affd700 1 --2- 172.21.15.16:0/1868050007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f507c09d630 0x7f507c1338c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:40.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.980+0000 7f507a7fc700 1 -- 172.21.15.16:0/1868050007 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f507c138d80 msgr2=0x7f507c134340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:40.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.980+0000 7f507a7fc700 1 --2- 172.21.15.16:0/1868050007 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f507c138d80 0x7f507c134340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:40.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.980+0000 7f507a7fc700 1 -- 172.21.15.16:0/1868050007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f507c09d630 msgr2=0x7f507c1338c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:40.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.980+0000 7f507a7fc700 1 --2- 172.21.15.16:0/1868050007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f507c09d630 0x7f507c1338c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:40.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.980+0000 7f507a7fc700 1 -- 172.21.15.16:0/1868050007 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50740094a0 con 0x7f507c0a0a30 2023-09-30T22:35:40.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.980+0000 7f507b7fe700 1 --2- 172.21.15.16:0/1868050007 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f507c138d80 0x7f507c134340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:35:40.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.980+0000 7f507a7fc700 1 --2- 172.21.15.16:0/1868050007 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f507c0a0a30 0x7f507c133e00 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f507000c780 tx=0x7f507000cb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:40.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.980+0000 7f50637fe700 1 -- 172.21.15.16:0/1868050007 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5070004c80 con 0x7f507c0a0a30 2023-09-30T22:35:40.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.980+0000 7f50637fe700 1 -- 172.21.15.16:0/1868050007 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5070004de0 con 0x7f507c0a0a30 2023-09-30T22:35:40.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.980+0000 7f50637fe700 1 -- 172.21.15.16:0/1868050007 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5070007310 con 0x7f507c0a0a30 2023-09-30T22:35:40.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.980+0000 7f5081614700 1 -- 172.21.15.16:0/1868050007 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f507c1362d0 con 0x7f507c0a0a30 2023-09-30T22:35:40.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.980+0000 7f5081614700 1 -- 172.21.15.16:0/1868050007 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f507c137b70 con 0x7f507c0a0a30 2023-09-30T22:35:40.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.981+0000 7f50637fe700 1 -- 172.21.15.16:0/1868050007 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5070007470 con 0x7f507c0a0a30 2023-09-30T22:35:40.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.981+0000 7f5081614700 1 -- 172.21.15.16:0/1868050007 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f507c0747f0 con 0x7f507c0a0a30 2023-09-30T22:35:40.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.981+0000 7f50637fe700 1 -- 172.21.15.16:0/1868050007 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f507003e900 con 0x7f507c0a0a30 2023-09-30T22:35:40.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:40.986+0000 7f50637fe700 1 -- 172.21.15.16:0/1868050007 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5070007720 con 0x7f507c0a0a30 2023-09-30T22:35:41.058 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.057+0000 7fa298473700 1 -- 172.21.15.16:0/298934175 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa290136e90 con 0x7fa2900a0a30 2023-09-30T22:35:41.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.058+0000 7fa286ffd700 1 -- 172.21.15.16:0/298934175 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa290136e90 con 0x7fa2900a0a30 2023-09-30T22:35:41.059 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:35:41.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.060+0000 7fa298473700 1 -- 172.21.15.16:0/298934175 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa2900a0a30 msgr2=0x7fa290133cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:41.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.060+0000 7fa298473700 1 --2- 172.21.15.16:0/298934175 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa2900a0a30 0x7fa290133cc0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fa28000a010 tx=0x7fa280004750 comp rx=0 tx=0).stop 2023-09-30T22:35:41.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.060+0000 7fa298473700 1 -- 172.21.15.16:0/298934175 shutdown_connections 2023-09-30T22:35:41.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.060+0000 7fa298473700 1 --2- 172.21.15.16:0/298934175 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa29009d630 0x7fa290133780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:41.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.060+0000 7fa298473700 1 --2- 172.21.15.16:0/298934175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa290138d80 0x7fa290134200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:41.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.060+0000 7fa298473700 1 --2- 172.21.15.16:0/298934175 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa2900a0a30 0x7fa290133cc0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:41.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.060+0000 7fa298473700 1 -- 172.21.15.16:0/298934175 >> 172.21.15.16:0/298934175 conn(0x7fa2900a3050 msgr2=0x7fa29009ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:41.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.060+0000 7fa298473700 1 -- 172.21.15.16:0/298934175 shutdown_connections 2023-09-30T22:35:41.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.060+0000 7fa298473700 1 -- 172.21.15.16:0/298934175 wait complete. 2023-09-30T22:35:41.076 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:35:41.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.096+0000 7f1330329700 1 -- 172.21.15.16:0/3369105664 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f13281a92d0 con 0x7f132809e930 2023-09-30T22:35:41.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.097+0000 7f131effd700 1 -- 172.21.15.16:0/3369105664 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1324021070 con 0x7f132809e930 2023-09-30T22:35:41.099 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:35:41.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.099+0000 7f1330329700 1 -- 172.21.15.16:0/3369105664 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f132809e930 msgr2=0x7f13281c8600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:41.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.099+0000 7f1330329700 1 --2- 172.21.15.16:0/3369105664 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f132809e930 0x7f13281c8600 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f132400ec10 tx=0x7f132400ef20 comp rx=0 tx=0).stop 2023-09-30T22:35:41.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.099+0000 7f1330329700 1 -- 172.21.15.16:0/3369105664 shutdown_connections 2023-09-30T22:35:41.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.099+0000 7f1330329700 1 --2- 172.21.15.16:0/3369105664 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f132809f650 0x7f13281c4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:41.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.099+0000 7f1330329700 1 --2- 172.21.15.16:0/3369105664 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f132809e930 0x7f13281c8600 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:41.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.099+0000 7f1330329700 1 --2- 172.21.15.16:0/3369105664 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f132809f240 0x7f13281c8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:41.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.099+0000 7f1330329700 1 -- 172.21.15.16:0/3369105664 >> 172.21.15.16:0/3369105664 conn(0x7f13280a4210 msgr2=0x7f13281baf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:41.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.099+0000 7f1330329700 1 -- 172.21.15.16:0/3369105664 shutdown_connections 2023-09-30T22:35:41.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.099+0000 7f1330329700 1 -- 172.21.15.16:0/3369105664 wait complete. 2023-09-30T22:35:41.114 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:35:41.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.196+0000 7f5081614700 1 -- 172.21.15.16:0/1868050007 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f507c072980 con 0x7f507c0a0a30 2023-09-30T22:35:41.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.197+0000 7f50637fe700 1 -- 172.21.15.16:0/1868050007 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5070018030 con 0x7f507c0a0a30 2023-09-30T22:35:41.198 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:35:41.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.199+0000 7f5081614700 1 -- 172.21.15.16:0/1868050007 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f507c0a0a30 msgr2=0x7f507c133e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:41.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.199+0000 7f5081614700 1 --2- 172.21.15.16:0/1868050007 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f507c0a0a30 0x7f507c133e00 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f507000c780 tx=0x7f507000cb40 comp rx=0 tx=0).stop 2023-09-30T22:35:41.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.199+0000 7f5081614700 1 -- 172.21.15.16:0/1868050007 shutdown_connections 2023-09-30T22:35:41.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.199+0000 7f5081614700 1 --2- 172.21.15.16:0/1868050007 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f507c138d80 0x7f507c134340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:41.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.199+0000 7f5081614700 1 --2- 172.21.15.16:0/1868050007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f507c09d630 0x7f507c1338c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:41.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.199+0000 7f5081614700 1 --2- 172.21.15.16:0/1868050007 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f507c0a0a30 0x7f507c133e00 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:41.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.199+0000 7f5081614700 1 -- 172.21.15.16:0/1868050007 >> 172.21.15.16:0/1868050007 conn(0x7f507c0a3050 msgr2=0x7f507c09f620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:41.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.199+0000 7f5081614700 1 -- 172.21.15.16:0/1868050007 shutdown_connections 2023-09-30T22:35:41.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.199+0000 7f5081614700 1 -- 172.21.15.16:0/1868050007 wait complete. 2023-09-30T22:35:41.213 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:35:41.477 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:35:41.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.623+0000 7fca28c46700 1 Processor -- start 2023-09-30T22:35:41.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.623+0000 7fca28c46700 1 -- start start 2023-09-30T22:35:41.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.624+0000 7fca28c46700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fca2406b250 0x7fca24136840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:41.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.624+0000 7fca28c46700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca24136d80 0x7fca24139160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:41.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.624+0000 7fca28c46700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca2409e810 con 0x7fca2406a770 2023-09-30T22:35:41.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.624+0000 7fca28c46700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca2409e070 con 0x7fca2406b250 2023-09-30T22:35:41.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.624+0000 7fca28c46700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fca2409ead0 con 0x7fca24136d80 2023-09-30T22:35:41.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.624+0000 7fca2259c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fca2406a770 0x7fca2406ab40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40572/0 (socket says 172.21.15.16:40572) 2023-09-30T22:35:41.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.624+0000 7fca22d9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca24136d80 0x7fca24139160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:41.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.624+0000 7fca2259c700 1 -- 172.21.15.16:0/376406511 learned_addr learned my addr 172.21.15.16:0/376406511 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:41.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.624+0000 7fca21d9b700 1 --2- 172.21.15.16:0/376406511 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fca2406b250 0x7fca24136840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:41.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.624+0000 7fca2159a700 1 -- 172.21.15.16:0/376406511 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4054111634 0 0) 0x7fca2409e810 con 0x7fca2406a770 2023-09-30T22:35:41.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.624+0000 7fca22d9d700 1 -- 172.21.15.16:0/376406511 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fca2406b250 msgr2=0x7fca24136840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:41.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.624+0000 7fca22d9d700 1 --2- 172.21.15.16:0/376406511 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fca2406b250 0x7fca24136840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:41.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.624+0000 7fca22d9d700 1 -- 172.21.15.16:0/376406511 >> v1:172.21.15.16:6789/0 conn(0x7fca2406a770 legacy=0x7fca2406ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:41.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.624+0000 7fca22d9d700 1 -- 172.21.15.16:0/376406511 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca2409daa0 con 0x7fca24136d80 2023-09-30T22:35:41.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.624+0000 7fca21d9b700 1 --2- 172.21.15.16:0/376406511 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fca2406b250 0x7fca24136840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:41.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.624+0000 7fca22d9d700 1 --2- 172.21.15.16:0/376406511 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca24136d80 0x7fca24139160 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fca1c009df0 tx=0x7fca1c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cad5a7a5f6b32c1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:41.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.624+0000 7fca2159a700 1 -- 172.21.15.16:0/376406511 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c00e070 con 0x7fca24136d80 2023-09-30T22:35:41.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.625+0000 7fca2159a700 1 -- 172.21.15.16:0/376406511 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca1c00bae0 con 0x7fca24136d80 2023-09-30T22:35:41.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.625+0000 7fca2159a700 1 -- 172.21.15.16:0/376406511 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c00be90 con 0x7fca24136d80 2023-09-30T22:35:41.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.625+0000 7fca28c46700 1 -- 172.21.15.16:0/376406511 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca24136d80 msgr2=0x7fca24139160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:41.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.625+0000 7fca28c46700 1 --2- 172.21.15.16:0/376406511 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca24136d80 0x7fca24139160 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fca1c009df0 tx=0x7fca1c00b330 comp rx=0 tx=0).stop 2023-09-30T22:35:41.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.625+0000 7fca28c46700 1 -- 172.21.15.16:0/376406511 shutdown_connections 2023-09-30T22:35:41.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.625+0000 7fca28c46700 1 --2- 172.21.15.16:0/376406511 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca24136d80 0x7fca24139160 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:41.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.625+0000 7fca28c46700 1 --2- 172.21.15.16:0/376406511 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fca2406b250 0x7fca24136840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:41.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.625+0000 7fca28c46700 1 -- 172.21.15.16:0/376406511 >> 172.21.15.16:0/376406511 conn(0x7fca24066f50 msgr2=0x7fca2409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:41.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.625+0000 7fca28c46700 1 -- 172.21.15.16:0/376406511 shutdown_connections 2023-09-30T22:35:41.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.625+0000 7fca28c46700 1 -- 172.21.15.16:0/376406511 wait complete. 2023-09-30T22:35:41.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.626+0000 7fca28c46700 1 Processor -- start 2023-09-30T22:35:41.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.626+0000 7fca28c46700 1 -- start start 2023-09-30T22:35:41.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.627+0000 7fca28c46700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca2406a770 0x7fca241c82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:41.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.627+0000 7fca28c46700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fca2406b250 0x7fca241c87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:41.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.627+0000 7fca28c46700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca24136d80 0x7fca241c8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:41.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.627+0000 7fca28c46700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca241264d0 con 0x7fca2406a770 2023-09-30T22:35:41.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.627+0000 7fca28c46700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca24125740 con 0x7fca2406b250 2023-09-30T22:35:41.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.627+0000 7fca28c46700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fca2409bf70 con 0x7fca24136d80 2023-09-30T22:35:41.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.627+0000 7fca21d9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fca2406b250 0x7fca241c87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:41.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.627+0000 7fca22d9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca24136d80 0x7fca241c8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:41.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.627+0000 7fca2259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca2406a770 0x7fca241c82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:41.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.627+0000 7fca21d9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fca2406b250 0x7fca241c87e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47618/0 (socket says 172.21.15.16:47618) 2023-09-30T22:35:41.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.627+0000 7fca21d9b700 1 -- 172.21.15.16:0/3241095441 learned_addr learned my addr 172.21.15.16:0/3241095441 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:41.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.627+0000 7fca22d9d700 1 -- 172.21.15.16:0/3241095441 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fca2406b250 msgr2=0x7fca241c87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:41.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.627+0000 7fca22d9d700 1 --2- 172.21.15.16:0/3241095441 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fca2406b250 0x7fca241c87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:41.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.627+0000 7fca22d9d700 1 -- 172.21.15.16:0/3241095441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca2406a770 msgr2=0x7fca241c82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:41.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.627+0000 7fca22d9d700 1 --2- 172.21.15.16:0/3241095441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca2406a770 0x7fca241c82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:41.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.627+0000 7fca22d9d700 1 -- 172.21.15.16:0/3241095441 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca180094a0 con 0x7fca24136d80 2023-09-30T22:35:41.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.627+0000 7fca21d9b700 1 --2- 172.21.15.16:0/3241095441 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fca2406b250 0x7fca241c87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:41.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.628+0000 7fca22d9d700 1 --2- 172.21.15.16:0/3241095441 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca24136d80 0x7fca241c8d20 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fca1c016000 tx=0x7fca1c004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:41.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.628+0000 7fca12ffd700 1 -- 172.21.15.16:0/3241095441 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c027480 con 0x7fca24136d80 2023-09-30T22:35:41.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.628+0000 7fca12ffd700 1 -- 172.21.15.16:0/3241095441 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca1c020070 con 0x7fca24136d80 2023-09-30T22:35:41.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.628+0000 7fca28c46700 1 -- 172.21.15.16:0/3241095441 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca1c009a70 con 0x7fca24136d80 2023-09-30T22:35:41.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.628+0000 7fca12ffd700 1 -- 172.21.15.16:0/3241095441 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c027820 con 0x7fca24136d80 2023-09-30T22:35:41.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.628+0000 7fca28c46700 1 -- 172.21.15.16:0/3241095441 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca24071ed0 con 0x7fca24136d80 2023-09-30T22:35:41.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.629+0000 7fca12ffd700 1 -- 172.21.15.16:0/3241095441 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fca1c01b400 con 0x7fca24136d80 2023-09-30T22:35:41.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.629+0000 7fca12ffd700 1 -- 172.21.15.16:0/3241095441 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fca1c022070 con 0x7fca24136d80 2023-09-30T22:35:41.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.629+0000 7fca28c46700 1 -- 172.21.15.16:0/3241095441 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca240747f0 con 0x7fca24136d80 2023-09-30T22:35:41.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.634+0000 7fca12ffd700 1 -- 172.21.15.16:0/3241095441 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fca1c01c400 con 0x7fca24136d80 2023-09-30T22:35:41.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.839+0000 7fca28c46700 1 -- 172.21.15.16:0/3241095441 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fca2406c360 con 0x7fca24136d80 2023-09-30T22:35:41.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.839+0000 7fca12ffd700 1 -- 172.21.15.16:0/3241095441 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fca1c02c020 con 0x7fca24136d80 2023-09-30T22:35:41.840 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:35:41.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.841+0000 7fca28c46700 1 -- 172.21.15.16:0/3241095441 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca24136d80 msgr2=0x7fca241c8d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:41.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.841+0000 7fca28c46700 1 --2- 172.21.15.16:0/3241095441 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca24136d80 0x7fca241c8d20 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fca1c016000 tx=0x7fca1c004ad0 comp rx=0 tx=0).stop 2023-09-30T22:35:41.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.841+0000 7fca28c46700 1 -- 172.21.15.16:0/3241095441 shutdown_connections 2023-09-30T22:35:41.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.841+0000 7fca28c46700 1 --2- 172.21.15.16:0/3241095441 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fca24136d80 0x7fca241c8d20 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:41.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.841+0000 7fca28c46700 1 --2- 172.21.15.16:0/3241095441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fca2406a770 0x7fca241c82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:41.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.841+0000 7fca28c46700 1 --2- 172.21.15.16:0/3241095441 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fca2406b250 0x7fca241c87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:41.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.841+0000 7fca28c46700 1 -- 172.21.15.16:0/3241095441 >> 172.21.15.16:0/3241095441 conn(0x7fca24066f50 msgr2=0x7fca2406ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:41.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.841+0000 7fca28c46700 1 -- 172.21.15.16:0/3241095441 shutdown_connections 2023-09-30T22:35:41.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:41.841+0000 7fca28c46700 1 -- 172.21.15.16:0/3241095441 wait complete. 2023-09-30T22:35:41.856 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:35:42.076 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:35:42.115 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:35:42.214 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:35:42.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.225+0000 7f7abc2b6700 1 Processor -- start 2023-09-30T22:35:42.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.225+0000 7f7abc2b6700 1 -- start start 2023-09-30T22:35:42.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.226+0000 7f7abc2b6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7ab409e1d0 0x7f7ab4135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:42.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.226+0000 7f7abc2b6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7ab4135e50 0x7f7ab4138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:42.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.226+0000 7f7abc2b6700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ab409f130 con 0x7f7ab409d6f0 2023-09-30T22:35:42.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.226+0000 7f7abc2b6700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab409f520 con 0x7f7ab409e1d0 2023-09-30T22:35:42.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.226+0000 7f7abc2b6700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab409e930 con 0x7f7ab4135e50 2023-09-30T22:35:42.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.226+0000 7f7ab9851700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7ab409e1d0 0x7f7ab4135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:42.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.226+0000 7f7aba052700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7ab409d6f0 0x7f7ab409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40578/0 (socket says 172.21.15.16:40578) 2023-09-30T22:35:42.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.226+0000 7f7aba052700 1 -- 172.21.15.16:0/3365291757 learned_addr learned my addr 172.21.15.16:0/3365291757 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:42.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.227+0000 7f7aba853700 1 --2- 172.21.15.16:0/3365291757 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7ab4135e50 0x7f7ab4138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:42.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.227+0000 7f7ab9851700 1 -- 172.21.15.16:0/3365291757 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7ab4135e50 msgr2=0x7f7ab4138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:42.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.227+0000 7f7ab9851700 1 --2- 172.21.15.16:0/3365291757 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7ab4135e50 0x7f7ab4138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:42.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.227+0000 7f7ab9851700 1 -- 172.21.15.16:0/3365291757 >> v1:172.21.15.16:6789/0 conn(0x7f7ab409d6f0 legacy=0x7f7ab409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:42.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.227+0000 7f7ab9050700 1 -- 172.21.15.16:0/3365291757 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2656350338 0 0) 0x7f7ab409f130 con 0x7f7ab409d6f0 2023-09-30T22:35:42.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.227+0000 7f7ab9851700 1 -- 172.21.15.16:0/3365291757 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab409b2d0 con 0x7f7ab409e1d0 2023-09-30T22:35:42.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.227+0000 7f7aba853700 1 --2- 172.21.15.16:0/3365291757 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7ab4135e50 0x7f7ab4138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:42.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.227+0000 7f7ab9851700 1 --2- 172.21.15.16:0/3365291757 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7ab409e1d0 0x7f7ab4135910 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f7aa40098a0 tx=0x7f7aa4009bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=215706192b7b85 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:42.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.227+0000 7f7ab9050700 1 -- 172.21.15.16:0/3365291757 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa4011830 con 0x7f7ab409e1d0 2023-09-30T22:35:42.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.227+0000 7f7ab9050700 1 -- 172.21.15.16:0/3365291757 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aa4011990 con 0x7f7ab409e1d0 2023-09-30T22:35:42.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.227+0000 7f7ab9050700 1 -- 172.21.15.16:0/3365291757 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa4011d20 con 0x7f7ab409e1d0 2023-09-30T22:35:42.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.228+0000 7f7abc2b6700 1 -- 172.21.15.16:0/3365291757 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7ab409e1d0 msgr2=0x7f7ab4135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:42.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.228+0000 7f7abc2b6700 1 --2- 172.21.15.16:0/3365291757 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7ab409e1d0 0x7f7ab4135910 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f7aa40098a0 tx=0x7f7aa4009bb0 comp rx=0 tx=0).stop 2023-09-30T22:35:42.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.229+0000 7f7abc2b6700 1 -- 172.21.15.16:0/3365291757 shutdown_connections 2023-09-30T22:35:42.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.229+0000 7f7abc2b6700 1 --2- 172.21.15.16:0/3365291757 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7ab4135e50 0x7f7ab4138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:42.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.229+0000 7f7abc2b6700 1 --2- 172.21.15.16:0/3365291757 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7ab409e1d0 0x7f7ab4135910 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:42.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.229+0000 7f7abc2b6700 1 -- 172.21.15.16:0/3365291757 >> 172.21.15.16:0/3365291757 conn(0x7f7ab4066f50 msgr2=0x7f7ab409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:42.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.229+0000 7f7abc2b6700 1 -- 172.21.15.16:0/3365291757 shutdown_connections 2023-09-30T22:35:42.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.229+0000 7f7abc2b6700 1 -- 172.21.15.16:0/3365291757 wait complete. 2023-09-30T22:35:42.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.230+0000 7f7abc2b6700 1 Processor -- start 2023-09-30T22:35:42.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.230+0000 7f7abc2b6700 1 -- start start 2023-09-30T22:35:42.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.231+0000 7f7abc2b6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7ab409d6f0 0x7f7ab411b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:42.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.231+0000 7f7abc2b6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ab409e1d0 0x7f7ab411ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:42.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.231+0000 7f7abc2b6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7ab4135e50 0x7f7ab411bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:42.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.231+0000 7f7abc2b6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab411f9b0 con 0x7f7ab409e1d0 2023-09-30T22:35:42.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.231+0000 7f7abc2b6700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab411eff0 con 0x7f7ab4135e50 2023-09-30T22:35:42.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.231+0000 7f7abc2b6700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab411e330 con 0x7f7ab409d6f0 2023-09-30T22:35:42.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.231+0000 7f7ab9851700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ab409e1d0 0x7f7ab411ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:42.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.231+0000 7f7aba052700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7ab409d6f0 0x7f7ab411b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:42.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.231+0000 7f7ab9851700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ab409e1d0 0x7f7ab411ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40834/0 (socket says 172.21.15.16:40834) 2023-09-30T22:35:42.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.231+0000 7f7ab9851700 1 -- 172.21.15.16:0/2231315421 learned_addr learned my addr 172.21.15.16:0/2231315421 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:42.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.231+0000 7f7aba052700 1 -- 172.21.15.16:0/2231315421 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7ab4135e50 msgr2=0x7f7ab411bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:42.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.231+0000 7f7aba052700 1 --2- 172.21.15.16:0/2231315421 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7ab4135e50 0x7f7ab411bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:42.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.231+0000 7f7aba052700 1 -- 172.21.15.16:0/2231315421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ab409e1d0 msgr2=0x7f7ab411ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:42.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.231+0000 7f7aba052700 1 --2- 172.21.15.16:0/2231315421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ab409e1d0 0x7f7ab411ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:42.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.231+0000 7f7aba052700 1 -- 172.21.15.16:0/2231315421 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aac009a70 con 0x7f7ab409d6f0 2023-09-30T22:35:42.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.232+0000 7f7ab9851700 1 --2- 172.21.15.16:0/2231315421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ab409e1d0 0x7f7ab411ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:42.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.232+0000 7f7aba052700 1 --2- 172.21.15.16:0/2231315421 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7ab409d6f0 0x7f7ab411b520 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f7ab000c4a0 tx=0x7f7ab000c7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:42.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.232+0000 7f7aaaffd700 1 -- 172.21.15.16:0/2231315421 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab000cea0 con 0x7f7ab409d6f0 2023-09-30T22:35:42.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.232+0000 7f7aaaffd700 1 -- 172.21.15.16:0/2231315421 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ab0004c80 con 0x7f7ab409d6f0 2023-09-30T22:35:42.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.232+0000 7f7abc2b6700 1 -- 172.21.15.16:0/2231315421 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7aa40094a0 con 0x7f7ab409d6f0 2023-09-30T22:35:42.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.232+0000 7f7aaaffd700 1 -- 172.21.15.16:0/2231315421 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ab00070d0 con 0x7f7ab409d6f0 2023-09-30T22:35:42.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.232+0000 7f7abc2b6700 1 -- 172.21.15.16:0/2231315421 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ab411d0e0 con 0x7f7ab409d6f0 2023-09-30T22:35:42.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.233+0000 7f7abc2b6700 1 -- 172.21.15.16:0/2231315421 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ab4071700 con 0x7f7ab409d6f0 2023-09-30T22:35:42.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.233+0000 7f7aaaffd700 1 -- 172.21.15.16:0/2231315421 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7ab0007230 con 0x7f7ab409d6f0 2023-09-30T22:35:42.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.233+0000 7f7aaaffd700 1 -- 172.21.15.16:0/2231315421 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f7ab003f2b0 con 0x7f7ab409d6f0 2023-09-30T22:35:42.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.238+0000 7f7aaaffd700 1 -- 172.21.15.16:0/2231315421 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7ab001bb70 con 0x7f7ab409d6f0 2023-09-30T22:35:42.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.274+0000 7f6c4f154700 1 Processor -- start 2023-09-30T22:35:42.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.274+0000 7f6c4f154700 1 -- start start 2023-09-30T22:35:42.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.275+0000 7f6c4f154700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6c480a0a30 0x7f6c480a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:42.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.275+0000 7f6c4f154700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6c4809d630 0x7f6c480a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:42.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.275+0000 7f6c4f154700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c480a2a70 con 0x7f6c48133930 2023-09-30T22:35:42.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.275+0000 7f6c4f154700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c480a01d0 con 0x7f6c4809d630 2023-09-30T22:35:42.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.275+0000 7f6c4f154700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4809e2b0 con 0x7f6c480a0a30 2023-09-30T22:35:42.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.275+0000 7f6c47fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6c4809d630 0x7f6c480a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:42.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.275+0000 7f6c4d6f1700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6c48133930 0x7f6c48133db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40592/0 (socket says 172.21.15.16:40592) 2023-09-30T22:35:42.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.275+0000 7f6c4d6f1700 1 -- 172.21.15.16:0/229797182 learned_addr learned my addr 172.21.15.16:0/229797182 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:42.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.275+0000 7f6c4cef0700 1 --2- 172.21.15.16:0/229797182 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6c480a0a30 0x7f6c480a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:42.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.276+0000 7f6c47fff700 1 -- 172.21.15.16:0/229797182 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6c480a0a30 msgr2=0x7f6c480a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:42.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.276+0000 7f6c47fff700 1 --2- 172.21.15.16:0/229797182 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6c480a0a30 0x7f6c480a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:42.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.276+0000 7f6c47fff700 1 -- 172.21.15.16:0/229797182 >> v1:172.21.15.16:6789/0 conn(0x7f6c48133930 legacy=0x7f6c48133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:42.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.276+0000 7f6c477fe700 1 -- 172.21.15.16:0/229797182 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2316906646 0 0) 0x7f6c480a2a70 con 0x7f6c48133930 2023-09-30T22:35:42.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.276+0000 7f6c47fff700 1 -- 172.21.15.16:0/229797182 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c4809dc50 con 0x7f6c4809d630 2023-09-30T22:35:42.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.276+0000 7f6c47fff700 1 --2- 172.21.15.16:0/229797182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c4809d630 0x7f6c480a1340 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c009840 tx=0x7f6c3c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb2e2855b9ff7ea5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:42.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.276+0000 7f6c477fe700 1 -- 172.21.15.16:0/229797182 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c011830 con 0x7f6c4809d630 2023-09-30T22:35:42.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.276+0000 7f6c477fe700 1 -- 172.21.15.16:0/229797182 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c3c011990 con 0x7f6c4809d630 2023-09-30T22:35:42.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.276+0000 7f6c477fe700 1 -- 172.21.15.16:0/229797182 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c011d20 con 0x7f6c4809d630 2023-09-30T22:35:42.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.277+0000 7f6c4f154700 1 -- 172.21.15.16:0/229797182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c4809d630 msgr2=0x7f6c480a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:42.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.277+0000 7f6c4f154700 1 --2- 172.21.15.16:0/229797182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c4809d630 0x7f6c480a1340 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c009840 tx=0x7f6c3c009b50 comp rx=0 tx=0).stop 2023-09-30T22:35:42.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.277+0000 7f6c4f154700 1 -- 172.21.15.16:0/229797182 shutdown_connections 2023-09-30T22:35:42.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.277+0000 7f6c4f154700 1 --2- 172.21.15.16:0/229797182 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6c480a0a30 0x7f6c480a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:42.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.277+0000 7f6c4f154700 1 --2- 172.21.15.16:0/229797182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c4809d630 0x7f6c480a1340 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:42.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.277+0000 7f6c4f154700 1 -- 172.21.15.16:0/229797182 >> 172.21.15.16:0/229797182 conn(0x7f6c480a3050 msgr2=0x7f6c480a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:42.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.277+0000 7f6c4f154700 1 -- 172.21.15.16:0/229797182 shutdown_connections 2023-09-30T22:35:42.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.277+0000 7f6c4f154700 1 -- 172.21.15.16:0/229797182 wait complete. 2023-09-30T22:35:42.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.278+0000 7f6c4f154700 1 Processor -- start 2023-09-30T22:35:42.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.278+0000 7f6c4f154700 1 -- start start 2023-09-30T22:35:42.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.279+0000 7f6c4f154700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6c4809d630 0x7f6c4811f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:42.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.279+0000 7f6c4f154700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c480a0a30 0x7f6c4811f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:42.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.279+0000 7f6c4f154700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6c48133930 0x7f6c481c8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:42.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.279+0000 7f6c4f154700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c481223a0 con 0x7f6c480a0a30 2023-09-30T22:35:42.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.279+0000 7f6c4f154700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c48122580 con 0x7f6c4809d630 2023-09-30T22:35:42.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.279+0000 7f6c4f154700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6c481333f0 con 0x7f6c48133930 2023-09-30T22:35:42.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.279+0000 7f6c47fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c480a0a30 0x7f6c4811f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:42.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.279+0000 7f6c47fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c480a0a30 0x7f6c4811f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40858/0 (socket says 172.21.15.16:40858) 2023-09-30T22:35:42.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.279+0000 7f6c47fff700 1 -- 172.21.15.16:0/404945567 learned_addr learned my addr 172.21.15.16:0/404945567 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:42.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.279+0000 7f6c4d6f1700 1 --2- 172.21.15.16:0/404945567 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6c48133930 0x7f6c481c8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:42.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.279+0000 7f6c4cef0700 1 --2- 172.21.15.16:0/404945567 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6c4809d630 0x7f6c4811f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:42.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.279+0000 7f6c47fff700 1 -- 172.21.15.16:0/404945567 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6c48133930 msgr2=0x7f6c481c8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:42.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.279+0000 7f6c47fff700 1 --2- 172.21.15.16:0/404945567 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6c48133930 0x7f6c481c8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:42.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.279+0000 7f6c47fff700 1 -- 172.21.15.16:0/404945567 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6c4809d630 msgr2=0x7f6c4811f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:42.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.279+0000 7f6c47fff700 1 --2- 172.21.15.16:0/404945567 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6c4809d630 0x7f6c4811f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:42.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.279+0000 7f6c47fff700 1 -- 172.21.15.16:0/404945567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c3c0094a0 con 0x7f6c480a0a30 2023-09-30T22:35:42.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.280+0000 7f6c47fff700 1 --2- 172.21.15.16:0/404945567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c480a0a30 0x7f6c4811f970 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c0090b0 tx=0x7f6c3c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:42.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.280+0000 7f6c457fa700 1 -- 172.21.15.16:0/404945567 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c004400 con 0x7f6c480a0a30 2023-09-30T22:35:42.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.280+0000 7f6c457fa700 1 -- 172.21.15.16:0/404945567 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c3c01f070 con 0x7f6c480a0a30 2023-09-30T22:35:42.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.280+0000 7f6c4f154700 1 -- 172.21.15.16:0/404945567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c4811feb0 con 0x7f6c480a0a30 2023-09-30T22:35:42.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.280+0000 7f6c457fa700 1 -- 172.21.15.16:0/404945567 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c0038e0 con 0x7f6c480a0a30 2023-09-30T22:35:42.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.280+0000 7f6c4f154700 1 -- 172.21.15.16:0/404945567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c4809d380 con 0x7f6c480a0a30 2023-09-30T22:35:42.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.281+0000 7f6c457fa700 1 -- 172.21.15.16:0/404945567 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6c3c021030 con 0x7f6c480a0a30 2023-09-30T22:35:42.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.281+0000 7f6c457fa700 1 -- 172.21.15.16:0/404945567 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f6c3c040780 con 0x7f6c480a0a30 2023-09-30T22:35:42.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.281+0000 7f6c4f154700 1 -- 172.21.15.16:0/404945567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c48120ec0 con 0x7f6c480a0a30 2023-09-30T22:35:42.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.286+0000 7f6c457fa700 1 -- 172.21.15.16:0/404945567 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6c3c004560 con 0x7f6c480a0a30 2023-09-30T22:35:42.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.356+0000 7fa30568c700 1 Processor -- start 2023-09-30T22:35:42.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.356+0000 7fa30568c700 1 -- start start 2023-09-30T22:35:42.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.357+0000 7fa30568c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa30009d6f0 0x7fa30009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:42.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.357+0000 7fa30568c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa30009e000 0x7fa3001339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:42.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.357+0000 7fa30568c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa30009e410 0x7fa300133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:42.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.357+0000 7fa30568c700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa30006b460 con 0x7fa30009e000 2023-09-30T22:35:42.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.357+0000 7fa30568c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa30009b120 con 0x7fa30009e410 2023-09-30T22:35:42.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.357+0000 7fa30568c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa30009a7e0 con 0x7fa30009d6f0 2023-09-30T22:35:42.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.357+0000 7fa2ff7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa30009e410 0x7fa300133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:42.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.357+0000 7fa2ff7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa30009e410 0x7fa300133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47660/0 (socket says 172.21.15.16:47660) 2023-09-30T22:35:42.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.357+0000 7fa2ff7fe700 1 -- 172.21.15.16:0/44971260 learned_addr learned my addr 172.21.15.16:0/44971260 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:42.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.357+0000 7fa2feffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa30009d6f0 0x7fa30009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:42.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.357+0000 7fa2fe7fc700 1 --2- 172.21.15.16:0/44971260 >> v2:172.21.15.16:3300/0 conn(0x7fa30009e000 0x7fa3001339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:42.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.358+0000 7fa2feffd700 1 -- 172.21.15.16:0/44971260 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa30009e410 msgr2=0x7fa300133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:42.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.358+0000 7fa2feffd700 1 --2- 172.21.15.16:0/44971260 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa30009e410 0x7fa300133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:42.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.358+0000 7fa2feffd700 1 -- 172.21.15.16:0/44971260 >> v2:172.21.15.16:3300/0 conn(0x7fa30009e000 msgr2=0x7fa3001339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:42.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.358+0000 7fa2feffd700 1 --2- 172.21.15.16:0/44971260 >> v2:172.21.15.16:3300/0 conn(0x7fa30009e000 0x7fa3001339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:42.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.358+0000 7fa2feffd700 1 -- 172.21.15.16:0/44971260 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa30009a500 con 0x7fa30009d6f0 2023-09-30T22:35:42.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.358+0000 7fa2ff7fe700 1 --2- 172.21.15.16:0/44971260 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa30009e410 0x7fa300133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:42.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.358+0000 7fa2feffd700 1 --2- 172.21.15.16:0/44971260 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa30009d6f0 0x7fa30009dac0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fa2f0009870 tx=0x7fa2f0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=515dd559894dd26e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:42.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.358+0000 7fa2fdffb700 1 -- 172.21.15.16:0/44971260 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f0011830 con 0x7fa30009d6f0 2023-09-30T22:35:42.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.358+0000 7fa2fdffb700 1 -- 172.21.15.16:0/44971260 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2f0011990 con 0x7fa30009d6f0 2023-09-30T22:35:42.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.358+0000 7fa2fdffb700 1 -- 172.21.15.16:0/44971260 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f0011d20 con 0x7fa30009d6f0 2023-09-30T22:35:42.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.359+0000 7fa30568c700 1 -- 172.21.15.16:0/44971260 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa30009d6f0 msgr2=0x7fa30009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:42.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.359+0000 7fa30568c700 1 --2- 172.21.15.16:0/44971260 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa30009d6f0 0x7fa30009dac0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fa2f0009870 tx=0x7fa2f0009b80 comp rx=0 tx=0).stop 2023-09-30T22:35:42.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.359+0000 7fa30568c700 1 -- 172.21.15.16:0/44971260 shutdown_connections 2023-09-30T22:35:42.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.359+0000 7fa30568c700 1 --2- 172.21.15.16:0/44971260 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa30009e410 0x7fa300133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:42.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.359+0000 7fa30568c700 1 --2- 172.21.15.16:0/44971260 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa30009d6f0 0x7fa30009dac0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:42.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.359+0000 7fa30568c700 1 --2- 172.21.15.16:0/44971260 >> v2:172.21.15.16:3300/0 conn(0x7fa30009e000 0x7fa3001339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:42.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.359+0000 7fa30568c700 1 -- 172.21.15.16:0/44971260 >> 172.21.15.16:0/44971260 conn(0x7fa300066f50 msgr2=0x7fa30009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:42.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.359+0000 7fa30568c700 1 -- 172.21.15.16:0/44971260 shutdown_connections 2023-09-30T22:35:42.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.359+0000 7fa30568c700 1 -- 172.21.15.16:0/44971260 wait complete. 2023-09-30T22:35:42.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.360+0000 7fa30568c700 1 Processor -- start 2023-09-30T22:35:42.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.361+0000 7fa30568c700 1 -- start start 2023-09-30T22:35:42.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.361+0000 7fa30568c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa30009d6f0 0x7fa3001a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:42.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.361+0000 7fa30568c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa30009e000 0x7fa3001a65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:42.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.361+0000 7fa30568c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa30009e410 0x7fa3001a6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:42.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.361+0000 7fa30568c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3001b93d0 con 0x7fa30009e410 2023-09-30T22:35:42.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.361+0000 7fa30568c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3001a8e40 con 0x7fa30009d6f0 2023-09-30T22:35:42.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.361+0000 7fa30568c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa3001a81d0 con 0x7fa30009e000 2023-09-30T22:35:42.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.361+0000 7fa2fe7fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa30009e000 0x7fa3001a65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:42.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.361+0000 7fa2ff7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa30009e410 0x7fa3001a6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:42.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.361+0000 7fa2fe7fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa30009e000 0x7fa3001a65b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:50714/0 (socket says 172.21.15.16:50714) 2023-09-30T22:35:42.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.361+0000 7fa2ff7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa30009e410 0x7fa3001a6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40874/0 (socket says 172.21.15.16:40874) 2023-09-30T22:35:42.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.361+0000 7fa2fe7fc700 1 -- 172.21.15.16:0/3487062473 learned_addr learned my addr 172.21.15.16:0/3487062473 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:42.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.361+0000 7fa2feffd700 1 --2- 172.21.15.16:0/3487062473 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa30009d6f0 0x7fa3001a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:42.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.362+0000 7fa2ff7fe700 1 -- 172.21.15.16:0/3487062473 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa30009e000 msgr2=0x7fa3001a65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:42.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.362+0000 7fa2ff7fe700 1 --2- 172.21.15.16:0/3487062473 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa30009e000 0x7fa3001a65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:42.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.362+0000 7fa2ff7fe700 1 -- 172.21.15.16:0/3487062473 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa30009d6f0 msgr2=0x7fa3001a6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:42.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.362+0000 7fa2ff7fe700 1 --2- 172.21.15.16:0/3487062473 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa30009d6f0 0x7fa3001a6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:42.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.362+0000 7fa2ff7fe700 1 -- 172.21.15.16:0/3487062473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2f8009b30 con 0x7fa30009e410 2023-09-30T22:35:42.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.362+0000 7fa2ff7fe700 1 --2- 172.21.15.16:0/3487062473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa30009e410 0x7fa3001a6af0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8009e50 tx=0x7fa2f800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:42.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.362+0000 7fa2e77fe700 1 -- 172.21.15.16:0/3487062473 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f8018070 con 0x7fa30009e410 2023-09-30T22:35:42.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.362+0000 7fa2e77fe700 1 -- 172.21.15.16:0/3487062473 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2f800f040 con 0x7fa30009e410 2023-09-30T22:35:42.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.362+0000 7fa30568c700 1 -- 172.21.15.16:0/3487062473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2f00094a0 con 0x7fa30009e410 2023-09-30T22:35:42.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.362+0000 7fa2e77fe700 1 -- 172.21.15.16:0/3487062473 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f8003c60 con 0x7fa30009e410 2023-09-30T22:35:42.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.362+0000 7fa30568c700 1 -- 172.21.15.16:0/3487062473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3000a21d0 con 0x7fa30009e410 2023-09-30T22:35:42.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.363+0000 7fa2e77fe700 1 -- 172.21.15.16:0/3487062473 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa2f800c880 con 0x7fa30009e410 2023-09-30T22:35:42.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.363+0000 7fa2e77fe700 1 -- 172.21.15.16:0/3487062473 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fa2f803f6e0 con 0x7fa30009e410 2023-09-30T22:35:42.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.363+0000 7fa30568c700 1 -- 172.21.15.16:0/3487062473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa300074ea0 con 0x7fa30009e410 2023-09-30T22:35:42.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.373+0000 7fa2e77fe700 1 -- 172.21.15.16:0/3487062473 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa2f8003dc0 con 0x7fa30009e410 2023-09-30T22:35:42.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.475+0000 7f7abc2b6700 1 -- 172.21.15.16:0/2231315421 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f7ab4074ea0 con 0x7f7ab409d6f0 2023-09-30T22:35:42.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.475+0000 7f7aaaffd700 1 -- 172.21.15.16:0/2231315421 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7ab001b340 con 0x7f7ab409d6f0 2023-09-30T22:35:42.476 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:35:42.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.479+0000 7f7abc2b6700 1 -- 172.21.15.16:0/2231315421 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7ab409d6f0 msgr2=0x7f7ab411b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:42.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.479+0000 7f7abc2b6700 1 --2- 172.21.15.16:0/2231315421 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7ab409d6f0 0x7f7ab411b520 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f7ab000c4a0 tx=0x7f7ab000c7b0 comp rx=0 tx=0).stop 2023-09-30T22:35:42.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.479+0000 7f7abc2b6700 1 -- 172.21.15.16:0/2231315421 shutdown_connections 2023-09-30T22:35:42.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.479+0000 7f7abc2b6700 1 --2- 172.21.15.16:0/2231315421 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7ab4135e50 0x7f7ab411bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:42.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.479+0000 7f7abc2b6700 1 --2- 172.21.15.16:0/2231315421 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7ab409d6f0 0x7f7ab411b520 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:42.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.479+0000 7f7abc2b6700 1 --2- 172.21.15.16:0/2231315421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ab409e1d0 0x7f7ab411ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:42.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.479+0000 7f7abc2b6700 1 -- 172.21.15.16:0/2231315421 >> 172.21.15.16:0/2231315421 conn(0x7f7ab4066f50 msgr2=0x7f7ab406e840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:42.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.479+0000 7f7abc2b6700 1 -- 172.21.15.16:0/2231315421 shutdown_connections 2023-09-30T22:35:42.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.479+0000 7f7abc2b6700 1 -- 172.21.15.16:0/2231315421 wait complete. 2023-09-30T22:35:42.494 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:35:42.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.514+0000 7f6c4f154700 1 -- 172.21.15.16:0/404945567 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f6c480747f0 con 0x7f6c480a0a30 2023-09-30T22:35:42.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.515+0000 7f6c457fa700 1 -- 172.21.15.16:0/404945567 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6c3c015020 con 0x7f6c480a0a30 2023-09-30T22:35:42.516 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:35:42.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.517+0000 7f6c4f154700 1 -- 172.21.15.16:0/404945567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c480a0a30 msgr2=0x7f6c4811f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:42.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.517+0000 7f6c4f154700 1 --2- 172.21.15.16:0/404945567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c480a0a30 0x7f6c4811f970 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c0090b0 tx=0x7f6c3c004750 comp rx=0 tx=0).stop 2023-09-30T22:35:42.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.517+0000 7f6c4f154700 1 -- 172.21.15.16:0/404945567 shutdown_connections 2023-09-30T22:35:42.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.517+0000 7f6c4f154700 1 --2- 172.21.15.16:0/404945567 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6c4809d630 0x7f6c4811f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:42.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.517+0000 7f6c4f154700 1 --2- 172.21.15.16:0/404945567 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6c48133930 0x7f6c481c8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:42.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.517+0000 7f6c4f154700 1 --2- 172.21.15.16:0/404945567 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c480a0a30 0x7f6c4811f970 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:42.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.517+0000 7f6c4f154700 1 -- 172.21.15.16:0/404945567 >> 172.21.15.16:0/404945567 conn(0x7f6c480a3050 msgr2=0x7f6c481bbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:42.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.517+0000 7f6c4f154700 1 -- 172.21.15.16:0/404945567 shutdown_connections 2023-09-30T22:35:42.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.517+0000 7f6c4f154700 1 -- 172.21.15.16:0/404945567 wait complete. 2023-09-30T22:35:42.532 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:35:42.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.579+0000 7fa30568c700 1 -- 172.21.15.16:0/3487062473 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa30006c360 con 0x7fa30009e410 2023-09-30T22:35:42.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.579+0000 7fa2e77fe700 1 -- 172.21.15.16:0/3487062473 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa2f801de00 con 0x7fa30009e410 2023-09-30T22:35:42.580 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:35:42.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.581+0000 7fa30568c700 1 -- 172.21.15.16:0/3487062473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa30009e410 msgr2=0x7fa3001a6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:42.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.581+0000 7fa30568c700 1 --2- 172.21.15.16:0/3487062473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa30009e410 0x7fa3001a6af0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fa2f8009e50 tx=0x7fa2f800c3a0 comp rx=0 tx=0).stop 2023-09-30T22:35:42.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.581+0000 7fa30568c700 1 -- 172.21.15.16:0/3487062473 shutdown_connections 2023-09-30T22:35:42.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.581+0000 7fa30568c700 1 --2- 172.21.15.16:0/3487062473 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa30009d6f0 0x7fa3001a6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:42.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.581+0000 7fa30568c700 1 --2- 172.21.15.16:0/3487062473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa30009e410 0x7fa3001a6af0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:42.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.581+0000 7fa30568c700 1 --2- 172.21.15.16:0/3487062473 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa30009e000 0x7fa3001a65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:42.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.581+0000 7fa30568c700 1 -- 172.21.15.16:0/3487062473 >> 172.21.15.16:0/3487062473 conn(0x7fa300066f50 msgr2=0x7fa3001bad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:42.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.581+0000 7fa30568c700 1 -- 172.21.15.16:0/3487062473 shutdown_connections 2023-09-30T22:35:42.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:42.581+0000 7fa30568c700 1 -- 172.21.15.16:0/3487062473 wait complete. 2023-09-30T22:35:42.596 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:35:42.857 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:35:43.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.002+0000 7fa85e28f700 1 Processor -- start 2023-09-30T22:35:43.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.002+0000 7fa85e28f700 1 -- start start 2023-09-30T22:35:43.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.002+0000 7fa85e28f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa8580a0a30 0x7fa8580a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:43.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.002+0000 7fa85e28f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa858134170 0x7fa85813bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:43.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.002+0000 7fa85e28f700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa85809edd0 con 0x7fa8580a1340 2023-09-30T22:35:43.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.002+0000 7fa85e28f700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa85809e1f0 con 0x7fa858134170 2023-09-30T22:35:43.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.002+0000 7fa85e28f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa85809dc50 con 0x7fa8580a0a30 2023-09-30T22:35:43.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.003+0000 7fa85c82c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa858134170 0x7fa85813bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:43.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.003+0000 7fa857fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa8580a0a30 0x7fa8580a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:43.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.003+0000 7fa85c82c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa858134170 0x7fa85813bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40880/0 (socket says 172.21.15.16:40880) 2023-09-30T22:35:43.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.003+0000 7fa85c82c700 1 -- 172.21.15.16:0/1357068417 learned_addr learned my addr 172.21.15.16:0/1357068417 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:43.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.003+0000 7fa8577fe700 1 --1- 172.21.15.16:0/1357068417 >> v1:172.21.15.16:6789/0 conn(0x7fa8580a1340 0x7fa858133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40602/0 (socket says 172.21.15.16:40602) 2023-09-30T22:35:43.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.003+0000 7fa857fff700 1 -- 172.21.15.16:0/1357068417 >> v2:172.21.15.16:3300/0 conn(0x7fa858134170 msgr2=0x7fa85813bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:43.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.003+0000 7fa857fff700 1 --2- 172.21.15.16:0/1357068417 >> v2:172.21.15.16:3300/0 conn(0x7fa858134170 0x7fa85813bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:43.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.003+0000 7fa857fff700 1 -- 172.21.15.16:0/1357068417 >> v1:172.21.15.16:6789/0 conn(0x7fa8580a1340 legacy=0x7fa858133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:43.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.003+0000 7fa857fff700 1 -- 172.21.15.16:0/1357068417 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8580a1fe0 con 0x7fa8580a0a30 2023-09-30T22:35:43.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.003+0000 7fa856ffd700 1 -- 172.21.15.16:0/1357068417 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2201445037 0 0) 0x7fa85809edd0 con 0x7fa8580a1340 2023-09-30T22:35:43.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.003+0000 7fa85c82c700 1 --2- 172.21.15.16:0/1357068417 >> v2:172.21.15.16:3300/0 conn(0x7fa858134170 0x7fa85813bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:43.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.003+0000 7fa857fff700 1 --2- 172.21.15.16:0/1357068417 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa8580a0a30 0x7fa8580a0e00 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fa848009840 tx=0x7fa848009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1429650e02adb661 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:43.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.003+0000 7fa856ffd700 1 -- 172.21.15.16:0/1357068417 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa848011830 con 0x7fa8580a0a30 2023-09-30T22:35:43.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.003+0000 7fa856ffd700 1 -- 172.21.15.16:0/1357068417 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa848011990 con 0x7fa8580a0a30 2023-09-30T22:35:43.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.003+0000 7fa856ffd700 1 -- 172.21.15.16:0/1357068417 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa848011d20 con 0x7fa8580a0a30 2023-09-30T22:35:43.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.004+0000 7fa85e28f700 1 -- 172.21.15.16:0/1357068417 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa8580a0a30 msgr2=0x7fa8580a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:43.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.004+0000 7fa85e28f700 1 --2- 172.21.15.16:0/1357068417 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa8580a0a30 0x7fa8580a0e00 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fa848009840 tx=0x7fa848009b50 comp rx=0 tx=0).stop 2023-09-30T22:35:43.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.004+0000 7fa85e28f700 1 -- 172.21.15.16:0/1357068417 shutdown_connections 2023-09-30T22:35:43.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.004+0000 7fa85e28f700 1 --2- 172.21.15.16:0/1357068417 >> v2:172.21.15.16:3300/0 conn(0x7fa858134170 0x7fa85813bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:43.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.004+0000 7fa85e28f700 1 --2- 172.21.15.16:0/1357068417 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa8580a0a30 0x7fa8580a0e00 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:43.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.004+0000 7fa85e28f700 1 -- 172.21.15.16:0/1357068417 >> 172.21.15.16:0/1357068417 conn(0x7fa8580a6550 msgr2=0x7fa8580a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:43.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.004+0000 7fa85e28f700 1 -- 172.21.15.16:0/1357068417 shutdown_connections 2023-09-30T22:35:43.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.004+0000 7fa85e28f700 1 -- 172.21.15.16:0/1357068417 wait complete. 2023-09-30T22:35:43.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.005+0000 7fa85e28f700 1 Processor -- start 2023-09-30T22:35:43.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.005+0000 7fa85e28f700 1 -- start start 2023-09-30T22:35:43.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.006+0000 7fa85e28f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8580a0a30 0x7fa858123240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:43.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.006+0000 7fa85e28f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa8580a1340 0x7fa858123780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:43.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.006+0000 7fa85e28f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa858134170 0x7fa8581c8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:43.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.006+0000 7fa85e28f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa858125ef0 con 0x7fa8580a0a30 2023-09-30T22:35:43.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.006+0000 7fa85e28f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa858126120 con 0x7fa858134170 2023-09-30T22:35:43.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.006+0000 7fa85e28f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa8581244f0 con 0x7fa8580a1340 2023-09-30T22:35:43.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.006+0000 7fa857fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8580a0a30 0x7fa858123240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:43.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.006+0000 7fa857fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8580a0a30 0x7fa858123240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40890/0 (socket says 172.21.15.16:40890) 2023-09-30T22:35:43.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.006+0000 7fa857fff700 1 -- 172.21.15.16:0/3629165971 learned_addr learned my addr 172.21.15.16:0/3629165971 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:43.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.006+0000 7fa85c82c700 1 --2- 172.21.15.16:0/3629165971 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa858134170 0x7fa8581c8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:43.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.006+0000 7fa8577fe700 1 --2- 172.21.15.16:0/3629165971 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa8580a1340 0x7fa858123780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:43.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.006+0000 7fa857fff700 1 -- 172.21.15.16:0/3629165971 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa8580a1340 msgr2=0x7fa858123780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:43.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.006+0000 7fa857fff700 1 --2- 172.21.15.16:0/3629165971 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa8580a1340 0x7fa858123780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:43.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.006+0000 7fa857fff700 1 -- 172.21.15.16:0/3629165971 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa858134170 msgr2=0x7fa8581c8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:43.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.006+0000 7fa857fff700 1 --2- 172.21.15.16:0/3629165971 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa858134170 0x7fa8581c8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:43.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.006+0000 7fa857fff700 1 -- 172.21.15.16:0/3629165971 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa850009b30 con 0x7fa8580a0a30 2023-09-30T22:35:43.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.006+0000 7fa85c82c700 1 --2- 172.21.15.16:0/3629165971 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa858134170 0x7fa8581c8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:43.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.006+0000 7fa857fff700 1 --2- 172.21.15.16:0/3629165971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8580a0a30 0x7fa858123240 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fa848009fd0 tx=0x7fa848004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:43.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.007+0000 7fa854ff9700 1 -- 172.21.15.16:0/3629165971 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8480036a0 con 0x7fa8580a0a30 2023-09-30T22:35:43.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.007+0000 7fa854ff9700 1 -- 172.21.15.16:0/3629165971 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa84801f070 con 0x7fa8580a0a30 2023-09-30T22:35:43.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.007+0000 7fa85e28f700 1 -- 172.21.15.16:0/3629165971 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8480094a0 con 0x7fa8580a0a30 2023-09-30T22:35:43.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.007+0000 7fa854ff9700 1 -- 172.21.15.16:0/3629165971 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa848003a60 con 0x7fa8580a0a30 2023-09-30T22:35:43.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.007+0000 7fa85e28f700 1 -- 172.21.15.16:0/3629165971 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa858124ce0 con 0x7fa8580a0a30 2023-09-30T22:35:43.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.007+0000 7fa854ff9700 1 -- 172.21.15.16:0/3629165971 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa848003bc0 con 0x7fa8580a0a30 2023-09-30T22:35:43.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.008+0000 7fa854ff9700 1 -- 172.21.15.16:0/3629165971 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fa848021070 con 0x7fa8580a0a30 2023-09-30T22:35:43.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.008+0000 7fa85e28f700 1 -- 172.21.15.16:0/3629165971 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8580a1130 con 0x7fa8580a0a30 2023-09-30T22:35:43.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.012+0000 7fa854ff9700 1 -- 172.21.15.16:0/3629165971 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa848015020 con 0x7fa8580a0a30 2023-09-30T22:35:43.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.221+0000 7fa85e28f700 1 -- 172.21.15.16:0/3629165971 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa858074ea0 con 0x7fa8580a0a30 2023-09-30T22:35:43.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.221+0000 7fa854ff9700 1 -- 172.21.15.16:0/3629165971 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa8480253e0 con 0x7fa8580a0a30 2023-09-30T22:35:43.222 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:35:43.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.223+0000 7fa85e28f700 1 -- 172.21.15.16:0/3629165971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8580a0a30 msgr2=0x7fa858123240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:43.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.223+0000 7fa85e28f700 1 --2- 172.21.15.16:0/3629165971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8580a0a30 0x7fa858123240 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fa848009fd0 tx=0x7fa848004750 comp rx=0 tx=0).stop 2023-09-30T22:35:43.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.223+0000 7fa85e28f700 1 -- 172.21.15.16:0/3629165971 shutdown_connections 2023-09-30T22:35:43.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.223+0000 7fa85e28f700 1 --2- 172.21.15.16:0/3629165971 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa858134170 0x7fa8581c8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:43.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.223+0000 7fa85e28f700 1 --2- 172.21.15.16:0/3629165971 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa8580a0a30 0x7fa858123240 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:43.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.223+0000 7fa85e28f700 1 --2- 172.21.15.16:0/3629165971 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa8580a1340 0x7fa858123780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:43.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.223+0000 7fa85e28f700 1 -- 172.21.15.16:0/3629165971 >> 172.21.15.16:0/3629165971 conn(0x7fa8580a6550 msgr2=0x7fa85809edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:43.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.223+0000 7fa85e28f700 1 -- 172.21.15.16:0/3629165971 shutdown_connections 2023-09-30T22:35:43.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.223+0000 7fa85e28f700 1 -- 172.21.15.16:0/3629165971 wait complete. 2023-09-30T22:35:43.238 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:35:43.496 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:35:43.533 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:35:43.597 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:35:43.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.648+0000 7fcafc7dd700 1 Processor -- start 2023-09-30T22:35:43.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.648+0000 7fcafc7dd700 1 -- start start 2023-09-30T22:35:43.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.650+0000 7fcafc7dd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcaf409e1d0 0x7fcaf4138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:43.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.650+0000 7fcafc7dd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcaf4139170 0x7fcaf413b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:43.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.650+0000 7fcafc7dd700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaf409b120 con 0x7fcaf409d6f0 2023-09-30T22:35:43.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.650+0000 7fcafc7dd700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaf406b460 con 0x7fcaf4139170 2023-09-30T22:35:43.653 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.650+0000 7fcafc7dd700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf409a7e0 con 0x7fcaf409e1d0 2023-09-30T22:35:43.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.650+0000 7fcaf9d78700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcaf409e1d0 0x7fcaf4138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:43.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.650+0000 7fcafa579700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fcaf409d6f0 0x7fcaf409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40614/0 (socket says 172.21.15.16:40614) 2023-09-30T22:35:43.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.650+0000 7fcafa579700 1 -- 172.21.15.16:0/1523528619 learned_addr learned my addr 172.21.15.16:0/1523528619 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:43.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.650+0000 7fcafad7a700 1 --2- 172.21.15.16:0/1523528619 >> v2:172.21.15.16:3300/0 conn(0x7fcaf4139170 0x7fcaf413b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:43.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.650+0000 7fcaf9d78700 1 -- 172.21.15.16:0/1523528619 >> v2:172.21.15.16:3300/0 conn(0x7fcaf4139170 msgr2=0x7fcaf413b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:43.654 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.650+0000 7fcaf9d78700 1 --2- 172.21.15.16:0/1523528619 >> v2:172.21.15.16:3300/0 conn(0x7fcaf4139170 0x7fcaf413b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:43.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.650+0000 7fcaf9d78700 1 -- 172.21.15.16:0/1523528619 >> v1:172.21.15.16:6789/0 conn(0x7fcaf409d6f0 legacy=0x7fcaf409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:43.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.650+0000 7fcaf9d78700 1 -- 172.21.15.16:0/1523528619 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf409a500 con 0x7fcaf409e1d0 2023-09-30T22:35:43.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.650+0000 7fcafad7a700 1 --2- 172.21.15.16:0/1523528619 >> v2:172.21.15.16:3300/0 conn(0x7fcaf4139170 0x7fcaf413b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:43.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.650+0000 7fcaf9d78700 1 --2- 172.21.15.16:0/1523528619 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcaf409e1d0 0x7fcaf4138c30 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fcae4009870 tx=0x7fcae4009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c217a5ea34e1619 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:43.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.651+0000 7fcaf9577700 1 -- 172.21.15.16:0/1523528619 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae4011830 con 0x7fcaf409e1d0 2023-09-30T22:35:43.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.651+0000 7fcaf9577700 1 -- 172.21.15.16:0/1523528619 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcae4011990 con 0x7fcaf409e1d0 2023-09-30T22:35:43.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.651+0000 7fcaf9577700 1 -- 172.21.15.16:0/1523528619 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae4011d20 con 0x7fcaf409e1d0 2023-09-30T22:35:43.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.651+0000 7fcafc7dd700 1 -- 172.21.15.16:0/1523528619 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcaf409e1d0 msgr2=0x7fcaf4138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:43.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.651+0000 7fcafc7dd700 1 --2- 172.21.15.16:0/1523528619 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcaf409e1d0 0x7fcaf4138c30 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fcae4009870 tx=0x7fcae4009b80 comp rx=0 tx=0).stop 2023-09-30T22:35:43.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.651+0000 7fcafc7dd700 1 -- 172.21.15.16:0/1523528619 shutdown_connections 2023-09-30T22:35:43.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.651+0000 7fcafc7dd700 1 --2- 172.21.15.16:0/1523528619 >> v2:172.21.15.16:3300/0 conn(0x7fcaf4139170 0x7fcaf413b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:43.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.651+0000 7fcafc7dd700 1 --2- 172.21.15.16:0/1523528619 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcaf409e1d0 0x7fcaf4138c30 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:43.656 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.651+0000 7fcafc7dd700 1 -- 172.21.15.16:0/1523528619 >> 172.21.15.16:0/1523528619 conn(0x7fcaf4066f50 msgr2=0x7fcaf409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:43.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.651+0000 7fcafc7dd700 1 -- 172.21.15.16:0/1523528619 shutdown_connections 2023-09-30T22:35:43.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.652+0000 7fcafc7dd700 1 -- 172.21.15.16:0/1523528619 wait complete. 2023-09-30T22:35:43.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.653+0000 7fcafc7dd700 1 Processor -- start 2023-09-30T22:35:43.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.653+0000 7fcafc7dd700 1 -- start start 2023-09-30T22:35:43.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.654+0000 7fcafc7dd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcaf409d6f0 0x7fcaf41c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:43.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.654+0000 7fcafc7dd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcaf409e1d0 0x7fcaf41c8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:43.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.654+0000 7fcafc7dd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcaf4139170 0x7fcaf411e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:43.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.654+0000 7fcafc7dd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf41220a0 con 0x7fcaf409e1d0 2023-09-30T22:35:43.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.654+0000 7fcafc7dd700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf4122280 con 0x7fcaf409d6f0 2023-09-30T22:35:43.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.654+0000 7fcafc7dd700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcaf41be250 con 0x7fcaf4139170 2023-09-30T22:35:43.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.654+0000 7fcafa579700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcaf409d6f0 0x7fcaf41c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:43.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.654+0000 7fcafa579700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcaf409d6f0 0x7fcaf41c86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47720/0 (socket says 172.21.15.16:47720) 2023-09-30T22:35:43.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.654+0000 7fcafa579700 1 -- 172.21.15.16:0/975761837 learned_addr learned my addr 172.21.15.16:0/975761837 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:43.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.654+0000 7fcaf9d78700 1 --2- 172.21.15.16:0/975761837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcaf409e1d0 0x7fcaf41c8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:43.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.654+0000 7fcafad7a700 1 --2- 172.21.15.16:0/975761837 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcaf4139170 0x7fcaf411e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:43.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.654+0000 7fcafa579700 1 -- 172.21.15.16:0/975761837 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcaf4139170 msgr2=0x7fcaf411e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:43.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.654+0000 7fcafa579700 1 --2- 172.21.15.16:0/975761837 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcaf4139170 0x7fcaf411e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:43.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.654+0000 7fcafa579700 1 -- 172.21.15.16:0/975761837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcaf409e1d0 msgr2=0x7fcaf41c8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:43.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.654+0000 7fcafa579700 1 --2- 172.21.15.16:0/975761837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcaf409e1d0 0x7fcaf41c8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:43.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.654+0000 7fcafa579700 1 -- 172.21.15.16:0/975761837 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaec009a70 con 0x7fcaf409d6f0 2023-09-30T22:35:43.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.654+0000 7fcaf9d78700 1 --2- 172.21.15.16:0/975761837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcaf409e1d0 0x7fcaf41c8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:43.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.655+0000 7fcafa579700 1 --2- 172.21.15.16:0/975761837 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcaf409d6f0 0x7fcaf41c86b0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fcaf000c4a0 tx=0x7fcaf000c7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:43.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.655+0000 7fcaeaffd700 1 -- 172.21.15.16:0/975761837 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf000cea0 con 0x7fcaf409d6f0 2023-09-30T22:35:43.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.655+0000 7fcaeaffd700 1 -- 172.21.15.16:0/975761837 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaf0005d70 con 0x7fcaf409d6f0 2023-09-30T22:35:43.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.655+0000 7fcaeaffd700 1 -- 172.21.15.16:0/975761837 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf00072d0 con 0x7fcaf409d6f0 2023-09-30T22:35:43.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.655+0000 7fcafc7dd700 1 -- 172.21.15.16:0/975761837 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcae40094a0 con 0x7fcaf409d6f0 2023-09-30T22:35:43.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.655+0000 7fcafc7dd700 1 -- 172.21.15.16:0/975761837 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaf40a29f0 con 0x7fcaf409d6f0 2023-09-30T22:35:43.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.656+0000 7fcaeaffd700 1 -- 172.21.15.16:0/975761837 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcaf0007430 con 0x7fcaf409d6f0 2023-09-30T22:35:43.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.656+0000 7fcaeaffd700 1 -- 172.21.15.16:0/975761837 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fcaf003e950 con 0x7fcaf409d6f0 2023-09-30T22:35:43.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.656+0000 7fcafc7dd700 1 -- 172.21.15.16:0/975761837 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaf4121ee0 con 0x7fcaf409d6f0 2023-09-30T22:35:43.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.661+0000 7fcaeaffd700 1 -- 172.21.15.16:0/975761837 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcaf4121ee0 con 0x7fcaf409d6f0 2023-09-30T22:35:43.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.686+0000 7f3f8b57e700 1 Processor -- start 2023-09-30T22:35:43.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.687+0000 7f3f8b57e700 1 -- start start 2023-09-30T22:35:43.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.687+0000 7f3f8b57e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3f8409f410 0x7f3f84136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:43.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.687+0000 7f3f8b57e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3f84136fe0 0x7f3f841393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:43.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.687+0000 7f3f8b57e700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f84070940 con 0x7f3f8409e930 2023-09-30T22:35:43.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.687+0000 7f3f8b57e700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f8406fd60 con 0x7f3f84136fe0 2023-09-30T22:35:43.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.687+0000 7f3f8b57e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8406fa80 con 0x7f3f8409f410 2023-09-30T22:35:43.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.687+0000 7f3f88b19700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3f8409f410 0x7f3f84136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:43.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.687+0000 7f3f88b19700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3f8409f410 0x7f3f84136aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47732/0 (socket says 172.21.15.16:47732) 2023-09-30T22:35:43.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.687+0000 7f3f88b19700 1 -- 172.21.15.16:0/3414878515 learned_addr learned my addr 172.21.15.16:0/3414878515 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:43.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.687+0000 7f3f89b1b700 1 --2- 172.21.15.16:0/3414878515 >> v2:172.21.15.16:3300/0 conn(0x7f3f84136fe0 0x7f3f841393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:43.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.688+0000 7f3f88b19700 1 -- 172.21.15.16:0/3414878515 >> v2:172.21.15.16:3300/0 conn(0x7f3f84136fe0 msgr2=0x7f3f841393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:43.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.688+0000 7f3f88b19700 1 --2- 172.21.15.16:0/3414878515 >> v2:172.21.15.16:3300/0 conn(0x7f3f84136fe0 0x7f3f841393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:43.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.688+0000 7f3f88b19700 1 -- 172.21.15.16:0/3414878515 >> v1:172.21.15.16:6789/0 conn(0x7f3f8409e930 legacy=0x7f3f8409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:43.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.688+0000 7f3f88b19700 1 -- 172.21.15.16:0/3414878515 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f8409fb60 con 0x7f3f8409f410 2023-09-30T22:35:43.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.688+0000 7f3f88b19700 1 --2- 172.21.15.16:0/3414878515 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3f8409f410 0x7f3f84136aa0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f3f740098f0 tx=0x7f3f74009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbaf8f8cd7381797 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:43.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.688+0000 7f3f7bfff700 1 -- 172.21.15.16:0/3414878515 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f74011830 con 0x7f3f8409f410 2023-09-30T22:35:43.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.688+0000 7f3f7bfff700 1 -- 172.21.15.16:0/3414878515 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f74011990 con 0x7f3f8409f410 2023-09-30T22:35:43.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.688+0000 7f3f7bfff700 1 -- 172.21.15.16:0/3414878515 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f74011d20 con 0x7f3f8409f410 2023-09-30T22:35:43.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.689+0000 7f3f8b57e700 1 -- 172.21.15.16:0/3414878515 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3f8409f410 msgr2=0x7f3f84136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:43.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.689+0000 7f3f8b57e700 1 --2- 172.21.15.16:0/3414878515 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3f8409f410 0x7f3f84136aa0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f3f740098f0 tx=0x7f3f74009c00 comp rx=0 tx=0).stop 2023-09-30T22:35:43.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.689+0000 7f3f8b57e700 1 -- 172.21.15.16:0/3414878515 shutdown_connections 2023-09-30T22:35:43.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.689+0000 7f3f8b57e700 1 --2- 172.21.15.16:0/3414878515 >> v2:172.21.15.16:3300/0 conn(0x7f3f84136fe0 0x7f3f841393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:43.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.689+0000 7f3f8b57e700 1 --2- 172.21.15.16:0/3414878515 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3f8409f410 0x7f3f84136aa0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:43.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.689+0000 7f3f8b57e700 1 -- 172.21.15.16:0/3414878515 >> 172.21.15.16:0/3414878515 conn(0x7f3f840a4210 msgr2=0x7f3f840a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:43.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.689+0000 7f3f8b57e700 1 -- 172.21.15.16:0/3414878515 shutdown_connections 2023-09-30T22:35:43.692 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.689+0000 7f3f8b57e700 1 -- 172.21.15.16:0/3414878515 wait complete. 2023-09-30T22:35:43.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.690+0000 7f3f8b57e700 1 Processor -- start 2023-09-30T22:35:43.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.690+0000 7f3f8b57e700 1 -- start start 2023-09-30T22:35:43.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.690+0000 7f3f8b57e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f8409e930 0x7f3f8406c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:43.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.690+0000 7f3f8b57e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3f8409f410 0x7f3f8406c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:43.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.690+0000 7f3f8b57e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3f84136fe0 0x7f3f841c8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:43.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.691+0000 7f3f8b57e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8406bd80 con 0x7f3f8409e930 2023-09-30T22:35:43.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.691+0000 7f3f8b57e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f841321b0 con 0x7f3f8409f410 2023-09-30T22:35:43.693 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.691+0000 7f3f8b57e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3f84071d20 con 0x7f3f84136fe0 2023-09-30T22:35:43.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.691+0000 7f3f88b19700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3f8409f410 0x7f3f8406c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:43.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.691+0000 7f3f88b19700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3f8409f410 0x7f3f8406c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47734/0 (socket says 172.21.15.16:47734) 2023-09-30T22:35:43.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.691+0000 7f3f88b19700 1 -- 172.21.15.16:0/680012819 learned_addr learned my addr 172.21.15.16:0/680012819 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:43.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.691+0000 7f3f89b1b700 1 --2- 172.21.15.16:0/680012819 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3f84136fe0 0x7f3f841c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:43.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.691+0000 7f3f8931a700 1 --2- 172.21.15.16:0/680012819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f8409e930 0x7f3f8406c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:43.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.691+0000 7f3f88b19700 1 -- 172.21.15.16:0/680012819 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3f84136fe0 msgr2=0x7f3f841c8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:43.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.691+0000 7f3f88b19700 1 --2- 172.21.15.16:0/680012819 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3f84136fe0 0x7f3f841c8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:43.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.691+0000 7f3f88b19700 1 -- 172.21.15.16:0/680012819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f8409e930 msgr2=0x7f3f8406c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:43.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.691+0000 7f3f88b19700 1 --2- 172.21.15.16:0/680012819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f8409e930 0x7f3f8406c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:43.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.691+0000 7f3f88b19700 1 -- 172.21.15.16:0/680012819 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f74009550 con 0x7f3f8409f410 2023-09-30T22:35:43.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.691+0000 7f3f88b19700 1 --2- 172.21.15.16:0/680012819 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3f8409f410 0x7f3f8406c8b0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f3f74000c00 tx=0x7f3f74004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:43.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.691+0000 7f3f79ffb700 1 -- 172.21.15.16:0/680012819 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f74004400 con 0x7f3f8409f410 2023-09-30T22:35:43.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.691+0000 7f3f79ffb700 1 -- 172.21.15.16:0/680012819 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f7401f070 con 0x7f3f8409f410 2023-09-30T22:35:43.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.692+0000 7f3f8b57e700 1 -- 172.21.15.16:0/680012819 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f84122450 con 0x7f3f8409f410 2023-09-30T22:35:43.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.692+0000 7f3f79ffb700 1 -- 172.21.15.16:0/680012819 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f740038e0 con 0x7f3f8409f410 2023-09-30T22:35:43.695 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.692+0000 7f3f8b57e700 1 -- 172.21.15.16:0/680012819 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f84121db0 con 0x7f3f8409f410 2023-09-30T22:35:43.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.693+0000 7f3f79ffb700 1 -- 172.21.15.16:0/680012819 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3f74021030 con 0x7f3f8409f410 2023-09-30T22:35:43.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.693+0000 7f3f79ffb700 1 -- 172.21.15.16:0/680012819 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f3f74040990 con 0x7f3f8409f410 2023-09-30T22:35:43.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.693+0000 7f3f8b57e700 1 -- 172.21.15.16:0/680012819 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f841222b0 con 0x7f3f8409f410 2023-09-30T22:35:43.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.703+0000 7f3f79ffb700 1 -- 172.21.15.16:0/680012819 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3f841222b0 con 0x7f3f8409f410 2023-09-30T22:35:43.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.743+0000 7feab202b700 1 Processor -- start 2023-09-30T22:35:43.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.743+0000 7feab202b700 1 -- start start 2023-09-30T22:35:43.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.744+0000 7feab202b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7feaac09d6f0 0x7feaac09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:43.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.744+0000 7feab202b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7feaac09e000 0x7feaac130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:43.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.744+0000 7feab202b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaac09f130 con 0x7feaac09e410 2023-09-30T22:35:43.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.744+0000 7feab202b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaac09f520 con 0x7feaac09d6f0 2023-09-30T22:35:43.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.744+0000 7feab202b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feaac09e930 con 0x7feaac09e000 2023-09-30T22:35:43.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.744+0000 7feaabfff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7feaac09e410 0x7feaac130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40632/0 (socket says 172.21.15.16:40632) 2023-09-30T22:35:43.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.744+0000 7feaabfff700 1 -- 172.21.15.16:0/3315166006 learned_addr learned my addr 172.21.15.16:0/3315166006 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:43.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.744+0000 7feaab7fe700 1 --2- 172.21.15.16:0/3315166006 >> v2:172.21.15.16:3300/0 conn(0x7feaac09d6f0 0x7feaac09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:43.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.744+0000 7feaaaffd700 1 --2- 172.21.15.16:0/3315166006 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7feaac09e000 0x7feaac130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:43.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.745+0000 7feaab7fe700 1 -- 172.21.15.16:0/3315166006 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7feaac09e000 msgr2=0x7feaac130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:43.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.745+0000 7feaab7fe700 1 --2- 172.21.15.16:0/3315166006 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7feaac09e000 0x7feaac130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:43.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.745+0000 7feaaa7fc700 1 -- 172.21.15.16:0/3315166006 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 502856440 0 0) 0x7feaac09f130 con 0x7feaac09e410 2023-09-30T22:35:43.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.745+0000 7feaab7fe700 1 -- 172.21.15.16:0/3315166006 >> v1:172.21.15.16:6789/0 conn(0x7feaac09e410 legacy=0x7feaac130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:43.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.745+0000 7feaab7fe700 1 -- 172.21.15.16:0/3315166006 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaac09b2d0 con 0x7feaac09d6f0 2023-09-30T22:35:43.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.745+0000 7feaab7fe700 1 --2- 172.21.15.16:0/3315166006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feaac09d6f0 0x7feaac09dac0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7feaa4009d90 tx=0x7feaa400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0c3e3795609b021 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:43.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.745+0000 7feaaa7fc700 1 -- 172.21.15.16:0/3315166006 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa400e070 con 0x7feaac09d6f0 2023-09-30T22:35:43.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.745+0000 7feaaa7fc700 1 -- 172.21.15.16:0/3315166006 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaa400bb20 con 0x7feaac09d6f0 2023-09-30T22:35:43.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.745+0000 7feaaa7fc700 1 -- 172.21.15.16:0/3315166006 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa4012430 con 0x7feaac09d6f0 2023-09-30T22:35:43.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.746+0000 7feab202b700 1 -- 172.21.15.16:0/3315166006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feaac09d6f0 msgr2=0x7feaac09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:43.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.746+0000 7feab202b700 1 --2- 172.21.15.16:0/3315166006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feaac09d6f0 0x7feaac09dac0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7feaa4009d90 tx=0x7feaa400b330 comp rx=0 tx=0).stop 2023-09-30T22:35:43.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.746+0000 7feab202b700 1 -- 172.21.15.16:0/3315166006 shutdown_connections 2023-09-30T22:35:43.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.746+0000 7feab202b700 1 --2- 172.21.15.16:0/3315166006 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feaac09d6f0 0x7feaac09dac0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:43.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.746+0000 7feab202b700 1 --2- 172.21.15.16:0/3315166006 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7feaac09e000 0x7feaac130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:43.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.746+0000 7feab202b700 1 -- 172.21.15.16:0/3315166006 >> 172.21.15.16:0/3315166006 conn(0x7feaac066f50 msgr2=0x7feaac09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:43.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.746+0000 7feab202b700 1 -- 172.21.15.16:0/3315166006 shutdown_connections 2023-09-30T22:35:43.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.746+0000 7feab202b700 1 -- 172.21.15.16:0/3315166006 wait complete. 2023-09-30T22:35:43.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.747+0000 7feab202b700 1 Processor -- start 2023-09-30T22:35:43.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.747+0000 7feab202b700 1 -- start start 2023-09-30T22:35:43.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.748+0000 7feab202b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7feaac09d6f0 0x7feaac131630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:43.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.748+0000 7feab202b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7feaac09e000 0x7feaac131b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:43.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.748+0000 7feab202b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feaac09e410 0x7feaac1320b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:43.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.748+0000 7feab202b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaac130fa0 con 0x7feaac09e410 2023-09-30T22:35:43.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.748+0000 7feab202b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaac133340 con 0x7feaac09d6f0 2023-09-30T22:35:43.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.748+0000 7feab202b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7feaac135370 con 0x7feaac09e000 2023-09-30T22:35:43.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.748+0000 7feaab7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7feaac09d6f0 0x7feaac131630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:43.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.748+0000 7feaab7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7feaac09d6f0 0x7feaac131630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47748/0 (socket says 172.21.15.16:47748) 2023-09-30T22:35:43.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.748+0000 7feaab7fe700 1 -- 172.21.15.16:0/1202632770 learned_addr learned my addr 172.21.15.16:0/1202632770 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:43.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.748+0000 7feaaaffd700 1 --2- 172.21.15.16:0/1202632770 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7feaac09e000 0x7feaac131b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:43.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.748+0000 7feaabfff700 1 --2- 172.21.15.16:0/1202632770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feaac09e410 0x7feaac1320b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:43.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.748+0000 7feaab7fe700 1 -- 172.21.15.16:0/1202632770 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7feaac09e000 msgr2=0x7feaac131b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:43.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.748+0000 7feaab7fe700 1 --2- 172.21.15.16:0/1202632770 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7feaac09e000 0x7feaac131b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:43.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.748+0000 7feaab7fe700 1 -- 172.21.15.16:0/1202632770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feaac09e410 msgr2=0x7feaac1320b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:43.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.748+0000 7feaab7fe700 1 --2- 172.21.15.16:0/1202632770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feaac09e410 0x7feaac1320b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:43.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.748+0000 7feaab7fe700 1 -- 172.21.15.16:0/1202632770 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaa4009a70 con 0x7feaac09d6f0 2023-09-30T22:35:43.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.749+0000 7feaabfff700 1 --2- 172.21.15.16:0/1202632770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feaac09e410 0x7feaac1320b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:35:43.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.749+0000 7feaab7fe700 1 --2- 172.21.15.16:0/1202632770 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7feaac09d6f0 0x7feaac131630 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7feaa4014040 tx=0x7feaa4004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:43.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.749+0000 7fea93fff700 1 -- 172.21.15.16:0/1202632770 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa4012430 con 0x7feaac09d6f0 2023-09-30T22:35:43.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.749+0000 7fea93fff700 1 -- 172.21.15.16:0/1202632770 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaa4020070 con 0x7feaac09d6f0 2023-09-30T22:35:43.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.749+0000 7feab202b700 1 -- 172.21.15.16:0/1202632770 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaac0a1e70 con 0x7feaac09d6f0 2023-09-30T22:35:43.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.749+0000 7fea93fff700 1 -- 172.21.15.16:0/1202632770 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa4004280 con 0x7feaac09d6f0 2023-09-30T22:35:43.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.749+0000 7feab202b700 1 -- 172.21.15.16:0/1202632770 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaac132a70 con 0x7feaac09d6f0 2023-09-30T22:35:43.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.750+0000 7fea93fff700 1 -- 172.21.15.16:0/1202632770 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7feaa40043e0 con 0x7feaac09d6f0 2023-09-30T22:35:43.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.750+0000 7feab202b700 1 -- 172.21.15.16:0/1202632770 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaac134500 con 0x7feaac09d6f0 2023-09-30T22:35:43.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.750+0000 7fea93fff700 1 -- 172.21.15.16:0/1202632770 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7feaa4022070 con 0x7feaac09d6f0 2023-09-30T22:35:43.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.759+0000 7fea93fff700 1 -- 172.21.15.16:0/1202632770 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7feaac134500 con 0x7feaac09d6f0 2023-09-30T22:35:43.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.889+0000 7fcafc7dd700 1 -- 172.21.15.16:0/975761837 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fcaf4121ee0 con 0x7fcaf409d6f0 2023-09-30T22:35:43.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.889+0000 7fcaeaffd700 1 -- 172.21.15.16:0/975761837 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcaf4121ee0 con 0x7fcaf409d6f0 2023-09-30T22:35:43.890 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:35:43.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.892+0000 7fcafc7dd700 1 -- 172.21.15.16:0/975761837 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcaf409d6f0 msgr2=0x7fcaf41c86b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:43.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.892+0000 7fcafc7dd700 1 --2- 172.21.15.16:0/975761837 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcaf409d6f0 0x7fcaf41c86b0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fcaf000c4a0 tx=0x7fcaf000c7b0 comp rx=0 tx=0).stop 2023-09-30T22:35:43.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.892+0000 7fcafc7dd700 1 -- 172.21.15.16:0/975761837 shutdown_connections 2023-09-30T22:35:43.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.892+0000 7fcafc7dd700 1 --2- 172.21.15.16:0/975761837 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcaf409d6f0 0x7fcaf41c86b0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:43.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.892+0000 7fcafc7dd700 1 --2- 172.21.15.16:0/975761837 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcaf4139170 0x7fcaf411e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:43.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.892+0000 7fcafc7dd700 1 --2- 172.21.15.16:0/975761837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcaf409e1d0 0x7fcaf41c8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:43.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.892+0000 7fcafc7dd700 1 -- 172.21.15.16:0/975761837 >> 172.21.15.16:0/975761837 conn(0x7fcaf4066f50 msgr2=0x7fcaf40a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:43.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.892+0000 7fcafc7dd700 1 -- 172.21.15.16:0/975761837 shutdown_connections 2023-09-30T22:35:43.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.892+0000 7fcafc7dd700 1 -- 172.21.15.16:0/975761837 wait complete. 2023-09-30T22:35:43.906 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:35:43.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.918+0000 7f3f8b57e700 1 -- 172.21.15.16:0/680012819 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3f841222b0 con 0x7f3f8409f410 2023-09-30T22:35:43.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.918+0000 7f3f79ffb700 1 -- 172.21.15.16:0/680012819 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3f841222b0 con 0x7f3f8409f410 2023-09-30T22:35:43.920 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:35:43.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.920+0000 7f3f8b57e700 1 -- 172.21.15.16:0/680012819 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3f8409f410 msgr2=0x7f3f8406c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:43.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.920+0000 7f3f8b57e700 1 --2- 172.21.15.16:0/680012819 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3f8409f410 0x7f3f8406c8b0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f3f74000c00 tx=0x7f3f74004750 comp rx=0 tx=0).stop 2023-09-30T22:35:43.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.920+0000 7f3f8b57e700 1 -- 172.21.15.16:0/680012819 shutdown_connections 2023-09-30T22:35:43.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.920+0000 7f3f8b57e700 1 --2- 172.21.15.16:0/680012819 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3f84136fe0 0x7f3f841c8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:43.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.920+0000 7f3f8b57e700 1 --2- 172.21.15.16:0/680012819 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3f8409e930 0x7f3f8406c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:43.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.920+0000 7f3f8b57e700 1 --2- 172.21.15.16:0/680012819 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3f8409f410 0x7f3f8406c8b0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:43.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.920+0000 7f3f8b57e700 1 -- 172.21.15.16:0/680012819 >> 172.21.15.16:0/680012819 conn(0x7f3f840a4210 msgr2=0x7f3f840a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:43.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.921+0000 7f3f8b57e700 1 -- 172.21.15.16:0/680012819 shutdown_connections 2023-09-30T22:35:43.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.921+0000 7f3f8b57e700 1 -- 172.21.15.16:0/680012819 wait complete. 2023-09-30T22:35:43.935 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:35:43.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.966+0000 7feab202b700 1 -- 172.21.15.16:0/1202632770 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7feaac134500 con 0x7feaac09d6f0 2023-09-30T22:35:43.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.966+0000 7fea93fff700 1 -- 172.21.15.16:0/1202632770 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7feaac134500 con 0x7feaac09d6f0 2023-09-30T22:35:43.967 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:35:43.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.969+0000 7feab202b700 1 -- 172.21.15.16:0/1202632770 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7feaac09d6f0 msgr2=0x7feaac131630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:43.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.969+0000 7feab202b700 1 --2- 172.21.15.16:0/1202632770 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7feaac09d6f0 0x7feaac131630 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7feaa4014040 tx=0x7feaa4004ad0 comp rx=0 tx=0).stop 2023-09-30T22:35:43.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.969+0000 7feab202b700 1 -- 172.21.15.16:0/1202632770 shutdown_connections 2023-09-30T22:35:43.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.969+0000 7feab202b700 1 --2- 172.21.15.16:0/1202632770 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7feaac09d6f0 0x7feaac131630 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:43.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.969+0000 7feab202b700 1 --2- 172.21.15.16:0/1202632770 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feaac09e410 0x7feaac1320b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:43.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.969+0000 7feab202b700 1 --2- 172.21.15.16:0/1202632770 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7feaac09e000 0x7feaac131b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:43.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.969+0000 7feab202b700 1 -- 172.21.15.16:0/1202632770 >> 172.21.15.16:0/1202632770 conn(0x7feaac066f50 msgr2=0x7feaac070e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:43.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.969+0000 7feab202b700 1 -- 172.21.15.16:0/1202632770 shutdown_connections 2023-09-30T22:35:43.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:43.969+0000 7feab202b700 1 -- 172.21.15.16:0/1202632770 wait complete. 2023-09-30T22:35:43.983 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:35:44.239 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:35:44.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.384+0000 7fd8b0b43700 1 Processor -- start 2023-09-30T22:35:44.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.385+0000 7fd8b0b43700 1 -- start start 2023-09-30T22:35:44.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.385+0000 7fd8b0b43700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd8ac09d6f0 0x7fd8ac09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:44.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.385+0000 7fd8b0b43700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd8ac09e000 0x7fd8ac1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:44.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.385+0000 7fd8b0b43700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8ac09b120 con 0x7fd8ac09e410 2023-09-30T22:35:44.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.385+0000 7fd8b0b43700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8ac06b460 con 0x7fd8ac09e000 2023-09-30T22:35:44.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.385+0000 7fd8b0b43700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ac09a7e0 con 0x7fd8ac09d6f0 2023-09-30T22:35:44.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.385+0000 7fd8a9d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd8ac09e000 0x7fd8ac1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:44.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.385+0000 7fd8aa59c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd8ac09d6f0 0x7fd8ac09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:44.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.385+0000 7fd8a9d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd8ac09e000 0x7fd8ac1339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40964/0 (socket says 172.21.15.16:40964) 2023-09-30T22:35:44.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.385+0000 7fd8aad9d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd8ac09e410 0x7fd8ac133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40638/0 (socket says 172.21.15.16:40638) 2023-09-30T22:35:44.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.385+0000 7fd8aad9d700 1 -- 172.21.15.16:0/2541285945 learned_addr learned my addr 172.21.15.16:0/2541285945 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:44.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.385+0000 7fd8aa59c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd8ac09d6f0 0x7fd8ac09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47764/0 (socket says 172.21.15.16:47764) 2023-09-30T22:35:44.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.386+0000 7fd8a9d9b700 1 -- 172.21.15.16:0/2541285945 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd8ac09d6f0 msgr2=0x7fd8ac09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:44.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.386+0000 7fd8a9d9b700 1 --2- 172.21.15.16:0/2541285945 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd8ac09d6f0 0x7fd8ac09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:44.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.386+0000 7fd8a9d9b700 1 -- 172.21.15.16:0/2541285945 >> v1:172.21.15.16:6789/0 conn(0x7fd8ac09e410 legacy=0x7fd8ac133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:44.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.386+0000 7fd8a9d9b700 1 -- 172.21.15.16:0/2541285945 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8ac09a500 con 0x7fd8ac09e000 2023-09-30T22:35:44.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.386+0000 7fd8a959a700 1 -- 172.21.15.16:0/2541285945 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1551085933 0 0) 0x7fd8ac09b120 con 0x7fd8ac09e410 2023-09-30T22:35:44.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.386+0000 7fd8aa59c700 1 --2- 172.21.15.16:0/2541285945 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd8ac09d6f0 0x7fd8ac09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:44.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.386+0000 7fd8a9d9b700 1 --2- 172.21.15.16:0/2541285945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8ac09e000 0x7fd8ac1339f0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0009840 tx=0x7fd8a0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddc863faa719cb8a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:44.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.386+0000 7fd8a959a700 1 -- 172.21.15.16:0/2541285945 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a0011830 con 0x7fd8ac09e000 2023-09-30T22:35:44.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.386+0000 7fd8a959a700 1 -- 172.21.15.16:0/2541285945 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a0011990 con 0x7fd8ac09e000 2023-09-30T22:35:44.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.386+0000 7fd8a959a700 1 -- 172.21.15.16:0/2541285945 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a0011d20 con 0x7fd8ac09e000 2023-09-30T22:35:44.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.387+0000 7fd8b0b43700 1 -- 172.21.15.16:0/2541285945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8ac09e000 msgr2=0x7fd8ac1339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:44.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.387+0000 7fd8b0b43700 1 --2- 172.21.15.16:0/2541285945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8ac09e000 0x7fd8ac1339f0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fd8a0009840 tx=0x7fd8a0009b50 comp rx=0 tx=0).stop 2023-09-30T22:35:44.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.387+0000 7fd8b0b43700 1 -- 172.21.15.16:0/2541285945 shutdown_connections 2023-09-30T22:35:44.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.387+0000 7fd8b0b43700 1 --2- 172.21.15.16:0/2541285945 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd8ac09d6f0 0x7fd8ac09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:44.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.387+0000 7fd8b0b43700 1 --2- 172.21.15.16:0/2541285945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8ac09e000 0x7fd8ac1339f0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:44.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.387+0000 7fd8b0b43700 1 -- 172.21.15.16:0/2541285945 >> 172.21.15.16:0/2541285945 conn(0x7fd8ac066f50 msgr2=0x7fd8ac09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:44.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.387+0000 7fd8b0b43700 1 -- 172.21.15.16:0/2541285945 shutdown_connections 2023-09-30T22:35:44.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.387+0000 7fd8b0b43700 1 -- 172.21.15.16:0/2541285945 wait complete. 2023-09-30T22:35:44.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.388+0000 7fd8b0b43700 1 Processor -- start 2023-09-30T22:35:44.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.389+0000 7fd8b0b43700 1 -- start start 2023-09-30T22:35:44.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.389+0000 7fd8b0b43700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd8ac09d6f0 0x7fd8ac1a5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:44.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.389+0000 7fd8b0b43700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8ac09e000 0x7fd8ac1a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:44.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.389+0000 7fd8b0b43700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd8ac09e410 0x7fd8ac1a6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:44.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.389+0000 7fd8b0b43700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ac1a9150 con 0x7fd8ac09e000 2023-09-30T22:35:44.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.389+0000 7fd8b0b43700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ac09c710 con 0x7fd8ac09d6f0 2023-09-30T22:35:44.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.389+0000 7fd8b0b43700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ac0a2e40 con 0x7fd8ac09e410 2023-09-30T22:35:44.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.389+0000 7fd8aa59c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd8ac09d6f0 0x7fd8ac1a5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:44.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.389+0000 7fd8a9d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8ac09e000 0x7fd8ac1a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:44.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.389+0000 7fd8aa59c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd8ac09d6f0 0x7fd8ac1a5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47770/0 (socket says 172.21.15.16:47770) 2023-09-30T22:35:44.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.389+0000 7fd8aa59c700 1 -- 172.21.15.16:0/4235261311 learned_addr learned my addr 172.21.15.16:0/4235261311 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:44.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.390+0000 7fd8aad9d700 1 --2- 172.21.15.16:0/4235261311 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd8ac09e410 0x7fd8ac1a6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:44.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.390+0000 7fd8a9d9b700 1 -- 172.21.15.16:0/4235261311 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd8ac09e410 msgr2=0x7fd8ac1a6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:44.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.390+0000 7fd8a9d9b700 1 --2- 172.21.15.16:0/4235261311 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd8ac09e410 0x7fd8ac1a6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:44.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.390+0000 7fd8a9d9b700 1 -- 172.21.15.16:0/4235261311 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd8ac09d6f0 msgr2=0x7fd8ac1a5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:44.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.390+0000 7fd8a9d9b700 1 --2- 172.21.15.16:0/4235261311 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd8ac09d6f0 0x7fd8ac1a5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:44.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.390+0000 7fd8a9d9b700 1 -- 172.21.15.16:0/4235261311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd89c0094a0 con 0x7fd8ac09e000 2023-09-30T22:35:44.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.390+0000 7fd8aa59c700 1 --2- 172.21.15.16:0/4235261311 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd8ac09d6f0 0x7fd8ac1a5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:44.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.390+0000 7fd8a9d9b700 1 --2- 172.21.15.16:0/4235261311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8ac09e000 0x7fd8ac1a63e0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac06e510 tx=0x7fd8a0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:44.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.390+0000 7fd89affd700 1 -- 172.21.15.16:0/4235261311 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a001a7e0 con 0x7fd8ac09e000 2023-09-30T22:35:44.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.390+0000 7fd89affd700 1 -- 172.21.15.16:0/4235261311 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a001a960 con 0x7fd8ac09e000 2023-09-30T22:35:44.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.390+0000 7fd8b0b43700 1 -- 172.21.15.16:0/4235261311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8a00094a0 con 0x7fd8ac09e000 2023-09-30T22:35:44.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.390+0000 7fd89affd700 1 -- 172.21.15.16:0/4235261311 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a001ad10 con 0x7fd8ac09e000 2023-09-30T22:35:44.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.390+0000 7fd8b0b43700 1 -- 172.21.15.16:0/4235261311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8ac1a8d00 con 0x7fd8ac09e000 2023-09-30T22:35:44.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.391+0000 7fd89affd700 1 -- 172.21.15.16:0/4235261311 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd8a0022070 con 0x7fd8ac09e000 2023-09-30T22:35:44.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.391+0000 7fd89affd700 1 -- 172.21.15.16:0/4235261311 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd8a0040650 con 0x7fd8ac09e000 2023-09-30T22:35:44.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.391+0000 7fd8b0b43700 1 -- 172.21.15.16:0/4235261311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8ac074ea0 con 0x7fd8ac09e000 2023-09-30T22:35:44.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.396+0000 7fd89affd700 1 -- 172.21.15.16:0/4235261311 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd8a0015080 con 0x7fd8ac09e000 2023-09-30T22:35:44.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.604+0000 7fd8b0b43700 1 -- 172.21.15.16:0/4235261311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd8ac1a9540 con 0x7fd8ac09e000 2023-09-30T22:35:44.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.604+0000 7fd89affd700 1 -- 172.21.15.16:0/4235261311 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd8a0029070 con 0x7fd8ac09e000 2023-09-30T22:35:44.605 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:35:44.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.606+0000 7fd8b0b43700 1 -- 172.21.15.16:0/4235261311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8ac09e000 msgr2=0x7fd8ac1a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:44.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.606+0000 7fd8b0b43700 1 --2- 172.21.15.16:0/4235261311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8ac09e000 0x7fd8ac1a63e0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac06e510 tx=0x7fd8a0004750 comp rx=0 tx=0).stop 2023-09-30T22:35:44.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.606+0000 7fd8b0b43700 1 -- 172.21.15.16:0/4235261311 shutdown_connections 2023-09-30T22:35:44.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.606+0000 7fd8b0b43700 1 --2- 172.21.15.16:0/4235261311 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd8ac09d6f0 0x7fd8ac1a5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:44.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.606+0000 7fd8b0b43700 1 --2- 172.21.15.16:0/4235261311 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd8ac09e410 0x7fd8ac1a6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:44.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.606+0000 7fd8b0b43700 1 --2- 172.21.15.16:0/4235261311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8ac09e000 0x7fd8ac1a63e0 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:44.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.606+0000 7fd8b0b43700 1 -- 172.21.15.16:0/4235261311 >> 172.21.15.16:0/4235261311 conn(0x7fd8ac066f50 msgr2=0x7fd8ac1bad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:44.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.607+0000 7fd8b0b43700 1 -- 172.21.15.16:0/4235261311 shutdown_connections 2023-09-30T22:35:44.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:44.607+0000 7fd8b0b43700 1 -- 172.21.15.16:0/4235261311 wait complete. 2023-09-30T22:35:44.621 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:35:44.907 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:35:44.936 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:35:44.983 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:35:45.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.059+0000 7f3cdc27c700 1 Processor -- start 2023-09-30T22:35:45.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.059+0000 7f3cdc27c700 1 -- start start 2023-09-30T22:35:45.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.060+0000 7f3cdc27c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3cd409d6f0 0x7f3cd409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:45.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.060+0000 7f3cdc27c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3cd409e410 0x7f3cd4134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:45.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.060+0000 7f3cdc27c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cd409b120 con 0x7f3cd409e000 2023-09-30T22:35:45.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.060+0000 7f3cdc27c700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cd406b460 con 0x7f3cd409e410 2023-09-30T22:35:45.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.060+0000 7f3cdc27c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd409a7e0 con 0x7f3cd409d6f0 2023-09-30T22:35:45.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.060+0000 7f3cda018700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3cd409d6f0 0x7f3cd409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:45.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.060+0000 7f3cda819700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3cd409e410 0x7f3cd4134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:45.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.060+0000 7f3cd9817700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f3cd409e000 0x7f3cd41339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40650/0 (socket says 172.21.15.16:40650) 2023-09-30T22:35:45.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.060+0000 7f3cd9817700 1 -- 172.21.15.16:0/4025518879 learned_addr learned my addr 172.21.15.16:0/4025518879 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:45.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.060+0000 7f3cda819700 1 -- 172.21.15.16:0/4025518879 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3cd409d6f0 msgr2=0x7f3cd409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:45.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.060+0000 7f3cda819700 1 --2- 172.21.15.16:0/4025518879 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3cd409d6f0 0x7f3cd409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:45.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.060+0000 7f3cda819700 1 -- 172.21.15.16:0/4025518879 >> v1:172.21.15.16:6789/0 conn(0x7f3cd409e000 legacy=0x7f3cd41339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:45.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.060+0000 7f3cd9016700 1 -- 172.21.15.16:0/4025518879 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 900730387 0 0) 0x7f3cd409b120 con 0x7f3cd409e000 2023-09-30T22:35:45.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.060+0000 7f3cda819700 1 -- 172.21.15.16:0/4025518879 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd409a500 con 0x7f3cd409e410 2023-09-30T22:35:45.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.060+0000 7f3cda018700 1 --2- 172.21.15.16:0/4025518879 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3cd409d6f0 0x7f3cd409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:45.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.061+0000 7f3cda819700 1 --2- 172.21.15.16:0/4025518879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cd409e410 0x7f3cd4134100 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc009d90 tx=0x7f3ccc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fc382715a47917 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:45.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.061+0000 7f3cd9016700 1 -- 172.21.15.16:0/4025518879 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc00e070 con 0x7f3cd409e410 2023-09-30T22:35:45.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.061+0000 7f3cd9016700 1 -- 172.21.15.16:0/4025518879 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ccc00bb20 con 0x7f3cd409e410 2023-09-30T22:35:45.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.061+0000 7f3cd9016700 1 -- 172.21.15.16:0/4025518879 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc012430 con 0x7f3cd409e410 2023-09-30T22:35:45.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.061+0000 7f3cdc27c700 1 -- 172.21.15.16:0/4025518879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cd409e410 msgr2=0x7f3cd4134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:45.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.061+0000 7f3cdc27c700 1 --2- 172.21.15.16:0/4025518879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cd409e410 0x7f3cd4134100 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc009d90 tx=0x7f3ccc00b330 comp rx=0 tx=0).stop 2023-09-30T22:35:45.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.062+0000 7f3cdc27c700 1 -- 172.21.15.16:0/4025518879 shutdown_connections 2023-09-30T22:35:45.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.062+0000 7f3cdc27c700 1 --2- 172.21.15.16:0/4025518879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cd409e410 0x7f3cd4134100 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:45.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.062+0000 7f3cdc27c700 1 --2- 172.21.15.16:0/4025518879 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3cd409d6f0 0x7f3cd409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:45.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.062+0000 7f3cdc27c700 1 -- 172.21.15.16:0/4025518879 >> 172.21.15.16:0/4025518879 conn(0x7f3cd4066f50 msgr2=0x7f3cd409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:45.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.062+0000 7f3cdc27c700 1 -- 172.21.15.16:0/4025518879 shutdown_connections 2023-09-30T22:35:45.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.062+0000 7f3cdc27c700 1 -- 172.21.15.16:0/4025518879 wait complete. 2023-09-30T22:35:45.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.063+0000 7f3cdc27c700 1 Processor -- start 2023-09-30T22:35:45.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.063+0000 7f3cdc27c700 1 -- start start 2023-09-30T22:35:45.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.063+0000 7f3cdc27c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3cd409d6f0 0x7f3cd41c8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:45.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.063+0000 7f3cdc27c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3cd409e000 0x7f3cd41c8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:45.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.063+0000 7f3cdc27c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cd409e410 0x7f3cd41c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:45.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.063+0000 7f3cdc27c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd41a8a30 con 0x7f3cd409e410 2023-09-30T22:35:45.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.063+0000 7f3cdc27c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd41a9780 con 0x7f3cd409d6f0 2023-09-30T22:35:45.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.063+0000 7f3cdc27c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd41c8e80 con 0x7f3cd409e000 2023-09-30T22:35:45.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.063+0000 7f3cda819700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cd409e410 0x7f3cd41c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:45.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.063+0000 7f3cd9817700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3cd409e000 0x7f3cd41c8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:45.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.063+0000 7f3cd9817700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3cd409e000 0x7f3cd41c8940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:50762/0 (socket says 172.21.15.16:50762) 2023-09-30T22:35:45.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.063+0000 7f3cda819700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cd409e410 0x7f3cd41c49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40992/0 (socket says 172.21.15.16:40992) 2023-09-30T22:35:45.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.063+0000 7f3cd9817700 1 -- 172.21.15.16:0/3224919392 learned_addr learned my addr 172.21.15.16:0/3224919392 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:45.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.064+0000 7f3cda018700 1 --2- 172.21.15.16:0/3224919392 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3cd409d6f0 0x7f3cd41c8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:45.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.064+0000 7f3cd9817700 1 -- 172.21.15.16:0/3224919392 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3cd409d6f0 msgr2=0x7f3cd41c8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:45.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.064+0000 7f3cd9817700 1 --2- 172.21.15.16:0/3224919392 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3cd409d6f0 0x7f3cd41c8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:45.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.064+0000 7f3cd9817700 1 -- 172.21.15.16:0/3224919392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cd409e410 msgr2=0x7f3cd41c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:45.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.064+0000 7f3cd9817700 1 --2- 172.21.15.16:0/3224919392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cd409e410 0x7f3cd41c49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:45.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.064+0000 7f3cd9817700 1 -- 172.21.15.16:0/3224919392 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd00094a0 con 0x7f3cd409e000 2023-09-30T22:35:45.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.064+0000 7f3cd9817700 1 --2- 172.21.15.16:0/3224919392 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3cd409e000 0x7f3cd41c8940 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f3cc400c520 tx=0x7f3cc400c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:45.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.064+0000 7f3ccaffd700 1 -- 172.21.15.16:0/3224919392 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc4004c80 con 0x7f3cd409e000 2023-09-30T22:35:45.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.064+0000 7f3ccaffd700 1 -- 172.21.15.16:0/3224919392 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cc4004de0 con 0x7f3cd409e000 2023-09-30T22:35:45.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.064+0000 7f3cdc27c700 1 -- 172.21.15.16:0/3224919392 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ccc009a70 con 0x7f3cd409e000 2023-09-30T22:35:45.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.064+0000 7f3ccaffd700 1 -- 172.21.15.16:0/3224919392 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc4007310 con 0x7f3cd409e000 2023-09-30T22:35:45.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.064+0000 7f3cdc27c700 1 -- 172.21.15.16:0/3224919392 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cd41a85e0 con 0x7f3cd409e000 2023-09-30T22:35:45.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.065+0000 7f3ccaffd700 1 -- 172.21.15.16:0/3224919392 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3cc4007470 con 0x7f3cd409e000 2023-09-30T22:35:45.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.065+0000 7f3cdc27c700 1 -- 172.21.15.16:0/3224919392 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cd41a9240 con 0x7f3cd409e000 2023-09-30T22:35:45.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.066+0000 7f3ccaffd700 1 -- 172.21.15.16:0/3224919392 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f3cc403f220 con 0x7f3cd409e000 2023-09-30T22:35:45.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.075+0000 7f3ccaffd700 1 -- 172.21.15.16:0/3224919392 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3cd41a9240 con 0x7f3cd409e000 2023-09-30T22:35:45.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.095+0000 7ff00753a700 1 Processor -- start 2023-09-30T22:35:45.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.096+0000 7ff00753a700 1 -- start start 2023-09-30T22:35:45.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.096+0000 7ff00753a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff00009e1d0 0x7ff000138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:45.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.096+0000 7ff00753a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff000139170 0x7ff00013b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:45.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.096+0000 7ff00753a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff00009b120 con 0x7ff00009d6f0 2023-09-30T22:35:45.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.096+0000 7ff00753a700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff00006b460 con 0x7ff000139170 2023-09-30T22:35:45.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.096+0000 7ff00753a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff00009a7e0 con 0x7ff00009e1d0 2023-09-30T22:35:45.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.096+0000 7ff005ad7700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff000139170 0x7ff00013b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:45.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.096+0000 7ff004ad5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff00009e1d0 0x7ff000138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:45.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.096+0000 7ff0052d6700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff00009d6f0 0x7ff00009dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40652/0 (socket says 172.21.15.16:40652) 2023-09-30T22:35:45.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.096+0000 7ff0052d6700 1 -- 172.21.15.16:0/3992893747 learned_addr learned my addr 172.21.15.16:0/3992893747 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:45.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.096+0000 7ff005ad7700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff000139170 0x7ff00013b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41004/0 (socket says 172.21.15.16:41004) 2023-09-30T22:35:45.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.097+0000 7ff005ad7700 1 -- 172.21.15.16:0/3992893747 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff00009e1d0 msgr2=0x7ff000138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:45.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.097+0000 7ff005ad7700 1 --2- 172.21.15.16:0/3992893747 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff00009e1d0 0x7ff000138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:45.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.097+0000 7ff005ad7700 1 -- 172.21.15.16:0/3992893747 >> v1:172.21.15.16:6789/0 conn(0x7ff00009d6f0 legacy=0x7ff00009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:45.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.097+0000 7feff7fff700 1 -- 172.21.15.16:0/3992893747 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4202739697 0 0) 0x7ff00009b120 con 0x7ff00009d6f0 2023-09-30T22:35:45.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.097+0000 7ff005ad7700 1 -- 172.21.15.16:0/3992893747 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff00009a500 con 0x7ff000139170 2023-09-30T22:35:45.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.097+0000 7ff004ad5700 1 --2- 172.21.15.16:0/3992893747 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff00009e1d0 0x7ff000138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:45.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.097+0000 7ff005ad7700 1 --2- 172.21.15.16:0/3992893747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff000139170 0x7ff00013b550 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7feff8009d90 tx=0x7feff800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a5182cc917f33bb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:45.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.097+0000 7feff7fff700 1 -- 172.21.15.16:0/3992893747 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff800e070 con 0x7ff000139170 2023-09-30T22:35:45.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.097+0000 7feff7fff700 1 -- 172.21.15.16:0/3992893747 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feff800bb20 con 0x7ff000139170 2023-09-30T22:35:45.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.097+0000 7feff7fff700 1 -- 172.21.15.16:0/3992893747 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff8012430 con 0x7ff000139170 2023-09-30T22:35:45.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.098+0000 7ff00753a700 1 -- 172.21.15.16:0/3992893747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff000139170 msgr2=0x7ff00013b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:45.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.098+0000 7ff00753a700 1 --2- 172.21.15.16:0/3992893747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff000139170 0x7ff00013b550 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7feff8009d90 tx=0x7feff800b330 comp rx=0 tx=0).stop 2023-09-30T22:35:45.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.098+0000 7ff00753a700 1 -- 172.21.15.16:0/3992893747 shutdown_connections 2023-09-30T22:35:45.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.098+0000 7ff00753a700 1 --2- 172.21.15.16:0/3992893747 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff000139170 0x7ff00013b550 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:45.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.098+0000 7ff00753a700 1 --2- 172.21.15.16:0/3992893747 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff00009e1d0 0x7ff000138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:45.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.098+0000 7ff00753a700 1 -- 172.21.15.16:0/3992893747 >> 172.21.15.16:0/3992893747 conn(0x7ff000066f50 msgr2=0x7ff00009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:45.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.098+0000 7ff00753a700 1 -- 172.21.15.16:0/3992893747 shutdown_connections 2023-09-30T22:35:45.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.098+0000 7ff00753a700 1 -- 172.21.15.16:0/3992893747 wait complete. 2023-09-30T22:35:45.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.099+0000 7ff00753a700 1 Processor -- start 2023-09-30T22:35:45.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.100+0000 7ff00753a700 1 -- start start 2023-09-30T22:35:45.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.100+0000 7ff00753a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff00009d6f0 0x7ff0001c8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:45.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.100+0000 7ff00753a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff00009e1d0 0x7ff0001c89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:45.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.100+0000 7ff00753a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff000139170 0x7ff0001c8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:45.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.100+0000 7ff00753a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0001254a0 con 0x7ff00009e1d0 2023-09-30T22:35:45.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.100+0000 7ff00753a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff000123770 con 0x7ff00009d6f0 2023-09-30T22:35:45.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.100+0000 7ff00753a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff00009c710 con 0x7ff000139170 2023-09-30T22:35:45.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.100+0000 7ff004ad5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff00009e1d0 0x7ff0001c89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:45.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.100+0000 7ff004ad5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff00009e1d0 0x7ff0001c89a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41018/0 (socket says 172.21.15.16:41018) 2023-09-30T22:35:45.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.100+0000 7ff004ad5700 1 -- 172.21.15.16:0/1462990092 learned_addr learned my addr 172.21.15.16:0/1462990092 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:45.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.100+0000 7ff005ad7700 1 --2- 172.21.15.16:0/1462990092 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff000139170 0x7ff0001c8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:45.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.100+0000 7ff0052d6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff00009d6f0 0x7ff0001c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:45.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.101+0000 7ff004ad5700 1 -- 172.21.15.16:0/1462990092 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff000139170 msgr2=0x7ff0001c8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:45.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.101+0000 7ff004ad5700 1 --2- 172.21.15.16:0/1462990092 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff000139170 0x7ff0001c8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:45.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.101+0000 7ff004ad5700 1 -- 172.21.15.16:0/1462990092 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff00009d6f0 msgr2=0x7ff0001c8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:45.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.101+0000 7ff004ad5700 1 --2- 172.21.15.16:0/1462990092 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff00009d6f0 0x7ff0001c8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:45.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.101+0000 7ff004ad5700 1 -- 172.21.15.16:0/1462990092 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff00094a0 con 0x7ff00009e1d0 2023-09-30T22:35:45.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.101+0000 7ff0052d6700 1 --2- 172.21.15.16:0/1462990092 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff00009d6f0 0x7ff0001c8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:45.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.101+0000 7ff004ad5700 1 --2- 172.21.15.16:0/1462990092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff00009e1d0 0x7ff0001c89a0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7feff000ebe0 tx=0x7feff000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:45.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.101+0000 7feff5ffb700 1 -- 172.21.15.16:0/1462990092 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff000c850 con 0x7ff00009e1d0 2023-09-30T22:35:45.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.101+0000 7feff5ffb700 1 -- 172.21.15.16:0/1462990092 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feff000c9b0 con 0x7ff00009e1d0 2023-09-30T22:35:45.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.101+0000 7ff00753a700 1 -- 172.21.15.16:0/1462990092 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feff8009a70 con 0x7ff00009e1d0 2023-09-30T22:35:45.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.101+0000 7feff5ffb700 1 -- 172.21.15.16:0/1462990092 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff000cd40 con 0x7ff00009e1d0 2023-09-30T22:35:45.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.101+0000 7ff00753a700 1 -- 172.21.15.16:0/1462990092 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0000a3070 con 0x7ff00009e1d0 2023-09-30T22:35:45.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.102+0000 7feff5ffb700 1 -- 172.21.15.16:0/1462990092 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7feff0013470 con 0x7ff00009e1d0 2023-09-30T22:35:45.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.103+0000 7ff00753a700 1 -- 172.21.15.16:0/1462990092 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff000074ea0 con 0x7ff00009e1d0 2023-09-30T22:35:45.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.103+0000 7feff5ffb700 1 -- 172.21.15.16:0/1462990092 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7feff003f790 con 0x7ff00009e1d0 2023-09-30T22:35:45.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.107+0000 7feff5ffb700 1 -- 172.21.15.16:0/1462990092 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7feff0013720 con 0x7ff00009e1d0 2023-09-30T22:35:45.140 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.140+0000 7f84c8038700 1 Processor -- start 2023-09-30T22:35:45.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.140+0000 7f84c8038700 1 -- start start 2023-09-30T22:35:45.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.141+0000 7f84c8038700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f84c009e1d0 0x7f84c0135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:45.142 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.141+0000 7f84c8038700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f84c0135e50 0x7f84c0138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:45.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.141+0000 7f84c8038700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84c009f130 con 0x7f84c009d6f0 2023-09-30T22:35:45.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.141+0000 7f84c8038700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84c009f520 con 0x7f84c009e1d0 2023-09-30T22:35:45.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.141+0000 7f84c8038700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84c009e930 con 0x7f84c0135e50 2023-09-30T22:35:45.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.141+0000 7f84c55d3700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f84c009e1d0 0x7f84c0135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:45.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.141+0000 7f84c5dd4700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f84c009d6f0 0x7f84c009dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40668/0 (socket says 172.21.15.16:40668) 2023-09-30T22:35:45.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.141+0000 7f84c5dd4700 1 -- 172.21.15.16:0/4102789414 learned_addr learned my addr 172.21.15.16:0/4102789414 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:45.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.141+0000 7f84c65d5700 1 --2- 172.21.15.16:0/4102789414 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f84c0135e50 0x7f84c0138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:45.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.142+0000 7f84c4dd2700 1 -- 172.21.15.16:0/4102789414 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3440986993 0 0) 0x7f84c009f130 con 0x7f84c009d6f0 2023-09-30T22:35:45.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.142+0000 7f84c4dd2700 1 -- 172.21.15.16:0/4102789414 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84ac0035b0 con 0x7f84c009d6f0 2023-09-30T22:35:45.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.142+0000 7f84c55d3700 1 -- 172.21.15.16:0/4102789414 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f84c0135e50 msgr2=0x7f84c0138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:45.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.142+0000 7f84c55d3700 1 --2- 172.21.15.16:0/4102789414 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f84c0135e50 0x7f84c0138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:45.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.142+0000 7f84c55d3700 1 -- 172.21.15.16:0/4102789414 >> v1:172.21.15.16:6789/0 conn(0x7f84c009d6f0 legacy=0x7f84c009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:45.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.142+0000 7f84c55d3700 1 -- 172.21.15.16:0/4102789414 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c009b2d0 con 0x7f84c009e1d0 2023-09-30T22:35:45.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.142+0000 7f84c65d5700 1 --2- 172.21.15.16:0/4102789414 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f84c0135e50 0x7f84c0138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:45.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.142+0000 7f84c55d3700 1 --2- 172.21.15.16:0/4102789414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84c009e1d0 0x7f84c0135910 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f84b0009840 tx=0x7f84b0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1970fd0cac272882 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:45.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.142+0000 7f84c4dd2700 1 -- 172.21.15.16:0/4102789414 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b0004230 con 0x7f84c009e1d0 2023-09-30T22:35:45.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.142+0000 7f84c4dd2700 1 -- 172.21.15.16:0/4102789414 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84b0004390 con 0x7f84c009e1d0 2023-09-30T22:35:45.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.142+0000 7f84c4dd2700 1 -- 172.21.15.16:0/4102789414 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b00036a0 con 0x7f84c009e1d0 2023-09-30T22:35:45.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.143+0000 7f84c8038700 1 -- 172.21.15.16:0/4102789414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84c009e1d0 msgr2=0x7f84c0135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:45.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.143+0000 7f84c8038700 1 --2- 172.21.15.16:0/4102789414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84c009e1d0 0x7f84c0135910 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f84b0009840 tx=0x7f84b0009b50 comp rx=0 tx=0).stop 2023-09-30T22:35:45.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.143+0000 7f84c8038700 1 -- 172.21.15.16:0/4102789414 shutdown_connections 2023-09-30T22:35:45.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.143+0000 7f84c8038700 1 --2- 172.21.15.16:0/4102789414 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f84c0135e50 0x7f84c0138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:45.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.143+0000 7f84c8038700 1 --2- 172.21.15.16:0/4102789414 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84c009e1d0 0x7f84c0135910 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:45.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.143+0000 7f84c8038700 1 -- 172.21.15.16:0/4102789414 >> 172.21.15.16:0/4102789414 conn(0x7f84c0066f50 msgr2=0x7f84c009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:45.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.143+0000 7f84c8038700 1 -- 172.21.15.16:0/4102789414 shutdown_connections 2023-09-30T22:35:45.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.143+0000 7f84c8038700 1 -- 172.21.15.16:0/4102789414 wait complete. 2023-09-30T22:35:45.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.144+0000 7f84c8038700 1 Processor -- start 2023-09-30T22:35:45.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.144+0000 7f84c8038700 1 -- start start 2023-09-30T22:35:45.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.145+0000 7f84c8038700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f84c009d6f0 0x7f84c0136250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:45.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.145+0000 7f84c8038700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f84c009e1d0 0x7f84c0136790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:45.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.145+0000 7f84c8038700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84c0135e50 0x7f84c0136cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:45.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.145+0000 7f84c8038700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c01333a0 con 0x7f84c0135e50 2023-09-30T22:35:45.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.145+0000 7f84c8038700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84c009c700 con 0x7f84c009e1d0 2023-09-30T22:35:45.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.145+0000 7f84c8038700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f84c01217d0 con 0x7f84c009d6f0 2023-09-30T22:35:45.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.145+0000 7f84c55d3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f84c009e1d0 0x7f84c0136790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:45.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.145+0000 7f84c55d3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f84c009e1d0 0x7f84c0136790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47812/0 (socket says 172.21.15.16:47812) 2023-09-30T22:35:45.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.145+0000 7f84c55d3700 1 -- 172.21.15.16:0/2091984981 learned_addr learned my addr 172.21.15.16:0/2091984981 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:45.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.145+0000 7f84c65d5700 1 --2- 172.21.15.16:0/2091984981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84c0135e50 0x7f84c0136cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:45.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.145+0000 7f84c5dd4700 1 --2- 172.21.15.16:0/2091984981 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f84c009d6f0 0x7f84c0136250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:45.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.145+0000 7f84c55d3700 1 -- 172.21.15.16:0/2091984981 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f84c009d6f0 msgr2=0x7f84c0136250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:45.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.145+0000 7f84c55d3700 1 --2- 172.21.15.16:0/2091984981 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f84c009d6f0 0x7f84c0136250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:45.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.145+0000 7f84c55d3700 1 -- 172.21.15.16:0/2091984981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84c0135e50 msgr2=0x7f84c0136cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:45.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.145+0000 7f84c55d3700 1 --2- 172.21.15.16:0/2091984981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84c0135e50 0x7f84c0136cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:45.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.145+0000 7f84c55d3700 1 -- 172.21.15.16:0/2091984981 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b8009a70 con 0x7f84c009e1d0 2023-09-30T22:35:45.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.145+0000 7f84c5dd4700 1 --2- 172.21.15.16:0/2091984981 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f84c009d6f0 0x7f84c0136250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:35:45.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.145+0000 7f84c55d3700 1 --2- 172.21.15.16:0/2091984981 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f84c009e1d0 0x7f84c0136790 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f84b0009b90 tx=0x7f84b0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:45.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.145+0000 7f84b67fc700 1 -- 172.21.15.16:0/2091984981 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b0003d90 con 0x7f84c009e1d0 2023-09-30T22:35:45.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.146+0000 7f84b67fc700 1 -- 172.21.15.16:0/2091984981 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84b0011630 con 0x7f84c009e1d0 2023-09-30T22:35:45.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.146+0000 7f84c8038700 1 -- 172.21.15.16:0/2091984981 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84b00094a0 con 0x7f84c009e1d0 2023-09-30T22:35:45.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.146+0000 7f84b67fc700 1 -- 172.21.15.16:0/2091984981 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b00119e0 con 0x7f84c009e1d0 2023-09-30T22:35:45.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.146+0000 7f84c8038700 1 -- 172.21.15.16:0/2091984981 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84c0120df0 con 0x7f84c009e1d0 2023-09-30T22:35:45.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.146+0000 7f84b67fc700 1 -- 172.21.15.16:0/2091984981 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f84b0018040 con 0x7f84c009e1d0 2023-09-30T22:35:45.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.147+0000 7f84b67fc700 1 -- 172.21.15.16:0/2091984981 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f84b003fea0 con 0x7f84c009e1d0 2023-09-30T22:35:45.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.147+0000 7f84c8038700 1 -- 172.21.15.16:0/2091984981 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84c01376b0 con 0x7f84c009e1d0 2023-09-30T22:35:45.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.156+0000 7f84b67fc700 1 -- 172.21.15.16:0/2091984981 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f84b002b080 con 0x7f84c009e1d0 2023-09-30T22:35:45.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.288+0000 7f3cdc27c700 1 -- 172.21.15.16:0/3224919392 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3cd4074ea0 con 0x7f3cd409e000 2023-09-30T22:35:45.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.288+0000 7f3ccaffd700 1 -- 172.21.15.16:0/3224919392 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3cc40133c0 con 0x7f3cd409e000 2023-09-30T22:35:45.289 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:35:45.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.290+0000 7f3cdc27c700 1 -- 172.21.15.16:0/3224919392 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3cd409e000 msgr2=0x7f3cd41c8940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:45.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.290+0000 7f3cdc27c700 1 --2- 172.21.15.16:0/3224919392 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3cd409e000 0x7f3cd41c8940 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f3cc400c520 tx=0x7f3cc400c830 comp rx=0 tx=0).stop 2023-09-30T22:35:45.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.290+0000 7f3cdc27c700 1 -- 172.21.15.16:0/3224919392 shutdown_connections 2023-09-30T22:35:45.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.290+0000 7f3cdc27c700 1 --2- 172.21.15.16:0/3224919392 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3cd409d6f0 0x7f3cd41c8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:45.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.290+0000 7f3cdc27c700 1 --2- 172.21.15.16:0/3224919392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3cd409e410 0x7f3cd41c49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:45.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.290+0000 7f3cdc27c700 1 --2- 172.21.15.16:0/3224919392 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3cd409e000 0x7f3cd41c8940 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:45.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.290+0000 7f3cdc27c700 1 -- 172.21.15.16:0/3224919392 >> 172.21.15.16:0/3224919392 conn(0x7f3cd4066f50 msgr2=0x7f3cd41bacf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:45.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.290+0000 7f3cdc27c700 1 -- 172.21.15.16:0/3224919392 shutdown_connections 2023-09-30T22:35:45.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.290+0000 7f3cdc27c700 1 -- 172.21.15.16:0/3224919392 wait complete. 2023-09-30T22:35:45.305 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:35:45.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.320+0000 7ff00753a700 1 -- 172.21.15.16:0/1462990092 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff000071e90 con 0x7ff00009e1d0 2023-09-30T22:35:45.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.320+0000 7feff5ffb700 1 -- 172.21.15.16:0/1462990092 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7feff0021070 con 0x7ff00009e1d0 2023-09-30T22:35:45.321 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:35:45.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.323+0000 7ff00753a700 1 -- 172.21.15.16:0/1462990092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff00009e1d0 msgr2=0x7ff0001c89a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:45.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.323+0000 7ff00753a700 1 --2- 172.21.15.16:0/1462990092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff00009e1d0 0x7ff0001c89a0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7feff000ebe0 tx=0x7feff000eef0 comp rx=0 tx=0).stop 2023-09-30T22:35:45.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.323+0000 7ff00753a700 1 -- 172.21.15.16:0/1462990092 shutdown_connections 2023-09-30T22:35:45.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.323+0000 7ff00753a700 1 --2- 172.21.15.16:0/1462990092 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff00009d6f0 0x7ff0001c8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:45.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.323+0000 7ff00753a700 1 --2- 172.21.15.16:0/1462990092 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff000139170 0x7ff0001c8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:45.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.323+0000 7ff00753a700 1 --2- 172.21.15.16:0/1462990092 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff00009e1d0 0x7ff0001c89a0 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:45.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.323+0000 7ff00753a700 1 -- 172.21.15.16:0/1462990092 >> 172.21.15.16:0/1462990092 conn(0x7ff000066f50 msgr2=0x7ff000070e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:45.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.323+0000 7ff00753a700 1 -- 172.21.15.16:0/1462990092 shutdown_connections 2023-09-30T22:35:45.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.323+0000 7ff00753a700 1 -- 172.21.15.16:0/1462990092 wait complete. 2023-09-30T22:35:45.336 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:35:45.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.363+0000 7f84c8038700 1 -- 172.21.15.16:0/2091984981 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f84c0071930 con 0x7f84c009e1d0 2023-09-30T22:35:45.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.364+0000 7f84b67fc700 1 -- 172.21.15.16:0/2091984981 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f84b0016020 con 0x7f84c009e1d0 2023-09-30T22:35:45.364 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:35:45.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.366+0000 7f84c8038700 1 -- 172.21.15.16:0/2091984981 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f84c009e1d0 msgr2=0x7f84c0136790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:45.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.366+0000 7f84c8038700 1 --2- 172.21.15.16:0/2091984981 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f84c009e1d0 0x7f84c0136790 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f84b0009b90 tx=0x7f84b0004750 comp rx=0 tx=0).stop 2023-09-30T22:35:45.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.366+0000 7f84c8038700 1 -- 172.21.15.16:0/2091984981 shutdown_connections 2023-09-30T22:35:45.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.366+0000 7f84c8038700 1 --2- 172.21.15.16:0/2091984981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84c0135e50 0x7f84c0136cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:45.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.366+0000 7f84c8038700 1 --2- 172.21.15.16:0/2091984981 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f84c009d6f0 0x7f84c0136250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:45.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.366+0000 7f84c8038700 1 --2- 172.21.15.16:0/2091984981 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f84c009e1d0 0x7f84c0136790 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:45.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.366+0000 7f84c8038700 1 -- 172.21.15.16:0/2091984981 >> 172.21.15.16:0/2091984981 conn(0x7f84c0066f50 msgr2=0x7f84c006ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:45.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.366+0000 7f84c8038700 1 -- 172.21.15.16:0/2091984981 shutdown_connections 2023-09-30T22:35:45.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.366+0000 7f84c8038700 1 -- 172.21.15.16:0/2091984981 wait complete. 2023-09-30T22:35:45.380 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:35:45.622 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:35:45.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.771+0000 7f6be979d700 1 Processor -- start 2023-09-30T22:35:45.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.771+0000 7f6be979d700 1 -- start start 2023-09-30T22:35:45.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.771+0000 7f6be979d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6be40a09c0 0x7f6be40a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:45.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.771+0000 7f6be979d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6be409d5c0 0x7f6be40a12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:45.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.771+0000 7f6be979d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6be4133920 0x7f6be4133d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:45.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.771+0000 7f6be979d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6be40a1810 con 0x7f6be40a09c0 2023-09-30T22:35:45.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.771+0000 7f6be979d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6be409ea10 con 0x7f6be409d5c0 2023-09-30T22:35:45.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.771+0000 7f6be979d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6be409e210 con 0x7f6be4133920 2023-09-30T22:35:45.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.771+0000 7f6be2ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6be40a09c0 0x7f6be40a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:45.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.771+0000 7f6be27fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6be409d5c0 0x7f6be40a12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:45.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.771+0000 7f6be37fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6be4133920 0x7f6be4133d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:45.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.771+0000 7f6be2ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6be40a09c0 0x7f6be40a0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41030/0 (socket says 172.21.15.16:41030) 2023-09-30T22:35:45.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.771+0000 7f6be2ffd700 1 -- 172.21.15.16:0/376428748 learned_addr learned my addr 172.21.15.16:0/376428748 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:45.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.772+0000 7f6be37fe700 1 -- 172.21.15.16:0/376428748 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6be409d5c0 msgr2=0x7f6be40a12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:45.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.772+0000 7f6be37fe700 1 --2- 172.21.15.16:0/376428748 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6be409d5c0 0x7f6be40a12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:45.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.772+0000 7f6be37fe700 1 -- 172.21.15.16:0/376428748 >> v2:172.21.15.16:3300/0 conn(0x7f6be40a09c0 msgr2=0x7f6be40a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:45.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.772+0000 7f6be37fe700 1 --2- 172.21.15.16:0/376428748 >> v2:172.21.15.16:3300/0 conn(0x7f6be40a09c0 0x7f6be40a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:45.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.772+0000 7f6be37fe700 1 -- 172.21.15.16:0/376428748 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be40a26f0 con 0x7f6be4133920 2023-09-30T22:35:45.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.772+0000 7f6be2ffd700 1 --2- 172.21.15.16:0/376428748 >> v2:172.21.15.16:3300/0 conn(0x7f6be40a09c0 0x7f6be40a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:45.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.772+0000 7f6be37fe700 1 --2- 172.21.15.16:0/376428748 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6be4133920 0x7f6be4133d70 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc009d90 tx=0x7f6bdc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f827acd208eb3b62 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:45.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.772+0000 7f6be1ffb700 1 -- 172.21.15.16:0/376428748 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc00e070 con 0x7f6be4133920 2023-09-30T22:35:45.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.772+0000 7f6be1ffb700 1 -- 172.21.15.16:0/376428748 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bdc00bb20 con 0x7f6be4133920 2023-09-30T22:35:45.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.772+0000 7f6be1ffb700 1 -- 172.21.15.16:0/376428748 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc012430 con 0x7f6be4133920 2023-09-30T22:35:45.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.773+0000 7f6be979d700 1 -- 172.21.15.16:0/376428748 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6be4133920 msgr2=0x7f6be4133d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:45.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.773+0000 7f6be979d700 1 --2- 172.21.15.16:0/376428748 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6be4133920 0x7f6be4133d70 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc009d90 tx=0x7f6bdc00b330 comp rx=0 tx=0).stop 2023-09-30T22:35:45.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.773+0000 7f6be979d700 1 -- 172.21.15.16:0/376428748 shutdown_connections 2023-09-30T22:35:45.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.773+0000 7f6be979d700 1 --2- 172.21.15.16:0/376428748 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6be4133920 0x7f6be4133d70 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:45.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.773+0000 7f6be979d700 1 --2- 172.21.15.16:0/376428748 >> v2:172.21.15.16:3300/0 conn(0x7f6be40a09c0 0x7f6be40a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:45.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.773+0000 7f6be979d700 1 --2- 172.21.15.16:0/376428748 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6be409d5c0 0x7f6be40a12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:45.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.773+0000 7f6be979d700 1 -- 172.21.15.16:0/376428748 >> 172.21.15.16:0/376428748 conn(0x7f6be40a2fe0 msgr2=0x7f6be40a33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:45.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.773+0000 7f6be979d700 1 -- 172.21.15.16:0/376428748 shutdown_connections 2023-09-30T22:35:45.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.773+0000 7f6be979d700 1 -- 172.21.15.16:0/376428748 wait complete. 2023-09-30T22:35:45.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.774+0000 7f6be979d700 1 Processor -- start 2023-09-30T22:35:45.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.774+0000 7f6be979d700 1 -- start start 2023-09-30T22:35:45.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.774+0000 7f6be979d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6be409d5c0 0x7f6be41c8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:45.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.774+0000 7f6be979d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6be40a09c0 0x7f6be41c8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:45.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.774+0000 7f6be979d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6be4133920 0x7f6be411e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:45.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.774+0000 7f6be979d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6be4133430 con 0x7f6be4133920 2023-09-30T22:35:45.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.774+0000 7f6be979d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6be40a59b0 con 0x7f6be409d5c0 2023-09-30T22:35:45.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.774+0000 7f6be979d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6be41bcfd0 con 0x7f6be40a09c0 2023-09-30T22:35:45.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.775+0000 7f6be37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6be4133920 0x7f6be411e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:45.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.775+0000 7f6be2ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6be409d5c0 0x7f6be41c8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:45.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.775+0000 7f6be27fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6be40a09c0 0x7f6be41c8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:45.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.775+0000 7f6be2ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6be409d5c0 0x7f6be41c8410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47830/0 (socket says 172.21.15.16:47830) 2023-09-30T22:35:45.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.775+0000 7f6be37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6be4133920 0x7f6be411e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41042/0 (socket says 172.21.15.16:41042) 2023-09-30T22:35:45.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.775+0000 7f6be2ffd700 1 -- 172.21.15.16:0/4053741068 learned_addr learned my addr 172.21.15.16:0/4053741068 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:45.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.775+0000 7f6be2ffd700 1 -- 172.21.15.16:0/4053741068 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6be40a09c0 msgr2=0x7f6be41c8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:45.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.775+0000 7f6be2ffd700 1 --2- 172.21.15.16:0/4053741068 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6be40a09c0 0x7f6be41c8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:45.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.775+0000 7f6be2ffd700 1 -- 172.21.15.16:0/4053741068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6be4133920 msgr2=0x7f6be411e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:45.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.775+0000 7f6be2ffd700 1 --2- 172.21.15.16:0/4053741068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6be4133920 0x7f6be411e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:45.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.775+0000 7f6be2ffd700 1 -- 172.21.15.16:0/4053741068 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bd8009550 con 0x7f6be409d5c0 2023-09-30T22:35:45.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.775+0000 7f6be37fe700 1 --2- 172.21.15.16:0/4053741068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6be4133920 0x7f6be411e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:45.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.775+0000 7f6be2ffd700 1 --2- 172.21.15.16:0/4053741068 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6be409d5c0 0x7f6be41c8410 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f6bd800ebe0 tx=0x7f6bd800eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:45.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.775+0000 7f6bcb7fe700 1 -- 172.21.15.16:0/4053741068 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd800c850 con 0x7f6be409d5c0 2023-09-30T22:35:45.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.775+0000 7f6bcb7fe700 1 -- 172.21.15.16:0/4053741068 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bd800c9b0 con 0x7f6be409d5c0 2023-09-30T22:35:45.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.775+0000 7f6bcb7fe700 1 -- 172.21.15.16:0/4053741068 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd800cd40 con 0x7f6be409d5c0 2023-09-30T22:35:45.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.775+0000 7f6be979d700 1 -- 172.21.15.16:0/4053741068 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bdc009a70 con 0x7f6be409d5c0 2023-09-30T22:35:45.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.775+0000 7f6be979d700 1 -- 172.21.15.16:0/4053741068 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6be41204b0 con 0x7f6be409d5c0 2023-09-30T22:35:45.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.776+0000 7f6bcb7fe700 1 -- 172.21.15.16:0/4053741068 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6bd8013470 con 0x7f6be409d5c0 2023-09-30T22:35:45.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.776+0000 7f6bcb7fe700 1 -- 172.21.15.16:0/4053741068 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f6bd803f7c0 con 0x7f6be409d5c0 2023-09-30T22:35:45.780 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.776+0000 7f6be979d700 1 -- 172.21.15.16:0/4053741068 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6be41c8ef0 con 0x7f6be409d5c0 2023-09-30T22:35:45.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.781+0000 7f6bcb7fe700 1 -- 172.21.15.16:0/4053741068 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6bd80232f0 con 0x7f6be409d5c0 2023-09-30T22:35:45.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.986+0000 7f6be979d700 1 -- 172.21.15.16:0/4053741068 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6be40747f0 con 0x7f6be409d5c0 2023-09-30T22:35:45.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.986+0000 7f6bcb7fe700 1 -- 172.21.15.16:0/4053741068 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6bd8017020 con 0x7f6be409d5c0 2023-09-30T22:35:45.988 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:35:45.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.989+0000 7f6be979d700 1 -- 172.21.15.16:0/4053741068 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6be409d5c0 msgr2=0x7f6be41c8410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:45.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.989+0000 7f6be979d700 1 --2- 172.21.15.16:0/4053741068 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6be409d5c0 0x7f6be41c8410 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f6bd800ebe0 tx=0x7f6bd800eef0 comp rx=0 tx=0).stop 2023-09-30T22:35:45.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.989+0000 7f6be979d700 1 -- 172.21.15.16:0/4053741068 shutdown_connections 2023-09-30T22:35:45.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.989+0000 7f6be979d700 1 --2- 172.21.15.16:0/4053741068 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6be409d5c0 0x7f6be41c8410 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:45.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.989+0000 7f6be979d700 1 --2- 172.21.15.16:0/4053741068 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6be4133920 0x7f6be411e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:45.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.989+0000 7f6be979d700 1 --2- 172.21.15.16:0/4053741068 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6be40a09c0 0x7f6be41c8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:45.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.989+0000 7f6be979d700 1 -- 172.21.15.16:0/4053741068 >> 172.21.15.16:0/4053741068 conn(0x7f6be40a2fe0 msgr2=0x7f6be41bbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:45.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.989+0000 7f6be979d700 1 -- 172.21.15.16:0/4053741068 shutdown_connections 2023-09-30T22:35:45.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:45.989+0000 7f6be979d700 1 -- 172.21.15.16:0/4053741068 wait complete. 2023-09-30T22:35:46.004 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:35:46.305 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:35:46.337 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:35:46.381 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:35:46.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.454+0000 7fa37527d700 1 Processor -- start 2023-09-30T22:35:46.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.454+0000 7fa37527d700 1 -- start start 2023-09-30T22:35:46.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.455+0000 7fa37527d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa3701338a0 0x7fa370133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:46.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.455+0000 7fa37527d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa37009e930 0x7fa37013b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:46.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.455+0000 7fa37527d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa37006ef30 con 0x7fa37009de50 2023-09-30T22:35:46.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.455+0000 7fa37527d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa37006fef0 con 0x7fa3701338a0 2023-09-30T22:35:46.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.455+0000 7fa37527d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa3701334b0 con 0x7fa37009e930 2023-09-30T22:35:46.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.455+0000 7fa36e7fc700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa37009de50 0x7fa37009e220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40684/0 (socket says 172.21.15.16:40684) 2023-09-30T22:35:46.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.455+0000 7fa36e7fc700 1 -- 172.21.15.16:0/215710734 learned_addr learned my addr 172.21.15.16:0/215710734 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:46.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.455+0000 7fa36effd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa3701338a0 0x7fa370133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:46.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.455+0000 7fa36f7fe700 1 --2- 172.21.15.16:0/215710734 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa37009e930 0x7fa37013b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:46.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.455+0000 7fa36dffb700 1 -- 172.21.15.16:0/215710734 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 289924868 0 0) 0x7fa37006ef30 con 0x7fa37009de50 2023-09-30T22:35:46.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.455+0000 7fa36dffb700 1 -- 172.21.15.16:0/215710734 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3580035b0 con 0x7fa37009de50 2023-09-30T22:35:46.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.455+0000 7fa36f7fe700 1 -- 172.21.15.16:0/215710734 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa3701338a0 msgr2=0x7fa370133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:46.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.455+0000 7fa36f7fe700 1 --2- 172.21.15.16:0/215710734 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa3701338a0 0x7fa370133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:46.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.455+0000 7fa36f7fe700 1 -- 172.21.15.16:0/215710734 >> v1:172.21.15.16:6789/0 conn(0x7fa37009de50 legacy=0x7fa37009e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:46.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.455+0000 7fa36f7fe700 1 -- 172.21.15.16:0/215710734 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa37006fc30 con 0x7fa37009e930 2023-09-30T22:35:46.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.455+0000 7fa36effd700 1 --2- 172.21.15.16:0/215710734 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa3701338a0 0x7fa370133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:46.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.456+0000 7fa36f7fe700 1 --2- 172.21.15.16:0/215710734 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa37009e930 0x7fa37013b740 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fa368009dc0 tx=0x7fa36800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9761570211ab9366 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:46.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.456+0000 7fa36dffb700 1 -- 172.21.15.16:0/215710734 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36800e070 con 0x7fa37009e930 2023-09-30T22:35:46.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.456+0000 7fa36dffb700 1 -- 172.21.15.16:0/215710734 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa36800bb20 con 0x7fa37009e930 2023-09-30T22:35:46.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.456+0000 7fa36dffb700 1 -- 172.21.15.16:0/215710734 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36800be90 con 0x7fa37009e930 2023-09-30T22:35:46.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.456+0000 7fa37527d700 1 -- 172.21.15.16:0/215710734 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa37009e930 msgr2=0x7fa37013b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:46.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.456+0000 7fa37527d700 1 --2- 172.21.15.16:0/215710734 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa37009e930 0x7fa37013b740 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fa368009dc0 tx=0x7fa36800b330 comp rx=0 tx=0).stop 2023-09-30T22:35:46.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.457+0000 7fa37527d700 1 -- 172.21.15.16:0/215710734 shutdown_connections 2023-09-30T22:35:46.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.457+0000 7fa37527d700 1 --2- 172.21.15.16:0/215710734 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa37009e930 0x7fa37013b740 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:46.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.457+0000 7fa37527d700 1 --2- 172.21.15.16:0/215710734 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa3701338a0 0x7fa370133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:46.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.457+0000 7fa37527d700 1 -- 172.21.15.16:0/215710734 >> 172.21.15.16:0/215710734 conn(0x7fa3700a2aa0 msgr2=0x7fa3700a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:46.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.457+0000 7fa37527d700 1 -- 172.21.15.16:0/215710734 shutdown_connections 2023-09-30T22:35:46.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.457+0000 7fa37527d700 1 -- 172.21.15.16:0/215710734 wait complete. 2023-09-30T22:35:46.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.458+0000 7fa37527d700 1 Processor -- start 2023-09-30T22:35:46.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.458+0000 7fa37527d700 1 -- start start 2023-09-30T22:35:46.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.458+0000 7fa37527d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa37009de50 0x7fa37011d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:46.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.458+0000 7fa37527d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa37009e930 0x7fa37011dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:46.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.458+0000 7fa37527d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3701338a0 0x7fa37011e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:46.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.458+0000 7fa37527d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa37011f200 con 0x7fa3701338a0 2023-09-30T22:35:46.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.458+0000 7fa37527d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa37006f640 con 0x7fa37009de50 2023-09-30T22:35:46.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.458+0000 7fa37527d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa370120e30 con 0x7fa37009e930 2023-09-30T22:35:46.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.459+0000 7fa36effd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa37009de50 0x7fa37011d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:46.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.459+0000 7fa36e7fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa37009e930 0x7fa37011dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:46.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.459+0000 7fa36effd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa37009de50 0x7fa37011d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47854/0 (socket says 172.21.15.16:47854) 2023-09-30T22:35:46.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.459+0000 7fa36effd700 1 -- 172.21.15.16:0/282104579 learned_addr learned my addr 172.21.15.16:0/282104579 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:46.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.459+0000 7fa36f7fe700 1 --2- 172.21.15.16:0/282104579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3701338a0 0x7fa37011e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:46.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.459+0000 7fa36e7fc700 1 -- 172.21.15.16:0/282104579 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa37009de50 msgr2=0x7fa37011d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:46.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.459+0000 7fa36e7fc700 1 --2- 172.21.15.16:0/282104579 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa37009de50 0x7fa37011d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:46.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.459+0000 7fa36e7fc700 1 -- 172.21.15.16:0/282104579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3701338a0 msgr2=0x7fa37011e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:46.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.459+0000 7fa36e7fc700 1 --2- 172.21.15.16:0/282104579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3701338a0 0x7fa37011e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:46.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.459+0000 7fa36e7fc700 1 -- 172.21.15.16:0/282104579 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3600094a0 con 0x7fa37009e930 2023-09-30T22:35:46.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.459+0000 7fa36e7fc700 1 --2- 172.21.15.16:0/282104579 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa37009e930 0x7fa37011dc10 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fa36400c400 tx=0x7fa36400c710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:46.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.459+0000 7fa3577fe700 1 -- 172.21.15.16:0/282104579 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa364005d70 con 0x7fa37009e930 2023-09-30T22:35:46.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.459+0000 7fa3577fe700 1 -- 172.21.15.16:0/282104579 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa36400ce90 con 0x7fa37009e930 2023-09-30T22:35:46.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.459+0000 7fa3577fe700 1 -- 172.21.15.16:0/282104579 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3640072f0 con 0x7fa37009e930 2023-09-30T22:35:46.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.459+0000 7fa37527d700 1 -- 172.21.15.16:0/282104579 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa368009a70 con 0x7fa37009e930 2023-09-30T22:35:46.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.460+0000 7fa37527d700 1 -- 172.21.15.16:0/282104579 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa370120450 con 0x7fa37009e930 2023-09-30T22:35:46.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.461+0000 7fa3577fe700 1 -- 172.21.15.16:0/282104579 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa364007450 con 0x7fa37009e930 2023-09-30T22:35:46.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.461+0000 7fa37527d700 1 -- 172.21.15.16:0/282104579 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa37011a470 con 0x7fa37009e930 2023-09-30T22:35:46.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.461+0000 7fa3577fe700 1 -- 172.21.15.16:0/282104579 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fa36403e670 con 0x7fa37009e930 2023-09-30T22:35:46.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.466+0000 7fa3577fe700 1 -- 172.21.15.16:0/282104579 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa364018af0 con 0x7fa37009e930 2023-09-30T22:35:46.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.496+0000 7f072b853700 1 Processor -- start 2023-09-30T22:35:46.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.496+0000 7f072b853700 1 -- start start 2023-09-30T22:35:46.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.497+0000 7f072b853700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0724133a80 0x7f0724133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:46.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.497+0000 7f072b853700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f072412ff20 0x7f0724130330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:46.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.497+0000 7f072b853700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0724130870 0x7f072413a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:46.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.497+0000 7f072b853700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f072406c570 con 0x7f0724130870 2023-09-30T22:35:46.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.497+0000 7f072b853700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f072406bc00 con 0x7f072412ff20 2023-09-30T22:35:46.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.497+0000 7f072b853700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f07240a0040 con 0x7f0724133a80 2023-09-30T22:35:46.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.497+0000 7f07295ef700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0724133a80 0x7f0724133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:46.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.497+0000 7f0729df0700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0724130870 0x7f072413a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:46.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.497+0000 7f07295ef700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0724133a80 0x7f0724133e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:50814/0 (socket says 172.21.15.16:50814) 2023-09-30T22:35:46.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.497+0000 7f0729df0700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0724130870 0x7f072413a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41068/0 (socket says 172.21.15.16:41068) 2023-09-30T22:35:46.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.497+0000 7f07295ef700 1 -- 172.21.15.16:0/1509184579 learned_addr learned my addr 172.21.15.16:0/1509184579 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:46.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.497+0000 7f0728dee700 1 --2- 172.21.15.16:0/1509184579 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f072412ff20 0x7f0724130330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:46.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.498+0000 7f0728dee700 1 -- 172.21.15.16:0/1509184579 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0724133a80 msgr2=0x7f0724133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:46.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.498+0000 7f0728dee700 1 --2- 172.21.15.16:0/1509184579 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0724133a80 0x7f0724133e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:46.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.498+0000 7f0728dee700 1 -- 172.21.15.16:0/1509184579 >> v2:172.21.15.16:3300/0 conn(0x7f0724130870 msgr2=0x7f072413a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:46.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.498+0000 7f0728dee700 1 --2- 172.21.15.16:0/1509184579 >> v2:172.21.15.16:3300/0 conn(0x7f0724130870 0x7f072413a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:46.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.498+0000 7f0728dee700 1 -- 172.21.15.16:0/1509184579 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0724133860 con 0x7f072412ff20 2023-09-30T22:35:46.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.498+0000 7f0729df0700 1 --2- 172.21.15.16:0/1509184579 >> v2:172.21.15.16:3300/0 conn(0x7f0724130870 0x7f072413a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:46.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.498+0000 7f0728dee700 1 --2- 172.21.15.16:0/1509184579 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f072412ff20 0x7f0724130330 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f0714009840 tx=0x7f0714009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1acd59ccd47d9d3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:46.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.498+0000 7f071bfff700 1 -- 172.21.15.16:0/1509184579 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0714011830 con 0x7f072412ff20 2023-09-30T22:35:46.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.498+0000 7f071bfff700 1 -- 172.21.15.16:0/1509184579 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0714011990 con 0x7f072412ff20 2023-09-30T22:35:46.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.498+0000 7f071bfff700 1 -- 172.21.15.16:0/1509184579 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0714011d20 con 0x7f072412ff20 2023-09-30T22:35:46.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.499+0000 7f072b853700 1 -- 172.21.15.16:0/1509184579 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f072412ff20 msgr2=0x7f0724130330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:46.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.499+0000 7f072b853700 1 --2- 172.21.15.16:0/1509184579 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f072412ff20 0x7f0724130330 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f0714009840 tx=0x7f0714009b50 comp rx=0 tx=0).stop 2023-09-30T22:35:46.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.499+0000 7f072b853700 1 -- 172.21.15.16:0/1509184579 shutdown_connections 2023-09-30T22:35:46.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.499+0000 7f072b853700 1 --2- 172.21.15.16:0/1509184579 >> v2:172.21.15.16:3300/0 conn(0x7f0724130870 0x7f072413a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:46.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.499+0000 7f072b853700 1 --2- 172.21.15.16:0/1509184579 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0724133a80 0x7f0724133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:46.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.499+0000 7f072b853700 1 --2- 172.21.15.16:0/1509184579 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f072412ff20 0x7f0724130330 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:46.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.499+0000 7f072b853700 1 -- 172.21.15.16:0/1509184579 >> 172.21.15.16:0/1509184579 conn(0x7f07240a4210 msgr2=0x7f07240a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:46.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.499+0000 7f072b853700 1 -- 172.21.15.16:0/1509184579 shutdown_connections 2023-09-30T22:35:46.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.499+0000 7f072b853700 1 -- 172.21.15.16:0/1509184579 wait complete. 2023-09-30T22:35:46.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.500+0000 7f072b853700 1 Processor -- start 2023-09-30T22:35:46.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.500+0000 7f072b853700 1 -- start start 2023-09-30T22:35:46.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.501+0000 7f072b853700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f072412ff20 0x7f072409c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:46.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.501+0000 7f072b853700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0724130870 0x7f072409c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:46.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.501+0000 7f072b853700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0724133a80 0x7f072409cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:46.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.501+0000 7f072b853700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072412db80 con 0x7f072412ff20 2023-09-30T22:35:46.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.501+0000 7f072b853700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0724120dd0 con 0x7f0724133a80 2023-09-30T22:35:46.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.501+0000 7f072b853700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f072411fc50 con 0x7f0724130870 2023-09-30T22:35:46.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.501+0000 7f07295ef700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f072412ff20 0x7f072409c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:46.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.501+0000 7f07295ef700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f072412ff20 0x7f072409c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41070/0 (socket says 172.21.15.16:41070) 2023-09-30T22:35:46.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.501+0000 7f07295ef700 1 -- 172.21.15.16:0/2671902137 learned_addr learned my addr 172.21.15.16:0/2671902137 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:46.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.501+0000 7f0728dee700 1 --2- 172.21.15.16:0/2671902137 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0724130870 0x7f072409c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:46.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.501+0000 7f0729df0700 1 --2- 172.21.15.16:0/2671902137 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0724133a80 0x7f072409cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:46.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.501+0000 7f07295ef700 1 -- 172.21.15.16:0/2671902137 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0724130870 msgr2=0x7f072409c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:46.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.501+0000 7f07295ef700 1 --2- 172.21.15.16:0/2671902137 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0724130870 0x7f072409c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:46.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.501+0000 7f07295ef700 1 -- 172.21.15.16:0/2671902137 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0724133a80 msgr2=0x7f072409cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:46.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.501+0000 7f07295ef700 1 --2- 172.21.15.16:0/2671902137 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0724133a80 0x7f072409cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:46.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.501+0000 7f07295ef700 1 -- 172.21.15.16:0/2671902137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f071c009a70 con 0x7f072412ff20 2023-09-30T22:35:46.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.501+0000 7f0729df0700 1 --2- 172.21.15.16:0/2671902137 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0724133a80 0x7f072409cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:46.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.501+0000 7f07295ef700 1 --2- 172.21.15.16:0/2671902137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f072412ff20 0x7f072409c480 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f072000e910 tx=0x7f072000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:46.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.502+0000 7f0719ffb700 1 -- 172.21.15.16:0/2671902137 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07200096a0 con 0x7f072412ff20 2023-09-30T22:35:46.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.502+0000 7f0719ffb700 1 -- 172.21.15.16:0/2671902137 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f072000ee50 con 0x7f072412ff20 2023-09-30T22:35:46.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.502+0000 7f072b853700 1 -- 172.21.15.16:0/2671902137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07140094a0 con 0x7f072412ff20 2023-09-30T22:35:46.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.502+0000 7f0719ffb700 1 -- 172.21.15.16:0/2671902137 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072000ca90 con 0x7f072412ff20 2023-09-30T22:35:46.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.502+0000 7f072b853700 1 -- 172.21.15.16:0/2671902137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f072411f940 con 0x7f072412ff20 2023-09-30T22:35:46.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.503+0000 7f0719ffb700 1 -- 172.21.15.16:0/2671902137 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f072000cbf0 con 0x7f072412ff20 2023-09-30T22:35:46.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.503+0000 7f0719ffb700 1 -- 172.21.15.16:0/2671902137 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f072003f740 con 0x7f072412ff20 2023-09-30T22:35:46.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.503+0000 7f072b853700 1 -- 172.21.15.16:0/2671902137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0724074ea0 con 0x7f072412ff20 2023-09-30T22:35:46.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.508+0000 7f0719ffb700 1 -- 172.21.15.16:0/2671902137 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0720017020 con 0x7f072412ff20 2023-09-30T22:35:46.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.538+0000 7fb0bf81e700 1 Processor -- start 2023-09-30T22:35:46.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.538+0000 7fb0bf81e700 1 -- start start 2023-09-30T22:35:46.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.539+0000 7fb0bf81e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb0b809d630 0x7fb0b80a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:46.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.539+0000 7fb0bf81e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb0b8138d80 0x7fb0b813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:46.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.539+0000 7fb0bf81e700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0b80a2a70 con 0x7fb0b80a0a30 2023-09-30T22:35:46.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.539+0000 7fb0bf81e700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0b80a01d0 con 0x7fb0b8138d80 2023-09-30T22:35:46.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.539+0000 7fb0bf81e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b809e2b0 con 0x7fb0b809d630 2023-09-30T22:35:46.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.539+0000 7fb0bcdb9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb0b809d630 0x7fb0b80a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:46.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.539+0000 7fb0bcdb9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb0b809d630 0x7fb0b80a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47878/0 (socket says 172.21.15.16:47878) 2023-09-30T22:35:46.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.539+0000 7fb0bcdb9700 1 -- 172.21.15.16:0/127950814 learned_addr learned my addr 172.21.15.16:0/127950814 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:46.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.539+0000 7fb0bddbb700 1 --2- 172.21.15.16:0/127950814 >> v2:172.21.15.16:3300/0 conn(0x7fb0b8138d80 0x7fb0b813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:46.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.539+0000 7fb0b7fff700 1 -- 172.21.15.16:0/127950814 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4252122074 0 0) 0x7fb0b80a2a70 con 0x7fb0b80a0a30 2023-09-30T22:35:46.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.539+0000 7fb0bcdb9700 1 -- 172.21.15.16:0/127950814 >> v2:172.21.15.16:3300/0 conn(0x7fb0b8138d80 msgr2=0x7fb0b813b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:46.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.539+0000 7fb0bcdb9700 1 --2- 172.21.15.16:0/127950814 >> v2:172.21.15.16:3300/0 conn(0x7fb0b8138d80 0x7fb0b813b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:46.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.539+0000 7fb0bcdb9700 1 -- 172.21.15.16:0/127950814 >> v1:172.21.15.16:6789/0 conn(0x7fb0b80a0a30 legacy=0x7fb0b80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:46.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.539+0000 7fb0bcdb9700 1 -- 172.21.15.16:0/127950814 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b809dc50 con 0x7fb0b809d630 2023-09-30T22:35:46.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.539+0000 7fb0bcdb9700 1 --2- 172.21.15.16:0/127950814 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb0b809d630 0x7fb0b80a1510 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac0098f0 tx=0x7fb0ac009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63a0355cc79f70ac server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:46.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.540+0000 7fb0b7fff700 1 -- 172.21.15.16:0/127950814 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac011830 con 0x7fb0b809d630 2023-09-30T22:35:46.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.540+0000 7fb0b7fff700 1 -- 172.21.15.16:0/127950814 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0ac011990 con 0x7fb0b809d630 2023-09-30T22:35:46.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.540+0000 7fb0b7fff700 1 -- 172.21.15.16:0/127950814 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0ac011d20 con 0x7fb0b809d630 2023-09-30T22:35:46.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.541+0000 7fb0bf81e700 1 -- 172.21.15.16:0/127950814 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb0b809d630 msgr2=0x7fb0b80a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:46.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.541+0000 7fb0bf81e700 1 --2- 172.21.15.16:0/127950814 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb0b809d630 0x7fb0b80a1510 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fb0ac0098f0 tx=0x7fb0ac009c00 comp rx=0 tx=0).stop 2023-09-30T22:35:46.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.541+0000 7fb0bf81e700 1 -- 172.21.15.16:0/127950814 shutdown_connections 2023-09-30T22:35:46.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.541+0000 7fb0bf81e700 1 --2- 172.21.15.16:0/127950814 >> v2:172.21.15.16:3300/0 conn(0x7fb0b8138d80 0x7fb0b813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:46.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.541+0000 7fb0bf81e700 1 --2- 172.21.15.16:0/127950814 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb0b809d630 0x7fb0b80a1510 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:46.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.541+0000 7fb0bf81e700 1 -- 172.21.15.16:0/127950814 >> 172.21.15.16:0/127950814 conn(0x7fb0b80a3050 msgr2=0x7fb0b80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:46.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.541+0000 7fb0bf81e700 1 -- 172.21.15.16:0/127950814 shutdown_connections 2023-09-30T22:35:46.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.541+0000 7fb0bf81e700 1 -- 172.21.15.16:0/127950814 wait complete. 2023-09-30T22:35:46.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.542+0000 7fb0bf81e700 1 Processor -- start 2023-09-30T22:35:46.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.542+0000 7fb0bf81e700 1 -- start start 2023-09-30T22:35:46.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.542+0000 7fb0bf81e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0b809d630 0x7fb0b81337d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:46.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.542+0000 7fb0bf81e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb0b80a0a30 0x7fb0b8133d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:46.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.542+0000 7fb0bf81e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb0b8138d80 0x7fb0b8134250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:46.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.542+0000 7fb0bf81e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b8137250 con 0x7fb0b809d630 2023-09-30T22:35:46.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.542+0000 7fb0bf81e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b8136590 con 0x7fb0b8138d80 2023-09-30T22:35:46.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.542+0000 7fb0bf81e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b813ac40 con 0x7fb0b80a0a30 2023-09-30T22:35:46.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.542+0000 7fb0bd5ba700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0b809d630 0x7fb0b81337d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:46.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.542+0000 7fb0bcdb9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb0b80a0a30 0x7fb0b8133d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:46.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.543+0000 7fb0bcdb9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb0b80a0a30 0x7fb0b8133d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:50828/0 (socket says 172.21.15.16:50828) 2023-09-30T22:35:46.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.543+0000 7fb0bcdb9700 1 -- 172.21.15.16:0/1082234769 learned_addr learned my addr 172.21.15.16:0/1082234769 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:46.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.543+0000 7fb0bd5ba700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0b809d630 0x7fb0b81337d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41100/0 (socket says 172.21.15.16:41100) 2023-09-30T22:35:46.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.543+0000 7fb0bddbb700 1 --2- 172.21.15.16:0/1082234769 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb0b8138d80 0x7fb0b8134250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:46.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.543+0000 7fb0bd5ba700 1 -- 172.21.15.16:0/1082234769 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb0b80a0a30 msgr2=0x7fb0b8133d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:46.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.543+0000 7fb0bd5ba700 1 --2- 172.21.15.16:0/1082234769 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb0b80a0a30 0x7fb0b8133d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:46.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.543+0000 7fb0bd5ba700 1 -- 172.21.15.16:0/1082234769 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb0b8138d80 msgr2=0x7fb0b8134250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:46.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.543+0000 7fb0bd5ba700 1 --2- 172.21.15.16:0/1082234769 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb0b8138d80 0x7fb0b8134250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:46.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.543+0000 7fb0bd5ba700 1 -- 172.21.15.16:0/1082234769 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0ac009550 con 0x7fb0b809d630 2023-09-30T22:35:46.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.543+0000 7fb0bd5ba700 1 --2- 172.21.15.16:0/1082234769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0b809d630 0x7fb0b81337d0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fb0a80012e0 tx=0x7fb0a800cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:46.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.543+0000 7fb0b5ffb700 1 -- 172.21.15.16:0/1082234769 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a8003e30 con 0x7fb0b809d630 2023-09-30T22:35:46.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.543+0000 7fb0b5ffb700 1 -- 172.21.15.16:0/1082234769 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0a8005070 con 0x7fb0b809d630 2023-09-30T22:35:46.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.543+0000 7fb0bf81e700 1 -- 172.21.15.16:0/1082234769 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0b80a0410 con 0x7fb0b809d630 2023-09-30T22:35:46.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.543+0000 7fb0b5ffb700 1 -- 172.21.15.16:0/1082234769 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a80053e0 con 0x7fb0b809d630 2023-09-30T22:35:46.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.544+0000 7fb0bf81e700 1 -- 172.21.15.16:0/1082234769 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0b8136d00 con 0x7fb0b809d630 2023-09-30T22:35:46.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.544+0000 7fb0b5ffb700 1 -- 172.21.15.16:0/1082234769 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb0a8005540 con 0x7fb0b809d630 2023-09-30T22:35:46.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.545+0000 7fb0b5ffb700 1 -- 172.21.15.16:0/1082234769 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fb0a803f3f0 con 0x7fb0b809d630 2023-09-30T22:35:46.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.545+0000 7fb0bf81e700 1 -- 172.21.15.16:0/1082234769 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0b80747f0 con 0x7fb0b809d630 2023-09-30T22:35:46.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.550+0000 7fb0b5ffb700 1 -- 172.21.15.16:0/1082234769 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb0a8010710 con 0x7fb0b809d630 2023-09-30T22:35:46.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.676+0000 7fa37527d700 1 -- 172.21.15.16:0/282104579 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa37006eae0 con 0x7fa37009e930 2023-09-30T22:35:46.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.677+0000 7fa3577fe700 1 -- 172.21.15.16:0/282104579 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa37006eae0 con 0x7fa37009e930 2023-09-30T22:35:46.677 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:35:46.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.679+0000 7fa37527d700 1 -- 172.21.15.16:0/282104579 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa37009e930 msgr2=0x7fa37011dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:46.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.679+0000 7fa37527d700 1 --2- 172.21.15.16:0/282104579 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa37009e930 0x7fa37011dc10 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fa36400c400 tx=0x7fa36400c710 comp rx=0 tx=0).stop 2023-09-30T22:35:46.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.679+0000 7fa37527d700 1 -- 172.21.15.16:0/282104579 shutdown_connections 2023-09-30T22:35:46.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.679+0000 7fa37527d700 1 --2- 172.21.15.16:0/282104579 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa37009de50 0x7fa37011d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:46.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.679+0000 7fa37527d700 1 --2- 172.21.15.16:0/282104579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3701338a0 0x7fa37011e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:46.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.679+0000 7fa37527d700 1 --2- 172.21.15.16:0/282104579 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa37009e930 0x7fa37011dc10 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:46.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.679+0000 7fa37527d700 1 -- 172.21.15.16:0/282104579 >> 172.21.15.16:0/282104579 conn(0x7fa3700a2aa0 msgr2=0x7fa3701329f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:46.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.679+0000 7fa37527d700 1 -- 172.21.15.16:0/282104579 shutdown_connections 2023-09-30T22:35:46.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.679+0000 7fa37527d700 1 -- 172.21.15.16:0/282104579 wait complete. 2023-09-30T22:35:46.694 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:35:46.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.717+0000 7f072b853700 1 -- 172.21.15.16:0/2671902137 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f07241209e0 con 0x7f072412ff20 2023-09-30T22:35:46.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.717+0000 7f0719ffb700 1 -- 172.21.15.16:0/2671902137 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f072000cea0 con 0x7f072412ff20 2023-09-30T22:35:46.718 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:35:46.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.720+0000 7f072b853700 1 -- 172.21.15.16:0/2671902137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f072412ff20 msgr2=0x7f072409c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:46.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.720+0000 7f072b853700 1 --2- 172.21.15.16:0/2671902137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f072412ff20 0x7f072409c480 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f072000e910 tx=0x7f072000ecd0 comp rx=0 tx=0).stop 2023-09-30T22:35:46.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.720+0000 7f072b853700 1 -- 172.21.15.16:0/2671902137 shutdown_connections 2023-09-30T22:35:46.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.720+0000 7f072b853700 1 --2- 172.21.15.16:0/2671902137 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0724133a80 0x7f072409cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:46.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.720+0000 7f072b853700 1 --2- 172.21.15.16:0/2671902137 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f072412ff20 0x7f072409c480 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:46.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.720+0000 7f072b853700 1 --2- 172.21.15.16:0/2671902137 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0724130870 0x7f072409c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:46.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.720+0000 7f072b853700 1 -- 172.21.15.16:0/2671902137 >> 172.21.15.16:0/2671902137 conn(0x7f07240a4210 msgr2=0x7f072409e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:46.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.720+0000 7f072b853700 1 -- 172.21.15.16:0/2671902137 shutdown_connections 2023-09-30T22:35:46.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.720+0000 7f072b853700 1 -- 172.21.15.16:0/2671902137 wait complete. 2023-09-30T22:35:46.735 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:35:46.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.759+0000 7fb0bf81e700 1 -- 172.21.15.16:0/1082234769 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb0b8072980 con 0x7fb0b809d630 2023-09-30T22:35:46.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.760+0000 7fb0b5ffb700 1 -- 172.21.15.16:0/1082234769 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb0a8029090 con 0x7fb0b809d630 2023-09-30T22:35:46.760 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:35:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.762+0000 7fb0bf81e700 1 -- 172.21.15.16:0/1082234769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0b809d630 msgr2=0x7fb0b81337d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.762+0000 7fb0bf81e700 1 --2- 172.21.15.16:0/1082234769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0b809d630 0x7fb0b81337d0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fb0a80012e0 tx=0x7fb0a800cac0 comp rx=0 tx=0).stop 2023-09-30T22:35:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.762+0000 7fb0bf81e700 1 -- 172.21.15.16:0/1082234769 shutdown_connections 2023-09-30T22:35:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.762+0000 7fb0bf81e700 1 --2- 172.21.15.16:0/1082234769 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb0b8138d80 0x7fb0b8134250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.762+0000 7fb0bf81e700 1 --2- 172.21.15.16:0/1082234769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb0b809d630 0x7fb0b81337d0 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.762+0000 7fb0bf81e700 1 --2- 172.21.15.16:0/1082234769 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb0b80a0a30 0x7fb0b8133d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.762+0000 7fb0bf81e700 1 -- 172.21.15.16:0/1082234769 >> 172.21.15.16:0/1082234769 conn(0x7fb0b80a3050 msgr2=0x7fb0b80a4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.762+0000 7fb0bf81e700 1 -- 172.21.15.16:0/1082234769 shutdown_connections 2023-09-30T22:35:46.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:46.762+0000 7fb0bf81e700 1 -- 172.21.15.16:0/1082234769 wait complete. 2023-09-30T22:35:46.776 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:35:47.005 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:35:47.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.152+0000 7f9f2c156700 1 Processor -- start 2023-09-30T22:35:47.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.152+0000 7f9f2c156700 1 -- start start 2023-09-30T22:35:47.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.152+0000 7f9f2c156700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9f2409e1d0 0x7f9f24135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:47.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.152+0000 7f9f2c156700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9f24135e50 0x7f9f24138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:47.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.152+0000 7f9f2c156700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f2409f130 con 0x7f9f2409d6f0 2023-09-30T22:35:47.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.152+0000 7f9f2c156700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2409f520 con 0x7f9f2409e1d0 2023-09-30T22:35:47.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.152+0000 7f9f2c156700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2409e930 con 0x7f9f24135e50 2023-09-30T22:35:47.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.152+0000 7f9f296f1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9f2409e1d0 0x7f9f24135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:47.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.152+0000 7f9f29ef2700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9f2409d6f0 0x7f9f2409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:40700/0 (socket says 172.21.15.16:40700) 2023-09-30T22:35:47.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.152+0000 7f9f29ef2700 1 -- 172.21.15.16:0/487507506 learned_addr learned my addr 172.21.15.16:0/487507506 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:47.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.152+0000 7f9f2a6f3700 1 --2- 172.21.15.16:0/487507506 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9f24135e50 0x7f9f24138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:47.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.153+0000 7f9f28ef0700 1 -- 172.21.15.16:0/487507506 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 406825970 0 0) 0x7f9f2409f130 con 0x7f9f2409d6f0 2023-09-30T22:35:47.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.153+0000 7f9f296f1700 1 -- 172.21.15.16:0/487507506 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9f24135e50 msgr2=0x7f9f24138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:47.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.153+0000 7f9f296f1700 1 --2- 172.21.15.16:0/487507506 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9f24135e50 0x7f9f24138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:47.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.153+0000 7f9f296f1700 1 -- 172.21.15.16:0/487507506 >> v1:172.21.15.16:6789/0 conn(0x7f9f2409d6f0 legacy=0x7f9f2409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:47.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.153+0000 7f9f296f1700 1 -- 172.21.15.16:0/487507506 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f2409b2d0 con 0x7f9f2409e1d0 2023-09-30T22:35:47.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.153+0000 7f9f2a6f3700 1 --2- 172.21.15.16:0/487507506 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9f24135e50 0x7f9f24138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:47.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.153+0000 7f9f296f1700 1 --2- 172.21.15.16:0/487507506 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9f2409e1d0 0x7f9f24135910 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f9f14009870 tx=0x7f9f14009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6572094fadbc0fc8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:47.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.153+0000 7f9f28ef0700 1 -- 172.21.15.16:0/487507506 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f14011830 con 0x7f9f2409e1d0 2023-09-30T22:35:47.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.153+0000 7f9f28ef0700 1 -- 172.21.15.16:0/487507506 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f14011990 con 0x7f9f2409e1d0 2023-09-30T22:35:47.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.153+0000 7f9f28ef0700 1 -- 172.21.15.16:0/487507506 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f14011d20 con 0x7f9f2409e1d0 2023-09-30T22:35:47.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.154+0000 7f9f2c156700 1 -- 172.21.15.16:0/487507506 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9f2409e1d0 msgr2=0x7f9f24135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:47.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.154+0000 7f9f2c156700 1 --2- 172.21.15.16:0/487507506 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9f2409e1d0 0x7f9f24135910 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f9f14009870 tx=0x7f9f14009b80 comp rx=0 tx=0).stop 2023-09-30T22:35:47.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.154+0000 7f9f2c156700 1 -- 172.21.15.16:0/487507506 shutdown_connections 2023-09-30T22:35:47.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.154+0000 7f9f2c156700 1 --2- 172.21.15.16:0/487507506 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9f24135e50 0x7f9f24138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:47.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.154+0000 7f9f2c156700 1 --2- 172.21.15.16:0/487507506 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9f2409e1d0 0x7f9f24135910 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:47.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.154+0000 7f9f2c156700 1 -- 172.21.15.16:0/487507506 >> 172.21.15.16:0/487507506 conn(0x7f9f24066f50 msgr2=0x7f9f2409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:47.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.154+0000 7f9f2c156700 1 -- 172.21.15.16:0/487507506 shutdown_connections 2023-09-30T22:35:47.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.154+0000 7f9f2c156700 1 -- 172.21.15.16:0/487507506 wait complete. 2023-09-30T22:35:47.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.155+0000 7f9f2c156700 1 Processor -- start 2023-09-30T22:35:47.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.155+0000 7f9f2c156700 1 -- start start 2023-09-30T22:35:47.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.155+0000 7f9f2c156700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9f2409d6f0 0x7f9f2411d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:47.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.155+0000 7f9f2c156700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f2409e1d0 0x7f9f2411da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:47.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.155+0000 7f9f2c156700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9f24135e50 0x7f9f2411dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:47.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.155+0000 7f9f2c156700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2411ea20 con 0x7f9f2409e1d0 2023-09-30T22:35:47.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.155+0000 7f9f2c156700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2411f110 con 0x7f9f2409d6f0 2023-09-30T22:35:47.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.155+0000 7f9f2c156700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9f24121020 con 0x7f9f24135e50 2023-09-30T22:35:47.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.156+0000 7f9f296f1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f2409e1d0 0x7f9f2411da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:47.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.156+0000 7f9f2a6f3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9f24135e50 0x7f9f2411dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:47.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.156+0000 7f9f296f1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f2409e1d0 0x7f9f2411da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41114/0 (socket says 172.21.15.16:41114) 2023-09-30T22:35:47.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.156+0000 7f9f296f1700 1 -- 172.21.15.16:0/785121073 learned_addr learned my addr 172.21.15.16:0/785121073 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:47.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.156+0000 7f9f29ef2700 1 --2- 172.21.15.16:0/785121073 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9f2409d6f0 0x7f9f2411d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:47.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.156+0000 7f9f2a6f3700 1 -- 172.21.15.16:0/785121073 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9f2409d6f0 msgr2=0x7f9f2411d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:47.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.156+0000 7f9f2a6f3700 1 --2- 172.21.15.16:0/785121073 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9f2409d6f0 0x7f9f2411d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:47.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.156+0000 7f9f2a6f3700 1 -- 172.21.15.16:0/785121073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f2409e1d0 msgr2=0x7f9f2411da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:47.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.156+0000 7f9f2a6f3700 1 --2- 172.21.15.16:0/785121073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f2409e1d0 0x7f9f2411da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:47.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.156+0000 7f9f2a6f3700 1 -- 172.21.15.16:0/785121073 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f1c009a70 con 0x7f9f24135e50 2023-09-30T22:35:47.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.156+0000 7f9f296f1700 1 --2- 172.21.15.16:0/785121073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f2409e1d0 0x7f9f2411da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:47.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.156+0000 7f9f2a6f3700 1 --2- 172.21.15.16:0/785121073 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9f24135e50 0x7f9f2411dfd0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c00ef90 tx=0x7f9f1c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:47.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.156+0000 7f9f1a7fc700 1 -- 172.21.15.16:0/785121073 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c00ca90 con 0x7f9f24135e50 2023-09-30T22:35:47.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.156+0000 7f9f1a7fc700 1 -- 172.21.15.16:0/785121073 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f1c00f040 con 0x7f9f24135e50 2023-09-30T22:35:47.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.156+0000 7f9f2c156700 1 -- 172.21.15.16:0/785121073 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f140094a0 con 0x7f9f24135e50 2023-09-30T22:35:47.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.156+0000 7f9f1a7fc700 1 -- 172.21.15.16:0/785121073 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c00ce30 con 0x7f9f24135e50 2023-09-30T22:35:47.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.156+0000 7f9f2c156700 1 -- 172.21.15.16:0/785121073 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f24120d40 con 0x7f9f24135e50 2023-09-30T22:35:47.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.157+0000 7f9f1a7fc700 1 -- 172.21.15.16:0/785121073 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9f1c014480 con 0x7f9f24135e50 2023-09-30T22:35:47.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.157+0000 7f9f1a7fc700 1 -- 172.21.15.16:0/785121073 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f9f1c03fc70 con 0x7f9f24135e50 2023-09-30T22:35:47.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.157+0000 7f9f2c156700 1 -- 172.21.15.16:0/785121073 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f241209e0 con 0x7f9f24135e50 2023-09-30T22:35:47.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.162+0000 7f9f1a7fc700 1 -- 172.21.15.16:0/785121073 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9f241209e0 con 0x7f9f24135e50 2023-09-30T22:35:47.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.366+0000 7f9f2c156700 1 -- 172.21.15.16:0/785121073 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9f241209e0 con 0x7f9f24135e50 2023-09-30T22:35:47.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.366+0000 7f9f1a7fc700 1 -- 172.21.15.16:0/785121073 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9f241209e0 con 0x7f9f24135e50 2023-09-30T22:35:47.367 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:35:47.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.368+0000 7f9f2c156700 1 -- 172.21.15.16:0/785121073 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9f24135e50 msgr2=0x7f9f2411dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:47.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.369+0000 7f9f2c156700 1 --2- 172.21.15.16:0/785121073 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9f24135e50 0x7f9f2411dfd0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c00ef90 tx=0x7f9f1c00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:35:47.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.369+0000 7f9f2c156700 1 -- 172.21.15.16:0/785121073 shutdown_connections 2023-09-30T22:35:47.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.369+0000 7f9f2c156700 1 --2- 172.21.15.16:0/785121073 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9f2409d6f0 0x7f9f2411d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:47.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.369+0000 7f9f2c156700 1 --2- 172.21.15.16:0/785121073 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9f24135e50 0x7f9f2411dfd0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:47.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.369+0000 7f9f2c156700 1 --2- 172.21.15.16:0/785121073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f2409e1d0 0x7f9f2411da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:47.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.369+0000 7f9f2c156700 1 -- 172.21.15.16:0/785121073 >> 172.21.15.16:0/785121073 conn(0x7f9f24066f50 msgr2=0x7f9f24134a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:47.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.369+0000 7f9f2c156700 1 -- 172.21.15.16:0/785121073 shutdown_connections 2023-09-30T22:35:47.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.369+0000 7f9f2c156700 1 -- 172.21.15.16:0/785121073 wait complete. 2023-09-30T22:35:47.383 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:35:47.694 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:35:47.736 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:35:47.777 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:35:47.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.845+0000 7f0253553700 1 Processor -- start 2023-09-30T22:35:47.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.846+0000 7f0253553700 1 -- start start 2023-09-30T22:35:47.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.846+0000 7f0253553700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f024c09e930 0x7f024c09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:47.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.846+0000 7f0253553700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f024c09f650 0x7f024c131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:47.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.846+0000 7f0253553700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f024c070940 con 0x7f024c09f240 2023-09-30T22:35:47.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.846+0000 7f0253553700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f024c06fd60 con 0x7f024c09f650 2023-09-30T22:35:47.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.846+0000 7f0253553700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f024c06fa80 con 0x7f024c09e930 2023-09-30T22:35:47.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.846+0000 7f0251af0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f024c09f650 0x7f024c131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:47.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.846+0000 7f02512ef700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f024c09e930 0x7f024c09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:47.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.846+0000 7f0251af0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f024c09f650 0x7f024c131f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47172/0 (socket says 172.21.15.16:47172) 2023-09-30T22:35:47.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.847+0000 7f02512ef700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f024c09e930 0x7f024c09ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:34884/0 (socket says 172.21.15.16:34884) 2023-09-30T22:35:47.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.847+0000 7f0251af0700 1 -- 172.21.15.16:0/1431844385 learned_addr learned my addr 172.21.15.16:0/1431844385 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:47.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.846+0000 7f0250aee700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f024c09f240 0x7f024c1317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56920/0 (socket says 172.21.15.16:56920) 2023-09-30T22:35:47.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.847+0000 7f0251af0700 1 -- 172.21.15.16:0/1431844385 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f024c09e930 msgr2=0x7f024c09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:47.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.847+0000 7f0243fff700 1 -- 172.21.15.16:0/1431844385 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3207526798 0 0) 0x7f024c070940 con 0x7f024c09f240 2023-09-30T22:35:47.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.847+0000 7f0251af0700 1 --2- 172.21.15.16:0/1431844385 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f024c09e930 0x7f024c09ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:47.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.847+0000 7f0251af0700 1 -- 172.21.15.16:0/1431844385 >> v1:172.21.15.16:6789/0 conn(0x7f024c09f240 legacy=0x7f024c1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:47.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.847+0000 7f0251af0700 1 -- 172.21.15.16:0/1431844385 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f024c09fb60 con 0x7f024c09f650 2023-09-30T22:35:47.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.847+0000 7f02512ef700 1 --2- 172.21.15.16:0/1431844385 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f024c09e930 0x7f024c09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:47.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.847+0000 7f0251af0700 1 --2- 172.21.15.16:0/1431844385 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f024c09f650 0x7f024c131f00 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f0244009e80 tx=0x7f024400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=477e3536bfb7952f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:47.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.847+0000 7f0243fff700 1 -- 172.21.15.16:0/1431844385 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024400e070 con 0x7f024c09f650 2023-09-30T22:35:47.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.847+0000 7f0243fff700 1 -- 172.21.15.16:0/1431844385 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f024400b910 con 0x7f024c09f650 2023-09-30T22:35:47.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.847+0000 7f0243fff700 1 -- 172.21.15.16:0/1431844385 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024400bcc0 con 0x7f024c09f650 2023-09-30T22:35:47.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.848+0000 7f0253553700 1 -- 172.21.15.16:0/1431844385 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f024c09f650 msgr2=0x7f024c131f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:47.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.848+0000 7f0253553700 1 --2- 172.21.15.16:0/1431844385 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f024c09f650 0x7f024c131f00 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f0244009e80 tx=0x7f024400b330 comp rx=0 tx=0).stop 2023-09-30T22:35:47.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.848+0000 7f0253553700 1 -- 172.21.15.16:0/1431844385 shutdown_connections 2023-09-30T22:35:47.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.848+0000 7f0253553700 1 --2- 172.21.15.16:0/1431844385 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f024c09f650 0x7f024c131f00 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:47.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.848+0000 7f0253553700 1 --2- 172.21.15.16:0/1431844385 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f024c09e930 0x7f024c09ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:47.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.848+0000 7f0253553700 1 -- 172.21.15.16:0/1431844385 >> 172.21.15.16:0/1431844385 conn(0x7f024c0a4210 msgr2=0x7f024c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:47.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.848+0000 7f0253553700 1 -- 172.21.15.16:0/1431844385 shutdown_connections 2023-09-30T22:35:47.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.848+0000 7f0253553700 1 -- 172.21.15.16:0/1431844385 wait complete. 2023-09-30T22:35:47.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.850+0000 7f0253553700 1 Processor -- start 2023-09-30T22:35:47.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.850+0000 7f0253553700 1 -- start start 2023-09-30T22:35:47.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.851+0000 7f0253553700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f024c09e930 0x7f024c1a3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:47.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.851+0000 7f0253553700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f024c09f240 0x7f024c1c5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:47.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.851+0000 7f0253553700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f024c09f650 0x7f024c1c6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:47.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.851+0000 7f0253553700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f024c1a4f60 con 0x7f024c09f240 2023-09-30T22:35:47.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.851+0000 7f0253553700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f024c1b6f70 con 0x7f024c09f650 2023-09-30T22:35:47.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.851+0000 7f0253553700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f024c071d20 con 0x7f024c09e930 2023-09-30T22:35:47.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.851+0000 7f0250aee700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f024c09f240 0x7f024c1c5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:47.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.851+0000 7f02512ef700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f024c09e930 0x7f024c1a3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:47.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.851+0000 7f0250aee700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f024c09f240 0x7f024c1c5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38490/0 (socket says 172.21.15.16:38490) 2023-09-30T22:35:47.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.851+0000 7f02512ef700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f024c09e930 0x7f024c1a3200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:34898/0 (socket says 172.21.15.16:34898) 2023-09-30T22:35:47.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.851+0000 7f0250aee700 1 -- 172.21.15.16:0/1228010495 learned_addr learned my addr 172.21.15.16:0/1228010495 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:47.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.851+0000 7f0251af0700 1 --2- 172.21.15.16:0/1228010495 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f024c09f650 0x7f024c1c6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:47.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.851+0000 7f0250aee700 1 -- 172.21.15.16:0/1228010495 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f024c09e930 msgr2=0x7f024c1a3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:47.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.851+0000 7f0250aee700 1 --2- 172.21.15.16:0/1228010495 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f024c09e930 0x7f024c1a3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:47.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.851+0000 7f0250aee700 1 -- 172.21.15.16:0/1228010495 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f024c09f650 msgr2=0x7f024c1c6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:47.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.851+0000 7f0250aee700 1 --2- 172.21.15.16:0/1228010495 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f024c09f650 0x7f024c1c6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:47.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.851+0000 7f0250aee700 1 -- 172.21.15.16:0/1228010495 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02480094a0 con 0x7f024c09f240 2023-09-30T22:35:47.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.851+0000 7f02512ef700 1 --2- 172.21.15.16:0/1228010495 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f024c09e930 0x7f024c1a3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:47.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.851+0000 7f0251af0700 1 --2- 172.21.15.16:0/1228010495 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f024c09f650 0x7f024c1c6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:47.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.851+0000 7f0250aee700 1 --2- 172.21.15.16:0/1228010495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f024c09f240 0x7f024c1c5e20 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f023c001eb0 tx=0x7f023c00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:47.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.852+0000 7f0241ffb700 1 -- 172.21.15.16:0/1228010495 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023c004c80 con 0x7f024c09f240 2023-09-30T22:35:47.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.852+0000 7f0241ffb700 1 -- 172.21.15.16:0/1228010495 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f023c004de0 con 0x7f024c09f240 2023-09-30T22:35:47.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.852+0000 7f0253553700 1 -- 172.21.15.16:0/1228010495 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0244009b30 con 0x7f024c09f240 2023-09-30T22:35:47.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.852+0000 7f0241ffb700 1 -- 172.21.15.16:0/1228010495 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023c007310 con 0x7f024c09f240 2023-09-30T22:35:47.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.852+0000 7f0253553700 1 -- 172.21.15.16:0/1228010495 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f024c0a0190 con 0x7f024c09f240 2023-09-30T22:35:47.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.853+0000 7f0241ffb700 1 -- 172.21.15.16:0/1228010495 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f023c007470 con 0x7f024c09f240 2023-09-30T22:35:47.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.853+0000 7f0241ffb700 1 -- 172.21.15.16:0/1228010495 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f023c03f1f0 con 0x7f024c09f240 2023-09-30T22:35:47.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.853+0000 7f0253553700 1 -- 172.21.15.16:0/1228010495 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f024c074ea0 con 0x7f024c09f240 2023-09-30T22:35:47.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.858+0000 7f0241ffb700 1 -- 172.21.15.16:0/1228010495 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f023c0123b0 con 0x7f024c09f240 2023-09-30T22:35:47.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.883+0000 7f28638f4700 1 Processor -- start 2023-09-30T22:35:47.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.884+0000 7f28638f4700 1 -- start start 2023-09-30T22:35:47.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.884+0000 7f28638f4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f285c0a0a30 0x7f285c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:47.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.884+0000 7f28638f4700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f285c138d80 0x7f285c13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:47.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.884+0000 7f28638f4700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f285c0a2a70 con 0x7f285c09d630 2023-09-30T22:35:47.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.884+0000 7f28638f4700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f285c0a01d0 con 0x7f285c138d80 2023-09-30T22:35:47.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.884+0000 7f28638f4700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f285c09e2b0 con 0x7f285c0a0a30 2023-09-30T22:35:47.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.884+0000 7f2861690700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f285c0a0a30 0x7f285c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:47.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.884+0000 7f2860e8f700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f285c09d630 0x7f285c0a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56936/0 (socket says 172.21.15.16:56936) 2023-09-30T22:35:47.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.884+0000 7f2860e8f700 1 -- 172.21.15.16:0/3431582425 learned_addr learned my addr 172.21.15.16:0/3431582425 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:47.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.884+0000 7f2861e91700 1 --2- 172.21.15.16:0/3431582425 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f285c138d80 0x7f285c13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:47.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.885+0000 7f2861690700 1 -- 172.21.15.16:0/3431582425 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f285c138d80 msgr2=0x7f285c13b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:47.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.885+0000 7f285bfff700 1 -- 172.21.15.16:0/3431582425 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3055405775 0 0) 0x7f285c0a2a70 con 0x7f285c09d630 2023-09-30T22:35:47.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.885+0000 7f2861690700 1 --2- 172.21.15.16:0/3431582425 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f285c138d80 0x7f285c13b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:47.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.885+0000 7f2861690700 1 -- 172.21.15.16:0/3431582425 >> v1:172.21.15.16:6789/0 conn(0x7f285c09d630 legacy=0x7f285c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:47.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.885+0000 7f2861690700 1 -- 172.21.15.16:0/3431582425 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f285c09dc50 con 0x7f285c0a0a30 2023-09-30T22:35:47.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.885+0000 7f2861690700 1 --2- 172.21.15.16:0/3431582425 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f285c0a0a30 0x7f285c0a0e00 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f2854009870 tx=0x7f2854009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a12fe7d39b7cf02a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:47.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.885+0000 7f285bfff700 1 -- 172.21.15.16:0/3431582425 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2854011830 con 0x7f285c0a0a30 2023-09-30T22:35:47.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.885+0000 7f285bfff700 1 -- 172.21.15.16:0/3431582425 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2854011990 con 0x7f285c0a0a30 2023-09-30T22:35:47.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.885+0000 7f285bfff700 1 -- 172.21.15.16:0/3431582425 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2854011d20 con 0x7f285c0a0a30 2023-09-30T22:35:47.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.886+0000 7f28638f4700 1 -- 172.21.15.16:0/3431582425 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f285c0a0a30 msgr2=0x7f285c0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:47.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.886+0000 7f28638f4700 1 --2- 172.21.15.16:0/3431582425 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f285c0a0a30 0x7f285c0a0e00 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f2854009870 tx=0x7f2854009b80 comp rx=0 tx=0).stop 2023-09-30T22:35:47.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.886+0000 7f28638f4700 1 -- 172.21.15.16:0/3431582425 shutdown_connections 2023-09-30T22:35:47.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.886+0000 7f28638f4700 1 --2- 172.21.15.16:0/3431582425 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f285c138d80 0x7f285c13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:47.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.886+0000 7f28638f4700 1 --2- 172.21.15.16:0/3431582425 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f285c0a0a30 0x7f285c0a0e00 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:47.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.886+0000 7f28638f4700 1 -- 172.21.15.16:0/3431582425 >> 172.21.15.16:0/3431582425 conn(0x7f285c0a3050 msgr2=0x7f285c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:47.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.886+0000 7f28638f4700 1 -- 172.21.15.16:0/3431582425 shutdown_connections 2023-09-30T22:35:47.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.886+0000 7f28638f4700 1 -- 172.21.15.16:0/3431582425 wait complete. 2023-09-30T22:35:47.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.887+0000 7f28638f4700 1 Processor -- start 2023-09-30T22:35:47.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.887+0000 7f28638f4700 1 -- start start 2023-09-30T22:35:47.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.888+0000 7f28638f4700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f285c09d630 0x7f285c1337e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:47.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.888+0000 7f28638f4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f285c0a0a30 0x7f285c133d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:47.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.888+0000 7f28638f4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f285c138d80 0x7f285c134260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:47.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.888+0000 7f28638f4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f285c136f80 con 0x7f285c0a0a30 2023-09-30T22:35:47.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.888+0000 7f28638f4700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f285c135970 con 0x7f285c09d630 2023-09-30T22:35:47.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.888+0000 7f28638f4700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f285c0a0480 con 0x7f285c138d80 2023-09-30T22:35:47.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.888+0000 7f2860e8f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f285c0a0a30 0x7f285c133d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:47.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.888+0000 7f2860e8f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f285c0a0a30 0x7f285c133d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38502/0 (socket says 172.21.15.16:38502) 2023-09-30T22:35:47.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.888+0000 7f2860e8f700 1 -- 172.21.15.16:0/3942985860 learned_addr learned my addr 172.21.15.16:0/3942985860 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:47.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.888+0000 7f2861690700 1 --2- 172.21.15.16:0/3942985860 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f285c09d630 0x7f285c1337e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:47.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.888+0000 7f2861690700 1 -- 172.21.15.16:0/3942985860 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f285c138d80 msgr2=0x7f285c134260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:47.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.888+0000 7f2861690700 1 --2- 172.21.15.16:0/3942985860 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f285c138d80 0x7f285c134260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:47.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.888+0000 7f2861690700 1 -- 172.21.15.16:0/3942985860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f285c0a0a30 msgr2=0x7f285c133d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:47.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.888+0000 7f2861690700 1 --2- 172.21.15.16:0/3942985860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f285c0a0a30 0x7f285c133d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:47.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.888+0000 7f2861690700 1 -- 172.21.15.16:0/3942985860 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28540094a0 con 0x7f285c09d630 2023-09-30T22:35:47.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.888+0000 7f2860e8f700 1 --2- 172.21.15.16:0/3942985860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f285c0a0a30 0x7f285c133d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:47.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.889+0000 7f2861690700 1 --2- 172.21.15.16:0/3942985860 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f285c09d630 0x7f285c1337e0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f285c0672b0 tx=0x7f2854004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:47.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.889+0000 7f2859ffb700 1 -- 172.21.15.16:0/3942985860 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2854004550 con 0x7f285c09d630 2023-09-30T22:35:47.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.889+0000 7f2859ffb700 1 -- 172.21.15.16:0/3942985860 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f285401f070 con 0x7f285c09d630 2023-09-30T22:35:47.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.889+0000 7f28638f4700 1 -- 172.21.15.16:0/3942985860 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f285c136cc0 con 0x7f285c09d630 2023-09-30T22:35:47.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.889+0000 7f2859ffb700 1 -- 172.21.15.16:0/3942985860 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28540038e0 con 0x7f285c09d630 2023-09-30T22:35:47.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.889+0000 7f28638f4700 1 -- 172.21.15.16:0/3942985860 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f285c135b40 con 0x7f285c09d630 2023-09-30T22:35:47.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.890+0000 7f2859ffb700 1 -- 172.21.15.16:0/3942985860 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2854003a40 con 0x7f285c09d630 2023-09-30T22:35:47.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.890+0000 7f2859ffb700 1 -- 172.21.15.16:0/3942985860 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f2854021070 con 0x7f285c09d630 2023-09-30T22:35:47.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.890+0000 7f28638f4700 1 -- 172.21.15.16:0/3942985860 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f285c13b620 con 0x7f285c09d630 2023-09-30T22:35:47.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.895+0000 7f2859ffb700 1 -- 172.21.15.16:0/3942985860 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2854040e60 con 0x7f285c09d630 2023-09-30T22:35:47.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.955+0000 7f81b2a9c700 1 Processor -- start 2023-09-30T22:35:47.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.955+0000 7f81b2a9c700 1 -- start start 2023-09-30T22:35:47.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.955+0000 7f81b2a9c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f81ac131810 0x7f81ac131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:47.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.955+0000 7f81b2a9c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f81ac099a40 0x7f81ac099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:47.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.955+0000 7f81b2a9c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f81ac09a3b0 0x7f81ac138650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:47.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.955+0000 7f81b2a9c700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81ac09b580 con 0x7f81ac09a3b0 2023-09-30T22:35:47.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.955+0000 7f81b2a9c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81ac1313c0 con 0x7f81ac131810 2023-09-30T22:35:47.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.955+0000 7f81b2a9c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f81ac06e8c0 con 0x7f81ac099a40 2023-09-30T22:35:47.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.956+0000 7f81b1039700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f81ac09a3b0 0x7f81ac138650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:47.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.956+0000 7f81b1039700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f81ac09a3b0 0x7f81ac138650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38508/0 (socket says 172.21.15.16:38508) 2023-09-30T22:35:47.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.956+0000 7f81b1039700 1 -- 172.21.15.16:0/4282878992 learned_addr learned my addr 172.21.15.16:0/4282878992 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:47.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.956+0000 7f81b0838700 1 --2- 172.21.15.16:0/4282878992 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f81ac131810 0x7f81ac131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:47.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.956+0000 7f81abfff700 1 --2- 172.21.15.16:0/4282878992 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f81ac099a40 0x7f81ac099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:47.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.956+0000 7f81b1039700 1 -- 172.21.15.16:0/4282878992 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f81ac099a40 msgr2=0x7f81ac099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:47.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.956+0000 7f81b1039700 1 --2- 172.21.15.16:0/4282878992 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f81ac099a40 0x7f81ac099e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:47.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.956+0000 7f81b1039700 1 -- 172.21.15.16:0/4282878992 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f81ac131810 msgr2=0x7f81ac131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:47.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.956+0000 7f81b1039700 1 --2- 172.21.15.16:0/4282878992 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f81ac131810 0x7f81ac131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:47.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.956+0000 7f81b1039700 1 -- 172.21.15.16:0/4282878992 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81ac09a820 con 0x7f81ac09a3b0 2023-09-30T22:35:47.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.956+0000 7f81b0838700 1 --2- 172.21.15.16:0/4282878992 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f81ac131810 0x7f81ac131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:47.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.956+0000 7f81abfff700 1 --2- 172.21.15.16:0/4282878992 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f81ac099a40 0x7f81ac099e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:35:47.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.956+0000 7f81b1039700 1 --2- 172.21.15.16:0/4282878992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81ac09a3b0 0x7f81ac138650 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f81a4009e50 tx=0x7f81a400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fbc786afe1ae851 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:47.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.956+0000 7f81ab7fe700 1 -- 172.21.15.16:0/4282878992 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a400e070 con 0x7f81ac09a3b0 2023-09-30T22:35:47.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.956+0000 7f81ab7fe700 1 -- 172.21.15.16:0/4282878992 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a400b910 con 0x7f81ac09a3b0 2023-09-30T22:35:47.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.956+0000 7f81ab7fe700 1 -- 172.21.15.16:0/4282878992 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a400bcc0 con 0x7f81ac09a3b0 2023-09-30T22:35:47.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.957+0000 7f81b2a9c700 1 -- 172.21.15.16:0/4282878992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81ac09a3b0 msgr2=0x7f81ac138650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:47.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.957+0000 7f81b2a9c700 1 --2- 172.21.15.16:0/4282878992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81ac09a3b0 0x7f81ac138650 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f81a4009e50 tx=0x7f81a400b330 comp rx=0 tx=0).stop 2023-09-30T22:35:47.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.957+0000 7f81b2a9c700 1 -- 172.21.15.16:0/4282878992 shutdown_connections 2023-09-30T22:35:47.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.957+0000 7f81b2a9c700 1 --2- 172.21.15.16:0/4282878992 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f81ac131810 0x7f81ac131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:47.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.957+0000 7f81b2a9c700 1 --2- 172.21.15.16:0/4282878992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81ac09a3b0 0x7f81ac138650 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:47.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.957+0000 7f81b2a9c700 1 --2- 172.21.15.16:0/4282878992 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f81ac099a40 0x7f81ac099e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:47.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.957+0000 7f81b2a9c700 1 -- 172.21.15.16:0/4282878992 >> 172.21.15.16:0/4282878992 conn(0x7f81ac0a2aa0 msgr2=0x7f81ac0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:47.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.957+0000 7f81b2a9c700 1 -- 172.21.15.16:0/4282878992 shutdown_connections 2023-09-30T22:35:47.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.957+0000 7f81b2a9c700 1 -- 172.21.15.16:0/4282878992 wait complete. 2023-09-30T22:35:47.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.958+0000 7f81b2a9c700 1 Processor -- start 2023-09-30T22:35:47.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.958+0000 7f81b2a9c700 1 -- start start 2023-09-30T22:35:47.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.959+0000 7f81b2a9c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f81ac099a40 0x7f81ac118cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:47.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.959+0000 7f81b2a9c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81ac09a3b0 0x7f81ac119210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:47.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.959+0000 7f81b2a9c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f81ac131810 0x7f81ac119750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:47.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.959+0000 7f81b2a9c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81ac09e6a0 con 0x7f81ac09a3b0 2023-09-30T22:35:47.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.959+0000 7f81b2a9c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81ac11c1d0 con 0x7f81ac131810 2023-09-30T22:35:47.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.959+0000 7f81b2a9c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f81ac130770 con 0x7f81ac099a40 2023-09-30T22:35:47.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.959+0000 7f81b0838700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f81ac099a40 0x7f81ac118cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:47.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.959+0000 7f81b1039700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f81ac131810 0x7f81ac119750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:47.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.959+0000 7f81b0838700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f81ac099a40 0x7f81ac118cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:34914/0 (socket says 172.21.15.16:34914) 2023-09-30T22:35:47.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.959+0000 7f81b1039700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f81ac131810 0x7f81ac119750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47228/0 (socket says 172.21.15.16:47228) 2023-09-30T22:35:47.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.959+0000 7f81b0838700 1 -- 172.21.15.16:0/1251583502 learned_addr learned my addr 172.21.15.16:0/1251583502 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:47.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.959+0000 7f81abfff700 1 --2- 172.21.15.16:0/1251583502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81ac09a3b0 0x7f81ac119210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:47.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.960+0000 7f81b1039700 1 -- 172.21.15.16:0/1251583502 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f81ac099a40 msgr2=0x7f81ac118cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:47.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.960+0000 7f81b1039700 1 --2- 172.21.15.16:0/1251583502 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f81ac099a40 0x7f81ac118cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:47.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.960+0000 7f81b1039700 1 -- 172.21.15.16:0/1251583502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81ac09a3b0 msgr2=0x7f81ac119210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:47.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.960+0000 7f81b1039700 1 --2- 172.21.15.16:0/1251583502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81ac09a3b0 0x7f81ac119210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:47.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.960+0000 7f81b1039700 1 -- 172.21.15.16:0/1251583502 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a00094a0 con 0x7f81ac131810 2023-09-30T22:35:47.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.960+0000 7f81b0838700 1 --2- 172.21.15.16:0/1251583502 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f81ac099a40 0x7f81ac118cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:47.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.960+0000 7f81abfff700 1 --2- 172.21.15.16:0/1251583502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81ac09a3b0 0x7f81ac119210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:47.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.960+0000 7f81b1039700 1 --2- 172.21.15.16:0/1251583502 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f81ac131810 0x7f81ac119750 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f81a4015000 tx=0x7f81a4004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:47.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.960+0000 7f81a97fa700 1 -- 172.21.15.16:0/1251583502 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a40092e0 con 0x7f81ac131810 2023-09-30T22:35:47.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.960+0000 7f81a97fa700 1 -- 172.21.15.16:0/1251583502 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81a4004040 con 0x7f81ac131810 2023-09-30T22:35:47.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.960+0000 7f81a97fa700 1 -- 172.21.15.16:0/1251583502 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81a40043f0 con 0x7f81ac131810 2023-09-30T22:35:47.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.960+0000 7f81b2a9c700 1 -- 172.21.15.16:0/1251583502 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81a4009b30 con 0x7f81ac131810 2023-09-30T22:35:47.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.960+0000 7f81b2a9c700 1 -- 172.21.15.16:0/1251583502 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81ac06eb00 con 0x7f81ac131810 2023-09-30T22:35:47.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.961+0000 7f81a97fa700 1 -- 172.21.15.16:0/1251583502 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f81a4004550 con 0x7f81ac131810 2023-09-30T22:35:47.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.961+0000 7f81b2a9c700 1 -- 172.21.15.16:0/1251583502 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81ac11c000 con 0x7f81ac131810 2023-09-30T22:35:47.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.962+0000 7f81a97fa700 1 -- 172.21.15.16:0/1251583502 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f81a4040810 con 0x7f81ac131810 2023-09-30T22:35:47.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:47.971+0000 7f81a97fa700 1 -- 172.21.15.16:0/1251583502 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f81a4019350 con 0x7f81ac131810 2023-09-30T22:35:48.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.106+0000 7f0253553700 1 -- 172.21.15.16:0/1228010495 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f024c09ffe0 con 0x7f024c09f240 2023-09-30T22:35:48.107 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.107+0000 7f0241ffb700 1 -- 172.21.15.16:0/1228010495 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f023c011360 con 0x7f024c09f240 2023-09-30T22:35:48.107 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:35:48.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.107+0000 7f28638f4700 1 -- 172.21.15.16:0/3942985860 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f285c0747f0 con 0x7f285c09d630 2023-09-30T22:35:48.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.108+0000 7f2859ffb700 1 -- 172.21.15.16:0/3942985860 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2854015020 con 0x7f285c09d630 2023-09-30T22:35:48.108 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:35:48.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.110+0000 7f0253553700 1 -- 172.21.15.16:0/1228010495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f024c09f240 msgr2=0x7f024c1c5e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:48.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.110+0000 7f0253553700 1 --2- 172.21.15.16:0/1228010495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f024c09f240 0x7f024c1c5e20 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f023c001eb0 tx=0x7f023c00c800 comp rx=0 tx=0).stop 2023-09-30T22:35:48.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.110+0000 7f0253553700 1 -- 172.21.15.16:0/1228010495 shutdown_connections 2023-09-30T22:35:48.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.110+0000 7f0253553700 1 --2- 172.21.15.16:0/1228010495 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f024c09f650 0x7f024c1c6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:48.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.110+0000 7f0253553700 1 --2- 172.21.15.16:0/1228010495 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f024c09e930 0x7f024c1a3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:48.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.110+0000 7f0253553700 1 --2- 172.21.15.16:0/1228010495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f024c09f240 0x7f024c1c5e20 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:48.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.110+0000 7f0253553700 1 -- 172.21.15.16:0/1228010495 >> 172.21.15.16:0/1228010495 conn(0x7f024c0a4210 msgr2=0x7f024c0a27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:48.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.110+0000 7f0253553700 1 -- 172.21.15.16:0/1228010495 shutdown_connections 2023-09-30T22:35:48.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.110+0000 7f0253553700 1 -- 172.21.15.16:0/1228010495 wait complete. 2023-09-30T22:35:48.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.111+0000 7f28638f4700 1 -- 172.21.15.16:0/3942985860 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f285c09d630 msgr2=0x7f285c1337e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:48.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.111+0000 7f28638f4700 1 --2- 172.21.15.16:0/3942985860 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f285c09d630 0x7f285c1337e0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f285c0672b0 tx=0x7f2854004750 comp rx=0 tx=0).stop 2023-09-30T22:35:48.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.111+0000 7f28638f4700 1 -- 172.21.15.16:0/3942985860 shutdown_connections 2023-09-30T22:35:48.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.111+0000 7f28638f4700 1 --2- 172.21.15.16:0/3942985860 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f285c09d630 0x7f285c1337e0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:48.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.111+0000 7f28638f4700 1 --2- 172.21.15.16:0/3942985860 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f285c138d80 0x7f285c134260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:48.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.111+0000 7f28638f4700 1 --2- 172.21.15.16:0/3942985860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f285c0a0a30 0x7f285c133d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:48.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.111+0000 7f28638f4700 1 -- 172.21.15.16:0/3942985860 >> 172.21.15.16:0/3942985860 conn(0x7f285c0a3050 msgr2=0x7f285c0a6960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:48.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.112+0000 7f28638f4700 1 -- 172.21.15.16:0/3942985860 shutdown_connections 2023-09-30T22:35:48.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.112+0000 7f28638f4700 1 -- 172.21.15.16:0/3942985860 wait complete. 2023-09-30T22:35:48.131 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:35:48.131 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:35:48.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.175+0000 7f81b2a9c700 1 -- 172.21.15.16:0/1251583502 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f81ac071d80 con 0x7f81ac131810 2023-09-30T22:35:48.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.175+0000 7f81a97fa700 1 -- 172.21.15.16:0/1251583502 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f81a4011b80 con 0x7f81ac131810 2023-09-30T22:35:48.176 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:35:48.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.177+0000 7f81b2a9c700 1 -- 172.21.15.16:0/1251583502 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f81ac131810 msgr2=0x7f81ac119750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:48.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.177+0000 7f81b2a9c700 1 --2- 172.21.15.16:0/1251583502 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f81ac131810 0x7f81ac119750 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f81a4015000 tx=0x7f81a4004ad0 comp rx=0 tx=0).stop 2023-09-30T22:35:48.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.177+0000 7f81b2a9c700 1 -- 172.21.15.16:0/1251583502 shutdown_connections 2023-09-30T22:35:48.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.177+0000 7f81b2a9c700 1 --2- 172.21.15.16:0/1251583502 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f81ac131810 0x7f81ac119750 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:48.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.177+0000 7f81b2a9c700 1 --2- 172.21.15.16:0/1251583502 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f81ac099a40 0x7f81ac118cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:48.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.177+0000 7f81b2a9c700 1 --2- 172.21.15.16:0/1251583502 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f81ac09a3b0 0x7f81ac119210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:48.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.177+0000 7f81b2a9c700 1 -- 172.21.15.16:0/1251583502 >> 172.21.15.16:0/1251583502 conn(0x7f81ac0a2aa0 msgr2=0x7f81ac067820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:48.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.177+0000 7f81b2a9c700 1 -- 172.21.15.16:0/1251583502 shutdown_connections 2023-09-30T22:35:48.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.177+0000 7f81b2a9c700 1 -- 172.21.15.16:0/1251583502 wait complete. 2023-09-30T22:35:48.191 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:35:48.384 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:35:48.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.530+0000 7faeea2cb700 1 Processor -- start 2023-09-30T22:35:48.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.530+0000 7faeea2cb700 1 -- start start 2023-09-30T22:35:48.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.531+0000 7faeea2cb700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faee406b250 0x7faee4136840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:48.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.531+0000 7faeea2cb700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faee4136d80 0x7faee4139160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:48.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.531+0000 7faeea2cb700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faee409ece0 con 0x7faee406a770 2023-09-30T22:35:48.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.531+0000 7faeea2cb700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faee406b640 con 0x7faee406b250 2023-09-30T22:35:48.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.531+0000 7faeea2cb700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faee409e760 con 0x7faee4136d80 2023-09-30T22:35:48.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.531+0000 7faee37fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faee406b250 0x7faee4136840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:48.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.531+0000 7faee8868700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faee4136d80 0x7faee4139160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:48.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.531+0000 7faee3fff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7faee406a770 0x7faee406ab40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56952/0 (socket says 172.21.15.16:56952) 2023-09-30T22:35:48.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.531+0000 7faee3fff700 1 -- 172.21.15.16:0/1084361983 learned_addr learned my addr 172.21.15.16:0/1084361983 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:48.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.531+0000 7faee2ffd700 1 -- 172.21.15.16:0/1084361983 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2190855265 0 0) 0x7faee409ece0 con 0x7faee406a770 2023-09-30T22:35:48.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.531+0000 7faee2ffd700 1 -- 172.21.15.16:0/1084361983 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faecc0035b0 con 0x7faee406a770 2023-09-30T22:35:48.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.532+0000 7faee37fe700 1 -- 172.21.15.16:0/1084361983 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faee4136d80 msgr2=0x7faee4139160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:48.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.532+0000 7faee37fe700 1 --2- 172.21.15.16:0/1084361983 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faee4136d80 0x7faee4139160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:48.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.532+0000 7faee37fe700 1 -- 172.21.15.16:0/1084361983 >> v1:172.21.15.16:6789/0 conn(0x7faee406a770 legacy=0x7faee406ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:48.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.532+0000 7faee37fe700 1 -- 172.21.15.16:0/1084361983 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faee409e040 con 0x7faee406b250 2023-09-30T22:35:48.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.532+0000 7faee8868700 1 --2- 172.21.15.16:0/1084361983 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faee4136d80 0x7faee4139160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:48.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.532+0000 7faee37fe700 1 --2- 172.21.15.16:0/1084361983 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faee406b250 0x7faee4136840 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7faed8009870 tx=0x7faed8009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cdd68e35cace213d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:48.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.532+0000 7faee2ffd700 1 -- 172.21.15.16:0/1084361983 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed8011830 con 0x7faee406b250 2023-09-30T22:35:48.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.532+0000 7faee2ffd700 1 -- 172.21.15.16:0/1084361983 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faed8011990 con 0x7faee406b250 2023-09-30T22:35:48.534 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.532+0000 7faee2ffd700 1 -- 172.21.15.16:0/1084361983 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed8011d20 con 0x7faee406b250 2023-09-30T22:35:48.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.533+0000 7faeea2cb700 1 -- 172.21.15.16:0/1084361983 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faee406b250 msgr2=0x7faee4136840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:48.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.533+0000 7faeea2cb700 1 --2- 172.21.15.16:0/1084361983 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faee406b250 0x7faee4136840 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7faed8009870 tx=0x7faed8009b80 comp rx=0 tx=0).stop 2023-09-30T22:35:48.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.533+0000 7faeea2cb700 1 -- 172.21.15.16:0/1084361983 shutdown_connections 2023-09-30T22:35:48.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.533+0000 7faeea2cb700 1 --2- 172.21.15.16:0/1084361983 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faee4136d80 0x7faee4139160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:48.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.533+0000 7faeea2cb700 1 --2- 172.21.15.16:0/1084361983 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faee406b250 0x7faee4136840 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:48.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.533+0000 7faeea2cb700 1 -- 172.21.15.16:0/1084361983 >> 172.21.15.16:0/1084361983 conn(0x7faee4066f50 msgr2=0x7faee409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:48.537 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.533+0000 7faeea2cb700 1 -- 172.21.15.16:0/1084361983 shutdown_connections 2023-09-30T22:35:48.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.533+0000 7faeea2cb700 1 -- 172.21.15.16:0/1084361983 wait complete. 2023-09-30T22:35:48.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.534+0000 7faeea2cb700 1 Processor -- start 2023-09-30T22:35:48.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.534+0000 7faeea2cb700 1 -- start start 2023-09-30T22:35:48.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.534+0000 7faeea2cb700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faee406a770 0x7faee409ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:48.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.534+0000 7faeea2cb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faee406b250 0x7faee409b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:48.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.534+0000 7faeea2cb700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faee4136d80 0x7faee41c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:48.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.534+0000 7faeea2cb700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee409a8e0 con 0x7faee406b250 2023-09-30T22:35:48.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.534+0000 7faeea2cb700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faee4132180 con 0x7faee406a770 2023-09-30T22:35:48.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.534+0000 7faeea2cb700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faee4122040 con 0x7faee4136d80 2023-09-30T22:35:48.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.534+0000 7faee37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faee406b250 0x7faee409b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:48.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.534+0000 7faee8868700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faee4136d80 0x7faee41c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:48.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.534+0000 7faee37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faee406b250 0x7faee409b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38532/0 (socket says 172.21.15.16:38532) 2023-09-30T22:35:48.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.534+0000 7faee8868700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faee4136d80 0x7faee41c81a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:34930/0 (socket says 172.21.15.16:34930) 2023-09-30T22:35:48.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.534+0000 7faee37fe700 1 -- 172.21.15.16:0/1211382430 learned_addr learned my addr 172.21.15.16:0/1211382430 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:48.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.534+0000 7faee3fff700 1 --2- 172.21.15.16:0/1211382430 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faee406a770 0x7faee409ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:48.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.535+0000 7faee3fff700 1 -- 172.21.15.16:0/1211382430 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faee4136d80 msgr2=0x7faee41c81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:48.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.535+0000 7faee3fff700 1 --2- 172.21.15.16:0/1211382430 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faee4136d80 0x7faee41c81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:48.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.535+0000 7faee3fff700 1 -- 172.21.15.16:0/1211382430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faee406b250 msgr2=0x7faee409b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:48.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.535+0000 7faee3fff700 1 --2- 172.21.15.16:0/1211382430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faee406b250 0x7faee409b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:48.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.535+0000 7faee3fff700 1 -- 172.21.15.16:0/1211382430 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faedc009a70 con 0x7faee406a770 2023-09-30T22:35:48.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.535+0000 7faee8868700 1 --2- 172.21.15.16:0/1211382430 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faee4136d80 0x7faee41c81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:48.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.535+0000 7faee37fe700 1 --2- 172.21.15.16:0/1211382430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faee406b250 0x7faee409b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:48.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.535+0000 7faee3fff700 1 --2- 172.21.15.16:0/1211382430 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faee406a770 0x7faee409ad20 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7faed400c460 tx=0x7faed400c770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:48.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.535+0000 7faee0ff9700 1 -- 172.21.15.16:0/1211382430 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed400cea0 con 0x7faee406a770 2023-09-30T22:35:48.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.535+0000 7faee0ff9700 1 -- 172.21.15.16:0/1211382430 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faed4005d70 con 0x7faee406a770 2023-09-30T22:35:48.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.535+0000 7faeea2cb700 1 -- 172.21.15.16:0/1211382430 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faed80094a0 con 0x7faee406a770 2023-09-30T22:35:48.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.535+0000 7faee0ff9700 1 -- 172.21.15.16:0/1211382430 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faed40072f0 con 0x7faee406a770 2023-09-30T22:35:48.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.535+0000 7faeea2cb700 1 -- 172.21.15.16:0/1211382430 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faee41323a0 con 0x7faee406a770 2023-09-30T22:35:48.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.536+0000 7faee0ff9700 1 -- 172.21.15.16:0/1211382430 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7faed4007450 con 0x7faee406a770 2023-09-30T22:35:48.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.536+0000 7faee0ff9700 1 -- 172.21.15.16:0/1211382430 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7faed403e8d0 con 0x7faee406a770 2023-09-30T22:35:48.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.536+0000 7faeea2cb700 1 -- 172.21.15.16:0/1211382430 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faee409c070 con 0x7faee406a770 2023-09-30T22:35:48.541 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.541+0000 7faee0ff9700 1 -- 172.21.15.16:0/1211382430 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7faed4019b40 con 0x7faee406a770 2023-09-30T22:35:48.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.749+0000 7faeea2cb700 1 -- 172.21.15.16:0/1211382430 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7faee409f6b0 con 0x7faee406a770 2023-09-30T22:35:48.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.749+0000 7faee0ff9700 1 -- 172.21.15.16:0/1211382430 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7faed4017030 con 0x7faee406a770 2023-09-30T22:35:48.750 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:35:48.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.752+0000 7faeea2cb700 1 -- 172.21.15.16:0/1211382430 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faee406a770 msgr2=0x7faee409ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:48.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.752+0000 7faeea2cb700 1 --2- 172.21.15.16:0/1211382430 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faee406a770 0x7faee409ad20 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7faed400c460 tx=0x7faed400c770 comp rx=0 tx=0).stop 2023-09-30T22:35:48.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.752+0000 7faeea2cb700 1 -- 172.21.15.16:0/1211382430 shutdown_connections 2023-09-30T22:35:48.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.752+0000 7faeea2cb700 1 --2- 172.21.15.16:0/1211382430 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faee406a770 0x7faee409ad20 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:48.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.752+0000 7faeea2cb700 1 --2- 172.21.15.16:0/1211382430 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faee4136d80 0x7faee41c81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:48.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.752+0000 7faeea2cb700 1 --2- 172.21.15.16:0/1211382430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faee406b250 0x7faee409b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:48.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.752+0000 7faeea2cb700 1 -- 172.21.15.16:0/1211382430 >> 172.21.15.16:0/1211382430 conn(0x7faee4066f50 msgr2=0x7faee40a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:48.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.752+0000 7faeea2cb700 1 -- 172.21.15.16:0/1211382430 shutdown_connections 2023-09-30T22:35:48.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:48.752+0000 7faeea2cb700 1 -- 172.21.15.16:0/1211382430 wait complete. 2023-09-30T22:35:48.766 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:35:49.132 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:35:49.132 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:35:49.192 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:35:49.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.279+0000 7f8b572fa700 1 Processor -- start 2023-09-30T22:35:49.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.279+0000 7ff109d97700 1 Processor -- start 2023-09-30T22:35:49.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.279+0000 7f8b572fa700 1 -- start start 2023-09-30T22:35:49.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.279+0000 7ff109d97700 1 -- start start 2023-09-30T22:35:49.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.280+0000 7ff109d97700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff10409c650 0x7ff10409ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:49.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.280+0000 7ff109d97700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff1040a0090 0x7ff10409cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:49.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.280+0000 7f8b572fa700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8b5009e930 0x7f8b5009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:49.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.280+0000 7f8b572fa700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8b5009f240 0x7f8b501317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:49.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.280+0000 7f8b572fa700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8b5009f650 0x7f8b50131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:49.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.280+0000 7f8b572fa700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b50070940 con 0x7f8b5009f650 2023-09-30T22:35:49.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.280+0000 7f8b572fa700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5006fd60 con 0x7f8b5009e930 2023-09-30T22:35:49.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.280+0000 7f8b572fa700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5006fa80 con 0x7f8b5009f240 2023-09-30T22:35:49.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.280+0000 7f8b54895700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8b5009f240 0x7f8b501317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:49.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.280+0000 7ff109d97700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff10409ba70 con 0x7ff104131970 2023-09-30T22:35:49.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.280+0000 7ff109d97700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff10409d4a0 con 0x7ff1040a0090 2023-09-30T22:35:49.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.280+0000 7ff109d97700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff10409e8f0 con 0x7ff10409c650 2023-09-30T22:35:49.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.280+0000 7f8b54895700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8b5009f240 0x7f8b501317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:34944/0 (socket says 172.21.15.16:34944) 2023-09-30T22:35:49.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.280+0000 7f8b54895700 1 -- 172.21.15.16:0/386007081 learned_addr learned my addr 172.21.15.16:0/386007081 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:49.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.280+0000 7ff1037fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff10409c650 0x7ff10409ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:49.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.280+0000 7ff1037fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff10409c650 0x7ff10409ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:34938/0 (socket says 172.21.15.16:34938) 2023-09-30T22:35:49.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.280+0000 7ff103fff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff104131970 0x7ff104131dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56958/0 (socket says 172.21.15.16:56958) 2023-09-30T22:35:49.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.280+0000 7ff1037fe700 1 -- 172.21.15.16:0/1933878655 learned_addr learned my addr 172.21.15.16:0/1933878655 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:49.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.280+0000 7f8b55897700 1 --2- 172.21.15.16:0/386007081 >> v2:172.21.15.16:3300/0 conn(0x7f8b5009f650 0x7f8b50131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:49.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.281+0000 7ff102ffd700 1 --2- 172.21.15.16:0/1933878655 >> v2:172.21.15.16:3300/0 conn(0x7ff1040a0090 0x7ff10409cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:49.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.281+0000 7ff1037fe700 1 -- 172.21.15.16:0/1933878655 >> v2:172.21.15.16:3300/0 conn(0x7ff1040a0090 msgr2=0x7ff10409cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:49.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.281+0000 7ff1037fe700 1 --2- 172.21.15.16:0/1933878655 >> v2:172.21.15.16:3300/0 conn(0x7ff1040a0090 0x7ff10409cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:49.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.281+0000 7ff1037fe700 1 -- 172.21.15.16:0/1933878655 >> v1:172.21.15.16:6789/0 conn(0x7ff104131970 legacy=0x7ff104131dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:49.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.281+0000 7ff1037fe700 1 -- 172.21.15.16:0/1933878655 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1040a0760 con 0x7ff10409c650 2023-09-30T22:35:49.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.281+0000 7ff1037fe700 1 --2- 172.21.15.16:0/1933878655 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff10409c650 0x7ff10409ca20 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7ff0f40098f0 tx=0x7ff0f4009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ff9e6863f34efd6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:49.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.281+0000 7f8b55897700 1 -- 172.21.15.16:0/386007081 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8b5009f240 msgr2=0x7f8b501317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:49.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.281+0000 7f8b55897700 1 --2- 172.21.15.16:0/386007081 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8b5009f240 0x7f8b501317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:49.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.281+0000 7f8b55897700 1 -- 172.21.15.16:0/386007081 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8b5009e930 msgr2=0x7f8b5009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:49.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.281+0000 7f8b55897700 1 --2- 172.21.15.16:0/386007081 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8b5009e930 0x7f8b5009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:49.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.281+0000 7f8b55897700 1 -- 172.21.15.16:0/386007081 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b5009fb60 con 0x7f8b5009f650 2023-09-30T22:35:49.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.281+0000 7f8b55897700 1 --2- 172.21.15.16:0/386007081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b5009f650 0x7f8b50131d30 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f8b48009d90 tx=0x7f8b4800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74d6209bd897012f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:49.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.281+0000 7ff1027fc700 1 -- 172.21.15.16:0/1933878655 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f400b8a0 con 0x7ff10409c650 2023-09-30T22:35:49.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.281+0000 7ff1027fc700 1 -- 172.21.15.16:0/1933878655 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0f400ba00 con 0x7ff10409c650 2023-09-30T22:35:49.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.281+0000 7ff1027fc700 1 -- 172.21.15.16:0/1933878655 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0f400bd90 con 0x7ff10409c650 2023-09-30T22:35:49.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.281+0000 7f8b47fff700 1 -- 172.21.15.16:0/386007081 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4800e070 con 0x7f8b5009f650 2023-09-30T22:35:49.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.281+0000 7f8b47fff700 1 -- 172.21.15.16:0/386007081 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b4800bb20 con 0x7f8b5009f650 2023-09-30T22:35:49.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.281+0000 7f8b47fff700 1 -- 172.21.15.16:0/386007081 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b48012430 con 0x7f8b5009f650 2023-09-30T22:35:49.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.282+0000 7ff109d97700 1 -- 172.21.15.16:0/1933878655 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff10409c650 msgr2=0x7ff10409ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:49.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.282+0000 7ff109d97700 1 --2- 172.21.15.16:0/1933878655 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff10409c650 0x7ff10409ca20 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7ff0f40098f0 tx=0x7ff0f4009c00 comp rx=0 tx=0).stop 2023-09-30T22:35:49.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.282+0000 7f8b572fa700 1 -- 172.21.15.16:0/386007081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b5009f650 msgr2=0x7f8b50131d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:49.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.282+0000 7f8b572fa700 1 --2- 172.21.15.16:0/386007081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b5009f650 0x7f8b50131d30 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f8b48009d90 tx=0x7f8b4800b330 comp rx=0 tx=0).stop 2023-09-30T22:35:49.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.282+0000 7f8b572fa700 1 -- 172.21.15.16:0/386007081 shutdown_connections 2023-09-30T22:35:49.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.282+0000 7f8b572fa700 1 --2- 172.21.15.16:0/386007081 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8b5009e930 0x7f8b5009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:49.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.282+0000 7f8b572fa700 1 --2- 172.21.15.16:0/386007081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b5009f650 0x7f8b50131d30 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:49.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.282+0000 7f8b572fa700 1 --2- 172.21.15.16:0/386007081 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8b5009f240 0x7f8b501317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:49.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.282+0000 7f8b572fa700 1 -- 172.21.15.16:0/386007081 >> 172.21.15.16:0/386007081 conn(0x7f8b500a4210 msgr2=0x7f8b500a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:49.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.282+0000 7ff109d97700 1 -- 172.21.15.16:0/1933878655 shutdown_connections 2023-09-30T22:35:49.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.282+0000 7ff109d97700 1 --2- 172.21.15.16:0/1933878655 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff10409c650 0x7ff10409ca20 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:49.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.282+0000 7ff109d97700 1 --2- 172.21.15.16:0/1933878655 >> v2:172.21.15.16:3300/0 conn(0x7ff1040a0090 0x7ff10409cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:49.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.282+0000 7ff109d97700 1 -- 172.21.15.16:0/1933878655 >> 172.21.15.16:0/1933878655 conn(0x7ff1040a3f70 msgr2=0x7ff1040a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:49.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.282+0000 7ff109d97700 1 -- 172.21.15.16:0/1933878655 shutdown_connections 2023-09-30T22:35:49.291 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.282+0000 7ff109d97700 1 -- 172.21.15.16:0/1933878655 wait complete. 2023-09-30T22:35:49.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.282+0000 7f8b572fa700 1 -- 172.21.15.16:0/386007081 shutdown_connections 2023-09-30T22:35:49.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.282+0000 7f8b572fa700 1 -- 172.21.15.16:0/386007081 wait complete. 2023-09-30T22:35:49.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.283+0000 7ff109d97700 1 Processor -- start 2023-09-30T22:35:49.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.283+0000 7f8b572fa700 1 Processor -- start 2023-09-30T22:35:49.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.284+0000 7f8b572fa700 1 -- start start 2023-09-30T22:35:49.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.284+0000 7ff109d97700 1 -- start start 2023-09-30T22:35:49.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.284+0000 7f8b572fa700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b5009e930 0x7f8b5006b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:49.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.284+0000 7f8b572fa700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8b5009f240 0x7f8b5006bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:49.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.284+0000 7f8b572fa700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8b5009f650 0x7f8b5006c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:49.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.284+0000 7f8b572fa700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b501b73c0 con 0x7f8b5009e930 2023-09-30T22:35:49.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.284+0000 7f8b572fa700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b501a9570 con 0x7f8b5009f240 2023-09-30T22:35:49.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.284+0000 7f8b572fa700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5006cd10 con 0x7f8b5009f650 2023-09-30T22:35:49.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.284+0000 7ff109d97700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff10409c650 0x7ff10411f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:49.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.284+0000 7ff109d97700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1040a0090 0x7ff10411fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:49.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.284+0000 7ff109d97700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff104131970 0x7ff1041202f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:49.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.284+0000 7ff109d97700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff104121480 con 0x7ff1040a0090 2023-09-30T22:35:49.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.284+0000 7ff109d97700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff10409d980 con 0x7ff10409c650 2023-09-30T22:35:49.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.284+0000 7ff109d97700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff104122df0 con 0x7ff104131970 2023-09-30T22:35:49.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.284+0000 7f8b55096700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b5009e930 0x7f8b5006b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:49.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.284+0000 7f8b55897700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8b5009f650 0x7f8b5006c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:49.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.284+0000 7f8b55096700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b5009e930 0x7f8b5006b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38558/0 (socket says 172.21.15.16:38558) 2023-09-30T22:35:49.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.284+0000 7f8b55096700 1 -- 172.21.15.16:0/2261222902 learned_addr learned my addr 172.21.15.16:0/2261222902 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:49.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.284+0000 7ff103fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff104131970 0x7ff1041202f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:49.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.284+0000 7ff103fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff104131970 0x7ff1041202f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:34952/0 (socket says 172.21.15.16:34952) 2023-09-30T22:35:49.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.284+0000 7ff103fff700 1 -- 172.21.15.16:0/1425372038 learned_addr learned my addr 172.21.15.16:0/1425372038 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:49.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.285+0000 7f8b55897700 1 -- 172.21.15.16:0/2261222902 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8b5009f240 msgr2=0x7f8b5006bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:49.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.285+0000 7f8b55897700 1 --2- 172.21.15.16:0/2261222902 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8b5009f240 0x7f8b5006bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:49.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.285+0000 7f8b55897700 1 -- 172.21.15.16:0/2261222902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b5009e930 msgr2=0x7f8b5006b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:49.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.285+0000 7f8b55897700 1 --2- 172.21.15.16:0/2261222902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b5009e930 0x7f8b5006b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:49.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.285+0000 7f8b55897700 1 -- 172.21.15.16:0/2261222902 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b48009a70 con 0x7f8b5009f650 2023-09-30T22:35:49.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.285+0000 7f8b55096700 1 --2- 172.21.15.16:0/2261222902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b5009e930 0x7f8b5006b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:49.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.285+0000 7ff103fff700 1 -- 172.21.15.16:0/1425372038 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff10409c650 msgr2=0x7ff10411f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:49.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.285+0000 7ff103fff700 1 --2- 172.21.15.16:0/1425372038 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff10409c650 0x7ff10411f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:49.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.285+0000 7ff103fff700 1 -- 172.21.15.16:0/1425372038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1040a0090 msgr2=0x7ff10411fdb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:35:49.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.285+0000 7ff103fff700 1 --2- 172.21.15.16:0/1425372038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1040a0090 0x7ff10411fdb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:49.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.285+0000 7ff103fff700 1 -- 172.21.15.16:0/1425372038 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0f4009550 con 0x7ff104131970 2023-09-30T22:35:49.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.285+0000 7f8b55897700 1 --2- 172.21.15.16:0/2261222902 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8b5009f650 0x7f8b5006c330 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f8b48016000 tx=0x7f8b48004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:49.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.285+0000 7ff103fff700 1 --2- 172.21.15.16:0/1425372038 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff104131970 0x7ff1041202f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc00cc00 tx=0x7ff0fc00a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:49.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.285+0000 7f8b45ffb700 1 -- 172.21.15.16:0/2261222902 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b480048b0 con 0x7f8b5009f650 2023-09-30T22:35:49.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.285+0000 7f8b45ffb700 1 -- 172.21.15.16:0/2261222902 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b48020070 con 0x7f8b5009f650 2023-09-30T22:35:49.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.285+0000 7f8b572fa700 1 -- 172.21.15.16:0/2261222902 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b500a0190 con 0x7f8b5009f650 2023-09-30T22:35:49.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.285+0000 7ff0ebfff700 1 -- 172.21.15.16:0/1425372038 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc00aba0 con 0x7ff104131970 2023-09-30T22:35:49.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.285+0000 7ff0ebfff700 1 -- 172.21.15.16:0/1425372038 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0fc00ad00 con 0x7ff104131970 2023-09-30T22:35:49.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.285+0000 7ff109d97700 1 -- 172.21.15.16:0/1425372038 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1041233f0 con 0x7ff104131970 2023-09-30T22:35:49.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.285+0000 7ff0ebfff700 1 -- 172.21.15.16:0/1425372038 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc007270 con 0x7ff104131970 2023-09-30T22:35:49.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.285+0000 7ff109d97700 1 -- 172.21.15.16:0/1425372038 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff104121c50 con 0x7ff104131970 2023-09-30T22:35:49.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.285+0000 7f8b45ffb700 1 -- 172.21.15.16:0/2261222902 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b480276c0 con 0x7f8b5009f650 2023-09-30T22:35:49.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.286+0000 7f8b572fa700 1 -- 172.21.15.16:0/2261222902 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b50071d20 con 0x7f8b5009f650 2023-09-30T22:35:49.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.287+0000 7ff0ebfff700 1 -- 172.21.15.16:0/1425372038 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff0fc0073d0 con 0x7ff104131970 2023-09-30T22:35:49.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.287+0000 7ff109d97700 1 -- 172.21.15.16:0/1425372038 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1040747f0 con 0x7ff104131970 2023-09-30T22:35:49.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.287+0000 7ff0ebfff700 1 -- 172.21.15.16:0/1425372038 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7ff0fc03f540 con 0x7ff104131970 2023-09-30T22:35:49.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.287+0000 7f8b45ffb700 1 -- 172.21.15.16:0/2261222902 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8b48012430 con 0x7f8b5009f650 2023-09-30T22:35:49.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.288+0000 7f8b45ffb700 1 -- 172.21.15.16:0/2261222902 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f8b48022070 con 0x7f8b5009f650 2023-09-30T22:35:49.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.288+0000 7f8b572fa700 1 -- 172.21.15.16:0/2261222902 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b501a0b30 con 0x7f8b5009f650 2023-09-30T22:35:49.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.292+0000 7ff0ebfff700 1 -- 172.21.15.16:0/1425372038 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff0fc017070 con 0x7ff104131970 2023-09-30T22:35:49.303 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.297+0000 7f8b45ffb700 1 -- 172.21.15.16:0/2261222902 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8b4801c400 con 0x7f8b5009f650 2023-09-30T22:35:49.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.337+0000 7eff06857700 1 Processor -- start 2023-09-30T22:35:49.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.337+0000 7eff06857700 1 -- start start 2023-09-30T22:35:49.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.338+0000 7eff06857700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7eff000a0a30 0x7eff000a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:49.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.338+0000 7eff06857700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7eff000a1340 0x7eff00133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:49.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.338+0000 7eff06857700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff0009edd0 con 0x7eff00133fa0 2023-09-30T22:35:49.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.338+0000 7eff06857700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff0009e1f0 con 0x7eff000a0a30 2023-09-30T22:35:49.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.338+0000 7eff06857700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7eff0009dc50 con 0x7eff000a1340 2023-09-30T22:35:49.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.338+0000 7eff04df4700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7eff00133fa0 0x7eff0013ad20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56966/0 (socket says 172.21.15.16:56966) 2023-09-30T22:35:49.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.338+0000 7efefffff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7eff000a0a30 0x7eff000a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:49.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.338+0000 7eff04df4700 1 -- 172.21.15.16:0/3221668866 learned_addr learned my addr 172.21.15.16:0/3221668866 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:49.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.338+0000 7efeff7fe700 1 --2- 172.21.15.16:0/3221668866 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7eff000a1340 0x7eff00133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:49.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.338+0000 7efefffff700 1 -- 172.21.15.16:0/3221668866 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7eff000a1340 msgr2=0x7eff00133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:49.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.338+0000 7efefffff700 1 --2- 172.21.15.16:0/3221668866 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7eff000a1340 0x7eff00133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:49.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.338+0000 7efefffff700 1 -- 172.21.15.16:0/3221668866 >> v1:172.21.15.16:6789/0 conn(0x7eff00133fa0 legacy=0x7eff0013ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:49.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.338+0000 7efefeffd700 1 -- 172.21.15.16:0/3221668866 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1308376497 0 0) 0x7eff0009edd0 con 0x7eff00133fa0 2023-09-30T22:35:49.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.338+0000 7efefffff700 1 -- 172.21.15.16:0/3221668866 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff000a1fe0 con 0x7eff000a0a30 2023-09-30T22:35:49.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.338+0000 7efeff7fe700 1 --2- 172.21.15.16:0/3221668866 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7eff000a1340 0x7eff00133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:49.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.338+0000 7efefffff700 1 --2- 172.21.15.16:0/3221668866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff000a0a30 0x7eff000a0e00 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7efef0009840 tx=0x7efef0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baf1cfc6473b6fd4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:49.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.339+0000 7efefeffd700 1 -- 172.21.15.16:0/3221668866 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef0011830 con 0x7eff000a0a30 2023-09-30T22:35:49.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.339+0000 7efefeffd700 1 -- 172.21.15.16:0/3221668866 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efef0011990 con 0x7eff000a0a30 2023-09-30T22:35:49.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.339+0000 7efefeffd700 1 -- 172.21.15.16:0/3221668866 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef0011d20 con 0x7eff000a0a30 2023-09-30T22:35:49.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.339+0000 7eff06857700 1 -- 172.21.15.16:0/3221668866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff000a0a30 msgr2=0x7eff000a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:49.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.339+0000 7eff06857700 1 --2- 172.21.15.16:0/3221668866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff000a0a30 0x7eff000a0e00 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7efef0009840 tx=0x7efef0009b50 comp rx=0 tx=0).stop 2023-09-30T22:35:49.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.339+0000 7eff06857700 1 -- 172.21.15.16:0/3221668866 shutdown_connections 2023-09-30T22:35:49.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.339+0000 7eff06857700 1 --2- 172.21.15.16:0/3221668866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff000a0a30 0x7eff000a0e00 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:49.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.339+0000 7eff06857700 1 --2- 172.21.15.16:0/3221668866 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7eff000a1340 0x7eff00133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:49.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.339+0000 7eff06857700 1 -- 172.21.15.16:0/3221668866 >> 172.21.15.16:0/3221668866 conn(0x7eff000a6550 msgr2=0x7eff000a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:49.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.340+0000 7eff06857700 1 -- 172.21.15.16:0/3221668866 shutdown_connections 2023-09-30T22:35:49.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.340+0000 7eff06857700 1 -- 172.21.15.16:0/3221668866 wait complete. 2023-09-30T22:35:49.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.341+0000 7eff06857700 1 Processor -- start 2023-09-30T22:35:49.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.341+0000 7eff06857700 1 -- start start 2023-09-30T22:35:49.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.341+0000 7eff06857700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7eff000a0a30 0x7eff00122790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:49.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.341+0000 7eff06857700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff000a1340 0x7eff00122cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:49.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.341+0000 7eff06857700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7eff00133fa0 0x7eff00123210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:49.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.341+0000 7eff06857700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff00124520 con 0x7eff000a1340 2023-09-30T22:35:49.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.341+0000 7eff06857700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff00133320 con 0x7eff000a0a30 2023-09-30T22:35:49.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.341+0000 7eff06857700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7eff001262b0 con 0x7eff00133fa0 2023-09-30T22:35:49.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.341+0000 7efeff7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff000a1340 0x7eff00122cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:49.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.341+0000 7efeff7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff000a1340 0x7eff00122cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38580/0 (socket says 172.21.15.16:38580) 2023-09-30T22:35:49.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.342+0000 7efeff7fe700 1 -- 172.21.15.16:0/3027849248 learned_addr learned my addr 172.21.15.16:0/3027849248 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:49.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.342+0000 7efefffff700 1 --2- 172.21.15.16:0/3027849248 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7eff000a0a30 0x7eff00122790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:49.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.342+0000 7eff04df4700 1 --2- 172.21.15.16:0/3027849248 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7eff00133fa0 0x7eff00123210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:49.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.342+0000 7efeff7fe700 1 -- 172.21.15.16:0/3027849248 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7eff00133fa0 msgr2=0x7eff00123210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:49.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.342+0000 7efeff7fe700 1 --2- 172.21.15.16:0/3027849248 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7eff00133fa0 0x7eff00123210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:49.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.342+0000 7efeff7fe700 1 -- 172.21.15.16:0/3027849248 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7eff000a0a30 msgr2=0x7eff00122790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:49.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.342+0000 7efeff7fe700 1 --2- 172.21.15.16:0/3027849248 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7eff000a0a30 0x7eff00122790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:49.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.342+0000 7efeff7fe700 1 -- 172.21.15.16:0/3027849248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef40094a0 con 0x7eff000a1340 2023-09-30T22:35:49.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.342+0000 7efefffff700 1 --2- 172.21.15.16:0/3027849248 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7eff000a0a30 0x7eff00122790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:49.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.342+0000 7efeff7fe700 1 --2- 172.21.15.16:0/3027849248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff000a1340 0x7eff00122cd0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7efef400ec10 tx=0x7efef400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:49.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.342+0000 7efefcff9700 1 -- 172.21.15.16:0/3027849248 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef400c850 con 0x7eff000a1340 2023-09-30T22:35:49.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.342+0000 7efefcff9700 1 -- 172.21.15.16:0/3027849248 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efef400c9b0 con 0x7eff000a1340 2023-09-30T22:35:49.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.342+0000 7efefcff9700 1 -- 172.21.15.16:0/3027849248 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef400cd20 con 0x7eff000a1340 2023-09-30T22:35:49.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.342+0000 7eff06857700 1 -- 172.21.15.16:0/3027849248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efef00094a0 con 0x7eff000a1340 2023-09-30T22:35:49.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.343+0000 7eff06857700 1 -- 172.21.15.16:0/3027849248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff000a0220 con 0x7eff000a1340 2023-09-30T22:35:49.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.343+0000 7efefcff9700 1 -- 172.21.15.16:0/3027849248 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7efef4013470 con 0x7eff000a1340 2023-09-30T22:35:49.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.343+0000 7eff06857700 1 -- 172.21.15.16:0/3027849248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff001336e0 con 0x7eff000a1340 2023-09-30T22:35:49.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.344+0000 7efefcff9700 1 -- 172.21.15.16:0/3027849248 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7efef403fbd0 con 0x7eff000a1340 2023-09-30T22:35:49.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.349+0000 7efefcff9700 1 -- 172.21.15.16:0/3027849248 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7efef4013720 con 0x7eff000a1340 2023-09-30T22:35:49.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.500+0000 7ff109d97700 1 -- 172.21.15.16:0/1425372038 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff104072840 con 0x7ff104131970 2023-09-30T22:35:49.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.500+0000 7ff0ebfff700 1 -- 172.21.15.16:0/1425372038 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff0fc0108f0 con 0x7ff104131970 2023-09-30T22:35:49.501 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:35:49.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.504+0000 7ff109d97700 1 -- 172.21.15.16:0/1425372038 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff104131970 msgr2=0x7ff1041202f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:49.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.504+0000 7ff109d97700 1 --2- 172.21.15.16:0/1425372038 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff104131970 0x7ff1041202f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc00cc00 tx=0x7ff0fc00a060 comp rx=0 tx=0).stop 2023-09-30T22:35:49.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.504+0000 7ff109d97700 1 -- 172.21.15.16:0/1425372038 shutdown_connections 2023-09-30T22:35:49.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.504+0000 7ff109d97700 1 --2- 172.21.15.16:0/1425372038 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff10409c650 0x7ff10411f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:49.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.504+0000 7ff109d97700 1 --2- 172.21.15.16:0/1425372038 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff104131970 0x7ff1041202f0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:49.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.504+0000 7ff109d97700 1 --2- 172.21.15.16:0/1425372038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff1040a0090 0x7ff10411fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:49.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.504+0000 7ff109d97700 1 -- 172.21.15.16:0/1425372038 >> 172.21.15.16:0/1425372038 conn(0x7ff1040a3f70 msgr2=0x7ff10409eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:49.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.504+0000 7ff109d97700 1 -- 172.21.15.16:0/1425372038 shutdown_connections 2023-09-30T22:35:49.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.504+0000 7ff109d97700 1 -- 172.21.15.16:0/1425372038 wait complete. 2023-09-30T22:35:49.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.507+0000 7f8b572fa700 1 -- 172.21.15.16:0/2261222902 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f8b50074ea0 con 0x7f8b5009f650 2023-09-30T22:35:49.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.507+0000 7f8b45ffb700 1 -- 172.21.15.16:0/2261222902 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8b48025380 con 0x7f8b5009f650 2023-09-30T22:35:49.507 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:35:49.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.509+0000 7f8b572fa700 1 -- 172.21.15.16:0/2261222902 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8b5009f650 msgr2=0x7f8b5006c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:49.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.509+0000 7f8b572fa700 1 --2- 172.21.15.16:0/2261222902 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8b5009f650 0x7f8b5006c330 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f8b48016000 tx=0x7f8b48004ad0 comp rx=0 tx=0).stop 2023-09-30T22:35:49.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.509+0000 7f8b572fa700 1 -- 172.21.15.16:0/2261222902 shutdown_connections 2023-09-30T22:35:49.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.509+0000 7f8b572fa700 1 --2- 172.21.15.16:0/2261222902 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8b5009f650 0x7f8b5006c330 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:49.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.509+0000 7f8b572fa700 1 --2- 172.21.15.16:0/2261222902 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8b5009e930 0x7f8b5006b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:49.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.509+0000 7f8b572fa700 1 --2- 172.21.15.16:0/2261222902 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8b5009f240 0x7f8b5006bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:49.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.509+0000 7f8b572fa700 1 -- 172.21.15.16:0/2261222902 >> 172.21.15.16:0/2261222902 conn(0x7f8b500a4210 msgr2=0x7f8b501b8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:49.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.509+0000 7f8b572fa700 1 -- 172.21.15.16:0/2261222902 shutdown_connections 2023-09-30T22:35:49.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.509+0000 7f8b572fa700 1 -- 172.21.15.16:0/2261222902 wait complete. 2023-09-30T22:35:49.521 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:35:49.524 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:35:49.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.556+0000 7eff06857700 1 -- 172.21.15.16:0/3027849248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7eff00074ea0 con 0x7eff000a1340 2023-09-30T22:35:49.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.556+0000 7efefcff9700 1 -- 172.21.15.16:0/3027849248 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7efef4011b00 con 0x7eff000a1340 2023-09-30T22:35:49.556 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:35:49.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.558+0000 7eff06857700 1 -- 172.21.15.16:0/3027849248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff000a1340 msgr2=0x7eff00122cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:49.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.558+0000 7eff06857700 1 --2- 172.21.15.16:0/3027849248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff000a1340 0x7eff00122cd0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7efef400ec10 tx=0x7efef400ef20 comp rx=0 tx=0).stop 2023-09-30T22:35:49.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.558+0000 7eff06857700 1 -- 172.21.15.16:0/3027849248 shutdown_connections 2023-09-30T22:35:49.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.558+0000 7eff06857700 1 --2- 172.21.15.16:0/3027849248 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7eff000a0a30 0x7eff00122790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:49.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.558+0000 7eff06857700 1 --2- 172.21.15.16:0/3027849248 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7eff00133fa0 0x7eff00123210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:49.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.558+0000 7eff06857700 1 --2- 172.21.15.16:0/3027849248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7eff000a1340 0x7eff00122cd0 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:49.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.558+0000 7eff06857700 1 -- 172.21.15.16:0/3027849248 >> 172.21.15.16:0/3027849248 conn(0x7eff000a6550 msgr2=0x7eff000a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:49.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.558+0000 7eff06857700 1 -- 172.21.15.16:0/3027849248 shutdown_connections 2023-09-30T22:35:49.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.558+0000 7eff06857700 1 -- 172.21.15.16:0/3027849248 wait complete. 2023-09-30T22:35:49.572 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:35:49.766 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:35:49.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.911+0000 7f2c74641700 1 Processor -- start 2023-09-30T22:35:49.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.911+0000 7f2c74641700 1 -- start start 2023-09-30T22:35:49.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.912+0000 7f2c74641700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c6c09e930 0x7f2c6c09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:49.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.912+0000 7f2c74641700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c6c09f650 0x7f2c6c131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:49.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.912+0000 7f2c74641700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c6c070940 con 0x7f2c6c09f240 2023-09-30T22:35:49.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.912+0000 7f2c74641700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6c06fd60 con 0x7f2c6c09f650 2023-09-30T22:35:49.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.912+0000 7f2c74641700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6c06fa80 con 0x7f2c6c09e930 2023-09-30T22:35:49.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.912+0000 7f2c723dd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c6c09e930 0x7f2c6c09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:49.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.912+0000 7f2c72bde700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c6c09f650 0x7f2c6c131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:49.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.912+0000 7f2c71bdc700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2c6c09f240 0x7f2c6c1317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56974/0 (socket says 172.21.15.16:56974) 2023-09-30T22:35:49.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.912+0000 7f2c71bdc700 1 -- 172.21.15.16:0/1467391939 learned_addr learned my addr 172.21.15.16:0/1467391939 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:49.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.913+0000 7f2c723dd700 1 -- 172.21.15.16:0/1467391939 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c6c09f650 msgr2=0x7f2c6c131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:49.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.913+0000 7f2c723dd700 1 --2- 172.21.15.16:0/1467391939 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c6c09f650 0x7f2c6c131f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:49.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.913+0000 7f2c723dd700 1 -- 172.21.15.16:0/1467391939 >> v1:172.21.15.16:6789/0 conn(0x7f2c6c09f240 legacy=0x7f2c6c1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:49.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.913+0000 7f2c723dd700 1 -- 172.21.15.16:0/1467391939 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c6c09fb60 con 0x7f2c6c09e930 2023-09-30T22:35:49.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.913+0000 7f2c713db700 1 -- 172.21.15.16:0/1467391939 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 861208993 0 0) 0x7f2c6c070940 con 0x7f2c6c09f240 2023-09-30T22:35:49.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.913+0000 7f2c72bde700 1 --2- 172.21.15.16:0/1467391939 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c6c09f650 0x7f2c6c131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:49.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.913+0000 7f2c723dd700 1 --2- 172.21.15.16:0/1467391939 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c6c09e930 0x7f2c6c09ed00 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f2c68009870 tx=0x7f2c68009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3256568d3b402456 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:49.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.913+0000 7f2c713db700 1 -- 172.21.15.16:0/1467391939 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c68011830 con 0x7f2c6c09e930 2023-09-30T22:35:49.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.913+0000 7f2c713db700 1 -- 172.21.15.16:0/1467391939 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c68011990 con 0x7f2c6c09e930 2023-09-30T22:35:49.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.913+0000 7f2c713db700 1 -- 172.21.15.16:0/1467391939 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c68011d20 con 0x7f2c6c09e930 2023-09-30T22:35:49.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.914+0000 7f2c74641700 1 -- 172.21.15.16:0/1467391939 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c6c09e930 msgr2=0x7f2c6c09ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:49.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.914+0000 7f2c74641700 1 --2- 172.21.15.16:0/1467391939 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c6c09e930 0x7f2c6c09ed00 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f2c68009870 tx=0x7f2c68009b80 comp rx=0 tx=0).stop 2023-09-30T22:35:49.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.914+0000 7f2c74641700 1 -- 172.21.15.16:0/1467391939 shutdown_connections 2023-09-30T22:35:49.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.914+0000 7f2c74641700 1 --2- 172.21.15.16:0/1467391939 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c6c09f650 0x7f2c6c131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:49.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.914+0000 7f2c74641700 1 --2- 172.21.15.16:0/1467391939 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c6c09e930 0x7f2c6c09ed00 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:49.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.914+0000 7f2c74641700 1 -- 172.21.15.16:0/1467391939 >> 172.21.15.16:0/1467391939 conn(0x7f2c6c0a4210 msgr2=0x7f2c6c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:49.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.914+0000 7f2c74641700 1 -- 172.21.15.16:0/1467391939 shutdown_connections 2023-09-30T22:35:49.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.914+0000 7f2c74641700 1 -- 172.21.15.16:0/1467391939 wait complete. 2023-09-30T22:35:49.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.915+0000 7f2c74641700 1 Processor -- start 2023-09-30T22:35:49.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.915+0000 7f2c74641700 1 -- start start 2023-09-30T22:35:49.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.915+0000 7f2c74641700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c6c09e930 0x7f2c6c1c63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:49.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.915+0000 7f2c74641700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c6c09f240 0x7f2c6c1c6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:49.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.915+0000 7f2c74641700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c6c09f650 0x7f2c6c1c2840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:49.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.915+0000 7f2c74641700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6c1a7320 con 0x7f2c6c09f240 2023-09-30T22:35:49.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.915+0000 7f2c74641700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6c1a6540 con 0x7f2c6c09e930 2023-09-30T22:35:49.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.915+0000 7f2c74641700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6c1a6920 con 0x7f2c6c09f650 2023-09-30T22:35:49.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.915+0000 7f2c723dd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c6c09e930 0x7f2c6c1c63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:49.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.915+0000 7f2c72bde700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c6c09f650 0x7f2c6c1c2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:49.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.915+0000 7f2c723dd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c6c09e930 0x7f2c6c1c63d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47294/0 (socket says 172.21.15.16:47294) 2023-09-30T22:35:49.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.915+0000 7f2c723dd700 1 -- 172.21.15.16:0/3313204355 learned_addr learned my addr 172.21.15.16:0/3313204355 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:49.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.915+0000 7f2c71bdc700 1 --2- 172.21.15.16:0/3313204355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c6c09f240 0x7f2c6c1c6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:49.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.916+0000 7f2c72bde700 1 -- 172.21.15.16:0/3313204355 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c6c09e930 msgr2=0x7f2c6c1c63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:49.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.916+0000 7f2c72bde700 1 --2- 172.21.15.16:0/3313204355 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c6c09e930 0x7f2c6c1c63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:49.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.916+0000 7f2c72bde700 1 -- 172.21.15.16:0/3313204355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c6c09f240 msgr2=0x7f2c6c1c6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:49.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.916+0000 7f2c72bde700 1 --2- 172.21.15.16:0/3313204355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c6c09f240 0x7f2c6c1c6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:49.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.916+0000 7f2c72bde700 1 -- 172.21.15.16:0/3313204355 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c64009a70 con 0x7f2c6c09f650 2023-09-30T22:35:49.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.916+0000 7f2c723dd700 1 --2- 172.21.15.16:0/3313204355 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c6c09e930 0x7f2c6c1c63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:49.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.916+0000 7f2c72bde700 1 --2- 172.21.15.16:0/3313204355 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c6c09f650 0x7f2c6c1c2840 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f2c6400efc0 tx=0x7f2c6400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:49.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.916+0000 7f2c5effd700 1 -- 172.21.15.16:0/3313204355 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6400cc90 con 0x7f2c6c09f650 2023-09-30T22:35:49.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.916+0000 7f2c5effd700 1 -- 172.21.15.16:0/3313204355 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c6400f040 con 0x7f2c6c09f650 2023-09-30T22:35:49.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.916+0000 7f2c74641700 1 -- 172.21.15.16:0/3313204355 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c680094a0 con 0x7f2c6c09f650 2023-09-30T22:35:49.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.916+0000 7f2c5effd700 1 -- 172.21.15.16:0/3313204355 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c64014480 con 0x7f2c6c09f650 2023-09-30T22:35:49.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.916+0000 7f2c74641700 1 -- 172.21.15.16:0/3313204355 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c6c0a0190 con 0x7f2c6c09f650 2023-09-30T22:35:49.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.917+0000 7f2c5effd700 1 -- 172.21.15.16:0/3313204355 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2c64014600 con 0x7f2c6c09f650 2023-09-30T22:35:49.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.917+0000 7f2c5effd700 1 -- 172.21.15.16:0/3313204355 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f2c6403ffa0 con 0x7f2c6c09f650 2023-09-30T22:35:49.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.917+0000 7f2c74641700 1 -- 172.21.15.16:0/3313204355 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c6c074ea0 con 0x7f2c6c09f650 2023-09-30T22:35:49.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:49.922+0000 7f2c5effd700 1 -- 172.21.15.16:0/3313204355 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2c64018080 con 0x7f2c6c09f650 2023-09-30T22:35:50.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.125+0000 7f2c74641700 1 -- 172.21.15.16:0/3313204355 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2c6c06b250 con 0x7f2c6c09f650 2023-09-30T22:35:50.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.125+0000 7f2c5effd700 1 -- 172.21.15.16:0/3313204355 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2c640114a0 con 0x7f2c6c09f650 2023-09-30T22:35:50.126 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:35:50.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.127+0000 7f2c74641700 1 -- 172.21.15.16:0/3313204355 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c6c09f650 msgr2=0x7f2c6c1c2840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:50.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.127+0000 7f2c74641700 1 --2- 172.21.15.16:0/3313204355 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c6c09f650 0x7f2c6c1c2840 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f2c6400efc0 tx=0x7f2c6400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:35:50.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.127+0000 7f2c74641700 1 -- 172.21.15.16:0/3313204355 shutdown_connections 2023-09-30T22:35:50.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.127+0000 7f2c74641700 1 --2- 172.21.15.16:0/3313204355 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2c6c09e930 0x7f2c6c1c63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:50.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.127+0000 7f2c74641700 1 --2- 172.21.15.16:0/3313204355 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2c6c09f650 0x7f2c6c1c2840 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:50.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.127+0000 7f2c74641700 1 --2- 172.21.15.16:0/3313204355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c6c09f240 0x7f2c6c1c6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:50.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.127+0000 7f2c74641700 1 -- 172.21.15.16:0/3313204355 >> 172.21.15.16:0/3313204355 conn(0x7f2c6c0a4210 msgr2=0x7f2c6c1b8a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:50.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.127+0000 7f2c74641700 1 -- 172.21.15.16:0/3313204355 shutdown_connections 2023-09-30T22:35:50.129 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.128+0000 7f2c74641700 1 -- 172.21.15.16:0/3313204355 wait complete. 2023-09-30T22:35:50.141 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:35:50.522 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:35:50.525 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:35:50.573 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:35:50.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.670+0000 7f7adc5e5700 1 Processor -- start 2023-09-30T22:35:50.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.671+0000 7f7adc5e5700 1 -- start start 2023-09-30T22:35:50.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.671+0000 7f7adc5e5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7ad409d6f0 0x7f7ad409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:50.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.671+0000 7f7adc5e5700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7ad409e000 0x7f7ad41339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:50.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.671+0000 7f7adc5e5700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ad409b120 con 0x7f7ad409e410 2023-09-30T22:35:50.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.671+0000 7f7adc5e5700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ad406b460 con 0x7f7ad409e000 2023-09-30T22:35:50.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.671+0000 7f7adc5e5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad409a7e0 con 0x7f7ad409d6f0 2023-09-30T22:35:50.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.671+0000 7f7ada381700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7ad409d6f0 0x7f7ad409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:50.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.672+0000 7f7ad9b80700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7ad409e000 0x7f7ad41339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:50.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.672+0000 7f7adab82700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7ad409e410 0x7f7ad4133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56982/0 (socket says 172.21.15.16:56982) 2023-09-30T22:35:50.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.672+0000 7f7ada381700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7ad409d6f0 0x7f7ad409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35002/0 (socket says 172.21.15.16:35002) 2023-09-30T22:35:50.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.672+0000 7f7adab82700 1 -- 172.21.15.16:0/688619592 learned_addr learned my addr 172.21.15.16:0/688619592 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:50.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.672+0000 7f7ad9b80700 1 -- 172.21.15.16:0/688619592 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7ad409d6f0 msgr2=0x7f7ad409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:50.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.672+0000 7f7ad9b80700 1 --2- 172.21.15.16:0/688619592 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7ad409d6f0 0x7f7ad409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:50.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.672+0000 7f7ad9b80700 1 -- 172.21.15.16:0/688619592 >> v1:172.21.15.16:6789/0 conn(0x7f7ad409e410 legacy=0x7f7ad4133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:50.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.672+0000 7f7ad9b80700 1 -- 172.21.15.16:0/688619592 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ad409a500 con 0x7f7ad409e000 2023-09-30T22:35:50.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.672+0000 7f7ada381700 1 --2- 172.21.15.16:0/688619592 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7ad409d6f0 0x7f7ad409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:50.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.672+0000 7f7ad937f700 1 -- 172.21.15.16:0/688619592 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 780503547 0 0) 0x7f7ad409b120 con 0x7f7ad409e410 2023-09-30T22:35:50.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.672+0000 7f7ad9b80700 1 --2- 172.21.15.16:0/688619592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ad409e000 0x7f7ad41339f0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f7ac4009840 tx=0x7f7ac4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5450f559cd2ec86 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:50.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.672+0000 7f7ad937f700 1 -- 172.21.15.16:0/688619592 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac4011830 con 0x7f7ad409e000 2023-09-30T22:35:50.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.672+0000 7f7ad937f700 1 -- 172.21.15.16:0/688619592 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ac4011990 con 0x7f7ad409e000 2023-09-30T22:35:50.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.672+0000 7f7ad937f700 1 -- 172.21.15.16:0/688619592 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac4011d20 con 0x7f7ad409e000 2023-09-30T22:35:50.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.673+0000 7f7adc5e5700 1 -- 172.21.15.16:0/688619592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ad409e000 msgr2=0x7f7ad41339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:50.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.673+0000 7f7adc5e5700 1 --2- 172.21.15.16:0/688619592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ad409e000 0x7f7ad41339f0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f7ac4009840 tx=0x7f7ac4009b50 comp rx=0 tx=0).stop 2023-09-30T22:35:50.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.673+0000 7f7adc5e5700 1 -- 172.21.15.16:0/688619592 shutdown_connections 2023-09-30T22:35:50.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.673+0000 7f7adc5e5700 1 --2- 172.21.15.16:0/688619592 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7ad409d6f0 0x7f7ad409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:50.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.673+0000 7f7adc5e5700 1 --2- 172.21.15.16:0/688619592 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ad409e000 0x7f7ad41339f0 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:50.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.673+0000 7f7adc5e5700 1 -- 172.21.15.16:0/688619592 >> 172.21.15.16:0/688619592 conn(0x7f7ad4066f50 msgr2=0x7f7ad409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:50.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.673+0000 7f7adc5e5700 1 -- 172.21.15.16:0/688619592 shutdown_connections 2023-09-30T22:35:50.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.673+0000 7f7adc5e5700 1 -- 172.21.15.16:0/688619592 wait complete. 2023-09-30T22:35:50.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.674+0000 7f7adc5e5700 1 Processor -- start 2023-09-30T22:35:50.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.674+0000 7f7adc5e5700 1 -- start start 2023-09-30T22:35:50.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.675+0000 7f7adc5e5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7ad409d6f0 0x7f7ad41a5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:50.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.675+0000 7f7adc5e5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ad409e000 0x7f7ad41a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:50.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.675+0000 7f7adc5e5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7ad409e410 0x7f7ad41a6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:50.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.675+0000 7f7adc5e5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad41a9150 con 0x7f7ad409e000 2023-09-30T22:35:50.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.675+0000 7f7adc5e5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad409c710 con 0x7f7ad409e410 2023-09-30T22:35:50.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.675+0000 7f7adc5e5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad40a2e40 con 0x7f7ad409d6f0 2023-09-30T22:35:50.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.675+0000 7f7ad9b80700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ad409e000 0x7f7ad41a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:50.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.675+0000 7f7ad9b80700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ad409e000 0x7f7ad41a63e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38610/0 (socket says 172.21.15.16:38610) 2023-09-30T22:35:50.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.675+0000 7f7ad9b80700 1 -- 172.21.15.16:0/1943356138 learned_addr learned my addr 172.21.15.16:0/1943356138 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:50.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.675+0000 7f7ada381700 1 --2- 172.21.15.16:0/1943356138 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7ad409d6f0 0x7f7ad41a5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:50.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.675+0000 7f7adab82700 1 --2- 172.21.15.16:0/1943356138 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7ad409e410 0x7f7ad41a6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:50.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.675+0000 7f7ada381700 1 -- 172.21.15.16:0/1943356138 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7ad409e410 msgr2=0x7f7ad41a6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:50.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.675+0000 7f7ada381700 1 --2- 172.21.15.16:0/1943356138 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7ad409e410 0x7f7ad41a6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:50.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.675+0000 7f7ada381700 1 -- 172.21.15.16:0/1943356138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ad409e000 msgr2=0x7f7ad41a63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:50.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.675+0000 7f7ada381700 1 --2- 172.21.15.16:0/1943356138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ad409e000 0x7f7ad41a63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:50.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.675+0000 7f7ada381700 1 -- 172.21.15.16:0/1943356138 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ad00094a0 con 0x7f7ad409d6f0 2023-09-30T22:35:50.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.675+0000 7f7ad9b80700 1 --2- 172.21.15.16:0/1943356138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ad409e000 0x7f7ad41a63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:50.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.676+0000 7f7ada381700 1 --2- 172.21.15.16:0/1943356138 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7ad409d6f0 0x7f7ad41a5ea0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f7ad000e9b0 tx=0x7f7ad000ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:50.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.676+0000 7f7acaffd700 1 -- 172.21.15.16:0/1943356138 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad000ee50 con 0x7f7ad409d6f0 2023-09-30T22:35:50.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.676+0000 7f7acaffd700 1 -- 172.21.15.16:0/1943356138 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ad000c850 con 0x7f7ad409d6f0 2023-09-30T22:35:50.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.676+0000 7f7adc5e5700 1 -- 172.21.15.16:0/1943356138 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ac40094a0 con 0x7f7ad409d6f0 2023-09-30T22:35:50.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.676+0000 7f7acaffd700 1 -- 172.21.15.16:0/1943356138 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad000cc00 con 0x7f7ad409d6f0 2023-09-30T22:35:50.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.676+0000 7f7adc5e5700 1 -- 172.21.15.16:0/1943356138 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ad41a8d00 con 0x7f7ad409d6f0 2023-09-30T22:35:50.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.677+0000 7f7adc5e5700 1 -- 172.21.15.16:0/1943356138 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ad4074ea0 con 0x7f7ad409d6f0 2023-09-30T22:35:50.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.677+0000 7f7acaffd700 1 -- 172.21.15.16:0/1943356138 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7ad0013470 con 0x7f7ad409d6f0 2023-09-30T22:35:50.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.678+0000 7f7acaffd700 1 -- 172.21.15.16:0/1943356138 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f7ad003fee0 con 0x7f7ad409d6f0 2023-09-30T22:35:50.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.682+0000 7f7acaffd700 1 -- 172.21.15.16:0/1943356138 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7ad0013720 con 0x7f7ad409d6f0 2023-09-30T22:35:50.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.710+0000 7f552b4c7700 1 Processor -- start 2023-09-30T22:35:50.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.710+0000 7f552b4c7700 1 -- start start 2023-09-30T22:35:50.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.711+0000 7f552b4c7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55240a0a30 0x7f55240a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:50.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.711+0000 7f552b4c7700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f55240a1340 0x7f5524133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:50.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.711+0000 7f552b4c7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5524133fa0 0x7f5524134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:50.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.711+0000 7f552b4c7700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f552409edd0 con 0x7f55240a1340 2023-09-30T22:35:50.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.711+0000 7f552b4c7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f552409e1f0 con 0x7f55240a0a30 2023-09-30T22:35:50.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.711+0000 7f552b4c7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f552409dc50 con 0x7f5524133fa0 2023-09-30T22:35:50.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.711+0000 7f5529263700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55240a0a30 0x7f55240a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:50.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.711+0000 7f5529263700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55240a0a30 0x7f55240a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47320/0 (socket says 172.21.15.16:47320) 2023-09-30T22:35:50.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.711+0000 7f5529263700 1 -- 172.21.15.16:0/1119416611 learned_addr learned my addr 172.21.15.16:0/1119416611 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:50.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.711+0000 7f5529a64700 1 --2- 172.21.15.16:0/1119416611 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5524133fa0 0x7f5524134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:50.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.711+0000 7f5528a62700 1 --2- 172.21.15.16:0/1119416611 >> v2:172.21.15.16:3300/0 conn(0x7f55240a1340 0x7f5524133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:50.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.711+0000 7f5529263700 1 -- 172.21.15.16:0/1119416611 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5524133fa0 msgr2=0x7f5524134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:50.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.711+0000 7f5529263700 1 --2- 172.21.15.16:0/1119416611 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5524133fa0 0x7f5524134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:50.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.711+0000 7f5529263700 1 -- 172.21.15.16:0/1119416611 >> v2:172.21.15.16:3300/0 conn(0x7f55240a1340 msgr2=0x7f5524133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:50.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.711+0000 7f5529263700 1 --2- 172.21.15.16:0/1119416611 >> v2:172.21.15.16:3300/0 conn(0x7f55240a1340 0x7f5524133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:50.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.711+0000 7f5529263700 1 -- 172.21.15.16:0/1119416611 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55240a1fe0 con 0x7f55240a0a30 2023-09-30T22:35:50.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.712+0000 7f5529263700 1 --2- 172.21.15.16:0/1119416611 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55240a0a30 0x7f55240a0e00 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f5520009e80 tx=0x7f552000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ff6778c946bbdedd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:50.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.712+0000 7f551bfff700 1 -- 172.21.15.16:0/1119416611 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f552000e070 con 0x7f55240a0a30 2023-09-30T22:35:50.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.712+0000 7f551bfff700 1 -- 172.21.15.16:0/1119416611 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f552000b8d0 con 0x7f55240a0a30 2023-09-30T22:35:50.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.712+0000 7f551bfff700 1 -- 172.21.15.16:0/1119416611 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f552000bc80 con 0x7f55240a0a30 2023-09-30T22:35:50.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.712+0000 7f552b4c7700 1 -- 172.21.15.16:0/1119416611 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55240a0a30 msgr2=0x7f55240a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:50.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.712+0000 7f552b4c7700 1 --2- 172.21.15.16:0/1119416611 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55240a0a30 0x7f55240a0e00 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f5520009e80 tx=0x7f552000b330 comp rx=0 tx=0).stop 2023-09-30T22:35:50.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.713+0000 7f552b4c7700 1 -- 172.21.15.16:0/1119416611 shutdown_connections 2023-09-30T22:35:50.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.713+0000 7f552b4c7700 1 --2- 172.21.15.16:0/1119416611 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55240a0a30 0x7f55240a0e00 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:50.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.713+0000 7f552b4c7700 1 --2- 172.21.15.16:0/1119416611 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5524133fa0 0x7f5524134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:50.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.713+0000 7f552b4c7700 1 --2- 172.21.15.16:0/1119416611 >> v2:172.21.15.16:3300/0 conn(0x7f55240a1340 0x7f5524133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:50.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.713+0000 7f552b4c7700 1 -- 172.21.15.16:0/1119416611 >> 172.21.15.16:0/1119416611 conn(0x7f55240a6550 msgr2=0x7f55240a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:50.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.713+0000 7f552b4c7700 1 -- 172.21.15.16:0/1119416611 shutdown_connections 2023-09-30T22:35:50.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.713+0000 7f552b4c7700 1 -- 172.21.15.16:0/1119416611 wait complete. 2023-09-30T22:35:50.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.714+0000 7f552b4c7700 1 Processor -- start 2023-09-30T22:35:50.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.714+0000 7f552b4c7700 1 -- start start 2023-09-30T22:35:50.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.714+0000 7f552b4c7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f55240a0a30 0x7f5524120060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:50.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.714+0000 7f552b4c7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55240a1340 0x7f55241205a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:50.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.714+0000 7f552b4c7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5524133fa0 0x7f5524120ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:50.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.714+0000 7f552b4c7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5524121d10 con 0x7f5524133fa0 2023-09-30T22:35:50.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.714+0000 7f552b4c7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55241238a0 con 0x7f55240a1340 2023-09-30T22:35:50.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.714+0000 7f552b4c7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5524122ec0 con 0x7f55240a0a30 2023-09-30T22:35:50.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.714+0000 7f5528a62700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55240a1340 0x7f55241205a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:50.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.714+0000 7f5528a62700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55240a1340 0x7f55241205a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47334/0 (socket says 172.21.15.16:47334) 2023-09-30T22:35:50.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.714+0000 7f5528a62700 1 -- 172.21.15.16:0/3106662950 learned_addr learned my addr 172.21.15.16:0/3106662950 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:50.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.714+0000 7f5529263700 1 --2- 172.21.15.16:0/3106662950 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f55240a0a30 0x7f5524120060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:50.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.714+0000 7f5529a64700 1 --2- 172.21.15.16:0/3106662950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5524133fa0 0x7f5524120ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:50.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.715+0000 7f5528a62700 1 -- 172.21.15.16:0/3106662950 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f55240a0a30 msgr2=0x7f5524120060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:50.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.715+0000 7f5528a62700 1 --2- 172.21.15.16:0/3106662950 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f55240a0a30 0x7f5524120060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:50.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.715+0000 7f5528a62700 1 -- 172.21.15.16:0/3106662950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5524133fa0 msgr2=0x7f5524120ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:50.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.715+0000 7f5528a62700 1 --2- 172.21.15.16:0/3106662950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5524133fa0 0x7f5524120ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:50.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.715+0000 7f5528a62700 1 -- 172.21.15.16:0/3106662950 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5520009b30 con 0x7f55240a1340 2023-09-30T22:35:50.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.715+0000 7f5528a62700 1 --2- 172.21.15.16:0/3106662950 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55240a1340 0x7f55241205a0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f551400e9b0 tx=0x7f551400ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:50.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.715+0000 7f5519ffb700 1 -- 172.21.15.16:0/3106662950 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5514009710 con 0x7f55240a1340 2023-09-30T22:35:50.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.715+0000 7f5519ffb700 1 -- 172.21.15.16:0/3106662950 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f551400ee50 con 0x7f55240a1340 2023-09-30T22:35:50.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.715+0000 7f5519ffb700 1 -- 172.21.15.16:0/3106662950 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551400ca50 con 0x7f55240a1340 2023-09-30T22:35:50.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.715+0000 7f552b4c7700 1 -- 172.21.15.16:0/3106662950 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55240a3530 con 0x7f55240a1340 2023-09-30T22:35:50.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.716+0000 7f552b4c7700 1 -- 172.21.15.16:0/3106662950 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5524123b80 con 0x7f55240a1340 2023-09-30T22:35:50.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.717+0000 7f5519ffb700 1 -- 172.21.15.16:0/3106662950 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f551400cbb0 con 0x7f55240a1340 2023-09-30T22:35:50.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.717+0000 7f552b4c7700 1 -- 172.21.15.16:0/3106662950 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5524123540 con 0x7f55240a1340 2023-09-30T22:35:50.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.717+0000 7f5519ffb700 1 -- 172.21.15.16:0/3106662950 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f551403fbd0 con 0x7f55240a1340 2023-09-30T22:35:50.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.722+0000 7f5519ffb700 1 -- 172.21.15.16:0/3106662950 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f551401b3b0 con 0x7f55240a1340 2023-09-30T22:35:50.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.754+0000 7f6838a19700 1 Processor -- start 2023-09-30T22:35:50.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.754+0000 7f6838a19700 1 -- start start 2023-09-30T22:35:50.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.755+0000 7f6838a19700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f68340a0a30 0x7f68340a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:50.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.755+0000 7f6838a19700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6834138d80 0x7f683413b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:50.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.755+0000 7f6838a19700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68340a2a70 con 0x7f683409d630 2023-09-30T22:35:50.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.755+0000 7f6838a19700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68340a01d0 con 0x7f6834138d80 2023-09-30T22:35:50.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.755+0000 7f6838a19700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f683409e2b0 con 0x7f68340a0a30 2023-09-30T22:35:50.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.755+0000 7f683259c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f68340a0a30 0x7f68340a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:50.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.755+0000 7f6832d9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6834138d80 0x7f683413b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:50.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.755+0000 7f6831d9b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f683409d630 0x7f68340a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:56990/0 (socket says 172.21.15.16:56990) 2023-09-30T22:35:50.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.755+0000 7f6831d9b700 1 -- 172.21.15.16:0/4032502582 learned_addr learned my addr 172.21.15.16:0/4032502582 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:50.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.756+0000 7f683159a700 1 -- 172.21.15.16:0/4032502582 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 851063021 0 0) 0x7f68340a2a70 con 0x7f683409d630 2023-09-30T22:35:50.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.756+0000 7f683159a700 1 -- 172.21.15.16:0/4032502582 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f681c0035b0 con 0x7f683409d630 2023-09-30T22:35:50.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.756+0000 7f683259c700 1 -- 172.21.15.16:0/4032502582 >> v2:172.21.15.16:3300/0 conn(0x7f6834138d80 msgr2=0x7f683413b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:50.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.756+0000 7f683259c700 1 --2- 172.21.15.16:0/4032502582 >> v2:172.21.15.16:3300/0 conn(0x7f6834138d80 0x7f683413b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:50.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.756+0000 7f683259c700 1 -- 172.21.15.16:0/4032502582 >> v1:172.21.15.16:6789/0 conn(0x7f683409d630 legacy=0x7f68340a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:50.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.756+0000 7f683259c700 1 -- 172.21.15.16:0/4032502582 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f683409dc50 con 0x7f68340a0a30 2023-09-30T22:35:50.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.756+0000 7f6832d9d700 1 --2- 172.21.15.16:0/4032502582 >> v2:172.21.15.16:3300/0 conn(0x7f6834138d80 0x7f683413b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:50.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.756+0000 7f683259c700 1 --2- 172.21.15.16:0/4032502582 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f68340a0a30 0x7f68340a0e00 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f6824009840 tx=0x7f6824009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d5ade753c1b30b3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:50.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.756+0000 7f683159a700 1 -- 172.21.15.16:0/4032502582 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6824011830 con 0x7f68340a0a30 2023-09-30T22:35:50.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.756+0000 7f683159a700 1 -- 172.21.15.16:0/4032502582 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6824011990 con 0x7f68340a0a30 2023-09-30T22:35:50.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.756+0000 7f683159a700 1 -- 172.21.15.16:0/4032502582 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6824011d20 con 0x7f68340a0a30 2023-09-30T22:35:50.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.757+0000 7f6838a19700 1 -- 172.21.15.16:0/4032502582 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f68340a0a30 msgr2=0x7f68340a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:50.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.757+0000 7f6838a19700 1 --2- 172.21.15.16:0/4032502582 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f68340a0a30 0x7f68340a0e00 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f6824009840 tx=0x7f6824009b50 comp rx=0 tx=0).stop 2023-09-30T22:35:50.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.757+0000 7f6838a19700 1 -- 172.21.15.16:0/4032502582 shutdown_connections 2023-09-30T22:35:50.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.757+0000 7f6838a19700 1 --2- 172.21.15.16:0/4032502582 >> v2:172.21.15.16:3300/0 conn(0x7f6834138d80 0x7f683413b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:50.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.757+0000 7f6838a19700 1 --2- 172.21.15.16:0/4032502582 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f68340a0a30 0x7f68340a0e00 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:50.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.757+0000 7f6838a19700 1 -- 172.21.15.16:0/4032502582 >> 172.21.15.16:0/4032502582 conn(0x7f68340a3050 msgr2=0x7f68340a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:50.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.757+0000 7f6838a19700 1 -- 172.21.15.16:0/4032502582 shutdown_connections 2023-09-30T22:35:50.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.757+0000 7f6838a19700 1 -- 172.21.15.16:0/4032502582 wait complete. 2023-09-30T22:35:50.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.758+0000 7f6838a19700 1 Processor -- start 2023-09-30T22:35:50.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.758+0000 7f6838a19700 1 -- start start 2023-09-30T22:35:50.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.759+0000 7f6838a19700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f683409d630 0x7f68341338b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:50.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.759+0000 7f6838a19700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f68340a0a30 0x7f6834133df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:50.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.759+0000 7f6838a19700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6834138d80 0x7f6834134330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:50.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.759+0000 7f6838a19700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6834136b20 con 0x7f6834138d80 2023-09-30T22:35:50.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.759+0000 7f6838a19700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6834134ca0 con 0x7f683409d630 2023-09-30T22:35:50.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.759+0000 7f6838a19700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f68340a0410 con 0x7f68340a0a30 2023-09-30T22:35:50.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.759+0000 7f6831d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f68340a0a30 0x7f6834133df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:50.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.759+0000 7f6831d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f68340a0a30 0x7f6834133df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35028/0 (socket says 172.21.15.16:35028) 2023-09-30T22:35:50.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.759+0000 7f6831d9b700 1 -- 172.21.15.16:0/3558701520 learned_addr learned my addr 172.21.15.16:0/3558701520 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:50.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.759+0000 7f6832d9d700 1 --2- 172.21.15.16:0/3558701520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6834138d80 0x7f6834134330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:50.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.759+0000 7f6831d9b700 1 -- 172.21.15.16:0/3558701520 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f683409d630 msgr2=0x7f68341338b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:50.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.759+0000 7f6831d9b700 1 --2- 172.21.15.16:0/3558701520 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f683409d630 0x7f68341338b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:50.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.759+0000 7f6831d9b700 1 -- 172.21.15.16:0/3558701520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6834138d80 msgr2=0x7f6834134330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:50.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.759+0000 7f6831d9b700 1 --2- 172.21.15.16:0/3558701520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6834138d80 0x7f6834134330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:50.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.759+0000 7f6831d9b700 1 -- 172.21.15.16:0/3558701520 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f682c009a70 con 0x7f68340a0a30 2023-09-30T22:35:50.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.759+0000 7f6831d9b700 1 --2- 172.21.15.16:0/3558701520 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f68340a0a30 0x7f6834133df0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f6828005910 tx=0x7f682800c9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:50.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.759+0000 7f6822ffd700 1 -- 172.21.15.16:0/3558701520 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6828005d70 con 0x7f68340a0a30 2023-09-30T22:35:50.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.760+0000 7f6822ffd700 1 -- 172.21.15.16:0/3558701520 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68280070d0 con 0x7f68340a0a30 2023-09-30T22:35:50.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.760+0000 7f6838a19700 1 -- 172.21.15.16:0/3558701520 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68240094a0 con 0x7f68340a0a30 2023-09-30T22:35:50.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.760+0000 7f6822ffd700 1 -- 172.21.15.16:0/3558701520 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6828007440 con 0x7f68340a0a30 2023-09-30T22:35:50.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.760+0000 7f6838a19700 1 -- 172.21.15.16:0/3558701520 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6834137f70 con 0x7f68340a0a30 2023-09-30T22:35:50.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.761+0000 7f6838a19700 1 -- 172.21.15.16:0/3558701520 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68340747f0 con 0x7f68340a0a30 2023-09-30T22:35:50.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.761+0000 7f6822ffd700 1 -- 172.21.15.16:0/3558701520 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f68280075a0 con 0x7f68340a0a30 2023-09-30T22:35:50.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.761+0000 7f6822ffd700 1 -- 172.21.15.16:0/3558701520 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f6828010300 con 0x7f68340a0a30 2023-09-30T22:35:50.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.766+0000 7f6822ffd700 1 -- 172.21.15.16:0/3558701520 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6828044060 con 0x7f68340a0a30 2023-09-30T22:35:50.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.895+0000 7f7adc5e5700 1 -- 172.21.15.16:0/1943356138 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f7ad41a9540 con 0x7f7ad409d6f0 2023-09-30T22:35:50.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.896+0000 7f7acaffd700 1 -- 172.21.15.16:0/1943356138 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7ad0021070 con 0x7f7ad409d6f0 2023-09-30T22:35:50.896 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:35:50.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.899+0000 7f7adc5e5700 1 -- 172.21.15.16:0/1943356138 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7ad409d6f0 msgr2=0x7f7ad41a5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:50.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.899+0000 7f7adc5e5700 1 --2- 172.21.15.16:0/1943356138 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7ad409d6f0 0x7f7ad41a5ea0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f7ad000e9b0 tx=0x7f7ad000ecc0 comp rx=0 tx=0).stop 2023-09-30T22:35:50.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.899+0000 7f7adc5e5700 1 -- 172.21.15.16:0/1943356138 shutdown_connections 2023-09-30T22:35:50.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.899+0000 7f7adc5e5700 1 --2- 172.21.15.16:0/1943356138 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7ad409e410 0x7f7ad41a6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:50.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.899+0000 7f7adc5e5700 1 --2- 172.21.15.16:0/1943356138 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7ad409d6f0 0x7f7ad41a5ea0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:50.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.899+0000 7f7adc5e5700 1 --2- 172.21.15.16:0/1943356138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ad409e000 0x7f7ad41a63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:50.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.899+0000 7f7adc5e5700 1 -- 172.21.15.16:0/1943356138 >> 172.21.15.16:0/1943356138 conn(0x7f7ad4066f50 msgr2=0x7f7ad41bad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:50.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.899+0000 7f7adc5e5700 1 -- 172.21.15.16:0/1943356138 shutdown_connections 2023-09-30T22:35:50.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.899+0000 7f7adc5e5700 1 -- 172.21.15.16:0/1943356138 wait complete. 2023-09-30T22:35:50.918 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:35:50.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.946+0000 7f552b4c7700 1 -- 172.21.15.16:0/3106662950 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f5524074ea0 con 0x7f55240a1340 2023-09-30T22:35:50.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.947+0000 7f5519ffb700 1 -- 172.21.15.16:0/3106662950 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5514021070 con 0x7f55240a1340 2023-09-30T22:35:50.947 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:35:50.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.949+0000 7f552b4c7700 1 -- 172.21.15.16:0/3106662950 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55240a1340 msgr2=0x7f55241205a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:50.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.949+0000 7f552b4c7700 1 --2- 172.21.15.16:0/3106662950 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55240a1340 0x7f55241205a0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f551400e9b0 tx=0x7f551400ecc0 comp rx=0 tx=0).stop 2023-09-30T22:35:50.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.949+0000 7f552b4c7700 1 -- 172.21.15.16:0/3106662950 shutdown_connections 2023-09-30T22:35:50.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.949+0000 7f552b4c7700 1 --2- 172.21.15.16:0/3106662950 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5524133fa0 0x7f5524120ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:50.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.949+0000 7f552b4c7700 1 --2- 172.21.15.16:0/3106662950 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f55240a0a30 0x7f5524120060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:50.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.949+0000 7f552b4c7700 1 --2- 172.21.15.16:0/3106662950 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55240a1340 0x7f55241205a0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:50.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.949+0000 7f552b4c7700 1 -- 172.21.15.16:0/3106662950 >> 172.21.15.16:0/3106662950 conn(0x7f55240a6550 msgr2=0x7f55241baf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:50.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.949+0000 7f552b4c7700 1 -- 172.21.15.16:0/3106662950 shutdown_connections 2023-09-30T22:35:50.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.949+0000 7f552b4c7700 1 -- 172.21.15.16:0/3106662950 wait complete. 2023-09-30T22:35:50.963 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:35:50.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.972+0000 7f6838a19700 1 -- 172.21.15.16:0/3558701520 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f6834135c70 con 0x7f68340a0a30 2023-09-30T22:35:50.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.973+0000 7f6822ffd700 1 -- 172.21.15.16:0/3558701520 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6828017070 con 0x7f68340a0a30 2023-09-30T22:35:50.973 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:35:50.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.975+0000 7f6838a19700 1 -- 172.21.15.16:0/3558701520 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f68340a0a30 msgr2=0x7f6834133df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:50.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.975+0000 7f6838a19700 1 --2- 172.21.15.16:0/3558701520 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f68340a0a30 0x7f6834133df0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f6828005910 tx=0x7f682800c9e0 comp rx=0 tx=0).stop 2023-09-30T22:35:50.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.975+0000 7f6838a19700 1 -- 172.21.15.16:0/3558701520 shutdown_connections 2023-09-30T22:35:50.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.975+0000 7f6838a19700 1 --2- 172.21.15.16:0/3558701520 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f683409d630 0x7f68341338b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:50.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.975+0000 7f6838a19700 1 --2- 172.21.15.16:0/3558701520 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6834138d80 0x7f6834134330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:50.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.975+0000 7f6838a19700 1 --2- 172.21.15.16:0/3558701520 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f68340a0a30 0x7f6834133df0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:50.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.975+0000 7f6838a19700 1 -- 172.21.15.16:0/3558701520 >> 172.21.15.16:0/3558701520 conn(0x7f68340a3050 msgr2=0x7f683409ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:50.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.975+0000 7f6838a19700 1 -- 172.21.15.16:0/3558701520 shutdown_connections 2023-09-30T22:35:50.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:50.975+0000 7f6838a19700 1 -- 172.21.15.16:0/3558701520 wait complete. 2023-09-30T22:35:50.989 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:35:51.142 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:35:51.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.284+0000 7faacd89e700 1 Processor -- start 2023-09-30T22:35:51.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.284+0000 7faacd89e700 1 -- start start 2023-09-30T22:35:51.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.284+0000 7faacd89e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faac80a09c0 0x7faac80a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:51.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.285+0000 7faacd89e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faac809d5c0 0x7faac80a12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:51.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.285+0000 7faacd89e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faac8133920 0x7faac8133d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:51.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.285+0000 7faacd89e700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faac80a1810 con 0x7faac8133920 2023-09-30T22:35:51.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.285+0000 7faacd89e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faac809ea10 con 0x7faac809d5c0 2023-09-30T22:35:51.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.285+0000 7faacd89e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faac809e210 con 0x7faac80a09c0 2023-09-30T22:35:51.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.285+0000 7faac67fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faac809d5c0 0x7faac80a12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:51.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.285+0000 7faac77fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faac8133920 0x7faac8133d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:51.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.285+0000 7faac67fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faac809d5c0 0x7faac80a12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47360/0 (socket says 172.21.15.16:47360) 2023-09-30T22:35:51.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.285+0000 7faac77fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faac8133920 0x7faac8133d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38658/0 (socket says 172.21.15.16:38658) 2023-09-30T22:35:51.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.285+0000 7faac6ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faac80a09c0 0x7faac80a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:51.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.285+0000 7faac67fc700 1 -- 172.21.15.16:0/4277621691 learned_addr learned my addr 172.21.15.16:0/4277621691 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:51.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.285+0000 7faac77fe700 1 -- 172.21.15.16:0/4277621691 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faac80a09c0 msgr2=0x7faac80a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:51.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.285+0000 7faac77fe700 1 --2- 172.21.15.16:0/4277621691 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faac80a09c0 0x7faac80a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:51.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.285+0000 7faac77fe700 1 -- 172.21.15.16:0/4277621691 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faac809d5c0 msgr2=0x7faac80a12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:51.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.285+0000 7faac77fe700 1 --2- 172.21.15.16:0/4277621691 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faac809d5c0 0x7faac80a12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:51.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.285+0000 7faac77fe700 1 -- 172.21.15.16:0/4277621691 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faac80a26f0 con 0x7faac8133920 2023-09-30T22:35:51.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.285+0000 7faac67fc700 1 --2- 172.21.15.16:0/4277621691 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faac809d5c0 0x7faac80a12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:51.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.285+0000 7faac6ffd700 1 --2- 172.21.15.16:0/4277621691 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faac80a09c0 0x7faac80a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:51.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.285+0000 7faac77fe700 1 --2- 172.21.15.16:0/4277621691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faac8133920 0x7faac8133d90 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7faac0009d90 tx=0x7faac000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5606916e3a52f448 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:51.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.285+0000 7faac5ffb700 1 -- 172.21.15.16:0/4277621691 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac000e070 con 0x7faac8133920 2023-09-30T22:35:51.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.285+0000 7faac5ffb700 1 -- 172.21.15.16:0/4277621691 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faac000bb20 con 0x7faac8133920 2023-09-30T22:35:51.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.286+0000 7faac5ffb700 1 -- 172.21.15.16:0/4277621691 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac0012430 con 0x7faac8133920 2023-09-30T22:35:51.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.286+0000 7faacd89e700 1 -- 172.21.15.16:0/4277621691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faac8133920 msgr2=0x7faac8133d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:51.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.286+0000 7faacd89e700 1 --2- 172.21.15.16:0/4277621691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faac8133920 0x7faac8133d90 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7faac0009d90 tx=0x7faac000b330 comp rx=0 tx=0).stop 2023-09-30T22:35:51.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.286+0000 7faacd89e700 1 -- 172.21.15.16:0/4277621691 shutdown_connections 2023-09-30T22:35:51.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.286+0000 7faacd89e700 1 --2- 172.21.15.16:0/4277621691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faac8133920 0x7faac8133d90 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:51.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.286+0000 7faacd89e700 1 --2- 172.21.15.16:0/4277621691 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faac80a09c0 0x7faac80a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:51.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.286+0000 7faacd89e700 1 --2- 172.21.15.16:0/4277621691 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faac809d5c0 0x7faac80a12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:51.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.286+0000 7faacd89e700 1 -- 172.21.15.16:0/4277621691 >> 172.21.15.16:0/4277621691 conn(0x7faac80a2fe0 msgr2=0x7faac80a33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:51.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.286+0000 7faacd89e700 1 -- 172.21.15.16:0/4277621691 shutdown_connections 2023-09-30T22:35:51.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.286+0000 7faacd89e700 1 -- 172.21.15.16:0/4277621691 wait complete. 2023-09-30T22:35:51.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.287+0000 7faacd89e700 1 Processor -- start 2023-09-30T22:35:51.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.287+0000 7faacd89e700 1 -- start start 2023-09-30T22:35:51.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.288+0000 7faacd89e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faac809d5c0 0x7faac81c8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:51.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.288+0000 7faacd89e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faac80a09c0 0x7faac81c8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:51.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.288+0000 7faacd89e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faac8133920 0x7faac811e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:51.292 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.288+0000 7faacd89e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac81c8fa0 con 0x7faac8133920 2023-09-30T22:35:51.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.288+0000 7faacd89e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faac81203f0 con 0x7faac809d5c0 2023-09-30T22:35:51.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.288+0000 7faacd89e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faac80a5e60 con 0x7faac80a09c0 2023-09-30T22:35:51.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.288+0000 7faac77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faac8133920 0x7faac811e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:51.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.288+0000 7faac67fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faac80a09c0 0x7faac81c8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:51.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.288+0000 7faac6ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faac809d5c0 0x7faac81c8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:51.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.288+0000 7faac67fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faac80a09c0 0x7faac81c8a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35044/0 (socket says 172.21.15.16:35044) 2023-09-30T22:35:51.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.288+0000 7faac6ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faac809d5c0 0x7faac81c8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47368/0 (socket says 172.21.15.16:47368) 2023-09-30T22:35:51.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.288+0000 7faac77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faac8133920 0x7faac811e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38672/0 (socket says 172.21.15.16:38672) 2023-09-30T22:35:51.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.288+0000 7faac67fc700 1 -- 172.21.15.16:0/3795206216 learned_addr learned my addr 172.21.15.16:0/3795206216 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:51.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.288+0000 7faac67fc700 1 -- 172.21.15.16:0/3795206216 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faac809d5c0 msgr2=0x7faac81c8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:51.293 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.288+0000 7faac67fc700 1 --2- 172.21.15.16:0/3795206216 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faac809d5c0 0x7faac81c8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:51.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.288+0000 7faac67fc700 1 -- 172.21.15.16:0/3795206216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faac8133920 msgr2=0x7faac811e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:51.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.288+0000 7faac67fc700 1 --2- 172.21.15.16:0/3795206216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faac8133920 0x7faac811e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:51.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.288+0000 7faac67fc700 1 -- 172.21.15.16:0/3795206216 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faab0009550 con 0x7faac80a09c0 2023-09-30T22:35:51.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.288+0000 7faac77fe700 1 --2- 172.21.15.16:0/3795206216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faac8133920 0x7faac811e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:51.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.288+0000 7faac6ffd700 1 --2- 172.21.15.16:0/3795206216 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faac809d5c0 0x7faac81c8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:51.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.289+0000 7faac67fc700 1 --2- 172.21.15.16:0/3795206216 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faac80a09c0 0x7faac81c8a60 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7faab000e9c0 tx=0x7faab000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:51.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.289+0000 7faabffff700 1 -- 172.21.15.16:0/3795206216 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab000ee60 con 0x7faac80a09c0 2023-09-30T22:35:51.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.289+0000 7faabffff700 1 -- 172.21.15.16:0/3795206216 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faab000c850 con 0x7faac80a09c0 2023-09-30T22:35:51.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.289+0000 7faacd89e700 1 -- 172.21.15.16:0/3795206216 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faab80094a0 con 0x7faac80a09c0 2023-09-30T22:35:51.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.289+0000 7faabffff700 1 -- 172.21.15.16:0/3795206216 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab000cc00 con 0x7faac80a09c0 2023-09-30T22:35:51.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.289+0000 7faacd89e700 1 -- 172.21.15.16:0/3795206216 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faac0009a70 con 0x7faac80a09c0 2023-09-30T22:35:51.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.290+0000 7faabffff700 1 -- 172.21.15.16:0/3795206216 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7faab000cd60 con 0x7faac80a09c0 2023-09-30T22:35:51.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.290+0000 7faabffff700 1 -- 172.21.15.16:0/3795206216 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7faab003fa90 con 0x7faac80a09c0 2023-09-30T22:35:51.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.290+0000 7faacd89e700 1 -- 172.21.15.16:0/3795206216 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faac80747f0 con 0x7faac80a09c0 2023-09-30T22:35:51.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.295+0000 7faabffff700 1 -- 172.21.15.16:0/3795206216 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7faab0021020 con 0x7faac80a09c0 2023-09-30T22:35:51.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.501+0000 7faacd89e700 1 -- 172.21.15.16:0/3795206216 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7faac8072980 con 0x7faac80a09c0 2023-09-30T22:35:51.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.502+0000 7faabffff700 1 -- 172.21.15.16:0/3795206216 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7faab0017070 con 0x7faac80a09c0 2023-09-30T22:35:51.503 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:35:51.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.504+0000 7faacd89e700 1 -- 172.21.15.16:0/3795206216 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faac80a09c0 msgr2=0x7faac81c8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:51.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.504+0000 7faacd89e700 1 --2- 172.21.15.16:0/3795206216 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faac80a09c0 0x7faac81c8a60 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7faab000e9c0 tx=0x7faab000ecd0 comp rx=0 tx=0).stop 2023-09-30T22:35:51.505 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.504+0000 7faacd89e700 1 -- 172.21.15.16:0/3795206216 shutdown_connections 2023-09-30T22:35:51.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.504+0000 7faacd89e700 1 --2- 172.21.15.16:0/3795206216 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faac809d5c0 0x7faac81c8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:51.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.504+0000 7faacd89e700 1 --2- 172.21.15.16:0/3795206216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faac8133920 0x7faac811e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:51.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.504+0000 7faacd89e700 1 --2- 172.21.15.16:0/3795206216 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faac80a09c0 0x7faac81c8a60 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:51.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.504+0000 7faacd89e700 1 -- 172.21.15.16:0/3795206216 >> 172.21.15.16:0/3795206216 conn(0x7faac80a2fe0 msgr2=0x7faac809ecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:51.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.504+0000 7faacd89e700 1 -- 172.21.15.16:0/3795206216 shutdown_connections 2023-09-30T22:35:51.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:51.504+0000 7faacd89e700 1 -- 172.21.15.16:0/3795206216 wait complete. 2023-09-30T22:35:51.519 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:35:51.919 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:35:51.964 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:35:51.990 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:35:52.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.073+0000 7f5f45a53700 1 Processor -- start 2023-09-30T22:35:52.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.074+0000 7f5f45a53700 1 -- start start 2023-09-30T22:35:52.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.074+0000 7f5f45a53700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5f401337f0 0x7f5f40138880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:52.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.074+0000 7f5f45a53700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5f40138dc0 0x7f5f4013b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:52.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.074+0000 7f5f45a53700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f4006b990 con 0x7f5f40133420 2023-09-30T22:35:52.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.074+0000 7f5f45a53700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f4009b0e0 con 0x7f5f40138dc0 2023-09-30T22:35:52.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.074+0000 7f5f45a53700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f40133c20 con 0x7f5f401337f0 2023-09-30T22:35:52.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.074+0000 7f5f3e7fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5f401337f0 0x7f5f40138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:52.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.074+0000 7f5f3e7fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5f401337f0 0x7f5f40138880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47378/0 (socket says 172.21.15.16:47378) 2023-09-30T22:35:52.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.074+0000 7f5f3e7fc700 1 -- 172.21.15.16:0/866206845 learned_addr learned my addr 172.21.15.16:0/866206845 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:52.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.074+0000 7f5f3f7fe700 1 --2- 172.21.15.16:0/866206845 >> v2:172.21.15.16:3300/0 conn(0x7f5f40138dc0 0x7f5f4013b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:52.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.075+0000 7f5f3e7fc700 1 -- 172.21.15.16:0/866206845 >> v2:172.21.15.16:3300/0 conn(0x7f5f40138dc0 msgr2=0x7f5f4013b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:52.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.075+0000 7f5f3e7fc700 1 --2- 172.21.15.16:0/866206845 >> v2:172.21.15.16:3300/0 conn(0x7f5f40138dc0 0x7f5f4013b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:52.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.075+0000 7f5f3e7fc700 1 -- 172.21.15.16:0/866206845 >> v1:172.21.15.16:6789/0 conn(0x7f5f40133420 legacy=0x7f5f40133e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:52.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.075+0000 7f5f3e7fc700 1 -- 172.21.15.16:0/866206845 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f400a0590 con 0x7f5f401337f0 2023-09-30T22:35:52.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.075+0000 7f5f3e7fc700 1 --2- 172.21.15.16:0/866206845 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5f401337f0 0x7f5f40138880 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f5f340098f0 tx=0x7f5f34009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22e6b5e425c4e701 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:52.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.075+0000 7f5f3dffb700 1 -- 172.21.15.16:0/866206845 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f34011830 con 0x7f5f401337f0 2023-09-30T22:35:52.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.075+0000 7f5f3dffb700 1 -- 172.21.15.16:0/866206845 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f34011990 con 0x7f5f401337f0 2023-09-30T22:35:52.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.075+0000 7f5f3dffb700 1 -- 172.21.15.16:0/866206845 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f34011d20 con 0x7f5f401337f0 2023-09-30T22:35:52.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.076+0000 7f5f45a53700 1 -- 172.21.15.16:0/866206845 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5f401337f0 msgr2=0x7f5f40138880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:52.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.076+0000 7f5f45a53700 1 --2- 172.21.15.16:0/866206845 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5f401337f0 0x7f5f40138880 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f5f340098f0 tx=0x7f5f34009c00 comp rx=0 tx=0).stop 2023-09-30T22:35:52.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.076+0000 7f5f45a53700 1 -- 172.21.15.16:0/866206845 shutdown_connections 2023-09-30T22:35:52.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.076+0000 7f5f45a53700 1 --2- 172.21.15.16:0/866206845 >> v2:172.21.15.16:3300/0 conn(0x7f5f40138dc0 0x7f5f4013b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:52.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.076+0000 7f5f45a53700 1 --2- 172.21.15.16:0/866206845 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5f401337f0 0x7f5f40138880 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:52.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.076+0000 7f5f45a53700 1 -- 172.21.15.16:0/866206845 >> 172.21.15.16:0/866206845 conn(0x7f5f400a4210 msgr2=0x7f5f400a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:52.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.076+0000 7f5f45a53700 1 -- 172.21.15.16:0/866206845 shutdown_connections 2023-09-30T22:35:52.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.076+0000 7f5f45a53700 1 -- 172.21.15.16:0/866206845 wait complete. 2023-09-30T22:35:52.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.077+0000 7f5f45a53700 1 Processor -- start 2023-09-30T22:35:52.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.077+0000 7f5f45a53700 1 -- start start 2023-09-30T22:35:52.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.078+0000 7f5f45a53700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f40133420 0x7f5f40137bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:52.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.078+0000 7f5f45a53700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5f401337f0 0x7f5f40138110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:52.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.078+0000 7f5f45a53700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5f40138dc0 0x7f5f4009c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:52.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.078+0000 7f5f45a53700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f40138650 con 0x7f5f40133420 2023-09-30T22:35:52.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.078+0000 7f5f45a53700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4012fc30 con 0x7f5f40138dc0 2023-09-30T22:35:52.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.078+0000 7f5f45a53700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4012e2d0 con 0x7f5f401337f0 2023-09-30T22:35:52.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.078+0000 7f5f3e7fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5f401337f0 0x7f5f40138110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:52.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.078+0000 7f5f3e7fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5f401337f0 0x7f5f40138110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35050/0 (socket says 172.21.15.16:35050) 2023-09-30T22:35:52.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.078+0000 7f5f3e7fc700 1 -- 172.21.15.16:0/4159573275 learned_addr learned my addr 172.21.15.16:0/4159573275 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:52.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.078+0000 7f5f3f7fe700 1 --2- 172.21.15.16:0/4159573275 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5f40138dc0 0x7f5f4009c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:52.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.078+0000 7f5f3effd700 1 --2- 172.21.15.16:0/4159573275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f40133420 0x7f5f40137bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:52.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.078+0000 7f5f3e7fc700 1 -- 172.21.15.16:0/4159573275 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5f40138dc0 msgr2=0x7f5f4009c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:52.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.078+0000 7f5f3e7fc700 1 --2- 172.21.15.16:0/4159573275 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5f40138dc0 0x7f5f4009c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:52.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.078+0000 7f5f3e7fc700 1 -- 172.21.15.16:0/4159573275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f40133420 msgr2=0x7f5f40137bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:52.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.078+0000 7f5f3e7fc700 1 --2- 172.21.15.16:0/4159573275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f40133420 0x7f5f40137bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:52.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.078+0000 7f5f3e7fc700 1 -- 172.21.15.16:0/4159573275 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f34009550 con 0x7f5f401337f0 2023-09-30T22:35:52.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.079+0000 7f5f3e7fc700 1 --2- 172.21.15.16:0/4159573275 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5f401337f0 0x7f5f40138110 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f5f34000c00 tx=0x7f5f34004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:52.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.079+0000 7f5f27fff700 1 -- 172.21.15.16:0/4159573275 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f340036a0 con 0x7f5f401337f0 2023-09-30T22:35:52.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.079+0000 7f5f45a53700 1 -- 172.21.15.16:0/4159573275 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f401346b0 con 0x7f5f401337f0 2023-09-30T22:35:52.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.079+0000 7f5f27fff700 1 -- 172.21.15.16:0/4159573275 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f34003800 con 0x7f5f401337f0 2023-09-30T22:35:52.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.079+0000 7f5f27fff700 1 -- 172.21.15.16:0/4159573275 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f34003bb0 con 0x7f5f401337f0 2023-09-30T22:35:52.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.079+0000 7f5f45a53700 1 -- 172.21.15.16:0/4159573275 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f40130810 con 0x7f5f401337f0 2023-09-30T22:35:52.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.080+0000 7f5f27fff700 1 -- 172.21.15.16:0/4159573275 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5f34021070 con 0x7f5f401337f0 2023-09-30T22:35:52.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.080+0000 7f5f27fff700 1 -- 172.21.15.16:0/4159573275 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f5f34040dc0 con 0x7f5f401337f0 2023-09-30T22:35:52.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.080+0000 7f5f45a53700 1 -- 172.21.15.16:0/4159573275 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f401309a0 con 0x7f5f401337f0 2023-09-30T22:35:52.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.085+0000 7f5f27fff700 1 -- 172.21.15.16:0/4159573275 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5f34029020 con 0x7f5f401337f0 2023-09-30T22:35:52.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.110+0000 7ff3451e4700 1 Processor -- start 2023-09-30T22:35:52.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.111+0000 7ff3451e4700 1 -- start start 2023-09-30T22:35:52.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.111+0000 7ff3451e4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff34009d6d0 0x7ff34009daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:52.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.111+0000 7ff3451e4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff34009dfe0 0x7ff3401335c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:52.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.111+0000 7ff3451e4700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff34009a570 con 0x7ff340133b00 2023-09-30T22:35:52.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.111+0000 7ff3451e4700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff34009b1c0 con 0x7ff34009dfe0 2023-09-30T22:35:52.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.111+0000 7ff3451e4700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff34009f240 con 0x7ff34009d6d0 2023-09-30T22:35:52.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.111+0000 7ff33ed9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff34009d6d0 0x7ff34009daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:52.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.111+0000 7ff33ed9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff34009d6d0 0x7ff34009daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35058/0 (socket says 172.21.15.16:35058) 2023-09-30T22:35:52.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.111+0000 7ff33ed9d700 1 -- 172.21.15.16:0/595138552 learned_addr learned my addr 172.21.15.16:0/595138552 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:52.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.111+0000 7ff33e59c700 1 --2- 172.21.15.16:0/595138552 >> v2:172.21.15.16:3300/0 conn(0x7ff34009dfe0 0x7ff3401335c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:52.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.112+0000 7ff33ed9d700 1 -- 172.21.15.16:0/595138552 >> v2:172.21.15.16:3300/0 conn(0x7ff34009dfe0 msgr2=0x7ff3401335c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:52.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.112+0000 7ff33ed9d700 1 --2- 172.21.15.16:0/595138552 >> v2:172.21.15.16:3300/0 conn(0x7ff34009dfe0 0x7ff3401335c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:52.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.112+0000 7ff33ed9d700 1 -- 172.21.15.16:0/595138552 >> v1:172.21.15.16:6789/0 conn(0x7ff340133b00 legacy=0x7ff34013aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:52.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.112+0000 7ff33ed9d700 1 -- 172.21.15.16:0/595138552 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff34009a2d0 con 0x7ff34009d6d0 2023-09-30T22:35:52.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.112+0000 7ff33e59c700 1 --2- 172.21.15.16:0/595138552 >> v2:172.21.15.16:3300/0 conn(0x7ff34009dfe0 0x7ff3401335c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:52.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.112+0000 7ff33ed9d700 1 --2- 172.21.15.16:0/595138552 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff34009d6d0 0x7ff34009daa0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7ff3300098f0 tx=0x7ff330009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=747988065bbe6519 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:52.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.112+0000 7ff33dd9b700 1 -- 172.21.15.16:0/595138552 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff330011830 con 0x7ff34009d6d0 2023-09-30T22:35:52.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.112+0000 7ff33dd9b700 1 -- 172.21.15.16:0/595138552 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff330011990 con 0x7ff34009d6d0 2023-09-30T22:35:52.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.112+0000 7ff33dd9b700 1 -- 172.21.15.16:0/595138552 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff330011d20 con 0x7ff34009d6d0 2023-09-30T22:35:52.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.113+0000 7ff3451e4700 1 -- 172.21.15.16:0/595138552 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff34009d6d0 msgr2=0x7ff34009daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:52.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.113+0000 7ff3451e4700 1 --2- 172.21.15.16:0/595138552 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff34009d6d0 0x7ff34009daa0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7ff3300098f0 tx=0x7ff330009c00 comp rx=0 tx=0).stop 2023-09-30T22:35:52.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.113+0000 7ff3451e4700 1 -- 172.21.15.16:0/595138552 shutdown_connections 2023-09-30T22:35:52.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.113+0000 7ff3451e4700 1 --2- 172.21.15.16:0/595138552 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff34009d6d0 0x7ff34009daa0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:52.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.113+0000 7ff3451e4700 1 --2- 172.21.15.16:0/595138552 >> v2:172.21.15.16:3300/0 conn(0x7ff34009dfe0 0x7ff3401335c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:52.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.113+0000 7ff3451e4700 1 -- 172.21.15.16:0/595138552 >> 172.21.15.16:0/595138552 conn(0x7ff340099c20 msgr2=0x7ff3400a2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:52.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.113+0000 7ff3451e4700 1 -- 172.21.15.16:0/595138552 shutdown_connections 2023-09-30T22:35:52.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.113+0000 7ff3451e4700 1 -- 172.21.15.16:0/595138552 wait complete. 2023-09-30T22:35:52.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.115+0000 7ff3451e4700 1 Processor -- start 2023-09-30T22:35:52.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.115+0000 7ff3451e4700 1 -- start start 2023-09-30T22:35:52.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.116+0000 7ff3451e4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff34009d6d0 0x7ff34011f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:52.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.116+0000 7ff3451e4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff34009dfe0 0x7ff34011f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:52.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.116+0000 7ff3451e4700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff340133b00 0x7ff34011fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:52.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.116+0000 7ff3451e4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff340132440 con 0x7ff34009dfe0 2023-09-30T22:35:52.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.116+0000 7ff3451e4700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff340121280 con 0x7ff340133b00 2023-09-30T22:35:52.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.116+0000 7ff3451e4700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff340121eb0 con 0x7ff34009d6d0 2023-09-30T22:35:52.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.116+0000 7ff33e59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff34009dfe0 0x7ff34011f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:52.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.116+0000 7ff33e59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff34009dfe0 0x7ff34011f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38712/0 (socket says 172.21.15.16:38712) 2023-09-30T22:35:52.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.116+0000 7ff33e59c700 1 -- 172.21.15.16:0/1672014276 learned_addr learned my addr 172.21.15.16:0/1672014276 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:52.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.116+0000 7ff33ed9d700 1 --2- 172.21.15.16:0/1672014276 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff34009d6d0 0x7ff34011f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:52.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.116+0000 7ff33f59e700 1 --2- 172.21.15.16:0/1672014276 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff340133b00 0x7ff34011fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:52.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.116+0000 7ff33ed9d700 1 -- 172.21.15.16:0/1672014276 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff340133b00 msgr2=0x7ff34011fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:52.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.116+0000 7ff33ed9d700 1 --2- 172.21.15.16:0/1672014276 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff340133b00 0x7ff34011fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:52.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.116+0000 7ff33ed9d700 1 -- 172.21.15.16:0/1672014276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff34009dfe0 msgr2=0x7ff34011f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:52.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.116+0000 7ff33ed9d700 1 --2- 172.21.15.16:0/1672014276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff34009dfe0 0x7ff34011f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:52.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.116+0000 7ff33ed9d700 1 -- 172.21.15.16:0/1672014276 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3340094a0 con 0x7ff34009d6d0 2023-09-30T22:35:52.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.116+0000 7ff33e59c700 1 --2- 172.21.15.16:0/1672014276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff34009dfe0 0x7ff34011f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:52.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.117+0000 7ff33ed9d700 1 --2- 172.21.15.16:0/1672014276 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff34009d6d0 0x7ff34011f050 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7ff330016000 tx=0x7ff330004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:52.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.117+0000 7ff3277fe700 1 -- 172.21.15.16:0/1672014276 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff330004510 con 0x7ff34009d6d0 2023-09-30T22:35:52.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.117+0000 7ff3277fe700 1 -- 172.21.15.16:0/1672014276 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff33001e070 con 0x7ff34009d6d0 2023-09-30T22:35:52.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.117+0000 7ff3451e4700 1 -- 172.21.15.16:0/1672014276 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff330009550 con 0x7ff34009d6d0 2023-09-30T22:35:52.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.117+0000 7ff3277fe700 1 -- 172.21.15.16:0/1672014276 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3300038c0 con 0x7ff34009d6d0 2023-09-30T22:35:52.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.117+0000 7ff3451e4700 1 -- 172.21.15.16:0/1672014276 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff34009c900 con 0x7ff34009d6d0 2023-09-30T22:35:52.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.118+0000 7ff3277fe700 1 -- 172.21.15.16:0/1672014276 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff330003a20 con 0x7ff34009d6d0 2023-09-30T22:35:52.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.118+0000 7ff3451e4700 1 -- 172.21.15.16:0/1672014276 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff34004f2a0 con 0x7ff34009d6d0 2023-09-30T22:35:52.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.118+0000 7ff3277fe700 1 -- 172.21.15.16:0/1672014276 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7ff3300404b0 con 0x7ff34009d6d0 2023-09-30T22:35:52.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.123+0000 7ff3277fe700 1 -- 172.21.15.16:0/1672014276 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff33006e750 con 0x7ff34009d6d0 2023-09-30T22:35:52.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.154+0000 7f6f5ad7d700 1 Processor -- start 2023-09-30T22:35:52.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.154+0000 7f6f5ad7d700 1 -- start start 2023-09-30T22:35:52.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.155+0000 7f6f5ad7d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6f540a0a30 0x7f6f540a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:52.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.155+0000 7f6f5ad7d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6f540a1340 0x7f6f54133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:52.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.155+0000 7f6f5ad7d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6f54133fa0 0x7f6f54134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:52.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.155+0000 7f6f5ad7d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f5409edd0 con 0x7f6f540a1340 2023-09-30T22:35:52.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.155+0000 7f6f5ad7d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5409e1f0 con 0x7f6f540a0a30 2023-09-30T22:35:52.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.155+0000 7f6f5ad7d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5409dc50 con 0x7f6f54133fa0 2023-09-30T22:35:52.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.155+0000 7f6f53fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6f540a1340 0x7f6f54133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:52.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.155+0000 7f6f53fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6f540a1340 0x7f6f54133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38724/0 (socket says 172.21.15.16:38724) 2023-09-30T22:35:52.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.155+0000 7f6f53fff700 1 -- 172.21.15.16:0/2046454168 learned_addr learned my addr 172.21.15.16:0/2046454168 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:52.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.155+0000 7f6f5931a700 1 --2- 172.21.15.16:0/2046454168 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6f54133fa0 0x7f6f54134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:52.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.155+0000 7f6f58b19700 1 --2- 172.21.15.16:0/2046454168 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6f540a0a30 0x7f6f540a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:52.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.155+0000 7f6f58b19700 1 -- 172.21.15.16:0/2046454168 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6f54133fa0 msgr2=0x7f6f54134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:52.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.155+0000 7f6f58b19700 1 --2- 172.21.15.16:0/2046454168 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6f54133fa0 0x7f6f54134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:52.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.155+0000 7f6f58b19700 1 -- 172.21.15.16:0/2046454168 >> v2:172.21.15.16:3300/0 conn(0x7f6f540a1340 msgr2=0x7f6f54133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:52.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.155+0000 7f6f58b19700 1 --2- 172.21.15.16:0/2046454168 >> v2:172.21.15.16:3300/0 conn(0x7f6f540a1340 0x7f6f54133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:52.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.155+0000 7f6f58b19700 1 -- 172.21.15.16:0/2046454168 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f540a1fe0 con 0x7f6f540a0a30 2023-09-30T22:35:52.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.155+0000 7f6f5931a700 1 --2- 172.21.15.16:0/2046454168 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6f54133fa0 0x7f6f54134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:52.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.156+0000 7f6f58b19700 1 --2- 172.21.15.16:0/2046454168 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6f540a0a30 0x7f6f540a0e00 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f6f44009870 tx=0x7f6f44009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a7bdfd90b5c2323 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:52.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.156+0000 7f6f537fe700 1 -- 172.21.15.16:0/2046454168 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f44011830 con 0x7f6f540a0a30 2023-09-30T22:35:52.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.156+0000 7f6f537fe700 1 -- 172.21.15.16:0/2046454168 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f44011990 con 0x7f6f540a0a30 2023-09-30T22:35:52.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.156+0000 7f6f537fe700 1 -- 172.21.15.16:0/2046454168 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f44011d20 con 0x7f6f540a0a30 2023-09-30T22:35:52.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.157+0000 7f6f5ad7d700 1 -- 172.21.15.16:0/2046454168 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6f540a0a30 msgr2=0x7f6f540a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:52.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.157+0000 7f6f5ad7d700 1 --2- 172.21.15.16:0/2046454168 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6f540a0a30 0x7f6f540a0e00 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f6f44009870 tx=0x7f6f44009b80 comp rx=0 tx=0).stop 2023-09-30T22:35:52.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.157+0000 7f6f5ad7d700 1 -- 172.21.15.16:0/2046454168 shutdown_connections 2023-09-30T22:35:52.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.157+0000 7f6f5ad7d700 1 --2- 172.21.15.16:0/2046454168 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6f540a0a30 0x7f6f540a0e00 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:52.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.157+0000 7f6f5ad7d700 1 --2- 172.21.15.16:0/2046454168 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6f54133fa0 0x7f6f54134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:52.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.157+0000 7f6f5ad7d700 1 --2- 172.21.15.16:0/2046454168 >> v2:172.21.15.16:3300/0 conn(0x7f6f540a1340 0x7f6f54133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:52.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.157+0000 7f6f5ad7d700 1 -- 172.21.15.16:0/2046454168 >> 172.21.15.16:0/2046454168 conn(0x7f6f540a6550 msgr2=0x7f6f540a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:52.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.157+0000 7f6f5ad7d700 1 -- 172.21.15.16:0/2046454168 shutdown_connections 2023-09-30T22:35:52.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.157+0000 7f6f5ad7d700 1 -- 172.21.15.16:0/2046454168 wait complete. 2023-09-30T22:35:52.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.158+0000 7f6f5ad7d700 1 Processor -- start 2023-09-30T22:35:52.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.158+0000 7f6f5ad7d700 1 -- start start 2023-09-30T22:35:52.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.158+0000 7f6f5ad7d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6f540a0a30 0x7f6f54121510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:52.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.158+0000 7f6f5ad7d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6f540a1340 0x7f6f54121a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:52.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.159+0000 7f6f5ad7d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f54133fa0 0x7f6f54121f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:52.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.159+0000 7f6f5ad7d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f540a36e0 con 0x7f6f54133fa0 2023-09-30T22:35:52.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.159+0000 7f6f5ad7d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f540a0220 con 0x7f6f540a1340 2023-09-30T22:35:52.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.159+0000 7f6f5ad7d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6f541230a0 con 0x7f6f540a0a30 2023-09-30T22:35:52.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.159+0000 7f6f58b19700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6f540a0a30 0x7f6f54121510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:52.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.159+0000 7f6f58b19700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6f540a0a30 0x7f6f54121510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35100/0 (socket says 172.21.15.16:35100) 2023-09-30T22:35:52.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.159+0000 7f6f58b19700 1 -- 172.21.15.16:0/934964403 learned_addr learned my addr 172.21.15.16:0/934964403 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:52.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.159+0000 7f6f53fff700 1 --2- 172.21.15.16:0/934964403 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6f540a1340 0x7f6f54121a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:52.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.159+0000 7f6f5931a700 1 --2- 172.21.15.16:0/934964403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f54133fa0 0x7f6f54121f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:52.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.159+0000 7f6f58b19700 1 -- 172.21.15.16:0/934964403 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6f540a1340 msgr2=0x7f6f54121a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:52.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.159+0000 7f6f58b19700 1 --2- 172.21.15.16:0/934964403 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6f540a1340 0x7f6f54121a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:52.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.159+0000 7f6f58b19700 1 -- 172.21.15.16:0/934964403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f54133fa0 msgr2=0x7f6f54121f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:52.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.159+0000 7f6f58b19700 1 --2- 172.21.15.16:0/934964403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f54133fa0 0x7f6f54121f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:52.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.159+0000 7f6f58b19700 1 -- 172.21.15.16:0/934964403 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f4c009a70 con 0x7f6f540a0a30 2023-09-30T22:35:52.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.159+0000 7f6f5931a700 1 --2- 172.21.15.16:0/934964403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f54133fa0 0x7f6f54121f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:52.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.159+0000 7f6f53fff700 1 --2- 172.21.15.16:0/934964403 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6f540a1340 0x7f6f54121a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:52.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.159+0000 7f6f58b19700 1 --2- 172.21.15.16:0/934964403 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6f540a0a30 0x7f6f54121510 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f6f44000c00 tx=0x7f6f44004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:52.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.159+0000 7f6f517fa700 1 -- 172.21.15.16:0/934964403 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f44004450 con 0x7f6f540a0a30 2023-09-30T22:35:52.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.160+0000 7f6f517fa700 1 -- 172.21.15.16:0/934964403 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f4401f070 con 0x7f6f540a0a30 2023-09-30T22:35:52.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.160+0000 7f6f517fa700 1 -- 172.21.15.16:0/934964403 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f440038e0 con 0x7f6f540a0a30 2023-09-30T22:35:52.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.160+0000 7f6f5ad7d700 1 -- 172.21.15.16:0/934964403 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f440094a0 con 0x7f6f540a0a30 2023-09-30T22:35:52.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.160+0000 7f6f5ad7d700 1 -- 172.21.15.16:0/934964403 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f54124f80 con 0x7f6f540a0a30 2023-09-30T22:35:52.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.160+0000 7f6f517fa700 1 -- 172.21.15.16:0/934964403 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6f44003a40 con 0x7f6f540a0a30 2023-09-30T22:35:52.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.161+0000 7f6f517fa700 1 -- 172.21.15.16:0/934964403 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f6f44021070 con 0x7f6f540a0a30 2023-09-30T22:35:52.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.161+0000 7f6f5ad7d700 1 -- 172.21.15.16:0/934964403 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f54124b50 con 0x7f6f540a0a30 2023-09-30T22:35:52.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.166+0000 7f6f517fa700 1 -- 172.21.15.16:0/934964403 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6f4402b080 con 0x7f6f540a0a30 2023-09-30T22:35:52.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.297+0000 7f5f45a53700 1 -- 172.21.15.16:0/4159573275 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f5f40074ea0 con 0x7f5f401337f0 2023-09-30T22:35:52.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.297+0000 7f5f27fff700 1 -- 172.21.15.16:0/4159573275 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5f34015070 con 0x7f5f401337f0 2023-09-30T22:35:52.298 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:35:52.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.299+0000 7f5f45a53700 1 -- 172.21.15.16:0/4159573275 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5f401337f0 msgr2=0x7f5f40138110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:52.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.299+0000 7f5f45a53700 1 --2- 172.21.15.16:0/4159573275 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5f401337f0 0x7f5f40138110 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f5f34000c00 tx=0x7f5f34004750 comp rx=0 tx=0).stop 2023-09-30T22:35:52.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.300+0000 7f5f45a53700 1 -- 172.21.15.16:0/4159573275 shutdown_connections 2023-09-30T22:35:52.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.300+0000 7f5f45a53700 1 --2- 172.21.15.16:0/4159573275 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5f40138dc0 0x7f5f4009c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:52.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.300+0000 7f5f45a53700 1 --2- 172.21.15.16:0/4159573275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5f40133420 0x7f5f40137bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:52.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.300+0000 7f5f45a53700 1 --2- 172.21.15.16:0/4159573275 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5f401337f0 0x7f5f40138110 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:52.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.300+0000 7f5f45a53700 1 -- 172.21.15.16:0/4159573275 >> 172.21.15.16:0/4159573275 conn(0x7f5f400a4210 msgr2=0x7f5f40131400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:52.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.300+0000 7f5f45a53700 1 -- 172.21.15.16:0/4159573275 shutdown_connections 2023-09-30T22:35:52.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.300+0000 7f5f45a53700 1 -- 172.21.15.16:0/4159573275 wait complete. 2023-09-30T22:35:52.314 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:35:52.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.334+0000 7ff3451e4700 1 -- 172.21.15.16:0/1672014276 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ff34006c230 con 0x7ff34009d6d0 2023-09-30T22:35:52.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.335+0000 7ff3277fe700 1 -- 172.21.15.16:0/1672014276 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff330015030 con 0x7ff34009d6d0 2023-09-30T22:35:52.335 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:35:52.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.337+0000 7ff3451e4700 1 -- 172.21.15.16:0/1672014276 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff34009d6d0 msgr2=0x7ff34011f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:52.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.337+0000 7ff3451e4700 1 --2- 172.21.15.16:0/1672014276 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff34009d6d0 0x7ff34011f050 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7ff330016000 tx=0x7ff330004750 comp rx=0 tx=0).stop 2023-09-30T22:35:52.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.337+0000 7ff3451e4700 1 -- 172.21.15.16:0/1672014276 shutdown_connections 2023-09-30T22:35:52.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.337+0000 7ff3451e4700 1 --2- 172.21.15.16:0/1672014276 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff340133b00 0x7ff34011fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:52.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.337+0000 7ff3451e4700 1 --2- 172.21.15.16:0/1672014276 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff34009d6d0 0x7ff34011f050 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:52.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.337+0000 7ff3451e4700 1 --2- 172.21.15.16:0/1672014276 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff34009dfe0 0x7ff34011f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:52.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.337+0000 7ff3451e4700 1 -- 172.21.15.16:0/1672014276 >> 172.21.15.16:0/1672014276 conn(0x7ff340099c20 msgr2=0x7ff3400a0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:52.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.337+0000 7ff3451e4700 1 -- 172.21.15.16:0/1672014276 shutdown_connections 2023-09-30T22:35:52.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.337+0000 7ff3451e4700 1 -- 172.21.15.16:0/1672014276 wait complete. 2023-09-30T22:35:52.351 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:35:52.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.371+0000 7f6f5ad7d700 1 -- 172.21.15.16:0/934964403 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f6f54074ea0 con 0x7f6f540a0a30 2023-09-30T22:35:52.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.371+0000 7f6f517fa700 1 -- 172.21.15.16:0/934964403 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6f44015020 con 0x7f6f540a0a30 2023-09-30T22:35:52.372 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:35:52.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.374+0000 7f6f5ad7d700 1 -- 172.21.15.16:0/934964403 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6f540a0a30 msgr2=0x7f6f54121510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:52.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.374+0000 7f6f5ad7d700 1 --2- 172.21.15.16:0/934964403 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6f540a0a30 0x7f6f54121510 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f6f44000c00 tx=0x7f6f44004750 comp rx=0 tx=0).stop 2023-09-30T22:35:52.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.374+0000 7f6f5ad7d700 1 -- 172.21.15.16:0/934964403 shutdown_connections 2023-09-30T22:35:52.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.374+0000 7f6f5ad7d700 1 --2- 172.21.15.16:0/934964403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6f54133fa0 0x7f6f54121f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:52.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.374+0000 7f6f5ad7d700 1 --2- 172.21.15.16:0/934964403 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6f540a0a30 0x7f6f54121510 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:52.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.374+0000 7f6f5ad7d700 1 --2- 172.21.15.16:0/934964403 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6f540a1340 0x7f6f54121a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:52.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.374+0000 7f6f5ad7d700 1 -- 172.21.15.16:0/934964403 >> 172.21.15.16:0/934964403 conn(0x7f6f540a6550 msgr2=0x7f6f540a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:52.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.374+0000 7f6f5ad7d700 1 -- 172.21.15.16:0/934964403 shutdown_connections 2023-09-30T22:35:52.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.374+0000 7f6f5ad7d700 1 -- 172.21.15.16:0/934964403 wait complete. 2023-09-30T22:35:52.388 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:35:52.519 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:35:52.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.663+0000 7fd87a5db700 1 Processor -- start 2023-09-30T22:35:52.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.663+0000 7fd87a5db700 1 -- start start 2023-09-30T22:35:52.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.663+0000 7fd87a5db700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd87409d6f0 0x7fd87409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:52.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.663+0000 7fd87a5db700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd87409e000 0x7fd874130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:52.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.663+0000 7fd87a5db700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd87409f130 con 0x7fd87409e410 2023-09-30T22:35:52.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.663+0000 7fd87a5db700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd87409f520 con 0x7fd87409e000 2023-09-30T22:35:52.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.663+0000 7fd87a5db700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd87409e930 con 0x7fd87409d6f0 2023-09-30T22:35:52.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.663+0000 7fd8737fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd87409e000 0x7fd874130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:52.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.663+0000 7fd873fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd87409d6f0 0x7fd87409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:52.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.664+0000 7fd878b78700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd87409e410 0x7fd874130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57016/0 (socket says 172.21.15.16:57016) 2023-09-30T22:35:52.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.664+0000 7fd878b78700 1 -- 172.21.15.16:0/1141938408 learned_addr learned my addr 172.21.15.16:0/1141938408 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:52.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.664+0000 7fd873fff700 1 -- 172.21.15.16:0/1141938408 >> v2:172.21.15.16:3300/0 conn(0x7fd87409e000 msgr2=0x7fd874130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:52.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.664+0000 7fd873fff700 1 --2- 172.21.15.16:0/1141938408 >> v2:172.21.15.16:3300/0 conn(0x7fd87409e000 0x7fd874130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:52.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.664+0000 7fd873fff700 1 -- 172.21.15.16:0/1141938408 >> v1:172.21.15.16:6789/0 conn(0x7fd87409e410 legacy=0x7fd874130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:52.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.664+0000 7fd873fff700 1 -- 172.21.15.16:0/1141938408 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd87409b2d0 con 0x7fd87409d6f0 2023-09-30T22:35:52.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.664+0000 7fd8737fe700 1 --2- 172.21.15.16:0/1141938408 >> v2:172.21.15.16:3300/0 conn(0x7fd87409e000 0x7fd874130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:52.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.664+0000 7fd872ffd700 1 -- 172.21.15.16:0/1141938408 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2358290319 0 0) 0x7fd87409f130 con 0x7fd87409e410 2023-09-30T22:35:52.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.664+0000 7fd873fff700 1 --2- 172.21.15.16:0/1141938408 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd87409d6f0 0x7fd87409dac0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fd864009840 tx=0x7fd864009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ecf2a3c727f5f1b5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:52.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.664+0000 7fd872ffd700 1 -- 172.21.15.16:0/1141938408 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd864011830 con 0x7fd87409d6f0 2023-09-30T22:35:52.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.664+0000 7fd872ffd700 1 -- 172.21.15.16:0/1141938408 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd864011990 con 0x7fd87409d6f0 2023-09-30T22:35:52.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.664+0000 7fd872ffd700 1 -- 172.21.15.16:0/1141938408 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd864011d20 con 0x7fd87409d6f0 2023-09-30T22:35:52.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.665+0000 7fd87a5db700 1 -- 172.21.15.16:0/1141938408 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd87409d6f0 msgr2=0x7fd87409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:52.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.665+0000 7fd87a5db700 1 --2- 172.21.15.16:0/1141938408 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd87409d6f0 0x7fd87409dac0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fd864009840 tx=0x7fd864009b50 comp rx=0 tx=0).stop 2023-09-30T22:35:52.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.665+0000 7fd87a5db700 1 -- 172.21.15.16:0/1141938408 shutdown_connections 2023-09-30T22:35:52.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.665+0000 7fd87a5db700 1 --2- 172.21.15.16:0/1141938408 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd87409d6f0 0x7fd87409dac0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:52.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.665+0000 7fd87a5db700 1 --2- 172.21.15.16:0/1141938408 >> v2:172.21.15.16:3300/0 conn(0x7fd87409e000 0x7fd874130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:52.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.665+0000 7fd87a5db700 1 -- 172.21.15.16:0/1141938408 >> 172.21.15.16:0/1141938408 conn(0x7fd874066f50 msgr2=0x7fd87409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:52.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.665+0000 7fd87a5db700 1 -- 172.21.15.16:0/1141938408 shutdown_connections 2023-09-30T22:35:52.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.665+0000 7fd87a5db700 1 -- 172.21.15.16:0/1141938408 wait complete. 2023-09-30T22:35:52.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.666+0000 7fd87a5db700 1 Processor -- start 2023-09-30T22:35:52.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.666+0000 7fd87a5db700 1 -- start start 2023-09-30T22:35:52.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.667+0000 7fd87a5db700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd87409d6f0 0x7fd87411ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:52.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.667+0000 7fd87a5db700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd87409e000 0x7fd8741204f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:52.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.667+0000 7fd87a5db700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd87409e410 0x7fd874120a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:52.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.667+0000 7fd87a5db700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd874130370 con 0x7fd87409d6f0 2023-09-30T22:35:52.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.667+0000 7fd87a5db700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd874123530 con 0x7fd87409e410 2023-09-30T22:35:52.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.667+0000 7fd87a5db700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd8741237f0 con 0x7fd87409e000 2023-09-30T22:35:52.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.667+0000 7fd873fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd87409d6f0 0x7fd87411ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:52.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.667+0000 7fd878b78700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd87409e410 0x7fd874120a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:52.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.667+0000 7fd8737fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd87409e000 0x7fd8741204f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:52.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.667+0000 7fd873fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd87409d6f0 0x7fd87411ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38744/0 (socket says 172.21.15.16:38744) 2023-09-30T22:35:52.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.667+0000 7fd873fff700 1 -- 172.21.15.16:0/2560901461 learned_addr learned my addr 172.21.15.16:0/2560901461 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:52.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.667+0000 7fd878b78700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd87409e410 0x7fd874120a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47434/0 (socket says 172.21.15.16:47434) 2023-09-30T22:35:52.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.667+0000 7fd873fff700 1 -- 172.21.15.16:0/2560901461 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd87409e000 msgr2=0x7fd8741204f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:52.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.667+0000 7fd873fff700 1 --2- 172.21.15.16:0/2560901461 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd87409e000 0x7fd8741204f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:52.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.667+0000 7fd873fff700 1 -- 172.21.15.16:0/2560901461 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd87409e410 msgr2=0x7fd874120a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:52.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.667+0000 7fd873fff700 1 --2- 172.21.15.16:0/2560901461 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd87409e410 0x7fd874120a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:52.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.667+0000 7fd873fff700 1 -- 172.21.15.16:0/2560901461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8680094a0 con 0x7fd87409d6f0 2023-09-30T22:35:52.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.667+0000 7fd8737fe700 1 --2- 172.21.15.16:0/2560901461 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd87409e000 0x7fd8741204f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:52.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.667+0000 7fd873fff700 1 --2- 172.21.15.16:0/2560901461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd87409d6f0 0x7fd87411ffb0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fd864009b90 tx=0x7fd864004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:52.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.668+0000 7fd870ff9700 1 -- 172.21.15.16:0/2560901461 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8640036a0 con 0x7fd87409d6f0 2023-09-30T22:35:52.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.668+0000 7fd870ff9700 1 -- 172.21.15.16:0/2560901461 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd86401e070 con 0x7fd87409d6f0 2023-09-30T22:35:52.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.668+0000 7fd87a5db700 1 -- 172.21.15.16:0/2560901461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8640094a0 con 0x7fd87409d6f0 2023-09-30T22:35:52.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.668+0000 7fd870ff9700 1 -- 172.21.15.16:0/2560901461 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd864003a60 con 0x7fd87409d6f0 2023-09-30T22:35:52.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.668+0000 7fd87a5db700 1 -- 172.21.15.16:0/2560901461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd874123260 con 0x7fd87409d6f0 2023-09-30T22:35:52.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.668+0000 7fd870ff9700 1 -- 172.21.15.16:0/2560901461 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd864003bc0 con 0x7fd87409d6f0 2023-09-30T22:35:52.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.669+0000 7fd870ff9700 1 -- 172.21.15.16:0/2560901461 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd8640405c0 con 0x7fd87409d6f0 2023-09-30T22:35:52.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.669+0000 7fd87a5db700 1 -- 172.21.15.16:0/2560901461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd874074ea0 con 0x7fd87409d6f0 2023-09-30T22:35:52.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.673+0000 7fd870ff9700 1 -- 172.21.15.16:0/2560901461 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd864015070 con 0x7fd87409d6f0 2023-09-30T22:35:52.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.880+0000 7fd87a5db700 1 -- 172.21.15.16:0/2560901461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd8741223c0 con 0x7fd87409d6f0 2023-09-30T22:35:52.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.880+0000 7fd870ff9700 1 -- 172.21.15.16:0/2560901461 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd86402d970 con 0x7fd87409d6f0 2023-09-30T22:35:52.881 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:35:52.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.882+0000 7fd87a5db700 1 -- 172.21.15.16:0/2560901461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd87409d6f0 msgr2=0x7fd87411ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:52.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.882+0000 7fd87a5db700 1 --2- 172.21.15.16:0/2560901461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd87409d6f0 0x7fd87411ffb0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fd864009b90 tx=0x7fd864004750 comp rx=0 tx=0).stop 2023-09-30T22:35:52.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.882+0000 7fd87a5db700 1 -- 172.21.15.16:0/2560901461 shutdown_connections 2023-09-30T22:35:52.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.882+0000 7fd87a5db700 1 --2- 172.21.15.16:0/2560901461 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd87409e410 0x7fd874120a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:52.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.882+0000 7fd87a5db700 1 --2- 172.21.15.16:0/2560901461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd87409d6f0 0x7fd87411ffb0 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:52.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.882+0000 7fd87a5db700 1 --2- 172.21.15.16:0/2560901461 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd87409e000 0x7fd8741204f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:52.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.882+0000 7fd87a5db700 1 -- 172.21.15.16:0/2560901461 >> 172.21.15.16:0/2560901461 conn(0x7fd874066f50 msgr2=0x7fd874071110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:52.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.882+0000 7fd87a5db700 1 -- 172.21.15.16:0/2560901461 shutdown_connections 2023-09-30T22:35:52.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:52.883+0000 7fd87a5db700 1 -- 172.21.15.16:0/2560901461 wait complete. 2023-09-30T22:35:52.897 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:35:53.315 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:35:53.352 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:35:53.389 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:35:53.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.465+0000 7f950ce4d700 1 Processor -- start 2023-09-30T22:35:53.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.465+0000 7f950ce4d700 1 -- start start 2023-09-30T22:35:53.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.465+0000 7f950ce4d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f950809d6f0 0x7f950809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:53.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.465+0000 7f950ce4d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f950809e000 0x7f95081339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:53.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.466+0000 7f950ce4d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f950809b120 con 0x7f950809e410 2023-09-30T22:35:53.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.466+0000 7f950ce4d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f950806b460 con 0x7f950809e000 2023-09-30T22:35:53.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.466+0000 7f950ce4d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f950809a7e0 con 0x7f950809d6f0 2023-09-30T22:35:53.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.466+0000 7f9505d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f950809e000 0x7f95081339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:53.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.466+0000 7f950659c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f950809d6f0 0x7f950809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:53.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.466+0000 7f950659c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f950809d6f0 0x7f950809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47440/0 (socket says 172.21.15.16:47440) 2023-09-30T22:35:53.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.466+0000 7f9505d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f950809e000 0x7f95081339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38758/0 (socket says 172.21.15.16:38758) 2023-09-30T22:35:53.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.466+0000 7f950659c700 1 -- 172.21.15.16:0/3418094896 learned_addr learned my addr 172.21.15.16:0/3418094896 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:53.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.466+0000 7f9505d9b700 1 -- 172.21.15.16:0/3418094896 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f950809d6f0 msgr2=0x7f950809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:53.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.466+0000 7f9505d9b700 1 --2- 172.21.15.16:0/3418094896 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f950809d6f0 0x7f950809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:53.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.466+0000 7f9505d9b700 1 -- 172.21.15.16:0/3418094896 >> v1:172.21.15.16:6789/0 conn(0x7f950809e410 legacy=0x7f9508133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:53.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.466+0000 7f9505d9b700 1 -- 172.21.15.16:0/3418094896 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f950809a500 con 0x7f950809e000 2023-09-30T22:35:53.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.466+0000 7f950559a700 1 -- 172.21.15.16:0/3418094896 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1903539520 0 0) 0x7f950809b120 con 0x7f950809e410 2023-09-30T22:35:53.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.466+0000 7f950659c700 1 --2- 172.21.15.16:0/3418094896 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f950809d6f0 0x7f950809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:53.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.466+0000 7f9505d9b700 1 --2- 172.21.15.16:0/3418094896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f950809e000 0x7f95081339f0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f94fc009840 tx=0x7f94fc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7c11788ccde6038 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:53.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.467+0000 7f950559a700 1 -- 172.21.15.16:0/3418094896 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94fc011830 con 0x7f950809e000 2023-09-30T22:35:53.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.467+0000 7f950559a700 1 -- 172.21.15.16:0/3418094896 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94fc011990 con 0x7f950809e000 2023-09-30T22:35:53.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.467+0000 7f950559a700 1 -- 172.21.15.16:0/3418094896 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94fc011d20 con 0x7f950809e000 2023-09-30T22:35:53.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.467+0000 7f950ce4d700 1 -- 172.21.15.16:0/3418094896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f950809e000 msgr2=0x7f95081339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:53.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.467+0000 7f950ce4d700 1 --2- 172.21.15.16:0/3418094896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f950809e000 0x7f95081339f0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f94fc009840 tx=0x7f94fc009b50 comp rx=0 tx=0).stop 2023-09-30T22:35:53.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.467+0000 7f950ce4d700 1 -- 172.21.15.16:0/3418094896 shutdown_connections 2023-09-30T22:35:53.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.467+0000 7f950ce4d700 1 --2- 172.21.15.16:0/3418094896 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f950809d6f0 0x7f950809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:53.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.467+0000 7f950ce4d700 1 --2- 172.21.15.16:0/3418094896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f950809e000 0x7f95081339f0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:53.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.467+0000 7f950ce4d700 1 -- 172.21.15.16:0/3418094896 >> 172.21.15.16:0/3418094896 conn(0x7f9508066f50 msgr2=0x7f950809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:53.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.468+0000 7f950ce4d700 1 -- 172.21.15.16:0/3418094896 shutdown_connections 2023-09-30T22:35:53.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.468+0000 7f950ce4d700 1 -- 172.21.15.16:0/3418094896 wait complete. 2023-09-30T22:35:53.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.469+0000 7f950ce4d700 1 Processor -- start 2023-09-30T22:35:53.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.469+0000 7f950ce4d700 1 -- start start 2023-09-30T22:35:53.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.469+0000 7f950ce4d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f950809d6f0 0x7f95081c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:53.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.469+0000 7f950ce4d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f950809e000 0x7f95081c8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:53.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.469+0000 7f950ce4d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f950809e410 0x7f95081c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:53.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.469+0000 7f950ce4d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95081a4ce0 con 0x7f950809e000 2023-09-30T22:35:53.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.469+0000 7f950ce4d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95081a8150 con 0x7f950809d6f0 2023-09-30T22:35:53.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.469+0000 7f950ce4d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f95081a6440 con 0x7f950809e410 2023-09-30T22:35:53.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.469+0000 7f950659c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f950809d6f0 0x7f95081c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:53.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.469+0000 7f9505d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f950809e000 0x7f95081c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:53.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.469+0000 7f950659c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f950809d6f0 0x7f95081c8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47446/0 (socket says 172.21.15.16:47446) 2023-09-30T22:35:53.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.470+0000 7f950659c700 1 -- 172.21.15.16:0/4071717772 learned_addr learned my addr 172.21.15.16:0/4071717772 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:53.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.469+0000 7f9505d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f950809e000 0x7f95081c8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38764/0 (socket says 172.21.15.16:38764) 2023-09-30T22:35:53.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.469+0000 7f9506d9d700 1 --2- 172.21.15.16:0/4071717772 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f950809e410 0x7f95081c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:53.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.470+0000 7f950659c700 1 -- 172.21.15.16:0/4071717772 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f950809e410 msgr2=0x7f95081c8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:53.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.470+0000 7f950659c700 1 --2- 172.21.15.16:0/4071717772 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f950809e410 0x7f95081c8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:53.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.470+0000 7f950659c700 1 -- 172.21.15.16:0/4071717772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f950809e000 msgr2=0x7f95081c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:53.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.470+0000 7f950659c700 1 --2- 172.21.15.16:0/4071717772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f950809e000 0x7f95081c8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:53.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.470+0000 7f950659c700 1 -- 172.21.15.16:0/4071717772 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f8009550 con 0x7f950809d6f0 2023-09-30T22:35:53.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.470+0000 7f9505d9b700 1 --2- 172.21.15.16:0/4071717772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f950809e000 0x7f95081c8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:53.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.470+0000 7f9506d9d700 1 --2- 172.21.15.16:0/4071717772 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f950809e410 0x7f95081c8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:53.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.470+0000 7f950659c700 1 --2- 172.21.15.16:0/4071717772 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f950809d6f0 0x7f95081c8040 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f94f800ebe0 tx=0x7f94f800eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:53.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.470+0000 7f94f6ffd700 1 -- 172.21.15.16:0/4071717772 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f800c850 con 0x7f950809d6f0 2023-09-30T22:35:53.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.470+0000 7f94f6ffd700 1 -- 172.21.15.16:0/4071717772 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94f800c9b0 con 0x7f950809d6f0 2023-09-30T22:35:53.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.470+0000 7f950ce4d700 1 -- 172.21.15.16:0/4071717772 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94fc0094a0 con 0x7f950809d6f0 2023-09-30T22:35:53.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.471+0000 7f94f6ffd700 1 -- 172.21.15.16:0/4071717772 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f800cd40 con 0x7f950809d6f0 2023-09-30T22:35:53.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.471+0000 7f950ce4d700 1 -- 172.21.15.16:0/4071717772 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95081a6c30 con 0x7f950809d6f0 2023-09-30T22:35:53.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.471+0000 7f950ce4d700 1 -- 172.21.15.16:0/4071717772 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9508074ea0 con 0x7f950809d6f0 2023-09-30T22:35:53.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.472+0000 7f94f6ffd700 1 -- 172.21.15.16:0/4071717772 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f94f8013470 con 0x7f950809d6f0 2023-09-30T22:35:53.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.472+0000 7f94f6ffd700 1 -- 172.21.15.16:0/4071717772 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f94f8040070 con 0x7f950809d6f0 2023-09-30T22:35:53.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.477+0000 7f94f6ffd700 1 -- 172.21.15.16:0/4071717772 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f94f8011760 con 0x7f950809d6f0 2023-09-30T22:35:53.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.513+0000 7f9a3e226700 1 Processor -- start 2023-09-30T22:35:53.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.513+0000 7f9a3e226700 1 -- start start 2023-09-30T22:35:53.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.513+0000 7f9a3e226700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9a3809d6f0 0x7f9a3809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:53.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.513+0000 7f9a3e226700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9a3809e000 0x7f9a381339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:53.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.513+0000 7f9a3e226700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9a3809e410 0x7f9a38133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:53.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.513+0000 7f9a3e226700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a3806b460 con 0x7f9a3809e410 2023-09-30T22:35:53.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.513+0000 7f9a3e226700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3809b120 con 0x7f9a3809d6f0 2023-09-30T22:35:53.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.513+0000 7f9a3e226700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3809a7e0 con 0x7f9a3809e000 2023-09-30T22:35:53.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.513+0000 7f9a377fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9a3809d6f0 0x7f9a3809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:53.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.513+0000 7f9a37fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9a3809e410 0x7f9a38133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:53.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.513+0000 7f9a36ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9a3809e000 0x7f9a381339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:53.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.513+0000 7f9a377fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9a3809d6f0 0x7f9a3809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47458/0 (socket says 172.21.15.16:47458) 2023-09-30T22:35:53.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.513+0000 7f9a377fe700 1 -- 172.21.15.16:0/3201690399 learned_addr learned my addr 172.21.15.16:0/3201690399 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:53.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.514+0000 7f9a377fe700 1 -- 172.21.15.16:0/3201690399 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9a3809e000 msgr2=0x7f9a381339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:53.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.514+0000 7f9a377fe700 1 --2- 172.21.15.16:0/3201690399 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9a3809e000 0x7f9a381339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:53.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.514+0000 7f9a377fe700 1 -- 172.21.15.16:0/3201690399 >> v2:172.21.15.16:3300/0 conn(0x7f9a3809e410 msgr2=0x7f9a38133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:53.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.514+0000 7f9a377fe700 1 --2- 172.21.15.16:0/3201690399 >> v2:172.21.15.16:3300/0 conn(0x7f9a3809e410 0x7f9a38133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:53.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.514+0000 7f9a377fe700 1 -- 172.21.15.16:0/3201690399 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a3809a500 con 0x7f9a3809d6f0 2023-09-30T22:35:53.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.514+0000 7f9a37fff700 1 --2- 172.21.15.16:0/3201690399 >> v2:172.21.15.16:3300/0 conn(0x7f9a3809e410 0x7f9a38133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:53.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.514+0000 7f9a377fe700 1 --2- 172.21.15.16:0/3201690399 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9a3809d6f0 0x7f9a3809dac0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f9a28009920 tx=0x7f9a28009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4e894c32f5d7fb6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:53.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.514+0000 7f9a367fc700 1 -- 172.21.15.16:0/3201690399 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a28011830 con 0x7f9a3809d6f0 2023-09-30T22:35:53.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.514+0000 7f9a367fc700 1 -- 172.21.15.16:0/3201690399 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a28011990 con 0x7f9a3809d6f0 2023-09-30T22:35:53.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.514+0000 7f9a367fc700 1 -- 172.21.15.16:0/3201690399 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a28011d20 con 0x7f9a3809d6f0 2023-09-30T22:35:53.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.515+0000 7f9a3e226700 1 -- 172.21.15.16:0/3201690399 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9a3809d6f0 msgr2=0x7f9a3809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:53.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.515+0000 7f9a3e226700 1 --2- 172.21.15.16:0/3201690399 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9a3809d6f0 0x7f9a3809dac0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f9a28009920 tx=0x7f9a28009c30 comp rx=0 tx=0).stop 2023-09-30T22:35:53.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.515+0000 7f9a3e226700 1 -- 172.21.15.16:0/3201690399 shutdown_connections 2023-09-30T22:35:53.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.515+0000 7f9a3e226700 1 --2- 172.21.15.16:0/3201690399 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9a3809d6f0 0x7f9a3809dac0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:53.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.515+0000 7f9a3e226700 1 --2- 172.21.15.16:0/3201690399 >> v2:172.21.15.16:3300/0 conn(0x7f9a3809e410 0x7f9a38133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:53.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.515+0000 7f9a3e226700 1 --2- 172.21.15.16:0/3201690399 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9a3809e000 0x7f9a381339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:53.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.515+0000 7f9a3e226700 1 -- 172.21.15.16:0/3201690399 >> 172.21.15.16:0/3201690399 conn(0x7f9a38066f50 msgr2=0x7f9a3809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:53.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.515+0000 7f9a3e226700 1 -- 172.21.15.16:0/3201690399 shutdown_connections 2023-09-30T22:35:53.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.515+0000 7f9a3e226700 1 -- 172.21.15.16:0/3201690399 wait complete. 2023-09-30T22:35:53.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.516+0000 7f9a3e226700 1 Processor -- start 2023-09-30T22:35:53.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.516+0000 7f9a3e226700 1 -- start start 2023-09-30T22:35:53.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.517+0000 7f9a3e226700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9a3809d6f0 0x7f9a381a5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:53.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.517+0000 7f9a3e226700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a3809e000 0x7f9a381a6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:53.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.517+0000 7f9a3e226700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9a3809e410 0x7f9a381a69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:53.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.517+0000 7f9a3e226700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a381a7ac0 con 0x7f9a3809e000 2023-09-30T22:35:53.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.517+0000 7f9a3e226700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a381a80e0 con 0x7f9a3809d6f0 2023-09-30T22:35:53.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.517+0000 7f9a3e226700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9a381a8d10 con 0x7f9a3809e410 2023-09-30T22:35:53.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.517+0000 7f9a377fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9a3809d6f0 0x7f9a381a5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:53.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.517+0000 7f9a377fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9a3809d6f0 0x7f9a381a5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47472/0 (socket says 172.21.15.16:47472) 2023-09-30T22:35:53.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.517+0000 7f9a377fe700 1 -- 172.21.15.16:0/2659279524 learned_addr learned my addr 172.21.15.16:0/2659279524 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:53.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.517+0000 7f9a37fff700 1 --2- 172.21.15.16:0/2659279524 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9a3809e410 0x7f9a381a69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:53.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.517+0000 7f9a36ffd700 1 --2- 172.21.15.16:0/2659279524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a3809e000 0x7f9a381a6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:53.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.517+0000 7f9a377fe700 1 -- 172.21.15.16:0/2659279524 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9a3809e410 msgr2=0x7f9a381a69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:53.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.517+0000 7f9a377fe700 1 --2- 172.21.15.16:0/2659279524 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9a3809e410 0x7f9a381a69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:53.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.517+0000 7f9a377fe700 1 -- 172.21.15.16:0/2659279524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a3809e000 msgr2=0x7f9a381a6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:53.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.517+0000 7f9a377fe700 1 --2- 172.21.15.16:0/2659279524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a3809e000 0x7f9a381a6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:53.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.517+0000 7f9a377fe700 1 -- 172.21.15.16:0/2659279524 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a30009a70 con 0x7f9a3809d6f0 2023-09-30T22:35:53.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.518+0000 7f9a37fff700 1 --2- 172.21.15.16:0/2659279524 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9a3809e410 0x7f9a381a69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:53.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.518+0000 7f9a377fe700 1 --2- 172.21.15.16:0/2659279524 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9a3809d6f0 0x7f9a381a5f40 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f9a28009c70 tx=0x7f9a28004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:53.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.518+0000 7f9a1ffff700 1 -- 172.21.15.16:0/2659279524 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a28004240 con 0x7f9a3809d6f0 2023-09-30T22:35:53.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.518+0000 7f9a1ffff700 1 -- 172.21.15.16:0/2659279524 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a2801f070 con 0x7f9a3809d6f0 2023-09-30T22:35:53.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.518+0000 7f9a3e226700 1 -- 172.21.15.16:0/2659279524 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a28009550 con 0x7f9a3809d6f0 2023-09-30T22:35:53.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.518+0000 7f9a1ffff700 1 -- 172.21.15.16:0/2659279524 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a280036a0 con 0x7f9a3809d6f0 2023-09-30T22:35:53.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.518+0000 7f9a3e226700 1 -- 172.21.15.16:0/2659279524 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a380a2140 con 0x7f9a3809d6f0 2023-09-30T22:35:53.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.519+0000 7f9a3e226700 1 -- 172.21.15.16:0/2659279524 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a3809c710 con 0x7f9a3809d6f0 2023-09-30T22:35:53.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.520+0000 7f9a1ffff700 1 -- 172.21.15.16:0/2659279524 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9a28021030 con 0x7f9a3809d6f0 2023-09-30T22:35:53.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.520+0000 7f9a1ffff700 1 -- 172.21.15.16:0/2659279524 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f9a28040c50 con 0x7f9a3809d6f0 2023-09-30T22:35:53.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.524+0000 7f9a1ffff700 1 -- 172.21.15.16:0/2659279524 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9a28040e60 con 0x7f9a3809d6f0 2023-09-30T22:35:53.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.554+0000 7fea40624700 1 Processor -- start 2023-09-30T22:35:53.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.555+0000 7fea40624700 1 -- start start 2023-09-30T22:35:53.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.555+0000 7fea40624700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea380a0a30 0x7fea380a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:53.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.555+0000 7fea40624700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fea380a1340 0x7fea38133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:53.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.555+0000 7fea40624700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea38133fa0 0x7fea38134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:53.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.555+0000 7fea40624700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea3809edd0 con 0x7fea380a1340 2023-09-30T22:35:53.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.555+0000 7fea40624700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea3809e1f0 con 0x7fea380a0a30 2023-09-30T22:35:53.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.555+0000 7fea40624700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fea3809dc50 con 0x7fea38133fa0 2023-09-30T22:35:53.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.556+0000 7fea3e3c0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea380a0a30 0x7fea380a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:53.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.556+0000 7fea3e3c0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea380a0a30 0x7fea380a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47478/0 (socket says 172.21.15.16:47478) 2023-09-30T22:35:53.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.556+0000 7fea3e3c0700 1 -- 172.21.15.16:0/3762229725 learned_addr learned my addr 172.21.15.16:0/3762229725 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:53.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.556+0000 7fea3ebc1700 1 --2- 172.21.15.16:0/3762229725 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea38133fa0 0x7fea38134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:53.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.556+0000 7fea3e3c0700 1 -- 172.21.15.16:0/3762229725 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea38133fa0 msgr2=0x7fea38134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:53.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.556+0000 7fea3e3c0700 1 --2- 172.21.15.16:0/3762229725 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea38133fa0 0x7fea38134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:53.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.556+0000 7fea3e3c0700 1 -- 172.21.15.16:0/3762229725 >> v2:172.21.15.16:3300/0 conn(0x7fea380a1340 msgr2=0x7fea38133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:53.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.556+0000 7fea3e3c0700 1 --2- 172.21.15.16:0/3762229725 >> v2:172.21.15.16:3300/0 conn(0x7fea380a1340 0x7fea38133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:53.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.556+0000 7fea3e3c0700 1 -- 172.21.15.16:0/3762229725 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea380a1fe0 con 0x7fea380a0a30 2023-09-30T22:35:53.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.556+0000 7fea3e3c0700 1 --2- 172.21.15.16:0/3762229725 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea380a0a30 0x7fea380a0e00 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fea34009920 tx=0x7fea34009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=122bc21f24fc4e40 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:53.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.556+0000 7fea3d3be700 1 -- 172.21.15.16:0/3762229725 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3400ba20 con 0x7fea380a0a30 2023-09-30T22:35:53.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.556+0000 7fea3d3be700 1 -- 172.21.15.16:0/3762229725 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea3400bb80 con 0x7fea380a0a30 2023-09-30T22:35:53.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.556+0000 7fea3d3be700 1 -- 172.21.15.16:0/3762229725 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea34009e70 con 0x7fea380a0a30 2023-09-30T22:35:53.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.557+0000 7fea40624700 1 -- 172.21.15.16:0/3762229725 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea380a0a30 msgr2=0x7fea380a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:53.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.557+0000 7fea40624700 1 --2- 172.21.15.16:0/3762229725 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea380a0a30 0x7fea380a0e00 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fea34009920 tx=0x7fea34009c30 comp rx=0 tx=0).stop 2023-09-30T22:35:53.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.557+0000 7fea40624700 1 -- 172.21.15.16:0/3762229725 shutdown_connections 2023-09-30T22:35:53.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.557+0000 7fea40624700 1 --2- 172.21.15.16:0/3762229725 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea380a0a30 0x7fea380a0e00 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:53.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.557+0000 7fea40624700 1 --2- 172.21.15.16:0/3762229725 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea38133fa0 0x7fea38134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:53.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.557+0000 7fea40624700 1 --2- 172.21.15.16:0/3762229725 >> v2:172.21.15.16:3300/0 conn(0x7fea380a1340 0x7fea38133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:53.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.557+0000 7fea40624700 1 -- 172.21.15.16:0/3762229725 >> 172.21.15.16:0/3762229725 conn(0x7fea380a6550 msgr2=0x7fea380a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:53.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.559+0000 7fea40624700 1 -- 172.21.15.16:0/3762229725 shutdown_connections 2023-09-30T22:35:53.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.559+0000 7fea40624700 1 -- 172.21.15.16:0/3762229725 wait complete. 2023-09-30T22:35:53.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.560+0000 7fea40624700 1 Processor -- start 2023-09-30T22:35:53.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.560+0000 7fea40624700 1 -- start start 2023-09-30T22:35:53.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.560+0000 7fea40624700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea380a0a30 0x7fea38120580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:53.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.560+0000 7fea40624700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea380a1340 0x7fea38120ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:53.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.560+0000 7fea40624700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea38133fa0 0x7fea38121000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:53.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.560+0000 7fea40624700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea381240a0 con 0x7fea380a0a30 2023-09-30T22:35:53.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.560+0000 7fea40624700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea381219c0 con 0x7fea38133fa0 2023-09-30T22:35:53.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.560+0000 7fea40624700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fea38122190 con 0x7fea380a1340 2023-09-30T22:35:53.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.560+0000 7fea3dbbf700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea380a1340 0x7fea38120ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:53.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.561+0000 7fea3dbbf700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea380a1340 0x7fea38120ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35160/0 (socket says 172.21.15.16:35160) 2023-09-30T22:35:53.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.561+0000 7fea3dbbf700 1 -- 172.21.15.16:0/565948465 learned_addr learned my addr 172.21.15.16:0/565948465 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:53.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.561+0000 7fea3ebc1700 1 --2- 172.21.15.16:0/565948465 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea38133fa0 0x7fea38121000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:53.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.561+0000 7fea3e3c0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea380a0a30 0x7fea38120580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:53.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.561+0000 7fea3dbbf700 1 -- 172.21.15.16:0/565948465 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea38133fa0 msgr2=0x7fea38121000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:53.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.561+0000 7fea3dbbf700 1 --2- 172.21.15.16:0/565948465 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea38133fa0 0x7fea38121000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:53.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.561+0000 7fea3dbbf700 1 -- 172.21.15.16:0/565948465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea380a0a30 msgr2=0x7fea38120580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:53.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.561+0000 7fea3dbbf700 1 --2- 172.21.15.16:0/565948465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea380a0a30 0x7fea38120580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:53.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.561+0000 7fea3dbbf700 1 -- 172.21.15.16:0/565948465 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea34009550 con 0x7fea380a1340 2023-09-30T22:35:53.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.561+0000 7fea3dbbf700 1 --2- 172.21.15.16:0/565948465 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea380a1340 0x7fea38120ac0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fea2800b560 tx=0x7fea2800b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:53.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.561+0000 7fea2effd700 1 -- 172.21.15.16:0/565948465 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea28011830 con 0x7fea380a1340 2023-09-30T22:35:53.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.561+0000 7fea2effd700 1 -- 172.21.15.16:0/565948465 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea28011990 con 0x7fea380a1340 2023-09-30T22:35:53.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.561+0000 7fea2effd700 1 -- 172.21.15.16:0/565948465 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea28011d40 con 0x7fea380a1340 2023-09-30T22:35:53.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.561+0000 7fea40624700 1 -- 172.21.15.16:0/565948465 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea38122740 con 0x7fea380a1340 2023-09-30T22:35:53.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.561+0000 7fea40624700 1 -- 172.21.15.16:0/565948465 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea38123b00 con 0x7fea380a1340 2023-09-30T22:35:53.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.562+0000 7fea2effd700 1 -- 172.21.15.16:0/565948465 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fea2801a400 con 0x7fea380a1340 2023-09-30T22:35:53.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.562+0000 7fea2effd700 1 -- 172.21.15.16:0/565948465 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fea2803f570 con 0x7fea380a1340 2023-09-30T22:35:53.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.563+0000 7fea40624700 1 -- 172.21.15.16:0/565948465 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea380a1920 con 0x7fea380a1340 2023-09-30T22:35:53.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.567+0000 7fea2effd700 1 -- 172.21.15.16:0/565948465 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fea28015020 con 0x7fea380a1340 2023-09-30T22:35:53.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.737+0000 7f950ce4d700 1 -- 172.21.15.16:0/4071717772 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f95081a7fb0 con 0x7f950809d6f0 2023-09-30T22:35:53.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.737+0000 7f94f6ffd700 1 -- 172.21.15.16:0/4071717772 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f95081a7fb0 con 0x7f950809d6f0 2023-09-30T22:35:53.738 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:35:53.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.740+0000 7f950ce4d700 1 -- 172.21.15.16:0/4071717772 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f950809d6f0 msgr2=0x7f95081c8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:53.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.740+0000 7f950ce4d700 1 --2- 172.21.15.16:0/4071717772 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f950809d6f0 0x7f95081c8040 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f94f800ebe0 tx=0x7f94f800eef0 comp rx=0 tx=0).stop 2023-09-30T22:35:53.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.740+0000 7f950ce4d700 1 -- 172.21.15.16:0/4071717772 shutdown_connections 2023-09-30T22:35:53.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.740+0000 7f950ce4d700 1 --2- 172.21.15.16:0/4071717772 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f950809d6f0 0x7f95081c8040 secure :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f94f800ebe0 tx=0x7f94f800eef0 comp rx=0 tx=0).stop 2023-09-30T22:35:53.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.740+0000 7f950ce4d700 1 --2- 172.21.15.16:0/4071717772 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f950809e410 0x7f95081c8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:53.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.740+0000 7f950ce4d700 1 --2- 172.21.15.16:0/4071717772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f950809e000 0x7f95081c8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:53.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.740+0000 7f950ce4d700 1 -- 172.21.15.16:0/4071717772 >> 172.21.15.16:0/4071717772 conn(0x7f9508066f50 msgr2=0x7f95081bbab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:53.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.741+0000 7f950ce4d700 1 -- 172.21.15.16:0/4071717772 shutdown_connections 2023-09-30T22:35:53.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.741+0000 7f950ce4d700 1 -- 172.21.15.16:0/4071717772 wait complete. 2023-09-30T22:35:53.755 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:35:53.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.774+0000 7fea40624700 1 -- 172.21.15.16:0/565948465 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fea38074ea0 con 0x7fea380a1340 2023-09-30T22:35:53.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.775+0000 7fea2effd700 1 -- 172.21.15.16:0/565948465 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fea28029090 con 0x7fea380a1340 2023-09-30T22:35:53.775 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:35:53.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.777+0000 7fea40624700 1 -- 172.21.15.16:0/565948465 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea380a1340 msgr2=0x7fea38120ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:53.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.777+0000 7fea40624700 1 --2- 172.21.15.16:0/565948465 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea380a1340 0x7fea38120ac0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fea2800b560 tx=0x7fea2800b870 comp rx=0 tx=0).stop 2023-09-30T22:35:53.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.777+0000 7fea40624700 1 -- 172.21.15.16:0/565948465 shutdown_connections 2023-09-30T22:35:53.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.777+0000 7fea40624700 1 --2- 172.21.15.16:0/565948465 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fea38133fa0 0x7fea38121000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:53.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.777+0000 7fea40624700 1 --2- 172.21.15.16:0/565948465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fea380a0a30 0x7fea38120580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:53.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.777+0000 7fea40624700 1 --2- 172.21.15.16:0/565948465 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fea380a1340 0x7fea38120ac0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:53.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.777+0000 7fea40624700 1 -- 172.21.15.16:0/565948465 >> 172.21.15.16:0/565948465 conn(0x7fea380a6550 msgr2=0x7fea3809ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:53.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.777+0000 7fea40624700 1 -- 172.21.15.16:0/565948465 shutdown_connections 2023-09-30T22:35:53.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.777+0000 7fea40624700 1 -- 172.21.15.16:0/565948465 wait complete. 2023-09-30T22:35:53.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.783+0000 7f9a3e226700 1 -- 172.21.15.16:0/2659279524 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9a38074ea0 con 0x7f9a3809d6f0 2023-09-30T22:35:53.783 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.783+0000 7f9a1ffff700 1 -- 172.21.15.16:0/2659279524 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9a28015030 con 0x7f9a3809d6f0 2023-09-30T22:35:53.783 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:35:53.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.785+0000 7f9a3e226700 1 -- 172.21.15.16:0/2659279524 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9a3809d6f0 msgr2=0x7f9a381a5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:53.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.785+0000 7f9a3e226700 1 --2- 172.21.15.16:0/2659279524 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9a3809d6f0 0x7f9a381a5f40 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f9a28009c70 tx=0x7f9a28004750 comp rx=0 tx=0).stop 2023-09-30T22:35:53.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.785+0000 7f9a3e226700 1 -- 172.21.15.16:0/2659279524 shutdown_connections 2023-09-30T22:35:53.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.785+0000 7f9a3e226700 1 --2- 172.21.15.16:0/2659279524 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9a3809d6f0 0x7f9a381a5f40 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:53.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.785+0000 7f9a3e226700 1 --2- 172.21.15.16:0/2659279524 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9a3809e410 0x7f9a381a69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:53.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.785+0000 7f9a3e226700 1 --2- 172.21.15.16:0/2659279524 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a3809e000 0x7f9a381a6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:53.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.785+0000 7f9a3e226700 1 -- 172.21.15.16:0/2659279524 >> 172.21.15.16:0/2659279524 conn(0x7f9a38066f50 msgr2=0x7f9a381bae60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:53.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.785+0000 7f9a3e226700 1 -- 172.21.15.16:0/2659279524 shutdown_connections 2023-09-30T22:35:53.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:53.785+0000 7f9a3e226700 1 -- 172.21.15.16:0/2659279524 wait complete. 2023-09-30T22:35:53.792 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:35:53.800 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:35:53.898 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:35:54.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.041+0000 7f4d401b0700 1 Processor -- start 2023-09-30T22:35:54.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.041+0000 7f4d401b0700 1 -- start start 2023-09-30T22:35:54.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.041+0000 7f4d401b0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4d3809e1d0 0x7f4d38135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:54.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.041+0000 7f4d401b0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4d38135e50 0x7f4d38138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:54.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.041+0000 7f4d401b0700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d3809f130 con 0x7f4d3809d6f0 2023-09-30T22:35:54.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.041+0000 7f4d401b0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3809f520 con 0x7f4d3809e1d0 2023-09-30T22:35:54.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.041+0000 7f4d401b0700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3809e930 con 0x7f4d38135e50 2023-09-30T22:35:54.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.042+0000 7f4d3d74b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4d3809e1d0 0x7f4d38135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:54.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.042+0000 7f4d3d74b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4d3809e1d0 0x7f4d38135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47496/0 (socket says 172.21.15.16:47496) 2023-09-30T22:35:54.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.042+0000 7f4d3d74b700 1 -- 172.21.15.16:0/1772361524 learned_addr learned my addr 172.21.15.16:0/1772361524 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:54.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.042+0000 7f4d3e74d700 1 --2- 172.21.15.16:0/1772361524 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4d38135e50 0x7f4d38138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:54.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.042+0000 7f4d3d74b700 1 -- 172.21.15.16:0/1772361524 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4d38135e50 msgr2=0x7f4d38138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:54.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.042+0000 7f4d3d74b700 1 --2- 172.21.15.16:0/1772361524 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4d38135e50 0x7f4d38138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:54.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.042+0000 7f4d3d74b700 1 -- 172.21.15.16:0/1772361524 >> v1:172.21.15.16:6789/0 conn(0x7f4d3809d6f0 legacy=0x7f4d3809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:54.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.042+0000 7f4d3cf4a700 1 -- 172.21.15.16:0/1772361524 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3669381543 0 0) 0x7f4d3809f130 con 0x7f4d3809d6f0 2023-09-30T22:35:54.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.042+0000 7f4d3d74b700 1 -- 172.21.15.16:0/1772361524 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d3809b2d0 con 0x7f4d3809e1d0 2023-09-30T22:35:54.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.042+0000 7f4d3d74b700 1 --2- 172.21.15.16:0/1772361524 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4d3809e1d0 0x7f4d38135910 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f4d2800bbc0 tx=0x7f4d2800bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ebf0dd1e74d385d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:54.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.042+0000 7f4d3cf4a700 1 -- 172.21.15.16:0/1772361524 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d2800dd30 con 0x7f4d3809e1d0 2023-09-30T22:35:54.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.042+0000 7f4d3cf4a700 1 -- 172.21.15.16:0/1772361524 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d2800de90 con 0x7f4d3809e1d0 2023-09-30T22:35:54.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.042+0000 7f4d3cf4a700 1 -- 172.21.15.16:0/1772361524 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d28010690 con 0x7f4d3809e1d0 2023-09-30T22:35:54.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.043+0000 7f4d401b0700 1 -- 172.21.15.16:0/1772361524 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4d3809e1d0 msgr2=0x7f4d38135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:54.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.043+0000 7f4d401b0700 1 --2- 172.21.15.16:0/1772361524 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4d3809e1d0 0x7f4d38135910 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f4d2800bbc0 tx=0x7f4d2800bed0 comp rx=0 tx=0).stop 2023-09-30T22:35:54.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.043+0000 7f4d401b0700 1 -- 172.21.15.16:0/1772361524 shutdown_connections 2023-09-30T22:35:54.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.043+0000 7f4d401b0700 1 --2- 172.21.15.16:0/1772361524 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4d38135e50 0x7f4d38138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:54.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.043+0000 7f4d401b0700 1 --2- 172.21.15.16:0/1772361524 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4d3809e1d0 0x7f4d38135910 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:54.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.043+0000 7f4d401b0700 1 -- 172.21.15.16:0/1772361524 >> 172.21.15.16:0/1772361524 conn(0x7f4d38066f50 msgr2=0x7f4d3809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:54.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.043+0000 7f4d401b0700 1 -- 172.21.15.16:0/1772361524 shutdown_connections 2023-09-30T22:35:54.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.043+0000 7f4d401b0700 1 -- 172.21.15.16:0/1772361524 wait complete. 2023-09-30T22:35:54.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.044+0000 7f4d401b0700 1 Processor -- start 2023-09-30T22:35:54.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.044+0000 7f4d401b0700 1 -- start start 2023-09-30T22:35:54.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.045+0000 7f4d401b0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d3809d6f0 0x7f4d381215e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:54.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.045+0000 7f4d401b0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4d3809e1d0 0x7f4d38121b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:54.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.045+0000 7f4d401b0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4d38135e50 0x7f4d381c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:54.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.045+0000 7f4d401b0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d381233f0 con 0x7f4d3809d6f0 2023-09-30T22:35:54.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.045+0000 7f4d401b0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d38135400 con 0x7f4d3809e1d0 2023-09-30T22:35:54.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.045+0000 7f4d401b0700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4d38123ff0 con 0x7f4d38135e50 2023-09-30T22:35:54.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.045+0000 7f4d3d74b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4d3809e1d0 0x7f4d38121b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:54.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.045+0000 7f4d3df4c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d3809d6f0 0x7f4d381215e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:54.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.045+0000 7f4d3d74b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4d3809e1d0 0x7f4d38121b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47504/0 (socket says 172.21.15.16:47504) 2023-09-30T22:35:54.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.045+0000 7f4d3d74b700 1 -- 172.21.15.16:0/3934290761 learned_addr learned my addr 172.21.15.16:0/3934290761 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:54.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.045+0000 7f4d3e74d700 1 --2- 172.21.15.16:0/3934290761 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4d38135e50 0x7f4d381c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:54.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.045+0000 7f4d3d74b700 1 -- 172.21.15.16:0/3934290761 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4d38135e50 msgr2=0x7f4d381c82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:54.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.045+0000 7f4d3d74b700 1 --2- 172.21.15.16:0/3934290761 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4d38135e50 0x7f4d381c82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:54.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.045+0000 7f4d3d74b700 1 -- 172.21.15.16:0/3934290761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d3809d6f0 msgr2=0x7f4d381215e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:54.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.045+0000 7f4d3d74b700 1 --2- 172.21.15.16:0/3934290761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d3809d6f0 0x7f4d381215e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:54.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.045+0000 7f4d3d74b700 1 -- 172.21.15.16:0/3934290761 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d2800b7f0 con 0x7f4d3809e1d0 2023-09-30T22:35:54.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.045+0000 7f4d3d74b700 1 --2- 172.21.15.16:0/3934290761 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4d3809e1d0 0x7f4d38121b20 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f4d28000c00 tx=0x7f4d28004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:54.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.045+0000 7f4d2e7fc700 1 -- 172.21.15.16:0/3934290761 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d28003b70 con 0x7f4d3809e1d0 2023-09-30T22:35:54.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.045+0000 7f4d2e7fc700 1 -- 172.21.15.16:0/3934290761 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d28003cd0 con 0x7f4d3809e1d0 2023-09-30T22:35:54.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.045+0000 7f4d401b0700 1 -- 172.21.15.16:0/3934290761 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d38121120 con 0x7f4d3809e1d0 2023-09-30T22:35:54.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.045+0000 7f4d2e7fc700 1 -- 172.21.15.16:0/3934290761 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d28010690 con 0x7f4d3809e1d0 2023-09-30T22:35:54.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.046+0000 7f4d401b0700 1 -- 172.21.15.16:0/3934290761 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d38124d20 con 0x7f4d3809e1d0 2023-09-30T22:35:54.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.046+0000 7f4d2e7fc700 1 -- 172.21.15.16:0/3934290761 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4d28019400 con 0x7f4d3809e1d0 2023-09-30T22:35:54.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.047+0000 7f4d401b0700 1 -- 172.21.15.16:0/3934290761 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d380a29a0 con 0x7f4d3809e1d0 2023-09-30T22:35:54.050 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.047+0000 7f4d2e7fc700 1 -- 172.21.15.16:0/3934290761 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f4d28040fb0 con 0x7f4d3809e1d0 2023-09-30T22:35:54.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.052+0000 7f4d2e7fc700 1 -- 172.21.15.16:0/3934290761 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4d280223e0 con 0x7f4d3809e1d0 2023-09-30T22:35:54.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.255+0000 7f4d401b0700 1 -- 172.21.15.16:0/3934290761 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4d38074ea0 con 0x7f4d3809e1d0 2023-09-30T22:35:54.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.255+0000 7f4d2e7fc700 1 -- 172.21.15.16:0/3934290761 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4d28019e60 con 0x7f4d3809e1d0 2023-09-30T22:35:54.256 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:35:54.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.257+0000 7f4d401b0700 1 -- 172.21.15.16:0/3934290761 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4d3809e1d0 msgr2=0x7f4d38121b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:54.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.257+0000 7f4d401b0700 1 --2- 172.21.15.16:0/3934290761 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4d3809e1d0 0x7f4d38121b20 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f4d28000c00 tx=0x7f4d28004750 comp rx=0 tx=0).stop 2023-09-30T22:35:54.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.257+0000 7f4d401b0700 1 -- 172.21.15.16:0/3934290761 shutdown_connections 2023-09-30T22:35:54.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.257+0000 7f4d401b0700 1 --2- 172.21.15.16:0/3934290761 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4d38135e50 0x7f4d381c82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:54.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.257+0000 7f4d401b0700 1 --2- 172.21.15.16:0/3934290761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d3809d6f0 0x7f4d381215e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:54.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.257+0000 7f4d401b0700 1 --2- 172.21.15.16:0/3934290761 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4d3809e1d0 0x7f4d38121b20 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:54.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.257+0000 7f4d401b0700 1 -- 172.21.15.16:0/3934290761 >> 172.21.15.16:0/3934290761 conn(0x7f4d38066f50 msgr2=0x7f4d3809fae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:54.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.257+0000 7f4d401b0700 1 -- 172.21.15.16:0/3934290761 shutdown_connections 2023-09-30T22:35:54.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.257+0000 7f4d401b0700 1 -- 172.21.15.16:0/3934290761 wait complete. 2023-09-30T22:35:54.271 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:35:54.756 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:35:54.793 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:35:54.801 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:35:54.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.906+0000 7fe6bb934700 1 Processor -- start 2023-09-30T22:35:54.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.907+0000 7fe6bb934700 1 -- start start 2023-09-30T22:35:54.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.907+0000 7fe6bb934700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe6b409e930 0x7fe6b409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:54.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.907+0000 7fe6bb934700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe6b409f650 0x7fe6b4131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:54.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.907+0000 7fe6bb934700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6b4070940 con 0x7fe6b409f240 2023-09-30T22:35:54.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.907+0000 7fe6bb934700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6b406fd60 con 0x7fe6b409e930 2023-09-30T22:35:54.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.907+0000 7fe6bb934700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b406fa80 con 0x7fe6b409f650 2023-09-30T22:35:54.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.907+0000 7fe6b8ecf700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe6b409f240 0x7fe6b41317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57042/0 (socket says 172.21.15.16:57042) 2023-09-30T22:35:54.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.907+0000 7fe6b8ecf700 1 -- 172.21.15.16:0/1464696097 learned_addr learned my addr 172.21.15.16:0/1464696097 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:54.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.907+0000 7fe6b96d0700 1 --2- 172.21.15.16:0/1464696097 >> v2:172.21.15.16:3300/0 conn(0x7fe6b409e930 0x7fe6b409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:54.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.907+0000 7fe6b9ed1700 1 --2- 172.21.15.16:0/1464696097 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe6b409f650 0x7fe6b4131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:54.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.908+0000 7fe6abfff700 1 -- 172.21.15.16:0/1464696097 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2369471387 0 0) 0x7fe6b4070940 con 0x7fe6b409f240 2023-09-30T22:35:54.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.908+0000 7fe6abfff700 1 -- 172.21.15.16:0/1464696097 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6a00035b0 con 0x7fe6b409f240 2023-09-30T22:35:54.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.908+0000 7fe6b96d0700 1 -- 172.21.15.16:0/1464696097 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe6b409f650 msgr2=0x7fe6b4131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:54.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.908+0000 7fe6b96d0700 1 --2- 172.21.15.16:0/1464696097 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe6b409f650 0x7fe6b4131f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:54.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.908+0000 7fe6b96d0700 1 -- 172.21.15.16:0/1464696097 >> v1:172.21.15.16:6789/0 conn(0x7fe6b409f240 legacy=0x7fe6b41317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:54.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.908+0000 7fe6b96d0700 1 -- 172.21.15.16:0/1464696097 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b409fb60 con 0x7fe6b409e930 2023-09-30T22:35:54.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.908+0000 7fe6b96d0700 1 --2- 172.21.15.16:0/1464696097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6b409e930 0x7fe6b409ed00 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0009840 tx=0x7fe6b0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4402fd48b07a9cd1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:54.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.908+0000 7fe6abfff700 1 -- 172.21.15.16:0/1464696097 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b0010a90 con 0x7fe6b409e930 2023-09-30T22:35:54.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.908+0000 7fe6abfff700 1 -- 172.21.15.16:0/1464696097 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6b0010bf0 con 0x7fe6b409e930 2023-09-30T22:35:54.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.908+0000 7fe6abfff700 1 -- 172.21.15.16:0/1464696097 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b0018440 con 0x7fe6b409e930 2023-09-30T22:35:54.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.909+0000 7fe6bb934700 1 -- 172.21.15.16:0/1464696097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6b409e930 msgr2=0x7fe6b409ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:54.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.909+0000 7fe6bb934700 1 --2- 172.21.15.16:0/1464696097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6b409e930 0x7fe6b409ed00 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0009840 tx=0x7fe6b0009b50 comp rx=0 tx=0).stop 2023-09-30T22:35:54.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.909+0000 7fe6bb934700 1 -- 172.21.15.16:0/1464696097 shutdown_connections 2023-09-30T22:35:54.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.909+0000 7fe6bb934700 1 --2- 172.21.15.16:0/1464696097 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe6b409f650 0x7fe6b4131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:54.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.909+0000 7fe6bb934700 1 --2- 172.21.15.16:0/1464696097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6b409e930 0x7fe6b409ed00 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:54.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.909+0000 7fe6bb934700 1 -- 172.21.15.16:0/1464696097 >> 172.21.15.16:0/1464696097 conn(0x7fe6b40a4210 msgr2=0x7fe6b40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:54.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.909+0000 7fe6bb934700 1 -- 172.21.15.16:0/1464696097 shutdown_connections 2023-09-30T22:35:54.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.909+0000 7fe6bb934700 1 -- 172.21.15.16:0/1464696097 wait complete. 2023-09-30T22:35:54.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.911+0000 7fe6bb934700 1 Processor -- start 2023-09-30T22:35:54.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.911+0000 7fe6bb934700 1 -- start start 2023-09-30T22:35:54.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.911+0000 7fe6bb934700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe6b409e930 0x7fe6b41c3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:54.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.911+0000 7fe6bb934700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6b409f240 0x7fe6b41c41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:54.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.911+0000 7fe6bb934700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe6b409f650 0x7fe6b41c4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:54.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.911+0000 7fe6bb934700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b41a3cc0 con 0x7fe6b409f240 2023-09-30T22:35:54.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.911+0000 7fe6bb934700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b41a2090 con 0x7fe6b409f650 2023-09-30T22:35:54.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.911+0000 7fe6bb934700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b41a2610 con 0x7fe6b409e930 2023-09-30T22:35:54.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.911+0000 7fe6b8ecf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6b409f240 0x7fe6b41c41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:54.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.911+0000 7fe6b8ecf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6b409f240 0x7fe6b41c41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38840/0 (socket says 172.21.15.16:38840) 2023-09-30T22:35:54.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.911+0000 7fe6b8ecf700 1 -- 172.21.15.16:0/52992759 learned_addr learned my addr 172.21.15.16:0/52992759 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:54.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.911+0000 7fe6b9ed1700 1 --2- 172.21.15.16:0/52992759 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe6b409f650 0x7fe6b41c4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:54.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.911+0000 7fe6b96d0700 1 --2- 172.21.15.16:0/52992759 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe6b409e930 0x7fe6b41c3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:54.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.912+0000 7fe6b8ecf700 1 -- 172.21.15.16:0/52992759 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe6b409e930 msgr2=0x7fe6b41c3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:54.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.912+0000 7fe6b8ecf700 1 --2- 172.21.15.16:0/52992759 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe6b409e930 0x7fe6b41c3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:54.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.912+0000 7fe6b8ecf700 1 -- 172.21.15.16:0/52992759 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe6b409f650 msgr2=0x7fe6b41c4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:54.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.912+0000 7fe6b8ecf700 1 --2- 172.21.15.16:0/52992759 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe6b409f650 0x7fe6b41c4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:54.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.912+0000 7fe6b8ecf700 1 -- 172.21.15.16:0/52992759 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b00094a0 con 0x7fe6b409f240 2023-09-30T22:35:54.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.912+0000 7fe6b8ecf700 1 --2- 172.21.15.16:0/52992759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6b409f240 0x7fe6b41c41f0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fe6a400c700 tx=0x7fe6a400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:54.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.912+0000 7fe6a9ffb700 1 -- 172.21.15.16:0/52992759 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a4005d70 con 0x7fe6b409f240 2023-09-30T22:35:54.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.912+0000 7fe6a9ffb700 1 -- 172.21.15.16:0/52992759 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6a40070d0 con 0x7fe6b409f240 2023-09-30T22:35:54.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.912+0000 7fe6a9ffb700 1 -- 172.21.15.16:0/52992759 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a4007460 con 0x7fe6b409f240 2023-09-30T22:35:54.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.912+0000 7fe6bb934700 1 -- 172.21.15.16:0/52992759 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6b40a0190 con 0x7fe6b409f240 2023-09-30T22:35:54.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.912+0000 7fe6bb934700 1 -- 172.21.15.16:0/52992759 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6b41a4730 con 0x7fe6b409f240 2023-09-30T22:35:54.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.913+0000 7fe6a9ffb700 1 -- 172.21.15.16:0/52992759 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe6a40075c0 con 0x7fe6b409f240 2023-09-30T22:35:54.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.913+0000 7fe6bb934700 1 -- 172.21.15.16:0/52992759 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6b41a3b00 con 0x7fe6b409f240 2023-09-30T22:35:54.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.914+0000 7fe6a9ffb700 1 -- 172.21.15.16:0/52992759 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fe6a403ec10 con 0x7fe6b409f240 2023-09-30T22:35:54.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.919+0000 7fe6a9ffb700 1 -- 172.21.15.16:0/52992759 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe6a4017020 con 0x7fe6b409f240 2023-09-30T22:35:54.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.947+0000 7fe6ee45f700 1 Processor -- start 2023-09-30T22:35:54.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.947+0000 7fe6ee45f700 1 -- start start 2023-09-30T22:35:54.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.947+0000 7efc884f8700 1 Processor -- start 2023-09-30T22:35:54.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.947+0000 7efc884f8700 1 -- start start 2023-09-30T22:35:54.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.948+0000 7fe6ee45f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe6e80a0a30 0x7fe6e80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:54.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.948+0000 7fe6ee45f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe6e809d630 0x7fe6e80a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:54.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.948+0000 7fe6ee45f700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6e80a2a70 con 0x7fe6e8133930 2023-09-30T22:35:54.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.948+0000 7fe6ee45f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e80a01d0 con 0x7fe6e809d630 2023-09-30T22:35:54.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.948+0000 7fe6ee45f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e809e2b0 con 0x7fe6e80a0a30 2023-09-30T22:35:54.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.948+0000 7efc884f8700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efc801337f0 0x7efc80138880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:54.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.948+0000 7efc884f8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc80138dc0 0x7efc8013b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:54.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.948+0000 7efc884f8700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc8006b990 con 0x7efc80133420 2023-09-30T22:35:54.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.948+0000 7efc884f8700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc8009b0e0 con 0x7efc801337f0 2023-09-30T22:35:54.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.948+0000 7efc884f8700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efc80133c20 con 0x7efc80138dc0 2023-09-30T22:35:54.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.948+0000 7efc86294700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7efc80133420 0x7efc80133e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57068/0 (socket says 172.21.15.16:57068) 2023-09-30T22:35:54.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.948+0000 7efc86294700 1 -- 172.21.15.16:0/1039450255 learned_addr learned my addr 172.21.15.16:0/1039450255 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:54.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.948+0000 7fe6ec9fc700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe6e8133930 0x7fe6e8133dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57056/0 (socket says 172.21.15.16:57056) 2023-09-30T22:35:54.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.948+0000 7fe6ec9fc700 1 -- 172.21.15.16:0/3453492056 learned_addr learned my addr 172.21.15.16:0/3453492056 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:54.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.948+0000 7efc86a95700 1 --2- 172.21.15.16:0/1039450255 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc80138dc0 0x7efc8013b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:54.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.948+0000 7fe6e6ffd700 1 -- 172.21.15.16:0/3453492056 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 814191208 0 0) 0x7fe6e80a2a70 con 0x7fe6e8133930 2023-09-30T22:35:54.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.948+0000 7efc85a93700 1 --2- 172.21.15.16:0/1039450255 >> v2:172.21.15.16:3300/0 conn(0x7efc801337f0 0x7efc80138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:54.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.948+0000 7fe6e6ffd700 1 -- 172.21.15.16:0/3453492056 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6d00035b0 con 0x7fe6e8133930 2023-09-30T22:35:54.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.948+0000 7efc85292700 1 -- 172.21.15.16:0/1039450255 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1824530258 0 0) 0x7efc8006b990 con 0x7efc80133420 2023-09-30T22:35:54.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.948+0000 7efc85292700 1 -- 172.21.15.16:0/1039450255 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc6c0035b0 con 0x7efc80133420 2023-09-30T22:35:54.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.948+0000 7fe6e77fe700 1 --2- 172.21.15.16:0/3453492056 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe6e809d630 0x7fe6e80a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:54.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.949+0000 7efc85292700 1 -- 172.21.15.16:0/1039450255 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2344412886 0 0) 0x7efc6c0035b0 con 0x7efc80133420 2023-09-30T22:35:54.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.949+0000 7efc85292700 1 -- 172.21.15.16:0/1039450255 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc80138dc0 msgr2=0x7efc8013b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:54.955 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.949+0000 7efc85292700 1 --2- 172.21.15.16:0/1039450255 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc80138dc0 0x7efc8013b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:54.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.949+0000 7efc85292700 1 -- 172.21.15.16:0/1039450255 >> v2:172.21.15.16:3300/0 conn(0x7efc801337f0 msgr2=0x7efc80138880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:54.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.949+0000 7efc85292700 1 --2- 172.21.15.16:0/1039450255 >> v2:172.21.15.16:3300/0 conn(0x7efc801337f0 0x7efc80138880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:54.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.949+0000 7efc85292700 1 -- 172.21.15.16:0/1039450255 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc800a0590 con 0x7efc80133420 2023-09-30T22:35:54.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.949+0000 7efc85292700 1 -- 172.21.15.16:0/1039450255 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1074163509 0 0) 0x7efc7c001670 con 0x7efc80133420 2023-09-30T22:35:54.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.949+0000 7efc85292700 1 -- 172.21.15.16:0/1039450255 >> v1:172.21.15.16:6789/0 conn(0x7efc80133420 legacy=0x7efc80133e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:54.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.949+0000 7efc85292700 1 --2- 172.21.15.16:0/1039450255 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efc6c003a00 0x7efc6c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:54.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.949+0000 7efc85292700 1 --2- 172.21.15.16:0/1039450255 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc80138dc0 0x7efc6c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:54.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.949+0000 7efc85292700 1 --2- 172.21.15.16:0/1039450255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc6c004d90 0x7efc6c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:54.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.949+0000 7efc85292700 1 -- 172.21.15.16:0/1039450255 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc8009b0e0 con 0x7efc6c004d90 2023-09-30T22:35:54.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.949+0000 7efc85292700 1 -- 172.21.15.16:0/1039450255 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc80133c20 con 0x7efc6c003a00 2023-09-30T22:35:54.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.949+0000 7efc85292700 1 -- 172.21.15.16:0/1039450255 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efc6c00bb10 con 0x7efc80138dc0 2023-09-30T22:35:54.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.949+0000 7efc86294700 1 --2- 172.21.15.16:0/1039450255 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efc6c003a00 0x7efc6c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:54.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.949+0000 7efc86a95700 1 --2- 172.21.15.16:0/1039450255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc6c004d90 0x7efc6c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:54.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.949+0000 7fe6e6ffd700 1 -- 172.21.15.16:0/3453492056 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3378741905 0 0) 0x7fe6d00035b0 con 0x7fe6e8133930 2023-09-30T22:35:54.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.949+0000 7fe6e6ffd700 1 -- 172.21.15.16:0/3453492056 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe6e80a0a30 msgr2=0x7fe6e80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:54.957 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.949+0000 7fe6e6ffd700 1 --2- 172.21.15.16:0/3453492056 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe6e80a0a30 0x7fe6e80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:54.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.949+0000 7fe6e6ffd700 1 -- 172.21.15.16:0/3453492056 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe6e809d630 msgr2=0x7fe6e80a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:54.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.949+0000 7fe6e6ffd700 1 --2- 172.21.15.16:0/3453492056 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe6e809d630 0x7fe6e80a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:54.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.949+0000 7fe6e6ffd700 1 -- 172.21.15.16:0/3453492056 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e809dc50 con 0x7fe6e8133930 2023-09-30T22:35:54.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.949+0000 7fe6e6ffd700 1 -- 172.21.15.16:0/3453492056 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1074163509 0 0) 0x7fe6e00019f0 con 0x7fe6e8133930 2023-09-30T22:35:54.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.949+0000 7fe6e6ffd700 1 -- 172.21.15.16:0/3453492056 >> v1:172.21.15.16:6789/0 conn(0x7fe6e8133930 legacy=0x7fe6e8133dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:54.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.949+0000 7fe6e6ffd700 1 --2- 172.21.15.16:0/3453492056 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe6d0003a00 0x7fe6d0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:54.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.949+0000 7fe6e6ffd700 1 --2- 172.21.15.16:0/3453492056 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe6d0004850 0x7fe6d0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:54.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.949+0000 7fe6e6ffd700 1 --2- 172.21.15.16:0/3453492056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6d0005200 0x7fe6d000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:54.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.949+0000 7fe6e6ffd700 1 -- 172.21.15.16:0/3453492056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e80a01d0 con 0x7fe6d0005200 2023-09-30T22:35:54.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.949+0000 7fe6e6ffd700 1 -- 172.21.15.16:0/3453492056 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e809e2b0 con 0x7fe6d0004850 2023-09-30T22:35:54.958 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.949+0000 7fe6e6ffd700 1 -- 172.21.15.16:0/3453492056 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe6d000c210 con 0x7fe6d0003a00 2023-09-30T22:35:54.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.949+0000 7fe6ec9fc700 1 --2- 172.21.15.16:0/3453492056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6d0005200 0x7fe6d000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:54.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.949+0000 7fe6e7fff700 1 --2- 172.21.15.16:0/3453492056 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe6d0003a00 0x7fe6d0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:54.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.949+0000 7fe6ec9fc700 1 -- 172.21.15.16:0/3453492056 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe6d0003a00 msgr2=0x7fe6d0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:54.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.949+0000 7fe6ec9fc700 1 --2- 172.21.15.16:0/3453492056 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe6d0003a00 0x7fe6d0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:54.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.949+0000 7fe6ec9fc700 1 -- 172.21.15.16:0/3453492056 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe6d0004850 msgr2=0x7fe6d0004cc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T22:35:54.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.949+0000 7fe6ec9fc700 1 --2- 172.21.15.16:0/3453492056 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe6d0004850 0x7fe6d0004cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:54.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.949+0000 7fe6ec9fc700 1 -- 172.21.15.16:0/3453492056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6d00035b0 con 0x7fe6d0005200 2023-09-30T22:35:54.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.949+0000 7fe6e7fff700 1 --2- 172.21.15.16:0/3453492056 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe6d0003a00 0x7fe6d0003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:35:54.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.950+0000 7efc86a95700 1 -- 172.21.15.16:0/1039450255 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc80138dc0 msgr2=0x7efc6c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:54.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.950+0000 7efc86a95700 1 --2- 172.21.15.16:0/1039450255 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc80138dc0 0x7efc6c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:54.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.950+0000 7efc86a95700 1 -- 172.21.15.16:0/1039450255 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efc6c003a00 msgr2=0x7efc6c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:54.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.950+0000 7efc86a95700 1 --2- 172.21.15.16:0/1039450255 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efc6c003a00 0x7efc6c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:54.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.950+0000 7efc86a95700 1 -- 172.21.15.16:0/1039450255 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc800a0590 con 0x7efc6c004d90 2023-09-30T22:35:54.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.950+0000 7fe6ec9fc700 1 --2- 172.21.15.16:0/3453492056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6d0005200 0x7fe6d000ba10 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fe6e000a780 tx=0x7fe6d0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87e1b47a469af030 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:54.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.950+0000 7fe6e6ffd700 1 -- 172.21.15.16:0/3453492056 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e00072d0 con 0x7fe6d0005200 2023-09-30T22:35:54.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.950+0000 7fe6e6ffd700 1 -- 172.21.15.16:0/3453492056 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6e0007430 con 0x7fe6d0005200 2023-09-30T22:35:54.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.950+0000 7fe6e6ffd700 1 -- 172.21.15.16:0/3453492056 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e00077c0 con 0x7fe6d0005200 2023-09-30T22:35:54.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.950+0000 7efc86a95700 1 --2- 172.21.15.16:0/1039450255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc6c004d90 0x7efc6c005200 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7efc78007330 tx=0x7efc6c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bcb8630ae8141ec server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:54.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.950+0000 7efc85292700 1 -- 172.21.15.16:0/1039450255 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc78007e50 con 0x7efc6c004d90 2023-09-30T22:35:54.961 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.950+0000 7efc85292700 1 -- 172.21.15.16:0/1039450255 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc78014400 con 0x7efc6c004d90 2023-09-30T22:35:54.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.950+0000 7efc85292700 1 -- 172.21.15.16:0/1039450255 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc780147b0 con 0x7efc6c004d90 2023-09-30T22:35:54.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.951+0000 7fe6ee45f700 1 -- 172.21.15.16:0/3453492056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6d0005200 msgr2=0x7fe6d000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:54.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.951+0000 7fe6ee45f700 1 --2- 172.21.15.16:0/3453492056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6d0005200 0x7fe6d000ba10 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fe6e000a780 tx=0x7fe6d0005880 comp rx=0 tx=0).stop 2023-09-30T22:35:54.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.951+0000 7fe6ee45f700 1 -- 172.21.15.16:0/3453492056 shutdown_connections 2023-09-30T22:35:54.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.951+0000 7fe6ee45f700 1 --2- 172.21.15.16:0/3453492056 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe6d0004850 0x7fe6d0004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:54.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.951+0000 7fe6ee45f700 1 --2- 172.21.15.16:0/3453492056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6d0005200 0x7fe6d000ba10 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:54.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.951+0000 7fe6ee45f700 1 --2- 172.21.15.16:0/3453492056 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe6d0003a00 0x7fe6d0003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:54.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.951+0000 7fe6ee45f700 1 -- 172.21.15.16:0/3453492056 >> 172.21.15.16:0/3453492056 conn(0x7fe6e80a3050 msgr2=0x7fe6e80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:54.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.951+0000 7fe6ee45f700 1 -- 172.21.15.16:0/3453492056 shutdown_connections 2023-09-30T22:35:54.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.951+0000 7fe6ee45f700 1 -- 172.21.15.16:0/3453492056 wait complete. 2023-09-30T22:35:54.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.951+0000 7efc884f8700 1 -- 172.21.15.16:0/1039450255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc6c004d90 msgr2=0x7efc6c005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:54.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.951+0000 7efc884f8700 1 --2- 172.21.15.16:0/1039450255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc6c004d90 0x7efc6c005200 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7efc78007330 tx=0x7efc6c005880 comp rx=0 tx=0).stop 2023-09-30T22:35:54.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.951+0000 7efc86a95700 1 -- 172.21.15.16:0/1039450255 reap_dead start 2023-09-30T22:35:54.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.951+0000 7efc884f8700 1 -- 172.21.15.16:0/1039450255 shutdown_connections 2023-09-30T22:35:54.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.951+0000 7efc884f8700 1 -- 172.21.15.16:0/1039450255 >> 172.21.15.16:0/1039450255 conn(0x7efc800a4210 msgr2=0x7efc800a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:54.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.951+0000 7efc884f8700 1 -- 172.21.15.16:0/1039450255 shutdown_connections 2023-09-30T22:35:54.964 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.951+0000 7efc884f8700 1 -- 172.21.15.16:0/1039450255 wait complete. 2023-09-30T22:35:54.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.952+0000 7fe6ee45f700 1 Processor -- start 2023-09-30T22:35:54.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.953+0000 7efc884f8700 1 Processor -- start 2023-09-30T22:35:54.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.953+0000 7efc884f8700 1 -- start start 2023-09-30T22:35:54.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.953+0000 7fe6ee45f700 1 -- start start 2023-09-30T22:35:54.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.953+0000 7efc884f8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efc6c004d90 0x7efc8012ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:54.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.953+0000 7fe6ee45f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe6d0003a00 0x7fe6e811f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:54.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.953+0000 7fe6ee45f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe6d0004850 0x7fe6e811fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:54.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.953+0000 7fe6ee45f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6d0005200 0x7fe6e81c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:54.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.953+0000 7fe6ee45f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e8122350 con 0x7fe6d0005200 2023-09-30T22:35:54.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.953+0000 7fe6ee45f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e8122530 con 0x7fe6d0004850 2023-09-30T22:35:54.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.953+0000 7fe6ee45f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e81333a0 con 0x7fe6d0003a00 2023-09-30T22:35:54.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.953+0000 7efc884f8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc8012f3b0 0x7efc801c8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:54.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.953+0000 7efc884f8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc801c8770 0x7efc801c8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:54.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.953+0000 7efc884f8700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc801c9100 con 0x7efc8012f3b0 2023-09-30T22:35:54.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.953+0000 7efc884f8700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc8012ea80 con 0x7efc6c004d90 2023-09-30T22:35:54.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.953+0000 7efc884f8700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efc80134db0 con 0x7efc801c8770 2023-09-30T22:35:54.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.954+0000 7efc85a93700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc8012f3b0 0x7efc801c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:54.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.954+0000 7efc85a93700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc8012f3b0 0x7efc801c8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38884/0 (socket says 172.21.15.16:38884) 2023-09-30T22:35:54.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.954+0000 7efc85a93700 1 -- 172.21.15.16:0/2317949028 learned_addr learned my addr 172.21.15.16:0/2317949028 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:54.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.954+0000 7efc86a95700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc801c8770 0x7efc801c8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:54.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.954+0000 7fe6ec9fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6d0005200 0x7fe6e81c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:54.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.954+0000 7fe6ec9fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6d0005200 0x7fe6e81c8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38888/0 (socket says 172.21.15.16:38888) 2023-09-30T22:35:54.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.954+0000 7fe6ec9fc700 1 -- 172.21.15.16:0/1517810418 learned_addr learned my addr 172.21.15.16:0/1517810418 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:54.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.954+0000 7efc85a93700 1 -- 172.21.15.16:0/2317949028 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc801c8770 msgr2=0x7efc801c8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:54.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.954+0000 7efc85a93700 1 --2- 172.21.15.16:0/2317949028 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc801c8770 0x7efc801c8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:54.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.954+0000 7efc85a93700 1 -- 172.21.15.16:0/2317949028 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efc6c004d90 msgr2=0x7efc8012ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:54.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.954+0000 7efc85a93700 1 --2- 172.21.15.16:0/2317949028 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efc6c004d90 0x7efc8012ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:54.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.954+0000 7efc85a93700 1 -- 172.21.15.16:0/2317949028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc80135070 con 0x7efc8012f3b0 2023-09-30T22:35:54.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.954+0000 7efc86a95700 1 --2- 172.21.15.16:0/2317949028 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc801c8770 0x7efc801c8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:54.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.954+0000 7fe6ec9fc700 1 -- 172.21.15.16:0/1517810418 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe6d0003a00 msgr2=0x7fe6e811f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:54.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.954+0000 7fe6ec9fc700 1 --2- 172.21.15.16:0/1517810418 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe6d0003a00 0x7fe6e811f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:54.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.954+0000 7fe6ec9fc700 1 -- 172.21.15.16:0/1517810418 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe6d0004850 msgr2=0x7fe6e811fa00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:35:54.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.954+0000 7fe6ec9fc700 1 --2- 172.21.15.16:0/1517810418 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe6d0004850 0x7fe6e811fa00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:54.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.954+0000 7fe6ec9fc700 1 -- 172.21.15.16:0/1517810418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e80a4030 con 0x7fe6d0005200 2023-09-30T22:35:54.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.954+0000 7efc85a93700 1 --2- 172.21.15.16:0/2317949028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc8012f3b0 0x7efc801c8230 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7efc7000e810 tx=0x7efc7000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:54.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.954+0000 7fe6ec9fc700 1 --2- 172.21.15.16:0/1517810418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6d0005200 0x7fe6e81c8020 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fe6e0002230 tx=0x7fe6e809e5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:54.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.954+0000 7efc76ffd700 1 -- 172.21.15.16:0/2317949028 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc70013680 con 0x7efc8012f3b0 2023-09-30T22:35:54.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.955+0000 7efc76ffd700 1 -- 172.21.15.16:0/2317949028 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc700137e0 con 0x7efc8012f3b0 2023-09-30T22:35:54.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.955+0000 7efc76ffd700 1 -- 172.21.15.16:0/2317949028 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc70013b90 con 0x7efc8012f3b0 2023-09-30T22:35:54.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.955+0000 7fe6e4ff9700 1 -- 172.21.15.16:0/1517810418 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e001f430 con 0x7fe6d0005200 2023-09-30T22:35:54.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.955+0000 7efc884f8700 1 -- 172.21.15.16:0/2317949028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc80130be0 con 0x7efc8012f3b0 2023-09-30T22:35:54.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.955+0000 7fe6e4ff9700 1 -- 172.21.15.16:0/1517810418 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6e001f590 con 0x7fe6d0005200 2023-09-30T22:35:54.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.955+0000 7fe6ee45f700 1 -- 172.21.15.16:0/1517810418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6e811ef20 con 0x7fe6d0005200 2023-09-30T22:35:54.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.955+0000 7fe6e4ff9700 1 -- 172.21.15.16:0/1517810418 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e001f920 con 0x7fe6d0005200 2023-09-30T22:35:54.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.955+0000 7fe6ee45f700 1 -- 172.21.15.16:0/1517810418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6e8120e10 con 0x7fe6d0005200 2023-09-30T22:35:54.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.955+0000 7efc884f8700 1 -- 172.21.15.16:0/2317949028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc80135da0 con 0x7efc8012f3b0 2023-09-30T22:35:54.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.956+0000 7fe6ee45f700 1 -- 172.21.15.16:0/1517810418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6e8120680 con 0x7fe6d0005200 2023-09-30T22:35:54.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.956+0000 7fe6e4ff9700 1 -- 172.21.15.16:0/1517810418 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe6e001fa80 con 0x7fe6d0005200 2023-09-30T22:35:54.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.956+0000 7efc884f8700 1 -- 172.21.15.16:0/2317949028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc80132f50 con 0x7efc8012f3b0 2023-09-30T22:35:54.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.956+0000 7fe6e4ff9700 1 -- 172.21.15.16:0/1517810418 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fe6e0041540 con 0x7fe6d0005200 2023-09-30T22:35:54.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.961+0000 7fe6e4ff9700 1 -- 172.21.15.16:0/1517810418 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe6e001fd30 con 0x7fe6d0005200 2023-09-30T22:35:54.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.962+0000 7efc76ffd700 1 -- 172.21.15.16:0/2317949028 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7efc70013cf0 con 0x7efc8012f3b0 2023-09-30T22:35:54.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.962+0000 7efc76ffd700 1 -- 172.21.15.16:0/2317949028 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7efc70040090 con 0x7efc8012f3b0 2023-09-30T22:35:54.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:54.966+0000 7efc76ffd700 1 -- 172.21.15.16:0/2317949028 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7efc7001a360 con 0x7efc8012f3b0 2023-09-30T22:35:55.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.133+0000 7fe6bb934700 1 -- 172.21.15.16:0/52992759 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe6b4074ea0 con 0x7fe6b409f240 2023-09-30T22:35:55.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.134+0000 7fe6a9ffb700 1 -- 172.21.15.16:0/52992759 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe6a400e280 con 0x7fe6b409f240 2023-09-30T22:35:55.134 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:35:55.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.136+0000 7fe6bb934700 1 -- 172.21.15.16:0/52992759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6b409f240 msgr2=0x7fe6b41c41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:55.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.136+0000 7fe6bb934700 1 --2- 172.21.15.16:0/52992759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6b409f240 0x7fe6b41c41f0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fe6a400c700 tx=0x7fe6a400ca10 comp rx=0 tx=0).stop 2023-09-30T22:35:55.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.136+0000 7fe6bb934700 1 -- 172.21.15.16:0/52992759 shutdown_connections 2023-09-30T22:35:55.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.136+0000 7fe6bb934700 1 --2- 172.21.15.16:0/52992759 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe6b409f650 0x7fe6b41c4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:55.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.136+0000 7fe6bb934700 1 --2- 172.21.15.16:0/52992759 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe6b409e930 0x7fe6b41c3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:55.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.136+0000 7fe6bb934700 1 --2- 172.21.15.16:0/52992759 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6b409f240 0x7fe6b41c41f0 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:55.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.136+0000 7fe6bb934700 1 -- 172.21.15.16:0/52992759 >> 172.21.15.16:0/52992759 conn(0x7fe6b40a4210 msgr2=0x7fe6b406c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:55.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.136+0000 7fe6bb934700 1 -- 172.21.15.16:0/52992759 shutdown_connections 2023-09-30T22:35:55.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.136+0000 7fe6bb934700 1 -- 172.21.15.16:0/52992759 wait complete. 2023-09-30T22:35:55.151 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:35:55.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.168+0000 7fe6ee45f700 1 -- 172.21.15.16:0/1517810418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe6e8074770 con 0x7fe6d0005200 2023-09-30T22:35:55.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.168+0000 7fe6e4ff9700 1 -- 172.21.15.16:0/1517810418 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe6e0018020 con 0x7fe6d0005200 2023-09-30T22:35:55.169 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:35:55.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.171+0000 7fe6ee45f700 1 -- 172.21.15.16:0/1517810418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6d0005200 msgr2=0x7fe6e81c8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:55.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.171+0000 7fe6ee45f700 1 --2- 172.21.15.16:0/1517810418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6d0005200 0x7fe6e81c8020 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fe6e0002230 tx=0x7fe6e809e5d0 comp rx=0 tx=0).stop 2023-09-30T22:35:55.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.171+0000 7fe6ee45f700 1 -- 172.21.15.16:0/1517810418 shutdown_connections 2023-09-30T22:35:55.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.171+0000 7fe6ee45f700 1 --2- 172.21.15.16:0/1517810418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6d0005200 0x7fe6e81c8020 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:55.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.171+0000 7fe6ee45f700 1 --2- 172.21.15.16:0/1517810418 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe6d0003a00 0x7fe6e811f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:55.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.171+0000 7fe6ee45f700 1 --2- 172.21.15.16:0/1517810418 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe6d0004850 0x7fe6e811fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:55.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.171+0000 7fe6ee45f700 1 -- 172.21.15.16:0/1517810418 >> 172.21.15.16:0/1517810418 conn(0x7fe6e80a3050 msgr2=0x7fe6e809ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:55.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.171+0000 7fe6ee45f700 1 -- 172.21.15.16:0/1517810418 shutdown_connections 2023-09-30T22:35:55.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.171+0000 7fe6ee45f700 1 -- 172.21.15.16:0/1517810418 wait complete. 2023-09-30T22:35:55.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.171+0000 7efc884f8700 1 -- 172.21.15.16:0/2317949028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7efc80135260 con 0x7efc8012f3b0 2023-09-30T22:35:55.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.172+0000 7efc76ffd700 1 -- 172.21.15.16:0/2317949028 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7efc70021070 con 0x7efc8012f3b0 2023-09-30T22:35:55.173 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:35:55.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.174+0000 7efc884f8700 1 -- 172.21.15.16:0/2317949028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc8012f3b0 msgr2=0x7efc801c8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:55.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.174+0000 7efc884f8700 1 --2- 172.21.15.16:0/2317949028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc8012f3b0 0x7efc801c8230 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7efc7000e810 tx=0x7efc7000eb20 comp rx=0 tx=0).stop 2023-09-30T22:35:55.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.174+0000 7efc884f8700 1 -- 172.21.15.16:0/2317949028 shutdown_connections 2023-09-30T22:35:55.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.174+0000 7efc884f8700 1 --2- 172.21.15.16:0/2317949028 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efc6c004d90 0x7efc8012ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:55.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.174+0000 7efc884f8700 1 --2- 172.21.15.16:0/2317949028 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc801c8770 0x7efc801c8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:55.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.174+0000 7efc884f8700 1 --2- 172.21.15.16:0/2317949028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc8012f3b0 0x7efc801c8230 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:55.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.174+0000 7efc884f8700 1 -- 172.21.15.16:0/2317949028 >> 172.21.15.16:0/2317949028 conn(0x7efc800a4210 msgr2=0x7efc800a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:55.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.174+0000 7efc884f8700 1 -- 172.21.15.16:0/2317949028 shutdown_connections 2023-09-30T22:35:55.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.174+0000 7efc884f8700 1 -- 172.21.15.16:0/2317949028 wait complete. 2023-09-30T22:35:55.187 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:35:55.189 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:35:55.272 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:35:55.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.415+0000 7fd5ff707700 1 Processor -- start 2023-09-30T22:35:55.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.416+0000 7fd5ff707700 1 -- start start 2023-09-30T22:35:55.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.416+0000 7fd5ff707700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd5f809d6f0 0x7fd5f809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:55.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.416+0000 7fd5ff707700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd5f809e000 0x7fd5f8130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:55.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.416+0000 7fd5ff707700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd5f809e410 0x7fd5f8130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:55.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.416+0000 7fd5ff707700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5f809f520 con 0x7fd5f809e000 2023-09-30T22:35:55.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.416+0000 7fd5ff707700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f809f130 con 0x7fd5f809d6f0 2023-09-30T22:35:55.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.416+0000 7fd5ff707700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f809e930 con 0x7fd5f809e410 2023-09-30T22:35:55.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.416+0000 7fd5fdca4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd5f809e410 0x7fd5f8130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:55.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.416+0000 7fd5fd4a3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd5f809d6f0 0x7fd5f809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:55.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.416+0000 7fd5fcca2700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd5f809e000 0x7fd5f8130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:55.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.416+0000 7fd5fdca4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd5f809e410 0x7fd5f8130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35270/0 (socket says 172.21.15.16:35270) 2023-09-30T22:35:55.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.416+0000 7fd5fdca4700 1 -- 172.21.15.16:0/3753126846 learned_addr learned my addr 172.21.15.16:0/3753126846 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:55.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.416+0000 7fd5fcca2700 1 --2- 172.21.15.16:0/3753126846 >> v2:172.21.15.16:3300/0 conn(0x7fd5f809e000 0x7fd5f8130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38898/0 (socket says 172.21.15.16:38898) 2023-09-30T22:35:55.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.417+0000 7fd5fd4a3700 1 -- 172.21.15.16:0/3753126846 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd5f809e410 msgr2=0x7fd5f8130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:55.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.417+0000 7fd5fd4a3700 1 --2- 172.21.15.16:0/3753126846 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd5f809e410 0x7fd5f8130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:55.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.417+0000 7fd5fd4a3700 1 -- 172.21.15.16:0/3753126846 >> v2:172.21.15.16:3300/0 conn(0x7fd5f809e000 msgr2=0x7fd5f8130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:55.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.417+0000 7fd5fd4a3700 1 --2- 172.21.15.16:0/3753126846 >> v2:172.21.15.16:3300/0 conn(0x7fd5f809e000 0x7fd5f8130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:55.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.417+0000 7fd5fd4a3700 1 -- 172.21.15.16:0/3753126846 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f809b2d0 con 0x7fd5f809d6f0 2023-09-30T22:35:55.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.417+0000 7fd5fdca4700 1 --2- 172.21.15.16:0/3753126846 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd5f809e410 0x7fd5f8130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:55.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.417+0000 7fd5fd4a3700 1 --2- 172.21.15.16:0/3753126846 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd5f809d6f0 0x7fd5f809dac0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fd5f400bae0 tx=0x7fd5f400bdf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e233b745d9c7fcd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:55.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.417+0000 7fd5ebfff700 1 -- 172.21.15.16:0/3753126846 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f400dc70 con 0x7fd5f809d6f0 2023-09-30T22:35:55.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.417+0000 7fd5ebfff700 1 -- 172.21.15.16:0/3753126846 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5f400ddd0 con 0x7fd5f809d6f0 2023-09-30T22:35:55.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.417+0000 7fd5ebfff700 1 -- 172.21.15.16:0/3753126846 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f4010690 con 0x7fd5f809d6f0 2023-09-30T22:35:55.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.418+0000 7fd5ff707700 1 -- 172.21.15.16:0/3753126846 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd5f809d6f0 msgr2=0x7fd5f809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:55.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.418+0000 7fd5ff707700 1 --2- 172.21.15.16:0/3753126846 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd5f809d6f0 0x7fd5f809dac0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fd5f400bae0 tx=0x7fd5f400bdf0 comp rx=0 tx=0).stop 2023-09-30T22:35:55.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.418+0000 7fd5ff707700 1 -- 172.21.15.16:0/3753126846 shutdown_connections 2023-09-30T22:35:55.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.418+0000 7fd5ff707700 1 --2- 172.21.15.16:0/3753126846 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd5f809d6f0 0x7fd5f809dac0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:55.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.418+0000 7fd5ff707700 1 --2- 172.21.15.16:0/3753126846 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd5f809e410 0x7fd5f8130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:55.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.418+0000 7fd5ff707700 1 --2- 172.21.15.16:0/3753126846 >> v2:172.21.15.16:3300/0 conn(0x7fd5f809e000 0x7fd5f8130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:55.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.418+0000 7fd5ff707700 1 -- 172.21.15.16:0/3753126846 >> 172.21.15.16:0/3753126846 conn(0x7fd5f8066f50 msgr2=0x7fd5f809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:55.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.418+0000 7fd5ff707700 1 -- 172.21.15.16:0/3753126846 shutdown_connections 2023-09-30T22:35:55.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.418+0000 7fd5ff707700 1 -- 172.21.15.16:0/3753126846 wait complete. 2023-09-30T22:35:55.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.419+0000 7fd5ff707700 1 Processor -- start 2023-09-30T22:35:55.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.419+0000 7fd5ff707700 1 -- start start 2023-09-30T22:35:55.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.419+0000 7fd5ff707700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5f809d6f0 0x7fd5f811e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:55.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.419+0000 7fd5ff707700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd5f809e000 0x7fd5f811e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:55.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.419+0000 7fd5ff707700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd5f809e410 0x7fd5f811eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:55.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.419+0000 7fd5ff707700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f8120540 con 0x7fd5f809d6f0 2023-09-30T22:35:55.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.419+0000 7fd5ff707700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f809c700 con 0x7fd5f809e000 2023-09-30T22:35:55.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.419+0000 7fd5ff707700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f80a18c0 con 0x7fd5f809e410 2023-09-30T22:35:55.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.419+0000 7fd5fcca2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd5f809e000 0x7fd5f811e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:55.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.419+0000 7fd5fdca4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd5f809e410 0x7fd5f811eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:55.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.419+0000 7fd5fcca2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd5f809e000 0x7fd5f811e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47562/0 (socket says 172.21.15.16:47562) 2023-09-30T22:35:55.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.419+0000 7fd5fcca2700 1 -- 172.21.15.16:0/3529163867 learned_addr learned my addr 172.21.15.16:0/3529163867 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:55.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.419+0000 7fd5fd4a3700 1 --2- 172.21.15.16:0/3529163867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5f809d6f0 0x7fd5f811e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:55.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.420+0000 7fd5fdca4700 1 -- 172.21.15.16:0/3529163867 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd5f809e000 msgr2=0x7fd5f811e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:55.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.420+0000 7fd5fdca4700 1 --2- 172.21.15.16:0/3529163867 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd5f809e000 0x7fd5f811e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:55.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.420+0000 7fd5fdca4700 1 -- 172.21.15.16:0/3529163867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5f809d6f0 msgr2=0x7fd5f811e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:55.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.420+0000 7fd5fdca4700 1 --2- 172.21.15.16:0/3529163867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5f809d6f0 0x7fd5f811e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:55.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.420+0000 7fd5fdca4700 1 -- 172.21.15.16:0/3529163867 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f0009b30 con 0x7fd5f809e410 2023-09-30T22:35:55.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.420+0000 7fd5fcca2700 1 --2- 172.21.15.16:0/3529163867 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd5f809e000 0x7fd5f811e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:55.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.420+0000 7fd5fd4a3700 1 --2- 172.21.15.16:0/3529163867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5f809d6f0 0x7fd5f811e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:55.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.420+0000 7fd5fdca4700 1 --2- 172.21.15.16:0/3529163867 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd5f809e410 0x7fd5f811eae0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fd5f000efc0 tx=0x7fd5f000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:55.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.420+0000 7fd5e9ffb700 1 -- 172.21.15.16:0/3529163867 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f0018070 con 0x7fd5f809e410 2023-09-30T22:35:55.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.420+0000 7fd5e9ffb700 1 -- 172.21.15.16:0/3529163867 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5f000f040 con 0x7fd5f809e410 2023-09-30T22:35:55.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.420+0000 7fd5ff707700 1 -- 172.21.15.16:0/3529163867 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5f400b740 con 0x7fd5f809e410 2023-09-30T22:35:55.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.420+0000 7fd5e9ffb700 1 -- 172.21.15.16:0/3529163867 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f0003c60 con 0x7fd5f809e410 2023-09-30T22:35:55.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.420+0000 7fd5ff707700 1 -- 172.21.15.16:0/3529163867 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5f81201e0 con 0x7fd5f809e410 2023-09-30T22:35:55.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.421+0000 7fd5e9ffb700 1 -- 172.21.15.16:0/3529163867 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd5f000c880 con 0x7fd5f809e410 2023-09-30T22:35:55.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.421+0000 7fd5e9ffb700 1 -- 172.21.15.16:0/3529163867 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd5f003f750 con 0x7fd5f809e410 2023-09-30T22:35:55.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.422+0000 7fd5ff707700 1 -- 172.21.15.16:0/3529163867 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5f8074ea0 con 0x7fd5f809e410 2023-09-30T22:35:55.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.427+0000 7fd5e9ffb700 1 -- 172.21.15.16:0/3529163867 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd5f0003dc0 con 0x7fd5f809e410 2023-09-30T22:35:55.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.634+0000 7fd5ff707700 1 -- 172.21.15.16:0/3529163867 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd5f806c360 con 0x7fd5f809e410 2023-09-30T22:35:55.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.634+0000 7fd5e9ffb700 1 -- 172.21.15.16:0/3529163867 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd5f001cbb0 con 0x7fd5f809e410 2023-09-30T22:35:55.635 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:35:55.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.636+0000 7fd5ff707700 1 -- 172.21.15.16:0/3529163867 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd5f809e410 msgr2=0x7fd5f811eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:55.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.636+0000 7fd5ff707700 1 --2- 172.21.15.16:0/3529163867 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd5f809e410 0x7fd5f811eae0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fd5f000efc0 tx=0x7fd5f000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:35:55.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.636+0000 7fd5ff707700 1 -- 172.21.15.16:0/3529163867 shutdown_connections 2023-09-30T22:35:55.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.636+0000 7fd5ff707700 1 --2- 172.21.15.16:0/3529163867 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd5f809e410 0x7fd5f811eae0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:55.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.636+0000 7fd5ff707700 1 --2- 172.21.15.16:0/3529163867 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd5f809d6f0 0x7fd5f811e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:55.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.636+0000 7fd5ff707700 1 --2- 172.21.15.16:0/3529163867 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd5f809e000 0x7fd5f811e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:55.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.636+0000 7fd5ff707700 1 -- 172.21.15.16:0/3529163867 >> 172.21.15.16:0/3529163867 conn(0x7fd5f8066f50 msgr2=0x7fd5f806a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:55.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.637+0000 7fd5ff707700 1 -- 172.21.15.16:0/3529163867 shutdown_connections 2023-09-30T22:35:55.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:55.637+0000 7fd5ff707700 1 -- 172.21.15.16:0/3529163867 wait complete. 2023-09-30T22:35:55.651 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:35:56.152 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:35:56.188 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:35:56.190 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:35:56.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.305+0000 7f93aa3a7700 1 Processor -- start 2023-09-30T22:35:56.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.306+0000 7f93aa3a7700 1 -- start start 2023-09-30T22:35:56.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.306+0000 7f93aa3a7700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f93a409f410 0x7f93a4136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:56.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.306+0000 7f93aa3a7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f93a4136fe0 0x7f93a41393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:56.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.306+0000 7f93aa3a7700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93a4070940 con 0x7f93a409e930 2023-09-30T22:35:56.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.306+0000 7f93aa3a7700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93a406fd60 con 0x7f93a409f410 2023-09-30T22:35:56.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.306+0000 7f93aa3a7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f93a406fa80 con 0x7f93a4136fe0 2023-09-30T22:35:56.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.307+0000 7f93a3fff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f93a409e930 0x7f93a409ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57078/0 (socket says 172.21.15.16:57078) 2023-09-30T22:35:56.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.307+0000 7f93a3fff700 1 -- 172.21.15.16:0/2669326501 learned_addr learned my addr 172.21.15.16:0/2669326501 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:56.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.307+0000 7f93a8944700 1 --2- 172.21.15.16:0/2669326501 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f93a4136fe0 0x7f93a41393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:56.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.307+0000 7f93a37fe700 1 --2- 172.21.15.16:0/2669326501 >> v2:172.21.15.16:3300/0 conn(0x7f93a409f410 0x7f93a4136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:56.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.307+0000 7f93a2ffd700 1 -- 172.21.15.16:0/2669326501 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2626222117 0 0) 0x7f93a4070940 con 0x7f93a409e930 2023-09-30T22:35:56.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.307+0000 7f93a2ffd700 1 -- 172.21.15.16:0/2669326501 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f938c0035b0 con 0x7f93a409e930 2023-09-30T22:35:56.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.307+0000 7f93a37fe700 1 -- 172.21.15.16:0/2669326501 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f93a4136fe0 msgr2=0x7f93a41393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:56.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.307+0000 7f93a37fe700 1 --2- 172.21.15.16:0/2669326501 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f93a4136fe0 0x7f93a41393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:56.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.307+0000 7f93a37fe700 1 -- 172.21.15.16:0/2669326501 >> v1:172.21.15.16:6789/0 conn(0x7f93a409e930 legacy=0x7f93a409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:56.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.307+0000 7f93a37fe700 1 -- 172.21.15.16:0/2669326501 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93a409fb60 con 0x7f93a409f410 2023-09-30T22:35:56.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.307+0000 7f93a8944700 1 --2- 172.21.15.16:0/2669326501 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f93a4136fe0 0x7f93a41393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:56.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.307+0000 7f93a37fe700 1 --2- 172.21.15.16:0/2669326501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93a409f410 0x7f93a4136aa0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f9398009840 tx=0x7f9398009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97b3d72ae1199a95 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:56.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.308+0000 7f93a2ffd700 1 -- 172.21.15.16:0/2669326501 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9398011830 con 0x7f93a409f410 2023-09-30T22:35:56.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.308+0000 7f93a2ffd700 1 -- 172.21.15.16:0/2669326501 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9398011990 con 0x7f93a409f410 2023-09-30T22:35:56.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.308+0000 7f93a2ffd700 1 -- 172.21.15.16:0/2669326501 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9398011d20 con 0x7f93a409f410 2023-09-30T22:35:56.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.309+0000 7f93aa3a7700 1 -- 172.21.15.16:0/2669326501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93a409f410 msgr2=0x7f93a4136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:56.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.309+0000 7f93aa3a7700 1 --2- 172.21.15.16:0/2669326501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93a409f410 0x7f93a4136aa0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f9398009840 tx=0x7f9398009b50 comp rx=0 tx=0).stop 2023-09-30T22:35:56.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.309+0000 7f93aa3a7700 1 -- 172.21.15.16:0/2669326501 shutdown_connections 2023-09-30T22:35:56.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.309+0000 7f93aa3a7700 1 --2- 172.21.15.16:0/2669326501 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f93a4136fe0 0x7f93a41393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:56.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.309+0000 7f93aa3a7700 1 --2- 172.21.15.16:0/2669326501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93a409f410 0x7f93a4136aa0 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:56.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.309+0000 7f93aa3a7700 1 -- 172.21.15.16:0/2669326501 >> 172.21.15.16:0/2669326501 conn(0x7f93a40a4210 msgr2=0x7f93a40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:56.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.309+0000 7f93aa3a7700 1 -- 172.21.15.16:0/2669326501 shutdown_connections 2023-09-30T22:35:56.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.309+0000 7f93aa3a7700 1 -- 172.21.15.16:0/2669326501 wait complete. 2023-09-30T22:35:56.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.310+0000 7f93aa3a7700 1 Processor -- start 2023-09-30T22:35:56.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.310+0000 7f93aa3a7700 1 -- start start 2023-09-30T22:35:56.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.311+0000 7f93aa3a7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f93a409e930 0x7f93a411f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:56.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.311+0000 7f93aa3a7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93a409f410 0x7f93a411fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:56.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.311+0000 7f93aa3a7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f93a4136fe0 0x7f93a411ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:56.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.311+0000 7f93aa3a7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a4134e40 con 0x7f93a409f410 2023-09-30T22:35:56.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.311+0000 7f93aa3a7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93a4135420 con 0x7f93a409e930 2023-09-30T22:35:56.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.311+0000 7f93aa3a7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f93a41219a0 con 0x7f93a4136fe0 2023-09-30T22:35:56.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.311+0000 7f93a37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93a409f410 0x7f93a411fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:56.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.311+0000 7f93a37fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93a409f410 0x7f93a411fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38908/0 (socket says 172.21.15.16:38908) 2023-09-30T22:35:56.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.311+0000 7f93a3fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f93a409e930 0x7f93a411f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:56.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.311+0000 7f93a37fe700 1 -- 172.21.15.16:0/544110465 learned_addr learned my addr 172.21.15.16:0/544110465 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:56.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.312+0000 7f93a8944700 1 --2- 172.21.15.16:0/544110465 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f93a4136fe0 0x7f93a411ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:56.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.312+0000 7f93a37fe700 1 -- 172.21.15.16:0/544110465 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f93a4136fe0 msgr2=0x7f93a411ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:56.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.312+0000 7f93a37fe700 1 --2- 172.21.15.16:0/544110465 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f93a4136fe0 0x7f93a411ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:56.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.312+0000 7f93a37fe700 1 -- 172.21.15.16:0/544110465 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f93a409e930 msgr2=0x7f93a411f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:56.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.312+0000 7f93a37fe700 1 --2- 172.21.15.16:0/544110465 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f93a409e930 0x7f93a411f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:56.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.312+0000 7f93a37fe700 1 -- 172.21.15.16:0/544110465 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f939c009a70 con 0x7f93a409f410 2023-09-30T22:35:56.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.312+0000 7f93a37fe700 1 --2- 172.21.15.16:0/544110465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93a409f410 0x7f93a411fa70 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f9398000c00 tx=0x7f9398004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:56.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.312+0000 7f93a0ff9700 1 -- 172.21.15.16:0/544110465 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9398004550 con 0x7f93a409f410 2023-09-30T22:35:56.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.312+0000 7f93a0ff9700 1 -- 172.21.15.16:0/544110465 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f939801f070 con 0x7f93a409f410 2023-09-30T22:35:56.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.312+0000 7f93aa3a7700 1 -- 172.21.15.16:0/544110465 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93980094a0 con 0x7f93a409f410 2023-09-30T22:35:56.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.312+0000 7f93a0ff9700 1 -- 172.21.15.16:0/544110465 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93980038e0 con 0x7f93a409f410 2023-09-30T22:35:56.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.312+0000 7f93aa3a7700 1 -- 172.21.15.16:0/544110465 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93a40a32f0 con 0x7f93a409f410 2023-09-30T22:35:56.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.313+0000 7f93a0ff9700 1 -- 172.21.15.16:0/544110465 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9398003a40 con 0x7f93a409f410 2023-09-30T22:35:56.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.313+0000 7f93a0ff9700 1 -- 172.21.15.16:0/544110465 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f9398021070 con 0x7f93a409f410 2023-09-30T22:35:56.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.313+0000 7f93aa3a7700 1 -- 172.21.15.16:0/544110465 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93a4074ea0 con 0x7f93a409f410 2023-09-30T22:35:56.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.318+0000 7f93a0ff9700 1 -- 172.21.15.16:0/544110465 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9398015020 con 0x7f93a409f410 2023-09-30T22:35:56.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.345+0000 7f17bae48700 1 Processor -- start 2023-09-30T22:35:56.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.345+0000 7f17bae48700 1 -- start start 2023-09-30T22:35:56.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.346+0000 7f17bae48700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f17b406a770 0x7f17b406ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:56.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.346+0000 7f17bae48700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f17b406b080 0x7f17b4131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:56.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.346+0000 7f17bae48700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17b409ece0 con 0x7f17b4131b40 2023-09-30T22:35:56.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.346+0000 7f17bae48700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17b409e760 con 0x7f17b406a770 2023-09-30T22:35:56.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.346+0000 7f17bae48700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f17b409e040 con 0x7f17b406b080 2023-09-30T22:35:56.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.346+0000 7f17b93e5700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f17b4131b40 0x7f17b4138850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57090/0 (socket says 172.21.15.16:57090) 2023-09-30T22:35:56.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.346+0000 7f17b93e5700 1 -- 172.21.15.16:0/3218835806 learned_addr learned my addr 172.21.15.16:0/3218835806 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:56.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.346+0000 7f17b8be4700 1 --2- 172.21.15.16:0/3218835806 >> v2:172.21.15.16:3300/0 conn(0x7f17b406a770 0x7f17b406ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:56.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.346+0000 7f17b3fff700 1 --2- 172.21.15.16:0/3218835806 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f17b406b080 0x7f17b4131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:56.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.347+0000 7f17b3fff700 1 -- 172.21.15.16:0/3218835806 >> v2:172.21.15.16:3300/0 conn(0x7f17b406a770 msgr2=0x7f17b406ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:56.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.347+0000 7f17b3fff700 1 --2- 172.21.15.16:0/3218835806 >> v2:172.21.15.16:3300/0 conn(0x7f17b406a770 0x7f17b406ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:56.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.347+0000 7f17b3fff700 1 -- 172.21.15.16:0/3218835806 >> v1:172.21.15.16:6789/0 conn(0x7f17b4131b40 legacy=0x7f17b4138850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:56.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.347+0000 7f17b3fff700 1 -- 172.21.15.16:0/3218835806 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b409f630 con 0x7f17b406b080 2023-09-30T22:35:56.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.347+0000 7f17b37fe700 1 -- 172.21.15.16:0/3218835806 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1637694271 0 0) 0x7f17b409ece0 con 0x7f17b4131b40 2023-09-30T22:35:56.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.347+0000 7f17b3fff700 1 --2- 172.21.15.16:0/3218835806 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f17b406b080 0x7f17b4131600 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f17a0009840 tx=0x7f17a0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70841b8c204a5620 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:56.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.347+0000 7fefc45c0700 1 Processor -- start 2023-09-30T22:35:56.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.347+0000 7f17b37fe700 1 -- 172.21.15.16:0/3218835806 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a0011830 con 0x7f17b406b080 2023-09-30T22:35:56.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.347+0000 7fefc45c0700 1 -- start start 2023-09-30T22:35:56.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.347+0000 7f17b37fe700 1 -- 172.21.15.16:0/3218835806 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17a0011990 con 0x7f17b406b080 2023-09-30T22:35:56.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.347+0000 7f17b37fe700 1 -- 172.21.15.16:0/3218835806 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a0011d20 con 0x7f17b406b080 2023-09-30T22:35:56.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.348+0000 7fefc45c0700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fefbc09d6f0 0x7fefbc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:56.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.348+0000 7fefc45c0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fefbc09e410 0x7fefbc130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:56.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.348+0000 7fefc45c0700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefbc09f130 con 0x7fefbc09e000 2023-09-30T22:35:56.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.348+0000 7fefc45c0700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefbc09f520 con 0x7fefbc09d6f0 2023-09-30T22:35:56.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.348+0000 7fefc45c0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefbc09e930 con 0x7fefbc09e410 2023-09-30T22:35:56.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.348+0000 7f17bae48700 1 -- 172.21.15.16:0/3218835806 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f17b406b080 msgr2=0x7f17b4131600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:56.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.348+0000 7f17bae48700 1 --2- 172.21.15.16:0/3218835806 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f17b406b080 0x7f17b4131600 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f17a0009840 tx=0x7f17a0009b50 comp rx=0 tx=0).stop 2023-09-30T22:35:56.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.348+0000 7f17bae48700 1 -- 172.21.15.16:0/3218835806 shutdown_connections 2023-09-30T22:35:56.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.348+0000 7f17bae48700 1 --2- 172.21.15.16:0/3218835806 >> v2:172.21.15.16:3300/0 conn(0x7f17b406a770 0x7f17b406ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:56.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.348+0000 7f17bae48700 1 --2- 172.21.15.16:0/3218835806 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f17b406b080 0x7f17b4131600 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:56.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.348+0000 7f17bae48700 1 -- 172.21.15.16:0/3218835806 >> 172.21.15.16:0/3218835806 conn(0x7f17b4066f50 msgr2=0x7f17b409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:56.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.348+0000 7fefc235c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fefbc09d6f0 0x7fefbc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:56.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.348+0000 7fefc235c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fefbc09d6f0 0x7fefbc09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38920/0 (socket says 172.21.15.16:38920) 2023-09-30T22:35:56.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.348+0000 7fefc235c700 1 -- 172.21.15.16:0/4056732432 learned_addr learned my addr 172.21.15.16:0/4056732432 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:56.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.348+0000 7fefc2b5d700 1 --2- 172.21.15.16:0/4056732432 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fefbc09e410 0x7fefbc130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:56.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.348+0000 7f17bae48700 1 -- 172.21.15.16:0/3218835806 shutdown_connections 2023-09-30T22:35:56.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.348+0000 7fefc135a700 1 -- 172.21.15.16:0/4056732432 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3098709013 0 0) 0x7fefbc09f130 con 0x7fefbc09e000 2023-09-30T22:35:56.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.348+0000 7fefc135a700 1 -- 172.21.15.16:0/4056732432 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fefa80035b0 con 0x7fefbc09e000 2023-09-30T22:35:56.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.348+0000 7f17bae48700 1 -- 172.21.15.16:0/3218835806 wait complete. 2023-09-30T22:35:56.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.348+0000 7fefc235c700 1 -- 172.21.15.16:0/4056732432 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fefbc09e410 msgr2=0x7fefbc130d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:56.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.348+0000 7fefc235c700 1 --2- 172.21.15.16:0/4056732432 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fefbc09e410 0x7fefbc130d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:56.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.348+0000 7fefc235c700 1 -- 172.21.15.16:0/4056732432 >> v1:172.21.15.16:6789/0 conn(0x7fefbc09e000 legacy=0x7fefbc130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:56.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.348+0000 7fefc235c700 1 -- 172.21.15.16:0/4056732432 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefbc09b2d0 con 0x7fefbc09d6f0 2023-09-30T22:35:56.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.349+0000 7fefc235c700 1 --2- 172.21.15.16:0/4056732432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefbc09d6f0 0x7fefbc09dac0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fefb80098f0 tx=0x7fefb8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b60f1f92dd8a216e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:56.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.349+0000 7fefc135a700 1 -- 172.21.15.16:0/4056732432 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb8011830 con 0x7fefbc09d6f0 2023-09-30T22:35:56.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.349+0000 7fefc135a700 1 -- 172.21.15.16:0/4056732432 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefb8011990 con 0x7fefbc09d6f0 2023-09-30T22:35:56.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.349+0000 7fefc135a700 1 -- 172.21.15.16:0/4056732432 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefb8011d20 con 0x7fefbc09d6f0 2023-09-30T22:35:56.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.350+0000 7fefc45c0700 1 -- 172.21.15.16:0/4056732432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefbc09d6f0 msgr2=0x7fefbc09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:56.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.350+0000 7fefc45c0700 1 --2- 172.21.15.16:0/4056732432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefbc09d6f0 0x7fefbc09dac0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fefb80098f0 tx=0x7fefb8009c00 comp rx=0 tx=0).stop 2023-09-30T22:35:56.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.350+0000 7fefc45c0700 1 -- 172.21.15.16:0/4056732432 shutdown_connections 2023-09-30T22:35:56.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.350+0000 7fefc45c0700 1 --2- 172.21.15.16:0/4056732432 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fefbc09e410 0x7fefbc130d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:56.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.350+0000 7fefc45c0700 1 --2- 172.21.15.16:0/4056732432 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefbc09d6f0 0x7fefbc09dac0 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:56.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.350+0000 7fefc45c0700 1 -- 172.21.15.16:0/4056732432 >> 172.21.15.16:0/4056732432 conn(0x7fefbc066f50 msgr2=0x7fefbc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:56.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.350+0000 7f17bae48700 1 Processor -- start 2023-09-30T22:35:56.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.350+0000 7fefc45c0700 1 -- 172.21.15.16:0/4056732432 shutdown_connections 2023-09-30T22:35:56.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.350+0000 7fefc45c0700 1 -- 172.21.15.16:0/4056732432 wait complete. 2023-09-30T22:35:56.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.350+0000 7f17bae48700 1 -- start start 2023-09-30T22:35:56.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.351+0000 7fefc45c0700 1 Processor -- start 2023-09-30T22:35:56.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.351+0000 7fefc45c0700 1 -- start start 2023-09-30T22:35:56.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.351+0000 7f17bae48700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f17b406a770 0x7f17b41c80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:56.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.351+0000 7f17bae48700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17b406b080 0x7f17b41c8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:56.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.351+0000 7f17bae48700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f17b4131b40 0x7f17b41c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:56.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.351+0000 7f17bae48700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b4131650 con 0x7f17b406b080 2023-09-30T22:35:56.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.351+0000 7f17bae48700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17b4121c30 con 0x7f17b4131b40 2023-09-30T22:35:56.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.351+0000 7f17bae48700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f17b41222b0 con 0x7f17b406a770 2023-09-30T22:35:56.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.351+0000 7f17b93e5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f17b4131b40 0x7f17b41c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:56.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.351+0000 7f17b93e5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f17b4131b40 0x7f17b41c8b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47586/0 (socket says 172.21.15.16:47586) 2023-09-30T22:35:56.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.351+0000 7f17b93e5700 1 -- 172.21.15.16:0/3830799805 learned_addr learned my addr 172.21.15.16:0/3830799805 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:56.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.351+0000 7fefc45c0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fefbc09d6f0 0x7fefbc122590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:56.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.351+0000 7fefc45c0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefbc09e000 0x7fefbc122ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:56.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.351+0000 7fefc45c0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fefbc09e410 0x7fefbc123010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:56.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.351+0000 7fefc45c0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefbc126060 con 0x7fefbc09e000 2023-09-30T22:35:56.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.351+0000 7fefc45c0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefbc0a1fa0 con 0x7fefbc09d6f0 2023-09-30T22:35:56.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.351+0000 7fefc45c0700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fefbc09c700 con 0x7fefbc09e410 2023-09-30T22:35:56.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.351+0000 7f17b3fff700 1 --2- 172.21.15.16:0/3830799805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17b406b080 0x7f17b41c8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:56.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.352+0000 7f17b93e5700 1 -- 172.21.15.16:0/3830799805 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f17b406a770 msgr2=0x7f17b41c80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:56.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.352+0000 7f17b93e5700 1 --2- 172.21.15.16:0/3830799805 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f17b406a770 0x7f17b41c80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:56.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.352+0000 7f17b93e5700 1 -- 172.21.15.16:0/3830799805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17b406b080 msgr2=0x7f17b41c8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:56.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.352+0000 7f17b93e5700 1 --2- 172.21.15.16:0/3830799805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17b406b080 0x7f17b41c8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:56.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.352+0000 7fefc1b5b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefbc09e000 0x7fefbc122ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:56.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.352+0000 7fefc1b5b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefbc09e000 0x7fefbc122ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38938/0 (socket says 172.21.15.16:38938) 2023-09-30T22:35:56.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.352+0000 7fefc1b5b700 1 -- 172.21.15.16:0/1780672143 learned_addr learned my addr 172.21.15.16:0/1780672143 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:56.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.352+0000 7f17b93e5700 1 -- 172.21.15.16:0/3830799805 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17a00094a0 con 0x7f17b4131b40 2023-09-30T22:35:56.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.352+0000 7fefc235c700 1 --2- 172.21.15.16:0/1780672143 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fefbc09d6f0 0x7fefbc122590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:56.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.352+0000 7fefc2b5d700 1 --2- 172.21.15.16:0/1780672143 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fefbc09e410 0x7fefbc123010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:56.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.352+0000 7f17b93e5700 1 --2- 172.21.15.16:0/3830799805 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f17b4131b40 0x7f17b41c8b70 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f17ac00cca0 tx=0x7f17ac00a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:56.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.352+0000 7fefc1b5b700 1 -- 172.21.15.16:0/1780672143 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fefbc09e410 msgr2=0x7fefbc123010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:56.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.352+0000 7fefc1b5b700 1 --2- 172.21.15.16:0/1780672143 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fefbc09e410 0x7fefbc123010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:56.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.352+0000 7fefc1b5b700 1 -- 172.21.15.16:0/1780672143 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fefbc09d6f0 msgr2=0x7fefbc122590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:56.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.352+0000 7fefc1b5b700 1 --2- 172.21.15.16:0/1780672143 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fefbc09d6f0 0x7fefbc122590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:56.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.352+0000 7fefc1b5b700 1 -- 172.21.15.16:0/1780672143 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb8009550 con 0x7fefbc09e000 2023-09-30T22:35:56.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.352+0000 7f17b17fa700 1 -- 172.21.15.16:0/3830799805 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17ac00ac20 con 0x7f17b4131b40 2023-09-30T22:35:56.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.352+0000 7f17b17fa700 1 -- 172.21.15.16:0/3830799805 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17ac00ad80 con 0x7f17b4131b40 2023-09-30T22:35:56.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.352+0000 7f17bae48700 1 -- 172.21.15.16:0/3830799805 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17b4120cb0 con 0x7f17b4131b40 2023-09-30T22:35:56.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.352+0000 7fefc1b5b700 1 --2- 172.21.15.16:0/1780672143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefbc09e000 0x7fefbc122ad0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fefac00c520 tx=0x7fefac00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:56.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.353+0000 7f17bae48700 1 -- 172.21.15.16:0/3830799805 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17b4131250 con 0x7f17b4131b40 2023-09-30T22:35:56.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.353+0000 7f17b17fa700 1 -- 172.21.15.16:0/3830799805 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17ac007310 con 0x7f17b4131b40 2023-09-30T22:35:56.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.353+0000 7fefb2ffd700 1 -- 172.21.15.16:0/1780672143 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefac0052d0 con 0x7fefbc09e000 2023-09-30T22:35:56.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.353+0000 7fefb2ffd700 1 -- 172.21.15.16:0/1780672143 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefac005430 con 0x7fefbc09e000 2023-09-30T22:35:56.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.353+0000 7fefc45c0700 1 -- 172.21.15.16:0/1780672143 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefbc1253a0 con 0x7fefbc09e000 2023-09-30T22:35:56.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.353+0000 7fefb2ffd700 1 -- 172.21.15.16:0/1780672143 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefac0038c0 con 0x7fefbc09e000 2023-09-30T22:35:56.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.353+0000 7fefc45c0700 1 -- 172.21.15.16:0/1780672143 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefbc1267f0 con 0x7fefbc09e000 2023-09-30T22:35:56.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.354+0000 7fefb2ffd700 1 -- 172.21.15.16:0/1780672143 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fefac003a20 con 0x7fefbc09e000 2023-09-30T22:35:56.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.354+0000 7f17b17fa700 1 -- 172.21.15.16:0/3830799805 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f17ac007470 con 0x7f17b4131b40 2023-09-30T22:35:56.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.354+0000 7f17bae48700 1 -- 172.21.15.16:0/3830799805 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17940052e0 con 0x7f17b4131b40 2023-09-30T22:35:56.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.354+0000 7f17b17fa700 1 -- 172.21.15.16:0/3830799805 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f17ac03f790 con 0x7f17b4131b40 2023-09-30T22:35:56.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.354+0000 7fefb2ffd700 1 -- 172.21.15.16:0/1780672143 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fefac040b30 con 0x7fefbc09e000 2023-09-30T22:35:56.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.354+0000 7fefc45c0700 1 -- 172.21.15.16:0/1780672143 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefbc074ea0 con 0x7fefbc09e000 2023-09-30T22:35:56.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.359+0000 7f17b17fa700 1 -- 172.21.15.16:0/3830799805 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f17ac01c3b0 con 0x7f17b4131b40 2023-09-30T22:35:56.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.359+0000 7fefb2ffd700 1 -- 172.21.15.16:0/1780672143 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fefac017030 con 0x7fefbc09e000 2023-09-30T22:35:56.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.529+0000 7f93aa3a7700 1 -- 172.21.15.16:0/544110465 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f93a406b250 con 0x7f93a409f410 2023-09-30T22:35:56.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.529+0000 7f93a0ff9700 1 -- 172.21.15.16:0/544110465 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9398015020 con 0x7f93a409f410 2023-09-30T22:35:56.530 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:35:56.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.531+0000 7f93aa3a7700 1 -- 172.21.15.16:0/544110465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93a409f410 msgr2=0x7f93a411fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:56.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.531+0000 7f93aa3a7700 1 --2- 172.21.15.16:0/544110465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93a409f410 0x7f93a411fa70 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f9398000c00 tx=0x7f9398004750 comp rx=0 tx=0).stop 2023-09-30T22:35:56.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.531+0000 7f93aa3a7700 1 -- 172.21.15.16:0/544110465 shutdown_connections 2023-09-30T22:35:56.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.531+0000 7f93aa3a7700 1 --2- 172.21.15.16:0/544110465 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f93a409e930 0x7f93a411f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:56.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.531+0000 7f93aa3a7700 1 --2- 172.21.15.16:0/544110465 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f93a4136fe0 0x7f93a411ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:56.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.531+0000 7f93aa3a7700 1 --2- 172.21.15.16:0/544110465 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f93a409f410 0x7f93a411fa70 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:56.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.531+0000 7f93aa3a7700 1 -- 172.21.15.16:0/544110465 >> 172.21.15.16:0/544110465 conn(0x7f93a40a4210 msgr2=0x7f93a40a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:56.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.531+0000 7f93aa3a7700 1 -- 172.21.15.16:0/544110465 shutdown_connections 2023-09-30T22:35:56.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.531+0000 7f93aa3a7700 1 -- 172.21.15.16:0/544110465 wait complete. 2023-09-30T22:35:56.545 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:35:56.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.563+0000 7f17bae48700 1 -- 172.21.15.16:0/3830799805 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1794005590 con 0x7f17b4131b40 2023-09-30T22:35:56.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.564+0000 7f17b17fa700 1 -- 172.21.15.16:0/3830799805 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f17ac018030 con 0x7f17b4131b40 2023-09-30T22:35:56.564 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:35:56.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.566+0000 7f17bae48700 1 -- 172.21.15.16:0/3830799805 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f17b4131b40 msgr2=0x7f17b41c8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:56.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.566+0000 7f17bae48700 1 --2- 172.21.15.16:0/3830799805 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f17b4131b40 0x7f17b41c8b70 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f17ac00cca0 tx=0x7f17ac00a060 comp rx=0 tx=0).stop 2023-09-30T22:35:56.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.566+0000 7f17bae48700 1 -- 172.21.15.16:0/3830799805 shutdown_connections 2023-09-30T22:35:56.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.566+0000 7f17bae48700 1 --2- 172.21.15.16:0/3830799805 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f17b4131b40 0x7f17b41c8b70 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:56.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.566+0000 7f17bae48700 1 --2- 172.21.15.16:0/3830799805 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f17b406a770 0x7f17b41c80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:56.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.566+0000 7f17bae48700 1 --2- 172.21.15.16:0/3830799805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17b406b080 0x7f17b41c8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:56.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.566+0000 7f17bae48700 1 -- 172.21.15.16:0/3830799805 >> 172.21.15.16:0/3830799805 conn(0x7f17b4066f50 msgr2=0x7f17b406ff70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:56.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.566+0000 7f17bae48700 1 -- 172.21.15.16:0/3830799805 shutdown_connections 2023-09-30T22:35:56.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.567+0000 7f17bae48700 1 -- 172.21.15.16:0/3830799805 wait complete. 2023-09-30T22:35:56.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.566+0000 7fefc45c0700 1 -- 172.21.15.16:0/1780672143 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fefbc06c360 con 0x7fefbc09e000 2023-09-30T22:35:56.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.567+0000 7fefb2ffd700 1 -- 172.21.15.16:0/1780672143 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fefac044060 con 0x7fefbc09e000 2023-09-30T22:35:56.568 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:35:56.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.569+0000 7fefc45c0700 1 -- 172.21.15.16:0/1780672143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefbc09e000 msgr2=0x7fefbc122ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:56.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.569+0000 7fefc45c0700 1 --2- 172.21.15.16:0/1780672143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefbc09e000 0x7fefbc122ad0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fefac00c520 tx=0x7fefac00c830 comp rx=0 tx=0).stop 2023-09-30T22:35:56.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.569+0000 7fefc45c0700 1 -- 172.21.15.16:0/1780672143 shutdown_connections 2023-09-30T22:35:56.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.569+0000 7fefc45c0700 1 --2- 172.21.15.16:0/1780672143 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fefbc09d6f0 0x7fefbc122590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:56.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.569+0000 7fefc45c0700 1 --2- 172.21.15.16:0/1780672143 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fefbc09e410 0x7fefbc123010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:56.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.569+0000 7fefc45c0700 1 --2- 172.21.15.16:0/1780672143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fefbc09e000 0x7fefbc122ad0 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:56.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.569+0000 7fefc45c0700 1 -- 172.21.15.16:0/1780672143 >> 172.21.15.16:0/1780672143 conn(0x7fefbc066f50 msgr2=0x7fefbc070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:56.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.569+0000 7fefc45c0700 1 -- 172.21.15.16:0/1780672143 shutdown_connections 2023-09-30T22:35:56.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.569+0000 7fefc45c0700 1 -- 172.21.15.16:0/1780672143 wait complete. 2023-09-30T22:35:56.582 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:35:56.584 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:35:56.651 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:35:56.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.791+0000 7ff3d0c17700 1 Processor -- start 2023-09-30T22:35:56.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.791+0000 7ff3d0c17700 1 -- start start 2023-09-30T22:35:56.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.792+0000 7ff3d0c17700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff3cc09e1d0 0x7ff3cc135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:56.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.792+0000 7ff3d0c17700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff3cc135e50 0x7ff3cc138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:56.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.792+0000 7ff3d0c17700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3cc09f130 con 0x7ff3cc09d6f0 2023-09-30T22:35:56.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.792+0000 7ff3d0c17700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3cc09f520 con 0x7ff3cc135e50 2023-09-30T22:35:56.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.792+0000 7ff3d0c17700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff3cc09e930 con 0x7ff3cc09e1d0 2023-09-30T22:35:56.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.792+0000 7ff3ca59c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff3cc09d6f0 0x7ff3cc09dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:57116/0 (socket says 172.21.15.16:57116) 2023-09-30T22:35:56.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.792+0000 7ff3ca59c700 1 -- 172.21.15.16:0/3266142647 learned_addr learned my addr 172.21.15.16:0/3266142647 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:56.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.792+0000 7ff3c9d9b700 1 --2- 172.21.15.16:0/3266142647 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff3cc09e1d0 0x7ff3cc135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:56.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.792+0000 7ff3cad9d700 1 --2- 172.21.15.16:0/3266142647 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff3cc135e50 0x7ff3cc138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:56.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.793+0000 7ff3c959a700 1 -- 172.21.15.16:0/3266142647 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2970626010 0 0) 0x7ff3cc09f130 con 0x7ff3cc09d6f0 2023-09-30T22:35:56.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.793+0000 7ff3c959a700 1 -- 172.21.15.16:0/3266142647 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3b40035b0 con 0x7ff3cc09d6f0 2023-09-30T22:35:56.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.793+0000 7ff3c9d9b700 1 -- 172.21.15.16:0/3266142647 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff3cc135e50 msgr2=0x7ff3cc138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:56.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.793+0000 7ff3c9d9b700 1 --2- 172.21.15.16:0/3266142647 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff3cc135e50 0x7ff3cc138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:56.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.793+0000 7ff3c9d9b700 1 -- 172.21.15.16:0/3266142647 >> v1:172.21.15.16:6789/0 conn(0x7ff3cc09d6f0 legacy=0x7ff3cc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:56.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.793+0000 7ff3c9d9b700 1 -- 172.21.15.16:0/3266142647 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3cc09b2d0 con 0x7ff3cc09e1d0 2023-09-30T22:35:56.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.793+0000 7ff3cad9d700 1 --2- 172.21.15.16:0/3266142647 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff3cc135e50 0x7ff3cc138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:56.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.793+0000 7ff3c9d9b700 1 --2- 172.21.15.16:0/3266142647 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff3cc09e1d0 0x7ff3cc135910 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0009870 tx=0x7ff3c0009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11b5249150135a60 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:56.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.793+0000 7ff3c959a700 1 -- 172.21.15.16:0/3266142647 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c0011830 con 0x7ff3cc09e1d0 2023-09-30T22:35:56.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.793+0000 7ff3c959a700 1 -- 172.21.15.16:0/3266142647 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3c0011990 con 0x7ff3cc09e1d0 2023-09-30T22:35:56.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.793+0000 7ff3c959a700 1 -- 172.21.15.16:0/3266142647 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c0011d20 con 0x7ff3cc09e1d0 2023-09-30T22:35:56.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.794+0000 7ff3d0c17700 1 -- 172.21.15.16:0/3266142647 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff3cc09e1d0 msgr2=0x7ff3cc135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:56.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.794+0000 7ff3d0c17700 1 --2- 172.21.15.16:0/3266142647 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff3cc09e1d0 0x7ff3cc135910 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0009870 tx=0x7ff3c0009b80 comp rx=0 tx=0).stop 2023-09-30T22:35:56.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.794+0000 7ff3d0c17700 1 -- 172.21.15.16:0/3266142647 shutdown_connections 2023-09-30T22:35:56.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.794+0000 7ff3d0c17700 1 --2- 172.21.15.16:0/3266142647 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff3cc135e50 0x7ff3cc138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:56.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.794+0000 7ff3d0c17700 1 --2- 172.21.15.16:0/3266142647 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff3cc09e1d0 0x7ff3cc135910 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:56.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.794+0000 7ff3d0c17700 1 -- 172.21.15.16:0/3266142647 >> 172.21.15.16:0/3266142647 conn(0x7ff3cc066f50 msgr2=0x7ff3cc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:56.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.794+0000 7ff3d0c17700 1 -- 172.21.15.16:0/3266142647 shutdown_connections 2023-09-30T22:35:56.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.794+0000 7ff3d0c17700 1 -- 172.21.15.16:0/3266142647 wait complete. 2023-09-30T22:35:56.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.795+0000 7ff3d0c17700 1 Processor -- start 2023-09-30T22:35:56.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.795+0000 7ff3d0c17700 1 -- start start 2023-09-30T22:35:56.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.796+0000 7ff3d0c17700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff3cc09d6f0 0x7ff3cc06f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:56.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.796+0000 7ff3d0c17700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3cc09e1d0 0x7ff3cc06f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:56.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.796+0000 7ff3d0c17700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff3cc135e50 0x7ff3cc1c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:56.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.796+0000 7ff3d0c17700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3cc1251b0 con 0x7ff3cc09e1d0 2023-09-30T22:35:56.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.796+0000 7ff3d0c17700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3cc134200 con 0x7ff3cc135e50 2023-09-30T22:35:56.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.796+0000 7ff3d0c17700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff3cc06ed30 con 0x7ff3cc09d6f0 2023-09-30T22:35:56.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.796+0000 7ff3c9d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3cc09e1d0 0x7ff3cc06f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:56.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.796+0000 7ff3cad9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff3cc135e50 0x7ff3cc1c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:56.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.796+0000 7ff3ca59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff3cc09d6f0 0x7ff3cc06f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:56.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.796+0000 7ff3cad9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff3cc135e50 0x7ff3cc1c82f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47630/0 (socket says 172.21.15.16:47630) 2023-09-30T22:35:56.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.796+0000 7ff3c9d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3cc09e1d0 0x7ff3cc06f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38940/0 (socket says 172.21.15.16:38940) 2023-09-30T22:35:56.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.796+0000 7ff3cad9d700 1 -- 172.21.15.16:0/1690700658 learned_addr learned my addr 172.21.15.16:0/1690700658 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:56.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.796+0000 7ff3ca59c700 1 -- 172.21.15.16:0/1690700658 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff3cc135e50 msgr2=0x7ff3cc1c82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:56.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.796+0000 7ff3ca59c700 1 --2- 172.21.15.16:0/1690700658 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff3cc135e50 0x7ff3cc1c82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:56.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.796+0000 7ff3ca59c700 1 -- 172.21.15.16:0/1690700658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3cc09e1d0 msgr2=0x7ff3cc06f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:56.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.796+0000 7ff3ca59c700 1 --2- 172.21.15.16:0/1690700658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3cc09e1d0 0x7ff3cc06f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:56.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.796+0000 7ff3ca59c700 1 -- 172.21.15.16:0/1690700658 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3c4009a70 con 0x7ff3cc09d6f0 2023-09-30T22:35:56.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.796+0000 7ff3cad9d700 1 --2- 172.21.15.16:0/1690700658 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff3cc135e50 0x7ff3cc1c82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:56.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.796+0000 7ff3c9d9b700 1 --2- 172.21.15.16:0/1690700658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3cc09e1d0 0x7ff3cc06f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:56.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.796+0000 7ff3ca59c700 1 --2- 172.21.15.16:0/1690700658 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff3cc09d6f0 0x7ff3cc06f2d0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc00c430 tx=0x7ff3bc00c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:56.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.796+0000 7ff3baffd700 1 -- 172.21.15.16:0/1690700658 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc005d70 con 0x7ff3cc09d6f0 2023-09-30T22:35:56.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.796+0000 7ff3baffd700 1 -- 172.21.15.16:0/1690700658 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3bc0070d0 con 0x7ff3cc09d6f0 2023-09-30T22:35:56.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.796+0000 7ff3d0c17700 1 -- 172.21.15.16:0/1690700658 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3c00094a0 con 0x7ff3cc09d6f0 2023-09-30T22:35:56.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.796+0000 7ff3baffd700 1 -- 172.21.15.16:0/1690700658 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc007460 con 0x7ff3cc09d6f0 2023-09-30T22:35:56.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.797+0000 7ff3d0c17700 1 -- 172.21.15.16:0/1690700658 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3cc070480 con 0x7ff3cc09d6f0 2023-09-30T22:35:56.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.797+0000 7ff3baffd700 1 -- 172.21.15.16:0/1690700658 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff3bc0075c0 con 0x7ff3cc09d6f0 2023-09-30T22:35:56.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.798+0000 7ff3baffd700 1 -- 172.21.15.16:0/1690700658 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7ff3bc03e7a0 con 0x7ff3cc09d6f0 2023-09-30T22:35:56.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.798+0000 7ff3d0c17700 1 -- 172.21.15.16:0/1690700658 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3cc0a1d00 con 0x7ff3cc09d6f0 2023-09-30T22:35:56.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:56.803+0000 7ff3baffd700 1 -- 172.21.15.16:0/1690700658 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff3bc017030 con 0x7ff3cc09d6f0 2023-09-30T22:35:57.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.010+0000 7ff3d0c17700 1 -- 172.21.15.16:0/1690700658 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff3cc09c700 con 0x7ff3cc09d6f0 2023-09-30T22:35:57.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.011+0000 7ff3baffd700 1 -- 172.21.15.16:0/1690700658 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff3bc0188c0 con 0x7ff3cc09d6f0 2023-09-30T22:35:57.011 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:35:57.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.013+0000 7ff3d0c17700 1 -- 172.21.15.16:0/1690700658 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff3cc09d6f0 msgr2=0x7ff3cc06f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:57.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.013+0000 7ff3d0c17700 1 --2- 172.21.15.16:0/1690700658 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff3cc09d6f0 0x7ff3cc06f2d0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc00c430 tx=0x7ff3bc00c740 comp rx=0 tx=0).stop 2023-09-30T22:35:57.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.013+0000 7ff3d0c17700 1 -- 172.21.15.16:0/1690700658 shutdown_connections 2023-09-30T22:35:57.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.013+0000 7ff3d0c17700 1 --2- 172.21.15.16:0/1690700658 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff3cc135e50 0x7ff3cc1c82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:57.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.013+0000 7ff3d0c17700 1 --2- 172.21.15.16:0/1690700658 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff3cc09d6f0 0x7ff3cc06f2d0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:57.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.013+0000 7ff3d0c17700 1 --2- 172.21.15.16:0/1690700658 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff3cc09e1d0 0x7ff3cc06f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:57.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.013+0000 7ff3d0c17700 1 -- 172.21.15.16:0/1690700658 >> 172.21.15.16:0/1690700658 conn(0x7ff3cc066f50 msgr2=0x7ff3cc134840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:57.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.013+0000 7ff3d0c17700 1 -- 172.21.15.16:0/1690700658 shutdown_connections 2023-09-30T22:35:57.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.013+0000 7ff3d0c17700 1 -- 172.21.15.16:0/1690700658 wait complete. 2023-09-30T22:35:57.027 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:35:57.546 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:35:57.582 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:35:57.584 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:35:57.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.697+0000 7fa213504700 1 Processor -- start 2023-09-30T22:35:57.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.697+0000 7fa213504700 1 -- start start 2023-09-30T22:35:57.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.698+0000 7fa213504700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa20c1338a0 0x7fa20c133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:57.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.698+0000 7fa213504700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa20c09e930 0x7fa20c13b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:57.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.698+0000 7fa213504700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa20c06ef30 con 0x7fa20c09de50 2023-09-30T22:35:57.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.698+0000 7fa213504700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa20c06fef0 con 0x7fa20c09e930 2023-09-30T22:35:57.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.698+0000 7fa213504700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa20c1334b0 con 0x7fa20c1338a0 2023-09-30T22:35:57.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.698+0000 7fa2112a0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa20c1338a0 0x7fa20c133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:57.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.698+0000 7fa210a9f700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa20c09de50 0x7fa20c09e220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38218/0 (socket says 172.21.15.16:38218) 2023-09-30T22:35:57.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.698+0000 7fa210a9f700 1 -- 172.21.15.16:0/3846749906 learned_addr learned my addr 172.21.15.16:0/3846749906 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:57.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.698+0000 7fa211aa1700 1 --2- 172.21.15.16:0/3846749906 >> v2:172.21.15.16:3300/0 conn(0x7fa20c09e930 0x7fa20c13b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:57.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.698+0000 7fa2112a0700 1 -- 172.21.15.16:0/3846749906 >> v2:172.21.15.16:3300/0 conn(0x7fa20c09e930 msgr2=0x7fa20c13b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:57.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.698+0000 7fa2112a0700 1 --2- 172.21.15.16:0/3846749906 >> v2:172.21.15.16:3300/0 conn(0x7fa20c09e930 0x7fa20c13b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:57.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.698+0000 7fa2112a0700 1 -- 172.21.15.16:0/3846749906 >> v1:172.21.15.16:6789/0 conn(0x7fa20c09de50 legacy=0x7fa20c09e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:57.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.698+0000 7fa2112a0700 1 -- 172.21.15.16:0/3846749906 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa20c06fc30 con 0x7fa20c1338a0 2023-09-30T22:35:57.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.698+0000 7fa211aa1700 1 --2- 172.21.15.16:0/3846749906 >> v2:172.21.15.16:3300/0 conn(0x7fa20c09e930 0x7fa20c13b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:57.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.698+0000 7fa2112a0700 1 --2- 172.21.15.16:0/3846749906 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa20c1338a0 0x7fa20c133c70 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fa208009840 tx=0x7fa208009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=333cd5cef0ff72b1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:57.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.699+0000 7fa203fff700 1 -- 172.21.15.16:0/3846749906 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa208011830 con 0x7fa20c1338a0 2023-09-30T22:35:57.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.699+0000 7fa203fff700 1 -- 172.21.15.16:0/3846749906 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa208011990 con 0x7fa20c1338a0 2023-09-30T22:35:57.700 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.699+0000 7fa203fff700 1 -- 172.21.15.16:0/3846749906 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa208011d20 con 0x7fa20c1338a0 2023-09-30T22:35:57.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.699+0000 7fa213504700 1 -- 172.21.15.16:0/3846749906 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa20c1338a0 msgr2=0x7fa20c133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:57.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.699+0000 7fa213504700 1 --2- 172.21.15.16:0/3846749906 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa20c1338a0 0x7fa20c133c70 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fa208009840 tx=0x7fa208009b50 comp rx=0 tx=0).stop 2023-09-30T22:35:57.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.700+0000 7fa213504700 1 -- 172.21.15.16:0/3846749906 shutdown_connections 2023-09-30T22:35:57.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.700+0000 7fa213504700 1 --2- 172.21.15.16:0/3846749906 >> v2:172.21.15.16:3300/0 conn(0x7fa20c09e930 0x7fa20c13b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:57.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.700+0000 7fa213504700 1 --2- 172.21.15.16:0/3846749906 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa20c1338a0 0x7fa20c133c70 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:57.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.700+0000 7fa213504700 1 -- 172.21.15.16:0/3846749906 >> 172.21.15.16:0/3846749906 conn(0x7fa20c0a2aa0 msgr2=0x7fa20c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:57.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.700+0000 7fa213504700 1 -- 172.21.15.16:0/3846749906 shutdown_connections 2023-09-30T22:35:57.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.700+0000 7fa213504700 1 -- 172.21.15.16:0/3846749906 wait complete. 2023-09-30T22:35:57.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.701+0000 7fa213504700 1 Processor -- start 2023-09-30T22:35:57.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.701+0000 7fa213504700 1 -- start start 2023-09-30T22:35:57.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.701+0000 7fa213504700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa20c09de50 0x7fa20c11d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:57.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.701+0000 7fa213504700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa20c09e930 0x7fa20c11dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:57.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.701+0000 7fa213504700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa20c1338a0 0x7fa20c11e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:57.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.701+0000 7fa213504700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa20c11f350 con 0x7fa20c1338a0 2023-09-30T22:35:57.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.701+0000 7fa213504700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa20c06f640 con 0x7fa20c09de50 2023-09-30T22:35:57.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.701+0000 7fa213504700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa20c120f80 con 0x7fa20c09e930 2023-09-30T22:35:57.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.701+0000 7fa210a9f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa20c09e930 0x7fa20c11dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:57.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.701+0000 7fa2112a0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa20c09de50 0x7fa20c11d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:57.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.701+0000 7fa211aa1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa20c1338a0 0x7fa20c11e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:57.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.701+0000 7fa2112a0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa20c09de50 0x7fa20c11d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47314/0 (socket says 172.21.15.16:47314) 2023-09-30T22:35:57.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.701+0000 7fa2112a0700 1 -- 172.21.15.16:0/2378272129 learned_addr learned my addr 172.21.15.16:0/2378272129 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:57.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.701+0000 7fa210a9f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa20c09e930 0x7fa20c11dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:40888/0 (socket says 172.21.15.16:40888) 2023-09-30T22:35:57.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.702+0000 7fa210a9f700 1 -- 172.21.15.16:0/2378272129 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa20c09de50 msgr2=0x7fa20c11d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:57.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.702+0000 7fa210a9f700 1 --2- 172.21.15.16:0/2378272129 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa20c09de50 0x7fa20c11d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:57.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.702+0000 7fa210a9f700 1 -- 172.21.15.16:0/2378272129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa20c1338a0 msgr2=0x7fa20c11e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:57.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.702+0000 7fa210a9f700 1 --2- 172.21.15.16:0/2378272129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa20c1338a0 0x7fa20c11e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:57.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.702+0000 7fa210a9f700 1 -- 172.21.15.16:0/2378272129 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa204009a70 con 0x7fa20c09e930 2023-09-30T22:35:57.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.702+0000 7fa211aa1700 1 --2- 172.21.15.16:0/2378272129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa20c1338a0 0x7fa20c11e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:57.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.702+0000 7fa210a9f700 1 --2- 172.21.15.16:0/2378272129 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa20c09e930 0x7fa20c11dd60 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc00c390 tx=0x7fa1fc00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:57.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.702+0000 7fa201ffb700 1 -- 172.21.15.16:0/2378272129 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc005b40 con 0x7fa20c09e930 2023-09-30T22:35:57.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.702+0000 7fa201ffb700 1 -- 172.21.15.16:0/2378272129 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1fc005ca0 con 0x7fa20c09e930 2023-09-30T22:35:57.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.702+0000 7fa213504700 1 -- 172.21.15.16:0/2378272129 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2080094a0 con 0x7fa20c09e930 2023-09-30T22:35:57.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.702+0000 7fa201ffb700 1 -- 172.21.15.16:0/2378272129 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc0072f0 con 0x7fa20c09e930 2023-09-30T22:35:57.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.703+0000 7fa213504700 1 -- 172.21.15.16:0/2378272129 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa20c0a1050 con 0x7fa20c09e930 2023-09-30T22:35:57.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.704+0000 7fa201ffb700 1 -- 172.21.15.16:0/2378272129 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa1fc007450 con 0x7fa20c09e930 2023-09-30T22:35:57.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.704+0000 7fa213504700 1 -- 172.21.15.16:0/2378272129 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa20c11a470 con 0x7fa20c09e930 2023-09-30T22:35:57.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.704+0000 7fa201ffb700 1 -- 172.21.15.16:0/2378272129 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fa1fc03e9a0 con 0x7fa20c09e930 2023-09-30T22:35:57.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.709+0000 7fa201ffb700 1 -- 172.21.15.16:0/2378272129 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa1fc01b3b0 con 0x7fa20c09e930 2023-09-30T22:35:57.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.736+0000 7fa9dfbdb700 1 Processor -- start 2023-09-30T22:35:57.737 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.737+0000 7fa9dfbdb700 1 -- start start 2023-09-30T22:35:57.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.737+0000 7fa9dfbdb700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa9d8131810 0x7fa9d8131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:57.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.737+0000 7fa9dfbdb700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa9d8099a40 0x7fa9d8099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:57.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.737+0000 7fa9dfbdb700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9d81313c0 con 0x7fa9d809a3b0 2023-09-30T22:35:57.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.737+0000 7fa9dfbdb700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d809b580 con 0x7fa9d8131810 2023-09-30T22:35:57.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.737+0000 7fa9dfbdb700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d806e8c0 con 0x7fa9d8099a40 2023-09-30T22:35:57.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.737+0000 7fa9de178700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa9d809a3b0 0x7fa9d8138650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38232/0 (socket says 172.21.15.16:38232) 2023-09-30T22:35:57.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.737+0000 7fa9de178700 1 -- 172.21.15.16:0/3642386364 learned_addr learned my addr 172.21.15.16:0/3642386364 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:57.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.737+0000 7fa9dd176700 1 --2- 172.21.15.16:0/3642386364 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa9d8099a40 0x7fa9d8099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:57.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.737+0000 7fa9dd977700 1 --2- 172.21.15.16:0/3642386364 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa9d8131810 0x7fa9d8131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:57.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.738+0000 7fa9dd176700 1 -- 172.21.15.16:0/3642386364 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa9d8131810 msgr2=0x7fa9d8131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:57.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.738+0000 7fa9dd176700 1 --2- 172.21.15.16:0/3642386364 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa9d8131810 0x7fa9d8131be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:57.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.738+0000 7fa9dd176700 1 -- 172.21.15.16:0/3642386364 >> v1:172.21.15.16:6789/0 conn(0x7fa9d809a3b0 legacy=0x7fa9d8138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:57.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.738+0000 7fa9dc975700 1 -- 172.21.15.16:0/3642386364 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1552915313 0 0) 0x7fa9d81313c0 con 0x7fa9d809a3b0 2023-09-30T22:35:57.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.738+0000 7fa9dd176700 1 -- 172.21.15.16:0/3642386364 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d809a820 con 0x7fa9d8099a40 2023-09-30T22:35:57.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.738+0000 7fa9dd176700 1 --2- 172.21.15.16:0/3642386364 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa9d8099a40 0x7fa9d8099e70 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8009870 tx=0x7fa9c8009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a91cdbbaa6cb731 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:57.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.738+0000 7fa9dc975700 1 -- 172.21.15.16:0/3642386364 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c8011830 con 0x7fa9d8099a40 2023-09-30T22:35:57.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.738+0000 7fa9dc975700 1 -- 172.21.15.16:0/3642386364 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9c8011990 con 0x7fa9d8099a40 2023-09-30T22:35:57.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.738+0000 7fa9dc975700 1 -- 172.21.15.16:0/3642386364 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9c8011d20 con 0x7fa9d8099a40 2023-09-30T22:35:57.743 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.739+0000 7fa9dfbdb700 1 -- 172.21.15.16:0/3642386364 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa9d8099a40 msgr2=0x7fa9d8099e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:57.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.739+0000 7fa9dfbdb700 1 --2- 172.21.15.16:0/3642386364 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa9d8099a40 0x7fa9d8099e70 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8009870 tx=0x7fa9c8009b80 comp rx=0 tx=0).stop 2023-09-30T22:35:57.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.739+0000 7fa9dfbdb700 1 -- 172.21.15.16:0/3642386364 shutdown_connections 2023-09-30T22:35:57.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.739+0000 7fa9dfbdb700 1 --2- 172.21.15.16:0/3642386364 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa9d8131810 0x7fa9d8131be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:57.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.739+0000 7fa9dfbdb700 1 --2- 172.21.15.16:0/3642386364 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa9d8099a40 0x7fa9d8099e70 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:57.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.739+0000 7fa9dfbdb700 1 -- 172.21.15.16:0/3642386364 >> 172.21.15.16:0/3642386364 conn(0x7fa9d80a2aa0 msgr2=0x7fa9d80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:57.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.739+0000 7fa9dfbdb700 1 -- 172.21.15.16:0/3642386364 shutdown_connections 2023-09-30T22:35:57.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.739+0000 7fa9dfbdb700 1 -- 172.21.15.16:0/3642386364 wait complete. 2023-09-30T22:35:57.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.740+0000 7fa9dfbdb700 1 Processor -- start 2023-09-30T22:35:57.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.740+0000 7fa9dfbdb700 1 -- start start 2023-09-30T22:35:57.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.741+0000 7fa9dfbdb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9d8099a40 0x7fa9d8118cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:57.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.741+0000 7fa9dfbdb700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa9d809a3b0 0x7fa9d8119210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:57.744 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.741+0000 7fa9dfbdb700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa9d8131810 0x7fa9d8119750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:57.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.741+0000 7fa9dfbdb700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d81307c0 con 0x7fa9d8099a40 2023-09-30T22:35:57.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.741+0000 7fa9dfbdb700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d812bc40 con 0x7fa9d8131810 2023-09-30T22:35:57.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.741+0000 7fa9dfbdb700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d811bff0 con 0x7fa9d809a3b0 2023-09-30T22:35:57.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.741+0000 7fa9de178700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa9d8131810 0x7fa9d8119750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:57.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.741+0000 7fa9de178700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa9d8131810 0x7fa9d8119750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47332/0 (socket says 172.21.15.16:47332) 2023-09-30T22:35:57.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.741+0000 7fa9de178700 1 -- 172.21.15.16:0/3854595161 learned_addr learned my addr 172.21.15.16:0/3854595161 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:57.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.741+0000 7fa9dd977700 1 --2- 172.21.15.16:0/3854595161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9d8099a40 0x7fa9d8118cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:57.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.742+0000 7fa9dd176700 1 --2- 172.21.15.16:0/3854595161 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa9d809a3b0 0x7fa9d8119210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:57.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.742+0000 7fa9de178700 1 -- 172.21.15.16:0/3854595161 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa9d809a3b0 msgr2=0x7fa9d8119210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:57.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.742+0000 7fa9de178700 1 --2- 172.21.15.16:0/3854595161 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa9d809a3b0 0x7fa9d8119210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:57.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.742+0000 7fa9de178700 1 -- 172.21.15.16:0/3854595161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9d8099a40 msgr2=0x7fa9d8118cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:57.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.742+0000 7fa9de178700 1 --2- 172.21.15.16:0/3854595161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9d8099a40 0x7fa9d8118cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:57.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.742+0000 7fa9de178700 1 -- 172.21.15.16:0/3854595161 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9c80094a0 con 0x7fa9d8131810 2023-09-30T22:35:57.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.742+0000 7fa9de178700 1 --2- 172.21.15.16:0/3854595161 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa9d8131810 0x7fa9d8119750 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fa9d000cca0 tx=0x7fa9d000a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:57.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.742+0000 7fa9ce7fc700 1 -- 172.21.15.16:0/3854595161 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d000ab70 con 0x7fa9d8131810 2023-09-30T22:35:57.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.742+0000 7fa9ce7fc700 1 -- 172.21.15.16:0/3854595161 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9d000acd0 con 0x7fa9d8131810 2023-09-30T22:35:57.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.742+0000 7fa9ce7fc700 1 -- 172.21.15.16:0/3854595161 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9d00117a0 con 0x7fa9d8131810 2023-09-30T22:35:57.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.742+0000 7fa9dfbdb700 1 -- 172.21.15.16:0/3854595161 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9d811b610 con 0x7fa9d8131810 2023-09-30T22:35:57.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.743+0000 7fa9dfbdb700 1 -- 172.21.15.16:0/3854595161 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9d811ca60 con 0x7fa9d8131810 2023-09-30T22:35:57.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.744+0000 7fa9ce7fc700 1 -- 172.21.15.16:0/3854595161 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa9d0017020 con 0x7fa9d8131810 2023-09-30T22:35:57.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.744+0000 7fa9dfbdb700 1 -- 172.21.15.16:0/3854595161 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9d8067ca0 con 0x7fa9d8131810 2023-09-30T22:35:57.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.747+0000 7f5d3ae96700 1 Processor -- start 2023-09-30T22:35:57.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.747+0000 7f5d3ae96700 1 -- start start 2023-09-30T22:35:57.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.747+0000 7f5d3ae96700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d3409d6f0 0x7f5d3409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:57.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.747+0000 7f5d3ae96700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d3409e000 0x7f5d34130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:57.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.747+0000 7f5d3ae96700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5d3409e410 0x7f5d34130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:57.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.747+0000 7f5d3ae96700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d3409f520 con 0x7f5d3409e410 2023-09-30T22:35:57.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.747+0000 7f5d3ae96700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3409f130 con 0x7f5d3409e000 2023-09-30T22:35:57.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.748+0000 7f5d3ae96700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3409e930 con 0x7f5d3409d6f0 2023-09-30T22:35:57.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.748+0000 7f5d38c32700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d3409d6f0 0x7f5d3409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:57.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.748+0000 7f5d38c32700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d3409d6f0 0x7f5d3409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:40928/0 (socket says 172.21.15.16:40928) 2023-09-30T22:35:57.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.748+0000 7f5d38c32700 1 -- 172.21.15.16:0/3106725275 learned_addr learned my addr 172.21.15.16:0/3106725275 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:57.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.748+0000 7f5d39433700 1 --2- 172.21.15.16:0/3106725275 >> v2:172.21.15.16:3300/0 conn(0x7f5d3409e410 0x7f5d34130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:57.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.748+0000 7f5d38c32700 1 -- 172.21.15.16:0/3106725275 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d3409e000 msgr2=0x7f5d34130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:57.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.748+0000 7f5d38c32700 1 --2- 172.21.15.16:0/3106725275 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d3409e000 0x7f5d34130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:57.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.748+0000 7f5d38c32700 1 -- 172.21.15.16:0/3106725275 >> v2:172.21.15.16:3300/0 conn(0x7f5d3409e410 msgr2=0x7f5d34130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:57.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.748+0000 7f5d38c32700 1 --2- 172.21.15.16:0/3106725275 >> v2:172.21.15.16:3300/0 conn(0x7f5d3409e410 0x7f5d34130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:57.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.748+0000 7f5d38c32700 1 -- 172.21.15.16:0/3106725275 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d3409b2d0 con 0x7f5d3409d6f0 2023-09-30T22:35:57.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.748+0000 7f5d38c32700 1 --2- 172.21.15.16:0/3106725275 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d3409d6f0 0x7f5d3409dac0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f5d28009920 tx=0x7f5d28009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c950a06673687f0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:57.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.748+0000 7f5d337fe700 1 -- 172.21.15.16:0/3106725275 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d28011830 con 0x7f5d3409d6f0 2023-09-30T22:35:57.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.748+0000 7f5d337fe700 1 -- 172.21.15.16:0/3106725275 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d28011990 con 0x7f5d3409d6f0 2023-09-30T22:35:57.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.748+0000 7f5d337fe700 1 -- 172.21.15.16:0/3106725275 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d28011d20 con 0x7f5d3409d6f0 2023-09-30T22:35:57.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.749+0000 7f5d3ae96700 1 -- 172.21.15.16:0/3106725275 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d3409d6f0 msgr2=0x7f5d3409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:57.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.749+0000 7f5d3ae96700 1 --2- 172.21.15.16:0/3106725275 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d3409d6f0 0x7f5d3409dac0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f5d28009920 tx=0x7f5d28009c30 comp rx=0 tx=0).stop 2023-09-30T22:35:57.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.749+0000 7f5d3ae96700 1 -- 172.21.15.16:0/3106725275 shutdown_connections 2023-09-30T22:35:57.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.749+0000 7f5d3ae96700 1 --2- 172.21.15.16:0/3106725275 >> v2:172.21.15.16:3300/0 conn(0x7f5d3409e410 0x7f5d34130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:57.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.749+0000 7f5d3ae96700 1 --2- 172.21.15.16:0/3106725275 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d3409d6f0 0x7f5d3409dac0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:57.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.749+0000 7f5d3ae96700 1 --2- 172.21.15.16:0/3106725275 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d3409e000 0x7f5d34130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:57.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.749+0000 7f5d3ae96700 1 -- 172.21.15.16:0/3106725275 >> 172.21.15.16:0/3106725275 conn(0x7f5d34066f50 msgr2=0x7f5d3409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:57.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.749+0000 7f5d3ae96700 1 -- 172.21.15.16:0/3106725275 shutdown_connections 2023-09-30T22:35:57.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.749+0000 7f5d3ae96700 1 -- 172.21.15.16:0/3106725275 wait complete. 2023-09-30T22:35:57.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.750+0000 7fa9ce7fc700 1 -- 172.21.15.16:0/3854595161 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fa9d0011e20 con 0x7fa9d8131810 2023-09-30T22:35:57.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.750+0000 7f5d3ae96700 1 Processor -- start 2023-09-30T22:35:57.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.750+0000 7f5d3ae96700 1 -- start start 2023-09-30T22:35:57.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.751+0000 7f5d3ae96700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d3409d6f0 0x7f5d341c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:57.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.751+0000 7f5d3ae96700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d3409e000 0x7f5d341c55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:57.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.751+0000 7f5d3ae96700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d3409e410 0x7f5d34131650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:57.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.751+0000 7f5d3ae96700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d341349c0 con 0x7f5d3409d6f0 2023-09-30T22:35:57.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.751+0000 7f5d3ae96700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d34134550 con 0x7f5d3409e000 2023-09-30T22:35:57.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.751+0000 7f5d3ae96700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5d341c4c70 con 0x7f5d3409e410 2023-09-30T22:35:57.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.751+0000 7f5d33fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d3409e000 0x7f5d341c55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:57.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.751+0000 7f5d39433700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d3409e410 0x7f5d34131650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:57.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.751+0000 7f5d33fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d3409e000 0x7f5d341c55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47350/0 (socket says 172.21.15.16:47350) 2023-09-30T22:35:57.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.751+0000 7f5d39433700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d3409e410 0x7f5d34131650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:40944/0 (socket says 172.21.15.16:40944) 2023-09-30T22:35:57.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.751+0000 7f5d33fff700 1 -- 172.21.15.16:0/2884039679 learned_addr learned my addr 172.21.15.16:0/2884039679 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:57.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.751+0000 7f5d38c32700 1 --2- 172.21.15.16:0/2884039679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d3409d6f0 0x7f5d341c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:57.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.752+0000 7f5d33fff700 1 -- 172.21.15.16:0/2884039679 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d3409e410 msgr2=0x7f5d34131650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:57.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.752+0000 7f5d33fff700 1 --2- 172.21.15.16:0/2884039679 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d3409e410 0x7f5d34131650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:57.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.752+0000 7f5d33fff700 1 -- 172.21.15.16:0/2884039679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d3409d6f0 msgr2=0x7f5d341c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:57.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.752+0000 7f5d33fff700 1 --2- 172.21.15.16:0/2884039679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d3409d6f0 0x7f5d341c50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:57.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.752+0000 7f5d33fff700 1 -- 172.21.15.16:0/2884039679 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d28009550 con 0x7f5d3409e000 2023-09-30T22:35:57.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.752+0000 7f5d39433700 1 --2- 172.21.15.16:0/2884039679 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d3409e410 0x7f5d34131650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:57.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.752+0000 7f5d38c32700 1 --2- 172.21.15.16:0/2884039679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d3409d6f0 0x7f5d341c50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:57.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.752+0000 7f5d33fff700 1 --2- 172.21.15.16:0/2884039679 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d3409e000 0x7f5d341c55e0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f5d2000b530 tx=0x7f5d2000b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:57.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.757+0000 7f5d317fa700 1 -- 172.21.15.16:0/2884039679 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d20011830 con 0x7f5d3409e000 2023-09-30T22:35:57.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.757+0000 7f5d317fa700 1 -- 172.21.15.16:0/2884039679 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d200119b0 con 0x7f5d3409e000 2023-09-30T22:35:57.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.757+0000 7f5d3ae96700 1 -- 172.21.15.16:0/2884039679 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d34072580 con 0x7f5d3409e000 2023-09-30T22:35:57.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.757+0000 7f5d317fa700 1 -- 172.21.15.16:0/2884039679 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d20011d60 con 0x7f5d3409e000 2023-09-30T22:35:57.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.757+0000 7fa9ce7fc700 1 -- 172.21.15.16:0/3854595161 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa9d0020030 con 0x7fa9d8131810 2023-09-30T22:35:57.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.757+0000 7f5d3ae96700 1 -- 172.21.15.16:0/2884039679 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d34132940 con 0x7f5d3409e000 2023-09-30T22:35:57.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.758+0000 7f5d317fa700 1 -- 172.21.15.16:0/2884039679 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5d2001a400 con 0x7f5d3409e000 2023-09-30T22:35:57.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.758+0000 7f5d317fa700 1 -- 172.21.15.16:0/2884039679 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f5d2003f5c0 con 0x7f5d3409e000 2023-09-30T22:35:57.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.758+0000 7f5d3ae96700 1 -- 172.21.15.16:0/2884039679 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d34134690 con 0x7f5d3409e000 2023-09-30T22:35:57.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.763+0000 7f5d317fa700 1 -- 172.21.15.16:0/2884039679 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5d34134690 con 0x7f5d3409e000 2023-09-30T22:35:57.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.932+0000 7fa213504700 1 -- 172.21.15.16:0/2378272129 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa20c074ea0 con 0x7fa20c09e930 2023-09-30T22:35:57.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.932+0000 7fa201ffb700 1 -- 172.21.15.16:0/2378272129 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa1fc017030 con 0x7fa20c09e930 2023-09-30T22:35:57.933 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:35:57.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.934+0000 7fa213504700 1 -- 172.21.15.16:0/2378272129 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa20c09e930 msgr2=0x7fa20c11dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:57.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.934+0000 7fa213504700 1 --2- 172.21.15.16:0/2378272129 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa20c09e930 0x7fa20c11dd60 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc00c390 tx=0x7fa1fc00c6a0 comp rx=0 tx=0).stop 2023-09-30T22:35:57.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.935+0000 7fa213504700 1 -- 172.21.15.16:0/2378272129 shutdown_connections 2023-09-30T22:35:57.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.935+0000 7fa213504700 1 --2- 172.21.15.16:0/2378272129 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa20c09de50 0x7fa20c11d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:57.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.935+0000 7fa213504700 1 --2- 172.21.15.16:0/2378272129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa20c1338a0 0x7fa20c11e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:57.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.935+0000 7fa213504700 1 --2- 172.21.15.16:0/2378272129 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa20c09e930 0x7fa20c11dd60 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:57.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.935+0000 7fa213504700 1 -- 172.21.15.16:0/2378272129 >> 172.21.15.16:0/2378272129 conn(0x7fa20c0a2aa0 msgr2=0x7fa20c1badd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:57.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.935+0000 7fa213504700 1 -- 172.21.15.16:0/2378272129 shutdown_connections 2023-09-30T22:35:57.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.935+0000 7fa213504700 1 -- 172.21.15.16:0/2378272129 wait complete. 2023-09-30T22:35:57.954 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:35:57.965 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.965+0000 7fa9dfbdb700 1 -- 172.21.15.16:0/3854595161 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa9d8074ea0 con 0x7fa9d8131810 2023-09-30T22:35:57.966 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.966+0000 7fa9ce7fc700 1 -- 172.21.15.16:0/3854595161 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa9d00070d0 con 0x7fa9d8131810 2023-09-30T22:35:57.966 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:35:57.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.968+0000 7fa9dfbdb700 1 -- 172.21.15.16:0/3854595161 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa9d8131810 msgr2=0x7fa9d8119750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:57.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.968+0000 7fa9dfbdb700 1 --2- 172.21.15.16:0/3854595161 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa9d8131810 0x7fa9d8119750 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fa9d000cca0 tx=0x7fa9d000a060 comp rx=0 tx=0).stop 2023-09-30T22:35:57.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.968+0000 7fa9dfbdb700 1 -- 172.21.15.16:0/3854595161 shutdown_connections 2023-09-30T22:35:57.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.968+0000 7fa9dfbdb700 1 --2- 172.21.15.16:0/3854595161 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa9d8131810 0x7fa9d8119750 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:57.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.968+0000 7fa9dfbdb700 1 --2- 172.21.15.16:0/3854595161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa9d8099a40 0x7fa9d8118cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:57.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.968+0000 7fa9dfbdb700 1 --2- 172.21.15.16:0/3854595161 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa9d809a3b0 0x7fa9d8119210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:57.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.968+0000 7fa9dfbdb700 1 -- 172.21.15.16:0/3854595161 >> 172.21.15.16:0/3854595161 conn(0x7fa9d80a2aa0 msgr2=0x7fa9d812e710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:57.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.968+0000 7fa9dfbdb700 1 -- 172.21.15.16:0/3854595161 shutdown_connections 2023-09-30T22:35:57.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.968+0000 7fa9dfbdb700 1 -- 172.21.15.16:0/3854595161 wait complete. 2023-09-30T22:35:57.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.977+0000 7f5d3ae96700 1 -- 172.21.15.16:0/2884039679 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f5d34134690 con 0x7f5d3409e000 2023-09-30T22:35:57.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.978+0000 7f5d317fa700 1 -- 172.21.15.16:0/2884039679 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5d34134690 con 0x7f5d3409e000 2023-09-30T22:35:57.979 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:35:57.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.980+0000 7f5d3ae96700 1 -- 172.21.15.16:0/2884039679 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d3409e000 msgr2=0x7f5d341c55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:57.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.980+0000 7f5d3ae96700 1 --2- 172.21.15.16:0/2884039679 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d3409e000 0x7f5d341c55e0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f5d2000b530 tx=0x7f5d2000b840 comp rx=0 tx=0).stop 2023-09-30T22:35:57.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.980+0000 7f5d3ae96700 1 -- 172.21.15.16:0/2884039679 shutdown_connections 2023-09-30T22:35:57.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.980+0000 7f5d3ae96700 1 --2- 172.21.15.16:0/2884039679 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5d3409e410 0x7f5d34131650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:57.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.980+0000 7f5d3ae96700 1 --2- 172.21.15.16:0/2884039679 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5d3409d6f0 0x7f5d341c50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:57.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.980+0000 7f5d3ae96700 1 --2- 172.21.15.16:0/2884039679 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5d3409e000 0x7f5d341c55e0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:57.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.980+0000 7f5d3ae96700 1 -- 172.21.15.16:0/2884039679 >> 172.21.15.16:0/2884039679 conn(0x7f5d34066f50 msgr2=0x7f5d3406a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:57.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.980+0000 7f5d3ae96700 1 -- 172.21.15.16:0/2884039679 shutdown_connections 2023-09-30T22:35:57.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:57.980+0000 7f5d3ae96700 1 -- 172.21.15.16:0/2884039679 wait complete. 2023-09-30T22:35:57.983 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:35:57.994 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:35:58.028 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:35:58.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.175+0000 7f8383b9a700 1 Processor -- start 2023-09-30T22:35:58.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.175+0000 7f8383b9a700 1 -- start start 2023-09-30T22:35:58.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.175+0000 7f8383b9a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f837c09d6f0 0x7f837c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:58.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.176+0000 7f8383b9a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f837c09e000 0x7f837c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:58.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.176+0000 7f8383b9a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f837c09e410 0x7f837c133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:58.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.176+0000 7f8383b9a700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f837c06b460 con 0x7f837c09e000 2023-09-30T22:35:58.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.176+0000 7f8383b9a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f837c09b120 con 0x7f837c09d6f0 2023-09-30T22:35:58.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.176+0000 7f8383b9a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f837c09a7e0 con 0x7f837c09e410 2023-09-30T22:35:58.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.176+0000 7f8381135700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f837c09e000 0x7f837c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:58.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.176+0000 7f8382137700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f837c09e410 0x7f837c133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:58.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.176+0000 7f8381936700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f837c09d6f0 0x7f837c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:58.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.176+0000 7f8382137700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f837c09e410 0x7f837c133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:40948/0 (socket says 172.21.15.16:40948) 2023-09-30T22:35:58.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.176+0000 7f8381135700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f837c09e000 0x7f837c1339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39566/0 (socket says 172.21.15.16:39566) 2023-09-30T22:35:58.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.176+0000 7f8382137700 1 -- 172.21.15.16:0/758326480 learned_addr learned my addr 172.21.15.16:0/758326480 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:58.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.176+0000 7f8381936700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f837c09d6f0 0x7f837c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47354/0 (socket says 172.21.15.16:47354) 2023-09-30T22:35:58.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.176+0000 7f8382137700 1 -- 172.21.15.16:0/758326480 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f837c09d6f0 msgr2=0x7f837c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:58.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.176+0000 7f8382137700 1 --2- 172.21.15.16:0/758326480 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f837c09d6f0 0x7f837c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:58.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.176+0000 7f8382137700 1 -- 172.21.15.16:0/758326480 >> v2:172.21.15.16:3300/0 conn(0x7f837c09e000 msgr2=0x7f837c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:58.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.176+0000 7f8382137700 1 --2- 172.21.15.16:0/758326480 >> v2:172.21.15.16:3300/0 conn(0x7f837c09e000 0x7f837c1339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:58.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.176+0000 7f8382137700 1 -- 172.21.15.16:0/758326480 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f837c09a500 con 0x7f837c09e410 2023-09-30T22:35:58.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.176+0000 7f8381936700 1 --2- 172.21.15.16:0/758326480 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f837c09d6f0 0x7f837c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:58.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.176+0000 7f8381135700 1 --2- 172.21.15.16:0/758326480 >> v2:172.21.15.16:3300/0 conn(0x7f837c09e000 0x7f837c1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:58.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.176+0000 7f8382137700 1 --2- 172.21.15.16:0/758326480 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f837c09e410 0x7f837c133f30 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f8374009e80 tx=0x7f837400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4760f890413ad970 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:58.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.176+0000 7f8380934700 1 -- 172.21.15.16:0/758326480 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f837400e070 con 0x7f837c09e410 2023-09-30T22:35:58.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.177+0000 7f8380934700 1 -- 172.21.15.16:0/758326480 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f837400b8d0 con 0x7f837c09e410 2023-09-30T22:35:58.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.177+0000 7f8380934700 1 -- 172.21.15.16:0/758326480 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f837400bc80 con 0x7f837c09e410 2023-09-30T22:35:58.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.177+0000 7f8383b9a700 1 -- 172.21.15.16:0/758326480 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f837c09e410 msgr2=0x7f837c133f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:58.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.177+0000 7f8383b9a700 1 --2- 172.21.15.16:0/758326480 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f837c09e410 0x7f837c133f30 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f8374009e80 tx=0x7f837400b330 comp rx=0 tx=0).stop 2023-09-30T22:35:58.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.177+0000 7f8383b9a700 1 -- 172.21.15.16:0/758326480 shutdown_connections 2023-09-30T22:35:58.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.177+0000 7f8383b9a700 1 --2- 172.21.15.16:0/758326480 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f837c09d6f0 0x7f837c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:58.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.177+0000 7f8383b9a700 1 --2- 172.21.15.16:0/758326480 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f837c09e410 0x7f837c133f30 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:58.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.177+0000 7f8383b9a700 1 --2- 172.21.15.16:0/758326480 >> v2:172.21.15.16:3300/0 conn(0x7f837c09e000 0x7f837c1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:58.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.177+0000 7f8383b9a700 1 -- 172.21.15.16:0/758326480 >> 172.21.15.16:0/758326480 conn(0x7f837c066f50 msgr2=0x7f837c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:58.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.177+0000 7f8383b9a700 1 -- 172.21.15.16:0/758326480 shutdown_connections 2023-09-30T22:35:58.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.177+0000 7f8383b9a700 1 -- 172.21.15.16:0/758326480 wait complete. 2023-09-30T22:35:58.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.178+0000 7f8383b9a700 1 Processor -- start 2023-09-30T22:35:58.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.178+0000 7f8383b9a700 1 -- start start 2023-09-30T22:35:58.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.179+0000 7f8383b9a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f837c09d6f0 0x7f837c1c5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:58.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.179+0000 7f8383b9a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f837c09e000 0x7f837c1c5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:58.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.179+0000 7f8383b9a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f837c09e410 0x7f837c1c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:58.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.179+0000 7f8383b9a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f837c1a7240 con 0x7f837c09e410 2023-09-30T22:35:58.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.179+0000 7f8383b9a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f837c1c5ad0 con 0x7f837c09d6f0 2023-09-30T22:35:58.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.179+0000 7f8383b9a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f837c1a9820 con 0x7f837c09e000 2023-09-30T22:35:58.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.179+0000 7f8381936700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f837c09d6f0 0x7f837c1c5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:58.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.179+0000 7f8381135700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f837c09e000 0x7f837c1c5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:58.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.179+0000 7f8381936700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f837c09d6f0 0x7f837c1c5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47370/0 (socket says 172.21.15.16:47370) 2023-09-30T22:35:58.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.179+0000 7f8381936700 1 -- 172.21.15.16:0/3588339769 learned_addr learned my addr 172.21.15.16:0/3588339769 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:58.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.179+0000 7f8382137700 1 --2- 172.21.15.16:0/3588339769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f837c09e410 0x7f837c1c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:58.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.179+0000 7f8381936700 1 -- 172.21.15.16:0/3588339769 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f837c09e000 msgr2=0x7f837c1c5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:58.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.179+0000 7f8381936700 1 --2- 172.21.15.16:0/3588339769 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f837c09e000 0x7f837c1c5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:58.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.179+0000 7f8381936700 1 -- 172.21.15.16:0/3588339769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f837c09e410 msgr2=0x7f837c1c8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:58.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.179+0000 7f8381936700 1 --2- 172.21.15.16:0/3588339769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f837c09e410 0x7f837c1c8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:58.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.179+0000 7f8381936700 1 -- 172.21.15.16:0/3588339769 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83780094a0 con 0x7f837c09d6f0 2023-09-30T22:35:58.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.179+0000 7f8381135700 1 --2- 172.21.15.16:0/3588339769 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f837c09e000 0x7f837c1c5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:58.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.179+0000 7f8382137700 1 --2- 172.21.15.16:0/3588339769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f837c09e410 0x7f837c1c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:58.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.179+0000 7f8381936700 1 --2- 172.21.15.16:0/3588339769 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f837c09d6f0 0x7f837c1c5050 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f837800ec10 tx=0x7f837800ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:58.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.179+0000 7f83727fc700 1 -- 172.21.15.16:0/3588339769 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f837800c850 con 0x7f837c09d6f0 2023-09-30T22:35:58.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.180+0000 7f83727fc700 1 -- 172.21.15.16:0/3588339769 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f837800c9b0 con 0x7f837c09d6f0 2023-09-30T22:35:58.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.180+0000 7f8383b9a700 1 -- 172.21.15.16:0/3588339769 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f836c0094a0 con 0x7f837c09d6f0 2023-09-30T22:35:58.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.180+0000 7f83727fc700 1 -- 172.21.15.16:0/3588339769 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f837800cd20 con 0x7f837c09d6f0 2023-09-30T22:35:58.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.180+0000 7f8383b9a700 1 -- 172.21.15.16:0/3588339769 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8374009b30 con 0x7f837c09d6f0 2023-09-30T22:35:58.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.180+0000 7f83727fc700 1 -- 172.21.15.16:0/3588339769 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8378013470 con 0x7f837c09d6f0 2023-09-30T22:35:58.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.181+0000 7f83727fc700 1 -- 172.21.15.16:0/3588339769 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f837803f7a0 con 0x7f837c09d6f0 2023-09-30T22:35:58.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.180+0000 7f8383b9a700 1 -- 172.21.15.16:0/3588339769 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f837c074ea0 con 0x7f837c09d6f0 2023-09-30T22:35:58.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.185+0000 7f83727fc700 1 -- 172.21.15.16:0/3588339769 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8378021020 con 0x7f837c09d6f0 2023-09-30T22:35:58.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.388+0000 7f8383b9a700 1 -- 172.21.15.16:0/3588339769 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f837c06c360 con 0x7f837c09d6f0 2023-09-30T22:35:58.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.389+0000 7f83727fc700 1 -- 172.21.15.16:0/3588339769 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8378013720 con 0x7f837c09d6f0 2023-09-30T22:35:58.389 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:35:58.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.391+0000 7f8383b9a700 1 -- 172.21.15.16:0/3588339769 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f837c09d6f0 msgr2=0x7f837c1c5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:58.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.391+0000 7f8383b9a700 1 --2- 172.21.15.16:0/3588339769 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f837c09d6f0 0x7f837c1c5050 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f837800ec10 tx=0x7f837800ef20 comp rx=0 tx=0).stop 2023-09-30T22:35:58.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.391+0000 7f8383b9a700 1 -- 172.21.15.16:0/3588339769 shutdown_connections 2023-09-30T22:35:58.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.391+0000 7f8383b9a700 1 --2- 172.21.15.16:0/3588339769 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f837c09d6f0 0x7f837c1c5050 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:58.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.391+0000 7f8383b9a700 1 --2- 172.21.15.16:0/3588339769 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f837c09e410 0x7f837c1c8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:58.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.391+0000 7f8383b9a700 1 --2- 172.21.15.16:0/3588339769 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f837c09e000 0x7f837c1c5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:58.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.391+0000 7f8383b9a700 1 -- 172.21.15.16:0/3588339769 >> 172.21.15.16:0/3588339769 conn(0x7f837c066f50 msgr2=0x7f837c1bae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:58.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.391+0000 7f8383b9a700 1 -- 172.21.15.16:0/3588339769 shutdown_connections 2023-09-30T22:35:58.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:58.391+0000 7f8383b9a700 1 -- 172.21.15.16:0/3588339769 wait complete. 2023-09-30T22:35:58.405 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:35:58.954 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:35:58.984 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:35:58.996 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:35:59.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.104+0000 7efecc754700 1 Processor -- start 2023-09-30T22:35:59.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.104+0000 7efecc754700 1 -- start start 2023-09-30T22:35:59.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.105+0000 7efecc754700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efec409e930 0x7efec409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:59.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.105+0000 7efecc754700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efec409f240 0x7efec41317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:59.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.105+0000 7efecc754700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efec409f650 0x7efec4131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:59.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.105+0000 7efecc754700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efec4070940 con 0x7efec409f240 2023-09-30T22:35:59.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.105+0000 7efecc754700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efec406fd60 con 0x7efec409f650 2023-09-30T22:35:59.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.105+0000 7efecc754700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efec406fa80 con 0x7efec409e930 2023-09-30T22:35:59.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.105+0000 7efeca4f0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efec409e930 0x7efec409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:59.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.105+0000 7efecacf1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efec409f650 0x7efec4131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:59.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.105+0000 7efeca4f0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efec409e930 0x7efec409ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:40966/0 (socket says 172.21.15.16:40966) 2023-09-30T22:35:59.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.105+0000 7efeca4f0700 1 -- 172.21.15.16:0/704669553 learned_addr learned my addr 172.21.15.16:0/704669553 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:59.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.105+0000 7efecacf1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efec409f650 0x7efec4131d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47372/0 (socket says 172.21.15.16:47372) 2023-09-30T22:35:59.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.105+0000 7efec9cef700 1 --2- 172.21.15.16:0/704669553 >> v2:172.21.15.16:3300/0 conn(0x7efec409f240 0x7efec41317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:59.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.106+0000 7efecacf1700 1 -- 172.21.15.16:0/704669553 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efec409e930 msgr2=0x7efec409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:59.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.106+0000 7efecacf1700 1 --2- 172.21.15.16:0/704669553 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efec409e930 0x7efec409ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:59.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.106+0000 7efecacf1700 1 -- 172.21.15.16:0/704669553 >> v2:172.21.15.16:3300/0 conn(0x7efec409f240 msgr2=0x7efec41317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:59.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.106+0000 7efecacf1700 1 --2- 172.21.15.16:0/704669553 >> v2:172.21.15.16:3300/0 conn(0x7efec409f240 0x7efec41317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:59.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.106+0000 7efecacf1700 1 -- 172.21.15.16:0/704669553 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efec409fb60 con 0x7efec409f650 2023-09-30T22:35:59.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.106+0000 7efeca4f0700 1 --2- 172.21.15.16:0/704669553 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efec409e930 0x7efec409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:59.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.106+0000 7efec9cef700 1 --2- 172.21.15.16:0/704669553 >> v2:172.21.15.16:3300/0 conn(0x7efec409f240 0x7efec41317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:59.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.106+0000 7efecacf1700 1 --2- 172.21.15.16:0/704669553 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efec409f650 0x7efec4131d30 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7efebc009840 tx=0x7efebc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc3428dcb8000730 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:59.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.106+0000 7efec94ee700 1 -- 172.21.15.16:0/704669553 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efebc011830 con 0x7efec409f650 2023-09-30T22:35:59.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.106+0000 7efec94ee700 1 -- 172.21.15.16:0/704669553 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efebc011990 con 0x7efec409f650 2023-09-30T22:35:59.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.106+0000 7efec94ee700 1 -- 172.21.15.16:0/704669553 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efebc011d20 con 0x7efec409f650 2023-09-30T22:35:59.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.107+0000 7efecc754700 1 -- 172.21.15.16:0/704669553 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efec409f650 msgr2=0x7efec4131d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:59.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.107+0000 7efecc754700 1 --2- 172.21.15.16:0/704669553 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efec409f650 0x7efec4131d30 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7efebc009840 tx=0x7efebc009b50 comp rx=0 tx=0).stop 2023-09-30T22:35:59.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.107+0000 7efecc754700 1 -- 172.21.15.16:0/704669553 shutdown_connections 2023-09-30T22:35:59.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.107+0000 7efecc754700 1 --2- 172.21.15.16:0/704669553 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efec409f650 0x7efec4131d30 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:59.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.107+0000 7efecc754700 1 --2- 172.21.15.16:0/704669553 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efec409e930 0x7efec409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:59.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.107+0000 7efecc754700 1 --2- 172.21.15.16:0/704669553 >> v2:172.21.15.16:3300/0 conn(0x7efec409f240 0x7efec41317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:59.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.107+0000 7efecc754700 1 -- 172.21.15.16:0/704669553 >> 172.21.15.16:0/704669553 conn(0x7efec40a4210 msgr2=0x7efec40a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:59.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.107+0000 7efecc754700 1 -- 172.21.15.16:0/704669553 shutdown_connections 2023-09-30T22:35:59.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.107+0000 7efecc754700 1 -- 172.21.15.16:0/704669553 wait complete. 2023-09-30T22:35:59.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.108+0000 7efecc754700 1 Processor -- start 2023-09-30T22:35:59.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.108+0000 7efecc754700 1 -- start start 2023-09-30T22:35:59.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.109+0000 7efecc754700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efec409e930 0x7efec41a3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:59.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.109+0000 7efecc754700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efec409f240 0x7efec41a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:59.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.109+0000 7efecc754700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efec409f650 0x7efec41a4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:59.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.109+0000 7efecc754700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efec41a80a0 con 0x7efec409f650 2023-09-30T22:35:59.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.109+0000 7efecc754700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efec41a6ee0 con 0x7efec409f240 2023-09-30T22:35:59.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.109+0000 7efecc754700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efec40a3510 con 0x7efec409e930 2023-09-30T22:35:59.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.109+0000 7efeca4f0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efec409e930 0x7efec41a3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:59.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.109+0000 7efecacf1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efec409f650 0x7efec41a4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:59.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.109+0000 7efeca4f0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efec409e930 0x7efec41a3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:40980/0 (socket says 172.21.15.16:40980) 2023-09-30T22:35:59.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.109+0000 7efecacf1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efec409f650 0x7efec41a4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39600/0 (socket says 172.21.15.16:39600) 2023-09-30T22:35:59.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.109+0000 7efeca4f0700 1 -- 172.21.15.16:0/2270149066 learned_addr learned my addr 172.21.15.16:0/2270149066 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:59.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.109+0000 7efeca4f0700 1 -- 172.21.15.16:0/2270149066 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efec409f240 msgr2=0x7efec41a41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:59.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.109+0000 7efeca4f0700 1 --2- 172.21.15.16:0/2270149066 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efec409f240 0x7efec41a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:59.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.109+0000 7efeca4f0700 1 -- 172.21.15.16:0/2270149066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efec409f650 msgr2=0x7efec41a4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:59.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.109+0000 7efeca4f0700 1 --2- 172.21.15.16:0/2270149066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efec409f650 0x7efec41a4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:59.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.109+0000 7efeca4f0700 1 -- 172.21.15.16:0/2270149066 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb4009b30 con 0x7efec409e930 2023-09-30T22:35:59.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.109+0000 7efeca4f0700 1 --2- 172.21.15.16:0/2270149066 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efec409e930 0x7efec41a3c80 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7efeb400ef30 tx=0x7efeb400b390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:59.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.110+0000 7efec2ffd700 1 -- 172.21.15.16:0/2270149066 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb4010040 con 0x7efec409e930 2023-09-30T22:35:59.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.110+0000 7efec2ffd700 1 -- 172.21.15.16:0/2270149066 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeb400bba0 con 0x7efec409e930 2023-09-30T22:35:59.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.110+0000 7efec2ffd700 1 -- 172.21.15.16:0/2270149066 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeb4014400 con 0x7efec409e930 2023-09-30T22:35:59.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.110+0000 7efecc754700 1 -- 172.21.15.16:0/2270149066 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeb80094a0 con 0x7efec409e930 2023-09-30T22:35:59.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.110+0000 7efecc754700 1 -- 172.21.15.16:0/2270149066 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efebc0094a0 con 0x7efec409e930 2023-09-30T22:35:59.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.111+0000 7efec2ffd700 1 -- 172.21.15.16:0/2270149066 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7efeb4014560 con 0x7efec409e930 2023-09-30T22:35:59.118 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.111+0000 7efecc754700 1 -- 172.21.15.16:0/2270149066 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efec41a0b30 con 0x7efec409e930 2023-09-30T22:35:59.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.111+0000 7efec2ffd700 1 -- 172.21.15.16:0/2270149066 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7efeb403f8d0 con 0x7efec409e930 2023-09-30T22:35:59.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.120+0000 7efec2ffd700 1 -- 172.21.15.16:0/2270149066 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7efeb4014810 con 0x7efec409e930 2023-09-30T22:35:59.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.144+0000 7fd497724700 1 Processor -- start 2023-09-30T22:35:59.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.144+0000 7fd497724700 1 -- start start 2023-09-30T22:35:59.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.145+0000 7fd497724700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd49006b250 0x7fd490136840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:59.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.145+0000 7fd497724700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd490136d80 0x7fd490139160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:59.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.145+0000 7fd497724700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd49009ece0 con 0x7fd49006a770 2023-09-30T22:35:59.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.145+0000 7fd497724700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd49006b640 con 0x7fd490136d80 2023-09-30T22:35:59.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.145+0000 7fd497724700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd49009e760 con 0x7fd49006b250 2023-09-30T22:35:59.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.145+0000 7fd494cbf700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd49006b250 0x7fd490136840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:59.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.145+0000 7fd4954c0700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd49006a770 0x7fd49006ab40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38236/0 (socket says 172.21.15.16:38236) 2023-09-30T22:35:59.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.145+0000 7fd4954c0700 1 -- 172.21.15.16:0/2436312538 learned_addr learned my addr 172.21.15.16:0/2436312538 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:35:59.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.145+0000 7fd495cc1700 1 --2- 172.21.15.16:0/2436312538 >> v2:172.21.15.16:3300/0 conn(0x7fd490136d80 0x7fd490139160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:59.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.145+0000 7fd487fff700 1 -- 172.21.15.16:0/2436312538 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1038180661 0 0) 0x7fd49009ece0 con 0x7fd49006a770 2023-09-30T22:35:59.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.145+0000 7fd487fff700 1 -- 172.21.15.16:0/2436312538 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd47c0035b0 con 0x7fd49006a770 2023-09-30T22:35:59.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.145+0000 7fd495cc1700 1 -- 172.21.15.16:0/2436312538 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd49006b250 msgr2=0x7fd490136840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:59.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.145+0000 7fd495cc1700 1 --2- 172.21.15.16:0/2436312538 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd49006b250 0x7fd490136840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:59.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.145+0000 7fd495cc1700 1 -- 172.21.15.16:0/2436312538 >> v1:172.21.15.16:6789/0 conn(0x7fd49006a770 legacy=0x7fd49006ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:59.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.145+0000 7fd495cc1700 1 -- 172.21.15.16:0/2436312538 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd49009e040 con 0x7fd490136d80 2023-09-30T22:35:59.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.146+0000 7fd494cbf700 1 --2- 172.21.15.16:0/2436312538 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd49006b250 0x7fd490136840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:59.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.146+0000 7fd495cc1700 1 --2- 172.21.15.16:0/2436312538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd490136d80 0x7fd490139160 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fd488009d90 tx=0x7fd48800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e1450d6973eca40 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:59.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.146+0000 7fd487fff700 1 -- 172.21.15.16:0/2436312538 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48800e070 con 0x7fd490136d80 2023-09-30T22:35:59.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.146+0000 7fd487fff700 1 -- 172.21.15.16:0/2436312538 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd48800bb20 con 0x7fd490136d80 2023-09-30T22:35:59.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.146+0000 7fd487fff700 1 -- 172.21.15.16:0/2436312538 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd488012430 con 0x7fd490136d80 2023-09-30T22:35:59.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.147+0000 7fd497724700 1 -- 172.21.15.16:0/2436312538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd490136d80 msgr2=0x7fd490139160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:59.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.147+0000 7fd497724700 1 --2- 172.21.15.16:0/2436312538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd490136d80 0x7fd490139160 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fd488009d90 tx=0x7fd48800b330 comp rx=0 tx=0).stop 2023-09-30T22:35:59.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.147+0000 7fd497724700 1 -- 172.21.15.16:0/2436312538 shutdown_connections 2023-09-30T22:35:59.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.147+0000 7fd497724700 1 --2- 172.21.15.16:0/2436312538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd490136d80 0x7fd490139160 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:59.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.147+0000 7fd497724700 1 --2- 172.21.15.16:0/2436312538 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd49006b250 0x7fd490136840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:59.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.147+0000 7fd497724700 1 -- 172.21.15.16:0/2436312538 >> 172.21.15.16:0/2436312538 conn(0x7fd490066f50 msgr2=0x7fd49009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:59.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.147+0000 7fd497724700 1 -- 172.21.15.16:0/2436312538 shutdown_connections 2023-09-30T22:35:59.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.147+0000 7fd497724700 1 -- 172.21.15.16:0/2436312538 wait complete. 2023-09-30T22:35:59.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.148+0000 7fd497724700 1 Processor -- start 2023-09-30T22:35:59.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.149+0000 7fd497724700 1 -- start start 2023-09-30T22:35:59.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.149+0000 7fd497724700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd49006a770 0x7fd4901c7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:59.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.149+0000 7fd497724700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd49006b250 0x7fd4901c84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:59.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.149+0000 7fd497724700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd490136d80 0x7fd4901c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:59.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.149+0000 7fd497724700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd490071e60 con 0x7fd49006a770 2023-09-30T22:35:59.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.149+0000 7fd497724700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd49009c110 con 0x7fd490136d80 2023-09-30T22:35:59.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.149+0000 7fd497724700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd490124090 con 0x7fd49006b250 2023-09-30T22:35:59.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.149+0000 7fd4954c0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd49006a770 0x7fd4901c7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:59.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.149+0000 7fd4954c0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd49006a770 0x7fd4901c7f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39606/0 (socket says 172.21.15.16:39606) 2023-09-30T22:35:59.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.149+0000 7fd495cc1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd490136d80 0x7fd4901c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:59.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.149+0000 7fd4954c0700 1 -- 172.21.15.16:0/2247305021 learned_addr learned my addr 172.21.15.16:0/2247305021 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:59.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.150+0000 7fd494cbf700 1 --2- 172.21.15.16:0/2247305021 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd49006b250 0x7fd4901c84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:59.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.150+0000 7fd495cc1700 1 -- 172.21.15.16:0/2247305021 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd49006b250 msgr2=0x7fd4901c84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:59.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.150+0000 7fd495cc1700 1 --2- 172.21.15.16:0/2247305021 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd49006b250 0x7fd4901c84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:59.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.150+0000 7fd495cc1700 1 -- 172.21.15.16:0/2247305021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd49006a770 msgr2=0x7fd4901c7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:59.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.150+0000 7fd495cc1700 1 --2- 172.21.15.16:0/2247305021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd49006a770 0x7fd4901c7f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:59.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.150+0000 7fd495cc1700 1 -- 172.21.15.16:0/2247305021 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4800094a0 con 0x7fd490136d80 2023-09-30T22:35:59.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.150+0000 7fd495cc1700 1 --2- 172.21.15.16:0/2247305021 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd490136d80 0x7fd4901c8a10 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fd488016000 tx=0x7fd488004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:59.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.150+0000 7fd485ffb700 1 -- 172.21.15.16:0/2247305021 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd488012430 con 0x7fd490136d80 2023-09-30T22:35:59.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.150+0000 7fd485ffb700 1 -- 172.21.15.16:0/2247305021 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd488020070 con 0x7fd490136d80 2023-09-30T22:35:59.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.150+0000 7fd485ffb700 1 -- 172.21.15.16:0/2247305021 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd488004280 con 0x7fd490136d80 2023-09-30T22:35:59.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.150+0000 7fd497724700 1 -- 172.21.15.16:0/2247305021 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd488009a70 con 0x7fd490136d80 2023-09-30T22:35:59.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.151+0000 7fd497724700 1 -- 172.21.15.16:0/2247305021 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4901340a0 con 0x7fd490136d80 2023-09-30T22:35:59.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.151+0000 7fd485ffb700 1 -- 172.21.15.16:0/2247305021 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd4880043e0 con 0x7fd490136d80 2023-09-30T22:35:59.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.152+0000 7fd485ffb700 1 -- 172.21.15.16:0/2247305021 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd488022070 con 0x7fd490136d80 2023-09-30T22:35:59.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.152+0000 7fd497724700 1 -- 172.21.15.16:0/2247305021 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd490123cd0 con 0x7fd490136d80 2023-09-30T22:35:59.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.154+0000 7f1c6959f700 1 Processor -- start 2023-09-30T22:35:59.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.154+0000 7f1c6959f700 1 -- start start 2023-09-30T22:35:59.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.154+0000 7f1c6959f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1c6409d6f0 0x7f1c6409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:59.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.154+0000 7f1c6959f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1c6409e440 0x7f1c64134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:59.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.154+0000 7f1c6959f700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c6409b120 con 0x7f1c6409e000 2023-09-30T22:35:59.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.154+0000 7f1c6959f700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c6406b460 con 0x7f1c6409d6f0 2023-09-30T22:35:59.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.154+0000 7f1c6959f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6409a7e0 con 0x7f1c6409e440 2023-09-30T22:35:59.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.155+0000 7f1c62ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1c6409d6f0 0x7f1c6409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:59.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.155+0000 7f1c62ffd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1c6409d6f0 0x7f1c6409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39612/0 (socket says 172.21.15.16:39612) 2023-09-30T22:35:59.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.155+0000 7f1c637fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1c6409e440 0x7f1c64134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:59.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.155+0000 7f1c62ffd700 1 -- 172.21.15.16:0/3451928259 learned_addr learned my addr 172.21.15.16:0/3451928259 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:59.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.155+0000 7f1c627fc700 1 --1- 172.21.15.16:0/3451928259 >> v1:172.21.15.16:6789/0 conn(0x7f1c6409e000 0x7f1c641339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38246/0 (socket says 172.21.15.16:38246) 2023-09-30T22:35:59.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.155+0000 7f1c61ffb700 1 -- 172.21.15.16:0/3451928259 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3487673217 0 0) 0x7f1c6409b120 con 0x7f1c6409e000 2023-09-30T22:35:59.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.155+0000 7f1c637fe700 1 -- 172.21.15.16:0/3451928259 >> v2:172.21.15.16:3300/0 conn(0x7f1c6409d6f0 msgr2=0x7f1c6409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:59.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.155+0000 7f1c637fe700 1 --2- 172.21.15.16:0/3451928259 >> v2:172.21.15.16:3300/0 conn(0x7f1c6409d6f0 0x7f1c6409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:59.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.155+0000 7f1c637fe700 1 -- 172.21.15.16:0/3451928259 >> v1:172.21.15.16:6789/0 conn(0x7f1c6409e000 legacy=0x7f1c641339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:59.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.155+0000 7f1c637fe700 1 -- 172.21.15.16:0/3451928259 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c6409a500 con 0x7f1c6409e440 2023-09-30T22:35:59.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.155+0000 7f1c637fe700 1 --2- 172.21.15.16:0/3451928259 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1c6409e440 0x7f1c64134100 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c009dc0 tx=0x7f1c5c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=301eeb5ebb621979 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:59.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.157+0000 7f1c61ffb700 1 -- 172.21.15.16:0/3451928259 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c00e070 con 0x7f1c6409e440 2023-09-30T22:35:59.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.157+0000 7f1c61ffb700 1 -- 172.21.15.16:0/3451928259 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c5c00bae0 con 0x7f1c6409e440 2023-09-30T22:35:59.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.157+0000 7f1c61ffb700 1 -- 172.21.15.16:0/3451928259 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5c00be90 con 0x7f1c6409e440 2023-09-30T22:35:59.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.157+0000 7fd485ffb700 1 -- 172.21.15.16:0/2247305021 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd488004690 con 0x7fd490136d80 2023-09-30T22:35:59.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.158+0000 7f1c6959f700 1 -- 172.21.15.16:0/3451928259 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1c6409e440 msgr2=0x7f1c64134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:59.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.158+0000 7f1c6959f700 1 --2- 172.21.15.16:0/3451928259 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1c6409e440 0x7f1c64134100 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c009dc0 tx=0x7f1c5c00b330 comp rx=0 tx=0).stop 2023-09-30T22:35:59.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.158+0000 7f1c6959f700 1 -- 172.21.15.16:0/3451928259 shutdown_connections 2023-09-30T22:35:59.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.158+0000 7f1c6959f700 1 --2- 172.21.15.16:0/3451928259 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1c6409e440 0x7f1c64134100 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:59.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.158+0000 7f1c6959f700 1 --2- 172.21.15.16:0/3451928259 >> v2:172.21.15.16:3300/0 conn(0x7f1c6409d6f0 0x7f1c6409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:59.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.158+0000 7f1c6959f700 1 -- 172.21.15.16:0/3451928259 >> 172.21.15.16:0/3451928259 conn(0x7f1c64066f50 msgr2=0x7f1c6409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:59.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.158+0000 7f1c6959f700 1 -- 172.21.15.16:0/3451928259 shutdown_connections 2023-09-30T22:35:59.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.158+0000 7f1c6959f700 1 -- 172.21.15.16:0/3451928259 wait complete. 2023-09-30T22:35:59.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.159+0000 7f1c6959f700 1 Processor -- start 2023-09-30T22:35:59.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.160+0000 7f1c6959f700 1 -- start start 2023-09-30T22:35:59.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.160+0000 7f1c6959f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1c6409d6f0 0x7f1c641cc950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:59.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.160+0000 7f1c6959f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c6409e000 0x7f1c641cce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:59.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.160+0000 7f1c6959f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1c6409e440 0x7f1c6411b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:59.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.160+0000 7f1c6959f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c641ac6e0 con 0x7f1c6409e000 2023-09-30T22:35:59.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.160+0000 7f1c6959f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c641c13e0 con 0x7f1c6409d6f0 2023-09-30T22:35:59.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.160+0000 7f1c6959f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1c640a2d20 con 0x7f1c6409e440 2023-09-30T22:35:59.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.160+0000 7f1c62ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1c6409d6f0 0x7f1c641cc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:59.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.160+0000 7f1c62ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1c6409d6f0 0x7f1c641cc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47420/0 (socket says 172.21.15.16:47420) 2023-09-30T22:35:59.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.160+0000 7f1c62ffd700 1 -- 172.21.15.16:0/3595205677 learned_addr learned my addr 172.21.15.16:0/3595205677 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:59.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.160+0000 7f1c627fc700 1 --2- 172.21.15.16:0/3595205677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c6409e000 0x7f1c641cce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:59.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.160+0000 7f1c637fe700 1 --2- 172.21.15.16:0/3595205677 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1c6409e440 0x7f1c6411b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:59.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.160+0000 7f1c627fc700 1 -- 172.21.15.16:0/3595205677 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1c6409e440 msgr2=0x7f1c6411b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:59.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.160+0000 7f1c627fc700 1 --2- 172.21.15.16:0/3595205677 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1c6409e440 0x7f1c6411b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:59.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.160+0000 7f1c627fc700 1 -- 172.21.15.16:0/3595205677 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1c6409d6f0 msgr2=0x7f1c641cc950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:59.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.160+0000 7f1c627fc700 1 --2- 172.21.15.16:0/3595205677 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1c6409d6f0 0x7f1c641cc950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:59.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.160+0000 7f1c627fc700 1 -- 172.21.15.16:0/3595205677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c5c009a70 con 0x7f1c6409e000 2023-09-30T22:35:59.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.160+0000 7f1c637fe700 1 --2- 172.21.15.16:0/3595205677 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1c6409e440 0x7f1c6411b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:59.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.161+0000 7f1c627fc700 1 --2- 172.21.15.16:0/3595205677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c6409e000 0x7f1c641cce90 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f1c5800c4a0 tx=0x7f1c5800c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:59.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.161+0000 7f1c4b7fe700 1 -- 172.21.15.16:0/3595205677 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c58004c80 con 0x7f1c6409e000 2023-09-30T22:35:59.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.161+0000 7f1c4b7fe700 1 -- 172.21.15.16:0/3595205677 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c58004de0 con 0x7f1c6409e000 2023-09-30T22:35:59.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.161+0000 7f1c6959f700 1 -- 172.21.15.16:0/3595205677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c641bd980 con 0x7f1c6409e000 2023-09-30T22:35:59.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.161+0000 7f1c4b7fe700 1 -- 172.21.15.16:0/3595205677 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c58007310 con 0x7f1c6409e000 2023-09-30T22:35:59.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.161+0000 7f1c6959f700 1 -- 172.21.15.16:0/3595205677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c6409c710 con 0x7f1c6409e000 2023-09-30T22:35:59.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.162+0000 7f1c6959f700 1 -- 172.21.15.16:0/3595205677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c6406abf0 con 0x7f1c6409e000 2023-09-30T22:35:59.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.162+0000 7f1c4b7fe700 1 -- 172.21.15.16:0/3595205677 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1c58007470 con 0x7f1c6409e000 2023-09-30T22:35:59.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.162+0000 7f1c4b7fe700 1 -- 172.21.15.16:0/3595205677 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f1c5803f220 con 0x7f1c6409e000 2023-09-30T22:35:59.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.167+0000 7f1c4b7fe700 1 -- 172.21.15.16:0/3595205677 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1c58007720 con 0x7f1c6409e000 2023-09-30T22:35:59.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.354+0000 7efecc754700 1 -- 172.21.15.16:0/2270149066 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7efec4074ea0 con 0x7efec409e930 2023-09-30T22:35:59.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.354+0000 7efec2ffd700 1 -- 172.21.15.16:0/2270149066 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7efeb401bb00 con 0x7efec409e930 2023-09-30T22:35:59.355 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:35:59.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.357+0000 7efecc754700 1 -- 172.21.15.16:0/2270149066 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efec409e930 msgr2=0x7efec41a3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:59.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.357+0000 7efecc754700 1 --2- 172.21.15.16:0/2270149066 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efec409e930 0x7efec41a3c80 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7efeb400ef30 tx=0x7efeb400b390 comp rx=0 tx=0).stop 2023-09-30T22:35:59.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.357+0000 7efecc754700 1 -- 172.21.15.16:0/2270149066 shutdown_connections 2023-09-30T22:35:59.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.357+0000 7efecc754700 1 --2- 172.21.15.16:0/2270149066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efec409f650 0x7efec41a4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:59.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.357+0000 7efecc754700 1 --2- 172.21.15.16:0/2270149066 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efec409e930 0x7efec41a3c80 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:59.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.357+0000 7efecc754700 1 --2- 172.21.15.16:0/2270149066 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efec409f240 0x7efec41a41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:59.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.357+0000 7efecc754700 1 -- 172.21.15.16:0/2270149066 >> 172.21.15.16:0/2270149066 conn(0x7efec40a4210 msgr2=0x7efec41b8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:59.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.357+0000 7efecc754700 1 -- 172.21.15.16:0/2270149066 shutdown_connections 2023-09-30T22:35:59.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.357+0000 7efecc754700 1 -- 172.21.15.16:0/2270149066 wait complete. 2023-09-30T22:35:59.372 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:35:59.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.372+0000 7f1c6959f700 1 -- 172.21.15.16:0/3595205677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1c6409f420 con 0x7f1c6409e000 2023-09-30T22:35:59.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.373+0000 7f1c4b7fe700 1 -- 172.21.15.16:0/3595205677 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1c58011360 con 0x7f1c6409e000 2023-09-30T22:35:59.373 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:35:59.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.375+0000 7f1c6959f700 1 -- 172.21.15.16:0/3595205677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c6409e000 msgr2=0x7f1c641cce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:59.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.375+0000 7f1c6959f700 1 --2- 172.21.15.16:0/3595205677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c6409e000 0x7f1c641cce90 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f1c5800c4a0 tx=0x7f1c5800c7b0 comp rx=0 tx=0).stop 2023-09-30T22:35:59.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.375+0000 7f1c6959f700 1 -- 172.21.15.16:0/3595205677 shutdown_connections 2023-09-30T22:35:59.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.375+0000 7f1c6959f700 1 --2- 172.21.15.16:0/3595205677 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1c6409d6f0 0x7f1c641cc950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:59.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.375+0000 7f1c6959f700 1 --2- 172.21.15.16:0/3595205677 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1c6409e440 0x7f1c6411b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:59.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.375+0000 7f1c6959f700 1 --2- 172.21.15.16:0/3595205677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c6409e000 0x7f1c641cce90 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:59.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.375+0000 7f1c6959f700 1 -- 172.21.15.16:0/3595205677 >> 172.21.15.16:0/3595205677 conn(0x7f1c64066f50 msgr2=0x7f1c641c00e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:59.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.375+0000 7f1c6959f700 1 -- 172.21.15.16:0/3595205677 shutdown_connections 2023-09-30T22:35:59.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.375+0000 7f1c6959f700 1 -- 172.21.15.16:0/3595205677 wait complete. 2023-09-30T22:35:59.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.384+0000 7fd497724700 1 -- 172.21.15.16:0/2247305021 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd490074ea0 con 0x7fd490136d80 2023-09-30T22:35:59.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.385+0000 7fd485ffb700 1 -- 172.21.15.16:0/2247305021 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd4880263e0 con 0x7fd490136d80 2023-09-30T22:35:59.385 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:35:59.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.387+0000 7fd497724700 1 -- 172.21.15.16:0/2247305021 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd490136d80 msgr2=0x7fd4901c8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:59.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.387+0000 7fd497724700 1 --2- 172.21.15.16:0/2247305021 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd490136d80 0x7fd4901c8a10 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fd488016000 tx=0x7fd488004ad0 comp rx=0 tx=0).stop 2023-09-30T22:35:59.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.387+0000 7fd497724700 1 -- 172.21.15.16:0/2247305021 shutdown_connections 2023-09-30T22:35:59.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.387+0000 7fd497724700 1 --2- 172.21.15.16:0/2247305021 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd490136d80 0x7fd4901c8a10 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:59.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.387+0000 7fd497724700 1 --2- 172.21.15.16:0/2247305021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd49006a770 0x7fd4901c7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:59.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.387+0000 7fd497724700 1 --2- 172.21.15.16:0/2247305021 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd49006b250 0x7fd4901c84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:59.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.387+0000 7fd497724700 1 -- 172.21.15.16:0/2247305021 >> 172.21.15.16:0/2247305021 conn(0x7fd490066f50 msgr2=0x7fd4901346a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:59.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.387+0000 7fd497724700 1 -- 172.21.15.16:0/2247305021 shutdown_connections 2023-09-30T22:35:59.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.387+0000 7fd497724700 1 -- 172.21.15.16:0/2247305021 wait complete. 2023-09-30T22:35:59.389 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:35:59.401 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:35:59.407 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:35:59.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.582+0000 7fef724d0700 1 Processor -- start 2023-09-30T22:35:59.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.582+0000 7fef724d0700 1 -- start start 2023-09-30T22:35:59.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.582+0000 7fef724d0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fef6c1316c0 0x7fef6c131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:59.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.582+0000 7fef724d0700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fef6c06ae80 0x7fef6c139550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:59.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.582+0000 7fef724d0700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef6c09e130 con 0x7fef6c09d9f0 2023-09-30T22:35:59.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.582+0000 7fef724d0700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef6c1312b0 con 0x7fef6c06ae80 2023-09-30T22:35:59.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.582+0000 7fef724d0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef6c09f6e0 con 0x7fef6c1316c0 2023-09-30T22:35:59.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.582+0000 7fef70a6d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fef6c06ae80 0x7fef6c139550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:59.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.582+0000 7fef6bfff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fef6c1316c0 0x7fef6c131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:59.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.582+0000 7fef70a6d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fef6c06ae80 0x7fef6c139550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39616/0 (socket says 172.21.15.16:39616) 2023-09-30T22:35:59.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.582+0000 7fef70a6d700 1 -- 172.21.15.16:0/3531318287 learned_addr learned my addr 172.21.15.16:0/3531318287 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:59.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.583+0000 7fef6bfff700 1 -- 172.21.15.16:0/3531318287 >> v2:172.21.15.16:3300/0 conn(0x7fef6c06ae80 msgr2=0x7fef6c139550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:35:59.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.583+0000 7fef6bfff700 1 --2- 172.21.15.16:0/3531318287 >> v2:172.21.15.16:3300/0 conn(0x7fef6c06ae80 0x7fef6c139550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:59.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.583+0000 7fef6bfff700 1 -- 172.21.15.16:0/3531318287 >> v1:172.21.15.16:6789/0 conn(0x7fef6c09d9f0 legacy=0x7fef6c06a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:59.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.583+0000 7fef6bfff700 1 -- 172.21.15.16:0/3531318287 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef6c09e760 con 0x7fef6c1316c0 2023-09-30T22:35:59.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.583+0000 7fef70a6d700 1 --2- 172.21.15.16:0/3531318287 >> v2:172.21.15.16:3300/0 conn(0x7fef6c06ae80 0x7fef6c139550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:35:59.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.583+0000 7fef6bfff700 1 --2- 172.21.15.16:0/3531318287 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fef6c1316c0 0x7fef6c131a90 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fef5c009840 tx=0x7fef5c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=785fa4fabd13e546 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:59.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.583+0000 7fef6affd700 1 -- 172.21.15.16:0/3531318287 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5c011830 con 0x7fef6c1316c0 2023-09-30T22:35:59.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.583+0000 7fef6affd700 1 -- 172.21.15.16:0/3531318287 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef5c011990 con 0x7fef6c1316c0 2023-09-30T22:35:59.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.583+0000 7fef6affd700 1 -- 172.21.15.16:0/3531318287 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5c011d20 con 0x7fef6c1316c0 2023-09-30T22:35:59.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.584+0000 7fef724d0700 1 -- 172.21.15.16:0/3531318287 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fef6c1316c0 msgr2=0x7fef6c131a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:59.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.584+0000 7fef724d0700 1 --2- 172.21.15.16:0/3531318287 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fef6c1316c0 0x7fef6c131a90 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fef5c009840 tx=0x7fef5c009b50 comp rx=0 tx=0).stop 2023-09-30T22:35:59.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.584+0000 7fef724d0700 1 -- 172.21.15.16:0/3531318287 shutdown_connections 2023-09-30T22:35:59.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.584+0000 7fef724d0700 1 --2- 172.21.15.16:0/3531318287 >> v2:172.21.15.16:3300/0 conn(0x7fef6c06ae80 0x7fef6c139550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:59.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.584+0000 7fef724d0700 1 --2- 172.21.15.16:0/3531318287 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fef6c1316c0 0x7fef6c131a90 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:59.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.584+0000 7fef724d0700 1 -- 172.21.15.16:0/3531318287 >> 172.21.15.16:0/3531318287 conn(0x7fef6c066f50 msgr2=0x7fef6c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:59.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.584+0000 7fef724d0700 1 -- 172.21.15.16:0/3531318287 shutdown_connections 2023-09-30T22:35:59.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.584+0000 7fef724d0700 1 -- 172.21.15.16:0/3531318287 wait complete. 2023-09-30T22:35:59.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.585+0000 7fef724d0700 1 Processor -- start 2023-09-30T22:35:59.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.585+0000 7fef724d0700 1 -- start start 2023-09-30T22:35:59.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.585+0000 7fef724d0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fef6c06ae80 0x7fef6c1c83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:59.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.585+0000 7fef724d0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef6c09d9f0 0x7fef6c1c88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:59.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.585+0000 7fef724d0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fef6c1316c0 0x7fef6c1c8e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:35:59.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.585+0000 7fef724d0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef6c072600 con 0x7fef6c09d9f0 2023-09-30T22:35:59.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.585+0000 7fef724d0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef6c0712f0 con 0x7fef6c06ae80 2023-09-30T22:35:59.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.585+0000 7fef724d0700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fef6c070330 con 0x7fef6c1316c0 2023-09-30T22:35:59.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.585+0000 7fef6bfff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fef6c06ae80 0x7fef6c1c83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:59.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.585+0000 7fef6bfff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fef6c06ae80 0x7fef6c1c83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47434/0 (socket says 172.21.15.16:47434) 2023-09-30T22:35:59.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.585+0000 7fef6bfff700 1 -- 172.21.15.16:0/4245586423 learned_addr learned my addr 172.21.15.16:0/4245586423 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:35:59.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.586+0000 7fef70a6d700 1 --2- 172.21.15.16:0/4245586423 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fef6c1316c0 0x7fef6c1c8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:59.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.586+0000 7fef6b7fe700 1 --2- 172.21.15.16:0/4245586423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef6c09d9f0 0x7fef6c1c88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:35:59.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.586+0000 7fef6bfff700 1 -- 172.21.15.16:0/4245586423 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fef6c1316c0 msgr2=0x7fef6c1c8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:59.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.586+0000 7fef6bfff700 1 --2- 172.21.15.16:0/4245586423 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fef6c1316c0 0x7fef6c1c8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:59.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.586+0000 7fef6bfff700 1 -- 172.21.15.16:0/4245586423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef6c09d9f0 msgr2=0x7fef6c1c88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:59.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.586+0000 7fef6bfff700 1 --2- 172.21.15.16:0/4245586423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef6c09d9f0 0x7fef6c1c88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:59.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.586+0000 7fef6bfff700 1 -- 172.21.15.16:0/4245586423 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef64009b30 con 0x7fef6c06ae80 2023-09-30T22:35:59.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.586+0000 7fef6bfff700 1 --2- 172.21.15.16:0/4245586423 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fef6c06ae80 0x7fef6c1c83a0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fef5c009fd0 tx=0x7fef5c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:35:59.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.586+0000 7fef68ff9700 1 -- 172.21.15.16:0/4245586423 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5c0036a0 con 0x7fef6c06ae80 2023-09-30T22:35:59.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.586+0000 7fef68ff9700 1 -- 172.21.15.16:0/4245586423 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef5c01f070 con 0x7fef6c06ae80 2023-09-30T22:35:59.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.586+0000 7fef724d0700 1 -- 172.21.15.16:0/4245586423 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef5c0094a0 con 0x7fef6c06ae80 2023-09-30T22:35:59.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.586+0000 7fef68ff9700 1 -- 172.21.15.16:0/4245586423 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5c003a60 con 0x7fef6c06ae80 2023-09-30T22:35:59.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.586+0000 7fef724d0700 1 -- 172.21.15.16:0/4245586423 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef6c06feb0 con 0x7fef6c06ae80 2023-09-30T22:35:59.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.587+0000 7fef68ff9700 1 -- 172.21.15.16:0/4245586423 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fef5c003bc0 con 0x7fef6c06ae80 2023-09-30T22:35:59.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.587+0000 7fef68ff9700 1 -- 172.21.15.16:0/4245586423 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fef5c021070 con 0x7fef6c06ae80 2023-09-30T22:35:59.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.587+0000 7fef724d0700 1 -- 172.21.15.16:0/4245586423 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef6c1313f0 con 0x7fef6c06ae80 2023-09-30T22:35:59.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.592+0000 7fef68ff9700 1 -- 172.21.15.16:0/4245586423 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fef5c015080 con 0x7fef6c06ae80 2023-09-30T22:35:59.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.798+0000 7fef724d0700 1 -- 172.21.15.16:0/4245586423 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fef6c12b6b0 con 0x7fef6c06ae80 2023-09-30T22:35:59.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.798+0000 7fef68ff9700 1 -- 172.21.15.16:0/4245586423 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fef5c01a950 con 0x7fef6c06ae80 2023-09-30T22:35:59.799 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:35:59.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.801+0000 7fef724d0700 1 -- 172.21.15.16:0/4245586423 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fef6c06ae80 msgr2=0x7fef6c1c83a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:35:59.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.801+0000 7fef724d0700 1 --2- 172.21.15.16:0/4245586423 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fef6c06ae80 0x7fef6c1c83a0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fef5c009fd0 tx=0x7fef5c004750 comp rx=0 tx=0).stop 2023-09-30T22:35:59.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.801+0000 7fef724d0700 1 -- 172.21.15.16:0/4245586423 shutdown_connections 2023-09-30T22:35:59.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.801+0000 7fef724d0700 1 --2- 172.21.15.16:0/4245586423 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fef6c06ae80 0x7fef6c1c83a0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:59.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.801+0000 7fef724d0700 1 --2- 172.21.15.16:0/4245586423 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fef6c1316c0 0x7fef6c1c8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:59.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.801+0000 7fef724d0700 1 --2- 172.21.15.16:0/4245586423 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef6c09d9f0 0x7fef6c1c88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:35:59.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.801+0000 7fef724d0700 1 -- 172.21.15.16:0/4245586423 >> 172.21.15.16:0/4245586423 conn(0x7fef6c066f50 msgr2=0x7fef6c0a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:35:59.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.801+0000 7fef724d0700 1 -- 172.21.15.16:0/4245586423 shutdown_connections 2023-09-30T22:35:59.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:35:59.801+0000 7fef724d0700 1 -- 172.21.15.16:0/4245586423 wait complete. 2023-09-30T22:35:59.815 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:36:00.372 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:36:00.390 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:36:00.402 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:36:00.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.523+0000 7f4d58e07700 1 Processor -- start 2023-09-30T22:36:00.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.523+0000 7f4d58e07700 1 -- start start 2023-09-30T22:36:00.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.524+0000 7f4d58e07700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4d541316c0 0x7f4d54131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:00.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.524+0000 7f4d58e07700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4d5409d9f0 0x7f4d5406a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:00.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.524+0000 7f4d58e07700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4d5406acb0 0x7f4d5406b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:00.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.524+0000 7f4d58e07700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d5409e130 con 0x7f4d5406acb0 2023-09-30T22:36:00.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.524+0000 7f4d58e07700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d541312b0 con 0x7f4d5409d9f0 2023-09-30T22:36:00.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.524+0000 7f4d58e07700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5409f6e0 con 0x7f4d541316c0 2023-09-30T22:36:00.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.524+0000 7f4d52d9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4d5406acb0 0x7f4d5406b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:00.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.524+0000 7f4d52d9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4d5406acb0 0x7f4d5406b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39634/0 (socket says 172.21.15.16:39634) 2023-09-30T22:36:00.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.524+0000 7f4d5259c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4d541316c0 0x7f4d54131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:00.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.524+0000 7f4d52d9d700 1 -- 172.21.15.16:0/1503343697 learned_addr learned my addr 172.21.15.16:0/1503343697 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:00.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.524+0000 7f4d49d9b700 1 --2- 172.21.15.16:0/1503343697 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4d5409d9f0 0x7f4d5406a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:00.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.524+0000 7f4d52d9d700 1 -- 172.21.15.16:0/1503343697 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4d541316c0 msgr2=0x7f4d54131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:00.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.524+0000 7f4d52d9d700 1 --2- 172.21.15.16:0/1503343697 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4d541316c0 0x7f4d54131a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:00.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.524+0000 7f4d52d9d700 1 -- 172.21.15.16:0/1503343697 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4d5409d9f0 msgr2=0x7f4d5406a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:00.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.524+0000 7f4d52d9d700 1 --2- 172.21.15.16:0/1503343697 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4d5409d9f0 0x7f4d5406a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:00.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.524+0000 7f4d52d9d700 1 -- 172.21.15.16:0/1503343697 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d5409e760 con 0x7f4d5406acb0 2023-09-30T22:36:00.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.524+0000 7f4d49d9b700 1 --2- 172.21.15.16:0/1503343697 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4d5409d9f0 0x7f4d5406a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:00.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.525+0000 7f4d52d9d700 1 --2- 172.21.15.16:0/1503343697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d5406acb0 0x7f4d5406b110 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c009e50 tx=0x7f4d4c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bd56543be816fdc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:00.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.525+0000 7f4d51d9b700 1 -- 172.21.15.16:0/1503343697 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c00e070 con 0x7f4d5406acb0 2023-09-30T22:36:00.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.525+0000 7f4d51d9b700 1 -- 172.21.15.16:0/1503343697 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d4c00b910 con 0x7f4d5406acb0 2023-09-30T22:36:00.527 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.525+0000 7f4d51d9b700 1 -- 172.21.15.16:0/1503343697 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c00bcc0 con 0x7f4d5406acb0 2023-09-30T22:36:00.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.526+0000 7f4d58e07700 1 -- 172.21.15.16:0/1503343697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d5406acb0 msgr2=0x7f4d5406b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:00.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.526+0000 7f4d58e07700 1 --2- 172.21.15.16:0/1503343697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d5406acb0 0x7f4d5406b110 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c009e50 tx=0x7f4d4c00b330 comp rx=0 tx=0).stop 2023-09-30T22:36:00.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.526+0000 7f4d58e07700 1 -- 172.21.15.16:0/1503343697 shutdown_connections 2023-09-30T22:36:00.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.526+0000 7f4d58e07700 1 --2- 172.21.15.16:0/1503343697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d5406acb0 0x7f4d5406b110 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:00.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.526+0000 7f4d58e07700 1 --2- 172.21.15.16:0/1503343697 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4d541316c0 0x7f4d54131a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:00.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.526+0000 7f4d58e07700 1 --2- 172.21.15.16:0/1503343697 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4d5409d9f0 0x7f4d5406a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:00.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.526+0000 7f4d58e07700 1 -- 172.21.15.16:0/1503343697 >> 172.21.15.16:0/1503343697 conn(0x7f4d54066f50 msgr2=0x7f4d5409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:00.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.526+0000 7f4d58e07700 1 -- 172.21.15.16:0/1503343697 shutdown_connections 2023-09-30T22:36:00.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.526+0000 7f4d58e07700 1 -- 172.21.15.16:0/1503343697 wait complete. 2023-09-30T22:36:00.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.528+0000 7f4d58e07700 1 Processor -- start 2023-09-30T22:36:00.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.528+0000 7f4d58e07700 1 -- start start 2023-09-30T22:36:00.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.529+0000 7f4d58e07700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4d5406acb0 0x7f4d5409a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:00.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.529+0000 7f4d58e07700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4d5409d9f0 0x7f4d5409a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:00.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.529+0000 7f4d58e07700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d541316c0 0x7f4d5409ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:00.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.529+0000 7f4d58e07700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5411d9b0 con 0x7f4d541316c0 2023-09-30T22:36:00.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.529+0000 7f4d58e07700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5411dfb0 con 0x7f4d5409d9f0 2023-09-30T22:36:00.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.529+0000 7f4d58e07700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5411e220 con 0x7f4d5406acb0 2023-09-30T22:36:00.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.529+0000 7f4d52d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d541316c0 0x7f4d5409ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:00.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.529+0000 7f4d49d9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4d5409d9f0 0x7f4d5409a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:00.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.529+0000 7f4d52d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d541316c0 0x7f4d5409ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39644/0 (socket says 172.21.15.16:39644) 2023-09-30T22:36:00.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.529+0000 7f4d52d9d700 1 -- 172.21.15.16:0/3440948541 learned_addr learned my addr 172.21.15.16:0/3440948541 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:00.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.529+0000 7f4d5259c700 1 --2- 172.21.15.16:0/3440948541 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4d5406acb0 0x7f4d5409a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:00.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.529+0000 7f4d52d9d700 1 -- 172.21.15.16:0/3440948541 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4d5406acb0 msgr2=0x7f4d5409a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:00.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.529+0000 7f4d52d9d700 1 --2- 172.21.15.16:0/3440948541 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4d5406acb0 0x7f4d5409a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:00.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.529+0000 7f4d52d9d700 1 -- 172.21.15.16:0/3440948541 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4d5409d9f0 msgr2=0x7f4d5409a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:00.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.529+0000 7f4d52d9d700 1 --2- 172.21.15.16:0/3440948541 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4d5409d9f0 0x7f4d5409a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:00.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.529+0000 7f4d52d9d700 1 -- 172.21.15.16:0/3440948541 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d3c0094a0 con 0x7f4d541316c0 2023-09-30T22:36:00.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.529+0000 7f4d49d9b700 1 --2- 172.21.15.16:0/3440948541 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4d5409d9f0 0x7f4d5409a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:00.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.529+0000 7f4d5259c700 1 --2- 172.21.15.16:0/3440948541 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4d5406acb0 0x7f4d5409a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:00.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.529+0000 7f4d52d9d700 1 --2- 172.21.15.16:0/3440948541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d541316c0 0x7f4d5409ae50 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f4d5409e270 tx=0x7f4d4c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:00.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.530+0000 7f4d4b7fe700 1 -- 172.21.15.16:0/3440948541 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c0092e0 con 0x7f4d541316c0 2023-09-30T22:36:00.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.530+0000 7f4d4b7fe700 1 -- 172.21.15.16:0/3440948541 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d4c00ce80 con 0x7f4d541316c0 2023-09-30T22:36:00.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.530+0000 7f4d58e07700 1 -- 172.21.15.16:0/3440948541 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d4c009b30 con 0x7f4d541316c0 2023-09-30T22:36:00.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.530+0000 7f4d4b7fe700 1 -- 172.21.15.16:0/3440948541 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c0044b0 con 0x7f4d541316c0 2023-09-30T22:36:00.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.530+0000 7f4d58e07700 1 -- 172.21.15.16:0/3440948541 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d5412b900 con 0x7f4d541316c0 2023-09-30T22:36:00.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.531+0000 7f4d4b7fe700 1 -- 172.21.15.16:0/3440948541 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4d4c004610 con 0x7f4d541316c0 2023-09-30T22:36:00.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.531+0000 7f4d4b7fe700 1 -- 172.21.15.16:0/3440948541 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f4d4c040570 con 0x7f4d541316c0 2023-09-30T22:36:00.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.531+0000 7f4d58e07700 1 -- 172.21.15.16:0/3440948541 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d54074ea0 con 0x7f4d541316c0 2023-09-30T22:36:00.540 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.540+0000 7f4d4b7fe700 1 -- 172.21.15.16:0/3440948541 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4d4c011c10 con 0x7f4d541316c0 2023-09-30T22:36:00.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.563+0000 7f39b7bfe700 1 Processor -- start 2023-09-30T22:36:00.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.563+0000 7f39b7bfe700 1 -- start start 2023-09-30T22:36:00.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.564+0000 7f39b7bfe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f39b00a0a30 0x7f39b00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:00.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.564+0000 7f39b7bfe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f39b009d630 0x7f39b00a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:00.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.564+0000 7f39b7bfe700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39b00a2a70 con 0x7f39b0133930 2023-09-30T22:36:00.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.564+0000 7f39b7bfe700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39b00a01d0 con 0x7f39b00a0a30 2023-09-30T22:36:00.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.564+0000 7f39b7bfe700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f39b009e2b0 con 0x7f39b009d630 2023-09-30T22:36:00.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.564+0000 7f39b5199700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f39b009d630 0x7f39b00a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:00.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.564+0000 7f39b619b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f39b0133930 0x7f39b0133d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38268/0 (socket says 172.21.15.16:38268) 2023-09-30T22:36:00.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.564+0000 7f39b619b700 1 -- 172.21.15.16:0/2004718859 learned_addr learned my addr 172.21.15.16:0/2004718859 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:36:00.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.564+0000 7f39b599a700 1 --2- 172.21.15.16:0/2004718859 >> v2:172.21.15.16:3300/0 conn(0x7f39b00a0a30 0x7f39b00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:00.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.564+0000 7f39b5199700 1 -- 172.21.15.16:0/2004718859 >> v2:172.21.15.16:3300/0 conn(0x7f39b00a0a30 msgr2=0x7f39b00a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:00.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.564+0000 7f39b5199700 1 --2- 172.21.15.16:0/2004718859 >> v2:172.21.15.16:3300/0 conn(0x7f39b00a0a30 0x7f39b00a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:00.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.564+0000 7f39b5199700 1 -- 172.21.15.16:0/2004718859 >> v1:172.21.15.16:6789/0 conn(0x7f39b0133930 legacy=0x7f39b0133d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:00.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.564+0000 7f39b5199700 1 -- 172.21.15.16:0/2004718859 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39b009dc50 con 0x7f39b009d630 2023-09-30T22:36:00.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.564+0000 7f39b4998700 1 -- 172.21.15.16:0/2004718859 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3822783094 0 0) 0x7f39b00a2a70 con 0x7f39b0133930 2023-09-30T22:36:00.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.564+0000 7f39b599a700 1 --2- 172.21.15.16:0/2004718859 >> v2:172.21.15.16:3300/0 conn(0x7f39b00a0a30 0x7f39b00a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:00.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.565+0000 7f39b5199700 1 --2- 172.21.15.16:0/2004718859 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f39b009d630 0x7f39b00a1340 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f39a0009840 tx=0x7f39a0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f059a06ed704f5c3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:00.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.565+0000 7f39b4998700 1 -- 172.21.15.16:0/2004718859 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a0011830 con 0x7f39b009d630 2023-09-30T22:36:00.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.565+0000 7f39b4998700 1 -- 172.21.15.16:0/2004718859 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39a0011990 con 0x7f39b009d630 2023-09-30T22:36:00.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.565+0000 7f39b4998700 1 -- 172.21.15.16:0/2004718859 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a0011d20 con 0x7f39b009d630 2023-09-30T22:36:00.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.565+0000 7f39b7bfe700 1 -- 172.21.15.16:0/2004718859 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f39b009d630 msgr2=0x7f39b00a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:00.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.565+0000 7f39b7bfe700 1 --2- 172.21.15.16:0/2004718859 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f39b009d630 0x7f39b00a1340 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f39a0009840 tx=0x7f39a0009b50 comp rx=0 tx=0).stop 2023-09-30T22:36:00.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.565+0000 7f39b7bfe700 1 -- 172.21.15.16:0/2004718859 shutdown_connections 2023-09-30T22:36:00.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.565+0000 7f39b7bfe700 1 --2- 172.21.15.16:0/2004718859 >> v2:172.21.15.16:3300/0 conn(0x7f39b00a0a30 0x7f39b00a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:00.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.565+0000 7f39b7bfe700 1 --2- 172.21.15.16:0/2004718859 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f39b009d630 0x7f39b00a1340 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:00.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.565+0000 7f39b7bfe700 1 -- 172.21.15.16:0/2004718859 >> 172.21.15.16:0/2004718859 conn(0x7f39b00a3050 msgr2=0x7f39b00a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:00.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.566+0000 7f39b7bfe700 1 -- 172.21.15.16:0/2004718859 shutdown_connections 2023-09-30T22:36:00.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.566+0000 7f39b7bfe700 1 -- 172.21.15.16:0/2004718859 wait complete. 2023-09-30T22:36:00.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.567+0000 7f39b7bfe700 1 Processor -- start 2023-09-30T22:36:00.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.567+0000 7f39b7bfe700 1 -- start start 2023-09-30T22:36:00.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.567+0000 7f4ca47c1700 1 Processor -- start 2023-09-30T22:36:00.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.567+0000 7f4ca47c1700 1 -- start start 2023-09-30T22:36:00.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.567+0000 7f39b7bfe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f39b009d630 0x7f39b01c8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:00.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.567+0000 7f39b7bfe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f39b00a0a30 0x7f39b01c8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:00.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.567+0000 7f39b7bfe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39b0133930 0x7f39b011e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:00.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.567+0000 7f39b7bfe700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b0122190 con 0x7f39b0133930 2023-09-30T22:36:00.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.567+0000 7f39b7bfe700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39b00a5360 con 0x7f39b009d630 2023-09-30T22:36:00.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.567+0000 7f39b7bfe700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f39b0133410 con 0x7f39b00a0a30 2023-09-30T22:36:00.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.567+0000 7f39b5199700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f39b00a0a30 0x7f39b01c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:00.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.567+0000 7f39b5199700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f39b00a0a30 0x7f39b01c8990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:41044/0 (socket says 172.21.15.16:41044) 2023-09-30T22:36:00.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.567+0000 7f39b5199700 1 -- 172.21.15.16:0/211180155 learned_addr learned my addr 172.21.15.16:0/211180155 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:00.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.567+0000 7f39b599a700 1 --2- 172.21.15.16:0/211180155 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f39b009d630 0x7f39b01c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:00.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.567+0000 7f39b619b700 1 --2- 172.21.15.16:0/211180155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39b0133930 0x7f39b011e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:00.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.568+0000 7f4ca47c1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4c9c09e1d0 0x7f4c9c138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:00.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.568+0000 7f4ca47c1700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4c9c139170 0x7f4c9c13b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:00.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.568+0000 7f4ca47c1700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c9c09b120 con 0x7f4c9c09d6f0 2023-09-30T22:36:00.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.568+0000 7f4ca47c1700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c9c06b460 con 0x7f4c9c139170 2023-09-30T22:36:00.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.568+0000 7f4ca47c1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9c09a7e0 con 0x7f4c9c09e1d0 2023-09-30T22:36:00.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.568+0000 7f39b599a700 1 -- 172.21.15.16:0/211180155 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f39b00a0a30 msgr2=0x7f39b01c8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:00.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.568+0000 7f39b599a700 1 --2- 172.21.15.16:0/211180155 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f39b00a0a30 0x7f39b01c8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:00.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.568+0000 7f39b599a700 1 -- 172.21.15.16:0/211180155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39b0133930 msgr2=0x7f39b011e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:00.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.568+0000 7f39b599a700 1 --2- 172.21.15.16:0/211180155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39b0133930 0x7f39b011e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:00.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.568+0000 7f39b599a700 1 -- 172.21.15.16:0/211180155 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39ac0094a0 con 0x7f39b009d630 2023-09-30T22:36:00.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.568+0000 7f4ca1d5c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4c9c09e1d0 0x7f4c9c138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:00.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.568+0000 7f39b5199700 1 --2- 172.21.15.16:0/211180155 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f39b00a0a30 0x7f39b01c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:00.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.568+0000 7f4ca1d5c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4c9c09e1d0 0x7f4c9c138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:41060/0 (socket says 172.21.15.16:41060) 2023-09-30T22:36:00.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.568+0000 7f4ca1d5c700 1 -- 172.21.15.16:0/91171399 learned_addr learned my addr 172.21.15.16:0/91171399 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:00.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.568+0000 7f39b599a700 1 --2- 172.21.15.16:0/211180155 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f39b009d630 0x7f39b01c8450 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f39ac00e9e0 tx=0x7f39ac00ecf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:00.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.568+0000 7f39a67fc700 1 -- 172.21.15.16:0/211180155 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ac00ee80 con 0x7f39b009d630 2023-09-30T22:36:00.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.568+0000 7f4ca1d5c700 1 -- 172.21.15.16:0/91171399 >> v2:172.21.15.16:3300/0 conn(0x7f4c9c139170 msgr2=0x7f4c9c13b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:00.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.568+0000 7f4ca1d5c700 1 --2- 172.21.15.16:0/91171399 >> v2:172.21.15.16:3300/0 conn(0x7f4c9c139170 0x7f4c9c13b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:00.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.568+0000 7f4ca1d5c700 1 -- 172.21.15.16:0/91171399 >> v1:172.21.15.16:6789/0 conn(0x7f4c9c09d6f0 legacy=0x7f4c9c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:00.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.568+0000 7f4ca1d5c700 1 -- 172.21.15.16:0/91171399 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c9c09a500 con 0x7f4c9c09e1d0 2023-09-30T22:36:00.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.568+0000 7f4ca1d5c700 1 --2- 172.21.15.16:0/91171399 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4c9c09e1d0 0x7f4c9c138c30 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c009920 tx=0x7f4c8c009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7daadb3d4195a16d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:00.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.569+0000 7f4ca155b700 1 -- 172.21.15.16:0/91171399 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c00ba20 con 0x7f4c9c09e1d0 2023-09-30T22:36:00.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.569+0000 7f4ca155b700 1 -- 172.21.15.16:0/91171399 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c8c00bb80 con 0x7f4c9c09e1d0 2023-09-30T22:36:00.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.569+0000 7f4ca155b700 1 -- 172.21.15.16:0/91171399 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c009e70 con 0x7f4c9c09e1d0 2023-09-30T22:36:00.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.568+0000 7f39b7bfe700 1 -- 172.21.15.16:0/211180155 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39a00094a0 con 0x7f39b009d630 2023-09-30T22:36:00.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.568+0000 7f39a67fc700 1 -- 172.21.15.16:0/211180155 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39ac00c850 con 0x7f39b009d630 2023-09-30T22:36:00.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.568+0000 7f39a67fc700 1 -- 172.21.15.16:0/211180155 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39ac00cbe0 con 0x7f39b009d630 2023-09-30T22:36:00.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.568+0000 7f39b7bfe700 1 -- 172.21.15.16:0/211180155 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39b01c8ed0 con 0x7f39b009d630 2023-09-30T22:36:00.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.569+0000 7f39a67fc700 1 -- 172.21.15.16:0/211180155 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f39ac00cd40 con 0x7f39b009d630 2023-09-30T22:36:00.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.570+0000 7f39a67fc700 1 -- 172.21.15.16:0/211180155 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f39ac03f720 con 0x7f39b009d630 2023-09-30T22:36:00.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.570+0000 7f39b7bfe700 1 -- 172.21.15.16:0/211180155 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39b00747f0 con 0x7f39b009d630 2023-09-30T22:36:00.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.570+0000 7f4ca47c1700 1 -- 172.21.15.16:0/91171399 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4c9c09e1d0 msgr2=0x7f4c9c138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:00.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.570+0000 7f4ca47c1700 1 --2- 172.21.15.16:0/91171399 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4c9c09e1d0 0x7f4c9c138c30 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c009920 tx=0x7f4c8c009c30 comp rx=0 tx=0).stop 2023-09-30T22:36:00.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.570+0000 7f4ca47c1700 1 -- 172.21.15.16:0/91171399 shutdown_connections 2023-09-30T22:36:00.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.570+0000 7f4ca47c1700 1 --2- 172.21.15.16:0/91171399 >> v2:172.21.15.16:3300/0 conn(0x7f4c9c139170 0x7f4c9c13b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:00.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.570+0000 7f4ca47c1700 1 --2- 172.21.15.16:0/91171399 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4c9c09e1d0 0x7f4c9c138c30 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:00.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.570+0000 7f4ca47c1700 1 -- 172.21.15.16:0/91171399 >> 172.21.15.16:0/91171399 conn(0x7f4c9c066f50 msgr2=0x7f4c9c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:00.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.570+0000 7f4ca47c1700 1 -- 172.21.15.16:0/91171399 shutdown_connections 2023-09-30T22:36:00.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.570+0000 7f4ca47c1700 1 -- 172.21.15.16:0/91171399 wait complete. 2023-09-30T22:36:00.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.571+0000 7f4ca47c1700 1 Processor -- start 2023-09-30T22:36:00.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.571+0000 7f4ca47c1700 1 -- start start 2023-09-30T22:36:00.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.572+0000 7f4ca47c1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4c9c09d6f0 0x7f4c9c11e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:00.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.572+0000 7f4ca47c1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c9c09e1d0 0x7f4c9c11edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:00.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.572+0000 7f4ca47c1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4c9c139170 0x7f4c9c11f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:00.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.572+0000 7f4ca47c1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9c1343e0 con 0x7f4c9c09e1d0 2023-09-30T22:36:00.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.572+0000 7f4ca47c1700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9c1205d0 con 0x7f4c9c139170 2023-09-30T22:36:00.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.572+0000 7f4ca47c1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9c121e30 con 0x7f4c9c09d6f0 2023-09-30T22:36:00.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.572+0000 7f4ca1d5c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c9c09e1d0 0x7f4c9c11edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:00.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.572+0000 7f4ca255d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4c9c09d6f0 0x7f4c9c11e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:00.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.572+0000 7f4ca1d5c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c9c09e1d0 0x7f4c9c11edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39674/0 (socket says 172.21.15.16:39674) 2023-09-30T22:36:00.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.572+0000 7f4ca2d5e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4c9c139170 0x7f4c9c11f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:00.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.572+0000 7f4ca1d5c700 1 -- 172.21.15.16:0/1584611914 learned_addr learned my addr 172.21.15.16:0/1584611914 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:00.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.573+0000 7f4ca255d700 1 -- 172.21.15.16:0/1584611914 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4c9c139170 msgr2=0x7f4c9c11f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:00.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.573+0000 7f4ca255d700 1 --2- 172.21.15.16:0/1584611914 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4c9c139170 0x7f4c9c11f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:00.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.573+0000 7f4ca255d700 1 -- 172.21.15.16:0/1584611914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c9c09e1d0 msgr2=0x7f4c9c11edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:00.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.573+0000 7f4ca255d700 1 --2- 172.21.15.16:0/1584611914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c9c09e1d0 0x7f4c9c11edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:00.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.573+0000 7f4ca255d700 1 -- 172.21.15.16:0/1584611914 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c8c009550 con 0x7f4c9c09d6f0 2023-09-30T22:36:00.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.573+0000 7f4ca2d5e700 1 --2- 172.21.15.16:0/1584611914 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4c9c139170 0x7f4c9c11f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:00.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.573+0000 7f4ca255d700 1 --2- 172.21.15.16:0/1584611914 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4c9c09d6f0 0x7f4c9c11e8b0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f4c98000f60 tx=0x7f4c9800c730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:00.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.573+0000 7f4c92ffd700 1 -- 172.21.15.16:0/1584611914 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c980052d0 con 0x7f4c9c09d6f0 2023-09-30T22:36:00.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.573+0000 7f4c92ffd700 1 -- 172.21.15.16:0/1584611914 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c98005430 con 0x7f4c9c09d6f0 2023-09-30T22:36:00.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.573+0000 7f4ca47c1700 1 -- 172.21.15.16:0/1584611914 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c9c120a70 con 0x7f4c9c09d6f0 2023-09-30T22:36:00.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.573+0000 7f4c92ffd700 1 -- 172.21.15.16:0/1584611914 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c980038c0 con 0x7f4c9c09d6f0 2023-09-30T22:36:00.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.573+0000 7f4ca47c1700 1 -- 172.21.15.16:0/1584611914 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c9c120cb0 con 0x7f4c9c09d6f0 2023-09-30T22:36:00.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.574+0000 7f4c92ffd700 1 -- 172.21.15.16:0/1584611914 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4c98003a20 con 0x7f4c9c09d6f0 2023-09-30T22:36:00.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.574+0000 7f4c92ffd700 1 -- 172.21.15.16:0/1584611914 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f4c9803e9b0 con 0x7f4c9c09d6f0 2023-09-30T22:36:00.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.575+0000 7f4ca47c1700 1 -- 172.21.15.16:0/1584611914 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c9c1347a0 con 0x7f4c9c09d6f0 2023-09-30T22:36:00.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.576+0000 7f39a67fc700 1 -- 172.21.15.16:0/211180155 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f39ac0232f0 con 0x7f39b009d630 2023-09-30T22:36:00.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.580+0000 7f4c92ffd700 1 -- 172.21.15.16:0/1584611914 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4c98017080 con 0x7f4c9c09d6f0 2023-09-30T22:36:00.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.759+0000 7f4d58e07700 1 -- 172.21.15.16:0/3440948541 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f4d540725c0 con 0x7f4d541316c0 2023-09-30T22:36:00.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.759+0000 7f4d4b7fe700 1 -- 172.21.15.16:0/3440948541 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4d4c011800 con 0x7f4d541316c0 2023-09-30T22:36:00.760 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:36:00.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.762+0000 7f4d58e07700 1 -- 172.21.15.16:0/3440948541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d541316c0 msgr2=0x7f4d5409ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:00.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.762+0000 7f4d58e07700 1 --2- 172.21.15.16:0/3440948541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d541316c0 0x7f4d5409ae50 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f4d5409e270 tx=0x7f4d4c004ad0 comp rx=0 tx=0).stop 2023-09-30T22:36:00.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.762+0000 7f4d58e07700 1 -- 172.21.15.16:0/3440948541 shutdown_connections 2023-09-30T22:36:00.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.762+0000 7f4d58e07700 1 --2- 172.21.15.16:0/3440948541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4d541316c0 0x7f4d5409ae50 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:00.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.762+0000 7f4d58e07700 1 --2- 172.21.15.16:0/3440948541 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4d5406acb0 0x7f4d5409a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:00.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.762+0000 7f4d58e07700 1 --2- 172.21.15.16:0/3440948541 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4d5409d9f0 0x7f4d5409a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:00.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.762+0000 7f4d58e07700 1 -- 172.21.15.16:0/3440948541 >> 172.21.15.16:0/3440948541 conn(0x7f4d54066f50 msgr2=0x7f4d5412bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:00.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.762+0000 7f4d58e07700 1 -- 172.21.15.16:0/3440948541 shutdown_connections 2023-09-30T22:36:00.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.762+0000 7f4d58e07700 1 -- 172.21.15.16:0/3440948541 wait complete. 2023-09-30T22:36:00.776 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:36:00.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.786+0000 7f39b7bfe700 1 -- 172.21.15.16:0/211180155 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f39b0072980 con 0x7f39b009d630 2023-09-30T22:36:00.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.786+0000 7f39a67fc700 1 -- 172.21.15.16:0/211180155 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f39ac021070 con 0x7f39b009d630 2023-09-30T22:36:00.787 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:36:00.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.789+0000 7f39b7bfe700 1 -- 172.21.15.16:0/211180155 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f39b009d630 msgr2=0x7f39b01c8450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:00.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.789+0000 7f39b7bfe700 1 --2- 172.21.15.16:0/211180155 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f39b009d630 0x7f39b01c8450 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f39ac00e9e0 tx=0x7f39ac00ecf0 comp rx=0 tx=0).stop 2023-09-30T22:36:00.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.789+0000 7f39b7bfe700 1 -- 172.21.15.16:0/211180155 shutdown_connections 2023-09-30T22:36:00.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.789+0000 7f39b7bfe700 1 --2- 172.21.15.16:0/211180155 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f39b009d630 0x7f39b01c8450 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:00.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.789+0000 7f39b7bfe700 1 --2- 172.21.15.16:0/211180155 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39b0133930 0x7f39b011e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:00.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.789+0000 7f39b7bfe700 1 --2- 172.21.15.16:0/211180155 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f39b00a0a30 0x7f39b01c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:00.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.789+0000 7f39b7bfe700 1 -- 172.21.15.16:0/211180155 >> 172.21.15.16:0/211180155 conn(0x7f39b00a3050 msgr2=0x7f39b01bbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:00.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.789+0000 7f39b7bfe700 1 -- 172.21.15.16:0/211180155 shutdown_connections 2023-09-30T22:36:00.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.789+0000 7f39b7bfe700 1 -- 172.21.15.16:0/211180155 wait complete. 2023-09-30T22:36:00.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.793+0000 7f4ca47c1700 1 -- 172.21.15.16:0/1584611914 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4c9c09f420 con 0x7f4c9c09d6f0 2023-09-30T22:36:00.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.794+0000 7f4c92ffd700 1 -- 172.21.15.16:0/1584611914 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4c98011590 con 0x7f4c9c09d6f0 2023-09-30T22:36:00.794 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:36:00.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.796+0000 7f4ca47c1700 1 -- 172.21.15.16:0/1584611914 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4c9c09d6f0 msgr2=0x7f4c9c11e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:00.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.796+0000 7f4ca47c1700 1 --2- 172.21.15.16:0/1584611914 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4c9c09d6f0 0x7f4c9c11e8b0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f4c98000f60 tx=0x7f4c9800c730 comp rx=0 tx=0).stop 2023-09-30T22:36:00.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.796+0000 7f4ca47c1700 1 -- 172.21.15.16:0/1584611914 shutdown_connections 2023-09-30T22:36:00.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.796+0000 7f4ca47c1700 1 --2- 172.21.15.16:0/1584611914 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4c9c139170 0x7f4c9c11f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:00.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.796+0000 7f4ca47c1700 1 --2- 172.21.15.16:0/1584611914 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4c9c09d6f0 0x7f4c9c11e8b0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:00.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.796+0000 7f4ca47c1700 1 --2- 172.21.15.16:0/1584611914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c9c09e1d0 0x7f4c9c11edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:00.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.796+0000 7f4ca47c1700 1 -- 172.21.15.16:0/1584611914 >> 172.21.15.16:0/1584611914 conn(0x7f4c9c066f50 msgr2=0x7f4c9c138170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:00.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.796+0000 7f4ca47c1700 1 -- 172.21.15.16:0/1584611914 shutdown_connections 2023-09-30T22:36:00.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.796+0000 7f4ca47c1700 1 -- 172.21.15.16:0/1584611914 wait complete. 2023-09-30T22:36:00.804 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:36:00.811 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:36:00.816 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:36:00.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.990+0000 7faf6c27f700 1 Processor -- start 2023-09-30T22:36:00.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.990+0000 7faf6c27f700 1 -- start start 2023-09-30T22:36:00.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.991+0000 7faf6c27f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faf64133420 0x7faf641337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:00.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.991+0000 7faf6c27f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf64133e30 0x7faf6412ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:00.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.991+0000 7faf6c27f700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf6406b990 con 0x7faf64134260 2023-09-30T22:36:00.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.991+0000 7faf6c27f700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf640a0590 con 0x7faf64133420 2023-09-30T22:36:00.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.991+0000 7faf6c27f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf6409fb80 con 0x7faf64133e30 2023-09-30T22:36:00.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.991+0000 7faf6981a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf64133e30 0x7faf6412ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:00.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.991+0000 7faf6a01b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faf64133420 0x7faf641337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:00.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.991+0000 7faf6a01b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faf64133420 0x7faf641337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39684/0 (socket says 172.21.15.16:39684) 2023-09-30T22:36:00.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.991+0000 7faf6981a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf64133e30 0x7faf6412ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47482/0 (socket says 172.21.15.16:47482) 2023-09-30T22:36:00.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.991+0000 7faf6a01b700 1 -- 172.21.15.16:0/2810995974 learned_addr learned my addr 172.21.15.16:0/2810995974 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:00.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.991+0000 7faf6a01b700 1 -- 172.21.15.16:0/2810995974 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf64133e30 msgr2=0x7faf6412ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:00.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.991+0000 7faf6a01b700 1 --2- 172.21.15.16:0/2810995974 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf64133e30 0x7faf6412ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:00.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.991+0000 7faf6a01b700 1 -- 172.21.15.16:0/2810995974 >> v1:172.21.15.16:6789/0 conn(0x7faf64134260 legacy=0x7faf641304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:00.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.991+0000 7faf6a01b700 1 -- 172.21.15.16:0/2810995974 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf640a0030 con 0x7faf64133420 2023-09-30T22:36:00.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.992+0000 7faf6981a700 1 --2- 172.21.15.16:0/2810995974 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf64133e30 0x7faf6412ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:00.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.991+0000 7faf69019700 1 -- 172.21.15.16:0/2810995974 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2584104231 0 0) 0x7faf6406b990 con 0x7faf64134260 2023-09-30T22:36:00.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.992+0000 7faf6a01b700 1 --2- 172.21.15.16:0/2810995974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf64133420 0x7faf641337f0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7faf600098f0 tx=0x7faf60009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c2e437f31ebdb0b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:00.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.992+0000 7faf69019700 1 -- 172.21.15.16:0/2810995974 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf60011830 con 0x7faf64133420 2023-09-30T22:36:00.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.992+0000 7faf69019700 1 -- 172.21.15.16:0/2810995974 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf60011990 con 0x7faf64133420 2023-09-30T22:36:00.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.992+0000 7faf69019700 1 -- 172.21.15.16:0/2810995974 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf60011d20 con 0x7faf64133420 2023-09-30T22:36:00.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.992+0000 7faf6c27f700 1 -- 172.21.15.16:0/2810995974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf64133420 msgr2=0x7faf641337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:00.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.992+0000 7faf6c27f700 1 --2- 172.21.15.16:0/2810995974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf64133420 0x7faf641337f0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7faf600098f0 tx=0x7faf60009c00 comp rx=0 tx=0).stop 2023-09-30T22:36:00.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.993+0000 7faf6c27f700 1 -- 172.21.15.16:0/2810995974 shutdown_connections 2023-09-30T22:36:00.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.993+0000 7faf6c27f700 1 --2- 172.21.15.16:0/2810995974 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf64133420 0x7faf641337f0 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:00.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.993+0000 7faf6c27f700 1 --2- 172.21.15.16:0/2810995974 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf64133e30 0x7faf6412ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:00.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.993+0000 7faf6c27f700 1 -- 172.21.15.16:0/2810995974 >> 172.21.15.16:0/2810995974 conn(0x7faf640a4210 msgr2=0x7faf640a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:00.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.993+0000 7faf6c27f700 1 -- 172.21.15.16:0/2810995974 shutdown_connections 2023-09-30T22:36:00.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.993+0000 7faf6c27f700 1 -- 172.21.15.16:0/2810995974 wait complete. 2023-09-30T22:36:00.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.994+0000 7faf6c27f700 1 Processor -- start 2023-09-30T22:36:00.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.994+0000 7faf6c27f700 1 -- start start 2023-09-30T22:36:00.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.994+0000 7faf6c27f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf64133420 0x7faf6409c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:00.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.994+0000 7faf6c27f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf64133e30 0x7faf6409c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:00.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.994+0000 7faf6c27f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf64134260 0x7faf6409cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:00.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.994+0000 7faf6c27f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf64120110 con 0x7faf64133e30 2023-09-30T22:36:00.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.994+0000 7faf6c27f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf6411f080 con 0x7faf64133420 2023-09-30T22:36:00.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.994+0000 7faf6c27f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faf6409deb0 con 0x7faf64134260 2023-09-30T22:36:00.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.994+0000 7faf6a01b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf64133420 0x7faf6409c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:00.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.994+0000 7faf6a01b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf64133420 0x7faf6409c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47488/0 (socket says 172.21.15.16:47488) 2023-09-30T22:36:00.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.994+0000 7faf6a01b700 1 -- 172.21.15.16:0/620904991 learned_addr learned my addr 172.21.15.16:0/620904991 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:00.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.994+0000 7faf6981a700 1 --2- 172.21.15.16:0/620904991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf64133e30 0x7faf6409c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:00.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.994+0000 7faf6a81c700 1 --2- 172.21.15.16:0/620904991 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf64134260 0x7faf6409cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:00.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.994+0000 7faf6a01b700 1 -- 172.21.15.16:0/620904991 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf64134260 msgr2=0x7faf6409cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:00.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.994+0000 7faf6a01b700 1 --2- 172.21.15.16:0/620904991 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf64134260 0x7faf6409cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:00.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.994+0000 7faf6a01b700 1 -- 172.21.15.16:0/620904991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf64133e30 msgr2=0x7faf6409c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:00.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.994+0000 7faf6a01b700 1 --2- 172.21.15.16:0/620904991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf64133e30 0x7faf6409c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:00.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.995+0000 7faf6a01b700 1 -- 172.21.15.16:0/620904991 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf580094a0 con 0x7faf64133420 2023-09-30T22:36:00.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.995+0000 7faf6a01b700 1 --2- 172.21.15.16:0/620904991 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf64133420 0x7faf6409c480 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7faf60009c40 tx=0x7faf60004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:00.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.995+0000 7faf56ffd700 1 -- 172.21.15.16:0/620904991 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf60004300 con 0x7faf64133420 2023-09-30T22:36:00.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.995+0000 7faf56ffd700 1 -- 172.21.15.16:0/620904991 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf6001f070 con 0x7faf64133420 2023-09-30T22:36:00.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.995+0000 7faf6c27f700 1 -- 172.21.15.16:0/620904991 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf60009550 con 0x7faf64133420 2023-09-30T22:36:00.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.995+0000 7faf56ffd700 1 -- 172.21.15.16:0/620904991 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf600036a0 con 0x7faf64133420 2023-09-30T22:36:01.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.995+0000 7faf6c27f700 1 -- 172.21.15.16:0/620904991 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf6411fd90 con 0x7faf64133420 2023-09-30T22:36:01.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.996+0000 7faf56ffd700 1 -- 172.21.15.16:0/620904991 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7faf60021030 con 0x7faf64133420 2023-09-30T22:36:01.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.996+0000 7faf6c27f700 1 -- 172.21.15.16:0/620904991 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf64074ea0 con 0x7faf64133420 2023-09-30T22:36:01.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:00.996+0000 7faf56ffd700 1 -- 172.21.15.16:0/620904991 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7faf60040980 con 0x7faf64133420 2023-09-30T22:36:01.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.001+0000 7faf56ffd700 1 -- 172.21.15.16:0/620904991 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7faf60019770 con 0x7faf64133420 2023-09-30T22:36:01.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.204+0000 7faf6c27f700 1 -- 172.21.15.16:0/620904991 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7faf6406b250 con 0x7faf64133420 2023-09-30T22:36:01.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.205+0000 7faf56ffd700 1 -- 172.21.15.16:0/620904991 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7faf60015070 con 0x7faf64133420 2023-09-30T22:36:01.205 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:36:01.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.207+0000 7faf6c27f700 1 -- 172.21.15.16:0/620904991 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf64133420 msgr2=0x7faf6409c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:01.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.207+0000 7faf6c27f700 1 --2- 172.21.15.16:0/620904991 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf64133420 0x7faf6409c480 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7faf60009c40 tx=0x7faf60004750 comp rx=0 tx=0).stop 2023-09-30T22:36:01.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.207+0000 7faf6c27f700 1 -- 172.21.15.16:0/620904991 shutdown_connections 2023-09-30T22:36:01.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.207+0000 7faf6c27f700 1 --2- 172.21.15.16:0/620904991 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faf64133420 0x7faf6409c480 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:01.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.207+0000 7faf6c27f700 1 --2- 172.21.15.16:0/620904991 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faf64134260 0x7faf6409cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:01.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.207+0000 7faf6c27f700 1 --2- 172.21.15.16:0/620904991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faf64133e30 0x7faf6409c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:01.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.207+0000 7faf6c27f700 1 -- 172.21.15.16:0/620904991 >> 172.21.15.16:0/620904991 conn(0x7faf640a4210 msgr2=0x7faf6412de00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:01.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.207+0000 7faf6c27f700 1 -- 172.21.15.16:0/620904991 shutdown_connections 2023-09-30T22:36:01.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.207+0000 7faf6c27f700 1 -- 172.21.15.16:0/620904991 wait complete. 2023-09-30T22:36:01.221 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:36:01.777 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:36:01.806 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:36:01.811 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:36:01.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.929+0000 7fe02ecf8700 1 Processor -- start 2023-09-30T22:36:01.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.929+0000 7fe02ecf8700 1 -- start start 2023-09-30T22:36:01.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.930+0000 7fe02ecf8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe02806a770 0x7fe02806ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:01.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.930+0000 7fe02ecf8700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe028131d10 0x7fe028139860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:01.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.930+0000 7fe02ecf8700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe02809ece0 con 0x7fe02806b080 2023-09-30T22:36:01.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.930+0000 7fe02ecf8700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe02809e760 con 0x7fe028131d10 2023-09-30T22:36:01.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.930+0000 7fe02ecf8700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe02809e040 con 0x7fe02806a770 2023-09-30T22:36:01.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.930+0000 7fe02d295700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe028131d10 0x7fe028139860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:01.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.930+0000 7fe02ca94700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe02806a770 0x7fe02806ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:01.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.930+0000 7fe02d295700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe028131d10 0x7fe028139860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39706/0 (socket says 172.21.15.16:39706) 2023-09-30T22:36:01.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.930+0000 7fe02d295700 1 -- 172.21.15.16:0/45289908 learned_addr learned my addr 172.21.15.16:0/45289908 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:01.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.930+0000 7fe027fff700 1 --1- 172.21.15.16:0/45289908 >> v1:172.21.15.16:6789/0 conn(0x7fe02806b080 0x7fe028131600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38298/0 (socket says 172.21.15.16:38298) 2023-09-30T22:36:01.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.931+0000 7fe02ca94700 1 -- 172.21.15.16:0/45289908 >> v2:172.21.15.16:3300/0 conn(0x7fe028131d10 msgr2=0x7fe028139860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:01.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.931+0000 7fe02ca94700 1 --2- 172.21.15.16:0/45289908 >> v2:172.21.15.16:3300/0 conn(0x7fe028131d10 0x7fe028139860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:01.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.931+0000 7fe02ca94700 1 -- 172.21.15.16:0/45289908 >> v1:172.21.15.16:6789/0 conn(0x7fe02806b080 legacy=0x7fe028131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:01.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.931+0000 7fe02ca94700 1 -- 172.21.15.16:0/45289908 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe02809f630 con 0x7fe02806a770 2023-09-30T22:36:01.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.931+0000 7fe0277fe700 1 -- 172.21.15.16:0/45289908 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1182838601 0 0) 0x7fe02809ece0 con 0x7fe02806b080 2023-09-30T22:36:01.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.931+0000 7fe02d295700 1 --2- 172.21.15.16:0/45289908 >> v2:172.21.15.16:3300/0 conn(0x7fe028131d10 0x7fe028139860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:01.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.931+0000 7fe02ca94700 1 --2- 172.21.15.16:0/45289908 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe02806a770 0x7fe02806ab40 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fe01c009840 tx=0x7fe01c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ad3e917918ac13d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:01.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.931+0000 7fe0277fe700 1 -- 172.21.15.16:0/45289908 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01c011830 con 0x7fe02806a770 2023-09-30T22:36:01.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.931+0000 7fe0277fe700 1 -- 172.21.15.16:0/45289908 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe01c011990 con 0x7fe02806a770 2023-09-30T22:36:01.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.931+0000 7fe0277fe700 1 -- 172.21.15.16:0/45289908 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01c011d20 con 0x7fe02806a770 2023-09-30T22:36:01.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.932+0000 7fe02ecf8700 1 -- 172.21.15.16:0/45289908 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe02806a770 msgr2=0x7fe02806ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:01.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.932+0000 7fe02ecf8700 1 --2- 172.21.15.16:0/45289908 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe02806a770 0x7fe02806ab40 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fe01c009840 tx=0x7fe01c009b50 comp rx=0 tx=0).stop 2023-09-30T22:36:01.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.932+0000 7fe02ecf8700 1 -- 172.21.15.16:0/45289908 shutdown_connections 2023-09-30T22:36:01.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.932+0000 7fe02ecf8700 1 --2- 172.21.15.16:0/45289908 >> v2:172.21.15.16:3300/0 conn(0x7fe028131d10 0x7fe028139860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:01.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.932+0000 7fe02ecf8700 1 --2- 172.21.15.16:0/45289908 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe02806a770 0x7fe02806ab40 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:01.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.932+0000 7fe02ecf8700 1 -- 172.21.15.16:0/45289908 >> 172.21.15.16:0/45289908 conn(0x7fe028066f50 msgr2=0x7fe02809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:01.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.932+0000 7fe02ecf8700 1 -- 172.21.15.16:0/45289908 shutdown_connections 2023-09-30T22:36:01.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.932+0000 7fe02ecf8700 1 -- 172.21.15.16:0/45289908 wait complete. 2023-09-30T22:36:01.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.933+0000 7fe02ecf8700 1 Processor -- start 2023-09-30T22:36:01.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.933+0000 7fe02ecf8700 1 -- start start 2023-09-30T22:36:01.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.934+0000 7fe02ecf8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe02806a770 0x7fe02811e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:01.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.934+0000 7fe02ecf8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe02806b080 0x7fe02811e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:01.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.934+0000 7fe02ecf8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe028131d10 0x7fe02811ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:01.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.934+0000 7fe02ecf8700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe028121ae0 con 0x7fe02806b080 2023-09-30T22:36:01.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.934+0000 7fe02ecf8700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe028130ce0 con 0x7fe02806a770 2023-09-30T22:36:01.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.934+0000 7fe02ecf8700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe02811fe70 con 0x7fe028131d10 2023-09-30T22:36:01.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.934+0000 7fe02ca94700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe02806a770 0x7fe02811e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:01.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.934+0000 7fe027fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe02806b080 0x7fe02811e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:01.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.934+0000 7fe02ca94700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe02806a770 0x7fe02811e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47490/0 (socket says 172.21.15.16:47490) 2023-09-30T22:36:01.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.934+0000 7fe02ca94700 1 -- 172.21.15.16:0/4279701879 learned_addr learned my addr 172.21.15.16:0/4279701879 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:01.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.934+0000 7fe02d295700 1 --2- 172.21.15.16:0/4279701879 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe028131d10 0x7fe02811ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:01.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.934+0000 7fe02ca94700 1 -- 172.21.15.16:0/4279701879 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe028131d10 msgr2=0x7fe02811ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:01.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.934+0000 7fe02ca94700 1 --2- 172.21.15.16:0/4279701879 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe028131d10 0x7fe02811ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:01.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.934+0000 7fe02ca94700 1 -- 172.21.15.16:0/4279701879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe02806b080 msgr2=0x7fe02811e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:01.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.934+0000 7fe02ca94700 1 --2- 172.21.15.16:0/4279701879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe02806b080 0x7fe02811e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:01.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.934+0000 7fe02ca94700 1 -- 172.21.15.16:0/4279701879 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe020009b30 con 0x7fe02806a770 2023-09-30T22:36:01.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.934+0000 7fe027fff700 1 --2- 172.21.15.16:0/4279701879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe02806b080 0x7fe02811e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:01.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.934+0000 7fe02ca94700 1 --2- 172.21.15.16:0/4279701879 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe02806a770 0x7fe02811e1c0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fe01c0090b0 tx=0x7fe01c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:01.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.935+0000 7fe0257fa700 1 -- 172.21.15.16:0/4279701879 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01c0036a0 con 0x7fe02806a770 2023-09-30T22:36:01.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.935+0000 7fe0257fa700 1 -- 172.21.15.16:0/4279701879 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe01c01f070 con 0x7fe02806a770 2023-09-30T22:36:01.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.935+0000 7fe02ecf8700 1 -- 172.21.15.16:0/4279701879 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe01c0094a0 con 0x7fe02806a770 2023-09-30T22:36:01.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.935+0000 7fe0257fa700 1 -- 172.21.15.16:0/4279701879 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01c003a40 con 0x7fe02806a770 2023-09-30T22:36:01.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.935+0000 7fe02ecf8700 1 -- 172.21.15.16:0/4279701879 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe028071eb0 con 0x7fe02806a770 2023-09-30T22:36:01.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.936+0000 7fe0257fa700 1 -- 172.21.15.16:0/4279701879 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe01c003ba0 con 0x7fe02806a770 2023-09-30T22:36:01.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.936+0000 7fe02ecf8700 1 -- 172.21.15.16:0/4279701879 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe028074ea0 con 0x7fe02806a770 2023-09-30T22:36:01.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.936+0000 7fe0257fa700 1 -- 172.21.15.16:0/4279701879 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fe01c021070 con 0x7fe02806a770 2023-09-30T22:36:01.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.941+0000 7fe0257fa700 1 -- 172.21.15.16:0/4279701879 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe01c019d40 con 0x7fe02806a770 2023-09-30T22:36:01.967 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.967+0000 7f18702de700 1 Processor -- start 2023-09-30T22:36:01.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.967+0000 7f18702de700 1 -- start start 2023-09-30T22:36:01.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.967+0000 7fdf2e6b5700 1 Processor -- start 2023-09-30T22:36:01.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.967+0000 7fdf2e6b5700 1 -- start start 2023-09-30T22:36:01.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.968+0000 7fdf2e6b5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdf2809d6f0 0x7fdf2809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:01.968 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.968+0000 7fdf2e6b5700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdf2809e000 0x7fdf281339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:01.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.968+0000 7f18702de700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f186809d6f0 0x7f186809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:01.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.968+0000 7f18702de700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f186809e000 0x7f18681339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:01.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.968+0000 7fdf2e6b5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdf2809e410 0x7fdf28133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:01.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.968+0000 7fdf2e6b5700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf2806b460 con 0x7fdf2809e000 2023-09-30T22:36:01.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.968+0000 7fdf2e6b5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2809b120 con 0x7fdf2809e410 2023-09-30T22:36:01.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.968+0000 7fdf2e6b5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2809a7e0 con 0x7fdf2809d6f0 2023-09-30T22:36:01.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.968+0000 7f18702de700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f186809e410 0x7f1868133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:01.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.968+0000 7f18702de700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f186806b460 con 0x7f186809d6f0 2023-09-30T22:36:01.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.968+0000 7f18702de700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f186809b120 con 0x7f186809e410 2023-09-30T22:36:01.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.968+0000 7f18702de700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f186809a7e0 con 0x7f186809e000 2023-09-30T22:36:01.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.968+0000 7fdf27fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdf2809d6f0 0x7fdf2809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:01.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.968+0000 7f186d879700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f186809e000 0x7f18681339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:01.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.968+0000 7f186d879700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f186809e000 0x7f18681339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:41112/0 (socket says 172.21.15.16:41112) 2023-09-30T22:36:01.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.968+0000 7f186d879700 1 -- 172.21.15.16:0/2715030688 learned_addr learned my addr 172.21.15.16:0/2715030688 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:01.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.968+0000 7f186e07a700 1 --2- 172.21.15.16:0/2715030688 >> v2:172.21.15.16:3300/0 conn(0x7f186809d6f0 0x7f186809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:01.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.968+0000 7fdf27fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdf2809d6f0 0x7fdf2809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:41100/0 (socket says 172.21.15.16:41100) 2023-09-30T22:36:01.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.968+0000 7fdf27fff700 1 -- 172.21.15.16:0/3346201410 learned_addr learned my addr 172.21.15.16:0/3346201410 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:01.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.968+0000 7fdf277fe700 1 --2- 172.21.15.16:0/3346201410 >> v2:172.21.15.16:3300/0 conn(0x7fdf2809e000 0x7fdf281339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:01.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.968+0000 7fdf2cc52700 1 --2- 172.21.15.16:0/3346201410 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdf2809e410 0x7fdf28133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:01.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.969+0000 7f186d879700 1 -- 172.21.15.16:0/2715030688 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f186809e410 msgr2=0x7f1868133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:01.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.969+0000 7f186d879700 1 --2- 172.21.15.16:0/2715030688 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f186809e410 0x7f1868133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:01.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.969+0000 7f186d879700 1 -- 172.21.15.16:0/2715030688 >> v2:172.21.15.16:3300/0 conn(0x7f186809d6f0 msgr2=0x7f186809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:01.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.969+0000 7f186d879700 1 --2- 172.21.15.16:0/2715030688 >> v2:172.21.15.16:3300/0 conn(0x7f186809d6f0 0x7f186809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:01.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.969+0000 7f186d879700 1 -- 172.21.15.16:0/2715030688 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f186809a500 con 0x7f186809e000 2023-09-30T22:36:01.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.969+0000 7f186e07a700 1 --2- 172.21.15.16:0/2715030688 >> v2:172.21.15.16:3300/0 conn(0x7f186809d6f0 0x7f186809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:01.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.969+0000 7fdf2cc52700 1 -- 172.21.15.16:0/3346201410 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdf2809d6f0 msgr2=0x7fdf2809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:01.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.969+0000 7fdf2cc52700 1 --2- 172.21.15.16:0/3346201410 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdf2809d6f0 0x7fdf2809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:01.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.969+0000 7fdf2cc52700 1 -- 172.21.15.16:0/3346201410 >> v2:172.21.15.16:3300/0 conn(0x7fdf2809e000 msgr2=0x7fdf281339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:01.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.969+0000 7fdf2cc52700 1 --2- 172.21.15.16:0/3346201410 >> v2:172.21.15.16:3300/0 conn(0x7fdf2809e000 0x7fdf281339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:01.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.969+0000 7fdf2cc52700 1 -- 172.21.15.16:0/3346201410 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf2809a500 con 0x7fdf2809e410 2023-09-30T22:36:01.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.969+0000 7f186d879700 1 --2- 172.21.15.16:0/2715030688 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f186809e000 0x7f18681339f0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f1858009920 tx=0x7f1858009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40bc91c7351c4186 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:01.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.969+0000 7fdf2cc52700 1 --2- 172.21.15.16:0/3346201410 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdf2809e410 0x7fdf28133f30 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fdf20009dc0 tx=0x7fdf2000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87e9b6a0620542da server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:01.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.969+0000 7f186d078700 1 -- 172.21.15.16:0/2715030688 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1858011830 con 0x7f186809e000 2023-09-30T22:36:01.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.969+0000 7fdf26ffd700 1 -- 172.21.15.16:0/3346201410 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2000e070 con 0x7fdf2809e410 2023-09-30T22:36:01.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.969+0000 7f186d078700 1 -- 172.21.15.16:0/2715030688 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1858011990 con 0x7f186809e000 2023-09-30T22:36:01.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.969+0000 7f186d078700 1 -- 172.21.15.16:0/2715030688 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1858011d20 con 0x7f186809e000 2023-09-30T22:36:01.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.969+0000 7fdf26ffd700 1 -- 172.21.15.16:0/3346201410 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf2000bae0 con 0x7fdf2809e410 2023-09-30T22:36:01.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.969+0000 7fdf26ffd700 1 -- 172.21.15.16:0/3346201410 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2000be70 con 0x7fdf2809e410 2023-09-30T22:36:01.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.970+0000 7f18702de700 1 -- 172.21.15.16:0/2715030688 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f186809e000 msgr2=0x7f18681339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:01.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.970+0000 7f18702de700 1 --2- 172.21.15.16:0/2715030688 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f186809e000 0x7f18681339f0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f1858009920 tx=0x7f1858009c30 comp rx=0 tx=0).stop 2023-09-30T22:36:01.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.970+0000 7fdf2e6b5700 1 -- 172.21.15.16:0/3346201410 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdf2809e410 msgr2=0x7fdf28133f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:01.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.970+0000 7fdf2e6b5700 1 --2- 172.21.15.16:0/3346201410 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdf2809e410 0x7fdf28133f30 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fdf20009dc0 tx=0x7fdf2000b330 comp rx=0 tx=0).stop 2023-09-30T22:36:01.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.970+0000 7fdf2e6b5700 1 -- 172.21.15.16:0/3346201410 shutdown_connections 2023-09-30T22:36:01.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.970+0000 7fdf2e6b5700 1 --2- 172.21.15.16:0/3346201410 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdf2809e410 0x7fdf28133f30 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:01.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.970+0000 7fdf2e6b5700 1 --2- 172.21.15.16:0/3346201410 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdf2809d6f0 0x7fdf2809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:01.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.970+0000 7fdf2e6b5700 1 --2- 172.21.15.16:0/3346201410 >> v2:172.21.15.16:3300/0 conn(0x7fdf2809e000 0x7fdf281339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:01.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.970+0000 7fdf2e6b5700 1 -- 172.21.15.16:0/3346201410 >> 172.21.15.16:0/3346201410 conn(0x7fdf28066f50 msgr2=0x7fdf2809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:01.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.970+0000 7fdf2e6b5700 1 -- 172.21.15.16:0/3346201410 shutdown_connections 2023-09-30T22:36:01.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.970+0000 7f18702de700 1 -- 172.21.15.16:0/2715030688 shutdown_connections 2023-09-30T22:36:01.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.970+0000 7fdf2e6b5700 1 -- 172.21.15.16:0/3346201410 wait complete. 2023-09-30T22:36:01.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.970+0000 7f18702de700 1 --2- 172.21.15.16:0/2715030688 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f186809e410 0x7f1868133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:01.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.970+0000 7f18702de700 1 --2- 172.21.15.16:0/2715030688 >> v2:172.21.15.16:3300/0 conn(0x7f186809d6f0 0x7f186809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:01.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.970+0000 7f18702de700 1 --2- 172.21.15.16:0/2715030688 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f186809e000 0x7f18681339f0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:01.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.970+0000 7f18702de700 1 -- 172.21.15.16:0/2715030688 >> 172.21.15.16:0/2715030688 conn(0x7f1868066f50 msgr2=0x7f186809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:01.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.970+0000 7f18702de700 1 -- 172.21.15.16:0/2715030688 shutdown_connections 2023-09-30T22:36:01.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.970+0000 7f18702de700 1 -- 172.21.15.16:0/2715030688 wait complete. 2023-09-30T22:36:01.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.971+0000 7fdf2e6b5700 1 Processor -- start 2023-09-30T22:36:01.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.971+0000 7f18702de700 1 Processor -- start 2023-09-30T22:36:01.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.971+0000 7f18702de700 1 -- start start 2023-09-30T22:36:01.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.971+0000 7fdf2e6b5700 1 -- start start 2023-09-30T22:36:01.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.972+0000 7f18702de700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f186809d6f0 0x7f18681a5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:01.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.972+0000 7f18702de700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f186809e000 0x7f18681a6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:01.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.972+0000 7f18702de700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f186809e410 0x7f18681a6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:01.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.972+0000 7f18702de700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18681a7320 con 0x7f186809e000 2023-09-30T22:36:01.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.972+0000 7f18702de700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18681a80b0 con 0x7f186809e410 2023-09-30T22:36:01.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.972+0000 7f18702de700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f18681a9170 con 0x7f186809d6f0 2023-09-30T22:36:01.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.972+0000 7f186e07a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f186809d6f0 0x7f18681a5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:01.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.972+0000 7f186e07a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f186809d6f0 0x7f18681a5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:41118/0 (socket says 172.21.15.16:41118) 2023-09-30T22:36:01.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.972+0000 7f186e07a700 1 -- 172.21.15.16:0/1593034062 learned_addr learned my addr 172.21.15.16:0/1593034062 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:01.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.972+0000 7f186d879700 1 --2- 172.21.15.16:0/1593034062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f186809e000 0x7f18681a6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:01.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.972+0000 7f186e87b700 1 --2- 172.21.15.16:0/1593034062 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f186809e410 0x7f18681a6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:01.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.972+0000 7f186e07a700 1 -- 172.21.15.16:0/1593034062 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f186809e410 msgr2=0x7f18681a6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:01.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.972+0000 7f186e07a700 1 --2- 172.21.15.16:0/1593034062 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f186809e410 0x7f18681a6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:01.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.973+0000 7f186e07a700 1 -- 172.21.15.16:0/1593034062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f186809e000 msgr2=0x7f18681a6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:01.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.973+0000 7f186e07a700 1 --2- 172.21.15.16:0/1593034062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f186809e000 0x7f18681a6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:01.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.973+0000 7f186e07a700 1 -- 172.21.15.16:0/1593034062 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18640094a0 con 0x7f186809d6f0 2023-09-30T22:36:01.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.973+0000 7f186e07a700 1 --2- 172.21.15.16:0/1593034062 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f186809d6f0 0x7f18681a5ec0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f186400ec10 tx=0x7f186400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:01.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.973+0000 7f185effd700 1 -- 172.21.15.16:0/1593034062 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f186400c910 con 0x7f186809d6f0 2023-09-30T22:36:01.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.973+0000 7f18702de700 1 -- 172.21.15.16:0/1593034062 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1858009550 con 0x7f186809d6f0 2023-09-30T22:36:01.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.973+0000 7f185effd700 1 -- 172.21.15.16:0/1593034062 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f186400ca70 con 0x7f186809d6f0 2023-09-30T22:36:01.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.973+0000 7f185effd700 1 -- 172.21.15.16:0/1593034062 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f186400cde0 con 0x7f186809d6f0 2023-09-30T22:36:01.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.973+0000 7f18702de700 1 -- 172.21.15.16:0/1593034062 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18680a2140 con 0x7f186809d6f0 2023-09-30T22:36:01.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.972+0000 7fdf2e6b5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdf2809d6f0 0x7fdf281a5d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:01.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.972+0000 7fdf2e6b5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf2809e000 0x7fdf281a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:01.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.972+0000 7fdf2e6b5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdf2809e410 0x7fdf281a6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:01.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.972+0000 7fdf2e6b5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf281aa1f0 con 0x7fdf2809e000 2023-09-30T22:36:01.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.972+0000 7fdf2e6b5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf281a7920 con 0x7fdf2809d6f0 2023-09-30T22:36:01.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.972+0000 7fdf2e6b5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdf281a7f40 con 0x7fdf2809e410 2023-09-30T22:36:01.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.972+0000 7fdf2cc52700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdf2809e410 0x7fdf281a6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:01.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.972+0000 7fdf2cc52700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdf2809e410 0x7fdf281a6790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:41126/0 (socket says 172.21.15.16:41126) 2023-09-30T22:36:01.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.972+0000 7fdf2cc52700 1 -- 172.21.15.16:0/2573640284 learned_addr learned my addr 172.21.15.16:0/2573640284 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:01.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.972+0000 7fdf27fff700 1 --2- 172.21.15.16:0/2573640284 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdf2809d6f0 0x7fdf281a5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:01.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.972+0000 7fdf277fe700 1 --2- 172.21.15.16:0/2573640284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf2809e000 0x7fdf281a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:01.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.973+0000 7fdf27fff700 1 -- 172.21.15.16:0/2573640284 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdf2809e410 msgr2=0x7fdf281a6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:01.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.973+0000 7fdf27fff700 1 --2- 172.21.15.16:0/2573640284 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdf2809e410 0x7fdf281a6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:01.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.973+0000 7fdf27fff700 1 -- 172.21.15.16:0/2573640284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf2809e000 msgr2=0x7fdf281a6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:01.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.973+0000 7fdf27fff700 1 --2- 172.21.15.16:0/2573640284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf2809e000 0x7fdf281a6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:01.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.973+0000 7fdf27fff700 1 -- 172.21.15.16:0/2573640284 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf20009a70 con 0x7fdf2809d6f0 2023-09-30T22:36:01.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.973+0000 7fdf2cc52700 1 --2- 172.21.15.16:0/2573640284 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdf2809e410 0x7fdf281a6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:36:01.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.973+0000 7fdf27fff700 1 --2- 172.21.15.16:0/2573640284 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdf2809d6f0 0x7fdf281a5d10 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fdf1800e910 tx=0x7fdf1800ec20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:01.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.974+0000 7fdf24ff9700 1 -- 172.21.15.16:0/2573640284 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1800ee60 con 0x7fdf2809d6f0 2023-09-30T22:36:01.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.974+0000 7fdf24ff9700 1 -- 172.21.15.16:0/2573640284 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf1800c850 con 0x7fdf2809d6f0 2023-09-30T22:36:01.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.974+0000 7fdf24ff9700 1 -- 172.21.15.16:0/2573640284 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1800cbe0 con 0x7fdf2809d6f0 2023-09-30T22:36:01.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.974+0000 7f185effd700 1 -- 172.21.15.16:0/1593034062 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1864013470 con 0x7f186809d6f0 2023-09-30T22:36:01.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.974+0000 7f185effd700 1 -- 172.21.15.16:0/1593034062 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f186403fc90 con 0x7f186809d6f0 2023-09-30T22:36:01.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.974+0000 7f18702de700 1 -- 172.21.15.16:0/1593034062 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18681a7b30 con 0x7f186809d6f0 2023-09-30T22:36:01.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.975+0000 7fdf2e6b5700 1 -- 172.21.15.16:0/2573640284 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf281a8b70 con 0x7fdf2809d6f0 2023-09-30T22:36:01.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.975+0000 7fdf2e6b5700 1 -- 172.21.15.16:0/2573640284 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf281a95b0 con 0x7fdf2809d6f0 2023-09-30T22:36:01.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.975+0000 7fdf24ff9700 1 -- 172.21.15.16:0/2573640284 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdf1800cd40 con 0x7fdf2809d6f0 2023-09-30T22:36:01.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.976+0000 7fdf24ff9700 1 -- 172.21.15.16:0/2573640284 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fdf1803f740 con 0x7fdf2809d6f0 2023-09-30T22:36:01.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.976+0000 7fdf2e6b5700 1 -- 172.21.15.16:0/2573640284 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf2809c710 con 0x7fdf2809d6f0 2023-09-30T22:36:01.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.980+0000 7f185effd700 1 -- 172.21.15.16:0/1593034062 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1864017080 con 0x7f186809d6f0 2023-09-30T22:36:01.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:01.982+0000 7fdf24ff9700 1 -- 172.21.15.16:0/2573640284 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdf18017080 con 0x7fdf2809d6f0 2023-09-30T22:36:02.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.169+0000 7fe02ecf8700 1 -- 172.21.15.16:0/4279701879 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe0281224d0 con 0x7fe02806a770 2023-09-30T22:36:02.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.169+0000 7fe0257fa700 1 -- 172.21.15.16:0/4279701879 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe01c034020 con 0x7fe02806a770 2023-09-30T22:36:02.170 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:36:02.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.171+0000 7fe02ecf8700 1 -- 172.21.15.16:0/4279701879 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe02806a770 msgr2=0x7fe02811e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:02.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.171+0000 7fe02ecf8700 1 --2- 172.21.15.16:0/4279701879 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe02806a770 0x7fe02811e1c0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fe01c0090b0 tx=0x7fe01c004750 comp rx=0 tx=0).stop 2023-09-30T22:36:02.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.172+0000 7fe02ecf8700 1 -- 172.21.15.16:0/4279701879 shutdown_connections 2023-09-30T22:36:02.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.172+0000 7fe02ecf8700 1 --2- 172.21.15.16:0/4279701879 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe02806a770 0x7fe02811e1c0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:02.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.172+0000 7fe02ecf8700 1 --2- 172.21.15.16:0/4279701879 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe028131d10 0x7fe02811ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:02.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.172+0000 7fe02ecf8700 1 --2- 172.21.15.16:0/4279701879 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe02806b080 0x7fe02811e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:02.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.172+0000 7fe02ecf8700 1 -- 172.21.15.16:0/4279701879 >> 172.21.15.16:0/4279701879 conn(0x7fe028066f50 msgr2=0x7fe0280a1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:02.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.172+0000 7fe02ecf8700 1 -- 172.21.15.16:0/4279701879 shutdown_connections 2023-09-30T22:36:02.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.172+0000 7fe02ecf8700 1 -- 172.21.15.16:0/4279701879 wait complete. 2023-09-30T22:36:02.186 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:36:02.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.188+0000 7f18702de700 1 -- 172.21.15.16:0/1593034062 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f18681a9580 con 0x7f186809d6f0 2023-09-30T22:36:02.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.189+0000 7f185effd700 1 -- 172.21.15.16:0/1593034062 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f18681a9580 con 0x7f186809d6f0 2023-09-30T22:36:02.189 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:36:02.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.191+0000 7f18702de700 1 -- 172.21.15.16:0/1593034062 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f186809d6f0 msgr2=0x7f18681a5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:02.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.191+0000 7f18702de700 1 --2- 172.21.15.16:0/1593034062 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f186809d6f0 0x7f18681a5ec0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f186400ec10 tx=0x7f186400ef20 comp rx=0 tx=0).stop 2023-09-30T22:36:02.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.191+0000 7f18702de700 1 -- 172.21.15.16:0/1593034062 shutdown_connections 2023-09-30T22:36:02.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.191+0000 7f18702de700 1 --2- 172.21.15.16:0/1593034062 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f186809e410 0x7f18681a6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:02.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.191+0000 7f18702de700 1 --2- 172.21.15.16:0/1593034062 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f186809d6f0 0x7f18681a5ec0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:02.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.191+0000 7f18702de700 1 --2- 172.21.15.16:0/1593034062 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f186809e000 0x7f18681a6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:02.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.191+0000 7f18702de700 1 -- 172.21.15.16:0/1593034062 >> 172.21.15.16:0/1593034062 conn(0x7f1868066f50 msgr2=0x7f18681bae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:02.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.191+0000 7f18702de700 1 -- 172.21.15.16:0/1593034062 shutdown_connections 2023-09-30T22:36:02.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.191+0000 7f18702de700 1 -- 172.21.15.16:0/1593034062 wait complete. 2023-09-30T22:36:02.205 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:36:02.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.208+0000 7fdf2e6b5700 1 -- 172.21.15.16:0/2573640284 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fdf28074ea0 con 0x7fdf2809d6f0 2023-09-30T22:36:02.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.208+0000 7fdf24ff9700 1 -- 172.21.15.16:0/2573640284 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdf1801b3b0 con 0x7fdf2809d6f0 2023-09-30T22:36:02.208 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:36:02.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.210+0000 7fdf2e6b5700 1 -- 172.21.15.16:0/2573640284 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdf2809d6f0 msgr2=0x7fdf281a5d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:02.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.210+0000 7fdf2e6b5700 1 --2- 172.21.15.16:0/2573640284 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdf2809d6f0 0x7fdf281a5d10 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fdf1800e910 tx=0x7fdf1800ec20 comp rx=0 tx=0).stop 2023-09-30T22:36:02.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.210+0000 7fdf2e6b5700 1 -- 172.21.15.16:0/2573640284 shutdown_connections 2023-09-30T22:36:02.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.210+0000 7fdf2e6b5700 1 --2- 172.21.15.16:0/2573640284 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdf2809d6f0 0x7fdf281a5d10 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:02.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.210+0000 7fdf2e6b5700 1 --2- 172.21.15.16:0/2573640284 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdf2809e410 0x7fdf281a6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:02.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.210+0000 7fdf2e6b5700 1 --2- 172.21.15.16:0/2573640284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdf2809e000 0x7fdf281a6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:02.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.210+0000 7fdf2e6b5700 1 -- 172.21.15.16:0/2573640284 >> 172.21.15.16:0/2573640284 conn(0x7fdf28066f50 msgr2=0x7fdf281bac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:02.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.210+0000 7fdf2e6b5700 1 -- 172.21.15.16:0/2573640284 shutdown_connections 2023-09-30T22:36:02.211 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.210+0000 7fdf2e6b5700 1 -- 172.21.15.16:0/2573640284 wait complete. 2023-09-30T22:36:02.222 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:36:02.225 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:36:02.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.359+0000 7f72dd1d3700 1 Processor -- start 2023-09-30T22:36:02.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.359+0000 7f72dd1d3700 1 -- start start 2023-09-30T22:36:02.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.359+0000 7f72dd1d3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72d81337f0 0x7f72d8138880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:02.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.359+0000 7f72dd1d3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72d8138dc0 0x7f72d813b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:02.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.359+0000 7f72dd1d3700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72d806b990 con 0x7f72d8133420 2023-09-30T22:36:02.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.359+0000 7f72dd1d3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72d809b0e0 con 0x7f72d8138dc0 2023-09-30T22:36:02.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.359+0000 7f72dd1d3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f72d8133c20 con 0x7f72d81337f0 2023-09-30T22:36:02.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.359+0000 7f72d659c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72d81337f0 0x7f72d8138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:02.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.359+0000 7f72d759e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72d8138dc0 0x7f72d813b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:02.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.359+0000 7f72d6d9d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f72d8133420 0x7f72d8133e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38308/0 (socket says 172.21.15.16:38308) 2023-09-30T22:36:02.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.359+0000 7f72d659c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72d81337f0 0x7f72d8138880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:41134/0 (socket says 172.21.15.16:41134) 2023-09-30T22:36:02.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.359+0000 7f72d6d9d700 1 -- 172.21.15.16:0/3956963674 learned_addr learned my addr 172.21.15.16:0/3956963674 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:36:02.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.360+0000 7f72d759e700 1 -- 172.21.15.16:0/3956963674 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72d81337f0 msgr2=0x7f72d8138880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:02.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.360+0000 7f72d759e700 1 --2- 172.21.15.16:0/3956963674 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72d81337f0 0x7f72d8138880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:02.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.360+0000 7f72d5d9b700 1 -- 172.21.15.16:0/3956963674 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1630744271 0 0) 0x7f72d806b990 con 0x7f72d8133420 2023-09-30T22:36:02.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.360+0000 7f72d759e700 1 -- 172.21.15.16:0/3956963674 >> v1:172.21.15.16:6789/0 conn(0x7f72d8133420 legacy=0x7f72d8133e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:02.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.360+0000 7f72d759e700 1 -- 172.21.15.16:0/3956963674 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d80a0590 con 0x7f72d8138dc0 2023-09-30T22:36:02.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.360+0000 7f72d659c700 1 --2- 172.21.15.16:0/3956963674 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72d81337f0 0x7f72d8138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:02.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.360+0000 7f72d759e700 1 --2- 172.21.15.16:0/3956963674 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72d8138dc0 0x7f72d813b1a0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f72d0009870 tx=0x7f72d0009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc8b3e7b813cb4e9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:02.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.360+0000 7f72d5d9b700 1 -- 172.21.15.16:0/3956963674 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d0011830 con 0x7f72d8138dc0 2023-09-30T22:36:02.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.360+0000 7f72d5d9b700 1 -- 172.21.15.16:0/3956963674 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72d0011990 con 0x7f72d8138dc0 2023-09-30T22:36:02.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.360+0000 7f72d5d9b700 1 -- 172.21.15.16:0/3956963674 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d0011d20 con 0x7f72d8138dc0 2023-09-30T22:36:02.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.361+0000 7f72dd1d3700 1 -- 172.21.15.16:0/3956963674 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72d8138dc0 msgr2=0x7f72d813b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:02.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.361+0000 7f72dd1d3700 1 --2- 172.21.15.16:0/3956963674 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72d8138dc0 0x7f72d813b1a0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f72d0009870 tx=0x7f72d0009b80 comp rx=0 tx=0).stop 2023-09-30T22:36:02.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.361+0000 7f72dd1d3700 1 -- 172.21.15.16:0/3956963674 shutdown_connections 2023-09-30T22:36:02.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.361+0000 7f72dd1d3700 1 --2- 172.21.15.16:0/3956963674 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72d8138dc0 0x7f72d813b1a0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:02.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.361+0000 7f72dd1d3700 1 --2- 172.21.15.16:0/3956963674 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72d81337f0 0x7f72d8138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:02.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.361+0000 7f72dd1d3700 1 -- 172.21.15.16:0/3956963674 >> 172.21.15.16:0/3956963674 conn(0x7f72d80a4210 msgr2=0x7f72d80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:02.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.361+0000 7f72dd1d3700 1 -- 172.21.15.16:0/3956963674 shutdown_connections 2023-09-30T22:36:02.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.361+0000 7f72dd1d3700 1 -- 172.21.15.16:0/3956963674 wait complete. 2023-09-30T22:36:02.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.362+0000 7f72dd1d3700 1 Processor -- start 2023-09-30T22:36:02.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.362+0000 7f72dd1d3700 1 -- start start 2023-09-30T22:36:02.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.363+0000 7f72dd1d3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72d8133420 0x7f72d812ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:02.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.363+0000 7f72dd1d3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72d81337f0 0x7f72d812f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:02.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.363+0000 7f72dd1d3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72d8138dc0 0x7f72d812f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:02.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.363+0000 7f72dd1d3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d809c4e0 con 0x7f72d81337f0 2023-09-30T22:36:02.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.363+0000 7f72dd1d3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72d809cd10 con 0x7f72d8133420 2023-09-30T22:36:02.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.363+0000 7f72dd1d3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f72d809e460 con 0x7f72d8138dc0 2023-09-30T22:36:02.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.363+0000 7f72d6d9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72d8133420 0x7f72d812ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:02.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.363+0000 7f72d759e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72d8138dc0 0x7f72d812f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:02.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.363+0000 7f72d659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72d81337f0 0x7f72d812f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:02.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.363+0000 7f72d6d9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72d8133420 0x7f72d812ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47548/0 (socket says 172.21.15.16:47548) 2023-09-30T22:36:02.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.363+0000 7f72d6d9d700 1 -- 172.21.15.16:0/3162669165 learned_addr learned my addr 172.21.15.16:0/3162669165 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:02.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.363+0000 7f72d759e700 1 -- 172.21.15.16:0/3162669165 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72d8133420 msgr2=0x7f72d812ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:02.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.363+0000 7f72d759e700 1 --2- 172.21.15.16:0/3162669165 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72d8133420 0x7f72d812ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:02.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.363+0000 7f72d759e700 1 -- 172.21.15.16:0/3162669165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72d81337f0 msgr2=0x7f72d812f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:02.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.363+0000 7f72d759e700 1 --2- 172.21.15.16:0/3162669165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72d81337f0 0x7f72d812f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:02.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.363+0000 7f72d759e700 1 -- 172.21.15.16:0/3162669165 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72cc0094a0 con 0x7f72d8138dc0 2023-09-30T22:36:02.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.363+0000 7f72d6d9d700 1 --2- 172.21.15.16:0/3162669165 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72d8133420 0x7f72d812ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:02.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.363+0000 7f72d659c700 1 --2- 172.21.15.16:0/3162669165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72d81337f0 0x7f72d812f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:02.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.363+0000 7f72d759e700 1 --2- 172.21.15.16:0/3162669165 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72d8138dc0 0x7f72d812f8d0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f72d0016000 tx=0x7f72d0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:02.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.364+0000 7f72bf7fe700 1 -- 172.21.15.16:0/3162669165 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d0004450 con 0x7f72d8138dc0 2023-09-30T22:36:02.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.364+0000 7f72bf7fe700 1 -- 172.21.15.16:0/3162669165 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72d00045d0 con 0x7f72d8138dc0 2023-09-30T22:36:02.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.364+0000 7f72dd1d3700 1 -- 172.21.15.16:0/3162669165 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72d00094a0 con 0x7f72d8138dc0 2023-09-30T22:36:02.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.364+0000 7f72bf7fe700 1 -- 172.21.15.16:0/3162669165 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d00038e0 con 0x7f72d8138dc0 2023-09-30T22:36:02.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.364+0000 7f72dd1d3700 1 -- 172.21.15.16:0/3162669165 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72d809b9d0 con 0x7f72d8138dc0 2023-09-30T22:36:02.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.364+0000 7f72bf7fe700 1 -- 172.21.15.16:0/3162669165 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f72d0003a40 con 0x7f72d8138dc0 2023-09-30T22:36:02.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.365+0000 7f72bf7fe700 1 -- 172.21.15.16:0/3162669165 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f72d0040420 con 0x7f72d8138dc0 2023-09-30T22:36:02.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.365+0000 7f72dd1d3700 1 -- 172.21.15.16:0/3162669165 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72d8074ea0 con 0x7f72d8138dc0 2023-09-30T22:36:02.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.370+0000 7f72bf7fe700 1 -- 172.21.15.16:0/3162669165 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f72d0015080 con 0x7f72d8138dc0 2023-09-30T22:36:02.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.571+0000 7f72dd1d3700 1 -- 172.21.15.16:0/3162669165 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f72d806b250 con 0x7f72d8138dc0 2023-09-30T22:36:02.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.571+0000 7f72bf7fe700 1 -- 172.21.15.16:0/3162669165 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f72d0028070 con 0x7f72d8138dc0 2023-09-30T22:36:02.572 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:36:02.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.573+0000 7f72dd1d3700 1 -- 172.21.15.16:0/3162669165 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72d8138dc0 msgr2=0x7f72d812f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:02.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.573+0000 7f72dd1d3700 1 --2- 172.21.15.16:0/3162669165 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72d8138dc0 0x7f72d812f8d0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f72d0016000 tx=0x7f72d0004750 comp rx=0 tx=0).stop 2023-09-30T22:36:02.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.573+0000 7f72dd1d3700 1 -- 172.21.15.16:0/3162669165 shutdown_connections 2023-09-30T22:36:02.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.573+0000 7f72dd1d3700 1 --2- 172.21.15.16:0/3162669165 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f72d8133420 0x7f72d812ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:02.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.573+0000 7f72dd1d3700 1 --2- 172.21.15.16:0/3162669165 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f72d8138dc0 0x7f72d812f8d0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:02.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.573+0000 7f72dd1d3700 1 --2- 172.21.15.16:0/3162669165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72d81337f0 0x7f72d812f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:02.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.573+0000 7f72dd1d3700 1 -- 172.21.15.16:0/3162669165 >> 172.21.15.16:0/3162669165 conn(0x7f72d80a4210 msgr2=0x7f72d80a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:02.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.573+0000 7f72dd1d3700 1 -- 172.21.15.16:0/3162669165 shutdown_connections 2023-09-30T22:36:02.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:02.573+0000 7f72dd1d3700 1 -- 172.21.15.16:0/3162669165 wait complete. 2023-09-30T22:36:02.587 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:36:03.187 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:36:03.206 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:36:03.226 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:36:03.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.356+0000 7f7f7104b700 1 Processor -- start 2023-09-30T22:36:03.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.357+0000 7f7f7104b700 1 -- start start 2023-09-30T22:36:03.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.357+0000 7f7f7104b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f6c09d6f0 0x7f7f6c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:03.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.357+0000 7f7f7104b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f6c09e000 0x7f7f6c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:03.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.357+0000 7f7f7104b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f6c09b120 con 0x7f7f6c09e410 2023-09-30T22:36:03.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.357+0000 7f7f7104b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6c06b460 con 0x7f7f6c09e000 2023-09-30T22:36:03.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.357+0000 7f7f7104b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6c09a7e0 con 0x7f7f6c09d6f0 2023-09-30T22:36:03.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.358+0000 7f7f6a59c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f6c09e000 0x7f7f6c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:03.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.358+0000 7f7f6b59e700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7f6c09e410 0x7f7f6c133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38310/0 (socket says 172.21.15.16:38310) 2023-09-30T22:36:03.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.358+0000 7f7f6b59e700 1 -- 172.21.15.16:0/3388989518 learned_addr learned my addr 172.21.15.16:0/3388989518 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:36:03.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.358+0000 7f7f6ad9d700 1 --2- 172.21.15.16:0/3388989518 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f6c09d6f0 0x7f7f6c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:03.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.358+0000 7f7f6a59c700 1 -- 172.21.15.16:0/3388989518 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f6c09d6f0 msgr2=0x7f7f6c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:03.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.358+0000 7f7f6a59c700 1 --2- 172.21.15.16:0/3388989518 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f6c09d6f0 0x7f7f6c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:03.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.358+0000 7f7f6a59c700 1 -- 172.21.15.16:0/3388989518 >> v1:172.21.15.16:6789/0 conn(0x7f7f6c09e410 legacy=0x7f7f6c133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:03.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.358+0000 7f7f6a59c700 1 -- 172.21.15.16:0/3388989518 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f6c09a500 con 0x7f7f6c09e000 2023-09-30T22:36:03.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.358+0000 7f7f69d9b700 1 -- 172.21.15.16:0/3388989518 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1946613567 0 0) 0x7f7f6c09b120 con 0x7f7f6c09e410 2023-09-30T22:36:03.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.358+0000 7f7f6ad9d700 1 --2- 172.21.15.16:0/3388989518 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f6c09d6f0 0x7f7f6c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:03.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.358+0000 7f7f6a59c700 1 --2- 172.21.15.16:0/3388989518 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f6c09e000 0x7f7f6c1339f0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f7f60009870 tx=0x7f7f60009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56eb59e30f08845f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:03.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.358+0000 7f7f69d9b700 1 -- 172.21.15.16:0/3388989518 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f60011830 con 0x7f7f6c09e000 2023-09-30T22:36:03.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.358+0000 7f7f69d9b700 1 -- 172.21.15.16:0/3388989518 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f60011990 con 0x7f7f6c09e000 2023-09-30T22:36:03.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.359+0000 7f7f69d9b700 1 -- 172.21.15.16:0/3388989518 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f60011d20 con 0x7f7f6c09e000 2023-09-30T22:36:03.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.359+0000 7f7f7104b700 1 -- 172.21.15.16:0/3388989518 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f6c09e000 msgr2=0x7f7f6c1339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:03.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.359+0000 7f7f7104b700 1 --2- 172.21.15.16:0/3388989518 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f6c09e000 0x7f7f6c1339f0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f7f60009870 tx=0x7f7f60009b80 comp rx=0 tx=0).stop 2023-09-30T22:36:03.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.359+0000 7f7f7104b700 1 -- 172.21.15.16:0/3388989518 shutdown_connections 2023-09-30T22:36:03.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.359+0000 7f7f7104b700 1 --2- 172.21.15.16:0/3388989518 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f6c09d6f0 0x7f7f6c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:03.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.360+0000 7f7f7104b700 1 --2- 172.21.15.16:0/3388989518 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f6c09e000 0x7f7f6c1339f0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:03.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.360+0000 7f7f7104b700 1 -- 172.21.15.16:0/3388989518 >> 172.21.15.16:0/3388989518 conn(0x7f7f6c066f50 msgr2=0x7f7f6c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:03.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.360+0000 7f7f7104b700 1 -- 172.21.15.16:0/3388989518 shutdown_connections 2023-09-30T22:36:03.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.360+0000 7f7f7104b700 1 -- 172.21.15.16:0/3388989518 wait complete. 2023-09-30T22:36:03.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.361+0000 7f7f7104b700 1 Processor -- start 2023-09-30T22:36:03.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.362+0000 7f7f7104b700 1 -- start start 2023-09-30T22:36:03.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.362+0000 7f7f7104b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f6c09d6f0 0x7f7f6c122470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:03.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.362+0000 7f7f7104b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f6c09e000 0x7f7f6c1229b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:03.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.362+0000 7f7f7104b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f6c09e410 0x7f7f6c122ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:03.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.362+0000 7f7f7104b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6c125cb0 con 0x7f7f6c09e410 2023-09-30T22:36:03.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.362+0000 7f7f7104b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6c125f90 con 0x7f7f6c09d6f0 2023-09-30T22:36:03.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.362+0000 7f7f7104b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6c0a30d0 con 0x7f7f6c09e000 2023-09-30T22:36:03.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.362+0000 7f7f6a59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f6c09e000 0x7f7f6c1229b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:03.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.362+0000 7f7f6ad9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f6c09d6f0 0x7f7f6c122470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:03.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.363+0000 7f7f6a59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f6c09e000 0x7f7f6c1229b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:41158/0 (socket says 172.21.15.16:41158) 2023-09-30T22:36:03.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.363+0000 7f7f6a59c700 1 -- 172.21.15.16:0/3380773215 learned_addr learned my addr 172.21.15.16:0/3380773215 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:03.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.363+0000 7f7f6b59e700 1 --2- 172.21.15.16:0/3380773215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f6c09e410 0x7f7f6c122ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:03.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.363+0000 7f7f6a59c700 1 -- 172.21.15.16:0/3380773215 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f6c09d6f0 msgr2=0x7f7f6c122470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:03.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.363+0000 7f7f6a59c700 1 --2- 172.21.15.16:0/3380773215 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f6c09d6f0 0x7f7f6c122470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:03.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.363+0000 7f7f6a59c700 1 -- 172.21.15.16:0/3380773215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f6c09e410 msgr2=0x7f7f6c122ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:03.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.363+0000 7f7f6a59c700 1 --2- 172.21.15.16:0/3380773215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f6c09e410 0x7f7f6c122ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:03.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.363+0000 7f7f6a59c700 1 -- 172.21.15.16:0/3380773215 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f5c0094a0 con 0x7f7f6c09e000 2023-09-30T22:36:03.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.363+0000 7f7f6b59e700 1 --2- 172.21.15.16:0/3380773215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f6c09e410 0x7f7f6c122ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:36:03.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.363+0000 7f7f6ad9d700 1 --2- 172.21.15.16:0/3380773215 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f6c09d6f0 0x7f7f6c122470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:03.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.363+0000 7f7f6a59c700 1 --2- 172.21.15.16:0/3380773215 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f6c09e000 0x7f7f6c1229b0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f7f600090b0 tx=0x7f7f60004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:03.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.363+0000 7f7f537fe700 1 -- 172.21.15.16:0/3380773215 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6001a6e0 con 0x7f7f6c09e000 2023-09-30T22:36:03.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.363+0000 7f7f537fe700 1 -- 172.21.15.16:0/3380773215 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f6001a840 con 0x7f7f6c09e000 2023-09-30T22:36:03.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.364+0000 7f7f7104b700 1 -- 172.21.15.16:0/3380773215 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f600094a0 con 0x7f7f6c09e000 2023-09-30T22:36:03.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.364+0000 7f7f537fe700 1 -- 172.21.15.16:0/3380773215 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6001abb0 con 0x7f7f6c09e000 2023-09-30T22:36:03.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.364+0000 7f7f7104b700 1 -- 172.21.15.16:0/3380773215 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f6c124ea0 con 0x7f7f6c09e000 2023-09-30T22:36:03.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.364+0000 7f7f537fe700 1 -- 172.21.15.16:0/3380773215 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7f60022070 con 0x7f7f6c09e000 2023-09-30T22:36:03.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.365+0000 7f7f537fe700 1 -- 172.21.15.16:0/3380773215 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f7f60040690 con 0x7f7f6c09e000 2023-09-30T22:36:03.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.365+0000 7f7f7104b700 1 -- 172.21.15.16:0/3380773215 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f6c125b10 con 0x7f7f6c09e000 2023-09-30T22:36:03.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.370+0000 7f7f537fe700 1 -- 172.21.15.16:0/3380773215 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7f6c125b10 con 0x7f7f6c09e000 2023-09-30T22:36:03.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.386+0000 7f3500ea8700 1 Processor -- start 2023-09-30T22:36:03.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.386+0000 7f3500ea8700 1 -- start start 2023-09-30T22:36:03.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.387+0000 7f639d59b700 1 Processor -- start 2023-09-30T22:36:03.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.387+0000 7f639d59b700 1 -- start start 2023-09-30T22:36:03.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.387+0000 7f3500ea8700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f34fc09e930 0x7f34fc09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:03.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.387+0000 7f3500ea8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f34fc09f240 0x7f34fc1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:03.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.387+0000 7f3500ea8700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34fc070940 con 0x7f34fc09f650 2023-09-30T22:36:03.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.387+0000 7f3500ea8700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34fc06fd60 con 0x7f34fc09e930 2023-09-30T22:36:03.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.387+0000 7f3500ea8700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34fc06fa80 con 0x7f34fc09f240 2023-09-30T22:36:03.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.387+0000 7f34fa59c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f34fc09e930 0x7f34fc09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:03.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.387+0000 7f34fad9d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f34fc09f650 0x7f34fc131d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38318/0 (socket says 172.21.15.16:38318) 2023-09-30T22:36:03.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.387+0000 7f34fa59c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f34fc09e930 0x7f34fc09ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39780/0 (socket says 172.21.15.16:39780) 2023-09-30T22:36:03.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.387+0000 7f34fad9d700 1 -- 172.21.15.16:0/567226297 learned_addr learned my addr 172.21.15.16:0/567226297 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:36:03.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.387+0000 7f34f9d9b700 1 --2- 172.21.15.16:0/567226297 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f34fc09f240 0x7f34fc1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:03.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.388+0000 7f34fa59c700 1 -- 172.21.15.16:0/567226297 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f34fc09f240 msgr2=0x7f34fc1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:03.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.388+0000 7f34fa59c700 1 --2- 172.21.15.16:0/567226297 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f34fc09f240 0x7f34fc1317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:03.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.388+0000 7f34fa59c700 1 -- 172.21.15.16:0/567226297 >> v1:172.21.15.16:6789/0 conn(0x7f34fc09f650 legacy=0x7f34fc131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:03.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.387+0000 7f639d59b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f639809e1d0 0x7f6398138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:03.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.387+0000 7f639d59b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6398139170 0x7f639813b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:03.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.388+0000 7f639d59b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f639809b120 con 0x7f639809d6f0 2023-09-30T22:36:03.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.388+0000 7f639d59b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f639806b460 con 0x7f6398139170 2023-09-30T22:36:03.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.388+0000 7f639d59b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f639809a7e0 con 0x7f639809e1d0 2023-09-30T22:36:03.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.388+0000 7f34fa59c700 1 -- 172.21.15.16:0/567226297 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34fc09fb60 con 0x7f34fc09e930 2023-09-30T22:36:03.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.388+0000 7f63967fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f639809e1d0 0x7f6398138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:03.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.388+0000 7f63967fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f639809e1d0 0x7f6398138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47582/0 (socket says 172.21.15.16:47582) 2023-09-30T22:36:03.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.388+0000 7f63967fc700 1 -- 172.21.15.16:0/1902031061 learned_addr learned my addr 172.21.15.16:0/1902031061 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:03.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.388+0000 7f63977fe700 1 --2- 172.21.15.16:0/1902031061 >> v2:172.21.15.16:3300/0 conn(0x7f6398139170 0x7f639813b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:03.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.388+0000 7f34fa59c700 1 --2- 172.21.15.16:0/567226297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34fc09e930 0x7f34fc09ed00 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f34ec009840 tx=0x7f34ec009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f52a38e879573a52 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:03.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.388+0000 7f63967fc700 1 -- 172.21.15.16:0/1902031061 >> v2:172.21.15.16:3300/0 conn(0x7f6398139170 msgr2=0x7f639813b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:03.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.388+0000 7f63967fc700 1 --2- 172.21.15.16:0/1902031061 >> v2:172.21.15.16:3300/0 conn(0x7f6398139170 0x7f639813b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:03.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.388+0000 7f34f959a700 1 -- 172.21.15.16:0/567226297 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34ec011830 con 0x7f34fc09e930 2023-09-30T22:36:03.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.388+0000 7f63967fc700 1 -- 172.21.15.16:0/1902031061 >> v1:172.21.15.16:6789/0 conn(0x7f639809d6f0 legacy=0x7f639809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:03.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.388+0000 7f63967fc700 1 -- 172.21.15.16:0/1902031061 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f639809a500 con 0x7f639809e1d0 2023-09-30T22:36:03.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.388+0000 7f6395ffb700 1 -- 172.21.15.16:0/1902031061 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3161177957 0 0) 0x7f639809b120 con 0x7f639809d6f0 2023-09-30T22:36:03.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.388+0000 7f34f959a700 1 -- 172.21.15.16:0/567226297 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34ec011990 con 0x7f34fc09e930 2023-09-30T22:36:03.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.388+0000 7f34f959a700 1 -- 172.21.15.16:0/567226297 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34ec011d20 con 0x7f34fc09e930 2023-09-30T22:36:03.393 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.388+0000 7f63967fc700 1 --2- 172.21.15.16:0/1902031061 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f639809e1d0 0x7f6398138c30 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f638c009920 tx=0x7f638c009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3002eb455f6a27a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:03.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.389+0000 7f6395ffb700 1 -- 172.21.15.16:0/1902031061 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638c011830 con 0x7f639809e1d0 2023-09-30T22:36:03.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.389+0000 7f6395ffb700 1 -- 172.21.15.16:0/1902031061 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f638c011990 con 0x7f639809e1d0 2023-09-30T22:36:03.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.389+0000 7f6395ffb700 1 -- 172.21.15.16:0/1902031061 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638c011d20 con 0x7f639809e1d0 2023-09-30T22:36:03.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.389+0000 7f639d59b700 1 -- 172.21.15.16:0/1902031061 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f639809e1d0 msgr2=0x7f6398138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:03.394 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.389+0000 7f639d59b700 1 --2- 172.21.15.16:0/1902031061 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f639809e1d0 0x7f6398138c30 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f638c009920 tx=0x7f638c009c30 comp rx=0 tx=0).stop 2023-09-30T22:36:03.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.389+0000 7f3500ea8700 1 -- 172.21.15.16:0/567226297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34fc09e930 msgr2=0x7f34fc09ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:03.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.389+0000 7f3500ea8700 1 --2- 172.21.15.16:0/567226297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34fc09e930 0x7f34fc09ed00 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f34ec009840 tx=0x7f34ec009b50 comp rx=0 tx=0).stop 2023-09-30T22:36:03.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.389+0000 7f639d59b700 1 -- 172.21.15.16:0/1902031061 shutdown_connections 2023-09-30T22:36:03.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.389+0000 7f639d59b700 1 --2- 172.21.15.16:0/1902031061 >> v2:172.21.15.16:3300/0 conn(0x7f6398139170 0x7f639813b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:03.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.389+0000 7f639d59b700 1 --2- 172.21.15.16:0/1902031061 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f639809e1d0 0x7f6398138c30 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:03.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.389+0000 7f639d59b700 1 -- 172.21.15.16:0/1902031061 >> 172.21.15.16:0/1902031061 conn(0x7f6398066f50 msgr2=0x7f639809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:03.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.390+0000 7f639d59b700 1 -- 172.21.15.16:0/1902031061 shutdown_connections 2023-09-30T22:36:03.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.390+0000 7f3500ea8700 1 -- 172.21.15.16:0/567226297 shutdown_connections 2023-09-30T22:36:03.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.390+0000 7f3500ea8700 1 --2- 172.21.15.16:0/567226297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34fc09e930 0x7f34fc09ed00 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:03.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.390+0000 7f639d59b700 1 -- 172.21.15.16:0/1902031061 wait complete. 2023-09-30T22:36:03.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.390+0000 7f3500ea8700 1 --2- 172.21.15.16:0/567226297 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f34fc09f240 0x7f34fc1317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:03.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.390+0000 7f3500ea8700 1 -- 172.21.15.16:0/567226297 >> 172.21.15.16:0/567226297 conn(0x7f34fc0a4210 msgr2=0x7f34fc0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:03.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.390+0000 7f3500ea8700 1 -- 172.21.15.16:0/567226297 shutdown_connections 2023-09-30T22:36:03.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.390+0000 7f3500ea8700 1 -- 172.21.15.16:0/567226297 wait complete. 2023-09-30T22:36:03.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.391+0000 7f639d59b700 1 Processor -- start 2023-09-30T22:36:03.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.391+0000 7f3500ea8700 1 Processor -- start 2023-09-30T22:36:03.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.391+0000 7f639d59b700 1 -- start start 2023-09-30T22:36:03.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.391+0000 7f3500ea8700 1 -- start start 2023-09-30T22:36:03.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.391+0000 7f639d59b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f639809d6f0 0x7f63981207a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:03.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.391+0000 7f639d59b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f639809e1d0 0x7f6398120ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:03.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.391+0000 7f639d59b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6398139170 0x7f6398121220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:03.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.391+0000 7f639d59b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6398124360 con 0x7f639809e1d0 2023-09-30T22:36:03.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.391+0000 7f639d59b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6398123a50 con 0x7f6398139170 2023-09-30T22:36:03.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.391+0000 7f639d59b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f63980a2f40 con 0x7f639809d6f0 2023-09-30T22:36:03.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.392+0000 7f63977fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6398139170 0x7f6398121220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:03.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.392+0000 7f63977fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6398139170 0x7f6398121220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47592/0 (socket says 172.21.15.16:47592) 2023-09-30T22:36:03.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.392+0000 7f63977fe700 1 -- 172.21.15.16:0/821688763 learned_addr learned my addr 172.21.15.16:0/821688763 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:03.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.392+0000 7f6396ffd700 1 --2- 172.21.15.16:0/821688763 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f639809d6f0 0x7f63981207a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:03.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.391+0000 7f3500ea8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f34fc09e930 0x7f34fc06b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:03.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.391+0000 7f3500ea8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34fc09f240 0x7f34fc06bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:03.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.391+0000 7f3500ea8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f34fc09f650 0x7f34fc06c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:03.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.391+0000 7f3500ea8700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34fc1a89c0 con 0x7f34fc09f240 2023-09-30T22:36:03.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.391+0000 7f3500ea8700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34fc1a9a00 con 0x7f34fc09e930 2023-09-30T22:36:03.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.391+0000 7f3500ea8700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f34fc1a94f0 con 0x7f34fc09f650 2023-09-30T22:36:03.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.392+0000 7f34fad9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f34fc09f650 0x7f34fc06c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:03.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.392+0000 7f34fad9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f34fc09f650 0x7f34fc06c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:41168/0 (socket says 172.21.15.16:41168) 2023-09-30T22:36:03.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.392+0000 7f34fad9d700 1 -- 172.21.15.16:0/839601616 learned_addr learned my addr 172.21.15.16:0/839601616 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:03.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.392+0000 7f34f9d9b700 1 --2- 172.21.15.16:0/839601616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34fc09f240 0x7f34fc06bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:03.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.392+0000 7f63967fc700 1 --2- 172.21.15.16:0/821688763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f639809e1d0 0x7f6398120ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:03.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.392+0000 7f34fa59c700 1 --2- 172.21.15.16:0/839601616 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f34fc09e930 0x7f34fc06b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:03.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.392+0000 7f63977fe700 1 -- 172.21.15.16:0/821688763 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f639809d6f0 msgr2=0x7f63981207a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:03.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.392+0000 7f63977fe700 1 --2- 172.21.15.16:0/821688763 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f639809d6f0 0x7f63981207a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:03.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.392+0000 7f63977fe700 1 -- 172.21.15.16:0/821688763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f639809e1d0 msgr2=0x7f6398120ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:03.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.392+0000 7f63977fe700 1 --2- 172.21.15.16:0/821688763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f639809e1d0 0x7f6398120ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:03.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.392+0000 7f63977fe700 1 -- 172.21.15.16:0/821688763 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f638c009550 con 0x7f6398139170 2023-09-30T22:36:03.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.392+0000 7f6396ffd700 1 --2- 172.21.15.16:0/821688763 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f639809d6f0 0x7f63981207a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:03.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.392+0000 7f34fad9d700 1 -- 172.21.15.16:0/839601616 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f34fc09e930 msgr2=0x7f34fc06b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:03.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.392+0000 7f34fad9d700 1 --2- 172.21.15.16:0/839601616 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f34fc09e930 0x7f34fc06b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:03.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.392+0000 7f34fad9d700 1 -- 172.21.15.16:0/839601616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34fc09f240 msgr2=0x7f34fc06bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:03.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.392+0000 7f34fad9d700 1 --2- 172.21.15.16:0/839601616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34fc09f240 0x7f34fc06bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:03.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.392+0000 7f34fad9d700 1 -- 172.21.15.16:0/839601616 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34ec0094a0 con 0x7f34fc09f650 2023-09-30T22:36:03.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.392+0000 7f63977fe700 1 --2- 172.21.15.16:0/821688763 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6398139170 0x7f6398121220 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f639000efc0 tx=0x7f639000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:03.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.393+0000 7f637f7fe700 1 -- 172.21.15.16:0/821688763 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6390009ea0 con 0x7f6398139170 2023-09-30T22:36:03.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.393+0000 7f34fad9d700 1 --2- 172.21.15.16:0/839601616 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f34fc09f650 0x7f34fc06c330 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f34f400cbc0 tx=0x7f34f400ced0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:03.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.393+0000 7f637f7fe700 1 -- 172.21.15.16:0/821688763 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f639000f040 con 0x7f6398139170 2023-09-30T22:36:03.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.393+0000 7f639d59b700 1 -- 172.21.15.16:0/821688763 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f639809c710 con 0x7f6398139170 2023-09-30T22:36:03.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.393+0000 7f637f7fe700 1 -- 172.21.15.16:0/821688763 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6390014480 con 0x7f6398139170 2023-09-30T22:36:03.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.393+0000 7f639d59b700 1 -- 172.21.15.16:0/821688763 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6398122b60 con 0x7f6398139170 2023-09-30T22:36:03.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.393+0000 7f34eaffd700 1 -- 172.21.15.16:0/839601616 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f400aaa0 con 0x7f34fc09f650 2023-09-30T22:36:03.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.393+0000 7f34eaffd700 1 -- 172.21.15.16:0/839601616 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34f400ac00 con 0x7f34fc09f650 2023-09-30T22:36:03.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.393+0000 7f34eaffd700 1 -- 172.21.15.16:0/839601616 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f4012400 con 0x7f34fc09f650 2023-09-30T22:36:03.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.394+0000 7f637f7fe700 1 -- 172.21.15.16:0/821688763 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f63900145e0 con 0x7f6398139170 2023-09-30T22:36:03.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.394+0000 7f637f7fe700 1 -- 172.21.15.16:0/821688763 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f639003ff50 con 0x7f6398139170 2023-09-30T22:36:03.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.394+0000 7f3500ea8700 1 -- 172.21.15.16:0/839601616 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34fc0a0190 con 0x7f34fc09f650 2023-09-30T22:36:03.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.394+0000 7f639d59b700 1 -- 172.21.15.16:0/821688763 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6398123e20 con 0x7f6398139170 2023-09-30T22:36:03.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.394+0000 7f3500ea8700 1 -- 172.21.15.16:0/839601616 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34fc0a3670 con 0x7f34fc09f650 2023-09-30T22:36:03.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.395+0000 7f34eaffd700 1 -- 172.21.15.16:0/839601616 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f34f401b3b0 con 0x7f34fc09f650 2023-09-30T22:36:03.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.395+0000 7f34eaffd700 1 -- 172.21.15.16:0/839601616 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f34f403f320 con 0x7f34fc09f650 2023-09-30T22:36:03.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.395+0000 7f3500ea8700 1 -- 172.21.15.16:0/839601616 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34fc071d80 con 0x7f34fc09f650 2023-09-30T22:36:03.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.399+0000 7f637f7fe700 1 -- 172.21.15.16:0/821688763 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6390018070 con 0x7f6398139170 2023-09-30T22:36:03.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.400+0000 7f34eaffd700 1 -- 172.21.15.16:0/839601616 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f34f4017080 con 0x7f34fc09f650 2023-09-30T22:36:03.588 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:36:03.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.607+0000 7f639d59b700 1 -- 172.21.15.16:0/821688763 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f6398074ea0 con 0x7f6398139170 2023-09-30T22:36:03.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.607+0000 7f637f7fe700 1 -- 172.21.15.16:0/821688763 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f639002b090 con 0x7f6398139170 2023-09-30T22:36:03.608 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:36:03.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.609+0000 7f639d59b700 1 -- 172.21.15.16:0/821688763 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6398139170 msgr2=0x7f6398121220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:03.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.609+0000 7f639d59b700 1 --2- 172.21.15.16:0/821688763 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6398139170 0x7f6398121220 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f639000efc0 tx=0x7f639000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:36:03.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.609+0000 7f639d59b700 1 -- 172.21.15.16:0/821688763 shutdown_connections 2023-09-30T22:36:03.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.609+0000 7f639d59b700 1 --2- 172.21.15.16:0/821688763 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6398139170 0x7f6398121220 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:03.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.609+0000 7f639d59b700 1 --2- 172.21.15.16:0/821688763 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f639809d6f0 0x7f63981207a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:03.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.609+0000 7f639d59b700 1 --2- 172.21.15.16:0/821688763 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f639809e1d0 0x7f6398120ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:03.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.609+0000 7f639d59b700 1 -- 172.21.15.16:0/821688763 >> 172.21.15.16:0/821688763 conn(0x7f6398066f50 msgr2=0x7f6398070d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:03.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.610+0000 7f639d59b700 1 -- 172.21.15.16:0/821688763 shutdown_connections 2023-09-30T22:36:03.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.610+0000 7f639d59b700 1 -- 172.21.15.16:0/821688763 wait complete. 2023-09-30T22:36:03.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.629+0000 7f7f7104b700 1 -- 172.21.15.16:0/3380773215 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f7f6c125b10 con 0x7f7f6c09e000 2023-09-30T22:36:03.630 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:36:03.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.629+0000 7f7f537fe700 1 -- 172.21.15.16:0/3380773215 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7f60036020 con 0x7f7f6c09e000 2023-09-30T22:36:03.631 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:36:03.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.631+0000 7f7f7104b700 1 -- 172.21.15.16:0/3380773215 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f6c09e000 msgr2=0x7f7f6c1229b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:03.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.631+0000 7f7f7104b700 1 --2- 172.21.15.16:0/3380773215 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f6c09e000 0x7f7f6c1229b0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f7f600090b0 tx=0x7f7f60004750 comp rx=0 tx=0).stop 2023-09-30T22:36:03.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.632+0000 7f7f7104b700 1 -- 172.21.15.16:0/3380773215 shutdown_connections 2023-09-30T22:36:03.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.632+0000 7f7f7104b700 1 --2- 172.21.15.16:0/3380773215 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7f6c09d6f0 0x7f7f6c122470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:03.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.632+0000 7f7f7104b700 1 --2- 172.21.15.16:0/3380773215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7f6c09e410 0x7f7f6c122ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:03.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.632+0000 7f7f7104b700 1 --2- 172.21.15.16:0/3380773215 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7f6c09e000 0x7f7f6c1229b0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:03.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.632+0000 7f7f7104b700 1 -- 172.21.15.16:0/3380773215 >> 172.21.15.16:0/3380773215 conn(0x7f7f6c066f50 msgr2=0x7f7f6c070e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:03.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.632+0000 7f7f7104b700 1 -- 172.21.15.16:0/3380773215 shutdown_connections 2023-09-30T22:36:03.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.632+0000 7f7f7104b700 1 -- 172.21.15.16:0/3380773215 wait complete. 2023-09-30T22:36:03.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.642+0000 7f3500ea8700 1 -- 172.21.15.16:0/839601616 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f34fc1a0b30 con 0x7f34fc09f650 2023-09-30T22:36:03.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.643+0000 7f34eaffd700 1 -- 172.21.15.16:0/839601616 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f34f40192c0 con 0x7f34fc09f650 2023-09-30T22:36:03.643 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:36:03.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.645+0000 7f3500ea8700 1 -- 172.21.15.16:0/839601616 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f34fc09f650 msgr2=0x7f34fc06c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:03.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.645+0000 7f3500ea8700 1 --2- 172.21.15.16:0/839601616 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f34fc09f650 0x7f34fc06c330 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f34f400cbc0 tx=0x7f34f400ced0 comp rx=0 tx=0).stop 2023-09-30T22:36:03.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.645+0000 7f3500ea8700 1 -- 172.21.15.16:0/839601616 shutdown_connections 2023-09-30T22:36:03.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.645+0000 7f3500ea8700 1 --2- 172.21.15.16:0/839601616 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f34fc09e930 0x7f34fc06b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:03.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.645+0000 7f3500ea8700 1 --2- 172.21.15.16:0/839601616 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f34fc09f650 0x7f34fc06c330 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:03.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.645+0000 7f3500ea8700 1 --2- 172.21.15.16:0/839601616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f34fc09f240 0x7f34fc06bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:03.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.645+0000 7f3500ea8700 1 -- 172.21.15.16:0/839601616 >> 172.21.15.16:0/839601616 conn(0x7f34fc0a4210 msgr2=0x7f34fc1b8da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:03.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.645+0000 7f3500ea8700 1 -- 172.21.15.16:0/839601616 shutdown_connections 2023-09-30T22:36:03.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.645+0000 7f3500ea8700 1 -- 172.21.15.16:0/839601616 wait complete. 2023-09-30T22:36:03.647 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:36:03.660 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:36:03.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.730+0000 7f749e6e5700 1 Processor -- start 2023-09-30T22:36:03.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.730+0000 7f749e6e5700 1 -- start start 2023-09-30T22:36:03.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.731+0000 7f749e6e5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f749809d6f0 0x7f749809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:03.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.731+0000 7f749e6e5700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f749809e410 0x7f7498130d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:03.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.731+0000 7f749e6e5700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f749809f130 con 0x7f749809e000 2023-09-30T22:36:03.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.731+0000 7f749e6e5700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f749809f520 con 0x7f749809e410 2023-09-30T22:36:03.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.731+0000 7f749e6e5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f749809e930 con 0x7f749809d6f0 2023-09-30T22:36:03.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.731+0000 7f749cc82700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f749809e410 0x7f7498130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:03.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.731+0000 7f74977fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f749809e000 0x7f7498130660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38330/0 (socket says 172.21.15.16:38330) 2023-09-30T22:36:03.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.731+0000 7f7497fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f749809d6f0 0x7f749809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:03.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.731+0000 7f74977fe700 1 -- 172.21.15.16:0/3106596097 learned_addr learned my addr 172.21.15.16:0/3106596097 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:36:03.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.731+0000 7f749cc82700 1 -- 172.21.15.16:0/3106596097 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f749809d6f0 msgr2=0x7f749809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:03.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.731+0000 7f749cc82700 1 --2- 172.21.15.16:0/3106596097 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f749809d6f0 0x7f749809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:03.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.731+0000 7f749cc82700 1 -- 172.21.15.16:0/3106596097 >> v1:172.21.15.16:6789/0 conn(0x7f749809e000 legacy=0x7f7498130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:03.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.731+0000 7f749cc82700 1 -- 172.21.15.16:0/3106596097 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f749809b2d0 con 0x7f749809e410 2023-09-30T22:36:03.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.731+0000 7f7496ffd700 1 -- 172.21.15.16:0/3106596097 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1310528105 0 0) 0x7f749809f130 con 0x7f749809e000 2023-09-30T22:36:03.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.731+0000 7f7497fff700 1 --2- 172.21.15.16:0/3106596097 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f749809d6f0 0x7f749809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:03.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.732+0000 7f749cc82700 1 --2- 172.21.15.16:0/3106596097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f749809e410 0x7f7498130d70 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f7490009d90 tx=0x7f749000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b51b5ec45b6ab943 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:03.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.732+0000 7f7496ffd700 1 -- 172.21.15.16:0/3106596097 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749000e070 con 0x7f749809e410 2023-09-30T22:36:03.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.732+0000 7f7496ffd700 1 -- 172.21.15.16:0/3106596097 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f749000bb20 con 0x7f749809e410 2023-09-30T22:36:03.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.732+0000 7f7496ffd700 1 -- 172.21.15.16:0/3106596097 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7490012430 con 0x7f749809e410 2023-09-30T22:36:03.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.732+0000 7f749e6e5700 1 -- 172.21.15.16:0/3106596097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f749809e410 msgr2=0x7f7498130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:03.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.732+0000 7f749e6e5700 1 --2- 172.21.15.16:0/3106596097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f749809e410 0x7f7498130d70 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f7490009d90 tx=0x7f749000b330 comp rx=0 tx=0).stop 2023-09-30T22:36:03.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.732+0000 7f749e6e5700 1 -- 172.21.15.16:0/3106596097 shutdown_connections 2023-09-30T22:36:03.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.732+0000 7f749e6e5700 1 --2- 172.21.15.16:0/3106596097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f749809e410 0x7f7498130d70 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:03.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.732+0000 7f749e6e5700 1 --2- 172.21.15.16:0/3106596097 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f749809d6f0 0x7f749809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:03.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.732+0000 7f749e6e5700 1 -- 172.21.15.16:0/3106596097 >> 172.21.15.16:0/3106596097 conn(0x7f7498066f50 msgr2=0x7f749809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:03.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.732+0000 7f749e6e5700 1 -- 172.21.15.16:0/3106596097 shutdown_connections 2023-09-30T22:36:03.735 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.733+0000 7f749e6e5700 1 -- 172.21.15.16:0/3106596097 wait complete. 2023-09-30T22:36:03.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.733+0000 7f749e6e5700 1 Processor -- start 2023-09-30T22:36:03.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.734+0000 7f749e6e5700 1 -- start start 2023-09-30T22:36:03.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.734+0000 7f749e6e5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f749809d6f0 0x7f7498122610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:03.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.734+0000 7f749e6e5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f749809e000 0x7f7498122b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:03.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.734+0000 7f749e6e5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f749809e410 0x7f7498123090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:03.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.734+0000 7f749e6e5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7498123ae0 con 0x7f749809e410 2023-09-30T22:36:03.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.734+0000 7f749e6e5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74981260e0 con 0x7f749809e000 2023-09-30T22:36:03.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.734+0000 7f749e6e5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f74981241d0 con 0x7f749809d6f0 2023-09-30T22:36:03.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.734+0000 7f749cc82700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f749809e410 0x7f7498123090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:03.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.734+0000 7f749cc82700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f749809e410 0x7f7498123090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39836/0 (socket says 172.21.15.16:39836) 2023-09-30T22:36:03.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.734+0000 7f749cc82700 1 -- 172.21.15.16:0/3251892495 learned_addr learned my addr 172.21.15.16:0/3251892495 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:03.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.734+0000 7f74977fe700 1 --2- 172.21.15.16:0/3251892495 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f749809e000 0x7f7498122b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:03.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.734+0000 7f7497fff700 1 --2- 172.21.15.16:0/3251892495 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f749809d6f0 0x7f7498122610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:03.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.735+0000 7f749cc82700 1 -- 172.21.15.16:0/3251892495 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f749809d6f0 msgr2=0x7f7498122610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:03.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.735+0000 7f749cc82700 1 --2- 172.21.15.16:0/3251892495 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f749809d6f0 0x7f7498122610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:03.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.735+0000 7f749cc82700 1 -- 172.21.15.16:0/3251892495 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f749809e000 msgr2=0x7f7498122b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:03.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.735+0000 7f749cc82700 1 --2- 172.21.15.16:0/3251892495 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f749809e000 0x7f7498122b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:03.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.735+0000 7f749cc82700 1 -- 172.21.15.16:0/3251892495 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74880094a0 con 0x7f749809e410 2023-09-30T22:36:03.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.735+0000 7f74977fe700 1 --2- 172.21.15.16:0/3251892495 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f749809e000 0x7f7498122b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:03.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.735+0000 7f749cc82700 1 --2- 172.21.15.16:0/3251892495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f749809e410 0x7f7498123090 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f7490014040 tx=0x7f7490004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:03.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.735+0000 7f7494ff9700 1 -- 172.21.15.16:0/3251892495 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7490012430 con 0x7f749809e410 2023-09-30T22:36:03.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.735+0000 7f7494ff9700 1 -- 172.21.15.16:0/3251892495 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7490020070 con 0x7f749809e410 2023-09-30T22:36:03.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.735+0000 7f749e6e5700 1 -- 172.21.15.16:0/3251892495 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7490009a70 con 0x7f749809e410 2023-09-30T22:36:03.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.735+0000 7f7494ff9700 1 -- 172.21.15.16:0/3251892495 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7490004280 con 0x7f749809e410 2023-09-30T22:36:03.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.735+0000 7f749e6e5700 1 -- 172.21.15.16:0/3251892495 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74981304e0 con 0x7f749809e410 2023-09-30T22:36:03.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.736+0000 7f7494ff9700 1 -- 172.21.15.16:0/3251892495 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f74900043e0 con 0x7f749809e410 2023-09-30T22:36:03.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.736+0000 7f7494ff9700 1 -- 172.21.15.16:0/3251892495 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f7490022070 con 0x7f749809e410 2023-09-30T22:36:03.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.736+0000 7f749e6e5700 1 -- 172.21.15.16:0/3251892495 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7498074ea0 con 0x7f749809e410 2023-09-30T22:36:03.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.741+0000 7f7494ff9700 1 -- 172.21.15.16:0/3251892495 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f749001b400 con 0x7f749809e410 2023-09-30T22:36:03.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.946+0000 7f749e6e5700 1 -- 172.21.15.16:0/3251892495 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7498125c60 con 0x7f749809e410 2023-09-30T22:36:03.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.946+0000 7f7494ff9700 1 -- 172.21.15.16:0/3251892495 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7498125c60 con 0x7f749809e410 2023-09-30T22:36:03.947 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:36:03.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.949+0000 7f749e6e5700 1 -- 172.21.15.16:0/3251892495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f749809e410 msgr2=0x7f7498123090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:03.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.949+0000 7f749e6e5700 1 --2- 172.21.15.16:0/3251892495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f749809e410 0x7f7498123090 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f7490014040 tx=0x7f7490004ad0 comp rx=0 tx=0).stop 2023-09-30T22:36:03.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.949+0000 7f749e6e5700 1 -- 172.21.15.16:0/3251892495 shutdown_connections 2023-09-30T22:36:03.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.949+0000 7f749e6e5700 1 --2- 172.21.15.16:0/3251892495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f749809e410 0x7f7498123090 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:03.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.949+0000 7f749e6e5700 1 --2- 172.21.15.16:0/3251892495 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f749809d6f0 0x7f7498122610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:03.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.949+0000 7f749e6e5700 1 --2- 172.21.15.16:0/3251892495 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f749809e000 0x7f7498122b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:03.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.949+0000 7f749e6e5700 1 -- 172.21.15.16:0/3251892495 >> 172.21.15.16:0/3251892495 conn(0x7f7498066f50 msgr2=0x7f749806a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:03.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.949+0000 7f749e6e5700 1 -- 172.21.15.16:0/3251892495 shutdown_connections 2023-09-30T22:36:03.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:03.949+0000 7f749e6e5700 1 -- 172.21.15.16:0/3251892495 wait complete. 2023-09-30T22:36:03.963 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:36:04.631 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:36:04.649 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:36:04.661 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:36:04.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.784+0000 7f9041858700 1 Processor -- start 2023-09-30T22:36:04.785 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.784+0000 7f9041858700 1 -- start start 2023-09-30T22:36:04.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.785+0000 7f9041858700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f903c0a0a30 0x7f903c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:04.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.785+0000 7f9041858700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f903c134170 0x7f903c13bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:04.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.785+0000 7f9041858700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f903c09edd0 con 0x7f903c0a1340 2023-09-30T22:36:04.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.785+0000 7f9041858700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f903c09e1f0 con 0x7f903c0a0a30 2023-09-30T22:36:04.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.785+0000 7f9041858700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f903c09dc50 con 0x7f903c134170 2023-09-30T22:36:04.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.785+0000 7f903affd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f903c0a0a30 0x7f903c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:04.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.785+0000 7f903affd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f903c0a0a30 0x7f903c0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39848/0 (socket says 172.21.15.16:39848) 2023-09-30T22:36:04.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.785+0000 7f903affd700 1 -- 172.21.15.16:0/3630152070 learned_addr learned my addr 172.21.15.16:0/3630152070 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:04.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.785+0000 7f903b7fe700 1 --2- 172.21.15.16:0/3630152070 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f903c134170 0x7f903c13bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:04.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.786+0000 7f903b7fe700 1 -- 172.21.15.16:0/3630152070 >> v2:172.21.15.16:3300/0 conn(0x7f903c0a0a30 msgr2=0x7f903c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:04.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.786+0000 7f903b7fe700 1 --2- 172.21.15.16:0/3630152070 >> v2:172.21.15.16:3300/0 conn(0x7f903c0a0a30 0x7f903c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:04.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.786+0000 7f903b7fe700 1 -- 172.21.15.16:0/3630152070 >> v1:172.21.15.16:6789/0 conn(0x7f903c0a1340 legacy=0x7f903c133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:04.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.786+0000 7f903b7fe700 1 -- 172.21.15.16:0/3630152070 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f903c0a1fe0 con 0x7f903c134170 2023-09-30T22:36:04.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.786+0000 7f903affd700 1 --2- 172.21.15.16:0/3630152070 >> v2:172.21.15.16:3300/0 conn(0x7f903c0a0a30 0x7f903c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:04.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.786+0000 7f903b7fe700 1 --2- 172.21.15.16:0/3630152070 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f903c134170 0x7f903c13bd30 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f9034009d90 tx=0x7f903400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=729a9d6e3f42cb3e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:04.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.786+0000 7f9039ffb700 1 -- 172.21.15.16:0/3630152070 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903400e070 con 0x7f903c134170 2023-09-30T22:36:04.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.786+0000 7f9039ffb700 1 -- 172.21.15.16:0/3630152070 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f903400bb20 con 0x7f903c134170 2023-09-30T22:36:04.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.786+0000 7f9039ffb700 1 -- 172.21.15.16:0/3630152070 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9034012430 con 0x7f903c134170 2023-09-30T22:36:04.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.787+0000 7f9041858700 1 -- 172.21.15.16:0/3630152070 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f903c134170 msgr2=0x7f903c13bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:04.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.787+0000 7f9041858700 1 --2- 172.21.15.16:0/3630152070 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f903c134170 0x7f903c13bd30 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f9034009d90 tx=0x7f903400b330 comp rx=0 tx=0).stop 2023-09-30T22:36:04.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.787+0000 7f9041858700 1 -- 172.21.15.16:0/3630152070 shutdown_connections 2023-09-30T22:36:04.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.787+0000 7f9041858700 1 --2- 172.21.15.16:0/3630152070 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f903c134170 0x7f903c13bd30 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:04.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.787+0000 7f9041858700 1 --2- 172.21.15.16:0/3630152070 >> v2:172.21.15.16:3300/0 conn(0x7f903c0a0a30 0x7f903c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:04.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.787+0000 7f9041858700 1 -- 172.21.15.16:0/3630152070 >> 172.21.15.16:0/3630152070 conn(0x7f903c0a6550 msgr2=0x7f903c0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:04.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.787+0000 7f9041858700 1 -- 172.21.15.16:0/3630152070 shutdown_connections 2023-09-30T22:36:04.792 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.787+0000 7f9041858700 1 -- 172.21.15.16:0/3630152070 wait complete. 2023-09-30T22:36:04.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.788+0000 7f9041858700 1 Processor -- start 2023-09-30T22:36:04.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.789+0000 7f9041858700 1 -- start start 2023-09-30T22:36:04.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.789+0000 7f9041858700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f903c0a0a30 0x7f903c11e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:04.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.789+0000 7f9041858700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f903c0a1340 0x7f903c11ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:04.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.789+0000 7f9041858700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f903c134170 0x7f903c11f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:04.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.789+0000 7f9041858700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f903c11fbe0 con 0x7f903c0a1340 2023-09-30T22:36:04.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.789+0000 7f9041858700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f903c1221a0 con 0x7f903c0a0a30 2023-09-30T22:36:04.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.789+0000 7f9041858700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f903c120290 con 0x7f903c134170 2023-09-30T22:36:04.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.789+0000 7f903a7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f903c0a1340 0x7f903c11ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:04.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.789+0000 7f903affd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f903c0a0a30 0x7f903c11e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:04.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.790+0000 7f903a7fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f903c0a1340 0x7f903c11ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39852/0 (socket says 172.21.15.16:39852) 2023-09-30T22:36:04.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.790+0000 7f903a7fc700 1 -- 172.21.15.16:0/3010885784 learned_addr learned my addr 172.21.15.16:0/3010885784 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:04.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.790+0000 7f903b7fe700 1 --2- 172.21.15.16:0/3010885784 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f903c134170 0x7f903c11f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:04.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.790+0000 7f903affd700 1 -- 172.21.15.16:0/3010885784 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f903c134170 msgr2=0x7f903c11f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:04.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.790+0000 7f903affd700 1 --2- 172.21.15.16:0/3010885784 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f903c134170 0x7f903c11f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:04.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.790+0000 7f903affd700 1 -- 172.21.15.16:0/3010885784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f903c0a1340 msgr2=0x7f903c11ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:04.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.790+0000 7f903affd700 1 --2- 172.21.15.16:0/3010885784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f903c0a1340 0x7f903c11ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:04.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.790+0000 7f903affd700 1 -- 172.21.15.16:0/3010885784 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f902c009550 con 0x7f903c0a0a30 2023-09-30T22:36:04.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.790+0000 7f903affd700 1 --2- 172.21.15.16:0/3010885784 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f903c0a0a30 0x7f903c11e710 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f902c00e9a0 tx=0x7f902c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:04.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.790+0000 7f9023fff700 1 -- 172.21.15.16:0/3010885784 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f902c004030 con 0x7f903c0a0a30 2023-09-30T22:36:04.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.790+0000 7f9023fff700 1 -- 172.21.15.16:0/3010885784 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f902c004190 con 0x7f903c0a0a30 2023-09-30T22:36:04.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.790+0000 7f9041858700 1 -- 172.21.15.16:0/3010885784 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9034009a70 con 0x7f903c0a0a30 2023-09-30T22:36:04.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.790+0000 7f9023fff700 1 -- 172.21.15.16:0/3010885784 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f902c004520 con 0x7f903c0a0a30 2023-09-30T22:36:04.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.791+0000 7f9041858700 1 -- 172.21.15.16:0/3010885784 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f903c0a25e0 con 0x7f903c0a0a30 2023-09-30T22:36:04.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.792+0000 7f9023fff700 1 -- 172.21.15.16:0/3010885784 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f902c0036a0 con 0x7f903c0a0a30 2023-09-30T22:36:04.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.792+0000 7f9041858700 1 -- 172.21.15.16:0/3010885784 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f903c074ea0 con 0x7f903c0a0a30 2023-09-30T22:36:04.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.792+0000 7f9023fff700 1 -- 172.21.15.16:0/3010885784 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f902c03f920 con 0x7f903c0a0a30 2023-09-30T22:36:04.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.797+0000 7f9023fff700 1 -- 172.21.15.16:0/3010885784 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f902c011d90 con 0x7f903c0a0a30 2023-09-30T22:36:04.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.822+0000 7fc1dfc84700 1 Processor -- start 2023-09-30T22:36:04.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.822+0000 7f61045b3700 1 Processor -- start 2023-09-30T22:36:04.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.822+0000 7f61045b3700 1 -- start start 2023-09-30T22:36:04.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.822+0000 7fc1dfc84700 1 -- start start 2023-09-30T22:36:04.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.823+0000 7fc1dfc84700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc1d809d6f0 0x7fc1d809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:04.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.823+0000 7fc1dfc84700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc1d809e000 0x7fc1d8130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:04.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.823+0000 7fc1dfc84700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1d809f130 con 0x7fc1d809e410 2023-09-30T22:36:04.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.823+0000 7fc1dfc84700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d809f520 con 0x7fc1d809e000 2023-09-30T22:36:04.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.823+0000 7fc1dfc84700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d809e930 con 0x7fc1d809d6f0 2023-09-30T22:36:04.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.823+0000 7f61045b3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f60fc09e1d0 0x7f60fc138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:04.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.823+0000 7f61045b3700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f60fc139170 0x7f60fc13b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:04.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.823+0000 7f61045b3700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60fc09b120 con 0x7f60fc09d6f0 2023-09-30T22:36:04.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.823+0000 7f61045b3700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60fc06b460 con 0x7f60fc139170 2023-09-30T22:36:04.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.823+0000 7f61045b3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60fc09a7e0 con 0x7f60fc09e1d0 2023-09-30T22:36:04.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.823+0000 7f6102b50700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f60fc139170 0x7f60fc13b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:04.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.823+0000 7f6102b50700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f60fc139170 0x7f60fc13b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39868/0 (socket says 172.21.15.16:39868) 2023-09-30T22:36:04.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.823+0000 7f6102b50700 1 -- 172.21.15.16:0/4236093115 learned_addr learned my addr 172.21.15.16:0/4236093115 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:04.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.823+0000 7f6101b4e700 1 --2- 172.21.15.16:0/4236093115 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f60fc09e1d0 0x7f60fc138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:04.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.823+0000 7fc1de221700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc1d809e410 0x7fc1d8130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38336/0 (socket says 172.21.15.16:38336) 2023-09-30T22:36:04.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.823+0000 7fc1de221700 1 -- 172.21.15.16:0/1234854000 learned_addr learned my addr 172.21.15.16:0/1234854000 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:36:04.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.824+0000 7f6101b4e700 1 -- 172.21.15.16:0/4236093115 >> v2:172.21.15.16:3300/0 conn(0x7f60fc139170 msgr2=0x7f60fc13b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:04.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.824+0000 7f6101b4e700 1 --2- 172.21.15.16:0/4236093115 >> v2:172.21.15.16:3300/0 conn(0x7f60fc139170 0x7f60fc13b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:04.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.824+0000 7f6101b4e700 1 -- 172.21.15.16:0/4236093115 >> v1:172.21.15.16:6789/0 conn(0x7f60fc09d6f0 legacy=0x7f60fc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:04.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.824+0000 7f6101b4e700 1 -- 172.21.15.16:0/4236093115 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60fc09a500 con 0x7f60fc09e1d0 2023-09-30T22:36:04.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.824+0000 7fc1dd21f700 1 --2- 172.21.15.16:0/1234854000 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc1d809e000 0x7fc1d8130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:04.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.824+0000 7fc1dda20700 1 --2- 172.21.15.16:0/1234854000 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc1d809d6f0 0x7fc1d809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:04.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.824+0000 7fc1dca1e700 1 -- 172.21.15.16:0/1234854000 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1937943022 0 0) 0x7fc1d809f130 con 0x7fc1d809e410 2023-09-30T22:36:04.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.824+0000 7fc1dca1e700 1 -- 172.21.15.16:0/1234854000 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1c40035b0 con 0x7fc1d809e410 2023-09-30T22:36:04.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.824+0000 7fc1dd21f700 1 -- 172.21.15.16:0/1234854000 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc1d809d6f0 msgr2=0x7fc1d809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:04.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.824+0000 7fc1dd21f700 1 --2- 172.21.15.16:0/1234854000 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc1d809d6f0 0x7fc1d809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:04.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.824+0000 7fc1dd21f700 1 -- 172.21.15.16:0/1234854000 >> v1:172.21.15.16:6789/0 conn(0x7fc1d809e410 legacy=0x7fc1d8130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:04.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.824+0000 7fc1dd21f700 1 -- 172.21.15.16:0/1234854000 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1d809b2d0 con 0x7fc1d809e000 2023-09-30T22:36:04.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.824+0000 7f6101b4e700 1 --2- 172.21.15.16:0/4236093115 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f60fc09e1d0 0x7f60fc138c30 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f60ec009870 tx=0x7f60ec009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7940a63b21c24452 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:04.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.824+0000 7f610134d700 1 -- 172.21.15.16:0/4236093115 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ec011830 con 0x7f60fc09e1d0 2023-09-30T22:36:04.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.824+0000 7fc1dd21f700 1 --2- 172.21.15.16:0/1234854000 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc1d809e000 0x7fc1d8130660 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8009870 tx=0x7fc1c8009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=130567f443c27f57 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:04.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.824+0000 7f610134d700 1 -- 172.21.15.16:0/4236093115 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60ec011990 con 0x7f60fc09e1d0 2023-09-30T22:36:04.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.824+0000 7f610134d700 1 -- 172.21.15.16:0/4236093115 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ec011d20 con 0x7f60fc09e1d0 2023-09-30T22:36:04.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.825+0000 7fc1dca1e700 1 -- 172.21.15.16:0/1234854000 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c800d980 con 0x7fc1d809e000 2023-09-30T22:36:04.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.825+0000 7fc1dca1e700 1 -- 172.21.15.16:0/1234854000 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1c800dae0 con 0x7fc1d809e000 2023-09-30T22:36:04.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.825+0000 7fc1dca1e700 1 -- 172.21.15.16:0/1234854000 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c800de70 con 0x7fc1d809e000 2023-09-30T22:36:04.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.825+0000 7f61045b3700 1 -- 172.21.15.16:0/4236093115 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f60fc09e1d0 msgr2=0x7f60fc138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:04.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.825+0000 7f61045b3700 1 --2- 172.21.15.16:0/4236093115 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f60fc09e1d0 0x7f60fc138c30 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f60ec009870 tx=0x7f60ec009b80 comp rx=0 tx=0).stop 2023-09-30T22:36:04.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.825+0000 7f61045b3700 1 -- 172.21.15.16:0/4236093115 shutdown_connections 2023-09-30T22:36:04.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.825+0000 7f61045b3700 1 --2- 172.21.15.16:0/4236093115 >> v2:172.21.15.16:3300/0 conn(0x7f60fc139170 0x7f60fc13b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:04.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.825+0000 7f61045b3700 1 --2- 172.21.15.16:0/4236093115 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f60fc09e1d0 0x7f60fc138c30 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:04.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.825+0000 7f61045b3700 1 -- 172.21.15.16:0/4236093115 >> 172.21.15.16:0/4236093115 conn(0x7f60fc066f50 msgr2=0x7f60fc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:04.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.825+0000 7fc1dfc84700 1 -- 172.21.15.16:0/1234854000 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc1d809e000 msgr2=0x7fc1d8130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:04.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.825+0000 7fc1dfc84700 1 --2- 172.21.15.16:0/1234854000 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc1d809e000 0x7fc1d8130660 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8009870 tx=0x7fc1c8009b80 comp rx=0 tx=0).stop 2023-09-30T22:36:04.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.825+0000 7f61045b3700 1 -- 172.21.15.16:0/4236093115 shutdown_connections 2023-09-30T22:36:04.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.825+0000 7f61045b3700 1 -- 172.21.15.16:0/4236093115 wait complete. 2023-09-30T22:36:04.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.825+0000 7fc1dfc84700 1 -- 172.21.15.16:0/1234854000 shutdown_connections 2023-09-30T22:36:04.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.825+0000 7fc1dfc84700 1 --2- 172.21.15.16:0/1234854000 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc1d809d6f0 0x7fc1d809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:04.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.825+0000 7fc1dfc84700 1 --2- 172.21.15.16:0/1234854000 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc1d809e000 0x7fc1d8130660 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:04.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.825+0000 7fc1dfc84700 1 -- 172.21.15.16:0/1234854000 >> 172.21.15.16:0/1234854000 conn(0x7fc1d8066f50 msgr2=0x7fc1d809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:04.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.826+0000 7fc1dfc84700 1 -- 172.21.15.16:0/1234854000 shutdown_connections 2023-09-30T22:36:04.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.826+0000 7fc1dfc84700 1 -- 172.21.15.16:0/1234854000 wait complete. 2023-09-30T22:36:04.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.826+0000 7f61045b3700 1 Processor -- start 2023-09-30T22:36:04.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.826+0000 7f61045b3700 1 -- start start 2023-09-30T22:36:04.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.827+0000 7fc1dfc84700 1 Processor -- start 2023-09-30T22:36:04.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.827+0000 7fc1dfc84700 1 -- start start 2023-09-30T22:36:04.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.827+0000 7f61045b3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f60fc09d6f0 0x7f60fc11e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:04.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.827+0000 7f61045b3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60fc09e1d0 0x7f60fc11ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:04.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.827+0000 7f61045b3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f60fc139170 0x7f60fc11f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:04.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.827+0000 7f61045b3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60fc09c710 con 0x7f60fc09e1d0 2023-09-30T22:36:04.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.827+0000 7f61045b3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60fc0a2a30 con 0x7f60fc139170 2023-09-30T22:36:04.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.827+0000 7f61045b3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f60fc1204e0 con 0x7f60fc09d6f0 2023-09-30T22:36:04.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.827+0000 7f6102b50700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f60fc139170 0x7f60fc11f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:04.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.827+0000 7f6102b50700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f60fc139170 0x7f60fc11f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47640/0 (socket says 172.21.15.16:47640) 2023-09-30T22:36:04.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.827+0000 7f6102b50700 1 -- 172.21.15.16:0/1223829007 learned_addr learned my addr 172.21.15.16:0/1223829007 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:04.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.827+0000 7fc1dfc84700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc1d809e000 0x7fc1d8122460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:04.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.827+0000 7fc1dfc84700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc1d809e410 0x7fc1d81229a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:04.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.827+0000 7fc1dfc84700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1d8122ee0 0x7fc1d81c8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:04.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.827+0000 7fc1dfc84700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d8124660 con 0x7fc1d8122ee0 2023-09-30T22:36:04.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.827+0000 7fc1dfc84700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d8125f40 con 0x7fc1d809e000 2023-09-30T22:36:04.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.827+0000 7fc1dfc84700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d80a1d00 con 0x7fc1d809e410 2023-09-30T22:36:04.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.827+0000 7f6102b50700 1 -- 172.21.15.16:0/1223829007 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f60fc09d6f0 msgr2=0x7f60fc11e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:04.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.827+0000 7f6102b50700 1 --2- 172.21.15.16:0/1223829007 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f60fc09d6f0 0x7f60fc11e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:04.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.827+0000 7f6102b50700 1 -- 172.21.15.16:0/1223829007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60fc09e1d0 msgr2=0x7f60fc11ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:04.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.828+0000 7f6102b50700 1 --2- 172.21.15.16:0/1223829007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60fc09e1d0 0x7f60fc11ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:04.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.828+0000 7f6102b50700 1 -- 172.21.15.16:0/1223829007 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60ec0094a0 con 0x7f60fc139170 2023-09-30T22:36:04.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.827+0000 7fc1dda20700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc1d809e000 0x7fc1d8122460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:04.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.828+0000 7fc1dda20700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc1d809e000 0x7fc1d8122460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47646/0 (socket says 172.21.15.16:47646) 2023-09-30T22:36:04.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.828+0000 7fc1dda20700 1 -- 172.21.15.16:0/2196048816 learned_addr learned my addr 172.21.15.16:0/2196048816 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:04.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.828+0000 7fc1dd21f700 1 --2- 172.21.15.16:0/2196048816 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc1d809e410 0x7fc1d81229a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:04.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.828+0000 7f6102b50700 1 --2- 172.21.15.16:0/1223829007 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f60fc139170 0x7f60fc11f2b0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f60f400efc0 tx=0x7f60f400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:04.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.828+0000 7fc1dda20700 1 -- 172.21.15.16:0/2196048816 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc1d809e410 msgr2=0x7fc1d81229a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:04.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.828+0000 7fc1dda20700 1 --2- 172.21.15.16:0/2196048816 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc1d809e410 0x7fc1d81229a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:04.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.828+0000 7fc1dda20700 1 -- 172.21.15.16:0/2196048816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1d8122ee0 msgr2=0x7fc1d81c8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:04.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.828+0000 7fc1dda20700 1 --2- 172.21.15.16:0/2196048816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1d8122ee0 0x7fc1d81c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:04.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.828+0000 7fc1dda20700 1 -- 172.21.15.16:0/2196048816 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1c80094a0 con 0x7fc1d809e000 2023-09-30T22:36:04.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.828+0000 7f60f2ffd700 1 -- 172.21.15.16:0/1223829007 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60f4003c20 con 0x7f60fc139170 2023-09-30T22:36:04.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.828+0000 7f60f2ffd700 1 -- 172.21.15.16:0/1223829007 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60f400f040 con 0x7f60fc139170 2023-09-30T22:36:04.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.828+0000 7f60f2ffd700 1 -- 172.21.15.16:0/1223829007 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60f400c930 con 0x7f60fc139170 2023-09-30T22:36:04.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.828+0000 7f61045b3700 1 -- 172.21.15.16:0/1223829007 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60fc121ae0 con 0x7f60fc139170 2023-09-30T22:36:04.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.828+0000 7f61045b3700 1 -- 172.21.15.16:0/1223829007 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60fc120be0 con 0x7f60fc139170 2023-09-30T22:36:04.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.828+0000 7fc1dda20700 1 --2- 172.21.15.16:0/2196048816 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc1d809e000 0x7fc1d8122460 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fc1d400e790 tx=0x7fc1d400eaa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:04.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.829+0000 7fc1ce7fc700 1 -- 172.21.15.16:0/2196048816 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d400c070 con 0x7fc1d809e000 2023-09-30T22:36:04.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.829+0000 7fc1ce7fc700 1 -- 172.21.15.16:0/2196048816 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1d4013630 con 0x7fc1d809e000 2023-09-30T22:36:04.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.829+0000 7fc1dfc84700 1 -- 172.21.15.16:0/2196048816 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1d8124030 con 0x7fc1d809e000 2023-09-30T22:36:04.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.829+0000 7fc1ce7fc700 1 -- 172.21.15.16:0/2196048816 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d40139e0 con 0x7fc1d809e000 2023-09-30T22:36:04.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.829+0000 7fc1dfc84700 1 -- 172.21.15.16:0/2196048816 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1d8130340 con 0x7fc1d809e000 2023-09-30T22:36:04.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.829+0000 7f60f2ffd700 1 -- 172.21.15.16:0/1223829007 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f60f400ca90 con 0x7f60fc139170 2023-09-30T22:36:04.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.829+0000 7f60f2ffd700 1 -- 172.21.15.16:0/1223829007 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f60f403fb10 con 0x7f60fc139170 2023-09-30T22:36:04.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.830+0000 7f61045b3700 1 -- 172.21.15.16:0/1223829007 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60fc121eb0 con 0x7f60fc139170 2023-09-30T22:36:04.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.830+0000 7fc1dfc84700 1 -- 172.21.15.16:0/2196048816 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1d81258e0 con 0x7fc1d809e000 2023-09-30T22:36:04.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.830+0000 7fc1ce7fc700 1 -- 172.21.15.16:0/2196048816 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc1d4013b40 con 0x7fc1d809e000 2023-09-30T22:36:04.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.831+0000 7fc1ce7fc700 1 -- 172.21.15.16:0/2196048816 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fc1d403f7b0 con 0x7fc1d809e000 2023-09-30T22:36:04.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.835+0000 7f60f2ffd700 1 -- 172.21.15.16:0/1223829007 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f60f4014440 con 0x7f60fc139170 2023-09-30T22:36:04.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:04.840+0000 7fc1ce7fc700 1 -- 172.21.15.16:0/2196048816 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc1d4012960 con 0x7fc1d809e000 2023-09-30T22:36:04.963 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:36:05.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.010+0000 7f9041858700 1 -- 172.21.15.16:0/3010885784 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f903c0a28e0 con 0x7f903c0a0a30 2023-09-30T22:36:05.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.010+0000 7f9023fff700 1 -- 172.21.15.16:0/3010885784 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f902c011350 con 0x7f903c0a0a30 2023-09-30T22:36:05.011 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:36:05.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.013+0000 7f9041858700 1 -- 172.21.15.16:0/3010885784 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f903c0a0a30 msgr2=0x7f903c11e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:05.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.013+0000 7f9041858700 1 --2- 172.21.15.16:0/3010885784 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f903c0a0a30 0x7f903c11e710 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f902c00e9a0 tx=0x7f902c00ecb0 comp rx=0 tx=0).stop 2023-09-30T22:36:05.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.013+0000 7f9041858700 1 -- 172.21.15.16:0/3010885784 shutdown_connections 2023-09-30T22:36:05.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.013+0000 7f9041858700 1 --2- 172.21.15.16:0/3010885784 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f903c0a0a30 0x7f903c11e710 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:05.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.013+0000 7f9041858700 1 --2- 172.21.15.16:0/3010885784 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f903c134170 0x7f903c11f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:05.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.013+0000 7f9041858700 1 --2- 172.21.15.16:0/3010885784 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f903c0a1340 0x7f903c11ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:05.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.013+0000 7f9041858700 1 -- 172.21.15.16:0/3010885784 >> 172.21.15.16:0/3010885784 conn(0x7f903c0a6550 msgr2=0x7f903c09ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:05.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.013+0000 7f9041858700 1 -- 172.21.15.16:0/3010885784 shutdown_connections 2023-09-30T22:36:05.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.013+0000 7f9041858700 1 -- 172.21.15.16:0/3010885784 wait complete. 2023-09-30T22:36:05.029 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:36:05.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.043+0000 7f61045b3700 1 -- 172.21.15.16:0/1223829007 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f60fc09f420 con 0x7f60fc139170 2023-09-30T22:36:05.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.044+0000 7f60f2ffd700 1 -- 172.21.15.16:0/1223829007 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f60f40133e0 con 0x7f60fc139170 2023-09-30T22:36:05.044 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:36:05.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.048+0000 7f61045b3700 1 -- 172.21.15.16:0/1223829007 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f60fc139170 msgr2=0x7f60fc11f2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:05.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.048+0000 7f61045b3700 1 --2- 172.21.15.16:0/1223829007 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f60fc139170 0x7f60fc11f2b0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f60f400efc0 tx=0x7f60f400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:36:05.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.048+0000 7f61045b3700 1 -- 172.21.15.16:0/1223829007 shutdown_connections 2023-09-30T22:36:05.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.048+0000 7f61045b3700 1 --2- 172.21.15.16:0/1223829007 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f60fc139170 0x7f60fc11f2b0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:05.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.048+0000 7f61045b3700 1 --2- 172.21.15.16:0/1223829007 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f60fc09d6f0 0x7f60fc11e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:05.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.048+0000 7f61045b3700 1 --2- 172.21.15.16:0/1223829007 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f60fc09e1d0 0x7f60fc11ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:05.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.048+0000 7f61045b3700 1 -- 172.21.15.16:0/1223829007 >> 172.21.15.16:0/1223829007 conn(0x7f60fc066f50 msgr2=0x7f60fc0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:05.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.048+0000 7f61045b3700 1 -- 172.21.15.16:0/1223829007 shutdown_connections 2023-09-30T22:36:05.049 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.048+0000 7f61045b3700 1 -- 172.21.15.16:0/1223829007 wait complete. 2023-09-30T22:36:05.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.053+0000 7fc1dfc84700 1 -- 172.21.15.16:0/2196048816 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc1d81258e0 con 0x7fc1d809e000 2023-09-30T22:36:05.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.054+0000 7fc1ce7fc700 1 -- 172.21.15.16:0/2196048816 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc1d4012370 con 0x7fc1d809e000 2023-09-30T22:36:05.054 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:36:05.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.056+0000 7fc1dfc84700 1 -- 172.21.15.16:0/2196048816 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc1d809e000 msgr2=0x7fc1d8122460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:05.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.056+0000 7fc1dfc84700 1 --2- 172.21.15.16:0/2196048816 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc1d809e000 0x7fc1d8122460 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fc1d400e790 tx=0x7fc1d400eaa0 comp rx=0 tx=0).stop 2023-09-30T22:36:05.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.056+0000 7fc1dfc84700 1 -- 172.21.15.16:0/2196048816 shutdown_connections 2023-09-30T22:36:05.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.056+0000 7fc1dfc84700 1 --2- 172.21.15.16:0/2196048816 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc1d809e000 0x7fc1d8122460 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:05.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.056+0000 7fc1dfc84700 1 --2- 172.21.15.16:0/2196048816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1d8122ee0 0x7fc1d81c8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:05.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.056+0000 7fc1dfc84700 1 --2- 172.21.15.16:0/2196048816 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc1d809e410 0x7fc1d81229a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:05.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.056+0000 7fc1dfc84700 1 -- 172.21.15.16:0/2196048816 >> 172.21.15.16:0/2196048816 conn(0x7fc1d8066f50 msgr2=0x7fc1d80a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:05.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.056+0000 7fc1dfc84700 1 -- 172.21.15.16:0/2196048816 shutdown_connections 2023-09-30T22:36:05.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.056+0000 7fc1dfc84700 1 -- 172.21.15.16:0/2196048816 wait complete. 2023-09-30T22:36:05.069 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:36:05.071 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:36:05.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.111+0000 7f882e8ba700 1 Processor -- start 2023-09-30T22:36:05.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.111+0000 7f882e8ba700 1 -- start start 2023-09-30T22:36:05.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.112+0000 7f882e8ba700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f882809e930 0x7f882809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:05.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.112+0000 7f882e8ba700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f882809f240 0x7f88281317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:05.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.112+0000 7f882e8ba700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8828070940 con 0x7f882809f650 2023-09-30T22:36:05.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.112+0000 7f882e8ba700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f882806fd60 con 0x7f882809f240 2023-09-30T22:36:05.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.112+0000 7f882e8ba700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f882806fa80 con 0x7f882809e930 2023-09-30T22:36:05.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.112+0000 7f88277fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f882809f240 0x7f88281317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:05.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.112+0000 7f8827fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f882809e930 0x7f882809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:05.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.112+0000 7f882ce57700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f882809f650 0x7f8828131d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38362/0 (socket says 172.21.15.16:38362) 2023-09-30T22:36:05.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.112+0000 7f882ce57700 1 -- 172.21.15.16:0/2427319861 learned_addr learned my addr 172.21.15.16:0/2427319861 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:36:05.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.112+0000 7f88277fe700 1 -- 172.21.15.16:0/2427319861 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f882809e930 msgr2=0x7f882809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:05.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.112+0000 7f88277fe700 1 --2- 172.21.15.16:0/2427319861 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f882809e930 0x7f882809ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:05.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.112+0000 7f88277fe700 1 -- 172.21.15.16:0/2427319861 >> v1:172.21.15.16:6789/0 conn(0x7f882809f650 legacy=0x7f8828131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:05.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.113+0000 7f88277fe700 1 -- 172.21.15.16:0/2427319861 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f882809fb60 con 0x7f882809f240 2023-09-30T22:36:05.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.113+0000 7f8826ffd700 1 -- 172.21.15.16:0/2427319861 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2994026238 0 0) 0x7f8828070940 con 0x7f882809f650 2023-09-30T22:36:05.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.113+0000 7f8827fff700 1 --2- 172.21.15.16:0/2427319861 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f882809e930 0x7f882809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:05.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.113+0000 7f88277fe700 1 --2- 172.21.15.16:0/2427319861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f882809f240 0x7f88281317f0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f881c009840 tx=0x7f881c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd82e63803a8a7e4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:05.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.113+0000 7f8826ffd700 1 -- 172.21.15.16:0/2427319861 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881c011830 con 0x7f882809f240 2023-09-30T22:36:05.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.113+0000 7f8826ffd700 1 -- 172.21.15.16:0/2427319861 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f881c011990 con 0x7f882809f240 2023-09-30T22:36:05.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.113+0000 7f8826ffd700 1 -- 172.21.15.16:0/2427319861 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881c011d20 con 0x7f882809f240 2023-09-30T22:36:05.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.113+0000 7f882e8ba700 1 -- 172.21.15.16:0/2427319861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f882809f240 msgr2=0x7f88281317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:05.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.113+0000 7f882e8ba700 1 --2- 172.21.15.16:0/2427319861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f882809f240 0x7f88281317f0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f881c009840 tx=0x7f881c009b50 comp rx=0 tx=0).stop 2023-09-30T22:36:05.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.114+0000 7f882e8ba700 1 -- 172.21.15.16:0/2427319861 shutdown_connections 2023-09-30T22:36:05.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.114+0000 7f882e8ba700 1 --2- 172.21.15.16:0/2427319861 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f882809e930 0x7f882809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:05.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.114+0000 7f882e8ba700 1 --2- 172.21.15.16:0/2427319861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f882809f240 0x7f88281317f0 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:05.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.114+0000 7f882e8ba700 1 -- 172.21.15.16:0/2427319861 >> 172.21.15.16:0/2427319861 conn(0x7f88280a4210 msgr2=0x7f88280a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:05.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.114+0000 7f882e8ba700 1 -- 172.21.15.16:0/2427319861 shutdown_connections 2023-09-30T22:36:05.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.114+0000 7f882e8ba700 1 -- 172.21.15.16:0/2427319861 wait complete. 2023-09-30T22:36:05.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.115+0000 7f882e8ba700 1 Processor -- start 2023-09-30T22:36:05.119 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.115+0000 7f882e8ba700 1 -- start start 2023-09-30T22:36:05.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.115+0000 7f882e8ba700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f882809e930 0x7f88281a3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:05.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.115+0000 7f882e8ba700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f882809f240 0x7f88281a4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:05.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.115+0000 7f882e8ba700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f882809f650 0x7f88281a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:05.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.115+0000 7f882e8ba700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88281a5f20 con 0x7f882809f650 2023-09-30T22:36:05.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.115+0000 7f882e8ba700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88281a6be0 con 0x7f882809e930 2023-09-30T22:36:05.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.115+0000 7f882e8ba700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f88280a32f0 con 0x7f882809f240 2023-09-30T22:36:05.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.115+0000 7f8827fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f882809e930 0x7f88281a3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:05.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.115+0000 7f8827fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f882809e930 0x7f88281a3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47664/0 (socket says 172.21.15.16:47664) 2023-09-30T22:36:05.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.115+0000 7f8827fff700 1 -- 172.21.15.16:0/2330567470 learned_addr learned my addr 172.21.15.16:0/2330567470 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:05.120 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.115+0000 7f88277fe700 1 --2- 172.21.15.16:0/2330567470 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f882809f240 0x7f88281a4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:05.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.115+0000 7f882ce57700 1 --2- 172.21.15.16:0/2330567470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f882809f650 0x7f88281a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:05.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.116+0000 7f8827fff700 1 -- 172.21.15.16:0/2330567470 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f882809f240 msgr2=0x7f88281a4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:05.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.116+0000 7f8827fff700 1 --2- 172.21.15.16:0/2330567470 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f882809f240 0x7f88281a4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:05.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.116+0000 7f8827fff700 1 -- 172.21.15.16:0/2330567470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f882809f650 msgr2=0x7f88281a4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:05.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.116+0000 7f8827fff700 1 --2- 172.21.15.16:0/2330567470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f882809f650 0x7f88281a4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:05.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.116+0000 7f8827fff700 1 -- 172.21.15.16:0/2330567470 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88180094a0 con 0x7f882809e930 2023-09-30T22:36:05.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.116+0000 7f88277fe700 1 --2- 172.21.15.16:0/2330567470 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f882809f240 0x7f88281a4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:05.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.116+0000 7f882ce57700 1 --2- 172.21.15.16:0/2330567470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f882809f650 0x7f88281a4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:05.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.116+0000 7f8827fff700 1 --2- 172.21.15.16:0/2330567470 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f882809e930 0x7f88281a3cf0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f881800ebe0 tx=0x7f881800eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:05.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.116+0000 7f8824ff9700 1 -- 172.21.15.16:0/2330567470 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881800c850 con 0x7f882809e930 2023-09-30T22:36:05.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.116+0000 7f8824ff9700 1 -- 172.21.15.16:0/2330567470 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f881800c9b0 con 0x7f882809e930 2023-09-30T22:36:05.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.116+0000 7f882e8ba700 1 -- 172.21.15.16:0/2330567470 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f881c0094a0 con 0x7f882809e930 2023-09-30T22:36:05.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.116+0000 7f8824ff9700 1 -- 172.21.15.16:0/2330567470 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881800cd40 con 0x7f882809e930 2023-09-30T22:36:05.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.116+0000 7f882e8ba700 1 -- 172.21.15.16:0/2330567470 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88280a0190 con 0x7f882809e930 2023-09-30T22:36:05.122 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.117+0000 7f8824ff9700 1 -- 172.21.15.16:0/2330567470 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8818013470 con 0x7f882809e930 2023-09-30T22:36:05.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.117+0000 7f8824ff9700 1 -- 172.21.15.16:0/2330567470 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f881803fb50 con 0x7f882809e930 2023-09-30T22:36:05.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.117+0000 7f882e8ba700 1 -- 172.21.15.16:0/2330567470 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88281a0b30 con 0x7f882809e930 2023-09-30T22:36:05.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.122+0000 7f8824ff9700 1 -- 172.21.15.16:0/2330567470 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8818013720 con 0x7f882809e930 2023-09-30T22:36:05.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.328+0000 7f882e8ba700 1 -- 172.21.15.16:0/2330567470 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8828074ea0 con 0x7f882809e930 2023-09-30T22:36:05.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.328+0000 7f8824ff9700 1 -- 172.21.15.16:0/2330567470 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8818021030 con 0x7f882809e930 2023-09-30T22:36:05.329 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:36:05.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.331+0000 7f882e8ba700 1 -- 172.21.15.16:0/2330567470 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f882809e930 msgr2=0x7f88281a3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:05.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.331+0000 7f882e8ba700 1 --2- 172.21.15.16:0/2330567470 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f882809e930 0x7f88281a3cf0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f881800ebe0 tx=0x7f881800eef0 comp rx=0 tx=0).stop 2023-09-30T22:36:05.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.331+0000 7f882e8ba700 1 -- 172.21.15.16:0/2330567470 shutdown_connections 2023-09-30T22:36:05.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.331+0000 7f882e8ba700 1 --2- 172.21.15.16:0/2330567470 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f882809e930 0x7f88281a3cf0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:05.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.331+0000 7f882e8ba700 1 --2- 172.21.15.16:0/2330567470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f882809f650 0x7f88281a4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:05.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.331+0000 7f882e8ba700 1 --2- 172.21.15.16:0/2330567470 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f882809f240 0x7f88281a4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:05.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.331+0000 7f882e8ba700 1 -- 172.21.15.16:0/2330567470 >> 172.21.15.16:0/2330567470 conn(0x7f88280a4210 msgr2=0x7f88281b8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:05.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.331+0000 7f882e8ba700 1 -- 172.21.15.16:0/2330567470 shutdown_connections 2023-09-30T22:36:05.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:05.331+0000 7f882e8ba700 1 -- 172.21.15.16:0/2330567470 wait complete. 2023-09-30T22:36:05.345 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:36:06.030 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:36:06.070 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:36:06.071 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:36:06.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.180+0000 7f8eed022700 1 Processor -- start 2023-09-30T22:36:06.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.180+0000 7f8eed022700 1 -- start start 2023-09-30T22:36:06.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.180+0000 7f8eed022700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8ee806a770 0x7f8ee806ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:06.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.181+0000 7f8eed022700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f8ee806b080 0x7f8ee8131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:06.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.181+0000 7f8eed022700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ee809ece0 con 0x7f8ee8131b40 2023-09-30T22:36:06.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.181+0000 7f8eed022700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ee809e760 con 0x7f8ee806b080 2023-09-30T22:36:06.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.181+0000 7f8eed022700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee809e040 con 0x7f8ee806a770 2023-09-30T22:36:06.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.181+0000 7f8ee6d9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8ee806a770 0x7f8ee806ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:06.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.181+0000 7f8ee6d9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8ee806a770 0x7f8ee806ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47678/0 (socket says 172.21.15.16:47678) 2023-09-30T22:36:06.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.181+0000 7f8ee6d9d700 1 -- 172.21.15.16:0/1030093045 learned_addr learned my addr 172.21.15.16:0/1030093045 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:06.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.181+0000 7f8ee659c700 1 --2- 172.21.15.16:0/1030093045 >> v2:172.21.15.16:3300/0 conn(0x7f8ee806b080 0x7f8ee8131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:06.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.181+0000 7f8ee6d9d700 1 -- 172.21.15.16:0/1030093045 >> v2:172.21.15.16:3300/0 conn(0x7f8ee806b080 msgr2=0x7f8ee8131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:06.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.181+0000 7f8ee6d9d700 1 --2- 172.21.15.16:0/1030093045 >> v2:172.21.15.16:3300/0 conn(0x7f8ee806b080 0x7f8ee8131600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:06.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.181+0000 7f8ee6d9d700 1 -- 172.21.15.16:0/1030093045 >> v1:172.21.15.16:6789/0 conn(0x7f8ee8131b40 legacy=0x7f8ee8138850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:06.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.181+0000 7f8ee6d9d700 1 -- 172.21.15.16:0/1030093045 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ee809f630 con 0x7f8ee806a770 2023-09-30T22:36:06.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.181+0000 7f8ee5d9b700 1 -- 172.21.15.16:0/1030093045 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3118330321 0 0) 0x7f8ee809ece0 con 0x7f8ee8131b40 2023-09-30T22:36:06.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.181+0000 7f8ee659c700 1 --2- 172.21.15.16:0/1030093045 >> v2:172.21.15.16:3300/0 conn(0x7f8ee806b080 0x7f8ee8131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:06.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.181+0000 7f8ee6d9d700 1 --2- 172.21.15.16:0/1030093045 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8ee806a770 0x7f8ee806ab40 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f8ed80098f0 tx=0x7f8ed8009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29e3ab6b9c8a097 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:06.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.182+0000 7f8ee5d9b700 1 -- 172.21.15.16:0/1030093045 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed8011830 con 0x7f8ee806a770 2023-09-30T22:36:06.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.182+0000 7f8ee5d9b700 1 -- 172.21.15.16:0/1030093045 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ed8011990 con 0x7f8ee806a770 2023-09-30T22:36:06.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.182+0000 7f8ee5d9b700 1 -- 172.21.15.16:0/1030093045 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed8011d20 con 0x7f8ee806a770 2023-09-30T22:36:06.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.182+0000 7f8eed022700 1 -- 172.21.15.16:0/1030093045 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8ee806a770 msgr2=0x7f8ee806ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:06.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.182+0000 7f8eed022700 1 --2- 172.21.15.16:0/1030093045 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8ee806a770 0x7f8ee806ab40 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f8ed80098f0 tx=0x7f8ed8009c00 comp rx=0 tx=0).stop 2023-09-30T22:36:06.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.182+0000 7f8eed022700 1 -- 172.21.15.16:0/1030093045 shutdown_connections 2023-09-30T22:36:06.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.182+0000 7f8eed022700 1 --2- 172.21.15.16:0/1030093045 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8ee806a770 0x7f8ee806ab40 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:06.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.182+0000 7f8eed022700 1 --2- 172.21.15.16:0/1030093045 >> v2:172.21.15.16:3300/0 conn(0x7f8ee806b080 0x7f8ee8131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:06.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.182+0000 7f8eed022700 1 -- 172.21.15.16:0/1030093045 >> 172.21.15.16:0/1030093045 conn(0x7f8ee8066f50 msgr2=0x7f8ee809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:06.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.182+0000 7f8eed022700 1 -- 172.21.15.16:0/1030093045 shutdown_connections 2023-09-30T22:36:06.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.182+0000 7f8eed022700 1 -- 172.21.15.16:0/1030093045 wait complete. 2023-09-30T22:36:06.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.184+0000 7f8eed022700 1 Processor -- start 2023-09-30T22:36:06.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.184+0000 7f8eed022700 1 -- start start 2023-09-30T22:36:06.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.184+0000 7f8eed022700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8ee806a770 0x7f8ee81210a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:06.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.184+0000 7f8eed022700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ee806b080 0x7f8ee81215e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:06.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.184+0000 7f8eed022700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8ee8131b40 0x7f8ee81c5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:06.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.184+0000 7f8eed022700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee806ed20 con 0x7f8ee806b080 2023-09-30T22:36:06.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.184+0000 7f8eed022700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee8120cc0 con 0x7f8ee8131b40 2023-09-30T22:36:06.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.184+0000 7f8eed022700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee8131210 con 0x7f8ee806a770 2023-09-30T22:36:06.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.184+0000 7f8ee659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ee806b080 0x7f8ee81215e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:06.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.185+0000 7f8ee6d9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8ee806a770 0x7f8ee81210a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:06.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.185+0000 7f8ee659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ee806b080 0x7f8ee81215e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39920/0 (socket says 172.21.15.16:39920) 2023-09-30T22:36:06.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.185+0000 7f8ee659c700 1 -- 172.21.15.16:0/1913991833 learned_addr learned my addr 172.21.15.16:0/1913991833 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:06.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.185+0000 7f8ee759e700 1 --2- 172.21.15.16:0/1913991833 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8ee8131b40 0x7f8ee81c5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:06.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.185+0000 7f8ee6d9d700 1 -- 172.21.15.16:0/1913991833 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8ee8131b40 msgr2=0x7f8ee81c5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:06.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.185+0000 7f8ee6d9d700 1 --2- 172.21.15.16:0/1913991833 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8ee8131b40 0x7f8ee81c5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:06.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.185+0000 7f8ee6d9d700 1 -- 172.21.15.16:0/1913991833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ee806b080 msgr2=0x7f8ee81215e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:06.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.185+0000 7f8ee6d9d700 1 --2- 172.21.15.16:0/1913991833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ee806b080 0x7f8ee81215e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:06.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.185+0000 7f8ee6d9d700 1 -- 172.21.15.16:0/1913991833 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8edc0094a0 con 0x7f8ee806a770 2023-09-30T22:36:06.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.185+0000 7f8ee659c700 1 --2- 172.21.15.16:0/1913991833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ee806b080 0x7f8ee81215e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:06.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.185+0000 7f8ee6d9d700 1 --2- 172.21.15.16:0/1913991833 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8ee806a770 0x7f8ee81210a0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8016000 tx=0x7f8ed8004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:06.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.185+0000 7f8ecf7fe700 1 -- 172.21.15.16:0/1913991833 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed8004300 con 0x7f8ee806a770 2023-09-30T22:36:06.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.185+0000 7f8ecf7fe700 1 -- 172.21.15.16:0/1913991833 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ed801f070 con 0x7f8ee806a770 2023-09-30T22:36:06.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.185+0000 7f8eed022700 1 -- 172.21.15.16:0/1913991833 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ed8009550 con 0x7f8ee806a770 2023-09-30T22:36:06.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.186+0000 7f8ecf7fe700 1 -- 172.21.15.16:0/1913991833 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed80036a0 con 0x7f8ee806a770 2023-09-30T22:36:06.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.186+0000 7f8eed022700 1 -- 172.21.15.16:0/1913991833 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ee8123c90 con 0x7f8ee806a770 2023-09-30T22:36:06.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.186+0000 7f8ecf7fe700 1 -- 172.21.15.16:0/1913991833 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8ed8021030 con 0x7f8ee806a770 2023-09-30T22:36:06.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.187+0000 7f8ecf7fe700 1 -- 172.21.15.16:0/1913991833 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f8ed8040910 con 0x7f8ee806a770 2023-09-30T22:36:06.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.187+0000 7f8eed022700 1 -- 172.21.15.16:0/1913991833 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ee806aea0 con 0x7f8ee806a770 2023-09-30T22:36:06.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.192+0000 7f8ecf7fe700 1 -- 172.21.15.16:0/1913991833 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8ed8015080 con 0x7f8ee806a770 2023-09-30T22:36:06.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.219+0000 7f7ab3a55700 1 Processor -- start 2023-09-30T22:36:06.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.219+0000 7fd06f63c700 1 Processor -- start 2023-09-30T22:36:06.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.220+0000 7fd06f63c700 1 -- start start 2023-09-30T22:36:06.220 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.220+0000 7f7ab3a55700 1 -- start start 2023-09-30T22:36:06.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.220+0000 7fd06f63c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd068131810 0x7fd068131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:06.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.220+0000 7fd06f63c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd068099a40 0x7fd068099e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:06.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.220+0000 7fd06f63c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd06809a3b0 0x7fd068138650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:06.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.220+0000 7fd06f63c700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0681313c0 con 0x7fd068131810 2023-09-30T22:36:06.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.220+0000 7fd06f63c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd06806e8c0 con 0x7fd06809a3b0 2023-09-30T22:36:06.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.220+0000 7fd06f63c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd068071d20 con 0x7fd068099a40 2023-09-30T22:36:06.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.220+0000 7f7ab3a55700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7aac133540 0x7f7aac135920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:06.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.220+0000 7f7ab3a55700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7aac135e60 0x7f7aac138240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:06.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.220+0000 7f7ab3a55700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7aac0716a0 con 0x7f7aac130540 2023-09-30T22:36:06.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.220+0000 7f7ab3a55700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aac071290 con 0x7f7aac133540 2023-09-30T22:36:06.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.220+0000 7f7ab3a55700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7aac130340 con 0x7f7aac135e60 2023-09-30T22:36:06.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.221+0000 7f7ab1ff2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7aac135e60 0x7f7aac138240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:06.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.221+0000 7f7ab17f1700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f7aac130540 0x7f7aac130910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38388/0 (socket says 172.21.15.16:38388) 2023-09-30T22:36:06.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.221+0000 7f7ab17f1700 1 -- 172.21.15.16:0/1754925457 learned_addr learned my addr 172.21.15.16:0/1754925457 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:36:06.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.221+0000 7fd06cbd7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd068099a40 0x7fd068099e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:06.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.221+0000 7fd06d3d8700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd068131810 0x7fd068131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:06.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.221+0000 7fd06cbd7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd068099a40 0x7fd068099e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:41266/0 (socket says 172.21.15.16:41266) 2023-09-30T22:36:06.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.221+0000 7fd06cbd7700 1 -- 172.21.15.16:0/2899811484 learned_addr learned my addr 172.21.15.16:0/2899811484 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:06.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.221+0000 7fd06d3d8700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd068131810 0x7fd068131be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39928/0 (socket says 172.21.15.16:39928) 2023-09-30T22:36:06.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.221+0000 7fd06dbd9700 1 --2- 172.21.15.16:0/2899811484 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd06809a3b0 0x7fd068138650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:06.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.221+0000 7f7ab0ff0700 1 --2- 172.21.15.16:0/1754925457 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7aac133540 0x7f7aac135920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:06.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.221+0000 7f7ab1ff2700 1 -- 172.21.15.16:0/1754925457 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7aac133540 msgr2=0x7f7aac135920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:06.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.221+0000 7f7ab1ff2700 1 --2- 172.21.15.16:0/1754925457 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7aac133540 0x7f7aac135920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:06.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.221+0000 7f7aa3fff700 1 -- 172.21.15.16:0/1754925457 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1514030988 0 0) 0x7f7aac0716a0 con 0x7f7aac130540 2023-09-30T22:36:06.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.221+0000 7f7ab1ff2700 1 -- 172.21.15.16:0/1754925457 >> v1:172.21.15.16:6789/0 conn(0x7f7aac130540 legacy=0x7f7aac130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:06.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.221+0000 7f7ab1ff2700 1 -- 172.21.15.16:0/1754925457 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aac070b80 con 0x7f7aac135e60 2023-09-30T22:36:06.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.221+0000 7f7ab1ff2700 1 --2- 172.21.15.16:0/1754925457 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7aac135e60 0x7f7aac138240 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4009dc0 tx=0x7f7aa400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa723f425b6923c1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:06.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.221+0000 7fd06d3d8700 1 -- 172.21.15.16:0/2899811484 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd068099a40 msgr2=0x7fd068099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:06.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.221+0000 7fd06d3d8700 1 --2- 172.21.15.16:0/2899811484 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd068099a40 0x7fd068099e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:06.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.221+0000 7fd06d3d8700 1 -- 172.21.15.16:0/2899811484 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd06809a3b0 msgr2=0x7fd068138650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:06.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.221+0000 7fd06d3d8700 1 --2- 172.21.15.16:0/2899811484 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd06809a3b0 0x7fd068138650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:06.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.221+0000 7fd06d3d8700 1 -- 172.21.15.16:0/2899811484 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0680721a0 con 0x7fd068131810 2023-09-30T22:36:06.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.222+0000 7fd06d3d8700 1 --2- 172.21.15.16:0/2899811484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd068131810 0x7fd068131be0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fd064009840 tx=0x7fd064009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0a4733db6848aa8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:06.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.222+0000 7f7aa3fff700 1 -- 172.21.15.16:0/1754925457 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa400e070 con 0x7f7aac135e60 2023-09-30T22:36:06.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.222+0000 7f7aa3fff700 1 -- 172.21.15.16:0/1754925457 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aa400bb20 con 0x7f7aac135e60 2023-09-30T22:36:06.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.222+0000 7f7aa3fff700 1 -- 172.21.15.16:0/1754925457 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa4012430 con 0x7f7aac135e60 2023-09-30T22:36:06.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.222+0000 7fd05ffff700 1 -- 172.21.15.16:0/2899811484 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd064011830 con 0x7fd068131810 2023-09-30T22:36:06.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.222+0000 7fd05ffff700 1 -- 172.21.15.16:0/2899811484 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd064011990 con 0x7fd068131810 2023-09-30T22:36:06.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.222+0000 7fd05ffff700 1 -- 172.21.15.16:0/2899811484 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd064011d20 con 0x7fd068131810 2023-09-30T22:36:06.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.223+0000 7f7ab3a55700 1 -- 172.21.15.16:0/1754925457 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7aac135e60 msgr2=0x7f7aac138240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:06.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.223+0000 7f7ab3a55700 1 --2- 172.21.15.16:0/1754925457 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7aac135e60 0x7f7aac138240 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4009dc0 tx=0x7f7aa400b330 comp rx=0 tx=0).stop 2023-09-30T22:36:06.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.223+0000 7f7ab3a55700 1 -- 172.21.15.16:0/1754925457 shutdown_connections 2023-09-30T22:36:06.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.223+0000 7f7ab3a55700 1 --2- 172.21.15.16:0/1754925457 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7aac135e60 0x7f7aac138240 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:06.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.223+0000 7f7ab3a55700 1 --2- 172.21.15.16:0/1754925457 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7aac133540 0x7f7aac135920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:06.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.223+0000 7f7ab3a55700 1 -- 172.21.15.16:0/1754925457 >> 172.21.15.16:0/1754925457 conn(0x7f7aac0a2aa0 msgr2=0x7f7aac0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:06.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.223+0000 7f7ab3a55700 1 -- 172.21.15.16:0/1754925457 shutdown_connections 2023-09-30T22:36:06.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.223+0000 7f7ab3a55700 1 -- 172.21.15.16:0/1754925457 wait complete. 2023-09-30T22:36:06.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.223+0000 7fd06f63c700 1 -- 172.21.15.16:0/2899811484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd068131810 msgr2=0x7fd068131be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:06.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.223+0000 7fd06f63c700 1 --2- 172.21.15.16:0/2899811484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd068131810 0x7fd068131be0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fd064009840 tx=0x7fd064009b50 comp rx=0 tx=0).stop 2023-09-30T22:36:06.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.223+0000 7fd06f63c700 1 -- 172.21.15.16:0/2899811484 shutdown_connections 2023-09-30T22:36:06.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.223+0000 7fd06f63c700 1 --2- 172.21.15.16:0/2899811484 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd06809a3b0 0x7fd068138650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:06.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.223+0000 7fd06f63c700 1 --2- 172.21.15.16:0/2899811484 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd068131810 0x7fd068131be0 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:06.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.223+0000 7fd06f63c700 1 --2- 172.21.15.16:0/2899811484 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd068099a40 0x7fd068099e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:06.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.223+0000 7fd06f63c700 1 -- 172.21.15.16:0/2899811484 >> 172.21.15.16:0/2899811484 conn(0x7fd0680a2aa0 msgr2=0x7fd0680a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:06.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.224+0000 7f7ab3a55700 1 Processor -- start 2023-09-30T22:36:06.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.224+0000 7fd06f63c700 1 -- 172.21.15.16:0/2899811484 shutdown_connections 2023-09-30T22:36:06.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.224+0000 7fd06f63c700 1 -- 172.21.15.16:0/2899811484 wait complete. 2023-09-30T22:36:06.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.224+0000 7f7ab3a55700 1 -- start start 2023-09-30T22:36:06.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.224+0000 7f7ab3a55700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7aac130540 0x7f7aac09bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:06.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.224+0000 7f7ab3a55700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7aac133540 0x7f7aac09c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:06.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.224+0000 7f7ab3a55700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7aac135e60 0x7f7aac09ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:06.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.224+0000 7f7ab3a55700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aac137930 con 0x7f7aac135e60 2023-09-30T22:36:06.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.224+0000 7f7ab3a55700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aac09d450 con 0x7f7aac133540 2023-09-30T22:36:06.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.224+0000 7f7ab3a55700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7aac1366b0 con 0x7f7aac130540 2023-09-30T22:36:06.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.224+0000 7f7ab17f1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7aac130540 0x7f7aac09bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:06.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.225+0000 7f7ab0ff0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7aac133540 0x7f7aac09c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:06.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.225+0000 7f7ab17f1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7aac130540 0x7f7aac09bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:41276/0 (socket says 172.21.15.16:41276) 2023-09-30T22:36:06.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.225+0000 7f7ab17f1700 1 -- 172.21.15.16:0/1736665116 learned_addr learned my addr 172.21.15.16:0/1736665116 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:06.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.225+0000 7f7ab1ff2700 1 --2- 172.21.15.16:0/1736665116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7aac135e60 0x7f7aac09ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:06.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.225+0000 7f7ab0ff0700 1 -- 172.21.15.16:0/1736665116 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7aac130540 msgr2=0x7f7aac09bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:06.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.225+0000 7f7ab0ff0700 1 --2- 172.21.15.16:0/1736665116 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7aac130540 0x7f7aac09bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:06.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.225+0000 7f7ab0ff0700 1 -- 172.21.15.16:0/1736665116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7aac135e60 msgr2=0x7f7aac09ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:06.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.225+0000 7f7ab0ff0700 1 --2- 172.21.15.16:0/1736665116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7aac135e60 0x7f7aac09ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:06.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.225+0000 7f7ab0ff0700 1 -- 172.21.15.16:0/1736665116 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aa4009a70 con 0x7f7aac133540 2023-09-30T22:36:06.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.225+0000 7f7ab0ff0700 1 --2- 172.21.15.16:0/1736665116 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7aac133540 0x7f7aac09c530 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c00e900 tx=0x7f7a9c00ec10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:06.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.225+0000 7f7aa1ffb700 1 -- 172.21.15.16:0/1736665116 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c013680 con 0x7f7aac133540 2023-09-30T22:36:06.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.225+0000 7f7aa1ffb700 1 -- 172.21.15.16:0/1736665116 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a9c0137e0 con 0x7f7aac133540 2023-09-30T22:36:06.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.225+0000 7fd06f63c700 1 Processor -- start 2023-09-30T22:36:06.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.225+0000 7fd06f63c700 1 -- start start 2023-09-30T22:36:06.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.225+0000 7f7ab3a55700 1 -- 172.21.15.16:0/1736665116 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7aac137d80 con 0x7f7aac133540 2023-09-30T22:36:06.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.226+0000 7f7aa1ffb700 1 -- 172.21.15.16:0/1736665116 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c013b50 con 0x7f7aac133540 2023-09-30T22:36:06.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.226+0000 7f7ab3a55700 1 -- 172.21.15.16:0/1736665116 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aac137280 con 0x7f7aac133540 2023-09-30T22:36:06.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.226+0000 7fd06f63c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd068099a40 0x7fd0681d5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:06.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.226+0000 7fd06f63c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd06809a3b0 0x7fd0681d5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:06.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.226+0000 7fd06f63c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd068131810 0x7fd06813af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:06.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.226+0000 7fd06f63c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0681d2cc0 con 0x7fd06809a3b0 2023-09-30T22:36:06.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.226+0000 7fd06f63c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd068134140 con 0x7fd068131810 2023-09-30T22:36:06.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.226+0000 7fd06f63c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd0681d2ef0 con 0x7fd068099a40 2023-09-30T22:36:06.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.226+0000 7fd06d3d8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd068099a40 0x7fd0681d5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:06.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.226+0000 7fd06d3d8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd068099a40 0x7fd0681d5130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:41290/0 (socket says 172.21.15.16:41290) 2023-09-30T22:36:06.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.226+0000 7fd06d3d8700 1 -- 172.21.15.16:0/2349782911 learned_addr learned my addr 172.21.15.16:0/2349782911 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:06.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.226+0000 7fd06dbd9700 1 --2- 172.21.15.16:0/2349782911 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd068131810 0x7fd06813af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:06.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.226+0000 7fd06cbd7700 1 --2- 172.21.15.16:0/2349782911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd06809a3b0 0x7fd0681d5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:06.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.227+0000 7fd06dbd9700 1 -- 172.21.15.16:0/2349782911 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd068099a40 msgr2=0x7fd0681d5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:06.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.227+0000 7fd06dbd9700 1 --2- 172.21.15.16:0/2349782911 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd068099a40 0x7fd0681d5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:06.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.227+0000 7fd06dbd9700 1 -- 172.21.15.16:0/2349782911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd06809a3b0 msgr2=0x7fd0681d5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:06.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.227+0000 7fd06dbd9700 1 --2- 172.21.15.16:0/2349782911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd06809a3b0 0x7fd0681d5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:06.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.227+0000 7fd06dbd9700 1 -- 172.21.15.16:0/2349782911 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0640094a0 con 0x7fd068131810 2023-09-30T22:36:06.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.227+0000 7fd06d3d8700 1 --2- 172.21.15.16:0/2349782911 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd068099a40 0x7fd0681d5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:06.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.227+0000 7fd06dbd9700 1 --2- 172.21.15.16:0/2349782911 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd068131810 0x7fd06813af20 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fd060009d90 tx=0x7fd06000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:06.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.227+0000 7f7ab3a55700 1 -- 172.21.15.16:0/1736665116 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7aac074ea0 con 0x7f7aac133540 2023-09-30T22:36:06.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.227+0000 7f7aa1ffb700 1 -- 172.21.15.16:0/1736665116 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7a9c013cb0 con 0x7f7aac133540 2023-09-30T22:36:06.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.227+0000 7fd05dffb700 1 -- 172.21.15.16:0/2349782911 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd060009c00 con 0x7fd068131810 2023-09-30T22:36:06.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.227+0000 7fd05dffb700 1 -- 172.21.15.16:0/2349782911 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd06000f040 con 0x7fd068131810 2023-09-30T22:36:06.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.227+0000 7fd06f63c700 1 -- 172.21.15.16:0/2349782911 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd068133ac0 con 0x7fd068131810 2023-09-30T22:36:06.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.227+0000 7fd05dffb700 1 -- 172.21.15.16:0/2349782911 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd060014480 con 0x7fd068131810 2023-09-30T22:36:06.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.227+0000 7fd06f63c700 1 -- 172.21.15.16:0/2349782911 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd068133d80 con 0x7fd068131810 2023-09-30T22:36:06.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.229+0000 7fd06f63c700 1 -- 172.21.15.16:0/2349782911 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd068133f70 con 0x7fd068131810 2023-09-30T22:36:06.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.229+0000 7f7aa1ffb700 1 -- 172.21.15.16:0/1736665116 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f7a9c011350 con 0x7f7aac133540 2023-09-30T22:36:06.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.232+0000 7f7aa1ffb700 1 -- 172.21.15.16:0/1736665116 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7a9c01b3b0 con 0x7f7aac133540 2023-09-30T22:36:06.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.233+0000 7fd05dffb700 1 -- 172.21.15.16:0/2349782911 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd0600145e0 con 0x7fd068131810 2023-09-30T22:36:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.233+0000 7fd05dffb700 1 -- 172.21.15.16:0/2349782911 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd060040710 con 0x7fd068131810 2023-09-30T22:36:06.242 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.237+0000 7fd05dffb700 1 -- 172.21.15.16:0/2349782911 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd060012930 con 0x7fd068131810 2023-09-30T22:36:06.346 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:36:06.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.403+0000 7f8eed022700 1 -- 172.21.15.16:0/1913991833 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f8ee8124190 con 0x7f8ee806a770 2023-09-30T22:36:06.404 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.404+0000 7f8ecf7fe700 1 -- 172.21.15.16:0/1913991833 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8ed80239c0 con 0x7f8ee806a770 2023-09-30T22:36:06.404 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:36:06.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.407+0000 7f8eed022700 1 -- 172.21.15.16:0/1913991833 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8ee806a770 msgr2=0x7f8ee81210a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:06.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.407+0000 7f8eed022700 1 --2- 172.21.15.16:0/1913991833 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8ee806a770 0x7f8ee81210a0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f8ed8016000 tx=0x7f8ed8004750 comp rx=0 tx=0).stop 2023-09-30T22:36:06.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.408+0000 7f8eed022700 1 -- 172.21.15.16:0/1913991833 shutdown_connections 2023-09-30T22:36:06.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.408+0000 7f8eed022700 1 --2- 172.21.15.16:0/1913991833 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8ee8131b40 0x7f8ee81c5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:06.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.408+0000 7f8eed022700 1 --2- 172.21.15.16:0/1913991833 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8ee806a770 0x7f8ee81210a0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:06.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.408+0000 7f8eed022700 1 --2- 172.21.15.16:0/1913991833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8ee806b080 0x7f8ee81215e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:06.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.408+0000 7f8eed022700 1 -- 172.21.15.16:0/1913991833 >> 172.21.15.16:0/1913991833 conn(0x7f8ee8066f50 msgr2=0x7f8ee806ff90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:06.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.408+0000 7f8eed022700 1 -- 172.21.15.16:0/1913991833 shutdown_connections 2023-09-30T22:36:06.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.408+0000 7f8eed022700 1 -- 172.21.15.16:0/1913991833 wait complete. 2023-09-30T22:36:06.428 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:36:06.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.448+0000 7fd06f63c700 1 -- 172.21.15.16:0/2349782911 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd068074ea0 con 0x7fd068131810 2023-09-30T22:36:06.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.449+0000 7fd05dffb700 1 -- 172.21.15.16:0/2349782911 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd06002a030 con 0x7fd068131810 2023-09-30T22:36:06.449 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:36:06.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.451+0000 7fd06f63c700 1 -- 172.21.15.16:0/2349782911 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd068131810 msgr2=0x7fd06813af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:06.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.451+0000 7fd06f63c700 1 --2- 172.21.15.16:0/2349782911 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd068131810 0x7fd06813af20 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fd060009d90 tx=0x7fd06000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:36:06.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.451+0000 7fd06f63c700 1 -- 172.21.15.16:0/2349782911 shutdown_connections 2023-09-30T22:36:06.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.451+0000 7fd06f63c700 1 --2- 172.21.15.16:0/2349782911 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd068131810 0x7fd06813af20 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:06.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.451+0000 7fd06f63c700 1 --2- 172.21.15.16:0/2349782911 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd068099a40 0x7fd0681d5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:06.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.451+0000 7fd06f63c700 1 --2- 172.21.15.16:0/2349782911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd06809a3b0 0x7fd0681d5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:06.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.451+0000 7fd06f63c700 1 -- 172.21.15.16:0/2349782911 >> 172.21.15.16:0/2349782911 conn(0x7fd0680a2aa0 msgr2=0x7fd068130220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:06.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.451+0000 7fd06f63c700 1 -- 172.21.15.16:0/2349782911 shutdown_connections 2023-09-30T22:36:06.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.451+0000 7fd06f63c700 1 -- 172.21.15.16:0/2349782911 wait complete. 2023-09-30T22:36:06.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.454+0000 7f7ab3a55700 1 -- 172.21.15.16:0/1736665116 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7aac0a1df0 con 0x7f7aac133540 2023-09-30T22:36:06.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.455+0000 7f7aa1ffb700 1 -- 172.21.15.16:0/1736665116 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7a9c017020 con 0x7f7aac133540 2023-09-30T22:36:06.455 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:36:06.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.457+0000 7f7ab3a55700 1 -- 172.21.15.16:0/1736665116 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7aac133540 msgr2=0x7f7aac09c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:06.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.457+0000 7f7ab3a55700 1 --2- 172.21.15.16:0/1736665116 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7aac133540 0x7f7aac09c530 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c00e900 tx=0x7f7a9c00ec10 comp rx=0 tx=0).stop 2023-09-30T22:36:06.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.457+0000 7f7ab3a55700 1 -- 172.21.15.16:0/1736665116 shutdown_connections 2023-09-30T22:36:06.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.457+0000 7f7ab3a55700 1 --2- 172.21.15.16:0/1736665116 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7aac135e60 0x7f7aac09ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:06.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.457+0000 7f7ab3a55700 1 --2- 172.21.15.16:0/1736665116 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7aac130540 0x7f7aac09bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:06.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.457+0000 7f7ab3a55700 1 --2- 172.21.15.16:0/1736665116 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7aac133540 0x7f7aac09c530 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:06.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.457+0000 7f7ab3a55700 1 -- 172.21.15.16:0/1736665116 >> 172.21.15.16:0/1736665116 conn(0x7f7aac0a2aa0 msgr2=0x7f7aac131320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:06.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.457+0000 7f7ab3a55700 1 -- 172.21.15.16:0/1736665116 shutdown_connections 2023-09-30T22:36:06.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.457+0000 7f7ab3a55700 1 -- 172.21.15.16:0/1736665116 wait complete. 2023-09-30T22:36:06.468 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:36:06.473 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:36:06.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.491+0000 7fe0c3fdb700 1 Processor -- start 2023-09-30T22:36:06.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.491+0000 7fe0c3fdb700 1 -- start start 2023-09-30T22:36:06.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.491+0000 7fe0c3fdb700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe0bc133420 0x7fe0bc1337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:06.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.491+0000 7fe0c3fdb700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe0bc133e30 0x7fe0bc12ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:06.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.492+0000 7fe0c3fdb700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0bc06b990 con 0x7fe0bc134260 2023-09-30T22:36:06.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.492+0000 7fe0c3fdb700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0bc0a0590 con 0x7fe0bc133420 2023-09-30T22:36:06.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.492+0000 7fe0c3fdb700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe0bc09fb80 con 0x7fe0bc133e30 2023-09-30T22:36:06.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.492+0000 7fe0c2578700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe0bc134260 0x7fe0bc1304d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:38402/0 (socket says 172.21.15.16:38402) 2023-09-30T22:36:06.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.492+0000 7fe0c2578700 1 -- 172.21.15.16:0/4112767336 learned_addr learned my addr 172.21.15.16:0/4112767336 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:36:06.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.492+0000 7fe0c1576700 1 --2- 172.21.15.16:0/4112767336 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe0bc133e30 0x7fe0bc12ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:06.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.492+0000 7fe0c1d77700 1 --2- 172.21.15.16:0/4112767336 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe0bc133420 0x7fe0bc1337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:06.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.492+0000 7fe0c0d75700 1 -- 172.21.15.16:0/4112767336 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1431908927 0 0) 0x7fe0bc06b990 con 0x7fe0bc134260 2023-09-30T22:36:06.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.492+0000 7fe0c0d75700 1 -- 172.21.15.16:0/4112767336 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0a80035b0 con 0x7fe0bc134260 2023-09-30T22:36:06.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.492+0000 7fe0c1576700 1 -- 172.21.15.16:0/4112767336 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe0bc133420 msgr2=0x7fe0bc1337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:06.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.492+0000 7fe0c1576700 1 --2- 172.21.15.16:0/4112767336 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe0bc133420 0x7fe0bc1337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:06.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.492+0000 7fe0c1576700 1 -- 172.21.15.16:0/4112767336 >> v1:172.21.15.16:6789/0 conn(0x7fe0bc134260 legacy=0x7fe0bc1304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:06.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.492+0000 7fe0c1576700 1 -- 172.21.15.16:0/4112767336 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0bc0a0030 con 0x7fe0bc133e30 2023-09-30T22:36:06.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.492+0000 7fe0c1d77700 1 --2- 172.21.15.16:0/4112767336 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe0bc133420 0x7fe0bc1337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:06.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.492+0000 7fe0c1576700 1 --2- 172.21.15.16:0/4112767336 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe0bc133e30 0x7fe0bc12ff90 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac009870 tx=0x7fe0ac009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e386774393ef63d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:06.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.492+0000 7fe0c0d75700 1 -- 172.21.15.16:0/4112767336 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac011830 con 0x7fe0bc133e30 2023-09-30T22:36:06.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.493+0000 7fe0c0d75700 1 -- 172.21.15.16:0/4112767336 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0ac011990 con 0x7fe0bc133e30 2023-09-30T22:36:06.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.493+0000 7fe0c0d75700 1 -- 172.21.15.16:0/4112767336 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac011d20 con 0x7fe0bc133e30 2023-09-30T22:36:06.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.493+0000 7fe0c3fdb700 1 -- 172.21.15.16:0/4112767336 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe0bc133e30 msgr2=0x7fe0bc12ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:06.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.493+0000 7fe0c3fdb700 1 --2- 172.21.15.16:0/4112767336 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe0bc133e30 0x7fe0bc12ff90 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac009870 tx=0x7fe0ac009b80 comp rx=0 tx=0).stop 2023-09-30T22:36:06.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.493+0000 7fe0c3fdb700 1 -- 172.21.15.16:0/4112767336 shutdown_connections 2023-09-30T22:36:06.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.493+0000 7fe0c3fdb700 1 --2- 172.21.15.16:0/4112767336 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe0bc133420 0x7fe0bc1337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:06.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.493+0000 7fe0c3fdb700 1 --2- 172.21.15.16:0/4112767336 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe0bc133e30 0x7fe0bc12ff90 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:06.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.493+0000 7fe0c3fdb700 1 -- 172.21.15.16:0/4112767336 >> 172.21.15.16:0/4112767336 conn(0x7fe0bc0a4210 msgr2=0x7fe0bc0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:06.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.493+0000 7fe0c3fdb700 1 -- 172.21.15.16:0/4112767336 shutdown_connections 2023-09-30T22:36:06.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.493+0000 7fe0c3fdb700 1 -- 172.21.15.16:0/4112767336 wait complete. 2023-09-30T22:36:06.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.494+0000 7fe0c3fdb700 1 Processor -- start 2023-09-30T22:36:06.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.494+0000 7fe0c3fdb700 1 -- start start 2023-09-30T22:36:06.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.495+0000 7fe0c3fdb700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe0bc133420 0x7fe0bc09c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:06.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.495+0000 7fe0c3fdb700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe0bc133e30 0x7fe0bc09c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:06.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.495+0000 7fe0c3fdb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0bc134260 0x7fe0bc09cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:06.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.495+0000 7fe0c3fdb700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0bc11ff20 con 0x7fe0bc134260 2023-09-30T22:36:06.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.495+0000 7fe0c3fdb700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0bc120100 con 0x7fe0bc133420 2023-09-30T22:36:06.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.495+0000 7fe0c3fdb700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe0bc12d920 con 0x7fe0bc133e30 2023-09-30T22:36:06.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.495+0000 7fe0c1d77700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe0bc133420 0x7fe0bc09c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:06.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.495+0000 7fe0c1576700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe0bc133e30 0x7fe0bc09c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:06.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.495+0000 7fe0c1d77700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe0bc133420 0x7fe0bc09c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:47752/0 (socket says 172.21.15.16:47752) 2023-09-30T22:36:06.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.495+0000 7fe0c1d77700 1 -- 172.21.15.16:0/3741456014 learned_addr learned my addr 172.21.15.16:0/3741456014 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:06.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.495+0000 7fe0c2578700 1 --2- 172.21.15.16:0/3741456014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0bc134260 0x7fe0bc09cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:06.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.495+0000 7fe0c1d77700 1 -- 172.21.15.16:0/3741456014 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe0bc133e30 msgr2=0x7fe0bc09c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:06.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.495+0000 7fe0c1d77700 1 --2- 172.21.15.16:0/3741456014 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe0bc133e30 0x7fe0bc09c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:06.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.495+0000 7fe0c1d77700 1 -- 172.21.15.16:0/3741456014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0bc134260 msgr2=0x7fe0bc09cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:06.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.495+0000 7fe0c1d77700 1 --2- 172.21.15.16:0/3741456014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0bc134260 0x7fe0bc09cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:06.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.495+0000 7fe0c1d77700 1 -- 172.21.15.16:0/3741456014 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b80094a0 con 0x7fe0bc133420 2023-09-30T22:36:06.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.495+0000 7fe0c1d77700 1 --2- 172.21.15.16:0/3741456014 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe0bc133420 0x7fe0bc09c480 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fe0b800ec40 tx=0x7fe0b800ef50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:06.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.495+0000 7fe0b27fc700 1 -- 172.21.15.16:0/3741456014 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b800c850 con 0x7fe0bc133420 2023-09-30T22:36:06.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.496+0000 7fe0b27fc700 1 -- 172.21.15.16:0/3741456014 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0b800c9b0 con 0x7fe0bc133420 2023-09-30T22:36:06.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.496+0000 7fe0c3fdb700 1 -- 172.21.15.16:0/3741456014 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0ac0094a0 con 0x7fe0bc133420 2023-09-30T22:36:06.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.496+0000 7fe0b27fc700 1 -- 172.21.15.16:0/3741456014 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b800cd20 con 0x7fe0bc133420 2023-09-30T22:36:06.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.496+0000 7fe0c3fdb700 1 -- 172.21.15.16:0/3741456014 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0bc11fb60 con 0x7fe0bc133420 2023-09-30T22:36:06.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.497+0000 7fe0b27fc700 1 -- 172.21.15.16:0/3741456014 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe0b8013470 con 0x7fe0bc133420 2023-09-30T22:36:06.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.497+0000 7fe0b27fc700 1 -- 172.21.15.16:0/3741456014 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fe0b803fb50 con 0x7fe0bc133420 2023-09-30T22:36:06.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.497+0000 7fe0c3fdb700 1 -- 172.21.15.16:0/3741456014 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0bc11fd50 con 0x7fe0bc133420 2023-09-30T22:36:06.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.502+0000 7fe0b27fc700 1 -- 172.21.15.16:0/3741456014 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe0b8021080 con 0x7fe0bc133420 2023-09-30T22:36:06.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.710+0000 7fe0c3fdb700 1 -- 172.21.15.16:0/3741456014 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe0bc11f480 con 0x7fe0bc133420 2023-09-30T22:36:06.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.710+0000 7fe0b27fc700 1 -- 172.21.15.16:0/3741456014 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe0b8013720 con 0x7fe0bc133420 2023-09-30T22:36:06.711 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:36:06.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.712+0000 7fe0c3fdb700 1 -- 172.21.15.16:0/3741456014 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe0bc133420 msgr2=0x7fe0bc09c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:06.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.712+0000 7fe0c3fdb700 1 --2- 172.21.15.16:0/3741456014 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe0bc133420 0x7fe0bc09c480 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fe0b800ec40 tx=0x7fe0b800ef50 comp rx=0 tx=0).stop 2023-09-30T22:36:06.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.712+0000 7fe0c3fdb700 1 -- 172.21.15.16:0/3741456014 shutdown_connections 2023-09-30T22:36:06.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.712+0000 7fe0c3fdb700 1 --2- 172.21.15.16:0/3741456014 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe0bc133420 0x7fe0bc09c480 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:06.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.712+0000 7fe0c3fdb700 1 --2- 172.21.15.16:0/3741456014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe0bc134260 0x7fe0bc09cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:06.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.712+0000 7fe0c3fdb700 1 --2- 172.21.15.16:0/3741456014 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe0bc133e30 0x7fe0bc09c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:06.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.713+0000 7fe0c3fdb700 1 -- 172.21.15.16:0/3741456014 >> 172.21.15.16:0/3741456014 conn(0x7fe0bc0a4210 msgr2=0x7fe0bc130460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:06.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.713+0000 7fe0c3fdb700 1 -- 172.21.15.16:0/3741456014 shutdown_connections 2023-09-30T22:36:06.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:06.713+0000 7fe0c3fdb700 1 -- 172.21.15.16:0/3741456014 wait complete. 2023-09-30T22:36:06.727 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:36:07.429 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:36:07.469 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:36:07.473 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:36:07.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.583+0000 7f999ae5c700 1 Processor -- start 2023-09-30T22:36:07.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.583+0000 7f999ae5c700 1 -- start start 2023-09-30T22:36:07.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.584+0000 7f999ae5c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f99940a0a30 0x7f99940a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:07.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.584+0000 7f999ae5c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9994134170 0x7f999413bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:07.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.584+0000 7f999ae5c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f999409edd0 con 0x7f99940a1340 2023-09-30T22:36:07.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.584+0000 7f999ae5c700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f999409e1f0 con 0x7f99940a0a30 2023-09-30T22:36:07.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.584+0000 7f999ae5c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f999409dc50 con 0x7f9994134170 2023-09-30T22:36:07.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.584+0000 7f9998bf8700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f99940a0a30 0x7f99940a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:07.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.584+0000 7f9998bf8700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f99940a0a30 0x7f99940a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51672/0 (socket says 172.21.15.16:51672) 2023-09-30T22:36:07.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.584+0000 7f9998bf8700 1 -- 172.21.15.16:0/1034572531 learned_addr learned my addr 172.21.15.16:0/1034572531 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:07.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.584+0000 7f99993f9700 1 --2- 172.21.15.16:0/1034572531 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9994134170 0x7f999413bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:07.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.584+0000 7f9998bf8700 1 -- 172.21.15.16:0/1034572531 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9994134170 msgr2=0x7f999413bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:07.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.584+0000 7f9998bf8700 1 --2- 172.21.15.16:0/1034572531 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9994134170 0x7f999413bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:07.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.584+0000 7f9998bf8700 1 -- 172.21.15.16:0/1034572531 >> v1:172.21.15.16:6789/0 conn(0x7f99940a1340 legacy=0x7f9994133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:07.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.584+0000 7f9998bf8700 1 -- 172.21.15.16:0/1034572531 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99940a1fe0 con 0x7f99940a0a30 2023-09-30T22:36:07.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.584+0000 7f99993f9700 1 --2- 172.21.15.16:0/1034572531 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9994134170 0x7f999413bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:07.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.584+0000 7f99937fe700 1 -- 172.21.15.16:0/1034572531 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3192642480 0 0) 0x7f999409edd0 con 0x7f99940a1340 2023-09-30T22:36:07.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.584+0000 7f9998bf8700 1 --2- 172.21.15.16:0/1034572531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99940a0a30 0x7f99940a0e00 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f99880098f0 tx=0x7f9988009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=188e0b15c8bbd133 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:07.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.585+0000 7f99937fe700 1 -- 172.21.15.16:0/1034572531 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9988011830 con 0x7f99940a0a30 2023-09-30T22:36:07.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.585+0000 7f99937fe700 1 -- 172.21.15.16:0/1034572531 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9988011990 con 0x7f99940a0a30 2023-09-30T22:36:07.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.585+0000 7f99937fe700 1 -- 172.21.15.16:0/1034572531 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9988011d20 con 0x7f99940a0a30 2023-09-30T22:36:07.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.585+0000 7f999ae5c700 1 -- 172.21.15.16:0/1034572531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99940a0a30 msgr2=0x7f99940a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:07.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.585+0000 7f999ae5c700 1 --2- 172.21.15.16:0/1034572531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99940a0a30 0x7f99940a0e00 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f99880098f0 tx=0x7f9988009c00 comp rx=0 tx=0).stop 2023-09-30T22:36:07.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.585+0000 7f999ae5c700 1 -- 172.21.15.16:0/1034572531 shutdown_connections 2023-09-30T22:36:07.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.585+0000 7f999ae5c700 1 --2- 172.21.15.16:0/1034572531 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9994134170 0x7f999413bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:07.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.585+0000 7f999ae5c700 1 --2- 172.21.15.16:0/1034572531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99940a0a30 0x7f99940a0e00 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:07.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.585+0000 7f999ae5c700 1 -- 172.21.15.16:0/1034572531 >> 172.21.15.16:0/1034572531 conn(0x7f99940a6550 msgr2=0x7f99940a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:07.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.586+0000 7f999ae5c700 1 -- 172.21.15.16:0/1034572531 shutdown_connections 2023-09-30T22:36:07.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.586+0000 7f999ae5c700 1 -- 172.21.15.16:0/1034572531 wait complete. 2023-09-30T22:36:07.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.587+0000 7f999ae5c700 1 Processor -- start 2023-09-30T22:36:07.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.587+0000 7f999ae5c700 1 -- start start 2023-09-30T22:36:07.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.588+0000 7f999ae5c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f99940a0a30 0x7f9994122670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:07.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.588+0000 7f999ae5c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99940a1340 0x7f9994122bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:07.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.588+0000 7f999ae5c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9994134170 0x7f99941230f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:07.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.588+0000 7f999ae5c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9994126190 con 0x7f99940a1340 2023-09-30T22:36:07.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.588+0000 7f999ae5c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9994125bf0 con 0x7f99940a0a30 2023-09-30T22:36:07.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.588+0000 7f999ae5c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9994124320 con 0x7f9994134170 2023-09-30T22:36:07.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.588+0000 7f9993fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99940a1340 0x7f9994122bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:07.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.588+0000 7f9993fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99940a1340 0x7f9994122bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51678/0 (socket says 172.21.15.16:51678) 2023-09-30T22:36:07.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.588+0000 7f9993fff700 1 -- 172.21.15.16:0/138153605 learned_addr learned my addr 172.21.15.16:0/138153605 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:07.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.588+0000 7f99993f9700 1 --2- 172.21.15.16:0/138153605 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9994134170 0x7f99941230f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:07.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.588+0000 7f9993fff700 1 -- 172.21.15.16:0/138153605 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9994134170 msgr2=0x7f99941230f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:07.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.588+0000 7f9993fff700 1 --2- 172.21.15.16:0/138153605 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9994134170 0x7f99941230f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:07.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.588+0000 7f9993fff700 1 -- 172.21.15.16:0/138153605 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f99940a0a30 msgr2=0x7f9994122670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:07.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.588+0000 7f9993fff700 1 --2- 172.21.15.16:0/138153605 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f99940a0a30 0x7f9994122670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:07.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.588+0000 7f9993fff700 1 -- 172.21.15.16:0/138153605 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f998c009a70 con 0x7f99940a1340 2023-09-30T22:36:07.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.588+0000 7f99993f9700 1 --2- 172.21.15.16:0/138153605 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9994134170 0x7f99941230f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:07.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.589+0000 7f9993fff700 1 --2- 172.21.15.16:0/138153605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99940a1340 0x7f9994122bb0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f9980001eb0 tx=0x7f998000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:07.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.589+0000 7f99917fa700 1 -- 172.21.15.16:0/138153605 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99800054e0 con 0x7f99940a1340 2023-09-30T22:36:07.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.589+0000 7f99917fa700 1 -- 172.21.15.16:0/138153605 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9980003680 con 0x7f99940a1340 2023-09-30T22:36:07.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.589+0000 7f999ae5c700 1 -- 172.21.15.16:0/138153605 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9988009550 con 0x7f99940a1340 2023-09-30T22:36:07.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.589+0000 7f99917fa700 1 -- 172.21.15.16:0/138153605 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99800039f0 con 0x7f99940a1340 2023-09-30T22:36:07.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.589+0000 7f999ae5c700 1 -- 172.21.15.16:0/138153605 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99940a25e0 con 0x7f99940a1340 2023-09-30T22:36:07.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.590+0000 7f99917fa700 1 -- 172.21.15.16:0/138153605 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9980003b50 con 0x7f99940a1340 2023-09-30T22:36:07.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.590+0000 7f99917fa700 1 -- 172.21.15.16:0/138153605 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f998003ead0 con 0x7f99940a1340 2023-09-30T22:36:07.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.590+0000 7f999ae5c700 1 -- 172.21.15.16:0/138153605 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99940a0220 con 0x7f99940a1340 2023-09-30T22:36:07.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.595+0000 7f99917fa700 1 -- 172.21.15.16:0/138153605 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9980017020 con 0x7f99940a1340 2023-09-30T22:36:07.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.620+0000 7fdcace8a700 1 Processor -- start 2023-09-30T22:36:07.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.620+0000 7fdcace8a700 1 -- start start 2023-09-30T22:36:07.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.621+0000 7fdcace8a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdca81338a0 0x7fdca8133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:07.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.621+0000 7fdcace8a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdca809de50 0x7fdca809e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:07.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.621+0000 7fdcace8a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdca806ef30 con 0x7fdca809e760 2023-09-30T22:36:07.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.621+0000 7fdcace8a700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdca806fef0 con 0x7fdca81338a0 2023-09-30T22:36:07.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.621+0000 7fdcace8a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdca81334b0 con 0x7fdca809de50 2023-09-30T22:36:07.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.621+0000 7fdca659c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdca81338a0 0x7fdca8133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:07.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.621+0000 7fdca6d9d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fdca809e760 0x7fdca813a730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52144/0 (socket says 172.21.15.16:52144) 2023-09-30T22:36:07.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.621+0000 7fdca6d9d700 1 -- 172.21.15.16:0/1396605132 learned_addr learned my addr 172.21.15.16:0/1396605132 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:36:07.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.621+0000 7fdca5d9b700 1 --2- 172.21.15.16:0/1396605132 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdca809de50 0x7fdca809e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:07.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.622+0000 7fdca659c700 1 -- 172.21.15.16:0/1396605132 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdca809de50 msgr2=0x7fdca809e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:07.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.622+0000 7fdca659c700 1 --2- 172.21.15.16:0/1396605132 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdca809de50 0x7fdca809e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:07.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.622+0000 7fdca659c700 1 -- 172.21.15.16:0/1396605132 >> v1:172.21.15.16:6789/0 conn(0x7fdca809e760 legacy=0x7fdca813a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:07.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.622+0000 7fdca559a700 1 -- 172.21.15.16:0/1396605132 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3897833240 0 0) 0x7fdca806ef30 con 0x7fdca809e760 2023-09-30T22:36:07.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.622+0000 7fdca659c700 1 -- 172.21.15.16:0/1396605132 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdca806fc30 con 0x7fdca81338a0 2023-09-30T22:36:07.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.622+0000 7fdca659c700 1 --2- 172.21.15.16:0/1396605132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdca81338a0 0x7fdca8133c70 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fdc98009840 tx=0x7fdc98009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cd1a4373ba354c3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:07.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.622+0000 7fdca559a700 1 -- 172.21.15.16:0/1396605132 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc98011830 con 0x7fdca81338a0 2023-09-30T22:36:07.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.622+0000 7fdca559a700 1 -- 172.21.15.16:0/1396605132 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc98011990 con 0x7fdca81338a0 2023-09-30T22:36:07.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.622+0000 7fdca559a700 1 -- 172.21.15.16:0/1396605132 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc98011d20 con 0x7fdca81338a0 2023-09-30T22:36:07.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.623+0000 7fdcace8a700 1 -- 172.21.15.16:0/1396605132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdca81338a0 msgr2=0x7fdca8133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:07.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.623+0000 7fdcace8a700 1 --2- 172.21.15.16:0/1396605132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdca81338a0 0x7fdca8133c70 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fdc98009840 tx=0x7fdc98009b50 comp rx=0 tx=0).stop 2023-09-30T22:36:07.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.623+0000 7fdcace8a700 1 -- 172.21.15.16:0/1396605132 shutdown_connections 2023-09-30T22:36:07.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.623+0000 7fdcace8a700 1 --2- 172.21.15.16:0/1396605132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdca81338a0 0x7fdca8133c70 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:07.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.623+0000 7fdcace8a700 1 --2- 172.21.15.16:0/1396605132 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdca809de50 0x7fdca809e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:07.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.623+0000 7fdcace8a700 1 -- 172.21.15.16:0/1396605132 >> 172.21.15.16:0/1396605132 conn(0x7fdca80a2aa0 msgr2=0x7fdca80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:07.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.623+0000 7fdcace8a700 1 -- 172.21.15.16:0/1396605132 shutdown_connections 2023-09-30T22:36:07.625 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.623+0000 7fdcace8a700 1 -- 172.21.15.16:0/1396605132 wait complete. 2023-09-30T22:36:07.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.624+0000 7fdcace8a700 1 Processor -- start 2023-09-30T22:36:07.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.624+0000 7fdcace8a700 1 -- start start 2023-09-30T22:36:07.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.625+0000 7fdcace8a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdca809de50 0x7fdca811a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:07.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.625+0000 7fdcace8a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdca809e760 0x7fdca811ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:07.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.625+0000 7fdcace8a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdca81338a0 0x7fdca811b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:07.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.625+0000 7fdcace8a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca811daf0 con 0x7fdca809de50 2023-09-30T22:36:07.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.625+0000 7fdcace8a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdca811ea50 con 0x7fdca809e760 2023-09-30T22:36:07.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.625+0000 7fdcace8a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdca806f640 con 0x7fdca81338a0 2023-09-30T22:36:07.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.625+0000 7fdca659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdca809de50 0x7fdca811a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:07.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.625+0000 7fdca659c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdca809de50 0x7fdca811a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51702/0 (socket says 172.21.15.16:51702) 2023-09-30T22:36:07.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.625+0000 7fdca659c700 1 -- 172.21.15.16:0/839831731 learned_addr learned my addr 172.21.15.16:0/839831731 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:07.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.625+0000 7fdca6d9d700 1 --2- 172.21.15.16:0/839831731 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdca81338a0 0x7fdca811b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:07.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.625+0000 7fdca5d9b700 1 --2- 172.21.15.16:0/839831731 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdca809e760 0x7fdca811ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:07.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.625+0000 7fdca659c700 1 -- 172.21.15.16:0/839831731 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdca81338a0 msgr2=0x7fdca811b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:07.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.625+0000 7fdca659c700 1 --2- 172.21.15.16:0/839831731 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdca81338a0 0x7fdca811b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:07.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.625+0000 7fdca659c700 1 -- 172.21.15.16:0/839831731 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdca809e760 msgr2=0x7fdca811ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:07.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.625+0000 7fdca659c700 1 --2- 172.21.15.16:0/839831731 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdca809e760 0x7fdca811ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:07.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.625+0000 7fdca659c700 1 -- 172.21.15.16:0/839831731 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc980094a0 con 0x7fdca809de50 2023-09-30T22:36:07.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.626+0000 7fdca659c700 1 --2- 172.21.15.16:0/839831731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdca809de50 0x7fdca811a650 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fdc980090b0 tx=0x7fdc98004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:07.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.626+0000 7fdc96ffd700 1 -- 172.21.15.16:0/839831731 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc980036a0 con 0x7fdca809de50 2023-09-30T22:36:07.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.626+0000 7fdc96ffd700 1 -- 172.21.15.16:0/839831731 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc9801f070 con 0x7fdca809de50 2023-09-30T22:36:07.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.626+0000 7fdcace8a700 1 -- 172.21.15.16:0/839831731 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdca811d3d0 con 0x7fdca809de50 2023-09-30T22:36:07.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.626+0000 7fdc96ffd700 1 -- 172.21.15.16:0/839831731 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc98003a60 con 0x7fdca809de50 2023-09-30T22:36:07.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.626+0000 7fdcace8a700 1 -- 172.21.15.16:0/839831731 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdca811ddb0 con 0x7fdca809de50 2023-09-30T22:36:07.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.627+0000 7fdc96ffd700 1 -- 172.21.15.16:0/839831731 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdc98003bc0 con 0x7fdca809de50 2023-09-30T22:36:07.629 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.627+0000 7fdc96ffd700 1 -- 172.21.15.16:0/839831731 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fdc98021070 con 0x7fdca809de50 2023-09-30T22:36:07.630 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.627+0000 7fdcace8a700 1 -- 172.21.15.16:0/839831731 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdca8074ea0 con 0x7fdca809de50 2023-09-30T22:36:07.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.631+0000 7f96bc51c700 1 Processor -- start 2023-09-30T22:36:07.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.631+0000 7f96bc51c700 1 -- start start 2023-09-30T22:36:07.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.632+0000 7f96bc51c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f96b4134630 0x7f96b4136a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:07.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.632+0000 7f96bc51c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f96b4136f50 0x7f96b4139330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:07.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.632+0000 7f96bc51c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96b409b4f0 con 0x7f96b4131810 2023-09-30T22:36:07.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.632+0000 7f96bc51c700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96b41313c0 con 0x7f96b4134630 2023-09-30T22:36:07.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.632+0000 7f96bc51c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f96b406e8c0 con 0x7f96b4136f50 2023-09-30T22:36:07.633 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.632+0000 7f96baab9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f96b4136f50 0x7f96b4139330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:07.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.632+0000 7f96baab9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f96b4136f50 0x7f96b4139330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:43766/0 (socket says 172.21.15.16:43766) 2023-09-30T22:36:07.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.632+0000 7f96baab9700 1 -- 172.21.15.16:0/2806614479 learned_addr learned my addr 172.21.15.16:0/2806614479 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:07.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.632+0000 7f96b9ab7700 1 --2- 172.21.15.16:0/2806614479 >> v2:172.21.15.16:3300/0 conn(0x7f96b4134630 0x7f96b4136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:07.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.632+0000 7f96baab9700 1 -- 172.21.15.16:0/2806614479 >> v2:172.21.15.16:3300/0 conn(0x7f96b4134630 msgr2=0x7f96b4136a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:07.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.632+0000 7f96baab9700 1 --2- 172.21.15.16:0/2806614479 >> v2:172.21.15.16:3300/0 conn(0x7f96b4134630 0x7f96b4136a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:07.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.632+0000 7f96baab9700 1 -- 172.21.15.16:0/2806614479 >> v1:172.21.15.16:6789/0 conn(0x7f96b4131810 legacy=0x7f96b4131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:07.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.633+0000 7f96baab9700 1 -- 172.21.15.16:0/2806614479 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b4071d20 con 0x7f96b4136f50 2023-09-30T22:36:07.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.633+0000 7f96b9ab7700 1 --2- 172.21.15.16:0/2806614479 >> v2:172.21.15.16:3300/0 conn(0x7f96b4134630 0x7f96b4136a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:07.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.633+0000 7f96baab9700 1 --2- 172.21.15.16:0/2806614479 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f96b4136f50 0x7f96b4139330 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f96ac009e50 tx=0x7f96ac00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ca09c04964417e0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:07.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.633+0000 7f96b92b6700 1 -- 172.21.15.16:0/2806614479 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ac00e070 con 0x7f96b4136f50 2023-09-30T22:36:07.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.633+0000 7f96b92b6700 1 -- 172.21.15.16:0/2806614479 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96ac00b910 con 0x7f96b4136f50 2023-09-30T22:36:07.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.633+0000 7f96b92b6700 1 -- 172.21.15.16:0/2806614479 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ac00bcc0 con 0x7f96b4136f50 2023-09-30T22:36:07.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.634+0000 7f96bc51c700 1 -- 172.21.15.16:0/2806614479 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f96b4136f50 msgr2=0x7f96b4139330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:07.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.634+0000 7f96bc51c700 1 --2- 172.21.15.16:0/2806614479 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f96b4136f50 0x7f96b4139330 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f96ac009e50 tx=0x7f96ac00b330 comp rx=0 tx=0).stop 2023-09-30T22:36:07.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.634+0000 7f96bc51c700 1 -- 172.21.15.16:0/2806614479 shutdown_connections 2023-09-30T22:36:07.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.634+0000 7f96bc51c700 1 --2- 172.21.15.16:0/2806614479 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f96b4136f50 0x7f96b4139330 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:07.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.634+0000 7f96bc51c700 1 --2- 172.21.15.16:0/2806614479 >> v2:172.21.15.16:3300/0 conn(0x7f96b4134630 0x7f96b4136a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:07.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.634+0000 7f96bc51c700 1 -- 172.21.15.16:0/2806614479 >> 172.21.15.16:0/2806614479 conn(0x7f96b40a2aa0 msgr2=0x7f96b40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:07.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.634+0000 7f96bc51c700 1 -- 172.21.15.16:0/2806614479 shutdown_connections 2023-09-30T22:36:07.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.634+0000 7f96bc51c700 1 -- 172.21.15.16:0/2806614479 wait complete. 2023-09-30T22:36:07.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.634+0000 7fdc96ffd700 1 -- 172.21.15.16:0/839831731 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdc98015020 con 0x7fdca809de50 2023-09-30T22:36:07.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.635+0000 7f96bc51c700 1 Processor -- start 2023-09-30T22:36:07.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.635+0000 7f96bc51c700 1 -- start start 2023-09-30T22:36:07.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.636+0000 7f96bc51c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96b4131810 0x7f96b406ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:07.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.636+0000 7f96bc51c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f96b4134630 0x7f96b4070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:07.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.636+0000 7f96bc51c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f96b4136f50 0x7f96b41c85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:07.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.636+0000 7f96bc51c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b411e870 con 0x7f96b4131810 2023-09-30T22:36:07.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.636+0000 7f96bc51c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96b411e010 con 0x7f96b4136f50 2023-09-30T22:36:07.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.636+0000 7f96bc51c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f96b406fad0 con 0x7f96b4134630 2023-09-30T22:36:07.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.636+0000 7f96b9ab7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f96b4134630 0x7f96b4070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:07.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.636+0000 7f96b9ab7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f96b4134630 0x7f96b4070470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:43776/0 (socket says 172.21.15.16:43776) 2023-09-30T22:36:07.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.636+0000 7f96b9ab7700 1 -- 172.21.15.16:0/3039171161 learned_addr learned my addr 172.21.15.16:0/3039171161 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:07.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.636+0000 7f96ba2b8700 1 --2- 172.21.15.16:0/3039171161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96b4131810 0x7f96b406ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:07.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.636+0000 7f96ba2b8700 1 -- 172.21.15.16:0/3039171161 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f96b4134630 msgr2=0x7f96b4070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:07.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.636+0000 7f96ba2b8700 1 --2- 172.21.15.16:0/3039171161 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f96b4134630 0x7f96b4070470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:07.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.637+0000 7f96ba2b8700 1 -- 172.21.15.16:0/3039171161 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f96b4136f50 msgr2=0x7f96b41c85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:07.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.637+0000 7f96ba2b8700 1 --2- 172.21.15.16:0/3039171161 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f96b4136f50 0x7f96b41c85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:07.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.637+0000 7f96ba2b8700 1 -- 172.21.15.16:0/3039171161 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96a40094a0 con 0x7f96b4131810 2023-09-30T22:36:07.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.637+0000 7f96b9ab7700 1 --2- 172.21.15.16:0/3039171161 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f96b4134630 0x7f96b4070470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:07.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.637+0000 7f96ba2b8700 1 --2- 172.21.15.16:0/3039171161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96b4131810 0x7f96b406ff30 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f96b0000f60 tx=0x7f96b000c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:07.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.637+0000 7f96aaffd700 1 -- 172.21.15.16:0/3039171161 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b00052d0 con 0x7f96b4131810 2023-09-30T22:36:07.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.637+0000 7f96aaffd700 1 -- 172.21.15.16:0/3039171161 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96b000ce90 con 0x7f96b4131810 2023-09-30T22:36:07.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.637+0000 7f96bc51c700 1 -- 172.21.15.16:0/3039171161 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96ac009b30 con 0x7f96b4131810 2023-09-30T22:36:07.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.637+0000 7f96aaffd700 1 -- 172.21.15.16:0/3039171161 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b0003680 con 0x7f96b4131810 2023-09-30T22:36:07.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.637+0000 7f96bc51c700 1 -- 172.21.15.16:0/3039171161 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96b40709b0 con 0x7f96b4131810 2023-09-30T22:36:07.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.638+0000 7f96aaffd700 1 -- 172.21.15.16:0/3039171161 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f96b00037e0 con 0x7f96b4131810 2023-09-30T22:36:07.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.638+0000 7f96aaffd700 1 -- 172.21.15.16:0/3039171161 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f96b003ea30 con 0x7f96b4131810 2023-09-30T22:36:07.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.638+0000 7f96bc51c700 1 -- 172.21.15.16:0/3039171161 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96b4074ea0 con 0x7f96b4131810 2023-09-30T22:36:07.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.643+0000 7f96aaffd700 1 -- 172.21.15.16:0/3039171161 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f96b0011590 con 0x7f96b4131810 2023-09-30T22:36:07.728 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:36:07.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.839+0000 7f999ae5c700 1 -- 172.21.15.16:0/138153605 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9994074ea0 con 0x7f99940a1340 2023-09-30T22:36:07.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.839+0000 7f99917fa700 1 -- 172.21.15.16:0/138153605 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f998001b3b0 con 0x7f99940a1340 2023-09-30T22:36:07.840 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:36:07.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.843+0000 7f999ae5c700 1 -- 172.21.15.16:0/138153605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99940a1340 msgr2=0x7f9994122bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:07.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.843+0000 7f999ae5c700 1 --2- 172.21.15.16:0/138153605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99940a1340 0x7f9994122bb0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f9980001eb0 tx=0x7f998000c8b0 comp rx=0 tx=0).stop 2023-09-30T22:36:07.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.844+0000 7f999ae5c700 1 -- 172.21.15.16:0/138153605 shutdown_connections 2023-09-30T22:36:07.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.844+0000 7f999ae5c700 1 --2- 172.21.15.16:0/138153605 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f99940a0a30 0x7f9994122670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:07.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.844+0000 7f999ae5c700 1 --2- 172.21.15.16:0/138153605 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9994134170 0x7f99941230f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:07.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.844+0000 7f999ae5c700 1 --2- 172.21.15.16:0/138153605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f99940a1340 0x7f9994122bb0 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:07.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.844+0000 7f999ae5c700 1 -- 172.21.15.16:0/138153605 >> 172.21.15.16:0/138153605 conn(0x7f99940a6550 msgr2=0x7f999409edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:07.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.844+0000 7f999ae5c700 1 -- 172.21.15.16:0/138153605 shutdown_connections 2023-09-30T22:36:07.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.844+0000 7f999ae5c700 1 -- 172.21.15.16:0/138153605 wait complete. 2023-09-30T22:36:07.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.857+0000 7f96bc51c700 1 -- 172.21.15.16:0/3039171161 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f96b4066980 con 0x7f96b4131810 2023-09-30T22:36:07.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.857+0000 7f96aaffd700 1 -- 172.21.15.16:0/3039171161 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f96b00192c0 con 0x7f96b4131810 2023-09-30T22:36:07.857 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:36:07.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.859+0000 7f96bc51c700 1 -- 172.21.15.16:0/3039171161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96b4131810 msgr2=0x7f96b406ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:07.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.859+0000 7f96bc51c700 1 --2- 172.21.15.16:0/3039171161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96b4131810 0x7f96b406ff30 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f96b0000f60 tx=0x7f96b000c6e0 comp rx=0 tx=0).stop 2023-09-30T22:36:07.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.859+0000 7f96bc51c700 1 -- 172.21.15.16:0/3039171161 shutdown_connections 2023-09-30T22:36:07.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.859+0000 7f96bc51c700 1 --2- 172.21.15.16:0/3039171161 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f96b4136f50 0x7f96b41c85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:07.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.859+0000 7f96bc51c700 1 --2- 172.21.15.16:0/3039171161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96b4131810 0x7f96b406ff30 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:07.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.859+0000 7f96bc51c700 1 --2- 172.21.15.16:0/3039171161 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f96b4134630 0x7f96b4070470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:07.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.859+0000 7f96bc51c700 1 -- 172.21.15.16:0/3039171161 >> 172.21.15.16:0/3039171161 conn(0x7f96b40a2aa0 msgr2=0x7f96b412f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:07.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.859+0000 7f96bc51c700 1 -- 172.21.15.16:0/3039171161 shutdown_connections 2023-09-30T22:36:07.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.859+0000 7f96bc51c700 1 -- 172.21.15.16:0/3039171161 wait complete. 2023-09-30T22:36:07.864 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:36:07.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.863+0000 7fdcace8a700 1 -- 172.21.15.16:0/839831731 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fdca806b9e0 con 0x7fdca809de50 2023-09-30T22:36:07.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.864+0000 7fdc96ffd700 1 -- 172.21.15.16:0/839831731 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdc9801a430 con 0x7fdca809de50 2023-09-30T22:36:07.864 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:36:07.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.866+0000 7fdcace8a700 1 -- 172.21.15.16:0/839831731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdca809de50 msgr2=0x7fdca811a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:07.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.866+0000 7fdcace8a700 1 --2- 172.21.15.16:0/839831731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdca809de50 0x7fdca811a650 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7fdc980090b0 tx=0x7fdc98004750 comp rx=0 tx=0).stop 2023-09-30T22:36:07.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.866+0000 7fdcace8a700 1 -- 172.21.15.16:0/839831731 shutdown_connections 2023-09-30T22:36:07.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.866+0000 7fdcace8a700 1 --2- 172.21.15.16:0/839831731 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdca81338a0 0x7fdca811b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:07.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.866+0000 7fdcace8a700 1 --2- 172.21.15.16:0/839831731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdca809de50 0x7fdca811a650 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:07.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.866+0000 7fdcace8a700 1 --2- 172.21.15.16:0/839831731 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdca809e760 0x7fdca811ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:07.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.866+0000 7fdcace8a700 1 -- 172.21.15.16:0/839831731 >> 172.21.15.16:0/839831731 conn(0x7fdca80a2aa0 msgr2=0x7fdca8130730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:07.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.867+0000 7fdcace8a700 1 -- 172.21.15.16:0/839831731 shutdown_connections 2023-09-30T22:36:07.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.867+0000 7fdcace8a700 1 -- 172.21.15.16:0/839831731 wait complete. 2023-09-30T22:36:07.875 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:36:07.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.878+0000 7f82ac169700 1 Processor -- start 2023-09-30T22:36:07.878 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.878+0000 7f82ac169700 1 -- start start 2023-09-30T22:36:07.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.879+0000 7f82ac169700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f82a41338c0 0x7f82a4133c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:07.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.879+0000 7f82ac169700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f82a41341d0 0x7f82a409de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:07.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.879+0000 7f82ac169700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f82a409e390 0x7f82a409e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:07.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.879+0000 7f82ac169700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82a4070c80 con 0x7f82a41341d0 2023-09-30T22:36:07.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.879+0000 7f82ac169700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82a409ed20 con 0x7f82a409e390 2023-09-30T22:36:07.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.879+0000 7f82ac169700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f82a406f720 con 0x7f82a41338c0 2023-09-30T22:36:07.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.879+0000 7f82a9f05700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f82a41338c0 0x7f82a4133c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:07.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.879+0000 7f82aa706700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f82a409e390 0x7f82a409e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:07.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.879+0000 7f82a9f05700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f82a41338c0 0x7f82a4133c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:43780/0 (socket says 172.21.15.16:43780) 2023-09-30T22:36:07.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.879+0000 7f82aa706700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f82a409e390 0x7f82a409e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:33170/0 (socket says 172.21.15.16:33170) 2023-09-30T22:36:07.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.879+0000 7f82a9f05700 1 -- 172.21.15.16:0/2176473453 learned_addr learned my addr 172.21.15.16:0/2176473453 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:07.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.879+0000 7f82a9704700 1 --2- 172.21.15.16:0/2176473453 >> v2:172.21.15.16:3300/0 conn(0x7f82a41341d0 0x7f82a409de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:07.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.879+0000 7f82aa706700 1 -- 172.21.15.16:0/2176473453 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f82a41338c0 msgr2=0x7f82a4133c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:07.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.879+0000 7f82aa706700 1 --2- 172.21.15.16:0/2176473453 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f82a41338c0 0x7f82a4133c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:07.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.879+0000 7f82aa706700 1 -- 172.21.15.16:0/2176473453 >> v2:172.21.15.16:3300/0 conn(0x7f82a41341d0 msgr2=0x7f82a409de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:07.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.879+0000 7f82aa706700 1 --2- 172.21.15.16:0/2176473453 >> v2:172.21.15.16:3300/0 conn(0x7f82a41341d0 0x7f82a409de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:07.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.879+0000 7f82aa706700 1 -- 172.21.15.16:0/2176473453 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a406fce0 con 0x7f82a409e390 2023-09-30T22:36:07.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.879+0000 7f82a9f05700 1 --2- 172.21.15.16:0/2176473453 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f82a41338c0 0x7f82a4133c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:07.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.879+0000 7f82a9704700 1 --2- 172.21.15.16:0/2176473453 >> v2:172.21.15.16:3300/0 conn(0x7f82a41341d0 0x7f82a409de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:07.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.879+0000 7f82aa706700 1 --2- 172.21.15.16:0/2176473453 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f82a409e390 0x7f82a409e7e0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f829c009dc0 tx=0x7f829c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a86451f802f4539b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:07.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.880+0000 7f82a8f03700 1 -- 172.21.15.16:0/2176473453 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829c00e070 con 0x7f82a409e390 2023-09-30T22:36:07.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.880+0000 7f82a8f03700 1 -- 172.21.15.16:0/2176473453 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f829c00bae0 con 0x7f82a409e390 2023-09-30T22:36:07.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.880+0000 7f82a8f03700 1 -- 172.21.15.16:0/2176473453 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829c00be90 con 0x7f82a409e390 2023-09-30T22:36:07.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.880+0000 7f82ac169700 1 -- 172.21.15.16:0/2176473453 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f82a409e390 msgr2=0x7f82a409e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:07.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.880+0000 7f82ac169700 1 --2- 172.21.15.16:0/2176473453 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f82a409e390 0x7f82a409e7e0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f829c009dc0 tx=0x7f829c00b330 comp rx=0 tx=0).stop 2023-09-30T22:36:07.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.880+0000 7f82ac169700 1 -- 172.21.15.16:0/2176473453 shutdown_connections 2023-09-30T22:36:07.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.880+0000 7f82ac169700 1 --2- 172.21.15.16:0/2176473453 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f82a409e390 0x7f82a409e7e0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:07.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.880+0000 7f82ac169700 1 --2- 172.21.15.16:0/2176473453 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f82a41338c0 0x7f82a4133c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:07.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.880+0000 7f82ac169700 1 --2- 172.21.15.16:0/2176473453 >> v2:172.21.15.16:3300/0 conn(0x7f82a41341d0 0x7f82a409de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:07.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.881+0000 7f82ac169700 1 -- 172.21.15.16:0/2176473453 >> 172.21.15.16:0/2176473453 conn(0x7f82a40a2aa0 msgr2=0x7f82a40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:07.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.881+0000 7f82ac169700 1 -- 172.21.15.16:0/2176473453 shutdown_connections 2023-09-30T22:36:07.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.881+0000 7f82ac169700 1 -- 172.21.15.16:0/2176473453 wait complete. 2023-09-30T22:36:07.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.882+0000 7f82ac169700 1 Processor -- start 2023-09-30T22:36:07.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.882+0000 7f82ac169700 1 -- start start 2023-09-30T22:36:07.886 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:36:07.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.882+0000 7f82ac169700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82a409e390 0x7f82a411a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:07.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.882+0000 7f82ac169700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f82a41338c0 0x7f82a411add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:07.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.882+0000 7f82ac169700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f82a41341d0 0x7f82a411b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:07.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.882+0000 7f82ac169700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a411ca30 con 0x7f82a409e390 2023-09-30T22:36:07.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.882+0000 7f82ac169700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82a412dbf0 con 0x7f82a41341d0 2023-09-30T22:36:07.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.882+0000 7f82ac169700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f82a406f960 con 0x7f82a41338c0 2023-09-30T22:36:07.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.882+0000 7f82a9704700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f82a41338c0 0x7f82a411add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:07.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.882+0000 7f82aa706700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f82a41341d0 0x7f82a411b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:07.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.882+0000 7f82a9f05700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82a409e390 0x7f82a411a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:07.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.882+0000 7f82a9704700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f82a41338c0 0x7f82a411add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:43784/0 (socket says 172.21.15.16:43784) 2023-09-30T22:36:07.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.882+0000 7f82a9704700 1 -- 172.21.15.16:0/3036035615 learned_addr learned my addr 172.21.15.16:0/3036035615 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:07.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.883+0000 7f82a9f05700 1 -- 172.21.15.16:0/3036035615 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f82a41338c0 msgr2=0x7f82a411add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:07.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.883+0000 7f82a9f05700 1 --2- 172.21.15.16:0/3036035615 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f82a41338c0 0x7f82a411add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:07.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.883+0000 7f82a9f05700 1 -- 172.21.15.16:0/3036035615 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f82a41341d0 msgr2=0x7f82a411b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:07.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.883+0000 7f82a9f05700 1 --2- 172.21.15.16:0/3036035615 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f82a41341d0 0x7f82a411b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:07.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.883+0000 7f82a9f05700 1 -- 172.21.15.16:0/3036035615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8294009550 con 0x7f82a409e390 2023-09-30T22:36:07.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.883+0000 7f82a9704700 1 --2- 172.21.15.16:0/3036035615 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f82a41338c0 0x7f82a411add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:07.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.883+0000 7f82aa706700 1 --2- 172.21.15.16:0/3036035615 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f82a41341d0 0x7f82a411b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:07.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.883+0000 7f82a9f05700 1 --2- 172.21.15.16:0/3036035615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82a409e390 0x7f82a411a890 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f829400e9a0 tx=0x7f829400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:07.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.883+0000 7f829a7fc700 1 -- 172.21.15.16:0/3036035615 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8294004030 con 0x7f82a409e390 2023-09-30T22:36:07.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.883+0000 7f829a7fc700 1 -- 172.21.15.16:0/3036035615 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8294004190 con 0x7f82a409e390 2023-09-30T22:36:07.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.883+0000 7f829a7fc700 1 -- 172.21.15.16:0/3036035615 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8294004520 con 0x7f82a409e390 2023-09-30T22:36:07.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.883+0000 7f82ac169700 1 -- 172.21.15.16:0/3036035615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82a00094a0 con 0x7f82a409e390 2023-09-30T22:36:07.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.883+0000 7f82ac169700 1 -- 172.21.15.16:0/3036035615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f829c009a70 con 0x7f82a409e390 2023-09-30T22:36:07.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.884+0000 7f829a7fc700 1 -- 172.21.15.16:0/3036035615 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f82940036a0 con 0x7f82a409e390 2023-09-30T22:36:07.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.884+0000 7f829a7fc700 1 -- 172.21.15.16:0/3036035615 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f829403f650 con 0x7f82a409e390 2023-09-30T22:36:07.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.884+0000 7f82ac169700 1 -- 172.21.15.16:0/3036035615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82a409ea40 con 0x7f82a409e390 2023-09-30T22:36:07.890 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:07.889+0000 7f829a7fc700 1 -- 172.21.15.16:0/3036035615 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8294019d60 con 0x7f82a409e390 2023-09-30T22:36:08.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:08.092+0000 7f82ac169700 1 -- 172.21.15.16:0/3036035615 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f82a4074ea0 con 0x7f82a409e390 2023-09-30T22:36:08.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:08.093+0000 7f829a7fc700 1 -- 172.21.15.16:0/3036035615 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8294019300 con 0x7f82a409e390 2023-09-30T22:36:08.093 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:36:08.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:08.095+0000 7f82ac169700 1 -- 172.21.15.16:0/3036035615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82a409e390 msgr2=0x7f82a411a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:08.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:08.095+0000 7f82ac169700 1 --2- 172.21.15.16:0/3036035615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82a409e390 0x7f82a411a890 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f829400e9a0 tx=0x7f829400ecb0 comp rx=0 tx=0).stop 2023-09-30T22:36:08.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:08.095+0000 7f82ac169700 1 -- 172.21.15.16:0/3036035615 shutdown_connections 2023-09-30T22:36:08.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:08.095+0000 7f82ac169700 1 --2- 172.21.15.16:0/3036035615 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f82a41341d0 0x7f82a411b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:08.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:08.095+0000 7f82ac169700 1 --2- 172.21.15.16:0/3036035615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82a409e390 0x7f82a411a890 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:08.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:08.095+0000 7f82ac169700 1 --2- 172.21.15.16:0/3036035615 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f82a41338c0 0x7f82a411add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:08.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:08.095+0000 7f82ac169700 1 -- 172.21.15.16:0/3036035615 >> 172.21.15.16:0/3036035615 conn(0x7f82a40a2aa0 msgr2=0x7f82a412f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:08.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:08.095+0000 7f82ac169700 1 -- 172.21.15.16:0/3036035615 shutdown_connections 2023-09-30T22:36:08.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:08.095+0000 7f82ac169700 1 -- 172.21.15.16:0/3036035615 wait complete. 2023-09-30T22:36:08.108 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:36:08.865 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:36:08.876 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:36:08.887 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:36:09.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.015+0000 7f995cdcf700 1 Processor -- start 2023-09-30T22:36:09.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.015+0000 7f995cdcf700 1 -- start start 2023-09-30T22:36:09.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.015+0000 7f995cdcf700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f99581338a0 0x7f9958133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:09.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.015+0000 7f995cdcf700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f995809e930 0x7f995813b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:09.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.015+0000 7f995cdcf700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f995806ef30 con 0x7f995809de50 2023-09-30T22:36:09.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.015+0000 7f995cdcf700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f995806fef0 con 0x7f995809e930 2023-09-30T22:36:09.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.015+0000 7f995cdcf700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f99581334b0 con 0x7f99581338a0 2023-09-30T22:36:09.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.016+0000 7f9956d9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f995809e930 0x7f995813b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:09.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.016+0000 7f995659c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f99581338a0 0x7f9958133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:09.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.016+0000 7f9956d9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f995809e930 0x7f995813b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:33184/0 (socket says 172.21.15.16:33184) 2023-09-30T22:36:09.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.016+0000 7f9955d9b700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f995809de50 0x7f995809e220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52162/0 (socket says 172.21.15.16:52162) 2023-09-30T22:36:09.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.016+0000 7f9956d9d700 1 -- 172.21.15.16:0/1462751580 learned_addr learned my addr 172.21.15.16:0/1462751580 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:09.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.016+0000 7f995659c700 1 -- 172.21.15.16:0/1462751580 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f995809e930 msgr2=0x7f995813b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:09.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.016+0000 7f995659c700 1 --2- 172.21.15.16:0/1462751580 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f995809e930 0x7f995813b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:09.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.016+0000 7f995659c700 1 -- 172.21.15.16:0/1462751580 >> v1:172.21.15.16:6789/0 conn(0x7f995809de50 legacy=0x7f995809e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:09.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.016+0000 7f995659c700 1 -- 172.21.15.16:0/1462751580 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f995806fc30 con 0x7f99581338a0 2023-09-30T22:36:09.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.016+0000 7f995559a700 1 -- 172.21.15.16:0/1462751580 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4027855267 0 0) 0x7f995806ef30 con 0x7f995809de50 2023-09-30T22:36:09.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.016+0000 7f9956d9d700 1 --2- 172.21.15.16:0/1462751580 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f995809e930 0x7f995813b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:09.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.016+0000 7f995659c700 1 --2- 172.21.15.16:0/1462751580 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f99581338a0 0x7f9958133c70 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f9948009870 tx=0x7f9948009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df812b08345f3fb5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:09.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.016+0000 7f995559a700 1 -- 172.21.15.16:0/1462751580 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9948011830 con 0x7f99581338a0 2023-09-30T22:36:09.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.016+0000 7f995559a700 1 -- 172.21.15.16:0/1462751580 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9948011990 con 0x7f99581338a0 2023-09-30T22:36:09.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.016+0000 7f995559a700 1 -- 172.21.15.16:0/1462751580 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9948011d20 con 0x7f99581338a0 2023-09-30T22:36:09.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.017+0000 7f995cdcf700 1 -- 172.21.15.16:0/1462751580 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f99581338a0 msgr2=0x7f9958133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:09.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.017+0000 7f995cdcf700 1 --2- 172.21.15.16:0/1462751580 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f99581338a0 0x7f9958133c70 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f9948009870 tx=0x7f9948009b80 comp rx=0 tx=0).stop 2023-09-30T22:36:09.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.017+0000 7f995cdcf700 1 -- 172.21.15.16:0/1462751580 shutdown_connections 2023-09-30T22:36:09.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.017+0000 7f995cdcf700 1 --2- 172.21.15.16:0/1462751580 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f995809e930 0x7f995813b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:09.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.017+0000 7f995cdcf700 1 --2- 172.21.15.16:0/1462751580 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f99581338a0 0x7f9958133c70 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:09.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.017+0000 7f995cdcf700 1 -- 172.21.15.16:0/1462751580 >> 172.21.15.16:0/1462751580 conn(0x7f99580a2aa0 msgr2=0x7f99580a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:09.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.017+0000 7f995cdcf700 1 -- 172.21.15.16:0/1462751580 shutdown_connections 2023-09-30T22:36:09.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.017+0000 7f995cdcf700 1 -- 172.21.15.16:0/1462751580 wait complete. 2023-09-30T22:36:09.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.019+0000 7f995cdcf700 1 Processor -- start 2023-09-30T22:36:09.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.019+0000 7f995cdcf700 1 -- start start 2023-09-30T22:36:09.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.019+0000 7f995cdcf700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f995809de50 0x7f995811b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:09.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.019+0000 7f995cdcf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f995809e930 0x7f995811bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:09.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.019+0000 7f995cdcf700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f99581338a0 0x7f995811c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:09.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.019+0000 7f995cdcf700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995811e520 con 0x7f995809e930 2023-09-30T22:36:09.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.019+0000 7f995cdcf700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f995811d8f0 con 0x7f995809de50 2023-09-30T22:36:09.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.019+0000 7f995cdcf700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f995806f640 con 0x7f99581338a0 2023-09-30T22:36:09.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.019+0000 7f9955d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f995809e930 0x7f995811bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:09.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.019+0000 7f995659c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f995809de50 0x7f995811b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:09.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.019+0000 7f9955d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f995809e930 0x7f995811bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51746/0 (socket says 172.21.15.16:51746) 2023-09-30T22:36:09.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.019+0000 7f9955d9b700 1 -- 172.21.15.16:0/1500197571 learned_addr learned my addr 172.21.15.16:0/1500197571 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:09.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.019+0000 7f9956d9d700 1 --2- 172.21.15.16:0/1500197571 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f99581338a0 0x7f995811c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:09.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.020+0000 7f9956d9d700 1 -- 172.21.15.16:0/1500197571 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f995809de50 msgr2=0x7f995811b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:09.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.020+0000 7f9956d9d700 1 --2- 172.21.15.16:0/1500197571 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f995809de50 0x7f995811b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:09.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.020+0000 7f9956d9d700 1 -- 172.21.15.16:0/1500197571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f995809e930 msgr2=0x7f995811bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:09.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.020+0000 7f9956d9d700 1 --2- 172.21.15.16:0/1500197571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f995809e930 0x7f995811bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:09.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.020+0000 7f9956d9d700 1 -- 172.21.15.16:0/1500197571 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9950009b30 con 0x7f99581338a0 2023-09-30T22:36:09.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.020+0000 7f995659c700 1 --2- 172.21.15.16:0/1500197571 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f995809de50 0x7f995811b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:09.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.020+0000 7f9955d9b700 1 --2- 172.21.15.16:0/1500197571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f995809e930 0x7f995811bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:09.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.020+0000 7f9956d9d700 1 --2- 172.21.15.16:0/1500197571 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f99581338a0 0x7f995811c310 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f9950009e50 tx=0x7f995000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:09.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.020+0000 7f9946ffd700 1 -- 172.21.15.16:0/1500197571 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9950018070 con 0x7f99581338a0 2023-09-30T22:36:09.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.020+0000 7f9946ffd700 1 -- 172.21.15.16:0/1500197571 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f995000f040 con 0x7f99581338a0 2023-09-30T22:36:09.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.020+0000 7f995cdcf700 1 -- 172.21.15.16:0/1500197571 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99480094a0 con 0x7f99581338a0 2023-09-30T22:36:09.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.020+0000 7f9946ffd700 1 -- 172.21.15.16:0/1500197571 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9950003c40 con 0x7f99581338a0 2023-09-30T22:36:09.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.020+0000 7f995cdcf700 1 -- 172.21.15.16:0/1500197571 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f995811e970 con 0x7f99581338a0 2023-09-30T22:36:09.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.021+0000 7f995cdcf700 1 -- 172.21.15.16:0/1500197571 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99581babb0 con 0x7f99581338a0 2023-09-30T22:36:09.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.022+0000 7f9946ffd700 1 -- 172.21.15.16:0/1500197571 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f995000c880 con 0x7f99581338a0 2023-09-30T22:36:09.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.022+0000 7f9946ffd700 1 -- 172.21.15.16:0/1500197571 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f99500483f0 con 0x7f99581338a0 2023-09-30T22:36:09.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.027+0000 7f9946ffd700 1 -- 172.21.15.16:0/1500197571 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f995001cd90 con 0x7f99581338a0 2023-09-30T22:36:09.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.054+0000 7f1b7ecd6700 1 Processor -- start 2023-09-30T22:36:09.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.054+0000 7f1b7ecd6700 1 -- start start 2023-09-30T22:36:09.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.054+0000 7f1b7ecd6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1b7809e930 0x7f1b7809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:09.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.054+0000 7f1b7ecd6700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1b7809f240 0x7f1b781317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:09.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.054+0000 7f1b7ecd6700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b78070940 con 0x7f1b7809f650 2023-09-30T22:36:09.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.054+0000 7f1b7ecd6700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b7806fd60 con 0x7f1b7809f240 2023-09-30T22:36:09.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.054+0000 7f1b7ecd6700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7806fa80 con 0x7f1b7809e930 2023-09-30T22:36:09.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.054+0000 7f1b7ca72700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1b7809e930 0x7f1b7809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:09.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.054+0000 7f1b7ca72700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1b7809e930 0x7f1b7809ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:33206/0 (socket says 172.21.15.16:33206) 2023-09-30T22:36:09.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.054+0000 7f1b7ca72700 1 -- 172.21.15.16:0/2678097281 learned_addr learned my addr 172.21.15.16:0/2678097281 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:09.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.055+0000 7f1b77fff700 1 --2- 172.21.15.16:0/2678097281 >> v2:172.21.15.16:3300/0 conn(0x7f1b7809f240 0x7f1b781317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:09.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.055+0000 7f1b7ca72700 1 -- 172.21.15.16:0/2678097281 >> v2:172.21.15.16:3300/0 conn(0x7f1b7809f240 msgr2=0x7f1b781317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:09.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.055+0000 7f1b7ca72700 1 --2- 172.21.15.16:0/2678097281 >> v2:172.21.15.16:3300/0 conn(0x7f1b7809f240 0x7f1b781317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:09.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.055+0000 7f1b7ca72700 1 -- 172.21.15.16:0/2678097281 >> v1:172.21.15.16:6789/0 conn(0x7f1b7809f650 legacy=0x7f1b78131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:09.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.055+0000 7f1b777fe700 1 -- 172.21.15.16:0/2678097281 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1171291002 0 0) 0x7f1b78070940 con 0x7f1b7809f650 2023-09-30T22:36:09.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.055+0000 7f1b7ca72700 1 -- 172.21.15.16:0/2678097281 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b7809fb60 con 0x7f1b7809e930 2023-09-30T22:36:09.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.055+0000 7f1b7ca72700 1 --2- 172.21.15.16:0/2678097281 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1b7809e930 0x7f1b7809ed00 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f1b680098f0 tx=0x7f1b68009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c29fc72e05399d9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:09.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.055+0000 7f1b777fe700 1 -- 172.21.15.16:0/2678097281 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b68011830 con 0x7f1b7809e930 2023-09-30T22:36:09.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.055+0000 7f1b777fe700 1 -- 172.21.15.16:0/2678097281 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b68011990 con 0x7f1b7809e930 2023-09-30T22:36:09.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.055+0000 7f1b777fe700 1 -- 172.21.15.16:0/2678097281 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b68011d20 con 0x7f1b7809e930 2023-09-30T22:36:09.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.056+0000 7f1b7ecd6700 1 -- 172.21.15.16:0/2678097281 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1b7809e930 msgr2=0x7f1b7809ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:09.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.056+0000 7f1b7ecd6700 1 --2- 172.21.15.16:0/2678097281 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1b7809e930 0x7f1b7809ed00 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f1b680098f0 tx=0x7f1b68009c00 comp rx=0 tx=0).stop 2023-09-30T22:36:09.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.056+0000 7f1b7ecd6700 1 -- 172.21.15.16:0/2678097281 shutdown_connections 2023-09-30T22:36:09.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.056+0000 7f1b7ecd6700 1 --2- 172.21.15.16:0/2678097281 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1b7809e930 0x7f1b7809ed00 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:09.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.056+0000 7f1b7ecd6700 1 --2- 172.21.15.16:0/2678097281 >> v2:172.21.15.16:3300/0 conn(0x7f1b7809f240 0x7f1b781317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:09.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.056+0000 7f1b7ecd6700 1 -- 172.21.15.16:0/2678097281 >> 172.21.15.16:0/2678097281 conn(0x7f1b780a4210 msgr2=0x7f1b780a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:09.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.057+0000 7f1b7ecd6700 1 -- 172.21.15.16:0/2678097281 shutdown_connections 2023-09-30T22:36:09.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.057+0000 7f1b7ecd6700 1 -- 172.21.15.16:0/2678097281 wait complete. 2023-09-30T22:36:09.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.058+0000 7f1b7ecd6700 1 Processor -- start 2023-09-30T22:36:09.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.058+0000 7f1b7ecd6700 1 -- start start 2023-09-30T22:36:09.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.058+0000 7f1b7ecd6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1b7809e930 0x7f1b7806b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:09.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.058+0000 7f1b7ecd6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1b7809f240 0x7f1b7806bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:09.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.058+0000 7f1b7ecd6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b7809f650 0x7f1b7806c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:09.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.058+0000 7f1b7ecd6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b781a8800 con 0x7f1b7809f650 2023-09-30T22:36:09.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.058+0000 7f1b7ecd6700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b781a9810 con 0x7f1b7809e930 2023-09-30T22:36:09.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.058+0000 7f1b7ecd6700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1b781a9300 con 0x7f1b7809f240 2023-09-30T22:36:09.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.059+0000 7f1b77fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1b7809f240 0x7f1b7806bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:09.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.059+0000 7f1b77fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1b7809f240 0x7f1b7806bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:43806/0 (socket says 172.21.15.16:43806) 2023-09-30T22:36:09.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.059+0000 7f1b77fff700 1 -- 172.21.15.16:0/3367230513 learned_addr learned my addr 172.21.15.16:0/3367230513 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:09.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.059+0000 7f1b7d273700 1 --2- 172.21.15.16:0/3367230513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b7809f650 0x7f1b7806c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:09.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.059+0000 7f1b77fff700 1 -- 172.21.15.16:0/3367230513 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1b7809e930 msgr2=0x7f1b7806b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:09.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.059+0000 7f1b77fff700 1 --2- 172.21.15.16:0/3367230513 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1b7809e930 0x7f1b7806b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:09.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.059+0000 7f1b77fff700 1 -- 172.21.15.16:0/3367230513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b7809f650 msgr2=0x7f1b7806c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:09.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.059+0000 7f1b77fff700 1 --2- 172.21.15.16:0/3367230513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b7809f650 0x7f1b7806c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:09.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.059+0000 7f1b77fff700 1 -- 172.21.15.16:0/3367230513 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b68009550 con 0x7f1b7809f240 2023-09-30T22:36:09.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.059+0000 7f1b7d273700 1 --2- 172.21.15.16:0/3367230513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b7809f650 0x7f1b7806c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:09.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.059+0000 7f1b77fff700 1 --2- 172.21.15.16:0/3367230513 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1b7809f240 0x7f1b7806bdc0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c00e9b0 tx=0x7f1b6c00ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:09.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.059+0000 7f1b757fa700 1 -- 172.21.15.16:0/3367230513 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c009710 con 0x7f1b7809f240 2023-09-30T22:36:09.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.059+0000 7f1b7ecd6700 1 -- 172.21.15.16:0/3367230513 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b780a0190 con 0x7f1b7809f240 2023-09-30T22:36:09.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.059+0000 7f1b757fa700 1 -- 172.21.15.16:0/3367230513 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b6c004020 con 0x7f1b7809f240 2023-09-30T22:36:09.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.059+0000 7f1b7ecd6700 1 -- 172.21.15.16:0/3367230513 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b78071d20 con 0x7f1b7809f240 2023-09-30T22:36:09.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.060+0000 7f1b757fa700 1 -- 172.21.15.16:0/3367230513 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c004390 con 0x7f1b7809f240 2023-09-30T22:36:09.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.061+0000 7f1b7ecd6700 1 -- 172.21.15.16:0/3367230513 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b580052e0 con 0x7f1b7809f240 2023-09-30T22:36:09.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.061+0000 7f1b757fa700 1 -- 172.21.15.16:0/3367230513 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1b6c00c850 con 0x7f1b7809f240 2023-09-30T22:36:09.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.061+0000 7f1b757fa700 1 -- 172.21.15.16:0/3367230513 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f1b6c03f9d0 con 0x7f1b7809f240 2023-09-30T22:36:09.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.069+0000 7f1b757fa700 1 -- 172.21.15.16:0/3367230513 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1b6c01b3b0 con 0x7f1b7809f240 2023-09-30T22:36:09.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.070+0000 7ffa5a9c2700 1 Processor -- start 2023-09-30T22:36:09.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.070+0000 7ffa5a9c2700 1 -- start start 2023-09-30T22:36:09.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.070+0000 7ffa5a9c2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa5409d6f0 0x7ffa5409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:09.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.070+0000 7ffa5a9c2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa5409e000 0x7ffa541339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:09.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.070+0000 7ffa5a9c2700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ffa5409e410 0x7ffa54133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:09.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.070+0000 7ffa5a9c2700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa5406b460 con 0x7ffa5409e410 2023-09-30T22:36:09.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.070+0000 7ffa5a9c2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5409b120 con 0x7ffa5409e000 2023-09-30T22:36:09.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.070+0000 7ffa5a9c2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5409a7e0 con 0x7ffa5409d6f0 2023-09-30T22:36:09.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.071+0000 7ffa53fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa5409d6f0 0x7ffa5409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:09.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.071+0000 7ffa53fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa5409d6f0 0x7ffa5409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:43816/0 (socket says 172.21.15.16:43816) 2023-09-30T22:36:09.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.071+0000 7ffa53fff700 1 -- 172.21.15.16:0/1932215637 learned_addr learned my addr 172.21.15.16:0/1932215637 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:09.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.071+0000 7ffa537fe700 1 --2- 172.21.15.16:0/1932215637 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa5409e000 0x7ffa541339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:09.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.071+0000 7ffa58f5f700 1 --2- 172.21.15.16:0/1932215637 >> v2:172.21.15.16:3300/0 conn(0x7ffa5409e410 0x7ffa54133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:09.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.071+0000 7ffa53fff700 1 -- 172.21.15.16:0/1932215637 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa5409e000 msgr2=0x7ffa541339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:09.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.071+0000 7ffa53fff700 1 --2- 172.21.15.16:0/1932215637 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa5409e000 0x7ffa541339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:09.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.071+0000 7ffa53fff700 1 -- 172.21.15.16:0/1932215637 >> v2:172.21.15.16:3300/0 conn(0x7ffa5409e410 msgr2=0x7ffa54133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:09.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.071+0000 7ffa53fff700 1 --2- 172.21.15.16:0/1932215637 >> v2:172.21.15.16:3300/0 conn(0x7ffa5409e410 0x7ffa54133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:09.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.071+0000 7ffa53fff700 1 -- 172.21.15.16:0/1932215637 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa5409a500 con 0x7ffa5409d6f0 2023-09-30T22:36:09.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.071+0000 7ffa537fe700 1 --2- 172.21.15.16:0/1932215637 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa5409e000 0x7ffa541339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:09.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.071+0000 7ffa58f5f700 1 --2- 172.21.15.16:0/1932215637 >> v2:172.21.15.16:3300/0 conn(0x7ffa5409e410 0x7ffa54133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:09.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.071+0000 7ffa53fff700 1 --2- 172.21.15.16:0/1932215637 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa5409d6f0 0x7ffa5409dac0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7ffa44009920 tx=0x7ffa44009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a38246c5ede23a7f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:09.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.072+0000 7ffa52ffd700 1 -- 172.21.15.16:0/1932215637 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa44011830 con 0x7ffa5409d6f0 2023-09-30T22:36:09.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.072+0000 7ffa52ffd700 1 -- 172.21.15.16:0/1932215637 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa44011990 con 0x7ffa5409d6f0 2023-09-30T22:36:09.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.072+0000 7ffa52ffd700 1 -- 172.21.15.16:0/1932215637 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa44011d20 con 0x7ffa5409d6f0 2023-09-30T22:36:09.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.072+0000 7ffa5a9c2700 1 -- 172.21.15.16:0/1932215637 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa5409d6f0 msgr2=0x7ffa5409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:09.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.072+0000 7ffa5a9c2700 1 --2- 172.21.15.16:0/1932215637 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa5409d6f0 0x7ffa5409dac0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7ffa44009920 tx=0x7ffa44009c30 comp rx=0 tx=0).stop 2023-09-30T22:36:09.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.072+0000 7ffa5a9c2700 1 -- 172.21.15.16:0/1932215637 shutdown_connections 2023-09-30T22:36:09.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.072+0000 7ffa5a9c2700 1 --2- 172.21.15.16:0/1932215637 >> v2:172.21.15.16:3300/0 conn(0x7ffa5409e410 0x7ffa54133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:09.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.072+0000 7ffa5a9c2700 1 --2- 172.21.15.16:0/1932215637 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa5409d6f0 0x7ffa5409dac0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:09.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.072+0000 7ffa5a9c2700 1 --2- 172.21.15.16:0/1932215637 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa5409e000 0x7ffa541339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:09.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.072+0000 7ffa5a9c2700 1 -- 172.21.15.16:0/1932215637 >> 172.21.15.16:0/1932215637 conn(0x7ffa54066f50 msgr2=0x7ffa5409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:09.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.073+0000 7ffa5a9c2700 1 -- 172.21.15.16:0/1932215637 shutdown_connections 2023-09-30T22:36:09.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.073+0000 7ffa5a9c2700 1 -- 172.21.15.16:0/1932215637 wait complete. 2023-09-30T22:36:09.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.074+0000 7ffa5a9c2700 1 Processor -- start 2023-09-30T22:36:09.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.074+0000 7ffa5a9c2700 1 -- start start 2023-09-30T22:36:09.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.075+0000 7ffa5a9c2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa5409d6f0 0x7ffa541c8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:09.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.075+0000 7ffa5a9c2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa5409e000 0x7ffa541c8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:09.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.075+0000 7ffa5a9c2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa5409e410 0x7ffa541c8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:09.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.075+0000 7ffa5a9c2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa541a81b0 con 0x7ffa5409e000 2023-09-30T22:36:09.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.075+0000 7ffa5a9c2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa541a64f0 con 0x7ffa5409e410 2023-09-30T22:36:09.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.075+0000 7ffa5a9c2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffa540a30d0 con 0x7ffa5409d6f0 2023-09-30T22:36:09.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.075+0000 7ffa53fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa5409d6f0 0x7ffa541c8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:09.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.075+0000 7ffa53fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa5409d6f0 0x7ffa541c8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:43828/0 (socket says 172.21.15.16:43828) 2023-09-30T22:36:09.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.075+0000 7ffa53fff700 1 -- 172.21.15.16:0/788538518 learned_addr learned my addr 172.21.15.16:0/788538518 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:09.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.075+0000 7ffa58f5f700 1 --2- 172.21.15.16:0/788538518 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa5409e410 0x7ffa541c8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:09.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.075+0000 7ffa58f5f700 1 -- 172.21.15.16:0/788538518 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa5409d6f0 msgr2=0x7ffa541c8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:09.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.075+0000 7ffa58f5f700 1 --2- 172.21.15.16:0/788538518 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa5409d6f0 0x7ffa541c8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:09.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.075+0000 7ffa58f5f700 1 -- 172.21.15.16:0/788538518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa5409e000 msgr2=0x7ffa541c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:09.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.075+0000 7ffa58f5f700 1 --2- 172.21.15.16:0/788538518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa5409e000 0x7ffa541c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:09.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.075+0000 7ffa58f5f700 1 -- 172.21.15.16:0/788538518 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa480094a0 con 0x7ffa5409e410 2023-09-30T22:36:09.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.075+0000 7ffa53fff700 1 --2- 172.21.15.16:0/788538518 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa5409d6f0 0x7ffa541c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:09.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.076+0000 7ffa58f5f700 1 --2- 172.21.15.16:0/788538518 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa5409e410 0x7ffa541c8ba0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c00ef90 tx=0x7ffa4c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:09.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.076+0000 7ffa50ff9700 1 -- 172.21.15.16:0/788538518 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c00cd40 con 0x7ffa5409e410 2023-09-30T22:36:09.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.076+0000 7ffa50ff9700 1 -- 172.21.15.16:0/788538518 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa4c00f040 con 0x7ffa5409e410 2023-09-30T22:36:09.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.076+0000 7ffa5a9c2700 1 -- 172.21.15.16:0/788538518 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa4c009a70 con 0x7ffa5409e410 2023-09-30T22:36:09.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.076+0000 7ffa50ff9700 1 -- 172.21.15.16:0/788538518 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c0146c0 con 0x7ffa5409e410 2023-09-30T22:36:09.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.076+0000 7ffa5a9c2700 1 -- 172.21.15.16:0/788538518 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa44009550 con 0x7ffa5409e410 2023-09-30T22:36:09.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.077+0000 7ffa5a9c2700 1 -- 172.21.15.16:0/788538518 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa54074ea0 con 0x7ffa5409e410 2023-09-30T22:36:09.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.077+0000 7ffa50ff9700 1 -- 172.21.15.16:0/788538518 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ffa4c014820 con 0x7ffa5409e410 2023-09-30T22:36:09.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.077+0000 7ffa50ff9700 1 -- 172.21.15.16:0/788538518 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7ffa4c0404c0 con 0x7ffa5409e410 2023-09-30T22:36:09.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.082+0000 7ffa50ff9700 1 -- 172.21.15.16:0/788538518 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ffa4c012da0 con 0x7ffa5409e410 2023-09-30T22:36:09.109 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:36:09.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.250+0000 7f4179bc2700 1 Processor -- start 2023-09-30T22:36:09.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.250+0000 7f4179bc2700 1 -- start start 2023-09-30T22:36:09.252 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.251+0000 7f4179bc2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f417409e930 0x7f417409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:09.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.251+0000 7f4179bc2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f417409f240 0x7f41741317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:09.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.251+0000 7f4179bc2700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f417409f650 0x7f4174131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:09.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.251+0000 7f4179bc2700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4174070940 con 0x7f417409f650 2023-09-30T22:36:09.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.251+0000 7f4179bc2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f417406fd60 con 0x7f417409f240 2023-09-30T22:36:09.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.251+0000 7f4179bc2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f417406fa80 con 0x7f417409e930 2023-09-30T22:36:09.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.251+0000 7f41737fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f417409e930 0x7f417409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:09.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.251+0000 7f4173fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f417409f650 0x7f4174131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:09.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.251+0000 7f41737fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f417409e930 0x7f417409ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:43842/0 (socket says 172.21.15.16:43842) 2023-09-30T22:36:09.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.251+0000 7f41737fe700 1 -- 172.21.15.16:0/1591415517 learned_addr learned my addr 172.21.15.16:0/1591415517 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:09.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.251+0000 7f4172ffd700 1 --2- 172.21.15.16:0/1591415517 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f417409f240 0x7f41741317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:09.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.252+0000 7f4172ffd700 1 -- 172.21.15.16:0/1591415517 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f417409e930 msgr2=0x7f417409ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:09.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.252+0000 7f4172ffd700 1 --2- 172.21.15.16:0/1591415517 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f417409e930 0x7f417409ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:09.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.252+0000 7f4172ffd700 1 -- 172.21.15.16:0/1591415517 >> v2:172.21.15.16:3300/0 conn(0x7f417409f650 msgr2=0x7f4174131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:09.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.252+0000 7f4172ffd700 1 --2- 172.21.15.16:0/1591415517 >> v2:172.21.15.16:3300/0 conn(0x7f417409f650 0x7f4174131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:09.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.252+0000 7f4172ffd700 1 -- 172.21.15.16:0/1591415517 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f417409fb60 con 0x7f417409f240 2023-09-30T22:36:09.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.252+0000 7f4173fff700 1 --2- 172.21.15.16:0/1591415517 >> v2:172.21.15.16:3300/0 conn(0x7f417409f650 0x7f4174131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:09.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.252+0000 7f41737fe700 1 --2- 172.21.15.16:0/1591415517 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f417409e930 0x7f417409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:09.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.252+0000 7f4172ffd700 1 --2- 172.21.15.16:0/1591415517 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f417409f240 0x7f41741317f0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f4168009840 tx=0x7f4168009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7d939994d4844f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:09.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.252+0000 7f41727fc700 1 -- 172.21.15.16:0/1591415517 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4168011830 con 0x7f417409f240 2023-09-30T22:36:09.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.252+0000 7f41727fc700 1 -- 172.21.15.16:0/1591415517 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4168011990 con 0x7f417409f240 2023-09-30T22:36:09.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.252+0000 7f41727fc700 1 -- 172.21.15.16:0/1591415517 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4168011d20 con 0x7f417409f240 2023-09-30T22:36:09.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.253+0000 7f4179bc2700 1 -- 172.21.15.16:0/1591415517 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f417409f240 msgr2=0x7f41741317f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:09.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.253+0000 7f4179bc2700 1 --2- 172.21.15.16:0/1591415517 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f417409f240 0x7f41741317f0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f4168009840 tx=0x7f4168009b50 comp rx=0 tx=0).stop 2023-09-30T22:36:09.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.253+0000 7f4179bc2700 1 -- 172.21.15.16:0/1591415517 shutdown_connections 2023-09-30T22:36:09.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.253+0000 7f4179bc2700 1 --2- 172.21.15.16:0/1591415517 >> v2:172.21.15.16:3300/0 conn(0x7f417409f650 0x7f4174131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:09.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.253+0000 7f4179bc2700 1 --2- 172.21.15.16:0/1591415517 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f417409e930 0x7f417409ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:09.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.253+0000 7f4179bc2700 1 --2- 172.21.15.16:0/1591415517 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f417409f240 0x7f41741317f0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:09.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.253+0000 7f4179bc2700 1 -- 172.21.15.16:0/1591415517 >> 172.21.15.16:0/1591415517 conn(0x7f41740a4210 msgr2=0x7f41740a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:09.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.253+0000 7f4179bc2700 1 -- 172.21.15.16:0/1591415517 shutdown_connections 2023-09-30T22:36:09.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.253+0000 7f4179bc2700 1 -- 172.21.15.16:0/1591415517 wait complete. 2023-09-30T22:36:09.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.254+0000 7f4179bc2700 1 Processor -- start 2023-09-30T22:36:09.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.255+0000 7f4179bc2700 1 -- start start 2023-09-30T22:36:09.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.255+0000 7f4179bc2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f417409e930 0x7f417406b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:09.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.255+0000 7f4179bc2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f417409f240 0x7f417406bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:09.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.255+0000 7f4179bc2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f417409f650 0x7f417406c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:09.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.255+0000 7f4179bc2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41741a7960 con 0x7f417409f650 2023-09-30T22:36:09.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.255+0000 7f4179bc2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f417406d530 con 0x7f417409f240 2023-09-30T22:36:09.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.255+0000 7f4179bc2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f41740a3580 con 0x7f417409e930 2023-09-30T22:36:09.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.256+0000 7f4173fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f417409f650 0x7f417406c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:09.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.256+0000 7f4173fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f417409f650 0x7f417406c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51792/0 (socket says 172.21.15.16:51792) 2023-09-30T22:36:09.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.256+0000 7f4173fff700 1 -- 172.21.15.16:0/2437137422 learned_addr learned my addr 172.21.15.16:0/2437137422 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:09.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.256+0000 7f41737fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f417409e930 0x7f417406b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:09.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.256+0000 7f4172ffd700 1 --2- 172.21.15.16:0/2437137422 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f417409f240 0x7f417406bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:09.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.256+0000 7f4172ffd700 1 -- 172.21.15.16:0/2437137422 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f417409e930 msgr2=0x7f417406b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:09.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.256+0000 7f4172ffd700 1 --2- 172.21.15.16:0/2437137422 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f417409e930 0x7f417406b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:09.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.256+0000 7f4172ffd700 1 -- 172.21.15.16:0/2437137422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f417409f650 msgr2=0x7f417406c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:09.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.256+0000 7f4172ffd700 1 --2- 172.21.15.16:0/2437137422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f417409f650 0x7f417406c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:09.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.256+0000 7f4172ffd700 1 -- 172.21.15.16:0/2437137422 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4164009550 con 0x7f417409f240 2023-09-30T22:36:09.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.256+0000 7f41737fe700 1 --2- 172.21.15.16:0/2437137422 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f417409e930 0x7f417406b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:09.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.256+0000 7f4173fff700 1 --2- 172.21.15.16:0/2437137422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f417409f650 0x7f417406c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:09.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.256+0000 7f4172ffd700 1 --2- 172.21.15.16:0/2437137422 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f417409f240 0x7f417406bdc0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f4168016000 tx=0x7f4168004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:09.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.257+0000 7f415bfff700 1 -- 172.21.15.16:0/2437137422 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4168004450 con 0x7f417409f240 2023-09-30T22:36:09.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.257+0000 7f415bfff700 1 -- 172.21.15.16:0/2437137422 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f416801f070 con 0x7f417409f240 2023-09-30T22:36:09.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.257+0000 7f415bfff700 1 -- 172.21.15.16:0/2437137422 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41680038e0 con 0x7f417409f240 2023-09-30T22:36:09.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.257+0000 7f4179bc2700 1 -- 172.21.15.16:0/2437137422 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f416c009a70 con 0x7f417409f240 2023-09-30T22:36:09.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.257+0000 7f4179bc2700 1 -- 172.21.15.16:0/2437137422 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41680094a0 con 0x7f417409f240 2023-09-30T22:36:09.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.258+0000 7f415bfff700 1 -- 172.21.15.16:0/2437137422 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4168003a40 con 0x7f417409f240 2023-09-30T22:36:09.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.258+0000 7f415bfff700 1 -- 172.21.15.16:0/2437137422 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f4168021070 con 0x7f417409f240 2023-09-30T22:36:09.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.258+0000 7f4179bc2700 1 -- 172.21.15.16:0/2437137422 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4174074ea0 con 0x7f417409f240 2023-09-30T22:36:09.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.259+0000 7f995cdcf700 1 -- 172.21.15.16:0/1500197571 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9958074ea0 con 0x7f99581338a0 2023-09-30T22:36:09.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.259+0000 7f9946ffd700 1 -- 172.21.15.16:0/1500197571 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f995001de00 con 0x7f99581338a0 2023-09-30T22:36:09.265 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:36:09.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.263+0000 7f995cdcf700 1 -- 172.21.15.16:0/1500197571 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f99581338a0 msgr2=0x7f995811c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:09.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.263+0000 7f995cdcf700 1 --2- 172.21.15.16:0/1500197571 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f99581338a0 0x7f995811c310 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f9950009e50 tx=0x7f995000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:36:09.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.263+0000 7f995cdcf700 1 -- 172.21.15.16:0/1500197571 shutdown_connections 2023-09-30T22:36:09.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.263+0000 7f995cdcf700 1 --2- 172.21.15.16:0/1500197571 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f995809de50 0x7f995811b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:09.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.263+0000 7f995cdcf700 1 --2- 172.21.15.16:0/1500197571 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f99581338a0 0x7f995811c310 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:09.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.263+0000 7f995cdcf700 1 --2- 172.21.15.16:0/1500197571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f995809e930 0x7f995811bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:09.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.263+0000 7f995cdcf700 1 -- 172.21.15.16:0/1500197571 >> 172.21.15.16:0/1500197571 conn(0x7f99580a2aa0 msgr2=0x7f99581306c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:09.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.263+0000 7f995cdcf700 1 -- 172.21.15.16:0/1500197571 shutdown_connections 2023-09-30T22:36:09.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.263+0000 7f995cdcf700 1 -- 172.21.15.16:0/1500197571 wait complete. 2023-09-30T22:36:09.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.264+0000 7f415bfff700 1 -- 172.21.15.16:0/2437137422 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f416802b020 con 0x7f417409f240 2023-09-30T22:36:09.278 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:36:09.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.286+0000 7f1b7ecd6700 1 -- 172.21.15.16:0/3367230513 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1b58005590 con 0x7f1b7809f240 2023-09-30T22:36:09.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.286+0000 7f1b757fa700 1 -- 172.21.15.16:0/3367230513 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1b6c01ab50 con 0x7f1b7809f240 2023-09-30T22:36:09.287 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:36:09.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.289+0000 7f1b7ecd6700 1 -- 172.21.15.16:0/3367230513 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1b7809f240 msgr2=0x7f1b7806bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:09.289 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.289+0000 7f1b7ecd6700 1 --2- 172.21.15.16:0/3367230513 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1b7809f240 0x7f1b7806bdc0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c00e9b0 tx=0x7f1b6c00ecc0 comp rx=0 tx=0).stop 2023-09-30T22:36:09.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.289+0000 7f1b7ecd6700 1 -- 172.21.15.16:0/3367230513 shutdown_connections 2023-09-30T22:36:09.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.289+0000 7f1b7ecd6700 1 --2- 172.21.15.16:0/3367230513 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1b7809e930 0x7f1b7806b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:09.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.289+0000 7f1b7ecd6700 1 --2- 172.21.15.16:0/3367230513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b7809f650 0x7f1b7806c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:09.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.289+0000 7f1b7ecd6700 1 --2- 172.21.15.16:0/3367230513 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1b7809f240 0x7f1b7806bdc0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:09.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.289+0000 7f1b7ecd6700 1 -- 172.21.15.16:0/3367230513 >> 172.21.15.16:0/3367230513 conn(0x7f1b780a4210 msgr2=0x7f1b781b8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:09.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.289+0000 7f1b7ecd6700 1 -- 172.21.15.16:0/3367230513 shutdown_connections 2023-09-30T22:36:09.290 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.289+0000 7f1b7ecd6700 1 -- 172.21.15.16:0/3367230513 wait complete. 2023-09-30T22:36:09.303 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:36:09.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.328+0000 7ffa5a9c2700 1 -- 172.21.15.16:0/788538518 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ffa541a7da0 con 0x7ffa5409e410 2023-09-30T22:36:09.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.329+0000 7ffa50ff9700 1 -- 172.21.15.16:0/788538518 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ffa4c018030 con 0x7ffa5409e410 2023-09-30T22:36:09.329 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:36:09.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.331+0000 7ffa5a9c2700 1 -- 172.21.15.16:0/788538518 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa5409e410 msgr2=0x7ffa541c8ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:09.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.331+0000 7ffa5a9c2700 1 --2- 172.21.15.16:0/788538518 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa5409e410 0x7ffa541c8ba0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c00ef90 tx=0x7ffa4c00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:36:09.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.331+0000 7ffa5a9c2700 1 -- 172.21.15.16:0/788538518 shutdown_connections 2023-09-30T22:36:09.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.331+0000 7ffa5a9c2700 1 --2- 172.21.15.16:0/788538518 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffa5409e410 0x7ffa541c8ba0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:09.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.331+0000 7ffa5a9c2700 1 --2- 172.21.15.16:0/788538518 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffa5409d6f0 0x7ffa541c8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:09.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.331+0000 7ffa5a9c2700 1 --2- 172.21.15.16:0/788538518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa5409e000 0x7ffa541c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:09.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.331+0000 7ffa5a9c2700 1 -- 172.21.15.16:0/788538518 >> 172.21.15.16:0/788538518 conn(0x7ffa54066f50 msgr2=0x7ffa541bbb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:09.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.331+0000 7ffa5a9c2700 1 -- 172.21.15.16:0/788538518 shutdown_connections 2023-09-30T22:36:09.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.331+0000 7ffa5a9c2700 1 -- 172.21.15.16:0/788538518 wait complete. 2023-09-30T22:36:09.345 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:36:09.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.503+0000 7f4179bc2700 1 -- 172.21.15.16:0/2437137422 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f417406b250 con 0x7f417409f240 2023-09-30T22:36:09.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.503+0000 7f415bfff700 1 -- 172.21.15.16:0/2437137422 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4168034070 con 0x7f417409f240 2023-09-30T22:36:09.504 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:36:09.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.505+0000 7f4179bc2700 1 -- 172.21.15.16:0/2437137422 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f417409f240 msgr2=0x7f417406bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:09.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.505+0000 7f4179bc2700 1 --2- 172.21.15.16:0/2437137422 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f417409f240 0x7f417406bdc0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f4168016000 tx=0x7f4168004750 comp rx=0 tx=0).stop 2023-09-30T22:36:09.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.505+0000 7f4179bc2700 1 -- 172.21.15.16:0/2437137422 shutdown_connections 2023-09-30T22:36:09.506 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.505+0000 7f4179bc2700 1 --2- 172.21.15.16:0/2437137422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f417409f650 0x7f417406c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:09.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.505+0000 7f4179bc2700 1 --2- 172.21.15.16:0/2437137422 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f417409e930 0x7f417406b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:09.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.505+0000 7f4179bc2700 1 --2- 172.21.15.16:0/2437137422 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f417409f240 0x7f417406bdc0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:09.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.505+0000 7f4179bc2700 1 -- 172.21.15.16:0/2437137422 >> 172.21.15.16:0/2437137422 conn(0x7f41740a4210 msgr2=0x7f41741b8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:09.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.506+0000 7f4179bc2700 1 -- 172.21.15.16:0/2437137422 shutdown_connections 2023-09-30T22:36:09.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:09.506+0000 7f4179bc2700 1 -- 172.21.15.16:0/2437137422 wait complete. 2023-09-30T22:36:09.523 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:36:10.279 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:36:10.304 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:36:10.346 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:36:10.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.428+0000 7faed41db700 1 Processor -- start 2023-09-30T22:36:10.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.428+0000 7faed41db700 1 -- start start 2023-09-30T22:36:10.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.429+0000 7faed41db700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faecc133420 0x7faecc1337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:10.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.429+0000 7faed41db700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faecc133e30 0x7faecc12ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:10.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.429+0000 7faed41db700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faecc06b990 con 0x7faecc134260 2023-09-30T22:36:10.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.429+0000 7faed41db700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faecc0a0590 con 0x7faecc133e30 2023-09-30T22:36:10.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.429+0000 7faed41db700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faecc09fb80 con 0x7faecc133420 2023-09-30T22:36:10.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.429+0000 7faed1f77700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faecc133420 0x7faecc1337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:10.430 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.429+0000 7faed1f77700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faecc133420 0x7faecc1337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:33268/0 (socket says 172.21.15.16:33268) 2023-09-30T22:36:10.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.429+0000 7faed1f77700 1 -- 172.21.15.16:0/2737286766 learned_addr learned my addr 172.21.15.16:0/2737286766 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:10.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.429+0000 7faed1776700 1 --2- 172.21.15.16:0/2737286766 >> v2:172.21.15.16:3300/0 conn(0x7faecc133e30 0x7faecc12ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:10.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.429+0000 7faed1f77700 1 -- 172.21.15.16:0/2737286766 >> v2:172.21.15.16:3300/0 conn(0x7faecc133e30 msgr2=0x7faecc12ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:10.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.429+0000 7faed1f77700 1 --2- 172.21.15.16:0/2737286766 >> v2:172.21.15.16:3300/0 conn(0x7faecc133e30 0x7faecc12ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:10.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.429+0000 7faed1f77700 1 -- 172.21.15.16:0/2737286766 >> v1:172.21.15.16:6789/0 conn(0x7faecc134260 legacy=0x7faecc1304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:10.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.429+0000 7faed1f77700 1 -- 172.21.15.16:0/2737286766 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faecc0a0030 con 0x7faecc133420 2023-09-30T22:36:10.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.429+0000 7faed1f77700 1 --2- 172.21.15.16:0/2737286766 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faecc133420 0x7faecc1337f0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7faec80098f0 tx=0x7faec8009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7fe8ac4909a07ef server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:10.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.429+0000 7faed0f75700 1 -- 172.21.15.16:0/2737286766 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec8011830 con 0x7faecc133420 2023-09-30T22:36:10.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.430+0000 7faed0f75700 1 -- 172.21.15.16:0/2737286766 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faec8011990 con 0x7faecc133420 2023-09-30T22:36:10.431 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.430+0000 7faed0f75700 1 -- 172.21.15.16:0/2737286766 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faec8011d20 con 0x7faecc133420 2023-09-30T22:36:10.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.430+0000 7faed41db700 1 -- 172.21.15.16:0/2737286766 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faecc133420 msgr2=0x7faecc1337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:10.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.430+0000 7faed41db700 1 --2- 172.21.15.16:0/2737286766 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faecc133420 0x7faecc1337f0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7faec80098f0 tx=0x7faec8009c00 comp rx=0 tx=0).stop 2023-09-30T22:36:10.432 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.430+0000 7faed41db700 1 -- 172.21.15.16:0/2737286766 shutdown_connections 2023-09-30T22:36:10.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.430+0000 7faed41db700 1 --2- 172.21.15.16:0/2737286766 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faecc133420 0x7faecc1337f0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:10.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.430+0000 7faed41db700 1 --2- 172.21.15.16:0/2737286766 >> v2:172.21.15.16:3300/0 conn(0x7faecc133e30 0x7faecc12ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:10.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.430+0000 7faed41db700 1 -- 172.21.15.16:0/2737286766 >> 172.21.15.16:0/2737286766 conn(0x7faecc0a4210 msgr2=0x7faecc0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:10.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.430+0000 7faed41db700 1 -- 172.21.15.16:0/2737286766 shutdown_connections 2023-09-30T22:36:10.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.431+0000 7faed41db700 1 -- 172.21.15.16:0/2737286766 wait complete. 2023-09-30T22:36:10.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.432+0000 7faed41db700 1 Processor -- start 2023-09-30T22:36:10.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.432+0000 7faed41db700 1 -- start start 2023-09-30T22:36:10.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.433+0000 7faed41db700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faecc133420 0x7faecc09c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:10.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.433+0000 7faed41db700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faecc133e30 0x7faecc09c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:10.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.433+0000 7faed41db700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faecc134260 0x7faecc09cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:10.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.433+0000 7faed41db700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faecc11f0f0 con 0x7faecc133420 2023-09-30T22:36:10.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.433+0000 7faed41db700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faecc09dfd0 con 0x7faecc133e30 2023-09-30T22:36:10.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.433+0000 7faed41db700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faecc120100 con 0x7faecc134260 2023-09-30T22:36:10.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.433+0000 7faed1776700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faecc133e30 0x7faecc09c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:10.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.433+0000 7faed1f77700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faecc133420 0x7faecc09c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:10.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.433+0000 7faed1776700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faecc133e30 0x7faecc09c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:33272/0 (socket says 172.21.15.16:33272) 2023-09-30T22:36:10.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.433+0000 7faed1776700 1 -- 172.21.15.16:0/2545103171 learned_addr learned my addr 172.21.15.16:0/2545103171 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:10.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.433+0000 7faed1776700 1 -- 172.21.15.16:0/2545103171 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faecc134260 msgr2=0x7faecc09cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:10.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.433+0000 7faed1776700 1 --2- 172.21.15.16:0/2545103171 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faecc134260 0x7faecc09cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:10.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.433+0000 7faed1776700 1 -- 172.21.15.16:0/2545103171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faecc133420 msgr2=0x7faecc09c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:10.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.433+0000 7faed1776700 1 --2- 172.21.15.16:0/2545103171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faecc133420 0x7faecc09c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:10.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.433+0000 7faed1776700 1 -- 172.21.15.16:0/2545103171 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faec8009550 con 0x7faecc133e30 2023-09-30T22:36:10.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.433+0000 7faed1f77700 1 --2- 172.21.15.16:0/2545103171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faecc133420 0x7faecc09c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:10.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.433+0000 7faed1776700 1 --2- 172.21.15.16:0/2545103171 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faecc133e30 0x7faecc09c9c0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7faebc00e980 tx=0x7faebc00ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:10.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.434+0000 7faec27fc700 1 -- 172.21.15.16:0/2545103171 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faebc009710 con 0x7faecc133e30 2023-09-30T22:36:10.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.434+0000 7faec27fc700 1 -- 172.21.15.16:0/2545103171 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faebc00eed0 con 0x7faecc133e30 2023-09-30T22:36:10.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.434+0000 7faec27fc700 1 -- 172.21.15.16:0/2545103171 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faebc00ca70 con 0x7faecc133e30 2023-09-30T22:36:10.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.434+0000 7faed41db700 1 -- 172.21.15.16:0/2545103171 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faecc11f4a0 con 0x7faecc133e30 2023-09-30T22:36:10.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.434+0000 7faed41db700 1 -- 172.21.15.16:0/2545103171 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faecc11fe20 con 0x7faecc133e30 2023-09-30T22:36:10.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.435+0000 7faec27fc700 1 -- 172.21.15.16:0/2545103171 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7faebc00cbd0 con 0x7faecc133e30 2023-09-30T22:36:10.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.435+0000 7faec27fc700 1 -- 172.21.15.16:0/2545103171 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7faebc03f610 con 0x7faecc133e30 2023-09-30T22:36:10.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.435+0000 7faed41db700 1 -- 172.21.15.16:0/2545103171 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faecc11f440 con 0x7faecc133e30 2023-09-30T22:36:10.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.440+0000 7faec27fc700 1 -- 172.21.15.16:0/2545103171 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7faebc013470 con 0x7faecc133e30 2023-09-30T22:36:10.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.469+0000 7fd4e873e700 1 Processor -- start 2023-09-30T22:36:10.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.469+0000 7fd4e873e700 1 -- start start 2023-09-30T22:36:10.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.470+0000 7fd4e873e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd4e00a0a30 0x7fd4e00a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:10.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.470+0000 7fd4e873e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4e0134170 0x7fd4e013bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:10.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.470+0000 7fd4e873e700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4e009edd0 con 0x7fd4e00a1340 2023-09-30T22:36:10.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.470+0000 7fd4e873e700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4e009e1f0 con 0x7fd4e00a0a30 2023-09-30T22:36:10.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.470+0000 7fd4e873e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e009dc50 con 0x7fd4e0134170 2023-09-30T22:36:10.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.470+0000 7fd4e64da700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd4e00a0a30 0x7fd4e00a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:10.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.470+0000 7fd4e5cd9700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd4e00a1340 0x7fd4e0133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52182/0 (socket says 172.21.15.16:52182) 2023-09-30T22:36:10.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.470+0000 7fd4e5cd9700 1 -- 172.21.15.16:0/3407337191 learned_addr learned my addr 172.21.15.16:0/3407337191 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:36:10.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.470+0000 7fd4e6cdb700 1 --2- 172.21.15.16:0/3407337191 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4e0134170 0x7fd4e013bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:10.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.470+0000 7fd4e54d8700 1 -- 172.21.15.16:0/3407337191 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3941393340 0 0) 0x7fd4e009edd0 con 0x7fd4e00a1340 2023-09-30T22:36:10.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.470+0000 7fd4e54d8700 1 -- 172.21.15.16:0/3407337191 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4cc0035b0 con 0x7fd4e00a1340 2023-09-30T22:36:10.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.470+0000 7fd4e6cdb700 1 -- 172.21.15.16:0/3407337191 >> v2:172.21.15.16:3300/0 conn(0x7fd4e00a0a30 msgr2=0x7fd4e00a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:10.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.470+0000 7fd4e6cdb700 1 --2- 172.21.15.16:0/3407337191 >> v2:172.21.15.16:3300/0 conn(0x7fd4e00a0a30 0x7fd4e00a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:10.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.470+0000 7fd4e6cdb700 1 -- 172.21.15.16:0/3407337191 >> v1:172.21.15.16:6789/0 conn(0x7fd4e00a1340 legacy=0x7fd4e0133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:10.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.470+0000 7fd4e6cdb700 1 -- 172.21.15.16:0/3407337191 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e00a1fe0 con 0x7fd4e0134170 2023-09-30T22:36:10.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.470+0000 7fd4e64da700 1 --2- 172.21.15.16:0/3407337191 >> v2:172.21.15.16:3300/0 conn(0x7fd4e00a0a30 0x7fd4e00a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:10.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.471+0000 7fd4e6cdb700 1 --2- 172.21.15.16:0/3407337191 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4e0134170 0x7fd4e013bd30 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8009d90 tx=0x7fd4d800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0b7c296fdb708e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:10.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.471+0000 7fd4e54d8700 1 -- 172.21.15.16:0/3407337191 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d800e070 con 0x7fd4e0134170 2023-09-30T22:36:10.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.471+0000 7fd4e54d8700 1 -- 172.21.15.16:0/3407337191 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4d800bb20 con 0x7fd4e0134170 2023-09-30T22:36:10.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.471+0000 7fd4e54d8700 1 -- 172.21.15.16:0/3407337191 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d8012430 con 0x7fd4e0134170 2023-09-30T22:36:10.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.472+0000 7fd4e873e700 1 -- 172.21.15.16:0/3407337191 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4e0134170 msgr2=0x7fd4e013bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:10.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.472+0000 7fd4e873e700 1 --2- 172.21.15.16:0/3407337191 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4e0134170 0x7fd4e013bd30 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8009d90 tx=0x7fd4d800b330 comp rx=0 tx=0).stop 2023-09-30T22:36:10.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.472+0000 7fd4e873e700 1 -- 172.21.15.16:0/3407337191 shutdown_connections 2023-09-30T22:36:10.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.472+0000 7fd4e873e700 1 --2- 172.21.15.16:0/3407337191 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4e0134170 0x7fd4e013bd30 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:10.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.472+0000 7fd4e873e700 1 --2- 172.21.15.16:0/3407337191 >> v2:172.21.15.16:3300/0 conn(0x7fd4e00a0a30 0x7fd4e00a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:10.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.472+0000 7fd4e873e700 1 -- 172.21.15.16:0/3407337191 >> 172.21.15.16:0/3407337191 conn(0x7fd4e00a6550 msgr2=0x7fd4e00a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:10.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.472+0000 7fd4e873e700 1 -- 172.21.15.16:0/3407337191 shutdown_connections 2023-09-30T22:36:10.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.472+0000 7fd4e873e700 1 -- 172.21.15.16:0/3407337191 wait complete. 2023-09-30T22:36:10.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.474+0000 7fd4e873e700 1 Processor -- start 2023-09-30T22:36:10.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.474+0000 7fd4e873e700 1 -- start start 2023-09-30T22:36:10.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.474+0000 7fd4e873e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4e00a0a30 0x7fd4e01c83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:10.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.474+0000 7fd4e873e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd4e00a1340 0x7fd4e01c8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:10.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.474+0000 7fd4e873e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4e0134170 0x7fd4e01c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:10.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.474+0000 7fd4e873e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e0124d50 con 0x7fd4e0134170 2023-09-30T22:36:10.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.474+0000 7fd4e873e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e0122de0 con 0x7fd4e00a0a30 2023-09-30T22:36:10.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.474+0000 7fd4e873e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e01263d0 con 0x7fd4e00a1340 2023-09-30T22:36:10.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.475+0000 7fd4e64da700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4e00a0a30 0x7fd4e01c83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:10.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.475+0000 7fd4e64da700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4e00a0a30 0x7fd4e01c83e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:33280/0 (socket says 172.21.15.16:33280) 2023-09-30T22:36:10.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.475+0000 7fd4e64da700 1 -- 172.21.15.16:0/1500220570 learned_addr learned my addr 172.21.15.16:0/1500220570 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:10.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.475+0000 7fd4e5cd9700 1 --2- 172.21.15.16:0/1500220570 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd4e00a1340 0x7fd4e01c8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:10.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.475+0000 7fd4e6cdb700 1 --2- 172.21.15.16:0/1500220570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4e0134170 0x7fd4e01c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:10.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.475+0000 7fd4e64da700 1 -- 172.21.15.16:0/1500220570 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd4e00a1340 msgr2=0x7fd4e01c8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:10.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.475+0000 7fd4e64da700 1 --2- 172.21.15.16:0/1500220570 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd4e00a1340 0x7fd4e01c8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:10.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.475+0000 7fd4e64da700 1 -- 172.21.15.16:0/1500220570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4e0134170 msgr2=0x7fd4e01c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:10.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.475+0000 7fd4e64da700 1 --2- 172.21.15.16:0/1500220570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4e0134170 0x7fd4e01c8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:10.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.475+0000 7fd4e64da700 1 -- 172.21.15.16:0/1500220570 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4dc0094a0 con 0x7fd4e00a0a30 2023-09-30T22:36:10.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.475+0000 7fd4e64da700 1 --2- 172.21.15.16:0/1500220570 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4e00a0a30 0x7fd4e01c83e0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc00ebe0 tx=0x7fd4dc00eef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:10.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.475+0000 7fd4d6ffd700 1 -- 172.21.15.16:0/1500220570 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc00c850 con 0x7fd4e00a0a30 2023-09-30T22:36:10.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.475+0000 7fd4d6ffd700 1 -- 172.21.15.16:0/1500220570 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4dc00c9b0 con 0x7fd4e00a0a30 2023-09-30T22:36:10.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.475+0000 7fd4e873e700 1 -- 172.21.15.16:0/1500220570 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4d8009a70 con 0x7fd4e00a0a30 2023-09-30T22:36:10.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.475+0000 7fd4d6ffd700 1 -- 172.21.15.16:0/1500220570 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc00cd20 con 0x7fd4e00a0a30 2023-09-30T22:36:10.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.476+0000 7fd4e873e700 1 -- 172.21.15.16:0/1500220570 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4e00a25e0 con 0x7fd4e00a0a30 2023-09-30T22:36:10.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.476+0000 7fd4d6ffd700 1 -- 172.21.15.16:0/1500220570 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd4dc013470 con 0x7fd4e00a0a30 2023-09-30T22:36:10.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.476+0000 7fd4d6ffd700 1 -- 172.21.15.16:0/1500220570 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd4dc03fcf0 con 0x7fd4e00a0a30 2023-09-30T22:36:10.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.477+0000 7fd4e873e700 1 -- 172.21.15.16:0/1500220570 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4e00a4940 con 0x7fd4e00a0a30 2023-09-30T22:36:10.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.482+0000 7fd4d6ffd700 1 -- 172.21.15.16:0/1500220570 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd4dc017080 con 0x7fd4e00a0a30 2023-09-30T22:36:10.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.510+0000 7fcac5374700 1 Processor -- start 2023-09-30T22:36:10.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.510+0000 7fcac5374700 1 -- start start 2023-09-30T22:36:10.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.510+0000 7fcac5374700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcac009d6f0 0x7fcac009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:10.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.510+0000 7fcac5374700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcac009e000 0x7fcac0130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:10.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.510+0000 7fcac5374700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcac009e410 0x7fcac0130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:10.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.510+0000 7fcac5374700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcac009f520 con 0x7fcac009d6f0 2023-09-30T22:36:10.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.510+0000 7fcac5374700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcac009f130 con 0x7fcac009e000 2023-09-30T22:36:10.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.510+0000 7fcac5374700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcac009e930 con 0x7fcac009e410 2023-09-30T22:36:10.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.511+0000 7fcabe7fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcac009e000 0x7fcac0130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:10.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.511+0000 7fcabe7fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcac009e000 0x7fcac0130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:33286/0 (socket says 172.21.15.16:33286) 2023-09-30T22:36:10.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.511+0000 7fcabe7fc700 1 -- 172.21.15.16:0/2449800634 learned_addr learned my addr 172.21.15.16:0/2449800634 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:10.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.511+0000 7fcabf7fe700 1 --2- 172.21.15.16:0/2449800634 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcac009e410 0x7fcac0130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:10.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.511+0000 7fcabeffd700 1 --2- 172.21.15.16:0/2449800634 >> v2:172.21.15.16:3300/0 conn(0x7fcac009d6f0 0x7fcac009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:10.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.511+0000 7fcabe7fc700 1 -- 172.21.15.16:0/2449800634 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcac009e410 msgr2=0x7fcac0130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:10.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.511+0000 7fcabe7fc700 1 --2- 172.21.15.16:0/2449800634 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcac009e410 0x7fcac0130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:10.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.511+0000 7fcabe7fc700 1 -- 172.21.15.16:0/2449800634 >> v2:172.21.15.16:3300/0 conn(0x7fcac009d6f0 msgr2=0x7fcac009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:10.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.511+0000 7fcabe7fc700 1 --2- 172.21.15.16:0/2449800634 >> v2:172.21.15.16:3300/0 conn(0x7fcac009d6f0 0x7fcac009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:10.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.511+0000 7fcabe7fc700 1 -- 172.21.15.16:0/2449800634 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac009b2d0 con 0x7fcac009e000 2023-09-30T22:36:10.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.511+0000 7fcabeffd700 1 --2- 172.21.15.16:0/2449800634 >> v2:172.21.15.16:3300/0 conn(0x7fcac009d6f0 0x7fcac009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:10.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.511+0000 7fcabf7fe700 1 --2- 172.21.15.16:0/2449800634 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcac009e410 0x7fcac0130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:10.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.512+0000 7fcabe7fc700 1 --2- 172.21.15.16:0/2449800634 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcac009e000 0x7fcac0130660 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fcab400bb90 tx=0x7fcab400bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eadbed276df186f6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:10.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.512+0000 7fcabdffb700 1 -- 172.21.15.16:0/2449800634 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab400dd30 con 0x7fcac009e000 2023-09-30T22:36:10.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.512+0000 7fcabdffb700 1 -- 172.21.15.16:0/2449800634 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab400de90 con 0x7fcac009e000 2023-09-30T22:36:10.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.512+0000 7fcabdffb700 1 -- 172.21.15.16:0/2449800634 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab4010690 con 0x7fcac009e000 2023-09-30T22:36:10.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.512+0000 7fcac5374700 1 -- 172.21.15.16:0/2449800634 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcac009e000 msgr2=0x7fcac0130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:10.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.512+0000 7fcac5374700 1 --2- 172.21.15.16:0/2449800634 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcac009e000 0x7fcac0130660 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fcab400bb90 tx=0x7fcab400bea0 comp rx=0 tx=0).stop 2023-09-30T22:36:10.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.512+0000 7fcac5374700 1 -- 172.21.15.16:0/2449800634 shutdown_connections 2023-09-30T22:36:10.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.512+0000 7fcac5374700 1 --2- 172.21.15.16:0/2449800634 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcac009e410 0x7fcac0130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:10.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.512+0000 7fcac5374700 1 --2- 172.21.15.16:0/2449800634 >> v2:172.21.15.16:3300/0 conn(0x7fcac009d6f0 0x7fcac009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:10.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.512+0000 7fcac5374700 1 --2- 172.21.15.16:0/2449800634 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcac009e000 0x7fcac0130660 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:10.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.512+0000 7fcac5374700 1 -- 172.21.15.16:0/2449800634 >> 172.21.15.16:0/2449800634 conn(0x7fcac0066f50 msgr2=0x7fcac009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:10.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.512+0000 7fcac5374700 1 -- 172.21.15.16:0/2449800634 shutdown_connections 2023-09-30T22:36:10.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.512+0000 7fcac5374700 1 -- 172.21.15.16:0/2449800634 wait complete. 2023-09-30T22:36:10.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.514+0000 7fcac5374700 1 Processor -- start 2023-09-30T22:36:10.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.514+0000 7fcac5374700 1 -- start start 2023-09-30T22:36:10.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.515+0000 7fcac5374700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcac009d6f0 0x7fcac011e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:10.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.515+0000 7fcac5374700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcac009e000 0x7fcac011e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:10.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.515+0000 7fcac5374700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcac009e410 0x7fcac011eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:10.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.515+0000 7fcac5374700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac0121bc0 con 0x7fcac009d6f0 2023-09-30T22:36:10.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.515+0000 7fcac5374700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcac00a1aa0 con 0x7fcac009e000 2023-09-30T22:36:10.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.515+0000 7fcac5374700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcac009c700 con 0x7fcac009e410 2023-09-30T22:36:10.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.515+0000 7fcabe7fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcac009e000 0x7fcac011e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:10.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.515+0000 7fcabe7fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcac009e000 0x7fcac011e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:33288/0 (socket says 172.21.15.16:33288) 2023-09-30T22:36:10.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.515+0000 7fcabe7fc700 1 -- 172.21.15.16:0/71053455 learned_addr learned my addr 172.21.15.16:0/71053455 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:10.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.515+0000 7fcabf7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcac009e410 0x7fcac011eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:10.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.515+0000 7fcabeffd700 1 --2- 172.21.15.16:0/71053455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcac009d6f0 0x7fcac011e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:10.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.515+0000 7fcabe7fc700 1 -- 172.21.15.16:0/71053455 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcac009e410 msgr2=0x7fcac011eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:10.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.515+0000 7fcabe7fc700 1 --2- 172.21.15.16:0/71053455 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcac009e410 0x7fcac011eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:10.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.516+0000 7fcabe7fc700 1 -- 172.21.15.16:0/71053455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcac009d6f0 msgr2=0x7fcac011e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:10.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.516+0000 7fcabe7fc700 1 --2- 172.21.15.16:0/71053455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcac009d6f0 0x7fcac011e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:10.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.516+0000 7fcabe7fc700 1 -- 172.21.15.16:0/71053455 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab00094a0 con 0x7fcac009e000 2023-09-30T22:36:10.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.516+0000 7fcabe7fc700 1 --2- 172.21.15.16:0/71053455 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcac009e000 0x7fcac011e630 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fcab4000c00 tx=0x7fcab4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:10.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.516+0000 7fcaa77fe700 1 -- 172.21.15.16:0/71053455 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab401f070 con 0x7fcac009e000 2023-09-30T22:36:10.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.516+0000 7fcaa77fe700 1 -- 172.21.15.16:0/71053455 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab40038b0 con 0x7fcac009e000 2023-09-30T22:36:10.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.516+0000 7fcac5374700 1 -- 172.21.15.16:0/71053455 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcab8009a70 con 0x7fcac009e000 2023-09-30T22:36:10.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.516+0000 7fcaa77fe700 1 -- 172.21.15.16:0/71053455 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab4003c60 con 0x7fcac009e000 2023-09-30T22:36:10.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.516+0000 7fcac5374700 1 -- 172.21.15.16:0/71053455 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcab400b7f0 con 0x7fcac009e000 2023-09-30T22:36:10.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.517+0000 7fcaa77fe700 1 -- 172.21.15.16:0/71053455 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcab4019400 con 0x7fcac009e000 2023-09-30T22:36:10.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.517+0000 7fcaa77fe700 1 -- 172.21.15.16:0/71053455 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fcab4042610 con 0x7fcac009e000 2023-09-30T22:36:10.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.517+0000 7fcac5374700 1 -- 172.21.15.16:0/71053455 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcac0121470 con 0x7fcac009e000 2023-09-30T22:36:10.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.522+0000 7fcaa77fe700 1 -- 172.21.15.16:0/71053455 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcab4021310 con 0x7fcac009e000 2023-09-30T22:36:10.524 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:36:10.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.661+0000 7fd6e8a64700 1 Processor -- start 2023-09-30T22:36:10.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.661+0000 7fd6e8a64700 1 -- start start 2023-09-30T22:36:10.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.662+0000 7fd6e8a64700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6e409d6f0 0x7fd6e409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:10.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.662+0000 7fd6e8a64700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd6e409e000 0x7fd6e4130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:10.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.662+0000 7fd6e8a64700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6e409f130 con 0x7fd6e409e410 2023-09-30T22:36:10.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.663+0000 7fd6e8a64700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e409f520 con 0x7fd6e409d6f0 2023-09-30T22:36:10.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.663+0000 7fd6e8a64700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e409e930 con 0x7fd6e409e000 2023-09-30T22:36:10.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.663+0000 7fd6e259c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6e409d6f0 0x7fd6e409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:10.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.663+0000 7fd6e259c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6e409d6f0 0x7fd6e409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:33304/0 (socket says 172.21.15.16:33304) 2023-09-30T22:36:10.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.663+0000 7fd6e259c700 1 -- 172.21.15.16:0/2057768124 learned_addr learned my addr 172.21.15.16:0/2057768124 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:10.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.663+0000 7fd6e2d9d700 1 --1- 172.21.15.16:0/2057768124 >> v1:172.21.15.16:6789/0 conn(0x7fd6e409e410 0x7fd6e4130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52184/0 (socket says 172.21.15.16:52184) 2023-09-30T22:36:10.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.663+0000 7fd6e1d9b700 1 --2- 172.21.15.16:0/2057768124 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd6e409e000 0x7fd6e4130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:10.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.663+0000 7fd6e159a700 1 -- 172.21.15.16:0/2057768124 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1701929901 0 0) 0x7fd6e409f130 con 0x7fd6e409e410 2023-09-30T22:36:10.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.663+0000 7fd6e259c700 1 -- 172.21.15.16:0/2057768124 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd6e409e000 msgr2=0x7fd6e4130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:10.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.663+0000 7fd6e259c700 1 --2- 172.21.15.16:0/2057768124 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd6e409e000 0x7fd6e4130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:10.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.663+0000 7fd6e259c700 1 -- 172.21.15.16:0/2057768124 >> v1:172.21.15.16:6789/0 conn(0x7fd6e409e410 legacy=0x7fd6e4130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:10.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.663+0000 7fd6e259c700 1 -- 172.21.15.16:0/2057768124 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e409b2d0 con 0x7fd6e409d6f0 2023-09-30T22:36:10.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.664+0000 7fd6e259c700 1 --2- 172.21.15.16:0/2057768124 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6e409d6f0 0x7fd6e409dac0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4009920 tx=0x7fd6d4009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c083f11d2a8cf9c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:10.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.664+0000 7fd6e159a700 1 -- 172.21.15.16:0/2057768124 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d4011830 con 0x7fd6e409d6f0 2023-09-30T22:36:10.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.664+0000 7fd6e159a700 1 -- 172.21.15.16:0/2057768124 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6d4011990 con 0x7fd6e409d6f0 2023-09-30T22:36:10.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.664+0000 7fd6e159a700 1 -- 172.21.15.16:0/2057768124 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d4011d20 con 0x7fd6e409d6f0 2023-09-30T22:36:10.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.665+0000 7fd6e8a64700 1 -- 172.21.15.16:0/2057768124 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6e409d6f0 msgr2=0x7fd6e409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:10.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.665+0000 7fd6e8a64700 1 --2- 172.21.15.16:0/2057768124 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6e409d6f0 0x7fd6e409dac0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4009920 tx=0x7fd6d4009c30 comp rx=0 tx=0).stop 2023-09-30T22:36:10.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.665+0000 7fd6e8a64700 1 -- 172.21.15.16:0/2057768124 shutdown_connections 2023-09-30T22:36:10.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.665+0000 7fd6e8a64700 1 --2- 172.21.15.16:0/2057768124 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6e409d6f0 0x7fd6e409dac0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:10.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.665+0000 7fd6e8a64700 1 --2- 172.21.15.16:0/2057768124 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd6e409e000 0x7fd6e4130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:10.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.665+0000 7fd6e8a64700 1 -- 172.21.15.16:0/2057768124 >> 172.21.15.16:0/2057768124 conn(0x7fd6e4066f50 msgr2=0x7fd6e409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:10.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.665+0000 7fd6e8a64700 1 -- 172.21.15.16:0/2057768124 shutdown_connections 2023-09-30T22:36:10.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.665+0000 7fd6e8a64700 1 -- 172.21.15.16:0/2057768124 wait complete. 2023-09-30T22:36:10.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.667+0000 7fd6e8a64700 1 Processor -- start 2023-09-30T22:36:10.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.667+0000 7fd6e8a64700 1 -- start start 2023-09-30T22:36:10.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.667+0000 7fd6e8a64700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6e409d6f0 0x7fd6e41c2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:10.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.667+0000 7fd6e8a64700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd6e409e000 0x7fd6e41c31f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:10.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.667+0000 7fd6e8a64700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6e409e410 0x7fd6e4131670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:10.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.667+0000 7fd6e8a64700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e409c700 con 0x7fd6e409e410 2023-09-30T22:36:10.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.667+0000 7fd6e8a64700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e4134510 con 0x7fd6e409d6f0 2023-09-30T22:36:10.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.667+0000 7fd6e8a64700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e41c2880 con 0x7fd6e409e000 2023-09-30T22:36:10.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.668+0000 7fd6e1d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd6e409e000 0x7fd6e41c31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:10.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.668+0000 7fd6e1d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd6e409e000 0x7fd6e41c31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:43916/0 (socket says 172.21.15.16:43916) 2023-09-30T22:36:10.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.668+0000 7fd6e1d9b700 1 -- 172.21.15.16:0/3922332588 learned_addr learned my addr 172.21.15.16:0/3922332588 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:10.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.668+0000 7fd6e2d9d700 1 --2- 172.21.15.16:0/3922332588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6e409e410 0x7fd6e4131670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:10.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.668+0000 7fd6e259c700 1 --2- 172.21.15.16:0/3922332588 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6e409d6f0 0x7fd6e41c2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:10.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.668+0000 7fd6e1d9b700 1 -- 172.21.15.16:0/3922332588 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6e409d6f0 msgr2=0x7fd6e41c2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:10.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.668+0000 7fd6e1d9b700 1 --2- 172.21.15.16:0/3922332588 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6e409d6f0 0x7fd6e41c2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:10.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.668+0000 7fd6e1d9b700 1 -- 172.21.15.16:0/3922332588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6e409e410 msgr2=0x7fd6e4131670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:10.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.668+0000 7fd6e1d9b700 1 --2- 172.21.15.16:0/3922332588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6e409e410 0x7fd6e4131670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:10.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.668+0000 7fd6e1d9b700 1 -- 172.21.15.16:0/3922332588 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d4009550 con 0x7fd6e409e000 2023-09-30T22:36:10.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.668+0000 7fd6e1d9b700 1 --2- 172.21.15.16:0/3922332588 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd6e409e000 0x7fd6e41c31f0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fd6d800e9b0 tx=0x7fd6d800ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:10.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.668+0000 7fd6d2ffd700 1 -- 172.21.15.16:0/3922332588 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d8009710 con 0x7fd6e409e000 2023-09-30T22:36:10.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.668+0000 7fd6d2ffd700 1 -- 172.21.15.16:0/3922332588 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6d800ee50 con 0x7fd6e409e000 2023-09-30T22:36:10.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.668+0000 7fd6e8a64700 1 -- 172.21.15.16:0/3922332588 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6e40a3980 con 0x7fd6e409e000 2023-09-30T22:36:10.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.668+0000 7fd6d2ffd700 1 -- 172.21.15.16:0/3922332588 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d800ca70 con 0x7fd6e409e000 2023-09-30T22:36:10.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.669+0000 7fd6e8a64700 1 -- 172.21.15.16:0/3922332588 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6e41328e0 con 0x7fd6e409e000 2023-09-30T22:36:10.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.670+0000 7fd6d2ffd700 1 -- 172.21.15.16:0/3922332588 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd6d800cbd0 con 0x7fd6e409e000 2023-09-30T22:36:10.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.670+0000 7fd6d2ffd700 1 -- 172.21.15.16:0/3922332588 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd6d803fa30 con 0x7fd6e409e000 2023-09-30T22:36:10.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.670+0000 7fd6e8a64700 1 -- 172.21.15.16:0/3922332588 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6e4134290 con 0x7fd6e409e000 2023-09-30T22:36:10.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.675+0000 7fd6d2ffd700 1 -- 172.21.15.16:0/3922332588 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd6d8017080 con 0x7fd6e409e000 2023-09-30T22:36:10.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.682+0000 7faed41db700 1 -- 172.21.15.16:0/2545103171 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7faecc074ea0 con 0x7faecc133e30 2023-09-30T22:36:10.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.682+0000 7faec27fc700 1 -- 172.21.15.16:0/2545103171 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7faebc017020 con 0x7faecc133e30 2023-09-30T22:36:10.683 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:36:10.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.686+0000 7faed41db700 1 -- 172.21.15.16:0/2545103171 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faecc133e30 msgr2=0x7faecc09c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:10.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.686+0000 7faed41db700 1 --2- 172.21.15.16:0/2545103171 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faecc133e30 0x7faecc09c9c0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7faebc00e980 tx=0x7faebc00ec90 comp rx=0 tx=0).stop 2023-09-30T22:36:10.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.686+0000 7faed41db700 1 -- 172.21.15.16:0/2545103171 shutdown_connections 2023-09-30T22:36:10.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.686+0000 7faed41db700 1 --2- 172.21.15.16:0/2545103171 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faecc134260 0x7faecc09cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:10.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.686+0000 7faed41db700 1 --2- 172.21.15.16:0/2545103171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faecc133420 0x7faecc09c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:10.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.686+0000 7faed41db700 1 --2- 172.21.15.16:0/2545103171 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faecc133e30 0x7faecc09c9c0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:10.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.686+0000 7faed41db700 1 -- 172.21.15.16:0/2545103171 >> 172.21.15.16:0/2545103171 conn(0x7faecc0a4210 msgr2=0x7faecc12f600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:10.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.686+0000 7faed41db700 1 -- 172.21.15.16:0/2545103171 shutdown_connections 2023-09-30T22:36:10.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.686+0000 7faed41db700 1 -- 172.21.15.16:0/2545103171 wait complete. 2023-09-30T22:36:10.706 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:36:10.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.705+0000 7fd4e873e700 1 -- 172.21.15.16:0/1500220570 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fd4e0125d90 con 0x7fd4e00a0a30 2023-09-30T22:36:10.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.705+0000 7fd4d6ffd700 1 -- 172.21.15.16:0/1500220570 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd4e0125d90 con 0x7fd4e00a0a30 2023-09-30T22:36:10.707 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:36:10.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.708+0000 7fd4e873e700 1 -- 172.21.15.16:0/1500220570 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4e00a0a30 msgr2=0x7fd4e01c83e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:10.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.708+0000 7fd4e873e700 1 --2- 172.21.15.16:0/1500220570 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4e00a0a30 0x7fd4e01c83e0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc00ebe0 tx=0x7fd4dc00eef0 comp rx=0 tx=0).stop 2023-09-30T22:36:10.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.709+0000 7fd4e873e700 1 -- 172.21.15.16:0/1500220570 shutdown_connections 2023-09-30T22:36:10.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.709+0000 7fd4e873e700 1 --2- 172.21.15.16:0/1500220570 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd4e00a0a30 0x7fd4e01c83e0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:10.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.709+0000 7fd4e873e700 1 --2- 172.21.15.16:0/1500220570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4e0134170 0x7fd4e01c8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:10.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.709+0000 7fd4e873e700 1 --2- 172.21.15.16:0/1500220570 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd4e00a1340 0x7fd4e01c8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:10.709 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.709+0000 7fd4e873e700 1 -- 172.21.15.16:0/1500220570 >> 172.21.15.16:0/1500220570 conn(0x7fd4e00a6550 msgr2=0x7fd4e009edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:10.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.709+0000 7fd4e873e700 1 -- 172.21.15.16:0/1500220570 shutdown_connections 2023-09-30T22:36:10.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.709+0000 7fd4e873e700 1 -- 172.21.15.16:0/1500220570 wait complete. 2023-09-30T22:36:10.723 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:36:10.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.753+0000 7fcac5374700 1 -- 172.21.15.16:0/71053455 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fcac0074ea0 con 0x7fcac009e000 2023-09-30T22:36:10.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.753+0000 7fcaa77fe700 1 -- 172.21.15.16:0/71053455 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcab4028020 con 0x7fcac009e000 2023-09-30T22:36:10.754 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:36:10.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.756+0000 7fcac5374700 1 -- 172.21.15.16:0/71053455 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcac009e000 msgr2=0x7fcac011e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:10.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.756+0000 7fcac5374700 1 --2- 172.21.15.16:0/71053455 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcac009e000 0x7fcac011e630 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fcab4000c00 tx=0x7fcab4004750 comp rx=0 tx=0).stop 2023-09-30T22:36:10.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.756+0000 7fcac5374700 1 -- 172.21.15.16:0/71053455 shutdown_connections 2023-09-30T22:36:10.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.756+0000 7fcac5374700 1 --2- 172.21.15.16:0/71053455 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcac009e410 0x7fcac011eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:10.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.756+0000 7fcac5374700 1 --2- 172.21.15.16:0/71053455 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcac009d6f0 0x7fcac011e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:10.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.756+0000 7fcac5374700 1 --2- 172.21.15.16:0/71053455 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcac009e000 0x7fcac011e630 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:10.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.756+0000 7fcac5374700 1 -- 172.21.15.16:0/71053455 >> 172.21.15.16:0/71053455 conn(0x7fcac0066f50 msgr2=0x7fcac0070f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:10.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.756+0000 7fcac5374700 1 -- 172.21.15.16:0/71053455 shutdown_connections 2023-09-30T22:36:10.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.756+0000 7fcac5374700 1 -- 172.21.15.16:0/71053455 wait complete. 2023-09-30T22:36:10.770 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:36:10.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.891+0000 7fd6e8a64700 1 -- 172.21.15.16:0/3922332588 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd6e4134650 con 0x7fd6e409e000 2023-09-30T22:36:10.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.892+0000 7fd6d2ffd700 1 -- 172.21.15.16:0/3922332588 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd6e4134650 con 0x7fd6e409e000 2023-09-30T22:36:10.892 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:36:10.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.894+0000 7fd6e8a64700 1 -- 172.21.15.16:0/3922332588 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd6e409e000 msgr2=0x7fd6e41c31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:10.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.894+0000 7fd6e8a64700 1 --2- 172.21.15.16:0/3922332588 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd6e409e000 0x7fd6e41c31f0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fd6d800e9b0 tx=0x7fd6d800ecc0 comp rx=0 tx=0).stop 2023-09-30T22:36:10.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.894+0000 7fd6e8a64700 1 -- 172.21.15.16:0/3922332588 shutdown_connections 2023-09-30T22:36:10.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.894+0000 7fd6e8a64700 1 --2- 172.21.15.16:0/3922332588 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd6e409d6f0 0x7fd6e41c2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:10.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.894+0000 7fd6e8a64700 1 --2- 172.21.15.16:0/3922332588 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd6e409e410 0x7fd6e4131670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:10.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.894+0000 7fd6e8a64700 1 --2- 172.21.15.16:0/3922332588 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd6e409e000 0x7fd6e41c31f0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:10.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.894+0000 7fd6e8a64700 1 -- 172.21.15.16:0/3922332588 >> 172.21.15.16:0/3922332588 conn(0x7fd6e4066f50 msgr2=0x7fd6e4070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:10.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.894+0000 7fd6e8a64700 1 -- 172.21.15.16:0/3922332588 shutdown_connections 2023-09-30T22:36:10.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:10.894+0000 7fd6e8a64700 1 -- 172.21.15.16:0/3922332588 wait complete. 2023-09-30T22:36:10.908 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:36:11.706 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:36:11.724 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:36:11.771 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:36:11.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.856+0000 7fc2d70ae700 1 Processor -- start 2023-09-30T22:36:11.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.856+0000 7fc2d70ae700 1 -- start start 2023-09-30T22:36:11.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.857+0000 7fc2d70ae700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2d009d6f0 0x7fc2d009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:11.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.857+0000 7fc2d70ae700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2d009e000 0x7fc2d01339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:11.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.857+0000 7fc2d70ae700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc2d009e410 0x7fc2d0133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:11.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.857+0000 7fc2d70ae700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2d006b460 con 0x7fc2d009e410 2023-09-30T22:36:11.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.857+0000 7fc2d70ae700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d009b120 con 0x7fc2d009e000 2023-09-30T22:36:11.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.857+0000 7fc2d70ae700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d009a7e0 con 0x7fc2d009d6f0 2023-09-30T22:36:11.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.857+0000 7fc2d4e4a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2d009d6f0 0x7fc2d009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:11.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.857+0000 7fc2d4e4a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2d009d6f0 0x7fc2d009dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:43930/0 (socket says 172.21.15.16:43930) 2023-09-30T22:36:11.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.857+0000 7fc2d4e4a700 1 -- 172.21.15.16:0/920520510 learned_addr learned my addr 172.21.15.16:0/920520510 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:11.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.857+0000 7fc2d564b700 1 --2- 172.21.15.16:0/920520510 >> v2:172.21.15.16:3300/0 conn(0x7fc2d009e410 0x7fc2d0133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:11.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.857+0000 7fc2cffff700 1 --2- 172.21.15.16:0/920520510 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2d009e000 0x7fc2d01339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:11.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.857+0000 7fc2d4e4a700 1 -- 172.21.15.16:0/920520510 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2d009e000 msgr2=0x7fc2d01339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:11.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.857+0000 7fc2d4e4a700 1 --2- 172.21.15.16:0/920520510 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2d009e000 0x7fc2d01339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:11.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.857+0000 7fc2d4e4a700 1 -- 172.21.15.16:0/920520510 >> v2:172.21.15.16:3300/0 conn(0x7fc2d009e410 msgr2=0x7fc2d0133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:11.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.857+0000 7fc2d4e4a700 1 --2- 172.21.15.16:0/920520510 >> v2:172.21.15.16:3300/0 conn(0x7fc2d009e410 0x7fc2d0133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:11.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.857+0000 7fc2d4e4a700 1 -- 172.21.15.16:0/920520510 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d009a500 con 0x7fc2d009d6f0 2023-09-30T22:36:11.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.857+0000 7fc2d564b700 1 --2- 172.21.15.16:0/920520510 >> v2:172.21.15.16:3300/0 conn(0x7fc2d009e410 0x7fc2d0133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:11.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.858+0000 7fc2d4e4a700 1 --2- 172.21.15.16:0/920520510 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2d009d6f0 0x7fc2d009dac0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4009920 tx=0x7fc2c4009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe39d9d29733c6e7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:11.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.858+0000 7fc2cf7fe700 1 -- 172.21.15.16:0/920520510 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c4011830 con 0x7fc2d009d6f0 2023-09-30T22:36:11.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.858+0000 7fc2cf7fe700 1 -- 172.21.15.16:0/920520510 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2c4011990 con 0x7fc2d009d6f0 2023-09-30T22:36:11.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.858+0000 7fc2cf7fe700 1 -- 172.21.15.16:0/920520510 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c4011d20 con 0x7fc2d009d6f0 2023-09-30T22:36:11.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.859+0000 7fc2d70ae700 1 -- 172.21.15.16:0/920520510 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2d009d6f0 msgr2=0x7fc2d009dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:11.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.859+0000 7fc2d70ae700 1 --2- 172.21.15.16:0/920520510 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2d009d6f0 0x7fc2d009dac0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4009920 tx=0x7fc2c4009c30 comp rx=0 tx=0).stop 2023-09-30T22:36:11.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.859+0000 7fc2d70ae700 1 -- 172.21.15.16:0/920520510 shutdown_connections 2023-09-30T22:36:11.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.859+0000 7fc2d70ae700 1 --2- 172.21.15.16:0/920520510 >> v2:172.21.15.16:3300/0 conn(0x7fc2d009e410 0x7fc2d0133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:11.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.859+0000 7fc2d70ae700 1 --2- 172.21.15.16:0/920520510 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2d009d6f0 0x7fc2d009dac0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:11.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.859+0000 7fc2d70ae700 1 --2- 172.21.15.16:0/920520510 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2d009e000 0x7fc2d01339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:11.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.859+0000 7fc2d70ae700 1 -- 172.21.15.16:0/920520510 >> 172.21.15.16:0/920520510 conn(0x7fc2d0066f50 msgr2=0x7fc2d009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:11.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.859+0000 7fc2d70ae700 1 -- 172.21.15.16:0/920520510 shutdown_connections 2023-09-30T22:36:11.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.859+0000 7fc2d70ae700 1 -- 172.21.15.16:0/920520510 wait complete. 2023-09-30T22:36:11.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.860+0000 7fc2d70ae700 1 Processor -- start 2023-09-30T22:36:11.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.860+0000 7fc2d70ae700 1 -- start start 2023-09-30T22:36:11.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.861+0000 7fc2d70ae700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2d009d6f0 0x7fc2d01c8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:11.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.861+0000 7fc2d70ae700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2d009e000 0x7fc2d01c86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:11.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.861+0000 7fc2d70ae700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2d009e410 0x7fc2d01c8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:11.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.861+0000 7fc2d70ae700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d01a5590 con 0x7fc2d009d6f0 2023-09-30T22:36:11.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.861+0000 7fc2d70ae700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d01a53a0 con 0x7fc2d009e410 2023-09-30T22:36:11.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.861+0000 7fc2d70ae700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d00a30c0 con 0x7fc2d009e000 2023-09-30T22:36:11.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.861+0000 7fc2d4e4a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2d009d6f0 0x7fc2d01c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:11.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.861+0000 7fc2d4e4a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2d009d6f0 0x7fc2d01c8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51868/0 (socket says 172.21.15.16:51868) 2023-09-30T22:36:11.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.861+0000 7fc2d4e4a700 1 -- 172.21.15.16:0/2601905101 learned_addr learned my addr 172.21.15.16:0/2601905101 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:11.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.861+0000 7fc2cffff700 1 --2- 172.21.15.16:0/2601905101 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2d009e000 0x7fc2d01c86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:11.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.862+0000 7fc2d564b700 1 --2- 172.21.15.16:0/2601905101 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2d009e410 0x7fc2d01c8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:11.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.862+0000 7fc2d4e4a700 1 -- 172.21.15.16:0/2601905101 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2d009e000 msgr2=0x7fc2d01c86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:11.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.862+0000 7fc2d4e4a700 1 --2- 172.21.15.16:0/2601905101 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2d009e000 0x7fc2d01c86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:11.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.862+0000 7fc2d4e4a700 1 -- 172.21.15.16:0/2601905101 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2d009e410 msgr2=0x7fc2d01c8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:11.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.862+0000 7fc2d4e4a700 1 --2- 172.21.15.16:0/2601905101 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2d009e410 0x7fc2d01c8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:11.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.862+0000 7fc2d4e4a700 1 -- 172.21.15.16:0/2601905101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2c8009a70 con 0x7fc2d009d6f0 2023-09-30T22:36:11.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.862+0000 7fc2d564b700 1 --2- 172.21.15.16:0/2601905101 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2d009e410 0x7fc2d01c8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:36:11.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.862+0000 7fc2cffff700 1 --2- 172.21.15.16:0/2601905101 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2d009e000 0x7fc2d01c86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:11.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.862+0000 7fc2d4e4a700 1 --2- 172.21.15.16:0/2601905101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2d009d6f0 0x7fc2d01c8190 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4000c00 tx=0x7fc2c4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:11.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.862+0000 7fc2cd7fa700 1 -- 172.21.15.16:0/2601905101 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c4004380 con 0x7fc2d009d6f0 2023-09-30T22:36:11.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.862+0000 7fc2cd7fa700 1 -- 172.21.15.16:0/2601905101 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2c401f070 con 0x7fc2d009d6f0 2023-09-30T22:36:11.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.862+0000 7fc2d70ae700 1 -- 172.21.15.16:0/2601905101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2c4009550 con 0x7fc2d009d6f0 2023-09-30T22:36:11.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.862+0000 7fc2cd7fa700 1 -- 172.21.15.16:0/2601905101 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c40036a0 con 0x7fc2d009d6f0 2023-09-30T22:36:11.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.862+0000 7fc2d70ae700 1 -- 172.21.15.16:0/2601905101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2d01a7320 con 0x7fc2d009d6f0 2023-09-30T22:36:11.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.863+0000 7fc2cd7fa700 1 -- 172.21.15.16:0/2601905101 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc2c4021030 con 0x7fc2d009d6f0 2023-09-30T22:36:11.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.863+0000 7fc2cd7fa700 1 -- 172.21.15.16:0/2601905101 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fc2c40409f0 con 0x7fc2d009d6f0 2023-09-30T22:36:11.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.864+0000 7fc2d70ae700 1 -- 172.21.15.16:0/2601905101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2d0074ea0 con 0x7fc2d009d6f0 2023-09-30T22:36:11.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.869+0000 7fc2cd7fa700 1 -- 172.21.15.16:0/2601905101 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc2c400b9d0 con 0x7fc2d009d6f0 2023-09-30T22:36:11.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.895+0000 7f9127b01700 1 Processor -- start 2023-09-30T22:36:11.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.896+0000 7f9127b01700 1 -- start start 2023-09-30T22:36:11.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.896+0000 7f9127b01700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f91200a0a30 0x7f91200a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:11.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.896+0000 7f9127b01700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f91200a1340 0x7f9120133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:11.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.896+0000 7f9127b01700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f912009edd0 con 0x7f9120133fa0 2023-09-30T22:36:11.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.896+0000 7f9127b01700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f912009e1f0 con 0x7f91200a1340 2023-09-30T22:36:11.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.896+0000 7f9127b01700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f912009dc50 con 0x7f91200a0a30 2023-09-30T22:36:11.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.896+0000 7f912589d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f91200a0a30 0x7f91200a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:11.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.896+0000 7f912589d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f91200a0a30 0x7f91200a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:33356/0 (socket says 172.21.15.16:33356) 2023-09-30T22:36:11.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.896+0000 7f912589d700 1 -- 172.21.15.16:0/769199996 learned_addr learned my addr 172.21.15.16:0/769199996 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:11.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.896+0000 7f912509c700 1 --2- 172.21.15.16:0/769199996 >> v2:172.21.15.16:3300/0 conn(0x7f91200a1340 0x7f9120133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:11.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.897+0000 7f912589d700 1 -- 172.21.15.16:0/769199996 >> v2:172.21.15.16:3300/0 conn(0x7f91200a1340 msgr2=0x7f9120133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:11.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.897+0000 7f912589d700 1 --2- 172.21.15.16:0/769199996 >> v2:172.21.15.16:3300/0 conn(0x7f91200a1340 0x7f9120133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:11.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.897+0000 7f912589d700 1 -- 172.21.15.16:0/769199996 >> v1:172.21.15.16:6789/0 conn(0x7f9120133fa0 legacy=0x7f912013ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:11.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.897+0000 7f912589d700 1 -- 172.21.15.16:0/769199996 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91200a1fe0 con 0x7f91200a0a30 2023-09-30T22:36:11.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.897+0000 7f912589d700 1 --2- 172.21.15.16:0/769199996 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f91200a0a30 0x7f91200a0e00 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f9110009e50 tx=0x7f911000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a57be09839bf6adf server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:11.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.897+0000 7f912489b700 1 -- 172.21.15.16:0/769199996 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911000e070 con 0x7f91200a0a30 2023-09-30T22:36:11.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.897+0000 7f912489b700 1 -- 172.21.15.16:0/769199996 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f911000b910 con 0x7f91200a0a30 2023-09-30T22:36:11.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.897+0000 7f912489b700 1 -- 172.21.15.16:0/769199996 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911000bcc0 con 0x7f91200a0a30 2023-09-30T22:36:11.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.898+0000 7f9127b01700 1 -- 172.21.15.16:0/769199996 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f91200a0a30 msgr2=0x7f91200a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:11.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.898+0000 7f9127b01700 1 --2- 172.21.15.16:0/769199996 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f91200a0a30 0x7f91200a0e00 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f9110009e50 tx=0x7f911000b330 comp rx=0 tx=0).stop 2023-09-30T22:36:11.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.898+0000 7f9127b01700 1 -- 172.21.15.16:0/769199996 shutdown_connections 2023-09-30T22:36:11.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.898+0000 7f9127b01700 1 --2- 172.21.15.16:0/769199996 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f91200a0a30 0x7f91200a0e00 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:11.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.898+0000 7f9127b01700 1 --2- 172.21.15.16:0/769199996 >> v2:172.21.15.16:3300/0 conn(0x7f91200a1340 0x7f9120133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:11.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.898+0000 7f9127b01700 1 -- 172.21.15.16:0/769199996 >> 172.21.15.16:0/769199996 conn(0x7f91200a6550 msgr2=0x7f91200a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:11.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.898+0000 7f9127b01700 1 -- 172.21.15.16:0/769199996 shutdown_connections 2023-09-30T22:36:11.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.898+0000 7f9127b01700 1 -- 172.21.15.16:0/769199996 wait complete. 2023-09-30T22:36:11.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.899+0000 7f9127b01700 1 Processor -- start 2023-09-30T22:36:11.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.899+0000 7f9127b01700 1 -- start start 2023-09-30T22:36:11.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.900+0000 7f9127b01700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f91200a0a30 0x7f912011e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:11.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.900+0000 7f9127b01700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f91200a1340 0x7f912011eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:11.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.900+0000 7f9127b01700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9120133fa0 0x7f912011f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:11.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.900+0000 7f9127b01700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91200a0290 con 0x7f9120133fa0 2023-09-30T22:36:11.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.900+0000 7f9127b01700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91201209e0 con 0x7f91200a0a30 2023-09-30T22:36:11.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.900+0000 7f9127b01700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9120122010 con 0x7f91200a1340 2023-09-30T22:36:11.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.900+0000 7f912509c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f91200a1340 0x7f912011eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:11.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.900+0000 7f912509c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f91200a1340 0x7f912011eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:43942/0 (socket says 172.21.15.16:43942) 2023-09-30T22:36:11.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.900+0000 7f912509c700 1 -- 172.21.15.16:0/1166475124 learned_addr learned my addr 172.21.15.16:0/1166475124 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:11.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.900+0000 7f912589d700 1 --2- 172.21.15.16:0/1166475124 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f91200a0a30 0x7f912011e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:11.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.900+0000 7f912609e700 1 --2- 172.21.15.16:0/1166475124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9120133fa0 0x7f912011f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:11.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.900+0000 7f912509c700 1 -- 172.21.15.16:0/1166475124 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f91200a0a30 msgr2=0x7f912011e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:11.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.900+0000 7f912509c700 1 --2- 172.21.15.16:0/1166475124 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f91200a0a30 0x7f912011e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:11.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.900+0000 7f912509c700 1 -- 172.21.15.16:0/1166475124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9120133fa0 msgr2=0x7f912011f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:11.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.900+0000 7f912509c700 1 --2- 172.21.15.16:0/1166475124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9120133fa0 0x7f912011f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:11.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.900+0000 7f912509c700 1 -- 172.21.15.16:0/1166475124 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9110009b30 con 0x7f91200a1340 2023-09-30T22:36:11.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.900+0000 7f912509c700 1 --2- 172.21.15.16:0/1166475124 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f91200a1340 0x7f912011eda0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f911400e9b0 tx=0x7f911400ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:11.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.901+0000 7f911e7fc700 1 -- 172.21.15.16:0/1166475124 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9114009710 con 0x7f91200a1340 2023-09-30T22:36:11.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.901+0000 7f911e7fc700 1 -- 172.21.15.16:0/1166475124 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f911400c850 con 0x7f91200a1340 2023-09-30T22:36:11.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.901+0000 7f9127b01700 1 -- 172.21.15.16:0/1166475124 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9120120560 con 0x7f91200a1340 2023-09-30T22:36:11.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.901+0000 7f911e7fc700 1 -- 172.21.15.16:0/1166475124 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911400cbc0 con 0x7f91200a1340 2023-09-30T22:36:11.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.901+0000 7f9127b01700 1 -- 172.21.15.16:0/1166475124 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91200a38a0 con 0x7f91200a1340 2023-09-30T22:36:11.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.902+0000 7f911e7fc700 1 -- 172.21.15.16:0/1166475124 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f911400cd20 con 0x7f91200a1340 2023-09-30T22:36:11.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.902+0000 7f911e7fc700 1 -- 172.21.15.16:0/1166475124 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f911403fb60 con 0x7f91200a1340 2023-09-30T22:36:11.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.902+0000 7f9127b01700 1 -- 172.21.15.16:0/1166475124 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9120074ea0 con 0x7f91200a1340 2023-09-30T22:36:11.908 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.907+0000 7f911e7fc700 1 -- 172.21.15.16:0/1166475124 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9114017080 con 0x7f91200a1340 2023-09-30T22:36:11.909 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:36:11.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.942+0000 7f7fa486a700 1 Processor -- start 2023-09-30T22:36:11.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.942+0000 7f7fa486a700 1 -- start start 2023-09-30T22:36:11.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.942+0000 7f7fa486a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7fa006e570 0x7f7fa006e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:11.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.942+0000 7f7fa486a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7fa006ee80 0x7f7fa0134d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:11.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.942+0000 7f7fa486a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fa0070ee0 con 0x7f7fa009eca0 2023-09-30T22:36:11.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.942+0000 7f7fa486a700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fa0071480 con 0x7f7fa006ee80 2023-09-30T22:36:11.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.942+0000 7f7fa486a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa0071b90 con 0x7f7fa006e570 2023-09-30T22:36:11.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.943+0000 7f7f9e59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7fa006e570 0x7f7fa006e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:11.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.943+0000 7f7f9e59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7fa006e570 0x7f7fa006e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:43944/0 (socket says 172.21.15.16:43944) 2023-09-30T22:36:11.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.943+0000 7f7f9e59c700 1 -- 172.21.15.16:0/982409839 learned_addr learned my addr 172.21.15.16:0/982409839 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:11.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.943+0000 7f7f9e59c700 1 -- 172.21.15.16:0/982409839 >> v2:172.21.15.16:3300/0 conn(0x7f7fa006ee80 msgr2=0x7f7fa0134d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:11.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.943+0000 7f7f9e59c700 1 --2- 172.21.15.16:0/982409839 >> v2:172.21.15.16:3300/0 conn(0x7f7fa006ee80 0x7f7fa0134d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:11.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.943+0000 7f7f9e59c700 1 -- 172.21.15.16:0/982409839 >> v1:172.21.15.16:6789/0 conn(0x7f7fa009eca0 legacy=0x7f7fa0134610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:11.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.943+0000 7f7f9e59c700 1 -- 172.21.15.16:0/982409839 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa00711a0 con 0x7f7fa006e570 2023-09-30T22:36:11.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.943+0000 7f7f9e59c700 1 --2- 172.21.15.16:0/982409839 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7fa006e570 0x7f7fa006e940 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f7f900098f0 tx=0x7f7f90009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20b785ea5e94187f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:11.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.944+0000 7f7f9d59a700 1 -- 172.21.15.16:0/982409839 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f90004230 con 0x7f7fa006e570 2023-09-30T22:36:11.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.944+0000 7f7f9d59a700 1 -- 172.21.15.16:0/982409839 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f90004390 con 0x7f7fa006e570 2023-09-30T22:36:11.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.944+0000 7f7f9d59a700 1 -- 172.21.15.16:0/982409839 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f900036a0 con 0x7f7fa006e570 2023-09-30T22:36:11.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.944+0000 7f7fa486a700 1 -- 172.21.15.16:0/982409839 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7fa006e570 msgr2=0x7f7fa006e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:11.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.944+0000 7f7fa486a700 1 --2- 172.21.15.16:0/982409839 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7fa006e570 0x7f7fa006e940 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f7f900098f0 tx=0x7f7f90009c00 comp rx=0 tx=0).stop 2023-09-30T22:36:11.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.945+0000 7f7fa486a700 1 -- 172.21.15.16:0/982409839 shutdown_connections 2023-09-30T22:36:11.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.945+0000 7f7fa486a700 1 --2- 172.21.15.16:0/982409839 >> v2:172.21.15.16:3300/0 conn(0x7f7fa006ee80 0x7f7fa0134d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:11.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.945+0000 7f7fa486a700 1 --2- 172.21.15.16:0/982409839 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7fa006e570 0x7f7fa006e940 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:11.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.945+0000 7f7fa486a700 1 -- 172.21.15.16:0/982409839 >> 172.21.15.16:0/982409839 conn(0x7f7fa00a2aa0 msgr2=0x7f7fa00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:11.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.945+0000 7f7fa486a700 1 -- 172.21.15.16:0/982409839 shutdown_connections 2023-09-30T22:36:11.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.945+0000 7f7fa486a700 1 -- 172.21.15.16:0/982409839 wait complete. 2023-09-30T22:36:11.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.946+0000 7f7fa486a700 1 Processor -- start 2023-09-30T22:36:11.947 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.946+0000 7f7fa486a700 1 -- start start 2023-09-30T22:36:11.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.947+0000 7f7fa486a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7fa006e570 0x7f7fa0134ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:11.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.947+0000 7f7fa486a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fa006ee80 0x7f7fa01388a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:11.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.947+0000 7f7fa486a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7fa0138de0 0x7f7fa0139210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:11.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.947+0000 7f7fa486a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa0139a10 con 0x7f7fa006ee80 2023-09-30T22:36:11.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.947+0000 7f7fa486a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa0139750 con 0x7f7fa0138de0 2023-09-30T22:36:11.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.947+0000 7f7fa486a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa0135a90 con 0x7f7fa006e570 2023-09-30T22:36:11.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.947+0000 7f7f9ed9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7fa0138de0 0x7f7fa0139210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:11.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.947+0000 7f7f9e59c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7fa006e570 0x7f7fa0134ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:11.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.947+0000 7f7f9ed9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7fa0138de0 0x7f7fa0139210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:33376/0 (socket says 172.21.15.16:33376) 2023-09-30T22:36:11.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.947+0000 7f7f9ed9d700 1 -- 172.21.15.16:0/364128187 learned_addr learned my addr 172.21.15.16:0/364128187 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:11.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.947+0000 7f7f9dd9b700 1 --2- 172.21.15.16:0/364128187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fa006ee80 0x7f7fa01388a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:11.948 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.947+0000 7f7f9e59c700 1 -- 172.21.15.16:0/364128187 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7fa0138de0 msgr2=0x7f7fa0139210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:11.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.947+0000 7f7f9e59c700 1 --2- 172.21.15.16:0/364128187 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7fa0138de0 0x7f7fa0139210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:11.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.947+0000 7f7f9e59c700 1 -- 172.21.15.16:0/364128187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fa006ee80 msgr2=0x7f7fa01388a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:11.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.947+0000 7f7f9e59c700 1 --2- 172.21.15.16:0/364128187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fa006ee80 0x7f7fa01388a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:11.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.947+0000 7f7f9e59c700 1 -- 172.21.15.16:0/364128187 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f90009550 con 0x7f7fa006e570 2023-09-30T22:36:11.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.947+0000 7f7f9dd9b700 1 --2- 172.21.15.16:0/364128187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fa006ee80 0x7f7fa01388a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:36:11.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.947+0000 7f7f9e59c700 1 --2- 172.21.15.16:0/364128187 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7fa006e570 0x7f7fa0134ad0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f7f90015000 tx=0x7f7f90004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:11.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.948+0000 7f7f8effd700 1 -- 172.21.15.16:0/364128187 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f90003d80 con 0x7f7fa006e570 2023-09-30T22:36:11.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.948+0000 7f7f8effd700 1 -- 172.21.15.16:0/364128187 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f9001f070 con 0x7f7fa006e570 2023-09-30T22:36:11.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.948+0000 7f7f8effd700 1 -- 172.21.15.16:0/364128187 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9000b8a0 con 0x7f7fa006e570 2023-09-30T22:36:11.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.948+0000 7f7fa486a700 1 -- 172.21.15.16:0/364128187 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fa0070610 con 0x7f7fa006e570 2023-09-30T22:36:11.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.948+0000 7f7fa486a700 1 -- 172.21.15.16:0/364128187 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fa0140860 con 0x7f7fa006e570 2023-09-30T22:36:11.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.949+0000 7f7fa486a700 1 -- 172.21.15.16:0/364128187 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fa0137960 con 0x7f7fa006e570 2023-09-30T22:36:11.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.950+0000 7f7f8effd700 1 -- 172.21.15.16:0/364128187 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7f90021030 con 0x7f7fa006e570 2023-09-30T22:36:11.951 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.950+0000 7f7f8effd700 1 -- 172.21.15.16:0/364128187 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f7f90041540 con 0x7f7fa006e570 2023-09-30T22:36:11.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:11.959+0000 7f7f8effd700 1 -- 172.21.15.16:0/364128187 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7f9000be80 con 0x7f7fa006e570 2023-09-30T22:36:12.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.052+0000 7fe2a2948700 1 Processor -- start 2023-09-30T22:36:12.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.052+0000 7fe2a2948700 1 -- start start 2023-09-30T22:36:12.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.052+0000 7fe2a2948700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe29c09e1d0 0x7fe29c135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:12.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.052+0000 7fe2a2948700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe29c135e50 0x7fe29c138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:12.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.053+0000 7fe2a2948700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe29c09f130 con 0x7fe29c09d6f0 2023-09-30T22:36:12.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.053+0000 7fe2a2948700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe29c09f520 con 0x7fe29c135e50 2023-09-30T22:36:12.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.053+0000 7fe2a2948700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c09e930 con 0x7fe29c09e1d0 2023-09-30T22:36:12.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.053+0000 7fe29b7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe29c09e1d0 0x7fe29c135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:12.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.053+0000 7fe29b7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe29c09e1d0 0x7fe29c135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:33382/0 (socket says 172.21.15.16:33382) 2023-09-30T22:36:12.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.053+0000 7fe29b7fe700 1 -- 172.21.15.16:0/2292324606 learned_addr learned my addr 172.21.15.16:0/2292324606 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:12.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.053+0000 7fe2a0ee5700 1 --2- 172.21.15.16:0/2292324606 >> v2:172.21.15.16:3300/0 conn(0x7fe29c135e50 0x7fe29c138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:12.055 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.053+0000 7fe29b7fe700 1 -- 172.21.15.16:0/2292324606 >> v2:172.21.15.16:3300/0 conn(0x7fe29c135e50 msgr2=0x7fe29c138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:12.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.053+0000 7fe29b7fe700 1 --2- 172.21.15.16:0/2292324606 >> v2:172.21.15.16:3300/0 conn(0x7fe29c135e50 0x7fe29c138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:12.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.053+0000 7fe29b7fe700 1 -- 172.21.15.16:0/2292324606 >> v1:172.21.15.16:6789/0 conn(0x7fe29c09d6f0 legacy=0x7fe29c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:12.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.053+0000 7fe29b7fe700 1 -- 172.21.15.16:0/2292324606 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe29c09b2d0 con 0x7fe29c09e1d0 2023-09-30T22:36:12.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.053+0000 7fe29b7fe700 1 --2- 172.21.15.16:0/2292324606 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe29c09e1d0 0x7fe29c135910 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fe290009920 tx=0x7fe290009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d94f58e8eb725dbf server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:12.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.053+0000 7fe29affd700 1 -- 172.21.15.16:0/2292324606 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe290011830 con 0x7fe29c09e1d0 2023-09-30T22:36:12.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.054+0000 7fe29affd700 1 -- 172.21.15.16:0/2292324606 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe290011990 con 0x7fe29c09e1d0 2023-09-30T22:36:12.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.054+0000 7fe29affd700 1 -- 172.21.15.16:0/2292324606 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe290011d20 con 0x7fe29c09e1d0 2023-09-30T22:36:12.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.054+0000 7fe2a2948700 1 -- 172.21.15.16:0/2292324606 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe29c09e1d0 msgr2=0x7fe29c135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:12.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.054+0000 7fe2a2948700 1 --2- 172.21.15.16:0/2292324606 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe29c09e1d0 0x7fe29c135910 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fe290009920 tx=0x7fe290009c30 comp rx=0 tx=0).stop 2023-09-30T22:36:12.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.054+0000 7fe2a2948700 1 -- 172.21.15.16:0/2292324606 shutdown_connections 2023-09-30T22:36:12.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.054+0000 7fe2a2948700 1 --2- 172.21.15.16:0/2292324606 >> v2:172.21.15.16:3300/0 conn(0x7fe29c135e50 0x7fe29c138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:12.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.054+0000 7fe2a2948700 1 --2- 172.21.15.16:0/2292324606 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe29c09e1d0 0x7fe29c135910 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:12.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.054+0000 7fe2a2948700 1 -- 172.21.15.16:0/2292324606 >> 172.21.15.16:0/2292324606 conn(0x7fe29c066f50 msgr2=0x7fe29c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:12.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.054+0000 7fe2a2948700 1 -- 172.21.15.16:0/2292324606 shutdown_connections 2023-09-30T22:36:12.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.054+0000 7fe2a2948700 1 -- 172.21.15.16:0/2292324606 wait complete. 2023-09-30T22:36:12.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.055+0000 7fe2a2948700 1 Processor -- start 2023-09-30T22:36:12.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.055+0000 7fe2a2948700 1 -- start start 2023-09-30T22:36:12.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.056+0000 7fe2a2948700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe29c09d6f0 0x7fe29c11e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:12.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.056+0000 7fe2a2948700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe29c09e1d0 0x7fe29c11ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:12.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.056+0000 7fe2a2948700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe29c135e50 0x7fe29c11f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:12.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.056+0000 7fe2a2948700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c13a260 con 0x7fe29c135e50 2023-09-30T22:36:12.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.056+0000 7fe2a2948700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c09c700 con 0x7fe29c09e1d0 2023-09-30T22:36:12.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.056+0000 7fe2a2948700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe29c121ba0 con 0x7fe29c09d6f0 2023-09-30T22:36:12.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.056+0000 7fe29b7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe29c09e1d0 0x7fe29c11ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:12.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.056+0000 7fe29b7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe29c09e1d0 0x7fe29c11ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:33388/0 (socket says 172.21.15.16:33388) 2023-09-30T22:36:12.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.056+0000 7fe29b7fe700 1 -- 172.21.15.16:0/4254212261 learned_addr learned my addr 172.21.15.16:0/4254212261 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:12.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.056+0000 7fe2a0ee5700 1 --2- 172.21.15.16:0/4254212261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe29c135e50 0x7fe29c11f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:12.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.056+0000 7fe29bfff700 1 --2- 172.21.15.16:0/4254212261 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe29c09d6f0 0x7fe29c11e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:12.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.057+0000 7fe29b7fe700 1 -- 172.21.15.16:0/4254212261 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe29c09d6f0 msgr2=0x7fe29c11e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:12.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.057+0000 7fe29b7fe700 1 --2- 172.21.15.16:0/4254212261 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe29c09d6f0 0x7fe29c11e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:12.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.057+0000 7fe29b7fe700 1 -- 172.21.15.16:0/4254212261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe29c135e50 msgr2=0x7fe29c11f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:12.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.057+0000 7fe29b7fe700 1 --2- 172.21.15.16:0/4254212261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe29c135e50 0x7fe29c11f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:12.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.057+0000 7fe29b7fe700 1 -- 172.21.15.16:0/4254212261 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe290009550 con 0x7fe29c09e1d0 2023-09-30T22:36:12.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.057+0000 7fe2a0ee5700 1 --2- 172.21.15.16:0/4254212261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe29c135e50 0x7fe29c11f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:12.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.057+0000 7fe29b7fe700 1 --2- 172.21.15.16:0/4254212261 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe29c09e1d0 0x7fe29c11ee30 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fe2900098f0 tx=0x7fe290004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:12.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.057+0000 7fe298ff9700 1 -- 172.21.15.16:0/4254212261 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe290004380 con 0x7fe29c09e1d0 2023-09-30T22:36:12.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.057+0000 7fe298ff9700 1 -- 172.21.15.16:0/4254212261 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe29001f070 con 0x7fe29c09e1d0 2023-09-30T22:36:12.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.057+0000 7fe2a2948700 1 -- 172.21.15.16:0/4254212261 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe29c122130 con 0x7fe29c09e1d0 2023-09-30T22:36:12.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.057+0000 7fe298ff9700 1 -- 172.21.15.16:0/4254212261 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2900036a0 con 0x7fe29c09e1d0 2023-09-30T22:36:12.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.057+0000 7fe2a2948700 1 -- 172.21.15.16:0/4254212261 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe29c122ba0 con 0x7fe29c09e1d0 2023-09-30T22:36:12.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.058+0000 7fe2a2948700 1 -- 172.21.15.16:0/4254212261 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe29c074ea0 con 0x7fe29c09e1d0 2023-09-30T22:36:12.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.058+0000 7fe298ff9700 1 -- 172.21.15.16:0/4254212261 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe290021030 con 0x7fe29c09e1d0 2023-09-30T22:36:12.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.059+0000 7fe298ff9700 1 -- 172.21.15.16:0/4254212261 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fe290040920 con 0x7fe29c09e1d0 2023-09-30T22:36:12.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.068+0000 7fe298ff9700 1 -- 172.21.15.16:0/4254212261 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe2900239c0 con 0x7fe29c09e1d0 2023-09-30T22:36:12.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.131+0000 7f9127b01700 1 -- 172.21.15.16:0/1166475124 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f91200a28e0 con 0x7f91200a1340 2023-09-30T22:36:12.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.131+0000 7f911e7fc700 1 -- 172.21.15.16:0/1166475124 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9114021020 con 0x7f91200a1340 2023-09-30T22:36:12.132 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:36:12.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.134+0000 7f9127b01700 1 -- 172.21.15.16:0/1166475124 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f91200a1340 msgr2=0x7f912011eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:12.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.134+0000 7f9127b01700 1 --2- 172.21.15.16:0/1166475124 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f91200a1340 0x7f912011eda0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f911400e9b0 tx=0x7f911400ecc0 comp rx=0 tx=0).stop 2023-09-30T22:36:12.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.134+0000 7f9127b01700 1 -- 172.21.15.16:0/1166475124 shutdown_connections 2023-09-30T22:36:12.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.134+0000 7f9127b01700 1 --2- 172.21.15.16:0/1166475124 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f91200a0a30 0x7f912011e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:12.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.134+0000 7f9127b01700 1 --2- 172.21.15.16:0/1166475124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9120133fa0 0x7f912011f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:12.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.134+0000 7f9127b01700 1 --2- 172.21.15.16:0/1166475124 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f91200a1340 0x7f912011eda0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:12.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.134+0000 7f9127b01700 1 -- 172.21.15.16:0/1166475124 >> 172.21.15.16:0/1166475124 conn(0x7f91200a6550 msgr2=0x7f912009a830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:12.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.134+0000 7f9127b01700 1 -- 172.21.15.16:0/1166475124 shutdown_connections 2023-09-30T22:36:12.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.134+0000 7f9127b01700 1 -- 172.21.15.16:0/1166475124 wait complete. 2023-09-30T22:36:12.149 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:36:12.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.165+0000 7fc2d70ae700 1 -- 172.21.15.16:0/2601905101 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc2d01b84f0 con 0x7fc2d009d6f0 2023-09-30T22:36:12.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.165+0000 7fc2cd7fa700 1 -- 172.21.15.16:0/2601905101 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc2c4015020 con 0x7fc2d009d6f0 2023-09-30T22:36:12.166 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:36:12.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.168+0000 7fc2d70ae700 1 -- 172.21.15.16:0/2601905101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2d009d6f0 msgr2=0x7fc2d01c8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:12.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.168+0000 7fc2d70ae700 1 --2- 172.21.15.16:0/2601905101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2d009d6f0 0x7fc2d01c8190 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4000c00 tx=0x7fc2c4004750 comp rx=0 tx=0).stop 2023-09-30T22:36:12.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.168+0000 7fc2d70ae700 1 -- 172.21.15.16:0/2601905101 shutdown_connections 2023-09-30T22:36:12.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.168+0000 7fc2d70ae700 1 --2- 172.21.15.16:0/2601905101 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2d009e410 0x7fc2d01c8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:12.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.168+0000 7fc2d70ae700 1 --2- 172.21.15.16:0/2601905101 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2d009d6f0 0x7fc2d01c8190 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:12.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.168+0000 7fc2d70ae700 1 --2- 172.21.15.16:0/2601905101 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2d009e000 0x7fc2d01c86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:12.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.168+0000 7fc2d70ae700 1 -- 172.21.15.16:0/2601905101 >> 172.21.15.16:0/2601905101 conn(0x7fc2d0066f50 msgr2=0x7fc2d01bbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:12.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.168+0000 7fc2d70ae700 1 -- 172.21.15.16:0/2601905101 shutdown_connections 2023-09-30T22:36:12.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.168+0000 7fc2d70ae700 1 -- 172.21.15.16:0/2601905101 wait complete. 2023-09-30T22:36:12.182 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:36:12.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.246+0000 7f7fa486a700 1 -- 172.21.15.16:0/364128187 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7fa0137960 con 0x7f7fa006e570 2023-09-30T22:36:12.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.246+0000 7f7f8effd700 1 -- 172.21.15.16:0/364128187 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7f90023bc0 con 0x7f7fa006e570 2023-09-30T22:36:12.247 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:36:12.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.249+0000 7f7fa486a700 1 -- 172.21.15.16:0/364128187 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7fa006e570 msgr2=0x7f7fa0134ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:12.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.249+0000 7f7fa486a700 1 --2- 172.21.15.16:0/364128187 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7fa006e570 0x7f7fa0134ad0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f7f90015000 tx=0x7f7f90004750 comp rx=0 tx=0).stop 2023-09-30T22:36:12.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.249+0000 7f7fa486a700 1 -- 172.21.15.16:0/364128187 shutdown_connections 2023-09-30T22:36:12.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.249+0000 7f7fa486a700 1 --2- 172.21.15.16:0/364128187 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7fa0138de0 0x7f7fa0139210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:12.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.249+0000 7f7fa486a700 1 --2- 172.21.15.16:0/364128187 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7fa006e570 0x7f7fa0134ad0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:12.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.249+0000 7f7fa486a700 1 --2- 172.21.15.16:0/364128187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fa006ee80 0x7f7fa01388a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:12.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.249+0000 7f7fa486a700 1 -- 172.21.15.16:0/364128187 >> 172.21.15.16:0/364128187 conn(0x7f7fa00a2aa0 msgr2=0x7f7fa00a1410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:12.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.249+0000 7f7fa486a700 1 -- 172.21.15.16:0/364128187 shutdown_connections 2023-09-30T22:36:12.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.249+0000 7f7fa486a700 1 -- 172.21.15.16:0/364128187 wait complete. 2023-09-30T22:36:12.263 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:36:12.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.275+0000 7fe2a2948700 1 -- 172.21.15.16:0/4254212261 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe29c121d10 con 0x7fe29c09e1d0 2023-09-30T22:36:12.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.275+0000 7fe298ff9700 1 -- 172.21.15.16:0/4254212261 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe290015030 con 0x7fe29c09e1d0 2023-09-30T22:36:12.276 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:36:12.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.278+0000 7fe2a2948700 1 -- 172.21.15.16:0/4254212261 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe29c09e1d0 msgr2=0x7fe29c11ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:12.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.278+0000 7fe2a2948700 1 --2- 172.21.15.16:0/4254212261 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe29c09e1d0 0x7fe29c11ee30 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fe2900098f0 tx=0x7fe290004750 comp rx=0 tx=0).stop 2023-09-30T22:36:12.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.278+0000 7fe2a2948700 1 -- 172.21.15.16:0/4254212261 shutdown_connections 2023-09-30T22:36:12.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.278+0000 7fe2a2948700 1 --2- 172.21.15.16:0/4254212261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe29c135e50 0x7fe29c11f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:12.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.278+0000 7fe2a2948700 1 --2- 172.21.15.16:0/4254212261 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe29c09d6f0 0x7fe29c11e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:12.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.278+0000 7fe2a2948700 1 --2- 172.21.15.16:0/4254212261 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe29c09e1d0 0x7fe29c11ee30 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:12.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.278+0000 7fe2a2948700 1 -- 172.21.15.16:0/4254212261 >> 172.21.15.16:0/4254212261 conn(0x7fe29c066f50 msgr2=0x7fe29c0a1580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:12.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.278+0000 7fe2a2948700 1 -- 172.21.15.16:0/4254212261 shutdown_connections 2023-09-30T22:36:12.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:12.278+0000 7fe2a2948700 1 -- 172.21.15.16:0/4254212261 wait complete. 2023-09-30T22:36:12.292 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:36:13.150 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:36:13.183 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:36:13.263 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:36:13.292 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:36:13.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.308+0000 7fc6a1c89700 1 Processor -- start 2023-09-30T22:36:13.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.308+0000 7fc6a1c89700 1 -- start start 2023-09-30T22:36:13.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.309+0000 7fc6a1c89700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc69c130540 0x7fc69c130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:13.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.309+0000 7fc6a1c89700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc69c130e50 0x7fc69c06f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:13.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.309+0000 7fc6a1c89700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc69c06fb20 0x7fc69c06ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:13.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.309+0000 7fc6a1c89700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc69c09a390 con 0x7fc69c130540 2023-09-30T22:36:13.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.309+0000 7fc6a1c89700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c0716a0 con 0x7fc69c06fb20 2023-09-30T22:36:13.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.309+0000 7fc6a1c89700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c071290 con 0x7fc69c130e50 2023-09-30T22:36:13.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.309+0000 7fc69b7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc69c130540 0x7fc69c130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:13.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.309+0000 7fc69b7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc69c130540 0x7fc69c130910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51926/0 (socket says 172.21.15.16:51926) 2023-09-30T22:36:13.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.309+0000 7fc69b7fe700 1 -- 172.21.15.16:0/2362794370 learned_addr learned my addr 172.21.15.16:0/2362794370 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:13.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.309+0000 7fc69affd700 1 --2- 172.21.15.16:0/2362794370 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc69c130e50 0x7fc69c06f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:13.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.309+0000 7fc69bfff700 1 --2- 172.21.15.16:0/2362794370 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc69c06fb20 0x7fc69c06ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:13.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.309+0000 7fc69b7fe700 1 -- 172.21.15.16:0/2362794370 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc69c130e50 msgr2=0x7fc69c06f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:13.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.309+0000 7fc69b7fe700 1 --2- 172.21.15.16:0/2362794370 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc69c130e50 0x7fc69c06f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:13.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.309+0000 7fc69b7fe700 1 -- 172.21.15.16:0/2362794370 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc69c06fb20 msgr2=0x7fc69c06ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:13.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.309+0000 7fc69b7fe700 1 --2- 172.21.15.16:0/2362794370 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc69c06fb20 0x7fc69c06ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:13.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.309+0000 7fc69b7fe700 1 -- 172.21.15.16:0/2362794370 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc69c130340 con 0x7fc69c130540 2023-09-30T22:36:13.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.309+0000 7fc69bfff700 1 --2- 172.21.15.16:0/2362794370 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc69c06fb20 0x7fc69c06ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:36:13.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.310+0000 7fc69b7fe700 1 --2- 172.21.15.16:0/2362794370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc69c130540 0x7fc69c130910 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fc68c0098f0 tx=0x7fc68c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dc049e25f669feb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:13.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.310+0000 7fc69a7fc700 1 -- 172.21.15.16:0/2362794370 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68c011830 con 0x7fc69c130540 2023-09-30T22:36:13.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.310+0000 7fc69a7fc700 1 -- 172.21.15.16:0/2362794370 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc68c011990 con 0x7fc69c130540 2023-09-30T22:36:13.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.310+0000 7fc69a7fc700 1 -- 172.21.15.16:0/2362794370 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc68c011d20 con 0x7fc69c130540 2023-09-30T22:36:13.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.311+0000 7fc6a1c89700 1 -- 172.21.15.16:0/2362794370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc69c130540 msgr2=0x7fc69c130910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:13.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.311+0000 7fc6a1c89700 1 --2- 172.21.15.16:0/2362794370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc69c130540 0x7fc69c130910 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fc68c0098f0 tx=0x7fc68c009c00 comp rx=0 tx=0).stop 2023-09-30T22:36:13.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.311+0000 7fc6a1c89700 1 -- 172.21.15.16:0/2362794370 shutdown_connections 2023-09-30T22:36:13.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.311+0000 7fc6a1c89700 1 --2- 172.21.15.16:0/2362794370 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc69c06fb20 0x7fc69c06ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:13.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.311+0000 7fc6a1c89700 1 --2- 172.21.15.16:0/2362794370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc69c130540 0x7fc69c130910 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:13.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.311+0000 7fc6a1c89700 1 --2- 172.21.15.16:0/2362794370 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc69c130e50 0x7fc69c06f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:13.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.311+0000 7fc6a1c89700 1 -- 172.21.15.16:0/2362794370 >> 172.21.15.16:0/2362794370 conn(0x7fc69c0a2aa0 msgr2=0x7fc69c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:13.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.311+0000 7fc6a1c89700 1 -- 172.21.15.16:0/2362794370 shutdown_connections 2023-09-30T22:36:13.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.311+0000 7fc6a1c89700 1 -- 172.21.15.16:0/2362794370 wait complete. 2023-09-30T22:36:13.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.312+0000 7fc6a1c89700 1 Processor -- start 2023-09-30T22:36:13.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.313+0000 7fc6a1c89700 1 -- start start 2023-09-30T22:36:13.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.313+0000 7fc6a1c89700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc69c06fb20 0x7fc69c11a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:13.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.313+0000 7fc6a1c89700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc69c130540 0x7fc69c11aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:13.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.313+0000 7fc6a1c89700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc69c130e50 0x7fc69c11af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:13.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.313+0000 7fc6a1c89700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c0a0b50 con 0x7fc69c06fb20 2023-09-30T22:36:13.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.313+0000 7fc6a1c89700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c11d3a0 con 0x7fc69c130540 2023-09-30T22:36:13.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.313+0000 7fc6a1c89700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc69c0718e0 con 0x7fc69c130e50 2023-09-30T22:36:13.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.314+0000 7fc69b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc69c06fb20 0x7fc69c11a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:13.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.314+0000 7fc69affd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc69c130540 0x7fc69c11aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:13.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.314+0000 7fc69bfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc69c130e50 0x7fc69c11af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:13.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.314+0000 7fc69b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc69c06fb20 0x7fc69c11a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51940/0 (socket says 172.21.15.16:51940) 2023-09-30T22:36:13.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.314+0000 7fc69bfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc69c130e50 0x7fc69c11af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:43980/0 (socket says 172.21.15.16:43980) 2023-09-30T22:36:13.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.314+0000 7fc69affd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc69c130540 0x7fc69c11aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:33408/0 (socket says 172.21.15.16:33408) 2023-09-30T22:36:13.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.314+0000 7fc69b7fe700 1 -- 172.21.15.16:0/4059561687 learned_addr learned my addr 172.21.15.16:0/4059561687 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:13.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.314+0000 7fc69bfff700 1 -- 172.21.15.16:0/4059561687 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc69c130540 msgr2=0x7fc69c11aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:13.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.314+0000 7fc69bfff700 1 --2- 172.21.15.16:0/4059561687 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc69c130540 0x7fc69c11aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:13.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.314+0000 7fc69bfff700 1 -- 172.21.15.16:0/4059561687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc69c06fb20 msgr2=0x7fc69c11a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:13.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.314+0000 7fc69bfff700 1 --2- 172.21.15.16:0/4059561687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc69c06fb20 0x7fc69c11a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:13.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.314+0000 7fc69bfff700 1 -- 172.21.15.16:0/4059561687 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc68c009550 con 0x7fc69c130e50 2023-09-30T22:36:13.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.314+0000 7fc69b7fe700 1 --2- 172.21.15.16:0/4059561687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc69c06fb20 0x7fc69c11a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:13.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.314+0000 7fc69bfff700 1 --2- 172.21.15.16:0/4059561687 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc69c130e50 0x7fc69c11af50 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fc694007ef0 tx=0x7fc69400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:13.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.315+0000 7fc683fff700 1 -- 172.21.15.16:0/4059561687 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69400dde0 con 0x7fc69c130e50 2023-09-30T22:36:13.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.315+0000 7fc683fff700 1 -- 172.21.15.16:0/4059561687 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc69400f040 con 0x7fc69c130e50 2023-09-30T22:36:13.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.315+0000 7fc6a1c89700 1 -- 172.21.15.16:0/4059561687 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc69c11c1f0 con 0x7fc69c130e50 2023-09-30T22:36:13.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.315+0000 7fc683fff700 1 -- 172.21.15.16:0/4059561687 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc694003c60 con 0x7fc69c130e50 2023-09-30T22:36:13.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.315+0000 7fc6a1c89700 1 -- 172.21.15.16:0/4059561687 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc69c11dff0 con 0x7fc69c130e50 2023-09-30T22:36:13.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.316+0000 7fc683fff700 1 -- 172.21.15.16:0/4059561687 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc694004030 con 0x7fc69c130e50 2023-09-30T22:36:13.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.316+0000 7fc683fff700 1 -- 172.21.15.16:0/4059561687 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fc69403f880 con 0x7fc69c130e50 2023-09-30T22:36:13.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.316+0000 7fc6a1c89700 1 -- 172.21.15.16:0/4059561687 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc69c09cd40 con 0x7fc69c130e50 2023-09-30T22:36:13.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.322+0000 7fc683fff700 1 -- 172.21.15.16:0/4059561687 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc69401a020 con 0x7fc69c130e50 2023-09-30T22:36:13.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.364+0000 7faaaef52700 1 Processor -- start 2023-09-30T22:36:13.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.364+0000 7faaaef52700 1 -- start start 2023-09-30T22:36:13.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.365+0000 7faaaef52700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faaa8133420 0x7faaa81337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:13.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.365+0000 7faaaef52700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faaa8133e30 0x7faaa812ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:13.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.365+0000 7faaaef52700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faaa8134260 0x7faaa81304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:13.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.365+0000 7faaaef52700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faaa806b990 con 0x7faaa8133e30 2023-09-30T22:36:13.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.365+0000 7faaaef52700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaa80a0590 con 0x7faaa8133420 2023-09-30T22:36:13.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.365+0000 7faaaef52700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faaa809fb80 con 0x7faaa8134260 2023-09-30T22:36:13.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.365+0000 7faaad4ef700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faaa8134260 0x7faaa81304d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:13.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.365+0000 7faaa7fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7faaa8133e30 0x7faaa812ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:13.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.365+0000 7faaaccee700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faaa8133420 0x7faaa81337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:13.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.365+0000 7faaad4ef700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faaa8134260 0x7faaa81304d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:43984/0 (socket says 172.21.15.16:43984) 2023-09-30T22:36:13.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.365+0000 7faaad4ef700 1 -- 172.21.15.16:0/2235025789 learned_addr learned my addr 172.21.15.16:0/2235025789 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:13.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.365+0000 7faaad4ef700 1 -- 172.21.15.16:0/2235025789 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faaa8133420 msgr2=0x7faaa81337f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:13.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.365+0000 7faaad4ef700 1 --2- 172.21.15.16:0/2235025789 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faaa8133420 0x7faaa81337f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:13.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.365+0000 7faaad4ef700 1 -- 172.21.15.16:0/2235025789 >> v2:172.21.15.16:3300/0 conn(0x7faaa8133e30 msgr2=0x7faaa812ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:13.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.365+0000 7faaad4ef700 1 --2- 172.21.15.16:0/2235025789 >> v2:172.21.15.16:3300/0 conn(0x7faaa8133e30 0x7faaa812ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:13.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.365+0000 7faaad4ef700 1 -- 172.21.15.16:0/2235025789 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaa80a0030 con 0x7faaa8134260 2023-09-30T22:36:13.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.365+0000 7faaa7fff700 1 --2- 172.21.15.16:0/2235025789 >> v2:172.21.15.16:3300/0 conn(0x7faaa8133e30 0x7faaa812ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:13.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.366+0000 7faaad4ef700 1 --2- 172.21.15.16:0/2235025789 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faaa8134260 0x7faaa81304d0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7faaa0009e50 tx=0x7faaa000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d34b84ecb901039 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:13.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.366+0000 7faaa77fe700 1 -- 172.21.15.16:0/2235025789 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa000e070 con 0x7faaa8134260 2023-09-30T22:36:13.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.366+0000 7faaa77fe700 1 -- 172.21.15.16:0/2235025789 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaa000b910 con 0x7faaa8134260 2023-09-30T22:36:13.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.366+0000 7faaa77fe700 1 -- 172.21.15.16:0/2235025789 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa000bcc0 con 0x7faaa8134260 2023-09-30T22:36:13.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.367+0000 7faaaef52700 1 -- 172.21.15.16:0/2235025789 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faaa8134260 msgr2=0x7faaa81304d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:13.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.367+0000 7faaaef52700 1 --2- 172.21.15.16:0/2235025789 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faaa8134260 0x7faaa81304d0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7faaa0009e50 tx=0x7faaa000b330 comp rx=0 tx=0).stop 2023-09-30T22:36:13.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.367+0000 7faaaef52700 1 -- 172.21.15.16:0/2235025789 shutdown_connections 2023-09-30T22:36:13.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.367+0000 7faaaef52700 1 --2- 172.21.15.16:0/2235025789 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faaa8133420 0x7faaa81337f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:13.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.367+0000 7faaaef52700 1 --2- 172.21.15.16:0/2235025789 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faaa8134260 0x7faaa81304d0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:13.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.367+0000 7faaaef52700 1 --2- 172.21.15.16:0/2235025789 >> v2:172.21.15.16:3300/0 conn(0x7faaa8133e30 0x7faaa812ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:13.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.367+0000 7faaaef52700 1 -- 172.21.15.16:0/2235025789 >> 172.21.15.16:0/2235025789 conn(0x7faaa80a4210 msgr2=0x7faaa80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:13.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.367+0000 7faaaef52700 1 -- 172.21.15.16:0/2235025789 shutdown_connections 2023-09-30T22:36:13.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.367+0000 7faaaef52700 1 -- 172.21.15.16:0/2235025789 wait complete. 2023-09-30T22:36:13.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.368+0000 7faaaef52700 1 Processor -- start 2023-09-30T22:36:13.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.368+0000 7faaaef52700 1 -- start start 2023-09-30T22:36:13.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.369+0000 7faaaef52700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faaa8133420 0x7faaa809c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:13.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.369+0000 7faaaef52700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faaa8133e30 0x7faaa809c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:13.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.369+0000 7faaaef52700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faaa8134260 0x7faaa809cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:13.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.369+0000 7faaaef52700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa809de50 con 0x7faaa8133e30 2023-09-30T22:36:13.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.369+0000 7faaaef52700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaa809dc60 con 0x7faaa8134260 2023-09-30T22:36:13.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.369+0000 7faaaef52700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7faaa811f0c0 con 0x7faaa8133420 2023-09-30T22:36:13.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.369+0000 7faaaccee700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faaa8133420 0x7faaa809c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:13.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.369+0000 7faaaccee700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faaa8133420 0x7faaa809c0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:43994/0 (socket says 172.21.15.16:43994) 2023-09-30T22:36:13.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.369+0000 7faaaccee700 1 -- 172.21.15.16:0/214522477 learned_addr learned my addr 172.21.15.16:0/214522477 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:13.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.369+0000 7faaad4ef700 1 --2- 172.21.15.16:0/214522477 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faaa8134260 0x7faaa809cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:13.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.369+0000 7faaa7fff700 1 --2- 172.21.15.16:0/214522477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faaa8133e30 0x7faaa809c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:13.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.369+0000 7faaaccee700 1 -- 172.21.15.16:0/214522477 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faaa8134260 msgr2=0x7faaa809cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:13.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.369+0000 7faaaccee700 1 --2- 172.21.15.16:0/214522477 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faaa8134260 0x7faaa809cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:13.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.369+0000 7faaaccee700 1 -- 172.21.15.16:0/214522477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faaa8133e30 msgr2=0x7faaa809c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:13.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.369+0000 7faaaccee700 1 --2- 172.21.15.16:0/214522477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faaa8133e30 0x7faaa809c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:13.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.369+0000 7faaaccee700 1 -- 172.21.15.16:0/214522477 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa940094a0 con 0x7faaa8133420 2023-09-30T22:36:13.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.370+0000 7faaaccee700 1 --2- 172.21.15.16:0/214522477 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faaa8133420 0x7faaa809c0a0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7faa9c00e9b0 tx=0x7faa9c00ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:13.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.370+0000 7faaa57fa700 1 -- 172.21.15.16:0/214522477 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9c00ee50 con 0x7faaa8133420 2023-09-30T22:36:13.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.370+0000 7faaa57fa700 1 -- 172.21.15.16:0/214522477 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa9c00c850 con 0x7faaa8133420 2023-09-30T22:36:13.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.370+0000 7faaa57fa700 1 -- 172.21.15.16:0/214522477 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9c00cbc0 con 0x7faaa8133420 2023-09-30T22:36:13.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.370+0000 7faaaef52700 1 -- 172.21.15.16:0/214522477 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaa0009b30 con 0x7faaa8133420 2023-09-30T22:36:13.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.370+0000 7faaaef52700 1 -- 172.21.15.16:0/214522477 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaa811fb60 con 0x7faaa8133420 2023-09-30T22:36:13.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.372+0000 7faaaef52700 1 -- 172.21.15.16:0/214522477 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaa8131400 con 0x7faaa8133420 2023-09-30T22:36:13.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.372+0000 7faaa57fa700 1 -- 172.21.15.16:0/214522477 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7faa9c00cd20 con 0x7faaa8133420 2023-09-30T22:36:13.376 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.373+0000 7faaa57fa700 1 -- 172.21.15.16:0/214522477 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7faa9c03fff0 con 0x7faaa8133420 2023-09-30T22:36:13.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.381+0000 7faaa57fa700 1 -- 172.21.15.16:0/214522477 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7faa9c01a360 con 0x7faaa8133420 2023-09-30T22:36:13.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.411+0000 7f2b18c29700 1 Processor -- start 2023-09-30T22:36:13.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.411+0000 7f2b18c29700 1 -- start start 2023-09-30T22:36:13.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.412+0000 7f2b18c29700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2b14133510 0x7f2b141358f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:13.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.412+0000 7f2b18c29700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2b14135e30 0x7f2b14138210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:13.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.412+0000 7f2b18c29700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b14130260 con 0x7f2b14130670 2023-09-30T22:36:13.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.412+0000 7f2b18c29700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b1409a350 con 0x7f2b14135e30 2023-09-30T22:36:13.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.412+0000 7f2b18c29700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b14071ff0 con 0x7f2b14133510 2023-09-30T22:36:13.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.412+0000 7f2b1259c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2b14130670 0x7f2b14130a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52220/0 (socket says 172.21.15.16:52220) 2023-09-30T22:36:13.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.412+0000 7f2b1259c700 1 -- 172.21.15.16:0/1124571119 learned_addr learned my addr 172.21.15.16:0/1124571119 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:36:13.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.412+0000 7f2b11d9b700 1 --2- 172.21.15.16:0/1124571119 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2b14133510 0x7f2b141358f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:13.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.412+0000 7f2b12d9d700 1 --2- 172.21.15.16:0/1124571119 >> v2:172.21.15.16:3300/0 conn(0x7f2b14135e30 0x7f2b14138210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:13.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.413+0000 7f2b12d9d700 1 -- 172.21.15.16:0/1124571119 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2b14133510 msgr2=0x7f2b141358f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:13.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.413+0000 7f2b12d9d700 1 --2- 172.21.15.16:0/1124571119 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2b14133510 0x7f2b141358f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:13.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.413+0000 7f2b12d9d700 1 -- 172.21.15.16:0/1124571119 >> v1:172.21.15.16:6789/0 conn(0x7f2b14130670 legacy=0x7f2b14130a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:13.420 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.413+0000 7f2b12d9d700 1 -- 172.21.15.16:0/1124571119 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b1409ecf0 con 0x7f2b14135e30 2023-09-30T22:36:13.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.413+0000 7f2b12d9d700 1 --2- 172.21.15.16:0/1124571119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b14135e30 0x7f2b14138210 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c009d90 tx=0x7f2b0c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a19d0cf4150e6e8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:13.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.413+0000 7f2b1159a700 1 -- 172.21.15.16:0/1124571119 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c00bd20 con 0x7f2b14135e30 2023-09-30T22:36:13.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.413+0000 7f2b1159a700 1 -- 172.21.15.16:0/1124571119 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b0c00be80 con 0x7f2b14135e30 2023-09-30T22:36:13.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.413+0000 7f2b1159a700 1 -- 172.21.15.16:0/1124571119 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c0116a0 con 0x7f2b14135e30 2023-09-30T22:36:13.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.414+0000 7f2b18c29700 1 -- 172.21.15.16:0/1124571119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b14135e30 msgr2=0x7f2b14138210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:13.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.414+0000 7f2b18c29700 1 --2- 172.21.15.16:0/1124571119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b14135e30 0x7f2b14138210 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c009d90 tx=0x7f2b0c00b330 comp rx=0 tx=0).stop 2023-09-30T22:36:13.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.414+0000 7f2b18c29700 1 -- 172.21.15.16:0/1124571119 shutdown_connections 2023-09-30T22:36:13.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.414+0000 7f2b18c29700 1 --2- 172.21.15.16:0/1124571119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b14135e30 0x7f2b14138210 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:13.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.414+0000 7f2b18c29700 1 --2- 172.21.15.16:0/1124571119 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2b14133510 0x7f2b141358f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:13.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.414+0000 7f2b18c29700 1 -- 172.21.15.16:0/1124571119 >> 172.21.15.16:0/1124571119 conn(0x7f2b140a2aa0 msgr2=0x7f2b140a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:13.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.414+0000 7f2b18c29700 1 -- 172.21.15.16:0/1124571119 shutdown_connections 2023-09-30T22:36:13.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.414+0000 7f2b18c29700 1 -- 172.21.15.16:0/1124571119 wait complete. 2023-09-30T22:36:13.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.416+0000 7f2b18c29700 1 Processor -- start 2023-09-30T22:36:13.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.416+0000 7f2b18c29700 1 -- start start 2023-09-30T22:36:13.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.416+0000 7f2b18c29700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2b14130670 0x7f2b1409a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:13.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.416+0000 7f2b18c29700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b14133510 0x7f2b1409ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:13.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.416+0000 7f2b18c29700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2b14135e30 0x7f2b1409b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:13.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.416+0000 7f2b18c29700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1409d950 con 0x7f2b14133510 2023-09-30T22:36:13.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.416+0000 7f2b18c29700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1409c1d0 con 0x7f2b14130670 2023-09-30T22:36:13.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.416+0000 7f2b18c29700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1409e2c0 con 0x7f2b14135e30 2023-09-30T22:36:13.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.417+0000 7f2b11d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b14133510 0x7f2b1409ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:13.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.417+0000 7f2b1259c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2b14130670 0x7f2b1409a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:13.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.417+0000 7f2b11d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b14133510 0x7f2b1409ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51980/0 (socket says 172.21.15.16:51980) 2023-09-30T22:36:13.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.417+0000 7f2b11d9b700 1 -- 172.21.15.16:0/364196309 learned_addr learned my addr 172.21.15.16:0/364196309 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:13.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.417+0000 7f2b12d9d700 1 --2- 172.21.15.16:0/364196309 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2b14135e30 0x7f2b1409b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:13.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.417+0000 7f2b11d9b700 1 -- 172.21.15.16:0/364196309 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2b14135e30 msgr2=0x7f2b1409b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:13.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.417+0000 7f2b11d9b700 1 --2- 172.21.15.16:0/364196309 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2b14135e30 0x7f2b1409b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:13.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.417+0000 7f2b11d9b700 1 -- 172.21.15.16:0/364196309 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2b14130670 msgr2=0x7f2b1409a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:13.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.417+0000 7f2b11d9b700 1 --2- 172.21.15.16:0/364196309 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2b14130670 0x7f2b1409a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:13.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.417+0000 7f2b11d9b700 1 -- 172.21.15.16:0/364196309 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b0c009a70 con 0x7f2b14133510 2023-09-30T22:36:13.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.417+0000 7f2b1259c700 1 --2- 172.21.15.16:0/364196309 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2b14130670 0x7f2b1409a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:13.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.417+0000 7f2b11d9b700 1 --2- 172.21.15.16:0/364196309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b14133510 0x7f2b1409ab00 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f2b0800e9b0 tx=0x7f2b0800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:13.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.417+0000 7f2b02ffd700 1 -- 172.21.15.16:0/364196309 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b08009710 con 0x7f2b14133510 2023-09-30T22:36:13.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.418+0000 7f2b02ffd700 1 -- 172.21.15.16:0/364196309 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b0800ee50 con 0x7f2b14133510 2023-09-30T22:36:13.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.418+0000 7f2b18c29700 1 -- 172.21.15.16:0/364196309 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b14135900 con 0x7f2b14133510 2023-09-30T22:36:13.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.418+0000 7f2b02ffd700 1 -- 172.21.15.16:0/364196309 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0800ca50 con 0x7f2b14133510 2023-09-30T22:36:13.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.418+0000 7f2b18c29700 1 -- 172.21.15.16:0/364196309 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b1409de00 con 0x7f2b14133510 2023-09-30T22:36:13.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.419+0000 7f2b02ffd700 1 -- 172.21.15.16:0/364196309 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2b0800cbb0 con 0x7f2b14133510 2023-09-30T22:36:13.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.419+0000 7f2b02ffd700 1 -- 172.21.15.16:0/364196309 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f2b0803f770 con 0x7f2b14133510 2023-09-30T22:36:13.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.419+0000 7f2b18c29700 1 -- 172.21.15.16:0/364196309 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b14074ea0 con 0x7f2b14133510 2023-09-30T22:36:13.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.424+0000 7f2b02ffd700 1 -- 172.21.15.16:0/364196309 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2b08021020 con 0x7f2b14133510 2023-09-30T22:36:13.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.454+0000 7fcf98251700 1 Processor -- start 2023-09-30T22:36:13.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.454+0000 7fcf98251700 1 -- start start 2023-09-30T22:36:13.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.455+0000 7fcf98251700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcf900a0a30 0x7fcf900a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:13.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.455+0000 7fcf98251700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcf90138d80 0x7fcf9013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:13.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.455+0000 7fcf98251700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf900a2a70 con 0x7fcf9009d630 2023-09-30T22:36:13.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.455+0000 7fcf98251700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf900a01d0 con 0x7fcf90138d80 2023-09-30T22:36:13.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.455+0000 7fcf98251700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf9009e2b0 con 0x7fcf900a0a30 2023-09-30T22:36:13.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.455+0000 7fcf95fed700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcf900a0a30 0x7fcf900a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:13.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.455+0000 7fcf967ee700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcf90138d80 0x7fcf9013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:13.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.455+0000 7fcf95fed700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcf900a0a30 0x7fcf900a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:33440/0 (socket says 172.21.15.16:33440) 2023-09-30T22:36:13.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.455+0000 7fcf967ee700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcf90138d80 0x7fcf9013b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51996/0 (socket says 172.21.15.16:51996) 2023-09-30T22:36:13.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.455+0000 7fcf95fed700 1 -- 172.21.15.16:0/1529181220 learned_addr learned my addr 172.21.15.16:0/1529181220 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:13.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.456+0000 7fcf967ee700 1 -- 172.21.15.16:0/1529181220 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcf900a0a30 msgr2=0x7fcf900a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:13.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.456+0000 7fcf967ee700 1 --2- 172.21.15.16:0/1529181220 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcf900a0a30 0x7fcf900a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:13.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.456+0000 7fcf967ee700 1 -- 172.21.15.16:0/1529181220 >> v1:172.21.15.16:6789/0 conn(0x7fcf9009d630 legacy=0x7fcf900a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:13.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.456+0000 7fcf967ee700 1 -- 172.21.15.16:0/1529181220 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf9009dc50 con 0x7fcf90138d80 2023-09-30T22:36:13.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.456+0000 7fcf967ee700 1 --2- 172.21.15.16:0/1529181220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf90138d80 0x7fcf9013b160 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fcf88009d90 tx=0x7fcf8800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c32cce52d88b3409 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:13.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.456+0000 7fcf94feb700 1 -- 172.21.15.16:0/1529181220 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf8800e070 con 0x7fcf90138d80 2023-09-30T22:36:13.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.456+0000 7fcf94feb700 1 -- 172.21.15.16:0/1529181220 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf8800bb20 con 0x7fcf90138d80 2023-09-30T22:36:13.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.456+0000 7fcf94feb700 1 -- 172.21.15.16:0/1529181220 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf88012430 con 0x7fcf90138d80 2023-09-30T22:36:13.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.457+0000 7fcf98251700 1 -- 172.21.15.16:0/1529181220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf90138d80 msgr2=0x7fcf9013b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:13.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.457+0000 7fcf98251700 1 --2- 172.21.15.16:0/1529181220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf90138d80 0x7fcf9013b160 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fcf88009d90 tx=0x7fcf8800b330 comp rx=0 tx=0).stop 2023-09-30T22:36:13.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.457+0000 7fcf98251700 1 -- 172.21.15.16:0/1529181220 shutdown_connections 2023-09-30T22:36:13.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.457+0000 7fcf98251700 1 --2- 172.21.15.16:0/1529181220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf90138d80 0x7fcf9013b160 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:13.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.457+0000 7fcf98251700 1 --2- 172.21.15.16:0/1529181220 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcf900a0a30 0x7fcf900a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:13.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.457+0000 7fcf98251700 1 -- 172.21.15.16:0/1529181220 >> 172.21.15.16:0/1529181220 conn(0x7fcf900a3050 msgr2=0x7fcf900a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:13.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.457+0000 7fcf98251700 1 -- 172.21.15.16:0/1529181220 shutdown_connections 2023-09-30T22:36:13.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.457+0000 7fcf98251700 1 -- 172.21.15.16:0/1529181220 wait complete. 2023-09-30T22:36:13.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.458+0000 7fcf98251700 1 Processor -- start 2023-09-30T22:36:13.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.458+0000 7fcf98251700 1 -- start start 2023-09-30T22:36:13.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.459+0000 7fcf98251700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcf9009d630 0x7fcf9011eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:13.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.459+0000 7fcf98251700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcf900a0a30 0x7fcf9011f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:13.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.459+0000 7fcf98251700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf90138d80 0x7fcf9011f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:13.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.459+0000 7fcf98251700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf90120d10 con 0x7fcf90138d80 2023-09-30T22:36:13.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.459+0000 7fcf98251700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf900a04a0 con 0x7fcf9009d630 2023-09-30T22:36:13.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.459+0000 7fcf98251700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcf900a4b80 con 0x7fcf900a0a30 2023-09-30T22:36:13.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.459+0000 7fcf967ee700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf90138d80 0x7fcf9011f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:13.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.459+0000 7fcf967ee700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf90138d80 0x7fcf9011f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52010/0 (socket says 172.21.15.16:52010) 2023-09-30T22:36:13.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.459+0000 7fcf967ee700 1 -- 172.21.15.16:0/1882571716 learned_addr learned my addr 172.21.15.16:0/1882571716 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:13.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.459+0000 7fcf95fed700 1 --2- 172.21.15.16:0/1882571716 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcf9009d630 0x7fcf9011eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:13.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.459+0000 7fcf957ec700 1 --2- 172.21.15.16:0/1882571716 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcf900a0a30 0x7fcf9011f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:13.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.460+0000 7fcf967ee700 1 -- 172.21.15.16:0/1882571716 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcf900a0a30 msgr2=0x7fcf9011f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:13.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.460+0000 7fcf967ee700 1 --2- 172.21.15.16:0/1882571716 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcf900a0a30 0x7fcf9011f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:13.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.460+0000 7fcf967ee700 1 -- 172.21.15.16:0/1882571716 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcf9009d630 msgr2=0x7fcf9011eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:13.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.460+0000 7fcf967ee700 1 --2- 172.21.15.16:0/1882571716 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcf9009d630 0x7fcf9011eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:13.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.460+0000 7fcf967ee700 1 -- 172.21.15.16:0/1882571716 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf88009a70 con 0x7fcf90138d80 2023-09-30T22:36:13.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.460+0000 7fcf95fed700 1 --2- 172.21.15.16:0/1882571716 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcf9009d630 0x7fcf9011eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:13.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.460+0000 7fcf967ee700 1 --2- 172.21.15.16:0/1882571716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf90138d80 0x7fcf9011f5f0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fcf88013000 tx=0x7fcf88004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:13.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.460+0000 7fcf867fc700 1 -- 172.21.15.16:0/1882571716 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf88012430 con 0x7fcf90138d80 2023-09-30T22:36:13.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.460+0000 7fcf867fc700 1 -- 172.21.15.16:0/1882571716 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf88020070 con 0x7fcf90138d80 2023-09-30T22:36:13.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.460+0000 7fcf867fc700 1 -- 172.21.15.16:0/1882571716 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf88003ea0 con 0x7fcf90138d80 2023-09-30T22:36:13.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.460+0000 7fcf98251700 1 -- 172.21.15.16:0/1882571716 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf901206b0 con 0x7fcf90138d80 2023-09-30T22:36:13.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.460+0000 7fcf98251700 1 -- 172.21.15.16:0/1882571716 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf90120ea0 con 0x7fcf90138d80 2023-09-30T22:36:13.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.462+0000 7fcf98251700 1 -- 172.21.15.16:0/1882571716 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf900747f0 con 0x7fcf90138d80 2023-09-30T22:36:13.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.462+0000 7fcf867fc700 1 -- 172.21.15.16:0/1882571716 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcf88004280 con 0x7fcf90138d80 2023-09-30T22:36:13.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.462+0000 7fcf867fc700 1 -- 172.21.15.16:0/1882571716 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fcf88022070 con 0x7fcf90138d80 2023-09-30T22:36:13.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.467+0000 7fcf867fc700 1 -- 172.21.15.16:0/1882571716 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcf88012e60 con 0x7fcf90138d80 2023-09-30T22:36:13.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.617+0000 7fc6a1c89700 1 -- 172.21.15.16:0/4059561687 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc69c074ea0 con 0x7fc69c130e50 2023-09-30T22:36:13.618 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.618+0000 7fc683fff700 1 -- 172.21.15.16:0/4059561687 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc69401d360 con 0x7fc69c130e50 2023-09-30T22:36:13.618 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:36:13.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.621+0000 7fc6a1c89700 1 -- 172.21.15.16:0/4059561687 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc69c130e50 msgr2=0x7fc69c11af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:13.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.621+0000 7fc6a1c89700 1 --2- 172.21.15.16:0/4059561687 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc69c130e50 0x7fc69c11af50 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fc694007ef0 tx=0x7fc69400d3b0 comp rx=0 tx=0).stop 2023-09-30T22:36:13.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.621+0000 7fc6a1c89700 1 -- 172.21.15.16:0/4059561687 shutdown_connections 2023-09-30T22:36:13.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.621+0000 7fc6a1c89700 1 --2- 172.21.15.16:0/4059561687 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc69c130e50 0x7fc69c11af50 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:13.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.621+0000 7fc6a1c89700 1 --2- 172.21.15.16:0/4059561687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc69c06fb20 0x7fc69c11a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:13.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.621+0000 7fc6a1c89700 1 --2- 172.21.15.16:0/4059561687 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc69c130540 0x7fc69c11aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:13.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.621+0000 7fc6a1c89700 1 -- 172.21.15.16:0/4059561687 >> 172.21.15.16:0/4059561687 conn(0x7fc69c0a2aa0 msgr2=0x7fc69c12d570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:13.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.621+0000 7fc6a1c89700 1 -- 172.21.15.16:0/4059561687 shutdown_connections 2023-09-30T22:36:13.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.621+0000 7fc6a1c89700 1 -- 172.21.15.16:0/4059561687 wait complete. 2023-09-30T22:36:13.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.639+0000 7f2b18c29700 1 -- 172.21.15.16:0/364196309 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f2b14072210 con 0x7f2b14133510 2023-09-30T22:36:13.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.639+0000 7f2b02ffd700 1 -- 172.21.15.16:0/364196309 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2b0801be10 con 0x7f2b14133510 2023-09-30T22:36:13.640 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:36:13.641 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:36:13.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.642+0000 7f2b18c29700 1 -- 172.21.15.16:0/364196309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b14133510 msgr2=0x7f2b1409ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:13.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.642+0000 7f2b18c29700 1 --2- 172.21.15.16:0/364196309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b14133510 0x7f2b1409ab00 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f2b0800e9b0 tx=0x7f2b0800ecc0 comp rx=0 tx=0).stop 2023-09-30T22:36:13.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.642+0000 7f2b18c29700 1 -- 172.21.15.16:0/364196309 shutdown_connections 2023-09-30T22:36:13.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.642+0000 7f2b18c29700 1 --2- 172.21.15.16:0/364196309 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2b14130670 0x7f2b1409a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:13.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.642+0000 7f2b18c29700 1 --2- 172.21.15.16:0/364196309 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2b14135e30 0x7f2b1409b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:13.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.642+0000 7f2b18c29700 1 --2- 172.21.15.16:0/364196309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b14133510 0x7f2b1409ab00 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:13.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.642+0000 7f2b18c29700 1 -- 172.21.15.16:0/364196309 >> 172.21.15.16:0/364196309 conn(0x7f2b140a2aa0 msgr2=0x7f2b14132390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:13.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.642+0000 7f2b18c29700 1 -- 172.21.15.16:0/364196309 shutdown_connections 2023-09-30T22:36:13.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.642+0000 7f2b18c29700 1 -- 172.21.15.16:0/364196309 wait complete. 2023-09-30T22:36:13.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.655+0000 7faaaef52700 1 -- 172.21.15.16:0/214522477 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7faaa8074ea0 con 0x7faaa8133420 2023-09-30T22:36:13.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.655+0000 7faaa57fa700 1 -- 172.21.15.16:0/214522477 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7faa9c021020 con 0x7faaa8133420 2023-09-30T22:36:13.656 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:36:13.656 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:36:13.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.659+0000 7faaaef52700 1 -- 172.21.15.16:0/214522477 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faaa8133420 msgr2=0x7faaa809c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:13.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.659+0000 7faaaef52700 1 --2- 172.21.15.16:0/214522477 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faaa8133420 0x7faaa809c0a0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7faa9c00e9b0 tx=0x7faa9c00ecc0 comp rx=0 tx=0).stop 2023-09-30T22:36:13.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.659+0000 7faaaef52700 1 -- 172.21.15.16:0/214522477 shutdown_connections 2023-09-30T22:36:13.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.659+0000 7faaaef52700 1 --2- 172.21.15.16:0/214522477 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7faaa8134260 0x7faaa809cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:13.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.659+0000 7faaaef52700 1 --2- 172.21.15.16:0/214522477 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7faaa8133420 0x7faaa809c0a0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:13.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.659+0000 7faaaef52700 1 --2- 172.21.15.16:0/214522477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7faaa8133e30 0x7faaa809c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:13.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.659+0000 7faaaef52700 1 -- 172.21.15.16:0/214522477 >> 172.21.15.16:0/214522477 conn(0x7faaa80a4210 msgr2=0x7faaa80a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:13.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.659+0000 7faaaef52700 1 -- 172.21.15.16:0/214522477 shutdown_connections 2023-09-30T22:36:13.660 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.659+0000 7faaaef52700 1 -- 172.21.15.16:0/214522477 wait complete. 2023-09-30T22:36:13.678 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:36:13.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.698+0000 7fcf98251700 1 -- 172.21.15.16:0/1882571716 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fcf90072980 con 0x7fcf90138d80 2023-09-30T22:36:13.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.698+0000 7fcf867fc700 1 -- 172.21.15.16:0/1882571716 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcf8801a380 con 0x7fcf90138d80 2023-09-30T22:36:13.699 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:36:13.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.700+0000 7fcf98251700 1 -- 172.21.15.16:0/1882571716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf90138d80 msgr2=0x7fcf9011f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:13.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.700+0000 7fcf98251700 1 --2- 172.21.15.16:0/1882571716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf90138d80 0x7fcf9011f5f0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fcf88013000 tx=0x7fcf88004ad0 comp rx=0 tx=0).stop 2023-09-30T22:36:13.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.700+0000 7fcf98251700 1 -- 172.21.15.16:0/1882571716 shutdown_connections 2023-09-30T22:36:13.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.700+0000 7fcf98251700 1 --2- 172.21.15.16:0/1882571716 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcf9009d630 0x7fcf9011eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:13.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.700+0000 7fcf98251700 1 --2- 172.21.15.16:0/1882571716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcf90138d80 0x7fcf9011f5f0 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:13.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.700+0000 7fcf98251700 1 --2- 172.21.15.16:0/1882571716 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcf900a0a30 0x7fcf9011f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:13.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.700+0000 7fcf98251700 1 -- 172.21.15.16:0/1882571716 >> 172.21.15.16:0/1882571716 conn(0x7fcf900a3050 msgr2=0x7fcf900a4f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:13.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.700+0000 7fcf98251700 1 -- 172.21.15.16:0/1882571716 shutdown_connections 2023-09-30T22:36:13.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:13.700+0000 7fcf98251700 1 -- 172.21.15.16:0/1882571716 wait complete. 2023-09-30T22:36:13.714 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:36:14.642 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:36:14.657 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:36:14.679 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:36:14.715 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:36:14.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.797+0000 7f244ed9c700 1 Processor -- start 2023-09-30T22:36:14.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.798+0000 7f244ed9c700 1 -- start start 2023-09-30T22:36:14.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.798+0000 7f244ed9c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f244812cf60 0x7f244806f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:14.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.798+0000 7f244ed9c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f244812f8b0 0x7f244812fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:14.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.798+0000 7f244ed9c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2448130260 0x7f2448136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:14.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.798+0000 7f244ed9c700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f244812d330 con 0x7f244812f8b0 2023-09-30T22:36:14.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.798+0000 7f244ed9c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f244812cd90 con 0x7f2448130260 2023-09-30T22:36:14.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.798+0000 7f244ed9c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f24481306d0 con 0x7f244812cf60 2023-09-30T22:36:14.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.799+0000 7f244cb38700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f244812cf60 0x7f244806f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:14.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.799+0000 7f244cb38700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f244812cf60 0x7f244806f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:44024/0 (socket says 172.21.15.16:44024) 2023-09-30T22:36:14.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.799+0000 7f244cb38700 1 -- 172.21.15.16:0/3016645388 learned_addr learned my addr 172.21.15.16:0/3016645388 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:14.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.799+0000 7f2447fff700 1 --2- 172.21.15.16:0/3016645388 >> v2:172.21.15.16:3300/0 conn(0x7f244812f8b0 0x7f244812fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:14.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.799+0000 7f2447fff700 1 -- 172.21.15.16:0/3016645388 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f244812cf60 msgr2=0x7f244806f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:14.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.799+0000 7f2447fff700 1 --2- 172.21.15.16:0/3016645388 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f244812cf60 0x7f244806f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:14.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.799+0000 7f2447fff700 1 -- 172.21.15.16:0/3016645388 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2448130260 msgr2=0x7f2448136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:14.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.799+0000 7f2447fff700 1 --2- 172.21.15.16:0/3016645388 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2448130260 0x7f2448136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:14.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.799+0000 7f2447fff700 1 -- 172.21.15.16:0/3016645388 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2448071780 con 0x7f244812f8b0 2023-09-30T22:36:14.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.799+0000 7f244cb38700 1 --2- 172.21.15.16:0/3016645388 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f244812cf60 0x7f244806f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:14.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.799+0000 7f2447fff700 1 --2- 172.21.15.16:0/3016645388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f244812f8b0 0x7f244812fd20 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f2434009870 tx=0x7f2434009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f48baea72ac3be40 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:14.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.799+0000 7f24477fe700 1 -- 172.21.15.16:0/3016645388 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2434011830 con 0x7f244812f8b0 2023-09-30T22:36:14.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.800+0000 7f24477fe700 1 -- 172.21.15.16:0/3016645388 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2434011990 con 0x7f244812f8b0 2023-09-30T22:36:14.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.800+0000 7f24477fe700 1 -- 172.21.15.16:0/3016645388 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2434011d20 con 0x7f244812f8b0 2023-09-30T22:36:14.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.801+0000 7f244ed9c700 1 -- 172.21.15.16:0/3016645388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f244812f8b0 msgr2=0x7f244812fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:14.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.801+0000 7f244ed9c700 1 --2- 172.21.15.16:0/3016645388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f244812f8b0 0x7f244812fd20 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f2434009870 tx=0x7f2434009b80 comp rx=0 tx=0).stop 2023-09-30T22:36:14.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.801+0000 7f244ed9c700 1 -- 172.21.15.16:0/3016645388 shutdown_connections 2023-09-30T22:36:14.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.801+0000 7f244ed9c700 1 --2- 172.21.15.16:0/3016645388 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2448130260 0x7f2448136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:14.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.801+0000 7f244ed9c700 1 --2- 172.21.15.16:0/3016645388 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f244812cf60 0x7f244806f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:14.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.801+0000 7f244ed9c700 1 --2- 172.21.15.16:0/3016645388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f244812f8b0 0x7f244812fd20 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:14.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.801+0000 7f244ed9c700 1 -- 172.21.15.16:0/3016645388 >> 172.21.15.16:0/3016645388 conn(0x7f24480a2aa0 msgr2=0x7f24480a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:14.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.801+0000 7f244ed9c700 1 -- 172.21.15.16:0/3016645388 shutdown_connections 2023-09-30T22:36:14.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.801+0000 7f244ed9c700 1 -- 172.21.15.16:0/3016645388 wait complete. 2023-09-30T22:36:14.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.802+0000 7f244ed9c700 1 Processor -- start 2023-09-30T22:36:14.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.802+0000 7f244ed9c700 1 -- start start 2023-09-30T22:36:14.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.803+0000 7f244ed9c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f244812cf60 0x7f24481309e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:14.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.803+0000 7f244ed9c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f244812f8b0 0x7f2448130f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:14.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.803+0000 7f244ed9c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2448130260 0x7f2448131460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:14.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.803+0000 7f244ed9c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24481332c0 con 0x7f2448130260 2023-09-30T22:36:14.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.803+0000 7f244ed9c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f244809ea00 con 0x7f244812cf60 2023-09-30T22:36:14.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.803+0000 7f244ed9c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f244812f420 con 0x7f244812f8b0 2023-09-30T22:36:14.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.804+0000 7f2447fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f244812f8b0 0x7f2448130f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:14.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.804+0000 7f244cb38700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f244812cf60 0x7f24481309e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:14.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.804+0000 7f2447fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f244812f8b0 0x7f2448130f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:44040/0 (socket says 172.21.15.16:44040) 2023-09-30T22:36:14.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.804+0000 7f2447fff700 1 -- 172.21.15.16:0/3847963851 learned_addr learned my addr 172.21.15.16:0/3847963851 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:14.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.804+0000 7f244d339700 1 --2- 172.21.15.16:0/3847963851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2448130260 0x7f2448131460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:14.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.804+0000 7f244cb38700 1 -- 172.21.15.16:0/3847963851 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f244812f8b0 msgr2=0x7f2448130f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:14.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.804+0000 7f244cb38700 1 --2- 172.21.15.16:0/3847963851 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f244812f8b0 0x7f2448130f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:14.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.804+0000 7f244cb38700 1 -- 172.21.15.16:0/3847963851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2448130260 msgr2=0x7f2448131460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:14.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.804+0000 7f244cb38700 1 --2- 172.21.15.16:0/3847963851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2448130260 0x7f2448131460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:14.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.804+0000 7f244cb38700 1 -- 172.21.15.16:0/3847963851 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f243c009550 con 0x7f244812cf60 2023-09-30T22:36:14.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.804+0000 7f244cb38700 1 --2- 172.21.15.16:0/3847963851 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f244812cf60 0x7f24481309e0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f243c00e9d0 tx=0x7f243c00ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:14.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.804+0000 7f24457fa700 1 -- 172.21.15.16:0/3847963851 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243c00ee70 con 0x7f244812cf60 2023-09-30T22:36:14.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.805+0000 7f24457fa700 1 -- 172.21.15.16:0/3847963851 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f243c004030 con 0x7f244812cf60 2023-09-30T22:36:14.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.805+0000 7f24457fa700 1 -- 172.21.15.16:0/3847963851 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243c0043c0 con 0x7f244812cf60 2023-09-30T22:36:14.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.805+0000 7f244ed9c700 1 -- 172.21.15.16:0/3847963851 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24340094a0 con 0x7f244812cf60 2023-09-30T22:36:14.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.805+0000 7f244ed9c700 1 -- 172.21.15.16:0/3847963851 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2448139880 con 0x7f244812cf60 2023-09-30T22:36:14.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.806+0000 7f24457fa700 1 -- 172.21.15.16:0/3847963851 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f243c00c850 con 0x7f244812cf60 2023-09-30T22:36:14.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.806+0000 7f24457fa700 1 -- 172.21.15.16:0/3847963851 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f243c03f9c0 con 0x7f244812cf60 2023-09-30T22:36:14.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.809+0000 7f244ed9c700 1 -- 172.21.15.16:0/3847963851 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2448074ea0 con 0x7f244812cf60 2023-09-30T22:36:14.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.814+0000 7f24457fa700 1 -- 172.21.15.16:0/3847963851 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f243c011d90 con 0x7f244812cf60 2023-09-30T22:36:14.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.839+0000 7f79a0ad9700 1 Processor -- start 2023-09-30T22:36:14.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.839+0000 7f79a0ad9700 1 -- start start 2023-09-30T22:36:14.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.839+0000 7f2eee5be700 1 Processor -- start 2023-09-30T22:36:14.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.839+0000 7f2eee5be700 1 -- start start 2023-09-30T22:36:14.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.840+0000 7f79a0ad9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f799c06e570 0x7f799c06e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:14.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.840+0000 7f79a0ad9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f799c09eca0 0x7f799c06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:14.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.840+0000 7f2eee5be700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2ee80a09c0 0x7f2ee80a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:14.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.840+0000 7f79a0ad9700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f799c134610 0x7f799c134ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:14.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.840+0000 7f79a0ad9700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f799c070ee0 con 0x7f799c134610 2023-09-30T22:36:14.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.840+0000 7f79a0ad9700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f799c071480 con 0x7f799c06e570 2023-09-30T22:36:14.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.840+0000 7f79a0ad9700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f799c06f3c0 con 0x7f799c09eca0 2023-09-30T22:36:14.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.840+0000 7f2eee5be700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2ee80a12d0 0x7f2ee8134030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:14.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.840+0000 7f2eee5be700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ee809ea10 con 0x7f2ee809d5c0 2023-09-30T22:36:14.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.840+0000 7f2eee5be700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ee809e210 con 0x7f2ee80a12d0 2023-09-30T22:36:14.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.840+0000 7f2eee5be700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee80a26f0 con 0x7f2ee80a09c0 2023-09-30T22:36:14.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.840+0000 7f799ad9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f799c134610 0x7f799c134ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:14.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.840+0000 7f799ad9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f799c134610 0x7f799c134ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52042/0 (socket says 172.21.15.16:52042) 2023-09-30T22:36:14.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.840+0000 7f799ad9d700 1 -- 172.21.15.16:0/483290074 learned_addr learned my addr 172.21.15.16:0/483290074 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:14.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.840+0000 7f2eecb5b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2ee80a12d0 0x7f2ee8134030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:14.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.840+0000 7f2eecb5b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2ee80a12d0 0x7f2ee8134030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52036/0 (socket says 172.21.15.16:52036) 2023-09-30T22:36:14.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.840+0000 7f2eecb5b700 1 -- 172.21.15.16:0/342514123 learned_addr learned my addr 172.21.15.16:0/342514123 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:14.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.840+0000 7f2ee77fe700 1 --1- 172.21.15.16:0/342514123 >> v1:172.21.15.16:6789/0 conn(0x7f2ee809d5c0 0x7f2ee8133920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52232/0 (socket says 172.21.15.16:52232) 2023-09-30T22:36:14.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.841+0000 7f7999d9b700 1 --2- 172.21.15.16:0/483290074 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f799c09eca0 0x7f799c06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:14.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.841+0000 7f799ad9d700 1 -- 172.21.15.16:0/483290074 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f799c09eca0 msgr2=0x7f799c06ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:14.845 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.841+0000 7f799ad9d700 1 --2- 172.21.15.16:0/483290074 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f799c09eca0 0x7f799c06ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:14.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.841+0000 7f2ee7fff700 1 --2- 172.21.15.16:0/342514123 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2ee80a09c0 0x7f2ee80a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:14.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.841+0000 7f799ad9d700 1 -- 172.21.15.16:0/483290074 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f799c06e570 msgr2=0x7f799c06e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:14.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.841+0000 7f799ad9d700 1 --2- 172.21.15.16:0/483290074 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f799c06e570 0x7f799c06e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:14.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.841+0000 7f799ad9d700 1 -- 172.21.15.16:0/483290074 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f799c071b90 con 0x7f799c134610 2023-09-30T22:36:14.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.841+0000 7f7999d9b700 1 --2- 172.21.15.16:0/483290074 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f799c09eca0 0x7f799c06ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:36:14.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.841+0000 7f2eecb5b700 1 -- 172.21.15.16:0/342514123 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2ee80a09c0 msgr2=0x7f2ee80a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:14.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.841+0000 7f2eecb5b700 1 --2- 172.21.15.16:0/342514123 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2ee80a09c0 0x7f2ee80a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:14.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.841+0000 7f2eecb5b700 1 -- 172.21.15.16:0/342514123 >> v1:172.21.15.16:6789/0 conn(0x7f2ee809d5c0 legacy=0x7f2ee8133920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:14.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.841+0000 7f2eecb5b700 1 -- 172.21.15.16:0/342514123 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee809db70 con 0x7f2ee80a12d0 2023-09-30T22:36:14.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.841+0000 7f799ad9d700 1 --2- 172.21.15.16:0/483290074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f799c134610 0x7f799c134ab0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f7994009e50 tx=0x7f799400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cb8e1e8490c5a5b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:14.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.841+0000 7f2eecb5b700 1 --2- 172.21.15.16:0/342514123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ee80a12d0 0x7f2ee8134030 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0009e50 tx=0x7f2ee000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5f520440861ee15 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:14.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.841+0000 7f799959a700 1 -- 172.21.15.16:0/483290074 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799400e070 con 0x7f799c134610 2023-09-30T22:36:14.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.842+0000 7f799959a700 1 -- 172.21.15.16:0/483290074 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f799400b9c0 con 0x7f799c134610 2023-09-30T22:36:14.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.842+0000 7f799959a700 1 -- 172.21.15.16:0/483290074 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799400bd70 con 0x7f799c134610 2023-09-30T22:36:14.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.842+0000 7f2ee6ffd700 1 -- 172.21.15.16:0/342514123 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee000e070 con 0x7f2ee80a12d0 2023-09-30T22:36:14.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.842+0000 7f2ee6ffd700 1 -- 172.21.15.16:0/342514123 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ee0003a20 con 0x7f2ee80a12d0 2023-09-30T22:36:14.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.842+0000 7f2ee6ffd700 1 -- 172.21.15.16:0/342514123 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee0003dd0 con 0x7f2ee80a12d0 2023-09-30T22:36:14.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.842+0000 7f2eee5be700 1 -- 172.21.15.16:0/342514123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ee80a12d0 msgr2=0x7f2ee8134030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:14.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.842+0000 7f2eee5be700 1 --2- 172.21.15.16:0/342514123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ee80a12d0 0x7f2ee8134030 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0009e50 tx=0x7f2ee000b330 comp rx=0 tx=0).stop 2023-09-30T22:36:14.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.843+0000 7f2eee5be700 1 -- 172.21.15.16:0/342514123 shutdown_connections 2023-09-30T22:36:14.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.843+0000 7f2eee5be700 1 --2- 172.21.15.16:0/342514123 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ee80a12d0 0x7f2ee8134030 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:14.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.843+0000 7f2eee5be700 1 --2- 172.21.15.16:0/342514123 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2ee80a09c0 0x7f2ee80a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:14.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.843+0000 7f79a0ad9700 1 -- 172.21.15.16:0/483290074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f799c134610 msgr2=0x7f799c134ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:14.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.843+0000 7f79a0ad9700 1 --2- 172.21.15.16:0/483290074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f799c134610 0x7f799c134ab0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f7994009e50 tx=0x7f799400b330 comp rx=0 tx=0).stop 2023-09-30T22:36:14.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.843+0000 7f2eee5be700 1 -- 172.21.15.16:0/342514123 >> 172.21.15.16:0/342514123 conn(0x7f2ee80a2fe0 msgr2=0x7f2ee80a33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:14.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.843+0000 7f2eee5be700 1 -- 172.21.15.16:0/342514123 shutdown_connections 2023-09-30T22:36:14.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.843+0000 7f2eee5be700 1 -- 172.21.15.16:0/342514123 wait complete. 2023-09-30T22:36:14.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.843+0000 7f79a0ad9700 1 -- 172.21.15.16:0/483290074 shutdown_connections 2023-09-30T22:36:14.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.843+0000 7f79a0ad9700 1 --2- 172.21.15.16:0/483290074 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f799c06e570 0x7f799c06e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:14.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.843+0000 7f79a0ad9700 1 --2- 172.21.15.16:0/483290074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f799c134610 0x7f799c134ab0 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:14.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.843+0000 7f79a0ad9700 1 --2- 172.21.15.16:0/483290074 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f799c09eca0 0x7f799c06ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:14.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.843+0000 7f79a0ad9700 1 -- 172.21.15.16:0/483290074 >> 172.21.15.16:0/483290074 conn(0x7f799c0a2aa0 msgr2=0x7f799c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:14.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.844+0000 7f79a0ad9700 1 -- 172.21.15.16:0/483290074 shutdown_connections 2023-09-30T22:36:14.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.844+0000 7f79a0ad9700 1 -- 172.21.15.16:0/483290074 wait complete. 2023-09-30T22:36:14.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.845+0000 7f79a0ad9700 1 Processor -- start 2023-09-30T22:36:14.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.844+0000 7f2eee5be700 1 Processor -- start 2023-09-30T22:36:14.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.845+0000 7f79a0ad9700 1 -- start start 2023-09-30T22:36:14.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.845+0000 7f79a0ad9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f799c06e570 0x7f799c1223f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:14.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.845+0000 7f79a0ad9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f799c134610 0x7f799c122930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:14.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.845+0000 7f79a0ad9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f799c122e70 0x7f799c1cca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:14.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.845+0000 7f79a0ad9700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f799c1255c0 con 0x7f799c134610 2023-09-30T22:36:14.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.845+0000 7f79a0ad9700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f799c132f20 con 0x7f799c122e70 2023-09-30T22:36:14.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.845+0000 7f79a0ad9700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f799c123f60 con 0x7f799c06e570 2023-09-30T22:36:14.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.846+0000 7f7999d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f799c134610 0x7f799c122930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:14.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.846+0000 7f7999d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f799c134610 0x7f799c122930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52046/0 (socket says 172.21.15.16:52046) 2023-09-30T22:36:14.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.846+0000 7f7999d9b700 1 -- 172.21.15.16:0/548815794 learned_addr learned my addr 172.21.15.16:0/548815794 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:14.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.846+0000 7f799ad9d700 1 --2- 172.21.15.16:0/548815794 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f799c122e70 0x7f799c1cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:14.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.846+0000 7f799a59c700 1 --2- 172.21.15.16:0/548815794 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f799c06e570 0x7f799c1223f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:14.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.845+0000 7f2eee5be700 1 -- start start 2023-09-30T22:36:14.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.846+0000 7f2eee5be700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2ee809d5c0 0x7f2ee8121170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:14.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.846+0000 7f2eee5be700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2ee80a09c0 0x7f2ee81216b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:14.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.846+0000 7f2eee5be700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ee80a12d0 0x7f2ee8121bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:14.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.846+0000 7f2eee5be700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee8124c90 con 0x7f2ee80a12d0 2023-09-30T22:36:14.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.846+0000 7f2eee5be700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee8124060 con 0x7f2ee809d5c0 2023-09-30T22:36:14.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.846+0000 7f2eee5be700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee80a52c0 con 0x7f2ee80a09c0 2023-09-30T22:36:14.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.846+0000 7f2ee77fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2ee80a09c0 0x7f2ee81216b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:14.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.846+0000 7f2ee77fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2ee80a09c0 0x7f2ee81216b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:44070/0 (socket says 172.21.15.16:44070) 2023-09-30T22:36:14.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.846+0000 7f2ee77fe700 1 -- 172.21.15.16:0/3194892234 learned_addr learned my addr 172.21.15.16:0/3194892234 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:14.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.847+0000 7f2eecb5b700 1 --2- 172.21.15.16:0/3194892234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ee80a12d0 0x7f2ee8121bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:14.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.847+0000 7f2ee7fff700 1 --2- 172.21.15.16:0/3194892234 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2ee809d5c0 0x7f2ee8121170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:14.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.846+0000 7f7999d9b700 1 -- 172.21.15.16:0/548815794 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f799c06e570 msgr2=0x7f799c1223f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:14.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.846+0000 7f7999d9b700 1 --2- 172.21.15.16:0/548815794 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f799c06e570 0x7f799c1223f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:14.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.846+0000 7f7999d9b700 1 -- 172.21.15.16:0/548815794 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f799c122e70 msgr2=0x7f799c1cca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:14.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.846+0000 7f7999d9b700 1 --2- 172.21.15.16:0/548815794 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f799c122e70 0x7f799c1cca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:14.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.846+0000 7f7999d9b700 1 -- 172.21.15.16:0/548815794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7994009b30 con 0x7f799c134610 2023-09-30T22:36:14.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.846+0000 7f799a59c700 1 --2- 172.21.15.16:0/548815794 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f799c06e570 0x7f799c1223f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:36:14.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.846+0000 7f7999d9b700 1 --2- 172.21.15.16:0/548815794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f799c134610 0x7f799c122930 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f799000c750 tx=0x7f799000ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:14.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.847+0000 7f798affd700 1 -- 172.21.15.16:0/548815794 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7990011800 con 0x7f799c134610 2023-09-30T22:36:14.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.847+0000 7f798affd700 1 -- 172.21.15.16:0/548815794 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7990011960 con 0x7f799c134610 2023-09-30T22:36:14.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.847+0000 7f798affd700 1 -- 172.21.15.16:0/548815794 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7990011cd0 con 0x7f799c134610 2023-09-30T22:36:14.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.847+0000 7f79a0ad9700 1 -- 172.21.15.16:0/548815794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f799c125e70 con 0x7f799c134610 2023-09-30T22:36:14.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.847+0000 7f79a0ad9700 1 -- 172.21.15.16:0/548815794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f799c0a2420 con 0x7f799c134610 2023-09-30T22:36:14.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.847+0000 7f2ee77fe700 1 -- 172.21.15.16:0/3194892234 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2ee809d5c0 msgr2=0x7f2ee8121170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:14.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.847+0000 7f2ee77fe700 1 --2- 172.21.15.16:0/3194892234 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2ee809d5c0 0x7f2ee8121170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:14.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.847+0000 7f2ee77fe700 1 -- 172.21.15.16:0/3194892234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ee80a12d0 msgr2=0x7f2ee8121bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:14.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.847+0000 7f2ee77fe700 1 --2- 172.21.15.16:0/3194892234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ee80a12d0 0x7f2ee8121bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:14.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.847+0000 7f2ee77fe700 1 -- 172.21.15.16:0/3194892234 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee0009b30 con 0x7f2ee80a09c0 2023-09-30T22:36:14.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.847+0000 7f2ee77fe700 1 --2- 172.21.15.16:0/3194892234 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2ee80a09c0 0x7f2ee81216b0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f2edc00c750 tx=0x7f2edc00ca60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:14.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.847+0000 7f2ee4ff9700 1 -- 172.21.15.16:0/3194892234 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2edc004c80 con 0x7f2ee80a09c0 2023-09-30T22:36:14.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.847+0000 7f2ee4ff9700 1 -- 172.21.15.16:0/3194892234 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2edc004de0 con 0x7f2ee80a09c0 2023-09-30T22:36:14.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.847+0000 7f2ee4ff9700 1 -- 172.21.15.16:0/3194892234 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2edc007310 con 0x7f2ee80a09c0 2023-09-30T22:36:14.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.847+0000 7f2eee5be700 1 -- 172.21.15.16:0/3194892234 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ee809cd10 con 0x7f2ee80a09c0 2023-09-30T22:36:14.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.848+0000 7f2eee5be700 1 -- 172.21.15.16:0/3194892234 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ee8122d80 con 0x7f2ee80a09c0 2023-09-30T22:36:14.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.848+0000 7f79a0ad9700 1 -- 172.21.15.16:0/548815794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f799c0706d0 con 0x7f799c134610 2023-09-30T22:36:14.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.849+0000 7f798affd700 1 -- 172.21.15.16:0/548815794 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7990023480 con 0x7f799c134610 2023-09-30T22:36:14.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.850+0000 7f798affd700 1 -- 172.21.15.16:0/548815794 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f799003fa00 con 0x7f799c134610 2023-09-30T22:36:14.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.849+0000 7f2ee4ff9700 1 -- 172.21.15.16:0/3194892234 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2edc007470 con 0x7f2ee80a09c0 2023-09-30T22:36:14.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.849+0000 7f2ee4ff9700 1 -- 172.21.15.16:0/3194892234 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f2edc03f580 con 0x7f2ee80a09c0 2023-09-30T22:36:14.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.849+0000 7f2eee5be700 1 -- 172.21.15.16:0/3194892234 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ee8124810 con 0x7f2ee80a09c0 2023-09-30T22:36:14.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.858+0000 7f798affd700 1 -- 172.21.15.16:0/548815794 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7990023730 con 0x7f799c134610 2023-09-30T22:36:14.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.859+0000 7f2ee4ff9700 1 -- 172.21.15.16:0/3194892234 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2ee8124810 con 0x7f2ee80a09c0 2023-09-30T22:36:14.889 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.889+0000 7f14b29e0700 1 Processor -- start 2023-09-30T22:36:14.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.889+0000 7f14b29e0700 1 -- start start 2023-09-30T22:36:14.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.890+0000 7f14b29e0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f14ac09d6f0 0x7f14ac09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:14.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.890+0000 7f14b29e0700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f14ac09e000 0x7f14ac130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:14.891 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.890+0000 7f14b29e0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f14ac09e410 0x7f14ac130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:14.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.890+0000 7f14b29e0700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14ac09f520 con 0x7f14ac09e000 2023-09-30T22:36:14.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.890+0000 7f14b29e0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14ac09f130 con 0x7f14ac09e410 2023-09-30T22:36:14.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.890+0000 7f14b29e0700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f14ac09e930 con 0x7f14ac09d6f0 2023-09-30T22:36:14.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.890+0000 7f14ab7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f14ac09e000 0x7f14ac130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:14.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.890+0000 7f14abfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f14ac09d6f0 0x7f14ac09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:14.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.890+0000 7f14ab7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f14ac09e000 0x7f14ac130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52070/0 (socket says 172.21.15.16:52070) 2023-09-30T22:36:14.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.890+0000 7f14ab7fe700 1 -- 172.21.15.16:0/1627823161 learned_addr learned my addr 172.21.15.16:0/1627823161 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:14.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.890+0000 7f14abfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f14ac09d6f0 0x7f14ac09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:44076/0 (socket says 172.21.15.16:44076) 2023-09-30T22:36:14.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.890+0000 7f14b0f7d700 1 --2- 172.21.15.16:0/1627823161 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f14ac09e410 0x7f14ac130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:14.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.890+0000 7f14ab7fe700 1 -- 172.21.15.16:0/1627823161 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f14ac09d6f0 msgr2=0x7f14ac09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:14.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.890+0000 7f14ab7fe700 1 --2- 172.21.15.16:0/1627823161 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f14ac09d6f0 0x7f14ac09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:14.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.890+0000 7f14ab7fe700 1 -- 172.21.15.16:0/1627823161 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f14ac09e410 msgr2=0x7f14ac130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:14.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.890+0000 7f14ab7fe700 1 --2- 172.21.15.16:0/1627823161 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f14ac09e410 0x7f14ac130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:14.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.890+0000 7f14ab7fe700 1 -- 172.21.15.16:0/1627823161 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14ac09b2d0 con 0x7f14ac09e000 2023-09-30T22:36:14.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.891+0000 7f14abfff700 1 --2- 172.21.15.16:0/1627823161 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f14ac09d6f0 0x7f14ac09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:14.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.891+0000 7f14ab7fe700 1 --2- 172.21.15.16:0/1627823161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14ac09e000 0x7f14ac130660 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f14a0009920 tx=0x7f14a0009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad7dc5698262cc25 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:14.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.891+0000 7f14aaffd700 1 -- 172.21.15.16:0/1627823161 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a0011830 con 0x7f14ac09e000 2023-09-30T22:36:14.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.891+0000 7f14aaffd700 1 -- 172.21.15.16:0/1627823161 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a0011990 con 0x7f14ac09e000 2023-09-30T22:36:14.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.891+0000 7f14aaffd700 1 -- 172.21.15.16:0/1627823161 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a0011d20 con 0x7f14ac09e000 2023-09-30T22:36:14.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.892+0000 7f14b29e0700 1 -- 172.21.15.16:0/1627823161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14ac09e000 msgr2=0x7f14ac130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:14.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.892+0000 7f14b29e0700 1 --2- 172.21.15.16:0/1627823161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14ac09e000 0x7f14ac130660 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f14a0009920 tx=0x7f14a0009c30 comp rx=0 tx=0).stop 2023-09-30T22:36:14.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.892+0000 7f14b29e0700 1 -- 172.21.15.16:0/1627823161 shutdown_connections 2023-09-30T22:36:14.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.892+0000 7f14b29e0700 1 --2- 172.21.15.16:0/1627823161 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f14ac09e410 0x7f14ac130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:14.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.892+0000 7f14b29e0700 1 --2- 172.21.15.16:0/1627823161 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f14ac09d6f0 0x7f14ac09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:14.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.892+0000 7f14b29e0700 1 --2- 172.21.15.16:0/1627823161 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14ac09e000 0x7f14ac130660 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:14.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.892+0000 7f14b29e0700 1 -- 172.21.15.16:0/1627823161 >> 172.21.15.16:0/1627823161 conn(0x7f14ac066f50 msgr2=0x7f14ac09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:14.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.892+0000 7f14b29e0700 1 -- 172.21.15.16:0/1627823161 shutdown_connections 2023-09-30T22:36:14.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.892+0000 7f14b29e0700 1 -- 172.21.15.16:0/1627823161 wait complete. 2023-09-30T22:36:14.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.893+0000 7f14b29e0700 1 Processor -- start 2023-09-30T22:36:14.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.893+0000 7f14b29e0700 1 -- start start 2023-09-30T22:36:14.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.894+0000 7f14b29e0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f14ac09d6f0 0x7f14ac11e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:14.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.894+0000 7f14b29e0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f14ac09e000 0x7f14ac11e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:14.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.894+0000 7f14b29e0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14ac09e410 0x7f14ac11ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:14.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.894+0000 7f14b29e0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ac09c700 con 0x7f14ac09e410 2023-09-30T22:36:14.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.894+0000 7f14b29e0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14ac1202d0 con 0x7f14ac09e000 2023-09-30T22:36:14.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.894+0000 7f14b29e0700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f14ac0a2020 con 0x7f14ac09d6f0 2023-09-30T22:36:14.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.894+0000 7f14b0f7d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14ac09e410 0x7f14ac11ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:14.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.894+0000 7f14b0f7d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14ac09e410 0x7f14ac11ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52078/0 (socket says 172.21.15.16:52078) 2023-09-30T22:36:14.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.894+0000 7f14b0f7d700 1 -- 172.21.15.16:0/1047734134 learned_addr learned my addr 172.21.15.16:0/1047734134 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:14.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.894+0000 7f14ab7fe700 1 --2- 172.21.15.16:0/1047734134 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f14ac09e000 0x7f14ac11e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:14.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.894+0000 7f14abfff700 1 --2- 172.21.15.16:0/1047734134 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f14ac09d6f0 0x7f14ac11e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:14.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.894+0000 7f14ab7fe700 1 -- 172.21.15.16:0/1047734134 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f14ac09d6f0 msgr2=0x7f14ac11e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:14.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.894+0000 7f14ab7fe700 1 --2- 172.21.15.16:0/1047734134 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f14ac09d6f0 0x7f14ac11e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:14.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.894+0000 7f14ab7fe700 1 -- 172.21.15.16:0/1047734134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14ac09e410 msgr2=0x7f14ac11ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:14.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.894+0000 7f14ab7fe700 1 --2- 172.21.15.16:0/1047734134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14ac09e410 0x7f14ac11ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:14.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.894+0000 7f14ab7fe700 1 -- 172.21.15.16:0/1047734134 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f149c0094a0 con 0x7f14ac09e000 2023-09-30T22:36:14.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.894+0000 7f14b0f7d700 1 --2- 172.21.15.16:0/1047734134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14ac09e410 0x7f14ac11ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:14.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.894+0000 7f14ab7fe700 1 --2- 172.21.15.16:0/1047734134 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f14ac09e000 0x7f14ac11e6e0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f14a0016000 tx=0x7f14a0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:14.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.895+0000 7f14a8ff9700 1 -- 172.21.15.16:0/1047734134 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a0004240 con 0x7f14ac09e000 2023-09-30T22:36:14.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.895+0000 7f14a8ff9700 1 -- 172.21.15.16:0/1047734134 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a001f070 con 0x7f14ac09e000 2023-09-30T22:36:14.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.895+0000 7f14a8ff9700 1 -- 172.21.15.16:0/1047734134 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a00036a0 con 0x7f14ac09e000 2023-09-30T22:36:14.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.895+0000 7f14b29e0700 1 -- 172.21.15.16:0/1047734134 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14a0009550 con 0x7f14ac09e000 2023-09-30T22:36:14.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.895+0000 7f14b29e0700 1 -- 172.21.15.16:0/1047734134 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14ac121950 con 0x7f14ac09e000 2023-09-30T22:36:14.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.896+0000 7f14a8ff9700 1 -- 172.21.15.16:0/1047734134 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f14a0021030 con 0x7f14ac09e000 2023-09-30T22:36:14.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.896+0000 7f14a8ff9700 1 -- 172.21.15.16:0/1047734134 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f14a0040950 con 0x7f14ac09e000 2023-09-30T22:36:14.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.896+0000 7f14b29e0700 1 -- 172.21.15.16:0/1047734134 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14ac120f70 con 0x7f14ac09e000 2023-09-30T22:36:14.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:14.906+0000 7f14a8ff9700 1 -- 172.21.15.16:0/1047734134 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f14a00239c0 con 0x7f14ac09e000 2023-09-30T22:36:15.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:15.035+0000 7f244ed9c700 1 -- 172.21.15.16:0/3847963851 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f24481330b0 con 0x7f244812cf60 2023-09-30T22:36:15.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:15.036+0000 7f24457fa700 1 -- 172.21.15.16:0/3847963851 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f243c011350 con 0x7f244812cf60 2023-09-30T22:36:15.037 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:36:15.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:15.039+0000 7f244ed9c700 1 -- 172.21.15.16:0/3847963851 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f244812cf60 msgr2=0x7f24481309e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:15.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:15.039+0000 7f244ed9c700 1 --2- 172.21.15.16:0/3847963851 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f244812cf60 0x7f24481309e0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f243c00e9d0 tx=0x7f243c00ece0 comp rx=0 tx=0).stop 2023-09-30T22:36:15.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:15.040+0000 7f244ed9c700 1 -- 172.21.15.16:0/3847963851 shutdown_connections 2023-09-30T22:36:15.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:15.040+0000 7f244ed9c700 1 --2- 172.21.15.16:0/3847963851 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f244812cf60 0x7f24481309e0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:15.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:15.040+0000 7f244ed9c700 1 --2- 172.21.15.16:0/3847963851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2448130260 0x7f2448131460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:15.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:15.040+0000 7f244ed9c700 1 --2- 172.21.15.16:0/3847963851 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f244812f8b0 0x7f2448130f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:15.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:15.040+0000 7f244ed9c700 1 -- 172.21.15.16:0/3847963851 >> 172.21.15.16:0/3847963851 conn(0x7f24480a2aa0 msgr2=0x7f244806e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:15.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:15.040+0000 7f244ed9c700 1 -- 172.21.15.16:0/3847963851 shutdown_connections 2023-09-30T22:36:15.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:15.040+0000 7f244ed9c700 1 -- 172.21.15.16:0/3847963851 wait complete. 2023-09-30T22:36:15.055 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:36:15.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:15.073+0000 7f2eee5be700 1 -- 172.21.15.16:0/3194892234 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f2ee8124810 con 0x7f2ee80a09c0 2023-09-30T22:36:15.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:15.073+0000 7f2ee4ff9700 1 -- 172.21.15.16:0/3194892234 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2ee8124810 con 0x7f2ee80a09c0 2023-09-30T22:36:15.074 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:36:15.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:15.076+0000 7f2eee5be700 1 -- 172.21.15.16:0/3194892234 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2ee80a09c0 msgr2=0x7f2ee81216b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:15.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:15.076+0000 7f2eee5be700 1 --2- 172.21.15.16:0/3194892234 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2ee80a09c0 0x7f2ee81216b0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f2edc00c750 tx=0x7f2edc00ca60 comp rx=0 tx=0).stop 2023-09-30T22:36:15.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:15.076+0000 7f2eee5be700 1 -- 172.21.15.16:0/3194892234 shutdown_connections 2023-09-30T22:36:15.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:15.076+0000 7f2eee5be700 1 --2- 172.21.15.16:0/3194892234 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2ee809d5c0 0x7f2ee8121170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:15.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:15.076+0000 7f2eee5be700 1 --2- 172.21.15.16:0/3194892234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ee80a12d0 0x7f2ee8121bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:15.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:15.076+0000 7f2eee5be700 1 --2- 172.21.15.16:0/3194892234 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2ee80a09c0 0x7f2ee81216b0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:15.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:15.076+0000 7f2eee5be700 1 -- 172.21.15.16:0/3194892234 >> 172.21.15.16:0/3194892234 conn(0x7f2ee80a2fe0 msgr2=0x7f2ee81bafb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:15.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:15.076+0000 7f2eee5be700 1 -- 172.21.15.16:0/3194892234 shutdown_connections 2023-09-30T22:36:15.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:15.076+0000 7f2eee5be700 1 -- 172.21.15.16:0/3194892234 wait complete. 2023-09-30T22:36:15.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:15.086+0000 7f79a0ad9700 1 -- 172.21.15.16:0/548815794 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f799c0706d0 con 0x7f799c134610 2023-09-30T22:36:15.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:15.086+0000 7f798affd700 1 -- 172.21.15.16:0/548815794 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f799c0706d0 con 0x7f799c134610 2023-09-30T22:36:15.087 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:36:15.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:15.089+0000 7f79a0ad9700 1 -- 172.21.15.16:0/548815794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f799c134610 msgr2=0x7f799c122930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:15.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:15.089+0000 7f79a0ad9700 1 --2- 172.21.15.16:0/548815794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f799c134610 0x7f799c122930 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f799000c750 tx=0x7f799000ca60 comp rx=0 tx=0).stop 2023-09-30T22:36:15.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:15.089+0000 7f79a0ad9700 1 -- 172.21.15.16:0/548815794 shutdown_connections 2023-09-30T22:36:15.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:15.089+0000 7f79a0ad9700 1 --2- 172.21.15.16:0/548815794 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f799c122e70 0x7f799c1cca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:15.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:15.089+0000 7f79a0ad9700 1 --2- 172.21.15.16:0/548815794 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f799c06e570 0x7f799c1223f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:15.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:15.089+0000 7f79a0ad9700 1 --2- 172.21.15.16:0/548815794 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f799c134610 0x7f799c122930 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:15.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:15.089+0000 7f79a0ad9700 1 -- 172.21.15.16:0/548815794 >> 172.21.15.16:0/548815794 conn(0x7f799c0a2aa0 msgr2=0x7f799c0a0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:15.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:15.089+0000 7f79a0ad9700 1 -- 172.21.15.16:0/548815794 shutdown_connections 2023-09-30T22:36:15.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:15.089+0000 7f79a0ad9700 1 -- 172.21.15.16:0/548815794 wait complete. 2023-09-30T22:36:15.091 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:36:15.103 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:36:15.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:15.131+0000 7f14b29e0700 1 -- 172.21.15.16:0/1047734134 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f14ac074ea0 con 0x7f14ac09e000 2023-09-30T22:36:15.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:15.131+0000 7f14a8ff9700 1 -- 172.21.15.16:0/1047734134 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f14a0015020 con 0x7f14ac09e000 2023-09-30T22:36:15.132 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:36:15.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:15.134+0000 7f14b29e0700 1 -- 172.21.15.16:0/1047734134 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f14ac09e000 msgr2=0x7f14ac11e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:15.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:15.134+0000 7f14b29e0700 1 --2- 172.21.15.16:0/1047734134 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f14ac09e000 0x7f14ac11e6e0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f14a0016000 tx=0x7f14a0004750 comp rx=0 tx=0).stop 2023-09-30T22:36:15.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:15.134+0000 7f14b29e0700 1 -- 172.21.15.16:0/1047734134 shutdown_connections 2023-09-30T22:36:15.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:15.134+0000 7f14b29e0700 1 --2- 172.21.15.16:0/1047734134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14ac09e410 0x7f14ac11ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:15.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:15.134+0000 7f14b29e0700 1 --2- 172.21.15.16:0/1047734134 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f14ac09d6f0 0x7f14ac11e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:15.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:15.134+0000 7f14b29e0700 1 --2- 172.21.15.16:0/1047734134 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f14ac09e000 0x7f14ac11e6e0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:15.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:15.134+0000 7f14b29e0700 1 -- 172.21.15.16:0/1047734134 >> 172.21.15.16:0/1047734134 conn(0x7f14ac066f50 msgr2=0x7f14ac070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:15.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:15.134+0000 7f14b29e0700 1 -- 172.21.15.16:0/1047734134 shutdown_connections 2023-09-30T22:36:15.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:15.134+0000 7f14b29e0700 1 -- 172.21.15.16:0/1047734134 wait complete. 2023-09-30T22:36:15.148 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:36:16.056 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:36:16.092 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:36:16.104 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:36:16.149 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:36:16.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.205+0000 7fce9255b700 1 Processor -- start 2023-09-30T22:36:16.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.205+0000 7fce9255b700 1 -- start start 2023-09-30T22:36:16.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.206+0000 7fce9255b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fce8c1338a0 0x7fce8c133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:16.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.206+0000 7fce9255b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fce8c09de50 0x7fce8c09e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:16.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.206+0000 7fce9255b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fce8c09e760 0x7fce8c13a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:16.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.206+0000 7fce9255b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce8c06fef0 con 0x7fce8c1338a0 2023-09-30T22:36:16.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.206+0000 7fce9255b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce8c06ef30 con 0x7fce8c09e760 2023-09-30T22:36:16.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.206+0000 7fce9255b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fce8c1334b0 con 0x7fce8c09de50 2023-09-30T22:36:16.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.206+0000 7fce8b7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fce8c09de50 0x7fce8c09e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:16.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.206+0000 7fce8b7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fce8c09de50 0x7fce8c09e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:44094/0 (socket says 172.21.15.16:44094) 2023-09-30T22:36:16.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.206+0000 7fce8b7fe700 1 -- 172.21.15.16:0/1349211425 learned_addr learned my addr 172.21.15.16:0/1349211425 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:16.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.206+0000 7fce90af8700 1 --2- 172.21.15.16:0/1349211425 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fce8c09e760 0x7fce8c13a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:16.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.206+0000 7fce8b7fe700 1 -- 172.21.15.16:0/1349211425 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fce8c09e760 msgr2=0x7fce8c13a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:16.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.206+0000 7fce8b7fe700 1 --2- 172.21.15.16:0/1349211425 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fce8c09e760 0x7fce8c13a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:16.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.206+0000 7fce8b7fe700 1 -- 172.21.15.16:0/1349211425 >> v2:172.21.15.16:3300/0 conn(0x7fce8c1338a0 msgr2=0x7fce8c133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:16.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.206+0000 7fce8b7fe700 1 --2- 172.21.15.16:0/1349211425 >> v2:172.21.15.16:3300/0 conn(0x7fce8c1338a0 0x7fce8c133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:16.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.206+0000 7fce8b7fe700 1 -- 172.21.15.16:0/1349211425 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce8c06fc30 con 0x7fce8c09de50 2023-09-30T22:36:16.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.206+0000 7fce90af8700 1 --2- 172.21.15.16:0/1349211425 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fce8c09e760 0x7fce8c13a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:16.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.207+0000 7fce8b7fe700 1 --2- 172.21.15.16:0/1349211425 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fce8c09de50 0x7fce8c09e220 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fce800098f0 tx=0x7fce80009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bb5ce7d8b3846da server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:16.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.207+0000 7fce8affd700 1 -- 172.21.15.16:0/1349211425 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce80011830 con 0x7fce8c09de50 2023-09-30T22:36:16.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.207+0000 7fce8affd700 1 -- 172.21.15.16:0/1349211425 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce80011990 con 0x7fce8c09de50 2023-09-30T22:36:16.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.207+0000 7fce8affd700 1 -- 172.21.15.16:0/1349211425 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce80011d20 con 0x7fce8c09de50 2023-09-30T22:36:16.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.208+0000 7fce9255b700 1 -- 172.21.15.16:0/1349211425 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fce8c09de50 msgr2=0x7fce8c09e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:16.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.208+0000 7fce9255b700 1 --2- 172.21.15.16:0/1349211425 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fce8c09de50 0x7fce8c09e220 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fce800098f0 tx=0x7fce80009c00 comp rx=0 tx=0).stop 2023-09-30T22:36:16.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.208+0000 7fce9255b700 1 -- 172.21.15.16:0/1349211425 shutdown_connections 2023-09-30T22:36:16.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.208+0000 7fce9255b700 1 --2- 172.21.15.16:0/1349211425 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fce8c09e760 0x7fce8c13a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:16.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.208+0000 7fce9255b700 1 --2- 172.21.15.16:0/1349211425 >> v2:172.21.15.16:3300/0 conn(0x7fce8c1338a0 0x7fce8c133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:16.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.208+0000 7fce9255b700 1 --2- 172.21.15.16:0/1349211425 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fce8c09de50 0x7fce8c09e220 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:16.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.208+0000 7fce9255b700 1 -- 172.21.15.16:0/1349211425 >> 172.21.15.16:0/1349211425 conn(0x7fce8c0a2aa0 msgr2=0x7fce8c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:16.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.208+0000 7fce9255b700 1 -- 172.21.15.16:0/1349211425 shutdown_connections 2023-09-30T22:36:16.212 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.208+0000 7fce9255b700 1 -- 172.21.15.16:0/1349211425 wait complete. 2023-09-30T22:36:16.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.209+0000 7fce9255b700 1 Processor -- start 2023-09-30T22:36:16.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.209+0000 7fce9255b700 1 -- start start 2023-09-30T22:36:16.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.210+0000 7fce9255b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce8c09de50 0x7fce8c11ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:16.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.210+0000 7fce9255b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fce8c09e760 0x7fce8c11f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:16.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.210+0000 7fce9255b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fce8c1338a0 0x7fce8c11f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:16.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.210+0000 7fce9255b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce8c121d70 con 0x7fce8c09de50 2023-09-30T22:36:16.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.210+0000 7fce9255b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce8c120710 con 0x7fce8c09e760 2023-09-30T22:36:16.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.210+0000 7fce9255b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fce8c122690 con 0x7fce8c1338a0 2023-09-30T22:36:16.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.210+0000 7fce90af8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fce8c1338a0 0x7fce8c11f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:16.213 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.210+0000 7fce90af8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fce8c1338a0 0x7fce8c11f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:44098/0 (socket says 172.21.15.16:44098) 2023-09-30T22:36:16.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.210+0000 7fce90af8700 1 -- 172.21.15.16:0/1971111672 learned_addr learned my addr 172.21.15.16:0/1971111672 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:16.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.210+0000 7fce8b7fe700 1 --2- 172.21.15.16:0/1971111672 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fce8c09e760 0x7fce8c11f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:16.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.210+0000 7fce8bfff700 1 --2- 172.21.15.16:0/1971111672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce8c09de50 0x7fce8c11ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:16.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.210+0000 7fce90af8700 1 -- 172.21.15.16:0/1971111672 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fce8c09e760 msgr2=0x7fce8c11f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:16.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.210+0000 7fce90af8700 1 --2- 172.21.15.16:0/1971111672 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fce8c09e760 0x7fce8c11f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:16.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.210+0000 7fce90af8700 1 -- 172.21.15.16:0/1971111672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce8c09de50 msgr2=0x7fce8c11ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:16.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.210+0000 7fce90af8700 1 --2- 172.21.15.16:0/1971111672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce8c09de50 0x7fce8c11ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:16.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.210+0000 7fce90af8700 1 -- 172.21.15.16:0/1971111672 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce84009a70 con 0x7fce8c1338a0 2023-09-30T22:36:16.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.210+0000 7fce8bfff700 1 --2- 172.21.15.16:0/1971111672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce8c09de50 0x7fce8c11ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:36:16.214 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.211+0000 7fce90af8700 1 --2- 172.21.15.16:0/1971111672 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fce8c1338a0 0x7fce8c11f670 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fce8400efc0 tx=0x7fce8400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:16.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.211+0000 7fce88ff9700 1 -- 172.21.15.16:0/1971111672 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8400cb40 con 0x7fce8c1338a0 2023-09-30T22:36:16.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.211+0000 7fce88ff9700 1 -- 172.21.15.16:0/1971111672 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce8400f040 con 0x7fce8c1338a0 2023-09-30T22:36:16.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.211+0000 7fce9255b700 1 -- 172.21.15.16:0/1971111672 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce80009550 con 0x7fce8c1338a0 2023-09-30T22:36:16.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.211+0000 7fce88ff9700 1 -- 172.21.15.16:0/1971111672 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce84003a20 con 0x7fce8c1338a0 2023-09-30T22:36:16.215 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.211+0000 7fce9255b700 1 -- 172.21.15.16:0/1971111672 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce8c0a1100 con 0x7fce8c1338a0 2023-09-30T22:36:16.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.212+0000 7fce88ff9700 1 -- 172.21.15.16:0/1971111672 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fce84014480 con 0x7fce8c1338a0 2023-09-30T22:36:16.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.212+0000 7fce9255b700 1 -- 172.21.15.16:0/1971111672 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce8c06f6a0 con 0x7fce8c1338a0 2023-09-30T22:36:16.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.212+0000 7fce88ff9700 1 -- 172.21.15.16:0/1971111672 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fce8403fff0 con 0x7fce8c1338a0 2023-09-30T22:36:16.218 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.218+0000 7fce88ff9700 1 -- 172.21.15.16:0/1971111672 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fce84012930 con 0x7fce8c1338a0 2023-09-30T22:36:16.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.246+0000 7f20c4afc700 1 Processor -- start 2023-09-30T22:36:16.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.246+0000 7f20c4afc700 1 -- start start 2023-09-30T22:36:16.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.246+0000 7f20c4afc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f20c006f5d0 0x7f20c012ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:16.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.246+0000 7f20c4afc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f20c0135c20 0x7f20c0138000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:16.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.247+0000 7f20c4afc700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20c006fa40 con 0x7f20c012cf60 2023-09-30T22:36:16.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.247+0000 7f20c4afc700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20c012cd90 con 0x7f20c006f5d0 2023-09-30T22:36:16.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.247+0000 7f20c4afc700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f20c0071780 con 0x7f20c0135c20 2023-09-30T22:36:16.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.247+0000 7f20bed9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f20c0135c20 0x7f20c0138000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:16.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.247+0000 7f20be59c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f20c012cf60 0x7f20c012f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52234/0 (socket says 172.21.15.16:52234) 2023-09-30T22:36:16.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.247+0000 7f20bed9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f20c0135c20 0x7f20c0138000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:44108/0 (socket says 172.21.15.16:44108) 2023-09-30T22:36:16.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.247+0000 7f20be59c700 1 -- 172.21.15.16:0/73707279 learned_addr learned my addr 172.21.15.16:0/73707279 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:36:16.248 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.247+0000 7f20bdd9b700 1 --2- 172.21.15.16:0/73707279 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f20c006f5d0 0x7f20c012ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:16.249 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.247+0000 7f20bd59a700 1 -- 172.21.15.16:0/73707279 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3787917552 0 0) 0x7f20c006fa40 con 0x7f20c012cf60 2023-09-30T22:36:16.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.247+0000 7f20bd59a700 1 -- 172.21.15.16:0/73707279 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20a80034d0 con 0x7f20c012cf60 2023-09-30T22:36:16.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.247+0000 7f20bed9d700 1 -- 172.21.15.16:0/73707279 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f20c006f5d0 msgr2=0x7f20c012ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:16.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.247+0000 7f20bed9d700 1 --2- 172.21.15.16:0/73707279 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f20c006f5d0 0x7f20c012ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:16.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.247+0000 7f20bed9d700 1 -- 172.21.15.16:0/73707279 >> v1:172.21.15.16:6789/0 conn(0x7f20c012cf60 legacy=0x7f20c012f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:16.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.247+0000 7f20bed9d700 1 -- 172.21.15.16:0/73707279 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c0071a40 con 0x7f20c0135c20 2023-09-30T22:36:16.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.247+0000 7f20bdd9b700 1 --2- 172.21.15.16:0/73707279 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f20c006f5d0 0x7f20c012ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:16.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.248+0000 7f20bed9d700 1 --2- 172.21.15.16:0/73707279 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f20c0135c20 0x7f20c0138000 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f20b8009df0 tx=0x7f20b800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a4ce9ef55f23ee5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:16.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.248+0000 7f20bd59a700 1 -- 172.21.15.16:0/73707279 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b800e070 con 0x7f20c0135c20 2023-09-30T22:36:16.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.248+0000 7f20bd59a700 1 -- 172.21.15.16:0/73707279 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20b800bae0 con 0x7f20c0135c20 2023-09-30T22:36:16.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.248+0000 7f20bd59a700 1 -- 172.21.15.16:0/73707279 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b800be70 con 0x7f20c0135c20 2023-09-30T22:36:16.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.249+0000 7f20c4afc700 1 -- 172.21.15.16:0/73707279 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f20c0135c20 msgr2=0x7f20c0138000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:16.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.249+0000 7f20c4afc700 1 --2- 172.21.15.16:0/73707279 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f20c0135c20 0x7f20c0138000 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f20b8009df0 tx=0x7f20b800b330 comp rx=0 tx=0).stop 2023-09-30T22:36:16.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.249+0000 7f20c4afc700 1 -- 172.21.15.16:0/73707279 shutdown_connections 2023-09-30T22:36:16.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.249+0000 7f20c4afc700 1 --2- 172.21.15.16:0/73707279 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f20c0135c20 0x7f20c0138000 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:16.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.249+0000 7f20c4afc700 1 --2- 172.21.15.16:0/73707279 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f20c006f5d0 0x7f20c012ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:16.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.249+0000 7f20c4afc700 1 -- 172.21.15.16:0/73707279 >> 172.21.15.16:0/73707279 conn(0x7f20c00a2aa0 msgr2=0x7f20c00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:16.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.249+0000 7f20c4afc700 1 -- 172.21.15.16:0/73707279 shutdown_connections 2023-09-30T22:36:16.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.249+0000 7f20c4afc700 1 -- 172.21.15.16:0/73707279 wait complete. 2023-09-30T22:36:16.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.250+0000 7f20c4afc700 1 Processor -- start 2023-09-30T22:36:16.253 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.250+0000 7f20c4afc700 1 -- start start 2023-09-30T22:36:16.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.251+0000 7f20c4afc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f20c006f5d0 0x7f20c01c3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:16.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.251+0000 7f20c4afc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20c012cf60 0x7f20c01c44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:16.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.251+0000 7f20c4afc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f20c0135c20 0x7f20c01c49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:16.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.251+0000 7f20c4afc700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c009c990 con 0x7f20c012cf60 2023-09-30T22:36:16.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.251+0000 7f20c4afc700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20c00a1500 con 0x7f20c006f5d0 2023-09-30T22:36:16.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.251+0000 7f20c4afc700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f20c0129fa0 con 0x7f20c0135c20 2023-09-30T22:36:16.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.251+0000 7f20bdd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20c012cf60 0x7f20c01c44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:16.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.251+0000 7f20bdd9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20c012cf60 0x7f20c01c44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52094/0 (socket says 172.21.15.16:52094) 2023-09-30T22:36:16.254 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.251+0000 7f20be59c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f20c006f5d0 0x7f20c01c3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:16.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.251+0000 7f20bdd9b700 1 -- 172.21.15.16:0/1687285558 learned_addr learned my addr 172.21.15.16:0/1687285558 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:16.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.252+0000 7f20bed9d700 1 --2- 172.21.15.16:0/1687285558 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f20c0135c20 0x7f20c01c49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:16.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.252+0000 7f20bed9d700 1 -- 172.21.15.16:0/1687285558 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f20c006f5d0 msgr2=0x7f20c01c3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:16.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.252+0000 7f20bed9d700 1 --2- 172.21.15.16:0/1687285558 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f20c006f5d0 0x7f20c01c3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:16.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.252+0000 7f20bed9d700 1 -- 172.21.15.16:0/1687285558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20c012cf60 msgr2=0x7f20c01c44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:16.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.252+0000 7f20bed9d700 1 --2- 172.21.15.16:0/1687285558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20c012cf60 0x7f20c01c44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:16.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.252+0000 7f20bed9d700 1 -- 172.21.15.16:0/1687285558 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b40094a0 con 0x7f20c0135c20 2023-09-30T22:36:16.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.252+0000 7f20be59c700 1 --2- 172.21.15.16:0/1687285558 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f20c006f5d0 0x7f20c01c3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:16.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.252+0000 7f20bdd9b700 1 --2- 172.21.15.16:0/1687285558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20c012cf60 0x7f20c01c44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:16.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.252+0000 7f20bed9d700 1 --2- 172.21.15.16:0/1687285558 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f20c0135c20 0x7f20c01c49f0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f20c0070520 tx=0x7f20b8004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:16.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.252+0000 7f20aeffd700 1 -- 172.21.15.16:0/1687285558 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b80044d0 con 0x7f20c0135c20 2023-09-30T22:36:16.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.253+0000 7f20aeffd700 1 -- 172.21.15.16:0/1687285558 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20b8004670 con 0x7f20c0135c20 2023-09-30T22:36:16.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.253+0000 7f20aeffd700 1 -- 172.21.15.16:0/1687285558 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b8012430 con 0x7f20c0135c20 2023-09-30T22:36:16.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.253+0000 7f20c4afc700 1 -- 172.21.15.16:0/1687285558 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20b8009a70 con 0x7f20c0135c20 2023-09-30T22:36:16.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.253+0000 7f20c4afc700 1 -- 172.21.15.16:0/1687285558 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20c009d850 con 0x7f20c0135c20 2023-09-30T22:36:16.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.254+0000 7f20c4afc700 1 -- 172.21.15.16:0/1687285558 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20c0074ea0 con 0x7f20c0135c20 2023-09-30T22:36:16.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.255+0000 7f20aeffd700 1 -- 172.21.15.16:0/1687285558 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f20b8012590 con 0x7f20c0135c20 2023-09-30T22:36:16.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.255+0000 7f20aeffd700 1 -- 172.21.15.16:0/1687285558 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f20b80414a0 con 0x7f20c0135c20 2023-09-30T22:36:16.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.264+0000 7f20aeffd700 1 -- 172.21.15.16:0/1687285558 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f20b802e940 con 0x7f20c0135c20 2023-09-30T22:36:16.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.294+0000 7fb4ee28a700 1 Processor -- start 2023-09-30T22:36:16.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.294+0000 7fb4ee28a700 1 -- start start 2023-09-30T22:36:16.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.294+0000 7fb4ee28a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb4e80954c0 0x7fb4e8095890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:16.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.295+0000 7fb4ee28a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb4e8099680 0x7fb4e8099ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:16.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.295+0000 7fb4ee28a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb4e8099ff0 0x7fb4e8137590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:16.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.295+0000 7fb4ee28a700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4e809aaf0 con 0x7fb4e80954c0 2023-09-30T22:36:16.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.295+0000 7fb4ee28a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e809c280 con 0x7fb4e8099680 2023-09-30T22:36:16.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.295+0000 7fb4ee28a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e809b960 con 0x7fb4e8099ff0 2023-09-30T22:36:16.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.295+0000 7fb4ec827700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb4e8099ff0 0x7fb4e8137590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:16.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.295+0000 7fb4ec827700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb4e8099ff0 0x7fb4e8137590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:44136/0 (socket says 172.21.15.16:44136) 2023-09-30T22:36:16.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.295+0000 7fb4ec827700 1 -- 172.21.15.16:0/1589596519 learned_addr learned my addr 172.21.15.16:0/1589596519 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:16.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.295+0000 7fb4e7fff700 1 --2- 172.21.15.16:0/1589596519 >> v2:172.21.15.16:3300/0 conn(0x7fb4e80954c0 0x7fb4e8095890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:16.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.295+0000 7fb4dffff700 1 --2- 172.21.15.16:0/1589596519 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb4e8099680 0x7fb4e8099ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:16.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.295+0000 7fb4ec827700 1 -- 172.21.15.16:0/1589596519 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb4e8099680 msgr2=0x7fb4e8099ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:16.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.295+0000 7fb4ec827700 1 --2- 172.21.15.16:0/1589596519 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb4e8099680 0x7fb4e8099ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:16.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.295+0000 7fb4ec827700 1 -- 172.21.15.16:0/1589596519 >> v2:172.21.15.16:3300/0 conn(0x7fb4e80954c0 msgr2=0x7fb4e8095890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:16.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.295+0000 7fb4ec827700 1 --2- 172.21.15.16:0/1589596519 >> v2:172.21.15.16:3300/0 conn(0x7fb4e80954c0 0x7fb4e8095890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:16.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.295+0000 7fb4ec827700 1 -- 172.21.15.16:0/1589596519 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4e809a7b0 con 0x7fb4e8099ff0 2023-09-30T22:36:16.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.295+0000 7fb4ec827700 1 --2- 172.21.15.16:0/1589596519 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb4e8099ff0 0x7fb4e8137590 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0009e50 tx=0x7fb4e000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48577b062d3ef289 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:16.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.296+0000 7fb4e77fe700 1 -- 172.21.15.16:0/1589596519 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e000e070 con 0x7fb4e8099ff0 2023-09-30T22:36:16.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.296+0000 7fb4e77fe700 1 -- 172.21.15.16:0/1589596519 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4e000b910 con 0x7fb4e8099ff0 2023-09-30T22:36:16.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.296+0000 7fb4e77fe700 1 -- 172.21.15.16:0/1589596519 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e000bcc0 con 0x7fb4e8099ff0 2023-09-30T22:36:16.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.297+0000 7fb4ee28a700 1 -- 172.21.15.16:0/1589596519 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb4e8099ff0 msgr2=0x7fb4e8137590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:16.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.297+0000 7fb4ee28a700 1 --2- 172.21.15.16:0/1589596519 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb4e8099ff0 0x7fb4e8137590 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0009e50 tx=0x7fb4e000b330 comp rx=0 tx=0).stop 2023-09-30T22:36:16.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.297+0000 7fb4ee28a700 1 -- 172.21.15.16:0/1589596519 shutdown_connections 2023-09-30T22:36:16.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.297+0000 7fb4ee28a700 1 --2- 172.21.15.16:0/1589596519 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb4e8099ff0 0x7fb4e8137590 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:16.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.297+0000 7fb4ee28a700 1 --2- 172.21.15.16:0/1589596519 >> v2:172.21.15.16:3300/0 conn(0x7fb4e80954c0 0x7fb4e8095890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:16.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.297+0000 7fb4ee28a700 1 --2- 172.21.15.16:0/1589596519 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb4e8099680 0x7fb4e8099ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:16.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.297+0000 7fb4ee28a700 1 -- 172.21.15.16:0/1589596519 >> 172.21.15.16:0/1589596519 conn(0x7fb4e80a2360 msgr2=0x7fb4e80a2760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:16.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.297+0000 7fb4ee28a700 1 -- 172.21.15.16:0/1589596519 shutdown_connections 2023-09-30T22:36:16.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.297+0000 7fb4ee28a700 1 -- 172.21.15.16:0/1589596519 wait complete. 2023-09-30T22:36:16.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.298+0000 7fb4ee28a700 1 Processor -- start 2023-09-30T22:36:16.302 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.298+0000 7fb4ee28a700 1 -- start start 2023-09-30T22:36:16.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.299+0000 7fb4ee28a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e80954c0 0x7fb4e808f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:16.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.299+0000 7fb4ee28a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb4e8099680 0x7fb4e808cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:16.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.299+0000 7fb4ee28a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb4e8099ff0 0x7fb4e808d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:16.304 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.299+0000 7fb4ee28a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e8090880 con 0x7fb4e80954c0 2023-09-30T22:36:16.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.299+0000 7fb4ee28a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e809b000 con 0x7fb4e8099680 2023-09-30T22:36:16.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.299+0000 7fb4ee28a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e80a05f0 con 0x7fb4e8099ff0 2023-09-30T22:36:16.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.299+0000 7fb4e7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e80954c0 0x7fb4e808f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:16.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.299+0000 7fb4e7fff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e80954c0 0x7fb4e808f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52110/0 (socket says 172.21.15.16:52110) 2023-09-30T22:36:16.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.299+0000 7fb4e7fff700 1 -- 172.21.15.16:0/3654570375 learned_addr learned my addr 172.21.15.16:0/3654570375 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:16.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.299+0000 7fb4ec827700 1 --2- 172.21.15.16:0/3654570375 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb4e8099ff0 0x7fb4e808d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:16.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.299+0000 7fb4dffff700 1 --2- 172.21.15.16:0/3654570375 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb4e8099680 0x7fb4e808cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:16.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.299+0000 7fb4ec827700 1 -- 172.21.15.16:0/3654570375 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb4e8099680 msgr2=0x7fb4e808cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:16.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.299+0000 7fb4ec827700 1 --2- 172.21.15.16:0/3654570375 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb4e8099680 0x7fb4e808cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:16.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.299+0000 7fb4ec827700 1 -- 172.21.15.16:0/3654570375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e80954c0 msgr2=0x7fb4e808f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:16.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.299+0000 7fb4ec827700 1 --2- 172.21.15.16:0/3654570375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e80954c0 0x7fb4e808f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:16.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.299+0000 7fb4ec827700 1 -- 172.21.15.16:0/3654570375 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4e0009b30 con 0x7fb4e8099ff0 2023-09-30T22:36:16.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.299+0000 7fb4ec827700 1 --2- 172.21.15.16:0/3654570375 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb4e8099ff0 0x7fb4e808d2a0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0015000 tx=0x7fb4e0004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:16.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.300+0000 7fb4e57fa700 1 -- 172.21.15.16:0/3654570375 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e0004450 con 0x7fb4e8099ff0 2023-09-30T22:36:16.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.300+0000 7fb4e57fa700 1 -- 172.21.15.16:0/3654570375 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4e00045b0 con 0x7fb4e8099ff0 2023-09-30T22:36:16.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.300+0000 7fb4ee28a700 1 -- 172.21.15.16:0/3654570375 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4e81375e0 con 0x7fb4e8099ff0 2023-09-30T22:36:16.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.300+0000 7fb4e57fa700 1 -- 172.21.15.16:0/3654570375 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4e0004960 con 0x7fb4e8099ff0 2023-09-30T22:36:16.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.300+0000 7fb4ee28a700 1 -- 172.21.15.16:0/3654570375 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4e808fe30 con 0x7fb4e8099ff0 2023-09-30T22:36:16.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.301+0000 7fb4e57fa700 1 -- 172.21.15.16:0/3654570375 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb4e001c410 con 0x7fb4e8099ff0 2023-09-30T22:36:16.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.301+0000 7fb4ee28a700 1 -- 172.21.15.16:0/3654570375 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4e8074ea0 con 0x7fb4e8099ff0 2023-09-30T22:36:16.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.301+0000 7fb4e57fa700 1 -- 172.21.15.16:0/3654570375 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fb4e00410f0 con 0x7fb4e8099ff0 2023-09-30T22:36:16.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.310+0000 7fb4e57fa700 1 -- 172.21.15.16:0/3654570375 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb4e00247d0 con 0x7fb4e8099ff0 2023-09-30T22:36:16.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.347+0000 7fc7243bb700 1 Processor -- start 2023-09-30T22:36:16.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.348+0000 7fc7243bb700 1 -- start start 2023-09-30T22:36:16.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.349+0000 7fc7243bb700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc71c09d630 0x7fc71c0a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:16.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.349+0000 7fc7243bb700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc71c138d80 0x7fc71c13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:16.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.349+0000 7fc7243bb700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc71c0a2a70 con 0x7fc71c0a0a30 2023-09-30T22:36:16.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.349+0000 7fc7243bb700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc71c0a01d0 con 0x7fc71c09d630 2023-09-30T22:36:16.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.349+0000 7fc7243bb700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc71c09e2b0 con 0x7fc71c138d80 2023-09-30T22:36:16.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.349+0000 7fc721956700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc71c09d630 0x7fc71c0a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:16.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.349+0000 7fc722958700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc71c138d80 0x7fc71c13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:16.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.349+0000 7fc722157700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc71c0a0a30 0x7fc71c0a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:52248/0 (socket says 172.21.15.16:52248) 2023-09-30T22:36:16.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.349+0000 7fc722157700 1 -- 172.21.15.16:0/2487210643 learned_addr learned my addr 172.21.15.16:0/2487210643 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:36:16.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.349+0000 7fc721155700 1 -- 172.21.15.16:0/2487210643 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 353706061 0 0) 0x7fc71c0a2a70 con 0x7fc71c0a0a30 2023-09-30T22:36:16.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.349+0000 7fc722958700 1 -- 172.21.15.16:0/2487210643 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc71c09d630 msgr2=0x7fc71c0a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:16.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.349+0000 7fc722958700 1 --2- 172.21.15.16:0/2487210643 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc71c09d630 0x7fc71c0a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:16.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.349+0000 7fc722958700 1 -- 172.21.15.16:0/2487210643 >> v1:172.21.15.16:6789/0 conn(0x7fc71c0a0a30 legacy=0x7fc71c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:16.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.349+0000 7fc722958700 1 -- 172.21.15.16:0/2487210643 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc71c09dc50 con 0x7fc71c138d80 2023-09-30T22:36:16.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.349+0000 7fc722958700 1 --2- 172.21.15.16:0/2487210643 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc71c138d80 0x7fc71c13b160 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fc714009dc0 tx=0x7fc71400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4c5496a759c2306 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:16.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.350+0000 7fc721155700 1 -- 172.21.15.16:0/2487210643 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71400e070 con 0x7fc71c138d80 2023-09-30T22:36:16.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.350+0000 7fc721155700 1 -- 172.21.15.16:0/2487210643 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc71400bb20 con 0x7fc71c138d80 2023-09-30T22:36:16.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.350+0000 7fc721155700 1 -- 172.21.15.16:0/2487210643 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc714012430 con 0x7fc71c138d80 2023-09-30T22:36:16.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.351+0000 7fc7243bb700 1 -- 172.21.15.16:0/2487210643 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc71c138d80 msgr2=0x7fc71c13b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:16.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.351+0000 7fc7243bb700 1 --2- 172.21.15.16:0/2487210643 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc71c138d80 0x7fc71c13b160 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fc714009dc0 tx=0x7fc71400b330 comp rx=0 tx=0).stop 2023-09-30T22:36:16.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.351+0000 7fc7243bb700 1 -- 172.21.15.16:0/2487210643 shutdown_connections 2023-09-30T22:36:16.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.351+0000 7fc7243bb700 1 --2- 172.21.15.16:0/2487210643 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc71c138d80 0x7fc71c13b160 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:16.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.351+0000 7fc7243bb700 1 --2- 172.21.15.16:0/2487210643 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc71c09d630 0x7fc71c0a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:16.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.351+0000 7fc7243bb700 1 -- 172.21.15.16:0/2487210643 >> 172.21.15.16:0/2487210643 conn(0x7fc71c0a3050 msgr2=0x7fc71c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:16.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.351+0000 7fc7243bb700 1 -- 172.21.15.16:0/2487210643 shutdown_connections 2023-09-30T22:36:16.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.351+0000 7fc7243bb700 1 -- 172.21.15.16:0/2487210643 wait complete. 2023-09-30T22:36:16.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.352+0000 7fc7243bb700 1 Processor -- start 2023-09-30T22:36:16.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.352+0000 7fc7243bb700 1 -- start start 2023-09-30T22:36:16.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.353+0000 7fc7243bb700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc71c09d630 0x7fc71c133770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:16.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.353+0000 7fc7243bb700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc71c0a0a30 0x7fc71c133cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:16.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.353+0000 7fc7243bb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc71c138d80 0x7fc71c1341f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:16.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.353+0000 7fc7243bb700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc71c134c40 con 0x7fc71c138d80 2023-09-30T22:36:16.356 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.353+0000 7fc7243bb700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc71c1361b0 con 0x7fc71c09d630 2023-09-30T22:36:16.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.353+0000 7fc7243bb700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc71c136ed0 con 0x7fc71c0a0a30 2023-09-30T22:36:16.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.353+0000 7fc722958700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc71c138d80 0x7fc71c1341f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:16.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.353+0000 7fc722958700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc71c138d80 0x7fc71c1341f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52118/0 (socket says 172.21.15.16:52118) 2023-09-30T22:36:16.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.353+0000 7fc722958700 1 -- 172.21.15.16:0/4159355709 learned_addr learned my addr 172.21.15.16:0/4159355709 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:16.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.353+0000 7fc722157700 1 --2- 172.21.15.16:0/4159355709 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc71c09d630 0x7fc71c133770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:16.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.353+0000 7fc721956700 1 --2- 172.21.15.16:0/4159355709 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc71c0a0a30 0x7fc71c133cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:16.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.354+0000 7fc722958700 1 -- 172.21.15.16:0/4159355709 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc71c0a0a30 msgr2=0x7fc71c133cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:16.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.354+0000 7fc722958700 1 --2- 172.21.15.16:0/4159355709 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc71c0a0a30 0x7fc71c133cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:16.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.354+0000 7fc722958700 1 -- 172.21.15.16:0/4159355709 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc71c09d630 msgr2=0x7fc71c133770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:16.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.354+0000 7fc722958700 1 --2- 172.21.15.16:0/4159355709 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc71c09d630 0x7fc71c133770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:16.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.354+0000 7fc722958700 1 -- 172.21.15.16:0/4159355709 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc714009a70 con 0x7fc71c138d80 2023-09-30T22:36:16.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.354+0000 7fc722958700 1 --2- 172.21.15.16:0/4159355709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc71c138d80 0x7fc71c1341f0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fc714014040 tx=0x7fc714004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:16.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.354+0000 7fc712ffd700 1 -- 172.21.15.16:0/4159355709 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc714012430 con 0x7fc71c138d80 2023-09-30T22:36:16.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.354+0000 7fc712ffd700 1 -- 172.21.15.16:0/4159355709 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc714020070 con 0x7fc71c138d80 2023-09-30T22:36:16.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.354+0000 7fc7243bb700 1 -- 172.21.15.16:0/4159355709 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc71c0a4c00 con 0x7fc71c138d80 2023-09-30T22:36:16.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.354+0000 7fc712ffd700 1 -- 172.21.15.16:0/4159355709 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc714004040 con 0x7fc71c138d80 2023-09-30T22:36:16.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.354+0000 7fc7243bb700 1 -- 172.21.15.16:0/4159355709 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc71c137940 con 0x7fc71c138d80 2023-09-30T22:36:16.358 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.355+0000 7fc712ffd700 1 -- 172.21.15.16:0/4159355709 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc7140041a0 con 0x7fc71c138d80 2023-09-30T22:36:16.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.356+0000 7fc712ffd700 1 -- 172.21.15.16:0/4159355709 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fc714022070 con 0x7fc71c138d80 2023-09-30T22:36:16.359 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.356+0000 7fc7243bb700 1 -- 172.21.15.16:0/4159355709 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc71c136b70 con 0x7fc71c138d80 2023-09-30T22:36:16.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.361+0000 7fc712ffd700 1 -- 172.21.15.16:0/4159355709 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc71401a970 con 0x7fc71c138d80 2023-09-30T22:36:16.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.432+0000 7fce9255b700 1 -- 172.21.15.16:0/1971111672 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fce8c074ea0 con 0x7fce8c1338a0 2023-09-30T22:36:16.433 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.433+0000 7fce88ff9700 1 -- 172.21.15.16:0/1971111672 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fce8401c3b0 con 0x7fce8c1338a0 2023-09-30T22:36:16.434 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:36:16.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.435+0000 7fce9255b700 1 -- 172.21.15.16:0/1971111672 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fce8c1338a0 msgr2=0x7fce8c11f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:16.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.435+0000 7fce9255b700 1 --2- 172.21.15.16:0/1971111672 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fce8c1338a0 0x7fce8c11f670 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fce8400efc0 tx=0x7fce8400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:36:16.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.435+0000 7fce9255b700 1 -- 172.21.15.16:0/1971111672 shutdown_connections 2023-09-30T22:36:16.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.435+0000 7fce9255b700 1 --2- 172.21.15.16:0/1971111672 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fce8c1338a0 0x7fce8c11f670 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:16.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.435+0000 7fce9255b700 1 --2- 172.21.15.16:0/1971111672 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce8c09de50 0x7fce8c11ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:16.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.435+0000 7fce9255b700 1 --2- 172.21.15.16:0/1971111672 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fce8c09e760 0x7fce8c11f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:16.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.435+0000 7fce9255b700 1 -- 172.21.15.16:0/1971111672 >> 172.21.15.16:0/1971111672 conn(0x7fce8c0a2aa0 msgr2=0x7fce8c09ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:16.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.436+0000 7fce9255b700 1 -- 172.21.15.16:0/1971111672 shutdown_connections 2023-09-30T22:36:16.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.438+0000 7fce9255b700 1 -- 172.21.15.16:0/1971111672 wait complete. 2023-09-30T22:36:16.453 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:36:16.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.478+0000 7f20c4afc700 1 -- 172.21.15.16:0/1687285558 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f20c0135a00 con 0x7f20c0135c20 2023-09-30T22:36:16.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.478+0000 7f20aeffd700 1 -- 172.21.15.16:0/1687285558 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f20b802e350 con 0x7f20c0135c20 2023-09-30T22:36:16.479 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:36:16.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.481+0000 7f20c4afc700 1 -- 172.21.15.16:0/1687285558 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f20c0135c20 msgr2=0x7f20c01c49f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:16.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.481+0000 7f20c4afc700 1 --2- 172.21.15.16:0/1687285558 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f20c0135c20 0x7f20c01c49f0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f20c0070520 tx=0x7f20b8004ad0 comp rx=0 tx=0).stop 2023-09-30T22:36:16.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.481+0000 7f20c4afc700 1 -- 172.21.15.16:0/1687285558 shutdown_connections 2023-09-30T22:36:16.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.481+0000 7f20c4afc700 1 --2- 172.21.15.16:0/1687285558 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f20c006f5d0 0x7f20c01c3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:16.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.481+0000 7f20c4afc700 1 --2- 172.21.15.16:0/1687285558 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f20c0135c20 0x7f20c01c49f0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:16.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.481+0000 7f20c4afc700 1 --2- 172.21.15.16:0/1687285558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20c012cf60 0x7f20c01c44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:16.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.481+0000 7f20c4afc700 1 -- 172.21.15.16:0/1687285558 >> 172.21.15.16:0/1687285558 conn(0x7f20c00a2aa0 msgr2=0x7f20c00a1830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:16.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.481+0000 7f20c4afc700 1 -- 172.21.15.16:0/1687285558 shutdown_connections 2023-09-30T22:36:16.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.481+0000 7f20c4afc700 1 -- 172.21.15.16:0/1687285558 wait complete. 2023-09-30T22:36:16.496 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:36:16.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.548+0000 7fb4ee28a700 1 -- 172.21.15.16:0/3654570375 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fb4e806b700 con 0x7fb4e8099ff0 2023-09-30T22:36:16.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.548+0000 7fb4e57fa700 1 -- 172.21.15.16:0/3654570375 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb4e0011390 con 0x7fb4e8099ff0 2023-09-30T22:36:16.549 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:36:16.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.551+0000 7fb4ee28a700 1 -- 172.21.15.16:0/3654570375 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb4e8099ff0 msgr2=0x7fb4e808d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:16.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.551+0000 7fb4ee28a700 1 --2- 172.21.15.16:0/3654570375 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb4e8099ff0 0x7fb4e808d2a0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0015000 tx=0x7fb4e0004ad0 comp rx=0 tx=0).stop 2023-09-30T22:36:16.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.551+0000 7fb4ee28a700 1 -- 172.21.15.16:0/3654570375 shutdown_connections 2023-09-30T22:36:16.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.551+0000 7fb4ee28a700 1 --2- 172.21.15.16:0/3654570375 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb4e8099ff0 0x7fb4e808d2a0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:16.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.551+0000 7fb4ee28a700 1 --2- 172.21.15.16:0/3654570375 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4e80954c0 0x7fb4e808f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:16.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.551+0000 7fb4ee28a700 1 --2- 172.21.15.16:0/3654570375 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb4e8099680 0x7fb4e808cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:16.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.551+0000 7fb4ee28a700 1 -- 172.21.15.16:0/3654570375 >> 172.21.15.16:0/3654570375 conn(0x7fb4e80a2360 msgr2=0x7fb4e809d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:16.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.551+0000 7fb4ee28a700 1 -- 172.21.15.16:0/3654570375 shutdown_connections 2023-09-30T22:36:16.555 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.551+0000 7fb4ee28a700 1 -- 172.21.15.16:0/3654570375 wait complete. 2023-09-30T22:36:16.566 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:36:16.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.587+0000 7fc7243bb700 1 -- 172.21.15.16:0/4159355709 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc71c0747f0 con 0x7fc71c138d80 2023-09-30T22:36:16.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.587+0000 7fc712ffd700 1 -- 172.21.15.16:0/4159355709 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc7140303a0 con 0x7fc71c138d80 2023-09-30T22:36:16.588 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:36:16.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.589+0000 7fc7243bb700 1 -- 172.21.15.16:0/4159355709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc71c138d80 msgr2=0x7fc71c1341f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:16.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.589+0000 7fc7243bb700 1 --2- 172.21.15.16:0/4159355709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc71c138d80 0x7fc71c1341f0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fc714014040 tx=0x7fc714004ad0 comp rx=0 tx=0).stop 2023-09-30T22:36:16.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.589+0000 7fc7243bb700 1 -- 172.21.15.16:0/4159355709 shutdown_connections 2023-09-30T22:36:16.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.589+0000 7fc7243bb700 1 --2- 172.21.15.16:0/4159355709 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc71c09d630 0x7fc71c133770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:16.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.589+0000 7fc7243bb700 1 --2- 172.21.15.16:0/4159355709 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc71c138d80 0x7fc71c1341f0 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:16.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.589+0000 7fc7243bb700 1 --2- 172.21.15.16:0/4159355709 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc71c0a0a30 0x7fc71c133cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:16.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.589+0000 7fc7243bb700 1 -- 172.21.15.16:0/4159355709 >> 172.21.15.16:0/4159355709 conn(0x7fc71c0a3050 msgr2=0x7fc71c0a4180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:16.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.589+0000 7fc7243bb700 1 -- 172.21.15.16:0/4159355709 shutdown_connections 2023-09-30T22:36:16.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:16.589+0000 7fc7243bb700 1 -- 172.21.15.16:0/4159355709 wait complete. 2023-09-30T22:36:16.603 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:36:17.454 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:36:17.497 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:36:17.568 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:36:17.604 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:36:17.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.613+0000 7f690b6e7700 1 Processor -- start 2023-09-30T22:36:17.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.613+0000 7f690b6e7700 1 -- start start 2023-09-30T22:36:17.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.614+0000 7f690b6e7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f690409d6f0 0x7f690409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:17.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.614+0000 7f690b6e7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f690409e000 0x7f69041339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:17.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.614+0000 7f690b6e7700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f690409e410 0x7f6904133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:17.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.614+0000 7f690b6e7700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f690406b460 con 0x7f690409e410 2023-09-30T22:36:17.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.614+0000 7f690b6e7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f690409b120 con 0x7f690409e000 2023-09-30T22:36:17.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.614+0000 7f690b6e7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f690409a7e0 con 0x7f690409d6f0 2023-09-30T22:36:17.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.614+0000 7f6909c84700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f690409e410 0x7f6904133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:17.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.614+0000 7f6909483700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f690409d6f0 0x7f690409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:17.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.614+0000 7f6909c84700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f690409e410 0x7f6904133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41830/0 (socket says 172.21.15.16:41830) 2023-09-30T22:36:17.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.614+0000 7f6909c84700 1 -- 172.21.15.16:0/2766022217 learned_addr learned my addr 172.21.15.16:0/2766022217 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:17.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.614+0000 7f6909483700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f690409d6f0 0x7f690409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:56974/0 (socket says 172.21.15.16:56974) 2023-09-30T22:36:17.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.614+0000 7f6909c84700 1 -- 172.21.15.16:0/2766022217 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f690409d6f0 msgr2=0x7f690409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:17.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.614+0000 7f6909c84700 1 --2- 172.21.15.16:0/2766022217 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f690409d6f0 0x7f690409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:17.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.614+0000 7f6909c84700 1 -- 172.21.15.16:0/2766022217 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f690409e000 msgr2=0x7f69041339f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T22:36:17.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.614+0000 7f6909c84700 1 --2- 172.21.15.16:0/2766022217 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f690409e000 0x7f69041339f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:17.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.614+0000 7f6909c84700 1 -- 172.21.15.16:0/2766022217 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f690409a500 con 0x7f690409e410 2023-09-30T22:36:17.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.614+0000 7f6909483700 1 --2- 172.21.15.16:0/2766022217 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f690409d6f0 0x7f690409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:17.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.615+0000 7f6909c84700 1 --2- 172.21.15.16:0/2766022217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f690409e410 0x7f6904133f30 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f68fc009e50 tx=0x7f68fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=713b5f079ccc0b0f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:17.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.615+0000 7f68fbfff700 1 -- 172.21.15.16:0/2766022217 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68fc00e070 con 0x7f690409e410 2023-09-30T22:36:17.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.615+0000 7f68fbfff700 1 -- 172.21.15.16:0/2766022217 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68fc00b910 con 0x7f690409e410 2023-09-30T22:36:17.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.615+0000 7f68fbfff700 1 -- 172.21.15.16:0/2766022217 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68fc00bcc0 con 0x7f690409e410 2023-09-30T22:36:17.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.616+0000 7f690b6e7700 1 -- 172.21.15.16:0/2766022217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f690409e410 msgr2=0x7f6904133f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:17.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.616+0000 7f690b6e7700 1 --2- 172.21.15.16:0/2766022217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f690409e410 0x7f6904133f30 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f68fc009e50 tx=0x7f68fc00b330 comp rx=0 tx=0).stop 2023-09-30T22:36:17.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.616+0000 7f690b6e7700 1 -- 172.21.15.16:0/2766022217 shutdown_connections 2023-09-30T22:36:17.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.616+0000 7f690b6e7700 1 --2- 172.21.15.16:0/2766022217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f690409e410 0x7f6904133f30 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:17.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.616+0000 7f690b6e7700 1 --2- 172.21.15.16:0/2766022217 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f690409d6f0 0x7f690409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:17.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.616+0000 7f690b6e7700 1 --2- 172.21.15.16:0/2766022217 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f690409e000 0x7f69041339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:17.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.616+0000 7f690b6e7700 1 -- 172.21.15.16:0/2766022217 >> 172.21.15.16:0/2766022217 conn(0x7f6904066f50 msgr2=0x7f690409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:17.619 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.616+0000 7f690b6e7700 1 -- 172.21.15.16:0/2766022217 shutdown_connections 2023-09-30T22:36:17.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.616+0000 7f690b6e7700 1 -- 172.21.15.16:0/2766022217 wait complete. 2023-09-30T22:36:17.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.617+0000 7f690b6e7700 1 Processor -- start 2023-09-30T22:36:17.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.617+0000 7f690b6e7700 1 -- start start 2023-09-30T22:36:17.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.618+0000 7f690b6e7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f690409d6f0 0x7f69041a5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:17.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.618+0000 7f690b6e7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f690409e000 0x7f69041a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:17.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.618+0000 7f690b6e7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f690409e410 0x7f69041a6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:17.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.618+0000 7f690b6e7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69041a98e0 con 0x7f690409e410 2023-09-30T22:36:17.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.618+0000 7f690b6e7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69041a7ff0 con 0x7f690409e000 2023-09-30T22:36:17.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.618+0000 7f690b6e7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f69041aa280 con 0x7f690409d6f0 2023-09-30T22:36:17.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.618+0000 7f6908c82700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f690409e000 0x7f69041a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:17.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.618+0000 7f6908c82700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f690409e000 0x7f69041a6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38088/0 (socket says 172.21.15.16:38088) 2023-09-30T22:36:17.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.618+0000 7f6908c82700 1 -- 172.21.15.16:0/3022886561 learned_addr learned my addr 172.21.15.16:0/3022886561 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:17.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.618+0000 7f6909c84700 1 --2- 172.21.15.16:0/3022886561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f690409e410 0x7f69041a6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:17.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.618+0000 7f6909483700 1 --2- 172.21.15.16:0/3022886561 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f690409d6f0 0x7f69041a5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:17.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.619+0000 7f6908c82700 1 -- 172.21.15.16:0/3022886561 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f690409d6f0 msgr2=0x7f69041a5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:17.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.619+0000 7f6908c82700 1 --2- 172.21.15.16:0/3022886561 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f690409d6f0 0x7f69041a5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:17.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.619+0000 7f6908c82700 1 -- 172.21.15.16:0/3022886561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f690409e410 msgr2=0x7f69041a6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:17.622 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.619+0000 7f6908c82700 1 --2- 172.21.15.16:0/3022886561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f690409e410 0x7f69041a6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:17.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.619+0000 7f6908c82700 1 -- 172.21.15.16:0/3022886561 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69000094a0 con 0x7f690409e000 2023-09-30T22:36:17.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.619+0000 7f6908c82700 1 --2- 172.21.15.16:0/3022886561 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f690409e000 0x7f69041a6300 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f68f400b580 tx=0x7f68f400b890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:17.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.619+0000 7f68f9ffb700 1 -- 172.21.15.16:0/3022886561 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f4011830 con 0x7f690409e000 2023-09-30T22:36:17.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.619+0000 7f68f9ffb700 1 -- 172.21.15.16:0/3022886561 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68f400be90 con 0x7f690409e000 2023-09-30T22:36:17.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.619+0000 7f690b6e7700 1 -- 172.21.15.16:0/3022886561 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68fc009b30 con 0x7f690409e000 2023-09-30T22:36:17.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.619+0000 7f68f9ffb700 1 -- 172.21.15.16:0/3022886561 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f4011bd0 con 0x7f690409e000 2023-09-30T22:36:17.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.620+0000 7f690b6e7700 1 -- 172.21.15.16:0/3022886561 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69040a2fd0 con 0x7f690409e000 2023-09-30T22:36:17.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.621+0000 7f68f9ffb700 1 -- 172.21.15.16:0/3022886561 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f68f4011d30 con 0x7f690409e000 2023-09-30T22:36:17.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.621+0000 7f68f9ffb700 1 -- 172.21.15.16:0/3022886561 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f68f4017c40 con 0x7f690409e000 2023-09-30T22:36:17.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.621+0000 7f690b6e7700 1 -- 172.21.15.16:0/3022886561 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6904074ea0 con 0x7f690409e000 2023-09-30T22:36:17.627 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.627+0000 7f68f9ffb700 1 -- 172.21.15.16:0/3022886561 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f68f4044060 con 0x7f690409e000 2023-09-30T22:36:17.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.655+0000 7f68d948e700 1 Processor -- start 2023-09-30T22:36:17.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.655+0000 7f68d948e700 1 -- start start 2023-09-30T22:36:17.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.655+0000 7f68d948e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f68d41338a0 0x7f68d4133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:17.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.655+0000 7f68d948e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f68d409e930 0x7f68d413b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:17.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.655+0000 7f68d948e700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68d406ef30 con 0x7f68d409de50 2023-09-30T22:36:17.657 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.655+0000 7f68d948e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68d406fef0 con 0x7f68d41338a0 2023-09-30T22:36:17.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.656+0000 7f68d948e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f68d41334b0 con 0x7f68d409e930 2023-09-30T22:36:17.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.656+0000 7f68d2ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f68d41338a0 0x7f68d4133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:17.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.656+0000 7f68d2ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f68d41338a0 0x7f68d4133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38098/0 (socket says 172.21.15.16:38098) 2023-09-30T22:36:17.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.656+0000 7f68d2ffd700 1 -- 172.21.15.16:0/1213629665 learned_addr learned my addr 172.21.15.16:0/1213629665 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:17.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.656+0000 7f68d2ffd700 1 -- 172.21.15.16:0/1213629665 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f68d409e930 msgr2=0x7f68d413b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:17.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.656+0000 7f68d2ffd700 1 --2- 172.21.15.16:0/1213629665 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f68d409e930 0x7f68d413b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:17.658 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.656+0000 7f68d2ffd700 1 -- 172.21.15.16:0/1213629665 >> v1:172.21.15.16:6789/0 conn(0x7f68d409de50 legacy=0x7f68d409e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:17.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.656+0000 7f68d2ffd700 1 -- 172.21.15.16:0/1213629665 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d406fc30 con 0x7f68d41338a0 2023-09-30T22:36:17.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.656+0000 7f68d2ffd700 1 --2- 172.21.15.16:0/1213629665 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f68d41338a0 0x7f68d4133c70 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f68c4009920 tx=0x7f68c4009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8eb4c6e8420c94fc server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:17.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.657+0000 7f68d1ffb700 1 -- 172.21.15.16:0/1213629665 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c400ba60 con 0x7f68d41338a0 2023-09-30T22:36:17.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.657+0000 7f68d1ffb700 1 -- 172.21.15.16:0/1213629665 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68c400bbc0 con 0x7f68d41338a0 2023-09-30T22:36:17.659 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.657+0000 7f68d1ffb700 1 -- 172.21.15.16:0/1213629665 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c4009e70 con 0x7f68d41338a0 2023-09-30T22:36:17.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.657+0000 7f68d948e700 1 -- 172.21.15.16:0/1213629665 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f68d41338a0 msgr2=0x7f68d4133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:17.661 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.657+0000 7f68d948e700 1 --2- 172.21.15.16:0/1213629665 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f68d41338a0 0x7f68d4133c70 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f68c4009920 tx=0x7f68c4009c30 comp rx=0 tx=0).stop 2023-09-30T22:36:17.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.658+0000 7f68d948e700 1 -- 172.21.15.16:0/1213629665 shutdown_connections 2023-09-30T22:36:17.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.658+0000 7f68d948e700 1 --2- 172.21.15.16:0/1213629665 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f68d409e930 0x7f68d413b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:17.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.658+0000 7f68d948e700 1 --2- 172.21.15.16:0/1213629665 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f68d41338a0 0x7f68d4133c70 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:17.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.658+0000 7f68d948e700 1 -- 172.21.15.16:0/1213629665 >> 172.21.15.16:0/1213629665 conn(0x7f68d40a2aa0 msgr2=0x7f68d40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:17.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.658+0000 7f68d948e700 1 -- 172.21.15.16:0/1213629665 shutdown_connections 2023-09-30T22:36:17.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.658+0000 7f68d948e700 1 -- 172.21.15.16:0/1213629665 wait complete. 2023-09-30T22:36:17.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.659+0000 7f68d948e700 1 Processor -- start 2023-09-30T22:36:17.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.659+0000 7f68d948e700 1 -- start start 2023-09-30T22:36:17.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.659+0000 7f68d948e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f68d409de50 0x7f68d411eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:17.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.660+0000 7f68d948e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68d409e930 0x7f68d411f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:17.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.660+0000 7f68d948e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f68d41338a0 0x7f68d411f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:17.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.660+0000 7f68d948e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68d411fff0 con 0x7f68d409e930 2023-09-30T22:36:17.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.660+0000 7f68d948e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68d41220a0 con 0x7f68d41338a0 2023-09-30T22:36:17.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.660+0000 7f68d948e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f68d412fa70 con 0x7f68d409de50 2023-09-30T22:36:17.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.660+0000 7f68d27fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68d409e930 0x7f68d411f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:17.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.660+0000 7f68d27fc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68d409e930 0x7f68d411f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41848/0 (socket says 172.21.15.16:41848) 2023-09-30T22:36:17.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.660+0000 7f68d27fc700 1 -- 172.21.15.16:0/1150694818 learned_addr learned my addr 172.21.15.16:0/1150694818 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:17.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.660+0000 7f68d2ffd700 1 --2- 172.21.15.16:0/1150694818 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f68d409de50 0x7f68d411eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:17.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.660+0000 7f68d37fe700 1 --2- 172.21.15.16:0/1150694818 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f68d41338a0 0x7f68d411f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:17.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.660+0000 7f68d27fc700 1 -- 172.21.15.16:0/1150694818 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f68d409de50 msgr2=0x7f68d411eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:17.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.660+0000 7f68d27fc700 1 --2- 172.21.15.16:0/1150694818 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f68d409de50 0x7f68d411eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:17.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.660+0000 7f68d27fc700 1 -- 172.21.15.16:0/1150694818 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f68d41338a0 msgr2=0x7f68d411f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:17.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.660+0000 7f68d27fc700 1 --2- 172.21.15.16:0/1150694818 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f68d41338a0 0x7f68d411f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:17.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.660+0000 7f68d27fc700 1 -- 172.21.15.16:0/1150694818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c4009550 con 0x7f68d409e930 2023-09-30T22:36:17.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.660+0000 7f68d27fc700 1 --2- 172.21.15.16:0/1150694818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68d409e930 0x7f68d411f060 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f68c800c420 tx=0x7f68c800c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:17.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.661+0000 7f68bb7fe700 1 -- 172.21.15.16:0/1150694818 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c80052d0 con 0x7f68d409e930 2023-09-30T22:36:17.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.661+0000 7f68d948e700 1 -- 172.21.15.16:0/1150694818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68d406f640 con 0x7f68d409e930 2023-09-30T22:36:17.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.661+0000 7f68d948e700 1 -- 172.21.15.16:0/1150694818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68d4122dd0 con 0x7f68d409e930 2023-09-30T22:36:17.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.661+0000 7f68bb7fe700 1 -- 172.21.15.16:0/1150694818 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68c8005430 con 0x7f68d409e930 2023-09-30T22:36:17.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.661+0000 7f68bb7fe700 1 -- 172.21.15.16:0/1150694818 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c80038c0 con 0x7f68d409e930 2023-09-30T22:36:17.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.662+0000 7f68bb7fe700 1 -- 172.21.15.16:0/1150694818 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f68c8003a20 con 0x7f68d409e930 2023-09-30T22:36:17.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.662+0000 7f68d948e700 1 -- 172.21.15.16:0/1150694818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68b40052e0 con 0x7f68d409e930 2023-09-30T22:36:17.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.662+0000 7f68bb7fe700 1 -- 172.21.15.16:0/1150694818 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f68c803eaf0 con 0x7f68d409e930 2023-09-30T22:36:17.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.671+0000 7f68bb7fe700 1 -- 172.21.15.16:0/1150694818 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f68c8017070 con 0x7f68d409e930 2023-09-30T22:36:17.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.724+0000 7fa294198700 1 Processor -- start 2023-09-30T22:36:17.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.725+0000 7fa294198700 1 -- start start 2023-09-30T22:36:17.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.725+0000 7fa294198700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa28c133420 0x7fa28c1337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:17.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.725+0000 7fa294198700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa28c133e30 0x7fa28c12ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:17.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.725+0000 7fa294198700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa28c06b990 con 0x7fa28c134260 2023-09-30T22:36:17.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.725+0000 7fa294198700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c0a0590 con 0x7fa28c133e30 2023-09-30T22:36:17.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.725+0000 7fa294198700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c09fb80 con 0x7fa28c133420 2023-09-30T22:36:17.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.726+0000 7fa291f34700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa28c133420 0x7fa28c1337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:17.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.726+0000 7fa291f34700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa28c133420 0x7fa28c1337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:57002/0 (socket says 172.21.15.16:57002) 2023-09-30T22:36:17.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.726+0000 7fa291f34700 1 -- 172.21.15.16:0/677844284 learned_addr learned my addr 172.21.15.16:0/677844284 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:17.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.726+0000 7fa291733700 1 --2- 172.21.15.16:0/677844284 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa28c133e30 0x7fa28c12ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:17.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.726+0000 7fa291f34700 1 -- 172.21.15.16:0/677844284 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa28c133e30 msgr2=0x7fa28c12ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:17.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.726+0000 7fa291f34700 1 --2- 172.21.15.16:0/677844284 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa28c133e30 0x7fa28c12ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:17.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.726+0000 7fa291f34700 1 -- 172.21.15.16:0/677844284 >> v1:172.21.15.16:6789/0 conn(0x7fa28c134260 legacy=0x7fa28c1304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:17.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.726+0000 7fa291f34700 1 -- 172.21.15.16:0/677844284 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa28c0a0030 con 0x7fa28c133420 2023-09-30T22:36:17.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.726+0000 7fa291f34700 1 --2- 172.21.15.16:0/677844284 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa28c133420 0x7fa28c1337f0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fa288009920 tx=0x7fa288009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=189cee5feb5862a6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:17.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.727+0000 7fa290f32700 1 -- 172.21.15.16:0/677844284 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa288011830 con 0x7fa28c133420 2023-09-30T22:36:17.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.727+0000 7fa290f32700 1 -- 172.21.15.16:0/677844284 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa288011990 con 0x7fa28c133420 2023-09-30T22:36:17.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.727+0000 7fa290f32700 1 -- 172.21.15.16:0/677844284 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa288011d20 con 0x7fa28c133420 2023-09-30T22:36:17.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.727+0000 7fa294198700 1 -- 172.21.15.16:0/677844284 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa28c133420 msgr2=0x7fa28c1337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:17.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.727+0000 7fa294198700 1 --2- 172.21.15.16:0/677844284 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa28c133420 0x7fa28c1337f0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fa288009920 tx=0x7fa288009c30 comp rx=0 tx=0).stop 2023-09-30T22:36:17.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.727+0000 7fa294198700 1 -- 172.21.15.16:0/677844284 shutdown_connections 2023-09-30T22:36:17.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.727+0000 7fa294198700 1 --2- 172.21.15.16:0/677844284 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa28c133420 0x7fa28c1337f0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:17.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.727+0000 7fa294198700 1 --2- 172.21.15.16:0/677844284 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa28c133e30 0x7fa28c12ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:17.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.727+0000 7fa294198700 1 -- 172.21.15.16:0/677844284 >> 172.21.15.16:0/677844284 conn(0x7fa28c0a4210 msgr2=0x7fa28c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:17.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.727+0000 7fa294198700 1 -- 172.21.15.16:0/677844284 shutdown_connections 2023-09-30T22:36:17.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.727+0000 7fa294198700 1 -- 172.21.15.16:0/677844284 wait complete. 2023-09-30T22:36:17.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.729+0000 7fa294198700 1 Processor -- start 2023-09-30T22:36:17.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.729+0000 7fa294198700 1 -- start start 2023-09-30T22:36:17.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.730+0000 7fa294198700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa28c133420 0x7fa28c09c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:17.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.730+0000 7fa294198700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa28c133e30 0x7fa28c09cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:17.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.730+0000 7fa294198700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa28c134260 0x7fa28c09d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:17.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.730+0000 7fa294198700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c120f90 con 0x7fa28c134260 2023-09-30T22:36:17.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.730+0000 7fa294198700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c09db20 con 0x7fa28c133e30 2023-09-30T22:36:17.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.730+0000 7fa294198700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c120140 con 0x7fa28c133420 2023-09-30T22:36:17.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.730+0000 7fa292735700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa28c134260 0x7fa28c09d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:17.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.730+0000 7fa292735700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa28c134260 0x7fa28c09d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41860/0 (socket says 172.21.15.16:41860) 2023-09-30T22:36:17.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.730+0000 7fa292735700 1 -- 172.21.15.16:0/115595590 learned_addr learned my addr 172.21.15.16:0/115595590 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:17.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.730+0000 7fa291733700 1 --2- 172.21.15.16:0/115595590 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa28c133e30 0x7fa28c09cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:17.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.730+0000 7fa291f34700 1 --2- 172.21.15.16:0/115595590 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa28c133420 0x7fa28c09c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:17.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.730+0000 7fa291733700 1 -- 172.21.15.16:0/115595590 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa28c133420 msgr2=0x7fa28c09c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:17.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.730+0000 7fa291733700 1 --2- 172.21.15.16:0/115595590 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa28c133420 0x7fa28c09c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:17.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.730+0000 7fa291733700 1 -- 172.21.15.16:0/115595590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa28c134260 msgr2=0x7fa28c09d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:17.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.730+0000 7fa291733700 1 --2- 172.21.15.16:0/115595590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa28c134260 0x7fa28c09d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:17.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.730+0000 7fa291733700 1 -- 172.21.15.16:0/115595590 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa288009550 con 0x7fa28c133e30 2023-09-30T22:36:17.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.731+0000 7fa291733700 1 --2- 172.21.15.16:0/115595590 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa28c133e30 0x7fa28c09cc20 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fa27c00e8d0 tx=0x7fa27c00ebe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:17.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.731+0000 7fa2827fc700 1 -- 172.21.15.16:0/115595590 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27c00c690 con 0x7fa28c133e30 2023-09-30T22:36:17.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.731+0000 7fa2827fc700 1 -- 172.21.15.16:0/115595590 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa27c00c7f0 con 0x7fa28c133e30 2023-09-30T22:36:17.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.731+0000 7fa294198700 1 -- 172.21.15.16:0/115595590 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa28c131720 con 0x7fa28c133e30 2023-09-30T22:36:17.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.731+0000 7fa2827fc700 1 -- 172.21.15.16:0/115595590 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27c00cb60 con 0x7fa28c133e30 2023-09-30T22:36:17.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.731+0000 7fa294198700 1 -- 172.21.15.16:0/115595590 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa28c1203b0 con 0x7fa28c133e30 2023-09-30T22:36:17.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.732+0000 7fa2827fc700 1 -- 172.21.15.16:0/115595590 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa27c00ccc0 con 0x7fa28c133e30 2023-09-30T22:36:17.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.732+0000 7fa2827fc700 1 -- 172.21.15.16:0/115595590 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fa27c03f810 con 0x7fa28c133e30 2023-09-30T22:36:17.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.733+0000 7fa294198700 1 -- 172.21.15.16:0/115595590 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa28c133060 con 0x7fa28c133e30 2023-09-30T22:36:17.738 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.738+0000 7fa2827fc700 1 -- 172.21.15.16:0/115595590 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa27c021020 con 0x7fa28c133e30 2023-09-30T22:36:17.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.761+0000 7f364848c700 1 Processor -- start 2023-09-30T22:36:17.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.761+0000 7f364848c700 1 -- start start 2023-09-30T22:36:17.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.761+0000 7f364848c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f36400a0a30 0x7f36400a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:17.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.762+0000 7f364848c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3640138d80 0x7f364013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:17.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.762+0000 7f364848c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36400a2a70 con 0x7f364009d630 2023-09-30T22:36:17.762 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.762+0000 7f364848c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36400a01d0 con 0x7f36400a0a30 2023-09-30T22:36:17.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.762+0000 7f364848c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f364009e2b0 con 0x7f3640138d80 2023-09-30T22:36:17.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.762+0000 7f3645a27700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f364009d630 0x7f36400a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60148/0 (socket says 172.21.15.16:60148) 2023-09-30T22:36:17.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.762+0000 7f3645a27700 1 -- 172.21.15.16:0/832863576 learned_addr learned my addr 172.21.15.16:0/832863576 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:36:17.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.762+0000 7f3646228700 1 --2- 172.21.15.16:0/832863576 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f36400a0a30 0x7f36400a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:17.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.762+0000 7f3646a29700 1 --2- 172.21.15.16:0/832863576 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3640138d80 0x7f364013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:17.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.762+0000 7f3645226700 1 -- 172.21.15.16:0/832863576 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 480594863 0 0) 0x7f36400a2a70 con 0x7f364009d630 2023-09-30T22:36:17.763 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.762+0000 7f3645226700 1 -- 172.21.15.16:0/832863576 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f362c0035b0 con 0x7f364009d630 2023-09-30T22:36:17.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.762+0000 7f3646228700 1 -- 172.21.15.16:0/832863576 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3640138d80 msgr2=0x7f364013b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:17.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.762+0000 7f3646228700 1 --2- 172.21.15.16:0/832863576 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3640138d80 0x7f364013b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:17.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.762+0000 7f3646228700 1 -- 172.21.15.16:0/832863576 >> v1:172.21.15.16:6789/0 conn(0x7f364009d630 legacy=0x7f36400a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:17.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.762+0000 7f3646228700 1 -- 172.21.15.16:0/832863576 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f364009dc50 con 0x7f36400a0a30 2023-09-30T22:36:17.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.763+0000 7f3646228700 1 --2- 172.21.15.16:0/832863576 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f36400a0a30 0x7f36400a0e00 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f363c009870 tx=0x7f363c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=384f3038cc94ec6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:17.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.763+0000 7f3645226700 1 -- 172.21.15.16:0/832863576 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363c011830 con 0x7f36400a0a30 2023-09-30T22:36:17.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.763+0000 7f3645226700 1 -- 172.21.15.16:0/832863576 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f363c011990 con 0x7f36400a0a30 2023-09-30T22:36:17.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.763+0000 7f3645226700 1 -- 172.21.15.16:0/832863576 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363c011d20 con 0x7f36400a0a30 2023-09-30T22:36:17.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.763+0000 7f364848c700 1 -- 172.21.15.16:0/832863576 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f36400a0a30 msgr2=0x7f36400a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:17.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.763+0000 7f364848c700 1 --2- 172.21.15.16:0/832863576 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f36400a0a30 0x7f36400a0e00 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f363c009870 tx=0x7f363c009b80 comp rx=0 tx=0).stop 2023-09-30T22:36:17.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.764+0000 7f364848c700 1 -- 172.21.15.16:0/832863576 shutdown_connections 2023-09-30T22:36:17.765 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.764+0000 7f364848c700 1 --2- 172.21.15.16:0/832863576 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3640138d80 0x7f364013b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:17.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.764+0000 7f364848c700 1 --2- 172.21.15.16:0/832863576 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f36400a0a30 0x7f36400a0e00 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:17.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.764+0000 7f364848c700 1 -- 172.21.15.16:0/832863576 >> 172.21.15.16:0/832863576 conn(0x7f36400a3050 msgr2=0x7f36400a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:17.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.764+0000 7f364848c700 1 -- 172.21.15.16:0/832863576 shutdown_connections 2023-09-30T22:36:17.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.764+0000 7f364848c700 1 -- 172.21.15.16:0/832863576 wait complete. 2023-09-30T22:36:17.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.765+0000 7f364848c700 1 Processor -- start 2023-09-30T22:36:17.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.765+0000 7f364848c700 1 -- start start 2023-09-30T22:36:17.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.765+0000 7f364848c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f364009d630 0x7f3640133760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:17.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.765+0000 7f364848c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f36400a0a30 0x7f3640133ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:17.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.765+0000 7f364848c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3640138d80 0x7f36401341e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:17.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.765+0000 7f364848c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36401365a0 con 0x7f364009d630 2023-09-30T22:36:17.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.765+0000 7f364848c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3640138b10 con 0x7f36400a0a30 2023-09-30T22:36:17.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.765+0000 7f364848c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3640136f10 con 0x7f3640138d80 2023-09-30T22:36:17.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.765+0000 7f3646228700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f364009d630 0x7f3640133760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:17.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.765+0000 7f3645a27700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f36400a0a30 0x7f3640133ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:17.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.765+0000 7f3646228700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f364009d630 0x7f3640133760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41874/0 (socket says 172.21.15.16:41874) 2023-09-30T22:36:17.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.765+0000 7f3646228700 1 -- 172.21.15.16:0/427494995 learned_addr learned my addr 172.21.15.16:0/427494995 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:17.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.766+0000 7f3646a29700 1 --2- 172.21.15.16:0/427494995 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3640138d80 0x7f36401341e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:17.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.766+0000 7f3646228700 1 -- 172.21.15.16:0/427494995 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3640138d80 msgr2=0x7f36401341e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:17.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.766+0000 7f3646228700 1 --2- 172.21.15.16:0/427494995 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3640138d80 0x7f36401341e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:17.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.766+0000 7f3646228700 1 -- 172.21.15.16:0/427494995 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f36400a0a30 msgr2=0x7f3640133ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:17.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.766+0000 7f3646228700 1 --2- 172.21.15.16:0/427494995 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f36400a0a30 0x7f3640133ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:17.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.766+0000 7f3646228700 1 -- 172.21.15.16:0/427494995 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f363c0094a0 con 0x7f364009d630 2023-09-30T22:36:17.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.766+0000 7f3646228700 1 --2- 172.21.15.16:0/427494995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f364009d630 0x7f3640133760 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f36400a2c00 tx=0x7f363c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:17.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.766+0000 7f3636ffd700 1 -- 172.21.15.16:0/427494995 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363c004240 con 0x7f364009d630 2023-09-30T22:36:17.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.766+0000 7f3636ffd700 1 -- 172.21.15.16:0/427494995 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f363c01f070 con 0x7f364009d630 2023-09-30T22:36:17.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.766+0000 7f364848c700 1 -- 172.21.15.16:0/427494995 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36400a4b20 con 0x7f364009d630 2023-09-30T22:36:17.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.766+0000 7f3636ffd700 1 -- 172.21.15.16:0/427494995 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363c0045e0 con 0x7f364009d630 2023-09-30T22:36:17.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.766+0000 7f364848c700 1 -- 172.21.15.16:0/427494995 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36401376f0 con 0x7f364009d630 2023-09-30T22:36:17.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.767+0000 7f3636ffd700 1 -- 172.21.15.16:0/427494995 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f363c021030 con 0x7f364009d630 2023-09-30T22:36:17.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.767+0000 7f3636ffd700 1 -- 172.21.15.16:0/427494995 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f363c040700 con 0x7f364009d630 2023-09-30T22:36:17.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.767+0000 7f364848c700 1 -- 172.21.15.16:0/427494995 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3640137050 con 0x7f364009d630 2023-09-30T22:36:17.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.772+0000 7f3636ffd700 1 -- 172.21.15.16:0/427494995 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3640137050 con 0x7f364009d630 2023-09-30T22:36:17.843 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.843+0000 7f690b6e7700 1 -- 172.21.15.16:0/3022886561 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f69041a8c20 con 0x7f690409e000 2023-09-30T22:36:17.844 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.843+0000 7f68f9ffb700 1 -- 172.21.15.16:0/3022886561 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f68f4017a60 con 0x7f690409e000 2023-09-30T22:36:17.844 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:36:17.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.846+0000 7f690b6e7700 1 -- 172.21.15.16:0/3022886561 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f690409e000 msgr2=0x7f69041a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:17.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.846+0000 7f690b6e7700 1 --2- 172.21.15.16:0/3022886561 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f690409e000 0x7f69041a6300 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f68f400b580 tx=0x7f68f400b890 comp rx=0 tx=0).stop 2023-09-30T22:36:17.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.846+0000 7f690b6e7700 1 -- 172.21.15.16:0/3022886561 shutdown_connections 2023-09-30T22:36:17.846 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.846+0000 7f690b6e7700 1 --2- 172.21.15.16:0/3022886561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f690409e410 0x7f69041a6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:17.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.846+0000 7f690b6e7700 1 --2- 172.21.15.16:0/3022886561 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f690409d6f0 0x7f69041a5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:17.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.846+0000 7f690b6e7700 1 --2- 172.21.15.16:0/3022886561 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f690409e000 0x7f69041a6300 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:17.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.846+0000 7f690b6e7700 1 -- 172.21.15.16:0/3022886561 >> 172.21.15.16:0/3022886561 conn(0x7f6904066f50 msgr2=0x7f69041bad60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:17.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.846+0000 7f690b6e7700 1 -- 172.21.15.16:0/3022886561 shutdown_connections 2023-09-30T22:36:17.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.846+0000 7f690b6e7700 1 -- 172.21.15.16:0/3022886561 wait complete. 2023-09-30T22:36:17.860 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:36:17.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.927+0000 7f68d948e700 1 -- 172.21.15.16:0/1150694818 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f68b4005590 con 0x7f68d409e930 2023-09-30T22:36:17.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.927+0000 7f68bb7fe700 1 -- 172.21.15.16:0/1150694818 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f68c8017070 con 0x7f68d409e930 2023-09-30T22:36:17.928 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:36:17.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.930+0000 7f68d948e700 1 -- 172.21.15.16:0/1150694818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68d409e930 msgr2=0x7f68d411f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:17.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.930+0000 7f68d948e700 1 --2- 172.21.15.16:0/1150694818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68d409e930 0x7f68d411f060 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f68c800c420 tx=0x7f68c800c7e0 comp rx=0 tx=0).stop 2023-09-30T22:36:17.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.930+0000 7f68d948e700 1 -- 172.21.15.16:0/1150694818 shutdown_connections 2023-09-30T22:36:17.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.930+0000 7f68d948e700 1 --2- 172.21.15.16:0/1150694818 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f68d41338a0 0x7f68d411f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:17.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.930+0000 7f68d948e700 1 --2- 172.21.15.16:0/1150694818 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f68d409de50 0x7f68d411eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:17.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.930+0000 7f68d948e700 1 --2- 172.21.15.16:0/1150694818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68d409e930 0x7f68d411f060 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:17.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.930+0000 7f68d948e700 1 -- 172.21.15.16:0/1150694818 >> 172.21.15.16:0/1150694818 conn(0x7f68d40a2aa0 msgr2=0x7f68d409ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:17.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.930+0000 7f68d948e700 1 -- 172.21.15.16:0/1150694818 shutdown_connections 2023-09-30T22:36:17.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.930+0000 7f68d948e700 1 -- 172.21.15.16:0/1150694818 wait complete. 2023-09-30T22:36:17.944 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:36:17.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.979+0000 7fa294198700 1 -- 172.21.15.16:0/115595590 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa28c074ea0 con 0x7fa28c133e30 2023-09-30T22:36:17.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.980+0000 7fa2827fc700 1 -- 172.21.15.16:0/115595590 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa27c01be10 con 0x7fa28c133e30 2023-09-30T22:36:17.981 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:36:17.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.981+0000 7f364848c700 1 -- 172.21.15.16:0/427494995 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f3640137050 con 0x7f364009d630 2023-09-30T22:36:17.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.981+0000 7f3636ffd700 1 -- 172.21.15.16:0/427494995 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3640137050 con 0x7f364009d630 2023-09-30T22:36:17.983 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:36:17.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.982+0000 7fa294198700 1 -- 172.21.15.16:0/115595590 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa28c133e30 msgr2=0x7fa28c09cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:17.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.982+0000 7fa294198700 1 --2- 172.21.15.16:0/115595590 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa28c133e30 0x7fa28c09cc20 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fa27c00e8d0 tx=0x7fa27c00ebe0 comp rx=0 tx=0).stop 2023-09-30T22:36:17.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.982+0000 7fa294198700 1 -- 172.21.15.16:0/115595590 shutdown_connections 2023-09-30T22:36:17.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.982+0000 7fa294198700 1 --2- 172.21.15.16:0/115595590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa28c134260 0x7fa28c09d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:17.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.982+0000 7fa294198700 1 --2- 172.21.15.16:0/115595590 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa28c133420 0x7fa28c09c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:17.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.982+0000 7fa294198700 1 --2- 172.21.15.16:0/115595590 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa28c133e30 0x7fa28c09cc20 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:17.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.982+0000 7fa294198700 1 -- 172.21.15.16:0/115595590 >> 172.21.15.16:0/115595590 conn(0x7fa28c0a4210 msgr2=0x7fa28c0a3780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:17.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.982+0000 7fa294198700 1 -- 172.21.15.16:0/115595590 shutdown_connections 2023-09-30T22:36:17.986 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.982+0000 7fa294198700 1 -- 172.21.15.16:0/115595590 wait complete. 2023-09-30T22:36:17.987 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.983+0000 7f364848c700 1 -- 172.21.15.16:0/427494995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f364009d630 msgr2=0x7f3640133760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:17.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.983+0000 7f364848c700 1 --2- 172.21.15.16:0/427494995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f364009d630 0x7f3640133760 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f36400a2c00 tx=0x7f363c004750 comp rx=0 tx=0).stop 2023-09-30T22:36:17.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.983+0000 7f364848c700 1 -- 172.21.15.16:0/427494995 shutdown_connections 2023-09-30T22:36:17.988 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.983+0000 7f364848c700 1 --2- 172.21.15.16:0/427494995 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3640138d80 0x7f36401341e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:17.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.983+0000 7f364848c700 1 --2- 172.21.15.16:0/427494995 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f364009d630 0x7f3640133760 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:17.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.983+0000 7f364848c700 1 --2- 172.21.15.16:0/427494995 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f36400a0a30 0x7f3640133ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:17.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.983+0000 7f364848c700 1 -- 172.21.15.16:0/427494995 >> 172.21.15.16:0/427494995 conn(0x7f36400a3050 msgr2=0x7f364013ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:17.989 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.983+0000 7f364848c700 1 -- 172.21.15.16:0/427494995 shutdown_connections 2023-09-30T22:36:17.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:17.984+0000 7f364848c700 1 -- 172.21.15.16:0/427494995 wait complete. 2023-09-30T22:36:17.997 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:36:17.998 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:36:18.861 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:36:18.945 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:36:18.998 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:36:18.998 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:36:19.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.012+0000 7ff7e27a4700 1 Processor -- start 2023-09-30T22:36:19.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.012+0000 7ff7e27a4700 1 -- start start 2023-09-30T22:36:19.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.013+0000 7ff7e27a4700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff7dc09eca0 0x7ff7dc13cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:19.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.013+0000 7ff7e27a4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff7dc06f050 0x7ff7dc13f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:19.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.013+0000 7ff7e27a4700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7dc070ee0 con 0x7ff7dc06e570 2023-09-30T22:36:19.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.013+0000 7ff7e27a4700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7dc071980 con 0x7ff7dc09eca0 2023-09-30T22:36:19.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.013+0000 7ff7e27a4700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff7dc071480 con 0x7ff7dc06f050 2023-09-30T22:36:19.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.013+0000 7ff7d3fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff7dc09eca0 0x7ff7dc13cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:19.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.013+0000 7ff7d3fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff7dc09eca0 0x7ff7dc13cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41890/0 (socket says 172.21.15.16:41890) 2023-09-30T22:36:19.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.013+0000 7ff7d3fff700 1 -- 172.21.15.16:0/2548499887 learned_addr learned my addr 172.21.15.16:0/2548499887 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:19.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.013+0000 7ff7db7fe700 1 -- 172.21.15.16:0/2548499887 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1304146520 0 0) 0x7ff7dc070ee0 con 0x7ff7dc06e570 2023-09-30T22:36:19.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.013+0000 7ff7d3fff700 1 -- 172.21.15.16:0/2548499887 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff7dc06f050 msgr2=0x7ff7dc13f3f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T22:36:19.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.013+0000 7ff7d3fff700 1 --2- 172.21.15.16:0/2548499887 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff7dc06f050 0x7ff7dc13f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:19.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.013+0000 7ff7d3fff700 1 -- 172.21.15.16:0/2548499887 >> v1:172.21.15.16:6789/0 conn(0x7ff7dc06e570 legacy=0x7ff7dc06e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:19.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.013+0000 7ff7d3fff700 1 -- 172.21.15.16:0/2548499887 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7dc071b90 con 0x7ff7dc09eca0 2023-09-30T22:36:19.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.014+0000 7ff7d3fff700 1 --2- 172.21.15.16:0/2548499887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7dc09eca0 0x7ff7dc13cea0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7ff7c40098f0 tx=0x7ff7c4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a158bc9c0c2d0bc9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:19.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.014+0000 7ff7db7fe700 1 -- 172.21.15.16:0/2548499887 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c400b8a0 con 0x7ff7dc09eca0 2023-09-30T22:36:19.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.014+0000 7ff7db7fe700 1 -- 172.21.15.16:0/2548499887 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7c400ba00 con 0x7ff7dc09eca0 2023-09-30T22:36:19.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.014+0000 7ff7db7fe700 1 -- 172.21.15.16:0/2548499887 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c400bd90 con 0x7ff7dc09eca0 2023-09-30T22:36:19.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.015+0000 7ff7e27a4700 1 -- 172.21.15.16:0/2548499887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7dc09eca0 msgr2=0x7ff7dc13cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:19.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.015+0000 7ff7e27a4700 1 --2- 172.21.15.16:0/2548499887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7dc09eca0 0x7ff7dc13cea0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7ff7c40098f0 tx=0x7ff7c4009c00 comp rx=0 tx=0).stop 2023-09-30T22:36:19.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.015+0000 7ff7e27a4700 1 -- 172.21.15.16:0/2548499887 shutdown_connections 2023-09-30T22:36:19.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.015+0000 7ff7e27a4700 1 --2- 172.21.15.16:0/2548499887 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff7dc06f050 0x7ff7dc13f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:19.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.015+0000 7ff7e27a4700 1 --2- 172.21.15.16:0/2548499887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7dc09eca0 0x7ff7dc13cea0 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:19.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.015+0000 7ff7e27a4700 1 -- 172.21.15.16:0/2548499887 >> 172.21.15.16:0/2548499887 conn(0x7ff7dc0a2aa0 msgr2=0x7ff7dc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:19.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.015+0000 7ff7e27a4700 1 -- 172.21.15.16:0/2548499887 shutdown_connections 2023-09-30T22:36:19.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.015+0000 7ff7e27a4700 1 -- 172.21.15.16:0/2548499887 wait complete. 2023-09-30T22:36:19.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.016+0000 7ff7e27a4700 1 Processor -- start 2023-09-30T22:36:19.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.016+0000 7ff7e27a4700 1 -- start start 2023-09-30T22:36:19.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.017+0000 7ff7e27a4700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7dc06e570 0x7ff7dc1214e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:19.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.017+0000 7ff7e27a4700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff7dc06f050 0x7ff7dc121a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:19.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.017+0000 7ff7e27a4700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff7dc121f60 0x7ff7dc1cc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:19.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.017+0000 7ff7e27a4700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7dc070610 con 0x7ff7dc06e570 2023-09-30T22:36:19.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.017+0000 7ff7e27a4700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7dc124870 con 0x7ff7dc121f60 2023-09-30T22:36:19.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.017+0000 7ff7e27a4700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff7dc13e750 con 0x7ff7dc06f050 2023-09-30T22:36:19.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.017+0000 7ff7d3fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff7dc06f050 0x7ff7dc121a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:19.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.017+0000 7ff7d3fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff7dc06f050 0x7ff7dc121a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:57050/0 (socket says 172.21.15.16:57050) 2023-09-30T22:36:19.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.017+0000 7ff7d3fff700 1 -- 172.21.15.16:0/1973499316 learned_addr learned my addr 172.21.15.16:0/1973499316 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:19.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.017+0000 7ff7e0d41700 1 --2- 172.21.15.16:0/1973499316 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff7dc121f60 0x7ff7dc1cc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:19.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.017+0000 7ff7dbfff700 1 --2- 172.21.15.16:0/1973499316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7dc06e570 0x7ff7dc1214e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:19.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.018+0000 7ff7d3fff700 1 -- 172.21.15.16:0/1973499316 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff7dc121f60 msgr2=0x7ff7dc1cc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:19.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.018+0000 7ff7d3fff700 1 --2- 172.21.15.16:0/1973499316 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff7dc121f60 0x7ff7dc1cc9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:19.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.018+0000 7ff7d3fff700 1 -- 172.21.15.16:0/1973499316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7dc06e570 msgr2=0x7ff7dc1214e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:19.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.018+0000 7ff7d3fff700 1 --2- 172.21.15.16:0/1973499316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7dc06e570 0x7ff7dc1214e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:19.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.018+0000 7ff7d3fff700 1 -- 172.21.15.16:0/1973499316 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7c4009550 con 0x7ff7dc06f050 2023-09-30T22:36:19.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.018+0000 7ff7dbfff700 1 --2- 172.21.15.16:0/1973499316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7dc06e570 0x7ff7dc1214e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:19.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.018+0000 7ff7e0d41700 1 --2- 172.21.15.16:0/1973499316 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff7dc121f60 0x7ff7dc1cc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:19.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.018+0000 7ff7d3fff700 1 --2- 172.21.15.16:0/1973499316 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff7dc06f050 0x7ff7dc121a20 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4009c40 tx=0x7ff7c4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:19.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.018+0000 7ff7d97fa700 1 -- 172.21.15.16:0/1973499316 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c4004380 con 0x7ff7dc06f050 2023-09-30T22:36:19.024 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.019+0000 7ff7d97fa700 1 -- 172.21.15.16:0/1973499316 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7c401f070 con 0x7ff7dc06f050 2023-09-30T22:36:19.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.019+0000 7ff7d97fa700 1 -- 172.21.15.16:0/1973499316 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c40036a0 con 0x7ff7dc06f050 2023-09-30T22:36:19.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.019+0000 7ff7e27a4700 1 -- 172.21.15.16:0/1973499316 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7dc1232d0 con 0x7ff7dc06f050 2023-09-30T22:36:19.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.019+0000 7ff7e27a4700 1 -- 172.21.15.16:0/1973499316 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7dc123ef0 con 0x7ff7dc06f050 2023-09-30T22:36:19.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.020+0000 7ff7e27a4700 1 -- 172.21.15.16:0/1973499316 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7dc123850 con 0x7ff7dc06f050 2023-09-30T22:36:19.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.020+0000 7ff7d97fa700 1 -- 172.21.15.16:0/1973499316 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff7c4021030 con 0x7ff7dc06f050 2023-09-30T22:36:19.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.021+0000 7ff7d97fa700 1 -- 172.21.15.16:0/1973499316 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7ff7c4040b60 con 0x7ff7dc06f050 2023-09-30T22:36:19.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.028+0000 7ff7d97fa700 1 -- 172.21.15.16:0/1973499316 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff7dc123850 con 0x7ff7dc06f050 2023-09-30T22:36:19.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.095+0000 7f61769c0700 1 Processor -- start 2023-09-30T22:36:19.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.095+0000 7f61769c0700 1 -- start start 2023-09-30T22:36:19.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.095+0000 7f61769c0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6170133a80 0x7f6170133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:19.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.095+0000 7f61769c0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f617012ff20 0x7f6170130330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:19.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.095+0000 7f61769c0700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6170130870 0x7f617013a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:19.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.095+0000 7f61769c0700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f617006c570 con 0x7f6170130870 2023-09-30T22:36:19.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.095+0000 7f61769c0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f617006bc00 con 0x7f6170133a80 2023-09-30T22:36:19.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.095+0000 7f61769c0700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f61700a0040 con 0x7f617012ff20 2023-09-30T22:36:19.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.096+0000 7f6174f5d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6170130870 0x7f617013a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:19.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.096+0000 7f6174f5d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6170130870 0x7f617013a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41900/0 (socket says 172.21.15.16:41900) 2023-09-30T22:36:19.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.096+0000 7f6174f5d700 1 -- 172.21.15.16:0/3682585300 learned_addr learned my addr 172.21.15.16:0/3682585300 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:19.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.096+0000 7f616f7fe700 1 --2- 172.21.15.16:0/3682585300 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f617012ff20 0x7f6170130330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:19.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.096+0000 7f616ffff700 1 --2- 172.21.15.16:0/3682585300 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6170133a80 0x7f6170133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:19.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.096+0000 7f6174f5d700 1 -- 172.21.15.16:0/3682585300 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f617012ff20 msgr2=0x7f6170130330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:19.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.096+0000 7f6174f5d700 1 --2- 172.21.15.16:0/3682585300 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f617012ff20 0x7f6170130330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:19.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.096+0000 7f6174f5d700 1 -- 172.21.15.16:0/3682585300 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6170133a80 msgr2=0x7f6170133e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:19.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.096+0000 7f6174f5d700 1 --2- 172.21.15.16:0/3682585300 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6170133a80 0x7f6170133e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:19.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.096+0000 7f6174f5d700 1 -- 172.21.15.16:0/3682585300 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6170133860 con 0x7f6170130870 2023-09-30T22:36:19.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.096+0000 7f6174f5d700 1 --2- 172.21.15.16:0/3682585300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6170130870 0x7f617013a840 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f6168009e50 tx=0x7f616800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd91bae3fec61542 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:19.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.097+0000 7f616effd700 1 -- 172.21.15.16:0/3682585300 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616800e070 con 0x7f6170130870 2023-09-30T22:36:19.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.097+0000 7f616effd700 1 -- 172.21.15.16:0/3682585300 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f616800b910 con 0x7f6170130870 2023-09-30T22:36:19.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.097+0000 7f616effd700 1 -- 172.21.15.16:0/3682585300 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f616800bcc0 con 0x7f6170130870 2023-09-30T22:36:19.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.097+0000 7f61769c0700 1 -- 172.21.15.16:0/3682585300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6170130870 msgr2=0x7f617013a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:19.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.097+0000 7f61769c0700 1 --2- 172.21.15.16:0/3682585300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6170130870 0x7f617013a840 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f6168009e50 tx=0x7f616800b330 comp rx=0 tx=0).stop 2023-09-30T22:36:19.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.098+0000 7f61769c0700 1 -- 172.21.15.16:0/3682585300 shutdown_connections 2023-09-30T22:36:19.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.098+0000 7f61769c0700 1 --2- 172.21.15.16:0/3682585300 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6170133a80 0x7f6170133e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:19.100 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.098+0000 7f61769c0700 1 --2- 172.21.15.16:0/3682585300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6170130870 0x7f617013a840 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:19.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.098+0000 7f61769c0700 1 --2- 172.21.15.16:0/3682585300 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f617012ff20 0x7f6170130330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:19.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.098+0000 7f61769c0700 1 -- 172.21.15.16:0/3682585300 >> 172.21.15.16:0/3682585300 conn(0x7f61700a4210 msgr2=0x7f61700a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:19.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.098+0000 7f61769c0700 1 -- 172.21.15.16:0/3682585300 shutdown_connections 2023-09-30T22:36:19.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.098+0000 7f61769c0700 1 -- 172.21.15.16:0/3682585300 wait complete. 2023-09-30T22:36:19.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.099+0000 7f61769c0700 1 Processor -- start 2023-09-30T22:36:19.101 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.099+0000 7f61769c0700 1 -- start start 2023-09-30T22:36:19.102 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.099+0000 7f61769c0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f617012ff20 0x7f617009c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:19.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.099+0000 7f61769c0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6170130870 0x7f617009c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:19.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.099+0000 7f61769c0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6170133a80 0x7f617009cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:19.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.099+0000 7f61769c0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f617011f080 con 0x7f6170133a80 2023-09-30T22:36:19.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.099+0000 7f61769c0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f617011f520 con 0x7f6170130870 2023-09-30T22:36:19.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.099+0000 7f61769c0700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f617012da30 con 0x7f617012ff20 2023-09-30T22:36:19.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.100+0000 7f616ffff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f617012ff20 0x7f617009c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:19.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.100+0000 7f616f7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6170130870 0x7f617009c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:19.103 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.100+0000 7f616ffff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f617012ff20 0x7f617009c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:57076/0 (socket says 172.21.15.16:57076) 2023-09-30T22:36:19.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.100+0000 7f616f7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6170130870 0x7f617009c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38158/0 (socket says 172.21.15.16:38158) 2023-09-30T22:36:19.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.100+0000 7f616ffff700 1 -- 172.21.15.16:0/3829206584 learned_addr learned my addr 172.21.15.16:0/3829206584 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:19.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.100+0000 7f6174f5d700 1 --2- 172.21.15.16:0/3829206584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6170133a80 0x7f617009cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:19.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.100+0000 7f616f7fe700 1 -- 172.21.15.16:0/3829206584 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f617012ff20 msgr2=0x7f617009c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:19.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.100+0000 7f616f7fe700 1 --2- 172.21.15.16:0/3829206584 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f617012ff20 0x7f617009c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:19.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.100+0000 7f616f7fe700 1 -- 172.21.15.16:0/3829206584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6170133a80 msgr2=0x7f617009cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:19.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.100+0000 7f616f7fe700 1 --2- 172.21.15.16:0/3829206584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6170133a80 0x7f617009cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:19.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.100+0000 7f616f7fe700 1 -- 172.21.15.16:0/3829206584 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6168009b30 con 0x7f6170130870 2023-09-30T22:36:19.104 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.100+0000 7f616ffff700 1 --2- 172.21.15.16:0/3829206584 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f617012ff20 0x7f617009c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:19.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.100+0000 7f616f7fe700 1 --2- 172.21.15.16:0/3829206584 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6170130870 0x7f617009c9c0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f616400e900 tx=0x7f616400ec10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:19.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.100+0000 7f616cff9700 1 -- 172.21.15.16:0/3829206584 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6164013680 con 0x7f6170130870 2023-09-30T22:36:19.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.101+0000 7f616cff9700 1 -- 172.21.15.16:0/3829206584 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61640137e0 con 0x7f6170130870 2023-09-30T22:36:19.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.101+0000 7f61769c0700 1 -- 172.21.15.16:0/3829206584 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61700a21e0 con 0x7f6170130870 2023-09-30T22:36:19.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.101+0000 7f616cff9700 1 -- 172.21.15.16:0/3829206584 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6164013b50 con 0x7f6170130870 2023-09-30T22:36:19.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.101+0000 7f61769c0700 1 -- 172.21.15.16:0/3829206584 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f617009ddf0 con 0x7f6170130870 2023-09-30T22:36:19.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.102+0000 7f616cff9700 1 -- 172.21.15.16:0/3829206584 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6164013cb0 con 0x7f6170130870 2023-09-30T22:36:19.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.102+0000 7f616cff9700 1 -- 172.21.15.16:0/3829206584 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f616403fa10 con 0x7f6170130870 2023-09-30T22:36:19.105 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.102+0000 7f61769c0700 1 -- 172.21.15.16:0/3829206584 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6170074ea0 con 0x7f6170130870 2023-09-30T22:36:19.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.112+0000 7f616cff9700 1 -- 172.21.15.16:0/3829206584 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f616401adc0 con 0x7f6170130870 2023-09-30T22:36:19.146 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.146+0000 7f20c1636700 1 Processor -- start 2023-09-30T22:36:19.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.146+0000 7f20c1636700 1 -- start start 2023-09-30T22:36:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.147+0000 7f20c1636700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f20bc1338a0 0x7f20bc133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.147+0000 7f20c1636700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f20bc09de50 0x7f20bc09e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.147+0000 7f20c1636700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20bc06ef30 con 0x7f20bc09e760 2023-09-30T22:36:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.147+0000 7f20c1636700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20bc06fef0 con 0x7f20bc09de50 2023-09-30T22:36:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.147+0000 7f20c1636700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f20bc1334b0 con 0x7f20bc1338a0 2023-09-30T22:36:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.147+0000 7f20baffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f20bc1338a0 0x7f20bc133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.147+0000 7f20bb7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f20bc09e760 0x7f20bc13a730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60168/0 (socket says 172.21.15.16:60168) 2023-09-30T22:36:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.147+0000 7f20bb7fe700 1 -- 172.21.15.16:0/556486571 learned_addr learned my addr 172.21.15.16:0/556486571 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:36:19.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.147+0000 7f20ba7fc700 1 --2- 172.21.15.16:0/556486571 >> v2:172.21.15.16:3300/0 conn(0x7f20bc09de50 0x7f20bc09e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:19.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.147+0000 7f20b9ffb700 1 -- 172.21.15.16:0/556486571 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3875236283 0 0) 0x7f20bc06ef30 con 0x7f20bc09e760 2023-09-30T22:36:19.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.147+0000 7f20ba7fc700 1 -- 172.21.15.16:0/556486571 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f20bc1338a0 msgr2=0x7f20bc133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:19.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.147+0000 7f20ba7fc700 1 --2- 172.21.15.16:0/556486571 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f20bc1338a0 0x7f20bc133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:19.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.147+0000 7f20ba7fc700 1 -- 172.21.15.16:0/556486571 >> v1:172.21.15.16:6789/0 conn(0x7f20bc09e760 legacy=0x7f20bc13a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:19.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.147+0000 7f20ba7fc700 1 -- 172.21.15.16:0/556486571 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20bc06fc30 con 0x7f20bc09de50 2023-09-30T22:36:19.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.148+0000 7f20ba7fc700 1 --2- 172.21.15.16:0/556486571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20bc09de50 0x7f20bc09e220 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f20b0009840 tx=0x7f20b0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd9d0a6fcbbfb7c6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:19.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.149+0000 7f20b9ffb700 1 -- 172.21.15.16:0/556486571 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b0011830 con 0x7f20bc09de50 2023-09-30T22:36:19.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.149+0000 7f20b9ffb700 1 -- 172.21.15.16:0/556486571 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20b0011990 con 0x7f20bc09de50 2023-09-30T22:36:19.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.149+0000 7f20b9ffb700 1 -- 172.21.15.16:0/556486571 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b0011d20 con 0x7f20bc09de50 2023-09-30T22:36:19.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.149+0000 7f20c1636700 1 -- 172.21.15.16:0/556486571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20bc09de50 msgr2=0x7f20bc09e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:19.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.149+0000 7f20c1636700 1 --2- 172.21.15.16:0/556486571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20bc09de50 0x7f20bc09e220 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f20b0009840 tx=0x7f20b0009b50 comp rx=0 tx=0).stop 2023-09-30T22:36:19.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.149+0000 7f20c1636700 1 -- 172.21.15.16:0/556486571 shutdown_connections 2023-09-30T22:36:19.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.149+0000 7f20c1636700 1 --2- 172.21.15.16:0/556486571 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f20bc1338a0 0x7f20bc133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:19.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.149+0000 7f20c1636700 1 --2- 172.21.15.16:0/556486571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20bc09de50 0x7f20bc09e220 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:19.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.149+0000 7f20c1636700 1 -- 172.21.15.16:0/556486571 >> 172.21.15.16:0/556486571 conn(0x7f20bc0a2aa0 msgr2=0x7f20bc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:19.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.149+0000 7f20c1636700 1 -- 172.21.15.16:0/556486571 shutdown_connections 2023-09-30T22:36:19.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.149+0000 7f20c1636700 1 -- 172.21.15.16:0/556486571 wait complete. 2023-09-30T22:36:19.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.151+0000 7f20c1636700 1 Processor -- start 2023-09-30T22:36:19.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.151+0000 7f20c1636700 1 -- start start 2023-09-30T22:36:19.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.152+0000 7f20c1636700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f20bc09de50 0x7f20bc11ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:19.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.152+0000 7f20c1636700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f20bc09e760 0x7f20bc11ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:19.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.152+0000 7f20c1636700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20bc1338a0 0x7f20bc11f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:19.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.152+0000 7f20c1636700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20bc06f6b0 con 0x7f20bc1338a0 2023-09-30T22:36:19.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.152+0000 7f20c1636700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20bc11fe40 con 0x7f20bc09e760 2023-09-30T22:36:19.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.152+0000 7f20c1636700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f20bc1221f0 con 0x7f20bc09de50 2023-09-30T22:36:19.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.152+0000 7f20ba7fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f20bc09e760 0x7f20bc11ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:19.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.152+0000 7f20baffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f20bc09de50 0x7f20bc11ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:19.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.152+0000 7f20ba7fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f20bc09e760 0x7f20bc11ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38174/0 (socket says 172.21.15.16:38174) 2023-09-30T22:36:19.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.152+0000 7f20ba7fc700 1 -- 172.21.15.16:0/1634167615 learned_addr learned my addr 172.21.15.16:0/1634167615 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:19.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.152+0000 7f20bb7fe700 1 --2- 172.21.15.16:0/1634167615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20bc1338a0 0x7f20bc11f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:19.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.152+0000 7f20ba7fc700 1 -- 172.21.15.16:0/1634167615 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f20bc09de50 msgr2=0x7f20bc11ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:19.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.152+0000 7f20ba7fc700 1 --2- 172.21.15.16:0/1634167615 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f20bc09de50 0x7f20bc11ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:19.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.152+0000 7f20ba7fc700 1 -- 172.21.15.16:0/1634167615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20bc1338a0 msgr2=0x7f20bc11f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:19.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.152+0000 7f20ba7fc700 1 --2- 172.21.15.16:0/1634167615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20bc1338a0 0x7f20bc11f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:19.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.152+0000 7f20ba7fc700 1 -- 172.21.15.16:0/1634167615 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b00094a0 con 0x7f20bc09e760 2023-09-30T22:36:19.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.153+0000 7f20ba7fc700 1 --2- 172.21.15.16:0/1634167615 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f20bc09e760 0x7f20bc11ef40 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f20b00090b0 tx=0x7f20b0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:19.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.153+0000 7f20a37fe700 1 -- 172.21.15.16:0/1634167615 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b0004510 con 0x7f20bc09e760 2023-09-30T22:36:19.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.153+0000 7f20a37fe700 1 -- 172.21.15.16:0/1634167615 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20b001e070 con 0x7f20bc09e760 2023-09-30T22:36:19.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.153+0000 7f20c1636700 1 -- 172.21.15.16:0/1634167615 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20bc120de0 con 0x7f20bc09e760 2023-09-30T22:36:19.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.153+0000 7f20a37fe700 1 -- 172.21.15.16:0/1634167615 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b00038e0 con 0x7f20bc09e760 2023-09-30T22:36:19.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.153+0000 7f20c1636700 1 -- 172.21.15.16:0/1634167615 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20bc12f900 con 0x7f20bc09e760 2023-09-30T22:36:19.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.154+0000 7f20a37fe700 1 -- 172.21.15.16:0/1634167615 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f20b0003a40 con 0x7f20bc09e760 2023-09-30T22:36:19.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.154+0000 7f20a37fe700 1 -- 172.21.15.16:0/1634167615 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f20b0040460 con 0x7f20bc09e760 2023-09-30T22:36:19.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.155+0000 7f20c1636700 1 -- 172.21.15.16:0/1634167615 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20bc074ea0 con 0x7f20bc09e760 2023-09-30T22:36:19.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.157+0000 7fe58e6f6700 1 Processor -- start 2023-09-30T22:36:19.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.157+0000 7fe58e6f6700 1 -- start start 2023-09-30T22:36:19.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.157+0000 7fe58e6f6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe588134630 0x7fe588136a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:19.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.157+0000 7fe58e6f6700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe588136f50 0x7fe588139330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:19.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.157+0000 7fe58e6f6700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe58809b4f0 con 0x7fe588131810 2023-09-30T22:36:19.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.157+0000 7fe58e6f6700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5881313c0 con 0x7fe588136f50 2023-09-30T22:36:19.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.157+0000 7fe58e6f6700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe58806e8c0 con 0x7fe588134630 2023-09-30T22:36:19.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.157+0000 7fe5877fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe588134630 0x7fe588136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:19.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.157+0000 7fe5877fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe588134630 0x7fe588136a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:57094/0 (socket says 172.21.15.16:57094) 2023-09-30T22:36:19.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.158+0000 7fe5877fe700 1 -- 172.21.15.16:0/1814066539 learned_addr learned my addr 172.21.15.16:0/1814066539 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:19.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.158+0000 7fe58cc93700 1 --2- 172.21.15.16:0/1814066539 >> v2:172.21.15.16:3300/0 conn(0x7fe588136f50 0x7fe588139330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:19.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.158+0000 7fe5877fe700 1 -- 172.21.15.16:0/1814066539 >> v2:172.21.15.16:3300/0 conn(0x7fe588136f50 msgr2=0x7fe588139330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:19.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.158+0000 7fe5877fe700 1 --2- 172.21.15.16:0/1814066539 >> v2:172.21.15.16:3300/0 conn(0x7fe588136f50 0x7fe588139330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:19.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.158+0000 7fe5877fe700 1 -- 172.21.15.16:0/1814066539 >> v1:172.21.15.16:6789/0 conn(0x7fe588131810 legacy=0x7fe588131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:19.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.158+0000 7fe5877fe700 1 -- 172.21.15.16:0/1814066539 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe588071d20 con 0x7fe588134630 2023-09-30T22:36:19.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.158+0000 7fe58cc93700 1 --2- 172.21.15.16:0/1814066539 >> v2:172.21.15.16:3300/0 conn(0x7fe588136f50 0x7fe588139330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:36:19.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.158+0000 7fe5877fe700 1 --2- 172.21.15.16:0/1814066539 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe588134630 0x7fe588136a10 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fe57c0098f0 tx=0x7fe57c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a2e2d6d32998d41 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:19.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.158+0000 7fe586ffd700 1 -- 172.21.15.16:0/1814066539 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57c004230 con 0x7fe588134630 2023-09-30T22:36:19.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.158+0000 7fe586ffd700 1 -- 172.21.15.16:0/1814066539 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe57c004390 con 0x7fe588134630 2023-09-30T22:36:19.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.158+0000 7fe586ffd700 1 -- 172.21.15.16:0/1814066539 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57c0036a0 con 0x7fe588134630 2023-09-30T22:36:19.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.159+0000 7fe58e6f6700 1 -- 172.21.15.16:0/1814066539 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe588134630 msgr2=0x7fe588136a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:19.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.159+0000 7fe58e6f6700 1 --2- 172.21.15.16:0/1814066539 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe588134630 0x7fe588136a10 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fe57c0098f0 tx=0x7fe57c009c00 comp rx=0 tx=0).stop 2023-09-30T22:36:19.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.159+0000 7fe58e6f6700 1 -- 172.21.15.16:0/1814066539 shutdown_connections 2023-09-30T22:36:19.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.159+0000 7fe58e6f6700 1 --2- 172.21.15.16:0/1814066539 >> v2:172.21.15.16:3300/0 conn(0x7fe588136f50 0x7fe588139330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:19.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.159+0000 7fe58e6f6700 1 --2- 172.21.15.16:0/1814066539 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe588134630 0x7fe588136a10 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:19.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.159+0000 7fe58e6f6700 1 -- 172.21.15.16:0/1814066539 >> 172.21.15.16:0/1814066539 conn(0x7fe5880a2aa0 msgr2=0x7fe5880a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:19.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.159+0000 7fe58e6f6700 1 -- 172.21.15.16:0/1814066539 shutdown_connections 2023-09-30T22:36:19.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.159+0000 7fe58e6f6700 1 -- 172.21.15.16:0/1814066539 wait complete. 2023-09-30T22:36:19.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.160+0000 7fe58e6f6700 1 Processor -- start 2023-09-30T22:36:19.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.160+0000 7f20a37fe700 1 -- 172.21.15.16:0/1634167615 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f20b0015020 con 0x7f20bc09e760 2023-09-30T22:36:19.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.161+0000 7fe58e6f6700 1 -- start start 2023-09-30T22:36:19.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.161+0000 7fe58e6f6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe588131810 0x7fe58809d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:19.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.161+0000 7fe58e6f6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe588134630 0x7fe58811b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:19.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.161+0000 7fe58e6f6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe588136f50 0x7fe58811b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:19.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.161+0000 7fe58e6f6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe58811a630 con 0x7fe588136f50 2023-09-30T22:36:19.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.161+0000 7fe58e6f6700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe588118d40 con 0x7fe588131810 2023-09-30T22:36:19.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.161+0000 7fe58e6f6700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe5880a1f20 con 0x7fe588134630 2023-09-30T22:36:19.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.161+0000 7fe587fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe588131810 0x7fe58809d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:19.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.161+0000 7fe587fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe588131810 0x7fe58809d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38178/0 (socket says 172.21.15.16:38178) 2023-09-30T22:36:19.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.161+0000 7fe587fff700 1 -- 172.21.15.16:0/1946258419 learned_addr learned my addr 172.21.15.16:0/1946258419 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:19.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.161+0000 7fe587fff700 1 -- 172.21.15.16:0/1946258419 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe588134630 msgr2=0x7fe58811b210 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:36:19.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.161+0000 7fe587fff700 1 --2- 172.21.15.16:0/1946258419 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe588134630 0x7fe58811b210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:19.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.161+0000 7fe587fff700 1 -- 172.21.15.16:0/1946258419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe588136f50 msgr2=0x7fe58811b750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:36:19.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.161+0000 7fe587fff700 1 --2- 172.21.15.16:0/1946258419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe588136f50 0x7fe58811b750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:19.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.161+0000 7fe587fff700 1 -- 172.21.15.16:0/1946258419 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe57c009550 con 0x7fe588131810 2023-09-30T22:36:19.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.162+0000 7fe587fff700 1 --2- 172.21.15.16:0/1946258419 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe588131810 0x7fe58809d5b0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fe57800b4d0 tx=0x7fe57800b890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:19.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.162+0000 7fe584ff9700 1 -- 172.21.15.16:0/1946258419 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe578011830 con 0x7fe588131810 2023-09-30T22:36:19.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.162+0000 7fe584ff9700 1 -- 172.21.15.16:0/1946258419 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe578011990 con 0x7fe588131810 2023-09-30T22:36:19.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.162+0000 7fe58e6f6700 1 -- 172.21.15.16:0/1946258419 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe58806eb00 con 0x7fe588131810 2023-09-30T22:36:19.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.162+0000 7fe584ff9700 1 -- 172.21.15.16:0/1946258419 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe578011d40 con 0x7fe588131810 2023-09-30T22:36:19.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.162+0000 7fe58e6f6700 1 -- 172.21.15.16:0/1946258419 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe588118f10 con 0x7fe588131810 2023-09-30T22:36:19.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.163+0000 7fe58e6f6700 1 -- 172.21.15.16:0/1946258419 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe588119ff0 con 0x7fe588131810 2023-09-30T22:36:19.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.164+0000 7fe584ff9700 1 -- 172.21.15.16:0/1946258419 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe57801a400 con 0x7fe588131810 2023-09-30T22:36:19.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.164+0000 7fe584ff9700 1 -- 172.21.15.16:0/1946258419 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fe57803f490 con 0x7fe588131810 2023-09-30T22:36:19.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.169+0000 7fe584ff9700 1 -- 172.21.15.16:0/1946258419 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe588119ff0 con 0x7fe588131810 2023-09-30T22:36:19.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.246+0000 7ff7e27a4700 1 -- 172.21.15.16:0/1973499316 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff7dc123850 con 0x7ff7dc06f050 2023-09-30T22:36:19.247 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.246+0000 7ff7d97fa700 1 -- 172.21.15.16:0/1973499316 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff7dc123850 con 0x7ff7dc06f050 2023-09-30T22:36:19.247 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:36:19.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.250+0000 7ff7e27a4700 1 -- 172.21.15.16:0/1973499316 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff7dc06f050 msgr2=0x7ff7dc121a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:19.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.250+0000 7ff7e27a4700 1 --2- 172.21.15.16:0/1973499316 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff7dc06f050 0x7ff7dc121a20 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4009c40 tx=0x7ff7c4004750 comp rx=0 tx=0).stop 2023-09-30T22:36:19.250 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.250+0000 7ff7e27a4700 1 -- 172.21.15.16:0/1973499316 shutdown_connections 2023-09-30T22:36:19.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.250+0000 7ff7e27a4700 1 --2- 172.21.15.16:0/1973499316 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff7dc121f60 0x7ff7dc1cc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:19.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.250+0000 7ff7e27a4700 1 --2- 172.21.15.16:0/1973499316 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7dc06e570 0x7ff7dc1214e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:19.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.250+0000 7ff7e27a4700 1 --2- 172.21.15.16:0/1973499316 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff7dc06f050 0x7ff7dc121a20 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:19.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.250+0000 7ff7e27a4700 1 -- 172.21.15.16:0/1973499316 >> 172.21.15.16:0/1973499316 conn(0x7ff7dc0a2aa0 msgr2=0x7ff7dc0a0580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:19.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.250+0000 7ff7e27a4700 1 -- 172.21.15.16:0/1973499316 shutdown_connections 2023-09-30T22:36:19.251 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.250+0000 7ff7e27a4700 1 -- 172.21.15.16:0/1973499316 wait complete. 2023-09-30T22:36:19.267 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:36:19.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.347+0000 7f61769c0700 1 -- 172.21.15.16:0/3829206584 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f617011fdd0 con 0x7f6170130870 2023-09-30T22:36:19.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.347+0000 7f616cff9700 1 -- 172.21.15.16:0/3829206584 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f616402b090 con 0x7f6170130870 2023-09-30T22:36:19.348 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:36:19.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.350+0000 7f61769c0700 1 -- 172.21.15.16:0/3829206584 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6170130870 msgr2=0x7f617009c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:19.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.350+0000 7f61769c0700 1 --2- 172.21.15.16:0/3829206584 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6170130870 0x7f617009c9c0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f616400e900 tx=0x7f616400ec10 comp rx=0 tx=0).stop 2023-09-30T22:36:19.350 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.350+0000 7f61769c0700 1 -- 172.21.15.16:0/3829206584 shutdown_connections 2023-09-30T22:36:19.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.350+0000 7f61769c0700 1 --2- 172.21.15.16:0/3829206584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6170133a80 0x7f617009cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:19.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.350+0000 7f61769c0700 1 --2- 172.21.15.16:0/3829206584 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f617012ff20 0x7f617009c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:19.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.350+0000 7f61769c0700 1 --2- 172.21.15.16:0/3829206584 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6170130870 0x7f617009c9c0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:19.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.350+0000 7f61769c0700 1 -- 172.21.15.16:0/3829206584 >> 172.21.15.16:0/3829206584 conn(0x7f61700a4210 msgr2=0x7f61700a0920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:19.351 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.350+0000 7f61769c0700 1 -- 172.21.15.16:0/3829206584 shutdown_connections 2023-09-30T22:36:19.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.350+0000 7f61769c0700 1 -- 172.21.15.16:0/3829206584 wait complete. 2023-09-30T22:36:19.364 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:36:19.369 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.369+0000 7f20c1636700 1 -- 172.21.15.16:0/1634167615 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f20bc06b9e0 con 0x7f20bc09e760 2023-09-30T22:36:19.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.369+0000 7f20a37fe700 1 -- 172.21.15.16:0/1634167615 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f20b002d7d0 con 0x7f20bc09e760 2023-09-30T22:36:19.370 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:36:19.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.371+0000 7f20c1636700 1 -- 172.21.15.16:0/1634167615 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f20bc09e760 msgr2=0x7f20bc11ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:19.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.371+0000 7f20c1636700 1 --2- 172.21.15.16:0/1634167615 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f20bc09e760 0x7f20bc11ef40 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f20b00090b0 tx=0x7f20b0004750 comp rx=0 tx=0).stop 2023-09-30T22:36:19.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.371+0000 7f20c1636700 1 -- 172.21.15.16:0/1634167615 shutdown_connections 2023-09-30T22:36:19.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.371+0000 7f20c1636700 1 --2- 172.21.15.16:0/1634167615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20bc1338a0 0x7f20bc11f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:19.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.371+0000 7f20c1636700 1 --2- 172.21.15.16:0/1634167615 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f20bc09de50 0x7f20bc11ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:19.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.371+0000 7f20c1636700 1 --2- 172.21.15.16:0/1634167615 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f20bc09e760 0x7f20bc11ef40 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:19.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.371+0000 7f20c1636700 1 -- 172.21.15.16:0/1634167615 >> 172.21.15.16:0/1634167615 conn(0x7f20bc0a2aa0 msgr2=0x7f20bc09ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:19.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.372+0000 7f20c1636700 1 -- 172.21.15.16:0/1634167615 shutdown_connections 2023-09-30T22:36:19.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.372+0000 7f20c1636700 1 -- 172.21.15.16:0/1634167615 wait complete. 2023-09-30T22:36:19.386 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:36:19.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.395+0000 7fe58e6f6700 1 -- 172.21.15.16:0/1946258419 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fe588119ff0 con 0x7fe588131810 2023-09-30T22:36:19.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.395+0000 7fe584ff9700 1 -- 172.21.15.16:0/1946258419 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe588119ff0 con 0x7fe588131810 2023-09-30T22:36:19.396 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:36:19.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.397+0000 7fe58e6f6700 1 -- 172.21.15.16:0/1946258419 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe588131810 msgr2=0x7fe58809d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:19.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.397+0000 7fe58e6f6700 1 --2- 172.21.15.16:0/1946258419 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe588131810 0x7fe58809d5b0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fe57800b4d0 tx=0x7fe57800b890 comp rx=0 tx=0).stop 2023-09-30T22:36:19.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.397+0000 7fe58e6f6700 1 -- 172.21.15.16:0/1946258419 shutdown_connections 2023-09-30T22:36:19.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.397+0000 7fe58e6f6700 1 --2- 172.21.15.16:0/1946258419 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe588131810 0x7fe58809d5b0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:19.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.397+0000 7fe58e6f6700 1 --2- 172.21.15.16:0/1946258419 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe588136f50 0x7fe58811b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:19.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.397+0000 7fe58e6f6700 1 --2- 172.21.15.16:0/1946258419 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe588134630 0x7fe58811b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:19.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.397+0000 7fe58e6f6700 1 -- 172.21.15.16:0/1946258419 >> 172.21.15.16:0/1946258419 conn(0x7fe5880a2aa0 msgr2=0x7fe588133ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:19.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.397+0000 7fe58e6f6700 1 -- 172.21.15.16:0/1946258419 shutdown_connections 2023-09-30T22:36:19.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:19.397+0000 7fe58e6f6700 1 -- 172.21.15.16:0/1946258419 wait complete. 2023-09-30T22:36:19.411 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:36:20.268 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:36:20.365 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:36:20.387 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:36:20.412 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:36:20.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.416+0000 7f0f2eac7700 1 Processor -- start 2023-09-30T22:36:20.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.416+0000 7f0f2eac7700 1 -- start start 2023-09-30T22:36:20.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.416+0000 7f0f2eac7700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0f2809d6f0 0x7f0f2809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:20.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.416+0000 7f0f2eac7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0f2809e000 0x7f0f28130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:20.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.416+0000 7f0f2eac7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0f2809e410 0x7f0f28130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:20.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.416+0000 7f0f2eac7700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f2809f520 con 0x7f0f2809d6f0 2023-09-30T22:36:20.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.416+0000 7f0f2eac7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2809f130 con 0x7f0f2809e410 2023-09-30T22:36:20.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.416+0000 7f0f2eac7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2809e930 con 0x7f0f2809e000 2023-09-30T22:36:20.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.416+0000 7f0f27fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0f2809e000 0x7f0f28130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:20.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.416+0000 7f0f27fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0f2809e000 0x7f0f28130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:57100/0 (socket says 172.21.15.16:57100) 2023-09-30T22:36:20.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.416+0000 7f0f27fff700 1 -- 172.21.15.16:0/3121288195 learned_addr learned my addr 172.21.15.16:0/3121288195 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:20.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.417+0000 7f0f2d064700 1 --2- 172.21.15.16:0/3121288195 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0f2809e410 0x7f0f28130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:20.419 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.417+0000 7f0f2c863700 1 --2- 172.21.15.16:0/3121288195 >> v2:172.21.15.16:3300/0 conn(0x7f0f2809d6f0 0x7f0f2809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:20.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.417+0000 7f0f27fff700 1 -- 172.21.15.16:0/3121288195 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0f2809e410 msgr2=0x7f0f28130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:20.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.417+0000 7f0f27fff700 1 --2- 172.21.15.16:0/3121288195 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0f2809e410 0x7f0f28130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:20.421 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.417+0000 7f0f27fff700 1 -- 172.21.15.16:0/3121288195 >> v2:172.21.15.16:3300/0 conn(0x7f0f2809d6f0 msgr2=0x7f0f2809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:20.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.417+0000 7f0f27fff700 1 --2- 172.21.15.16:0/3121288195 >> v2:172.21.15.16:3300/0 conn(0x7f0f2809d6f0 0x7f0f2809dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:20.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.417+0000 7f0f27fff700 1 -- 172.21.15.16:0/3121288195 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f2809b2d0 con 0x7f0f2809e000 2023-09-30T22:36:20.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.417+0000 7f0f2c863700 1 --2- 172.21.15.16:0/3121288195 >> v2:172.21.15.16:3300/0 conn(0x7f0f2809d6f0 0x7f0f2809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:36:20.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.417+0000 7f0f27fff700 1 --2- 172.21.15.16:0/3121288195 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0f2809e000 0x7f0f28130660 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f0f1400bbc0 tx=0x7f0f1400bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cab00a38ac29b4a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:20.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.417+0000 7f0f277fe700 1 -- 172.21.15.16:0/3121288195 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f1400dcf0 con 0x7f0f2809e000 2023-09-30T22:36:20.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.417+0000 7f0f277fe700 1 -- 172.21.15.16:0/3121288195 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f1400de50 con 0x7f0f2809e000 2023-09-30T22:36:20.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.417+0000 7f0f277fe700 1 -- 172.21.15.16:0/3121288195 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f14010670 con 0x7f0f2809e000 2023-09-30T22:36:20.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.418+0000 7f0f2eac7700 1 -- 172.21.15.16:0/3121288195 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0f2809e000 msgr2=0x7f0f28130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:20.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.418+0000 7f0f2eac7700 1 --2- 172.21.15.16:0/3121288195 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0f2809e000 0x7f0f28130660 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f0f1400bbc0 tx=0x7f0f1400bed0 comp rx=0 tx=0).stop 2023-09-30T22:36:20.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.419+0000 7f0f2eac7700 1 -- 172.21.15.16:0/3121288195 shutdown_connections 2023-09-30T22:36:20.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.419+0000 7f0f2eac7700 1 --2- 172.21.15.16:0/3121288195 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0f2809e410 0x7f0f28130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:20.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.419+0000 7f0f2eac7700 1 --2- 172.21.15.16:0/3121288195 >> v2:172.21.15.16:3300/0 conn(0x7f0f2809d6f0 0x7f0f2809dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:20.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.419+0000 7f0f2eac7700 1 --2- 172.21.15.16:0/3121288195 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0f2809e000 0x7f0f28130660 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:20.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.419+0000 7f0f2eac7700 1 -- 172.21.15.16:0/3121288195 >> 172.21.15.16:0/3121288195 conn(0x7f0f28066f50 msgr2=0x7f0f2809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:20.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.419+0000 7f0f2eac7700 1 -- 172.21.15.16:0/3121288195 shutdown_connections 2023-09-30T22:36:20.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.419+0000 7f0f2eac7700 1 -- 172.21.15.16:0/3121288195 wait complete. 2023-09-30T22:36:20.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.420+0000 7f0f2eac7700 1 Processor -- start 2023-09-30T22:36:20.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.420+0000 7f0f2eac7700 1 -- start start 2023-09-30T22:36:20.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.421+0000 7f0f2eac7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f2809d6f0 0x7f0f28122270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:20.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.421+0000 7f0f2eac7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0f2809e000 0x7f0f281227b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:20.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.421+0000 7f0f2eac7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0f2809e410 0x7f0f28122cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:20.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.421+0000 7f0f2eac7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f28124430 con 0x7f0f2809d6f0 2023-09-30T22:36:20.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.421+0000 7f0f2eac7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2809c790 con 0x7f0f2809e410 2023-09-30T22:36:20.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.421+0000 7f0f2eac7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0f28123ef0 con 0x7f0f2809e000 2023-09-30T22:36:20.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.421+0000 7f0f2c863700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f2809d6f0 0x7f0f28122270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:20.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.421+0000 7f0f2c863700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f2809d6f0 0x7f0f28122270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41964/0 (socket says 172.21.15.16:41964) 2023-09-30T22:36:20.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.421+0000 7f0f2c863700 1 -- 172.21.15.16:0/2055767737 learned_addr learned my addr 172.21.15.16:0/2055767737 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:20.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.421+0000 7f0f27fff700 1 --2- 172.21.15.16:0/2055767737 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0f2809e000 0x7f0f281227b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:20.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.421+0000 7f0f2d064700 1 --2- 172.21.15.16:0/2055767737 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0f2809e410 0x7f0f28122cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:20.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.421+0000 7f0f2d064700 1 -- 172.21.15.16:0/2055767737 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0f2809e000 msgr2=0x7f0f281227b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:20.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.421+0000 7f0f2d064700 1 --2- 172.21.15.16:0/2055767737 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0f2809e000 0x7f0f281227b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:20.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.421+0000 7f0f2d064700 1 -- 172.21.15.16:0/2055767737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f2809d6f0 msgr2=0x7f0f28122270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:20.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.421+0000 7f0f2d064700 1 --2- 172.21.15.16:0/2055767737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f2809d6f0 0x7f0f28122270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:20.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.421+0000 7f0f2d064700 1 -- 172.21.15.16:0/2055767737 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f1400b7f0 con 0x7f0f2809e410 2023-09-30T22:36:20.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.422+0000 7f0f2d064700 1 --2- 172.21.15.16:0/2055767737 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0f2809e410 0x7f0f28122cf0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f0f20009d90 tx=0x7f0f2000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:20.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.422+0000 7f0f257fa700 1 -- 172.21.15.16:0/2055767737 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f20009c00 con 0x7f0f2809e410 2023-09-30T22:36:20.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.422+0000 7f0f257fa700 1 -- 172.21.15.16:0/2055767737 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f2000f040 con 0x7f0f2809e410 2023-09-30T22:36:20.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.422+0000 7f0f257fa700 1 -- 172.21.15.16:0/2055767737 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f20014480 con 0x7f0f2809e410 2023-09-30T22:36:20.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.422+0000 7f0f2eac7700 1 -- 172.21.15.16:0/2055767737 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f28125a10 con 0x7f0f2809e410 2023-09-30T22:36:20.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.422+0000 7f0f2eac7700 1 -- 172.21.15.16:0/2055767737 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f28125cf0 con 0x7f0f2809e410 2023-09-30T22:36:20.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.424+0000 7f0f2eac7700 1 -- 172.21.15.16:0/2055767737 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f080052e0 con 0x7f0f2809e410 2023-09-30T22:36:20.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.424+0000 7f0f257fa700 1 -- 172.21.15.16:0/2055767737 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0f200145e0 con 0x7f0f2809e410 2023-09-30T22:36:20.428 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.424+0000 7f0f257fa700 1 -- 172.21.15.16:0/2055767737 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f0f20040770 con 0x7f0f2809e410 2023-09-30T22:36:20.429 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.429+0000 7f0f257fa700 1 -- 172.21.15.16:0/2055767737 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0f20012930 con 0x7f0f2809e410 2023-09-30T22:36:20.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.513+0000 7f9fa4636700 1 Processor -- start 2023-09-30T22:36:20.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.513+0000 7f9fa4636700 1 -- start start 2023-09-30T22:36:20.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.513+0000 7f9fa4636700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9f9c06a770 0x7f9f9c09d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:20.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.513+0000 7f9fa4636700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f9f9c13d1f0 0x7f9f9c13f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:20.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.513+0000 7f9fa4636700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f9c09a810 con 0x7f9f9c09dc30 2023-09-30T22:36:20.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.513+0000 7f9fa4636700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f9c06ab40 con 0x7f9f9c13d1f0 2023-09-30T22:36:20.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.513+0000 7f9fa4636700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9c09ebc0 con 0x7f9f9c06a770 2023-09-30T22:36:20.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.514+0000 7f9fa23d2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9f9c06a770 0x7f9f9c09d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:20.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.514+0000 7f9fa1bd1700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9f9c09dc30 0x7f9f9c09e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60186/0 (socket says 172.21.15.16:60186) 2023-09-30T22:36:20.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.514+0000 7f9fa1bd1700 1 -- 172.21.15.16:0/1174262999 learned_addr learned my addr 172.21.15.16:0/1174262999 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:36:20.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.514+0000 7f9fa2bd3700 1 --2- 172.21.15.16:0/1174262999 >> v2:172.21.15.16:3300/0 conn(0x7f9f9c13d1f0 0x7f9f9c13f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:20.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.514+0000 7f9fa13d0700 1 -- 172.21.15.16:0/1174262999 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1598681107 0 0) 0x7f9f9c09a810 con 0x7f9f9c09dc30 2023-09-30T22:36:20.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.514+0000 7f9fa13d0700 1 -- 172.21.15.16:0/1174262999 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f880035b0 con 0x7f9f9c09dc30 2023-09-30T22:36:20.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.514+0000 7f9fa2bd3700 1 -- 172.21.15.16:0/1174262999 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9f9c06a770 msgr2=0x7f9f9c09d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:20.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.514+0000 7f9fa2bd3700 1 --2- 172.21.15.16:0/1174262999 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9f9c06a770 0x7f9f9c09d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:20.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.514+0000 7f9fa2bd3700 1 -- 172.21.15.16:0/1174262999 >> v1:172.21.15.16:6789/0 conn(0x7f9f9c09dc30 legacy=0x7f9f9c09e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:20.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.514+0000 7f9fa2bd3700 1 -- 172.21.15.16:0/1174262999 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f9c09e920 con 0x7f9f9c13d1f0 2023-09-30T22:36:20.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.515+0000 7f9fa2bd3700 1 --2- 172.21.15.16:0/1174262999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f9c13d1f0 0x7f9f9c13f5d0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f9f94009dc0 tx=0x7f9f9400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d05b4440ef2bda server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:20.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.515+0000 7f9fa13d0700 1 -- 172.21.15.16:0/1174262999 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9400e070 con 0x7f9f9c13d1f0 2023-09-30T22:36:20.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.515+0000 7f9fa13d0700 1 -- 172.21.15.16:0/1174262999 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f9400bae0 con 0x7f9f9c13d1f0 2023-09-30T22:36:20.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.515+0000 7f9fa13d0700 1 -- 172.21.15.16:0/1174262999 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9400be90 con 0x7f9f9c13d1f0 2023-09-30T22:36:20.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.515+0000 7f9fa4636700 1 -- 172.21.15.16:0/1174262999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f9c13d1f0 msgr2=0x7f9f9c13f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:20.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.515+0000 7f9fa4636700 1 --2- 172.21.15.16:0/1174262999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f9c13d1f0 0x7f9f9c13f5d0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f9f94009dc0 tx=0x7f9f9400b330 comp rx=0 tx=0).stop 2023-09-30T22:36:20.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.516+0000 7f9fa4636700 1 -- 172.21.15.16:0/1174262999 shutdown_connections 2023-09-30T22:36:20.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.516+0000 7f9fa4636700 1 --2- 172.21.15.16:0/1174262999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f9c13d1f0 0x7f9f9c13f5d0 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:20.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.516+0000 7f9fa4636700 1 --2- 172.21.15.16:0/1174262999 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9f9c06a770 0x7f9f9c09d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:20.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.516+0000 7f9fa4636700 1 -- 172.21.15.16:0/1174262999 >> 172.21.15.16:0/1174262999 conn(0x7f9f9c066f50 msgr2=0x7f9f9c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:20.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.516+0000 7f9fa4636700 1 -- 172.21.15.16:0/1174262999 shutdown_connections 2023-09-30T22:36:20.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.516+0000 7f9fa4636700 1 -- 172.21.15.16:0/1174262999 wait complete. 2023-09-30T22:36:20.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.517+0000 7f9fa4636700 1 Processor -- start 2023-09-30T22:36:20.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.517+0000 7f9fa4636700 1 -- start start 2023-09-30T22:36:20.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.517+0000 7f9fa4636700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f9c06a770 0x7f9f9c12e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:20.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.518+0000 7f9fa4636700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9f9c09dc30 0x7f9f9c11a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:20.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.518+0000 7f9fa4636700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9f9c13d1f0 0x7f9f9c11ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:20.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.518+0000 7f9fa4636700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9c140530 con 0x7f9f9c06a770 2023-09-30T22:36:20.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.518+0000 7f9fa4636700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9c11c430 con 0x7f9f9c09dc30 2023-09-30T22:36:20.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.518+0000 7f9fa4636700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9c11d2c0 con 0x7f9f9c13d1f0 2023-09-30T22:36:20.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.518+0000 7f9fa23d2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f9c06a770 0x7f9f9c12e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:20.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.518+0000 7f9fa23d2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f9c06a770 0x7f9f9c12e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41990/0 (socket says 172.21.15.16:41990) 2023-09-30T22:36:20.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.518+0000 7f9fa23d2700 1 -- 172.21.15.16:0/2030436470 learned_addr learned my addr 172.21.15.16:0/2030436470 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:20.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.518+0000 7f9fa2bd3700 1 --2- 172.21.15.16:0/2030436470 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9f9c13d1f0 0x7f9f9c11ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:20.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.518+0000 7f9fa23d2700 1 -- 172.21.15.16:0/2030436470 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9f9c13d1f0 msgr2=0x7f9f9c11ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:20.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.518+0000 7f9fa23d2700 1 --2- 172.21.15.16:0/2030436470 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9f9c13d1f0 0x7f9f9c11ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:20.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.518+0000 7f9fa23d2700 1 -- 172.21.15.16:0/2030436470 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9f9c09dc30 msgr2=0x7f9f9c11a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:20.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.518+0000 7f9fa23d2700 1 --2- 172.21.15.16:0/2030436470 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9f9c09dc30 0x7f9f9c11a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:20.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.518+0000 7f9fa23d2700 1 -- 172.21.15.16:0/2030436470 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f94009a70 con 0x7f9f9c06a770 2023-09-30T22:36:20.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.518+0000 7f9fa23d2700 1 --2- 172.21.15.16:0/2030436470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f9c06a770 0x7f9f9c12e390 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f9f9800e8a0 tx=0x7f9f9800ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:20.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.518+0000 7f9f92ffd700 1 -- 172.21.15.16:0/2030436470 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f98009590 con 0x7f9f9c06a770 2023-09-30T22:36:20.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.519+0000 7f9f92ffd700 1 -- 172.21.15.16:0/2030436470 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f98013680 con 0x7f9f9c06a770 2023-09-30T22:36:20.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.519+0000 7f9fa4636700 1 -- 172.21.15.16:0/2030436470 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f9c11e420 con 0x7f9f9c06a770 2023-09-30T22:36:20.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.519+0000 7f9fa4636700 1 -- 172.21.15.16:0/2030436470 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f9c11cc20 con 0x7f9f9c06a770 2023-09-30T22:36:20.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.519+0000 7f9f92ffd700 1 -- 172.21.15.16:0/2030436470 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f980139f0 con 0x7f9f9c06a770 2023-09-30T22:36:20.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.520+0000 7f9f92ffd700 1 -- 172.21.15.16:0/2030436470 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9f98013b50 con 0x7f9f9c06a770 2023-09-30T22:36:20.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.520+0000 7f9fa4636700 1 -- 172.21.15.16:0/2030436470 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f840052e0 con 0x7f9f9c06a770 2023-09-30T22:36:20.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.520+0000 7f9f92ffd700 1 -- 172.21.15.16:0/2030436470 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f9f9803fa90 con 0x7f9f9c06a770 2023-09-30T22:36:20.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.530+0000 7f9f92ffd700 1 -- 172.21.15.16:0/2030436470 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9f9800c480 con 0x7f9f9c06a770 2023-09-30T22:36:20.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.554+0000 7f7b312ab700 1 Processor -- start 2023-09-30T22:36:20.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.554+0000 7f7b312ab700 1 -- start start 2023-09-30T22:36:20.557 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.555+0000 7f7b312ab700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7b2c09d6f0 0x7f7b2c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:20.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.555+0000 7f7b312ab700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7b2c09e000 0x7f7b2c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:20.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.555+0000 7f7b312ab700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7b2c09e410 0x7f7b2c133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:20.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.555+0000 7f7b312ab700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b2c06b460 con 0x7f7b2c09d6f0 2023-09-30T22:36:20.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.555+0000 7f7b312ab700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2c09b120 con 0x7f7b2c09e000 2023-09-30T22:36:20.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.555+0000 7f7b312ab700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2c09a7e0 con 0x7f7b2c09e410 2023-09-30T22:36:20.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.555+0000 7f7b2affd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7b2c09d6f0 0x7f7b2c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:20.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.555+0000 7f7b2a7fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7b2c09e000 0x7f7b2c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:20.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.555+0000 7f7b2affd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7b2c09d6f0 0x7f7b2c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41994/0 (socket says 172.21.15.16:41994) 2023-09-30T22:36:20.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.555+0000 7f7b2affd700 1 -- 172.21.15.16:0/1950362202 learned_addr learned my addr 172.21.15.16:0/1950362202 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:20.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.555+0000 7f7b2a7fc700 1 -- 172.21.15.16:0/1950362202 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7b2c09e410 msgr2=0x7f7b2c133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:20.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.555+0000 7f7b2a7fc700 1 --2- 172.21.15.16:0/1950362202 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7b2c09e410 0x7f7b2c133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:20.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.555+0000 7f7b2a7fc700 1 -- 172.21.15.16:0/1950362202 >> v2:172.21.15.16:3300/0 conn(0x7f7b2c09d6f0 msgr2=0x7f7b2c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:20.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.555+0000 7f7b2a7fc700 1 --2- 172.21.15.16:0/1950362202 >> v2:172.21.15.16:3300/0 conn(0x7f7b2c09d6f0 0x7f7b2c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:20.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.555+0000 7f7b2a7fc700 1 -- 172.21.15.16:0/1950362202 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b2c09a500 con 0x7f7b2c09e000 2023-09-30T22:36:20.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.555+0000 7f7b2affd700 1 --2- 172.21.15.16:0/1950362202 >> v2:172.21.15.16:3300/0 conn(0x7f7b2c09d6f0 0x7f7b2c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:20.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.556+0000 7f7b2a7fc700 1 --2- 172.21.15.16:0/1950362202 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7b2c09e000 0x7f7b2c1339f0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f7b20009870 tx=0x7f7b20009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36cf91350f4ed3d7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:20.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.556+0000 7f7b29ffb700 1 -- 172.21.15.16:0/1950362202 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b20011830 con 0x7f7b2c09e000 2023-09-30T22:36:20.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.556+0000 7f7b29ffb700 1 -- 172.21.15.16:0/1950362202 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b20011990 con 0x7f7b2c09e000 2023-09-30T22:36:20.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.556+0000 7f7b29ffb700 1 -- 172.21.15.16:0/1950362202 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b20011d20 con 0x7f7b2c09e000 2023-09-30T22:36:20.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.556+0000 7f7b312ab700 1 -- 172.21.15.16:0/1950362202 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7b2c09e000 msgr2=0x7f7b2c1339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:20.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.556+0000 7f7b312ab700 1 --2- 172.21.15.16:0/1950362202 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7b2c09e000 0x7f7b2c1339f0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f7b20009870 tx=0x7f7b20009b80 comp rx=0 tx=0).stop 2023-09-30T22:36:20.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.557+0000 7f7b312ab700 1 -- 172.21.15.16:0/1950362202 shutdown_connections 2023-09-30T22:36:20.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.557+0000 7f7b312ab700 1 --2- 172.21.15.16:0/1950362202 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7b2c09e410 0x7f7b2c133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:20.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.557+0000 7f7b312ab700 1 --2- 172.21.15.16:0/1950362202 >> v2:172.21.15.16:3300/0 conn(0x7f7b2c09d6f0 0x7f7b2c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:20.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.557+0000 7f7b312ab700 1 --2- 172.21.15.16:0/1950362202 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7b2c09e000 0x7f7b2c1339f0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:20.560 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.557+0000 7f7b312ab700 1 -- 172.21.15.16:0/1950362202 >> 172.21.15.16:0/1950362202 conn(0x7f7b2c066f50 msgr2=0x7f7b2c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:20.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.557+0000 7f7b312ab700 1 -- 172.21.15.16:0/1950362202 shutdown_connections 2023-09-30T22:36:20.561 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.557+0000 7f7b312ab700 1 -- 172.21.15.16:0/1950362202 wait complete. 2023-09-30T22:36:20.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.558+0000 7f7b312ab700 1 Processor -- start 2023-09-30T22:36:20.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.558+0000 7f7b312ab700 1 -- start start 2023-09-30T22:36:20.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.559+0000 7f7b312ab700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b2c09d6f0 0x7f7b2c1a4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:20.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.559+0000 7f7b312ab700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7b2c09e000 0x7f7b2c1a54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:20.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.559+0000 7f7b312ab700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7b2c09e410 0x7f7b2c1a59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:20.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.559+0000 7f7b312ab700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2c0a2090 con 0x7f7b2c09d6f0 2023-09-30T22:36:20.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.559+0000 7f7b312ab700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2c1a8420 con 0x7f7b2c09e000 2023-09-30T22:36:20.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.559+0000 7f7b312ab700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2c1a86e0 con 0x7f7b2c09e410 2023-09-30T22:36:20.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.559+0000 7f7b2affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b2c09d6f0 0x7f7b2c1a4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:20.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.559+0000 7f7b2a7fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7b2c09e000 0x7f7b2c1a54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:20.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.559+0000 7f7b2affd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b2c09d6f0 0x7f7b2c1a4f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42008/0 (socket says 172.21.15.16:42008) 2023-09-30T22:36:20.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.559+0000 7f7b2affd700 1 -- 172.21.15.16:0/1562435952 learned_addr learned my addr 172.21.15.16:0/1562435952 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:20.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.559+0000 7f7b2a7fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7b2c09e000 0x7f7b2c1a54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38232/0 (socket says 172.21.15.16:38232) 2023-09-30T22:36:20.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.559+0000 7f7b2b7fe700 1 --2- 172.21.15.16:0/1562435952 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7b2c09e410 0x7f7b2c1a59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:20.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.559+0000 7f7b2a7fc700 1 -- 172.21.15.16:0/1562435952 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7b2c09e410 msgr2=0x7f7b2c1a59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:20.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.559+0000 7f7b2a7fc700 1 --2- 172.21.15.16:0/1562435952 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7b2c09e410 0x7f7b2c1a59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:20.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.559+0000 7f7b2a7fc700 1 -- 172.21.15.16:0/1562435952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b2c09d6f0 msgr2=0x7f7b2c1a4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:20.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.559+0000 7f7b2a7fc700 1 --2- 172.21.15.16:0/1562435952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b2c09d6f0 0x7f7b2c1a4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:20.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.559+0000 7f7b2a7fc700 1 -- 172.21.15.16:0/1562435952 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b1c009550 con 0x7f7b2c09e000 2023-09-30T22:36:20.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.560+0000 7f7b2a7fc700 1 --2- 172.21.15.16:0/1562435952 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7b2c09e000 0x7f7b2c1a54b0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f7b20016000 tx=0x7f7b20004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:20.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.560+0000 7f7b137fe700 1 -- 172.21.15.16:0/1562435952 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b20004510 con 0x7f7b2c09e000 2023-09-30T22:36:20.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.560+0000 7f7b137fe700 1 -- 172.21.15.16:0/1562435952 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b2001f070 con 0x7f7b2c09e000 2023-09-30T22:36:20.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.560+0000 7f7b312ab700 1 -- 172.21.15.16:0/1562435952 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b200094a0 con 0x7f7b2c09e000 2023-09-30T22:36:20.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.560+0000 7f7b312ab700 1 -- 172.21.15.16:0/1562435952 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b2c1a89c0 con 0x7f7b2c09e000 2023-09-30T22:36:20.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.560+0000 7f7b137fe700 1 -- 172.21.15.16:0/1562435952 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b200038c0 con 0x7f7b2c09e000 2023-09-30T22:36:20.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.561+0000 7f7b137fe700 1 -- 172.21.15.16:0/1562435952 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7b20003a20 con 0x7f7b2c09e000 2023-09-30T22:36:20.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.561+0000 7f7b312ab700 1 -- 172.21.15.16:0/1562435952 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b0c0052e0 con 0x7f7b2c09e000 2023-09-30T22:36:20.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.561+0000 7f7b137fe700 1 -- 172.21.15.16:0/1562435952 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f7b20021070 con 0x7f7b2c09e000 2023-09-30T22:36:20.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.566+0000 7f7b137fe700 1 -- 172.21.15.16:0/1562435952 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7b200253e0 con 0x7f7b2c09e000 2023-09-30T22:36:20.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.574+0000 7f89afd4b700 1 Processor -- start 2023-09-30T22:36:20.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.574+0000 7f89afd4b700 1 -- start start 2023-09-30T22:36:20.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.575+0000 7f89afd4b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f89a809e930 0x7f89a809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:20.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.575+0000 7f89afd4b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f89a809f240 0x7f89a81317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:20.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.575+0000 7f89afd4b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89a8070940 con 0x7f89a809f650 2023-09-30T22:36:20.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.575+0000 7f89afd4b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89a806fd60 con 0x7f89a809e930 2023-09-30T22:36:20.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.575+0000 7f89afd4b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f89a806fa80 con 0x7f89a809f240 2023-09-30T22:36:20.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.575+0000 7f89ad2e6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f89a809f240 0x7f89a81317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:20.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.575+0000 7f89ad2e6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f89a809f240 0x7f89a81317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:57162/0 (socket says 172.21.15.16:57162) 2023-09-30T22:36:20.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.575+0000 7f89ad2e6700 1 -- 172.21.15.16:0/4208091918 learned_addr learned my addr 172.21.15.16:0/4208091918 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:20.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.575+0000 7f89ae2e8700 1 --1- 172.21.15.16:0/4208091918 >> v1:172.21.15.16:6789/0 conn(0x7f89a809f650 0x7f89a8131d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60202/0 (socket says 172.21.15.16:60202) 2023-09-30T22:36:20.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.575+0000 7f89adae7700 1 --2- 172.21.15.16:0/4208091918 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f89a809e930 0x7f89a809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:20.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.575+0000 7f89acae5700 1 -- 172.21.15.16:0/4208091918 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1171298855 0 0) 0x7f89a8070940 con 0x7f89a809f650 2023-09-30T22:36:20.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.575+0000 7f89acae5700 1 -- 172.21.15.16:0/4208091918 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89940035b0 con 0x7f89a809f650 2023-09-30T22:36:20.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.575+0000 7f89adae7700 1 -- 172.21.15.16:0/4208091918 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f89a809f240 msgr2=0x7f89a81317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:20.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.575+0000 7f89adae7700 1 --2- 172.21.15.16:0/4208091918 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f89a809f240 0x7f89a81317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:20.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.575+0000 7f89adae7700 1 -- 172.21.15.16:0/4208091918 >> v1:172.21.15.16:6789/0 conn(0x7f89a809f650 legacy=0x7f89a8131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:20.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.575+0000 7f89adae7700 1 -- 172.21.15.16:0/4208091918 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a809fb60 con 0x7f89a809e930 2023-09-30T22:36:20.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.576+0000 7f89adae7700 1 --2- 172.21.15.16:0/4208091918 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f89a809e930 0x7f89a809ed00 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f89a4009870 tx=0x7f89a4009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b69ac625fc519e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:20.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.576+0000 7f89acae5700 1 -- 172.21.15.16:0/4208091918 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a4011830 con 0x7f89a809e930 2023-09-30T22:36:20.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.576+0000 7f89acae5700 1 -- 172.21.15.16:0/4208091918 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89a4011990 con 0x7f89a809e930 2023-09-30T22:36:20.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.576+0000 7f89acae5700 1 -- 172.21.15.16:0/4208091918 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a4011d20 con 0x7f89a809e930 2023-09-30T22:36:20.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.577+0000 7f89afd4b700 1 -- 172.21.15.16:0/4208091918 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f89a809e930 msgr2=0x7f89a809ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:20.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.577+0000 7f89afd4b700 1 --2- 172.21.15.16:0/4208091918 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f89a809e930 0x7f89a809ed00 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f89a4009870 tx=0x7f89a4009b80 comp rx=0 tx=0).stop 2023-09-30T22:36:20.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.577+0000 7f89afd4b700 1 -- 172.21.15.16:0/4208091918 shutdown_connections 2023-09-30T22:36:20.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.577+0000 7f89afd4b700 1 --2- 172.21.15.16:0/4208091918 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f89a809e930 0x7f89a809ed00 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:20.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.577+0000 7f89afd4b700 1 --2- 172.21.15.16:0/4208091918 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f89a809f240 0x7f89a81317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:20.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.577+0000 7f89afd4b700 1 -- 172.21.15.16:0/4208091918 >> 172.21.15.16:0/4208091918 conn(0x7f89a80a4210 msgr2=0x7f89a80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:20.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.577+0000 7f89afd4b700 1 -- 172.21.15.16:0/4208091918 shutdown_connections 2023-09-30T22:36:20.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.577+0000 7f89afd4b700 1 -- 172.21.15.16:0/4208091918 wait complete. 2023-09-30T22:36:20.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.579+0000 7f89afd4b700 1 Processor -- start 2023-09-30T22:36:20.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.579+0000 7f89afd4b700 1 -- start start 2023-09-30T22:36:20.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.580+0000 7f89afd4b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f89a809e930 0x7f89a81a3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:20.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.580+0000 7f89afd4b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89a809f240 0x7f89a81a4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:20.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.580+0000 7f89afd4b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f89a809f650 0x7f89a81a46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:20.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.580+0000 7f89afd4b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a8071ce0 con 0x7f89a809f240 2023-09-30T22:36:20.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.580+0000 7f89afd4b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89a81a5d50 con 0x7f89a809e930 2023-09-30T22:36:20.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.580+0000 7f89afd4b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f89a80a3410 con 0x7f89a809f650 2023-09-30T22:36:20.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.580+0000 7f89ad2e6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89a809f240 0x7f89a81a4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:20.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.580+0000 7f89ad2e6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89a809f240 0x7f89a81a4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42022/0 (socket says 172.21.15.16:42022) 2023-09-30T22:36:20.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.580+0000 7f89ad2e6700 1 -- 172.21.15.16:0/461809486 learned_addr learned my addr 172.21.15.16:0/461809486 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:20.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.580+0000 7f89adae7700 1 --2- 172.21.15.16:0/461809486 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f89a809e930 0x7f89a81a3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:20.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.580+0000 7f89ae2e8700 1 --2- 172.21.15.16:0/461809486 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f89a809f650 0x7f89a81a46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:20.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.580+0000 7f89adae7700 1 -- 172.21.15.16:0/461809486 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f89a809f650 msgr2=0x7f89a81a46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:20.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.580+0000 7f89adae7700 1 --2- 172.21.15.16:0/461809486 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f89a809f650 0x7f89a81a46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:20.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.580+0000 7f89adae7700 1 -- 172.21.15.16:0/461809486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89a809f240 msgr2=0x7f89a81a4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:20.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.580+0000 7f89adae7700 1 --2- 172.21.15.16:0/461809486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89a809f240 0x7f89a81a4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:20.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.580+0000 7f89adae7700 1 -- 172.21.15.16:0/461809486 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a40094a0 con 0x7f89a809e930 2023-09-30T22:36:20.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.581+0000 7f89adae7700 1 --2- 172.21.15.16:0/461809486 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f89a809e930 0x7f89a81a3c30 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f89a4016000 tx=0x7f89a4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:20.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.581+0000 7f899e7fc700 1 -- 172.21.15.16:0/461809486 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a4004510 con 0x7f89a809e930 2023-09-30T22:36:20.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.581+0000 7f899e7fc700 1 -- 172.21.15.16:0/461809486 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89a401f070 con 0x7f89a809e930 2023-09-30T22:36:20.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.581+0000 7f89afd4b700 1 -- 172.21.15.16:0/461809486 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89a80a0190 con 0x7f89a809e930 2023-09-30T22:36:20.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.581+0000 7f899e7fc700 1 -- 172.21.15.16:0/461809486 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a40038e0 con 0x7f89a809e930 2023-09-30T22:36:20.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.581+0000 7f89afd4b700 1 -- 172.21.15.16:0/461809486 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89a81a5770 con 0x7f89a809e930 2023-09-30T22:36:20.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.582+0000 7f89afd4b700 1 -- 172.21.15.16:0/461809486 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89a81a0b30 con 0x7f89a809e930 2023-09-30T22:36:20.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.583+0000 7f899e7fc700 1 -- 172.21.15.16:0/461809486 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f89a4003a40 con 0x7f89a809e930 2023-09-30T22:36:20.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.583+0000 7f899e7fc700 1 -- 172.21.15.16:0/461809486 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f89a4021070 con 0x7f89a809e930 2023-09-30T22:36:20.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.592+0000 7f899e7fc700 1 -- 172.21.15.16:0/461809486 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f89a4026480 con 0x7f89a809e930 2023-09-30T22:36:20.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.671+0000 7f0f2eac7700 1 -- 172.21.15.16:0/2055767737 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f0f08005860 con 0x7f0f2809e410 2023-09-30T22:36:20.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.671+0000 7f0f257fa700 1 -- 172.21.15.16:0/2055767737 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0f2002a070 con 0x7f0f2809e410 2023-09-30T22:36:20.672 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:36:20.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.674+0000 7f0f2eac7700 1 -- 172.21.15.16:0/2055767737 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0f2809e410 msgr2=0x7f0f28122cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:20.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.674+0000 7f0f2eac7700 1 --2- 172.21.15.16:0/2055767737 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0f2809e410 0x7f0f28122cf0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f0f20009d90 tx=0x7f0f2000c3a0 comp rx=0 tx=0).stop 2023-09-30T22:36:20.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.675+0000 7f0f2eac7700 1 -- 172.21.15.16:0/2055767737 shutdown_connections 2023-09-30T22:36:20.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.675+0000 7f0f2eac7700 1 --2- 172.21.15.16:0/2055767737 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0f2809e410 0x7f0f28122cf0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:20.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.675+0000 7f0f2eac7700 1 --2- 172.21.15.16:0/2055767737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f2809d6f0 0x7f0f28122270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:20.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.675+0000 7f0f2eac7700 1 --2- 172.21.15.16:0/2055767737 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0f2809e000 0x7f0f281227b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:20.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.675+0000 7f0f2eac7700 1 -- 172.21.15.16:0/2055767737 >> 172.21.15.16:0/2055767737 conn(0x7f0f28066f50 msgr2=0x7f0f28130990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:20.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.675+0000 7f0f2eac7700 1 -- 172.21.15.16:0/2055767737 shutdown_connections 2023-09-30T22:36:20.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.675+0000 7f0f2eac7700 1 -- 172.21.15.16:0/2055767737 wait complete. 2023-09-30T22:36:20.695 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:36:20.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.745+0000 7f9fa4636700 1 -- 172.21.15.16:0/2030436470 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f9f84005590 con 0x7f9f9c06a770 2023-09-30T22:36:20.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.745+0000 7f9f92ffd700 1 -- 172.21.15.16:0/2030436470 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9f98017020 con 0x7f9f9c06a770 2023-09-30T22:36:20.746 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:36:20.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.748+0000 7f9fa4636700 1 -- 172.21.15.16:0/2030436470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f9c06a770 msgr2=0x7f9f9c12e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:20.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.748+0000 7f9fa4636700 1 --2- 172.21.15.16:0/2030436470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f9c06a770 0x7f9f9c12e390 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f9f9800e8a0 tx=0x7f9f9800ebb0 comp rx=0 tx=0).stop 2023-09-30T22:36:20.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.748+0000 7f9fa4636700 1 -- 172.21.15.16:0/2030436470 shutdown_connections 2023-09-30T22:36:20.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.748+0000 7f9fa4636700 1 --2- 172.21.15.16:0/2030436470 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9f9c13d1f0 0x7f9f9c11ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:20.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.748+0000 7f9fa4636700 1 --2- 172.21.15.16:0/2030436470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f9c06a770 0x7f9f9c12e390 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:20.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.748+0000 7f9fa4636700 1 --2- 172.21.15.16:0/2030436470 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9f9c09dc30 0x7f9f9c11a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:20.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.748+0000 7f9fa4636700 1 -- 172.21.15.16:0/2030436470 >> 172.21.15.16:0/2030436470 conn(0x7f9f9c066f50 msgr2=0x7f9f9c0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:20.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.748+0000 7f9fa4636700 1 -- 172.21.15.16:0/2030436470 shutdown_connections 2023-09-30T22:36:20.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.748+0000 7f9fa4636700 1 -- 172.21.15.16:0/2030436470 wait complete. 2023-09-30T22:36:20.762 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:36:20.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.790+0000 7f7b312ab700 1 -- 172.21.15.16:0/1562435952 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7b0c005590 con 0x7f7b2c09e000 2023-09-30T22:36:20.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.790+0000 7f7b137fe700 1 -- 172.21.15.16:0/1562435952 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7b20015030 con 0x7f7b2c09e000 2023-09-30T22:36:20.791 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:36:20.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.793+0000 7f7b312ab700 1 -- 172.21.15.16:0/1562435952 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7b2c09e000 msgr2=0x7f7b2c1a54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:20.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.793+0000 7f7b312ab700 1 --2- 172.21.15.16:0/1562435952 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7b2c09e000 0x7f7b2c1a54b0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f7b20016000 tx=0x7f7b20004750 comp rx=0 tx=0).stop 2023-09-30T22:36:20.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.793+0000 7f7b312ab700 1 -- 172.21.15.16:0/1562435952 shutdown_connections 2023-09-30T22:36:20.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.793+0000 7f7b312ab700 1 --2- 172.21.15.16:0/1562435952 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7b2c09e410 0x7f7b2c1a59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:20.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.793+0000 7f7b312ab700 1 --2- 172.21.15.16:0/1562435952 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b2c09d6f0 0x7f7b2c1a4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:20.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.793+0000 7f7b312ab700 1 --2- 172.21.15.16:0/1562435952 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7b2c09e000 0x7f7b2c1a54b0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:20.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.793+0000 7f7b312ab700 1 -- 172.21.15.16:0/1562435952 >> 172.21.15.16:0/1562435952 conn(0x7f7b2c066f50 msgr2=0x7f7b2c06f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:20.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.793+0000 7f7b312ab700 1 -- 172.21.15.16:0/1562435952 shutdown_connections 2023-09-30T22:36:20.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.793+0000 7f7b312ab700 1 -- 172.21.15.16:0/1562435952 wait complete. 2023-09-30T22:36:20.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.802+0000 7f89afd4b700 1 -- 172.21.15.16:0/461809486 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f89a8074ea0 con 0x7f89a809e930 2023-09-30T22:36:20.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.802+0000 7f899e7fc700 1 -- 172.21.15.16:0/461809486 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f89a402b020 con 0x7f89a809e930 2023-09-30T22:36:20.802 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:36:20.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.804+0000 7f89afd4b700 1 -- 172.21.15.16:0/461809486 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f89a809e930 msgr2=0x7f89a81a3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:20.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.804+0000 7f89afd4b700 1 --2- 172.21.15.16:0/461809486 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f89a809e930 0x7f89a81a3c30 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f89a4016000 tx=0x7f89a4004750 comp rx=0 tx=0).stop 2023-09-30T22:36:20.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.804+0000 7f89afd4b700 1 -- 172.21.15.16:0/461809486 shutdown_connections 2023-09-30T22:36:20.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.804+0000 7f89afd4b700 1 --2- 172.21.15.16:0/461809486 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f89a809e930 0x7f89a81a3c30 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:20.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.804+0000 7f89afd4b700 1 --2- 172.21.15.16:0/461809486 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f89a809f650 0x7f89a81a46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:20.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.804+0000 7f89afd4b700 1 --2- 172.21.15.16:0/461809486 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f89a809f240 0x7f89a81a4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:20.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.804+0000 7f89afd4b700 1 -- 172.21.15.16:0/461809486 >> 172.21.15.16:0/461809486 conn(0x7f89a80a4210 msgr2=0x7f89a81b8c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:20.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.804+0000 7f89afd4b700 1 -- 172.21.15.16:0/461809486 shutdown_connections 2023-09-30T22:36:20.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:20.804+0000 7f89afd4b700 1 -- 172.21.15.16:0/461809486 wait complete. 2023-09-30T22:36:20.807 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:36:20.818 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:36:21.696 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:36:21.763 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:36:21.808 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:36:21.819 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:36:21.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.846+0000 7f59d638a700 1 Processor -- start 2023-09-30T22:36:21.847 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.846+0000 7f59d638a700 1 -- start start 2023-09-30T22:36:21.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.847+0000 7f59d638a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f59d0130540 0x7f59d0130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:21.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.847+0000 7f59d638a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f59d0130e50 0x7f59d006f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:21.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.847+0000 7f59d638a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59d00716a0 con 0x7f59d006fb20 2023-09-30T22:36:21.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.847+0000 7f59d638a700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59d009a390 con 0x7f59d0130e50 2023-09-30T22:36:21.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.847+0000 7f59d638a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59d0071290 con 0x7f59d0130540 2023-09-30T22:36:21.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.847+0000 7f59cffff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f59d0130540 0x7f59d0130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:21.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.847+0000 7f59d4927700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f59d006fb20 0x7f59d0137560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60212/0 (socket says 172.21.15.16:60212) 2023-09-30T22:36:21.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.847+0000 7f59d4927700 1 -- 172.21.15.16:0/244818396 learned_addr learned my addr 172.21.15.16:0/244818396 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:36:21.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.847+0000 7f59cf7fe700 1 --2- 172.21.15.16:0/244818396 >> v2:172.21.15.16:3300/0 conn(0x7f59d0130e50 0x7f59d006f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:21.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.847+0000 7f59cffff700 1 -- 172.21.15.16:0/244818396 >> v2:172.21.15.16:3300/0 conn(0x7f59d0130e50 msgr2=0x7f59d006f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:21.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.847+0000 7f59cffff700 1 --2- 172.21.15.16:0/244818396 >> v2:172.21.15.16:3300/0 conn(0x7f59d0130e50 0x7f59d006f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:21.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.847+0000 7f59cffff700 1 -- 172.21.15.16:0/244818396 >> v1:172.21.15.16:6789/0 conn(0x7f59d006fb20 legacy=0x7f59d0137560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:21.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.847+0000 7f59cffff700 1 -- 172.21.15.16:0/244818396 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d0130340 con 0x7f59d0130540 2023-09-30T22:36:21.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.848+0000 7f59cffff700 1 --2- 172.21.15.16:0/244818396 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f59d0130540 0x7f59d0130910 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f59c0009870 tx=0x7f59c0009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6904c9e74eb545bd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:21.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.848+0000 7f59ceffd700 1 -- 172.21.15.16:0/244818396 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c0004230 con 0x7f59d0130540 2023-09-30T22:36:21.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.848+0000 7f59ceffd700 1 -- 172.21.15.16:0/244818396 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59c0004390 con 0x7f59d0130540 2023-09-30T22:36:21.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.848+0000 7f59ceffd700 1 -- 172.21.15.16:0/244818396 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c00036a0 con 0x7f59d0130540 2023-09-30T22:36:21.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.849+0000 7f59d638a700 1 -- 172.21.15.16:0/244818396 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f59d0130540 msgr2=0x7f59d0130910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:21.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.849+0000 7f59d638a700 1 --2- 172.21.15.16:0/244818396 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f59d0130540 0x7f59d0130910 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f59c0009870 tx=0x7f59c0009b80 comp rx=0 tx=0).stop 2023-09-30T22:36:21.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.849+0000 7f59d638a700 1 -- 172.21.15.16:0/244818396 shutdown_connections 2023-09-30T22:36:21.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.849+0000 7f59d638a700 1 --2- 172.21.15.16:0/244818396 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f59d0130540 0x7f59d0130910 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:21.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.849+0000 7f59d638a700 1 --2- 172.21.15.16:0/244818396 >> v2:172.21.15.16:3300/0 conn(0x7f59d0130e50 0x7f59d006f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:21.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.849+0000 7f59d638a700 1 -- 172.21.15.16:0/244818396 >> 172.21.15.16:0/244818396 conn(0x7f59d00a2aa0 msgr2=0x7f59d00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:21.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.850+0000 7f59d638a700 1 -- 172.21.15.16:0/244818396 shutdown_connections 2023-09-30T22:36:21.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.850+0000 7f59d638a700 1 -- 172.21.15.16:0/244818396 wait complete. 2023-09-30T22:36:21.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.851+0000 7f59d638a700 1 Processor -- start 2023-09-30T22:36:21.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.851+0000 7f59d638a700 1 -- start start 2023-09-30T22:36:21.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.852+0000 7f59d638a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f59d006fb20 0x7f59d0118cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:21.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.852+0000 7f59d638a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59d0130540 0x7f59d0119210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:21.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.852+0000 7f59d638a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f59d0130e50 0x7f59d0119750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:21.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.852+0000 7f59d638a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d011a2f0 con 0x7f59d0130540 2023-09-30T22:36:21.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.852+0000 7f59d638a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59d011b000 con 0x7f59d0130e50 2023-09-30T22:36:21.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.852+0000 7f59d638a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f59d011a100 con 0x7f59d006fb20 2023-09-30T22:36:21.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.852+0000 7f59cffff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f59d006fb20 0x7f59d0118cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:21.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.852+0000 7f59cffff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f59d006fb20 0x7f59d0118cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:57188/0 (socket says 172.21.15.16:57188) 2023-09-30T22:36:21.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.852+0000 7f59cffff700 1 -- 172.21.15.16:0/2552076741 learned_addr learned my addr 172.21.15.16:0/2552076741 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:21.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.853+0000 7f59cf7fe700 1 --2- 172.21.15.16:0/2552076741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59d0130540 0x7f59d0119210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:21.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.853+0000 7f59d4927700 1 --2- 172.21.15.16:0/2552076741 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f59d0130e50 0x7f59d0119750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:21.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.853+0000 7f59cffff700 1 -- 172.21.15.16:0/2552076741 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f59d0130e50 msgr2=0x7f59d0119750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:21.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.853+0000 7f59cffff700 1 --2- 172.21.15.16:0/2552076741 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f59d0130e50 0x7f59d0119750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:21.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.853+0000 7f59cffff700 1 -- 172.21.15.16:0/2552076741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59d0130540 msgr2=0x7f59d0119210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:21.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.853+0000 7f59cffff700 1 --2- 172.21.15.16:0/2552076741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59d0130540 0x7f59d0119210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:21.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.853+0000 7f59cffff700 1 -- 172.21.15.16:0/2552076741 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59c00094a0 con 0x7f59d006fb20 2023-09-30T22:36:21.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.853+0000 7f59d4927700 1 --2- 172.21.15.16:0/2552076741 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f59d0130e50 0x7f59d0119750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:36:21.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.854+0000 7f59cffff700 1 --2- 172.21.15.16:0/2552076741 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f59d006fb20 0x7f59d0118cd0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f59d00713d0 tx=0x7f59c0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:21.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.854+0000 7f59ccff9700 1 -- 172.21.15.16:0/2552076741 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c0011630 con 0x7f59d006fb20 2023-09-30T22:36:21.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.854+0000 7f59ccff9700 1 -- 172.21.15.16:0/2552076741 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59c001e070 con 0x7f59d006fb20 2023-09-30T22:36:21.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.854+0000 7f59ccff9700 1 -- 172.21.15.16:0/2552076741 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c00119d0 con 0x7f59d006fb20 2023-09-30T22:36:21.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.854+0000 7f59d638a700 1 -- 172.21.15.16:0/2552076741 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59d00a1ea0 con 0x7f59d006fb20 2023-09-30T22:36:21.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.854+0000 7f59d638a700 1 -- 172.21.15.16:0/2552076741 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59d00718e0 con 0x7f59d006fb20 2023-09-30T22:36:21.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.855+0000 7f59ccff9700 1 -- 172.21.15.16:0/2552076741 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f59c0011b30 con 0x7f59d006fb20 2023-09-30T22:36:21.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.855+0000 7f59ccff9700 1 -- 172.21.15.16:0/2552076741 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f59c0040780 con 0x7f59d006fb20 2023-09-30T22:36:21.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.855+0000 7f59d638a700 1 -- 172.21.15.16:0/2552076741 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59d011a960 con 0x7f59d006fb20 2023-09-30T22:36:21.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.861+0000 7f59ccff9700 1 -- 172.21.15.16:0/2552076741 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f59c001a430 con 0x7f59d006fb20 2023-09-30T22:36:21.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.910+0000 7fe215e25700 1 Processor -- start 2023-09-30T22:36:21.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.910+0000 7fe215e25700 1 -- start start 2023-09-30T22:36:21.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.911+0000 7fe215e25700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe21006e570 0x7fe21006e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:21.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.911+0000 7fe215e25700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe21006ee80 0x7fe210134d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:21.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.911+0000 7fe215e25700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe210070ee0 con 0x7fe21009eca0 2023-09-30T22:36:21.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.911+0000 7fe215e25700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe210071480 con 0x7fe21006e570 2023-09-30T22:36:21.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.911+0000 7fe215e25700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe210071b90 con 0x7fe21006ee80 2023-09-30T22:36:21.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.911+0000 7fe20ffff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe21006ee80 0x7fe210134d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:21.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.911+0000 7fe20effd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe21009eca0 0x7fe210134610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60218/0 (socket says 172.21.15.16:60218) 2023-09-30T22:36:21.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.911+0000 7fe20effd700 1 -- 172.21.15.16:0/4142590934 learned_addr learned my addr 172.21.15.16:0/4142590934 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:36:21.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.911+0000 7fe20f7fe700 1 --2- 172.21.15.16:0/4142590934 >> v2:172.21.15.16:3300/0 conn(0x7fe21006e570 0x7fe21006e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:21.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.911+0000 7fe20e7fc700 1 -- 172.21.15.16:0/4142590934 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1215371886 0 0) 0x7fe210070ee0 con 0x7fe21009eca0 2023-09-30T22:36:21.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.911+0000 7fe20e7fc700 1 -- 172.21.15.16:0/4142590934 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1f80035b0 con 0x7fe21009eca0 2023-09-30T22:36:21.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.911+0000 7fe20ffff700 1 -- 172.21.15.16:0/4142590934 >> v2:172.21.15.16:3300/0 conn(0x7fe21006e570 msgr2=0x7fe21006e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:21.915 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.911+0000 7fe20ffff700 1 --2- 172.21.15.16:0/4142590934 >> v2:172.21.15.16:3300/0 conn(0x7fe21006e570 0x7fe21006e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:21.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.911+0000 7fe20ffff700 1 -- 172.21.15.16:0/4142590934 >> v1:172.21.15.16:6789/0 conn(0x7fe21009eca0 legacy=0x7fe210134610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:21.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.912+0000 7fe20ffff700 1 -- 172.21.15.16:0/4142590934 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2100711a0 con 0x7fe21006ee80 2023-09-30T22:36:21.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.912+0000 7fe20ffff700 1 --2- 172.21.15.16:0/4142590934 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe21006ee80 0x7fe210134d20 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fe208009d90 tx=0x7fe20800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16cfc6f3eadeb795 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:21.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.912+0000 7fe20e7fc700 1 -- 172.21.15.16:0/4142590934 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20800e070 con 0x7fe21006ee80 2023-09-30T22:36:21.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.912+0000 7fe20e7fc700 1 -- 172.21.15.16:0/4142590934 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe20800bb20 con 0x7fe21006ee80 2023-09-30T22:36:21.916 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.912+0000 7fe20e7fc700 1 -- 172.21.15.16:0/4142590934 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe208012430 con 0x7fe21006ee80 2023-09-30T22:36:21.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.913+0000 7fe215e25700 1 -- 172.21.15.16:0/4142590934 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe21006ee80 msgr2=0x7fe210134d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:21.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.913+0000 7fe215e25700 1 --2- 172.21.15.16:0/4142590934 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe21006ee80 0x7fe210134d20 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fe208009d90 tx=0x7fe20800b330 comp rx=0 tx=0).stop 2023-09-30T22:36:21.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.913+0000 7fe215e25700 1 -- 172.21.15.16:0/4142590934 shutdown_connections 2023-09-30T22:36:21.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.913+0000 7fe215e25700 1 --2- 172.21.15.16:0/4142590934 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe21006ee80 0x7fe210134d20 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:21.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.913+0000 7fe215e25700 1 --2- 172.21.15.16:0/4142590934 >> v2:172.21.15.16:3300/0 conn(0x7fe21006e570 0x7fe21006e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:21.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.913+0000 7fe215e25700 1 -- 172.21.15.16:0/4142590934 >> 172.21.15.16:0/4142590934 conn(0x7fe2100a2aa0 msgr2=0x7fe2100a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:21.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.913+0000 7fe215e25700 1 -- 172.21.15.16:0/4142590934 shutdown_connections 2023-09-30T22:36:21.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.913+0000 7fe215e25700 1 -- 172.21.15.16:0/4142590934 wait complete. 2023-09-30T22:36:21.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.914+0000 7fe215e25700 1 Processor -- start 2023-09-30T22:36:21.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.914+0000 7fe215e25700 1 -- start start 2023-09-30T22:36:21.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.915+0000 7fe215e25700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe21006e570 0x7fe21013b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:21.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.915+0000 7fe215e25700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe21006ee80 0x7fe2101348b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:21.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.915+0000 7fe215e25700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe210134df0 0x7fe210139e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:21.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.915+0000 7fe215e25700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21013d0c0 con 0x7fe210134df0 2023-09-30T22:36:21.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.915+0000 7fe215e25700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe21013d380 con 0x7fe21006ee80 2023-09-30T22:36:21.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.915+0000 7fe215e25700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe21013ddf0 con 0x7fe21006e570 2023-09-30T22:36:21.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.915+0000 7fe20ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe210134df0 0x7fe210139e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:21.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.915+0000 7fe20ffff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe210134df0 0x7fe210139e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42060/0 (socket says 172.21.15.16:42060) 2023-09-30T22:36:21.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.915+0000 7fe20ffff700 1 -- 172.21.15.16:0/2829542657 learned_addr learned my addr 172.21.15.16:0/2829542657 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:21.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.915+0000 7fe20effd700 1 --2- 172.21.15.16:0/2829542657 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe21006ee80 0x7fe2101348b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:21.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.916+0000 7fe20f7fe700 1 --2- 172.21.15.16:0/2829542657 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe21006e570 0x7fe21013b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:21.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.916+0000 7fe20ffff700 1 -- 172.21.15.16:0/2829542657 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe21006e570 msgr2=0x7fe21013b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:21.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.916+0000 7fe20ffff700 1 --2- 172.21.15.16:0/2829542657 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe21006e570 0x7fe21013b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:21.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.916+0000 7fe20ffff700 1 -- 172.21.15.16:0/2829542657 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe21006ee80 msgr2=0x7fe2101348b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:21.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.916+0000 7fe20ffff700 1 --2- 172.21.15.16:0/2829542657 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe21006ee80 0x7fe2101348b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:21.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.916+0000 7fe20ffff700 1 -- 172.21.15.16:0/2829542657 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe208009a70 con 0x7fe210134df0 2023-09-30T22:36:21.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.916+0000 7fe20ffff700 1 --2- 172.21.15.16:0/2829542657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe210134df0 0x7fe210139e80 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fe208014040 tx=0x7fe208004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:21.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.916+0000 7fe1f7fff700 1 -- 172.21.15.16:0/2829542657 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2080129f0 con 0x7fe210134df0 2023-09-30T22:36:21.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.916+0000 7fe1f7fff700 1 -- 172.21.15.16:0/2829542657 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe208020070 con 0x7fe210134df0 2023-09-30T22:36:21.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.916+0000 7fe215e25700 1 -- 172.21.15.16:0/2829542657 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe21013d660 con 0x7fe210134df0 2023-09-30T22:36:21.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.916+0000 7fe1f7fff700 1 -- 172.21.15.16:0/2829542657 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe208012db0 con 0x7fe210134df0 2023-09-30T22:36:21.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.916+0000 7fe215e25700 1 -- 172.21.15.16:0/2829542657 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe21013a5a0 con 0x7fe210134df0 2023-09-30T22:36:21.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.917+0000 7fe1f7fff700 1 -- 172.21.15.16:0/2829542657 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe208004040 con 0x7fe210134df0 2023-09-30T22:36:21.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.917+0000 7fe215e25700 1 -- 172.21.15.16:0/2829542657 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe210074ea0 con 0x7fe210134df0 2023-09-30T22:36:21.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.918+0000 7fe1f7fff700 1 -- 172.21.15.16:0/2829542657 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fe208022070 con 0x7fe210134df0 2023-09-30T22:36:21.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.927+0000 7fe1f7fff700 1 -- 172.21.15.16:0/2829542657 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe20802c080 con 0x7fe210134df0 2023-09-30T22:36:21.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.993+0000 7f28408ae700 1 Processor -- start 2023-09-30T22:36:21.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.993+0000 7f28408ae700 1 -- start start 2023-09-30T22:36:21.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.994+0000 7fc0c828b700 1 Processor -- start 2023-09-30T22:36:21.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.994+0000 7fc0c828b700 1 -- start start 2023-09-30T22:36:21.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.994+0000 7f28408ae700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f283c09e1d0 0x7f283c135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:21.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.994+0000 7f28408ae700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f283c135e50 0x7f283c138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:21.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.994+0000 7f28408ae700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f283c09f130 con 0x7f283c09d6f0 2023-09-30T22:36:21.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.994+0000 7f28408ae700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f283c09f520 con 0x7f283c09e1d0 2023-09-30T22:36:21.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.994+0000 7f28408ae700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f283c09e930 con 0x7f283c135e50 2023-09-30T22:36:21.996 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.994+0000 7f283ad9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f283c135e50 0x7f283c138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:21.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.994+0000 7f283ad9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f283c135e50 0x7f283c138230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:57204/0 (socket says 172.21.15.16:57204) 2023-09-30T22:36:21.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.994+0000 7f283ad9d700 1 -- 172.21.15.16:0/697108045 learned_addr learned my addr 172.21.15.16:0/697108045 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:21.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.994+0000 7f2839d9b700 1 --2- 172.21.15.16:0/697108045 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f283c09e1d0 0x7f283c135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:21.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.995+0000 7fc0c828b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc0c009e930 0x7fc0c009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:21.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.995+0000 7fc0c828b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc0c009f650 0x7fc0c0131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:21.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.995+0000 7fc0c828b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0c0070940 con 0x7fc0c009f240 2023-09-30T22:36:21.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.995+0000 7fc0c828b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c006fd60 con 0x7fc0c009e930 2023-09-30T22:36:21.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.995+0000 7fc0c828b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c006fa80 con 0x7fc0c009f650 2023-09-30T22:36:21.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.995+0000 7f2839d9b700 1 -- 172.21.15.16:0/697108045 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f283c135e50 msgr2=0x7f283c138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:21.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.995+0000 7f2839d9b700 1 --2- 172.21.15.16:0/697108045 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f283c135e50 0x7f283c138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:21.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.995+0000 7f2839d9b700 1 -- 172.21.15.16:0/697108045 >> v1:172.21.15.16:6789/0 conn(0x7f283c09d6f0 legacy=0x7f283c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:21.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.995+0000 7f2839d9b700 1 -- 172.21.15.16:0/697108045 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f283c09b2d0 con 0x7f283c09e1d0 2023-09-30T22:36:21.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.995+0000 7f283ad9d700 1 --2- 172.21.15.16:0/697108045 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f283c135e50 0x7f283c138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:22.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.995+0000 7fc0c6828700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc0c009f650 0x7fc0c0131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:22.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.995+0000 7fc0c6828700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc0c009f650 0x7fc0c0131f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:57216/0 (socket says 172.21.15.16:57216) 2023-09-30T22:36:22.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.995+0000 7fc0c6828700 1 -- 172.21.15.16:0/182747745 learned_addr learned my addr 172.21.15.16:0/182747745 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:22.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.995+0000 7fc0c6027700 1 --2- 172.21.15.16:0/182747745 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc0c009e930 0x7fc0c009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:22.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.995+0000 7f2839d9b700 1 --2- 172.21.15.16:0/697108045 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f283c09e1d0 0x7f283c135910 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f2830009870 tx=0x7f2830009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44c01f622f998572 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:22.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.995+0000 7fc0c6828700 1 -- 172.21.15.16:0/182747745 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc0c009e930 msgr2=0x7fc0c009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:22.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.995+0000 7fc0c6828700 1 --2- 172.21.15.16:0/182747745 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc0c009e930 0x7fc0c009ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:22.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.995+0000 7fc0c6828700 1 -- 172.21.15.16:0/182747745 >> v1:172.21.15.16:6789/0 conn(0x7fc0c009f240 legacy=0x7fc0c01317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:22.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.995+0000 7fc0c6828700 1 -- 172.21.15.16:0/182747745 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c009fb60 con 0x7fc0c009f650 2023-09-30T22:36:22.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.996+0000 7fc0c6828700 1 --2- 172.21.15.16:0/182747745 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc0c009f650 0x7fc0c0131f00 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8009e80 tx=0x7fc0b800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c41327e47b0152c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:22.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.996+0000 7f283959a700 1 -- 172.21.15.16:0/697108045 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283000f6a0 con 0x7f283c09e1d0 2023-09-30T22:36:22.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.996+0000 7f283959a700 1 -- 172.21.15.16:0/697108045 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f283000f800 con 0x7f283c09e1d0 2023-09-30T22:36:22.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.996+0000 7f283959a700 1 -- 172.21.15.16:0/697108045 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283000fb90 con 0x7f283c09e1d0 2023-09-30T22:36:22.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.997+0000 7f28408ae700 1 -- 172.21.15.16:0/697108045 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f283c09e1d0 msgr2=0x7f283c135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:22.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.997+0000 7f28408ae700 1 --2- 172.21.15.16:0/697108045 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f283c09e1d0 0x7f283c135910 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f2830009870 tx=0x7f2830009b80 comp rx=0 tx=0).stop 2023-09-30T22:36:22.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.997+0000 7f28408ae700 1 -- 172.21.15.16:0/697108045 shutdown_connections 2023-09-30T22:36:22.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.997+0000 7f28408ae700 1 --2- 172.21.15.16:0/697108045 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f283c135e50 0x7f283c138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:22.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.997+0000 7f28408ae700 1 --2- 172.21.15.16:0/697108045 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f283c09e1d0 0x7f283c135910 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:22.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.997+0000 7f28408ae700 1 -- 172.21.15.16:0/697108045 >> 172.21.15.16:0/697108045 conn(0x7f283c066f50 msgr2=0x7f283c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:22.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.997+0000 7fc0c5025700 1 -- 172.21.15.16:0/182747745 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b800e070 con 0x7fc0c009f650 2023-09-30T22:36:22.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.997+0000 7f28408ae700 1 -- 172.21.15.16:0/697108045 shutdown_connections 2023-09-30T22:36:22.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.997+0000 7fc0c5025700 1 -- 172.21.15.16:0/182747745 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0b800b9c0 con 0x7fc0c009f650 2023-09-30T22:36:22.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.997+0000 7fc0c5025700 1 -- 172.21.15.16:0/182747745 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b800bd70 con 0x7fc0c009f650 2023-09-30T22:36:22.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.997+0000 7f28408ae700 1 -- 172.21.15.16:0/697108045 wait complete. 2023-09-30T22:36:22.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.998+0000 7fc0c828b700 1 -- 172.21.15.16:0/182747745 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc0c009f650 msgr2=0x7fc0c0131f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:22.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.998+0000 7fc0c828b700 1 --2- 172.21.15.16:0/182747745 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc0c009f650 0x7fc0c0131f00 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8009e80 tx=0x7fc0b800b330 comp rx=0 tx=0).stop 2023-09-30T22:36:22.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.998+0000 7fc0c828b700 1 -- 172.21.15.16:0/182747745 shutdown_connections 2023-09-30T22:36:22.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.998+0000 7fc0c828b700 1 --2- 172.21.15.16:0/182747745 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc0c009f650 0x7fc0c0131f00 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:22.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.998+0000 7fc0c828b700 1 --2- 172.21.15.16:0/182747745 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc0c009e930 0x7fc0c009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:22.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.998+0000 7fc0c828b700 1 -- 172.21.15.16:0/182747745 >> 172.21.15.16:0/182747745 conn(0x7fc0c00a4210 msgr2=0x7fc0c00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:22.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.998+0000 7fc0c828b700 1 -- 172.21.15.16:0/182747745 shutdown_connections 2023-09-30T22:36:22.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.998+0000 7fc0c828b700 1 -- 172.21.15.16:0/182747745 wait complete. 2023-09-30T22:36:22.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.999+0000 7f28408ae700 1 Processor -- start 2023-09-30T22:36:22.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.999+0000 7f28408ae700 1 -- start start 2023-09-30T22:36:22.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.999+0000 7fc0c828b700 1 Processor -- start 2023-09-30T22:36:22.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.999+0000 7fc0c828b700 1 -- start start 2023-09-30T22:36:22.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.999+0000 7fc0c828b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc0c009e930 0x7fc0c01c5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:22.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.999+0000 7fc0c828b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0c009f240 0x7fc0c01c5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:22.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.999+0000 7fc0c828b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc0c009f650 0x7fc0c01c3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:22.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.999+0000 7fc0c828b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c0071d20 con 0x7fc0c009f240 2023-09-30T22:36:22.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.999+0000 7fc0c828b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c012f7d0 con 0x7fc0c009f650 2023-09-30T22:36:22.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.999+0000 7fc0c828b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c01303d0 con 0x7fc0c009e930 2023-09-30T22:36:22.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.999+0000 7f28408ae700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f283c09d6f0 0x7f283c119720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:22.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.999+0000 7f28408ae700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f283c09e1d0 0x7f283c119c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:22.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.999+0000 7f28408ae700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f283c135e50 0x7f283c11a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:22.012 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.999+0000 7f28408ae700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f283c11c7d0 con 0x7f283c09e1d0 2023-09-30T22:36:22.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.999+0000 7f28408ae700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f283c11b1f0 con 0x7f283c135e50 2023-09-30T22:36:22.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.999+0000 7f28408ae700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f283c11ca50 con 0x7f283c09d6f0 2023-09-30T22:36:22.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:21.999+0000 7fc0c6027700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc0c009e930 0x7fc0c01c5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:22.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.000+0000 7fc0c6027700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc0c009e930 0x7fc0c01c5220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:57226/0 (socket says 172.21.15.16:57226) 2023-09-30T22:36:22.013 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.000+0000 7fc0c6027700 1 -- 172.21.15.16:0/493771191 learned_addr learned my addr 172.21.15.16:0/493771191 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:22.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.000+0000 7f2839d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f283c09e1d0 0x7f283c119c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:22.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.000+0000 7f2839d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f283c09e1d0 0x7f283c119c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42070/0 (socket says 172.21.15.16:42070) 2023-09-30T22:36:22.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.000+0000 7f2839d9b700 1 -- 172.21.15.16:0/2413998261 learned_addr learned my addr 172.21.15.16:0/2413998261 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:22.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.000+0000 7f283a59c700 1 --2- 172.21.15.16:0/2413998261 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f283c09d6f0 0x7f283c119720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:22.014 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.000+0000 7f2839d9b700 1 -- 172.21.15.16:0/2413998261 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f283c09d6f0 msgr2=0x7f283c119720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:22.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.000+0000 7f2839d9b700 1 --2- 172.21.15.16:0/2413998261 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f283c09d6f0 0x7f283c119720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:22.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.000+0000 7f2839d9b700 1 -- 172.21.15.16:0/2413998261 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f283c135e50 msgr2=0x7f283c11a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:22.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.000+0000 7f2839d9b700 1 --2- 172.21.15.16:0/2413998261 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f283c135e50 0x7f283c11a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:22.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.000+0000 7f2839d9b700 1 -- 172.21.15.16:0/2413998261 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2834009b30 con 0x7f283c09e1d0 2023-09-30T22:36:22.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.000+0000 7fc0c6027700 1 -- 172.21.15.16:0/493771191 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc0c009f650 msgr2=0x7fc0c01c3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:22.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.000+0000 7fc0c6027700 1 --2- 172.21.15.16:0/493771191 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc0c009f650 0x7fc0c01c3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:22.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.000+0000 7fc0c6027700 1 -- 172.21.15.16:0/493771191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0c009f240 msgr2=0x7fc0c01c5760 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:36:22.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.000+0000 7fc0c6027700 1 --2- 172.21.15.16:0/493771191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0c009f240 0x7fc0c01c5760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:22.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.000+0000 7fc0c6027700 1 -- 172.21.15.16:0/493771191 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0b8009b30 con 0x7fc0c009e930 2023-09-30T22:36:22.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.000+0000 7fc0c6027700 1 --2- 172.21.15.16:0/493771191 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc0c009e930 0x7fc0c01c5220 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc00e810 tx=0x7fc0bc00ebd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:22.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.000+0000 7f2839d9b700 1 --2- 172.21.15.16:0/2413998261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f283c09e1d0 0x7f283c119c60 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f28300090b0 tx=0x7f2830004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:22.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.000+0000 7fc0b6ffd700 1 -- 172.21.15.16:0/493771191 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc013610 con 0x7fc0c009e930 2023-09-30T22:36:22.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.001+0000 7fc0c828b700 1 -- 172.21.15.16:0/493771191 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0c00a0190 con 0x7fc0c009e930 2023-09-30T22:36:22.017 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.001+0000 7f282affd700 1 -- 172.21.15.16:0/2413998261 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283001e070 con 0x7f283c09e1d0 2023-09-30T22:36:22.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.001+0000 7fc0c828b700 1 -- 172.21.15.16:0/493771191 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0c012e150 con 0x7fc0c009e930 2023-09-30T22:36:22.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.001+0000 7fc0b6ffd700 1 -- 172.21.15.16:0/493771191 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0bc013770 con 0x7fc0c009e930 2023-09-30T22:36:22.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.001+0000 7fc0b6ffd700 1 -- 172.21.15.16:0/493771191 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc013ae0 con 0x7fc0c009e930 2023-09-30T22:36:22.018 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.002+0000 7fc0c828b700 1 -- 172.21.15.16:0/493771191 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0a80052e0 con 0x7fc0c009e930 2023-09-30T22:36:22.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.001+0000 7f282affd700 1 -- 172.21.15.16:0/2413998261 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f283000fe30 con 0x7f283c09e1d0 2023-09-30T22:36:22.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.001+0000 7f282affd700 1 -- 172.21.15.16:0/2413998261 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283000f6a0 con 0x7f283c09e1d0 2023-09-30T22:36:22.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.003+0000 7f28408ae700 1 -- 172.21.15.16:0/2413998261 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28300094a0 con 0x7f283c09e1d0 2023-09-30T22:36:22.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.003+0000 7f28408ae700 1 -- 172.21.15.16:0/2413998261 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f283c09c760 con 0x7f283c09e1d0 2023-09-30T22:36:22.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.003+0000 7fc0b6ffd700 1 -- 172.21.15.16:0/493771191 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc0bc013c40 con 0x7fc0c009e930 2023-09-30T22:36:22.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.003+0000 7fc0b6ffd700 1 -- 172.21.15.16:0/493771191 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fc0bc03f910 con 0x7fc0c009e930 2023-09-30T22:36:22.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.004+0000 7f282affd700 1 -- 172.21.15.16:0/2413998261 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2830022070 con 0x7f283c09e1d0 2023-09-30T22:36:22.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.004+0000 7f28408ae700 1 -- 172.21.15.16:0/2413998261 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f283c074ea0 con 0x7f283c09e1d0 2023-09-30T22:36:22.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.004+0000 7f282affd700 1 -- 172.21.15.16:0/2413998261 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f2830019e40 con 0x7f283c09e1d0 2023-09-30T22:36:22.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.009+0000 7f282affd700 1 -- 172.21.15.16:0/2413998261 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f283002c770 con 0x7f283c09e1d0 2023-09-30T22:36:22.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.011+0000 7fc0b6ffd700 1 -- 172.21.15.16:0/493771191 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc0bc01c7d0 con 0x7fc0c009e930 2023-09-30T22:36:22.143 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.142+0000 7f59d638a700 1 -- 172.21.15.16:0/2552076741 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f59d011ba40 con 0x7f59d006fb20 2023-09-30T22:36:22.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.143+0000 7f59ccff9700 1 -- 172.21.15.16:0/2552076741 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f59d011ba40 con 0x7f59d006fb20 2023-09-30T22:36:22.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.143+0000 7fe215e25700 1 -- 172.21.15.16:0/2829542657 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe21006b9e0 con 0x7fe210134df0 2023-09-30T22:36:22.144 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:36:22.145 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.143+0000 7fe1f7fff700 1 -- 172.21.15.16:0/2829542657 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe20801a970 con 0x7fe210134df0 2023-09-30T22:36:22.145 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:36:22.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.146+0000 7f59d638a700 1 -- 172.21.15.16:0/2552076741 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f59d006fb20 msgr2=0x7f59d0118cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:22.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.146+0000 7f59d638a700 1 --2- 172.21.15.16:0/2552076741 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f59d006fb20 0x7f59d0118cd0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f59d00713d0 tx=0x7f59c0004750 comp rx=0 tx=0).stop 2023-09-30T22:36:22.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.146+0000 7f59d638a700 1 -- 172.21.15.16:0/2552076741 shutdown_connections 2023-09-30T22:36:22.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.146+0000 7f59d638a700 1 --2- 172.21.15.16:0/2552076741 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f59d0130e50 0x7f59d0119750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:22.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.146+0000 7f59d638a700 1 --2- 172.21.15.16:0/2552076741 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f59d006fb20 0x7f59d0118cd0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:22.148 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.147+0000 7f59d638a700 1 --2- 172.21.15.16:0/2552076741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59d0130540 0x7f59d0119210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:22.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.147+0000 7f59d638a700 1 -- 172.21.15.16:0/2552076741 >> 172.21.15.16:0/2552076741 conn(0x7f59d00a2aa0 msgr2=0x7f59d012d550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:22.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.147+0000 7f59d638a700 1 -- 172.21.15.16:0/2552076741 shutdown_connections 2023-09-30T22:36:22.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.147+0000 7fe215e25700 1 -- 172.21.15.16:0/2829542657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe210134df0 msgr2=0x7fe210139e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:22.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.147+0000 7fe215e25700 1 --2- 172.21.15.16:0/2829542657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe210134df0 0x7fe210139e80 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fe208014040 tx=0x7fe208004ad0 comp rx=0 tx=0).stop 2023-09-30T22:36:22.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.147+0000 7f59d638a700 1 -- 172.21.15.16:0/2552076741 wait complete. 2023-09-30T22:36:22.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.148+0000 7fe215e25700 1 -- 172.21.15.16:0/2829542657 shutdown_connections 2023-09-30T22:36:22.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.148+0000 7fe215e25700 1 --2- 172.21.15.16:0/2829542657 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe210134df0 0x7fe210139e80 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:22.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.148+0000 7fe215e25700 1 --2- 172.21.15.16:0/2829542657 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe21006e570 0x7fe21013b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:22.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.148+0000 7fe215e25700 1 --2- 172.21.15.16:0/2829542657 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe21006ee80 0x7fe2101348b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:22.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.148+0000 7fe215e25700 1 -- 172.21.15.16:0/2829542657 >> 172.21.15.16:0/2829542657 conn(0x7fe2100a2aa0 msgr2=0x7fe2100a1410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:22.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.148+0000 7fe215e25700 1 -- 172.21.15.16:0/2829542657 shutdown_connections 2023-09-30T22:36:22.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.148+0000 7fe215e25700 1 -- 172.21.15.16:0/2829542657 wait complete. 2023-09-30T22:36:22.168 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:36:22.168 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:36:22.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.216+0000 7f28408ae700 1 -- 172.21.15.16:0/2413998261 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f283c06c360 con 0x7f283c09e1d0 2023-09-30T22:36:22.216 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.216+0000 7f282affd700 1 -- 172.21.15.16:0/2413998261 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2830014020 con 0x7f283c09e1d0 2023-09-30T22:36:22.217 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:36:22.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.218+0000 7f28408ae700 1 -- 172.21.15.16:0/2413998261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f283c09e1d0 msgr2=0x7f283c119c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:22.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.218+0000 7f28408ae700 1 --2- 172.21.15.16:0/2413998261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f283c09e1d0 0x7f283c119c60 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f28300090b0 tx=0x7f2830004750 comp rx=0 tx=0).stop 2023-09-30T22:36:22.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.218+0000 7f28408ae700 1 -- 172.21.15.16:0/2413998261 shutdown_connections 2023-09-30T22:36:22.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.218+0000 7f28408ae700 1 --2- 172.21.15.16:0/2413998261 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f283c135e50 0x7f283c11a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:22.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.218+0000 7f28408ae700 1 --2- 172.21.15.16:0/2413998261 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f283c09d6f0 0x7f283c119720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:22.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.218+0000 7f28408ae700 1 --2- 172.21.15.16:0/2413998261 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f283c09e1d0 0x7f283c119c60 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:22.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.218+0000 7f28408ae700 1 -- 172.21.15.16:0/2413998261 >> 172.21.15.16:0/2413998261 conn(0x7f283c066f50 msgr2=0x7f283c1329b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:22.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.218+0000 7f28408ae700 1 -- 172.21.15.16:0/2413998261 shutdown_connections 2023-09-30T22:36:22.219 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.218+0000 7f28408ae700 1 -- 172.21.15.16:0/2413998261 wait complete. 2023-09-30T22:36:22.232 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:36:22.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.237+0000 7fc0c828b700 1 -- 172.21.15.16:0/493771191 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc0a8005860 con 0x7fc0c009e930 2023-09-30T22:36:22.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.237+0000 7fc0b6ffd700 1 -- 172.21.15.16:0/493771191 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc0bc02a020 con 0x7fc0c009e930 2023-09-30T22:36:22.238 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:36:22.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.240+0000 7fc0c828b700 1 -- 172.21.15.16:0/493771191 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc0c009e930 msgr2=0x7fc0c01c5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:22.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.240+0000 7fc0c828b700 1 --2- 172.21.15.16:0/493771191 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc0c009e930 0x7fc0c01c5220 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc00e810 tx=0x7fc0bc00ebd0 comp rx=0 tx=0).stop 2023-09-30T22:36:22.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.240+0000 7fc0c828b700 1 -- 172.21.15.16:0/493771191 shutdown_connections 2023-09-30T22:36:22.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.240+0000 7fc0c828b700 1 --2- 172.21.15.16:0/493771191 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc0c009f650 0x7fc0c01c3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:22.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.240+0000 7fc0c828b700 1 --2- 172.21.15.16:0/493771191 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc0c009e930 0x7fc0c01c5220 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:22.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.240+0000 7fc0c828b700 1 --2- 172.21.15.16:0/493771191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc0c009f240 0x7fc0c01c5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:22.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.240+0000 7fc0c828b700 1 -- 172.21.15.16:0/493771191 >> 172.21.15.16:0/493771191 conn(0x7fc0c00a4210 msgr2=0x7fc0c006c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:22.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.240+0000 7fc0c828b700 1 -- 172.21.15.16:0/493771191 shutdown_connections 2023-09-30T22:36:22.241 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:22.240+0000 7fc0c828b700 1 -- 172.21.15.16:0/493771191 wait complete. 2023-09-30T22:36:22.253 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:36:23.169 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:36:23.169 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:36:23.233 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:36:23.255 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:36:23.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.319+0000 7fdeff479700 1 Processor -- start 2023-09-30T22:36:23.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.319+0000 7fdeff479700 1 -- start start 2023-09-30T22:36:23.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.320+0000 7fdeff479700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdef806b250 0x7fdef8136840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:23.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.320+0000 7fdeff479700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdef8136d80 0x7fdef8139160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:23.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.320+0000 7fdeff479700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdef809ece0 con 0x7fdef806a770 2023-09-30T22:36:23.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.320+0000 7fdeff479700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdef806b640 con 0x7fdef8136d80 2023-09-30T22:36:23.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.320+0000 7fdeff479700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdef809e760 con 0x7fdef806b250 2023-09-30T22:36:23.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.320+0000 7fdefda16700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdef8136d80 0x7fdef8139160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:23.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.320+0000 7fdefda16700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fdef8136d80 0x7fdef8139160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42080/0 (socket says 172.21.15.16:42080) 2023-09-30T22:36:23.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.320+0000 7fdefd215700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fdef806a770 0x7fdef806ab40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60240/0 (socket says 172.21.15.16:60240) 2023-09-30T22:36:23.321 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.320+0000 7fdefda16700 1 -- 172.21.15.16:0/1419032194 learned_addr learned my addr 172.21.15.16:0/1419032194 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:23.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.320+0000 7fdefda16700 1 -- 172.21.15.16:0/1419032194 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdef806b250 msgr2=0x7fdef8136840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:23.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.320+0000 7fdefda16700 1 --2- 172.21.15.16:0/1419032194 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdef806b250 0x7fdef8136840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:23.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.320+0000 7fdefda16700 1 -- 172.21.15.16:0/1419032194 >> v1:172.21.15.16:6789/0 conn(0x7fdef806a770 legacy=0x7fdef806ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:23.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.321+0000 7fdeeffff700 1 -- 172.21.15.16:0/1419032194 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 804185681 0 0) 0x7fdef809ece0 con 0x7fdef806a770 2023-09-30T22:36:23.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.321+0000 7fdefda16700 1 -- 172.21.15.16:0/1419032194 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef809e040 con 0x7fdef8136d80 2023-09-30T22:36:23.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.321+0000 7fdefda16700 1 --2- 172.21.15.16:0/1419032194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdef8136d80 0x7fdef8139160 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fdef0009e50 tx=0x7fdef000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc5db36c85e9c4fb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:23.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.321+0000 7fdeeffff700 1 -- 172.21.15.16:0/1419032194 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef000e070 con 0x7fdef8136d80 2023-09-30T22:36:23.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.321+0000 7fdeeffff700 1 -- 172.21.15.16:0/1419032194 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef000b9c0 con 0x7fdef8136d80 2023-09-30T22:36:23.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.321+0000 7fdeeffff700 1 -- 172.21.15.16:0/1419032194 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef000bd70 con 0x7fdef8136d80 2023-09-30T22:36:23.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.321+0000 7f665886c700 1 Processor -- start 2023-09-30T22:36:23.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.321+0000 7f665886c700 1 -- start start 2023-09-30T22:36:23.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.322+0000 7fdeff479700 1 -- 172.21.15.16:0/1419032194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdef8136d80 msgr2=0x7fdef8139160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:23.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.322+0000 7fdeff479700 1 --2- 172.21.15.16:0/1419032194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdef8136d80 0x7fdef8139160 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fdef0009e50 tx=0x7fdef000b330 comp rx=0 tx=0).stop 2023-09-30T22:36:23.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.322+0000 7fdeff479700 1 -- 172.21.15.16:0/1419032194 shutdown_connections 2023-09-30T22:36:23.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.322+0000 7fdeff479700 1 --2- 172.21.15.16:0/1419032194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdef8136d80 0x7fdef8139160 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:23.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.322+0000 7fdeff479700 1 --2- 172.21.15.16:0/1419032194 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdef806b250 0x7fdef8136840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:23.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.322+0000 7fdeff479700 1 -- 172.21.15.16:0/1419032194 >> 172.21.15.16:0/1419032194 conn(0x7fdef8066f50 msgr2=0x7fdef809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:23.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.322+0000 7f665886c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f665409cb40 0x7f665406e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:23.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.322+0000 7f665886c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f665409cf10 0x7f665409d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:23.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.322+0000 7f665886c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f665409c6f0 con 0x7f6654131370 2023-09-30T22:36:23.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.322+0000 7f665886c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f665409e420 con 0x7f665409cf10 2023-09-30T22:36:23.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.322+0000 7f665886c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f66540723a0 con 0x7f665409cb40 2023-09-30T22:36:23.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.323+0000 7f6651d9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f665409cf10 0x7f665409d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:23.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.323+0000 7f6652d9d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6654131370 0x7f6654131810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60254/0 (socket says 172.21.15.16:60254) 2023-09-30T22:36:23.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.323+0000 7f6652d9d700 1 -- 172.21.15.16:0/3695211868 learned_addr learned my addr 172.21.15.16:0/3695211868 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:36:23.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.323+0000 7fdeff479700 1 -- 172.21.15.16:0/1419032194 shutdown_connections 2023-09-30T22:36:23.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.323+0000 7fdeff479700 1 -- 172.21.15.16:0/1419032194 wait complete. 2023-09-30T22:36:23.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.323+0000 7f665159a700 1 -- 172.21.15.16:0/3695211868 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3139479243 0 0) 0x7f665409c6f0 con 0x7f6654131370 2023-09-30T22:36:23.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.323+0000 7f6651d9b700 1 -- 172.21.15.16:0/3695211868 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f665409cb40 msgr2=0x7f665406e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:23.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.323+0000 7f6651d9b700 1 --2- 172.21.15.16:0/3695211868 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f665409cb40 0x7f665406e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:23.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.323+0000 7f6651d9b700 1 -- 172.21.15.16:0/3695211868 >> v1:172.21.15.16:6789/0 conn(0x7f6654131370 legacy=0x7f6654131810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:23.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.323+0000 7f6651d9b700 1 -- 172.21.15.16:0/3695211868 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f665406e080 con 0x7f665409cf10 2023-09-30T22:36:23.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.323+0000 7f6651d9b700 1 --2- 172.21.15.16:0/3695211868 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f665409cf10 0x7f665409d340 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f6648009870 tx=0x7f6648009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c68be4e5fa9538a2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:23.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.323+0000 7f665159a700 1 -- 172.21.15.16:0/3695211868 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6648011830 con 0x7f665409cf10 2023-09-30T22:36:23.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.323+0000 7f665159a700 1 -- 172.21.15.16:0/3695211868 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6648011990 con 0x7f665409cf10 2023-09-30T22:36:23.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.324+0000 7f665159a700 1 -- 172.21.15.16:0/3695211868 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6648011d20 con 0x7f665409cf10 2023-09-30T22:36:23.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.324+0000 7fdeff479700 1 Processor -- start 2023-09-30T22:36:23.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.324+0000 7fdeff479700 1 -- start start 2023-09-30T22:36:23.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.324+0000 7fdeff479700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdef806a770 0x7fdef8127850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:23.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.324+0000 7fdeff479700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdef806b250 0x7fdef8127d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:23.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.324+0000 7fdeff479700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdef8136d80 0x7fdef81cc430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:23.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.325+0000 7fdeff479700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef8127470 con 0x7fdef806a770 2023-09-30T22:36:23.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.325+0000 7fdeff479700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdef812a440 con 0x7fdef8136d80 2023-09-30T22:36:23.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.325+0000 7fdeff479700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdef81282d0 con 0x7fdef806b250 2023-09-30T22:36:23.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.325+0000 7fdefda16700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdef8136d80 0x7fdef81cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:23.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.325+0000 7fdefda16700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdef8136d80 0x7fdef81cc430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38360/0 (socket says 172.21.15.16:38360) 2023-09-30T22:36:23.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.325+0000 7fdefda16700 1 -- 172.21.15.16:0/1328998107 learned_addr learned my addr 172.21.15.16:0/1328998107 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:23.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.325+0000 7fdefd215700 1 --2- 172.21.15.16:0/1328998107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdef806a770 0x7fdef8127850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:23.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.325+0000 7fdefca14700 1 --2- 172.21.15.16:0/1328998107 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdef806b250 0x7fdef8127d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:23.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.325+0000 7f665886c700 1 -- 172.21.15.16:0/3695211868 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f665409cf10 msgr2=0x7f665409d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:23.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.325+0000 7f665886c700 1 --2- 172.21.15.16:0/3695211868 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f665409cf10 0x7f665409d340 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f6648009870 tx=0x7f6648009b80 comp rx=0 tx=0).stop 2023-09-30T22:36:23.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.325+0000 7f665886c700 1 -- 172.21.15.16:0/3695211868 shutdown_connections 2023-09-30T22:36:23.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.325+0000 7f665886c700 1 --2- 172.21.15.16:0/3695211868 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f665409cb40 0x7f665406e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:23.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.325+0000 7f665886c700 1 --2- 172.21.15.16:0/3695211868 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f665409cf10 0x7f665409d340 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:23.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.325+0000 7f665886c700 1 -- 172.21.15.16:0/3695211868 >> 172.21.15.16:0/3695211868 conn(0x7f665409e7e0 msgr2=0x7f665409ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:23.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.325+0000 7fdefd215700 1 -- 172.21.15.16:0/1328998107 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdef806b250 msgr2=0x7fdef8127d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:23.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.325+0000 7fdefd215700 1 --2- 172.21.15.16:0/1328998107 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdef806b250 0x7fdef8127d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:23.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.325+0000 7fdefd215700 1 -- 172.21.15.16:0/1328998107 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdef8136d80 msgr2=0x7fdef81cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:23.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.325+0000 7fdefd215700 1 --2- 172.21.15.16:0/1328998107 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdef8136d80 0x7fdef81cc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:23.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.325+0000 7fdefd215700 1 -- 172.21.15.16:0/1328998107 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef0009b30 con 0x7fdef806a770 2023-09-30T22:36:23.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.325+0000 7fdefda16700 1 --2- 172.21.15.16:0/1328998107 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdef8136d80 0x7fdef81cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:23.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.325+0000 7fdefd215700 1 --2- 172.21.15.16:0/1328998107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdef806a770 0x7fdef8127850 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fdef400c470 tx=0x7fdef400c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:23.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.325+0000 7f665886c700 1 -- 172.21.15.16:0/3695211868 shutdown_connections 2023-09-30T22:36:23.332 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.325+0000 7f665886c700 1 -- 172.21.15.16:0/3695211868 wait complete. 2023-09-30T22:36:23.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.326+0000 7fdeedffb700 1 -- 172.21.15.16:0/1328998107 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef4004c80 con 0x7fdef806a770 2023-09-30T22:36:23.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.326+0000 7fdeff479700 1 -- 172.21.15.16:0/1328998107 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdef8129be0 con 0x7fdef806a770 2023-09-30T22:36:23.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.326+0000 7fdeedffb700 1 -- 172.21.15.16:0/1328998107 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef4004de0 con 0x7fdef806a770 2023-09-30T22:36:23.333 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.326+0000 7fdeedffb700 1 -- 172.21.15.16:0/1328998107 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef4007310 con 0x7fdef806a770 2023-09-30T22:36:23.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.326+0000 7fdeff479700 1 -- 172.21.15.16:0/1328998107 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdef8129740 con 0x7fdef806a770 2023-09-30T22:36:23.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.327+0000 7fdeedffb700 1 -- 172.21.15.16:0/1328998107 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdef4007470 con 0x7fdef806a770 2023-09-30T22:36:23.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.327+0000 7fdeedffb700 1 -- 172.21.15.16:0/1328998107 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fdef4011d30 con 0x7fdef806a770 2023-09-30T22:36:23.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.327+0000 7f665886c700 1 Processor -- start 2023-09-30T22:36:23.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.327+0000 7f665886c700 1 -- start start 2023-09-30T22:36:23.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.328+0000 7f665886c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f665409cb40 0x7f6654095db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:23.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.328+0000 7f665886c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f665409cf10 0x7f66540962f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:23.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.328+0000 7f665886c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6654131370 0x7f6654096830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:23.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.328+0000 7f665886c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66540a2650 con 0x7f665409cf10 2023-09-30T22:36:23.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.328+0000 7f665886c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66540725e0 con 0x7f665409cb40 2023-09-30T22:36:23.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.328+0000 7f665886c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f66540934f0 con 0x7f6654131370 2023-09-30T22:36:23.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.328+0000 7f6651d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f665409cf10 0x7f66540962f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:23.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.328+0000 7f6651d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f665409cf10 0x7f66540962f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42106/0 (socket says 172.21.15.16:42106) 2023-09-30T22:36:23.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.328+0000 7f6651d9b700 1 -- 172.21.15.16:0/3790245852 learned_addr learned my addr 172.21.15.16:0/3790245852 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:23.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.328+0000 7f665259c700 1 --2- 172.21.15.16:0/3790245852 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f665409cb40 0x7f6654095db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:23.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.328+0000 7f6652d9d700 1 --2- 172.21.15.16:0/3790245852 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6654131370 0x7f6654096830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:23.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.328+0000 7f6651d9b700 1 -- 172.21.15.16:0/3790245852 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6654131370 msgr2=0x7f6654096830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:23.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.328+0000 7f6651d9b700 1 --2- 172.21.15.16:0/3790245852 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6654131370 0x7f6654096830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:23.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.328+0000 7f6651d9b700 1 -- 172.21.15.16:0/3790245852 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f665409cb40 msgr2=0x7f6654095db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:23.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.328+0000 7f6651d9b700 1 --2- 172.21.15.16:0/3790245852 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f665409cb40 0x7f6654095db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:23.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.328+0000 7f6651d9b700 1 -- 172.21.15.16:0/3790245852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66480094a0 con 0x7f665409cf10 2023-09-30T22:36:23.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.328+0000 7fdeff479700 1 -- 172.21.15.16:0/1328998107 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdef8074ea0 con 0x7fdef806a770 2023-09-30T22:36:23.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.328+0000 7f6651d9b700 1 --2- 172.21.15.16:0/3790245852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f665409cf10 0x7f66540962f0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f6648013000 tx=0x7f6648004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:23.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.337+0000 7f6642ffd700 1 -- 172.21.15.16:0/3790245852 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6648004510 con 0x7f665409cf10 2023-09-30T22:36:23.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.337+0000 7f6642ffd700 1 -- 172.21.15.16:0/3790245852 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f664801f070 con 0x7f665409cf10 2023-09-30T22:36:23.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.337+0000 7f665886c700 1 -- 172.21.15.16:0/3790245852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6654097ff0 con 0x7f665409cf10 2023-09-30T22:36:23.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.337+0000 7f6642ffd700 1 -- 172.21.15.16:0/3790245852 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66480038c0 con 0x7f665409cf10 2023-09-30T22:36:23.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.337+0000 7fdeedffb700 1 -- 172.21.15.16:0/1328998107 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdef4044060 con 0x7fdef806a770 2023-09-30T22:36:23.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.337+0000 7f665886c700 1 -- 172.21.15.16:0/3790245852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6654127990 con 0x7f665409cf10 2023-09-30T22:36:23.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.338+0000 7f6642ffd700 1 -- 172.21.15.16:0/3790245852 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6648003a20 con 0x7f665409cf10 2023-09-30T22:36:23.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.338+0000 7f6642ffd700 1 -- 172.21.15.16:0/3790245852 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f6648021070 con 0x7f665409cf10 2023-09-30T22:36:23.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.338+0000 7f665886c700 1 -- 172.21.15.16:0/3790245852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6654127bb0 con 0x7f665409cf10 2023-09-30T22:36:23.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.347+0000 7f6642ffd700 1 -- 172.21.15.16:0/3790245852 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6654127bb0 con 0x7f665409cf10 2023-09-30T22:36:23.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.397+0000 7fb62c0c7700 1 Processor -- start 2023-09-30T22:36:23.398 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.398+0000 7fb62c0c7700 1 -- start start 2023-09-30T22:36:23.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.398+0000 7fb62c0c7700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb6241337f0 0x7fb624138880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:23.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.398+0000 7fb62c0c7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb624138dc0 0x7fb62413b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:23.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.398+0000 7fb62c0c7700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb62406b990 con 0x7fb624133420 2023-09-30T22:36:23.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.398+0000 7fb62c0c7700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb62409b0e0 con 0x7fb6241337f0 2023-09-30T22:36:23.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.398+0000 7fb62c0c7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb624133c20 con 0x7fb624138dc0 2023-09-30T22:36:23.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.399+0000 7fb629662700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb6241337f0 0x7fb624138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:23.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.399+0000 7fb62a664700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb624138dc0 0x7fb62413b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:23.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.399+0000 7fb629662700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb6241337f0 0x7fb624138880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42110/0 (socket says 172.21.15.16:42110) 2023-09-30T22:36:23.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.399+0000 7fb629662700 1 -- 172.21.15.16:0/3701352004 learned_addr learned my addr 172.21.15.16:0/3701352004 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:23.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.399+0000 7fb629e63700 1 --1- 172.21.15.16:0/3701352004 >> v1:172.21.15.16:6789/0 conn(0x7fb624133420 0x7fb624133e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60256/0 (socket says 172.21.15.16:60256) 2023-09-30T22:36:23.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.399+0000 7fb62a664700 1 -- 172.21.15.16:0/3701352004 >> v2:172.21.15.16:3300/0 conn(0x7fb6241337f0 msgr2=0x7fb624138880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:23.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.399+0000 7fb62a664700 1 --2- 172.21.15.16:0/3701352004 >> v2:172.21.15.16:3300/0 conn(0x7fb6241337f0 0x7fb624138880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:23.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.399+0000 7fb62a664700 1 -- 172.21.15.16:0/3701352004 >> v1:172.21.15.16:6789/0 conn(0x7fb624133420 legacy=0x7fb624133e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:23.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.399+0000 7fb62a664700 1 -- 172.21.15.16:0/3701352004 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6240a0590 con 0x7fb624138dc0 2023-09-30T22:36:23.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.399+0000 7fb628e61700 1 -- 172.21.15.16:0/3701352004 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2298850600 0 0) 0x7fb62406b990 con 0x7fb624133420 2023-09-30T22:36:23.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.399+0000 7fb629662700 1 --2- 172.21.15.16:0/3701352004 >> v2:172.21.15.16:3300/0 conn(0x7fb6241337f0 0x7fb624138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:23.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.399+0000 7fb62a664700 1 --2- 172.21.15.16:0/3701352004 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb624138dc0 0x7fb62413b1a0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fb61c009d90 tx=0x7fb61c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3075ca16489ddf4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:23.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.400+0000 7fb628e61700 1 -- 172.21.15.16:0/3701352004 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c00e070 con 0x7fb624138dc0 2023-09-30T22:36:23.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.400+0000 7fb628e61700 1 -- 172.21.15.16:0/3701352004 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb61c00bb20 con 0x7fb624138dc0 2023-09-30T22:36:23.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.400+0000 7fb628e61700 1 -- 172.21.15.16:0/3701352004 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c012430 con 0x7fb624138dc0 2023-09-30T22:36:23.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.400+0000 7fb62c0c7700 1 -- 172.21.15.16:0/3701352004 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb624138dc0 msgr2=0x7fb62413b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:23.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.400+0000 7fb62c0c7700 1 --2- 172.21.15.16:0/3701352004 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb624138dc0 0x7fb62413b1a0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fb61c009d90 tx=0x7fb61c00b330 comp rx=0 tx=0).stop 2023-09-30T22:36:23.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.400+0000 7fb62c0c7700 1 -- 172.21.15.16:0/3701352004 shutdown_connections 2023-09-30T22:36:23.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.400+0000 7fb62c0c7700 1 --2- 172.21.15.16:0/3701352004 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb624138dc0 0x7fb62413b1a0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:23.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.400+0000 7fb62c0c7700 1 --2- 172.21.15.16:0/3701352004 >> v2:172.21.15.16:3300/0 conn(0x7fb6241337f0 0x7fb624138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:23.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.401+0000 7fb62c0c7700 1 -- 172.21.15.16:0/3701352004 >> 172.21.15.16:0/3701352004 conn(0x7fb6240a4210 msgr2=0x7fb6240a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:23.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.401+0000 7fb62c0c7700 1 -- 172.21.15.16:0/3701352004 shutdown_connections 2023-09-30T22:36:23.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.401+0000 7fb62c0c7700 1 -- 172.21.15.16:0/3701352004 wait complete. 2023-09-30T22:36:23.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.402+0000 7fb62c0c7700 1 Processor -- start 2023-09-30T22:36:23.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.402+0000 7fb62c0c7700 1 -- start start 2023-09-30T22:36:23.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.403+0000 7fb62c0c7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb624133420 0x7fb62412ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:23.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.403+0000 7fb62c0c7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6241337f0 0x7fb62412f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:23.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.403+0000 7fb62c0c7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb624138dc0 0x7fb62412f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:23.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.403+0000 7fb62c0c7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb62409cb90 con 0x7fb6241337f0 2023-09-30T22:36:23.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.403+0000 7fb62c0c7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb62409d6d0 con 0x7fb624138dc0 2023-09-30T22:36:23.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.403+0000 7fb62c0c7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb62409b6a0 con 0x7fb624133420 2023-09-30T22:36:23.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.403+0000 7fb62a664700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb624138dc0 0x7fb62412f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:23.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.403+0000 7fb62a664700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb624138dc0 0x7fb62412f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38390/0 (socket says 172.21.15.16:38390) 2023-09-30T22:36:23.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.403+0000 7fb62a664700 1 -- 172.21.15.16:0/2685245044 learned_addr learned my addr 172.21.15.16:0/2685245044 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:23.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.403+0000 7fb629662700 1 --2- 172.21.15.16:0/2685245044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6241337f0 0x7fb62412f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:23.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.403+0000 7fb62a664700 1 -- 172.21.15.16:0/2685245044 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb624133420 msgr2=0x7fb62412ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:23.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.403+0000 7fb62a664700 1 --2- 172.21.15.16:0/2685245044 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb624133420 0x7fb62412ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:23.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.403+0000 7fb62a664700 1 -- 172.21.15.16:0/2685245044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6241337f0 msgr2=0x7fb62412f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:23.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.403+0000 7fb62a664700 1 --2- 172.21.15.16:0/2685245044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6241337f0 0x7fb62412f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:23.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.404+0000 7fb62a664700 1 -- 172.21.15.16:0/2685245044 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb614009550 con 0x7fb624138dc0 2023-09-30T22:36:23.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.404+0000 7fb62a664700 1 --2- 172.21.15.16:0/2685245044 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb624138dc0 0x7fb62412f8d0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fb624133d60 tx=0x7fb61c004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:23.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.404+0000 7fb61a7fc700 1 -- 172.21.15.16:0/2685245044 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c012430 con 0x7fb624138dc0 2023-09-30T22:36:23.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.404+0000 7fb61a7fc700 1 -- 172.21.15.16:0/2685245044 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb61c020070 con 0x7fb624138dc0 2023-09-30T22:36:23.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.404+0000 7fb62c0c7700 1 -- 172.21.15.16:0/2685245044 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb61c009a70 con 0x7fb624138dc0 2023-09-30T22:36:23.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.404+0000 7fb61a7fc700 1 -- 172.21.15.16:0/2685245044 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb61c004280 con 0x7fb624138dc0 2023-09-30T22:36:23.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.404+0000 7fb62c0c7700 1 -- 172.21.15.16:0/2685245044 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb62409bda0 con 0x7fb624138dc0 2023-09-30T22:36:23.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.405+0000 7fb62c0c7700 1 -- 172.21.15.16:0/2685245044 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb60c0052e0 con 0x7fb624138dc0 2023-09-30T22:36:23.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.406+0000 7fb61a7fc700 1 -- 172.21.15.16:0/2685245044 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb61c0043e0 con 0x7fb624138dc0 2023-09-30T22:36:23.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.406+0000 7fb61a7fc700 1 -- 172.21.15.16:0/2685245044 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fb61c022070 con 0x7fb624138dc0 2023-09-30T22:36:23.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.415+0000 7fb61a7fc700 1 -- 172.21.15.16:0/2685245044 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb61c031430 con 0x7fb624138dc0 2023-09-30T22:36:23.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.439+0000 7fc74f86b700 1 Processor -- start 2023-09-30T22:36:23.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.440+0000 7fc74f86b700 1 -- start start 2023-09-30T22:36:23.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.440+0000 7fc74f86b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc74809e7e0 0x7fc74809ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:23.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.440+0000 7fc74f86b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc74809f0f0 0x7fc748137ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:23.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.440+0000 7fc74f86b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc748137ff0 0x7fc74813a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:23.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.440+0000 7fc74f86b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc74806bd70 con 0x7fc748137ff0 2023-09-30T22:36:23.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.440+0000 7fc74f86b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc74806cc30 con 0x7fc74809f0f0 2023-09-30T22:36:23.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.440+0000 7fc74f86b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc74806d870 con 0x7fc74809e7e0 2023-09-30T22:36:23.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.440+0000 7fc74ce06700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc74809f0f0 0x7fc748137ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:23.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.441+0000 7fc74ce06700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc74809f0f0 0x7fc748137ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38400/0 (socket says 172.21.15.16:38400) 2023-09-30T22:36:23.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.441+0000 7fc74ce06700 1 -- 172.21.15.16:0/2973776633 learned_addr learned my addr 172.21.15.16:0/2973776633 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:23.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.441+0000 7fc74de08700 1 --2- 172.21.15.16:0/2973776633 >> v2:172.21.15.16:3300/0 conn(0x7fc748137ff0 0x7fc74813a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:23.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.441+0000 7fc74d607700 1 --2- 172.21.15.16:0/2973776633 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc74809e7e0 0x7fc74809ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:23.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.441+0000 7fc74ce06700 1 -- 172.21.15.16:0/2973776633 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc74809e7e0 msgr2=0x7fc74809ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:23.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.441+0000 7fc74ce06700 1 --2- 172.21.15.16:0/2973776633 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc74809e7e0 0x7fc74809ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:23.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.441+0000 7fc74ce06700 1 -- 172.21.15.16:0/2973776633 >> v2:172.21.15.16:3300/0 conn(0x7fc748137ff0 msgr2=0x7fc74813a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:23.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.441+0000 7fc74ce06700 1 --2- 172.21.15.16:0/2973776633 >> v2:172.21.15.16:3300/0 conn(0x7fc748137ff0 0x7fc74813a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:23.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.441+0000 7fc74ce06700 1 -- 172.21.15.16:0/2973776633 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc74806bab0 con 0x7fc74809f0f0 2023-09-30T22:36:23.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.441+0000 7fc74ce06700 1 --2- 172.21.15.16:0/2973776633 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc74809f0f0 0x7fc748137ab0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fc7380098f0 tx=0x7fc738009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e9298685a5de8b8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:23.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.441+0000 7fc73ffff700 1 -- 172.21.15.16:0/2973776633 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc738011830 con 0x7fc74809f0f0 2023-09-30T22:36:23.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.441+0000 7fc73ffff700 1 -- 172.21.15.16:0/2973776633 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc738011990 con 0x7fc74809f0f0 2023-09-30T22:36:23.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.441+0000 7fc73ffff700 1 -- 172.21.15.16:0/2973776633 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc738011d20 con 0x7fc74809f0f0 2023-09-30T22:36:23.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.442+0000 7fc74f86b700 1 -- 172.21.15.16:0/2973776633 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc74809f0f0 msgr2=0x7fc748137ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:23.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.442+0000 7fc74f86b700 1 --2- 172.21.15.16:0/2973776633 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc74809f0f0 0x7fc748137ab0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fc7380098f0 tx=0x7fc738009c00 comp rx=0 tx=0).stop 2023-09-30T22:36:23.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.442+0000 7fc74f86b700 1 -- 172.21.15.16:0/2973776633 shutdown_connections 2023-09-30T22:36:23.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.442+0000 7fc74f86b700 1 --2- 172.21.15.16:0/2973776633 >> v2:172.21.15.16:3300/0 conn(0x7fc748137ff0 0x7fc74813a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:23.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.442+0000 7fc74f86b700 1 --2- 172.21.15.16:0/2973776633 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc74809e7e0 0x7fc74809ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:23.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.442+0000 7fc74f86b700 1 --2- 172.21.15.16:0/2973776633 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc74809f0f0 0x7fc748137ab0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:23.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.442+0000 7fc74f86b700 1 -- 172.21.15.16:0/2973776633 >> 172.21.15.16:0/2973776633 conn(0x7fc7480a4300 msgr2=0x7fc7480a4700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:23.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.442+0000 7fc74f86b700 1 -- 172.21.15.16:0/2973776633 shutdown_connections 2023-09-30T22:36:23.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.442+0000 7fc74f86b700 1 -- 172.21.15.16:0/2973776633 wait complete. 2023-09-30T22:36:23.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.444+0000 7fc74f86b700 1 Processor -- start 2023-09-30T22:36:23.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.444+0000 7fc74f86b700 1 -- start start 2023-09-30T22:36:23.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.445+0000 7fc74f86b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc74809e7e0 0x7fc748121870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:23.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.445+0000 7fc74f86b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc74809f0f0 0x7fc748121db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:23.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.445+0000 7fc74f86b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc748137ff0 0x7fc7481222f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:23.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.445+0000 7fc74f86b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc748125140 con 0x7fc74809f0f0 2023-09-30T22:36:23.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.445+0000 7fc74f86b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc748125340 con 0x7fc74809e7e0 2023-09-30T22:36:23.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.445+0000 7fc74f86b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc7481362d0 con 0x7fc748137ff0 2023-09-30T22:36:23.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.445+0000 7fc74ce06700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc74809f0f0 0x7fc748121db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:23.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.445+0000 7fc74d607700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc74809e7e0 0x7fc748121870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:23.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.445+0000 7fc74ce06700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc74809f0f0 0x7fc748121db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42124/0 (socket says 172.21.15.16:42124) 2023-09-30T22:36:23.447 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.445+0000 7fc74ce06700 1 -- 172.21.15.16:0/1818408008 learned_addr learned my addr 172.21.15.16:0/1818408008 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:23.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.445+0000 7fc74d607700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc74809e7e0 0x7fc748121870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38414/0 (socket says 172.21.15.16:38414) 2023-09-30T22:36:23.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.445+0000 7fc74de08700 1 --2- 172.21.15.16:0/1818408008 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc748137ff0 0x7fc7481222f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:23.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.445+0000 7fc74d607700 1 -- 172.21.15.16:0/1818408008 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc748137ff0 msgr2=0x7fc7481222f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:23.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.445+0000 7fc74d607700 1 --2- 172.21.15.16:0/1818408008 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc748137ff0 0x7fc7481222f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:23.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.445+0000 7fc74d607700 1 -- 172.21.15.16:0/1818408008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc74809f0f0 msgr2=0x7fc748121db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:23.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.445+0000 7fc74d607700 1 --2- 172.21.15.16:0/1818408008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc74809f0f0 0x7fc748121db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:23.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.445+0000 7fc74d607700 1 -- 172.21.15.16:0/1818408008 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc738009550 con 0x7fc74809e7e0 2023-09-30T22:36:23.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.446+0000 7fc74d607700 1 --2- 172.21.15.16:0/1818408008 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc74809e7e0 0x7fc748121870 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fc74400e810 tx=0x7fc74400eb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:23.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.446+0000 7fc73dffb700 1 -- 172.21.15.16:0/1818408008 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74400c070 con 0x7fc74809e7e0 2023-09-30T22:36:23.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.446+0000 7fc73dffb700 1 -- 172.21.15.16:0/1818408008 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc744013610 con 0x7fc74809e7e0 2023-09-30T22:36:23.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.446+0000 7fc74f86b700 1 -- 172.21.15.16:0/1818408008 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc74806f9d0 con 0x7fc74809e7e0 2023-09-30T22:36:23.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.446+0000 7fc73dffb700 1 -- 172.21.15.16:0/1818408008 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7440139a0 con 0x7fc74809e7e0 2023-09-30T22:36:23.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.446+0000 7fc74f86b700 1 -- 172.21.15.16:0/1818408008 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7480a0620 con 0x7fc74809e7e0 2023-09-30T22:36:23.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.447+0000 7fc73dffb700 1 -- 172.21.15.16:0/1818408008 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc744013b00 con 0x7fc74809e7e0 2023-09-30T22:36:23.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.447+0000 7fc73dffb700 1 -- 172.21.15.16:0/1818408008 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fc7440407e0 con 0x7fc74809e7e0 2023-09-30T22:36:23.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.447+0000 7fc74f86b700 1 -- 172.21.15.16:0/1818408008 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc74804f2a0 con 0x7fc74809e7e0 2023-09-30T22:36:23.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.457+0000 7fc73dffb700 1 -- 172.21.15.16:0/1818408008 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc744044060 con 0x7fc74809e7e0 2023-09-30T22:36:23.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.562+0000 7fdeff479700 1 -- 172.21.15.16:0/1328998107 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fdef812b1d0 con 0x7fdef806a770 2023-09-30T22:36:23.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.564+0000 7fdeedffb700 1 -- 172.21.15.16:0/1328998107 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdef4007720 con 0x7fdef806a770 2023-09-30T22:36:23.564 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:36:23.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.566+0000 7fdeff479700 1 -- 172.21.15.16:0/1328998107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdef806a770 msgr2=0x7fdef8127850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:23.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.566+0000 7fdeff479700 1 --2- 172.21.15.16:0/1328998107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdef806a770 0x7fdef8127850 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fdef400c470 tx=0x7fdef400c780 comp rx=0 tx=0).stop 2023-09-30T22:36:23.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.569+0000 7fdeff479700 1 -- 172.21.15.16:0/1328998107 shutdown_connections 2023-09-30T22:36:23.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.569+0000 7fdeff479700 1 --2- 172.21.15.16:0/1328998107 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdef8136d80 0x7fdef81cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:23.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.569+0000 7fdeff479700 1 --2- 172.21.15.16:0/1328998107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdef806a770 0x7fdef8127850 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:23.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.569+0000 7fdeff479700 1 --2- 172.21.15.16:0/1328998107 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdef806b250 0x7fdef8127d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:23.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.569+0000 7fdeff479700 1 -- 172.21.15.16:0/1328998107 >> 172.21.15.16:0/1328998107 conn(0x7fdef8066f50 msgr2=0x7fdef80708f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:23.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.573+0000 7fdeff479700 1 -- 172.21.15.16:0/1328998107 shutdown_connections 2023-09-30T22:36:23.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.573+0000 7fdeff479700 1 -- 172.21.15.16:0/1328998107 wait complete. 2023-09-30T22:36:23.588 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:36:23.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.592+0000 7f665886c700 1 -- 172.21.15.16:0/3790245852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6654127bb0 con 0x7f665409cf10 2023-09-30T22:36:23.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.593+0000 7f6642ffd700 1 -- 172.21.15.16:0/3790245852 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6654127bb0 con 0x7f665409cf10 2023-09-30T22:36:23.593 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:36:23.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.595+0000 7f665886c700 1 -- 172.21.15.16:0/3790245852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f665409cf10 msgr2=0x7f66540962f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:23.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.595+0000 7f665886c700 1 --2- 172.21.15.16:0/3790245852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f665409cf10 0x7f66540962f0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f6648013000 tx=0x7f6648004750 comp rx=0 tx=0).stop 2023-09-30T22:36:23.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.595+0000 7f665886c700 1 -- 172.21.15.16:0/3790245852 shutdown_connections 2023-09-30T22:36:23.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.595+0000 7f665886c700 1 --2- 172.21.15.16:0/3790245852 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f665409cb40 0x7f6654095db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:23.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.595+0000 7f665886c700 1 --2- 172.21.15.16:0/3790245852 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6654131370 0x7f6654096830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:23.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.595+0000 7f665886c700 1 --2- 172.21.15.16:0/3790245852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f665409cf10 0x7f66540962f0 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:23.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.595+0000 7f665886c700 1 -- 172.21.15.16:0/3790245852 >> 172.21.15.16:0/3790245852 conn(0x7f665409e7e0 msgr2=0x7f66540a0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:23.596 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.595+0000 7f665886c700 1 -- 172.21.15.16:0/3790245852 shutdown_connections 2023-09-30T22:36:23.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.595+0000 7f665886c700 1 -- 172.21.15.16:0/3790245852 wait complete. 2023-09-30T22:36:23.609 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:36:23.635 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.635+0000 7fb62c0c7700 1 -- 172.21.15.16:0/2685245044 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb60c005860 con 0x7fb624138dc0 2023-09-30T22:36:23.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.635+0000 7fb61a7fc700 1 -- 172.21.15.16:0/2685245044 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb61c035090 con 0x7fb624138dc0 2023-09-30T22:36:23.636 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:36:23.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.638+0000 7fb62c0c7700 1 -- 172.21.15.16:0/2685245044 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb624138dc0 msgr2=0x7fb62412f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:23.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.638+0000 7fb62c0c7700 1 --2- 172.21.15.16:0/2685245044 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb624138dc0 0x7fb62412f8d0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fb624133d60 tx=0x7fb61c004ad0 comp rx=0 tx=0).stop 2023-09-30T22:36:23.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.638+0000 7fb62c0c7700 1 -- 172.21.15.16:0/2685245044 shutdown_connections 2023-09-30T22:36:23.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.638+0000 7fb62c0c7700 1 --2- 172.21.15.16:0/2685245044 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb624138dc0 0x7fb62412f8d0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:23.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.638+0000 7fb62c0c7700 1 --2- 172.21.15.16:0/2685245044 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb624133420 0x7fb62412ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:23.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.638+0000 7fb62c0c7700 1 --2- 172.21.15.16:0/2685245044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6241337f0 0x7fb62412f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:23.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.638+0000 7fb62c0c7700 1 -- 172.21.15.16:0/2685245044 >> 172.21.15.16:0/2685245044 conn(0x7fb6240a4210 msgr2=0x7fb62413aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:23.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.638+0000 7fb62c0c7700 1 -- 172.21.15.16:0/2685245044 shutdown_connections 2023-09-30T22:36:23.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.638+0000 7fb62c0c7700 1 -- 172.21.15.16:0/2685245044 wait complete. 2023-09-30T22:36:23.652 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:36:23.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.673+0000 7fc74f86b700 1 -- 172.21.15.16:0/1818408008 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc74806b120 con 0x7fc74809e7e0 2023-09-30T22:36:23.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.673+0000 7fc73dffb700 1 -- 172.21.15.16:0/1818408008 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc74401c400 con 0x7fc74809e7e0 2023-09-30T22:36:23.673 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:36:23.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.675+0000 7fc74f86b700 1 -- 172.21.15.16:0/1818408008 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc74809e7e0 msgr2=0x7fc748121870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:23.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.675+0000 7fc74f86b700 1 --2- 172.21.15.16:0/1818408008 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc74809e7e0 0x7fc748121870 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fc74400e810 tx=0x7fc74400eb20 comp rx=0 tx=0).stop 2023-09-30T22:36:23.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.675+0000 7fc74f86b700 1 -- 172.21.15.16:0/1818408008 shutdown_connections 2023-09-30T22:36:23.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.675+0000 7fc74f86b700 1 --2- 172.21.15.16:0/1818408008 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc74809e7e0 0x7fc748121870 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:23.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.675+0000 7fc74f86b700 1 --2- 172.21.15.16:0/1818408008 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc748137ff0 0x7fc7481222f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:23.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.675+0000 7fc74f86b700 1 --2- 172.21.15.16:0/1818408008 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc74809f0f0 0x7fc748121db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:23.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.675+0000 7fc74f86b700 1 -- 172.21.15.16:0/1818408008 >> 172.21.15.16:0/1818408008 conn(0x7fc7480a4300 msgr2=0x7fc748070df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:23.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.675+0000 7fc74f86b700 1 -- 172.21.15.16:0/1818408008 shutdown_connections 2023-09-30T22:36:23.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:23.675+0000 7fc74f86b700 1 -- 172.21.15.16:0/1818408008 wait complete. 2023-09-30T22:36:23.689 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:36:24.589 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:36:24.610 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:36:24.653 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:36:24.690 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:36:24.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.750+0000 7fcd83dbd700 1 Processor -- start 2023-09-30T22:36:24.751 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.750+0000 7fcd83dbd700 1 -- start start 2023-09-30T22:36:24.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.751+0000 7fcd83dbd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd7c06e570 0x7fcd7c06e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:24.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.751+0000 7fcd83dbd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd7c09eca0 0x7fcd7c06ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:24.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.751+0000 7fcd83dbd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcd7c134610 0x7fcd7c134ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:24.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.751+0000 7fcd83dbd700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd7c070ee0 con 0x7fcd7c134610 2023-09-30T22:36:24.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.751+0000 7fcd83dbd700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c071480 con 0x7fcd7c09eca0 2023-09-30T22:36:24.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.751+0000 7fcd83dbd700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c06f3c0 con 0x7fcd7c06e570 2023-09-30T22:36:24.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.751+0000 7fcd81358700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd7c09eca0 0x7fcd7c06ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:24.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.752+0000 7fcd81358700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd7c09eca0 0x7fcd7c06ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38420/0 (socket says 172.21.15.16:38420) 2023-09-30T22:36:24.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.752+0000 7fcd81358700 1 -- 172.21.15.16:0/3996561975 learned_addr learned my addr 172.21.15.16:0/3996561975 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:24.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.752+0000 7fcd8235a700 1 --2- 172.21.15.16:0/3996561975 >> v2:172.21.15.16:3300/0 conn(0x7fcd7c134610 0x7fcd7c134ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:24.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.752+0000 7fcd81b59700 1 --2- 172.21.15.16:0/3996561975 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd7c06e570 0x7fcd7c06e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:24.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.752+0000 7fcd81358700 1 -- 172.21.15.16:0/3996561975 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd7c06e570 msgr2=0x7fcd7c06e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:24.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.752+0000 7fcd81358700 1 --2- 172.21.15.16:0/3996561975 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd7c06e570 0x7fcd7c06e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:24.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.752+0000 7fcd81358700 1 -- 172.21.15.16:0/3996561975 >> v2:172.21.15.16:3300/0 conn(0x7fcd7c134610 msgr2=0x7fcd7c134ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:24.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.752+0000 7fcd81358700 1 --2- 172.21.15.16:0/3996561975 >> v2:172.21.15.16:3300/0 conn(0x7fcd7c134610 0x7fcd7c134ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:24.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.752+0000 7fcd81358700 1 -- 172.21.15.16:0/3996561975 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd7c071b90 con 0x7fcd7c09eca0 2023-09-30T22:36:24.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.752+0000 7fcd81358700 1 --2- 172.21.15.16:0/3996561975 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd7c09eca0 0x7fcd7c06ee80 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c0098f0 tx=0x7fcd6c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=accaee20ca863553 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:24.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.752+0000 7fcd80b57700 1 -- 172.21.15.16:0/3996561975 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c011830 con 0x7fcd7c09eca0 2023-09-30T22:36:24.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.753+0000 7fcd80b57700 1 -- 172.21.15.16:0/3996561975 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd6c011990 con 0x7fcd7c09eca0 2023-09-30T22:36:24.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.753+0000 7fcd80b57700 1 -- 172.21.15.16:0/3996561975 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c011d20 con 0x7fcd7c09eca0 2023-09-30T22:36:24.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.754+0000 7fcd83dbd700 1 -- 172.21.15.16:0/3996561975 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd7c09eca0 msgr2=0x7fcd7c06ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:24.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.754+0000 7fcd83dbd700 1 --2- 172.21.15.16:0/3996561975 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd7c09eca0 0x7fcd7c06ee80 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c0098f0 tx=0x7fcd6c009c00 comp rx=0 tx=0).stop 2023-09-30T22:36:24.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.754+0000 7fcd83dbd700 1 -- 172.21.15.16:0/3996561975 shutdown_connections 2023-09-30T22:36:24.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.754+0000 7fcd83dbd700 1 --2- 172.21.15.16:0/3996561975 >> v2:172.21.15.16:3300/0 conn(0x7fcd7c134610 0x7fcd7c134ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:24.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.754+0000 7fcd83dbd700 1 --2- 172.21.15.16:0/3996561975 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd7c06e570 0x7fcd7c06e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:24.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.754+0000 7fcd83dbd700 1 --2- 172.21.15.16:0/3996561975 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd7c09eca0 0x7fcd7c06ee80 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:24.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.754+0000 7fcd83dbd700 1 -- 172.21.15.16:0/3996561975 >> 172.21.15.16:0/3996561975 conn(0x7fcd7c0a2aa0 msgr2=0x7fcd7c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:24.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.754+0000 7fcd83dbd700 1 -- 172.21.15.16:0/3996561975 shutdown_connections 2023-09-30T22:36:24.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.754+0000 7fcd83dbd700 1 -- 172.21.15.16:0/3996561975 wait complete. 2023-09-30T22:36:24.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.755+0000 7fcd83dbd700 1 Processor -- start 2023-09-30T22:36:24.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.755+0000 7fcd83dbd700 1 -- start start 2023-09-30T22:36:24.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.755+0000 7fcd83dbd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd7c06e570 0x7fcd7c1214e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:24.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.755+0000 7fcd83dbd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd7c134610 0x7fcd7c121a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:24.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.756+0000 7fcd83dbd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd7c121f60 0x7fcd7c1d2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:24.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.756+0000 7fcd83dbd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c1228b0 con 0x7fcd7c134610 2023-09-30T22:36:24.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.756+0000 7fcd83dbd700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c1342f0 con 0x7fcd7c06e570 2023-09-30T22:36:24.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.756+0000 7fcd83dbd700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7c123130 con 0x7fcd7c121f60 2023-09-30T22:36:24.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.756+0000 7fcd8235a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd7c121f60 0x7fcd7c1d2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:24.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.756+0000 7fcd8235a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd7c121f60 0x7fcd7c1d2d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:57348/0 (socket says 172.21.15.16:57348) 2023-09-30T22:36:24.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.756+0000 7fcd8235a700 1 -- 172.21.15.16:0/3275937703 learned_addr learned my addr 172.21.15.16:0/3275937703 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:24.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.756+0000 7fcd81358700 1 --2- 172.21.15.16:0/3275937703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd7c134610 0x7fcd7c121a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:24.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.756+0000 7fcd8235a700 1 -- 172.21.15.16:0/3275937703 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd7c06e570 msgr2=0x7fcd7c1214e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:24.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.756+0000 7fcd8235a700 1 --2- 172.21.15.16:0/3275937703 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd7c06e570 0x7fcd7c1214e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:24.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.756+0000 7fcd8235a700 1 -- 172.21.15.16:0/3275937703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd7c134610 msgr2=0x7fcd7c121a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:24.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.756+0000 7fcd8235a700 1 --2- 172.21.15.16:0/3275937703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd7c134610 0x7fcd7c121a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:24.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.756+0000 7fcd8235a700 1 -- 172.21.15.16:0/3275937703 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd6c009550 con 0x7fcd7c121f60 2023-09-30T22:36:24.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.757+0000 7fcd8235a700 1 --2- 172.21.15.16:0/3275937703 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd7c121f60 0x7fcd7c1d2d20 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fcd7400efc0 tx=0x7fcd7400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:24.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.757+0000 7fcd727fc700 1 -- 172.21.15.16:0/3275937703 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd74009ea0 con 0x7fcd7c121f60 2023-09-30T22:36:24.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.757+0000 7fcd727fc700 1 -- 172.21.15.16:0/3275937703 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd7400f040 con 0x7fcd7c121f60 2023-09-30T22:36:24.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.757+0000 7fcd83dbd700 1 -- 172.21.15.16:0/3275937703 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd7c123d60 con 0x7fcd7c121f60 2023-09-30T22:36:24.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.757+0000 7fcd83dbd700 1 -- 172.21.15.16:0/3275937703 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd7c0a2250 con 0x7fcd7c121f60 2023-09-30T22:36:24.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.757+0000 7fcd727fc700 1 -- 172.21.15.16:0/3275937703 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd74014480 con 0x7fcd7c121f60 2023-09-30T22:36:24.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.758+0000 7fcd83dbd700 1 -- 172.21.15.16:0/3275937703 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd640052e0 con 0x7fcd7c121f60 2023-09-30T22:36:24.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.759+0000 7fcd727fc700 1 -- 172.21.15.16:0/3275937703 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcd740145e0 con 0x7fcd7c121f60 2023-09-30T22:36:24.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.759+0000 7fcd727fc700 1 -- 172.21.15.16:0/3275937703 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fcd74040b50 con 0x7fcd7c121f60 2023-09-30T22:36:24.764 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.764+0000 7fcd727fc700 1 -- 172.21.15.16:0/3275937703 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcd74012930 con 0x7fcd7c121f60 2023-09-30T22:36:24.786 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.786+0000 7f51d73aa700 1 Processor -- start 2023-09-30T22:36:24.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.787+0000 7f51d73aa700 1 -- start start 2023-09-30T22:36:24.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.787+0000 7f51d73aa700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f51d009eca0 0x7f51d013cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:24.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.787+0000 7f51d73aa700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f51d006f050 0x7f51d013f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:24.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.787+0000 7f51d73aa700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51d0070ee0 con 0x7f51d006e570 2023-09-30T22:36:24.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.787+0000 7f51d73aa700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51d0071980 con 0x7f51d006f050 2023-09-30T22:36:24.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.787+0000 7f51d73aa700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51d0071480 con 0x7f51d009eca0 2023-09-30T22:36:24.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.787+0000 7f51d5146700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f51d006e570 0x7f51d006e940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60258/0 (socket says 172.21.15.16:60258) 2023-09-30T22:36:24.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.787+0000 7f51d5146700 1 -- 172.21.15.16:0/2377176680 learned_addr learned my addr 172.21.15.16:0/2377176680 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:36:24.789 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.787+0000 7f51d5947700 1 --2- 172.21.15.16:0/2377176680 >> v2:172.21.15.16:3300/0 conn(0x7f51d006f050 0x7f51d013f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:24.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.787+0000 7f51d4945700 1 --2- 172.21.15.16:0/2377176680 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f51d009eca0 0x7f51d013cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:24.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.788+0000 7f51c7fff700 1 -- 172.21.15.16:0/2377176680 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1810529208 0 0) 0x7f51d0070ee0 con 0x7f51d006e570 2023-09-30T22:36:24.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.788+0000 7f51c7fff700 1 -- 172.21.15.16:0/2377176680 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51bc0035b0 con 0x7f51d006e570 2023-09-30T22:36:24.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.788+0000 7f51d5947700 1 -- 172.21.15.16:0/2377176680 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f51d009eca0 msgr2=0x7f51d013cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:24.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.788+0000 7f51d5947700 1 --2- 172.21.15.16:0/2377176680 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f51d009eca0 0x7f51d013cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:24.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.788+0000 7f51d5947700 1 -- 172.21.15.16:0/2377176680 >> v1:172.21.15.16:6789/0 conn(0x7f51d006e570 legacy=0x7f51d006e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:24.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.788+0000 7f51d5947700 1 -- 172.21.15.16:0/2377176680 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d0071b90 con 0x7f51d006f050 2023-09-30T22:36:24.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.788+0000 7f51d4945700 1 --2- 172.21.15.16:0/2377176680 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f51d009eca0 0x7f51d013cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:24.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.788+0000 7f51d5947700 1 --2- 172.21.15.16:0/2377176680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51d006f050 0x7f51d013f3f0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f51c8009d90 tx=0x7f51c800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=754c8b819385587f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:24.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.788+0000 7f51c7fff700 1 -- 172.21.15.16:0/2377176680 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c800e070 con 0x7f51d006f050 2023-09-30T22:36:24.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.788+0000 7f51c7fff700 1 -- 172.21.15.16:0/2377176680 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51c800bb20 con 0x7f51d006f050 2023-09-30T22:36:24.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.788+0000 7f51c7fff700 1 -- 172.21.15.16:0/2377176680 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c8012430 con 0x7f51d006f050 2023-09-30T22:36:24.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.789+0000 7f51d73aa700 1 -- 172.21.15.16:0/2377176680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51d006f050 msgr2=0x7f51d013f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:24.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.789+0000 7f51d73aa700 1 --2- 172.21.15.16:0/2377176680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51d006f050 0x7f51d013f3f0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f51c8009d90 tx=0x7f51c800b330 comp rx=0 tx=0).stop 2023-09-30T22:36:24.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.789+0000 7f51d73aa700 1 -- 172.21.15.16:0/2377176680 shutdown_connections 2023-09-30T22:36:24.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.789+0000 7f51d73aa700 1 --2- 172.21.15.16:0/2377176680 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51d006f050 0x7f51d013f3f0 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:24.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.789+0000 7f51d73aa700 1 --2- 172.21.15.16:0/2377176680 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f51d009eca0 0x7f51d013cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:24.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.789+0000 7f51d73aa700 1 -- 172.21.15.16:0/2377176680 >> 172.21.15.16:0/2377176680 conn(0x7f51d00a2aa0 msgr2=0x7f51d00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:24.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.789+0000 7f51d73aa700 1 -- 172.21.15.16:0/2377176680 shutdown_connections 2023-09-30T22:36:24.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.789+0000 7f51d73aa700 1 -- 172.21.15.16:0/2377176680 wait complete. 2023-09-30T22:36:24.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.790+0000 7f51d73aa700 1 Processor -- start 2023-09-30T22:36:24.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.790+0000 7f51d73aa700 1 -- start start 2023-09-30T22:36:24.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.791+0000 7f51d73aa700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f51d006e570 0x7f51d0136480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:24.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.791+0000 7f51d73aa700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51d006f050 0x7f51d01369c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:24.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.791+0000 7f51d73aa700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f51d013a4a0 0x7f51d0136f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:24.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.791+0000 7f51d73aa700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d0138280 con 0x7f51d006f050 2023-09-30T22:36:24.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.791+0000 7f51d73aa700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51d0138090 con 0x7f51d006e570 2023-09-30T22:36:24.799 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.791+0000 7f51d73aa700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f51d0138960 con 0x7f51d013a4a0 2023-09-30T22:36:24.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.791+0000 7f51d5146700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f51d006e570 0x7f51d0136480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:24.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.791+0000 7f51d5146700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f51d006e570 0x7f51d0136480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38442/0 (socket says 172.21.15.16:38442) 2023-09-30T22:36:24.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.791+0000 7f51d5146700 1 -- 172.21.15.16:0/1872261218 learned_addr learned my addr 172.21.15.16:0/1872261218 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:24.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.791+0000 7f51d5947700 1 --2- 172.21.15.16:0/1872261218 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f51d013a4a0 0x7f51d0136f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:24.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.791+0000 7f51d4945700 1 --2- 172.21.15.16:0/1872261218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51d006f050 0x7f51d01369c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:24.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.792+0000 7f51d5146700 1 -- 172.21.15.16:0/1872261218 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f51d013a4a0 msgr2=0x7f51d0136f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:24.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.792+0000 7f51d5146700 1 --2- 172.21.15.16:0/1872261218 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f51d013a4a0 0x7f51d0136f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:24.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.792+0000 7f51d5146700 1 -- 172.21.15.16:0/1872261218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51d006f050 msgr2=0x7f51d01369c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:24.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.792+0000 7f51d5146700 1 --2- 172.21.15.16:0/1872261218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51d006f050 0x7f51d01369c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:24.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.792+0000 7f51d5146700 1 -- 172.21.15.16:0/1872261218 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c00094a0 con 0x7f51d006e570 2023-09-30T22:36:24.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.792+0000 7f51d5146700 1 --2- 172.21.15.16:0/1872261218 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f51d006e570 0x7f51d0136480 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f51cc00c700 tx=0x7f51cc00ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:24.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.792+0000 7f51c5ffb700 1 -- 172.21.15.16:0/1872261218 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51cc005d70 con 0x7f51d006e570 2023-09-30T22:36:24.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.792+0000 7f51c5ffb700 1 -- 172.21.15.16:0/1872261218 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51cc0070d0 con 0x7f51d006e570 2023-09-30T22:36:24.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.792+0000 7f51d73aa700 1 -- 172.21.15.16:0/1872261218 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51c8009a70 con 0x7f51d006e570 2023-09-30T22:36:24.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.792+0000 7f51c5ffb700 1 -- 172.21.15.16:0/1872261218 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51cc007460 con 0x7f51d006e570 2023-09-30T22:36:24.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.792+0000 7f51d73aa700 1 -- 172.21.15.16:0/1872261218 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51d0139ff0 con 0x7f51d006e570 2023-09-30T22:36:24.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.793+0000 7f51d73aa700 1 -- 172.21.15.16:0/1872261218 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51d006f560 con 0x7f51d006e570 2023-09-30T22:36:24.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.794+0000 7f51c5ffb700 1 -- 172.21.15.16:0/1872261218 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f51cc0075c0 con 0x7f51d006e570 2023-09-30T22:36:24.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.796+0000 7f51c5ffb700 1 -- 172.21.15.16:0/1872261218 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f51cc011360 con 0x7f51d006e570 2023-09-30T22:36:24.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.803+0000 7f51c5ffb700 1 -- 172.21.15.16:0/1872261218 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f51d0139ff0 con 0x7f51d006e570 2023-09-30T22:36:24.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.830+0000 7fc2f8e22700 1 Processor -- start 2023-09-30T22:36:24.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.830+0000 7fc2f8e22700 1 -- start start 2023-09-30T22:36:24.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.831+0000 7fc2f8e22700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2f409de00 0x7fc2f409e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:24.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.831+0000 7fc2f8e22700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2f413d1f0 0x7fc2f413f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:24.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.831+0000 7fc2f8e22700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2f409a810 con 0x7fc2f406a770 2023-09-30T22:36:24.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.831+0000 7fc2f8e22700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f406ab40 con 0x7fc2f409de00 2023-09-30T22:36:24.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.831+0000 7fc2f8e22700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f409ebc0 con 0x7fc2f413d1f0 2023-09-30T22:36:24.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.831+0000 7fc2f259c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc2f406a770 0x7fc2f409d6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60266/0 (socket says 172.21.15.16:60266) 2023-09-30T22:36:24.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.831+0000 7fc2f259c700 1 -- 172.21.15.16:0/1793303129 learned_addr learned my addr 172.21.15.16:0/1793303129 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:36:24.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.831+0000 7fc2f2d9d700 1 --2- 172.21.15.16:0/1793303129 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2f413d1f0 0x7fc2f413f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:24.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.831+0000 7fc2f1d9b700 1 --2- 172.21.15.16:0/1793303129 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2f409de00 0x7fc2f409e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:24.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.831+0000 7fc2f1d9b700 1 -- 172.21.15.16:0/1793303129 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2f413d1f0 msgr2=0x7fc2f413f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:24.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.831+0000 7fc2f1d9b700 1 --2- 172.21.15.16:0/1793303129 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2f413d1f0 0x7fc2f413f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:24.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.831+0000 7fc2f1d9b700 1 -- 172.21.15.16:0/1793303129 >> v1:172.21.15.16:6789/0 conn(0x7fc2f406a770 legacy=0x7fc2f409d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:24.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.831+0000 7fc2f1d9b700 1 -- 172.21.15.16:0/1793303129 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f409e920 con 0x7fc2f409de00 2023-09-30T22:36:24.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.832+0000 7fc2f1d9b700 1 --2- 172.21.15.16:0/1793303129 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2f409de00 0x7fc2f409e250 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8009870 tx=0x7fc2e8009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa49fa00b4549a29 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:24.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.832+0000 7fc2f159a700 1 -- 172.21.15.16:0/1793303129 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e8004230 con 0x7fc2f409de00 2023-09-30T22:36:24.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.832+0000 7fc2f159a700 1 -- 172.21.15.16:0/1793303129 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2e8004390 con 0x7fc2f409de00 2023-09-30T22:36:24.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.832+0000 7fc2f159a700 1 -- 172.21.15.16:0/1793303129 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e80036a0 con 0x7fc2f409de00 2023-09-30T22:36:24.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.832+0000 7fc2f8e22700 1 -- 172.21.15.16:0/1793303129 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2f409de00 msgr2=0x7fc2f409e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:24.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.833+0000 7fc2f8e22700 1 --2- 172.21.15.16:0/1793303129 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2f409de00 0x7fc2f409e250 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8009870 tx=0x7fc2e8009b80 comp rx=0 tx=0).stop 2023-09-30T22:36:24.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.833+0000 7fc2f8e22700 1 -- 172.21.15.16:0/1793303129 shutdown_connections 2023-09-30T22:36:24.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.833+0000 7fc2f8e22700 1 --2- 172.21.15.16:0/1793303129 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2f413d1f0 0x7fc2f413f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:24.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.833+0000 7fc2f8e22700 1 --2- 172.21.15.16:0/1793303129 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2f409de00 0x7fc2f409e250 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:24.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.833+0000 7fc2f8e22700 1 -- 172.21.15.16:0/1793303129 >> 172.21.15.16:0/1793303129 conn(0x7fc2f4066f50 msgr2=0x7fc2f409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:24.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.833+0000 7fc2f8e22700 1 -- 172.21.15.16:0/1793303129 shutdown_connections 2023-09-30T22:36:24.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.833+0000 7fc2f8e22700 1 -- 172.21.15.16:0/1793303129 wait complete. 2023-09-30T22:36:24.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.834+0000 7fc2f8e22700 1 Processor -- start 2023-09-30T22:36:24.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.834+0000 7fc2f8e22700 1 -- start start 2023-09-30T22:36:24.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.834+0000 7fc2f8e22700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2f406a770 0x7fc2f41384d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:24.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.834+0000 7fc2f8e22700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2f409de00 0x7fc2f4138a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:24.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.835+0000 7fc2f8e22700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2f413d1f0 0x7fc2f4138f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:24.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.835+0000 7fc2f8e22700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f4072120 con 0x7fc2f406a770 2023-09-30T22:36:24.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.835+0000 7fc2f8e22700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f413a230 con 0x7fc2f409de00 2023-09-30T22:36:24.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.835+0000 7fc2f8e22700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f40a1be0 con 0x7fc2f413d1f0 2023-09-30T22:36:24.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.835+0000 7fc2f259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2f406a770 0x7fc2f41384d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:24.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.835+0000 7fc2f259c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2f406a770 0x7fc2f41384d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42166/0 (socket says 172.21.15.16:42166) 2023-09-30T22:36:24.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.835+0000 7fc2f259c700 1 -- 172.21.15.16:0/6295838 learned_addr learned my addr 172.21.15.16:0/6295838 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:24.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.835+0000 7fc2f1d9b700 1 --2- 172.21.15.16:0/6295838 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2f409de00 0x7fc2f4138a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:24.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.835+0000 7fc2f2d9d700 1 --2- 172.21.15.16:0/6295838 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2f413d1f0 0x7fc2f4138f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:24.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.835+0000 7fc2f2d9d700 1 -- 172.21.15.16:0/6295838 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2f409de00 msgr2=0x7fc2f4138a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:24.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.835+0000 7fc2f2d9d700 1 --2- 172.21.15.16:0/6295838 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2f409de00 0x7fc2f4138a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:24.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.835+0000 7fc2f2d9d700 1 -- 172.21.15.16:0/6295838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2f406a770 msgr2=0x7fc2f41384d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:24.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.835+0000 7fc2f2d9d700 1 --2- 172.21.15.16:0/6295838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2f406a770 0x7fc2f41384d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:24.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.835+0000 7fc2f2d9d700 1 -- 172.21.15.16:0/6295838 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2e80094a0 con 0x7fc2f413d1f0 2023-09-30T22:36:24.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.835+0000 7fc2f259c700 1 --2- 172.21.15.16:0/6295838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2f406a770 0x7fc2f41384d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:24.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.835+0000 7fc2f1d9b700 1 --2- 172.21.15.16:0/6295838 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2f409de00 0x7fc2f4138a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:24.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.836+0000 7fc2f2d9d700 1 --2- 172.21.15.16:0/6295838 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2f413d1f0 0x7fc2f4138f50 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec00efc0 tx=0x7fc2ec00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:24.840 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.836+0000 7fc2e2ffd700 1 -- 172.21.15.16:0/6295838 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec00cc90 con 0x7fc2f413d1f0 2023-09-30T22:36:24.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.836+0000 7fc2e2ffd700 1 -- 172.21.15.16:0/6295838 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2ec00f040 con 0x7fc2f413d1f0 2023-09-30T22:36:24.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.836+0000 7fc2f8e22700 1 -- 172.21.15.16:0/6295838 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2f409f1a0 con 0x7fc2f413d1f0 2023-09-30T22:36:24.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.836+0000 7fc2e2ffd700 1 -- 172.21.15.16:0/6295838 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec0146c0 con 0x7fc2f413d1f0 2023-09-30T22:36:24.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.836+0000 7fc2f8e22700 1 -- 172.21.15.16:0/6295838 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2f413aa70 con 0x7fc2f413d1f0 2023-09-30T22:36:24.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.837+0000 7fc2e2ffd700 1 -- 172.21.15.16:0/6295838 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc2ec014820 con 0x7fc2f413d1f0 2023-09-30T22:36:24.841 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.837+0000 7fc2f8e22700 1 -- 172.21.15.16:0/6295838 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2f411cc30 con 0x7fc2f413d1f0 2023-09-30T22:36:24.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.838+0000 7fc2e2ffd700 1 -- 172.21.15.16:0/6295838 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fc2ec0401d0 con 0x7fc2f413d1f0 2023-09-30T22:36:24.842 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.842+0000 7fc2e2ffd700 1 -- 172.21.15.16:0/6295838 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc2ec007900 con 0x7fc2f413d1f0 2023-09-30T22:36:24.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.879+0000 7f9272039700 1 Processor -- start 2023-09-30T22:36:24.879 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.879+0000 7f9272039700 1 -- start start 2023-09-30T22:36:24.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.880+0000 7f9272039700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f926c09f410 0x7f926c136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:24.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.880+0000 7f9272039700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f926c136fe0 0x7f926c1393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:24.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.880+0000 7f9272039700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f926c070940 con 0x7f926c09e930 2023-09-30T22:36:24.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.880+0000 7f9272039700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f926c06fd60 con 0x7f926c09f410 2023-09-30T22:36:24.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.880+0000 7f9272039700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f926c06fa80 con 0x7f926c136fe0 2023-09-30T22:36:24.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.880+0000 7f926b7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f926c09e930 0x7f926c09ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60274/0 (socket says 172.21.15.16:60274) 2023-09-30T22:36:24.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.880+0000 7f926affd700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f926c09f410 0x7f926c136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:24.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.880+0000 7f926b7fe700 1 -- 172.21.15.16:0/3523768987 learned_addr learned my addr 172.21.15.16:0/3523768987 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:36:24.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.880+0000 7f926bfff700 1 --2- 172.21.15.16:0/3523768987 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f926c136fe0 0x7f926c1393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:24.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.880+0000 7f926a7fc700 1 -- 172.21.15.16:0/3523768987 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4137200460 0 0) 0x7f926c070940 con 0x7f926c09e930 2023-09-30T22:36:24.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.880+0000 7f926a7fc700 1 -- 172.21.15.16:0/3523768987 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92540035b0 con 0x7f926c09e930 2023-09-30T22:36:24.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.881+0000 7f926bfff700 1 -- 172.21.15.16:0/3523768987 >> v2:172.21.15.16:3300/0 conn(0x7f926c09f410 msgr2=0x7f926c136aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:24.882 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.881+0000 7f926bfff700 1 --2- 172.21.15.16:0/3523768987 >> v2:172.21.15.16:3300/0 conn(0x7f926c09f410 0x7f926c136aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:24.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.881+0000 7f926bfff700 1 -- 172.21.15.16:0/3523768987 >> v1:172.21.15.16:6789/0 conn(0x7f926c09e930 legacy=0x7f926c09ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:24.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.881+0000 7f926bfff700 1 -- 172.21.15.16:0/3523768987 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f926c09fb60 con 0x7f926c136fe0 2023-09-30T22:36:24.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.881+0000 7f926bfff700 1 --2- 172.21.15.16:0/3523768987 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f926c136fe0 0x7f926c1393c0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f9264009d90 tx=0x7f926400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17e452033df7d9ac server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:24.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.881+0000 7f926a7fc700 1 -- 172.21.15.16:0/3523768987 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f926400e070 con 0x7f926c136fe0 2023-09-30T22:36:24.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.881+0000 7f926a7fc700 1 -- 172.21.15.16:0/3523768987 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f926400bb20 con 0x7f926c136fe0 2023-09-30T22:36:24.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.881+0000 7f926a7fc700 1 -- 172.21.15.16:0/3523768987 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f926400be90 con 0x7f926c136fe0 2023-09-30T22:36:24.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.882+0000 7f9272039700 1 -- 172.21.15.16:0/3523768987 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f926c136fe0 msgr2=0x7f926c1393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:24.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.882+0000 7f9272039700 1 --2- 172.21.15.16:0/3523768987 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f926c136fe0 0x7f926c1393c0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f9264009d90 tx=0x7f926400b330 comp rx=0 tx=0).stop 2023-09-30T22:36:24.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.882+0000 7f9272039700 1 -- 172.21.15.16:0/3523768987 shutdown_connections 2023-09-30T22:36:24.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.882+0000 7f9272039700 1 --2- 172.21.15.16:0/3523768987 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f926c136fe0 0x7f926c1393c0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:24.883 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.882+0000 7f9272039700 1 --2- 172.21.15.16:0/3523768987 >> v2:172.21.15.16:3300/0 conn(0x7f926c09f410 0x7f926c136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:24.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.882+0000 7f9272039700 1 -- 172.21.15.16:0/3523768987 >> 172.21.15.16:0/3523768987 conn(0x7f926c0a4210 msgr2=0x7f926c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:24.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.882+0000 7f9272039700 1 -- 172.21.15.16:0/3523768987 shutdown_connections 2023-09-30T22:36:24.884 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.882+0000 7f9272039700 1 -- 172.21.15.16:0/3523768987 wait complete. 2023-09-30T22:36:24.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.883+0000 7f9272039700 1 Processor -- start 2023-09-30T22:36:24.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.883+0000 7f9272039700 1 -- start start 2023-09-30T22:36:24.885 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.884+0000 7f9272039700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f926c09e930 0x7f926c11f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:24.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.884+0000 7f9272039700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f926c09f410 0x7f926c11fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:24.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.884+0000 7f9272039700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f926c136fe0 0x7f926c120110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:24.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.884+0000 7f9272039700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926c1218b0 con 0x7f926c09e930 2023-09-30T22:36:24.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.884+0000 7f9272039700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f926c123170 con 0x7f926c09f410 2023-09-30T22:36:24.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.884+0000 7f9272039700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f926c0a32f0 con 0x7f926c136fe0 2023-09-30T22:36:24.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.884+0000 7f926affd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f926c09f410 0x7f926c11fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:24.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.884+0000 7f926affd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f926c09f410 0x7f926c11fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38478/0 (socket says 172.21.15.16:38478) 2023-09-30T22:36:24.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.884+0000 7f926affd700 1 -- 172.21.15.16:0/3593418097 learned_addr learned my addr 172.21.15.16:0/3593418097 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:24.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.884+0000 7f926b7fe700 1 --2- 172.21.15.16:0/3593418097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f926c09e930 0x7f926c11f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:24.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.884+0000 7f926bfff700 1 --2- 172.21.15.16:0/3593418097 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f926c136fe0 0x7f926c120110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:24.886 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.884+0000 7f926affd700 1 -- 172.21.15.16:0/3593418097 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f926c136fe0 msgr2=0x7f926c120110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:24.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.884+0000 7f926affd700 1 --2- 172.21.15.16:0/3593418097 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f926c136fe0 0x7f926c120110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:24.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.884+0000 7f926affd700 1 -- 172.21.15.16:0/3593418097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f926c09e930 msgr2=0x7f926c11f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:24.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.884+0000 7f926affd700 1 --2- 172.21.15.16:0/3593418097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f926c09e930 0x7f926c11f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:24.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.884+0000 7f926affd700 1 -- 172.21.15.16:0/3593418097 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9264009a70 con 0x7f926c09f410 2023-09-30T22:36:24.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.885+0000 7f926bfff700 1 --2- 172.21.15.16:0/3593418097 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f926c136fe0 0x7f926c120110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:24.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.885+0000 7f926affd700 1 --2- 172.21.15.16:0/3593418097 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f926c09f410 0x7f926c11fbd0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f926000e9b0 tx=0x7f926000ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:24.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.885+0000 7f9253fff700 1 -- 172.21.15.16:0/3593418097 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9260009710 con 0x7f926c09f410 2023-09-30T22:36:24.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.885+0000 7f9253fff700 1 -- 172.21.15.16:0/3593418097 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f926000ee50 con 0x7f926c09f410 2023-09-30T22:36:24.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.885+0000 7f9272039700 1 -- 172.21.15.16:0/3593418097 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f926c122bd0 con 0x7f926c09f410 2023-09-30T22:36:24.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.885+0000 7f9253fff700 1 -- 172.21.15.16:0/3593418097 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9260013a60 con 0x7f926c09f410 2023-09-30T22:36:24.887 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.885+0000 7f9272039700 1 -- 172.21.15.16:0/3593418097 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f926c121a50 con 0x7f926c09f410 2023-09-30T22:36:24.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.886+0000 7f9253fff700 1 -- 172.21.15.16:0/3593418097 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9260013bc0 con 0x7f926c09f410 2023-09-30T22:36:24.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.886+0000 7f9253fff700 1 -- 172.21.15.16:0/3593418097 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f9260043560 con 0x7f926c09f410 2023-09-30T22:36:24.888 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.886+0000 7f9272039700 1 -- 172.21.15.16:0/3593418097 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f926c135540 con 0x7f926c09f410 2023-09-30T22:36:24.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:24.892+0000 7f9253fff700 1 -- 172.21.15.16:0/3593418097 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9260020070 con 0x7f926c09f410 2023-09-30T22:36:25.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:25.001+0000 7fcd83dbd700 1 -- 172.21.15.16:0/3275937703 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fcd64005860 con 0x7fcd7c121f60 2023-09-30T22:36:25.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:25.002+0000 7fcd727fc700 1 -- 172.21.15.16:0/3275937703 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcd74012340 con 0x7fcd7c121f60 2023-09-30T22:36:25.002 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:36:25.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:25.004+0000 7fcd83dbd700 1 -- 172.21.15.16:0/3275937703 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd7c121f60 msgr2=0x7fcd7c1d2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:25.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:25.004+0000 7fcd83dbd700 1 --2- 172.21.15.16:0/3275937703 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd7c121f60 0x7fcd7c1d2d20 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fcd7400efc0 tx=0x7fcd7400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:36:25.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:25.004+0000 7fcd83dbd700 1 -- 172.21.15.16:0/3275937703 shutdown_connections 2023-09-30T22:36:25.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:25.004+0000 7fcd83dbd700 1 --2- 172.21.15.16:0/3275937703 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd7c06e570 0x7fcd7c1214e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:25.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:25.004+0000 7fcd83dbd700 1 --2- 172.21.15.16:0/3275937703 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd7c121f60 0x7fcd7c1d2d20 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:25.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:25.004+0000 7fcd83dbd700 1 --2- 172.21.15.16:0/3275937703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd7c134610 0x7fcd7c121a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:25.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:25.004+0000 7fcd83dbd700 1 -- 172.21.15.16:0/3275937703 >> 172.21.15.16:0/3275937703 conn(0x7fcd7c0a2aa0 msgr2=0x7fcd7c1bf2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:25.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:25.004+0000 7fcd83dbd700 1 -- 172.21.15.16:0/3275937703 shutdown_connections 2023-09-30T22:36:25.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:25.004+0000 7fcd83dbd700 1 -- 172.21.15.16:0/3275937703 wait complete. 2023-09-30T22:36:25.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:25.015+0000 7f51d73aa700 1 -- 172.21.15.16:0/1872261218 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f51d0139ff0 con 0x7f51d006e570 2023-09-30T22:36:25.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:25.015+0000 7f51c5ffb700 1 -- 172.21.15.16:0/1872261218 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f51d0139ff0 con 0x7f51d006e570 2023-09-30T22:36:25.016 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:36:25.019 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:36:25.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:25.019+0000 7f51d73aa700 1 -- 172.21.15.16:0/1872261218 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f51d006e570 msgr2=0x7f51d0136480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:25.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:25.019+0000 7f51d73aa700 1 --2- 172.21.15.16:0/1872261218 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f51d006e570 0x7f51d0136480 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f51cc00c700 tx=0x7f51cc00ca10 comp rx=0 tx=0).stop 2023-09-30T22:36:25.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:25.019+0000 7f51d73aa700 1 -- 172.21.15.16:0/1872261218 shutdown_connections 2023-09-30T22:36:25.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:25.019+0000 7f51d73aa700 1 --2- 172.21.15.16:0/1872261218 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f51d006e570 0x7f51d0136480 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:25.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:25.019+0000 7f51d73aa700 1 --2- 172.21.15.16:0/1872261218 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f51d013a4a0 0x7f51d0136f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:25.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:25.019+0000 7f51d73aa700 1 --2- 172.21.15.16:0/1872261218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f51d006f050 0x7f51d01369c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:25.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:25.019+0000 7f51d73aa700 1 -- 172.21.15.16:0/1872261218 >> 172.21.15.16:0/1872261218 conn(0x7f51d00a2aa0 msgr2=0x7f51d009c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:25.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:25.019+0000 7f51d73aa700 1 -- 172.21.15.16:0/1872261218 shutdown_connections 2023-09-30T22:36:25.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:25.019+0000 7f51d73aa700 1 -- 172.21.15.16:0/1872261218 wait complete. 2023-09-30T22:36:25.038 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:36:25.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:25.053+0000 7fc2f8e22700 1 -- 172.21.15.16:0/6295838 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc2f4074ea0 con 0x7fc2f413d1f0 2023-09-30T22:36:25.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:25.053+0000 7fc2e2ffd700 1 -- 172.21.15.16:0/6295838 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc2ec018020 con 0x7fc2f413d1f0 2023-09-30T22:36:25.054 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:36:25.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:25.055+0000 7fc2f8e22700 1 -- 172.21.15.16:0/6295838 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2f413d1f0 msgr2=0x7fc2f4138f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:25.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:25.055+0000 7fc2f8e22700 1 --2- 172.21.15.16:0/6295838 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2f413d1f0 0x7fc2f4138f50 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec00efc0 tx=0x7fc2ec00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:36:25.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:25.055+0000 7fc2f8e22700 1 -- 172.21.15.16:0/6295838 shutdown_connections 2023-09-30T22:36:25.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:25.056+0000 7fc2f8e22700 1 --2- 172.21.15.16:0/6295838 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc2f413d1f0 0x7fc2f4138f50 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:25.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:25.056+0000 7fc2f8e22700 1 --2- 172.21.15.16:0/6295838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2f406a770 0x7fc2f41384d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:25.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:25.056+0000 7fc2f8e22700 1 --2- 172.21.15.16:0/6295838 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc2f409de00 0x7fc2f4138a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:25.056 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:25.056+0000 7fc2f8e22700 1 -- 172.21.15.16:0/6295838 >> 172.21.15.16:0/6295838 conn(0x7fc2f4066f50 msgr2=0x7fc2f40a2010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:25.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:25.056+0000 7fc2f8e22700 1 -- 172.21.15.16:0/6295838 shutdown_connections 2023-09-30T22:36:25.057 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:25.056+0000 7fc2f8e22700 1 -- 172.21.15.16:0/6295838 wait complete. 2023-09-30T22:36:25.070 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:36:25.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:25.128+0000 7f9272039700 1 -- 172.21.15.16:0/3593418097 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f926c074ea0 con 0x7f926c09f410 2023-09-30T22:36:25.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:25.128+0000 7f9253fff700 1 -- 172.21.15.16:0/3593418097 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f926001b440 con 0x7f926c09f410 2023-09-30T22:36:25.129 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:36:25.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:25.130+0000 7f9272039700 1 -- 172.21.15.16:0/3593418097 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f926c09f410 msgr2=0x7f926c11fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:25.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:25.130+0000 7f9272039700 1 --2- 172.21.15.16:0/3593418097 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f926c09f410 0x7f926c11fbd0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f926000e9b0 tx=0x7f926000ecc0 comp rx=0 tx=0).stop 2023-09-30T22:36:25.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:25.130+0000 7f9272039700 1 -- 172.21.15.16:0/3593418097 shutdown_connections 2023-09-30T22:36:25.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:25.130+0000 7f9272039700 1 --2- 172.21.15.16:0/3593418097 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f926c136fe0 0x7f926c120110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:25.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:25.130+0000 7f9272039700 1 --2- 172.21.15.16:0/3593418097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f926c09e930 0x7f926c11f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:25.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:25.130+0000 7f9272039700 1 --2- 172.21.15.16:0/3593418097 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f926c09f410 0x7f926c11fbd0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:25.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:25.130+0000 7f9272039700 1 -- 172.21.15.16:0/3593418097 >> 172.21.15.16:0/3593418097 conn(0x7f926c0a4210 msgr2=0x7f926c135ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:25.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:25.130+0000 7f9272039700 1 -- 172.21.15.16:0/3593418097 shutdown_connections 2023-09-30T22:36:25.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:25.130+0000 7f9272039700 1 -- 172.21.15.16:0/3593418097 wait complete. 2023-09-30T22:36:25.144 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:36:26.020 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:36:26.039 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:36:26.071 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:36:26.145 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:36:26.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.174+0000 7fe84451d700 1 Processor -- start 2023-09-30T22:36:26.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.174+0000 7fe84451d700 1 -- start start 2023-09-30T22:36:26.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.175+0000 7fe84451d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe83c1338a0 0x7fe83c133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:26.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.175+0000 7fe84451d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe83c09e930 0x7fe83c13b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:26.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.175+0000 7fe84451d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe83c06ef30 con 0x7fe83c09de50 2023-09-30T22:36:26.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.175+0000 7fe84451d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe83c06fef0 con 0x7fe83c1338a0 2023-09-30T22:36:26.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.175+0000 7fe84451d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe83c1334b0 con 0x7fe83c09e930 2023-09-30T22:36:26.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.175+0000 7fe8422b9700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe83c1338a0 0x7fe83c133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:26.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.175+0000 7fe8422b9700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe83c1338a0 0x7fe83c133c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42184/0 (socket says 172.21.15.16:42184) 2023-09-30T22:36:26.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.175+0000 7fe8422b9700 1 -- 172.21.15.16:0/3289225558 learned_addr learned my addr 172.21.15.16:0/3289225558 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:26.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.175+0000 7fe842aba700 1 --2- 172.21.15.16:0/3289225558 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe83c09e930 0x7fe83c13b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:26.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.175+0000 7fe8422b9700 1 -- 172.21.15.16:0/3289225558 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe83c09e930 msgr2=0x7fe83c13b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:26.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.175+0000 7fe8422b9700 1 --2- 172.21.15.16:0/3289225558 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe83c09e930 0x7fe83c13b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:26.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.175+0000 7fe8422b9700 1 -- 172.21.15.16:0/3289225558 >> v1:172.21.15.16:6789/0 conn(0x7fe83c09de50 legacy=0x7fe83c09e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:26.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.175+0000 7fe8422b9700 1 -- 172.21.15.16:0/3289225558 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe83c06fc30 con 0x7fe83c1338a0 2023-09-30T22:36:26.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.176+0000 7fe8422b9700 1 --2- 172.21.15.16:0/3289225558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe83c1338a0 0x7fe83c133c70 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fe8380098f0 tx=0x7fe838009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f908bef17656a5b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:26.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.176+0000 7fe8412b7700 1 -- 172.21.15.16:0/3289225558 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe838011830 con 0x7fe83c1338a0 2023-09-30T22:36:26.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.176+0000 7fe8412b7700 1 -- 172.21.15.16:0/3289225558 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe838011990 con 0x7fe83c1338a0 2023-09-30T22:36:26.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.176+0000 7fe8412b7700 1 -- 172.21.15.16:0/3289225558 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe838011d20 con 0x7fe83c1338a0 2023-09-30T22:36:26.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.177+0000 7fe84451d700 1 -- 172.21.15.16:0/3289225558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe83c1338a0 msgr2=0x7fe83c133c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:26.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.177+0000 7fe84451d700 1 --2- 172.21.15.16:0/3289225558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe83c1338a0 0x7fe83c133c70 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fe8380098f0 tx=0x7fe838009c00 comp rx=0 tx=0).stop 2023-09-30T22:36:26.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.177+0000 7fe84451d700 1 -- 172.21.15.16:0/3289225558 shutdown_connections 2023-09-30T22:36:26.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.177+0000 7fe84451d700 1 --2- 172.21.15.16:0/3289225558 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe83c09e930 0x7fe83c13b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:26.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.177+0000 7fe84451d700 1 --2- 172.21.15.16:0/3289225558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe83c1338a0 0x7fe83c133c70 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:26.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.177+0000 7fe84451d700 1 -- 172.21.15.16:0/3289225558 >> 172.21.15.16:0/3289225558 conn(0x7fe83c0a2aa0 msgr2=0x7fe83c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:26.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.177+0000 7fe84451d700 1 -- 172.21.15.16:0/3289225558 shutdown_connections 2023-09-30T22:36:26.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.177+0000 7fe84451d700 1 -- 172.21.15.16:0/3289225558 wait complete. 2023-09-30T22:36:26.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.178+0000 7fe84451d700 1 Processor -- start 2023-09-30T22:36:26.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.178+0000 7fe84451d700 1 -- start start 2023-09-30T22:36:26.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.179+0000 7fe84451d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe83c09de50 0x7fe83c11eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:26.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.179+0000 7fe84451d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe83c09e930 0x7fe83c11f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:26.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.179+0000 7fe84451d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe83c1338a0 0x7fe83c11f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:26.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.179+0000 7fe84451d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe83c121ea0 con 0x7fe83c09e930 2023-09-30T22:36:26.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.179+0000 7fe84451d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe83c122080 con 0x7fe83c09de50 2023-09-30T22:36:26.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.179+0000 7fe84451d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe83c12f690 con 0x7fe83c1338a0 2023-09-30T22:36:26.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.179+0000 7fe8422b9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe83c09de50 0x7fe83c11eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:26.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.179+0000 7fe841ab8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe83c09e930 0x7fe83c11f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:26.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.179+0000 7fe8422b9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe83c09de50 0x7fe83c11eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:38502/0 (socket says 172.21.15.16:38502) 2023-09-30T22:36:26.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.179+0000 7fe8422b9700 1 -- 172.21.15.16:0/4179547444 learned_addr learned my addr 172.21.15.16:0/4179547444 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:26.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.179+0000 7fe842aba700 1 --2- 172.21.15.16:0/4179547444 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe83c1338a0 0x7fe83c11f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:26.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.180+0000 7fe841ab8700 1 -- 172.21.15.16:0/4179547444 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe83c1338a0 msgr2=0x7fe83c11f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:26.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.180+0000 7fe841ab8700 1 --2- 172.21.15.16:0/4179547444 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe83c1338a0 0x7fe83c11f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:26.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.180+0000 7fe841ab8700 1 -- 172.21.15.16:0/4179547444 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe83c09de50 msgr2=0x7fe83c11eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:26.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.180+0000 7fe841ab8700 1 --2- 172.21.15.16:0/4179547444 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe83c09de50 0x7fe83c11eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:26.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.180+0000 7fe841ab8700 1 -- 172.21.15.16:0/4179547444 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe838009550 con 0x7fe83c09e930 2023-09-30T22:36:26.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.180+0000 7fe841ab8700 1 --2- 172.21.15.16:0/4179547444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe83c09e930 0x7fe83c11f0e0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fe82c00c400 tx=0x7fe82c00c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:26.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.180+0000 7fe832ffd700 1 -- 172.21.15.16:0/4179547444 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe82c0052d0 con 0x7fe83c09e930 2023-09-30T22:36:26.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.180+0000 7fe832ffd700 1 -- 172.21.15.16:0/4179547444 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe82c005430 con 0x7fe83c09e930 2023-09-30T22:36:26.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.180+0000 7fe84451d700 1 -- 172.21.15.16:0/4179547444 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe83c0a1060 con 0x7fe83c09e930 2023-09-30T22:36:26.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.180+0000 7fe832ffd700 1 -- 172.21.15.16:0/4179547444 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe82c0038a0 con 0x7fe83c09e930 2023-09-30T22:36:26.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.180+0000 7fe84451d700 1 -- 172.21.15.16:0/4179547444 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe83c122340 con 0x7fe83c09e930 2023-09-30T22:36:26.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.181+0000 7fe832ffd700 1 -- 172.21.15.16:0/4179547444 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe82c003a00 con 0x7fe83c09e930 2023-09-30T22:36:26.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.181+0000 7fe832ffd700 1 -- 172.21.15.16:0/4179547444 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fe82c03e960 con 0x7fe83c09e930 2023-09-30T22:36:26.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.182+0000 7fe84451d700 1 -- 172.21.15.16:0/4179547444 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8240052e0 con 0x7fe83c09e930 2023-09-30T22:36:26.187 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.187+0000 7fe832ffd700 1 -- 172.21.15.16:0/4179547444 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe82c017020 con 0x7fe83c09e930 2023-09-30T22:36:26.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.223+0000 7f1941fe5700 1 Processor -- start 2023-09-30T22:36:26.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.223+0000 7f1941fe5700 1 -- start start 2023-09-30T22:36:26.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.224+0000 7f1941fe5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f193c09d6f0 0x7f193c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:26.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.224+0000 7f1941fe5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f193c09e000 0x7f193c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:26.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.224+0000 7f1941fe5700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f193c133f30 0x7f193c1343d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:26.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.224+0000 7f1941fe5700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f193c09b0f0 con 0x7f193c133f30 2023-09-30T22:36:26.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.224+0000 7f1941fe5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f193c09d2f0 con 0x7f193c09e000 2023-09-30T22:36:26.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.224+0000 7f1941fe5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f193c09a810 con 0x7f193c09d6f0 2023-09-30T22:36:26.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.224+0000 7f193b7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f193c09d6f0 0x7f193c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:26.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.224+0000 7f193b7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f193c09d6f0 0x7f193c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:57404/0 (socket says 172.21.15.16:57404) 2023-09-30T22:36:26.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.224+0000 7f193b7fe700 1 -- 172.21.15.16:0/52018451 learned_addr learned my addr 172.21.15.16:0/52018451 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:26.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.225+0000 7f193bfff700 1 --2- 172.21.15.16:0/52018451 >> v2:172.21.15.16:3300/0 conn(0x7f193c133f30 0x7f193c1343d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:26.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.225+0000 7f193affd700 1 --2- 172.21.15.16:0/52018451 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f193c09e000 0x7f193c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:26.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.225+0000 7f193bfff700 1 -- 172.21.15.16:0/52018451 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f193c09d6f0 msgr2=0x7f193c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:26.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.225+0000 7f193bfff700 1 --2- 172.21.15.16:0/52018451 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f193c09d6f0 0x7f193c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:26.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.225+0000 7f193bfff700 1 -- 172.21.15.16:0/52018451 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f193c09e000 msgr2=0x7f193c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:26.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.225+0000 7f193bfff700 1 --2- 172.21.15.16:0/52018451 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f193c09e000 0x7f193c1339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:26.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.225+0000 7f193bfff700 1 -- 172.21.15.16:0/52018451 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f193c09a470 con 0x7f193c133f30 2023-09-30T22:36:26.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.225+0000 7f193bfff700 1 --2- 172.21.15.16:0/52018451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f193c133f30 0x7f193c1343d0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f1934009d90 tx=0x7f193400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f28dc98cab46d9e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:26.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.225+0000 7f193a7fc700 1 -- 172.21.15.16:0/52018451 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193400e070 con 0x7f193c133f30 2023-09-30T22:36:26.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.225+0000 7f193a7fc700 1 -- 172.21.15.16:0/52018451 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f193400bb20 con 0x7f193c133f30 2023-09-30T22:36:26.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.225+0000 7f193a7fc700 1 -- 172.21.15.16:0/52018451 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1934012430 con 0x7f193c133f30 2023-09-30T22:36:26.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.226+0000 7f1941fe5700 1 -- 172.21.15.16:0/52018451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f193c133f30 msgr2=0x7f193c1343d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:26.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.226+0000 7f1941fe5700 1 --2- 172.21.15.16:0/52018451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f193c133f30 0x7f193c1343d0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f1934009d90 tx=0x7f193400b330 comp rx=0 tx=0).stop 2023-09-30T22:36:26.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.226+0000 7f1941fe5700 1 -- 172.21.15.16:0/52018451 shutdown_connections 2023-09-30T22:36:26.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.226+0000 7f1941fe5700 1 --2- 172.21.15.16:0/52018451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f193c133f30 0x7f193c1343d0 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:26.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.226+0000 7f1941fe5700 1 --2- 172.21.15.16:0/52018451 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f193c09d6f0 0x7f193c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:26.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.226+0000 7f1941fe5700 1 --2- 172.21.15.16:0/52018451 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f193c09e000 0x7f193c1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:26.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.226+0000 7f1941fe5700 1 -- 172.21.15.16:0/52018451 >> 172.21.15.16:0/52018451 conn(0x7f193c066f50 msgr2=0x7f193c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:26.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.226+0000 7f1941fe5700 1 -- 172.21.15.16:0/52018451 shutdown_connections 2023-09-30T22:36:26.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.226+0000 7f1941fe5700 1 -- 172.21.15.16:0/52018451 wait complete. 2023-09-30T22:36:26.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.228+0000 7f1941fe5700 1 Processor -- start 2023-09-30T22:36:26.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.228+0000 7f1941fe5700 1 -- start start 2023-09-30T22:36:26.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.229+0000 7f1941fe5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f193c09d6f0 0x7f193c1c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:26.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.229+0000 7f1941fe5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f193c09e000 0x7f193c1c8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:26.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.229+0000 7f1941fe5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f193c133f30 0x7f193c1c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:26.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.229+0000 7f1941fe5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f193c120840 con 0x7f193c09d6f0 2023-09-30T22:36:26.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.229+0000 7f1941fe5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f193c09c640 con 0x7f193c133f30 2023-09-30T22:36:26.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.229+0000 7f1941fe5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f193c1335f0 con 0x7f193c09e000 2023-09-30T22:36:26.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.229+0000 7f193b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f193c09d6f0 0x7f193c1c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:26.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.229+0000 7f193b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f193c09d6f0 0x7f193c1c8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42210/0 (socket says 172.21.15.16:42210) 2023-09-30T22:36:26.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.229+0000 7f193b7fe700 1 -- 172.21.15.16:0/509226084 learned_addr learned my addr 172.21.15.16:0/509226084 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:26.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.229+0000 7f193affd700 1 --2- 172.21.15.16:0/509226084 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f193c09e000 0x7f193c1c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:26.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.229+0000 7f193bfff700 1 --2- 172.21.15.16:0/509226084 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f193c133f30 0x7f193c1c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:26.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.229+0000 7f193b7fe700 1 -- 172.21.15.16:0/509226084 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f193c09e000 msgr2=0x7f193c1c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:26.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.229+0000 7f193b7fe700 1 --2- 172.21.15.16:0/509226084 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f193c09e000 0x7f193c1c8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:26.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.230+0000 7f193b7fe700 1 -- 172.21.15.16:0/509226084 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f193c133f30 msgr2=0x7f193c1c8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:26.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.230+0000 7f193b7fe700 1 --2- 172.21.15.16:0/509226084 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f193c133f30 0x7f193c1c8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:26.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.230+0000 7f193b7fe700 1 -- 172.21.15.16:0/509226084 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1934009a70 con 0x7f193c09d6f0 2023-09-30T22:36:26.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.230+0000 7f193b7fe700 1 --2- 172.21.15.16:0/509226084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f193c09d6f0 0x7f193c1c8040 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f192c00e910 tx=0x7f192c00ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:26.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.230+0000 7f1923fff700 1 -- 172.21.15.16:0/509226084 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192c0096a0 con 0x7f193c09d6f0 2023-09-30T22:36:26.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.230+0000 7f1923fff700 1 -- 172.21.15.16:0/509226084 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f192c013680 con 0x7f193c09d6f0 2023-09-30T22:36:26.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.230+0000 7f1923fff700 1 -- 172.21.15.16:0/509226084 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192c0139f0 con 0x7f193c09d6f0 2023-09-30T22:36:26.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.230+0000 7f1941fe5700 1 -- 172.21.15.16:0/509226084 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f193c122030 con 0x7f193c09d6f0 2023-09-30T22:36:26.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.230+0000 7f1941fe5700 1 -- 172.21.15.16:0/509226084 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f193c122d60 con 0x7f193c09d6f0 2023-09-30T22:36:26.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.231+0000 7f1923fff700 1 -- 172.21.15.16:0/509226084 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f192c013b50 con 0x7f193c09d6f0 2023-09-30T22:36:26.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.231+0000 7f1923fff700 1 -- 172.21.15.16:0/509226084 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f192c03f690 con 0x7f193c09d6f0 2023-09-30T22:36:26.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.231+0000 7f1941fe5700 1 -- 172.21.15.16:0/509226084 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f193c074ea0 con 0x7f193c09d6f0 2023-09-30T22:36:26.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.237+0000 7f1923fff700 1 -- 172.21.15.16:0/509226084 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f192c017080 con 0x7f193c09d6f0 2023-09-30T22:36:26.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.265+0000 7f9622eaa700 1 Processor -- start 2023-09-30T22:36:26.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.265+0000 7f9622eaa700 1 -- start start 2023-09-30T22:36:26.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.266+0000 7f9622eaa700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f961c09d6f0 0x7f961c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:26.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.266+0000 7f9622eaa700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f961c09e410 0x7f961c134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:26.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.266+0000 7f9622eaa700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f961c09b120 con 0x7f961c09e000 2023-09-30T22:36:26.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.266+0000 7f9622eaa700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f961c06b460 con 0x7f961c09e410 2023-09-30T22:36:26.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.266+0000 7f9622eaa700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f961c09a7e0 con 0x7f961c09d6f0 2023-09-30T22:36:26.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.266+0000 7f961bfff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f961c09e000 0x7f961c1339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60292/0 (socket says 172.21.15.16:60292) 2023-09-30T22:36:26.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.266+0000 7f961bfff700 1 -- 172.21.15.16:0/1489961334 learned_addr learned my addr 172.21.15.16:0/1489961334 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:36:26.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.266+0000 7f9621447700 1 --2- 172.21.15.16:0/1489961334 >> v2:172.21.15.16:3300/0 conn(0x7f961c09e410 0x7f961c134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:26.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.266+0000 7f9620c46700 1 --2- 172.21.15.16:0/1489961334 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f961c09d6f0 0x7f961c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:26.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.266+0000 7f961b7fe700 1 -- 172.21.15.16:0/1489961334 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3466702044 0 0) 0x7f961c09b120 con 0x7f961c09e000 2023-09-30T22:36:26.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.266+0000 7f961b7fe700 1 -- 172.21.15.16:0/1489961334 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f960c0035b0 con 0x7f961c09e000 2023-09-30T22:36:26.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.266+0000 7f9621447700 1 -- 172.21.15.16:0/1489961334 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f961c09d6f0 msgr2=0x7f961c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:26.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.266+0000 7f9621447700 1 --2- 172.21.15.16:0/1489961334 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f961c09d6f0 0x7f961c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:26.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.266+0000 7f9621447700 1 -- 172.21.15.16:0/1489961334 >> v1:172.21.15.16:6789/0 conn(0x7f961c09e000 legacy=0x7f961c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:26.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.266+0000 7f9621447700 1 -- 172.21.15.16:0/1489961334 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f961c09a500 con 0x7f961c09e410 2023-09-30T22:36:26.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.267+0000 7f9621447700 1 --2- 172.21.15.16:0/1489961334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f961c09e410 0x7f961c134100 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f9614009d90 tx=0x7f961400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=724db352cd3b0c7c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:26.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.267+0000 7f961b7fe700 1 -- 172.21.15.16:0/1489961334 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f961400e070 con 0x7f961c09e410 2023-09-30T22:36:26.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.267+0000 7f961b7fe700 1 -- 172.21.15.16:0/1489961334 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f961400bb20 con 0x7f961c09e410 2023-09-30T22:36:26.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.267+0000 7f961b7fe700 1 -- 172.21.15.16:0/1489961334 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f961400be90 con 0x7f961c09e410 2023-09-30T22:36:26.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.268+0000 7f9622eaa700 1 -- 172.21.15.16:0/1489961334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f961c09e410 msgr2=0x7f961c134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:26.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.268+0000 7f9622eaa700 1 --2- 172.21.15.16:0/1489961334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f961c09e410 0x7f961c134100 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f9614009d90 tx=0x7f961400b330 comp rx=0 tx=0).stop 2023-09-30T22:36:26.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.268+0000 7f9622eaa700 1 -- 172.21.15.16:0/1489961334 shutdown_connections 2023-09-30T22:36:26.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.268+0000 7f9622eaa700 1 --2- 172.21.15.16:0/1489961334 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f961c09e410 0x7f961c134100 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:26.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.268+0000 7f9622eaa700 1 --2- 172.21.15.16:0/1489961334 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f961c09d6f0 0x7f961c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:26.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.268+0000 7f9622eaa700 1 -- 172.21.15.16:0/1489961334 >> 172.21.15.16:0/1489961334 conn(0x7f961c066f50 msgr2=0x7f961c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:26.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.268+0000 7f9622eaa700 1 -- 172.21.15.16:0/1489961334 shutdown_connections 2023-09-30T22:36:26.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.268+0000 7f9622eaa700 1 -- 172.21.15.16:0/1489961334 wait complete. 2023-09-30T22:36:26.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.269+0000 7f9622eaa700 1 Processor -- start 2023-09-30T22:36:26.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.269+0000 7f9622eaa700 1 -- start start 2023-09-30T22:36:26.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.270+0000 7f9622eaa700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f961c09d6f0 0x7f961c1c8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:26.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.270+0000 7f9622eaa700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f961c09e000 0x7f961c1c8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:26.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.270+0000 7f9622eaa700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f961c09e410 0x7f961c1c8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:26.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.270+0000 7f9622eaa700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961c1a7680 con 0x7f961c09e410 2023-09-30T22:36:26.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.270+0000 7f9622eaa700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f961c09c710 con 0x7f961c09d6f0 2023-09-30T22:36:26.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.270+0000 7f9622eaa700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f961c0a2d20 con 0x7f961c09e000 2023-09-30T22:36:26.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.270+0000 7f9621447700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f961c09e410 0x7f961c1c8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:26.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.270+0000 7f9621447700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f961c09e410 0x7f961c1c8bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42216/0 (socket says 172.21.15.16:42216) 2023-09-30T22:36:26.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.270+0000 7f9621447700 1 -- 172.21.15.16:0/1124321457 learned_addr learned my addr 172.21.15.16:0/1124321457 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:26.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.271+0000 7f961bfff700 1 --2- 172.21.15.16:0/1124321457 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f961c09e000 0x7f961c1c8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:26.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.271+0000 7f9620c46700 1 --2- 172.21.15.16:0/1124321457 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f961c09d6f0 0x7f961c1c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:26.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.271+0000 7f9621447700 1 -- 172.21.15.16:0/1124321457 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f961c09e000 msgr2=0x7f961c1c8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:26.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.271+0000 7f9621447700 1 --2- 172.21.15.16:0/1124321457 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f961c09e000 0x7f961c1c8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:26.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.271+0000 7f9621447700 1 -- 172.21.15.16:0/1124321457 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f961c09d6f0 msgr2=0x7f961c1c8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:26.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.271+0000 7f9621447700 1 --2- 172.21.15.16:0/1124321457 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f961c09d6f0 0x7f961c1c8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:26.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.271+0000 7f9621447700 1 -- 172.21.15.16:0/1124321457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9614009a70 con 0x7f961c09e410 2023-09-30T22:36:26.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.271+0000 7f9620c46700 1 --2- 172.21.15.16:0/1124321457 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f961c09d6f0 0x7f961c1c8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:36:26.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.271+0000 7f9621447700 1 --2- 172.21.15.16:0/1124321457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f961c09e410 0x7f961c1c8bd0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f9614014040 tx=0x7f9614004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:26.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.271+0000 7f96197fa700 1 -- 172.21.15.16:0/1124321457 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9614007000 con 0x7f961c09e410 2023-09-30T22:36:26.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.271+0000 7f96197fa700 1 -- 172.21.15.16:0/1124321457 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9614012630 con 0x7f961c09e410 2023-09-30T22:36:26.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.271+0000 7f96197fa700 1 -- 172.21.15.16:0/1124321457 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96140129e0 con 0x7f961c09e410 2023-09-30T22:36:26.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.272+0000 7f9622eaa700 1 -- 172.21.15.16:0/1124321457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f961c1a6670 con 0x7f961c09e410 2023-09-30T22:36:26.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.272+0000 7f9622eaa700 1 -- 172.21.15.16:0/1124321457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f961c1a7c80 con 0x7f961c09e410 2023-09-30T22:36:26.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.273+0000 7f96197fa700 1 -- 172.21.15.16:0/1124321457 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9614012b40 con 0x7f961c09e410 2023-09-30T22:36:26.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.273+0000 7f96197fa700 1 -- 172.21.15.16:0/1124321457 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f9614003e10 con 0x7f961c09e410 2023-09-30T22:36:26.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.273+0000 7f9622eaa700 1 -- 172.21.15.16:0/1124321457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f961c09c2b0 con 0x7f961c09e410 2023-09-30T22:36:26.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.282+0000 7f96197fa700 1 -- 172.21.15.16:0/1124321457 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9614046060 con 0x7f961c09e410 2023-09-30T22:36:26.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.305+0000 7f12f126b700 1 Processor -- start 2023-09-30T22:36:26.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.306+0000 7f12f126b700 1 -- start start 2023-09-30T22:36:26.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.306+0000 7f12f126b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12ec0a0a30 0x7f12ec0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:26.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.306+0000 7f12f126b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f12ec134170 0x7f12ec13bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:26.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.306+0000 7f12f126b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12ec09edd0 con 0x7f12ec0a1340 2023-09-30T22:36:26.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.306+0000 7f12f126b700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12ec09e1f0 con 0x7f12ec134170 2023-09-30T22:36:26.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.306+0000 7f12f126b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f12ec09dc50 con 0x7f12ec0a0a30 2023-09-30T22:36:26.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.306+0000 7f12eb7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f12ec134170 0x7f12ec13bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:26.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.306+0000 7f12eaffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12ec0a0a30 0x7f12ec0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:26.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.306+0000 7f12eaffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12ec0a0a30 0x7f12ec0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:57442/0 (socket says 172.21.15.16:57442) 2023-09-30T22:36:26.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.306+0000 7f12eb7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f12ec134170 0x7f12ec13bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42226/0 (socket says 172.21.15.16:42226) 2023-09-30T22:36:26.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.306+0000 7f12eaffd700 1 -- 172.21.15.16:0/3129044208 learned_addr learned my addr 172.21.15.16:0/3129044208 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:26.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.307+0000 7f12eaffd700 1 -- 172.21.15.16:0/3129044208 >> v2:172.21.15.16:3300/0 conn(0x7f12ec134170 msgr2=0x7f12ec13bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:26.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.307+0000 7f12eaffd700 1 --2- 172.21.15.16:0/3129044208 >> v2:172.21.15.16:3300/0 conn(0x7f12ec134170 0x7f12ec13bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:26.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.307+0000 7f12eaffd700 1 -- 172.21.15.16:0/3129044208 >> v1:172.21.15.16:6789/0 conn(0x7f12ec0a1340 legacy=0x7f12ec133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:26.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.307+0000 7f12eaffd700 1 -- 172.21.15.16:0/3129044208 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12ec0a1fe0 con 0x7f12ec0a0a30 2023-09-30T22:36:26.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.307+0000 7f12eaffd700 1 --2- 172.21.15.16:0/3129044208 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12ec0a0a30 0x7f12ec0a0e00 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f12dc0098f0 tx=0x7f12dc009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=290467679c3087ca server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:26.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.307+0000 7f12e9ffb700 1 -- 172.21.15.16:0/3129044208 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12dc011830 con 0x7f12ec0a0a30 2023-09-30T22:36:26.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.307+0000 7f12e9ffb700 1 -- 172.21.15.16:0/3129044208 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12dc011990 con 0x7f12ec0a0a30 2023-09-30T22:36:26.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.307+0000 7f12e9ffb700 1 -- 172.21.15.16:0/3129044208 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12dc011d20 con 0x7f12ec0a0a30 2023-09-30T22:36:26.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.308+0000 7f12f126b700 1 -- 172.21.15.16:0/3129044208 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12ec0a0a30 msgr2=0x7f12ec0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:26.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.308+0000 7f12f126b700 1 --2- 172.21.15.16:0/3129044208 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12ec0a0a30 0x7f12ec0a0e00 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f12dc0098f0 tx=0x7f12dc009c00 comp rx=0 tx=0).stop 2023-09-30T22:36:26.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.308+0000 7f12f126b700 1 -- 172.21.15.16:0/3129044208 shutdown_connections 2023-09-30T22:36:26.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.308+0000 7f12f126b700 1 --2- 172.21.15.16:0/3129044208 >> v2:172.21.15.16:3300/0 conn(0x7f12ec134170 0x7f12ec13bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:26.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.308+0000 7f12f126b700 1 --2- 172.21.15.16:0/3129044208 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12ec0a0a30 0x7f12ec0a0e00 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:26.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.308+0000 7f12f126b700 1 -- 172.21.15.16:0/3129044208 >> 172.21.15.16:0/3129044208 conn(0x7f12ec0a6550 msgr2=0x7f12ec0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:26.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.308+0000 7f12f126b700 1 -- 172.21.15.16:0/3129044208 shutdown_connections 2023-09-30T22:36:26.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.308+0000 7f12f126b700 1 -- 172.21.15.16:0/3129044208 wait complete. 2023-09-30T22:36:26.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.309+0000 7f12f126b700 1 Processor -- start 2023-09-30T22:36:26.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.309+0000 7f12f126b700 1 -- start start 2023-09-30T22:36:26.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.310+0000 7f12f126b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12ec0a0a30 0x7f12ec1c8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:26.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.310+0000 7f12f126b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12ec0a1340 0x7f12ec1c89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:26.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.310+0000 7f12f126b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12ec134170 0x7f12ec1c8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:26.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.310+0000 7f12f126b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12ec1244d0 con 0x7f12ec0a0a30 2023-09-30T22:36:26.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.310+0000 7f12f126b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12ec124da0 con 0x7f12ec0a1340 2023-09-30T22:36:26.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.310+0000 7f12f126b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f12ec1263e0 con 0x7f12ec134170 2023-09-30T22:36:26.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.310+0000 7f12eaffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12ec0a0a30 0x7f12ec1c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:26.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.310+0000 7f12eb7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12ec134170 0x7f12ec1c8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:26.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.310+0000 7f12eaffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12ec0a0a30 0x7f12ec1c8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42238/0 (socket says 172.21.15.16:42238) 2023-09-30T22:36:26.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.310+0000 7f12eaffd700 1 -- 172.21.15.16:0/1940251460 learned_addr learned my addr 172.21.15.16:0/1940251460 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:26.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.310+0000 7f12eaffd700 1 -- 172.21.15.16:0/1940251460 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12ec134170 msgr2=0x7f12ec1c8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:26.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.310+0000 7f12eaffd700 1 --2- 172.21.15.16:0/1940251460 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12ec134170 0x7f12ec1c8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:26.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.310+0000 7f12eaffd700 1 -- 172.21.15.16:0/1940251460 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12ec0a1340 msgr2=0x7f12ec1c89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:26.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.310+0000 7f12eaffd700 1 --2- 172.21.15.16:0/1940251460 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12ec0a1340 0x7f12ec1c89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:26.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.310+0000 7f12eaffd700 1 -- 172.21.15.16:0/1940251460 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12dc009550 con 0x7f12ec0a0a30 2023-09-30T22:36:26.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.310+0000 7f12eaffd700 1 --2- 172.21.15.16:0/1940251460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12ec0a0a30 0x7f12ec1c8460 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f12dc000c00 tx=0x7f12dc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:26.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.311+0000 7f12d37fe700 1 -- 172.21.15.16:0/1940251460 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12dc004480 con 0x7f12ec0a0a30 2023-09-30T22:36:26.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.311+0000 7f12d37fe700 1 -- 172.21.15.16:0/1940251460 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12dc01f070 con 0x7f12ec0a0a30 2023-09-30T22:36:26.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.311+0000 7f12f126b700 1 -- 172.21.15.16:0/1940251460 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12ec124a80 con 0x7f12ec0a0a30 2023-09-30T22:36:26.318 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.311+0000 7f12d37fe700 1 -- 172.21.15.16:0/1940251460 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12dc0038c0 con 0x7f12ec0a0a30 2023-09-30T22:36:26.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.311+0000 7f12f126b700 1 -- 172.21.15.16:0/1940251460 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12ec126100 con 0x7f12ec0a0a30 2023-09-30T22:36:26.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.312+0000 7f12f126b700 1 -- 172.21.15.16:0/1940251460 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12ec074ea0 con 0x7f12ec0a0a30 2023-09-30T22:36:26.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.312+0000 7f12d37fe700 1 -- 172.21.15.16:0/1940251460 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f12dc021030 con 0x7f12ec0a0a30 2023-09-30T22:36:26.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.312+0000 7f12d37fe700 1 -- 172.21.15.16:0/1940251460 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f12dc040ab0 con 0x7f12ec0a0a30 2023-09-30T22:36:26.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.317+0000 7f12d37fe700 1 -- 172.21.15.16:0/1940251460 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f12dc0199a0 con 0x7f12ec0a0a30 2023-09-30T22:36:26.476 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.476+0000 7fe84451d700 1 -- 172.21.15.16:0/4179547444 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe824005590 con 0x7fe83c09e930 2023-09-30T22:36:26.477 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.476+0000 7fe832ffd700 1 -- 172.21.15.16:0/4179547444 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe82c019d20 con 0x7fe83c09e930 2023-09-30T22:36:26.477 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:36:26.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.479+0000 7fe84451d700 1 -- 172.21.15.16:0/4179547444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe83c09e930 msgr2=0x7fe83c11f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:26.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.479+0000 7fe84451d700 1 --2- 172.21.15.16:0/4179547444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe83c09e930 0x7fe83c11f0e0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fe82c00c400 tx=0x7fe82c00c710 comp rx=0 tx=0).stop 2023-09-30T22:36:26.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.479+0000 7fe84451d700 1 -- 172.21.15.16:0/4179547444 shutdown_connections 2023-09-30T22:36:26.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.479+0000 7fe84451d700 1 --2- 172.21.15.16:0/4179547444 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe83c09de50 0x7fe83c11eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:26.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.479+0000 7fe84451d700 1 --2- 172.21.15.16:0/4179547444 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe83c1338a0 0x7fe83c11f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:26.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.479+0000 7fe84451d700 1 --2- 172.21.15.16:0/4179547444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe83c09e930 0x7fe83c11f0e0 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:26.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.479+0000 7fe84451d700 1 -- 172.21.15.16:0/4179547444 >> 172.21.15.16:0/4179547444 conn(0x7fe83c0a2aa0 msgr2=0x7fe83c09ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:26.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.479+0000 7fe84451d700 1 -- 172.21.15.16:0/4179547444 shutdown_connections 2023-09-30T22:36:26.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.479+0000 7fe84451d700 1 -- 172.21.15.16:0/4179547444 wait complete. 2023-09-30T22:36:26.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.496+0000 7f9622eaa700 1 -- 172.21.15.16:0/1124321457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f961c074ea0 con 0x7f961c09e410 2023-09-30T22:36:26.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.496+0000 7f96197fa700 1 -- 172.21.15.16:0/1124321457 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9614026410 con 0x7f961c09e410 2023-09-30T22:36:26.497 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:36:26.498 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:36:26.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.499+0000 7f9622eaa700 1 -- 172.21.15.16:0/1124321457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f961c09e410 msgr2=0x7f961c1c8bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:26.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.499+0000 7f9622eaa700 1 --2- 172.21.15.16:0/1124321457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f961c09e410 0x7f961c1c8bd0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f9614014040 tx=0x7f9614004ad0 comp rx=0 tx=0).stop 2023-09-30T22:36:26.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.499+0000 7f9622eaa700 1 -- 172.21.15.16:0/1124321457 shutdown_connections 2023-09-30T22:36:26.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.499+0000 7f9622eaa700 1 --2- 172.21.15.16:0/1124321457 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f961c09d6f0 0x7f961c1c8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:26.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.499+0000 7f9622eaa700 1 --2- 172.21.15.16:0/1124321457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f961c09e410 0x7f961c1c8bd0 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:26.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.499+0000 7f9622eaa700 1 --2- 172.21.15.16:0/1124321457 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f961c09e000 0x7f961c1c8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:26.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.499+0000 7f9622eaa700 1 -- 172.21.15.16:0/1124321457 >> 172.21.15.16:0/1124321457 conn(0x7f961c066f50 msgr2=0x7f961c1bb9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:26.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.499+0000 7f9622eaa700 1 -- 172.21.15.16:0/1124321457 shutdown_connections 2023-09-30T22:36:26.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.499+0000 7f9622eaa700 1 -- 172.21.15.16:0/1124321457 wait complete. 2023-09-30T22:36:26.514 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:36:26.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.518+0000 7f1941fe5700 1 -- 172.21.15.16:0/509226084 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f193c06c360 con 0x7f193c09d6f0 2023-09-30T22:36:26.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.519+0000 7f1923fff700 1 -- 172.21.15.16:0/509226084 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f192c01bdf0 con 0x7f193c09d6f0 2023-09-30T22:36:26.520 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:36:26.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.521+0000 7f1941fe5700 1 -- 172.21.15.16:0/509226084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f193c09d6f0 msgr2=0x7f193c1c8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:26.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.521+0000 7f1941fe5700 1 --2- 172.21.15.16:0/509226084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f193c09d6f0 0x7f193c1c8040 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f192c00e910 tx=0x7f192c00ec20 comp rx=0 tx=0).stop 2023-09-30T22:36:26.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.521+0000 7f1941fe5700 1 -- 172.21.15.16:0/509226084 shutdown_connections 2023-09-30T22:36:26.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.521+0000 7f1941fe5700 1 --2- 172.21.15.16:0/509226084 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f193c133f30 0x7f193c1c8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:26.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.521+0000 7f1941fe5700 1 --2- 172.21.15.16:0/509226084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f193c09d6f0 0x7f193c1c8040 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:26.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.521+0000 7f1941fe5700 1 --2- 172.21.15.16:0/509226084 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f193c09e000 0x7f193c1c8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:26.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.521+0000 7f1941fe5700 1 -- 172.21.15.16:0/509226084 >> 172.21.15.16:0/509226084 conn(0x7f193c066f50 msgr2=0x7f193c1bbb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:26.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.521+0000 7f1941fe5700 1 -- 172.21.15.16:0/509226084 shutdown_connections 2023-09-30T22:36:26.523 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.522+0000 7f1941fe5700 1 -- 172.21.15.16:0/509226084 wait complete. 2023-09-30T22:36:26.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.524+0000 7f12f126b700 1 -- 172.21.15.16:0/1940251460 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f12ec125da0 con 0x7f12ec0a0a30 2023-09-30T22:36:26.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.524+0000 7f12d37fe700 1 -- 172.21.15.16:0/1940251460 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f12ec125da0 con 0x7f12ec0a0a30 2023-09-30T22:36:26.525 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:36:26.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.527+0000 7f12f126b700 1 -- 172.21.15.16:0/1940251460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12ec0a0a30 msgr2=0x7f12ec1c8460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:26.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.527+0000 7f12f126b700 1 --2- 172.21.15.16:0/1940251460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12ec0a0a30 0x7f12ec1c8460 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f12dc000c00 tx=0x7f12dc004750 comp rx=0 tx=0).stop 2023-09-30T22:36:26.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.527+0000 7f12f126b700 1 -- 172.21.15.16:0/1940251460 shutdown_connections 2023-09-30T22:36:26.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.527+0000 7f12f126b700 1 --2- 172.21.15.16:0/1940251460 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12ec134170 0x7f12ec1c8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:26.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.527+0000 7f12f126b700 1 --2- 172.21.15.16:0/1940251460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12ec0a0a30 0x7f12ec1c8460 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:26.528 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.527+0000 7f12f126b700 1 --2- 172.21.15.16:0/1940251460 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12ec0a1340 0x7f12ec1c89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:26.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.527+0000 7f12f126b700 1 -- 172.21.15.16:0/1940251460 >> 172.21.15.16:0/1940251460 conn(0x7f12ec0a6550 msgr2=0x7f12ec09ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:26.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.527+0000 7f12f126b700 1 -- 172.21.15.16:0/1940251460 shutdown_connections 2023-09-30T22:36:26.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:26.527+0000 7f12f126b700 1 -- 172.21.15.16:0/1940251460 wait complete. 2023-09-30T22:36:26.537 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:36:26.543 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:36:27.499 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:36:27.515 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:36:27.538 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:36:27.543 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:36:27.662 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.662+0000 7fa1642b5700 1 Processor -- start 2023-09-30T22:36:27.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.662+0000 7fa1642b5700 1 -- start start 2023-09-30T22:36:27.663 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.663+0000 7fa1642b5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa15c09d9f0 0x7fa15c136530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:27.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.663+0000 7fa1642b5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa15c136a70 0x7fa15c138e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:27.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.663+0000 7fa1642b5700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa15c09e130 con 0x7fa15c1316c0 2023-09-30T22:36:27.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.663+0000 7fa1642b5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa15c1321a0 con 0x7fa15c09d9f0 2023-09-30T22:36:27.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.663+0000 7fa1642b5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa15c1312b0 con 0x7fa15c136a70 2023-09-30T22:36:27.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.663+0000 7fa162051700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fa15c1316c0 0x7fa15c131a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47852/0 (socket says 172.21.15.16:47852) 2023-09-30T22:36:27.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.663+0000 7fa162852700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa15c136a70 0x7fa15c138e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:27.664 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.663+0000 7fa162051700 1 -- 172.21.15.16:0/3611607201 learned_addr learned my addr 172.21.15.16:0/3611607201 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:36:27.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.663+0000 7fa161850700 1 --2- 172.21.15.16:0/3611607201 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa15c09d9f0 0x7fa15c136530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:27.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.664+0000 7fa161850700 1 -- 172.21.15.16:0/3611607201 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa15c136a70 msgr2=0x7fa15c138e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:27.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.664+0000 7fa161850700 1 --2- 172.21.15.16:0/3611607201 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa15c136a70 0x7fa15c138e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:27.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.664+0000 7fa161850700 1 -- 172.21.15.16:0/3611607201 >> v1:172.21.15.16:6789/0 conn(0x7fa15c1316c0 legacy=0x7fa15c131a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:27.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.664+0000 7fa161850700 1 -- 172.21.15.16:0/3611607201 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa15c09f6e0 con 0x7fa15c09d9f0 2023-09-30T22:36:27.666 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.664+0000 7fa161850700 1 --2- 172.21.15.16:0/3611607201 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa15c09d9f0 0x7fa15c136530 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fa14c009870 tx=0x7fa14c009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=787b35abe0164b8c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:27.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.664+0000 7fa16104f700 1 -- 172.21.15.16:0/3611607201 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14c011830 con 0x7fa15c09d9f0 2023-09-30T22:36:27.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.664+0000 7fa16104f700 1 -- 172.21.15.16:0/3611607201 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa14c011990 con 0x7fa15c09d9f0 2023-09-30T22:36:27.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.664+0000 7fa16104f700 1 -- 172.21.15.16:0/3611607201 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14c011d20 con 0x7fa15c09d9f0 2023-09-30T22:36:27.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.665+0000 7fa1642b5700 1 -- 172.21.15.16:0/3611607201 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa15c09d9f0 msgr2=0x7fa15c136530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:27.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.666+0000 7fa1642b5700 1 --2- 172.21.15.16:0/3611607201 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa15c09d9f0 0x7fa15c136530 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fa14c009870 tx=0x7fa14c009b80 comp rx=0 tx=0).stop 2023-09-30T22:36:27.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.666+0000 7fa1642b5700 1 -- 172.21.15.16:0/3611607201 shutdown_connections 2023-09-30T22:36:27.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.666+0000 7fa1642b5700 1 --2- 172.21.15.16:0/3611607201 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa15c136a70 0x7fa15c138e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:27.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.666+0000 7fa1642b5700 1 --2- 172.21.15.16:0/3611607201 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa15c09d9f0 0x7fa15c136530 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:27.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.666+0000 7fa1642b5700 1 -- 172.21.15.16:0/3611607201 >> 172.21.15.16:0/3611607201 conn(0x7fa15c066f50 msgr2=0x7fa15c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:27.667 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.666+0000 7fa1642b5700 1 -- 172.21.15.16:0/3611607201 shutdown_connections 2023-09-30T22:36:27.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.666+0000 7fa1642b5700 1 -- 172.21.15.16:0/3611607201 wait complete. 2023-09-30T22:36:27.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.667+0000 7fa1642b5700 1 Processor -- start 2023-09-30T22:36:27.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.667+0000 7fa1642b5700 1 -- start start 2023-09-30T22:36:27.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.668+0000 7fa1642b5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa15c09d9f0 0x7fa15c070f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:27.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.668+0000 7fa1642b5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa15c1316c0 0x7fa15c0714b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:27.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.668+0000 7fa1642b5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa15c136a70 0x7fa15c0719f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:27.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.668+0000 7fa1642b5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa15c06fbe0 con 0x7fa15c1316c0 2023-09-30T22:36:27.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.668+0000 7fa1642b5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa15c070390 con 0x7fa15c09d9f0 2023-09-30T22:36:27.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.668+0000 7fa1642b5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa15c136160 con 0x7fa15c136a70 2023-09-30T22:36:27.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.668+0000 7fa162051700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa15c09d9f0 0x7fa15c070f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:27.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.668+0000 7fa162051700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa15c09d9f0 0x7fa15c070f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36008/0 (socket says 172.21.15.16:36008) 2023-09-30T22:36:27.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.668+0000 7fa162051700 1 -- 172.21.15.16:0/2650483067 learned_addr learned my addr 172.21.15.16:0/2650483067 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:27.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.668+0000 7fa162852700 1 --2- 172.21.15.16:0/2650483067 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa15c136a70 0x7fa15c0719f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:27.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.668+0000 7fa161850700 1 --2- 172.21.15.16:0/2650483067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa15c1316c0 0x7fa15c0714b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:27.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.669+0000 7fa162852700 1 -- 172.21.15.16:0/2650483067 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa15c09d9f0 msgr2=0x7fa15c070f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:27.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.669+0000 7fa162852700 1 --2- 172.21.15.16:0/2650483067 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa15c09d9f0 0x7fa15c070f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:27.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.669+0000 7fa162852700 1 -- 172.21.15.16:0/2650483067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa15c1316c0 msgr2=0x7fa15c0714b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:27.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.669+0000 7fa162852700 1 --2- 172.21.15.16:0/2650483067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa15c1316c0 0x7fa15c0714b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:27.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.669+0000 7fa162852700 1 -- 172.21.15.16:0/2650483067 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa14c0094a0 con 0x7fa15c136a70 2023-09-30T22:36:27.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.669+0000 7fa161850700 1 --2- 172.21.15.16:0/2650483067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa15c1316c0 0x7fa15c0714b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:27.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.669+0000 7fa162051700 1 --2- 172.21.15.16:0/2650483067 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa15c09d9f0 0x7fa15c070f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:27.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.669+0000 7fa162852700 1 --2- 172.21.15.16:0/2650483067 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa15c136a70 0x7fa15c0719f0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fa15c12edd0 tx=0x7fa15400ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:27.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.669+0000 7fa152ffd700 1 -- 172.21.15.16:0/2650483067 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15400cc20 con 0x7fa15c136a70 2023-09-30T22:36:27.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.669+0000 7fa152ffd700 1 -- 172.21.15.16:0/2650483067 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa15400f040 con 0x7fa15c136a70 2023-09-30T22:36:27.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.669+0000 7fa1642b5700 1 -- 172.21.15.16:0/2650483067 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa15c06e8e0 con 0x7fa15c136a70 2023-09-30T22:36:27.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.669+0000 7fa152ffd700 1 -- 172.21.15.16:0/2650483067 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa154009e60 con 0x7fa15c136a70 2023-09-30T22:36:27.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.670+0000 7fa1642b5700 1 -- 172.21.15.16:0/2650483067 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa15c070160 con 0x7fa15c136a70 2023-09-30T22:36:27.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.671+0000 7fa152ffd700 1 -- 172.21.15.16:0/2650483067 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa154014480 con 0x7fa15c136a70 2023-09-30T22:36:27.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.671+0000 7fa152ffd700 1 -- 172.21.15.16:0/2650483067 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fa154040120 con 0x7fa15c136a70 2023-09-30T22:36:27.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.671+0000 7fa1642b5700 1 -- 172.21.15.16:0/2650483067 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa15c074ea0 con 0x7fa15c136a70 2023-09-30T22:36:27.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.676+0000 7fa152ffd700 1 -- 172.21.15.16:0/2650483067 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa154018030 con 0x7fa15c136a70 2023-09-30T22:36:27.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.693+0000 7fcd20e61700 1 Processor -- start 2023-09-30T22:36:27.694 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.694+0000 7fcd20e61700 1 -- start start 2023-09-30T22:36:27.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.694+0000 7fcd20e61700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcd1c09de00 0x7fcd1c09e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:27.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.694+0000 7fcd20e61700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd1c13d1f0 0x7fcd1c13f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:27.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.694+0000 7fcd20e61700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd1c09a810 con 0x7fcd1c06a770 2023-09-30T22:36:27.696 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.694+0000 7fcd20e61700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd1c06ab40 con 0x7fcd1c09de00 2023-09-30T22:36:27.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.694+0000 7fcd20e61700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c09ebc0 con 0x7fcd1c13d1f0 2023-09-30T22:36:27.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.695+0000 7fcd1ad9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd1c13d1f0 0x7fcd1c13f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:27.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.695+0000 7fcd1a59c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fcd1c06a770 0x7fcd1c09d6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47862/0 (socket says 172.21.15.16:47862) 2023-09-30T22:36:27.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.695+0000 7fcd1a59c700 1 -- 172.21.15.16:0/141739327 learned_addr learned my addr 172.21.15.16:0/141739327 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:36:27.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.695+0000 7fcd19d9b700 1 --2- 172.21.15.16:0/141739327 >> v2:172.21.15.16:3300/0 conn(0x7fcd1c09de00 0x7fcd1c09e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:27.697 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.695+0000 7fcd1ad9d700 1 -- 172.21.15.16:0/141739327 >> v2:172.21.15.16:3300/0 conn(0x7fcd1c09de00 msgr2=0x7fcd1c09e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:27.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.695+0000 7fcd1ad9d700 1 --2- 172.21.15.16:0/141739327 >> v2:172.21.15.16:3300/0 conn(0x7fcd1c09de00 0x7fcd1c09e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:27.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.695+0000 7fcd1ad9d700 1 -- 172.21.15.16:0/141739327 >> v1:172.21.15.16:6789/0 conn(0x7fcd1c06a770 legacy=0x7fcd1c09d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:27.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.695+0000 7fcd1ad9d700 1 -- 172.21.15.16:0/141739327 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd1c09e920 con 0x7fcd1c13d1f0 2023-09-30T22:36:27.698 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.695+0000 7fcd1ad9d700 1 --2- 172.21.15.16:0/141739327 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd1c13d1f0 0x7fcd1c13f5d0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fcd14009d90 tx=0x7fcd1400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19d78111bf57e15d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:27.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.695+0000 7fcd1959a700 1 -- 172.21.15.16:0/141739327 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1400e070 con 0x7fcd1c13d1f0 2023-09-30T22:36:27.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.695+0000 7fcd1959a700 1 -- 172.21.15.16:0/141739327 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd1400bb20 con 0x7fcd1c13d1f0 2023-09-30T22:36:27.699 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.695+0000 7fcd1959a700 1 -- 172.21.15.16:0/141739327 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd14012430 con 0x7fcd1c13d1f0 2023-09-30T22:36:27.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.696+0000 7fcd20e61700 1 -- 172.21.15.16:0/141739327 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd1c13d1f0 msgr2=0x7fcd1c13f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:27.701 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.696+0000 7fcd20e61700 1 --2- 172.21.15.16:0/141739327 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd1c13d1f0 0x7fcd1c13f5d0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fcd14009d90 tx=0x7fcd1400b330 comp rx=0 tx=0).stop 2023-09-30T22:36:27.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.696+0000 7fcd20e61700 1 -- 172.21.15.16:0/141739327 shutdown_connections 2023-09-30T22:36:27.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.696+0000 7fcd20e61700 1 --2- 172.21.15.16:0/141739327 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd1c13d1f0 0x7fcd1c13f5d0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:27.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.696+0000 7fcd20e61700 1 --2- 172.21.15.16:0/141739327 >> v2:172.21.15.16:3300/0 conn(0x7fcd1c09de00 0x7fcd1c09e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:27.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.696+0000 7fcd20e61700 1 -- 172.21.15.16:0/141739327 >> 172.21.15.16:0/141739327 conn(0x7fcd1c066f50 msgr2=0x7fcd1c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:27.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.696+0000 7fcd20e61700 1 -- 172.21.15.16:0/141739327 shutdown_connections 2023-09-30T22:36:27.702 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.696+0000 7fcd20e61700 1 -- 172.21.15.16:0/141739327 wait complete. 2023-09-30T22:36:27.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.698+0000 7fcd20e61700 1 Processor -- start 2023-09-30T22:36:27.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.698+0000 7fcd20e61700 1 -- start start 2023-09-30T22:36:27.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.698+0000 7fcd20e61700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd1c06a770 0x7fcd1c070ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:27.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.698+0000 7fcd20e61700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd1c09de00 0x7fcd1c071000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:27.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.698+0000 7fcd20e61700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd1c13d1f0 0x7fcd1c071540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:27.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.698+0000 7fcd20e61700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c11c5b0 con 0x7fcd1c09de00 2023-09-30T22:36:27.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.698+0000 7fcd20e61700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c11cdd0 con 0x7fcd1c06a770 2023-09-30T22:36:27.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.698+0000 7fcd20e61700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c06f940 con 0x7fcd1c13d1f0 2023-09-30T22:36:27.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.698+0000 7fcd1ad9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd1c13d1f0 0x7fcd1c071540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:27.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.698+0000 7fcd19d9b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd1c09de00 0x7fcd1c071000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:27.704 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.698+0000 7fcd1ad9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd1c13d1f0 0x7fcd1c071540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:36318/0 (socket says 172.21.15.16:36318) 2023-09-30T22:36:27.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.698+0000 7fcd1ad9d700 1 -- 172.21.15.16:0/1242805070 learned_addr learned my addr 172.21.15.16:0/1242805070 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:27.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.699+0000 7fcd1a59c700 1 --2- 172.21.15.16:0/1242805070 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd1c06a770 0x7fcd1c070ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:27.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.699+0000 7fcd19d9b700 1 -- 172.21.15.16:0/1242805070 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd1c13d1f0 msgr2=0x7fcd1c071540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:27.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.699+0000 7fcd19d9b700 1 --2- 172.21.15.16:0/1242805070 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd1c13d1f0 0x7fcd1c071540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:27.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.699+0000 7fcd19d9b700 1 -- 172.21.15.16:0/1242805070 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd1c06a770 msgr2=0x7fcd1c070ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:27.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.699+0000 7fcd19d9b700 1 --2- 172.21.15.16:0/1242805070 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd1c06a770 0x7fcd1c070ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:27.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.699+0000 7fcd19d9b700 1 -- 172.21.15.16:0/1242805070 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd14009a70 con 0x7fcd1c09de00 2023-09-30T22:36:27.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.699+0000 7fcd1ad9d700 1 --2- 172.21.15.16:0/1242805070 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd1c13d1f0 0x7fcd1c071540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:27.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.699+0000 7fcd19d9b700 1 --2- 172.21.15.16:0/1242805070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd1c09de00 0x7fcd1c071000 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fcd1000e900 tx=0x7fcd1000ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:27.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.699+0000 7fcd0affd700 1 -- 172.21.15.16:0/1242805070 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1000ee50 con 0x7fcd1c09de00 2023-09-30T22:36:27.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.699+0000 7fcd0affd700 1 -- 172.21.15.16:0/1242805070 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd1000c850 con 0x7fcd1c09de00 2023-09-30T22:36:27.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.699+0000 7fcd0affd700 1 -- 172.21.15.16:0/1242805070 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1000cbc0 con 0x7fcd1c09de00 2023-09-30T22:36:27.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.700+0000 7fcd20e61700 1 -- 172.21.15.16:0/1242805070 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd1c06f200 con 0x7fcd1c09de00 2023-09-30T22:36:27.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.700+0000 7fcd20e61700 1 -- 172.21.15.16:0/1242805070 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd1c070670 con 0x7fcd1c09de00 2023-09-30T22:36:27.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.701+0000 7fcd20e61700 1 -- 172.21.15.16:0/1242805070 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd1c13b300 con 0x7fcd1c09de00 2023-09-30T22:36:27.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.701+0000 7fcd0affd700 1 -- 172.21.15.16:0/1242805070 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcd1000cd20 con 0x7fcd1c09de00 2023-09-30T22:36:27.707 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.701+0000 7fcd0affd700 1 -- 172.21.15.16:0/1242805070 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fcd1003f6d0 con 0x7fcd1c09de00 2023-09-30T22:36:27.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.710+0000 7fcd0affd700 1 -- 172.21.15.16:0/1242805070 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcd100232f0 con 0x7fcd1c09de00 2023-09-30T22:36:27.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.711+0000 7f25db9e0700 1 Processor -- start 2023-09-30T22:36:27.713 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.711+0000 7f25db9e0700 1 -- start start 2023-09-30T22:36:27.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.712+0000 7f25db9e0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f25d409d6f0 0x7f25d409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:27.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.712+0000 7f25db9e0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f25d409e000 0x7f25d41339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:27.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.712+0000 7f25db9e0700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25d409b120 con 0x7f25d409e410 2023-09-30T22:36:27.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.712+0000 7f25db9e0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25d406b460 con 0x7f25d409e000 2023-09-30T22:36:27.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.712+0000 7f25db9e0700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f25d409a7e0 con 0x7f25d409d6f0 2023-09-30T22:36:27.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.712+0000 7f25d8f7b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f25d409e000 0x7f25d41339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:27.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.712+0000 7f25d977c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f25d409d6f0 0x7f25d409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:27.714 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.712+0000 7f25d9f7d700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f25d409e410 0x7f25d4133f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47874/0 (socket says 172.21.15.16:47874) 2023-09-30T22:36:27.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.712+0000 7f25d9f7d700 1 -- 172.21.15.16:0/2522359450 learned_addr learned my addr 172.21.15.16:0/2522359450 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:36:27.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.712+0000 7f25d977c700 1 -- 172.21.15.16:0/2522359450 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f25d409e000 msgr2=0x7f25d41339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:27.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.712+0000 7f25d977c700 1 --2- 172.21.15.16:0/2522359450 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f25d409e000 0x7f25d41339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:27.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.712+0000 7f25d977c700 1 -- 172.21.15.16:0/2522359450 >> v1:172.21.15.16:6789/0 conn(0x7f25d409e410 legacy=0x7f25d4133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:27.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.712+0000 7f25d977c700 1 -- 172.21.15.16:0/2522359450 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d409a500 con 0x7f25d409d6f0 2023-09-30T22:36:27.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.712+0000 7f25d8f7b700 1 --2- 172.21.15.16:0/2522359450 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f25d409e000 0x7f25d41339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:27.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.713+0000 7f25d977c700 1 --2- 172.21.15.16:0/2522359450 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f25d409d6f0 0x7f25d409dac0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f25d00098a0 tx=0x7f25d0009bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e96d692e767901c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:27.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.713+0000 7f25cbfff700 1 -- 172.21.15.16:0/2522359450 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d0011830 con 0x7f25d409d6f0 2023-09-30T22:36:27.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.713+0000 7f25cbfff700 1 -- 172.21.15.16:0/2522359450 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25d0011990 con 0x7f25d409d6f0 2023-09-30T22:36:27.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.713+0000 7f25cbfff700 1 -- 172.21.15.16:0/2522359450 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d0011d20 con 0x7f25d409d6f0 2023-09-30T22:36:27.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.714+0000 7f25db9e0700 1 -- 172.21.15.16:0/2522359450 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f25d409d6f0 msgr2=0x7f25d409dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:27.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.714+0000 7f25db9e0700 1 --2- 172.21.15.16:0/2522359450 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f25d409d6f0 0x7f25d409dac0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f25d00098a0 tx=0x7f25d0009bb0 comp rx=0 tx=0).stop 2023-09-30T22:36:27.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.714+0000 7f25db9e0700 1 -- 172.21.15.16:0/2522359450 shutdown_connections 2023-09-30T22:36:27.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.714+0000 7f25db9e0700 1 --2- 172.21.15.16:0/2522359450 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f25d409d6f0 0x7f25d409dac0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:27.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.714+0000 7f25db9e0700 1 --2- 172.21.15.16:0/2522359450 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f25d409e000 0x7f25d41339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:27.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.714+0000 7f25db9e0700 1 -- 172.21.15.16:0/2522359450 >> 172.21.15.16:0/2522359450 conn(0x7f25d4066f50 msgr2=0x7f25d409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:27.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.714+0000 7f25db9e0700 1 -- 172.21.15.16:0/2522359450 shutdown_connections 2023-09-30T22:36:27.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.714+0000 7f25db9e0700 1 -- 172.21.15.16:0/2522359450 wait complete. 2023-09-30T22:36:27.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.715+0000 7f25db9e0700 1 Processor -- start 2023-09-30T22:36:27.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.715+0000 7f25db9e0700 1 -- start start 2023-09-30T22:36:27.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.716+0000 7f25db9e0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25d409d6f0 0x7f25d41a5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:27.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.716+0000 7f25db9e0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f25d409e000 0x7f25d41a63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:27.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.716+0000 7f25db9e0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f25d409e410 0x7f25d41a6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:27.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.716+0000 7f25db9e0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d41a8d60 con 0x7f25d409d6f0 2023-09-30T22:36:27.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.716+0000 7f25db9e0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25d41a7340 con 0x7f25d409e410 2023-09-30T22:36:27.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.716+0000 7f25db9e0700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f25d41a97b0 con 0x7f25d409e000 2023-09-30T22:36:27.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.716+0000 7f25d8f7b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f25d409e000 0x7f25d41a63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:27.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.716+0000 7f25d8f7b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f25d409e000 0x7f25d41a63d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:36326/0 (socket says 172.21.15.16:36326) 2023-09-30T22:36:27.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.716+0000 7f25d8f7b700 1 -- 172.21.15.16:0/1330794656 learned_addr learned my addr 172.21.15.16:0/1330794656 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:27.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.716+0000 7f25d977c700 1 --2- 172.21.15.16:0/1330794656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25d409d6f0 0x7f25d41a5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:27.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.716+0000 7f25d9f7d700 1 --2- 172.21.15.16:0/1330794656 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f25d409e410 0x7f25d41a6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:27.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.716+0000 7f25d8f7b700 1 -- 172.21.15.16:0/1330794656 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f25d409e410 msgr2=0x7f25d41a6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:27.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.716+0000 7f25d8f7b700 1 --2- 172.21.15.16:0/1330794656 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f25d409e410 0x7f25d41a6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:27.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.716+0000 7f25d8f7b700 1 -- 172.21.15.16:0/1330794656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25d409d6f0 msgr2=0x7f25d41a5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:27.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.716+0000 7f25d8f7b700 1 --2- 172.21.15.16:0/1330794656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25d409d6f0 0x7f25d41a5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:27.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.716+0000 7f25d8f7b700 1 -- 172.21.15.16:0/1330794656 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c40094a0 con 0x7f25d409e000 2023-09-30T22:36:27.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.716+0000 7f25d8f7b700 1 --2- 172.21.15.16:0/1330794656 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f25d409e000 0x7f25d41a63d0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f25c400ebe0 tx=0x7f25c400eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:27.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.717+0000 7f25c9ffb700 1 -- 172.21.15.16:0/1330794656 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c400c910 con 0x7f25d409e000 2023-09-30T22:36:27.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.717+0000 7f25db9e0700 1 -- 172.21.15.16:0/1330794656 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25d00094a0 con 0x7f25d409e000 2023-09-30T22:36:27.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.717+0000 7f25c9ffb700 1 -- 172.21.15.16:0/1330794656 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25c400ca70 con 0x7f25d409e000 2023-09-30T22:36:27.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.717+0000 7f25c9ffb700 1 -- 172.21.15.16:0/1330794656 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c400cde0 con 0x7f25d409e000 2023-09-30T22:36:27.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.717+0000 7f25db9e0700 1 -- 172.21.15.16:0/1330794656 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25d40a32a0 con 0x7f25d409e000 2023-09-30T22:36:27.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.718+0000 7f25c9ffb700 1 -- 172.21.15.16:0/1330794656 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f25c4013470 con 0x7f25d409e000 2023-09-30T22:36:27.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.719+0000 7f25c9ffb700 1 -- 172.21.15.16:0/1330794656 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f25c4050a50 con 0x7f25d409e000 2023-09-30T22:36:27.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.719+0000 7f25db9e0700 1 -- 172.21.15.16:0/1330794656 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25d409c710 con 0x7f25d409e000 2023-09-30T22:36:27.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.724+0000 7f25c9ffb700 1 -- 172.21.15.16:0/1330794656 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f25c40348e0 con 0x7f25d409e000 2023-09-30T22:36:27.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.738+0000 7f9043a82700 1 Processor -- start 2023-09-30T22:36:27.739 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.738+0000 7f9043a82700 1 -- start start 2023-09-30T22:36:27.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.739+0000 7f9043a82700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f903c133540 0x7f903c135920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:27.740 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.739+0000 7f9043a82700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f903c135e60 0x7f903c138240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:27.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.739+0000 7f9043a82700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f903c0716a0 con 0x7f903c130540 2023-09-30T22:36:27.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.739+0000 7f9043a82700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f903c071290 con 0x7f903c135e60 2023-09-30T22:36:27.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.739+0000 7f9043a82700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f903c130340 con 0x7f903c133540 2023-09-30T22:36:27.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.739+0000 7f904101d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f903c133540 0x7f903c135920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:27.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.739+0000 7f904181e700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f903c130540 0x7f903c130910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47884/0 (socket says 172.21.15.16:47884) 2023-09-30T22:36:27.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.739+0000 7f904101d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f903c133540 0x7f903c135920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:36336/0 (socket says 172.21.15.16:36336) 2023-09-30T22:36:27.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.739+0000 7f904181e700 1 -- 172.21.15.16:0/2051908497 learned_addr learned my addr 172.21.15.16:0/2051908497 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:36:27.741 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.740+0000 7f904101d700 1 -- 172.21.15.16:0/2051908497 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f903c135e60 msgr2=0x7f903c138240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:27.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.740+0000 7f904101d700 1 --2- 172.21.15.16:0/2051908497 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f903c135e60 0x7f903c138240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:27.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.740+0000 7f904081c700 1 -- 172.21.15.16:0/2051908497 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 782512602 0 0) 0x7f903c0716a0 con 0x7f903c130540 2023-09-30T22:36:27.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.740+0000 7f904101d700 1 -- 172.21.15.16:0/2051908497 >> v1:172.21.15.16:6789/0 conn(0x7f903c130540 legacy=0x7f903c130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:27.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.740+0000 7f904101d700 1 -- 172.21.15.16:0/2051908497 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f903c070b80 con 0x7f903c133540 2023-09-30T22:36:27.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.740+0000 7f904101d700 1 --2- 172.21.15.16:0/2051908497 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f903c133540 0x7f903c135920 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f902c009870 tx=0x7f902c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99834d21906ed931 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:27.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.740+0000 7f904081c700 1 -- 172.21.15.16:0/2051908497 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f902c011830 con 0x7f903c133540 2023-09-30T22:36:27.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.740+0000 7f904081c700 1 -- 172.21.15.16:0/2051908497 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f902c011990 con 0x7f903c133540 2023-09-30T22:36:27.742 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.740+0000 7f904081c700 1 -- 172.21.15.16:0/2051908497 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f902c011d20 con 0x7f903c133540 2023-09-30T22:36:27.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.741+0000 7f9043a82700 1 -- 172.21.15.16:0/2051908497 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f903c133540 msgr2=0x7f903c135920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:27.745 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.741+0000 7f9043a82700 1 --2- 172.21.15.16:0/2051908497 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f903c133540 0x7f903c135920 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f902c009870 tx=0x7f902c009b80 comp rx=0 tx=0).stop 2023-09-30T22:36:27.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.741+0000 7f9043a82700 1 -- 172.21.15.16:0/2051908497 shutdown_connections 2023-09-30T22:36:27.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.741+0000 7f9043a82700 1 --2- 172.21.15.16:0/2051908497 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f903c135e60 0x7f903c138240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:27.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.741+0000 7f9043a82700 1 --2- 172.21.15.16:0/2051908497 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f903c133540 0x7f903c135920 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:27.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.741+0000 7f9043a82700 1 -- 172.21.15.16:0/2051908497 >> 172.21.15.16:0/2051908497 conn(0x7f903c0a2aa0 msgr2=0x7f903c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:27.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.741+0000 7f9043a82700 1 -- 172.21.15.16:0/2051908497 shutdown_connections 2023-09-30T22:36:27.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.741+0000 7f9043a82700 1 -- 172.21.15.16:0/2051908497 wait complete. 2023-09-30T22:36:27.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.742+0000 7f9043a82700 1 Processor -- start 2023-09-30T22:36:27.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.742+0000 7f9043a82700 1 -- start start 2023-09-30T22:36:27.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.743+0000 7f9043a82700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f903c130540 0x7f903c09a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:27.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.743+0000 7f9043a82700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f903c133540 0x7f903c09aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:27.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.743+0000 7f9043a82700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f903c135e60 0x7f903c09b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:27.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.743+0000 7f9043a82700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f903c09cc80 con 0x7f903c133540 2023-09-30T22:36:27.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.743+0000 7f9043a82700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f903c09bbb0 con 0x7f903c135e60 2023-09-30T22:36:27.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.743+0000 7f9043a82700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f903c139380 con 0x7f903c130540 2023-09-30T22:36:27.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.743+0000 7f904101d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f903c133540 0x7f903c09aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:27.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.743+0000 7f904101d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f903c133540 0x7f903c09aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51238/0 (socket says 172.21.15.16:51238) 2023-09-30T22:36:27.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.743+0000 7f904101d700 1 -- 172.21.15.16:0/3128410021 learned_addr learned my addr 172.21.15.16:0/3128410021 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:27.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.743+0000 7f904181e700 1 --2- 172.21.15.16:0/3128410021 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f903c130540 0x7f903c09a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:27.747 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.743+0000 7f904201f700 1 --2- 172.21.15.16:0/3128410021 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f903c135e60 0x7f903c09b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:27.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.743+0000 7f904101d700 1 -- 172.21.15.16:0/3128410021 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f903c130540 msgr2=0x7f903c09a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:27.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.743+0000 7f904101d700 1 --2- 172.21.15.16:0/3128410021 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f903c130540 0x7f903c09a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:27.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.743+0000 7f904101d700 1 -- 172.21.15.16:0/3128410021 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f903c135e60 msgr2=0x7f903c09b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:27.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.743+0000 7f904101d700 1 --2- 172.21.15.16:0/3128410021 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f903c135e60 0x7f903c09b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:27.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.743+0000 7f904101d700 1 -- 172.21.15.16:0/3128410021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f902c0094a0 con 0x7f903c133540 2023-09-30T22:36:27.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.743+0000 7f904101d700 1 --2- 172.21.15.16:0/3128410021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f903c133540 0x7f903c09aad0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f902c0090b0 tx=0x7f902c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:27.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.744+0000 7f90327fc700 1 -- 172.21.15.16:0/3128410021 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f902c004550 con 0x7f903c133540 2023-09-30T22:36:27.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.744+0000 7f90327fc700 1 -- 172.21.15.16:0/3128410021 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f902c01f070 con 0x7f903c133540 2023-09-30T22:36:27.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.744+0000 7f90327fc700 1 -- 172.21.15.16:0/3128410021 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f902c0038a0 con 0x7f903c133540 2023-09-30T22:36:27.748 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.744+0000 7f9043a82700 1 -- 172.21.15.16:0/3128410021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f903c09c6e0 con 0x7f903c133540 2023-09-30T22:36:27.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.744+0000 7f9043a82700 1 -- 172.21.15.16:0/3128410021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f903c09b650 con 0x7f903c133540 2023-09-30T22:36:27.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.745+0000 7f90327fc700 1 -- 172.21.15.16:0/3128410021 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f902c003a00 con 0x7f903c133540 2023-09-30T22:36:27.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.745+0000 7f9043a82700 1 -- 172.21.15.16:0/3128410021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f903c09d860 con 0x7f903c133540 2023-09-30T22:36:27.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.745+0000 7f90327fc700 1 -- 172.21.15.16:0/3128410021 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f902c021070 con 0x7f903c133540 2023-09-30T22:36:27.750 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.750+0000 7f90327fc700 1 -- 172.21.15.16:0/3128410021 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f902c0199a0 con 0x7f903c133540 2023-09-30T22:36:27.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.934+0000 7f25db9e0700 1 -- 172.21.15.16:0/1330794656 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f25d41a95a0 con 0x7f25d409e000 2023-09-30T22:36:27.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.935+0000 7f25c9ffb700 1 -- 172.21.15.16:0/1330794656 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f25c4032030 con 0x7f25d409e000 2023-09-30T22:36:27.935 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:36:27.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.937+0000 7f25db9e0700 1 -- 172.21.15.16:0/1330794656 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f25d409e000 msgr2=0x7f25d41a63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:27.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.937+0000 7f25db9e0700 1 --2- 172.21.15.16:0/1330794656 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f25d409e000 0x7f25d41a63d0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f25c400ebe0 tx=0x7f25c400eef0 comp rx=0 tx=0).stop 2023-09-30T22:36:27.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.938+0000 7f25db9e0700 1 -- 172.21.15.16:0/1330794656 shutdown_connections 2023-09-30T22:36:27.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.938+0000 7f25db9e0700 1 --2- 172.21.15.16:0/1330794656 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f25d409e410 0x7f25d41a6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:27.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.938+0000 7f25db9e0700 1 --2- 172.21.15.16:0/1330794656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f25d409d6f0 0x7f25d41a5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:27.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.938+0000 7f25db9e0700 1 --2- 172.21.15.16:0/1330794656 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f25d409e000 0x7f25d41a63d0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:27.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.938+0000 7f25db9e0700 1 -- 172.21.15.16:0/1330794656 >> 172.21.15.16:0/1330794656 conn(0x7f25d4066f50 msgr2=0x7f25d41baeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:27.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.938+0000 7f25db9e0700 1 -- 172.21.15.16:0/1330794656 shutdown_connections 2023-09-30T22:36:27.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.938+0000 7f25db9e0700 1 -- 172.21.15.16:0/1330794656 wait complete. 2023-09-30T22:36:27.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.940+0000 7fa1642b5700 1 -- 172.21.15.16:0/2650483067 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa15c09e980 con 0x7fa15c136a70 2023-09-30T22:36:27.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.940+0000 7fa152ffd700 1 -- 172.21.15.16:0/2650483067 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa154007920 con 0x7fa15c136a70 2023-09-30T22:36:27.941 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:36:27.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.943+0000 7fa1642b5700 1 -- 172.21.15.16:0/2650483067 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa15c136a70 msgr2=0x7fa15c0719f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:27.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.943+0000 7fa1642b5700 1 --2- 172.21.15.16:0/2650483067 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa15c136a70 0x7fa15c0719f0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fa15c12edd0 tx=0x7fa15400ef90 comp rx=0 tx=0).stop 2023-09-30T22:36:27.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.943+0000 7fa1642b5700 1 -- 172.21.15.16:0/2650483067 shutdown_connections 2023-09-30T22:36:27.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.943+0000 7fa1642b5700 1 --2- 172.21.15.16:0/2650483067 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa15c09d9f0 0x7fa15c070f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:27.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.943+0000 7fa1642b5700 1 --2- 172.21.15.16:0/2650483067 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa15c136a70 0x7fa15c0719f0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:27.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.943+0000 7fa1642b5700 1 --2- 172.21.15.16:0/2650483067 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa15c1316c0 0x7fa15c0714b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:27.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.943+0000 7fa1642b5700 1 -- 172.21.15.16:0/2650483067 >> 172.21.15.16:0/2650483067 conn(0x7fa15c066f50 msgr2=0x7fa15c136e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:27.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.946+0000 7fa1642b5700 1 -- 172.21.15.16:0/2650483067 shutdown_connections 2023-09-30T22:36:27.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.946+0000 7fa1642b5700 1 -- 172.21.15.16:0/2650483067 wait complete. 2023-09-30T22:36:27.953 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:36:27.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.956+0000 7f9043a82700 1 -- 172.21.15.16:0/3128410021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f903c074ea0 con 0x7f903c133540 2023-09-30T22:36:27.956 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.956+0000 7f90327fc700 1 -- 172.21.15.16:0/3128410021 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f902c0193b0 con 0x7f903c133540 2023-09-30T22:36:27.957 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:36:27.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.958+0000 7f9043a82700 1 -- 172.21.15.16:0/3128410021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f903c133540 msgr2=0x7f903c09aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:27.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.958+0000 7f9043a82700 1 --2- 172.21.15.16:0/3128410021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f903c133540 0x7f903c09aad0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f902c0090b0 tx=0x7f902c004750 comp rx=0 tx=0).stop 2023-09-30T22:36:27.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.958+0000 7f9043a82700 1 -- 172.21.15.16:0/3128410021 shutdown_connections 2023-09-30T22:36:27.959 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.958+0000 7f9043a82700 1 --2- 172.21.15.16:0/3128410021 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f903c135e60 0x7f903c09b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:27.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.958+0000 7f9043a82700 1 --2- 172.21.15.16:0/3128410021 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f903c130540 0x7f903c09a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:27.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.958+0000 7f9043a82700 1 --2- 172.21.15.16:0/3128410021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f903c133540 0x7f903c09aad0 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:27.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.958+0000 7f9043a82700 1 -- 172.21.15.16:0/3128410021 >> 172.21.15.16:0/3128410021 conn(0x7f903c0a2aa0 msgr2=0x7f903c1350d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:27.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.959+0000 7f9043a82700 1 -- 172.21.15.16:0/3128410021 shutdown_connections 2023-09-30T22:36:27.960 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.959+0000 7f9043a82700 1 -- 172.21.15.16:0/3128410021 wait complete. 2023-09-30T22:36:27.961 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:36:27.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.962+0000 7fcd20e61700 1 -- 172.21.15.16:0/1242805070 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fcd1c074ea0 con 0x7fcd1c09de00 2023-09-30T22:36:27.963 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.962+0000 7fcd0affd700 1 -- 172.21.15.16:0/1242805070 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcd10021070 con 0x7fcd1c09de00 2023-09-30T22:36:27.963 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:36:27.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.965+0000 7fcd20e61700 1 -- 172.21.15.16:0/1242805070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd1c09de00 msgr2=0x7fcd1c071000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:27.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.965+0000 7fcd20e61700 1 --2- 172.21.15.16:0/1242805070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd1c09de00 0x7fcd1c071000 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fcd1000e900 tx=0x7fcd1000ec10 comp rx=0 tx=0).stop 2023-09-30T22:36:27.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.965+0000 7fcd20e61700 1 -- 172.21.15.16:0/1242805070 shutdown_connections 2023-09-30T22:36:27.969 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.965+0000 7fcd20e61700 1 --2- 172.21.15.16:0/1242805070 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd1c06a770 0x7fcd1c070ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:27.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.965+0000 7fcd20e61700 1 --2- 172.21.15.16:0/1242805070 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd1c13d1f0 0x7fcd1c071540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:27.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.965+0000 7fcd20e61700 1 --2- 172.21.15.16:0/1242805070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd1c09de00 0x7fcd1c071000 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:27.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.965+0000 7fcd20e61700 1 -- 172.21.15.16:0/1242805070 >> 172.21.15.16:0/1242805070 conn(0x7fcd1c066f50 msgr2=0x7fcd1c138440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:27.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.965+0000 7fcd20e61700 1 -- 172.21.15.16:0/1242805070 shutdown_connections 2023-09-30T22:36:27.970 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:27.965+0000 7fcd20e61700 1 -- 172.21.15.16:0/1242805070 wait complete. 2023-09-30T22:36:27.974 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:36:27.979 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:36:28.955 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:36:28.962 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:36:28.974 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:36:28.980 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:36:29.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.124+0000 7fd24322a700 1 Processor -- start 2023-09-30T22:36:29.124 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.124+0000 7fd24322a700 1 -- start start 2023-09-30T22:36:29.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.125+0000 7fd24322a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd23c09d6f0 0x7fd23c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:29.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.125+0000 7fd24322a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd23c09e000 0x7fd23c1339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:29.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.125+0000 7fd24322a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd23c09e410 0x7fd23c133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:29.125 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.125+0000 7fd24322a700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd23c06b460 con 0x7fd23c09d6f0 2023-09-30T22:36:29.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.125+0000 7fd24322a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd23c09b120 con 0x7fd23c09e410 2023-09-30T22:36:29.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.125+0000 7fd24322a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd23c09a7e0 con 0x7fd23c09e000 2023-09-30T22:36:29.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.125+0000 7fd240fc6700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd23c09d6f0 0x7fd23c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:29.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.125+0000 7fd240fc6700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd23c09d6f0 0x7fd23c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51244/0 (socket says 172.21.15.16:51244) 2023-09-30T22:36:29.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.125+0000 7fd240fc6700 1 -- 172.21.15.16:0/3332339447 learned_addr learned my addr 172.21.15.16:0/3332339447 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:29.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.125+0000 7fd2417c7700 1 --2- 172.21.15.16:0/3332339447 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd23c09e410 0x7fd23c133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:29.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.125+0000 7fd23bfff700 1 --2- 172.21.15.16:0/3332339447 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd23c09e000 0x7fd23c1339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:29.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.125+0000 7fd240fc6700 1 -- 172.21.15.16:0/3332339447 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd23c09e000 msgr2=0x7fd23c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:29.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.125+0000 7fd240fc6700 1 --2- 172.21.15.16:0/3332339447 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd23c09e000 0x7fd23c1339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:29.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.125+0000 7fd240fc6700 1 -- 172.21.15.16:0/3332339447 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd23c09e410 msgr2=0x7fd23c133f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:29.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.125+0000 7fd240fc6700 1 --2- 172.21.15.16:0/3332339447 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd23c09e410 0x7fd23c133f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:29.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.125+0000 7fd240fc6700 1 -- 172.21.15.16:0/3332339447 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd23c09a500 con 0x7fd23c09d6f0 2023-09-30T22:36:29.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.126+0000 7fd240fc6700 1 --2- 172.21.15.16:0/3332339447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd23c09d6f0 0x7fd23c09dac0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fd2300098f0 tx=0x7fd230009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72e210c7e3f0efe6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:29.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.126+0000 7fd23b7fe700 1 -- 172.21.15.16:0/3332339447 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd230011830 con 0x7fd23c09d6f0 2023-09-30T22:36:29.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.126+0000 7fd23b7fe700 1 -- 172.21.15.16:0/3332339447 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd230011990 con 0x7fd23c09d6f0 2023-09-30T22:36:29.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.126+0000 7fd23b7fe700 1 -- 172.21.15.16:0/3332339447 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd230011d20 con 0x7fd23c09d6f0 2023-09-30T22:36:29.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.127+0000 7fd24322a700 1 -- 172.21.15.16:0/3332339447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd23c09d6f0 msgr2=0x7fd23c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:29.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.127+0000 7fd24322a700 1 --2- 172.21.15.16:0/3332339447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd23c09d6f0 0x7fd23c09dac0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fd2300098f0 tx=0x7fd230009c00 comp rx=0 tx=0).stop 2023-09-30T22:36:29.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.131+0000 7fd24322a700 1 -- 172.21.15.16:0/3332339447 shutdown_connections 2023-09-30T22:36:29.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.131+0000 7fd24322a700 1 --2- 172.21.15.16:0/3332339447 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd23c09e410 0x7fd23c133f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:29.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.131+0000 7fd24322a700 1 --2- 172.21.15.16:0/3332339447 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd23c09d6f0 0x7fd23c09dac0 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:29.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.131+0000 7fd24322a700 1 --2- 172.21.15.16:0/3332339447 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd23c09e000 0x7fd23c1339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:29.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.131+0000 7fd24322a700 1 -- 172.21.15.16:0/3332339447 >> 172.21.15.16:0/3332339447 conn(0x7fd23c066f50 msgr2=0x7fd23c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:29.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.131+0000 7fd24322a700 1 -- 172.21.15.16:0/3332339447 shutdown_connections 2023-09-30T22:36:29.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.131+0000 7fd24322a700 1 -- 172.21.15.16:0/3332339447 wait complete. 2023-09-30T22:36:29.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.132+0000 7fd24322a700 1 Processor -- start 2023-09-30T22:36:29.133 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.132+0000 7fd24322a700 1 -- start start 2023-09-30T22:36:29.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.133+0000 7fd24322a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd23c09d6f0 0x7fd23c1c5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:29.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.133+0000 7fd24322a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd23c09e000 0x7fd23c1c5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:29.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.133+0000 7fd24322a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd23c09e410 0x7fd23c1c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:29.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.133+0000 7fd24322a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd23c1a95a0 con 0x7fd23c09e410 2023-09-30T22:36:29.134 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.133+0000 7fd24322a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd23c1a9780 con 0x7fd23c09d6f0 2023-09-30T22:36:29.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.133+0000 7fd24322a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd23c1c5aa0 con 0x7fd23c09e000 2023-09-30T22:36:29.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.134+0000 7fd240fc6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd23c09d6f0 0x7fd23c1c5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:29.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.134+0000 7fd240fc6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd23c09d6f0 0x7fd23c1c5020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36060/0 (socket says 172.21.15.16:36060) 2023-09-30T22:36:29.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.134+0000 7fd240fc6700 1 -- 172.21.15.16:0/4252064692 learned_addr learned my addr 172.21.15.16:0/4252064692 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:29.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.134+0000 7fd23bfff700 1 --2- 172.21.15.16:0/4252064692 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd23c09e000 0x7fd23c1c5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:29.135 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.134+0000 7fd2417c7700 1 --2- 172.21.15.16:0/4252064692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd23c09e410 0x7fd23c1c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:29.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.134+0000 7fd2417c7700 1 -- 172.21.15.16:0/4252064692 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd23c09e000 msgr2=0x7fd23c1c5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:29.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.134+0000 7fd2417c7700 1 --2- 172.21.15.16:0/4252064692 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd23c09e000 0x7fd23c1c5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:29.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.134+0000 7fd2417c7700 1 -- 172.21.15.16:0/4252064692 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd23c09d6f0 msgr2=0x7fd23c1c5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:29.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.134+0000 7fd2417c7700 1 --2- 172.21.15.16:0/4252064692 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd23c09d6f0 0x7fd23c1c5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:29.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.134+0000 7fd2417c7700 1 -- 172.21.15.16:0/4252064692 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd230009550 con 0x7fd23c09e410 2023-09-30T22:36:29.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.134+0000 7fd23bfff700 1 --2- 172.21.15.16:0/4252064692 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd23c09e000 0x7fd23c1c5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:29.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.134+0000 7fd2417c7700 1 --2- 172.21.15.16:0/4252064692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd23c09e410 0x7fd23c1c8040 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fd234009d90 tx=0x7fd23400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:29.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.135+0000 7fd2397fa700 1 -- 172.21.15.16:0/4252064692 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd234009c00 con 0x7fd23c09e410 2023-09-30T22:36:29.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.135+0000 7fd2397fa700 1 -- 172.21.15.16:0/4252064692 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd23400f040 con 0x7fd23c09e410 2023-09-30T22:36:29.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.135+0000 7fd24322a700 1 -- 172.21.15.16:0/4252064692 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd23c0a2090 con 0x7fd23c09e410 2023-09-30T22:36:29.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.135+0000 7fd2397fa700 1 -- 172.21.15.16:0/4252064692 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd234014480 con 0x7fd23c09e410 2023-09-30T22:36:29.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.135+0000 7fd24322a700 1 -- 172.21.15.16:0/4252064692 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd23c1a7870 con 0x7fd23c09e410 2023-09-30T22:36:29.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.136+0000 7fd2397fa700 1 -- 172.21.15.16:0/4252064692 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd2340145e0 con 0x7fd23c09e410 2023-09-30T22:36:29.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.136+0000 7fd2397fa700 1 -- 172.21.15.16:0/4252064692 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd234040160 con 0x7fd23c09e410 2023-09-30T22:36:29.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.136+0000 7fd24322a700 1 -- 172.21.15.16:0/4252064692 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd23c074ea0 con 0x7fd23c09e410 2023-09-30T22:36:29.141 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.141+0000 7fd2397fa700 1 -- 172.21.15.16:0/4252064692 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd234018030 con 0x7fd23c09e410 2023-09-30T22:36:29.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.150+0000 7f299a7fe700 1 Processor -- start 2023-09-30T22:36:29.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.150+0000 7f299a7fe700 1 -- start start 2023-09-30T22:36:29.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.150+0000 7f299a7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f299409e930 0x7f299409ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:29.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.150+0000 7f299a7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f299409f240 0x7f29941317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:29.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.151+0000 7f299a7fe700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2994070940 con 0x7f299409f650 2023-09-30T22:36:29.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.151+0000 7f299a7fe700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f299406fd60 con 0x7f299409f240 2023-09-30T22:36:29.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.151+0000 7f299a7fe700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f299406fa80 con 0x7f299409e930 2023-09-30T22:36:29.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.151+0000 7f29937fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f299409f240 0x7f29941317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:29.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.151+0000 7f29937fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f299409f240 0x7f29941317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51262/0 (socket says 172.21.15.16:51262) 2023-09-30T22:36:29.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.151+0000 7f29937fe700 1 -- 172.21.15.16:0/3405960136 learned_addr learned my addr 172.21.15.16:0/3405960136 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:29.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.151+0000 7f2993fff700 1 --2- 172.21.15.16:0/3405960136 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f299409e930 0x7f299409ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:29.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.151+0000 7f2992ffd700 1 -- 172.21.15.16:0/3405960136 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2605508809 0 0) 0x7f2994070940 con 0x7f299409f650 2023-09-30T22:36:29.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.151+0000 7f2992ffd700 1 -- 172.21.15.16:0/3405960136 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f297c0035b0 con 0x7f299409f650 2023-09-30T22:36:29.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.151+0000 7f2993fff700 1 -- 172.21.15.16:0/3405960136 >> v2:172.21.15.16:3300/0 conn(0x7f299409f240 msgr2=0x7f29941317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:29.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.151+0000 7f2993fff700 1 --2- 172.21.15.16:0/3405960136 >> v2:172.21.15.16:3300/0 conn(0x7f299409f240 0x7f29941317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:29.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.151+0000 7f2993fff700 1 -- 172.21.15.16:0/3405960136 >> v1:172.21.15.16:6789/0 conn(0x7f299409f650 legacy=0x7f2994131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:29.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.151+0000 7f2993fff700 1 -- 172.21.15.16:0/3405960136 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f299409fb60 con 0x7f299409e930 2023-09-30T22:36:29.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.151+0000 7f2993fff700 1 --2- 172.21.15.16:0/3405960136 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f299409e930 0x7f299409ed00 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f2984009840 tx=0x7f2984009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=165dfee4109ab280 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:29.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.152+0000 7f2992ffd700 1 -- 172.21.15.16:0/3405960136 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2984011830 con 0x7f299409e930 2023-09-30T22:36:29.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.152+0000 7f2992ffd700 1 -- 172.21.15.16:0/3405960136 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2984011990 con 0x7f299409e930 2023-09-30T22:36:29.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.152+0000 7f2992ffd700 1 -- 172.21.15.16:0/3405960136 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2984011d00 con 0x7f299409e930 2023-09-30T22:36:29.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.152+0000 7f299a7fe700 1 -- 172.21.15.16:0/3405960136 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f299409e930 msgr2=0x7f299409ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:29.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.152+0000 7f299a7fe700 1 --2- 172.21.15.16:0/3405960136 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f299409e930 0x7f299409ed00 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f2984009840 tx=0x7f2984009b50 comp rx=0 tx=0).stop 2023-09-30T22:36:29.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.153+0000 7f299a7fe700 1 -- 172.21.15.16:0/3405960136 shutdown_connections 2023-09-30T22:36:29.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.153+0000 7f299a7fe700 1 --2- 172.21.15.16:0/3405960136 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f299409e930 0x7f299409ed00 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:29.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.153+0000 7f299a7fe700 1 --2- 172.21.15.16:0/3405960136 >> v2:172.21.15.16:3300/0 conn(0x7f299409f240 0x7f29941317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:29.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.153+0000 7f299a7fe700 1 -- 172.21.15.16:0/3405960136 >> 172.21.15.16:0/3405960136 conn(0x7f29940a4210 msgr2=0x7f29940a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:29.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.153+0000 7f299a7fe700 1 -- 172.21.15.16:0/3405960136 shutdown_connections 2023-09-30T22:36:29.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.153+0000 7f299a7fe700 1 -- 172.21.15.16:0/3405960136 wait complete. 2023-09-30T22:36:29.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.154+0000 7f299a7fe700 1 Processor -- start 2023-09-30T22:36:29.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.154+0000 7f35db94d700 1 Processor -- start 2023-09-30T22:36:29.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.154+0000 7f299a7fe700 1 -- start start 2023-09-30T22:36:29.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.154+0000 7f35db94d700 1 -- start start 2023-09-30T22:36:29.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.154+0000 7f299a7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f299409e930 0x7f29941c2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:29.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.154+0000 7f299a7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f299409f240 0x7f29941c3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:29.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.154+0000 7f299a7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f299409f650 0x7f29941c5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:29.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.154+0000 7f299a7fe700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29941a7400 con 0x7f299409f240 2023-09-30T22:36:29.159 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.154+0000 7f299a7fe700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29941a75e0 con 0x7f299409e930 2023-09-30T22:36:29.160 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.154+0000 7f299a7fe700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f29941c38a0 con 0x7f299409f650 2023-09-30T22:36:29.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.155+0000 7f35db94d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f35d412cf60 0x7f35d406f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:29.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.155+0000 7f35db94d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f35d412f8b0 0x7f35d412fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:29.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.155+0000 7f35db94d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f35d4130260 0x7f35d4136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:29.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.155+0000 7f35db94d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35d412d330 con 0x7f35d4130260 2023-09-30T22:36:29.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.155+0000 7f35db94d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d412cd90 con 0x7f35d412f8b0 2023-09-30T22:36:29.162 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.155+0000 7f35db94d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f35d41306d0 con 0x7f35d412cf60 2023-09-30T22:36:29.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.155+0000 7f2998d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f299409f650 0x7f29941c5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:29.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.155+0000 7f2998d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f299409f650 0x7f29941c5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:36380/0 (socket says 172.21.15.16:36380) 2023-09-30T22:36:29.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.155+0000 7f2998d9b700 1 -- 172.21.15.16:0/2984284955 learned_addr learned my addr 172.21.15.16:0/2984284955 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:29.163 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.155+0000 7f29937fe700 1 --2- 172.21.15.16:0/2984284955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f299409f240 0x7f29941c3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:29.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.155+0000 7f35d96e9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f35d412cf60 0x7f35d406f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:29.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.155+0000 7f35d96e9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f35d412cf60 0x7f35d406f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:36396/0 (socket says 172.21.15.16:36396) 2023-09-30T22:36:29.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.155+0000 7f35d96e9700 1 -- 172.21.15.16:0/442499560 learned_addr learned my addr 172.21.15.16:0/442499560 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:29.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.155+0000 7f2998d9b700 1 -- 172.21.15.16:0/2984284955 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f299409e930 msgr2=0x7f29941c2e20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:36:29.165 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.155+0000 7f2998d9b700 1 --2- 172.21.15.16:0/2984284955 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f299409e930 0x7f29941c2e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:29.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.155+0000 7f2998d9b700 1 -- 172.21.15.16:0/2984284955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f299409f240 msgr2=0x7f29941c3360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:29.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.155+0000 7f2998d9b700 1 --2- 172.21.15.16:0/2984284955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f299409f240 0x7f29941c3360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:29.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.155+0000 7f2998d9b700 1 -- 172.21.15.16:0/2984284955 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29840094a0 con 0x7f299409f650 2023-09-30T22:36:29.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.155+0000 7f35d9eea700 1 --2- 172.21.15.16:0/442499560 >> v2:172.21.15.16:3300/0 conn(0x7f35d4130260 0x7f35d4136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:29.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.155+0000 7f35d96e9700 1 -- 172.21.15.16:0/442499560 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f35d412f8b0 msgr2=0x7f35d412fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:29.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.156+0000 7f35d96e9700 1 --2- 172.21.15.16:0/442499560 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f35d412f8b0 0x7f35d412fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:29.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.156+0000 7f35d96e9700 1 -- 172.21.15.16:0/442499560 >> v2:172.21.15.16:3300/0 conn(0x7f35d4130260 msgr2=0x7f35d4136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:29.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.156+0000 7f35d96e9700 1 --2- 172.21.15.16:0/442499560 >> v2:172.21.15.16:3300/0 conn(0x7f35d4130260 0x7f35d4136c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:29.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.156+0000 7f35d96e9700 1 -- 172.21.15.16:0/442499560 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d4071780 con 0x7f35d412cf60 2023-09-30T22:36:29.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.156+0000 7f2998d9b700 1 --2- 172.21.15.16:0/2984284955 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f299409f650 0x7f29941c5e40 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f298c00cc20 tx=0x7f298c00cf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:29.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.156+0000 7f35d96e9700 1 --2- 172.21.15.16:0/442499560 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f35d412cf60 0x7f35d406f5d0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f35d00098f0 tx=0x7f35d0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=181912bd4bd6bda6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:29.169 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.156+0000 7f2990ff9700 1 -- 172.21.15.16:0/2984284955 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f298c003c00 con 0x7f299409f650 2023-09-30T22:36:29.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.156+0000 7f35cbfff700 1 -- 172.21.15.16:0/442499560 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d000b8a0 con 0x7f35d412cf60 2023-09-30T22:36:29.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.156+0000 7f2990ff9700 1 -- 172.21.15.16:0/2984284955 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f298c003d60 con 0x7f299409f650 2023-09-30T22:36:29.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.156+0000 7f299a7fe700 1 -- 172.21.15.16:0/2984284955 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29940a0190 con 0x7f299409f650 2023-09-30T22:36:29.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.156+0000 7f35cbfff700 1 -- 172.21.15.16:0/442499560 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35d000ba00 con 0x7f35d412cf60 2023-09-30T22:36:29.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.156+0000 7f35cbfff700 1 -- 172.21.15.16:0/442499560 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35d000bd90 con 0x7f35d412cf60 2023-09-30T22:36:29.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.156+0000 7f299a7fe700 1 -- 172.21.15.16:0/2984284955 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29941a7040 con 0x7f299409f650 2023-09-30T22:36:29.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.156+0000 7f2990ff9700 1 -- 172.21.15.16:0/2984284955 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f298c00ab70 con 0x7f299409f650 2023-09-30T22:36:29.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.157+0000 7f35db94d700 1 -- 172.21.15.16:0/442499560 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f35d412cf60 msgr2=0x7f35d406f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:29.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.157+0000 7f35db94d700 1 --2- 172.21.15.16:0/442499560 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f35d412cf60 0x7f35d406f5d0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f35d00098f0 tx=0x7f35d0009c00 comp rx=0 tx=0).stop 2023-09-30T22:36:29.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.157+0000 7f35db94d700 1 -- 172.21.15.16:0/442499560 shutdown_connections 2023-09-30T22:36:29.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.157+0000 7f35db94d700 1 --2- 172.21.15.16:0/442499560 >> v2:172.21.15.16:3300/0 conn(0x7f35d4130260 0x7f35d4136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:29.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.157+0000 7f35db94d700 1 --2- 172.21.15.16:0/442499560 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f35d412cf60 0x7f35d406f5d0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:29.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.157+0000 7f35db94d700 1 --2- 172.21.15.16:0/442499560 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f35d412f8b0 0x7f35d412fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:29.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.157+0000 7f35db94d700 1 -- 172.21.15.16:0/442499560 >> 172.21.15.16:0/442499560 conn(0x7f35d40a2aa0 msgr2=0x7f35d40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:29.173 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.157+0000 7f35db94d700 1 -- 172.21.15.16:0/442499560 shutdown_connections 2023-09-30T22:36:29.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.157+0000 7f299a7fe700 1 -- 172.21.15.16:0/2984284955 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29941a7230 con 0x7f299409f650 2023-09-30T22:36:29.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.158+0000 7f2990ff9700 1 -- 172.21.15.16:0/2984284955 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f298c00acd0 con 0x7f299409f650 2023-09-30T22:36:29.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.158+0000 7f2990ff9700 1 -- 172.21.15.16:0/2984284955 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f298c040210 con 0x7f299409f650 2023-09-30T22:36:29.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.160+0000 7f35db94d700 1 -- 172.21.15.16:0/442499560 wait complete. 2023-09-30T22:36:29.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.161+0000 7f35db94d700 1 Processor -- start 2023-09-30T22:36:29.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.161+0000 7f35db94d700 1 -- start start 2023-09-30T22:36:29.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.161+0000 7f35db94d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f35d412cf60 0x7f35d4099d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:29.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.161+0000 7f35db94d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35d412f8b0 0x7f35d409a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:29.176 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.161+0000 7f35db94d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f35d4130260 0x7f35d409a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:29.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.161+0000 7f35db94d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d4129ca0 con 0x7f35d412f8b0 2023-09-30T22:36:29.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.161+0000 7f35db94d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d412f520 con 0x7f35d4130260 2023-09-30T22:36:29.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.161+0000 7f35db94d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f35d4129fa0 con 0x7f35d412cf60 2023-09-30T22:36:29.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.161+0000 7f35d9eea700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f35d4130260 0x7f35d409a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:29.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.161+0000 7f35d9eea700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f35d4130260 0x7f35d409a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36086/0 (socket says 172.21.15.16:36086) 2023-09-30T22:36:29.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.161+0000 7f35d9eea700 1 -- 172.21.15.16:0/3827011818 learned_addr learned my addr 172.21.15.16:0/3827011818 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:29.177 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.162+0000 7f35d96e9700 1 --2- 172.21.15.16:0/3827011818 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f35d412cf60 0x7f35d4099d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:29.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.162+0000 7f35d9eea700 1 -- 172.21.15.16:0/3827011818 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f35d412cf60 msgr2=0x7f35d4099d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:29.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.162+0000 7f35d9eea700 1 --2- 172.21.15.16:0/3827011818 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f35d412cf60 0x7f35d4099d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:29.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.162+0000 7f35d9eea700 1 -- 172.21.15.16:0/3827011818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35d412f8b0 msgr2=0x7f35d409a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:29.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.162+0000 7f35d9eea700 1 --2- 172.21.15.16:0/3827011818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35d412f8b0 0x7f35d409a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:29.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.162+0000 7f35d9eea700 1 -- 172.21.15.16:0/3827011818 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d0009550 con 0x7f35d4130260 2023-09-30T22:36:29.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.162+0000 7f35d96e9700 1 --2- 172.21.15.16:0/3827011818 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f35d412cf60 0x7f35d4099d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:36:29.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.162+0000 7f35d9eea700 1 --2- 172.21.15.16:0/3827011818 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f35d4130260 0x7f35d409a7d0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f35cc007e30 tx=0x7f35cc00e3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:29.178 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.162+0000 7f35c9ffb700 1 -- 172.21.15.16:0/3827011818 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35cc00eea0 con 0x7f35d4130260 2023-09-30T22:36:29.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.163+0000 7f094c2d7700 1 Processor -- start 2023-09-30T22:36:29.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.162+0000 7f35c9ffb700 1 -- 172.21.15.16:0/3827011818 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35cc014420 con 0x7f35d4130260 2023-09-30T22:36:29.179 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.162+0000 7f35c9ffb700 1 -- 172.21.15.16:0/3827011818 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35cc0147d0 con 0x7f35d4130260 2023-09-30T22:36:29.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.163+0000 7f35db94d700 1 -- 172.21.15.16:0/3827011818 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35d411fd80 con 0x7f35d4130260 2023-09-30T22:36:29.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.163+0000 7f35db94d700 1 -- 172.21.15.16:0/3827011818 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35d4120ab0 con 0x7f35d4130260 2023-09-30T22:36:29.180 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.163+0000 7f2990ff9700 1 -- 172.21.15.16:0/2984284955 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f298c011740 con 0x7f299409f650 2023-09-30T22:36:29.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.163+0000 7f094c2d7700 1 -- start start 2023-09-30T22:36:29.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.164+0000 7f094c2d7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f094409eca0 0x7f094413cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:29.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.164+0000 7f094c2d7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f094406f050 0x7f094413f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:29.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.164+0000 7f094c2d7700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0944070ee0 con 0x7f094406e570 2023-09-30T22:36:29.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.164+0000 7f094c2d7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0944071980 con 0x7f094409eca0 2023-09-30T22:36:29.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.164+0000 7f094c2d7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0944071480 con 0x7f094406f050 2023-09-30T22:36:29.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.164+0000 7f094a073700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f094406e570 0x7f094406e940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47912/0 (socket says 172.21.15.16:47912) 2023-09-30T22:36:29.181 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.164+0000 7f094a073700 1 -- 172.21.15.16:0/1982036530 learned_addr learned my addr 172.21.15.16:0/1982036530 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:36:29.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.164+0000 7f0949872700 1 --2- 172.21.15.16:0/1982036530 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f094409eca0 0x7f094413cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:29.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.164+0000 7f094a874700 1 --2- 172.21.15.16:0/1982036530 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f094406f050 0x7f094413f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:29.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.165+0000 7f0949071700 1 -- 172.21.15.16:0/1982036530 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3059502075 0 0) 0x7f0944070ee0 con 0x7f094406e570 2023-09-30T22:36:29.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.165+0000 7f0949071700 1 -- 172.21.15.16:0/1982036530 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09300035b0 con 0x7f094406e570 2023-09-30T22:36:29.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.164+0000 7f35c9ffb700 1 -- 172.21.15.16:0/3827011818 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f35cc014930 con 0x7f35d4130260 2023-09-30T22:36:29.182 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.165+0000 7f35db94d700 1 -- 172.21.15.16:0/3827011818 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35d411fbb0 con 0x7f35d4130260 2023-09-30T22:36:29.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.165+0000 7f0949872700 1 -- 172.21.15.16:0/1982036530 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f094406f050 msgr2=0x7f094413f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:29.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.165+0000 7f0949872700 1 --2- 172.21.15.16:0/1982036530 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f094406f050 0x7f094413f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:29.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.165+0000 7f0949872700 1 -- 172.21.15.16:0/1982036530 >> v1:172.21.15.16:6789/0 conn(0x7f094406e570 legacy=0x7f094406e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:29.183 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.165+0000 7f35c9ffb700 1 -- 172.21.15.16:0/3827011818 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f35cc03fff0 con 0x7f35d4130260 2023-09-30T22:36:29.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.165+0000 7f0949872700 1 -- 172.21.15.16:0/1982036530 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0944071b90 con 0x7f094409eca0 2023-09-30T22:36:29.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.165+0000 7f0949872700 1 --2- 172.21.15.16:0/1982036530 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f094409eca0 0x7f094413cea0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f0934009870 tx=0x7f0934009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cea0055d9d0dca4 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:29.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.170+0000 7f0949071700 1 -- 172.21.15.16:0/1982036530 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0934011830 con 0x7f094409eca0 2023-09-30T22:36:29.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.170+0000 7f0949071700 1 -- 172.21.15.16:0/1982036530 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0934011990 con 0x7f094409eca0 2023-09-30T22:36:29.184 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.170+0000 7f0949071700 1 -- 172.21.15.16:0/1982036530 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0934011d00 con 0x7f094409eca0 2023-09-30T22:36:29.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.170+0000 7f35c9ffb700 1 -- 172.21.15.16:0/3827011818 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f35cc014be0 con 0x7f35d4130260 2023-09-30T22:36:29.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.171+0000 7f094c2d7700 1 -- 172.21.15.16:0/1982036530 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f094409eca0 msgr2=0x7f094413cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:29.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.171+0000 7f094c2d7700 1 --2- 172.21.15.16:0/1982036530 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f094409eca0 0x7f094413cea0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f0934009870 tx=0x7f0934009b80 comp rx=0 tx=0).stop 2023-09-30T22:36:29.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.171+0000 7f094c2d7700 1 -- 172.21.15.16:0/1982036530 shutdown_connections 2023-09-30T22:36:29.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.171+0000 7f094c2d7700 1 --2- 172.21.15.16:0/1982036530 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f094406f050 0x7f094413f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:29.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.171+0000 7f094c2d7700 1 --2- 172.21.15.16:0/1982036530 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f094409eca0 0x7f094413cea0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:29.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.171+0000 7f094c2d7700 1 -- 172.21.15.16:0/1982036530 >> 172.21.15.16:0/1982036530 conn(0x7f09440a2aa0 msgr2=0x7f09440a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:29.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.174+0000 7f094c2d7700 1 -- 172.21.15.16:0/1982036530 shutdown_connections 2023-09-30T22:36:29.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.174+0000 7f094c2d7700 1 -- 172.21.15.16:0/1982036530 wait complete. 2023-09-30T22:36:29.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.175+0000 7f094c2d7700 1 Processor -- start 2023-09-30T22:36:29.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.175+0000 7f094c2d7700 1 -- start start 2023-09-30T22:36:29.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.176+0000 7f094c2d7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f094406e570 0x7f0944136600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:29.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.176+0000 7f094c2d7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f094406f050 0x7f0944136b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:29.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.176+0000 7f094c2d7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f094413e8b0 0x7f0944137080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:29.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.176+0000 7f094c2d7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0944138400 con 0x7f094406f050 2023-09-30T22:36:29.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.176+0000 7f094c2d7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0944138210 con 0x7f094406e570 2023-09-30T22:36:29.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.176+0000 7f094c2d7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0944138ae0 con 0x7f094413e8b0 2023-09-30T22:36:29.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.176+0000 7f094a073700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f094406e570 0x7f0944136600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:29.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.176+0000 7f094a073700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f094406e570 0x7f0944136600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36106/0 (socket says 172.21.15.16:36106) 2023-09-30T22:36:29.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.176+0000 7f094a073700 1 -- 172.21.15.16:0/685785392 learned_addr learned my addr 172.21.15.16:0/685785392 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:29.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.176+0000 7f094a874700 1 --2- 172.21.15.16:0/685785392 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f094413e8b0 0x7f0944137080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:29.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.176+0000 7f0949872700 1 --2- 172.21.15.16:0/685785392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f094406f050 0x7f0944136b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:29.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.177+0000 7f094a874700 1 -- 172.21.15.16:0/685785392 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f094406e570 msgr2=0x7f0944136600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:29.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.177+0000 7f094a874700 1 --2- 172.21.15.16:0/685785392 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f094406e570 0x7f0944136600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:29.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.177+0000 7f094a874700 1 -- 172.21.15.16:0/685785392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f094406f050 msgr2=0x7f0944136b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:29.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.177+0000 7f094a874700 1 --2- 172.21.15.16:0/685785392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f094406f050 0x7f0944136b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:29.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.177+0000 7f094a874700 1 -- 172.21.15.16:0/685785392 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09340094a0 con 0x7f094413e8b0 2023-09-30T22:36:29.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.177+0000 7f094a073700 1 --2- 172.21.15.16:0/685785392 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f094406e570 0x7f0944136600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:29.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.177+0000 7f094a874700 1 --2- 172.21.15.16:0/685785392 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f094413e8b0 0x7f0944137080 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f093c009d90 tx=0x7f093c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:29.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.177+0000 7f093affd700 1 -- 172.21.15.16:0/685785392 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093c009c00 con 0x7f094413e8b0 2023-09-30T22:36:29.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.177+0000 7f093affd700 1 -- 172.21.15.16:0/685785392 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f093c00f040 con 0x7f094413e8b0 2023-09-30T22:36:29.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.177+0000 7f094c2d7700 1 -- 172.21.15.16:0/685785392 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0944070610 con 0x7f094413e8b0 2023-09-30T22:36:29.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.177+0000 7f093affd700 1 -- 172.21.15.16:0/685785392 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093c014480 con 0x7f094413e8b0 2023-09-30T22:36:29.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.177+0000 7f094c2d7700 1 -- 172.21.15.16:0/685785392 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0944139b80 con 0x7f094413e8b0 2023-09-30T22:36:29.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.178+0000 7f093affd700 1 -- 172.21.15.16:0/685785392 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f093c0145e0 con 0x7f094413e8b0 2023-09-30T22:36:29.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.178+0000 7f093affd700 1 -- 172.21.15.16:0/685785392 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f093c0401c0 con 0x7f094413e8b0 2023-09-30T22:36:29.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.179+0000 7f094c2d7700 1 -- 172.21.15.16:0/685785392 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0944074ea0 con 0x7f094413e8b0 2023-09-30T22:36:29.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.183+0000 7f093affd700 1 -- 172.21.15.16:0/685785392 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f093c007510 con 0x7f094413e8b0 2023-09-30T22:36:29.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.361+0000 7fd24322a700 1 -- 172.21.15.16:0/4252064692 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd23c06c360 con 0x7fd23c09e410 2023-09-30T22:36:29.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.361+0000 7fd2397fa700 1 -- 172.21.15.16:0/4252064692 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd23402b090 con 0x7fd23c09e410 2023-09-30T22:36:29.364 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:36:29.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.363+0000 7fd24322a700 1 -- 172.21.15.16:0/4252064692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd23c09e410 msgr2=0x7fd23c1c8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:29.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.363+0000 7fd24322a700 1 --2- 172.21.15.16:0/4252064692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd23c09e410 0x7fd23c1c8040 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fd234009d90 tx=0x7fd23400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:36:29.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.364+0000 7fd24322a700 1 -- 172.21.15.16:0/4252064692 shutdown_connections 2023-09-30T22:36:29.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.364+0000 7fd24322a700 1 --2- 172.21.15.16:0/4252064692 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd23c09d6f0 0x7fd23c1c5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:29.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.364+0000 7fd24322a700 1 --2- 172.21.15.16:0/4252064692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd23c09e410 0x7fd23c1c8040 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:29.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.364+0000 7fd24322a700 1 --2- 172.21.15.16:0/4252064692 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd23c09e000 0x7fd23c1c5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:29.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.364+0000 7fd24322a700 1 -- 172.21.15.16:0/4252064692 >> 172.21.15.16:0/4252064692 conn(0x7fd23c066f50 msgr2=0x7fd23c1bad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:29.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.364+0000 7fd24322a700 1 -- 172.21.15.16:0/4252064692 shutdown_connections 2023-09-30T22:36:29.368 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.364+0000 7fd24322a700 1 -- 172.21.15.16:0/4252064692 wait complete. 2023-09-30T22:36:29.379 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:36:29.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.381+0000 7f35db94d700 1 -- 172.21.15.16:0/3827011818 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f35d411f660 con 0x7f35d4130260 2023-09-30T22:36:29.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.382+0000 7f35c9ffb700 1 -- 172.21.15.16:0/3827011818 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f35cc01bb00 con 0x7f35d4130260 2023-09-30T22:36:29.382 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:36:29.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.384+0000 7f35db94d700 1 -- 172.21.15.16:0/3827011818 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f35d4130260 msgr2=0x7f35d409a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:29.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.384+0000 7f35db94d700 1 --2- 172.21.15.16:0/3827011818 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f35d4130260 0x7f35d409a7d0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f35cc007e30 tx=0x7f35cc00e3c0 comp rx=0 tx=0).stop 2023-09-30T22:36:29.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.384+0000 7f35db94d700 1 -- 172.21.15.16:0/3827011818 shutdown_connections 2023-09-30T22:36:29.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.384+0000 7f35db94d700 1 --2- 172.21.15.16:0/3827011818 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f35d4130260 0x7f35d409a7d0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:29.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.384+0000 7f35db94d700 1 --2- 172.21.15.16:0/3827011818 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f35d412cf60 0x7f35d4099d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:29.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.384+0000 7f35db94d700 1 --2- 172.21.15.16:0/3827011818 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f35d412f8b0 0x7f35d409a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:29.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.384+0000 7f35db94d700 1 -- 172.21.15.16:0/3827011818 >> 172.21.15.16:0/3827011818 conn(0x7f35d40a2aa0 msgr2=0x7f35d409e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:29.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.384+0000 7f35db94d700 1 -- 172.21.15.16:0/3827011818 shutdown_connections 2023-09-30T22:36:29.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.384+0000 7f35db94d700 1 -- 172.21.15.16:0/3827011818 wait complete. 2023-09-30T22:36:29.396 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.396+0000 7f299a7fe700 1 -- 172.21.15.16:0/2984284955 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2994074ea0 con 0x7f299409f650 2023-09-30T22:36:29.397 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.396+0000 7f2990ff9700 1 -- 172.21.15.16:0/2984284955 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f298c011380 con 0x7f299409f650 2023-09-30T22:36:29.397 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:36:29.399 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:36:29.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.400+0000 7f299a7fe700 1 -- 172.21.15.16:0/2984284955 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f299409f650 msgr2=0x7f29941c5e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:29.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.400+0000 7f299a7fe700 1 --2- 172.21.15.16:0/2984284955 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f299409f650 0x7f29941c5e40 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f298c00cc20 tx=0x7f298c00cf30 comp rx=0 tx=0).stop 2023-09-30T22:36:29.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.400+0000 7f299a7fe700 1 -- 172.21.15.16:0/2984284955 shutdown_connections 2023-09-30T22:36:29.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.400+0000 7f299a7fe700 1 --2- 172.21.15.16:0/2984284955 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f299409e930 0x7f29941c2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:29.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.400+0000 7f299a7fe700 1 --2- 172.21.15.16:0/2984284955 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f299409f650 0x7f29941c5e40 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:29.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.400+0000 7f299a7fe700 1 --2- 172.21.15.16:0/2984284955 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f299409f240 0x7f29941c3360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:29.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.400+0000 7f299a7fe700 1 -- 172.21.15.16:0/2984284955 >> 172.21.15.16:0/2984284955 conn(0x7f29940a4210 msgr2=0x7f29941b8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:29.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.400+0000 7f299a7fe700 1 -- 172.21.15.16:0/2984284955 shutdown_connections 2023-09-30T22:36:29.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.400+0000 7f299a7fe700 1 -- 172.21.15.16:0/2984284955 wait complete. 2023-09-30T22:36:29.419 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:36:29.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.422+0000 7f094c2d7700 1 -- 172.21.15.16:0/685785392 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f09440a1da0 con 0x7f094413e8b0 2023-09-30T22:36:29.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.423+0000 7f093affd700 1 -- 172.21.15.16:0/685785392 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f093c02c090 con 0x7f094413e8b0 2023-09-30T22:36:29.424 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:36:29.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.425+0000 7f094c2d7700 1 -- 172.21.15.16:0/685785392 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f094413e8b0 msgr2=0x7f0944137080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:29.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.425+0000 7f094c2d7700 1 --2- 172.21.15.16:0/685785392 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f094413e8b0 0x7f0944137080 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f093c009d90 tx=0x7f093c00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:36:29.426 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.425+0000 7f094c2d7700 1 -- 172.21.15.16:0/685785392 shutdown_connections 2023-09-30T22:36:29.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.425+0000 7f094c2d7700 1 --2- 172.21.15.16:0/685785392 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f094406e570 0x7f0944136600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:29.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.425+0000 7f094c2d7700 1 --2- 172.21.15.16:0/685785392 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f094413e8b0 0x7f0944137080 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:29.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.425+0000 7f094c2d7700 1 --2- 172.21.15.16:0/685785392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f094406f050 0x7f0944136b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:29.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.425+0000 7f094c2d7700 1 -- 172.21.15.16:0/685785392 >> 172.21.15.16:0/685785392 conn(0x7f09440a2aa0 msgr2=0x7f094409ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:29.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.425+0000 7f094c2d7700 1 -- 172.21.15.16:0/685785392 shutdown_connections 2023-09-30T22:36:29.427 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:29.425+0000 7f094c2d7700 1 -- 172.21.15.16:0/685785392 wait complete. 2023-09-30T22:36:29.440 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:36:30.379 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:36:30.400 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:36:30.419 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:36:30.440 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:36:30.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.543+0000 7f4ad6a69700 1 Processor -- start 2023-09-30T22:36:30.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.543+0000 7f4ad6a69700 1 -- start start 2023-09-30T22:36:30.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.543+0000 7f4ad6a69700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4ad009e930 0x7f4ad009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:30.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.544+0000 7f4ad6a69700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4ad009f240 0x7f4ad01317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:30.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.544+0000 7f4ad6a69700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ad0070940 con 0x7f4ad009f650 2023-09-30T22:36:30.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.544+0000 7f4ad6a69700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ad006fd60 con 0x7f4ad009f240 2023-09-30T22:36:30.544 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.544+0000 7f4ad6a69700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad006fa80 con 0x7f4ad009e930 2023-09-30T22:36:30.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.544+0000 7f4ad4805700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4ad009e930 0x7f4ad009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:30.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.544+0000 7f4ad5006700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f4ad009f650 0x7f4ad0131d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47914/0 (socket says 172.21.15.16:47914) 2023-09-30T22:36:30.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.544+0000 7f4acffff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4ad009f240 0x7f4ad01317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:30.545 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.544+0000 7f4ad5006700 1 -- 172.21.15.16:0/978660479 learned_addr learned my addr 172.21.15.16:0/978660479 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:36:30.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.544+0000 7f4ad4805700 1 -- 172.21.15.16:0/978660479 >> v2:172.21.15.16:3300/0 conn(0x7f4ad009f240 msgr2=0x7f4ad01317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:30.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.544+0000 7f4ad4805700 1 --2- 172.21.15.16:0/978660479 >> v2:172.21.15.16:3300/0 conn(0x7f4ad009f240 0x7f4ad01317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:30.546 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.544+0000 7f4ad4805700 1 -- 172.21.15.16:0/978660479 >> v1:172.21.15.16:6789/0 conn(0x7f4ad009f650 legacy=0x7f4ad0131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:30.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.544+0000 7f4ad4805700 1 -- 172.21.15.16:0/978660479 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad009fb60 con 0x7f4ad009e930 2023-09-30T22:36:30.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.545+0000 7f4ad4805700 1 --2- 172.21.15.16:0/978660479 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4ad009e930 0x7f4ad009ed00 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f4ac4009840 tx=0x7f4ac4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=908c7ed8ad87d3e5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:30.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.545+0000 7f4acf7fe700 1 -- 172.21.15.16:0/978660479 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac4004230 con 0x7f4ad009e930 2023-09-30T22:36:30.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.545+0000 7f4acf7fe700 1 -- 172.21.15.16:0/978660479 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ac4004390 con 0x7f4ad009e930 2023-09-30T22:36:30.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.545+0000 7f4acf7fe700 1 -- 172.21.15.16:0/978660479 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac40036a0 con 0x7f4ad009e930 2023-09-30T22:36:30.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.546+0000 7f4ad6a69700 1 -- 172.21.15.16:0/978660479 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4ad009e930 msgr2=0x7f4ad009ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:30.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.546+0000 7f4ad6a69700 1 --2- 172.21.15.16:0/978660479 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4ad009e930 0x7f4ad009ed00 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f4ac4009840 tx=0x7f4ac4009b50 comp rx=0 tx=0).stop 2023-09-30T22:36:30.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.546+0000 7f4ad6a69700 1 -- 172.21.15.16:0/978660479 shutdown_connections 2023-09-30T22:36:30.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.546+0000 7f4ad6a69700 1 --2- 172.21.15.16:0/978660479 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4ad009e930 0x7f4ad009ed00 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:30.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.546+0000 7f4ad6a69700 1 --2- 172.21.15.16:0/978660479 >> v2:172.21.15.16:3300/0 conn(0x7f4ad009f240 0x7f4ad01317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:30.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.546+0000 7f4ad6a69700 1 -- 172.21.15.16:0/978660479 >> 172.21.15.16:0/978660479 conn(0x7f4ad00a4210 msgr2=0x7f4ad00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:30.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.546+0000 7f4ad6a69700 1 -- 172.21.15.16:0/978660479 shutdown_connections 2023-09-30T22:36:30.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.546+0000 7f4ad6a69700 1 -- 172.21.15.16:0/978660479 wait complete. 2023-09-30T22:36:30.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.547+0000 7f4ad6a69700 1 Processor -- start 2023-09-30T22:36:30.548 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.548+0000 7f4ad6a69700 1 -- start start 2023-09-30T22:36:30.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.548+0000 7f4ad6a69700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4ad009e930 0x7f4ad01c0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:30.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.548+0000 7f4ad6a69700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4ad009f240 0x7f4ad01c1340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:30.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.548+0000 7f4ad6a69700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ad009f650 0x7f4ad01c3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:30.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.548+0000 7f4ad6a69700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad01a4e20 con 0x7f4ad009f650 2023-09-30T22:36:30.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.548+0000 7f4ad6a69700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad01c1880 con 0x7f4ad009e930 2023-09-30T22:36:30.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.548+0000 7f4ad6a69700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad0071d20 con 0x7f4ad009f240 2023-09-30T22:36:30.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.549+0000 7f4acffff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4ad009f240 0x7f4ad01c1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:30.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.549+0000 7f4acffff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4ad009f240 0x7f4ad01c1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:36454/0 (socket says 172.21.15.16:36454) 2023-09-30T22:36:30.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.549+0000 7f4acffff700 1 -- 172.21.15.16:0/1212677212 learned_addr learned my addr 172.21.15.16:0/1212677212 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:30.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.549+0000 7f4ad4805700 1 --2- 172.21.15.16:0/1212677212 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4ad009e930 0x7f4ad01c0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:30.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.549+0000 7f4ad5006700 1 --2- 172.21.15.16:0/1212677212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ad009f650 0x7f4ad01c3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:30.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.549+0000 7f4acffff700 1 -- 172.21.15.16:0/1212677212 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4ad009e930 msgr2=0x7f4ad01c0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:30.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.549+0000 7f4acffff700 1 --2- 172.21.15.16:0/1212677212 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4ad009e930 0x7f4ad01c0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:30.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.549+0000 7f4acffff700 1 -- 172.21.15.16:0/1212677212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ad009f650 msgr2=0x7f4ad01c3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:30.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.549+0000 7f4acffff700 1 --2- 172.21.15.16:0/1212677212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ad009f650 0x7f4ad01c3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:30.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.549+0000 7f4acffff700 1 -- 172.21.15.16:0/1212677212 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac40094a0 con 0x7f4ad009f240 2023-09-30T22:36:30.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.550+0000 7f4acffff700 1 --2- 172.21.15.16:0/1212677212 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4ad009f240 0x7f4ad01c1340 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f4abc00bea0 tx=0x7f4abc00bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:30.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.550+0000 7f4acd7fa700 1 -- 172.21.15.16:0/1212677212 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4abc00dbb0 con 0x7f4ad009f240 2023-09-30T22:36:30.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.550+0000 7f4acd7fa700 1 -- 172.21.15.16:0/1212677212 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4abc00f7a0 con 0x7f4ad009f240 2023-09-30T22:36:30.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.550+0000 7f4ad6a69700 1 -- 172.21.15.16:0/1212677212 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ad00a0190 con 0x7f4ad009f240 2023-09-30T22:36:30.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.550+0000 7f4acd7fa700 1 -- 172.21.15.16:0/1212677212 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4abc00fb30 con 0x7f4ad009f240 2023-09-30T22:36:30.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.550+0000 7f4ad6a69700 1 -- 172.21.15.16:0/1212677212 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ad00a33f0 con 0x7f4ad009f240 2023-09-30T22:36:30.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.551+0000 7f4acd7fa700 1 -- 172.21.15.16:0/1212677212 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4abc014070 con 0x7f4ad009f240 2023-09-30T22:36:30.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.551+0000 7f4acd7fa700 1 -- 172.21.15.16:0/1212677212 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f4abc01ec60 con 0x7f4ad009f240 2023-09-30T22:36:30.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.551+0000 7f4ad6a69700 1 -- 172.21.15.16:0/1212677212 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ad0074ea0 con 0x7f4ad009f240 2023-09-30T22:36:30.556 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.556+0000 7f4acd7fa700 1 -- 172.21.15.16:0/1212677212 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4abc01b070 con 0x7f4ad009f240 2023-09-30T22:36:30.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.572+0000 7f21b4771700 1 Processor -- start 2023-09-30T22:36:30.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.572+0000 7f21b4771700 1 -- start start 2023-09-30T22:36:30.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.573+0000 7f21b4771700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f21ac06c0e0 0x7f21ac09ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:30.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.573+0000 7f21b4771700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f21ac1380f0 0x7f21ac13fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:30.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.573+0000 7f21b4771700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21ac134980 con 0x7f21ac134ec0 2023-09-30T22:36:30.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.573+0000 7f21b4771700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21ac06c4b0 con 0x7f21ac1380f0 2023-09-30T22:36:30.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.573+0000 7f21b4771700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac0a09c0 con 0x7f21ac06c0e0 2023-09-30T22:36:30.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.573+0000 7f21b250d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f21ac06c0e0 0x7f21ac09ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:30.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.573+0000 7f21b250d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f21ac06c0e0 0x7f21ac09ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36140/0 (socket says 172.21.15.16:36140) 2023-09-30T22:36:30.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.573+0000 7f21b250d700 1 -- 172.21.15.16:0/3846865253 learned_addr learned my addr 172.21.15.16:0/3846865253 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:30.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.573+0000 7f21b2d0e700 1 --2- 172.21.15.16:0/3846865253 >> v2:172.21.15.16:3300/0 conn(0x7f21ac1380f0 0x7f21ac13fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:30.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.574+0000 7f21b250d700 1 -- 172.21.15.16:0/3846865253 >> v2:172.21.15.16:3300/0 conn(0x7f21ac1380f0 msgr2=0x7f21ac13fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:30.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.574+0000 7f21b250d700 1 --2- 172.21.15.16:0/3846865253 >> v2:172.21.15.16:3300/0 conn(0x7f21ac1380f0 0x7f21ac13fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:30.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.574+0000 7f21b250d700 1 -- 172.21.15.16:0/3846865253 >> v1:172.21.15.16:6789/0 conn(0x7f21ac134ec0 legacy=0x7f21ac1379e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:30.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.574+0000 7f21b250d700 1 -- 172.21.15.16:0/3846865253 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21ac0a0600 con 0x7f21ac06c0e0 2023-09-30T22:36:30.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.574+0000 7f21b250d700 1 --2- 172.21.15.16:0/3846865253 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f21ac06c0e0 0x7f21ac09ac90 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f21a80098f0 tx=0x7f21a8009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a689daf6af02909 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:30.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.574+0000 7f21b150b700 1 -- 172.21.15.16:0/3846865253 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a8011830 con 0x7f21ac06c0e0 2023-09-30T22:36:30.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.574+0000 7f21b150b700 1 -- 172.21.15.16:0/3846865253 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21a8011990 con 0x7f21ac06c0e0 2023-09-30T22:36:30.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.574+0000 7f21b150b700 1 -- 172.21.15.16:0/3846865253 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a8011d20 con 0x7f21ac06c0e0 2023-09-30T22:36:30.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.575+0000 7f21b4771700 1 -- 172.21.15.16:0/3846865253 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f21ac06c0e0 msgr2=0x7f21ac09ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:30.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.575+0000 7f21b4771700 1 --2- 172.21.15.16:0/3846865253 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f21ac06c0e0 0x7f21ac09ac90 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f21a80098f0 tx=0x7f21a8009c00 comp rx=0 tx=0).stop 2023-09-30T22:36:30.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.575+0000 7f21b4771700 1 -- 172.21.15.16:0/3846865253 shutdown_connections 2023-09-30T22:36:30.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.575+0000 7f21b4771700 1 --2- 172.21.15.16:0/3846865253 >> v2:172.21.15.16:3300/0 conn(0x7f21ac1380f0 0x7f21ac13fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:30.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.575+0000 7f21b4771700 1 --2- 172.21.15.16:0/3846865253 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f21ac06c0e0 0x7f21ac09ac90 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:30.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.575+0000 7f21b4771700 1 -- 172.21.15.16:0/3846865253 >> 172.21.15.16:0/3846865253 conn(0x7f21ac0a4210 msgr2=0x7f21ac0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:30.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.575+0000 7f21b4771700 1 -- 172.21.15.16:0/3846865253 shutdown_connections 2023-09-30T22:36:30.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.575+0000 7f21b4771700 1 -- 172.21.15.16:0/3846865253 wait complete. 2023-09-30T22:36:30.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.576+0000 7f21b4771700 1 Processor -- start 2023-09-30T22:36:30.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.576+0000 7f21b4771700 1 -- start start 2023-09-30T22:36:30.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.577+0000 7f21b4771700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21ac06c0e0 0x7f21ac11a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:30.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.577+0000 7f21b4771700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f21ac134ec0 0x7f21ac11abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:30.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.577+0000 7f21b4771700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f21ac1380f0 0x7f21ac11b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:30.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.577+0000 7f21b4771700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac11da70 con 0x7f21ac06c0e0 2023-09-30T22:36:30.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.577+0000 7f21b4771700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac11c6d0 con 0x7f21ac134ec0 2023-09-30T22:36:30.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.577+0000 7f21b4771700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac11d6c0 con 0x7f21ac1380f0 2023-09-30T22:36:30.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.577+0000 7f21b250d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21ac06c0e0 0x7f21ac11a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:30.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.577+0000 7f21b250d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21ac06c0e0 0x7f21ac11a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51320/0 (socket says 172.21.15.16:51320) 2023-09-30T22:36:30.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.577+0000 7f21b250d700 1 -- 172.21.15.16:0/4056812299 learned_addr learned my addr 172.21.15.16:0/4056812299 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:30.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.577+0000 7f21b1d0c700 1 --2- 172.21.15.16:0/4056812299 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f21ac134ec0 0x7f21ac11abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:30.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.577+0000 7f21b250d700 1 -- 172.21.15.16:0/4056812299 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f21ac1380f0 msgr2=0x7f21ac11b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:30.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.577+0000 7f21b250d700 1 --2- 172.21.15.16:0/4056812299 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f21ac1380f0 0x7f21ac11b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:30.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.577+0000 7f21b250d700 1 -- 172.21.15.16:0/4056812299 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f21ac134ec0 msgr2=0x7f21ac11abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:30.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.577+0000 7f21b250d700 1 --2- 172.21.15.16:0/4056812299 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f21ac134ec0 0x7f21ac11abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:30.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.577+0000 7f21b250d700 1 -- 172.21.15.16:0/4056812299 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21a8009550 con 0x7f21ac06c0e0 2023-09-30T22:36:30.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.577+0000 7f21b1d0c700 1 --2- 172.21.15.16:0/4056812299 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f21ac134ec0 0x7f21ac11abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:36:30.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.577+0000 7f21b250d700 1 --2- 172.21.15.16:0/4056812299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21ac06c0e0 0x7f21ac11a680 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f21a8014040 tx=0x7f21a8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:30.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.578+0000 7f21a2ffd700 1 -- 172.21.15.16:0/4056812299 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a80036a0 con 0x7f21ac06c0e0 2023-09-30T22:36:30.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.578+0000 7f21a2ffd700 1 -- 172.21.15.16:0/4056812299 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21a801e070 con 0x7f21ac06c0e0 2023-09-30T22:36:30.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.578+0000 7f21b4771700 1 -- 172.21.15.16:0/4056812299 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21ac11d270 con 0x7f21ac06c0e0 2023-09-30T22:36:30.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.578+0000 7f21a2ffd700 1 -- 172.21.15.16:0/4056812299 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a8003a60 con 0x7f21ac06c0e0 2023-09-30T22:36:30.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.578+0000 7f21b4771700 1 -- 172.21.15.16:0/4056812299 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21ac11c390 con 0x7f21ac06c0e0 2023-09-30T22:36:30.587 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.579+0000 7f21a2ffd700 1 -- 172.21.15.16:0/4056812299 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f21a8003bc0 con 0x7f21ac06c0e0 2023-09-30T22:36:30.588 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.579+0000 7f0442f43700 1 Processor -- start 2023-09-30T22:36:30.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.579+0000 7f21a2ffd700 1 -- 172.21.15.16:0/4056812299 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f21a8040790 con 0x7f21ac06c0e0 2023-09-30T22:36:30.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.579+0000 7f0442f43700 1 -- start start 2023-09-30T22:36:30.590 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.579+0000 7f21b4771700 1 -- 172.21.15.16:0/4056812299 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21ac11dbb0 con 0x7f21ac06c0e0 2023-09-30T22:36:30.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.580+0000 7f0442f43700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f043c09e930 0x7f043c09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:30.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.580+0000 7f0442f43700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f043c09f650 0x7f043c131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:30.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.580+0000 7f0442f43700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f043c070940 con 0x7f043c09f240 2023-09-30T22:36:30.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.580+0000 7f0442f43700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f043c06fd60 con 0x7f043c09e930 2023-09-30T22:36:30.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.580+0000 7f0442f43700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f043c06fa80 con 0x7f043c09f650 2023-09-30T22:36:30.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.580+0000 7f0440cdf700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f043c09e930 0x7f043c09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:30.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.580+0000 7f0440cdf700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f043c09e930 0x7f043c09ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51322/0 (socket says 172.21.15.16:51322) 2023-09-30T22:36:30.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.580+0000 7f0440cdf700 1 -- 172.21.15.16:0/3450650936 learned_addr learned my addr 172.21.15.16:0/3450650936 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:30.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.580+0000 7f04414e0700 1 --2- 172.21.15.16:0/3450650936 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f043c09f650 0x7f043c131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:30.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.581+0000 7f0440cdf700 1 -- 172.21.15.16:0/3450650936 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f043c09f650 msgr2=0x7f043c131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:30.593 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.581+0000 7f0440cdf700 1 --2- 172.21.15.16:0/3450650936 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f043c09f650 0x7f043c131f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:30.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.581+0000 7f0440cdf700 1 -- 172.21.15.16:0/3450650936 >> v1:172.21.15.16:6789/0 conn(0x7f043c09f240 legacy=0x7f043c1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:30.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.581+0000 7f0440cdf700 1 -- 172.21.15.16:0/3450650936 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f043c09fb60 con 0x7f043c09e930 2023-09-30T22:36:30.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.581+0000 7f0440cdf700 1 --2- 172.21.15.16:0/3450650936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f043c09e930 0x7f043c09ed00 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f04300098f0 tx=0x7f0430009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb26240dbd7cb545 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:30.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.585+0000 7f043b7fe700 1 -- 172.21.15.16:0/3450650936 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0430004230 con 0x7f043c09e930 2023-09-30T22:36:30.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.585+0000 7f043b7fe700 1 -- 172.21.15.16:0/3450650936 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0430004390 con 0x7f043c09e930 2023-09-30T22:36:30.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.585+0000 7f043b7fe700 1 -- 172.21.15.16:0/3450650936 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04300036a0 con 0x7f043c09e930 2023-09-30T22:36:30.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.585+0000 7f21a2ffd700 1 -- 172.21.15.16:0/4056812299 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f21a8015080 con 0x7f21ac06c0e0 2023-09-30T22:36:30.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.586+0000 7f0442f43700 1 -- 172.21.15.16:0/3450650936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f043c09e930 msgr2=0x7f043c09ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:30.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.586+0000 7f0442f43700 1 --2- 172.21.15.16:0/3450650936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f043c09e930 0x7f043c09ed00 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f04300098f0 tx=0x7f0430009c00 comp rx=0 tx=0).stop 2023-09-30T22:36:30.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.586+0000 7f0442f43700 1 -- 172.21.15.16:0/3450650936 shutdown_connections 2023-09-30T22:36:30.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.586+0000 7f0442f43700 1 --2- 172.21.15.16:0/3450650936 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f043c09f650 0x7f043c131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:30.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.586+0000 7f0442f43700 1 --2- 172.21.15.16:0/3450650936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f043c09e930 0x7f043c09ed00 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:30.597 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.586+0000 7f0442f43700 1 -- 172.21.15.16:0/3450650936 >> 172.21.15.16:0/3450650936 conn(0x7f043c0a4210 msgr2=0x7f043c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:30.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.586+0000 7f0442f43700 1 -- 172.21.15.16:0/3450650936 shutdown_connections 2023-09-30T22:36:30.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.586+0000 7f0442f43700 1 -- 172.21.15.16:0/3450650936 wait complete. 2023-09-30T22:36:30.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.587+0000 7f0442f43700 1 Processor -- start 2023-09-30T22:36:30.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.587+0000 7f0442f43700 1 -- start start 2023-09-30T22:36:30.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.588+0000 7f0442f43700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f043c09e930 0x7f043c1a3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:30.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.588+0000 7f0442f43700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f043c09f240 0x7f043c1a4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:30.598 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.588+0000 7f0442f43700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f043c09f650 0x7f043c1a46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:30.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.588+0000 7f0442f43700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043c1a5850 con 0x7f043c09e930 2023-09-30T22:36:30.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.588+0000 7f0442f43700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f043c1a6aa0 con 0x7f043c09f650 2023-09-30T22:36:30.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.588+0000 7f0442f43700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f043c1a5e30 con 0x7f043c09f240 2023-09-30T22:36:30.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.589+0000 7f0440cdf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f043c09e930 0x7f043c1a3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:30.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.589+0000 7f0440cdf700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f043c09e930 0x7f043c1a3c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51336/0 (socket says 172.21.15.16:51336) 2023-09-30T22:36:30.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.589+0000 7f0440cdf700 1 -- 172.21.15.16:0/1600392962 learned_addr learned my addr 172.21.15.16:0/1600392962 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:30.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.589+0000 7f04414e0700 1 --2- 172.21.15.16:0/1600392962 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f043c09f650 0x7f043c1a46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:30.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.589+0000 7f0440cdf700 1 -- 172.21.15.16:0/1600392962 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f043c09f240 msgr2=0x7f043c1a4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:30.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.589+0000 7f0440cdf700 1 --2- 172.21.15.16:0/1600392962 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f043c09f240 0x7f043c1a4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:30.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.589+0000 7f0440cdf700 1 -- 172.21.15.16:0/1600392962 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f043c09f650 msgr2=0x7f043c1a46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:30.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.589+0000 7f0440cdf700 1 --2- 172.21.15.16:0/1600392962 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f043c09f650 0x7f043c1a46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:30.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.589+0000 7f0440cdf700 1 -- 172.21.15.16:0/1600392962 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0430009550 con 0x7f043c09e930 2023-09-30T22:36:30.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.589+0000 7f0440cdf700 1 --2- 172.21.15.16:0/1600392962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f043c09e930 0x7f043c1a3c40 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f043000b0e0 tx=0x7f0430004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:30.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.589+0000 7f04397fa700 1 -- 172.21.15.16:0/1600392962 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0430003cf0 con 0x7f043c09e930 2023-09-30T22:36:30.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.589+0000 7f04397fa700 1 -- 172.21.15.16:0/1600392962 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0430027070 con 0x7f043c09e930 2023-09-30T22:36:30.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.589+0000 7f04397fa700 1 -- 172.21.15.16:0/1600392962 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f043000b8a0 con 0x7f043c09e930 2023-09-30T22:36:30.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.589+0000 7f0442f43700 1 -- 172.21.15.16:0/1600392962 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f043c0a0190 con 0x7f043c09e930 2023-09-30T22:36:30.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.590+0000 7f0442f43700 1 -- 172.21.15.16:0/1600392962 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f043c1a74e0 con 0x7f043c09e930 2023-09-30T22:36:30.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.590+0000 7f04397fa700 1 -- 172.21.15.16:0/1600392962 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0430029030 con 0x7f043c09e930 2023-09-30T22:36:30.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.591+0000 7f04397fa700 1 -- 172.21.15.16:0/1600392962 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f043004d280 con 0x7f043c09e930 2023-09-30T22:36:30.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.591+0000 7f0442f43700 1 -- 172.21.15.16:0/1600392962 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f043c1a0b30 con 0x7f043c09e930 2023-09-30T22:36:30.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.596+0000 7f04397fa700 1 -- 172.21.15.16:0/1600392962 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f043001a440 con 0x7f043c09e930 2023-09-30T22:36:30.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.639+0000 7fcd7494c700 1 Processor -- start 2023-09-30T22:36:30.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.639+0000 7fcd7494c700 1 -- start start 2023-09-30T22:36:30.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.640+0000 7fcd7494c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd7009e1d0 0x7fcd70138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:30.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.640+0000 7fcd7494c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fcd70139170 0x7fcd7013b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:30.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.640+0000 7fcd7494c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd7009b120 con 0x7fcd7009d6f0 2023-09-30T22:36:30.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.640+0000 7fcd7494c700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd7006b460 con 0x7fcd70139170 2023-09-30T22:36:30.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.640+0000 7fcd7494c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7009a7e0 con 0x7fcd7009e1d0 2023-09-30T22:36:30.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.640+0000 7fcd6dd9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd7009e1d0 0x7fcd70138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:30.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.640+0000 7fcd6dd9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd7009e1d0 0x7fcd70138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:36490/0 (socket says 172.21.15.16:36490) 2023-09-30T22:36:30.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.640+0000 7fcd6dd9b700 1 -- 172.21.15.16:0/2252965578 learned_addr learned my addr 172.21.15.16:0/2252965578 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:30.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.640+0000 7fcd6ed9d700 1 --2- 172.21.15.16:0/2252965578 >> v2:172.21.15.16:3300/0 conn(0x7fcd70139170 0x7fcd7013b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:30.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.640+0000 7fcd6dd9b700 1 -- 172.21.15.16:0/2252965578 >> v2:172.21.15.16:3300/0 conn(0x7fcd70139170 msgr2=0x7fcd7013b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:30.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.640+0000 7fcd6dd9b700 1 --2- 172.21.15.16:0/2252965578 >> v2:172.21.15.16:3300/0 conn(0x7fcd70139170 0x7fcd7013b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:30.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.640+0000 7fcd6dd9b700 1 -- 172.21.15.16:0/2252965578 >> v1:172.21.15.16:6789/0 conn(0x7fcd7009d6f0 legacy=0x7fcd7009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:30.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.641+0000 7fcd6dd9b700 1 -- 172.21.15.16:0/2252965578 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd7009a500 con 0x7fcd7009e1d0 2023-09-30T22:36:30.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.641+0000 7fcd6dd9b700 1 --2- 172.21.15.16:0/2252965578 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd7009e1d0 0x7fcd70138c30 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fcd64009920 tx=0x7fcd64009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=465635bf726dd69a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:30.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.641+0000 7fcd6d59a700 1 -- 172.21.15.16:0/2252965578 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd64011830 con 0x7fcd7009e1d0 2023-09-30T22:36:30.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.641+0000 7fcd6d59a700 1 -- 172.21.15.16:0/2252965578 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd64011990 con 0x7fcd7009e1d0 2023-09-30T22:36:30.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.641+0000 7fcd6d59a700 1 -- 172.21.15.16:0/2252965578 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd64011d20 con 0x7fcd7009e1d0 2023-09-30T22:36:30.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.642+0000 7fcd7494c700 1 -- 172.21.15.16:0/2252965578 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd7009e1d0 msgr2=0x7fcd70138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:30.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.642+0000 7fcd7494c700 1 --2- 172.21.15.16:0/2252965578 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd7009e1d0 0x7fcd70138c30 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fcd64009920 tx=0x7fcd64009c30 comp rx=0 tx=0).stop 2023-09-30T22:36:30.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.642+0000 7fcd7494c700 1 -- 172.21.15.16:0/2252965578 shutdown_connections 2023-09-30T22:36:30.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.642+0000 7fcd7494c700 1 --2- 172.21.15.16:0/2252965578 >> v2:172.21.15.16:3300/0 conn(0x7fcd70139170 0x7fcd7013b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:30.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.642+0000 7fcd7494c700 1 --2- 172.21.15.16:0/2252965578 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd7009e1d0 0x7fcd70138c30 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:30.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.642+0000 7fcd7494c700 1 -- 172.21.15.16:0/2252965578 >> 172.21.15.16:0/2252965578 conn(0x7fcd70066f50 msgr2=0x7fcd7009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:30.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.642+0000 7fcd7494c700 1 -- 172.21.15.16:0/2252965578 shutdown_connections 2023-09-30T22:36:30.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.642+0000 7fcd7494c700 1 -- 172.21.15.16:0/2252965578 wait complete. 2023-09-30T22:36:30.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.644+0000 7fcd7494c700 1 Processor -- start 2023-09-30T22:36:30.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.644+0000 7fcd7494c700 1 -- start start 2023-09-30T22:36:30.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.644+0000 7fcd7494c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd7009d6f0 0x7fcd7011e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:30.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.644+0000 7fcd7494c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd7009e1d0 0x7fcd7011eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:30.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.644+0000 7fcd7494c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd70139170 0x7fcd7011f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:30.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.644+0000 7fcd7494c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7009c710 con 0x7fcd7009e1d0 2023-09-30T22:36:30.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.644+0000 7fcd7494c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd70120510 con 0x7fcd7009d6f0 2023-09-30T22:36:30.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.644+0000 7fcd7494c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcd701220a0 con 0x7fcd70139170 2023-09-30T22:36:30.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.644+0000 7fcd6e59c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd7009d6f0 0x7fcd7011e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:30.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.645+0000 7fcd6e59c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd7009d6f0 0x7fcd7011e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36172/0 (socket says 172.21.15.16:36172) 2023-09-30T22:36:30.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.645+0000 7fcd6e59c700 1 -- 172.21.15.16:0/4134716343 learned_addr learned my addr 172.21.15.16:0/4134716343 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:30.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.645+0000 7fcd6dd9b700 1 --2- 172.21.15.16:0/4134716343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd7009e1d0 0x7fcd7011eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:30.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.645+0000 7fcd6ed9d700 1 --2- 172.21.15.16:0/4134716343 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd70139170 0x7fcd7011f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:30.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.645+0000 7fcd6dd9b700 1 -- 172.21.15.16:0/4134716343 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd70139170 msgr2=0x7fcd7011f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:30.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.645+0000 7fcd6dd9b700 1 --2- 172.21.15.16:0/4134716343 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd70139170 0x7fcd7011f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:30.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.645+0000 7fcd6dd9b700 1 -- 172.21.15.16:0/4134716343 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd7009d6f0 msgr2=0x7fcd7011e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:30.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.645+0000 7fcd6dd9b700 1 --2- 172.21.15.16:0/4134716343 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd7009d6f0 0x7fcd7011e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:30.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.645+0000 7fcd6dd9b700 1 -- 172.21.15.16:0/4134716343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd64009550 con 0x7fcd7009e1d0 2023-09-30T22:36:30.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.645+0000 7fcd6e59c700 1 --2- 172.21.15.16:0/4134716343 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd7009d6f0 0x7fcd7011e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:30.649 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.645+0000 7fcd6dd9b700 1 --2- 172.21.15.16:0/4134716343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd7009e1d0 0x7fcd7011eda0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fcd64016000 tx=0x7fcd64004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:30.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.645+0000 7fcd5effd700 1 -- 172.21.15.16:0/4134716343 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd64004590 con 0x7fcd7009e1d0 2023-09-30T22:36:30.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.646+0000 7fcd5effd700 1 -- 172.21.15.16:0/4134716343 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd640036a0 con 0x7fcd7009e1d0 2023-09-30T22:36:30.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.646+0000 7fcd7494c700 1 -- 172.21.15.16:0/4134716343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd700a20b0 con 0x7fcd7009e1d0 2023-09-30T22:36:30.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.646+0000 7fcd5effd700 1 -- 172.21.15.16:0/4134716343 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd64003a50 con 0x7fcd7009e1d0 2023-09-30T22:36:30.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.646+0000 7fcd7494c700 1 -- 172.21.15.16:0/4134716343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd70122b10 con 0x7fcd7009e1d0 2023-09-30T22:36:30.650 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.647+0000 7fcd5effd700 1 -- 172.21.15.16:0/4134716343 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcd64003bb0 con 0x7fcd7009e1d0 2023-09-30T22:36:30.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.647+0000 7fcd7494c700 1 -- 172.21.15.16:0/4134716343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd70074ea0 con 0x7fcd7009e1d0 2023-09-30T22:36:30.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.647+0000 7fcd5effd700 1 -- 172.21.15.16:0/4134716343 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fcd64046060 con 0x7fcd7009e1d0 2023-09-30T22:36:30.652 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.652+0000 7fcd5effd700 1 -- 172.21.15.16:0/4134716343 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcd64015070 con 0x7fcd7009e1d0 2023-09-30T22:36:30.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.798+0000 7f21b4771700 1 -- 172.21.15.16:0/4056812299 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f21ac11ddb0 con 0x7f21ac06c0e0 2023-09-30T22:36:30.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.798+0000 7f21a2ffd700 1 -- 172.21.15.16:0/4056812299 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f21a801a420 con 0x7f21ac06c0e0 2023-09-30T22:36:30.799 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:36:30.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.800+0000 7f21b4771700 1 -- 172.21.15.16:0/4056812299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21ac06c0e0 msgr2=0x7f21ac11a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:30.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.800+0000 7f21b4771700 1 --2- 172.21.15.16:0/4056812299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21ac06c0e0 0x7f21ac11a680 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f21a8014040 tx=0x7f21a8004750 comp rx=0 tx=0).stop 2023-09-30T22:36:30.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.800+0000 7f21b4771700 1 -- 172.21.15.16:0/4056812299 shutdown_connections 2023-09-30T22:36:30.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.800+0000 7f21b4771700 1 --2- 172.21.15.16:0/4056812299 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f21ac1380f0 0x7f21ac11b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:30.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.800+0000 7f21b4771700 1 --2- 172.21.15.16:0/4056812299 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f21ac06c0e0 0x7f21ac11a680 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:30.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.800+0000 7f21b4771700 1 --2- 172.21.15.16:0/4056812299 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f21ac134ec0 0x7f21ac11abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:30.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.800+0000 7f21b4771700 1 -- 172.21.15.16:0/4056812299 >> 172.21.15.16:0/4056812299 conn(0x7f21ac0a4210 msgr2=0x7f21ac071960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:30.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.800+0000 7f21b4771700 1 -- 172.21.15.16:0/4056812299 shutdown_connections 2023-09-30T22:36:30.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.800+0000 7f21b4771700 1 -- 172.21.15.16:0/4056812299 wait complete. 2023-09-30T22:36:30.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.815+0000 7f4ad6a69700 1 -- 172.21.15.16:0/1212677212 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4ad006b250 con 0x7f4ad009f240 2023-09-30T22:36:30.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.816+0000 7f4acd7fa700 1 -- 172.21.15.16:0/1212677212 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4abc01b030 con 0x7f4ad009f240 2023-09-30T22:36:30.816 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:36:30.819 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:36:30.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.819+0000 7f4ad6a69700 1 -- 172.21.15.16:0/1212677212 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4ad009f240 msgr2=0x7f4ad01c1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:30.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.819+0000 7f4ad6a69700 1 --2- 172.21.15.16:0/1212677212 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4ad009f240 0x7f4ad01c1340 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f4abc00bea0 tx=0x7f4abc00bed0 comp rx=0 tx=0).stop 2023-09-30T22:36:30.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.819+0000 7f4ad6a69700 1 -- 172.21.15.16:0/1212677212 shutdown_connections 2023-09-30T22:36:30.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.819+0000 7f4ad6a69700 1 --2- 172.21.15.16:0/1212677212 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4ad009e930 0x7f4ad01c0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:30.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.819+0000 7f4ad6a69700 1 --2- 172.21.15.16:0/1212677212 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ad009f650 0x7f4ad01c3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:30.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.819+0000 7f4ad6a69700 1 --2- 172.21.15.16:0/1212677212 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4ad009f240 0x7f4ad01c1340 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:30.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.819+0000 7f4ad6a69700 1 -- 172.21.15.16:0/1212677212 >> 172.21.15.16:0/1212677212 conn(0x7f4ad00a4210 msgr2=0x7f4ad01b69a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:30.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.819+0000 7f4ad6a69700 1 -- 172.21.15.16:0/1212677212 shutdown_connections 2023-09-30T22:36:30.821 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.820+0000 7f4ad6a69700 1 -- 172.21.15.16:0/1212677212 wait complete. 2023-09-30T22:36:30.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.828+0000 7f0442f43700 1 -- 172.21.15.16:0/1600392962 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f043c1a0b30 con 0x7f043c09e930 2023-09-30T22:36:30.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.828+0000 7f04397fa700 1 -- 172.21.15.16:0/1600392962 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0430010b70 con 0x7f043c09e930 2023-09-30T22:36:30.829 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:36:30.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.830+0000 7f0442f43700 1 -- 172.21.15.16:0/1600392962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f043c09e930 msgr2=0x7f043c1a3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:30.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.830+0000 7f0442f43700 1 --2- 172.21.15.16:0/1600392962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f043c09e930 0x7f043c1a3c40 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f043000b0e0 tx=0x7f0430004750 comp rx=0 tx=0).stop 2023-09-30T22:36:30.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.831+0000 7f0442f43700 1 -- 172.21.15.16:0/1600392962 shutdown_connections 2023-09-30T22:36:30.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.831+0000 7f0442f43700 1 --2- 172.21.15.16:0/1600392962 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f043c09f650 0x7f043c1a46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:30.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.831+0000 7f0442f43700 1 --2- 172.21.15.16:0/1600392962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f043c09e930 0x7f043c1a3c40 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:30.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.831+0000 7f0442f43700 1 --2- 172.21.15.16:0/1600392962 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f043c09f240 0x7f043c1a4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:30.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.831+0000 7f0442f43700 1 -- 172.21.15.16:0/1600392962 >> 172.21.15.16:0/1600392962 conn(0x7f043c0a4210 msgr2=0x7f043c1b8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:30.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.831+0000 7f0442f43700 1 -- 172.21.15.16:0/1600392962 shutdown_connections 2023-09-30T22:36:30.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.831+0000 7f0442f43700 1 -- 172.21.15.16:0/1600392962 wait complete. 2023-09-30T22:36:30.834 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:36:30.845 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:36:30.892 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.892+0000 7fcd7494c700 1 -- 172.21.15.16:0/4134716343 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fcd7006c360 con 0x7fcd7009e1d0 2023-09-30T22:36:30.893 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.892+0000 7fcd5effd700 1 -- 172.21.15.16:0/4134716343 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcd64015020 con 0x7fcd7009e1d0 2023-09-30T22:36:30.893 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:36:30.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.895+0000 7fcd7494c700 1 -- 172.21.15.16:0/4134716343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd7009e1d0 msgr2=0x7fcd7011eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:30.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.895+0000 7fcd7494c700 1 --2- 172.21.15.16:0/4134716343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd7009e1d0 0x7fcd7011eda0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fcd64016000 tx=0x7fcd64004750 comp rx=0 tx=0).stop 2023-09-30T22:36:30.895 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.895+0000 7fcd7494c700 1 -- 172.21.15.16:0/4134716343 shutdown_connections 2023-09-30T22:36:30.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.895+0000 7fcd7494c700 1 --2- 172.21.15.16:0/4134716343 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcd7009d6f0 0x7fcd7011e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:30.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.895+0000 7fcd7494c700 1 --2- 172.21.15.16:0/4134716343 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcd70139170 0x7fcd7011f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:30.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.895+0000 7fcd7494c700 1 --2- 172.21.15.16:0/4134716343 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd7009e1d0 0x7fcd7011eda0 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:30.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.895+0000 7fcd7494c700 1 -- 172.21.15.16:0/4134716343 >> 172.21.15.16:0/4134716343 conn(0x7fcd70066f50 msgr2=0x7fcd70136c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:30.896 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.895+0000 7fcd7494c700 1 -- 172.21.15.16:0/4134716343 shutdown_connections 2023-09-30T22:36:30.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:30.895+0000 7fcd7494c700 1 -- 172.21.15.16:0/4134716343 wait complete. 2023-09-30T22:36:30.908 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:36:31.819 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:36:31.835 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:36:31.846 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:36:31.910 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:36:31.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:31.974+0000 7f984bb51700 1 Processor -- start 2023-09-30T22:36:31.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:31.974+0000 7f984bb51700 1 -- start start 2023-09-30T22:36:31.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:31.975+0000 7f984bb51700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f984409d6f0 0x7f984409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:31.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:31.975+0000 7f984bb51700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f984409e000 0x7f98441339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:31.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:31.975+0000 7f984bb51700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f984409e410 0x7f9844133f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:31.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:31.975+0000 7f984bb51700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f984406b460 con 0x7f984409d6f0 2023-09-30T22:36:31.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:31.975+0000 7f984bb51700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f984409b120 con 0x7f984409e410 2023-09-30T22:36:31.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:31.975+0000 7f984bb51700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f984409a7e0 con 0x7f984409e000 2023-09-30T22:36:31.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:31.975+0000 7f984a0ee700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f984409e410 0x7f9844133f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:31.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:31.975+0000 7f984a0ee700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f984409e410 0x7f9844133f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36178/0 (socket says 172.21.15.16:36178) 2023-09-30T22:36:31.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:31.975+0000 7f984a0ee700 1 -- 172.21.15.16:0/373035947 learned_addr learned my addr 172.21.15.16:0/373035947 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:31.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:31.975+0000 7f98498ed700 1 --2- 172.21.15.16:0/373035947 >> v2:172.21.15.16:3300/0 conn(0x7f984409d6f0 0x7f984409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:31.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:31.976+0000 7f98490ec700 1 --2- 172.21.15.16:0/373035947 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f984409e000 0x7f98441339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:31.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:31.976+0000 7f984a0ee700 1 -- 172.21.15.16:0/373035947 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f984409e000 msgr2=0x7f98441339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:31.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:31.976+0000 7f984a0ee700 1 --2- 172.21.15.16:0/373035947 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f984409e000 0x7f98441339f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:31.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:31.976+0000 7f984a0ee700 1 -- 172.21.15.16:0/373035947 >> v2:172.21.15.16:3300/0 conn(0x7f984409d6f0 msgr2=0x7f984409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:31.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:31.976+0000 7f984a0ee700 1 --2- 172.21.15.16:0/373035947 >> v2:172.21.15.16:3300/0 conn(0x7f984409d6f0 0x7f984409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:31.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:31.976+0000 7f984a0ee700 1 -- 172.21.15.16:0/373035947 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f984409a500 con 0x7f984409e410 2023-09-30T22:36:31.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:31.976+0000 7f984a0ee700 1 --2- 172.21.15.16:0/373035947 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f984409e410 0x7f9844133f30 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f983c009e80 tx=0x7f983c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84c0e32113fdde41 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:31.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:31.976+0000 7f98488eb700 1 -- 172.21.15.16:0/373035947 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983c00e070 con 0x7f984409e410 2023-09-30T22:36:31.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:31.976+0000 7f98488eb700 1 -- 172.21.15.16:0/373035947 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f983c00b8d0 con 0x7f984409e410 2023-09-30T22:36:31.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:31.976+0000 7f98488eb700 1 -- 172.21.15.16:0/373035947 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983c00bc80 con 0x7f984409e410 2023-09-30T22:36:31.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:31.977+0000 7f984bb51700 1 -- 172.21.15.16:0/373035947 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f984409e410 msgr2=0x7f9844133f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:31.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:31.977+0000 7f984bb51700 1 --2- 172.21.15.16:0/373035947 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f984409e410 0x7f9844133f30 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f983c009e80 tx=0x7f983c00b330 comp rx=0 tx=0).stop 2023-09-30T22:36:31.982 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:31.977+0000 7f984bb51700 1 -- 172.21.15.16:0/373035947 shutdown_connections 2023-09-30T22:36:31.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:31.977+0000 7f984bb51700 1 --2- 172.21.15.16:0/373035947 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f984409e410 0x7f9844133f30 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:31.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:31.977+0000 7f984bb51700 1 --2- 172.21.15.16:0/373035947 >> v2:172.21.15.16:3300/0 conn(0x7f984409d6f0 0x7f984409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:31.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:31.977+0000 7f984bb51700 1 --2- 172.21.15.16:0/373035947 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f984409e000 0x7f98441339f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:31.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:31.977+0000 7f984bb51700 1 -- 172.21.15.16:0/373035947 >> 172.21.15.16:0/373035947 conn(0x7f9844066f50 msgr2=0x7f984409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:31.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:31.977+0000 7f984bb51700 1 -- 172.21.15.16:0/373035947 shutdown_connections 2023-09-30T22:36:31.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:31.977+0000 7f984bb51700 1 -- 172.21.15.16:0/373035947 wait complete. 2023-09-30T22:36:31.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:31.978+0000 7f984bb51700 1 Processor -- start 2023-09-30T22:36:31.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:31.979+0000 7f984bb51700 1 -- start start 2023-09-30T22:36:31.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:31.979+0000 7f984bb51700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f984409d6f0 0x7f98441a25a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:31.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:31.979+0000 7f984bb51700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f984409e000 0x7f98441a2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:31.983 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:31.979+0000 7f984bb51700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f984409e410 0x7f98441a3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:31.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:31.979+0000 7f984bb51700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9844072350 con 0x7f984409e410 2023-09-30T22:36:31.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:31.979+0000 7f984bb51700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9844072120 con 0x7f984409d6f0 2023-09-30T22:36:31.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:31.979+0000 7f984bb51700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f98441a5380 con 0x7f984409e000 2023-09-30T22:36:31.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:31.979+0000 7f98498ed700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f984409d6f0 0x7f98441a25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:31.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:31.979+0000 7f98498ed700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f984409d6f0 0x7f98441a25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36188/0 (socket says 172.21.15.16:36188) 2023-09-30T22:36:31.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:31.979+0000 7f98498ed700 1 -- 172.21.15.16:0/2887914175 learned_addr learned my addr 172.21.15.16:0/2887914175 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:31.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:31.979+0000 7f98490ec700 1 --2- 172.21.15.16:0/2887914175 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f984409e000 0x7f98441a2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:31.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:31.980+0000 7f98498ed700 1 -- 172.21.15.16:0/2887914175 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f984409e000 msgr2=0x7f98441a2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:31.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:31.980+0000 7f98498ed700 1 --2- 172.21.15.16:0/2887914175 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f984409e000 0x7f98441a2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:31.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:31.980+0000 7f98498ed700 1 -- 172.21.15.16:0/2887914175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f984409e410 msgr2=0x7f98441a3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:31.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:31.980+0000 7f98498ed700 1 --2- 172.21.15.16:0/2887914175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f984409e410 0x7f98441a3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:31.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:31.980+0000 7f98498ed700 1 -- 172.21.15.16:0/2887914175 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f983c009b30 con 0x7f984409d6f0 2023-09-30T22:36:31.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:31.980+0000 7f98498ed700 1 --2- 172.21.15.16:0/2887914175 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f984409d6f0 0x7f98441a25a0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f984000e9b0 tx=0x7f984000ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:31.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:31.980+0000 7f983a7fc700 1 -- 172.21.15.16:0/2887914175 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9840009710 con 0x7f984409d6f0 2023-09-30T22:36:31.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:31.980+0000 7f983a7fc700 1 -- 172.21.15.16:0/2887914175 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f984000c7f0 con 0x7f984409d6f0 2023-09-30T22:36:31.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:31.980+0000 7f984bb51700 1 -- 172.21.15.16:0/2887914175 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98441b65b0 con 0x7f984409d6f0 2023-09-30T22:36:31.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:31.980+0000 7f983a7fc700 1 -- 172.21.15.16:0/2887914175 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984000cb60 con 0x7f984409d6f0 2023-09-30T22:36:31.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:31.980+0000 7f984bb51700 1 -- 172.21.15.16:0/2887914175 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f984409c760 con 0x7f984409d6f0 2023-09-30T22:36:31.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:31.981+0000 7f983a7fc700 1 -- 172.21.15.16:0/2887914175 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f984000ccc0 con 0x7f984409d6f0 2023-09-30T22:36:31.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:31.982+0000 7f983a7fc700 1 -- 172.21.15.16:0/2887914175 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f984003f7c0 con 0x7f984409d6f0 2023-09-30T22:36:31.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:31.982+0000 7f984bb51700 1 -- 172.21.15.16:0/2887914175 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9844074ea0 con 0x7f984409d6f0 2023-09-30T22:36:31.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:31.991+0000 7f983a7fc700 1 -- 172.21.15.16:0/2887914175 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f984001be10 con 0x7f984409d6f0 2023-09-30T22:36:32.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.015+0000 7f520bdd5700 1 Processor -- start 2023-09-30T22:36:32.016 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.016+0000 7f520bdd5700 1 -- start start 2023-09-30T22:36:32.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.016+0000 7f520bdd5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f520409d6f0 0x7f520409dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:32.019 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.016+0000 7f520bdd5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f520409e000 0x7f5204130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:32.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.016+0000 7f520bdd5700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f520409e410 0x7f5204130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:32.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.016+0000 7f520bdd5700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f520409f520 con 0x7f520409e410 2023-09-30T22:36:32.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.016+0000 7f520bdd5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f520409f130 con 0x7f520409d6f0 2023-09-30T22:36:32.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.016+0000 7f520bdd5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f520409e930 con 0x7f520409e000 2023-09-30T22:36:32.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.017+0000 7f5209b71700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f520409d6f0 0x7f520409dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:32.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.017+0000 7f520a372700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f520409e410 0x7f5204130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:32.020 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.017+0000 7f520a372700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f520409e410 0x7f5204130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51374/0 (socket says 172.21.15.16:51374) 2023-09-30T22:36:32.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.017+0000 7f5209b71700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f520409d6f0 0x7f520409dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36202/0 (socket says 172.21.15.16:36202) 2023-09-30T22:36:32.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.017+0000 7f520a372700 1 -- 172.21.15.16:0/2311321083 learned_addr learned my addr 172.21.15.16:0/2311321083 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:32.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.017+0000 7f5209370700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f520409e000 0x7f5204130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:32.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.017+0000 7f520a372700 1 -- 172.21.15.16:0/2311321083 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f520409e000 msgr2=0x7f5204130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:32.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.017+0000 7f520a372700 1 --2- 172.21.15.16:0/2311321083 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f520409e000 0x7f5204130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:32.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.017+0000 7f520a372700 1 -- 172.21.15.16:0/2311321083 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f520409d6f0 msgr2=0x7f520409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:32.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.017+0000 7f520a372700 1 --2- 172.21.15.16:0/2311321083 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f520409d6f0 0x7f520409dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:32.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.017+0000 7f520a372700 1 -- 172.21.15.16:0/2311321083 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f520409b2d0 con 0x7f520409e410 2023-09-30T22:36:32.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.017+0000 7f520a372700 1 --2- 172.21.15.16:0/2311321083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f520409e410 0x7f5204130ba0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f51fc009e50 tx=0x7f51fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69decb781c86ed87 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:32.021 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.018+0000 7f5208b6f700 1 -- 172.21.15.16:0/2311321083 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51fc00e070 con 0x7f520409e410 2023-09-30T22:36:32.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.018+0000 7f5208b6f700 1 -- 172.21.15.16:0/2311321083 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51fc00b910 con 0x7f520409e410 2023-09-30T22:36:32.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.018+0000 7f5208b6f700 1 -- 172.21.15.16:0/2311321083 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51fc00bcc0 con 0x7f520409e410 2023-09-30T22:36:32.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.018+0000 7f520bdd5700 1 -- 172.21.15.16:0/2311321083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f520409e410 msgr2=0x7f5204130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:32.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.018+0000 7f520bdd5700 1 --2- 172.21.15.16:0/2311321083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f520409e410 0x7f5204130ba0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f51fc009e50 tx=0x7f51fc00b330 comp rx=0 tx=0).stop 2023-09-30T22:36:32.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.018+0000 7f520bdd5700 1 -- 172.21.15.16:0/2311321083 shutdown_connections 2023-09-30T22:36:32.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.018+0000 7f520bdd5700 1 --2- 172.21.15.16:0/2311321083 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f520409d6f0 0x7f520409dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:32.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.018+0000 7f520bdd5700 1 --2- 172.21.15.16:0/2311321083 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f520409e410 0x7f5204130ba0 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:32.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.018+0000 7f520bdd5700 1 --2- 172.21.15.16:0/2311321083 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f520409e000 0x7f5204130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:32.022 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.018+0000 7f520bdd5700 1 -- 172.21.15.16:0/2311321083 >> 172.21.15.16:0/2311321083 conn(0x7f5204066f50 msgr2=0x7f520409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:32.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.019+0000 7f520bdd5700 1 -- 172.21.15.16:0/2311321083 shutdown_connections 2023-09-30T22:36:32.023 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.019+0000 7f520bdd5700 1 -- 172.21.15.16:0/2311321083 wait complete. 2023-09-30T22:36:32.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.020+0000 7f520bdd5700 1 Processor -- start 2023-09-30T22:36:32.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.020+0000 7f520bdd5700 1 -- start start 2023-09-30T22:36:32.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.020+0000 7f520bdd5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f520409d6f0 0x7f52041222a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:32.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.020+0000 7f520bdd5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f520409e000 0x7f52041227e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:32.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.020+0000 7f520bdd5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f520409e410 0x7f5204122d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:32.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.020+0000 7f520bdd5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5204125a90 con 0x7f520409e000 2023-09-30T22:36:32.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.020+0000 7f520bdd5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5204123f70 con 0x7f520409d6f0 2023-09-30T22:36:32.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.020+0000 7f520bdd5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f520409c700 con 0x7f520409e410 2023-09-30T22:36:32.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.020+0000 7f5209370700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f520409e000 0x7f52041227e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:32.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.020+0000 7f5209370700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f520409e000 0x7f52041227e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51388/0 (socket says 172.21.15.16:51388) 2023-09-30T22:36:32.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.020+0000 7f5209370700 1 -- 172.21.15.16:0/750700532 learned_addr learned my addr 172.21.15.16:0/750700532 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:32.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.020+0000 7f520a372700 1 --2- 172.21.15.16:0/750700532 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f520409e410 0x7f5204122d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:32.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.021+0000 7f5209b71700 1 --2- 172.21.15.16:0/750700532 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f520409d6f0 0x7f52041222a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:32.029 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.021+0000 7f520a372700 1 -- 172.21.15.16:0/750700532 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f520409d6f0 msgr2=0x7f52041222a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:32.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.021+0000 7f520a372700 1 --2- 172.21.15.16:0/750700532 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f520409d6f0 0x7f52041222a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:32.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.021+0000 7f520a372700 1 -- 172.21.15.16:0/750700532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f520409e000 msgr2=0x7f52041227e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:32.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.021+0000 7f520a372700 1 --2- 172.21.15.16:0/750700532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f520409e000 0x7f52041227e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:32.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.021+0000 7f520a372700 1 -- 172.21.15.16:0/750700532 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51fc009b30 con 0x7f520409e410 2023-09-30T22:36:32.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.021+0000 7f5209b71700 1 --2- 172.21.15.16:0/750700532 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f520409d6f0 0x7f52041222a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:36:32.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.021+0000 7f5209370700 1 --2- 172.21.15.16:0/750700532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f520409e000 0x7f52041227e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:32.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.021+0000 7f520a372700 1 --2- 172.21.15.16:0/750700532 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f520409e410 0x7f5204122d20 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f51fc015000 tx=0x7f51fc004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:32.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.021+0000 7f51fa7fc700 1 -- 172.21.15.16:0/750700532 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51fc004450 con 0x7f520409e410 2023-09-30T22:36:32.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.021+0000 7f51fa7fc700 1 -- 172.21.15.16:0/750700532 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51fc0045b0 con 0x7f520409e410 2023-09-30T22:36:32.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.021+0000 7f520bdd5700 1 -- 172.21.15.16:0/750700532 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52040a15d0 con 0x7f520409e410 2023-09-30T22:36:32.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.021+0000 7f51fa7fc700 1 -- 172.21.15.16:0/750700532 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51fc004960 con 0x7f520409e410 2023-09-30T22:36:32.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.022+0000 7f520bdd5700 1 -- 172.21.15.16:0/750700532 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5204126500 con 0x7f520409e410 2023-09-30T22:36:32.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.023+0000 7f520bdd5700 1 -- 172.21.15.16:0/750700532 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5204074ea0 con 0x7f520409e410 2023-09-30T22:36:32.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.023+0000 7f51fa7fc700 1 -- 172.21.15.16:0/750700532 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f51fc025460 con 0x7f520409e410 2023-09-30T22:36:32.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.024+0000 7fa341f23700 1 Processor -- start 2023-09-30T22:36:32.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.024+0000 7f51fa7fc700 1 -- 172.21.15.16:0/750700532 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f51fc0415d0 con 0x7f520409e410 2023-09-30T22:36:32.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.024+0000 7fa341f23700 1 -- start start 2023-09-30T22:36:32.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.024+0000 7fa341f23700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa33c06a770 0x7fa33c06ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:32.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.024+0000 7fa341f23700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa33c06b080 0x7fa33c131600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:32.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.024+0000 7fa341f23700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa33c09ece0 con 0x7fa33c131b40 2023-09-30T22:36:32.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.024+0000 7fa341f23700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa33c09e760 con 0x7fa33c06a770 2023-09-30T22:36:32.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.024+0000 7fa341f23700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa33c09e040 con 0x7fa33c06b080 2023-09-30T22:36:32.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.025+0000 7fa33b7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa33c06a770 0x7fa33c06ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:32.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.025+0000 7fa33b7fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa33c06a770 0x7fa33c06ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51402/0 (socket says 172.21.15.16:51402) 2023-09-30T22:36:32.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.025+0000 7fa33b7fe700 1 -- 172.21.15.16:0/1411941550 learned_addr learned my addr 172.21.15.16:0/1411941550 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:32.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.025+0000 7fa33bfff700 1 --1- 172.21.15.16:0/1411941550 >> v1:172.21.15.16:6789/0 conn(0x7fa33c131b40 0x7fa33c138850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47944/0 (socket says 172.21.15.16:47944) 2023-09-30T22:36:32.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.025+0000 7fa33affd700 1 --2- 172.21.15.16:0/1411941550 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa33c06b080 0x7fa33c131600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:32.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.025+0000 7fa33b7fe700 1 -- 172.21.15.16:0/1411941550 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa33c06b080 msgr2=0x7fa33c131600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:32.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.025+0000 7fa33b7fe700 1 --2- 172.21.15.16:0/1411941550 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa33c06b080 0x7fa33c131600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:32.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.025+0000 7fa33b7fe700 1 -- 172.21.15.16:0/1411941550 >> v1:172.21.15.16:6789/0 conn(0x7fa33c131b40 legacy=0x7fa33c138850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:32.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.025+0000 7fa33b7fe700 1 -- 172.21.15.16:0/1411941550 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa33c09f630 con 0x7fa33c06a770 2023-09-30T22:36:32.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.025+0000 7fa33a7fc700 1 -- 172.21.15.16:0/1411941550 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2827403252 0 0) 0x7fa33c09ece0 con 0x7fa33c131b40 2023-09-30T22:36:32.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.025+0000 7fa33b7fe700 1 --2- 172.21.15.16:0/1411941550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa33c06a770 0x7fa33c06ab40 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7fa3300098f0 tx=0x7fa330009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d829931cc24fcb9a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:32.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.025+0000 7fa33a7fc700 1 -- 172.21.15.16:0/1411941550 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa330011830 con 0x7fa33c06a770 2023-09-30T22:36:32.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.026+0000 7fa33a7fc700 1 -- 172.21.15.16:0/1411941550 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa330011990 con 0x7fa33c06a770 2023-09-30T22:36:32.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.026+0000 7fa33a7fc700 1 -- 172.21.15.16:0/1411941550 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa330011d20 con 0x7fa33c06a770 2023-09-30T22:36:32.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.026+0000 7fa341f23700 1 -- 172.21.15.16:0/1411941550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa33c06a770 msgr2=0x7fa33c06ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:32.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.026+0000 7fa341f23700 1 --2- 172.21.15.16:0/1411941550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa33c06a770 0x7fa33c06ab40 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7fa3300098f0 tx=0x7fa330009c00 comp rx=0 tx=0).stop 2023-09-30T22:36:32.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.026+0000 7fa341f23700 1 -- 172.21.15.16:0/1411941550 shutdown_connections 2023-09-30T22:36:32.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.026+0000 7fa341f23700 1 --2- 172.21.15.16:0/1411941550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa33c06a770 0x7fa33c06ab40 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:32.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.026+0000 7fa341f23700 1 --2- 172.21.15.16:0/1411941550 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa33c06b080 0x7fa33c131600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:32.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.026+0000 7fa341f23700 1 -- 172.21.15.16:0/1411941550 >> 172.21.15.16:0/1411941550 conn(0x7fa33c066f50 msgr2=0x7fa33c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:32.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.026+0000 7fa341f23700 1 -- 172.21.15.16:0/1411941550 shutdown_connections 2023-09-30T22:36:32.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.026+0000 7fa341f23700 1 -- 172.21.15.16:0/1411941550 wait complete. 2023-09-30T22:36:32.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.027+0000 7fa341f23700 1 Processor -- start 2023-09-30T22:36:32.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.028+0000 7fa341f23700 1 -- start start 2023-09-30T22:36:32.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.028+0000 7fa341f23700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa33c06a770 0x7fa33c1270f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:32.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.028+0000 7fa341f23700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa33c06b080 0x7fa33c127630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:32.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.028+0000 7fa341f23700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa33c131b40 0x7fa33c127b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:32.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.028+0000 7fa341f23700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa33c128330 con 0x7fa33c131b40 2023-09-30T22:36:32.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.028+0000 7fa341f23700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa33c129f40 con 0x7fa33c06b080 2023-09-30T22:36:32.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.028+0000 7fa341f23700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa33c129820 con 0x7fa33c06a770 2023-09-30T22:36:32.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.028+0000 7fa33bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa33c131b40 0x7fa33c127b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:32.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.028+0000 7fa33affd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa33c06b080 0x7fa33c127630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.028+0000 7fa33b7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa33c06a770 0x7fa33c1270f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.028+0000 7fa33bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa33c131b40 0x7fa33c127b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51408/0 (socket says 172.21.15.16:51408) 2023-09-30T22:36:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.028+0000 7fa33bfff700 1 -- 172.21.15.16:0/589855100 learned_addr learned my addr 172.21.15.16:0/589855100 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.028+0000 7fa33affd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa33c06b080 0x7fa33c127630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36236/0 (socket says 172.21.15.16:36236) 2023-09-30T22:36:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.029+0000 7fa33affd700 1 -- 172.21.15.16:0/589855100 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa33c06a770 msgr2=0x7fa33c1270f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.029+0000 7fa33affd700 1 --2- 172.21.15.16:0/589855100 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa33c06a770 0x7fa33c1270f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.029+0000 7fa33affd700 1 -- 172.21.15.16:0/589855100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa33c131b40 msgr2=0x7fa33c127b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.029+0000 7fa33affd700 1 --2- 172.21.15.16:0/589855100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa33c131b40 0x7fa33c127b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.029+0000 7fa33affd700 1 -- 172.21.15.16:0/589855100 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa330009550 con 0x7fa33c06b080 2023-09-30T22:36:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.029+0000 7fa33bfff700 1 --2- 172.21.15.16:0/589855100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa33c131b40 0x7fa33c127b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.029+0000 7fa33affd700 1 --2- 172.21.15.16:0/589855100 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa33c06b080 0x7fa33c127630 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fa32c00e900 tx=0x7fa32c00ec10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:32.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.029+0000 7fa323fff700 1 -- 172.21.15.16:0/589855100 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32c013680 con 0x7fa33c06b080 2023-09-30T22:36:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.029+0000 7fa323fff700 1 -- 172.21.15.16:0/589855100 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa32c0137e0 con 0x7fa33c06b080 2023-09-30T22:36:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.029+0000 7fa323fff700 1 -- 172.21.15.16:0/589855100 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa32c013b50 con 0x7fa33c06b080 2023-09-30T22:36:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.030+0000 7fa341f23700 1 -- 172.21.15.16:0/589855100 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa33c129c70 con 0x7fa33c06b080 2023-09-30T22:36:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.030+0000 7fa341f23700 1 -- 172.21.15.16:0/589855100 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa33c131590 con 0x7fa33c06b080 2023-09-30T22:36:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.031+0000 7fa323fff700 1 -- 172.21.15.16:0/589855100 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa32c013cb0 con 0x7fa33c06b080 2023-09-30T22:36:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.031+0000 7fa323fff700 1 -- 172.21.15.16:0/589855100 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fa32c03f9a0 con 0x7fa33c06b080 2023-09-30T22:36:32.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.032+0000 7fa341f23700 1 -- 172.21.15.16:0/589855100 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa33c11ac20 con 0x7fa33c06b080 2023-09-30T22:36:32.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.033+0000 7f51fa7fc700 1 -- 172.21.15.16:0/750700532 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f51fc025710 con 0x7f520409e410 2023-09-30T22:36:32.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.037+0000 7fa323fff700 1 -- 172.21.15.16:0/589855100 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa32c017080 con 0x7fa33c06b080 2023-09-30T22:36:32.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.059+0000 7f8be02df700 1 Processor -- start 2023-09-30T22:36:32.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.059+0000 7f8be02df700 1 -- start start 2023-09-30T22:36:32.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.060+0000 7f8be02df700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8bd809d6f0 0x7f8bd809dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:32.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.060+0000 7f8be02df700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8bd809e000 0x7f8bd8130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:32.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.060+0000 7f8be02df700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bd809f130 con 0x7f8bd809e410 2023-09-30T22:36:32.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.060+0000 7f8be02df700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd809f520 con 0x7f8bd809d6f0 2023-09-30T22:36:32.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.060+0000 7f8be02df700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd809e930 con 0x7f8bd809e000 2023-09-30T22:36:32.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.060+0000 7f8bde07b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8bd809d6f0 0x7f8bd809dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:32.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.060+0000 7f8bde07b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8bd809d6f0 0x7f8bd809dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36250/0 (socket says 172.21.15.16:36250) 2023-09-30T22:36:32.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.060+0000 7f8bde07b700 1 -- 172.21.15.16:0/1126055474 learned_addr learned my addr 172.21.15.16:0/1126055474 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:32.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.060+0000 7f8bdd87a700 1 --2- 172.21.15.16:0/1126055474 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8bd809e000 0x7f8bd8130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:32.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.061+0000 7f8bde07b700 1 -- 172.21.15.16:0/1126055474 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8bd809e000 msgr2=0x7f8bd8130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:32.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.061+0000 7f8bde07b700 1 --2- 172.21.15.16:0/1126055474 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8bd809e000 0x7f8bd8130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:32.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.061+0000 7f8bdd079700 1 -- 172.21.15.16:0/1126055474 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3898770066 0 0) 0x7f8bd809f130 con 0x7f8bd809e410 2023-09-30T22:36:32.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.061+0000 7f8bde07b700 1 -- 172.21.15.16:0/1126055474 >> v1:172.21.15.16:6789/0 conn(0x7f8bd809e410 legacy=0x7f8bd8130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:32.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.061+0000 7f8bde07b700 1 -- 172.21.15.16:0/1126055474 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bd809b2d0 con 0x7f8bd809d6f0 2023-09-30T22:36:32.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.061+0000 7f8bde07b700 1 --2- 172.21.15.16:0/1126055474 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8bd809d6f0 0x7f8bd809dac0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f8bd400de80 tx=0x7f8bd400f330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e4ae86b7697887b server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:32.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.061+0000 7f8bdd079700 1 -- 172.21.15.16:0/1126055474 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd4012070 con 0x7f8bd809d6f0 2023-09-30T22:36:32.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.061+0000 7f8bdd079700 1 -- 172.21.15.16:0/1126055474 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bd400f910 con 0x7f8bd809d6f0 2023-09-30T22:36:32.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.061+0000 7f8bdd079700 1 -- 172.21.15.16:0/1126055474 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd400fcc0 con 0x7f8bd809d6f0 2023-09-30T22:36:32.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.062+0000 7f8be02df700 1 -- 172.21.15.16:0/1126055474 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8bd809d6f0 msgr2=0x7f8bd809dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:32.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.062+0000 7f8be02df700 1 --2- 172.21.15.16:0/1126055474 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8bd809d6f0 0x7f8bd809dac0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f8bd400de80 tx=0x7f8bd400f330 comp rx=0 tx=0).stop 2023-09-30T22:36:32.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.062+0000 7f8be02df700 1 -- 172.21.15.16:0/1126055474 shutdown_connections 2023-09-30T22:36:32.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.062+0000 7f8be02df700 1 --2- 172.21.15.16:0/1126055474 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8bd809d6f0 0x7f8bd809dac0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:32.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.062+0000 7f8be02df700 1 --2- 172.21.15.16:0/1126055474 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8bd809e000 0x7f8bd8130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:32.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.062+0000 7f8be02df700 1 -- 172.21.15.16:0/1126055474 >> 172.21.15.16:0/1126055474 conn(0x7f8bd8066f50 msgr2=0x7f8bd809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:32.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.063+0000 7f8be02df700 1 -- 172.21.15.16:0/1126055474 shutdown_connections 2023-09-30T22:36:32.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.063+0000 7f8be02df700 1 -- 172.21.15.16:0/1126055474 wait complete. 2023-09-30T22:36:32.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.064+0000 7f8be02df700 1 Processor -- start 2023-09-30T22:36:32.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.064+0000 7f8be02df700 1 -- start start 2023-09-30T22:36:32.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.064+0000 7f8be02df700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8bd809d6f0 0x7f8bd811ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:32.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.064+0000 7f8be02df700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bd809e000 0x7f8bd811e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:32.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.064+0000 7f8be02df700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8bd809e410 0x7f8bd811e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:32.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.064+0000 7f8be02df700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd81222d0 con 0x7f8bd809e000 2023-09-30T22:36:32.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.064+0000 7f8be02df700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd8121910 con 0x7f8bd809e410 2023-09-30T22:36:32.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.064+0000 7f8be02df700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd8120c50 con 0x7f8bd809d6f0 2023-09-30T22:36:32.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.064+0000 7f8bde87c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8bd809e410 0x7f8bd811e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:32.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.064+0000 7f8bde87c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8bd809e410 0x7f8bd811e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36260/0 (socket says 172.21.15.16:36260) 2023-09-30T22:36:32.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.064+0000 7f8bde87c700 1 -- 172.21.15.16:0/2092583011 learned_addr learned my addr 172.21.15.16:0/2092583011 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:32.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.064+0000 7f8bdd87a700 1 --2- 172.21.15.16:0/2092583011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bd809e000 0x7f8bd811e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:32.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.065+0000 7f8bde87c700 1 -- 172.21.15.16:0/2092583011 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8bd809d6f0 msgr2=0x7f8bd811ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:32.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.065+0000 7f8bde87c700 1 --2- 172.21.15.16:0/2092583011 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8bd809d6f0 0x7f8bd811ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:32.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.065+0000 7f8bde87c700 1 -- 172.21.15.16:0/2092583011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bd809e000 msgr2=0x7f8bd811e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:32.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.065+0000 7f8bde87c700 1 --2- 172.21.15.16:0/2092583011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bd809e000 0x7f8bd811e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:32.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.065+0000 7f8bde87c700 1 -- 172.21.15.16:0/2092583011 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bd400db30 con 0x7f8bd809e410 2023-09-30T22:36:32.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.065+0000 7f8bde87c700 1 --2- 172.21.15.16:0/2092583011 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8bd809e410 0x7f8bd811e870 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f8bd000c870 tx=0x7f8bd000cb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:32.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.065+0000 7f8bceffd700 1 -- 172.21.15.16:0/2092583011 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd0003e30 con 0x7f8bd809e410 2023-09-30T22:36:32.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.065+0000 7f8bceffd700 1 -- 172.21.15.16:0/2092583011 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bd0005070 con 0x7f8bd809e410 2023-09-30T22:36:32.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.065+0000 7f8be02df700 1 -- 172.21.15.16:0/2092583011 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bd8071fd0 con 0x7f8bd809e410 2023-09-30T22:36:32.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.065+0000 7f8bceffd700 1 -- 172.21.15.16:0/2092583011 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd0005420 con 0x7f8bd809e410 2023-09-30T22:36:32.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.065+0000 7f8be02df700 1 -- 172.21.15.16:0/2092583011 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bd81210a0 con 0x7f8bd809e410 2023-09-30T22:36:32.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.066+0000 7f8bceffd700 1 -- 172.21.15.16:0/2092583011 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8bd0005580 con 0x7f8bd809e410 2023-09-30T22:36:32.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.067+0000 7f8bceffd700 1 -- 172.21.15.16:0/2092583011 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f8bd003ee10 con 0x7f8bd809e410 2023-09-30T22:36:32.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.070+0000 7f8be02df700 1 -- 172.21.15.16:0/2092583011 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bd811ac20 con 0x7f8bd809e410 2023-09-30T22:36:32.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.079+0000 7f8bceffd700 1 -- 172.21.15.16:0/2092583011 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8bd0017020 con 0x7f8bd809e410 2023-09-30T22:36:32.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.231+0000 7f984bb51700 1 -- 172.21.15.16:0/2887914175 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f984406c360 con 0x7f984409d6f0 2023-09-30T22:36:32.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.232+0000 7f983a7fc700 1 -- 172.21.15.16:0/2887914175 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f984001b3b0 con 0x7f984409d6f0 2023-09-30T22:36:32.232 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:36:32.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.234+0000 7f984bb51700 1 -- 172.21.15.16:0/2887914175 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f984409d6f0 msgr2=0x7f98441a25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:32.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.234+0000 7f984bb51700 1 --2- 172.21.15.16:0/2887914175 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f984409d6f0 0x7f98441a25a0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f984000e9b0 tx=0x7f984000ecc0 comp rx=0 tx=0).stop 2023-09-30T22:36:32.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.234+0000 7f984bb51700 1 -- 172.21.15.16:0/2887914175 shutdown_connections 2023-09-30T22:36:32.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.234+0000 7f984bb51700 1 --2- 172.21.15.16:0/2887914175 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f984409d6f0 0x7f98441a25a0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:32.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.234+0000 7f984bb51700 1 --2- 172.21.15.16:0/2887914175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f984409e410 0x7f98441a3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:32.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.234+0000 7f984bb51700 1 --2- 172.21.15.16:0/2887914175 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f984409e000 0x7f98441a2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:32.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.234+0000 7f984bb51700 1 -- 172.21.15.16:0/2887914175 >> 172.21.15.16:0/2887914175 conn(0x7f9844066f50 msgr2=0x7f98440a1580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:32.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.234+0000 7f984bb51700 1 -- 172.21.15.16:0/2887914175 shutdown_connections 2023-09-30T22:36:32.236 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.234+0000 7f984bb51700 1 -- 172.21.15.16:0/2887914175 wait complete. 2023-09-30T22:36:32.249 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:36:32.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.256+0000 7f520bdd5700 1 -- 172.21.15.16:0/750700532 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f520406c360 con 0x7f520409e410 2023-09-30T22:36:32.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.257+0000 7f51fa7fc700 1 -- 172.21.15.16:0/750700532 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f51fc02ecf0 con 0x7f520409e410 2023-09-30T22:36:32.257 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:36:32.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.259+0000 7f520bdd5700 1 -- 172.21.15.16:0/750700532 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f520409e410 msgr2=0x7f5204122d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:32.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.259+0000 7f520bdd5700 1 --2- 172.21.15.16:0/750700532 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f520409e410 0x7f5204122d20 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f51fc015000 tx=0x7f51fc004ad0 comp rx=0 tx=0).stop 2023-09-30T22:36:32.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.259+0000 7f520bdd5700 1 -- 172.21.15.16:0/750700532 shutdown_connections 2023-09-30T22:36:32.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.259+0000 7f520bdd5700 1 --2- 172.21.15.16:0/750700532 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f520409d6f0 0x7f52041222a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:32.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.259+0000 7f520bdd5700 1 --2- 172.21.15.16:0/750700532 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f520409e410 0x7f5204122d20 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:32.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.259+0000 7f520bdd5700 1 --2- 172.21.15.16:0/750700532 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f520409e000 0x7f52041227e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:32.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.259+0000 7f520bdd5700 1 -- 172.21.15.16:0/750700532 >> 172.21.15.16:0/750700532 conn(0x7f5204066f50 msgr2=0x7f5204070ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:32.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.259+0000 7f520bdd5700 1 -- 172.21.15.16:0/750700532 shutdown_connections 2023-09-30T22:36:32.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.259+0000 7f520bdd5700 1 -- 172.21.15.16:0/750700532 wait complete. 2023-09-30T22:36:32.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.261+0000 7fa341f23700 1 -- 172.21.15.16:0/589855100 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa33c074ea0 con 0x7fa33c06b080 2023-09-30T22:36:32.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.262+0000 7fa323fff700 1 -- 172.21.15.16:0/589855100 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa32c01b7c0 con 0x7fa33c06b080 2023-09-30T22:36:32.262 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:36:32.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.264+0000 7fa341f23700 1 -- 172.21.15.16:0/589855100 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa33c06b080 msgr2=0x7fa33c127630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:32.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.264+0000 7fa341f23700 1 --2- 172.21.15.16:0/589855100 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa33c06b080 0x7fa33c127630 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fa32c00e900 tx=0x7fa32c00ec10 comp rx=0 tx=0).stop 2023-09-30T22:36:32.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.264+0000 7fa341f23700 1 -- 172.21.15.16:0/589855100 shutdown_connections 2023-09-30T22:36:32.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.264+0000 7fa341f23700 1 --2- 172.21.15.16:0/589855100 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa33c131b40 0x7fa33c127b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:32.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.264+0000 7fa341f23700 1 --2- 172.21.15.16:0/589855100 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa33c06a770 0x7fa33c1270f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:32.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.264+0000 7fa341f23700 1 --2- 172.21.15.16:0/589855100 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa33c06b080 0x7fa33c127630 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:32.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.264+0000 7fa341f23700 1 -- 172.21.15.16:0/589855100 >> 172.21.15.16:0/589855100 conn(0x7fa33c066f50 msgr2=0x7fa33c0a1580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:32.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.264+0000 7fa341f23700 1 -- 172.21.15.16:0/589855100 shutdown_connections 2023-09-30T22:36:32.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.264+0000 7fa341f23700 1 -- 172.21.15.16:0/589855100 wait complete. 2023-09-30T22:36:32.275 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:36:32.279 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:36:32.294 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.294+0000 7f8be02df700 1 -- 172.21.15.16:0/2092583011 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f8bd81214b0 con 0x7f8bd809e410 2023-09-30T22:36:32.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.294+0000 7f8bceffd700 1 -- 172.21.15.16:0/2092583011 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8bd0010910 con 0x7f8bd809e410 2023-09-30T22:36:32.295 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:36:32.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.296+0000 7f8be02df700 1 -- 172.21.15.16:0/2092583011 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8bd809e410 msgr2=0x7f8bd811e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:32.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.296+0000 7f8be02df700 1 --2- 172.21.15.16:0/2092583011 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8bd809e410 0x7f8bd811e870 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f8bd000c870 tx=0x7f8bd000cb80 comp rx=0 tx=0).stop 2023-09-30T22:36:32.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.296+0000 7f8be02df700 1 -- 172.21.15.16:0/2092583011 shutdown_connections 2023-09-30T22:36:32.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.296+0000 7f8be02df700 1 --2- 172.21.15.16:0/2092583011 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8bd809e410 0x7f8bd811e870 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:32.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.296+0000 7f8be02df700 1 --2- 172.21.15.16:0/2092583011 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8bd809d6f0 0x7f8bd811ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:32.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.296+0000 7f8be02df700 1 --2- 172.21.15.16:0/2092583011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bd809e000 0x7f8bd811e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:32.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.296+0000 7f8be02df700 1 -- 172.21.15.16:0/2092583011 >> 172.21.15.16:0/2092583011 conn(0x7f8bd8066f50 msgr2=0x7f8bd8130920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:32.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.296+0000 7f8be02df700 1 -- 172.21.15.16:0/2092583011 shutdown_connections 2023-09-30T22:36:32.298 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:32.296+0000 7f8be02df700 1 -- 172.21.15.16:0/2092583011 wait complete. 2023-09-30T22:36:32.310 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:36:33.250 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:36:33.277 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:36:33.281 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:36:33.311 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:36:33.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.408+0000 7fe6310af700 1 Processor -- start 2023-09-30T22:36:33.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.408+0000 7fe6310af700 1 -- start start 2023-09-30T22:36:33.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.409+0000 7fe6310af700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe62c09e930 0x7fe62c09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:33.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.409+0000 7fe6310af700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe62c09f650 0x7fe62c131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:33.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.409+0000 7fe6310af700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe62c070940 con 0x7fe62c09f240 2023-09-30T22:36:33.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.409+0000 7fe6310af700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe62c06fd60 con 0x7fe62c09e930 2023-09-30T22:36:33.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.409+0000 7fe6310af700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe62c06fa80 con 0x7fe62c09f650 2023-09-30T22:36:33.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.409+0000 7fe62a59c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe62c09f240 0x7fe62c1317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47964/0 (socket says 172.21.15.16:47964) 2023-09-30T22:36:33.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.409+0000 7fe62a59c700 1 -- 172.21.15.16:0/234015755 learned_addr learned my addr 172.21.15.16:0/234015755 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:36:33.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.409+0000 7fe62ad9d700 1 --2- 172.21.15.16:0/234015755 >> v2:172.21.15.16:3300/0 conn(0x7fe62c09e930 0x7fe62c09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:33.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.409+0000 7fe62b59e700 1 --2- 172.21.15.16:0/234015755 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe62c09f650 0x7fe62c131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:33.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.409+0000 7fe629d9b700 1 -- 172.21.15.16:0/234015755 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3376596501 0 0) 0x7fe62c070940 con 0x7fe62c09f240 2023-09-30T22:36:33.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.409+0000 7fe629d9b700 1 -- 172.21.15.16:0/234015755 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6140035b0 con 0x7fe62c09f240 2023-09-30T22:36:33.412 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.409+0000 7fe62ad9d700 1 -- 172.21.15.16:0/234015755 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe62c09f650 msgr2=0x7fe62c131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:33.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.409+0000 7fe62ad9d700 1 --2- 172.21.15.16:0/234015755 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe62c09f650 0x7fe62c131f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:33.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.409+0000 7fe62ad9d700 1 -- 172.21.15.16:0/234015755 >> v1:172.21.15.16:6789/0 conn(0x7fe62c09f240 legacy=0x7fe62c1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:33.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.409+0000 7fe62ad9d700 1 -- 172.21.15.16:0/234015755 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe62c09fb60 con 0x7fe62c09e930 2023-09-30T22:36:33.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.410+0000 7fe62ad9d700 1 --2- 172.21.15.16:0/234015755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe62c09e930 0x7fe62c09ed00 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fe61c009840 tx=0x7fe61c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c539815f6fd69ee6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:33.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.410+0000 7fe629d9b700 1 -- 172.21.15.16:0/234015755 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61c011830 con 0x7fe62c09e930 2023-09-30T22:36:33.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.410+0000 7fe629d9b700 1 -- 172.21.15.16:0/234015755 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe61c011990 con 0x7fe62c09e930 2023-09-30T22:36:33.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.410+0000 7fe629d9b700 1 -- 172.21.15.16:0/234015755 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61c011d20 con 0x7fe62c09e930 2023-09-30T22:36:33.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.410+0000 7fe6310af700 1 -- 172.21.15.16:0/234015755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe62c09e930 msgr2=0x7fe62c09ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:33.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.410+0000 7fe6310af700 1 --2- 172.21.15.16:0/234015755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe62c09e930 0x7fe62c09ed00 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fe61c009840 tx=0x7fe61c009b50 comp rx=0 tx=0).stop 2023-09-30T22:36:33.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.411+0000 7fe6310af700 1 -- 172.21.15.16:0/234015755 shutdown_connections 2023-09-30T22:36:33.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.411+0000 7fe6310af700 1 --2- 172.21.15.16:0/234015755 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe62c09f650 0x7fe62c131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:33.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.411+0000 7fe6310af700 1 --2- 172.21.15.16:0/234015755 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe62c09e930 0x7fe62c09ed00 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:33.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.411+0000 7fe6310af700 1 -- 172.21.15.16:0/234015755 >> 172.21.15.16:0/234015755 conn(0x7fe62c0a4210 msgr2=0x7fe62c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:33.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.411+0000 7fe6310af700 1 -- 172.21.15.16:0/234015755 shutdown_connections 2023-09-30T22:36:33.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.411+0000 7fe6310af700 1 -- 172.21.15.16:0/234015755 wait complete. 2023-09-30T22:36:33.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.413+0000 7fe6310af700 1 Processor -- start 2023-09-30T22:36:33.414 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.413+0000 7fe6310af700 1 -- start start 2023-09-30T22:36:33.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.413+0000 7fe6310af700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe62c09e930 0x7fe62c1a0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:33.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.413+0000 7fe6310af700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe62c09f240 0x7fe62c1a14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:33.415 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.413+0000 7fe6310af700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe62c09f650 0x7fe62c1c3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:33.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.413+0000 7fe6310af700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe62c1a0b70 con 0x7fe62c09f240 2023-09-30T22:36:33.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.413+0000 7fe6310af700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe62c1b5260 con 0x7fe62c09f650 2023-09-30T22:36:33.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.413+0000 7fe6310af700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe62c1b5020 con 0x7fe62c09e930 2023-09-30T22:36:33.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.414+0000 7fe62a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe62c09f240 0x7fe62c1a14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:33.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.414+0000 7fe62ad9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe62c09e930 0x7fe62c1a0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:33.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.414+0000 7fe62a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe62c09f240 0x7fe62c1a14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51436/0 (socket says 172.21.15.16:51436) 2023-09-30T22:36:33.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.414+0000 7fe62a59c700 1 -- 172.21.15.16:0/3101564063 learned_addr learned my addr 172.21.15.16:0/3101564063 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:33.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.414+0000 7fe62b59e700 1 --2- 172.21.15.16:0/3101564063 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe62c09f650 0x7fe62c1c3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:33.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.414+0000 7fe62a59c700 1 -- 172.21.15.16:0/3101564063 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe62c09e930 msgr2=0x7fe62c1a0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:33.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.414+0000 7fe62a59c700 1 --2- 172.21.15.16:0/3101564063 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe62c09e930 0x7fe62c1a0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:33.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.414+0000 7fe62a59c700 1 -- 172.21.15.16:0/3101564063 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe62c09f650 msgr2=0x7fe62c1c3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:33.416 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.414+0000 7fe62a59c700 1 --2- 172.21.15.16:0/3101564063 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe62c09f650 0x7fe62c1c3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:33.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.414+0000 7fe62a59c700 1 -- 172.21.15.16:0/3101564063 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe61c0094a0 con 0x7fe62c09f240 2023-09-30T22:36:33.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.414+0000 7fe62a59c700 1 --2- 172.21.15.16:0/3101564063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe62c09f240 0x7fe62c1a14f0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fe62000c730 tx=0x7fe62000ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:33.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.415+0000 7fe6137fe700 1 -- 172.21.15.16:0/3101564063 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe620005d70 con 0x7fe62c09f240 2023-09-30T22:36:33.417 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.415+0000 7fe6137fe700 1 -- 172.21.15.16:0/3101564063 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6200070d0 con 0x7fe62c09f240 2023-09-30T22:36:33.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.415+0000 7fe6137fe700 1 -- 172.21.15.16:0/3101564063 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe620007440 con 0x7fe62c09f240 2023-09-30T22:36:33.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.415+0000 7fe6310af700 1 -- 172.21.15.16:0/3101564063 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe62c0a0190 con 0x7fe62c09f240 2023-09-30T22:36:33.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.415+0000 7fe6310af700 1 -- 172.21.15.16:0/3101564063 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe62c0a2f40 con 0x7fe62c09f240 2023-09-30T22:36:33.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.416+0000 7fe6310af700 1 -- 172.21.15.16:0/3101564063 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe62c1a2850 con 0x7fe62c09f240 2023-09-30T22:36:33.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.417+0000 7fe6137fe700 1 -- 172.21.15.16:0/3101564063 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe6200075a0 con 0x7fe62c09f240 2023-09-30T22:36:33.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.417+0000 7fe6137fe700 1 -- 172.21.15.16:0/3101564063 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fe62003f2c0 con 0x7fe62c09f240 2023-09-30T22:36:33.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.422+0000 7fe6137fe700 1 -- 172.21.15.16:0/3101564063 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe620023b70 con 0x7fe62c09f240 2023-09-30T22:36:33.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.446+0000 7f48ad6a8700 1 Processor -- start 2023-09-30T22:36:33.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.446+0000 7f3927dd6700 1 Processor -- start 2023-09-30T22:36:33.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.446+0000 7f48ad6a8700 1 -- start start 2023-09-30T22:36:33.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.447+0000 7f3927dd6700 1 -- start start 2023-09-30T22:36:33.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.447+0000 7f48ad6a8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f48a80a0a30 0x7f48a80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:33.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.447+0000 7f48ad6a8700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f48a809d630 0x7f48a80a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:33.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.447+0000 7f48ad6a8700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48a80a2a70 con 0x7f48a8133930 2023-09-30T22:36:33.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.447+0000 7f48ad6a8700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48a80a01d0 con 0x7f48a809d630 2023-09-30T22:36:33.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.447+0000 7f48ad6a8700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48a809e2b0 con 0x7f48a80a0a30 2023-09-30T22:36:33.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.447+0000 7f48a6ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f48a80a0a30 0x7f48a80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:33.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.447+0000 7f48a67fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f48a809d630 0x7f48a80a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:33.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.447+0000 7f48a6ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f48a80a0a30 0x7f48a80a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36300/0 (socket says 172.21.15.16:36300) 2023-09-30T22:36:33.450 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.447+0000 7f48a6ffd700 1 -- 172.21.15.16:0/995471583 learned_addr learned my addr 172.21.15.16:0/995471583 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:33.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.447+0000 7f3927dd6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f39200a0a30 0x7f39200a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:33.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.447+0000 7f3927dd6700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f392009d630 0x7f39200a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:33.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.447+0000 7f3927dd6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3920133930 0x7f3920133dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:33.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.447+0000 7f3927dd6700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39200a2a70 con 0x7f392009d630 2023-09-30T22:36:33.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.447+0000 7f3927dd6700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39200a01d0 con 0x7f39200a0a30 2023-09-30T22:36:33.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.447+0000 7f3927dd6700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f392009e2b0 con 0x7f3920133930 2023-09-30T22:36:33.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.448+0000 7f3926373700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3920133930 0x7f3920133dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:33.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.448+0000 7f3926373700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3920133930 0x7f3920133dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:36582/0 (socket says 172.21.15.16:36582) 2023-09-30T22:36:33.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.448+0000 7f3926373700 1 -- 172.21.15.16:0/2937636970 learned_addr learned my addr 172.21.15.16:0/2937636970 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:33.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.448+0000 7f3925371700 1 --2- 172.21.15.16:0/2937636970 >> v2:172.21.15.16:3300/0 conn(0x7f392009d630 0x7f39200a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:33.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.448+0000 7f3925b72700 1 --2- 172.21.15.16:0/2937636970 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f39200a0a30 0x7f39200a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:33.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.448+0000 7f48a67fc700 1 -- 172.21.15.16:0/995471583 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f48a80a0a30 msgr2=0x7f48a80a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:33.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.448+0000 7f48a67fc700 1 --2- 172.21.15.16:0/995471583 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f48a80a0a30 0x7f48a80a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:33.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.448+0000 7f48a67fc700 1 -- 172.21.15.16:0/995471583 >> v1:172.21.15.16:6789/0 conn(0x7f48a8133930 legacy=0x7f48a8133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:33.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.448+0000 7f48a67fc700 1 -- 172.21.15.16:0/995471583 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a809dc50 con 0x7f48a809d630 2023-09-30T22:36:33.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.448+0000 7f3926373700 1 -- 172.21.15.16:0/2937636970 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f39200a0a30 msgr2=0x7f39200a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:33.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.448+0000 7f3926373700 1 --2- 172.21.15.16:0/2937636970 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f39200a0a30 0x7f39200a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:33.453 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.448+0000 7f3926373700 1 -- 172.21.15.16:0/2937636970 >> v2:172.21.15.16:3300/0 conn(0x7f392009d630 msgr2=0x7f39200a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:33.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.448+0000 7f3926373700 1 --2- 172.21.15.16:0/2937636970 >> v2:172.21.15.16:3300/0 conn(0x7f392009d630 0x7f39200a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:33.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.448+0000 7f3926373700 1 -- 172.21.15.16:0/2937636970 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f392009dc50 con 0x7f3920133930 2023-09-30T22:36:33.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.448+0000 7f3926373700 1 --2- 172.21.15.16:0/2937636970 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3920133930 0x7f3920133dd0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f3918009e50 tx=0x7f391800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b180aae126a54b4d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:33.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.448+0000 7f48a67fc700 1 --2- 172.21.15.16:0/995471583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48a809d630 0x7f48a80a1340 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f489c009840 tx=0x7f489c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20ceb902446e2587 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:33.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.449+0000 7f3924b70700 1 -- 172.21.15.16:0/2937636970 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391800e070 con 0x7f3920133930 2023-09-30T22:36:33.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.449+0000 7f3924b70700 1 -- 172.21.15.16:0/2937636970 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f391800b9c0 con 0x7f3920133930 2023-09-30T22:36:33.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.449+0000 7f3924b70700 1 -- 172.21.15.16:0/2937636970 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391800bd70 con 0x7f3920133930 2023-09-30T22:36:33.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.449+0000 7f3927dd6700 1 -- 172.21.15.16:0/2937636970 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3920133930 msgr2=0x7f3920133dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:33.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.449+0000 7f3927dd6700 1 --2- 172.21.15.16:0/2937636970 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3920133930 0x7f3920133dd0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f3918009e50 tx=0x7f391800b330 comp rx=0 tx=0).stop 2023-09-30T22:36:33.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.449+0000 7f3927dd6700 1 -- 172.21.15.16:0/2937636970 shutdown_connections 2023-09-30T22:36:33.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.449+0000 7f3927dd6700 1 --2- 172.21.15.16:0/2937636970 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f39200a0a30 0x7f39200a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:33.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.449+0000 7f3927dd6700 1 --2- 172.21.15.16:0/2937636970 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3920133930 0x7f3920133dd0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:33.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.449+0000 7f3927dd6700 1 --2- 172.21.15.16:0/2937636970 >> v2:172.21.15.16:3300/0 conn(0x7f392009d630 0x7f39200a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:33.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.449+0000 7f3927dd6700 1 -- 172.21.15.16:0/2937636970 >> 172.21.15.16:0/2937636970 conn(0x7f39200a3050 msgr2=0x7f39200a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:33.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.450+0000 7f48a5ffb700 1 -- 172.21.15.16:0/995471583 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489c011830 con 0x7f48a809d630 2023-09-30T22:36:33.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.450+0000 7f48a5ffb700 1 -- 172.21.15.16:0/995471583 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f489c011990 con 0x7f48a809d630 2023-09-30T22:36:33.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.450+0000 7f48a5ffb700 1 -- 172.21.15.16:0/995471583 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489c011d20 con 0x7f48a809d630 2023-09-30T22:36:33.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.450+0000 7f3927dd6700 1 -- 172.21.15.16:0/2937636970 shutdown_connections 2023-09-30T22:36:33.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.450+0000 7f3927dd6700 1 -- 172.21.15.16:0/2937636970 wait complete. 2023-09-30T22:36:33.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.450+0000 7f48ad6a8700 1 -- 172.21.15.16:0/995471583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48a809d630 msgr2=0x7f48a80a1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:33.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.450+0000 7f48ad6a8700 1 --2- 172.21.15.16:0/995471583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48a809d630 0x7f48a80a1340 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f489c009840 tx=0x7f489c009b50 comp rx=0 tx=0).stop 2023-09-30T22:36:33.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.451+0000 7f48ad6a8700 1 -- 172.21.15.16:0/995471583 shutdown_connections 2023-09-30T22:36:33.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.451+0000 7f48ad6a8700 1 --2- 172.21.15.16:0/995471583 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f48a80a0a30 0x7f48a80a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:33.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.451+0000 7f48ad6a8700 1 --2- 172.21.15.16:0/995471583 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48a809d630 0x7f48a80a1340 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:33.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.451+0000 7f48ad6a8700 1 -- 172.21.15.16:0/995471583 >> 172.21.15.16:0/995471583 conn(0x7f48a80a3050 msgr2=0x7f48a80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:33.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.451+0000 7f48ad6a8700 1 -- 172.21.15.16:0/995471583 shutdown_connections 2023-09-30T22:36:33.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.451+0000 7f48ad6a8700 1 -- 172.21.15.16:0/995471583 wait complete. 2023-09-30T22:36:33.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.451+0000 7f3927dd6700 1 Processor -- start 2023-09-30T22:36:33.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.451+0000 7f3927dd6700 1 -- start start 2023-09-30T22:36:33.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.451+0000 7f3927dd6700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f39200a0a30 0x7f392011f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:33.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.451+0000 7f3927dd6700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3920133930 0x7f392011f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:33.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.451+0000 7f3927dd6700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39201c8100 0x7f39201c8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:33.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.451+0000 7f3927dd6700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3920133330 con 0x7f39201c8100 2023-09-30T22:36:33.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.451+0000 7f3927dd6700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39200a0410 con 0x7f3920133930 2023-09-30T22:36:33.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.451+0000 7f3927dd6700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f39201225c0 con 0x7f39200a0a30 2023-09-30T22:36:33.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.452+0000 7f3925b72700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f39200a0a30 0x7f392011f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:33.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.452+0000 7f3925b72700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f39200a0a30 0x7f392011f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:36588/0 (socket says 172.21.15.16:36588) 2023-09-30T22:36:33.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.452+0000 7f3925b72700 1 -- 172.21.15.16:0/4026126667 learned_addr learned my addr 172.21.15.16:0/4026126667 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:33.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.452+0000 7f3926373700 1 --2- 172.21.15.16:0/4026126667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39201c8100 0x7f39201c8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:33.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.452+0000 7f3925371700 1 --2- 172.21.15.16:0/4026126667 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3920133930 0x7f392011f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:33.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.452+0000 7f3926373700 1 -- 172.21.15.16:0/4026126667 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f39200a0a30 msgr2=0x7f392011f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:33.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.452+0000 7f3926373700 1 --2- 172.21.15.16:0/4026126667 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f39200a0a30 0x7f392011f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:33.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.452+0000 7f3926373700 1 -- 172.21.15.16:0/4026126667 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3920133930 msgr2=0x7f392011f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:33.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.452+0000 7f3926373700 1 --2- 172.21.15.16:0/4026126667 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3920133930 0x7f392011f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:33.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.452+0000 7f3926373700 1 -- 172.21.15.16:0/4026126667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3918009b30 con 0x7f39201c8100 2023-09-30T22:36:33.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.452+0000 7f3925b72700 1 --2- 172.21.15.16:0/4026126667 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f39200a0a30 0x7f392011f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:33.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.452+0000 7f48ad6a8700 1 Processor -- start 2023-09-30T22:36:33.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.452+0000 7f3926373700 1 --2- 172.21.15.16:0/4026126667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39201c8100 0x7f39201c8570 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f3918015000 tx=0x7f3918004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:33.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.453+0000 7f39167fc700 1 -- 172.21.15.16:0/4026126667 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3918004450 con 0x7f39201c8100 2023-09-30T22:36:33.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.453+0000 7f39167fc700 1 -- 172.21.15.16:0/4026126667 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39180092e0 con 0x7f39201c8100 2023-09-30T22:36:33.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.453+0000 7f39167fc700 1 -- 172.21.15.16:0/4026126667 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3918004830 con 0x7f39201c8100 2023-09-30T22:36:33.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.453+0000 7f48ad6a8700 1 -- start start 2023-09-30T22:36:33.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.453+0000 7f48ad6a8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f48a809d630 0x7f48a8129cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:33.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.453+0000 7f48ad6a8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f48a80a0a30 0x7f48a8126020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:33.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.453+0000 7f48ad6a8700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48a8133930 0x7f48a8126560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:33.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.453+0000 7f48ad6a8700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a812a3a0 con 0x7f48a8133930 2023-09-30T22:36:33.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.453+0000 7f48ad6a8700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48a80a0410 con 0x7f48a80a0a30 2023-09-30T22:36:33.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.453+0000 7f48ad6a8700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f48a8129220 con 0x7f48a809d630 2023-09-30T22:36:33.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.454+0000 7f48a77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48a8133930 0x7f48a8126560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:33.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.454+0000 7f48a77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48a8133930 0x7f48a8126560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51456/0 (socket says 172.21.15.16:51456) 2023-09-30T22:36:33.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.454+0000 7f48a77fe700 1 -- 172.21.15.16:0/4122046571 learned_addr learned my addr 172.21.15.16:0/4122046571 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:33.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.454+0000 7f48a67fc700 1 --2- 172.21.15.16:0/4122046571 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f48a80a0a30 0x7f48a8126020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:33.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.454+0000 7f48a6ffd700 1 --2- 172.21.15.16:0/4122046571 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f48a809d630 0x7f48a8129cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:33.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.453+0000 7f3927dd6700 1 -- 172.21.15.16:0/4026126667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39200a41e0 con 0x7f39201c8100 2023-09-30T22:36:33.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.453+0000 7f3927dd6700 1 -- 172.21.15.16:0/4026126667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39201205b0 con 0x7f39201c8100 2023-09-30T22:36:33.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.454+0000 7f3927dd6700 1 -- 172.21.15.16:0/4026126667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3920122400 con 0x7f39201c8100 2023-09-30T22:36:33.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.454+0000 7f48a67fc700 1 -- 172.21.15.16:0/4122046571 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f48a809d630 msgr2=0x7f48a8129cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:33.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.454+0000 7f48a67fc700 1 --2- 172.21.15.16:0/4122046571 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f48a809d630 0x7f48a8129cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:33.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.454+0000 7f48a67fc700 1 -- 172.21.15.16:0/4122046571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48a8133930 msgr2=0x7f48a8126560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:33.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.454+0000 7f48a67fc700 1 --2- 172.21.15.16:0/4122046571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48a8133930 0x7f48a8126560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:33.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.454+0000 7f48a67fc700 1 -- 172.21.15.16:0/4122046571 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f489c0094a0 con 0x7f48a80a0a30 2023-09-30T22:36:33.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.454+0000 7f48a77fe700 1 --2- 172.21.15.16:0/4122046571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48a8133930 0x7f48a8126560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:33.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.454+0000 7f48a67fc700 1 --2- 172.21.15.16:0/4122046571 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f48a80a0a30 0x7f48a8126020 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f489c016000 tx=0x7f489c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:33.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.455+0000 7f488f7fe700 1 -- 172.21.15.16:0/4122046571 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489c004480 con 0x7f48a80a0a30 2023-09-30T22:36:33.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.455+0000 7f48ad6a8700 1 -- 172.21.15.16:0/4122046571 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48a81294e0 con 0x7f48a80a0a30 2023-09-30T22:36:33.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.455+0000 7f48ad6a8700 1 -- 172.21.15.16:0/4122046571 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48a8126aa0 con 0x7f48a80a0a30 2023-09-30T22:36:33.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.455+0000 7f39167fc700 1 -- 172.21.15.16:0/4026126667 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f391801c410 con 0x7f39201c8100 2023-09-30T22:36:33.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.455+0000 7f488f7fe700 1 -- 172.21.15.16:0/4122046571 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f489c01f070 con 0x7f48a80a0a30 2023-09-30T22:36:33.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.455+0000 7f488f7fe700 1 -- 172.21.15.16:0/4122046571 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489c0045e0 con 0x7f48a80a0a30 2023-09-30T22:36:33.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.455+0000 7f39167fc700 1 -- 172.21.15.16:0/4026126667 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f3918040f90 con 0x7f39201c8100 2023-09-30T22:36:33.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.456+0000 7f488f7fe700 1 -- 172.21.15.16:0/4122046571 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f489c021030 con 0x7f48a80a0a30 2023-09-30T22:36:33.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.456+0000 7f488f7fe700 1 -- 172.21.15.16:0/4122046571 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f489c0407d0 con 0x7f48a80a0a30 2023-09-30T22:36:33.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.460+0000 7f39167fc700 1 -- 172.21.15.16:0/4026126667 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f391801a3c0 con 0x7f39201c8100 2023-09-30T22:36:33.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.461+0000 7f48ad6a8700 1 -- 172.21.15.16:0/4122046571 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48880052e0 con 0x7f48a80a0a30 2023-09-30T22:36:33.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.466+0000 7f488f7fe700 1 -- 172.21.15.16:0/4122046571 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f489c015080 con 0x7f48a80a0a30 2023-09-30T22:36:33.520 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.518+0000 7fd731fa2700 1 Processor -- start 2023-09-30T22:36:33.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.518+0000 7fd731fa2700 1 -- start start 2023-09-30T22:36:33.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.519+0000 7fd731fa2700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd72c09d800 0x7fd72c09dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:33.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.519+0000 7fd731fa2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd72c09e110 0x7fd72c133860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:33.521 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.519+0000 7fd731fa2700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd72c099e10 con 0x7fd72c133da0 2023-09-30T22:36:33.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.519+0000 7fd731fa2700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd72c09f560 con 0x7fd72c09d800 2023-09-30T22:36:33.522 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.519+0000 7fd731fa2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd72c06b4d0 con 0x7fd72c09e110 2023-09-30T22:36:33.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.520+0000 7fd7317a1700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd72c133da0 0x7fd72c13aab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:47984/0 (socket says 172.21.15.16:47984) 2023-09-30T22:36:33.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.520+0000 7fd7317a1700 1 -- 172.21.15.16:0/2704189348 learned_addr learned my addr 172.21.15.16:0/2704189348 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:36:33.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.520+0000 7fd72bfff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd72c09e110 0x7fd72c133860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:33.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.520+0000 7fd730fa0700 1 --2- 172.21.15.16:0/2704189348 >> v2:172.21.15.16:3300/0 conn(0x7fd72c09d800 0x7fd72c09dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:33.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.520+0000 7fd72bfff700 1 -- 172.21.15.16:0/2704189348 >> v2:172.21.15.16:3300/0 conn(0x7fd72c09d800 msgr2=0x7fd72c09dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:33.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.520+0000 7fd72bfff700 1 --2- 172.21.15.16:0/2704189348 >> v2:172.21.15.16:3300/0 conn(0x7fd72c09d800 0x7fd72c09dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:33.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.520+0000 7fd72bfff700 1 -- 172.21.15.16:0/2704189348 >> v1:172.21.15.16:6789/0 conn(0x7fd72c133da0 legacy=0x7fd72c13aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:33.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.520+0000 7fd72bfff700 1 -- 172.21.15.16:0/2704189348 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd72c09a060 con 0x7fd72c09e110 2023-09-30T22:36:33.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.520+0000 7fd72bfff700 1 --2- 172.21.15.16:0/2704189348 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd72c09e110 0x7fd72c133860 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fd718009820 tx=0x7fd718009b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8c7178c8a9c5e1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:33.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.520+0000 7fd72b7fe700 1 -- 172.21.15.16:0/2704189348 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd718005e30 con 0x7fd72c09e110 2023-09-30T22:36:33.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.521+0000 7fd72b7fe700 1 -- 172.21.15.16:0/2704189348 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd718006030 con 0x7fd72c09e110 2023-09-30T22:36:33.525 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.521+0000 7fd72b7fe700 1 -- 172.21.15.16:0/2704189348 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7180063c0 con 0x7fd72c09e110 2023-09-30T22:36:33.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.522+0000 7fd731fa2700 1 -- 172.21.15.16:0/2704189348 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd72c09e110 msgr2=0x7fd72c133860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:33.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.522+0000 7fd731fa2700 1 --2- 172.21.15.16:0/2704189348 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd72c09e110 0x7fd72c133860 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fd718009820 tx=0x7fd718009b30 comp rx=0 tx=0).stop 2023-09-30T22:36:33.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.522+0000 7fd731fa2700 1 -- 172.21.15.16:0/2704189348 shutdown_connections 2023-09-30T22:36:33.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.522+0000 7fd731fa2700 1 --2- 172.21.15.16:0/2704189348 >> v2:172.21.15.16:3300/0 conn(0x7fd72c09d800 0x7fd72c09dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:33.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.522+0000 7fd731fa2700 1 --2- 172.21.15.16:0/2704189348 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd72c09e110 0x7fd72c133860 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:33.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.522+0000 7fd731fa2700 1 -- 172.21.15.16:0/2704189348 >> 172.21.15.16:0/2704189348 conn(0x7fd72c0a2e20 msgr2=0x7fd72c0a3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:33.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.522+0000 7fd731fa2700 1 -- 172.21.15.16:0/2704189348 shutdown_connections 2023-09-30T22:36:33.526 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.522+0000 7fd731fa2700 1 -- 172.21.15.16:0/2704189348 wait complete. 2023-09-30T22:36:33.529 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.523+0000 7fd731fa2700 1 Processor -- start 2023-09-30T22:36:33.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.523+0000 7fd731fa2700 1 -- start start 2023-09-30T22:36:33.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.524+0000 7fd731fa2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd72c09d800 0x7fd72c122770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:33.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.524+0000 7fd731fa2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd72c09e110 0x7fd72c122cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:33.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.524+0000 7fd731fa2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd72c133da0 0x7fd72c1231f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:33.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.524+0000 7fd731fa2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd72c133240 con 0x7fd72c133da0 2023-09-30T22:36:33.530 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.524+0000 7fd731fa2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd72c123c20 con 0x7fd72c09d800 2023-09-30T22:36:33.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.524+0000 7fd731fa2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd72c125f60 con 0x7fd72c09e110 2023-09-30T22:36:33.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.525+0000 7fd7317a1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd72c133da0 0x7fd72c1231f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:33.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.525+0000 7fd72bfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd72c09e110 0x7fd72c122cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:33.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.525+0000 7fd7317a1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd72c133da0 0x7fd72c1231f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51468/0 (socket says 172.21.15.16:51468) 2023-09-30T22:36:33.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.525+0000 7fd7317a1700 1 -- 172.21.15.16:0/2765748478 learned_addr learned my addr 172.21.15.16:0/2765748478 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:33.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.525+0000 7fd730fa0700 1 --2- 172.21.15.16:0/2765748478 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd72c09d800 0x7fd72c122770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:33.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.525+0000 7fd72bfff700 1 -- 172.21.15.16:0/2765748478 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd72c09d800 msgr2=0x7fd72c122770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:33.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.525+0000 7fd72bfff700 1 --2- 172.21.15.16:0/2765748478 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd72c09d800 0x7fd72c122770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:33.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.525+0000 7fd72bfff700 1 -- 172.21.15.16:0/2765748478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd72c133da0 msgr2=0x7fd72c1231f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:33.531 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.525+0000 7fd72bfff700 1 --2- 172.21.15.16:0/2765748478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd72c133da0 0x7fd72c1231f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:33.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.525+0000 7fd72bfff700 1 -- 172.21.15.16:0/2765748478 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd718009430 con 0x7fd72c09e110 2023-09-30T22:36:33.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.526+0000 7fd72bfff700 1 --2- 172.21.15.16:0/2765748478 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd72c09e110 0x7fd72c122cb0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fd72c06ae00 tx=0x7fd718004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:33.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.526+0000 7fd7297fa700 1 -- 172.21.15.16:0/2765748478 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd718004240 con 0x7fd72c09e110 2023-09-30T22:36:33.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.526+0000 7fd7297fa700 1 -- 172.21.15.16:0/2765748478 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7180043a0 con 0x7fd72c09e110 2023-09-30T22:36:33.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.526+0000 7fd7297fa700 1 -- 172.21.15.16:0/2765748478 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7180036a0 con 0x7fd72c09e110 2023-09-30T22:36:33.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.527+0000 7fd731fa2700 1 -- 172.21.15.16:0/2765748478 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd72c124440 con 0x7fd72c09e110 2023-09-30T22:36:33.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.527+0000 7fd731fa2700 1 -- 172.21.15.16:0/2765748478 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd72c125ca0 con 0x7fd72c09e110 2023-09-30T22:36:33.532 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.528+0000 7fd731fa2700 1 -- 172.21.15.16:0/2765748478 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd72c04f2a0 con 0x7fd72c09e110 2023-09-30T22:36:33.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.529+0000 7fd7297fa700 1 -- 172.21.15.16:0/2765748478 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd718022020 con 0x7fd72c09e110 2023-09-30T22:36:33.533 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.530+0000 7fd7297fa700 1 -- 172.21.15.16:0/2765748478 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd718040b20 con 0x7fd72c09e110 2023-09-30T22:36:33.535 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.534+0000 7fd7297fa700 1 -- 172.21.15.16:0/2765748478 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd7180199a0 con 0x7fd72c09e110 2023-09-30T22:36:33.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.673+0000 7f3927dd6700 1 -- 172.21.15.16:0/4026126667 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3920122400 con 0x7f39201c8100 2023-09-30T22:36:33.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.674+0000 7f39167fc700 1 -- 172.21.15.16:0/4026126667 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3920122400 con 0x7f39201c8100 2023-09-30T22:36:33.675 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:36:33.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.676+0000 7f3927dd6700 1 -- 172.21.15.16:0/4026126667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39201c8100 msgr2=0x7f39201c8570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:33.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.676+0000 7f3927dd6700 1 --2- 172.21.15.16:0/4026126667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39201c8100 0x7f39201c8570 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f3918015000 tx=0x7f3918004ad0 comp rx=0 tx=0).stop 2023-09-30T22:36:33.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.676+0000 7f3927dd6700 1 -- 172.21.15.16:0/4026126667 shutdown_connections 2023-09-30T22:36:33.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.676+0000 7f3927dd6700 1 --2- 172.21.15.16:0/4026126667 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39201c8100 0x7f39201c8570 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:33.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.676+0000 7f3927dd6700 1 --2- 172.21.15.16:0/4026126667 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f39200a0a30 0x7f392011f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:33.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.676+0000 7f3927dd6700 1 --2- 172.21.15.16:0/4026126667 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3920133930 0x7f392011f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:33.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.676+0000 7f3927dd6700 1 -- 172.21.15.16:0/4026126667 >> 172.21.15.16:0/4026126667 conn(0x7f39200a3050 msgr2=0x7f392009ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:33.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.676+0000 7f3927dd6700 1 -- 172.21.15.16:0/4026126667 shutdown_connections 2023-09-30T22:36:33.678 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.676+0000 7f3927dd6700 1 -- 172.21.15.16:0/4026126667 wait complete. 2023-09-30T22:36:33.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.677+0000 7fe6310af700 1 -- 172.21.15.16:0/3101564063 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe62c09ffe0 con 0x7fe62c09f240 2023-09-30T22:36:33.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.677+0000 7fe6137fe700 1 -- 172.21.15.16:0/3101564063 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe620017020 con 0x7fe62c09f240 2023-09-30T22:36:33.679 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:36:33.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.680+0000 7fe6310af700 1 -- 172.21.15.16:0/3101564063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe62c09f240 msgr2=0x7fe62c1a14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:33.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.680+0000 7fe6310af700 1 --2- 172.21.15.16:0/3101564063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe62c09f240 0x7fe62c1a14f0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fe62000c730 tx=0x7fe62000ca40 comp rx=0 tx=0).stop 2023-09-30T22:36:33.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.680+0000 7fe6310af700 1 -- 172.21.15.16:0/3101564063 shutdown_connections 2023-09-30T22:36:33.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.680+0000 7fe6310af700 1 --2- 172.21.15.16:0/3101564063 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe62c09f650 0x7fe62c1c3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:33.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.680+0000 7fe6310af700 1 --2- 172.21.15.16:0/3101564063 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe62c09e930 0x7fe62c1a0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:33.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.680+0000 7fe6310af700 1 --2- 172.21.15.16:0/3101564063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe62c09f240 0x7fe62c1a14f0 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:33.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.680+0000 7fe6310af700 1 -- 172.21.15.16:0/3101564063 >> 172.21.15.16:0/3101564063 conn(0x7fe62c0a4210 msgr2=0x7fe62c0a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:33.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.680+0000 7f48ad6a8700 1 -- 172.21.15.16:0/4122046571 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4888005150 con 0x7f48a80a0a30 2023-09-30T22:36:33.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.680+0000 7fe6310af700 1 -- 172.21.15.16:0/3101564063 shutdown_connections 2023-09-30T22:36:33.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.680+0000 7fe6310af700 1 -- 172.21.15.16:0/3101564063 wait complete. 2023-09-30T22:36:33.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.680+0000 7f488f7fe700 1 -- 172.21.15.16:0/4122046571 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f489c019770 con 0x7f48a80a0a30 2023-09-30T22:36:33.683 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:36:33.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.684+0000 7f48ad6a8700 1 -- 172.21.15.16:0/4122046571 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f48a80a0a30 msgr2=0x7f48a8126020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:33.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.684+0000 7f48ad6a8700 1 --2- 172.21.15.16:0/4122046571 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f48a80a0a30 0x7f48a8126020 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f489c016000 tx=0x7f489c004750 comp rx=0 tx=0).stop 2023-09-30T22:36:33.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.686+0000 7f48ad6a8700 1 -- 172.21.15.16:0/4122046571 shutdown_connections 2023-09-30T22:36:33.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.686+0000 7f48ad6a8700 1 --2- 172.21.15.16:0/4122046571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48a8133930 0x7f48a8126560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:33.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.686+0000 7f48ad6a8700 1 --2- 172.21.15.16:0/4122046571 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f48a809d630 0x7f48a8129cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:33.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.686+0000 7f48ad6a8700 1 --2- 172.21.15.16:0/4122046571 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f48a80a0a30 0x7f48a8126020 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:33.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.686+0000 7f48ad6a8700 1 -- 172.21.15.16:0/4122046571 >> 172.21.15.16:0/4122046571 conn(0x7f48a80a3050 msgr2=0x7f48a809ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:33.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.686+0000 7f48ad6a8700 1 -- 172.21.15.16:0/4122046571 shutdown_connections 2023-09-30T22:36:33.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.686+0000 7f48ad6a8700 1 -- 172.21.15.16:0/4122046571 wait complete. 2023-09-30T22:36:33.695 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:36:33.698 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:36:33.708 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:36:33.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.754+0000 7fd731fa2700 1 -- 172.21.15.16:0/2765748478 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd72c06c0a0 con 0x7fd72c09e110 2023-09-30T22:36:33.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.755+0000 7fd7297fa700 1 -- 172.21.15.16:0/2765748478 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd71802a030 con 0x7fd72c09e110 2023-09-30T22:36:33.756 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:36:33.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.757+0000 7fd731fa2700 1 -- 172.21.15.16:0/2765748478 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd72c09e110 msgr2=0x7fd72c122cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:33.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.757+0000 7fd731fa2700 1 --2- 172.21.15.16:0/2765748478 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd72c09e110 0x7fd72c122cb0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fd72c06ae00 tx=0x7fd718004750 comp rx=0 tx=0).stop 2023-09-30T22:36:33.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.757+0000 7fd731fa2700 1 -- 172.21.15.16:0/2765748478 shutdown_connections 2023-09-30T22:36:33.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.757+0000 7fd731fa2700 1 --2- 172.21.15.16:0/2765748478 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd72c09d800 0x7fd72c122770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:33.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.757+0000 7fd731fa2700 1 --2- 172.21.15.16:0/2765748478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd72c133da0 0x7fd72c1231f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:33.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.757+0000 7fd731fa2700 1 --2- 172.21.15.16:0/2765748478 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd72c09e110 0x7fd72c122cb0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:33.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.757+0000 7fd731fa2700 1 -- 172.21.15.16:0/2765748478 >> 172.21.15.16:0/2765748478 conn(0x7fd72c0a2e20 msgr2=0x7fd72c070b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:33.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.757+0000 7fd731fa2700 1 -- 172.21.15.16:0/2765748478 shutdown_connections 2023-09-30T22:36:33.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:33.757+0000 7fd731fa2700 1 -- 172.21.15.16:0/2765748478 wait complete. 2023-09-30T22:36:33.772 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:36:34.696 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:36:34.699 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:36:34.709 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:36:34.772 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:36:34.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.850+0000 7f88d2da2700 1 Processor -- start 2023-09-30T22:36:34.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.850+0000 7f195c7d7700 1 Processor -- start 2023-09-30T22:36:34.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.850+0000 7f195c7d7700 1 -- start start 2023-09-30T22:36:34.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.851+0000 7f88d2da2700 1 -- start start 2023-09-30T22:36:34.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.851+0000 7f88d2da2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f88cc09d800 0x7f88cc09dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:34.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.851+0000 7f88d2da2700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f88cc09e110 0x7f88cc133860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:34.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.851+0000 7f88d2da2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f88cc133da0 0x7f88cc134240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:34.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.851+0000 7f88d2da2700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88cc09f560 con 0x7f88cc09e110 2023-09-30T22:36:34.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.851+0000 7f88d2da2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88cc099e10 con 0x7f88cc133da0 2023-09-30T22:36:34.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.851+0000 7f88d2da2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f88cc06b4d0 con 0x7f88cc09d800 2023-09-30T22:36:34.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.851+0000 7f88d159f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f88cc09e110 0x7f88cc133860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:34.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.851+0000 7f88d159f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f88cc09e110 0x7f88cc133860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51478/0 (socket says 172.21.15.16:51478) 2023-09-30T22:36:34.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.851+0000 7f88d159f700 1 -- 172.21.15.16:0/1395560880 learned_addr learned my addr 172.21.15.16:0/1395560880 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:34.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.851+0000 7f88d1da0700 1 --2- 172.21.15.16:0/1395560880 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f88cc09d800 0x7f88cc09dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:34.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.851+0000 7f195c7d7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1954136830 0x7f1954138c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:34.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.851+0000 7f195c7d7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1954139150 0x7f195413b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:34.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.852+0000 7f195c7d7700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f195406f720 con 0x7f19541338c0 2023-09-30T22:36:34.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.852+0000 7f195c7d7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1954070c80 con 0x7f1954139150 2023-09-30T22:36:34.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.852+0000 7f195c7d7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f195406fce0 con 0x7f1954136830 2023-09-30T22:36:34.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.852+0000 7f1959d72700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1954136830 0x7f1954138c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:34.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.852+0000 7f1959d72700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1954136830 0x7f1954138c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:36618/0 (socket says 172.21.15.16:36618) 2023-09-30T22:36:34.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.852+0000 7f195ad74700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1954139150 0x7f195413b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:34.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.852+0000 7f1959d72700 1 -- 172.21.15.16:0/1640236076 learned_addr learned my addr 172.21.15.16:0/1640236076 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:34.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.852+0000 7f88d25a1700 1 --2- 172.21.15.16:0/1395560880 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f88cc133da0 0x7f88cc134240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:34.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.852+0000 7f88d159f700 1 -- 172.21.15.16:0/1395560880 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f88cc09d800 msgr2=0x7f88cc09dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:34.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.852+0000 7f88d159f700 1 --2- 172.21.15.16:0/1395560880 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f88cc09d800 0x7f88cc09dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:34.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.852+0000 7f88d159f700 1 -- 172.21.15.16:0/1395560880 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f88cc133da0 msgr2=0x7f88cc134240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:34.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.852+0000 7f88d159f700 1 --2- 172.21.15.16:0/1395560880 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f88cc133da0 0x7f88cc134240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:34.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.852+0000 7f88d159f700 1 -- 172.21.15.16:0/1395560880 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88cc09a060 con 0x7f88cc09e110 2023-09-30T22:36:34.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.852+0000 7f88d25a1700 1 --2- 172.21.15.16:0/1395560880 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f88cc133da0 0x7f88cc134240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:36:34.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.852+0000 7f195ad74700 1 -- 172.21.15.16:0/1640236076 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1954136830 msgr2=0x7f1954138c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:34.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.852+0000 7f195ad74700 1 --2- 172.21.15.16:0/1640236076 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1954136830 0x7f1954138c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:34.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.852+0000 7f195ad74700 1 -- 172.21.15.16:0/1640236076 >> v1:172.21.15.16:6789/0 conn(0x7f19541338c0 legacy=0x7f1954133c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:34.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.852+0000 7f195ad74700 1 -- 172.21.15.16:0/1640236076 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19541336a0 con 0x7f1954139150 2023-09-30T22:36:34.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.852+0000 7f88d159f700 1 --2- 172.21.15.16:0/1395560880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88cc09e110 0x7f88cc133860 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f88bc006fd0 tx=0x7f88bc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43a44a411f7e05c5 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:34.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.852+0000 7f88d0d9e700 1 -- 172.21.15.16:0/1395560880 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88bc005e30 con 0x7f88cc09e110 2023-09-30T22:36:34.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.853+0000 7f88d0d9e700 1 -- 172.21.15.16:0/1395560880 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88bc004030 con 0x7f88cc09e110 2023-09-30T22:36:34.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.853+0000 7f88d0d9e700 1 -- 172.21.15.16:0/1395560880 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88bc0043e0 con 0x7f88cc09e110 2023-09-30T22:36:34.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.853+0000 7f195ad74700 1 --2- 172.21.15.16:0/1640236076 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1954139150 0x7f195413b530 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f194c009dc0 tx=0x7f194c00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e504b3c99b7c072 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:34.859 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.853+0000 7f88d2da2700 1 -- 172.21.15.16:0/1395560880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88cc09e110 msgr2=0x7f88cc133860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:34.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.853+0000 7f88d2da2700 1 --2- 172.21.15.16:0/1395560880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88cc09e110 0x7f88cc133860 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f88bc006fd0 tx=0x7f88bc009bd0 comp rx=0 tx=0).stop 2023-09-30T22:36:34.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.853+0000 7f88d2da2700 1 -- 172.21.15.16:0/1395560880 shutdown_connections 2023-09-30T22:36:34.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.853+0000 7f88d2da2700 1 --2- 172.21.15.16:0/1395560880 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f88cc133da0 0x7f88cc134240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:34.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.853+0000 7f88d2da2700 1 --2- 172.21.15.16:0/1395560880 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f88cc09d800 0x7f88cc09dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:34.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.853+0000 7f88d2da2700 1 --2- 172.21.15.16:0/1395560880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88cc09e110 0x7f88cc133860 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:34.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.853+0000 7f88d2da2700 1 -- 172.21.15.16:0/1395560880 >> 172.21.15.16:0/1395560880 conn(0x7f88cc0a2e20 msgr2=0x7f88cc0a3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:34.860 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.853+0000 7f1959571700 1 -- 172.21.15.16:0/1640236076 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194c00e070 con 0x7f1954139150 2023-09-30T22:36:34.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.854+0000 7f1959571700 1 -- 172.21.15.16:0/1640236076 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f194c00bb20 con 0x7f1954139150 2023-09-30T22:36:34.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.854+0000 7f1959571700 1 -- 172.21.15.16:0/1640236076 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194c012430 con 0x7f1954139150 2023-09-30T22:36:34.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.854+0000 7f88d2da2700 1 -- 172.21.15.16:0/1395560880 shutdown_connections 2023-09-30T22:36:34.861 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.854+0000 7f88d2da2700 1 -- 172.21.15.16:0/1395560880 wait complete. 2023-09-30T22:36:34.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.854+0000 7f195c7d7700 1 -- 172.21.15.16:0/1640236076 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1954139150 msgr2=0x7f195413b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:34.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.854+0000 7f195c7d7700 1 --2- 172.21.15.16:0/1640236076 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1954139150 0x7f195413b530 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f194c009dc0 tx=0x7f194c00b330 comp rx=0 tx=0).stop 2023-09-30T22:36:34.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.854+0000 7f195c7d7700 1 -- 172.21.15.16:0/1640236076 shutdown_connections 2023-09-30T22:36:34.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.854+0000 7f195c7d7700 1 --2- 172.21.15.16:0/1640236076 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1954139150 0x7f195413b530 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:34.862 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.854+0000 7f195c7d7700 1 --2- 172.21.15.16:0/1640236076 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1954136830 0x7f1954138c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:34.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.854+0000 7f195c7d7700 1 -- 172.21.15.16:0/1640236076 >> 172.21.15.16:0/1640236076 conn(0x7f19540a2aa0 msgr2=0x7f19540a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:34.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.855+0000 7f195c7d7700 1 -- 172.21.15.16:0/1640236076 shutdown_connections 2023-09-30T22:36:34.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.855+0000 7f195c7d7700 1 -- 172.21.15.16:0/1640236076 wait complete. 2023-09-30T22:36:34.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.855+0000 7f88d2da2700 1 Processor -- start 2023-09-30T22:36:34.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.855+0000 7f88d2da2700 1 -- start start 2023-09-30T22:36:34.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.856+0000 7f88d2da2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88cc09d800 0x7f88cc122160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:34.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.856+0000 7f88d2da2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f88cc09e110 0x7f88cc1226a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:34.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.856+0000 7f88d2da2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f88cc133da0 0x7f88cc122be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:34.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.856+0000 7f88d2da2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88cc124f70 con 0x7f88cc09d800 2023-09-30T22:36:34.864 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.856+0000 7f88d2da2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88cc123ea0 con 0x7f88cc133da0 2023-09-30T22:36:34.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.856+0000 7f88d2da2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f88cc124bb0 con 0x7f88cc09e110 2023-09-30T22:36:34.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.856+0000 7f88d1da0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88cc09d800 0x7f88cc122160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:34.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.856+0000 7f88d1da0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88cc09d800 0x7f88cc122160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51494/0 (socket says 172.21.15.16:51494) 2023-09-30T22:36:34.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.856+0000 7f88d1da0700 1 -- 172.21.15.16:0/3906553825 learned_addr learned my addr 172.21.15.16:0/3906553825 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:34.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.856+0000 7f88d25a1700 1 --2- 172.21.15.16:0/3906553825 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f88cc133da0 0x7f88cc122be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:34.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.856+0000 7f195c7d7700 1 Processor -- start 2023-09-30T22:36:34.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.857+0000 7f88d1da0700 1 -- 172.21.15.16:0/3906553825 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f88cc09e110 msgr2=0x7f88cc1226a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:36:34.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.857+0000 7f88d1da0700 1 --2- 172.21.15.16:0/3906553825 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f88cc09e110 0x7f88cc1226a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:34.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.857+0000 7f88d1da0700 1 -- 172.21.15.16:0/3906553825 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f88cc133da0 msgr2=0x7f88cc122be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:34.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.857+0000 7f88d1da0700 1 --2- 172.21.15.16:0/3906553825 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f88cc133da0 0x7f88cc122be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:34.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.857+0000 7f88d1da0700 1 -- 172.21.15.16:0/3906553825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88bc009500 con 0x7f88cc09d800 2023-09-30T22:36:34.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.857+0000 7f195c7d7700 1 -- start start 2023-09-30T22:36:34.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.857+0000 7f88d1da0700 1 --2- 172.21.15.16:0/3906553825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88cc09d800 0x7f88cc122160 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f88c800e810 tx=0x7f88c800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:34.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.857+0000 7f88c27fc700 1 -- 172.21.15.16:0/3906553825 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c8013610 con 0x7f88cc09d800 2023-09-30T22:36:34.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.857+0000 7f88c27fc700 1 -- 172.21.15.16:0/3906553825 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88c8013790 con 0x7f88cc09d800 2023-09-30T22:36:34.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.857+0000 7f88d2da2700 1 -- 172.21.15.16:0/3906553825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88cc125c90 con 0x7f88cc09d800 2023-09-30T22:36:34.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.857+0000 7f88c27fc700 1 -- 172.21.15.16:0/3906553825 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c8013b40 con 0x7f88cc09d800 2023-09-30T22:36:34.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.857+0000 7f88d2da2700 1 -- 172.21.15.16:0/3906553825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88cc125950 con 0x7f88cc09d800 2023-09-30T22:36:34.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.857+0000 7f195c7d7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f19541338c0 0x7f1954099ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:34.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.857+0000 7f195c7d7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1954136830 0x7f195409a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:34.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.857+0000 7f195c7d7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1954139150 0x7f19541cc600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:34.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.857+0000 7f195c7d7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f195409cff0 con 0x7f1954136830 2023-09-30T22:36:34.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.857+0000 7f195c7d7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f195409b720 con 0x7f19541338c0 2023-09-30T22:36:34.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.857+0000 7f195c7d7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f195412fd70 con 0x7f1954139150 2023-09-30T22:36:34.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.857+0000 7f1959d72700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1954136830 0x7f195409a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:34.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.857+0000 7f1959d72700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1954136830 0x7f195409a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51496/0 (socket says 172.21.15.16:51496) 2023-09-30T22:36:34.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.858+0000 7f1959d72700 1 -- 172.21.15.16:0/978298017 learned_addr learned my addr 172.21.15.16:0/978298017 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:34.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.858+0000 7f195ad74700 1 --2- 172.21.15.16:0/978298017 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1954139150 0x7f19541cc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:34.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.858+0000 7f195ad74700 1 -- 172.21.15.16:0/978298017 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f19541338c0 msgr2=0x7f1954099ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:34.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.858+0000 7f195ad74700 1 --2- 172.21.15.16:0/978298017 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f19541338c0 0x7f1954099ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:34.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.858+0000 7f195ad74700 1 -- 172.21.15.16:0/978298017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1954136830 msgr2=0x7f195409a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:34.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.858+0000 7f195ad74700 1 --2- 172.21.15.16:0/978298017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1954136830 0x7f195409a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:34.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.858+0000 7f195ad74700 1 -- 172.21.15.16:0/978298017 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f194c009a70 con 0x7f1954139150 2023-09-30T22:36:34.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.858+0000 7f195ad74700 1 --2- 172.21.15.16:0/978298017 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1954139150 0x7f19541cc600 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f194c016000 tx=0x7f194c004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:34.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.859+0000 7f1946ffd700 1 -- 172.21.15.16:0/978298017 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194c0048b0 con 0x7f1954139150 2023-09-30T22:36:34.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.859+0000 7f1946ffd700 1 -- 172.21.15.16:0/978298017 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f194c020070 con 0x7f1954139150 2023-09-30T22:36:34.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.859+0000 7f1946ffd700 1 -- 172.21.15.16:0/978298017 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194c0276c0 con 0x7f1954139150 2023-09-30T22:36:34.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.859+0000 7f195c7d7700 1 -- 172.21.15.16:0/978298017 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f195409c570 con 0x7f1954139150 2023-09-30T22:36:34.870 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.859+0000 7f195c7d7700 1 -- 172.21.15.16:0/978298017 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1954122640 con 0x7f1954139150 2023-09-30T22:36:34.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.858+0000 7f88d2da2700 1 -- 172.21.15.16:0/3906553825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88cc11eea0 con 0x7f88cc09d800 2023-09-30T22:36:34.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.860+0000 7f1946ffd700 1 -- 172.21.15.16:0/978298017 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f194c012430 con 0x7f1954139150 2023-09-30T22:36:34.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.860+0000 7f1946ffd700 1 -- 172.21.15.16:0/978298017 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f194c022070 con 0x7f1954139150 2023-09-30T22:36:34.871 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.860+0000 7f195c7d7700 1 -- 172.21.15.16:0/978298017 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1954074ea0 con 0x7f1954139150 2023-09-30T22:36:34.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.860+0000 7f88c27fc700 1 -- 172.21.15.16:0/3906553825 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f88c8013ca0 con 0x7f88cc09d800 2023-09-30T22:36:34.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.861+0000 7f88c27fc700 1 -- 172.21.15.16:0/3906553825 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f88c803f880 con 0x7f88cc09d800 2023-09-30T22:36:34.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.865+0000 7f88c27fc700 1 -- 172.21.15.16:0/3906553825 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f88c8009440 con 0x7f88cc09d800 2023-09-30T22:36:34.872 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.870+0000 7f1946ffd700 1 -- 172.21.15.16:0/978298017 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f194c01c400 con 0x7f1954139150 2023-09-30T22:36:34.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.896+0000 7f0bc2ee7700 1 Processor -- start 2023-09-30T22:36:34.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.896+0000 7f0bc2ee7700 1 -- start start 2023-09-30T22:36:34.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.896+0000 7f0bc2ee7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0bbc130540 0x7f0bbc130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:34.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.896+0000 7f0bc2ee7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0bbc130e50 0x7f0bbc06f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:34.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.896+0000 7f0bc2ee7700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f0bbc06fb20 0x7f0bbc06ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:34.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.896+0000 7f0bc2ee7700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bbc09a390 con 0x7f0bbc06fb20 2023-09-30T22:36:34.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.896+0000 7f0bc2ee7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bbc0716a0 con 0x7f0bbc130540 2023-09-30T22:36:34.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.896+0000 7f0bc2ee7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0bbc071290 con 0x7f0bbc130e50 2023-09-30T22:36:34.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.897+0000 7f0bbbfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0bbc130e50 0x7f0bbc06f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:34.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.897+0000 7f0bbbfff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0bbc130e50 0x7f0bbc06f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:36634/0 (socket says 172.21.15.16:36634) 2023-09-30T22:36:34.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.897+0000 7f0bbbfff700 1 -- 172.21.15.16:0/4116476373 learned_addr learned my addr 172.21.15.16:0/4116476373 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:34.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.897+0000 7f0bc0c83700 1 --2- 172.21.15.16:0/4116476373 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0bbc130540 0x7f0bbc130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:34.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.897+0000 7f0bbbfff700 1 -- 172.21.15.16:0/4116476373 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0bbc130540 msgr2=0x7f0bbc130910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:34.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.897+0000 7f0bbbfff700 1 --2- 172.21.15.16:0/4116476373 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0bbc130540 0x7f0bbc130910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:34.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.897+0000 7f0bbbfff700 1 -- 172.21.15.16:0/4116476373 >> v2:172.21.15.16:3300/0 conn(0x7f0bbc06fb20 msgr2=0x7f0bbc06ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:34.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.897+0000 7f0bbbfff700 1 --2- 172.21.15.16:0/4116476373 >> v2:172.21.15.16:3300/0 conn(0x7f0bbc06fb20 0x7f0bbc06ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:34.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.897+0000 7f0bbbfff700 1 -- 172.21.15.16:0/4116476373 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bbc130340 con 0x7f0bbc130e50 2023-09-30T22:36:34.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.897+0000 7f0bbbfff700 1 --2- 172.21.15.16:0/4116476373 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0bbc130e50 0x7f0bbc06f5e0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f0ba800bbc0 tx=0x7f0ba800bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3d42887cc14b916 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:34.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.897+0000 7f0bbb7fe700 1 -- 172.21.15.16:0/4116476373 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba800dcf0 con 0x7f0bbc130e50 2023-09-30T22:36:34.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.898+0000 7f0bbb7fe700 1 -- 172.21.15.16:0/4116476373 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ba800de50 con 0x7f0bbc130e50 2023-09-30T22:36:34.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.898+0000 7f0bbb7fe700 1 -- 172.21.15.16:0/4116476373 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba8010690 con 0x7f0bbc130e50 2023-09-30T22:36:34.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.898+0000 7f0bc2ee7700 1 -- 172.21.15.16:0/4116476373 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0bbc130e50 msgr2=0x7f0bbc06f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:34.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.898+0000 7f0bc2ee7700 1 --2- 172.21.15.16:0/4116476373 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0bbc130e50 0x7f0bbc06f5e0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f0ba800bbc0 tx=0x7f0ba800bed0 comp rx=0 tx=0).stop 2023-09-30T22:36:34.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.898+0000 7f0bc2ee7700 1 -- 172.21.15.16:0/4116476373 shutdown_connections 2023-09-30T22:36:34.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.898+0000 7f0bc2ee7700 1 --2- 172.21.15.16:0/4116476373 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0bbc130540 0x7f0bbc130910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:34.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.898+0000 7f0bc2ee7700 1 --2- 172.21.15.16:0/4116476373 >> v2:172.21.15.16:3300/0 conn(0x7f0bbc06fb20 0x7f0bbc06ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:34.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.898+0000 7f0bc2ee7700 1 --2- 172.21.15.16:0/4116476373 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0bbc130e50 0x7f0bbc06f5e0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:34.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.898+0000 7f0bc2ee7700 1 -- 172.21.15.16:0/4116476373 >> 172.21.15.16:0/4116476373 conn(0x7f0bbc0a2aa0 msgr2=0x7f0bbc0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:34.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.898+0000 7f0bc2ee7700 1 -- 172.21.15.16:0/4116476373 shutdown_connections 2023-09-30T22:36:34.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.898+0000 7f0bc2ee7700 1 -- 172.21.15.16:0/4116476373 wait complete. 2023-09-30T22:36:34.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.900+0000 7f0bc2ee7700 1 Processor -- start 2023-09-30T22:36:34.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.900+0000 7f0bc2ee7700 1 -- start start 2023-09-30T22:36:34.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.901+0000 7f0bc2ee7700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0bbc06fb20 0x7f0bbc1c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:34.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.901+0000 7f0bc2ee7700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bbc130540 0x7f0bbc1c8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:34.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.901+0000 7f0bc2ee7700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0bbc130e50 0x7f0bbc11fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:34.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.901+0000 7f0bc2ee7700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bbc121a60 con 0x7f0bbc130540 2023-09-30T22:36:34.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.901+0000 7f0bc2ee7700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bbc1237f0 con 0x7f0bbc06fb20 2023-09-30T22:36:34.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.901+0000 7f0bc2ee7700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f0bbc072660 con 0x7f0bbc130e50 2023-09-30T22:36:34.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.901+0000 7f0bc1484700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0bbc130e50 0x7f0bbc11fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:34.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.901+0000 7f0bc1484700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0bbc130e50 0x7f0bbc11fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:36650/0 (socket says 172.21.15.16:36650) 2023-09-30T22:36:34.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.901+0000 7f0bc1484700 1 -- 172.21.15.16:0/1803176570 learned_addr learned my addr 172.21.15.16:0/1803176570 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:34.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.901+0000 7f0bc0c83700 1 --2- 172.21.15.16:0/1803176570 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0bbc06fb20 0x7f0bbc1c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:34.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.901+0000 7f0bbbfff700 1 --2- 172.21.15.16:0/1803176570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bbc130540 0x7f0bbc1c8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:34.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.901+0000 7f0bc1484700 1 -- 172.21.15.16:0/1803176570 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0bbc06fb20 msgr2=0x7f0bbc1c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:34.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.901+0000 7f0bc1484700 1 --2- 172.21.15.16:0/1803176570 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0bbc06fb20 0x7f0bbc1c86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:34.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.901+0000 7f0bc1484700 1 -- 172.21.15.16:0/1803176570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bbc130540 msgr2=0x7f0bbc1c8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:34.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.901+0000 7f0bc1484700 1 --2- 172.21.15.16:0/1803176570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bbc130540 0x7f0bbc1c8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:34.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.901+0000 7f0bc1484700 1 -- 172.21.15.16:0/1803176570 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ba800b7f0 con 0x7f0bbc130e50 2023-09-30T22:36:34.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.901+0000 7f0bbbfff700 1 --2- 172.21.15.16:0/1803176570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bbc130540 0x7f0bbc1c8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:36:34.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.902+0000 7f0bc1484700 1 --2- 172.21.15.16:0/1803176570 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0bbc130e50 0x7f0bbc11fcd0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f0bb400b970 tx=0x7f0bb400bd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:34.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.902+0000 7f0bb97fa700 1 -- 172.21.15.16:0/1803176570 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb400c9b0 con 0x7f0bbc130e50 2023-09-30T22:36:34.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.902+0000 7f0bb97fa700 1 -- 172.21.15.16:0/1803176570 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bb400cb30 con 0x7f0bbc130e50 2023-09-30T22:36:34.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.902+0000 7f0bc2ee7700 1 -- 172.21.15.16:0/1803176570 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bbc0718e0 con 0x7f0bbc130e50 2023-09-30T22:36:34.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.902+0000 7f0bb97fa700 1 -- 172.21.15.16:0/1803176570 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb4014440 con 0x7f0bbc130e50 2023-09-30T22:36:34.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.902+0000 7f0bc2ee7700 1 -- 172.21.15.16:0/1803176570 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bbc123510 con 0x7f0bbc130e50 2023-09-30T22:36:34.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.903+0000 7f0bb97fa700 1 -- 172.21.15.16:0/1803176570 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f0bb401d3b0 con 0x7f0bbc130e50 2023-09-30T22:36:34.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.903+0000 7f0bb97fa700 1 -- 172.21.15.16:0/1803176570 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f0bb4006e40 con 0x7f0bbc130e50 2023-09-30T22:36:34.907 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.903+0000 7f0bc2ee7700 1 -- 172.21.15.16:0/1803176570 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bbc074ea0 con 0x7f0bbc130e50 2023-09-30T22:36:34.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.908+0000 7f0bb97fa700 1 -- 172.21.15.16:0/1803176570 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f0bb4019070 con 0x7f0bbc130e50 2023-09-30T22:36:34.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.935+0000 7f2cdf823700 1 Processor -- start 2023-09-30T22:36:34.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.936+0000 7f2cdf823700 1 -- start start 2023-09-30T22:36:34.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.936+0000 7f2cdf823700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2cd81316c0 0x7f2cd8131a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:34.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.937+0000 7f2cdf823700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f2cd806ae80 0x7f2cd8139550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:34.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.937+0000 7f2cdf823700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cd809e130 con 0x7f2cd809d9f0 2023-09-30T22:36:34.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.937+0000 7f2cdf823700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cd81312b0 con 0x7f2cd806ae80 2023-09-30T22:36:34.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.937+0000 7f2cdf823700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd809f6e0 con 0x7f2cd81316c0 2023-09-30T22:36:34.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.937+0000 7f2cdcdbe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f2cd809d9f0 0x7f2cd806a770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48006/0 (socket says 172.21.15.16:48006) 2023-09-30T22:36:34.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.937+0000 7f2cdd5bf700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2cd81316c0 0x7f2cd8131a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:34.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.937+0000 7f2cdcdbe700 1 -- 172.21.15.16:0/1086184706 learned_addr learned my addr 172.21.15.16:0/1086184706 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:36:34.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.937+0000 7f2cdddc0700 1 --2- 172.21.15.16:0/1086184706 >> v2:172.21.15.16:3300/0 conn(0x7f2cd806ae80 0x7f2cd8139550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:34.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.937+0000 7f2cdd5bf700 1 -- 172.21.15.16:0/1086184706 >> v2:172.21.15.16:3300/0 conn(0x7f2cd806ae80 msgr2=0x7f2cd8139550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:34.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.937+0000 7f2cdd5bf700 1 --2- 172.21.15.16:0/1086184706 >> v2:172.21.15.16:3300/0 conn(0x7f2cd806ae80 0x7f2cd8139550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:34.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.937+0000 7f2cdd5bf700 1 -- 172.21.15.16:0/1086184706 >> v1:172.21.15.16:6789/0 conn(0x7f2cd809d9f0 legacy=0x7f2cd806a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:34.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.937+0000 7f2cdd5bf700 1 -- 172.21.15.16:0/1086184706 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd809e760 con 0x7f2cd81316c0 2023-09-30T22:36:34.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.937+0000 7f2cdd5bf700 1 --2- 172.21.15.16:0/1086184706 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2cd81316c0 0x7f2cd8131a90 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4009840 tx=0x7f2cd4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d8fd2c3646ec00d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:34.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.938+0000 7f2ccffff700 1 -- 172.21.15.16:0/1086184706 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd4011830 con 0x7f2cd81316c0 2023-09-30T22:36:34.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.938+0000 7f2ccffff700 1 -- 172.21.15.16:0/1086184706 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cd4011990 con 0x7f2cd81316c0 2023-09-30T22:36:34.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.938+0000 7f2ccffff700 1 -- 172.21.15.16:0/1086184706 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd4011d20 con 0x7f2cd81316c0 2023-09-30T22:36:34.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.939+0000 7f2cdf823700 1 -- 172.21.15.16:0/1086184706 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2cd81316c0 msgr2=0x7f2cd8131a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:34.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.939+0000 7f2cdf823700 1 --2- 172.21.15.16:0/1086184706 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2cd81316c0 0x7f2cd8131a90 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4009840 tx=0x7f2cd4009b50 comp rx=0 tx=0).stop 2023-09-30T22:36:34.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.939+0000 7f2cdf823700 1 -- 172.21.15.16:0/1086184706 shutdown_connections 2023-09-30T22:36:34.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.939+0000 7f2cdf823700 1 --2- 172.21.15.16:0/1086184706 >> v2:172.21.15.16:3300/0 conn(0x7f2cd806ae80 0x7f2cd8139550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:34.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.939+0000 7f2cdf823700 1 --2- 172.21.15.16:0/1086184706 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2cd81316c0 0x7f2cd8131a90 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:34.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.939+0000 7f2cdf823700 1 -- 172.21.15.16:0/1086184706 >> 172.21.15.16:0/1086184706 conn(0x7f2cd8066f50 msgr2=0x7f2cd809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:34.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.939+0000 7f2cdf823700 1 -- 172.21.15.16:0/1086184706 shutdown_connections 2023-09-30T22:36:34.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.939+0000 7f2cdf823700 1 -- 172.21.15.16:0/1086184706 wait complete. 2023-09-30T22:36:34.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.940+0000 7f2cdf823700 1 Processor -- start 2023-09-30T22:36:34.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.940+0000 7f2cdf823700 1 -- start start 2023-09-30T22:36:34.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.941+0000 7f2cdf823700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2cd806ae80 0x7f2cd8071350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:34.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.941+0000 7f2cdf823700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cd809d9f0 0x7f2cd8071890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:34.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.941+0000 7f2cdf823700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2cd81316c0 0x7f2cd8071dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:34.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.941+0000 7f2cdf823700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd8072820 con 0x7f2cd809d9f0 2023-09-30T22:36:34.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.941+0000 7f2cdf823700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd8134750 con 0x7f2cd81316c0 2023-09-30T22:36:34.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.941+0000 7f2cdf823700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd8133640 con 0x7f2cd806ae80 2023-09-30T22:36:34.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.941+0000 7f2cdddc0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2cd81316c0 0x7f2cd8071dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:34.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.941+0000 7f2cdd5bf700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2cd806ae80 0x7f2cd8071350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:34.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.941+0000 7f2cdddc0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2cd81316c0 0x7f2cd8071dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36422/0 (socket says 172.21.15.16:36422) 2023-09-30T22:36:34.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.941+0000 7f2cdddc0700 1 -- 172.21.15.16:0/2494019407 learned_addr learned my addr 172.21.15.16:0/2494019407 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:34.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.941+0000 7f2cdd5bf700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2cd806ae80 0x7f2cd8071350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:36662/0 (socket says 172.21.15.16:36662) 2023-09-30T22:36:34.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.941+0000 7f2cdcdbe700 1 --2- 172.21.15.16:0/2494019407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cd809d9f0 0x7f2cd8071890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:34.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.942+0000 7f2cdd5bf700 1 -- 172.21.15.16:0/2494019407 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2cd81316c0 msgr2=0x7f2cd8071dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:34.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.942+0000 7f2cdd5bf700 1 --2- 172.21.15.16:0/2494019407 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2cd81316c0 0x7f2cd8071dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:34.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.942+0000 7f2cdd5bf700 1 -- 172.21.15.16:0/2494019407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cd809d9f0 msgr2=0x7f2cd8071890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:34.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.942+0000 7f2cdd5bf700 1 --2- 172.21.15.16:0/2494019407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cd809d9f0 0x7f2cd8071890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:34.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.942+0000 7f2cdd5bf700 1 -- 172.21.15.16:0/2494019407 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd40094a0 con 0x7f2cd806ae80 2023-09-30T22:36:34.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.942+0000 7f2cdddc0700 1 --2- 172.21.15.16:0/2494019407 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2cd81316c0 0x7f2cd8071dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:34.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.942+0000 7f2cdcdbe700 1 --2- 172.21.15.16:0/2494019407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cd809d9f0 0x7f2cd8071890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:34.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.942+0000 7f2cdd5bf700 1 --2- 172.21.15.16:0/2494019407 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2cd806ae80 0x7f2cd8071350 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4016000 tx=0x7f2cd4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:34.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.942+0000 7f2ccdffb700 1 -- 172.21.15.16:0/2494019407 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd40036a0 con 0x7f2cd806ae80 2023-09-30T22:36:34.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.942+0000 7f2ccdffb700 1 -- 172.21.15.16:0/2494019407 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cd401f070 con 0x7f2cd806ae80 2023-09-30T22:36:34.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.942+0000 7f2ccdffb700 1 -- 172.21.15.16:0/2494019407 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd4003a60 con 0x7f2cd806ae80 2023-09-30T22:36:34.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.942+0000 7f2cdf823700 1 -- 172.21.15.16:0/2494019407 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cd81328e0 con 0x7f2cd806ae80 2023-09-30T22:36:34.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.943+0000 7f2cdf823700 1 -- 172.21.15.16:0/2494019407 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cd812b900 con 0x7f2cd806ae80 2023-09-30T22:36:34.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.944+0000 7f2ccdffb700 1 -- 172.21.15.16:0/2494019407 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2cd4003bc0 con 0x7f2cd806ae80 2023-09-30T22:36:34.946 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.944+0000 7f2cdf823700 1 -- 172.21.15.16:0/2494019407 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cd809e480 con 0x7f2cd806ae80 2023-09-30T22:36:34.950 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.949+0000 7f2ccdffb700 1 -- 172.21.15.16:0/2494019407 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f2cd4021070 con 0x7f2cd806ae80 2023-09-30T22:36:34.954 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:34.953+0000 7f2ccdffb700 1 -- 172.21.15.16:0/2494019407 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2cd40253e0 con 0x7f2cd806ae80 2023-09-30T22:36:35.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:35.078+0000 7f195c7d7700 1 -- 172.21.15.16:0/978298017 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f195406b9e0 con 0x7f1954139150 2023-09-30T22:36:35.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:35.078+0000 7f1946ffd700 1 -- 172.21.15.16:0/978298017 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f194c02c030 con 0x7f1954139150 2023-09-30T22:36:35.080 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:36:35.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:35.080+0000 7f88d2da2700 1 -- 172.21.15.16:0/3906553825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f88cc04f2a0 con 0x7f88cc09d800 2023-09-30T22:36:35.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:35.080+0000 7f88c27fc700 1 -- 172.21.15.16:0/3906553825 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f88c8021070 con 0x7f88cc09d800 2023-09-30T22:36:35.082 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:36:35.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:35.080+0000 7f195c7d7700 1 -- 172.21.15.16:0/978298017 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1954139150 msgr2=0x7f19541cc600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:35.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:35.080+0000 7f195c7d7700 1 --2- 172.21.15.16:0/978298017 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1954139150 0x7f19541cc600 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f194c016000 tx=0x7f194c004ad0 comp rx=0 tx=0).stop 2023-09-30T22:36:35.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:35.080+0000 7f195c7d7700 1 -- 172.21.15.16:0/978298017 shutdown_connections 2023-09-30T22:36:35.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:35.080+0000 7f195c7d7700 1 --2- 172.21.15.16:0/978298017 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f19541338c0 0x7f1954099ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:35.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:35.080+0000 7f195c7d7700 1 --2- 172.21.15.16:0/978298017 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1954139150 0x7f19541cc600 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:35.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:35.080+0000 7f195c7d7700 1 --2- 172.21.15.16:0/978298017 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1954136830 0x7f195409a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:35.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:35.080+0000 7f195c7d7700 1 -- 172.21.15.16:0/978298017 >> 172.21.15.16:0/978298017 conn(0x7f19540a2aa0 msgr2=0x7f1954134610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:35.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:35.081+0000 7f195c7d7700 1 -- 172.21.15.16:0/978298017 shutdown_connections 2023-09-30T22:36:35.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:35.081+0000 7f195c7d7700 1 -- 172.21.15.16:0/978298017 wait complete. 2023-09-30T22:36:35.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:35.082+0000 7f88d2da2700 1 -- 172.21.15.16:0/3906553825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88cc09d800 msgr2=0x7f88cc122160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:35.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:35.082+0000 7f88d2da2700 1 --2- 172.21.15.16:0/3906553825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88cc09d800 0x7f88cc122160 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f88c800e810 tx=0x7f88c800eb20 comp rx=0 tx=0).stop 2023-09-30T22:36:35.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:35.084+0000 7f88d2da2700 1 -- 172.21.15.16:0/3906553825 shutdown_connections 2023-09-30T22:36:35.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:35.084+0000 7f88d2da2700 1 --2- 172.21.15.16:0/3906553825 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f88cc133da0 0x7f88cc122be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:35.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:35.084+0000 7f88d2da2700 1 --2- 172.21.15.16:0/3906553825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88cc09d800 0x7f88cc122160 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:35.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:35.084+0000 7f88d2da2700 1 --2- 172.21.15.16:0/3906553825 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f88cc09e110 0x7f88cc1226a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:35.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:35.084+0000 7f88d2da2700 1 -- 172.21.15.16:0/3906553825 >> 172.21.15.16:0/3906553825 conn(0x7f88cc0a2e20 msgr2=0x7f88cc071100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:35.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:35.085+0000 7f88d2da2700 1 -- 172.21.15.16:0/3906553825 shutdown_connections 2023-09-30T22:36:35.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:35.088+0000 7f88d2da2700 1 -- 172.21.15.16:0/3906553825 wait complete. 2023-09-30T22:36:35.103 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:36:35.104 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:36:35.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:35.143+0000 7f0bc2ee7700 1 -- 172.21.15.16:0/1803176570 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f0bbc070b80 con 0x7f0bbc130e50 2023-09-30T22:36:35.144 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:35.143+0000 7f0bb97fa700 1 -- 172.21.15.16:0/1803176570 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0bb4007370 con 0x7f0bbc130e50 2023-09-30T22:36:35.145 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:36:35.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:35.145+0000 7f0bc2ee7700 1 -- 172.21.15.16:0/1803176570 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0bbc130e50 msgr2=0x7f0bbc11fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:35.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:35.145+0000 7f0bc2ee7700 1 --2- 172.21.15.16:0/1803176570 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0bbc130e50 0x7f0bbc11fcd0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f0bb400b970 tx=0x7f0bb400bd30 comp rx=0 tx=0).stop 2023-09-30T22:36:35.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:35.145+0000 7f0bc2ee7700 1 -- 172.21.15.16:0/1803176570 shutdown_connections 2023-09-30T22:36:35.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:35.145+0000 7f0bc2ee7700 1 --2- 172.21.15.16:0/1803176570 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f0bbc06fb20 0x7f0bbc1c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:35.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:35.145+0000 7f0bc2ee7700 1 --2- 172.21.15.16:0/1803176570 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f0bbc130e50 0x7f0bbc11fcd0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:35.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:35.145+0000 7f0bc2ee7700 1 --2- 172.21.15.16:0/1803176570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0bbc130540 0x7f0bbc1c8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:35.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:35.145+0000 7f0bc2ee7700 1 -- 172.21.15.16:0/1803176570 >> 172.21.15.16:0/1803176570 conn(0x7f0bbc0a2aa0 msgr2=0x7f0bbc12f950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:35.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:35.145+0000 7f0bc2ee7700 1 -- 172.21.15.16:0/1803176570 shutdown_connections 2023-09-30T22:36:35.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:35.145+0000 7f0bc2ee7700 1 -- 172.21.15.16:0/1803176570 wait complete. 2023-09-30T22:36:35.161 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:36:35.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:35.170+0000 7f2cdf823700 1 -- 172.21.15.16:0/2494019407 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f2cd8074ea0 con 0x7f2cd806ae80 2023-09-30T22:36:35.172 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:35.170+0000 7f2ccdffb700 1 -- 172.21.15.16:0/2494019407 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2cd402b020 con 0x7f2cd806ae80 2023-09-30T22:36:35.172 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:36:35.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:35.173+0000 7f2cdf823700 1 -- 172.21.15.16:0/2494019407 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2cd806ae80 msgr2=0x7f2cd8071350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:35.174 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:35.173+0000 7f2cdf823700 1 --2- 172.21.15.16:0/2494019407 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2cd806ae80 0x7f2cd8071350 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4016000 tx=0x7f2cd4004750 comp rx=0 tx=0).stop 2023-09-30T22:36:35.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:35.173+0000 7f2cdf823700 1 -- 172.21.15.16:0/2494019407 shutdown_connections 2023-09-30T22:36:35.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:35.173+0000 7f2cdf823700 1 --2- 172.21.15.16:0/2494019407 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2cd81316c0 0x7f2cd8071dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:35.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:35.173+0000 7f2cdf823700 1 --2- 172.21.15.16:0/2494019407 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2cd806ae80 0x7f2cd8071350 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:35.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:35.173+0000 7f2cdf823700 1 --2- 172.21.15.16:0/2494019407 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2cd809d9f0 0x7f2cd8071890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:35.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:35.173+0000 7f2cdf823700 1 -- 172.21.15.16:0/2494019407 >> 172.21.15.16:0/2494019407 conn(0x7f2cd8066f50 msgr2=0x7f2cd8130a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:35.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:35.173+0000 7f2cdf823700 1 -- 172.21.15.16:0/2494019407 shutdown_connections 2023-09-30T22:36:35.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:35.173+0000 7f2cdf823700 1 -- 172.21.15.16:0/2494019407 wait complete. 2023-09-30T22:36:35.187 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:36:36.104 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:36:36.105 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:36:36.162 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:36:36.188 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:36:36.255 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.254+0000 7f2433a57700 1 Processor -- start 2023-09-30T22:36:36.256 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.254+0000 7f2433a57700 1 -- start start 2023-09-30T22:36:36.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.255+0000 7f2433a57700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f242c0a0a30 0x7f242c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:36.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.255+0000 7f2433a57700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f242c0a1340 0x7f242c133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:36.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.255+0000 7f2433a57700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f242c09edd0 con 0x7f242c133fa0 2023-09-30T22:36:36.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.255+0000 7f2433a57700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f242c09e1f0 con 0x7f242c0a0a30 2023-09-30T22:36:36.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.255+0000 7f2433a57700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f242c09dc50 con 0x7f242c0a1340 2023-09-30T22:36:36.257 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.255+0000 7f2430ff2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f242c0a1340 0x7f242c133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:36.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.255+0000 7f2430ff2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f242c0a1340 0x7f242c133a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:36666/0 (socket says 172.21.15.16:36666) 2023-09-30T22:36:36.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.255+0000 7f2430ff2700 1 -- 172.21.15.16:0/747117795 learned_addr learned my addr 172.21.15.16:0/747117795 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:36.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.255+0000 7f24317f3700 1 --2- 172.21.15.16:0/747117795 >> v2:172.21.15.16:3300/0 conn(0x7f242c0a0a30 0x7f242c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:36.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.255+0000 7f2430ff2700 1 -- 172.21.15.16:0/747117795 >> v2:172.21.15.16:3300/0 conn(0x7f242c0a0a30 msgr2=0x7f242c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:36.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.255+0000 7f2430ff2700 1 --2- 172.21.15.16:0/747117795 >> v2:172.21.15.16:3300/0 conn(0x7f242c0a0a30 0x7f242c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:36.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.255+0000 7f2430ff2700 1 -- 172.21.15.16:0/747117795 >> v1:172.21.15.16:6789/0 conn(0x7f242c133fa0 legacy=0x7f242c13ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:36.258 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.256+0000 7f2430ff2700 1 -- 172.21.15.16:0/747117795 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f242c0a1fe0 con 0x7f242c0a1340 2023-09-30T22:36:36.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.256+0000 7f24317f3700 1 --2- 172.21.15.16:0/747117795 >> v2:172.21.15.16:3300/0 conn(0x7f242c0a0a30 0x7f242c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:36:36.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.256+0000 7ff72f50d700 1 Processor -- start 2023-09-30T22:36:36.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.256+0000 7f2430ff2700 1 --2- 172.21.15.16:0/747117795 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f242c0a1340 0x7f242c133a60 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f241c0098f0 tx=0x7f241c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a0bfccbd1d3897b6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:36.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.256+0000 7f2423fff700 1 -- 172.21.15.16:0/747117795 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241c00b8a0 con 0x7f242c0a1340 2023-09-30T22:36:36.260 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.256+0000 7ff72f50d700 1 -- start start 2023-09-30T22:36:36.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.256+0000 7f2423fff700 1 -- 172.21.15.16:0/747117795 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f241c00ba00 con 0x7f242c0a1340 2023-09-30T22:36:36.261 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.256+0000 7f2423fff700 1 -- 172.21.15.16:0/747117795 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241c00bd90 con 0x7f242c0a1340 2023-09-30T22:36:36.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.257+0000 7ff72f50d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff7280a0a30 0x7ff7280a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:36.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.257+0000 7ff72f50d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff728134170 0x7ff72813bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:36.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.257+0000 7ff72f50d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff72809edd0 con 0x7ff7280a1340 2023-09-30T22:36:36.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.257+0000 7ff72f50d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff72809e1f0 con 0x7ff7280a0a30 2023-09-30T22:36:36.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.257+0000 7ff72f50d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff72809dc50 con 0x7ff728134170 2023-09-30T22:36:36.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.257+0000 7ff72daaa700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff728134170 0x7ff72813bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:36.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.257+0000 7ff72daaa700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff728134170 0x7ff72813bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:36680/0 (socket says 172.21.15.16:36680) 2023-09-30T22:36:36.262 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.257+0000 7ff72daaa700 1 -- 172.21.15.16:0/3165478212 learned_addr learned my addr 172.21.15.16:0/3165478212 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:36.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.257+0000 7f2433a57700 1 -- 172.21.15.16:0/747117795 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f242c0a1340 msgr2=0x7f242c133a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:36.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.257+0000 7f2433a57700 1 --2- 172.21.15.16:0/747117795 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f242c0a1340 0x7f242c133a60 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f241c0098f0 tx=0x7f241c009c00 comp rx=0 tx=0).stop 2023-09-30T22:36:36.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.257+0000 7f2433a57700 1 -- 172.21.15.16:0/747117795 shutdown_connections 2023-09-30T22:36:36.263 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.257+0000 7f2433a57700 1 --2- 172.21.15.16:0/747117795 >> v2:172.21.15.16:3300/0 conn(0x7f242c0a0a30 0x7f242c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:36.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.257+0000 7f2433a57700 1 --2- 172.21.15.16:0/747117795 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f242c0a1340 0x7f242c133a60 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:36.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.257+0000 7f2433a57700 1 -- 172.21.15.16:0/747117795 >> 172.21.15.16:0/747117795 conn(0x7f242c0a6550 msgr2=0x7f242c0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:36.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.257+0000 7ff72daaa700 1 -- 172.21.15.16:0/3165478212 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff7280a0a30 msgr2=0x7ff7280a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:36.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.257+0000 7ff72daaa700 1 --2- 172.21.15.16:0/3165478212 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff7280a0a30 0x7ff7280a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:36.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.257+0000 7ff72daaa700 1 -- 172.21.15.16:0/3165478212 >> v1:172.21.15.16:6789/0 conn(0x7ff7280a1340 legacy=0x7ff728133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:36.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.257+0000 7ff72daaa700 1 -- 172.21.15.16:0/3165478212 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7280a1fe0 con 0x7ff728134170 2023-09-30T22:36:36.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.257+0000 7f2433a57700 1 -- 172.21.15.16:0/747117795 shutdown_connections 2023-09-30T22:36:36.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.257+0000 7f2433a57700 1 -- 172.21.15.16:0/747117795 wait complete. 2023-09-30T22:36:36.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.257+0000 7ff72daaa700 1 --2- 172.21.15.16:0/3165478212 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff728134170 0x7ff72813bd30 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7ff720009e80 tx=0x7ff72000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=437401065c0fa985 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:36.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.258+0000 7ff71ffff700 1 -- 172.21.15.16:0/3165478212 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72000e070 con 0x7ff728134170 2023-09-30T22:36:36.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.258+0000 7ff71ffff700 1 -- 172.21.15.16:0/3165478212 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff72000b9c0 con 0x7ff728134170 2023-09-30T22:36:36.266 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.258+0000 7ff71ffff700 1 -- 172.21.15.16:0/3165478212 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72000bd70 con 0x7ff728134170 2023-09-30T22:36:36.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.258+0000 7ff72f50d700 1 -- 172.21.15.16:0/3165478212 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff728134170 msgr2=0x7ff72813bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:36.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.258+0000 7ff72f50d700 1 --2- 172.21.15.16:0/3165478212 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff728134170 0x7ff72813bd30 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7ff720009e80 tx=0x7ff72000b330 comp rx=0 tx=0).stop 2023-09-30T22:36:36.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.259+0000 7ff72f50d700 1 -- 172.21.15.16:0/3165478212 shutdown_connections 2023-09-30T22:36:36.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.259+0000 7ff72f50d700 1 --2- 172.21.15.16:0/3165478212 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff728134170 0x7ff72813bd30 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:36.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.259+0000 7ff72f50d700 1 --2- 172.21.15.16:0/3165478212 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff7280a0a30 0x7ff7280a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:36.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.259+0000 7ff72f50d700 1 -- 172.21.15.16:0/3165478212 >> 172.21.15.16:0/3165478212 conn(0x7ff7280a6550 msgr2=0x7ff7280a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:36.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.259+0000 7f2433a57700 1 Processor -- start 2023-09-30T22:36:36.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.259+0000 7ff72f50d700 1 -- 172.21.15.16:0/3165478212 shutdown_connections 2023-09-30T22:36:36.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.259+0000 7f2433a57700 1 -- start start 2023-09-30T22:36:36.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.259+0000 7ff72f50d700 1 -- 172.21.15.16:0/3165478212 wait complete. 2023-09-30T22:36:36.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.260+0000 7f2433a57700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f242c0a0a30 0x7f242c123330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:36.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.260+0000 7f2433a57700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f242c0a1340 0x7f242c123870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:36.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.260+0000 7f2433a57700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f242c133fa0 0x7f242c1c8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:36.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.260+0000 7f2433a57700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242c0a0290 con 0x7f242c133fa0 2023-09-30T22:36:36.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.260+0000 7f2433a57700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f242c122f50 con 0x7f242c0a0a30 2023-09-30T22:36:36.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.260+0000 7f2433a57700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f242c133410 con 0x7f242c0a1340 2023-09-30T22:36:36.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.260+0000 7f2430ff2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f242c0a1340 0x7f242c123870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:36.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.260+0000 7f2430ff2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f242c0a1340 0x7f242c123870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:36692/0 (socket says 172.21.15.16:36692) 2023-09-30T22:36:36.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.260+0000 7f2430ff2700 1 -- 172.21.15.16:0/924687507 learned_addr learned my addr 172.21.15.16:0/924687507 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:36.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.260+0000 7f24317f3700 1 --2- 172.21.15.16:0/924687507 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f242c0a0a30 0x7f242c123330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:36.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.260+0000 7f2430ff2700 1 -- 172.21.15.16:0/924687507 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f242c0a0a30 msgr2=0x7f242c123330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:36.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.260+0000 7f2430ff2700 1 --2- 172.21.15.16:0/924687507 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f242c0a0a30 0x7f242c123330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:36.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.260+0000 7f2430ff2700 1 -- 172.21.15.16:0/924687507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f242c133fa0 msgr2=0x7f242c1c8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:36.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.260+0000 7f2431ff4700 1 --2- 172.21.15.16:0/924687507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f242c133fa0 0x7f242c1c8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:36.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.260+0000 7f2430ff2700 1 --2- 172.21.15.16:0/924687507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f242c133fa0 0x7f242c1c8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:36.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.260+0000 7f2430ff2700 1 -- 172.21.15.16:0/924687507 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f241c009550 con 0x7f242c0a1340 2023-09-30T22:36:36.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.260+0000 7f2431ff4700 1 --2- 172.21.15.16:0/924687507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f242c133fa0 0x7f242c1c8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:36:36.272 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.261+0000 7f2430ff2700 1 --2- 172.21.15.16:0/924687507 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f242c0a1340 0x7f242c123870 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f241c000c00 tx=0x7f241c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:36.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.261+0000 7ff72f50d700 1 Processor -- start 2023-09-30T22:36:36.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.261+0000 7ff72f50d700 1 -- start start 2023-09-30T22:36:36.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.261+0000 7f2421ffb700 1 -- 172.21.15.16:0/924687507 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241c004590 con 0x7f242c0a1340 2023-09-30T22:36:36.273 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.261+0000 7f2421ffb700 1 -- 172.21.15.16:0/924687507 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f241c0036a0 con 0x7f242c0a1340 2023-09-30T22:36:36.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.261+0000 7f2433a57700 1 -- 172.21.15.16:0/924687507 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f242c123db0 con 0x7f242c0a1340 2023-09-30T22:36:36.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.261+0000 7f2421ffb700 1 -- 172.21.15.16:0/924687507 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241c003a50 con 0x7f242c0a1340 2023-09-30T22:36:36.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.261+0000 7f2433a57700 1 -- 172.21.15.16:0/924687507 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f242c126150 con 0x7f242c0a1340 2023-09-30T22:36:36.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.261+0000 7ff72f50d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff7280a0a30 0x7ff7281c8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:36.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.261+0000 7ff72f50d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7280a1340 0x7ff7281c8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:36.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.261+0000 7ff72f50d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff728134170 0x7ff7281c8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:36.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.261+0000 7ff72f50d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7281245d0 con 0x7ff7280a1340 2023-09-30T22:36:36.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.261+0000 7ff72f50d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff728125f40 con 0x7ff728134170 2023-09-30T22:36:36.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.261+0000 7ff72f50d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff7280a4f10 con 0x7ff7280a0a30 2023-09-30T22:36:36.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.262+0000 7ff72daaa700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff728134170 0x7ff7281c8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:36.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.262+0000 7ff72daaa700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff728134170 0x7ff7281c8ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:36454/0 (socket says 172.21.15.16:36454) 2023-09-30T22:36:36.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.262+0000 7ff72daaa700 1 -- 172.21.15.16:0/3653177181 learned_addr learned my addr 172.21.15.16:0/3653177181 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:36.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.262+0000 7ff72daaa700 1 -- 172.21.15.16:0/3653177181 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff7280a0a30 msgr2=0x7ff7281c8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:36.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.262+0000 7ff72daaa700 1 --2- 172.21.15.16:0/3653177181 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff7280a0a30 0x7ff7281c8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:36.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.262+0000 7ff72daaa700 1 -- 172.21.15.16:0/3653177181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7280a1340 msgr2=0x7ff7281c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:36.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.262+0000 7ff72daaa700 1 --2- 172.21.15.16:0/3653177181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7280a1340 0x7ff7281c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:36.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.262+0000 7ff72daaa700 1 -- 172.21.15.16:0/3653177181 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff720009b30 con 0x7ff728134170 2023-09-30T22:36:36.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.262+0000 7ff72daaa700 1 --2- 172.21.15.16:0/3653177181 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff728134170 0x7ff7281c8ea0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7ff72000b370 tx=0x7ff720004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:36.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.263+0000 7ff71dffb700 1 -- 172.21.15.16:0/3653177181 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff720004450 con 0x7ff728134170 2023-09-30T22:36:36.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.263+0000 7ff71dffb700 1 -- 172.21.15.16:0/3653177181 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7200045d0 con 0x7ff728134170 2023-09-30T22:36:36.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.263+0000 7ff71dffb700 1 -- 172.21.15.16:0/3653177181 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff720025460 con 0x7ff728134170 2023-09-30T22:36:36.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.263+0000 7ff72f50d700 1 -- 172.21.15.16:0/3653177181 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7280a25e0 con 0x7ff728134170 2023-09-30T22:36:36.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.263+0000 7f2421ffb700 1 -- 172.21.15.16:0/924687507 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f241c003bb0 con 0x7f242c0a1340 2023-09-30T22:36:36.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.263+0000 7ff72f50d700 1 -- 172.21.15.16:0/3653177181 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff728125240 con 0x7ff728134170 2023-09-30T22:36:36.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.263+0000 7f2421ffb700 1 -- 172.21.15.16:0/924687507 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f241c021030 con 0x7f242c0a1340 2023-09-30T22:36:36.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.264+0000 7ff71dffb700 1 -- 172.21.15.16:0/3653177181 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff72001c410 con 0x7ff728134170 2023-09-30T22:36:36.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.264+0000 7ff72f50d700 1 -- 172.21.15.16:0/3653177181 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff728074ea0 con 0x7ff728134170 2023-09-30T22:36:36.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.265+0000 7ff71dffb700 1 -- 172.21.15.16:0/3653177181 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7ff720041610 con 0x7ff728134170 2023-09-30T22:36:36.279 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.265+0000 7f2433a57700 1 -- 172.21.15.16:0/924687507 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f242c0a1d00 con 0x7f242c0a1340 2023-09-30T22:36:36.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.270+0000 7f2421ffb700 1 -- 172.21.15.16:0/924687507 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f241c01a430 con 0x7f242c0a1340 2023-09-30T22:36:36.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.270+0000 7ff71dffb700 1 -- 172.21.15.16:0/3653177181 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff720011390 con 0x7ff728134170 2023-09-30T22:36:36.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.317+0000 7ff0d560c700 1 Processor -- start 2023-09-30T22:36:36.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.317+0000 7ff0d560c700 1 -- start start 2023-09-30T22:36:36.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.318+0000 7ff0d560c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff0d009f410 0x7ff0d0136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:36.322 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.318+0000 7ff0d560c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff0d0136fe0 0x7ff0d01393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:36.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.318+0000 7ff0d560c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0d0070940 con 0x7ff0d009e930 2023-09-30T22:36:36.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.318+0000 7ff0d560c700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0d006fd60 con 0x7ff0d009f410 2023-09-30T22:36:36.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.318+0000 7ff0d560c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d006fa80 con 0x7ff0d0136fe0 2023-09-30T22:36:36.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.318+0000 7ff0ceffd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff0d009e930 0x7ff0d009ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48022/0 (socket says 172.21.15.16:48022) 2023-09-30T22:36:36.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.318+0000 7ff0ce7fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff0d009f410 0x7ff0d0136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:36.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.318+0000 7ff0ceffd700 1 -- 172.21.15.16:0/2293082363 learned_addr learned my addr 172.21.15.16:0/2293082363 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:36:36.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.319+0000 7ff0cf7fe700 1 --2- 172.21.15.16:0/2293082363 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff0d0136fe0 0x7ff0d01393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:36.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.319+0000 7ff0cdffb700 1 -- 172.21.15.16:0/2293082363 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2474500420 0 0) 0x7ff0d0070940 con 0x7ff0d009e930 2023-09-30T22:36:36.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.319+0000 7ff0cdffb700 1 -- 172.21.15.16:0/2293082363 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0b80035b0 con 0x7ff0d009e930 2023-09-30T22:36:36.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.319+0000 7ff0ce7fc700 1 -- 172.21.15.16:0/2293082363 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff0d0136fe0 msgr2=0x7ff0d01393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:36.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.319+0000 7ff0ce7fc700 1 --2- 172.21.15.16:0/2293082363 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff0d0136fe0 0x7ff0d01393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:36.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.319+0000 7ff0ce7fc700 1 -- 172.21.15.16:0/2293082363 >> v1:172.21.15.16:6789/0 conn(0x7ff0d009e930 legacy=0x7ff0d009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:36.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.319+0000 7ff0ce7fc700 1 -- 172.21.15.16:0/2293082363 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0d009fb60 con 0x7ff0d009f410 2023-09-30T22:36:36.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.319+0000 7ff0ce7fc700 1 --2- 172.21.15.16:0/2293082363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0d009f410 0x7ff0d0136aa0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4009840 tx=0x7ff0c4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99fb6349a67bfae2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:36.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.319+0000 7ff0cdffb700 1 -- 172.21.15.16:0/2293082363 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4011830 con 0x7ff0d009f410 2023-09-30T22:36:36.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.319+0000 7ff0cdffb700 1 -- 172.21.15.16:0/2293082363 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c4011990 con 0x7ff0d009f410 2023-09-30T22:36:36.325 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.320+0000 7ff0cdffb700 1 -- 172.21.15.16:0/2293082363 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4011d00 con 0x7ff0d009f410 2023-09-30T22:36:36.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.320+0000 7ff0d560c700 1 -- 172.21.15.16:0/2293082363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0d009f410 msgr2=0x7ff0d0136aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:36.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.320+0000 7ff0d560c700 1 --2- 172.21.15.16:0/2293082363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0d009f410 0x7ff0d0136aa0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4009840 tx=0x7ff0c4009b50 comp rx=0 tx=0).stop 2023-09-30T22:36:36.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.320+0000 7ff0d560c700 1 -- 172.21.15.16:0/2293082363 shutdown_connections 2023-09-30T22:36:36.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.320+0000 7ff0d560c700 1 --2- 172.21.15.16:0/2293082363 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff0d0136fe0 0x7ff0d01393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:36.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.320+0000 7ff0d560c700 1 --2- 172.21.15.16:0/2293082363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0d009f410 0x7ff0d0136aa0 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:36.326 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.320+0000 7ff0d560c700 1 -- 172.21.15.16:0/2293082363 >> 172.21.15.16:0/2293082363 conn(0x7ff0d00a4210 msgr2=0x7ff0d00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:36.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.320+0000 7ff0d560c700 1 -- 172.21.15.16:0/2293082363 shutdown_connections 2023-09-30T22:36:36.327 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.320+0000 7ff0d560c700 1 -- 172.21.15.16:0/2293082363 wait complete. 2023-09-30T22:36:36.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.322+0000 7ff0d560c700 1 Processor -- start 2023-09-30T22:36:36.328 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.322+0000 7ff0d560c700 1 -- start start 2023-09-30T22:36:36.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.323+0000 7ff0d560c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0d009e930 0x7ff0d01c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:36.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.323+0000 7ff0d560c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff0d009f410 0x7ff0d01c8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:36.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.323+0000 7ff0d560c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff0d0136fe0 0x7ff0d01c8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:36.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.323+0000 7ff0d560c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d0134320 con 0x7ff0d009e930 2023-09-30T22:36:36.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.323+0000 7ff0d560c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d0071d20 con 0x7ff0d009f410 2023-09-30T22:36:36.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.323+0000 7ff0d560c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d01241c0 con 0x7ff0d0136fe0 2023-09-30T22:36:36.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.323+0000 7ff0ceffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0d009e930 0x7ff0d01c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:36.329 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.323+0000 7ff0ceffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0d009e930 0x7ff0d01c82f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51574/0 (socket says 172.21.15.16:51574) 2023-09-30T22:36:36.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.323+0000 7ff0ceffd700 1 -- 172.21.15.16:0/3488969413 learned_addr learned my addr 172.21.15.16:0/3488969413 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:36.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.323+0000 7ff0ce7fc700 1 --2- 172.21.15.16:0/3488969413 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff0d009f410 0x7ff0d01c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:36.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.324+0000 7ff0ceffd700 1 -- 172.21.15.16:0/3488969413 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff0d0136fe0 msgr2=0x7ff0d01c8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:36.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.324+0000 7ff0ceffd700 1 --2- 172.21.15.16:0/3488969413 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff0d0136fe0 0x7ff0d01c8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:36.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.324+0000 7ff0ceffd700 1 -- 172.21.15.16:0/3488969413 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff0d009f410 msgr2=0x7ff0d01c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:36.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.324+0000 7ff0ceffd700 1 --2- 172.21.15.16:0/3488969413 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff0d009f410 0x7ff0d01c8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:36.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.324+0000 7ff0ceffd700 1 -- 172.21.15.16:0/3488969413 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c40094a0 con 0x7ff0d009e930 2023-09-30T22:36:36.330 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.324+0000 7ff0ceffd700 1 --2- 172.21.15.16:0/3488969413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0d009e930 0x7ff0d01c82f0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7ff0c000c760 tx=0x7ff0c000ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:36.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.324+0000 7ff0b77fe700 1 -- 172.21.15.16:0/3488969413 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c0005d70 con 0x7ff0d009e930 2023-09-30T22:36:36.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.324+0000 7ff0b77fe700 1 -- 172.21.15.16:0/3488969413 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c00070d0 con 0x7ff0d009e930 2023-09-30T22:36:36.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.324+0000 7ff0b77fe700 1 -- 172.21.15.16:0/3488969413 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c0007460 con 0x7ff0d009e930 2023-09-30T22:36:36.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.324+0000 7ff0d560c700 1 -- 172.21.15.16:0/3488969413 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0d006d470 con 0x7ff0d009e930 2023-09-30T22:36:36.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.324+0000 7ff0d560c700 1 -- 172.21.15.16:0/3488969413 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0d0124b50 con 0x7ff0d009e930 2023-09-30T22:36:36.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.325+0000 7ff0d560c700 1 -- 172.21.15.16:0/3488969413 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0d0074ea0 con 0x7ff0d009e930 2023-09-30T22:36:36.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.325+0000 7ff0b77fe700 1 -- 172.21.15.16:0/3488969413 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff0c00075c0 con 0x7ff0d009e930 2023-09-30T22:36:36.331 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.326+0000 7ff0b77fe700 1 -- 172.21.15.16:0/3488969413 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7ff0c003eb90 con 0x7ff0d009e930 2023-09-30T22:36:36.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.335+0000 7ff0b77fe700 1 -- 172.21.15.16:0/3488969413 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff0c000f260 con 0x7ff0d009e930 2023-09-30T22:36:36.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.358+0000 7f12fccd3700 1 Processor -- start 2023-09-30T22:36:36.360 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.359+0000 7f12fccd3700 1 -- start start 2023-09-30T22:36:36.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.359+0000 7f12fccd3700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f12f809e930 0x7f12f809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:36.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.359+0000 7f12fccd3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12f809f240 0x7f12f81317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:36.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.359+0000 7f12fccd3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12f809f650 0x7f12f8131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:36.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.359+0000 7f12fccd3700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12f8070940 con 0x7f12f809e930 2023-09-30T22:36:36.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.359+0000 7f12fccd3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12f806fd60 con 0x7f12f809f650 2023-09-30T22:36:36.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.359+0000 7f12fccd3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f12f806fa80 con 0x7f12f809f240 2023-09-30T22:36:36.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.359+0000 7f12f659c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f12f809e930 0x7f12f809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:36.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.360+0000 7f12f659c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f12f809e930 0x7f12f809ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51578/0 (socket says 172.21.15.16:51578) 2023-09-30T22:36:36.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.360+0000 7f12f659c700 1 -- 172.21.15.16:0/4231870751 learned_addr learned my addr 172.21.15.16:0/4231870751 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:36.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.360+0000 7f12f6d9d700 1 --2- 172.21.15.16:0/4231870751 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12f809f650 0x7f12f8131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:36.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.360+0000 7f12f5d9b700 1 --2- 172.21.15.16:0/4231870751 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12f809f240 0x7f12f81317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:36.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.360+0000 7f12f6d9d700 1 -- 172.21.15.16:0/4231870751 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12f809f240 msgr2=0x7f12f81317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:36.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.360+0000 7f12f6d9d700 1 --2- 172.21.15.16:0/4231870751 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12f809f240 0x7f12f81317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:36.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.360+0000 7f12f6d9d700 1 -- 172.21.15.16:0/4231870751 >> v2:172.21.15.16:3300/0 conn(0x7f12f809e930 msgr2=0x7f12f809ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:36.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.360+0000 7f12f6d9d700 1 --2- 172.21.15.16:0/4231870751 >> v2:172.21.15.16:3300/0 conn(0x7f12f809e930 0x7f12f809ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:36.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.360+0000 7f12f6d9d700 1 -- 172.21.15.16:0/4231870751 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12f809fb60 con 0x7f12f809f650 2023-09-30T22:36:36.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.360+0000 7f12f5d9b700 1 --2- 172.21.15.16:0/4231870751 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12f809f240 0x7f12f81317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:36.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.360+0000 7f12f6d9d700 1 --2- 172.21.15.16:0/4231870751 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12f809f650 0x7f12f8131d30 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f12f0009d90 tx=0x7f12f000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f093cc902c6a9a88 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:36.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.360+0000 7f12f559a700 1 -- 172.21.15.16:0/4231870751 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f000e070 con 0x7f12f809f650 2023-09-30T22:36:36.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.361+0000 7f12f559a700 1 -- 172.21.15.16:0/4231870751 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12f000bb20 con 0x7f12f809f650 2023-09-30T22:36:36.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.361+0000 7f12f559a700 1 -- 172.21.15.16:0/4231870751 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f0012430 con 0x7f12f809f650 2023-09-30T22:36:36.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.361+0000 7f12fccd3700 1 -- 172.21.15.16:0/4231870751 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12f809f650 msgr2=0x7f12f8131d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:36.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.361+0000 7f12fccd3700 1 --2- 172.21.15.16:0/4231870751 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12f809f650 0x7f12f8131d30 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f12f0009d90 tx=0x7f12f000b330 comp rx=0 tx=0).stop 2023-09-30T22:36:36.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.361+0000 7f12fccd3700 1 -- 172.21.15.16:0/4231870751 shutdown_connections 2023-09-30T22:36:36.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.361+0000 7f12fccd3700 1 --2- 172.21.15.16:0/4231870751 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12f809f650 0x7f12f8131d30 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:36.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.361+0000 7f12fccd3700 1 --2- 172.21.15.16:0/4231870751 >> v2:172.21.15.16:3300/0 conn(0x7f12f809e930 0x7f12f809ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:36.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.361+0000 7f12fccd3700 1 --2- 172.21.15.16:0/4231870751 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12f809f240 0x7f12f81317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:36.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.361+0000 7f12fccd3700 1 -- 172.21.15.16:0/4231870751 >> 172.21.15.16:0/4231870751 conn(0x7f12f80a4210 msgr2=0x7f12f80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:36.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.361+0000 7f12fccd3700 1 -- 172.21.15.16:0/4231870751 shutdown_connections 2023-09-30T22:36:36.367 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.361+0000 7f12fccd3700 1 -- 172.21.15.16:0/4231870751 wait complete. 2023-09-30T22:36:36.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.363+0000 7f12fccd3700 1 Processor -- start 2023-09-30T22:36:36.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.363+0000 7f12fccd3700 1 -- start start 2023-09-30T22:36:36.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.364+0000 7f12fccd3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12f809e930 0x7f12f809c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:36.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.364+0000 7f12fccd3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12f809f240 0x7f12f809c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:36.370 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.364+0000 7f12fccd3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12f809f650 0x7f12f809cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:36.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.364+0000 7f12fccd3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12f81242b0 con 0x7f12f809f240 2023-09-30T22:36:36.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.364+0000 7f12fccd3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12f8071d20 con 0x7f12f809f650 2023-09-30T22:36:36.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.364+0000 7f12fccd3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f12f809d670 con 0x7f12f809e930 2023-09-30T22:36:36.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.364+0000 7f12f659c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12f809e930 0x7f12f809c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:36.371 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.364+0000 7f12f659c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12f809e930 0x7f12f809c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:36730/0 (socket says 172.21.15.16:36730) 2023-09-30T22:36:36.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.364+0000 7f12f659c700 1 -- 172.21.15.16:0/2212501433 learned_addr learned my addr 172.21.15.16:0/2212501433 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:36.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.364+0000 7f12f6d9d700 1 --2- 172.21.15.16:0/2212501433 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12f809f650 0x7f12f809cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:36.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.364+0000 7f12f5d9b700 1 --2- 172.21.15.16:0/2212501433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12f809f240 0x7f12f809c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:36.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.364+0000 7f12f6d9d700 1 -- 172.21.15.16:0/2212501433 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12f809e930 msgr2=0x7f12f809c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:36.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.364+0000 7f12f6d9d700 1 --2- 172.21.15.16:0/2212501433 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12f809e930 0x7f12f809c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:36.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.364+0000 7f12f6d9d700 1 -- 172.21.15.16:0/2212501433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12f809f240 msgr2=0x7f12f809c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:36.372 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.364+0000 7f12f6d9d700 1 --2- 172.21.15.16:0/2212501433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12f809f240 0x7f12f809c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:36.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.364+0000 7f12f6d9d700 1 -- 172.21.15.16:0/2212501433 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12ec0094a0 con 0x7f12f809f650 2023-09-30T22:36:36.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.365+0000 7f12f6d9d700 1 --2- 172.21.15.16:0/2212501433 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12f809f650 0x7f12f809cd00 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f12f0016000 tx=0x7f12f0004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:36.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.365+0000 7f12e6ffd700 1 -- 172.21.15.16:0/2212501433 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f0004800 con 0x7f12f809f650 2023-09-30T22:36:36.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.365+0000 7f12e6ffd700 1 -- 172.21.15.16:0/2212501433 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12f0020070 con 0x7f12f809f650 2023-09-30T22:36:36.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.365+0000 7f12fccd3700 1 -- 172.21.15.16:0/2212501433 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12f0009a70 con 0x7f12f809f650 2023-09-30T22:36:36.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.365+0000 7f12fccd3700 1 -- 172.21.15.16:0/2212501433 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12f80a0190 con 0x7f12f809f650 2023-09-30T22:36:36.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.365+0000 7f12e6ffd700 1 -- 172.21.15.16:0/2212501433 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12f00276c0 con 0x7f12f809f650 2023-09-30T22:36:36.373 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.366+0000 7f12fccd3700 1 -- 172.21.15.16:0/2212501433 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12f8074ea0 con 0x7f12f809f650 2023-09-30T22:36:36.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.367+0000 7f12e6ffd700 1 -- 172.21.15.16:0/2212501433 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f12f0012430 con 0x7f12f809f650 2023-09-30T22:36:36.374 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.367+0000 7f12e6ffd700 1 -- 172.21.15.16:0/2212501433 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f12f0022070 con 0x7f12f809f650 2023-09-30T22:36:36.375 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.372+0000 7f12e6ffd700 1 -- 172.21.15.16:0/2212501433 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f12f0012e40 con 0x7f12f809f650 2023-09-30T22:36:36.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.487+0000 7ff72f50d700 1 -- 172.21.15.16:0/3653177181 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff728126440 con 0x7ff728134170 2023-09-30T22:36:36.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.487+0000 7ff71dffb700 1 -- 172.21.15.16:0/3653177181 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff720011390 con 0x7ff728134170 2023-09-30T22:36:36.489 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:36:36.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.490+0000 7ff72f50d700 1 -- 172.21.15.16:0/3653177181 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff728134170 msgr2=0x7ff7281c8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:36.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.490+0000 7ff72f50d700 1 --2- 172.21.15.16:0/3653177181 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff728134170 0x7ff7281c8ea0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7ff72000b370 tx=0x7ff720004ad0 comp rx=0 tx=0).stop 2023-09-30T22:36:36.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.491+0000 7ff72f50d700 1 -- 172.21.15.16:0/3653177181 shutdown_connections 2023-09-30T22:36:36.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.491+0000 7ff72f50d700 1 --2- 172.21.15.16:0/3653177181 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff728134170 0x7ff7281c8ea0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:36.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.491+0000 7ff72f50d700 1 --2- 172.21.15.16:0/3653177181 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff7280a0a30 0x7ff7281c8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:36.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.491+0000 7ff72f50d700 1 --2- 172.21.15.16:0/3653177181 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7280a1340 0x7ff7281c8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:36.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.491+0000 7ff72f50d700 1 -- 172.21.15.16:0/3653177181 >> 172.21.15.16:0/3653177181 conn(0x7ff7280a6550 msgr2=0x7ff72809ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:36.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.491+0000 7ff72f50d700 1 -- 172.21.15.16:0/3653177181 shutdown_connections 2023-09-30T22:36:36.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.491+0000 7ff72f50d700 1 -- 172.21.15.16:0/3653177181 wait complete. 2023-09-30T22:36:36.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.495+0000 7f2433a57700 1 -- 172.21.15.16:0/924687507 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f242c074ea0 con 0x7f242c0a1340 2023-09-30T22:36:36.497 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.496+0000 7f2421ffb700 1 -- 172.21.15.16:0/924687507 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f241c02b030 con 0x7f242c0a1340 2023-09-30T22:36:36.497 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:36:36.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.500+0000 7f2433a57700 1 -- 172.21.15.16:0/924687507 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f242c0a1340 msgr2=0x7f242c123870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:36.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.500+0000 7f2433a57700 1 --2- 172.21.15.16:0/924687507 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f242c0a1340 0x7f242c123870 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f241c000c00 tx=0x7f241c004750 comp rx=0 tx=0).stop 2023-09-30T22:36:36.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.502+0000 7f2433a57700 1 -- 172.21.15.16:0/924687507 shutdown_connections 2023-09-30T22:36:36.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.502+0000 7f2433a57700 1 --2- 172.21.15.16:0/924687507 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f242c0a0a30 0x7f242c123330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:36.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.502+0000 7f2433a57700 1 --2- 172.21.15.16:0/924687507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f242c133fa0 0x7f242c1c8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:36.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.502+0000 7f2433a57700 1 --2- 172.21.15.16:0/924687507 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f242c0a1340 0x7f242c123870 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:36.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.502+0000 7f2433a57700 1 -- 172.21.15.16:0/924687507 >> 172.21.15.16:0/924687507 conn(0x7f242c0a6550 msgr2=0x7f242c0a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:36.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.502+0000 7f2433a57700 1 -- 172.21.15.16:0/924687507 shutdown_connections 2023-09-30T22:36:36.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.502+0000 7f2433a57700 1 -- 172.21.15.16:0/924687507 wait complete. 2023-09-30T22:36:36.509 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:36:36.523 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:36:36.550 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.549+0000 7ff0d560c700 1 -- 172.21.15.16:0/3488969413 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff0d009dbd0 con 0x7ff0d009e930 2023-09-30T22:36:36.551 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.550+0000 7ff0b77fe700 1 -- 172.21.15.16:0/3488969413 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff0c0017030 con 0x7ff0d009e930 2023-09-30T22:36:36.551 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:36:36.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.552+0000 7ff0d560c700 1 -- 172.21.15.16:0/3488969413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0d009e930 msgr2=0x7ff0d01c82f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:36.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.552+0000 7ff0d560c700 1 --2- 172.21.15.16:0/3488969413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0d009e930 0x7ff0d01c82f0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7ff0c000c760 tx=0x7ff0c000ca70 comp rx=0 tx=0).stop 2023-09-30T22:36:36.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.552+0000 7ff0d560c700 1 -- 172.21.15.16:0/3488969413 shutdown_connections 2023-09-30T22:36:36.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.552+0000 7ff0d560c700 1 --2- 172.21.15.16:0/3488969413 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff0d0136fe0 0x7ff0d01c8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:36.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.552+0000 7ff0d560c700 1 --2- 172.21.15.16:0/3488969413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0d009e930 0x7ff0d01c82f0 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:36.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.552+0000 7ff0d560c700 1 --2- 172.21.15.16:0/3488969413 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff0d009f410 0x7ff0d01c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:36.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.552+0000 7ff0d560c700 1 -- 172.21.15.16:0/3488969413 >> 172.21.15.16:0/3488969413 conn(0x7ff0d00a4210 msgr2=0x7ff0d0134890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:36.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.552+0000 7ff0d560c700 1 -- 172.21.15.16:0/3488969413 shutdown_connections 2023-09-30T22:36:36.554 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.552+0000 7ff0d560c700 1 -- 172.21.15.16:0/3488969413 wait complete. 2023-09-30T22:36:36.567 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:36:36.591 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.590+0000 7f12fccd3700 1 -- 172.21.15.16:0/2212501433 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f12f806d800 con 0x7f12f809f650 2023-09-30T22:36:36.592 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.590+0000 7f12e6ffd700 1 -- 172.21.15.16:0/2212501433 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f12f002f350 con 0x7f12f809f650 2023-09-30T22:36:36.592 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:36:36.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.593+0000 7f12fccd3700 1 -- 172.21.15.16:0/2212501433 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12f809f650 msgr2=0x7f12f809cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:36.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.593+0000 7f12fccd3700 1 --2- 172.21.15.16:0/2212501433 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12f809f650 0x7f12f809cd00 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f12f0016000 tx=0x7f12f0004ad0 comp rx=0 tx=0).stop 2023-09-30T22:36:36.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.593+0000 7f12fccd3700 1 -- 172.21.15.16:0/2212501433 shutdown_connections 2023-09-30T22:36:36.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.593+0000 7f12fccd3700 1 --2- 172.21.15.16:0/2212501433 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f12f809f650 0x7f12f809cd00 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:36.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.593+0000 7f12fccd3700 1 --2- 172.21.15.16:0/2212501433 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f12f809e930 0x7f12f809c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:36.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.593+0000 7f12fccd3700 1 --2- 172.21.15.16:0/2212501433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12f809f240 0x7f12f809c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:36.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.593+0000 7f12fccd3700 1 -- 172.21.15.16:0/2212501433 >> 172.21.15.16:0/2212501433 conn(0x7f12f80a4210 msgr2=0x7f12f806c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:36.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.593+0000 7f12fccd3700 1 -- 172.21.15.16:0/2212501433 shutdown_connections 2023-09-30T22:36:36.595 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:36.593+0000 7f12fccd3700 1 -- 172.21.15.16:0/2212501433 wait complete. 2023-09-30T22:36:36.608 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:36:37.510 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:36:37.524 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:36:37.568 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:36:37.609 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:36:37.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.670+0000 7f22c0624700 1 Processor -- start 2023-09-30T22:36:37.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.670+0000 7f22c0624700 1 -- start start 2023-09-30T22:36:37.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.671+0000 7f22c0624700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f22b8130540 0x7f22b8130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:37.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.671+0000 7f22c0624700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f22b8130e50 0x7f22b806f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:37.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.671+0000 7f22c0624700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f22b806fb20 0x7f22b806ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:37.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.671+0000 7f22c0624700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22b809a390 con 0x7f22b8130540 2023-09-30T22:36:37.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.671+0000 7f22c0624700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22b80716a0 con 0x7f22b8130e50 2023-09-30T22:36:37.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.671+0000 7f22c0624700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f22b8071290 con 0x7f22b806fb20 2023-09-30T22:36:37.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.672+0000 7f22bdbbf700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f22b8130e50 0x7f22b806f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:37.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.672+0000 7f22bdbbf700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f22b8130e50 0x7f22b806f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:59136/0 (socket says 172.21.15.16:59136) 2023-09-30T22:36:37.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.672+0000 7f22bdbbf700 1 -- 172.21.15.16:0/2974396392 learned_addr learned my addr 172.21.15.16:0/2974396392 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:37.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.672+0000 7f22be3c0700 1 --2- 172.21.15.16:0/2974396392 >> v2:172.21.15.16:3300/0 conn(0x7f22b8130540 0x7f22b8130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:37.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.672+0000 7f22bebc1700 1 --2- 172.21.15.16:0/2974396392 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f22b806fb20 0x7f22b806ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:37.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.672+0000 7f22be3c0700 1 -- 172.21.15.16:0/2974396392 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f22b806fb20 msgr2=0x7f22b806ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:37.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.672+0000 7f22be3c0700 1 --2- 172.21.15.16:0/2974396392 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f22b806fb20 0x7f22b806ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:37.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.672+0000 7f22be3c0700 1 -- 172.21.15.16:0/2974396392 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f22b8130e50 msgr2=0x7f22b806f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:37.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.672+0000 7f22be3c0700 1 --2- 172.21.15.16:0/2974396392 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f22b8130e50 0x7f22b806f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:37.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.672+0000 7f22be3c0700 1 -- 172.21.15.16:0/2974396392 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b8130340 con 0x7f22b8130540 2023-09-30T22:36:37.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.672+0000 7f22be3c0700 1 --2- 172.21.15.16:0/2974396392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22b8130540 0x7f22b8130910 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f22b4009840 tx=0x7f22b4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76fe5d188f2c34e2 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:37.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.673+0000 7f22bd3be700 1 -- 172.21.15.16:0/2974396392 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b4011830 con 0x7f22b8130540 2023-09-30T22:36:37.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.673+0000 7f22bd3be700 1 -- 172.21.15.16:0/2974396392 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22b4011990 con 0x7f22b8130540 2023-09-30T22:36:37.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.673+0000 7f22bd3be700 1 -- 172.21.15.16:0/2974396392 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b4011d20 con 0x7f22b8130540 2023-09-30T22:36:37.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.673+0000 7f22c0624700 1 -- 172.21.15.16:0/2974396392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22b8130540 msgr2=0x7f22b8130910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:37.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.673+0000 7f22c0624700 1 --2- 172.21.15.16:0/2974396392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22b8130540 0x7f22b8130910 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f22b4009840 tx=0x7f22b4009b50 comp rx=0 tx=0).stop 2023-09-30T22:36:37.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.674+0000 7f22c0624700 1 -- 172.21.15.16:0/2974396392 shutdown_connections 2023-09-30T22:36:37.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.674+0000 7f22c0624700 1 --2- 172.21.15.16:0/2974396392 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f22b806fb20 0x7f22b806ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:37.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.674+0000 7f22c0624700 1 --2- 172.21.15.16:0/2974396392 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22b8130540 0x7f22b8130910 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:37.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.674+0000 7f22c0624700 1 --2- 172.21.15.16:0/2974396392 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f22b8130e50 0x7f22b806f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:37.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.674+0000 7f22c0624700 1 -- 172.21.15.16:0/2974396392 >> 172.21.15.16:0/2974396392 conn(0x7f22b80a2aa0 msgr2=0x7f22b80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:37.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.674+0000 7f22c0624700 1 -- 172.21.15.16:0/2974396392 shutdown_connections 2023-09-30T22:36:37.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.674+0000 7f22c0624700 1 -- 172.21.15.16:0/2974396392 wait complete. 2023-09-30T22:36:37.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.675+0000 7f22c0624700 1 Processor -- start 2023-09-30T22:36:37.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.675+0000 7f22c0624700 1 -- start start 2023-09-30T22:36:37.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.676+0000 7f22c0624700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22b806fb20 0x7f22b81c4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:37.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.676+0000 7f22c0624700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f22b8130540 0x7f22b81c4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:37.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.676+0000 7f22c0624700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f22b8130e50 0x7f22b8118cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:37.679 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.676+0000 7f22c0624700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b812bcb0 con 0x7f22b806fb20 2023-09-30T22:36:37.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.676+0000 7f22c0624700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22b811b6c0 con 0x7f22b8130540 2023-09-30T22:36:37.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.676+0000 7f22c0624700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f22b811b190 con 0x7f22b8130e50 2023-09-30T22:36:37.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.676+0000 7f22bdbbf700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f22b8130540 0x7f22b81c4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:37.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.676+0000 7f22bdbbf700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f22b8130540 0x7f22b81c4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:59150/0 (socket says 172.21.15.16:59150) 2023-09-30T22:36:37.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.676+0000 7f22bdbbf700 1 -- 172.21.15.16:0/3295163001 learned_addr learned my addr 172.21.15.16:0/3295163001 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:37.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.677+0000 7f22be3c0700 1 --2- 172.21.15.16:0/3295163001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22b806fb20 0x7f22b81c4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:37.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.677+0000 7f22bebc1700 1 --2- 172.21.15.16:0/3295163001 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f22b8130e50 0x7f22b8118cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:37.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.677+0000 7f22be3c0700 1 -- 172.21.15.16:0/3295163001 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f22b8130e50 msgr2=0x7f22b8118cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:37.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.677+0000 7f22be3c0700 1 --2- 172.21.15.16:0/3295163001 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f22b8130e50 0x7f22b8118cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:37.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.677+0000 7f22be3c0700 1 -- 172.21.15.16:0/3295163001 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f22b8130540 msgr2=0x7f22b81c4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:37.680 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.677+0000 7f22be3c0700 1 --2- 172.21.15.16:0/3295163001 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f22b8130540 0x7f22b81c4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:37.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.677+0000 7f22be3c0700 1 -- 172.21.15.16:0/3295163001 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b40094a0 con 0x7f22b806fb20 2023-09-30T22:36:37.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.677+0000 7f22bebc1700 1 --2- 172.21.15.16:0/3295163001 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f22b8130e50 0x7f22b8118cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:37.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.677+0000 7f22bdbbf700 1 --2- 172.21.15.16:0/3295163001 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f22b8130540 0x7f22b81c4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:37.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.677+0000 7f22be3c0700 1 --2- 172.21.15.16:0/3295163001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22b806fb20 0x7f22b81c4020 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f22b4016000 tx=0x7f22b4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:37.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.678+0000 7f22aeffd700 1 -- 172.21.15.16:0/3295163001 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b4004240 con 0x7f22b806fb20 2023-09-30T22:36:37.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.678+0000 7f22aeffd700 1 -- 172.21.15.16:0/3295163001 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22b401f070 con 0x7f22b806fb20 2023-09-30T22:36:37.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.678+0000 7f22c0624700 1 -- 172.21.15.16:0/3295163001 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22b81c3b60 con 0x7f22b806fb20 2023-09-30T22:36:37.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.678+0000 7f22aeffd700 1 -- 172.21.15.16:0/3295163001 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22b40036a0 con 0x7f22b806fb20 2023-09-30T22:36:37.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.678+0000 7f22c0624700 1 -- 172.21.15.16:0/3295163001 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22b8119f90 con 0x7f22b806fb20 2023-09-30T22:36:37.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.679+0000 7f22aeffd700 1 -- 172.21.15.16:0/3295163001 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f22b4021030 con 0x7f22b806fb20 2023-09-30T22:36:37.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.679+0000 7f22aeffd700 1 -- 172.21.15.16:0/3295163001 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f22b4040880 con 0x7f22b806fb20 2023-09-30T22:36:37.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.679+0000 7f22c0624700 1 -- 172.21.15.16:0/3295163001 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22b81c4aa0 con 0x7f22b806fb20 2023-09-30T22:36:37.690 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.688+0000 7f22aeffd700 1 -- 172.21.15.16:0/3295163001 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f22b4015020 con 0x7f22b806fb20 2023-09-30T22:36:37.708 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.707+0000 7f1c29e7d700 1 Processor -- start 2023-09-30T22:36:37.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.707+0000 7f1c29e7d700 1 -- start start 2023-09-30T22:36:37.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.707+0000 7f1c29e7d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1c2409de00 0x7f1c2409e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:37.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.707+0000 7f1c29e7d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1c2413d1f0 0x7f1c2413f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:37.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.707+0000 7f1c29e7d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c2409a810 con 0x7f1c2406a770 2023-09-30T22:36:37.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.707+0000 7f1c29e7d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2406ab40 con 0x7f1c2409de00 2023-09-30T22:36:37.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.707+0000 7f1c29e7d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2409ebc0 con 0x7f1c2413d1f0 2023-09-30T22:36:37.710 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.707+0000 7f1c237fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1c2406a770 0x7f1c2409d6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60012/0 (socket says 172.21.15.16:60012) 2023-09-30T22:36:37.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.707+0000 7f1c22ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1c2409de00 0x7f1c2409e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:37.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.707+0000 7f1c237fe700 1 -- 172.21.15.16:0/1978662179 learned_addr learned my addr 172.21.15.16:0/1978662179 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:36:37.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.708+0000 7f1c23fff700 1 --2- 172.21.15.16:0/1978662179 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1c2413d1f0 0x7f1c2413f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:37.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.708+0000 7f1c22ffd700 1 -- 172.21.15.16:0/1978662179 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1c2413d1f0 msgr2=0x7f1c2413f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:37.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.708+0000 7f1c22ffd700 1 --2- 172.21.15.16:0/1978662179 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1c2413d1f0 0x7f1c2413f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:37.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.708+0000 7f1c22ffd700 1 -- 172.21.15.16:0/1978662179 >> v1:172.21.15.16:6789/0 conn(0x7f1c2406a770 legacy=0x7f1c2409d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:37.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.708+0000 7f1c227fc700 1 -- 172.21.15.16:0/1978662179 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3094936492 0 0) 0x7f1c2409a810 con 0x7f1c2406a770 2023-09-30T22:36:37.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.708+0000 7f1c22ffd700 1 -- 172.21.15.16:0/1978662179 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c2409e920 con 0x7f1c2409de00 2023-09-30T22:36:37.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.708+0000 7f1c22ffd700 1 --2- 172.21.15.16:0/1978662179 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1c2409de00 0x7f1c2409e250 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f1c18009870 tx=0x7f1c18009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2b24c14447bf473 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:37.711 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.708+0000 7f1c227fc700 1 -- 172.21.15.16:0/1978662179 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c18011830 con 0x7f1c2409de00 2023-09-30T22:36:37.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.708+0000 7f1c227fc700 1 -- 172.21.15.16:0/1978662179 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c18011990 con 0x7f1c2409de00 2023-09-30T22:36:37.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.708+0000 7f1c227fc700 1 -- 172.21.15.16:0/1978662179 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c18011d20 con 0x7f1c2409de00 2023-09-30T22:36:37.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.709+0000 7f1c29e7d700 1 -- 172.21.15.16:0/1978662179 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1c2409de00 msgr2=0x7f1c2409e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:37.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.709+0000 7f1c29e7d700 1 --2- 172.21.15.16:0/1978662179 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1c2409de00 0x7f1c2409e250 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f1c18009870 tx=0x7f1c18009b80 comp rx=0 tx=0).stop 2023-09-30T22:36:37.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.709+0000 7f1c29e7d700 1 -- 172.21.15.16:0/1978662179 shutdown_connections 2023-09-30T22:36:37.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.709+0000 7f1c29e7d700 1 --2- 172.21.15.16:0/1978662179 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1c2413d1f0 0x7f1c2413f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:37.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.709+0000 7f1c29e7d700 1 --2- 172.21.15.16:0/1978662179 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1c2409de00 0x7f1c2409e250 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:37.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.709+0000 7f1c29e7d700 1 -- 172.21.15.16:0/1978662179 >> 172.21.15.16:0/1978662179 conn(0x7f1c24066f50 msgr2=0x7f1c2409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:37.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.709+0000 7f1c29e7d700 1 -- 172.21.15.16:0/1978662179 shutdown_connections 2023-09-30T22:36:37.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.709+0000 7f1c29e7d700 1 -- 172.21.15.16:0/1978662179 wait complete. 2023-09-30T22:36:37.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.711+0000 7f1c29e7d700 1 Processor -- start 2023-09-30T22:36:37.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.711+0000 7f1c29e7d700 1 -- start start 2023-09-30T22:36:37.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.712+0000 7f1c29e7d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1c2406a770 0x7f1c240716e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:37.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.712+0000 7f1c29e7d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1c2409de00 0x7f1c2413b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:37.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.712+0000 7f1c29e7d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c2413d1f0 0x7f1c2413bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:37.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.712+0000 7f1c29e7d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2411f030 con 0x7f1c2413d1f0 2023-09-30T22:36:37.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.712+0000 7f1c29e7d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2413cdf0 con 0x7f1c2409de00 2023-09-30T22:36:37.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.712+0000 7f1c29e7d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2411eb00 con 0x7f1c2406a770 2023-09-30T22:36:37.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.712+0000 7f1c237fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1c2406a770 0x7f1c240716e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:37.717 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.712+0000 7f1c237fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1c2406a770 0x7f1c240716e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35432/0 (socket says 172.21.15.16:35432) 2023-09-30T22:36:37.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.712+0000 7f1c237fe700 1 -- 172.21.15.16:0/1503124734 learned_addr learned my addr 172.21.15.16:0/1503124734 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:37.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.712+0000 7f1c22ffd700 1 --2- 172.21.15.16:0/1503124734 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1c2409de00 0x7f1c2413b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:37.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.712+0000 7f1c23fff700 1 --2- 172.21.15.16:0/1503124734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c2413d1f0 0x7f1c2413bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:37.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.712+0000 7f1c23fff700 1 -- 172.21.15.16:0/1503124734 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1c2406a770 msgr2=0x7f1c240716e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:37.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.712+0000 7f1c23fff700 1 --2- 172.21.15.16:0/1503124734 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1c2406a770 0x7f1c240716e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:37.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.712+0000 7f1c23fff700 1 -- 172.21.15.16:0/1503124734 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1c2409de00 msgr2=0x7f1c2413b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:37.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.712+0000 7f1c23fff700 1 --2- 172.21.15.16:0/1503124734 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1c2409de00 0x7f1c2413b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:37.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.712+0000 7f1c23fff700 1 -- 172.21.15.16:0/1503124734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c180094a0 con 0x7f1c2413d1f0 2023-09-30T22:36:37.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.712+0000 7f1c237fe700 1 --2- 172.21.15.16:0/1503124734 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1c2406a770 0x7f1c240716e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:37.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.712+0000 7f1c23fff700 1 --2- 172.21.15.16:0/1503124734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c2413d1f0 0x7f1c2413bae0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c00efc0 tx=0x7f1c1c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:37.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.713+0000 7f1c0bfff700 1 -- 172.21.15.16:0/1503124734 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c00cc90 con 0x7f1c2413d1f0 2023-09-30T22:36:37.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.713+0000 7f1c29e7d700 1 -- 172.21.15.16:0/1503124734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c2411ed50 con 0x7f1c2413d1f0 2023-09-30T22:36:37.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.713+0000 7f1c0bfff700 1 -- 172.21.15.16:0/1503124734 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c1c00f040 con 0x7f1c2413d1f0 2023-09-30T22:36:37.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.713+0000 7f1c0bfff700 1 -- 172.21.15.16:0/1503124734 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c0146c0 con 0x7f1c2413d1f0 2023-09-30T22:36:37.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.713+0000 7f1c29e7d700 1 -- 172.21.15.16:0/1503124734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c240721a0 con 0x7f1c2413d1f0 2023-09-30T22:36:37.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.714+0000 7f1c29e7d700 1 -- 172.21.15.16:0/1503124734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c24074ea0 con 0x7f1c2413d1f0 2023-09-30T22:36:37.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.715+0000 7f1c0bfff700 1 -- 172.21.15.16:0/1503124734 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1c1c014820 con 0x7f1c2413d1f0 2023-09-30T22:36:37.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.715+0000 7f1c0bfff700 1 -- 172.21.15.16:0/1503124734 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f1c1c040830 con 0x7f1c2413d1f0 2023-09-30T22:36:37.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.723+0000 7f1c0bfff700 1 -- 172.21.15.16:0/1503124734 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1c1c012930 con 0x7f1c2413d1f0 2023-09-30T22:36:37.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.766+0000 7fbc0c891700 1 Processor -- start 2023-09-30T22:36:37.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.766+0000 7fbc0c891700 1 -- start start 2023-09-30T22:36:37.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.767+0000 7fbc0c891700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbc0809e1d0 0x7fbc08138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:37.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.767+0000 7fbc0c891700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbc08139170 0x7fbc0813b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:37.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.767+0000 7fbc0c891700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc0809b120 con 0x7fbc0809d6f0 2023-09-30T22:36:37.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.767+0000 7fbc0c891700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0806b460 con 0x7fbc0809e1d0 2023-09-30T22:36:37.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.767+0000 7fbc0c891700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0809a7e0 con 0x7fbc08139170 2023-09-30T22:36:37.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.767+0000 7fbc0659c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fbc0809d6f0 0x7fbc0809dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60018/0 (socket says 172.21.15.16:60018) 2023-09-30T22:36:37.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.767+0000 7fbc0659c700 1 -- 172.21.15.16:0/2095044113 learned_addr learned my addr 172.21.15.16:0/2095044113 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:36:37.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.767+0000 7fbc06d9d700 1 --2- 172.21.15.16:0/2095044113 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbc08139170 0x7fbc0813b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:37.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.767+0000 7fbc05d9b700 1 --2- 172.21.15.16:0/2095044113 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbc0809e1d0 0x7fbc08138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:37.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.768+0000 7fbc05d9b700 1 -- 172.21.15.16:0/2095044113 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbc08139170 msgr2=0x7fbc0813b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:37.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.768+0000 7fbc05d9b700 1 --2- 172.21.15.16:0/2095044113 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbc08139170 0x7fbc0813b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:37.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.768+0000 7fbc0559a700 1 -- 172.21.15.16:0/2095044113 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4152604410 0 0) 0x7fbc0809b120 con 0x7fbc0809d6f0 2023-09-30T22:36:37.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.768+0000 7fbc05d9b700 1 -- 172.21.15.16:0/2095044113 >> v1:172.21.15.16:6789/0 conn(0x7fbc0809d6f0 legacy=0x7fbc0809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:37.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.768+0000 7fbc05d9b700 1 -- 172.21.15.16:0/2095044113 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc0809a500 con 0x7fbc0809e1d0 2023-09-30T22:36:37.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.768+0000 7fbc06d9d700 1 --2- 172.21.15.16:0/2095044113 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbc08139170 0x7fbc0813b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:37.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.768+0000 7fbc05d9b700 1 --2- 172.21.15.16:0/2095044113 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbc0809e1d0 0x7fbc08138c30 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc009870 tx=0x7fbbfc009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69236403806a679a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:37.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.768+0000 7fbc0559a700 1 -- 172.21.15.16:0/2095044113 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbfc011830 con 0x7fbc0809e1d0 2023-09-30T22:36:37.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.768+0000 7fbc0559a700 1 -- 172.21.15.16:0/2095044113 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbfc011990 con 0x7fbc0809e1d0 2023-09-30T22:36:37.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.768+0000 7fbc0559a700 1 -- 172.21.15.16:0/2095044113 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbfc011d20 con 0x7fbc0809e1d0 2023-09-30T22:36:37.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.769+0000 7fbc0c891700 1 -- 172.21.15.16:0/2095044113 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbc0809e1d0 msgr2=0x7fbc08138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:37.773 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.769+0000 7fbc0c891700 1 --2- 172.21.15.16:0/2095044113 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbc0809e1d0 0x7fbc08138c30 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fbbfc009870 tx=0x7fbbfc009b80 comp rx=0 tx=0).stop 2023-09-30T22:36:37.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.769+0000 7fbc0c891700 1 -- 172.21.15.16:0/2095044113 shutdown_connections 2023-09-30T22:36:37.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.769+0000 7fbc0c891700 1 --2- 172.21.15.16:0/2095044113 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbc08139170 0x7fbc0813b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:37.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.769+0000 7fbc0c891700 1 --2- 172.21.15.16:0/2095044113 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbc0809e1d0 0x7fbc08138c30 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:37.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.769+0000 7fbc0c891700 1 -- 172.21.15.16:0/2095044113 >> 172.21.15.16:0/2095044113 conn(0x7fbc08066f50 msgr2=0x7fbc0809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:37.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.769+0000 7fbc0c891700 1 -- 172.21.15.16:0/2095044113 shutdown_connections 2023-09-30T22:36:37.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.769+0000 7fbc0c891700 1 -- 172.21.15.16:0/2095044113 wait complete. 2023-09-30T22:36:37.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.770+0000 7fbc0c891700 1 Processor -- start 2023-09-30T22:36:37.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.770+0000 7fbc0c891700 1 -- start start 2023-09-30T22:36:37.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.771+0000 7fbc0c891700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbc0809d6f0 0x7fbc08120770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:37.774 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.771+0000 7fbc0c891700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc0809e1d0 0x7fbc08120cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:37.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.771+0000 7fbc0c891700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbc08139170 0x7fbc081211f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:37.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.771+0000 7fbc0c891700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc08122ba0 con 0x7fbc0809e1d0 2023-09-30T22:36:37.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.771+0000 7fbc0c891700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc08136610 con 0x7fbc0809d6f0 2023-09-30T22:36:37.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.771+0000 7fbc0c891700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0809c710 con 0x7fbc08139170 2023-09-30T22:36:37.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.771+0000 7fbc0659c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbc0809d6f0 0x7fbc08120770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:37.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.771+0000 7fbc0659c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbc0809d6f0 0x7fbc08120770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:59180/0 (socket says 172.21.15.16:59180) 2023-09-30T22:36:37.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.771+0000 7fbc0659c700 1 -- 172.21.15.16:0/1609231119 learned_addr learned my addr 172.21.15.16:0/1609231119 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:37.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.771+0000 7fbc06d9d700 1 --2- 172.21.15.16:0/1609231119 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbc08139170 0x7fbc081211f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:37.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.771+0000 7fbc05d9b700 1 --2- 172.21.15.16:0/1609231119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc0809e1d0 0x7fbc08120cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:37.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.771+0000 7fbc0659c700 1 -- 172.21.15.16:0/1609231119 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbc08139170 msgr2=0x7fbc081211f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:37.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.771+0000 7fbc0659c700 1 --2- 172.21.15.16:0/1609231119 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbc08139170 0x7fbc081211f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:37.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.771+0000 7fbc0659c700 1 -- 172.21.15.16:0/1609231119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc0809e1d0 msgr2=0x7fbc08120cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:37.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.771+0000 7fbc0659c700 1 --2- 172.21.15.16:0/1609231119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc0809e1d0 0x7fbc08120cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:37.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.771+0000 7fbc0659c700 1 -- 172.21.15.16:0/1609231119 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc00009a70 con 0x7fbc0809d6f0 2023-09-30T22:36:37.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.772+0000 7fbc0659c700 1 --2- 172.21.15.16:0/1609231119 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbc0809d6f0 0x7fbc08120770 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fbbf800c520 tx=0x7fbbf800c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:37.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.772+0000 7fbbf6ffd700 1 -- 172.21.15.16:0/1609231119 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf8004c80 con 0x7fbc0809d6f0 2023-09-30T22:36:37.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.772+0000 7fbbf6ffd700 1 -- 172.21.15.16:0/1609231119 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbf8004e00 con 0x7fbc0809d6f0 2023-09-30T22:36:37.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.772+0000 7fbc0c891700 1 -- 172.21.15.16:0/1609231119 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbfc0094a0 con 0x7fbc0809d6f0 2023-09-30T22:36:37.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.772+0000 7fbc0c891700 1 -- 172.21.15.16:0/1609231119 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc08072150 con 0x7fbc0809d6f0 2023-09-30T22:36:37.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.772+0000 7fbbf6ffd700 1 -- 172.21.15.16:0/1609231119 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf8007310 con 0x7fbc0809d6f0 2023-09-30T22:36:37.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.773+0000 7fbbf6ffd700 1 -- 172.21.15.16:0/1609231119 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fbbf8007470 con 0x7fbc0809d6f0 2023-09-30T22:36:37.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.773+0000 7fbc0c891700 1 -- 172.21.15.16:0/1609231119 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbec0052e0 con 0x7fbc0809d6f0 2023-09-30T22:36:37.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.773+0000 7fbbf6ffd700 1 -- 172.21.15.16:0/1609231119 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fbbf803f490 con 0x7fbc0809d6f0 2023-09-30T22:36:37.779 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.778+0000 7fbbf6ffd700 1 -- 172.21.15.16:0/1609231119 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fbbf80123b0 con 0x7fbc0809d6f0 2023-09-30T22:36:37.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.803+0000 7fb461029700 1 Processor -- start 2023-09-30T22:36:37.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.803+0000 7fb461029700 1 -- start start 2023-09-30T22:36:37.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.804+0000 7fb461029700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb45c0a0a30 0x7fb45c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:37.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.804+0000 7fb461029700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb45c138d80 0x7fb45c13b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:37.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.804+0000 7fb461029700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb45c0a2a70 con 0x7fb45c09d630 2023-09-30T22:36:37.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.804+0000 7fb461029700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb45c0a01d0 con 0x7fb45c138d80 2023-09-30T22:36:37.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.804+0000 7fb461029700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb45c09e2b0 con 0x7fb45c0a0a30 2023-09-30T22:36:37.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.804+0000 7fb45ad9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb45c0a0a30 0x7fb45c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:37.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.804+0000 7fb45ad9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb45c0a0a30 0x7fb45c0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:59192/0 (socket says 172.21.15.16:59192) 2023-09-30T22:36:37.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.804+0000 7fb45ad9d700 1 -- 172.21.15.16:0/1875241523 learned_addr learned my addr 172.21.15.16:0/1875241523 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:37.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.804+0000 7fb45b59e700 1 --2- 172.21.15.16:0/1875241523 >> v2:172.21.15.16:3300/0 conn(0x7fb45c138d80 0x7fb45c13b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:37.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.804+0000 7fb45ad9d700 1 -- 172.21.15.16:0/1875241523 >> v2:172.21.15.16:3300/0 conn(0x7fb45c138d80 msgr2=0x7fb45c13b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:37.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.804+0000 7fb45ad9d700 1 --2- 172.21.15.16:0/1875241523 >> v2:172.21.15.16:3300/0 conn(0x7fb45c138d80 0x7fb45c13b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:37.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.804+0000 7fb45ad9d700 1 -- 172.21.15.16:0/1875241523 >> v1:172.21.15.16:6789/0 conn(0x7fb45c09d630 legacy=0x7fb45c0a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:37.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.804+0000 7fb45ad9d700 1 -- 172.21.15.16:0/1875241523 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb45c09dc50 con 0x7fb45c0a0a30 2023-09-30T22:36:37.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.804+0000 7fb45b59e700 1 --2- 172.21.15.16:0/1875241523 >> v2:172.21.15.16:3300/0 conn(0x7fb45c138d80 0x7fb45c13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:37.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.805+0000 7fb45ad9d700 1 --2- 172.21.15.16:0/1875241523 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb45c0a0a30 0x7fb45c0a0e00 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fb44c0098f0 tx=0x7fb44c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5f6c88461f5923c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:37.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.805+0000 7fb459d9b700 1 -- 172.21.15.16:0/1875241523 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb44c011830 con 0x7fb45c0a0a30 2023-09-30T22:36:37.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.805+0000 7fb459d9b700 1 -- 172.21.15.16:0/1875241523 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb44c011990 con 0x7fb45c0a0a30 2023-09-30T22:36:37.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.805+0000 7fb459d9b700 1 -- 172.21.15.16:0/1875241523 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb44c011d20 con 0x7fb45c0a0a30 2023-09-30T22:36:37.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.805+0000 7fb461029700 1 -- 172.21.15.16:0/1875241523 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb45c0a0a30 msgr2=0x7fb45c0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:37.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.805+0000 7fb461029700 1 --2- 172.21.15.16:0/1875241523 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb45c0a0a30 0x7fb45c0a0e00 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fb44c0098f0 tx=0x7fb44c009c00 comp rx=0 tx=0).stop 2023-09-30T22:36:37.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.805+0000 7fb461029700 1 -- 172.21.15.16:0/1875241523 shutdown_connections 2023-09-30T22:36:37.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.805+0000 7fb461029700 1 --2- 172.21.15.16:0/1875241523 >> v2:172.21.15.16:3300/0 conn(0x7fb45c138d80 0x7fb45c13b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:37.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.805+0000 7fb461029700 1 --2- 172.21.15.16:0/1875241523 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb45c0a0a30 0x7fb45c0a0e00 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:37.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.805+0000 7fb461029700 1 -- 172.21.15.16:0/1875241523 >> 172.21.15.16:0/1875241523 conn(0x7fb45c0a3050 msgr2=0x7fb45c0a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:37.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.806+0000 7fb461029700 1 -- 172.21.15.16:0/1875241523 shutdown_connections 2023-09-30T22:36:37.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.806+0000 7fb461029700 1 -- 172.21.15.16:0/1875241523 wait complete. 2023-09-30T22:36:37.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.807+0000 7fb461029700 1 Processor -- start 2023-09-30T22:36:37.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.807+0000 7fb461029700 1 -- start start 2023-09-30T22:36:37.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.808+0000 7fb461029700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb45c09d630 0x7fb45c1208f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:37.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.808+0000 7fb461029700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb45c0a0a30 0x7fb45c120e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:37.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.808+0000 7fb461029700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb45c138d80 0x7fb45c121370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:37.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.808+0000 7fb461029700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb45c123700 con 0x7fb45c0a0a30 2023-09-30T22:36:37.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.808+0000 7fb461029700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb45c0a0410 con 0x7fb45c138d80 2023-09-30T22:36:37.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.808+0000 7fb461029700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb45c1224b0 con 0x7fb45c09d630 2023-09-30T22:36:37.812 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.808+0000 7fb45a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb45c0a0a30 0x7fb45c120e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:37.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.808+0000 7fb45a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb45c0a0a30 0x7fb45c120e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36288/0 (socket says 172.21.15.16:36288) 2023-09-30T22:36:37.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.808+0000 7fb45a59c700 1 -- 172.21.15.16:0/3535376309 learned_addr learned my addr 172.21.15.16:0/3535376309 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:37.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.809+0000 7fb45b59e700 1 --2- 172.21.15.16:0/3535376309 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb45c138d80 0x7fb45c121370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:37.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.809+0000 7fb45ad9d700 1 --2- 172.21.15.16:0/3535376309 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb45c09d630 0x7fb45c1208f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:37.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.809+0000 7fb45a59c700 1 -- 172.21.15.16:0/3535376309 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb45c09d630 msgr2=0x7fb45c1208f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:37.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.809+0000 7fb45a59c700 1 --2- 172.21.15.16:0/3535376309 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb45c09d630 0x7fb45c1208f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:37.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.809+0000 7fb45a59c700 1 -- 172.21.15.16:0/3535376309 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb45c138d80 msgr2=0x7fb45c121370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:37.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.809+0000 7fb45a59c700 1 --2- 172.21.15.16:0/3535376309 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb45c138d80 0x7fb45c121370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:37.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.809+0000 7fb45a59c700 1 -- 172.21.15.16:0/3535376309 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb454009a70 con 0x7fb45c0a0a30 2023-09-30T22:36:37.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.809+0000 7fb45a59c700 1 --2- 172.21.15.16:0/3535376309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb45c0a0a30 0x7fb45c120e30 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7fb45000c4e0 tx=0x7fb45000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:37.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.809+0000 7fb4437fe700 1 -- 172.21.15.16:0/3535376309 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4500052d0 con 0x7fb45c0a0a30 2023-09-30T22:36:37.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.809+0000 7fb4437fe700 1 -- 172.21.15.16:0/3535376309 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb450005430 con 0x7fb45c0a0a30 2023-09-30T22:36:37.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.809+0000 7fb4437fe700 1 -- 172.21.15.16:0/3535376309 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4500038c0 con 0x7fb45c0a0a30 2023-09-30T22:36:37.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.809+0000 7fb461029700 1 -- 172.21.15.16:0/3535376309 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb44c009550 con 0x7fb45c0a0a30 2023-09-30T22:36:37.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.809+0000 7fb461029700 1 -- 172.21.15.16:0/3535376309 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb45c123b50 con 0x7fb45c0a0a30 2023-09-30T22:36:37.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.811+0000 7fb4437fe700 1 -- 172.21.15.16:0/3535376309 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb450003a20 con 0x7fb45c0a0a30 2023-09-30T22:36:37.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.811+0000 7fb461029700 1 -- 172.21.15.16:0/3535376309 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb45c0747f0 con 0x7fb45c0a0a30 2023-09-30T22:36:37.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.811+0000 7fb4437fe700 1 -- 172.21.15.16:0/3535376309 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fb45003ec20 con 0x7fb45c0a0a30 2023-09-30T22:36:37.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.816+0000 7fb4437fe700 1 -- 172.21.15.16:0/3535376309 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb45000e280 con 0x7fb45c0a0a30 2023-09-30T22:36:37.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.933+0000 7f22c0624700 1 -- 172.21.15.16:0/3295163001 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f22b8074ea0 con 0x7f22b806fb20 2023-09-30T22:36:37.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.933+0000 7f22aeffd700 1 -- 172.21.15.16:0/3295163001 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f22b40239c0 con 0x7f22b806fb20 2023-09-30T22:36:37.935 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:36:37.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.934+0000 7f1c29e7d700 1 -- 172.21.15.16:0/1503124734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1c2409a3d0 con 0x7f1c2413d1f0 2023-09-30T22:36:37.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.934+0000 7f1c0bfff700 1 -- 172.21.15.16:0/1503124734 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1c1c02a020 con 0x7f1c2413d1f0 2023-09-30T22:36:37.935 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:36:37.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.935+0000 7f22c0624700 1 -- 172.21.15.16:0/3295163001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22b806fb20 msgr2=0x7f22b81c4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:37.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.935+0000 7f22c0624700 1 --2- 172.21.15.16:0/3295163001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22b806fb20 0x7f22b81c4020 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f22b4016000 tx=0x7f22b4004750 comp rx=0 tx=0).stop 2023-09-30T22:36:37.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.935+0000 7f22c0624700 1 -- 172.21.15.16:0/3295163001 shutdown_connections 2023-09-30T22:36:37.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.935+0000 7f22c0624700 1 --2- 172.21.15.16:0/3295163001 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f22b8130e50 0x7f22b8118cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:37.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.935+0000 7f22c0624700 1 --2- 172.21.15.16:0/3295163001 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f22b806fb20 0x7f22b81c4020 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:37.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.935+0000 7f22c0624700 1 --2- 172.21.15.16:0/3295163001 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f22b8130540 0x7f22b81c4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:37.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.935+0000 7f22c0624700 1 -- 172.21.15.16:0/3295163001 >> 172.21.15.16:0/3295163001 conn(0x7f22b80a2aa0 msgr2=0x7f22b812d340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:37.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.935+0000 7f22c0624700 1 -- 172.21.15.16:0/3295163001 shutdown_connections 2023-09-30T22:36:37.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.935+0000 7f22c0624700 1 -- 172.21.15.16:0/3295163001 wait complete. 2023-09-30T22:36:37.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.936+0000 7f1c29e7d700 1 -- 172.21.15.16:0/1503124734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c2413d1f0 msgr2=0x7f1c2413bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:37.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.936+0000 7f1c29e7d700 1 --2- 172.21.15.16:0/1503124734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c2413d1f0 0x7f1c2413bae0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c00efc0 tx=0x7f1c1c00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:36:37.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.937+0000 7f1c29e7d700 1 -- 172.21.15.16:0/1503124734 shutdown_connections 2023-09-30T22:36:37.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.937+0000 7f1c29e7d700 1 --2- 172.21.15.16:0/1503124734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1c2413d1f0 0x7f1c2413bae0 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:37.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.937+0000 7f1c29e7d700 1 --2- 172.21.15.16:0/1503124734 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1c2406a770 0x7f1c240716e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:37.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.937+0000 7f1c29e7d700 1 --2- 172.21.15.16:0/1503124734 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1c2409de00 0x7f1c2413b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:37.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.937+0000 7f1c29e7d700 1 -- 172.21.15.16:0/1503124734 >> 172.21.15.16:0/1503124734 conn(0x7f1c24066f50 msgr2=0x7f1c2409b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:37.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.937+0000 7f1c29e7d700 1 -- 172.21.15.16:0/1503124734 shutdown_connections 2023-09-30T22:36:37.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.937+0000 7f1c29e7d700 1 -- 172.21.15.16:0/1503124734 wait complete. 2023-09-30T22:36:37.951 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:36:37.959 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:36:37.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.996+0000 7fbc0c891700 1 -- 172.21.15.16:0/1609231119 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fbbec005590 con 0x7fbc0809d6f0 2023-09-30T22:36:37.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.996+0000 7fbbf6ffd700 1 -- 172.21.15.16:0/1609231119 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbbf8011360 con 0x7fbc0809d6f0 2023-09-30T22:36:37.998 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:36:38.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.999+0000 7fbc0c891700 1 -- 172.21.15.16:0/1609231119 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbc0809d6f0 msgr2=0x7fbc08120770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:38.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.999+0000 7fbc0c891700 1 --2- 172.21.15.16:0/1609231119 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbc0809d6f0 0x7fbc08120770 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fbbf800c520 tx=0x7fbbf800c8e0 comp rx=0 tx=0).stop 2023-09-30T22:36:38.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.999+0000 7fbc0c891700 1 -- 172.21.15.16:0/1609231119 shutdown_connections 2023-09-30T22:36:38.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.999+0000 7fbc0c891700 1 --2- 172.21.15.16:0/1609231119 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fbc0809d6f0 0x7fbc08120770 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:38.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.999+0000 7fbc0c891700 1 --2- 172.21.15.16:0/1609231119 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fbc08139170 0x7fbc081211f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:38.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.999+0000 7fbc0c891700 1 --2- 172.21.15.16:0/1609231119 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbc0809e1d0 0x7fbc08120cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:38.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.999+0000 7fbc0c891700 1 -- 172.21.15.16:0/1609231119 >> 172.21.15.16:0/1609231119 conn(0x7fbc08066f50 msgr2=0x7fbc08136b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:38.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.999+0000 7fbc0c891700 1 -- 172.21.15.16:0/1609231119 shutdown_connections 2023-09-30T22:36:38.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:37.999+0000 7fbc0c891700 1 -- 172.21.15.16:0/1609231119 wait complete. 2023-09-30T22:36:38.014 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:36:38.027 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:38.026+0000 7fb461029700 1 -- 172.21.15.16:0/3535376309 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb45c124040 con 0x7fb45c0a0a30 2023-09-30T22:36:38.028 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:38.027+0000 7fb4437fe700 1 -- 172.21.15.16:0/3535376309 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb450017070 con 0x7fb45c0a0a30 2023-09-30T22:36:38.028 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:36:38.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:38.029+0000 7fb461029700 1 -- 172.21.15.16:0/3535376309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb45c0a0a30 msgr2=0x7fb45c120e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:38.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:38.029+0000 7fb461029700 1 --2- 172.21.15.16:0/3535376309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb45c0a0a30 0x7fb45c120e30 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7fb45000c4e0 tx=0x7fb45000c8a0 comp rx=0 tx=0).stop 2023-09-30T22:36:38.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:38.029+0000 7fb461029700 1 -- 172.21.15.16:0/3535376309 shutdown_connections 2023-09-30T22:36:38.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:38.029+0000 7fb461029700 1 --2- 172.21.15.16:0/3535376309 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb45c138d80 0x7fb45c121370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:38.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:38.029+0000 7fb461029700 1 --2- 172.21.15.16:0/3535376309 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb45c09d630 0x7fb45c1208f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:38.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:38.029+0000 7fb461029700 1 --2- 172.21.15.16:0/3535376309 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb45c0a0a30 0x7fb45c120e30 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:38.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:38.029+0000 7fb461029700 1 -- 172.21.15.16:0/3535376309 >> 172.21.15.16:0/3535376309 conn(0x7fb45c0a3050 msgr2=0x7fb45c09ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:38.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:38.029+0000 7fb461029700 1 -- 172.21.15.16:0/3535376309 shutdown_connections 2023-09-30T22:36:38.031 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:38.029+0000 7fb461029700 1 -- 172.21.15.16:0/3535376309 wait complete. 2023-09-30T22:36:38.044 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:36:38.953 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:36:38.959 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:36:39.014 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:36:39.045 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:36:39.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.104+0000 7f645228c700 1 Processor -- start 2023-09-30T22:36:39.106 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.105+0000 7f645228c700 1 -- start start 2023-09-30T22:36:39.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.105+0000 7f645228c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f644c09d6f0 0x7f644c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:39.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.105+0000 7f645228c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f644c09e410 0x7f644c134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:39.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.105+0000 7f645228c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f644c09b120 con 0x7f644c09e000 2023-09-30T22:36:39.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.105+0000 7f645228c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f644c06b460 con 0x7f644c09d6f0 2023-09-30T22:36:39.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.105+0000 7f645228c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f644c09a7e0 con 0x7f644c09e410 2023-09-30T22:36:39.108 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.106+0000 7f644bfff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f644c09d6f0 0x7f644c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:39.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.106+0000 7f6450829700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f644c09e410 0x7f644c134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:39.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.106+0000 7f644bfff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f644c09d6f0 0x7f644c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:59210/0 (socket says 172.21.15.16:59210) 2023-09-30T22:36:39.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.106+0000 7f644bfff700 1 -- 172.21.15.16:0/3971545842 learned_addr learned my addr 172.21.15.16:0/3971545842 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:39.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.106+0000 7f644bfff700 1 -- 172.21.15.16:0/3971545842 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f644c09e410 msgr2=0x7f644c134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:39.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.106+0000 7f644bfff700 1 --2- 172.21.15.16:0/3971545842 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f644c09e410 0x7f644c134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:39.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.106+0000 7f644bfff700 1 -- 172.21.15.16:0/3971545842 >> v1:172.21.15.16:6789/0 conn(0x7f644c09e000 legacy=0x7f644c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:39.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.106+0000 7f644bfff700 1 -- 172.21.15.16:0/3971545842 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f644c09a500 con 0x7f644c09d6f0 2023-09-30T22:36:39.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.106+0000 7f6450829700 1 --2- 172.21.15.16:0/3971545842 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f644c09e410 0x7f644c134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:39.109 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.106+0000 7f644bfff700 1 --2- 172.21.15.16:0/3971545842 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f644c09d6f0 0x7f644c09dac0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f643c009920 tx=0x7f643c009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f18def6d7498c02 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:39.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.107+0000 7f644affd700 1 -- 172.21.15.16:0/3971545842 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643c011830 con 0x7f644c09d6f0 2023-09-30T22:36:39.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.107+0000 7f644affd700 1 -- 172.21.15.16:0/3971545842 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f643c011990 con 0x7f644c09d6f0 2023-09-30T22:36:39.110 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.107+0000 7f644affd700 1 -- 172.21.15.16:0/3971545842 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643c011d20 con 0x7f644c09d6f0 2023-09-30T22:36:39.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.109+0000 7f645228c700 1 -- 172.21.15.16:0/3971545842 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f644c09d6f0 msgr2=0x7f644c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:39.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.109+0000 7f645228c700 1 --2- 172.21.15.16:0/3971545842 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f644c09d6f0 0x7f644c09dac0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f643c009920 tx=0x7f643c009c30 comp rx=0 tx=0).stop 2023-09-30T22:36:39.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.109+0000 7f645228c700 1 -- 172.21.15.16:0/3971545842 shutdown_connections 2023-09-30T22:36:39.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.109+0000 7f645228c700 1 --2- 172.21.15.16:0/3971545842 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f644c09e410 0x7f644c134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:39.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.109+0000 7f645228c700 1 --2- 172.21.15.16:0/3971545842 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f644c09d6f0 0x7f644c09dac0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:39.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.109+0000 7f645228c700 1 -- 172.21.15.16:0/3971545842 >> 172.21.15.16:0/3971545842 conn(0x7f644c066f50 msgr2=0x7f644c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:39.111 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.109+0000 7f645228c700 1 -- 172.21.15.16:0/3971545842 shutdown_connections 2023-09-30T22:36:39.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.109+0000 7f645228c700 1 -- 172.21.15.16:0/3971545842 wait complete. 2023-09-30T22:36:39.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.110+0000 7f645228c700 1 Processor -- start 2023-09-30T22:36:39.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.110+0000 7f645228c700 1 -- start start 2023-09-30T22:36:39.112 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.111+0000 7f645228c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f644c09d6f0 0x7f644c1225c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:39.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.111+0000 7f645228c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f644c09e000 0x7f644c122b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:39.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.111+0000 7f645228c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f644c09e410 0x7f644c123040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:39.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.111+0000 7f645228c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f644c125ab0 con 0x7f644c09e410 2023-09-30T22:36:39.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.111+0000 7f645228c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f644c1249a0 con 0x7f644c09d6f0 2023-09-30T22:36:39.113 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.111+0000 7f645228c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f644c126050 con 0x7f644c09e000 2023-09-30T22:36:39.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.111+0000 7f6450829700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f644c09e410 0x7f644c123040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:39.114 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.111+0000 7f6450829700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f644c09e410 0x7f644c123040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36300/0 (socket says 172.21.15.16:36300) 2023-09-30T22:36:39.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.111+0000 7f6450829700 1 -- 172.21.15.16:0/2770121978 learned_addr learned my addr 172.21.15.16:0/2770121978 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:39.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.111+0000 7f644b7fe700 1 --2- 172.21.15.16:0/2770121978 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f644c09e000 0x7f644c122b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:39.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.112+0000 7f6450829700 1 -- 172.21.15.16:0/2770121978 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f644c09e000 msgr2=0x7f644c122b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:39.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.112+0000 7f6450829700 1 --2- 172.21.15.16:0/2770121978 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f644c09e000 0x7f644c122b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:39.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.112+0000 7f6450829700 1 -- 172.21.15.16:0/2770121978 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f644c09d6f0 msgr2=0x7f644c1225c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:39.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.112+0000 7f6450829700 1 --2- 172.21.15.16:0/2770121978 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f644c09d6f0 0x7f644c1225c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:39.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.112+0000 7f6450829700 1 -- 172.21.15.16:0/2770121978 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6444009a70 con 0x7f644c09e410 2023-09-30T22:36:39.115 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.112+0000 7f644b7fe700 1 --2- 172.21.15.16:0/2770121978 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f644c09e000 0x7f644c122b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:39.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.112+0000 7f6450829700 1 --2- 172.21.15.16:0/2770121978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f644c09e410 0x7f644c123040 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f644400efc0 tx=0x7f644400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:39.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.112+0000 7f6448ff9700 1 -- 172.21.15.16:0/2770121978 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644400cb40 con 0x7f644c09e410 2023-09-30T22:36:39.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.112+0000 7f6448ff9700 1 -- 172.21.15.16:0/2770121978 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f644400f040 con 0x7f644c09e410 2023-09-30T22:36:39.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.112+0000 7f645228c700 1 -- 172.21.15.16:0/2770121978 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f643c009550 con 0x7f644c09e410 2023-09-30T22:36:39.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.112+0000 7f6448ff9700 1 -- 172.21.15.16:0/2770121978 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6444003a20 con 0x7f644c09e410 2023-09-30T22:36:39.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.112+0000 7f645228c700 1 -- 172.21.15.16:0/2770121978 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f644c09c710 con 0x7f644c09e410 2023-09-30T22:36:39.116 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.113+0000 7f6448ff9700 1 -- 172.21.15.16:0/2770121978 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6444014480 con 0x7f644c09e410 2023-09-30T22:36:39.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.114+0000 7f6448ff9700 1 -- 172.21.15.16:0/2770121978 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f6444040090 con 0x7f644c09e410 2023-09-30T22:36:39.117 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.114+0000 7f645228c700 1 -- 172.21.15.16:0/2770121978 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f644c074ea0 con 0x7f644c09e410 2023-09-30T22:36:39.121 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.120+0000 7f6448ff9700 1 -- 172.21.15.16:0/2770121978 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6444012930 con 0x7f644c09e410 2023-09-30T22:36:39.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.146+0000 7f346deb3700 1 Processor -- start 2023-09-30T22:36:39.147 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.146+0000 7f346deb3700 1 -- start start 2023-09-30T22:36:39.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.146+0000 7f346deb3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3468131810 0x7f3468131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:39.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.146+0000 7f346deb3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f346809a580 0x7f3468139660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:39.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.146+0000 7f346deb3700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34681313c0 con 0x7f3468099a40 2023-09-30T22:36:39.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.146+0000 7f346deb3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f346806e8c0 con 0x7f3468131810 2023-09-30T22:36:39.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.146+0000 7f346deb3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3468071d20 con 0x7f346809a580 2023-09-30T22:36:39.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.147+0000 7f34677fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3468131810 0x7f3468131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:39.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.147+0000 7f34677fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3468131810 0x7f3468131be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:59216/0 (socket says 172.21.15.16:59216) 2023-09-30T22:36:39.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.147+0000 7f34677fe700 1 -- 172.21.15.16:0/3170685821 learned_addr learned my addr 172.21.15.16:0/3170685821 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:39.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.147+0000 7f3467fff700 1 --2- 172.21.15.16:0/3170685821 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f346809a580 0x7f3468139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:39.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.147+0000 7f34677fe700 1 -- 172.21.15.16:0/3170685821 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f346809a580 msgr2=0x7f3468139660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:39.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.147+0000 7f34677fe700 1 --2- 172.21.15.16:0/3170685821 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f346809a580 0x7f3468139660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:39.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.147+0000 7f34677fe700 1 -- 172.21.15.16:0/3170685821 >> v1:172.21.15.16:6789/0 conn(0x7f3468099a40 legacy=0x7f3468099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:39.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.147+0000 7f34677fe700 1 -- 172.21.15.16:0/3170685821 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34680721a0 con 0x7f3468131810 2023-09-30T22:36:39.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.147+0000 7f34677fe700 1 --2- 172.21.15.16:0/3170685821 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3468131810 0x7f3468131be0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f3458009920 tx=0x7f3458009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bab6b3cb76d3e809 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:39.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.148+0000 7f34667fc700 1 -- 172.21.15.16:0/3170685821 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345800ba60 con 0x7f3468131810 2023-09-30T22:36:39.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.148+0000 7f34667fc700 1 -- 172.21.15.16:0/3170685821 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f345800bbc0 con 0x7f3468131810 2023-09-30T22:36:39.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.148+0000 7f34667fc700 1 -- 172.21.15.16:0/3170685821 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3458009e70 con 0x7f3468131810 2023-09-30T22:36:39.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.148+0000 7f346deb3700 1 -- 172.21.15.16:0/3170685821 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3468131810 msgr2=0x7f3468131be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:39.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.148+0000 7f346deb3700 1 --2- 172.21.15.16:0/3170685821 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3468131810 0x7f3468131be0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f3458009920 tx=0x7f3458009c30 comp rx=0 tx=0).stop 2023-09-30T22:36:39.151 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.148+0000 7f346deb3700 1 -- 172.21.15.16:0/3170685821 shutdown_connections 2023-09-30T22:36:39.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.148+0000 7f346deb3700 1 --2- 172.21.15.16:0/3170685821 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f346809a580 0x7f3468139660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:39.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.148+0000 7f346deb3700 1 --2- 172.21.15.16:0/3170685821 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3468131810 0x7f3468131be0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:39.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.148+0000 7f346deb3700 1 -- 172.21.15.16:0/3170685821 >> 172.21.15.16:0/3170685821 conn(0x7f34680a2aa0 msgr2=0x7f34680a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:39.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.149+0000 7f346deb3700 1 -- 172.21.15.16:0/3170685821 shutdown_connections 2023-09-30T22:36:39.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.149+0000 7f346deb3700 1 -- 172.21.15.16:0/3170685821 wait complete. 2023-09-30T22:36:39.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.150+0000 7f346deb3700 1 Processor -- start 2023-09-30T22:36:39.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.150+0000 7f346deb3700 1 -- start start 2023-09-30T22:36:39.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.150+0000 7f346deb3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3468099a40 0x7f346811b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:39.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.150+0000 7f346deb3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f346809a580 0x7f346811bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:39.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.150+0000 7f346deb3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3468131810 0x7f346811c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:39.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.150+0000 7f346deb3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f346811f1f0 con 0x7f3468131810 2023-09-30T22:36:39.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.150+0000 7f346deb3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f346811d900 con 0x7f3468099a40 2023-09-30T22:36:39.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.150+0000 7f346deb3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f346811e5c0 con 0x7f346809a580 2023-09-30T22:36:39.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.151+0000 7f34677fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3468099a40 0x7f346811b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:39.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.151+0000 7f34677fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3468099a40 0x7f346811b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:59220/0 (socket says 172.21.15.16:59220) 2023-09-30T22:36:39.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.151+0000 7f34677fe700 1 -- 172.21.15.16:0/191367615 learned_addr learned my addr 172.21.15.16:0/191367615 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:39.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.151+0000 7f3467fff700 1 --2- 172.21.15.16:0/191367615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3468131810 0x7f346811c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:39.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.151+0000 7f34677fe700 1 -- 172.21.15.16:0/191367615 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f346809a580 msgr2=0x7f346811bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:39.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.151+0000 7f34677fe700 1 --2- 172.21.15.16:0/191367615 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f346809a580 0x7f346811bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:39.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.151+0000 7f34677fe700 1 -- 172.21.15.16:0/191367615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3468131810 msgr2=0x7f346811c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:39.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.151+0000 7f34677fe700 1 --2- 172.21.15.16:0/191367615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3468131810 0x7f346811c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:39.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.151+0000 7f34677fe700 1 -- 172.21.15.16:0/191367615 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3458009550 con 0x7f3468099a40 2023-09-30T22:36:39.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.152+0000 7f34677fe700 1 --2- 172.21.15.16:0/191367615 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3468099a40 0x7f346811b6d0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f3458000c00 tx=0x7f3458004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:39.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.152+0000 7f344ffff700 1 -- 172.21.15.16:0/191367615 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3458004450 con 0x7f3468099a40 2023-09-30T22:36:39.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.152+0000 7f344ffff700 1 -- 172.21.15.16:0/191367615 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34580045b0 con 0x7f3468099a40 2023-09-30T22:36:39.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.152+0000 7f346deb3700 1 -- 172.21.15.16:0/191367615 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34680a1e30 con 0x7f3468099a40 2023-09-30T22:36:39.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.152+0000 7f344ffff700 1 -- 172.21.15.16:0/191367615 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34580038e0 con 0x7f3468099a40 2023-09-30T22:36:39.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.152+0000 7f346deb3700 1 -- 172.21.15.16:0/191367615 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f346806eb00 con 0x7f3468099a40 2023-09-30T22:36:39.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.153+0000 7f346deb3700 1 -- 172.21.15.16:0/191367615 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f346811ebb0 con 0x7f3468099a40 2023-09-30T22:36:39.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.154+0000 7f344ffff700 1 -- 172.21.15.16:0/191367615 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3458003a40 con 0x7f3468099a40 2023-09-30T22:36:39.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.154+0000 7f344ffff700 1 -- 172.21.15.16:0/191367615 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f34580409e0 con 0x7f3468099a40 2023-09-30T22:36:39.164 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.163+0000 7f344ffff700 1 -- 172.21.15.16:0/191367615 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3458003cf0 con 0x7f3468099a40 2023-09-30T22:36:39.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.188+0000 7ff83738a700 1 Processor -- start 2023-09-30T22:36:39.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.188+0000 7ff83738a700 1 -- start start 2023-09-30T22:36:39.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.189+0000 7ff83738a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff83006a770 0x7ff83009d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:39.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.189+0000 7ff83738a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff83013d1f0 0x7ff83013f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:39.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.189+0000 7ff83738a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff83009a810 con 0x7ff83009dc30 2023-09-30T22:36:39.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.189+0000 7ff83738a700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff83006ab40 con 0x7ff83013d1f0 2023-09-30T22:36:39.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.189+0000 7ff83738a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff83009ebc0 con 0x7ff83006a770 2023-09-30T22:36:39.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.189+0000 7ff834925700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ff83009dc30 0x7ff83009e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60064/0 (socket says 172.21.15.16:60064) 2023-09-30T22:36:39.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.189+0000 7ff834925700 1 -- 172.21.15.16:0/1582059992 learned_addr learned my addr 172.21.15.16:0/1582059992 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:36:39.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.189+0000 7ff835927700 1 --2- 172.21.15.16:0/1582059992 >> v2:172.21.15.16:3300/0 conn(0x7ff83013d1f0 0x7ff83013f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:39.191 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.189+0000 7ff835126700 1 --2- 172.21.15.16:0/1582059992 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff83006a770 0x7ff83009d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:39.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.190+0000 7ff835927700 1 -- 172.21.15.16:0/1582059992 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff83006a770 msgr2=0x7ff83009d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:39.192 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.190+0000 7ff835927700 1 --2- 172.21.15.16:0/1582059992 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff83006a770 0x7ff83009d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:39.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.190+0000 7ff835927700 1 -- 172.21.15.16:0/1582059992 >> v1:172.21.15.16:6789/0 conn(0x7ff83009dc30 legacy=0x7ff83009e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:39.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.190+0000 7ff835927700 1 -- 172.21.15.16:0/1582059992 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff83009e920 con 0x7ff83013d1f0 2023-09-30T22:36:39.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.190+0000 7ff835927700 1 --2- 172.21.15.16:0/1582059992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff83013d1f0 0x7ff83013f5d0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7ff828009dc0 tx=0x7ff82800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=780aab3d28b7bba8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:39.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.190+0000 7ff827fff700 1 -- 172.21.15.16:0/1582059992 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82800e070 con 0x7ff83013d1f0 2023-09-30T22:36:39.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.190+0000 7ff827fff700 1 -- 172.21.15.16:0/1582059992 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff82800bae0 con 0x7ff83013d1f0 2023-09-30T22:36:39.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.190+0000 7ff827fff700 1 -- 172.21.15.16:0/1582059992 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82800be90 con 0x7ff83013d1f0 2023-09-30T22:36:39.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.191+0000 7ff83738a700 1 -- 172.21.15.16:0/1582059992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff83013d1f0 msgr2=0x7ff83013f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:39.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.191+0000 7ff83738a700 1 --2- 172.21.15.16:0/1582059992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff83013d1f0 0x7ff83013f5d0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7ff828009dc0 tx=0x7ff82800b330 comp rx=0 tx=0).stop 2023-09-30T22:36:39.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.191+0000 7ff83738a700 1 -- 172.21.15.16:0/1582059992 shutdown_connections 2023-09-30T22:36:39.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.191+0000 7ff83738a700 1 --2- 172.21.15.16:0/1582059992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff83013d1f0 0x7ff83013f5d0 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:39.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.191+0000 7ff83738a700 1 --2- 172.21.15.16:0/1582059992 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff83006a770 0x7ff83009d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:39.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.191+0000 7ff83738a700 1 -- 172.21.15.16:0/1582059992 >> 172.21.15.16:0/1582059992 conn(0x7ff830066f50 msgr2=0x7ff83009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:39.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.191+0000 7ff83738a700 1 -- 172.21.15.16:0/1582059992 shutdown_connections 2023-09-30T22:36:39.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.191+0000 7ff83738a700 1 -- 172.21.15.16:0/1582059992 wait complete. 2023-09-30T22:36:39.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.192+0000 7ff83738a700 1 Processor -- start 2023-09-30T22:36:39.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.192+0000 7ff83738a700 1 -- start start 2023-09-30T22:36:39.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.192+0000 7ff83738a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff83006a770 0x7ff830071770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:39.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.192+0000 7ff83738a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff83009dc30 0x7ff83013b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:39.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.193+0000 7ff83738a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff83013d1f0 0x7ff83013bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:39.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.193+0000 7ff83738a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff83009f1a0 con 0x7ff83009dc30 2023-09-30T22:36:39.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.193+0000 7ff83738a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff83011f040 con 0x7ff83006a770 2023-09-30T22:36:39.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.193+0000 7ff83738a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff83011e380 con 0x7ff83013d1f0 2023-09-30T22:36:39.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.193+0000 7ff834925700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff83009dc30 0x7ff83013b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:39.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.193+0000 7ff835927700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff83013d1f0 0x7ff83013bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:39.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.193+0000 7ff834925700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff83009dc30 0x7ff83013b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36332/0 (socket says 172.21.15.16:36332) 2023-09-30T22:36:39.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.193+0000 7ff834925700 1 -- 172.21.15.16:0/1021817927 learned_addr learned my addr 172.21.15.16:0/1021817927 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:39.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.193+0000 7ff835126700 1 --2- 172.21.15.16:0/1021817927 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff83006a770 0x7ff830071770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:39.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.193+0000 7ff834925700 1 -- 172.21.15.16:0/1021817927 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff83013d1f0 msgr2=0x7ff83013bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:39.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.193+0000 7ff834925700 1 --2- 172.21.15.16:0/1021817927 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff83013d1f0 0x7ff83013bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:39.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.193+0000 7ff834925700 1 -- 172.21.15.16:0/1021817927 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff83006a770 msgr2=0x7ff830071770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:39.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.193+0000 7ff834925700 1 --2- 172.21.15.16:0/1021817927 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff83006a770 0x7ff830071770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:39.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.193+0000 7ff834925700 1 -- 172.21.15.16:0/1021817927 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff828009a70 con 0x7ff83009dc30 2023-09-30T22:36:39.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.194+0000 7ff834925700 1 --2- 172.21.15.16:0/1021817927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff83009dc30 0x7ff83013b6b0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7ff82000c500 tx=0x7ff82000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:39.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.194+0000 7ff825ffb700 1 -- 172.21.15.16:0/1021817927 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff820005d70 con 0x7ff83009dc30 2023-09-30T22:36:39.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.194+0000 7ff825ffb700 1 -- 172.21.15.16:0/1021817927 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8200070d0 con 0x7ff83009dc30 2023-09-30T22:36:39.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.194+0000 7ff83738a700 1 -- 172.21.15.16:0/1021817927 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff83011eaa0 con 0x7ff83009dc30 2023-09-30T22:36:39.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.194+0000 7ff825ffb700 1 -- 172.21.15.16:0/1021817927 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff820007440 con 0x7ff83009dc30 2023-09-30T22:36:39.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.194+0000 7ff83738a700 1 -- 172.21.15.16:0/1021817927 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff83011ed60 con 0x7ff83009dc30 2023-09-30T22:36:39.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.195+0000 7ff825ffb700 1 -- 172.21.15.16:0/1021817927 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff8200075a0 con 0x7ff83009dc30 2023-09-30T22:36:39.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.195+0000 7ff825ffb700 1 -- 172.21.15.16:0/1021817927 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7ff82003eb80 con 0x7ff83009dc30 2023-09-30T22:36:39.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.196+0000 7ff83738a700 1 -- 172.21.15.16:0/1021817927 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff830074ea0 con 0x7ff83009dc30 2023-09-30T22:36:39.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.205+0000 7ff825ffb700 1 -- 172.21.15.16:0/1021817927 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff82003ee60 con 0x7ff83009dc30 2023-09-30T22:36:39.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.229+0000 7f873d6d5700 1 Processor -- start 2023-09-30T22:36:39.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.229+0000 7f873d6d5700 1 -- start start 2023-09-30T22:36:39.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.230+0000 7f873d6d5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8738133420 0x7f87381337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:39.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.230+0000 7f873d6d5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8738133e30 0x7f873812ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:39.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.230+0000 7f873d6d5700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f873806b990 con 0x7f8738134260 2023-09-30T22:36:39.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.230+0000 7f873d6d5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87380a0590 con 0x7f8738133420 2023-09-30T22:36:39.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.230+0000 7f873d6d5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f873809fb80 con 0x7f8738133e30 2023-09-30T22:36:39.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.230+0000 7f8736ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8738133420 0x7f87381337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:39.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.230+0000 7f8736ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8738133420 0x7f87381337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:59264/0 (socket says 172.21.15.16:59264) 2023-09-30T22:36:39.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.230+0000 7f8736ffd700 1 -- 172.21.15.16:0/883265046 learned_addr learned my addr 172.21.15.16:0/883265046 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:39.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.231+0000 7f87367fc700 1 --2- 172.21.15.16:0/883265046 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8738133e30 0x7f873812ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:39.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.231+0000 7f8736ffd700 1 -- 172.21.15.16:0/883265046 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8738133e30 msgr2=0x7f873812ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:39.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.231+0000 7f8736ffd700 1 --2- 172.21.15.16:0/883265046 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8738133e30 0x7f873812ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:39.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.231+0000 7f8736ffd700 1 -- 172.21.15.16:0/883265046 >> v1:172.21.15.16:6789/0 conn(0x7f8738134260 legacy=0x7f87381304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:39.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.231+0000 7f8736ffd700 1 -- 172.21.15.16:0/883265046 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87380a0030 con 0x7f8738133420 2023-09-30T22:36:39.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.231+0000 7f8736ffd700 1 --2- 172.21.15.16:0/883265046 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8738133420 0x7f87381337f0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f8728009920 tx=0x7f8728009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc36f66227fac04a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:39.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.231+0000 7f8735ffb700 1 -- 172.21.15.16:0/883265046 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872800ba60 con 0x7f8738133420 2023-09-30T22:36:39.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.231+0000 7f8735ffb700 1 -- 172.21.15.16:0/883265046 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f872800bbc0 con 0x7f8738133420 2023-09-30T22:36:39.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.231+0000 7f8735ffb700 1 -- 172.21.15.16:0/883265046 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8728009e70 con 0x7f8738133420 2023-09-30T22:36:39.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.232+0000 7f873d6d5700 1 -- 172.21.15.16:0/883265046 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8738133420 msgr2=0x7f87381337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:39.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.232+0000 7f873d6d5700 1 --2- 172.21.15.16:0/883265046 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8738133420 0x7f87381337f0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f8728009920 tx=0x7f8728009c30 comp rx=0 tx=0).stop 2023-09-30T22:36:39.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.232+0000 7f873d6d5700 1 -- 172.21.15.16:0/883265046 shutdown_connections 2023-09-30T22:36:39.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.232+0000 7f873d6d5700 1 --2- 172.21.15.16:0/883265046 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8738133420 0x7f87381337f0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:39.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.232+0000 7f873d6d5700 1 --2- 172.21.15.16:0/883265046 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8738133e30 0x7f873812ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:39.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.232+0000 7f873d6d5700 1 -- 172.21.15.16:0/883265046 >> 172.21.15.16:0/883265046 conn(0x7f87380a4210 msgr2=0x7f87380a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:39.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.232+0000 7f873d6d5700 1 -- 172.21.15.16:0/883265046 shutdown_connections 2023-09-30T22:36:39.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.232+0000 7f873d6d5700 1 -- 172.21.15.16:0/883265046 wait complete. 2023-09-30T22:36:39.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.233+0000 7f873d6d5700 1 Processor -- start 2023-09-30T22:36:39.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.233+0000 7f873d6d5700 1 -- start start 2023-09-30T22:36:39.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.234+0000 7f873d6d5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8738133420 0x7f873809bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:39.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.234+0000 7f873d6d5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8738133e30 0x7f873809c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:39.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.234+0000 7f873d6d5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8738134260 0x7f873809c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:39.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.234+0000 7f873d6d5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873811aa00 con 0x7f8738133420 2023-09-30T22:36:39.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.234+0000 7f873d6d5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f873809d390 con 0x7f8738134260 2023-09-30T22:36:39.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.234+0000 7f873d6d5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f873811ad60 con 0x7f8738133e30 2023-09-30T22:36:39.237 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.234+0000 7f8736ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8738133420 0x7f873809bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:39.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.234+0000 7f8736ffd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8738133420 0x7f873809bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36340/0 (socket says 172.21.15.16:36340) 2023-09-30T22:36:39.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.234+0000 7f8736ffd700 1 -- 172.21.15.16:0/188906265 learned_addr learned my addr 172.21.15.16:0/188906265 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:39.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.234+0000 7f87377fe700 1 --2- 172.21.15.16:0/188906265 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8738134260 0x7f873809c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:39.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.234+0000 7f87367fc700 1 --2- 172.21.15.16:0/188906265 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8738133e30 0x7f873809c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:39.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.235+0000 7f8736ffd700 1 -- 172.21.15.16:0/188906265 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8738133e30 msgr2=0x7f873809c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:39.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.235+0000 7f8736ffd700 1 --2- 172.21.15.16:0/188906265 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8738133e30 0x7f873809c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:39.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.235+0000 7f8736ffd700 1 -- 172.21.15.16:0/188906265 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8738134260 msgr2=0x7f873809c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:39.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.235+0000 7f8736ffd700 1 --2- 172.21.15.16:0/188906265 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8738134260 0x7f873809c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:39.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.235+0000 7f8736ffd700 1 -- 172.21.15.16:0/188906265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8728009550 con 0x7f8738133420 2023-09-30T22:36:39.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.235+0000 7f87377fe700 1 --2- 172.21.15.16:0/188906265 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8738134260 0x7f873809c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:39.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.235+0000 7f8736ffd700 1 --2- 172.21.15.16:0/188906265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8738133420 0x7f873809bee0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f8728009130 tx=0x7f8728004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:39.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.235+0000 7f871f7fe700 1 -- 172.21.15.16:0/188906265 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8728004300 con 0x7f8738133420 2023-09-30T22:36:39.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.235+0000 7f871f7fe700 1 -- 172.21.15.16:0/188906265 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f872801f070 con 0x7f8738133420 2023-09-30T22:36:39.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.235+0000 7f873d6d5700 1 -- 172.21.15.16:0/188906265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f873809dbe0 con 0x7f8738133420 2023-09-30T22:36:39.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.235+0000 7f871f7fe700 1 -- 172.21.15.16:0/188906265 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87280036a0 con 0x7f8738133420 2023-09-30T22:36:39.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.235+0000 7f873d6d5700 1 -- 172.21.15.16:0/188906265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f873812dbf0 con 0x7f8738133420 2023-09-30T22:36:39.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.236+0000 7f873d6d5700 1 -- 172.21.15.16:0/188906265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8738074ea0 con 0x7f8738133420 2023-09-30T22:36:39.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.237+0000 7f871f7fe700 1 -- 172.21.15.16:0/188906265 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8728021030 con 0x7f8738133420 2023-09-30T22:36:39.240 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.237+0000 7f871f7fe700 1 -- 172.21.15.16:0/188906265 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f87280115e0 con 0x7f8738133420 2023-09-30T22:36:39.243 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.242+0000 7f871f7fe700 1 -- 172.21.15.16:0/188906265 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f87280199a0 con 0x7f8738133420 2023-09-30T22:36:39.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.336+0000 7f645228c700 1 -- 172.21.15.16:0/2770121978 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f644c06c360 con 0x7f644c09e410 2023-09-30T22:36:39.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.336+0000 7f6448ff9700 1 -- 172.21.15.16:0/2770121978 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6444009d70 con 0x7f644c09e410 2023-09-30T22:36:39.338 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:36:39.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.339+0000 7f645228c700 1 -- 172.21.15.16:0/2770121978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f644c09e410 msgr2=0x7f644c123040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:39.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.339+0000 7f645228c700 1 --2- 172.21.15.16:0/2770121978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f644c09e410 0x7f644c123040 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f644400efc0 tx=0x7f644400c3a0 comp rx=0 tx=0).stop 2023-09-30T22:36:39.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.339+0000 7f645228c700 1 -- 172.21.15.16:0/2770121978 shutdown_connections 2023-09-30T22:36:39.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.339+0000 7f645228c700 1 --2- 172.21.15.16:0/2770121978 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f644c09d6f0 0x7f644c1225c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:39.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.339+0000 7f645228c700 1 --2- 172.21.15.16:0/2770121978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f644c09e410 0x7f644c123040 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:39.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.339+0000 7f645228c700 1 --2- 172.21.15.16:0/2770121978 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f644c09e000 0x7f644c122b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:39.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.339+0000 7f645228c700 1 -- 172.21.15.16:0/2770121978 >> 172.21.15.16:0/2770121978 conn(0x7f644c066f50 msgr2=0x7f644c0a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:39.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.339+0000 7f645228c700 1 -- 172.21.15.16:0/2770121978 shutdown_connections 2023-09-30T22:36:39.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.339+0000 7f645228c700 1 -- 172.21.15.16:0/2770121978 wait complete. 2023-09-30T22:36:39.354 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:36:39.377 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.376+0000 7f346deb3700 1 -- 172.21.15.16:0/191367615 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f346811ebb0 con 0x7f3468099a40 2023-09-30T22:36:39.378 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.377+0000 7f344ffff700 1 -- 172.21.15.16:0/191367615 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3458028070 con 0x7f3468099a40 2023-09-30T22:36:39.378 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:36:39.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.379+0000 7f346deb3700 1 -- 172.21.15.16:0/191367615 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3468099a40 msgr2=0x7f346811b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:39.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.379+0000 7f346deb3700 1 --2- 172.21.15.16:0/191367615 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3468099a40 0x7f346811b6d0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f3458000c00 tx=0x7f3458004750 comp rx=0 tx=0).stop 2023-09-30T22:36:39.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.379+0000 7f346deb3700 1 -- 172.21.15.16:0/191367615 shutdown_connections 2023-09-30T22:36:39.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.379+0000 7f346deb3700 1 --2- 172.21.15.16:0/191367615 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3468099a40 0x7f346811b6d0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:39.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.379+0000 7f346deb3700 1 --2- 172.21.15.16:0/191367615 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3468131810 0x7f346811c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:39.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.379+0000 7f346deb3700 1 --2- 172.21.15.16:0/191367615 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f346809a580 0x7f346811bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:39.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.379+0000 7f346deb3700 1 -- 172.21.15.16:0/191367615 >> 172.21.15.16:0/191367615 conn(0x7f34680a2aa0 msgr2=0x7f34681b8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:39.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.379+0000 7f346deb3700 1 -- 172.21.15.16:0/191367615 shutdown_connections 2023-09-30T22:36:39.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.379+0000 7f346deb3700 1 -- 172.21.15.16:0/191367615 wait complete. 2023-09-30T22:36:39.395 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:36:39.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.417+0000 7ff83738a700 1 -- 172.21.15.16:0/1021817927 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff83009a3d0 con 0x7ff83009dc30 2023-09-30T22:36:39.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.417+0000 7ff825ffb700 1 -- 172.21.15.16:0/1021817927 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff820017020 con 0x7ff83009dc30 2023-09-30T22:36:39.422 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:36:39.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.420+0000 7ff83738a700 1 -- 172.21.15.16:0/1021817927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff83009dc30 msgr2=0x7ff83013b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:39.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.420+0000 7ff83738a700 1 --2- 172.21.15.16:0/1021817927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff83009dc30 0x7ff83013b6b0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7ff82000c500 tx=0x7ff82000c810 comp rx=0 tx=0).stop 2023-09-30T22:36:39.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.420+0000 7ff83738a700 1 -- 172.21.15.16:0/1021817927 shutdown_connections 2023-09-30T22:36:39.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.420+0000 7ff83738a700 1 --2- 172.21.15.16:0/1021817927 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff83006a770 0x7ff830071770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:39.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.420+0000 7ff83738a700 1 --2- 172.21.15.16:0/1021817927 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff83013d1f0 0x7ff83013bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:39.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.420+0000 7ff83738a700 1 --2- 172.21.15.16:0/1021817927 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff83009dc30 0x7ff83013b6b0 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:39.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.420+0000 7ff83738a700 1 -- 172.21.15.16:0/1021817927 >> 172.21.15.16:0/1021817927 conn(0x7ff830066f50 msgr2=0x7ff83013d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:39.424 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.420+0000 7ff83738a700 1 -- 172.21.15.16:0/1021817927 shutdown_connections 2023-09-30T22:36:39.425 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.420+0000 7ff83738a700 1 -- 172.21.15.16:0/1021817927 wait complete. 2023-09-30T22:36:39.436 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:36:39.451 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.450+0000 7f873d6d5700 1 -- 172.21.15.16:0/188906265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f873811b430 con 0x7f8738133420 2023-09-30T22:36:39.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.450+0000 7f871f7fe700 1 -- 172.21.15.16:0/188906265 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f872800a0d0 con 0x7f8738133420 2023-09-30T22:36:39.452 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:36:39.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.453+0000 7f873d6d5700 1 -- 172.21.15.16:0/188906265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8738133420 msgr2=0x7f873809bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:39.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.453+0000 7f873d6d5700 1 --2- 172.21.15.16:0/188906265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8738133420 0x7f873809bee0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f8728009130 tx=0x7f8728004750 comp rx=0 tx=0).stop 2023-09-30T22:36:39.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.453+0000 7f873d6d5700 1 -- 172.21.15.16:0/188906265 shutdown_connections 2023-09-30T22:36:39.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.453+0000 7f873d6d5700 1 --2- 172.21.15.16:0/188906265 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8738134260 0x7f873809c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:39.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.453+0000 7f873d6d5700 1 --2- 172.21.15.16:0/188906265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8738133420 0x7f873809bee0 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:39.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.453+0000 7f873d6d5700 1 --2- 172.21.15.16:0/188906265 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8738133e30 0x7f873809c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:39.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.453+0000 7f873d6d5700 1 -- 172.21.15.16:0/188906265 >> 172.21.15.16:0/188906265 conn(0x7f87380a4210 msgr2=0x7f8738130150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:39.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.453+0000 7f873d6d5700 1 -- 172.21.15.16:0/188906265 shutdown_connections 2023-09-30T22:36:39.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:39.453+0000 7f873d6d5700 1 -- 172.21.15.16:0/188906265 wait complete. 2023-09-30T22:36:39.467 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:36:40.355 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:36:40.396 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:36:40.437 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:36:40.469 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:36:40.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.505+0000 7fc978281700 1 Processor -- start 2023-09-30T22:36:40.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.505+0000 7fc978281700 1 -- start start 2023-09-30T22:36:40.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.506+0000 7fc978281700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc97009d6f0 0x7fc97009dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:40.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.506+0000 7fc978281700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc97009e000 0x7fc970130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:40.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.506+0000 7fc978281700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc97009f130 con 0x7fc97009e410 2023-09-30T22:36:40.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.506+0000 7fc978281700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc97009f520 con 0x7fc97009e000 2023-09-30T22:36:40.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.506+0000 7fc978281700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc97009e930 con 0x7fc97009d6f0 2023-09-30T22:36:40.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.507+0000 7fc97581c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc97009e000 0x7fc970130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:40.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.507+0000 7fc97681e700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fc97009e410 0x7fc970130ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60080/0 (socket says 172.21.15.16:60080) 2023-09-30T22:36:40.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.507+0000 7fc97581c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc97009e000 0x7fc970130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36342/0 (socket says 172.21.15.16:36342) 2023-09-30T22:36:40.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.507+0000 7fc97681e700 1 -- 172.21.15.16:0/1538122364 learned_addr learned my addr 172.21.15.16:0/1538122364 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:36:40.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.507+0000 7fc97601d700 1 --2- 172.21.15.16:0/1538122364 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc97009d6f0 0x7fc97009dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:40.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.507+0000 7fc97581c700 1 -- 172.21.15.16:0/1538122364 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc97009d6f0 msgr2=0x7fc97009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:40.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.507+0000 7fc97581c700 1 --2- 172.21.15.16:0/1538122364 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc97009d6f0 0x7fc97009dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:40.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.507+0000 7fc97581c700 1 -- 172.21.15.16:0/1538122364 >> v1:172.21.15.16:6789/0 conn(0x7fc97009e410 legacy=0x7fc970130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:40.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.507+0000 7fc97581c700 1 -- 172.21.15.16:0/1538122364 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc97009b2d0 con 0x7fc97009e000 2023-09-30T22:36:40.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.507+0000 7fc97501b700 1 -- 172.21.15.16:0/1538122364 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2390804704 0 0) 0x7fc97009f130 con 0x7fc97009e410 2023-09-30T22:36:40.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.507+0000 7fc97581c700 1 --2- 172.21.15.16:0/1538122364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc97009e000 0x7fc970130660 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fc96000bae0 tx=0x7fc96000bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfd75cdca8c8c0db server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:40.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.507+0000 7fc97501b700 1 -- 172.21.15.16:0/1538122364 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96000dd30 con 0x7fc97009e000 2023-09-30T22:36:40.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.507+0000 7fc97501b700 1 -- 172.21.15.16:0/1538122364 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc96000de90 con 0x7fc97009e000 2023-09-30T22:36:40.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.508+0000 7fc97501b700 1 -- 172.21.15.16:0/1538122364 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc960010690 con 0x7fc97009e000 2023-09-30T22:36:40.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.509+0000 7fc978281700 1 -- 172.21.15.16:0/1538122364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc97009e000 msgr2=0x7fc970130660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:40.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.509+0000 7fc978281700 1 --2- 172.21.15.16:0/1538122364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc97009e000 0x7fc970130660 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7fc96000bae0 tx=0x7fc96000bdf0 comp rx=0 tx=0).stop 2023-09-30T22:36:40.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.509+0000 7fc978281700 1 -- 172.21.15.16:0/1538122364 shutdown_connections 2023-09-30T22:36:40.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.509+0000 7fc978281700 1 --2- 172.21.15.16:0/1538122364 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc97009d6f0 0x7fc97009dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:40.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.509+0000 7fc978281700 1 --2- 172.21.15.16:0/1538122364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc97009e000 0x7fc970130660 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:40.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.509+0000 7fc978281700 1 -- 172.21.15.16:0/1538122364 >> 172.21.15.16:0/1538122364 conn(0x7fc970066f50 msgr2=0x7fc97009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:40.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.509+0000 7fc978281700 1 -- 172.21.15.16:0/1538122364 shutdown_connections 2023-09-30T22:36:40.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.509+0000 7fc978281700 1 -- 172.21.15.16:0/1538122364 wait complete. 2023-09-30T22:36:40.512 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.510+0000 7fc978281700 1 Processor -- start 2023-09-30T22:36:40.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.510+0000 7fc978281700 1 -- start start 2023-09-30T22:36:40.515 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.511+0000 7fc978281700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc97009d6f0 0x7fc97011de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:40.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.511+0000 7fc978281700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc97009e000 0x7fc97011e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:40.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.511+0000 7fc978281700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc97009e410 0x7fc97011e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:40.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.511+0000 7fc978281700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97011f280 con 0x7fc97009d6f0 2023-09-30T22:36:40.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.511+0000 7fc978281700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc970120010 con 0x7fc97009e000 2023-09-30T22:36:40.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.511+0000 7fc978281700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc9701210d0 con 0x7fc97009e410 2023-09-30T22:36:40.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.511+0000 7fc97601d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc97009d6f0 0x7fc97011de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:40.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.511+0000 7fc97601d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc97009d6f0 0x7fc97011de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36358/0 (socket says 172.21.15.16:36358) 2023-09-30T22:36:40.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.511+0000 7fc97601d700 1 -- 172.21.15.16:0/1867350765 learned_addr learned my addr 172.21.15.16:0/1867350765 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:40.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.511+0000 7fc97581c700 1 --2- 172.21.15.16:0/1867350765 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc97009e000 0x7fc97011e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:40.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.512+0000 7fc97601d700 1 -- 172.21.15.16:0/1867350765 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc97009e410 msgr2=0x7fc97011e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:40.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.512+0000 7fc97601d700 1 --2- 172.21.15.16:0/1867350765 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc97009e410 0x7fc97011e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:40.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.512+0000 7fc97601d700 1 -- 172.21.15.16:0/1867350765 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc97009e000 msgr2=0x7fc97011e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:40.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.512+0000 7fc97601d700 1 --2- 172.21.15.16:0/1867350765 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc97009e000 0x7fc97011e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:40.517 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.512+0000 7fc97601d700 1 -- 172.21.15.16:0/1867350765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc96000b740 con 0x7fc97009d6f0 2023-09-30T22:36:40.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.512+0000 7fc97581c700 1 --2- 172.21.15.16:0/1867350765 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc97009e000 0x7fc97011e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:40.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.512+0000 7fc97601d700 1 --2- 172.21.15.16:0/1867350765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc97009d6f0 0x7fc97011de20 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7fc96c00d830 tx=0x7fc96c00db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:40.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.512+0000 7fc966ffd700 1 -- 172.21.15.16:0/1867350765 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96c0125f0 con 0x7fc97009d6f0 2023-09-30T22:36:40.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.512+0000 7fc966ffd700 1 -- 172.21.15.16:0/1867350765 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc96c012750 con 0x7fc97009d6f0 2023-09-30T22:36:40.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.512+0000 7fc966ffd700 1 -- 172.21.15.16:0/1867350765 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96c012b00 con 0x7fc97009d6f0 2023-09-30T22:36:40.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.512+0000 7fc978281700 1 -- 172.21.15.16:0/1867350765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc97009c700 con 0x7fc97009d6f0 2023-09-30T22:36:40.518 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.512+0000 7fc978281700 1 -- 172.21.15.16:0/1867350765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc97011fa30 con 0x7fc97009d6f0 2023-09-30T22:36:40.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.513+0000 7fc966ffd700 1 -- 172.21.15.16:0/1867350765 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc96c012c60 con 0x7fc97009d6f0 2023-09-30T22:36:40.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.514+0000 7fc966ffd700 1 -- 172.21.15.16:0/1867350765 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fc96c03f9e0 con 0x7fc97009d6f0 2023-09-30T22:36:40.519 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.514+0000 7fc978281700 1 -- 172.21.15.16:0/1867350765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc97011ac20 con 0x7fc97009d6f0 2023-09-30T22:36:40.524 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.523+0000 7fc966ffd700 1 -- 172.21.15.16:0/1867350765 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc96c014430 con 0x7fc97009d6f0 2023-09-30T22:36:40.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.562+0000 7fd329ab2700 1 Processor -- start 2023-09-30T22:36:40.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.562+0000 7fd329ab2700 1 -- start start 2023-09-30T22:36:40.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.563+0000 7fd329ab2700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd324133420 0x7fd3241337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:40.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.563+0000 7fd329ab2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd324134260 0x7fd3241306a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:40.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.563+0000 7fd329ab2700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd32406b990 con 0x7fd324133e30 2023-09-30T22:36:40.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.563+0000 7fd329ab2700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3240a0590 con 0x7fd324133420 2023-09-30T22:36:40.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.563+0000 7fd329ab2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd32409fb80 con 0x7fd324134260 2023-09-30T22:36:40.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.563+0000 7fd3237fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd324133420 0x7fd3241337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:40.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.563+0000 7fd323fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd324134260 0x7fd3241306a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:40.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.563+0000 7fd322ffd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd324133e30 0x7fd32412ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60090/0 (socket says 172.21.15.16:60090) 2023-09-30T22:36:40.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.563+0000 7fd322ffd700 1 -- 172.21.15.16:0/806474796 learned_addr learned my addr 172.21.15.16:0/806474796 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:36:40.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.563+0000 7fd3237fe700 1 -- 172.21.15.16:0/806474796 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd324134260 msgr2=0x7fd3241306a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:40.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.563+0000 7fd3237fe700 1 --2- 172.21.15.16:0/806474796 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd324134260 0x7fd3241306a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:40.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.563+0000 7fd3237fe700 1 -- 172.21.15.16:0/806474796 >> v1:172.21.15.16:6789/0 conn(0x7fd324133e30 legacy=0x7fd32412ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:40.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.563+0000 7fd3237fe700 1 -- 172.21.15.16:0/806474796 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3240a0030 con 0x7fd324133420 2023-09-30T22:36:40.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.563+0000 7fd3227fc700 1 -- 172.21.15.16:0/806474796 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1498381062 0 0) 0x7fd32406b990 con 0x7fd324133e30 2023-09-30T22:36:40.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.563+0000 7fd323fff700 1 --2- 172.21.15.16:0/806474796 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd324134260 0x7fd3241306a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:40.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.564+0000 7fd3237fe700 1 --2- 172.21.15.16:0/806474796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd324133420 0x7fd3241337f0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fd314009840 tx=0x7fd314009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e79e8bdc1569c150 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:40.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.564+0000 7fd3227fc700 1 -- 172.21.15.16:0/806474796 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd314011830 con 0x7fd324133420 2023-09-30T22:36:40.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.564+0000 7fd3227fc700 1 -- 172.21.15.16:0/806474796 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd314011990 con 0x7fd324133420 2023-09-30T22:36:40.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.564+0000 7fd3227fc700 1 -- 172.21.15.16:0/806474796 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd314011d20 con 0x7fd324133420 2023-09-30T22:36:40.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.565+0000 7fd329ab2700 1 -- 172.21.15.16:0/806474796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd324133420 msgr2=0x7fd3241337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:40.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.565+0000 7fd329ab2700 1 --2- 172.21.15.16:0/806474796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd324133420 0x7fd3241337f0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fd314009840 tx=0x7fd314009b50 comp rx=0 tx=0).stop 2023-09-30T22:36:40.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.565+0000 7fd329ab2700 1 -- 172.21.15.16:0/806474796 shutdown_connections 2023-09-30T22:36:40.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.565+0000 7fd329ab2700 1 --2- 172.21.15.16:0/806474796 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd324134260 0x7fd3241306a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:40.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.565+0000 7fd329ab2700 1 --2- 172.21.15.16:0/806474796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd324133420 0x7fd3241337f0 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:40.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.565+0000 7fd329ab2700 1 -- 172.21.15.16:0/806474796 >> 172.21.15.16:0/806474796 conn(0x7fd3240a4210 msgr2=0x7fd3240a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:40.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.565+0000 7fd329ab2700 1 -- 172.21.15.16:0/806474796 shutdown_connections 2023-09-30T22:36:40.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.565+0000 7fd329ab2700 1 -- 172.21.15.16:0/806474796 wait complete. 2023-09-30T22:36:40.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.567+0000 7fd329ab2700 1 Processor -- start 2023-09-30T22:36:40.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.567+0000 7fd329ab2700 1 -- start start 2023-09-30T22:36:40.570 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.568+0000 7fd329ab2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd324133420 0x7fd32409c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:40.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.568+0000 7fd329ab2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd324133e30 0x7fd32409c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:40.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.568+0000 7fd329ab2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd324134260 0x7fd32409cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:40.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.568+0000 7fd329ab2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd32411f980 con 0x7fd324134260 2023-09-30T22:36:40.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.568+0000 7fd329ab2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd32409dee0 con 0x7fd324133420 2023-09-30T22:36:40.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.568+0000 7fd329ab2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd32411ff10 con 0x7fd324133e30 2023-09-30T22:36:40.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.568+0000 7fd322ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd324133e30 0x7fd32409c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:40.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.568+0000 7fd322ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd324133e30 0x7fd32409c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35566/0 (socket says 172.21.15.16:35566) 2023-09-30T22:36:40.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.568+0000 7fd322ffd700 1 -- 172.21.15.16:0/3211087597 learned_addr learned my addr 172.21.15.16:0/3211087597 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:40.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.568+0000 7fd323fff700 1 --2- 172.21.15.16:0/3211087597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd324134260 0x7fd32409cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:40.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.569+0000 7fd322ffd700 1 -- 172.21.15.16:0/3211087597 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd324133420 msgr2=0x7fd32409c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:40.571 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.569+0000 7fd322ffd700 1 --2- 172.21.15.16:0/3211087597 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd324133420 0x7fd32409c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:40.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.569+0000 7fd322ffd700 1 -- 172.21.15.16:0/3211087597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd324134260 msgr2=0x7fd32409cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:40.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.569+0000 7fd322ffd700 1 --2- 172.21.15.16:0/3211087597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd324134260 0x7fd32409cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:40.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.569+0000 7fd322ffd700 1 -- 172.21.15.16:0/3211087597 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd31c009a70 con 0x7fd324133e30 2023-09-30T22:36:40.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.569+0000 7fd323fff700 1 --2- 172.21.15.16:0/3211087597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd324134260 0x7fd32409cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:40.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.569+0000 7fd322ffd700 1 --2- 172.21.15.16:0/3211087597 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd324133e30 0x7fd32409c9c0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fd31800c520 tx=0x7fd31800c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:40.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.569+0000 7fd30bfff700 1 -- 172.21.15.16:0/3211087597 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd318004c80 con 0x7fd324133e30 2023-09-30T22:36:40.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.569+0000 7fd30bfff700 1 -- 172.21.15.16:0/3211087597 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd318004de0 con 0x7fd324133e30 2023-09-30T22:36:40.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.569+0000 7fd329ab2700 1 -- 172.21.15.16:0/3211087597 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3140094a0 con 0x7fd324133e30 2023-09-30T22:36:40.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.569+0000 7fd30bfff700 1 -- 172.21.15.16:0/3211087597 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd318007310 con 0x7fd324133e30 2023-09-30T22:36:40.572 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.569+0000 7fd329ab2700 1 -- 172.21.15.16:0/3211087597 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3241206f0 con 0x7fd324133e30 2023-09-30T22:36:40.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.571+0000 7fd30bfff700 1 -- 172.21.15.16:0/3211087597 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd318007470 con 0x7fd324133e30 2023-09-30T22:36:40.573 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.571+0000 7fd329ab2700 1 -- 172.21.15.16:0/3211087597 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3040052e0 con 0x7fd324133e30 2023-09-30T22:36:40.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.571+0000 7fd30bfff700 1 -- 172.21.15.16:0/3211087597 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd31803f4b0 con 0x7fd324133e30 2023-09-30T22:36:40.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.576+0000 7fd30bfff700 1 -- 172.21.15.16:0/3211087597 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd3180106d0 con 0x7fd324133e30 2023-09-30T22:36:40.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.599+0000 7fe867594700 1 Processor -- start 2023-09-30T22:36:40.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.599+0000 7fe867594700 1 -- start start 2023-09-30T22:36:40.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.600+0000 7fe867594700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe86009f410 0x7fe860136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:40.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.600+0000 7fe867594700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fe860136fe0 0x7fe8601393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:40.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.600+0000 7fe867594700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe860070940 con 0x7fe86009e930 2023-09-30T22:36:40.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.600+0000 7fe867594700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe86006fd60 con 0x7fe860136fe0 2023-09-30T22:36:40.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.600+0000 7fe867594700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe86006fa80 con 0x7fe86009f410 2023-09-30T22:36:40.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.600+0000 7fe865330700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fe86009e930 0x7fe86009ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60104/0 (socket says 172.21.15.16:60104) 2023-09-30T22:36:40.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.600+0000 7fe865330700 1 -- 172.21.15.16:0/2253102941 learned_addr learned my addr 172.21.15.16:0/2253102941 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:36:40.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.600+0000 7fe864b2f700 1 --2- 172.21.15.16:0/2253102941 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe86009f410 0x7fe860136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:40.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.601+0000 7fe865b31700 1 --2- 172.21.15.16:0/2253102941 >> v2:172.21.15.16:3300/0 conn(0x7fe860136fe0 0x7fe8601393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:40.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.601+0000 7fe857fff700 1 -- 172.21.15.16:0/2253102941 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4254344221 0 0) 0x7fe860070940 con 0x7fe86009e930 2023-09-30T22:36:40.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.601+0000 7fe857fff700 1 -- 172.21.15.16:0/2253102941 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe84c0035b0 con 0x7fe86009e930 2023-09-30T22:36:40.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.601+0000 7fe865b31700 1 -- 172.21.15.16:0/2253102941 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe86009f410 msgr2=0x7fe860136aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:40.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.601+0000 7fe865b31700 1 --2- 172.21.15.16:0/2253102941 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe86009f410 0x7fe860136aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:40.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.601+0000 7fe865b31700 1 -- 172.21.15.16:0/2253102941 >> v1:172.21.15.16:6789/0 conn(0x7fe86009e930 legacy=0x7fe86009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:40.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.601+0000 7fe865b31700 1 -- 172.21.15.16:0/2253102941 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe86009fb60 con 0x7fe860136fe0 2023-09-30T22:36:40.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.601+0000 7fe865b31700 1 --2- 172.21.15.16:0/2253102941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe860136fe0 0x7fe8601393c0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fe858009d90 tx=0x7fe85800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcebc777033d007f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:40.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.601+0000 7fe857fff700 1 -- 172.21.15.16:0/2253102941 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe85800e070 con 0x7fe860136fe0 2023-09-30T22:36:40.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.602+0000 7fe857fff700 1 -- 172.21.15.16:0/2253102941 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe85800bb20 con 0x7fe860136fe0 2023-09-30T22:36:40.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.602+0000 7fe857fff700 1 -- 172.21.15.16:0/2253102941 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe858012430 con 0x7fe860136fe0 2023-09-30T22:36:40.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.602+0000 7fe867594700 1 -- 172.21.15.16:0/2253102941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe860136fe0 msgr2=0x7fe8601393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:40.605 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.602+0000 7fe867594700 1 --2- 172.21.15.16:0/2253102941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe860136fe0 0x7fe8601393c0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fe858009d90 tx=0x7fe85800b330 comp rx=0 tx=0).stop 2023-09-30T22:36:40.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.602+0000 7fe867594700 1 -- 172.21.15.16:0/2253102941 shutdown_connections 2023-09-30T22:36:40.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.602+0000 7fe867594700 1 --2- 172.21.15.16:0/2253102941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe860136fe0 0x7fe8601393c0 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:40.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.602+0000 7fe867594700 1 --2- 172.21.15.16:0/2253102941 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe86009f410 0x7fe860136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:40.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.602+0000 7fe867594700 1 -- 172.21.15.16:0/2253102941 >> 172.21.15.16:0/2253102941 conn(0x7fe8600a4210 msgr2=0x7fe8600a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:40.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.603+0000 7fe867594700 1 -- 172.21.15.16:0/2253102941 shutdown_connections 2023-09-30T22:36:40.606 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.603+0000 7fe867594700 1 -- 172.21.15.16:0/2253102941 wait complete. 2023-09-30T22:36:40.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.604+0000 7fe867594700 1 Processor -- start 2023-09-30T22:36:40.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.604+0000 7fe867594700 1 -- start start 2023-09-30T22:36:40.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.605+0000 7fe867594700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe86009e930 0x7fe86011e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:40.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.605+0000 7fe867594700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe86009f410 0x7fe86011ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:40.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.605+0000 7fe867594700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe860136fe0 0x7fe86011f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:40.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.605+0000 7fe867594700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe860120290 con 0x7fe860136fe0 2023-09-30T22:36:40.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.605+0000 7fe867594700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8601208b0 con 0x7fe86009e930 2023-09-30T22:36:40.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.605+0000 7fe867594700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fe860071d90 con 0x7fe86009f410 2023-09-30T22:36:40.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.605+0000 7fe865330700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe86009e930 0x7fe86011e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:40.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.605+0000 7fe865330700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe86009e930 0x7fe86011e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:59304/0 (socket says 172.21.15.16:59304) 2023-09-30T22:36:40.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.605+0000 7fe865330700 1 -- 172.21.15.16:0/347186469 learned_addr learned my addr 172.21.15.16:0/347186469 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:40.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.605+0000 7fe865b31700 1 --2- 172.21.15.16:0/347186469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe860136fe0 0x7fe86011f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:40.609 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.605+0000 7fe865330700 1 -- 172.21.15.16:0/347186469 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe86009f410 msgr2=0x7fe86011ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:40.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.605+0000 7fe864b2f700 1 --2- 172.21.15.16:0/347186469 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe86009f410 0x7fe86011ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:40.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.605+0000 7fe865330700 1 --2- 172.21.15.16:0/347186469 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe86009f410 0x7fe86011ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:40.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.605+0000 7fe865330700 1 -- 172.21.15.16:0/347186469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe860136fe0 msgr2=0x7fe86011f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:40.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.605+0000 7fe865330700 1 --2- 172.21.15.16:0/347186469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe860136fe0 0x7fe86011f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:40.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.605+0000 7fe865330700 1 -- 172.21.15.16:0/347186469 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe858009a70 con 0x7fe86009e930 2023-09-30T22:36:40.610 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.605+0000 7fe864b2f700 1 --2- 172.21.15.16:0/347186469 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe86009f410 0x7fe86011ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:36:40.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.606+0000 7fe865330700 1 --2- 172.21.15.16:0/347186469 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe86009e930 0x7fe86011e680 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fe85c00c730 tx=0x7fe85c00ca40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:40.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.606+0000 7fe855ffb700 1 -- 172.21.15.16:0/347186469 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe85c005d70 con 0x7fe86009e930 2023-09-30T22:36:40.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.606+0000 7fe855ffb700 1 -- 172.21.15.16:0/347186469 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe85c0070d0 con 0x7fe86009e930 2023-09-30T22:36:40.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.606+0000 7fe867594700 1 -- 172.21.15.16:0/347186469 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe86009de30 con 0x7fe86009e930 2023-09-30T22:36:40.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.606+0000 7fe855ffb700 1 -- 172.21.15.16:0/347186469 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe85c007440 con 0x7fe86009e930 2023-09-30T22:36:40.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.606+0000 7fe867594700 1 -- 172.21.15.16:0/347186469 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe860121c50 con 0x7fe86009e930 2023-09-30T22:36:40.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.607+0000 7fe855ffb700 1 -- 172.21.15.16:0/347186469 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fe85c0075a0 con 0x7fe86009e930 2023-09-30T22:36:40.611 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.607+0000 7fe855ffb700 1 -- 172.21.15.16:0/347186469 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fe85c03eb90 con 0x7fe86009e930 2023-09-30T22:36:40.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.608+0000 7fe867594700 1 -- 172.21.15.16:0/347186469 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe860074ea0 con 0x7fe86009e930 2023-09-30T22:36:40.614 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.613+0000 7fe855ffb700 1 -- 172.21.15.16:0/347186469 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fe85c019b40 con 0x7fe86009e930 2023-09-30T22:36:40.638 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.636+0000 7fc92a4fd700 1 Processor -- start 2023-09-30T22:36:40.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.637+0000 7fc92a4fd700 1 -- start start 2023-09-30T22:36:40.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.637+0000 7fc92a4fd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc924130540 0x7fc924130910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:40.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.637+0000 7fc92a4fd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc92406fcf0 0x7fc924138570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:40.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.637+0000 7fc92a4fd700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9240716a0 con 0x7fc924130e50 2023-09-30T22:36:40.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.637+0000 7fc92a4fd700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc92409a390 con 0x7fc92406fcf0 2023-09-30T22:36:40.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.637+0000 7fc92a4fd700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc924071290 con 0x7fc924130540 2023-09-30T22:36:40.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.637+0000 7fc923fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc924130540 0x7fc924130910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:40.639 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.637+0000 7fc923fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc924130540 0x7fc924130910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35580/0 (socket says 172.21.15.16:35580) 2023-09-30T22:36:40.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.637+0000 7fc923fff700 1 -- 172.21.15.16:0/1505744582 learned_addr learned my addr 172.21.15.16:0/1505744582 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:40.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.637+0000 7fc928a9a700 1 --2- 172.21.15.16:0/1505744582 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc92406fcf0 0x7fc924138570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:40.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.638+0000 7fc922ffd700 1 -- 172.21.15.16:0/1505744582 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1912138353 0 0) 0x7fc9240716a0 con 0x7fc924130e50 2023-09-30T22:36:40.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.638+0000 7fc922ffd700 1 -- 172.21.15.16:0/1505744582 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc90c0057c0 con 0x7fc924130e50 2023-09-30T22:36:40.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.638+0000 7fc923fff700 1 -- 172.21.15.16:0/1505744582 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc92406fcf0 msgr2=0x7fc924138570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:40.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.638+0000 7fc923fff700 1 --2- 172.21.15.16:0/1505744582 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc92406fcf0 0x7fc924138570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:40.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.638+0000 7fc923fff700 1 -- 172.21.15.16:0/1505744582 >> v1:172.21.15.16:6789/0 conn(0x7fc924130e50 legacy=0x7fc92406f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:40.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.638+0000 7fc923fff700 1 -- 172.21.15.16:0/1505744582 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc924130340 con 0x7fc924130540 2023-09-30T22:36:40.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.638+0000 7fc923fff700 1 --2- 172.21.15.16:0/1505744582 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc924130540 0x7fc924130910 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fc914009920 tx=0x7fc914009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71429428cf118b46 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:40.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.638+0000 7fc922ffd700 1 -- 172.21.15.16:0/1505744582 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc914011830 con 0x7fc924130540 2023-09-30T22:36:40.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.638+0000 7fc922ffd700 1 -- 172.21.15.16:0/1505744582 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc914011990 con 0x7fc924130540 2023-09-30T22:36:40.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.638+0000 7fc922ffd700 1 -- 172.21.15.16:0/1505744582 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc914011d20 con 0x7fc924130540 2023-09-30T22:36:40.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.639+0000 7fc92a4fd700 1 -- 172.21.15.16:0/1505744582 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc924130540 msgr2=0x7fc924130910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:40.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.639+0000 7fc92a4fd700 1 --2- 172.21.15.16:0/1505744582 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc924130540 0x7fc924130910 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fc914009920 tx=0x7fc914009c30 comp rx=0 tx=0).stop 2023-09-30T22:36:40.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.640+0000 7fc92a4fd700 1 -- 172.21.15.16:0/1505744582 shutdown_connections 2023-09-30T22:36:40.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.640+0000 7fc92a4fd700 1 --2- 172.21.15.16:0/1505744582 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc92406fcf0 0x7fc924138570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:40.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.640+0000 7fc92a4fd700 1 --2- 172.21.15.16:0/1505744582 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc924130540 0x7fc924130910 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:40.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.640+0000 7fc92a4fd700 1 -- 172.21.15.16:0/1505744582 >> 172.21.15.16:0/1505744582 conn(0x7fc9240a2aa0 msgr2=0x7fc9240a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:40.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.640+0000 7fc92a4fd700 1 -- 172.21.15.16:0/1505744582 shutdown_connections 2023-09-30T22:36:40.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.640+0000 7fc92a4fd700 1 -- 172.21.15.16:0/1505744582 wait complete. 2023-09-30T22:36:40.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.641+0000 7fc92a4fd700 1 Processor -- start 2023-09-30T22:36:40.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.641+0000 7fc92a4fd700 1 -- start start 2023-09-30T22:36:40.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.641+0000 7fc92a4fd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc92406fcf0 0x7fc92411a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:40.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.641+0000 7fc92a4fd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc924130540 0x7fc92411ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:40.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.641+0000 7fc92a4fd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc924130e50 0x7fc92411b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:40.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.641+0000 7fc92a4fd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92411e1f0 con 0x7fc924130e50 2023-09-30T22:36:40.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.641+0000 7fc92a4fd700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc92411d5a0 con 0x7fc924130540 2023-09-30T22:36:40.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.642+0000 7fc92a4fd700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc92412cb70 con 0x7fc92406fcf0 2023-09-30T22:36:40.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.642+0000 7fc923fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc92406fcf0 0x7fc92411a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:40.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.642+0000 7fc923fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc92406fcf0 0x7fc92411a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35592/0 (socket says 172.21.15.16:35592) 2023-09-30T22:36:40.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.642+0000 7fc923fff700 1 -- 172.21.15.16:0/4126801596 learned_addr learned my addr 172.21.15.16:0/4126801596 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:40.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.642+0000 7fc928a9a700 1 --2- 172.21.15.16:0/4126801596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc924130e50 0x7fc92411b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:40.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.642+0000 7fc9237fe700 1 --2- 172.21.15.16:0/4126801596 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc924130540 0x7fc92411ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:40.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.642+0000 7fc923fff700 1 -- 172.21.15.16:0/4126801596 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc924130540 msgr2=0x7fc92411ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:40.646 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.642+0000 7fc923fff700 1 --2- 172.21.15.16:0/4126801596 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc924130540 0x7fc92411ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:40.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.642+0000 7fc923fff700 1 -- 172.21.15.16:0/4126801596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc924130e50 msgr2=0x7fc92411b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:40.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.642+0000 7fc923fff700 1 --2- 172.21.15.16:0/4126801596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc924130e50 0x7fc92411b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:40.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.642+0000 7fc923fff700 1 -- 172.21.15.16:0/4126801596 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc914009550 con 0x7fc92406fcf0 2023-09-30T22:36:40.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.642+0000 7fc923fff700 1 --2- 172.21.15.16:0/4126801596 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc92406fcf0 0x7fc92411a6d0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fc914000c00 tx=0x7fc914004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:40.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.643+0000 7fc920ff9700 1 -- 172.21.15.16:0/4126801596 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc914004510 con 0x7fc92406fcf0 2023-09-30T22:36:40.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.643+0000 7fc920ff9700 1 -- 172.21.15.16:0/4126801596 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9140036a0 con 0x7fc92406fcf0 2023-09-30T22:36:40.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.643+0000 7fc920ff9700 1 -- 172.21.15.16:0/4126801596 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc914003a50 con 0x7fc92406fcf0 2023-09-30T22:36:40.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.643+0000 7fc92a4fd700 1 -- 172.21.15.16:0/4126801596 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9240a0830 con 0x7fc92406fcf0 2023-09-30T22:36:40.647 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.643+0000 7fc92a4fd700 1 -- 172.21.15.16:0/4126801596 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc92411e980 con 0x7fc92406fcf0 2023-09-30T22:36:40.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.644+0000 7fc92a4fd700 1 -- 172.21.15.16:0/4126801596 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc92406f690 con 0x7fc92406fcf0 2023-09-30T22:36:40.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.645+0000 7fc920ff9700 1 -- 172.21.15.16:0/4126801596 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc914003bb0 con 0x7fc92406fcf0 2023-09-30T22:36:40.648 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.645+0000 7fc920ff9700 1 -- 172.21.15.16:0/4126801596 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fc914021030 con 0x7fc92406fcf0 2023-09-30T22:36:40.655 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.654+0000 7fc920ff9700 1 -- 172.21.15.16:0/4126801596 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc914030480 con 0x7fc92406fcf0 2023-09-30T22:36:40.787 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.786+0000 7fc978281700 1 -- 172.21.15.16:0/1867350765 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc970074ea0 con 0x7fc97009d6f0 2023-09-30T22:36:40.788 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.786+0000 7fc966ffd700 1 -- 172.21.15.16:0/1867350765 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc96c01c360 con 0x7fc97009d6f0 2023-09-30T22:36:40.788 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:36:40.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.789+0000 7fc978281700 1 -- 172.21.15.16:0/1867350765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc97009d6f0 msgr2=0x7fc97011de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:40.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.789+0000 7fc978281700 1 --2- 172.21.15.16:0/1867350765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc97009d6f0 0x7fc97011de20 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7fc96c00d830 tx=0x7fc96c00db40 comp rx=0 tx=0).stop 2023-09-30T22:36:40.790 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.789+0000 7fc978281700 1 -- 172.21.15.16:0/1867350765 shutdown_connections 2023-09-30T22:36:40.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.789+0000 7fc978281700 1 --2- 172.21.15.16:0/1867350765 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc97009e410 0x7fc97011e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:40.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.789+0000 7fc978281700 1 --2- 172.21.15.16:0/1867350765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc97009d6f0 0x7fc97011de20 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:40.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.789+0000 7fc978281700 1 --2- 172.21.15.16:0/1867350765 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc97009e000 0x7fc97011e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:40.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.789+0000 7fc978281700 1 -- 172.21.15.16:0/1867350765 >> 172.21.15.16:0/1867350765 conn(0x7fc970066f50 msgr2=0x7fc970130a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:40.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.789+0000 7fc978281700 1 -- 172.21.15.16:0/1867350765 shutdown_connections 2023-09-30T22:36:40.791 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.789+0000 7fc978281700 1 -- 172.21.15.16:0/1867350765 wait complete. 2023-09-30T22:36:40.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.796+0000 7fd329ab2700 1 -- 172.21.15.16:0/3211087597 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fd304005590 con 0x7fd324133e30 2023-09-30T22:36:40.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.796+0000 7fd30bfff700 1 -- 172.21.15.16:0/3211087597 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd318011360 con 0x7fd324133e30 2023-09-30T22:36:40.798 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:36:40.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.799+0000 7fd329ab2700 1 -- 172.21.15.16:0/3211087597 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd324133e30 msgr2=0x7fd32409c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:40.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.799+0000 7fd329ab2700 1 --2- 172.21.15.16:0/3211087597 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd324133e30 0x7fd32409c9c0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fd31800c520 tx=0x7fd31800c830 comp rx=0 tx=0).stop 2023-09-30T22:36:40.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.799+0000 7fd329ab2700 1 -- 172.21.15.16:0/3211087597 shutdown_connections 2023-09-30T22:36:40.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.799+0000 7fd329ab2700 1 --2- 172.21.15.16:0/3211087597 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd324133420 0x7fd32409c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:40.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.799+0000 7fd329ab2700 1 --2- 172.21.15.16:0/3211087597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd324134260 0x7fd32409cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:40.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.799+0000 7fd329ab2700 1 --2- 172.21.15.16:0/3211087597 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd324133e30 0x7fd32409c9c0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:40.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.799+0000 7fd329ab2700 1 -- 172.21.15.16:0/3211087597 >> 172.21.15.16:0/3211087597 conn(0x7fd3240a4210 msgr2=0x7fd3240a0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:40.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.799+0000 7fd329ab2700 1 -- 172.21.15.16:0/3211087597 shutdown_connections 2023-09-30T22:36:40.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.799+0000 7fd329ab2700 1 -- 172.21.15.16:0/3211087597 wait complete. 2023-09-30T22:36:40.805 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:36:40.820 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:36:40.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.849+0000 7fe867594700 1 -- 172.21.15.16:0/347186469 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fe8600a2e20 con 0x7fe86009e930 2023-09-30T22:36:40.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.849+0000 7fe855ffb700 1 -- 172.21.15.16:0/347186469 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe85c017020 con 0x7fe86009e930 2023-09-30T22:36:40.851 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:36:40.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.851+0000 7fe867594700 1 -- 172.21.15.16:0/347186469 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe86009e930 msgr2=0x7fe86011e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:40.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.851+0000 7fe867594700 1 --2- 172.21.15.16:0/347186469 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe86009e930 0x7fe86011e680 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fe85c00c730 tx=0x7fe85c00ca40 comp rx=0 tx=0).stop 2023-09-30T22:36:40.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.851+0000 7fe867594700 1 -- 172.21.15.16:0/347186469 shutdown_connections 2023-09-30T22:36:40.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.851+0000 7fe867594700 1 --2- 172.21.15.16:0/347186469 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fe86009e930 0x7fe86011e680 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:40.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.851+0000 7fe867594700 1 --2- 172.21.15.16:0/347186469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe860136fe0 0x7fe86011f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:40.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.851+0000 7fe867594700 1 --2- 172.21.15.16:0/347186469 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fe86009f410 0x7fe86011ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:40.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.851+0000 7fe867594700 1 -- 172.21.15.16:0/347186469 >> 172.21.15.16:0/347186469 conn(0x7fe8600a4210 msgr2=0x7fe860134900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:40.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.851+0000 7fe867594700 1 -- 172.21.15.16:0/347186469 shutdown_connections 2023-09-30T22:36:40.854 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.851+0000 7fe867594700 1 -- 172.21.15.16:0/347186469 wait complete. 2023-09-30T22:36:40.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.864+0000 7fc92a4fd700 1 -- 172.21.15.16:0/4126801596 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fc924074ea0 con 0x7fc92406fcf0 2023-09-30T22:36:40.865 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.864+0000 7fc920ff9700 1 -- 172.21.15.16:0/4126801596 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc914015070 con 0x7fc92406fcf0 2023-09-30T22:36:40.866 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:36:40.867 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:36:40.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.866+0000 7fc92a4fd700 1 -- 172.21.15.16:0/4126801596 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc92406fcf0 msgr2=0x7fc92411a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:40.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.866+0000 7fc92a4fd700 1 --2- 172.21.15.16:0/4126801596 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc92406fcf0 0x7fc92411a6d0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fc914000c00 tx=0x7fc914004750 comp rx=0 tx=0).stop 2023-09-30T22:36:40.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.866+0000 7fc92a4fd700 1 -- 172.21.15.16:0/4126801596 shutdown_connections 2023-09-30T22:36:40.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.866+0000 7fc92a4fd700 1 --2- 172.21.15.16:0/4126801596 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc924130e50 0x7fc92411b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:40.868 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.866+0000 7fc92a4fd700 1 --2- 172.21.15.16:0/4126801596 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc92406fcf0 0x7fc92411a6d0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:40.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.866+0000 7fc92a4fd700 1 --2- 172.21.15.16:0/4126801596 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc924130540 0x7fc92411ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:40.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.866+0000 7fc92a4fd700 1 -- 172.21.15.16:0/4126801596 >> 172.21.15.16:0/4126801596 conn(0x7fc9240a2aa0 msgr2=0x7fc92412d370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:40.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.866+0000 7fc92a4fd700 1 -- 172.21.15.16:0/4126801596 shutdown_connections 2023-09-30T22:36:40.869 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:40.866+0000 7fc92a4fd700 1 -- 172.21.15.16:0/4126801596 wait complete. 2023-09-30T22:36:40.881 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:36:41.806 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:36:41.821 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:36:41.868 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:36:41.882 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:36:41.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.970+0000 7ffb03b10700 1 Processor -- start 2023-09-30T22:36:41.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.970+0000 7ffb03b10700 1 -- start start 2023-09-30T22:36:41.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.971+0000 7ffb03b10700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffafc09d6f0 0x7ffafc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:41.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.971+0000 7ffb03b10700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffafc09e410 0x7ffafc134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:41.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.971+0000 7ffb03b10700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffafc09b120 con 0x7ffafc09e000 2023-09-30T22:36:41.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.971+0000 7ffb03b10700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffafc06b460 con 0x7ffafc09d6f0 2023-09-30T22:36:41.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.971+0000 7ffb03b10700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffafc09a7e0 con 0x7ffafc09e410 2023-09-30T22:36:41.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.971+0000 7ffb020ad700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffafc09e410 0x7ffafc134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:41.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.971+0000 7ffb010ab700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7ffafc09e000 0x7ffafc1339f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60130/0 (socket says 172.21.15.16:60130) 2023-09-30T22:36:41.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.971+0000 7ffb010ab700 1 -- 172.21.15.16:0/537174979 learned_addr learned my addr 172.21.15.16:0/537174979 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:36:41.973 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.971+0000 7ffb018ac700 1 --2- 172.21.15.16:0/537174979 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffafc09d6f0 0x7ffafc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:41.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.971+0000 7ffb008aa700 1 -- 172.21.15.16:0/537174979 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4036989829 0 0) 0x7ffafc09b120 con 0x7ffafc09e000 2023-09-30T22:36:41.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.971+0000 7ffb008aa700 1 -- 172.21.15.16:0/537174979 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffae80035b0 con 0x7ffafc09e000 2023-09-30T22:36:41.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.972+0000 7ffb020ad700 1 -- 172.21.15.16:0/537174979 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffafc09d6f0 msgr2=0x7ffafc09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:41.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.972+0000 7ffb020ad700 1 --2- 172.21.15.16:0/537174979 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffafc09d6f0 0x7ffafc09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:41.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.972+0000 7ffb020ad700 1 -- 172.21.15.16:0/537174979 >> v1:172.21.15.16:6789/0 conn(0x7ffafc09e000 legacy=0x7ffafc1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:41.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.972+0000 7ffb020ad700 1 -- 172.21.15.16:0/537174979 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffafc09a500 con 0x7ffafc09e410 2023-09-30T22:36:41.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.972+0000 7ffb020ad700 1 --2- 172.21.15.16:0/537174979 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffafc09e410 0x7ffafc134100 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4009df0 tx=0x7ffaf400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a8cb0a06b0def40 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:41.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.972+0000 7ffb008aa700 1 -- 172.21.15.16:0/537174979 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf400e070 con 0x7ffafc09e410 2023-09-30T22:36:41.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.972+0000 7ffb008aa700 1 -- 172.21.15.16:0/537174979 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaf400bae0 con 0x7ffafc09e410 2023-09-30T22:36:41.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.972+0000 7ffb008aa700 1 -- 172.21.15.16:0/537174979 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf400be90 con 0x7ffafc09e410 2023-09-30T22:36:41.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.973+0000 7ffb03b10700 1 -- 172.21.15.16:0/537174979 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffafc09e410 msgr2=0x7ffafc134100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:41.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.973+0000 7ffb03b10700 1 --2- 172.21.15.16:0/537174979 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffafc09e410 0x7ffafc134100 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4009df0 tx=0x7ffaf400b330 comp rx=0 tx=0).stop 2023-09-30T22:36:41.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.973+0000 7ffb03b10700 1 -- 172.21.15.16:0/537174979 shutdown_connections 2023-09-30T22:36:41.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.973+0000 7ffb03b10700 1 --2- 172.21.15.16:0/537174979 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffafc09e410 0x7ffafc134100 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:41.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.973+0000 7ffb03b10700 1 --2- 172.21.15.16:0/537174979 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffafc09d6f0 0x7ffafc09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:41.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.973+0000 7ffb03b10700 1 -- 172.21.15.16:0/537174979 >> 172.21.15.16:0/537174979 conn(0x7ffafc066f50 msgr2=0x7ffafc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:41.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.973+0000 7ffb03b10700 1 -- 172.21.15.16:0/537174979 shutdown_connections 2023-09-30T22:36:41.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.973+0000 7ffb03b10700 1 -- 172.21.15.16:0/537174979 wait complete. 2023-09-30T22:36:41.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.975+0000 7ffb03b10700 1 Processor -- start 2023-09-30T22:36:41.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.975+0000 7ffb03b10700 1 -- start start 2023-09-30T22:36:41.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.976+0000 7ffb03b10700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffafc09d6f0 0x7ffafc1222b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:41.977 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.976+0000 7ffb03b10700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffafc09e000 0x7ffafc1227f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:41.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.976+0000 7ffb03b10700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffafc09e410 0x7ffafc122d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:41.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.976+0000 7ffb03b10700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffafc125b90 con 0x7ffafc09e000 2023-09-30T22:36:41.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.976+0000 7ffb03b10700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffafc133680 con 0x7ffafc09e410 2023-09-30T22:36:41.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.976+0000 7ffb03b10700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ffafc125830 con 0x7ffafc09d6f0 2023-09-30T22:36:41.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.976+0000 7ffb020ad700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffafc09e410 0x7ffafc122d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:41.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.976+0000 7ffb020ad700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffafc09e410 0x7ffafc122d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:59342/0 (socket says 172.21.15.16:59342) 2023-09-30T22:36:41.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.976+0000 7ffb020ad700 1 -- 172.21.15.16:0/605498718 learned_addr learned my addr 172.21.15.16:0/605498718 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:41.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.976+0000 7ffb018ac700 1 --2- 172.21.15.16:0/605498718 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffafc09d6f0 0x7ffafc1222b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:41.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.976+0000 7ffb010ab700 1 --2- 172.21.15.16:0/605498718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffafc09e000 0x7ffafc1227f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:41.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.976+0000 7ffb020ad700 1 -- 172.21.15.16:0/605498718 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffafc09d6f0 msgr2=0x7ffafc1222b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:41.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.976+0000 7ffb020ad700 1 --2- 172.21.15.16:0/605498718 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffafc09d6f0 0x7ffafc1222b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:41.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.976+0000 7ffb020ad700 1 -- 172.21.15.16:0/605498718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffafc09e000 msgr2=0x7ffafc1227f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:41.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.976+0000 7ffb020ad700 1 --2- 172.21.15.16:0/605498718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffafc09e000 0x7ffafc1227f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:41.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.976+0000 7ffb020ad700 1 -- 172.21.15.16:0/605498718 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf4009a70 con 0x7ffafc09e410 2023-09-30T22:36:41.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.977+0000 7ffb020ad700 1 --2- 172.21.15.16:0/605498718 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffafc09e410 0x7ffafc122d30 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7ffaf400b370 tx=0x7ffaf4004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:41.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.977+0000 7ffaf27fc700 1 -- 172.21.15.16:0/605498718 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf40044d0 con 0x7ffafc09e410 2023-09-30T22:36:41.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.977+0000 7ffaf27fc700 1 -- 172.21.15.16:0/605498718 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaf4020070 con 0x7ffafc09e410 2023-09-30T22:36:41.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.977+0000 7ffaf27fc700 1 -- 172.21.15.16:0/605498718 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf40048b0 con 0x7ffafc09e410 2023-09-30T22:36:41.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.977+0000 7ffb03b10700 1 -- 172.21.15.16:0/605498718 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffafc124d50 con 0x7ffafc09e410 2023-09-30T22:36:41.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.977+0000 7ffb03b10700 1 -- 172.21.15.16:0/605498718 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffafc125560 con 0x7ffafc09e410 2023-09-30T22:36:41.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.978+0000 7ffaf27fc700 1 -- 172.21.15.16:0/605498718 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ffaf4012430 con 0x7ffafc09e410 2023-09-30T22:36:41.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.978+0000 7ffb03b10700 1 -- 172.21.15.16:0/605498718 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffafc11f030 con 0x7ffafc09e410 2023-09-30T22:36:41.981 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.979+0000 7ffaf27fc700 1 -- 172.21.15.16:0/605498718 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7ffaf4022070 con 0x7ffafc09e410 2023-09-30T22:36:41.985 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.984+0000 7ffaf27fc700 1 -- 172.21.15.16:0/605498718 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ffaf401c400 con 0x7ffafc09e410 2023-09-30T22:36:41.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.997+0000 7f64cb86a700 1 Processor -- start 2023-09-30T22:36:41.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.997+0000 7f64cb86a700 1 -- start start 2023-09-30T22:36:41.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.998+0000 7f64cb86a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f64c409e1d0 0x7f64c4135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:41.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.998+0000 7f64cb86a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f64c4135e50 0x7f64c4138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:41.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.998+0000 7f64cb86a700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64c409f130 con 0x7f64c409d6f0 2023-09-30T22:36:42.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.998+0000 7f64cb86a700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64c409f520 con 0x7f64c409e1d0 2023-09-30T22:36:42.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.998+0000 7f64cb86a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f64c409e930 con 0x7f64c4135e50 2023-09-30T22:36:42.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.998+0000 7f64c8e05700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f64c409e1d0 0x7f64c4135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:42.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.998+0000 7f64c9606700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f64c409d6f0 0x7f64c409dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60144/0 (socket says 172.21.15.16:60144) 2023-09-30T22:36:42.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.998+0000 7f64c8e05700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f64c409e1d0 0x7f64c4135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36426/0 (socket says 172.21.15.16:36426) 2023-09-30T22:36:42.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.998+0000 7f64c9606700 1 -- 172.21.15.16:0/3888526201 learned_addr learned my addr 172.21.15.16:0/3888526201 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:36:42.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.998+0000 7f64c9e07700 1 --2- 172.21.15.16:0/3888526201 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f64c4135e50 0x7f64c4138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:42.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.998+0000 7f64c8e05700 1 -- 172.21.15.16:0/3888526201 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f64c4135e50 msgr2=0x7f64c4138230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:42.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.998+0000 7f64c8e05700 1 --2- 172.21.15.16:0/3888526201 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f64c4135e50 0x7f64c4138230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:42.001 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.998+0000 7f64c8e05700 1 -- 172.21.15.16:0/3888526201 >> v1:172.21.15.16:6789/0 conn(0x7f64c409d6f0 legacy=0x7f64c409dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:42.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.999+0000 7f64c8e05700 1 -- 172.21.15.16:0/3888526201 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c409b2d0 con 0x7f64c409e1d0 2023-09-30T22:36:42.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.999+0000 7f64c8e05700 1 --2- 172.21.15.16:0/3888526201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64c409e1d0 0x7f64c4135910 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f64b4009840 tx=0x7f64b4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1956c27cb89a8ac7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:42.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.999+0000 7f64bbfff700 1 -- 172.21.15.16:0/3888526201 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b4004230 con 0x7f64c409e1d0 2023-09-30T22:36:42.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.999+0000 7f64bbfff700 1 -- 172.21.15.16:0/3888526201 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64b4004390 con 0x7f64c409e1d0 2023-09-30T22:36:42.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:41.999+0000 7f64bbfff700 1 -- 172.21.15.16:0/3888526201 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b40036a0 con 0x7f64c409e1d0 2023-09-30T22:36:42.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.000+0000 7f64cb86a700 1 -- 172.21.15.16:0/3888526201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64c409e1d0 msgr2=0x7f64c4135910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:42.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.000+0000 7f64cb86a700 1 --2- 172.21.15.16:0/3888526201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64c409e1d0 0x7f64c4135910 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f64b4009840 tx=0x7f64b4009b50 comp rx=0 tx=0).stop 2023-09-30T22:36:42.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.000+0000 7f64cb86a700 1 -- 172.21.15.16:0/3888526201 shutdown_connections 2023-09-30T22:36:42.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.000+0000 7f64cb86a700 1 --2- 172.21.15.16:0/3888526201 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f64c4135e50 0x7f64c4138230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:42.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.000+0000 7f64cb86a700 1 --2- 172.21.15.16:0/3888526201 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64c409e1d0 0x7f64c4135910 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:42.002 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.000+0000 7f64cb86a700 1 -- 172.21.15.16:0/3888526201 >> 172.21.15.16:0/3888526201 conn(0x7f64c4066f50 msgr2=0x7f64c409f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:42.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.000+0000 7f64cb86a700 1 -- 172.21.15.16:0/3888526201 shutdown_connections 2023-09-30T22:36:42.003 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.000+0000 7f64cb86a700 1 -- 172.21.15.16:0/3888526201 wait complete. 2023-09-30T22:36:42.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.001+0000 7f64cb86a700 1 Processor -- start 2023-09-30T22:36:42.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.001+0000 7f64cb86a700 1 -- start start 2023-09-30T22:36:42.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.002+0000 7f64cb86a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f64c409d6f0 0x7f64c411c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:42.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.002+0000 7f64cb86a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f64c409e1d0 0x7f64c411c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:42.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.002+0000 7f64cb86a700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64c4135e50 0x7f64c411cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:42.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.002+0000 7f64cb86a700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c411fb90 con 0x7f64c4135e50 2023-09-30T22:36:42.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.002+0000 7f64cb86a700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64c411e2a0 con 0x7f64c409d6f0 2023-09-30T22:36:42.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.002+0000 7f64cb86a700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f64c409c770 con 0x7f64c409e1d0 2023-09-30T22:36:42.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.002+0000 7f64c9e07700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64c4135e50 0x7f64c411cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:42.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.002+0000 7f64c9e07700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64c4135e50 0x7f64c411cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36430/0 (socket says 172.21.15.16:36430) 2023-09-30T22:36:42.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.002+0000 7f64c9e07700 1 -- 172.21.15.16:0/4183324457 learned_addr learned my addr 172.21.15.16:0/4183324457 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:42.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.002+0000 7f64c8e05700 1 --2- 172.21.15.16:0/4183324457 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f64c409e1d0 0x7f64c411c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:42.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.002+0000 7f64c9606700 1 --2- 172.21.15.16:0/4183324457 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f64c409d6f0 0x7f64c411c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:42.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.003+0000 7f64c9e07700 1 -- 172.21.15.16:0/4183324457 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f64c409e1d0 msgr2=0x7f64c411c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:42.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.003+0000 7f64c9e07700 1 --2- 172.21.15.16:0/4183324457 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f64c409e1d0 0x7f64c411c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:42.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.003+0000 7f64c9e07700 1 -- 172.21.15.16:0/4183324457 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f64c409d6f0 msgr2=0x7f64c411c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:42.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.003+0000 7f64c9e07700 1 --2- 172.21.15.16:0/4183324457 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f64c409d6f0 0x7f64c411c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:42.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.003+0000 7f64c9e07700 1 -- 172.21.15.16:0/4183324457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64b40094a0 con 0x7f64c4135e50 2023-09-30T22:36:42.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.003+0000 7f64c9e07700 1 --2- 172.21.15.16:0/4183324457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64c4135e50 0x7f64c411cb40 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f64bc009e30 tx=0x7f64bc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:42.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.003+0000 7f64b9ffb700 1 -- 172.21.15.16:0/4183324457 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64bc009ca0 con 0x7f64c4135e50 2023-09-30T22:36:42.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.003+0000 7f64b9ffb700 1 -- 172.21.15.16:0/4183324457 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64bc00f040 con 0x7f64c4135e50 2023-09-30T22:36:42.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.003+0000 7f64b9ffb700 1 -- 172.21.15.16:0/4183324457 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64bc00ce90 con 0x7f64c4135e50 2023-09-30T22:36:42.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.003+0000 7f64cb86a700 1 -- 172.21.15.16:0/4183324457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64c40a1f10 con 0x7f64c4135e50 2023-09-30T22:36:42.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.004+0000 7f64cb86a700 1 -- 172.21.15.16:0/4183324457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64c411f650 con 0x7f64c4135e50 2023-09-30T22:36:42.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.005+0000 7f64b9ffb700 1 -- 172.21.15.16:0/4183324457 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f64bc014480 con 0x7f64c4135e50 2023-09-30T22:36:42.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.005+0000 7f64cb86a700 1 -- 172.21.15.16:0/4183324457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64c411eed0 con 0x7f64c4135e50 2023-09-30T22:36:42.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.005+0000 7f64b9ffb700 1 -- 172.21.15.16:0/4183324457 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f64bc040380 con 0x7f64c4135e50 2023-09-30T22:36:42.015 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.014+0000 7f64b9ffb700 1 -- 172.21.15.16:0/4183324457 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f64bc012930 con 0x7f64c4135e50 2023-09-30T22:36:42.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.056+0000 7f2f531fd700 1 Processor -- start 2023-09-30T22:36:42.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.056+0000 7f2f531fd700 1 -- start start 2023-09-30T22:36:42.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.057+0000 7f2f531fd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f4c133420 0x7f2f4c1337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:42.059 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.057+0000 7f2f531fd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f4c133e30 0x7f2f4c12ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:42.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.057+0000 7f2f531fd700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f4c06b990 con 0x7f2f4c134260 2023-09-30T22:36:42.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.057+0000 7f2f531fd700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4c0a0590 con 0x7f2f4c133420 2023-09-30T22:36:42.060 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.057+0000 7f2f531fd700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4c09fb80 con 0x7f2f4c133e30 2023-09-30T22:36:42.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.057+0000 7f33d46ae700 1 Processor -- start 2023-09-30T22:36:42.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.057+0000 7f33d46ae700 1 -- start start 2023-09-30T22:36:42.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.057+0000 7f2f50f99700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f4c133420 0x7f2f4c1337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:42.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.057+0000 7f2f50f99700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f4c133420 0x7f2f4c1337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:59352/0 (socket says 172.21.15.16:59352) 2023-09-30T22:36:42.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.057+0000 7f2f50f99700 1 -- 172.21.15.16:0/4118112604 learned_addr learned my addr 172.21.15.16:0/4118112604 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:42.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.057+0000 7f2f4bfff700 1 --2- 172.21.15.16:0/4118112604 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f4c133e30 0x7f2f4c12ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:42.062 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.058+0000 7f33d46ae700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f33cc09d6f0 0x7f33cc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:42.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.058+0000 7f2f50f99700 1 -- 172.21.15.16:0/4118112604 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f4c133e30 msgr2=0x7f2f4c12ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:42.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.058+0000 7f2f50f99700 1 --2- 172.21.15.16:0/4118112604 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f4c133e30 0x7f2f4c12ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:42.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.058+0000 7f2f50f99700 1 -- 172.21.15.16:0/4118112604 >> v1:172.21.15.16:6789/0 conn(0x7f2f4c134260 legacy=0x7f2f4c1304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:42.063 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.058+0000 7f2f50f99700 1 -- 172.21.15.16:0/4118112604 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f4c0a0030 con 0x7f2f4c133420 2023-09-30T22:36:42.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.058+0000 7f2f50f99700 1 --2- 172.21.15.16:0/4118112604 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f4c133420 0x7f2f4c1337f0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f2f40009920 tx=0x7f2f40009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=367722adb52fcb77 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:42.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.058+0000 7f2f4b7fe700 1 -- 172.21.15.16:0/4118112604 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f40011830 con 0x7f2f4c133420 2023-09-30T22:36:42.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.058+0000 7f2f4b7fe700 1 -- 172.21.15.16:0/4118112604 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f40011990 con 0x7f2f4c133420 2023-09-30T22:36:42.064 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.058+0000 7f2f4b7fe700 1 -- 172.21.15.16:0/4118112604 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f40011d20 con 0x7f2f4c133420 2023-09-30T22:36:42.065 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.058+0000 7f33d46ae700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f33cc09e000 0x7f33cc130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:42.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.058+0000 7f33d46ae700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f33cc09e410 0x7f33cc130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:42.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.058+0000 7f33d46ae700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33cc09f520 con 0x7f33cc09d6f0 2023-09-30T22:36:42.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.058+0000 7f33d46ae700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33cc09f130 con 0x7f33cc09e000 2023-09-30T22:36:42.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.058+0000 7f33d46ae700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f33cc09e930 con 0x7f33cc09e410 2023-09-30T22:36:42.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.058+0000 7f33d244a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f33cc09d6f0 0x7f33cc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:42.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.058+0000 7f33d2c4b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f33cc09e410 0x7f33cc130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:42.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.059+0000 7f33d2c4b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f33cc09e410 0x7f33cc130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35648/0 (socket says 172.21.15.16:35648) 2023-09-30T22:36:42.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.059+0000 7f33d244a700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f33cc09d6f0 0x7f33cc09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36432/0 (socket says 172.21.15.16:36432) 2023-09-30T22:36:42.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.059+0000 7f33d2c4b700 1 -- 172.21.15.16:0/1266072228 learned_addr learned my addr 172.21.15.16:0/1266072228 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:42.066 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.059+0000 7f33d1c49700 1 --2- 172.21.15.16:0/1266072228 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f33cc09e000 0x7f33cc130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:42.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.059+0000 7f33d244a700 1 -- 172.21.15.16:0/1266072228 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f33cc09e410 msgr2=0x7f33cc130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:42.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.059+0000 7f33d244a700 1 --2- 172.21.15.16:0/1266072228 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f33cc09e410 0x7f33cc130ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:42.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.059+0000 7f33d244a700 1 -- 172.21.15.16:0/1266072228 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f33cc09e000 msgr2=0x7f33cc130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:42.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.059+0000 7f33d244a700 1 --2- 172.21.15.16:0/1266072228 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f33cc09e000 0x7f33cc130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:42.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.059+0000 7f33d244a700 1 -- 172.21.15.16:0/1266072228 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33cc09b2d0 con 0x7f33cc09d6f0 2023-09-30T22:36:42.067 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.059+0000 7f33d244a700 1 --2- 172.21.15.16:0/1266072228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33cc09d6f0 0x7f33cc09dac0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f33c8009870 tx=0x7f33c8009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bff011a1d46dd0e6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:42.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.059+0000 7f2f531fd700 1 -- 172.21.15.16:0/4118112604 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f4c133420 msgr2=0x7f2f4c1337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:42.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.059+0000 7f2f531fd700 1 --2- 172.21.15.16:0/4118112604 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f4c133420 0x7f2f4c1337f0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f2f40009920 tx=0x7f2f40009c30 comp rx=0 tx=0).stop 2023-09-30T22:36:42.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.059+0000 7f2f531fd700 1 -- 172.21.15.16:0/4118112604 shutdown_connections 2023-09-30T22:36:42.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.059+0000 7f2f531fd700 1 --2- 172.21.15.16:0/4118112604 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f4c133420 0x7f2f4c1337f0 secure :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f2f40009920 tx=0x7f2f40009c30 comp rx=0 tx=0).stop 2023-09-30T22:36:42.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.059+0000 7f2f531fd700 1 --2- 172.21.15.16:0/4118112604 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f4c133e30 0x7f2f4c12ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:42.068 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.059+0000 7f2f531fd700 1 -- 172.21.15.16:0/4118112604 >> 172.21.15.16:0/4118112604 conn(0x7f2f4c0a4210 msgr2=0x7f2f4c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:42.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.059+0000 7f33d1448700 1 -- 172.21.15.16:0/1266072228 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c8011830 con 0x7f33cc09d6f0 2023-09-30T22:36:42.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.059+0000 7f33d1448700 1 -- 172.21.15.16:0/1266072228 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33c8011990 con 0x7f33cc09d6f0 2023-09-30T22:36:42.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.059+0000 7f33d1448700 1 -- 172.21.15.16:0/1266072228 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c8011d00 con 0x7f33cc09d6f0 2023-09-30T22:36:42.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.059+0000 7f2f531fd700 1 -- 172.21.15.16:0/4118112604 shutdown_connections 2023-09-30T22:36:42.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.060+0000 7f2f531fd700 1 -- 172.21.15.16:0/4118112604 wait complete. 2023-09-30T22:36:42.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.060+0000 7f2f531fd700 1 Processor -- start 2023-09-30T22:36:42.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.060+0000 7f33d46ae700 1 -- 172.21.15.16:0/1266072228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33cc09d6f0 msgr2=0x7f33cc09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:42.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.060+0000 7f33d46ae700 1 --2- 172.21.15.16:0/1266072228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33cc09d6f0 0x7f33cc09dac0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f33c8009870 tx=0x7f33c8009b80 comp rx=0 tx=0).stop 2023-09-30T22:36:42.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.061+0000 7f2f531fd700 1 -- start start 2023-09-30T22:36:42.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.061+0000 7f33d46ae700 1 -- 172.21.15.16:0/1266072228 shutdown_connections 2023-09-30T22:36:42.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.061+0000 7f33d46ae700 1 --2- 172.21.15.16:0/1266072228 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f33cc09e410 0x7f33cc130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:42.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.061+0000 7f33d46ae700 1 --2- 172.21.15.16:0/1266072228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33cc09d6f0 0x7f33cc09dac0 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:42.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.061+0000 7f33d46ae700 1 --2- 172.21.15.16:0/1266072228 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f33cc09e000 0x7f33cc130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:42.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.061+0000 7f33d46ae700 1 -- 172.21.15.16:0/1266072228 >> 172.21.15.16:0/1266072228 conn(0x7f33cc066f50 msgr2=0x7f33cc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:42.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.061+0000 7f33d46ae700 1 -- 172.21.15.16:0/1266072228 shutdown_connections 2023-09-30T22:36:42.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.061+0000 7f33d46ae700 1 -- 172.21.15.16:0/1266072228 wait complete. 2023-09-30T22:36:42.072 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.061+0000 7f2f531fd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f4c133e30 0x7f2f4c070de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:42.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.062+0000 7f2f531fd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f4c134260 0x7f2f4c071320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:42.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.062+0000 7f2f531fd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f4c071860 0x7f2f4c1c8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:42.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.062+0000 7f2f531fd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4c072220 con 0x7f2f4c133e30 2023-09-30T22:36:42.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.062+0000 7f2f531fd700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4c121290 con 0x7f2f4c134260 2023-09-30T22:36:42.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.062+0000 7f2f531fd700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4c131a70 con 0x7f2f4c071860 2023-09-30T22:36:42.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.062+0000 7f2f4bfff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f4c134260 0x7f2f4c071320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:42.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.062+0000 7f2f4bfff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f4c134260 0x7f2f4c071320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:59370/0 (socket says 172.21.15.16:59370) 2023-09-30T22:36:42.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.062+0000 7f2f4bfff700 1 -- 172.21.15.16:0/1989800566 learned_addr learned my addr 172.21.15.16:0/1989800566 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:42.073 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.062+0000 7f2f5179a700 1 --2- 172.21.15.16:0/1989800566 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f4c071860 0x7f2f4c1c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:42.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.063+0000 7f2f5179a700 1 -- 172.21.15.16:0/1989800566 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f4c134260 msgr2=0x7f2f4c071320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:42.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.063+0000 7f2f5179a700 1 --2- 172.21.15.16:0/1989800566 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f4c134260 0x7f2f4c071320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:42.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.063+0000 7f2f5179a700 1 -- 172.21.15.16:0/1989800566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f4c133e30 msgr2=0x7f2f4c070de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:42.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.063+0000 7f2f5179a700 1 --2- 172.21.15.16:0/1989800566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f4c133e30 0x7f2f4c070de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:42.074 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.063+0000 7f2f5179a700 1 -- 172.21.15.16:0/1989800566 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f40009550 con 0x7f2f4c071860 2023-09-30T22:36:42.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.062+0000 7f33d46ae700 1 Processor -- start 2023-09-30T22:36:42.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.062+0000 7f33d46ae700 1 -- start start 2023-09-30T22:36:42.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.063+0000 7f33d46ae700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33cc09d6f0 0x7f33cc11e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:42.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.063+0000 7f33d46ae700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f33cc09e000 0x7f33cc11e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:42.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.063+0000 7f33d46ae700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f33cc09e410 0x7f33cc11eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:42.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.063+0000 7f33d46ae700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33cc130560 con 0x7f33cc09d6f0 2023-09-30T22:36:42.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.063+0000 7f33d46ae700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33cc09c700 con 0x7f33cc09e410 2023-09-30T22:36:42.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.063+0000 7f33d46ae700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f33cc121c30 con 0x7f33cc09e000 2023-09-30T22:36:42.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.063+0000 7f33d2c4b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f33cc09e410 0x7f33cc11eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:42.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.063+0000 7f33d2c4b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f33cc09e410 0x7f33cc11eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:59382/0 (socket says 172.21.15.16:59382) 2023-09-30T22:36:42.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.063+0000 7f33d2c4b700 1 -- 172.21.15.16:0/3158704315 learned_addr learned my addr 172.21.15.16:0/3158704315 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:42.075 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.063+0000 7f33d1c49700 1 --2- 172.21.15.16:0/3158704315 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f33cc09e000 0x7f33cc11e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:42.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.063+0000 7f2f4bfff700 1 --2- 172.21.15.16:0/1989800566 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f4c134260 0x7f2f4c071320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:42.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.063+0000 7f33d1c49700 1 -- 172.21.15.16:0/3158704315 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f33cc09e410 msgr2=0x7f33cc11eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:42.076 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.063+0000 7f33d1c49700 1 --2- 172.21.15.16:0/3158704315 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f33cc09e410 0x7f33cc11eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:42.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.063+0000 7f33d1c49700 1 -- 172.21.15.16:0/3158704315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33cc09d6f0 msgr2=0x7f33cc11e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:42.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.063+0000 7f33d1c49700 1 --2- 172.21.15.16:0/3158704315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33cc09d6f0 0x7f33cc11e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:42.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.063+0000 7f33d1c49700 1 -- 172.21.15.16:0/3158704315 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33c80094a0 con 0x7f33cc09e000 2023-09-30T22:36:42.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.064+0000 7f33d1c49700 1 --2- 172.21.15.16:0/3158704315 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f33cc09e000 0x7f33cc11e650 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f33bc00e870 tx=0x7f33bc00eb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:42.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.064+0000 7f2f5179a700 1 --2- 172.21.15.16:0/1989800566 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f4c071860 0x7f2f4c1c8660 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f2f4400cb90 tx=0x7f2f4400cea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:42.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.064+0000 7f33c2ffd700 1 -- 172.21.15.16:0/3158704315 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33bc013680 con 0x7f33cc09e000 2023-09-30T22:36:42.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.064+0000 7f2f497fa700 1 -- 172.21.15.16:0/1989800566 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f44003c00 con 0x7f2f4c071860 2023-09-30T22:36:42.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.064+0000 7f33c2ffd700 1 -- 172.21.15.16:0/3158704315 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33bc0137e0 con 0x7f33cc09e000 2023-09-30T22:36:42.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.064+0000 7f33c2ffd700 1 -- 172.21.15.16:0/3158704315 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33bc013b50 con 0x7f33cc09e000 2023-09-30T22:36:42.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.064+0000 7f2f497fa700 1 -- 172.21.15.16:0/1989800566 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f44003d60 con 0x7f2f4c071860 2023-09-30T22:36:42.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.064+0000 7f2f531fd700 1 -- 172.21.15.16:0/1989800566 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f4c120070 con 0x7f2f4c071860 2023-09-30T22:36:42.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.064+0000 7f33d46ae700 1 -- 172.21.15.16:0/3158704315 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33cc1202d0 con 0x7f33cc09e000 2023-09-30T22:36:42.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.064+0000 7f33d46ae700 1 -- 172.21.15.16:0/3158704315 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33cc1213c0 con 0x7f33cc09e000 2023-09-30T22:36:42.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.065+0000 7f33c2ffd700 1 -- 172.21.15.16:0/3158704315 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f33bc013cb0 con 0x7f33cc09e000 2023-09-30T22:36:42.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.064+0000 7f2f531fd700 1 -- 172.21.15.16:0/1989800566 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f4c120da0 con 0x7f2f4c071860 2023-09-30T22:36:42.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.065+0000 7f2f497fa700 1 -- 172.21.15.16:0/1989800566 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f4400aaa0 con 0x7f2f4c071860 2023-09-30T22:36:42.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.065+0000 7f33c2ffd700 1 -- 172.21.15.16:0/3158704315 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f33bc03fb80 con 0x7f33cc09e000 2023-09-30T22:36:42.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.066+0000 7f33d46ae700 1 -- 172.21.15.16:0/3158704315 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33cc074ea0 con 0x7f33cc09e000 2023-09-30T22:36:42.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.066+0000 7f2f531fd700 1 -- 172.21.15.16:0/1989800566 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f2c0052e0 con 0x7f2f4c071860 2023-09-30T22:36:42.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.071+0000 7f2f497fa700 1 -- 172.21.15.16:0/1989800566 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f2f4400ac00 con 0x7f2f4c071860 2023-09-30T22:36:42.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.072+0000 7f2f497fa700 1 -- 172.21.15.16:0/1989800566 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f2f44040220 con 0x7f2f4c071860 2023-09-30T22:36:42.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.072+0000 7f33c2ffd700 1 -- 172.21.15.16:0/3158704315 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f33bc017070 con 0x7f33cc09e000 2023-09-30T22:36:42.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.076+0000 7f2f497fa700 1 -- 172.21.15.16:0/1989800566 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f2f44001c20 con 0x7f2f4c071860 2023-09-30T22:36:42.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.205+0000 7ffb03b10700 1 -- 172.21.15.16:0/605498718 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ffafc074ea0 con 0x7ffafc09e410 2023-09-30T22:36:42.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.206+0000 7ffaf27fc700 1 -- 172.21.15.16:0/605498718 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ffaf4025380 con 0x7ffafc09e410 2023-09-30T22:36:42.207 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:36:42.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.208+0000 7ffb03b10700 1 -- 172.21.15.16:0/605498718 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffafc09e410 msgr2=0x7ffafc122d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:42.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.208+0000 7ffb03b10700 1 --2- 172.21.15.16:0/605498718 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffafc09e410 0x7ffafc122d30 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7ffaf400b370 tx=0x7ffaf4004ad0 comp rx=0 tx=0).stop 2023-09-30T22:36:42.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.208+0000 7ffb03b10700 1 -- 172.21.15.16:0/605498718 shutdown_connections 2023-09-30T22:36:42.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.208+0000 7ffb03b10700 1 --2- 172.21.15.16:0/605498718 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ffafc09e410 0x7ffafc122d30 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:42.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.208+0000 7ffb03b10700 1 --2- 172.21.15.16:0/605498718 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ffafc09d6f0 0x7ffafc1222b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:42.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.208+0000 7ffb03b10700 1 --2- 172.21.15.16:0/605498718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffafc09e000 0x7ffafc1227f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:42.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.208+0000 7ffb03b10700 1 -- 172.21.15.16:0/605498718 >> 172.21.15.16:0/605498718 conn(0x7ffafc066f50 msgr2=0x7ffafc133c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:42.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.208+0000 7ffb03b10700 1 -- 172.21.15.16:0/605498718 shutdown_connections 2023-09-30T22:36:42.210 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.208+0000 7ffb03b10700 1 -- 172.21.15.16:0/605498718 wait complete. 2023-09-30T22:36:42.228 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:36:42.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.279+0000 7f33d46ae700 1 -- 172.21.15.16:0/3158704315 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f33cc0722f0 con 0x7f33cc09e000 2023-09-30T22:36:42.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.280+0000 7f33c2ffd700 1 -- 172.21.15.16:0/3158704315 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f33bc01abe0 con 0x7f33cc09e000 2023-09-30T22:36:42.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.280+0000 7f64cb86a700 1 -- 172.21.15.16:0/4183324457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f64c4071c40 con 0x7f64c4135e50 2023-09-30T22:36:42.282 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:36:42.283 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.280+0000 7f64b9ffb700 1 -- 172.21.15.16:0/4183324457 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f64bc012340 con 0x7f64c4135e50 2023-09-30T22:36:42.283 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:36:42.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.282+0000 7f33d46ae700 1 -- 172.21.15.16:0/3158704315 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f33cc09e000 msgr2=0x7f33cc11e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:42.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.282+0000 7f33d46ae700 1 --2- 172.21.15.16:0/3158704315 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f33cc09e000 0x7f33cc11e650 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f33bc00e870 tx=0x7f33bc00eb80 comp rx=0 tx=0).stop 2023-09-30T22:36:42.284 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.282+0000 7f33d46ae700 1 -- 172.21.15.16:0/3158704315 shutdown_connections 2023-09-30T22:36:42.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.282+0000 7f33d46ae700 1 --2- 172.21.15.16:0/3158704315 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f33cc09e410 0x7f33cc11eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:42.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.282+0000 7f33d46ae700 1 --2- 172.21.15.16:0/3158704315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f33cc09d6f0 0x7f33cc11e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:42.285 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.282+0000 7f33d46ae700 1 --2- 172.21.15.16:0/3158704315 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f33cc09e000 0x7f33cc11e650 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:42.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.282+0000 7f33d46ae700 1 -- 172.21.15.16:0/3158704315 >> 172.21.15.16:0/3158704315 conn(0x7f33cc066f50 msgr2=0x7f33cc06a770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:42.286 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.282+0000 7f64cb86a700 1 -- 172.21.15.16:0/4183324457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64c4135e50 msgr2=0x7f64c411cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:42.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.282+0000 7f64cb86a700 1 --2- 172.21.15.16:0/4183324457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64c4135e50 0x7f64c411cb40 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f64bc009e30 tx=0x7f64bc00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:36:42.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.283+0000 7f64cb86a700 1 -- 172.21.15.16:0/4183324457 shutdown_connections 2023-09-30T22:36:42.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.283+0000 7f64cb86a700 1 --2- 172.21.15.16:0/4183324457 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f64c409d6f0 0x7f64c411c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:42.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.283+0000 7f64cb86a700 1 --2- 172.21.15.16:0/4183324457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64c4135e50 0x7f64c411cb40 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:42.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.283+0000 7f64cb86a700 1 --2- 172.21.15.16:0/4183324457 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f64c409e1d0 0x7f64c411c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:42.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.283+0000 7f64cb86a700 1 -- 172.21.15.16:0/4183324457 >> 172.21.15.16:0/4183324457 conn(0x7f64c4066f50 msgr2=0x7f64c406ed80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:42.287 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.283+0000 7f33d46ae700 1 -- 172.21.15.16:0/3158704315 shutdown_connections 2023-09-30T22:36:42.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.283+0000 7f33d46ae700 1 -- 172.21.15.16:0/3158704315 wait complete. 2023-09-30T22:36:42.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.283+0000 7f64cb86a700 1 -- 172.21.15.16:0/4183324457 shutdown_connections 2023-09-30T22:36:42.288 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.283+0000 7f64cb86a700 1 -- 172.21.15.16:0/4183324457 wait complete. 2023-09-30T22:36:42.305 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:36:42.306 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:36:42.319 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.318+0000 7f2f531fd700 1 -- 172.21.15.16:0/1989800566 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2f2c005860 con 0x7f2f4c071860 2023-09-30T22:36:42.320 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.318+0000 7f2f497fa700 1 -- 172.21.15.16:0/1989800566 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f2f44011380 con 0x7f2f4c071860 2023-09-30T22:36:42.320 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:36:42.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.321+0000 7f2f531fd700 1 -- 172.21.15.16:0/1989800566 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f4c071860 msgr2=0x7f2f4c1c8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:42.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.321+0000 7f2f531fd700 1 --2- 172.21.15.16:0/1989800566 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f4c071860 0x7f2f4c1c8660 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f2f4400cb90 tx=0x7f2f4400cea0 comp rx=0 tx=0).stop 2023-09-30T22:36:42.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.321+0000 7f2f531fd700 1 -- 172.21.15.16:0/1989800566 shutdown_connections 2023-09-30T22:36:42.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.321+0000 7f2f531fd700 1 --2- 172.21.15.16:0/1989800566 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f2f4c071860 0x7f2f4c1c8660 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:42.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.321+0000 7f2f531fd700 1 --2- 172.21.15.16:0/1989800566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f4c133e30 0x7f2f4c070de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:42.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.321+0000 7f2f531fd700 1 --2- 172.21.15.16:0/1989800566 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f2f4c134260 0x7f2f4c071320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:42.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.321+0000 7f2f531fd700 1 -- 172.21.15.16:0/1989800566 >> 172.21.15.16:0/1989800566 conn(0x7f2f4c0a4210 msgr2=0x7f2f4c0a0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:42.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.321+0000 7f2f531fd700 1 -- 172.21.15.16:0/1989800566 shutdown_connections 2023-09-30T22:36:42.323 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:42.321+0000 7f2f531fd700 1 -- 172.21.15.16:0/1989800566 wait complete. 2023-09-30T22:36:42.336 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:36:43.230 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:36:43.307 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:36:43.308 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:36:43.337 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:36:43.379 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.378+0000 7fb052f67700 1 Processor -- start 2023-09-30T22:36:43.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.378+0000 7fb052f67700 1 -- start start 2023-09-30T22:36:43.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.378+0000 7fb052f67700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb04c09d6f0 0x7fb04c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:43.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.378+0000 7fb052f67700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb04c09e000 0x7fb04c130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:43.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.378+0000 7fb052f67700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb04c09e410 0x7fb04c130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:43.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.378+0000 7fb052f67700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb04c09f520 con 0x7fb04c09e410 2023-09-30T22:36:43.380 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.378+0000 7fb052f67700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb04c09f130 con 0x7fb04c09e000 2023-09-30T22:36:43.381 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.378+0000 7fb052f67700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb04c09e930 con 0x7fb04c09d6f0 2023-09-30T22:36:43.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.380+0000 7fb051504700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb04c09e410 0x7fb04c130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:43.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.380+0000 7fb051504700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb04c09e410 0x7fb04c130ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36460/0 (socket says 172.21.15.16:36460) 2023-09-30T22:36:43.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.380+0000 7fb051504700 1 -- 172.21.15.16:0/1114657656 learned_addr learned my addr 172.21.15.16:0/1114657656 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:43.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.380+0000 7fb050d03700 1 --2- 172.21.15.16:0/1114657656 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb04c09d6f0 0x7fb04c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:43.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.380+0000 7fb051504700 1 -- 172.21.15.16:0/1114657656 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb04c09d6f0 msgr2=0x7fb04c09dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:43.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.380+0000 7fb051504700 1 --2- 172.21.15.16:0/1114657656 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb04c09d6f0 0x7fb04c09dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:43.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.380+0000 7fb051504700 1 -- 172.21.15.16:0/1114657656 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb04c09e000 msgr2=0x7fb04c130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:43.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.380+0000 7fb051504700 1 --2- 172.21.15.16:0/1114657656 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb04c09e000 0x7fb04c130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:43.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.380+0000 7fb051504700 1 -- 172.21.15.16:0/1114657656 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb04c09b2d0 con 0x7fb04c09e410 2023-09-30T22:36:43.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.380+0000 7fb051504700 1 --2- 172.21.15.16:0/1114657656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb04c09e410 0x7fb04c130ba0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fb04400bf60 tx=0x7fb04400bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7db20c4e71c5835e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:43.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.381+0000 7fb04b7fe700 1 -- 172.21.15.16:0/1114657656 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04400b4e0 con 0x7fb04c09e410 2023-09-30T22:36:43.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.381+0000 7fb04b7fe700 1 -- 172.21.15.16:0/1114657656 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb04400da00 con 0x7fb04c09e410 2023-09-30T22:36:43.383 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.381+0000 7fb04b7fe700 1 -- 172.21.15.16:0/1114657656 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04400ddb0 con 0x7fb04c09e410 2023-09-30T22:36:43.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.381+0000 7fb052f67700 1 -- 172.21.15.16:0/1114657656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb04c09e410 msgr2=0x7fb04c130ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:43.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.381+0000 7fb052f67700 1 --2- 172.21.15.16:0/1114657656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb04c09e410 0x7fb04c130ba0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fb04400bf60 tx=0x7fb04400bf90 comp rx=0 tx=0).stop 2023-09-30T22:36:43.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.381+0000 7fb052f67700 1 -- 172.21.15.16:0/1114657656 shutdown_connections 2023-09-30T22:36:43.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.381+0000 7fb052f67700 1 --2- 172.21.15.16:0/1114657656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb04c09e410 0x7fb04c130ba0 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:43.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.381+0000 7fb052f67700 1 --2- 172.21.15.16:0/1114657656 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb04c09d6f0 0x7fb04c09dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:43.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.381+0000 7fb052f67700 1 --2- 172.21.15.16:0/1114657656 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb04c09e000 0x7fb04c130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:43.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.381+0000 7fb052f67700 1 -- 172.21.15.16:0/1114657656 >> 172.21.15.16:0/1114657656 conn(0x7fb04c066f50 msgr2=0x7fb04c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:43.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.381+0000 7fb052f67700 1 -- 172.21.15.16:0/1114657656 shutdown_connections 2023-09-30T22:36:43.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.382+0000 7fb052f67700 1 -- 172.21.15.16:0/1114657656 wait complete. 2023-09-30T22:36:43.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.382+0000 7fb052f67700 1 Processor -- start 2023-09-30T22:36:43.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.383+0000 7fb052f67700 1 -- start start 2023-09-30T22:36:43.387 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.383+0000 7fb052f67700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb04c09d6f0 0x7fb04c11df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:43.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.383+0000 7fb052f67700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb04c09e000 0x7fb04c11e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:43.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.383+0000 7fb052f67700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb04c09e410 0x7fb04c11ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:43.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.383+0000 7fb052f67700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04c130360 con 0x7fb04c09d6f0 2023-09-30T22:36:43.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.383+0000 7fb052f67700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb04c120de0 con 0x7fb04c09e000 2023-09-30T22:36:43.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.383+0000 7fb052f67700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb04c1217c0 con 0x7fb04c09e410 2023-09-30T22:36:43.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.383+0000 7fb04bfff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb04c09e000 0x7fb04c11e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:43.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.384+0000 7fb04bfff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb04c09e000 0x7fb04c11e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:59400/0 (socket says 172.21.15.16:59400) 2023-09-30T22:36:43.388 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.384+0000 7fb04bfff700 1 -- 172.21.15.16:0/2120386631 learned_addr learned my addr 172.21.15.16:0/2120386631 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:43.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.384+0000 7fb050d03700 1 --2- 172.21.15.16:0/2120386631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb04c09d6f0 0x7fb04c11df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:43.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.384+0000 7fb050d03700 1 -- 172.21.15.16:0/2120386631 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb04c09e410 msgr2=0x7fb04c11ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:43.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.384+0000 7fb050d03700 1 --2- 172.21.15.16:0/2120386631 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb04c09e410 0x7fb04c11ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:43.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.384+0000 7fb050d03700 1 -- 172.21.15.16:0/2120386631 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb04c09e000 msgr2=0x7fb04c11e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:43.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.384+0000 7fb050d03700 1 --2- 172.21.15.16:0/2120386631 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb04c09e000 0x7fb04c11e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:43.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.384+0000 7fb050d03700 1 -- 172.21.15.16:0/2120386631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb04400bd40 con 0x7fb04c09d6f0 2023-09-30T22:36:43.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.384+0000 7fb04bfff700 1 --2- 172.21.15.16:0/2120386631 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb04c09e000 0x7fb04c11e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:43.389 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.384+0000 7fb050d03700 1 --2- 172.21.15.16:0/2120386631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb04c09d6f0 0x7fb04c11df80 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7fb04000e900 tx=0x7fb04000ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:43.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.384+0000 7fb0497fa700 1 -- 172.21.15.16:0/2120386631 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04000ee50 con 0x7fb04c09d6f0 2023-09-30T22:36:43.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.385+0000 7fb0497fa700 1 -- 172.21.15.16:0/2120386631 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb04000c850 con 0x7fb04c09d6f0 2023-09-30T22:36:43.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.385+0000 7fb052f67700 1 -- 172.21.15.16:0/2120386631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb04c120150 con 0x7fb04c09d6f0 2023-09-30T22:36:43.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.385+0000 7fb0497fa700 1 -- 172.21.15.16:0/2120386631 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04000cbc0 con 0x7fb04c09d6f0 2023-09-30T22:36:43.390 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.385+0000 7fb052f67700 1 -- 172.21.15.16:0/2120386631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb04c09c700 con 0x7fb04c09d6f0 2023-09-30T22:36:43.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.386+0000 7fb0497fa700 1 -- 172.21.15.16:0/2120386631 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb04000cd20 con 0x7fb04c09d6f0 2023-09-30T22:36:43.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.386+0000 7fb0497fa700 1 -- 172.21.15.16:0/2120386631 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fb04003fa90 con 0x7fb04c09d6f0 2023-09-30T22:36:43.391 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.386+0000 7fb052f67700 1 -- 172.21.15.16:0/2120386631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb04c11ac20 con 0x7fb04c09d6f0 2023-09-30T22:36:43.392 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.391+0000 7fb0497fa700 1 -- 172.21.15.16:0/2120386631 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb04001adc0 con 0x7fb04c09d6f0 2023-09-30T22:36:43.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.452+0000 7f1873ca3700 1 Processor -- start 2023-09-30T22:36:43.455 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.453+0000 7f1873ca3700 1 -- start start 2023-09-30T22:36:43.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.453+0000 7f1873ca3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f186c136830 0x7f186c138c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:43.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.453+0000 7f1873ca3700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f186c139150 0x7f186c13b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:43.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.453+0000 7f1873ca3700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f186c06f720 con 0x7f186c1338c0 2023-09-30T22:36:43.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.453+0000 7f1873ca3700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f186c070c80 con 0x7f186c139150 2023-09-30T22:36:43.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.453+0000 7f1873ca3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f186c06fce0 con 0x7f186c136830 2023-09-30T22:36:43.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.454+0000 7f1871a3f700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f186c1338c0 0x7f186c133c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60170/0 (socket says 172.21.15.16:60170) 2023-09-30T22:36:43.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.454+0000 7f1871a3f700 1 -- 172.21.15.16:0/623092500 learned_addr learned my addr 172.21.15.16:0/623092500 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:36:43.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.454+0000 7f1872240700 1 --2- 172.21.15.16:0/623092500 >> v2:172.21.15.16:3300/0 conn(0x7f186c139150 0x7f186c13b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:43.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.454+0000 7f187123e700 1 --2- 172.21.15.16:0/623092500 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f186c136830 0x7f186c138c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:43.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.454+0000 7f1872240700 1 -- 172.21.15.16:0/623092500 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f186c136830 msgr2=0x7f186c138c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:43.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.454+0000 7f1872240700 1 --2- 172.21.15.16:0/623092500 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f186c136830 0x7f186c138c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:43.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.454+0000 7f1872240700 1 -- 172.21.15.16:0/623092500 >> v1:172.21.15.16:6789/0 conn(0x7f186c1338c0 legacy=0x7f186c133c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:43.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.454+0000 7f1872240700 1 -- 172.21.15.16:0/623092500 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f186c1336a0 con 0x7f186c139150 2023-09-30T22:36:43.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.454+0000 7f1872240700 1 --2- 172.21.15.16:0/623092500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f186c139150 0x7f186c13b530 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f1864009d90 tx=0x7f186400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eee29a67028f111 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:43.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.454+0000 7f1870a3d700 1 -- 172.21.15.16:0/623092500 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f186400e070 con 0x7f186c139150 2023-09-30T22:36:43.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.455+0000 7f1870a3d700 1 -- 172.21.15.16:0/623092500 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f186400bbd0 con 0x7f186c139150 2023-09-30T22:36:43.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.455+0000 7f1870a3d700 1 -- 172.21.15.16:0/623092500 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1864012430 con 0x7f186c139150 2023-09-30T22:36:43.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.455+0000 7f1873ca3700 1 -- 172.21.15.16:0/623092500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f186c139150 msgr2=0x7f186c13b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:43.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.455+0000 7f1873ca3700 1 --2- 172.21.15.16:0/623092500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f186c139150 0x7f186c13b530 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f1864009d90 tx=0x7f186400b330 comp rx=0 tx=0).stop 2023-09-30T22:36:43.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.456+0000 7f1873ca3700 1 -- 172.21.15.16:0/623092500 shutdown_connections 2023-09-30T22:36:43.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.456+0000 7f1873ca3700 1 --2- 172.21.15.16:0/623092500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f186c139150 0x7f186c13b530 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:43.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.456+0000 7f1873ca3700 1 --2- 172.21.15.16:0/623092500 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f186c136830 0x7f186c138c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:43.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.456+0000 7f1873ca3700 1 -- 172.21.15.16:0/623092500 >> 172.21.15.16:0/623092500 conn(0x7f186c0a2aa0 msgr2=0x7f186c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:43.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.456+0000 7f1873ca3700 1 -- 172.21.15.16:0/623092500 shutdown_connections 2023-09-30T22:36:43.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.456+0000 7f1873ca3700 1 -- 172.21.15.16:0/623092500 wait complete. 2023-09-30T22:36:43.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.457+0000 7f1873ca3700 1 Processor -- start 2023-09-30T22:36:43.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.457+0000 7f1873ca3700 1 -- start start 2023-09-30T22:36:43.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.458+0000 7f5a9f59e700 1 Processor -- start 2023-09-30T22:36:43.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.458+0000 7f5a9f59e700 1 -- start start 2023-09-30T22:36:43.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.458+0000 7f1873ca3700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f186c1338c0 0x7f186c1c85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:43.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.458+0000 7f1873ca3700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f186c136830 0x7f186c1c8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:43.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.458+0000 7f1873ca3700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f186c139150 0x7f186c119e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:43.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.458+0000 7f1873ca3700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f186c1360e0 con 0x7f186c1338c0 2023-09-30T22:36:43.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.458+0000 7f1873ca3700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f186c1be2d0 con 0x7f186c139150 2023-09-30T22:36:43.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.458+0000 7f1873ca3700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f186c11d920 con 0x7f186c136830 2023-09-30T22:36:43.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.458+0000 7f187123e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f186c136830 0x7f186c1c8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:43.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.458+0000 7f187123e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f186c136830 0x7f186c1c8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35686/0 (socket says 172.21.15.16:35686) 2023-09-30T22:36:43.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.458+0000 7f187123e700 1 -- 172.21.15.16:0/852421213 learned_addr learned my addr 172.21.15.16:0/852421213 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:43.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.459+0000 7f5a9f59e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5aa009e670 0x7f5aa009ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:43.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.459+0000 7f5a9f59e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5aa009f8a0 0x7f5aa009ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:43.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.459+0000 7f5a9f59e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5aa0133960 0x7f5aa0133db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:43.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.459+0000 7f187123e700 1 -- 172.21.15.16:0/852421213 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f186c139150 msgr2=0x7f186c119e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:43.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.459+0000 7f187123e700 1 --2- 172.21.15.16:0/852421213 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f186c139150 0x7f186c119e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:43.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.459+0000 7f187123e700 1 -- 172.21.15.16:0/852421213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f186c1338c0 msgr2=0x7f186c1c85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:43.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.459+0000 7f187123e700 1 --2- 172.21.15.16:0/852421213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f186c1338c0 0x7f186c1c85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:43.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.459+0000 7f187123e700 1 -- 172.21.15.16:0/852421213 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1864009a70 con 0x7f186c136830 2023-09-30T22:36:43.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.459+0000 7f5a9f59e700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5aa009f4c0 con 0x7f5aa0133960 2023-09-30T22:36:43.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.459+0000 7f5a9f59e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa00a0240 con 0x7f5aa009e670 2023-09-30T22:36:43.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.459+0000 7f5a9f59e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa006c9a0 con 0x7f5aa009f8a0 2023-09-30T22:36:43.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.459+0000 7f5a9dd9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5aa009f8a0 0x7f5aa009ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:43.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.459+0000 7f187123e700 1 --2- 172.21.15.16:0/852421213 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f186c136830 0x7f186c1c8af0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f185c00e810 tx=0x7f185c00ebd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:43.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.459+0000 7f5a9dd9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5aa009f8a0 0x7f5aa009ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35694/0 (socket says 172.21.15.16:35694) 2023-09-30T22:36:43.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.459+0000 7f5a9dd9b700 1 -- 172.21.15.16:0/3820429100 learned_addr learned my addr 172.21.15.16:0/3820429100 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:43.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.459+0000 7f18627fc700 1 -- 172.21.15.16:0/852421213 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185c013610 con 0x7f186c136830 2023-09-30T22:36:43.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.459+0000 7f1873ca3700 1 -- 172.21.15.16:0/852421213 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f186c12fcd0 con 0x7f186c136830 2023-09-30T22:36:43.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.459+0000 7f1873ca3700 1 -- 172.21.15.16:0/852421213 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f186c11e0b0 con 0x7f186c136830 2023-09-30T22:36:43.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.459+0000 7f18627fc700 1 -- 172.21.15.16:0/852421213 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f185c013770 con 0x7f186c136830 2023-09-30T22:36:43.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.459+0000 7f18627fc700 1 -- 172.21.15.16:0/852421213 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185c013ae0 con 0x7f186c136830 2023-09-30T22:36:43.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.459+0000 7f5a9dd9b700 1 -- 172.21.15.16:0/3820429100 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5aa009e670 msgr2=0x7f5aa009ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:43.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.459+0000 7f5a9dd9b700 1 --2- 172.21.15.16:0/3820429100 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5aa009e670 0x7f5aa009ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:43.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.459+0000 7f5a9dd9b700 1 -- 172.21.15.16:0/3820429100 >> v2:172.21.15.16:3300/0 conn(0x7f5aa0133960 msgr2=0x7f5aa0133db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:43.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.459+0000 7f5a9dd9b700 1 --2- 172.21.15.16:0/3820429100 >> v2:172.21.15.16:3300/0 conn(0x7f5aa0133960 0x7f5aa0133db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:43.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.459+0000 7f5a9dd9b700 1 -- 172.21.15.16:0/3820429100 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa006b880 con 0x7f5aa009f8a0 2023-09-30T22:36:43.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.461+0000 7f1873ca3700 1 -- 172.21.15.16:0/852421213 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18540052e0 con 0x7f186c136830 2023-09-30T22:36:43.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.461+0000 7f5a9dd9b700 1 --2- 172.21.15.16:0/3820429100 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5aa009f8a0 0x7f5aa009ef80 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f5a940098f0 tx=0x7f5a94009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f982aa7b7d064c45 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:43.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.461+0000 7f18627fc700 1 -- 172.21.15.16:0/852421213 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f185c013c40 con 0x7f186c136830 2023-09-30T22:36:43.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.461+0000 7f18627fc700 1 -- 172.21.15.16:0/852421213 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f185c03f9a0 con 0x7f186c136830 2023-09-30T22:36:43.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.466+0000 7f5a9d59a700 1 -- 172.21.15.16:0/3820429100 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9400b8a0 con 0x7f5aa009f8a0 2023-09-30T22:36:43.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.466+0000 7f5a9d59a700 1 -- 172.21.15.16:0/3820429100 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a9400ba00 con 0x7f5aa009f8a0 2023-09-30T22:36:43.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.466+0000 7f5a9d59a700 1 -- 172.21.15.16:0/3820429100 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9400bd90 con 0x7f5aa009f8a0 2023-09-30T22:36:43.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.466+0000 7f18627fc700 1 -- 172.21.15.16:0/852421213 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f185c01c7d0 con 0x7f186c136830 2023-09-30T22:36:43.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.467+0000 7f5a9f59e700 1 -- 172.21.15.16:0/3820429100 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5aa009f8a0 msgr2=0x7f5aa009ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:43.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.467+0000 7f5a9f59e700 1 --2- 172.21.15.16:0/3820429100 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5aa009f8a0 0x7f5aa009ef80 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f5a940098f0 tx=0x7f5a94009c00 comp rx=0 tx=0).stop 2023-09-30T22:36:43.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.467+0000 7f5a9f59e700 1 -- 172.21.15.16:0/3820429100 shutdown_connections 2023-09-30T22:36:43.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.467+0000 7f5a9f59e700 1 --2- 172.21.15.16:0/3820429100 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5aa009e670 0x7f5aa009ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:43.478 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.467+0000 7f5a9f59e700 1 --2- 172.21.15.16:0/3820429100 >> v2:172.21.15.16:3300/0 conn(0x7f5aa0133960 0x7f5aa0133db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:43.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.467+0000 7f5a9f59e700 1 --2- 172.21.15.16:0/3820429100 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5aa009f8a0 0x7f5aa009ef80 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:43.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.467+0000 7f5a9f59e700 1 -- 172.21.15.16:0/3820429100 >> 172.21.15.16:0/3820429100 conn(0x7f5aa00a0c90 msgr2=0x7f5aa00a1090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:43.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.467+0000 7f5a9f59e700 1 -- 172.21.15.16:0/3820429100 shutdown_connections 2023-09-30T22:36:43.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.467+0000 7f5a9f59e700 1 -- 172.21.15.16:0/3820429100 wait complete. 2023-09-30T22:36:43.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.468+0000 7f5a9f59e700 1 Processor -- start 2023-09-30T22:36:43.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.468+0000 7f5a9f59e700 1 -- start start 2023-09-30T22:36:43.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.469+0000 7f5a9f59e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5aa009e670 0x7f5aa01200d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:43.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.469+0000 7f5a9f59e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5aa009f8a0 0x7f5aa0120610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:43.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.469+0000 7f5a9f59e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5aa0133960 0x7f5aa0120b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:43.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.469+0000 7f5a9f59e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa0123380 con 0x7f5aa0133960 2023-09-30T22:36:43.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.469+0000 7f5a9f59e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa00a0480 con 0x7f5aa009e670 2023-09-30T22:36:43.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.469+0000 7f5a9f59e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa0071c30 con 0x7f5aa009f8a0 2023-09-30T22:36:43.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.469+0000 7f5a9e59c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5aa009e670 0x7f5aa01200d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:43.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.469+0000 7f5a9e59c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5aa009e670 0x7f5aa01200d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:59452/0 (socket says 172.21.15.16:59452) 2023-09-30T22:36:43.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.469+0000 7f5a9e59c700 1 -- 172.21.15.16:0/13656983 learned_addr learned my addr 172.21.15.16:0/13656983 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:43.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.469+0000 7f5a9ed9d700 1 --2- 172.21.15.16:0/13656983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5aa0133960 0x7f5aa0120b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:43.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.469+0000 7f5a9dd9b700 1 --2- 172.21.15.16:0/13656983 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5aa009f8a0 0x7f5aa0120610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:43.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.470+0000 7f5a9ed9d700 1 -- 172.21.15.16:0/13656983 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5aa009f8a0 msgr2=0x7f5aa0120610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:43.480 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.470+0000 7f5a9ed9d700 1 --2- 172.21.15.16:0/13656983 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5aa009f8a0 0x7f5aa0120610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:43.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.470+0000 7f5a9ed9d700 1 -- 172.21.15.16:0/13656983 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5aa009e670 msgr2=0x7f5aa01200d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:43.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.470+0000 7f5a9ed9d700 1 --2- 172.21.15.16:0/13656983 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5aa009e670 0x7f5aa01200d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:43.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.470+0000 7f5a9ed9d700 1 -- 172.21.15.16:0/13656983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a94009550 con 0x7f5aa0133960 2023-09-30T22:36:43.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.470+0000 7f5a9ed9d700 1 --2- 172.21.15.16:0/13656983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5aa0133960 0x7f5aa0120b50 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f5a9800be20 tx=0x7f5a98009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:43.481 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.470+0000 7f5a8effd700 1 -- 172.21.15.16:0/13656983 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9800cb90 con 0x7f5aa0133960 2023-09-30T22:36:43.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.470+0000 7f5a8effd700 1 -- 172.21.15.16:0/13656983 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a9800cd10 con 0x7f5aa0133960 2023-09-30T22:36:43.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.470+0000 7f5a8effd700 1 -- 172.21.15.16:0/13656983 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a98014680 con 0x7f5aa0133960 2023-09-30T22:36:43.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.470+0000 7f5a9f59e700 1 -- 172.21.15.16:0/13656983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aa0121ce0 con 0x7f5aa0133960 2023-09-30T22:36:43.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.471+0000 7f5a9f59e700 1 -- 172.21.15.16:0/13656983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aa01240f0 con 0x7f5aa0133960 2023-09-30T22:36:43.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.472+0000 7f5a8effd700 1 -- 172.21.15.16:0/13656983 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5a9801d3b0 con 0x7f5aa0133960 2023-09-30T22:36:43.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.472+0000 7f5a9f59e700 1 -- 172.21.15.16:0/13656983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5aa00a3410 con 0x7f5aa0133960 2023-09-30T22:36:43.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.472+0000 7f5a8effd700 1 -- 172.21.15.16:0/13656983 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f5a98006e40 con 0x7f5aa0133960 2023-09-30T22:36:43.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.481+0000 7f5a8effd700 1 -- 172.21.15.16:0/13656983 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5a9801c360 con 0x7f5aa0133960 2023-09-30T22:36:43.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.497+0000 7fdb03152700 1 Processor -- start 2023-09-30T22:36:43.499 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.497+0000 7fdb03152700 1 -- start start 2023-09-30T22:36:43.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.498+0000 7fdb03152700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdafc09d6f0 0x7fdafc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:43.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.498+0000 7fdb03152700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdafc09e410 0x7fdafc134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:43.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.498+0000 7fdb03152700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdafc09b120 con 0x7fdafc09e000 2023-09-30T22:36:43.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.498+0000 7fdb03152700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdafc06b460 con 0x7fdafc09e410 2023-09-30T22:36:43.500 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.498+0000 7fdb03152700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdafc09a7e0 con 0x7fdafc09d6f0 2023-09-30T22:36:43.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.498+0000 7fdb00eee700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdafc09d6f0 0x7fdafc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:43.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.498+0000 7fdb00eee700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdafc09d6f0 0x7fdafc09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35708/0 (socket says 172.21.15.16:35708) 2023-09-30T22:36:43.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.498+0000 7fdb00eee700 1 -- 172.21.15.16:0/553434889 learned_addr learned my addr 172.21.15.16:0/553434889 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:43.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.498+0000 7fdb016ef700 1 --2- 172.21.15.16:0/553434889 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdafc09e410 0x7fdafc134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:43.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.499+0000 7fdb00eee700 1 -- 172.21.15.16:0/553434889 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdafc09e410 msgr2=0x7fdafc134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:43.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.499+0000 7fdb00eee700 1 --2- 172.21.15.16:0/553434889 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdafc09e410 0x7fdafc134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:43.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.499+0000 7fdb00eee700 1 -- 172.21.15.16:0/553434889 >> v1:172.21.15.16:6789/0 conn(0x7fdafc09e000 legacy=0x7fdafc1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:43.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.499+0000 7fdb00eee700 1 -- 172.21.15.16:0/553434889 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdafc09a500 con 0x7fdafc09d6f0 2023-09-30T22:36:43.501 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.499+0000 7fdb00eee700 1 --2- 172.21.15.16:0/553434889 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdafc09d6f0 0x7fdafc09dac0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0009950 tx=0x7fdaf0009c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4c304d96bdd27eb server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:43.502 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.499+0000 7fdafb7fe700 1 -- 172.21.15.16:0/553434889 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf0004230 con 0x7fdafc09d6f0 2023-09-30T22:36:43.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.499+0000 7fdafb7fe700 1 -- 172.21.15.16:0/553434889 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaf0004390 con 0x7fdafc09d6f0 2023-09-30T22:36:43.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.499+0000 7fdafb7fe700 1 -- 172.21.15.16:0/553434889 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf00036a0 con 0x7fdafc09d6f0 2023-09-30T22:36:43.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.500+0000 7fdb03152700 1 -- 172.21.15.16:0/553434889 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdafc09d6f0 msgr2=0x7fdafc09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:43.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.500+0000 7fdb03152700 1 --2- 172.21.15.16:0/553434889 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdafc09d6f0 0x7fdafc09dac0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0009950 tx=0x7fdaf0009c60 comp rx=0 tx=0).stop 2023-09-30T22:36:43.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.500+0000 7fdb03152700 1 -- 172.21.15.16:0/553434889 shutdown_connections 2023-09-30T22:36:43.503 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.500+0000 7fdb03152700 1 --2- 172.21.15.16:0/553434889 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdafc09e410 0x7fdafc134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:43.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.500+0000 7fdb03152700 1 --2- 172.21.15.16:0/553434889 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdafc09d6f0 0x7fdafc09dac0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:43.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.500+0000 7fdb03152700 1 -- 172.21.15.16:0/553434889 >> 172.21.15.16:0/553434889 conn(0x7fdafc066f50 msgr2=0x7fdafc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:43.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.500+0000 7fdb03152700 1 -- 172.21.15.16:0/553434889 shutdown_connections 2023-09-30T22:36:43.504 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.500+0000 7fdb03152700 1 -- 172.21.15.16:0/553434889 wait complete. 2023-09-30T22:36:43.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.502+0000 7fdb03152700 1 Processor -- start 2023-09-30T22:36:43.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.502+0000 7fdb03152700 1 -- start start 2023-09-30T22:36:43.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.503+0000 7fdb03152700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdafc09d6f0 0x7fdafc1225a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:43.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.503+0000 7fdb03152700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdafc09e000 0x7fdafc122ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:43.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.503+0000 7fdb03152700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdafc09e410 0x7fdafc123020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:43.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.503+0000 7fdb03152700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdafc125d60 con 0x7fdafc09d6f0 2023-09-30T22:36:43.507 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.503+0000 7fdb03152700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdafc1337b0 con 0x7fdafc09e000 2023-09-30T22:36:43.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.503+0000 7fdb03152700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fdafc125f60 con 0x7fdafc09e410 2023-09-30T22:36:43.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.503+0000 7fdb00eee700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdafc09d6f0 0x7fdafc1225a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:43.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.503+0000 7fdb00eee700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdafc09d6f0 0x7fdafc1225a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36538/0 (socket says 172.21.15.16:36538) 2023-09-30T22:36:43.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.503+0000 7fdb00eee700 1 -- 172.21.15.16:0/1704990915 learned_addr learned my addr 172.21.15.16:0/1704990915 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:43.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.503+0000 7fdb016ef700 1 --2- 172.21.15.16:0/1704990915 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdafc09e410 0x7fdafc123020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:43.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.503+0000 7fdafbfff700 1 --2- 172.21.15.16:0/1704990915 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdafc09e000 0x7fdafc122ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:43.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.503+0000 7fdb00eee700 1 -- 172.21.15.16:0/1704990915 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdafc09e410 msgr2=0x7fdafc123020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:43.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.504+0000 7fdb00eee700 1 --2- 172.21.15.16:0/1704990915 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdafc09e410 0x7fdafc123020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:43.508 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.504+0000 7fdb00eee700 1 -- 172.21.15.16:0/1704990915 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdafc09e000 msgr2=0x7fdafc122ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:43.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.504+0000 7fdb00eee700 1 --2- 172.21.15.16:0/1704990915 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdafc09e000 0x7fdafc122ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:43.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.504+0000 7fdb00eee700 1 -- 172.21.15.16:0/1704990915 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf0009550 con 0x7fdafc09d6f0 2023-09-30T22:36:43.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.504+0000 7fdb016ef700 1 --2- 172.21.15.16:0/1704990915 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdafc09e410 0x7fdafc123020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:43.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.504+0000 7fdb00eee700 1 --2- 172.21.15.16:0/1704990915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdafc09d6f0 0x7fdafc1225a0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0004000 tx=0x7fdaf0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:43.509 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.504+0000 7fdaf97fa700 1 -- 172.21.15.16:0/1704990915 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf0003c70 con 0x7fdafc09d6f0 2023-09-30T22:36:43.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.504+0000 7fdaf97fa700 1 -- 172.21.15.16:0/1704990915 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaf0003e10 con 0x7fdafc09d6f0 2023-09-30T22:36:43.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.504+0000 7fdb03152700 1 -- 172.21.15.16:0/1704990915 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdafc124590 con 0x7fdafc09d6f0 2023-09-30T22:36:43.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.504+0000 7fdaf97fa700 1 -- 172.21.15.16:0/1704990915 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf0011870 con 0x7fdafc09d6f0 2023-09-30T22:36:43.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.504+0000 7fdb03152700 1 -- 172.21.15.16:0/1704990915 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdafc1266f0 con 0x7fdafc09d6f0 2023-09-30T22:36:43.510 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.505+0000 7fdb03152700 1 -- 172.21.15.16:0/1704990915 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdafc074ea0 con 0x7fdafc09d6f0 2023-09-30T22:36:43.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.506+0000 7fdaf97fa700 1 -- 172.21.15.16:0/1704990915 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fdaf0021070 con 0x7fdafc09d6f0 2023-09-30T22:36:43.511 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.506+0000 7fdaf97fa700 1 -- 172.21.15.16:0/1704990915 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fdaf0041140 con 0x7fdafc09d6f0 2023-09-30T22:36:43.516 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.515+0000 7fdaf97fa700 1 -- 172.21.15.16:0/1704990915 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fdaf002d990 con 0x7fdafc09d6f0 2023-09-30T22:36:43.620 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.619+0000 7fb052f67700 1 -- 172.21.15.16:0/2120386631 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb04c074ea0 con 0x7fb04c09d6f0 2023-09-30T22:36:43.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.619+0000 7fb0497fa700 1 -- 172.21.15.16:0/2120386631 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb040021030 con 0x7fb04c09d6f0 2023-09-30T22:36:43.621 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:36:43.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.621+0000 7fb052f67700 1 -- 172.21.15.16:0/2120386631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb04c09d6f0 msgr2=0x7fb04c11df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:43.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.621+0000 7fb052f67700 1 --2- 172.21.15.16:0/2120386631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb04c09d6f0 0x7fb04c11df80 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7fb04000e900 tx=0x7fb04000ec10 comp rx=0 tx=0).stop 2023-09-30T22:36:43.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.622+0000 7fb052f67700 1 -- 172.21.15.16:0/2120386631 shutdown_connections 2023-09-30T22:36:43.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.622+0000 7fb052f67700 1 --2- 172.21.15.16:0/2120386631 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb04c09e410 0x7fb04c11ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:43.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.622+0000 7fb052f67700 1 --2- 172.21.15.16:0/2120386631 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb04c09d6f0 0x7fb04c11df80 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:43.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.622+0000 7fb052f67700 1 --2- 172.21.15.16:0/2120386631 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb04c09e000 0x7fb04c11e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:43.623 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.622+0000 7fb052f67700 1 -- 172.21.15.16:0/2120386631 >> 172.21.15.16:0/2120386631 conn(0x7fb04c066f50 msgr2=0x7fb04c130890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:43.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.623+0000 7fb052f67700 1 -- 172.21.15.16:0/2120386631 shutdown_connections 2023-09-30T22:36:43.624 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.623+0000 7fb052f67700 1 -- 172.21.15.16:0/2120386631 wait complete. 2023-09-30T22:36:43.643 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:36:43.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.681+0000 7f1873ca3700 1 -- 172.21.15.16:0/852421213 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1854005860 con 0x7f186c136830 2023-09-30T22:36:43.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.682+0000 7f18627fc700 1 -- 172.21.15.16:0/852421213 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f185c017030 con 0x7f186c136830 2023-09-30T22:36:43.684 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:36:43.686 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.684+0000 7f1873ca3700 1 -- 172.21.15.16:0/852421213 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f186c136830 msgr2=0x7f186c1c8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:43.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.684+0000 7f1873ca3700 1 --2- 172.21.15.16:0/852421213 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f186c136830 0x7f186c1c8af0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f185c00e810 tx=0x7f185c00ebd0 comp rx=0 tx=0).stop 2023-09-30T22:36:43.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.684+0000 7f1873ca3700 1 -- 172.21.15.16:0/852421213 shutdown_connections 2023-09-30T22:36:43.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.684+0000 7f1873ca3700 1 --2- 172.21.15.16:0/852421213 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f186c139150 0x7f186c119e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:43.687 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.684+0000 7f1873ca3700 1 --2- 172.21.15.16:0/852421213 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f186c1338c0 0x7f186c1c85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:43.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.684+0000 7f1873ca3700 1 --2- 172.21.15.16:0/852421213 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f186c136830 0x7f186c1c8af0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:43.688 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.684+0000 7f1873ca3700 1 -- 172.21.15.16:0/852421213 >> 172.21.15.16:0/852421213 conn(0x7f186c0a2aa0 msgr2=0x7f186c134610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:43.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.684+0000 7f1873ca3700 1 -- 172.21.15.16:0/852421213 shutdown_connections 2023-09-30T22:36:43.689 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.684+0000 7f1873ca3700 1 -- 172.21.15.16:0/852421213 wait complete. 2023-09-30T22:36:43.700 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:36:43.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.701+0000 7f5a9f59e700 1 -- 172.21.15.16:0/13656983 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f5aa01235b0 con 0x7f5aa0133960 2023-09-30T22:36:43.703 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.701+0000 7f5a8effd700 1 -- 172.21.15.16:0/13656983 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5a98019070 con 0x7f5aa0133960 2023-09-30T22:36:43.704 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:36:43.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.703+0000 7f5a9f59e700 1 -- 172.21.15.16:0/13656983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5aa0133960 msgr2=0x7f5aa0120b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:43.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.703+0000 7f5a9f59e700 1 --2- 172.21.15.16:0/13656983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5aa0133960 0x7f5aa0120b50 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f5a9800be20 tx=0x7f5a98009ed0 comp rx=0 tx=0).stop 2023-09-30T22:36:43.705 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.703+0000 7f5a9f59e700 1 -- 172.21.15.16:0/13656983 shutdown_connections 2023-09-30T22:36:43.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.703+0000 7f5a9f59e700 1 --2- 172.21.15.16:0/13656983 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5aa009e670 0x7f5aa01200d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:43.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.703+0000 7f5a9f59e700 1 --2- 172.21.15.16:0/13656983 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5aa0133960 0x7f5aa0120b50 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:43.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.703+0000 7f5a9f59e700 1 --2- 172.21.15.16:0/13656983 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f5aa009f8a0 0x7f5aa0120610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:43.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.703+0000 7f5a9f59e700 1 -- 172.21.15.16:0/13656983 >> 172.21.15.16:0/13656983 conn(0x7f5aa00a0c90 msgr2=0x7f5aa01bae90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:43.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.703+0000 7f5a9f59e700 1 -- 172.21.15.16:0/13656983 shutdown_connections 2023-09-30T22:36:43.706 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.703+0000 7f5a9f59e700 1 -- 172.21.15.16:0/13656983 wait complete. 2023-09-30T22:36:43.719 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:36:43.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.724+0000 7fdb03152700 1 -- 172.21.15.16:0/1704990915 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fdafc06c360 con 0x7fdafc09d6f0 2023-09-30T22:36:43.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.724+0000 7fdaf97fa700 1 -- 172.21.15.16:0/1704990915 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fdaf002e3f0 con 0x7fdafc09d6f0 2023-09-30T22:36:43.726 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:36:43.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.727+0000 7fdb03152700 1 -- 172.21.15.16:0/1704990915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdafc09d6f0 msgr2=0x7fdafc1225a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:43.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.727+0000 7fdb03152700 1 --2- 172.21.15.16:0/1704990915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdafc09d6f0 0x7fdafc1225a0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0004000 tx=0x7fdaf0004750 comp rx=0 tx=0).stop 2023-09-30T22:36:43.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.727+0000 7fdb03152700 1 -- 172.21.15.16:0/1704990915 shutdown_connections 2023-09-30T22:36:43.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.727+0000 7fdb03152700 1 --2- 172.21.15.16:0/1704990915 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fdafc09e410 0x7fdafc123020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:43.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.727+0000 7fdb03152700 1 --2- 172.21.15.16:0/1704990915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdafc09d6f0 0x7fdafc1225a0 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:43.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.727+0000 7fdb03152700 1 --2- 172.21.15.16:0/1704990915 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fdafc09e000 0x7fdafc122ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:43.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.727+0000 7fdb03152700 1 -- 172.21.15.16:0/1704990915 >> 172.21.15.16:0/1704990915 conn(0x7fdafc066f50 msgr2=0x7fdafc0711f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:43.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.727+0000 7fdb03152700 1 -- 172.21.15.16:0/1704990915 shutdown_connections 2023-09-30T22:36:43.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:43.727+0000 7fdb03152700 1 -- 172.21.15.16:0/1704990915 wait complete. 2023-09-30T22:36:43.742 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:36:44.644 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:36:44.701 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:36:44.720 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:36:44.742 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:36:44.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.797+0000 7f03d035c700 1 Processor -- start 2023-09-30T22:36:44.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.797+0000 7f03d035c700 1 -- start start 2023-09-30T22:36:44.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.797+0000 7f03d035c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f03c809f330 0x7f03c8138f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:44.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.797+0000 7f03d035c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f03c8139450 0x7f03c813b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:44.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.797+0000 7f03d035c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03c806bec0 con 0x7f03c809e850 2023-09-30T22:36:44.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.797+0000 7f03d035c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03c80a0030 con 0x7f03c8139450 2023-09-30T22:36:44.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.797+0000 7f03d035c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f03c80a0830 con 0x7f03c809f330 2023-09-30T22:36:44.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.798+0000 7f03ce0f8700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f03c809e850 0x7f03c809ec20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60184/0 (socket says 172.21.15.16:60184) 2023-09-30T22:36:44.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.798+0000 7f03ce0f8700 1 -- 172.21.15.16:0/864198735 learned_addr learned my addr 172.21.15.16:0/864198735 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:36:44.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.798+0000 7f03cd8f7700 1 --2- 172.21.15.16:0/864198735 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f03c809f330 0x7f03c8138f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:44.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.798+0000 7f03ce8f9700 1 --2- 172.21.15.16:0/864198735 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f03c8139450 0x7f03c813b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:44.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.798+0000 7f03cd0f6700 1 -- 172.21.15.16:0/864198735 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 436002911 0 0) 0x7f03c806bec0 con 0x7f03c809e850 2023-09-30T22:36:44.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.798+0000 7f03cd0f6700 1 -- 172.21.15.16:0/864198735 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03b40035b0 con 0x7f03c809e850 2023-09-30T22:36:44.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.798+0000 7f03cd8f7700 1 -- 172.21.15.16:0/864198735 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f03c8139450 msgr2=0x7f03c813b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:44.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.798+0000 7f03cd8f7700 1 --2- 172.21.15.16:0/864198735 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f03c8139450 0x7f03c813b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:44.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.798+0000 7f03cd8f7700 1 -- 172.21.15.16:0/864198735 >> v1:172.21.15.16:6789/0 conn(0x7f03c809e850 legacy=0x7f03c809ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:44.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.798+0000 7f03cd8f7700 1 -- 172.21.15.16:0/864198735 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03c806caa0 con 0x7f03c809f330 2023-09-30T22:36:44.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.798+0000 7f03cd8f7700 1 --2- 172.21.15.16:0/864198735 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f03c809f330 0x7f03c8138f10 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f03b8009870 tx=0x7f03b8009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebbab5b2aef6a108 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:44.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.799+0000 7f03cd0f6700 1 -- 172.21.15.16:0/864198735 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b8011830 con 0x7f03c809f330 2023-09-30T22:36:44.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.799+0000 7f03cd0f6700 1 -- 172.21.15.16:0/864198735 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03b8011990 con 0x7f03c809f330 2023-09-30T22:36:44.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.799+0000 7f03cd0f6700 1 -- 172.21.15.16:0/864198735 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b8011d00 con 0x7f03c809f330 2023-09-30T22:36:44.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.799+0000 7f03d035c700 1 -- 172.21.15.16:0/864198735 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f03c809f330 msgr2=0x7f03c8138f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:44.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.799+0000 7f03d035c700 1 --2- 172.21.15.16:0/864198735 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f03c809f330 0x7f03c8138f10 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f03b8009870 tx=0x7f03b8009b80 comp rx=0 tx=0).stop 2023-09-30T22:36:44.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.800+0000 7f03d035c700 1 -- 172.21.15.16:0/864198735 shutdown_connections 2023-09-30T22:36:44.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.800+0000 7f03d035c700 1 --2- 172.21.15.16:0/864198735 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f03c8139450 0x7f03c813b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:44.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.800+0000 7f03d035c700 1 --2- 172.21.15.16:0/864198735 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f03c809f330 0x7f03c8138f10 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:44.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.800+0000 7f03d035c700 1 -- 172.21.15.16:0/864198735 >> 172.21.15.16:0/864198735 conn(0x7f03c80a3f70 msgr2=0x7f03c80a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:44.805 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.800+0000 7f03d035c700 1 -- 172.21.15.16:0/864198735 shutdown_connections 2023-09-30T22:36:44.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.800+0000 7f03d035c700 1 -- 172.21.15.16:0/864198735 wait complete. 2023-09-30T22:36:44.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.801+0000 7f03d035c700 1 Processor -- start 2023-09-30T22:36:44.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.801+0000 7f03d035c700 1 -- start start 2023-09-30T22:36:44.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.801+0000 7f03d035c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f03c809e850 0x7f03c81228d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:44.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.801+0000 7f03d035c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f03c809f330 0x7f03c8122e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:44.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.801+0000 7f03d035c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03c8139450 0x7f03c8123350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:44.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.801+0000 7f03d035c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c806ccc0 con 0x7f03c8139450 2023-09-30T22:36:44.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.801+0000 7f03d035c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03c8125ba0 con 0x7f03c809e850 2023-09-30T22:36:44.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.801+0000 7f03d035c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f03c81260c0 con 0x7f03c809f330 2023-09-30T22:36:44.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.801+0000 7f03ce0f8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f03c809e850 0x7f03c81228d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:44.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.802+0000 7f03ce0f8700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f03c809e850 0x7f03c81228d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:59494/0 (socket says 172.21.15.16:59494) 2023-09-30T22:36:44.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.802+0000 7f03ce0f8700 1 -- 172.21.15.16:0/142986353 learned_addr learned my addr 172.21.15.16:0/142986353 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:44.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.802+0000 7f03ce8f9700 1 --2- 172.21.15.16:0/142986353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03c8139450 0x7f03c8123350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:44.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.802+0000 7f03ce0f8700 1 -- 172.21.15.16:0/142986353 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f03c809f330 msgr2=0x7f03c8122e10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:36:44.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.802+0000 7f03ce0f8700 1 --2- 172.21.15.16:0/142986353 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f03c809f330 0x7f03c8122e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:44.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.802+0000 7f03ce0f8700 1 -- 172.21.15.16:0/142986353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03c8139450 msgr2=0x7f03c8123350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:44.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.802+0000 7f03ce0f8700 1 --2- 172.21.15.16:0/142986353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03c8139450 0x7f03c8123350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:44.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.802+0000 7f03ce0f8700 1 -- 172.21.15.16:0/142986353 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b80094a0 con 0x7f03c809e850 2023-09-30T22:36:44.807 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.802+0000 7f03ce0f8700 1 --2- 172.21.15.16:0/142986353 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f03c809e850 0x7f03c81228d0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f03c400c700 tx=0x7f03c400ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:44.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.802+0000 7f03beffd700 1 -- 172.21.15.16:0/142986353 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c4005d70 con 0x7f03c809e850 2023-09-30T22:36:44.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.803+0000 7f03beffd700 1 -- 172.21.15.16:0/142986353 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03c40070d0 con 0x7f03c809e850 2023-09-30T22:36:44.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.803+0000 7f03d035c700 1 -- 172.21.15.16:0/142986353 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03c80a16f0 con 0x7f03c809e850 2023-09-30T22:36:44.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.803+0000 7f03beffd700 1 -- 172.21.15.16:0/142986353 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c4007440 con 0x7f03c809e850 2023-09-30T22:36:44.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.803+0000 7f03d035c700 1 -- 172.21.15.16:0/142986353 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03c8126b30 con 0x7f03c809e850 2023-09-30T22:36:44.808 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.804+0000 7f03beffd700 1 -- 172.21.15.16:0/142986353 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f03c40075a0 con 0x7f03c809e850 2023-09-30T22:36:44.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.804+0000 7f03beffd700 1 -- 172.21.15.16:0/142986353 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f03c4040c30 con 0x7f03c809e850 2023-09-30T22:36:44.809 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.804+0000 7f03d035c700 1 -- 172.21.15.16:0/142986353 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03c8125d30 con 0x7f03c809e850 2023-09-30T22:36:44.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.814+0000 7f03beffd700 1 -- 172.21.15.16:0/142986353 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f03c4017070 con 0x7f03c809e850 2023-09-30T22:36:44.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.847+0000 7f448bdf5700 1 Processor -- start 2023-09-30T22:36:44.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.847+0000 7f448bdf5700 1 -- start start 2023-09-30T22:36:44.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.848+0000 7f448bdf5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f448412cf60 0x7f448406f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:44.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.848+0000 7f448bdf5700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4484130430 0x7f4484137c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:44.849 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.848+0000 7f448bdf5700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f448412cd90 con 0x7f448412f8b0 2023-09-30T22:36:44.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.848+0000 7f448bdf5700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f448412d330 con 0x7f4484130430 2023-09-30T22:36:44.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.848+0000 7f448bdf5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4484071780 con 0x7f448412cf60 2023-09-30T22:36:44.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.848+0000 7f448a392700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4484130430 0x7f4484137c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:44.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.848+0000 7f4489b91700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f448412cf60 0x7f448406f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:44.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.848+0000 7f448a392700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4484130430 0x7f4484137c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36562/0 (socket says 172.21.15.16:36562) 2023-09-30T22:36:44.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.848+0000 7f4489b91700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f448412cf60 0x7f448406f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:59504/0 (socket says 172.21.15.16:59504) 2023-09-30T22:36:44.850 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.848+0000 7f448a392700 1 -- 172.21.15.16:0/2823265559 learned_addr learned my addr 172.21.15.16:0/2823265559 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:44.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.848+0000 7f448a392700 1 -- 172.21.15.16:0/2823265559 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f448412cf60 msgr2=0x7f448406f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:44.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.848+0000 7f448a392700 1 --2- 172.21.15.16:0/2823265559 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f448412cf60 0x7f448406f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:44.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.848+0000 7f448a392700 1 -- 172.21.15.16:0/2823265559 >> v1:172.21.15.16:6789/0 conn(0x7f448412f8b0 legacy=0x7f448412fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:44.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.849+0000 7f448a392700 1 -- 172.21.15.16:0/2823265559 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4484071a40 con 0x7f4484130430 2023-09-30T22:36:44.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.849+0000 7f4488b8f700 1 -- 172.21.15.16:0/2823265559 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1648979253 0 0) 0x7f448412cd90 con 0x7f448412f8b0 2023-09-30T22:36:44.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.849+0000 7f448a392700 1 --2- 172.21.15.16:0/2823265559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4484130430 0x7f4484137c30 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f447c009e50 tx=0x7f447c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b826d9db9cb91608 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:44.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.849+0000 7f4488b8f700 1 -- 172.21.15.16:0/2823265559 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447c00e070 con 0x7f4484130430 2023-09-30T22:36:44.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.849+0000 7f4488b8f700 1 -- 172.21.15.16:0/2823265559 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f447c00b910 con 0x7f4484130430 2023-09-30T22:36:44.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.849+0000 7f4488b8f700 1 -- 172.21.15.16:0/2823265559 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f447c00bcc0 con 0x7f4484130430 2023-09-30T22:36:44.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.850+0000 7f448bdf5700 1 -- 172.21.15.16:0/2823265559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4484130430 msgr2=0x7f4484137c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:44.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.850+0000 7f448bdf5700 1 --2- 172.21.15.16:0/2823265559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4484130430 0x7f4484137c30 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f447c009e50 tx=0x7f447c00b330 comp rx=0 tx=0).stop 2023-09-30T22:36:44.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.850+0000 7f448bdf5700 1 -- 172.21.15.16:0/2823265559 shutdown_connections 2023-09-30T22:36:44.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.850+0000 7f448bdf5700 1 --2- 172.21.15.16:0/2823265559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4484130430 0x7f4484137c30 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:44.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.850+0000 7f448bdf5700 1 --2- 172.21.15.16:0/2823265559 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f448412cf60 0x7f448406f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:44.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.850+0000 7f448bdf5700 1 -- 172.21.15.16:0/2823265559 >> 172.21.15.16:0/2823265559 conn(0x7f44840a2aa0 msgr2=0x7f44840a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:44.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.850+0000 7f448bdf5700 1 -- 172.21.15.16:0/2823265559 shutdown_connections 2023-09-30T22:36:44.853 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.850+0000 7f448bdf5700 1 -- 172.21.15.16:0/2823265559 wait complete. 2023-09-30T22:36:44.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.851+0000 7f448bdf5700 1 Processor -- start 2023-09-30T22:36:44.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.851+0000 7f448bdf5700 1 -- start start 2023-09-30T22:36:44.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.852+0000 7f448bdf5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f448412cf60 0x7f44841309e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:44.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.852+0000 7f448bdf5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f448412f8b0 0x7f4484130f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:44.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.852+0000 7f448bdf5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4484130430 0x7f4484131460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:44.855 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.852+0000 7f448bdf5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448409ea80 con 0x7f448412cf60 2023-09-30T22:36:44.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.852+0000 7f448bdf5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44841330b0 con 0x7f4484130430 2023-09-30T22:36:44.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.852+0000 7f448bdf5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f448412d8d0 con 0x7f448412f8b0 2023-09-30T22:36:44.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.852+0000 7f4489390700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f448412f8b0 0x7f4484130f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:44.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.852+0000 7f4489390700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f448412f8b0 0x7f4484130f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35738/0 (socket says 172.21.15.16:35738) 2023-09-30T22:36:44.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.852+0000 7f4489390700 1 -- 172.21.15.16:0/46560190 learned_addr learned my addr 172.21.15.16:0/46560190 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:44.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.852+0000 7f4489b91700 1 --2- 172.21.15.16:0/46560190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f448412cf60 0x7f44841309e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:44.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.852+0000 7f448a392700 1 --2- 172.21.15.16:0/46560190 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4484130430 0x7f4484131460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:44.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.853+0000 7f4489390700 1 -- 172.21.15.16:0/46560190 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4484130430 msgr2=0x7f4484131460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:44.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.853+0000 7f4489390700 1 --2- 172.21.15.16:0/46560190 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4484130430 0x7f4484131460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:44.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.853+0000 7f4489390700 1 -- 172.21.15.16:0/46560190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f448412cf60 msgr2=0x7f44841309e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:44.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.853+0000 7f4489390700 1 --2- 172.21.15.16:0/46560190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f448412cf60 0x7f44841309e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:44.856 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.853+0000 7f4489390700 1 -- 172.21.15.16:0/46560190 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f447c009b30 con 0x7f448412f8b0 2023-09-30T22:36:44.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.853+0000 7f4489b91700 1 --2- 172.21.15.16:0/46560190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f448412cf60 0x7f44841309e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:44.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.853+0000 7f4489390700 1 --2- 172.21.15.16:0/46560190 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f448412f8b0 0x7f4484130f20 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f447400c550 tx=0x7f447400c860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:44.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.854+0000 7f447a7fc700 1 -- 172.21.15.16:0/46560190 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44740054e0 con 0x7f448412f8b0 2023-09-30T22:36:44.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.854+0000 7f447a7fc700 1 -- 172.21.15.16:0/46560190 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4474003680 con 0x7f448412f8b0 2023-09-30T22:36:44.857 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.854+0000 7f448bdf5700 1 -- 172.21.15.16:0/46560190 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f448409e070 con 0x7f448412f8b0 2023-09-30T22:36:44.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.854+0000 7f447a7fc700 1 -- 172.21.15.16:0/46560190 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4474003a30 con 0x7f448412f8b0 2023-09-30T22:36:44.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.854+0000 7f448bdf5700 1 -- 172.21.15.16:0/46560190 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4484133390 con 0x7f448412f8b0 2023-09-30T22:36:44.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.855+0000 7f448bdf5700 1 -- 172.21.15.16:0/46560190 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4484074ea0 con 0x7f448412f8b0 2023-09-30T22:36:44.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.855+0000 7f447a7fc700 1 -- 172.21.15.16:0/46560190 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4474003b90 con 0x7f448412f8b0 2023-09-30T22:36:44.858 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.856+0000 7f447a7fc700 1 -- 172.21.15.16:0/46560190 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f447403ef50 con 0x7f448412f8b0 2023-09-30T22:36:44.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.864+0000 7f447a7fc700 1 -- 172.21.15.16:0/46560190 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4474003e40 con 0x7f448412f8b0 2023-09-30T22:36:44.897 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.896+0000 7f4023ae1700 1 Processor -- start 2023-09-30T22:36:44.898 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.896+0000 7f4023ae1700 1 -- start start 2023-09-30T22:36:44.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.897+0000 7f4023ae1700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f401c0a0a30 0x7f401c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:44.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.897+0000 7f4023ae1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f401c0a1340 0x7f401c133a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:44.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.897+0000 7f4023ae1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f401c133fa0 0x7f401c134410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:44.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.897+0000 7f4023ae1700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f401c09edd0 con 0x7f401c0a0a30 2023-09-30T22:36:44.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.897+0000 7f4023ae1700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f401c09e1f0 con 0x7f401c0a1340 2023-09-30T22:36:44.901 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.897+0000 7f4023ae1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f401c09dc50 con 0x7f401c133fa0 2023-09-30T22:36:44.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.897+0000 7f402187d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f401c0a0a30 0x7f401c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:44.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.897+0000 7f402187d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f401c0a0a30 0x7f401c0a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36584/0 (socket says 172.21.15.16:36584) 2023-09-30T22:36:44.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.897+0000 7f402187d700 1 -- 172.21.15.16:0/3301674802 learned_addr learned my addr 172.21.15.16:0/3301674802 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:44.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.897+0000 7f402207e700 1 --2- 172.21.15.16:0/3301674802 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f401c133fa0 0x7f401c134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:44.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.898+0000 7f402107c700 1 --2- 172.21.15.16:0/3301674802 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f401c0a1340 0x7f401c133a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:44.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.898+0000 7f402187d700 1 -- 172.21.15.16:0/3301674802 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f401c133fa0 msgr2=0x7f401c134410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:44.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.898+0000 7f402187d700 1 --2- 172.21.15.16:0/3301674802 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f401c133fa0 0x7f401c134410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:44.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.898+0000 7f402187d700 1 -- 172.21.15.16:0/3301674802 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f401c0a1340 msgr2=0x7f401c133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:44.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.898+0000 7f402187d700 1 --2- 172.21.15.16:0/3301674802 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f401c0a1340 0x7f401c133a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:44.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.898+0000 7f402187d700 1 -- 172.21.15.16:0/3301674802 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f401c0a1fe0 con 0x7f401c0a0a30 2023-09-30T22:36:44.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.898+0000 7f402107c700 1 --2- 172.21.15.16:0/3301674802 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f401c0a1340 0x7f401c133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:36:44.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.898+0000 7f402187d700 1 --2- 172.21.15.16:0/3301674802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f401c0a0a30 0x7f401c0a0e00 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f40180098f0 tx=0x7f4018009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc25eb064a3e1828 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:44.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.898+0000 7f402087b700 1 -- 172.21.15.16:0/3301674802 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4018011830 con 0x7f401c0a0a30 2023-09-30T22:36:44.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.898+0000 7f402087b700 1 -- 172.21.15.16:0/3301674802 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4018011990 con 0x7f401c0a0a30 2023-09-30T22:36:44.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.898+0000 7f402087b700 1 -- 172.21.15.16:0/3301674802 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4018011d20 con 0x7f401c0a0a30 2023-09-30T22:36:44.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.899+0000 7f4023ae1700 1 -- 172.21.15.16:0/3301674802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f401c0a0a30 msgr2=0x7f401c0a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:44.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.899+0000 7f4023ae1700 1 --2- 172.21.15.16:0/3301674802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f401c0a0a30 0x7f401c0a0e00 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7f40180098f0 tx=0x7f4018009c00 comp rx=0 tx=0).stop 2023-09-30T22:36:44.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.899+0000 7f4023ae1700 1 -- 172.21.15.16:0/3301674802 shutdown_connections 2023-09-30T22:36:44.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.899+0000 7f4023ae1700 1 --2- 172.21.15.16:0/3301674802 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f401c133fa0 0x7f401c134410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:44.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.899+0000 7f4023ae1700 1 --2- 172.21.15.16:0/3301674802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f401c0a0a30 0x7f401c0a0e00 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:44.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.899+0000 7f4023ae1700 1 --2- 172.21.15.16:0/3301674802 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f401c0a1340 0x7f401c133a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:44.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.899+0000 7f4023ae1700 1 -- 172.21.15.16:0/3301674802 >> 172.21.15.16:0/3301674802 conn(0x7f401c0a6550 msgr2=0x7f401c0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:44.905 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.899+0000 7f4023ae1700 1 -- 172.21.15.16:0/3301674802 shutdown_connections 2023-09-30T22:36:44.906 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.899+0000 7f4023ae1700 1 -- 172.21.15.16:0/3301674802 wait complete. 2023-09-30T22:36:44.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.901+0000 7f4023ae1700 1 Processor -- start 2023-09-30T22:36:44.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.901+0000 7f4023ae1700 1 -- start start 2023-09-30T22:36:44.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.901+0000 7f4023ae1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f401c0a0a30 0x7f401c121220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:44.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.901+0000 7f4023ae1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f401c0a1340 0x7f401c121760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:44.909 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.901+0000 7f4023ae1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f401c133fa0 0x7f401c121ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:44.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.901+0000 7f4023ae1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f401c1325f0 con 0x7f401c0a1340 2023-09-30T22:36:44.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.901+0000 7f4023ae1700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f401c122680 con 0x7f401c133fa0 2023-09-30T22:36:44.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.901+0000 7f4023ae1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f401c123cc0 con 0x7f401c0a0a30 2023-09-30T22:36:44.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.901+0000 7f402187d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f401c0a0a30 0x7f401c121220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:44.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.902+0000 7f402187d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f401c0a0a30 0x7f401c121220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35746/0 (socket says 172.21.15.16:35746) 2023-09-30T22:36:44.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.902+0000 7f402187d700 1 -- 172.21.15.16:0/508468281 learned_addr learned my addr 172.21.15.16:0/508468281 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:44.910 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.902+0000 7f402107c700 1 --2- 172.21.15.16:0/508468281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f401c0a1340 0x7f401c121760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:44.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.902+0000 7f402207e700 1 --2- 172.21.15.16:0/508468281 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f401c133fa0 0x7f401c121ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:44.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.902+0000 7f402187d700 1 -- 172.21.15.16:0/508468281 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f401c133fa0 msgr2=0x7f401c121ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:44.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.902+0000 7f402187d700 1 --2- 172.21.15.16:0/508468281 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f401c133fa0 0x7f401c121ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:44.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.902+0000 7f402187d700 1 -- 172.21.15.16:0/508468281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f401c0a1340 msgr2=0x7f401c121760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:44.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.902+0000 7f402187d700 1 --2- 172.21.15.16:0/508468281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f401c0a1340 0x7f401c121760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:44.911 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.902+0000 7f402187d700 1 -- 172.21.15.16:0/508468281 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4018009550 con 0x7f401c0a0a30 2023-09-30T22:36:44.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.902+0000 7f402107c700 1 --2- 172.21.15.16:0/508468281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f401c0a1340 0x7f401c121760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:44.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.902+0000 7f402187d700 1 --2- 172.21.15.16:0/508468281 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f401c0a0a30 0x7f401c121220 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f4018009f50 tx=0x7f4018004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:44.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.902+0000 7f40127fc700 1 -- 172.21.15.16:0/508468281 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4018004300 con 0x7f401c0a0a30 2023-09-30T22:36:44.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.902+0000 7f40127fc700 1 -- 172.21.15.16:0/508468281 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f401801f070 con 0x7f401c0a0a30 2023-09-30T22:36:44.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.902+0000 7f40127fc700 1 -- 172.21.15.16:0/508468281 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40180036a0 con 0x7f401c0a0a30 2023-09-30T22:36:44.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.902+0000 7f4023ae1700 1 -- 172.21.15.16:0/508468281 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f401c1247a0 con 0x7f401c0a0a30 2023-09-30T22:36:44.912 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.902+0000 7f4023ae1700 1 -- 172.21.15.16:0/508468281 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f401c0a0220 con 0x7f401c0a0a30 2023-09-30T22:36:44.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.904+0000 7f4023ae1700 1 -- 172.21.15.16:0/508468281 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f401c124930 con 0x7f401c0a0a30 2023-09-30T22:36:44.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.904+0000 7f40127fc700 1 -- 172.21.15.16:0/508468281 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4018021030 con 0x7f401c0a0a30 2023-09-30T22:36:44.913 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.904+0000 7fc742c83700 1 Processor -- start 2023-09-30T22:36:44.914 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.904+0000 7f40127fc700 1 -- 172.21.15.16:0/508468281 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f4018040930 con 0x7f401c0a0a30 2023-09-30T22:36:44.917 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.904+0000 7fc742c83700 1 -- start start 2023-09-30T22:36:44.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.905+0000 7fc742c83700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc73c133420 0x7fc73c1337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:44.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.905+0000 7fc742c83700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc73c133e30 0x7fc73c12ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:44.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.905+0000 7fc742c83700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc73c134260 0x7fc73c1304d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:44.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.905+0000 7fc742c83700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc73c06b990 con 0x7fc73c133420 2023-09-30T22:36:44.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.905+0000 7fc742c83700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc73c0a0590 con 0x7fc73c134260 2023-09-30T22:36:44.918 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.905+0000 7fc742c83700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc73c09fb80 con 0x7fc73c133e30 2023-09-30T22:36:44.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.905+0000 7fc740a1f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc73c133420 0x7fc73c1337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:44.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.905+0000 7fc740a1f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc73c133420 0x7fc73c1337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36614/0 (socket says 172.21.15.16:36614) 2023-09-30T22:36:44.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.906+0000 7fc740a1f700 1 -- 172.21.15.16:0/3717611957 learned_addr learned my addr 172.21.15.16:0/3717611957 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:44.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.906+0000 7fc741220700 1 --2- 172.21.15.16:0/3717611957 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc73c134260 0x7fc73c1304d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:44.919 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.906+0000 7fc740a1f700 1 -- 172.21.15.16:0/3717611957 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc73c133e30 msgr2=0x7fc73c12ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:44.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.906+0000 7fc73bfff700 1 --2- 172.21.15.16:0/3717611957 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc73c133e30 0x7fc73c12ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:44.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.906+0000 7fc740a1f700 1 --2- 172.21.15.16:0/3717611957 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc73c133e30 0x7fc73c12ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:44.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.906+0000 7fc740a1f700 1 -- 172.21.15.16:0/3717611957 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc73c134260 msgr2=0x7fc73c1304d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:44.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.906+0000 7fc740a1f700 1 --2- 172.21.15.16:0/3717611957 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc73c134260 0x7fc73c1304d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:44.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.906+0000 7fc740a1f700 1 -- 172.21.15.16:0/3717611957 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc73c0a0030 con 0x7fc73c133420 2023-09-30T22:36:44.920 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.906+0000 7fc73bfff700 1 --2- 172.21.15.16:0/3717611957 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc73c133e30 0x7fc73c12ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:36:44.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.906+0000 7fc741220700 1 --2- 172.21.15.16:0/3717611957 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc73c134260 0x7fc73c1304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:44.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.906+0000 7fc740a1f700 1 --2- 172.21.15.16:0/3717611957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc73c133420 0x7fc73c1337f0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fc7300098f0 tx=0x7fc730009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d794b6d46a8f048c server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:44.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.906+0000 7fc73b7fe700 1 -- 172.21.15.16:0/3717611957 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc730011830 con 0x7fc73c133420 2023-09-30T22:36:44.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.906+0000 7fc73b7fe700 1 -- 172.21.15.16:0/3717611957 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc730011990 con 0x7fc73c133420 2023-09-30T22:36:44.921 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.906+0000 7fc73b7fe700 1 -- 172.21.15.16:0/3717611957 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc730011d20 con 0x7fc73c133420 2023-09-30T22:36:44.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.907+0000 7fc742c83700 1 -- 172.21.15.16:0/3717611957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc73c133420 msgr2=0x7fc73c1337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:44.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.907+0000 7fc742c83700 1 --2- 172.21.15.16:0/3717611957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc73c133420 0x7fc73c1337f0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fc7300098f0 tx=0x7fc730009c00 comp rx=0 tx=0).stop 2023-09-30T22:36:44.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.907+0000 7fc742c83700 1 -- 172.21.15.16:0/3717611957 shutdown_connections 2023-09-30T22:36:44.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.907+0000 7fc742c83700 1 --2- 172.21.15.16:0/3717611957 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc73c134260 0x7fc73c1304d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:44.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.907+0000 7fc742c83700 1 --2- 172.21.15.16:0/3717611957 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc73c133420 0x7fc73c1337f0 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:44.922 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.907+0000 7fc742c83700 1 --2- 172.21.15.16:0/3717611957 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc73c133e30 0x7fc73c12ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:44.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.907+0000 7fc742c83700 1 -- 172.21.15.16:0/3717611957 >> 172.21.15.16:0/3717611957 conn(0x7fc73c0a4210 msgr2=0x7fc73c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:44.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.907+0000 7fc742c83700 1 -- 172.21.15.16:0/3717611957 shutdown_connections 2023-09-30T22:36:44.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.907+0000 7fc742c83700 1 -- 172.21.15.16:0/3717611957 wait complete. 2023-09-30T22:36:44.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.908+0000 7fc742c83700 1 Processor -- start 2023-09-30T22:36:44.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.908+0000 7fc742c83700 1 -- start start 2023-09-30T22:36:44.923 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.909+0000 7fc742c83700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc73c133420 0x7fc73c09bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:44.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.909+0000 7fc742c83700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc73c133e30 0x7fc73c09c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:44.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.909+0000 7fc742c83700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc73c134260 0x7fc73c09c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:44.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.909+0000 7fc742c83700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc73c11c2e0 con 0x7fc73c133e30 2023-09-30T22:36:44.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.909+0000 7fc742c83700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc73c09d8c0 con 0x7fc73c133420 2023-09-30T22:36:44.924 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.909+0000 7fc742c83700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc73c11acf0 con 0x7fc73c134260 2023-09-30T22:36:44.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.909+0000 7fc73bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc73c133e30 0x7fc73c09c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:44.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.909+0000 7fc73bfff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc73c133e30 0x7fc73c09c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36628/0 (socket says 172.21.15.16:36628) 2023-09-30T22:36:44.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.909+0000 7fc73bfff700 1 -- 172.21.15.16:0/502422966 learned_addr learned my addr 172.21.15.16:0/502422966 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:44.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.909+0000 7fc740a1f700 1 --2- 172.21.15.16:0/502422966 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc73c133420 0x7fc73c09bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:44.925 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.910+0000 7fc741220700 1 --2- 172.21.15.16:0/502422966 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc73c134260 0x7fc73c09c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:44.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.910+0000 7fc73bfff700 1 -- 172.21.15.16:0/502422966 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc73c134260 msgr2=0x7fc73c09c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:44.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.910+0000 7fc73bfff700 1 --2- 172.21.15.16:0/502422966 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc73c134260 0x7fc73c09c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:44.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.910+0000 7fc73bfff700 1 -- 172.21.15.16:0/502422966 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc73c133420 msgr2=0x7fc73c09bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:44.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.910+0000 7fc73bfff700 1 --2- 172.21.15.16:0/502422966 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc73c133420 0x7fc73c09bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:44.926 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.910+0000 7fc73bfff700 1 -- 172.21.15.16:0/502422966 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc734009a70 con 0x7fc73c133e30 2023-09-30T22:36:44.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.910+0000 7fc741220700 1 --2- 172.21.15.16:0/502422966 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc73c134260 0x7fc73c09c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:36:44.927 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.910+0000 7f40127fc700 1 -- 172.21.15.16:0/508468281 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4018006d80 con 0x7f401c0a0a30 2023-09-30T22:36:44.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.910+0000 7fc73bfff700 1 --2- 172.21.15.16:0/502422966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc73c133e30 0x7fc73c09c1f0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fc72800c780 tx=0x7fc72800ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:44.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.910+0000 7fc7397fa700 1 -- 172.21.15.16:0/502422966 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc728011800 con 0x7fc73c133e30 2023-09-30T22:36:44.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.910+0000 7fc742c83700 1 -- 172.21.15.16:0/502422966 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc730009550 con 0x7fc73c133e30 2023-09-30T22:36:44.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.910+0000 7fc7397fa700 1 -- 172.21.15.16:0/502422966 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc728011960 con 0x7fc73c133e30 2023-09-30T22:36:44.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.910+0000 7fc7397fa700 1 -- 172.21.15.16:0/502422966 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc728011cd0 con 0x7fc73c133e30 2023-09-30T22:36:44.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.910+0000 7fc742c83700 1 -- 172.21.15.16:0/502422966 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc73c131400 con 0x7fc73c133e30 2023-09-30T22:36:44.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.911+0000 7fc7397fa700 1 -- 172.21.15.16:0/502422966 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc728021350 con 0x7fc73c133e30 2023-09-30T22:36:44.930 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.912+0000 7fc7397fa700 1 -- 172.21.15.16:0/502422966 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fc72803f6a0 con 0x7fc73c133e30 2023-09-30T22:36:44.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.912+0000 7fc742c83700 1 -- 172.21.15.16:0/502422966 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc73c11a680 con 0x7fc73c133e30 2023-09-30T22:36:44.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:44.922+0000 7fc7397fa700 1 -- 172.21.15.16:0/502422966 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc728011e30 con 0x7fc73c133e30 2023-09-30T22:36:45.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:45.033+0000 7f03d035c700 1 -- 172.21.15.16:0/142986353 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f03c80747f0 con 0x7f03c809e850 2023-09-30T22:36:45.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:45.034+0000 7f03beffd700 1 -- 172.21.15.16:0/142986353 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f03c4017030 con 0x7f03c809e850 2023-09-30T22:36:45.035 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:36:45.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:45.036+0000 7f03d035c700 1 -- 172.21.15.16:0/142986353 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f03c809e850 msgr2=0x7f03c81228d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:45.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:45.036+0000 7f03d035c700 1 --2- 172.21.15.16:0/142986353 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f03c809e850 0x7f03c81228d0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f03c400c700 tx=0x7f03c400ca10 comp rx=0 tx=0).stop 2023-09-30T22:36:45.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:45.036+0000 7f03d035c700 1 -- 172.21.15.16:0/142986353 shutdown_connections 2023-09-30T22:36:45.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:45.036+0000 7f03d035c700 1 --2- 172.21.15.16:0/142986353 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f03c809e850 0x7f03c81228d0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:45.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:45.036+0000 7f03d035c700 1 --2- 172.21.15.16:0/142986353 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03c8139450 0x7f03c8123350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:45.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:45.036+0000 7f03d035c700 1 --2- 172.21.15.16:0/142986353 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f03c809f330 0x7f03c8122e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:45.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:45.036+0000 7f03d035c700 1 -- 172.21.15.16:0/142986353 >> 172.21.15.16:0/142986353 conn(0x7f03c80a3f70 msgr2=0x7f03c8070de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:45.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:45.036+0000 7f03d035c700 1 -- 172.21.15.16:0/142986353 shutdown_connections 2023-09-30T22:36:45.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:45.036+0000 7f03d035c700 1 -- 172.21.15.16:0/142986353 wait complete. 2023-09-30T22:36:45.056 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:36:45.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:45.091+0000 7f448bdf5700 1 -- 172.21.15.16:0/46560190 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f44840713a0 con 0x7f448412f8b0 2023-09-30T22:36:45.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:45.091+0000 7f447a7fc700 1 -- 172.21.15.16:0/46560190 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f44740192c0 con 0x7f448412f8b0 2023-09-30T22:36:45.093 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:36:45.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:45.093+0000 7f448bdf5700 1 -- 172.21.15.16:0/46560190 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f448412f8b0 msgr2=0x7f4484130f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:45.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:45.093+0000 7f448bdf5700 1 --2- 172.21.15.16:0/46560190 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f448412f8b0 0x7f4484130f20 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f447400c550 tx=0x7f447400c860 comp rx=0 tx=0).stop 2023-09-30T22:36:45.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:45.094+0000 7f448bdf5700 1 -- 172.21.15.16:0/46560190 shutdown_connections 2023-09-30T22:36:45.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:45.094+0000 7f448bdf5700 1 --2- 172.21.15.16:0/46560190 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4484130430 0x7f4484131460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:45.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:45.094+0000 7f448bdf5700 1 --2- 172.21.15.16:0/46560190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f448412cf60 0x7f44841309e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:45.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:45.094+0000 7f448bdf5700 1 --2- 172.21.15.16:0/46560190 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f448412f8b0 0x7f4484130f20 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:45.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:45.094+0000 7f448bdf5700 1 -- 172.21.15.16:0/46560190 >> 172.21.15.16:0/46560190 conn(0x7f44840a2aa0 msgr2=0x7f448406e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:45.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:45.094+0000 7f448bdf5700 1 -- 172.21.15.16:0/46560190 shutdown_connections 2023-09-30T22:36:45.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:45.094+0000 7f448bdf5700 1 -- 172.21.15.16:0/46560190 wait complete. 2023-09-30T22:36:45.109 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:36:45.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:45.121+0000 7f4023ae1700 1 -- 172.21.15.16:0/508468281 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f401c074ea0 con 0x7f401c0a0a30 2023-09-30T22:36:45.123 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:45.121+0000 7f40127fc700 1 -- 172.21.15.16:0/508468281 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4018015030 con 0x7f401c0a0a30 2023-09-30T22:36:45.123 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:36:45.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:45.124+0000 7f4023ae1700 1 -- 172.21.15.16:0/508468281 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f401c0a0a30 msgr2=0x7f401c121220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:45.126 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:45.124+0000 7f4023ae1700 1 --2- 172.21.15.16:0/508468281 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f401c0a0a30 0x7f401c121220 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f4018009f50 tx=0x7f4018004750 comp rx=0 tx=0).stop 2023-09-30T22:36:45.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:45.124+0000 7f4023ae1700 1 -- 172.21.15.16:0/508468281 shutdown_connections 2023-09-30T22:36:45.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:45.124+0000 7f4023ae1700 1 --2- 172.21.15.16:0/508468281 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f401c133fa0 0x7f401c121ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:45.127 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:45.124+0000 7f4023ae1700 1 --2- 172.21.15.16:0/508468281 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f401c0a0a30 0x7f401c121220 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:45.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:45.124+0000 7f4023ae1700 1 --2- 172.21.15.16:0/508468281 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f401c0a1340 0x7f401c121760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:45.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:45.124+0000 7f4023ae1700 1 -- 172.21.15.16:0/508468281 >> 172.21.15.16:0/508468281 conn(0x7f401c0a6550 msgr2=0x7f401c0a4030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:45.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:45.124+0000 7f4023ae1700 1 -- 172.21.15.16:0/508468281 shutdown_connections 2023-09-30T22:36:45.128 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:45.124+0000 7f4023ae1700 1 -- 172.21.15.16:0/508468281 wait complete. 2023-09-30T22:36:45.131 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:45.130+0000 7fc742c83700 1 -- 172.21.15.16:0/502422966 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fc73c12dbf0 con 0x7fc73c133e30 2023-09-30T22:36:45.132 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:45.130+0000 7fc7397fa700 1 -- 172.21.15.16:0/502422966 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc728017af0 con 0x7fc73c133e30 2023-09-30T22:36:45.132 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:36:45.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:45.132+0000 7fc742c83700 1 -- 172.21.15.16:0/502422966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc73c133e30 msgr2=0x7fc73c09c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:45.136 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:45.132+0000 7fc742c83700 1 --2- 172.21.15.16:0/502422966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc73c133e30 0x7fc73c09c1f0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fc72800c780 tx=0x7fc72800ca90 comp rx=0 tx=0).stop 2023-09-30T22:36:45.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:45.132+0000 7fc742c83700 1 -- 172.21.15.16:0/502422966 shutdown_connections 2023-09-30T22:36:45.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:45.133+0000 7fc742c83700 1 --2- 172.21.15.16:0/502422966 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc73c133420 0x7fc73c09bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:45.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:45.133+0000 7fc742c83700 1 --2- 172.21.15.16:0/502422966 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc73c134260 0x7fc73c09c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:45.137 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:45.133+0000 7fc742c83700 1 --2- 172.21.15.16:0/502422966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc73c133e30 0x7fc73c09c1f0 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:45.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:45.133+0000 7fc742c83700 1 -- 172.21.15.16:0/502422966 >> 172.21.15.16:0/502422966 conn(0x7fc73c0a4210 msgr2=0x7fc73c1301c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:45.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:45.133+0000 7fc742c83700 1 -- 172.21.15.16:0/502422966 shutdown_connections 2023-09-30T22:36:45.138 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:45.133+0000 7fc742c83700 1 -- 172.21.15.16:0/502422966 wait complete. 2023-09-30T22:36:45.140 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:36:45.149 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:36:46.057 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:36:46.110 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:36:46.141 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:36:46.150 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:36:46.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.224+0000 7f91ae88b700 1 Processor -- start 2023-09-30T22:36:46.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.224+0000 7f91ae88b700 1 -- start start 2023-09-30T22:36:46.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.225+0000 7f91ae88b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f91a809e1d0 0x7f91a8138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:46.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.225+0000 7f91ae88b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f91a8139170 0x7f91a813b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:46.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.225+0000 7f91ae88b700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91a809b120 con 0x7f91a809d6f0 2023-09-30T22:36:46.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.225+0000 7f91ae88b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91a806b460 con 0x7f91a809e1d0 2023-09-30T22:36:46.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.225+0000 7f91ae88b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f91a809a7e0 con 0x7f91a8139170 2023-09-30T22:36:46.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.225+0000 7f91a77fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f91a809e1d0 0x7f91a8138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:46.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.225+0000 7f91a77fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f91a809e1d0 0x7f91a8138c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:59556/0 (socket says 172.21.15.16:59556) 2023-09-30T22:36:46.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.225+0000 7f91a7fff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f91a809d6f0 0x7f91a809dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60208/0 (socket says 172.21.15.16:60208) 2023-09-30T22:36:46.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.225+0000 7f91a77fe700 1 -- 172.21.15.16:0/2870292338 learned_addr learned my addr 172.21.15.16:0/2870292338 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:46.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.225+0000 7f91ace28700 1 --2- 172.21.15.16:0/2870292338 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f91a8139170 0x7f91a813b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:46.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.225+0000 7f91a6ffd700 1 -- 172.21.15.16:0/2870292338 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2357459682 0 0) 0x7f91a809b120 con 0x7f91a809d6f0 2023-09-30T22:36:46.227 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.225+0000 7f91a77fe700 1 -- 172.21.15.16:0/2870292338 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f91a8139170 msgr2=0x7f91a813b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:46.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.225+0000 7f91a77fe700 1 --2- 172.21.15.16:0/2870292338 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f91a8139170 0x7f91a813b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:46.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.225+0000 7f91a77fe700 1 -- 172.21.15.16:0/2870292338 >> v1:172.21.15.16:6789/0 conn(0x7f91a809d6f0 legacy=0x7f91a809dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:46.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.225+0000 7f91a77fe700 1 -- 172.21.15.16:0/2870292338 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91a809a500 con 0x7f91a809e1d0 2023-09-30T22:36:46.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.226+0000 7f91a77fe700 1 --2- 172.21.15.16:0/2870292338 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f91a809e1d0 0x7f91a8138c30 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f919c009920 tx=0x7f919c009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9bc77604190ed623 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:46.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.226+0000 7f91a6ffd700 1 -- 172.21.15.16:0/2870292338 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f919c011830 con 0x7f91a809e1d0 2023-09-30T22:36:46.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.226+0000 7f91a6ffd700 1 -- 172.21.15.16:0/2870292338 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f919c011990 con 0x7f91a809e1d0 2023-09-30T22:36:46.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.226+0000 7f91a6ffd700 1 -- 172.21.15.16:0/2870292338 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f919c011d20 con 0x7f91a809e1d0 2023-09-30T22:36:46.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.227+0000 7f91ae88b700 1 -- 172.21.15.16:0/2870292338 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f91a809e1d0 msgr2=0x7f91a8138c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:46.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.227+0000 7f91ae88b700 1 --2- 172.21.15.16:0/2870292338 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f91a809e1d0 0x7f91a8138c30 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f919c009920 tx=0x7f919c009c30 comp rx=0 tx=0).stop 2023-09-30T22:36:46.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.227+0000 7f91ae88b700 1 -- 172.21.15.16:0/2870292338 shutdown_connections 2023-09-30T22:36:46.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.227+0000 7f91ae88b700 1 --2- 172.21.15.16:0/2870292338 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f91a8139170 0x7f91a813b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:46.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.227+0000 7f91ae88b700 1 --2- 172.21.15.16:0/2870292338 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f91a809e1d0 0x7f91a8138c30 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:46.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.227+0000 7f91ae88b700 1 -- 172.21.15.16:0/2870292338 >> 172.21.15.16:0/2870292338 conn(0x7f91a8066f50 msgr2=0x7f91a809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:46.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.227+0000 7f91ae88b700 1 -- 172.21.15.16:0/2870292338 shutdown_connections 2023-09-30T22:36:46.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.227+0000 7f91ae88b700 1 -- 172.21.15.16:0/2870292338 wait complete. 2023-09-30T22:36:46.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.229+0000 7f91ae88b700 1 Processor -- start 2023-09-30T22:36:46.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.229+0000 7f91ae88b700 1 -- start start 2023-09-30T22:36:46.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.230+0000 7f91ae88b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f91a809d6f0 0x7f91a81205f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:46.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.230+0000 7f91ae88b700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91a809e1d0 0x7f91a8120b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:46.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.230+0000 7f91ae88b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f91a8139170 0x7f91a8121070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:46.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.230+0000 7f91ae88b700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91a81230a0 con 0x7f91a809e1d0 2023-09-30T22:36:46.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.230+0000 7f91ae88b700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91a8124110 con 0x7f91a809d6f0 2023-09-30T22:36:46.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.230+0000 7f91ae88b700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f91a81234c0 con 0x7f91a8139170 2023-09-30T22:36:46.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.230+0000 7f91a7fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f91a809d6f0 0x7f91a81205f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:46.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.230+0000 7f91a77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91a809e1d0 0x7f91a8120b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:46.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.230+0000 7f91a7fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f91a809d6f0 0x7f91a81205f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:59564/0 (socket says 172.21.15.16:59564) 2023-09-30T22:36:46.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.230+0000 7f91a77fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91a809e1d0 0x7f91a8120b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36630/0 (socket says 172.21.15.16:36630) 2023-09-30T22:36:46.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.230+0000 7f91a7fff700 1 -- 172.21.15.16:0/330506087 learned_addr learned my addr 172.21.15.16:0/330506087 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:46.232 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.230+0000 7f91ace28700 1 --2- 172.21.15.16:0/330506087 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f91a8139170 0x7f91a8121070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:46.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.230+0000 7f91a7fff700 1 -- 172.21.15.16:0/330506087 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f91a8139170 msgr2=0x7f91a8121070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:46.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.230+0000 7f91a7fff700 1 --2- 172.21.15.16:0/330506087 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f91a8139170 0x7f91a8121070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:46.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.231+0000 7f91a7fff700 1 -- 172.21.15.16:0/330506087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91a809e1d0 msgr2=0x7f91a8120b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:46.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.231+0000 7f91a7fff700 1 --2- 172.21.15.16:0/330506087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91a809e1d0 0x7f91a8120b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:46.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.231+0000 7f91a7fff700 1 -- 172.21.15.16:0/330506087 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f919c009550 con 0x7f91a809d6f0 2023-09-30T22:36:46.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.231+0000 7f91a77fe700 1 --2- 172.21.15.16:0/330506087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91a809e1d0 0x7f91a8120b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:46.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.231+0000 7f91a7fff700 1 --2- 172.21.15.16:0/330506087 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f91a809d6f0 0x7f91a81205f0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f919800c520 tx=0x7f919800c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:46.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.231+0000 7f91a4ff9700 1 -- 172.21.15.16:0/330506087 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9198004c80 con 0x7f91a809d6f0 2023-09-30T22:36:46.233 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.231+0000 7f91a4ff9700 1 -- 172.21.15.16:0/330506087 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9198004e00 con 0x7f91a809d6f0 2023-09-30T22:36:46.234 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.231+0000 7f91ae88b700 1 -- 172.21.15.16:0/330506087 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91a80a3b20 con 0x7f91a809d6f0 2023-09-30T22:36:46.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.231+0000 7f91a4ff9700 1 -- 172.21.15.16:0/330506087 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9198007310 con 0x7f91a809d6f0 2023-09-30T22:36:46.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.231+0000 7f91ae88b700 1 -- 172.21.15.16:0/330506087 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91a8136820 con 0x7f91a809d6f0 2023-09-30T22:36:46.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.232+0000 7f91a4ff9700 1 -- 172.21.15.16:0/330506087 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9198007470 con 0x7f91a809d6f0 2023-09-30T22:36:46.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.232+0000 7f91a4ff9700 1 -- 172.21.15.16:0/330506087 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f919803f320 con 0x7f91a809d6f0 2023-09-30T22:36:46.235 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.233+0000 7f91ae88b700 1 -- 172.21.15.16:0/330506087 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91a8074ea0 con 0x7f91a809d6f0 2023-09-30T22:36:46.238 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.238+0000 7f91a4ff9700 1 -- 172.21.15.16:0/330506087 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f91980123b0 con 0x7f91a809d6f0 2023-09-30T22:36:46.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.263+0000 7f6336902700 1 Processor -- start 2023-09-30T22:36:46.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.263+0000 7f6336902700 1 -- start start 2023-09-30T22:36:46.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.264+0000 7f6336902700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6330134630 0x7f6330136a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:46.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.264+0000 7f6336902700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6330136f50 0x7f6330139330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:46.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.264+0000 7f6336902700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f633009b4f0 con 0x7f6330131810 2023-09-30T22:36:46.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.264+0000 7f6336902700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63301313c0 con 0x7f6330136f50 2023-09-30T22:36:46.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.264+0000 7f6336902700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f633006e8c0 con 0x7f6330134630 2023-09-30T22:36:46.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.264+0000 7f632f7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6330134630 0x7f6330136a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:46.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.264+0000 7f632ffff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6330131810 0x7f6330131be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60210/0 (socket says 172.21.15.16:60210) 2023-09-30T22:36:46.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.264+0000 7f632ffff700 1 -- 172.21.15.16:0/2952975816 learned_addr learned my addr 172.21.15.16:0/2952975816 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:36:46.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.264+0000 7f6334e9f700 1 --2- 172.21.15.16:0/2952975816 >> v2:172.21.15.16:3300/0 conn(0x7f6330136f50 0x7f6330139330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:46.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.264+0000 7f632f7fe700 1 -- 172.21.15.16:0/2952975816 >> v2:172.21.15.16:3300/0 conn(0x7f6330136f50 msgr2=0x7f6330139330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:46.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.264+0000 7f632f7fe700 1 --2- 172.21.15.16:0/2952975816 >> v2:172.21.15.16:3300/0 conn(0x7f6330136f50 0x7f6330139330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:46.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.264+0000 7f632f7fe700 1 -- 172.21.15.16:0/2952975816 >> v1:172.21.15.16:6789/0 conn(0x7f6330131810 legacy=0x7f6330131be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:46.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.264+0000 7f632f7fe700 1 -- 172.21.15.16:0/2952975816 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6330071d20 con 0x7f6330134630 2023-09-30T22:36:46.269 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.265+0000 7f632f7fe700 1 --2- 172.21.15.16:0/2952975816 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6330134630 0x7f6330136a10 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f6324009840 tx=0x7f6324009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ce2990c4dd308fd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:46.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.265+0000 7f632effd700 1 -- 172.21.15.16:0/2952975816 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6324011830 con 0x7f6330134630 2023-09-30T22:36:46.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.265+0000 7f632effd700 1 -- 172.21.15.16:0/2952975816 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6324011990 con 0x7f6330134630 2023-09-30T22:36:46.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.265+0000 7f632effd700 1 -- 172.21.15.16:0/2952975816 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6324011d20 con 0x7f6330134630 2023-09-30T22:36:46.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.266+0000 7f6336902700 1 -- 172.21.15.16:0/2952975816 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6330134630 msgr2=0x7f6330136a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:46.270 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.266+0000 7f6336902700 1 --2- 172.21.15.16:0/2952975816 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6330134630 0x7f6330136a10 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f6324009840 tx=0x7f6324009b50 comp rx=0 tx=0).stop 2023-09-30T22:36:46.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.266+0000 7f6336902700 1 -- 172.21.15.16:0/2952975816 shutdown_connections 2023-09-30T22:36:46.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.266+0000 7f6336902700 1 --2- 172.21.15.16:0/2952975816 >> v2:172.21.15.16:3300/0 conn(0x7f6330136f50 0x7f6330139330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:46.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.266+0000 7f6336902700 1 --2- 172.21.15.16:0/2952975816 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6330134630 0x7f6330136a10 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:46.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.266+0000 7f6336902700 1 -- 172.21.15.16:0/2952975816 >> 172.21.15.16:0/2952975816 conn(0x7f63300a2aa0 msgr2=0x7f63300a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:46.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.266+0000 7f6336902700 1 -- 172.21.15.16:0/2952975816 shutdown_connections 2023-09-30T22:36:46.271 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.266+0000 7f6336902700 1 -- 172.21.15.16:0/2952975816 wait complete. 2023-09-30T22:36:46.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.267+0000 7f6336902700 1 Processor -- start 2023-09-30T22:36:46.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.268+0000 7f6336902700 1 -- start start 2023-09-30T22:36:46.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.268+0000 7f6336902700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6330131810 0x7f63301c6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:46.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.268+0000 7f6336902700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6330134630 0x7f63301c6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:46.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.268+0000 7f6336902700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6330136f50 0x7f63301c6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:46.274 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.268+0000 7f6336902700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633011a910 con 0x7f6330136f50 2023-09-30T22:36:46.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.268+0000 7f6336902700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f633009e5e0 con 0x7f6330134630 2023-09-30T22:36:46.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.268+0000 7f6336902700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f633011ba30 con 0x7f6330131810 2023-09-30T22:36:46.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.269+0000 7f632ffff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6330131810 0x7f63301c6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:46.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.269+0000 7f632ffff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6330131810 0x7f63301c6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35794/0 (socket says 172.21.15.16:35794) 2023-09-30T22:36:46.275 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.269+0000 7f632ffff700 1 -- 172.21.15.16:0/2782611331 learned_addr learned my addr 172.21.15.16:0/2782611331 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:46.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.269+0000 7f6334e9f700 1 --2- 172.21.15.16:0/2782611331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6330136f50 0x7f63301c6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:46.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.269+0000 7f632ffff700 1 -- 172.21.15.16:0/2782611331 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6330134630 msgr2=0x7f63301c6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:46.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.269+0000 7f632ffff700 1 --2- 172.21.15.16:0/2782611331 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6330134630 0x7f63301c6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:46.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.269+0000 7f632ffff700 1 -- 172.21.15.16:0/2782611331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6330136f50 msgr2=0x7f63301c6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:46.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.269+0000 7f632ffff700 1 --2- 172.21.15.16:0/2782611331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6330136f50 0x7f63301c6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:46.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.269+0000 7f632ffff700 1 -- 172.21.15.16:0/2782611331 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63240094a0 con 0x7f6330131810 2023-09-30T22:36:46.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.269+0000 7f6334e9f700 1 --2- 172.21.15.16:0/2782611331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6330136f50 0x7f63301c6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:46.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.269+0000 7f632ffff700 1 --2- 172.21.15.16:0/2782611331 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6330131810 0x7f63301c6240 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f632000c4b0 tx=0x7f632000c7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:46.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.269+0000 7f632cff9700 1 -- 172.21.15.16:0/2782611331 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6320005d70 con 0x7f6330131810 2023-09-30T22:36:46.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.270+0000 7f632cff9700 1 -- 172.21.15.16:0/2782611331 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63200070d0 con 0x7f6330131810 2023-09-30T22:36:46.276 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.270+0000 7f6336902700 1 -- 172.21.15.16:0/2782611331 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f633011a340 con 0x7f6330131810 2023-09-30T22:36:46.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.270+0000 7f632cff9700 1 -- 172.21.15.16:0/2782611331 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6320007440 con 0x7f6330131810 2023-09-30T22:36:46.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.270+0000 7f6336902700 1 -- 172.21.15.16:0/2782611331 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f633011b750 con 0x7f6330131810 2023-09-30T22:36:46.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.270+0000 7f632cff9700 1 -- 172.21.15.16:0/2782611331 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f63200075a0 con 0x7f6330131810 2023-09-30T22:36:46.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.271+0000 7f632cff9700 1 -- 172.21.15.16:0/2782611331 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f632003ebb0 con 0x7f6330131810 2023-09-30T22:36:46.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.271+0000 7f6336902700 1 -- 172.21.15.16:0/2782611331 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f633004f2a0 con 0x7f6330131810 2023-09-30T22:36:46.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.276+0000 7f632cff9700 1 -- 172.21.15.16:0/2782611331 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6320019b40 con 0x7f6330131810 2023-09-30T22:36:46.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.307+0000 7f398a131700 1 Processor -- start 2023-09-30T22:36:46.308 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.307+0000 7f398a131700 1 -- start start 2023-09-30T22:36:46.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.308+0000 7f398a131700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3984130670 0x7f3984130a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:46.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.308+0000 7f398a131700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f39840701a0 0x7f3984138540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:46.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.308+0000 7f398a131700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3984130260 con 0x7f398406f680 2023-09-30T22:36:46.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.308+0000 7f398a131700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3984071ff0 con 0x7f39840701a0 2023-09-30T22:36:46.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.308+0000 7f398a131700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f398409ecf0 con 0x7f3984130670 2023-09-30T22:36:46.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.308+0000 7f39837fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3984130670 0x7f3984130a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:46.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.308+0000 7f3982ffd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f398406f680 0x7f398406fa90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60224/0 (socket says 172.21.15.16:60224) 2023-09-30T22:36:46.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.308+0000 7f3982ffd700 1 -- 172.21.15.16:0/1412622489 learned_addr learned my addr 172.21.15.16:0/1412622489 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:36:46.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.308+0000 7f39837fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3984130670 0x7f3984130a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35796/0 (socket says 172.21.15.16:35796) 2023-09-30T22:36:46.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.308+0000 7f3983fff700 1 --2- 172.21.15.16:0/1412622489 >> v2:172.21.15.16:3300/0 conn(0x7f39840701a0 0x7f3984138540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:46.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.308+0000 7f39827fc700 1 -- 172.21.15.16:0/1412622489 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4243761976 0 0) 0x7f3984130260 con 0x7f398406f680 2023-09-30T22:36:46.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.308+0000 7f39837fe700 1 -- 172.21.15.16:0/1412622489 >> v2:172.21.15.16:3300/0 conn(0x7f39840701a0 msgr2=0x7f3984138540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:46.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.308+0000 7f39837fe700 1 --2- 172.21.15.16:0/1412622489 >> v2:172.21.15.16:3300/0 conn(0x7f39840701a0 0x7f3984138540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:46.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.308+0000 7f39837fe700 1 -- 172.21.15.16:0/1412622489 >> v1:172.21.15.16:6789/0 conn(0x7f398406f680 legacy=0x7f398406fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:46.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.308+0000 7f39837fe700 1 -- 172.21.15.16:0/1412622489 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3984070e70 con 0x7f3984130670 2023-09-30T22:36:46.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.309+0000 7f39837fe700 1 --2- 172.21.15.16:0/1412622489 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3984130670 0x7f3984130a40 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f397400bae0 tx=0x7f397400bdf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51b273c3eed41d0e server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:46.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.309+0000 7f39827fc700 1 -- 172.21.15.16:0/1412622489 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397400dc70 con 0x7f3984130670 2023-09-30T22:36:46.310 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.309+0000 7f39827fc700 1 -- 172.21.15.16:0/1412622489 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f397400ddd0 con 0x7f3984130670 2023-09-30T22:36:46.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.309+0000 7f39827fc700 1 -- 172.21.15.16:0/1412622489 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3974010690 con 0x7f3984130670 2023-09-30T22:36:46.311 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.309+0000 7f398a131700 1 -- 172.21.15.16:0/1412622489 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3984130670 msgr2=0x7f3984130a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:46.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.309+0000 7f398a131700 1 --2- 172.21.15.16:0/1412622489 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3984130670 0x7f3984130a40 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f397400bae0 tx=0x7f397400bdf0 comp rx=0 tx=0).stop 2023-09-30T22:36:46.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.310+0000 7f398a131700 1 -- 172.21.15.16:0/1412622489 shutdown_connections 2023-09-30T22:36:46.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.310+0000 7f398a131700 1 --2- 172.21.15.16:0/1412622489 >> v2:172.21.15.16:3300/0 conn(0x7f39840701a0 0x7f3984138540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:46.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.310+0000 7f398a131700 1 --2- 172.21.15.16:0/1412622489 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3984130670 0x7f3984130a40 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:46.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.310+0000 7f398a131700 1 -- 172.21.15.16:0/1412622489 >> 172.21.15.16:0/1412622489 conn(0x7f39840a2aa0 msgr2=0x7f39840a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:46.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.310+0000 7f398a131700 1 -- 172.21.15.16:0/1412622489 shutdown_connections 2023-09-30T22:36:46.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.310+0000 7f398a131700 1 -- 172.21.15.16:0/1412622489 wait complete. 2023-09-30T22:36:46.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.311+0000 7f398a131700 1 Processor -- start 2023-09-30T22:36:46.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.311+0000 7f398a131700 1 -- start start 2023-09-30T22:36:46.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.312+0000 7f398a131700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f398406f680 0x7f398409be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:46.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.312+0000 7f398a131700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f39840701a0 0x7f398409c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:46.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.312+0000 7f398a131700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3984130670 0x7f398409c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:46.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.312+0000 7f398a131700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398411a930 con 0x7f398406f680 2023-09-30T22:36:46.314 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.312+0000 7f398a131700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f398409da60 con 0x7f3984130670 2023-09-30T22:36:46.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.312+0000 7f398a131700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f39841192b0 con 0x7f39840701a0 2023-09-30T22:36:46.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.312+0000 7f39837fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f398406f680 0x7f398409be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:46.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.312+0000 7f39837fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f398406f680 0x7f398409be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36680/0 (socket says 172.21.15.16:36680) 2023-09-30T22:36:46.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.312+0000 7f39837fe700 1 -- 172.21.15.16:0/826226216 learned_addr learned my addr 172.21.15.16:0/826226216 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:46.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.312+0000 7f3983fff700 1 --2- 172.21.15.16:0/826226216 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3984130670 0x7f398409c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:46.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.312+0000 7f3982ffd700 1 --2- 172.21.15.16:0/826226216 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f39840701a0 0x7f398409c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:46.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.313+0000 7f39837fe700 1 -- 172.21.15.16:0/826226216 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f39840701a0 msgr2=0x7f398409c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:46.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.313+0000 7f39837fe700 1 --2- 172.21.15.16:0/826226216 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f39840701a0 0x7f398409c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:46.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.313+0000 7f39837fe700 1 -- 172.21.15.16:0/826226216 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3984130670 msgr2=0x7f398409c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:46.315 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.313+0000 7f39837fe700 1 --2- 172.21.15.16:0/826226216 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3984130670 0x7f398409c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:46.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.313+0000 7f39837fe700 1 -- 172.21.15.16:0/826226216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f397400b740 con 0x7f398406f680 2023-09-30T22:36:46.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.313+0000 7f3982ffd700 1 --2- 172.21.15.16:0/826226216 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f39840701a0 0x7f398409c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:46.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.313+0000 7f39837fe700 1 --2- 172.21.15.16:0/826226216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f398406f680 0x7f398409be50 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f398409ee30 tx=0x7f3974004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:46.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.313+0000 7f396bfff700 1 -- 172.21.15.16:0/826226216 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397401f070 con 0x7f398406f680 2023-09-30T22:36:46.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.313+0000 7f396bfff700 1 -- 172.21.15.16:0/826226216 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39740038b0 con 0x7f398406f680 2023-09-30T22:36:46.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.313+0000 7f398a131700 1 -- 172.21.15.16:0/826226216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3984119c90 con 0x7f398406f680 2023-09-30T22:36:46.316 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.313+0000 7f396bfff700 1 -- 172.21.15.16:0/826226216 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3974003c60 con 0x7f398406f680 2023-09-30T22:36:46.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.313+0000 7f398a131700 1 -- 172.21.15.16:0/826226216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3984071820 con 0x7f398406f680 2023-09-30T22:36:46.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.314+0000 7f396bfff700 1 -- 172.21.15.16:0/826226216 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3974010470 con 0x7f398406f680 2023-09-30T22:36:46.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.315+0000 7f398a131700 1 -- 172.21.15.16:0/826226216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3984074ea0 con 0x7f398406f680 2023-09-30T22:36:46.317 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.315+0000 7f396bfff700 1 -- 172.21.15.16:0/826226216 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f3974042c90 con 0x7f398406f680 2023-09-30T22:36:46.324 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.324+0000 7f396bfff700 1 -- 172.21.15.16:0/826226216 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3974029020 con 0x7f398406f680 2023-09-30T22:36:46.334 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.334+0000 7fd9c5aa1700 1 Processor -- start 2023-09-30T22:36:46.335 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.334+0000 7fd9c5aa1700 1 -- start start 2023-09-30T22:36:46.336 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.335+0000 7fd9c5aa1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd9c009e930 0x7fd9c009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:46.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.335+0000 7fd9c5aa1700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd9c009f650 0x7fd9c0131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:46.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.335+0000 7fd9c5aa1700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9c0070940 con 0x7fd9c009f240 2023-09-30T22:36:46.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.335+0000 7fd9c5aa1700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9c006fd60 con 0x7fd9c009f650 2023-09-30T22:36:46.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.335+0000 7fd9c5aa1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c006fa80 con 0x7fd9c009e930 2023-09-30T22:36:46.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.335+0000 7fd9beffd700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd9c009f240 0x7fd9c01317f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60230/0 (socket says 172.21.15.16:60230) 2023-09-30T22:36:46.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.335+0000 7fd9beffd700 1 -- 172.21.15.16:0/325477539 learned_addr learned my addr 172.21.15.16:0/325477539 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:36:46.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.335+0000 7fd9bffff700 1 --2- 172.21.15.16:0/325477539 >> v2:172.21.15.16:3300/0 conn(0x7fd9c009f650 0x7fd9c0131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:46.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.335+0000 7fd9be7fc700 1 -- 172.21.15.16:0/325477539 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2622706139 0 0) 0x7fd9c0070940 con 0x7fd9c009f240 2023-09-30T22:36:46.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.335+0000 7fd9be7fc700 1 -- 172.21.15.16:0/325477539 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9a80035b0 con 0x7fd9c009f240 2023-09-30T22:36:46.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.335+0000 7fd9bf7fe700 1 --2- 172.21.15.16:0/325477539 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd9c009e930 0x7fd9c009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:46.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.335+0000 7fd9be7fc700 1 -- 172.21.15.16:0/325477539 <== mon.0 v1:172.21.15.16:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3053117799 0 0) 0x7fd9a80035b0 con 0x7fd9c009f240 2023-09-30T22:36:46.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.335+0000 7fd9be7fc700 1 -- 172.21.15.16:0/325477539 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd9c009e930 msgr2=0x7fd9c009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:46.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.335+0000 7fd9be7fc700 1 --2- 172.21.15.16:0/325477539 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd9c009e930 0x7fd9c009ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:46.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.336+0000 7fd9be7fc700 1 -- 172.21.15.16:0/325477539 >> v2:172.21.15.16:3300/0 conn(0x7fd9c009f650 msgr2=0x7fd9c0131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:46.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.336+0000 7fd9be7fc700 1 --2- 172.21.15.16:0/325477539 >> v2:172.21.15.16:3300/0 conn(0x7fd9c009f650 0x7fd9c0131f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:46.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.336+0000 7fd9be7fc700 1 -- 172.21.15.16:0/325477539 --> v1:172.21.15.16:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c009fb60 con 0x7fd9c009f240 2023-09-30T22:36:46.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.336+0000 7fd9be7fc700 1 -- 172.21.15.16:0/325477539 <== mon.0 v1:172.21.15.16:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1074163509 0 0) 0x7fd9b4001670 con 0x7fd9c009f240 2023-09-30T22:36:46.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.336+0000 7fd9bffff700 1 --2- 172.21.15.16:0/325477539 >> v2:172.21.15.16:3300/0 conn(0x7fd9c009f650 0x7fd9c0131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:46.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.336+0000 7fd9be7fc700 1 -- 172.21.15.16:0/325477539 >> v1:172.21.15.16:6789/0 conn(0x7fd9c009f240 legacy=0x7fd9c01317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:46.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.336+0000 7fd9be7fc700 1 --2- 172.21.15.16:0/325477539 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd9c009e930 0x7fd9a8003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:46.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.336+0000 7fd9be7fc700 1 --2- 172.21.15.16:0/325477539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9a8003f80 0x7fd9a8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:46.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.336+0000 7fd9be7fc700 1 --2- 172.21.15.16:0/325477539 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd9a8004d90 0x7fd9a8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:46.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.336+0000 7fd9be7fc700 1 -- 172.21.15.16:0/325477539 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c006fd60 con 0x7fd9a8003f80 2023-09-30T22:36:46.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.336+0000 7fd9be7fc700 1 -- 172.21.15.16:0/325477539 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c006fa80 con 0x7fd9a8004d90 2023-09-30T22:36:46.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.336+0000 7fd9be7fc700 1 -- 172.21.15.16:0/325477539 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a800ba40 con 0x7fd9c009e930 2023-09-30T22:36:46.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.336+0000 7fd9beffd700 1 --2- 172.21.15.16:0/325477539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9a8003f80 0x7fd9a8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:46.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.336+0000 7fd9bffff700 1 --2- 172.21.15.16:0/325477539 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd9a8004d90 0x7fd9a8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:46.339 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.336+0000 7fd9bf7fe700 1 --2- 172.21.15.16:0/325477539 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd9c009e930 0x7fd9a8003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:46.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.336+0000 7fd9bf7fe700 1 -- 172.21.15.16:0/325477539 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd9a8004d90 msgr2=0x7fd9a8005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:46.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.336+0000 7fd9bf7fe700 1 --2- 172.21.15.16:0/325477539 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd9a8004d90 0x7fd9a8005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:46.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.336+0000 7fd9bf7fe700 1 -- 172.21.15.16:0/325477539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9a8003f80 msgr2=0x7fd9a8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:46.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.336+0000 7fd9bf7fe700 1 --2- 172.21.15.16:0/325477539 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9a8003f80 0x7fd9a8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:46.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.336+0000 7fd9bf7fe700 1 -- 172.21.15.16:0/325477539 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a80035b0 con 0x7fd9c009e930 2023-09-30T22:36:46.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.336+0000 7fd9bf7fe700 1 --2- 172.21.15.16:0/325477539 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd9c009e930 0x7fd9a8003a00 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0009b60 tx=0x7fd9a8005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec0ce61dd57fa2f0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:46.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.337+0000 7fd9be7fc700 1 -- 172.21.15.16:0/325477539 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b000e660 con 0x7fd9c009e930 2023-09-30T22:36:46.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.337+0000 7fd9be7fc700 1 -- 172.21.15.16:0/325477539 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b000e7c0 con 0x7fd9c009e930 2023-09-30T22:36:46.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.337+0000 7fd9be7fc700 1 -- 172.21.15.16:0/325477539 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b000eb30 con 0x7fd9c009e930 2023-09-30T22:36:46.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.338+0000 7fd9c5aa1700 1 -- 172.21.15.16:0/325477539 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd9c009e930 msgr2=0x7fd9a8003a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:46.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.338+0000 7fd9c5aa1700 1 --2- 172.21.15.16:0/325477539 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd9c009e930 0x7fd9a8003a00 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0009b60 tx=0x7fd9a8005880 comp rx=0 tx=0).stop 2023-09-30T22:36:46.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.338+0000 7fd9bffff700 1 -- 172.21.15.16:0/325477539 reap_dead start 2023-09-30T22:36:46.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.338+0000 7fd9c5aa1700 1 -- 172.21.15.16:0/325477539 shutdown_connections 2023-09-30T22:36:46.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.338+0000 7fd9c5aa1700 1 -- 172.21.15.16:0/325477539 >> 172.21.15.16:0/325477539 conn(0x7fd9c00a4210 msgr2=0x7fd9c00a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:46.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.338+0000 7fd9c5aa1700 1 -- 172.21.15.16:0/325477539 shutdown_connections 2023-09-30T22:36:46.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.338+0000 7fd9c5aa1700 1 -- 172.21.15.16:0/325477539 wait complete. 2023-09-30T22:36:46.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.340+0000 7fd9c5aa1700 1 Processor -- start 2023-09-30T22:36:46.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.340+0000 7fd9c5aa1700 1 -- start start 2023-09-30T22:36:46.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.341+0000 7fd9c5aa1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd9c009e930 0x7fd9c01a1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:46.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.341+0000 7fd9c5aa1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd9c01b4f60 0x7fd9c01a2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:46.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.341+0000 7fd9c5aa1700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9c01a26d0 0x7fd9c01ca820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:46.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.341+0000 7fd9c5aa1700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c01a2b00 con 0x7fd9c01a26d0 2023-09-30T22:36:46.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.341+0000 7fd9c5aa1700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c01a3780 con 0x7fd9c01b4f60 2023-09-30T22:36:46.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.341+0000 7fd9c5aa1700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c01a3da0 con 0x7fd9c009e930 2023-09-30T22:36:46.344 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.341+0000 7fd9beffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd9c01b4f60 0x7fd9c01a2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:46.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.341+0000 7fd9bf7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd9c009e930 0x7fd9c01a1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:46.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.341+0000 7fd9bf7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd9c009e930 0x7fd9c01a1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:35826/0 (socket says 172.21.15.16:35826) 2023-09-30T22:36:46.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.341+0000 7fd9bf7fe700 1 -- 172.21.15.16:0/4225681260 learned_addr learned my addr 172.21.15.16:0/4225681260 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:46.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.341+0000 7fd9bffff700 1 --2- 172.21.15.16:0/4225681260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9c01a26d0 0x7fd9c01ca820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:46.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.341+0000 7fd9bf7fe700 1 -- 172.21.15.16:0/4225681260 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd9c01b4f60 msgr2=0x7fd9c01a2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:46.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.341+0000 7fd9bf7fe700 1 --2- 172.21.15.16:0/4225681260 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd9c01b4f60 0x7fd9c01a2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:46.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.341+0000 7fd9bf7fe700 1 -- 172.21.15.16:0/4225681260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9c01a26d0 msgr2=0x7fd9c01ca820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:46.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.341+0000 7fd9bf7fe700 1 --2- 172.21.15.16:0/4225681260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9c01a26d0 0x7fd9c01ca820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:46.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.341+0000 7fd9bf7fe700 1 -- 172.21.15.16:0/4225681260 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c00a0190 con 0x7fd9c009e930 2023-09-30T22:36:46.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.342+0000 7fd9bf7fe700 1 --2- 172.21.15.16:0/4225681260 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd9c009e930 0x7fd9c01a1c50 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fd9a800bbd0 tx=0x7fd9b00047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:46.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.342+0000 7fd9a7fff700 1 -- 172.21.15.16:0/4225681260 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b0021070 con 0x7fd9c009e930 2023-09-30T22:36:46.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.342+0000 7fd9a7fff700 1 -- 172.21.15.16:0/4225681260 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b0004430 con 0x7fd9c009e930 2023-09-30T22:36:46.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.342+0000 7fd9c5aa1700 1 -- 172.21.15.16:0/4225681260 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9c01a49d0 con 0x7fd9c009e930 2023-09-30T22:36:46.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.342+0000 7fd9a7fff700 1 -- 172.21.15.16:0/4225681260 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b000e660 con 0x7fd9c009e930 2023-09-30T22:36:46.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.342+0000 7fd9c5aa1700 1 -- 172.21.15.16:0/4225681260 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9c01a5fc0 con 0x7fd9c009e930 2023-09-30T22:36:46.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.343+0000 7fd9a7fff700 1 -- 172.21.15.16:0/4225681260 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd9b0025070 con 0x7fd9c009e930 2023-09-30T22:36:46.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.343+0000 7fd9a7fff700 1 -- 172.21.15.16:0/4225681260 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd9b0040960 con 0x7fd9c009e930 2023-09-30T22:36:46.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.343+0000 7fd9c5aa1700 1 -- 172.21.15.16:0/4225681260 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9c00a34a0 con 0x7fd9c009e930 2023-09-30T22:36:46.353 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.353+0000 7fd9a7fff700 1 -- 172.21.15.16:0/4225681260 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd9b0017080 con 0x7fd9c009e930 2023-09-30T22:36:46.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.491+0000 7f6336902700 1 -- 172.21.15.16:0/2782611331 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f6330074ea0 con 0x7f6330131810 2023-09-30T22:36:46.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.492+0000 7f632cff9700 1 -- 172.21.15.16:0/2782611331 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6320017020 con 0x7f6330131810 2023-09-30T22:36:46.492 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:36:46.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.494+0000 7f6336902700 1 -- 172.21.15.16:0/2782611331 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6330131810 msgr2=0x7f63301c6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:46.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.494+0000 7f6336902700 1 --2- 172.21.15.16:0/2782611331 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6330131810 0x7f63301c6240 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f632000c4b0 tx=0x7f632000c7c0 comp rx=0 tx=0).stop 2023-09-30T22:36:46.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.494+0000 7f6336902700 1 -- 172.21.15.16:0/2782611331 shutdown_connections 2023-09-30T22:36:46.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.494+0000 7f6336902700 1 --2- 172.21.15.16:0/2782611331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6330136f50 0x7f63301c6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:46.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.494+0000 7f6336902700 1 --2- 172.21.15.16:0/2782611331 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6330131810 0x7f63301c6240 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:46.495 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.494+0000 7f6336902700 1 --2- 172.21.15.16:0/2782611331 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6330134630 0x7f63301c6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:46.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.494+0000 7f6336902700 1 -- 172.21.15.16:0/2782611331 >> 172.21.15.16:0/2782611331 conn(0x7f63300a2aa0 msgr2=0x7f633012e030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:46.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.495+0000 7f6336902700 1 -- 172.21.15.16:0/2782611331 shutdown_connections 2023-09-30T22:36:46.496 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.495+0000 7f6336902700 1 -- 172.21.15.16:0/2782611331 wait complete. 2023-09-30T22:36:46.509 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:36:46.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.538+0000 7f398a131700 1 -- 172.21.15.16:0/826226216 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3984072210 con 0x7f398406f680 2023-09-30T22:36:46.538 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.538+0000 7f396bfff700 1 -- 172.21.15.16:0/826226216 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f39740233d0 con 0x7f398406f680 2023-09-30T22:36:46.539 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:36:46.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.541+0000 7f398a131700 1 -- 172.21.15.16:0/826226216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f398406f680 msgr2=0x7f398409be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:46.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.541+0000 7f398a131700 1 --2- 172.21.15.16:0/826226216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f398406f680 0x7f398409be50 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f398409ee30 tx=0x7f3974004750 comp rx=0 tx=0).stop 2023-09-30T22:36:46.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.541+0000 7f398a131700 1 -- 172.21.15.16:0/826226216 shutdown_connections 2023-09-30T22:36:46.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.541+0000 7f398a131700 1 --2- 172.21.15.16:0/826226216 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3984130670 0x7f398409c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:46.542 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.541+0000 7f398a131700 1 --2- 172.21.15.16:0/826226216 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f398406f680 0x7f398409be50 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:46.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.541+0000 7f398a131700 1 --2- 172.21.15.16:0/826226216 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f39840701a0 0x7f398409c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:46.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.541+0000 7f398a131700 1 -- 172.21.15.16:0/826226216 >> 172.21.15.16:0/826226216 conn(0x7f39840a2aa0 msgr2=0x7f398412b450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:46.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.542+0000 7f398a131700 1 -- 172.21.15.16:0/826226216 shutdown_connections 2023-09-30T22:36:46.543 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.542+0000 7f398a131700 1 -- 172.21.15.16:0/826226216 wait complete. 2023-09-30T22:36:46.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.546+0000 7f91ae88b700 1 -- 172.21.15.16:0/330506087 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f91a8124900 con 0x7f91a809d6f0 2023-09-30T22:36:46.547 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.547+0000 7f91a4ff9700 1 -- 172.21.15.16:0/330506087 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9198011360 con 0x7f91a809d6f0 2023-09-30T22:36:46.547 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:36:46.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.551+0000 7f91ae88b700 1 -- 172.21.15.16:0/330506087 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f91a809d6f0 msgr2=0x7f91a81205f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:46.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.551+0000 7f91ae88b700 1 --2- 172.21.15.16:0/330506087 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f91a809d6f0 0x7f91a81205f0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f919800c520 tx=0x7f919800c830 comp rx=0 tx=0).stop 2023-09-30T22:36:46.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.551+0000 7f91ae88b700 1 -- 172.21.15.16:0/330506087 shutdown_connections 2023-09-30T22:36:46.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.551+0000 7f91ae88b700 1 --2- 172.21.15.16:0/330506087 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f91a809d6f0 0x7f91a81205f0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:46.552 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.551+0000 7f91ae88b700 1 --2- 172.21.15.16:0/330506087 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f91a8139170 0x7f91a8121070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:46.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.551+0000 7f91ae88b700 1 --2- 172.21.15.16:0/330506087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91a809e1d0 0x7f91a8120b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:46.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.551+0000 7f91ae88b700 1 -- 172.21.15.16:0/330506087 >> 172.21.15.16:0/330506087 conn(0x7f91a8066f50 msgr2=0x7f91a8136a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:46.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.551+0000 7f91ae88b700 1 -- 172.21.15.16:0/330506087 shutdown_connections 2023-09-30T22:36:46.553 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.551+0000 7f91ae88b700 1 -- 172.21.15.16:0/330506087 wait complete. 2023-09-30T22:36:46.559 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:36:46.566 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:36:46.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.611+0000 7fd9c5aa1700 1 -- 172.21.15.16:0/4225681260 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd9c0071d20 con 0x7fd9c009e930 2023-09-30T22:36:46.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.612+0000 7fd9a7fff700 1 -- 172.21.15.16:0/4225681260 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd9b002c020 con 0x7fd9c009e930 2023-09-30T22:36:46.612 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:36:46.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.614+0000 7fd9c5aa1700 1 -- 172.21.15.16:0/4225681260 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd9c009e930 msgr2=0x7fd9c01a1c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:46.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.614+0000 7fd9c5aa1700 1 --2- 172.21.15.16:0/4225681260 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd9c009e930 0x7fd9c01a1c50 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fd9a800bbd0 tx=0x7fd9b00047c0 comp rx=0 tx=0).stop 2023-09-30T22:36:46.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.614+0000 7fd9c5aa1700 1 -- 172.21.15.16:0/4225681260 shutdown_connections 2023-09-30T22:36:46.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.614+0000 7fd9c5aa1700 1 --2- 172.21.15.16:0/4225681260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd9c01a26d0 0x7fd9c01ca820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:46.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.614+0000 7fd9c5aa1700 1 --2- 172.21.15.16:0/4225681260 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd9c009e930 0x7fd9c01a1c50 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:46.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.614+0000 7fd9c5aa1700 1 --2- 172.21.15.16:0/4225681260 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd9c01b4f60 0x7fd9c01a2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:46.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.614+0000 7fd9c5aa1700 1 -- 172.21.15.16:0/4225681260 >> 172.21.15.16:0/4225681260 conn(0x7fd9c00a4210 msgr2=0x7fd9c01b6c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:46.615 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.614+0000 7fd9c5aa1700 1 -- 172.21.15.16:0/4225681260 shutdown_connections 2023-09-30T22:36:46.616 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:46.614+0000 7fd9c5aa1700 1 -- 172.21.15.16:0/4225681260 wait complete. 2023-09-30T22:36:46.628 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:36:47.510 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:36:47.560 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:36:47.567 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:36:47.629 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:36:47.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.664+0000 7f3c34178700 1 Processor -- start 2023-09-30T22:36:47.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.664+0000 7f3c34178700 1 -- start start 2023-09-30T22:36:47.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.665+0000 7f3c34178700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f3c2c1338a0 0x7f3c2c133c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:47.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.665+0000 7f3c34178700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3c2c09de50 0x7f3c2c09e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:47.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.665+0000 7f3c34178700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3c2c09e760 0x7f3c2c13a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:47.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.665+0000 7f3c34178700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c2c06fef0 con 0x7f3c2c1338a0 2023-09-30T22:36:47.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.665+0000 7f3c34178700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2c06ef30 con 0x7f3c2c09e760 2023-09-30T22:36:47.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.665+0000 7f3c34178700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2c1334b0 con 0x7f3c2c09de50 2023-09-30T22:36:47.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.665+0000 7f3c32715700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3c2c09e760 0x7f3c2c13a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:47.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.665+0000 7f3c32715700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3c2c09e760 0x7f3c2c13a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37610/0 (socket says 172.21.15.16:37610) 2023-09-30T22:36:47.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.665+0000 7f3c32715700 1 -- 172.21.15.16:0/4149012230 learned_addr learned my addr 172.21.15.16:0/4149012230 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:47.674 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.665+0000 7f3c31713700 1 --2- 172.21.15.16:0/4149012230 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3c2c09de50 0x7f3c2c09e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:47.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.665+0000 7f3c31f14700 1 --2- 172.21.15.16:0/4149012230 >> v2:172.21.15.16:3300/0 conn(0x7f3c2c1338a0 0x7f3c2c133c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:47.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.665+0000 7f3c32715700 1 -- 172.21.15.16:0/4149012230 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3c2c09de50 msgr2=0x7f3c2c09e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:47.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.665+0000 7f3c32715700 1 --2- 172.21.15.16:0/4149012230 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3c2c09de50 0x7f3c2c09e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:47.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.665+0000 7f3c32715700 1 -- 172.21.15.16:0/4149012230 >> v2:172.21.15.16:3300/0 conn(0x7f3c2c1338a0 msgr2=0x7f3c2c133c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:47.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.665+0000 7f3c32715700 1 --2- 172.21.15.16:0/4149012230 >> v2:172.21.15.16:3300/0 conn(0x7f3c2c1338a0 0x7f3c2c133c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:47.675 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.665+0000 7f3c32715700 1 -- 172.21.15.16:0/4149012230 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c2c06fc30 con 0x7f3c2c09e760 2023-09-30T22:36:47.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.666+0000 7f3c32715700 1 --2- 172.21.15.16:0/4149012230 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3c2c09e760 0x7f3c2c13a730 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f3c24009e50 tx=0x7f3c2400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ac11bad2cfd3cd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:47.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.666+0000 7f3c30f12700 1 -- 172.21.15.16:0/4149012230 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2400e070 con 0x7f3c2c09e760 2023-09-30T22:36:47.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.666+0000 7f3c30f12700 1 -- 172.21.15.16:0/4149012230 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c2400b910 con 0x7f3c2c09e760 2023-09-30T22:36:47.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.666+0000 7f3c30f12700 1 -- 172.21.15.16:0/4149012230 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c2400bcc0 con 0x7f3c2c09e760 2023-09-30T22:36:47.676 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.667+0000 7f3c34178700 1 -- 172.21.15.16:0/4149012230 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3c2c09e760 msgr2=0x7f3c2c13a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:47.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.667+0000 7f3c34178700 1 --2- 172.21.15.16:0/4149012230 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3c2c09e760 0x7f3c2c13a730 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f3c24009e50 tx=0x7f3c2400b330 comp rx=0 tx=0).stop 2023-09-30T22:36:47.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.667+0000 7f3c34178700 1 -- 172.21.15.16:0/4149012230 shutdown_connections 2023-09-30T22:36:47.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.667+0000 7f3c34178700 1 --2- 172.21.15.16:0/4149012230 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3c2c09e760 0x7f3c2c13a730 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:47.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.667+0000 7f3c34178700 1 --2- 172.21.15.16:0/4149012230 >> v2:172.21.15.16:3300/0 conn(0x7f3c2c1338a0 0x7f3c2c133c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:47.677 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.667+0000 7f3c34178700 1 --2- 172.21.15.16:0/4149012230 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3c2c09de50 0x7f3c2c09e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:47.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.667+0000 7f3c34178700 1 -- 172.21.15.16:0/4149012230 >> 172.21.15.16:0/4149012230 conn(0x7f3c2c0a2aa0 msgr2=0x7f3c2c0a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:47.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.667+0000 7f3c34178700 1 -- 172.21.15.16:0/4149012230 shutdown_connections 2023-09-30T22:36:47.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.667+0000 7f3c34178700 1 -- 172.21.15.16:0/4149012230 wait complete. 2023-09-30T22:36:47.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.668+0000 7f3c34178700 1 Processor -- start 2023-09-30T22:36:47.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.668+0000 7f3c34178700 1 -- start start 2023-09-30T22:36:47.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.669+0000 7f3c34178700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c2c09de50 0x7f3c2c119760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:47.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.669+0000 7f3c34178700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3c2c09e760 0x7f3c2c119ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:47.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.669+0000 7f3c34178700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3c2c1338a0 0x7f3c2c11a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:47.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.669+0000 7f3c34178700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2c11cec0 con 0x7f3c2c09de50 2023-09-30T22:36:47.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.669+0000 7f3c34178700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2c11da60 con 0x7f3c2c1338a0 2023-09-30T22:36:47.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.669+0000 7f3c34178700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2c11d0a0 con 0x7f3c2c09e760 2023-09-30T22:36:47.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.669+0000 7f3c31f14700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c2c09de50 0x7f3c2c119760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:47.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.669+0000 7f3c31f14700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c2c09de50 0x7f3c2c119760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45972/0 (socket says 172.21.15.16:45972) 2023-09-30T22:36:47.683 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.669+0000 7f3c31f14700 1 -- 172.21.15.16:0/1087063689 learned_addr learned my addr 172.21.15.16:0/1087063689 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:47.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.669+0000 7f3c32715700 1 --2- 172.21.15.16:0/1087063689 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3c2c1338a0 0x7f3c2c11a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:47.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.669+0000 7f3c31f14700 1 -- 172.21.15.16:0/1087063689 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3c2c09e760 msgr2=0x7f3c2c119ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:47.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.669+0000 7f3c31f14700 1 --2- 172.21.15.16:0/1087063689 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3c2c09e760 0x7f3c2c119ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:47.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.669+0000 7f3c31f14700 1 -- 172.21.15.16:0/1087063689 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3c2c1338a0 msgr2=0x7f3c2c11a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:47.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.669+0000 7f3c31f14700 1 --2- 172.21.15.16:0/1087063689 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3c2c1338a0 0x7f3c2c11a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:47.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.669+0000 7f3c31f14700 1 -- 172.21.15.16:0/1087063689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c24009b30 con 0x7f3c2c09de50 2023-09-30T22:36:47.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.670+0000 7f3c31f14700 1 --2- 172.21.15.16:0/1087063689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c2c09de50 0x7f3c2c119760 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f3c280143c0 tx=0x7f3c280146d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:47.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.670+0000 7f3c227fc700 1 -- 172.21.15.16:0/1087063689 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c28016050 con 0x7f3c2c09de50 2023-09-30T22:36:47.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.670+0000 7f3c227fc700 1 -- 172.21.15.16:0/1087063689 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c28017040 con 0x7f3c2c09de50 2023-09-30T22:36:47.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.670+0000 7f3c227fc700 1 -- 172.21.15.16:0/1087063689 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c280114d0 con 0x7f3c2c09de50 2023-09-30T22:36:47.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.670+0000 7f3c34178700 1 -- 172.21.15.16:0/1087063689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c2c11b230 con 0x7f3c2c09de50 2023-09-30T22:36:47.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.670+0000 7f3c34178700 1 -- 172.21.15.16:0/1087063689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c2c0a0c40 con 0x7f3c2c09de50 2023-09-30T22:36:47.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.671+0000 7f3c34178700 1 -- 172.21.15.16:0/1087063689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c2c074ea0 con 0x7f3c2c09de50 2023-09-30T22:36:47.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.671+0000 7f3c227fc700 1 -- 172.21.15.16:0/1087063689 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f3c2800f850 con 0x7f3c2c09de50 2023-09-30T22:36:47.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.672+0000 7f3c227fc700 1 -- 172.21.15.16:0/1087063689 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f3c280476b0 con 0x7f3c2c09de50 2023-09-30T22:36:47.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.676+0000 7f3c227fc700 1 -- 172.21.15.16:0/1087063689 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f3c28011630 con 0x7f3c2c09de50 2023-09-30T22:36:47.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.716+0000 7f891464d700 1 Processor -- start 2023-09-30T22:36:47.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.716+0000 7f891464d700 1 -- start start 2023-09-30T22:36:47.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.717+0000 7f891464d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f890c1337f0 0x7f890c138880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:47.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.717+0000 7f891464d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f890c138dc0 0x7f890c13b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:47.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.717+0000 7f891464d700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f890c06b990 con 0x7f890c133420 2023-09-30T22:36:47.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.717+0000 7f891464d700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f890c09b0e0 con 0x7f890c1337f0 2023-09-30T22:36:47.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.717+0000 7f891464d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f890c133c20 con 0x7f890c138dc0 2023-09-30T22:36:47.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.717+0000 7f8912bea700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f890c138dc0 0x7f890c13b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:47.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.717+0000 7f8911be8700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f890c1337f0 0x7f890c138880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:47.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.717+0000 7f8912bea700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f890c138dc0 0x7f890c13b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:50976/0 (socket says 172.21.15.16:50976) 2023-09-30T22:36:47.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.717+0000 7f8912bea700 1 -- 172.21.15.16:0/205323740 learned_addr learned my addr 172.21.15.16:0/205323740 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:47.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.717+0000 7f89123e9700 1 --1- 172.21.15.16:0/205323740 >> v1:172.21.15.16:6789/0 conn(0x7f890c133420 0x7f890c133e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48892/0 (socket says 172.21.15.16:48892) 2023-09-30T22:36:47.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.718+0000 7f8912bea700 1 -- 172.21.15.16:0/205323740 >> v2:172.21.15.16:3300/0 conn(0x7f890c1337f0 msgr2=0x7f890c138880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:47.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.718+0000 7f8912bea700 1 --2- 172.21.15.16:0/205323740 >> v2:172.21.15.16:3300/0 conn(0x7f890c1337f0 0x7f890c138880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:47.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.718+0000 7f8912bea700 1 -- 172.21.15.16:0/205323740 >> v1:172.21.15.16:6789/0 conn(0x7f890c133420 legacy=0x7f890c133e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:47.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.718+0000 7f8912bea700 1 -- 172.21.15.16:0/205323740 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f890c0a0590 con 0x7f890c138dc0 2023-09-30T22:36:47.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.718+0000 7f8911be8700 1 --2- 172.21.15.16:0/205323740 >> v2:172.21.15.16:3300/0 conn(0x7f890c1337f0 0x7f890c138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:47.722 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.718+0000 7f8912bea700 1 --2- 172.21.15.16:0/205323740 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f890c138dc0 0x7f890c13b1a0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f8904009e50 tx=0x7f890400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f1cf4784f7ac316 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:47.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.718+0000 7f89113e7700 1 -- 172.21.15.16:0/205323740 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f890400e070 con 0x7f890c138dc0 2023-09-30T22:36:47.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.718+0000 7f89113e7700 1 -- 172.21.15.16:0/205323740 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f890400b910 con 0x7f890c138dc0 2023-09-30T22:36:47.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.718+0000 7f89113e7700 1 -- 172.21.15.16:0/205323740 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f890400bcc0 con 0x7f890c138dc0 2023-09-30T22:36:47.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.719+0000 7f891464d700 1 -- 172.21.15.16:0/205323740 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f890c138dc0 msgr2=0x7f890c13b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:47.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.719+0000 7f891464d700 1 --2- 172.21.15.16:0/205323740 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f890c138dc0 0x7f890c13b1a0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f8904009e50 tx=0x7f890400b330 comp rx=0 tx=0).stop 2023-09-30T22:36:47.723 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.719+0000 7f891464d700 1 -- 172.21.15.16:0/205323740 shutdown_connections 2023-09-30T22:36:47.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.719+0000 7f891464d700 1 --2- 172.21.15.16:0/205323740 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f890c138dc0 0x7f890c13b1a0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:47.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.719+0000 7f891464d700 1 --2- 172.21.15.16:0/205323740 >> v2:172.21.15.16:3300/0 conn(0x7f890c1337f0 0x7f890c138880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:47.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.719+0000 7f891464d700 1 -- 172.21.15.16:0/205323740 >> 172.21.15.16:0/205323740 conn(0x7f890c0a4210 msgr2=0x7f890c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:47.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.720+0000 7f891464d700 1 -- 172.21.15.16:0/205323740 shutdown_connections 2023-09-30T22:36:47.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.720+0000 7f891464d700 1 -- 172.21.15.16:0/205323740 wait complete. 2023-09-30T22:36:47.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.721+0000 7f891464d700 1 Processor -- start 2023-09-30T22:36:47.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.721+0000 7f891464d700 1 -- start start 2023-09-30T22:36:47.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.722+0000 7f891464d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f890c133420 0x7f890c09c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:47.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.722+0000 7f891464d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f890c1337f0 0x7f890c09c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:47.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.722+0000 7f891464d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f890c138dc0 0x7f890c09cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:47.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.722+0000 7f891464d700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f890c12eee0 con 0x7f890c1337f0 2023-09-30T22:36:47.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.722+0000 7f891464d700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f890c09dfb0 con 0x7f890c133420 2023-09-30T22:36:47.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.722+0000 7f891464d700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f890c130a00 con 0x7f890c138dc0 2023-09-30T22:36:47.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.722+0000 7f89123e9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f890c133420 0x7f890c09c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:47.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.722+0000 7f89123e9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f890c133420 0x7f890c09c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37630/0 (socket says 172.21.15.16:37630) 2023-09-30T22:36:47.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.722+0000 7f89123e9700 1 -- 172.21.15.16:0/152682483 learned_addr learned my addr 172.21.15.16:0/152682483 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:47.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.722+0000 7f8911be8700 1 --2- 172.21.15.16:0/152682483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f890c1337f0 0x7f890c09c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:47.729 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.722+0000 7f8912bea700 1 --2- 172.21.15.16:0/152682483 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f890c138dc0 0x7f890c09cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:47.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.722+0000 7f89123e9700 1 -- 172.21.15.16:0/152682483 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f890c138dc0 msgr2=0x7f890c09cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:47.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.722+0000 7f89123e9700 1 --2- 172.21.15.16:0/152682483 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f890c138dc0 0x7f890c09cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:47.730 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.722+0000 7f89123e9700 1 -- 172.21.15.16:0/152682483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f890c1337f0 msgr2=0x7f890c09c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:47.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.722+0000 7f89123e9700 1 --2- 172.21.15.16:0/152682483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f890c1337f0 0x7f890c09c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:47.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.722+0000 7f89123e9700 1 -- 172.21.15.16:0/152682483 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88fc0094a0 con 0x7f890c133420 2023-09-30T22:36:47.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.723+0000 7f89123e9700 1 --2- 172.21.15.16:0/152682483 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f890c133420 0x7f890c09c480 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f890800c520 tx=0x7f890800c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:47.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.723+0000 7f8902ffd700 1 -- 172.21.15.16:0/152682483 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8908005cb0 con 0x7f890c133420 2023-09-30T22:36:47.731 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.723+0000 7f8902ffd700 1 -- 172.21.15.16:0/152682483 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8908005e10 con 0x7f890c133420 2023-09-30T22:36:47.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.723+0000 7f8902ffd700 1 -- 172.21.15.16:0/152682483 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89080072d0 con 0x7f890c133420 2023-09-30T22:36:47.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.723+0000 7f891464d700 1 -- 172.21.15.16:0/152682483 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8904009b30 con 0x7f890c133420 2023-09-30T22:36:47.732 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.723+0000 7f891464d700 1 -- 172.21.15.16:0/152682483 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f890c12f440 con 0x7f890c133420 2023-09-30T22:36:47.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.724+0000 7f8902ffd700 1 -- 172.21.15.16:0/152682483 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8908007430 con 0x7f890c133420 2023-09-30T22:36:47.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.725+0000 7f891464d700 1 -- 172.21.15.16:0/152682483 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f890c074ea0 con 0x7f890c133420 2023-09-30T22:36:47.733 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.725+0000 7f8902ffd700 1 -- 172.21.15.16:0/152682483 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f890803ea30 con 0x7f890c133420 2023-09-30T22:36:47.734 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.734+0000 7f8902ffd700 1 -- 172.21.15.16:0/152682483 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f890801b3b0 con 0x7f890c133420 2023-09-30T22:36:47.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.761+0000 7f76b855f700 1 Processor -- start 2023-09-30T22:36:47.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.761+0000 7f76b855f700 1 -- start start 2023-09-30T22:36:47.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.762+0000 7f76b855f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f76b006e570 0x7f76b006e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:47.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.762+0000 7f76b855f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f76b009eca0 0x7f76b006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:47.766 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.762+0000 7f76b855f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f76b0134610 0x7f76b0134a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:47.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.762+0000 7f76b855f700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76b0070ee0 con 0x7f76b009eca0 2023-09-30T22:36:47.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.762+0000 7f76b855f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76b0071480 con 0x7f76b006e570 2023-09-30T22:36:47.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.762+0000 7f76b855f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f76b006f3c0 con 0x7f76b0134610 2023-09-30T22:36:47.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.762+0000 7f76b5afa700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f76b009eca0 0x7f76b006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:47.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.762+0000 7f76b5afa700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f76b009eca0 0x7f76b006ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:45998/0 (socket says 172.21.15.16:45998) 2023-09-30T22:36:47.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.762+0000 7f76b5afa700 1 -- 172.21.15.16:0/542134934 learned_addr learned my addr 172.21.15.16:0/542134934 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:47.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.762+0000 7f76b6afc700 1 --2- 172.21.15.16:0/542134934 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f76b0134610 0x7f76b0134a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:47.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.762+0000 7f76b62fb700 1 --2- 172.21.15.16:0/542134934 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f76b006e570 0x7f76b006e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:47.767 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.762+0000 7f76b5afa700 1 -- 172.21.15.16:0/542134934 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f76b0134610 msgr2=0x7f76b0134a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:47.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.762+0000 7f76b5afa700 1 --2- 172.21.15.16:0/542134934 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f76b0134610 0x7f76b0134a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:47.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.762+0000 7f76b5afa700 1 -- 172.21.15.16:0/542134934 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f76b006e570 msgr2=0x7f76b006e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:47.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.762+0000 7f76b5afa700 1 --2- 172.21.15.16:0/542134934 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f76b006e570 0x7f76b006e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:47.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.762+0000 7f76b5afa700 1 -- 172.21.15.16:0/542134934 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76b0071b90 con 0x7f76b009eca0 2023-09-30T22:36:47.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.762+0000 7f76b62fb700 1 --2- 172.21.15.16:0/542134934 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f76b006e570 0x7f76b006e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:36:47.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.763+0000 7f76b5afa700 1 --2- 172.21.15.16:0/542134934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76b009eca0 0x7f76b006ee80 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f76a00098f0 tx=0x7f76a0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9b61ebee064727d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:47.768 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.763+0000 7f76b52f9700 1 -- 172.21.15.16:0/542134934 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a0011830 con 0x7f76b009eca0 2023-09-30T22:36:47.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.763+0000 7f76b52f9700 1 -- 172.21.15.16:0/542134934 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76a0011990 con 0x7f76b009eca0 2023-09-30T22:36:47.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.763+0000 7f76b52f9700 1 -- 172.21.15.16:0/542134934 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a0011d20 con 0x7f76b009eca0 2023-09-30T22:36:47.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.763+0000 7f76b855f700 1 -- 172.21.15.16:0/542134934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76b009eca0 msgr2=0x7f76b006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:47.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.763+0000 7f76b855f700 1 --2- 172.21.15.16:0/542134934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76b009eca0 0x7f76b006ee80 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f76a00098f0 tx=0x7f76a0009c00 comp rx=0 tx=0).stop 2023-09-30T22:36:47.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.764+0000 7f76b855f700 1 -- 172.21.15.16:0/542134934 shutdown_connections 2023-09-30T22:36:47.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.764+0000 7f76b855f700 1 --2- 172.21.15.16:0/542134934 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f76b006e570 0x7f76b006e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:47.769 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.764+0000 7f76b855f700 1 --2- 172.21.15.16:0/542134934 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f76b0134610 0x7f76b0134a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:47.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.764+0000 7f76b855f700 1 --2- 172.21.15.16:0/542134934 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76b009eca0 0x7f76b006ee80 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:47.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.764+0000 7f76b855f700 1 -- 172.21.15.16:0/542134934 >> 172.21.15.16:0/542134934 conn(0x7f76b00a2aa0 msgr2=0x7f76b00a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:47.770 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.764+0000 7f76b855f700 1 -- 172.21.15.16:0/542134934 shutdown_connections 2023-09-30T22:36:47.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.764+0000 7f76b855f700 1 -- 172.21.15.16:0/542134934 wait complete. 2023-09-30T22:36:47.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.765+0000 7f76b855f700 1 Processor -- start 2023-09-30T22:36:47.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.765+0000 7f76b855f700 1 -- start start 2023-09-30T22:36:47.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.765+0000 7f76b855f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f76b006e570 0x7f76b01214e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:47.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.765+0000 7f76b855f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76b0134610 0x7f76b0121a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:47.771 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.765+0000 7f76b855f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f76b01340c0 0x7f76b0121f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:47.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.765+0000 7f76b855f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b0124350 con 0x7f76b0134610 2023-09-30T22:36:47.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.765+0000 7f76b855f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76b00a2070 con 0x7f76b01340c0 2023-09-30T22:36:47.772 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.765+0000 7f76b855f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f76b0123770 con 0x7f76b006e570 2023-09-30T22:36:47.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.766+0000 7f76b5afa700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76b0134610 0x7f76b0121a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:47.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.766+0000 7f76b5afa700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76b0134610 0x7f76b0121a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46006/0 (socket says 172.21.15.16:46006) 2023-09-30T22:36:47.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.766+0000 7f76b5afa700 1 -- 172.21.15.16:0/1407426179 learned_addr learned my addr 172.21.15.16:0/1407426179 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:47.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.766+0000 7f76b62fb700 1 --2- 172.21.15.16:0/1407426179 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f76b006e570 0x7f76b01214e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:47.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.766+0000 7f76b6afc700 1 --2- 172.21.15.16:0/1407426179 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f76b01340c0 0x7f76b0121f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:47.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.766+0000 7f76b5afa700 1 -- 172.21.15.16:0/1407426179 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f76b006e570 msgr2=0x7f76b01214e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:47.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.766+0000 7f76b5afa700 1 --2- 172.21.15.16:0/1407426179 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f76b006e570 0x7f76b01214e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:47.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.766+0000 7f76b5afa700 1 -- 172.21.15.16:0/1407426179 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f76b01340c0 msgr2=0x7f76b0121f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:47.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.766+0000 7f76b5afa700 1 --2- 172.21.15.16:0/1407426179 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f76b01340c0 0x7f76b0121f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:47.776 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.766+0000 7f76b5afa700 1 -- 172.21.15.16:0/1407426179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a0009550 con 0x7f76b0134610 2023-09-30T22:36:47.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.766+0000 7f76b5afa700 1 --2- 172.21.15.16:0/1407426179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76b0134610 0x7f76b0121a20 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f76a0009c40 tx=0x7f76a0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:47.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.766+0000 7f76a6ffd700 1 -- 172.21.15.16:0/1407426179 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a00044d0 con 0x7f76b0134610 2023-09-30T22:36:47.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.766+0000 7f76a6ffd700 1 -- 172.21.15.16:0/1407426179 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76a001f070 con 0x7f76b0134610 2023-09-30T22:36:47.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.766+0000 7f76b855f700 1 -- 172.21.15.16:0/1407426179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76b0122af0 con 0x7f76b0134610 2023-09-30T22:36:47.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.766+0000 7f76a6ffd700 1 -- 172.21.15.16:0/1407426179 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a00038c0 con 0x7f76b0134610 2023-09-30T22:36:47.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.766+0000 7f76b855f700 1 -- 172.21.15.16:0/1407426179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76b0123170 con 0x7f76b0134610 2023-09-30T22:36:47.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.767+0000 7f76a4ff9700 1 -- 172.21.15.16:0/1407426179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f768c0052e0 con 0x7f76b0134610 2023-09-30T22:36:47.777 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.768+0000 7f76a6ffd700 1 -- 172.21.15.16:0/1407426179 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f76a0003a20 con 0x7f76b0134610 2023-09-30T22:36:47.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.770+0000 7f76a6ffd700 1 -- 172.21.15.16:0/1407426179 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f76a0021070 con 0x7f76b0134610 2023-09-30T22:36:47.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.778+0000 7f76a6ffd700 1 -- 172.21.15.16:0/1407426179 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f76a0026480 con 0x7f76b0134610 2023-09-30T22:36:47.810 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.809+0000 7ff073299700 1 Processor -- start 2023-09-30T22:36:47.811 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.810+0000 7ff073299700 1 -- start start 2023-09-30T22:36:47.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.810+0000 7ff073299700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff06c0a0a30 0x7ff06c0a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:47.813 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.810+0000 7ff073299700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff06c134170 0x7ff06c13bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:47.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.810+0000 7ff073299700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff06c09edd0 con 0x7ff06c0a1340 2023-09-30T22:36:47.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.810+0000 7ff073299700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff06c09e1f0 con 0x7ff06c134170 2023-09-30T22:36:47.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.810+0000 7ff073299700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff06c09dc50 con 0x7ff06c0a0a30 2023-09-30T22:36:47.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.810+0000 7ff071836700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff06c134170 0x7ff06c13bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:47.814 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.811+0000 7ff071836700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff06c134170 0x7ff06c13bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46010/0 (socket says 172.21.15.16:46010) 2023-09-30T22:36:47.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.811+0000 7ff071836700 1 -- 172.21.15.16:0/3966705122 learned_addr learned my addr 172.21.15.16:0/3966705122 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:47.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.811+0000 7ff070834700 1 --1- 172.21.15.16:0/3966705122 >> v1:172.21.15.16:6789/0 conn(0x7ff06c0a1340 0x7ff06c133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48902/0 (socket says 172.21.15.16:48902) 2023-09-30T22:36:47.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.811+0000 7ff071035700 1 --2- 172.21.15.16:0/3966705122 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff06c0a0a30 0x7ff06c0a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:47.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.811+0000 7ff071836700 1 -- 172.21.15.16:0/3966705122 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff06c0a0a30 msgr2=0x7ff06c0a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:47.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.811+0000 7ff071836700 1 --2- 172.21.15.16:0/3966705122 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff06c0a0a30 0x7ff06c0a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:47.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.811+0000 7ff071836700 1 -- 172.21.15.16:0/3966705122 >> v1:172.21.15.16:6789/0 conn(0x7ff06c0a1340 legacy=0x7ff06c133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:47.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.811+0000 7ff071836700 1 -- 172.21.15.16:0/3966705122 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff06c0a1fe0 con 0x7ff06c134170 2023-09-30T22:36:47.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.811+0000 7ff071836700 1 --2- 172.21.15.16:0/3966705122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff06c134170 0x7ff06c13bd30 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7ff064009e50 tx=0x7ff06400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=393d0e5d4a00bd8a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:47.816 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.811+0000 7ff063fff700 1 -- 172.21.15.16:0/3966705122 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06400e070 con 0x7ff06c134170 2023-09-30T22:36:47.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.811+0000 7ff063fff700 1 -- 172.21.15.16:0/3966705122 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff06400b9c0 con 0x7ff06c134170 2023-09-30T22:36:47.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.811+0000 7ff063fff700 1 -- 172.21.15.16:0/3966705122 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06400bd70 con 0x7ff06c134170 2023-09-30T22:36:47.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.812+0000 7ff073299700 1 -- 172.21.15.16:0/3966705122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff06c134170 msgr2=0x7ff06c13bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:47.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.812+0000 7ff073299700 1 --2- 172.21.15.16:0/3966705122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff06c134170 0x7ff06c13bd30 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7ff064009e50 tx=0x7ff06400b330 comp rx=0 tx=0).stop 2023-09-30T22:36:47.817 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.812+0000 7ff073299700 1 -- 172.21.15.16:0/3966705122 shutdown_connections 2023-09-30T22:36:47.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.812+0000 7ff073299700 1 --2- 172.21.15.16:0/3966705122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff06c134170 0x7ff06c13bd30 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:47.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.812+0000 7ff073299700 1 --2- 172.21.15.16:0/3966705122 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff06c0a0a30 0x7ff06c0a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:47.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.812+0000 7ff073299700 1 -- 172.21.15.16:0/3966705122 >> 172.21.15.16:0/3966705122 conn(0x7ff06c0a6550 msgr2=0x7ff06c0a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:47.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.813+0000 7ff073299700 1 -- 172.21.15.16:0/3966705122 shutdown_connections 2023-09-30T22:36:47.818 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.813+0000 7ff073299700 1 -- 172.21.15.16:0/3966705122 wait complete. 2023-09-30T22:36:47.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.813+0000 7ff073299700 1 Processor -- start 2023-09-30T22:36:47.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.814+0000 7ff073299700 1 -- start start 2023-09-30T22:36:47.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.814+0000 7ff073299700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff06c0a0a30 0x7ff06c1228d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:47.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.814+0000 7ff073299700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff06c0a1340 0x7ff06c122e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:47.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.814+0000 7ff073299700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff06c134170 0x7ff06c123350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:47.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.814+0000 7ff073299700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff06c1245d0 con 0x7ff06c0a0a30 2023-09-30T22:36:47.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.814+0000 7ff073299700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff06c126360 con 0x7ff06c0a1340 2023-09-30T22:36:47.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.814+0000 7ff073299700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff06c124c90 con 0x7ff06c134170 2023-09-30T22:36:47.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.814+0000 7ff071836700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff06c134170 0x7ff06c123350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:47.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.814+0000 7ff071836700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff06c134170 0x7ff06c123350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:51030/0 (socket says 172.21.15.16:51030) 2023-09-30T22:36:47.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.814+0000 7ff071836700 1 -- 172.21.15.16:0/1490857708 learned_addr learned my addr 172.21.15.16:0/1490857708 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:47.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.814+0000 7ff070834700 1 --2- 172.21.15.16:0/1490857708 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff06c0a1340 0x7ff06c122e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:47.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.814+0000 7ff071035700 1 --2- 172.21.15.16:0/1490857708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff06c0a0a30 0x7ff06c1228d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:47.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.814+0000 7ff071836700 1 -- 172.21.15.16:0/1490857708 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff06c0a1340 msgr2=0x7ff06c122e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:47.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.814+0000 7ff071836700 1 --2- 172.21.15.16:0/1490857708 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff06c0a1340 0x7ff06c122e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:47.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.815+0000 7ff071836700 1 -- 172.21.15.16:0/1490857708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff06c0a0a30 msgr2=0x7ff06c1228d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:47.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.815+0000 7ff071836700 1 --2- 172.21.15.16:0/1490857708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff06c0a0a30 0x7ff06c1228d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:47.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.815+0000 7ff071836700 1 -- 172.21.15.16:0/1490857708 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff064009b30 con 0x7ff06c134170 2023-09-30T22:36:47.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.815+0000 7ff071035700 1 --2- 172.21.15.16:0/1490857708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff06c0a0a30 0x7ff06c1228d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:36:47.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.815+0000 7ff071836700 1 --2- 172.21.15.16:0/1490857708 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff06c134170 0x7ff06c123350 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7ff06c09e330 tx=0x7ff064004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:47.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.815+0000 7ff061ffb700 1 -- 172.21.15.16:0/1490857708 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0640048b0 con 0x7ff06c134170 2023-09-30T22:36:47.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.815+0000 7ff061ffb700 1 -- 172.21.15.16:0/1490857708 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0640092e0 con 0x7ff06c134170 2023-09-30T22:36:47.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.815+0000 7ff073299700 1 -- 172.21.15.16:0/1490857708 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff06c124a00 con 0x7ff06c134170 2023-09-30T22:36:47.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.815+0000 7ff061ffb700 1 -- 172.21.15.16:0/1490857708 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06401c660 con 0x7ff06c134170 2023-09-30T22:36:47.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.815+0000 7ff073299700 1 -- 172.21.15.16:0/1490857708 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff06c126080 con 0x7ff06c134170 2023-09-30T22:36:47.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.816+0000 7ff061ffb700 1 -- 172.21.15.16:0/1490857708 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff06401c7c0 con 0x7ff06c134170 2023-09-30T22:36:47.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.816+0000 7ff073299700 1 -- 172.21.15.16:0/1490857708 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff06c0a1d40 con 0x7ff06c134170 2023-09-30T22:36:47.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.817+0000 7ff061ffb700 1 -- 172.21.15.16:0/1490857708 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7ff0640412b0 con 0x7ff06c134170 2023-09-30T22:36:47.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.822+0000 7ff061ffb700 1 -- 172.21.15.16:0/1490857708 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff06402d940 con 0x7ff06c134170 2023-09-30T22:36:47.899 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.899+0000 7f3c34178700 1 -- 172.21.15.16:0/1087063689 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f3c2c06f640 con 0x7f3c2c09de50 2023-09-30T22:36:47.900 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.899+0000 7f3c227fc700 1 -- 172.21.15.16:0/1087063689 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3c28022b00 con 0x7f3c2c09de50 2023-09-30T22:36:47.901 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:36:47.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.902+0000 7f3c34178700 1 -- 172.21.15.16:0/1087063689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c2c09de50 msgr2=0x7f3c2c119760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:47.902 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.902+0000 7f3c34178700 1 --2- 172.21.15.16:0/1087063689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c2c09de50 0x7f3c2c119760 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f3c280143c0 tx=0x7f3c280146d0 comp rx=0 tx=0).stop 2023-09-30T22:36:47.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.902+0000 7f3c34178700 1 -- 172.21.15.16:0/1087063689 shutdown_connections 2023-09-30T22:36:47.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.902+0000 7f3c34178700 1 --2- 172.21.15.16:0/1087063689 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f3c2c1338a0 0x7f3c2c11a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:47.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.902+0000 7f3c34178700 1 --2- 172.21.15.16:0/1087063689 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3c2c09de50 0x7f3c2c119760 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:47.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.902+0000 7f3c34178700 1 --2- 172.21.15.16:0/1087063689 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f3c2c09e760 0x7f3c2c119ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:47.903 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.902+0000 7f3c34178700 1 -- 172.21.15.16:0/1087063689 >> 172.21.15.16:0/1087063689 conn(0x7f3c2c0a2aa0 msgr2=0x7f3c2c09ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:47.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.902+0000 7f3c34178700 1 -- 172.21.15.16:0/1087063689 shutdown_connections 2023-09-30T22:36:47.904 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.902+0000 7f3c34178700 1 -- 172.21.15.16:0/1087063689 wait complete. 2023-09-30T22:36:47.917 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:36:47.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.975+0000 7f891464d700 1 -- 172.21.15.16:0/152682483 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f890c06b250 con 0x7f890c133420 2023-09-30T22:36:47.976 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.975+0000 7f8902ffd700 1 -- 172.21.15.16:0/152682483 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8908017030 con 0x7f890c133420 2023-09-30T22:36:47.976 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:36:47.978 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.977+0000 7f891464d700 1 -- 172.21.15.16:0/152682483 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f890c133420 msgr2=0x7f890c09c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:47.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.977+0000 7f891464d700 1 --2- 172.21.15.16:0/152682483 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f890c133420 0x7f890c09c480 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f890800c520 tx=0x7f890800c830 comp rx=0 tx=0).stop 2023-09-30T22:36:47.979 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.978+0000 7f891464d700 1 -- 172.21.15.16:0/152682483 shutdown_connections 2023-09-30T22:36:47.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.978+0000 7f891464d700 1 --2- 172.21.15.16:0/152682483 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f890c133420 0x7f890c09c480 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:47.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.978+0000 7f891464d700 1 --2- 172.21.15.16:0/152682483 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f890c138dc0 0x7f890c09cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:47.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.978+0000 7f891464d700 1 --2- 172.21.15.16:0/152682483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f890c1337f0 0x7f890c09c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:47.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.978+0000 7f891464d700 1 -- 172.21.15.16:0/152682483 >> 172.21.15.16:0/152682483 conn(0x7f890c0a4210 msgr2=0x7f890c0a0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:47.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.978+0000 7f891464d700 1 -- 172.21.15.16:0/152682483 shutdown_connections 2023-09-30T22:36:47.980 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:47.978+0000 7f891464d700 1 -- 172.21.15.16:0/152682483 wait complete. 2023-09-30T22:36:47.992 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:36:48.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:48.004+0000 7f76a4ff9700 1 -- 172.21.15.16:0/1407426179 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f768c005860 con 0x7f76b0134610 2023-09-30T22:36:48.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:48.004+0000 7f76a6ffd700 1 -- 172.21.15.16:0/1407426179 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f76a002b020 con 0x7f76b0134610 2023-09-30T22:36:48.006 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:36:48.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:48.007+0000 7f76a4ff9700 1 -- 172.21.15.16:0/1407426179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76b0134610 msgr2=0x7f76b0121a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:48.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:48.007+0000 7f76a4ff9700 1 --2- 172.21.15.16:0/1407426179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76b0134610 0x7f76b0121a20 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f76a0009c40 tx=0x7f76a0004750 comp rx=0 tx=0).stop 2023-09-30T22:36:48.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:48.007+0000 7f76a4ff9700 1 -- 172.21.15.16:0/1407426179 shutdown_connections 2023-09-30T22:36:48.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:48.007+0000 7f76a4ff9700 1 --2- 172.21.15.16:0/1407426179 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f76b01340c0 0x7f76b0121f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:48.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:48.007+0000 7f76a4ff9700 1 --2- 172.21.15.16:0/1407426179 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f76b006e570 0x7f76b01214e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:48.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:48.007+0000 7f76a4ff9700 1 --2- 172.21.15.16:0/1407426179 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76b0134610 0x7f76b0121a20 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:48.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:48.007+0000 7f76a4ff9700 1 -- 172.21.15.16:0/1407426179 >> 172.21.15.16:0/1407426179 conn(0x7f76b00a2aa0 msgr2=0x7f76b01bf210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:48.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:48.007+0000 7f76a4ff9700 1 -- 172.21.15.16:0/1407426179 shutdown_connections 2023-09-30T22:36:48.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:48.007+0000 7f76a4ff9700 1 -- 172.21.15.16:0/1407426179 wait complete. 2023-09-30T22:36:48.021 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:36:48.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:48.030+0000 7ff073299700 1 -- 172.21.15.16:0/1490857708 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7ff06c074ea0 con 0x7ff06c134170 2023-09-30T22:36:48.030 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:48.030+0000 7ff061ffb700 1 -- 172.21.15.16:0/1490857708 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff06402d350 con 0x7ff06c134170 2023-09-30T22:36:48.031 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:36:48.033 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:48.033+0000 7ff073299700 1 -- 172.21.15.16:0/1490857708 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff06c134170 msgr2=0x7ff06c123350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:48.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:48.033+0000 7ff073299700 1 --2- 172.21.15.16:0/1490857708 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff06c134170 0x7ff06c123350 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7ff06c09e330 tx=0x7ff064004ad0 comp rx=0 tx=0).stop 2023-09-30T22:36:48.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:48.033+0000 7ff073299700 1 -- 172.21.15.16:0/1490857708 shutdown_connections 2023-09-30T22:36:48.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:48.033+0000 7ff073299700 1 --2- 172.21.15.16:0/1490857708 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff06c134170 0x7ff06c123350 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:48.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:48.033+0000 7ff073299700 1 --2- 172.21.15.16:0/1490857708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff06c0a0a30 0x7ff06c1228d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:48.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:48.033+0000 7ff073299700 1 --2- 172.21.15.16:0/1490857708 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff06c0a1340 0x7ff06c122e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:48.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:48.033+0000 7ff073299700 1 -- 172.21.15.16:0/1490857708 >> 172.21.15.16:0/1490857708 conn(0x7ff06c0a6550 msgr2=0x7ff06c09ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:48.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:48.033+0000 7ff073299700 1 -- 172.21.15.16:0/1490857708 shutdown_connections 2023-09-30T22:36:48.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:48.033+0000 7ff073299700 1 -- 172.21.15.16:0/1490857708 wait complete. 2023-09-30T22:36:48.047 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:36:48.918 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:36:48.993 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:36:49.022 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:36:49.049 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:36:49.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.067+0000 7fb6643d5700 1 Processor -- start 2023-09-30T22:36:49.069 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.067+0000 7fb6643d5700 1 -- start start 2023-09-30T22:36:49.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.068+0000 7fb6643d5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb65c09d6f0 0x7fb65c09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:49.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.068+0000 7fb6643d5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb65c09e410 0x7fb65c134100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:49.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.068+0000 7fb6643d5700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb65c09b120 con 0x7fb65c09e000 2023-09-30T22:36:49.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.068+0000 7fb6643d5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb65c06b460 con 0x7fb65c09d6f0 2023-09-30T22:36:49.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.068+0000 7fb6643d5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb65c09a7e0 con 0x7fb65c09e410 2023-09-30T22:36:49.070 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.068+0000 7fb662171700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb65c09d6f0 0x7fb65c09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:49.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.068+0000 7fb662171700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb65c09d6f0 0x7fb65c09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37672/0 (socket says 172.21.15.16:37672) 2023-09-30T22:36:49.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.068+0000 7fb662972700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb65c09e410 0x7fb65c134100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:49.071 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.068+0000 7fb662171700 1 -- 172.21.15.16:0/1605062642 learned_addr learned my addr 172.21.15.16:0/1605062642 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:49.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.068+0000 7fb662171700 1 -- 172.21.15.16:0/1605062642 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb65c09e410 msgr2=0x7fb65c134100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:49.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.068+0000 7fb662171700 1 --2- 172.21.15.16:0/1605062642 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb65c09e410 0x7fb65c134100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:49.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.068+0000 7fb662171700 1 -- 172.21.15.16:0/1605062642 >> v1:172.21.15.16:6789/0 conn(0x7fb65c09e000 legacy=0x7fb65c1339f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:49.077 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.068+0000 7fb662171700 1 -- 172.21.15.16:0/1605062642 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb65c09a500 con 0x7fb65c09d6f0 2023-09-30T22:36:49.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.068+0000 7fb662972700 1 --2- 172.21.15.16:0/1605062642 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb65c09e410 0x7fb65c134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:49.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.069+0000 7fb662171700 1 --2- 172.21.15.16:0/1605062642 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb65c09d6f0 0x7fb65c09dac0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fb658009920 tx=0x7fb658009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cf5e87df50e7040 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:49.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.069+0000 7fb66116f700 1 -- 172.21.15.16:0/1605062642 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb658011830 con 0x7fb65c09d6f0 2023-09-30T22:36:49.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.069+0000 7fb66116f700 1 -- 172.21.15.16:0/1605062642 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb658011990 con 0x7fb65c09d6f0 2023-09-30T22:36:49.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.069+0000 7fb66116f700 1 -- 172.21.15.16:0/1605062642 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb658011d20 con 0x7fb65c09d6f0 2023-09-30T22:36:49.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.070+0000 7fb6643d5700 1 -- 172.21.15.16:0/1605062642 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb65c09d6f0 msgr2=0x7fb65c09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:49.078 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.070+0000 7fb6643d5700 1 --2- 172.21.15.16:0/1605062642 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb65c09d6f0 0x7fb65c09dac0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fb658009920 tx=0x7fb658009c30 comp rx=0 tx=0).stop 2023-09-30T22:36:49.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.070+0000 7fb6643d5700 1 -- 172.21.15.16:0/1605062642 shutdown_connections 2023-09-30T22:36:49.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.070+0000 7fb6643d5700 1 --2- 172.21.15.16:0/1605062642 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb65c09e410 0x7fb65c134100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:49.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.070+0000 7fb6643d5700 1 --2- 172.21.15.16:0/1605062642 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb65c09d6f0 0x7fb65c09dac0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:49.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.070+0000 7fb6643d5700 1 -- 172.21.15.16:0/1605062642 >> 172.21.15.16:0/1605062642 conn(0x7fb65c066f50 msgr2=0x7fb65c09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:49.079 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.070+0000 7fb6643d5700 1 -- 172.21.15.16:0/1605062642 shutdown_connections 2023-09-30T22:36:49.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.070+0000 7fb6643d5700 1 -- 172.21.15.16:0/1605062642 wait complete. 2023-09-30T22:36:49.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.071+0000 7fb6643d5700 1 Processor -- start 2023-09-30T22:36:49.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.071+0000 7fb6643d5700 1 -- start start 2023-09-30T22:36:49.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.072+0000 7fb6643d5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb65c09d6f0 0x7fb65c1a6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:49.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.072+0000 7fb6643d5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb65c09e000 0x7fb65c1a6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:49.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.072+0000 7fb6643d5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb65c09e410 0x7fb65c1a6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:49.080 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.072+0000 7fb6643d5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65c1a74e0 con 0x7fb65c09d6f0 2023-09-30T22:36:49.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.072+0000 7fb6643d5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb65c1a7b60 con 0x7fb65c09e000 2023-09-30T22:36:49.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.072+0000 7fb6643d5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb65c1a8db0 con 0x7fb65c09e410 2023-09-30T22:36:49.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.072+0000 7fb662171700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb65c09d6f0 0x7fb65c1a6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:49.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.072+0000 7fb661970700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb65c09e000 0x7fb65c1a6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:49.081 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.072+0000 7fb662171700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb65c09d6f0 0x7fb65c1a6030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46020/0 (socket says 172.21.15.16:46020) 2023-09-30T22:36:49.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.072+0000 7fb661970700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb65c09e000 0x7fb65c1a6570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37684/0 (socket says 172.21.15.16:37684) 2023-09-30T22:36:49.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.072+0000 7fb662171700 1 -- 172.21.15.16:0/2843580171 learned_addr learned my addr 172.21.15.16:0/2843580171 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:49.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.072+0000 7fb662972700 1 --2- 172.21.15.16:0/2843580171 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb65c09e410 0x7fb65c1a6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:49.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.073+0000 7fb662171700 1 -- 172.21.15.16:0/2843580171 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb65c09e410 msgr2=0x7fb65c1a6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:49.082 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.073+0000 7fb662171700 1 --2- 172.21.15.16:0/2843580171 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb65c09e410 0x7fb65c1a6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:49.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.073+0000 7fb662171700 1 -- 172.21.15.16:0/2843580171 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb65c09e000 msgr2=0x7fb65c1a6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:49.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.073+0000 7fb662171700 1 --2- 172.21.15.16:0/2843580171 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb65c09e000 0x7fb65c1a6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:49.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.073+0000 7fb662171700 1 -- 172.21.15.16:0/2843580171 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb654009a70 con 0x7fb65c09d6f0 2023-09-30T22:36:49.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.073+0000 7fb661970700 1 --2- 172.21.15.16:0/2843580171 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb65c09e000 0x7fb65c1a6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:49.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.073+0000 7fb662171700 1 --2- 172.21.15.16:0/2843580171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb65c09d6f0 0x7fb65c1a6030 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fb6580098f0 tx=0x7fb658004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:49.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.073+0000 7fb652ffd700 1 -- 172.21.15.16:0/2843580171 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb658004240 con 0x7fb65c09d6f0 2023-09-30T22:36:49.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.073+0000 7fb652ffd700 1 -- 172.21.15.16:0/2843580171 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb65801f070 con 0x7fb65c09d6f0 2023-09-30T22:36:49.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.073+0000 7fb6643d5700 1 -- 172.21.15.16:0/2843580171 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb658009550 con 0x7fb65c09d6f0 2023-09-30T22:36:49.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.073+0000 7fb652ffd700 1 -- 172.21.15.16:0/2843580171 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6580045c0 con 0x7fb65c09d6f0 2023-09-30T22:36:49.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.073+0000 7fb6643d5700 1 -- 172.21.15.16:0/2843580171 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb65c0a22d0 con 0x7fb65c09d6f0 2023-09-30T22:36:49.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.074+0000 7fb652ffd700 1 -- 172.21.15.16:0/2843580171 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb658021030 con 0x7fb65c09d6f0 2023-09-30T22:36:49.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.074+0000 7fb652ffd700 1 -- 172.21.15.16:0/2843580171 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fb658040800 con 0x7fb65c09d6f0 2023-09-30T22:36:49.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.074+0000 7fb6643d5700 1 -- 172.21.15.16:0/2843580171 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb65c1a9680 con 0x7fb65c09d6f0 2023-09-30T22:36:49.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.080+0000 7fb652ffd700 1 -- 172.21.15.16:0/2843580171 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb65801a450 con 0x7fb65c09d6f0 2023-09-30T22:36:49.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.148+0000 7fce75eff700 1 Processor -- start 2023-09-30T22:36:49.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.148+0000 7fce75eff700 1 -- start start 2023-09-30T22:36:49.149 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.149+0000 7fce75eff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fce7009d630 0x7fce700a1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:49.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.149+0000 7fce75eff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fce70138d80 0x7fce7013b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:49.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.149+0000 7fce75eff700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce700a2a70 con 0x7fce700a0a30 2023-09-30T22:36:49.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.149+0000 7fce75eff700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce700a01d0 con 0x7fce70138d80 2023-09-30T22:36:49.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.149+0000 7fce75eff700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fce7009e2b0 con 0x7fce7009d630 2023-09-30T22:36:49.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.149+0000 7fce6f7fe700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fce700a0a30 0x7fce700a0e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48920/0 (socket says 172.21.15.16:48920) 2023-09-30T22:36:49.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.149+0000 7fce6f7fe700 1 -- 172.21.15.16:0/1053199819 learned_addr learned my addr 172.21.15.16:0/1053199819 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:36:49.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.149+0000 7fce6ffff700 1 --2- 172.21.15.16:0/1053199819 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fce70138d80 0x7fce7013b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:49.150 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.149+0000 7fce6effd700 1 --2- 172.21.15.16:0/1053199819 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fce7009d630 0x7fce700a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:49.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.150+0000 7fce6ffff700 1 -- 172.21.15.16:0/1053199819 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fce7009d630 msgr2=0x7fce700a1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:49.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.150+0000 7fce6ffff700 1 --2- 172.21.15.16:0/1053199819 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fce7009d630 0x7fce700a1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:49.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.150+0000 7fce6ffff700 1 -- 172.21.15.16:0/1053199819 >> v1:172.21.15.16:6789/0 conn(0x7fce700a0a30 legacy=0x7fce700a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:49.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.150+0000 7fce6ffff700 1 -- 172.21.15.16:0/1053199819 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce7009dc50 con 0x7fce70138d80 2023-09-30T22:36:49.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.150+0000 7fce6ffff700 1 --2- 172.21.15.16:0/1053199819 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fce70138d80 0x7fce7013b160 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fce68009dc0 tx=0x7fce6800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab558762cb0346e8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:49.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.150+0000 7fce6e7fc700 1 -- 172.21.15.16:0/1053199819 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6800e070 con 0x7fce70138d80 2023-09-30T22:36:49.152 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.150+0000 7fce6e7fc700 1 -- 172.21.15.16:0/1053199819 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce6800bb20 con 0x7fce70138d80 2023-09-30T22:36:49.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.150+0000 7fce6e7fc700 1 -- 172.21.15.16:0/1053199819 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce68012430 con 0x7fce70138d80 2023-09-30T22:36:49.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.151+0000 7fce75eff700 1 -- 172.21.15.16:0/1053199819 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fce70138d80 msgr2=0x7fce7013b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:49.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.151+0000 7fce75eff700 1 --2- 172.21.15.16:0/1053199819 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fce70138d80 0x7fce7013b160 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fce68009dc0 tx=0x7fce6800b330 comp rx=0 tx=0).stop 2023-09-30T22:36:49.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.151+0000 7fce75eff700 1 -- 172.21.15.16:0/1053199819 shutdown_connections 2023-09-30T22:36:49.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.151+0000 7fce75eff700 1 --2- 172.21.15.16:0/1053199819 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fce70138d80 0x7fce7013b160 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:49.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.151+0000 7fce75eff700 1 --2- 172.21.15.16:0/1053199819 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fce7009d630 0x7fce700a1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:49.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.151+0000 7fce75eff700 1 -- 172.21.15.16:0/1053199819 >> 172.21.15.16:0/1053199819 conn(0x7fce700a3050 msgr2=0x7fce700a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:49.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.151+0000 7fce75eff700 1 -- 172.21.15.16:0/1053199819 shutdown_connections 2023-09-30T22:36:49.153 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.151+0000 7fce75eff700 1 -- 172.21.15.16:0/1053199819 wait complete. 2023-09-30T22:36:49.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.153+0000 7fce75eff700 1 Processor -- start 2023-09-30T22:36:49.154 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.153+0000 7fce75eff700 1 -- start start 2023-09-30T22:36:49.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.154+0000 7fce75eff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fce7009d630 0x7fce701382c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:49.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.154+0000 7fce75eff700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce700a0a30 0x7fce7006dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:49.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.154+0000 7fce75eff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fce70138d80 0x7fce7006e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:49.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.154+0000 7fce75eff700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce70070c70 con 0x7fce700a0a30 2023-09-30T22:36:49.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.154+0000 7fce75eff700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce70133940 con 0x7fce70138d80 2023-09-30T22:36:49.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.154+0000 7fce75eff700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fce7013af10 con 0x7fce7009d630 2023-09-30T22:36:49.155 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.154+0000 7fce6f7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fce7009d630 0x7fce701382c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:49.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.154+0000 7fce6f7fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fce7009d630 0x7fce701382c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:51054/0 (socket says 172.21.15.16:51054) 2023-09-30T22:36:49.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.154+0000 7fce6f7fe700 1 -- 172.21.15.16:0/80774906 learned_addr learned my addr 172.21.15.16:0/80774906 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:49.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.154+0000 7fce6ffff700 1 --2- 172.21.15.16:0/80774906 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fce70138d80 0x7fce7006e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:49.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.154+0000 7fce6effd700 1 --2- 172.21.15.16:0/80774906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce700a0a30 0x7fce7006dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:49.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.154+0000 7fce6ffff700 1 -- 172.21.15.16:0/80774906 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fce7009d630 msgr2=0x7fce701382c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:49.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.154+0000 7fce6ffff700 1 --2- 172.21.15.16:0/80774906 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fce7009d630 0x7fce701382c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:49.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.154+0000 7fce6ffff700 1 -- 172.21.15.16:0/80774906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce700a0a30 msgr2=0x7fce7006dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:49.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.154+0000 7fce6ffff700 1 --2- 172.21.15.16:0/80774906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce700a0a30 0x7fce7006dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:49.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.154+0000 7fce6ffff700 1 -- 172.21.15.16:0/80774906 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce68009a70 con 0x7fce70138d80 2023-09-30T22:36:49.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.155+0000 7fce6ffff700 1 --2- 172.21.15.16:0/80774906 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fce70138d80 0x7fce7006e000 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fce68016000 tx=0x7fce68004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:49.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.155+0000 7fce57fff700 1 -- 172.21.15.16:0/80774906 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce68012430 con 0x7fce70138d80 2023-09-30T22:36:49.156 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.155+0000 7fce57fff700 1 -- 172.21.15.16:0/80774906 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce68020070 con 0x7fce70138d80 2023-09-30T22:36:49.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.155+0000 7fce75eff700 1 -- 172.21.15.16:0/80774906 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce700a0410 con 0x7fce70138d80 2023-09-30T22:36:49.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.155+0000 7fce57fff700 1 -- 172.21.15.16:0/80774906 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce68009430 con 0x7fce70138d80 2023-09-30T22:36:49.157 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.155+0000 7fce75eff700 1 -- 172.21.15.16:0/80774906 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce70138800 con 0x7fce70138d80 2023-09-30T22:36:49.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.156+0000 7fce57fff700 1 -- 172.21.15.16:0/80774906 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fce68004040 con 0x7fce70138d80 2023-09-30T22:36:49.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.156+0000 7fce75eff700 1 -- 172.21.15.16:0/80774906 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce701389f0 con 0x7fce70138d80 2023-09-30T22:36:49.158 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.156+0000 7fce57fff700 1 -- 172.21.15.16:0/80774906 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fce68022070 con 0x7fce70138d80 2023-09-30T22:36:49.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.166+0000 7fce57fff700 1 -- 172.21.15.16:0/80774906 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fce680042f0 con 0x7fce70138d80 2023-09-30T22:36:49.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.184+0000 7f02c8fbc700 1 Processor -- start 2023-09-30T22:36:49.185 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.184+0000 7f02c8fbc700 1 -- start start 2023-09-30T22:36:49.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.185+0000 7f02c8fbc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f02c412cf60 0x7f02c406f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:49.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.185+0000 7f02c8fbc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f02c4130430 0x7f02c4137c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:49.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.185+0000 7f02c8fbc700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02c412cd90 con 0x7f02c412f8b0 2023-09-30T22:36:49.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.185+0000 7f02c8fbc700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02c412d330 con 0x7f02c412cf60 2023-09-30T22:36:49.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.185+0000 7f02c8fbc700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f02c4071780 con 0x7f02c4130430 2023-09-30T22:36:49.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.185+0000 7f02c259c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f02c412cf60 0x7f02c406f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:49.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.186+0000 7f02c259c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f02c412cf60 0x7f02c406f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37710/0 (socket says 172.21.15.16:37710) 2023-09-30T22:36:49.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.186+0000 7f02c259c700 1 -- 172.21.15.16:0/3511871784 learned_addr learned my addr 172.21.15.16:0/3511871784 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:49.186 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.186+0000 7f02c2d9d700 1 --2- 172.21.15.16:0/3511871784 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f02c4130430 0x7f02c4137c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:49.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.186+0000 7f02c159a700 1 -- 172.21.15.16:0/3511871784 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1266389004 0 0) 0x7f02c412cd90 con 0x7f02c412f8b0 2023-09-30T22:36:49.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.186+0000 7f02c159a700 1 -- 172.21.15.16:0/3511871784 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02ac0035b0 con 0x7f02c412f8b0 2023-09-30T22:36:49.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.186+0000 7f02c259c700 1 -- 172.21.15.16:0/3511871784 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f02c4130430 msgr2=0x7f02c4137c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:49.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.186+0000 7f02c259c700 1 --2- 172.21.15.16:0/3511871784 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f02c4130430 0x7f02c4137c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:49.188 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.186+0000 7f02c259c700 1 -- 172.21.15.16:0/3511871784 >> v1:172.21.15.16:6789/0 conn(0x7f02c412f8b0 legacy=0x7f02c412fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:49.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.186+0000 7f02c259c700 1 -- 172.21.15.16:0/3511871784 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02c4071a40 con 0x7f02c412cf60 2023-09-30T22:36:49.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.186+0000 7f02c2d9d700 1 --2- 172.21.15.16:0/3511871784 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f02c4130430 0x7f02c4137c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:49.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.186+0000 7f02c259c700 1 --2- 172.21.15.16:0/3511871784 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f02c412cf60 0x7f02c406f5d0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f02b4009920 tx=0x7f02b4009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd98b6bc73aa17a8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:49.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.187+0000 7f02c159a700 1 -- 172.21.15.16:0/3511871784 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b4011830 con 0x7f02c412cf60 2023-09-30T22:36:49.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.187+0000 7f02c159a700 1 -- 172.21.15.16:0/3511871784 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02b4011990 con 0x7f02c412cf60 2023-09-30T22:36:49.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.187+0000 7f02c159a700 1 -- 172.21.15.16:0/3511871784 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b4011d20 con 0x7f02c412cf60 2023-09-30T22:36:49.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.187+0000 7f02c8fbc700 1 -- 172.21.15.16:0/3511871784 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f02c412cf60 msgr2=0x7f02c406f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:49.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.187+0000 7f02c8fbc700 1 --2- 172.21.15.16:0/3511871784 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f02c412cf60 0x7f02c406f5d0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f02b4009920 tx=0x7f02b4009c30 comp rx=0 tx=0).stop 2023-09-30T22:36:49.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.188+0000 7f02c8fbc700 1 -- 172.21.15.16:0/3511871784 shutdown_connections 2023-09-30T22:36:49.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.188+0000 7f02c8fbc700 1 --2- 172.21.15.16:0/3511871784 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f02c4130430 0x7f02c4137c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:49.189 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.188+0000 7f02c8fbc700 1 --2- 172.21.15.16:0/3511871784 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f02c412cf60 0x7f02c406f5d0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:49.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.188+0000 7f02c8fbc700 1 -- 172.21.15.16:0/3511871784 >> 172.21.15.16:0/3511871784 conn(0x7f02c40a2aa0 msgr2=0x7f02c40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:49.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.188+0000 7f02c8fbc700 1 -- 172.21.15.16:0/3511871784 shutdown_connections 2023-09-30T22:36:49.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.188+0000 7f02c8fbc700 1 -- 172.21.15.16:0/3511871784 wait complete. 2023-09-30T22:36:49.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.189+0000 7f02c8fbc700 1 Processor -- start 2023-09-30T22:36:49.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.189+0000 7f02c8fbc700 1 -- start start 2023-09-30T22:36:49.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.189+0000 7f02c8fbc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f02c412cf60 0x7f02c41c3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:49.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.189+0000 7f02c8fbc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f02c412f8b0 0x7f02c41c4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:49.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.189+0000 7f02c8fbc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02c4130430 0x7f02c41c49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:49.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.189+0000 7f02c8fbc700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c409ccf0 con 0x7f02c4130430 2023-09-30T22:36:49.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.189+0000 7f02c8fbc700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02c409e920 con 0x7f02c412cf60 2023-09-30T22:36:49.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.189+0000 7f02c8fbc700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f02c412f4f0 con 0x7f02c412f8b0 2023-09-30T22:36:49.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.190+0000 7f02c2d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02c4130430 0x7f02c41c49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:49.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.190+0000 7f02c2d9d700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02c4130430 0x7f02c41c49a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46036/0 (socket says 172.21.15.16:46036) 2023-09-30T22:36:49.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.190+0000 7f02c2d9d700 1 -- 172.21.15.16:0/1670861688 learned_addr learned my addr 172.21.15.16:0/1670861688 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:49.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.190+0000 7f02c1d9b700 1 --2- 172.21.15.16:0/1670861688 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f02c412f8b0 0x7f02c41c4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:49.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.190+0000 7f02c1d9b700 1 -- 172.21.15.16:0/1670861688 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f02c412cf60 msgr2=0x7f02c41c3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:49.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.190+0000 7f02c1d9b700 1 --2- 172.21.15.16:0/1670861688 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f02c412cf60 0x7f02c41c3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:49.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.190+0000 7f02c1d9b700 1 -- 172.21.15.16:0/1670861688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02c4130430 msgr2=0x7f02c41c49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:49.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.190+0000 7f02c1d9b700 1 --2- 172.21.15.16:0/1670861688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02c4130430 0x7f02c41c49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:49.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.190+0000 7f02c1d9b700 1 -- 172.21.15.16:0/1670861688 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02bc00bc80 con 0x7f02c412f8b0 2023-09-30T22:36:49.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.190+0000 7f02c2d9d700 1 --2- 172.21.15.16:0/1670861688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02c4130430 0x7f02c41c49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:49.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.190+0000 7f02c1d9b700 1 --2- 172.21.15.16:0/1670861688 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f02c412f8b0 0x7f02c41c4460 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f02b800c450 tx=0x7f02b800c760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:49.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.191+0000 7f02b2ffd700 1 -- 172.21.15.16:0/1670861688 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b80052d0 con 0x7f02c412f8b0 2023-09-30T22:36:49.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.191+0000 7f02b2ffd700 1 -- 172.21.15.16:0/1670861688 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02b8005430 con 0x7f02c412f8b0 2023-09-30T22:36:49.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.191+0000 7f02c8fbc700 1 -- 172.21.15.16:0/1670861688 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02b4009550 con 0x7f02c412f8b0 2023-09-30T22:36:49.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.191+0000 7f02b2ffd700 1 -- 172.21.15.16:0/1670861688 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b80038a0 con 0x7f02c412f8b0 2023-09-30T22:36:49.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.191+0000 7f02c8fbc700 1 -- 172.21.15.16:0/1670861688 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02c409d4e0 con 0x7f02c412f8b0 2023-09-30T22:36:49.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.192+0000 7f02c8fbc700 1 -- 172.21.15.16:0/1670861688 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02c4074ea0 con 0x7f02c412f8b0 2023-09-30T22:36:49.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.192+0000 7f02b2ffd700 1 -- 172.21.15.16:0/1670861688 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f02b8003a00 con 0x7f02c412f8b0 2023-09-30T22:36:49.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.193+0000 7f02b2ffd700 1 -- 172.21.15.16:0/1670861688 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f02b803eaa0 con 0x7f02c412f8b0 2023-09-30T22:36:49.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.198+0000 7f02b2ffd700 1 -- 172.21.15.16:0/1670861688 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f02b800f260 con 0x7f02c412f8b0 2023-09-30T22:36:49.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.221+0000 7fa660b9f700 1 Processor -- start 2023-09-30T22:36:49.221 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.221+0000 7fa660b9f700 1 -- start start 2023-09-30T22:36:49.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.222+0000 7fa660b9f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa65c09d8d0 0x7fa65c09dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:49.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.222+0000 7fa660b9f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa65c09e1e0 0x7fa65c130460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:49.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.222+0000 7fa660b9f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa65c09e610 0x7fa65c1309a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:49.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.222+0000 7fa660b9f700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa65c09b6e0 con 0x7fa65c09e610 2023-09-30T22:36:49.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.222+0000 7fa660b9f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa65c09b470 con 0x7fa65c09d8d0 2023-09-30T22:36:49.222 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.222+0000 7fa660b9f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa65c09eb80 con 0x7fa65c09e1e0 2023-09-30T22:36:49.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.222+0000 7fa65affd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa65c09e1e0 0x7fa65c130460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:49.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.222+0000 7fa65bfff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fa65c09e610 0x7fa65c1309a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:49.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.222+0000 7fa65affd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa65c09e1e0 0x7fa65c130460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:51086/0 (socket says 172.21.15.16:51086) 2023-09-30T22:36:49.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.222+0000 7fa65affd700 1 -- 172.21.15.16:0/2277075457 learned_addr learned my addr 172.21.15.16:0/2277075457 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:49.223 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.222+0000 7fa65b7fe700 1 --2- 172.21.15.16:0/2277075457 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa65c09d8d0 0x7fa65c09dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:49.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.222+0000 7fa65b7fe700 1 -- 172.21.15.16:0/2277075457 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa65c09e1e0 msgr2=0x7fa65c130460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:49.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.222+0000 7fa65b7fe700 1 --2- 172.21.15.16:0/2277075457 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa65c09e1e0 0x7fa65c130460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:49.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.222+0000 7fa65b7fe700 1 -- 172.21.15.16:0/2277075457 >> v2:172.21.15.16:3300/0 conn(0x7fa65c09e610 msgr2=0x7fa65c1309a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:49.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.222+0000 7fa65b7fe700 1 --2- 172.21.15.16:0/2277075457 >> v2:172.21.15.16:3300/0 conn(0x7fa65c09e610 0x7fa65c1309a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:49.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.222+0000 7fa65b7fe700 1 -- 172.21.15.16:0/2277075457 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa65c09ae00 con 0x7fa65c09d8d0 2023-09-30T22:36:49.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.222+0000 7fa65affd700 1 --2- 172.21.15.16:0/2277075457 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa65c09e1e0 0x7fa65c130460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:49.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.223+0000 7fa65b7fe700 1 --2- 172.21.15.16:0/2277075457 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa65c09d8d0 0x7fa65c09dca0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fa64c009840 tx=0x7fa64c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c055bd2ef89ad0b7 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:49.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.223+0000 7fa65a7fc700 1 -- 172.21.15.16:0/2277075457 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64c011830 con 0x7fa65c09d8d0 2023-09-30T22:36:49.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.223+0000 7fa65a7fc700 1 -- 172.21.15.16:0/2277075457 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa64c011990 con 0x7fa65c09d8d0 2023-09-30T22:36:49.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.223+0000 7fa65a7fc700 1 -- 172.21.15.16:0/2277075457 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64c011d20 con 0x7fa65c09d8d0 2023-09-30T22:36:49.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.224+0000 7fa660b9f700 1 -- 172.21.15.16:0/2277075457 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa65c09d8d0 msgr2=0x7fa65c09dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:49.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.224+0000 7fa660b9f700 1 --2- 172.21.15.16:0/2277075457 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa65c09d8d0 0x7fa65c09dca0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fa64c009840 tx=0x7fa64c009b50 comp rx=0 tx=0).stop 2023-09-30T22:36:49.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.224+0000 7fa660b9f700 1 -- 172.21.15.16:0/2277075457 shutdown_connections 2023-09-30T22:36:49.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.224+0000 7fa660b9f700 1 --2- 172.21.15.16:0/2277075457 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa65c09d8d0 0x7fa65c09dca0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:49.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.224+0000 7fa660b9f700 1 --2- 172.21.15.16:0/2277075457 >> v2:172.21.15.16:3300/0 conn(0x7fa65c09e610 0x7fa65c1309a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:49.225 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.224+0000 7fa660b9f700 1 --2- 172.21.15.16:0/2277075457 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa65c09e1e0 0x7fa65c130460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:49.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.224+0000 7fa660b9f700 1 -- 172.21.15.16:0/2277075457 >> 172.21.15.16:0/2277075457 conn(0x7fa65c0a2e20 msgr2=0x7fa65c0a3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:49.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.224+0000 7fa660b9f700 1 -- 172.21.15.16:0/2277075457 shutdown_connections 2023-09-30T22:36:49.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.224+0000 7fa660b9f700 1 -- 172.21.15.16:0/2277075457 wait complete. 2023-09-30T22:36:49.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.226+0000 7fa660b9f700 1 Processor -- start 2023-09-30T22:36:49.226 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.226+0000 7fa660b9f700 1 -- start start 2023-09-30T22:36:49.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.227+0000 7fa660b9f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa65c09d8d0 0x7fa65c11df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:49.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.227+0000 7fa660b9f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa65c09e1e0 0x7fa65c11e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:49.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.227+0000 7fa660b9f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa65c09e610 0x7fa65c11e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:49.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.227+0000 7fa660b9f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa65c0a1de0 con 0x7fa65c09d8d0 2023-09-30T22:36:49.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.227+0000 7fa660b9f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa65c1202f0 con 0x7fa65c09e1e0 2023-09-30T22:36:49.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.227+0000 7fa660b9f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fa65c09c600 con 0x7fa65c09e610 2023-09-30T22:36:49.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.227+0000 7fa65b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa65c09d8d0 0x7fa65c11df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:49.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.227+0000 7fa65b7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa65c09d8d0 0x7fa65c11df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46062/0 (socket says 172.21.15.16:46062) 2023-09-30T22:36:49.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.227+0000 7fa65affd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa65c09e1e0 0x7fa65c11e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:49.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.227+0000 7fa65b7fe700 1 -- 172.21.15.16:0/79948646 learned_addr learned my addr 172.21.15.16:0/79948646 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:49.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.227+0000 7fa65bfff700 1 --2- 172.21.15.16:0/79948646 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa65c09e610 0x7fa65c11e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:49.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.227+0000 7fa65b7fe700 1 -- 172.21.15.16:0/79948646 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa65c09e610 msgr2=0x7fa65c11e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:49.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.227+0000 7fa65b7fe700 1 --2- 172.21.15.16:0/79948646 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa65c09e610 0x7fa65c11e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:49.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.227+0000 7fa65b7fe700 1 -- 172.21.15.16:0/79948646 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa65c09e1e0 msgr2=0x7fa65c11e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:49.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.227+0000 7fa65b7fe700 1 --2- 172.21.15.16:0/79948646 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa65c09e1e0 0x7fa65c11e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:49.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.227+0000 7fa65b7fe700 1 -- 172.21.15.16:0/79948646 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa65000b5b0 con 0x7fa65c09d8d0 2023-09-30T22:36:49.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.227+0000 7fa65affd700 1 --2- 172.21.15.16:0/79948646 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa65c09e1e0 0x7fa65c11e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:49.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.228+0000 7fa65b7fe700 1 --2- 172.21.15.16:0/79948646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa65c09d8d0 0x7fa65c11df20 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fa64c000c00 tx=0x7fa64c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:49.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.228+0000 7fa643fff700 1 -- 172.21.15.16:0/79948646 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64c004450 con 0x7fa65c09d8d0 2023-09-30T22:36:49.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.228+0000 7fa643fff700 1 -- 172.21.15.16:0/79948646 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa64c01f070 con 0x7fa65c09d8d0 2023-09-30T22:36:49.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.228+0000 7fa660b9f700 1 -- 172.21.15.16:0/79948646 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa64c0094a0 con 0x7fa65c09d8d0 2023-09-30T22:36:49.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.228+0000 7fa643fff700 1 -- 172.21.15.16:0/79948646 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64c0038e0 con 0x7fa65c09d8d0 2023-09-30T22:36:49.230 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.228+0000 7fa660b9f700 1 -- 172.21.15.16:0/79948646 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa65c120910 con 0x7fa65c09d8d0 2023-09-30T22:36:49.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.229+0000 7fa660b9f700 1 -- 172.21.15.16:0/79948646 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa65c121ab0 con 0x7fa65c09d8d0 2023-09-30T22:36:49.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.229+0000 7fa643fff700 1 -- 172.21.15.16:0/79948646 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fa64c003a40 con 0x7fa65c09d8d0 2023-09-30T22:36:49.231 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.230+0000 7fa643fff700 1 -- 172.21.15.16:0/79948646 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fa64c021070 con 0x7fa65c09d8d0 2023-09-30T22:36:49.239 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.239+0000 7fa643fff700 1 -- 172.21.15.16:0/79948646 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fa64c010380 con 0x7fa65c09d8d0 2023-09-30T22:36:49.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.308+0000 7fb6643d5700 1 -- 172.21.15.16:0/2843580171 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb65c09c710 con 0x7fb65c09d6f0 2023-09-30T22:36:49.309 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.309+0000 7fb652ffd700 1 -- 172.21.15.16:0/2843580171 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb658015020 con 0x7fb65c09d6f0 2023-09-30T22:36:49.310 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:36:49.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.311+0000 7fb6643d5700 1 -- 172.21.15.16:0/2843580171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb65c09d6f0 msgr2=0x7fb65c1a6030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:49.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.311+0000 7fb6643d5700 1 --2- 172.21.15.16:0/2843580171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb65c09d6f0 0x7fb65c1a6030 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fb6580098f0 tx=0x7fb658004750 comp rx=0 tx=0).stop 2023-09-30T22:36:49.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.311+0000 7fb6643d5700 1 -- 172.21.15.16:0/2843580171 shutdown_connections 2023-09-30T22:36:49.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.311+0000 7fb6643d5700 1 --2- 172.21.15.16:0/2843580171 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb65c09e410 0x7fb65c1a6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:49.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.311+0000 7fb6643d5700 1 --2- 172.21.15.16:0/2843580171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb65c09d6f0 0x7fb65c1a6030 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:49.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.311+0000 7fb6643d5700 1 --2- 172.21.15.16:0/2843580171 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb65c09e000 0x7fb65c1a6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:49.312 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.311+0000 7fb6643d5700 1 -- 172.21.15.16:0/2843580171 >> 172.21.15.16:0/2843580171 conn(0x7fb65c066f50 msgr2=0x7fb65c1bafa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:49.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.311+0000 7fb6643d5700 1 -- 172.21.15.16:0/2843580171 shutdown_connections 2023-09-30T22:36:49.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.311+0000 7fb6643d5700 1 -- 172.21.15.16:0/2843580171 wait complete. 2023-09-30T22:36:49.326 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:36:49.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.381+0000 7fce75eff700 1 -- 172.21.15.16:0/80774906 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fce700747f0 con 0x7fce70138d80 2023-09-30T22:36:49.382 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.381+0000 7fce57fff700 1 -- 172.21.15.16:0/80774906 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fce68011390 con 0x7fce70138d80 2023-09-30T22:36:49.382 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:36:49.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.384+0000 7fce75eff700 1 -- 172.21.15.16:0/80774906 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fce70138d80 msgr2=0x7fce7006e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:49.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.384+0000 7fce75eff700 1 --2- 172.21.15.16:0/80774906 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fce70138d80 0x7fce7006e000 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fce68016000 tx=0x7fce68004ad0 comp rx=0 tx=0).stop 2023-09-30T22:36:49.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.384+0000 7fce75eff700 1 -- 172.21.15.16:0/80774906 shutdown_connections 2023-09-30T22:36:49.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.384+0000 7fce75eff700 1 --2- 172.21.15.16:0/80774906 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fce70138d80 0x7fce7006e000 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:49.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.384+0000 7fce75eff700 1 --2- 172.21.15.16:0/80774906 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fce7009d630 0x7fce701382c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:49.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.384+0000 7fce75eff700 1 --2- 172.21.15.16:0/80774906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fce700a0a30 0x7fce7006dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:49.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.384+0000 7fce75eff700 1 -- 172.21.15.16:0/80774906 >> 172.21.15.16:0/80774906 conn(0x7fce700a3050 msgr2=0x7fce7009ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:49.385 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.384+0000 7fce75eff700 1 -- 172.21.15.16:0/80774906 shutdown_connections 2023-09-30T22:36:49.386 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.384+0000 7fce75eff700 1 -- 172.21.15.16:0/80774906 wait complete. 2023-09-30T22:36:49.400 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:36:49.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.409+0000 7f02c8fbc700 1 -- 172.21.15.16:0/1670861688 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f02c40713a0 con 0x7f02c412f8b0 2023-09-30T22:36:49.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.410+0000 7f02b2ffd700 1 -- 172.21.15.16:0/1670861688 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f02b8017070 con 0x7f02c412f8b0 2023-09-30T22:36:49.411 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:36:49.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.412+0000 7f02c8fbc700 1 -- 172.21.15.16:0/1670861688 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f02c412f8b0 msgr2=0x7f02c41c4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:49.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.412+0000 7f02c8fbc700 1 --2- 172.21.15.16:0/1670861688 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f02c412f8b0 0x7f02c41c4460 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f02b800c450 tx=0x7f02b800c760 comp rx=0 tx=0).stop 2023-09-30T22:36:49.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.412+0000 7f02c8fbc700 1 -- 172.21.15.16:0/1670861688 shutdown_connections 2023-09-30T22:36:49.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.412+0000 7f02c8fbc700 1 --2- 172.21.15.16:0/1670861688 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f02c412cf60 0x7f02c41c3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:49.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.412+0000 7f02c8fbc700 1 --2- 172.21.15.16:0/1670861688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f02c4130430 0x7f02c41c49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:49.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.412+0000 7f02c8fbc700 1 --2- 172.21.15.16:0/1670861688 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f02c412f8b0 0x7f02c41c4460 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:49.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.412+0000 7f02c8fbc700 1 -- 172.21.15.16:0/1670861688 >> 172.21.15.16:0/1670861688 conn(0x7f02c40a2aa0 msgr2=0x7f02c406e510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:49.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.412+0000 7f02c8fbc700 1 -- 172.21.15.16:0/1670861688 shutdown_connections 2023-09-30T22:36:49.413 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.412+0000 7f02c8fbc700 1 -- 172.21.15.16:0/1670861688 wait complete. 2023-09-30T22:36:49.426 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:36:49.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.445+0000 7fa660b9f700 1 -- 172.21.15.16:0/79948646 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa65c04f2a0 con 0x7fa65c09d8d0 2023-09-30T22:36:49.446 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.445+0000 7fa643fff700 1 -- 172.21.15.16:0/79948646 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa64c034020 con 0x7fa65c09d8d0 2023-09-30T22:36:49.446 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:36:49.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.448+0000 7fa660b9f700 1 -- 172.21.15.16:0/79948646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa65c09d8d0 msgr2=0x7fa65c11df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:49.448 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.448+0000 7fa660b9f700 1 --2- 172.21.15.16:0/79948646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa65c09d8d0 0x7fa65c11df20 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fa64c000c00 tx=0x7fa64c004750 comp rx=0 tx=0).stop 2023-09-30T22:36:49.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.448+0000 7fa660b9f700 1 -- 172.21.15.16:0/79948646 shutdown_connections 2023-09-30T22:36:49.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.448+0000 7fa660b9f700 1 --2- 172.21.15.16:0/79948646 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fa65c09e610 0x7fa65c11e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:49.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.448+0000 7fa660b9f700 1 --2- 172.21.15.16:0/79948646 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa65c09d8d0 0x7fa65c11df20 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:49.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.448+0000 7fa660b9f700 1 --2- 172.21.15.16:0/79948646 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fa65c09e1e0 0x7fa65c11e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:49.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.448+0000 7fa660b9f700 1 -- 172.21.15.16:0/79948646 >> 172.21.15.16:0/79948646 conn(0x7fa65c0a2e20 msgr2=0x7fa65c070a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:49.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.448+0000 7fa660b9f700 1 -- 172.21.15.16:0/79948646 shutdown_connections 2023-09-30T22:36:49.449 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:49.448+0000 7fa660b9f700 1 -- 172.21.15.16:0/79948646 wait complete. 2023-09-30T22:36:49.462 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:36:50.327 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:36:50.401 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:36:50.427 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:36:50.463 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:36:50.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.478+0000 7fd177fb5700 1 Processor -- start 2023-09-30T22:36:50.479 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.479+0000 7fd177fb5700 1 -- start start 2023-09-30T22:36:50.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.479+0000 7fd177fb5700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd170131810 0x7fd170131be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:50.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.479+0000 7fd177fb5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd17009a580 0x7fd170139660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:50.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.479+0000 7fd177fb5700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1701313c0 con 0x7fd170099a40 2023-09-30T22:36:50.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.479+0000 7fd177fb5700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd17006e8c0 con 0x7fd170131810 2023-09-30T22:36:50.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.479+0000 7fd177fb5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd170071d20 con 0x7fd17009a580 2023-09-30T22:36:50.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.480+0000 7fd175d51700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd170131810 0x7fd170131be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:50.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.480+0000 7fd175d51700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd170131810 0x7fd170131be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46072/0 (socket says 172.21.15.16:46072) 2023-09-30T22:36:50.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.480+0000 7fd175550700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fd170099a40 0x7fd170099e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48944/0 (socket says 172.21.15.16:48944) 2023-09-30T22:36:50.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.480+0000 7fd175d51700 1 -- 172.21.15.16:0/276787366 learned_addr learned my addr 172.21.15.16:0/276787366 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:50.482 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.480+0000 7fd174d4f700 1 -- 172.21.15.16:0/276787366 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 88623007 0 0) 0x7fd1701313c0 con 0x7fd170099a40 2023-09-30T22:36:50.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.480+0000 7fd174d4f700 1 -- 172.21.15.16:0/276787366 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd15c0035b0 con 0x7fd170099a40 2023-09-30T22:36:50.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.480+0000 7fd175d51700 1 -- 172.21.15.16:0/276787366 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd17009a580 msgr2=0x7fd170139660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:50.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.480+0000 7fd175d51700 1 --2- 172.21.15.16:0/276787366 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd17009a580 0x7fd170139660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:50.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.480+0000 7fd175d51700 1 -- 172.21.15.16:0/276787366 >> v1:172.21.15.16:6789/0 conn(0x7fd170099a40 legacy=0x7fd170099e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:50.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.480+0000 7fd175d51700 1 -- 172.21.15.16:0/276787366 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1700721a0 con 0x7fd170131810 2023-09-30T22:36:50.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.480+0000 7fd175d51700 1 --2- 172.21.15.16:0/276787366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd170131810 0x7fd170131be0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fd16c0098f0 tx=0x7fd16c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9a7c94286800059 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:50.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.480+0000 7fd174d4f700 1 -- 172.21.15.16:0/276787366 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16c011830 con 0x7fd170131810 2023-09-30T22:36:50.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.481+0000 7fd174d4f700 1 -- 172.21.15.16:0/276787366 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd16c011990 con 0x7fd170131810 2023-09-30T22:36:50.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.481+0000 7fd174d4f700 1 -- 172.21.15.16:0/276787366 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd16c011d20 con 0x7fd170131810 2023-09-30T22:36:50.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.481+0000 7fd177fb5700 1 -- 172.21.15.16:0/276787366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd170131810 msgr2=0x7fd170131be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:50.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.481+0000 7fd177fb5700 1 --2- 172.21.15.16:0/276787366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd170131810 0x7fd170131be0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fd16c0098f0 tx=0x7fd16c009c00 comp rx=0 tx=0).stop 2023-09-30T22:36:50.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.481+0000 7fd177fb5700 1 -- 172.21.15.16:0/276787366 shutdown_connections 2023-09-30T22:36:50.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.481+0000 7fd177fb5700 1 --2- 172.21.15.16:0/276787366 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd17009a580 0x7fd170139660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:50.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.481+0000 7fd177fb5700 1 --2- 172.21.15.16:0/276787366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd170131810 0x7fd170131be0 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:50.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.481+0000 7fd177fb5700 1 -- 172.21.15.16:0/276787366 >> 172.21.15.16:0/276787366 conn(0x7fd1700a2aa0 msgr2=0x7fd1700a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:50.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.482+0000 7fd177fb5700 1 -- 172.21.15.16:0/276787366 shutdown_connections 2023-09-30T22:36:50.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.482+0000 7fd177fb5700 1 -- 172.21.15.16:0/276787366 wait complete. 2023-09-30T22:36:50.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.483+0000 7fd177fb5700 1 Processor -- start 2023-09-30T22:36:50.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.483+0000 7fd177fb5700 1 -- start start 2023-09-30T22:36:50.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.484+0000 7fd177fb5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd170099a40 0x7fd17011b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:50.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.484+0000 7fd177fb5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd17009a580 0x7fd17011bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:50.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.484+0000 7fd177fb5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd170131810 0x7fd17011c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:50.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.484+0000 7fd177fb5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17011ee30 con 0x7fd170099a40 2023-09-30T22:36:50.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.484+0000 7fd177fb5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd17006eb00 con 0x7fd17009a580 2023-09-30T22:36:50.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.484+0000 7fd177fb5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd17011d310 con 0x7fd170131810 2023-09-30T22:36:50.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.484+0000 7fd175550700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd17009a580 0x7fd17011bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:50.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.484+0000 7fd175550700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd17009a580 0x7fd17011bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37752/0 (socket says 172.21.15.16:37752) 2023-09-30T22:36:50.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.484+0000 7fd175550700 1 -- 172.21.15.16:0/3434102565 learned_addr learned my addr 172.21.15.16:0/3434102565 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:50.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.484+0000 7fd176552700 1 --2- 172.21.15.16:0/3434102565 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd170131810 0x7fd17011c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:50.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.485+0000 7fd175550700 1 -- 172.21.15.16:0/3434102565 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd170131810 msgr2=0x7fd17011c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:50.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.485+0000 7fd175550700 1 --2- 172.21.15.16:0/3434102565 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd170131810 0x7fd17011c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:50.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.485+0000 7fd175550700 1 -- 172.21.15.16:0/3434102565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd170099a40 msgr2=0x7fd17011b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:50.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.485+0000 7fd175550700 1 --2- 172.21.15.16:0/3434102565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd170099a40 0x7fd17011b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:50.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.485+0000 7fd175550700 1 -- 172.21.15.16:0/3434102565 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd16c009550 con 0x7fd17009a580 2023-09-30T22:36:50.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.485+0000 7fd175550700 1 --2- 172.21.15.16:0/3434102565 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd17009a580 0x7fd17011bb80 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fd16000c520 tx=0x7fd16000c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:50.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.485+0000 7fd1667fc700 1 -- 172.21.15.16:0/3434102565 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd160005cb0 con 0x7fd17009a580 2023-09-30T22:36:50.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.485+0000 7fd1667fc700 1 -- 172.21.15.16:0/3434102565 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd160005e10 con 0x7fd17009a580 2023-09-30T22:36:50.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.485+0000 7fd1667fc700 1 -- 172.21.15.16:0/3434102565 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1600072f0 con 0x7fd17009a580 2023-09-30T22:36:50.489 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.485+0000 7fd177fb5700 1 -- 172.21.15.16:0/3434102565 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1700a1af0 con 0x7fd17009a580 2023-09-30T22:36:50.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.485+0000 7fd177fb5700 1 -- 172.21.15.16:0/3434102565 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd17011f110 con 0x7fd17009a580 2023-09-30T22:36:50.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.486+0000 7fd1667fc700 1 -- 172.21.15.16:0/3434102565 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd160007450 con 0x7fd17009a580 2023-09-30T22:36:50.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.487+0000 7fd177fb5700 1 -- 172.21.15.16:0/3434102565 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd170074ea0 con 0x7fd17009a580 2023-09-30T22:36:50.491 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.487+0000 7fd1667fc700 1 -- 172.21.15.16:0/3434102565 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd16003ecc0 con 0x7fd17009a580 2023-09-30T22:36:50.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.492+0000 7fd1667fc700 1 -- 172.21.15.16:0/3434102565 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd160019b40 con 0x7fd17009a580 2023-09-30T22:36:50.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.577+0000 7f976dd05700 1 Processor -- start 2023-09-30T22:36:50.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.577+0000 7f976dd05700 1 -- start start 2023-09-30T22:36:50.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.577+0000 7f976dd05700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f976809e930 0x7f976809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:50.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.577+0000 7f976dd05700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f976809f240 0x7f97681317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:50.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.577+0000 7f976dd05700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9768070940 con 0x7f976809f650 2023-09-30T22:36:50.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.577+0000 7f976dd05700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f976806fd60 con 0x7f976809f240 2023-09-30T22:36:50.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.577+0000 7f976dd05700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f976806fa80 con 0x7f976809e930 2023-09-30T22:36:50.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.578+0000 7f9767fff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f976809f650 0x7f9768131d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48954/0 (socket says 172.21.15.16:48954) 2023-09-30T22:36:50.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.578+0000 7f9767fff700 1 -- 172.21.15.16:0/3611768962 learned_addr learned my addr 172.21.15.16:0/3611768962 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:36:50.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.578+0000 7f97677fe700 1 --2- 172.21.15.16:0/3611768962 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f976809e930 0x7f976809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:50.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.578+0000 7f9766ffd700 1 --2- 172.21.15.16:0/3611768962 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f976809f240 0x7f97681317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:50.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.578+0000 7f97667fc700 1 -- 172.21.15.16:0/3611768962 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3433407678 0 0) 0x7f9768070940 con 0x7f976809f650 2023-09-30T22:36:50.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.578+0000 7f97667fc700 1 -- 172.21.15.16:0/3611768962 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97500035b0 con 0x7f976809f650 2023-09-30T22:36:50.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.578+0000 7f97677fe700 1 -- 172.21.15.16:0/3611768962 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f976809f240 msgr2=0x7f97681317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:50.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.578+0000 7f97677fe700 1 --2- 172.21.15.16:0/3611768962 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f976809f240 0x7f97681317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:50.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.578+0000 7f97677fe700 1 -- 172.21.15.16:0/3611768962 >> v1:172.21.15.16:6789/0 conn(0x7f976809f650 legacy=0x7f9768131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:50.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.578+0000 7f97677fe700 1 -- 172.21.15.16:0/3611768962 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f976809fb60 con 0x7f976809e930 2023-09-30T22:36:50.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.578+0000 7f9766ffd700 1 --2- 172.21.15.16:0/3611768962 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f976809f240 0x7f97681317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:50.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.578+0000 7f97677fe700 1 --2- 172.21.15.16:0/3611768962 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f976809e930 0x7f976809ed00 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f9758009870 tx=0x7f9758009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5980f0aada6e74d6 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:50.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.579+0000 7f97667fc700 1 -- 172.21.15.16:0/3611768962 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9758011830 con 0x7f976809e930 2023-09-30T22:36:50.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.579+0000 7f97667fc700 1 -- 172.21.15.16:0/3611768962 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9758011990 con 0x7f976809e930 2023-09-30T22:36:50.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.579+0000 7f97667fc700 1 -- 172.21.15.16:0/3611768962 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9758011d00 con 0x7f976809e930 2023-09-30T22:36:50.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.579+0000 7f976dd05700 1 -- 172.21.15.16:0/3611768962 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f976809e930 msgr2=0x7f976809ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:50.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.579+0000 7f976dd05700 1 --2- 172.21.15.16:0/3611768962 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f976809e930 0x7f976809ed00 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f9758009870 tx=0x7f9758009b80 comp rx=0 tx=0).stop 2023-09-30T22:36:50.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.579+0000 7f976dd05700 1 -- 172.21.15.16:0/3611768962 shutdown_connections 2023-09-30T22:36:50.581 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.579+0000 7f976dd05700 1 --2- 172.21.15.16:0/3611768962 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f976809e930 0x7f976809ed00 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:50.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.579+0000 7f976dd05700 1 --2- 172.21.15.16:0/3611768962 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f976809f240 0x7f97681317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:50.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.579+0000 7f976dd05700 1 -- 172.21.15.16:0/3611768962 >> 172.21.15.16:0/3611768962 conn(0x7f97680a4210 msgr2=0x7f97680a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:50.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.580+0000 7f976dd05700 1 -- 172.21.15.16:0/3611768962 shutdown_connections 2023-09-30T22:36:50.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.580+0000 7f976dd05700 1 -- 172.21.15.16:0/3611768962 wait complete. 2023-09-30T22:36:50.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.581+0000 7f976dd05700 1 Processor -- start 2023-09-30T22:36:50.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.581+0000 7f976dd05700 1 -- start start 2023-09-30T22:36:50.583 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.581+0000 7f976dd05700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f976809e930 0x7f976806b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:50.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.581+0000 7f976dd05700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f976809f240 0x7f976806bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:50.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.581+0000 7f976dd05700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f976809f650 0x7f976806c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:50.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.581+0000 7f976dd05700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97681a87d0 con 0x7f976809e930 2023-09-30T22:36:50.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.581+0000 7f976dd05700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97681a97e0 con 0x7f976809f650 2023-09-30T22:36:50.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.581+0000 7f976dd05700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f97681a8f70 con 0x7f976809f240 2023-09-30T22:36:50.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.581+0000 7f97677fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f976809e930 0x7f976806b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:50.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.582+0000 7f97677fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f976809e930 0x7f976806b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46090/0 (socket says 172.21.15.16:46090) 2023-09-30T22:36:50.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.582+0000 7f97677fe700 1 -- 172.21.15.16:0/2488432877 learned_addr learned my addr 172.21.15.16:0/2488432877 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:50.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.582+0000 7f9767fff700 1 --2- 172.21.15.16:0/2488432877 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f976809f650 0x7f976806c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:50.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.582+0000 7f9766ffd700 1 --2- 172.21.15.16:0/2488432877 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f976809f240 0x7f976806bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:50.584 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.582+0000 7f97677fe700 1 -- 172.21.15.16:0/2488432877 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f976809f240 msgr2=0x7f976806bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:50.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.582+0000 7f97677fe700 1 --2- 172.21.15.16:0/2488432877 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f976809f240 0x7f976806bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:50.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.582+0000 7f97677fe700 1 -- 172.21.15.16:0/2488432877 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f976809f650 msgr2=0x7f976806c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:50.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.582+0000 7f97677fe700 1 --2- 172.21.15.16:0/2488432877 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f976809f650 0x7f976806c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:50.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.582+0000 7f97677fe700 1 -- 172.21.15.16:0/2488432877 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f975c0094a0 con 0x7f976809e930 2023-09-30T22:36:50.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.582+0000 7f9766ffd700 1 --2- 172.21.15.16:0/2488432877 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f976809f240 0x7f976806bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:36:50.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.582+0000 7f97677fe700 1 --2- 172.21.15.16:0/2488432877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f976809e930 0x7f976806b880 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f9758009bc0 tx=0x7f9758004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:50.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.582+0000 7f974ffff700 1 -- 172.21.15.16:0/2488432877 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97580045d0 con 0x7f976809e930 2023-09-30T22:36:50.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.582+0000 7f974ffff700 1 -- 172.21.15.16:0/2488432877 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f975801f070 con 0x7f976809e930 2023-09-30T22:36:50.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.582+0000 7f976dd05700 1 -- 172.21.15.16:0/2488432877 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97580094a0 con 0x7f976809e930 2023-09-30T22:36:50.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.582+0000 7f974ffff700 1 -- 172.21.15.16:0/2488432877 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97580038e0 con 0x7f976809e930 2023-09-30T22:36:50.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.583+0000 7f976dd05700 1 -- 172.21.15.16:0/2488432877 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97680a0190 con 0x7f976809e930 2023-09-30T22:36:50.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.584+0000 7f974ffff700 1 -- 172.21.15.16:0/2488432877 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9758003a40 con 0x7f976809e930 2023-09-30T22:36:50.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.584+0000 7f976dd05700 1 -- 172.21.15.16:0/2488432877 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f976806d4d0 con 0x7f976809e930 2023-09-30T22:36:50.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.584+0000 7f974ffff700 1 -- 172.21.15.16:0/2488432877 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f9758021070 con 0x7f976809e930 2023-09-30T22:36:50.589 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.589+0000 7f974ffff700 1 -- 172.21.15.16:0/2488432877 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9758024380 con 0x7f976809e930 2023-09-30T22:36:50.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.599+0000 7ff2ad021700 1 Processor -- start 2023-09-30T22:36:50.599 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.599+0000 7ff2ad021700 1 -- start start 2023-09-30T22:36:50.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.600+0000 7ff2ad021700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff2a809e930 0x7ff2a809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:50.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.600+0000 7ff2ad021700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff2a809f650 0x7ff2a8131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:50.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.600+0000 7ff2ad021700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2a8070940 con 0x7ff2a809f240 2023-09-30T22:36:50.600 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.600+0000 7ff2ad021700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2a806fd60 con 0x7ff2a809e930 2023-09-30T22:36:50.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.600+0000 7ff2ad021700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a806fa80 con 0x7ff2a809f650 2023-09-30T22:36:50.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.600+0000 7ff2a6d9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff2a809e930 0x7ff2a809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:50.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.600+0000 7ff2a6d9d700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff2a809e930 0x7ff2a809ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46106/0 (socket says 172.21.15.16:46106) 2023-09-30T22:36:50.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.600+0000 7ff2a6d9d700 1 -- 172.21.15.16:0/3021857861 learned_addr learned my addr 172.21.15.16:0/3021857861 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:50.601 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.600+0000 7ff2a759e700 1 --2- 172.21.15.16:0/3021857861 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff2a809f650 0x7ff2a8131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:50.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.600+0000 7ff2a6d9d700 1 -- 172.21.15.16:0/3021857861 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff2a809f650 msgr2=0x7ff2a8131f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:50.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.600+0000 7ff2a6d9d700 1 --2- 172.21.15.16:0/3021857861 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff2a809f650 0x7ff2a8131f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:50.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.600+0000 7ff2a6d9d700 1 -- 172.21.15.16:0/3021857861 >> v1:172.21.15.16:6789/0 conn(0x7ff2a809f240 legacy=0x7ff2a81317f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-09-30T22:36:50.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.600+0000 7ff2a6d9d700 1 -- 172.21.15.16:0/3021857861 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a809fb60 con 0x7ff2a809e930 2023-09-30T22:36:50.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.601+0000 7ff2a6d9d700 1 --2- 172.21.15.16:0/3021857861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2a809e930 0x7ff2a809ed00 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7ff2980098f0 tx=0x7ff298009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfa6e9eecff7bddd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:50.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.601+0000 7ff2a5d9b700 1 -- 172.21.15.16:0/3021857861 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff298004230 con 0x7ff2a809e930 2023-09-30T22:36:50.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.601+0000 7ff2a5d9b700 1 -- 172.21.15.16:0/3021857861 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff298004390 con 0x7ff2a809e930 2023-09-30T22:36:50.602 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.601+0000 7ff2a5d9b700 1 -- 172.21.15.16:0/3021857861 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2980036a0 con 0x7ff2a809e930 2023-09-30T22:36:50.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.602+0000 7ff2ad021700 1 -- 172.21.15.16:0/3021857861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2a809e930 msgr2=0x7ff2a809ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:50.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.602+0000 7ff2ad021700 1 --2- 172.21.15.16:0/3021857861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2a809e930 0x7ff2a809ed00 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7ff2980098f0 tx=0x7ff298009c00 comp rx=0 tx=0).stop 2023-09-30T22:36:50.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.602+0000 7ff2ad021700 1 -- 172.21.15.16:0/3021857861 shutdown_connections 2023-09-30T22:36:50.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.602+0000 7ff2ad021700 1 --2- 172.21.15.16:0/3021857861 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff2a809f650 0x7ff2a8131f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:50.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.602+0000 7ff2ad021700 1 --2- 172.21.15.16:0/3021857861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2a809e930 0x7ff2a809ed00 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:50.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.602+0000 7ff2ad021700 1 -- 172.21.15.16:0/3021857861 >> 172.21.15.16:0/3021857861 conn(0x7ff2a80a4210 msgr2=0x7ff2a80a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:50.603 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.602+0000 7ff2ad021700 1 -- 172.21.15.16:0/3021857861 shutdown_connections 2023-09-30T22:36:50.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.602+0000 7ff2ad021700 1 -- 172.21.15.16:0/3021857861 wait complete. 2023-09-30T22:36:50.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.604+0000 7ff2ad021700 1 Processor -- start 2023-09-30T22:36:50.604 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.604+0000 7ff2ad021700 1 -- start start 2023-09-30T22:36:50.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.605+0000 7ff2ad021700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2a809e930 0x7ff2a81a1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:50.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.605+0000 7ff2ad021700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff2a809f240 0x7ff2a81a2190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:50.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.605+0000 7ff2ad021700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff2a809f650 0x7ff2a81a26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:50.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.605+0000 7ff2ad021700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a81a3100 con 0x7ff2a809e930 2023-09-30T22:36:50.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.605+0000 7ff2ad021700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a81a3860 con 0x7ff2a809f650 2023-09-30T22:36:50.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.605+0000 7ff2ad021700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a81a4ab0 con 0x7ff2a809f240 2023-09-30T22:36:50.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.605+0000 7ff2a659c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff2a809f240 0x7ff2a81a2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:50.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.605+0000 7ff2a659c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff2a809f240 0x7ff2a81a2190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:51120/0 (socket says 172.21.15.16:51120) 2023-09-30T22:36:50.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.605+0000 7ff2a659c700 1 -- 172.21.15.16:0/809258582 learned_addr learned my addr 172.21.15.16:0/809258582 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:50.607 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.605+0000 7ff2a6d9d700 1 --2- 172.21.15.16:0/809258582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2a809e930 0x7ff2a81a1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:50.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.606+0000 7ff2a759e700 1 --2- 172.21.15.16:0/809258582 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff2a809f650 0x7ff2a81a26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:50.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.606+0000 7ff2a6d9d700 1 -- 172.21.15.16:0/809258582 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff2a809f240 msgr2=0x7ff2a81a2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:50.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.606+0000 7ff2a6d9d700 1 --2- 172.21.15.16:0/809258582 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff2a809f240 0x7ff2a81a2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:50.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.606+0000 7ff2a6d9d700 1 -- 172.21.15.16:0/809258582 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff2a809f650 msgr2=0x7ff2a81a26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:50.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.606+0000 7ff2a6d9d700 1 --2- 172.21.15.16:0/809258582 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff2a809f650 0x7ff2a81a26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:50.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.606+0000 7ff2a6d9d700 1 -- 172.21.15.16:0/809258582 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff298009550 con 0x7ff2a809e930 2023-09-30T22:36:50.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.606+0000 7ff2a6d9d700 1 --2- 172.21.15.16:0/809258582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2a809e930 0x7ff2a81a1c50 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7ff2a8072a80 tx=0x7ff298004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:50.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.606+0000 7ff28f7fe700 1 -- 172.21.15.16:0/809258582 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff298003ae0 con 0x7ff2a809e930 2023-09-30T22:36:50.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.606+0000 7ff28f7fe700 1 -- 172.21.15.16:0/809258582 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff29801f070 con 0x7ff2a809e930 2023-09-30T22:36:50.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.606+0000 7ff28f7fe700 1 -- 172.21.15.16:0/809258582 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff298003e40 con 0x7ff2a809e930 2023-09-30T22:36:50.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.606+0000 7ff2ad021700 1 -- 172.21.15.16:0/809258582 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2a80a0190 con 0x7ff2a809e930 2023-09-30T22:36:50.608 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.606+0000 7ff2ad021700 1 -- 172.21.15.16:0/809258582 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2a81a54f0 con 0x7ff2a809e930 2023-09-30T22:36:50.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.608+0000 7ff2ad021700 1 -- 172.21.15.16:0/809258582 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2a80a3590 con 0x7ff2a809e930 2023-09-30T22:36:50.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.608+0000 7ff28f7fe700 1 -- 172.21.15.16:0/809258582 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff298021030 con 0x7ff2a809e930 2023-09-30T22:36:50.612 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.609+0000 7ff28f7fe700 1 -- 172.21.15.16:0/809258582 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7ff298040e60 con 0x7ff2a809e930 2023-09-30T22:36:50.617 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.617+0000 7ff28f7fe700 1 -- 172.21.15.16:0/809258582 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff298023790 con 0x7ff2a809e930 2023-09-30T22:36:50.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.633+0000 7fb60675e700 1 Processor -- start 2023-09-30T22:36:50.634 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.633+0000 7fb60675e700 1 -- start start 2023-09-30T22:36:50.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.634+0000 7fb60675e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb60009e1d0 0x7fb600135910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:50.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.634+0000 7fb60675e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fb600135e50 0x7fb600138230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:50.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.634+0000 7fb60675e700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb60009f130 con 0x7fb60009d6f0 2023-09-30T22:36:50.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.634+0000 7fb60675e700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb60009f520 con 0x7fb600135e50 2023-09-30T22:36:50.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.634+0000 7fb60675e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb60009e930 con 0x7fb60009e1d0 2023-09-30T22:36:50.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.634+0000 7fb5ff7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb60009e1d0 0x7fb600135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:50.636 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.634+0000 7fb5ff7fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb60009e1d0 0x7fb600135910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37792/0 (socket says 172.21.15.16:37792) 2023-09-30T22:36:50.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.634+0000 7fb5ff7fe700 1 -- 172.21.15.16:0/2356106300 learned_addr learned my addr 172.21.15.16:0/2356106300 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:50.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.634+0000 7fb604cfb700 1 --2- 172.21.15.16:0/2356106300 >> v2:172.21.15.16:3300/0 conn(0x7fb600135e50 0x7fb600138230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:50.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.634+0000 7fb604cfb700 1 -- 172.21.15.16:0/2356106300 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb60009e1d0 msgr2=0x7fb600135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:50.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.634+0000 7fb604cfb700 1 --2- 172.21.15.16:0/2356106300 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb60009e1d0 0x7fb600135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:50.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.635+0000 7fb604cfb700 1 -- 172.21.15.16:0/2356106300 >> v1:172.21.15.16:6789/0 conn(0x7fb60009d6f0 legacy=0x7fb60009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:50.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.635+0000 7fb604cfb700 1 -- 172.21.15.16:0/2356106300 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb60009b2d0 con 0x7fb600135e50 2023-09-30T22:36:50.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.635+0000 7fb5ff7fe700 1 --2- 172.21.15.16:0/2356106300 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb60009e1d0 0x7fb600135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:50.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.635+0000 7fb604cfb700 1 --2- 172.21.15.16:0/2356106300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb600135e50 0x7fb600138230 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8009dc0 tx=0x7fb5f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b876c378187757f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:50.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.635+0000 7fb5feffd700 1 -- 172.21.15.16:0/2356106300 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f800e070 con 0x7fb600135e50 2023-09-30T22:36:50.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.635+0000 7fb5feffd700 1 -- 172.21.15.16:0/2356106300 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f800bae0 con 0x7fb600135e50 2023-09-30T22:36:50.637 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.635+0000 7fb5feffd700 1 -- 172.21.15.16:0/2356106300 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f800be90 con 0x7fb600135e50 2023-09-30T22:36:50.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.636+0000 7fb60675e700 1 -- 172.21.15.16:0/2356106300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb600135e50 msgr2=0x7fb600138230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:50.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.636+0000 7fb60675e700 1 --2- 172.21.15.16:0/2356106300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb600135e50 0x7fb600138230 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8009dc0 tx=0x7fb5f800b330 comp rx=0 tx=0).stop 2023-09-30T22:36:50.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.636+0000 7fb60675e700 1 -- 172.21.15.16:0/2356106300 shutdown_connections 2023-09-30T22:36:50.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.636+0000 7fb60675e700 1 --2- 172.21.15.16:0/2356106300 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb600135e50 0x7fb600138230 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:50.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.636+0000 7fb60675e700 1 --2- 172.21.15.16:0/2356106300 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb60009e1d0 0x7fb600135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:50.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.636+0000 7fb60675e700 1 -- 172.21.15.16:0/2356106300 >> 172.21.15.16:0/2356106300 conn(0x7fb600066f50 msgr2=0x7fb60009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:50.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.636+0000 7fb60675e700 1 -- 172.21.15.16:0/2356106300 shutdown_connections 2023-09-30T22:36:50.640 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.636+0000 7fb60675e700 1 -- 172.21.15.16:0/2356106300 wait complete. 2023-09-30T22:36:50.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.637+0000 7fb60675e700 1 Processor -- start 2023-09-30T22:36:50.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.638+0000 7fb60675e700 1 -- start start 2023-09-30T22:36:50.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.638+0000 7fb60675e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb60009d6f0 0x7fb600119480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:50.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.638+0000 7fb60675e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb60009e1d0 0x7fb6001199c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:50.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.638+0000 7fb60675e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb600135e50 0x7fb600119f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:50.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.638+0000 7fb60675e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60011cd00 con 0x7fb60009e1d0 2023-09-30T22:36:50.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.638+0000 7fb60675e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb60011b900 con 0x7fb60009d6f0 2023-09-30T22:36:50.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.638+0000 7fb60675e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fb600072650 con 0x7fb600135e50 2023-09-30T22:36:50.641 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.638+0000 7fb5ff7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb60009e1d0 0x7fb6001199c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:50.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.638+0000 7fb5ff7fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb60009e1d0 0x7fb6001199c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46138/0 (socket says 172.21.15.16:46138) 2023-09-30T22:36:50.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.638+0000 7fb5ff7fe700 1 -- 172.21.15.16:0/3014241728 learned_addr learned my addr 172.21.15.16:0/3014241728 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:50.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.638+0000 7fb604cfb700 1 --2- 172.21.15.16:0/3014241728 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb600135e50 0x7fb600119f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:50.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.638+0000 7fb5fffff700 1 --2- 172.21.15.16:0/3014241728 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb60009d6f0 0x7fb600119480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:50.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.639+0000 7fb604cfb700 1 -- 172.21.15.16:0/3014241728 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb60009d6f0 msgr2=0x7fb600119480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:50.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.639+0000 7fb604cfb700 1 --2- 172.21.15.16:0/3014241728 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb60009d6f0 0x7fb600119480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:50.642 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.639+0000 7fb604cfb700 1 -- 172.21.15.16:0/3014241728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb60009e1d0 msgr2=0x7fb6001199c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:50.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.639+0000 7fb604cfb700 1 --2- 172.21.15.16:0/3014241728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb60009e1d0 0x7fb6001199c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:50.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.639+0000 7fb604cfb700 1 -- 172.21.15.16:0/3014241728 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f400b7f0 con 0x7fb600135e50 2023-09-30T22:36:50.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.639+0000 7fb5ff7fe700 1 --2- 172.21.15.16:0/3014241728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb60009e1d0 0x7fb6001199c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:50.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.639+0000 7fb604cfb700 1 --2- 172.21.15.16:0/3014241728 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb600135e50 0x7fb600119f00 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8016000 tx=0x7fb5f8004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:50.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.639+0000 7fb5fcff9700 1 -- 172.21.15.16:0/3014241728 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f80045d0 con 0x7fb600135e50 2023-09-30T22:36:50.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.639+0000 7fb5fcff9700 1 -- 172.21.15.16:0/3014241728 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f8020070 con 0x7fb600135e50 2023-09-30T22:36:50.643 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.639+0000 7fb5fcff9700 1 -- 172.21.15.16:0/3014241728 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f8007060 con 0x7fb600135e50 2023-09-30T22:36:50.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.640+0000 7fb60675e700 1 -- 172.21.15.16:0/3014241728 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5f8009a70 con 0x7fb600135e50 2023-09-30T22:36:50.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.640+0000 7fb60675e700 1 -- 172.21.15.16:0/3014241728 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb60011ca40 con 0x7fb600135e50 2023-09-30T22:36:50.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.641+0000 7fb5fcff9700 1 -- 172.21.15.16:0/3014241728 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fb5f8027480 con 0x7fb600135e50 2023-09-30T22:36:50.644 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.641+0000 7fb60675e700 1 -- 172.21.15.16:0/3014241728 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb60009ef50 con 0x7fb600135e50 2023-09-30T22:36:50.645 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.641+0000 7fb5fcff9700 1 -- 172.21.15.16:0/3014241728 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fb5f8022070 con 0x7fb600135e50 2023-09-30T22:36:50.651 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.651+0000 7fb5fcff9700 1 -- 172.21.15.16:0/3014241728 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fb5f8011980 con 0x7fb600135e50 2023-09-30T22:36:50.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.755+0000 7fd177fb5700 1 -- 172.21.15.16:0/3434102565 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd17006b9e0 con 0x7fd17009a580 2023-09-30T22:36:50.756 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.756+0000 7fd1667fc700 1 -- 172.21.15.16:0/3434102565 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd160017030 con 0x7fd17009a580 2023-09-30T22:36:50.756 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:36:50.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.759+0000 7fd177fb5700 1 -- 172.21.15.16:0/3434102565 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd17009a580 msgr2=0x7fd17011bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:50.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.759+0000 7fd177fb5700 1 --2- 172.21.15.16:0/3434102565 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd17009a580 0x7fd17011bb80 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fd16000c520 tx=0x7fd16000c8e0 comp rx=0 tx=0).stop 2023-09-30T22:36:50.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.759+0000 7fd177fb5700 1 -- 172.21.15.16:0/3434102565 shutdown_connections 2023-09-30T22:36:50.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.759+0000 7fd177fb5700 1 --2- 172.21.15.16:0/3434102565 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd170131810 0x7fd17011c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:50.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.759+0000 7fd177fb5700 1 --2- 172.21.15.16:0/3434102565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd170099a40 0x7fd17011b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:50.760 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.759+0000 7fd177fb5700 1 --2- 172.21.15.16:0/3434102565 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd17009a580 0x7fd17011bb80 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:50.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.759+0000 7fd177fb5700 1 -- 172.21.15.16:0/3434102565 >> 172.21.15.16:0/3434102565 conn(0x7fd1700a2aa0 msgr2=0x7fd1701b8b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:50.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.759+0000 7fd177fb5700 1 -- 172.21.15.16:0/3434102565 shutdown_connections 2023-09-30T22:36:50.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.759+0000 7fd177fb5700 1 -- 172.21.15.16:0/3434102565 wait complete. 2023-09-30T22:36:50.779 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:36:50.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.814+0000 7f976dd05700 1 -- 172.21.15.16:0/2488432877 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9768074ea0 con 0x7f976809e930 2023-09-30T22:36:50.815 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.815+0000 7f974ffff700 1 -- 172.21.15.16:0/2488432877 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9758015020 con 0x7f976809e930 2023-09-30T22:36:50.816 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:36:50.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.818+0000 7f976dd05700 1 -- 172.21.15.16:0/2488432877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f976809e930 msgr2=0x7f976806b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:50.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.818+0000 7f976dd05700 1 --2- 172.21.15.16:0/2488432877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f976809e930 0x7f976806b880 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f9758009bc0 tx=0x7f9758004750 comp rx=0 tx=0).stop 2023-09-30T22:36:50.819 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.818+0000 7f976dd05700 1 -- 172.21.15.16:0/2488432877 shutdown_connections 2023-09-30T22:36:50.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.818+0000 7f976dd05700 1 --2- 172.21.15.16:0/2488432877 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f976809f650 0x7f976806c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:50.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.818+0000 7f976dd05700 1 --2- 172.21.15.16:0/2488432877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f976809e930 0x7f976806b880 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:50.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.818+0000 7f976dd05700 1 --2- 172.21.15.16:0/2488432877 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f976809f240 0x7f976806bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:50.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.818+0000 7f976dd05700 1 -- 172.21.15.16:0/2488432877 >> 172.21.15.16:0/2488432877 conn(0x7f97680a4210 msgr2=0x7f97681b8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:50.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.818+0000 7f976dd05700 1 -- 172.21.15.16:0/2488432877 shutdown_connections 2023-09-30T22:36:50.820 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.818+0000 7f976dd05700 1 -- 172.21.15.16:0/2488432877 wait complete. 2023-09-30T22:36:50.838 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:36:50.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.848+0000 7ff2ad021700 1 -- 172.21.15.16:0/809258582 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff2a80a3590 con 0x7ff2a809e930 2023-09-30T22:36:50.848 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.848+0000 7ff28f7fe700 1 -- 172.21.15.16:0/809258582 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff2980233d0 con 0x7ff2a809e930 2023-09-30T22:36:50.849 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:36:50.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.850+0000 7ff2ad021700 1 -- 172.21.15.16:0/809258582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2a809e930 msgr2=0x7ff2a81a1c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:50.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.850+0000 7ff2ad021700 1 --2- 172.21.15.16:0/809258582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2a809e930 0x7ff2a81a1c50 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7ff2a8072a80 tx=0x7ff298004750 comp rx=0 tx=0).stop 2023-09-30T22:36:50.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.850+0000 7ff2ad021700 1 -- 172.21.15.16:0/809258582 shutdown_connections 2023-09-30T22:36:50.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.850+0000 7ff2ad021700 1 --2- 172.21.15.16:0/809258582 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff2a809f650 0x7ff2a81a26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:50.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.850+0000 7ff2ad021700 1 --2- 172.21.15.16:0/809258582 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2a809e930 0x7ff2a81a1c50 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:50.851 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.850+0000 7ff2ad021700 1 --2- 172.21.15.16:0/809258582 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff2a809f240 0x7ff2a81a2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:50.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.850+0000 7ff2ad021700 1 -- 172.21.15.16:0/809258582 >> 172.21.15.16:0/809258582 conn(0x7ff2a80a4210 msgr2=0x7ff2a81b6b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:50.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.850+0000 7ff2ad021700 1 -- 172.21.15.16:0/809258582 shutdown_connections 2023-09-30T22:36:50.852 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.850+0000 7ff2ad021700 1 -- 172.21.15.16:0/809258582 wait complete. 2023-09-30T22:36:50.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.862+0000 7fb60675e700 1 -- 172.21.15.16:0/3014241728 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb600074ea0 con 0x7fb600135e50 2023-09-30T22:36:50.863 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.863+0000 7fb5fcff9700 1 -- 172.21.15.16:0/3014241728 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb5f802c020 con 0x7fb600135e50 2023-09-30T22:36:50.863 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:36:50.864 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:36:50.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.865+0000 7fb60675e700 1 -- 172.21.15.16:0/3014241728 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb600135e50 msgr2=0x7fb600119f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:50.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.865+0000 7fb60675e700 1 --2- 172.21.15.16:0/3014241728 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb600135e50 0x7fb600119f00 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8016000 tx=0x7fb5f8004ad0 comp rx=0 tx=0).stop 2023-09-30T22:36:50.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.865+0000 7fb60675e700 1 -- 172.21.15.16:0/3014241728 shutdown_connections 2023-09-30T22:36:50.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.865+0000 7fb60675e700 1 --2- 172.21.15.16:0/3014241728 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fb60009d6f0 0x7fb600119480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:50.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.865+0000 7fb60675e700 1 --2- 172.21.15.16:0/3014241728 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fb600135e50 0x7fb600119f00 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:50.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.865+0000 7fb60675e700 1 --2- 172.21.15.16:0/3014241728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb60009e1d0 0x7fb6001199c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:50.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.865+0000 7fb60675e700 1 -- 172.21.15.16:0/3014241728 >> 172.21.15.16:0/3014241728 conn(0x7fb600066f50 msgr2=0x7fb6000a0cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:50.866 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.865+0000 7fb60675e700 1 -- 172.21.15.16:0/3014241728 shutdown_connections 2023-09-30T22:36:50.867 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:50.865+0000 7fb60675e700 1 -- 172.21.15.16:0/3014241728 wait complete. 2023-09-30T22:36:50.879 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:36:51.779 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:36:51.839 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:36:51.866 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:36:51.880 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:36:51.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.933+0000 7f11606c5700 1 Processor -- start 2023-09-30T22:36:51.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.933+0000 7f11606c5700 1 -- start start 2023-09-30T22:36:51.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.934+0000 7f11606c5700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f115812cf60 0x7f115806f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:51.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.934+0000 7f11606c5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f115812f8b0 0x7f115812fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:51.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.934+0000 7f11606c5700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f115812cd90 con 0x7f1158130260 2023-09-30T22:36:51.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.934+0000 7f11606c5700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f115812d330 con 0x7f115812cf60 2023-09-30T22:36:51.934 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.934+0000 7f11606c5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f11581306d0 con 0x7f115812f8b0 2023-09-30T22:36:51.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.934+0000 7f115dc60700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f115812f8b0 0x7f115812fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:51.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.934+0000 7f115dc60700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f115812f8b0 0x7f115812fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:51144/0 (socket says 172.21.15.16:51144) 2023-09-30T22:36:51.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.934+0000 7f115dc60700 1 -- 172.21.15.16:0/830525480 learned_addr learned my addr 172.21.15.16:0/830525480 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:51.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.934+0000 7f115e461700 1 --2- 172.21.15.16:0/830525480 >> v2:172.21.15.16:3300/0 conn(0x7f115812cf60 0x7f115806f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:51.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.934+0000 7f115dc60700 1 -- 172.21.15.16:0/830525480 >> v2:172.21.15.16:3300/0 conn(0x7f115812cf60 msgr2=0x7f115806f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:51.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.934+0000 7f115dc60700 1 --2- 172.21.15.16:0/830525480 >> v2:172.21.15.16:3300/0 conn(0x7f115812cf60 0x7f115806f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:51.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.934+0000 7f115dc60700 1 -- 172.21.15.16:0/830525480 >> v1:172.21.15.16:6789/0 conn(0x7f1158130260 legacy=0x7f1158136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:51.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.934+0000 7f115d45f700 1 -- 172.21.15.16:0/830525480 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2041029910 0 0) 0x7f115812cd90 con 0x7f1158130260 2023-09-30T22:36:51.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.934+0000 7f115dc60700 1 -- 172.21.15.16:0/830525480 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1158071780 con 0x7f115812f8b0 2023-09-30T22:36:51.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.935+0000 7f115dc60700 1 --2- 172.21.15.16:0/830525480 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f115812f8b0 0x7f115812fd20 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f114800bb90 tx=0x7f114800bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4d40fdb496db538 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:51.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.935+0000 7f115d45f700 1 -- 172.21.15.16:0/830525480 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f114800dc20 con 0x7f115812f8b0 2023-09-30T22:36:51.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.935+0000 7f115d45f700 1 -- 172.21.15.16:0/830525480 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f114800dd80 con 0x7f115812f8b0 2023-09-30T22:36:51.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.935+0000 7f115d45f700 1 -- 172.21.15.16:0/830525480 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1148017650 con 0x7f115812f8b0 2023-09-30T22:36:51.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.936+0000 7f11606c5700 1 -- 172.21.15.16:0/830525480 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f115812f8b0 msgr2=0x7f115812fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:51.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.936+0000 7f11606c5700 1 --2- 172.21.15.16:0/830525480 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f115812f8b0 0x7f115812fd20 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f114800bb90 tx=0x7f114800bea0 comp rx=0 tx=0).stop 2023-09-30T22:36:51.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.936+0000 7f11606c5700 1 -- 172.21.15.16:0/830525480 shutdown_connections 2023-09-30T22:36:51.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.936+0000 7f11606c5700 1 --2- 172.21.15.16:0/830525480 >> v2:172.21.15.16:3300/0 conn(0x7f115812cf60 0x7f115806f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:51.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.936+0000 7f11606c5700 1 --2- 172.21.15.16:0/830525480 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f115812f8b0 0x7f115812fd20 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:51.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.936+0000 7f11606c5700 1 -- 172.21.15.16:0/830525480 >> 172.21.15.16:0/830525480 conn(0x7f11580a2aa0 msgr2=0x7f11580a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:51.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.936+0000 7f11606c5700 1 -- 172.21.15.16:0/830525480 shutdown_connections 2023-09-30T22:36:51.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.936+0000 7f11606c5700 1 -- 172.21.15.16:0/830525480 wait complete. 2023-09-30T22:36:51.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.937+0000 7f11606c5700 1 Processor -- start 2023-09-30T22:36:51.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.937+0000 7f11606c5700 1 -- start start 2023-09-30T22:36:51.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.937+0000 7f11606c5700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f115812cf60 0x7f11581c5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:51.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.937+0000 7f11606c5700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f115812f8b0 0x7f11581c5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:51.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.937+0000 7f11606c5700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1158130260 0x7f11581c8170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:51.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.937+0000 7f11606c5700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f115809c4f0 con 0x7f1158130260 2023-09-30T22:36:51.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.937+0000 7f11606c5700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11581c5bd0 con 0x7f115812f8b0 2023-09-30T22:36:51.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.937+0000 7f11606c5700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f115809e5c0 con 0x7f115812cf60 2023-09-30T22:36:51.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.938+0000 7f115dc60700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f115812f8b0 0x7f11581c5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:51.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.938+0000 7f115e461700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f115812cf60 0x7f11581c5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:51.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.938+0000 7f115dc60700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f115812f8b0 0x7f11581c5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37810/0 (socket says 172.21.15.16:37810) 2023-09-30T22:36:51.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.938+0000 7f115dc60700 1 -- 172.21.15.16:0/260434959 learned_addr learned my addr 172.21.15.16:0/260434959 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:51.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.938+0000 7f115ec62700 1 --2- 172.21.15.16:0/260434959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1158130260 0x7f11581c8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:51.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.938+0000 7f115dc60700 1 -- 172.21.15.16:0/260434959 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f115812cf60 msgr2=0x7f11581c5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:51.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.938+0000 7f115dc60700 1 --2- 172.21.15.16:0/260434959 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f115812cf60 0x7f11581c5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:51.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.938+0000 7f115dc60700 1 -- 172.21.15.16:0/260434959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1158130260 msgr2=0x7f11581c8170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:51.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.938+0000 7f115dc60700 1 --2- 172.21.15.16:0/260434959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1158130260 0x7f11581c8170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:51.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.938+0000 7f115dc60700 1 -- 172.21.15.16:0/260434959 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f114800b7f0 con 0x7f115812f8b0 2023-09-30T22:36:51.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.938+0000 7f115e461700 1 --2- 172.21.15.16:0/260434959 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f115812cf60 0x7f11581c5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:51.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.938+0000 7f115ec62700 1 --2- 172.21.15.16:0/260434959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1158130260 0x7f11581c8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:36:51.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.938+0000 7f115dc60700 1 --2- 172.21.15.16:0/260434959 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f115812f8b0 0x7f11581c5690 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f114800bee0 tx=0x7f1148004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:51.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.939+0000 7f114effd700 1 -- 172.21.15.16:0/260434959 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1148017800 con 0x7f115812f8b0 2023-09-30T22:36:51.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.939+0000 7f114effd700 1 -- 172.21.15.16:0/260434959 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1148017960 con 0x7f115812f8b0 2023-09-30T22:36:51.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.939+0000 7f11606c5700 1 -- 172.21.15.16:0/260434959 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f115809eb60 con 0x7f115812f8b0 2023-09-30T22:36:51.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.939+0000 7f114effd700 1 -- 172.21.15.16:0/260434959 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1148017cf0 con 0x7f115812f8b0 2023-09-30T22:36:51.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.939+0000 7f11606c5700 1 -- 172.21.15.16:0/260434959 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f115809cc50 con 0x7f115812f8b0 2023-09-30T22:36:51.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.940+0000 7f114effd700 1 -- 172.21.15.16:0/260434959 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1148015070 con 0x7f115812f8b0 2023-09-30T22:36:51.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.940+0000 7f11606c5700 1 -- 172.21.15.16:0/260434959 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f115809e750 con 0x7f115812f8b0 2023-09-30T22:36:51.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.940+0000 7f114effd700 1 -- 172.21.15.16:0/260434959 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f1148040e90 con 0x7f115812f8b0 2023-09-30T22:36:51.949 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.949+0000 7f114effd700 1 -- 172.21.15.16:0/260434959 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f115809e750 con 0x7f115812f8b0 2023-09-30T22:36:51.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.990+0000 7f6decdbc700 1 Processor -- start 2023-09-30T22:36:51.990 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.990+0000 7f6decdbc700 1 -- start start 2023-09-30T22:36:51.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.990+0000 7f6decdbc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6de806b250 0x7f6de8136840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:51.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.990+0000 7f6decdbc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6de8136d80 0x7f6de8139160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:51.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.991+0000 7f6decdbc700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6de809ece0 con 0x7f6de806a770 2023-09-30T22:36:51.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.991+0000 7f6decdbc700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6de809e760 con 0x7f6de806b250 2023-09-30T22:36:51.991 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.991+0000 7f6decdbc700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6de809e040 con 0x7f6de8136d80 2023-09-30T22:36:51.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.991+0000 7f6de5d9b700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f6de806b250 0x7f6de8136840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:51.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.991+0000 7f6de659c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f6de806a770 0x7f6de806ab40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48974/0 (socket says 172.21.15.16:48974) 2023-09-30T22:36:51.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.991+0000 7f6de6d9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6de8136d80 0x7f6de8139160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:51.992 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.991+0000 7f6de659c700 1 -- 172.21.15.16:0/975280651 learned_addr learned my addr 172.21.15.16:0/975280651 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:36:51.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.991+0000 7f6de5d9b700 1 --2- 172.21.15.16:0/975280651 >> v2:172.21.15.16:3300/0 conn(0x7f6de806b250 0x7f6de8136840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46176/0 (socket says 172.21.15.16:46176) 2023-09-30T22:36:51.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.991+0000 7f6de5d9b700 1 -- 172.21.15.16:0/975280651 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6de8136d80 msgr2=0x7f6de8139160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:51.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.991+0000 7f6de5d9b700 1 --2- 172.21.15.16:0/975280651 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6de8136d80 0x7f6de8139160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:51.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.991+0000 7f6de5d9b700 1 -- 172.21.15.16:0/975280651 >> v1:172.21.15.16:6789/0 conn(0x7f6de806a770 legacy=0x7f6de806ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:51.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.991+0000 7f6de5d9b700 1 -- 172.21.15.16:0/975280651 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de809f630 con 0x7f6de806b250 2023-09-30T22:36:51.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.991+0000 7f6de559a700 1 -- 172.21.15.16:0/975280651 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1723333203 0 0) 0x7f6de809ece0 con 0x7f6de806a770 2023-09-30T22:36:51.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.991+0000 7f6de6d9d700 1 --2- 172.21.15.16:0/975280651 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6de8136d80 0x7f6de8139160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:51.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.992+0000 7f6de5d9b700 1 --2- 172.21.15.16:0/975280651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6de806b250 0x7f6de8136840 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc00dc60 tx=0x7f6ddc00df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=296fc6dfad1488e8 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:51.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.992+0000 7f6de559a700 1 -- 172.21.15.16:0/975280651 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc012070 con 0x7f6de806b250 2023-09-30T22:36:51.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.992+0000 7f6de559a700 1 -- 172.21.15.16:0/975280651 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ddc00d2e0 con 0x7f6de806b250 2023-09-30T22:36:51.993 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.992+0000 7f6de559a700 1 -- 172.21.15.16:0/975280651 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc00f900 con 0x7f6de806b250 2023-09-30T22:36:51.994 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.993+0000 7f6decdbc700 1 -- 172.21.15.16:0/975280651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6de806b250 msgr2=0x7f6de8136840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:51.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.993+0000 7f6decdbc700 1 --2- 172.21.15.16:0/975280651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6de806b250 0x7f6de8136840 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc00dc60 tx=0x7f6ddc00df70 comp rx=0 tx=0).stop 2023-09-30T22:36:51.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.993+0000 7f6decdbc700 1 -- 172.21.15.16:0/975280651 shutdown_connections 2023-09-30T22:36:51.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.993+0000 7f6decdbc700 1 --2- 172.21.15.16:0/975280651 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6de8136d80 0x7f6de8139160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:51.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.993+0000 7f6decdbc700 1 --2- 172.21.15.16:0/975280651 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6de806b250 0x7f6de8136840 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:51.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.993+0000 7f6decdbc700 1 -- 172.21.15.16:0/975280651 >> 172.21.15.16:0/975280651 conn(0x7f6de8066f50 msgr2=0x7f6de809f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:51.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.993+0000 7f6decdbc700 1 -- 172.21.15.16:0/975280651 shutdown_connections 2023-09-30T22:36:51.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.993+0000 7f6decdbc700 1 -- 172.21.15.16:0/975280651 wait complete. 2023-09-30T22:36:51.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.994+0000 7f6decdbc700 1 Processor -- start 2023-09-30T22:36:51.995 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.994+0000 7f6decdbc700 1 -- start start 2023-09-30T22:36:51.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.995+0000 7f6decdbc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6de806a770 0x7f6de8119480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:51.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.995+0000 7f6decdbc700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6de806b250 0x7f6de81199c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:51.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.995+0000 7f6decdbc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6de8136d80 0x7f6de8119f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:51.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.995+0000 7f6decdbc700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de809a7a0 con 0x7f6de806b250 2023-09-30T22:36:51.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.995+0000 7f6decdbc700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6de8136780 con 0x7f6de8136d80 2023-09-30T22:36:51.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.995+0000 7f6decdbc700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f6de811ab30 con 0x7f6de806a770 2023-09-30T22:36:51.997 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.995+0000 7f6de659c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6de806a770 0x7f6de8119480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:51.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.995+0000 7f6de659c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6de806a770 0x7f6de8119480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:51180/0 (socket says 172.21.15.16:51180) 2023-09-30T22:36:51.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.995+0000 7f6de659c700 1 -- 172.21.15.16:0/589210122 learned_addr learned my addr 172.21.15.16:0/589210122 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:51.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.995+0000 7f6de5d9b700 1 --2- 172.21.15.16:0/589210122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6de806b250 0x7f6de81199c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:51.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.996+0000 7f6de6d9d700 1 --2- 172.21.15.16:0/589210122 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6de8136d80 0x7f6de8119f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:51.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.996+0000 7f6de659c700 1 -- 172.21.15.16:0/589210122 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6de8136d80 msgr2=0x7f6de8119f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:51.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.996+0000 7f6de659c700 1 --2- 172.21.15.16:0/589210122 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6de8136d80 0x7f6de8119f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:51.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.996+0000 7f6de659c700 1 -- 172.21.15.16:0/589210122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6de806b250 msgr2=0x7f6de81199c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:51.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.996+0000 7f6de659c700 1 --2- 172.21.15.16:0/589210122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6de806b250 0x7f6de81199c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:51.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.996+0000 7f6de659c700 1 -- 172.21.15.16:0/589210122 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de0009a70 con 0x7f6de806a770 2023-09-30T22:36:51.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.996+0000 7f6de5d9b700 1 --2- 172.21.15.16:0/589210122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6de806b250 0x7f6de81199c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:51.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.996+0000 7f6de6d9d700 1 --2- 172.21.15.16:0/589210122 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6de8136d80 0x7f6de8119f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:51.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.996+0000 7f6de659c700 1 --2- 172.21.15.16:0/589210122 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6de806a770 0x7f6de8119480 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f6dd800c520 tx=0x7f6dd800c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:51.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.996+0000 7f6dd6ffd700 1 -- 172.21.15.16:0/589210122 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd8004c80 con 0x7f6de806a770 2023-09-30T22:36:51.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.996+0000 7f6dd6ffd700 1 -- 172.21.15.16:0/589210122 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dd8004e00 con 0x7f6de806a770 2023-09-30T22:36:51.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.996+0000 7f6dd6ffd700 1 -- 172.21.15.16:0/589210122 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd8007310 con 0x7f6de806a770 2023-09-30T22:36:51.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.996+0000 7f6decdbc700 1 -- 172.21.15.16:0/589210122 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ddc00d940 con 0x7f6de806a770 2023-09-30T22:36:51.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.997+0000 7f6decdbc700 1 -- 172.21.15.16:0/589210122 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6de811baf0 con 0x7f6de806a770 2023-09-30T22:36:51.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.998+0000 7f6dd6ffd700 1 -- 172.21.15.16:0/589210122 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f6dd8007470 con 0x7f6de806a770 2023-09-30T22:36:52.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.998+0000 7f6decdbc700 1 -- 172.21.15.16:0/589210122 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6de8074ea0 con 0x7f6de806a770 2023-09-30T22:36:52.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:51.998+0000 7f6dd6ffd700 1 -- 172.21.15.16:0/589210122 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f6dd803f2e0 con 0x7f6de806a770 2023-09-30T22:36:52.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.007+0000 7f6dd6ffd700 1 -- 172.21.15.16:0/589210122 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f6dd80133c0 con 0x7f6de806a770 2023-09-30T22:36:52.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.031+0000 7ff64d280700 1 Processor -- start 2023-09-30T22:36:52.032 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.031+0000 7ff64d280700 1 -- start start 2023-09-30T22:36:52.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.032+0000 7ff64d280700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff64809e930 0x7ff64809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:52.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.032+0000 7ff64d280700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff64809f240 0x7ff6481317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:52.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.032+0000 7ff64d280700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff64809f650 0x7ff648131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:52.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.032+0000 7ff64d280700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff648070940 con 0x7ff64809f650 2023-09-30T22:36:52.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.032+0000 7ff64d280700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff64806fd60 con 0x7ff64809f240 2023-09-30T22:36:52.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.032+0000 7ff64d280700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff64806fa80 con 0x7ff64809e930 2023-09-30T22:36:52.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.032+0000 7ff646ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff64809e930 0x7ff64809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:52.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.032+0000 7ff646ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff64809e930 0x7ff64809ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:51186/0 (socket says 172.21.15.16:51186) 2023-09-30T22:36:52.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.032+0000 7ff6467fc700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff64809f240 0x7ff6481317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:52.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.032+0000 7ff646ffd700 1 -- 172.21.15.16:0/1952400436 learned_addr learned my addr 172.21.15.16:0/1952400436 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:52.034 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.033+0000 7ff6477fe700 1 --2- 172.21.15.16:0/1952400436 >> v2:172.21.15.16:3300/0 conn(0x7ff64809f650 0x7ff648131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:52.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.033+0000 7ff646ffd700 1 -- 172.21.15.16:0/1952400436 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff64809f240 msgr2=0x7ff6481317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:52.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.033+0000 7ff646ffd700 1 --2- 172.21.15.16:0/1952400436 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff64809f240 0x7ff6481317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:52.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.033+0000 7ff646ffd700 1 -- 172.21.15.16:0/1952400436 >> v2:172.21.15.16:3300/0 conn(0x7ff64809f650 msgr2=0x7ff648131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:52.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.033+0000 7ff646ffd700 1 --2- 172.21.15.16:0/1952400436 >> v2:172.21.15.16:3300/0 conn(0x7ff64809f650 0x7ff648131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:52.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.033+0000 7ff646ffd700 1 -- 172.21.15.16:0/1952400436 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff64809fb60 con 0x7ff64809e930 2023-09-30T22:36:52.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.033+0000 7ff6467fc700 1 --2- 172.21.15.16:0/1952400436 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff64809f240 0x7ff6481317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:52.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.033+0000 7ff6477fe700 1 --2- 172.21.15.16:0/1952400436 >> v2:172.21.15.16:3300/0 conn(0x7ff64809f650 0x7ff648131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:52.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.033+0000 7ff646ffd700 1 --2- 172.21.15.16:0/1952400436 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff64809e930 0x7ff64809ed00 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7ff6380098f0 tx=0x7ff638009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da780cfb155ba223 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:52.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.033+0000 7ff645ffb700 1 -- 172.21.15.16:0/1952400436 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff638011830 con 0x7ff64809e930 2023-09-30T22:36:52.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.033+0000 7ff645ffb700 1 -- 172.21.15.16:0/1952400436 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff638011990 con 0x7ff64809e930 2023-09-30T22:36:52.035 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.033+0000 7ff645ffb700 1 -- 172.21.15.16:0/1952400436 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff638011d20 con 0x7ff64809e930 2023-09-30T22:36:52.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.034+0000 7ff64d280700 1 -- 172.21.15.16:0/1952400436 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff64809e930 msgr2=0x7ff64809ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:52.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.034+0000 7ff64d280700 1 --2- 172.21.15.16:0/1952400436 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff64809e930 0x7ff64809ed00 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7ff6380098f0 tx=0x7ff638009c00 comp rx=0 tx=0).stop 2023-09-30T22:36:52.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.035+0000 7ff64d280700 1 -- 172.21.15.16:0/1952400436 shutdown_connections 2023-09-30T22:36:52.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.035+0000 7ff64d280700 1 --2- 172.21.15.16:0/1952400436 >> v2:172.21.15.16:3300/0 conn(0x7ff64809f650 0x7ff648131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:52.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.035+0000 7ff64d280700 1 --2- 172.21.15.16:0/1952400436 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff64809e930 0x7ff64809ed00 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:52.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.035+0000 7ff64d280700 1 --2- 172.21.15.16:0/1952400436 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff64809f240 0x7ff6481317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:52.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.035+0000 7ff64d280700 1 -- 172.21.15.16:0/1952400436 >> 172.21.15.16:0/1952400436 conn(0x7ff6480a4210 msgr2=0x7ff6480a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:52.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.035+0000 7ff64d280700 1 -- 172.21.15.16:0/1952400436 shutdown_connections 2023-09-30T22:36:52.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.035+0000 7ff64d280700 1 -- 172.21.15.16:0/1952400436 wait complete. 2023-09-30T22:36:52.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.036+0000 7ff64d280700 1 Processor -- start 2023-09-30T22:36:52.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.036+0000 7ff64d280700 1 -- start start 2023-09-30T22:36:52.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.037+0000 7ff64d280700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff64809e930 0x7ff6481a3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:52.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.037+0000 7ff64d280700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff64809f240 0x7ff6481a4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:52.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.037+0000 7ff64d280700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff64809f650 0x7ff6481a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:52.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.037+0000 7ff64d280700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6481a80e0 con 0x7ff64809e930 2023-09-30T22:36:52.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.037+0000 7ff64d280700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6481a7720 con 0x7ff64809f650 2023-09-30T22:36:52.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.037+0000 7ff64d280700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff6481a6a60 con 0x7ff64809f240 2023-09-30T22:36:52.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.037+0000 7ff6477fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff64809f650 0x7ff6481a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:52.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.037+0000 7ff6477fe700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff64809f650 0x7ff6481a4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37840/0 (socket says 172.21.15.16:37840) 2023-09-30T22:36:52.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.037+0000 7ff6477fe700 1 -- 172.21.15.16:0/2546582342 learned_addr learned my addr 172.21.15.16:0/2546582342 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:52.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.037+0000 7ff6467fc700 1 --2- 172.21.15.16:0/2546582342 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff64809f240 0x7ff6481a4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:52.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.038+0000 7ff646ffd700 1 --2- 172.21.15.16:0/2546582342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff64809e930 0x7ff6481a3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:52.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.038+0000 7ff6467fc700 1 -- 172.21.15.16:0/2546582342 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff64809f650 msgr2=0x7ff6481a4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:52.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.038+0000 7ff6467fc700 1 --2- 172.21.15.16:0/2546582342 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff64809f650 0x7ff6481a4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:52.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.038+0000 7ff6467fc700 1 -- 172.21.15.16:0/2546582342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff64809e930 msgr2=0x7ff6481a3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:52.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.038+0000 7ff6467fc700 1 --2- 172.21.15.16:0/2546582342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff64809e930 0x7ff6481a3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:52.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.038+0000 7ff6467fc700 1 -- 172.21.15.16:0/2546582342 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff63c0094a0 con 0x7ff64809f240 2023-09-30T22:36:52.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.038+0000 7ff6477fe700 1 --2- 172.21.15.16:0/2546582342 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff64809f650 0x7ff6481a4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:52.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.038+0000 7ff646ffd700 1 --2- 172.21.15.16:0/2546582342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff64809e930 0x7ff6481a3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:36:52.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.038+0000 7ff6467fc700 1 --2- 172.21.15.16:0/2546582342 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff64809f240 0x7ff6481a4140 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7ff63c00e980 tx=0x7ff63c00ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:52.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.038+0000 7ff62f7fe700 1 -- 172.21.15.16:0/2546582342 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63c00eed0 con 0x7ff64809f240 2023-09-30T22:36:52.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.038+0000 7ff62f7fe700 1 -- 172.21.15.16:0/2546582342 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff63c004240 con 0x7ff64809f240 2023-09-30T22:36:52.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.038+0000 7ff64d280700 1 -- 172.21.15.16:0/2546582342 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff640009a70 con 0x7ff64809f240 2023-09-30T22:36:52.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.038+0000 7ff62f7fe700 1 -- 172.21.15.16:0/2546582342 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63c0045b0 con 0x7ff64809f240 2023-09-30T22:36:52.041 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.039+0000 7ff64d280700 1 -- 172.21.15.16:0/2546582342 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff638009550 con 0x7ff64809f240 2023-09-30T22:36:52.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.040+0000 7f64c34a9700 1 Processor -- start 2023-09-30T22:36:52.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.040+0000 7ff64d280700 1 -- 172.21.15.16:0/2546582342 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6481a0b30 con 0x7ff64809f240 2023-09-30T22:36:52.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.040+0000 7f64c34a9700 1 -- start start 2023-09-30T22:36:52.042 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.041+0000 7f64c34a9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f64bc09d6f0 0x7f64bc09dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:52.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.041+0000 7f64c34a9700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f64bc09e000 0x7f64bc130660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:52.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.041+0000 7ff62f7fe700 1 -- 172.21.15.16:0/2546582342 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff63c0036a0 con 0x7ff64809f240 2023-09-30T22:36:52.043 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.041+0000 7ff62f7fe700 1 -- 172.21.15.16:0/2546582342 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7ff63c042220 con 0x7ff64809f240 2023-09-30T22:36:52.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.041+0000 7f64c34a9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f64bc09e410 0x7f64bc130ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:52.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.041+0000 7f64c34a9700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64bc09f520 con 0x7f64bc09e000 2023-09-30T22:36:52.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.041+0000 7f64c34a9700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64bc09f130 con 0x7f64bc09e410 2023-09-30T22:36:52.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.041+0000 7f64c34a9700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f64bc09e930 con 0x7f64bc09d6f0 2023-09-30T22:36:52.044 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.041+0000 7f64c0a44700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f64bc09e000 0x7f64bc130660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:52.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.041+0000 7f64c1245700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f64bc09d6f0 0x7f64bc09dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:52.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.041+0000 7f64c0a44700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f64bc09e000 0x7f64bc130660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46208/0 (socket says 172.21.15.16:46208) 2023-09-30T22:36:52.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.041+0000 7f64c0a44700 1 -- 172.21.15.16:0/1802585243 learned_addr learned my addr 172.21.15.16:0/1802585243 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:52.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.041+0000 7f64c1245700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f64bc09d6f0 0x7f64bc09dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:51202/0 (socket says 172.21.15.16:51202) 2023-09-30T22:36:52.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.042+0000 7f64c1245700 1 -- 172.21.15.16:0/1802585243 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f64bc09e410 msgr2=0x7f64bc130ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:52.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.042+0000 7f64c1245700 1 --2- 172.21.15.16:0/1802585243 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f64bc09e410 0x7f64bc130ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:52.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.042+0000 7f64c1245700 1 -- 172.21.15.16:0/1802585243 >> v2:172.21.15.16:3300/0 conn(0x7f64bc09e000 msgr2=0x7f64bc130660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:52.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.042+0000 7f64c1245700 1 --2- 172.21.15.16:0/1802585243 >> v2:172.21.15.16:3300/0 conn(0x7f64bc09e000 0x7f64bc130660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:52.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.042+0000 7f64c1245700 1 -- 172.21.15.16:0/1802585243 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64bc09b2d0 con 0x7f64bc09d6f0 2023-09-30T22:36:52.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.042+0000 7f64c0a44700 1 --2- 172.21.15.16:0/1802585243 >> v2:172.21.15.16:3300/0 conn(0x7f64bc09e000 0x7f64bc130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:52.045 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.042+0000 7f64c1245700 1 --2- 172.21.15.16:0/1802585243 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f64bc09d6f0 0x7f64bc09dac0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f64b4009dc0 tx=0x7f64b400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1aff4a5e70da3829 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:52.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.046+0000 7f64b3fff700 1 -- 172.21.15.16:0/1802585243 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b400e070 con 0x7f64bc09d6f0 2023-09-30T22:36:52.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.046+0000 7f64b3fff700 1 -- 172.21.15.16:0/1802585243 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64b400bae0 con 0x7f64bc09d6f0 2023-09-30T22:36:52.046 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.046+0000 7f64b3fff700 1 -- 172.21.15.16:0/1802585243 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b400be90 con 0x7f64bc09d6f0 2023-09-30T22:36:52.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.046+0000 7ff62f7fe700 1 -- 172.21.15.16:0/2546582342 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff63c01be10 con 0x7ff64809f240 2023-09-30T22:36:52.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.046+0000 7f64c34a9700 1 -- 172.21.15.16:0/1802585243 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f64bc09d6f0 msgr2=0x7f64bc09dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:52.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.046+0000 7f64c34a9700 1 --2- 172.21.15.16:0/1802585243 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f64bc09d6f0 0x7f64bc09dac0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f64b4009dc0 tx=0x7f64b400b330 comp rx=0 tx=0).stop 2023-09-30T22:36:52.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.047+0000 7f64c34a9700 1 -- 172.21.15.16:0/1802585243 shutdown_connections 2023-09-30T22:36:52.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.047+0000 7f64c34a9700 1 --2- 172.21.15.16:0/1802585243 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f64bc09e410 0x7f64bc130ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:52.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.047+0000 7f64c34a9700 1 --2- 172.21.15.16:0/1802585243 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f64bc09d6f0 0x7f64bc09dac0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:52.047 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.047+0000 7f64c34a9700 1 --2- 172.21.15.16:0/1802585243 >> v2:172.21.15.16:3300/0 conn(0x7f64bc09e000 0x7f64bc130660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:52.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.047+0000 7f64c34a9700 1 -- 172.21.15.16:0/1802585243 >> 172.21.15.16:0/1802585243 conn(0x7f64bc066f50 msgr2=0x7f64bc09f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:52.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.047+0000 7f64c34a9700 1 -- 172.21.15.16:0/1802585243 shutdown_connections 2023-09-30T22:36:52.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.047+0000 7f64c34a9700 1 -- 172.21.15.16:0/1802585243 wait complete. 2023-09-30T22:36:52.048 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.048+0000 7f64c34a9700 1 Processor -- start 2023-09-30T22:36:52.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.048+0000 7f64c34a9700 1 -- start start 2023-09-30T22:36:52.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.049+0000 7f64c34a9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64bc09d6f0 0x7f64bc119480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:52.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.049+0000 7f64c34a9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f64bc09e000 0x7f64bc1199c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:52.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.049+0000 7f64c34a9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f64bc09e410 0x7f64bc119f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:52.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.049+0000 7f64c34a9700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64bc11cbc0 con 0x7f64bc09d6f0 2023-09-30T22:36:52.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.049+0000 7f64c34a9700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64bc11a850 con 0x7f64bc09e000 2023-09-30T22:36:52.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.049+0000 7f64c34a9700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f64bc11cda0 con 0x7f64bc09e410 2023-09-30T22:36:52.051 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.049+0000 7f64c1245700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64bc09d6f0 0x7f64bc119480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:52.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.049+0000 7f64c1245700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64bc09d6f0 0x7f64bc119480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46220/0 (socket says 172.21.15.16:46220) 2023-09-30T22:36:52.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.049+0000 7f64c1245700 1 -- 172.21.15.16:0/20119173 learned_addr learned my addr 172.21.15.16:0/20119173 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:52.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.049+0000 7f64c0a44700 1 --2- 172.21.15.16:0/20119173 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f64bc09e000 0x7f64bc1199c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:52.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.049+0000 7f64c1245700 1 -- 172.21.15.16:0/20119173 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f64bc09e410 msgr2=0x7f64bc119f00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-09-30T22:36:52.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.049+0000 7f64c1245700 1 --2- 172.21.15.16:0/20119173 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f64bc09e410 0x7f64bc119f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:52.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.049+0000 7f64c1245700 1 -- 172.21.15.16:0/20119173 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f64bc09e000 msgr2=0x7f64bc1199c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:52.052 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.049+0000 7f64c1245700 1 --2- 172.21.15.16:0/20119173 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f64bc09e000 0x7f64bc1199c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:52.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.049+0000 7f64c1245700 1 -- 172.21.15.16:0/20119173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64b800bb30 con 0x7f64bc09d6f0 2023-09-30T22:36:52.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.049+0000 7f64c0a44700 1 --2- 172.21.15.16:0/20119173 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f64bc09e000 0x7f64bc1199c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-09-30T22:36:52.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.049+0000 7f64c1245700 1 --2- 172.21.15.16:0/20119173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64bc09d6f0 0x7f64bc119480 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f64b400b370 tx=0x7f64b4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:52.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.050+0000 7f64b1ffb700 1 -- 172.21.15.16:0/20119173 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b4004730 con 0x7f64bc09d6f0 2023-09-30T22:36:52.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.050+0000 7f64b1ffb700 1 -- 172.21.15.16:0/20119173 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64b4020070 con 0x7f64bc09d6f0 2023-09-30T22:36:52.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.050+0000 7f64b1ffb700 1 -- 172.21.15.16:0/20119173 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b4027480 con 0x7f64bc09d6f0 2023-09-30T22:36:52.053 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.050+0000 7f64c34a9700 1 -- 172.21.15.16:0/20119173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64b4009a70 con 0x7f64bc09d6f0 2023-09-30T22:36:52.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.050+0000 7f64c34a9700 1 -- 172.21.15.16:0/20119173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64bc11c800 con 0x7f64bc09d6f0 2023-09-30T22:36:52.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.051+0000 7f64c34a9700 1 -- 172.21.15.16:0/20119173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64bc074ea0 con 0x7f64bc09d6f0 2023-09-30T22:36:52.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.051+0000 7f64b1ffb700 1 -- 172.21.15.16:0/20119173 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f64b4012430 con 0x7f64bc09d6f0 2023-09-30T22:36:52.054 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.051+0000 7f64b1ffb700 1 -- 172.21.15.16:0/20119173 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f64b4022070 con 0x7f64bc09d6f0 2023-09-30T22:36:52.061 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.061+0000 7f64b1ffb700 1 -- 172.21.15.16:0/20119173 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f64b401c400 con 0x7f64bc09d6f0 2023-09-30T22:36:52.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.170+0000 7f11606c5700 1 -- 172.21.15.16:0/260434959 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f115809e700 con 0x7f115812f8b0 2023-09-30T22:36:52.171 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.171+0000 7f114effd700 1 -- 172.21.15.16:0/260434959 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f1148029770 con 0x7f115812f8b0 2023-09-30T22:36:52.171 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:36:52.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.174+0000 7f11606c5700 1 -- 172.21.15.16:0/260434959 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f115812f8b0 msgr2=0x7f11581c5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:52.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.174+0000 7f11606c5700 1 --2- 172.21.15.16:0/260434959 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f115812f8b0 0x7f11581c5690 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f114800bee0 tx=0x7f1148004750 comp rx=0 tx=0).stop 2023-09-30T22:36:52.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.174+0000 7f11606c5700 1 -- 172.21.15.16:0/260434959 shutdown_connections 2023-09-30T22:36:52.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.174+0000 7f11606c5700 1 --2- 172.21.15.16:0/260434959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1158130260 0x7f11581c8170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:52.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.174+0000 7f11606c5700 1 --2- 172.21.15.16:0/260434959 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f115812cf60 0x7f11581c5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:52.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.174+0000 7f11606c5700 1 --2- 172.21.15.16:0/260434959 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f115812f8b0 0x7f11581c5690 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:52.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.174+0000 7f11606c5700 1 -- 172.21.15.16:0/260434959 >> 172.21.15.16:0/260434959 conn(0x7f11580a2aa0 msgr2=0x7f115812dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:52.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.175+0000 7f11606c5700 1 -- 172.21.15.16:0/260434959 shutdown_connections 2023-09-30T22:36:52.175 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.175+0000 7f11606c5700 1 -- 172.21.15.16:0/260434959 wait complete. 2023-09-30T22:36:52.194 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:36:52.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.224+0000 7f6decdbc700 1 -- 172.21.15.16:0/589210122 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f6de811b790 con 0x7f6de806a770 2023-09-30T22:36:52.224 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.224+0000 7f6dd6ffd700 1 -- 172.21.15.16:0/589210122 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6dd8011360 con 0x7f6de806a770 2023-09-30T22:36:52.225 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:36:52.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.227+0000 7f6decdbc700 1 -- 172.21.15.16:0/589210122 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6de806a770 msgr2=0x7f6de8119480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:52.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.227+0000 7f6decdbc700 1 --2- 172.21.15.16:0/589210122 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6de806a770 0x7f6de8119480 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f6dd800c520 tx=0x7f6dd800c830 comp rx=0 tx=0).stop 2023-09-30T22:36:52.228 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.228+0000 7f6decdbc700 1 -- 172.21.15.16:0/589210122 shutdown_connections 2023-09-30T22:36:52.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.228+0000 7f6decdbc700 1 --2- 172.21.15.16:0/589210122 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f6de8136d80 0x7f6de8119f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:52.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.228+0000 7f6decdbc700 1 --2- 172.21.15.16:0/589210122 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f6de806a770 0x7f6de8119480 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:52.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.228+0000 7f6decdbc700 1 --2- 172.21.15.16:0/589210122 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6de806b250 0x7f6de81199c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:52.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.228+0000 7f6decdbc700 1 -- 172.21.15.16:0/589210122 >> 172.21.15.16:0/589210122 conn(0x7f6de8066f50 msgr2=0x7f6de8132780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:52.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.228+0000 7f6decdbc700 1 -- 172.21.15.16:0/589210122 shutdown_connections 2023-09-30T22:36:52.229 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.228+0000 7f6decdbc700 1 -- 172.21.15.16:0/589210122 wait complete. 2023-09-30T22:36:52.242 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:36:52.264 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.264+0000 7ff64d280700 1 -- 172.21.15.16:0/2546582342 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff648074ea0 con 0x7ff64809f240 2023-09-30T22:36:52.265 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.264+0000 7ff62f7fe700 1 -- 172.21.15.16:0/2546582342 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff63c01b3b0 con 0x7ff64809f240 2023-09-30T22:36:52.265 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:36:52.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.266+0000 7ff64d280700 1 -- 172.21.15.16:0/2546582342 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff64809f240 msgr2=0x7ff6481a4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:52.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.266+0000 7ff64d280700 1 --2- 172.21.15.16:0/2546582342 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff64809f240 0x7ff6481a4140 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7ff63c00e980 tx=0x7ff63c00ec90 comp rx=0 tx=0).stop 2023-09-30T22:36:52.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.266+0000 7ff64d280700 1 -- 172.21.15.16:0/2546582342 shutdown_connections 2023-09-30T22:36:52.267 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.266+0000 7ff64d280700 1 --2- 172.21.15.16:0/2546582342 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff64809f650 0x7ff6481a4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:52.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.266+0000 7ff64d280700 1 --2- 172.21.15.16:0/2546582342 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff64809e930 0x7ff6481a3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:52.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.266+0000 7ff64d280700 1 --2- 172.21.15.16:0/2546582342 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff64809f240 0x7ff6481a4140 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:52.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.267+0000 7ff64d280700 1 -- 172.21.15.16:0/2546582342 >> 172.21.15.16:0/2546582342 conn(0x7ff6480a4210 msgr2=0x7ff6481b8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:52.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.267+0000 7ff64d280700 1 -- 172.21.15.16:0/2546582342 shutdown_connections 2023-09-30T22:36:52.268 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.267+0000 7ff64d280700 1 -- 172.21.15.16:0/2546582342 wait complete. 2023-09-30T22:36:52.277 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.277+0000 7f64c34a9700 1 -- 172.21.15.16:0/20119173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f64bc11ae90 con 0x7f64bc09d6f0 2023-09-30T22:36:52.278 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.278+0000 7f64b1ffb700 1 -- 172.21.15.16:0/20119173 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f64b402c070 con 0x7f64bc09d6f0 2023-09-30T22:36:52.278 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:36:52.280 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.280+0000 7f64c34a9700 1 -- 172.21.15.16:0/20119173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64bc09d6f0 msgr2=0x7f64bc119480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:52.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.280+0000 7f64c34a9700 1 --2- 172.21.15.16:0/20119173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64bc09d6f0 0x7f64bc119480 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f64b400b370 tx=0x7f64b4004ad0 comp rx=0 tx=0).stop 2023-09-30T22:36:52.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.280+0000 7f64c34a9700 1 -- 172.21.15.16:0/20119173 shutdown_connections 2023-09-30T22:36:52.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.280+0000 7f64c34a9700 1 --2- 172.21.15.16:0/20119173 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f64bc09e410 0x7f64bc119f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:52.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.280+0000 7f64c34a9700 1 --2- 172.21.15.16:0/20119173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f64bc09d6f0 0x7f64bc119480 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:52.281 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.280+0000 7f64c34a9700 1 --2- 172.21.15.16:0/20119173 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f64bc09e000 0x7f64bc1199c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:52.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.280+0000 7f64c34a9700 1 -- 172.21.15.16:0/20119173 >> 172.21.15.16:0/20119173 conn(0x7f64bc066f50 msgr2=0x7f64bc0a16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:52.282 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:36:52.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.280+0000 7f64c34a9700 1 -- 172.21.15.16:0/20119173 shutdown_connections 2023-09-30T22:36:52.282 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:52.280+0000 7f64c34a9700 1 -- 172.21.15.16:0/20119173 wait complete. 2023-09-30T22:36:52.294 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:36:53.195 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:36:53.243 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-09-30T22:36:53.283 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-09-30T22:36:53.295 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-09-30T22:36:53.352 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.351+0000 7fd8fdd7f700 1 Processor -- start 2023-09-30T22:36:53.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.351+0000 7fd8fdd7f700 1 -- start start 2023-09-30T22:36:53.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.352+0000 7fd8fdd7f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fd8f812cf60 0x7fd8f806f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:53.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.352+0000 7fd8fdd7f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd8f8130430 0x7fd8f8137c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:53.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.352+0000 7fd8fdd7f700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8f812cd90 con 0x7fd8f812f8b0 2023-09-30T22:36:53.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.352+0000 7fd8fdd7f700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8f812d330 con 0x7fd8f812cf60 2023-09-30T22:36:53.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.352+0000 7fd8fdd7f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f8071780 con 0x7fd8f8130430 2023-09-30T22:36:53.354 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.352+0000 7fd8f7fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd8f8130430 0x7fd8f8137c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:53.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.352+0000 7fd8f7fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd8f8130430 0x7fd8f8137c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37870/0 (socket says 172.21.15.16:37870) 2023-09-30T22:36:53.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.352+0000 7fd8f7fff700 1 -- 172.21.15.16:0/758491644 learned_addr learned my addr 172.21.15.16:0/758491644 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:53.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.352+0000 7fd8f77fe700 1 --2- 172.21.15.16:0/758491644 >> v2:172.21.15.16:3300/0 conn(0x7fd8f812cf60 0x7fd8f806f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:53.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.353+0000 7fd8f7fff700 1 -- 172.21.15.16:0/758491644 >> v2:172.21.15.16:3300/0 conn(0x7fd8f812cf60 msgr2=0x7fd8f806f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:53.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.353+0000 7fd8f7fff700 1 --2- 172.21.15.16:0/758491644 >> v2:172.21.15.16:3300/0 conn(0x7fd8f812cf60 0x7fd8f806f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:53.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.353+0000 7fd8f7fff700 1 -- 172.21.15.16:0/758491644 >> v1:172.21.15.16:6789/0 conn(0x7fd8f812f8b0 legacy=0x7fd8f812fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:53.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.353+0000 7fd8f7fff700 1 -- 172.21.15.16:0/758491644 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f8071a40 con 0x7fd8f8130430 2023-09-30T22:36:53.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.353+0000 7fd8f7fff700 1 --2- 172.21.15.16:0/758491644 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd8f8130430 0x7fd8f8137c30 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fd8f000bf60 tx=0x7fd8f000bf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f602f3d41fa3c5c3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:53.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.353+0000 7fd8f67fc700 1 -- 172.21.15.16:0/758491644 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f0003a20 con 0x7fd8f8130430 2023-09-30T22:36:53.355 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.353+0000 7fd8f67fc700 1 -- 172.21.15.16:0/758491644 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8f0003b80 con 0x7fd8f8130430 2023-09-30T22:36:53.357 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.353+0000 7fd8f67fc700 1 -- 172.21.15.16:0/758491644 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f000d9c0 con 0x7fd8f8130430 2023-09-30T22:36:53.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.354+0000 7fd8fdd7f700 1 -- 172.21.15.16:0/758491644 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd8f8130430 msgr2=0x7fd8f8137c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:53.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.354+0000 7fd8fdd7f700 1 --2- 172.21.15.16:0/758491644 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd8f8130430 0x7fd8f8137c30 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fd8f000bf60 tx=0x7fd8f000bf90 comp rx=0 tx=0).stop 2023-09-30T22:36:53.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.354+0000 7fd8fdd7f700 1 -- 172.21.15.16:0/758491644 shutdown_connections 2023-09-30T22:36:53.361 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.354+0000 7fd8fdd7f700 1 --2- 172.21.15.16:0/758491644 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd8f8130430 0x7fd8f8137c30 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:53.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.354+0000 7fd8fdd7f700 1 --2- 172.21.15.16:0/758491644 >> v2:172.21.15.16:3300/0 conn(0x7fd8f812cf60 0x7fd8f806f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:53.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.354+0000 7fd8fdd7f700 1 -- 172.21.15.16:0/758491644 >> 172.21.15.16:0/758491644 conn(0x7fd8f80a2aa0 msgr2=0x7fd8f80a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:53.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.354+0000 7fd8fdd7f700 1 -- 172.21.15.16:0/758491644 shutdown_connections 2023-09-30T22:36:53.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.354+0000 7fd8fdd7f700 1 -- 172.21.15.16:0/758491644 wait complete. 2023-09-30T22:36:53.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.356+0000 7fd8fdd7f700 1 Processor -- start 2023-09-30T22:36:53.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.356+0000 7fd8fdd7f700 1 -- start start 2023-09-30T22:36:53.362 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.357+0000 7fd8fdd7f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd8f812cf60 0x7fd8f81c4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:53.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.357+0000 7fd8fdd7f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd8f812f8b0 0x7fd8f81c4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:53.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.357+0000 7fd8fdd7f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8f8130430 0x7fd8f81c4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:53.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.357+0000 7fd8fdd7f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f809d090 con 0x7fd8f8130430 2023-09-30T22:36:53.363 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.357+0000 7fd8fdd7f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f8129fa0 con 0x7fd8f812f8b0 2023-09-30T22:36:53.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.357+0000 7fd8fdd7f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f809df20 con 0x7fd8f812cf60 2023-09-30T22:36:53.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.357+0000 7fd8f77fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd8f812cf60 0x7fd8f81c4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:53.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.357+0000 7fd8f77fe700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd8f812cf60 0x7fd8f81c4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:51210/0 (socket says 172.21.15.16:51210) 2023-09-30T22:36:53.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.357+0000 7fd8f77fe700 1 -- 172.21.15.16:0/3297782829 learned_addr learned my addr 172.21.15.16:0/3297782829 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:53.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.357+0000 7fd8f7fff700 1 --2- 172.21.15.16:0/3297782829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8f8130430 0x7fd8f81c4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:53.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.357+0000 7fd8f6ffd700 1 --2- 172.21.15.16:0/3297782829 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd8f812f8b0 0x7fd8f81c4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:53.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.357+0000 7fd8f77fe700 1 -- 172.21.15.16:0/3297782829 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd8f812f8b0 msgr2=0x7fd8f81c4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:53.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.357+0000 7fd8f77fe700 1 --2- 172.21.15.16:0/3297782829 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd8f812f8b0 0x7fd8f81c4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:53.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.357+0000 7fd8f77fe700 1 -- 172.21.15.16:0/3297782829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8f8130430 msgr2=0x7fd8f81c4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:53.364 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.357+0000 7fd8f77fe700 1 --2- 172.21.15.16:0/3297782829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8f8130430 0x7fd8f81c4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:53.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.357+0000 7fd8f77fe700 1 -- 172.21.15.16:0/3297782829 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f000bd40 con 0x7fd8f812cf60 2023-09-30T22:36:53.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.358+0000 7fd8f77fe700 1 --2- 172.21.15.16:0/3297782829 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd8f812cf60 0x7fd8f81c4130 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fd8e800e9b0 tx=0x7fd8e800ecc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:53.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.358+0000 7fd8dffff700 1 -- 172.21.15.16:0/3297782829 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e8009710 con 0x7fd8f812cf60 2023-09-30T22:36:53.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.358+0000 7fd8dffff700 1 -- 172.21.15.16:0/3297782829 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8e800ee50 con 0x7fd8f812cf60 2023-09-30T22:36:53.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.358+0000 7fd8dffff700 1 -- 172.21.15.16:0/3297782829 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e800ca50 con 0x7fd8f812cf60 2023-09-30T22:36:53.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.358+0000 7fd8fdd7f700 1 -- 172.21.15.16:0/3297782829 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8f809e880 con 0x7fd8f812cf60 2023-09-30T22:36:53.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.358+0000 7fd8fdd7f700 1 -- 172.21.15.16:0/3297782829 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8f81193d0 con 0x7fd8f812cf60 2023-09-30T22:36:53.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.359+0000 7fd8dffff700 1 -- 172.21.15.16:0/3297782829 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fd8e800cbb0 con 0x7fd8f812cf60 2023-09-30T22:36:53.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.359+0000 7fd8fdd7f700 1 -- 172.21.15.16:0/3297782829 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8f8074ea0 con 0x7fd8f812cf60 2023-09-30T22:36:53.365 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.359+0000 7fd8dffff700 1 -- 172.21.15.16:0/3297782829 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fd8e803f770 con 0x7fd8f812cf60 2023-09-30T22:36:53.366 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.364+0000 7fd8dffff700 1 -- 172.21.15.16:0/3297782829 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fd8e801be10 con 0x7fd8f812cf60 2023-09-30T22:36:53.395 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.395+0000 7f7fdabb9700 1 Processor -- start 2023-09-30T22:36:53.399 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.395+0000 7f7fdabb9700 1 -- start start 2023-09-30T22:36:53.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.396+0000 7f7fdabb9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7fd412cf60 0x7f7fd406f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:53.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.396+0000 7f7fdabb9700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f7fd412f8b0 0x7f7fd412fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:53.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.396+0000 7f7fdabb9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7fd4130260 0x7f7fd4136c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:53.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.396+0000 7f7fdabb9700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fd412d330 con 0x7f7fd412f8b0 2023-09-30T22:36:53.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.396+0000 7f7fdabb9700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd412cd90 con 0x7f7fd412cf60 2023-09-30T22:36:53.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.396+0000 7f7fdabb9700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd41306d0 con 0x7f7fd4130260 2023-09-30T22:36:53.400 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.396+0000 7f7fd9156700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7fd4130260 0x7f7fd4136c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:53.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.396+0000 7f7fd9156700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7fd4130260 0x7f7fd4136c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:51226/0 (socket says 172.21.15.16:51226) 2023-09-30T22:36:53.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.396+0000 7f7fd9156700 1 -- 172.21.15.16:0/3109640268 learned_addr learned my addr 172.21.15.16:0/3109640268 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:53.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.396+0000 7f7fd8955700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7fd412cf60 0x7f7fd406f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:53.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.396+0000 7f7fd8955700 1 -- 172.21.15.16:0/3109640268 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7fd4130260 msgr2=0x7f7fd4136c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:53.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.396+0000 7f7fd8955700 1 --2- 172.21.15.16:0/3109640268 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7fd4130260 0x7f7fd4136c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:53.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.396+0000 7f7fd8955700 1 -- 172.21.15.16:0/3109640268 >> v2:172.21.15.16:3300/0 conn(0x7f7fd412f8b0 msgr2=0x7f7fd412fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:53.401 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.396+0000 7f7fd3fff700 1 --2- 172.21.15.16:0/3109640268 >> v2:172.21.15.16:3300/0 conn(0x7f7fd412f8b0 0x7f7fd412fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:53.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.396+0000 7f7fd8955700 1 --2- 172.21.15.16:0/3109640268 >> v2:172.21.15.16:3300/0 conn(0x7f7fd412f8b0 0x7f7fd412fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:53.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.396+0000 7f7fd8955700 1 -- 172.21.15.16:0/3109640268 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fd4071780 con 0x7f7fd412cf60 2023-09-30T22:36:53.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.396+0000 7f7fd9156700 1 --2- 172.21.15.16:0/3109640268 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7fd4130260 0x7f7fd4136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:53.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.397+0000 7f7fd3fff700 1 --2- 172.21.15.16:0/3109640268 >> v2:172.21.15.16:3300/0 conn(0x7f7fd412f8b0 0x7f7fd412fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:36:53.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.397+0000 7f7fd8955700 1 --2- 172.21.15.16:0/3109640268 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7fd412cf60 0x7f7fd406f5d0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8009840 tx=0x7f7fc8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32933f97781b9f78 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:53.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.397+0000 7f7fd37fe700 1 -- 172.21.15.16:0/3109640268 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc8011830 con 0x7f7fd412cf60 2023-09-30T22:36:53.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.397+0000 7f7fd37fe700 1 -- 172.21.15.16:0/3109640268 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fc8011990 con 0x7f7fd412cf60 2023-09-30T22:36:53.402 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.397+0000 7f7fd37fe700 1 -- 172.21.15.16:0/3109640268 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc8011d20 con 0x7f7fd412cf60 2023-09-30T22:36:53.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.398+0000 7f7fdabb9700 1 -- 172.21.15.16:0/3109640268 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7fd412cf60 msgr2=0x7f7fd406f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:53.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.398+0000 7f7fdabb9700 1 --2- 172.21.15.16:0/3109640268 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7fd412cf60 0x7f7fd406f5d0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8009840 tx=0x7f7fc8009b50 comp rx=0 tx=0).stop 2023-09-30T22:36:53.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.398+0000 7f7fdabb9700 1 -- 172.21.15.16:0/3109640268 shutdown_connections 2023-09-30T22:36:53.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.398+0000 7f7fdabb9700 1 --2- 172.21.15.16:0/3109640268 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7fd412cf60 0x7f7fd406f5d0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:53.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.398+0000 7f7fdabb9700 1 --2- 172.21.15.16:0/3109640268 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7fd4130260 0x7f7fd4136c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:53.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.398+0000 7f7fdabb9700 1 --2- 172.21.15.16:0/3109640268 >> v2:172.21.15.16:3300/0 conn(0x7f7fd412f8b0 0x7f7fd412fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:53.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.398+0000 7f7fdabb9700 1 -- 172.21.15.16:0/3109640268 >> 172.21.15.16:0/3109640268 conn(0x7f7fd40a2aa0 msgr2=0x7f7fd40a2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:53.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.398+0000 7f7fdabb9700 1 -- 172.21.15.16:0/3109640268 shutdown_connections 2023-09-30T22:36:53.403 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.398+0000 7f7fdabb9700 1 -- 172.21.15.16:0/3109640268 wait complete. 2023-09-30T22:36:53.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.400+0000 7f7fdabb9700 1 Processor -- start 2023-09-30T22:36:53.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.400+0000 7f7fdabb9700 1 -- start start 2023-09-30T22:36:53.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.401+0000 7f7fdabb9700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fd412cf60 0x7f7fd409b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:53.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.401+0000 7f7fdabb9700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7fd412f8b0 0x7f7fd409b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:53.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.401+0000 7f7fdabb9700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7fd4130260 0x7f7fd409be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:53.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.401+0000 7f7fdabb9700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd411d0e0 con 0x7f7fd412cf60 2023-09-30T22:36:53.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.401+0000 7f7fdabb9700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd4129fa0 con 0x7f7fd412f8b0 2023-09-30T22:36:53.406 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.401+0000 7f7fdabb9700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd409e6a0 con 0x7f7fd4130260 2023-09-30T22:36:53.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.401+0000 7f7fd8955700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fd412cf60 0x7f7fd409b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:53.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.401+0000 7f7fd8955700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fd412cf60 0x7f7fd409b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46242/0 (socket says 172.21.15.16:46242) 2023-09-30T22:36:53.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.401+0000 7f7fd8955700 1 -- 172.21.15.16:0/2317473171 learned_addr learned my addr 172.21.15.16:0/2317473171 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:53.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.401+0000 7f7fd9156700 1 --2- 172.21.15.16:0/2317473171 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7fd4130260 0x7f7fd409be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:53.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.401+0000 7f7fd8955700 1 -- 172.21.15.16:0/2317473171 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7fd4130260 msgr2=0x7f7fd409be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:53.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.401+0000 7f7fd8955700 1 --2- 172.21.15.16:0/2317473171 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7fd4130260 0x7f7fd409be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:53.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.401+0000 7f7fd8955700 1 -- 172.21.15.16:0/2317473171 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7fd412f8b0 msgr2=0x7f7fd409b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:53.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.401+0000 7f7fd8955700 1 --2- 172.21.15.16:0/2317473171 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7fd412f8b0 0x7f7fd409b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:53.407 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.401+0000 7f7fd8955700 1 -- 172.21.15.16:0/2317473171 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fcc00bd40 con 0x7f7fd412cf60 2023-09-30T22:36:53.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.401+0000 7f7fd8955700 1 --2- 172.21.15.16:0/2317473171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fd412cf60 0x7f7fd409b3f0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8000c00 tx=0x7f7fc8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:53.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.402+0000 7f7fd17fa700 1 -- 172.21.15.16:0/2317473171 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc8004590 con 0x7f7fd412cf60 2023-09-30T22:36:53.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.402+0000 7f7fd17fa700 1 -- 172.21.15.16:0/2317473171 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fc801f070 con 0x7f7fd412cf60 2023-09-30T22:36:53.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.402+0000 7f7fdabb9700 1 -- 172.21.15.16:0/2317473171 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fc80094a0 con 0x7f7fd412cf60 2023-09-30T22:36:53.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.402+0000 7f7fd17fa700 1 -- 172.21.15.16:0/2317473171 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc80038e0 con 0x7f7fd412cf60 2023-09-30T22:36:53.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.402+0000 7f7fdabb9700 1 -- 172.21.15.16:0/2317473171 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fd409e970 con 0x7f7fd412cf60 2023-09-30T22:36:53.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.403+0000 7f7fd17fa700 1 -- 172.21.15.16:0/2317473171 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f7fc8003a40 con 0x7f7fd412cf60 2023-09-30T22:36:53.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.403+0000 7f7fdabb9700 1 -- 172.21.15.16:0/2317473171 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fd404ea50 con 0x7f7fd412cf60 2023-09-30T22:36:53.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.403+0000 7f7fd17fa700 1 -- 172.21.15.16:0/2317473171 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f7fc8021070 con 0x7f7fd412cf60 2023-09-30T22:36:53.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.408+0000 7f7fd17fa700 1 -- 172.21.15.16:0/2317473171 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f7fc8026480 con 0x7f7fd412cf60 2023-09-30T22:36:53.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.435+0000 7f5856927700 1 Processor -- start 2023-09-30T22:36:53.435 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.435+0000 7f5856927700 1 -- start start 2023-09-30T22:36:53.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.435+0000 7f5856927700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f585009e1d0 0x7f5850138c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:53.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.435+0000 7f5856927700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5850139170 0x7f585013b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:53.436 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.435+0000 7f5856927700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f585009b120 con 0x7f585009d6f0 2023-09-30T22:36:53.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.435+0000 7f5856927700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f585006b460 con 0x7f585009e1d0 2023-09-30T22:36:53.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.435+0000 7f5856927700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f585009a7e0 con 0x7f5850139170 2023-09-30T22:36:53.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.436+0000 7f5854ec4700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5850139170 0x7f585013b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:53.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.436+0000 7f584ffff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f585009d6f0 0x7f585009dac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:48980/0 (socket says 172.21.15.16:48980) 2023-09-30T22:36:53.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.436+0000 7f584ffff700 1 -- 172.21.15.16:0/3921187360 learned_addr learned my addr 172.21.15.16:0/3921187360 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:36:53.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.436+0000 7f584f7fe700 1 --2- 172.21.15.16:0/3921187360 >> v2:172.21.15.16:3300/0 conn(0x7f585009e1d0 0x7f5850138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:53.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.436+0000 7f5854ec4700 1 -- 172.21.15.16:0/3921187360 >> v2:172.21.15.16:3300/0 conn(0x7f585009e1d0 msgr2=0x7f5850138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:53.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.436+0000 7f5854ec4700 1 --2- 172.21.15.16:0/3921187360 >> v2:172.21.15.16:3300/0 conn(0x7f585009e1d0 0x7f5850138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:53.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.436+0000 7f5854ec4700 1 -- 172.21.15.16:0/3921187360 >> v1:172.21.15.16:6789/0 conn(0x7f585009d6f0 legacy=0x7f585009dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:53.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.436+0000 7f5854ec4700 1 -- 172.21.15.16:0/3921187360 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f585009a500 con 0x7f5850139170 2023-09-30T22:36:53.437 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.436+0000 7f5854ec4700 1 --2- 172.21.15.16:0/3921187360 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5850139170 0x7f585013b550 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f5848009dc0 tx=0x7f584800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c59ae232e0889547 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:53.438 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.436+0000 7f584effd700 1 -- 172.21.15.16:0/3921187360 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584800e070 con 0x7f5850139170 2023-09-30T22:36:53.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.437+0000 7f584effd700 1 -- 172.21.15.16:0/3921187360 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f584800bae0 con 0x7f5850139170 2023-09-30T22:36:53.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.437+0000 7f584effd700 1 -- 172.21.15.16:0/3921187360 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584800be90 con 0x7f5850139170 2023-09-30T22:36:53.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.437+0000 7f5856927700 1 -- 172.21.15.16:0/3921187360 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5850139170 msgr2=0x7f585013b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:53.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.437+0000 7f5856927700 1 --2- 172.21.15.16:0/3921187360 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5850139170 0x7f585013b550 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f5848009dc0 tx=0x7f584800b330 comp rx=0 tx=0).stop 2023-09-30T22:36:53.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.437+0000 7f5856927700 1 -- 172.21.15.16:0/3921187360 shutdown_connections 2023-09-30T22:36:53.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.437+0000 7f5856927700 1 --2- 172.21.15.16:0/3921187360 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5850139170 0x7f585013b550 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:53.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.437+0000 7f5856927700 1 --2- 172.21.15.16:0/3921187360 >> v2:172.21.15.16:3300/0 conn(0x7f585009e1d0 0x7f5850138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:53.439 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.437+0000 7f5856927700 1 -- 172.21.15.16:0/3921187360 >> 172.21.15.16:0/3921187360 conn(0x7f5850066f50 msgr2=0x7f585009f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:53.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.437+0000 7f5856927700 1 -- 172.21.15.16:0/3921187360 shutdown_connections 2023-09-30T22:36:53.440 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.437+0000 7f5856927700 1 -- 172.21.15.16:0/3921187360 wait complete. 2023-09-30T22:36:53.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.439+0000 7f5856927700 1 Processor -- start 2023-09-30T22:36:53.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.439+0000 7f5856927700 1 -- start start 2023-09-30T22:36:53.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.440+0000 7f5856927700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f585009d6f0 0x7f585011e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:53.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.440+0000 7f5856927700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f585009e1d0 0x7f585011ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:53.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.440+0000 7f5856927700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5850139170 0x7f585011f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:53.441 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.440+0000 7f5856927700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5850122140 con 0x7f585009e1d0 2023-09-30T22:36:53.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.440+0000 7f5856927700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f585009c710 con 0x7f5850139170 2023-09-30T22:36:53.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.440+0000 7f5856927700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5850122340 con 0x7f585009d6f0 2023-09-30T22:36:53.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.440+0000 7f584ffff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f585009d6f0 0x7f585011e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:53.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.440+0000 7f5854ec4700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5850139170 0x7f585011f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:53.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.440+0000 7f584ffff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f585009d6f0 0x7f585011e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:51236/0 (socket says 172.21.15.16:51236) 2023-09-30T22:36:53.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.440+0000 7f584ffff700 1 -- 172.21.15.16:0/1014960652 learned_addr learned my addr 172.21.15.16:0/1014960652 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:53.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.440+0000 7f584f7fe700 1 --2- 172.21.15.16:0/1014960652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f585009e1d0 0x7f585011ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:53.442 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.441+0000 7f5854ec4700 1 -- 172.21.15.16:0/1014960652 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f585009d6f0 msgr2=0x7f585011e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:53.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.441+0000 7f5854ec4700 1 --2- 172.21.15.16:0/1014960652 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f585009d6f0 0x7f585011e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:53.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.441+0000 7f5854ec4700 1 -- 172.21.15.16:0/1014960652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f585009e1d0 msgr2=0x7f585011ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:53.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.441+0000 7f5854ec4700 1 --2- 172.21.15.16:0/1014960652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f585009e1d0 0x7f585011ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:53.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.441+0000 7f5854ec4700 1 -- 172.21.15.16:0/1014960652 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5848009a70 con 0x7f5850139170 2023-09-30T22:36:53.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.441+0000 7f584ffff700 1 --2- 172.21.15.16:0/1014960652 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f585009d6f0 0x7f585011e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:53.443 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.441+0000 7f5854ec4700 1 --2- 172.21.15.16:0/1014960652 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5850139170 0x7f585011f320 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f5848016000 tx=0x7f5848004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:53.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.441+0000 7f584cff9700 1 -- 172.21.15.16:0/1014960652 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58480045d0 con 0x7f5850139170 2023-09-30T22:36:53.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.441+0000 7f584cff9700 1 -- 172.21.15.16:0/1014960652 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5848020070 con 0x7f5850139170 2023-09-30T22:36:53.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.441+0000 7f584cff9700 1 -- 172.21.15.16:0/1014960652 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5848027480 con 0x7f5850139170 2023-09-30T22:36:53.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.441+0000 7f5856927700 1 -- 172.21.15.16:0/1014960652 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58500720a0 con 0x7f5850139170 2023-09-30T22:36:53.444 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.441+0000 7f5856927700 1 -- 172.21.15.16:0/1014960652 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58501204b0 con 0x7f5850139170 2023-09-30T22:36:53.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.442+0000 7f5856927700 1 -- 172.21.15.16:0/1014960652 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5850074ea0 con 0x7f5850139170 2023-09-30T22:36:53.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.443+0000 7f584cff9700 1 -- 172.21.15.16:0/1014960652 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f58480275e0 con 0x7f5850139170 2023-09-30T22:36:53.445 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.443+0000 7f584cff9700 1 -- 172.21.15.16:0/1014960652 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f5848022070 con 0x7f5850139170 2023-09-30T22:36:53.452 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.452+0000 7f584cff9700 1 -- 172.21.15.16:0/1014960652 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5848027890 con 0x7f5850139170 2023-09-30T22:36:53.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.482+0000 7f82e302c700 1 Processor -- start 2023-09-30T22:36:53.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.482+0000 7f82e302c700 1 -- start start 2023-09-30T22:36:53.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.483+0000 7f82e302c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f82dc133a80 0x7f82dc133e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:53.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.483+0000 7f82e302c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f82dc130a70 0x7f82dc13b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:53.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.483+0000 7f82e302c700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82dc06bc00 con 0x7f82dc12ff20 2023-09-30T22:36:53.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.483+0000 7f82e302c700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82dc06c570 con 0x7f82dc133a80 2023-09-30T22:36:53.484 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.483+0000 7f82e302c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc0a0040 con 0x7f82dc130a70 2023-09-30T22:36:53.485 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.483+0000 7f82e0dc8700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f82dc133a80 0x7f82dc133e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:53.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.483+0000 7f82e0dc8700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f82dc133a80 0x7f82dc133e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46268/0 (socket says 172.21.15.16:46268) 2023-09-30T22:36:53.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.483+0000 7f82e0dc8700 1 -- 172.21.15.16:0/2310217890 learned_addr learned my addr 172.21.15.16:0/2310217890 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:53.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.483+0000 7f82e15c9700 1 --2- 172.21.15.16:0/2310217890 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f82dc130a70 0x7f82dc13b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:53.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.484+0000 7f82e0dc8700 1 -- 172.21.15.16:0/2310217890 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f82dc130a70 msgr2=0x7f82dc13b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:53.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.484+0000 7f82e0dc8700 1 --2- 172.21.15.16:0/2310217890 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f82dc130a70 0x7f82dc13b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:53.486 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.484+0000 7f82e0dc8700 1 -- 172.21.15.16:0/2310217890 >> v1:172.21.15.16:6789/0 conn(0x7f82dc12ff20 legacy=0x7f82dc130360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:53.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.484+0000 7f82e0dc8700 1 -- 172.21.15.16:0/2310217890 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82dc133860 con 0x7f82dc133a80 2023-09-30T22:36:53.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.484+0000 7f82e0dc8700 1 --2- 172.21.15.16:0/2310217890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82dc133a80 0x7f82dc133e50 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f82d00098f0 tx=0x7f82d0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b65b32466119641f server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:53.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.484+0000 7f82db7fe700 1 -- 172.21.15.16:0/2310217890 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d0011830 con 0x7f82dc133a80 2023-09-30T22:36:53.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.484+0000 7f82db7fe700 1 -- 172.21.15.16:0/2310217890 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82d0011990 con 0x7f82dc133a80 2023-09-30T22:36:53.487 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.484+0000 7f82db7fe700 1 -- 172.21.15.16:0/2310217890 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d0011d20 con 0x7f82dc133a80 2023-09-30T22:36:53.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.485+0000 7f82e302c700 1 -- 172.21.15.16:0/2310217890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82dc133a80 msgr2=0x7f82dc133e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:53.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.485+0000 7f82e302c700 1 --2- 172.21.15.16:0/2310217890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82dc133a80 0x7f82dc133e50 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f82d00098f0 tx=0x7f82d0009c00 comp rx=0 tx=0).stop 2023-09-30T22:36:53.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.485+0000 7f82e302c700 1 -- 172.21.15.16:0/2310217890 shutdown_connections 2023-09-30T22:36:53.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.485+0000 7f82e302c700 1 --2- 172.21.15.16:0/2310217890 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f82dc130a70 0x7f82dc13b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:53.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.485+0000 7f82e302c700 1 --2- 172.21.15.16:0/2310217890 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82dc133a80 0x7f82dc133e50 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:53.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.485+0000 7f82e302c700 1 -- 172.21.15.16:0/2310217890 >> 172.21.15.16:0/2310217890 conn(0x7f82dc0a4210 msgr2=0x7f82dc0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:53.488 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.485+0000 7f82e302c700 1 -- 172.21.15.16:0/2310217890 shutdown_connections 2023-09-30T22:36:53.490 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.485+0000 7f82e302c700 1 -- 172.21.15.16:0/2310217890 wait complete. 2023-09-30T22:36:53.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.486+0000 7f82e302c700 1 Processor -- start 2023-09-30T22:36:53.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.486+0000 7f82e302c700 1 -- start start 2023-09-30T22:36:53.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.487+0000 7f82e302c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f82dc12ff20 0x7f82dc09c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:53.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.487+0000 7f82e302c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f82dc130a70 0x7f82dc09c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:53.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.487+0000 7f82e302c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82dc133a80 0x7f82dc09cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:53.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.487+0000 7f82e302c700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc11fcc0 con 0x7f82dc133a80 2023-09-30T22:36:53.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.487+0000 7f82e302c700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc12d260 con 0x7f82dc130a70 2023-09-30T22:36:53.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.487+0000 7f82e302c700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f82dc09e2b0 con 0x7f82dc12ff20 2023-09-30T22:36:53.492 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.487+0000 7f82e0dc8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f82dc12ff20 0x7f82dc09c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:53.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.487+0000 7f82e0dc8700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f82dc12ff20 0x7f82dc09c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:51244/0 (socket says 172.21.15.16:51244) 2023-09-30T22:36:53.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.487+0000 7f82e0dc8700 1 -- 172.21.15.16:0/1198567767 learned_addr learned my addr 172.21.15.16:0/1198567767 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:53.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.487+0000 7f82dbfff700 1 --2- 172.21.15.16:0/1198567767 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f82dc130a70 0x7f82dc09c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:53.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.487+0000 7f82e15c9700 1 --2- 172.21.15.16:0/1198567767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82dc133a80 0x7f82dc09cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:53.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.487+0000 7f82dbfff700 1 -- 172.21.15.16:0/1198567767 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f82dc12ff20 msgr2=0x7f82dc09c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:53.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.487+0000 7f82dbfff700 1 --2- 172.21.15.16:0/1198567767 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f82dc12ff20 0x7f82dc09c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:53.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.487+0000 7f82dbfff700 1 -- 172.21.15.16:0/1198567767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82dc133a80 msgr2=0x7f82dc09cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:53.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.487+0000 7f82dbfff700 1 --2- 172.21.15.16:0/1198567767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82dc133a80 0x7f82dc09cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:53.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.487+0000 7f82dbfff700 1 -- 172.21.15.16:0/1198567767 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d0009550 con 0x7f82dc130a70 2023-09-30T22:36:53.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.487+0000 7f82e15c9700 1 --2- 172.21.15.16:0/1198567767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82dc133a80 0x7f82dc09cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-09-30T22:36:53.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.488+0000 7f82dbfff700 1 --2- 172.21.15.16:0/1198567767 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f82dc130a70 0x7f82dc09c9c0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f82c8000f60 tx=0x7f82c800c730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:53.493 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.488+0000 7f82d97fa700 1 -- 172.21.15.16:0/1198567767 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c80052d0 con 0x7f82dc130a70 2023-09-30T22:36:53.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.488+0000 7f82d97fa700 1 -- 172.21.15.16:0/1198567767 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82c8005430 con 0x7f82dc130a70 2023-09-30T22:36:53.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.488+0000 7f82e302c700 1 -- 172.21.15.16:0/1198567767 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82dc1209f0 con 0x7f82dc130a70 2023-09-30T22:36:53.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.488+0000 7f82d97fa700 1 -- 172.21.15.16:0/1198567767 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c80038c0 con 0x7f82dc130a70 2023-09-30T22:36:53.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.488+0000 7f82e302c700 1 -- 172.21.15.16:0/1198567767 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82dc09df00 con 0x7f82dc130a70 2023-09-30T22:36:53.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.489+0000 7f82d97fa700 1 -- 172.21.15.16:0/1198567767 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f82c8003a20 con 0x7f82dc130a70 2023-09-30T22:36:53.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.489+0000 7f82e302c700 1 -- 172.21.15.16:0/1198567767 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82dc09d8a0 con 0x7f82dc130a70 2023-09-30T22:36:53.494 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.489+0000 7f82d97fa700 1 -- 172.21.15.16:0/1198567767 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f82c803ebc0 con 0x7f82dc130a70 2023-09-30T22:36:53.498 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.498+0000 7f82d97fa700 1 -- 172.21.15.16:0/1198567767 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f82c8018260 con 0x7f82dc130a70 2023-09-30T22:36:53.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.582+0000 7fd8fdd7f700 1 -- 172.21.15.16:0/3297782829 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd8f8067dd0 con 0x7fd8f812cf60 2023-09-30T22:36:53.582 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.582+0000 7fd8dffff700 1 -- 172.21.15.16:0/3297782829 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd8e8021020 con 0x7fd8f812cf60 2023-09-30T22:36:53.583 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:36:53.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.584+0000 7fd8fdd7f700 1 -- 172.21.15.16:0/3297782829 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd8f812cf60 msgr2=0x7fd8f81c4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:53.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.584+0000 7fd8fdd7f700 1 --2- 172.21.15.16:0/3297782829 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd8f812cf60 0x7fd8f81c4130 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fd8e800e9b0 tx=0x7fd8e800ecc0 comp rx=0 tx=0).stop 2023-09-30T22:36:53.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.584+0000 7fd8fdd7f700 1 -- 172.21.15.16:0/3297782829 shutdown_connections 2023-09-30T22:36:53.585 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.584+0000 7fd8fdd7f700 1 --2- 172.21.15.16:0/3297782829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd8f8130430 0x7fd8f81c4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:53.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.584+0000 7fd8fdd7f700 1 --2- 172.21.15.16:0/3297782829 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fd8f812cf60 0x7fd8f81c4130 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:53.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.584+0000 7fd8fdd7f700 1 --2- 172.21.15.16:0/3297782829 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fd8f812f8b0 0x7fd8f81c4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:53.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.584+0000 7fd8fdd7f700 1 -- 172.21.15.16:0/3297782829 >> 172.21.15.16:0/3297782829 conn(0x7fd8f80a2aa0 msgr2=0x7fd8f81b7990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:53.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.585+0000 7fd8fdd7f700 1 -- 172.21.15.16:0/3297782829 shutdown_connections 2023-09-30T22:36:53.586 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.585+0000 7fd8fdd7f700 1 -- 172.21.15.16:0/3297782829 wait complete. 2023-09-30T22:36:53.600 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:36:53.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.627+0000 7f7fdabb9700 1 -- 172.21.15.16:0/2317473171 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7fd4074ea0 con 0x7f7fd412cf60 2023-09-30T22:36:53.628 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.628+0000 7f7fd17fa700 1 -- 172.21.15.16:0/2317473171 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7fc802b070 con 0x7f7fd412cf60 2023-09-30T22:36:53.628 INFO:teuthology.orchestra.run.smithi016.stdout:124554051586 2023-09-30T22:36:53.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.631+0000 7f7fdabb9700 1 -- 172.21.15.16:0/2317473171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fd412cf60 msgr2=0x7f7fd409b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:53.631 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.631+0000 7f7fdabb9700 1 --2- 172.21.15.16:0/2317473171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fd412cf60 0x7f7fd409b3f0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f7fc8000c00 tx=0x7f7fc8004750 comp rx=0 tx=0).stop 2023-09-30T22:36:53.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.631+0000 7f7fdabb9700 1 -- 172.21.15.16:0/2317473171 shutdown_connections 2023-09-30T22:36:53.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.631+0000 7f7fdabb9700 1 --2- 172.21.15.16:0/2317473171 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f7fd4130260 0x7f7fd409be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:53.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.631+0000 7f7fdabb9700 1 --2- 172.21.15.16:0/2317473171 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fd412cf60 0x7f7fd409b3f0 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:53.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.631+0000 7f7fdabb9700 1 --2- 172.21.15.16:0/2317473171 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f7fd412f8b0 0x7f7fd409b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:53.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.631+0000 7f7fdabb9700 1 -- 172.21.15.16:0/2317473171 >> 172.21.15.16:0/2317473171 conn(0x7f7fd40a2aa0 msgr2=0x7f7fd41b7a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:53.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.631+0000 7f7fdabb9700 1 -- 172.21.15.16:0/2317473171 shutdown_connections 2023-09-30T22:36:53.632 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.631+0000 7f7fdabb9700 1 -- 172.21.15.16:0/2317473171 wait complete. 2023-09-30T22:36:53.650 INFO:tasks.ceph.ceph_manager.ceph:need seq 124554051588 got 124554051586 for osd.1 2023-09-30T22:36:53.681 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.681+0000 7f5856927700 1 -- 172.21.15.16:0/1014960652 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f585009f420 con 0x7f5850139170 2023-09-30T22:36:53.682 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.681+0000 7f584cff9700 1 -- 172.21.15.16:0/1014960652 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f584801b400 con 0x7f5850139170 2023-09-30T22:36:53.682 INFO:teuthology.orchestra.run.smithi016.stdout:120259084290 2023-09-30T22:36:53.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.683+0000 7f5856927700 1 -- 172.21.15.16:0/1014960652 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5850139170 msgr2=0x7f585011f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:53.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.683+0000 7f5856927700 1 --2- 172.21.15.16:0/1014960652 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5850139170 0x7f585011f320 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f5848016000 tx=0x7f5848004ad0 comp rx=0 tx=0).stop 2023-09-30T22:36:53.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.684+0000 7f5856927700 1 -- 172.21.15.16:0/1014960652 shutdown_connections 2023-09-30T22:36:53.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.684+0000 7f5856927700 1 --2- 172.21.15.16:0/1014960652 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f5850139170 0x7f585011f320 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:53.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.684+0000 7f5856927700 1 --2- 172.21.15.16:0/1014960652 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f585009d6f0 0x7f585011e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:53.684 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.684+0000 7f5856927700 1 --2- 172.21.15.16:0/1014960652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f585009e1d0 0x7f585011ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:53.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.684+0000 7f5856927700 1 -- 172.21.15.16:0/1014960652 >> 172.21.15.16:0/1014960652 conn(0x7f5850066f50 msgr2=0x7f5850136c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:53.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.684+0000 7f5856927700 1 -- 172.21.15.16:0/1014960652 shutdown_connections 2023-09-30T22:36:53.685 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.684+0000 7f5856927700 1 -- 172.21.15.16:0/1014960652 wait complete. 2023-09-30T22:36:53.697 INFO:tasks.ceph.ceph_manager.ceph:need seq 120259084292 got 120259084290 for osd.0 2023-09-30T22:36:53.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.712+0000 7f82e302c700 1 -- 172.21.15.16:0/1198567767 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f82dc074ea0 con 0x7f82dc130a70 2023-09-30T22:36:53.712 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.712+0000 7f82d97fa700 1 -- 172.21.15.16:0/1198567767 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f82c8017070 con 0x7f82dc130a70 2023-09-30T22:36:53.712 INFO:teuthology.orchestra.run.smithi016.stdout:137438953474 2023-09-30T22:36:53.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.714+0000 7f82e302c700 1 -- 172.21.15.16:0/1198567767 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f82dc130a70 msgr2=0x7f82dc09c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:53.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.714+0000 7f82e302c700 1 --2- 172.21.15.16:0/1198567767 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f82dc130a70 0x7f82dc09c9c0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f82c8000f60 tx=0x7f82c800c730 comp rx=0 tx=0).stop 2023-09-30T22:36:53.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.714+0000 7f82e302c700 1 -- 172.21.15.16:0/1198567767 shutdown_connections 2023-09-30T22:36:53.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.714+0000 7f82e302c700 1 --2- 172.21.15.16:0/1198567767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82dc133a80 0x7f82dc09cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:53.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.714+0000 7f82e302c700 1 --2- 172.21.15.16:0/1198567767 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f82dc12ff20 0x7f82dc09c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:53.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.714+0000 7f82e302c700 1 --2- 172.21.15.16:0/1198567767 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f82dc130a70 0x7f82dc09c9c0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:53.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.714+0000 7f82e302c700 1 -- 172.21.15.16:0/1198567767 >> 172.21.15.16:0/1198567767 conn(0x7f82dc0a4210 msgr2=0x7f82dc09e930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:53.715 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.715+0000 7f82e302c700 1 -- 172.21.15.16:0/1198567767 shutdown_connections 2023-09-30T22:36:53.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:53.715+0000 7f82e302c700 1 -- 172.21.15.16:0/1198567767 wait complete. 2023-09-30T22:36:53.728 INFO:tasks.ceph.ceph_manager.ceph:need seq 137438953475 got 137438953474 for osd.3 2023-09-30T22:36:54.601 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-09-30T22:36:54.651 INFO:tasks.ceph:ignoring flush pg stats error, probably testing upgrade: timed out waiting for mon to be updated with osd.1: 124554051586 < 124554051588 2023-09-30T22:36:54.651 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-09-30T22:36:54.652 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-30T22:36:54.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.749+0000 7f83cb68f700 1 Processor -- start 2023-09-30T22:36:54.749 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.749+0000 7f83cb68f700 1 -- start start 2023-09-30T22:36:54.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.749+0000 7f83cb68f700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f83c40a0a30 0x7f83c40a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:54.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.749+0000 7f83cb68f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f83c409d630 0x7f83c40a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:54.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.749+0000 7f83cb68f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83c4133930 0x7f83c4133db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:54.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.749+0000 7f83cb68f700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83c40a2a70 con 0x7f83c40a0a30 2023-09-30T22:36:54.752 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.749+0000 7f83cb68f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83c40a01d0 con 0x7f83c4133930 2023-09-30T22:36:54.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.749+0000 7f83cb68f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f83c409e2b0 con 0x7f83c409d630 2023-09-30T22:36:54.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.750+0000 7f83c9c2c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83c4133930 0x7f83c4133db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:54.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.750+0000 7f83c8c2a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f83c409d630 0x7f83c40a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:54.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.750+0000 7f83c9c2c700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83c4133930 0x7f83c4133db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37928/0 (socket says 172.21.15.16:37928) 2023-09-30T22:36:54.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.750+0000 7f83c9c2c700 1 -- 172.21.15.16:0/3699353709 learned_addr learned my addr 172.21.15.16:0/3699353709 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:54.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.750+0000 7f83c942b700 1 --2- 172.21.15.16:0/3699353709 >> v2:172.21.15.16:3300/0 conn(0x7f83c40a0a30 0x7f83c40a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:54.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.750+0000 7f83c9c2c700 1 -- 172.21.15.16:0/3699353709 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f83c409d630 msgr2=0x7f83c40a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:54.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.750+0000 7f83c9c2c700 1 --2- 172.21.15.16:0/3699353709 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f83c409d630 0x7f83c40a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:54.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.750+0000 7f83c9c2c700 1 -- 172.21.15.16:0/3699353709 >> v2:172.21.15.16:3300/0 conn(0x7f83c40a0a30 msgr2=0x7f83c40a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:54.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.750+0000 7f83c9c2c700 1 --2- 172.21.15.16:0/3699353709 >> v2:172.21.15.16:3300/0 conn(0x7f83c40a0a30 0x7f83c40a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:54.753 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.750+0000 7f83c9c2c700 1 -- 172.21.15.16:0/3699353709 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83c409dc50 con 0x7f83c4133930 2023-09-30T22:36:54.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.750+0000 7f83c942b700 1 --2- 172.21.15.16:0/3699353709 >> v2:172.21.15.16:3300/0 conn(0x7f83c40a0a30 0x7f83c40a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:54.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.750+0000 7f83c9c2c700 1 --2- 172.21.15.16:0/3699353709 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83c4133930 0x7f83c4133db0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f83bc009e50 tx=0x7f83bc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8b2a3c74d763f68 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:54.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.750+0000 7f83c3fff700 1 -- 172.21.15.16:0/3699353709 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83bc00e070 con 0x7f83c4133930 2023-09-30T22:36:54.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.750+0000 7f83c3fff700 1 -- 172.21.15.16:0/3699353709 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83bc00b910 con 0x7f83c4133930 2023-09-30T22:36:54.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.750+0000 7f83c3fff700 1 -- 172.21.15.16:0/3699353709 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83bc00bcc0 con 0x7f83c4133930 2023-09-30T22:36:54.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.751+0000 7f83cb68f700 1 -- 172.21.15.16:0/3699353709 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83c4133930 msgr2=0x7f83c4133db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:54.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.751+0000 7f83cb68f700 1 --2- 172.21.15.16:0/3699353709 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83c4133930 0x7f83c4133db0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f83bc009e50 tx=0x7f83bc00b330 comp rx=0 tx=0).stop 2023-09-30T22:36:54.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.751+0000 7f83cb68f700 1 -- 172.21.15.16:0/3699353709 shutdown_connections 2023-09-30T22:36:54.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.751+0000 7f83cb68f700 1 --2- 172.21.15.16:0/3699353709 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83c4133930 0x7f83c4133db0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:54.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.751+0000 7f83cb68f700 1 --2- 172.21.15.16:0/3699353709 >> v2:172.21.15.16:3300/0 conn(0x7f83c40a0a30 0x7f83c40a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:54.754 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.751+0000 7f83cb68f700 1 --2- 172.21.15.16:0/3699353709 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f83c409d630 0x7f83c40a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:54.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.751+0000 7f83cb68f700 1 -- 172.21.15.16:0/3699353709 >> 172.21.15.16:0/3699353709 conn(0x7f83c40a3050 msgr2=0x7f83c40a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:54.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.751+0000 7f83cb68f700 1 -- 172.21.15.16:0/3699353709 shutdown_connections 2023-09-30T22:36:54.755 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.751+0000 7f83cb68f700 1 -- 172.21.15.16:0/3699353709 wait complete. 2023-09-30T22:36:54.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.752+0000 7f83cb68f700 1 Processor -- start 2023-09-30T22:36:54.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.752+0000 7f83cb68f700 1 -- start start 2023-09-30T22:36:54.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.753+0000 7f83cb68f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83c409d630 0x7f83c4121340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:54.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.753+0000 7f83cb68f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f83c40a0a30 0x7f83c4121880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:54.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.753+0000 7f83cb68f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83c4133930 0x7f83c4121dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:54.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.753+0000 7f83cb68f700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83c4124960 con 0x7f83c4133930 2023-09-30T22:36:54.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.753+0000 7f83cb68f700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83c40a4030 con 0x7f83c409d630 2023-09-30T22:36:54.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.753+0000 7f83cb68f700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f83c4122cd0 con 0x7f83c40a0a30 2023-09-30T22:36:54.757 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.753+0000 7f83c942b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83c409d630 0x7f83c4121340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:54.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.753+0000 7f83c9c2c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83c4133930 0x7f83c4121dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:54.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.753+0000 7f83c9c2c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83c4133930 0x7f83c4121dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46292/0 (socket says 172.21.15.16:46292) 2023-09-30T22:36:54.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.753+0000 7f83c942b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83c409d630 0x7f83c4121340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:37932/0 (socket says 172.21.15.16:37932) 2023-09-30T22:36:54.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.753+0000 7f83c9c2c700 1 -- 172.21.15.16:0/1912715694 learned_addr learned my addr 172.21.15.16:0/1912715694 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:54.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.753+0000 7f83c942b700 1 -- 172.21.15.16:0/1912715694 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f83c40a0a30 msgr2=0x7f83c4121880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:54.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.753+0000 7f83c942b700 1 --2- 172.21.15.16:0/1912715694 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f83c40a0a30 0x7f83c4121880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:54.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.753+0000 7f83c942b700 1 -- 172.21.15.16:0/1912715694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83c4133930 msgr2=0x7f83c4121dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:54.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.753+0000 7f83c942b700 1 --2- 172.21.15.16:0/1912715694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83c4133930 0x7f83c4121dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:54.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.753+0000 7f83c942b700 1 -- 172.21.15.16:0/1912715694 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b40094a0 con 0x7f83c409d630 2023-09-30T22:36:54.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.753+0000 7f83c9c2c700 1 --2- 172.21.15.16:0/1912715694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83c4133930 0x7f83c4121dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:54.758 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.753+0000 7f83c942b700 1 --2- 172.21.15.16:0/1912715694 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83c409d630 0x7f83c4121340 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f83b400e9b0 tx=0x7f83b400ecc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:54.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.754+0000 7f83c1ffb700 1 -- 172.21.15.16:0/1912715694 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b400c850 con 0x7f83c409d630 2023-09-30T22:36:54.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.754+0000 7f83c1ffb700 1 -- 172.21.15.16:0/1912715694 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83b400c9b0 con 0x7f83c409d630 2023-09-30T22:36:54.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.754+0000 7f83cb68f700 1 -- 172.21.15.16:0/1912715694 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83bc009b30 con 0x7f83c409d630 2023-09-30T22:36:54.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.754+0000 7f83c1ffb700 1 -- 172.21.15.16:0/1912715694 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b400cd60 con 0x7f83c409d630 2023-09-30T22:36:54.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.754+0000 7f83cb68f700 1 -- 172.21.15.16:0/1912715694 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83c40a0410 con 0x7f83c409d630 2023-09-30T22:36:54.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.755+0000 7f83c1ffb700 1 -- 172.21.15.16:0/1912715694 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f83b4013470 con 0x7f83c409d630 2023-09-30T22:36:54.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.755+0000 7f83c1ffb700 1 -- 172.21.15.16:0/1912715694 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f83b403fab0 con 0x7f83c409d630 2023-09-30T22:36:54.759 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.756+0000 7f83cb68f700 1 -- 172.21.15.16:0/1912715694 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83c411df60 con 0x7f83c409d630 2023-09-30T22:36:54.761 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.761+0000 7f83c1ffb700 1 -- 172.21.15.16:0/1912715694 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f83b401b3b0 con 0x7f83c409d630 2023-09-30T22:36:54.793 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.793+0000 7f4f0ce25700 1 Processor -- start 2023-09-30T22:36:54.794 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.793+0000 7f4f0ce25700 1 -- start start 2023-09-30T22:36:54.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.794+0000 7f4f0ce25700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4f080a0a30 0x7f4f080a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:54.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.794+0000 7f4f0ce25700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4f08134170 0x7f4f0813bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:54.795 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.794+0000 7f4f0ce25700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f0809edd0 con 0x7f4f080a1340 2023-09-30T22:36:54.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.794+0000 7f4f0ce25700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f0809e1f0 con 0x7f4f080a0a30 2023-09-30T22:36:54.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.794+0000 7f4f0ce25700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4f0809dc50 con 0x7f4f08134170 2023-09-30T22:36:54.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.794+0000 7f4f0659c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4f080a0a30 0x7f4f080a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:54.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.794+0000 7f4f06d9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4f08134170 0x7f4f0813bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:54.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.794+0000 7f4f06d9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4f08134170 0x7f4f0813bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:51258/0 (socket says 172.21.15.16:51258) 2023-09-30T22:36:54.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.794+0000 7f4f0659c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f4f080a0a30 0x7f4f080a0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46300/0 (socket says 172.21.15.16:46300) 2023-09-30T22:36:54.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.794+0000 7f4f06d9d700 1 -- 172.21.15.16:0/3282073693 learned_addr learned my addr 172.21.15.16:0/3282073693 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:54.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.794+0000 7f4f05d9b700 1 --1- 172.21.15.16:0/3282073693 >> v1:172.21.15.16:6789/0 conn(0x7f4f080a1340 0x7f4f08133a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:49006/0 (socket says 172.21.15.16:49006) 2023-09-30T22:36:54.796 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.794+0000 7f4f06d9d700 1 -- 172.21.15.16:0/3282073693 >> v2:172.21.15.16:3300/0 conn(0x7f4f080a0a30 msgr2=0x7f4f080a0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:36:54.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.794+0000 7f4f06d9d700 1 --2- 172.21.15.16:0/3282073693 >> v2:172.21.15.16:3300/0 conn(0x7f4f080a0a30 0x7f4f080a0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:54.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.794+0000 7f4f06d9d700 1 -- 172.21.15.16:0/3282073693 >> v1:172.21.15.16:6789/0 conn(0x7f4f080a1340 legacy=0x7f4f08133a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:54.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.794+0000 7f4f06d9d700 1 -- 172.21.15.16:0/3282073693 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f080a1fe0 con 0x7f4f08134170 2023-09-30T22:36:54.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.794+0000 7f4f0659c700 1 --2- 172.21.15.16:0/3282073693 >> v2:172.21.15.16:3300/0 conn(0x7f4f080a0a30 0x7f4f080a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:54.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.795+0000 7f4f06d9d700 1 --2- 172.21.15.16:0/3282073693 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4f08134170 0x7f4f0813bd30 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f4f00009e50 tx=0x7f4f0000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=487ea7ce447c3c81 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:54.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.795+0000 7f4f0559a700 1 -- 172.21.15.16:0/3282073693 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0000e070 con 0x7f4f08134170 2023-09-30T22:36:54.797 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.795+0000 7f4f0559a700 1 -- 172.21.15.16:0/3282073693 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f0000b910 con 0x7f4f08134170 2023-09-30T22:36:54.798 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.795+0000 7f4f0559a700 1 -- 172.21.15.16:0/3282073693 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0000bcc0 con 0x7f4f08134170 2023-09-30T22:36:54.800 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.796+0000 7f4f0ce25700 1 -- 172.21.15.16:0/3282073693 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4f08134170 msgr2=0x7f4f0813bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:54.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.796+0000 7f4f0ce25700 1 --2- 172.21.15.16:0/3282073693 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4f08134170 0x7f4f0813bd30 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f4f00009e50 tx=0x7f4f0000b330 comp rx=0 tx=0).stop 2023-09-30T22:36:54.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.796+0000 7f4f0ce25700 1 -- 172.21.15.16:0/3282073693 shutdown_connections 2023-09-30T22:36:54.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.796+0000 7f4f0ce25700 1 --2- 172.21.15.16:0/3282073693 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4f08134170 0x7f4f0813bd30 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:54.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.796+0000 7f4f0ce25700 1 --2- 172.21.15.16:0/3282073693 >> v2:172.21.15.16:3300/0 conn(0x7f4f080a0a30 0x7f4f080a0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:54.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.796+0000 7f4f0ce25700 1 -- 172.21.15.16:0/3282073693 >> 172.21.15.16:0/3282073693 conn(0x7f4f080a6550 msgr2=0x7f4f080a6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:54.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.796+0000 7f4f0ce25700 1 -- 172.21.15.16:0/3282073693 shutdown_connections 2023-09-30T22:36:54.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.796+0000 7f4f0ce25700 1 -- 172.21.15.16:0/3282073693 wait complete. 2023-09-30T22:36:54.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.797+0000 7f4f0ce25700 1 Processor -- start 2023-09-30T22:36:54.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.798+0000 7f4f0ce25700 1 -- start start 2023-09-30T22:36:54.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.798+0000 7f4f0ce25700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4f080a0a30 0x7f4f0811e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:54.801 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.798+0000 7f4f0ce25700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f080a1340 0x7f4f0811ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:54.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.798+0000 7f4f0ce25700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4f08134170 0x7f4f0811f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:36:54.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.798+0000 7f4f0ce25700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f0811fbe0 con 0x7f4f080a1340 2023-09-30T22:36:54.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.798+0000 7f4f0ce25700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f081221a0 con 0x7f4f08134170 2023-09-30T22:36:54.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.798+0000 7f4f0ce25700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f4f08120290 con 0x7f4f080a0a30 2023-09-30T22:36:54.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.798+0000 7f4f0659c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4f080a0a30 0x7f4f0811e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:54.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.798+0000 7f4f0659c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4f080a0a30 0x7f4f0811e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:51260/0 (socket says 172.21.15.16:51260) 2023-09-30T22:36:54.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.798+0000 7f4f06d9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4f08134170 0x7f4f0811f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:54.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.798+0000 7f4f0659c700 1 -- 172.21.15.16:0/525459803 learned_addr learned my addr 172.21.15.16:0/525459803 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:36:54.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.798+0000 7f4f05d9b700 1 --2- 172.21.15.16:0/525459803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f080a1340 0x7f4f0811ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:36:54.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.799+0000 7f4f0659c700 1 -- 172.21.15.16:0/525459803 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4f08134170 msgr2=0x7f4f0811f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:54.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.799+0000 7f4f0659c700 1 --2- 172.21.15.16:0/525459803 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4f08134170 0x7f4f0811f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:54.802 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.799+0000 7f4f0659c700 1 -- 172.21.15.16:0/525459803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f080a1340 msgr2=0x7f4f0811ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:54.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.799+0000 7f4f0659c700 1 --2- 172.21.15.16:0/525459803 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4f080a1340 0x7f4f0811ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:54.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.799+0000 7f4f0659c700 1 -- 172.21.15.16:0/525459803 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef80094a0 con 0x7f4f080a0a30 2023-09-30T22:36:54.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.799+0000 7f4f06d9d700 1 --2- 172.21.15.16:0/525459803 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f4f08134170 0x7f4f0811f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:36:54.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.799+0000 7f4f0659c700 1 --2- 172.21.15.16:0/525459803 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f4f080a0a30 0x7f4f0811e710 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f4ef800ec10 tx=0x7f4ef800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:36:54.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.799+0000 7f4efeffd700 1 -- 172.21.15.16:0/525459803 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef800c850 con 0x7f4f080a0a30 2023-09-30T22:36:54.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.799+0000 7f4efeffd700 1 -- 172.21.15.16:0/525459803 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ef800c9b0 con 0x7f4f080a0a30 2023-09-30T22:36:54.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.799+0000 7f4f0ce25700 1 -- 172.21.15.16:0/525459803 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f00009b30 con 0x7f4f080a0a30 2023-09-30T22:36:54.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.799+0000 7f4efeffd700 1 -- 172.21.15.16:0/525459803 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef800cd20 con 0x7f4f080a0a30 2023-09-30T22:36:54.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.799+0000 7f4f0ce25700 1 -- 172.21.15.16:0/525459803 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f080a25e0 con 0x7f4f080a0a30 2023-09-30T22:36:54.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.800+0000 7f4efeffd700 1 -- 172.21.15.16:0/525459803 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f4ef8013470 con 0x7f4f080a0a30 2023-09-30T22:36:54.803 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.800+0000 7f4efeffd700 1 -- 172.21.15.16:0/525459803 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f4ef803fd30 con 0x7f4f080a0a30 2023-09-30T22:36:54.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.801+0000 7f4f0ce25700 1 -- 172.21.15.16:0/525459803 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f08074ea0 con 0x7f4f080a0a30 2023-09-30T22:36:54.806 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.806+0000 7f4efeffd700 1 -- 172.21.15.16:0/525459803 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f4ef8017070 con 0x7f4f080a0a30 2023-09-30T22:36:54.971 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.971+0000 7f83cb68f700 1 -- 172.21.15.16:0/1912715694 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f83c40747f0 con 0x7f83c409d630 2023-09-30T22:36:54.972 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.971+0000 7f83c1ffb700 1 -- 172.21.15.16:0/1912715694 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f83b4017020 con 0x7f83c409d630 2023-09-30T22:36:54.972 INFO:teuthology.orchestra.run.smithi016.stdout:133143986178 2023-09-30T22:36:54.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.974+0000 7f83cb68f700 1 -- 172.21.15.16:0/1912715694 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83c409d630 msgr2=0x7f83c4121340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:36:54.974 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.974+0000 7f83cb68f700 1 --2- 172.21.15.16:0/1912715694 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83c409d630 0x7f83c4121340 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f83b400e9b0 tx=0x7f83b400ecc0 comp rx=0 tx=0).stop 2023-09-30T22:36:54.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.974+0000 7f83cb68f700 1 -- 172.21.15.16:0/1912715694 shutdown_connections 2023-09-30T22:36:54.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.974+0000 7f83cb68f700 1 --2- 172.21.15.16:0/1912715694 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f83c409d630 0x7f83c4121340 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:54.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.974+0000 7f83cb68f700 1 --2- 172.21.15.16:0/1912715694 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f83c4133930 0x7f83c4121dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:54.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.974+0000 7f83cb68f700 1 --2- 172.21.15.16:0/1912715694 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f83c40a0a30 0x7f83c4121880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:36:54.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.974+0000 7f83cb68f700 1 -- 172.21.15.16:0/1912715694 >> 172.21.15.16:0/1912715694 conn(0x7f83c40a3050 msgr2=0x7f83c41331a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:36:54.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.974+0000 7f83cb68f700 1 -- 172.21.15.16:0/1912715694 shutdown_connections 2023-09-30T22:36:54.975 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:36:54.974+0000 7f83cb68f700 1 -- 172.21.15.16:0/1912715694 wait complete. 2023-09-30T22:36:54.989 INFO:tasks.ceph.ceph_manager.ceph:need seq 133143986179 got 133143986178 for osd.2 2023-09-30T22:38:54.691 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-09-30T22:38:54.692 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph.py", line 1496, in healthy manager.wait_for_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2742, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2574, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2340, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 1595, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi016 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2023-09-30T22:38:54.762 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=8548a7ba8a8d44719f5b310a888e0fc8 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph.py", line 1496, in healthy manager.wait_for_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2742, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2574, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2340, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 1595, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi016 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2023-09-30T22:38:54.767 DEBUG:teuthology.run_tasks:Unwinding manager ceph.restart 2023-09-30T22:38:54.778 DEBUG:teuthology.run_tasks:Unwinding manager ceph.stop 2023-09-30T22:38:54.804 DEBUG:teuthology.run_tasks:Unwinding manager ceph.restart 2023-09-30T22:38:54.815 DEBUG:teuthology.run_tasks:Unwinding manager ceph.restart 2023-09-30T22:38:54.825 DEBUG:teuthology.run_tasks:Unwinding manager install.upgrade 2023-09-30T22:38:54.835 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-09-30T22:38:54.845 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-09-30T22:38:54.845 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-09-30T22:38:54.845 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-30T22:38:54.874 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-09-30T22:38:54.874 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-30T22:38:54.937 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:38:55.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.081+0000 7ff2370ed700 1 Processor -- start 2023-09-30T22:38:55.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.081+0000 7ff2370ed700 1 -- start start 2023-09-30T22:38:55.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.081+0000 7ff2370ed700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff23009e930 0x7ff23009ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:55.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.081+0000 7ff2370ed700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff23009f240 0x7ff2301317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:55.083 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.081+0000 7ff2370ed700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff23009f650 0x7ff230131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:55.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.081+0000 7ff2370ed700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff230070940 con 0x7ff23009f240 2023-09-30T22:38:55.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.081+0000 7ff2370ed700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff23006fd60 con 0x7ff23009e930 2023-09-30T22:38:55.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.081+0000 7ff2370ed700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff23006fa80 con 0x7ff23009f650 2023-09-30T22:38:55.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.082+0000 7ff22ffff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff23009f240 0x7ff2301317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:55.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.082+0000 7ff23568a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff23009f650 0x7ff230131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:55.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.082+0000 7ff22ffff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7ff23009f240 0x7ff2301317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48088/0 (socket says 172.21.15.16:48088) 2023-09-30T22:38:55.084 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.082+0000 7ff234e89700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff23009e930 0x7ff23009ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:55.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.082+0000 7ff23568a700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff23009f650 0x7ff230131d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37182/0 (socket says 172.21.15.16:37182) 2023-09-30T22:38:55.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.082+0000 7ff22ffff700 1 -- 172.21.15.16:0/4097171783 learned_addr learned my addr 172.21.15.16:0/4097171783 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:38:55.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.082+0000 7ff23568a700 1 -- 172.21.15.16:0/4097171783 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff23009e930 msgr2=0x7ff23009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:38:55.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.082+0000 7ff23568a700 1 --2- 172.21.15.16:0/4097171783 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff23009e930 0x7ff23009ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:55.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.082+0000 7ff23568a700 1 -- 172.21.15.16:0/4097171783 >> v2:172.21.15.16:3300/0 conn(0x7ff23009f240 msgr2=0x7ff2301317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:38:55.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.082+0000 7ff23568a700 1 --2- 172.21.15.16:0/4097171783 >> v2:172.21.15.16:3300/0 conn(0x7ff23009f240 0x7ff2301317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:55.085 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.082+0000 7ff23568a700 1 -- 172.21.15.16:0/4097171783 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff23009fb60 con 0x7ff23009f650 2023-09-30T22:38:55.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.082+0000 7ff22ffff700 1 --2- 172.21.15.16:0/4097171783 >> v2:172.21.15.16:3300/0 conn(0x7ff23009f240 0x7ff2301317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:38:55.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.082+0000 7ff23568a700 1 --2- 172.21.15.16:0/4097171783 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff23009f650 0x7ff230131d30 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7ff228009d90 tx=0x7ff22800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2bef09640520814 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:38:55.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.082+0000 7ff22f7fe700 1 -- 172.21.15.16:0/4097171783 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22800e070 con 0x7ff23009f650 2023-09-30T22:38:55.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.082+0000 7ff22f7fe700 1 -- 172.21.15.16:0/4097171783 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff22800bb20 con 0x7ff23009f650 2023-09-30T22:38:55.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.082+0000 7ff22f7fe700 1 -- 172.21.15.16:0/4097171783 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff228012430 con 0x7ff23009f650 2023-09-30T22:38:55.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.083+0000 7ff2370ed700 1 -- 172.21.15.16:0/4097171783 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff23009f650 msgr2=0x7ff230131d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:38:55.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.083+0000 7ff2370ed700 1 --2- 172.21.15.16:0/4097171783 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff23009f650 0x7ff230131d30 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7ff228009d90 tx=0x7ff22800b330 comp rx=0 tx=0).stop 2023-09-30T22:38:55.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.083+0000 7ff2370ed700 1 -- 172.21.15.16:0/4097171783 shutdown_connections 2023-09-30T22:38:55.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.083+0000 7ff2370ed700 1 --2- 172.21.15.16:0/4097171783 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff23009e930 0x7ff23009ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:55.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.083+0000 7ff2370ed700 1 --2- 172.21.15.16:0/4097171783 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff23009f650 0x7ff230131d30 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:55.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.083+0000 7ff2370ed700 1 --2- 172.21.15.16:0/4097171783 >> v2:172.21.15.16:3300/0 conn(0x7ff23009f240 0x7ff2301317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:55.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.083+0000 7ff2370ed700 1 -- 172.21.15.16:0/4097171783 >> 172.21.15.16:0/4097171783 conn(0x7ff2300a4210 msgr2=0x7ff2300a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:38:55.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.083+0000 7ff2370ed700 1 -- 172.21.15.16:0/4097171783 shutdown_connections 2023-09-30T22:38:55.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.083+0000 7ff2370ed700 1 -- 172.21.15.16:0/4097171783 wait complete. 2023-09-30T22:38:55.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.084+0000 7ff2370ed700 1 Processor -- start 2023-09-30T22:38:55.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.084+0000 7ff2370ed700 1 -- start start 2023-09-30T22:38:55.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.085+0000 7ff2370ed700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff23009e930 0x7ff2301203f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:55.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.085+0000 7ff2370ed700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff23009f240 0x7ff230120930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:55.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.085+0000 7ff2370ed700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff23009f650 0x7ff230120e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:55.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.085+0000 7ff2370ed700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff230121850 con 0x7ff23009e930 2023-09-30T22:38:55.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.085+0000 7ff2370ed700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff230071d20 con 0x7ff23009f650 2023-09-30T22:38:55.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.085+0000 7ff2370ed700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7ff230122810 con 0x7ff23009f240 2023-09-30T22:38:55.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.085+0000 7ff234e89700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff23009e930 0x7ff2301203f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:55.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.085+0000 7ff234e89700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff23009e930 0x7ff2301203f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48102/0 (socket says 172.21.15.16:48102) 2023-09-30T22:38:55.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.085+0000 7ff22ffff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff23009f240 0x7ff230120930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:55.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.085+0000 7ff23568a700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff23009f650 0x7ff230120e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:55.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.085+0000 7ff234e89700 1 -- 172.21.15.16:0/931363580 learned_addr learned my addr 172.21.15.16:0/931363580 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:38:55.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.085+0000 7ff22ffff700 1 -- 172.21.15.16:0/931363580 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff23009f650 msgr2=0x7ff230120e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:38:55.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.085+0000 7ff22ffff700 1 --2- 172.21.15.16:0/931363580 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff23009f650 0x7ff230120e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:55.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.085+0000 7ff22ffff700 1 -- 172.21.15.16:0/931363580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff23009e930 msgr2=0x7ff2301203f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:38:55.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.085+0000 7ff22ffff700 1 --2- 172.21.15.16:0/931363580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff23009e930 0x7ff2301203f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:55.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.085+0000 7ff22ffff700 1 -- 172.21.15.16:0/931363580 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff21c009550 con 0x7ff23009f240 2023-09-30T22:38:55.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.085+0000 7ff234e89700 1 --2- 172.21.15.16:0/931363580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff23009e930 0x7ff2301203f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:38:55.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.085+0000 7ff23568a700 1 --2- 172.21.15.16:0/931363580 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff23009f650 0x7ff230120e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:38:55.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.085+0000 7ff22ffff700 1 --2- 172.21.15.16:0/931363580 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff23009f240 0x7ff230120930 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7ff21c00e9a0 tx=0x7ff21c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:38:55.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.086+0000 7ff22d7fa700 1 -- 172.21.15.16:0/931363580 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21c004240 con 0x7ff23009f240 2023-09-30T22:38:55.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.086+0000 7ff22d7fa700 1 -- 172.21.15.16:0/931363580 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff21c0043a0 con 0x7ff23009f240 2023-09-30T22:38:55.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.086+0000 7ff2370ed700 1 -- 172.21.15.16:0/931363580 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff228009a70 con 0x7ff23009f240 2023-09-30T22:38:55.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.086+0000 7ff22d7fa700 1 -- 172.21.15.16:0/931363580 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21c0036a0 con 0x7ff23009f240 2023-09-30T22:38:55.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.086+0000 7ff2370ed700 1 -- 172.21.15.16:0/931363580 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2300a0190 con 0x7ff23009f240 2023-09-30T22:38:55.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.086+0000 7ff22d7fa700 1 -- 172.21.15.16:0/931363580 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7ff21c003800 con 0x7ff23009f240 2023-09-30T22:38:55.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.087+0000 7ff22d7fa700 1 -- 172.21.15.16:0/931363580 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7ff21c03fb80 con 0x7ff23009f240 2023-09-30T22:38:55.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.087+0000 7ff2370ed700 1 -- 172.21.15.16:0/931363580 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff230074ea0 con 0x7ff23009f240 2023-09-30T22:38:55.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.092+0000 7ff22d7fa700 1 -- 172.21.15.16:0/931363580 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7ff21c03fe60 con 0x7ff23009f240 2023-09-30T22:38:55.295 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.295+0000 7ff2370ed700 1 -- 172.21.15.16:0/931363580 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff2301312f0 con 0x7ff23009f240 2023-09-30T22:38:55.296 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.296+0000 7ff22d7fa700 1 -- 172.21.15.16:0/931363580 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+13447 (secure 0 0 0) 0x7ff21c02b090 con 0x7ff23009f240 2023-09-30T22:38:55.297 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:38:55.297 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":40,"fsid":"7ede3e14-2bd0-4726-9437-6d13c90cd342","created":"2023-09-30T22:26:35.483697+0000","modified":"2023-09-30T22:30:13.497628+0000","last_up_change":"2023-09-30T22:29:35.702639+0000","last_in_change":"2023-09-30T22:26:37.120906+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T22:26:44.285133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{},"mgr_devicehealth":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:26:45.132921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:26:49.353683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:26:50.655685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"19946985-0ee2-487f-9347-929670dbe5ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":22,"up_from":28,"up_thru":32,"down_at":23,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6804","nonce":129539}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6805","nonce":129539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6807","nonce":129539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6806","nonce":129539}]},"public_addr":"172.21.15.16:6804/129539","cluster_addr":"172.21.15.16:6805/129539","heartbeat_back_addr":"172.21.15.16:6807/129539","heartbeat_front_addr":"172.21.15.16:6806/129539","state":["exists","up"]},{"osd":1,"uuid":"8efd4dbc-6ae4-4b82-b530-3b3eac2be445","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":22,"up_from":29,"up_thru":32,"down_at":24,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6808","nonce":129637}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6809","nonce":129637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6811","nonce":129637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6810","nonce":129637}]},"public_addr":"172.21.15.16:6808/129637","cluster_addr":"172.21.15.16:6809/129637","heartbeat_back_addr":"172.21.15.16:6811/129637","heartbeat_front_addr":"172.21.15.16:6810/129637","state":["exists","up"]},{"osd":2,"uuid":"b47bb1ce-27af-4c0c-868a-b76ebd0d9499","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":24,"up_from":31,"up_thru":32,"down_at":25,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6800","nonce":129747}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6801","nonce":129747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6803","nonce":129747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6802","nonce":129747}]},"public_addr":"172.21.15.16:6800/129747","cluster_addr":"172.21.15.16:6801/129747","heartbeat_back_addr":"172.21.15.16:6803/129747","heartbeat_front_addr":"172.21.15.16:6802/129747","state":["exists","up"]},{"osd":3,"uuid":"03b53d64-d10d-448a-a081-39ecceab589b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":25,"up_from":32,"up_thru":32,"down_at":26,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6812","nonce":130087}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6813","nonce":130087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6815","nonce":130087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6814","nonce":130087}]},"public_addr":"172.21.15.16:6812/130087","cluster_addr":"172.21.15.16:6813/130087","heartbeat_back_addr":"172.21.15.16:6815/130087","heartbeat_front_addr":"172.21.15.16:6814/130087","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.305953+0000","dead_epoch":23},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.307488+0000","dead_epoch":24},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.341023+0000","dead_epoch":25},{"osd":3,"down_stamp":"2023-09-30T22:29:28.519844+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.192071+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.8","mappings":[{"from":3,"to":2}]},{"pgid":"4.d","mappings":[{"from":3,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":0}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]},{"pgid":"4.17","mappings":[{"from":3,"to":2}]},{"pgid":"4.19","mappings":[{"from":3,"to":0}]},{"pgid":"4.3a","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/3403658744":"2023-10-01T22:30:13.497576+0000","172.21.15.16:0/1216430525":"2023-10-01T22:30:13.497576+0000","172.21.15.16:0/3418827610":"2023-10-01T22:30:13.497576+0000","172.21.15.16:0/1101648925":"2023-10-01T22:29:14.475978+0000","172.21.15.16:0/2249754484":"2023-10-01T22:29:14.475978+0000","172.21.15.16:0/1879378585":"2023-10-01T22:30:13.497576+0000","172.21.15.16:6816/118285":"2023-10-01T22:29:14.475978+0000","172.21.15.16:6816/126811":"2023-10-01T22:30:13.497576+0000","172.21.15.16:6817/1424366610":"2023-10-01T22:29:40.259230+0000","172.21.15.16:0/2178963721":"2023-10-01T22:30:13.497576+0000","172.21.15.16:0/3146063150":"2023-10-01T22:30:13.497576+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:38:55.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.299+0000 7ff2370ed700 1 -- 172.21.15.16:0/931363580 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff23009f240 msgr2=0x7ff230120930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:38:55.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.299+0000 7ff2370ed700 1 --2- 172.21.15.16:0/931363580 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff23009f240 0x7ff230120930 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7ff21c00e9a0 tx=0x7ff21c00ecb0 comp rx=0 tx=0).stop 2023-09-30T22:38:55.299 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.299+0000 7ff2370ed700 1 -- 172.21.15.16:0/931363580 shutdown_connections 2023-09-30T22:38:55.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.299+0000 7ff2370ed700 1 --2- 172.21.15.16:0/931363580 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7ff23009f650 0x7ff230120e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:55.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.299+0000 7ff2370ed700 1 --2- 172.21.15.16:0/931363580 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff23009e930 0x7ff2301203f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:55.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.299+0000 7ff2370ed700 1 --2- 172.21.15.16:0/931363580 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7ff23009f240 0x7ff230120930 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:55.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.299+0000 7ff2370ed700 1 -- 172.21.15.16:0/931363580 >> 172.21.15.16:0/931363580 conn(0x7ff2300a4210 msgr2=0x7ff2300a1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:38:55.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.299+0000 7ff2370ed700 1 -- 172.21.15.16:0/931363580 shutdown_connections 2023-09-30T22:38:55.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.299+0000 7ff2370ed700 1 -- 172.21.15.16:0/931363580 wait complete. 2023-09-30T22:38:55.313 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-30T22:26:44.285133+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}, 'mgr_devicehealth': {}}, 'read_balance': {'score_acting': 4, 'score_stable': 4, 'optimal_score': 0.5, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-09-30T22:26:45.132921+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-30T22:26:49.353683+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-09-30T22:26:50.655685+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-30T22:38:55.314 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-30T22:38:55.456 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.453+0000 7f593e7ac700 1 Processor -- start 2023-09-30T22:38:55.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.454+0000 7f593e7ac700 1 -- start start 2023-09-30T22:38:55.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.454+0000 7f593e7ac700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f59381303c0 0x7f5938130790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:55.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.454+0000 7f593e7ac700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f593806a770 0x7f593806ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:55.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.454+0000 7f593e7ac700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f593809f620 con 0x7f593806b0c0 2023-09-30T22:38:55.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.454+0000 7f593e7ac700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5938130280 con 0x7f593806a770 2023-09-30T22:38:55.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.454+0000 7f593e7ac700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f593809e1b0 con 0x7f59381303c0 2023-09-30T22:38:55.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.454+0000 7f5937fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f59381303c0 0x7f5938130790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:55.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.454+0000 7f59377fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f593806a770 0x7f593806ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:55.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.454+0000 7f593cd49700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f593806b0c0 0x7f593813a7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58876/0 (socket says 172.21.15.16:58876) 2023-09-30T22:38:55.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.454+0000 7f5937fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f59381303c0 0x7f5938130790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37198/0 (socket says 172.21.15.16:37198) 2023-09-30T22:38:55.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.454+0000 7f593cd49700 1 -- 172.21.15.16:0/666850310 learned_addr learned my addr 172.21.15.16:0/666850310 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:38:55.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.454+0000 7f59377fe700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f593806a770 0x7f593806ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48104/0 (socket says 172.21.15.16:48104) 2023-09-30T22:38:55.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.455+0000 7f59377fe700 1 -- 172.21.15.16:0/666850310 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f59381303c0 msgr2=0x7f5938130790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:38:55.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.455+0000 7f59377fe700 1 --2- 172.21.15.16:0/666850310 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f59381303c0 0x7f5938130790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:55.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.455+0000 7f59377fe700 1 -- 172.21.15.16:0/666850310 >> v1:172.21.15.16:6789/0 conn(0x7f593806b0c0 legacy=0x7f593813a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:38:55.459 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.455+0000 7f59377fe700 1 -- 172.21.15.16:0/666850310 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f593806b540 con 0x7f593806a770 2023-09-30T22:38:55.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.455+0000 7f5936ffd700 1 -- 172.21.15.16:0/666850310 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 69339210 0 0) 0x7f593809f620 con 0x7f593806b0c0 2023-09-30T22:38:55.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.455+0000 7f5937fff700 1 --2- 172.21.15.16:0/666850310 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f59381303c0 0x7f5938130790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:38:55.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.455+0000 7f59377fe700 1 --2- 172.21.15.16:0/666850310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f593806a770 0x7f593806ab80 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f5928009840 tx=0x7f5928009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c86661a651b6ed9 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:38:55.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.455+0000 7f5936ffd700 1 -- 172.21.15.16:0/666850310 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5928011830 con 0x7f593806a770 2023-09-30T22:38:55.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.455+0000 7f5936ffd700 1 -- 172.21.15.16:0/666850310 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5928011990 con 0x7f593806a770 2023-09-30T22:38:55.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.455+0000 7f5936ffd700 1 -- 172.21.15.16:0/666850310 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5928011d20 con 0x7f593806a770 2023-09-30T22:38:55.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.456+0000 7f593e7ac700 1 -- 172.21.15.16:0/666850310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f593806a770 msgr2=0x7f593806ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:38:55.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.456+0000 7f593e7ac700 1 --2- 172.21.15.16:0/666850310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f593806a770 0x7f593806ab80 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f5928009840 tx=0x7f5928009b50 comp rx=0 tx=0).stop 2023-09-30T22:38:55.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.456+0000 7f593e7ac700 1 -- 172.21.15.16:0/666850310 shutdown_connections 2023-09-30T22:38:55.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.456+0000 7f593e7ac700 1 --2- 172.21.15.16:0/666850310 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f59381303c0 0x7f5938130790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:55.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.456+0000 7f593e7ac700 1 --2- 172.21.15.16:0/666850310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f593806a770 0x7f593806ab80 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:55.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.456+0000 7f593e7ac700 1 -- 172.21.15.16:0/666850310 >> 172.21.15.16:0/666850310 conn(0x7f5938066d70 msgr2=0x7f59380a2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:38:55.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.456+0000 7f593e7ac700 1 -- 172.21.15.16:0/666850310 shutdown_connections 2023-09-30T22:38:55.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.456+0000 7f593e7ac700 1 -- 172.21.15.16:0/666850310 wait complete. 2023-09-30T22:38:55.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.457+0000 7f593e7ac700 1 Processor -- start 2023-09-30T22:38:55.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.457+0000 7f593e7ac700 1 -- start start 2023-09-30T22:38:55.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.457+0000 7f593e7ac700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f593806a770 0x7f593811d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:55.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.457+0000 7f593e7ac700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f593806b0c0 0x7f593811de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:55.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.457+0000 7f593e7ac700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59381303c0 0x7f593811e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:55.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.457+0000 7f593e7ac700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f593811fd40 con 0x7f59381303c0 2023-09-30T22:38:55.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.457+0000 7f593e7ac700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5938071860 con 0x7f593806b0c0 2023-09-30T22:38:55.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.457+0000 7f593e7ac700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5938071fe0 con 0x7f593806a770 2023-09-30T22:38:55.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.457+0000 7f5937fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f593806a770 0x7f593811d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:55.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.457+0000 7f5937fff700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f593806a770 0x7f593811d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37200/0 (socket says 172.21.15.16:37200) 2023-09-30T22:38:55.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.458+0000 7f5937fff700 1 -- 172.21.15.16:0/3790165358 learned_addr learned my addr 172.21.15.16:0/3790165358 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:38:55.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.458+0000 7f593cd49700 1 --2- 172.21.15.16:0/3790165358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59381303c0 0x7f593811e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:55.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.458+0000 7f59377fe700 1 --2- 172.21.15.16:0/3790165358 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f593806b0c0 0x7f593811de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:55.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.458+0000 7f5937fff700 1 -- 172.21.15.16:0/3790165358 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f593806b0c0 msgr2=0x7f593811de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:38:55.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.458+0000 7f5937fff700 1 --2- 172.21.15.16:0/3790165358 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f593806b0c0 0x7f593811de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:55.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.458+0000 7f5937fff700 1 -- 172.21.15.16:0/3790165358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59381303c0 msgr2=0x7f593811e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:38:55.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.458+0000 7f5937fff700 1 --2- 172.21.15.16:0/3790165358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59381303c0 0x7f593811e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:55.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.458+0000 7f5937fff700 1 -- 172.21.15.16:0/3790165358 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59200094a0 con 0x7f593806a770 2023-09-30T22:38:55.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.458+0000 7f593cd49700 1 --2- 172.21.15.16:0/3790165358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59381303c0 0x7f593811e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:38:55.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.458+0000 7f5937fff700 1 --2- 172.21.15.16:0/3790165358 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f593806a770 0x7f593811d930 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f592000ec10 tx=0x7f592000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:38:55.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.458+0000 7f5934ff9700 1 -- 172.21.15.16:0/3790165358 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f592000c7f0 con 0x7f593806a770 2023-09-30T22:38:55.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.458+0000 7f5934ff9700 1 -- 172.21.15.16:0/3790165358 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f592000c950 con 0x7f593806a770 2023-09-30T22:38:55.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.458+0000 7f593e7ac700 1 -- 172.21.15.16:0/3790165358 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59280094a0 con 0x7f593806a770 2023-09-30T22:38:55.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.458+0000 7f5934ff9700 1 -- 172.21.15.16:0/3790165358 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f592000ccc0 con 0x7f593806a770 2023-09-30T22:38:55.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.458+0000 7f593e7ac700 1 -- 172.21.15.16:0/3790165358 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f593811eda0 con 0x7f593806a770 2023-09-30T22:38:55.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.459+0000 7f5934ff9700 1 -- 172.21.15.16:0/3790165358 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f5920013470 con 0x7f593806a770 2023-09-30T22:38:55.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.459+0000 7f5934ff9700 1 -- 172.21.15.16:0/3790165358 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f592003f760 con 0x7f593806a770 2023-09-30T22:38:55.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.459+0000 7f593e7ac700 1 -- 172.21.15.16:0/3790165358 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f593804f2a0 con 0x7f593806a770 2023-09-30T22:38:55.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.464+0000 7f5934ff9700 1 -- 172.21.15.16:0/3790165358 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f5920013720 con 0x7f593806a770 2023-09-30T22:38:55.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.664+0000 7f593e7ac700 1 -- 172.21.15.16:0/3790165358 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f593811f1d0 con 0x7f593806a770 2023-09-30T22:38:55.665 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.665+0000 7f5934ff9700 1 -- 172.21.15.16:0/3790165358 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5920021020 con 0x7f593806a770 2023-09-30T22:38:55.666 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-09-30T22:38:55.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.667+0000 7f593e7ac700 1 -- 172.21.15.16:0/3790165358 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f593806a770 msgr2=0x7f593811d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:38:55.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.667+0000 7f593e7ac700 1 --2- 172.21.15.16:0/3790165358 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f593806a770 0x7f593811d930 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f592000ec10 tx=0x7f592000ef20 comp rx=0 tx=0).stop 2023-09-30T22:38:55.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.667+0000 7f593e7ac700 1 -- 172.21.15.16:0/3790165358 shutdown_connections 2023-09-30T22:38:55.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.667+0000 7f593e7ac700 1 --2- 172.21.15.16:0/3790165358 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59381303c0 0x7f593811e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:55.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.667+0000 7f593e7ac700 1 --2- 172.21.15.16:0/3790165358 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f593806a770 0x7f593811d930 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:55.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.667+0000 7f593e7ac700 1 --2- 172.21.15.16:0/3790165358 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f593806b0c0 0x7f593811de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:55.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.667+0000 7f593e7ac700 1 -- 172.21.15.16:0/3790165358 >> 172.21.15.16:0/3790165358 conn(0x7f5938066d70 msgr2=0x7f59380a26d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:38:55.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.667+0000 7f593e7ac700 1 -- 172.21.15.16:0/3790165358 shutdown_connections 2023-09-30T22:38:55.669 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.667+0000 7f593e7ac700 1 -- 172.21.15.16:0/3790165358 wait complete. 2023-09-30T22:38:55.682 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-09-30T22:38:55.822 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.822+0000 7f9f20860700 1 Processor -- start 2023-09-30T22:38:55.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.822+0000 7f9f20860700 1 -- start start 2023-09-30T22:38:55.825 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.822+0000 7f9f20860700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9f1c136780 0x7f9f1c138b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:55.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.822+0000 7f9f20860700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9f1c1390a0 0x7f9f1c13b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:55.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.822+0000 7f9f20860700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f1c1301c0 con 0x7f9f1c1303c0 2023-09-30T22:38:55.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.822+0000 7f9f20860700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1c09f620 con 0x7f9f1c136780 2023-09-30T22:38:55.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.822+0000 7f9f20860700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1c09e1b0 con 0x7f9f1c1390a0 2023-09-30T22:38:55.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.823+0000 7f9f1ad9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9f1c1390a0 0x7f9f1c13b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:55.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.823+0000 7f9f19d9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9f1c136780 0x7f9f1c138b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:55.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.823+0000 7f9f1a59c700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f9f1c1303c0 0x7f9f1c130790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58878/0 (socket says 172.21.15.16:58878) 2023-09-30T22:38:55.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.823+0000 7f9f1ad9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9f1c1390a0 0x7f9f1c13b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37202/0 (socket says 172.21.15.16:37202) 2023-09-30T22:38:55.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.823+0000 7f9f1a59c700 1 -- 172.21.15.16:0/2223096748 learned_addr learned my addr 172.21.15.16:0/2223096748 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:38:55.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.823+0000 7f9f1ad9d700 1 -- 172.21.15.16:0/2223096748 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9f1c136780 msgr2=0x7f9f1c138b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:38:55.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.823+0000 7f9f1ad9d700 1 --2- 172.21.15.16:0/2223096748 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9f1c136780 0x7f9f1c138b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:55.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.823+0000 7f9f1ad9d700 1 -- 172.21.15.16:0/2223096748 >> v1:172.21.15.16:6789/0 conn(0x7f9f1c1303c0 legacy=0x7f9f1c130790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:38:55.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.823+0000 7f9f1ad9d700 1 -- 172.21.15.16:0/2223096748 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f1c0a3a60 con 0x7f9f1c1390a0 2023-09-30T22:38:55.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.823+0000 7f9f1959a700 1 -- 172.21.15.16:0/2223096748 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1844383406 0 0) 0x7f9f1c1301c0 con 0x7f9f1c1303c0 2023-09-30T22:38:55.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.823+0000 7f9f1ad9d700 1 --2- 172.21.15.16:0/2223096748 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9f1c1390a0 0x7f9f1c13b480 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f9f14009dc0 tx=0x7f9f1400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5fe262ee9068fd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:38:55.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.823+0000 7f9f1959a700 1 -- 172.21.15.16:0/2223096748 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1400e070 con 0x7f9f1c1390a0 2023-09-30T22:38:55.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.823+0000 7f9f1959a700 1 -- 172.21.15.16:0/2223096748 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f1400bb20 con 0x7f9f1c1390a0 2023-09-30T22:38:55.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.823+0000 7f9f1959a700 1 -- 172.21.15.16:0/2223096748 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f14012430 con 0x7f9f1c1390a0 2023-09-30T22:38:55.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.824+0000 7f9f20860700 1 -- 172.21.15.16:0/2223096748 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9f1c1390a0 msgr2=0x7f9f1c13b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:38:55.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.824+0000 7f9f20860700 1 --2- 172.21.15.16:0/2223096748 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9f1c1390a0 0x7f9f1c13b480 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f9f14009dc0 tx=0x7f9f1400b330 comp rx=0 tx=0).stop 2023-09-30T22:38:55.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.824+0000 7f9f20860700 1 -- 172.21.15.16:0/2223096748 shutdown_connections 2023-09-30T22:38:55.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.824+0000 7f9f20860700 1 --2- 172.21.15.16:0/2223096748 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9f1c1390a0 0x7f9f1c13b480 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:55.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.824+0000 7f9f20860700 1 --2- 172.21.15.16:0/2223096748 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9f1c136780 0x7f9f1c138b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:55.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.824+0000 7f9f20860700 1 -- 172.21.15.16:0/2223096748 >> 172.21.15.16:0/2223096748 conn(0x7f9f1c066d70 msgr2=0x7f9f1c0a2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:38:55.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.824+0000 7f9f20860700 1 -- 172.21.15.16:0/2223096748 shutdown_connections 2023-09-30T22:38:55.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.824+0000 7f9f20860700 1 -- 172.21.15.16:0/2223096748 wait complete. 2023-09-30T22:38:55.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.825+0000 7f9f20860700 1 Processor -- start 2023-09-30T22:38:55.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.825+0000 7f9f20860700 1 -- start start 2023-09-30T22:38:55.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.826+0000 7f9f20860700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f1c1303c0 0x7f9f1c12cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:55.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.826+0000 7f9f20860700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9f1c136780 0x7f9f1c12d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:55.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.826+0000 7f9f20860700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9f1c1390a0 0x7f9f1c12d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:55.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.826+0000 7f9f20860700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1c070590 con 0x7f9f1c1303c0 2023-09-30T22:38:55.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.826+0000 7f9f20860700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1c0a29a0 con 0x7f9f1c136780 2023-09-30T22:38:55.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.826+0000 7f9f20860700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1c06fff0 con 0x7f9f1c1390a0 2023-09-30T22:38:55.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.826+0000 7f9f1a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f1c1303c0 0x7f9f1c12cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:55.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.826+0000 7f9f19d9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9f1c136780 0x7f9f1c12d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:55.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.826+0000 7f9f1ad9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9f1c1390a0 0x7f9f1c12d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:55.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.826+0000 7f9f19d9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9f1c136780 0x7f9f1c12d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:45980/0 (socket says 172.21.15.16:45980) 2023-09-30T22:38:55.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.826+0000 7f9f19d9b700 1 -- 172.21.15.16:0/2216331861 learned_addr learned my addr 172.21.15.16:0/2216331861 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:38:55.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.826+0000 7f9f1a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f1c1303c0 0x7f9f1c12cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48110/0 (socket says 172.21.15.16:48110) 2023-09-30T22:38:55.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.826+0000 7f9f1a59c700 1 -- 172.21.15.16:0/2216331861 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9f1c1390a0 msgr2=0x7f9f1c12d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:38:55.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.826+0000 7f9f1a59c700 1 --2- 172.21.15.16:0/2216331861 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9f1c1390a0 0x7f9f1c12d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:55.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.826+0000 7f9f1a59c700 1 -- 172.21.15.16:0/2216331861 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9f1c136780 msgr2=0x7f9f1c12d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:38:55.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.826+0000 7f9f1a59c700 1 --2- 172.21.15.16:0/2216331861 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9f1c136780 0x7f9f1c12d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:55.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.826+0000 7f9f1a59c700 1 -- 172.21.15.16:0/2216331861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f14009a70 con 0x7f9f1c1303c0 2023-09-30T22:38:55.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.826+0000 7f9f19d9b700 1 --2- 172.21.15.16:0/2216331861 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9f1c136780 0x7f9f1c12d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:38:55.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.826+0000 7f9f1ad9d700 1 --2- 172.21.15.16:0/2216331861 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9f1c1390a0 0x7f9f1c12d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:38:55.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.826+0000 7f9f1a59c700 1 --2- 172.21.15.16:0/2216331861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f1c1303c0 0x7f9f1c12cb40 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c00c470 tx=0x7f9f0c00c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:38:55.837 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.827+0000 7f9f0affd700 1 -- 172.21.15.16:0/2216331861 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f0c004c80 con 0x7f9f1c1303c0 2023-09-30T22:38:55.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.827+0000 7f9f0affd700 1 -- 172.21.15.16:0/2216331861 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f0c004de0 con 0x7f9f1c1303c0 2023-09-30T22:38:55.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.827+0000 7f9f20860700 1 -- 172.21.15.16:0/2216331861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f1c06ee70 con 0x7f9f1c1303c0 2023-09-30T22:38:55.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.827+0000 7f9f0affd700 1 -- 172.21.15.16:0/2216331861 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f0c007310 con 0x7f9f1c1303c0 2023-09-30T22:38:55.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.827+0000 7f9f20860700 1 -- 172.21.15.16:0/2216331861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f1c070d20 con 0x7f9f1c1303c0 2023-09-30T22:38:55.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.827+0000 7f9f0affd700 1 -- 172.21.15.16:0/2216331861 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9f0c007470 con 0x7f9f1c1303c0 2023-09-30T22:38:55.838 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.828+0000 7f9f0affd700 1 -- 172.21.15.16:0/2216331861 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f9f0c03f1a0 con 0x7f9f1c1303c0 2023-09-30T22:38:55.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.828+0000 7f9f20860700 1 -- 172.21.15.16:0/2216331861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f1c04f2a0 con 0x7f9f1c1303c0 2023-09-30T22:38:55.839 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:55.833+0000 7f9f0affd700 1 -- 172.21.15.16:0/2216331861 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f9f0c007720 con 0x7f9f1c1303c0 2023-09-30T22:38:56.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.035+0000 7f9f20860700 1 -- 172.21.15.16:0/2216331861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9f1c06c380 con 0x7f9f1c1303c0 2023-09-30T22:38:56.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.036+0000 7f9f0affd700 1 -- 172.21.15.16:0/2216331861 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v40) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9f0c011360 con 0x7f9f1c1303c0 2023-09-30T22:38:56.036 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 8 2023-09-30T22:38:56.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.038+0000 7f9f20860700 1 -- 172.21.15.16:0/2216331861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f1c1303c0 msgr2=0x7f9f1c12cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:38:56.038 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.038+0000 7f9f20860700 1 --2- 172.21.15.16:0/2216331861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f1c1303c0 0x7f9f1c12cb40 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f9f0c00c470 tx=0x7f9f0c00c780 comp rx=0 tx=0).stop 2023-09-30T22:38:56.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.038+0000 7f9f20860700 1 -- 172.21.15.16:0/2216331861 shutdown_connections 2023-09-30T22:38:56.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.038+0000 7f9f20860700 1 --2- 172.21.15.16:0/2216331861 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f9f1c1390a0 0x7f9f1c12d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:56.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.038+0000 7f9f20860700 1 --2- 172.21.15.16:0/2216331861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f1c1303c0 0x7f9f1c12cb40 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:56.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.038+0000 7f9f20860700 1 --2- 172.21.15.16:0/2216331861 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f9f1c136780 0x7f9f1c12d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:56.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.038+0000 7f9f20860700 1 -- 172.21.15.16:0/2216331861 >> 172.21.15.16:0/2216331861 conn(0x7f9f1c066d70 msgr2=0x7f9f1c09b200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:38:56.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.038+0000 7f9f20860700 1 -- 172.21.15.16:0/2216331861 shutdown_connections 2023-09-30T22:38:56.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.038+0000 7f9f20860700 1 -- 172.21.15.16:0/2216331861 wait complete. 2023-09-30T22:38:56.052 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-30T22:38:56.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.190+0000 7fcc3c8bd700 1 Processor -- start 2023-09-30T22:38:56.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.190+0000 7fcc3c8bd700 1 -- start start 2023-09-30T22:38:56.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.191+0000 7fcc3c8bd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc3809d7d0 0x7fcc3809dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:56.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.191+0000 7fcc3c8bd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcc38130e80 0x7fcc38138a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:56.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.191+0000 7fcc3c8bd700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc3809b800 con 0x7fcc3809e0e0 2023-09-30T22:38:56.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.191+0000 7fcc3c8bd700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc3809b620 con 0x7fcc38130e80 2023-09-30T22:38:56.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.191+0000 7fcc3c8bd700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcc3809f370 con 0x7fcc3809d7d0 2023-09-30T22:38:56.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.191+0000 7fcc3659c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc3809d7d0 0x7fcc3809dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:56.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.191+0000 7fcc36d9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcc38130e80 0x7fcc38138a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:56.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.191+0000 7fcc36d9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcc38130e80 0x7fcc38138a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:45996/0 (socket says 172.21.15.16:45996) 2023-09-30T22:38:56.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.191+0000 7fcc36d9d700 1 -- 172.21.15.16:0/1225388192 learned_addr learned my addr 172.21.15.16:0/1225388192 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:38:56.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.191+0000 7fcc3659c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc3809d7d0 0x7fcc3809dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37224/0 (socket says 172.21.15.16:37224) 2023-09-30T22:38:56.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.191+0000 7fcc35d9b700 1 --1- 172.21.15.16:0/1225388192 >> v1:172.21.15.16:6789/0 conn(0x7fcc3809e0e0 0x7fcc38130770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58880/0 (socket says 172.21.15.16:58880) 2023-09-30T22:38:56.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.191+0000 7fcc36d9d700 1 -- 172.21.15.16:0/1225388192 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc3809d7d0 msgr2=0x7fcc3809dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:38:56.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.191+0000 7fcc36d9d700 1 --2- 172.21.15.16:0/1225388192 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc3809d7d0 0x7fcc3809dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:56.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.191+0000 7fcc3559a700 1 -- 172.21.15.16:0/1225388192 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 314635887 0 0) 0x7fcc3809b800 con 0x7fcc3809e0e0 2023-09-30T22:38:56.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.191+0000 7fcc36d9d700 1 -- 172.21.15.16:0/1225388192 >> v1:172.21.15.16:6789/0 conn(0x7fcc3809e0e0 legacy=0x7fcc38130770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:38:56.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.191+0000 7fcc36d9d700 1 -- 172.21.15.16:0/1225388192 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc3809ef80 con 0x7fcc38130e80 2023-09-30T22:38:56.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.191+0000 7fcc3659c700 1 --2- 172.21.15.16:0/1225388192 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc3809d7d0 0x7fcc3809dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:38:56.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.191+0000 7fcc36d9d700 1 --2- 172.21.15.16:0/1225388192 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcc38130e80 0x7fcc38138a40 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fcc3000d180 tx=0x7fcc3000d490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=446183bc2a9a9515 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:38:56.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.192+0000 7fcc3559a700 1 -- 172.21.15.16:0/1225388192 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc30010070 con 0x7fcc38130e80 2023-09-30T22:38:56.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.192+0000 7fcc3559a700 1 -- 172.21.15.16:0/1225388192 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc3000dd40 con 0x7fcc38130e80 2023-09-30T22:38:56.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.192+0000 7fcc3559a700 1 -- 172.21.15.16:0/1225388192 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3000b310 con 0x7fcc38130e80 2023-09-30T22:38:56.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.192+0000 7fcc3c8bd700 1 -- 172.21.15.16:0/1225388192 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcc38130e80 msgr2=0x7fcc38138a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:38:56.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.192+0000 7fcc3c8bd700 1 --2- 172.21.15.16:0/1225388192 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcc38130e80 0x7fcc38138a40 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fcc3000d180 tx=0x7fcc3000d490 comp rx=0 tx=0).stop 2023-09-30T22:38:56.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.192+0000 7fcc3c8bd700 1 -- 172.21.15.16:0/1225388192 shutdown_connections 2023-09-30T22:38:56.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.192+0000 7fcc3c8bd700 1 --2- 172.21.15.16:0/1225388192 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcc38130e80 0x7fcc38138a40 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:56.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.192+0000 7fcc3c8bd700 1 --2- 172.21.15.16:0/1225388192 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc3809d7d0 0x7fcc3809dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:56.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.192+0000 7fcc3c8bd700 1 -- 172.21.15.16:0/1225388192 >> 172.21.15.16:0/1225388192 conn(0x7fcc38066d70 msgr2=0x7fcc380a2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:38:56.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.192+0000 7fcc3c8bd700 1 -- 172.21.15.16:0/1225388192 shutdown_connections 2023-09-30T22:38:56.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.192+0000 7fcc3c8bd700 1 -- 172.21.15.16:0/1225388192 wait complete. 2023-09-30T22:38:56.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.194+0000 7fcc3c8bd700 1 Processor -- start 2023-09-30T22:38:56.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.194+0000 7fcc3c8bd700 1 -- start start 2023-09-30T22:38:56.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.194+0000 7fcc3c8bd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc3809d7d0 0x7fcc38121360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:56.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.194+0000 7fcc3c8bd700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc3809e0e0 0x7fcc381218a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:56.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.194+0000 7fcc3c8bd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcc38130e80 0x7fcc38121de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:56.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.194+0000 7fcc3c8bd700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc38124ba0 con 0x7fcc3809e0e0 2023-09-30T22:38:56.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.194+0000 7fcc3c8bd700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc38123080 con 0x7fcc38130e80 2023-09-30T22:38:56.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.194+0000 7fcc3c8bd700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fcc380a1e80 con 0x7fcc3809d7d0 2023-09-30T22:38:56.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.194+0000 7fcc3659c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc3809d7d0 0x7fcc38121360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:56.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.194+0000 7fcc36d9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcc38130e80 0x7fcc38121de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:56.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.194+0000 7fcc36d9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcc38130e80 0x7fcc38121de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:46004/0 (socket says 172.21.15.16:46004) 2023-09-30T22:38:56.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.194+0000 7fcc3659c700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc3809d7d0 0x7fcc38121360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37240/0 (socket says 172.21.15.16:37240) 2023-09-30T22:38:56.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.194+0000 7fcc36d9d700 1 -- 172.21.15.16:0/110970633 learned_addr learned my addr 172.21.15.16:0/110970633 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:38:56.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.194+0000 7fcc35d9b700 1 --2- 172.21.15.16:0/110970633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc3809e0e0 0x7fcc381218a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:56.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.195+0000 7fcc36d9d700 1 -- 172.21.15.16:0/110970633 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc3809d7d0 msgr2=0x7fcc38121360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:38:56.206 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.195+0000 7fcc36d9d700 1 --2- 172.21.15.16:0/110970633 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc3809d7d0 0x7fcc38121360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:56.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.195+0000 7fcc36d9d700 1 -- 172.21.15.16:0/110970633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc3809e0e0 msgr2=0x7fcc381218a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:38:56.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.195+0000 7fcc36d9d700 1 --2- 172.21.15.16:0/110970633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc3809e0e0 0x7fcc381218a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:56.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.195+0000 7fcc36d9d700 1 -- 172.21.15.16:0/110970633 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc200094a0 con 0x7fcc38130e80 2023-09-30T22:38:56.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.195+0000 7fcc35d9b700 1 --2- 172.21.15.16:0/110970633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc3809e0e0 0x7fcc381218a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:38:56.207 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.195+0000 7fcc36d9d700 1 --2- 172.21.15.16:0/110970633 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcc38130e80 0x7fcc38121de0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fcc30009830 tx=0x7fcc30004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:38:56.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.195+0000 7fcc2effd700 1 -- 172.21.15.16:0/110970633 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc30009e20 con 0x7fcc38130e80 2023-09-30T22:38:56.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.195+0000 7fcc2effd700 1 -- 172.21.15.16:0/110970633 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc3001d430 con 0x7fcc38130e80 2023-09-30T22:38:56.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.195+0000 7fcc3c8bd700 1 -- 172.21.15.16:0/110970633 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc3000bd40 con 0x7fcc38130e80 2023-09-30T22:38:56.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.195+0000 7fcc2effd700 1 -- 172.21.15.16:0/110970633 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3001d7e0 con 0x7fcc38130e80 2023-09-30T22:38:56.208 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.195+0000 7fcc3c8bd700 1 -- 172.21.15.16:0/110970633 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc38125610 con 0x7fcc38130e80 2023-09-30T22:38:56.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.196+0000 7fcc2effd700 1 -- 172.21.15.16:0/110970633 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fcc3001d940 con 0x7fcc38130e80 2023-09-30T22:38:56.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.196+0000 7fcc2effd700 1 -- 172.21.15.16:0/110970633 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fcc30040fd0 con 0x7fcc38130e80 2023-09-30T22:38:56.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.196+0000 7fcc3c8bd700 1 -- 172.21.15.16:0/110970633 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc3804f2a0 con 0x7fcc38130e80 2023-09-30T22:38:56.209 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.201+0000 7fcc2effd700 1 -- 172.21.15.16:0/110970633 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fcc30027be0 con 0x7fcc38130e80 2023-09-30T22:38:56.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.404+0000 7fcc3c8bd700 1 -- 172.21.15.16:0/110970633 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcc3806c380 con 0x7fcc38130e80 2023-09-30T22:38:56.405 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.404+0000 7fcc2effd700 1 -- 172.21.15.16:0/110970633 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v40) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcc300273b0 con 0x7fcc38130e80 2023-09-30T22:38:56.406 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-09-30T22:38:56.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.407+0000 7fcc3c8bd700 1 -- 172.21.15.16:0/110970633 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcc38130e80 msgr2=0x7fcc38121de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:38:56.408 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.407+0000 7fcc3c8bd700 1 --2- 172.21.15.16:0/110970633 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcc38130e80 0x7fcc38121de0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fcc30009830 tx=0x7fcc30004ad0 comp rx=0 tx=0).stop 2023-09-30T22:38:56.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.407+0000 7fcc3c8bd700 1 -- 172.21.15.16:0/110970633 shutdown_connections 2023-09-30T22:38:56.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.407+0000 7fcc3c8bd700 1 --2- 172.21.15.16:0/110970633 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fcc38130e80 0x7fcc38121de0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:56.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.407+0000 7fcc3c8bd700 1 --2- 172.21.15.16:0/110970633 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fcc3809d7d0 0x7fcc38121360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:56.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.407+0000 7fcc3c8bd700 1 --2- 172.21.15.16:0/110970633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc3809e0e0 0x7fcc381218a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:56.410 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.407+0000 7fcc3c8bd700 1 -- 172.21.15.16:0/110970633 >> 172.21.15.16:0/110970633 conn(0x7fcc38066d70 msgr2=0x7fcc380721d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:38:56.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.407+0000 7fcc3c8bd700 1 -- 172.21.15.16:0/110970633 shutdown_connections 2023-09-30T22:38:56.411 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.407+0000 7fcc3c8bd700 1 -- 172.21.15.16:0/110970633 wait complete. 2023-09-30T22:38:56.422 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-30T22:38:56.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.561+0000 7f559a9d2700 1 Processor -- start 2023-09-30T22:38:56.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.561+0000 7f559a9d2700 1 -- start start 2023-09-30T22:38:56.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.561+0000 7f559a9d2700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5594133cc0 0x7f5594134090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:56.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.561+0000 7f559a9d2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55941345d0 0x7f55940a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:56.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.561+0000 7f559a9d2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f55940a0f70 0x7f55940a1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:56.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.562+0000 7f559a9d2700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55940a0640 con 0x7f5594133cc0 2023-09-30T22:38:56.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.562+0000 7f559a9d2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f559409e1f0 con 0x7f55941345d0 2023-09-30T22:38:56.564 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.562+0000 7f559a9d2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f55940a23b0 con 0x7f55940a0f70 2023-09-30T22:38:56.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.562+0000 7f5593fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5594133cc0 0x7f5594134090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:56.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.562+0000 7f5598f6f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f55940a0f70 0x7f55940a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:56.565 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.562+0000 7f5593fff700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f5594133cc0 0x7f5594134090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48120/0 (socket says 172.21.15.16:48120) 2023-09-30T22:38:56.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.562+0000 7f5593fff700 1 -- 172.21.15.16:0/1891920698 learned_addr learned my addr 172.21.15.16:0/1891920698 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:38:56.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.562+0000 7f5598f6f700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f55940a0f70 0x7f55940a1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37254/0 (socket says 172.21.15.16:37254) 2023-09-30T22:38:56.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.562+0000 7f55937fe700 1 --2- 172.21.15.16:0/1891920698 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55941345d0 0x7f55940a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:56.566 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.562+0000 7f5593fff700 1 -- 172.21.15.16:0/1891920698 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f55940a0f70 msgr2=0x7f55940a1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:38:56.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.562+0000 7f5593fff700 1 --2- 172.21.15.16:0/1891920698 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f55940a0f70 0x7f55940a1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:56.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.562+0000 7f5593fff700 1 -- 172.21.15.16:0/1891920698 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55941345d0 msgr2=0x7f55940a0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:38:56.567 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.562+0000 7f5593fff700 1 --2- 172.21.15.16:0/1891920698 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55941345d0 0x7f55940a0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:56.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.562+0000 7f5593fff700 1 -- 172.21.15.16:0/1891920698 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f559409df30 con 0x7f5594133cc0 2023-09-30T22:38:56.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.562+0000 7f5598f6f700 1 --2- 172.21.15.16:0/1891920698 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f55940a0f70 0x7f55940a1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:38:56.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.562+0000 7f55937fe700 1 --2- 172.21.15.16:0/1891920698 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55941345d0 0x7f55940a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:38:56.568 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.562+0000 7f5593fff700 1 --2- 172.21.15.16:0/1891920698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5594133cc0 0x7f5594134090 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f5584009920 tx=0x7f5584009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4787f5a440013d60 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:38:56.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.562+0000 7f5592ffd700 1 -- 172.21.15.16:0/1891920698 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5584011830 con 0x7f5594133cc0 2023-09-30T22:38:56.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.562+0000 7f5592ffd700 1 -- 172.21.15.16:0/1891920698 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5584011990 con 0x7f5594133cc0 2023-09-30T22:38:56.569 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.563+0000 7f5592ffd700 1 -- 172.21.15.16:0/1891920698 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5584011d20 con 0x7f5594133cc0 2023-09-30T22:38:56.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.563+0000 7f559a9d2700 1 -- 172.21.15.16:0/1891920698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5594133cc0 msgr2=0x7f5594134090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:38:56.574 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.563+0000 7f559a9d2700 1 --2- 172.21.15.16:0/1891920698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5594133cc0 0x7f5594134090 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f5584009920 tx=0x7f5584009c30 comp rx=0 tx=0).stop 2023-09-30T22:38:56.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.563+0000 7f559a9d2700 1 -- 172.21.15.16:0/1891920698 shutdown_connections 2023-09-30T22:38:56.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.563+0000 7f559a9d2700 1 --2- 172.21.15.16:0/1891920698 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f55940a0f70 0x7f55940a1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:56.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.563+0000 7f559a9d2700 1 --2- 172.21.15.16:0/1891920698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5594133cc0 0x7f5594134090 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:56.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.563+0000 7f559a9d2700 1 --2- 172.21.15.16:0/1891920698 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55941345d0 0x7f55940a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:56.575 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.563+0000 7f559a9d2700 1 -- 172.21.15.16:0/1891920698 >> 172.21.15.16:0/1891920698 conn(0x7f55940a3050 msgr2=0x7f55940a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:38:56.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.563+0000 7f559a9d2700 1 -- 172.21.15.16:0/1891920698 shutdown_connections 2023-09-30T22:38:56.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.563+0000 7f559a9d2700 1 -- 172.21.15.16:0/1891920698 wait complete. 2023-09-30T22:38:56.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.564+0000 7f559a9d2700 1 Processor -- start 2023-09-30T22:38:56.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.564+0000 7f559a9d2700 1 -- start start 2023-09-30T22:38:56.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.565+0000 7f559a9d2700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f55940a0f70 0x7f5594122490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:56.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.565+0000 7f559a9d2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5594133cc0 0x7f55941229d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:56.576 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.565+0000 7f559a9d2700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55941345d0 0x7f5594122f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:56.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.565+0000 7f559a9d2700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5594124800 con 0x7f5594133cc0 2023-09-30T22:38:56.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.565+0000 7f559a9d2700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f559409ff30 con 0x7f55941345d0 2023-09-30T22:38:56.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.565+0000 7f559a9d2700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f5594124e30 con 0x7f55940a0f70 2023-09-30T22:38:56.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.565+0000 7f5598f6f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55941345d0 0x7f5594122f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:56.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.565+0000 7f55937fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5594133cc0 0x7f55941229d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:56.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.565+0000 7f5598f6f700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55941345d0 0x7f5594122f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:46028/0 (socket says 172.21.15.16:46028) 2023-09-30T22:38:56.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.565+0000 7f55937fe700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5594133cc0 0x7f55941229d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48128/0 (socket says 172.21.15.16:48128) 2023-09-30T22:38:56.577 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.565+0000 7f5598f6f700 1 -- 172.21.15.16:0/527555834 learned_addr learned my addr 172.21.15.16:0/527555834 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:38:56.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.565+0000 7f5593fff700 1 --2- 172.21.15.16:0/527555834 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f55940a0f70 0x7f5594122490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:56.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.565+0000 7f5598f6f700 1 -- 172.21.15.16:0/527555834 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f55940a0f70 msgr2=0x7f5594122490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:38:56.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.565+0000 7f5598f6f700 1 --2- 172.21.15.16:0/527555834 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f55940a0f70 0x7f5594122490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:56.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.565+0000 7f5598f6f700 1 -- 172.21.15.16:0/527555834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5594133cc0 msgr2=0x7f55941229d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:38:56.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.565+0000 7f5598f6f700 1 --2- 172.21.15.16:0/527555834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5594133cc0 0x7f55941229d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:56.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.565+0000 7f5598f6f700 1 -- 172.21.15.16:0/527555834 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f558c009a70 con 0x7f55941345d0 2023-09-30T22:38:56.578 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.565+0000 7f55937fe700 1 --2- 172.21.15.16:0/527555834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5594133cc0 0x7f55941229d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:38:56.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.565+0000 7f5598f6f700 1 --2- 172.21.15.16:0/527555834 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55941345d0 0x7f5594122f10 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f558c00efc0 tx=0x7f558c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:38:56.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.566+0000 7f5590ff9700 1 -- 172.21.15.16:0/527555834 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f558c00ca90 con 0x7f55941345d0 2023-09-30T22:38:56.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.566+0000 7f5590ff9700 1 -- 172.21.15.16:0/527555834 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f558c00f040 con 0x7f55941345d0 2023-09-30T22:38:56.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.566+0000 7f559a9d2700 1 -- 172.21.15.16:0/527555834 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f557c0094a0 con 0x7f55941345d0 2023-09-30T22:38:56.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.566+0000 7f5590ff9700 1 -- 172.21.15.16:0/527555834 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f558c00ce30 con 0x7f55941345d0 2023-09-30T22:38:56.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.566+0000 7f559a9d2700 1 -- 172.21.15.16:0/527555834 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5584009550 con 0x7f55941345d0 2023-09-30T22:38:56.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.566+0000 7f5590ff9700 1 -- 172.21.15.16:0/527555834 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f558c014480 con 0x7f55941345d0 2023-09-30T22:38:56.579 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.567+0000 7f5590ff9700 1 -- 172.21.15.16:0/527555834 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f558c03fe40 con 0x7f55941345d0 2023-09-30T22:38:56.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.567+0000 7f559a9d2700 1 -- 172.21.15.16:0/527555834 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f559404f2a0 con 0x7f55941345d0 2023-09-30T22:38:56.580 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.572+0000 7f5590ff9700 1 -- 172.21.15.16:0/527555834 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f558c012930 con 0x7f55941345d0 2023-09-30T22:38:56.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.778+0000 7f559a9d2700 1 -- 172.21.15.16:0/527555834 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f559406d570 con 0x7f55941345d0 2023-09-30T22:38:56.778 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.778+0000 7f5590ff9700 1 -- 172.21.15.16:0/527555834 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v40) v1 ==== 100+0+11 (secure 0 0 0) 0x7f558c02a0a0 con 0x7f55941345d0 2023-09-30T22:38:56.779 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-09-30T22:38:56.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.780+0000 7f559a9d2700 1 -- 172.21.15.16:0/527555834 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55941345d0 msgr2=0x7f5594122f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:38:56.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.780+0000 7f559a9d2700 1 --2- 172.21.15.16:0/527555834 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55941345d0 0x7f5594122f10 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f558c00efc0 tx=0x7f558c00c3a0 comp rx=0 tx=0).stop 2023-09-30T22:38:56.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.780+0000 7f559a9d2700 1 -- 172.21.15.16:0/527555834 shutdown_connections 2023-09-30T22:38:56.781 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.781+0000 7f559a9d2700 1 --2- 172.21.15.16:0/527555834 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f55941345d0 0x7f5594122f10 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:56.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.781+0000 7f559a9d2700 1 --2- 172.21.15.16:0/527555834 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f55940a0f70 0x7f5594122490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:56.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.781+0000 7f559a9d2700 1 --2- 172.21.15.16:0/527555834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5594133cc0 0x7f55941229d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:56.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.781+0000 7f559a9d2700 1 -- 172.21.15.16:0/527555834 >> 172.21.15.16:0/527555834 conn(0x7f55940a3050 msgr2=0x7f559409a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:38:56.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.781+0000 7f559a9d2700 1 -- 172.21.15.16:0/527555834 shutdown_connections 2023-09-30T22:38:56.782 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.781+0000 7f559a9d2700 1 -- 172.21.15.16:0/527555834 wait complete. 2023-09-30T22:38:56.795 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-30T22:38:56.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.932+0000 7f96533c0700 1 Processor -- start 2023-09-30T22:38:56.933 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.932+0000 7f96533c0700 1 -- start start 2023-09-30T22:38:56.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.932+0000 7f96533c0700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f964c09e930 0x7f964c09ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:56.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.932+0000 7f96533c0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f964c09f240 0x7f964c1317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:56.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.933+0000 7f96533c0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f964c09f650 0x7f964c131d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:56.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.933+0000 7f96533c0700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f964c070940 con 0x7f964c09e930 2023-09-30T22:38:56.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.933+0000 7f96533c0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f964c06fd60 con 0x7f964c09f240 2023-09-30T22:38:56.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.933+0000 7f96533c0700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f964c06fa80 con 0x7f964c09f650 2023-09-30T22:38:56.935 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.933+0000 7f965095b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f964c09f240 0x7f964c1317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:56.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.933+0000 7f965115c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f964c09e930 0x7f964c09ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:56.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.933+0000 7f965195d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f964c09f650 0x7f964c131d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:56.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.933+0000 7f965115c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f964c09e930 0x7f964c09ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48134/0 (socket says 172.21.15.16:48134) 2023-09-30T22:38:56.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.933+0000 7f965195d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f964c09f650 0x7f964c131d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37272/0 (socket says 172.21.15.16:37272) 2023-09-30T22:38:56.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.933+0000 7f965115c700 1 -- 172.21.15.16:0/36363638 learned_addr learned my addr 172.21.15.16:0/36363638 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:38:56.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.933+0000 7f965095b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f964c09f240 0x7f964c1317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:46032/0 (socket says 172.21.15.16:46032) 2023-09-30T22:38:56.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.933+0000 7f965115c700 1 -- 172.21.15.16:0/36363638 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f964c09f650 msgr2=0x7f964c131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:38:56.936 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.933+0000 7f965115c700 1 --2- 172.21.15.16:0/36363638 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f964c09f650 0x7f964c131d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:56.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.933+0000 7f965115c700 1 -- 172.21.15.16:0/36363638 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f964c09f240 msgr2=0x7f964c1317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:38:56.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.933+0000 7f965115c700 1 --2- 172.21.15.16:0/36363638 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f964c09f240 0x7f964c1317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:56.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.933+0000 7f965115c700 1 -- 172.21.15.16:0/36363638 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f964c09fb60 con 0x7f964c09e930 2023-09-30T22:38:56.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.933+0000 7f965195d700 1 --2- 172.21.15.16:0/36363638 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f964c09f650 0x7f964c131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:38:56.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.933+0000 7f965115c700 1 --2- 172.21.15.16:0/36363638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f964c09e930 0x7f964c09ed00 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f96480098f0 tx=0x7f9648009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b9ea72c2b697cf1 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:38:56.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.933+0000 7f9643fff700 1 -- 172.21.15.16:0/36363638 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9648011830 con 0x7f964c09e930 2023-09-30T22:38:56.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.933+0000 7f9643fff700 1 -- 172.21.15.16:0/36363638 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9648011990 con 0x7f964c09e930 2023-09-30T22:38:56.937 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.934+0000 7f9643fff700 1 -- 172.21.15.16:0/36363638 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9648011d20 con 0x7f964c09e930 2023-09-30T22:38:56.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.934+0000 7f96533c0700 1 -- 172.21.15.16:0/36363638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f964c09e930 msgr2=0x7f964c09ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:38:56.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.934+0000 7f96533c0700 1 --2- 172.21.15.16:0/36363638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f964c09e930 0x7f964c09ed00 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f96480098f0 tx=0x7f9648009c00 comp rx=0 tx=0).stop 2023-09-30T22:38:56.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.934+0000 7f96533c0700 1 -- 172.21.15.16:0/36363638 shutdown_connections 2023-09-30T22:38:56.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.934+0000 7f96533c0700 1 --2- 172.21.15.16:0/36363638 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f964c09f650 0x7f964c131d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:56.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.934+0000 7f96533c0700 1 --2- 172.21.15.16:0/36363638 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f964c09e930 0x7f964c09ed00 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:56.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.934+0000 7f96533c0700 1 --2- 172.21.15.16:0/36363638 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f964c09f240 0x7f964c1317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:56.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.934+0000 7f96533c0700 1 -- 172.21.15.16:0/36363638 >> 172.21.15.16:0/36363638 conn(0x7f964c0a4210 msgr2=0x7f964c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:38:56.938 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.934+0000 7f96533c0700 1 -- 172.21.15.16:0/36363638 shutdown_connections 2023-09-30T22:38:56.939 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.934+0000 7f96533c0700 1 -- 172.21.15.16:0/36363638 wait complete. 2023-09-30T22:38:56.940 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.935+0000 7f96533c0700 1 Processor -- start 2023-09-30T22:38:56.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.935+0000 7f96533c0700 1 -- start start 2023-09-30T22:38:56.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.936+0000 7f96533c0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f964c09e930 0x7f964c1a7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:56.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.936+0000 7f96533c0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f964c09f240 0x7f964c1a7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:56.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.936+0000 7f96533c0700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f964c09f650 0x7f964c1a7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:56.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.936+0000 7f96533c0700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f964c1aad50 con 0x7f964c09e930 2023-09-30T22:38:56.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.936+0000 7f96533c0700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f964c1aa840 con 0x7f964c09f240 2023-09-30T22:38:56.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.936+0000 7f96533c0700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f964c1a9cd0 con 0x7f964c09f650 2023-09-30T22:38:56.941 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.936+0000 7f965095b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f964c09f240 0x7f964c1a7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:56.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.936+0000 7f965115c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f964c09e930 0x7f964c1a7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:56.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.936+0000 7f965195d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f964c09f650 0x7f964c1a7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:56.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.936+0000 7f965095b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f964c09f240 0x7f964c1a7770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:46046/0 (socket says 172.21.15.16:46046) 2023-09-30T22:38:56.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.936+0000 7f965095b700 1 -- 172.21.15.16:0/1399232132 learned_addr learned my addr 172.21.15.16:0/1399232132 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:38:56.942 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.936+0000 7f965115c700 1 -- 172.21.15.16:0/1399232132 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f964c09f650 msgr2=0x7f964c1a7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:38:56.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.936+0000 7f965115c700 1 --2- 172.21.15.16:0/1399232132 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f964c09f650 0x7f964c1a7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:56.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.936+0000 7f965115c700 1 -- 172.21.15.16:0/1399232132 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f964c09f240 msgr2=0x7f964c1a7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:38:56.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.936+0000 7f965115c700 1 --2- 172.21.15.16:0/1399232132 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f964c09f240 0x7f964c1a7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:56.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.936+0000 7f965115c700 1 -- 172.21.15.16:0/1399232132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9644009a70 con 0x7f964c09e930 2023-09-30T22:38:56.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.936+0000 7f965195d700 1 --2- 172.21.15.16:0/1399232132 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f964c09f650 0x7f964c1a7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:38:56.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.936+0000 7f965095b700 1 --2- 172.21.15.16:0/1399232132 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f964c09f240 0x7f964c1a7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:38:56.943 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.936+0000 7f965115c700 1 --2- 172.21.15.16:0/1399232132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f964c09e930 0x7f964c1a7230 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f9648016000 tx=0x7f9648004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:38:56.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.937+0000 7f9641ffb700 1 -- 172.21.15.16:0/1399232132 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9648004240 con 0x7f964c09e930 2023-09-30T22:38:56.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.937+0000 7f9641ffb700 1 -- 172.21.15.16:0/1399232132 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f964801f070 con 0x7f964c09e930 2023-09-30T22:38:56.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.937+0000 7f9641ffb700 1 -- 172.21.15.16:0/1399232132 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96480036a0 con 0x7f964c09e930 2023-09-30T22:38:56.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.937+0000 7f96533c0700 1 -- 172.21.15.16:0/1399232132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9648009550 con 0x7f964c09e930 2023-09-30T22:38:56.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.937+0000 7f96533c0700 1 -- 172.21.15.16:0/1399232132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f964c0a0190 con 0x7f964c09e930 2023-09-30T22:38:56.944 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.938+0000 7f9641ffb700 1 -- 172.21.15.16:0/1399232132 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f9648021030 con 0x7f964c09e930 2023-09-30T22:38:56.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.938+0000 7f9641ffb700 1 -- 172.21.15.16:0/1399232132 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f9648040660 con 0x7f964c09e930 2023-09-30T22:38:56.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.938+0000 7f96533c0700 1 -- 172.21.15.16:0/1399232132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f964c0a3580 con 0x7f964c09e930 2023-09-30T22:38:56.945 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:56.943+0000 7f9641ffb700 1 -- 172.21.15.16:0/1399232132 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f96480239c0 con 0x7f964c09e930 2023-09-30T22:38:57.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.160+0000 7f96533c0700 1 -- 172.21.15.16:0/1399232132 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f964c074ea0 con 0x7f964c09e930 2023-09-30T22:38:57.161 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.161+0000 7f9641ffb700 1 -- 172.21.15.16:0/1399232132 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 11 entries v40) v1 ==== 81+0+639 (secure 0 0 0) 0x7f96480233d0 con 0x7f964c09e930 2023-09-30T22:38:57.162 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3403658744 2023-10-01T22:30:13.497576+0000 2023-09-30T22:38:57.162 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/1216430525 2023-10-01T22:30:13.497576+0000 2023-09-30T22:38:57.162 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3418827610 2023-10-01T22:30:13.497576+0000 2023-09-30T22:38:57.163 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/1101648925 2023-10-01T22:29:14.475978+0000 2023-09-30T22:38:57.163 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/2249754484 2023-10-01T22:29:14.475978+0000 2023-09-30T22:38:57.163 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/1879378585 2023-10-01T22:30:13.497576+0000 2023-09-30T22:38:57.163 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6816/118285 2023-10-01T22:29:14.475978+0000 2023-09-30T22:38:57.164 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6816/126811 2023-10-01T22:30:13.497576+0000 2023-09-30T22:38:57.164 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6817/1424366610 2023-10-01T22:29:40.259230+0000 2023-09-30T22:38:57.164 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/2178963721 2023-10-01T22:30:13.497576+0000 2023-09-30T22:38:57.164 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3146063150 2023-10-01T22:30:13.497576+0000 2023-09-30T22:38:57.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.163+0000 7f96533c0700 1 -- 172.21.15.16:0/1399232132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f964c09e930 msgr2=0x7f964c1a7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:38:57.166 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.163+0000 7f96533c0700 1 --2- 172.21.15.16:0/1399232132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f964c09e930 0x7f964c1a7230 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f9648016000 tx=0x7f9648004750 comp rx=0 tx=0).stop 2023-09-30T22:38:57.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.163+0000 7f96533c0700 1 -- 172.21.15.16:0/1399232132 shutdown_connections 2023-09-30T22:38:57.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.163+0000 7f96533c0700 1 --2- 172.21.15.16:0/1399232132 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f964c09f650 0x7f964c1a7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:57.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.163+0000 7f96533c0700 1 --2- 172.21.15.16:0/1399232132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f964c09e930 0x7f964c1a7230 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:57.167 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.163+0000 7f96533c0700 1 --2- 172.21.15.16:0/1399232132 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f964c09f240 0x7f964c1a7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:57.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.163+0000 7f96533c0700 1 -- 172.21.15.16:0/1399232132 >> 172.21.15.16:0/1399232132 conn(0x7f964c0a4210 msgr2=0x7f964c06b880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:38:57.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.163+0000 7f96533c0700 1 -- 172.21.15.16:0/1399232132 shutdown_connections 2023-09-30T22:38:57.168 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.163+0000 7f96533c0700 1 -- 172.21.15.16:0/1399232132 wait complete. 2023-09-30T22:38:57.168 INFO:teuthology.orchestra.run.smithi016.stderr:listed 11 entries 2023-09-30T22:38:57.177 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-09-30T22:38:57.177 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-09-30T22:38:57.200 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-09-30T22:38:57.337 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.337+0000 7efc28356700 1 Processor -- start 2023-09-30T22:38:57.338 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.337+0000 7efc28356700 1 -- start start 2023-09-30T22:38:57.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.338+0000 7efc28356700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc2009f410 0x7efc20136aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:57.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.338+0000 7efc28356700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efc20136fe0 0x7efc201393c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:57.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.338+0000 7efc28356700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc20070940 con 0x7efc2009e930 2023-09-30T22:38:57.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.338+0000 7efc28356700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc2006fd60 con 0x7efc20136fe0 2023-09-30T22:38:57.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.338+0000 7efc28356700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efc2006fa80 con 0x7efc2009f410 2023-09-30T22:38:57.340 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.338+0000 7efc268f3700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7efc20136fe0 0x7efc201393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:57.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.338+0000 7efc258f1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc2009f410 0x7efc20136aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:57.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.338+0000 7efc260f2700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7efc2009e930 0x7efc2009ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:58884/0 (socket says 172.21.15.16:58884) 2023-09-30T22:38:57.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.338+0000 7efc260f2700 1 -- 172.21.15.16:0/3916112842 learned_addr learned my addr 172.21.15.16:0/3916112842 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:38:57.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.338+0000 7efc268f3700 1 -- 172.21.15.16:0/3916112842 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc2009f410 msgr2=0x7efc20136aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:38:57.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.338+0000 7efc268f3700 1 --2- 172.21.15.16:0/3916112842 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc2009f410 0x7efc20136aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:57.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.338+0000 7efc268f3700 1 -- 172.21.15.16:0/3916112842 >> v1:172.21.15.16:6789/0 conn(0x7efc2009e930 legacy=0x7efc2009ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:38:57.341 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.338+0000 7efc250f0700 1 -- 172.21.15.16:0/3916112842 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4189957563 0 0) 0x7efc20070940 con 0x7efc2009e930 2023-09-30T22:38:57.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.338+0000 7efc268f3700 1 -- 172.21.15.16:0/3916112842 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc2009fb60 con 0x7efc20136fe0 2023-09-30T22:38:57.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.338+0000 7efc258f1700 1 --2- 172.21.15.16:0/3916112842 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc2009f410 0x7efc20136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:38:57.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.339+0000 7efc268f3700 1 --2- 172.21.15.16:0/3916112842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc20136fe0 0x7efc201393c0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7efc18009d90 tx=0x7efc1800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d770906c6074b257 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:38:57.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.339+0000 7efc250f0700 1 -- 172.21.15.16:0/3916112842 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1800e070 con 0x7efc20136fe0 2023-09-30T22:38:57.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.339+0000 7efc250f0700 1 -- 172.21.15.16:0/3916112842 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc1800bb20 con 0x7efc20136fe0 2023-09-30T22:38:57.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.339+0000 7efc250f0700 1 -- 172.21.15.16:0/3916112842 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc18012430 con 0x7efc20136fe0 2023-09-30T22:38:57.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.339+0000 7efc28356700 1 -- 172.21.15.16:0/3916112842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc20136fe0 msgr2=0x7efc201393c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:38:57.342 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.339+0000 7efc28356700 1 --2- 172.21.15.16:0/3916112842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc20136fe0 0x7efc201393c0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7efc18009d90 tx=0x7efc1800b330 comp rx=0 tx=0).stop 2023-09-30T22:38:57.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.339+0000 7efc28356700 1 -- 172.21.15.16:0/3916112842 shutdown_connections 2023-09-30T22:38:57.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.339+0000 7efc28356700 1 --2- 172.21.15.16:0/3916112842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc20136fe0 0x7efc201393c0 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:57.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.339+0000 7efc28356700 1 --2- 172.21.15.16:0/3916112842 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc2009f410 0x7efc20136aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:57.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.339+0000 7efc28356700 1 -- 172.21.15.16:0/3916112842 >> 172.21.15.16:0/3916112842 conn(0x7efc200a4210 msgr2=0x7efc200a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:38:57.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.339+0000 7efc28356700 1 -- 172.21.15.16:0/3916112842 shutdown_connections 2023-09-30T22:38:57.343 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.340+0000 7efc28356700 1 -- 172.21.15.16:0/3916112842 wait complete. 2023-09-30T22:38:57.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.340+0000 7efc28356700 1 Processor -- start 2023-09-30T22:38:57.345 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.340+0000 7efc28356700 1 -- start start 2023-09-30T22:38:57.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.341+0000 7efc28356700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc2009e930 0x7efc201c8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:57.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.341+0000 7efc28356700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc2009f410 0x7efc201c8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:57.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.341+0000 7efc28356700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efc20136fe0 0x7efc2006b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:57.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.341+0000 7efc28356700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc20132560 con 0x7efc2009e930 2023-09-30T22:38:57.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.341+0000 7efc28356700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc2006d640 con 0x7efc20136fe0 2023-09-30T22:38:57.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.341+0000 7efc28356700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7efc20120a50 con 0x7efc2009f410 2023-09-30T22:38:57.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.341+0000 7efc260f2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc2009e930 0x7efc201c8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:57.346 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.341+0000 7efc258f1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc2009f410 0x7efc201c8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:57.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.341+0000 7efc260f2700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc2009e930 0x7efc201c8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48150/0 (socket says 172.21.15.16:48150) 2023-09-30T22:38:57.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.341+0000 7efc258f1700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc2009f410 0x7efc201c8a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:37292/0 (socket says 172.21.15.16:37292) 2023-09-30T22:38:57.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.341+0000 7efc260f2700 1 -- 172.21.15.16:0/901639715 learned_addr learned my addr 172.21.15.16:0/901639715 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:38:57.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.341+0000 7efc268f3700 1 --2- 172.21.15.16:0/901639715 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efc20136fe0 0x7efc2006b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:57.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.341+0000 7efc260f2700 1 -- 172.21.15.16:0/901639715 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc2009f410 msgr2=0x7efc201c8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:38:57.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.341+0000 7efc260f2700 1 --2- 172.21.15.16:0/901639715 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc2009f410 0x7efc201c8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:57.347 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.341+0000 7efc260f2700 1 -- 172.21.15.16:0/901639715 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efc20136fe0 msgr2=0x7efc2006b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:38:57.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.341+0000 7efc260f2700 1 --2- 172.21.15.16:0/901639715 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efc20136fe0 0x7efc2006b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:57.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.341+0000 7efc260f2700 1 -- 172.21.15.16:0/901639715 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc140094a0 con 0x7efc2009e930 2023-09-30T22:38:57.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.341+0000 7efc258f1700 1 --2- 172.21.15.16:0/901639715 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc2009f410 0x7efc201c8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:38:57.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.342+0000 7efc260f2700 1 --2- 172.21.15.16:0/901639715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc2009e930 0x7efc201c8520 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7efc1c00c550 tx=0x7efc1c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:38:57.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.342+0000 7efc12ffd700 1 -- 172.21.15.16:0/901639715 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1c004c80 con 0x7efc2009e930 2023-09-30T22:38:57.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.342+0000 7efc12ffd700 1 -- 172.21.15.16:0/901639715 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc1c004e00 con 0x7efc2009e930 2023-09-30T22:38:57.348 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.342+0000 7efc28356700 1 -- 172.21.15.16:0/901639715 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc18009a70 con 0x7efc2009e930 2023-09-30T22:38:57.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.342+0000 7efc12ffd700 1 -- 172.21.15.16:0/901639715 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc1c007310 con 0x7efc2009e930 2023-09-30T22:38:57.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.342+0000 7efc28356700 1 -- 172.21.15.16:0/901639715 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc201c8fa0 con 0x7efc2009e930 2023-09-30T22:38:57.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.343+0000 7efc12ffd700 1 -- 172.21.15.16:0/901639715 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7efc1c007470 con 0x7efc2009e930 2023-09-30T22:38:57.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.343+0000 7efc12ffd700 1 -- 172.21.15.16:0/901639715 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7efc1c03f320 con 0x7efc2009e930 2023-09-30T22:38:57.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.343+0000 7efc28356700 1 -- 172.21.15.16:0/901639715 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc20071d20 con 0x7efc2009e930 2023-09-30T22:38:57.349 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.348+0000 7efc12ffd700 1 -- 172.21.15.16:0/901639715 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7efc1c0123b0 con 0x7efc2009e930 2023-09-30T22:38:57.558 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.558+0000 7efc28356700 1 -- 172.21.15.16:0/901639715 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efc20074ea0 con 0x7efc2009e930 2023-09-30T22:38:57.559 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.559+0000 7efc12ffd700 1 -- 172.21.15.16:0/901639715 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+13447 (secure 0 0 0) 0x7efc1c011360 con 0x7efc2009e930 2023-09-30T22:38:57.559 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-09-30T22:38:57.560 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":40,"fsid":"7ede3e14-2bd0-4726-9437-6d13c90cd342","created":"2023-09-30T22:26:35.483697+0000","modified":"2023-09-30T22:30:13.497628+0000","last_up_change":"2023-09-30T22:29:35.702639+0000","last_in_change":"2023-09-30T22:26:37.120906+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":4,"require_min_compat_client":"pacific","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-09-30T22:26:44.285133+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{},"mgr_devicehealth":{}},"read_balance":{"score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-09-30T22:26:45.132921+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-09-30T22:26:49.353683+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-09-30T22:26:50.655685+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"19946985-0ee2-487f-9347-929670dbe5ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":22,"up_from":28,"up_thru":32,"down_at":23,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6804","nonce":129539}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6805","nonce":129539}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6807","nonce":129539}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6806","nonce":129539}]},"public_addr":"172.21.15.16:6804/129539","cluster_addr":"172.21.15.16:6805/129539","heartbeat_back_addr":"172.21.15.16:6807/129539","heartbeat_front_addr":"172.21.15.16:6806/129539","state":["exists","up"]},{"osd":1,"uuid":"8efd4dbc-6ae4-4b82-b530-3b3eac2be445","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":22,"up_from":29,"up_thru":32,"down_at":24,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6808","nonce":129637}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6809","nonce":129637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6811","nonce":129637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6810","nonce":129637}]},"public_addr":"172.21.15.16:6808/129637","cluster_addr":"172.21.15.16:6809/129637","heartbeat_back_addr":"172.21.15.16:6811/129637","heartbeat_front_addr":"172.21.15.16:6810/129637","state":["exists","up"]},{"osd":2,"uuid":"b47bb1ce-27af-4c0c-868a-b76ebd0d9499","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":24,"up_from":31,"up_thru":32,"down_at":25,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6800","nonce":129747}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6801","nonce":129747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6803","nonce":129747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6802","nonce":129747}]},"public_addr":"172.21.15.16:6800/129747","cluster_addr":"172.21.15.16:6801/129747","heartbeat_back_addr":"172.21.15.16:6803/129747","heartbeat_front_addr":"172.21.15.16:6802/129747","state":["exists","up"]},{"osd":3,"uuid":"03b53d64-d10d-448a-a081-39ecceab589b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":25,"up_from":32,"up_thru":32,"down_at":26,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6812","nonce":130087}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6813","nonce":130087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6815","nonce":130087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.16:6814","nonce":130087}]},"public_addr":"172.21.15.16:6812/130087","cluster_addr":"172.21.15.16:6813/130087","heartbeat_back_addr":"172.21.15.16:6815/130087","heartbeat_front_addr":"172.21.15.16:6814/130087","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.305953+0000","dead_epoch":23},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.307488+0000","dead_epoch":24},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.341023+0000","dead_epoch":25},{"osd":3,"down_stamp":"2023-09-30T22:29:28.519844+0000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-09-30T22:26:42.192071+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.8","mappings":[{"from":3,"to":2}]},{"pgid":"4.d","mappings":[{"from":3,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":0}]},{"pgid":"4.14","mappings":[{"from":3,"to":2}]},{"pgid":"4.17","mappings":[{"from":3,"to":2}]},{"pgid":"4.19","mappings":[{"from":3,"to":0}]},{"pgid":"4.3a","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:0/3403658744":"2023-10-01T22:30:13.497576+0000","172.21.15.16:0/1216430525":"2023-10-01T22:30:13.497576+0000","172.21.15.16:0/3418827610":"2023-10-01T22:30:13.497576+0000","172.21.15.16:0/1101648925":"2023-10-01T22:29:14.475978+0000","172.21.15.16:0/2249754484":"2023-10-01T22:29:14.475978+0000","172.21.15.16:0/1879378585":"2023-10-01T22:30:13.497576+0000","172.21.15.16:6816/118285":"2023-10-01T22:29:14.475978+0000","172.21.15.16:6816/126811":"2023-10-01T22:30:13.497576+0000","172.21.15.16:6817/1424366610":"2023-10-01T22:29:40.259230+0000","172.21.15.16:0/2178963721":"2023-10-01T22:30:13.497576+0000","172.21.15.16:0/3146063150":"2023-10-01T22:30:13.497576+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-09-30T22:38:57.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.561+0000 7efc28356700 1 -- 172.21.15.16:0/901639715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc2009e930 msgr2=0x7efc201c8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:38:57.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.561+0000 7efc28356700 1 --2- 172.21.15.16:0/901639715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc2009e930 0x7efc201c8520 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7efc1c00c550 tx=0x7efc1c00c860 comp rx=0 tx=0).stop 2023-09-30T22:38:57.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.561+0000 7efc28356700 1 -- 172.21.15.16:0/901639715 shutdown_connections 2023-09-30T22:38:57.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.561+0000 7efc28356700 1 --2- 172.21.15.16:0/901639715 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7efc20136fe0 0x7efc2006b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:57.562 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.561+0000 7efc28356700 1 --2- 172.21.15.16:0/901639715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efc2009e930 0x7efc201c8520 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:57.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.561+0000 7efc28356700 1 --2- 172.21.15.16:0/901639715 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7efc2009f410 0x7efc201c8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:57.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.561+0000 7efc28356700 1 -- 172.21.15.16:0/901639715 >> 172.21.15.16:0/901639715 conn(0x7efc200a4210 msgr2=0x7efc20134990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:38:57.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.562+0000 7efc28356700 1 -- 172.21.15.16:0/901639715 shutdown_connections 2023-09-30T22:38:57.563 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.562+0000 7efc28356700 1 -- 172.21.15.16:0/901639715 wait complete. 2023-09-30T22:38:57.576 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-09-30T22:26:44.285133+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}, 'mgr_devicehealth': {}}, 'read_balance': {'score_acting': 4, 'score_stable': 4, 'optimal_score': 0.5, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-09-30T22:26:45.132921+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-09-30T22:26:49.353683+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-09-30T22:26:50.655685+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.1299999952316284, 'score_stable': 1.1299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.1299999952316284, 'raw_score_stable': 1.1299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-09-30T22:38:57.576 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-09-30T22:38:57.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.715+0000 7fc11ed94700 1 Processor -- start 2023-09-30T22:38:57.716 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.715+0000 7fc11ed94700 1 -- start start 2023-09-30T22:38:57.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.716+0000 7fc11ed94700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc11812e180 0x7fc11812e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:57.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.716+0000 7fc11ed94700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc11806b270 0x7fc118139590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:57.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.716+0000 7fc11ed94700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc11812df80 con 0x7fc11806a770 2023-09-30T22:38:57.718 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.716+0000 7fc11ed94700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc11809e640 con 0x7fc11806b270 2023-09-30T22:38:57.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.716+0000 7fc11ed94700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc11809f7c0 con 0x7fc11812e180 2023-09-30T22:38:57.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.716+0000 7fc11d331700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc11806b270 0x7fc118139590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:57.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.716+0000 7fc11cb30700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc11812e180 0x7fc11812e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:57.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.716+0000 7fc11d331700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fc11806b270 0x7fc118139590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34766/0 (socket says 172.21.15.16:34766) 2023-09-30T22:38:57.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.716+0000 7fc11cb30700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc11812e180 0x7fc11812e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:39688/0 (socket says 172.21.15.16:39688) 2023-09-30T22:38:57.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.716+0000 7fc11d331700 1 -- 172.21.15.16:0/824595508 learned_addr learned my addr 172.21.15.16:0/824595508 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:38:57.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.717+0000 7fc11d331700 1 -- 172.21.15.16:0/824595508 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc11812e180 msgr2=0x7fc11812e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:38:57.719 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.717+0000 7fc11d331700 1 --2- 172.21.15.16:0/824595508 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc11812e180 0x7fc11812e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:57.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.717+0000 7fc11d331700 1 -- 172.21.15.16:0/824595508 >> v1:172.21.15.16:6789/0 conn(0x7fc11806a770 legacy=0x7fc11806ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:38:57.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.717+0000 7fc11d331700 1 -- 172.21.15.16:0/824595508 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc11809e900 con 0x7fc11806b270 2023-09-30T22:38:57.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.717+0000 7fc1177fe700 1 -- 172.21.15.16:0/824595508 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1548559608 0 0) 0x7fc11812df80 con 0x7fc11806a770 2023-09-30T22:38:57.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.717+0000 7fc11cb30700 1 --2- 172.21.15.16:0/824595508 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc11812e180 0x7fc11812e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:38:57.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.717+0000 7fc11d331700 1 --2- 172.21.15.16:0/824595508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc11806b270 0x7fc118139590 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fc110009e50 tx=0x7fc11000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f8ac10e1ee8a8e3 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:38:57.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.717+0000 7fc1177fe700 1 -- 172.21.15.16:0/824595508 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11000e070 con 0x7fc11806b270 2023-09-30T22:38:57.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.717+0000 7fc1177fe700 1 -- 172.21.15.16:0/824595508 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc11000b910 con 0x7fc11806b270 2023-09-30T22:38:57.720 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.717+0000 7fc1177fe700 1 -- 172.21.15.16:0/824595508 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc11000bcc0 con 0x7fc11806b270 2023-09-30T22:38:57.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.718+0000 7fc11ed94700 1 -- 172.21.15.16:0/824595508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc11806b270 msgr2=0x7fc118139590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:38:57.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.718+0000 7fc11ed94700 1 --2- 172.21.15.16:0/824595508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc11806b270 0x7fc118139590 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fc110009e50 tx=0x7fc11000b330 comp rx=0 tx=0).stop 2023-09-30T22:38:57.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.718+0000 7fc11ed94700 1 -- 172.21.15.16:0/824595508 shutdown_connections 2023-09-30T22:38:57.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.718+0000 7fc11ed94700 1 --2- 172.21.15.16:0/824595508 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc11806b270 0x7fc118139590 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:57.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.718+0000 7fc11ed94700 1 --2- 172.21.15.16:0/824595508 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc11812e180 0x7fc11812e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:57.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.718+0000 7fc11ed94700 1 -- 172.21.15.16:0/824595508 >> 172.21.15.16:0/824595508 conn(0x7fc118066d70 msgr2=0x7fc1180a2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:38:57.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.718+0000 7fc11ed94700 1 -- 172.21.15.16:0/824595508 shutdown_connections 2023-09-30T22:38:57.721 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.718+0000 7fc11ed94700 1 -- 172.21.15.16:0/824595508 wait complete. 2023-09-30T22:38:57.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.719+0000 7fc11ed94700 1 Processor -- start 2023-09-30T22:38:57.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.719+0000 7fc11ed94700 1 -- start start 2023-09-30T22:38:57.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.719+0000 7fc11ed94700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc11806a770 0x7fc11809a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:57.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.719+0000 7fc11ed94700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc11806b270 0x7fc11809a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:57.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.719+0000 7fc11ed94700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc11812e180 0x7fc11809ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:57.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.719+0000 7fc11ed94700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11811dcb0 con 0x7fc11806a770 2023-09-30T22:38:57.724 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.719+0000 7fc11ed94700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc11811de90 con 0x7fc11806b270 2023-09-30T22:38:57.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.719+0000 7fc11ed94700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fc11812b840 con 0x7fc11812e180 2023-09-30T22:38:57.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.719+0000 7fc11cb30700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc11806a770 0x7fc11809a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:57.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.719+0000 7fc11cb30700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc11806a770 0x7fc11809a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34780/0 (socket says 172.21.15.16:34780) 2023-09-30T22:38:57.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.719+0000 7fc117fff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc11806b270 0x7fc11809a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:57.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.719+0000 7fc11cb30700 1 -- 172.21.15.16:0/2536913864 learned_addr learned my addr 172.21.15.16:0/2536913864 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:38:57.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.720+0000 7fc11d331700 1 --2- 172.21.15.16:0/2536913864 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc11812e180 0x7fc11809ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:57.725 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.720+0000 7fc11cb30700 1 -- 172.21.15.16:0/2536913864 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc11812e180 msgr2=0x7fc11809ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:38:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.720+0000 7fc11cb30700 1 --2- 172.21.15.16:0/2536913864 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc11812e180 0x7fc11809ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.720+0000 7fc11cb30700 1 -- 172.21.15.16:0/2536913864 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc11806b270 msgr2=0x7fc11809a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:38:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.720+0000 7fc11cb30700 1 --2- 172.21.15.16:0/2536913864 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc11806b270 0x7fc11809a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.720+0000 7fc11cb30700 1 -- 172.21.15.16:0/2536913864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1080094a0 con 0x7fc11806a770 2023-09-30T22:38:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.720+0000 7fc11d331700 1 --2- 172.21.15.16:0/2536913864 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc11812e180 0x7fc11809ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:38:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.720+0000 7fc117fff700 1 --2- 172.21.15.16:0/2536913864 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc11806b270 0x7fc11809a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:38:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.720+0000 7fc11cb30700 1 --2- 172.21.15.16:0/2536913864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc11806a770 0x7fc11809a3d0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fc10800ec10 tx=0x7fc10800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:38:57.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.720+0000 7fc1157fa700 1 -- 172.21.15.16:0/2536913864 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10800c850 con 0x7fc11806a770 2023-09-30T22:38:57.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.720+0000 7fc1157fa700 1 -- 172.21.15.16:0/2536913864 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc10800c9b0 con 0x7fc11806a770 2023-09-30T22:38:57.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.720+0000 7fc11ed94700 1 -- 172.21.15.16:0/2536913864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc110009b30 con 0x7fc11806a770 2023-09-30T22:38:57.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.720+0000 7fc1157fa700 1 -- 172.21.15.16:0/2536913864 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc10800cd20 con 0x7fc11806a770 2023-09-30T22:38:57.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.720+0000 7fc11ed94700 1 -- 172.21.15.16:0/2536913864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc11812af00 con 0x7fc11806a770 2023-09-30T22:38:57.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.721+0000 7fc1157fa700 1 -- 172.21.15.16:0/2536913864 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fc108013470 con 0x7fc11806a770 2023-09-30T22:38:57.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.721+0000 7fc11ed94700 1 -- 172.21.15.16:0/2536913864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc11811dac0 con 0x7fc11806a770 2023-09-30T22:38:57.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.721+0000 7fc1157fa700 1 -- 172.21.15.16:0/2536913864 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fc10803f810 con 0x7fc11806a770 2023-09-30T22:38:57.728 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.726+0000 7fc1157fa700 1 -- 172.21.15.16:0/2536913864 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fc108013720 con 0x7fc11806a770 2023-09-30T22:38:57.928 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.928+0000 7fc11ed94700 1 -- 172.21.15.16:0/2536913864 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc11804f2a0 con 0x7fc11806a770 2023-09-30T22:38:57.929 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.928+0000 7fc1157fa700 1 -- 172.21.15.16:0/2536913864 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc10801bb60 con 0x7fc11806a770 2023-09-30T22:38:57.929 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2023-09-30T22:38:57.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.931+0000 7fc11ed94700 1 -- 172.21.15.16:0/2536913864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc11806a770 msgr2=0x7fc11809a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:38:57.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.931+0000 7fc11ed94700 1 --2- 172.21.15.16:0/2536913864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc11806a770 0x7fc11809a3d0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fc10800ec10 tx=0x7fc10800ef20 comp rx=0 tx=0).stop 2023-09-30T22:38:57.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.931+0000 7fc11ed94700 1 -- 172.21.15.16:0/2536913864 shutdown_connections 2023-09-30T22:38:57.931 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.931+0000 7fc11ed94700 1 --2- 172.21.15.16:0/2536913864 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fc11812e180 0x7fc11809ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:57.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.931+0000 7fc11ed94700 1 --2- 172.21.15.16:0/2536913864 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc11806a770 0x7fc11809a3d0 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:57.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.931+0000 7fc11ed94700 1 --2- 172.21.15.16:0/2536913864 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fc11806b270 0x7fc11809a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:57.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.931+0000 7fc11ed94700 1 -- 172.21.15.16:0/2536913864 >> 172.21.15.16:0/2536913864 conn(0x7fc118066d70 msgr2=0x7fc1181b8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:38:57.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.931+0000 7fc11ed94700 1 -- 172.21.15.16:0/2536913864 shutdown_connections 2023-09-30T22:38:57.932 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:57.931+0000 7fc11ed94700 1 -- 172.21.15.16:0/2536913864 wait complete. 2023-09-30T22:38:57.945 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-09-30T22:38:58.086 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.084+0000 7fde9b59e700 1 Processor -- start 2023-09-30T22:38:58.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.084+0000 7fde9b59e700 1 -- start start 2023-09-30T22:38:58.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.084+0000 7fde9b59e700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fde9c06a770 0x7fde9c133830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:58.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.084+0000 7fde9b59e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fde9c133280 0x7fde9c133d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:58.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.084+0000 7fde9b59e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fde9c12fcc0 0x7fde9c130110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:58.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.084+0000 7fde9b59e700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde9c06ab40 con 0x7fde9c06a770 2023-09-30T22:38:58.087 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.084+0000 7fde9b59e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c09a2c0 con 0x7fde9c12fcc0 2023-09-30T22:38:58.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.084+0000 7fde9b59e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c09f440 con 0x7fde9c133280 2023-09-30T22:38:58.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.084+0000 7fde99d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fde9c133280 0x7fde9c133d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:58.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.084+0000 7fde9ad9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fde9c12fcc0 0x7fde9c130110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:58.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.084+0000 7fde9a59c700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fde9c06a770 0x7fde9c133830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:58.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.085+0000 7fde99d9b700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fde9c133280 0x7fde9c133d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:39050/0 (socket says 172.21.15.16:39050) 2023-09-30T22:38:58.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.085+0000 7fde9ad9d700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fde9c12fcc0 0x7fde9c130110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:39700/0 (socket says 172.21.15.16:39700) 2023-09-30T22:38:58.088 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.085+0000 7fde99d9b700 1 -- 172.21.15.16:0/22992196 learned_addr learned my addr 172.21.15.16:0/22992196 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:38:58.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.085+0000 7fde99d9b700 1 -- 172.21.15.16:0/22992196 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fde9c12fcc0 msgr2=0x7fde9c130110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:38:58.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.085+0000 7fde99d9b700 1 --2- 172.21.15.16:0/22992196 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fde9c12fcc0 0x7fde9c130110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:58.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.085+0000 7fde99d9b700 1 -- 172.21.15.16:0/22992196 >> v2:172.21.15.16:3300/0 conn(0x7fde9c06a770 msgr2=0x7fde9c133830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:38:58.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.085+0000 7fde99d9b700 1 --2- 172.21.15.16:0/22992196 >> v2:172.21.15.16:3300/0 conn(0x7fde9c06a770 0x7fde9c133830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:58.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.085+0000 7fde99d9b700 1 -- 172.21.15.16:0/22992196 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde9c06b250 con 0x7fde9c133280 2023-09-30T22:38:58.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.085+0000 7fde9ad9d700 1 --2- 172.21.15.16:0/22992196 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fde9c12fcc0 0x7fde9c130110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:38:58.089 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.085+0000 7fde99d9b700 1 --2- 172.21.15.16:0/22992196 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fde9c133280 0x7fde9c133d70 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fde900098f0 tx=0x7fde90009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9ddfc1f5231cb73 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:38:58.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.085+0000 7fde9959a700 1 -- 172.21.15.16:0/22992196 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde90011830 con 0x7fde9c133280 2023-09-30T22:38:58.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.085+0000 7fde9959a700 1 -- 172.21.15.16:0/22992196 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde90011990 con 0x7fde9c133280 2023-09-30T22:38:58.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.085+0000 7fde9959a700 1 -- 172.21.15.16:0/22992196 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde90011d20 con 0x7fde9c133280 2023-09-30T22:38:58.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.086+0000 7fde9b59e700 1 -- 172.21.15.16:0/22992196 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fde9c133280 msgr2=0x7fde9c133d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:38:58.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.086+0000 7fde9b59e700 1 --2- 172.21.15.16:0/22992196 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fde9c133280 0x7fde9c133d70 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fde900098f0 tx=0x7fde90009c00 comp rx=0 tx=0).stop 2023-09-30T22:38:58.090 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.086+0000 7fde9b59e700 1 -- 172.21.15.16:0/22992196 shutdown_connections 2023-09-30T22:38:58.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.086+0000 7fde9b59e700 1 --2- 172.21.15.16:0/22992196 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fde9c12fcc0 0x7fde9c130110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:58.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.086+0000 7fde9b59e700 1 --2- 172.21.15.16:0/22992196 >> v2:172.21.15.16:3300/0 conn(0x7fde9c06a770 0x7fde9c133830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:58.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.086+0000 7fde9b59e700 1 --2- 172.21.15.16:0/22992196 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fde9c133280 0x7fde9c133d70 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:58.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.086+0000 7fde9b59e700 1 -- 172.21.15.16:0/22992196 >> 172.21.15.16:0/22992196 conn(0x7fde9c0a2cc0 msgr2=0x7fde9c0a30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:38:58.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.086+0000 7fde9b59e700 1 -- 172.21.15.16:0/22992196 shutdown_connections 2023-09-30T22:38:58.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.086+0000 7fde9b59e700 1 -- 172.21.15.16:0/22992196 wait complete. 2023-09-30T22:38:58.093 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.087+0000 7fde9b59e700 1 Processor -- start 2023-09-30T22:38:58.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.087+0000 7fde9b59e700 1 -- start start 2023-09-30T22:38:58.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.088+0000 7fde9b59e700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde9c06a770 0x7fde9c11d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:58.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.088+0000 7fde9b59e700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fde9c12fcc0 0x7fde9c11dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:58.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.088+0000 7fde9b59e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fde9c133280 0x7fde9c11e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:58.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.088+0000 7fde9b59e700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c1207b0 con 0x7fde9c06a770 2023-09-30T22:38:58.094 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.088+0000 7fde9b59e700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c11f8c0 con 0x7fde9c12fcc0 2023-09-30T22:38:58.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.088+0000 7fde9b59e700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c11f560 con 0x7fde9c133280 2023-09-30T22:38:58.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.088+0000 7fde9ad9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fde9c133280 0x7fde9c11e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:58.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.088+0000 7fde9a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde9c06a770 0x7fde9c11d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:58.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.088+0000 7fde99d9b700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fde9c12fcc0 0x7fde9c11dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:58.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.088+0000 7fde9a59c700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde9c06a770 0x7fde9c11d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34800/0 (socket says 172.21.15.16:34800) 2023-09-30T22:38:58.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.088+0000 7fde9ad9d700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fde9c133280 0x7fde9c11e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:39060/0 (socket says 172.21.15.16:39060) 2023-09-30T22:38:58.095 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.088+0000 7fde9a59c700 1 -- 172.21.15.16:0/2384776585 learned_addr learned my addr 172.21.15.16:0/2384776585 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:38:58.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.088+0000 7fde9a59c700 1 -- 172.21.15.16:0/2384776585 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fde9c133280 msgr2=0x7fde9c11e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:38:58.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.088+0000 7fde9a59c700 1 --2- 172.21.15.16:0/2384776585 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fde9c133280 0x7fde9c11e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:58.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.088+0000 7fde9a59c700 1 -- 172.21.15.16:0/2384776585 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fde9c12fcc0 msgr2=0x7fde9c11dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:38:58.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.088+0000 7fde9a59c700 1 --2- 172.21.15.16:0/2384776585 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fde9c12fcc0 0x7fde9c11dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:58.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.088+0000 7fde9a59c700 1 -- 172.21.15.16:0/2384776585 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde94009a70 con 0x7fde9c06a770 2023-09-30T22:38:58.096 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.088+0000 7fde99d9b700 1 --2- 172.21.15.16:0/2384776585 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fde9c12fcc0 0x7fde9c11dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:38:58.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.088+0000 7fde9a59c700 1 --2- 172.21.15.16:0/2384776585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde9c06a770 0x7fde9c11d720 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fde8c00e9e0 tx=0x7fde8c00ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:38:58.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.088+0000 7fde8affd700 1 -- 172.21.15.16:0/2384776585 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8c009710 con 0x7fde9c06a770 2023-09-30T22:38:58.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.088+0000 7fde8affd700 1 -- 172.21.15.16:0/2384776585 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde8c00ee70 con 0x7fde9c06a770 2023-09-30T22:38:58.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.089+0000 7fde8affd700 1 -- 172.21.15.16:0/2384776585 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8c00ca50 con 0x7fde9c06a770 2023-09-30T22:38:58.097 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.089+0000 7fde9b59e700 1 -- 172.21.15.16:0/2384776585 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde90009550 con 0x7fde9c06a770 2023-09-30T22:38:58.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.089+0000 7fde9b59e700 1 -- 172.21.15.16:0/2384776585 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde9c11fdd0 con 0x7fde9c06a770 2023-09-30T22:38:58.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.089+0000 7fde8affd700 1 -- 172.21.15.16:0/2384776585 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fde8c00cbb0 con 0x7fde9c06a770 2023-09-30T22:38:58.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.089+0000 7fde8affd700 1 -- 172.21.15.16:0/2384776585 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fde8c03fa90 con 0x7fde9c06a770 2023-09-30T22:38:58.098 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.090+0000 7fde9b59e700 1 -- 172.21.15.16:0/2384776585 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde9c04f2a0 con 0x7fde9c06a770 2023-09-30T22:38:58.099 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.094+0000 7fde8affd700 1 -- 172.21.15.16:0/2384776585 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fde8c021080 con 0x7fde9c06a770 2023-09-30T22:38:58.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.296+0000 7fde9b59e700 1 -- 172.21.15.16:0/2384776585 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fde9c0715d0 con 0x7fde9c06a770 2023-09-30T22:38:58.297 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.297+0000 7fde8affd700 1 -- 172.21.15.16:0/2384776585 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v40) v1 ==== 92+0+10 (secure 0 0 0) 0x7fde8c017020 con 0x7fde9c06a770 2023-09-30T22:38:58.298 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 8 2023-09-30T22:38:58.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.299+0000 7fde9b59e700 1 -- 172.21.15.16:0/2384776585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde9c06a770 msgr2=0x7fde9c11d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:38:58.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.299+0000 7fde9b59e700 1 --2- 172.21.15.16:0/2384776585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde9c06a770 0x7fde9c11d720 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fde8c00e9e0 tx=0x7fde8c00ecf0 comp rx=0 tx=0).stop 2023-09-30T22:38:58.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.299+0000 7fde9b59e700 1 -- 172.21.15.16:0/2384776585 shutdown_connections 2023-09-30T22:38:58.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.299+0000 7fde9b59e700 1 --2- 172.21.15.16:0/2384776585 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fde9c133280 0x7fde9c11e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:58.300 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.299+0000 7fde9b59e700 1 --2- 172.21.15.16:0/2384776585 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde9c06a770 0x7fde9c11d720 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:58.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.299+0000 7fde9b59e700 1 --2- 172.21.15.16:0/2384776585 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fde9c12fcc0 0x7fde9c11dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:58.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.299+0000 7fde9b59e700 1 -- 172.21.15.16:0/2384776585 >> 172.21.15.16:0/2384776585 conn(0x7fde9c0a2cc0 msgr2=0x7fde9c09d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:38:58.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.299+0000 7fde9b59e700 1 -- 172.21.15.16:0/2384776585 shutdown_connections 2023-09-30T22:38:58.301 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.299+0000 7fde9b59e700 1 -- 172.21.15.16:0/2384776585 wait complete. 2023-09-30T22:38:58.314 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-09-30T22:38:58.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.454+0000 7f8d76f92700 1 Processor -- start 2023-09-30T22:38:58.454 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.454+0000 7f8d76f92700 1 -- start start 2023-09-30T22:38:58.460 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.454+0000 7f8d76f92700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8d70131b30 0x7f8d70131f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:58.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.454+0000 7f8d76f92700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8d7009f390 0x7f8d701399d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:58.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.454+0000 7f8d76f92700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d700a01c0 con 0x7f8d7009e850 2023-09-30T22:38:58.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.454+0000 7f8d76f92700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7009f830 con 0x7f8d7009f390 2023-09-30T22:38:58.461 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.454+0000 7f8d76f92700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7009b560 con 0x7f8d70131b30 2023-09-30T22:38:58.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.454+0000 7f8d74d2e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8d70131b30 0x7f8d70131f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:58.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.454+0000 7f8d6ffff700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f8d7009e850 0x7f8d7009ec80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60708/0 (socket says 172.21.15.16:60708) 2023-09-30T22:38:58.462 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.454+0000 7f8d6ffff700 1 -- 172.21.15.16:0/999913687 learned_addr learned my addr 172.21.15.16:0/999913687 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:38:58.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.454+0000 7f8d7552f700 1 --2- 172.21.15.16:0/999913687 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8d7009f390 0x7f8d701399d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:58.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.455+0000 7f8d6f7fe700 1 -- 172.21.15.16:0/999913687 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4000035998 0 0) 0x7f8d700a01c0 con 0x7f8d7009e850 2023-09-30T22:38:58.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.455+0000 7f8d6f7fe700 1 -- 172.21.15.16:0/999913687 --> v1:172.21.15.16:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d5c0035b0 con 0x7f8d7009e850 2023-09-30T22:38:58.463 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.455+0000 7f8d74d2e700 1 -- 172.21.15.16:0/999913687 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8d7009f390 msgr2=0x7f8d701399d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:38:58.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.455+0000 7f8d74d2e700 1 --2- 172.21.15.16:0/999913687 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8d7009f390 0x7f8d701399d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:58.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.455+0000 7f8d74d2e700 1 -- 172.21.15.16:0/999913687 >> v1:172.21.15.16:6789/0 conn(0x7f8d7009e850 legacy=0x7f8d7009ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:38:58.464 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.455+0000 7f8d74d2e700 1 -- 172.21.15.16:0/999913687 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d7009a260 con 0x7f8d70131b30 2023-09-30T22:38:58.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.455+0000 7f8d7552f700 1 --2- 172.21.15.16:0/999913687 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8d7009f390 0x7f8d701399d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:38:58.465 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.455+0000 7f8d74d2e700 1 --2- 172.21.15.16:0/999913687 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8d70131b30 0x7f8d70131f00 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f8d60009870 tx=0x7f8d60009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ad039298d9d5d server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:38:58.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.455+0000 7f8d6f7fe700 1 -- 172.21.15.16:0/999913687 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d60011830 con 0x7f8d70131b30 2023-09-30T22:38:58.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.455+0000 7f8d6f7fe700 1 -- 172.21.15.16:0/999913687 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d60011990 con 0x7f8d70131b30 2023-09-30T22:38:58.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.455+0000 7f8d6f7fe700 1 -- 172.21.15.16:0/999913687 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d60011d20 con 0x7f8d70131b30 2023-09-30T22:38:58.466 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.456+0000 7f8d76f92700 1 -- 172.21.15.16:0/999913687 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8d70131b30 msgr2=0x7f8d70131f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:38:58.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.456+0000 7f8d76f92700 1 --2- 172.21.15.16:0/999913687 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8d70131b30 0x7f8d70131f00 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f8d60009870 tx=0x7f8d60009b80 comp rx=0 tx=0).stop 2023-09-30T22:38:58.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.456+0000 7f8d76f92700 1 -- 172.21.15.16:0/999913687 shutdown_connections 2023-09-30T22:38:58.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.456+0000 7f8d76f92700 1 --2- 172.21.15.16:0/999913687 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8d7009f390 0x7f8d701399d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:58.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.456+0000 7f8d76f92700 1 --2- 172.21.15.16:0/999913687 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8d70131b30 0x7f8d70131f00 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:58.467 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.456+0000 7f8d76f92700 1 -- 172.21.15.16:0/999913687 >> 172.21.15.16:0/999913687 conn(0x7f8d700a4130 msgr2=0x7f8d700a4530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:38:58.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.456+0000 7f8d76f92700 1 -- 172.21.15.16:0/999913687 shutdown_connections 2023-09-30T22:38:58.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.456+0000 7f8d76f92700 1 -- 172.21.15.16:0/999913687 wait complete. 2023-09-30T22:38:58.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.457+0000 7f8d76f92700 1 Processor -- start 2023-09-30T22:38:58.468 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.457+0000 7f8d76f92700 1 -- start start 2023-09-30T22:38:58.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.457+0000 7f8d76f92700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8d7009e850 0x7f8d7011def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:58.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.458+0000 7f8d76f92700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8d7009f390 0x7f8d7011e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:58.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.458+0000 7f8d76f92700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d70131b30 0x7f8d7011e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:58.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.458+0000 7f8d76f92700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d701209a0 con 0x7f8d70131b30 2023-09-30T22:38:58.469 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.458+0000 7f8d76f92700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7011fc80 con 0x7f8d7009f390 2023-09-30T22:38:58.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.458+0000 7f8d76f92700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f8d70120dc0 con 0x7f8d7009e850 2023-09-30T22:38:58.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.458+0000 7f8d74d2e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8d7009e850 0x7f8d7011def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:58.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.458+0000 7f8d7552f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d70131b30 0x7f8d7011e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:58.470 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.458+0000 7f8d6ffff700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8d7009f390 0x7f8d7011e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:58.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.458+0000 7f8d7552f700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d70131b30 0x7f8d7011e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34810/0 (socket says 172.21.15.16:34810) 2023-09-30T22:38:58.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.458+0000 7f8d74d2e700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8d7009e850 0x7f8d7011def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:39086/0 (socket says 172.21.15.16:39086) 2023-09-30T22:38:58.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.458+0000 7f8d7552f700 1 -- 172.21.15.16:0/1739439045 learned_addr learned my addr 172.21.15.16:0/1739439045 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:38:58.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.458+0000 7f8d6ffff700 1 -- 172.21.15.16:0/1739439045 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8d7009e850 msgr2=0x7f8d7011def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:38:58.471 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.458+0000 7f8d6ffff700 1 --2- 172.21.15.16:0/1739439045 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8d7009e850 0x7f8d7011def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:58.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.458+0000 7f8d6ffff700 1 -- 172.21.15.16:0/1739439045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d70131b30 msgr2=0x7f8d7011e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:38:58.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.458+0000 7f8d6ffff700 1 --2- 172.21.15.16:0/1739439045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d70131b30 0x7f8d7011e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:58.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.458+0000 7f8d6ffff700 1 -- 172.21.15.16:0/1739439045 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d68009a70 con 0x7f8d7009f390 2023-09-30T22:38:58.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.458+0000 7f8d7552f700 1 --2- 172.21.15.16:0/1739439045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d70131b30 0x7f8d7011e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:38:58.472 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.458+0000 7f8d74d2e700 1 --2- 172.21.15.16:0/1739439045 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8d7009e850 0x7f8d7011def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:38:58.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.458+0000 7f8d6ffff700 1 --2- 172.21.15.16:0/1739439045 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8d7009f390 0x7f8d7011e430 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f8d58001800 tx=0x7f8d58001b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:38:58.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.458+0000 7f8d6d7fa700 1 -- 172.21.15.16:0/1739439045 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d58013070 con 0x7f8d7009f390 2023-09-30T22:38:58.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.458+0000 7f8d6d7fa700 1 -- 172.21.15.16:0/1739439045 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d5800e340 con 0x7f8d7009f390 2023-09-30T22:38:58.473 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.458+0000 7f8d76f92700 1 -- 172.21.15.16:0/1739439045 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d600094a0 con 0x7f8d7009f390 2023-09-30T22:38:58.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.459+0000 7f8d6d7fa700 1 -- 172.21.15.16:0/1739439045 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d58005b00 con 0x7f8d7009f390 2023-09-30T22:38:58.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.459+0000 7f8d76f92700 1 -- 172.21.15.16:0/1739439045 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d7009da10 con 0x7f8d7009f390 2023-09-30T22:38:58.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.459+0000 7f8d6d7fa700 1 -- 172.21.15.16:0/1739439045 <== mon.1 v2:172.21.15.16:3301/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f8d58005c60 con 0x7f8d7009f390 2023-09-30T22:38:58.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.460+0000 7f8d6d7fa700 1 -- 172.21.15.16:0/1739439045 <== mon.1 v2:172.21.15.16:3301/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f8d58042b50 con 0x7f8d7009f390 2023-09-30T22:38:58.474 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.460+0000 7f8d76f92700 1 -- 172.21.15.16:0/1739439045 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d70121690 con 0x7f8d7009f390 2023-09-30T22:38:58.475 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.465+0000 7f8d6d7fa700 1 -- 172.21.15.16:0/1739439045 <== mon.1 v2:172.21.15.16:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f8d70121690 con 0x7f8d7009f390 2023-09-30T22:38:58.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.667+0000 7f8d76f92700 1 -- 172.21.15.16:0/1739439045 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8d70121690 con 0x7f8d7009f390 2023-09-30T22:38:58.668 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.667+0000 7f8d6d7fa700 1 -- 172.21.15.16:0/1739439045 <== mon.1 v2:172.21.15.16:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v40) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8d70121690 con 0x7f8d7009f390 2023-09-30T22:38:58.668 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2023-09-30T22:38:58.670 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.670+0000 7f8d76f92700 1 -- 172.21.15.16:0/1739439045 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8d7009f390 msgr2=0x7f8d7011e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:38:58.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.670+0000 7f8d76f92700 1 --2- 172.21.15.16:0/1739439045 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8d7009f390 0x7f8d7011e430 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f8d58001800 tx=0x7f8d58001b10 comp rx=0 tx=0).stop 2023-09-30T22:38:58.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.670+0000 7f8d76f92700 1 -- 172.21.15.16:0/1739439045 shutdown_connections 2023-09-30T22:38:58.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.670+0000 7f8d76f92700 1 --2- 172.21.15.16:0/1739439045 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d70131b30 0x7f8d7011e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:58.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.670+0000 7f8d76f92700 1 --2- 172.21.15.16:0/1739439045 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f8d7009e850 0x7f8d7011def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:58.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.670+0000 7f8d76f92700 1 --2- 172.21.15.16:0/1739439045 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f8d7009f390 0x7f8d7011e430 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:58.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.670+0000 7f8d76f92700 1 -- 172.21.15.16:0/1739439045 >> 172.21.15.16:0/1739439045 conn(0x7f8d700a4130 msgr2=0x7f8d7009cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:38:58.671 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.670+0000 7f8d76f92700 1 -- 172.21.15.16:0/1739439045 shutdown_connections 2023-09-30T22:38:58.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.670+0000 7f8d76f92700 1 -- 172.21.15.16:0/1739439045 wait complete. 2023-09-30T22:38:58.684 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-09-30T22:38:58.823 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.823+0000 7f87d7a60700 1 Processor -- start 2023-09-30T22:38:58.824 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.823+0000 7f87d7a60700 1 -- start start 2023-09-30T22:38:58.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.824+0000 7f87d7a60700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f87d0133cc0 0x7f87d0134090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:58.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.824+0000 7f87d7a60700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f87d01345d0 0x7f87d00a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:58.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.824+0000 7f87d7a60700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f87d00a0f70 0x7f87d00a1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:58.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.824+0000 7f87d7a60700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87d00a0640 con 0x7f87d0133cc0 2023-09-30T22:38:58.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.824+0000 7f87d7a60700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87d009e1f0 con 0x7f87d01345d0 2023-09-30T22:38:58.826 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.824+0000 7f87d7a60700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f87d00a23b0 con 0x7f87d00a0f70 2023-09-30T22:38:58.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.824+0000 7f87d4ffb700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f87d01345d0 0x7f87d00a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:58.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.824+0000 7f87d57fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f87d0133cc0 0x7f87d0134090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:58.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.824+0000 7f87d57fc700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f87d0133cc0 0x7f87d0134090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34822/0 (socket says 172.21.15.16:34822) 2023-09-30T22:38:58.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.824+0000 7f87d5ffd700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f87d00a0f70 0x7f87d00a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:58.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.824+0000 7f87d57fc700 1 -- 172.21.15.16:0/214029421 learned_addr learned my addr 172.21.15.16:0/214029421 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:38:58.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.824+0000 7f87d4ffb700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f87d01345d0 0x7f87d00a0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:39746/0 (socket says 172.21.15.16:39746) 2023-09-30T22:38:58.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.824+0000 7f87d57fc700 1 -- 172.21.15.16:0/214029421 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f87d00a0f70 msgr2=0x7f87d00a1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:38:58.827 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.824+0000 7f87d57fc700 1 --2- 172.21.15.16:0/214029421 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f87d00a0f70 0x7f87d00a1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:58.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.824+0000 7f87d57fc700 1 -- 172.21.15.16:0/214029421 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f87d01345d0 msgr2=0x7f87d00a0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:38:58.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.824+0000 7f87d57fc700 1 --2- 172.21.15.16:0/214029421 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f87d01345d0 0x7f87d00a0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:58.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.824+0000 7f87d57fc700 1 -- 172.21.15.16:0/214029421 --> v2:172.21.15.16:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d009df30 con 0x7f87d0133cc0 2023-09-30T22:38:58.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.824+0000 7f87d5ffd700 1 --2- 172.21.15.16:0/214029421 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f87d00a0f70 0x7f87d00a1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:38:58.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.824+0000 7f87d4ffb700 1 --2- 172.21.15.16:0/214029421 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f87d01345d0 0x7f87d00a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:38:58.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.825+0000 7f87d57fc700 1 --2- 172.21.15.16:0/214029421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87d0133cc0 0x7f87d0134090 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f87cc009920 tx=0x7f87cc009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4111f71300da5bd server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:38:58.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.825+0000 7f87c7fff700 1 -- 172.21.15.16:0/214029421 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc011830 con 0x7f87d0133cc0 2023-09-30T22:38:58.828 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.825+0000 7f87c7fff700 1 -- 172.21.15.16:0/214029421 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87cc011990 con 0x7f87d0133cc0 2023-09-30T22:38:58.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.825+0000 7f87c7fff700 1 -- 172.21.15.16:0/214029421 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc011d20 con 0x7f87d0133cc0 2023-09-30T22:38:58.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.825+0000 7f87d7a60700 1 -- 172.21.15.16:0/214029421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87d0133cc0 msgr2=0x7f87d0134090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:38:58.829 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.825+0000 7f87d7a60700 1 --2- 172.21.15.16:0/214029421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87d0133cc0 0x7f87d0134090 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f87cc009920 tx=0x7f87cc009c30 comp rx=0 tx=0).stop 2023-09-30T22:38:58.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.825+0000 7f87d7a60700 1 -- 172.21.15.16:0/214029421 shutdown_connections 2023-09-30T22:38:58.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.825+0000 7f87d7a60700 1 --2- 172.21.15.16:0/214029421 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f87d00a0f70 0x7f87d00a1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:58.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.825+0000 7f87d7a60700 1 --2- 172.21.15.16:0/214029421 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87d0133cc0 0x7f87d0134090 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:58.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.825+0000 7f87d7a60700 1 --2- 172.21.15.16:0/214029421 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f87d01345d0 0x7f87d00a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:58.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.825+0000 7f87d7a60700 1 -- 172.21.15.16:0/214029421 >> 172.21.15.16:0/214029421 conn(0x7f87d00a3050 msgr2=0x7f87d00a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:38:58.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.826+0000 7f87d7a60700 1 -- 172.21.15.16:0/214029421 shutdown_connections 2023-09-30T22:38:58.830 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.826+0000 7f87d7a60700 1 -- 172.21.15.16:0/214029421 wait complete. 2023-09-30T22:38:58.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.827+0000 7f87d7a60700 1 Processor -- start 2023-09-30T22:38:58.832 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.827+0000 7f87d7a60700 1 -- start start 2023-09-30T22:38:58.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.828+0000 7f87d7a60700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f87d00a0f70 0x7f87d01225b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:58.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.828+0000 7f87d7a60700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87d0133cc0 0x7f87d0122af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:58.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.828+0000 7f87d7a60700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f87d01345d0 0x7f87d0123030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:58.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.828+0000 7f87d7a60700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d0124260 con 0x7f87d0133cc0 2023-09-30T22:38:58.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.828+0000 7f87d7a60700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87d0125b30 con 0x7f87d01345d0 2023-09-30T22:38:58.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.828+0000 7f87d7a60700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f87d009ca70 con 0x7f87d00a0f70 2023-09-30T22:38:58.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.828+0000 7f87d57fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f87d00a0f70 0x7f87d01225b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:58.833 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.828+0000 7f87d57fc700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f87d00a0f70 0x7f87d01225b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3302/0 says I am v2:172.21.15.16:39116/0 (socket says 172.21.15.16:39116) 2023-09-30T22:38:58.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.828+0000 7f87d5ffd700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f87d01345d0 0x7f87d0123030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:58.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.828+0000 7f87d4ffb700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87d0133cc0 0x7f87d0122af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:58.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.828+0000 7f87d57fc700 1 -- 172.21.15.16:0/1795070422 learned_addr learned my addr 172.21.15.16:0/1795070422 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:38:58.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.828+0000 7f87d57fc700 1 -- 172.21.15.16:0/1795070422 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f87d01345d0 msgr2=0x7f87d0123030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:38:58.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.828+0000 7f87d57fc700 1 --2- 172.21.15.16:0/1795070422 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f87d01345d0 0x7f87d0123030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:58.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.828+0000 7f87d57fc700 1 -- 172.21.15.16:0/1795070422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87d0133cc0 msgr2=0x7f87d0122af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:38:58.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.828+0000 7f87d57fc700 1 --2- 172.21.15.16:0/1795070422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87d0133cc0 0x7f87d0122af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:58.834 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.828+0000 7f87d57fc700 1 -- 172.21.15.16:0/1795070422 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87c8009a70 con 0x7f87d00a0f70 2023-09-30T22:38:58.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.828+0000 7f87d5ffd700 1 --2- 172.21.15.16:0/1795070422 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f87d01345d0 0x7f87d0123030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:38:58.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.828+0000 7f87d4ffb700 1 --2- 172.21.15.16:0/1795070422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87d0133cc0 0x7f87d0122af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:38:58.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.828+0000 7f87d57fc700 1 --2- 172.21.15.16:0/1795070422 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f87d00a0f70 0x7f87d01225b0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f87cc009870 tx=0x7f87cc004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:38:58.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.829+0000 7f87c5ffb700 1 -- 172.21.15.16:0/1795070422 <== mon.2 v2:172.21.15.16:3302/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc004240 con 0x7f87d00a0f70 2023-09-30T22:38:58.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.829+0000 7f87c5ffb700 1 -- 172.21.15.16:0/1795070422 <== mon.2 v2:172.21.15.16:3302/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87cc01f070 con 0x7f87d00a0f70 2023-09-30T22:38:58.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.829+0000 7f87c5ffb700 1 -- 172.21.15.16:0/1795070422 <== mon.2 v2:172.21.15.16:3302/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87cc0045e0 con 0x7f87d00a0f70 2023-09-30T22:38:58.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.829+0000 7f87d7a60700 1 -- 172.21.15.16:0/1795070422 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87c00094a0 con 0x7f87d00a0f70 2023-09-30T22:38:58.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.829+0000 7f87d7a60700 1 -- 172.21.15.16:0/1795070422 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87cc009550 con 0x7f87d00a0f70 2023-09-30T22:38:58.835 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.830+0000 7f87d7a60700 1 -- 172.21.15.16:0/1795070422 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87d0133110 con 0x7f87d00a0f70 2023-09-30T22:38:58.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.830+0000 7f87c5ffb700 1 -- 172.21.15.16:0/1795070422 <== mon.2 v2:172.21.15.16:3302/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f87cc021030 con 0x7f87d00a0f70 2023-09-30T22:38:58.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.830+0000 7f87c5ffb700 1 -- 172.21.15.16:0/1795070422 <== mon.2 v2:172.21.15.16:3302/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f87cc040e60 con 0x7f87d00a0f70 2023-09-30T22:38:58.836 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:58.835+0000 7f87c5ffb700 1 -- 172.21.15.16:0/1795070422 <== mon.2 v2:172.21.15.16:3302/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f87cc0237e0 con 0x7f87d00a0f70 2023-09-30T22:38:59.036 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.036+0000 7f87d7a60700 1 -- 172.21.15.16:0/1795070422 --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f87d009db60 con 0x7f87d00a0f70 2023-09-30T22:38:59.037 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.036+0000 7f87c5ffb700 1 -- 172.21.15.16:0/1795070422 <== mon.2 v2:172.21.15.16:3302/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v40) v1 ==== 100+0+11 (secure 0 0 0) 0x7f87cc015030 con 0x7f87d00a0f70 2023-09-30T22:38:59.037 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2023-09-30T22:38:59.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.038+0000 7f87d7a60700 1 -- 172.21.15.16:0/1795070422 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f87d00a0f70 msgr2=0x7f87d01225b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:38:59.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.038+0000 7f87d7a60700 1 --2- 172.21.15.16:0/1795070422 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f87d00a0f70 0x7f87d01225b0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f87cc009870 tx=0x7f87cc004750 comp rx=0 tx=0).stop 2023-09-30T22:38:59.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.038+0000 7f87d7a60700 1 -- 172.21.15.16:0/1795070422 shutdown_connections 2023-09-30T22:38:59.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.038+0000 7f87d7a60700 1 --2- 172.21.15.16:0/1795070422 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f87d01345d0 0x7f87d0123030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:59.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.038+0000 7f87d7a60700 1 --2- 172.21.15.16:0/1795070422 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f87d00a0f70 0x7f87d01225b0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:59.039 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.038+0000 7f87d7a60700 1 --2- 172.21.15.16:0/1795070422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f87d0133cc0 0x7f87d0122af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:59.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.038+0000 7f87d7a60700 1 -- 172.21.15.16:0/1795070422 >> 172.21.15.16:0/1795070422 conn(0x7f87d00a3050 msgr2=0x7f87d009a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:38:59.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.039+0000 7f87d7a60700 1 -- 172.21.15.16:0/1795070422 shutdown_connections 2023-09-30T22:38:59.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.039+0000 7f87d7a60700 1 -- 172.21.15.16:0/1795070422 wait complete. 2023-09-30T22:38:59.052 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-09-30T22:38:59.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.189+0000 7f1ebff24700 1 Processor -- start 2023-09-30T22:38:59.190 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.190+0000 7f1ebff24700 1 -- start start 2023-09-30T22:38:59.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.190+0000 7f1ebff24700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1eb80a0a30 0x7f1eb80a0e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:59.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.190+0000 7f1ebff24700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1eb8138d80 0x7f1eb813b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:59.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.190+0000 7f1ebff24700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1eb80a2a70 con 0x7f1eb809d630 2023-09-30T22:38:59.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.190+0000 7f1ebff24700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1eb80a01d0 con 0x7f1eb8138d80 2023-09-30T22:38:59.193 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.190+0000 7f1ebff24700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb809e2b0 con 0x7f1eb80a0a30 2023-09-30T22:38:59.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.190+0000 7f1ebe4c1700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7f1eb8138d80 0x7f1eb813b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:59.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.190+0000 7f1ebdcc0700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1eb80a0a30 0x7f1eb80a0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:59.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.190+0000 7f1ebd4bf700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7f1eb809d630 0x7f1eb80a1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60716/0 (socket says 172.21.15.16:60716) 2023-09-30T22:38:59.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.190+0000 7f1ebd4bf700 1 -- 172.21.15.16:0/832134484 learned_addr learned my addr 172.21.15.16:0/832134484 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:38:59.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.191+0000 7f1ebdcc0700 1 -- 172.21.15.16:0/832134484 >> v2:172.21.15.16:3300/0 conn(0x7f1eb8138d80 msgr2=0x7f1eb813b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:38:59.194 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.191+0000 7f1ebdcc0700 1 --2- 172.21.15.16:0/832134484 >> v2:172.21.15.16:3300/0 conn(0x7f1eb8138d80 0x7f1eb813b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:59.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.191+0000 7f1ebdcc0700 1 -- 172.21.15.16:0/832134484 >> v1:172.21.15.16:6789/0 conn(0x7f1eb809d630 legacy=0x7f1eb80a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:38:59.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.191+0000 7f1ebdcc0700 1 -- 172.21.15.16:0/832134484 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb809dc50 con 0x7f1eb80a0a30 2023-09-30T22:38:59.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.191+0000 7f1ebe4c1700 1 --2- 172.21.15.16:0/832134484 >> v2:172.21.15.16:3300/0 conn(0x7f1eb8138d80 0x7f1eb813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:38:59.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.191+0000 7f1ebdcc0700 1 --2- 172.21.15.16:0/832134484 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1eb80a0a30 0x7f1eb80a0e00 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4009840 tx=0x7f1eb4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30cea3dc5995e89a server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:38:59.195 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.191+0000 7f1ebccbe700 1 -- 172.21.15.16:0/832134484 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb4011830 con 0x7f1eb80a0a30 2023-09-30T22:38:59.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.191+0000 7f1ebccbe700 1 -- 172.21.15.16:0/832134484 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb4011990 con 0x7f1eb80a0a30 2023-09-30T22:38:59.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.191+0000 7f1ebccbe700 1 -- 172.21.15.16:0/832134484 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb4011d20 con 0x7f1eb80a0a30 2023-09-30T22:38:59.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.191+0000 7f1ebff24700 1 -- 172.21.15.16:0/832134484 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1eb80a0a30 msgr2=0x7f1eb80a0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:38:59.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.191+0000 7f1ebff24700 1 --2- 172.21.15.16:0/832134484 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1eb80a0a30 0x7f1eb80a0e00 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4009840 tx=0x7f1eb4009b50 comp rx=0 tx=0).stop 2023-09-30T22:38:59.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.192+0000 7f1ebff24700 1 -- 172.21.15.16:0/832134484 shutdown_connections 2023-09-30T22:38:59.196 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.192+0000 7f1ebff24700 1 --2- 172.21.15.16:0/832134484 >> v2:172.21.15.16:3300/0 conn(0x7f1eb8138d80 0x7f1eb813b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:59.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.192+0000 7f1ebff24700 1 --2- 172.21.15.16:0/832134484 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1eb80a0a30 0x7f1eb80a0e00 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:59.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.192+0000 7f1ebff24700 1 -- 172.21.15.16:0/832134484 >> 172.21.15.16:0/832134484 conn(0x7f1eb80a3050 msgr2=0x7f1eb80a3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:38:59.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.192+0000 7f1ebff24700 1 -- 172.21.15.16:0/832134484 shutdown_connections 2023-09-30T22:38:59.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.192+0000 7f1ebff24700 1 -- 172.21.15.16:0/832134484 wait complete. 2023-09-30T22:38:59.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.193+0000 7f1ebff24700 1 Processor -- start 2023-09-30T22:38:59.197 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.193+0000 7f1ebff24700 1 -- start start 2023-09-30T22:38:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.193+0000 7f1ebff24700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1eb809d630 0x7f1eb81208f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:59.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.193+0000 7f1ebff24700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1eb80a0a30 0x7f1eb8120e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:59.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.193+0000 7f1ebff24700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1eb8138d80 0x7f1eb8121370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:38:59.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.193+0000 7f1ebff24700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb8122470 con 0x7f1eb809d630 2023-09-30T22:38:59.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.193+0000 7f1ebff24700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb81237c0 con 0x7f1eb8138d80 2023-09-30T22:38:59.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.193+0000 7f1ebff24700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb8121d50 con 0x7f1eb80a0a30 2023-09-30T22:38:59.200 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.193+0000 7f1ebdcc0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1eb809d630 0x7f1eb81208f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:59.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.193+0000 7f1ebe4c1700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1eb8138d80 0x7f1eb8121370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:59.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.193+0000 7f1ebdcc0700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1eb809d630 0x7f1eb81208f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34850/0 (socket says 172.21.15.16:34850) 2023-09-30T22:38:59.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.193+0000 7f1ebdcc0700 1 -- 172.21.15.16:0/2917970275 learned_addr learned my addr 172.21.15.16:0/2917970275 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:38:59.201 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.193+0000 7f1ebd4bf700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1eb80a0a30 0x7f1eb8120e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:38:59.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.194+0000 7f1ebdcc0700 1 -- 172.21.15.16:0/2917970275 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1eb80a0a30 msgr2=0x7f1eb8120e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:38:59.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.194+0000 7f1ebdcc0700 1 --2- 172.21.15.16:0/2917970275 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1eb80a0a30 0x7f1eb8120e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:59.202 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.194+0000 7f1ebdcc0700 1 -- 172.21.15.16:0/2917970275 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1eb8138d80 msgr2=0x7f1eb8121370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:38:59.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.194+0000 7f1ebdcc0700 1 --2- 172.21.15.16:0/2917970275 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1eb8138d80 0x7f1eb8121370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:59.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.194+0000 7f1ebdcc0700 1 -- 172.21.15.16:0/2917970275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb0009a70 con 0x7f1eb809d630 2023-09-30T22:38:59.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.194+0000 7f1ebdcc0700 1 --2- 172.21.15.16:0/2917970275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1eb809d630 0x7f1eb81208f0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4009fd0 tx=0x7f1eb4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:38:59.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.194+0000 7f1eae7fc700 1 -- 172.21.15.16:0/2917970275 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb40036a0 con 0x7f1eb809d630 2023-09-30T22:38:59.203 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.194+0000 7f1eae7fc700 1 -- 172.21.15.16:0/2917970275 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb401f070 con 0x7f1eb809d630 2023-09-30T22:38:59.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.194+0000 7f1ebff24700 1 -- 172.21.15.16:0/2917970275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1eb40094a0 con 0x7f1eb809d630 2023-09-30T22:38:59.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.194+0000 7f1eae7fc700 1 -- 172.21.15.16:0/2917970275 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb4003a40 con 0x7f1eb809d630 2023-09-30T22:38:59.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.194+0000 7f1ebff24700 1 -- 172.21.15.16:0/2917970275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eb80a0410 con 0x7f1eb809d630 2023-09-30T22:38:59.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.195+0000 7f1eae7fc700 1 -- 172.21.15.16:0/2917970275 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7f1eb4003ba0 con 0x7f1eb809d630 2023-09-30T22:38:59.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.195+0000 7f1eae7fc700 1 -- 172.21.15.16:0/2917970275 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7f1eb4021070 con 0x7f1eb809d630 2023-09-30T22:38:59.204 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.195+0000 7f1ebff24700 1 -- 172.21.15.16:0/2917970275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1eb8123cc0 con 0x7f1eb809d630 2023-09-30T22:38:59.205 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.200+0000 7f1eae7fc700 1 -- 172.21.15.16:0/2917970275 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7f1eb402b070 con 0x7f1eb809d630 2023-09-30T22:38:59.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.417+0000 7f1ebff24700 1 -- 172.21.15.16:0/2917970275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1eb80747f0 con 0x7f1eb809d630 2023-09-30T22:38:59.418 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.418+0000 7f1eae7fc700 1 -- 172.21.15.16:0/2917970275 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 11 entries v40) v1 ==== 81+0+639 (secure 0 0 0) 0x7f1eb4015030 con 0x7f1eb809d630 2023-09-30T22:38:59.419 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3403658744 2023-10-01T22:30:13.497576+0000 2023-09-30T22:38:59.419 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/1216430525 2023-10-01T22:30:13.497576+0000 2023-09-30T22:38:59.419 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3418827610 2023-10-01T22:30:13.497576+0000 2023-09-30T22:38:59.419 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/1101648925 2023-10-01T22:29:14.475978+0000 2023-09-30T22:38:59.419 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/2249754484 2023-10-01T22:29:14.475978+0000 2023-09-30T22:38:59.419 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/1879378585 2023-10-01T22:30:13.497576+0000 2023-09-30T22:38:59.420 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6816/118285 2023-10-01T22:29:14.475978+0000 2023-09-30T22:38:59.420 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6816/126811 2023-10-01T22:30:13.497576+0000 2023-09-30T22:38:59.420 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6817/1424366610 2023-10-01T22:29:40.259230+0000 2023-09-30T22:38:59.420 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/2178963721 2023-10-01T22:30:13.497576+0000 2023-09-30T22:38:59.420 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3146063150 2023-10-01T22:30:13.497576+0000 2023-09-30T22:38:59.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.420+0000 7f1ebff24700 1 -- 172.21.15.16:0/2917970275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1eb809d630 msgr2=0x7f1eb81208f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:38:59.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.420+0000 7f1ebff24700 1 --2- 172.21.15.16:0/2917970275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1eb809d630 0x7f1eb81208f0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4009fd0 tx=0x7f1eb4004750 comp rx=0 tx=0).stop 2023-09-30T22:38:59.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.420+0000 7f1ebff24700 1 -- 172.21.15.16:0/2917970275 shutdown_connections 2023-09-30T22:38:59.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.420+0000 7f1ebff24700 1 --2- 172.21.15.16:0/2917970275 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7f1eb8138d80 0x7f1eb8121370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:59.422 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.420+0000 7f1ebff24700 1 --2- 172.21.15.16:0/2917970275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1eb809d630 0x7f1eb81208f0 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:59.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.420+0000 7f1ebff24700 1 --2- 172.21.15.16:0/2917970275 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7f1eb80a0a30 0x7f1eb8120e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:38:59.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.420+0000 7f1ebff24700 1 -- 172.21.15.16:0/2917970275 >> 172.21.15.16:0/2917970275 conn(0x7f1eb80a3050 msgr2=0x7f1eb8139150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:38:59.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.420+0000 7f1ebff24700 1 -- 172.21.15.16:0/2917970275 shutdown_connections 2023-09-30T22:38:59.423 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:38:59.420+0000 7f1ebff24700 1 -- 172.21.15.16:0/2917970275 wait complete. 2023-09-30T22:38:59.423 INFO:teuthology.orchestra.run.smithi016.stderr:listed 11 entries 2023-09-30T22:38:59.434 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi027.front.sepia.ceph.com... 2023-09-30T22:38:59.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:38:59.435 DEBUG:teuthology.orchestra.run.smithi027:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-09-30T22:38:59.477 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:ceph-fuse[112319]: fuse finished with error 0 and tester_r 0 2023-09-30T22:38:59.479 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T22:39:05.581 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi027.front.sepia.ceph.com 2023-09-30T22:39:05.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:39:05.581 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-09-30T22:39:05.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:39:05.614 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2023-09-30T22:39:05.677 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-09-30T22:39:05.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:39:05.678 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-09-30T22:39:05.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-09-30T22:39:05.763 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link delete ceph-brx 2023-09-30T22:39:05.840 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-09-30T22:39:05.852 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-09-30T22:39:05.852 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-09-30T22:39:05.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:05.997+0000 7fba8faba700 1 Processor -- start 2023-09-30T22:39:05.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:05.997+0000 7fba8faba700 1 -- start start 2023-09-30T22:39:05.998 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:05.997+0000 7fba8faba700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fba8809e930 0x7fba8809ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:39:05.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:05.997+0000 7fba8faba700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fba8809f240 0x7fba881317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:39:05.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:05.997+0000 7fba8faba700 1 -- --> v1:172.21.15.16:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba88070940 con 0x7fba8809f650 2023-09-30T22:39:05.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:05.997+0000 7fba8faba700 1 -- --> v2:172.21.15.16:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba8806fd60 con 0x7fba8809f240 2023-09-30T22:39:05.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:05.997+0000 7fba8faba700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba8806fa80 con 0x7fba8809e930 2023-09-30T22:39:05.999 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:05.998+0000 7fba8d055700 1 --2- >> v2:172.21.15.16:3300/0 conn(0x7fba8809f240 0x7fba881317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:39:06.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:05.998+0000 7fba8e057700 1 --1- >> v1:172.21.15.16:6789/0 conn(0x7fba8809f650 0x7fba88131d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.16:6789/0 says I am v1:172.21.15.16:60728/0 (socket says 172.21.15.16:60728) 2023-09-30T22:39:06.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:05.998+0000 7fba8e057700 1 -- 172.21.15.16:0/1855425746 learned_addr learned my addr 172.21.15.16:0/1855425746 (peer_addr_for_me v1:172.21.15.16:0/0) 2023-09-30T22:39:06.000 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:05.998+0000 7fba8d856700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fba8809e930 0x7fba8809ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:39:06.004 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:05.998+0000 7fba8d856700 1 -- 172.21.15.16:0/1855425746 >> v2:172.21.15.16:3300/0 conn(0x7fba8809f240 msgr2=0x7fba881317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-09-30T22:39:06.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:05.998+0000 7fba8d856700 1 --2- 172.21.15.16:0/1855425746 >> v2:172.21.15.16:3300/0 conn(0x7fba8809f240 0x7fba881317f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:39:06.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:05.998+0000 7fba8d856700 1 -- 172.21.15.16:0/1855425746 >> v1:172.21.15.16:6789/0 conn(0x7fba8809f650 legacy=0x7fba88131d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:39:06.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:05.998+0000 7fba8d856700 1 -- 172.21.15.16:0/1855425746 --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba8809fb60 con 0x7fba8809e930 2023-09-30T22:39:06.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:05.998+0000 7fba8c854700 1 -- 172.21.15.16:0/1855425746 <== mon.0 v1:172.21.15.16:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 462777470 0 0) 0x7fba88070940 con 0x7fba8809f650 2023-09-30T22:39:06.005 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:05.998+0000 7fba8d055700 1 --2- 172.21.15.16:0/1855425746 >> v2:172.21.15.16:3300/0 conn(0x7fba8809f240 0x7fba881317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:39:06.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:05.998+0000 7fba8d856700 1 --2- 172.21.15.16:0/1855425746 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fba8809e930 0x7fba8809ed00 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fba84009840 tx=0x7fba84009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1aa74c79d07ee642 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:39:06.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:05.998+0000 7fba8c854700 1 -- 172.21.15.16:0/1855425746 <== mon.1 v2:172.21.15.16:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba84011830 con 0x7fba8809e930 2023-09-30T22:39:06.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:05.998+0000 7fba8c854700 1 -- 172.21.15.16:0/1855425746 <== mon.1 v2:172.21.15.16:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba84011990 con 0x7fba8809e930 2023-09-30T22:39:06.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:05.998+0000 7fba8c854700 1 -- 172.21.15.16:0/1855425746 <== mon.1 v2:172.21.15.16:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba84011d20 con 0x7fba8809e930 2023-09-30T22:39:06.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:05.999+0000 7fba8faba700 1 -- 172.21.15.16:0/1855425746 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fba8809e930 msgr2=0x7fba8809ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:39:06.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:05.999+0000 7fba8faba700 1 --2- 172.21.15.16:0/1855425746 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fba8809e930 0x7fba8809ed00 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fba84009840 tx=0x7fba84009b50 comp rx=0 tx=0).stop 2023-09-30T22:39:06.006 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:05.999+0000 7fba8faba700 1 -- 172.21.15.16:0/1855425746 shutdown_connections 2023-09-30T22:39:06.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:05.999+0000 7fba8faba700 1 --2- 172.21.15.16:0/1855425746 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fba8809e930 0x7fba8809ed00 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:39:06.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:05.999+0000 7fba8faba700 1 --2- 172.21.15.16:0/1855425746 >> v2:172.21.15.16:3300/0 conn(0x7fba8809f240 0x7fba881317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:39:06.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:05.999+0000 7fba8faba700 1 -- 172.21.15.16:0/1855425746 >> 172.21.15.16:0/1855425746 conn(0x7fba880a4210 msgr2=0x7fba880a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-09-30T22:39:06.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:05.999+0000 7fba8faba700 1 -- 172.21.15.16:0/1855425746 shutdown_connections 2023-09-30T22:39:06.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:05.999+0000 7fba8faba700 1 -- 172.21.15.16:0/1855425746 wait complete. 2023-09-30T22:39:06.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:06.000+0000 7fba8faba700 1 Processor -- start 2023-09-30T22:39:06.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:06.000+0000 7fba8faba700 1 -- start start 2023-09-30T22:39:06.007 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:06.000+0000 7fba8faba700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba8809e930 0x7fba881a3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:39:06.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:06.000+0000 7fba8faba700 1 --2- >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fba8809f240 0x7fba881a3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:39:06.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:06.001+0000 7fba8faba700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fba8809f650 0x7fba881c5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-09-30T22:39:06.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:06.001+0000 7fba8faba700 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba881a3a90 con 0x7fba8809e930 2023-09-30T22:39:06.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:06.001+0000 7fba8faba700 1 -- --> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba881b6ed0 con 0x7fba8809f650 2023-09-30T22:39:06.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:06.001+0000 7fba8faba700 1 -- --> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] -- mon_getmap magic: 0 v1 -- 0x7fba881a2b20 con 0x7fba8809f240 2023-09-30T22:39:06.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:06.001+0000 7fba8d856700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba8809e930 0x7fba881a3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:39:06.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:06.001+0000 7fba8e057700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fba8809f650 0x7fba881c5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:39:06.008 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:06.001+0000 7fba8d856700 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba8809e930 0x7fba881a3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34874/0 (socket says 172.21.15.16:34874) 2023-09-30T22:39:06.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:06.001+0000 7fba8e057700 1 --2- >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fba8809f650 0x7fba881c5db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3301/0 says I am v2:172.21.15.16:39800/0 (socket says 172.21.15.16:39800) 2023-09-30T22:39:06.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:06.001+0000 7fba8d856700 1 -- 172.21.15.16:0/2274693418 learned_addr learned my addr 172.21.15.16:0/2274693418 (peer_addr_for_me v2:172.21.15.16:0/0) 2023-09-30T22:39:06.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:06.001+0000 7fba8d055700 1 --2- 172.21.15.16:0/2274693418 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fba8809f240 0x7fba881a3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-09-30T22:39:06.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:06.001+0000 7fba8d856700 1 -- 172.21.15.16:0/2274693418 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fba8809f240 msgr2=0x7fba881a3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:39:06.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:06.001+0000 7fba8d856700 1 --2- 172.21.15.16:0/2274693418 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fba8809f240 0x7fba881a3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:39:06.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:06.001+0000 7fba8d856700 1 -- 172.21.15.16:0/2274693418 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fba8809f650 msgr2=0x7fba881c5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-09-30T22:39:06.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:06.001+0000 7fba8d856700 1 --2- 172.21.15.16:0/2274693418 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fba8809f650 0x7fba881c5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-09-30T22:39:06.009 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:06.001+0000 7fba8d856700 1 -- 172.21.15.16:0/2274693418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba780094a0 con 0x7fba8809e930 2023-09-30T22:39:06.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:06.001+0000 7fba8e057700 1 --2- 172.21.15.16:0/2274693418 >> [v2:172.21.15.16:3301/0,v1:172.21.15.16:6790/0] conn(0x7fba8809f650 0x7fba881c5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:39:06.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:06.001+0000 7fba8d055700 1 --2- 172.21.15.16:0/2274693418 >> [v2:172.21.15.16:3302/0,v1:172.21.15.16:6791/0] conn(0x7fba8809f240 0x7fba881a3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-09-30T22:39:06.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:06.001+0000 7fba8d856700 1 --2- 172.21.15.16:0/2274693418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba8809e930 0x7fba881a3010 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7fba84009fd0 tx=0x7fba84004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-09-30T22:39:06.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:06.001+0000 7fba7e7fc700 1 -- 172.21.15.16:0/2274693418 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba84004550 con 0x7fba8809e930 2023-09-30T22:39:06.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:06.001+0000 7fba7e7fc700 1 -- 172.21.15.16:0/2274693418 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba8401f070 con 0x7fba8809e930 2023-09-30T22:39:06.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:06.001+0000 7fba8faba700 1 -- 172.21.15.16:0/2274693418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba840094a0 con 0x7fba8809e930 2023-09-30T22:39:06.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:06.001+0000 7fba7e7fc700 1 -- 172.21.15.16:0/2274693418 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba840038e0 con 0x7fba8809e930 2023-09-30T22:39:06.010 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:06.002+0000 7fba8faba700 1 -- 172.21.15.16:0/2274693418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba880a0190 con 0x7fba8809e930 2023-09-30T22:39:06.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:06.002+0000 7fba8faba700 1 -- 172.21.15.16:0/2274693418 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba88074ea0 con 0x7fba8809e930 2023-09-30T22:39:06.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:06.003+0000 7fba7e7fc700 1 -- 172.21.15.16:0/2274693418 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 10) v1 ==== 35252+0+0 (secure 0 0 0) 0x7fba84003a40 con 0x7fba8809e930 2023-09-30T22:39:06.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:06.003+0000 7fba7e7fc700 1 -- 172.21.15.16:0/2274693418 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4393+0+0 (secure 0 0 0) 0x7fba84021070 con 0x7fba8809e930 2023-09-30T22:39:06.011 INFO:teuthology.orchestra.run.smithi016.stderr:2023-09-30T22:39:06.007+0000 7fba7e7fc700 1 -- 172.21.15.16:0/2274693418 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173876 (secure 0 0 0) 0x7fba840253e0 con 0x7fba8809e930 2023-09-30T22:41:05.895 DEBUG:teuthology.orchestra.run:got remote process result: 124 2023-09-30T22:41:05.896 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2742, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2574, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2340, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 1595, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi016 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2023-09-30T22:41:05.904 INFO:teuthology.misc:Shutting down mds daemons... 2023-09-30T22:41:05.904 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-09-30T22:41:05.905 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T22:41:05.919 INFO:tasks.ceph.mds.a:Stopped 2023-09-30T22:41:05.919 INFO:teuthology.misc:Shutting down osd daemons... 2023-09-30T22:41:05.919 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-09-30T22:41:05.919 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T22:41:05.988 INFO:tasks.ceph.osd.0:Stopped 2023-09-30T22:41:05.989 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-09-30T22:41:05.989 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T22:41:06.059 INFO:tasks.ceph.osd.1:Stopped 2023-09-30T22:41:06.059 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-09-30T22:41:06.059 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T22:41:06.127 INFO:tasks.ceph.osd.2:Stopped 2023-09-30T22:41:06.128 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-09-30T22:41:06.128 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T22:41:06.196 INFO:tasks.ceph.osd.3:Stopped 2023-09-30T22:41:06.196 INFO:teuthology.misc:Shutting down mgr daemons... 2023-09-30T22:41:06.196 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-09-30T22:41:06.196 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T22:41:06.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T22:41:06.197 ERROR:teuthology.orchestra.daemon.state:Error while waiting for process to exit Traceback (most recent call last): File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph.py", line 1469, in run_daemon yield File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2742, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2574, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2340, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 1595, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi016 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/daemon/state.py", line 139, in stop run.wait([self.proc], timeout=timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi016 with status 1: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x' 2023-09-30T22:41:06.208 INFO:tasks.ceph.mgr.x:Stopped 2023-09-30T22:41:06.208 INFO:teuthology.misc:Shutting down mon daemons... 2023-09-30T22:41:06.208 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-09-30T22:41:06.208 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T22:41:06.250 INFO:tasks.ceph.mon.a:Stopped 2023-09-30T22:41:06.250 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-09-30T22:41:06.250 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T22:41:06.304 INFO:tasks.ceph.mon.b:Stopped 2023-09-30T22:41:06.304 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-09-30T22:41:06.305 INFO:teuthology.orchestra.run:waiting for 300 2023-09-30T22:41:06.358 INFO:tasks.ceph.mon.c:Stopped 2023-09-30T22:41:06.358 INFO:tasks.ceph:Checking cluster log for badness... 2023-09-30T22:41:06.358 DEBUG:teuthology.orchestra.run.smithi016:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v 'overall HEALTH_' | egrep -v '\(FS_' | egrep -v '\(MDS_' | egrep -v '\(OSD_' | egrep -v '\(MON_DOWN\)' | egrep -v '\(CACHE_POOL_' | egrep -v '\(POOL_' | egrep -v '\(MGR_DOWN\)' | egrep -v '\(PG_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v 'Monitor daemon marked osd' | egrep -v 'Behind on trimming' | egrep -v 'Manager daemon' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'scrub mismatch' | egrep -v ScrubResult | egrep -v 'wrongly marked' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(SLOW_OPS\)' | egrep -v 'overall HEALTH_' | egrep -v '\(MON_MSGR2_NOT_ENABLED\)' | egrep -v 'slow request' | head -n 1 2023-09-30T22:41:06.447 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi016.front.sepia.ceph.com 2023-09-30T22:41:06.447 DEBUG:teuthology.orchestra.run.smithi016:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-09-30T22:41:06.618 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi016.front.sepia.ceph.com 2023-09-30T22:41:06.618 DEBUG:teuthology.orchestra.run.smithi016:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-09-30T22:41:06.708 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi016.front.sepia.ceph.com 2023-09-30T22:41:06.708 DEBUG:teuthology.orchestra.run.smithi016:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-09-30T22:41:06.815 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi016.front.sepia.ceph.com 2023-09-30T22:41:06.815 DEBUG:teuthology.orchestra.run.smithi016:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-09-30T22:41:06.881 INFO:tasks.ceph:Archiving mon data... 2023-09-30T22:41:06.882 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7408044/data/mon.a.tgz 2023-09-30T22:41:06.882 DEBUG:teuthology.orchestra.run.smithi016:> mktemp 2023-09-30T22:41:06.905 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/tmp.uH9tohRmRs 2023-09-30T22:41:06.905 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.uH9tohRmRs 2023-09-30T22:41:07.054 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0666 /tmp/tmp.uH9tohRmRs 2023-09-30T22:41:07.151 DEBUG:teuthology.orchestra.remote:smithi016:/tmp/tmp.uH9tohRmRs is 329KB 2023-09-30T22:41:07.250 DEBUG:teuthology.orchestra.run.smithi016:> rm -fr /tmp/tmp.uH9tohRmRs 2023-09-30T22:41:07.273 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7408044/data/mon.b.tgz 2023-09-30T22:41:07.273 DEBUG:teuthology.orchestra.run.smithi016:> mktemp 2023-09-30T22:41:07.336 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/tmp.VcHelhigsZ 2023-09-30T22:41:07.337 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.VcHelhigsZ 2023-09-30T22:41:07.480 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0666 /tmp/tmp.VcHelhigsZ 2023-09-30T22:41:07.576 DEBUG:teuthology.orchestra.remote:smithi016:/tmp/tmp.VcHelhigsZ is 332KB 2023-09-30T22:41:07.670 DEBUG:teuthology.orchestra.run.smithi016:> rm -fr /tmp/tmp.VcHelhigsZ 2023-09-30T22:41:07.692 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7408044/data/mon.c.tgz 2023-09-30T22:41:07.693 DEBUG:teuthology.orchestra.run.smithi016:> mktemp 2023-09-30T22:41:07.755 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/tmp.Zys1lXTCSJ 2023-09-30T22:41:07.756 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.Zys1lXTCSJ 2023-09-30T22:41:07.898 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0666 /tmp/tmp.Zys1lXTCSJ 2023-09-30T22:41:07.994 DEBUG:teuthology.orchestra.remote:smithi016:/tmp/tmp.Zys1lXTCSJ is 332KB 2023-09-30T22:41:08.068 DEBUG:teuthology.orchestra.run.smithi016:> rm -fr /tmp/tmp.Zys1lXTCSJ 2023-09-30T22:41:08.091 INFO:tasks.ceph:Cleaning ceph cluster... 2023-09-30T22:41:08.091 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-09-30T22:41:08.133 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-09-30T22:41:08.171 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-09-30T22:41:08.171 DEBUG:teuthology.orchestra.run.smithi016:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-09-30T22:41:08.209 DEBUG:teuthology.orchestra.run.smithi027:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-09-30T22:41:08.243 INFO:teuthology.orchestra.run.smithi016.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-09-30T22:41:08.245 INFO:teuthology.orchestra.run.smithi027.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-09-30T22:41:08.252 INFO:tasks.ceph:Archiving crash dumps... 2023-09-30T22:41:08.254 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/lib/ceph/crash to /home/teuthworker/archive/dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7408044/remote/smithi016/crash 2023-09-30T22:41:08.255 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-09-30T22:41:08.327 DEBUG:teuthology.misc:Transferring archived files from smithi027:/var/lib/ceph/crash to /home/teuthworker/archive/dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7408044/remote/smithi027/crash 2023-09-30T22:41:08.328 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-09-30T22:41:08.382 INFO:tasks.ceph:Compressing logs... 2023-09-30T22:41:08.382 DEBUG:teuthology.orchestra.run.smithi016:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-09-30T22:41:08.385 DEBUG:teuthology.orchestra.run.smithi027:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-09-30T22:41:08.414 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.112319.log 2023-09-30T22:41:08.419 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167117.log 2023-09-30T22:41:08.419 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170449.log 2023-09-30T22:41:08.420 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.167117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165183.log 2023-09-30T22:41:08.427 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/ceph-client.0.112319.log: 96.6% -- replaced with /var/log/ceph/ceph-client.0.112319.log.gz 2023-09-30T22:41:08.427 INFO:teuthology.orchestra.run.smithi027.stderr: 2023-09-30T22:41:08.427 INFO:teuthology.orchestra.run.smithi027.stderr:real 0m0.020s 2023-09-30T22:41:08.427 INFO:teuthology.orchestra.run.smithi027.stderr:user 0m0.013s 2023-09-30T22:41:08.427 INFO:teuthology.orchestra.run.smithi027.stderr:sys 0m0.017s 2023-09-30T22:41:08.442 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.167117.log.gz 2023-09-30T22:41:08.443 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.170449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178124.log 2023-09-30T22:41:08.443 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.165183.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.170449.log.gz 2023-09-30T22:41:08.443 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140806.log 2023-09-30T22:41:08.443 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.165183.log.gz 2023-09-30T22:41:08.443 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189220.log 2023-09-30T22:41:08.443 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.178124.log: /var/log/ceph/ceph-client.admin.140806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190344.log 2023-09-30T22:41:08.443 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.178124.log.gz 2023-09-30T22:41:08.444 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.140806.log.gz 2023-09-30T22:41:08.444 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.189220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136121.log 2023-09-30T22:41:08.444 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.189220.log.gz 2023-09-30T22:41:08.444 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.190344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142591.log 2023-09-30T22:41:08.444 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.190344.log.gz 2023-09-30T22:41:08.444 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.136121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163779.log 2023-09-30T22:41:08.444 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.142591.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.136121.log.gz 2023-09-30T22:41:08.445 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189756.log 2023-09-30T22:41:08.445 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.142591.log.gz 2023-09-30T22:41:08.445 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.163779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155741.log 2023-09-30T22:41:08.445 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.163779.log.gz 2023-09-30T22:41:08.445 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.189756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158129.log 2023-09-30T22:41:08.445 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.155741.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.189756.log.gz 2023-09-30T22:41:08.445 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148332.log 2023-09-30T22:41:08.445 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.155741.log.gz 2023-09-30T22:41:08.446 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.158129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174265.log 2023-09-30T22:41:08.446 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.158129.log.gz 2023-09-30T22:41:08.446 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.148332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136656.log 2023-09-30T22:41:08.446 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.174265.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.148332.log.gz 2023-09-30T22:41:08.446 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143106.log 2023-09-30T22:41:08.446 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.174265.log.gz 2023-09-30T22:41:08.446 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.136656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162601.log 2023-09-30T22:41:08.447 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.136656.log.gz 2023-09-30T22:41:08.447 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.143106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183669.log 2023-09-30T22:41:08.447 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.162601.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.143106.log.gz 2023-09-30T22:41:08.447 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180559.log 2023-09-30T22:41:08.447 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.162601.log.gz 2023-09-30T22:41:08.447 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.183669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153403.log 2023-09-30T22:41:08.447 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.183669.log.gz 2023-09-30T22:41:08.447 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.180559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174471.log 2023-09-30T22:41:08.448 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.153403.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.180559.log.gz 2023-09-30T22:41:08.448 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179643.log 2023-09-30T22:41:08.448 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.153403.log.gz 2023-09-30T22:41:08.448 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.174471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140657.log 2023-09-30T22:41:08.448 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.174471.log.gz 2023-09-30T22:41:08.448 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.179643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118222.log 2023-09-30T22:41:08.449 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.140657.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.179643.log.gz 2023-09-30T22:41:08.449 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138198.log 2023-09-30T22:41:08.449 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.140657.log.gz 2023-09-30T22:41:08.449 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.118222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169639.log 2023-09-30T22:41:08.449 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.138198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157139.log 2023-09-30T22:41:08.449 INFO:teuthology.orchestra.run.smithi016.stderr: 93.5% -- replaced with /var/log/ceph/ceph-client.admin.118222.log.gz 2023-09-30T22:41:08.449 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.169639.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.138198.log.gz 2023-09-30T22:41:08.449 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152278.log 2023-09-30T22:41:08.450 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.169639.log.gz 2023-09-30T22:41:08.450 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.157139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188795.log 2023-09-30T22:41:08.450 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.157139.log.gz 2023-09-30T22:41:08.450 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.152278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192760.log 2023-09-30T22:41:08.450 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.152278.log.gz 2023-09-30T22:41:08.450 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.188795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154434.log 2023-09-30T22:41:08.450 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.188795.log.gz 2023-09-30T22:41:08.450 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143498.log 2023-09-30T22:41:08.451 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.192760.log: /var/log/ceph/ceph-client.admin.154434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168097.log 2023-09-30T22:41:08.451 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.192760.log.gz 2023-09-30T22:41:08.451 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.143498.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.154434.log.gz 2023-09-30T22:41:08.451 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122561.log 2023-09-30T22:41:08.451 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.143498.log.gz 2023-09-30T22:41:08.451 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.168097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173567.log 2023-09-30T22:41:08.451 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.168097.log.gz 2023-09-30T22:41:08.451 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.122561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154407.log 2023-09-30T22:41:08.452 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.122561.log.gz 2023-09-30T22:41:08.452 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.173567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145363.log 2023-09-30T22:41:08.452 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.173567.log.gz 2023-09-30T22:41:08.452 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.154407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189416.log 2023-09-30T22:41:08.452 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.154407.log.gz 2023-09-30T22:41:08.452 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.145363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177980.log 2023-09-30T22:41:08.452 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.145363.log.gz 2023-09-30T22:41:08.452 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.189416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176703.log 2023-09-30T22:41:08.453 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.189416.log.gz 2023-09-30T22:41:08.453 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.177980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159857.log 2023-09-30T22:41:08.453 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.176703.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.177980.log.gz 2023-09-30T22:41:08.453 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164739.log 2023-09-30T22:41:08.453 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.176703.log.gz 2023-09-30T22:41:08.453 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.159857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157774.log 2023-09-30T22:41:08.453 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.159857.log.gz 2023-09-30T22:41:08.453 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.164739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163262.log 2023-09-30T22:41:08.454 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.157774.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.164739.log.gz 2023-09-30T22:41:08.454 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154186.log 2023-09-30T22:41:08.454 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.157774.log.gz 2023-09-30T22:41:08.454 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.163262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140950.log 2023-09-30T22:41:08.454 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.163262.log.gz 2023-09-30T22:41:08.454 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.154186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134082.log 2023-09-30T22:41:08.454 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.154186.log.gz 2023-09-30T22:41:08.454 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.140950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144553.log 2023-09-30T22:41:08.455 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.140950.log.gz 2023-09-30T22:41:08.455 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.134082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160811.log 2023-09-30T22:41:08.455 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.134082.log.gz 2023-09-30T22:41:08.455 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.144553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135844.log 2023-09-30T22:41:08.455 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.160811.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.144553.log.gz 2023-09-30T22:41:08.455 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193936.log 2023-09-30T22:41:08.455 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.160811.log.gz 2023-09-30T22:41:08.455 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.135844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147371.log 2023-09-30T22:41:08.456 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.135844.log.gz 2023-09-30T22:41:08.456 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.193936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125831.log 2023-09-30T22:41:08.456 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.147371.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.193936.log.gz 2023-09-30T22:41:08.456 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146432.log 2023-09-30T22:41:08.456 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.147371.log.gz 2023-09-30T22:41:08.456 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.125831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129368.log 2023-09-30T22:41:08.456 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.125831.log.gz 2023-09-30T22:41:08.457 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.146432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139408.log 2023-09-30T22:41:08.457 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.146432.log.gz 2023-09-30T22:41:08.457 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.129368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188572.log 2023-09-30T22:41:08.457 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.129368.log.gz 2023-09-30T22:41:08.457 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.139408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138812.log 2023-09-30T22:41:08.457 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.139408.log.gz 2023-09-30T22:41:08.457 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.188572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182248.log 2023-09-30T22:41:08.457 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.138812.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.188572.log.gz 2023-09-30T22:41:08.458 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182052.log 2023-09-30T22:41:08.458 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.138812.log.gz 2023-09-30T22:41:08.458 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.182248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153159.log 2023-09-30T22:41:08.458 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.182248.log.gz 2023-09-30T22:41:08.458 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.182052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127910.log 2023-09-30T22:41:08.458 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.153159.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.182052.log.gz 2023-09-30T22:41:08.458 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157113.log 2023-09-30T22:41:08.458 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.153159.log.gz 2023-09-30T22:41:08.459 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.127910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165719.log 2023-09-30T22:41:08.459 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.127910.log.gz 2023-09-30T22:41:08.459 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.157113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183891.log 2023-09-30T22:41:08.459 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.157113.log.gz 2023-09-30T22:41:08.459 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.165719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166725.log 2023-09-30T22:41:08.459 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.165719.log.gz 2023-09-30T22:41:08.459 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.183891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156721.log 2023-09-30T22:41:08.459 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.183891.log.gz 2023-09-30T22:41:08.460 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.166725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168659.log 2023-09-30T22:41:08.460 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.166725.log.gz 2023-09-30T22:41:08.460 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.156721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167483.log 2023-09-30T22:41:08.556 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.168659.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.156721.log.gz 2023-09-30T22:41:08.556 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133884.log 2023-09-30T22:41:08.556 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.168659.log.gz 2023-09-30T22:41:08.556 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.167483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181121.log 2023-09-30T22:41:08.557 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.167483.log.gz 2023-09-30T22:41:08.557 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.133884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143598.log 2023-09-30T22:41:08.557 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.181121.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.133884.log.gz 2023-09-30T22:41:08.557 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142983.log 2023-09-30T22:41:08.557 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.181121.log.gz 2023-09-30T22:41:08.557 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.143598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136926.log 2023-09-30T22:41:08.557 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.143598.log.gz 2023-09-30T22:41:08.558 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.142983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171037.log 2023-09-30T22:41:08.558 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.142983.log.gz 2023-09-30T22:41:08.558 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.136926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163849.log 2023-09-30T22:41:08.558 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.136926.log.gz 2023-09-30T22:41:08.558 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.171037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186465.log 2023-09-30T22:41:08.558 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.171037.log.gz 2023-09-30T22:41:08.558 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.163849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169521.log 2023-09-30T22:41:08.559 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.186465.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.163849.log.gz 2023-09-30T22:41:08.559 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156551.log 2023-09-30T22:41:08.559 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.186465.log.gz 2023-09-30T22:41:08.559 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.169521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145167.log 2023-09-30T22:41:08.559 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.169521.log.gz 2023-09-30T22:41:08.559 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.156551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166384.log 2023-09-30T22:41:08.559 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.145167.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.156551.log.gz 2023-09-30T22:41:08.559 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178960.log 2023-09-30T22:41:08.560 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.145167.log.gz 2023-09-30T22:41:08.560 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.166384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128625.log 2023-09-30T22:41:08.560 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.166384.log.gz 2023-09-30T22:41:08.560 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.178960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140244.log 2023-09-30T22:41:08.560 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.128625.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.178960.log.gz 2023-09-30T22:41:08.560 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172783.log 2023-09-30T22:41:08.560 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.128625.log.gz 2023-09-30T22:41:08.561 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.140244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180280.log 2023-09-30T22:41:08.561 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.140244.log.gz 2023-09-30T22:41:08.561 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.172783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152665.log 2023-09-30T22:41:08.561 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.180280.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.172783.log.gz 2023-09-30T22:41:08.561 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165941.log 2023-09-30T22:41:08.561 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.180280.log.gz 2023-09-30T22:41:08.561 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.152665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188768.log 2023-09-30T22:41:08.561 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.152665.log.gz 2023-09-30T22:41:08.562 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.165941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158558.log 2023-09-30T22:41:08.562 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.165941.log.gz 2023-09-30T22:41:08.562 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.188768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147697.log 2023-09-30T22:41:08.562 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.188768.log.gz 2023-09-30T22:41:08.562 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.158558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162797.log 2023-09-30T22:41:08.562 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.158558.log.gz 2023-09-30T22:41:08.562 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.147697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141656.log 2023-09-30T22:41:08.562 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.162797.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.147697.log.gz 2023-09-30T22:41:08.563 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176115.log 2023-09-30T22:41:08.563 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.162797.log.gz 2023-09-30T22:41:08.563 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.141656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175331.log 2023-09-30T22:41:08.563 INFO:teuthology.orchestra.run.smithi016.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.141656.log.gz 2023-09-30T22:41:08.563 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.176115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185041.log 2023-09-30T22:41:08.563 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.175331.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.176115.log.gz 2023-09-30T22:41:08.563 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158877.log 2023-09-30T22:41:08.563 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.175331.log.gz 2023-09-30T22:41:08.564 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.185041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187251.log 2023-09-30T22:41:08.564 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.185041.log.gz 2023-09-30T22:41:08.564 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.158877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170867.log 2023-09-30T22:41:08.564 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.187251.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.158877.log.gz 2023-09-30T22:41:08.564 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136119.log 2023-09-30T22:41:08.564 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.187251.log.gz 2023-09-30T22:41:08.564 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.170867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168366.log 2023-09-30T22:41:08.564 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.170867.log.gz 2023-09-30T22:41:08.565 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.136119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157971.log 2023-09-30T22:41:08.565 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.168366.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.136119.log.gz 2023-09-30T22:41:08.565 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145141.log 2023-09-30T22:41:08.565 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.168366.log.gz 2023-09-30T22:41:08.565 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.157971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182391.log 2023-09-30T22:41:08.565 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.157971.log.gz 2023-09-30T22:41:08.565 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.145141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136066.log 2023-09-30T22:41:08.565 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.182391.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.145141.log.gz 2023-09-30T22:41:08.566 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127426.log 2023-09-30T22:41:08.566 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.182391.log.gz 2023-09-30T22:41:08.566 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.136066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134864.log 2023-09-30T22:41:08.566 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.136066.log.gz 2023-09-30T22:41:08.566 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.127426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169076.log 2023-09-30T22:41:08.566 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.127426.log.gz 2023-09-30T22:41:08.566 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.134864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169690.log 2023-09-30T22:41:08.566 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.134864.log.gz 2023-09-30T22:41:08.567 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.169076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123112.log 2023-09-30T22:41:08.567 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.169076.log.gz 2023-09-30T22:41:08.567 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.169690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165131.log 2023-09-30T22:41:08.567 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.123112.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.169690.log.gz 2023-09-30T22:41:08.567 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170619.log 2023-09-30T22:41:08.567 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.123112.log.gz 2023-09-30T22:41:08.567 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.165131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184257.log 2023-09-30T22:41:08.567 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.165131.log.gz 2023-09-30T22:41:08.568 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.170619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160105.log 2023-09-30T22:41:08.568 INFO:teuthology.orchestra.run.smithi016.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.170619.log.gz 2023-09-30T22:41:08.568 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.184257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146317.log 2023-09-30T22:41:08.568 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.184257.log.gz 2023-09-30T22:41:08.568 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.160105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146709.log 2023-09-30T22:41:08.568 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.160105.log.gz 2023-09-30T22:41:08.568 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.146317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122659.log 2023-09-30T22:41:08.568 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.146317.log.gz 2023-09-30T22:41:08.569 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.146709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149721.log 2023-09-30T22:41:08.569 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.146709.log.gz 2023-09-30T22:41:08.569 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.122659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164053.log 2023-09-30T22:41:08.569 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.122659.log.gz 2023-09-30T22:41:08.569 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.149721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124018.log 2023-09-30T22:41:08.569 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.149721.log.gz 2023-09-30T22:41:08.569 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.164053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138839.log 2023-09-30T22:41:08.569 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.164053.log.gz 2023-09-30T22:41:08.570 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.124018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152179.log 2023-09-30T22:41:08.570 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.124018.log.gz 2023-09-30T22:41:08.570 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.138839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150833.log 2023-09-30T22:41:08.570 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.138839.log.gz 2023-09-30T22:41:08.570 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.152179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122904.log 2023-09-30T22:41:08.570 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.152179.log.gz 2023-09-30T22:41:08.570 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.150833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174054.log 2023-09-30T22:41:08.570 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.150833.log.gz 2023-09-30T22:41:08.571 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.122904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136630.log 2023-09-30T22:41:08.571 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.122904.log.gz 2023-09-30T22:41:08.571 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.174054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161058.log 2023-09-30T22:41:08.571 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.174054.log.gz 2023-09-30T22:41:08.571 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.136630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187888.log 2023-09-30T22:41:08.571 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.136630.log.gz 2023-09-30T22:41:08.571 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.161058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178150.log 2023-09-30T22:41:08.571 INFO:teuthology.orchestra.run.smithi016.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.161058.log.gz 2023-09-30T22:41:08.571 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.187888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170671.log 2023-09-30T22:41:08.572 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.187888.log.gz 2023-09-30T22:41:08.572 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.178150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171625.log 2023-09-30T22:41:08.572 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.170671.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.178150.log.gz 2023-09-30T22:41:08.572 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146879.log 2023-09-30T22:41:08.572 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.170671.log.gz 2023-09-30T22:41:08.572 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.171625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161475.log 2023-09-30T22:41:08.572 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.171625.log.gz 2023-09-30T22:41:08.572 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.146879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148528.log 2023-09-30T22:41:08.573 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.161475.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.146879.log.gz 2023-09-30T22:41:08.573 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191350.log 2023-09-30T22:41:08.573 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.161475.log.gz 2023-09-30T22:41:08.573 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.148528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170841.log 2023-09-30T22:41:08.573 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.148528.log.gz 2023-09-30T22:41:08.573 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.191350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143207.log 2023-09-30T22:41:08.573 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.191350.log.gz 2023-09-30T22:41:08.573 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.170841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158459.log 2023-09-30T22:41:08.574 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.170841.log.gz 2023-09-30T22:41:08.574 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.143207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144775.log 2023-09-30T22:41:08.574 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.143207.log.gz 2023-09-30T22:41:08.574 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.158459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162674.log 2023-09-30T22:41:08.574 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.144775.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.158459.log.gz 2023-09-30T22:41:08.574 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154754.log 2023-09-30T22:41:08.574 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.144775.log.gz 2023-09-30T22:41:08.574 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.162674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129417.log 2023-09-30T22:41:08.575 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.162674.log.gz 2023-09-30T22:41:08.575 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.154754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140558.log 2023-09-30T22:41:08.575 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.129417.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.154754.log.gz 2023-09-30T22:41:08.575 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147530.log 2023-09-30T22:41:08.575 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.140558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181735.log 2023-09-30T22:41:08.575 INFO:teuthology.orchestra.run.smithi016.stderr: 91.6% -- replaced with /var/log/ceph/ceph-client.admin.129417.log.gz 2023-09-30T22:41:08.575 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.140558.log.gz 2023-09-30T22:41:08.575 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.147530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180084.log 2023-09-30T22:41:08.576 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.147530.log.gz 2023-09-30T22:41:08.576 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.181735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144579.log 2023-09-30T22:41:08.576 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.181735.log.gz 2023-09-30T22:41:08.576 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.180084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145899.log 2023-09-30T22:41:08.576 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.180084.log.gz 2023-09-30T22:41:08.576 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.144579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191154.log 2023-09-30T22:41:08.576 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.144579.log.gz 2023-09-30T22:41:08.576 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.145899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181709.log 2023-09-30T22:41:08.577 INFO:teuthology.orchestra.run.smithi016.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.145899.log.gz 2023-09-30T22:41:08.577 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.191154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143402.log 2023-09-30T22:41:08.577 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.191154.log.gz 2023-09-30T22:41:08.577 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.181709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175447.log 2023-09-30T22:41:08.577 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.181709.log.gz 2023-09-30T22:41:08.577 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.143402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146979.log 2023-09-30T22:41:08.577 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.143402.log.gz 2023-09-30T22:41:08.577 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.175447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126945.log 2023-09-30T22:41:08.578 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.175447.log.gz 2023-09-30T22:41:08.578 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.146979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184675.log 2023-09-30T22:41:08.578 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.146979.log.gz 2023-09-30T22:41:08.578 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.126945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186661.log 2023-09-30T22:41:08.578 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9%/var/log/ceph/ceph-client.admin.184675.log: -- replaced with /var/log/ceph/ceph-client.admin.126945.log.gz 2023-09-30T22:41:08.578 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163190.log 2023-09-30T22:41:08.578 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.184675.log.gz 2023-09-30T22:41:08.579 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.186661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172069.log 2023-09-30T22:41:08.579 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.186661.log.gz 2023-09-30T22:41:08.579 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.163190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136529.log 2023-09-30T22:41:08.579 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.163190.log.gz 2023-09-30T22:41:08.579 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.172069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186364.log 2023-09-30T22:41:08.579 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.172069.log.gz 2023-09-30T22:41:08.579 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.136529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161647.log 2023-09-30T22:41:08.579 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.136529.log.gz 2023-09-30T22:41:08.580 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.186364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140243.log 2023-09-30T22:41:08.580 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.186364.log.gz 2023-09-30T22:41:08.580 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.161647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140754.log 2023-09-30T22:41:08.580 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.161647.log.gz 2023-09-30T22:41:08.580 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.140243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177928.log 2023-09-30T22:41:08.580 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.140754.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.140243.log.gz 2023-09-30T22:41:08.580 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143302.log 2023-09-30T22:41:08.580 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.140754.log.gz 2023-09-30T22:41:08.581 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.177928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175647.log 2023-09-30T22:41:08.581 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.177928.log.gz 2023-09-30T22:41:08.581 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.143302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193166.log 2023-09-30T22:41:08.581 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.143302.log.gz 2023-09-30T22:41:08.581 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.175647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134550.log 2023-09-30T22:41:08.581 INFO:teuthology.orchestra.run.smithi016.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.175647.log.gz 2023-09-30T22:41:08.581 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.193166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144331.log 2023-09-30T22:41:08.581 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.193166.log.gz 2023-09-30T22:41:08.582 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.134550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183303.log 2023-09-30T22:41:08.582 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.144331.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.134550.log.gz 2023-09-30T22:41:08.582 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182689.log 2023-09-30T22:41:08.582 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.144331.log.gz 2023-09-30T22:41:08.582 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.183303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138054.log 2023-09-30T22:41:08.582 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.183303.log.gz 2023-09-30T22:41:08.582 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.182689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178346.log 2023-09-30T22:41:08.582 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.138054.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.182689.log.gz 2023-09-30T22:41:08.583 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170306.log 2023-09-30T22:41:08.583 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.138054.log.gz 2023-09-30T22:41:08.583 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147906.log 2023-09-30T22:41:08.583 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.178346.log: /var/log/ceph/ceph-client.admin.170306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170109.log 2023-09-30T22:41:08.583 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.178346.log.gz 2023-09-30T22:41:08.583 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.170306.log.gz 2023-09-30T22:41:08.583 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.147906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163458.log 2023-09-30T22:41:08.584 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.147906.log.gz 2023-09-30T22:41:08.584 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.170109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127399.log 2023-09-30T22:41:08.584 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.163458.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.170109.log.gz 2023-09-30T22:41:08.584 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137832.log 2023-09-30T22:41:08.584 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.163458.log.gz 2023-09-30T22:41:08.584 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.127399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134694.log 2023-09-30T22:41:08.584 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.127399.log.gz 2023-09-30T22:41:08.584 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.137832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125586.log 2023-09-30T22:41:08.585 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.137832.log.gz 2023-09-30T22:41:08.585 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.134694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150246.log 2023-09-30T22:41:08.585 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.125586.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.134694.log.gz 2023-09-30T22:41:08.585 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134548.log 2023-09-30T22:41:08.585 INFO:teuthology.orchestra.run.smithi016.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.125586.log.gz 2023-09-30T22:41:08.585 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.150246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134276.log 2023-09-30T22:41:08.585 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.150246.log.gz 2023-09-30T22:41:08.585 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.134548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192381.log 2023-09-30T22:41:08.586 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.134276.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.134548.log.gz 2023-09-30T22:41:08.586 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172490.log 2023-09-30T22:41:08.586 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.134276.log.gz 2023-09-30T22:41:08.586 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.192381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125635.log 2023-09-30T22:41:08.586 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.172490.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.192381.log.gz 2023-09-30T22:41:08.586 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168685.log 2023-09-30T22:41:08.586 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.172490.log.gz 2023-09-30T22:41:08.587 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.125635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177144.log 2023-09-30T22:41:08.587 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.125635.log.gz 2023-09-30T22:41:08.587 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.168685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150023.log 2023-09-30T22:41:08.587 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.177144.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.168685.log.gz 2023-09-30T22:41:08.587 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156133.log 2023-09-30T22:41:08.587 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.177144.log.gz/var/log/ceph/ceph-client.admin.150023.log: 2023-09-30T22:41:08.587 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149043.log 2023-09-30T22:41:08.587 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.150023.log.gz 2023-09-30T22:41:08.588 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.156133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144478.log 2023-09-30T22:41:08.588 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.156133.log.gz 2023-09-30T22:41:08.588 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.149043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145115.log 2023-09-30T22:41:08.588 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.144478.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.149043.log.gz 2023-09-30T22:41:08.588 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184649.log 2023-09-30T22:41:08.588 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.144478.log.gz 2023-09-30T22:41:08.588 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.145115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173642.log 2023-09-30T22:41:08.588 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.145115.log.gz 2023-09-30T22:41:08.589 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.184649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185776.log 2023-09-30T22:41:08.589 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.173642.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.184649.log.gz 2023-09-30T22:41:08.589 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135334.log 2023-09-30T22:41:08.589 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.173642.log.gz 2023-09-30T22:41:08.589 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187814.log 2023-09-30T22:41:08.589 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.185776.log: /var/log/ceph/ceph-client.admin.135334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177291.log 2023-09-30T22:41:08.589 INFO:teuthology.orchestra.run.smithi016.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.185776.log.gz 2023-09-30T22:41:08.590 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.187814.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.135334.log.gz 2023-09-30T22:41:08.590 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189463.log 2023-09-30T22:41:08.590 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.187814.log.gz 2023-09-30T22:41:08.590 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.177291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135673.log 2023-09-30T22:41:08.590 INFO:teuthology.orchestra.run.smithi016.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.177291.log.gz 2023-09-30T22:41:08.590 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.189463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175723.log 2023-09-30T22:41:08.590 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.135673.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.189463.log.gz 2023-09-30T22:41:08.590 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167705.log 2023-09-30T22:41:08.591 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.135673.log.gz 2023-09-30T22:41:08.591 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.175723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184283.log 2023-09-30T22:41:08.591 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.175723.log.gz 2023-09-30T22:41:08.591 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.167705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192552.log 2023-09-30T22:41:08.591 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.184283.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.167705.log.gz 2023-09-30T22:41:08.591 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129319.log 2023-09-30T22:41:08.591 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.184283.log.gz 2023-09-30T22:41:08.592 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.192552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127270.log 2023-09-30T22:41:08.592 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.192552.log.gz 2023-09-30T22:41:08.592 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.129319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152670.log 2023-09-30T22:41:08.592 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.129319.log.gz 2023-09-30T22:41:08.592 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.127270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157898.log 2023-09-30T22:41:08.592 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.152670.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.127270.log.gz 2023-09-30T22:41:08.592 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118670.log 2023-09-30T22:41:08.592 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.152670.log.gz 2023-09-30T22:41:08.593 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.157898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128113.log 2023-09-30T22:41:08.593 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.118670.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.157898.log.gz 2023-09-30T22:41:08.593 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172686.log 2023-09-30T22:41:08.593 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.118670.log.gz 2023-09-30T22:41:08.593 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.128113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185825.log 2023-09-30T22:41:08.593 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.128113.log.gz 2023-09-30T22:41:08.593 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.172686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186268.log 2023-09-30T22:41:08.593 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.185825.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.172686.log.gz 2023-09-30T22:41:08.594 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177000.log 2023-09-30T22:41:08.594 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.185825.log.gz 2023-09-30T22:41:08.594 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.186268.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-09-30T22:41:08.594 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.186268.log.gz 2023-09-30T22:41:08.594 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.177000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190591.log 2023-09-30T22:41:08.594 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-osd.2.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.177000.log.gz 2023-09-30T22:41:08.594 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142714.log 2023-09-30T22:41:08.594 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.190591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180333.log 2023-09-30T22:41:08.595 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.190591.log.gz 2023-09-30T22:41:08.595 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.142714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174743.log 2023-09-30T22:41:08.595 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.142714.log.gz 2023-09-30T22:41:08.595 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.180333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178568.log 2023-09-30T22:41:08.595 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.180333.log.gz 2023-09-30T22:41:08.595 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184992.log 2023-09-30T22:41:08.595 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.174743.log: /var/log/ceph/ceph-client.admin.178568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171429.log 2023-09-30T22:41:08.595 INFO:teuthology.orchestra.run.smithi016.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.174743.log.gz 2023-09-30T22:41:08.596 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.184992.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.178568.log.gz 2023-09-30T22:41:08.596 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139481.log 2023-09-30T22:41:08.596 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.184992.log.gz 2023-09-30T22:41:08.596 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162673.log 2023-09-30T22:41:08.596 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.171429.log: /var/log/ceph/ceph-client.admin.139481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155911.log 2023-09-30T22:41:08.596 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.171429.log.gz 2023-09-30T22:41:08.596 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.139481.log.gz 2023-09-30T22:41:08.596 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.162673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176778.log 2023-09-30T22:41:08.597 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.162673.log.gz 2023-09-30T22:41:08.597 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.155911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158708.log 2023-09-30T22:41:08.597 INFO:teuthology.orchestra.run.smithi016.stderr:gzip /var/log/ceph/ceph-client.admin.176778.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.143401.log 2023-09-30T22:41:08.597 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.155911.log.gz 2023-09-30T22:41:08.597 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.176778.log.gz 2023-09-30T22:41:08.597 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.158708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139434.log 2023-09-30T22:41:08.597 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.158708.log.gz 2023-09-30T22:41:08.597 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.143401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175643.log 2023-09-30T22:41:08.598 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.143401.log.gz 2023-09-30T22:41:08.598 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.139434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157283.log 2023-09-30T22:41:08.598 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.139434.log.gz 2023-09-30T22:41:08.598 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.175643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188279.log 2023-09-30T22:41:08.598 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.157283.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.175643.log.gz 2023-09-30T22:41:08.598 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172613.log 2023-09-30T22:41:08.598 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.157283.log.gz 2023-09-30T22:41:08.598 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.188279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167145.log 2023-09-30T22:41:08.599 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.172613.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.188279.log.gz 2023-09-30T22:41:08.599 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157187.log 2023-09-30T22:41:08.599 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.167145.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.172613.log.gz 2023-09-30T22:41:08.599 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155937.log 2023-09-30T22:41:08.599 INFO:teuthology.orchestra.run.smithi016.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.167145.log.gz 2023-09-30T22:41:08.599 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.157187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167509.log 2023-09-30T22:41:08.599 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.157187.log.gz 2023-09-30T22:41:08.599 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.155937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172195.log 2023-09-30T22:41:08.600 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.167509.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.155937.log.gz 2023-09-30T22:41:08.600 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163163.log 2023-09-30T22:41:08.600 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.167509.log.gz 2023-09-30T22:41:08.600 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.172195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156191.log 2023-09-30T22:41:08.600 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.172195.log.gz 2023-09-30T22:41:08.600 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.163163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154823.log 2023-09-30T22:41:08.600 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.156191.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.163163.log.gz 2023-09-30T22:41:08.601 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162038.log 2023-09-30T22:41:08.601 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.156191.log.gz 2023-09-30T22:41:08.601 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.154823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186750.log 2023-09-30T22:41:08.601 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.154823.log.gz 2023-09-30T22:41:08.601 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.162038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185877.log 2023-09-30T22:41:08.601 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.186750.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.162038.log.gz 2023-09-30T22:41:08.601 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146344.log 2023-09-30T22:41:08.601 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.186750.log.gz 2023-09-30T22:41:08.602 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.185877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187841.log 2023-09-30T22:41:08.602 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.185877.log.gz 2023-09-30T22:41:08.602 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.146344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166555.log 2023-09-30T22:41:08.602 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.187841.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.146344.log.gz 2023-09-30T22:41:08.602 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151003.log 2023-09-30T22:41:08.602 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.187841.log.gz 2023-09-30T22:41:08.602 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.166555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182323.log 2023-09-30T22:41:08.602 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.166555.log.gz 2023-09-30T22:41:08.603 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.151003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185093.log 2023-09-30T22:41:08.603 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.182323.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.151003.log.gz 2023-09-30T22:41:08.603 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155367.log 2023-09-30T22:41:08.603 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.182323.log.gz 2023-09-30T22:41:08.603 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.185093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176214.log 2023-09-30T22:41:08.603 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.185093.log.gz 2023-09-30T22:41:08.603 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.155367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158655.log 2023-09-30T22:41:08.604 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.176214.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.155367.log.gz 2023-09-30T22:41:08.604 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158067.log 2023-09-30T22:41:08.604 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.176214.log.gz 2023-09-30T22:41:08.604 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.158655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168855.log 2023-09-30T22:41:08.604 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.158655.log.gz 2023-09-30T22:41:08.604 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.158067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174634.log 2023-09-30T22:41:08.604 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.168855.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.158067.log.gz 2023-09-30T22:41:08.604 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172489.log 2023-09-30T22:41:08.605 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.168855.log.gz 2023-09-30T22:41:08.605 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.174634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176899.log 2023-09-30T22:41:08.605 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.174634.log.gz 2023-09-30T22:41:08.605 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.172489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138297.log 2023-09-30T22:41:08.605 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.176899.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.172489.log.gz 2023-09-30T22:41:08.605 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166752.log 2023-09-30T22:41:08.605 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.176899.log.gz 2023-09-30T22:41:08.605 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.138297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166699.log 2023-09-30T22:41:08.606 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.138297.log.gz 2023-09-30T22:41:08.606 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.166752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175770.log 2023-09-30T22:41:08.606 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.166699.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.166752.log.gz 2023-09-30T22:41:08.606 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138982.log 2023-09-30T22:41:08.606 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.166699.log.gz 2023-09-30T22:41:08.606 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.175770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151983.log 2023-09-30T22:41:08.606 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.175770.log.gz 2023-09-30T22:41:08.606 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.138982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175966.log 2023-09-30T22:41:08.606 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.138982.log.gz 2023-09-30T22:41:08.607 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.151983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179496.log 2023-09-30T22:41:08.607 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.175966.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.151983.log.gz 2023-09-30T22:41:08.607 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181317.log 2023-09-30T22:41:08.607 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.175966.log.gz 2023-09-30T22:41:08.607 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.179496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142780.log 2023-09-30T22:41:08.607 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.179496.log.gz 2023-09-30T22:41:08.607 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.181317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155119.log 2023-09-30T22:41:08.608 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.142780.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.181317.log.gz 2023-09-30T22:41:08.608 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170512.log 2023-09-30T22:41:08.608 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.142780.log.gz 2023-09-30T22:41:08.608 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.155119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171063.log 2023-09-30T22:41:08.608 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.155119.log.gz 2023-09-30T22:41:08.608 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.170512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150441.log 2023-09-30T22:41:08.608 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.171063.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.170512.log.gz 2023-09-30T22:41:08.608 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188036.log 2023-09-30T22:41:08.609 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.171063.log.gz 2023-09-30T22:41:08.609 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.150441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139677.log 2023-09-30T22:41:08.609 INFO:teuthology.orchestra.run.smithi016.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.150441.log.gz 2023-09-30T22:41:08.609 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.188036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190199.log 2023-09-30T22:41:08.609 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.139677.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.188036.log.gz 2023-09-30T22:41:08.609 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193114.log 2023-09-30T22:41:08.609 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.139677.log.gz 2023-09-30T22:41:08.609 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.190199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138394.log 2023-09-30T22:41:08.610 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.190199.log.gz 2023-09-30T22:41:08.610 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.193114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133958.log 2023-09-30T22:41:08.610 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.193114.log.gz 2023-09-30T22:41:08.610 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.138394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153867.log 2023-09-30T22:41:08.610 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.138394.log.gz 2023-09-30T22:41:08.610 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.133958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182885.log 2023-09-30T22:41:08.610 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.133958.log.gz 2023-09-30T22:41:08.611 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.153867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192108.log 2023-09-30T22:41:08.611 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.153867.log.gz 2023-09-30T22:41:08.611 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.182885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156525.log 2023-09-30T22:41:08.611 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.182885.log.gz 2023-09-30T22:41:08.611 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.192108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140166.log 2023-09-30T22:41:08.611 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.156525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153258.log 2023-09-30T22:41:08.611 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.192108.log.gz 2023-09-30T22:41:08.611 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.156525.log.gz 2023-09-30T22:41:08.612 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.140166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127810.log 2023-09-30T22:41:08.612 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.140166.log.gz 2023-09-30T22:41:08.612 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.153258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174377.log 2023-09-30T22:41:08.612 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.153258.log.gz 2023-09-30T22:41:08.612 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.127810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192918.log 2023-09-30T22:41:08.612 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.127810.log.gz 2023-09-30T22:41:08.612 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.174377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138686.log 2023-09-30T22:41:08.613 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.174377.log.gz 2023-09-30T22:41:08.613 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.192918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156891.log 2023-09-30T22:41:08.613 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.192918.log.gz 2023-09-30T22:41:08.613 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.138686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165992.log 2023-09-30T22:41:08.613 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.138686.log.gz 2023-09-30T22:41:08.613 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.156891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170057.log 2023-09-30T22:41:08.613 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.165992.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.156891.log.gz 2023-09-30T22:41:08.613 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158903.log 2023-09-30T22:41:08.614 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.170057.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.165992.log.gz 2023-09-30T22:41:08.614 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187341.log 2023-09-30T22:41:08.614 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.170057.log.gz 2023-09-30T22:41:08.614 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.158903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147475.log 2023-09-30T22:41:08.614 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.158903.log.gz 2023-09-30T22:41:08.614 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.187341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176554.log 2023-09-30T22:41:08.614 INFO:teuthology.orchestra.run.smithi016.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.187341.log.gz 2023-09-30T22:41:08.615 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.147475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128927.log 2023-09-30T22:41:08.615 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.147475.log.gz 2023-09-30T22:41:08.615 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.176554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124584.log 2023-09-30T22:41:08.615 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.128927.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.176554.log.gz 2023-09-30T22:41:08.615 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189026.log 2023-09-30T22:41:08.615 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.124584.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.128927.log.gz 2023-09-30T22:41:08.616 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136335.log 2023-09-30T22:41:08.616 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.124584.log.gz 2023-09-30T22:41:08.616 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.189026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166189.log 2023-09-30T22:41:08.616 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.189026.log.gz 2023-09-30T22:41:08.616 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.136335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193310.log 2023-09-30T22:41:08.616 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.166189.log: gzip -5 89.1%--verbose -- -- replaced with /var/log/ceph/ceph-client.admin.136335.log.gz/var/log/ceph/ceph-client.admin.185484.log 2023-09-30T22:41:08.616 INFO:teuthology.orchestra.run.smithi016.stderr: 2023-09-30T22:41:08.616 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.193310.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.166189.log.gz 2023-09-30T22:41:08.617 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181856.log 2023-09-30T22:41:08.617 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.193310.log.gz 2023-09-30T22:41:08.617 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.185484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192991.log 2023-09-30T22:41:08.617 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.185484.log.gz 2023-09-30T22:41:08.617 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.181856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184309.log 2023-09-30T22:41:08.617 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.181856.log.gz 2023-09-30T22:41:08.617 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.192991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158127.log 2023-09-30T22:41:08.618 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.184309.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.192991.log.gz 2023-09-30T22:41:08.618 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190814.log 2023-09-30T22:41:08.618 INFO:teuthology.orchestra.run.smithi016.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.184309.log.gz 2023-09-30T22:41:08.618 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.158127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137440.log 2023-09-30T22:41:08.618 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.158127.log.gz 2023-09-30T22:41:08.618 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.190814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146040.log 2023-09-30T22:41:08.618 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.137440.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.190814.log.gz 2023-09-30T22:41:08.618 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169273.log 2023-09-30T22:41:08.619 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.137440.log.gz 2023-09-30T22:41:08.619 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.146040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161476.log 2023-09-30T22:41:08.619 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.146040.log.gz 2023-09-30T22:41:08.619 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.169273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155964.log 2023-09-30T22:41:08.619 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.169273.log.gz 2023-09-30T22:41:08.619 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.161476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151591.log 2023-09-30T22:41:08.619 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.155964.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.161476.log.gz 2023-09-30T22:41:08.619 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176507.log 2023-09-30T22:41:08.620 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.155964.log.gz 2023-09-30T22:41:08.620 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.151591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185237.log 2023-09-30T22:41:08.620 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.151591.log.gz 2023-09-30T22:41:08.620 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.176507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139286.log 2023-09-30T22:41:08.620 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.185237.log: gzip -5 --verbose 88.0%-- /var/log/ceph/ceph-client.admin.176604.log 2023-09-30T22:41:08.620 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.176507.log.gz 2023-09-30T22:41:08.620 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.185237.log.gz 2023-09-30T22:41:08.620 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.139286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173099.log 2023-09-30T22:41:08.621 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.176604.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.139286.log.gz 2023-09-30T22:41:08.621 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127217.log 2023-09-30T22:41:08.621 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.176604.log.gz 2023-09-30T22:41:08.621 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.173099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184113.log 2023-09-30T22:41:08.621 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.173099.log.gz 2023-09-30T22:41:08.621 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.127217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151686.log 2023-09-30T22:41:08.621 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4%/var/log/ceph/ceph-client.admin.184113.log: -- replaced with /var/log/ceph/ceph-client.admin.127217.log.gz 2023-09-30T22:41:08.621 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172068.log 2023-09-30T22:41:08.622 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.184113.log.gz 2023-09-30T22:41:08.622 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.151686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177683.log 2023-09-30T22:41:08.622 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.151686.log.gz 2023-09-30T22:41:08.622 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.172068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149116.log 2023-09-30T22:41:08.622 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.172068.log.gz 2023-09-30T22:41:08.622 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.177683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173074.log 2023-09-30T22:41:08.622 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.177683.log.gz 2023-09-30T22:41:08.623 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.149116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136331.log 2023-09-30T22:41:08.623 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.173074.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.149116.log.gz 2023-09-30T22:41:08.623 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188867.log 2023-09-30T22:41:08.623 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.173074.log.gz 2023-09-30T22:41:08.623 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.136331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151619.log 2023-09-30T22:41:08.623 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.136331.log.gz 2023-09-30T22:41:08.623 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.188867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183722.log 2023-09-30T22:41:08.623 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.151619.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.188867.log.gz 2023-09-30T22:41:08.624 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137688.log 2023-09-30T22:41:08.624 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.151619.log.gz 2023-09-30T22:41:08.624 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.183722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164935.log 2023-09-30T22:41:08.624 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9%/var/log/ceph/ceph-client.admin.137688.log: -- replaced with /var/log/ceph/ceph-client.admin.183722.log.gz 2023-09-30T22:41:08.624 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139034.log 2023-09-30T22:41:08.624 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.164935.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.137688.log.gz 2023-09-30T22:41:08.624 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189612.log 2023-09-30T22:41:08.624 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.164935.log.gz 2023-09-30T22:41:08.625 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.139034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136852.log 2023-09-30T22:41:08.625 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.139034.log.gz 2023-09-30T22:41:08.625 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.189612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171676.log 2023-09-30T22:41:08.625 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.136852.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.189612.log.gz 2023-09-30T22:41:08.625 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165327.log 2023-09-30T22:41:08.625 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.136852.log.gz 2023-09-30T22:41:08.625 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.171676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160694.log 2023-09-30T22:41:08.626 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.171676.log.gz 2023-09-30T22:41:08.626 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.165327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124829.log 2023-09-30T22:41:08.626 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.165327.log.gz 2023-09-30T22:41:08.626 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.160694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141368.log 2023-09-30T22:41:08.626 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.160694.log.gz 2023-09-30T22:41:08.626 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.124829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193167.log 2023-09-30T22:41:08.626 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.124829.log.gz 2023-09-30T22:41:08.626 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.141368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147867.log 2023-09-30T22:41:08.627 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.141368.log.gz 2023-09-30T22:41:08.627 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.193167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177561.log 2023-09-30T22:41:08.627 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.147867.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.193167.log.gz 2023-09-30T22:41:08.627 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174574.log 2023-09-30T22:41:08.627 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.147867.log.gz 2023-09-30T22:41:08.627 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.177561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141760.log 2023-09-30T22:41:08.627 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.177561.log.gz 2023-09-30T22:41:08.628 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.174574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187592.log 2023-09-30T22:41:08.628 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.141760.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.174574.log.gz 2023-09-30T22:41:08.628 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143794.log 2023-09-30T22:41:08.628 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.141760.log.gz 2023-09-30T22:41:08.628 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.187592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149631.log 2023-09-30T22:41:08.628 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.187592.log.gz 2023-09-30T22:41:08.628 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.143794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167778.log 2023-09-30T22:41:08.628 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.149631.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.143794.log.gz 2023-09-30T22:41:08.629 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144527.log 2023-09-30T22:41:08.629 INFO:teuthology.orchestra.run.smithi016.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.149631.log.gz 2023-09-30T22:41:08.629 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.167778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185972.log 2023-09-30T22:41:08.629 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.167778.log.gz 2023-09-30T22:41:08.629 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.144527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143205.log 2023-09-30T22:41:08.629 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.185972.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.144527.log.gz 2023-09-30T22:41:08.629 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174842.log 2023-09-30T22:41:08.629 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.185972.log.gz 2023-09-30T22:41:08.630 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.143205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171110.log 2023-09-30T22:41:08.630 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.143205.log.gz 2023-09-30T22:41:08.630 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.174842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147177.log 2023-09-30T22:41:08.630 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.171110.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.174842.log.gz 2023-09-30T22:41:08.630 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156402.log 2023-09-30T22:41:08.630 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.171110.log.gz 2023-09-30T22:41:08.630 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.147177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149534.log 2023-09-30T22:41:08.630 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.147177.log.gz 2023-09-30T22:41:08.631 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.156402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164987.log 2023-09-30T22:41:08.631 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.149534.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.156402.log.gz 2023-09-30T22:41:08.631 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150611.log 2023-09-30T22:41:08.631 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.149534.log.gz 2023-09-30T22:41:08.631 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.164987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177588.log 2023-09-30T22:41:08.631 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.164987.log.gz 2023-09-30T22:41:08.631 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.150611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162826.log 2023-09-30T22:41:08.631 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.177588.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.150611.log.gz 2023-09-30T22:41:08.632 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168071.log 2023-09-30T22:41:08.632 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.177588.log.gz 2023-09-30T22:41:08.632 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.162826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148651.log 2023-09-30T22:41:08.632 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.162826.log.gz 2023-09-30T22:41:08.632 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.168071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141219.log 2023-09-30T22:41:08.632 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.148651.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.168071.log.gz 2023-09-30T22:41:08.632 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165745.log 2023-09-30T22:41:08.632 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.148651.log.gz 2023-09-30T22:41:08.633 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.141219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134179.log 2023-09-30T22:41:08.633 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.141219.log.gz 2023-09-30T22:41:08.633 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.165745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176999.log 2023-09-30T22:41:08.633 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.134179.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.165745.log.gz 2023-09-30T22:41:08.633 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156599.log 2023-09-30T22:41:08.633 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.134179.log.gz 2023-09-30T22:41:08.633 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.176999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128576.log 2023-09-30T22:41:08.633 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.176999.log.gz 2023-09-30T22:41:08.634 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.156599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163385.log 2023-09-30T22:41:08.634 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.128576.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.156599.log.gz 2023-09-30T22:41:08.634 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171599.log 2023-09-30T22:41:08.634 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.128576.log.gz 2023-09-30T22:41:08.634 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.163385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129123.log 2023-09-30T22:41:08.634 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.163385.log.gz 2023-09-30T22:41:08.634 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152082.log 2023-09-30T22:41:08.634 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.171599.log: /var/log/ceph/ceph-client.admin.129123.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.171599.log.gz 2023-09-30T22:41:08.635 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157479.log 2023-09-30T22:41:08.635 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.129123.log.gz 2023-09-30T22:41:08.635 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.152082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125537.log 2023-09-30T22:41:08.635 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.152082.log.gz 2023-09-30T22:41:08.635 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.157479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138616.log 2023-09-30T22:41:08.635 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.157479.log.gz 2023-09-30T22:41:08.635 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.125537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142518.log 2023-09-30T22:41:08.635 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.125537.log.gz 2023-09-30T22:41:08.636 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.138616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180754.log 2023-09-30T22:41:08.636 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.138616.log.gz 2023-09-30T22:41:08.636 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.142518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169835.log 2023-09-30T22:41:08.636 INFO:teuthology.orchestra.run.smithi016.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.142518.log.gz 2023-09-30T22:41:08.636 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.180754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189834.log 2023-09-30T22:41:08.636 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.169835.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.180754.log.gz 2023-09-30T22:41:08.636 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173005.log 2023-09-30T22:41:08.636 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.169835.log.gz 2023-09-30T22:41:08.637 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.189834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176412.log 2023-09-30T22:41:08.637 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.189834.log.gz 2023-09-30T22:41:08.637 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.173005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135725.log 2023-09-30T22:41:08.637 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.176412.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.173005.log.gz 2023-09-30T22:41:08.637 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135726.log 2023-09-30T22:41:08.637 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.176412.log.gz 2023-09-30T22:41:08.637 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.135725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135895.log 2023-09-30T22:41:08.637 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.135725.log.gz 2023-09-30T22:41:08.638 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.135726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189807.log 2023-09-30T22:41:08.638 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.135895.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.135726.log.gz 2023-09-30T22:41:08.638 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166895.log 2023-09-30T22:41:08.638 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.135895.log.gz 2023-09-30T22:41:08.638 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.189807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142028.log 2023-09-30T22:41:08.638 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.189807.log.gz 2023-09-30T22:41:08.638 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.166895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123359.log 2023-09-30T22:41:08.638 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.142028.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.166895.log.gz 2023-09-30T22:41:08.639 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148259.log 2023-09-30T22:41:08.639 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.142028.log.gz 2023-09-30T22:41:08.639 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.123359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161007.log 2023-09-30T22:41:08.639 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.123359.log.gz 2023-09-30T22:41:08.639 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.148259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157578.log 2023-09-30T22:41:08.639 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.161007.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.148259.log.gz 2023-09-30T22:41:08.639 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169494.log 2023-09-30T22:41:08.639 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.161007.log.gz 2023-09-30T22:41:08.640 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.157578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147966.log 2023-09-30T22:41:08.640 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.157578.log.gz 2023-09-30T22:41:08.640 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.169494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129172.log 2023-09-30T22:41:08.640 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.147966.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.169494.log.gz 2023-09-30T22:41:08.640 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-09-30T22:41:08.640 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.147966.log.gz 2023-09-30T22:41:08.640 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.129172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134131.log 2023-09-30T22:41:08.640 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.129172.log.gz 2023-09-30T22:41:08.641 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151885.log 2023-09-30T22:41:08.641 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.134131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139081.log 2023-09-30T22:41:08.641 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.134131.log.gz 2023-09-30T22:41:08.641 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.151885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128772.log 2023-09-30T22:41:08.641 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.151885.log.gz 2023-09-30T22:41:08.641 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.139081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175208.log 2023-09-30T22:41:08.641 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.139081.log.gz 2023-09-30T22:41:08.641 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.128772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175833.log 2023-09-30T22:41:08.642 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.128772.log.gz 2023-09-30T22:41:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.175208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128723.log 2023-09-30T22:41:08.642 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.175208.log.gz 2023-09-30T22:41:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.175833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125025.log 2023-09-30T22:41:08.642 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.175833.log.gz 2023-09-30T22:41:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168489.log 2023-09-30T22:41:08.642 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.128723.log: /var/log/ceph/ceph-client.admin.125025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137319.log 2023-09-30T22:41:08.642 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.128723.log.gz 2023-09-30T22:41:08.643 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.125025.log.gz 2023-09-30T22:41:08.643 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.168489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187145.log 2023-09-30T22:41:08.643 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165013.log 2023-09-30T22:41:08.643 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.137319.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.168489.log.gz 2023-09-30T22:41:08.643 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.187145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140362.log 2023-09-30T22:41:08.643 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.137319.log.gz 2023-09-30T22:41:08.643 INFO:teuthology.orchestra.run.smithi016.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.187145.log.gz 2023-09-30T22:41:08.643 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.165013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189250.log 2023-09-30T22:41:08.643 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.140362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186072.log 2023-09-30T22:41:08.644 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.165013.log.gz 2023-09-30T22:41:08.644 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.140362.log.gz 2023-09-30T22:41:08.644 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.189250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190984.log 2023-09-30T22:41:08.644 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185288.log /var/log/ceph/ceph-client.admin.186072.log: 2023-09-30T22:41:08.644 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.189250.log.gz 2023-09-30T22:41:08.644 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.190984.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.186072.log.gz 2023-09-30T22:41:08.644 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165622.log 2023-09-30T22:41:08.645 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.190984.log.gz 2023-09-30T22:41:08.645 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193088.log 2023-09-30T22:41:08.645 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.185288.log: /var/log/ceph/ceph-client.admin.165622.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.185288.log.gz 2023-09-30T22:41:08.645 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149730.log 2023-09-30T22:41:08.645 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174816.log 2023-09-30T22:41:08.645 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.165622.log.gz 2023-09-30T22:41:08.645 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.193088.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.193088.log.gz 2023-09-30T22:41:08.645 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.149730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161987.log 2023-09-30T22:41:08.646 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136826.log 2023-09-30T22:41:08.646 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.149730.log.gz 2023-09-30T22:41:08.646 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.174816.log: /var/log/ceph/ceph-client.admin.161987.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.174816.log.gz 2023-09-30T22:41:08.646 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153598.log 2023-09-30T22:41:08.646 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.161987.log.gz 2023-09-30T22:41:08.646 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124192.log 2023-09-30T22:41:08.646 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.136826.log: /var/log/ceph/ceph-client.admin.153598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192696.log 2023-09-30T22:41:08.646 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.136826.log.gz 2023-09-30T22:41:08.647 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.124192.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.153598.log.gz 2023-09-30T22:41:08.647 INFO:teuthology.orchestra.run.smithi016.stderr: 95.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-09-30T22:41:08.647 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.124192.log.gz 2023-09-30T22:41:08.647 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.192696.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.192696.log.gz 2023-09-30T22:41:08.647 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123877.log 2023-09-30T22:41:08.647 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161869.log 2023-09-30T22:41:08.745 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.123877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160223.log 2023-09-30T22:41:08.745 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.161869.log: 86.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.123877.log.gz-5 2023-09-30T22:41:08.745 INFO:teuthology.orchestra.run.smithi016.stderr:--verbose -- /var/log/ceph/ceph-client.admin.169325.log 2023-09-30T22:41:08.745 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.161869.log.gz 2023-09-30T22:41:08.745 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.160223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151225.log 2023-09-30T22:41:08.745 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.169325.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.160223.log.gz 2023-09-30T22:41:08.745 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146683.log 2023-09-30T22:41:08.746 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.169325.log.gz 2023-09-30T22:41:08.746 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.151225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147127.log 2023-09-30T22:41:08.746 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.151225.log.gz 2023-09-30T22:41:08.746 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.146683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146095.log 2023-09-30T22:41:08.746 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.147127.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.146683.log.gz 2023-09-30T22:41:08.746 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162405.log 2023-09-30T22:41:08.746 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.147127.log.gz 2023-09-30T22:41:08.747 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.146095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134941.log 2023-09-30T22:41:08.747 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.146095.log.gz 2023-09-30T22:41:08.747 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.162405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152571.log 2023-09-30T22:41:08.747 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.162405.log.gz 2023-09-30T22:41:08.747 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.134941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171403.log 2023-09-30T22:41:08.747 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.134941.log.gz 2023-09-30T22:41:08.747 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.152571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186660.log 2023-09-30T22:41:08.747 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.152571.log.gz 2023-09-30T22:41:08.748 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.171403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129221.log 2023-09-30T22:41:08.748 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.171403.log.gz 2023-09-30T22:41:08.748 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.186660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157944.log 2023-09-30T22:41:08.748 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.186660.log.gz 2023-09-30T22:41:08.748 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.129221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157309.log 2023-09-30T22:41:08.748 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.157944.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.129221.log.gz 2023-09-30T22:41:08.748 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142225.log 2023-09-30T22:41:08.748 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.157944.log.gz 2023-09-30T22:41:08.749 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.157309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125439.log 2023-09-30T22:41:08.749 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.157309.log.gz 2023-09-30T22:41:08.749 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.142225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174646.log 2023-09-30T22:41:08.749 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.142225.log.gz 2023-09-30T22:41:08.749 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.125439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156011.log 2023-09-30T22:41:08.749 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.174646.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.125439.log.gz 2023-09-30T22:41:08.749 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187396.log 2023-09-30T22:41:08.749 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.174646.log.gz 2023-09-30T22:41:08.750 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.156011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189071.log 2023-09-30T22:41:08.750 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.156011.log.gz 2023-09-30T22:41:08.750 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.187396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146931.log 2023-09-30T22:41:08.750 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.187396.log.gz 2023-09-30T22:41:08.750 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.189071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146735.log 2023-09-30T22:41:08.750 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.189071.log.gz 2023-09-30T22:41:08.750 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.146931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140976.log 2023-09-30T22:41:08.750 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.146931.log.gz 2023-09-30T22:41:08.751 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.146735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.194181.log 2023-09-30T22:41:08.751 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.140976.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.146735.log.gz 2023-09-30T22:41:08.751 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151395.log 2023-09-30T22:41:08.751 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.140976.log.gz 2023-09-30T22:41:08.751 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.194181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159491.log 2023-09-30T22:41:08.751 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.151395.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.194181.log.gz 2023-09-30T22:41:08.751 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122708.log 2023-09-30T22:41:08.751 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.151395.log.gz 2023-09-30T22:41:08.752 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.159491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190396.log 2023-09-30T22:41:08.752 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.159491.log.gz 2023-09-30T22:41:08.752 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.122708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154213.log 2023-09-30T22:41:08.752 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.122708.log.gz 2023-09-30T22:41:08.752 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.190396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168758.log 2023-09-30T22:41:08.752 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.154213.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.190396.log.gz 2023-09-30T22:41:08.752 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150700.log 2023-09-30T22:41:08.752 INFO:teuthology.orchestra.run.smithi016.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.154213.log.gz 2023-09-30T22:41:08.753 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.168758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140049.log 2023-09-30T22:41:08.753 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.150700.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.168758.log.gz 2023-09-30T22:41:08.753 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166602.log 2023-09-30T22:41:08.753 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.140049.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.150700.log.gz 2023-09-30T22:41:08.753 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177879.log 2023-09-30T22:41:08.753 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.140049.log.gz 2023-09-30T22:41:08.753 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.166602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167091.log 2023-09-30T22:41:08.753 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.177879.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.166602.log.gz 2023-09-30T22:41:08.754 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139854.log 2023-09-30T22:41:08.754 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.177879.log.gz 2023-09-30T22:41:08.754 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.167091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172682.log 2023-09-30T22:41:08.754 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.167091.log.gz 2023-09-30T22:41:08.754 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.139854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167901.log 2023-09-30T22:41:08.754 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.139854.log.gz 2023-09-30T22:41:08.754 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.172682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181072.log 2023-09-30T22:41:08.754 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.172682.log.gz 2023-09-30T22:41:08.755 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.167901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127271.log 2023-09-30T22:41:08.755 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.167901.log.gz 2023-09-30T22:41:08.755 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.181072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167875.log 2023-09-30T22:41:08.755 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.181072.log.gz 2023-09-30T22:41:08.755 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.127271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127504.log 2023-09-30T22:41:08.755 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.167875.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.127271.log.gz 2023-09-30T22:41:08.755 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134205.log 2023-09-30T22:41:08.756 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.167875.log.gz 2023-09-30T22:41:08.756 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.127504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124731.log 2023-09-30T22:41:08.756 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.127504.log.gz 2023-09-30T22:41:08.756 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.134205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176411.log 2023-09-30T22:41:08.756 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.134205.log.gz/var/log/ceph/ceph-client.admin.124731.log: 2023-09-30T22:41:08.756 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142395.log 2023-09-30T22:41:08.756 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.124731.log.gz 2023-09-30T22:41:08.756 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.176411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183424.log 2023-09-30T22:41:08.757 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.176411.log.gz 2023-09-30T22:41:08.757 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.142395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123751.log 2023-09-30T22:41:08.757 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.183424.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.142395.log.gz 2023-09-30T22:41:08.757 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183917.log 2023-09-30T22:41:08.757 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.183424.log.gz 2023-09-30T22:41:08.757 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.123751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141611.log 2023-09-30T22:41:08.757 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.123751.log.gz 2023-09-30T22:41:08.757 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.183917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190957.log 2023-09-30T22:41:08.758 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.141611.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.183917.log.gz 2023-09-30T22:41:08.758 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122955.log 2023-09-30T22:41:08.758 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.141611.log.gz 2023-09-30T22:41:08.758 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.190957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136040.log 2023-09-30T22:41:08.758 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.190957.log.gz 2023-09-30T22:41:08.758 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.122955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162235.log 2023-09-30T22:41:08.758 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.122955.log.gz 2023-09-30T22:41:08.758 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.136040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183620.log 2023-09-30T22:41:08.759 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.162235.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.136040.log.gz 2023-09-30T22:41:08.759 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148151.log 2023-09-30T22:41:08.759 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.162235.log.gz 2023-09-30T22:41:08.759 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.183620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162261.log 2023-09-30T22:41:08.759 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.183620.log.gz 2023-09-30T22:41:08.759 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.148151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165549.log 2023-09-30T22:41:08.759 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.148151.log.gz 2023-09-30T22:41:08.759 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.162261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191520.log 2023-09-30T22:41:08.760 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.162261.log.gz 2023-09-30T22:41:08.760 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.165549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125292.log 2023-09-30T22:41:08.760 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.165549.log.gz 2023-09-30T22:41:08.760 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.191520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171233.log 2023-09-30T22:41:08.760 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.125292.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.191520.log.gz 2023-09-30T22:41:08.760 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180755.log 2023-09-30T22:41:08.760 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.125292.log.gz 2023-09-30T22:41:08.760 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.171233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161254.log 2023-09-30T22:41:08.761 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.180755.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.171233.log.gz 2023-09-30T22:41:08.761 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189560.log 2023-09-30T22:41:08.761 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.180755.log.gz 2023-09-30T22:41:08.761 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.161254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124927.log 2023-09-30T22:41:08.761 INFO:teuthology.orchestra.run.smithi016.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.161254.log.gz 2023-09-30T22:41:08.761 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.189560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180876.log 2023-09-30T22:41:08.761 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.189560.log.gz 2023-09-30T22:41:08.761 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.124927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186269.log 2023-09-30T22:41:08.762 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.124927.log.gz 2023-09-30T22:41:08.762 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.180876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192526.log 2023-09-30T22:41:08.762 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.180876.log.gz 2023-09-30T22:41:08.762 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.186269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185067.log 2023-09-30T22:41:08.762 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.192526.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.186269.log.gz 2023-09-30T22:41:08.762 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190592.log 2023-09-30T22:41:08.762 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.192526.log.gz 2023-09-30T22:41:08.762 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.185067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172587.log 2023-09-30T22:41:08.763 INFO:teuthology.orchestra.run.smithi016.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.185067.log.gz 2023-09-30T22:41:08.763 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.190592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142590.log 2023-09-30T22:41:08.763 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.190592.log.gz 2023-09-30T22:41:08.763 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.172587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144357.log 2023-09-30T22:41:08.763 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.172587.log.gz 2023-09-30T22:41:08.763 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.142590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150514.log 2023-09-30T22:41:08.763 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.142590.log.gz 2023-09-30T22:41:08.763 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.144357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123555.log 2023-09-30T22:41:08.764 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.150514.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.144357.log.gz 2023-09-30T22:41:08.764 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164986.log 2023-09-30T22:41:08.764 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.150514.log.gz 2023-09-30T22:41:08.764 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.123555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171011.log 2023-09-30T22:41:08.764 INFO:teuthology.orchestra.run.smithi016.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.123555.log.gz 2023-09-30T22:41:08.764 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.164986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128976.log 2023-09-30T22:41:08.764 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.164986.log.gz 2023-09-30T22:41:08.764 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.171011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139578.log 2023-09-30T22:41:08.764 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.171011.log.gz 2023-09-30T22:41:08.765 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.128976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134690.log 2023-09-30T22:41:08.765 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.128976.log.gz 2023-09-30T22:41:08.765 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.139578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158485.log 2023-09-30T22:41:08.765 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.139578.log.gz 2023-09-30T22:41:08.765 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.134690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136434.log 2023-09-30T22:41:08.765 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.134690.log.gz 2023-09-30T22:41:08.765 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.158485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172221.log 2023-09-30T22:41:08.765 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.158485.log.gz 2023-09-30T22:41:08.766 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.136434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118822.log 2023-09-30T22:41:08.766 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.136434.log.gz 2023-09-30T22:41:08.766 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.172221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165379.log 2023-09-30T22:41:08.766 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.172221.log.gz 2023-09-30T22:41:08.766 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.118822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129579.log 2023-09-30T22:41:08.766 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.118822.log.gz 2023-09-30T22:41:08.766 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.165379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143694.log 2023-09-30T22:41:08.766 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.165379.log.gz 2023-09-30T22:41:08.767 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.129579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123875.log 2023-09-30T22:41:08.767 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.129579.log.gz 2023-09-30T22:41:08.767 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.143694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170423.log 2023-09-30T22:41:08.767 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.143694.log.gz 2023-09-30T22:41:08.767 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.123875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147770.log 2023-09-30T22:41:08.767 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.170423.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.123875.log.gz 2023-09-30T22:41:08.767 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191324.log 2023-09-30T22:41:08.767 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.170423.log.gz 2023-09-30T22:41:08.768 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.147770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145755.log 2023-09-30T22:41:08.768 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.191324.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.147770.log.gz 2023-09-30T22:41:08.768 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-09-30T22:41:08.768 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.191324.log.gz 2023-09-30T22:41:08.768 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.145755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186805.log 2023-09-30T22:41:08.768 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.145755.log.gz 2023-09-30T22:41:08.768 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148947.log 2023-09-30T22:41:08.768 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.186805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170718.log 2023-09-30T22:41:08.769 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.186805.log.gz 2023-09-30T22:41:08.769 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.148947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125123.log 2023-09-30T22:41:08.769 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.148947.log.gz 2023-09-30T22:41:08.769 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.170718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151495.log 2023-09-30T22:41:08.769 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.170718.log.gz 2023-09-30T22:41:08.769 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.125123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153062.log 2023-09-30T22:41:08.769 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.125123.log.gz 2023-09-30T22:41:08.769 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.151495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181411.log 2023-09-30T22:41:08.770 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.151495.log.gz 2023-09-30T22:41:08.770 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.153062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153624.log 2023-09-30T22:41:08.770 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.153062.log.gz 2023-09-30T22:41:08.770 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.181411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179326.log 2023-09-30T22:41:08.770 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.181411.log.gz 2023-09-30T22:41:08.770 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.153624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129270.log 2023-09-30T22:41:08.770 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.179326.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.153624.log.gz 2023-09-30T22:41:08.770 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167313.log 2023-09-30T22:41:08.771 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.179326.log.gz 2023-09-30T22:41:08.771 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176039.log 2023-09-30T22:41:08.771 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.129270.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.129270.log.gz 2023-09-30T22:41:08.771 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.167313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179130.log 2023-09-30T22:41:08.771 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.167313.log.gz 2023-09-30T22:41:08.771 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.176039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153454.log 2023-09-30T22:41:08.771 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.176039.log.gz 2023-09-30T22:41:08.771 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.179130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142029.log 2023-09-30T22:41:08.772 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184208.log 2023-09-30T22:41:08.772 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0%/var/log/ceph/ceph-client.admin.153454.log: -- replaced with /var/log/ceph/ceph-client.admin.179130.log.gz 2023-09-30T22:41:08.772 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.153454.log.gz 2023-09-30T22:41:08.772 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.142029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158513.log 2023-09-30T22:41:08.772 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142787.log 2023-09-30T22:41:08.772 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.142029.log.gz 2023-09-30T22:41:08.772 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.184208.log: /var/log/ceph/ceph-client.admin.158513.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.184208.log.gz 2023-09-30T22:41:08.772 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161673.log 2023-09-30T22:41:08.773 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174228.log 2023-09-30T22:41:08.773 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.142787.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.158513.log.gz 2023-09-30T22:41:08.773 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.161673.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.142787.log.gz 2023-09-30T22:41:08.773 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186946.log 2023-09-30T22:41:08.773 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.161673.log.gz 2023-09-30T22:41:08.773 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.174228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135308.log 2023-09-30T22:41:08.773 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.186946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167732.log 2023-09-30T22:41:08.773 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.174228.log.gz 2023-09-30T22:41:08.774 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.186946.log.gz 2023-09-30T22:41:08.774 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.135308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192795.log 2023-09-30T22:41:08.774 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141049.log 2023-09-30T22:41:08.774 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1%/var/log/ceph/ceph-client.admin.167732.log: -- replaced with /var/log/ceph/ceph-client.admin.135308.log.gz 2023-09-30T22:41:08.774 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.192795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124486.log 2023-09-30T22:41:08.774 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.167732.log.gz 2023-09-30T22:41:08.774 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.192795.log.gz 2023-09-30T22:41:08.774 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.141049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143795.log 2023-09-30T22:41:08.775 INFO:teuthology.orchestra.run.smithi016.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.141049.log.gz 2023-09-30T22:41:08.775 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141249.log 2023-09-30T22:41:08.775 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.124486.log: /var/log/ceph/ceph-client.admin.143795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144945.log 2023-09-30T22:41:08.775 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.124486.log.gz 2023-09-30T22:41:08.775 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.143795.log.gz 2023-09-30T22:41:08.775 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.141249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147101.log 2023-09-30T22:41:08.775 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose 87.8%-- /var/log/ceph/ceph-osd.0.log 2023-09-30T22:41:08.775 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ceph-client.admin.141249.log.gz 2023-09-30T22:41:08.776 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.144945.log: /var/log/ceph/ceph-client.admin.147101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127072.log 2023-09-30T22:41:08.776 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.144945.log.gz 2023-09-30T22:41:08.776 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-osd.0.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.147101.log.gz 2023-09-30T22:41:08.776 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158755.log 2023-09-30T22:41:08.776 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.127072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154063.log 2023-09-30T22:41:08.776 INFO:teuthology.orchestra.run.smithi016.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.127072.log.gz 2023-09-30T22:41:08.776 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.158755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134035.log 2023-09-30T22:41:08.776 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.154063.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.158755.log.gz 2023-09-30T22:41:08.777 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.154063.log.gz 2023-09-30T22:41:08.777 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.134035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180925.log 2023-09-30T22:41:08.777 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.134035.log.gz 2023-09-30T22:41:08.777 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171821.log 2023-09-30T22:41:08.777 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179363.log 2023-09-30T22:41:08.777 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.180925.log: /var/log/ceph/ceph-client.admin.171821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193593.log 2023-09-30T22:41:08.777 INFO:teuthology.orchestra.run.smithi016.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.180925.log.gz 2023-09-30T22:41:08.777 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.171821.log.gz 2023-09-30T22:41:08.777 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.179363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122418.log 2023-09-30T22:41:08.778 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.179363.log.gz 2023-09-30T22:41:08.778 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.193593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150906.log 2023-09-30T22:41:08.778 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.122418.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.193593.log.gz 2023-09-30T22:41:08.778 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.122418.log.gz 2023-09-30T22:41:08.778 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136461.log 2023-09-30T22:41:08.778 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.150906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169717.log 2023-09-30T22:41:08.778 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.150906.log.gz 2023-09-30T22:41:08.779 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.136461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129902.log 2023-09-30T22:41:08.779 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.136461.log.gz 2023-09-30T22:41:08.779 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.169717.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.169717.log.gz 2023-09-30T22:41:08.779 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.129902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162065.log 2023-09-30T22:41:08.779 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.129902.log.gz 2023-09-30T22:41:08.779 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149914.log 2023-09-30T22:41:08.779 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.162065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183816.log 2023-09-30T22:41:08.779 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.162065.log.gz 2023-09-30T22:41:08.780 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.149914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182493.log 2023-09-30T22:41:08.780 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.149914.log.gz-5 2023-09-30T22:41:08.780 INFO:teuthology.orchestra.run.smithi016.stderr:--verbose -- /var/log/ceph/ceph-client.admin.155610.log 2023-09-30T22:41:08.780 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.183816.log: /var/log/ceph/ceph-client.admin.182493.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.183816.log.gz 2023-09-30T22:41:08.780 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151299.log 2023-09-30T22:41:08.780 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.155610.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.182493.log.gz 2023-09-30T22:41:08.780 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.155610.log.gz 2023-09-30T22:41:08.780 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146291.log 2023-09-30T22:41:08.781 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187537.log 2023-09-30T22:41:08.781 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.151299.log: /var/log/ceph/ceph-client.admin.146291.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.151299.log.gz 2023-09-30T22:41:08.781 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139774.log 2023-09-30T22:41:08.781 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.146291.log.gz 2023-09-30T22:41:08.781 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.187537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157382.log 2023-09-30T22:41:08.781 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.139774.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.187537.log.gz 2023-09-30T22:41:08.781 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178763.log 2023-09-30T22:41:08.781 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.139774.log.gz 2023-09-30T22:41:08.782 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.157382.log: 89.0%/var/log/ceph/ceph-client.admin.178763.log: -- replaced with /var/log/ceph/ceph-client.admin.157382.log.gz 2023-09-30T22:41:08.782 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.178763.log.gz 2023-09-30T22:41:08.782 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187618.log 2023-09-30T22:41:08.782 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163555.log 2023-09-30T22:41:08.782 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.187618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193284.log 2023-09-30T22:41:08.782 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.187618.log.gz 2023-09-30T22:41:08.782 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.163555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140048.log 2023-09-30T22:41:08.782 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.163555.log.gz 2023-09-30T22:41:08.783 INFO:teuthology.orchestra.run.smithi016.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.149070.log /var/log/ceph/ceph-client.admin.193284.log: 2023-09-30T22:41:08.783 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.140048.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.193284.log.gz 2023-09-30T22:41:08.783 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.149070.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.140048.log.gz 2023-09-30T22:41:08.783 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.149070.log.gz 2023-09-30T22:41:08.783 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170227.log 2023-09-30T22:41:08.783 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188429.log 2023-09-30T22:41:08.783 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.170227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165426.log 2023-09-30T22:41:08.783 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.170227.log.gz 2023-09-30T22:41:08.783 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.188429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179718.log 2023-09-30T22:41:08.784 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.188429.log.gz 2023-09-30T22:41:08.784 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.165426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180631.log 2023-09-30T22:41:08.784 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.165426.log.gz 2023-09-30T22:41:08.784 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.179718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168170.log 2023-09-30T22:41:08.784 INFO:teuthology.orchestra.run.smithi016.stderr:gzip 88.3%-5 -- replaced with /var/log/ceph/ceph-client.admin.179718.log.gz--verbose 2023-09-30T22:41:08.784 INFO:teuthology.orchestra.run.smithi016.stderr:-- /var/log/ceph/ceph-client.admin.169691.log 2023-09-30T22:41:08.784 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.180631.log: /var/log/ceph/ceph-client.admin.168170.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.180631.log.gz 2023-09-30T22:41:08.784 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.168170.log.gz 2023-09-30T22:41:08.785 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.169691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191546.log 2023-09-30T22:41:08.785 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.169691.log.gz 2023-09-30T22:41:08.785 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192304.log 2023-09-30T22:41:08.785 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.191546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147326.log 2023-09-30T22:41:08.785 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.191546.log.gz 2023-09-30T22:41:08.785 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.192304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139970.log 2023-09-30T22:41:08.785 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.147326.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.192304.log.gz 2023-09-30T22:41:08.785 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.147326.log.gz 2023-09-30T22:41:08.786 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.139970.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.139970.log.gz 2023-09-30T22:41:08.786 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157338.log 2023-09-30T22:41:08.786 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123653.log 2023-09-30T22:41:08.786 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.157338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181607.log 2023-09-30T22:41:08.786 INFO:teuthology.orchestra.run.smithi016.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.157338.log.gz 2023-09-30T22:41:08.786 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.123653.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.123653.log.gz 2023-09-30T22:41:08.786 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.181607.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.181607.log.gz 2023-09-30T22:41:08.786 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164624.log 2023-09-30T22:41:08.787 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124022.log 2023-09-30T22:41:08.787 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.164624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156917.log 2023-09-30T22:41:08.787 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.164624.log.gz 2023-09-30T22:41:08.787 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.124022.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.111939.log 2023-09-30T22:41:08.787 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.124022.log.gz 2023-09-30T22:41:08.787 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.156917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164569.log 2023-09-30T22:41:08.787 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.156917.log.gz 2023-09-30T22:41:08.787 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph.tmp-client.admin.111939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153042.log 2023-09-30T22:41:08.788 INFO:teuthology.orchestra.run.smithi016.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.111939.log.gz 2023-09-30T22:41:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.164569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191742.log 2023-09-30T22:41:08.788 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.164569.log.gz 2023-09-30T22:41:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.153042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187788.log 2023-09-30T22:41:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.191742.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.153042.log.gz 2023-09-30T22:41:08.788 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.191742.log.gz 2023-09-30T22:41:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.187788.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.187788.log.gz 2023-09-30T22:41:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145507.log 2023-09-30T22:41:08.789 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136236.log 2023-09-30T22:41:08.789 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.145507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161085.log 2023-09-30T22:41:08.789 INFO:teuthology.orchestra.run.smithi016.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.145507.log.gz 2023-09-30T22:41:08.789 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.136236.log: /var/log/ceph/ceph-client.admin.161085.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.136236.log.gz 2023-09-30T22:41:08.789 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145452.log 2023-09-30T22:41:08.789 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.161085.log.gz 2023-09-30T22:41:08.789 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124270.log 2023-09-30T22:41:08.789 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.145452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160497.log 2023-09-30T22:41:08.790 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.145452.log.gz 2023-09-30T22:41:08.790 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.124270.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.124270.log.gz 2023-09-30T22:41:08.790 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.160497.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.160497.log.gz 2023-09-30T22:41:08.790 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163654.log 2023-09-30T22:41:08.790 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172903.log 2023-09-30T22:41:08.790 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.163654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165993.log 2023-09-30T22:41:08.790 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.163654.log.gz 2023-09-30T22:41:08.790 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.172903.log: /var/log/ceph/ceph-client.admin.165993.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.172903.log.gz 2023-09-30T22:41:08.791 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.165993.log.gz 2023-09-30T22:41:08.791 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182127.log 2023-09-30T22:41:08.791 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155315.log 2023-09-30T22:41:08.791 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.182127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135674.log 2023-09-30T22:41:08.791 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.182127.log.gz/var/log/ceph/ceph-client.admin.155315.log: 2023-09-30T22:41:08.791 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.155315.log.gz 2023-09-30T22:41:08.791 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.135674.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.135674.log.gz 2023-09-30T22:41:08.792 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128878.log 2023-09-30T22:41:08.792 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182715.log 2023-09-30T22:41:08.792 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.128878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134423.log 2023-09-30T22:41:08.792 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.128878.log.gz 2023-09-30T22:41:08.792 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.182715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166529.log 2023-09-30T22:41:08.792 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.134423.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.182715.log.gz 2023-09-30T22:41:08.792 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181539.log 2023-09-30T22:41:08.792 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.134423.log.gz 2023-09-30T22:41:08.792 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.166529.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.166529.log.gz 2023-09-30T22:41:08.793 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.181539.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.181539.log.gz 2023-09-30T22:41:08.793 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136727.log 2023-09-30T22:41:08.793 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177784.log 2023-09-30T22:41:08.793 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.136727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160104.log 2023-09-30T22:41:08.793 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.136727.log.gz 2023-09-30T22:41:08.793 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.177784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153650.log 2023-09-30T22:41:08.793 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.177784.log.gz 2023-09-30T22:41:08.793 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.160104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125733.log 2023-09-30T22:41:08.794 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.160104.log.gz 2023-09-30T22:41:08.794 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.153650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140388.log 2023-09-30T22:41:08.794 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.153650.log.gz 2023-09-30T22:41:08.794 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.125733.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.125733.log.gz 2023-09-30T22:41:08.794 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.140388.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.140388.log.gz 2023-09-30T22:41:08.794 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186217.log 2023-09-30T22:41:08.794 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134745.log 2023-09-30T22:41:08.794 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.186217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193887.log 2023-09-30T22:41:08.795 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.134745.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.186217.log.gz 2023-09-30T22:41:08.795 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.134745.log.gz 2023-09-30T22:41:08.795 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.193887.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.193887.log.gz 2023-09-30T22:41:08.795 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155218.log 2023-09-30T22:41:08.795 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193383.log 2023-09-30T22:41:08.795 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.155218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161255.log 2023-09-30T22:41:08.795 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.155218.log.gz 2023-09-30T22:41:08.795 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.193383.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.193383.log.gz 2023-09-30T22:41:08.796 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.161255.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.161255.log.gz 2023-09-30T22:41:08.796 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134325.log 2023-09-30T22:41:08.796 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140461.log 2023-09-30T22:41:08.796 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.134325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159539.log 2023-09-30T22:41:08.796 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.134325.log.gz 2023-09-30T22:41:08.796 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.140461.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.140461.log.gz 2023-09-30T22:41:08.796 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.159539.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.159539.log.gz 2023-09-30T22:41:08.796 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159661.log 2023-09-30T22:41:08.797 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189194.log 2023-09-30T22:41:08.797 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.159661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173279.log 2023-09-30T22:41:08.797 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.159661.log.gz 2023-09-30T22:41:08.797 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.189194.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.189194.log.gz 2023-09-30T22:41:08.797 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.173279.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.173279.log.gz 2023-09-30T22:41:08.797 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184087.log 2023-09-30T22:41:08.797 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140611.log 2023-09-30T22:41:08.797 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.184087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169128.log 2023-09-30T22:41:08.798 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.184087.log.gz/var/log/ceph/ceph-client.admin.140611.log: 2023-09-30T22:41:08.798 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.140611.log.gz 2023-09-30T22:41:08.798 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.169128.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.169128.log.gz 2023-09-30T22:41:08.798 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192600.log 2023-09-30T22:41:08.798 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163984.log 2023-09-30T22:41:08.798 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.192600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142617.log 2023-09-30T22:41:08.798 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.192600.log.gz 2023-09-30T22:41:08.798 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.163984.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.163984.log.gz 2023-09-30T22:41:08.799 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.142617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149461.log 2023-09-30T22:41:08.799 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.142617.log.gz 2023-09-30T22:41:08.799 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145648.log 2023-09-30T22:41:08.799 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.149461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163004.log 2023-09-30T22:41:08.799 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.145648.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.149461.log.gz 2023-09-30T22:41:08.799 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.163004.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.145648.log.gz 2023-09-30T22:41:08.799 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.163004.log.gz 2023-09-30T22:41:08.799 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149435.log 2023-09-30T22:41:08.800 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176608.log 2023-09-30T22:41:08.800 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.149435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174181.log 2023-09-30T22:41:08.800 INFO:teuthology.orchestra.run.smithi016.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.149435.log.gz 2023-09-30T22:41:08.800 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.176608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173491.log 2023-09-30T22:41:08.800 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.174181.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.176608.log.gz 2023-09-30T22:41:08.800 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.174181.log.gz 2023-09-30T22:41:08.800 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.173491.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.173491.log.gz 2023-09-30T22:41:08.800 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158336.log 2023-09-30T22:41:08.801 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178959.log 2023-09-30T22:41:08.801 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.158336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172809.log 2023-09-30T22:41:08.801 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.158336.log.gz 2023-09-30T22:41:08.801 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.178959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154949.log 2023-09-30T22:41:08.801 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.172809.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.178959.log.gz 2023-09-30T22:41:08.801 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193642.log 2023-09-30T22:41:08.801 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.172809.log.gz 2023-09-30T22:41:08.801 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.154949.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.154949.log.gz 2023-09-30T22:41:08.802 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.193642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142813.log 2023-09-30T22:41:08.802 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169077.log 2023-09-30T22:41:08.802 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.193642.log.gz 2023-09-30T22:41:08.802 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.142813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125880.log 2023-09-30T22:41:08.802 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.142813.log.gz 2023-09-30T22:41:08.802 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.169077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158289.log 2023-09-30T22:41:08.802 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.169077.log.gz 2023-09-30T22:41:08.802 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144188.log 2023-09-30T22:41:08.803 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.125880.log: /var/log/ceph/ceph-client.admin.158289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152767.log 2023-09-30T22:41:08.803 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.144188.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.158289.log.gz 2023-09-30T22:41:08.803 INFO:teuthology.orchestra.run.smithi016.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.125880.log.gz 2023-09-30T22:41:08.803 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.144188.log.gz 2023-09-30T22:41:08.803 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.152767.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.152767.log.gz 2023-09-30T22:41:08.803 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170108.log 2023-09-30T22:41:08.803 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181019.log 2023-09-30T22:41:08.804 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.170108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185580.log 2023-09-30T22:41:08.804 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.181019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183371.log 2023-09-30T22:41:08.804 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.170108.log.gz 2023-09-30T22:41:08.804 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.181019.log.gz 2023-09-30T22:41:08.804 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.185580.log: /var/log/ceph/ceph-client.admin.183371.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.185580.log.gz 2023-09-30T22:41:08.804 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.183371.log.gz 2023-09-30T22:41:08.804 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164543.log 2023-09-30T22:41:08.804 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159100.log 2023-09-30T22:41:08.805 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.164543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185188.log 2023-09-30T22:41:08.805 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.164543.log.gz 2023-09-30T22:41:08.805 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.159100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138250.log 2023-09-30T22:41:08.805 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.185188.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.159100.log.gz 2023-09-30T22:41:08.805 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173295.log 2023-09-30T22:41:08.805 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.185188.log.gz 2023-09-30T22:41:08.805 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.138250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182640.log 2023-09-30T22:41:08.805 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.138250.log.gz 2023-09-30T22:41:08.806 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.173295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137048.log 2023-09-30T22:41:08.806 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.173295.log.gz 2023-09-30T22:41:08.806 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.182640.log: /var/log/ceph/ceph-client.admin.137048.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.182640.log.gz 2023-09-30T22:41:08.806 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.137048.log.gz 2023-09-30T22:41:08.806 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174986.log 2023-09-30T22:41:08.806 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135060.log 2023-09-30T22:41:08.806 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.174986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157748.log 2023-09-30T22:41:08.807 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.174986.log.gz 2023-09-30T22:41:08.807 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.135060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160301.log 2023-09-30T22:41:08.807 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182836.log 2023-09-30T22:41:08.807 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.135060.log.gz 2023-09-30T22:41:08.807 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.157748.log: /var/log/ceph/ceph-client.admin.160301.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.157748.log.gz 2023-09-30T22:41:08.807 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.160301.log.gz 2023-09-30T22:41:08.807 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.182836.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.182836.log.gz 2023-09-30T22:41:08.807 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175012.log 2023-09-30T22:41:08.808 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124269.log 2023-09-30T22:41:08.808 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.175012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123878.log 2023-09-30T22:41:08.808 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.175012.log.gz 2023-09-30T22:41:08.808 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.124269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185681.log 2023-09-30T22:41:08.808 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149827.log 2023-09-30T22:41:08.808 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.124269.log.gz 2023-09-30T22:41:08.808 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.123878.log: /var/log/ceph/ceph-client.admin.185681.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.123878.log.gz 2023-09-30T22:41:08.808 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162183.log 2023-09-30T22:41:08.809 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.185681.log.gz 2023-09-30T22:41:08.809 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.149827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182979.log 2023-09-30T22:41:08.809 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.162183.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.149827.log.gz 2023-09-30T22:41:08.809 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.182979.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.162183.log.gz 2023-09-30T22:41:08.809 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.182979.log.gz 2023-09-30T22:41:08.809 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179104.log 2023-09-30T22:41:08.809 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169912.log 2023-09-30T22:41:08.809 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.179104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170815.log 2023-09-30T22:41:08.810 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.179104.log.gz 2023-09-30T22:41:08.810 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.169912.log: /var/log/ceph/ceph-client.admin.170815.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.169912.log.gz 2023-09-30T22:41:08.810 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.170815.log.gz 2023-09-30T22:41:08.810 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182587.log 2023-09-30T22:41:08.810 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152258.log 2023-09-30T22:41:08.810 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.182587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140852.log 2023-09-30T22:41:08.810 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.182587.log.gz 2023-09-30T22:41:08.810 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.152258.log: /var/log/ceph/ceph-client.admin.140852.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.152258.log.gz 2023-09-30T22:41:08.810 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.140852.log.gz 2023-09-30T22:41:08.811 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152597.log 2023-09-30T22:41:08.811 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173836.log 2023-09-30T22:41:08.811 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.152597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152861.log 2023-09-30T22:41:08.811 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.152597.log.gz 2023-09-30T22:41:08.811 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.173836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173789.log 2023-09-30T22:41:08.811 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145533.log 2023-09-30T22:41:08.811 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.173836.log.gz 2023-09-30T22:41:08.811 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.152861.log: /var/log/ceph/ceph-client.admin.173789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122806.log 2023-09-30T22:41:08.812 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.152861.log.gz 2023-09-30T22:41:08.812 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.145533.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.173789.log.gz 2023-09-30T22:41:08.812 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.145533.log.gz 2023-09-30T22:41:08.812 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.122806.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.122806.log.gz 2023-09-30T22:41:08.812 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155341.log 2023-09-30T22:41:08.812 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185433.log 2023-09-30T22:41:08.812 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.155341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167679.log 2023-09-30T22:41:08.812 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.185433.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.155341.log.gz 2023-09-30T22:41:08.813 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127837.log 2023-09-30T22:41:08.813 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.185433.log.gz 2023-09-30T22:41:08.813 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180306.log 2023-09-30T22:41:08.813 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.167679.log: /var/log/ceph/ceph-client.admin.127837.log: 87.8%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.167679.log.gz--verbose 2023-09-30T22:41:08.813 INFO:teuthology.orchestra.run.smithi016.stderr:-- /var/log/ceph/ceph-client.admin.150895.log 2023-09-30T22:41:08.813 INFO:teuthology.orchestra.run.smithi016.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.127837.log.gz 2023-09-30T22:41:08.813 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.180306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179447.log 2023-09-30T22:41:08.813 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.150895.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.180306.log.gz 2023-09-30T22:41:08.814 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167927.log 2023-09-30T22:41:08.814 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.179447.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.150895.log.gz 2023-09-30T22:41:08.814 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118567.log 2023-09-30T22:41:08.814 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.179447.log.gz 2023-09-30T22:41:08.814 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.167927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189586.log 2023-09-30T22:41:08.814 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.167927.log.gz 2023-09-30T22:41:08.814 INFO:teuthology.orchestra.run.smithi016.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.127452.log 2023-09-30T22:41:08.814 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.118567.log: /var/log/ceph/ceph-client.admin.189586.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.118567.log.gz 2023-09-30T22:41:08.815 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.189586.log.gz 2023-09-30T22:41:08.815 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.127452.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.127452.log.gz 2023-09-30T22:41:08.815 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160445.log 2023-09-30T22:41:08.815 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185289.log 2023-09-30T22:41:08.815 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.160445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146487.log 2023-09-30T22:41:08.815 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.160445.log.gz 2023-09-30T22:41:08.815 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.185289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136533.log 2023-09-30T22:41:08.815 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143009.log 2023-09-30T22:41:08.816 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.146487.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.185289.log.gz 2023-09-30T22:41:08.816 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.136533.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.146487.log.gz 2023-09-30T22:41:08.816 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.136533.log.gz 2023-09-30T22:41:08.816 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.143009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176213.log 2023-09-30T22:41:08.816 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.143009.log.gz 2023-09-30T22:41:08.816 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124437.log 2023-09-30T22:41:08.816 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138101.log 2023-09-30T22:41:08.816 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.176213.log: /var/log/ceph/ceph-client.admin.124437.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.176213.log.gz 2023-09-30T22:41:08.817 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.124437.log.gz 2023-09-30T22:41:08.817 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.138101.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.138101.log.gz 2023-09-30T22:41:08.817 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137273.log 2023-09-30T22:41:08.817 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154569.log 2023-09-30T22:41:08.817 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.137273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123310.log 2023-09-30T22:41:08.817 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.137273.log.gz 2023-09-30T22:41:08.817 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.154569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167341.log 2023-09-30T22:41:08.817 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.123310.log: gzip 89.0%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.154569.log.gz-- 2023-09-30T22:41:08.818 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.144749.log 2023-09-30T22:41:08.818 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.167341.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.123310.log.gz 2023-09-30T22:41:08.818 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.167341.log.gz 2023-09-30T22:41:08.818 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.144749.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.144749.log.gz 2023-09-30T22:41:08.818 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189193.log 2023-09-30T22:41:08.818 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182783.log 2023-09-30T22:41:08.818 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.189193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191793.log 2023-09-30T22:41:08.818 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.189193.log.gz 2023-09-30T22:41:08.819 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.182783.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.182783.log.gz 2023-09-30T22:41:08.819 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.191793.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.191793.log.gz 2023-09-30T22:41:08.819 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167583.log 2023-09-30T22:41:08.819 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149853.log 2023-09-30T22:41:08.819 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.167583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152963.log 2023-09-30T22:41:08.819 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.167583.log.gz 2023-09-30T22:41:08.819 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.149853.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.149853.log.gz 2023-09-30T22:41:08.819 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.152963.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.152963.log.gz 2023-09-30T22:41:08.820 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155768.log 2023-09-30T22:41:08.820 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142199.log 2023-09-30T22:41:08.820 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.155768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137414.log 2023-09-30T22:41:08.820 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3%/var/log/ceph/ceph-client.admin.142199.log: -- replaced with /var/log/ceph/ceph-client.admin.155768.log.gz 2023-09-30T22:41:08.820 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.142199.log.gz 2023-09-30T22:41:08.820 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.137414.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.137414.log.gz 2023-09-30T22:41:08.820 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148555.log 2023-09-30T22:41:08.821 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169495.log 2023-09-30T22:41:08.821 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.148555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141172.log 2023-09-30T22:41:08.821 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.169495.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.148555.log.gz 2023-09-30T22:41:08.821 INFO:teuthology.orchestra.run.smithi016.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.169495.log.gz 2023-09-30T22:41:08.821 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.141172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193985.log 2023-09-30T22:41:08.821 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.141172.log.gz 2023-09-30T22:41:08.821 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156329.log 2023-09-30T22:41:08.821 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.193985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176311.log 2023-09-30T22:41:08.822 INFO:teuthology.orchestra.run.smithi016.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.193985.log.gz 2023-09-30T22:41:08.822 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.156329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171284.log 2023-09-30T22:41:08.822 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166111.log 2023-09-30T22:41:08.822 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.156329.log.gz 2023-09-30T22:41:08.822 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.176311.log: /var/log/ceph/ceph-client.admin.171284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138885.log 2023-09-30T22:41:08.822 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.176311.log.gz 2023-09-30T22:41:08.822 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.171284.log.gz 2023-09-30T22:41:08.822 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.166111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188670.log 2023-09-30T22:41:08.823 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.138885.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.166111.log.gz 2023-09-30T22:41:08.823 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.188670.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.138885.log.gz 2023-09-30T22:41:08.823 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.188670.log.gz 2023-09-30T22:41:08.823 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166921.log 2023-09-30T22:41:08.823 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161477.log 2023-09-30T22:41:08.823 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.166921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190540.log 2023-09-30T22:41:08.823 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.166921.log.gz 2023-09-30T22:41:08.823 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.161477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180823.log 2023-09-30T22:41:08.824 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145952.log 2023-09-30T22:41:08.824 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.161477.log.gz 2023-09-30T22:41:08.824 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.190540.log: /var/log/ceph/ceph-client.admin.180823.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.190540.log.gz 2023-09-30T22:41:08.824 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128217.log 2023-09-30T22:41:08.824 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.145952.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.180823.log.gz 2023-09-30T22:41:08.824 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140414.log 2023-09-30T22:41:08.824 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.145952.log.gz 2023-09-30T22:41:08.824 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.128217.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.128217.log.gz 2023-09-30T22:41:08.825 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.140414.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.140414.log.gz 2023-09-30T22:41:08.825 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191423.log 2023-09-30T22:41:08.825 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152474.log 2023-09-30T22:41:08.825 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.191423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186413.log 2023-09-30T22:41:08.825 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.191423.log.gz 2023-09-30T22:41:08.825 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180139.log 2023-09-30T22:41:08.825 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.152474.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.152474.log.gz 2023-09-30T22:41:08.826 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.186413.log: /var/log/ceph/ceph-client.admin.180139.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.186413.log.gz 2023-09-30T22:41:08.826 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137884.log 2023-09-30T22:41:08.826 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.180139.log.gz 2023-09-30T22:41:08.826 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191012.log 2023-09-30T22:41:08.826 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.137884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148358.log 2023-09-30T22:41:08.826 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.137884.log.gz 2023-09-30T22:41:08.826 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.191012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122610.log 2023-09-30T22:41:08.826 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.191012.log.gz--verbose 2023-09-30T22:41:08.827 INFO:teuthology.orchestra.run.smithi016.stderr:-- /var/log/ceph/ceph-client.admin.135504.log 2023-09-30T22:41:08.827 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.148358.log: /var/log/ceph/ceph-client.admin.122610.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.148358.log.gz 2023-09-30T22:41:08.827 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.135504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138224.log 2023-09-30T22:41:08.827 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.122610.log.gz 2023-09-30T22:41:08.827 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.135504.log.gz 2023-09-30T22:41:08.827 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190029.log 2023-09-30T22:41:08.827 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172272.log 2023-09-30T22:41:08.827 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.138224.log: /var/log/ceph/ceph-client.admin.190029.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.138224.log.gz 2023-09-30T22:41:08.828 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.190029.log.gz 2023-09-30T22:41:08.828 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.172272.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.172272.log.gz 2023-09-30T22:41:08.828 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135086.log 2023-09-30T22:41:08.828 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179522.log 2023-09-30T22:41:08.828 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.135086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177391.log 2023-09-30T22:41:08.828 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.135086.log.gz 2023-09-30T22:41:08.828 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.179522.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.179522.log.gz 2023-09-30T22:41:08.829 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.177391.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.177391.log.gz 2023-09-30T22:41:08.829 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143206.log 2023-09-30T22:41:08.829 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165575.log 2023-09-30T22:41:08.829 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.143206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163582.log 2023-09-30T22:41:08.829 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.165575.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.143206.log.gz 2023-09-30T22:41:08.829 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151199.log 2023-09-30T22:41:08.829 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.163582.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.165575.log.gz 2023-09-30T22:41:08.829 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159243.log 2023-09-30T22:41:08.830 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.163582.log.gz 2023-09-30T22:41:08.830 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.151199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156132.log 2023-09-30T22:41:08.830 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.159243.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.151199.log.gz 2023-09-30T22:41:08.830 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184404.log 2023-09-30T22:41:08.830 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.159243.log.gz 2023-09-30T22:41:08.830 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.156132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150807.log 2023-09-30T22:41:08.830 INFO:teuthology.orchestra.run.smithi016.stderr:gzip 88.9%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.156132.log.gz-- 2023-09-30T22:41:08.830 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.155511.log 2023-09-30T22:41:08.831 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.184404.log: /var/log/ceph/ceph-client.admin.150807.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.184404.log.gz 2023-09-30T22:41:08.831 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178271.log 2023-09-30T22:41:08.831 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.150807.log.gz 2023-09-30T22:41:08.831 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.155511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166503.log 2023-09-30T22:41:08.831 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.155511.log.gz 2023-09-30T22:41:08.831 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.178271.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.178271.log.gz 2023-09-30T22:41:08.831 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.166503.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.166503.log.gz 2023-09-30T22:41:08.831 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158362.log 2023-09-30T22:41:08.832 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163040.log 2023-09-30T22:41:08.832 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.158362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186609.log 2023-09-30T22:41:08.832 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.158362.log.gz 2023-09-30T22:41:08.832 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.163040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148502.log 2023-09-30T22:41:08.832 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.163040.log.gz 2023-09-30T22:41:08.832 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-09-30T22:41:08.832 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.186609.log: /var/log/ceph/ceph-client.admin.148502.log: gzip -5 88.2%--verbose -- -- replaced with /var/log/ceph/ceph-client.admin.186609.log.gz/var/log/ceph/ceph-client.admin.123702.log 2023-09-30T22:41:08.832 INFO:teuthology.orchestra.run.smithi016.stderr: 2023-09-30T22:41:08.833 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-mgr.x.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.148502.log.gz 2023-09-30T22:41:08.833 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118930.log 2023-09-30T22:41:08.833 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.123702.log: /var/log/ceph/ceph-client.admin.118930.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.123702.log.gz 2023-09-30T22:41:08.833 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.118930.log.gz 2023-09-30T22:41:08.833 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123063.log 2023-09-30T22:41:08.833 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189808.log 2023-09-30T22:41:08.833 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.123063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179957.log 2023-09-30T22:41:08.833 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.189808.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.123063.log.gz 2023-09-30T22:41:08.834 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.179957.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.189808.log.gz 2023-09-30T22:41:08.834 INFO:teuthology.orchestra.run.smithi016.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.179957.log.gz 2023-09-30T22:41:08.834 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166385.log 2023-09-30T22:41:08.834 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133178.log 2023-09-30T22:41:08.834 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.166385.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.166385.log.gz 2023-09-30T22:41:08.834 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.133178.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.133178.log.gz 2023-09-30T22:41:08.834 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123212.log 2023-09-30T22:41:08.834 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183499.log 2023-09-30T22:41:08.835 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.123212.log: /var/log/ceph/ceph-client.admin.183499.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.123212.log.gz 2023-09-30T22:41:08.835 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.183499.log.gz 2023-09-30T22:41:08.835 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184600.log 2023-09-30T22:41:08.835 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149533.log 2023-09-30T22:41:08.835 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.184600.log: /var/log/ceph/ceph-client.admin.149533.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.184600.log.gz 2023-09-30T22:41:08.835 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.149533.log.gz 2023-09-30T22:41:08.835 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150049.log 2023-09-30T22:41:08.835 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141734.log 2023-09-30T22:41:08.836 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.150049.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.150049.log.gz/var/log/ceph/ceph-client.admin.141734.log: 2023-09-30T22:41:08.836 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.141734.log.gz 2023-09-30T22:41:08.836 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193838.log 2023-09-30T22:41:08.836 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150637.log 2023-09-30T22:41:08.836 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.193838.log: /var/log/ceph/ceph-client.admin.150637.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.193838.log.gz 2023-09-30T22:41:08.836 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.150637.log.gz 2023-09-30T22:41:08.836 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163850.log 2023-09-30T22:41:08.836 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140584.log 2023-09-30T22:41:08.836 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.163850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176358.log 2023-09-30T22:41:08.837 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.140584.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.163850.log.gz 2023-09-30T22:41:08.837 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.176358.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.140584.log.gz 2023-09-30T22:41:08.837 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.176358.log.gz 2023-09-30T22:41:08.837 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166137.log 2023-09-30T22:41:08.837 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164347.log 2023-09-30T22:41:08.837 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.166137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193336.log 2023-09-30T22:41:08.837 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.164347.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.166137.log.gz 2023-09-30T22:41:08.838 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.164347.log.gz 2023-09-30T22:41:08.838 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.193336.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.193336.log.gz 2023-09-30T22:41:08.838 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184061.log 2023-09-30T22:41:08.838 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166994.log 2023-09-30T22:41:08.838 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.184061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157001.log 2023-09-30T22:41:08.838 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.184061.log.gz 2023-09-30T22:41:08.838 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.166994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147297.log 2023-09-30T22:41:08.838 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.166994.log.gz 2023-09-30T22:41:08.839 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.157001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163651.log 2023-09-30T22:41:08.839 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.157001.log.gz 2023-09-30T22:41:08.839 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.147297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151787.log 2023-09-30T22:41:08.839 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.147297.log.gz 2023-09-30T22:41:08.839 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.163651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148455.log 2023-09-30T22:41:08.839 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.163651.log.gz 2023-09-30T22:41:08.839 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.151787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127604.log 2023-09-30T22:41:08.839 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.151787.log.gz 2023-09-30T22:41:08.840 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.148455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192500.log 2023-09-30T22:41:08.840 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.148455.log.gz 2023-09-30T22:41:08.840 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.127604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141538.log 2023-09-30T22:41:08.840 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.127604.log.gz 2023-09-30T22:41:08.840 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.192500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164445.log 2023-09-30T22:41:08.840 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.192500.log.gz 2023-09-30T22:41:08.840 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.141538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171677.log 2023-09-30T22:41:08.840 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8%/var/log/ceph/ceph-client.admin.164445.log: -- replaced with /var/log/ceph/ceph-client.admin.141538.log.gz 2023-09-30T22:41:08.841 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.164445.log.gz 2023-09-30T22:41:08.841 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.171677.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.171677.log.gz 2023-09-30T22:41:08.841 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191964.log 2023-09-30T22:41:08.841 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185485.log 2023-09-30T22:41:08.841 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.191964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157871.log 2023-09-30T22:41:08.841 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.191964.log.gz 2023-09-30T22:41:08.841 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.185485.log: /var/log/ceph/ceph-client.admin.157871.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.185485.log.gz 2023-09-30T22:41:08.841 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.157871.log.gz 2023-09-30T22:41:08.842 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146905.log 2023-09-30T22:41:08.842 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172856.log 2023-09-30T22:41:08.842 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.146905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178859.log 2023-09-30T22:41:08.842 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.146905.log.gz 2023-09-30T22:41:08.842 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.172856.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.172856.log.gz 2023-09-30T22:41:08.842 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.178859.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.178859.log.gz 2023-09-30T22:41:08.842 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163066.log 2023-09-30T22:41:08.842 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146806.log 2023-09-30T22:41:08.843 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.163066.log: /var/log/ceph/ceph-client.admin.146806.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.163066.log.gz 2023-09-30T22:41:08.843 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.146806.log.gz 2023-09-30T22:41:08.843 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134746.log 2023-09-30T22:41:08.843 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127073.log 2023-09-30T22:41:08.843 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.134746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187252.log 2023-09-30T22:41:08.843 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.134746.log.gz 2023-09-30T22:41:08.843 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.127073.log: /var/log/ceph/ceph-client.admin.187252.log: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.127073.log.gz 2023-09-30T22:41:08.843 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.187252.log.gz 2023-09-30T22:41:08.844 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139676.log 2023-09-30T22:41:08.844 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190226.log 2023-09-30T22:41:08.844 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.139676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192330.log 2023-09-30T22:41:08.844 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.190226.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.139676.log.gz 2023-09-30T22:41:08.844 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.190226.log.gz 2023-09-30T22:41:08.844 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.192330.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.192330.log.gz 2023-09-30T22:41:08.844 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159073.log 2023-09-30T22:41:08.844 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144086.log 2023-09-30T22:41:08.844 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.159073.log: /var/log/ceph/ceph-client.admin.144086.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.159073.log.gz 2023-09-30T22:41:08.845 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.144086.log.gz 2023-09-30T22:41:08.845 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143547.log 2023-09-30T22:41:08.845 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174058.log 2023-09-30T22:41:08.845 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.143547.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.143547.log.gz 2023-09-30T22:41:08.845 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.174058.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.174058.log.gz 2023-09-30T22:41:08.845 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142910.log 2023-09-30T22:41:08.845 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178176.log 2023-09-30T22:41:08.845 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.142910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166307.log 2023-09-30T22:41:08.846 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.142910.log.gz 2023-09-30T22:41:08.846 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.178176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162870.log 2023-09-30T22:41:08.846 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.178176.log.gz 2023-09-30T22:41:08.846 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.166307.log: /var/log/ceph/ceph-client.admin.162870.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.166307.log.gz 2023-09-30T22:41:08.846 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162039.log 2023-09-30T22:41:08.846 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.162870.log.gz 2023-09-30T22:41:08.846 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145703.log 2023-09-30T22:41:08.846 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.162039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173763.log 2023-09-30T22:41:08.847 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.162039.log.gz 2023-09-30T22:41:08.847 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.145703.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.145703.log.gz 2023-09-30T22:41:08.847 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.173763.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.173763.log.gz 2023-09-30T22:41:08.847 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165353.log 2023-09-30T22:41:08.847 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187723.log 2023-09-30T22:41:08.847 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.165353.log: /var/log/ceph/ceph-client.admin.187723.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.165353.log.gz 2023-09-30T22:41:08.847 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144383.log 2023-09-30T22:41:08.847 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.187723.log.gz 2023-09-30T22:41:08.848 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180680.log 2023-09-30T22:41:08.848 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.144383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138786.log 2023-09-30T22:41:08.848 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.180680.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.144383.log.gz 2023-09-30T22:41:08.848 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.180680.log.gz 2023-09-30T22:41:08.848 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.138786.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.138786.log.gz 2023-09-30T22:41:08.848 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138028.log 2023-09-30T22:41:08.848 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168881.log 2023-09-30T22:41:08.848 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.138028.log: /var/log/ceph/ceph-client.admin.168881.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.138028.log.gz 2023-09-30T22:41:08.849 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.168881.log.gz 2023-09-30T22:41:08.849 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139238.log 2023-09-30T22:41:08.849 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134890.log 2023-09-30T22:41:08.849 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.139238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178541.log 2023-09-30T22:41:08.849 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.134890.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.139238.log.gz 2023-09-30T22:41:08.849 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.178541.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.134890.log.gz 2023-09-30T22:41:08.849 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173472.log 2023-09-30T22:41:08.883 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.178541.log.gz 2023-09-30T22:41:08.883 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190958.log 2023-09-30T22:41:08.883 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.173472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146236.log 2023-09-30T22:41:08.883 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.173472.log.gz 2023-09-30T22:41:08.884 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.190958.log: 90.8% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-09-30T22:41:08.884 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.146236.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.190958.log.gz 2023-09-30T22:41:08.884 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.146236.log.gz 2023-09-30T22:41:08.884 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147747.log 2023-09-30T22:41:08.884 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155414.log 2023-09-30T22:41:08.884 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.147747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168319.log 2023-09-30T22:41:08.884 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8%/var/log/ceph/ceph-client.admin.155414.log: -- replaced with /var/log/ceph/ceph-client.admin.147747.log.gz 2023-09-30T22:41:08.885 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.155414.log.gz 2023-09-30T22:41:08.885 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.168319.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.168319.log.gz 2023-09-30T22:41:08.885 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141565.log 2023-09-30T22:41:08.885 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174939.log 2023-09-30T22:41:08.885 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.141565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175182.log 2023-09-30T22:41:08.885 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5%/var/log/ceph/ceph-client.admin.174939.log: -- replaced with /var/log/ceph/ceph-client.admin.141565.log.gz 2023-09-30T22:41:08.885 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.174939.log.gz 2023-09-30T22:41:08.885 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.175182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124535.log 2023-09-30T22:41:08.886 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.175182.log.gz 2023-09-30T22:41:08.886 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164151.log 2023-09-30T22:41:08.886 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.124535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161203.log 2023-09-30T22:41:08.886 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.124535.log.gz 2023-09-30T22:41:08.886 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.164151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188972.log 2023-09-30T22:41:08.886 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.164151.log.gz 2023-09-30T22:41:08.886 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150104.log 2023-09-30T22:41:08.887 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.161203.log: /var/log/ceph/ceph-client.admin.188972.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.161203.log.gz 2023-09-30T22:41:08.887 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.188972.log.gz/var/log/ceph/ceph-client.admin.150104.log: 2023-09-30T22:41:08.887 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146147.log 2023-09-30T22:41:08.887 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.150104.log.gz 2023-09-30T22:41:08.887 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.146147.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.146147.log.gz 2023-09-30T22:41:08.887 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139996.log 2023-09-30T22:41:08.887 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143890.log 2023-09-30T22:41:08.887 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.139996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188671.log 2023-09-30T22:41:08.888 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.139996.log.gz 2023-09-30T22:41:08.888 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.143890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141003.log 2023-09-30T22:41:08.888 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124218.log 2023-09-30T22:41:08.888 INFO:teuthology.orchestra.run.smithi016.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.143890.log.gz 2023-09-30T22:41:08.888 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.188671.log: /var/log/ceph/ceph-client.admin.141003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178542.log 2023-09-30T22:41:08.888 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.188671.log.gz 2023-09-30T22:41:08.888 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.141003.log.gz 2023-09-30T22:41:08.889 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.124218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145258.log 2023-09-30T22:41:08.889 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.124218.log.gz 2023-09-30T22:41:08.889 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.178542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176212.log 2023-09-30T22:41:08.889 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.178542.log.gz 2023-09-30T22:41:08.889 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.145258.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.145258.log.gz 2023-09-30T22:41:08.889 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.176212.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.176212.log.gz 2023-09-30T22:41:08.889 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140780.log 2023-09-30T22:41:08.889 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186464.log 2023-09-30T22:41:08.890 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.140780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139212.log 2023-09-30T22:41:08.890 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.140780.log.gz 2023-09-30T22:41:08.890 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.186464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180951.log 2023-09-30T22:41:08.890 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.186464.log.gz--verbose 2023-09-30T22:41:08.890 INFO:teuthology.orchestra.run.smithi016.stderr:-- /var/log/ceph/ceph-client.admin.138002.log 2023-09-30T22:41:08.890 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.139212.log: /var/log/ceph/ceph-client.admin.180951.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.139212.log.gz 2023-09-30T22:41:08.890 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.138002.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.180951.log.gz 2023-09-30T22:41:08.891 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.138002.log.gz 2023-09-30T22:41:08.891 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124780.log 2023-09-30T22:41:08.891 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155537.log 2023-09-30T22:41:08.891 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.124780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181931.log 2023-09-30T22:41:08.891 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.124780.log.gz 2023-09-30T22:41:08.891 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.155537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178467.log 2023-09-30T22:41:08.891 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162456.log 2023-09-30T22:41:08.892 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.155537.log.gz 2023-09-30T22:41:08.892 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.181931.log: /var/log/ceph/ceph-client.admin.178467.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.181931.log.gz 2023-09-30T22:41:08.892 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175448.log 2023-09-30T22:41:08.892 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.178467.log.gz 2023-09-30T22:41:08.892 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.162456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183277.log 2023-09-30T22:41:08.892 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.162456.log.gz 2023-09-30T22:41:08.892 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.175448.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.175448.log.gz 2023-09-30T22:41:08.893 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.183277.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.183277.log.gz 2023-09-30T22:41:08.893 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122855.log 2023-09-30T22:41:08.893 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191180.log 2023-09-30T22:41:08.893 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.122855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177392.log 2023-09-30T22:41:08.893 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.191180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123457.log 2023-09-30T22:41:08.893 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.122855.log.gz 2023-09-30T22:41:08.893 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.191180.log.gz 2023-09-30T22:41:08.894 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.177392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164177.log 2023-09-30T22:41:08.894 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158951.log 2023-09-30T22:41:08.894 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.123457.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.177392.log.gz 2023-09-30T22:41:08.894 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.164177.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.123457.log.gz 2023-09-30T22:41:08.894 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171480.log 2023-09-30T22:41:08.894 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.164177.log.gz 2023-09-30T22:41:08.894 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.158951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158681.log 2023-09-30T22:41:08.894 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.158951.log.gz 2023-09-30T22:41:08.895 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.171480.log: /var/log/ceph/ceph-client.admin.158681.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.171480.log.gz 2023-09-30T22:41:08.895 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.158681.log.gz 2023-09-30T22:41:08.895 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157542.log 2023-09-30T22:41:08.895 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172017.log 2023-09-30T22:41:08.895 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.157542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181464.log 2023-09-30T22:41:08.895 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.157542.log.gz 2023-09-30T22:41:08.895 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.172017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190761.log 2023-09-30T22:41:08.896 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175135.log 2023-09-30T22:41:08.896 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.172017.log.gz 2023-09-30T22:41:08.896 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.181464.log: /var/log/ceph/ceph-client.admin.190761.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.181464.log.gz 2023-09-30T22:41:08.896 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162379.log 2023-09-30T22:41:08.896 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.190761.log.gz 2023-09-30T22:41:08.896 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.175135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156695.log 2023-09-30T22:41:08.896 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.175135.log.gz 2023-09-30T22:41:08.896 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.162379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153216.log 2023-09-30T22:41:08.897 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177562.log 2023-09-30T22:41:08.897 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.162379.log.gz 2023-09-30T22:41:08.897 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.156695.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.156695.log.gz 2023-09-30T22:41:08.897 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.153216.log: /var/log/ceph/ceph-client.admin.177562.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.153216.log.gz 2023-09-30T22:41:08.897 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.177562.log.gz 2023-09-30T22:41:08.897 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148847.log 2023-09-30T22:41:08.897 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191128.log 2023-09-30T22:41:08.897 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.148847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168954.log 2023-09-30T22:41:08.898 INFO:teuthology.orchestra.run.smithi016.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.148847.log.gz 2023-09-30T22:41:08.898 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.191128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137022.log 2023-09-30T22:41:08.898 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156747.log 2023-09-30T22:41:08.898 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.168954.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.191128.log.gz 2023-09-30T22:41:08.898 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.137022.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.168954.log.gz 2023-09-30T22:41:08.898 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.137022.log.gz 2023-09-30T22:41:08.898 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.156747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160641.log 2023-09-30T22:41:08.899 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.156747.log.gz 2023-09-30T22:41:08.899 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181803.log 2023-09-30T22:41:08.899 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.160641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149265.log 2023-09-30T22:41:08.899 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.160641.log.gz 2023-09-30T22:41:08.899 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.181803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141415.log 2023-09-30T22:41:08.899 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.181803.log.gz 2023-09-30T22:41:08.899 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.149265.log: /var/log/ceph/ceph-client.admin.141415.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.149265.log.gz 2023-09-30T22:41:08.899 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.141415.log.gz 2023-09-30T22:41:08.900 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173371.log 2023-09-30T22:41:08.900 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-09-30T22:41:08.900 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.173371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143599.log 2023-09-30T22:41:08.900 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.173371.log.gz 2023-09-30T22:41:08.900 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168318.log 2023-09-30T22:41:08.900 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152404.log 2023-09-30T22:41:08.900 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.143599.log: /var/log/ceph/ceph-client.admin.168318.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.143599.log.gz 2023-09-30T22:41:08.900 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.168318.log.gz 2023-09-30T22:41:08.901 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.152404.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.152404.log.gz 2023-09-30T22:41:08.901 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137503.log 2023-09-30T22:41:08.901 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134229.log 2023-09-30T22:41:08.901 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.137503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175235.log 2023-09-30T22:41:08.901 INFO:teuthology.orchestra.run.smithi016.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.137503.log.gz 2023-09-30T22:41:08.901 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.134229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151094.log 2023-09-30T22:41:08.901 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.134229.log.gz 2023-09-30T22:41:08.901 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.175235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153990.log 2023-09-30T22:41:08.902 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.151094.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.175235.log.gz 2023-09-30T22:41:08.902 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.151094.log.gz 2023-09-30T22:41:08.902 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.153990.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.153990.log.gz 2023-09-30T22:41:08.902 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124682.log 2023-09-30T22:41:08.902 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186073.log 2023-09-30T22:41:08.902 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.124682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122369.log 2023-09-30T22:41:08.902 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.186073.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.124682.log.gz 2023-09-30T22:41:08.903 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.186073.log.gz 2023-09-30T22:41:08.903 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.122369.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.122369.log.gz 2023-09-30T22:41:08.903 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160889.log 2023-09-30T22:41:08.903 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184479.log 2023-09-30T22:41:08.903 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.160889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188232.log 2023-09-30T22:41:08.903 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.160889.log.gz 2023-09-30T22:41:08.903 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.184479.log: /var/log/ceph/ceph-client.admin.188232.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.184479.log.gz 2023-09-30T22:41:08.903 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.188232.log.gz 2023-09-30T22:41:08.904 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181215.log 2023-09-30T22:41:08.904 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-09-30T22:41:08.904 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.181215.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-09-30T22:41:08.904 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.181215.log.gz 2023-09-30T22:41:08.904 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164373.log 2023-09-30T22:41:08.904 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.164373.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.164373.log.gz 2023-09-30T22:41:08.904 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180558.log 2023-09-30T22:41:08.905 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.180558.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.180558.log.gz 2023-09-30T22:41:08.905 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192008.log 2023-09-30T22:41:08.905 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.192008.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.192008.log.gz 2023-09-30T22:41:08.905 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149334.log 2023-09-30T22:41:08.905 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.149334.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.149334.log.gz 2023-09-30T22:41:08.905 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158263.log 2023-09-30T22:41:08.905 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.158263.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.158263.log.gz 2023-09-30T22:41:08.906 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123408.log 2023-09-30T22:41:08.906 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.123408.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.123408.log.gz 2023-09-30T22:41:08.906 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150415.log 2023-09-30T22:41:08.906 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.150415.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.150415.log.gz 2023-09-30T22:41:08.906 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165915.log 2023-09-30T22:41:08.906 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.165915.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.165915.log.gz 2023-09-30T22:41:08.906 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154335.log 2023-09-30T22:41:08.906 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.154335.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.154335.log.gz 2023-09-30T22:41:08.907 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128480.log 2023-09-30T22:41:08.907 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.128480.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.128480.log.gz 2023-09-30T22:41:08.907 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164790.log 2023-09-30T22:41:08.907 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.164790.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.164790.log.gz 2023-09-30T22:41:08.907 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134942.log 2023-09-30T22:41:08.907 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.134942.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.134942.log.gz 2023-09-30T22:41:08.907 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190618.log 2023-09-30T22:41:08.907 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.190618.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.190618.log.gz 2023-09-30T22:41:08.908 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179888.log 2023-09-30T22:41:08.908 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.179888.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.179888.log.gz 2023-09-30T22:41:08.908 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149657.log 2023-09-30T22:41:08.908 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.149657.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.149657.log.gz 2023-09-30T22:41:08.908 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.194132.log 2023-09-30T22:41:08.908 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.194132.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.194132.log.gz 2023-09-30T22:41:08.908 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188998.log 2023-09-30T22:41:08.909 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.188998.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.188998.log.gz 2023-09-30T22:41:08.909 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137687.log 2023-09-30T22:41:08.909 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.137687.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.137687.log.gz 2023-09-30T22:41:08.909 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144971.log 2023-09-30T22:41:08.909 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.144971.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.144971.log.gz 2023-09-30T22:41:08.909 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162575.log 2023-09-30T22:41:08.909 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144723.log 2023-09-30T22:41:08.909 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.162575.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.162575.log.gz 2023-09-30T22:41:08.909 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.144723.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.144723.log.gz 2023-09-30T22:41:08.910 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159047.log 2023-09-30T22:41:08.910 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.159047.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.159047.log.gz 2023-09-30T22:41:08.910 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145844.log 2023-09-30T22:41:08.910 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.145844.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.145844.log.gz 2023-09-30T22:41:08.910 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171991.log 2023-09-30T22:41:08.910 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.171991.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.171991.log.gz 2023-09-30T22:41:08.910 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147671.log 2023-09-30T22:41:08.910 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.147671.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.147671.log.gz 2023-09-30T22:41:08.911 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184871.log 2023-09-30T22:41:08.911 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.184871.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.184871.log.gz 2023-09-30T22:41:08.911 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164817.log 2023-09-30T22:41:08.911 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.164817.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.164817.log.gz 2023-09-30T22:41:08.911 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122757.log 2023-09-30T22:41:08.911 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.122757.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.122757.log.gz 2023-09-30T22:41:08.911 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145337.log 2023-09-30T22:41:08.911 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.145337.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.145337.log.gz 2023-09-30T22:41:08.912 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123006.log 2023-09-30T22:41:08.912 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.123006.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.123006.log.gz 2023-09-30T22:41:08.912 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175378.log 2023-09-30T22:41:08.912 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.175378.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.175378.log.gz 2023-09-30T22:41:08.912 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125172.log 2023-09-30T22:41:08.912 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.125172.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.125172.log.gz 2023-09-30T22:41:08.912 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168562.log 2023-09-30T22:41:08.912 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.168562.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.168562.log.gz 2023-09-30T22:41:08.913 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192382.log 2023-09-30T22:41:08.913 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.192382.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.192382.log.gz 2023-09-30T22:41:08.913 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182519.log 2023-09-30T22:41:08.913 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.182519.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.182519.log.gz 2023-09-30T22:41:08.913 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188402.log 2023-09-30T22:41:08.913 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.188402.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.188402.log.gz 2023-09-30T22:41:08.913 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164244.log 2023-09-30T22:41:08.913 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.164244.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.164244.log.gz 2023-09-30T22:41:08.914 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142349.log 2023-09-30T22:41:08.914 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.142349.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.142349.log.gz 2023-09-30T22:41:08.914 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174800.log 2023-09-30T22:41:08.914 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.174800.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.174800.log.gz 2023-09-30T22:41:08.914 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188477.log 2023-09-30T22:41:08.914 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.188477.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.188477.log.gz 2023-09-30T22:41:08.914 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177758.log 2023-09-30T22:41:08.915 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.177758.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.177758.log.gz 2023-09-30T22:41:08.915 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184897.log 2023-09-30T22:41:08.915 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.184897.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.184897.log.gz 2023-09-30T22:41:08.915 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123161.log 2023-09-30T22:41:08.915 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.123161.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.123161.log.gz 2023-09-30T22:41:08.915 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152793.log 2023-09-30T22:41:08.915 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.152793.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.152793.log.gz 2023-09-30T22:41:08.915 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159439.log 2023-09-30T22:41:08.915 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160027.log 2023-09-30T22:41:08.916 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.159439.log: /var/log/ceph/ceph-client.admin.160027.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.159439.log.gz 2023-09-30T22:41:08.916 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.160027.log.gz 2023-09-30T22:41:08.916 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172417.log 2023-09-30T22:41:08.916 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.172417.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.172417.log.gz 2023-09-30T22:41:08.916 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127074.log 2023-09-30T22:41:08.916 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.127074.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.127074.log.gz 2023-09-30T22:41:08.916 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185680.log 2023-09-30T22:41:08.917 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.185680.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.185680.log.gz 2023-09-30T22:41:08.917 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128189.log 2023-09-30T22:41:08.917 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.128189.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.128189.log.gz 2023-09-30T22:41:08.917 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169913.log 2023-09-30T22:41:08.917 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.169913.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.169913.log.gz 2023-09-30T22:41:08.917 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135896.log 2023-09-30T22:41:08.917 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.135896.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.135896.log.gz 2023-09-30T22:41:08.917 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173864.log 2023-09-30T22:41:08.918 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.173864.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.173864.log.gz 2023-09-30T22:41:08.918 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156499.log 2023-09-30T22:41:08.918 INFO:teuthology.orchestra.run.smithi016.stderr: 94.7% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-09-30T22:41:08.918 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.156499.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.156499.log.gz 2023-09-30T22:41:08.918 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178372.log 2023-09-30T22:41:08.918 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148678.log 2023-09-30T22:41:08.918 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.178372.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.178372.log.gz 2023-09-30T22:41:08.918 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.148678.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.148678.log.gz 2023-09-30T22:41:08.919 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160496.log 2023-09-30T22:41:08.919 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166333.log 2023-09-30T22:41:08.919 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.160496.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.160496.log.gz 2023-09-30T22:41:08.919 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.166333.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.166333.log.gz 2023-09-30T22:41:08.919 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127725.log 2023-09-30T22:41:08.919 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161817.log 2023-09-30T22:41:08.919 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.127725.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.127725.log.gz 2023-09-30T22:41:08.919 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.161817.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.161817.log.gz 2023-09-30T22:41:08.920 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164054.log 2023-09-30T22:41:08.920 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146539.log 2023-09-30T22:41:08.920 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.164054.log: /var/log/ceph/ceph-client.admin.146539.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.164054.log.gz 2023-09-30T22:41:08.920 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.146539.log.gz 2023-09-30T22:41:08.920 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179765.log 2023-09-30T22:41:08.920 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177196.log 2023-09-30T22:41:08.920 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.179765.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.179765.log.gz 2023-09-30T22:41:08.920 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.177196.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.177196.log.gz 2023-09-30T22:41:08.921 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159831.log 2023-09-30T22:41:08.921 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171795.log 2023-09-30T22:41:08.921 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.159831.log: /var/log/ceph/ceph-client.admin.171795.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.159831.log.gz 2023-09-30T22:41:08.921 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.171795.log.gz 2023-09-30T22:41:08.921 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159908.log 2023-09-30T22:41:08.921 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190422.log 2023-09-30T22:41:08.921 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.159908.log: /var/log/ceph/ceph-client.admin.190422.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.159908.log.gz 2023-09-30T22:41:08.922 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.190422.log.gz 2023-09-30T22:41:08.922 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-09-30T22:41:08.922 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137686.log 2023-09-30T22:41:08.922 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-mon.c.log: /var/log/ceph/ceph-client.admin.137686.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.137686.log.gz 2023-09-30T22:41:08.922 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127211.log 2023-09-30T22:41:08.922 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.127211.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.127211.log.gz 2023-09-30T22:41:08.922 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163412.log 2023-09-30T22:41:08.922 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.163412.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.163412.log.gz 2023-09-30T22:41:08.923 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129074.log 2023-09-30T22:41:08.923 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.129074.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.129074.log.gz 2023-09-30T22:41:08.923 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143403.log 2023-09-30T22:41:08.923 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.143403.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.143403.log.gz 2023-09-30T22:41:08.923 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190395.log 2023-09-30T22:41:08.923 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.190395.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.190395.log.gz 2023-09-30T22:41:08.923 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138590.log 2023-09-30T22:41:08.924 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.138590.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.138590.log.gz 2023-09-30T22:41:08.924 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166188.log 2023-09-30T22:41:08.924 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.166188.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.166188.log.gz 2023-09-30T22:41:08.924 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160300.log 2023-09-30T22:41:08.924 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.160300.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.160300.log.gz 2023-09-30T22:41:08.924 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139604.log 2023-09-30T22:41:08.924 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.139604.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.139604.log.gz 2023-09-30T22:41:08.924 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142322.log 2023-09-30T22:41:08.925 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.142322.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.142322.log.gz 2023-09-30T22:41:08.925 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139008.log 2023-09-30T22:41:08.925 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.139008.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.139008.log.gz 2023-09-30T22:41:08.925 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188376.log 2023-09-30T22:41:08.925 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.188376.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.188376.log.gz 2023-09-30T22:41:08.925 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159635.log 2023-09-30T22:41:08.925 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.159635.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.159635.log.gz 2023-09-30T22:41:08.926 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157505.log 2023-09-30T22:41:08.926 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.157505.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.157505.log.gz 2023-09-30T22:41:08.926 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148306.log 2023-09-30T22:41:08.926 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.148306.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.148306.log.gz 2023-09-30T22:41:08.926 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167386.log 2023-09-30T22:41:08.926 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.167386.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.167386.log.gz 2023-09-30T22:41:08.926 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176758.log 2023-09-30T22:41:08.926 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.176758.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.176758.log.gz 2023-09-30T22:41:08.927 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183175.log 2023-09-30T22:41:08.927 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.183175.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.183175.log.gz 2023-09-30T22:41:08.927 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161399.log 2023-09-30T22:41:08.927 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.161399.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.161399.log.gz 2023-09-30T22:41:08.927 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146121.log 2023-09-30T22:41:08.927 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.146121.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.146121.log.gz 2023-09-30T22:41:08.927 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163359.log 2023-09-30T22:41:08.927 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.163359.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.163359.log.gz 2023-09-30T22:41:08.928 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182911.log 2023-09-30T22:41:08.940 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.182911.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.182911.log.gz 2023-09-30T22:41:08.940 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143991.log 2023-09-30T22:41:08.940 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.143991.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.143991.log.gz 2023-09-30T22:41:08.940 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160249.log 2023-09-30T22:41:08.940 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.160249.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.160249.log.gz 2023-09-30T22:41:08.941 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128449.log 2023-09-30T22:41:08.941 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.128449.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.128449.log.gz 2023-09-30T22:41:08.941 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142126.log 2023-09-30T22:41:08.941 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.142126.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.142126.log.gz 2023-09-30T22:41:08.941 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152070.log 2023-09-30T22:41:08.941 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.152070.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.152070.log.gz 2023-09-30T22:41:08.941 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153551.log 2023-09-30T22:41:08.942 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.153551.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.153551.log.gz 2023-09-30T22:41:08.942 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175574.log 2023-09-30T22:41:08.942 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.175574.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.175574.log.gz 2023-09-30T22:41:08.942 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153943.log 2023-09-30T22:41:08.942 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.153943.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.153943.log.gz 2023-09-30T22:41:08.942 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137831.log 2023-09-30T22:41:08.942 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.137831.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.137831.log.gz 2023-09-30T22:41:08.942 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125488.log 2023-09-30T22:41:08.943 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.125488.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.125488.log.gz 2023-09-30T22:41:08.943 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172273.log 2023-09-30T22:41:08.943 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.172273.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.172273.log.gz 2023-09-30T22:41:08.943 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167974.log 2023-09-30T22:41:08.943 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.167974.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.167974.log.gz 2023-09-30T22:41:08.943 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176024.log 2023-09-30T22:41:08.943 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.176024.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.176024.log.gz 2023-09-30T22:41:08.944 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176948.log 2023-09-30T22:41:08.944 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.176948.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.176948.log.gz 2023-09-30T22:41:08.944 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183032.log 2023-09-30T22:41:08.944 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.183032.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.183032.log.gz 2023-09-30T22:41:08.944 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148152.log 2023-09-30T22:41:08.944 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.148152.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.148152.log.gz 2023-09-30T22:41:08.944 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169443.log 2023-09-30T22:41:08.944 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.169443.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.169443.log.gz 2023-09-30T22:41:08.945 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170645.log 2023-09-30T22:41:08.945 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.170645.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.170645.log.gz 2023-09-30T22:41:08.945 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142423.log 2023-09-30T22:41:08.945 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.142423.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.142423.log.gz 2023-09-30T22:41:08.945 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172979.log 2023-09-30T22:41:08.945 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.172979.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.172979.log.gz 2023-09-30T22:41:08.945 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125341.log 2023-09-30T22:41:08.945 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.125341.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.125341.log.gz 2023-09-30T22:41:08.946 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183107.log 2023-09-30T22:41:08.946 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.183107.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.183107.log.gz 2023-09-30T22:41:08.946 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191716.log 2023-09-30T22:41:08.946 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.191716.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.191716.log.gz 2023-09-30T22:41:08.946 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152866.log 2023-09-30T22:41:08.946 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.152866.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.152866.log.gz 2023-09-30T22:41:08.946 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134547.log 2023-09-30T22:41:08.947 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.134547.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.134547.log.gz 2023-09-30T22:41:08.947 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151815.log 2023-09-30T22:41:08.947 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169272.log 2023-09-30T22:41:08.947 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.151815.log: /var/log/ceph/ceph-client.admin.169272.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.151815.log.gz 2023-09-30T22:41:08.947 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.169272.log.gz 2023-09-30T22:41:08.947 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153428.log 2023-09-30T22:41:08.947 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.153428.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.153428.log.gz 2023-09-30T22:41:08.948 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154531.log 2023-09-30T22:41:08.948 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.154531.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.154531.log.gz 2023-09-30T22:41:08.948 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126793.log 2023-09-30T22:41:08.948 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.126793.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.126793.log.gz 2023-09-30T22:41:08.948 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186856.log 2023-09-30T22:41:08.948 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.186856.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.186856.log.gz 2023-09-30T22:41:08.948 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135256.log 2023-09-30T22:41:08.948 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.135256.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.135256.log.gz 2023-09-30T22:41:08.949 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.172391.log 2023-09-30T22:41:08.949 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.172391.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.172391.log.gz 2023-09-30T22:41:08.949 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147574.log 2023-09-30T22:41:08.949 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.147574.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.147574.log.gz 2023-09-30T22:41:08.949 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182101.log 2023-09-30T22:41:08.949 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.182101.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.182101.log.gz 2023-09-30T22:41:08.949 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148724.log 2023-09-30T22:41:08.950 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178075.log 2023-09-30T22:41:08.950 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.148724.log: /var/log/ceph/ceph-client.admin.178075.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.148724.log.gz 2023-09-30T22:41:08.950 INFO:teuthology.orchestra.run.smithi016.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.178075.log.gz 2023-09-30T22:41:08.950 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175843.log 2023-09-30T22:41:08.950 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.175843.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.175843.log.gz 2023-09-30T22:41:08.950 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189952.log 2023-09-30T22:41:08.950 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.189952.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.189952.log.gz 2023-09-30T22:41:08.950 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125221.log 2023-09-30T22:41:08.951 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.125221.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.125221.log.gz 2023-09-30T22:41:08.951 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163955.log 2023-09-30T22:41:08.951 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.163955.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.163955.log.gz 2023-09-30T22:41:08.951 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190200.log 2023-09-30T22:41:08.951 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.190200.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.190200.log.gz 2023-09-30T22:41:08.951 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177757.log 2023-09-30T22:41:08.951 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.177757.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.177757.log.gz 2023-09-30T22:41:08.951 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138494.log 2023-09-30T22:41:08.952 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.138494.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.138494.log.gz 2023-09-30T22:41:08.952 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129490.log 2023-09-30T22:41:08.952 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.129490.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.129490.log.gz 2023-09-30T22:41:08.952 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154923.log 2023-09-30T22:41:08.952 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.154923.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.154923.log.gz 2023-09-30T22:41:08.952 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141442.log 2023-09-30T22:41:08.952 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.141442.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.141442.log.gz 2023-09-30T22:41:08.953 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137858.log 2023-09-30T22:41:08.953 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.137858.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.137858.log.gz 2023-09-30T22:41:08.953 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159909.log 2023-09-30T22:41:08.953 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.159909.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.159909.log.gz 2023-09-30T22:41:08.953 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-09-30T22:41:08.957 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph.log: 94.2% -- replaced with /var/log/ceph/ceph.log.gz 2023-09-30T22:41:08.958 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183865.log 2023-09-30T22:41:08.958 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.183865.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.183865.log.gz 2023-09-30T22:41:08.958 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177954.log 2023-09-30T22:41:08.958 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.177954.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.177954.log.gz 2023-09-30T22:41:08.958 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127656.log 2023-09-30T22:41:08.958 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.127656.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.127656.log.gz 2023-09-30T22:41:08.958 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148162.log 2023-09-30T22:41:08.959 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.148162.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.148162.log.gz 2023-09-30T22:41:08.959 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168463.log 2023-09-30T22:41:08.959 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.168463.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.168463.log.gz 2023-09-30T22:41:08.959 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177195.log 2023-09-30T22:41:08.959 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.177195.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.177195.log.gz 2023-09-30T22:41:08.959 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192187.log 2023-09-30T22:41:08.959 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.192187.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.192187.log.gz 2023-09-30T22:41:08.959 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167536.log 2023-09-30T22:41:08.960 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.167536.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.167536.log.gz 2023-09-30T22:41:08.960 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192892.log 2023-09-30T22:41:08.960 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.192892.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.192892.log.gz 2023-09-30T22:41:08.960 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128364.log 2023-09-30T22:41:08.960 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.128364.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.128364.log.gz 2023-09-30T22:41:08.960 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175919.log 2023-09-30T22:41:08.960 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.175919.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.175919.log.gz 2023-09-30T22:41:08.960 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144673.log 2023-09-30T22:41:08.961 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.144673.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.144673.log.gz 2023-09-30T22:41:08.961 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158851.log 2023-09-30T22:41:08.961 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.158851.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.158851.log.gz 2023-09-30T22:41:08.961 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148920.log 2023-09-30T22:41:08.961 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.148920.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.148920.log.gz 2023-09-30T22:41:08.961 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140192.log 2023-09-30T22:41:08.961 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.140192.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.140192.log.gz 2023-09-30T22:41:08.961 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.158166.log 2023-09-30T22:41:08.963 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.158166.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.158166.log.gz 2023-09-30T22:41:08.963 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188863.log 2023-09-30T22:41:08.963 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.188863.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.188863.log.gz 2023-09-30T22:41:08.963 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145559.log 2023-09-30T22:41:08.964 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.145559.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.145559.log.gz 2023-09-30T22:41:08.964 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135452.log 2023-09-30T22:41:08.964 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.135452.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.135452.log.gz 2023-09-30T22:41:08.964 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189364.log 2023-09-30T22:41:08.964 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.189364.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.189364.log.gz 2023-09-30T22:41:08.964 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134155.log 2023-09-30T22:41:08.965 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.134155.log: 93.8% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-09-30T22:41:08.965 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.134155.log.gz 2023-09-30T22:41:08.966 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191963.log 2023-09-30T22:41:08.966 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180231.log 2023-09-30T22:41:08.966 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.191963.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.191963.log.gz 2023-09-30T22:41:08.966 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.180231.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.180231.log.gz 2023-09-30T22:41:08.966 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193506.log 2023-09-30T22:41:08.967 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162457.log 2023-09-30T22:41:08.967 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.193506.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.193506.log.gz 2023-09-30T22:41:08.967 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.162457.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.162457.log.gz 2023-09-30T22:41:08.967 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137218.log 2023-09-30T22:41:08.968 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165157.log 2023-09-30T22:41:08.968 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.137218.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.137218.log.gz 2023-09-30T22:41:08.969 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.165157.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.165157.log.gz 2023-09-30T22:41:08.969 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177095.log 2023-09-30T22:41:08.969 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161868.log 2023-09-30T22:41:08.969 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.177095.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.177095.log.gz 2023-09-30T22:41:08.970 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.161868.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.161868.log.gz 2023-09-30T22:41:08.970 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153820.log 2023-09-30T22:41:08.970 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151886.log 2023-09-30T22:41:08.971 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.153820.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.153820.log.gz 2023-09-30T22:41:08.971 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.151886.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.151886.log.gz 2023-09-30T22:41:08.972 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161595.log 2023-09-30T22:41:08.972 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157702.log 2023-09-30T22:41:08.973 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.161595.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.161595.log.gz 2023-09-30T22:41:08.973 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.157702.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.157702.log.gz 2023-09-30T22:41:08.973 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154611.log 2023-09-30T22:41:08.974 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156794.log 2023-09-30T22:41:08.974 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.154611.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.154611.log.gz 2023-09-30T22:41:08.975 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.156794.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.156794.log.gz 2023-09-30T22:41:08.975 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192186.log 2023-09-30T22:41:08.975 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188206.log 2023-09-30T22:41:08.976 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.192186.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.192186.log.gz/var/log/ceph/ceph-client.admin.188206.log: 2023-09-30T22:41:08.976 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.188206.log.gz 2023-09-30T22:41:08.977 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122300.log 2023-09-30T22:41:08.977 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154139.log 2023-09-30T22:41:08.977 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.122300.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.122300.log.gz 2023-09-30T22:41:08.978 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.154139.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.154139.log.gz 2023-09-30T22:41:08.978 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153747.log 2023-09-30T22:41:08.978 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182195.log 2023-09-30T22:41:08.979 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.153747.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.153747.log.gz 2023-09-30T22:41:08.979 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.182195.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.182195.log.gz 2023-09-30T22:41:08.979 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179055.log 2023-09-30T22:41:08.981 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135138.log 2023-09-30T22:41:08.981 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.179055.log: /var/log/ceph/ceph-client.admin.135138.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.179055.log.gz 2023-09-30T22:41:08.981 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.135138.log.gz 2023-09-30T22:41:08.981 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145925.log 2023-09-30T22:41:08.982 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189390.log 2023-09-30T22:41:08.982 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.145925.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.145925.log.gz 2023-09-30T22:41:08.982 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.189390.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.189390.log.gz 2023-09-30T22:41:08.983 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186857.log 2023-09-30T22:41:08.983 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150479.log 2023-09-30T22:41:08.984 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.186857.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.186857.log.gz 2023-09-30T22:41:08.984 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.150479.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.150479.log.gz 2023-09-30T22:41:08.984 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159340.log 2023-09-30T22:41:08.984 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179569.log 2023-09-30T22:41:08.985 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.159340.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.159340.log.gz 2023-09-30T22:41:08.985 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.179569.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.179569.log.gz 2023-09-30T22:41:08.985 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173959.log 2023-09-30T22:41:08.986 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145729.log 2023-09-30T22:41:08.986 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.173959.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.173959.log.gz 2023-09-30T22:41:08.987 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.145729.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.145729.log.gz 2023-09-30T22:41:08.987 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124388.log 2023-09-30T22:41:08.987 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118518.log 2023-09-30T22:41:08.988 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.124388.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.124388.log.gz 2023-09-30T22:41:08.988 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.118518.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.118518.log.gz 2023-09-30T22:41:08.989 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139382.log 2023-09-30T22:41:08.989 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139853.log 2023-09-30T22:41:08.990 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.139382.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.139382.log.gz 2023-09-30T22:41:08.990 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.139853.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.139853.log.gz 2023-09-30T22:41:08.990 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164250.log 2023-09-30T22:41:08.990 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169324.log 2023-09-30T22:41:08.991 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.164250.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.164250.log.gz 2023-09-30T22:41:08.991 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.169324.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.169324.log.gz 2023-09-30T22:41:08.992 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125684.log 2023-09-30T22:41:08.992 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170031.log 2023-09-30T22:41:08.993 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.125684.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.125684.log.gz 2023-09-30T22:41:08.993 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.170031.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.170031.log.gz 2023-09-30T22:41:08.993 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180035.log 2023-09-30T22:41:08.993 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151690.log 2023-09-30T22:41:08.994 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.180035.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.180035.log.gz 2023-09-30T22:41:08.994 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.151690.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.151690.log.gz 2023-09-30T22:41:08.994 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151102.log 2023-09-30T22:41:08.995 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135137.log 2023-09-30T22:41:08.996 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.151102.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.151102.log.gz 2023-09-30T22:41:08.996 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.135137.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.135137.log.gz 2023-09-30T22:41:08.996 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145311.log 2023-09-30T22:41:08.996 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162771.log 2023-09-30T22:41:08.997 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.145311.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.145311.log.gz 2023-09-30T22:41:08.997 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.162771.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.162771.log.gz 2023-09-30T22:41:08.997 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191794.log 2023-09-30T22:41:08.998 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163751.log 2023-09-30T22:41:08.999 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.191794.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.191794.log.gz 2023-09-30T22:41:08.999 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.163751.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.163751.log.gz 2023-09-30T22:41:08.999 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168267.log 2023-09-30T22:41:08.999 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173175.log 2023-09-30T22:41:09.000 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.168267.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.168267.log.gz 2023-09-30T22:41:09.000 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.173175.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.173175.log.gz 2023-09-30T22:41:09.001 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141146.log 2023-09-30T22:41:09.001 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146513.log 2023-09-30T22:41:09.001 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.141146.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.141146.log.gz 2023-09-30T22:41:09.002 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.146513.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.146513.log.gz 2023-09-30T22:41:09.002 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192949.log 2023-09-30T22:41:09.003 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173985.log 2023-09-30T22:41:09.003 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.192949.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.192949.log.gz 2023-09-30T22:41:09.003 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.173985.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.173985.log.gz 2023-09-30T22:41:09.004 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161791.log 2023-09-30T22:41:09.004 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152473.log 2023-09-30T22:41:09.005 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.161791.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.161791.log.gz 2023-09-30T22:41:09.005 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.152473.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.152473.log.gz 2023-09-30T22:41:09.005 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178712.log 2023-09-30T22:41:09.006 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125074.log 2023-09-30T22:41:09.006 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.178712.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.178712.log.gz 2023-09-30T22:41:09.007 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.125074.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.125074.log.gz 2023-09-30T22:41:09.007 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137513.log 2023-09-30T22:41:09.007 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150219.log 2023-09-30T22:41:09.008 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.137513.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.137513.log.gz 2023-09-30T22:41:09.008 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.150219.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.150219.log.gz 2023-09-30T22:41:09.008 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184701.log 2023-09-30T22:41:09.009 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138643.log 2023-09-30T22:41:09.010 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.184701.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.184701.log.gz 2023-09-30T22:41:09.010 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.138643.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.138643.log.gz 2023-09-30T22:41:09.010 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179251.log 2023-09-30T22:41:09.010 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144868.log 2023-09-30T22:41:09.012 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.179251.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.179251.log.gz 2023-09-30T22:41:09.012 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.144868.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.144868.log.gz 2023-09-30T22:41:09.012 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180484.log 2023-09-30T22:41:09.012 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178908.log 2023-09-30T22:41:09.018 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.180484.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.180484.log.gz 2023-09-30T22:41:09.018 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.178908.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.178908.log.gz 2023-09-30T22:41:09.018 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170305.log 2023-09-30T22:41:09.018 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.170305.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.170305.log.gz 2023-09-30T22:41:09.019 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193740.log 2023-09-30T22:41:09.019 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161646.log 2023-09-30T22:41:09.019 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.193740.log: /var/log/ceph/ceph-client.admin.161646.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.193740.log.gz 2023-09-30T22:41:09.019 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.161646.log.gz 2023-09-30T22:41:09.019 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159687.log 2023-09-30T22:41:09.019 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166799.log 2023-09-30T22:41:09.019 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.159687.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.159687.log.gz 2023-09-30T22:41:09.019 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.166799.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.166799.log.gz 2023-09-30T22:41:09.020 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178663.log 2023-09-30T22:41:09.020 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.178663.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.178663.log.gz 2023-09-30T22:41:09.020 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179839.log 2023-09-30T22:41:09.021 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155563.log 2023-09-30T22:41:09.021 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.179839.log: /var/log/ceph/ceph-client.admin.155563.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.179839.log.gz 2023-09-30T22:41:09.021 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.155563.log.gz 2023-09-30T22:41:09.021 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151272.log 2023-09-30T22:41:09.021 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147965.log 2023-09-30T22:41:09.021 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.151272.log: /var/log/ceph/ceph-client.admin.147965.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.151272.log.gz 2023-09-30T22:41:09.022 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.147965.log.gz 2023-09-30T22:41:09.022 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154630.log 2023-09-30T22:41:09.023 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190028.log 2023-09-30T22:41:09.023 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.154630.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.154630.log.gz 2023-09-30T22:41:09.023 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144282.log 2023-09-30T22:41:09.023 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.190028.log: /var/log/ceph/ceph-client.admin.144282.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.190028.log.gz 2023-09-30T22:41:09.024 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.144282.log.gz 2023-09-30T22:41:09.024 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135530.log 2023-09-30T22:41:09.025 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.194034.log 2023-09-30T22:41:09.025 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.135530.log: /var/log/ceph/ceph-client.admin.194034.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.135530.log.gz 2023-09-30T22:41:09.025 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.194034.log.gz 2023-09-30T22:41:09.025 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160837.log 2023-09-30T22:41:09.026 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155004.log 2023-09-30T22:41:09.026 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.160837.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.160837.log.gz 2023-09-30T22:41:09.028 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160615.log 2023-09-30T22:41:09.028 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.155004.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.155004.log.gz 2023-09-30T22:41:09.028 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.160615.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.160615.log.gz 2023-09-30T22:41:09.029 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179158.log 2023-09-30T22:41:09.029 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193691.log 2023-09-30T22:41:09.029 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.179158.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.179158.log.gz 2023-09-30T22:41:09.030 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.193691.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.193691.log.gz 2023-09-30T22:41:09.030 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127925.log 2023-09-30T22:41:09.031 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123604.log 2023-09-30T22:41:09.031 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.127925.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.127925.log.gz 2023-09-30T22:41:09.031 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.123604.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.123604.log.gz 2023-09-30T22:41:09.032 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171894.log 2023-09-30T22:41:09.032 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180427.log 2023-09-30T22:41:09.033 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.171894.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.171894.log.gz 2023-09-30T22:41:09.033 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.180427.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.180427.log.gz 2023-09-30T22:41:09.033 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188598.log 2023-09-30T22:41:09.034 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142173.log 2023-09-30T22:41:09.034 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.188598.log: /var/log/ceph/ceph-client.admin.142173.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.188598.log.gz 2023-09-30T22:41:09.034 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.142173.log.gz 2023-09-30T22:41:09.034 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187200.log 2023-09-30T22:41:09.035 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174547.log 2023-09-30T22:41:09.035 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.187200.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.187200.log.gz 2023-09-30T22:41:09.036 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.174547.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.174547.log.gz 2023-09-30T22:41:09.036 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177487.log 2023-09-30T22:41:09.037 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167287.log 2023-09-30T22:41:09.037 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.177487.log: /var/log/ceph/ceph-client.admin.167287.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.177487.log.gz 2023-09-30T22:41:09.037 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.167287.log.gz 2023-09-30T22:41:09.037 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152277.log 2023-09-30T22:41:09.038 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165820.log 2023-09-30T22:41:09.038 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.152277.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.152277.log.gz 2023-09-30T22:41:09.039 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.165820.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.165820.log.gz 2023-09-30T22:41:09.039 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144135.log 2023-09-30T22:41:09.040 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144919.log 2023-09-30T22:41:09.040 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.144135.log: /var/log/ceph/ceph-client.admin.144919.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.144135.log.gz 2023-09-30T22:41:09.040 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.144919.log.gz 2023-09-30T22:41:09.040 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184012.log 2023-09-30T22:41:09.041 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.184012.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.184012.log.gz 2023-09-30T22:41:09.041 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181660.log 2023-09-30T22:41:09.042 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138420.log 2023-09-30T22:41:09.042 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.181660.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.181660.log.gz 2023-09-30T22:41:09.044 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159161.log 2023-09-30T22:41:09.044 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.138420.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.138420.log.gz 2023-09-30T22:41:09.044 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.159161.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.159161.log.gz 2023-09-30T22:41:09.044 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125782.log 2023-09-30T22:41:09.045 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185876.log 2023-09-30T22:41:09.045 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.125782.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.125782.log.gz 2023-09-30T22:41:09.046 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.185876.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.185876.log.gz 2023-09-30T22:41:09.046 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135922.log 2023-09-30T22:41:09.047 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156303.log 2023-09-30T22:41:09.047 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.135922.log: /var/log/ceph/ceph-client.admin.156303.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.135922.log.gz 2023-09-30T22:41:09.047 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.156303.log.gz 2023-09-30T22:41:09.047 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149916.log 2023-09-30T22:41:09.048 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124074.log 2023-09-30T22:41:09.048 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.149916.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.149916.log.gz 2023-09-30T22:41:09.049 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.124074.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.124074.log.gz 2023-09-30T22:41:09.049 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150710.log 2023-09-30T22:41:09.050 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182297.log 2023-09-30T22:41:09.050 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.150710.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.150710.log.gz 2023-09-30T22:41:09.050 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.182297.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.182297.log.gz 2023-09-30T22:41:09.050 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129787.log 2023-09-30T22:41:09.051 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161059.log 2023-09-30T22:41:09.051 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.129787.log: /var/log/ceph/ceph-client.admin.161059.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.129787.log.gz 2023-09-30T22:41:09.052 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.161059.log.gz 2023-09-30T22:41:09.052 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126893.log 2023-09-30T22:41:09.053 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181268.log 2023-09-30T22:41:09.053 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.126893.log: /var/log/ceph/ceph-client.admin.181268.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.126893.log.gz 2023-09-30T22:41:09.053 INFO:teuthology.orchestra.run.smithi016.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.181268.log.gz 2023-09-30T22:41:09.053 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171847.log 2023-09-30T22:41:09.054 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191619.log 2023-09-30T22:41:09.054 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.171847.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.171847.log.gz 2023-09-30T22:41:09.055 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.191619.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.191619.log.gz 2023-09-30T22:41:09.055 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181999.log 2023-09-30T22:41:09.056 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185384.log 2023-09-30T22:41:09.056 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.181999.log: /var/log/ceph/ceph-client.admin.185384.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.181999.log.gz 2023-09-30T22:41:09.056 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.185384.log.gz 2023-09-30T22:41:09.056 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141977.log 2023-09-30T22:41:09.057 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184478.log 2023-09-30T22:41:09.057 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.141977.log: /var/log/ceph/ceph-client.admin.184478.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.141977.log.gz 2023-09-30T22:41:09.058 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.184478.log.gz 2023-09-30T22:41:09.058 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155145.log 2023-09-30T22:41:09.059 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159465.log 2023-09-30T22:41:09.059 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.155145.log: /var/log/ceph/ceph-client.admin.159465.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.155145.log.gz 2023-09-30T22:41:09.059 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.159465.log.gz 2023-09-30T22:41:09.059 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154016.log 2023-09-30T22:41:09.060 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178764.log 2023-09-30T22:41:09.060 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.154016.log: /var/log/ceph/ceph-client.admin.178764.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.154016.log.gz 2023-09-30T22:41:09.061 INFO:teuthology.orchestra.run.smithi016.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.178764.log.gz 2023-09-30T22:41:09.061 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137123.log 2023-09-30T22:41:09.061 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154727.log 2023-09-30T22:41:09.062 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.137123.log: /var/log/ceph/ceph-client.admin.154727.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.137123.log.gz 2023-09-30T22:41:09.062 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.154727.log.gz 2023-09-30T22:41:09.062 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149338.log 2023-09-30T22:41:09.063 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118614.log 2023-09-30T22:41:09.063 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.149338.log: /var/log/ceph/ceph-client.admin.118614.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.149338.log.gz 2023-09-30T22:41:09.064 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.118614.log.gz 2023-09-30T22:41:09.064 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143990.log 2023-09-30T22:41:09.064 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164791.log 2023-09-30T22:41:09.065 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.143990.log: /var/log/ceph/ceph-client.admin.164791.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.143990.log.gz 2023-09-30T22:41:09.065 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.164791.log.gz 2023-09-30T22:41:09.065 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168756.log 2023-09-30T22:41:09.066 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190788.log 2023-09-30T22:41:09.066 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.168756.log: /var/log/ceph/ceph-client.admin.190788.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.168756.log.gz 2023-09-30T22:41:09.066 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.190788.log.gz 2023-09-30T22:41:09.067 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148894.log 2023-09-30T22:41:09.067 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165772.log 2023-09-30T22:41:09.068 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.148894.log: /var/log/ceph/ceph-client.admin.165772.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.148894.log.gz 2023-09-30T22:41:09.068 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.165772.log.gz 2023-09-30T22:41:09.068 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151029.log 2023-09-30T22:41:09.069 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191376.log 2023-09-30T22:41:09.069 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.151029.log: /var/log/ceph/ceph-client.admin.191376.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.151029.log.gz 2023-09-30T22:41:09.070 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.191376.log.gz 2023-09-30T22:41:09.070 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128821.log 2023-09-30T22:41:09.071 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143743.log 2023-09-30T22:41:09.071 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.128821.log: /var/log/ceph/ceph-client.admin.143743.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.128821.log.gz 2023-09-30T22:41:09.071 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.143743.log.gz 2023-09-30T22:41:09.071 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186021.log 2023-09-30T22:41:09.072 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193789.log 2023-09-30T22:41:09.072 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.186021.log: 89.5%/var/log/ceph/ceph-client.admin.193789.log: -- replaced with /var/log/ceph/ceph-client.admin.186021.log.gz 2023-09-30T22:41:09.072 INFO:teuthology.orchestra.run.smithi016.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.193789.log.gz 2023-09-30T22:41:09.073 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134374.log 2023-09-30T22:41:09.073 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148765.log 2023-09-30T22:41:09.073 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.134374.log: /var/log/ceph/ceph-client.admin.148765.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.134374.log.gz 2023-09-30T22:41:09.074 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.148765.log.gz 2023-09-30T22:41:09.074 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156945.log 2023-09-30T22:41:09.075 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165523.log 2023-09-30T22:41:09.075 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.156945.log: /var/log/ceph/ceph-client.admin.165523.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.156945.log.gz 2023-09-30T22:41:09.075 INFO:teuthology.orchestra.run.smithi016.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.165523.log.gz 2023-09-30T22:41:09.076 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164623.log 2023-09-30T22:41:09.076 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175039.log 2023-09-30T22:41:09.076 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.164623.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.164623.log.gz/var/log/ceph/ceph-client.admin.175039.log: 2023-09-30T22:41:09.077 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.175039.log.gz 2023-09-30T22:41:09.077 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186168.log 2023-09-30T22:41:09.078 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128674.log 2023-09-30T22:41:09.078 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.186168.log: /var/log/ceph/ceph-client.admin.128674.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.186168.log.gz 2023-09-30T22:41:09.078 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.128674.log.gz 2023-09-30T22:41:09.079 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123876.log 2023-09-30T22:41:09.079 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.186556.log 2023-09-30T22:41:09.079 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.123876.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.123876.log.gz 2023-09-30T22:41:09.080 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.186556.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.186556.log.gz 2023-09-30T22:41:09.080 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191225.log 2023-09-30T22:41:09.081 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154826.log 2023-09-30T22:41:09.081 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.191225.log: /var/log/ceph/ceph-client.admin.154826.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.191225.log.gz 2023-09-30T22:41:09.081 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.154826.log.gz 2023-09-30T22:41:09.082 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159269.log 2023-09-30T22:41:09.082 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184796.log 2023-09-30T22:41:09.082 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.159269.log: /var/log/ceph/ceph-client.admin.184796.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.159269.log.gz 2023-09-30T22:41:09.083 INFO:teuthology.orchestra.run.smithi016.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.184796.log.gz 2023-09-30T22:41:09.083 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169911.log 2023-09-30T22:41:09.084 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.163261.log 2023-09-30T22:41:09.084 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.169911.log: /var/log/ceph/ceph-client.admin.163261.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.169911.log.gz 2023-09-30T22:41:09.084 INFO:teuthology.orchestra.run.smithi016.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.163261.log.gz 2023-09-30T22:41:09.084 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191912.log 2023-09-30T22:41:09.086 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171207.log 2023-09-30T22:41:09.086 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.191912.log: /var/log/ceph/ceph-client.admin.171207.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.191912.log.gz 2023-09-30T22:41:09.086 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.171207.log.gz 2023-09-30T22:41:09.086 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149144.log 2023-09-30T22:41:09.087 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160888.log 2023-09-30T22:41:09.087 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.149144.log: /var/log/ceph/ceph-client.admin.160888.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.149144.log.gz 2023-09-30T22:41:09.087 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.160888.log.gz 2023-09-30T22:41:09.087 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160053.log 2023-09-30T22:41:09.088 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162234.log 2023-09-30T22:41:09.088 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.160053.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.160053.log.gz 2023-09-30T22:41:09.089 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.162234.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.162234.log.gz 2023-09-30T22:41:09.089 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187422.log 2023-09-30T22:41:09.090 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147075.log 2023-09-30T22:41:09.090 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.187422.log: /var/log/ceph/ceph-client.admin.147075.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.187422.log.gz 2023-09-30T22:41:09.090 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.147075.log.gz 2023-09-30T22:41:09.091 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.185629.log 2023-09-30T22:41:09.092 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125390.log 2023-09-30T22:41:09.092 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.185629.log: /var/log/ceph/ceph-client.admin.125390.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.185629.log.gz 2023-09-30T22:41:09.092 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.125390.log.gz 2023-09-30T22:41:09.092 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173687.log 2023-09-30T22:41:09.093 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143939.log 2023-09-30T22:41:09.093 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.173687.log: /var/log/ceph/ceph-client.admin.143939.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.173687.log.gz 2023-09-30T22:41:09.093 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.143939.log.gz 2023-09-30T22:41:09.094 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192722.log 2023-09-30T22:41:09.094 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124878.log 2023-09-30T22:41:09.094 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.192722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187004.log 2023-09-30T22:41:09.095 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.192722.log.gz 2023-09-30T22:41:09.095 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.124878.log: /var/log/ceph/ceph-client.admin.187004.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.124878.log.gz 2023-09-30T22:41:09.095 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.187004.log.gz 2023-09-30T22:41:09.095 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187055.log 2023-09-30T22:41:09.096 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181513.log 2023-09-30T22:41:09.096 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.187055.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.187055.log.gz 2023-09-30T22:41:09.097 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.181513.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.181513.log.gz 2023-09-30T22:41:09.097 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174155.log 2023-09-30T22:41:09.098 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.174155.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.174155.log.gz 2023-09-30T22:41:09.098 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190762.log 2023-09-30T22:41:09.098 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136928.log 2023-09-30T22:41:09.099 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.190762.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.190762.log.gz 2023-09-30T22:41:09.099 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.136928.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.136928.log.gz 2023-09-30T22:41:09.099 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179300.log 2023-09-30T22:41:09.100 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168561.log 2023-09-30T22:41:09.100 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.179300.log: /var/log/ceph/ceph-client.admin.168561.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.179300.log.gz 2023-09-30T22:41:09.101 INFO:teuthology.orchestra.run.smithi016.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.168561.log.gz 2023-09-30T22:41:09.101 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129677.log 2023-09-30T22:41:09.102 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155816.log 2023-09-30T22:41:09.102 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.129677.log: /var/log/ceph/ceph-client.admin.155816.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.129677.log.gz 2023-09-30T22:41:09.102 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.155816.log.gz 2023-09-30T22:41:09.102 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174351.log 2023-09-30T22:41:09.103 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153794.log 2023-09-30T22:41:09.103 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.174351.log: /var/log/ceph/ceph-client.admin.153794.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.174351.log.gz 2023-09-30T22:41:09.104 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.153794.log.gz 2023-09-30T22:41:09.104 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160419.log 2023-09-30T22:41:09.105 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155715.log 2023-09-30T22:41:09.105 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.160419.log: /var/log/ceph/ceph-client.admin.155715.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.160419.log.gz 2023-09-30T22:41:09.105 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.155715.log.gz 2023-09-30T22:41:09.105 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183547.log 2023-09-30T22:41:09.106 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136263.log 2023-09-30T22:41:09.106 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.183547.log: /var/log/ceph/ceph-client.admin.136263.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.183547.log.gz 2023-09-30T22:41:09.107 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.136263.log.gz 2023-09-30T22:41:09.107 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137610.log 2023-09-30T22:41:09.107 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.177340.log 2023-09-30T22:41:09.108 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.137610.log: /var/log/ceph/ceph-client.admin.177340.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.137610.log.gz 2023-09-30T22:41:09.108 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.177340.log.gz 2023-09-30T22:41:09.108 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179914.log 2023-09-30T22:41:09.109 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.189659.log 2023-09-30T22:41:09.109 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.179914.log: /var/log/ceph/ceph-client.admin.189659.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.179914.log.gz 2023-09-30T22:41:09.110 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.189659.log.gz 2023-09-30T22:41:09.110 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157087.log 2023-09-30T22:41:09.110 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137079.log 2023-09-30T22:41:09.111 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.157087.log: /var/log/ceph/ceph-client.admin.137079.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.157087.log.gz 2023-09-30T22:41:09.111 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.137079.log.gz 2023-09-30T22:41:09.111 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.162967.log 2023-09-30T22:41:09.111 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147271.log 2023-09-30T22:41:09.112 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.162967.log: /var/log/ceph/ceph-client.admin.147271.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.162967.log.gz 2023-09-30T22:41:09.112 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.147271.log.gz 2023-09-30T22:41:09.113 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154259.log 2023-09-30T22:41:09.113 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153355.log 2023-09-30T22:41:09.114 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.154259.log: /var/log/ceph/ceph-client.admin.153355.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.154259.log.gz 2023-09-30T22:41:09.114 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.153355.log.gz 2023-09-30T22:41:09.114 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139800.log 2023-09-30T22:41:09.115 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144161.log 2023-09-30T22:41:09.115 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.139800.log: /var/log/ceph/ceph-client.admin.144161.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.139800.log.gz 2023-09-30T22:41:09.115 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.144161.log.gz 2023-09-30T22:41:09.116 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.169129.log 2023-09-30T22:41:09.116 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.149239.log 2023-09-30T22:41:09.116 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.169129.log: /var/log/ceph/ceph-client.admin.149239.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.169129.log.gz 2023-09-30T22:41:09.117 INFO:teuthology.orchestra.run.smithi016.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.149239.log.gz 2023-09-30T22:41:09.117 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184845.log 2023-09-30T22:41:09.118 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.161281.log 2023-09-30T22:41:09.118 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.184845.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.184845.log.gz 2023-09-30T22:41:09.118 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.161281.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.161281.log.gz 2023-09-30T22:41:09.119 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134549.log 2023-09-30T22:41:09.119 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.184505.log 2023-09-30T22:41:09.120 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.134549.log: /var/log/ceph/ceph-client.admin.184505.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.134549.log.gz 2023-09-30T22:41:09.120 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.184505.log.gz 2023-09-30T22:41:09.120 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171285.log 2023-09-30T22:41:09.120 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123506.log 2023-09-30T22:41:09.121 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.171285.log: /var/log/ceph/ceph-client.admin.123506.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.171285.log.gz 2023-09-30T22:41:09.121 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.123506.log.gz 2023-09-30T22:41:09.122 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153237.log 2023-09-30T22:41:09.122 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.160693.log 2023-09-30T22:41:09.123 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.153237.log: /var/log/ceph/ceph-client.admin.160693.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.153237.log.gz 2023-09-30T22:41:09.123 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.160693.log.gz 2023-09-30T22:41:09.123 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141342.log 2023-09-30T22:41:09.124 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141833.log 2023-09-30T22:41:09.124 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.141342.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.141342.log.gz 2023-09-30T22:41:09.125 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.141833.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.141833.log.gz 2023-09-30T22:41:09.125 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151448.log 2023-09-30T22:41:09.125 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-09-30T22:41:09.126 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.151448.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.151448.log.gz 2023-09-30T22:41:09.129 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170253.log 2023-09-30T22:41:09.130 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.170253.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.170253.log.gz 2023-09-30T22:41:09.130 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137244.log 2023-09-30T22:41:09.131 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.137244.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.137244.log.gz 2023-09-30T22:41:09.131 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123261.log 2023-09-30T22:41:09.132 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.123261.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.123261.log.gz 2023-09-30T22:41:09.132 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155171.log 2023-09-30T22:41:09.134 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.155171.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.155171.log.gz 2023-09-30T22:41:09.134 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128400.log 2023-09-30T22:41:09.135 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.128400.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.128400.log.gz 2023-09-30T22:41:09.135 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159295.log 2023-09-30T22:41:09.136 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173397.log 2023-09-30T22:41:09.136 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.159295.log: /var/log/ceph/ceph-client.admin.173397.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.159295.log.gz 2023-09-30T22:41:09.136 INFO:teuthology.orchestra.run.smithi016.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.173397.log.gz 2023-09-30T22:41:09.136 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.165230.log 2023-09-30T22:41:09.137 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.165230.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.165230.log.gz 2023-09-30T22:41:09.137 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135503.log 2023-09-30T22:41:09.138 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.135503.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.135503.log.gz 2023-09-30T22:41:09.138 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183228.log 2023-09-30T22:41:09.139 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.183228.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.183228.log.gz 2023-09-30T22:41:09.139 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.154408.log 2023-09-30T22:41:09.140 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.154408.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.154408.log.gz 2023-09-30T22:41:09.141 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.193480.log 2023-09-30T22:41:09.142 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.193480.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.193480.log.gz 2023-09-30T22:41:09.142 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.179692.log 2023-09-30T22:41:09.143 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.179692.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.179692.log.gz 2023-09-30T22:41:09.143 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170914.log 2023-09-30T22:41:09.144 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.170914.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.170914.log.gz 2023-09-30T22:41:09.144 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.164446.log 2023-09-30T22:41:09.145 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.164446.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.164446.log.gz 2023-09-30T22:41:09.145 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183473.log 2023-09-30T22:41:09.146 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.183473.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.183473.log.gz 2023-09-30T22:41:09.146 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190148.log 2023-09-30T22:41:09.147 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.190148.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.190148.log.gz 2023-09-30T22:41:09.147 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.194083.log 2023-09-30T22:41:09.148 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.194083.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.194083.log.gz 2023-09-30T22:41:09.149 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152375.log 2023-09-30T22:41:09.150 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.152375.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.152375.log.gz 2023-09-30T22:41:09.150 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128143.log 2023-09-30T22:41:09.150 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.128143.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.128143.log.gz 2023-09-30T22:41:09.151 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173201.log 2023-09-30T22:41:09.152 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.173201.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.173201.log.gz 2023-09-30T22:41:09.152 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138446.log 2023-09-30T22:41:09.153 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.138446.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.138446.log.gz 2023-09-30T22:41:09.153 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124976.log 2023-09-30T22:41:09.154 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.124976.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.124976.log.gz 2023-09-30T22:41:09.154 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127071.log 2023-09-30T22:41:09.155 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.127071.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.127071.log.gz 2023-09-30T22:41:09.155 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141805.log 2023-09-30T22:41:09.156 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.141805.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.141805.log.gz 2023-09-30T22:41:09.156 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.147501.log 2023-09-30T22:41:09.157 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.147501.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.147501.log.gz 2023-09-30T22:41:09.157 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.173593.log 2023-09-30T22:41:09.158 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.173593.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.173593.log.gz 2023-09-30T22:41:09.158 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.191572.log 2023-09-30T22:41:09.159 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.191572.log: 93.8% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-09-30T22:41:09.159 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168907.log 2023-09-30T22:41:09.159 INFO:teuthology.orchestra.run.smithi016.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.191572.log.gz 2023-09-30T22:41:09.160 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.159734.log 2023-09-30T22:41:09.160 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.168907.log: /var/log/ceph/ceph-client.admin.159734.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.168907.log.gz 2023-09-30T22:41:09.161 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.192134.log 2023-09-30T22:41:09.161 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.159734.log.gz 2023-09-30T22:41:09.162 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156159.log 2023-09-30T22:41:09.162 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.192134.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.192134.log.gz 2023-09-30T22:41:09.162 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.156159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.176804.log 2023-09-30T22:41:09.163 INFO:teuthology.orchestra.run.smithi016.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.156159.log.gz 2023-09-30T22:41:09.163 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.176804.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.176804.log.gz 2023-09-30T22:41:09.163 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145062.log 2023-09-30T22:41:09.164 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187448.log 2023-09-30T22:41:09.164 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.145062.log: /var/log/ceph/ceph-client.admin.187448.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.145062.log.gz 2023-09-30T22:41:09.164 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150314.log 2023-09-30T22:41:09.165 INFO:teuthology.orchestra.run.smithi016.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.187448.log.gz 2023-09-30T22:41:09.165 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122492.log 2023-09-30T22:41:09.167 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.150314.log: /var/log/ceph/ceph-client.admin.122492.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.150314.log.gz 2023-09-30T22:41:09.167 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.180110.log 2023-09-30T22:41:09.167 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.122492.log.gz 2023-09-30T22:41:09.167 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141930.log 2023-09-30T22:41:09.167 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.180110.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.180110.log.gz 2023-09-30T22:41:09.167 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.141930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187984.log 2023-09-30T22:41:09.168 INFO:teuthology.orchestra.run.smithi016.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.141930.log.gz 2023-09-30T22:41:09.168 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181343.log 2023-09-30T22:41:09.169 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.187984.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.187984.log.gz 2023-09-30T22:41:09.169 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.181343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150318.log 2023-09-30T22:41:09.169 INFO:teuthology.orchestra.run.smithi016.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.181343.log.gz 2023-09-30T22:41:09.169 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139186.log 2023-09-30T22:41:09.169 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.150318.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.150318.log.gz 2023-09-30T22:41:09.170 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.139186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187644.log 2023-09-30T22:41:09.170 INFO:teuthology.orchestra.run.smithi016.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.139186.log.gz 2023-09-30T22:41:09.170 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.187056.log 2023-09-30T22:41:09.171 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.187644.log: /var/log/ceph/ceph-client.admin.187056.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.187644.log.gz 2023-09-30T22:41:09.171 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188035.log 2023-09-30T22:41:09.171 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.187056.log.gz 2023-09-30T22:41:09.172 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.157675.log 2023-09-30T22:41:09.172 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.188035.log: 88.1%/var/log/ceph/ceph-client.admin.157675.log: -- replaced with /var/log/ceph/ceph-client.admin.188035.log.gz 2023-09-30T22:41:09.172 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.171481.log 2023-09-30T22:41:09.172 INFO:teuthology.orchestra.run.smithi016.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.157675.log.gz 2023-09-30T22:41:09.172 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181147.log 2023-09-30T22:41:09.174 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.171481.log: /var/log/ceph/ceph-client.admin.181147.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.171481.log.gz 2023-09-30T22:41:09.174 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135307.log 2023-09-30T22:41:09.174 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.181147.log.gz 2023-09-30T22:41:09.174 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127630.log 2023-09-30T22:41:09.174 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.135307.log: /var/log/ceph/ceph-client.admin.127630.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.135307.log.gz 2023-09-30T22:41:09.174 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188180.log 2023-09-30T22:41:09.175 INFO:teuthology.orchestra.run.smithi016.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.127630.log.gz 2023-09-30T22:41:09.175 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.168123.log 2023-09-30T22:41:09.176 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.188180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.188062.log 2023-09-30T22:41:09.176 INFO:teuthology.orchestra.run.smithi016.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.188180.log.gz 2023-09-30T22:41:09.176 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.168123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.156355.log 2023-09-30T22:41:09.176 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.168123.log.gz 2023-09-30T22:41:09.176 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.188062.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.188062.log.gz 2023-09-30T22:41:09.176 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.156355.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.156355.log.gz 2023-09-30T22:41:09.177 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142957.log 2023-09-30T22:41:09.177 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136729.log 2023-09-30T22:41:09.177 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.142957.log: /var/log/ceph/ceph-client.admin.136729.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.142957.log.gz 2023-09-30T22:41:09.178 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.167190.log 2023-09-30T22:41:09.178 INFO:teuthology.orchestra.run.smithi016.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.136729.log.gz 2023-09-30T22:41:09.178 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.181905.log 2023-09-30T22:41:09.179 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.167190.log: /var/log/ceph/ceph-client.admin.181905.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.167190.log.gz 2023-09-30T22:41:09.179 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.181905.log.gz 2023-09-30T22:41:09.179 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.174424.log 2023-09-30T22:41:09.179 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.178345.log 2023-09-30T22:41:09.180 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.174424.log: /var/log/ceph/ceph-client.admin.178345.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.174424.log.gz 2023-09-30T22:41:09.180 INFO:teuthology.orchestra.run.smithi016.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.178345.log.gz 2023-09-30T22:41:09.180 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.155022.log 2023-09-30T22:41:09.181 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.166958.log 2023-09-30T22:41:09.181 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.155022.log: /var/log/ceph/ceph-client.admin.166958.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.155022.log.gz 2023-09-30T22:41:09.181 INFO:teuthology.orchestra.run.smithi016.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.166958.log.gz 2023-09-30T22:41:09.181 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183081.log 2023-09-30T22:41:09.181 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.182444.log 2023-09-30T22:41:09.182 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.183081.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.183081.log.gz 2023-09-30T22:41:09.182 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.182444.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.182444.log.gz 2023-09-30T22:41:09.182 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.151421.log 2023-09-30T22:41:09.183 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.152065.log 2023-09-30T22:41:09.183 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.151421.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.151421.log.gz 2023-09-30T22:41:09.184 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.152065.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.152065.log.gz 2023-09-30T22:41:09.184 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124633.log 2023-09-30T22:41:09.185 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146628.log 2023-09-30T22:41:09.185 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.124633.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.124633.log.gz 2023-09-30T22:41:09.185 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.146628.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.146628.log.gz 2023-09-30T22:41:09.186 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.170475.log 2023-09-30T22:41:09.186 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.190030.log 2023-09-30T22:41:09.186 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.170475.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.170475.log.gz 2023-09-30T22:41:09.187 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.190030.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.190030.log.gz 2023-09-30T22:41:09.187 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.175527.log 2023-09-30T22:41:09.187 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124073.log 2023-09-30T22:41:09.187 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.175527.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.175527.log.gz 2023-09-30T22:41:09.188 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.124073.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.124073.log.gz 2023-09-30T22:41:09.188 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.148063.log 2023-09-30T22:41:09.188 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129025.log 2023-09-30T22:41:09.188 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.148063.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.148063.log.gz 2023-09-30T22:41:09.189 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.129025.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.129025.log.gz 2023-09-30T22:41:09.189 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.183695.log 2023-09-30T22:41:09.190 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.153000.log 2023-09-30T22:41:09.190 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.183695.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.183695.log.gz 2023-09-30T22:41:09.191 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.153000.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.153000.log.gz 2023-09-30T22:41:09.191 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.150122.log 2023-09-30T22:41:09.191 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ceph-client.admin.150122.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.150122.log.gz 2023-09-30T22:41:09.250 INFO:teuthology.orchestra.run.smithi016.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-09-30T22:41:09.253 INFO:teuthology.orchestra.run.smithi016.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-09-30T22:41:09.295 INFO:teuthology.orchestra.run.smithi016.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-09-30T22:41:09.383 INFO:teuthology.orchestra.run.smithi016.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-09-30T22:41:09.607 INFO:teuthology.orchestra.run.smithi016.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-09-30T22:41:09.609 INFO:teuthology.orchestra.run.smithi016.stderr: 2023-09-30T22:41:09.609 INFO:teuthology.orchestra.run.smithi016.stderr:real 0m1.204s 2023-09-30T22:41:09.609 INFO:teuthology.orchestra.run.smithi016.stderr:user 0m4.285s 2023-09-30T22:41:09.610 INFO:teuthology.orchestra.run.smithi016.stderr:sys 0m0.969s 2023-09-30T22:41:09.610 INFO:tasks.ceph:Archiving logs... 2023-09-30T22:41:09.610 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/log/ceph to /home/teuthworker/archive/dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7408044/remote/smithi016/log 2023-09-30T22:41:09.611 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /var/log/ceph -- . 2023-09-30T22:41:11.268 DEBUG:teuthology.misc:Transferring archived files from smithi027:/var/log/ceph to /home/teuthworker/archive/dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7408044/remote/smithi027/log 2023-09-30T22:41:11.269 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar c -f - -C /var/log/ceph -- . 2023-09-30T22:41:11.304 ERROR:teuthology.run_tasks:Manager failed: ceph Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2742, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2574, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2340, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 1595, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi016 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2023-09-30T22:41:11.315 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-09-30T22:41:11.332 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2742, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2574, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2340, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 1595, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi016 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2023-09-30T22:41:11.337 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-09-30T22:41:11.337 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-09-30T22:41:11.343 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-09-30T22:41:11.468 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-09-30T22:41:11.480 INFO:teuthology.task.clock:Checking final clock skew... 2023-09-30T22:41:11.480 DEBUG:teuthology.orchestra.run.smithi016:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-30T22:41:11.483 DEBUG:teuthology.orchestra.run.smithi027:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-09-30T22:41:11.502 INFO:teuthology.orchestra.run.smithi027.stderr:bash: ntpq: command not found 2023-09-30T22:41:11.503 INFO:teuthology.orchestra.run.smithi016.stderr:bash: ntpq: command not found 2023-09-30T22:41:11.507 INFO:teuthology.orchestra.run.smithi027.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-30T22:41:11.507 INFO:teuthology.orchestra.run.smithi027.stdout:=============================================================================== 2023-09-30T22:41:11.507 INFO:teuthology.orchestra.run.smithi027.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 12 -82us[ -82us] +/- 81ms 2023-09-30T22:41:11.507 INFO:teuthology.orchestra.run.smithi027.stdout:^* hv02.front.sepia.ceph.com 3 6 377 9 -503us[ -502us] +/- 61ms 2023-09-30T22:41:11.507 INFO:teuthology.orchestra.run.smithi027.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 10 +639us[ +639us] +/- 64ms 2023-09-30T22:41:11.508 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-09-30T22:41:11.509 INFO:teuthology.orchestra.run.smithi016.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-09-30T22:41:11.509 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================== 2023-09-30T22:41:11.509 INFO:teuthology.orchestra.run.smithi016.stdout:^+ hv01.front.sepia.ceph.com 2 7 377 76 -59us[ -56us] +/- 80ms 2023-09-30T22:41:11.509 INFO:teuthology.orchestra.run.smithi016.stdout:^* hv02.front.sepia.ceph.com 3 7 377 74 -483us[ -481us] +/- 60ms 2023-09-30T22:41:11.510 INFO:teuthology.orchestra.run.smithi016.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 13 +658us[ +658us] +/- 64ms 2023-09-30T22:41:11.510 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-09-30T22:41:11.510 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-09-30T22:41:11.523 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-09-30T22:41:11.523 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-09-30T22:41:11.535 DEBUG:teuthology.orchestra.run.smithi016:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-09-30T22:41:11.606 DEBUG:teuthology.orchestra.run.smithi027:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-09-30T22:41:11.700 DEBUG:teuthology.orchestra.run.smithi016:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-30T22:41:11.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T22:41:11.738 DEBUG:teuthology.orchestra.run.smithi027:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-09-30T22:41:11.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T22:41:11.771 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-09-30T22:41:11.783 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-09-30T22:41:11.799 INFO:teuthology.task.internal:Duration was 1543.633278 seconds 2023-09-30T22:41:11.800 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-09-30T22:41:11.815 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-09-30T22:41:11.816 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-09-30T22:41:11.818 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-09-30T22:41:11.867 INFO:teuthology.orchestra.run.smithi027.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-30T22:41:11.874 INFO:teuthology.orchestra.run.smithi016.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-09-30T22:41:12.347 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-09-30T22:41:12.348 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi016.front.sepia.ceph.com 2023-09-30T22:41:12.348 DEBUG:teuthology.orchestra.run.smithi016:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-09-30T22:41:12.377 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi027.front.sepia.ceph.com 2023-09-30T22:41:12.377 DEBUG:teuthology.orchestra.run.smithi027:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-09-30T22:41:12.401 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-09-30T22:41:12.402 DEBUG:teuthology.orchestra.run.smithi016:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-09-30T22:41:12.420 DEBUG:teuthology.orchestra.run.smithi027:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-09-30T22:41:12.481 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-09-30T22:41:12.481 DEBUG:teuthology.orchestra.run.smithi016:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-09-30T22:41:12.483 DEBUG:teuthology.orchestra.run.smithi027:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-09-30T22:41:12.608 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-09-30T22:41:12.620 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-09-30T22:41:12.621 DEBUG:teuthology.orchestra.run.smithi016:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-09-30T22:41:12.652 DEBUG:teuthology.orchestra.run.smithi027:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-09-30T22:41:12.680 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-09-30T22:41:12.693 DEBUG:teuthology.orchestra.run.smithi016:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-09-30T22:41:12.696 DEBUG:teuthology.orchestra.run.smithi027:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-09-30T22:41:12.726 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = core 2023-09-30T22:41:12.748 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern = core 2023-09-30T22:41:12.761 DEBUG:teuthology.orchestra.run.smithi016:> test -e /home/ubuntu/cephtest/archive/coredump 2023-09-30T22:41:12.814 WARNING:teuthology.task.internal:Found coredumps on ubuntu@smithi016.front.sepia.ceph.com, flagging run as failed 2023-09-30T22:41:12.814 DEBUG:teuthology.orchestra.run.smithi027:> test -e /home/ubuntu/cephtest/archive/coredump 2023-09-30T22:41:12.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T22:41:12.832 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-09-30T22:41:12.844 INFO:teuthology.task.internal:Transferring archived files... 2023-09-30T22:41:12.845 DEBUG:teuthology.misc:Transferring archived files from smithi016:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7408044/remote/smithi016 2023-09-30T22:41:12.845 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-09-30T22:42:42.662 INFO:teuthology.task.internal:Transferring binaries for coredumps... 2023-09-30T22:42:42.672 INFO:teuthology.task.internal: core looks like: /home/teuthworker/archive/dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7408044/remote/smithi016/coredump/1696112982.126811.core.gz: gzip compressed data, was "1696112982.126811.core", last modified: Sat Sep 30 22:41:12 2023, max compression, original size modulo 2^32 1365295104 2023-09-30T22:42:42.673 INFO:teuthology.task.internal:core is compressed, try accessing gzip file ... 2023-09-30T22:42:52.212 INFO:teuthology.task.internal: core looks like: /tmp/tmpjd1hqjw2: ELF 64-bit LSB core file, x86-64, version 1 (SYSV), SVR4-style, from 'ceph-mgr -f --cluster ceph -i x', real uid: 0, effective uid: 0, real gid: 0, effective gid: 0, execfn: '/bin/ceph-mgr', platform: 'x86_64' 2023-09-30T22:42:52.353 INFO:teuthology.task.internal: dump_program: ceph-mgr -f --cluster ceph -i x 2023-09-30T22:42:52.353 DEBUG:teuthology.orchestra.run.smithi016:> which 'ceph-mgr -f --cluster ceph -i x' 2023-09-30T22:42:52.377 INFO:teuthology.orchestra.run.smithi016.stderr:/usr/bin/which: invalid option -- 'f' 2023-09-30T22:42:52.377 INFO:teuthology.orchestra.run.smithi016.stderr:/usr/bin/which: unrecognized option '--cluster' 2023-09-30T22:42:52.378 INFO:teuthology.orchestra.run.smithi016.stderr:/usr/bin/which: no x in (/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin) 2023-09-30T22:42:52.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T22:42:52.379 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph-mgr 2023-09-30T22:42:52.379 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph 2023-09-30T22:42:52.381 ERROR:teuthology.run_tasks:Manager failed: internal.archive Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/task/internal/__init__.py", line 392, in archive yield File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2742, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2574, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2340, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 1595, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi016 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/task/internal/__init__.py", line 420, in archive fetch_binaries_for_coredumps(path, rem) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/task/internal/__init__.py", line 345, in fetch_binaries_for_coredumps remote_path = remote.sh(['which', dump_program]).rstrip() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 96, in sh proc = self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi016 with status 1: "which 'ceph-mgr -f --cluster ceph -i x'" 2023-09-30T22:42:52.397 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-09-30T22:42:52.410 INFO:teuthology.task.internal:Not uploading archives. 2023-09-30T22:42:52.410 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-09-30T22:42:52.428 INFO:teuthology.task.internal:Tidying up after the test... 2023-09-30T22:42:52.429 DEBUG:teuthology.orchestra.run.smithi016:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-09-30T22:42:52.444 DEBUG:teuthology.orchestra.run.smithi027:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-09-30T22:42:52.457 INFO:teuthology.orchestra.run.smithi016.stdout: 393224 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Sep 30 22:41 /home/ubuntu/cephtest 2023-09-30T22:42:52.458 INFO:teuthology.orchestra.run.smithi016.stdout: 393225 4 drwxr-xr-x 5 ubuntu ubuntu 4096 Sep 30 22:41 /home/ubuntu/cephtest/archive 2023-09-30T22:42:52.458 INFO:teuthology.orchestra.run.smithi016.stdout: 393226 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 30 22:29 /home/ubuntu/cephtest/archive/coredump 2023-09-30T22:42:52.458 INFO:teuthology.orchestra.run.smithi016.stdout: 393232 364640 -rw------- 1 root root 1365295104 Sep 30 22:29 /home/ubuntu/cephtest/archive/coredump/1696112982.126811.core 2023-09-30T22:42:52.458 INFO:teuthology.orchestra.run.smithi016.stdout: 393253 4 drwxrwxr-x 2 ubuntu ubuntu 4096 Sep 30 22:41 /home/ubuntu/cephtest/archive/audit 2023-09-30T22:42:52.458 INFO:teuthology.orchestra.run.smithi016.stdout: 393255 28 -rw------- 1 ubuntu ubuntu 25515 Sep 30 22:41 /home/ubuntu/cephtest/archive/audit/audit.log.gz 2023-09-30T22:42:52.458 INFO:teuthology.orchestra.run.smithi016.stdout: 393227 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 30 22:41 /home/ubuntu/cephtest/archive/syslog 2023-09-30T22:42:52.459 INFO:teuthology.orchestra.run.smithi016.stdout: 393254 4 -rw-rw-rw- 1 ubuntu ubuntu 773 Sep 30 22:41 /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2023-09-30T22:42:52.459 INFO:teuthology.orchestra.run.smithi016.stdout: 393258 112 -rw-rw-r-- 1 ubuntu ubuntu 113941 Sep 30 22:41 /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-09-30T22:42:52.459 INFO:teuthology.orchestra.run.smithi016.stdout: 393256 132 -rw-rw-rw- 1 ubuntu ubuntu 133428 Sep 30 22:41 /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2023-09-30T22:42:52.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-09-30T22:42:52.460 INFO:teuthology.orchestra.run.smithi016.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2023-09-30T22:42:52.460 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph.py", line 1950, in task ctx.managers[config['cluster']].wait_for_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2742, in wait_for_clean num_active_clean = self.get_num_active_clean() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2574, in get_num_active_clean pgs = self.get_pg_stats() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 2340, in get_pg_stats out = self.raw_cluster_cmd('pg', 'dump', '--format=json') File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 1595, in raw_cluster_cmd return self.run_cluster_cmd(**kwargs).stdout.getvalue() File "/home/teuthworker/src/github.com_dparmar18_ceph_6973b24e3d99fd494b753449d2ea12077599cbd6/qa/tasks/ceph_manager.py", line 1586, in run_cluster_cmd return self.controller.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/remote.py", line 522, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi016 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_54e62bcbac4e53d9685e08328b790d3b20d71cae/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi016 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2023-09-30T22:42:52.472 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-09-30T22:42:52.486 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-09-30T22:42:52.503 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: CommandFailedError: Command failed on smithi016 with status 124: 'sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json' 2023-09-30T22:42:52.503 INFO:teuthology.nuke:Checking targets against current locks 2023-09-30T22:42:52.510 INFO:teuthology.orchestra.run.smithi027.stdout: 525357 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Sep 30 22:41 /home/ubuntu/cephtest 2023-09-30T22:42:52.510 INFO:teuthology.orchestra.run.smithi027.stdout: 525358 4 drwxr-xr-x 4 ubuntu ubuntu 4096 Sep 30 22:41 /home/ubuntu/cephtest/archive 2023-09-30T22:42:52.510 INFO:teuthology.orchestra.run.smithi027.stdout: 655785 4 drwxrwxr-x 2 ubuntu ubuntu 4096 Sep 30 22:41 /home/ubuntu/cephtest/archive/audit 2023-09-30T22:42:52.511 INFO:teuthology.orchestra.run.smithi027.stdout: 655787 220 -rw------- 1 ubuntu ubuntu 223422 Sep 30 22:41 /home/ubuntu/cephtest/archive/audit/audit.log.gz 2023-09-30T22:42:52.511 INFO:teuthology.orchestra.run.smithi027.stdout: 525360 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 30 22:41 /home/ubuntu/cephtest/archive/syslog 2023-09-30T22:42:52.511 INFO:teuthology.orchestra.run.smithi027.stdout: 525482 4 -rw-rw-rw- 1 ubuntu ubuntu 878 Sep 30 22:39 /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2023-09-30T22:42:52.511 INFO:teuthology.orchestra.run.smithi027.stdout: 525362 92 -rw-rw-r-- 1 ubuntu ubuntu 92543 Sep 30 22:41 /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-09-30T22:42:52.511 INFO:teuthology.orchestra.run.smithi027.stdout: 525361 88 -rw-rw-rw- 1 ubuntu ubuntu 86296 Sep 30 22:41 /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2023-09-30T22:42:52.512 INFO:teuthology.orchestra.run.smithi027.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2023-09-30T22:42:52.535 DEBUG:teuthology.nuke:shortname: smithi016 2023-09-30T22:42:52.535 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-30T22:42:52.561 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi016.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7408044', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-30 22:04:55.040715', 'locked_by': 'scheduled_dparmar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKk3kgw+DuBL1HteXloOxaFL1W90U3HWeJLYcbwgixcY6Xo/kjnjoqxZKByRRTDia3763Cwx7yT37uv+Zj3xRvw='} 2023-09-30T22:42:52.564 DEBUG:teuthology.nuke:shortname: smithi027 2023-09-30T22:42:52.564 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-09-30T22:42:52.586 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi027.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/dparmar-2023-09-30_21:53:45-fs:upgrade-fix-nfs-apply-err-reporting-distro-default-smithi/7408044', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-09-30 22:04:55.038787', 'locked_by': 'scheduled_dparmar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOLuHXzaNM+5O4CIcXkqdy7FJ6Fd0pbrWeXk9n3OuMUKk4LGzMdfrWpI9MHt6/S9S8/UPzCaNJKNF5P3SbAS+Sg='} 2023-09-30T22:42:52.611 INFO:teuthology.orchestra.console.smithi016:Power off 2023-09-30T22:42:52.612 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-30T22:42:52.642 DEBUG:teuthology.orchestra.console.smithi016:power off output: Chassis Power Control: Down/Off 2023-09-30T22:42:52.643 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-30T22:42:52.659 INFO:teuthology.orchestra.console.smithi027:Power off 2023-09-30T22:42:52.659 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-09-30T22:42:52.673 DEBUG:teuthology.orchestra.console.smithi016:check power output: Chassis Power is on 2023-09-30T22:42:52.682 DEBUG:teuthology.orchestra.console.smithi027:power off output: Chassis Power Control: Down/Off 2023-09-30T22:42:52.682 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-30T22:42:52.708 DEBUG:teuthology.orchestra.console.smithi027:check power output: Chassis Power is on 2023-09-30T22:42:56.675 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-30T22:42:56.709 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-30T22:42:56.785 DEBUG:teuthology.orchestra.console.smithi016:check power output: Chassis Power is on 2023-09-30T22:42:56.819 DEBUG:teuthology.orchestra.console.smithi027:check power output: Chassis Power is on 2023-09-30T22:43:00.790 DEBUG:teuthology.orchestra.console.smithi016:pexpect command: ipmitool -H smithi016.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-30T22:43:00.820 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-09-30T22:43:00.902 DEBUG:teuthology.orchestra.console.smithi016:check power output: Chassis Power is off 2023-09-30T22:43:00.931 DEBUG:teuthology.orchestra.console.smithi027:check power output: Chassis Power is off 2023-09-30T22:43:01.003 INFO:teuthology.orchestra.console.smithi016:Power off completed 2023-09-30T22:43:01.032 INFO:teuthology.orchestra.console.smithi027:Power off completed 2023-09-30T22:43:01.137 INFO:teuthology.run:Summary data: description: fs:upgrade/upgraded_client/{bluestore-bitmap branch/pacific centos_8 clusters/{1-mds-1-client-micro} conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn} tasks/{0-install 1-mount/mount/fuse 2-upgrade 3-workload/stress_tests/{0-client-upgrade 1-tests/kernel_untar_build}}} duration: 1543.633278131485 failure_reason: 'Command failed on smithi016 with status 124: ''sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json''' flavor: default owner: scheduled_dparmar@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=8548a7ba8a8d44719f5b310a888e0fc8 status: fail success: false 2023-09-30T22:43:01.138 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-09-30T22:43:01.216 INFO:teuthology.run:FAIL